2023-10-26T01:14:26.773 INFO:root:teuthology version: 0.0.1.dev201+g202b180 2023-10-26T01:14:26.775 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_pdonnell@teuthology --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/mds-full} --name pdonnell-2023-10-24_19:52:40-fs-wip-batrick-testing-20231024.144545-distro-default-smithi --archive /home/teuthworker/archive/pdonnell-2023-10-24_19:52:40-fs-wip-batrick-testing-20231024.144545-distro-default-smithi/7436434 --verbose -- /home/teuthworker/archive/pdonnell-2023-10-24_19:52:40-fs-wip-batrick-testing-20231024.144545-distro-default-smithi/7436434/orig.config.yaml 2023-10-26T01:14:26.825 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-26T01:14:26.917 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2023-10-24_19:52:40-fs-wip-batrick-testing-20231024.144545-distro-default-smithi/7436434 branch: wip-batrick-testing-20231024.144545 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/mds-full} email: pdonnell@redhat.com first_in_suite: false job_id: '7436434' kernel: client: branch: testing kdb: true sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: pdonnell-2023-10-24_19:52:40-fs-wip-batrick-testing-20231024.144545-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-batrick-testing-20231024.144545 ceph: cephfs: ec_profile: - disabled cluster-conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mon: mon op complaint time: 120 osd: osd op complaint time: 180 conf: client: pid file: '' client.0: debug client: 20 debug objectcacher: 20 debug objecter: 20 client.1: debug client: 20 debug objectcacher: 20 debug objecter: 20 global: ms die on skipped message: false mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon osd backfillfull ratio: 0.6 mon osd full ratio: 0.7 mon osd nearfull ratio: 0.6 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 memstore device bytes: 200000000 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 1.0 osd mon report interval: 5 osd objectstore: memstore flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL - PG_RECOVERY_FULL - PG_DEGRADED sha1: 384993681abceb8cb8affea10538d607052db4b4 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 384993681abceb8cb8affea10538d607052db4b4 kclient: syntax: v1 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-batrick-testing-20231024.144545 sha1: 384993681abceb8cb8affea10538d607052db4b4 owner: scheduled_pdonnell@teuthology priority: 75 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 7595 sha1: 384993681abceb8cb8affea10538d607052db4b4 sleep_before_teardown: 0 subset: 66/256 suite: fs suite_branch: wip-batrick-testing-20231024.144545 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_384993681abceb8cb8affea10538d607052db4b4/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 384993681abceb8cb8affea10538d607052db4b4 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi169.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBMurhwyAOgLDZYp8KLlJfM2tGAae+jI6aesQjNLyzgOViSaU190GbQt3iYkIuSCQpGqFwejukievn06h7IOmAQ= smithi170.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOTrugeqpQFQkbL89tlvCCxYzOw91Gi/g8LcDRa/4kSdgb6tjnu8eLV09Jn06FyUdFgjJqFrxgYM07un/LnXr3g= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - kclient: null - cephfs_test_runner: modules: - tasks.cephfs.test_full teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 202b180cb047e798fb131047314a862593f45403 timestamp: 2023-10-24_19:52:40 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.770726 2023-10-26T01:14:26.918 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_384993681abceb8cb8affea10538d607052db4b4/qa; will attempt to use it 2023-10-26T01:14:26.919 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_384993681abceb8cb8affea10538d607052db4b4/qa/tasks 2023-10-26T01:14:26.919 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-10-26T01:14:26.921 INFO:teuthology.task.internal:Checking packages... 2023-10-26T01:14:26.945 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '384993681abceb8cb8affea10538d607052db4b4' 2023-10-26T01:14:26.946 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-10-26T01:14:26.946 INFO:teuthology.packaging:ref: None 2023-10-26T01:14:26.947 INFO:teuthology.packaging:tag: None 2023-10-26T01:14:26.947 INFO:teuthology.packaging:branch: wip-batrick-testing-20231024.144545 2023-10-26T01:14:26.947 INFO:teuthology.packaging:sha1: 384993681abceb8cb8affea10538d607052db4b4 2023-10-26T01:14:26.948 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-batrick-testing-20231024.144545 2023-10-26T01:14:27.162 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-6863.g38499368 2023-10-26T01:14:27.163 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-10-26T01:14:27.170 INFO:teuthology.task.internal:no buildpackages task found 2023-10-26T01:14:27.170 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-10-26T01:14:27.203 INFO:teuthology.task.internal:Saving configuration 2023-10-26T01:14:27.220 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-10-26T01:14:27.226 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-26T01:14:27.252 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi169.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-10-24_19:52:40-fs-wip-batrick-testing-20231024.144545-distro-default-smithi/7436434', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-26 01:09:21.887394', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBMurhwyAOgLDZYp8KLlJfM2tGAae+jI6aesQjNLyzgOViSaU190GbQt3iYkIuSCQpGqFwejukievn06h7IOmAQ='} 2023-10-26T01:14:27.276 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi170.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-10-24_19:52:40-fs-wip-batrick-testing-20231024.144545-distro-default-smithi/7436434', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-26 01:09:21.888879', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOTrugeqpQFQkbL89tlvCCxYzOw91Gi/g8LcDRa/4kSdgb6tjnu8eLV09Jn06FyUdFgjJqFrxgYM07un/LnXr3g='} 2023-10-26T01:14:27.276 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-10-26T01:14:27.282 INFO:teuthology.task.internal:roles: ubuntu@smithi169.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-10-26T01:14:27.283 INFO:teuthology.task.internal:roles: ubuntu@smithi170.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-10-26T01:14:27.283 INFO:teuthology.run_tasks:Running task console_log... 2023-10-26T01:14:27.352 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7ffa426a9b80>, signals=[15]) 2023-10-26T01:14:27.353 INFO:teuthology.run_tasks:Running task internal.connect... 2023-10-26T01:14:27.359 INFO:teuthology.task.internal:Opening connections... 2023-10-26T01:14:27.359 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi169.front.sepia.ceph.com 2023-10-26T01:14:27.361 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-26T01:14:27.445 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi170.front.sepia.ceph.com 2023-10-26T01:14:27.446 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-26T01:14:27.528 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-10-26T01:14:27.535 DEBUG:teuthology.orchestra.run.smithi169:> uname -m 2023-10-26T01:14:27.553 INFO:teuthology.orchestra.run.smithi169.stdout:x86_64 2023-10-26T01:14:27.553 DEBUG:teuthology.orchestra.run.smithi169:> cat /etc/os-release 2023-10-26T01:14:27.609 INFO:teuthology.orchestra.run.smithi169.stdout:NAME="CentOS Stream" 2023-10-26T01:14:27.609 INFO:teuthology.orchestra.run.smithi169.stdout:VERSION="9" 2023-10-26T01:14:27.610 INFO:teuthology.orchestra.run.smithi169.stdout:ID="centos" 2023-10-26T01:14:27.610 INFO:teuthology.orchestra.run.smithi169.stdout:ID_LIKE="rhel fedora" 2023-10-26T01:14:27.610 INFO:teuthology.orchestra.run.smithi169.stdout:VERSION_ID="9" 2023-10-26T01:14:27.610 INFO:teuthology.orchestra.run.smithi169.stdout:PLATFORM_ID="platform:el9" 2023-10-26T01:14:27.610 INFO:teuthology.orchestra.run.smithi169.stdout:PRETTY_NAME="CentOS Stream 9" 2023-10-26T01:14:27.611 INFO:teuthology.orchestra.run.smithi169.stdout:ANSI_COLOR="0;31" 2023-10-26T01:14:27.611 INFO:teuthology.orchestra.run.smithi169.stdout:LOGO="fedora-logo-icon" 2023-10-26T01:14:27.611 INFO:teuthology.orchestra.run.smithi169.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-10-26T01:14:27.611 INFO:teuthology.orchestra.run.smithi169.stdout:HOME_URL="https://centos.org/" 2023-10-26T01:14:27.611 INFO:teuthology.orchestra.run.smithi169.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-26T01:14:27.612 INFO:teuthology.orchestra.run.smithi169.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-10-26T01:14:27.612 INFO:teuthology.orchestra.run.smithi169.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-10-26T01:14:27.612 INFO:teuthology.lock.ops:Updating smithi169.front.sepia.ceph.com on lock server 2023-10-26T01:14:27.639 DEBUG:teuthology.orchestra.run.smithi170:> uname -m 2023-10-26T01:14:27.656 INFO:teuthology.orchestra.run.smithi170.stdout:x86_64 2023-10-26T01:14:27.656 DEBUG:teuthology.orchestra.run.smithi170:> cat /etc/os-release 2023-10-26T01:14:27.712 INFO:teuthology.orchestra.run.smithi170.stdout:NAME="CentOS Stream" 2023-10-26T01:14:27.713 INFO:teuthology.orchestra.run.smithi170.stdout:VERSION="9" 2023-10-26T01:14:27.713 INFO:teuthology.orchestra.run.smithi170.stdout:ID="centos" 2023-10-26T01:14:27.713 INFO:teuthology.orchestra.run.smithi170.stdout:ID_LIKE="rhel fedora" 2023-10-26T01:14:27.713 INFO:teuthology.orchestra.run.smithi170.stdout:VERSION_ID="9" 2023-10-26T01:14:27.713 INFO:teuthology.orchestra.run.smithi170.stdout:PLATFORM_ID="platform:el9" 2023-10-26T01:14:27.714 INFO:teuthology.orchestra.run.smithi170.stdout:PRETTY_NAME="CentOS Stream 9" 2023-10-26T01:14:27.714 INFO:teuthology.orchestra.run.smithi170.stdout:ANSI_COLOR="0;31" 2023-10-26T01:14:27.714 INFO:teuthology.orchestra.run.smithi170.stdout:LOGO="fedora-logo-icon" 2023-10-26T01:14:27.714 INFO:teuthology.orchestra.run.smithi170.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-10-26T01:14:27.714 INFO:teuthology.orchestra.run.smithi170.stdout:HOME_URL="https://centos.org/" 2023-10-26T01:14:27.715 INFO:teuthology.orchestra.run.smithi170.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-26T01:14:27.715 INFO:teuthology.orchestra.run.smithi170.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-10-26T01:14:27.715 INFO:teuthology.orchestra.run.smithi170.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-10-26T01:14:27.715 INFO:teuthology.lock.ops:Updating smithi170.front.sepia.ceph.com on lock server 2023-10-26T01:14:27.739 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-10-26T01:14:27.748 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-10-26T01:14:27.754 INFO:teuthology.task.internal:Checking for old test directory... 2023-10-26T01:14:27.754 DEBUG:teuthology.orchestra.run.smithi169:> test '!' -e /home/ubuntu/cephtest 2023-10-26T01:14:27.757 DEBUG:teuthology.orchestra.run.smithi170:> test '!' -e /home/ubuntu/cephtest 2023-10-26T01:14:27.773 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-10-26T01:14:27.778 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-10-26T01:14:27.779 DEBUG:teuthology.orchestra.run.smithi169:> test -z $(ls -A /var/lib/ceph) 2023-10-26T01:14:27.814 DEBUG:teuthology.orchestra.run.smithi170:> test -z $(ls -A /var/lib/ceph) 2023-10-26T01:14:27.913 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-10-26T01:14:27.965 INFO:teuthology.run_tasks:Running task kernel... 2023-10-26T01:14:27.980 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'} 2023-10-26T01:14:27.981 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}, 'client.2': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}, 'client.3': {'branch': 'testing'}} 2023-10-26T01:14:27.982 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}}, timeout 300 2023-10-26T01:14:27.982 DEBUG:teuthology.orchestra.run.smithi169:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-26T01:14:27.983 DEBUG:teuthology.orchestra.run.smithi170:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-26T01:14:27.998 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-26T01:14:27.999 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2023-10-26T01:14:28.002 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-26T01:14:28.003 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2023-10-26T01:14:28.149 DEBUG:teuthology.task.kernel:sha1 for client.1 is 5b73517e0e2fa132ea79290e5fc06ea5349b2288 2023-10-26T01:14:28.150 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "5b73517e0e2fa132ea79290e5fc06ea5349b2288"... 2023-10-26T01:14:28.150 DEBUG:teuthology.orchestra.run.smithi170:> uname -r 2023-10-26T01:14:28.167 INFO:teuthology.orchestra.run.smithi170.stdout:5.14.0-361.el9.x86_64 2023-10-26T01:14:28.168 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs 5b73517e0e2fa132ea79290e5fc06ea5349b2288 2023-10-26T01:14:28.168 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-10-26T01:14:28.168 DEBUG:teuthology.orchestra.run.smithi170:> sudo yum upgrade -y linux-firmware 2023-10-26T01:14:28.228 DEBUG:teuthology.task.kernel:sha1 for client.0 is 5b73517e0e2fa132ea79290e5fc06ea5349b2288 2023-10-26T01:14:28.228 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5b73517e0e2fa132ea79290e5fc06ea5349b2288"... 2023-10-26T01:14:28.228 DEBUG:teuthology.orchestra.run.smithi169:> uname -r 2023-10-26T01:14:28.245 INFO:teuthology.orchestra.run.smithi169.stdout:5.14.0-361.el9.x86_64 2023-10-26T01:14:28.245 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs 5b73517e0e2fa132ea79290e5fc06ea5349b2288 2023-10-26T01:14:28.245 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-10-26T01:14:28.246 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum upgrade -y linux-firmware 2023-10-26T01:14:30.474 INFO:teuthology.orchestra.run.smithi170.stdout:CentOS-9 - AppStream 9.6 MB/s | 6.3 MB 00:00 2023-10-26T01:14:30.483 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS-9 - AppStream 11 MB/s | 6.3 MB 00:00 2023-10-26T01:14:31.791 INFO:teuthology.orchestra.run.smithi170.stdout:CentOS-9 - BaseOS 6.8 MB/s | 2.0 MB 00:00 2023-10-26T01:14:32.092 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS-9 - BaseOS 3.9 MB/s | 2.0 MB 00:00 2023-10-26T01:14:33.074 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 7.9 MB 00:00 2023-10-26T01:14:33.090 INFO:teuthology.orchestra.run.smithi170.stdout:CentOS Stream 9 - BaseOS 8.1 MB/s | 7.9 MB 00:00 2023-10-26T01:14:35.225 INFO:teuthology.orchestra.run.smithi170.stdout:CentOS Stream 9 - AppStream 16 MB/s | 18 MB 00:01 2023-10-26T01:14:36.011 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS Stream 9 - AppStream 9.4 MB/s | 18 MB 00:01 2023-10-26T01:14:39.986 INFO:teuthology.orchestra.run.smithi170.stdout:CentOS Stream 9 - CRB 4.0 MB/s | 5.7 MB 00:01 2023-10-26T01:14:40.476 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS Stream 9 - CRB 4.9 MB/s | 5.7 MB 00:01 2023-10-26T01:14:41.543 INFO:teuthology.orchestra.run.smithi170.stdout:CentOS Stream 9 - Extras packages 30 kB/s | 15 kB 00:00 2023-10-26T01:14:41.901 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS Stream 9 - Extras packages 43 kB/s | 15 kB 00:00 2023-10-26T01:14:44.654 INFO:teuthology.orchestra.run.smithi169.stdout:Extra Packages for Enterprise Linux 7.3 MB/s | 19 MB 00:02 2023-10-26T01:14:49.261 INFO:teuthology.orchestra.run.smithi169.stdout:lab-extras 13 kB/s | 1.7 kB 00:00 2023-10-26T01:14:49.571 INFO:teuthology.orchestra.run.smithi170.stdout:Extra Packages for Enterprise Linux 2.4 MB/s | 19 MB 00:07 2023-10-26T01:14:51.429 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2023-10-26T01:14:51.440 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2023-10-26T01:14:51.440 INFO:teuthology.orchestra.run.smithi169.stdout: Package Arch Version Repository Size 2023-10-26T01:14:51.440 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2023-10-26T01:14:51.440 INFO:teuthology.orchestra.run.smithi169.stdout:Upgrading: 2023-10-26T01:14:51.441 INFO:teuthology.orchestra.run.smithi169.stdout: linux-firmware noarch 20230926-140.el9 CentOS-BaseOS 326 M 2023-10-26T01:14:51.441 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:14:51.441 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction Summary 2023-10-26T01:14:51.441 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2023-10-26T01:14:51.441 INFO:teuthology.orchestra.run.smithi169.stdout:Upgrade 1 Package 2023-10-26T01:14:51.441 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:14:51.442 INFO:teuthology.orchestra.run.smithi169.stdout:Total download size: 326 M 2023-10-26T01:14:51.442 INFO:teuthology.orchestra.run.smithi169.stdout:Downloading Packages: 2023-10-26T01:14:54.118 INFO:teuthology.orchestra.run.smithi170.stdout:lab-extras 15 kB/s | 1.7 kB 00:00 2023-10-26T01:14:56.323 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2023-10-26T01:14:56.331 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2023-10-26T01:14:56.331 INFO:teuthology.orchestra.run.smithi170.stdout: Package Arch Version Repository Size 2023-10-26T01:14:56.331 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2023-10-26T01:14:56.332 INFO:teuthology.orchestra.run.smithi170.stdout:Upgrading: 2023-10-26T01:14:56.332 INFO:teuthology.orchestra.run.smithi170.stdout: linux-firmware noarch 20230926-140.el9 CentOS-BaseOS 326 M 2023-10-26T01:14:56.332 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-26T01:14:56.333 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction Summary 2023-10-26T01:14:56.333 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2023-10-26T01:14:56.333 INFO:teuthology.orchestra.run.smithi170.stdout:Upgrade 1 Package 2023-10-26T01:14:56.334 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-26T01:14:56.334 INFO:teuthology.orchestra.run.smithi170.stdout:Total download size: 326 M 2023-10-26T01:14:56.335 INFO:teuthology.orchestra.run.smithi170.stdout:Downloading Packages: 2023-10-26T01:15:00.469 INFO:teuthology.orchestra.run.smithi169.stdout:linux-firmware-20230926-140.el9.noarch.rpm 36 MB/s | 326 MB 00:09 2023-10-26T01:15:00.471 INFO:teuthology.orchestra.run.smithi169.stdout:-------------------------------------------------------------------------------- 2023-10-26T01:15:00.471 INFO:teuthology.orchestra.run.smithi169.stdout:Total 36 MB/s | 326 MB 00:09 2023-10-26T01:15:00.471 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction check 2023-10-26T01:15:00.557 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction check succeeded. 2023-10-26T01:15:00.557 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction test 2023-10-26T01:15:02.702 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction test succeeded. 2023-10-26T01:15:02.702 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction 2023-10-26T01:15:04.727 INFO:teuthology.orchestra.run.smithi170.stdout:linux-firmware-20230926-140.el9.noarch.rpm 39 MB/s | 326 MB 00:08 2023-10-26T01:15:04.728 INFO:teuthology.orchestra.run.smithi170.stdout:-------------------------------------------------------------------------------- 2023-10-26T01:15:04.728 INFO:teuthology.orchestra.run.smithi170.stdout:Total 39 MB/s | 326 MB 00:08 2023-10-26T01:15:04.729 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction check 2023-10-26T01:15:04.818 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction check succeeded. 2023-10-26T01:15:04.819 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction test 2023-10-26T01:15:06.888 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction test succeeded. 2023-10-26T01:15:06.888 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction 2023-10-26T01:15:08.364 INFO:teuthology.orchestra.run.smithi169.stdout: Preparing : 1/1 2023-10-26T01:15:08.769 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : linux-firmware-20230926-140.el9.noarch 1/2 2023-10-26T01:15:08.858 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2023-10-26T01:15:09.893 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2023-10-26T01:15:09.894 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : linux-firmware-20230926-140.el9.noarch 1/2 2023-10-26T01:15:10.269 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2023-10-26T01:15:10.269 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:15:10.270 INFO:teuthology.orchestra.run.smithi169.stdout:Upgraded: 2023-10-26T01:15:10.270 INFO:teuthology.orchestra.run.smithi169.stdout: linux-firmware-20230926-140.el9.noarch 2023-10-26T01:15:10.270 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:15:10.270 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2023-10-26T01:15:10.563 INFO:teuthology.task.kernel:Downloading kernel 5b73517e0e2fa132ea79290e5fc06ea5349b2288 on client.0... 2023-10-26T01:15:10.563 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5b73517e0e2fa132ea79290e5fc06ea5349b2288 2023-10-26T01:15:10.784 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/5b73517e0e2fa132ea79290e5fc06ea5349b2288/centos/9/flavors/default/x86_64/ 2023-10-26T01:15:10.784 DEBUG:teuthology.orchestra.run.smithi169:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.6.0_rc5_g5b73517e0e2f-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/5b73517e0e2fa132ea79290e5fc06ea5349b2288/centos/9/flavors/default/x86_64/ --input-file=- 2023-10-26T01:15:10.787 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi169.front.sepia.ceph.com to complete... 2023-10-26T01:15:12.549 INFO:teuthology.orchestra.run.smithi170.stdout: Preparing : 1/1 2023-10-26T01:15:13.042 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : linux-firmware-20230926-140.el9.noarch 1/2 2023-10-26T01:15:13.132 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2023-10-26T01:15:13.932 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2023-10-26T01:15:13.933 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : linux-firmware-20230926-140.el9.noarch 1/2 2023-10-26T01:15:14.336 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2023-10-26T01:15:14.336 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-26T01:15:14.336 INFO:teuthology.orchestra.run.smithi170.stdout:Upgraded: 2023-10-26T01:15:14.337 INFO:teuthology.orchestra.run.smithi170.stdout: linux-firmware-20230926-140.el9.noarch 2023-10-26T01:15:14.337 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-26T01:15:14.337 INFO:teuthology.orchestra.run.smithi170.stdout:Complete! 2023-10-26T01:15:14.656 INFO:teuthology.task.kernel:Downloading kernel 5b73517e0e2fa132ea79290e5fc06ea5349b2288 on client.1... 2023-10-26T01:15:14.657 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5b73517e0e2fa132ea79290e5fc06ea5349b2288 2023-10-26T01:15:14.803 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/5b73517e0e2fa132ea79290e5fc06ea5349b2288/centos/9/flavors/default/x86_64/ 2023-10-26T01:15:14.803 DEBUG:teuthology.orchestra.run.smithi170:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.6.0_rc5_g5b73517e0e2f-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/5b73517e0e2fa132ea79290e5fc06ea5349b2288/centos/9/flavors/default/x86_64/ --input-file=- 2023-10-26T01:15:14.805 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi170.front.sepia.ceph.com to complete... 2023-10-26T01:15:29.083 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26 01:15:29 URL:https://3.chacra.ceph.com/r/kernel/testing/5b73517e0e2fa132ea79290e5fc06ea5349b2288/centos/9/flavors/default/x86_64/kernel-6.6.0_rc5_g5b73517e0e2f-1.x86_64.rpm [320585487/320585487] -> "/tmp/kernel.x86_64.rpm" [1] 2023-10-26T01:15:29.083 INFO:teuthology.orchestra.run.smithi170.stderr:FINISHED --2023-10-26 01:15:29-- 2023-10-26T01:15:29.084 INFO:teuthology.orchestra.run.smithi170.stderr:Total wall clock time: 14s 2023-10-26T01:15:29.084 INFO:teuthology.orchestra.run.smithi170.stderr:Downloaded: 1 files, 306M in 14s (21.9 MB/s) 2023-10-26T01:15:29.085 INFO:teuthology.task.kernel:Installing kernel 5b73517e0e2fa132ea79290e5fc06ea5349b2288 on client.1... 2023-10-26T01:15:29.086 DEBUG:teuthology.orchestra.run.smithi170:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-10-26T01:15:30.938 INFO:teuthology.orchestra.run.smithi170.stdout:Verifying... ######################################## 2023-10-26T01:15:30.993 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing... ######################################## 2023-10-26T01:15:31.073 INFO:teuthology.orchestra.run.smithi170.stdout:Updating / installing... 2023-10-26T01:15:36.129 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26 01:15:36 URL:https://3.chacra.ceph.com/r/kernel/testing/5b73517e0e2fa132ea79290e5fc06ea5349b2288/centos/9/flavors/default/x86_64/kernel-6.6.0_rc5_g5b73517e0e2f-1.x86_64.rpm [320585487/320585487] -> "/tmp/kernel.x86_64.rpm" [1] 2023-10-26T01:15:36.129 INFO:teuthology.orchestra.run.smithi169.stderr:FINISHED --2023-10-26 01:15:36-- 2023-10-26T01:15:36.129 INFO:teuthology.orchestra.run.smithi169.stderr:Total wall clock time: 25s 2023-10-26T01:15:36.129 INFO:teuthology.orchestra.run.smithi169.stderr:Downloaded: 1 files, 306M in 25s (12.2 MB/s) 2023-10-26T01:15:36.131 INFO:teuthology.task.kernel:Installing kernel 5b73517e0e2fa132ea79290e5fc06ea5349b2288 on client.0... 2023-10-26T01:15:36.131 DEBUG:teuthology.orchestra.run.smithi169:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-10-26T01:15:37.985 INFO:teuthology.orchestra.run.smithi169.stdout:Verifying... ######################################## 2023-10-26T01:15:38.073 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing... ######################################## 2023-10-26T01:15:38.153 INFO:teuthology.orchestra.run.smithi169.stdout:Updating / installing... 2023-10-26T01:15:38.164 INFO:teuthology.orchestra.run.smithi170.stdout:kernel-6.6.0_rc5_g5b73517e0e2f-1 ######################################## 2023-10-26T01:15:45.264 INFO:teuthology.orchestra.run.smithi169.stdout:kernel-6.6.0_rc5_g5b73517e0e2f-1 ######################################## 2023-10-26T01:15:53.514 INFO:teuthology.orchestra.run.smithi170.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2023-10-26T01:16:00.611 INFO:teuthology.orchestra.run.smithi169.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2023-10-26T01:16:14.727 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi170.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-10-26T01:16:14.728 DEBUG:teuthology.orchestra.run.smithi170:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-10-26T01:16:14.806 INFO:teuthology.orchestra.run.smithi170.stdout:/boot/System.map-6.6.0-rc5-g5b73517e0e2f 2023-10-26T01:16:14.807 INFO:teuthology.orchestra.run.smithi170.stdout:/boot/config-6.6.0-rc5-g5b73517e0e2f 2023-10-26T01:16:14.807 INFO:teuthology.orchestra.run.smithi170.stdout:/boot/vmlinuz-6.6.0-rc5-g5b73517e0e2f 2023-10-26T01:16:14.807 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f 2023-10-26T01:16:14.807 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel 2023-10-26T01:16:14.808 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch 2023-10-26T01:16:14.808 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86 2023-10-26T01:16:14.808 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto 2023-10-26T01:16:14.808 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-10-26T01:16:14.808 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-10-26T01:16:14.809 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-10-26T01:16:14.809 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-10-26T01:16:14.809 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-10-26T01:16:14.809 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-10-26T01:16:14.809 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-10-26T01:16:14.810 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/crc32c-intel.ko 2023-10-26T01:16:14.810 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-10-26T01:16:14.810 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-10-26T01:16:14.810 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-10-26T01:16:14.811 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-10-26T01:16:14.811 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/serpent-avx2.ko 2023-10-26T01:16:14.811 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-10-26T01:16:14.811 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-10-26T01:16:14.811 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-10-26T01:16:14.812 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-10-26T01:16:14.812 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-10-26T01:16:14.812 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-10-26T01:16:14.812 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-10-26T01:16:14.812 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/kernel 2023-10-26T01:16:14.813 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/kernel/cpu 2023-10-26T01:16:14.813 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/kernel/cpu/mce 2023-10-26T01:16:14.813 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-10-26T01:16:14.813 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/kvm 2023-10-26T01:16:14.814 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/kvm/kvm-amd.ko 2023-10-26T01:16:14.814 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/kvm/kvm-intel.ko 2023-10-26T01:16:14.814 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/kvm/kvm.ko 2023-10-26T01:16:14.814 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto 2023-10-26T01:16:14.815 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/ansi_cprng.ko 2023-10-26T01:16:14.815 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/anubis.ko 2023-10-26T01:16:14.815 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/arc4.ko 2023-10-26T01:16:14.815 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/async_tx 2023-10-26T01:16:14.815 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/async_tx/async_memcpy.ko 2023-10-26T01:16:14.816 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/async_tx/async_pq.ko 2023-10-26T01:16:14.816 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/async_tx/async_raid6_recov.ko 2023-10-26T01:16:14.816 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/async_tx/async_tx.ko 2023-10-26T01:16:14.816 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/async_tx/async_xor.ko 2023-10-26T01:16:14.817 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/async_tx/raid6test.ko 2023-10-26T01:16:14.817 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/authenc.ko 2023-10-26T01:16:14.817 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/authencesn.ko 2023-10-26T01:16:14.817 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/blake2b_generic.ko 2023-10-26T01:16:14.817 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/blowfish_common.ko 2023-10-26T01:16:14.818 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/blowfish_generic.ko 2023-10-26T01:16:14.818 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/camellia_generic.ko 2023-10-26T01:16:14.818 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/cast5_generic.ko 2023-10-26T01:16:14.818 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/cast6_generic.ko 2023-10-26T01:16:14.818 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/cast_common.ko 2023-10-26T01:16:14.819 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/ccm.ko 2023-10-26T01:16:14.819 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/cmac.ko 2023-10-26T01:16:14.819 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/crc32_generic.ko 2023-10-26T01:16:14.819 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/crypto_engine.ko 2023-10-26T01:16:14.820 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/crypto_user.ko 2023-10-26T01:16:14.820 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/cts.ko 2023-10-26T01:16:14.820 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/des_generic.ko 2023-10-26T01:16:14.820 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/dh_generic.ko 2023-10-26T01:16:14.820 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/echainiv.ko 2023-10-26T01:16:14.821 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/essiv.ko 2023-10-26T01:16:14.821 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/fcrypt.ko 2023-10-26T01:16:14.821 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/khazad.ko 2023-10-26T01:16:14.821 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/lz4.ko 2023-10-26T01:16:14.821 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/lz4hc.ko 2023-10-26T01:16:14.822 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/md4.ko 2023-10-26T01:16:14.822 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/michael_mic.ko 2023-10-26T01:16:14.822 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/pcbc.ko 2023-10-26T01:16:14.822 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/pcrypt.ko 2023-10-26T01:16:14.823 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/rmd160.ko 2023-10-26T01:16:14.823 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/seed.ko 2023-10-26T01:16:14.823 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/serpent_generic.ko 2023-10-26T01:16:14.823 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/tcrypt.ko 2023-10-26T01:16:14.824 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/tea.ko 2023-10-26T01:16:14.824 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/twofish_common.ko 2023-10-26T01:16:14.824 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/twofish_generic.ko 2023-10-26T01:16:14.824 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/vmac.ko 2023-10-26T01:16:14.824 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/wp512.ko 2023-10-26T01:16:14.825 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/xcbc.ko 2023-10-26T01:16:14.825 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/xor.ko 2023-10-26T01:16:14.825 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/xxhash_generic.ko 2023-10-26T01:16:14.825 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers 2023-10-26T01:16:14.826 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi 2023-10-26T01:16:14.826 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi/acpi_ipmi.ko 2023-10-26T01:16:14.826 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi/acpi_pad.ko 2023-10-26T01:16:14.826 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi/custom_method.ko 2023-10-26T01:16:14.826 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi/ec_sys.ko 2023-10-26T01:16:14.827 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi/platform_profile.ko 2023-10-26T01:16:14.827 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi/sbs.ko 2023-10-26T01:16:14.827 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi/sbshc.ko 2023-10-26T01:16:14.827 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi/video.ko 2023-10-26T01:16:14.827 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata 2023-10-26T01:16:14.828 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/acard-ahci.ko 2023-10-26T01:16:14.828 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/ahci_platform.ko 2023-10-26T01:16:14.828 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/ata_generic.ko 2023-10-26T01:16:14.828 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/libahci_platform.ko 2023-10-26T01:16:14.829 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_acpi.ko 2023-10-26T01:16:14.829 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_ali.ko 2023-10-26T01:16:14.829 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_amd.ko 2023-10-26T01:16:14.829 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_artop.ko 2023-10-26T01:16:14.829 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_atiixp.ko 2023-10-26T01:16:14.830 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_atp867x.ko 2023-10-26T01:16:14.830 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_cmd640.ko 2023-10-26T01:16:14.830 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_cmd64x.ko 2023-10-26T01:16:14.830 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_cypress.ko 2023-10-26T01:16:14.831 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_efar.ko 2023-10-26T01:16:14.831 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_hpt366.ko 2023-10-26T01:16:14.831 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_hpt37x.ko 2023-10-26T01:16:14.831 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_hpt3x2n.ko 2023-10-26T01:16:14.831 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_hpt3x3.ko 2023-10-26T01:16:14.832 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_it8213.ko 2023-10-26T01:16:14.832 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_it821x.ko 2023-10-26T01:16:14.832 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_jmicron.ko 2023-10-26T01:16:14.832 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_marvell.ko 2023-10-26T01:16:14.833 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_mpiix.ko 2023-10-26T01:16:14.833 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_netcell.ko 2023-10-26T01:16:14.833 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_ninja32.ko 2023-10-26T01:16:14.833 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_ns87410.ko 2023-10-26T01:16:14.833 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_ns87415.ko 2023-10-26T01:16:14.834 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_oldpiix.ko 2023-10-26T01:16:14.834 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_opti.ko 2023-10-26T01:16:14.834 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_optidma.ko 2023-10-26T01:16:14.834 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_pcmcia.ko 2023-10-26T01:16:14.835 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_pdc2027x.ko 2023-10-26T01:16:14.835 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-10-26T01:16:14.835 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_piccolo.ko 2023-10-26T01:16:14.835 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_rdc.ko 2023-10-26T01:16:14.835 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_sch.ko 2023-10-26T01:16:14.836 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_serverworks.ko 2023-10-26T01:16:14.836 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_sil680.ko 2023-10-26T01:16:14.836 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_sis.ko 2023-10-26T01:16:14.836 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_sl82c105.ko 2023-10-26T01:16:14.836 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_triflex.ko 2023-10-26T01:16:14.837 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_via.ko 2023-10-26T01:16:14.837 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pdc_adma.ko 2023-10-26T01:16:14.837 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_inic162x.ko 2023-10-26T01:16:14.837 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_mv.ko 2023-10-26T01:16:14.838 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_nv.ko 2023-10-26T01:16:14.838 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_promise.ko 2023-10-26T01:16:14.838 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_qstor.ko 2023-10-26T01:16:14.838 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_sil.ko 2023-10-26T01:16:14.838 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_sil24.ko 2023-10-26T01:16:14.839 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_sis.ko 2023-10-26T01:16:14.839 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_svw.ko 2023-10-26T01:16:14.839 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_sx4.ko 2023-10-26T01:16:14.839 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_uli.ko 2023-10-26T01:16:14.840 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_via.ko 2023-10-26T01:16:14.840 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_vsc.ko 2023-10-26T01:16:14.840 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/auxdisplay 2023-10-26T01:16:14.840 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/auxdisplay/ks0108.ko 2023-10-26T01:16:14.840 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/base 2023-10-26T01:16:14.841 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/base/regmap 2023-10-26T01:16:14.841 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/base/regmap/regmap-i2c.ko 2023-10-26T01:16:14.841 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/base/regmap/regmap-mmio.ko 2023-10-26T01:16:14.841 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/bcma 2023-10-26T01:16:14.841 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/bcma/bcma.ko 2023-10-26T01:16:14.842 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block 2023-10-26T01:16:14.842 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/aoe 2023-10-26T01:16:14.842 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/aoe/aoe.ko 2023-10-26T01:16:14.842 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/brd.ko 2023-10-26T01:16:14.843 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/drbd 2023-10-26T01:16:14.843 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/drbd/drbd.ko 2023-10-26T01:16:14.843 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/floppy.ko 2023-10-26T01:16:14.843 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/loop.ko 2023-10-26T01:16:14.843 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/mtip32xx 2023-10-26T01:16:14.844 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-10-26T01:16:14.844 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/nbd.ko 2023-10-26T01:16:14.844 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/null_blk 2023-10-26T01:16:14.844 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/null_blk/null_blk.ko 2023-10-26T01:16:14.844 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/pktcdvd.ko 2023-10-26T01:16:14.845 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/rbd.ko 2023-10-26T01:16:14.845 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/virtio_blk.ko 2023-10-26T01:16:14.845 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/xen-blkback 2023-10-26T01:16:14.845 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-10-26T01:16:14.845 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/xen-blkfront.ko 2023-10-26T01:16:14.846 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/zram 2023-10-26T01:16:14.846 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/zram/zram.ko 2023-10-26T01:16:14.846 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char 2023-10-26T01:16:14.846 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/hangcheck-timer.ko 2023-10-26T01:16:14.847 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/hw_random 2023-10-26T01:16:14.847 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/hw_random/amd-rng.ko 2023-10-26T01:16:14.847 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/hw_random/intel-rng.ko 2023-10-26T01:16:14.847 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-10-26T01:16:14.847 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/hw_random/via-rng.ko 2023-10-26T01:16:14.848 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/hw_random/virtio-rng.ko 2023-10-26T01:16:14.848 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/ipmi 2023-10-26T01:16:14.848 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-10-26T01:16:14.848 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-10-26T01:16:14.849 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-10-26T01:16:14.849 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/ipmi/ipmi_si.ko 2023-10-26T01:16:14.849 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-10-26T01:16:14.849 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-10-26T01:16:14.849 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/lp.ko 2023-10-26T01:16:14.850 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/mwave 2023-10-26T01:16:14.850 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/mwave/mwave.ko 2023-10-26T01:16:14.850 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/ppdev.ko 2023-10-26T01:16:14.850 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/tlclk.ko 2023-10-26T01:16:14.851 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/tpm 2023-10-26T01:16:14.851 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/tpm/tpm.ko 2023-10-26T01:16:14.851 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/tpm/tpm_atmel.ko 2023-10-26T01:16:14.851 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/tpm/tpm_infineon.ko 2023-10-26T01:16:14.851 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/tpm/tpm_nsc.ko 2023-10-26T01:16:14.852 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/tpm/tpm_tis.ko 2023-10-26T01:16:14.852 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-10-26T01:16:14.852 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/uv_mmtimer.ko 2023-10-26T01:16:14.852 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/virtio_console.ko 2023-10-26T01:16:14.853 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/cpufreq 2023-10-26T01:16:14.853 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-10-26T01:16:14.853 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-10-26T01:16:14.854 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/cpufreq/p4-clockmod.ko 2023-10-26T01:16:14.854 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-10-26T01:16:14.854 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/cpufreq/powernow-k8.ko 2023-10-26T01:16:14.854 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/cpufreq/speedstep-lib.ko 2023-10-26T01:16:14.854 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto 2023-10-26T01:16:14.855 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/ccp 2023-10-26T01:16:14.855 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-10-26T01:16:14.855 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/ccp/ccp.ko 2023-10-26T01:16:14.855 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/intel 2023-10-26T01:16:14.855 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/intel/qat 2023-10-26T01:16:14.856 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/intel/qat/qat_common 2023-10-26T01:16:14.856 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-10-26T01:16:14.856 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-10-26T01:16:14.856 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-10-26T01:16:14.857 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/padlock-aes.ko 2023-10-26T01:16:14.857 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/padlock-sha.ko 2023-10-26T01:16:14.857 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/virtio 2023-10-26T01:16:14.857 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-10-26T01:16:14.857 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/dca 2023-10-26T01:16:14.858 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/dca/dca.ko 2023-10-26T01:16:14.858 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/devfreq 2023-10-26T01:16:14.858 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-10-26T01:16:14.858 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/dma 2023-10-26T01:16:14.858 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/dma/dw 2023-10-26T01:16:14.859 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/dma/dw/dw_dmac.ko 2023-10-26T01:16:14.859 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/dma/ioat 2023-10-26T01:16:14.859 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/dma/ioat/ioatdma.ko 2023-10-26T01:16:14.859 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac 2023-10-26T01:16:14.859 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/amd64_edac.ko 2023-10-26T01:16:14.860 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/e752x_edac.ko 2023-10-26T01:16:14.860 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/edac_mce_amd.ko 2023-10-26T01:16:14.860 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/i3000_edac.ko 2023-10-26T01:16:14.860 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/i3200_edac.ko 2023-10-26T01:16:14.861 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/i5100_edac.ko 2023-10-26T01:16:14.861 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/i5400_edac.ko 2023-10-26T01:16:14.861 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/i7300_edac.ko 2023-10-26T01:16:14.861 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/i7core_edac.ko 2023-10-26T01:16:14.861 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/i82975x_edac.ko 2023-10-26T01:16:14.862 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/ie31200_edac.ko 2023-10-26T01:16:14.862 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/sb_edac.ko 2023-10-26T01:16:14.862 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/x38_edac.ko 2023-10-26T01:16:14.862 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/firmware 2023-10-26T01:16:14.862 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/firmware/edd.ko 2023-10-26T01:16:14.863 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/firmware/iscsi_ibft.ko 2023-10-26T01:16:14.863 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/gpio 2023-10-26T01:16:14.863 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/gpio/gpio-ich.ko 2023-10-26T01:16:14.863 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/gpio/gpio-viperboard.ko 2023-10-26T01:16:14.863 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid 2023-10-26T01:16:14.864 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-appleir.ko 2023-10-26T01:16:14.864 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-aureal.ko 2023-10-26T01:16:14.864 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-axff.ko 2023-10-26T01:16:14.864 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-dr.ko 2023-10-26T01:16:14.864 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-elecom.ko 2023-10-26T01:16:14.865 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-elo.ko 2023-10-26T01:16:14.865 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-emsff.ko 2023-10-26T01:16:14.865 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-gaff.ko 2023-10-26T01:16:14.865 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-gt683r.ko 2023-10-26T01:16:14.866 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-gyration.ko 2023-10-26T01:16:14.866 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-holtek-kbd.ko 2023-10-26T01:16:14.866 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-holtek-mouse.ko 2023-10-26T01:16:14.866 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-holtekff.ko 2023-10-26T01:16:14.866 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-hyperv.ko 2023-10-26T01:16:14.867 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-icade.ko 2023-10-26T01:16:14.867 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-keytouch.ko 2023-10-26T01:16:14.867 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-kye.ko 2023-10-26T01:16:14.867 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-lcpower.ko 2023-10-26T01:16:14.867 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-led.ko 2023-10-26T01:16:14.868 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-lenovo.ko 2023-10-26T01:16:14.868 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-logitech-dj.ko 2023-10-26T01:16:14.868 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-10-26T01:16:14.868 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-multitouch.ko 2023-10-26T01:16:14.868 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-ortek.ko 2023-10-26T01:16:14.869 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-penmount.ko 2023-10-26T01:16:14.869 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-petalynx.ko 2023-10-26T01:16:14.869 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-picolcd.ko 2023-10-26T01:16:14.869 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-pl.ko 2023-10-26T01:16:14.870 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-plantronics.ko 2023-10-26T01:16:14.870 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-primax.ko 2023-10-26T01:16:14.870 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-rmi.ko 2023-10-26T01:16:14.870 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-arvo.ko 2023-10-26T01:16:14.870 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-common.ko 2023-10-26T01:16:14.871 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-isku.ko 2023-10-26T01:16:14.871 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-kone.ko 2023-10-26T01:16:14.871 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-10-26T01:16:14.871 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-konepure.ko 2023-10-26T01:16:14.872 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-10-26T01:16:14.872 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-lua.ko 2023-10-26T01:16:14.872 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-pyra.ko 2023-10-26T01:16:14.872 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-ryos.ko 2023-10-26T01:16:14.872 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-savu.ko 2023-10-26T01:16:14.873 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat.ko 2023-10-26T01:16:14.873 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-saitek.ko 2023-10-26T01:16:14.873 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-samsung.ko 2023-10-26T01:16:14.873 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-sensor-hub.ko 2023-10-26T01:16:14.873 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-sjoy.ko 2023-10-26T01:16:14.874 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-sony.ko 2023-10-26T01:16:14.874 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-speedlink.ko 2023-10-26T01:16:14.874 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-steelseries.ko 2023-10-26T01:16:14.874 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-sunplus.ko 2023-10-26T01:16:14.874 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-thrustmaster.ko 2023-10-26T01:16:14.875 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-tivo.ko 2023-10-26T01:16:14.875 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-tmff.ko 2023-10-26T01:16:14.875 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-topseed.ko 2023-10-26T01:16:14.875 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-twinhan.ko 2023-10-26T01:16:14.876 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-uclogic.ko 2023-10-26T01:16:14.876 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-waltop.ko 2023-10-26T01:16:14.876 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-wiimote.ko 2023-10-26T01:16:14.876 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-xinmo.ko 2023-10-26T01:16:14.876 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-zpff.ko 2023-10-26T01:16:14.877 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-zydacron.ko 2023-10-26T01:16:14.877 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/uhid.ko 2023-10-26T01:16:14.877 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/wacom.ko 2023-10-26T01:16:14.877 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hv 2023-10-26T01:16:14.878 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hv/hv_balloon.ko 2023-10-26T01:16:14.878 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hv/hv_utils.ko 2023-10-26T01:16:14.878 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hv/hv_vmbus.ko 2023-10-26T01:16:14.878 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon 2023-10-26T01:16:14.878 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/abituguru.ko 2023-10-26T01:16:14.879 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/abituguru3.ko 2023-10-26T01:16:14.879 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/acpi_power_meter.ko 2023-10-26T01:16:14.879 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ad7414.ko 2023-10-26T01:16:14.879 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ad7418.ko 2023-10-26T01:16:14.879 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adc128d818.ko 2023-10-26T01:16:14.880 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adm1025.ko 2023-10-26T01:16:14.880 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adm1026.ko 2023-10-26T01:16:14.880 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adm1029.ko 2023-10-26T01:16:14.880 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adm1031.ko 2023-10-26T01:16:14.881 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adm9240.ko 2023-10-26T01:16:14.881 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ads7828.ko 2023-10-26T01:16:14.881 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adt7410.ko 2023-10-26T01:16:14.881 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adt7411.ko 2023-10-26T01:16:14.881 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adt7462.ko 2023-10-26T01:16:14.882 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adt7470.ko 2023-10-26T01:16:14.882 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adt7475.ko 2023-10-26T01:16:14.882 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adt7x10.ko 2023-10-26T01:16:14.882 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/amc6821.ko 2023-10-26T01:16:14.882 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/applesmc.ko 2023-10-26T01:16:14.883 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/asb100.ko 2023-10-26T01:16:14.883 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/asc7621.ko 2023-10-26T01:16:14.883 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/asus_atk0110.ko 2023-10-26T01:16:14.883 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/atxp1.ko 2023-10-26T01:16:14.883 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/coretemp.ko 2023-10-26T01:16:14.884 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-10-26T01:16:14.884 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/dme1737.ko 2023-10-26T01:16:14.884 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ds1621.ko 2023-10-26T01:16:14.884 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ds620.ko 2023-10-26T01:16:14.885 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/emc1403.ko 2023-10-26T01:16:14.885 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/emc6w201.ko 2023-10-26T01:16:14.885 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/f71805f.ko 2023-10-26T01:16:14.885 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/f71882fg.ko 2023-10-26T01:16:14.885 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/f75375s.ko 2023-10-26T01:16:14.886 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/fam15h_power.ko 2023-10-26T01:16:14.886 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/fschmd.ko 2023-10-26T01:16:14.886 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/g760a.ko 2023-10-26T01:16:14.886 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/g762.ko 2023-10-26T01:16:14.886 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/gl518sm.ko 2023-10-26T01:16:14.887 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/gl520sm.ko 2023-10-26T01:16:14.887 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/hwmon-vid.ko 2023-10-26T01:16:14.887 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/i5500_temp.ko 2023-10-26T01:16:14.887 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/i5k_amb.ko 2023-10-26T01:16:14.887 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ibmaem.ko 2023-10-26T01:16:14.888 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ibmpex.ko 2023-10-26T01:16:14.888 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ina209.ko 2023-10-26T01:16:14.888 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ina2xx.ko 2023-10-26T01:16:14.888 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/it87.ko 2023-10-26T01:16:14.889 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/k10temp.ko 2023-10-26T01:16:14.889 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/k8temp.ko 2023-10-26T01:16:14.889 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lineage-pem.ko 2023-10-26T01:16:14.889 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm63.ko 2023-10-26T01:16:14.889 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm73.ko 2023-10-26T01:16:14.890 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm75.ko 2023-10-26T01:16:14.890 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm77.ko 2023-10-26T01:16:14.890 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm78.ko 2023-10-26T01:16:14.890 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm80.ko 2023-10-26T01:16:14.890 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm83.ko 2023-10-26T01:16:14.891 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm85.ko 2023-10-26T01:16:14.891 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm87.ko 2023-10-26T01:16:14.891 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm90.ko 2023-10-26T01:16:14.891 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm92.ko 2023-10-26T01:16:14.893 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm93.ko 2023-10-26T01:16:14.893 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm95234.ko 2023-10-26T01:16:14.893 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm95241.ko 2023-10-26T01:16:14.893 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm95245.ko 2023-10-26T01:16:14.894 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ltc2945.ko 2023-10-26T01:16:14.894 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ltc4151.ko 2023-10-26T01:16:14.894 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ltc4215.ko 2023-10-26T01:16:14.894 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ltc4222.ko 2023-10-26T01:16:14.894 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ltc4245.ko 2023-10-26T01:16:14.895 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ltc4260.ko 2023-10-26T01:16:14.895 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ltc4261.ko 2023-10-26T01:16:14.895 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/max16065.ko 2023-10-26T01:16:14.895 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/max1619.ko 2023-10-26T01:16:14.896 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/max1668.ko 2023-10-26T01:16:14.896 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/max197.ko 2023-10-26T01:16:14.896 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/max6639.ko 2023-10-26T01:16:14.896 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/max6650.ko 2023-10-26T01:16:14.896 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/max6697.ko 2023-10-26T01:16:14.897 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/mcp3021.ko 2023-10-26T01:16:14.897 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/nct6683.ko 2023-10-26T01:16:14.897 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/nct6775-core.ko 2023-10-26T01:16:14.897 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/nct6775.ko 2023-10-26T01:16:14.897 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/nct7802.ko 2023-10-26T01:16:14.898 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pc87360.ko 2023-10-26T01:16:14.898 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pc87427.ko 2023-10-26T01:16:14.898 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pcf8591.ko 2023-10-26T01:16:14.898 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus 2023-10-26T01:16:14.899 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-10-26T01:16:14.899 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-10-26T01:16:14.899 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-10-26T01:16:14.899 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/max16064.ko 2023-10-26T01:16:14.899 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/max34440.ko 2023-10-26T01:16:14.900 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/max8688.ko 2023-10-26T01:16:14.900 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-10-26T01:16:14.900 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-10-26T01:16:14.900 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-10-26T01:16:14.900 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-10-26T01:16:14.901 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-10-26T01:16:14.901 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-10-26T01:16:14.901 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/powr1220.ko 2023-10-26T01:16:14.901 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/sch5627.ko 2023-10-26T01:16:14.901 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/sch5636.ko 2023-10-26T01:16:14.902 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/sch56xx-common.ko 2023-10-26T01:16:14.902 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/sht15.ko 2023-10-26T01:16:14.902 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/sht21.ko 2023-10-26T01:16:14.902 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/shtc1.ko 2023-10-26T01:16:14.903 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/sis5595.ko 2023-10-26T01:16:14.903 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/smsc47b397.ko 2023-10-26T01:16:14.903 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/smsc47m1.ko 2023-10-26T01:16:14.903 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/smsc47m192.ko 2023-10-26T01:16:14.903 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/thmc50.ko 2023-10-26T01:16:14.904 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/tmp102.ko 2023-10-26T01:16:14.904 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/tmp103.ko 2023-10-26T01:16:14.904 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/tmp401.ko 2023-10-26T01:16:14.904 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/tmp421.ko 2023-10-26T01:16:14.904 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/via-cputemp.ko 2023-10-26T01:16:14.905 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/via686a.ko 2023-10-26T01:16:14.905 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/vt1211.ko 2023-10-26T01:16:14.905 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/vt8231.ko 2023-10-26T01:16:14.905 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83627ehf.ko 2023-10-26T01:16:14.905 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83627hf.ko 2023-10-26T01:16:14.906 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83781d.ko 2023-10-26T01:16:14.906 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83791d.ko 2023-10-26T01:16:14.906 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83792d.ko 2023-10-26T01:16:14.906 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83793.ko 2023-10-26T01:16:14.907 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83795.ko 2023-10-26T01:16:14.907 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83l785ts.ko 2023-10-26T01:16:14.907 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83l786ng.ko 2023-10-26T01:16:14.907 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c 2023-10-26T01:16:14.907 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/algos 2023-10-26T01:16:14.908 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-10-26T01:16:14.908 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-10-26T01:16:14.908 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses 2023-10-26T01:16:14.908 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-10-26T01:16:14.908 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-10-26T01:16:14.909 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-10-26T01:16:14.909 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-10-26T01:16:14.909 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-10-26T01:16:14.909 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-10-26T01:16:14.910 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-10-26T01:16:14.910 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-10-26T01:16:14.910 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-i801.ko 2023-10-26T01:16:14.910 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-isch.ko 2023-10-26T01:16:14.910 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-10-26T01:16:14.911 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-10-26T01:16:14.911 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-10-26T01:16:14.911 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-parport.ko 2023-10-26T01:16:14.911 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-10-26T01:16:14.912 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-10-26T01:16:14.912 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-10-26T01:16:14.912 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-10-26T01:16:14.912 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-10-26T01:16:14.912 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-10-26T01:16:14.913 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-via.ko 2023-10-26T01:16:14.913 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-10-26T01:16:14.913 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-10-26T01:16:14.913 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/i2c-dev.ko 2023-10-26T01:16:14.913 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/i2c-mux.ko 2023-10-26T01:16:14.914 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-10-26T01:16:14.914 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/i2c-smbus.ko 2023-10-26T01:16:14.914 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/i2c-stub.ko 2023-10-26T01:16:14.914 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband 2023-10-26T01:16:14.914 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/core 2023-10-26T01:16:14.915 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/core/ib_cm.ko 2023-10-26T01:16:14.915 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/core/ib_core.ko 2023-10-26T01:16:14.915 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/core/ib_umad.ko 2023-10-26T01:16:14.915 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-10-26T01:16:14.916 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/core/iw_cm.ko 2023-10-26T01:16:14.916 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/core/rdma_cm.ko 2023-10-26T01:16:14.916 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-10-26T01:16:14.916 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw 2023-10-26T01:16:14.916 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw/cxgb4 2023-10-26T01:16:14.917 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-10-26T01:16:14.917 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw/mlx4 2023-10-26T01:16:14.917 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-10-26T01:16:14.917 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw/mlx5 2023-10-26T01:16:14.917 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-10-26T01:16:14.918 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw/mthca 2023-10-26T01:16:14.918 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-10-26T01:16:14.918 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp 2023-10-26T01:16:14.918 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/ipoib 2023-10-26T01:16:14.919 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-10-26T01:16:14.919 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/iser 2023-10-26T01:16:14.919 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-10-26T01:16:14.919 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/isert 2023-10-26T01:16:14.919 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-10-26T01:16:14.920 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/srp 2023-10-26T01:16:14.920 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-10-26T01:16:14.920 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/srpt 2023-10-26T01:16:14.920 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-10-26T01:16:14.920 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input 2023-10-26T01:16:14.921 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/gameport 2023-10-26T01:16:14.921 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-10-26T01:16:14.921 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/gameport/fm801-gp.ko 2023-10-26T01:16:14.921 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/gameport/gameport.ko 2023-10-26T01:16:14.922 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/joydev.ko 2023-10-26T01:16:14.922 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/keyboard 2023-10-26T01:16:14.922 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/keyboard/gpio_keys.ko 2023-10-26T01:16:14.922 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse 2023-10-26T01:16:14.922 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse/appletouch.ko 2023-10-26T01:16:14.923 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse/bcm5974.ko 2023-10-26T01:16:14.923 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse/cyapatp.ko 2023-10-26T01:16:14.923 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse/elan_i2c.ko 2023-10-26T01:16:14.923 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse/sermouse.ko 2023-10-26T01:16:14.924 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-10-26T01:16:14.924 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse/synaptics_usb.ko 2023-10-26T01:16:14.924 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse/vsxxxaa.ko 2023-10-26T01:16:14.924 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/rmi4 2023-10-26T01:16:14.924 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/rmi4/rmi_core.ko 2023-10-26T01:16:14.925 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/serio 2023-10-26T01:16:14.925 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/serio/altera_ps2.ko 2023-10-26T01:16:14.925 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/serio/arc_ps2.ko 2023-10-26T01:16:14.925 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-10-26T01:16:14.926 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/serio/serio_raw.ko 2023-10-26T01:16:14.926 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/sparse-keymap.ko 2023-10-26T01:16:14.926 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/iommu 2023-10-26T01:16:14.926 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/iommu/amd 2023-10-26T01:16:14.927 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/iommu/amd/iommu_v2.ko 2023-10-26T01:16:14.927 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds 2023-10-26T01:16:14.927 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/leds-blinkm.ko 2023-10-26T01:16:14.927 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/leds-lm3530.ko 2023-10-26T01:16:14.927 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/leds-lp3944.ko 2023-10-26T01:16:14.928 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/leds-ss4200.ko 2023-10-26T01:16:14.928 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger 2023-10-26T01:16:14.928 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-10-26T01:16:14.928 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-10-26T01:16:14.928 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-10-26T01:16:14.929 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-10-26T01:16:14.929 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-10-26T01:16:14.929 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-10-26T01:16:14.929 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-10-26T01:16:14.930 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-10-26T01:16:14.930 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md 2023-10-26T01:16:14.930 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/bcache 2023-10-26T01:16:14.930 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/bcache/bcache.ko 2023-10-26T01:16:14.930 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-bio-prison.ko 2023-10-26T01:16:14.931 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-cache-smq.ko 2023-10-26T01:16:14.931 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-cache.ko 2023-10-26T01:16:14.931 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-crypt.ko 2023-10-26T01:16:14.931 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-delay.ko 2023-10-26T01:16:14.931 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-flakey.ko 2023-10-26T01:16:14.932 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-log-userspace.ko 2023-10-26T01:16:14.932 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-multipath.ko 2023-10-26T01:16:14.932 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-queue-length.ko 2023-10-26T01:16:14.932 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-raid.ko 2023-10-26T01:16:14.933 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-round-robin.ko 2023-10-26T01:16:14.933 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-service-time.ko 2023-10-26T01:16:14.933 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-switch.ko 2023-10-26T01:16:14.933 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-thin-pool.ko 2023-10-26T01:16:14.933 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-verity.ko 2023-10-26T01:16:14.934 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/faulty.ko 2023-10-26T01:16:14.934 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/linear.ko 2023-10-26T01:16:14.934 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/multipath.ko 2023-10-26T01:16:14.934 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/persistent-data 2023-10-26T01:16:14.935 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-10-26T01:16:14.935 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/raid0.ko 2023-10-26T01:16:14.935 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/raid1.ko 2023-10-26T01:16:14.935 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/raid10.ko 2023-10-26T01:16:14.935 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/raid456.ko 2023-10-26T01:16:14.936 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message 2023-10-26T01:16:14.936 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message/fusion 2023-10-26T01:16:14.936 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message/fusion/mptbase.ko 2023-10-26T01:16:14.936 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message/fusion/mptctl.ko 2023-10-26T01:16:14.937 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message/fusion/mptfc.ko 2023-10-26T01:16:14.937 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message/fusion/mptlan.ko 2023-10-26T01:16:14.937 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message/fusion/mptsas.ko 2023-10-26T01:16:14.937 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message/fusion/mptscsih.ko 2023-10-26T01:16:14.937 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message/fusion/mptspi.ko 2023-10-26T01:16:14.938 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/mfd 2023-10-26T01:16:14.938 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/mfd/lpc_ich.ko 2023-10-26T01:16:14.938 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/mfd/lpc_sch.ko 2023-10-26T01:16:14.938 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/mfd/mfd-core.ko 2023-10-26T01:16:14.938 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/mfd/sm501.ko 2023-10-26T01:16:14.939 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/mfd/viperboard.ko 2023-10-26T01:16:14.939 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/mfd/vx855.ko 2023-10-26T01:16:14.939 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/mfd/wl1273-core.ko 2023-10-26T01:16:14.939 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc 2023-10-26T01:16:14.940 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/altera-stapl 2023-10-26T01:16:14.940 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-10-26T01:16:14.940 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/apds9802als.ko 2023-10-26T01:16:14.940 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/apds990x.ko 2023-10-26T01:16:14.941 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/bh1770glc.ko 2023-10-26T01:16:14.941 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/cb710 2023-10-26T01:16:14.941 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/cb710/cb710.ko 2023-10-26T01:16:14.941 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/eeprom 2023-10-26T01:16:14.941 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/eeprom/at24.ko 2023-10-26T01:16:14.941 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/eeprom/eeprom.ko 2023-10-26T01:16:14.942 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-10-26T01:16:14.942 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/eeprom/max6875.ko 2023-10-26T01:16:14.942 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/enclosure.ko 2023-10-26T01:16:14.942 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/hpilo.ko 2023-10-26T01:16:14.943 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/isl29003.ko 2023-10-26T01:16:14.943 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/isl29020.ko 2023-10-26T01:16:14.943 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/lis3lv02d 2023-10-26T01:16:14.943 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-10-26T01:16:14.943 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-10-26T01:16:14.944 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/mei 2023-10-26T01:16:14.944 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/mei/mei-me.ko 2023-10-26T01:16:14.944 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/mei/mei-txe.ko 2023-10-26T01:16:14.944 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/mei/mei.ko 2023-10-26T01:16:14.944 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/sgi-gru 2023-10-26T01:16:14.945 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/sgi-gru/gru.ko 2023-10-26T01:16:14.945 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/sgi-xp 2023-10-26T01:16:14.945 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/sgi-xp/xp.ko 2023-10-26T01:16:14.945 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/sgi-xp/xpc.ko 2023-10-26T01:16:14.945 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-10-26T01:16:14.946 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/tifm_7xx1.ko 2023-10-26T01:16:14.946 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/tifm_core.ko 2023-10-26T01:16:14.946 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/tsl2550.ko 2023-10-26T01:16:14.946 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/vmw_balloon.ko 2023-10-26T01:16:14.946 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/vmw_vmci 2023-10-26T01:16:14.947 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-10-26T01:16:14.947 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net 2023-10-26T01:16:14.947 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/bonding 2023-10-26T01:16:14.947 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/bonding/bonding.ko 2023-10-26T01:16:14.948 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/dsa 2023-10-26T01:16:14.948 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/dsa/b53 2023-10-26T01:16:14.948 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/dsa/b53/b53_common.ko 2023-10-26T01:16:14.948 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/dsa/bcm-sf2.ko 2023-10-26T01:16:14.948 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/dsa/mv88e6060.ko 2023-10-26T01:16:14.949 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/dsa/mv88e6xxx 2023-10-26T01:16:14.949 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-10-26T01:16:14.949 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/dummy.ko 2023-10-26T01:16:14.949 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/eql.ko 2023-10-26T01:16:14.949 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet 2023-10-26T01:16:14.950 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/3com 2023-10-26T01:16:14.950 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-10-26T01:16:14.950 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-10-26T01:16:14.950 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-10-26T01:16:14.950 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-10-26T01:16:14.951 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/8390 2023-10-26T01:16:14.951 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/8390/8390.ko 2023-10-26T01:16:14.951 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-10-26T01:16:14.951 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-10-26T01:16:14.952 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-10-26T01:16:14.952 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/adaptec 2023-10-26T01:16:14.952 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-10-26T01:16:14.952 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/agere 2023-10-26T01:16:14.952 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/agere/et131x.ko 2023-10-26T01:16:14.953 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/alteon 2023-10-26T01:16:14.953 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-10-26T01:16:14.953 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/altera 2023-10-26T01:16:14.953 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-10-26T01:16:14.953 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/amd 2023-10-26T01:16:14.954 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-10-26T01:16:14.954 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-10-26T01:16:14.954 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-10-26T01:16:14.954 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros 2023-10-26T01:16:14.955 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/alx 2023-10-26T01:16:14.955 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-10-26T01:16:14.955 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/atl1c 2023-10-26T01:16:14.955 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-10-26T01:16:14.955 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/atl1e 2023-10-26T01:16:14.956 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-10-26T01:16:14.956 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/atlx 2023-10-26T01:16:14.956 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-10-26T01:16:14.957 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-10-26T01:16:14.957 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom 2023-10-26T01:16:14.958 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-10-26T01:16:14.958 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-10-26T01:16:14.958 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-10-26T01:16:14.958 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-10-26T01:16:14.959 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-10-26T01:16:14.959 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom/genet 2023-10-26T01:16:14.959 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-10-26T01:16:14.959 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-10-26T01:16:14.959 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/brocade 2023-10-26T01:16:14.960 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/brocade/bna 2023-10-26T01:16:14.960 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-10-26T01:16:14.960 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio 2023-10-26T01:16:14.960 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/cxgb 2023-10-26T01:16:14.960 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-10-26T01:16:14.961 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-10-26T01:16:14.961 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-10-26T01:16:14.961 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-10-26T01:16:14.961 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-10-26T01:16:14.962 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-10-26T01:16:14.962 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-10-26T01:16:14.962 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-10-26T01:16:14.962 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-10-26T01:16:14.962 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/cisco 2023-10-26T01:16:14.963 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/cisco/enic 2023-10-26T01:16:14.963 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-10-26T01:16:14.963 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dec 2023-10-26T01:16:14.963 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dec/tulip 2023-10-26T01:16:14.963 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-10-26T01:16:14.964 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-10-26T01:16:14.964 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-10-26T01:16:14.964 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-10-26T01:16:14.964 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-10-26T01:16:14.965 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-10-26T01:16:14.965 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dlink 2023-10-26T01:16:14.965 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-10-26T01:16:14.965 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-10-26T01:16:14.965 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dnet.ko 2023-10-26T01:16:14.966 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/emulex 2023-10-26T01:16:14.966 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/emulex/benet 2023-10-26T01:16:14.966 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-10-26T01:16:14.966 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/ethoc.ko 2023-10-26T01:16:14.966 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/fealnx.ko 2023-10-26T01:16:14.967 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel 2023-10-26T01:16:14.967 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/e100.ko 2023-10-26T01:16:14.967 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/e1000 2023-10-26T01:16:14.967 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-10-26T01:16:14.967 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/e1000e 2023-10-26T01:16:14.968 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-10-26T01:16:14.968 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/fm10k 2023-10-26T01:16:14.968 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-10-26T01:16:14.968 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/i40e 2023-10-26T01:16:14.969 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-10-26T01:16:14.969 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/iavf 2023-10-26T01:16:14.969 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-10-26T01:16:14.969 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/igb 2023-10-26T01:16:14.969 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-10-26T01:16:14.970 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/igbvf 2023-10-26T01:16:14.970 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-10-26T01:16:14.970 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/ixgbe 2023-10-26T01:16:14.970 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-10-26T01:16:14.970 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/ixgbevf 2023-10-26T01:16:14.971 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-10-26T01:16:14.971 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/jme.ko 2023-10-26T01:16:14.971 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/marvell 2023-10-26T01:16:14.971 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-10-26T01:16:14.972 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/marvell/skge.ko 2023-10-26T01:16:14.972 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-10-26T01:16:14.972 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/mellanox 2023-10-26T01:16:14.972 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/mellanox/mlx4 2023-10-26T01:16:14.972 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-10-26T01:16:14.973 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-10-26T01:16:14.973 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/mellanox/mlx5 2023-10-26T01:16:14.973 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-10-26T01:16:14.973 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-10-26T01:16:14.973 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/micrel 2023-10-26T01:16:14.974 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-10-26T01:16:14.974 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/myricom 2023-10-26T01:16:14.974 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/myricom/myri10ge 2023-10-26T01:16:14.974 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-10-26T01:16:14.975 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/natsemi 2023-10-26T01:16:14.975 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-10-26T01:16:14.975 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-10-26T01:16:14.975 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/neterion 2023-10-26T01:16:14.975 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-10-26T01:16:14.976 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/nvidia 2023-10-26T01:16:14.976 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-10-26T01:16:14.976 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/packetengines 2023-10-26T01:16:14.976 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-10-26T01:16:14.976 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-10-26T01:16:14.977 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/qlogic 2023-10-26T01:16:14.977 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/qlogic/netxen 2023-10-26T01:16:14.977 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-10-26T01:16:14.977 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-10-26T01:16:14.978 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-10-26T01:16:14.978 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-10-26T01:16:14.978 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/rdc 2023-10-26T01:16:14.978 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-10-26T01:16:14.978 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/realtek 2023-10-26T01:16:14.979 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-10-26T01:16:14.979 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-10-26T01:16:14.979 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/realtek/atp.ko 2023-10-26T01:16:14.979 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-10-26T01:16:14.979 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/rocker 2023-10-26T01:16:14.980 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-10-26T01:16:14.980 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sfc 2023-10-26T01:16:14.980 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-10-26T01:16:14.980 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/silan 2023-10-26T01:16:14.981 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-10-26T01:16:14.981 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sis 2023-10-26T01:16:14.981 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sis/sis190.ko 2023-10-26T01:16:14.981 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sis/sis900.ko 2023-10-26T01:16:14.981 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/smsc 2023-10-26T01:16:14.982 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-10-26T01:16:14.982 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-10-26T01:16:14.982 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-10-26T01:16:14.982 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-10-26T01:16:14.983 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/stmicro 2023-10-26T01:16:14.983 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/stmicro/stmmac 2023-10-26T01:16:14.983 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-10-26T01:16:14.983 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-10-26T01:16:14.983 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sun 2023-10-26T01:16:14.984 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sun/cassini.ko 2023-10-26T01:16:14.984 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sun/niu.ko 2023-10-26T01:16:14.984 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sun/sungem.ko 2023-10-26T01:16:14.984 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-10-26T01:16:14.984 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/tehuti 2023-10-26T01:16:14.985 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-10-26T01:16:14.985 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/ti 2023-10-26T01:16:14.985 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/ti/tlan.ko 2023-10-26T01:16:14.985 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/via 2023-10-26T01:16:14.986 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-10-26T01:16:14.986 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-10-26T01:16:14.986 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/wiznet 2023-10-26T01:16:14.986 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-10-26T01:16:14.986 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-10-26T01:16:14.987 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/xircom 2023-10-26T01:16:14.987 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-10-26T01:16:14.987 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/geneve.ko 2023-10-26T01:16:14.987 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/hyperv 2023-10-26T01:16:14.987 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-10-26T01:16:14.988 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ieee802154 2023-10-26T01:16:14.988 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ieee802154/fakelb.ko 2023-10-26T01:16:14.988 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ifb.ko 2023-10-26T01:16:14.988 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ipvlan 2023-10-26T01:16:14.988 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ipvlan/ipvlan.ko 2023-10-26T01:16:14.989 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/macvlan.ko 2023-10-26T01:16:14.989 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/macvtap.ko 2023-10-26T01:16:14.989 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/mdio 2023-10-26T01:16:14.989 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/mdio.ko 2023-10-26T01:16:14.989 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-10-26T01:16:14.990 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-10-26T01:16:14.990 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/mdio/mdio-regmap.ko 2023-10-26T01:16:14.990 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/mii.ko 2023-10-26T01:16:14.990 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/net_failover.ko 2023-10-26T01:16:14.991 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/netconsole.ko 2023-10-26T01:16:14.991 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/nlmon.ko 2023-10-26T01:16:14.991 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/pcs 2023-10-26T01:16:14.991 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/pcs/pcs-lynx.ko 2023-10-26T01:16:14.991 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-10-26T01:16:14.992 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy 2023-10-26T01:16:14.992 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/amd.ko 2023-10-26T01:16:14.992 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-10-26T01:16:14.992 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-10-26T01:16:14.992 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/bcm7xxx.ko 2023-10-26T01:16:14.993 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/bcm87xx.ko 2023-10-26T01:16:14.993 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/broadcom.ko 2023-10-26T01:16:14.993 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/cicada.ko 2023-10-26T01:16:14.993 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/davicom.ko 2023-10-26T01:16:14.993 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/dp83640.ko 2023-10-26T01:16:14.994 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/et1011c.ko 2023-10-26T01:16:14.994 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/icplus.ko 2023-10-26T01:16:14.994 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/lxt.ko 2023-10-26T01:16:14.994 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/marvell.ko 2023-10-26T01:16:14.995 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/micrel.ko 2023-10-26T01:16:14.995 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/national.ko 2023-10-26T01:16:14.995 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/phylink.ko 2023-10-26T01:16:14.995 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/qsemi.ko 2023-10-26T01:16:14.995 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/realtek.ko 2023-10-26T01:16:14.996 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/smsc.ko 2023-10-26T01:16:14.996 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/ste10Xp.ko 2023-10-26T01:16:14.996 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/vitesse.ko 2023-10-26T01:16:14.996 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp 2023-10-26T01:16:14.996 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/bsd_comp.ko 2023-10-26T01:16:14.997 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/ppp_async.ko 2023-10-26T01:16:14.997 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/ppp_deflate.ko 2023-10-26T01:16:14.997 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/ppp_generic.ko 2023-10-26T01:16:14.997 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/ppp_mppe.ko 2023-10-26T01:16:14.997 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/ppp_synctty.ko 2023-10-26T01:16:14.998 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/pppoe.ko 2023-10-26T01:16:14.998 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/pppox.ko 2023-10-26T01:16:14.998 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/pptp.ko 2023-10-26T01:16:14.998 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/slip 2023-10-26T01:16:14.999 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/slip/slhc.ko 2023-10-26T01:16:14.999 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/slip/slip.ko 2023-10-26T01:16:14.999 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/sungem_phy.ko 2023-10-26T01:16:14.999 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/tap.ko 2023-10-26T01:16:14.999 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/team 2023-10-26T01:16:15.000 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/team/team.ko 2023-10-26T01:16:15.000 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/team/team_mode_activebackup.ko 2023-10-26T01:16:15.000 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/team/team_mode_broadcast.ko 2023-10-26T01:16:15.000 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-10-26T01:16:15.000 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/team/team_mode_random.ko 2023-10-26T01:16:15.001 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-10-26T01:16:15.001 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/tun.ko 2023-10-26T01:16:15.001 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/veth.ko 2023-10-26T01:16:15.001 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/virtio_net.ko 2023-10-26T01:16:15.001 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/vmxnet3 2023-10-26T01:16:15.002 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-10-26T01:16:15.002 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/vxlan 2023-10-26T01:16:15.002 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/vxlan/vxlan.ko 2023-10-26T01:16:15.002 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/xen-netback 2023-10-26T01:16:15.002 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/xen-netback/xen-netback.ko 2023-10-26T01:16:15.003 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/xen-netfront.ko 2023-10-26T01:16:15.003 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ntb 2023-10-26T01:16:15.003 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ntb/ntb.ko 2023-10-26T01:16:15.003 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/nvme 2023-10-26T01:16:15.003 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/nvme/host 2023-10-26T01:16:15.004 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/nvme/host/nvme-core.ko 2023-10-26T01:16:15.004 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/nvme/host/nvme.ko 2023-10-26T01:16:15.004 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/parport 2023-10-26T01:16:15.004 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/parport/parport.ko 2023-10-26T01:16:15.005 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/parport/parport_cs.ko 2023-10-26T01:16:15.005 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/parport/parport_pc.ko 2023-10-26T01:16:15.005 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/parport/parport_serial.ko 2023-10-26T01:16:15.005 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pci 2023-10-26T01:16:15.005 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pci/hotplug 2023-10-26T01:16:15.006 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-10-26T01:16:15.006 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pci/pcie 2023-10-26T01:16:15.006 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pci/pcie/aer_inject.ko 2023-10-26T01:16:15.006 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pci/xen-pcifront.ko 2023-10-26T01:16:15.007 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pcmcia 2023-10-26T01:16:15.007 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pcmcia/i82092.ko 2023-10-26T01:16:15.007 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pcmcia/pd6729.ko 2023-10-26T01:16:15.007 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pcmcia/yenta_socket.ko 2023-10-26T01:16:15.007 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pinctrl 2023-10-26T01:16:15.008 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pinctrl/intel 2023-10-26T01:16:15.008 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-10-26T01:16:15.008 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform 2023-10-26T01:16:15.008 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86 2023-10-26T01:16:15.009 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/acer-wmi.ko 2023-10-26T01:16:15.009 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/acerhdf.ko 2023-10-26T01:16:15.009 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/apple-gmux.ko 2023-10-26T01:16:15.009 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/asus-laptop.ko 2023-10-26T01:16:15.009 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-10-26T01:16:15.010 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/asus-wmi.ko 2023-10-26T01:16:15.010 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/classmate-laptop.ko 2023-10-26T01:16:15.010 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-10-26T01:16:15.010 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-10-26T01:16:15.010 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-10-26T01:16:15.011 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-10-26T01:16:15.011 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/hdaps.ko 2023-10-26T01:16:15.011 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/intel 2023-10-26T01:16:15.011 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-10-26T01:16:15.012 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/intel_ips.ko 2023-10-26T01:16:15.012 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/msi-wmi.ko 2023-10-26T01:16:15.012 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/mxm-wmi.ko 2023-10-26T01:16:15.012 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-10-26T01:16:15.012 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/samsung-laptop.ko 2023-10-26T01:16:15.013 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/samsung-q10.ko 2023-10-26T01:16:15.013 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/topstar-laptop.ko 2023-10-26T01:16:15.013 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-10-26T01:16:15.013 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/toshiba_haps.ko 2023-10-26T01:16:15.014 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/wmi-bmof.ko 2023-10-26T01:16:15.014 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/wmi.ko 2023-10-26T01:16:15.014 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/powercap 2023-10-26T01:16:15.014 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/powercap/intel_rapl_common.ko 2023-10-26T01:16:15.014 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/powercap/intel_rapl_msr.ko 2023-10-26T01:16:15.015 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pps 2023-10-26T01:16:15.015 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pps/clients 2023-10-26T01:16:15.015 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pps/clients/pps-gpio.ko 2023-10-26T01:16:15.015 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pps/clients/pps-ldisc.ko 2023-10-26T01:16:15.015 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pps/clients/pps_parport.ko 2023-10-26T01:16:15.016 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pps/pps_core.ko 2023-10-26T01:16:15.016 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ptp 2023-10-26T01:16:15.016 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ptp/ptp.ko 2023-10-26T01:16:15.016 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ptp/ptp_kvm.ko 2023-10-26T01:16:15.016 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc 2023-10-26T01:16:15.017 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-bq32k.ko 2023-10-26T01:16:15.017 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds1286.ko 2023-10-26T01:16:15.017 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds1307.ko 2023-10-26T01:16:15.017 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds1374.ko 2023-10-26T01:16:15.017 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds1511.ko 2023-10-26T01:16:15.018 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds1553.ko 2023-10-26T01:16:15.018 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds1672.ko 2023-10-26T01:16:15.018 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds1742.ko 2023-10-26T01:16:15.018 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds2404.ko 2023-10-26T01:16:15.019 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds3232.ko 2023-10-26T01:16:15.019 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-em3027.ko 2023-10-26T01:16:15.019 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-fm3130.ko 2023-10-26T01:16:15.019 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-isl12022.ko 2023-10-26T01:16:15.019 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-isl1208.ko 2023-10-26T01:16:15.020 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-m41t80.ko 2023-10-26T01:16:15.020 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-m48t35.ko 2023-10-26T01:16:15.020 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-m48t59.ko 2023-10-26T01:16:15.020 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-max6900.ko 2023-10-26T01:16:15.020 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-msm6242.ko 2023-10-26T01:16:15.021 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-pcf2127.ko 2023-10-26T01:16:15.021 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-pcf85063.ko 2023-10-26T01:16:15.021 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-pcf8523.ko 2023-10-26T01:16:15.021 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-pcf8563.ko 2023-10-26T01:16:15.021 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-pcf8583.ko 2023-10-26T01:16:15.022 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-rp5c01.ko 2023-10-26T01:16:15.022 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-rs5c372.ko 2023-10-26T01:16:15.022 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-10-26T01:16:15.022 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-rx8025.ko 2023-10-26T01:16:15.023 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-rx8581.ko 2023-10-26T01:16:15.023 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-10-26T01:16:15.023 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-x1205.ko 2023-10-26T01:16:15.023 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi 2023-10-26T01:16:15.023 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/3w-9xxx.ko 2023-10-26T01:16:15.024 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/3w-sas.ko 2023-10-26T01:16:15.024 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/3w-xxxx.ko 2023-10-26T01:16:15.024 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/BusLogic.ko 2023-10-26T01:16:15.024 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/a100u2w.ko 2023-10-26T01:16:15.024 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/aacraid 2023-10-26T01:16:15.025 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/aacraid/aacraid.ko 2023-10-26T01:16:15.025 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/advansys.ko 2023-10-26T01:16:15.025 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/aic7xxx 2023-10-26T01:16:15.025 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-10-26T01:16:15.025 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-10-26T01:16:15.026 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/am53c974.ko 2023-10-26T01:16:15.026 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/arcmsr 2023-10-26T01:16:15.026 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-10-26T01:16:15.026 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/atp870u.ko 2023-10-26T01:16:15.027 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/be2iscsi 2023-10-26T01:16:15.027 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-10-26T01:16:15.027 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/bfa 2023-10-26T01:16:15.027 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/bfa/bfa.ko 2023-10-26T01:16:15.027 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/bnx2fc 2023-10-26T01:16:15.028 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-10-26T01:16:15.028 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/bnx2i 2023-10-26T01:16:15.028 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-10-26T01:16:15.028 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/ch.ko 2023-10-26T01:16:15.028 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/csiostor 2023-10-26T01:16:15.029 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/csiostor/csiostor.ko 2023-10-26T01:16:15.029 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/cxgbi 2023-10-26T01:16:15.029 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/cxgbi/cxgb3i 2023-10-26T01:16:15.029 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-10-26T01:16:15.029 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/cxgbi/cxgb4i 2023-10-26T01:16:15.030 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-10-26T01:16:15.030 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-10-26T01:16:15.030 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/dc395x.ko 2023-10-26T01:16:15.030 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/device_handler 2023-10-26T01:16:15.031 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-10-26T01:16:15.031 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-10-26T01:16:15.031 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-10-26T01:16:15.031 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-10-26T01:16:15.031 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/dmx3191d.ko 2023-10-26T01:16:15.032 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/esas2r 2023-10-26T01:16:15.032 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/esas2r/esas2r.ko 2023-10-26T01:16:15.032 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/esp_scsi.ko 2023-10-26T01:16:15.032 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/fcoe 2023-10-26T01:16:15.032 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/fcoe/fcoe.ko 2023-10-26T01:16:15.033 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-10-26T01:16:15.033 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/fnic 2023-10-26T01:16:15.033 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/fnic/fnic.ko 2023-10-26T01:16:15.033 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/hpsa.ko 2023-10-26T01:16:15.033 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/hptiop.ko 2023-10-26T01:16:15.034 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/hv_storvsc.ko 2023-10-26T01:16:15.034 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/initio.ko 2023-10-26T01:16:15.034 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/ipr.ko 2023-10-26T01:16:15.034 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/ips.ko 2023-10-26T01:16:15.034 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/isci 2023-10-26T01:16:15.035 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/isci/isci.ko 2023-10-26T01:16:15.037 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-10-26T01:16:15.037 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/iscsi_tcp.ko 2023-10-26T01:16:15.037 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/libfc 2023-10-26T01:16:15.038 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/libfc/libfc.ko 2023-10-26T01:16:15.038 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/libiscsi.ko 2023-10-26T01:16:15.038 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/libiscsi_tcp.ko 2023-10-26T01:16:15.038 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/libsas 2023-10-26T01:16:15.038 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/libsas/libsas.ko 2023-10-26T01:16:15.039 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/lpfc 2023-10-26T01:16:15.039 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/lpfc/lpfc.ko 2023-10-26T01:16:15.039 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/megaraid 2023-10-26T01:16:15.039 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/megaraid.ko 2023-10-26T01:16:15.039 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-10-26T01:16:15.040 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-10-26T01:16:15.040 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-10-26T01:16:15.040 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/mpt3sas 2023-10-26T01:16:15.040 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-10-26T01:16:15.040 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/mvsas 2023-10-26T01:16:15.041 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/mvsas/mvsas.ko 2023-10-26T01:16:15.041 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/mvumi.ko 2023-10-26T01:16:15.041 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/pm8001 2023-10-26T01:16:15.041 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-10-26T01:16:15.041 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/pmcraid.ko 2023-10-26T01:16:15.042 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/qla1280.ko 2023-10-26T01:16:15.042 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/qla2xxx 2023-10-26T01:16:15.042 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-10-26T01:16:15.042 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-10-26T01:16:15.042 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/qla4xxx 2023-10-26T01:16:15.043 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-10-26T01:16:15.043 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/raid_class.ko 2023-10-26T01:16:15.043 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/scsi_debug.ko 2023-10-26T01:16:15.043 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/scsi_transport_fc.ko 2023-10-26T01:16:15.044 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-10-26T01:16:15.044 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/scsi_transport_sas.ko 2023-10-26T01:16:15.044 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/scsi_transport_spi.ko 2023-10-26T01:16:15.044 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/scsi_transport_srp.ko 2023-10-26T01:16:15.044 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/ses.ko 2023-10-26T01:16:15.045 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/st.ko 2023-10-26T01:16:15.045 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/stex.ko 2023-10-26T01:16:15.045 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/sym53c8xx_2 2023-10-26T01:16:15.045 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-10-26T01:16:15.045 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/virtio_scsi.ko 2023-10-26T01:16:15.046 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/vmw_pvscsi.ko 2023-10-26T01:16:15.046 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/wd719x.ko 2023-10-26T01:16:15.046 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ssb 2023-10-26T01:16:15.046 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ssb/ssb.ko 2023-10-26T01:16:15.046 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target 2023-10-26T01:16:15.047 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/iscsi 2023-10-26T01:16:15.047 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-10-26T01:16:15.047 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/loopback 2023-10-26T01:16:15.047 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/loopback/tcm_loop.ko 2023-10-26T01:16:15.048 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/target_core_file.ko 2023-10-26T01:16:15.048 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/target_core_iblock.ko 2023-10-26T01:16:15.048 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/target_core_mod.ko 2023-10-26T01:16:15.048 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/target_core_pscsi.ko 2023-10-26T01:16:15.048 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/target_core_user.ko 2023-10-26T01:16:15.049 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/tcm_fc 2023-10-26T01:16:15.049 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-10-26T01:16:15.049 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal 2023-10-26T01:16:15.049 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel 2023-10-26T01:16:15.049 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal 2023-10-26T01:16:15.050 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-10-26T01:16:15.050 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-10-26T01:16:15.050 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-10-26T01:16:15.050 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-10-26T01:16:15.050 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-10-26T01:16:15.051 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-10-26T01:16:15.051 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-10-26T01:16:15.051 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-10-26T01:16:15.052 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-10-26T01:16:15.052 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-10-26T01:16:15.052 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-10-26T01:16:15.052 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-10-26T01:16:15.052 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-10-26T01:16:15.053 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-10-26T01:16:15.053 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-10-26T01:16:15.053 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty 2023-10-26T01:16:15.053 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/ipwireless 2023-10-26T01:16:15.054 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-10-26T01:16:15.054 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/n_gsm.ko 2023-10-26T01:16:15.054 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/n_hdlc.ko 2023-10-26T01:16:15.054 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/nozomi.ko 2023-10-26T01:16:15.054 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/serial 2023-10-26T01:16:15.055 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/serial/8250 2023-10-26T01:16:15.055 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-10-26T01:16:15.055 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/serial/arc_uart.ko 2023-10-26T01:16:15.055 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/serial/jsm 2023-10-26T01:16:15.055 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/serial/jsm/jsm.ko 2023-10-26T01:16:15.056 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ufs 2023-10-26T01:16:15.056 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ufs/core 2023-10-26T01:16:15.056 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ufs/core/ufshcd-core.ko 2023-10-26T01:16:15.056 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ufs/host 2023-10-26T01:16:15.056 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-10-26T01:16:15.057 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/uio 2023-10-26T01:16:15.057 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/uio/uio.ko 2023-10-26T01:16:15.057 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/uio/uio_aec.ko 2023-10-26T01:16:15.057 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/uio/uio_cif.ko 2023-10-26T01:16:15.058 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/uio/uio_pci_generic.ko 2023-10-26T01:16:15.058 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/uio/uio_sercos3.ko 2023-10-26T01:16:15.058 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb 2023-10-26T01:16:15.058 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/class 2023-10-26T01:16:15.058 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/class/cdc-acm.ko 2023-10-26T01:16:15.059 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/class/cdc-wdm.ko 2023-10-26T01:16:15.059 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/class/usblp.ko 2023-10-26T01:16:15.059 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/class/usbtmc.ko 2023-10-26T01:16:15.059 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/host 2023-10-26T01:16:15.059 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/host/sl811-hcd.ko 2023-10-26T01:16:15.060 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/image 2023-10-26T01:16:15.060 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/image/mdc800.ko 2023-10-26T01:16:15.060 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/image/microtek.ko 2023-10-26T01:16:15.060 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc 2023-10-26T01:16:15.060 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/adutux.ko 2023-10-26T01:16:15.061 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/appledisplay.ko 2023-10-26T01:16:15.061 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/emi26.ko 2023-10-26T01:16:15.061 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/emi62.ko 2023-10-26T01:16:15.061 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/ezusb.ko 2023-10-26T01:16:15.061 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/idmouse.ko 2023-10-26T01:16:15.062 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/iowarrior.ko 2023-10-26T01:16:15.062 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/isight_firmware.ko 2023-10-26T01:16:15.062 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/ldusb.ko 2023-10-26T01:16:15.062 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/legousbtower.ko 2023-10-26T01:16:15.063 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/sisusbvga 2023-10-26T01:16:15.063 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-10-26T01:16:15.063 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/trancevibrator.ko 2023-10-26T01:16:15.063 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/usb3503.ko 2023-10-26T01:16:15.063 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/usblcd.ko 2023-10-26T01:16:15.064 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/usbsevseg.ko 2023-10-26T01:16:15.064 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/uss720.ko 2023-10-26T01:16:15.064 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/yurex.ko 2023-10-26T01:16:15.064 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/phy 2023-10-26T01:16:15.064 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/phy/phy-generic.ko 2023-10-26T01:16:15.065 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial 2023-10-26T01:16:15.065 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/aircable.ko 2023-10-26T01:16:15.065 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/ark3116.ko 2023-10-26T01:16:15.065 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/belkin_sa.ko 2023-10-26T01:16:15.065 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/ch341.ko 2023-10-26T01:16:15.066 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/cp210x.ko 2023-10-26T01:16:15.066 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/cyberjack.ko 2023-10-26T01:16:15.066 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/cypress_m8.ko 2023-10-26T01:16:15.066 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/digi_acceleport.ko 2023-10-26T01:16:15.066 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/empeg.ko 2023-10-26T01:16:15.067 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/ftdi_sio.ko 2023-10-26T01:16:15.067 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/garmin_gps.ko 2023-10-26T01:16:15.067 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/io_edgeport.ko 2023-10-26T01:16:15.067 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/io_ti.ko 2023-10-26T01:16:15.067 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/ipaq.ko 2023-10-26T01:16:15.068 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/ipw.ko 2023-10-26T01:16:15.068 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/ir-usb.ko 2023-10-26T01:16:15.068 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-10-26T01:16:15.068 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/keyspan.ko 2023-10-26T01:16:15.068 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/keyspan_pda.ko 2023-10-26T01:16:15.069 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/kl5kusb105.ko 2023-10-26T01:16:15.069 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/kobil_sct.ko 2023-10-26T01:16:15.069 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/mct_u232.ko 2023-10-26T01:16:15.069 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/mos7720.ko 2023-10-26T01:16:15.069 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/mos7840.ko 2023-10-26T01:16:15.070 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/navman.ko 2023-10-26T01:16:15.070 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/omninet.ko 2023-10-26T01:16:15.070 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/opticon.ko 2023-10-26T01:16:15.070 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/option.ko 2023-10-26T01:16:15.070 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/oti6858.ko 2023-10-26T01:16:15.071 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/pl2303.ko 2023-10-26T01:16:15.071 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/qcaux.ko 2023-10-26T01:16:15.071 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/qcserial.ko 2023-10-26T01:16:15.071 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/quatech2.ko 2023-10-26T01:16:15.071 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/safe_serial.ko 2023-10-26T01:16:15.072 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/sierra.ko 2023-10-26T01:16:15.072 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/spcp8x5.ko 2023-10-26T01:16:15.072 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/ssu100.ko 2023-10-26T01:16:15.072 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/symbolserial.ko 2023-10-26T01:16:15.072 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-10-26T01:16:15.073 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-10-26T01:16:15.073 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/usb_debug.ko 2023-10-26T01:16:15.073 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/usb_wwan.ko 2023-10-26T01:16:15.073 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/visor.ko 2023-10-26T01:16:15.073 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/whiteheat.ko 2023-10-26T01:16:15.074 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/xsens_mt.ko 2023-10-26T01:16:15.074 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage 2023-10-26T01:16:15.074 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/uas.ko 2023-10-26T01:16:15.074 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-alauda.ko 2023-10-26T01:16:15.075 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-cypress.ko 2023-10-26T01:16:15.075 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-datafab.ko 2023-10-26T01:16:15.075 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-10-26T01:16:15.075 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-freecom.ko 2023-10-26T01:16:15.075 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-isd200.ko 2023-10-26T01:16:15.076 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-10-26T01:16:15.076 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-karma.ko 2023-10-26T01:16:15.076 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-onetouch.ko 2023-10-26T01:16:15.076 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-realtek.ko 2023-10-26T01:16:15.076 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-sddr09.ko 2023-10-26T01:16:15.077 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-sddr55.ko 2023-10-26T01:16:15.077 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-usbat.ko 2023-10-26T01:16:15.077 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/usb-storage.ko 2023-10-26T01:16:15.077 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/usbip 2023-10-26T01:16:15.077 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/usbip/usbip-core.ko 2023-10-26T01:16:15.078 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/usbip/usbip-host.ko 2023-10-26T01:16:15.078 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-10-26T01:16:15.078 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vfio 2023-10-26T01:16:15.078 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vfio/pci 2023-10-26T01:16:15.078 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-10-26T01:16:15.079 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vfio/pci/vfio-pci.ko 2023-10-26T01:16:15.079 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vfio/vfio.ko 2023-10-26T01:16:15.079 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-10-26T01:16:15.079 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vhost 2023-10-26T01:16:15.080 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vhost/vhost.ko 2023-10-26T01:16:15.080 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vhost/vhost_iotlb.ko 2023-10-26T01:16:15.080 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vhost/vhost_net.ko 2023-10-26T01:16:15.080 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vhost/vhost_scsi.ko 2023-10-26T01:16:15.080 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/video 2023-10-26T01:16:15.081 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/video/backlight 2023-10-26T01:16:15.081 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/video/backlight/apple_bl.ko 2023-10-26T01:16:15.081 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/video/backlight/lcd.ko 2023-10-26T01:16:15.081 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/video/backlight/platform_lcd.ko 2023-10-26T01:16:15.081 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/virtio 2023-10-26T01:16:15.082 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/virtio/virtio.ko 2023-10-26T01:16:15.082 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/virtio/virtio_balloon.ko 2023-10-26T01:16:15.082 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/virtio/virtio_mmio.ko 2023-10-26T01:16:15.082 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/virtio/virtio_pci.ko 2023-10-26T01:16:15.083 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-10-26T01:16:15.083 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-10-26T01:16:15.083 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/virtio/virtio_ring.ko 2023-10-26T01:16:15.083 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1 2023-10-26T01:16:15.084 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/masters 2023-10-26T01:16:15.084 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/masters/ds2482.ko 2023-10-26T01:16:15.084 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/masters/ds2490.ko 2023-10-26T01:16:15.084 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves 2023-10-26T01:16:15.084 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-10-26T01:16:15.085 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-10-26T01:16:15.085 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-10-26T01:16:15.085 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-10-26T01:16:15.085 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-10-26T01:16:15.085 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-10-26T01:16:15.086 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-10-26T01:16:15.086 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-10-26T01:16:15.086 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-10-26T01:16:15.086 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_smem.ko 2023-10-26T01:16:15.087 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_therm.ko 2023-10-26T01:16:15.087 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/wire.ko 2023-10-26T01:16:15.087 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog 2023-10-26T01:16:15.087 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/alim1535_wdt.ko 2023-10-26T01:16:15.087 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/alim7101_wdt.ko 2023-10-26T01:16:15.088 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/f71808e_wdt.ko 2023-10-26T01:16:15.088 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/hpwdt.ko 2023-10-26T01:16:15.088 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/i6300esb.ko 2023-10-26T01:16:15.088 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-10-26T01:16:15.088 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/iTCO_wdt.ko 2023-10-26T01:16:15.089 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/ib700wdt.ko 2023-10-26T01:16:15.089 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/ibmasr.ko 2023-10-26T01:16:15.089 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-10-26T01:16:15.089 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/it8712f_wdt.ko 2023-10-26T01:16:15.089 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/it87_wdt.ko 2023-10-26T01:16:15.090 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/machzwd.ko 2023-10-26T01:16:15.090 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/nv_tco.ko 2023-10-26T01:16:15.090 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/pcwd_pci.ko 2023-10-26T01:16:15.090 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/pcwd_usb.ko 2023-10-26T01:16:15.090 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-10-26T01:16:15.091 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/sch311x_wdt.ko 2023-10-26T01:16:15.091 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/softdog.ko 2023-10-26T01:16:15.091 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/sp5100_tco.ko 2023-10-26T01:16:15.091 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/via_wdt.ko 2023-10-26T01:16:15.092 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-10-26T01:16:15.092 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/w83877f_wdt.ko 2023-10-26T01:16:15.092 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/w83977f_wdt.ko 2023-10-26T01:16:15.092 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/wdt_pci.ko 2023-10-26T01:16:15.092 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/xen_wdt.ko 2023-10-26T01:16:15.093 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen 2023-10-26T01:16:15.093 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xen-acpi-processor.ko 2023-10-26T01:16:15.093 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xen-evtchn.ko 2023-10-26T01:16:15.093 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xen-gntalloc.ko 2023-10-26T01:16:15.093 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xen-gntdev.ko 2023-10-26T01:16:15.094 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xen-pciback 2023-10-26T01:16:15.094 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-10-26T01:16:15.094 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xen-privcmd.ko 2023-10-26T01:16:15.094 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xenfs 2023-10-26T01:16:15.094 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xenfs/xenfs.ko 2023-10-26T01:16:15.095 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs 2023-10-26T01:16:15.095 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/9p 2023-10-26T01:16:15.095 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/9p/9p.ko 2023-10-26T01:16:15.095 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/affs 2023-10-26T01:16:15.095 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/affs/affs.ko 2023-10-26T01:16:15.096 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/befs 2023-10-26T01:16:15.096 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/befs/befs.ko 2023-10-26T01:16:15.096 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/binfmt_misc.ko 2023-10-26T01:16:15.096 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/btrfs 2023-10-26T01:16:15.097 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/btrfs/btrfs.ko 2023-10-26T01:16:15.097 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/cachefiles 2023-10-26T01:16:15.097 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/cachefiles/cachefiles.ko 2023-10-26T01:16:15.097 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ceph 2023-10-26T01:16:15.097 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ceph/ceph.ko 2023-10-26T01:16:15.098 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/coda 2023-10-26T01:16:15.098 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/coda/coda.ko 2023-10-26T01:16:15.098 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/cramfs 2023-10-26T01:16:15.098 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/cramfs/cramfs.ko 2023-10-26T01:16:15.098 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/dlm 2023-10-26T01:16:15.099 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/dlm/dlm.ko 2023-10-26T01:16:15.099 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ecryptfs 2023-10-26T01:16:15.099 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ecryptfs/ecryptfs.ko 2023-10-26T01:16:15.099 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/f2fs 2023-10-26T01:16:15.099 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/f2fs/f2fs.ko 2023-10-26T01:16:15.100 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fat 2023-10-26T01:16:15.100 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fat/fat.ko 2023-10-26T01:16:15.100 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fat/msdos.ko 2023-10-26T01:16:15.100 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fat/vfat.ko 2023-10-26T01:16:15.100 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fscache 2023-10-26T01:16:15.101 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fscache/fscache.ko 2023-10-26T01:16:15.101 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fuse 2023-10-26T01:16:15.101 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fuse/cuse.ko 2023-10-26T01:16:15.101 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fuse/fuse.ko 2023-10-26T01:16:15.102 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/gfs2 2023-10-26T01:16:15.102 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/gfs2/gfs2.ko 2023-10-26T01:16:15.102 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/hfs 2023-10-26T01:16:15.102 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/hfs/hfs.ko 2023-10-26T01:16:15.102 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/hfsplus 2023-10-26T01:16:15.103 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/hfsplus/hfsplus.ko 2023-10-26T01:16:15.103 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/isofs 2023-10-26T01:16:15.103 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/isofs/isofs.ko 2023-10-26T01:16:15.103 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/jfs 2023-10-26T01:16:15.103 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/jfs/jfs.ko 2023-10-26T01:16:15.104 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/lockd 2023-10-26T01:16:15.104 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/lockd/lockd.ko 2023-10-26T01:16:15.104 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/minix 2023-10-26T01:16:15.104 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/minix/minix.ko 2023-10-26T01:16:15.104 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/netfs 2023-10-26T01:16:15.105 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/netfs/netfs.ko 2023-10-26T01:16:15.105 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs 2023-10-26T01:16:15.105 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/blocklayout 2023-10-26T01:16:15.105 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-10-26T01:16:15.105 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/filelayout 2023-10-26T01:16:15.106 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-10-26T01:16:15.106 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/flexfilelayout 2023-10-26T01:16:15.106 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-10-26T01:16:15.106 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/nfs.ko 2023-10-26T01:16:15.106 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/nfsv3.ko 2023-10-26T01:16:15.107 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/nfsv4.ko 2023-10-26T01:16:15.107 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs_common 2023-10-26T01:16:15.107 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs_common/grace.ko 2023-10-26T01:16:15.107 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs_common/nfs_acl.ko 2023-10-26T01:16:15.107 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfsd 2023-10-26T01:16:15.108 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfsd/nfsd.ko 2023-10-26T01:16:15.108 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nilfs2 2023-10-26T01:16:15.108 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nilfs2/nilfs2.ko 2023-10-26T01:16:15.108 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls 2023-10-26T01:16:15.108 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-celtic.ko 2023-10-26T01:16:15.109 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-centeuro.ko 2023-10-26T01:16:15.109 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-croatian.ko 2023-10-26T01:16:15.109 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-cyrillic.ko 2023-10-26T01:16:15.109 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-gaelic.ko 2023-10-26T01:16:15.109 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-greek.ko 2023-10-26T01:16:15.110 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-iceland.ko 2023-10-26T01:16:15.110 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-inuit.ko 2023-10-26T01:16:15.110 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-roman.ko 2023-10-26T01:16:15.110 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-romanian.ko 2023-10-26T01:16:15.110 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-turkish.ko 2023-10-26T01:16:15.111 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp1250.ko 2023-10-26T01:16:15.111 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp1251.ko 2023-10-26T01:16:15.111 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp1255.ko 2023-10-26T01:16:15.111 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp737.ko 2023-10-26T01:16:15.111 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp775.ko 2023-10-26T01:16:15.112 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp850.ko 2023-10-26T01:16:15.112 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp852.ko 2023-10-26T01:16:15.112 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp855.ko 2023-10-26T01:16:15.112 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp857.ko 2023-10-26T01:16:15.112 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp860.ko 2023-10-26T01:16:15.113 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp861.ko 2023-10-26T01:16:15.113 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp862.ko 2023-10-26T01:16:15.113 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp863.ko 2023-10-26T01:16:15.113 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp864.ko 2023-10-26T01:16:15.113 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp865.ko 2023-10-26T01:16:15.114 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp866.ko 2023-10-26T01:16:15.114 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp869.ko 2023-10-26T01:16:15.114 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp874.ko 2023-10-26T01:16:15.114 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp932.ko 2023-10-26T01:16:15.115 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp936.ko 2023-10-26T01:16:15.115 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp949.ko 2023-10-26T01:16:15.115 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp950.ko 2023-10-26T01:16:15.115 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_euc-jp.ko 2023-10-26T01:16:15.115 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-1.ko 2023-10-26T01:16:15.116 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-13.ko 2023-10-26T01:16:15.116 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-14.ko 2023-10-26T01:16:15.116 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-15.ko 2023-10-26T01:16:15.116 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-2.ko 2023-10-26T01:16:15.116 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-3.ko 2023-10-26T01:16:15.117 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-4.ko 2023-10-26T01:16:15.117 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-5.ko 2023-10-26T01:16:15.117 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-6.ko 2023-10-26T01:16:15.117 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-7.ko 2023-10-26T01:16:15.117 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-9.ko 2023-10-26T01:16:15.118 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_koi8-r.ko 2023-10-26T01:16:15.118 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_koi8-ru.ko 2023-10-26T01:16:15.118 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_koi8-u.ko 2023-10-26T01:16:15.118 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_ucs2_utils.ko 2023-10-26T01:16:15.118 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_utf8.ko 2023-10-26T01:16:15.119 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2 2023-10-26T01:16:15.119 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/cluster 2023-10-26T01:16:15.119 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-10-26T01:16:15.119 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/dlm 2023-10-26T01:16:15.120 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-10-26T01:16:15.120 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/dlmfs 2023-10-26T01:16:15.120 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-10-26T01:16:15.120 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/ocfs2.ko 2023-10-26T01:16:15.120 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-10-26T01:16:15.121 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-10-26T01:16:15.121 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-10-26T01:16:15.121 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/overlayfs 2023-10-26T01:16:15.121 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/overlayfs/overlay.ko 2023-10-26T01:16:15.121 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/pstore 2023-10-26T01:16:15.122 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/pstore/ramoops.ko 2023-10-26T01:16:15.122 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/reiserfs 2023-10-26T01:16:15.122 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/reiserfs/reiserfs.ko 2023-10-26T01:16:15.122 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/romfs 2023-10-26T01:16:15.123 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/romfs/romfs.ko 2023-10-26T01:16:15.123 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/smb 2023-10-26T01:16:15.123 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/smb/client 2023-10-26T01:16:15.123 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/smb/client/cifs.ko 2023-10-26T01:16:15.123 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/smb/common 2023-10-26T01:16:15.124 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/smb/common/cifs_arc4.ko 2023-10-26T01:16:15.124 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/smb/common/cifs_md4.ko 2023-10-26T01:16:15.124 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/squashfs 2023-10-26T01:16:15.124 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/squashfs/squashfs.ko 2023-10-26T01:16:15.124 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/sysv 2023-10-26T01:16:15.125 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/sysv/sysv.ko 2023-10-26T01:16:15.125 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/udf 2023-10-26T01:16:15.125 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/udf/udf.ko 2023-10-26T01:16:15.125 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ufs 2023-10-26T01:16:15.125 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ufs/ufs.ko 2023-10-26T01:16:15.126 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/xfs 2023-10-26T01:16:15.126 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/xfs/xfs.ko 2023-10-26T01:16:15.126 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/kernel 2023-10-26T01:16:15.126 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/kernel/rcu 2023-10-26T01:16:15.127 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/kernel/rcu/rcutorture.ko 2023-10-26T01:16:15.127 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/kernel/torture.ko 2023-10-26T01:16:15.127 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/kernel/trace 2023-10-26T01:16:15.127 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-10-26T01:16:15.127 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib 2023-10-26T01:16:15.128 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/asn1_encoder.ko 2023-10-26T01:16:15.128 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/crc-itu-t.ko 2023-10-26T01:16:15.128 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/crc8.ko 2023-10-26T01:16:15.128 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/crypto 2023-10-26T01:16:15.128 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/crypto/libarc4.ko 2023-10-26T01:16:15.129 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/crypto/libdes.ko 2023-10-26T01:16:15.129 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/libcrc32c.ko 2023-10-26T01:16:15.129 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/lru_cache.ko 2023-10-26T01:16:15.129 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/lz4 2023-10-26T01:16:15.129 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/lz4/lz4_compress.ko 2023-10-26T01:16:15.130 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/lz4/lz4hc_compress.ko 2023-10-26T01:16:15.130 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/math 2023-10-26T01:16:15.130 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/math/cordic.ko 2023-10-26T01:16:15.130 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/raid6 2023-10-26T01:16:15.130 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/raid6/raid6_pq.ko 2023-10-26T01:16:15.131 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/reed_solomon 2023-10-26T01:16:15.131 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/reed_solomon/reed_solomon.ko 2023-10-26T01:16:15.131 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/ts_bm.ko 2023-10-26T01:16:15.131 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/ts_fsm.ko 2023-10-26T01:16:15.132 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/ts_kmp.ko 2023-10-26T01:16:15.132 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/zstd 2023-10-26T01:16:15.132 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/zstd/zstd_compress.ko 2023-10-26T01:16:15.132 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/mm 2023-10-26T01:16:15.132 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/mm/hwpoison-inject.ko 2023-10-26T01:16:15.133 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net 2023-10-26T01:16:15.133 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan 2023-10-26T01:16:15.133 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan/6lowpan.ko 2023-10-26T01:16:15.133 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan/nhc_dest.ko 2023-10-26T01:16:15.133 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan/nhc_fragment.ko 2023-10-26T01:16:15.134 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan/nhc_hop.ko 2023-10-26T01:16:15.134 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan/nhc_ipv6.ko 2023-10-26T01:16:15.134 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan/nhc_mobility.ko 2023-10-26T01:16:15.134 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan/nhc_routing.ko 2023-10-26T01:16:15.134 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan/nhc_udp.ko 2023-10-26T01:16:15.135 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/802 2023-10-26T01:16:15.135 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/802/garp.ko 2023-10-26T01:16:15.135 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/802/mrp.ko 2023-10-26T01:16:15.135 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/802/p8022.ko 2023-10-26T01:16:15.136 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/802/psnap.ko 2023-10-26T01:16:15.136 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/802/stp.ko 2023-10-26T01:16:15.136 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/8021q 2023-10-26T01:16:15.136 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/8021q/8021q.ko 2023-10-26T01:16:15.136 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/9p 2023-10-26T01:16:15.137 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/9p/9pnet.ko 2023-10-26T01:16:15.137 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/9p/9pnet_fd.ko 2023-10-26T01:16:15.137 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/9p/9pnet_rdma.ko 2023-10-26T01:16:15.137 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/9p/9pnet_virtio.ko 2023-10-26T01:16:15.137 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/batman-adv 2023-10-26T01:16:15.138 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/batman-adv/batman-adv.ko 2023-10-26T01:16:15.138 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge 2023-10-26T01:16:15.138 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/br_netfilter.ko 2023-10-26T01:16:15.138 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/bridge.ko 2023-10-26T01:16:15.139 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter 2023-10-26T01:16:15.139 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-10-26T01:16:15.139 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_among.ko 2023-10-26T01:16:15.139 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_arp.ko 2023-10-26T01:16:15.139 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-10-26T01:16:15.140 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-10-26T01:16:15.140 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_ip.ko 2023-10-26T01:16:15.140 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-10-26T01:16:15.140 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_limit.ko 2023-10-26T01:16:15.140 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_log.ko 2023-10-26T01:16:15.141 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_mark.ko 2023-10-26T01:16:15.141 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-10-26T01:16:15.141 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-10-26T01:16:15.141 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-10-26T01:16:15.141 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-10-26T01:16:15.142 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_snat.ko 2023-10-26T01:16:15.142 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_stp.ko 2023-10-26T01:16:15.142 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-10-26T01:16:15.142 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-10-26T01:16:15.143 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-10-26T01:16:15.143 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-10-26T01:16:15.143 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebtables.ko 2023-10-26T01:16:15.143 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-10-26T01:16:15.144 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-10-26T01:16:15.144 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-10-26T01:16:15.144 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ceph 2023-10-26T01:16:15.145 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ceph/libceph.ko 2023-10-26T01:16:15.145 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/core 2023-10-26T01:16:15.145 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/core/failover.ko 2023-10-26T01:16:15.145 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/core/pktgen.ko 2023-10-26T01:16:15.146 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dccp 2023-10-26T01:16:15.146 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dccp/dccp.ko 2023-10-26T01:16:15.146 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dccp/dccp_diag.ko 2023-10-26T01:16:15.146 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dccp/dccp_ipv4.ko 2023-10-26T01:16:15.146 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dccp/dccp_ipv6.ko 2023-10-26T01:16:15.147 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dns_resolver 2023-10-26T01:16:15.147 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dns_resolver/dns_resolver.ko 2023-10-26T01:16:15.147 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dsa 2023-10-26T01:16:15.147 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dsa/dsa_core.ko 2023-10-26T01:16:15.147 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dsa/tag_brcm.ko 2023-10-26T01:16:15.148 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dsa/tag_dsa.ko 2023-10-26T01:16:15.148 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dsa/tag_none.ko 2023-10-26T01:16:15.148 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dsa/tag_trailer.ko 2023-10-26T01:16:15.148 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ieee802154 2023-10-26T01:16:15.148 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ieee802154/6lowpan 2023-10-26T01:16:15.149 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-10-26T01:16:15.149 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ieee802154/ieee802154.ko 2023-10-26T01:16:15.149 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ieee802154/ieee802154_socket.ko 2023-10-26T01:16:15.149 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4 2023-10-26T01:16:15.150 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/ah4.ko 2023-10-26T01:16:15.150 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/esp4.ko 2023-10-26T01:16:15.150 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/fou.ko 2023-10-26T01:16:15.150 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/gre.ko 2023-10-26T01:16:15.150 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/inet_diag.ko 2023-10-26T01:16:15.151 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/ip_gre.ko 2023-10-26T01:16:15.151 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/ip_tunnel.ko 2023-10-26T01:16:15.151 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/ip_vti.ko 2023-10-26T01:16:15.151 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/ipcomp.ko 2023-10-26T01:16:15.151 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/ipip.ko 2023-10-26T01:16:15.152 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter 2023-10-26T01:16:15.152 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/arp_tables.ko 2023-10-26T01:16:15.152 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-10-26T01:16:15.152 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-10-26T01:16:15.152 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-10-26T01:16:15.153 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-10-26T01:16:15.153 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-10-26T01:16:15.153 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-10-26T01:16:15.153 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-10-26T01:16:15.153 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-10-26T01:16:15.154 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-10-26T01:16:15.154 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/iptable_security.ko 2023-10-26T01:16:15.154 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-10-26T01:16:15.154 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-10-26T01:16:15.154 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-10-26T01:16:15.155 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-10-26T01:16:15.155 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-10-26T01:16:15.155 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-10-26T01:16:15.155 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-10-26T01:16:15.155 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-10-26T01:16:15.156 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-10-26T01:16:15.156 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-10-26T01:16:15.156 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_bic.ko 2023-10-26T01:16:15.156 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_dctcp.ko 2023-10-26T01:16:15.156 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_diag.ko 2023-10-26T01:16:15.157 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_highspeed.ko 2023-10-26T01:16:15.157 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_htcp.ko 2023-10-26T01:16:15.157 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_hybla.ko 2023-10-26T01:16:15.157 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_illinois.ko 2023-10-26T01:16:15.157 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_lp.ko 2023-10-26T01:16:15.158 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_scalable.ko 2023-10-26T01:16:15.158 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_vegas.ko 2023-10-26T01:16:15.158 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_veno.ko 2023-10-26T01:16:15.158 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_westwood.ko 2023-10-26T01:16:15.158 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_yeah.ko 2023-10-26T01:16:15.159 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tunnel4.ko 2023-10-26T01:16:15.159 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/udp_diag.ko 2023-10-26T01:16:15.159 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/udp_tunnel.ko 2023-10-26T01:16:15.159 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/xfrm4_tunnel.ko 2023-10-26T01:16:15.159 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6 2023-10-26T01:16:15.160 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/ah6.ko 2023-10-26T01:16:15.160 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/esp6.ko 2023-10-26T01:16:15.160 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/fou6.ko 2023-10-26T01:16:15.160 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/ip6_tunnel.ko 2023-10-26T01:16:15.161 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-10-26T01:16:15.161 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/ip6_vti.ko 2023-10-26T01:16:15.161 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/ipcomp6.ko 2023-10-26T01:16:15.161 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter 2023-10-26T01:16:15.161 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-10-26T01:16:15.162 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-10-26T01:16:15.162 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-10-26T01:16:15.162 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-10-26T01:16:15.162 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-10-26T01:16:15.162 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-10-26T01:16:15.163 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-10-26T01:16:15.163 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-10-26T01:16:15.163 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-10-26T01:16:15.163 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-10-26T01:16:15.163 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-10-26T01:16:15.164 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-10-26T01:16:15.164 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-10-26T01:16:15.164 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-10-26T01:16:15.164 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-10-26T01:16:15.164 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-10-26T01:16:15.165 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-10-26T01:16:15.165 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-10-26T01:16:15.165 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-10-26T01:16:15.165 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-10-26T01:16:15.166 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-10-26T01:16:15.166 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-10-26T01:16:15.166 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-10-26T01:16:15.166 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-10-26T01:16:15.167 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/sit.ko 2023-10-26T01:16:15.167 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/tunnel6.ko 2023-10-26T01:16:15.167 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/xfrm6_tunnel.ko 2023-10-26T01:16:15.167 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/key 2023-10-26T01:16:15.168 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/key/af_key.ko 2023-10-26T01:16:15.168 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/l2tp 2023-10-26T01:16:15.168 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/l2tp/l2tp_core.ko 2023-10-26T01:16:15.168 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/l2tp/l2tp_debugfs.ko 2023-10-26T01:16:15.169 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/l2tp/l2tp_eth.ko 2023-10-26T01:16:15.169 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/l2tp/l2tp_ip.ko 2023-10-26T01:16:15.169 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/l2tp/l2tp_ip6.ko 2023-10-26T01:16:15.169 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/l2tp/l2tp_netlink.ko 2023-10-26T01:16:15.170 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/l2tp/l2tp_ppp.ko 2023-10-26T01:16:15.170 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/llc 2023-10-26T01:16:15.170 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/llc/llc.ko 2023-10-26T01:16:15.170 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/mac802154 2023-10-26T01:16:15.170 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/mac802154/mac802154.ko 2023-10-26T01:16:15.171 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/mpls 2023-10-26T01:16:15.171 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/mpls/mpls_gso.ko 2023-10-26T01:16:15.171 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter 2023-10-26T01:16:15.171 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset 2023-10-26T01:16:15.172 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set.ko 2023-10-26T01:16:15.172 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-10-26T01:16:15.172 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-10-26T01:16:15.172 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-10-26T01:16:15.172 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-10-26T01:16:15.173 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-10-26T01:16:15.173 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-10-26T01:16:15.173 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-10-26T01:16:15.173 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-10-26T01:16:15.174 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-10-26T01:16:15.174 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-10-26T01:16:15.174 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-10-26T01:16:15.174 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-10-26T01:16:15.175 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-10-26T01:16:15.175 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-10-26T01:16:15.175 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-10-26T01:16:15.175 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs 2023-10-26T01:16:15.175 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs.ko 2023-10-26T01:16:15.176 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-10-26T01:16:15.176 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-10-26T01:16:15.176 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-10-26T01:16:15.176 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-10-26T01:16:15.177 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-10-26T01:16:15.177 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-10-26T01:16:15.177 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-10-26T01:16:15.178 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-10-26T01:16:15.178 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-10-26T01:16:15.178 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-10-26T01:16:15.178 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-10-26T01:16:15.179 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-10-26T01:16:15.179 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-10-26T01:16:15.179 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conncount.ko 2023-10-26T01:16:15.179 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack.ko 2023-10-26T01:16:15.179 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-10-26T01:16:15.180 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-10-26T01:16:15.180 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-10-26T01:16:15.180 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_h323.ko 2023-10-26T01:16:15.180 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_irc.ko 2023-10-26T01:16:15.181 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-10-26T01:16:15.181 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-10-26T01:16:15.181 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-10-26T01:16:15.181 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_sane.ko 2023-10-26T01:16:15.181 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_sip.ko 2023-10-26T01:16:15.182 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-10-26T01:16:15.182 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-10-26T01:16:15.182 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_dup_netdev.ko 2023-10-26T01:16:15.182 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_flow_table.ko 2023-10-26T01:16:15.182 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_flow_table_inet.ko 2023-10-26T01:16:15.183 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_log_syslog.ko 2023-10-26T01:16:15.183 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_nat.ko 2023-10-26T01:16:15.183 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_nat_amanda.ko 2023-10-26T01:16:15.183 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_nat_ftp.ko 2023-10-26T01:16:15.183 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_nat_irc.ko 2023-10-26T01:16:15.184 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_nat_sip.ko 2023-10-26T01:16:15.184 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_nat_tftp.ko 2023-10-26T01:16:15.184 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_synproxy_core.ko 2023-10-26T01:16:15.184 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_tables.ko 2023-10-26T01:16:15.184 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nfnetlink.ko 2023-10-26T01:16:15.185 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nfnetlink_acct.ko 2023-10-26T01:16:15.185 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nfnetlink_log.ko 2023-10-26T01:16:15.185 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nfnetlink_osf.ko 2023-10-26T01:16:15.185 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nfnetlink_queue.ko 2023-10-26T01:16:15.186 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_chain_nat.ko 2023-10-26T01:16:15.186 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_compat.ko 2023-10-26T01:16:15.186 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_ct.ko 2023-10-26T01:16:15.186 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_dup_netdev.ko 2023-10-26T01:16:15.186 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_fib.ko 2023-10-26T01:16:15.187 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_fwd_netdev.ko 2023-10-26T01:16:15.187 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_hash.ko 2023-10-26T01:16:15.187 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_limit.ko 2023-10-26T01:16:15.187 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_log.ko 2023-10-26T01:16:15.187 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_masq.ko 2023-10-26T01:16:15.188 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_nat.ko 2023-10-26T01:16:15.188 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_numgen.ko 2023-10-26T01:16:15.188 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_queue.ko 2023-10-26T01:16:15.188 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_quota.ko 2023-10-26T01:16:15.188 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_redir.ko 2023-10-26T01:16:15.189 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_reject.ko 2023-10-26T01:16:15.189 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_reject_inet.ko 2023-10-26T01:16:15.189 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_socket.ko 2023-10-26T01:16:15.189 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_tunnel.ko 2023-10-26T01:16:15.189 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_AUDIT.ko 2023-10-26T01:16:15.190 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_CHECKSUM.ko 2023-10-26T01:16:15.190 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_CLASSIFY.ko 2023-10-26T01:16:15.190 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-10-26T01:16:15.190 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_CT.ko 2023-10-26T01:16:15.191 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_DSCP.ko 2023-10-26T01:16:15.191 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_HL.ko 2023-10-26T01:16:15.191 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_HMARK.ko 2023-10-26T01:16:15.191 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_IDLETIMER.ko 2023-10-26T01:16:15.192 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_LED.ko 2023-10-26T01:16:15.192 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_LOG.ko 2023-10-26T01:16:15.192 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_MASQUERADE.ko 2023-10-26T01:16:15.192 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_NETMAP.ko 2023-10-26T01:16:15.192 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_NFLOG.ko 2023-10-26T01:16:15.193 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_NFQUEUE.ko 2023-10-26T01:16:15.193 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_RATEEST.ko 2023-10-26T01:16:15.193 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_REDIRECT.ko 2023-10-26T01:16:15.193 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_SECMARK.ko 2023-10-26T01:16:15.193 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_TCPMSS.ko 2023-10-26T01:16:15.194 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-10-26T01:16:15.194 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_TEE.ko 2023-10-26T01:16:15.194 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_TPROXY.ko 2023-10-26T01:16:15.194 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_TRACE.ko 2023-10-26T01:16:15.194 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_addrtype.ko 2023-10-26T01:16:15.195 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_bpf.ko 2023-10-26T01:16:15.195 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_cgroup.ko 2023-10-26T01:16:15.195 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_cluster.ko 2023-10-26T01:16:15.195 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_comment.ko 2023-10-26T01:16:15.195 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_connbytes.ko 2023-10-26T01:16:15.196 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_connlabel.ko 2023-10-26T01:16:15.196 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_connlimit.ko 2023-10-26T01:16:15.196 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_connmark.ko 2023-10-26T01:16:15.196 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_conntrack.ko 2023-10-26T01:16:15.196 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_cpu.ko 2023-10-26T01:16:15.197 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_dccp.ko 2023-10-26T01:16:15.197 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_devgroup.ko 2023-10-26T01:16:15.197 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_dscp.ko 2023-10-26T01:16:15.197 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_ecn.ko 2023-10-26T01:16:15.197 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_esp.ko 2023-10-26T01:16:15.198 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_hashlimit.ko 2023-10-26T01:16:15.198 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_helper.ko 2023-10-26T01:16:15.198 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_hl.ko 2023-10-26T01:16:15.198 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_ipcomp.ko 2023-10-26T01:16:15.199 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_iprange.ko 2023-10-26T01:16:15.199 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_ipvs.ko 2023-10-26T01:16:15.199 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_l2tp.ko 2023-10-26T01:16:15.199 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_length.ko 2023-10-26T01:16:15.200 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_limit.ko 2023-10-26T01:16:15.200 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_mac.ko 2023-10-26T01:16:15.200 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_mark.ko 2023-10-26T01:16:15.200 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_multiport.ko 2023-10-26T01:16:15.201 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_nat.ko 2023-10-26T01:16:15.201 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_nfacct.ko 2023-10-26T01:16:15.201 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_osf.ko 2023-10-26T01:16:15.201 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_owner.ko 2023-10-26T01:16:15.202 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_physdev.ko 2023-10-26T01:16:15.202 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_pkttype.ko 2023-10-26T01:16:15.202 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_policy.ko 2023-10-26T01:16:15.202 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_quota.ko 2023-10-26T01:16:15.203 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_rateest.ko 2023-10-26T01:16:15.203 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_realm.ko 2023-10-26T01:16:15.203 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_recent.ko 2023-10-26T01:16:15.203 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_sctp.ko 2023-10-26T01:16:15.203 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_set.ko 2023-10-26T01:16:15.204 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_socket.ko 2023-10-26T01:16:15.204 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_state.ko 2023-10-26T01:16:15.204 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_statistic.ko 2023-10-26T01:16:15.204 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_string.ko 2023-10-26T01:16:15.205 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_tcpmss.ko 2023-10-26T01:16:15.205 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_time.ko 2023-10-26T01:16:15.205 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_u32.ko 2023-10-26T01:16:15.205 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netlink 2023-10-26T01:16:15.206 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netlink/netlink_diag.ko 2023-10-26T01:16:15.206 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/nsh 2023-10-26T01:16:15.206 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/nsh/nsh.ko 2023-10-26T01:16:15.206 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/openvswitch 2023-10-26T01:16:15.206 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/openvswitch/openvswitch.ko 2023-10-26T01:16:15.207 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/openvswitch/vport-geneve.ko 2023-10-26T01:16:15.207 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/openvswitch/vport-gre.ko 2023-10-26T01:16:15.207 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/openvswitch/vport-vxlan.ko 2023-10-26T01:16:15.207 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/packet 2023-10-26T01:16:15.207 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/packet/af_packet_diag.ko 2023-10-26T01:16:15.208 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/rds 2023-10-26T01:16:15.208 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/rds/rds.ko 2023-10-26T01:16:15.208 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/rds/rds_rdma.ko 2023-10-26T01:16:15.209 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/rds/rds_tcp.ko 2023-10-26T01:16:15.209 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched 2023-10-26T01:16:15.209 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_csum.ko 2023-10-26T01:16:15.209 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_gact.ko 2023-10-26T01:16:15.209 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_ipt.ko 2023-10-26T01:16:15.210 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_mirred.ko 2023-10-26T01:16:15.210 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_nat.ko 2023-10-26T01:16:15.210 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_pedit.ko 2023-10-26T01:16:15.210 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_police.ko 2023-10-26T01:16:15.211 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_simple.ko 2023-10-26T01:16:15.211 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_skbedit.ko 2023-10-26T01:16:15.211 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_vlan.ko 2023-10-26T01:16:15.212 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/cls_basic.ko 2023-10-26T01:16:15.212 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/cls_bpf.ko 2023-10-26T01:16:15.212 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/cls_flow.ko 2023-10-26T01:16:15.212 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/cls_fw.ko 2023-10-26T01:16:15.213 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/cls_route.ko 2023-10-26T01:16:15.213 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/cls_u32.ko 2023-10-26T01:16:15.213 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/em_cmp.ko 2023-10-26T01:16:15.214 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/em_ipset.ko 2023-10-26T01:16:15.214 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/em_meta.ko 2023-10-26T01:16:15.214 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/em_nbyte.ko 2023-10-26T01:16:15.215 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/em_text.ko 2023-10-26T01:16:15.215 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/em_u32.ko 2023-10-26T01:16:15.215 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_choke.ko 2023-10-26T01:16:15.215 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_codel.ko 2023-10-26T01:16:15.215 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_drr.ko 2023-10-26T01:16:15.216 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_fq.ko 2023-10-26T01:16:15.216 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_gred.ko 2023-10-26T01:16:15.216 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_hfsc.ko 2023-10-26T01:16:15.216 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_hhf.ko 2023-10-26T01:16:15.216 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_htb.ko 2023-10-26T01:16:15.217 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_ingress.ko 2023-10-26T01:16:15.217 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_mqprio.ko 2023-10-26T01:16:15.217 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_mqprio_lib.ko 2023-10-26T01:16:15.217 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_multiq.ko 2023-10-26T01:16:15.217 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_netem.ko 2023-10-26T01:16:15.218 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_pie.ko 2023-10-26T01:16:15.218 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_plug.ko 2023-10-26T01:16:15.218 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_prio.ko 2023-10-26T01:16:15.218 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_qfq.ko 2023-10-26T01:16:15.218 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_red.ko 2023-10-26T01:16:15.219 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_sfb.ko 2023-10-26T01:16:15.219 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_sfq.ko 2023-10-26T01:16:15.219 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_tbf.ko 2023-10-26T01:16:15.219 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_teql.ko 2023-10-26T01:16:15.220 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sctp 2023-10-26T01:16:15.220 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sctp/sctp.ko 2023-10-26T01:16:15.220 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sctp/sctp_diag.ko 2023-10-26T01:16:15.220 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sunrpc 2023-10-26T01:16:15.220 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sunrpc/auth_gss 2023-10-26T01:16:15.221 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-10-26T01:16:15.221 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-10-26T01:16:15.221 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sunrpc/sunrpc.ko 2023-10-26T01:16:15.221 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sunrpc/xprtrdma 2023-10-26T01:16:15.221 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-10-26T01:16:15.222 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/tipc 2023-10-26T01:16:15.222 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/tipc/diag.ko 2023-10-26T01:16:15.222 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/tipc/tipc.ko 2023-10-26T01:16:15.222 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/unix 2023-10-26T01:16:15.222 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/unix/unix_diag.ko 2023-10-26T01:16:15.223 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/vmw_vsock 2023-10-26T01:16:15.223 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-10-26T01:16:15.223 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-10-26T01:16:15.223 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/vmw_vsock/vsock.ko 2023-10-26T01:16:15.223 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/vmw_vsock/vsock_diag.ko 2023-10-26T01:16:15.224 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/vmw_vsock/vsock_loopback.ko 2023-10-26T01:16:15.224 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/xfrm 2023-10-26T01:16:15.224 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/xfrm/xfrm_ipcomp.ko 2023-10-26T01:16:15.224 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/security 2023-10-26T01:16:15.224 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/security/keys 2023-10-26T01:16:15.225 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/security/keys/encrypted-keys 2023-10-26T01:16:15.225 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-10-26T01:16:15.225 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/security/keys/trusted-keys 2023-10-26T01:16:15.225 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/security/keys/trusted-keys/trusted.ko 2023-10-26T01:16:15.226 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/virt 2023-10-26T01:16:15.226 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/virt/lib 2023-10-26T01:16:15.226 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/virt/lib/irqbypass.ko 2023-10-26T01:16:15.226 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.alias 2023-10-26T01:16:15.226 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.alias.bin 2023-10-26T01:16:15.227 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.builtin 2023-10-26T01:16:15.227 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.builtin.alias.bin 2023-10-26T01:16:15.227 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.builtin.bin 2023-10-26T01:16:15.227 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.builtin.modinfo 2023-10-26T01:16:15.227 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.dep 2023-10-26T01:16:15.228 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.dep.bin 2023-10-26T01:16:15.228 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.devname 2023-10-26T01:16:15.228 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.order 2023-10-26T01:16:15.228 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.softdep 2023-10-26T01:16:15.228 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.symbols 2023-10-26T01:16:15.229 INFO:teuthology.orchestra.run.smithi170.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.symbols.bin 2023-10-26T01:16:15.229 DEBUG:teuthology.task.kernel:get_image_version: 6.6.0-rc5-g5b73517e0e2f 2023-10-26T01:16:15.230 DEBUG:teuthology.orchestra.run.smithi170:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-10-26T01:16:15.253 INFO:teuthology.orchestra.run.smithi170.stdout:postinstall scriptlet (using /bin/sh): 2023-10-26T01:16:15.253 INFO:teuthology.orchestra.run.smithi170.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.6.0-rc5-g5b73517e0e2f -a -r /boot/System.map-6.6.0-rc5-g5b73517e0e2f ]; then 2023-10-26T01:16:15.253 INFO:teuthology.orchestra.run.smithi170.stdout:cp /boot/vmlinuz-6.6.0-rc5-g5b73517e0e2f /boot/.vmlinuz-6.6.0-rc5-g5b73517e0e2f-rpm 2023-10-26T01:16:15.254 INFO:teuthology.orchestra.run.smithi170.stdout:cp /boot/System.map-6.6.0-rc5-g5b73517e0e2f /boot/.System.map-6.6.0-rc5-g5b73517e0e2f-rpm 2023-10-26T01:16:15.254 INFO:teuthology.orchestra.run.smithi170.stdout:rm -f /boot/vmlinuz-6.6.0-rc5-g5b73517e0e2f /boot/System.map-6.6.0-rc5-g5b73517e0e2f 2023-10-26T01:16:15.254 INFO:teuthology.orchestra.run.smithi170.stdout:/sbin/installkernel 6.6.0-rc5-g5b73517e0e2f /boot/.vmlinuz-6.6.0-rc5-g5b73517e0e2f-rpm /boot/.System.map-6.6.0-rc5-g5b73517e0e2f-rpm 2023-10-26T01:16:15.254 INFO:teuthology.orchestra.run.smithi170.stdout:rm -f /boot/.vmlinuz-6.6.0-rc5-g5b73517e0e2f-rpm /boot/.System.map-6.6.0-rc5-g5b73517e0e2f-rpm 2023-10-26T01:16:15.254 INFO:teuthology.orchestra.run.smithi170.stdout:fi 2023-10-26T01:16:15.255 INFO:teuthology.orchestra.run.smithi170.stdout:preuninstall scriptlet (using /bin/sh): 2023-10-26T01:16:15.255 INFO:teuthology.orchestra.run.smithi170.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-10-26T01:16:15.255 INFO:teuthology.orchestra.run.smithi170.stdout:new-kernel-pkg --remove 6.6.0-rc5-g5b73517e0e2f --rminitrd --initrdfile=/boot/initramfs-6.6.0-rc5-g5b73517e0e2f.img 2023-10-26T01:16:15.255 INFO:teuthology.orchestra.run.smithi170.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-10-26T01:16:15.255 INFO:teuthology.orchestra.run.smithi170.stdout:kernel-install remove 6.6.0-rc5-g5b73517e0e2f 2023-10-26T01:16:15.256 INFO:teuthology.orchestra.run.smithi170.stdout:fi 2023-10-26T01:16:15.256 INFO:teuthology.orchestra.run.smithi170.stdout:postuninstall scriptlet (using /bin/sh): 2023-10-26T01:16:15.256 INFO:teuthology.orchestra.run.smithi170.stdout:if [ -x /sbin/update-bootloader ]; then 2023-10-26T01:16:15.256 INFO:teuthology.orchestra.run.smithi170.stdout:/sbin/update-bootloader --remove 6.6.0-rc5-g5b73517e0e2f 2023-10-26T01:16:15.256 INFO:teuthology.orchestra.run.smithi170.stdout:fi 2023-10-26T01:16:15.257 DEBUG:teuthology.orchestra.run.smithi170:> sudo rpm -qi grub2-tools 2023-10-26T01:16:15.332 INFO:teuthology.orchestra.run.smithi170.stdout:Name : grub2-tools 2023-10-26T01:16:15.333 INFO:teuthology.orchestra.run.smithi170.stdout:Epoch : 1 2023-10-26T01:16:15.333 INFO:teuthology.orchestra.run.smithi170.stdout:Version : 2.06 2023-10-26T01:16:15.333 INFO:teuthology.orchestra.run.smithi170.stdout:Release : 68.el9 2023-10-26T01:16:15.333 INFO:teuthology.orchestra.run.smithi170.stdout:Architecture: x86_64 2023-10-26T01:16:15.334 INFO:teuthology.orchestra.run.smithi170.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2023-10-26T01:16:15.334 INFO:teuthology.orchestra.run.smithi170.stdout:Group : Unspecified 2023-10-26T01:16:15.334 INFO:teuthology.orchestra.run.smithi170.stdout:Size : 8264254 2023-10-26T01:16:15.335 INFO:teuthology.orchestra.run.smithi170.stdout:License : GPLv3+ 2023-10-26T01:16:15.335 INFO:teuthology.orchestra.run.smithi170.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2023-10-26T01:16:15.335 INFO:teuthology.orchestra.run.smithi170.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2023-10-26T01:16:15.335 INFO:teuthology.orchestra.run.smithi170.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2023-10-26T01:16:15.336 INFO:teuthology.orchestra.run.smithi170.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2023-10-26T01:16:15.336 INFO:teuthology.orchestra.run.smithi170.stdout:Packager : builder@centos.org 2023-10-26T01:16:15.336 INFO:teuthology.orchestra.run.smithi170.stdout:Vendor : CentOS 2023-10-26T01:16:15.336 INFO:teuthology.orchestra.run.smithi170.stdout:URL : http://www.gnu.org/software/grub/ 2023-10-26T01:16:15.336 INFO:teuthology.orchestra.run.smithi170.stdout:Summary : Support tools for GRUB. 2023-10-26T01:16:15.337 INFO:teuthology.orchestra.run.smithi170.stdout:Description : 2023-10-26T01:16:15.337 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-26T01:16:15.337 INFO:teuthology.orchestra.run.smithi170.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-10-26T01:16:15.337 INFO:teuthology.orchestra.run.smithi170.stdout:customizable bootloader with modular architecture. It supports a rich 2023-10-26T01:16:15.337 INFO:teuthology.orchestra.run.smithi170.stdout:variety of kernel formats, file systems, computer architectures and 2023-10-26T01:16:15.338 INFO:teuthology.orchestra.run.smithi170.stdout:hardware devices. 2023-10-26T01:16:15.338 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-26T01:16:15.338 INFO:teuthology.orchestra.run.smithi170.stdout:This subpackage provides tools for support of all platforms. 2023-10-26T01:16:15.339 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-10-26T01:16:15.339 INFO:teuthology.task.kernel:Updating grub on smithi170 to boot 6.6.0-rc5-g5b73517e0e2f 2023-10-26T01:16:15.339 DEBUG:teuthology.orchestra.run.smithi170:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-10-26T01:16:15.980 INFO:teuthology.orchestra.run.smithi170.stderr:Generating grub configuration file ... 2023-10-26T01:16:17.379 INFO:teuthology.orchestra.run.smithi170.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2023-10-26T01:16:17.416 INFO:teuthology.orchestra.run.smithi170.stderr:done 2023-10-26T01:16:17.420 DEBUG:teuthology.orchestra.run.smithi170:> mktemp 2023-10-26T01:16:17.439 INFO:teuthology.orchestra.run.smithi170.stdout:/tmp/tmp.TwmX4lBJP6 2023-10-26T01:16:17.439 DEBUG:teuthology.orchestra.run.smithi170:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.TwmX4lBJP6 2023-10-26T01:16:17.518 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod 0666 /tmp/tmp.TwmX4lBJP6 2023-10-26T01:16:17.655 DEBUG:teuthology.orchestra.remote:smithi170:/tmp/tmp.TwmX4lBJP6 is 6KB 2023-10-26T01:16:17.704 DEBUG:teuthology.orchestra.run.smithi170:> rm -fr /tmp/tmp.TwmX4lBJP6 2023-10-26T01:16:17.718 DEBUG:teuthology.orchestra.run.smithi170:> sudo /bin/ls /boot/loader/entries || true 2023-10-26T01:16:17.783 INFO:teuthology.orchestra.run.smithi170.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2023-10-26T01:16:17.783 INFO:teuthology.orchestra.run.smithi170.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2023-10-26T01:16:17.784 INFO:teuthology.orchestra.run.smithi170.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2023-10-26T01:16:17.784 INFO:teuthology.orchestra.run.smithi170.stdout:15949e3c88704300882482eddfd7baae-6.6.0-rc5-g5b73517e0e2f.conf 2023-10-26T01:16:17.785 DEBUG:teuthology.orchestra.run.smithi170:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.6.0-rc5-g5b73517e0e2f 2023-10-26T01:16:17.925 DEBUG:teuthology.orchestra.run.smithi170:> sudo shutdown -r now 2023-10-26T01:16:21.722 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi169.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-10-26T01:16:21.723 DEBUG:teuthology.orchestra.run.smithi169:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-10-26T01:16:21.795 INFO:teuthology.orchestra.run.smithi169.stdout:/boot/System.map-6.6.0-rc5-g5b73517e0e2f 2023-10-26T01:16:21.795 INFO:teuthology.orchestra.run.smithi169.stdout:/boot/config-6.6.0-rc5-g5b73517e0e2f 2023-10-26T01:16:21.795 INFO:teuthology.orchestra.run.smithi169.stdout:/boot/vmlinuz-6.6.0-rc5-g5b73517e0e2f 2023-10-26T01:16:21.796 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f 2023-10-26T01:16:21.796 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel 2023-10-26T01:16:21.796 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch 2023-10-26T01:16:21.796 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86 2023-10-26T01:16:21.796 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto 2023-10-26T01:16:21.797 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-10-26T01:16:21.797 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-10-26T01:16:21.797 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-10-26T01:16:21.797 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-10-26T01:16:21.797 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-10-26T01:16:21.798 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-10-26T01:16:21.798 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-10-26T01:16:21.798 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/crc32c-intel.ko 2023-10-26T01:16:21.798 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-10-26T01:16:21.798 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-10-26T01:16:21.799 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-10-26T01:16:21.799 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-10-26T01:16:21.799 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/serpent-avx2.ko 2023-10-26T01:16:21.799 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-10-26T01:16:21.799 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-10-26T01:16:21.800 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-10-26T01:16:21.800 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-10-26T01:16:21.800 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-10-26T01:16:21.800 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-10-26T01:16:21.800 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-10-26T01:16:21.800 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/kernel 2023-10-26T01:16:21.801 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/kernel/cpu 2023-10-26T01:16:21.801 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/kernel/cpu/mce 2023-10-26T01:16:21.801 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-10-26T01:16:21.801 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/kvm 2023-10-26T01:16:21.801 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/kvm/kvm-amd.ko 2023-10-26T01:16:21.802 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/kvm/kvm-intel.ko 2023-10-26T01:16:21.802 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/kvm/kvm.ko 2023-10-26T01:16:21.802 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto 2023-10-26T01:16:21.802 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/ansi_cprng.ko 2023-10-26T01:16:21.802 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/anubis.ko 2023-10-26T01:16:21.803 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/arc4.ko 2023-10-26T01:16:21.803 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/async_tx 2023-10-26T01:16:21.803 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/async_tx/async_memcpy.ko 2023-10-26T01:16:21.803 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/async_tx/async_pq.ko 2023-10-26T01:16:21.803 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/async_tx/async_raid6_recov.ko 2023-10-26T01:16:21.804 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/async_tx/async_tx.ko 2023-10-26T01:16:21.804 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/async_tx/async_xor.ko 2023-10-26T01:16:21.804 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/async_tx/raid6test.ko 2023-10-26T01:16:21.804 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/authenc.ko 2023-10-26T01:16:21.804 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/authencesn.ko 2023-10-26T01:16:21.805 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/blake2b_generic.ko 2023-10-26T01:16:21.805 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/blowfish_common.ko 2023-10-26T01:16:21.805 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/blowfish_generic.ko 2023-10-26T01:16:21.805 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/camellia_generic.ko 2023-10-26T01:16:21.805 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/cast5_generic.ko 2023-10-26T01:16:21.805 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/cast6_generic.ko 2023-10-26T01:16:21.806 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/cast_common.ko 2023-10-26T01:16:21.806 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/ccm.ko 2023-10-26T01:16:21.806 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/cmac.ko 2023-10-26T01:16:21.806 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/crc32_generic.ko 2023-10-26T01:16:21.806 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/crypto_engine.ko 2023-10-26T01:16:21.807 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/crypto_user.ko 2023-10-26T01:16:21.807 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/cts.ko 2023-10-26T01:16:21.807 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/des_generic.ko 2023-10-26T01:16:21.807 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/dh_generic.ko 2023-10-26T01:16:21.807 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/echainiv.ko 2023-10-26T01:16:21.808 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/essiv.ko 2023-10-26T01:16:21.808 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/fcrypt.ko 2023-10-26T01:16:21.808 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/khazad.ko 2023-10-26T01:16:21.808 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/lz4.ko 2023-10-26T01:16:21.808 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/lz4hc.ko 2023-10-26T01:16:21.809 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/md4.ko 2023-10-26T01:16:21.809 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/michael_mic.ko 2023-10-26T01:16:21.809 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/pcbc.ko 2023-10-26T01:16:21.809 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/pcrypt.ko 2023-10-26T01:16:21.809 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/rmd160.ko 2023-10-26T01:16:21.810 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/seed.ko 2023-10-26T01:16:21.810 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/serpent_generic.ko 2023-10-26T01:16:21.810 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/tcrypt.ko 2023-10-26T01:16:21.810 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/tea.ko 2023-10-26T01:16:21.810 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/twofish_common.ko 2023-10-26T01:16:21.811 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/twofish_generic.ko 2023-10-26T01:16:21.811 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/vmac.ko 2023-10-26T01:16:21.811 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/wp512.ko 2023-10-26T01:16:21.811 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/xcbc.ko 2023-10-26T01:16:21.811 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/xor.ko 2023-10-26T01:16:21.811 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/xxhash_generic.ko 2023-10-26T01:16:21.812 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers 2023-10-26T01:16:21.812 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi 2023-10-26T01:16:21.812 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi/acpi_ipmi.ko 2023-10-26T01:16:21.812 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi/acpi_pad.ko 2023-10-26T01:16:21.812 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi/custom_method.ko 2023-10-26T01:16:21.813 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi/ec_sys.ko 2023-10-26T01:16:21.813 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi/platform_profile.ko 2023-10-26T01:16:21.813 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi/sbs.ko 2023-10-26T01:16:21.813 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi/sbshc.ko 2023-10-26T01:16:21.813 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi/video.ko 2023-10-26T01:16:21.813 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata 2023-10-26T01:16:21.814 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/acard-ahci.ko 2023-10-26T01:16:21.814 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/ahci_platform.ko 2023-10-26T01:16:21.814 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/ata_generic.ko 2023-10-26T01:16:21.814 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/libahci_platform.ko 2023-10-26T01:16:21.815 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_acpi.ko 2023-10-26T01:16:21.815 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_ali.ko 2023-10-26T01:16:21.815 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_amd.ko 2023-10-26T01:16:21.815 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_artop.ko 2023-10-26T01:16:21.815 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_atiixp.ko 2023-10-26T01:16:21.815 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_atp867x.ko 2023-10-26T01:16:21.816 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_cmd640.ko 2023-10-26T01:16:21.816 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_cmd64x.ko 2023-10-26T01:16:21.816 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_cypress.ko 2023-10-26T01:16:21.816 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_efar.ko 2023-10-26T01:16:21.816 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_hpt366.ko 2023-10-26T01:16:21.817 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_hpt37x.ko 2023-10-26T01:16:21.817 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_hpt3x2n.ko 2023-10-26T01:16:21.817 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_hpt3x3.ko 2023-10-26T01:16:21.817 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_it8213.ko 2023-10-26T01:16:21.817 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_it821x.ko 2023-10-26T01:16:21.817 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_jmicron.ko 2023-10-26T01:16:21.818 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_marvell.ko 2023-10-26T01:16:21.818 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_mpiix.ko 2023-10-26T01:16:21.818 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_netcell.ko 2023-10-26T01:16:21.818 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_ninja32.ko 2023-10-26T01:16:21.819 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_ns87410.ko 2023-10-26T01:16:21.819 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_ns87415.ko 2023-10-26T01:16:21.819 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_oldpiix.ko 2023-10-26T01:16:21.819 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_opti.ko 2023-10-26T01:16:21.819 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_optidma.ko 2023-10-26T01:16:21.819 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_pcmcia.ko 2023-10-26T01:16:21.820 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_pdc2027x.ko 2023-10-26T01:16:21.820 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-10-26T01:16:21.820 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_piccolo.ko 2023-10-26T01:16:21.820 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_rdc.ko 2023-10-26T01:16:21.820 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_sch.ko 2023-10-26T01:16:21.821 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_serverworks.ko 2023-10-26T01:16:21.821 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_sil680.ko 2023-10-26T01:16:21.821 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_sis.ko 2023-10-26T01:16:21.821 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_sl82c105.ko 2023-10-26T01:16:21.821 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_triflex.ko 2023-10-26T01:16:21.822 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_via.ko 2023-10-26T01:16:21.822 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pdc_adma.ko 2023-10-26T01:16:21.822 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_inic162x.ko 2023-10-26T01:16:21.822 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_mv.ko 2023-10-26T01:16:21.822 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_nv.ko 2023-10-26T01:16:21.823 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_promise.ko 2023-10-26T01:16:21.823 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_qstor.ko 2023-10-26T01:16:21.823 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_sil.ko 2023-10-26T01:16:21.823 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_sil24.ko 2023-10-26T01:16:21.823 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_sis.ko 2023-10-26T01:16:21.823 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_svw.ko 2023-10-26T01:16:21.824 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_sx4.ko 2023-10-26T01:16:21.824 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_uli.ko 2023-10-26T01:16:21.824 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_via.ko 2023-10-26T01:16:21.824 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_vsc.ko 2023-10-26T01:16:21.824 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/auxdisplay 2023-10-26T01:16:21.825 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/auxdisplay/ks0108.ko 2023-10-26T01:16:21.825 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/base 2023-10-26T01:16:21.825 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/base/regmap 2023-10-26T01:16:21.825 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/base/regmap/regmap-i2c.ko 2023-10-26T01:16:21.825 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/base/regmap/regmap-mmio.ko 2023-10-26T01:16:21.826 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/bcma 2023-10-26T01:16:21.826 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/bcma/bcma.ko 2023-10-26T01:16:21.826 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block 2023-10-26T01:16:21.826 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/aoe 2023-10-26T01:16:21.826 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/aoe/aoe.ko 2023-10-26T01:16:21.827 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/brd.ko 2023-10-26T01:16:21.827 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/drbd 2023-10-26T01:16:21.827 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/drbd/drbd.ko 2023-10-26T01:16:21.827 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/floppy.ko 2023-10-26T01:16:21.827 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/loop.ko 2023-10-26T01:16:21.827 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/mtip32xx 2023-10-26T01:16:21.828 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-10-26T01:16:21.828 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/nbd.ko 2023-10-26T01:16:21.828 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/null_blk 2023-10-26T01:16:21.828 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/null_blk/null_blk.ko 2023-10-26T01:16:21.828 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/pktcdvd.ko 2023-10-26T01:16:21.829 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/rbd.ko 2023-10-26T01:16:21.829 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/virtio_blk.ko 2023-10-26T01:16:21.829 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/xen-blkback 2023-10-26T01:16:21.829 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-10-26T01:16:21.829 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/xen-blkfront.ko 2023-10-26T01:16:21.830 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/zram 2023-10-26T01:16:21.830 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/zram/zram.ko 2023-10-26T01:16:21.830 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char 2023-10-26T01:16:21.830 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/hangcheck-timer.ko 2023-10-26T01:16:21.830 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/hw_random 2023-10-26T01:16:21.831 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/hw_random/amd-rng.ko 2023-10-26T01:16:21.831 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/hw_random/intel-rng.ko 2023-10-26T01:16:21.831 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-10-26T01:16:21.831 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/hw_random/via-rng.ko 2023-10-26T01:16:21.831 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/hw_random/virtio-rng.ko 2023-10-26T01:16:21.832 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/ipmi 2023-10-26T01:16:21.832 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-10-26T01:16:21.832 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-10-26T01:16:21.832 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-10-26T01:16:21.833 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/ipmi/ipmi_si.ko 2023-10-26T01:16:21.833 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-10-26T01:16:21.833 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-10-26T01:16:21.833 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/lp.ko 2023-10-26T01:16:21.833 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/mwave 2023-10-26T01:16:21.834 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/mwave/mwave.ko 2023-10-26T01:16:21.834 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/ppdev.ko 2023-10-26T01:16:21.834 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/tlclk.ko 2023-10-26T01:16:21.834 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/tpm 2023-10-26T01:16:21.834 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/tpm/tpm.ko 2023-10-26T01:16:21.834 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/tpm/tpm_atmel.ko 2023-10-26T01:16:21.835 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/tpm/tpm_infineon.ko 2023-10-26T01:16:21.835 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/tpm/tpm_nsc.ko 2023-10-26T01:16:21.835 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/tpm/tpm_tis.ko 2023-10-26T01:16:21.835 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-10-26T01:16:21.835 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/uv_mmtimer.ko 2023-10-26T01:16:21.836 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/virtio_console.ko 2023-10-26T01:16:21.836 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/cpufreq 2023-10-26T01:16:21.836 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-10-26T01:16:21.836 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-10-26T01:16:21.836 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/cpufreq/p4-clockmod.ko 2023-10-26T01:16:21.837 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-10-26T01:16:21.837 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/cpufreq/powernow-k8.ko 2023-10-26T01:16:21.837 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/cpufreq/speedstep-lib.ko 2023-10-26T01:16:21.837 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto 2023-10-26T01:16:21.837 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/ccp 2023-10-26T01:16:21.837 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-10-26T01:16:21.838 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/ccp/ccp.ko 2023-10-26T01:16:21.838 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/intel 2023-10-26T01:16:21.838 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/intel/qat 2023-10-26T01:16:21.838 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/intel/qat/qat_common 2023-10-26T01:16:21.838 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-10-26T01:16:21.839 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-10-26T01:16:21.839 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-10-26T01:16:21.839 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/padlock-aes.ko 2023-10-26T01:16:21.839 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/padlock-sha.ko 2023-10-26T01:16:21.839 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/virtio 2023-10-26T01:16:21.840 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-10-26T01:16:21.840 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/dca 2023-10-26T01:16:21.840 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/dca/dca.ko 2023-10-26T01:16:21.840 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/devfreq 2023-10-26T01:16:21.840 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-10-26T01:16:21.840 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/dma 2023-10-26T01:16:21.841 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/dma/dw 2023-10-26T01:16:21.841 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/dma/dw/dw_dmac.ko 2023-10-26T01:16:21.841 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/dma/ioat 2023-10-26T01:16:21.841 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/dma/ioat/ioatdma.ko 2023-10-26T01:16:21.841 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac 2023-10-26T01:16:21.842 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/amd64_edac.ko 2023-10-26T01:16:21.842 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/e752x_edac.ko 2023-10-26T01:16:21.842 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/edac_mce_amd.ko 2023-10-26T01:16:21.842 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/i3000_edac.ko 2023-10-26T01:16:21.842 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/i3200_edac.ko 2023-10-26T01:16:21.843 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/i5100_edac.ko 2023-10-26T01:16:21.843 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/i5400_edac.ko 2023-10-26T01:16:21.843 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/i7300_edac.ko 2023-10-26T01:16:21.843 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/i7core_edac.ko 2023-10-26T01:16:21.844 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/i82975x_edac.ko 2023-10-26T01:16:21.844 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/ie31200_edac.ko 2023-10-26T01:16:21.844 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/sb_edac.ko 2023-10-26T01:16:21.844 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/x38_edac.ko 2023-10-26T01:16:21.844 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/firmware 2023-10-26T01:16:21.844 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/firmware/edd.ko 2023-10-26T01:16:21.845 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/firmware/iscsi_ibft.ko 2023-10-26T01:16:21.845 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/gpio 2023-10-26T01:16:21.845 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/gpio/gpio-ich.ko 2023-10-26T01:16:21.845 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/gpio/gpio-viperboard.ko 2023-10-26T01:16:21.845 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid 2023-10-26T01:16:21.846 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-appleir.ko 2023-10-26T01:16:21.846 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-aureal.ko 2023-10-26T01:16:21.846 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-axff.ko 2023-10-26T01:16:21.846 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-dr.ko 2023-10-26T01:16:21.846 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-elecom.ko 2023-10-26T01:16:21.847 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-elo.ko 2023-10-26T01:16:21.847 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-emsff.ko 2023-10-26T01:16:21.847 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-gaff.ko 2023-10-26T01:16:21.847 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-gt683r.ko 2023-10-26T01:16:21.847 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-gyration.ko 2023-10-26T01:16:21.847 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-holtek-kbd.ko 2023-10-26T01:16:21.848 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-holtek-mouse.ko 2023-10-26T01:16:21.848 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-holtekff.ko 2023-10-26T01:16:21.848 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-hyperv.ko 2023-10-26T01:16:21.848 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-icade.ko 2023-10-26T01:16:21.848 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-keytouch.ko 2023-10-26T01:16:21.849 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-kye.ko 2023-10-26T01:16:21.849 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-lcpower.ko 2023-10-26T01:16:21.849 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-led.ko 2023-10-26T01:16:21.849 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-lenovo.ko 2023-10-26T01:16:21.849 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-logitech-dj.ko 2023-10-26T01:16:21.849 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-10-26T01:16:21.850 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-multitouch.ko 2023-10-26T01:16:21.850 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-ortek.ko 2023-10-26T01:16:21.850 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-penmount.ko 2023-10-26T01:16:21.850 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-petalynx.ko 2023-10-26T01:16:21.850 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-picolcd.ko 2023-10-26T01:16:21.851 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-pl.ko 2023-10-26T01:16:21.851 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-plantronics.ko 2023-10-26T01:16:21.851 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-primax.ko 2023-10-26T01:16:21.851 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-rmi.ko 2023-10-26T01:16:21.851 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-arvo.ko 2023-10-26T01:16:21.852 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-common.ko 2023-10-26T01:16:21.852 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-isku.ko 2023-10-26T01:16:21.852 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-kone.ko 2023-10-26T01:16:21.852 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-10-26T01:16:21.852 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-konepure.ko 2023-10-26T01:16:21.853 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-10-26T01:16:21.853 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-lua.ko 2023-10-26T01:16:21.853 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-pyra.ko 2023-10-26T01:16:21.853 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-ryos.ko 2023-10-26T01:16:21.853 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-savu.ko 2023-10-26T01:16:21.853 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat.ko 2023-10-26T01:16:21.854 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-saitek.ko 2023-10-26T01:16:21.854 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-samsung.ko 2023-10-26T01:16:21.854 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-sensor-hub.ko 2023-10-26T01:16:21.854 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-sjoy.ko 2023-10-26T01:16:21.854 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-sony.ko 2023-10-26T01:16:21.855 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-speedlink.ko 2023-10-26T01:16:21.855 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-steelseries.ko 2023-10-26T01:16:21.855 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-sunplus.ko 2023-10-26T01:16:21.855 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-thrustmaster.ko 2023-10-26T01:16:21.855 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-tivo.ko 2023-10-26T01:16:21.856 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-tmff.ko 2023-10-26T01:16:21.856 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-topseed.ko 2023-10-26T01:16:21.856 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-twinhan.ko 2023-10-26T01:16:21.856 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-uclogic.ko 2023-10-26T01:16:21.856 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-waltop.ko 2023-10-26T01:16:21.857 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-wiimote.ko 2023-10-26T01:16:21.857 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-xinmo.ko 2023-10-26T01:16:21.857 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-zpff.ko 2023-10-26T01:16:21.857 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-zydacron.ko 2023-10-26T01:16:21.857 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/uhid.ko 2023-10-26T01:16:21.858 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/wacom.ko 2023-10-26T01:16:21.858 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hv 2023-10-26T01:16:21.858 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hv/hv_balloon.ko 2023-10-26T01:16:21.858 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hv/hv_utils.ko 2023-10-26T01:16:21.858 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hv/hv_vmbus.ko 2023-10-26T01:16:21.859 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon 2023-10-26T01:16:21.859 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/abituguru.ko 2023-10-26T01:16:21.859 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/abituguru3.ko 2023-10-26T01:16:21.859 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/acpi_power_meter.ko 2023-10-26T01:16:21.859 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ad7414.ko 2023-10-26T01:16:21.860 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ad7418.ko 2023-10-26T01:16:21.860 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adc128d818.ko 2023-10-26T01:16:21.860 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adm1025.ko 2023-10-26T01:16:21.860 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adm1026.ko 2023-10-26T01:16:21.860 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adm1029.ko 2023-10-26T01:16:21.861 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adm1031.ko 2023-10-26T01:16:21.861 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adm9240.ko 2023-10-26T01:16:21.861 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ads7828.ko 2023-10-26T01:16:21.861 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adt7410.ko 2023-10-26T01:16:21.861 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adt7411.ko 2023-10-26T01:16:21.862 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adt7462.ko 2023-10-26T01:16:21.862 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adt7470.ko 2023-10-26T01:16:21.862 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adt7475.ko 2023-10-26T01:16:21.862 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adt7x10.ko 2023-10-26T01:16:21.862 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/amc6821.ko 2023-10-26T01:16:21.862 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/applesmc.ko 2023-10-26T01:16:21.863 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/asb100.ko 2023-10-26T01:16:21.863 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/asc7621.ko 2023-10-26T01:16:21.863 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/asus_atk0110.ko 2023-10-26T01:16:21.863 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/atxp1.ko 2023-10-26T01:16:21.863 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/coretemp.ko 2023-10-26T01:16:21.864 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-10-26T01:16:21.864 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/dme1737.ko 2023-10-26T01:16:21.864 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ds1621.ko 2023-10-26T01:16:21.864 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ds620.ko 2023-10-26T01:16:21.864 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/emc1403.ko 2023-10-26T01:16:21.864 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/emc6w201.ko 2023-10-26T01:16:21.865 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/f71805f.ko 2023-10-26T01:16:21.865 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/f71882fg.ko 2023-10-26T01:16:21.865 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/f75375s.ko 2023-10-26T01:16:21.865 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/fam15h_power.ko 2023-10-26T01:16:21.865 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/fschmd.ko 2023-10-26T01:16:21.865 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/g760a.ko 2023-10-26T01:16:21.866 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/g762.ko 2023-10-26T01:16:21.866 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/gl518sm.ko 2023-10-26T01:16:21.866 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/gl520sm.ko 2023-10-26T01:16:21.866 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/hwmon-vid.ko 2023-10-26T01:16:21.866 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/i5500_temp.ko 2023-10-26T01:16:21.867 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/i5k_amb.ko 2023-10-26T01:16:21.867 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ibmaem.ko 2023-10-26T01:16:21.867 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ibmpex.ko 2023-10-26T01:16:21.867 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ina209.ko 2023-10-26T01:16:21.867 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ina2xx.ko 2023-10-26T01:16:21.868 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/it87.ko 2023-10-26T01:16:21.868 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/k10temp.ko 2023-10-26T01:16:21.868 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/k8temp.ko 2023-10-26T01:16:21.868 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lineage-pem.ko 2023-10-26T01:16:21.868 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm63.ko 2023-10-26T01:16:21.868 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm73.ko 2023-10-26T01:16:21.869 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm75.ko 2023-10-26T01:16:21.869 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm77.ko 2023-10-26T01:16:21.869 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm78.ko 2023-10-26T01:16:21.869 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm80.ko 2023-10-26T01:16:21.869 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm83.ko 2023-10-26T01:16:21.870 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm85.ko 2023-10-26T01:16:21.870 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm87.ko 2023-10-26T01:16:21.870 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm90.ko 2023-10-26T01:16:21.870 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm92.ko 2023-10-26T01:16:21.870 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm93.ko 2023-10-26T01:16:21.871 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm95234.ko 2023-10-26T01:16:21.871 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm95241.ko 2023-10-26T01:16:21.871 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm95245.ko 2023-10-26T01:16:21.871 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ltc2945.ko 2023-10-26T01:16:21.871 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ltc4151.ko 2023-10-26T01:16:21.872 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ltc4215.ko 2023-10-26T01:16:21.872 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ltc4222.ko 2023-10-26T01:16:21.872 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ltc4245.ko 2023-10-26T01:16:21.872 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ltc4260.ko 2023-10-26T01:16:21.872 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ltc4261.ko 2023-10-26T01:16:21.873 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/max16065.ko 2023-10-26T01:16:21.873 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/max1619.ko 2023-10-26T01:16:21.873 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/max1668.ko 2023-10-26T01:16:21.873 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/max197.ko 2023-10-26T01:16:21.873 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/max6639.ko 2023-10-26T01:16:21.873 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/max6650.ko 2023-10-26T01:16:21.874 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/max6697.ko 2023-10-26T01:16:21.874 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/mcp3021.ko 2023-10-26T01:16:21.874 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/nct6683.ko 2023-10-26T01:16:21.874 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/nct6775-core.ko 2023-10-26T01:16:21.874 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/nct6775.ko 2023-10-26T01:16:21.875 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/nct7802.ko 2023-10-26T01:16:21.875 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pc87360.ko 2023-10-26T01:16:21.875 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pc87427.ko 2023-10-26T01:16:21.875 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pcf8591.ko 2023-10-26T01:16:21.875 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus 2023-10-26T01:16:21.875 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-10-26T01:16:21.876 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-10-26T01:16:21.876 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-10-26T01:16:21.876 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/max16064.ko 2023-10-26T01:16:21.876 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/max34440.ko 2023-10-26T01:16:21.876 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/max8688.ko 2023-10-26T01:16:21.877 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-10-26T01:16:21.877 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-10-26T01:16:21.877 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-10-26T01:16:21.877 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-10-26T01:16:21.877 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-10-26T01:16:21.877 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-10-26T01:16:21.878 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/powr1220.ko 2023-10-26T01:16:21.878 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/sch5627.ko 2023-10-26T01:16:21.878 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/sch5636.ko 2023-10-26T01:16:21.878 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/sch56xx-common.ko 2023-10-26T01:16:21.878 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/sht15.ko 2023-10-26T01:16:21.879 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/sht21.ko 2023-10-26T01:16:21.879 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/shtc1.ko 2023-10-26T01:16:21.879 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/sis5595.ko 2023-10-26T01:16:21.879 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/smsc47b397.ko 2023-10-26T01:16:21.879 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/smsc47m1.ko 2023-10-26T01:16:21.880 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/smsc47m192.ko 2023-10-26T01:16:21.880 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/thmc50.ko 2023-10-26T01:16:21.880 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/tmp102.ko 2023-10-26T01:16:21.880 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/tmp103.ko 2023-10-26T01:16:21.880 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/tmp401.ko 2023-10-26T01:16:21.880 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/tmp421.ko 2023-10-26T01:16:21.881 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/via-cputemp.ko 2023-10-26T01:16:21.881 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/via686a.ko 2023-10-26T01:16:21.881 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/vt1211.ko 2023-10-26T01:16:21.881 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/vt8231.ko 2023-10-26T01:16:21.881 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83627ehf.ko 2023-10-26T01:16:21.882 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83627hf.ko 2023-10-26T01:16:21.882 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83781d.ko 2023-10-26T01:16:21.882 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83791d.ko 2023-10-26T01:16:21.882 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83792d.ko 2023-10-26T01:16:21.882 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83793.ko 2023-10-26T01:16:21.883 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83795.ko 2023-10-26T01:16:21.883 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83l785ts.ko 2023-10-26T01:16:21.883 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83l786ng.ko 2023-10-26T01:16:21.883 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c 2023-10-26T01:16:21.883 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/algos 2023-10-26T01:16:21.883 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-10-26T01:16:21.884 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-10-26T01:16:21.884 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses 2023-10-26T01:16:21.884 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-10-26T01:16:21.884 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-10-26T01:16:21.884 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-10-26T01:16:21.885 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-10-26T01:16:21.885 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-10-26T01:16:21.885 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-10-26T01:16:21.885 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-10-26T01:16:21.885 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-10-26T01:16:21.886 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-i801.ko 2023-10-26T01:16:21.886 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-isch.ko 2023-10-26T01:16:21.886 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-10-26T01:16:21.886 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-10-26T01:16:21.886 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-10-26T01:16:21.887 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-parport.ko 2023-10-26T01:16:21.887 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-10-26T01:16:21.887 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-10-26T01:16:21.887 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-10-26T01:16:21.887 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-10-26T01:16:21.887 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-10-26T01:16:21.888 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-10-26T01:16:21.888 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-via.ko 2023-10-26T01:16:21.888 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-10-26T01:16:21.888 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-10-26T01:16:21.888 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/i2c-dev.ko 2023-10-26T01:16:21.889 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/i2c-mux.ko 2023-10-26T01:16:21.889 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-10-26T01:16:21.889 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/i2c-smbus.ko 2023-10-26T01:16:21.889 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/i2c-stub.ko 2023-10-26T01:16:21.889 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband 2023-10-26T01:16:21.889 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/core 2023-10-26T01:16:21.890 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/core/ib_cm.ko 2023-10-26T01:16:21.890 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/core/ib_core.ko 2023-10-26T01:16:21.890 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/core/ib_umad.ko 2023-10-26T01:16:21.890 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-10-26T01:16:21.890 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/core/iw_cm.ko 2023-10-26T01:16:21.891 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/core/rdma_cm.ko 2023-10-26T01:16:21.891 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-10-26T01:16:21.891 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw 2023-10-26T01:16:21.891 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw/cxgb4 2023-10-26T01:16:21.891 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-10-26T01:16:21.891 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw/mlx4 2023-10-26T01:16:21.892 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-10-26T01:16:21.892 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw/mlx5 2023-10-26T01:16:21.892 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-10-26T01:16:21.892 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw/mthca 2023-10-26T01:16:21.892 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-10-26T01:16:21.893 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp 2023-10-26T01:16:21.893 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/ipoib 2023-10-26T01:16:21.893 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-10-26T01:16:21.893 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/iser 2023-10-26T01:16:21.893 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-10-26T01:16:21.894 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/isert 2023-10-26T01:16:21.894 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-10-26T01:16:21.894 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/srp 2023-10-26T01:16:21.894 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-10-26T01:16:21.894 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/srpt 2023-10-26T01:16:21.895 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-10-26T01:16:21.895 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input 2023-10-26T01:16:21.895 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/gameport 2023-10-26T01:16:21.895 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-10-26T01:16:21.895 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/gameport/fm801-gp.ko 2023-10-26T01:16:21.895 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/gameport/gameport.ko 2023-10-26T01:16:21.896 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/joydev.ko 2023-10-26T01:16:21.896 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/keyboard 2023-10-26T01:16:21.896 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/keyboard/gpio_keys.ko 2023-10-26T01:16:21.896 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse 2023-10-26T01:16:21.896 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse/appletouch.ko 2023-10-26T01:16:21.897 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse/bcm5974.ko 2023-10-26T01:16:21.897 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse/cyapatp.ko 2023-10-26T01:16:21.897 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse/elan_i2c.ko 2023-10-26T01:16:21.897 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse/sermouse.ko 2023-10-26T01:16:21.897 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-10-26T01:16:21.898 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse/synaptics_usb.ko 2023-10-26T01:16:21.898 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse/vsxxxaa.ko 2023-10-26T01:16:21.898 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/rmi4 2023-10-26T01:16:21.898 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/rmi4/rmi_core.ko 2023-10-26T01:16:21.898 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/serio 2023-10-26T01:16:21.899 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/serio/altera_ps2.ko 2023-10-26T01:16:21.899 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/serio/arc_ps2.ko 2023-10-26T01:16:21.899 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-10-26T01:16:21.899 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/serio/serio_raw.ko 2023-10-26T01:16:21.899 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/sparse-keymap.ko 2023-10-26T01:16:21.900 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/iommu 2023-10-26T01:16:21.900 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/iommu/amd 2023-10-26T01:16:21.900 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/iommu/amd/iommu_v2.ko 2023-10-26T01:16:21.900 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds 2023-10-26T01:16:21.900 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/leds-blinkm.ko 2023-10-26T01:16:21.901 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/leds-lm3530.ko 2023-10-26T01:16:21.901 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/leds-lp3944.ko 2023-10-26T01:16:21.901 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/leds-ss4200.ko 2023-10-26T01:16:21.901 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger 2023-10-26T01:16:21.901 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-10-26T01:16:21.901 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-10-26T01:16:21.902 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-10-26T01:16:21.902 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-10-26T01:16:21.902 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-10-26T01:16:21.902 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-10-26T01:16:21.902 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-10-26T01:16:21.903 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-10-26T01:16:21.903 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md 2023-10-26T01:16:21.903 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/bcache 2023-10-26T01:16:21.903 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/bcache/bcache.ko 2023-10-26T01:16:21.903 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-bio-prison.ko 2023-10-26T01:16:21.904 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-cache-smq.ko 2023-10-26T01:16:21.904 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-cache.ko 2023-10-26T01:16:21.904 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-crypt.ko 2023-10-26T01:16:21.904 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-delay.ko 2023-10-26T01:16:21.904 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-flakey.ko 2023-10-26T01:16:21.905 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-log-userspace.ko 2023-10-26T01:16:21.905 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-multipath.ko 2023-10-26T01:16:21.905 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-queue-length.ko 2023-10-26T01:16:21.905 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-raid.ko 2023-10-26T01:16:21.905 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-round-robin.ko 2023-10-26T01:16:21.905 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-service-time.ko 2023-10-26T01:16:21.906 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-switch.ko 2023-10-26T01:16:21.906 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-thin-pool.ko 2023-10-26T01:16:21.906 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-verity.ko 2023-10-26T01:16:21.906 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/faulty.ko 2023-10-26T01:16:21.906 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/linear.ko 2023-10-26T01:16:21.907 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/multipath.ko 2023-10-26T01:16:21.907 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/persistent-data 2023-10-26T01:16:21.907 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-10-26T01:16:21.907 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/raid0.ko 2023-10-26T01:16:21.907 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/raid1.ko 2023-10-26T01:16:21.908 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/raid10.ko 2023-10-26T01:16:21.908 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/raid456.ko 2023-10-26T01:16:21.908 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message 2023-10-26T01:16:21.908 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message/fusion 2023-10-26T01:16:21.908 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message/fusion/mptbase.ko 2023-10-26T01:16:21.909 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message/fusion/mptctl.ko 2023-10-26T01:16:21.909 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message/fusion/mptfc.ko 2023-10-26T01:16:21.909 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message/fusion/mptlan.ko 2023-10-26T01:16:21.909 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message/fusion/mptsas.ko 2023-10-26T01:16:21.909 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message/fusion/mptscsih.ko 2023-10-26T01:16:21.910 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message/fusion/mptspi.ko 2023-10-26T01:16:21.910 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/mfd 2023-10-26T01:16:21.910 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/mfd/lpc_ich.ko 2023-10-26T01:16:21.910 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/mfd/lpc_sch.ko 2023-10-26T01:16:21.910 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/mfd/mfd-core.ko 2023-10-26T01:16:21.911 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/mfd/sm501.ko 2023-10-26T01:16:21.911 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/mfd/viperboard.ko 2023-10-26T01:16:21.911 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/mfd/vx855.ko 2023-10-26T01:16:21.911 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/mfd/wl1273-core.ko 2023-10-26T01:16:21.911 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc 2023-10-26T01:16:21.912 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/altera-stapl 2023-10-26T01:16:21.912 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-10-26T01:16:21.912 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/apds9802als.ko 2023-10-26T01:16:21.912 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/apds990x.ko 2023-10-26T01:16:21.912 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/bh1770glc.ko 2023-10-26T01:16:21.912 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/cb710 2023-10-26T01:16:21.913 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/cb710/cb710.ko 2023-10-26T01:16:21.913 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/eeprom 2023-10-26T01:16:21.913 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/eeprom/at24.ko 2023-10-26T01:16:21.913 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/eeprom/eeprom.ko 2023-10-26T01:16:21.913 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-10-26T01:16:21.914 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/eeprom/max6875.ko 2023-10-26T01:16:21.914 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/enclosure.ko 2023-10-26T01:16:21.914 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/hpilo.ko 2023-10-26T01:16:21.914 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/isl29003.ko 2023-10-26T01:16:21.914 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/isl29020.ko 2023-10-26T01:16:21.915 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/lis3lv02d 2023-10-26T01:16:21.915 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-10-26T01:16:21.915 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-10-26T01:16:21.915 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/mei 2023-10-26T01:16:21.915 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/mei/mei-me.ko 2023-10-26T01:16:21.915 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/mei/mei-txe.ko 2023-10-26T01:16:21.916 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/mei/mei.ko 2023-10-26T01:16:21.916 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/sgi-gru 2023-10-26T01:16:21.916 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/sgi-gru/gru.ko 2023-10-26T01:16:21.916 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/sgi-xp 2023-10-26T01:16:21.916 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/sgi-xp/xp.ko 2023-10-26T01:16:21.917 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/sgi-xp/xpc.ko 2023-10-26T01:16:21.917 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-10-26T01:16:21.917 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/tifm_7xx1.ko 2023-10-26T01:16:21.917 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/tifm_core.ko 2023-10-26T01:16:21.917 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/tsl2550.ko 2023-10-26T01:16:21.918 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/vmw_balloon.ko 2023-10-26T01:16:21.918 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/vmw_vmci 2023-10-26T01:16:21.918 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-10-26T01:16:21.918 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net 2023-10-26T01:16:21.918 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/bonding 2023-10-26T01:16:21.918 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/bonding/bonding.ko 2023-10-26T01:16:21.919 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/dsa 2023-10-26T01:16:21.919 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/dsa/b53 2023-10-26T01:16:21.919 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/dsa/b53/b53_common.ko 2023-10-26T01:16:21.919 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/dsa/bcm-sf2.ko 2023-10-26T01:16:21.919 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/dsa/mv88e6060.ko 2023-10-26T01:16:21.920 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/dsa/mv88e6xxx 2023-10-26T01:16:21.920 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-10-26T01:16:21.920 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/dummy.ko 2023-10-26T01:16:21.920 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/eql.ko 2023-10-26T01:16:21.920 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet 2023-10-26T01:16:21.921 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/3com 2023-10-26T01:16:21.921 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-10-26T01:16:21.921 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-10-26T01:16:21.921 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-10-26T01:16:21.921 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-10-26T01:16:21.921 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/8390 2023-10-26T01:16:21.922 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/8390/8390.ko 2023-10-26T01:16:21.922 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-10-26T01:16:21.922 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-10-26T01:16:21.922 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-10-26T01:16:21.922 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/adaptec 2023-10-26T01:16:21.923 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-10-26T01:16:21.923 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/agere 2023-10-26T01:16:21.923 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/agere/et131x.ko 2023-10-26T01:16:21.923 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/alteon 2023-10-26T01:16:21.923 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-10-26T01:16:21.924 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/altera 2023-10-26T01:16:21.924 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-10-26T01:16:21.924 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/amd 2023-10-26T01:16:21.924 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-10-26T01:16:21.924 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-10-26T01:16:21.925 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-10-26T01:16:21.925 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros 2023-10-26T01:16:21.925 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/alx 2023-10-26T01:16:21.925 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-10-26T01:16:21.925 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/atl1c 2023-10-26T01:16:21.926 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-10-26T01:16:21.926 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/atl1e 2023-10-26T01:16:21.926 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-10-26T01:16:21.926 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/atlx 2023-10-26T01:16:21.926 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-10-26T01:16:21.927 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-10-26T01:16:21.927 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom 2023-10-26T01:16:21.927 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-10-26T01:16:21.927 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-10-26T01:16:21.927 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-10-26T01:16:21.927 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-10-26T01:16:21.928 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-10-26T01:16:21.928 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom/genet 2023-10-26T01:16:21.928 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-10-26T01:16:21.928 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-10-26T01:16:21.928 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/brocade 2023-10-26T01:16:21.929 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/brocade/bna 2023-10-26T01:16:21.929 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-10-26T01:16:21.929 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio 2023-10-26T01:16:21.929 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/cxgb 2023-10-26T01:16:21.929 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-10-26T01:16:21.930 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-10-26T01:16:21.930 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-10-26T01:16:21.930 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-10-26T01:16:21.930 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-10-26T01:16:21.930 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-10-26T01:16:21.931 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-10-26T01:16:21.931 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-10-26T01:16:21.931 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-10-26T01:16:21.931 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/cisco 2023-10-26T01:16:21.931 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/cisco/enic 2023-10-26T01:16:21.931 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-10-26T01:16:21.932 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dec 2023-10-26T01:16:21.932 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dec/tulip 2023-10-26T01:16:21.932 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-10-26T01:16:21.932 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-10-26T01:16:21.932 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-10-26T01:16:21.933 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-10-26T01:16:21.933 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-10-26T01:16:21.933 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-10-26T01:16:21.933 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dlink 2023-10-26T01:16:21.933 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-10-26T01:16:21.934 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-10-26T01:16:21.934 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dnet.ko 2023-10-26T01:16:21.934 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/emulex 2023-10-26T01:16:21.934 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/emulex/benet 2023-10-26T01:16:21.934 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-10-26T01:16:21.935 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/ethoc.ko 2023-10-26T01:16:21.935 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/fealnx.ko 2023-10-26T01:16:21.935 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel 2023-10-26T01:16:21.935 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/e100.ko 2023-10-26T01:16:21.935 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/e1000 2023-10-26T01:16:21.936 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-10-26T01:16:21.936 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/e1000e 2023-10-26T01:16:21.936 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-10-26T01:16:21.936 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/fm10k 2023-10-26T01:16:21.936 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-10-26T01:16:21.937 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/i40e 2023-10-26T01:16:21.937 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-10-26T01:16:21.937 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/iavf 2023-10-26T01:16:21.937 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-10-26T01:16:21.937 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/igb 2023-10-26T01:16:21.937 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-10-26T01:16:21.938 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/igbvf 2023-10-26T01:16:21.938 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-10-26T01:16:21.938 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/ixgbe 2023-10-26T01:16:21.938 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-10-26T01:16:21.938 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/ixgbevf 2023-10-26T01:16:21.939 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-10-26T01:16:21.939 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/jme.ko 2023-10-26T01:16:21.939 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/marvell 2023-10-26T01:16:21.939 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-10-26T01:16:21.939 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/marvell/skge.ko 2023-10-26T01:16:21.940 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-10-26T01:16:21.940 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/mellanox 2023-10-26T01:16:21.940 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/mellanox/mlx4 2023-10-26T01:16:21.940 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-10-26T01:16:21.940 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-10-26T01:16:21.940 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/mellanox/mlx5 2023-10-26T01:16:21.941 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-10-26T01:16:21.941 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-10-26T01:16:21.941 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/micrel 2023-10-26T01:16:21.941 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-10-26T01:16:21.941 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/myricom 2023-10-26T01:16:21.942 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/myricom/myri10ge 2023-10-26T01:16:21.942 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-10-26T01:16:21.942 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/natsemi 2023-10-26T01:16:21.942 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-10-26T01:16:21.942 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-10-26T01:16:21.943 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/neterion 2023-10-26T01:16:21.943 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-10-26T01:16:21.943 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/nvidia 2023-10-26T01:16:21.943 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-10-26T01:16:21.943 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/packetengines 2023-10-26T01:16:21.943 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-10-26T01:16:21.944 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-10-26T01:16:21.944 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/qlogic 2023-10-26T01:16:21.944 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/qlogic/netxen 2023-10-26T01:16:21.944 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-10-26T01:16:21.944 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-10-26T01:16:21.945 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-10-26T01:16:21.945 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-10-26T01:16:21.945 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/rdc 2023-10-26T01:16:21.945 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-10-26T01:16:21.945 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/realtek 2023-10-26T01:16:21.946 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-10-26T01:16:21.946 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-10-26T01:16:21.946 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/realtek/atp.ko 2023-10-26T01:16:21.946 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-10-26T01:16:21.946 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/rocker 2023-10-26T01:16:21.946 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-10-26T01:16:21.947 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sfc 2023-10-26T01:16:21.947 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-10-26T01:16:21.947 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/silan 2023-10-26T01:16:21.947 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-10-26T01:16:21.947 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sis 2023-10-26T01:16:21.948 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sis/sis190.ko 2023-10-26T01:16:21.948 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sis/sis900.ko 2023-10-26T01:16:21.948 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/smsc 2023-10-26T01:16:21.948 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-10-26T01:16:21.948 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-10-26T01:16:21.949 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-10-26T01:16:21.949 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-10-26T01:16:21.949 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/stmicro 2023-10-26T01:16:21.949 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/stmicro/stmmac 2023-10-26T01:16:21.949 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-10-26T01:16:21.949 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-10-26T01:16:21.950 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sun 2023-10-26T01:16:21.950 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sun/cassini.ko 2023-10-26T01:16:21.950 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sun/niu.ko 2023-10-26T01:16:21.950 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sun/sungem.ko 2023-10-26T01:16:21.950 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-10-26T01:16:21.951 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/tehuti 2023-10-26T01:16:21.951 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-10-26T01:16:21.951 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/ti 2023-10-26T01:16:21.951 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/ti/tlan.ko 2023-10-26T01:16:21.951 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/via 2023-10-26T01:16:21.952 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-10-26T01:16:21.952 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-10-26T01:16:21.952 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/wiznet 2023-10-26T01:16:21.952 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-10-26T01:16:21.952 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-10-26T01:16:21.952 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/xircom 2023-10-26T01:16:21.953 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-10-26T01:16:21.953 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/geneve.ko 2023-10-26T01:16:21.953 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/hyperv 2023-10-26T01:16:21.953 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-10-26T01:16:21.953 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ieee802154 2023-10-26T01:16:21.954 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ieee802154/fakelb.ko 2023-10-26T01:16:21.954 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ifb.ko 2023-10-26T01:16:21.954 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ipvlan 2023-10-26T01:16:21.954 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ipvlan/ipvlan.ko 2023-10-26T01:16:21.954 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/macvlan.ko 2023-10-26T01:16:21.955 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/macvtap.ko 2023-10-26T01:16:21.955 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/mdio 2023-10-26T01:16:21.955 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/mdio.ko 2023-10-26T01:16:21.955 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-10-26T01:16:21.955 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-10-26T01:16:21.955 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/mdio/mdio-regmap.ko 2023-10-26T01:16:21.956 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/mii.ko 2023-10-26T01:16:21.956 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/net_failover.ko 2023-10-26T01:16:21.956 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/netconsole.ko 2023-10-26T01:16:21.956 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/nlmon.ko 2023-10-26T01:16:21.956 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/pcs 2023-10-26T01:16:21.957 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/pcs/pcs-lynx.ko 2023-10-26T01:16:21.957 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-10-26T01:16:21.957 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy 2023-10-26T01:16:21.957 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/amd.ko 2023-10-26T01:16:21.957 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-10-26T01:16:21.958 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-10-26T01:16:21.958 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/bcm7xxx.ko 2023-10-26T01:16:21.958 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/bcm87xx.ko 2023-10-26T01:16:21.958 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/broadcom.ko 2023-10-26T01:16:21.958 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/cicada.ko 2023-10-26T01:16:21.958 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/davicom.ko 2023-10-26T01:16:21.959 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/dp83640.ko 2023-10-26T01:16:21.959 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/et1011c.ko 2023-10-26T01:16:21.959 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/icplus.ko 2023-10-26T01:16:21.959 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/lxt.ko 2023-10-26T01:16:21.959 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/marvell.ko 2023-10-26T01:16:21.960 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/micrel.ko 2023-10-26T01:16:21.960 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/national.ko 2023-10-26T01:16:21.960 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/phylink.ko 2023-10-26T01:16:21.960 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/qsemi.ko 2023-10-26T01:16:21.960 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/realtek.ko 2023-10-26T01:16:21.961 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/smsc.ko 2023-10-26T01:16:21.961 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/ste10Xp.ko 2023-10-26T01:16:21.961 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/vitesse.ko 2023-10-26T01:16:21.961 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp 2023-10-26T01:16:21.961 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/bsd_comp.ko 2023-10-26T01:16:21.962 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/ppp_async.ko 2023-10-26T01:16:21.962 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/ppp_deflate.ko 2023-10-26T01:16:21.962 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/ppp_generic.ko 2023-10-26T01:16:21.962 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/ppp_mppe.ko 2023-10-26T01:16:21.962 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/ppp_synctty.ko 2023-10-26T01:16:21.963 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/pppoe.ko 2023-10-26T01:16:21.963 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/pppox.ko 2023-10-26T01:16:21.963 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/pptp.ko 2023-10-26T01:16:21.963 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/slip 2023-10-26T01:16:21.963 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/slip/slhc.ko 2023-10-26T01:16:21.963 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/slip/slip.ko 2023-10-26T01:16:21.964 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/sungem_phy.ko 2023-10-26T01:16:21.964 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/tap.ko 2023-10-26T01:16:21.964 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/team 2023-10-26T01:16:21.964 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/team/team.ko 2023-10-26T01:16:21.964 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/team/team_mode_activebackup.ko 2023-10-26T01:16:21.964 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/team/team_mode_broadcast.ko 2023-10-26T01:16:21.965 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-10-26T01:16:21.965 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/team/team_mode_random.ko 2023-10-26T01:16:21.965 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-10-26T01:16:21.965 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/tun.ko 2023-10-26T01:16:21.965 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/veth.ko 2023-10-26T01:16:21.966 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/virtio_net.ko 2023-10-26T01:16:21.966 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/vmxnet3 2023-10-26T01:16:21.966 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-10-26T01:16:21.966 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/vxlan 2023-10-26T01:16:21.966 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/vxlan/vxlan.ko 2023-10-26T01:16:21.967 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/xen-netback 2023-10-26T01:16:21.967 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/xen-netback/xen-netback.ko 2023-10-26T01:16:21.967 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/xen-netfront.ko 2023-10-26T01:16:21.967 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ntb 2023-10-26T01:16:21.967 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ntb/ntb.ko 2023-10-26T01:16:21.968 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/nvme 2023-10-26T01:16:21.968 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/nvme/host 2023-10-26T01:16:21.968 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/nvme/host/nvme-core.ko 2023-10-26T01:16:21.968 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/nvme/host/nvme.ko 2023-10-26T01:16:21.968 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/parport 2023-10-26T01:16:21.969 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/parport/parport.ko 2023-10-26T01:16:21.969 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/parport/parport_cs.ko 2023-10-26T01:16:21.969 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/parport/parport_pc.ko 2023-10-26T01:16:21.969 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/parport/parport_serial.ko 2023-10-26T01:16:21.969 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pci 2023-10-26T01:16:21.970 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pci/hotplug 2023-10-26T01:16:21.970 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-10-26T01:16:21.970 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pci/pcie 2023-10-26T01:16:21.970 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pci/pcie/aer_inject.ko 2023-10-26T01:16:21.970 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pci/xen-pcifront.ko 2023-10-26T01:16:21.971 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pcmcia 2023-10-26T01:16:21.971 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pcmcia/i82092.ko 2023-10-26T01:16:21.971 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pcmcia/pd6729.ko 2023-10-26T01:16:21.971 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pcmcia/yenta_socket.ko 2023-10-26T01:16:21.971 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pinctrl 2023-10-26T01:16:21.971 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pinctrl/intel 2023-10-26T01:16:21.972 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-10-26T01:16:21.972 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform 2023-10-26T01:16:21.972 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86 2023-10-26T01:16:21.972 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/acer-wmi.ko 2023-10-26T01:16:21.972 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/acerhdf.ko 2023-10-26T01:16:21.973 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/apple-gmux.ko 2023-10-26T01:16:21.973 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/asus-laptop.ko 2023-10-26T01:16:21.973 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-10-26T01:16:21.973 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/asus-wmi.ko 2023-10-26T01:16:21.973 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/classmate-laptop.ko 2023-10-26T01:16:21.974 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-10-26T01:16:21.974 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-10-26T01:16:21.974 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-10-26T01:16:21.974 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-10-26T01:16:21.974 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/hdaps.ko 2023-10-26T01:16:21.974 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/intel 2023-10-26T01:16:21.975 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-10-26T01:16:21.975 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/intel_ips.ko 2023-10-26T01:16:21.975 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/msi-wmi.ko 2023-10-26T01:16:21.975 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/mxm-wmi.ko 2023-10-26T01:16:21.975 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-10-26T01:16:21.976 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/samsung-laptop.ko 2023-10-26T01:16:21.976 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/samsung-q10.ko 2023-10-26T01:16:21.976 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/topstar-laptop.ko 2023-10-26T01:16:21.976 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-10-26T01:16:21.976 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/toshiba_haps.ko 2023-10-26T01:16:21.977 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/wmi-bmof.ko 2023-10-26T01:16:21.977 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/wmi.ko 2023-10-26T01:16:21.977 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/powercap 2023-10-26T01:16:21.977 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/powercap/intel_rapl_common.ko 2023-10-26T01:16:21.977 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/powercap/intel_rapl_msr.ko 2023-10-26T01:16:21.977 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pps 2023-10-26T01:16:21.978 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pps/clients 2023-10-26T01:16:21.978 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pps/clients/pps-gpio.ko 2023-10-26T01:16:21.978 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pps/clients/pps-ldisc.ko 2023-10-26T01:16:21.978 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pps/clients/pps_parport.ko 2023-10-26T01:16:21.978 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pps/pps_core.ko 2023-10-26T01:16:21.979 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ptp 2023-10-26T01:16:21.979 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ptp/ptp.ko 2023-10-26T01:16:21.979 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ptp/ptp_kvm.ko 2023-10-26T01:16:21.979 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc 2023-10-26T01:16:21.979 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-bq32k.ko 2023-10-26T01:16:21.980 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds1286.ko 2023-10-26T01:16:21.980 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds1307.ko 2023-10-26T01:16:21.980 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds1374.ko 2023-10-26T01:16:21.980 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds1511.ko 2023-10-26T01:16:21.980 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds1553.ko 2023-10-26T01:16:21.981 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds1672.ko 2023-10-26T01:16:21.981 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds1742.ko 2023-10-26T01:16:21.981 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds2404.ko 2023-10-26T01:16:21.981 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds3232.ko 2023-10-26T01:16:21.981 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-em3027.ko 2023-10-26T01:16:21.981 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-fm3130.ko 2023-10-26T01:16:21.982 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-isl12022.ko 2023-10-26T01:16:21.982 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-isl1208.ko 2023-10-26T01:16:21.982 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-m41t80.ko 2023-10-26T01:16:21.982 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-m48t35.ko 2023-10-26T01:16:21.982 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-m48t59.ko 2023-10-26T01:16:21.983 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-max6900.ko 2023-10-26T01:16:21.983 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-msm6242.ko 2023-10-26T01:16:21.983 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-pcf2127.ko 2023-10-26T01:16:21.983 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-pcf85063.ko 2023-10-26T01:16:21.983 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-pcf8523.ko 2023-10-26T01:16:21.984 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-pcf8563.ko 2023-10-26T01:16:21.984 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-pcf8583.ko 2023-10-26T01:16:21.984 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-rp5c01.ko 2023-10-26T01:16:21.984 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-rs5c372.ko 2023-10-26T01:16:21.984 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-10-26T01:16:21.985 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-rx8025.ko 2023-10-26T01:16:21.985 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-rx8581.ko 2023-10-26T01:16:21.985 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-10-26T01:16:21.985 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-x1205.ko 2023-10-26T01:16:21.985 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi 2023-10-26T01:16:21.986 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/3w-9xxx.ko 2023-10-26T01:16:21.986 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/3w-sas.ko 2023-10-26T01:16:21.986 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/3w-xxxx.ko 2023-10-26T01:16:21.986 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/BusLogic.ko 2023-10-26T01:16:21.986 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/a100u2w.ko 2023-10-26T01:16:21.987 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/aacraid 2023-10-26T01:16:21.987 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/aacraid/aacraid.ko 2023-10-26T01:16:21.987 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/advansys.ko 2023-10-26T01:16:21.987 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/aic7xxx 2023-10-26T01:16:21.987 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-10-26T01:16:21.988 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-10-26T01:16:21.988 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/am53c974.ko 2023-10-26T01:16:21.988 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/arcmsr 2023-10-26T01:16:21.988 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-10-26T01:16:21.988 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/atp870u.ko 2023-10-26T01:16:21.988 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/be2iscsi 2023-10-26T01:16:21.989 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-10-26T01:16:21.989 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/bfa 2023-10-26T01:16:21.989 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/bfa/bfa.ko 2023-10-26T01:16:21.989 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/bnx2fc 2023-10-26T01:16:21.989 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-10-26T01:16:21.990 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/bnx2i 2023-10-26T01:16:21.990 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-10-26T01:16:21.990 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/ch.ko 2023-10-26T01:16:21.990 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/csiostor 2023-10-26T01:16:21.990 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/csiostor/csiostor.ko 2023-10-26T01:16:21.991 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/cxgbi 2023-10-26T01:16:21.991 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/cxgbi/cxgb3i 2023-10-26T01:16:21.991 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-10-26T01:16:21.991 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/cxgbi/cxgb4i 2023-10-26T01:16:21.991 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-10-26T01:16:21.991 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-10-26T01:16:21.992 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/dc395x.ko 2023-10-26T01:16:21.992 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/device_handler 2023-10-26T01:16:21.992 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-10-26T01:16:21.992 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-10-26T01:16:21.992 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-10-26T01:16:21.993 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-10-26T01:16:21.993 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/dmx3191d.ko 2023-10-26T01:16:21.993 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/esas2r 2023-10-26T01:16:21.993 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/esas2r/esas2r.ko 2023-10-26T01:16:21.993 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/esp_scsi.ko 2023-10-26T01:16:21.994 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/fcoe 2023-10-26T01:16:21.994 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/fcoe/fcoe.ko 2023-10-26T01:16:21.994 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-10-26T01:16:21.994 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/fnic 2023-10-26T01:16:21.994 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/fnic/fnic.ko 2023-10-26T01:16:21.994 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/hpsa.ko 2023-10-26T01:16:21.995 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/hptiop.ko 2023-10-26T01:16:21.995 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/hv_storvsc.ko 2023-10-26T01:16:21.995 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/initio.ko 2023-10-26T01:16:21.995 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/ipr.ko 2023-10-26T01:16:21.995 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/ips.ko 2023-10-26T01:16:21.996 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/isci 2023-10-26T01:16:21.996 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/isci/isci.ko 2023-10-26T01:16:21.996 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-10-26T01:16:21.996 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/iscsi_tcp.ko 2023-10-26T01:16:21.996 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/libfc 2023-10-26T01:16:21.996 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/libfc/libfc.ko 2023-10-26T01:16:21.997 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/libiscsi.ko 2023-10-26T01:16:21.997 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/libiscsi_tcp.ko 2023-10-26T01:16:21.997 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/libsas 2023-10-26T01:16:21.997 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/libsas/libsas.ko 2023-10-26T01:16:21.997 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/lpfc 2023-10-26T01:16:21.998 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/lpfc/lpfc.ko 2023-10-26T01:16:21.998 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/megaraid 2023-10-26T01:16:21.998 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/megaraid.ko 2023-10-26T01:16:21.998 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-10-26T01:16:21.998 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-10-26T01:16:21.999 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-10-26T01:16:21.999 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/mpt3sas 2023-10-26T01:16:21.999 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-10-26T01:16:21.999 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/mvsas 2023-10-26T01:16:21.999 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/mvsas/mvsas.ko 2023-10-26T01:16:22.000 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/mvumi.ko 2023-10-26T01:16:22.000 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/pm8001 2023-10-26T01:16:22.000 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-10-26T01:16:22.000 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/pmcraid.ko 2023-10-26T01:16:22.000 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/qla1280.ko 2023-10-26T01:16:22.000 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/qla2xxx 2023-10-26T01:16:22.001 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-10-26T01:16:22.001 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-10-26T01:16:22.001 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/qla4xxx 2023-10-26T01:16:22.001 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-10-26T01:16:22.001 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/raid_class.ko 2023-10-26T01:16:22.002 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/scsi_debug.ko 2023-10-26T01:16:22.002 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/scsi_transport_fc.ko 2023-10-26T01:16:22.002 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-10-26T01:16:22.002 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/scsi_transport_sas.ko 2023-10-26T01:16:22.003 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/scsi_transport_spi.ko 2023-10-26T01:16:22.003 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/scsi_transport_srp.ko 2023-10-26T01:16:22.003 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/ses.ko 2023-10-26T01:16:22.003 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/st.ko 2023-10-26T01:16:22.003 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/stex.ko 2023-10-26T01:16:22.003 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/sym53c8xx_2 2023-10-26T01:16:22.004 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-10-26T01:16:22.004 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/virtio_scsi.ko 2023-10-26T01:16:22.004 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/vmw_pvscsi.ko 2023-10-26T01:16:22.004 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/wd719x.ko 2023-10-26T01:16:22.004 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ssb 2023-10-26T01:16:22.005 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ssb/ssb.ko 2023-10-26T01:16:22.005 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target 2023-10-26T01:16:22.005 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/iscsi 2023-10-26T01:16:22.005 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-10-26T01:16:22.005 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/loopback 2023-10-26T01:16:22.006 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/loopback/tcm_loop.ko 2023-10-26T01:16:22.006 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/target_core_file.ko 2023-10-26T01:16:22.006 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/target_core_iblock.ko 2023-10-26T01:16:22.006 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/target_core_mod.ko 2023-10-26T01:16:22.006 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/target_core_pscsi.ko 2023-10-26T01:16:22.007 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/target_core_user.ko 2023-10-26T01:16:22.007 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/tcm_fc 2023-10-26T01:16:22.007 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-10-26T01:16:22.007 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal 2023-10-26T01:16:22.007 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel 2023-10-26T01:16:22.008 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal 2023-10-26T01:16:22.008 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-10-26T01:16:22.008 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-10-26T01:16:22.008 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-10-26T01:16:22.008 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-10-26T01:16:22.008 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-10-26T01:16:22.009 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-10-26T01:16:22.009 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-10-26T01:16:22.009 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-10-26T01:16:22.009 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-10-26T01:16:22.009 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-10-26T01:16:22.010 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-10-26T01:16:22.010 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-10-26T01:16:22.010 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-10-26T01:16:22.010 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-10-26T01:16:22.010 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-10-26T01:16:22.011 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty 2023-10-26T01:16:22.011 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/ipwireless 2023-10-26T01:16:22.011 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-10-26T01:16:22.011 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/n_gsm.ko 2023-10-26T01:16:22.011 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/n_hdlc.ko 2023-10-26T01:16:22.011 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/nozomi.ko 2023-10-26T01:16:22.012 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/serial 2023-10-26T01:16:22.012 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/serial/8250 2023-10-26T01:16:22.012 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-10-26T01:16:22.012 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/serial/arc_uart.ko 2023-10-26T01:16:22.012 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/serial/jsm 2023-10-26T01:16:22.013 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/serial/jsm/jsm.ko 2023-10-26T01:16:22.013 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ufs 2023-10-26T01:16:22.013 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ufs/core 2023-10-26T01:16:22.013 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ufs/core/ufshcd-core.ko 2023-10-26T01:16:22.013 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ufs/host 2023-10-26T01:16:22.013 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-10-26T01:16:22.014 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/uio 2023-10-26T01:16:22.014 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/uio/uio.ko 2023-10-26T01:16:22.014 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/uio/uio_aec.ko 2023-10-26T01:16:22.014 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/uio/uio_cif.ko 2023-10-26T01:16:22.014 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/uio/uio_pci_generic.ko 2023-10-26T01:16:22.015 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/uio/uio_sercos3.ko 2023-10-26T01:16:22.015 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb 2023-10-26T01:16:22.015 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/class 2023-10-26T01:16:22.015 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/class/cdc-acm.ko 2023-10-26T01:16:22.015 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/class/cdc-wdm.ko 2023-10-26T01:16:22.016 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/class/usblp.ko 2023-10-26T01:16:22.016 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/class/usbtmc.ko 2023-10-26T01:16:22.016 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/host 2023-10-26T01:16:22.016 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/host/sl811-hcd.ko 2023-10-26T01:16:22.016 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/image 2023-10-26T01:16:22.017 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/image/mdc800.ko 2023-10-26T01:16:22.017 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/image/microtek.ko 2023-10-26T01:16:22.017 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc 2023-10-26T01:16:22.017 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/adutux.ko 2023-10-26T01:16:22.017 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/appledisplay.ko 2023-10-26T01:16:22.018 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/emi26.ko 2023-10-26T01:16:22.018 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/emi62.ko 2023-10-26T01:16:22.018 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/ezusb.ko 2023-10-26T01:16:22.018 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/idmouse.ko 2023-10-26T01:16:22.018 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/iowarrior.ko 2023-10-26T01:16:22.019 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/isight_firmware.ko 2023-10-26T01:16:22.019 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/ldusb.ko 2023-10-26T01:16:22.019 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/legousbtower.ko 2023-10-26T01:16:22.019 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/sisusbvga 2023-10-26T01:16:22.019 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-10-26T01:16:22.020 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/trancevibrator.ko 2023-10-26T01:16:22.020 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/usb3503.ko 2023-10-26T01:16:22.020 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/usblcd.ko 2023-10-26T01:16:22.020 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/usbsevseg.ko 2023-10-26T01:16:22.020 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/uss720.ko 2023-10-26T01:16:22.020 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/yurex.ko 2023-10-26T01:16:22.021 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/phy 2023-10-26T01:16:22.021 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/phy/phy-generic.ko 2023-10-26T01:16:22.021 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial 2023-10-26T01:16:22.021 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/aircable.ko 2023-10-26T01:16:22.021 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/ark3116.ko 2023-10-26T01:16:22.022 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/belkin_sa.ko 2023-10-26T01:16:22.022 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/ch341.ko 2023-10-26T01:16:22.022 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/cp210x.ko 2023-10-26T01:16:22.022 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/cyberjack.ko 2023-10-26T01:16:22.022 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/cypress_m8.ko 2023-10-26T01:16:22.022 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/digi_acceleport.ko 2023-10-26T01:16:22.023 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/empeg.ko 2023-10-26T01:16:22.023 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/ftdi_sio.ko 2023-10-26T01:16:22.023 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/garmin_gps.ko 2023-10-26T01:16:22.023 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/io_edgeport.ko 2023-10-26T01:16:22.023 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/io_ti.ko 2023-10-26T01:16:22.024 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/ipaq.ko 2023-10-26T01:16:22.024 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/ipw.ko 2023-10-26T01:16:22.024 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/ir-usb.ko 2023-10-26T01:16:22.024 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-10-26T01:16:22.024 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/keyspan.ko 2023-10-26T01:16:22.025 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/keyspan_pda.ko 2023-10-26T01:16:22.025 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/kl5kusb105.ko 2023-10-26T01:16:22.025 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/kobil_sct.ko 2023-10-26T01:16:22.025 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/mct_u232.ko 2023-10-26T01:16:22.025 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/mos7720.ko 2023-10-26T01:16:22.025 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/mos7840.ko 2023-10-26T01:16:22.026 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/navman.ko 2023-10-26T01:16:22.026 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/omninet.ko 2023-10-26T01:16:22.026 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/opticon.ko 2023-10-26T01:16:22.026 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/option.ko 2023-10-26T01:16:22.026 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/oti6858.ko 2023-10-26T01:16:22.027 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/pl2303.ko 2023-10-26T01:16:22.027 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/qcaux.ko 2023-10-26T01:16:22.027 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/qcserial.ko 2023-10-26T01:16:22.027 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/quatech2.ko 2023-10-26T01:16:22.027 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/safe_serial.ko 2023-10-26T01:16:22.028 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/sierra.ko 2023-10-26T01:16:22.028 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/spcp8x5.ko 2023-10-26T01:16:22.028 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/ssu100.ko 2023-10-26T01:16:22.028 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/symbolserial.ko 2023-10-26T01:16:22.028 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-10-26T01:16:22.028 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-10-26T01:16:22.029 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/usb_debug.ko 2023-10-26T01:16:22.029 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/usb_wwan.ko 2023-10-26T01:16:22.029 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/visor.ko 2023-10-26T01:16:22.029 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/whiteheat.ko 2023-10-26T01:16:22.029 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/xsens_mt.ko 2023-10-26T01:16:22.030 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage 2023-10-26T01:16:22.030 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/uas.ko 2023-10-26T01:16:22.030 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-alauda.ko 2023-10-26T01:16:22.030 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-cypress.ko 2023-10-26T01:16:22.030 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-datafab.ko 2023-10-26T01:16:22.033 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-10-26T01:16:22.033 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-freecom.ko 2023-10-26T01:16:22.033 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-isd200.ko 2023-10-26T01:16:22.033 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-10-26T01:16:22.034 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-karma.ko 2023-10-26T01:16:22.034 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-onetouch.ko 2023-10-26T01:16:22.034 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-realtek.ko 2023-10-26T01:16:22.034 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-sddr09.ko 2023-10-26T01:16:22.034 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-sddr55.ko 2023-10-26T01:16:22.035 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-usbat.ko 2023-10-26T01:16:22.035 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/usb-storage.ko 2023-10-26T01:16:22.035 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/usbip 2023-10-26T01:16:22.035 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/usbip/usbip-core.ko 2023-10-26T01:16:22.035 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/usbip/usbip-host.ko 2023-10-26T01:16:22.036 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-10-26T01:16:22.036 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vfio 2023-10-26T01:16:22.036 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vfio/pci 2023-10-26T01:16:22.036 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-10-26T01:16:22.036 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vfio/pci/vfio-pci.ko 2023-10-26T01:16:22.036 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vfio/vfio.ko 2023-10-26T01:16:22.037 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-10-26T01:16:22.037 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vhost 2023-10-26T01:16:22.037 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vhost/vhost.ko 2023-10-26T01:16:22.037 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vhost/vhost_iotlb.ko 2023-10-26T01:16:22.038 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vhost/vhost_net.ko 2023-10-26T01:16:22.038 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vhost/vhost_scsi.ko 2023-10-26T01:16:22.038 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/video 2023-10-26T01:16:22.038 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/video/backlight 2023-10-26T01:16:22.038 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/video/backlight/apple_bl.ko 2023-10-26T01:16:22.039 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/video/backlight/lcd.ko 2023-10-26T01:16:22.039 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/video/backlight/platform_lcd.ko 2023-10-26T01:16:22.039 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/virtio 2023-10-26T01:16:22.039 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/virtio/virtio.ko 2023-10-26T01:16:22.039 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/virtio/virtio_balloon.ko 2023-10-26T01:16:22.040 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/virtio/virtio_mmio.ko 2023-10-26T01:16:22.040 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/virtio/virtio_pci.ko 2023-10-26T01:16:22.040 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-10-26T01:16:22.040 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-10-26T01:16:22.040 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/virtio/virtio_ring.ko 2023-10-26T01:16:22.040 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1 2023-10-26T01:16:22.041 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/masters 2023-10-26T01:16:22.041 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/masters/ds2482.ko 2023-10-26T01:16:22.041 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/masters/ds2490.ko 2023-10-26T01:16:22.041 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves 2023-10-26T01:16:22.041 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-10-26T01:16:22.042 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-10-26T01:16:22.042 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-10-26T01:16:22.042 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-10-26T01:16:22.042 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-10-26T01:16:22.042 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-10-26T01:16:22.043 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-10-26T01:16:22.043 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-10-26T01:16:22.043 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-10-26T01:16:22.043 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_smem.ko 2023-10-26T01:16:22.043 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_therm.ko 2023-10-26T01:16:22.044 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/wire.ko 2023-10-26T01:16:22.044 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog 2023-10-26T01:16:22.044 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/alim1535_wdt.ko 2023-10-26T01:16:22.044 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/alim7101_wdt.ko 2023-10-26T01:16:22.044 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/f71808e_wdt.ko 2023-10-26T01:16:22.045 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/hpwdt.ko 2023-10-26T01:16:22.045 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/i6300esb.ko 2023-10-26T01:16:22.045 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-10-26T01:16:22.045 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/iTCO_wdt.ko 2023-10-26T01:16:22.046 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/ib700wdt.ko 2023-10-26T01:16:22.046 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/ibmasr.ko 2023-10-26T01:16:22.046 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-10-26T01:16:22.046 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/it8712f_wdt.ko 2023-10-26T01:16:22.046 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/it87_wdt.ko 2023-10-26T01:16:22.047 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/machzwd.ko 2023-10-26T01:16:22.047 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/nv_tco.ko 2023-10-26T01:16:22.047 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/pcwd_pci.ko 2023-10-26T01:16:22.047 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/pcwd_usb.ko 2023-10-26T01:16:22.047 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-10-26T01:16:22.048 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/sch311x_wdt.ko 2023-10-26T01:16:22.048 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/softdog.ko 2023-10-26T01:16:22.048 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/sp5100_tco.ko 2023-10-26T01:16:22.048 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/via_wdt.ko 2023-10-26T01:16:22.048 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-10-26T01:16:22.049 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/w83877f_wdt.ko 2023-10-26T01:16:22.049 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/w83977f_wdt.ko 2023-10-26T01:16:22.049 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/wdt_pci.ko 2023-10-26T01:16:22.049 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/xen_wdt.ko 2023-10-26T01:16:22.049 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen 2023-10-26T01:16:22.049 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xen-acpi-processor.ko 2023-10-26T01:16:22.050 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xen-evtchn.ko 2023-10-26T01:16:22.050 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xen-gntalloc.ko 2023-10-26T01:16:22.050 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xen-gntdev.ko 2023-10-26T01:16:22.050 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xen-pciback 2023-10-26T01:16:22.050 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-10-26T01:16:22.051 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xen-privcmd.ko 2023-10-26T01:16:22.051 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xenfs 2023-10-26T01:16:22.051 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xenfs/xenfs.ko 2023-10-26T01:16:22.051 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs 2023-10-26T01:16:22.051 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/9p 2023-10-26T01:16:22.052 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/9p/9p.ko 2023-10-26T01:16:22.052 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/affs 2023-10-26T01:16:22.052 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/affs/affs.ko 2023-10-26T01:16:22.052 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/befs 2023-10-26T01:16:22.052 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/befs/befs.ko 2023-10-26T01:16:22.053 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/binfmt_misc.ko 2023-10-26T01:16:22.053 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/btrfs 2023-10-26T01:16:22.053 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/btrfs/btrfs.ko 2023-10-26T01:16:22.053 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/cachefiles 2023-10-26T01:16:22.053 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/cachefiles/cachefiles.ko 2023-10-26T01:16:22.054 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ceph 2023-10-26T01:16:22.054 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ceph/ceph.ko 2023-10-26T01:16:22.054 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/coda 2023-10-26T01:16:22.054 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/coda/coda.ko 2023-10-26T01:16:22.055 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/cramfs 2023-10-26T01:16:22.055 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/cramfs/cramfs.ko 2023-10-26T01:16:22.055 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/dlm 2023-10-26T01:16:22.055 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/dlm/dlm.ko 2023-10-26T01:16:22.055 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ecryptfs 2023-10-26T01:16:22.056 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ecryptfs/ecryptfs.ko 2023-10-26T01:16:22.056 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/f2fs 2023-10-26T01:16:22.056 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/f2fs/f2fs.ko 2023-10-26T01:16:22.056 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fat 2023-10-26T01:16:22.056 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fat/fat.ko 2023-10-26T01:16:22.057 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fat/msdos.ko 2023-10-26T01:16:22.057 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fat/vfat.ko 2023-10-26T01:16:22.057 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fscache 2023-10-26T01:16:22.057 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fscache/fscache.ko 2023-10-26T01:16:22.057 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fuse 2023-10-26T01:16:22.058 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fuse/cuse.ko 2023-10-26T01:16:22.058 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fuse/fuse.ko 2023-10-26T01:16:22.058 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/gfs2 2023-10-26T01:16:22.058 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/gfs2/gfs2.ko 2023-10-26T01:16:22.058 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/hfs 2023-10-26T01:16:22.059 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/hfs/hfs.ko 2023-10-26T01:16:22.059 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/hfsplus 2023-10-26T01:16:22.059 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/hfsplus/hfsplus.ko 2023-10-26T01:16:22.059 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/isofs 2023-10-26T01:16:22.059 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/isofs/isofs.ko 2023-10-26T01:16:22.060 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/jfs 2023-10-26T01:16:22.060 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/jfs/jfs.ko 2023-10-26T01:16:22.060 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/lockd 2023-10-26T01:16:22.060 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/lockd/lockd.ko 2023-10-26T01:16:22.060 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/minix 2023-10-26T01:16:22.061 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/minix/minix.ko 2023-10-26T01:16:22.061 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/netfs 2023-10-26T01:16:22.061 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/netfs/netfs.ko 2023-10-26T01:16:22.061 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs 2023-10-26T01:16:22.062 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/blocklayout 2023-10-26T01:16:22.062 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-10-26T01:16:22.062 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/filelayout 2023-10-26T01:16:22.062 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-10-26T01:16:22.062 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/flexfilelayout 2023-10-26T01:16:22.063 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-10-26T01:16:22.063 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/nfs.ko 2023-10-26T01:16:22.063 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/nfsv3.ko 2023-10-26T01:16:22.063 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/nfsv4.ko 2023-10-26T01:16:22.064 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs_common 2023-10-26T01:16:22.064 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs_common/grace.ko 2023-10-26T01:16:22.064 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs_common/nfs_acl.ko 2023-10-26T01:16:22.064 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfsd 2023-10-26T01:16:22.064 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfsd/nfsd.ko 2023-10-26T01:16:22.064 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nilfs2 2023-10-26T01:16:22.065 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nilfs2/nilfs2.ko 2023-10-26T01:16:22.065 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls 2023-10-26T01:16:22.065 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-celtic.ko 2023-10-26T01:16:22.065 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-centeuro.ko 2023-10-26T01:16:22.066 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-croatian.ko 2023-10-26T01:16:22.066 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-cyrillic.ko 2023-10-26T01:16:22.066 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-gaelic.ko 2023-10-26T01:16:22.066 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-greek.ko 2023-10-26T01:16:22.066 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-iceland.ko 2023-10-26T01:16:22.066 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-inuit.ko 2023-10-26T01:16:22.067 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-roman.ko 2023-10-26T01:16:22.067 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-romanian.ko 2023-10-26T01:16:22.067 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-turkish.ko 2023-10-26T01:16:22.067 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp1250.ko 2023-10-26T01:16:22.067 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp1251.ko 2023-10-26T01:16:22.068 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp1255.ko 2023-10-26T01:16:22.068 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp737.ko 2023-10-26T01:16:22.068 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp775.ko 2023-10-26T01:16:22.068 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp850.ko 2023-10-26T01:16:22.068 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp852.ko 2023-10-26T01:16:22.068 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp855.ko 2023-10-26T01:16:22.069 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp857.ko 2023-10-26T01:16:22.069 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp860.ko 2023-10-26T01:16:22.069 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp861.ko 2023-10-26T01:16:22.069 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp862.ko 2023-10-26T01:16:22.069 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp863.ko 2023-10-26T01:16:22.070 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp864.ko 2023-10-26T01:16:22.070 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp865.ko 2023-10-26T01:16:22.070 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp866.ko 2023-10-26T01:16:22.070 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp869.ko 2023-10-26T01:16:22.070 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp874.ko 2023-10-26T01:16:22.070 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp932.ko 2023-10-26T01:16:22.071 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp936.ko 2023-10-26T01:16:22.071 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp949.ko 2023-10-26T01:16:22.071 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp950.ko 2023-10-26T01:16:22.071 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_euc-jp.ko 2023-10-26T01:16:22.071 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-1.ko 2023-10-26T01:16:22.072 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-13.ko 2023-10-26T01:16:22.072 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-14.ko 2023-10-26T01:16:22.072 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-15.ko 2023-10-26T01:16:22.072 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-2.ko 2023-10-26T01:16:22.072 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-3.ko 2023-10-26T01:16:22.073 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-4.ko 2023-10-26T01:16:22.073 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-5.ko 2023-10-26T01:16:22.073 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-6.ko 2023-10-26T01:16:22.073 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-7.ko 2023-10-26T01:16:22.073 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-9.ko 2023-10-26T01:16:22.074 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_koi8-r.ko 2023-10-26T01:16:22.074 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_koi8-ru.ko 2023-10-26T01:16:22.074 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_koi8-u.ko 2023-10-26T01:16:22.074 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_ucs2_utils.ko 2023-10-26T01:16:22.074 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_utf8.ko 2023-10-26T01:16:22.075 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2 2023-10-26T01:16:22.075 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/cluster 2023-10-26T01:16:22.075 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-10-26T01:16:22.075 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/dlm 2023-10-26T01:16:22.076 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-10-26T01:16:22.076 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/dlmfs 2023-10-26T01:16:22.076 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-10-26T01:16:22.076 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/ocfs2.ko 2023-10-26T01:16:22.076 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-10-26T01:16:22.076 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-10-26T01:16:22.077 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-10-26T01:16:22.077 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/overlayfs 2023-10-26T01:16:22.077 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/overlayfs/overlay.ko 2023-10-26T01:16:22.077 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/pstore 2023-10-26T01:16:22.078 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/pstore/ramoops.ko 2023-10-26T01:16:22.078 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/reiserfs 2023-10-26T01:16:22.078 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/reiserfs/reiserfs.ko 2023-10-26T01:16:22.078 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/romfs 2023-10-26T01:16:22.078 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/romfs/romfs.ko 2023-10-26T01:16:22.079 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/smb 2023-10-26T01:16:22.079 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/smb/client 2023-10-26T01:16:22.079 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/smb/client/cifs.ko 2023-10-26T01:16:22.079 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/smb/common 2023-10-26T01:16:22.079 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/smb/common/cifs_arc4.ko 2023-10-26T01:16:22.079 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/smb/common/cifs_md4.ko 2023-10-26T01:16:22.080 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/squashfs 2023-10-26T01:16:22.080 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/squashfs/squashfs.ko 2023-10-26T01:16:22.080 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/sysv 2023-10-26T01:16:22.080 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/sysv/sysv.ko 2023-10-26T01:16:22.080 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/udf 2023-10-26T01:16:22.081 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/udf/udf.ko 2023-10-26T01:16:22.081 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ufs 2023-10-26T01:16:22.081 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ufs/ufs.ko 2023-10-26T01:16:22.081 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/xfs 2023-10-26T01:16:22.081 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/xfs/xfs.ko 2023-10-26T01:16:22.082 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/kernel 2023-10-26T01:16:22.082 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/kernel/rcu 2023-10-26T01:16:22.082 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/kernel/rcu/rcutorture.ko 2023-10-26T01:16:22.082 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/kernel/torture.ko 2023-10-26T01:16:22.082 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/kernel/trace 2023-10-26T01:16:22.083 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-10-26T01:16:22.083 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib 2023-10-26T01:16:22.083 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/asn1_encoder.ko 2023-10-26T01:16:22.083 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/crc-itu-t.ko 2023-10-26T01:16:22.083 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/crc8.ko 2023-10-26T01:16:22.084 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/crypto 2023-10-26T01:16:22.084 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/crypto/libarc4.ko 2023-10-26T01:16:22.084 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/crypto/libdes.ko 2023-10-26T01:16:22.084 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/libcrc32c.ko 2023-10-26T01:16:22.084 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/lru_cache.ko 2023-10-26T01:16:22.085 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/lz4 2023-10-26T01:16:22.085 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/lz4/lz4_compress.ko 2023-10-26T01:16:22.085 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/lz4/lz4hc_compress.ko 2023-10-26T01:16:22.085 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/math 2023-10-26T01:16:22.085 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/math/cordic.ko 2023-10-26T01:16:22.086 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/raid6 2023-10-26T01:16:22.086 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/raid6/raid6_pq.ko 2023-10-26T01:16:22.086 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/reed_solomon 2023-10-26T01:16:22.086 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/reed_solomon/reed_solomon.ko 2023-10-26T01:16:22.086 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/ts_bm.ko 2023-10-26T01:16:22.087 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/ts_fsm.ko 2023-10-26T01:16:22.087 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/ts_kmp.ko 2023-10-26T01:16:22.087 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/zstd 2023-10-26T01:16:22.087 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/zstd/zstd_compress.ko 2023-10-26T01:16:22.087 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/mm 2023-10-26T01:16:22.088 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/mm/hwpoison-inject.ko 2023-10-26T01:16:22.088 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net 2023-10-26T01:16:22.088 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan 2023-10-26T01:16:22.088 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan/6lowpan.ko 2023-10-26T01:16:22.088 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan/nhc_dest.ko 2023-10-26T01:16:22.088 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan/nhc_fragment.ko 2023-10-26T01:16:22.089 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan/nhc_hop.ko 2023-10-26T01:16:22.089 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan/nhc_ipv6.ko 2023-10-26T01:16:22.089 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan/nhc_mobility.ko 2023-10-26T01:16:22.089 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan/nhc_routing.ko 2023-10-26T01:16:22.090 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan/nhc_udp.ko 2023-10-26T01:16:22.090 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/802 2023-10-26T01:16:22.090 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/802/garp.ko 2023-10-26T01:16:22.090 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/802/mrp.ko 2023-10-26T01:16:22.090 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/802/p8022.ko 2023-10-26T01:16:22.090 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/802/psnap.ko 2023-10-26T01:16:22.091 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/802/stp.ko 2023-10-26T01:16:22.091 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/8021q 2023-10-26T01:16:22.091 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/8021q/8021q.ko 2023-10-26T01:16:22.091 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/9p 2023-10-26T01:16:22.092 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/9p/9pnet.ko 2023-10-26T01:16:22.092 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/9p/9pnet_fd.ko 2023-10-26T01:16:22.092 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/9p/9pnet_rdma.ko 2023-10-26T01:16:22.092 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/9p/9pnet_virtio.ko 2023-10-26T01:16:22.092 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/batman-adv 2023-10-26T01:16:22.092 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/batman-adv/batman-adv.ko 2023-10-26T01:16:22.093 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge 2023-10-26T01:16:22.093 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/br_netfilter.ko 2023-10-26T01:16:22.093 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/bridge.ko 2023-10-26T01:16:22.093 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter 2023-10-26T01:16:22.093 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-10-26T01:16:22.094 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_among.ko 2023-10-26T01:16:22.094 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_arp.ko 2023-10-26T01:16:22.094 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-10-26T01:16:22.094 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-10-26T01:16:22.094 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_ip.ko 2023-10-26T01:16:22.095 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-10-26T01:16:22.095 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_limit.ko 2023-10-26T01:16:22.095 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_log.ko 2023-10-26T01:16:22.095 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_mark.ko 2023-10-26T01:16:22.095 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-10-26T01:16:22.095 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-10-26T01:16:22.096 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-10-26T01:16:22.096 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-10-26T01:16:22.096 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_snat.ko 2023-10-26T01:16:22.096 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_stp.ko 2023-10-26T01:16:22.096 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-10-26T01:16:22.097 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-10-26T01:16:22.097 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-10-26T01:16:22.097 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-10-26T01:16:22.097 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebtables.ko 2023-10-26T01:16:22.097 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-10-26T01:16:22.098 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-10-26T01:16:22.098 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-10-26T01:16:22.098 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ceph 2023-10-26T01:16:22.098 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ceph/libceph.ko 2023-10-26T01:16:22.098 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/core 2023-10-26T01:16:22.099 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/core/failover.ko 2023-10-26T01:16:22.099 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/core/pktgen.ko 2023-10-26T01:16:22.099 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dccp 2023-10-26T01:16:22.099 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dccp/dccp.ko 2023-10-26T01:16:22.099 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dccp/dccp_diag.ko 2023-10-26T01:16:22.099 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dccp/dccp_ipv4.ko 2023-10-26T01:16:22.100 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dccp/dccp_ipv6.ko 2023-10-26T01:16:22.100 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dns_resolver 2023-10-26T01:16:22.100 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dns_resolver/dns_resolver.ko 2023-10-26T01:16:22.100 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dsa 2023-10-26T01:16:22.100 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dsa/dsa_core.ko 2023-10-26T01:16:22.101 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dsa/tag_brcm.ko 2023-10-26T01:16:22.101 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dsa/tag_dsa.ko 2023-10-26T01:16:22.101 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dsa/tag_none.ko 2023-10-26T01:16:22.101 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dsa/tag_trailer.ko 2023-10-26T01:16:22.101 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ieee802154 2023-10-26T01:16:22.102 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ieee802154/6lowpan 2023-10-26T01:16:22.102 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-10-26T01:16:22.102 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ieee802154/ieee802154.ko 2023-10-26T01:16:22.102 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ieee802154/ieee802154_socket.ko 2023-10-26T01:16:22.102 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4 2023-10-26T01:16:22.103 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/ah4.ko 2023-10-26T01:16:22.103 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/esp4.ko 2023-10-26T01:16:22.103 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/fou.ko 2023-10-26T01:16:22.103 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/gre.ko 2023-10-26T01:16:22.103 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/inet_diag.ko 2023-10-26T01:16:22.104 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/ip_gre.ko 2023-10-26T01:16:22.104 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/ip_tunnel.ko 2023-10-26T01:16:22.104 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/ip_vti.ko 2023-10-26T01:16:22.104 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/ipcomp.ko 2023-10-26T01:16:22.104 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/ipip.ko 2023-10-26T01:16:22.104 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter 2023-10-26T01:16:22.105 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/arp_tables.ko 2023-10-26T01:16:22.105 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-10-26T01:16:22.105 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-10-26T01:16:22.105 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-10-26T01:16:22.105 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-10-26T01:16:22.106 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-10-26T01:16:22.106 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-10-26T01:16:22.106 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-10-26T01:16:22.106 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-10-26T01:16:22.106 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-10-26T01:16:22.107 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/iptable_security.ko 2023-10-26T01:16:22.107 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-10-26T01:16:22.107 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-10-26T01:16:22.107 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-10-26T01:16:22.107 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-10-26T01:16:22.108 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-10-26T01:16:22.108 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-10-26T01:16:22.108 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-10-26T01:16:22.108 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-10-26T01:16:22.108 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-10-26T01:16:22.108 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-10-26T01:16:22.109 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_bic.ko 2023-10-26T01:16:22.109 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_dctcp.ko 2023-10-26T01:16:22.109 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_diag.ko 2023-10-26T01:16:22.109 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_highspeed.ko 2023-10-26T01:16:22.109 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_htcp.ko 2023-10-26T01:16:22.110 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_hybla.ko 2023-10-26T01:16:22.110 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_illinois.ko 2023-10-26T01:16:22.110 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_lp.ko 2023-10-26T01:16:22.110 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_scalable.ko 2023-10-26T01:16:22.110 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_vegas.ko 2023-10-26T01:16:22.111 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_veno.ko 2023-10-26T01:16:22.111 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_westwood.ko 2023-10-26T01:16:22.111 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_yeah.ko 2023-10-26T01:16:22.111 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tunnel4.ko 2023-10-26T01:16:22.111 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/udp_diag.ko 2023-10-26T01:16:22.111 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/udp_tunnel.ko 2023-10-26T01:16:22.112 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/xfrm4_tunnel.ko 2023-10-26T01:16:22.112 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6 2023-10-26T01:16:22.112 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/ah6.ko 2023-10-26T01:16:22.112 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/esp6.ko 2023-10-26T01:16:22.112 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/fou6.ko 2023-10-26T01:16:22.113 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/ip6_tunnel.ko 2023-10-26T01:16:22.113 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-10-26T01:16:22.113 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/ip6_vti.ko 2023-10-26T01:16:22.113 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/ipcomp6.ko 2023-10-26T01:16:22.113 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter 2023-10-26T01:16:22.114 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-10-26T01:16:22.114 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-10-26T01:16:22.114 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-10-26T01:16:22.114 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-10-26T01:16:22.114 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-10-26T01:16:22.115 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-10-26T01:16:22.115 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-10-26T01:16:22.115 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-10-26T01:16:22.115 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-10-26T01:16:22.115 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-10-26T01:16:22.115 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-10-26T01:16:22.116 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-10-26T01:16:22.116 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-10-26T01:16:22.116 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-10-26T01:16:22.116 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-10-26T01:16:22.116 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-10-26T01:16:22.117 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-10-26T01:16:22.117 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-10-26T01:16:22.117 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-10-26T01:16:22.117 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-10-26T01:16:22.117 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-10-26T01:16:22.118 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-10-26T01:16:22.118 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-10-26T01:16:22.118 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-10-26T01:16:22.118 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/sit.ko 2023-10-26T01:16:22.118 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/tunnel6.ko 2023-10-26T01:16:22.119 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/xfrm6_tunnel.ko 2023-10-26T01:16:22.119 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/key 2023-10-26T01:16:22.119 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/key/af_key.ko 2023-10-26T01:16:22.119 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/l2tp 2023-10-26T01:16:22.119 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/l2tp/l2tp_core.ko 2023-10-26T01:16:22.120 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/l2tp/l2tp_debugfs.ko 2023-10-26T01:16:22.120 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/l2tp/l2tp_eth.ko 2023-10-26T01:16:22.120 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/l2tp/l2tp_ip.ko 2023-10-26T01:16:22.120 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/l2tp/l2tp_ip6.ko 2023-10-26T01:16:22.120 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/l2tp/l2tp_netlink.ko 2023-10-26T01:16:22.121 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/l2tp/l2tp_ppp.ko 2023-10-26T01:16:22.121 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/llc 2023-10-26T01:16:22.121 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/llc/llc.ko 2023-10-26T01:16:22.121 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/mac802154 2023-10-26T01:16:22.121 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/mac802154/mac802154.ko 2023-10-26T01:16:22.122 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/mpls 2023-10-26T01:16:22.122 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/mpls/mpls_gso.ko 2023-10-26T01:16:22.122 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter 2023-10-26T01:16:22.122 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset 2023-10-26T01:16:22.122 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set.ko 2023-10-26T01:16:22.123 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-10-26T01:16:22.123 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-10-26T01:16:22.123 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-10-26T01:16:22.123 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-10-26T01:16:22.123 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-10-26T01:16:22.124 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-10-26T01:16:22.124 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-10-26T01:16:22.124 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-10-26T01:16:22.124 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-10-26T01:16:22.124 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-10-26T01:16:22.124 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-10-26T01:16:22.125 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-10-26T01:16:22.125 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-10-26T01:16:22.125 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-10-26T01:16:22.125 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-10-26T01:16:22.125 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs 2023-10-26T01:16:22.126 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs.ko 2023-10-26T01:16:22.126 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-10-26T01:16:22.126 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-10-26T01:16:22.126 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-10-26T01:16:22.126 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-10-26T01:16:22.126 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-10-26T01:16:22.127 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-10-26T01:16:22.127 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-10-26T01:16:22.127 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-10-26T01:16:22.127 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-10-26T01:16:22.127 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-10-26T01:16:22.128 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-10-26T01:16:22.128 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-10-26T01:16:22.128 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-10-26T01:16:22.128 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conncount.ko 2023-10-26T01:16:22.128 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack.ko 2023-10-26T01:16:22.129 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-10-26T01:16:22.129 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-10-26T01:16:22.129 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-10-26T01:16:22.129 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_h323.ko 2023-10-26T01:16:22.129 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_irc.ko 2023-10-26T01:16:22.130 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-10-26T01:16:22.130 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-10-26T01:16:22.130 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-10-26T01:16:22.130 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_sane.ko 2023-10-26T01:16:22.130 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_sip.ko 2023-10-26T01:16:22.131 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-10-26T01:16:22.131 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-10-26T01:16:22.131 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_dup_netdev.ko 2023-10-26T01:16:22.131 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_flow_table.ko 2023-10-26T01:16:22.131 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_flow_table_inet.ko 2023-10-26T01:16:22.132 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_log_syslog.ko 2023-10-26T01:16:22.132 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_nat.ko 2023-10-26T01:16:22.132 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_nat_amanda.ko 2023-10-26T01:16:22.132 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_nat_ftp.ko 2023-10-26T01:16:22.132 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_nat_irc.ko 2023-10-26T01:16:22.132 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_nat_sip.ko 2023-10-26T01:16:22.133 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_nat_tftp.ko 2023-10-26T01:16:22.133 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_synproxy_core.ko 2023-10-26T01:16:22.133 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_tables.ko 2023-10-26T01:16:22.133 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nfnetlink.ko 2023-10-26T01:16:22.133 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nfnetlink_acct.ko 2023-10-26T01:16:22.134 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nfnetlink_log.ko 2023-10-26T01:16:22.134 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nfnetlink_osf.ko 2023-10-26T01:16:22.134 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nfnetlink_queue.ko 2023-10-26T01:16:22.134 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_chain_nat.ko 2023-10-26T01:16:22.134 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_compat.ko 2023-10-26T01:16:22.135 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_ct.ko 2023-10-26T01:16:22.135 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_dup_netdev.ko 2023-10-26T01:16:22.135 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_fib.ko 2023-10-26T01:16:22.135 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_fwd_netdev.ko 2023-10-26T01:16:22.135 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_hash.ko 2023-10-26T01:16:22.136 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_limit.ko 2023-10-26T01:16:22.136 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_log.ko 2023-10-26T01:16:22.136 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_masq.ko 2023-10-26T01:16:22.136 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_nat.ko 2023-10-26T01:16:22.136 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_numgen.ko 2023-10-26T01:16:22.136 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_queue.ko 2023-10-26T01:16:22.137 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_quota.ko 2023-10-26T01:16:22.137 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_redir.ko 2023-10-26T01:16:22.137 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_reject.ko 2023-10-26T01:16:22.137 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_reject_inet.ko 2023-10-26T01:16:22.137 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_socket.ko 2023-10-26T01:16:22.138 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_tunnel.ko 2023-10-26T01:16:22.138 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_AUDIT.ko 2023-10-26T01:16:22.138 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_CHECKSUM.ko 2023-10-26T01:16:22.138 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_CLASSIFY.ko 2023-10-26T01:16:22.138 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-10-26T01:16:22.139 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_CT.ko 2023-10-26T01:16:22.139 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_DSCP.ko 2023-10-26T01:16:22.139 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_HL.ko 2023-10-26T01:16:22.139 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_HMARK.ko 2023-10-26T01:16:22.139 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_IDLETIMER.ko 2023-10-26T01:16:22.140 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_LED.ko 2023-10-26T01:16:22.140 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_LOG.ko 2023-10-26T01:16:22.140 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_MASQUERADE.ko 2023-10-26T01:16:22.140 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_NETMAP.ko 2023-10-26T01:16:22.140 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_NFLOG.ko 2023-10-26T01:16:22.141 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_NFQUEUE.ko 2023-10-26T01:16:22.141 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_RATEEST.ko 2023-10-26T01:16:22.141 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_REDIRECT.ko 2023-10-26T01:16:22.141 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_SECMARK.ko 2023-10-26T01:16:22.141 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_TCPMSS.ko 2023-10-26T01:16:22.141 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-10-26T01:16:22.142 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_TEE.ko 2023-10-26T01:16:22.142 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_TPROXY.ko 2023-10-26T01:16:22.142 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_TRACE.ko 2023-10-26T01:16:22.142 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_addrtype.ko 2023-10-26T01:16:22.142 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_bpf.ko 2023-10-26T01:16:22.143 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_cgroup.ko 2023-10-26T01:16:22.143 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_cluster.ko 2023-10-26T01:16:22.143 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_comment.ko 2023-10-26T01:16:22.143 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_connbytes.ko 2023-10-26T01:16:22.143 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_connlabel.ko 2023-10-26T01:16:22.144 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_connlimit.ko 2023-10-26T01:16:22.144 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_connmark.ko 2023-10-26T01:16:22.144 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_conntrack.ko 2023-10-26T01:16:22.144 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_cpu.ko 2023-10-26T01:16:22.144 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_dccp.ko 2023-10-26T01:16:22.145 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_devgroup.ko 2023-10-26T01:16:22.145 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_dscp.ko 2023-10-26T01:16:22.145 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_ecn.ko 2023-10-26T01:16:22.145 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_esp.ko 2023-10-26T01:16:22.145 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_hashlimit.ko 2023-10-26T01:16:22.145 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_helper.ko 2023-10-26T01:16:22.146 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_hl.ko 2023-10-26T01:16:22.146 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_ipcomp.ko 2023-10-26T01:16:22.146 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_iprange.ko 2023-10-26T01:16:22.146 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_ipvs.ko 2023-10-26T01:16:22.146 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_l2tp.ko 2023-10-26T01:16:22.146 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_length.ko 2023-10-26T01:16:22.147 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_limit.ko 2023-10-26T01:16:22.147 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_mac.ko 2023-10-26T01:16:22.147 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_mark.ko 2023-10-26T01:16:22.147 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_multiport.ko 2023-10-26T01:16:22.147 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_nat.ko 2023-10-26T01:16:22.148 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_nfacct.ko 2023-10-26T01:16:22.148 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_osf.ko 2023-10-26T01:16:22.148 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_owner.ko 2023-10-26T01:16:22.148 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_physdev.ko 2023-10-26T01:16:22.148 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_pkttype.ko 2023-10-26T01:16:22.149 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_policy.ko 2023-10-26T01:16:22.149 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_quota.ko 2023-10-26T01:16:22.149 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_rateest.ko 2023-10-26T01:16:22.149 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_realm.ko 2023-10-26T01:16:22.149 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_recent.ko 2023-10-26T01:16:22.149 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_sctp.ko 2023-10-26T01:16:22.150 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_set.ko 2023-10-26T01:16:22.150 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_socket.ko 2023-10-26T01:16:22.150 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_state.ko 2023-10-26T01:16:22.150 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_statistic.ko 2023-10-26T01:16:22.150 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_string.ko 2023-10-26T01:16:22.151 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_tcpmss.ko 2023-10-26T01:16:22.151 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_time.ko 2023-10-26T01:16:22.151 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_u32.ko 2023-10-26T01:16:22.151 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netlink 2023-10-26T01:16:22.151 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netlink/netlink_diag.ko 2023-10-26T01:16:22.151 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/nsh 2023-10-26T01:16:22.152 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/nsh/nsh.ko 2023-10-26T01:16:22.152 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/openvswitch 2023-10-26T01:16:22.152 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/openvswitch/openvswitch.ko 2023-10-26T01:16:22.152 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/openvswitch/vport-geneve.ko 2023-10-26T01:16:22.152 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/openvswitch/vport-gre.ko 2023-10-26T01:16:22.152 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/openvswitch/vport-vxlan.ko 2023-10-26T01:16:22.153 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/packet 2023-10-26T01:16:22.153 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/packet/af_packet_diag.ko 2023-10-26T01:16:22.153 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/rds 2023-10-26T01:16:22.153 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/rds/rds.ko 2023-10-26T01:16:22.153 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/rds/rds_rdma.ko 2023-10-26T01:16:22.153 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/rds/rds_tcp.ko 2023-10-26T01:16:22.154 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched 2023-10-26T01:16:22.154 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_csum.ko 2023-10-26T01:16:22.154 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_gact.ko 2023-10-26T01:16:22.154 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_ipt.ko 2023-10-26T01:16:22.154 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_mirred.ko 2023-10-26T01:16:22.155 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_nat.ko 2023-10-26T01:16:22.155 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_pedit.ko 2023-10-26T01:16:22.155 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_police.ko 2023-10-26T01:16:22.155 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_simple.ko 2023-10-26T01:16:22.155 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_skbedit.ko 2023-10-26T01:16:22.155 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_vlan.ko 2023-10-26T01:16:22.156 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/cls_basic.ko 2023-10-26T01:16:22.156 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/cls_bpf.ko 2023-10-26T01:16:22.156 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/cls_flow.ko 2023-10-26T01:16:22.156 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/cls_fw.ko 2023-10-26T01:16:22.156 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/cls_route.ko 2023-10-26T01:16:22.156 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/cls_u32.ko 2023-10-26T01:16:22.157 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/em_cmp.ko 2023-10-26T01:16:22.157 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/em_ipset.ko 2023-10-26T01:16:22.157 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/em_meta.ko 2023-10-26T01:16:22.157 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/em_nbyte.ko 2023-10-26T01:16:22.157 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/em_text.ko 2023-10-26T01:16:22.157 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/em_u32.ko 2023-10-26T01:16:22.158 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_choke.ko 2023-10-26T01:16:22.158 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_codel.ko 2023-10-26T01:16:22.158 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_drr.ko 2023-10-26T01:16:22.158 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_fq.ko 2023-10-26T01:16:22.158 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_gred.ko 2023-10-26T01:16:22.159 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_hfsc.ko 2023-10-26T01:16:22.159 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_hhf.ko 2023-10-26T01:16:22.159 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_htb.ko 2023-10-26T01:16:22.159 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_ingress.ko 2023-10-26T01:16:22.159 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_mqprio.ko 2023-10-26T01:16:22.159 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_mqprio_lib.ko 2023-10-26T01:16:22.160 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_multiq.ko 2023-10-26T01:16:22.160 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_netem.ko 2023-10-26T01:16:22.160 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_pie.ko 2023-10-26T01:16:22.160 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_plug.ko 2023-10-26T01:16:22.160 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_prio.ko 2023-10-26T01:16:22.161 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_qfq.ko 2023-10-26T01:16:22.161 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_red.ko 2023-10-26T01:16:22.161 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_sfb.ko 2023-10-26T01:16:22.161 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_sfq.ko 2023-10-26T01:16:22.161 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_tbf.ko 2023-10-26T01:16:22.162 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_teql.ko 2023-10-26T01:16:22.162 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sctp 2023-10-26T01:16:22.162 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sctp/sctp.ko 2023-10-26T01:16:22.162 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sctp/sctp_diag.ko 2023-10-26T01:16:22.162 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sunrpc 2023-10-26T01:16:22.162 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sunrpc/auth_gss 2023-10-26T01:16:22.163 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-10-26T01:16:22.163 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-10-26T01:16:22.163 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sunrpc/sunrpc.ko 2023-10-26T01:16:22.163 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sunrpc/xprtrdma 2023-10-26T01:16:22.163 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-10-26T01:16:22.164 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/tipc 2023-10-26T01:16:22.164 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/tipc/diag.ko 2023-10-26T01:16:22.164 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/tipc/tipc.ko 2023-10-26T01:16:22.164 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/unix 2023-10-26T01:16:22.164 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/unix/unix_diag.ko 2023-10-26T01:16:22.164 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/vmw_vsock 2023-10-26T01:16:22.165 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-10-26T01:16:22.165 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-10-26T01:16:22.165 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/vmw_vsock/vsock.ko 2023-10-26T01:16:22.165 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/vmw_vsock/vsock_diag.ko 2023-10-26T01:16:22.165 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/vmw_vsock/vsock_loopback.ko 2023-10-26T01:16:22.166 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/xfrm 2023-10-26T01:16:22.166 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/xfrm/xfrm_ipcomp.ko 2023-10-26T01:16:22.166 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/security 2023-10-26T01:16:22.166 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/security/keys 2023-10-26T01:16:22.166 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/security/keys/encrypted-keys 2023-10-26T01:16:22.167 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-10-26T01:16:22.167 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/security/keys/trusted-keys 2023-10-26T01:16:22.167 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/security/keys/trusted-keys/trusted.ko 2023-10-26T01:16:22.167 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/virt 2023-10-26T01:16:22.167 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/virt/lib 2023-10-26T01:16:22.168 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/virt/lib/irqbypass.ko 2023-10-26T01:16:22.168 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.alias 2023-10-26T01:16:22.168 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.alias.bin 2023-10-26T01:16:22.168 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.builtin 2023-10-26T01:16:22.168 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.builtin.alias.bin 2023-10-26T01:16:22.168 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.builtin.bin 2023-10-26T01:16:22.169 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.builtin.modinfo 2023-10-26T01:16:22.169 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.dep 2023-10-26T01:16:22.169 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.dep.bin 2023-10-26T01:16:22.169 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.devname 2023-10-26T01:16:22.169 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.order 2023-10-26T01:16:22.170 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.softdep 2023-10-26T01:16:22.170 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.symbols 2023-10-26T01:16:22.170 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.symbols.bin 2023-10-26T01:16:22.171 DEBUG:teuthology.task.kernel:get_image_version: 6.6.0-rc5-g5b73517e0e2f 2023-10-26T01:16:22.171 DEBUG:teuthology.orchestra.run.smithi169:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-10-26T01:16:22.194 INFO:teuthology.orchestra.run.smithi169.stdout:postinstall scriptlet (using /bin/sh): 2023-10-26T01:16:22.194 INFO:teuthology.orchestra.run.smithi169.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.6.0-rc5-g5b73517e0e2f -a -r /boot/System.map-6.6.0-rc5-g5b73517e0e2f ]; then 2023-10-26T01:16:22.194 INFO:teuthology.orchestra.run.smithi169.stdout:cp /boot/vmlinuz-6.6.0-rc5-g5b73517e0e2f /boot/.vmlinuz-6.6.0-rc5-g5b73517e0e2f-rpm 2023-10-26T01:16:22.194 INFO:teuthology.orchestra.run.smithi169.stdout:cp /boot/System.map-6.6.0-rc5-g5b73517e0e2f /boot/.System.map-6.6.0-rc5-g5b73517e0e2f-rpm 2023-10-26T01:16:22.195 INFO:teuthology.orchestra.run.smithi169.stdout:rm -f /boot/vmlinuz-6.6.0-rc5-g5b73517e0e2f /boot/System.map-6.6.0-rc5-g5b73517e0e2f 2023-10-26T01:16:22.195 INFO:teuthology.orchestra.run.smithi169.stdout:/sbin/installkernel 6.6.0-rc5-g5b73517e0e2f /boot/.vmlinuz-6.6.0-rc5-g5b73517e0e2f-rpm /boot/.System.map-6.6.0-rc5-g5b73517e0e2f-rpm 2023-10-26T01:16:22.195 INFO:teuthology.orchestra.run.smithi169.stdout:rm -f /boot/.vmlinuz-6.6.0-rc5-g5b73517e0e2f-rpm /boot/.System.map-6.6.0-rc5-g5b73517e0e2f-rpm 2023-10-26T01:16:22.195 INFO:teuthology.orchestra.run.smithi169.stdout:fi 2023-10-26T01:16:22.195 INFO:teuthology.orchestra.run.smithi169.stdout:preuninstall scriptlet (using /bin/sh): 2023-10-26T01:16:22.195 INFO:teuthology.orchestra.run.smithi169.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-10-26T01:16:22.196 INFO:teuthology.orchestra.run.smithi169.stdout:new-kernel-pkg --remove 6.6.0-rc5-g5b73517e0e2f --rminitrd --initrdfile=/boot/initramfs-6.6.0-rc5-g5b73517e0e2f.img 2023-10-26T01:16:22.196 INFO:teuthology.orchestra.run.smithi169.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-10-26T01:16:22.196 INFO:teuthology.orchestra.run.smithi169.stdout:kernel-install remove 6.6.0-rc5-g5b73517e0e2f 2023-10-26T01:16:22.196 INFO:teuthology.orchestra.run.smithi169.stdout:fi 2023-10-26T01:16:22.196 INFO:teuthology.orchestra.run.smithi169.stdout:postuninstall scriptlet (using /bin/sh): 2023-10-26T01:16:22.197 INFO:teuthology.orchestra.run.smithi169.stdout:if [ -x /sbin/update-bootloader ]; then 2023-10-26T01:16:22.197 INFO:teuthology.orchestra.run.smithi169.stdout:/sbin/update-bootloader --remove 6.6.0-rc5-g5b73517e0e2f 2023-10-26T01:16:22.197 INFO:teuthology.orchestra.run.smithi169.stdout:fi 2023-10-26T01:16:22.198 DEBUG:teuthology.orchestra.run.smithi169:> sudo rpm -qi grub2-tools 2023-10-26T01:16:22.274 INFO:teuthology.orchestra.run.smithi169.stdout:Name : grub2-tools 2023-10-26T01:16:22.274 INFO:teuthology.orchestra.run.smithi169.stdout:Epoch : 1 2023-10-26T01:16:22.274 INFO:teuthology.orchestra.run.smithi169.stdout:Version : 2.06 2023-10-26T01:16:22.274 INFO:teuthology.orchestra.run.smithi169.stdout:Release : 68.el9 2023-10-26T01:16:22.275 INFO:teuthology.orchestra.run.smithi169.stdout:Architecture: x86_64 2023-10-26T01:16:22.275 INFO:teuthology.orchestra.run.smithi169.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2023-10-26T01:16:22.275 INFO:teuthology.orchestra.run.smithi169.stdout:Group : Unspecified 2023-10-26T01:16:22.275 INFO:teuthology.orchestra.run.smithi169.stdout:Size : 8264254 2023-10-26T01:16:22.275 INFO:teuthology.orchestra.run.smithi169.stdout:License : GPLv3+ 2023-10-26T01:16:22.276 INFO:teuthology.orchestra.run.smithi169.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2023-10-26T01:16:22.276 INFO:teuthology.orchestra.run.smithi169.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2023-10-26T01:16:22.276 INFO:teuthology.orchestra.run.smithi169.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2023-10-26T01:16:22.276 INFO:teuthology.orchestra.run.smithi169.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2023-10-26T01:16:22.276 INFO:teuthology.orchestra.run.smithi169.stdout:Packager : builder@centos.org 2023-10-26T01:16:22.276 INFO:teuthology.orchestra.run.smithi169.stdout:Vendor : CentOS 2023-10-26T01:16:22.277 INFO:teuthology.orchestra.run.smithi169.stdout:URL : http://www.gnu.org/software/grub/ 2023-10-26T01:16:22.277 INFO:teuthology.orchestra.run.smithi169.stdout:Summary : Support tools for GRUB. 2023-10-26T01:16:22.277 INFO:teuthology.orchestra.run.smithi169.stdout:Description : 2023-10-26T01:16:22.277 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:16:22.277 INFO:teuthology.orchestra.run.smithi169.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-10-26T01:16:22.278 INFO:teuthology.orchestra.run.smithi169.stdout:customizable bootloader with modular architecture. It supports a rich 2023-10-26T01:16:22.278 INFO:teuthology.orchestra.run.smithi169.stdout:variety of kernel formats, file systems, computer architectures and 2023-10-26T01:16:22.278 INFO:teuthology.orchestra.run.smithi169.stdout:hardware devices. 2023-10-26T01:16:22.278 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:16:22.278 INFO:teuthology.orchestra.run.smithi169.stdout:This subpackage provides tools for support of all platforms. 2023-10-26T01:16:22.279 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-10-26T01:16:22.280 INFO:teuthology.task.kernel:Updating grub on smithi169 to boot 6.6.0-rc5-g5b73517e0e2f 2023-10-26T01:16:22.280 DEBUG:teuthology.orchestra.run.smithi169:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-10-26T01:16:22.939 INFO:teuthology.orchestra.run.smithi169.stderr:Generating grub configuration file ... 2023-10-26T01:16:24.256 INFO:teuthology.orchestra.run.smithi169.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2023-10-26T01:16:24.289 INFO:teuthology.orchestra.run.smithi169.stderr:done 2023-10-26T01:16:24.291 DEBUG:teuthology.orchestra.run.smithi169:> mktemp 2023-10-26T01:16:24.306 INFO:teuthology.orchestra.run.smithi169.stdout:/tmp/tmp.qWK8qXAxKa 2023-10-26T01:16:24.307 DEBUG:teuthology.orchestra.run.smithi169:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.qWK8qXAxKa 2023-10-26T01:16:24.385 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod 0666 /tmp/tmp.qWK8qXAxKa 2023-10-26T01:16:24.522 DEBUG:teuthology.orchestra.remote:smithi169:/tmp/tmp.qWK8qXAxKa is 6KB 2023-10-26T01:16:24.570 DEBUG:teuthology.orchestra.run.smithi169:> rm -fr /tmp/tmp.qWK8qXAxKa 2023-10-26T01:16:24.584 DEBUG:teuthology.orchestra.run.smithi169:> sudo /bin/ls /boot/loader/entries || true 2023-10-26T01:16:24.649 INFO:teuthology.orchestra.run.smithi169.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2023-10-26T01:16:24.649 INFO:teuthology.orchestra.run.smithi169.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2023-10-26T01:16:24.649 INFO:teuthology.orchestra.run.smithi169.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2023-10-26T01:16:24.650 INFO:teuthology.orchestra.run.smithi169.stdout:15949e3c88704300882482eddfd7baae-6.6.0-rc5-g5b73517e0e2f.conf 2023-10-26T01:16:24.651 DEBUG:teuthology.orchestra.run.smithi169:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.6.0-rc5-g5b73517e0e2f 2023-10-26T01:16:24.815 DEBUG:teuthology.orchestra.run.smithi169:> sudo shutdown -r now 2023-10-26T01:16:47.945 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2023-10-26T01:16:47.946 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi170.front.sepia.ceph.com' 2023-10-26T01:16:47.947 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-26T01:16:54.822 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2023-10-26T01:16:54.823 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi169.front.sepia.ceph.com' 2023-10-26T01:16:54.823 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-26T01:17:06.302 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.170 2023-10-26T01:17:13.214 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.169 2023-10-26T01:17:15.306 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi170.front.sepia.ceph.com' 2023-10-26T01:17:15.307 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-26T01:17:18.362 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.170 2023-10-26T01:17:22.219 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi169.front.sepia.ceph.com' 2023-10-26T01:17:22.220 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-26T01:17:25.274 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.169 2023-10-26T01:17:30.366 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi170.front.sepia.ceph.com' 2023-10-26T01:17:30.367 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-26T01:17:37.283 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi169.front.sepia.ceph.com' 2023-10-26T01:17:37.284 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-26T01:17:55.710 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.169 2023-10-26T01:18:02.457 DEBUG:teuthology.orchestra.run.smithi170:> true 2023-10-26T01:18:03.071 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi170.front.sepia.ceph.com' 2023-10-26T01:18:03.071 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "6.6.0-rc5-g5b73517e0e2f"... 2023-10-26T01:18:03.072 DEBUG:teuthology.orchestra.run.smithi170:> uname -r 2023-10-26T01:18:03.114 INFO:teuthology.orchestra.run.smithi170.stdout:6.6.0-rc5-g5b73517e0e2f 2023-10-26T01:18:03.115 DEBUG:teuthology.task.kernel:current kernel version is 6.6.0-rc5-g5b73517e0e2f vs 6.6.0-rc5-g5b73517e0e2f 2023-10-26T01:18:03.115 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-10-26T01:18:03.115 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-10-26T01:18:04.116 DEBUG:teuthology.parallel:result is None 2023-10-26T01:18:10.712 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi169.front.sepia.ceph.com' 2023-10-26T01:18:10.714 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-26T01:18:13.786 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.169 2023-10-26T01:18:31.788 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi169.front.sepia.ceph.com' 2023-10-26T01:18:31.789 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-26T01:18:32.242 DEBUG:teuthology.orchestra.run.smithi169:> true 2023-10-26T01:18:32.867 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi169.front.sepia.ceph.com' 2023-10-26T01:18:32.868 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.6.0-rc5-g5b73517e0e2f"... 2023-10-26T01:18:32.868 DEBUG:teuthology.orchestra.run.smithi169:> uname -r 2023-10-26T01:18:32.908 INFO:teuthology.orchestra.run.smithi169.stdout:6.6.0-rc5-g5b73517e0e2f 2023-10-26T01:18:32.909 DEBUG:teuthology.task.kernel:current kernel version is 6.6.0-rc5-g5b73517e0e2f vs 6.6.0-rc5-g5b73517e0e2f 2023-10-26T01:18:32.909 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-10-26T01:18:32.909 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-10-26T01:18:33.910 DEBUG:teuthology.parallel:result is None 2023-10-26T01:18:33.911 INFO:teuthology.run_tasks:Running task internal.base... 2023-10-26T01:18:33.919 INFO:teuthology.task.internal:Creating test directory... 2023-10-26T01:18:33.920 DEBUG:teuthology.orchestra.run.smithi169:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-26T01:18:33.928 DEBUG:teuthology.orchestra.run.smithi170:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-26T01:18:33.991 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-10-26T01:18:33.998 INFO:teuthology.run_tasks:Running task internal.archive... 2023-10-26T01:18:34.022 INFO:teuthology.task.internal:Creating archive directory... 2023-10-26T01:18:34.023 DEBUG:teuthology.orchestra.run.smithi169:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-26T01:18:34.031 DEBUG:teuthology.orchestra.run.smithi170:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-26T01:18:34.110 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-10-26T01:18:34.117 INFO:teuthology.task.internal:Enabling coredump saving... 2023-10-26T01:18:34.117 DEBUG:teuthology.orchestra.run.smithi169:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-26T01:18:34.126 DEBUG:teuthology.orchestra.run.smithi170:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-26T01:18:34.232 INFO:teuthology.orchestra.run.smithi169.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-26T01:18:34.275 INFO:teuthology.orchestra.run.smithi170.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-26T01:18:34.291 INFO:teuthology.orchestra.run.smithi169.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-26T01:18:34.337 INFO:teuthology.orchestra.run.smithi170.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-26T01:18:34.341 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-10-26T01:18:34.349 INFO:teuthology.task.internal:Configuring sudo... 2023-10-26T01:18:34.349 DEBUG:teuthology.orchestra.run.smithi169:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-26T01:18:34.359 DEBUG:teuthology.orchestra.run.smithi170:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-26T01:18:34.463 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-10-26T01:18:34.473 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-10-26T01:18:34.474 DEBUG:teuthology.orchestra.run.smithi169:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-26T01:18:34.481 DEBUG:teuthology.orchestra.run.smithi170:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-26T01:18:34.539 DEBUG:teuthology.orchestra.run.smithi169:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-26T01:18:34.620 DEBUG:teuthology.orchestra.run.smithi169:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-26T01:18:34.723 DEBUG:teuthology.orchestra.run.smithi169:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-26T01:18:34.798 DEBUG:teuthology.orchestra.run.smithi169:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-26T01:18:34.891 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:18:34.891 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-26T01:18:34.972 DEBUG:teuthology.orchestra.run.smithi170:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-26T01:18:35.063 DEBUG:teuthology.orchestra.run.smithi170:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-26T01:18:35.168 DEBUG:teuthology.orchestra.run.smithi170:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-26T01:18:35.256 DEBUG:teuthology.orchestra.run.smithi170:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-26T01:18:35.353 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-26T01:18:35.354 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-26T01:18:35.448 DEBUG:teuthology.orchestra.run.smithi169:> sudo service rsyslog restart 2023-10-26T01:18:35.457 DEBUG:teuthology.orchestra.run.smithi170:> sudo service rsyslog restart 2023-10-26T01:18:35.588 INFO:teuthology.orchestra.run.smithi169.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-26T01:18:35.623 INFO:teuthology.orchestra.run.smithi170.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-26T01:18:35.914 INFO:teuthology.run_tasks:Running task internal.timer... 2023-10-26T01:18:35.923 INFO:teuthology.task.internal:Starting timer... 2023-10-26T01:18:35.923 INFO:teuthology.run_tasks:Running task pcp... 2023-10-26T01:18:35.933 INFO:teuthology.run_tasks:Running task selinux... 2023-10-26T01:18:35.941 DEBUG:teuthology.orchestra.run.smithi169:> sudo service auditd rotate 2023-10-26T01:18:36.069 INFO:teuthology.orchestra.run.smithi169.stdout:Rotating logs: 2023-10-26T01:18:36.073 DEBUG:teuthology.orchestra.run.smithi170:> sudo service auditd rotate 2023-10-26T01:18:36.215 INFO:teuthology.orchestra.run.smithi170.stdout:Rotating logs: 2023-10-26T01:18:36.220 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-10-26T01:18:36.221 DEBUG:teuthology.orchestra.run.smithi169:> /usr/sbin/getenforce 2023-10-26T01:18:36.290 INFO:teuthology.orchestra.run.smithi169.stdout:Permissive 2023-10-26T01:18:36.290 DEBUG:teuthology.orchestra.run.smithi170:> /usr/sbin/getenforce 2023-10-26T01:18:36.356 INFO:teuthology.orchestra.run.smithi170.stdout:Permissive 2023-10-26T01:18:36.356 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi169.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi170.front.sepia.ceph.com': 'permissive'} 2023-10-26T01:18:36.357 DEBUG:teuthology.orchestra.run.smithi169:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-26T01:18:36.459 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-26T01:18:36.460 DEBUG:teuthology.orchestra.run.smithi170:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-26T01:18:36.555 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-26T01:18:36.558 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-10-26T01:18:36.559 DEBUG:teuthology.orchestra.run.smithi169:> sudo /usr/sbin/setenforce permissive 2023-10-26T01:18:36.669 DEBUG:teuthology.orchestra.run.smithi170:> sudo /usr/sbin/setenforce permissive 2023-10-26T01:18:36.794 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-10-26T01:18:36.804 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-10-26T01:18:36.804 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-10-26T01:18:36.824 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-10-26T01:18:36.826 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi169.front.sepia.ceph.com,smithi170.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-10-26T01:28:12.308 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi169.front.sepia.ceph.com'), Remote(name='ubuntu@smithi170.front.sepia.ceph.com')] 2023-10-26T01:28:12.375 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi169.front.sepia.ceph.com' 2023-10-26T01:28:12.376 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-26T01:28:12.518 DEBUG:teuthology.orchestra.run.smithi169:> true 2023-10-26T01:28:12.671 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi169.front.sepia.ceph.com' 2023-10-26T01:28:12.671 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi170.front.sepia.ceph.com' 2023-10-26T01:28:12.672 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-26T01:28:12.808 DEBUG:teuthology.orchestra.run.smithi170:> true 2023-10-26T01:28:12.972 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi170.front.sepia.ceph.com' 2023-10-26T01:28:12.972 INFO:teuthology.run_tasks:Running task clock... 2023-10-26T01:28:12.983 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-10-26T01:28:12.984 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-26T01:28:12.984 DEBUG:teuthology.orchestra.run.smithi169:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-26T01:28:12.993 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-26T01:28:12.993 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-26T01:28:13.090 INFO:teuthology.orchestra.run.smithi169.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-10-26T01:28:13.098 INFO:teuthology.orchestra.run.smithi170.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-10-26T01:28:13.142 INFO:teuthology.orchestra.run.smithi169.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-10-26T01:28:13.152 INFO:teuthology.orchestra.run.smithi170.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-10-26T01:28:13.234 INFO:teuthology.orchestra.run.smithi169.stderr:sudo: ntpd: command not found 2023-10-26T01:28:13.244 INFO:teuthology.orchestra.run.smithi170.stderr:sudo: ntpd: command not found 2023-10-26T01:28:13.277 INFO:teuthology.orchestra.run.smithi169.stdout:506 Cannot talk to daemon 2023-10-26T01:28:13.287 INFO:teuthology.orchestra.run.smithi170.stdout:506 Cannot talk to daemon 2023-10-26T01:28:13.329 INFO:teuthology.orchestra.run.smithi169.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-10-26T01:28:13.336 INFO:teuthology.orchestra.run.smithi170.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-10-26T01:28:13.384 INFO:teuthology.orchestra.run.smithi169.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-10-26T01:28:13.385 INFO:teuthology.orchestra.run.smithi170.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-10-26T01:28:13.506 INFO:teuthology.orchestra.run.smithi169.stderr:bash: line 1: ntpq: command not found 2023-10-26T01:28:13.507 INFO:teuthology.orchestra.run.smithi170.stderr:bash: line 1: ntpq: command not found 2023-10-26T01:28:13.513 INFO:teuthology.orchestra.run.smithi169.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-26T01:28:13.513 INFO:teuthology.orchestra.run.smithi169.stdout:=============================================================================== 2023-10-26T01:28:13.514 INFO:teuthology.orchestra.run.smithi169.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-26T01:28:13.514 INFO:teuthology.orchestra.run.smithi169.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-26T01:28:13.514 INFO:teuthology.orchestra.run.smithi169.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-26T01:28:13.514 INFO:teuthology.orchestra.run.smithi169.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-26T01:28:13.515 INFO:teuthology.orchestra.run.smithi170.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-26T01:28:13.516 INFO:teuthology.orchestra.run.smithi170.stdout:=============================================================================== 2023-10-26T01:28:13.516 INFO:teuthology.orchestra.run.smithi170.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-26T01:28:13.516 INFO:teuthology.orchestra.run.smithi170.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-26T01:28:13.516 INFO:teuthology.orchestra.run.smithi170.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-26T01:28:13.516 INFO:teuthology.orchestra.run.smithi170.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-26T01:28:13.517 INFO:teuthology.run_tasks:Running task install... 2023-10-26T01:28:13.526 DEBUG:teuthology.task.install:project ceph 2023-10-26T01:28:13.527 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '384993681abceb8cb8affea10538d607052db4b4'}} 2023-10-26T01:28:13.527 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '384993681abceb8cb8affea10538d607052db4b4'} 2023-10-26T01:28:13.527 INFO:teuthology.task.install:Using flavor: default 2023-10-26T01:28:13.534 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-10-26T01:28:13.535 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-10-26T01:28:13.535 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '384993681abceb8cb8affea10538d607052db4b4', 'tag': None, 'wait_for_package': False} 2023-10-26T01:28:13.536 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=384993681abceb8cb8affea10538d607052db4b4 2023-10-26T01:28:13.538 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '384993681abceb8cb8affea10538d607052db4b4', 'tag': None, 'wait_for_package': False} 2023-10-26T01:28:13.538 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=384993681abceb8cb8affea10538d607052db4b4 2023-10-26T01:28:13.691 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20231024.144545/384993681abceb8cb8affea10538d607052db4b4/centos/9/flavors/default/ 2023-10-26T01:28:13.691 INFO:teuthology.task.install.rpm:Package version is 18.0.0-6863.g38499368 2023-10-26T01:28:13.766 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20231024.144545/384993681abceb8cb8affea10538d607052db4b4/centos/9/flavors/default/ 2023-10-26T01:28:13.766 INFO:teuthology.task.install.rpm:Package version is 18.0.0-6863.g38499368 2023-10-26T01:28:13.807 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20231024.144545/384993681abceb8cb8affea10538d607052db4b4/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20231024.144545/384993681abceb8cb8affea10538d607052db4b4/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20231024.144545/384993681abceb8cb8affea10538d607052db4b4/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-10-26T01:28:13.807 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-26T01:28:13.807 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-10-26T01:28:13.916 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-10-26T01:28:13.917 DEBUG:teuthology.orchestra.run.smithi170:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/384993681abceb8cb8affea10538d607052db4b4/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-10-26T01:28:13.984 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20231024.144545/384993681abceb8cb8affea10538d607052db4b4/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20231024.144545/384993681abceb8cb8affea10538d607052db4b4/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20231024.144545/384993681abceb8cb8affea10538d607052db4b4/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-10-26T01:28:14.020 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:28:14.021 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-10-26T01:28:14.060 DEBUG:teuthology.orchestra.run.smithi170:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-10-26T01:28:14.122 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-10-26T01:28:14.122 DEBUG:teuthology.orchestra.run.smithi169:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/384993681abceb8cb8affea10538d607052db4b4/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-10-26T01:28:14.189 DEBUG:teuthology.orchestra.run.smithi170:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-10-26T01:28:14.267 DEBUG:teuthology.orchestra.run.smithi169:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-10-26T01:28:14.283 INFO:teuthology.orchestra.run.smithi170.stdout:check_obsoletes = 1 2023-10-26T01:28:14.287 DEBUG:teuthology.orchestra.run.smithi170:> sudo yum clean all 2023-10-26T01:28:14.405 DEBUG:teuthology.orchestra.run.smithi169:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-10-26T01:28:14.495 INFO:teuthology.orchestra.run.smithi169.stdout:check_obsoletes = 1 2023-10-26T01:28:14.499 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum clean all 2023-10-26T01:28:14.651 INFO:teuthology.orchestra.run.smithi170.stdout:72 files removed 2023-10-26T01:28:14.690 DEBUG:teuthology.orchestra.run.smithi170:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-10-26T01:28:14.851 INFO:teuthology.orchestra.run.smithi169.stdout:72 files removed 2023-10-26T01:28:14.890 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-10-26T01:28:15.658 INFO:teuthology.orchestra.run.smithi170.stdout:ceph packages for x86_64 172 kB/s | 78 kB 00:00 2023-10-26T01:28:15.963 INFO:teuthology.orchestra.run.smithi169.stdout:ceph packages for x86_64 140 kB/s | 78 kB 00:00 2023-10-26T01:28:16.248 INFO:teuthology.orchestra.run.smithi170.stdout:ceph noarch packages 19 kB/s | 9.4 kB 00:00 2023-10-26T01:28:16.522 INFO:teuthology.orchestra.run.smithi169.stdout:ceph noarch packages 20 kB/s | 9.4 kB 00:00 2023-10-26T01:28:16.776 INFO:teuthology.orchestra.run.smithi170.stdout:ceph source packages 4.2 kB/s | 1.9 kB 00:00 2023-10-26T01:28:17.031 INFO:teuthology.orchestra.run.smithi169.stdout:ceph source packages 4.3 kB/s | 1.9 kB 00:00 2023-10-26T01:28:17.430 INFO:teuthology.orchestra.run.smithi170.stdout:CentOS-9 - AppStream 11 MB/s | 6.3 MB 00:00 2023-10-26T01:28:17.818 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS-9 - AppStream 8.8 MB/s | 6.3 MB 00:00 2023-10-26T01:28:19.005 INFO:teuthology.orchestra.run.smithi170.stdout:CentOS-9 - BaseOS 6.0 MB/s | 2.0 MB 00:00 2023-10-26T01:28:19.427 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS-9 - BaseOS 5.2 MB/s | 2.0 MB 00:00 2023-10-26T01:28:20.934 INFO:teuthology.orchestra.run.smithi170.stdout:CentOS Stream 9 - BaseOS 5.1 MB/s | 7.9 MB 00:01 2023-10-26T01:28:21.031 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS Stream 9 - BaseOS 6.6 MB/s | 7.9 MB 00:01 2023-10-26T01:28:23.049 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS Stream 9 - AppStream 22 MB/s | 18 MB 00:00 2023-10-26T01:28:24.627 INFO:teuthology.orchestra.run.smithi170.stdout:CentOS Stream 9 - AppStream 7.1 MB/s | 18 MB 00:02 2023-10-26T01:28:27.366 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS Stream 9 - CRB 8.4 MB/s | 5.7 MB 00:00 2023-10-26T01:28:28.787 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS Stream 9 - Extras packages 83 kB/s | 15 kB 00:00 2023-10-26T01:28:29.636 INFO:teuthology.orchestra.run.smithi170.stdout:CentOS Stream 9 - CRB 4.3 MB/s | 5.7 MB 00:01 2023-10-26T01:28:30.031 INFO:teuthology.orchestra.run.smithi169.stdout:Extra Packages for Enterprise Linux 18 MB/s | 19 MB 00:01 2023-10-26T01:28:31.129 INFO:teuthology.orchestra.run.smithi170.stdout:CentOS Stream 9 - Extras packages 65 kB/s | 15 kB 00:00 2023-10-26T01:28:32.930 INFO:teuthology.orchestra.run.smithi170.stdout:Extra Packages for Enterprise Linux 12 MB/s | 19 MB 00:01 2023-10-26T01:28:35.214 INFO:teuthology.orchestra.run.smithi169.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2023-10-26T01:28:37.807 INFO:teuthology.orchestra.run.smithi169.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2023-10-26T01:28:37.809 INFO:teuthology.orchestra.run.smithi169.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2023-10-26T01:28:37.830 INFO:teuthology.orchestra.run.smithi169.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2023-10-26T01:28:37.832 INFO:teuthology.orchestra.run.smithi169.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2023-10-26T01:28:37.833 INFO:teuthology.orchestra.run.smithi169.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2023-10-26T01:28:37.835 INFO:teuthology.orchestra.run.smithi169.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2023-10-26T01:28:37.837 INFO:teuthology.orchestra.run.smithi169.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2023-10-26T01:28:37.841 INFO:teuthology.orchestra.run.smithi169.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2023-10-26T01:28:37.843 INFO:teuthology.orchestra.run.smithi169.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2023-10-26T01:28:37.921 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2023-10-26T01:28:37.934 INFO:teuthology.orchestra.run.smithi169.stdout:============================================================================================ 2023-10-26T01:28:37.935 INFO:teuthology.orchestra.run.smithi169.stdout: Package Arch Version Repository Size 2023-10-26T01:28:37.935 INFO:teuthology.orchestra.run.smithi169.stdout:============================================================================================ 2023-10-26T01:28:37.935 INFO:teuthology.orchestra.run.smithi169.stdout:Installing: 2023-10-26T01:28:37.935 INFO:teuthology.orchestra.run.smithi169.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2023-10-26T01:28:37.936 INFO:teuthology.orchestra.run.smithi169.stdout: ceph x86_64 2:18.0.0-6863.g38499368.el9 ceph 6.5 k 2023-10-26T01:28:37.936 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-base x86_64 2:18.0.0-6863.g38499368.el9 ceph 5.2 M 2023-10-26T01:28:37.936 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-fuse x86_64 2:18.0.0-6863.g38499368.el9 ceph 856 k 2023-10-26T01:28:37.936 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-6863.g38499368.el9 ceph 142 k 2023-10-26T01:28:37.936 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr x86_64 2:18.0.0-6863.g38499368.el9 ceph 1.5 M 2023-10-26T01:28:37.937 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-cephadm noarch 2:18.0.0-6863.g38499368.el9 ceph-noarch 132 k 2023-10-26T01:28:37.937 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-dashboard noarch 2:18.0.0-6863.g38499368.el9 ceph-noarch 1.8 M 2023-10-26T01:28:37.937 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-6863.g38499368.el9 ceph-noarch 7.4 M 2023-10-26T01:28:37.937 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-rook noarch 2:18.0.0-6863.g38499368.el9 ceph-noarch 48 k 2023-10-26T01:28:37.938 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-radosgw x86_64 2:18.0.0-6863.g38499368.el9 ceph 8.6 M 2023-10-26T01:28:37.938 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-test x86_64 2:18.0.0-6863.g38499368.el9 ceph 41 M 2023-10-26T01:28:37.938 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-volume noarch 2:18.0.0-6863.g38499368.el9 ceph-noarch 262 k 2023-10-26T01:28:37.938 INFO:teuthology.orchestra.run.smithi169.stdout: cephadm noarch 2:18.0.0-6863.g38499368.el9 ceph-noarch 270 k 2023-10-26T01:28:37.938 INFO:teuthology.orchestra.run.smithi169.stdout: cephfs-mirror x86_64 2:18.0.0-6863.g38499368.el9 ceph 218 k 2023-10-26T01:28:37.939 INFO:teuthology.orchestra.run.smithi169.stdout: cephfs-top noarch 2:18.0.0-6863.g38499368.el9 ceph-noarch 18 k 2023-10-26T01:28:37.939 INFO:teuthology.orchestra.run.smithi169.stdout: elfutils-libelf-devel x86_64 0.189-3.el9 appstream 22 k 2023-10-26T01:28:37.939 INFO:teuthology.orchestra.run.smithi169.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2023-10-26T01:28:37.939 INFO:teuthology.orchestra.run.smithi169.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2023-10-26T01:28:37.939 INFO:teuthology.orchestra.run.smithi169.stdout: libcephfs-devel x86_64 2:18.0.0-6863.g38499368.el9 ceph 31 k 2023-10-26T01:28:37.940 INFO:teuthology.orchestra.run.smithi169.stdout: libcephfs2 x86_64 2:18.0.0-6863.g38499368.el9 ceph 718 k 2023-10-26T01:28:37.940 INFO:teuthology.orchestra.run.smithi169.stdout: librados-devel x86_64 2:18.0.0-6863.g38499368.el9 ceph 127 k 2023-10-26T01:28:37.940 INFO:teuthology.orchestra.run.smithi169.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2023-10-26T01:28:37.940 INFO:teuthology.orchestra.run.smithi169.stdout: libuuid-devel x86_64 2.37.4-15.el9 appstream 26 k 2023-10-26T01:28:37.941 INFO:teuthology.orchestra.run.smithi169.stdout: openssl-devel x86_64 1:3.0.7-24.el9 appstream 4.1 M 2023-10-26T01:28:37.941 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2023-10-26T01:28:37.941 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2023-10-26T01:28:37.942 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2023-10-26T01:28:37.942 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cephfs x86_64 2:18.0.0-6863.g38499368.el9 ceph 162 k 2023-10-26T01:28:37.942 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rados x86_64 2:18.0.0-6863.g38499368.el9 ceph 321 k 2023-10-26T01:28:37.942 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rbd x86_64 2:18.0.0-6863.g38499368.el9 ceph 298 k 2023-10-26T01:28:37.942 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rgw x86_64 2:18.0.0-6863.g38499368.el9 ceph 99 k 2023-10-26T01:28:37.943 INFO:teuthology.orchestra.run.smithi169.stdout: rbd-fuse x86_64 2:18.0.0-6863.g38499368.el9 ceph 86 k 2023-10-26T01:28:37.943 INFO:teuthology.orchestra.run.smithi169.stdout: rbd-mirror x86_64 2:18.0.0-6863.g38499368.el9 ceph 3.0 M 2023-10-26T01:28:37.943 INFO:teuthology.orchestra.run.smithi169.stdout: rbd-nbd x86_64 2:18.0.0-6863.g38499368.el9 ceph 170 k 2023-10-26T01:28:37.943 INFO:teuthology.orchestra.run.smithi169.stdout: xfsprogs-devel x86_64 5.19.0-4.el9 appstream 86 k 2023-10-26T01:28:37.944 INFO:teuthology.orchestra.run.smithi169.stdout:Upgrading: 2023-10-26T01:28:37.944 INFO:teuthology.orchestra.run.smithi169.stdout: NetworkManager x86_64 1:1.45.5-1.el9 CentOS-BaseOS 2.2 M 2023-10-26T01:28:37.944 INFO:teuthology.orchestra.run.smithi169.stdout: NetworkManager-libnm x86_64 1:1.45.5-1.el9 CentOS-BaseOS 1.8 M 2023-10-26T01:28:37.944 INFO:teuthology.orchestra.run.smithi169.stdout: NetworkManager-team x86_64 1:1.45.5-1.el9 CentOS-BaseOS 42 k 2023-10-26T01:28:37.944 INFO:teuthology.orchestra.run.smithi169.stdout: NetworkManager-tui x86_64 1:1.45.5-1.el9 CentOS-BaseOS 245 k 2023-10-26T01:28:37.945 INFO:teuthology.orchestra.run.smithi169.stdout: acl x86_64 2.3.1-4.el9 CentOS-BaseOS 69 k 2023-10-26T01:28:37.945 INFO:teuthology.orchestra.run.smithi169.stdout: libacl x86_64 2.3.1-4.el9 CentOS-BaseOS 22 k 2023-10-26T01:28:37.945 INFO:teuthology.orchestra.run.smithi169.stdout: libacl-devel x86_64 2.3.1-4.el9 CentOS-AppStream 68 k 2023-10-26T01:28:37.945 INFO:teuthology.orchestra.run.smithi169.stdout: librados2 x86_64 2:18.0.0-6863.g38499368.el9 ceph 3.3 M 2023-10-26T01:28:37.945 INFO:teuthology.orchestra.run.smithi169.stdout: librbd1 x86_64 2:18.0.0-6863.g38499368.el9 ceph 3.0 M 2023-10-26T01:28:37.946 INFO:teuthology.orchestra.run.smithi169.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2023-10-26T01:28:37.946 INFO:teuthology.orchestra.run.smithi169.stdout:Installing dependencies: 2023-10-26T01:28:37.946 INFO:teuthology.orchestra.run.smithi169.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2023-10-26T01:28:37.946 INFO:teuthology.orchestra.run.smithi169.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2023-10-26T01:28:37.946 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-common x86_64 2:18.0.0-6863.g38499368.el9 ceph 19 M 2023-10-26T01:28:37.947 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-grafana-dashboards noarch 2:18.0.0-6863.g38499368.el9 ceph-noarch 23 k 2023-10-26T01:28:37.947 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mds x86_64 2:18.0.0-6863.g38499368.el9 ceph 2.1 M 2023-10-26T01:28:37.947 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-modules-core noarch 2:18.0.0-6863.g38499368.el9 ceph-noarch 243 k 2023-10-26T01:28:37.947 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mon x86_64 2:18.0.0-6863.g38499368.el9 ceph 4.4 M 2023-10-26T01:28:37.947 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-osd x86_64 2:18.0.0-6863.g38499368.el9 ceph 18 M 2023-10-26T01:28:37.947 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-prometheus-alerts noarch 2:18.0.0-6863.g38499368.el9 ceph-noarch 15 k 2023-10-26T01:28:37.948 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-selinux x86_64 2:18.0.0-6863.g38499368.el9 ceph 24 k 2023-10-26T01:28:37.948 INFO:teuthology.orchestra.run.smithi169.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2023-10-26T01:28:37.948 INFO:teuthology.orchestra.run.smithi169.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2023-10-26T01:28:37.948 INFO:teuthology.orchestra.run.smithi169.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2023-10-26T01:28:37.948 INFO:teuthology.orchestra.run.smithi169.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2023-10-26T01:28:37.949 INFO:teuthology.orchestra.run.smithi169.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2023-10-26T01:28:37.949 INFO:teuthology.orchestra.run.smithi169.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2023-10-26T01:28:37.949 INFO:teuthology.orchestra.run.smithi169.stdout: jq x86_64 1.6-15.el9 appstream 187 k 2023-10-26T01:28:37.949 INFO:teuthology.orchestra.run.smithi169.stdout: libarrow x86_64 9.0.0-9.el9 epel 4.4 M 2023-10-26T01:28:37.950 INFO:teuthology.orchestra.run.smithi169.stdout: libarrow-doc noarch 9.0.0-9.el9 epel 24 k 2023-10-26T01:28:37.950 INFO:teuthology.orchestra.run.smithi169.stdout: libcephsqlite x86_64 2:18.0.0-6863.g38499368.el9 ceph 165 k 2023-10-26T01:28:37.950 INFO:teuthology.orchestra.run.smithi169.stdout: libgfortran x86_64 11.4.1-2.3.el9 CentOS-BaseOS 803 k 2023-10-26T01:28:37.950 INFO:teuthology.orchestra.run.smithi169.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2023-10-26T01:28:37.950 INFO:teuthology.orchestra.run.smithi169.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2023-10-26T01:28:37.951 INFO:teuthology.orchestra.run.smithi169.stdout: libquadmath x86_64 11.4.1-2.3.el9 CentOS-BaseOS 195 k 2023-10-26T01:28:37.951 INFO:teuthology.orchestra.run.smithi169.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2023-10-26T01:28:37.951 INFO:teuthology.orchestra.run.smithi169.stdout: libradosstriper1 x86_64 2:18.0.0-6863.g38499368.el9 ceph 473 k 2023-10-26T01:28:37.951 INFO:teuthology.orchestra.run.smithi169.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2023-10-26T01:28:37.951 INFO:teuthology.orchestra.run.smithi169.stdout: librgw2 x86_64 2:18.0.0-6863.g38499368.el9 ceph 4.8 M 2023-10-26T01:28:37.952 INFO:teuthology.orchestra.run.smithi169.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2023-10-26T01:28:37.952 INFO:teuthology.orchestra.run.smithi169.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2023-10-26T01:28:37.952 INFO:teuthology.orchestra.run.smithi169.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2023-10-26T01:28:37.952 INFO:teuthology.orchestra.run.smithi169.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2023-10-26T01:28:37.952 INFO:teuthology.orchestra.run.smithi169.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2023-10-26T01:28:37.953 INFO:teuthology.orchestra.run.smithi169.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2023-10-26T01:28:37.953 INFO:teuthology.orchestra.run.smithi169.stdout: parquet-libs x86_64 9.0.0-9.el9 epel 837 k 2023-10-26T01:28:37.953 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2023-10-26T01:28:37.953 INFO:teuthology.orchestra.run.smithi169.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2023-10-26T01:28:37.953 INFO:teuthology.orchestra.run.smithi169.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2023-10-26T01:28:37.954 INFO:teuthology.orchestra.run.smithi169.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2023-10-26T01:28:37.954 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2023-10-26T01:28:37.954 INFO:teuthology.orchestra.run.smithi169.stdout: python3-ceph-argparse x86_64 2:18.0.0-6863.g38499368.el9 ceph 45 k 2023-10-26T01:28:37.954 INFO:teuthology.orchestra.run.smithi169.stdout: python3-ceph-common x86_64 2:18.0.0-6863.g38499368.el9 ceph 127 k 2023-10-26T01:28:37.955 INFO:teuthology.orchestra.run.smithi169.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2023-10-26T01:28:37.955 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2023-10-26T01:28:37.955 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2023-10-26T01:28:37.955 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2023-10-26T01:28:37.955 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2023-10-26T01:28:37.956 INFO:teuthology.orchestra.run.smithi169.stdout: python3-devel x86_64 3.9.17-2.el9 CentOS-AppStream 205 k 2023-10-26T01:28:37.956 INFO:teuthology.orchestra.run.smithi169.stdout: python3-google-auth noarch 1:2.23.3-1.el9 epel 197 k 2023-10-26T01:28:37.956 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2023-10-26T01:28:37.956 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2023-10-26T01:28:37.956 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2023-10-26T01:28:37.957 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2023-10-26T01:28:37.957 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2023-10-26T01:28:37.957 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jinja2 noarch 2.11.3-4.el9 CentOS-AppStream 228 k 2023-10-26T01:28:37.957 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2023-10-26T01:28:37.957 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2023-10-26T01:28:37.958 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2023-10-26T01:28:37.958 INFO:teuthology.orchestra.run.smithi169.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2023-10-26T01:28:37.958 INFO:teuthology.orchestra.run.smithi169.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2023-10-26T01:28:37.958 INFO:teuthology.orchestra.run.smithi169.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2023-10-26T01:28:37.958 INFO:teuthology.orchestra.run.smithi169.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2023-10-26T01:28:37.959 INFO:teuthology.orchestra.run.smithi169.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2023-10-26T01:28:37.959 INFO:teuthology.orchestra.run.smithi169.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2023-10-26T01:28:37.959 INFO:teuthology.orchestra.run.smithi169.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2023-10-26T01:28:37.959 INFO:teuthology.orchestra.run.smithi169.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2023-10-26T01:28:37.959 INFO:teuthology.orchestra.run.smithi169.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2023-10-26T01:28:37.960 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2023-10-26T01:28:37.960 INFO:teuthology.orchestra.run.smithi169.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2023-10-26T01:28:37.960 INFO:teuthology.orchestra.run.smithi169.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2023-10-26T01:28:37.960 INFO:teuthology.orchestra.run.smithi169.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2023-10-26T01:28:37.960 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2023-10-26T01:28:37.961 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2023-10-26T01:28:37.961 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2023-10-26T01:28:37.961 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2023-10-26T01:28:37.962 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2023-10-26T01:28:37.962 INFO:teuthology.orchestra.run.smithi169.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2023-10-26T01:28:37.962 INFO:teuthology.orchestra.run.smithi169.stdout: python3-requests noarch 2.25.1-7.el9 CentOS-BaseOS 113 k 2023-10-26T01:28:37.962 INFO:teuthology.orchestra.run.smithi169.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2023-10-26T01:28:37.962 INFO:teuthology.orchestra.run.smithi169.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2023-10-26T01:28:37.963 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2023-10-26T01:28:37.963 INFO:teuthology.orchestra.run.smithi169.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2023-10-26T01:28:37.963 INFO:teuthology.orchestra.run.smithi169.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2023-10-26T01:28:37.963 INFO:teuthology.orchestra.run.smithi169.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2023-10-26T01:28:37.963 INFO:teuthology.orchestra.run.smithi169.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2023-10-26T01:28:37.963 INFO:teuthology.orchestra.run.smithi169.stdout: python3-urllib3 noarch 1.26.5-4.el9 CentOS-BaseOS 187 k 2023-10-26T01:28:37.964 INFO:teuthology.orchestra.run.smithi169.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2023-10-26T01:28:37.964 INFO:teuthology.orchestra.run.smithi169.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2023-10-26T01:28:37.964 INFO:teuthology.orchestra.run.smithi169.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2023-10-26T01:28:37.964 INFO:teuthology.orchestra.run.smithi169.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2023-10-26T01:28:37.964 INFO:teuthology.orchestra.run.smithi169.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2023-10-26T01:28:37.965 INFO:teuthology.orchestra.run.smithi169.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2023-10-26T01:28:37.965 INFO:teuthology.orchestra.run.smithi169.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2023-10-26T01:28:37.965 INFO:teuthology.orchestra.run.smithi169.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2023-10-26T01:28:37.965 INFO:teuthology.orchestra.run.smithi169.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2023-10-26T01:28:37.965 INFO:teuthology.orchestra.run.smithi169.stdout: zlib-devel x86_64 1.2.11-40.el9 appstream 45 k 2023-10-26T01:28:37.966 INFO:teuthology.orchestra.run.smithi169.stdout:Installing weak dependencies: 2023-10-26T01:28:37.966 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2023-10-26T01:28:37.966 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:28:37.966 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction Summary 2023-10-26T01:28:37.967 INFO:teuthology.orchestra.run.smithi169.stdout:============================================================================================ 2023-10-26T01:28:37.967 INFO:teuthology.orchestra.run.smithi169.stdout:Install 133 Packages 2023-10-26T01:28:37.967 INFO:teuthology.orchestra.run.smithi169.stdout:Upgrade 10 Packages 2023-10-26T01:28:37.967 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:28:37.968 INFO:teuthology.orchestra.run.smithi169.stdout:Total download size: 200 M 2023-10-26T01:28:37.968 INFO:teuthology.orchestra.run.smithi169.stdout:Downloading Packages: 2023-10-26T01:28:38.180 INFO:teuthology.orchestra.run.smithi170.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2023-10-26T01:28:38.857 INFO:teuthology.orchestra.run.smithi169.stdout:(1/143): ceph-18.0.0-6863.g38499368.el9.x86_64. 14 kB/s | 6.5 kB 00:00 2023-10-26T01:28:39.949 INFO:teuthology.orchestra.run.smithi169.stdout:(2/143): ceph-fuse-18.0.0-6863.g38499368.el9.x8 786 kB/s | 856 kB 00:01 2023-10-26T01:28:40.141 INFO:teuthology.orchestra.run.smithi169.stdout:(3/143): ceph-immutable-object-cache-18.0.0-686 745 kB/s | 142 kB 00:00 2023-10-26T01:28:40.450 INFO:teuthology.orchestra.run.smithi169.stdout:(4/143): ceph-base-18.0.0-6863.g38499368.el9.x8 2.5 MB/s | 5.2 MB 00:02 2023-10-26T01:28:40.775 INFO:teuthology.orchestra.run.smithi169.stdout:(5/143): ceph-mgr-18.0.0-6863.g38499368.el9.x86 4.5 MB/s | 1.5 MB 00:00 2023-10-26T01:28:40.818 INFO:teuthology.orchestra.run.smithi170.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2023-10-26T01:28:40.820 INFO:teuthology.orchestra.run.smithi170.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2023-10-26T01:28:40.842 INFO:teuthology.orchestra.run.smithi170.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2023-10-26T01:28:40.843 INFO:teuthology.orchestra.run.smithi170.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2023-10-26T01:28:40.845 INFO:teuthology.orchestra.run.smithi170.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2023-10-26T01:28:40.846 INFO:teuthology.orchestra.run.smithi170.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2023-10-26T01:28:40.849 INFO:teuthology.orchestra.run.smithi170.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2023-10-26T01:28:40.853 INFO:teuthology.orchestra.run.smithi170.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2023-10-26T01:28:40.854 INFO:teuthology.orchestra.run.smithi170.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2023-10-26T01:28:40.933 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2023-10-26T01:28:40.962 INFO:teuthology.orchestra.run.smithi170.stdout:============================================================================================ 2023-10-26T01:28:40.962 INFO:teuthology.orchestra.run.smithi170.stdout: Package Arch Version Repository Size 2023-10-26T01:28:40.963 INFO:teuthology.orchestra.run.smithi170.stdout:============================================================================================ 2023-10-26T01:28:40.963 INFO:teuthology.orchestra.run.smithi170.stdout:Installing: 2023-10-26T01:28:40.963 INFO:teuthology.orchestra.run.smithi170.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2023-10-26T01:28:40.963 INFO:teuthology.orchestra.run.smithi170.stdout: ceph x86_64 2:18.0.0-6863.g38499368.el9 ceph 6.5 k 2023-10-26T01:28:40.963 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-base x86_64 2:18.0.0-6863.g38499368.el9 ceph 5.2 M 2023-10-26T01:28:40.964 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-fuse x86_64 2:18.0.0-6863.g38499368.el9 ceph 856 k 2023-10-26T01:28:40.964 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-6863.g38499368.el9 ceph 142 k 2023-10-26T01:28:40.964 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr x86_64 2:18.0.0-6863.g38499368.el9 ceph 1.5 M 2023-10-26T01:28:40.964 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-cephadm noarch 2:18.0.0-6863.g38499368.el9 ceph-noarch 132 k 2023-10-26T01:28:40.964 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-dashboard noarch 2:18.0.0-6863.g38499368.el9 ceph-noarch 1.8 M 2023-10-26T01:28:40.964 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-6863.g38499368.el9 ceph-noarch 7.4 M 2023-10-26T01:28:40.965 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-rook noarch 2:18.0.0-6863.g38499368.el9 ceph-noarch 48 k 2023-10-26T01:28:40.965 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-radosgw x86_64 2:18.0.0-6863.g38499368.el9 ceph 8.6 M 2023-10-26T01:28:40.965 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-test x86_64 2:18.0.0-6863.g38499368.el9 ceph 41 M 2023-10-26T01:28:40.965 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-volume noarch 2:18.0.0-6863.g38499368.el9 ceph-noarch 262 k 2023-10-26T01:28:40.965 INFO:teuthology.orchestra.run.smithi170.stdout: cephadm noarch 2:18.0.0-6863.g38499368.el9 ceph-noarch 270 k 2023-10-26T01:28:40.966 INFO:teuthology.orchestra.run.smithi170.stdout: cephfs-mirror x86_64 2:18.0.0-6863.g38499368.el9 ceph 218 k 2023-10-26T01:28:40.966 INFO:teuthology.orchestra.run.smithi170.stdout: cephfs-top noarch 2:18.0.0-6863.g38499368.el9 ceph-noarch 18 k 2023-10-26T01:28:40.966 INFO:teuthology.orchestra.run.smithi170.stdout: elfutils-libelf-devel x86_64 0.189-3.el9 appstream 22 k 2023-10-26T01:28:40.966 INFO:teuthology.orchestra.run.smithi170.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2023-10-26T01:28:40.966 INFO:teuthology.orchestra.run.smithi170.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2023-10-26T01:28:40.967 INFO:teuthology.orchestra.run.smithi170.stdout: libcephfs-devel x86_64 2:18.0.0-6863.g38499368.el9 ceph 31 k 2023-10-26T01:28:40.967 INFO:teuthology.orchestra.run.smithi170.stdout: libcephfs2 x86_64 2:18.0.0-6863.g38499368.el9 ceph 718 k 2023-10-26T01:28:40.967 INFO:teuthology.orchestra.run.smithi170.stdout: librados-devel x86_64 2:18.0.0-6863.g38499368.el9 ceph 127 k 2023-10-26T01:28:40.967 INFO:teuthology.orchestra.run.smithi170.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2023-10-26T01:28:40.967 INFO:teuthology.orchestra.run.smithi170.stdout: libuuid-devel x86_64 2.37.4-15.el9 appstream 26 k 2023-10-26T01:28:40.968 INFO:teuthology.orchestra.run.smithi170.stdout: openssl-devel x86_64 1:3.0.7-24.el9 appstream 4.1 M 2023-10-26T01:28:40.968 INFO:teuthology.orchestra.run.smithi170.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2023-10-26T01:28:40.968 INFO:teuthology.orchestra.run.smithi170.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2023-10-26T01:28:40.968 INFO:teuthology.orchestra.run.smithi170.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2023-10-26T01:28:40.968 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cephfs x86_64 2:18.0.0-6863.g38499368.el9 ceph 162 k 2023-10-26T01:28:40.969 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rados x86_64 2:18.0.0-6863.g38499368.el9 ceph 321 k 2023-10-26T01:28:40.969 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rbd x86_64 2:18.0.0-6863.g38499368.el9 ceph 298 k 2023-10-26T01:28:40.969 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rgw x86_64 2:18.0.0-6863.g38499368.el9 ceph 99 k 2023-10-26T01:28:40.969 INFO:teuthology.orchestra.run.smithi170.stdout: rbd-fuse x86_64 2:18.0.0-6863.g38499368.el9 ceph 86 k 2023-10-26T01:28:40.969 INFO:teuthology.orchestra.run.smithi170.stdout: rbd-mirror x86_64 2:18.0.0-6863.g38499368.el9 ceph 3.0 M 2023-10-26T01:28:40.970 INFO:teuthology.orchestra.run.smithi170.stdout: rbd-nbd x86_64 2:18.0.0-6863.g38499368.el9 ceph 170 k 2023-10-26T01:28:40.970 INFO:teuthology.orchestra.run.smithi170.stdout: xfsprogs-devel x86_64 5.19.0-4.el9 appstream 86 k 2023-10-26T01:28:40.970 INFO:teuthology.orchestra.run.smithi170.stdout:Upgrading: 2023-10-26T01:28:40.970 INFO:teuthology.orchestra.run.smithi170.stdout: NetworkManager x86_64 1:1.45.5-1.el9 CentOS-BaseOS 2.2 M 2023-10-26T01:28:40.970 INFO:teuthology.orchestra.run.smithi170.stdout: NetworkManager-libnm x86_64 1:1.45.5-1.el9 CentOS-BaseOS 1.8 M 2023-10-26T01:28:40.971 INFO:teuthology.orchestra.run.smithi170.stdout: NetworkManager-team x86_64 1:1.45.5-1.el9 CentOS-BaseOS 42 k 2023-10-26T01:28:40.971 INFO:teuthology.orchestra.run.smithi170.stdout: NetworkManager-tui x86_64 1:1.45.5-1.el9 CentOS-BaseOS 245 k 2023-10-26T01:28:40.971 INFO:teuthology.orchestra.run.smithi170.stdout: acl x86_64 2.3.1-4.el9 CentOS-BaseOS 69 k 2023-10-26T01:28:40.971 INFO:teuthology.orchestra.run.smithi170.stdout: libacl x86_64 2.3.1-4.el9 CentOS-BaseOS 22 k 2023-10-26T01:28:40.971 INFO:teuthology.orchestra.run.smithi170.stdout: libacl-devel x86_64 2.3.1-4.el9 CentOS-AppStream 68 k 2023-10-26T01:28:40.972 INFO:teuthology.orchestra.run.smithi170.stdout: librados2 x86_64 2:18.0.0-6863.g38499368.el9 ceph 3.3 M 2023-10-26T01:28:40.972 INFO:teuthology.orchestra.run.smithi170.stdout: librbd1 x86_64 2:18.0.0-6863.g38499368.el9 ceph 3.0 M 2023-10-26T01:28:40.972 INFO:teuthology.orchestra.run.smithi170.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2023-10-26T01:28:40.972 INFO:teuthology.orchestra.run.smithi170.stdout:Installing dependencies: 2023-10-26T01:28:40.972 INFO:teuthology.orchestra.run.smithi170.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2023-10-26T01:28:40.973 INFO:teuthology.orchestra.run.smithi170.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2023-10-26T01:28:40.973 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-common x86_64 2:18.0.0-6863.g38499368.el9 ceph 19 M 2023-10-26T01:28:40.973 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-grafana-dashboards noarch 2:18.0.0-6863.g38499368.el9 ceph-noarch 23 k 2023-10-26T01:28:40.973 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mds x86_64 2:18.0.0-6863.g38499368.el9 ceph 2.1 M 2023-10-26T01:28:40.973 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-modules-core noarch 2:18.0.0-6863.g38499368.el9 ceph-noarch 243 k 2023-10-26T01:28:40.974 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mon x86_64 2:18.0.0-6863.g38499368.el9 ceph 4.4 M 2023-10-26T01:28:40.974 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-osd x86_64 2:18.0.0-6863.g38499368.el9 ceph 18 M 2023-10-26T01:28:40.974 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-prometheus-alerts noarch 2:18.0.0-6863.g38499368.el9 ceph-noarch 15 k 2023-10-26T01:28:40.974 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-selinux x86_64 2:18.0.0-6863.g38499368.el9 ceph 24 k 2023-10-26T01:28:40.974 INFO:teuthology.orchestra.run.smithi170.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2023-10-26T01:28:40.975 INFO:teuthology.orchestra.run.smithi170.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2023-10-26T01:28:40.975 INFO:teuthology.orchestra.run.smithi170.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2023-10-26T01:28:40.975 INFO:teuthology.orchestra.run.smithi170.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2023-10-26T01:28:40.975 INFO:teuthology.orchestra.run.smithi170.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2023-10-26T01:28:40.975 INFO:teuthology.orchestra.run.smithi170.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2023-10-26T01:28:40.976 INFO:teuthology.orchestra.run.smithi170.stdout: jq x86_64 1.6-15.el9 appstream 187 k 2023-10-26T01:28:40.976 INFO:teuthology.orchestra.run.smithi170.stdout: libarrow x86_64 9.0.0-9.el9 epel 4.4 M 2023-10-26T01:28:40.976 INFO:teuthology.orchestra.run.smithi170.stdout: libarrow-doc noarch 9.0.0-9.el9 epel 24 k 2023-10-26T01:28:40.976 INFO:teuthology.orchestra.run.smithi170.stdout: libcephsqlite x86_64 2:18.0.0-6863.g38499368.el9 ceph 165 k 2023-10-26T01:28:40.976 INFO:teuthology.orchestra.run.smithi170.stdout: libgfortran x86_64 11.4.1-2.3.el9 CentOS-BaseOS 803 k 2023-10-26T01:28:40.977 INFO:teuthology.orchestra.run.smithi170.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2023-10-26T01:28:40.977 INFO:teuthology.orchestra.run.smithi170.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2023-10-26T01:28:40.977 INFO:teuthology.orchestra.run.smithi170.stdout: libquadmath x86_64 11.4.1-2.3.el9 CentOS-BaseOS 195 k 2023-10-26T01:28:40.977 INFO:teuthology.orchestra.run.smithi170.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2023-10-26T01:28:40.977 INFO:teuthology.orchestra.run.smithi170.stdout: libradosstriper1 x86_64 2:18.0.0-6863.g38499368.el9 ceph 473 k 2023-10-26T01:28:40.977 INFO:teuthology.orchestra.run.smithi170.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2023-10-26T01:28:40.978 INFO:teuthology.orchestra.run.smithi170.stdout: librgw2 x86_64 2:18.0.0-6863.g38499368.el9 ceph 4.8 M 2023-10-26T01:28:40.978 INFO:teuthology.orchestra.run.smithi170.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2023-10-26T01:28:40.978 INFO:teuthology.orchestra.run.smithi170.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2023-10-26T01:28:40.978 INFO:teuthology.orchestra.run.smithi170.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2023-10-26T01:28:40.978 INFO:teuthology.orchestra.run.smithi170.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2023-10-26T01:28:40.979 INFO:teuthology.orchestra.run.smithi170.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2023-10-26T01:28:40.979 INFO:teuthology.orchestra.run.smithi170.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2023-10-26T01:28:40.979 INFO:teuthology.orchestra.run.smithi170.stdout: parquet-libs x86_64 9.0.0-9.el9 epel 837 k 2023-10-26T01:28:40.979 INFO:teuthology.orchestra.run.smithi170.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2023-10-26T01:28:40.979 INFO:teuthology.orchestra.run.smithi170.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2023-10-26T01:28:40.980 INFO:teuthology.orchestra.run.smithi170.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2023-10-26T01:28:40.980 INFO:teuthology.orchestra.run.smithi170.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2023-10-26T01:28:40.980 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2023-10-26T01:28:40.980 INFO:teuthology.orchestra.run.smithi170.stdout: python3-ceph-argparse x86_64 2:18.0.0-6863.g38499368.el9 ceph 45 k 2023-10-26T01:28:40.980 INFO:teuthology.orchestra.run.smithi170.stdout: python3-ceph-common x86_64 2:18.0.0-6863.g38499368.el9 ceph 127 k 2023-10-26T01:28:40.981 INFO:teuthology.orchestra.run.smithi170.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2023-10-26T01:28:40.981 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2023-10-26T01:28:40.981 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2023-10-26T01:28:40.981 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2023-10-26T01:28:40.981 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2023-10-26T01:28:40.981 INFO:teuthology.orchestra.run.smithi170.stdout: python3-devel x86_64 3.9.17-2.el9 CentOS-AppStream 205 k 2023-10-26T01:28:40.982 INFO:teuthology.orchestra.run.smithi170.stdout: python3-google-auth noarch 1:2.23.3-1.el9 epel 197 k 2023-10-26T01:28:40.982 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2023-10-26T01:28:40.982 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2023-10-26T01:28:40.982 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2023-10-26T01:28:40.982 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2023-10-26T01:28:40.983 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2023-10-26T01:28:40.983 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jinja2 noarch 2.11.3-4.el9 CentOS-AppStream 228 k 2023-10-26T01:28:40.983 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2023-10-26T01:28:40.983 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2023-10-26T01:28:40.983 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2023-10-26T01:28:40.984 INFO:teuthology.orchestra.run.smithi170.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2023-10-26T01:28:40.984 INFO:teuthology.orchestra.run.smithi170.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2023-10-26T01:28:40.984 INFO:teuthology.orchestra.run.smithi170.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2023-10-26T01:28:40.984 INFO:teuthology.orchestra.run.smithi170.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2023-10-26T01:28:40.984 INFO:teuthology.orchestra.run.smithi170.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2023-10-26T01:28:40.985 INFO:teuthology.orchestra.run.smithi170.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2023-10-26T01:28:40.985 INFO:teuthology.orchestra.run.smithi170.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2023-10-26T01:28:40.985 INFO:teuthology.orchestra.run.smithi170.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2023-10-26T01:28:40.985 INFO:teuthology.orchestra.run.smithi170.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2023-10-26T01:28:40.985 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2023-10-26T01:28:40.986 INFO:teuthology.orchestra.run.smithi170.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2023-10-26T01:28:40.986 INFO:teuthology.orchestra.run.smithi170.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2023-10-26T01:28:40.986 INFO:teuthology.orchestra.run.smithi170.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2023-10-26T01:28:40.986 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2023-10-26T01:28:40.986 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2023-10-26T01:28:40.986 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2023-10-26T01:28:40.987 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2023-10-26T01:28:40.987 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2023-10-26T01:28:40.987 INFO:teuthology.orchestra.run.smithi170.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2023-10-26T01:28:40.987 INFO:teuthology.orchestra.run.smithi170.stdout: python3-requests noarch 2.25.1-7.el9 CentOS-BaseOS 113 k 2023-10-26T01:28:40.988 INFO:teuthology.orchestra.run.smithi170.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2023-10-26T01:28:40.988 INFO:teuthology.orchestra.run.smithi170.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2023-10-26T01:28:40.988 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2023-10-26T01:28:40.988 INFO:teuthology.orchestra.run.smithi170.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2023-10-26T01:28:40.988 INFO:teuthology.orchestra.run.smithi170.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2023-10-26T01:28:40.988 INFO:teuthology.orchestra.run.smithi170.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2023-10-26T01:28:40.989 INFO:teuthology.orchestra.run.smithi170.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2023-10-26T01:28:40.989 INFO:teuthology.orchestra.run.smithi170.stdout: python3-urllib3 noarch 1.26.5-4.el9 CentOS-BaseOS 187 k 2023-10-26T01:28:40.989 INFO:teuthology.orchestra.run.smithi170.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2023-10-26T01:28:40.989 INFO:teuthology.orchestra.run.smithi170.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2023-10-26T01:28:40.989 INFO:teuthology.orchestra.run.smithi170.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2023-10-26T01:28:40.990 INFO:teuthology.orchestra.run.smithi170.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2023-10-26T01:28:40.990 INFO:teuthology.orchestra.run.smithi170.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2023-10-26T01:28:40.990 INFO:teuthology.orchestra.run.smithi170.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2023-10-26T01:28:40.990 INFO:teuthology.orchestra.run.smithi170.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2023-10-26T01:28:40.990 INFO:teuthology.orchestra.run.smithi170.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2023-10-26T01:28:40.991 INFO:teuthology.orchestra.run.smithi170.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2023-10-26T01:28:40.991 INFO:teuthology.orchestra.run.smithi170.stdout: zlib-devel x86_64 1.2.11-40.el9 appstream 45 k 2023-10-26T01:28:40.991 INFO:teuthology.orchestra.run.smithi170.stdout:Installing weak dependencies: 2023-10-26T01:28:40.991 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2023-10-26T01:28:40.992 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-26T01:28:40.992 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction Summary 2023-10-26T01:28:40.992 INFO:teuthology.orchestra.run.smithi170.stdout:============================================================================================ 2023-10-26T01:28:40.992 INFO:teuthology.orchestra.run.smithi170.stdout:Install 133 Packages 2023-10-26T01:28:40.992 INFO:teuthology.orchestra.run.smithi170.stdout:Upgrade 10 Packages 2023-10-26T01:28:40.992 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-26T01:28:40.993 INFO:teuthology.orchestra.run.smithi170.stdout:Total download size: 200 M 2023-10-26T01:28:40.993 INFO:teuthology.orchestra.run.smithi170.stdout:Downloading Packages: 2023-10-26T01:28:41.500 INFO:teuthology.orchestra.run.smithi169.stdout:(6/143): ceph-mds-18.0.0-6863.g38499368.el9.x86 1.5 MB/s | 2.1 MB 00:01 2023-10-26T01:28:41.659 INFO:teuthology.orchestra.run.smithi169.stdout:(7/143): ceph-mon-18.0.0-6863.g38499368.el9.x86 5.0 MB/s | 4.4 MB 00:00 2023-10-26T01:28:42.065 INFO:teuthology.orchestra.run.smithi170.stdout:(1/143): ceph-18.0.0-6863.g38499368.el9.x86_64. 19 kB/s | 6.5 kB 00:00 2023-10-26T01:28:42.598 INFO:teuthology.orchestra.run.smithi170.stdout:(2/143): ceph-fuse-18.0.0-6863.g38499368.el9.x8 1.6 MB/s | 856 kB 00:00 2023-10-26T01:28:42.665 INFO:teuthology.orchestra.run.smithi170.stdout:(3/143): ceph-immutable-object-cache-18.0.0-686 2.1 MB/s | 142 kB 00:00 2023-10-26T01:28:42.718 INFO:teuthology.orchestra.run.smithi169.stdout:(8/143): ceph-common-18.0.0-6863.g38499368.el9. 4.4 MB/s | 19 MB 00:04 2023-10-26T01:28:42.885 INFO:teuthology.orchestra.run.smithi169.stdout:(9/143): ceph-radosgw-18.0.0-6863.g38499368.el9 7.0 MB/s | 8.6 MB 00:01 2023-10-26T01:28:42.901 INFO:teuthology.orchestra.run.smithi169.stdout:(10/143): ceph-selinux-18.0.0-6863.g38499368.el 132 kB/s | 24 kB 00:00 2023-10-26T01:28:42.993 INFO:teuthology.orchestra.run.smithi169.stdout:(11/143): cephfs-mirror-18.0.0-6863.g38499368.e 2.3 MB/s | 218 kB 00:00 2023-10-26T01:28:43.085 INFO:teuthology.orchestra.run.smithi169.stdout:(12/143): libcephfs-devel-18.0.0-6863.g38499368 343 kB/s | 31 kB 00:00 2023-10-26T01:28:43.174 INFO:teuthology.orchestra.run.smithi170.stdout:(4/143): ceph-mds-18.0.0-6863.g38499368.el9.x86 4.1 MB/s | 2.1 MB 00:00 2023-10-26T01:28:43.260 INFO:teuthology.orchestra.run.smithi169.stdout:(13/143): libcephfs2-18.0.0-6863.g38499368.el9. 4.0 MB/s | 718 kB 00:00 2023-10-26T01:28:43.369 INFO:teuthology.orchestra.run.smithi169.stdout:(14/143): libcephsqlite-18.0.0-6863.g38499368.e 1.5 MB/s | 165 kB 00:00 2023-10-26T01:28:43.478 INFO:teuthology.orchestra.run.smithi169.stdout:(15/143): librados-devel-18.0.0-6863.g38499368. 1.2 MB/s | 127 kB 00:00 2023-10-26T01:28:43.611 INFO:teuthology.orchestra.run.smithi169.stdout:(16/143): libradosstriper1-18.0.0-6863.g3849936 3.5 MB/s | 473 kB 00:00 2023-10-26T01:28:43.649 INFO:teuthology.orchestra.run.smithi170.stdout:(5/143): ceph-mgr-18.0.0-6863.g38499368.el9.x86 3.1 MB/s | 1.5 MB 00:00 2023-10-26T01:28:43.833 INFO:teuthology.orchestra.run.smithi170.stdout:(6/143): ceph-base-18.0.0-6863.g38499368.el9.x8 2.5 MB/s | 5.2 MB 00:02 2023-10-26T01:28:44.228 INFO:teuthology.orchestra.run.smithi169.stdout:(17/143): librgw2-18.0.0-6863.g38499368.el9.x86 7.8 MB/s | 4.8 MB 00:00 2023-10-26T01:28:44.312 INFO:teuthology.orchestra.run.smithi169.stdout:(18/143): python3-ceph-argparse-18.0.0-6863.g38 549 kB/s | 45 kB 00:00 2023-10-26T01:28:44.421 INFO:teuthology.orchestra.run.smithi169.stdout:(19/143): python3-ceph-common-18.0.0-6863.g3849 1.1 MB/s | 127 kB 00:00 2023-10-26T01:28:44.504 INFO:teuthology.orchestra.run.smithi169.stdout:(20/143): python3-cephfs-18.0.0-6863.g38499368. 1.9 MB/s | 162 kB 00:00 2023-10-26T01:28:44.580 INFO:teuthology.orchestra.run.smithi169.stdout:(21/143): python3-rados-18.0.0-6863.g38499368.e 4.2 MB/s | 321 kB 00:00 2023-10-26T01:28:44.657 INFO:teuthology.orchestra.run.smithi169.stdout:(22/143): python3-rbd-18.0.0-6863.g38499368.el9 3.9 MB/s | 298 kB 00:00 2023-10-26T01:28:44.666 INFO:teuthology.orchestra.run.smithi170.stdout:(7/143): ceph-common-18.0.0-6863.g38499368.el9. 6.4 MB/s | 19 MB 00:02 2023-10-26T01:28:44.722 INFO:teuthology.orchestra.run.smithi169.stdout:(23/143): python3-rgw-18.0.0-6863.g38499368.el9 1.5 MB/s | 99 kB 00:00 2023-10-26T01:28:44.814 INFO:teuthology.orchestra.run.smithi169.stdout:(24/143): rbd-fuse-18.0.0-6863.g38499368.el9.x8 945 kB/s | 86 kB 00:00 2023-10-26T01:28:45.142 INFO:teuthology.orchestra.run.smithi170.stdout:(8/143): ceph-mon-18.0.0-6863.g38499368.el9.x86 2.9 MB/s | 4.4 MB 00:01 2023-10-26T01:28:45.233 INFO:teuthology.orchestra.run.smithi170.stdout:(9/143): ceph-selinux-18.0.0-6863.g38499368.el9 265 kB/s | 24 kB 00:00 2023-10-26T01:28:45.289 INFO:teuthology.orchestra.run.smithi169.stdout:(25/143): rbd-mirror-18.0.0-6863.g38499368.el9. 6.4 MB/s | 3.0 MB 00:00 2023-10-26T01:28:45.406 INFO:teuthology.orchestra.run.smithi169.stdout:(26/143): rbd-nbd-18.0.0-6863.g38499368.el9.x86 1.4 MB/s | 170 kB 00:00 2023-10-26T01:28:45.515 INFO:teuthology.orchestra.run.smithi169.stdout:(27/143): ceph-grafana-dashboards-18.0.0-6863.g 215 kB/s | 23 kB 00:00 2023-10-26T01:28:45.632 INFO:teuthology.orchestra.run.smithi169.stdout:(28/143): ceph-mgr-cephadm-18.0.0-6863.g3849936 1.1 MB/s | 132 kB 00:00 2023-10-26T01:28:45.949 INFO:teuthology.orchestra.run.smithi169.stdout:(29/143): ceph-mgr-dashboard-18.0.0-6863.g38499 5.6 MB/s | 1.8 MB 00:00 2023-10-26T01:28:46.034 INFO:teuthology.orchestra.run.smithi170.stdout:(10/143): ceph-osd-18.0.0-6863.g38499368.el9.x8 8.1 MB/s | 18 MB 00:02 2023-10-26T01:28:46.201 INFO:teuthology.orchestra.run.smithi170.stdout:(11/143): ceph-radosgw-18.0.0-6863.g38499368.el 5.6 MB/s | 8.6 MB 00:01 2023-10-26T01:28:46.234 INFO:teuthology.orchestra.run.smithi170.stdout:(12/143): cephfs-mirror-18.0.0-6863.g38499368.e 1.1 MB/s | 218 kB 00:00 2023-10-26T01:28:46.259 INFO:teuthology.orchestra.run.smithi170.stdout:(13/143): libcephfs-devel-18.0.0-6863.g38499368 542 kB/s | 31 kB 00:00 2023-10-26T01:28:46.343 INFO:teuthology.orchestra.run.smithi170.stdout:(14/143): libcephsqlite-18.0.0-6863.g38499368.e 1.9 MB/s | 165 kB 00:00 2023-10-26T01:28:46.376 INFO:teuthology.orchestra.run.smithi170.stdout:(15/143): libcephfs2-18.0.0-6863.g38499368.el9. 5.0 MB/s | 718 kB 00:00 2023-10-26T01:28:46.410 INFO:teuthology.orchestra.run.smithi170.stdout:(16/143): librados-devel-18.0.0-6863.g38499368. 1.9 MB/s | 127 kB 00:00 2023-10-26T01:28:46.477 INFO:teuthology.orchestra.run.smithi170.stdout:(17/143): libradosstriper1-18.0.0-6863.g3849936 4.6 MB/s | 473 kB 00:00 2023-10-26T01:28:46.561 INFO:teuthology.orchestra.run.smithi170.stdout:(18/143): python3-ceph-argparse-18.0.0-6863.g38 548 kB/s | 45 kB 00:00 2023-10-26T01:28:46.658 INFO:teuthology.orchestra.run.smithi170.stdout:(19/143): python3-ceph-common-18.0.0-6863.g3849 1.3 MB/s | 127 kB 00:00 2023-10-26T01:28:46.733 INFO:teuthology.orchestra.run.smithi170.stdout:(20/143): python3-cephfs-18.0.0-6863.g38499368. 2.1 MB/s | 162 kB 00:00 2023-10-26T01:28:46.782 INFO:teuthology.orchestra.run.smithi169.stdout:(30/143): ceph-mgr-diskprediction-local-18.0.0- 8.9 MB/s | 7.4 MB 00:00 2023-10-26T01:28:46.816 INFO:teuthology.orchestra.run.smithi170.stdout:(21/143): python3-rados-18.0.0-6863.g38499368.e 3.8 MB/s | 321 kB 00:00 2023-10-26T01:28:46.894 INFO:teuthology.orchestra.run.smithi169.stdout:(31/143): ceph-mgr-modules-core-18.0.0-6863.g38 2.1 MB/s | 243 kB 00:00 2023-10-26T01:28:46.908 INFO:teuthology.orchestra.run.smithi170.stdout:(22/143): python3-rbd-18.0.0-6863.g38499368.el9 3.2 MB/s | 298 kB 00:00 2023-10-26T01:28:46.958 INFO:teuthology.orchestra.run.smithi169.stdout:(32/143): ceph-mgr-rook-18.0.0-6863.g38499368.e 764 kB/s | 48 kB 00:00 2023-10-26T01:28:46.984 INFO:teuthology.orchestra.run.smithi170.stdout:(23/143): python3-rgw-18.0.0-6863.g38499368.el9 1.3 MB/s | 99 kB 00:00 2023-10-26T01:28:47.016 INFO:teuthology.orchestra.run.smithi169.stdout:(33/143): ceph-prometheus-alerts-18.0.0-6863.g3 262 kB/s | 15 kB 00:00 2023-10-26T01:28:47.098 INFO:teuthology.orchestra.run.smithi169.stdout:(34/143): ceph-volume-18.0.0-6863.g38499368.el9 3.2 MB/s | 262 kB 00:00 2023-10-26T01:28:47.118 INFO:teuthology.orchestra.run.smithi170.stdout:(24/143): librgw2-18.0.0-6863.g38499368.el9.x86 6.8 MB/s | 4.8 MB 00:00 2023-10-26T01:28:47.143 INFO:teuthology.orchestra.run.smithi170.stdout:(25/143): rbd-fuse-18.0.0-6863.g38499368.el9.x8 545 kB/s | 86 kB 00:00 2023-10-26T01:28:47.173 INFO:teuthology.orchestra.run.smithi169.stdout:(35/143): cephadm-18.0.0-6863.g38499368.el9.noa 3.6 MB/s | 270 kB 00:00 2023-10-26T01:28:47.218 INFO:teuthology.orchestra.run.smithi170.stdout:(26/143): rbd-nbd-18.0.0-6863.g38499368.el9.x86 2.2 MB/s | 170 kB 00:00 2023-10-26T01:28:47.265 INFO:teuthology.orchestra.run.smithi169.stdout:(36/143): cephfs-top-18.0.0-6863.g38499368.el9. 201 kB/s | 18 kB 00:00 2023-10-26T01:28:47.293 INFO:teuthology.orchestra.run.smithi170.stdout:(27/143): ceph-grafana-dashboards-18.0.0-6863.g 313 kB/s | 23 kB 00:00 2023-10-26T01:28:47.377 INFO:teuthology.orchestra.run.smithi170.stdout:(28/143): ceph-mgr-cephadm-18.0.0-6863.g3849936 1.6 MB/s | 132 kB 00:00 2023-10-26T01:28:47.457 INFO:teuthology.orchestra.run.smithi169.stdout:(37/143): bison-3.7.4-5.el9.x86_64.rpm 4.7 MB/s | 920 kB 00:00 2023-10-26T01:28:47.477 INFO:teuthology.orchestra.run.smithi170.stdout:(29/143): rbd-mirror-18.0.0-6863.g38499368.el9. 8.4 MB/s | 3.0 MB 00:00 2023-10-26T01:28:47.499 INFO:teuthology.orchestra.run.smithi169.stdout:(38/143): boost-program-options-1.75.0-8.el9.x8 2.5 MB/s | 106 kB 00:00 2023-10-26T01:28:47.541 INFO:teuthology.orchestra.run.smithi169.stdout:(39/143): flex-2.6.4-9.el9.x86_64.rpm 7.4 MB/s | 307 kB 00:00 2023-10-26T01:28:47.599 INFO:teuthology.orchestra.run.smithi169.stdout:(40/143): flexiblas-3.0.4-8.el9.x86_64.rpm 550 kB/s | 32 kB 00:00 2023-10-26T01:28:47.621 INFO:teuthology.orchestra.run.smithi170.stdout:(30/143): ceph-mgr-dashboard-18.0.0-6863.g38499 7.3 MB/s | 1.8 MB 00:00 2023-10-26T01:28:47.711 INFO:teuthology.orchestra.run.smithi170.stdout:(31/143): ceph-mgr-modules-core-18.0.0-6863.g38 2.7 MB/s | 243 kB 00:00 2023-10-26T01:28:47.739 INFO:teuthology.orchestra.run.smithi169.stdout:(41/143): flexiblas-netlib-3.0.4-8.el9.x86_64.r 22 MB/s | 3.0 MB 00:00 2023-10-26T01:28:47.787 INFO:teuthology.orchestra.run.smithi170.stdout:(32/143): ceph-mgr-rook-18.0.0-6863.g38499368.e 644 kB/s | 48 kB 00:00 2023-10-26T01:28:47.808 INFO:teuthology.orchestra.run.smithi169.stdout:(42/143): flexiblas-openblas-openmp-3.0.4-8.el9 250 kB/s | 17 kB 00:00 2023-10-26T01:28:47.853 INFO:teuthology.orchestra.run.smithi170.stdout:(33/143): ceph-prometheus-alerts-18.0.0-6863.g3 230 kB/s | 15 kB 00:00 2023-10-26T01:28:47.867 INFO:teuthology.orchestra.run.smithi169.stdout:(43/143): libaio-devel-0.3.111-13.el9.x86_64.rp 175 kB/s | 10 kB 00:00 2023-10-26T01:28:47.909 INFO:teuthology.orchestra.run.smithi169.stdout:(44/143): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.8 MB/s | 159 kB 00:00 2023-10-26T01:28:47.937 INFO:teuthology.orchestra.run.smithi170.stdout:(34/143): ceph-volume-18.0.0-6863.g38499368.el9 3.1 MB/s | 262 kB 00:00 2023-10-26T01:28:47.967 INFO:teuthology.orchestra.run.smithi169.stdout:(45/143): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 661 kB 00:00 2023-10-26T01:28:48.018 INFO:teuthology.orchestra.run.smithi169.stdout:(46/143): libtool-2.4.6-45.el9.x86_64.rpm 12 MB/s | 578 kB 00:00 2023-10-26T01:28:48.021 INFO:teuthology.orchestra.run.smithi170.stdout:(35/143): cephadm-18.0.0-6863.g38499368.el9.noa 3.2 MB/s | 270 kB 00:00 2023-10-26T01:28:48.077 INFO:teuthology.orchestra.run.smithi169.stdout:(47/143): libzstd-devel-1.5.1-2.el9.x86_64.rpm 806 kB/s | 46 kB 00:00 2023-10-26T01:28:48.104 INFO:teuthology.orchestra.run.smithi170.stdout:(36/143): cephfs-top-18.0.0-6863.g38499368.el9. 221 kB/s | 18 kB 00:00 2023-10-26T01:28:48.118 INFO:teuthology.orchestra.run.smithi169.stdout:(48/143): lttng-ust-2.12.0-6.el9.x86_64.rpm 6.8 MB/s | 282 kB 00:00 2023-10-26T01:28:48.160 INFO:teuthology.orchestra.run.smithi169.stdout:(49/143): oniguruma-6.9.6-1.el9.5.x86_64.rpm 5.2 MB/s | 217 kB 00:00 2023-10-26T01:28:48.219 INFO:teuthology.orchestra.run.smithi169.stdout:(50/143): openblas-0.3.21-2.el9.x86_64.rpm 592 kB/s | 34 kB 00:00 2023-10-26T01:28:48.353 INFO:teuthology.orchestra.run.smithi169.stdout:(51/143): openblas-openmp-0.3.21-2.el9.x86_64.r 36 MB/s | 4.7 MB 00:00 2023-10-26T01:28:48.420 INFO:teuthology.orchestra.run.smithi169.stdout:(52/143): postgresql-13.11-1.el9.x86_64.rpm 23 MB/s | 1.5 MB 00:00 2023-10-26T01:28:48.422 INFO:teuthology.orchestra.run.smithi170.stdout:(37/143): ceph-mgr-diskprediction-local-18.0.0- 7.8 MB/s | 7.4 MB 00:00 2023-10-26T01:28:48.463 INFO:teuthology.orchestra.run.smithi170.stdout:(38/143): bison-3.7.4-5.el9.x86_64.rpm 2.5 MB/s | 920 kB 00:00 2023-10-26T01:28:48.478 INFO:teuthology.orchestra.run.smithi169.stdout:(53/143): postgresql-contrib-13.11-1.el9.x86_64 14 MB/s | 811 kB 00:00 2023-10-26T01:28:48.489 INFO:teuthology.orchestra.run.smithi170.stdout:(39/143): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 106 kB 00:00 2023-10-26T01:28:48.521 INFO:teuthology.orchestra.run.smithi169.stdout:(54/143): postgresql-private-libs-13.11-1.el9.x 3.2 MB/s | 134 kB 00:00 2023-10-26T01:28:48.522 INFO:teuthology.orchestra.run.smithi170.stdout:(40/143): flex-2.6.4-9.el9.x86_64.rpm 5.3 MB/s | 307 kB 00:00 2023-10-26T01:28:48.549 INFO:teuthology.orchestra.run.smithi170.stdout:(41/143): flexiblas-3.0.4-8.el9.x86_64.rpm 574 kB/s | 32 kB 00:00 2023-10-26T01:28:48.640 INFO:teuthology.orchestra.run.smithi170.stdout:(42/143): flexiblas-openblas-openmp-3.0.4-8.el9 191 kB/s | 17 kB 00:00 2023-10-26T01:28:48.687 INFO:teuthology.orchestra.run.smithi169.stdout:(55/143): postgresql-server-13.11-1.el9.x86_64. 34 MB/s | 5.7 MB 00:00 2023-10-26T01:28:48.723 INFO:teuthology.orchestra.run.smithi170.stdout:(43/143): flexiblas-netlib-3.0.4-8.el9.x86_64.r 15 MB/s | 3.0 MB 00:00 2023-10-26T01:28:48.757 INFO:teuthology.orchestra.run.smithi170.stdout:(44/143): libaio-devel-0.3.111-13.el9.x86_64.rp 90 kB/s | 10 kB 00:00 2023-10-26T01:28:48.782 INFO:teuthology.orchestra.run.smithi170.stdout:(45/143): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 159 kB 00:00 2023-10-26T01:28:48.849 INFO:teuthology.orchestra.run.smithi170.stdout:(46/143): librdkafka-1.6.1-102.el9.x86_64.rpm 7.2 MB/s | 661 kB 00:00 2023-10-26T01:28:48.891 INFO:teuthology.orchestra.run.smithi170.stdout:(47/143): libtool-2.4.6-45.el9.x86_64.rpm 5.5 MB/s | 578 kB 00:00 2023-10-26T01:28:48.913 INFO:teuthology.orchestra.run.smithi169.stdout:(56/143): ceph-osd-18.0.0-6863.g38499368.el9.x8 2.4 MB/s | 18 MB 00:07 2023-10-26T01:28:48.933 INFO:teuthology.orchestra.run.smithi170.stdout:(48/143): libzstd-devel-1.5.1-2.el9.x86_64.rpm 594 kB/s | 46 kB 00:00 2023-10-26T01:28:48.967 INFO:teuthology.orchestra.run.smithi170.stdout:(49/143): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.7 MB/s | 282 kB 00:00 2023-10-26T01:28:49.000 INFO:teuthology.orchestra.run.smithi170.stdout:(50/143): oniguruma-6.9.6-1.el9.5.x86_64.rpm 3.3 MB/s | 217 kB 00:00 2023-10-26T01:28:49.026 INFO:teuthology.orchestra.run.smithi170.stdout:(51/143): openblas-0.3.21-2.el9.x86_64.rpm 612 kB/s | 34 kB 00:00 2023-10-26T01:28:49.063 INFO:teuthology.orchestra.run.smithi169.stdout:(57/143): python3-babel-2.9.1-2.el9.noarch.rpm 15 MB/s | 5.8 MB 00:00 2023-10-26T01:28:49.088 INFO:teuthology.orchestra.run.smithi169.stdout:(58/143): python3-devel-3.9.17-2.el9.x86_64.rpm 1.2 MB/s | 205 kB 00:00 2023-10-26T01:28:49.138 INFO:teuthology.orchestra.run.smithi169.stdout:(59/143): python3-jinja2-2.11.3-4.el9.noarch.rp 4.6 MB/s | 228 kB 00:00 2023-10-26T01:28:49.160 INFO:teuthology.orchestra.run.smithi170.stdout:(52/143): openblas-openmp-0.3.21-2.el9.x86_64.r 30 MB/s | 4.7 MB 00:00 2023-10-26T01:28:49.180 INFO:teuthology.orchestra.run.smithi169.stdout:(60/143): python3-jsonpatch-1.21-16.el9.noarch. 269 kB/s | 24 kB 00:00 2023-10-26T01:28:49.226 INFO:teuthology.orchestra.run.smithi170.stdout:(53/143): postgresql-13.11-1.el9.x86_64.rpm 8.2 MB/s | 1.5 MB 00:00 2023-10-26T01:28:49.239 INFO:teuthology.orchestra.run.smithi169.stdout:(61/143): python3-jsonpointer-2.0-4.el9.noarch. 316 kB/s | 18 kB 00:00 2023-10-26T01:28:49.264 INFO:teuthology.orchestra.run.smithi169.stdout:(62/143): python3-mako-1.1.4-6.el9.noarch.rpm 1.7 MB/s | 144 kB 00:00 2023-10-26T01:28:49.268 INFO:teuthology.orchestra.run.smithi170.stdout:(54/143): postgresql-contrib-13.11-1.el9.x86_64 8.3 MB/s | 811 kB 00:00 2023-10-26T01:28:49.294 INFO:teuthology.orchestra.run.smithi170.stdout:(55/143): postgresql-private-libs-13.11-1.el9.x 2.2 MB/s | 134 kB 00:00 2023-10-26T01:28:49.314 INFO:teuthology.orchestra.run.smithi169.stdout:(63/143): python3-markupsafe-1.1.1-12.el9.x86_6 643 kB/s | 32 kB 00:00 2023-10-26T01:28:49.494 INFO:teuthology.orchestra.run.smithi170.stdout:(56/143): postgresql-server-13.11-1.el9.x86_64. 25 MB/s | 5.7 MB 00:00 2023-10-26T01:28:49.515 INFO:teuthology.orchestra.run.smithi169.stdout:(64/143): python3-numpy-1.20.1-5.el9.x86_64.rpm 19 MB/s | 4.8 MB 00:00 2023-10-26T01:28:49.540 INFO:teuthology.orchestra.run.smithi169.stdout:(65/143): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.0 MB/s | 217 kB 00:00 2023-10-26T01:28:49.582 INFO:teuthology.orchestra.run.smithi169.stdout:(66/143): python3-oauthlib-3.1.1-5.el9.noarch.r 4.2 MB/s | 174 kB 00:00 2023-10-26T01:28:49.624 INFO:teuthology.orchestra.run.smithi169.stdout:(67/143): python3-prettytable-0.7.2-27.el9.noar 502 kB/s | 41 kB 00:00 2023-10-26T01:28:49.644 INFO:teuthology.orchestra.run.smithi170.stdout:(57/143): python3-babel-2.9.1-2.el9.noarch.rpm 17 MB/s | 5.8 MB 00:00 2023-10-26T01:28:49.666 INFO:teuthology.orchestra.run.smithi169.stdout:(68/143): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.2 MB/s | 132 kB 00:00 2023-10-26T01:28:49.669 INFO:teuthology.orchestra.run.smithi170.stdout:(58/143): python3-devel-3.9.17-2.el9.x86_64.rpm 1.2 MB/s | 205 kB 00:00 2023-10-26T01:28:49.699 INFO:teuthology.orchestra.run.smithi169.stdout:(69/143): python3-pyasn1-modules-0.4.8-6.el9.no 2.8 MB/s | 211 kB 00:00 2023-10-26T01:28:49.711 INFO:teuthology.orchestra.run.smithi170.stdout:(59/143): python3-jsonpatch-1.21-16.el9.noarch. 607 kB/s | 24 kB 00:00 2023-10-26T01:28:49.750 INFO:teuthology.orchestra.run.smithi169.stdout:(70/143): python3-requests-oauthlib-1.3.0-12.el 525 kB/s | 43 kB 00:00 2023-10-26T01:28:49.753 INFO:teuthology.orchestra.run.smithi170.stdout:(60/143): python3-jinja2-2.11.3-4.el9.noarch.rp 2.7 MB/s | 228 kB 00:00 2023-10-26T01:28:49.779 INFO:teuthology.orchestra.run.smithi170.stdout:(61/143): python3-jsonpointer-2.0-4.el9.noarch. 281 kB/s | 18 kB 00:00 2023-10-26T01:28:49.804 INFO:teuthology.orchestra.run.smithi170.stdout:(62/143): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 144 kB 00:00 2023-10-26T01:28:49.829 INFO:teuthology.orchestra.run.smithi170.stdout:(63/143): python3-markupsafe-1.1.1-12.el9.x86_6 653 kB/s | 32 kB 00:00 2023-10-26T01:28:50.113 INFO:teuthology.orchestra.run.smithi170.stdout:(64/143): python3-numpy-1.20.1-5.el9.x86_64.rpm 16 MB/s | 4.8 MB 00:00 2023-10-26T01:28:50.138 INFO:teuthology.orchestra.run.smithi170.stdout:(65/143): python3-numpy-f2py-1.20.1-5.el9.x86_6 721 kB/s | 217 kB 00:00 2023-10-26T01:28:50.180 INFO:teuthology.orchestra.run.smithi170.stdout:(66/143): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 41 kB 00:00 2023-10-26T01:28:50.192 INFO:teuthology.orchestra.run.smithi169.stdout:(71/143): python3-scipy-1.6.2-8.el9.x86_64.rpm 32 MB/s | 16 MB 00:00 2023-10-26T01:28:50.205 INFO:teuthology.orchestra.run.smithi170.stdout:(67/143): python3-oauthlib-3.1.1-5.el9.noarch.r 2.6 MB/s | 174 kB 00:00 2023-10-26T01:28:50.240 INFO:teuthology.orchestra.run.smithi170.stdout:(68/143): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.2 MB/s | 132 kB 00:00 2023-10-26T01:28:50.274 INFO:teuthology.orchestra.run.smithi170.stdout:(69/143): python3-pyasn1-modules-0.4.8-6.el9.no 3.1 MB/s | 211 kB 00:00 2023-10-26T01:28:50.299 INFO:teuthology.orchestra.run.smithi170.stdout:(70/143): python3-requests-oauthlib-1.3.0-12.el 764 kB/s | 43 kB 00:00 2023-10-26T01:28:50.343 INFO:teuthology.orchestra.run.smithi169.stdout:(72/143): python3-toml-0.10.2-6.el9.noarch.rpm 67 kB/s | 39 kB 00:00 2023-10-26T01:28:50.650 INFO:teuthology.orchestra.run.smithi170.stdout:(71/143): python3-scipy-1.6.2-8.el9.x86_64.rpm 42 MB/s | 16 MB 00:00 2023-10-26T01:28:50.675 INFO:teuthology.orchestra.run.smithi170.stdout:(72/143): python3-toml-0.10.2-6.el9.noarch.rpm 107 kB/s | 39 kB 00:00 2023-10-26T01:28:50.725 INFO:teuthology.orchestra.run.smithi170.stdout:(73/143): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 56 kB 00:00 2023-10-26T01:28:50.759 INFO:teuthology.orchestra.run.smithi170.stdout:(74/143): socat-1.7.4.1-5.el9.x86_64.rpm 3.5 MB/s | 300 kB 00:00 2023-10-26T01:28:50.784 INFO:teuthology.orchestra.run.smithi169.stdout:(73/143): ceph-test-18.0.0-6863.g38499368.el9.x 5.2 MB/s | 41 MB 00:07 2023-10-26T01:28:50.785 INFO:teuthology.orchestra.run.smithi170.stdout:(75/143): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 63 kB 00:00 2023-10-26T01:28:50.809 INFO:teuthology.orchestra.run.smithi170.stdout:(76/143): fuse-2.9.9-15.el9.x86_64.rpm 1.6 MB/s | 79 kB 00:00 2023-10-26T01:28:50.817 INFO:teuthology.orchestra.run.smithi169.stdout:(74/143): socat-1.7.4.1-5.el9.x86_64.rpm 635 kB/s | 300 kB 00:00 2023-10-26T01:28:50.859 INFO:teuthology.orchestra.run.smithi169.stdout:(75/143): uuid-1.6.2-55.el9.x86_64.rpm 110 kB/s | 56 kB 00:00 2023-10-26T01:28:50.860 INFO:teuthology.orchestra.run.smithi170.stdout:(77/143): libgfortran-11.4.1-2.3.el9.x86_64.rpm 11 MB/s | 803 kB 00:00 2023-10-26T01:28:50.893 INFO:teuthology.orchestra.run.smithi170.stdout:(78/143): libquadmath-11.4.1-2.3.el9.x86_64.rpm 2.5 MB/s | 195 kB 00:00 2023-10-26T01:28:50.901 INFO:teuthology.orchestra.run.smithi169.stdout:(76/143): xmlstarlet-1.6.1-20.el9.x86_64.rpm 554 kB/s | 63 kB 00:00 2023-10-26T01:28:50.927 INFO:teuthology.orchestra.run.smithi170.stdout:(79/143): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.6 MB/s | 241 kB 00:00 2023-10-26T01:28:50.935 INFO:teuthology.orchestra.run.smithi169.stdout:(77/143): fuse-2.9.9-15.el9.x86_64.rpm 1.0 MB/s | 79 kB 00:00 2023-10-26T01:28:50.952 INFO:teuthology.orchestra.run.smithi170.stdout:(80/143): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 103 kB 00:00 2023-10-26T01:28:50.968 INFO:teuthology.orchestra.run.smithi169.stdout:(78/143): libgfortran-11.4.1-2.3.el9.x86_64.rpm 7.3 MB/s | 803 kB 00:00 2023-10-26T01:28:50.978 INFO:teuthology.orchestra.run.smithi170.stdout:(81/143): python3-pycparser-2.20-6.el9.noarch.r 2.5 MB/s | 124 kB 00:00 2023-10-26T01:28:51.002 INFO:teuthology.orchestra.run.smithi169.stdout:(79/143): libquadmath-11.4.1-2.3.el9.x86_64.rpm 2.1 MB/s | 195 kB 00:00 2023-10-26T01:28:51.003 INFO:teuthology.orchestra.run.smithi170.stdout:(82/143): python3-requests-2.25.1-7.el9.noarch. 2.3 MB/s | 113 kB 00:00 2023-10-26T01:28:51.038 INFO:teuthology.orchestra.run.smithi170.stdout:(83/143): python3-urllib3-1.26.5-4.el9.noarch.r 3.2 MB/s | 187 kB 00:00 2023-10-26T01:28:51.060 INFO:teuthology.orchestra.run.smithi169.stdout:(80/143): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.6 MB/s | 241 kB 00:00 2023-10-26T01:28:51.094 INFO:teuthology.orchestra.run.smithi169.stdout:(81/143): python3-ply-3.11-14.el9.noarch.rpm 840 kB/s | 103 kB 00:00 2023-10-26T01:28:51.119 INFO:teuthology.orchestra.run.smithi169.stdout:(82/143): python3-pycparser-2.20-6.el9.noarch.r 1.1 MB/s | 124 kB 00:00 2023-10-26T01:28:51.153 INFO:teuthology.orchestra.run.smithi169.stdout:(83/143): python3-requests-2.25.1-7.el9.noarch. 1.9 MB/s | 113 kB 00:00 2023-10-26T01:28:51.187 INFO:teuthology.orchestra.run.smithi169.stdout:(84/143): python3-urllib3-1.26.5-4.el9.noarch.r 2.0 MB/s | 187 kB 00:00 2023-10-26T01:28:51.345 INFO:teuthology.orchestra.run.smithi170.stdout:(84/143): python3-cryptography-36.0.1-4.el9.x86 3.6 MB/s | 1.2 MB 00:00 2023-10-26T01:28:51.487 INFO:teuthology.orchestra.run.smithi169.stdout:(85/143): elfutils-libelf-devel-0.189-3.el9.x86 75 kB/s | 22 kB 00:00 2023-10-26T01:28:51.579 INFO:teuthology.orchestra.run.smithi170.stdout:(85/143): elfutils-libelf-devel-0.189-3.el9.x86 96 kB/s | 22 kB 00:00 2023-10-26T01:28:51.837 INFO:teuthology.orchestra.run.smithi169.stdout:(86/143): jq-1.6-15.el9.x86_64.rpm 535 kB/s | 187 kB 00:00 2023-10-26T01:28:51.954 INFO:teuthology.orchestra.run.smithi169.stdout:(87/143): automake-1.16.2-8.el9.noarch.rpm 899 kB/s | 689 kB 00:00 2023-10-26T01:28:52.004 INFO:teuthology.orchestra.run.smithi169.stdout:(88/143): librabbitmq-0.11.0-7.el9.x86_64.rpm 272 kB/s | 45 kB 00:00 2023-10-26T01:28:52.079 INFO:teuthology.orchestra.run.smithi170.stdout:(86/143): ceph-test-18.0.0-6863.g38499368.el9.x 6.0 MB/s | 41 MB 00:06 2023-10-26T01:28:52.088 INFO:teuthology.orchestra.run.smithi169.stdout:(89/143): libuuid-devel-2.37.4-15.el9.x86_64.rp 192 kB/s | 26 kB 00:00 2023-10-26T01:28:52.121 INFO:teuthology.orchestra.run.smithi170.stdout:(87/143): automake-1.16.2-8.el9.noarch.rpm 636 kB/s | 689 kB 00:01 2023-10-26T01:28:52.138 INFO:teuthology.orchestra.run.smithi169.stdout:(90/143): python3-cryptography-36.0.1-4.el9.x86 1.2 MB/s | 1.2 MB 00:01 2023-10-26T01:28:52.230 INFO:teuthology.orchestra.run.smithi169.stdout:(91/143): python3-pytz-2021.1-5.el9.noarch.rpm 361 kB/s | 51 kB 00:00 2023-10-26T01:28:52.256 INFO:teuthology.orchestra.run.smithi169.stdout:(92/143): xfsprogs-devel-5.19.0-4.el9.x86_64.rp 741 kB/s | 86 kB 00:00 2023-10-26T01:28:52.346 INFO:teuthology.orchestra.run.smithi170.stdout:(88/143): jq-1.6-15.el9.x86_64.rpm 244 kB/s | 187 kB 00:00 2023-10-26T01:28:52.364 INFO:teuthology.orchestra.run.smithi169.stdout:(93/143): zlib-devel-1.2.11-40.el9.x86_64.rpm 340 kB/s | 45 kB 00:00 2023-10-26T01:28:52.371 INFO:teuthology.orchestra.run.smithi170.stdout:(89/143): librabbitmq-0.11.0-7.el9.x86_64.rpm 155 kB/s | 45 kB 00:00 2023-10-26T01:28:52.398 INFO:teuthology.orchestra.run.smithi169.stdout:(94/143): fmt-8.1.1-5.el9.x86_64.rpm 786 kB/s | 111 kB 00:00 2023-10-26T01:28:52.405 INFO:teuthology.orchestra.run.smithi170.stdout:(90/143): libuuid-devel-2.37.4-15.el9.x86_64.rp 90 kB/s | 26 kB 00:00 2023-10-26T01:28:52.465 INFO:teuthology.orchestra.run.smithi169.stdout:(95/143): gperftools-libs-2.9.1-2.el9.x86_64.rp 3.0 MB/s | 309 kB 00:00 2023-10-26T01:28:52.522 INFO:teuthology.orchestra.run.smithi170.stdout:(91/143): python3-pytz-2021.1-5.el9.noarch.rpm 341 kB/s | 51 kB 00:00 2023-10-26T01:28:52.598 INFO:teuthology.orchestra.run.smithi169.stdout:(96/143): libarrow-doc-9.0.0-9.el9.noarch.rpm 184 kB/s | 24 kB 00:00 2023-10-26T01:28:52.639 INFO:teuthology.orchestra.run.smithi170.stdout:(92/143): xfsprogs-devel-5.19.0-4.el9.x86_64.rp 370 kB/s | 86 kB 00:00 2023-10-26T01:28:52.665 INFO:teuthology.orchestra.run.smithi169.stdout:(97/143): liboath-2.6.7-2.el9.x86_64.rpm 739 kB/s | 49 kB 00:00 2023-10-26T01:28:52.774 INFO:teuthology.orchestra.run.smithi169.stdout:(98/143): openssl-devel-3.0.7-24.el9.x86_64.rpm 5.4 MB/s | 4.1 MB 00:00 2023-10-26T01:28:52.797 INFO:teuthology.orchestra.run.smithi170.stdout:(93/143): zlib-devel-1.2.11-40.el9.x86_64.rpm 164 kB/s | 45 kB 00:00 2023-10-26T01:28:52.799 INFO:teuthology.orchestra.run.smithi169.stdout:(99/143): libunwind-1.6.2-1.el9.x86_64.rpm 507 kB/s | 67 kB 00:00 2023-10-26T01:28:52.908 INFO:teuthology.orchestra.run.smithi169.stdout:(100/143): libarrow-9.0.0-9.el9.x86_64.rpm 8.7 MB/s | 4.4 MB 00:00 2023-10-26T01:28:52.949 INFO:teuthology.orchestra.run.smithi169.stdout:(101/143): python3-asyncssh-2.13.2-3.el9.noarch 3.6 MB/s | 548 kB 00:00 2023-10-26T01:28:52.992 INFO:teuthology.orchestra.run.smithi169.stdout:(102/143): python3-bcrypt-3.2.2-1.el9.x86_64.rp 526 kB/s | 43 kB 00:00 2023-10-26T01:28:53.064 INFO:teuthology.orchestra.run.smithi170.stdout:(94/143): fmt-8.1.1-5.el9.x86_64.rpm 261 kB/s | 111 kB 00:00 2023-10-26T01:28:53.067 INFO:teuthology.orchestra.run.smithi169.stdout:(103/143): python3-cachetools-4.2.4-1.el9.noarc 276 kB/s | 32 kB 00:00 2023-10-26T01:28:53.109 INFO:teuthology.orchestra.run.smithi169.stdout:(104/143): python3-certifi-2021.10.8-2.el9.noar 129 kB/s | 15 kB 00:00 2023-10-26T01:28:53.142 INFO:teuthology.orchestra.run.smithi169.stdout:(105/143): python3-cheroot-8.6.0-4.el9.noarch.r 2.3 MB/s | 172 kB 00:00 2023-10-26T01:28:53.193 INFO:teuthology.orchestra.run.smithi169.stdout:(106/143): parquet-libs-9.0.0-9.el9.x86_64.rpm 2.0 MB/s | 837 kB 00:00 2023-10-26T01:28:53.226 INFO:teuthology.orchestra.run.smithi169.stdout:(107/143): python3-google-auth-2.23.3-1.el9.noa 2.3 MB/s | 197 kB 00:00 2023-10-26T01:28:53.270 INFO:teuthology.orchestra.run.smithi169.stdout:(108/143): python3-jaraco-8.2.1-3.el9.noarch.rp 144 kB/s | 11 kB 00:00 2023-10-26T01:28:53.290 INFO:teuthology.orchestra.run.smithi170.stdout:(95/143): gperftools-libs-2.9.1-2.el9.x86_64.rp 629 kB/s | 309 kB 00:00 2023-10-26T01:28:53.302 INFO:teuthology.orchestra.run.smithi169.stdout:(109/143): python3-cherrypy-18.6.1-2.el9.noarch 1.8 MB/s | 358 kB 00:00 2023-10-26T01:28:53.344 INFO:teuthology.orchestra.run.smithi169.stdout:(110/143): python3-jaraco-classes-3.2.1-5.el9.n 152 kB/s | 18 kB 00:00 2023-10-26T01:28:53.365 INFO:teuthology.orchestra.run.smithi170.stdout:(96/143): libarrow-doc-9.0.0-9.el9.noarch.rpm 329 kB/s | 24 kB 00:00 2023-10-26T01:28:53.386 INFO:teuthology.orchestra.run.smithi169.stdout:(111/143): python3-jaraco-collections-3.0.0-8.e 199 kB/s | 23 kB 00:00 2023-10-26T01:28:53.428 INFO:teuthology.orchestra.run.smithi169.stdout:(112/143): python3-jaraco-functools-3.5.0-2.el9 235 kB/s | 19 kB 00:00 2023-10-26T01:28:53.457 INFO:teuthology.orchestra.run.smithi170.stdout:(97/143): liboath-2.6.7-2.el9.x86_64.rpm 535 kB/s | 49 kB 00:00 2023-10-26T01:28:53.470 INFO:teuthology.orchestra.run.smithi169.stdout:(113/143): python3-jaraco-text-3.2.0-6.el9.noar 158 kB/s | 20 kB 00:00 2023-10-26T01:28:53.512 INFO:teuthology.orchestra.run.smithi169.stdout:(114/143): python3-jwt+crypto-2.4.0-1.el9.noarc 72 kB/s | 9.0 kB 00:00 2023-10-26T01:28:53.557 INFO:teuthology.orchestra.run.smithi170.stdout:(98/143): libarrow-9.0.0-9.el9.x86_64.rpm 9.0 MB/s | 4.4 MB 00:00 2023-10-26T01:28:53.582 INFO:teuthology.orchestra.run.smithi170.stdout:(99/143): libunwind-1.6.2-1.el9.x86_64.rpm 542 kB/s | 67 kB 00:00 2023-10-26T01:28:53.587 INFO:teuthology.orchestra.run.smithi169.stdout:(115/143): python3-jwt-2.4.0-1.el9.noarch.rpm 353 kB/s | 41 kB 00:00 2023-10-26T01:28:53.629 INFO:teuthology.orchestra.run.smithi169.stdout:(116/143): python3-logutils-0.3.5-21.el9.noarch 399 kB/s | 46 kB 00:00 2023-10-26T01:28:53.655 INFO:teuthology.orchestra.run.smithi169.stdout:(117/143): python3-more-itertools-8.12.0-2.el9. 1.2 MB/s | 79 kB 00:00 2023-10-26T01:28:53.666 INFO:teuthology.orchestra.run.smithi170.stdout:(100/143): parquet-libs-9.0.0-9.el9.x86_64.rpm 7.6 MB/s | 837 kB 00:00 2023-10-26T01:28:53.705 INFO:teuthology.orchestra.run.smithi169.stdout:(118/143): python3-natsort-7.1.1-5.el9.noarch.r 776 kB/s | 58 kB 00:00 2023-10-26T01:28:53.741 INFO:teuthology.orchestra.run.smithi170.stdout:(101/143): python3-bcrypt-3.2.2-1.el9.x86_64.rp 588 kB/s | 43 kB 00:00 2023-10-26T01:28:53.746 INFO:teuthology.orchestra.run.smithi169.stdout:(119/143): python3-kubernetes-26.1.0-2.el9.noar 3.7 MB/s | 1.0 MB 00:00 2023-10-26T01:28:53.783 INFO:teuthology.orchestra.run.smithi170.stdout:(102/143): python3-asyncssh-2.13.2-3.el9.noarch 2.7 MB/s | 548 kB 00:00 2023-10-26T01:28:53.789 INFO:teuthology.orchestra.run.smithi169.stdout:(120/143): python3-portend-3.1.0-2.el9.noarch.r 198 kB/s | 16 kB 00:00 2023-10-26T01:28:53.814 INFO:teuthology.orchestra.run.smithi169.stdout:(121/143): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.3 MB/s | 90 kB 00:00 2023-10-26T01:28:53.825 INFO:teuthology.orchestra.run.smithi170.stdout:(103/143): python3-cachetools-4.2.4-1.el9.noarc 387 kB/s | 32 kB 00:00 2023-10-26T01:28:53.856 INFO:teuthology.orchestra.run.smithi169.stdout:(122/143): python3-pecan-1.4.2-3.el9.noarch.rpm 1.3 MB/s | 272 kB 00:00 2023-10-26T01:28:53.867 INFO:teuthology.orchestra.run.smithi170.stdout:(104/143): python3-certifi-2021.10.8-2.el9.noar 181 kB/s | 15 kB 00:00 2023-10-26T01:28:53.898 INFO:teuthology.orchestra.run.smithi169.stdout:(123/143): python3-repoze-lru-0.7-16.el9.noarch 285 kB/s | 31 kB 00:00 2023-10-26T01:28:53.909 INFO:teuthology.orchestra.run.smithi170.stdout:(105/143): python3-cheroot-8.6.0-4.el9.noarch.r 2.0 MB/s | 172 kB 00:00 2023-10-26T01:28:53.940 INFO:teuthology.orchestra.run.smithi169.stdout:(124/143): python3-rsa-4.9-2.el9.noarch.rpm 715 kB/s | 59 kB 00:00 2023-10-26T01:28:53.968 INFO:teuthology.orchestra.run.smithi170.stdout:(106/143): python3-cherrypy-18.6.1-2.el9.noarch 3.5 MB/s | 358 kB 00:00 2023-10-26T01:28:53.982 INFO:teuthology.orchestra.run.smithi169.stdout:(125/143): python3-tempora-5.0.0-2.el9.noarch.r 432 kB/s | 36 kB 00:00 2023-10-26T01:28:54.007 INFO:teuthology.orchestra.run.smithi169.stdout:(126/143): python3-routes-2.5.1-5.el9.noarch.rp 981 kB/s | 188 kB 00:00 2023-10-26T01:28:54.049 INFO:teuthology.orchestra.run.smithi169.stdout:(127/143): python3-typing-extensions-4.4.0-2.el 472 kB/s | 51 kB 00:00 2023-10-26T01:28:54.068 INFO:teuthology.orchestra.run.smithi170.stdout:(107/143): openssl-devel-3.0.7-24.el9.x86_64.rp 2.4 MB/s | 4.1 MB 00:01 2023-10-26T01:28:54.074 INFO:teuthology.orchestra.run.smithi169.stdout:(128/143): python3-websocket-client-1.2.3-2.el9 1.3 MB/s | 90 kB 00:00 2023-10-26T01:28:54.085 INFO:teuthology.orchestra.run.smithi170.stdout:(108/143): python3-jaraco-8.2.1-3.el9.noarch.rp 92 kB/s | 11 kB 00:00 2023-10-26T01:28:54.108 INFO:teuthology.orchestra.run.smithi169.stdout:(129/143): python3-webob-1.8.7-6.el9.noarch.rpm 1.8 MB/s | 230 kB 00:00 2023-10-26T01:28:54.118 INFO:teuthology.orchestra.run.smithi170.stdout:(109/143): python3-google-auth-2.23.3-1.el9.noa 945 kB/s | 197 kB 00:00 2023-10-26T01:28:54.135 INFO:teuthology.orchestra.run.smithi170.stdout:(110/143): python3-jaraco-classes-3.2.1-5.el9.n 267 kB/s | 18 kB 00:00 2023-10-26T01:28:54.150 INFO:teuthology.orchestra.run.smithi169.stdout:(130/143): python3-zc-lockfile-2.0-10.el9.noarc 268 kB/s | 20 kB 00:00 2023-10-26T01:28:54.183 INFO:teuthology.orchestra.run.smithi169.stdout:(131/143): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2023-10-26T01:28:54.236 INFO:teuthology.orchestra.run.smithi170.stdout:(111/143): python3-jaraco-functools-3.5.0-2.el9 167 kB/s | 19 kB 00:00 2023-10-26T01:28:54.286 INFO:teuthology.orchestra.run.smithi170.stdout:(112/143): python3-jaraco-text-3.2.0-6.el9.noar 247 kB/s | 20 kB 00:00 2023-10-26T01:28:54.325 INFO:teuthology.orchestra.run.smithi169.stdout:(132/143): re2-20211101-3.el9.x86_64.rpm 888 kB/s | 192 kB 00:00 2023-10-26T01:28:54.328 INFO:teuthology.orchestra.run.smithi170.stdout:(113/143): python3-jwt+crypto-2.4.0-1.el9.noarc 99 kB/s | 9.0 kB 00:00 2023-10-26T01:28:54.370 INFO:teuthology.orchestra.run.smithi170.stdout:(114/143): python3-jwt-2.4.0-1.el9.noarch.rpm 492 kB/s | 41 kB 00:00 2023-10-26T01:28:54.395 INFO:teuthology.orchestra.run.smithi170.stdout:(115/143): python3-jaraco-collections-3.0.0-8.e 75 kB/s | 23 kB 00:00 2023-10-26T01:28:54.454 INFO:teuthology.orchestra.run.smithi170.stdout:(116/143): python3-kubernetes-26.1.0-2.el9.noar 8.2 MB/s | 1.0 MB 00:00 2023-10-26T01:28:54.476 INFO:teuthology.orchestra.run.smithi169.stdout:(133/143): thrift-0.15.0-2.el9.x86_64.rpm 4.9 MB/s | 1.6 MB 00:00 2023-10-26T01:28:54.479 INFO:teuthology.orchestra.run.smithi170.stdout:(117/143): python3-logutils-0.3.5-21.el9.noarch 427 kB/s | 46 kB 00:00 2023-10-26T01:28:54.509 INFO:teuthology.orchestra.run.smithi169.stdout:(134/143): libacl-devel-2.3.1-4.el9.x86_64.rpm 2.0 MB/s | 68 kB 00:00 2023-10-26T01:28:54.546 INFO:teuthology.orchestra.run.smithi170.stdout:(118/143): python3-natsort-7.1.1-5.el9.noarch.r 633 kB/s | 58 kB 00:00 2023-10-26T01:28:54.588 INFO:teuthology.orchestra.run.smithi170.stdout:(119/143): python3-more-itertools-8.12.0-2.el9. 411 kB/s | 79 kB 00:00 2023-10-26T01:28:54.621 INFO:teuthology.orchestra.run.smithi170.stdout:(120/143): python3-pecan-1.4.2-3.el9.noarch.rpm 1.9 MB/s | 272 kB 00:00 2023-10-26T01:28:54.626 INFO:teuthology.orchestra.run.smithi169.stdout:(135/143): NetworkManager-1.45.5-1.el9.x86_64.r 19 MB/s | 2.2 MB 00:00 2023-10-26T01:28:54.647 INFO:teuthology.orchestra.run.smithi170.stdout:(121/143): python3-portend-3.1.0-2.el9.noarch.r 165 kB/s | 16 kB 00:00 2023-10-26T01:28:54.722 INFO:teuthology.orchestra.run.smithi170.stdout:(122/143): python3-repoze-lru-0.7-16.el9.noarch 309 kB/s | 31 kB 00:00 2023-10-26T01:28:54.743 INFO:teuthology.orchestra.run.smithi169.stdout:(136/143): NetworkManager-libnm-1.45.5-1.el9.x8 15 MB/s | 1.8 MB 00:00 2023-10-26T01:28:54.780 INFO:teuthology.orchestra.run.smithi170.stdout:(123/143): python3-pyOpenSSL-21.0.0-1.el9.noarc 469 kB/s | 90 kB 00:00 2023-10-26T01:28:54.793 INFO:teuthology.orchestra.run.smithi169.stdout:(137/143): NetworkManager-team-1.45.5-1.el9.x86 853 kB/s | 42 kB 00:00 2023-10-26T01:28:54.814 INFO:teuthology.orchestra.run.smithi170.stdout:(124/143): python3-routes-2.5.1-5.el9.noarch.rp 1.1 MB/s | 188 kB 00:00 2023-10-26T01:28:54.839 INFO:teuthology.orchestra.run.smithi170.stdout:(125/143): python3-rsa-4.9-2.el9.noarch.rpm 507 kB/s | 59 kB 00:00 2023-10-26T01:28:54.873 INFO:teuthology.orchestra.run.smithi170.stdout:(126/143): python3-tempora-5.0.0-2.el9.noarch.r 390 kB/s | 36 kB 00:00 2023-10-26T01:28:54.877 INFO:teuthology.orchestra.run.smithi169.stdout:(138/143): librbd1-18.0.0-6863.g38499368.el9.x8 5.5 MB/s | 3.0 MB 00:00 2023-10-26T01:28:54.907 INFO:teuthology.orchestra.run.smithi170.stdout:(127/143): python3-typing-extensions-4.4.0-2.el 558 kB/s | 51 kB 00:00 2023-10-26T01:28:54.910 INFO:teuthology.orchestra.run.smithi169.stdout:(139/143): NetworkManager-tui-1.45.5-1.el9.x86_ 2.1 MB/s | 245 kB 00:00 2023-10-26T01:28:54.936 INFO:teuthology.orchestra.run.smithi169.stdout:(140/143): acl-2.3.1-4.el9.x86_64.rpm 1.2 MB/s | 69 kB 00:00 2023-10-26T01:28:54.940 INFO:teuthology.orchestra.run.smithi170.stdout:(128/143): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2023-10-26T01:28:54.986 INFO:teuthology.orchestra.run.smithi169.stdout:(141/143): libacl-2.3.1-4.el9.x86_64.rpm 295 kB/s | 22 kB 00:00 2023-10-26T01:28:54.999 INFO:teuthology.orchestra.run.smithi170.stdout:(129/143): python3-websocket-client-1.2.3-2.el9 718 kB/s | 90 kB 00:00 2023-10-26T01:28:55.020 INFO:teuthology.orchestra.run.smithi169.stdout:(142/143): xfsdump-3.1.12-4.el9.x86_64.rpm 3.8 MB/s | 321 kB 00:00 2023-10-26T01:28:55.091 INFO:teuthology.orchestra.run.smithi170.stdout:(130/143): python3-werkzeug-2.0.3-3.el9.1.noarc 2.3 MB/s | 427 kB 00:00 2023-10-26T01:28:55.116 INFO:teuthology.orchestra.run.smithi170.stdout:(131/143): python3-zc-lockfile-2.0-10.el9.noarc 114 kB/s | 20 kB 00:00 2023-10-26T01:28:55.175 INFO:teuthology.orchestra.run.smithi170.stdout:(132/143): re2-20211101-3.el9.x86_64.rpm 2.3 MB/s | 192 kB 00:00 2023-10-26T01:28:55.278 INFO:teuthology.orchestra.run.smithi169.stdout:(143/143): librados2-18.0.0-6863.g38499368.el9. 3.0 MB/s | 3.3 MB 00:01 2023-10-26T01:28:55.286 INFO:teuthology.orchestra.run.smithi169.stdout:-------------------------------------------------------------------------------- 2023-10-26T01:28:55.287 INFO:teuthology.orchestra.run.smithi169.stdout:Total 12 MB/s | 200 MB 00:17 2023-10-26T01:28:55.425 INFO:teuthology.orchestra.run.smithi170.stdout:(133/143): thrift-0.15.0-2.el9.x86_64.rpm 4.8 MB/s | 1.6 MB 00:00 2023-10-26T01:28:55.467 INFO:teuthology.orchestra.run.smithi170.stdout:(134/143): libacl-devel-2.3.1-4.el9.x86_64.rpm 1.6 MB/s | 68 kB 00:00 2023-10-26T01:28:55.559 INFO:teuthology.orchestra.run.smithi170.stdout:(135/143): NetworkManager-1.45.5-1.el9.x86_64.r 25 MB/s | 2.2 MB 00:00 2023-10-26T01:28:55.643 INFO:teuthology.orchestra.run.smithi170.stdout:(136/143): librados2-18.0.0-6863.g38499368.el9. 6.3 MB/s | 3.3 MB 00:00 2023-10-26T01:28:55.710 INFO:teuthology.orchestra.run.smithi170.stdout:(137/143): NetworkManager-libnm-1.45.5-1.el9.x8 12 MB/s | 1.8 MB 00:00 2023-10-26T01:28:55.719 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction check 2023-10-26T01:28:55.735 INFO:teuthology.orchestra.run.smithi170.stdout:(138/143): NetworkManager-team-1.45.5-1.el9.x86 534 kB/s | 42 kB 00:00 2023-10-26T01:28:55.768 INFO:teuthology.orchestra.run.smithi170.stdout:(139/143): NetworkManager-tui-1.45.5-1.el9.x86_ 4.2 MB/s | 245 kB 00:00 2023-10-26T01:28:55.785 INFO:teuthology.orchestra.run.smithi170.stdout:(140/143): acl-2.3.1-4.el9.x86_64.rpm 1.4 MB/s | 69 kB 00:00 2023-10-26T01:28:55.811 INFO:teuthology.orchestra.run.smithi170.stdout:(141/143): libacl-2.3.1-4.el9.x86_64.rpm 542 kB/s | 22 kB 00:00 2023-10-26T01:28:55.844 INFO:teuthology.orchestra.run.smithi170.stdout:(142/143): xfsdump-3.1.12-4.el9.x86_64.rpm 5.5 MB/s | 321 kB 00:00 2023-10-26T01:28:55.922 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction check succeeded. 2023-10-26T01:28:55.922 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction test 2023-10-26T01:28:55.961 INFO:teuthology.orchestra.run.smithi170.stdout:(143/143): librbd1-18.0.0-6863.g38499368.el9.x8 3.9 MB/s | 3.0 MB 00:00 2023-10-26T01:28:55.975 INFO:teuthology.orchestra.run.smithi170.stdout:-------------------------------------------------------------------------------- 2023-10-26T01:28:55.975 INFO:teuthology.orchestra.run.smithi170.stdout:Total 13 MB/s | 200 MB 00:15 2023-10-26T01:28:56.409 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction check 2023-10-26T01:28:56.609 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction check succeeded. 2023-10-26T01:28:56.609 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction test 2023-10-26T01:28:58.073 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction test succeeded. 2023-10-26T01:28:58.073 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction 2023-10-26T01:28:58.767 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction test succeeded. 2023-10-26T01:28:58.768 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction 2023-10-26T01:29:04.166 INFO:teuthology.orchestra.run.smithi169.stdout: Preparing : 1/1 2023-10-26T01:29:04.262 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/153 2023-10-26T01:29:04.299 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/153 2023-10-26T01:29:04.340 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/153 2023-10-26T01:29:04.478 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/153 2023-10-26T01:29:04.871 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 5/153 2023-10-26T01:29:04.931 INFO:teuthology.orchestra.run.smithi170.stdout: Preparing : 1/1 2023-10-26T01:29:04.932 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : librados2-2:18.0.0-6863.g38499368.el9.x86_64 6/153 2023-10-26T01:29:05.047 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/153 2023-10-26T01:29:05.082 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/153 2023-10-26T01:29:05.107 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: librados2-2:18.0.0-6863.g38499368.el9.x86_64 6/153 2023-10-26T01:29:05.110 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/153 2023-10-26T01:29:05.141 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libcephfs2-2:18.0.0-6863.g38499368.el9.x86_64 7/153 2023-10-26T01:29:05.220 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/153 2023-10-26T01:29:05.280 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: libcephfs2-2:18.0.0-6863.g38499368.el9.x86_64 7/153 2023-10-26T01:29:05.363 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-rados-2:18.0.0-6863.g38499368.el9.x86_64 8/153 2023-10-26T01:29:05.416 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 9/153 2023-10-26T01:29:05.610 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 5/153 2023-10-26T01:29:05.657 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : librados2-2:18.0.0-6863.g38499368.el9.x86_64 6/153 2023-10-26T01:29:05.673 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 10/153 2023-10-26T01:29:05.746 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : postgresql-13.11-1.el9.x86_64 11/153 2023-10-26T01:29:05.771 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libcephsqlite-2:18.0.0-6863.g38499368.el9.x86_64 12/153 2023-10-26T01:29:05.807 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: librados2-2:18.0.0-6863.g38499368.el9.x86_64 6/153 2023-10-26T01:29:05.841 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libcephfs2-2:18.0.0-6863.g38499368.el9.x86_64 7/153 2023-10-26T01:29:05.921 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: libcephsqlite-2:18.0.0-6863.g38499368.el9.x86_64 12/153 2023-10-26T01:29:05.956 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libradosstriper1-2:18.0.0-6863.g38499368.el9.x86 13/153 2023-10-26T01:29:05.961 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: libcephfs2-2:18.0.0-6863.g38499368.el9.x86_64 7/153 2023-10-26T01:29:06.022 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-rados-2:18.0.0-6863.g38499368.el9.x86_64 8/153 2023-10-26T01:29:06.065 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: libradosstriper1-2:18.0.0-6863.g38499368.el9.x86 13/153 2023-10-26T01:29:06.069 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 9/153 2023-10-26T01:29:06.342 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 10/153 2023-10-26T01:29:06.396 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : postgresql-13.11-1.el9.x86_64 11/153 2023-10-26T01:29:06.420 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libcephsqlite-2:18.0.0-6863.g38499368.el9.x86_64 12/153 2023-10-26T01:29:06.465 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 14/153 2023-10-26T01:29:06.567 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: libcephsqlite-2:18.0.0-6863.g38499368.el9.x86_64 12/153 2023-10-26T01:29:06.605 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libradosstriper1-2:18.0.0-6863.g38499368.el9.x86 13/153 2023-10-26T01:29:06.628 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : NetworkManager-libnm-1:1.45.5-1.el9.x86_64 15/153 2023-10-26T01:29:06.688 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: libradosstriper1-2:18.0.0-6863.g38499368.el9.x86 13/153 2023-10-26T01:29:06.716 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: NetworkManager-libnm-1:1.45.5-1.el9.x86_64 15/153 2023-10-26T01:29:06.918 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: NetworkManager-1:1.45.5-1.el9.x86_64 16/153 2023-10-26T01:29:06.964 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : NetworkManager-1:1.45.5-1.el9.x86_64 16/153 2023-10-26T01:29:07.078 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 14/153 2023-10-26T01:29:07.195 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : NetworkManager-libnm-1:1.45.5-1.el9.x86_64 15/153 2023-10-26T01:29:07.284 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: NetworkManager-libnm-1:1.45.5-1.el9.x86_64 15/153 2023-10-26T01:29:07.395 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: NetworkManager-1:1.45.5-1.el9.x86_64 16/153 2023-10-26T01:29:07.436 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 17/153 2023-10-26T01:29:07.485 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : liboath-2.6.7-2.el9.x86_64 18/153 2023-10-26T01:29:07.487 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: NetworkManager-1:1.45.5-1.el9.x86_64 16/153 2023-10-26T01:29:07.573 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : NetworkManager-1:1.45.5-1.el9.x86_64 16/153 2023-10-26T01:29:07.591 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 19/153 2023-10-26T01:29:07.667 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-urllib3-1.26.5-4.el9.noarch 20/153 2023-10-26T01:29:07.716 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-requests-2.25.1-7.el9.noarch 21/153 2023-10-26T01:29:07.772 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libquadmath-11.4.1-2.3.el9.x86_64 22/153 2023-10-26T01:29:07.878 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libgfortran-11.4.1-2.3.el9.x86_64 23/153 2023-10-26T01:29:07.934 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/153 2023-10-26T01:29:07.968 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/153 2023-10-26T01:29:08.020 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: NetworkManager-1:1.45.5-1.el9.x86_64 16/153 2023-10-26T01:29:08.078 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/153 2023-10-26T01:29:08.091 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 17/153 2023-10-26T01:29:08.114 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-ceph-common-2:18.0.0-6863.g38499368.el9. 27/153 2023-10-26T01:29:08.159 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : liboath-2.6.7-2.el9.x86_64 18/153 2023-10-26T01:29:08.164 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-ceph-argparse-2:18.0.0-6863.g38499368.el 28/153 2023-10-26T01:29:08.266 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cephfs-2:18.0.0-6863.g38499368.el9.x86_6 29/153 2023-10-26T01:29:08.292 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 19/153 2023-10-26T01:29:08.368 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-urllib3-1.26.5-4.el9.noarch 20/153 2023-10-26T01:29:08.401 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-requests-2.25.1-7.el9.noarch 21/153 2023-10-26T01:29:08.450 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 30/153 2023-10-26T01:29:08.458 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libquadmath-11.4.1-2.3.el9.x86_64 22/153 2023-10-26T01:29:08.515 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 31/153 2023-10-26T01:29:08.565 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libgfortran-11.4.1-2.3.el9.x86_64 23/153 2023-10-26T01:29:08.620 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/153 2023-10-26T01:29:08.678 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/153 2023-10-26T01:29:08.807 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/153 2023-10-26T01:29:08.849 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-ceph-common-2:18.0.0-6863.g38499368.el9. 27/153 2023-10-26T01:29:08.899 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-ceph-argparse-2:18.0.0-6863.g38499368.el 28/153 2023-10-26T01:29:09.004 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-cephfs-2:18.0.0-6863.g38499368.el9.x86_6 29/153 2023-10-26T01:29:09.190 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 30/153 2023-10-26T01:29:09.219 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-rsa-4.9-2.el9.noarch 32/153 2023-10-26T01:29:09.278 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 31/153 2023-10-26T01:29:09.313 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 33/153 2023-10-26T01:29:09.434 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jinja2-2.11.3-4.el9.noarch 34/153 2023-10-26T01:29:09.466 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : librados-devel-2:18.0.0-6863.g38499368.el9.x86_6 35/153 2023-10-26T01:29:09.504 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 36/153 2023-10-26T01:29:09.655 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : re2-1:20211101-3.el9.x86_64 37/153 2023-10-26T01:29:09.862 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libarrow-9.0.0-9.el9.x86_64 38/153 2023-10-26T01:29:09.955 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 39/153 2023-10-26T01:29:09.971 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-rsa-4.9-2.el9.noarch 32/153 2023-10-26T01:29:10.066 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 33/153 2023-10-26T01:29:10.070 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 40/153 2023-10-26T01:29:10.108 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 41/153 2023-10-26T01:29:10.119 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jinja2-2.11.3-4.el9.noarch 34/153 2023-10-26T01:29:10.147 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 42/153 2023-10-26T01:29:10.157 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : librados-devel-2:18.0.0-6863.g38499368.el9.x86_6 35/153 2023-10-26T01:29:10.205 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 36/153 2023-10-26T01:29:10.266 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 43/153 2023-10-26T01:29:10.352 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : re2-1:20211101-3.el9.x86_64 37/153 2023-10-26T01:29:10.670 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libarrow-9.0.0-9.el9.x86_64 38/153 2023-10-26T01:29:10.685 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 44/153 2023-10-26T01:29:10.756 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 45/153 2023-10-26T01:29:10.765 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 39/153 2023-10-26T01:29:10.863 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 40/153 2023-10-26T01:29:10.958 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 46/153 2023-10-26T01:29:10.989 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 41/153 2023-10-26T01:29:11.007 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 47/153 2023-10-26T01:29:11.062 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 48/153 2023-10-26T01:29:11.193 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 49/153 2023-10-26T01:29:11.251 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-google-auth-1:2.23.3-1.el9.noarch 50/153 2023-10-26T01:29:11.317 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 51/153 2023-10-26T01:29:11.355 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 52/153 2023-10-26T01:29:11.393 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 42/153 2023-10-26T01:29:11.396 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libarrow-doc-9.0.0-9.el9.noarch 53/153 2023-10-26T01:29:11.438 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : fmt-8.1.1-5.el9.x86_64 54/153 2023-10-26T01:29:11.492 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 55/153 2023-10-26T01:29:11.493 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 43/153 2023-10-26T01:29:11.556 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 44/153 2023-10-26T01:29:11.622 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 45/153 2023-10-26T01:29:11.644 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libuuid-devel-2.37.4-15.el9.x86_64 56/153 2023-10-26T01:29:11.689 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : automake-1.16.2-8.el9.noarch 57/153 2023-10-26T01:29:11.759 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-ply-3.11-14.el9.noarch 58/153 2023-10-26T01:29:11.826 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 46/153 2023-10-26T01:29:11.845 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 59/153 2023-10-26T01:29:11.874 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 47/153 2023-10-26T01:29:11.960 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 48/153 2023-10-26T01:29:12.089 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 49/153 2023-10-26T01:29:12.139 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 60/153 2023-10-26T01:29:12.144 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-google-auth-1:2.23.3-1.el9.noarch 50/153 2023-10-26T01:29:12.201 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 61/153 2023-10-26T01:29:12.201 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 51/153 2023-10-26T01:29:12.240 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 52/153 2023-10-26T01:29:12.280 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libarrow-doc-9.0.0-9.el9.noarch 53/153 2023-10-26T01:29:12.323 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : fmt-8.1.1-5.el9.x86_64 54/153 2023-10-26T01:29:12.376 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 62/153 2023-10-26T01:29:12.377 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 55/153 2023-10-26T01:29:12.437 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 63/153 2023-10-26T01:29:12.469 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 64/153 2023-10-26T01:29:12.509 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 65/153 2023-10-26T01:29:12.531 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libuuid-devel-2.37.4-15.el9.x86_64 56/153 2023-10-26T01:29:12.552 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 66/153 2023-10-26T01:29:12.589 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : automake-1.16.2-8.el9.noarch 57/153 2023-10-26T01:29:12.594 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : fuse-2.9.9-15.el9.x86_64 67/153 2023-10-26T01:29:12.650 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/153 2023-10-26T01:29:12.659 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-ply-3.11-14.el9.noarch 58/153 2023-10-26T01:29:12.729 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 59/153 2023-10-26T01:29:12.732 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/153 2023-10-26T01:29:12.790 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 70/153 2023-10-26T01:29:12.830 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/153 2023-10-26T01:29:12.981 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/153 2023-10-26T01:29:12.989 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 60/153 2023-10-26T01:29:13.035 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 73/153 2023-10-26T01:29:13.041 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 61/153 2023-10-26T01:29:13.068 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 74/153 2023-10-26T01:29:13.121 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 75/153 2023-10-26T01:29:13.163 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 76/153 2023-10-26T01:29:13.241 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 62/153 2023-10-26T01:29:13.293 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 63/153 2023-10-26T01:29:13.308 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 77/153 2023-10-26T01:29:13.319 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 64/153 2023-10-26T01:29:13.345 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 78/153 2023-10-26T01:29:13.357 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 65/153 2023-10-26T01:29:13.400 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 66/153 2023-10-26T01:29:13.444 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : fuse-2.9.9-15.el9.x86_64 67/153 2023-10-26T01:29:13.473 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/153 2023-10-26T01:29:13.493 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/153 2023-10-26T01:29:13.552 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 80/153 2023-10-26T01:29:13.572 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/153 2023-10-26T01:29:13.630 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 70/153 2023-10-26T01:29:13.696 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/153 2023-10-26T01:29:13.828 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/153 2023-10-26T01:29:13.877 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 73/153 2023-10-26T01:29:13.920 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 74/153 2023-10-26T01:29:13.990 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 75/153 2023-10-26T01:29:14.035 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 76/153 2023-10-26T01:29:14.182 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 77/153 2023-10-26T01:29:14.257 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 78/153 2023-10-26T01:29:14.402 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/153 2023-10-26T01:29:14.472 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 80/153 2023-10-26T01:29:14.514 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 81/153 2023-10-26T01:29:14.576 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 82/153 2023-10-26T01:29:14.609 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 83/153 2023-10-26T01:29:14.739 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 84/153 2023-10-26T01:29:14.776 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 85/153 2023-10-26T01:29:15.001 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : openblas-0.3.21-2.el9.x86_64 86/153 2023-10-26T01:29:15.040 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 87/153 2023-10-26T01:29:15.132 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 88/153 2023-10-26T01:29:15.478 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 81/153 2023-10-26T01:29:15.538 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 82/153 2023-10-26T01:29:15.572 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 83/153 2023-10-26T01:29:15.703 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 84/153 2023-10-26T01:29:15.738 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 85/153 2023-10-26T01:29:15.965 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : openblas-0.3.21-2.el9.x86_64 86/153 2023-10-26T01:29:15.971 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 89/153 2023-10-26T01:29:16.036 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 87/153 2023-10-26T01:29:16.103 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 90/153 2023-10-26T01:29:16.152 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 88/153 2023-10-26T01:29:16.994 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 89/153 2023-10-26T01:29:17.125 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 90/153 2023-10-26T01:29:17.688 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 91/153 2023-10-26T01:29:17.769 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 92/153 2023-10-26T01:29:17.797 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 93/153 2023-10-26T01:29:17.828 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : jq-1.6-15.el9.x86_64 94/153 2023-10-26T01:29:17.888 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 95/153 2023-10-26T01:29:18.269 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 96/153 2023-10-26T01:29:18.309 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : librbd1-2:18.0.0-6863.g38499368.el9.x86_64 97/153 2023-10-26T01:29:18.429 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: librbd1-2:18.0.0-6863.g38499368.el9.x86_64 97/153 2023-10-26T01:29:18.466 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-rbd-2:18.0.0-6863.g38499368.el9.x86_64 98/153 2023-10-26T01:29:18.524 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 99/153 2023-10-26T01:29:18.766 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 91/153 2023-10-26T01:29:18.864 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 92/153 2023-10-26T01:29:18.902 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 93/153 2023-10-26T01:29:18.933 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : jq-1.6-15.el9.x86_64 94/153 2023-10-26T01:29:18.998 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 95/153 2023-10-26T01:29:19.017 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : parquet-libs-9.0.0-9.el9.x86_64 100/153 2023-10-26T01:29:19.053 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : librgw2-2:18.0.0-6863.g38499368.el9.x86_64 101/153 2023-10-26T01:29:19.145 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: librgw2-2:18.0.0-6863.g38499368.el9.x86_64 101/153 2023-10-26T01:29:19.172 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-rgw-2:18.0.0-6863.g38499368.el9.x86_64 102/153 2023-10-26T01:29:19.367 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 96/153 2023-10-26T01:29:19.396 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : librbd1-2:18.0.0-6863.g38499368.el9.x86_64 97/153 2023-10-26T01:29:19.553 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: librbd1-2:18.0.0-6863.g38499368.el9.x86_64 97/153 2023-10-26T01:29:19.586 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-rbd-2:18.0.0-6863.g38499368.el9.x86_64 98/153 2023-10-26T01:29:19.643 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 99/153 2023-10-26T01:29:20.123 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : parquet-libs-9.0.0-9.el9.x86_64 100/153 2023-10-26T01:29:20.166 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : librgw2-2:18.0.0-6863.g38499368.el9.x86_64 101/153 2023-10-26T01:29:20.267 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: librgw2-2:18.0.0-6863.g38499368.el9.x86_64 101/153 2023-10-26T01:29:20.301 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-rgw-2:18.0.0-6863.g38499368.el9.x86_64 102/153 2023-10-26T01:29:21.660 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-common-2:18.0.0-6863.g38499368.el9.x86_64 103/153 2023-10-26T01:29:21.702 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-common-2:18.0.0-6863.g38499368.el9.x86_64 103/153 2023-10-26T01:29:22.330 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-common-2:18.0.0-6863.g38499368.el9.x86_64 103/153 2023-10-26T01:29:22.372 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-base-2:18.0.0-6863.g38499368.el9.x86_64 104/153 2023-10-26T01:29:22.525 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-base-2:18.0.0-6863.g38499368.el9.x86_64 104/153 2023-10-26T01:29:22.525 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2023-10-26T01:29:22.526 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2023-10-26T01:29:22.526 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:29:22.573 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-selinux-2:18.0.0-6863.g38499368.el9.x86_64 105/153 2023-10-26T01:29:22.811 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-common-2:18.0.0-6863.g38499368.el9.x86_64 103/153 2023-10-26T01:29:22.848 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-common-2:18.0.0-6863.g38499368.el9.x86_64 103/153 2023-10-26T01:29:23.479 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-common-2:18.0.0-6863.g38499368.el9.x86_64 103/153 2023-10-26T01:29:23.526 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-base-2:18.0.0-6863.g38499368.el9.x86_64 104/153 2023-10-26T01:29:23.708 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-base-2:18.0.0-6863.g38499368.el9.x86_64 104/153 2023-10-26T01:29:23.708 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2023-10-26T01:29:23.708 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2023-10-26T01:29:23.709 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-26T01:29:23.753 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-selinux-2:18.0.0-6863.g38499368.el9.x86_64 105/153 2023-10-26T01:29:35.138 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-selinux-2:18.0.0-6863.g38499368.el9.x86_64 105/153 2023-10-26T01:29:35.139 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /sys 2023-10-26T01:29:35.139 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /proc 2023-10-26T01:29:35.139 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /mnt 2023-10-26T01:29:35.139 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /var/tmp 2023-10-26T01:29:35.140 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /home 2023-10-26T01:29:35.140 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /root 2023-10-26T01:29:35.140 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /tmp 2023-10-26T01:29:35.140 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:29:35.351 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-volume-2:18.0.0-6863.g38499368.el9.noarch 106/153 2023-10-26T01:29:35.395 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-volume-2:18.0.0-6863.g38499368.el9.noarch 106/153 2023-10-26T01:29:35.395 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-26T01:29:35.395 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2023-10-26T01:29:35.396 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:29:36.525 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-selinux-2:18.0.0-6863.g38499368.el9.x86_64 105/153 2023-10-26T01:29:36.526 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /sys 2023-10-26T01:29:36.526 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /proc 2023-10-26T01:29:36.526 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /mnt 2023-10-26T01:29:36.526 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /var/tmp 2023-10-26T01:29:36.526 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /home 2023-10-26T01:29:36.527 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /root 2023-10-26T01:29:36.527 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /tmp 2023-10-26T01:29:36.527 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-26T01:29:36.749 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-volume-2:18.0.0-6863.g38499368.el9.noarch 106/153 2023-10-26T01:29:36.801 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-volume-2:18.0.0-6863.g38499368.el9.noarch 106/153 2023-10-26T01:29:36.801 INFO:teuthology.orchestra.run.smithi170.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-26T01:29:36.802 INFO:teuthology.orchestra.run.smithi170.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2023-10-26T01:29:36.802 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-26T01:29:37.079 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-osd-2:18.0.0-6863.g38499368.el9.x86_64 107/153 2023-10-26T01:29:37.170 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-osd-2:18.0.0-6863.g38499368.el9.x86_64 107/153 2023-10-26T01:29:37.170 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-26T01:29:37.170 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2023-10-26T01:29:37.170 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-10-26T01:29:37.171 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-10-26T01:29:37.171 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:29:37.388 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mds-2:18.0.0-6863.g38499368.el9.x86_64 108/153 2023-10-26T01:29:37.466 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mds-2:18.0.0-6863.g38499368.el9.x86_64 108/153 2023-10-26T01:29:37.467 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-26T01:29:37.467 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2023-10-26T01:29:37.467 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-10-26T01:29:37.467 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-10-26T01:29:37.468 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:29:37.963 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mon-2:18.0.0-6863.g38499368.el9.x86_64 109/153 2023-10-26T01:29:38.058 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mon-2:18.0.0-6863.g38499368.el9.x86_64 109/153 2023-10-26T01:29:38.059 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-26T01:29:38.059 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2023-10-26T01:29:38.059 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-10-26T01:29:38.059 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-10-26T01:29:38.060 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:29:38.493 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-osd-2:18.0.0-6863.g38499368.el9.x86_64 107/153 2023-10-26T01:29:38.570 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-osd-2:18.0.0-6863.g38499368.el9.x86_64 107/153 2023-10-26T01:29:38.571 INFO:teuthology.orchestra.run.smithi170.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-26T01:29:38.571 INFO:teuthology.orchestra.run.smithi170.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2023-10-26T01:29:38.571 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-10-26T01:29:38.571 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-10-26T01:29:38.571 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-26T01:29:38.696 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: cephadm-2:18.0.0-6863.g38499368.el9.noarch 110/153 2023-10-26T01:29:38.735 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : cephadm-2:18.0.0-6863.g38499368.el9.noarch 110/153 2023-10-26T01:29:38.781 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-6863.g38499368.e 111/153 2023-10-26T01:29:38.802 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mds-2:18.0.0-6863.g38499368.el9.x86_64 108/153 2023-10-26T01:29:38.874 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mds-2:18.0.0-6863.g38499368.el9.x86_64 108/153 2023-10-26T01:29:38.874 INFO:teuthology.orchestra.run.smithi170.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-26T01:29:38.875 INFO:teuthology.orchestra.run.smithi170.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2023-10-26T01:29:38.875 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-10-26T01:29:38.875 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-10-26T01:29:38.875 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-26T01:29:38.893 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-6863.g38499368. 112/153 2023-10-26T01:29:38.919 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-6863.g38499368.el9.noa 113/153 2023-10-26T01:29:39.251 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-6863.g38499368.el9.noa 113/153 2023-10-26T01:29:39.306 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-6863.g38499368.el9.n 114/153 2023-10-26T01:29:39.361 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mon-2:18.0.0-6863.g38499368.el9.x86_64 109/153 2023-10-26T01:29:39.431 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mon-2:18.0.0-6863.g38499368.el9.x86_64 109/153 2023-10-26T01:29:39.432 INFO:teuthology.orchestra.run.smithi170.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-26T01:29:39.432 INFO:teuthology.orchestra.run.smithi170.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2023-10-26T01:29:39.432 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-10-26T01:29:39.432 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-10-26T01:29:39.433 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-26T01:29:40.086 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: cephadm-2:18.0.0-6863.g38499368.el9.noarch 110/153 2023-10-26T01:29:40.122 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : cephadm-2:18.0.0-6863.g38499368.el9.noarch 110/153 2023-10-26T01:29:40.193 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-6863.g38499368.e 111/153 2023-10-26T01:29:40.320 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-6863.g38499368. 112/153 2023-10-26T01:29:40.358 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-6863.g38499368.el9.noa 113/153 2023-10-26T01:29:40.411 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-6863.g38499368.el9.n 114/153 2023-10-26T01:29:40.437 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-6863.g384 115/153 2023-10-26T01:29:40.577 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-6863.g384 115/153 2023-10-26T01:29:40.689 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-6863.g38499368.el9.noa 113/153 2023-10-26T01:29:40.742 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-6863.g38499368.el9.n 114/153 2023-10-26T01:29:40.745 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-6863.g38499368.el 116/153 2023-10-26T01:29:40.773 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mgr-2:18.0.0-6863.g38499368.el9.x86_64 117/153 2023-10-26T01:29:40.853 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mgr-2:18.0.0-6863.g38499368.el9.x86_64 117/153 2023-10-26T01:29:40.853 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-26T01:29:40.854 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2023-10-26T01:29:40.854 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-10-26T01:29:40.854 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-10-26T01:29:40.854 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:29:40.914 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mgr-rook-2:18.0.0-6863.g38499368.el9.noarch 118/153 2023-10-26T01:29:40.952 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-6863.g38499368.el9.noarch 118/153 2023-10-26T01:29:41.832 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-2:18.0.0-6863.g38499368.el9.x86_64 119/153 2023-10-26T01:29:41.866 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-radosgw-2:18.0.0-6863.g38499368.el9.x86_64 120/153 2023-10-26T01:29:41.888 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-6863.g38499368.el9.n 114/153 2023-10-26T01:29:41.918 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-6863.g384 115/153 2023-10-26T01:29:41.947 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-6863.g38499368.el9.x86_64 120/153 2023-10-26T01:29:41.947 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-26T01:29:41.947 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2023-10-26T01:29:41.947 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-10-26T01:29:41.948 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-10-26T01:29:41.948 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:29:42.008 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-6863.g38499 121/153 2023-10-26T01:29:42.062 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-6863.g384 115/153 2023-10-26T01:29:42.100 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-6863.g38499 121/153 2023-10-26T01:29:42.101 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-26T01:29:42.101 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2023-10-26T01:29:42.101 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:29:42.176 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : cephfs-mirror-2:18.0.0-6863.g38499368.el9.x86_64 122/153 2023-10-26T01:29:42.226 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-6863.g38499368.el 116/153 2023-10-26T01:29:42.261 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mgr-2:18.0.0-6863.g38499368.el9.x86_64 117/153 2023-10-26T01:29:42.276 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-6863.g38499368.el9.x86_64 122/153 2023-10-26T01:29:42.276 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-26T01:29:42.276 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2023-10-26T01:29:42.276 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-10-26T01:29:42.276 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-10-26T01:29:42.277 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:29:42.331 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mgr-2:18.0.0-6863.g38499368.el9.x86_64 117/153 2023-10-26T01:29:42.332 INFO:teuthology.orchestra.run.smithi170.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-26T01:29:42.332 INFO:teuthology.orchestra.run.smithi170.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2023-10-26T01:29:42.332 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-10-26T01:29:42.332 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-10-26T01:29:42.333 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-26T01:29:42.379 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mgr-rook-2:18.0.0-6863.g38499368.el9.noarch 118/153 2023-10-26T01:29:42.417 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-6863.g38499368.el9.noarch 118/153 2023-10-26T01:29:42.659 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : rbd-mirror-2:18.0.0-6863.g38499368.el9.x86_64 123/153 2023-10-26T01:29:42.743 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: rbd-mirror-2:18.0.0-6863.g38499368.el9.x86_64 123/153 2023-10-26T01:29:42.743 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-26T01:29:42.743 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2023-10-26T01:29:42.744 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-10-26T01:29:42.744 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-10-26T01:29:42.744 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:29:43.356 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-2:18.0.0-6863.g38499368.el9.x86_64 119/153 2023-10-26T01:29:43.397 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-radosgw-2:18.0.0-6863.g38499368.el9.x86_64 120/153 2023-10-26T01:29:43.471 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-6863.g38499368.el9.x86_64 120/153 2023-10-26T01:29:43.471 INFO:teuthology.orchestra.run.smithi170.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-26T01:29:43.472 INFO:teuthology.orchestra.run.smithi170.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2023-10-26T01:29:43.472 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-10-26T01:29:43.472 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-10-26T01:29:43.472 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-26T01:29:43.522 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-6863.g38499 121/153 2023-10-26T01:29:43.592 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-6863.g38499 121/153 2023-10-26T01:29:43.593 INFO:teuthology.orchestra.run.smithi170.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-26T01:29:43.593 INFO:teuthology.orchestra.run.smithi170.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2023-10-26T01:29:43.593 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-26T01:29:43.698 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : cephfs-mirror-2:18.0.0-6863.g38499368.el9.x86_64 122/153 2023-10-26T01:29:43.782 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-6863.g38499368.el9.x86_64 122/153 2023-10-26T01:29:43.783 INFO:teuthology.orchestra.run.smithi170.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-26T01:29:43.783 INFO:teuthology.orchestra.run.smithi170.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2023-10-26T01:29:43.783 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-10-26T01:29:43.783 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-10-26T01:29:43.783 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-26T01:29:44.165 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : rbd-mirror-2:18.0.0-6863.g38499368.el9.x86_64 123/153 2023-10-26T01:29:44.240 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: rbd-mirror-2:18.0.0-6863.g38499368.el9.x86_64 123/153 2023-10-26T01:29:44.240 INFO:teuthology.orchestra.run.smithi170.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-26T01:29:44.241 INFO:teuthology.orchestra.run.smithi170.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2023-10-26T01:29:44.241 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-10-26T01:29:44.241 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-10-26T01:29:44.241 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-26T01:29:46.911 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-test-2:18.0.0-6863.g38499368.el9.x86_64 124/153 2023-10-26T01:29:46.968 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : rbd-fuse-2:18.0.0-6863.g38499368.el9.x86_64 125/153 2023-10-26T01:29:47.005 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : rbd-nbd-2:18.0.0-6863.g38499368.el9.x86_64 126/153 2023-10-26T01:29:47.279 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : elfutils-libelf-devel-0.189-3.el9.x86_64 127/153 2023-10-26T01:29:47.398 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 128/153 2023-10-26T01:29:47.461 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-fuse-2:18.0.0-6863.g38499368.el9.x86_64 129/153 2023-10-26T01:29:47.507 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libtool-2.4.6-45.el9.x86_64 130/153 2023-10-26T01:29:47.546 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : xfsprogs-devel-5.19.0-4.el9.x86_64 131/153 2023-10-26T01:29:47.579 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libcephfs-devel-2:18.0.0-6863.g38499368.el9.x86_ 132/153 2023-10-26T01:29:47.646 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : NetworkManager-team-1:1.45.5-1.el9.x86_64 133/153 2023-10-26T01:29:47.757 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : NetworkManager-tui-1:1.45.5-1.el9.x86_64 134/153 2023-10-26T01:29:47.829 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 135/153 2023-10-26T01:29:47.872 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 136/153 2023-10-26T01:29:48.399 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-test-2:18.0.0-6863.g38499368.el9.x86_64 124/153 2023-10-26T01:29:48.455 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : rbd-fuse-2:18.0.0-6863.g38499368.el9.x86_64 125/153 2023-10-26T01:29:48.495 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : rbd-nbd-2:18.0.0-6863.g38499368.el9.x86_64 126/153 2023-10-26T01:29:48.766 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 137/153 2023-10-26T01:29:48.779 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : elfutils-libelf-devel-0.189-3.el9.x86_64 127/153 2023-10-26T01:29:48.820 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 137/153 2023-10-26T01:29:48.876 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 137/153 2023-10-26T01:29:48.903 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 128/153 2023-10-26T01:29:48.939 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : cephfs-top-2:18.0.0-6863.g38499368.el9.noarch 138/153 2023-10-26T01:29:48.972 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-fuse-2:18.0.0-6863.g38499368.el9.x86_64 129/153 2023-10-26T01:29:49.023 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libtool-2.4.6-45.el9.x86_64 130/153 2023-10-26T01:29:49.059 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : xfsprogs-devel-5.19.0-4.el9.x86_64 131/153 2023-10-26T01:29:49.093 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libcephfs-devel-2:18.0.0-6863.g38499368.el9.x86_ 132/153 2023-10-26T01:29:49.137 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : NetworkManager-team-1:1.45.5-1.el9.x86_64 133/153 2023-10-26T01:29:49.244 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : NetworkManager-tui-1:1.45.5-1.el9.x86_64 134/153 2023-10-26T01:29:49.318 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 135/153 2023-10-26T01:29:49.360 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 136/153 2023-10-26T01:29:50.307 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 139/153 2023-10-26T01:29:50.324 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 137/153 2023-10-26T01:29:50.366 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 137/153 2023-10-26T01:29:50.409 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : openssl-devel-1:3.0.7-24.el9.x86_64 140/153 2023-10-26T01:29:50.423 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 137/153 2023-10-26T01:29:50.456 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 141/153 2023-10-26T01:29:50.494 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : cephfs-top-2:18.0.0-6863.g38499368.el9.noarch 138/153 2023-10-26T01:29:50.583 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : flex-2.6.4-9.el9.x86_64 142/153 2023-10-26T01:29:50.628 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : bison-3.7.4-5.el9.x86_64 143/153 2023-10-26T01:29:50.665 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 144/153 2023-10-26T01:29:50.665 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 145/153 2023-10-26T01:29:50.785 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 145/153 2023-10-26T01:29:50.816 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 146/153 2023-10-26T01:29:50.848 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 147/153 2023-10-26T01:29:50.881 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 148/153 2023-10-26T01:29:50.924 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 149/153 2023-10-26T01:29:50.924 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 149/153 2023-10-26T01:29:51.256 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 149/153 2023-10-26T01:29:51.257 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 150/153 2023-10-26T01:29:51.395 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 150/153 2023-10-26T01:29:51.426 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 151/153 2023-10-26T01:29:51.427 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 152/153 2023-10-26T01:29:51.565 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 152/153 2023-10-26T01:29:51.687 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 153/153 2023-10-26T01:29:51.930 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 139/153 2023-10-26T01:29:51.971 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: xfsdump-3.1.12-3.el9.x86_64 153/153 2023-10-26T01:29:51.971 INFO:teuthology.orchestra.run.smithi169.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-10-26T01:29:51.971 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:29:52.031 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : openssl-devel-1:3.0.7-24.el9.x86_64 140/153 2023-10-26T01:29:52.079 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 141/153 2023-10-26T01:29:52.208 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : flex-2.6.4-9.el9.x86_64 142/153 2023-10-26T01:29:52.242 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : bison-3.7.4-5.el9.x86_64 143/153 2023-10-26T01:29:52.279 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 144/153 2023-10-26T01:29:52.279 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 145/153 2023-10-26T01:29:52.406 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 145/153 2023-10-26T01:29:52.438 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 146/153 2023-10-26T01:29:52.470 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 147/153 2023-10-26T01:29:52.503 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 148/153 2023-10-26T01:29:52.543 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 149/153 2023-10-26T01:29:52.543 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 149/153 2023-10-26T01:29:52.870 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 149/153 2023-10-26T01:29:52.870 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 150/153 2023-10-26T01:29:53.018 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 150/153 2023-10-26T01:29:53.048 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 151/153 2023-10-26T01:29:53.049 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 152/153 2023-10-26T01:29:53.178 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 152/153 2023-10-26T01:29:53.302 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 153/153 2023-10-26T01:29:53.559 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: xfsdump-3.1.12-3.el9.x86_64 153/153 2023-10-26T01:29:53.559 INFO:teuthology.orchestra.run.smithi170.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-10-26T01:29:53.560 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-26T01:29:56.743 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-2:18.0.0-6863.g38499368.el9.x86_64 1/153 2023-10-26T01:29:56.743 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-base-2:18.0.0-6863.g38499368.el9.x86_64 2/153 2023-10-26T01:29:56.743 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-common-2:18.0.0-6863.g38499368.el9.x86_64 3/153 2023-10-26T01:29:56.744 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-fuse-2:18.0.0-6863.g38499368.el9.x86_64 4/153 2023-10-26T01:29:56.744 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-6863.g38499 5/153 2023-10-26T01:29:56.744 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mds-2:18.0.0-6863.g38499368.el9.x86_64 6/153 2023-10-26T01:29:56.744 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mgr-2:18.0.0-6863.g38499368.el9.x86_64 7/153 2023-10-26T01:29:56.744 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mon-2:18.0.0-6863.g38499368.el9.x86_64 8/153 2023-10-26T01:29:56.745 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-osd-2:18.0.0-6863.g38499368.el9.x86_64 9/153 2023-10-26T01:29:56.745 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-radosgw-2:18.0.0-6863.g38499368.el9.x86_64 10/153 2023-10-26T01:29:56.745 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-selinux-2:18.0.0-6863.g38499368.el9.x86_64 11/153 2023-10-26T01:29:56.745 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-test-2:18.0.0-6863.g38499368.el9.x86_64 12/153 2023-10-26T01:29:56.745 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : cephfs-mirror-2:18.0.0-6863.g38499368.el9.x86_64 13/153 2023-10-26T01:29:56.745 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libcephfs-devel-2:18.0.0-6863.g38499368.el9.x86_ 14/153 2023-10-26T01:29:56.746 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libcephfs2-2:18.0.0-6863.g38499368.el9.x86_64 15/153 2023-10-26T01:29:56.746 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libcephsqlite-2:18.0.0-6863.g38499368.el9.x86_64 16/153 2023-10-26T01:29:56.746 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librados-devel-2:18.0.0-6863.g38499368.el9.x86_6 17/153 2023-10-26T01:29:56.746 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libradosstriper1-2:18.0.0-6863.g38499368.el9.x86 18/153 2023-10-26T01:29:56.746 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librgw2-2:18.0.0-6863.g38499368.el9.x86_64 19/153 2023-10-26T01:29:56.747 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-ceph-argparse-2:18.0.0-6863.g38499368.el 20/153 2023-10-26T01:29:56.747 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-ceph-common-2:18.0.0-6863.g38499368.el9. 21/153 2023-10-26T01:29:56.747 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cephfs-2:18.0.0-6863.g38499368.el9.x86_6 22/153 2023-10-26T01:29:56.747 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-rados-2:18.0.0-6863.g38499368.el9.x86_64 23/153 2023-10-26T01:29:56.747 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-rbd-2:18.0.0-6863.g38499368.el9.x86_64 24/153 2023-10-26T01:29:56.748 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-rgw-2:18.0.0-6863.g38499368.el9.x86_64 25/153 2023-10-26T01:29:56.748 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : rbd-fuse-2:18.0.0-6863.g38499368.el9.x86_64 26/153 2023-10-26T01:29:56.748 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : rbd-mirror-2:18.0.0-6863.g38499368.el9.x86_64 27/153 2023-10-26T01:29:56.748 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : rbd-nbd-2:18.0.0-6863.g38499368.el9.x86_64 28/153 2023-10-26T01:29:56.749 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-6863.g38499368. 29/153 2023-10-26T01:29:56.749 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-6863.g38499368.el9.noa 30/153 2023-10-26T01:29:56.749 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-6863.g38499368.el9.n 31/153 2023-10-26T01:29:56.749 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-6863.g384 32/153 2023-10-26T01:29:56.749 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-6863.g38499368.el 33/153 2023-10-26T01:29:56.749 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mgr-rook-2:18.0.0-6863.g38499368.el9.noarch 34/153 2023-10-26T01:29:56.750 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-6863.g38499368.e 35/153 2023-10-26T01:29:56.750 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-volume-2:18.0.0-6863.g38499368.el9.noarch 36/153 2023-10-26T01:29:56.750 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : cephadm-2:18.0.0-6863.g38499368.el9.noarch 37/153 2023-10-26T01:29:56.750 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : cephfs-top-2:18.0.0-6863.g38499368.el9.noarch 38/153 2023-10-26T01:29:56.750 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : bison-3.7.4-5.el9.x86_64 39/153 2023-10-26T01:29:56.751 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 40/153 2023-10-26T01:29:56.751 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : flex-2.6.4-9.el9.x86_64 41/153 2023-10-26T01:29:56.751 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 42/153 2023-10-26T01:29:56.751 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 43/153 2023-10-26T01:29:56.752 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 44/153 2023-10-26T01:29:56.752 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 45/153 2023-10-26T01:29:56.752 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 46/153 2023-10-26T01:29:56.752 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 47/153 2023-10-26T01:29:56.752 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 48/153 2023-10-26T01:29:56.753 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 49/153 2023-10-26T01:29:56.753 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 50/153 2023-10-26T01:29:56.753 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 51/153 2023-10-26T01:29:56.753 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 52/153 2023-10-26T01:29:56.753 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 53/153 2023-10-26T01:29:56.753 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : postgresql-13.11-1.el9.x86_64 54/153 2023-10-26T01:29:56.754 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 55/153 2023-10-26T01:29:56.754 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 56/153 2023-10-26T01:29:56.754 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 57/153 2023-10-26T01:29:56.754 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 58/153 2023-10-26T01:29:56.755 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 59/153 2023-10-26T01:29:56.755 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jinja2-2.11.3-4.el9.noarch 60/153 2023-10-26T01:29:56.755 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 61/153 2023-10-26T01:29:56.755 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 62/153 2023-10-26T01:29:56.755 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 63/153 2023-10-26T01:29:56.755 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 64/153 2023-10-26T01:29:56.756 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 65/153 2023-10-26T01:29:56.756 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 66/153 2023-10-26T01:29:56.756 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 67/153 2023-10-26T01:29:56.756 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 68/153 2023-10-26T01:29:56.756 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 69/153 2023-10-26T01:29:56.757 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 70/153 2023-10-26T01:29:56.757 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 71/153 2023-10-26T01:29:56.757 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 72/153 2023-10-26T01:29:56.757 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 73/153 2023-10-26T01:29:56.757 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 74/153 2023-10-26T01:29:56.758 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 75/153 2023-10-26T01:29:56.758 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 76/153 2023-10-26T01:29:56.758 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 77/153 2023-10-26T01:29:56.758 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libgfortran-11.4.1-2.3.el9.x86_64 78/153 2023-10-26T01:29:56.758 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libquadmath-11.4.1-2.3.el9.x86_64 79/153 2023-10-26T01:29:56.759 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 80/153 2023-10-26T01:29:56.759 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-ply-3.11-14.el9.noarch 81/153 2023-10-26T01:29:56.759 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 82/153 2023-10-26T01:29:56.759 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-requests-2.25.1-7.el9.noarch 83/153 2023-10-26T01:29:56.759 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-urllib3-1.26.5-4.el9.noarch 84/153 2023-10-26T01:29:56.761 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 85/153 2023-10-26T01:29:56.761 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : automake-1.16.2-8.el9.noarch 86/153 2023-10-26T01:29:56.761 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : elfutils-libelf-devel-0.189-3.el9.x86_64 87/153 2023-10-26T01:29:56.762 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : jq-1.6-15.el9.x86_64 88/153 2023-10-26T01:29:56.762 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 89/153 2023-10-26T01:29:56.762 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libuuid-devel-2.37.4-15.el9.x86_64 90/153 2023-10-26T01:29:56.762 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : openssl-devel-1:3.0.7-24.el9.x86_64 91/153 2023-10-26T01:29:56.762 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/153 2023-10-26T01:29:56.762 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : xfsprogs-devel-5.19.0-4.el9.x86_64 93/153 2023-10-26T01:29:56.763 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 94/153 2023-10-26T01:29:56.763 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 95/153 2023-10-26T01:29:56.763 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 96/153 2023-10-26T01:29:56.763 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libarrow-9.0.0-9.el9.x86_64 97/153 2023-10-26T01:29:56.763 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libarrow-doc-9.0.0-9.el9.noarch 98/153 2023-10-26T01:29:56.764 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 99/153 2023-10-26T01:29:56.764 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 100/153 2023-10-26T01:29:56.764 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : parquet-libs-9.0.0-9.el9.x86_64 101/153 2023-10-26T01:29:56.764 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 102/153 2023-10-26T01:29:56.764 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 103/153 2023-10-26T01:29:56.765 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 104/153 2023-10-26T01:29:56.765 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 105/153 2023-10-26T01:29:56.765 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 106/153 2023-10-26T01:29:56.765 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 107/153 2023-10-26T01:29:56.765 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-google-auth-1:2.23.3-1.el9.noarch 108/153 2023-10-26T01:29:56.765 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 109/153 2023-10-26T01:29:56.766 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 110/153 2023-10-26T01:29:56.766 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 111/153 2023-10-26T01:29:56.766 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 112/153 2023-10-26T01:29:56.766 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 113/153 2023-10-26T01:29:56.767 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 114/153 2023-10-26T01:29:56.767 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 115/153 2023-10-26T01:29:56.767 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 116/153 2023-10-26T01:29:56.767 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 117/153 2023-10-26T01:29:56.767 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 118/153 2023-10-26T01:29:56.767 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 119/153 2023-10-26T01:29:56.768 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 120/153 2023-10-26T01:29:56.768 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 121/153 2023-10-26T01:29:56.768 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 122/153 2023-10-26T01:29:56.768 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 123/153 2023-10-26T01:29:56.769 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 124/153 2023-10-26T01:29:56.769 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 125/153 2023-10-26T01:29:56.769 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 126/153 2023-10-26T01:29:56.769 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 127/153 2023-10-26T01:29:56.769 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 128/153 2023-10-26T01:29:56.770 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 129/153 2023-10-26T01:29:56.770 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 130/153 2023-10-26T01:29:56.770 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 131/153 2023-10-26T01:29:56.770 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : re2-1:20211101-3.el9.x86_64 132/153 2023-10-26T01:29:56.770 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 133/153 2023-10-26T01:29:56.771 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librados2-2:18.0.0-6863.g38499368.el9.x86_64 134/153 2023-10-26T01:29:56.771 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 135/153 2023-10-26T01:29:56.771 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librbd1-2:18.0.0-6863.g38499368.el9.x86_64 136/153 2023-10-26T01:29:56.771 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 137/153 2023-10-26T01:29:56.771 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 138/153 2023-10-26T01:29:56.772 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 139/153 2023-10-26T01:29:56.772 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : NetworkManager-1:1.45.5-1.el9.x86_64 140/153 2023-10-26T01:29:56.772 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 141/153 2023-10-26T01:29:56.772 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : NetworkManager-libnm-1:1.45.5-1.el9.x86_64 142/153 2023-10-26T01:29:56.772 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 143/153 2023-10-26T01:29:56.773 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : NetworkManager-team-1:1.45.5-1.el9.x86_64 144/153 2023-10-26T01:29:56.773 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 145/153 2023-10-26T01:29:56.773 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : NetworkManager-tui-1:1.45.5-1.el9.x86_64 146/153 2023-10-26T01:29:56.773 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 147/153 2023-10-26T01:29:56.774 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : acl-2.3.1-4.el9.x86_64 148/153 2023-10-26T01:29:56.774 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : acl-2.3.1-3.el9.x86_64 149/153 2023-10-26T01:29:56.774 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 150/153 2023-10-26T01:29:56.774 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 151/153 2023-10-26T01:29:56.774 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 152/153 2023-10-26T01:29:58.341 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-2:18.0.0-6863.g38499368.el9.x86_64 1/153 2023-10-26T01:29:58.341 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-base-2:18.0.0-6863.g38499368.el9.x86_64 2/153 2023-10-26T01:29:58.342 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-common-2:18.0.0-6863.g38499368.el9.x86_64 3/153 2023-10-26T01:29:58.342 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-fuse-2:18.0.0-6863.g38499368.el9.x86_64 4/153 2023-10-26T01:29:58.342 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-6863.g38499 5/153 2023-10-26T01:29:58.342 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mds-2:18.0.0-6863.g38499368.el9.x86_64 6/153 2023-10-26T01:29:58.342 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mgr-2:18.0.0-6863.g38499368.el9.x86_64 7/153 2023-10-26T01:29:58.343 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mon-2:18.0.0-6863.g38499368.el9.x86_64 8/153 2023-10-26T01:29:58.343 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-osd-2:18.0.0-6863.g38499368.el9.x86_64 9/153 2023-10-26T01:29:58.343 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-radosgw-2:18.0.0-6863.g38499368.el9.x86_64 10/153 2023-10-26T01:29:58.343 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-selinux-2:18.0.0-6863.g38499368.el9.x86_64 11/153 2023-10-26T01:29:58.343 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-test-2:18.0.0-6863.g38499368.el9.x86_64 12/153 2023-10-26T01:29:58.344 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : cephfs-mirror-2:18.0.0-6863.g38499368.el9.x86_64 13/153 2023-10-26T01:29:58.344 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libcephfs-devel-2:18.0.0-6863.g38499368.el9.x86_ 14/153 2023-10-26T01:29:58.344 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libcephfs2-2:18.0.0-6863.g38499368.el9.x86_64 15/153 2023-10-26T01:29:58.344 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libcephsqlite-2:18.0.0-6863.g38499368.el9.x86_64 16/153 2023-10-26T01:29:58.344 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librados-devel-2:18.0.0-6863.g38499368.el9.x86_6 17/153 2023-10-26T01:29:58.345 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libradosstriper1-2:18.0.0-6863.g38499368.el9.x86 18/153 2023-10-26T01:29:58.345 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librgw2-2:18.0.0-6863.g38499368.el9.x86_64 19/153 2023-10-26T01:29:58.345 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-ceph-argparse-2:18.0.0-6863.g38499368.el 20/153 2023-10-26T01:29:58.345 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-ceph-common-2:18.0.0-6863.g38499368.el9. 21/153 2023-10-26T01:29:58.346 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-cephfs-2:18.0.0-6863.g38499368.el9.x86_6 22/153 2023-10-26T01:29:58.346 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-rados-2:18.0.0-6863.g38499368.el9.x86_64 23/153 2023-10-26T01:29:58.346 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-rbd-2:18.0.0-6863.g38499368.el9.x86_64 24/153 2023-10-26T01:29:58.346 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-rgw-2:18.0.0-6863.g38499368.el9.x86_64 25/153 2023-10-26T01:29:58.347 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : rbd-fuse-2:18.0.0-6863.g38499368.el9.x86_64 26/153 2023-10-26T01:29:58.347 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : rbd-mirror-2:18.0.0-6863.g38499368.el9.x86_64 27/153 2023-10-26T01:29:58.347 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : rbd-nbd-2:18.0.0-6863.g38499368.el9.x86_64 28/153 2023-10-26T01:29:58.357 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-6863.g38499368. 29/153 2023-10-26T01:29:58.357 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-6863.g38499368.el9.noa 30/153 2023-10-26T01:29:58.357 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-6863.g38499368.el9.n 31/153 2023-10-26T01:29:58.357 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-6863.g384 32/153 2023-10-26T01:29:58.358 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-6863.g38499368.el 33/153 2023-10-26T01:29:58.358 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mgr-rook-2:18.0.0-6863.g38499368.el9.noarch 34/153 2023-10-26T01:29:58.358 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-6863.g38499368.e 35/153 2023-10-26T01:29:58.358 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-volume-2:18.0.0-6863.g38499368.el9.noarch 36/153 2023-10-26T01:29:58.358 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : cephadm-2:18.0.0-6863.g38499368.el9.noarch 37/153 2023-10-26T01:29:58.359 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : cephfs-top-2:18.0.0-6863.g38499368.el9.noarch 38/153 2023-10-26T01:29:58.359 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : bison-3.7.4-5.el9.x86_64 39/153 2023-10-26T01:29:58.359 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 40/153 2023-10-26T01:29:58.359 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : flex-2.6.4-9.el9.x86_64 41/153 2023-10-26T01:29:58.359 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 42/153 2023-10-26T01:29:58.360 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 43/153 2023-10-26T01:29:58.360 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 44/153 2023-10-26T01:29:58.360 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 45/153 2023-10-26T01:29:58.360 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 46/153 2023-10-26T01:29:58.360 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 47/153 2023-10-26T01:29:58.361 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 48/153 2023-10-26T01:29:58.361 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 49/153 2023-10-26T01:29:58.361 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 50/153 2023-10-26T01:29:58.361 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 51/153 2023-10-26T01:29:58.361 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 52/153 2023-10-26T01:29:58.361 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 53/153 2023-10-26T01:29:58.362 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : postgresql-13.11-1.el9.x86_64 54/153 2023-10-26T01:29:58.362 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 55/153 2023-10-26T01:29:58.362 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 56/153 2023-10-26T01:29:58.362 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 57/153 2023-10-26T01:29:58.362 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 58/153 2023-10-26T01:29:58.363 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 59/153 2023-10-26T01:29:58.363 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jinja2-2.11.3-4.el9.noarch 60/153 2023-10-26T01:29:58.363 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 61/153 2023-10-26T01:29:58.363 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 62/153 2023-10-26T01:29:58.363 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 63/153 2023-10-26T01:29:58.364 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 64/153 2023-10-26T01:29:58.364 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 65/153 2023-10-26T01:29:58.364 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 66/153 2023-10-26T01:29:58.364 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 67/153 2023-10-26T01:29:58.364 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 68/153 2023-10-26T01:29:58.365 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 69/153 2023-10-26T01:29:58.365 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 70/153 2023-10-26T01:29:58.365 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 71/153 2023-10-26T01:29:58.365 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 72/153 2023-10-26T01:29:58.365 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 73/153 2023-10-26T01:29:58.366 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 74/153 2023-10-26T01:29:58.366 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 75/153 2023-10-26T01:29:58.366 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 76/153 2023-10-26T01:29:58.366 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 77/153 2023-10-26T01:29:58.366 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libgfortran-11.4.1-2.3.el9.x86_64 78/153 2023-10-26T01:29:58.366 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libquadmath-11.4.1-2.3.el9.x86_64 79/153 2023-10-26T01:29:58.367 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 80/153 2023-10-26T01:29:58.367 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-ply-3.11-14.el9.noarch 81/153 2023-10-26T01:29:58.367 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 82/153 2023-10-26T01:29:58.367 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-requests-2.25.1-7.el9.noarch 83/153 2023-10-26T01:29:58.368 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-urllib3-1.26.5-4.el9.noarch 84/153 2023-10-26T01:29:58.368 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 85/153 2023-10-26T01:29:58.368 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : automake-1.16.2-8.el9.noarch 86/153 2023-10-26T01:29:58.368 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : elfutils-libelf-devel-0.189-3.el9.x86_64 87/153 2023-10-26T01:29:58.368 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : jq-1.6-15.el9.x86_64 88/153 2023-10-26T01:29:58.368 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 89/153 2023-10-26T01:29:58.369 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libuuid-devel-2.37.4-15.el9.x86_64 90/153 2023-10-26T01:29:58.369 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : openssl-devel-1:3.0.7-24.el9.x86_64 91/153 2023-10-26T01:29:58.369 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/153 2023-10-26T01:29:58.369 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : xfsprogs-devel-5.19.0-4.el9.x86_64 93/153 2023-10-26T01:29:58.369 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 94/153 2023-10-26T01:29:58.370 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 95/153 2023-10-26T01:29:58.370 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 96/153 2023-10-26T01:29:58.370 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libarrow-9.0.0-9.el9.x86_64 97/153 2023-10-26T01:29:58.370 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libarrow-doc-9.0.0-9.el9.noarch 98/153 2023-10-26T01:29:58.370 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 99/153 2023-10-26T01:29:58.371 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 100/153 2023-10-26T01:29:58.371 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : parquet-libs-9.0.0-9.el9.x86_64 101/153 2023-10-26T01:29:58.371 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 102/153 2023-10-26T01:29:58.371 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 103/153 2023-10-26T01:29:58.371 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 104/153 2023-10-26T01:29:58.372 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 105/153 2023-10-26T01:29:58.372 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 106/153 2023-10-26T01:29:58.372 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 107/153 2023-10-26T01:29:58.372 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-google-auth-1:2.23.3-1.el9.noarch 108/153 2023-10-26T01:29:58.372 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 109/153 2023-10-26T01:29:58.373 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 110/153 2023-10-26T01:29:58.373 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 111/153 2023-10-26T01:29:58.373 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 112/153 2023-10-26T01:29:58.373 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 113/153 2023-10-26T01:29:58.373 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 114/153 2023-10-26T01:29:58.374 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 115/153 2023-10-26T01:29:58.374 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 116/153 2023-10-26T01:29:58.374 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 117/153 2023-10-26T01:29:58.374 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 118/153 2023-10-26T01:29:58.374 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 119/153 2023-10-26T01:29:58.375 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 120/153 2023-10-26T01:29:58.375 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 121/153 2023-10-26T01:29:58.375 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 122/153 2023-10-26T01:29:58.375 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 123/153 2023-10-26T01:29:58.375 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 124/153 2023-10-26T01:29:58.375 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 125/153 2023-10-26T01:29:58.376 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 126/153 2023-10-26T01:29:58.376 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 127/153 2023-10-26T01:29:58.376 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 128/153 2023-10-26T01:29:58.376 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 129/153 2023-10-26T01:29:58.376 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 130/153 2023-10-26T01:29:58.377 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 131/153 2023-10-26T01:29:58.377 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : re2-1:20211101-3.el9.x86_64 132/153 2023-10-26T01:29:58.377 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 133/153 2023-10-26T01:29:58.377 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librados2-2:18.0.0-6863.g38499368.el9.x86_64 134/153 2023-10-26T01:29:58.377 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 135/153 2023-10-26T01:29:58.378 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librbd1-2:18.0.0-6863.g38499368.el9.x86_64 136/153 2023-10-26T01:29:58.378 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 137/153 2023-10-26T01:29:58.378 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 138/153 2023-10-26T01:29:58.378 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 139/153 2023-10-26T01:29:58.378 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : NetworkManager-1:1.45.5-1.el9.x86_64 140/153 2023-10-26T01:29:58.378 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 141/153 2023-10-26T01:29:58.379 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : NetworkManager-libnm-1:1.45.5-1.el9.x86_64 142/153 2023-10-26T01:29:58.379 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 143/153 2023-10-26T01:29:58.379 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : NetworkManager-team-1:1.45.5-1.el9.x86_64 144/153 2023-10-26T01:29:58.379 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 145/153 2023-10-26T01:29:58.379 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : NetworkManager-tui-1:1.45.5-1.el9.x86_64 146/153 2023-10-26T01:29:58.380 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 147/153 2023-10-26T01:29:58.380 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : acl-2.3.1-4.el9.x86_64 148/153 2023-10-26T01:29:58.380 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : acl-2.3.1-3.el9.x86_64 149/153 2023-10-26T01:29:58.380 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 150/153 2023-10-26T01:29:58.380 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 151/153 2023-10-26T01:29:58.380 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 152/153 2023-10-26T01:29:58.936 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 153/153 2023-10-26T01:29:58.936 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:29:58.936 INFO:teuthology.orchestra.run.smithi169.stdout:Upgraded: 2023-10-26T01:29:58.937 INFO:teuthology.orchestra.run.smithi169.stdout: NetworkManager-1:1.45.5-1.el9.x86_64 2023-10-26T01:29:58.937 INFO:teuthology.orchestra.run.smithi169.stdout: NetworkManager-libnm-1:1.45.5-1.el9.x86_64 2023-10-26T01:29:58.937 INFO:teuthology.orchestra.run.smithi169.stdout: NetworkManager-team-1:1.45.5-1.el9.x86_64 2023-10-26T01:29:58.937 INFO:teuthology.orchestra.run.smithi169.stdout: NetworkManager-tui-1:1.45.5-1.el9.x86_64 2023-10-26T01:29:58.937 INFO:teuthology.orchestra.run.smithi169.stdout: acl-2.3.1-4.el9.x86_64 2023-10-26T01:29:58.938 INFO:teuthology.orchestra.run.smithi169.stdout: libacl-2.3.1-4.el9.x86_64 2023-10-26T01:29:58.938 INFO:teuthology.orchestra.run.smithi169.stdout: libacl-devel-2.3.1-4.el9.x86_64 2023-10-26T01:29:58.938 INFO:teuthology.orchestra.run.smithi169.stdout: librados2-2:18.0.0-6863.g38499368.el9.x86_64 2023-10-26T01:29:58.938 INFO:teuthology.orchestra.run.smithi169.stdout: librbd1-2:18.0.0-6863.g38499368.el9.x86_64 2023-10-26T01:29:58.938 INFO:teuthology.orchestra.run.smithi169.stdout: xfsdump-3.1.12-4.el9.x86_64 2023-10-26T01:29:58.939 INFO:teuthology.orchestra.run.smithi169.stdout:Installed: 2023-10-26T01:29:58.939 INFO:teuthology.orchestra.run.smithi169.stdout: automake-1.16.2-8.el9.noarch 2023-10-26T01:29:58.939 INFO:teuthology.orchestra.run.smithi169.stdout: bison-3.7.4-5.el9.x86_64 2023-10-26T01:29:58.939 INFO:teuthology.orchestra.run.smithi169.stdout: boost-program-options-1.75.0-8.el9.x86_64 2023-10-26T01:29:58.940 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-2:18.0.0-6863.g38499368.el9.x86_64 2023-10-26T01:29:58.940 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-base-2:18.0.0-6863.g38499368.el9.x86_64 2023-10-26T01:29:58.940 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-common-2:18.0.0-6863.g38499368.el9.x86_64 2023-10-26T01:29:58.940 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-fuse-2:18.0.0-6863.g38499368.el9.x86_64 2023-10-26T01:29:58.940 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-grafana-dashboards-2:18.0.0-6863.g38499368.el9.noarch 2023-10-26T01:29:58.940 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-immutable-object-cache-2:18.0.0-6863.g38499368.el9.x86_64 2023-10-26T01:29:58.941 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mds-2:18.0.0-6863.g38499368.el9.x86_64 2023-10-26T01:29:58.941 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-2:18.0.0-6863.g38499368.el9.x86_64 2023-10-26T01:29:58.941 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-cephadm-2:18.0.0-6863.g38499368.el9.noarch 2023-10-26T01:29:58.941 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-dashboard-2:18.0.0-6863.g38499368.el9.noarch 2023-10-26T01:29:58.941 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-diskprediction-local-2:18.0.0-6863.g38499368.el9.noarch 2023-10-26T01:29:58.942 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-modules-core-2:18.0.0-6863.g38499368.el9.noarch 2023-10-26T01:29:58.942 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-rook-2:18.0.0-6863.g38499368.el9.noarch 2023-10-26T01:29:58.942 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mon-2:18.0.0-6863.g38499368.el9.x86_64 2023-10-26T01:29:58.942 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-osd-2:18.0.0-6863.g38499368.el9.x86_64 2023-10-26T01:29:58.942 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-prometheus-alerts-2:18.0.0-6863.g38499368.el9.noarch 2023-10-26T01:29:58.943 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-radosgw-2:18.0.0-6863.g38499368.el9.x86_64 2023-10-26T01:29:58.943 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-selinux-2:18.0.0-6863.g38499368.el9.x86_64 2023-10-26T01:29:58.943 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-test-2:18.0.0-6863.g38499368.el9.x86_64 2023-10-26T01:29:58.943 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-volume-2:18.0.0-6863.g38499368.el9.noarch 2023-10-26T01:29:58.943 INFO:teuthology.orchestra.run.smithi169.stdout: cephadm-2:18.0.0-6863.g38499368.el9.noarch 2023-10-26T01:29:58.944 INFO:teuthology.orchestra.run.smithi169.stdout: cephfs-mirror-2:18.0.0-6863.g38499368.el9.x86_64 2023-10-26T01:29:58.944 INFO:teuthology.orchestra.run.smithi169.stdout: cephfs-top-2:18.0.0-6863.g38499368.el9.noarch 2023-10-26T01:29:58.944 INFO:teuthology.orchestra.run.smithi169.stdout: elfutils-libelf-devel-0.189-3.el9.x86_64 2023-10-26T01:29:58.944 INFO:teuthology.orchestra.run.smithi169.stdout: flex-2.6.4-9.el9.x86_64 2023-10-26T01:29:58.945 INFO:teuthology.orchestra.run.smithi169.stdout: flexiblas-3.0.4-8.el9.x86_64 2023-10-26T01:29:58.945 INFO:teuthology.orchestra.run.smithi169.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2023-10-26T01:29:58.945 INFO:teuthology.orchestra.run.smithi169.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2023-10-26T01:29:58.945 INFO:teuthology.orchestra.run.smithi169.stdout: fmt-8.1.1-5.el9.x86_64 2023-10-26T01:29:58.945 INFO:teuthology.orchestra.run.smithi169.stdout: fuse-2.9.9-15.el9.x86_64 2023-10-26T01:29:58.946 INFO:teuthology.orchestra.run.smithi169.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2023-10-26T01:29:58.946 INFO:teuthology.orchestra.run.smithi169.stdout: jq-1.6-15.el9.x86_64 2023-10-26T01:29:58.946 INFO:teuthology.orchestra.run.smithi169.stdout: libaio-devel-0.3.111-13.el9.x86_64 2023-10-26T01:29:58.946 INFO:teuthology.orchestra.run.smithi169.stdout: libarrow-9.0.0-9.el9.x86_64 2023-10-26T01:29:58.947 INFO:teuthology.orchestra.run.smithi169.stdout: libarrow-doc-9.0.0-9.el9.noarch 2023-10-26T01:29:58.947 INFO:teuthology.orchestra.run.smithi169.stdout: libcephfs-devel-2:18.0.0-6863.g38499368.el9.x86_64 2023-10-26T01:29:58.947 INFO:teuthology.orchestra.run.smithi169.stdout: libcephfs2-2:18.0.0-6863.g38499368.el9.x86_64 2023-10-26T01:29:58.947 INFO:teuthology.orchestra.run.smithi169.stdout: libcephsqlite-2:18.0.0-6863.g38499368.el9.x86_64 2023-10-26T01:29:58.948 INFO:teuthology.orchestra.run.smithi169.stdout: libgfortran-11.4.1-2.3.el9.x86_64 2023-10-26T01:29:58.948 INFO:teuthology.orchestra.run.smithi169.stdout: liboath-2.6.7-2.el9.x86_64 2023-10-26T01:29:58.948 INFO:teuthology.orchestra.run.smithi169.stdout: libpmemobj-1.12.1-1.el9.x86_64 2023-10-26T01:29:58.948 INFO:teuthology.orchestra.run.smithi169.stdout: libquadmath-11.4.1-2.3.el9.x86_64 2023-10-26T01:29:58.948 INFO:teuthology.orchestra.run.smithi169.stdout: librabbitmq-0.11.0-7.el9.x86_64 2023-10-26T01:29:58.948 INFO:teuthology.orchestra.run.smithi169.stdout: librados-devel-2:18.0.0-6863.g38499368.el9.x86_64 2023-10-26T01:29:58.949 INFO:teuthology.orchestra.run.smithi169.stdout: libradosstriper1-2:18.0.0-6863.g38499368.el9.x86_64 2023-10-26T01:29:58.949 INFO:teuthology.orchestra.run.smithi169.stdout: librdkafka-1.6.1-102.el9.x86_64 2023-10-26T01:29:58.949 INFO:teuthology.orchestra.run.smithi169.stdout: librgw2-2:18.0.0-6863.g38499368.el9.x86_64 2023-10-26T01:29:58.949 INFO:teuthology.orchestra.run.smithi169.stdout: libtool-2.4.6-45.el9.x86_64 2023-10-26T01:29:58.949 INFO:teuthology.orchestra.run.smithi169.stdout: libunwind-1.6.2-1.el9.x86_64 2023-10-26T01:29:58.950 INFO:teuthology.orchestra.run.smithi169.stdout: libuuid-devel-2.37.4-15.el9.x86_64 2023-10-26T01:29:58.950 INFO:teuthology.orchestra.run.smithi169.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2023-10-26T01:29:58.950 INFO:teuthology.orchestra.run.smithi169.stdout: lttng-ust-2.12.0-6.el9.x86_64 2023-10-26T01:29:58.950 INFO:teuthology.orchestra.run.smithi169.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2023-10-26T01:29:58.950 INFO:teuthology.orchestra.run.smithi169.stdout: openblas-0.3.21-2.el9.x86_64 2023-10-26T01:29:58.951 INFO:teuthology.orchestra.run.smithi169.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2023-10-26T01:29:58.951 INFO:teuthology.orchestra.run.smithi169.stdout: openssl-devel-1:3.0.7-24.el9.x86_64 2023-10-26T01:29:58.951 INFO:teuthology.orchestra.run.smithi169.stdout: parquet-libs-9.0.0-9.el9.x86_64 2023-10-26T01:29:58.951 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql-13.11-1.el9.x86_64 2023-10-26T01:29:58.951 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql-contrib-13.11-1.el9.x86_64 2023-10-26T01:29:58.951 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2023-10-26T01:29:58.952 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql-server-13.11-1.el9.x86_64 2023-10-26T01:29:58.952 INFO:teuthology.orchestra.run.smithi169.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2023-10-26T01:29:58.952 INFO:teuthology.orchestra.run.smithi169.stdout: python3-babel-2.9.1-2.el9.noarch 2023-10-26T01:29:58.952 INFO:teuthology.orchestra.run.smithi169.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2023-10-26T01:29:58.952 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cachetools-4.2.4-1.el9.noarch 2023-10-26T01:29:58.953 INFO:teuthology.orchestra.run.smithi169.stdout: python3-ceph-argparse-2:18.0.0-6863.g38499368.el9.x86_64 2023-10-26T01:29:58.953 INFO:teuthology.orchestra.run.smithi169.stdout: python3-ceph-common-2:18.0.0-6863.g38499368.el9.x86_64 2023-10-26T01:29:58.953 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cephfs-2:18.0.0-6863.g38499368.el9.x86_64 2023-10-26T01:29:58.953 INFO:teuthology.orchestra.run.smithi169.stdout: python3-certifi-2021.10.8-2.el9.noarch 2023-10-26T01:29:58.953 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cffi-1.14.5-5.el9.x86_64 2023-10-26T01:29:58.954 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cheroot-8.6.0-4.el9.noarch 2023-10-26T01:29:58.954 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2023-10-26T01:29:58.954 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2023-10-26T01:29:58.954 INFO:teuthology.orchestra.run.smithi169.stdout: python3-devel-3.9.17-2.el9.x86_64 2023-10-26T01:29:58.954 INFO:teuthology.orchestra.run.smithi169.stdout: python3-google-auth-1:2.23.3-1.el9.noarch 2023-10-26T01:29:58.955 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-8.2.1-3.el9.noarch 2023-10-26T01:29:58.955 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2023-10-26T01:29:58.955 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2023-10-26T01:29:58.955 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2023-10-26T01:29:58.956 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2023-10-26T01:29:58.956 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jinja2-2.11.3-4.el9.noarch 2023-10-26T01:29:58.956 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jsonpatch-1.21-16.el9.noarch 2023-10-26T01:29:58.956 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jsonpointer-2.0-4.el9.noarch 2023-10-26T01:29:58.956 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jwt-2.4.0-1.el9.noarch 2023-10-26T01:29:58.956 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2023-10-26T01:29:58.957 INFO:teuthology.orchestra.run.smithi169.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2023-10-26T01:29:58.957 INFO:teuthology.orchestra.run.smithi169.stdout: python3-logutils-0.3.5-21.el9.noarch 2023-10-26T01:29:58.957 INFO:teuthology.orchestra.run.smithi169.stdout: python3-mako-1.1.4-6.el9.noarch 2023-10-26T01:29:58.957 INFO:teuthology.orchestra.run.smithi169.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2023-10-26T01:29:58.957 INFO:teuthology.orchestra.run.smithi169.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2023-10-26T01:29:58.958 INFO:teuthology.orchestra.run.smithi169.stdout: python3-natsort-7.1.1-5.el9.noarch 2023-10-26T01:29:58.958 INFO:teuthology.orchestra.run.smithi169.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2023-10-26T01:29:58.958 INFO:teuthology.orchestra.run.smithi169.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2023-10-26T01:29:58.958 INFO:teuthology.orchestra.run.smithi169.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2023-10-26T01:29:58.958 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pecan-1.4.2-3.el9.noarch 2023-10-26T01:29:58.959 INFO:teuthology.orchestra.run.smithi169.stdout: python3-ply-3.11-14.el9.noarch 2023-10-26T01:29:58.959 INFO:teuthology.orchestra.run.smithi169.stdout: python3-portend-3.1.0-2.el9.noarch 2023-10-26T01:29:58.959 INFO:teuthology.orchestra.run.smithi169.stdout: python3-prettytable-0.7.2-27.el9.noarch 2023-10-26T01:29:58.959 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2023-10-26T01:29:58.959 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2023-10-26T01:29:58.959 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2023-10-26T01:29:58.960 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pycparser-2.20-6.el9.noarch 2023-10-26T01:29:58.960 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pytz-2021.1-5.el9.noarch 2023-10-26T01:29:58.960 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rados-2:18.0.0-6863.g38499368.el9.x86_64 2023-10-26T01:29:58.960 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rbd-2:18.0.0-6863.g38499368.el9.x86_64 2023-10-26T01:29:58.960 INFO:teuthology.orchestra.run.smithi169.stdout: python3-repoze-lru-0.7-16.el9.noarch 2023-10-26T01:29:58.961 INFO:teuthology.orchestra.run.smithi169.stdout: python3-requests-2.25.1-7.el9.noarch 2023-10-26T01:29:58.961 INFO:teuthology.orchestra.run.smithi169.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2023-10-26T01:29:58.961 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rgw-2:18.0.0-6863.g38499368.el9.x86_64 2023-10-26T01:29:58.961 INFO:teuthology.orchestra.run.smithi169.stdout: python3-routes-2.5.1-5.el9.noarch 2023-10-26T01:29:58.961 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rsa-4.9-2.el9.noarch 2023-10-26T01:29:58.962 INFO:teuthology.orchestra.run.smithi169.stdout: python3-scipy-1.6.2-8.el9.x86_64 2023-10-26T01:29:58.962 INFO:teuthology.orchestra.run.smithi169.stdout: python3-tempora-5.0.0-2.el9.noarch 2023-10-26T01:29:58.962 INFO:teuthology.orchestra.run.smithi169.stdout: python3-toml-0.10.2-6.el9.noarch 2023-10-26T01:29:58.962 INFO:teuthology.orchestra.run.smithi169.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2023-10-26T01:29:58.963 INFO:teuthology.orchestra.run.smithi169.stdout: python3-urllib3-1.26.5-4.el9.noarch 2023-10-26T01:29:58.963 INFO:teuthology.orchestra.run.smithi169.stdout: python3-webob-1.8.7-6.el9.noarch 2023-10-26T01:29:58.963 INFO:teuthology.orchestra.run.smithi169.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2023-10-26T01:29:58.963 INFO:teuthology.orchestra.run.smithi169.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2023-10-26T01:29:58.964 INFO:teuthology.orchestra.run.smithi169.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2023-10-26T01:29:58.964 INFO:teuthology.orchestra.run.smithi169.stdout: rbd-fuse-2:18.0.0-6863.g38499368.el9.x86_64 2023-10-26T01:29:58.964 INFO:teuthology.orchestra.run.smithi169.stdout: rbd-mirror-2:18.0.0-6863.g38499368.el9.x86_64 2023-10-26T01:29:58.964 INFO:teuthology.orchestra.run.smithi169.stdout: rbd-nbd-2:18.0.0-6863.g38499368.el9.x86_64 2023-10-26T01:29:58.964 INFO:teuthology.orchestra.run.smithi169.stdout: re2-1:20211101-3.el9.x86_64 2023-10-26T01:29:58.965 INFO:teuthology.orchestra.run.smithi169.stdout: socat-1.7.4.1-5.el9.x86_64 2023-10-26T01:29:58.965 INFO:teuthology.orchestra.run.smithi169.stdout: thrift-0.15.0-2.el9.x86_64 2023-10-26T01:29:58.965 INFO:teuthology.orchestra.run.smithi169.stdout: uuid-1.6.2-55.el9.x86_64 2023-10-26T01:29:58.965 INFO:teuthology.orchestra.run.smithi169.stdout: xfsprogs-devel-5.19.0-4.el9.x86_64 2023-10-26T01:29:58.966 INFO:teuthology.orchestra.run.smithi169.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2023-10-26T01:29:58.966 INFO:teuthology.orchestra.run.smithi169.stdout: zlib-devel-1.2.11-40.el9.x86_64 2023-10-26T01:29:58.966 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:29:58.967 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2023-10-26T01:29:59.471 DEBUG:teuthology.parallel:result is None 2023-10-26T01:30:00.515 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 153/153 2023-10-26T01:30:00.516 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-26T01:30:00.516 INFO:teuthology.orchestra.run.smithi170.stdout:Upgraded: 2023-10-26T01:30:00.516 INFO:teuthology.orchestra.run.smithi170.stdout: NetworkManager-1:1.45.5-1.el9.x86_64 2023-10-26T01:30:00.516 INFO:teuthology.orchestra.run.smithi170.stdout: NetworkManager-libnm-1:1.45.5-1.el9.x86_64 2023-10-26T01:30:00.516 INFO:teuthology.orchestra.run.smithi170.stdout: NetworkManager-team-1:1.45.5-1.el9.x86_64 2023-10-26T01:30:00.517 INFO:teuthology.orchestra.run.smithi170.stdout: NetworkManager-tui-1:1.45.5-1.el9.x86_64 2023-10-26T01:30:00.517 INFO:teuthology.orchestra.run.smithi170.stdout: acl-2.3.1-4.el9.x86_64 2023-10-26T01:30:00.517 INFO:teuthology.orchestra.run.smithi170.stdout: libacl-2.3.1-4.el9.x86_64 2023-10-26T01:30:00.517 INFO:teuthology.orchestra.run.smithi170.stdout: libacl-devel-2.3.1-4.el9.x86_64 2023-10-26T01:30:00.517 INFO:teuthology.orchestra.run.smithi170.stdout: librados2-2:18.0.0-6863.g38499368.el9.x86_64 2023-10-26T01:30:00.518 INFO:teuthology.orchestra.run.smithi170.stdout: librbd1-2:18.0.0-6863.g38499368.el9.x86_64 2023-10-26T01:30:00.518 INFO:teuthology.orchestra.run.smithi170.stdout: xfsdump-3.1.12-4.el9.x86_64 2023-10-26T01:30:00.518 INFO:teuthology.orchestra.run.smithi170.stdout:Installed: 2023-10-26T01:30:00.518 INFO:teuthology.orchestra.run.smithi170.stdout: automake-1.16.2-8.el9.noarch 2023-10-26T01:30:00.518 INFO:teuthology.orchestra.run.smithi170.stdout: bison-3.7.4-5.el9.x86_64 2023-10-26T01:30:00.518 INFO:teuthology.orchestra.run.smithi170.stdout: boost-program-options-1.75.0-8.el9.x86_64 2023-10-26T01:30:00.519 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-2:18.0.0-6863.g38499368.el9.x86_64 2023-10-26T01:30:00.519 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-base-2:18.0.0-6863.g38499368.el9.x86_64 2023-10-26T01:30:00.519 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-common-2:18.0.0-6863.g38499368.el9.x86_64 2023-10-26T01:30:00.519 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-fuse-2:18.0.0-6863.g38499368.el9.x86_64 2023-10-26T01:30:00.519 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-grafana-dashboards-2:18.0.0-6863.g38499368.el9.noarch 2023-10-26T01:30:00.520 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-immutable-object-cache-2:18.0.0-6863.g38499368.el9.x86_64 2023-10-26T01:30:00.520 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mds-2:18.0.0-6863.g38499368.el9.x86_64 2023-10-26T01:30:00.520 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-2:18.0.0-6863.g38499368.el9.x86_64 2023-10-26T01:30:00.520 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-cephadm-2:18.0.0-6863.g38499368.el9.noarch 2023-10-26T01:30:00.520 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-dashboard-2:18.0.0-6863.g38499368.el9.noarch 2023-10-26T01:30:00.520 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-diskprediction-local-2:18.0.0-6863.g38499368.el9.noarch 2023-10-26T01:30:00.521 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-modules-core-2:18.0.0-6863.g38499368.el9.noarch 2023-10-26T01:30:00.521 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-rook-2:18.0.0-6863.g38499368.el9.noarch 2023-10-26T01:30:00.521 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mon-2:18.0.0-6863.g38499368.el9.x86_64 2023-10-26T01:30:00.521 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-osd-2:18.0.0-6863.g38499368.el9.x86_64 2023-10-26T01:30:00.521 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-prometheus-alerts-2:18.0.0-6863.g38499368.el9.noarch 2023-10-26T01:30:00.521 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-radosgw-2:18.0.0-6863.g38499368.el9.x86_64 2023-10-26T01:30:00.522 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-selinux-2:18.0.0-6863.g38499368.el9.x86_64 2023-10-26T01:30:00.522 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-test-2:18.0.0-6863.g38499368.el9.x86_64 2023-10-26T01:30:00.522 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-volume-2:18.0.0-6863.g38499368.el9.noarch 2023-10-26T01:30:00.522 INFO:teuthology.orchestra.run.smithi170.stdout: cephadm-2:18.0.0-6863.g38499368.el9.noarch 2023-10-26T01:30:00.522 INFO:teuthology.orchestra.run.smithi170.stdout: cephfs-mirror-2:18.0.0-6863.g38499368.el9.x86_64 2023-10-26T01:30:00.523 INFO:teuthology.orchestra.run.smithi170.stdout: cephfs-top-2:18.0.0-6863.g38499368.el9.noarch 2023-10-26T01:30:00.523 INFO:teuthology.orchestra.run.smithi170.stdout: elfutils-libelf-devel-0.189-3.el9.x86_64 2023-10-26T01:30:00.523 INFO:teuthology.orchestra.run.smithi170.stdout: flex-2.6.4-9.el9.x86_64 2023-10-26T01:30:00.523 INFO:teuthology.orchestra.run.smithi170.stdout: flexiblas-3.0.4-8.el9.x86_64 2023-10-26T01:30:00.523 INFO:teuthology.orchestra.run.smithi170.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2023-10-26T01:30:00.524 INFO:teuthology.orchestra.run.smithi170.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2023-10-26T01:30:00.524 INFO:teuthology.orchestra.run.smithi170.stdout: fmt-8.1.1-5.el9.x86_64 2023-10-26T01:30:00.524 INFO:teuthology.orchestra.run.smithi170.stdout: fuse-2.9.9-15.el9.x86_64 2023-10-26T01:30:00.524 INFO:teuthology.orchestra.run.smithi170.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2023-10-26T01:30:00.524 INFO:teuthology.orchestra.run.smithi170.stdout: jq-1.6-15.el9.x86_64 2023-10-26T01:30:00.525 INFO:teuthology.orchestra.run.smithi170.stdout: libaio-devel-0.3.111-13.el9.x86_64 2023-10-26T01:30:00.525 INFO:teuthology.orchestra.run.smithi170.stdout: libarrow-9.0.0-9.el9.x86_64 2023-10-26T01:30:00.525 INFO:teuthology.orchestra.run.smithi170.stdout: libarrow-doc-9.0.0-9.el9.noarch 2023-10-26T01:30:00.525 INFO:teuthology.orchestra.run.smithi170.stdout: libcephfs-devel-2:18.0.0-6863.g38499368.el9.x86_64 2023-10-26T01:30:00.525 INFO:teuthology.orchestra.run.smithi170.stdout: libcephfs2-2:18.0.0-6863.g38499368.el9.x86_64 2023-10-26T01:30:00.525 INFO:teuthology.orchestra.run.smithi170.stdout: libcephsqlite-2:18.0.0-6863.g38499368.el9.x86_64 2023-10-26T01:30:00.526 INFO:teuthology.orchestra.run.smithi170.stdout: libgfortran-11.4.1-2.3.el9.x86_64 2023-10-26T01:30:00.526 INFO:teuthology.orchestra.run.smithi170.stdout: liboath-2.6.7-2.el9.x86_64 2023-10-26T01:30:00.526 INFO:teuthology.orchestra.run.smithi170.stdout: libpmemobj-1.12.1-1.el9.x86_64 2023-10-26T01:30:00.526 INFO:teuthology.orchestra.run.smithi170.stdout: libquadmath-11.4.1-2.3.el9.x86_64 2023-10-26T01:30:00.526 INFO:teuthology.orchestra.run.smithi170.stdout: librabbitmq-0.11.0-7.el9.x86_64 2023-10-26T01:30:00.527 INFO:teuthology.orchestra.run.smithi170.stdout: librados-devel-2:18.0.0-6863.g38499368.el9.x86_64 2023-10-26T01:30:00.527 INFO:teuthology.orchestra.run.smithi170.stdout: libradosstriper1-2:18.0.0-6863.g38499368.el9.x86_64 2023-10-26T01:30:00.527 INFO:teuthology.orchestra.run.smithi170.stdout: librdkafka-1.6.1-102.el9.x86_64 2023-10-26T01:30:00.527 INFO:teuthology.orchestra.run.smithi170.stdout: librgw2-2:18.0.0-6863.g38499368.el9.x86_64 2023-10-26T01:30:00.527 INFO:teuthology.orchestra.run.smithi170.stdout: libtool-2.4.6-45.el9.x86_64 2023-10-26T01:30:00.528 INFO:teuthology.orchestra.run.smithi170.stdout: libunwind-1.6.2-1.el9.x86_64 2023-10-26T01:30:00.528 INFO:teuthology.orchestra.run.smithi170.stdout: libuuid-devel-2.37.4-15.el9.x86_64 2023-10-26T01:30:00.528 INFO:teuthology.orchestra.run.smithi170.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2023-10-26T01:30:00.528 INFO:teuthology.orchestra.run.smithi170.stdout: lttng-ust-2.12.0-6.el9.x86_64 2023-10-26T01:30:00.528 INFO:teuthology.orchestra.run.smithi170.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2023-10-26T01:30:00.528 INFO:teuthology.orchestra.run.smithi170.stdout: openblas-0.3.21-2.el9.x86_64 2023-10-26T01:30:00.529 INFO:teuthology.orchestra.run.smithi170.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2023-10-26T01:30:00.529 INFO:teuthology.orchestra.run.smithi170.stdout: openssl-devel-1:3.0.7-24.el9.x86_64 2023-10-26T01:30:00.529 INFO:teuthology.orchestra.run.smithi170.stdout: parquet-libs-9.0.0-9.el9.x86_64 2023-10-26T01:30:00.529 INFO:teuthology.orchestra.run.smithi170.stdout: postgresql-13.11-1.el9.x86_64 2023-10-26T01:30:00.529 INFO:teuthology.orchestra.run.smithi170.stdout: postgresql-contrib-13.11-1.el9.x86_64 2023-10-26T01:30:00.530 INFO:teuthology.orchestra.run.smithi170.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2023-10-26T01:30:00.530 INFO:teuthology.orchestra.run.smithi170.stdout: postgresql-server-13.11-1.el9.x86_64 2023-10-26T01:30:00.530 INFO:teuthology.orchestra.run.smithi170.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2023-10-26T01:30:00.530 INFO:teuthology.orchestra.run.smithi170.stdout: python3-babel-2.9.1-2.el9.noarch 2023-10-26T01:30:00.530 INFO:teuthology.orchestra.run.smithi170.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2023-10-26T01:30:00.530 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cachetools-4.2.4-1.el9.noarch 2023-10-26T01:30:00.531 INFO:teuthology.orchestra.run.smithi170.stdout: python3-ceph-argparse-2:18.0.0-6863.g38499368.el9.x86_64 2023-10-26T01:30:00.531 INFO:teuthology.orchestra.run.smithi170.stdout: python3-ceph-common-2:18.0.0-6863.g38499368.el9.x86_64 2023-10-26T01:30:00.531 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cephfs-2:18.0.0-6863.g38499368.el9.x86_64 2023-10-26T01:30:00.531 INFO:teuthology.orchestra.run.smithi170.stdout: python3-certifi-2021.10.8-2.el9.noarch 2023-10-26T01:30:00.531 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cffi-1.14.5-5.el9.x86_64 2023-10-26T01:30:00.532 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cheroot-8.6.0-4.el9.noarch 2023-10-26T01:30:00.532 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2023-10-26T01:30:00.532 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2023-10-26T01:30:00.532 INFO:teuthology.orchestra.run.smithi170.stdout: python3-devel-3.9.17-2.el9.x86_64 2023-10-26T01:30:00.532 INFO:teuthology.orchestra.run.smithi170.stdout: python3-google-auth-1:2.23.3-1.el9.noarch 2023-10-26T01:30:00.532 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco-8.2.1-3.el9.noarch 2023-10-26T01:30:00.533 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2023-10-26T01:30:00.533 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2023-10-26T01:30:00.533 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2023-10-26T01:30:00.533 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2023-10-26T01:30:00.533 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jinja2-2.11.3-4.el9.noarch 2023-10-26T01:30:00.534 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jsonpatch-1.21-16.el9.noarch 2023-10-26T01:30:00.534 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jsonpointer-2.0-4.el9.noarch 2023-10-26T01:30:00.534 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jwt-2.4.0-1.el9.noarch 2023-10-26T01:30:00.534 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2023-10-26T01:30:00.534 INFO:teuthology.orchestra.run.smithi170.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2023-10-26T01:30:00.535 INFO:teuthology.orchestra.run.smithi170.stdout: python3-logutils-0.3.5-21.el9.noarch 2023-10-26T01:30:00.535 INFO:teuthology.orchestra.run.smithi170.stdout: python3-mako-1.1.4-6.el9.noarch 2023-10-26T01:30:00.535 INFO:teuthology.orchestra.run.smithi170.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2023-10-26T01:30:00.535 INFO:teuthology.orchestra.run.smithi170.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2023-10-26T01:30:00.535 INFO:teuthology.orchestra.run.smithi170.stdout: python3-natsort-7.1.1-5.el9.noarch 2023-10-26T01:30:00.536 INFO:teuthology.orchestra.run.smithi170.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2023-10-26T01:30:00.536 INFO:teuthology.orchestra.run.smithi170.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2023-10-26T01:30:00.536 INFO:teuthology.orchestra.run.smithi170.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2023-10-26T01:30:00.536 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pecan-1.4.2-3.el9.noarch 2023-10-26T01:30:00.536 INFO:teuthology.orchestra.run.smithi170.stdout: python3-ply-3.11-14.el9.noarch 2023-10-26T01:30:00.536 INFO:teuthology.orchestra.run.smithi170.stdout: python3-portend-3.1.0-2.el9.noarch 2023-10-26T01:30:00.537 INFO:teuthology.orchestra.run.smithi170.stdout: python3-prettytable-0.7.2-27.el9.noarch 2023-10-26T01:30:00.537 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2023-10-26T01:30:00.537 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2023-10-26T01:30:00.537 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2023-10-26T01:30:00.537 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pycparser-2.20-6.el9.noarch 2023-10-26T01:30:00.538 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pytz-2021.1-5.el9.noarch 2023-10-26T01:30:00.538 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rados-2:18.0.0-6863.g38499368.el9.x86_64 2023-10-26T01:30:00.538 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rbd-2:18.0.0-6863.g38499368.el9.x86_64 2023-10-26T01:30:00.538 INFO:teuthology.orchestra.run.smithi170.stdout: python3-repoze-lru-0.7-16.el9.noarch 2023-10-26T01:30:00.538 INFO:teuthology.orchestra.run.smithi170.stdout: python3-requests-2.25.1-7.el9.noarch 2023-10-26T01:30:00.539 INFO:teuthology.orchestra.run.smithi170.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2023-10-26T01:30:00.539 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rgw-2:18.0.0-6863.g38499368.el9.x86_64 2023-10-26T01:30:00.539 INFO:teuthology.orchestra.run.smithi170.stdout: python3-routes-2.5.1-5.el9.noarch 2023-10-26T01:30:00.539 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rsa-4.9-2.el9.noarch 2023-10-26T01:30:00.539 INFO:teuthology.orchestra.run.smithi170.stdout: python3-scipy-1.6.2-8.el9.x86_64 2023-10-26T01:30:00.540 INFO:teuthology.orchestra.run.smithi170.stdout: python3-tempora-5.0.0-2.el9.noarch 2023-10-26T01:30:00.540 INFO:teuthology.orchestra.run.smithi170.stdout: python3-toml-0.10.2-6.el9.noarch 2023-10-26T01:30:00.540 INFO:teuthology.orchestra.run.smithi170.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2023-10-26T01:30:00.540 INFO:teuthology.orchestra.run.smithi170.stdout: python3-urllib3-1.26.5-4.el9.noarch 2023-10-26T01:30:00.540 INFO:teuthology.orchestra.run.smithi170.stdout: python3-webob-1.8.7-6.el9.noarch 2023-10-26T01:30:00.540 INFO:teuthology.orchestra.run.smithi170.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2023-10-26T01:30:00.541 INFO:teuthology.orchestra.run.smithi170.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2023-10-26T01:30:00.541 INFO:teuthology.orchestra.run.smithi170.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2023-10-26T01:30:00.541 INFO:teuthology.orchestra.run.smithi170.stdout: rbd-fuse-2:18.0.0-6863.g38499368.el9.x86_64 2023-10-26T01:30:00.541 INFO:teuthology.orchestra.run.smithi170.stdout: rbd-mirror-2:18.0.0-6863.g38499368.el9.x86_64 2023-10-26T01:30:00.541 INFO:teuthology.orchestra.run.smithi170.stdout: rbd-nbd-2:18.0.0-6863.g38499368.el9.x86_64 2023-10-26T01:30:00.541 INFO:teuthology.orchestra.run.smithi170.stdout: re2-1:20211101-3.el9.x86_64 2023-10-26T01:30:00.542 INFO:teuthology.orchestra.run.smithi170.stdout: socat-1.7.4.1-5.el9.x86_64 2023-10-26T01:30:00.542 INFO:teuthology.orchestra.run.smithi170.stdout: thrift-0.15.0-2.el9.x86_64 2023-10-26T01:30:00.542 INFO:teuthology.orchestra.run.smithi170.stdout: uuid-1.6.2-55.el9.x86_64 2023-10-26T01:30:00.542 INFO:teuthology.orchestra.run.smithi170.stdout: xfsprogs-devel-5.19.0-4.el9.x86_64 2023-10-26T01:30:00.542 INFO:teuthology.orchestra.run.smithi170.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2023-10-26T01:30:00.543 INFO:teuthology.orchestra.run.smithi170.stdout: zlib-devel-1.2.11-40.el9.x86_64 2023-10-26T01:30:00.543 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-26T01:30:00.543 INFO:teuthology.orchestra.run.smithi170.stdout:Complete! 2023-10-26T01:30:01.034 DEBUG:teuthology.parallel:result is None 2023-10-26T01:30:01.035 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=384993681abceb8cb8affea10538d607052db4b4 2023-10-26T01:30:01.181 DEBUG:teuthology.orchestra.run.smithi169:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-10-26T01:30:01.242 INFO:teuthology.orchestra.run.smithi169.stdout:18.0.0-6863.g38499368.el9 2023-10-26T01:30:01.243 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-6863.g38499368.el9 2023-10-26T01:30:01.243 INFO:teuthology.task.install:The correct ceph version 18.0.0-6863.g38499368 is installed. 2023-10-26T01:30:01.244 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=384993681abceb8cb8affea10538d607052db4b4 2023-10-26T01:30:01.496 DEBUG:teuthology.orchestra.run.smithi170:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-10-26T01:30:01.557 INFO:teuthology.orchestra.run.smithi170.stdout:18.0.0-6863.g38499368.el9 2023-10-26T01:30:01.558 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-6863.g38499368.el9 2023-10-26T01:30:01.558 INFO:teuthology.task.install:The correct ceph version 18.0.0-6863.g38499368 is installed. 2023-10-26T01:30:01.559 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-10-26T01:30:01.559 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:30:01.560 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-26T01:30:01.645 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-26T01:30:01.646 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-26T01:30:01.724 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-10-26T01:30:01.725 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:30:01.725 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/usr/bin/daemon-helper 2023-10-26T01:30:01.802 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-26T01:30:01.917 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-26T01:30:01.917 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/usr/bin/daemon-helper 2023-10-26T01:30:02.013 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-26T01:30:02.106 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-10-26T01:30:02.107 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:30:02.107 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-26T01:30:02.186 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-26T01:30:02.302 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-26T01:30:02.302 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-26T01:30:02.386 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-26T01:30:02.502 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-10-26T01:30:02.503 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:30:02.503 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/usr/bin/stdin-killer 2023-10-26T01:30:02.585 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-26T01:30:02.698 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-26T01:30:02.699 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/usr/bin/stdin-killer 2023-10-26T01:30:02.779 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-26T01:30:02.894 INFO:teuthology.run_tasks:Running task ceph... 2023-10-26T01:30:02.965 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-10-26T01:30:02.965 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod 777 /var/log/ceph 2023-10-26T01:30:02.973 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod 777 /var/log/ceph 2023-10-26T01:30:03.053 INFO:tasks.ceph:Disabling ceph logrotate... 2023-10-26T01:30:03.053 DEBUG:teuthology.orchestra.run.smithi169:> sudo rm -f -- /etc/logrotate.d/ceph 2023-10-26T01:30:03.058 DEBUG:teuthology.orchestra.run.smithi170:> sudo rm -f -- /etc/logrotate.d/ceph 2023-10-26T01:30:03.141 INFO:tasks.ceph:Creating extra log directories... 2023-10-26T01:30:03.141 DEBUG:teuthology.orchestra.run.smithi169:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-10-26T01:30:03.146 DEBUG:teuthology.orchestra.run.smithi170:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-10-26T01:30:03.233 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-10-26T01:30:03.234 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-10-26T01:30:03.234 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-10-26T01:30:03.235 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:30:03.235 DEBUG:teuthology.orchestra.run.smithi169:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-10-26T01:30:03.276 DEBUG:teuthology.orchestra.run.smithi169:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-10-26T01:30:03.500 DEBUG:teuthology.orchestra.run.smithi169:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-10-26T01:30:03.593 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-26T01:30:03.594 DEBUG:teuthology.orchestra.run.smithi170:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-10-26T01:30:03.639 DEBUG:teuthology.orchestra.run.smithi170:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-10-26T01:30:03.859 DEBUG:teuthology.orchestra.run.smithi170:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-10-26T01:30:03.940 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-10-26T01:30:03.941 INFO:tasks.ceph:config {'conf': {'client': {'pid file': ''}, 'client.0': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'client.1': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'global': {'ms die on skipped message': False}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon osd backfillfull ratio': 0.6, 'mon osd full ratio': 0.7, 'mon osd nearfull ratio': 0.6}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'memstore device bytes': 200000000, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 1.0, 'osd mon report interval': 5, 'osd objectstore': 'memstore'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL', 'PG_RECOVERY_FULL', 'PG_DEGRADED'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-10-26T01:30:03.941 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/pdonnell-2023-10-24_19:52:40-fs-wip-batrick-testing-20231024.144545-distro-default-smithi/7436434', 'branch': 'wip-batrick-testing-20231024.144545', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/mds-full}', 'email': 'pdonnell@redhat.com', 'first_in_suite': False, 'job_id': '7436434', 'kernel': {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'}, 'ktype': 'upstream', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'pdonnell-2023-10-24_19:52:40-fs-wip-batrick-testing-20231024.144545-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-batrick-testing-20231024.144545'}, 'ceph': {'cephfs': {'ec_profile': ['disabled']}, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon': {'mon op complaint time': 120}, 'osd': {'osd op complaint time': 180}}, 'conf': {'client': {'pid file': ''}, 'client.0': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'client.1': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'global': {'ms die on skipped message': False}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon osd backfillfull ratio': 0.6, 'mon osd full ratio': 0.7, 'mon osd nearfull ratio': 0.6}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'memstore device bytes': 200000000, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 1.0, 'osd mon report interval': 5, 'osd objectstore': 'memstore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL', 'PG_RECOVERY_FULL', 'PG_DEGRADED'], 'sha1': '384993681abceb8cb8affea10538d607052db4b4'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '384993681abceb8cb8affea10538d607052db4b4'}}, 'kclient': {'syntax': 'v1'}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-batrick-testing-20231024.144545', 'sha1': '384993681abceb8cb8affea10538d607052db4b4'}}, 'owner': 'scheduled_pdonnell@teuthology', 'priority': 75, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 7595, 'sha1': '384993681abceb8cb8affea10538d607052db4b4', 'sleep_before_teardown': 0, 'subset': '66/256', 'suite': 'fs', 'suite_branch': 'wip-batrick-testing-20231024.144545', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_384993681abceb8cb8affea10538d607052db4b4/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '384993681abceb8cb8affea10538d607052db4b4', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi169.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBMurhwyAOgLDZYp8KLlJfM2tGAae+jI6aesQjNLyzgOViSaU190GbQt3iYkIuSCQpGqFwejukievn06h7IOmAQ=', 'smithi170.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOTrugeqpQFQkbL89tlvCCxYzOw91Gi/g8LcDRa/4kSdgb6tjnu8eLV09Jn06FyUdFgjJqFrxgYM07un/LnXr3g='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '384993681abceb8cb8affea10538d607052db4b4'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['disabled']}, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon': {'mon op complaint time': 120}, 'osd': {'osd op complaint time': 180}}, 'conf': {'client': {'pid file': ''}, 'client.0': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'client.1': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'global': {'ms die on skipped message': False}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon osd backfillfull ratio': 0.6, 'mon osd full ratio': 0.7, 'mon osd nearfull ratio': 0.6}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'memstore device bytes': 200000000, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 1.0, 'osd mon report interval': 5, 'osd objectstore': 'memstore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL', 'PG_RECOVERY_FULL', 'PG_DEGRADED'], 'sha1': '384993681abceb8cb8affea10538d607052db4b4', 'cluster': 'ceph'}}, {'kclient': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_full']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '202b180cb047e798fb131047314a862593f45403', 'timestamp': '2023-10-24_19:52:40', 'tube': 'smithi', 'user': 'pdonnell', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.770726'} 2023-10-26T01:30:03.942 DEBUG:teuthology.orchestra.run.smithi169:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-10-26T01:30:03.951 DEBUG:teuthology.orchestra.run.smithi170:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-10-26T01:30:04.001 DEBUG:teuthology.orchestra.run.smithi169:> sudo install -d -m0777 -- /var/run/ceph 2023-10-26T01:30:04.007 DEBUG:teuthology.orchestra.run.smithi170:> sudo install -d -m0777 -- /var/run/ceph 2023-10-26T01:30:04.127 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:30:04.127 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/scratch_devs of=/dev/stdout 2023-10-26T01:30:04.183 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-26T01:30:04.184 DEBUG:teuthology.orchestra.run.smithi169:> stat /dev/vg_nvme/lv_1 2023-10-26T01:30:04.235 INFO:teuthology.orchestra.run.smithi169.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-26T01:30:04.236 INFO:teuthology.orchestra.run.smithi169.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-26T01:30:04.236 INFO:teuthology.orchestra.run.smithi169.stdout:Device: 5h/5d Inode: 633 Links: 1 2023-10-26T01:30:04.236 INFO:teuthology.orchestra.run.smithi169.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-26T01:30:04.236 INFO:teuthology.orchestra.run.smithi169.stdout:Context: system_u:object_r:device_t:s0 2023-10-26T01:30:04.236 INFO:teuthology.orchestra.run.smithi169.stdout:Access: 2023-10-26 01:29:52.389002396 +0000 2023-10-26T01:30:04.237 INFO:teuthology.orchestra.run.smithi169.stdout:Modify: 2023-10-26 01:27:28.779223762 +0000 2023-10-26T01:30:04.237 INFO:teuthology.orchestra.run.smithi169.stdout:Change: 2023-10-26 01:27:28.779223762 +0000 2023-10-26T01:30:04.237 INFO:teuthology.orchestra.run.smithi169.stdout: Birth: 2023-10-26 01:27:28.779223762 +0000 2023-10-26T01:30:04.237 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-26T01:30:04.324 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records in 2023-10-26T01:30:04.325 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records out 2023-10-26T01:30:04.325 INFO:teuthology.orchestra.run.smithi169.stderr:512 bytes copied, 0.000366979 s, 1.4 MB/s 2023-10-26T01:30:04.328 DEBUG:teuthology.orchestra.run.smithi169:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-26T01:30:04.387 DEBUG:teuthology.orchestra.run.smithi169:> stat /dev/vg_nvme/lv_2 2023-10-26T01:30:04.439 INFO:teuthology.orchestra.run.smithi169.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-26T01:30:04.440 INFO:teuthology.orchestra.run.smithi169.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-26T01:30:04.440 INFO:teuthology.orchestra.run.smithi169.stdout:Device: 5h/5d Inode: 651 Links: 1 2023-10-26T01:30:04.440 INFO:teuthology.orchestra.run.smithi169.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-26T01:30:04.440 INFO:teuthology.orchestra.run.smithi169.stdout:Context: system_u:object_r:device_t:s0 2023-10-26T01:30:04.441 INFO:teuthology.orchestra.run.smithi169.stdout:Access: 2023-10-26 01:29:52.390002375 +0000 2023-10-26T01:30:04.441 INFO:teuthology.orchestra.run.smithi169.stdout:Modify: 2023-10-26 01:27:29.781202985 +0000 2023-10-26T01:30:04.441 INFO:teuthology.orchestra.run.smithi169.stdout:Change: 2023-10-26 01:27:29.781202985 +0000 2023-10-26T01:30:04.441 INFO:teuthology.orchestra.run.smithi169.stdout: Birth: 2023-10-26 01:27:29.781202985 +0000 2023-10-26T01:30:04.441 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-26T01:30:04.521 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records in 2023-10-26T01:30:04.522 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records out 2023-10-26T01:30:04.522 INFO:teuthology.orchestra.run.smithi169.stderr:512 bytes copied, 0.000328652 s, 1.6 MB/s 2023-10-26T01:30:04.524 DEBUG:teuthology.orchestra.run.smithi169:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-26T01:30:04.576 DEBUG:teuthology.orchestra.run.smithi169:> stat /dev/vg_nvme/lv_3 2023-10-26T01:30:04.624 INFO:teuthology.orchestra.run.smithi169.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-26T01:30:04.624 INFO:teuthology.orchestra.run.smithi169.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-26T01:30:04.625 INFO:teuthology.orchestra.run.smithi169.stdout:Device: 5h/5d Inode: 669 Links: 1 2023-10-26T01:30:04.625 INFO:teuthology.orchestra.run.smithi169.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-26T01:30:04.625 INFO:teuthology.orchestra.run.smithi169.stdout:Context: system_u:object_r:device_t:s0 2023-10-26T01:30:04.625 INFO:teuthology.orchestra.run.smithi169.stdout:Access: 2023-10-26 01:29:52.391002354 +0000 2023-10-26T01:30:04.625 INFO:teuthology.orchestra.run.smithi169.stdout:Modify: 2023-10-26 01:27:30.802181814 +0000 2023-10-26T01:30:04.626 INFO:teuthology.orchestra.run.smithi169.stdout:Change: 2023-10-26 01:27:30.802181814 +0000 2023-10-26T01:30:04.626 INFO:teuthology.orchestra.run.smithi169.stdout: Birth: 2023-10-26 01:27:30.802181814 +0000 2023-10-26T01:30:04.626 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-26T01:30:04.711 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records in 2023-10-26T01:30:04.711 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records out 2023-10-26T01:30:04.711 INFO:teuthology.orchestra.run.smithi169.stderr:512 bytes copied, 0.000351781 s, 1.5 MB/s 2023-10-26T01:30:04.715 DEBUG:teuthology.orchestra.run.smithi169:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-26T01:30:04.772 DEBUG:teuthology.orchestra.run.smithi169:> stat /dev/vg_nvme/lv_4 2023-10-26T01:30:04.822 INFO:teuthology.orchestra.run.smithi169.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-26T01:30:04.822 INFO:teuthology.orchestra.run.smithi169.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-26T01:30:04.822 INFO:teuthology.orchestra.run.smithi169.stdout:Device: 5h/5d Inode: 691 Links: 1 2023-10-26T01:30:04.823 INFO:teuthology.orchestra.run.smithi169.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-26T01:30:04.823 INFO:teuthology.orchestra.run.smithi169.stdout:Context: system_u:object_r:device_t:s0 2023-10-26T01:30:04.823 INFO:teuthology.orchestra.run.smithi169.stdout:Access: 2023-10-26 01:29:52.391002354 +0000 2023-10-26T01:30:04.823 INFO:teuthology.orchestra.run.smithi169.stdout:Modify: 2023-10-26 01:27:31.796161204 +0000 2023-10-26T01:30:04.823 INFO:teuthology.orchestra.run.smithi169.stdout:Change: 2023-10-26 01:27:31.796161204 +0000 2023-10-26T01:30:04.824 INFO:teuthology.orchestra.run.smithi169.stdout: Birth: 2023-10-26 01:27:31.796161204 +0000 2023-10-26T01:30:04.824 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-26T01:30:04.912 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records in 2023-10-26T01:30:04.912 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records out 2023-10-26T01:30:04.913 INFO:teuthology.orchestra.run.smithi169.stderr:512 bytes copied, 0.000322573 s, 1.6 MB/s 2023-10-26T01:30:04.916 DEBUG:teuthology.orchestra.run.smithi169:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-26T01:30:04.973 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-26T01:30:04.973 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-26T01:30:04.974 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/scratch_devs of=/dev/stdout 2023-10-26T01:30:05.033 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-26T01:30:05.033 DEBUG:teuthology.orchestra.run.smithi170:> stat /dev/vg_nvme/lv_1 2023-10-26T01:30:05.085 INFO:teuthology.orchestra.run.smithi170.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-26T01:30:05.085 INFO:teuthology.orchestra.run.smithi170.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-26T01:30:05.085 INFO:teuthology.orchestra.run.smithi170.stdout:Device: 5h/5d Inode: 633 Links: 1 2023-10-26T01:30:05.086 INFO:teuthology.orchestra.run.smithi170.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-26T01:30:05.086 INFO:teuthology.orchestra.run.smithi170.stdout:Context: system_u:object_r:device_t:s0 2023-10-26T01:30:05.086 INFO:teuthology.orchestra.run.smithi170.stdout:Access: 2023-10-26 01:29:53.995759799 +0000 2023-10-26T01:30:05.086 INFO:teuthology.orchestra.run.smithi170.stdout:Modify: 2023-10-26 01:27:28.741863833 +0000 2023-10-26T01:30:05.086 INFO:teuthology.orchestra.run.smithi170.stdout:Change: 2023-10-26 01:27:28.741863833 +0000 2023-10-26T01:30:05.087 INFO:teuthology.orchestra.run.smithi170.stdout: Birth: 2023-10-26 01:27:28.741863833 +0000 2023-10-26T01:30:05.087 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-26T01:30:05.175 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records in 2023-10-26T01:30:05.176 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records out 2023-10-26T01:30:05.176 INFO:teuthology.orchestra.run.smithi170.stderr:512 bytes copied, 0.000286119 s, 1.8 MB/s 2023-10-26T01:30:05.180 DEBUG:teuthology.orchestra.run.smithi170:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-26T01:30:05.237 DEBUG:teuthology.orchestra.run.smithi170:> stat /dev/vg_nvme/lv_2 2023-10-26T01:30:05.287 INFO:teuthology.orchestra.run.smithi170.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-26T01:30:05.287 INFO:teuthology.orchestra.run.smithi170.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-26T01:30:05.287 INFO:teuthology.orchestra.run.smithi170.stdout:Device: 5h/5d Inode: 650 Links: 1 2023-10-26T01:30:05.287 INFO:teuthology.orchestra.run.smithi170.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-26T01:30:05.288 INFO:teuthology.orchestra.run.smithi170.stdout:Context: system_u:object_r:device_t:s0 2023-10-26T01:30:05.288 INFO:teuthology.orchestra.run.smithi170.stdout:Access: 2023-10-26 01:29:53.995759799 +0000 2023-10-26T01:30:05.288 INFO:teuthology.orchestra.run.smithi170.stdout:Modify: 2023-10-26 01:27:29.738840795 +0000 2023-10-26T01:30:05.288 INFO:teuthology.orchestra.run.smithi170.stdout:Change: 2023-10-26 01:27:29.738840795 +0000 2023-10-26T01:30:05.288 INFO:teuthology.orchestra.run.smithi170.stdout: Birth: 2023-10-26 01:27:29.738840795 +0000 2023-10-26T01:30:05.289 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-26T01:30:05.377 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records in 2023-10-26T01:30:05.377 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records out 2023-10-26T01:30:05.378 INFO:teuthology.orchestra.run.smithi170.stderr:512 bytes copied, 0.000232202 s, 2.2 MB/s 2023-10-26T01:30:05.381 DEBUG:teuthology.orchestra.run.smithi170:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-26T01:30:05.439 DEBUG:teuthology.orchestra.run.smithi170:> stat /dev/vg_nvme/lv_3 2023-10-26T01:30:05.490 INFO:teuthology.orchestra.run.smithi170.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-26T01:30:05.491 INFO:teuthology.orchestra.run.smithi170.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-26T01:30:05.491 INFO:teuthology.orchestra.run.smithi170.stdout:Device: 5h/5d Inode: 669 Links: 1 2023-10-26T01:30:05.491 INFO:teuthology.orchestra.run.smithi170.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-26T01:30:05.491 INFO:teuthology.orchestra.run.smithi170.stdout:Context: system_u:object_r:device_t:s0 2023-10-26T01:30:05.492 INFO:teuthology.orchestra.run.smithi170.stdout:Access: 2023-10-26 01:29:53.997759753 +0000 2023-10-26T01:30:05.492 INFO:teuthology.orchestra.run.smithi170.stdout:Modify: 2023-10-26 01:27:30.783816648 +0000 2023-10-26T01:30:05.492 INFO:teuthology.orchestra.run.smithi170.stdout:Change: 2023-10-26 01:27:30.783816648 +0000 2023-10-26T01:30:05.492 INFO:teuthology.orchestra.run.smithi170.stdout: Birth: 2023-10-26 01:27:30.783816648 +0000 2023-10-26T01:30:05.493 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-26T01:30:05.578 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records in 2023-10-26T01:30:05.579 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records out 2023-10-26T01:30:05.579 INFO:teuthology.orchestra.run.smithi170.stderr:512 bytes copied, 0.00025451 s, 2.0 MB/s 2023-10-26T01:30:05.582 DEBUG:teuthology.orchestra.run.smithi170:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-26T01:30:05.637 DEBUG:teuthology.orchestra.run.smithi170:> stat /dev/vg_nvme/lv_4 2023-10-26T01:30:05.689 INFO:teuthology.orchestra.run.smithi170.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-26T01:30:05.689 INFO:teuthology.orchestra.run.smithi170.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-26T01:30:05.689 INFO:teuthology.orchestra.run.smithi170.stdout:Device: 5h/5d Inode: 688 Links: 1 2023-10-26T01:30:05.690 INFO:teuthology.orchestra.run.smithi170.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-26T01:30:05.690 INFO:teuthology.orchestra.run.smithi170.stdout:Context: system_u:object_r:device_t:s0 2023-10-26T01:30:05.690 INFO:teuthology.orchestra.run.smithi170.stdout:Access: 2023-10-26 01:29:53.997759753 +0000 2023-10-26T01:30:05.690 INFO:teuthology.orchestra.run.smithi170.stdout:Modify: 2023-10-26 01:27:31.794793287 +0000 2023-10-26T01:30:05.690 INFO:teuthology.orchestra.run.smithi170.stdout:Change: 2023-10-26 01:27:31.794793287 +0000 2023-10-26T01:30:05.691 INFO:teuthology.orchestra.run.smithi170.stdout: Birth: 2023-10-26 01:27:31.794793287 +0000 2023-10-26T01:30:05.691 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-26T01:30:05.782 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records in 2023-10-26T01:30:05.782 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records out 2023-10-26T01:30:05.782 INFO:teuthology.orchestra.run.smithi170.stderr:512 bytes copied, 0.00073966 s, 692 kB/s 2023-10-26T01:30:05.786 DEBUG:teuthology.orchestra.run.smithi170:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-26T01:30:05.838 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-26T01:30:05.838 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi169.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi170.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-10-26T01:30:05.839 INFO:tasks.ceph:Generating config... 2023-10-26T01:30:05.840 INFO:tasks.ceph:[client] pid file = 2023-10-26T01:30:05.840 INFO:tasks.ceph:[client.0] debug client = 20 2023-10-26T01:30:05.840 INFO:tasks.ceph:[client.0] debug objectcacher = 20 2023-10-26T01:30:05.841 INFO:tasks.ceph:[client.0] debug objecter = 20 2023-10-26T01:30:05.841 INFO:tasks.ceph:[client.1] debug client = 20 2023-10-26T01:30:05.841 INFO:tasks.ceph:[client.1] debug objectcacher = 20 2023-10-26T01:30:05.841 INFO:tasks.ceph:[client.1] debug objecter = 20 2023-10-26T01:30:05.842 INFO:tasks.ceph:[global] ms die on skipped message = False 2023-10-26T01:30:05.842 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-10-26T01:30:05.842 INFO:tasks.ceph:[mgr] debug ms = 1 2023-10-26T01:30:05.842 INFO:tasks.ceph:[mon] debug mon = 20 2023-10-26T01:30:05.843 INFO:tasks.ceph:[mon] debug ms = 1 2023-10-26T01:30:05.843 INFO:tasks.ceph:[mon] debug paxos = 20 2023-10-26T01:30:05.843 INFO:tasks.ceph:[mon] mon osd backfillfull ratio = 0.6 2023-10-26T01:30:05.843 INFO:tasks.ceph:[mon] mon osd full ratio = 0.7 2023-10-26T01:30:05.844 INFO:tasks.ceph:[mon] mon osd nearfull ratio = 0.6 2023-10-26T01:30:05.844 INFO:tasks.ceph:[osd] bdev async discard = True 2023-10-26T01:30:05.844 INFO:tasks.ceph:[osd] bdev enable discard = True 2023-10-26T01:30:05.844 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2023-10-26T01:30:05.845 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-10-26T01:30:05.845 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-10-26T01:30:05.845 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2023-10-26T01:30:05.845 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2023-10-26T01:30:05.846 INFO:tasks.ceph:[osd] debug ms = 1 2023-10-26T01:30:05.846 INFO:tasks.ceph:[osd] debug osd = 20 2023-10-26T01:30:05.846 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2023-10-26T01:30:05.846 INFO:tasks.ceph:[osd] memstore device bytes = 200000000 2023-10-26T01:30:05.847 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-10-26T01:30:05.847 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-10-26T01:30:05.847 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-10-26T01:30:05.847 INFO:tasks.ceph:[osd] osd failsafe full ratio = 1.0 2023-10-26T01:30:05.848 INFO:tasks.ceph:[osd] osd mon report interval = 5 2023-10-26T01:30:05.848 INFO:tasks.ceph:[osd] osd objectstore = memstore 2023-10-26T01:30:05.848 INFO:tasks.ceph:Setting up mon.a... 2023-10-26T01:30:05.849 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-10-26T01:30:05.992 INFO:teuthology.orchestra.run.smithi169.stdout:creating /etc/ceph/ceph.keyring 2023-10-26T01:30:06.000 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-10-26T01:30:06.133 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-26T01:30:06.215 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.169'), ('mon.b', '172.21.15.170'), ('mon.c', '[v2:172.21.15.170:3301,v1:172.21.15.170:6790]')] 2023-10-26T01:30:06.216 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.169,172.21.15.170,[v2:172.21.15.170:3301,v1:172.21.15.170:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'memstore device bytes': 200000000, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 1.0, 'osd mon report interval': 5, 'osd objectstore': 'memstore'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon osd backfillfull ratio': 0.6, 'mon osd full ratio': 0.7, 'mon osd nearfull ratio': 0.6}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'pid file': ''}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'client.0': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'client.1': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}} 2023-10-26T01:30:06.217 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:30:06.217 DEBUG:teuthology.orchestra.run.smithi169:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-10-26T01:30:06.262 DEBUG:teuthology.orchestra.run.smithi169:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.169 --add b 172.21.15.170 --addv c '[v2:172.21.15.170:3301,v1:172.21.15.170:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-10-26T01:30:06.415 INFO:teuthology.orchestra.run.smithi169.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-10-26T01:30:06.415 INFO:teuthology.orchestra.run.smithi169.stdout:monmaptool: generated fsid 30203b28-ed8a-44e8-a9a6-a9bf25f980f1 2023-10-26T01:30:06.415 INFO:teuthology.orchestra.run.smithi169.stdout:setting min_mon_release = pacific 2023-10-26T01:30:06.416 INFO:teuthology.orchestra.run.smithi169.stdout:epoch 0 2023-10-26T01:30:06.416 INFO:teuthology.orchestra.run.smithi169.stdout:fsid 30203b28-ed8a-44e8-a9a6-a9bf25f980f1 2023-10-26T01:30:06.416 INFO:teuthology.orchestra.run.smithi169.stdout:last_changed 2023-10-26T01:30:06.414410+0000 2023-10-26T01:30:06.416 INFO:teuthology.orchestra.run.smithi169.stdout:created 2023-10-26T01:30:06.414410+0000 2023-10-26T01:30:06.416 INFO:teuthology.orchestra.run.smithi169.stdout:min_mon_release 16 (pacific) 2023-10-26T01:30:06.417 INFO:teuthology.orchestra.run.smithi169.stdout:election_strategy: 1 2023-10-26T01:30:06.417 INFO:teuthology.orchestra.run.smithi169.stdout:0: [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] mon.a 2023-10-26T01:30:06.417 INFO:teuthology.orchestra.run.smithi169.stdout:1: [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] mon.b 2023-10-26T01:30:06.417 INFO:teuthology.orchestra.run.smithi169.stdout:2: [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] mon.c 2023-10-26T01:30:06.417 INFO:teuthology.orchestra.run.smithi169.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-10-26T01:30:06.419 DEBUG:teuthology.orchestra.run.smithi169:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-10-26T01:30:06.472 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 30203b28-ed8a-44e8-a9a6-a9bf25f980f1... 2023-10-26T01:30:06.473 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-26T01:30:06.479 DEBUG:teuthology.orchestra.run.smithi170:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-26T01:30:06.639 INFO:teuthology.orchestra.run.smithi169.stdout:[global] 2023-10-26T01:30:06.639 INFO:teuthology.orchestra.run.smithi169.stdout: chdir = "" 2023-10-26T01:30:06.639 INFO:teuthology.orchestra.run.smithi169.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-26T01:30:06.639 INFO:teuthology.orchestra.run.smithi169.stdout: auth supported = cephx 2023-10-26T01:30:06.640 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:30:06.640 INFO:teuthology.orchestra.run.smithi169.stdout: filestore xattr use omap = true 2023-10-26T01:30:06.640 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:30:06.640 INFO:teuthology.orchestra.run.smithi169.stdout: mon clock drift allowed = 1.000 2023-10-26T01:30:06.640 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:30:06.641 INFO:teuthology.orchestra.run.smithi169.stdout: osd crush chooseleaf type = 0 2023-10-26T01:30:06.641 INFO:teuthology.orchestra.run.smithi169.stdout: auth debug = true 2023-10-26T01:30:06.641 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:30:06.641 INFO:teuthology.orchestra.run.smithi169.stdout: ms die on old message = true 2023-10-26T01:30:06.641 INFO:teuthology.orchestra.run.smithi169.stdout: ms die on bug = true 2023-10-26T01:30:06.641 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:30:06.642 INFO:teuthology.orchestra.run.smithi169.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-26T01:30:06.642 INFO:teuthology.orchestra.run.smithi169.stdout: mon pg warn max object skew = 0 2023-10-26T01:30:06.642 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:30:06.642 INFO:teuthology.orchestra.run.smithi169.stdout: # disable pg_autoscaler by default for new pools 2023-10-26T01:30:06.642 INFO:teuthology.orchestra.run.smithi169.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-26T01:30:06.643 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:30:06.643 INFO:teuthology.orchestra.run.smithi169.stdout: osd pool default size = 2 2023-10-26T01:30:06.643 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:30:06.643 INFO:teuthology.orchestra.run.smithi169.stdout: mon osd allow primary affinity = true 2023-10-26T01:30:06.643 INFO:teuthology.orchestra.run.smithi169.stdout: mon osd allow pg remap = true 2023-10-26T01:30:06.644 INFO:teuthology.orchestra.run.smithi169.stdout: mon warn on legacy crush tunables = false 2023-10-26T01:30:06.644 INFO:teuthology.orchestra.run.smithi169.stdout: mon warn on crush straw calc version zero = false 2023-10-26T01:30:06.644 INFO:teuthology.orchestra.run.smithi169.stdout: mon warn on no sortbitwise = false 2023-10-26T01:30:06.644 INFO:teuthology.orchestra.run.smithi169.stdout: mon warn on osd down out interval zero = false 2023-10-26T01:30:06.644 INFO:teuthology.orchestra.run.smithi169.stdout: mon warn on too few osds = false 2023-10-26T01:30:06.645 INFO:teuthology.orchestra.run.smithi169.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-26T01:30:06.645 INFO:teuthology.orchestra.run.smithi169.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-26T01:30:06.645 INFO:teuthology.orchestra.run.smithi169.stdout: mon_allow_pool_size_one = true 2023-10-26T01:30:06.645 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:30:06.645 INFO:teuthology.orchestra.run.smithi169.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-26T01:30:06.646 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:30:06.646 INFO:teuthology.orchestra.run.smithi169.stdout: osd default data pool replay window = 5 2023-10-26T01:30:06.646 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:30:06.646 INFO:teuthology.orchestra.run.smithi169.stdout: mon allow pool delete = true 2023-10-26T01:30:06.646 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:30:06.647 INFO:teuthology.orchestra.run.smithi169.stdout: mon cluster log file level = debug 2023-10-26T01:30:06.647 INFO:teuthology.orchestra.run.smithi169.stdout: debug asserts on shutdown = true 2023-10-26T01:30:06.647 INFO:teuthology.orchestra.run.smithi169.stdout: mon health detail to clog = false 2023-10-26T01:30:06.647 INFO:teuthology.orchestra.run.smithi169.stdout: mon host = "172.21.15.169,172.21.15.170,[v2:172.21.15.170:3301,v1:172.21.15.170:6790]" 2023-10-26T01:30:06.647 INFO:teuthology.orchestra.run.smithi169.stdout: ms die on skipped message = False 2023-10-26T01:30:06.648 INFO:teuthology.orchestra.run.smithi169.stdout: fsid = 30203b28-ed8a-44e8-a9a6-a9bf25f980f1 2023-10-26T01:30:06.648 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:30:06.648 INFO:teuthology.orchestra.run.smithi169.stdout:[osd] 2023-10-26T01:30:06.648 INFO:teuthology.orchestra.run.smithi169.stdout: osd journal size = 100 2023-10-26T01:30:06.648 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:30:06.648 INFO:teuthology.orchestra.run.smithi169.stdout: osd scrub load threshold = 5.0 2023-10-26T01:30:06.649 INFO:teuthology.orchestra.run.smithi169.stdout: osd scrub max interval = 600 2023-10-26T01:30:06.649 INFO:teuthology.orchestra.run.smithi169.stdout: osd mclock profile = high_recovery_ops 2023-10-26T01:30:06.649 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:30:06.649 INFO:teuthology.orchestra.run.smithi169.stdout: osd recover clone overlap = true 2023-10-26T01:30:06.649 INFO:teuthology.orchestra.run.smithi169.stdout: osd recovery max chunk = 1048576 2023-10-26T01:30:06.650 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:30:06.650 INFO:teuthology.orchestra.run.smithi169.stdout: osd debug shutdown = true 2023-10-26T01:30:06.650 INFO:teuthology.orchestra.run.smithi169.stdout: osd debug op order = true 2023-10-26T01:30:06.650 INFO:teuthology.orchestra.run.smithi169.stdout: osd debug verify stray on activate = true 2023-10-26T01:30:06.650 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:30:06.651 INFO:teuthology.orchestra.run.smithi169.stdout: osd open classes on start = true 2023-10-26T01:30:06.651 INFO:teuthology.orchestra.run.smithi169.stdout: osd debug pg log writeout = true 2023-10-26T01:30:06.651 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:30:06.651 INFO:teuthology.orchestra.run.smithi169.stdout: osd deep scrub update digest min age = 30 2023-10-26T01:30:06.651 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:30:06.652 INFO:teuthology.orchestra.run.smithi169.stdout: osd map max advance = 10 2023-10-26T01:30:06.652 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:30:06.652 INFO:teuthology.orchestra.run.smithi169.stdout: journal zero on create = true 2023-10-26T01:30:06.652 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:30:06.652 INFO:teuthology.orchestra.run.smithi169.stdout: filestore ondisk finisher threads = 3 2023-10-26T01:30:06.653 INFO:teuthology.orchestra.run.smithi169.stdout: filestore apply finisher threads = 3 2023-10-26T01:30:06.653 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:30:06.653 INFO:teuthology.orchestra.run.smithi169.stdout: bdev debug aio = true 2023-10-26T01:30:06.653 INFO:teuthology.orchestra.run.smithi169.stdout: osd debug misdirected ops = true 2023-10-26T01:30:06.653 INFO:teuthology.orchestra.run.smithi169.stdout: bdev async discard = True 2023-10-26T01:30:06.653 INFO:teuthology.orchestra.run.smithi169.stdout: bdev enable discard = True 2023-10-26T01:30:06.654 INFO:teuthology.orchestra.run.smithi169.stdout: bluestore allocator = bitmap 2023-10-26T01:30:06.654 INFO:teuthology.orchestra.run.smithi169.stdout: bluestore block size = 96636764160 2023-10-26T01:30:06.654 INFO:teuthology.orchestra.run.smithi169.stdout: bluestore fsck on mount = True 2023-10-26T01:30:06.654 INFO:teuthology.orchestra.run.smithi169.stdout: debug bluefs = 1/20 2023-10-26T01:30:06.654 INFO:teuthology.orchestra.run.smithi169.stdout: debug bluestore = 1/20 2023-10-26T01:30:06.655 INFO:teuthology.orchestra.run.smithi169.stdout: debug ms = 1 2023-10-26T01:30:06.655 INFO:teuthology.orchestra.run.smithi169.stdout: debug osd = 20 2023-10-26T01:30:06.655 INFO:teuthology.orchestra.run.smithi169.stdout: debug rocksdb = 4/10 2023-10-26T01:30:06.655 INFO:teuthology.orchestra.run.smithi169.stdout: memstore device bytes = 200000000 2023-10-26T01:30:06.655 INFO:teuthology.orchestra.run.smithi169.stdout: mon osd backfillfull_ratio = 0.85 2023-10-26T01:30:06.656 INFO:teuthology.orchestra.run.smithi169.stdout: mon osd full ratio = 0.9 2023-10-26T01:30:06.656 INFO:teuthology.orchestra.run.smithi169.stdout: mon osd nearfull ratio = 0.8 2023-10-26T01:30:06.656 INFO:teuthology.orchestra.run.smithi169.stdout: osd failsafe full ratio = 1.0 2023-10-26T01:30:06.656 INFO:teuthology.orchestra.run.smithi169.stdout: osd mon report interval = 5 2023-10-26T01:30:06.656 INFO:teuthology.orchestra.run.smithi169.stdout: osd objectstore = memstore 2023-10-26T01:30:06.657 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:30:06.657 INFO:teuthology.orchestra.run.smithi169.stdout:[mgr] 2023-10-26T01:30:06.657 INFO:teuthology.orchestra.run.smithi169.stdout: debug ms = 1 2023-10-26T01:30:06.657 INFO:teuthology.orchestra.run.smithi169.stdout: debug mgr = 20 2023-10-26T01:30:06.657 INFO:teuthology.orchestra.run.smithi169.stdout: debug mon = 20 2023-10-26T01:30:06.658 INFO:teuthology.orchestra.run.smithi169.stdout: debug auth = 20 2023-10-26T01:30:06.658 INFO:teuthology.orchestra.run.smithi169.stdout: mon reweight min pgs per osd = 4 2023-10-26T01:30:06.658 INFO:teuthology.orchestra.run.smithi169.stdout: mon reweight min bytes per osd = 10 2023-10-26T01:30:06.658 INFO:teuthology.orchestra.run.smithi169.stdout: mgr/telemetry/nag = false 2023-10-26T01:30:06.658 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:30:06.659 INFO:teuthology.orchestra.run.smithi169.stdout:[mon] 2023-10-26T01:30:06.659 INFO:teuthology.orchestra.run.smithi169.stdout: debug ms = 1 2023-10-26T01:30:06.659 INFO:teuthology.orchestra.run.smithi169.stdout: debug mon = 20 2023-10-26T01:30:06.659 INFO:teuthology.orchestra.run.smithi169.stdout: debug paxos = 20 2023-10-26T01:30:06.659 INFO:teuthology.orchestra.run.smithi169.stdout: debug auth = 20 2023-10-26T01:30:06.660 INFO:teuthology.orchestra.run.smithi169.stdout: mon data avail warn = 5 2023-10-26T01:30:06.660 INFO:teuthology.orchestra.run.smithi169.stdout: mon mgr mkfs grace = 240 2023-10-26T01:30:06.660 INFO:teuthology.orchestra.run.smithi169.stdout: mon reweight min pgs per osd = 4 2023-10-26T01:30:06.660 INFO:teuthology.orchestra.run.smithi169.stdout: mon osd reporter subtree level = osd 2023-10-26T01:30:06.660 INFO:teuthology.orchestra.run.smithi169.stdout: mon osd prime pg temp = true 2023-10-26T01:30:06.660 INFO:teuthology.orchestra.run.smithi169.stdout: mon reweight min bytes per osd = 10 2023-10-26T01:30:06.661 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:30:06.661 INFO:teuthology.orchestra.run.smithi169.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-26T01:30:06.661 INFO:teuthology.orchestra.run.smithi169.stdout: auth mon ticket ttl = 660 # 11m 2023-10-26T01:30:06.661 INFO:teuthology.orchestra.run.smithi169.stdout: auth service ticket ttl = 240 # 4m 2023-10-26T01:30:06.661 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:30:06.662 INFO:teuthology.orchestra.run.smithi169.stdout: # don't complain about insecure global_id in the test suite 2023-10-26T01:30:06.662 INFO:teuthology.orchestra.run.smithi169.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-26T01:30:06.662 INFO:teuthology.orchestra.run.smithi169.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-26T01:30:06.662 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:30:06.662 INFO:teuthology.orchestra.run.smithi169.stdout: # 1m isn't quite enough 2023-10-26T01:30:06.663 INFO:teuthology.orchestra.run.smithi169.stdout: mon_down_mkfs_grace = 2m 2023-10-26T01:30:06.663 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:30:06.663 INFO:teuthology.orchestra.run.smithi169.stdout: mon_warn_on_filestore_osds = false 2023-10-26T01:30:06.663 INFO:teuthology.orchestra.run.smithi169.stdout: mon osd backfillfull ratio = 0.6 2023-10-26T01:30:06.663 INFO:teuthology.orchestra.run.smithi169.stdout: mon osd full ratio = 0.7 2023-10-26T01:30:06.664 INFO:teuthology.orchestra.run.smithi169.stdout: mon osd nearfull ratio = 0.6 2023-10-26T01:30:06.664 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:30:06.664 INFO:teuthology.orchestra.run.smithi169.stdout:[client] 2023-10-26T01:30:06.664 INFO:teuthology.orchestra.run.smithi169.stdout: rgw cache enabled = true 2023-10-26T01:30:06.664 INFO:teuthology.orchestra.run.smithi169.stdout: rgw enable ops log = true 2023-10-26T01:30:06.664 INFO:teuthology.orchestra.run.smithi169.stdout: rgw enable usage log = true 2023-10-26T01:30:06.665 INFO:teuthology.orchestra.run.smithi169.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-26T01:30:06.665 INFO:teuthology.orchestra.run.smithi169.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-26T01:30:06.665 INFO:teuthology.orchestra.run.smithi169.stdout: pid file = "" 2023-10-26T01:30:06.665 INFO:teuthology.orchestra.run.smithi169.stdout:[mon.a] 2023-10-26T01:30:06.666 INFO:teuthology.orchestra.run.smithi169.stdout:[mon.b] 2023-10-26T01:30:06.666 INFO:teuthology.orchestra.run.smithi169.stdout:[mon.c] 2023-10-26T01:30:06.666 INFO:teuthology.orchestra.run.smithi169.stdout:[mds.a] 2023-10-26T01:30:06.666 INFO:teuthology.orchestra.run.smithi169.stdout:[mds.c] 2023-10-26T01:30:06.666 INFO:teuthology.orchestra.run.smithi169.stdout:[mds.b] 2023-10-26T01:30:06.667 INFO:teuthology.orchestra.run.smithi169.stdout:[mds.d] 2023-10-26T01:30:06.667 INFO:teuthology.orchestra.run.smithi169.stdout:[client.0] 2023-10-26T01:30:06.667 INFO:teuthology.orchestra.run.smithi169.stdout: debug client = 20 2023-10-26T01:30:06.667 INFO:teuthology.orchestra.run.smithi169.stdout: debug objectcacher = 20 2023-10-26T01:30:06.667 INFO:teuthology.orchestra.run.smithi169.stdout: debug objecter = 20 2023-10-26T01:30:06.668 INFO:teuthology.orchestra.run.smithi169.stdout:[client.1] 2023-10-26T01:30:06.668 INFO:teuthology.orchestra.run.smithi169.stdout: debug client = 20 2023-10-26T01:30:06.668 INFO:teuthology.orchestra.run.smithi169.stdout: debug objectcacher = 20 2023-10-26T01:30:06.668 INFO:teuthology.orchestra.run.smithi169.stdout: debug objecter = 20 2023-10-26T01:30:06.669 INFO:teuthology.orchestra.run.smithi170.stdout:[global] 2023-10-26T01:30:06.669 INFO:teuthology.orchestra.run.smithi170.stdout: chdir = "" 2023-10-26T01:30:06.669 INFO:teuthology.orchestra.run.smithi170.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-26T01:30:06.669 INFO:teuthology.orchestra.run.smithi170.stdout: auth supported = cephx 2023-10-26T01:30:06.669 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-26T01:30:06.670 INFO:teuthology.orchestra.run.smithi170.stdout: filestore xattr use omap = true 2023-10-26T01:30:06.670 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-26T01:30:06.670 INFO:teuthology.orchestra.run.smithi170.stdout: mon clock drift allowed = 1.000 2023-10-26T01:30:06.670 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-26T01:30:06.670 INFO:teuthology.orchestra.run.smithi170.stdout: osd crush chooseleaf type = 0 2023-10-26T01:30:06.671 INFO:teuthology.orchestra.run.smithi170.stdout: auth debug = true 2023-10-26T01:30:06.671 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-26T01:30:06.671 INFO:teuthology.orchestra.run.smithi170.stdout: ms die on old message = true 2023-10-26T01:30:06.671 INFO:teuthology.orchestra.run.smithi170.stdout: ms die on bug = true 2023-10-26T01:30:06.672 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-26T01:30:06.672 INFO:teuthology.orchestra.run.smithi170.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-26T01:30:06.672 INFO:teuthology.orchestra.run.smithi170.stdout: mon pg warn max object skew = 0 2023-10-26T01:30:06.672 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-26T01:30:06.672 INFO:teuthology.orchestra.run.smithi170.stdout: # disable pg_autoscaler by default for new pools 2023-10-26T01:30:06.672 INFO:teuthology.orchestra.run.smithi170.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-26T01:30:06.673 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-26T01:30:06.673 INFO:teuthology.orchestra.run.smithi170.stdout: osd pool default size = 2 2023-10-26T01:30:06.673 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-26T01:30:06.673 INFO:teuthology.orchestra.run.smithi170.stdout: mon osd allow primary affinity = true 2023-10-26T01:30:06.673 INFO:teuthology.orchestra.run.smithi170.stdout: mon osd allow pg remap = true 2023-10-26T01:30:06.674 INFO:teuthology.orchestra.run.smithi170.stdout: mon warn on legacy crush tunables = false 2023-10-26T01:30:06.674 INFO:teuthology.orchestra.run.smithi170.stdout: mon warn on crush straw calc version zero = false 2023-10-26T01:30:06.674 INFO:teuthology.orchestra.run.smithi170.stdout: mon warn on no sortbitwise = false 2023-10-26T01:30:06.674 INFO:teuthology.orchestra.run.smithi170.stdout: mon warn on osd down out interval zero = false 2023-10-26T01:30:06.674 INFO:teuthology.orchestra.run.smithi170.stdout: mon warn on too few osds = false 2023-10-26T01:30:06.675 INFO:teuthology.orchestra.run.smithi170.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-26T01:30:06.675 INFO:teuthology.orchestra.run.smithi170.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-26T01:30:06.675 INFO:teuthology.orchestra.run.smithi170.stdout: mon_allow_pool_size_one = true 2023-10-26T01:30:06.675 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-26T01:30:06.675 INFO:teuthology.orchestra.run.smithi170.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-26T01:30:06.675 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-26T01:30:06.676 INFO:teuthology.orchestra.run.smithi170.stdout: osd default data pool replay window = 5 2023-10-26T01:30:06.676 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-26T01:30:06.676 INFO:teuthology.orchestra.run.smithi170.stdout: mon allow pool delete = true 2023-10-26T01:30:06.676 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-26T01:30:06.676 INFO:teuthology.orchestra.run.smithi170.stdout: mon cluster log file level = debug 2023-10-26T01:30:06.677 INFO:teuthology.orchestra.run.smithi170.stdout: debug asserts on shutdown = true 2023-10-26T01:30:06.677 INFO:teuthology.orchestra.run.smithi170.stdout: mon health detail to clog = false 2023-10-26T01:30:06.677 INFO:teuthology.orchestra.run.smithi170.stdout: mon host = "172.21.15.169,172.21.15.170,[v2:172.21.15.170:3301,v1:172.21.15.170:6790]" 2023-10-26T01:30:06.677 INFO:teuthology.orchestra.run.smithi170.stdout: ms die on skipped message = False 2023-10-26T01:30:06.677 INFO:teuthology.orchestra.run.smithi170.stdout: fsid = 30203b28-ed8a-44e8-a9a6-a9bf25f980f1 2023-10-26T01:30:06.678 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-26T01:30:06.678 INFO:teuthology.orchestra.run.smithi170.stdout:[osd] 2023-10-26T01:30:06.678 INFO:teuthology.orchestra.run.smithi170.stdout: osd journal size = 100 2023-10-26T01:30:06.678 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-26T01:30:06.678 INFO:teuthology.orchestra.run.smithi170.stdout: osd scrub load threshold = 5.0 2023-10-26T01:30:06.678 INFO:teuthology.orchestra.run.smithi170.stdout: osd scrub max interval = 600 2023-10-26T01:30:06.679 INFO:teuthology.orchestra.run.smithi170.stdout: osd mclock profile = high_recovery_ops 2023-10-26T01:30:06.679 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-26T01:30:06.679 INFO:teuthology.orchestra.run.smithi170.stdout: osd recover clone overlap = true 2023-10-26T01:30:06.679 INFO:teuthology.orchestra.run.smithi170.stdout: osd recovery max chunk = 1048576 2023-10-26T01:30:06.679 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-26T01:30:06.680 INFO:teuthology.orchestra.run.smithi170.stdout: osd debug shutdown = true 2023-10-26T01:30:06.680 INFO:teuthology.orchestra.run.smithi170.stdout: osd debug op order = true 2023-10-26T01:30:06.680 INFO:teuthology.orchestra.run.smithi170.stdout: osd debug verify stray on activate = true 2023-10-26T01:30:06.680 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-26T01:30:06.680 INFO:teuthology.orchestra.run.smithi170.stdout: osd open classes on start = true 2023-10-26T01:30:06.680 INFO:teuthology.orchestra.run.smithi170.stdout: osd debug pg log writeout = true 2023-10-26T01:30:06.681 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-26T01:30:06.681 INFO:teuthology.orchestra.run.smithi170.stdout: osd deep scrub update digest min age = 30 2023-10-26T01:30:06.681 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-26T01:30:06.681 INFO:teuthology.orchestra.run.smithi170.stdout: osd map max advance = 10 2023-10-26T01:30:06.681 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-26T01:30:06.682 INFO:teuthology.orchestra.run.smithi170.stdout: journal zero on create = true 2023-10-26T01:30:06.682 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-26T01:30:06.682 INFO:teuthology.orchestra.run.smithi170.stdout: filestore ondisk finisher threads = 3 2023-10-26T01:30:06.682 INFO:teuthology.orchestra.run.smithi170.stdout: filestore apply finisher threads = 3 2023-10-26T01:30:06.682 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-26T01:30:06.683 INFO:teuthology.orchestra.run.smithi170.stdout: bdev debug aio = true 2023-10-26T01:30:06.683 INFO:teuthology.orchestra.run.smithi170.stdout: osd debug misdirected ops = true 2023-10-26T01:30:06.683 INFO:teuthology.orchestra.run.smithi170.stdout: bdev async discard = True 2023-10-26T01:30:06.683 INFO:teuthology.orchestra.run.smithi170.stdout: bdev enable discard = True 2023-10-26T01:30:06.683 INFO:teuthology.orchestra.run.smithi170.stdout: bluestore allocator = bitmap 2023-10-26T01:30:06.684 INFO:teuthology.orchestra.run.smithi170.stdout: bluestore block size = 96636764160 2023-10-26T01:30:06.684 INFO:teuthology.orchestra.run.smithi170.stdout: bluestore fsck on mount = True 2023-10-26T01:30:06.684 INFO:teuthology.orchestra.run.smithi170.stdout: debug bluefs = 1/20 2023-10-26T01:30:06.684 INFO:teuthology.orchestra.run.smithi170.stdout: debug bluestore = 1/20 2023-10-26T01:30:06.684 INFO:teuthology.orchestra.run.smithi170.stdout: debug ms = 1 2023-10-26T01:30:06.684 INFO:teuthology.orchestra.run.smithi170.stdout: debug osd = 20 2023-10-26T01:30:06.685 INFO:teuthology.orchestra.run.smithi170.stdout: debug rocksdb = 4/10 2023-10-26T01:30:06.685 INFO:teuthology.orchestra.run.smithi170.stdout: memstore device bytes = 200000000 2023-10-26T01:30:06.685 INFO:teuthology.orchestra.run.smithi170.stdout: mon osd backfillfull_ratio = 0.85 2023-10-26T01:30:06.685 INFO:teuthology.orchestra.run.smithi170.stdout: mon osd full ratio = 0.9 2023-10-26T01:30:06.685 INFO:teuthology.orchestra.run.smithi170.stdout: mon osd nearfull ratio = 0.8 2023-10-26T01:30:06.686 INFO:teuthology.orchestra.run.smithi170.stdout: osd failsafe full ratio = 1.0 2023-10-26T01:30:06.686 INFO:teuthology.orchestra.run.smithi170.stdout: osd mon report interval = 5 2023-10-26T01:30:06.686 INFO:teuthology.orchestra.run.smithi170.stdout: osd objectstore = memstore 2023-10-26T01:30:06.686 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-26T01:30:06.686 INFO:teuthology.orchestra.run.smithi170.stdout:[mgr] 2023-10-26T01:30:06.687 INFO:teuthology.orchestra.run.smithi170.stdout: debug ms = 1 2023-10-26T01:30:06.687 INFO:teuthology.orchestra.run.smithi170.stdout: debug mgr = 20 2023-10-26T01:30:06.687 INFO:teuthology.orchestra.run.smithi170.stdout: debug mon = 20 2023-10-26T01:30:06.687 INFO:teuthology.orchestra.run.smithi170.stdout: debug auth = 20 2023-10-26T01:30:06.687 INFO:teuthology.orchestra.run.smithi170.stdout: mon reweight min pgs per osd = 4 2023-10-26T01:30:06.688 INFO:teuthology.orchestra.run.smithi170.stdout: mon reweight min bytes per osd = 10 2023-10-26T01:30:06.688 INFO:teuthology.orchestra.run.smithi170.stdout: mgr/telemetry/nag = false 2023-10-26T01:30:06.688 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-26T01:30:06.688 INFO:teuthology.orchestra.run.smithi170.stdout:[mon] 2023-10-26T01:30:06.688 INFO:teuthology.orchestra.run.smithi170.stdout: debug ms = 1 2023-10-26T01:30:06.689 INFO:teuthology.orchestra.run.smithi170.stdout: debug mon = 20 2023-10-26T01:30:06.689 INFO:teuthology.orchestra.run.smithi170.stdout: debug paxos = 20 2023-10-26T01:30:06.689 INFO:teuthology.orchestra.run.smithi170.stdout: debug auth = 20 2023-10-26T01:30:06.689 INFO:teuthology.orchestra.run.smithi170.stdout: mon data avail warn = 5 2023-10-26T01:30:06.689 INFO:teuthology.orchestra.run.smithi170.stdout: mon mgr mkfs grace = 240 2023-10-26T01:30:06.689 INFO:teuthology.orchestra.run.smithi170.stdout: mon reweight min pgs per osd = 4 2023-10-26T01:30:06.690 INFO:teuthology.orchestra.run.smithi170.stdout: mon osd reporter subtree level = osd 2023-10-26T01:30:06.690 INFO:teuthology.orchestra.run.smithi170.stdout: mon osd prime pg temp = true 2023-10-26T01:30:06.690 INFO:teuthology.orchestra.run.smithi170.stdout: mon reweight min bytes per osd = 10 2023-10-26T01:30:06.690 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-26T01:30:06.690 INFO:teuthology.orchestra.run.smithi170.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-26T01:30:06.691 INFO:teuthology.orchestra.run.smithi170.stdout: auth mon ticket ttl = 660 # 11m 2023-10-26T01:30:06.691 INFO:teuthology.orchestra.run.smithi170.stdout: auth service ticket ttl = 240 # 4m 2023-10-26T01:30:06.691 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-26T01:30:06.691 INFO:teuthology.orchestra.run.smithi170.stdout: # don't complain about insecure global_id in the test suite 2023-10-26T01:30:06.691 INFO:teuthology.orchestra.run.smithi170.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-26T01:30:06.692 INFO:teuthology.orchestra.run.smithi170.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-26T01:30:06.692 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-26T01:30:06.692 INFO:teuthology.orchestra.run.smithi170.stdout: # 1m isn't quite enough 2023-10-26T01:30:06.692 INFO:teuthology.orchestra.run.smithi170.stdout: mon_down_mkfs_grace = 2m 2023-10-26T01:30:06.692 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-26T01:30:06.693 INFO:teuthology.orchestra.run.smithi170.stdout: mon_warn_on_filestore_osds = false 2023-10-26T01:30:06.693 INFO:teuthology.orchestra.run.smithi170.stdout: mon osd backfillfull ratio = 0.6 2023-10-26T01:30:06.693 INFO:teuthology.orchestra.run.smithi170.stdout: mon osd full ratio = 0.7 2023-10-26T01:30:06.693 INFO:teuthology.orchestra.run.smithi170.stdout: mon osd nearfull ratio = 0.6 2023-10-26T01:30:06.693 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-26T01:30:06.693 INFO:teuthology.orchestra.run.smithi170.stdout:[client] 2023-10-26T01:30:06.694 INFO:teuthology.orchestra.run.smithi170.stdout: rgw cache enabled = true 2023-10-26T01:30:06.694 INFO:teuthology.orchestra.run.smithi170.stdout: rgw enable ops log = true 2023-10-26T01:30:06.694 INFO:teuthology.orchestra.run.smithi170.stdout: rgw enable usage log = true 2023-10-26T01:30:06.694 INFO:teuthology.orchestra.run.smithi170.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-26T01:30:06.694 INFO:teuthology.orchestra.run.smithi170.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-26T01:30:06.695 INFO:teuthology.orchestra.run.smithi170.stdout: pid file = "" 2023-10-26T01:30:06.695 INFO:teuthology.orchestra.run.smithi170.stdout:[mon.a] 2023-10-26T01:30:06.695 INFO:teuthology.orchestra.run.smithi170.stdout:[mon.b] 2023-10-26T01:30:06.695 INFO:teuthology.orchestra.run.smithi170.stdout:[mon.c] 2023-10-26T01:30:06.695 INFO:teuthology.orchestra.run.smithi170.stdout:[mds.a] 2023-10-26T01:30:06.695 INFO:teuthology.orchestra.run.smithi170.stdout:[mds.c] 2023-10-26T01:30:06.696 INFO:teuthology.orchestra.run.smithi170.stdout:[mds.b] 2023-10-26T01:30:06.696 INFO:teuthology.orchestra.run.smithi170.stdout:[mds.d] 2023-10-26T01:30:06.696 INFO:teuthology.orchestra.run.smithi170.stdout:[client.0] 2023-10-26T01:30:06.696 INFO:teuthology.orchestra.run.smithi170.stdout: debug client = 20 2023-10-26T01:30:06.696 INFO:teuthology.orchestra.run.smithi170.stdout: debug objectcacher = 20 2023-10-26T01:30:06.697 INFO:teuthology.orchestra.run.smithi170.stdout: debug objecter = 20 2023-10-26T01:30:06.697 INFO:teuthology.orchestra.run.smithi170.stdout:[client.1] 2023-10-26T01:30:06.697 INFO:teuthology.orchestra.run.smithi170.stdout: debug client = 20 2023-10-26T01:30:06.697 INFO:teuthology.orchestra.run.smithi170.stdout: debug objectcacher = 20 2023-10-26T01:30:06.697 INFO:teuthology.orchestra.run.smithi170.stdout: debug objecter = 20 2023-10-26T01:30:06.702 INFO:tasks.ceph:Creating admin key on mon.a... 2023-10-26T01:30:06.703 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-10-26T01:30:06.853 INFO:tasks.ceph:Copying monmap to all nodes... 2023-10-26T01:30:06.854 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:30:06.854 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-10-26T01:30:06.906 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:30:06.907 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-10-26T01:30:07.005 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi169.front.sepia.ceph.com 2023-10-26T01:30:07.006 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:30:07.006 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/etc/ceph/ceph.keyring 2023-10-26T01:30:07.006 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-26T01:30:07.155 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:30:07.155 DEBUG:teuthology.orchestra.run.smithi169:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-10-26T01:30:07.248 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi170.front.sepia.ceph.com 2023-10-26T01:30:07.249 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-26T01:30:07.249 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/etc/ceph/ceph.keyring 2023-10-26T01:30:07.249 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-26T01:30:07.406 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-26T01:30:07.406 DEBUG:teuthology.orchestra.run.smithi170:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-10-26T01:30:07.507 INFO:tasks.ceph:Setting up mon nodes... 2023-10-26T01:30:07.507 INFO:tasks.ceph:Setting up mgr nodes... 2023-10-26T01:30:07.508 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-10-26T01:30:07.663 INFO:teuthology.orchestra.run.smithi169.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-10-26T01:30:07.670 DEBUG:teuthology.orchestra.run.smithi170:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-10-26T01:30:07.869 INFO:teuthology.orchestra.run.smithi170.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-10-26T01:30:07.877 INFO:tasks.ceph:Setting up mds nodes... 2023-10-26T01:30:07.877 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-10-26T01:30:08.036 INFO:teuthology.orchestra.run.smithi169.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-10-26T01:30:08.042 DEBUG:teuthology.orchestra.run.smithi169:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-10-26T01:30:08.123 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-10-26T01:30:08.280 INFO:teuthology.orchestra.run.smithi169.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-10-26T01:30:08.286 DEBUG:teuthology.orchestra.run.smithi169:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-10-26T01:30:08.373 DEBUG:teuthology.orchestra.run.smithi170:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-10-26T01:30:08.523 INFO:teuthology.orchestra.run.smithi170.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-10-26T01:30:08.531 DEBUG:teuthology.orchestra.run.smithi170:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-10-26T01:30:08.615 DEBUG:teuthology.orchestra.run.smithi170:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-10-26T01:30:08.777 INFO:teuthology.orchestra.run.smithi170.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-10-26T01:30:08.783 DEBUG:teuthology.orchestra.run.smithi170:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-10-26T01:30:08.859 INFO:tasks.ceph_client:Setting up client nodes... 2023-10-26T01:30:08.860 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-10-26T01:30:08.988 INFO:teuthology.orchestra.run.smithi169.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-10-26T01:30:09.048 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-10-26T01:30:09.171 INFO:teuthology.orchestra.run.smithi169.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-10-26T01:30:09.217 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-10-26T01:30:09.335 INFO:teuthology.orchestra.run.smithi170.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-10-26T01:30:09.379 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-10-26T01:30:09.491 INFO:teuthology.orchestra.run.smithi170.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-10-26T01:30:09.541 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-10-26T01:30:09.542 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi169.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi170.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-10-26T01:30:09.542 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-10-26T01:30:09.647 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-26T01:30:09.647 INFO:tasks.ceph:role: osd.0 2023-10-26T01:30:09.648 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi169.front.sepia.ceph.com 2023-10-26T01:30:09.648 DEBUG:teuthology.orchestra.run.smithi169:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-10-26T01:30:09.751 INFO:teuthology.orchestra.run.smithi169.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-10-26T01:30:09.752 INFO:teuthology.orchestra.run.smithi169.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-26T01:30:09.752 INFO:teuthology.orchestra.run.smithi169.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-26T01:30:09.752 INFO:teuthology.orchestra.run.smithi169.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-10-26T01:30:09.752 INFO:teuthology.orchestra.run.smithi169.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-26T01:30:09.752 INFO:teuthology.orchestra.run.smithi169.stdout: = sunit=0 swidth=0 blks 2023-10-26T01:30:09.753 INFO:teuthology.orchestra.run.smithi169.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-26T01:30:09.753 INFO:teuthology.orchestra.run.smithi169.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-10-26T01:30:09.753 INFO:teuthology.orchestra.run.smithi169.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-26T01:30:09.753 INFO:teuthology.orchestra.run.smithi169.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-26T01:30:10.279 INFO:teuthology.orchestra.run.smithi169.stdout:Discarding blocks...Done. 2023-10-26T01:30:10.285 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi169.front.sepia.ceph.com -o noatime 2023-10-26T01:30:10.286 DEBUG:teuthology.orchestra.run.smithi169:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-10-26T01:30:10.471 DEBUG:teuthology.orchestra.run.smithi169:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-10-26T01:30:10.576 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-10-26T01:30:10.659 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-26T01:30:10.660 INFO:tasks.ceph:role: osd.1 2023-10-26T01:30:10.660 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi169.front.sepia.ceph.com 2023-10-26T01:30:10.660 DEBUG:teuthology.orchestra.run.smithi169:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-10-26T01:30:10.752 INFO:teuthology.orchestra.run.smithi169.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-10-26T01:30:10.753 INFO:teuthology.orchestra.run.smithi169.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-26T01:30:10.753 INFO:teuthology.orchestra.run.smithi169.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-26T01:30:10.753 INFO:teuthology.orchestra.run.smithi169.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-10-26T01:30:10.753 INFO:teuthology.orchestra.run.smithi169.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-26T01:30:10.754 INFO:teuthology.orchestra.run.smithi169.stdout: = sunit=0 swidth=0 blks 2023-10-26T01:30:10.754 INFO:teuthology.orchestra.run.smithi169.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-26T01:30:10.754 INFO:teuthology.orchestra.run.smithi169.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-10-26T01:30:10.754 INFO:teuthology.orchestra.run.smithi169.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-26T01:30:10.754 INFO:teuthology.orchestra.run.smithi169.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-26T01:30:11.227 INFO:teuthology.orchestra.run.smithi169.stdout:Discarding blocks...Done. 2023-10-26T01:30:11.233 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi169.front.sepia.ceph.com -o noatime 2023-10-26T01:30:11.233 DEBUG:teuthology.orchestra.run.smithi169:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-10-26T01:30:11.376 DEBUG:teuthology.orchestra.run.smithi169:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-10-26T01:30:11.470 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-10-26T01:30:11.565 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-26T01:30:11.565 INFO:tasks.ceph:role: osd.2 2023-10-26T01:30:11.565 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi169.front.sepia.ceph.com 2023-10-26T01:30:11.566 DEBUG:teuthology.orchestra.run.smithi169:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-10-26T01:30:11.667 INFO:teuthology.orchestra.run.smithi169.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-10-26T01:30:11.668 INFO:teuthology.orchestra.run.smithi169.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-26T01:30:11.668 INFO:teuthology.orchestra.run.smithi169.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-26T01:30:11.668 INFO:teuthology.orchestra.run.smithi169.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-10-26T01:30:11.669 INFO:teuthology.orchestra.run.smithi169.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-26T01:30:11.669 INFO:teuthology.orchestra.run.smithi169.stdout: = sunit=0 swidth=0 blks 2023-10-26T01:30:11.669 INFO:teuthology.orchestra.run.smithi169.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-26T01:30:11.669 INFO:teuthology.orchestra.run.smithi169.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-10-26T01:30:11.669 INFO:teuthology.orchestra.run.smithi169.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-26T01:30:11.670 INFO:teuthology.orchestra.run.smithi169.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-26T01:30:12.195 INFO:teuthology.orchestra.run.smithi169.stdout:Discarding blocks...Done. 2023-10-26T01:30:12.201 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi169.front.sepia.ceph.com -o noatime 2023-10-26T01:30:12.201 DEBUG:teuthology.orchestra.run.smithi169:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-10-26T01:30:12.360 DEBUG:teuthology.orchestra.run.smithi169:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-10-26T01:30:12.455 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-10-26T01:30:12.546 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-26T01:30:12.546 INFO:tasks.ceph:role: osd.3 2023-10-26T01:30:12.547 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi169.front.sepia.ceph.com 2023-10-26T01:30:12.547 DEBUG:teuthology.orchestra.run.smithi169:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-10-26T01:30:12.647 INFO:teuthology.orchestra.run.smithi169.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-10-26T01:30:12.648 INFO:teuthology.orchestra.run.smithi169.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-26T01:30:12.648 INFO:teuthology.orchestra.run.smithi169.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-26T01:30:12.648 INFO:teuthology.orchestra.run.smithi169.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-10-26T01:30:12.648 INFO:teuthology.orchestra.run.smithi169.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-26T01:30:12.648 INFO:teuthology.orchestra.run.smithi169.stdout: = sunit=0 swidth=0 blks 2023-10-26T01:30:12.649 INFO:teuthology.orchestra.run.smithi169.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-26T01:30:12.649 INFO:teuthology.orchestra.run.smithi169.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-10-26T01:30:12.649 INFO:teuthology.orchestra.run.smithi169.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-26T01:30:12.649 INFO:teuthology.orchestra.run.smithi169.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-26T01:30:13.104 INFO:teuthology.orchestra.run.smithi169.stdout:Discarding blocks...Done. 2023-10-26T01:30:13.110 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi169.front.sepia.ceph.com -o noatime 2023-10-26T01:30:13.110 DEBUG:teuthology.orchestra.run.smithi169:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-10-26T01:30:13.251 DEBUG:teuthology.orchestra.run.smithi169:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-10-26T01:30:13.354 DEBUG:teuthology.orchestra.run.smithi169:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-26T01:30:13.495 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:13.494+0000 7f16208a7500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-10-26T01:30:13.496 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:13.494+0000 7f16208a7500 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-10-26T01:30:13.505 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:13.504+0000 7f16208a7500 -1 memstore(/var/lib/ceph/osd/ceph-0) /var/lib/ceph/osd/ceph-0 2023-10-26T01:30:13.530 DEBUG:teuthology.orchestra.run.smithi169:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-10-26T01:30:13.617 DEBUG:teuthology.orchestra.run.smithi169:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-26T01:30:13.757 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:13.755+0000 7f1326f4a500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-10-26T01:30:13.759 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:13.758+0000 7f1326f4a500 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-10-26T01:30:13.768 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:13.767+0000 7f1326f4a500 -1 memstore(/var/lib/ceph/osd/ceph-1) /var/lib/ceph/osd/ceph-1 2023-10-26T01:30:13.805 DEBUG:teuthology.orchestra.run.smithi169:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-10-26T01:30:13.892 DEBUG:teuthology.orchestra.run.smithi169:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-26T01:30:14.031 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:14.029+0000 7f39e4174500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-10-26T01:30:14.037 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:14.036+0000 7f39e4174500 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-10-26T01:30:14.045 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:14.044+0000 7f39e4174500 -1 memstore(/var/lib/ceph/osd/ceph-2) /var/lib/ceph/osd/ceph-2 2023-10-26T01:30:14.081 DEBUG:teuthology.orchestra.run.smithi169:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-10-26T01:30:14.170 DEBUG:teuthology.orchestra.run.smithi169:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-26T01:30:14.304 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:14.303+0000 7fc9d7146500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-10-26T01:30:14.305 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:14.303+0000 7fc9d7146500 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-10-26T01:30:14.307 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:14.306+0000 7fc9d7146500 -1 memstore(/var/lib/ceph/osd/ceph-3) /var/lib/ceph/osd/ceph-3 2023-10-26T01:30:14.323 DEBUG:teuthology.orchestra.run.smithi169:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-10-26T01:30:14.412 DEBUG:teuthology.orchestra.run.smithi170:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-10-26T01:30:14.513 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-26T01:30:14.514 INFO:tasks.ceph:role: osd.4 2023-10-26T01:30:14.514 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi170.front.sepia.ceph.com 2023-10-26T01:30:14.514 DEBUG:teuthology.orchestra.run.smithi170:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-10-26T01:30:14.609 INFO:teuthology.orchestra.run.smithi170.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-10-26T01:30:14.610 INFO:teuthology.orchestra.run.smithi170.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-26T01:30:14.610 INFO:teuthology.orchestra.run.smithi170.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-26T01:30:14.610 INFO:teuthology.orchestra.run.smithi170.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-10-26T01:30:14.610 INFO:teuthology.orchestra.run.smithi170.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-26T01:30:14.611 INFO:teuthology.orchestra.run.smithi170.stdout: = sunit=0 swidth=0 blks 2023-10-26T01:30:14.611 INFO:teuthology.orchestra.run.smithi170.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-26T01:30:14.611 INFO:teuthology.orchestra.run.smithi170.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-10-26T01:30:14.611 INFO:teuthology.orchestra.run.smithi170.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-26T01:30:14.611 INFO:teuthology.orchestra.run.smithi170.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-26T01:30:14.644 INFO:teuthology.orchestra.run.smithi170.stdout:Discarding blocks...Done. 2023-10-26T01:30:14.650 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi170.front.sepia.ceph.com -o noatime 2023-10-26T01:30:14.651 DEBUG:teuthology.orchestra.run.smithi170:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-10-26T01:30:14.760 DEBUG:teuthology.orchestra.run.smithi170:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-10-26T01:30:14.842 DEBUG:teuthology.orchestra.run.smithi170:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-10-26T01:30:14.935 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-26T01:30:14.936 INFO:tasks.ceph:role: osd.5 2023-10-26T01:30:14.936 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi170.front.sepia.ceph.com 2023-10-26T01:30:14.937 DEBUG:teuthology.orchestra.run.smithi170:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-10-26T01:30:15.029 INFO:teuthology.orchestra.run.smithi170.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-10-26T01:30:15.030 INFO:teuthology.orchestra.run.smithi170.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-26T01:30:15.030 INFO:teuthology.orchestra.run.smithi170.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-26T01:30:15.030 INFO:teuthology.orchestra.run.smithi170.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-10-26T01:30:15.031 INFO:teuthology.orchestra.run.smithi170.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-26T01:30:15.031 INFO:teuthology.orchestra.run.smithi170.stdout: = sunit=0 swidth=0 blks 2023-10-26T01:30:15.031 INFO:teuthology.orchestra.run.smithi170.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-26T01:30:15.032 INFO:teuthology.orchestra.run.smithi170.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-10-26T01:30:15.032 INFO:teuthology.orchestra.run.smithi170.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-26T01:30:15.032 INFO:teuthology.orchestra.run.smithi170.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-26T01:30:15.064 INFO:teuthology.orchestra.run.smithi170.stdout:Discarding blocks...Done. 2023-10-26T01:30:15.070 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi170.front.sepia.ceph.com -o noatime 2023-10-26T01:30:15.070 DEBUG:teuthology.orchestra.run.smithi170:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-10-26T01:30:15.180 DEBUG:teuthology.orchestra.run.smithi170:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-10-26T01:30:15.267 DEBUG:teuthology.orchestra.run.smithi170:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-10-26T01:30:15.351 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-26T01:30:15.351 INFO:tasks.ceph:role: osd.6 2023-10-26T01:30:15.352 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi170.front.sepia.ceph.com 2023-10-26T01:30:15.352 DEBUG:teuthology.orchestra.run.smithi170:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-10-26T01:30:15.438 INFO:teuthology.orchestra.run.smithi170.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-10-26T01:30:15.439 INFO:teuthology.orchestra.run.smithi170.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-26T01:30:15.439 INFO:teuthology.orchestra.run.smithi170.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-26T01:30:15.439 INFO:teuthology.orchestra.run.smithi170.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-10-26T01:30:15.439 INFO:teuthology.orchestra.run.smithi170.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-26T01:30:15.440 INFO:teuthology.orchestra.run.smithi170.stdout: = sunit=0 swidth=0 blks 2023-10-26T01:30:15.440 INFO:teuthology.orchestra.run.smithi170.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-26T01:30:15.440 INFO:teuthology.orchestra.run.smithi170.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-10-26T01:30:15.440 INFO:teuthology.orchestra.run.smithi170.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-26T01:30:15.440 INFO:teuthology.orchestra.run.smithi170.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-26T01:30:15.473 INFO:teuthology.orchestra.run.smithi170.stdout:Discarding blocks...Done. 2023-10-26T01:30:15.477 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi170.front.sepia.ceph.com -o noatime 2023-10-26T01:30:15.478 DEBUG:teuthology.orchestra.run.smithi170:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-10-26T01:30:15.591 DEBUG:teuthology.orchestra.run.smithi170:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-10-26T01:30:15.682 DEBUG:teuthology.orchestra.run.smithi170:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-10-26T01:30:15.770 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-26T01:30:15.770 INFO:tasks.ceph:role: osd.7 2023-10-26T01:30:15.771 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi170.front.sepia.ceph.com 2023-10-26T01:30:15.771 DEBUG:teuthology.orchestra.run.smithi170:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-10-26T01:30:15.866 INFO:teuthology.orchestra.run.smithi170.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-10-26T01:30:15.866 INFO:teuthology.orchestra.run.smithi170.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-26T01:30:15.867 INFO:teuthology.orchestra.run.smithi170.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-26T01:30:15.867 INFO:teuthology.orchestra.run.smithi170.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-10-26T01:30:15.867 INFO:teuthology.orchestra.run.smithi170.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-26T01:30:15.867 INFO:teuthology.orchestra.run.smithi170.stdout: = sunit=0 swidth=0 blks 2023-10-26T01:30:15.867 INFO:teuthology.orchestra.run.smithi170.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-26T01:30:15.868 INFO:teuthology.orchestra.run.smithi170.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-10-26T01:30:15.868 INFO:teuthology.orchestra.run.smithi170.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-26T01:30:15.868 INFO:teuthology.orchestra.run.smithi170.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-26T01:30:15.900 INFO:teuthology.orchestra.run.smithi170.stdout:Discarding blocks...Done. 2023-10-26T01:30:15.907 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi170.front.sepia.ceph.com -o noatime 2023-10-26T01:30:15.908 DEBUG:teuthology.orchestra.run.smithi170:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-10-26T01:30:16.018 DEBUG:teuthology.orchestra.run.smithi170:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-10-26T01:30:16.096 DEBUG:teuthology.orchestra.run.smithi170:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-26T01:30:16.243 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:16.242+0000 7f1c89992500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-10-26T01:30:16.244 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:16.242+0000 7f1c89992500 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-10-26T01:30:16.246 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:16.245+0000 7f1c89992500 -1 memstore(/var/lib/ceph/osd/ceph-4) /var/lib/ceph/osd/ceph-4 2023-10-26T01:30:16.265 DEBUG:teuthology.orchestra.run.smithi170:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-10-26T01:30:16.354 DEBUG:teuthology.orchestra.run.smithi170:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-26T01:30:16.494 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:16.492+0000 7f3f6aa1d500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-10-26T01:30:16.494 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:16.492+0000 7f3f6aa1d500 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-10-26T01:30:16.495 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:16.494+0000 7f3f6aa1d500 -1 memstore(/var/lib/ceph/osd/ceph-5) /var/lib/ceph/osd/ceph-5 2023-10-26T01:30:16.513 DEBUG:teuthology.orchestra.run.smithi170:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-10-26T01:30:16.597 DEBUG:teuthology.orchestra.run.smithi170:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-26T01:30:16.738 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:16.737+0000 7f4a7c214500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-10-26T01:30:16.739 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:16.737+0000 7f4a7c214500 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-10-26T01:30:16.740 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:16.739+0000 7f4a7c214500 -1 memstore(/var/lib/ceph/osd/ceph-6) /var/lib/ceph/osd/ceph-6 2023-10-26T01:30:16.758 DEBUG:teuthology.orchestra.run.smithi170:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-10-26T01:30:16.848 DEBUG:teuthology.orchestra.run.smithi170:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-26T01:30:16.986 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:16.985+0000 7f52e4e16500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-10-26T01:30:16.987 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:16.985+0000 7f52e4e16500 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-10-26T01:30:16.988 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:16.987+0000 7f52e4e16500 -1 memstore(/var/lib/ceph/osd/ceph-7) /var/lib/ceph/osd/ceph-7 2023-10-26T01:30:17.005 DEBUG:teuthology.orchestra.run.smithi170:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-10-26T01:30:17.096 INFO:tasks.ceph:Reading keys from all nodes... 2023-10-26T01:30:17.096 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:30:17.097 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-10-26T01:30:17.202 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:30:17.202 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-10-26T01:30:17.300 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:30:17.301 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-10-26T01:30:17.402 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:30:17.402 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-10-26T01:30:17.493 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:30:17.493 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-10-26T01:30:17.589 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:30:17.589 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-10-26T01:30:17.688 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:30:17.688 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-10-26T01:30:17.775 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-26T01:30:17.776 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-10-26T01:30:17.880 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-26T01:30:17.880 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-10-26T01:30:17.981 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-26T01:30:17.981 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-10-26T01:30:18.080 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-26T01:30:18.081 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-10-26T01:30:18.175 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-26T01:30:18.175 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-10-26T01:30:18.268 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-26T01:30:18.268 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-10-26T01:30:18.363 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-26T01:30:18.363 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-10-26T01:30:18.458 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:30:18.459 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-10-26T01:30:18.519 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:30:18.520 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-10-26T01:30:18.574 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-26T01:30:18.575 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-10-26T01:30:18.636 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-26T01:30:18.637 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-10-26T01:30:18.695 INFO:tasks.ceph:Adding keys to all mons... 2023-10-26T01:30:18.695 DEBUG:teuthology.orchestra.run.smithi169:> sudo tee -a /etc/ceph/ceph.keyring 2023-10-26T01:30:18.704 DEBUG:teuthology.orchestra.run.smithi170:> sudo tee -a /etc/ceph/ceph.keyring 2023-10-26T01:30:18.787 INFO:teuthology.orchestra.run.smithi169.stdout:[mgr.y] 2023-10-26T01:30:18.787 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQAfwTllZc+DJxAAsZ2TWn8H7Kw/lNTc2qyKzA== 2023-10-26T01:30:18.787 INFO:teuthology.orchestra.run.smithi169.stdout:[mds.a] 2023-10-26T01:30:18.788 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQAgwTllWtUfAhAA28+12VwWEF/BUEu+ErSVyA== 2023-10-26T01:30:18.788 INFO:teuthology.orchestra.run.smithi169.stdout:[mds.c] 2023-10-26T01:30:18.788 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQAgwTllaCCqEBAAokeHvwOyp/Jv2x0yHDwysw== 2023-10-26T01:30:18.788 INFO:teuthology.orchestra.run.smithi169.stdout:[osd.0] 2023-10-26T01:30:18.788 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQAlwTll8RZ+HRAAt/sBK+Mi91YmTyQYbhKHLg== 2023-10-26T01:30:18.789 INFO:teuthology.orchestra.run.smithi169.stdout:[osd.1] 2023-10-26T01:30:18.789 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQAlwTllLXwWLRAAuHB2lJ+oDJou/IJBtsYY4Q== 2023-10-26T01:30:18.789 INFO:teuthology.orchestra.run.smithi169.stdout:[osd.2] 2023-10-26T01:30:18.789 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQAmwTllj1nSARAA5Jl34Q0X7a71ySd4wz17lw== 2023-10-26T01:30:18.789 INFO:teuthology.orchestra.run.smithi169.stdout:[osd.3] 2023-10-26T01:30:18.790 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQAmwTllC8MeEhAAfGxRVSwD+YbU/U5aeenO3w== 2023-10-26T01:30:18.790 INFO:teuthology.orchestra.run.smithi169.stdout:[mgr.x] 2023-10-26T01:30:18.790 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQAfwTllPu3FMxAAV8d5oRFXAeeZH7e1xzPEDg== 2023-10-26T01:30:18.790 INFO:teuthology.orchestra.run.smithi169.stdout:[mds.b] 2023-10-26T01:30:18.790 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQAgwTlleA8lHxAA+jMcMo+QnYAGllpKJ6eYgQ== 2023-10-26T01:30:18.790 INFO:teuthology.orchestra.run.smithi169.stdout:[mds.d] 2023-10-26T01:30:18.791 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQAgwTll51JQLhAA4Wzso4zQj+Htv6cXqvwRmg== 2023-10-26T01:30:18.791 INFO:teuthology.orchestra.run.smithi169.stdout:[osd.4] 2023-10-26T01:30:18.791 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQAowTll4Md6DhAAuXfadUNAdouMP28MiX/D3Q== 2023-10-26T01:30:18.791 INFO:teuthology.orchestra.run.smithi169.stdout:[osd.5] 2023-10-26T01:30:18.791 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQAowTllDdViHRAAec9QyhZCsjJ2rgGhhcjwpQ== 2023-10-26T01:30:18.792 INFO:teuthology.orchestra.run.smithi169.stdout:[osd.6] 2023-10-26T01:30:18.792 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQAowTll23X5KxAAsmQvG0yq7ilkVG5fNiaCUg== 2023-10-26T01:30:18.792 INFO:teuthology.orchestra.run.smithi169.stdout:[osd.7] 2023-10-26T01:30:18.792 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQAowTllhcy/OhAAY5Yjheh76ibOVREqN6WBpQ== 2023-10-26T01:30:18.792 INFO:teuthology.orchestra.run.smithi169.stdout:[client.0] 2023-10-26T01:30:18.792 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQAgwTllKC7gOhAAPsvxGcuUA8/p8xqbUgGteA== 2023-10-26T01:30:18.793 INFO:teuthology.orchestra.run.smithi169.stdout:[client.2] 2023-10-26T01:30:18.793 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQAhwTllLWMoChAAsUHzBc29GDqmEQAFnl7dHg== 2023-10-26T01:30:18.793 INFO:teuthology.orchestra.run.smithi169.stdout:[client.1] 2023-10-26T01:30:18.793 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQAhwTllhxb6ExAAeYTsurWpnjohWlrNIlLpsw== 2023-10-26T01:30:18.793 INFO:teuthology.orchestra.run.smithi169.stdout:[client.3] 2023-10-26T01:30:18.794 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQAhwTll875HHRAAGbwikoR/hDU8P/bj0OPWAw== 2023-10-26T01:30:18.795 INFO:teuthology.orchestra.run.smithi170.stdout:[mgr.y] 2023-10-26T01:30:18.795 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQAfwTllZc+DJxAAsZ2TWn8H7Kw/lNTc2qyKzA== 2023-10-26T01:30:18.795 INFO:teuthology.orchestra.run.smithi170.stdout:[mds.a] 2023-10-26T01:30:18.795 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQAgwTllWtUfAhAA28+12VwWEF/BUEu+ErSVyA== 2023-10-26T01:30:18.795 INFO:teuthology.orchestra.run.smithi170.stdout:[mds.c] 2023-10-26T01:30:18.796 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQAgwTllaCCqEBAAokeHvwOyp/Jv2x0yHDwysw== 2023-10-26T01:30:18.796 INFO:teuthology.orchestra.run.smithi170.stdout:[osd.0] 2023-10-26T01:30:18.796 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQAlwTll8RZ+HRAAt/sBK+Mi91YmTyQYbhKHLg== 2023-10-26T01:30:18.796 INFO:teuthology.orchestra.run.smithi170.stdout:[osd.1] 2023-10-26T01:30:18.796 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQAlwTllLXwWLRAAuHB2lJ+oDJou/IJBtsYY4Q== 2023-10-26T01:30:18.797 INFO:teuthology.orchestra.run.smithi170.stdout:[osd.2] 2023-10-26T01:30:18.797 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQAmwTllj1nSARAA5Jl34Q0X7a71ySd4wz17lw== 2023-10-26T01:30:18.797 INFO:teuthology.orchestra.run.smithi170.stdout:[osd.3] 2023-10-26T01:30:18.797 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQAmwTllC8MeEhAAfGxRVSwD+YbU/U5aeenO3w== 2023-10-26T01:30:18.797 INFO:teuthology.orchestra.run.smithi170.stdout:[mgr.x] 2023-10-26T01:30:18.797 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQAfwTllPu3FMxAAV8d5oRFXAeeZH7e1xzPEDg== 2023-10-26T01:30:18.798 INFO:teuthology.orchestra.run.smithi170.stdout:[mds.b] 2023-10-26T01:30:18.798 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQAgwTlleA8lHxAA+jMcMo+QnYAGllpKJ6eYgQ== 2023-10-26T01:30:18.798 INFO:teuthology.orchestra.run.smithi170.stdout:[mds.d] 2023-10-26T01:30:18.798 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQAgwTll51JQLhAA4Wzso4zQj+Htv6cXqvwRmg== 2023-10-26T01:30:18.798 INFO:teuthology.orchestra.run.smithi170.stdout:[osd.4] 2023-10-26T01:30:18.799 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQAowTll4Md6DhAAuXfadUNAdouMP28MiX/D3Q== 2023-10-26T01:30:18.799 INFO:teuthology.orchestra.run.smithi170.stdout:[osd.5] 2023-10-26T01:30:18.799 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQAowTllDdViHRAAec9QyhZCsjJ2rgGhhcjwpQ== 2023-10-26T01:30:18.799 INFO:teuthology.orchestra.run.smithi170.stdout:[osd.6] 2023-10-26T01:30:18.799 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQAowTll23X5KxAAsmQvG0yq7ilkVG5fNiaCUg== 2023-10-26T01:30:18.799 INFO:teuthology.orchestra.run.smithi170.stdout:[osd.7] 2023-10-26T01:30:18.800 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQAowTllhcy/OhAAY5Yjheh76ibOVREqN6WBpQ== 2023-10-26T01:30:18.800 INFO:teuthology.orchestra.run.smithi170.stdout:[client.0] 2023-10-26T01:30:18.800 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQAgwTllKC7gOhAAPsvxGcuUA8/p8xqbUgGteA== 2023-10-26T01:30:18.800 INFO:teuthology.orchestra.run.smithi170.stdout:[client.2] 2023-10-26T01:30:18.800 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQAhwTllLWMoChAAsUHzBc29GDqmEQAFnl7dHg== 2023-10-26T01:30:18.801 INFO:teuthology.orchestra.run.smithi170.stdout:[client.1] 2023-10-26T01:30:18.801 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQAhwTllhxb6ExAAeYTsurWpnjohWlrNIlLpsw== 2023-10-26T01:30:18.801 INFO:teuthology.orchestra.run.smithi170.stdout:[client.3] 2023-10-26T01:30:18.801 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQAhwTll875HHRAAGbwikoR/hDU8P/bj0OPWAw== 2023-10-26T01:30:18.802 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-26T01:30:18.843 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-26T01:30:18.995 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-26T01:30:19.004 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-26T01:30:19.153 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-26T01:30:19.159 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-26T01:30:19.308 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-26T01:30:19.317 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-26T01:30:19.452 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-26T01:30:19.457 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-26T01:30:19.595 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-26T01:30:19.600 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-26T01:30:19.738 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-26T01:30:19.743 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-26T01:30:19.887 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-26T01:30:19.896 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-26T01:30:20.041 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-26T01:30:20.050 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-26T01:30:20.192 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-26T01:30:20.199 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-26T01:30:20.338 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-26T01:30:20.343 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-26T01:30:20.484 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-26T01:30:20.493 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-26T01:30:20.639 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-26T01:30:20.646 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-26T01:30:20.789 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-26T01:30:20.797 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-26T01:30:20.940 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-26T01:30:20.948 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-26T01:30:21.091 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-26T01:30:21.096 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-26T01:30:21.241 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-26T01:30:21.246 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-26T01:30:21.396 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-26T01:30:21.405 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-26T01:30:21.553 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-10-26T01:30:21.554 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-10-26T01:30:21.652 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-26T01:30:21.815 DEBUG:teuthology.orchestra.run.smithi169:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-10-26T01:30:21.903 DEBUG:teuthology.orchestra.run.smithi170:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-10-26T01:30:22.008 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-26T01:30:22.169 DEBUG:teuthology.orchestra.run.smithi170:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-10-26T01:30:22.259 DEBUG:teuthology.orchestra.run.smithi170:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-10-26T01:30:22.354 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-26T01:30:22.509 DEBUG:teuthology.orchestra.run.smithi170:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-10-26T01:30:22.597 DEBUG:teuthology.orchestra.run.smithi169:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-10-26T01:30:22.606 DEBUG:teuthology.orchestra.run.smithi170:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-10-26T01:30:22.665 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-10-26T01:30:22.665 INFO:tasks.ceph.mon.a:Restarting daemon 2023-10-26T01:30:22.666 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-10-26T01:30:22.672 INFO:tasks.ceph.mon.a:Started 2023-10-26T01:30:22.672 INFO:tasks.ceph.mon.b:Restarting daemon 2023-10-26T01:30:22.672 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-10-26T01:30:22.681 INFO:tasks.ceph.mon.b:Started 2023-10-26T01:30:22.681 INFO:tasks.ceph.mon.c:Restarting daemon 2023-10-26T01:30:22.682 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-10-26T01:30:22.690 INFO:tasks.ceph.mon.c:Started 2023-10-26T01:30:22.691 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-10-26T01:30:22.691 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-10-26T01:30:22.691 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-10-26T01:30:22.697 INFO:tasks.ceph.mgr.y:Started 2023-10-26T01:30:22.698 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-10-26T01:30:22.698 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-10-26T01:30:22.703 INFO:tasks.ceph.mgr.x:Started 2023-10-26T01:30:22.704 INFO:tasks.ceph:setting config [client] client_mount_timeout = 600 2023-10-26T01:30:22.704 DEBUG:teuthology.orchestra.run.smithi169:> sudo ceph --cluster ceph config set client client_mount_timeout 600 2023-10-26T01:30:22.709 INFO:tasks.ceph:setting config [client] debug_client = 20 2023-10-26T01:30:22.709 DEBUG:teuthology.orchestra.run.smithi169:> sudo ceph --cluster ceph config set client debug_client 20 2023-10-26T01:30:22.757 INFO:tasks.ceph:setting config [client] debug_ms = 1 2023-10-26T01:30:22.757 DEBUG:teuthology.orchestra.run.smithi169:> sudo ceph --cluster ceph config set client debug_ms 1 2023-10-26T01:30:22.806 INFO:tasks.ceph:setting config [client] rados_mon_op_timeout = 900 2023-10-26T01:30:22.807 DEBUG:teuthology.orchestra.run.smithi169:> sudo ceph --cluster ceph config set client rados_mon_op_timeout 900 2023-10-26T01:30:22.852 INFO:tasks.ceph:setting config [client] rados_osd_op_timeout = 900 2023-10-26T01:30:22.853 DEBUG:teuthology.orchestra.run.smithi169:> sudo ceph --cluster ceph config set client rados_osd_op_timeout 900 2023-10-26T01:30:22.900 INFO:tasks.ceph:setting config [mds] debug_mds = 20 2023-10-26T01:30:22.901 DEBUG:teuthology.orchestra.run.smithi169:> sudo ceph --cluster ceph config set mds debug_mds 20 2023-10-26T01:30:22.948 INFO:tasks.ceph:setting config [mds] debug_mds_balancer = 20 2023-10-26T01:30:22.949 DEBUG:teuthology.orchestra.run.smithi169:> sudo ceph --cluster ceph config set mds debug_mds_balancer 20 2023-10-26T01:30:23.000 INFO:tasks.ceph:setting config [mds] debug_ms = 1 2023-10-26T01:30:23.001 DEBUG:teuthology.orchestra.run.smithi169:> sudo ceph --cluster ceph config set mds debug_ms 1 2023-10-26T01:30:23.049 INFO:tasks.ceph:setting config [mds] mds_debug_frag = True 2023-10-26T01:30:23.050 DEBUG:teuthology.orchestra.run.smithi169:> sudo ceph --cluster ceph config set mds mds_debug_frag True 2023-10-26T01:30:23.098 INFO:tasks.ceph:setting config [mds] mds_debug_scatterstat = True 2023-10-26T01:30:23.099 DEBUG:teuthology.orchestra.run.smithi169:> sudo ceph --cluster ceph config set mds mds_debug_scatterstat True 2023-10-26T01:30:23.151 INFO:tasks.ceph:setting config [mds] mds_op_complaint_time = 180 2023-10-26T01:30:23.152 DEBUG:teuthology.orchestra.run.smithi169:> sudo ceph --cluster ceph config set mds mds_op_complaint_time 180 2023-10-26T01:30:23.169 INFO:tasks.ceph:setting config [mds] mds_verify_scatter = True 2023-10-26T01:30:23.169 DEBUG:teuthology.orchestra.run.smithi169:> sudo ceph --cluster ceph config set mds mds_verify_scatter True 2023-10-26T01:30:23.223 INFO:tasks.ceph:setting config [mds] osd_op_complaint_time = 180 2023-10-26T01:30:23.224 DEBUG:teuthology.orchestra.run.smithi169:> sudo ceph --cluster ceph config set mds osd_op_complaint_time 180 2023-10-26T01:30:23.264 INFO:tasks.ceph:setting config [mds] rados_mon_op_timeout = 900 2023-10-26T01:30:23.264 DEBUG:teuthology.orchestra.run.smithi169:> sudo ceph --cluster ceph config set mds rados_mon_op_timeout 900 2023-10-26T01:30:23.289 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.287+0000 7fdf1962b640 1 -- 172.21.15.169:0/3815523556 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdf1406c460 msgr2=0x7fdf1406add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:23.289 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.287+0000 7fdf1962b640 1 --2- 172.21.15.169:0/3815523556 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdf1406c460 0x7fdf1406add0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fdf140700b0 tx=0x7fdf0802bdb0 comp rx=0 tx=0).stop 2023-10-26T01:30:23.305 INFO:tasks.ceph:setting config [mds] rados_osd_op_timeout = 900 2023-10-26T01:30:23.306 DEBUG:teuthology.orchestra.run.smithi169:> sudo ceph --cluster ceph config set mds rados_osd_op_timeout 900 2023-10-26T01:30:23.306 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.296+0000 7f7dbc402640 1 -- 172.21.15.169:0/3719995244 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7db40d21d0 msgr2=0x7f7db415af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:23.307 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.296+0000 7f7dbc402640 1 --2- 172.21.15.169:0/3719995244 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7db40d21d0 0x7f7db415af50 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7db000ea10 tx=0x7f7db000eed0 comp rx=0 tx=0).stop 2023-10-26T01:30:23.307 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.296+0000 7f7dbc402640 1 -- 172.21.15.169:0/3719995244 shutdown_connections 2023-10-26T01:30:23.307 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.296+0000 7f7dbc402640 1 --2- 172.21.15.169:0/3719995244 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7db40d21d0 0x7f7db415af50 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:23.308 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.296+0000 7f7dbc402640 1 --2- 172.21.15.169:0/3719995244 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7db40c9a20 0x7f7db40cab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:23.308 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.297+0000 7f7dbc402640 1 --2- 172.21.15.169:0/3719995244 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7db40d1df0 0x7f7db4155e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:23.308 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.297+0000 7f7dbc402640 1 -- 172.21.15.169:0/3719995244 >> 172.21.15.169:0/3719995244 conn(0x7f7db406b5c0 msgr2=0x7f7db415cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:23.308 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.297+0000 7f7dbc402640 1 -- 172.21.15.169:0/3719995244 shutdown_connections 2023-10-26T01:30:23.308 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.297+0000 7f7dbc402640 1 -- 172.21.15.169:0/3719995244 wait complete. 2023-10-26T01:30:23.309 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.291+0000 7fdf1962b640 1 -- 172.21.15.169:0/3815523556 shutdown_connections 2023-10-26T01:30:23.309 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.291+0000 7fdf1962b640 1 --2- 172.21.15.169:0/3815523556 >> v2:172.21.15.169:3300/0 conn(0x7fdf1406b310 0x7fdf1406b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:23.310 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.293+0000 7fdf1962b640 1 --2- 172.21.15.169:0/3815523556 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdf1406c460 0x7fdf1406add0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:23.310 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.294+0000 7fdf1962b640 1 --2- 172.21.15.169:0/3815523556 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdf1406c080 0x7fdf1415c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:23.310 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.294+0000 7fdf1962b640 1 -- 172.21.15.169:0/3815523556 >> 172.21.15.169:0/3815523556 conn(0x7fdf140b2800 msgr2=0x7fdf140b2be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:23.310 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.294+0000 7fdf1962b640 1 -- 172.21.15.169:0/3815523556 shutdown_connections 2023-10-26T01:30:23.311 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.295+0000 7fdf1962b640 1 -- 172.21.15.169:0/3815523556 wait complete. 2023-10-26T01:30:23.311 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.307+0000 7fdf1962b640 1 Processor -- start 2023-10-26T01:30:23.311 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.307+0000 7fdf1962b640 1 -- start start 2023-10-26T01:30:23.318 INFO:tasks.ceph:setting config [mon] mon_op_complaint_time = 120 2023-10-26T01:30:23.319 DEBUG:teuthology.orchestra.run.smithi169:> sudo ceph --cluster ceph config set mon mon_op_complaint_time 120 2023-10-26T01:30:23.320 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.313+0000 7fdf1962b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdf1406b310 0x7fdf140965e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:23.320 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.313+0000 7fdf1962b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdf1406c080 0x7fdf140af4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:23.321 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.313+0000 7fdf1962b640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdf1406c460 0x7fdf140af9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:23.321 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.313+0000 7fdf1962b640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf140aed80 con 0x7fdf1406b310 2023-10-26T01:30:23.321 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.313+0000 7fdf1962b640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf140adce0 con 0x7fdf1406c080 2023-10-26T01:30:23.321 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.313+0000 7fdf1962b640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf140adfc0 con 0x7fdf1406c460 2023-10-26T01:30:23.321 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.316+0000 7fdf127fc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdf1406c080 0x7fdf140af4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:23.322 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.316+0000 7fdf12ffd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdf1406b310 0x7fdf140965e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:23.322 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.316+0000 7fdf137fe640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdf1406c460 0x7fdf140af9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:23.322 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.316+0000 7fdf137fe640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdf1406c460 0x7fdf140af9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3301/0 says I am v2:172.21.15.169:48616/0 (socket says 172.21.15.169:48616) 2023-10-26T01:30:23.322 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.316+0000 7fdf12ffd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdf1406b310 0x7fdf140965e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:55336/0 (socket says 172.21.15.169:55336) 2023-10-26T01:30:23.323 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.316+0000 7fdf137fe640 1 -- 172.21.15.169:0/1732708010 learned_addr learned my addr 172.21.15.169:0/1732708010 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:30:23.323 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.318+0000 7fdf12ffd640 1 -- 172.21.15.169:0/1732708010 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdf1406c460 msgr2=0x7fdf140af9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:23.324 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.318+0000 7fdf12ffd640 1 --2- 172.21.15.169:0/1732708010 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdf1406c460 0x7fdf140af9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:23.324 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.318+0000 7fdf12ffd640 1 -- 172.21.15.169:0/1732708010 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdf1406c080 msgr2=0x7fdf140af4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:23.324 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.318+0000 7fdf12ffd640 1 --2- 172.21.15.169:0/1732708010 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdf1406c080 0x7fdf140af4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:23.325 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.318+0000 7fdf12ffd640 1 -- 172.21.15.169:0/1732708010 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf08009560 con 0x7fdf1406b310 2023-10-26T01:30:23.327 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.319+0000 7fdf12ffd640 1 --2- 172.21.15.169:0/1732708010 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdf1406b310 0x7fdf140965e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fdf0400e9f0 tx=0x7fdf0400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:23.327 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.320+0000 7fdefb7fe640 1 -- 172.21.15.169:0/1732708010 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf0400ccc0 con 0x7fdf1406b310 2023-10-26T01:30:23.327 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.320+0000 7fdefb7fe640 1 -- 172.21.15.169:0/1732708010 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(3 keys) v1 ==== 79+0+0 (secure 0 0 0) 0x7fdf0400ce20 con 0x7fdf1406b310 2023-10-26T01:30:23.328 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.320+0000 7fdefb7fe640 1 -- 172.21.15.169:0/1732708010 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf0400ccc0 con 0x7fdf1406b310 2023-10-26T01:30:23.328 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.320+0000 7fdf1962b640 1 -- 172.21.15.169:0/1732708010 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf140ae2a0 con 0x7fdf1406b310 2023-10-26T01:30:23.329 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.321+0000 7fdf1962b640 1 -- 172.21.15.169:0/1732708010 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf140ae860 con 0x7fdf1406b310 2023-10-26T01:30:23.329 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.323+0000 7fdefb7fe640 1 -- 172.21.15.169:0/1732708010 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fdf04004020 con 0x7fdf1406b310 2023-10-26T01:30:23.329 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.323+0000 7fdefb7fe640 1 -- 172.21.15.169:0/1732708010 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fdf04014070 con 0x7fdf1406b310 2023-10-26T01:30:23.339 INFO:tasks.ceph:setting config [osd] osd_op_complaint_time = 180 2023-10-26T01:30:23.339 DEBUG:teuthology.orchestra.run.smithi169:> sudo ceph --cluster ceph config set osd osd_op_complaint_time 180 2023-10-26T01:30:23.340 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.329+0000 7fdf1962b640 1 -- 172.21.15.169:0/1732708010 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf1404db10 con 0x7fdf1406b310 2023-10-26T01:30:23.340 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.334+0000 7fdefb7fe640 1 -- 172.21.15.169:0/1732708010 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7fdf0401c070 con 0x7fdf1406b310 2023-10-26T01:30:23.369 DEBUG:tasks.ceph:set 17 configs 2023-10-26T01:30:23.370 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'client_mount_timeout', '600'], check_status=True, hostname='smithi169') 2023-10-26T01:30:23.370 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'debug_client', '20'], check_status=True, hostname='smithi169') 2023-10-26T01:30:23.371 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'debug_ms', '1'], check_status=True, hostname='smithi169') 2023-10-26T01:30:23.371 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi169') 2023-10-26T01:30:23.419 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.417+0000 7fa125832640 1 -- 172.21.15.169:0/1985123166 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa1200cd7e0 msgr2=0x7fa1200cdc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:23.420 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.418+0000 7fd66ad16640 1 -- 172.21.15.169:0/3655462347 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd64c003970 msgr2=0x7fd64c003d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:23.420 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.418+0000 7fd66ad16640 1 --2- 172.21.15.169:0/3655462347 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd64c003970 0x7fd64c003d70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd65c00c640 tx=0x7fd64c0057f0 comp rx=0 tx=0).stop 2023-10-26T01:30:23.420 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.418+0000 7fd6692b3640 1 -- 172.21.15.169:0/3655462347 reap_dead start 2023-10-26T01:30:23.422 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.421+0000 7fd66ad16640 1 -- 172.21.15.169:0/3655462347 shutdown_connections 2023-10-26T01:30:23.423 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.421+0000 7fd66ad16640 1 -- 172.21.15.169:0/3655462347 >> 172.21.15.169:0/3655462347 conn(0x7fd6640b2800 msgr2=0x7fd6640b2be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:23.423 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.421+0000 7fa125832640 1 --2- 172.21.15.169:0/1985123166 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa1200cd7e0 0x7fa1200cdc90 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa1200b0520 tx=0x7fa11402bde0 comp rx=0 tx=0).stop 2023-10-26T01:30:23.423 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.422+0000 7fd66ad16640 1 -- 172.21.15.169:0/3655462347 shutdown_connections 2023-10-26T01:30:23.424 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.422+0000 7fd66ad16640 1 -- 172.21.15.169:0/3655462347 wait complete. 2023-10-26T01:30:23.432 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.425+0000 7fd66ad16640 1 Processor -- start 2023-10-26T01:30:23.432 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.428+0000 7fa125832640 1 -- 172.21.15.169:0/1985123166 shutdown_connections 2023-10-26T01:30:23.433 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.428+0000 7fa125832640 1 --2- 172.21.15.169:0/1985123166 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa12015b4a0 0x7fa12015b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:23.433 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.428+0000 7fa125832640 1 --2- 172.21.15.169:0/1985123166 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa1200cd7e0 0x7fa1200cdc90 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:23.433 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.428+0000 7fa125832640 1 --2- 172.21.15.169:0/1985123166 >> v2:172.21.15.169:3300/0 conn(0x7fa12006ce00 0x7fa1201550f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:23.433 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.428+0000 7fa125832640 1 -- 172.21.15.169:0/1985123166 >> 172.21.15.169:0/1985123166 conn(0x7fa120069930 msgr2=0x7fa120069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:23.433 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.428+0000 7fa125832640 1 -- 172.21.15.169:0/1985123166 shutdown_connections 2023-10-26T01:30:23.434 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.428+0000 7fa125832640 1 -- 172.21.15.169:0/1985123166 wait complete. 2023-10-26T01:30:23.435 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.433+0000 7fa125832640 1 Processor -- start 2023-10-26T01:30:23.435 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.434+0000 7fa125832640 1 -- start start 2023-10-26T01:30:23.437 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.436+0000 7fa125832640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa12006ce00 0x7fa1200cafb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:23.438 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.436+0000 7fa125832640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa1200cd7e0 0x7fa120071620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:23.438 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.436+0000 7fa125832640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa12015b4a0 0x7fa12006feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:23.439 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.437+0000 7fd66ad16640 1 -- start start 2023-10-26T01:30:23.439 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.437+0000 7fa125832640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa120095df0 con 0x7fa12015b4a0 2023-10-26T01:30:23.439 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.437+0000 7fa125832640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa12006f700 con 0x7fa12006ce00 2023-10-26T01:30:23.440 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.432+0000 7efcb5b2c640 1 -- 172.21.15.169:0/16766884 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) v1 -- 0x7efc94000e50 con 0x7efcb01682e0 2023-10-26T01:30:23.441 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.437+0000 7fa125832640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1200965e0 con 0x7fa1200cd7e0 2023-10-26T01:30:23.441 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.438+0000 7fa11e7fc640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa1200cd7e0 0x7fa120071620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:23.441 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.438+0000 7fa11e7fc640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa1200cd7e0 0x7fa120071620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3301/0 says I am v2:172.21.15.169:48642/0 (socket says 172.21.15.169:48642) 2023-10-26T01:30:23.441 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.438+0000 7fa11e7fc640 1 -- 172.21.15.169:0/376751244 learned_addr learned my addr 172.21.15.169:0/376751244 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:30:23.441 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.438+0000 7fa11f7fe640 1 --2- 172.21.15.169:0/376751244 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa12015b4a0 0x7fa12006feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:23.443 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.441+0000 7fa11f7fe640 1 -- 172.21.15.169:0/376751244 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa1200cd7e0 msgr2=0x7fa120071620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:23.443 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.441+0000 7fa11f7fe640 1 --2- 172.21.15.169:0/376751244 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa1200cd7e0 0x7fa120071620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:23.443 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.441+0000 7fa11f7fe640 1 -- 172.21.15.169:0/376751244 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa12006ce00 msgr2=0x7fa1200cafb0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-26T01:30:23.443 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.441+0000 7fa11f7fe640 1 --2- 172.21.15.169:0/376751244 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa12006ce00 0x7fa1200cafb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:23.444 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.441+0000 7fa11f7fe640 1 -- 172.21.15.169:0/376751244 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa114009560 con 0x7fa12015b4a0 2023-10-26T01:30:23.447 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.442+0000 7fd66ad16640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd64c003970 0x7fd66414fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:23.447 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.442+0000 7fd66ad16640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd6640cc960 0x7fd664096590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:23.448 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.442+0000 7fd66ad16640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd664096ad0 0x7fd6641f5f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:23.448 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.445+0000 7fd663fff640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd6640cc960 0x7fd664096590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:23.448 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.445+0000 7fd663fff640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd6640cc960 0x7fd664096590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3301/0 says I am v2:172.21.15.169:48652/0 (socket says 172.21.15.169:48652) 2023-10-26T01:30:23.448 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.446+0000 7fd668ab2640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd64c003970 0x7fd66414fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:23.448 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.446+0000 7fd668ab2640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd64c003970 0x7fd66414fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:55356/0 (socket says 172.21.15.169:55356) 2023-10-26T01:30:23.449 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.446+0000 7fd66ad16640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd664096f50 con 0x7fd64c003970 2023-10-26T01:30:23.449 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.447+0000 7fd6692b3640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd664096ad0 0x7fd6641f5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:23.450 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.447+0000 7fd6692b3640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd664096ad0 0x7fd6641f5f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.169:55546/0 (socket says 172.21.15.169:55546) 2023-10-26T01:30:23.450 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.447+0000 7fd6692b3640 1 -- 172.21.15.169:0/831209740 learned_addr learned my addr 172.21.15.169:0/831209740 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:30:23.450 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.447+0000 7f860759e640 1 -- 172.21.15.169:0/3083163079 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) v1 -- 0x7f85e00050c0 con 0x7f86080d03b0 2023-10-26T01:30:23.451 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.447+0000 7fd66ad16640 1 -- 172.21.15.169:0/831209740 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6640cb8c0 con 0x7fd664096ad0 2023-10-26T01:30:23.451 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.446+0000 7fa11f7fe640 1 --2- 172.21.15.169:0/376751244 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa12015b4a0 0x7fa12006feb0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa11800be20 tx=0x7fa11800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:23.452 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.447+0000 7fd66ad16640 1 -- 172.21.15.169:0/831209740 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6640cbba0 con 0x7fd6640cc960 2023-10-26T01:30:23.452 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.448+0000 7fd6692b3640 1 -- 172.21.15.169:0/831209740 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd6640cc960 msgr2=0x7fd664096590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:23.453 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.449+0000 7fd6692b3640 1 --2- 172.21.15.169:0/831209740 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd6640cc960 0x7fd664096590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:23.453 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.449+0000 7fd6692b3640 1 -- 172.21.15.169:0/831209740 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd64c003970 msgr2=0x7fd66414fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:23.453 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.449+0000 7fd6692b3640 1 --2- 172.21.15.169:0/831209740 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd64c003970 0x7fd66414fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:23.453 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.449+0000 7fd6692b3640 1 -- 172.21.15.169:0/831209740 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6640cbe80 con 0x7fd664096ad0 2023-10-26T01:30:23.453 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.450+0000 7fd668ab2640 1 --2- 172.21.15.169:0/831209740 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd64c003970 0x7fd66414fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2023-10-26T01:30:23.454 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.450+0000 7fd663fff640 1 --2- 172.21.15.169:0/831209740 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd6640cc960 0x7fd664096590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2023-10-26T01:30:23.454 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.450+0000 7fd6692b3640 1 --2- 172.21.15.169:0/831209740 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd664096ad0 0x7fd6641f5f80 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fd6640ce330 tx=0x7fd654009800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:23.454 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.451+0000 7fd6617fa640 1 -- 172.21.15.169:0/831209740 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6540052d0 con 0x7fd664096ad0 2023-10-26T01:30:23.455 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.451+0000 7fd6617fa640 1 -- 172.21.15.169:0/831209740 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(3 keys) v1 ==== 79+0+0 (secure 0 0 0) 0x7fd654005450 con 0x7fd664096ad0 2023-10-26T01:30:23.455 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.451+0000 7fd66ad16640 1 -- 172.21.15.169:0/831209740 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6640cc160 con 0x7fd664096ad0 2023-10-26T01:30:23.455 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.451+0000 7fd66ad16640 1 -- 172.21.15.169:0/831209740 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd66414b930 con 0x7fd664096ad0 2023-10-26T01:30:23.455 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.451+0000 7fd6617fa640 1 -- 172.21.15.169:0/831209740 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6540037c0 con 0x7fd664096ad0 2023-10-26T01:30:23.456 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.452+0000 7fd6617fa640 1 -- 172.21.15.169:0/831209740 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fd654003cc0 con 0x7fd664096ad0 2023-10-26T01:30:23.456 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.452+0000 7fd6617fa640 1 -- 172.21.15.169:0/831209740 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fd65401b6f0 con 0x7fd664096ad0 2023-10-26T01:30:23.456 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.455+0000 7fdefb7fe640 1 -- 172.21.15.169:0/1732708010 <== mon.0 v2:172.21.15.169:3300/0 7 ==== config(4 keys) v1 ==== 110+0+0 (secure 0 0 0) 0x7fdf04005020 con 0x7fdf1406b310 2023-10-26T01:30:23.457 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.456+0000 7fd6617fa640 1 -- 172.21.15.169:0/831209740 <== mon.1 v2:172.21.15.170:3300/0 6 ==== config(4 keys) v1 ==== 110+0+0 (secure 0 0 0) 0x7fd65401ba10 con 0x7fd664096ad0 2023-10-26T01:30:23.458 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.456+0000 7fa107fff640 1 -- 172.21.15.169:0/376751244 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa11800ce70 con 0x7fa12015b4a0 2023-10-26T01:30:23.458 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.456+0000 7fa125832640 1 -- 172.21.15.169:0/376751244 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1200cb4f0 con 0x7fa12015b4a0 2023-10-26T01:30:23.458 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.456+0000 7fa125832640 1 -- 172.21.15.169:0/376751244 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa120096000 con 0x7fa12015b4a0 2023-10-26T01:30:23.459 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.457+0000 7f85f6ffd640 1 -- 172.21.15.169:0/3083163079 <== mon.1 v2:172.21.15.170:3300/0 9 ==== config(4 keys) v1 ==== 110+0+0 (secure 0 0 0) 0x7f860001e670 con 0x7f86080d03b0 2023-10-26T01:30:23.459 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.457+0000 7fa107fff640 1 -- 172.21.15.169:0/376751244 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(4 keys) v1 ==== 110+0+0 (secure 0 0 0) 0x7fa118014510 con 0x7fa12015b4a0 2023-10-26T01:30:23.460 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.457+0000 7fa107fff640 1 -- 172.21.15.169:0/376751244 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa11800ce70 con 0x7fa12015b4a0 2023-10-26T01:30:23.462 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.459+0000 7efcacff9640 1 -- 172.21.15.169:0/16766884 <== mon.2 v2:172.21.15.170:3301/0 8 ==== config(4 keys) v1 ==== 110+0+0 (secure 0 0 0) 0x7efc980165e0 con 0x7efcb01682e0 2023-10-26T01:30:23.462 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.461+0000 7efcacff9640 1 -- 172.21.15.169:0/16766884 <== mon.2 v2:172.21.15.170:3301/0 9 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v5)=0 v5) v1 ==== 123+0+0 (secure 0 0 0) 0x7efc9802d0c0 con 0x7efcb01682e0 2023-10-26T01:30:23.463 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.462+0000 7fa107fff640 1 -- 172.21.15.169:0/376751244 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fa118014a40 con 0x7fa12015b4a0 2023-10-26T01:30:23.464 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.463+0000 7fa107fff640 1 -- 172.21.15.169:0/376751244 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fa11801d3b0 con 0x7fa12015b4a0 2023-10-26T01:30:23.465 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.464+0000 7fd66ad16640 1 -- 172.21.15.169:0/831209740 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd644005040 con 0x7fd664096ad0 2023-10-26T01:30:23.466 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.465+0000 7f85f6ffd640 1 -- 172.21.15.169:0/3083163079 <== mon.1 v2:172.21.15.170:3300/0 10 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f860001e670 con 0x7f86080d03b0 2023-10-26T01:30:23.468 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.466+0000 7efcacff9640 1 -- 172.21.15.169:0/16766884 <== mon.2 v2:172.21.15.170:3301/0 10 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7efc980165e0 con 0x7efcb01682e0 2023-10-26T01:30:23.468 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.467+0000 7fd6617fa640 1 -- 172.21.15.169:0/831209740 <== mon.1 v2:172.21.15.170:3300/0 7 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd65401be80 con 0x7fd664096ad0 2023-10-26T01:30:23.470 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.469+0000 7fa107fff640 1 -- 172.21.15.169:0/376751244 <== mon.0 v2:172.21.15.169:3300/0 6 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa11801d660 con 0x7fa12015b4a0 2023-10-26T01:30:23.472 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.470+0000 7fd6617fa640 1 -- 172.21.15.169:0/831209740 <== mon.1 v2:172.21.15.170:3300/0 8 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7fd654017070 con 0x7fd664096ad0 2023-10-26T01:30:23.472 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.468+0000 7fdefb7fe640 1 -- 172.21.15.169:0/1732708010 <== mon.0 v2:172.21.15.169:3300/0 8 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fdf04005250 con 0x7fdf1406b310 2023-10-26T01:30:23.477 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.472+0000 7fa125832640 1 -- 172.21.15.169:0/376751244 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0f8005250 con 0x7fa12015b4a0 2023-10-26T01:30:23.478 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.477+0000 7efcb5b2c640 1 -- 172.21.15.169:0/16766884 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7efcb01682e0 msgr2=0x7efcb0098ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:23.479 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.477+0000 7f85f6ffd640 1 -- 172.21.15.169:0/3083163079 <== mon.1 v2:172.21.15.170:3300/0 11 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v6)=0 v6) v1 ==== 123+0+0 (secure 0 0 0) 0x7f8600019070 con 0x7f86080d03b0 2023-10-26T01:30:23.479 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.477+0000 7efcb5b2c640 1 --2- 172.21.15.169:0/16766884 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7efcb01682e0 0x7efcb0098ab0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7efc98005fd0 tx=0x7efc9800be20 comp rx=0 tx=0).stop 2023-10-26T01:30:23.481 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.479+0000 7efcb5b2c640 1 -- 172.21.15.169:0/16766884 shutdown_connections 2023-10-26T01:30:23.481 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.479+0000 7efcb5b2c640 1 --2- 172.21.15.169:0/16766884 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efcb01686c0 0x7efcb0096350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:23.481 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.479+0000 7efcb5b2c640 1 --2- 172.21.15.169:0/16766884 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7efcb01682e0 0x7efcb0098ab0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:23.481 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.479+0000 7efcb5b2c640 1 --2- 172.21.15.169:0/16766884 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efcb00920b0 0x7efcb0205410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:23.482 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.479+0000 7efcb5b2c640 1 -- 172.21.15.169:0/16766884 >> 172.21.15.169:0/16766884 conn(0x7efcb00b2800 msgr2=0x7efcb00abef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:23.485 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.483+0000 7efcb5b2c640 1 -- 172.21.15.169:0/16766884 shutdown_connections 2023-10-26T01:30:23.485 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.484+0000 7efcb5b2c640 1 -- 172.21.15.169:0/16766884 wait complete. 2023-10-26T01:30:23.486 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.484+0000 7fa107fff640 1 -- 172.21.15.169:0/376751244 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7fa118019070 con 0x7fa12015b4a0 2023-10-26T01:30:23.486 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.485+0000 7f860759e640 1 -- 172.21.15.169:0/3083163079 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f86080d03b0 msgr2=0x7f8608150420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:23.486 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.485+0000 7f860759e640 1 --2- 172.21.15.169:0/3083163079 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f86080d03b0 0x7f8608150420 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f860000b6f0 tx=0x7f860000bbb0 comp rx=0 tx=0).stop 2023-10-26T01:30:23.490 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.488+0000 7f860759e640 1 -- 172.21.15.169:0/3083163079 shutdown_connections 2023-10-26T01:30:23.490 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.488+0000 7f860759e640 1 --2- 172.21.15.169:0/3083163079 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f86080d03b0 0x7f8608150420 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:23.491 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.488+0000 7f860759e640 1 --2- 172.21.15.169:0/3083163079 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f86080cca10 0x7f860814fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:23.491 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.488+0000 7f860759e640 1 --2- 172.21.15.169:0/3083163079 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86080cc630 0x7f86080ca750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:23.491 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.488+0000 7f860759e640 1 -- 172.21.15.169:0/3083163079 >> 172.21.15.169:0/3083163079 conn(0x7f86080d1360 msgr2=0x7f86080b1ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:23.492 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.488+0000 7f860759e640 1 -- 172.21.15.169:0/3083163079 shutdown_connections 2023-10-26T01:30:23.492 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.488+0000 7f860759e640 1 -- 172.21.15.169:0/3083163079 wait complete. 2023-10-26T01:30:23.533 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi169') 2023-10-26T01:30:23.540 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_mds', '20'], check_status=True, hostname='smithi169') 2023-10-26T01:30:23.590 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.585+0000 7f84c2e1f640 1 -- 172.21.15.169:0/1201564021 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f84bc15b2e0 msgr2=0x7f84bc15b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:23.591 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.585+0000 7f84c2e1f640 1 --2- 172.21.15.169:0/1201564021 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f84bc15b2e0 0x7f84bc15b770 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f84b4009830 tx=0x7f84b402bd50 comp rx=0 tx=0).stop 2023-10-26T01:30:23.591 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.589+0000 7f84c2e1f640 1 -- 172.21.15.169:0/1201564021 shutdown_connections 2023-10-26T01:30:23.592 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.589+0000 7f84c2e1f640 1 --2- 172.21.15.169:0/1201564021 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f84bc15b2e0 0x7f84bc15b770 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:23.592 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.589+0000 7f84c2e1f640 1 --2- 172.21.15.169:0/1201564021 >> v2:172.21.15.169:3300/0 conn(0x7f84bc0b0710 0x7f84bc0b3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:23.592 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.589+0000 7f84c2e1f640 1 -- 172.21.15.169:0/1201564021 >> 172.21.15.169:0/1201564021 conn(0x7f84bc0ce130 msgr2=0x7f84bc0ce510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:23.592 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.589+0000 7f84c2e1f640 1 -- 172.21.15.169:0/1201564021 shutdown_connections 2023-10-26T01:30:23.593 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.589+0000 7fb07d26f640 1 -- 172.21.15.169:0/1785608067 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb0780d20c0 msgr2=0x7fb0780ce090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:23.593 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.589+0000 7fb07d26f640 1 --2- 172.21.15.169:0/1785608067 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb0780d20c0 0x7fb0780ce090 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fb0780d1ac0 tx=0x7fb06c02bd40 comp rx=0 tx=0).stop 2023-10-26T01:30:23.593 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.589+0000 7fb07d26f640 1 -- 172.21.15.169:0/1785608067 shutdown_connections 2023-10-26T01:30:23.593 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.589+0000 7fb07d26f640 1 --2- 172.21.15.169:0/1785608067 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb0780d20c0 0x7fb0780ce090 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:23.594 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.589+0000 7fb07d26f640 1 --2- 172.21.15.169:0/1785608067 >> v2:172.21.15.169:3300/0 conn(0x7fb0780d0fc0 0x7fb0780d1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:23.594 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.589+0000 7fb07d26f640 1 --2- 172.21.15.169:0/1785608067 >> v2:172.21.15.170:3300/0 conn(0x7fb07815ba00 0x7fb0780b6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:23.594 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.589+0000 7fb07d26f640 1 -- 172.21.15.169:0/1785608067 >> 172.21.15.169:0/1785608067 conn(0x7fb07806b5c0 msgr2=0x7fb07806b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:23.595 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.589+0000 7f84c2e1f640 1 -- 172.21.15.169:0/1201564021 wait complete. 2023-10-26T01:30:23.595 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.590+0000 7fb07d26f640 1 -- 172.21.15.169:0/1785608067 shutdown_connections 2023-10-26T01:30:23.595 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.590+0000 7fb07d26f640 1 -- 172.21.15.169:0/1785608067 wait complete. 2023-10-26T01:30:23.597 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.594+0000 7f84c2e1f640 1 Processor -- start 2023-10-26T01:30:23.597 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.593+0000 7fb07d26f640 1 Processor -- start 2023-10-26T01:30:23.598 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.594+0000 7f84c2e1f640 1 -- start start 2023-10-26T01:30:23.598 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.596+0000 7fb07d26f640 1 -- start start 2023-10-26T01:30:23.601 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.600+0000 7f84c2e1f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f84bc0b0710 0x7f84bc15e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:23.603 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.600+0000 7f84c2e1f640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84bc0b3130 0x7f84bc0b4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:23.603 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.600+0000 7f84c2e1f640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f84bc15b2e0 0x7f84bc0c8780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:23.603 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.600+0000 7f84c0bbb640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f84bc0b0710 0x7f84bc15e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:23.604 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.601+0000 7f84c0bbb640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f84bc0b0710 0x7f84bc15e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.169:55572/0 (socket says 172.21.15.169:55572) 2023-10-26T01:30:23.604 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.601+0000 7f84c2e1f640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84bc0b6bc0 con 0x7f84bc0b3130 2023-10-26T01:30:23.604 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.601+0000 7f84c0bbb640 1 -- 172.21.15.169:0/3585463299 learned_addr learned my addr 172.21.15.169:0/3585463299 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:30:23.605 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.601+0000 7f84c2e1f640 1 -- 172.21.15.169:0/3585463299 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84bc0cb6b0 con 0x7f84bc0b0710 2023-10-26T01:30:23.605 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.601+0000 7f84c2e1f640 1 -- 172.21.15.169:0/3585463299 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84bc0b6640 con 0x7f84bc15b2e0 2023-10-26T01:30:23.605 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.601+0000 7f84c13bc640 1 --2- 172.21.15.169:0/3585463299 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f84bc15b2e0 0x7f84bc0c8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:23.606 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.601+0000 7f84b3fff640 1 --2- 172.21.15.169:0/3585463299 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84bc0b3130 0x7f84bc0b4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:23.607 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.605+0000 7fb07d26f640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb0780d0fc0 0x7fb07815ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:23.607 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.605+0000 7fb07d26f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0780d20c0 0x7fb0780c51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:23.608 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.605+0000 7fb07d26f640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb07815ba00 0x7fb0780c56f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:23.608 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.602+0000 7f84c13bc640 1 -- 172.21.15.169:0/3585463299 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f84bc0b0710 msgr2=0x7f84bc15e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:23.609 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.602+0000 7f84c13bc640 1 --2- 172.21.15.169:0/3585463299 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f84bc0b0710 0x7f84bc15e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:23.609 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.602+0000 7f84c13bc640 1 -- 172.21.15.169:0/3585463299 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84bc0b3130 msgr2=0x7f84bc0b4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:23.609 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.602+0000 7f84c13bc640 1 --2- 172.21.15.169:0/3585463299 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84bc0b3130 0x7f84bc0b4cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:23.610 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.602+0000 7f84c13bc640 1 -- 172.21.15.169:0/3585463299 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84b4009490 con 0x7f84bc15b2e0 2023-10-26T01:30:23.610 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.602+0000 7f84c0bbb640 1 --2- 172.21.15.169:0/3585463299 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f84bc0b0710 0x7f84bc15e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T01:30:23.610 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.602+0000 7f84b3fff640 1 --2- 172.21.15.169:0/3585463299 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84bc0b3130 0x7f84bc0b4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T01:30:23.611 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.603+0000 7f84c13bc640 1 --2- 172.21.15.169:0/3585463299 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f84bc15b2e0 0x7f84bc0c8780 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f84b4033000 tx=0x7f84b4003820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:23.611 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.603+0000 7f84b9ffb640 1 -- 172.21.15.169:0/3585463299 <== mon.2 v2:172.21.15.170:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b4003e20 con 0x7f84bc15b2e0 2023-10-26T01:30:23.613 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.603+0000 7f84c2e1f640 1 -- 172.21.15.169:0/3585463299 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84bc0cb0a0 con 0x7f84bc15b2e0 2023-10-26T01:30:23.613 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.603+0000 7f84c2e1f640 1 -- 172.21.15.169:0/3585463299 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84bc0cb340 con 0x7f84bc15b2e0 2023-10-26T01:30:23.614 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.604+0000 7f84b9ffb640 1 -- 172.21.15.169:0/3585463299 <== mon.2 v2:172.21.15.170:3301/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f84b402c9b0 con 0x7f84bc15b2e0 2023-10-26T01:30:23.614 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.604+0000 7f84b9ffb640 1 -- 172.21.15.169:0/3585463299 <== mon.2 v2:172.21.15.170:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b4003e20 con 0x7f84bc15b2e0 2023-10-26T01:30:23.614 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.604+0000 7f84b9ffb640 1 -- 172.21.15.169:0/3585463299 <== mon.2 v2:172.21.15.170:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f84b403b040 con 0x7f84bc15b2e0 2023-10-26T01:30:23.615 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.604+0000 7f84b9ffb640 1 -- 172.21.15.169:0/3585463299 <== mon.2 v2:172.21.15.170:3301/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f84b403d430 con 0x7f84bc15b2e0 2023-10-26T01:30:23.616 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.605+0000 7fb076ffd640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb0780d0fc0 0x7fb07815ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:23.617 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.605+0000 7fb076ffd640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb0780d0fc0 0x7fb07815ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3301/0 says I am v2:172.21.15.169:48688/0 (socket says 172.21.15.169:48688) 2023-10-26T01:30:23.617 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.606+0000 7fb0767fc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0780d20c0 0x7fb0780c51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:23.617 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.606+0000 7fb0767fc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0780d20c0 0x7fb0780c51b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.169:55586/0 (socket says 172.21.15.169:55586) 2023-10-26T01:30:23.618 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.607+0000 7fb076ffd640 1 -- 172.21.15.169:0/3200498440 learned_addr learned my addr 172.21.15.169:0/3200498440 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:30:23.618 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.607+0000 7fb07d26f640 1 -- 172.21.15.169:0/3200498440 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0781f5380 con 0x7fb07815ba00 2023-10-26T01:30:23.618 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.609+0000 7fb0777fe640 1 --2- 172.21.15.169:0/3200498440 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb07815ba00 0x7fb0780c56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:23.619 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.609+0000 7fb07d26f640 1 -- 172.21.15.169:0/3200498440 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb07815d6b0 con 0x7fb0780d20c0 2023-10-26T01:30:23.619 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.609+0000 7fb07d26f640 1 -- 172.21.15.169:0/3200498440 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb07815d990 con 0x7fb0780d0fc0 2023-10-26T01:30:23.619 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.611+0000 7f84c2e1f640 1 -- 172.21.15.169:0/3585463299 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84a0005250 con 0x7f84bc15b2e0 2023-10-26T01:30:23.621 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.610+0000 7fb0777fe640 1 -- 172.21.15.169:0/3200498440 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb0780d0fc0 msgr2=0x7fb07815ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:23.621 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.610+0000 7fb0777fe640 1 --2- 172.21.15.169:0/3200498440 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb0780d0fc0 0x7fb07815ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:23.621 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.610+0000 7fb0777fe640 1 -- 172.21.15.169:0/3200498440 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0780d20c0 msgr2=0x7fb0780c51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:23.621 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.610+0000 7fb0777fe640 1 --2- 172.21.15.169:0/3200498440 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0780d20c0 0x7fb0780c51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:23.622 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.610+0000 7fb0777fe640 1 -- 172.21.15.169:0/3200498440 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb06c009560 con 0x7fb07815ba00 2023-10-26T01:30:23.622 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.614+0000 7fb0777fe640 1 --2- 172.21.15.169:0/3200498440 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb07815ba00 0x7fb0780c56f0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb06c004000 tx=0x7fb06c02d900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:23.622 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.618+0000 7fb05f7fe640 1 -- 172.21.15.169:0/3200498440 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb06c02dc70 con 0x7fb07815ba00 2023-10-26T01:30:23.622 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.619+0000 7fb05f7fe640 1 -- 172.21.15.169:0/3200498440 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb06c004500 con 0x7fb07815ba00 2023-10-26T01:30:23.623 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.619+0000 7fb05f7fe640 1 -- 172.21.15.169:0/3200498440 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb06c003810 con 0x7fb07815ba00 2023-10-26T01:30:23.623 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.620+0000 7f84b9ffb640 1 -- 172.21.15.169:0/3585463299 <== mon.2 v2:172.21.15.170:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f84b4041020 con 0x7f84bc15b2e0 2023-10-26T01:30:23.633 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.619+0000 7fb07d26f640 1 -- 172.21.15.169:0/3200498440 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb07815dc70 con 0x7fb07815ba00 2023-10-26T01:30:23.633 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.619+0000 7fb07d26f640 1 -- 172.21.15.169:0/3200498440 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb07815e230 con 0x7fb07815ba00 2023-10-26T01:30:23.634 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.623+0000 7fb07d26f640 1 -- 172.21.15.169:0/3200498440 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb058005250 con 0x7fb07815ba00 2023-10-26T01:30:23.634 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.628+0000 7fb05f7fe640 1 -- 172.21.15.169:0/3200498440 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fb06c048070 con 0x7fb07815ba00 2023-10-26T01:30:23.638 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.633+0000 7fb05f7fe640 1 -- 172.21.15.169:0/3200498440 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fb06c034870 con 0x7fb07815ba00 2023-10-26T01:30:23.638 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.633+0000 7fb05f7fe640 1 -- 172.21.15.169:0/3200498440 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7fb06c04f070 con 0x7fb07815ba00 2023-10-26T01:30:23.660 INFO:tasks.ceph.mgr.x.smithi170.stderr:2023-10-26T01:30:23.657+0000 7f2a20213040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-26T01:30:23.660 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.655+0000 7fdf1962b640 1 -- 172.21.15.169:0/1732708010 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command([{prefix=config set, name=debug_mds}] v 0) v1 -- 0x7fdeec0052b0 con 0x7fdf1406b310 2023-10-26T01:30:23.661 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.660+0000 7efd567a5640 1 -- 172.21.15.169:0/2858974257 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd500cc8e0 msgr2=0x7efd500d03b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:23.662 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.660+0000 7efd567a5640 1 --2- 172.21.15.169:0/2858974257 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd500cc8e0 0x7efd500d03b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7efd500c4c90 tx=0x7efd4c02bdb0 comp rx=0 tx=0).stop 2023-10-26T01:30:23.666 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.665+0000 7fdefb7fe640 1 -- 172.21.15.169:0/1732708010 <== mon.0 v2:172.21.15.169:3300/0 9 ==== mon_command_ack([{prefix=config set, name=debug_mds}]=0 v7)=0 v7) v1 ==== 108+0+0 (secure 0 0 0) 0x7fdf04004b30 con 0x7fdf1406b310 2023-10-26T01:30:23.670 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.669+0000 7efd567a5640 1 -- 172.21.15.169:0/2858974257 shutdown_connections 2023-10-26T01:30:23.670 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.669+0000 7efd567a5640 1 --2- 172.21.15.169:0/2858974257 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd500cc8e0 0x7efd500d03b0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:23.671 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.669+0000 7efd567a5640 1 --2- 172.21.15.169:0/2858974257 >> v2:172.21.15.170:3300/0 conn(0x7efd500b4770 0x7efd500cc3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:23.671 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.669+0000 7efd567a5640 1 -- 172.21.15.169:0/2858974257 >> 172.21.15.169:0/2858974257 conn(0x7efd500d1360 msgr2=0x7efd500d1740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:23.672 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.670+0000 7fdf1962b640 1 -- 172.21.15.169:0/1732708010 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdf1406b310 msgr2=0x7fdf140965e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:23.672 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.670+0000 7fdf1962b640 1 --2- 172.21.15.169:0/1732708010 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdf1406b310 0x7fdf140965e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fdf0400e9f0 tx=0x7fdf0400eeb0 comp rx=0 tx=0).stop 2023-10-26T01:30:23.672 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.670+0000 7fdf1962b640 1 -- 172.21.15.169:0/1732708010 shutdown_connections 2023-10-26T01:30:23.672 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.670+0000 7fdf1962b640 1 --2- 172.21.15.169:0/1732708010 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdf1406c460 0x7fdf140af9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:23.673 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.670+0000 7fdf1962b640 1 --2- 172.21.15.169:0/1732708010 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdf1406c080 0x7fdf140af4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:23.673 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.670+0000 7fdf1962b640 1 --2- 172.21.15.169:0/1732708010 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdf1406b310 0x7fdf140965e0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:23.673 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.670+0000 7fdf1962b640 1 -- 172.21.15.169:0/1732708010 >> 172.21.15.169:0/1732708010 conn(0x7fdf140b2800 msgr2=0x7fdf140aafd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:23.673 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.670+0000 7fdf1962b640 1 -- 172.21.15.169:0/1732708010 shutdown_connections 2023-10-26T01:30:23.674 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.670+0000 7fdf1962b640 1 -- 172.21.15.169:0/1732708010 wait complete. 2023-10-26T01:30:23.674 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.672+0000 7efd567a5640 1 -- 172.21.15.169:0/2858974257 shutdown_connections 2023-10-26T01:30:23.674 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.672+0000 7efd567a5640 1 -- 172.21.15.169:0/2858974257 wait complete. 2023-10-26T01:30:23.680 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.674+0000 7efd567a5640 1 Processor -- start 2023-10-26T01:30:23.681 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.680+0000 7efd567a5640 1 -- start start 2023-10-26T01:30:23.683 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.682+0000 7efd567a5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd500b4770 0x7efd501f21e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:23.684 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.682+0000 7efd567a5640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd500cc8e0 0x7efd500c6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:23.685 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.683+0000 7efd557a3640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd500b4770 0x7efd501f21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:23.685 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.683+0000 7efd557a3640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd500b4770 0x7efd501f21e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.169:55602/0 (socket says 172.21.15.169:55602) 2023-10-26T01:30:23.685 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.683+0000 7efd557a3640 1 -- 172.21.15.169:0/2880188434 learned_addr learned my addr 172.21.15.169:0/2880188434 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:30:23.685 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.683+0000 7efd567a5640 1 --2- 172.21.15.169:0/2880188434 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7efd5015ba30 0x7efd500c9db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:23.686 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.684+0000 7efd567a5640 1 -- 172.21.15.169:0/2880188434 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd500b2650 con 0x7efd500cc8e0 2023-10-26T01:30:23.686 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.684+0000 7efd567a5640 1 -- 172.21.15.169:0/2880188434 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd501f5c00 con 0x7efd500b4770 2023-10-26T01:30:23.687 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.684+0000 7efd55fa4640 1 --2- 172.21.15.169:0/2880188434 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7efd5015ba30 0x7efd500c9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:23.687 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.684+0000 7efd567a5640 1 -- 172.21.15.169:0/2880188434 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd501f5ee0 con 0x7efd5015ba30 2023-10-26T01:30:23.688 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.685+0000 7efd54fa2640 1 --2- 172.21.15.169:0/2880188434 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd500cc8e0 0x7efd500c6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:23.688 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.685+0000 7efd557a3640 1 -- 172.21.15.169:0/2880188434 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7efd5015ba30 msgr2=0x7efd500c9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:23.688 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.685+0000 7efd557a3640 1 --2- 172.21.15.169:0/2880188434 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7efd5015ba30 0x7efd500c9db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:23.688 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.685+0000 7efd557a3640 1 -- 172.21.15.169:0/2880188434 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd500cc8e0 msgr2=0x7efd500c6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:23.689 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.686+0000 7efd557a3640 1 --2- 172.21.15.169:0/2880188434 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd500cc8e0 0x7efd500c6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:23.689 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.686+0000 7efd557a3640 1 -- 172.21.15.169:0/2880188434 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd4c009490 con 0x7efd500b4770 2023-10-26T01:30:23.689 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.686+0000 7efd55fa4640 1 --2- 172.21.15.169:0/2880188434 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7efd5015ba30 0x7efd500c9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T01:30:23.689 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.687+0000 7efd54fa2640 1 --2- 172.21.15.169:0/2880188434 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd500cc8e0 0x7efd500c6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:30:23.690 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.687+0000 7efd557a3640 1 --2- 172.21.15.169:0/2880188434 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd500b4770 0x7efd501f21e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7efd4800cbc0 tx=0x7efd4800a430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:23.690 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.687+0000 7efd45ffb640 1 -- 172.21.15.169:0/2880188434 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd4800ad40 con 0x7efd500b4770 2023-10-26T01:30:23.690 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.688+0000 7efd567a5640 1 -- 172.21.15.169:0/2880188434 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd501f61c0 con 0x7efd500b4770 2023-10-26T01:30:23.691 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.688+0000 7efd567a5640 1 -- 172.21.15.169:0/2880188434 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd500c7200 con 0x7efd500b4770 2023-10-26T01:30:23.691 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.688+0000 7efd45ffb640 1 -- 172.21.15.169:0/2880188434 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7efd48012420 con 0x7efd500b4770 2023-10-26T01:30:23.691 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.689+0000 7efd45ffb640 1 -- 172.21.15.169:0/2880188434 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd4800ad40 con 0x7efd500b4770 2023-10-26T01:30:23.693 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.691+0000 7efd45ffb640 1 -- 172.21.15.169:0/2880188434 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7efd48012950 con 0x7efd500b4770 2023-10-26T01:30:23.693 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.692+0000 7efd45ffb640 1 -- 172.21.15.169:0/2880188434 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7efd4801bb30 con 0x7efd500b4770 2023-10-26T01:30:23.693 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.692+0000 7efd567a5640 1 -- 172.21.15.169:0/2880188434 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd30005250 con 0x7efd500b4770 2023-10-26T01:30:23.699 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.698+0000 7f312f939640 1 -- 172.21.15.169:0/1615880345 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f31281574b0 msgr2=0x7f3128071c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:23.700 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.698+0000 7f312f939640 1 --2- 172.21.15.169:0/1615880345 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f31281574b0 0x7f3128071c30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f311c0083e0 tx=0x7f311c005f90 comp rx=0 tx=0).stop 2023-10-26T01:30:23.700 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.698+0000 7efd45ffb640 1 -- 172.21.15.169:0/2880188434 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7efd48016020 con 0x7efd500b4770 2023-10-26T01:30:23.701 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.699+0000 7fa125832640 1 -- 172.21.15.169:0/376751244 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) v1 -- 0x7fa0f80050c0 con 0x7fa12015b4a0 2023-10-26T01:30:23.701 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.699+0000 7f312f939640 1 -- 172.21.15.169:0/1615880345 shutdown_connections 2023-10-26T01:30:23.701 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.699+0000 7f312f939640 1 --2- 172.21.15.169:0/1615880345 >> v2:172.21.15.169:3300/0 conn(0x7f312806ad60 0x7f31280700b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:23.702 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.699+0000 7f312f939640 1 --2- 172.21.15.169:0/1615880345 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f31281574b0 0x7f3128071c30 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:23.702 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.699+0000 7f312f939640 1 -- 172.21.15.169:0/1615880345 >> 172.21.15.169:0/1615880345 conn(0x7f312806f200 msgr2=0x7f31280aee70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:23.704 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.703+0000 7f312f939640 1 -- 172.21.15.169:0/1615880345 shutdown_connections 2023-10-26T01:30:23.705 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.703+0000 7f312f939640 1 -- 172.21.15.169:0/1615880345 wait complete. 2023-10-26T01:30:23.712 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_mds_balancer', '20'], check_status=True, hostname='smithi169') 2023-10-26T01:30:23.712 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.710+0000 7fa107fff640 1 -- 172.21.15.169:0/376751244 <== mon.0 v2:172.21.15.169:3300/0 8 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v8)=0 v8) v1 ==== 106+0+0 (secure 0 0 0) 0x7fa11801e440 con 0x7fa12015b4a0 2023-10-26T01:30:23.714 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.711+0000 7f312f939640 1 Processor -- start 2023-10-26T01:30:23.719 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.711+0000 7f312f939640 1 -- start start 2023-10-26T01:30:23.722 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.721+0000 7fa125832640 1 -- 172.21.15.169:0/376751244 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa12015b4a0 msgr2=0x7fa12006feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:23.722 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.721+0000 7fa125832640 1 --2- 172.21.15.169:0/376751244 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa12015b4a0 0x7fa12006feb0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa11800be20 tx=0x7fa11800bf10 comp rx=0 tx=0).stop 2023-10-26T01:30:23.724 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.722+0000 7f312f939640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f312806ad60 0x7f31281e3030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:23.724 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.722+0000 7f312f939640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f31281574b0 0x7f31281e3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:23.725 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.723+0000 7fa125832640 1 -- 172.21.15.169:0/376751244 shutdown_connections 2023-10-26T01:30:23.725 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.723+0000 7fa125832640 1 --2- 172.21.15.169:0/376751244 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa12015b4a0 0x7fa12006feb0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:23.725 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.723+0000 7fa125832640 1 --2- 172.21.15.169:0/376751244 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa1200cd7e0 0x7fa120071620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:23.726 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.723+0000 7fa125832640 1 --2- 172.21.15.169:0/376751244 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa12006ce00 0x7fa1200cafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:23.726 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.723+0000 7fa125832640 1 -- 172.21.15.169:0/376751244 >> 172.21.15.169:0/376751244 conn(0x7fa120069930 msgr2=0x7fa12006b200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:23.726 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.723+0000 7fa125832640 1 -- 172.21.15.169:0/376751244 shutdown_connections 2023-10-26T01:30:23.726 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.723+0000 7fa125832640 1 -- 172.21.15.169:0/376751244 wait complete. 2023-10-26T01:30:23.727 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.722+0000 7f312f939640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3128157890 0x7f3128155270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:23.728 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.723+0000 7f312f939640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31280aec50 con 0x7f3128157890 2023-10-26T01:30:23.728 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.723+0000 7f312f939640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31280984b0 con 0x7f312806ad60 2023-10-26T01:30:23.728 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.723+0000 7f312f939640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31281e3ab0 con 0x7f31281574b0 2023-10-26T01:30:23.728 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.723+0000 7f312d6d5640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3128157890 0x7f3128155270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:23.728 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.724+0000 7f312d6d5640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3128157890 0x7f3128155270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:55450/0 (socket says 172.21.15.169:55450) 2023-10-26T01:30:23.729 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.724+0000 7f312d6d5640 1 -- 172.21.15.169:0/335637154 learned_addr learned my addr 172.21.15.169:0/335637154 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:30:23.729 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.725+0000 7f3127fff640 1 --2- 172.21.15.169:0/335637154 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f31281574b0 0x7f31281e3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:23.729 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.725+0000 7f312ced4640 1 --2- 172.21.15.169:0/335637154 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f312806ad60 0x7f31281e3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:23.729 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.726+0000 7f312d6d5640 1 -- 172.21.15.169:0/335637154 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f31281574b0 msgr2=0x7f31281e3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:23.730 INFO:tasks.ceph.mgr.x.smithi170.stderr:2023-10-26T01:30:23.727+0000 7f2a20213040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-26T01:30:23.731 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.726+0000 7f312d6d5640 1 --2- 172.21.15.169:0/335637154 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f31281574b0 0x7f31281e3570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:23.731 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.726+0000 7f312d6d5640 1 -- 172.21.15.169:0/335637154 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f312806ad60 msgr2=0x7f31281e3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:23.731 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.726+0000 7f312d6d5640 1 --2- 172.21.15.169:0/335637154 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f312806ad60 0x7f31281e3030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:23.732 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.726+0000 7f312d6d5640 1 -- 172.21.15.169:0/335637154 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f311c00ae30 con 0x7f3128157890 2023-10-26T01:30:23.732 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.727+0000 7f312d6d5640 1 --2- 172.21.15.169:0/335637154 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3128157890 0x7f3128155270 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f312000aef0 tx=0x7f3120006590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:23.732 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.728+0000 7f3125ffb640 1 -- 172.21.15.169:0/335637154 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3120011070 con 0x7f3128157890 2023-10-26T01:30:23.732 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.728+0000 7f3125ffb640 1 -- 172.21.15.169:0/335637154 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3120005050 con 0x7f3128157890 2023-10-26T01:30:23.732 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.728+0000 7f3125ffb640 1 -- 172.21.15.169:0/335637154 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f312001a680 con 0x7f3128157890 2023-10-26T01:30:23.732 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.728+0000 7f312f939640 1 -- 172.21.15.169:0/335637154 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31280af840 con 0x7f3128157890 2023-10-26T01:30:23.733 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.728+0000 7f312f939640 1 -- 172.21.15.169:0/335637154 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3128097e60 con 0x7f3128157890 2023-10-26T01:30:23.733 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.730+0000 7f312f939640 1 -- 172.21.15.169:0/335637154 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3108005250 con 0x7f3128157890 2023-10-26T01:30:23.736 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.735+0000 7f3125ffb640 1 -- 172.21.15.169:0/335637154 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f3120017020 con 0x7f3128157890 2023-10-26T01:30:23.736 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.735+0000 7f3125ffb640 1 -- 172.21.15.169:0/335637154 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f3120019980 con 0x7f3128157890 2023-10-26T01:30:23.739 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.738+0000 7f3125ffb640 1 -- 172.21.15.169:0/335637154 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f312001f060 con 0x7f3128157890 2023-10-26T01:30:23.753 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.751+0000 7f33a71ae640 1 -- 172.21.15.169:0/2938495810 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33980dedf0 msgr2=0x7f3398052d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:23.753 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.751+0000 7f33a71ae640 1 --2- 172.21.15.169:0/2938495810 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33980dedf0 0x7f3398052d20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f339804efe0 tx=0x7f339c02be40 comp rx=0 tx=0).stop 2023-10-26T01:30:23.757 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.752+0000 7f33a71ae640 1 -- 172.21.15.169:0/2938495810 shutdown_connections 2023-10-26T01:30:23.757 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.752+0000 7f33a71ae640 1 --2- 172.21.15.169:0/2938495810 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33980dedf0 0x7f3398052d20 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:23.757 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.752+0000 7f33a71ae640 1 --2- 172.21.15.169:0/2938495810 >> v2:172.21.15.169:3300/0 conn(0x7f33980e6f80 0x7f339804f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:23.758 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.752+0000 7f33a71ae640 1 -- 172.21.15.169:0/2938495810 >> 172.21.15.169:0/2938495810 conn(0x7f3398009f80 msgr2=0x7f339800a360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:23.758 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.752+0000 7f33a71ae640 1 -- 172.21.15.169:0/2938495810 shutdown_connections 2023-10-26T01:30:23.758 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.752+0000 7f33a71ae640 1 -- 172.21.15.169:0/2938495810 wait complete. 2023-10-26T01:30:23.758 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.755+0000 7f33a71ae640 1 Processor -- start 2023-10-26T01:30:23.760 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.755+0000 7f33a71ae640 1 -- start start 2023-10-26T01:30:23.761 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.757+0000 7f33a71ae640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33980dedf0 0x7f339804e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:23.761 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.757+0000 7f33a71ae640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f33980e6f80 0x7f33980d8440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:23.761 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.757+0000 7f33a71ae640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33980e7360 0x7f339804c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:23.767 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.758+0000 7f33a69ad640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33980e7360 0x7f339804c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:23.767 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.758+0000 7f33a69ad640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33980e7360 0x7f339804c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:55466/0 (socket says 172.21.15.169:55466) 2023-10-26T01:30:23.767 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.759+0000 7f33a61ac640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33980dedf0 0x7f339804e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:23.768 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.759+0000 7f33a59ab640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f33980e6f80 0x7f33980d8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:23.768 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.759+0000 7f33a59ab640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f33980e6f80 0x7f33980d8440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3301/0 says I am v2:172.21.15.169:48726/0 (socket says 172.21.15.169:48726) 2023-10-26T01:30:23.769 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.759+0000 7f33a61ac640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33980dedf0 0x7f339804e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.169:55636/0 (socket says 172.21.15.169:55636) 2023-10-26T01:30:23.769 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.759+0000 7f33a69ad640 1 -- 172.21.15.169:0/3662169647 learned_addr learned my addr 172.21.15.169:0/3662169647 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:30:23.769 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.759+0000 7f33a71ae640 1 -- 172.21.15.169:0/3662169647 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33980d8980 con 0x7f33980e7360 2023-10-26T01:30:23.770 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.760+0000 7f33a71ae640 1 -- 172.21.15.169:0/3662169647 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33980d7fb0 con 0x7f33980dedf0 2023-10-26T01:30:23.770 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.760+0000 7f33a71ae640 1 -- 172.21.15.169:0/3662169647 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33980397b0 con 0x7f33980e6f80 2023-10-26T01:30:23.770 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.761+0000 7f33a69ad640 1 -- 172.21.15.169:0/3662169647 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f33980e6f80 msgr2=0x7f33980d8440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:23.771 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.761+0000 7f33a69ad640 1 --2- 172.21.15.169:0/3662169647 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f33980e6f80 0x7f33980d8440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:23.771 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.761+0000 7f33a69ad640 1 -- 172.21.15.169:0/3662169647 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33980dedf0 msgr2=0x7f339804e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:23.771 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.761+0000 7f33a69ad640 1 --2- 172.21.15.169:0/3662169647 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33980dedf0 0x7f339804e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:23.773 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.761+0000 7f33a69ad640 1 -- 172.21.15.169:0/3662169647 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f339c009560 con 0x7f33980e7360 2023-10-26T01:30:23.773 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.761+0000 7f33a59ab640 1 --2- 172.21.15.169:0/3662169647 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f33980e6f80 0x7f33980d8440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T01:30:23.774 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.762+0000 7f33a61ac640 1 --2- 172.21.15.169:0/3662169647 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33980dedf0 0x7f339804e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:30:23.774 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.762+0000 7f33a69ad640 1 --2- 172.21.15.169:0/3662169647 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33980e7360 0x7f339804c470 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f339c036040 tx=0x7f339c02dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:23.774 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.762+0000 7f3396ffd640 1 -- 172.21.15.169:0/3662169647 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f339c0035f0 con 0x7f33980e7360 2023-10-26T01:30:23.775 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.762+0000 7f3396ffd640 1 -- 172.21.15.169:0/3662169647 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f339c003750 con 0x7f33980e7360 2023-10-26T01:30:23.775 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.762+0000 7f3396ffd640 1 -- 172.21.15.169:0/3662169647 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f339c003b20 con 0x7f33980e7360 2023-10-26T01:30:23.776 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.764+0000 7fd66ad16640 1 -- 172.21.15.169:0/831209740 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command([{prefix=config set, name=debug_mds_balancer}] v 0) v1 -- 0x7fd6440055c0 con 0x7fd664096ad0 2023-10-26T01:30:23.777 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.764+0000 7f33a71ae640 1 -- 172.21.15.169:0/3662169647 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33980d79a0 con 0x7f33980e7360 2023-10-26T01:30:23.778 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.764+0000 7f33a71ae640 1 -- 172.21.15.169:0/3662169647 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f339804bfe0 con 0x7f33980e7360 2023-10-26T01:30:23.778 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.766+0000 7f3396ffd640 1 -- 172.21.15.169:0/3662169647 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f339c03e450 con 0x7f33980e7360 2023-10-26T01:30:23.778 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.767+0000 7f3396ffd640 1 -- 172.21.15.169:0/3662169647 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f339c034510 con 0x7f33980e7360 2023-10-26T01:30:23.779 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.768+0000 7f33a71ae640 1 -- 172.21.15.169:0/3662169647 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33980089a0 con 0x7f33980e7360 2023-10-26T01:30:23.781 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.774+0000 7f1fcb1ce640 1 -- 172.21.15.169:0/794692278 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1fc415fa40 msgr2=0x7f1fc40b26e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:23.781 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.774+0000 7f1fcb1ce640 1 --2- 172.21.15.169:0/794692278 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1fc415fa40 0x7f1fc40b26e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f1fc406ba00 tx=0x7f1fb402bdb0 comp rx=0 tx=0).stop 2023-10-26T01:30:23.782 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.774+0000 7f1fcb1ce640 1 -- 172.21.15.169:0/794692278 shutdown_connections 2023-10-26T01:30:23.782 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.774+0000 7f1fcb1ce640 1 --2- 172.21.15.169:0/794692278 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1fc40a34b0 0x7f1fc40b3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:23.782 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.774+0000 7f1fcb1ce640 1 --2- 172.21.15.169:0/794692278 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1fc415fa40 0x7f1fc40b26e0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:23.783 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.774+0000 7f1fcb1ce640 1 --2- 172.21.15.169:0/794692278 >> v2:172.21.15.170:3300/0 conn(0x7f1fc40d5d00 0x7f1fc40d60e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:23.783 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.775+0000 7f1fcb1ce640 1 -- 172.21.15.169:0/794692278 >> 172.21.15.169:0/794692278 conn(0x7f1fc4069930 msgr2=0x7f1fc4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:23.783 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.775+0000 7f1fcb1ce640 1 -- 172.21.15.169:0/794692278 shutdown_connections 2023-10-26T01:30:23.784 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.775+0000 7f1fcb1ce640 1 -- 172.21.15.169:0/794692278 wait complete. 2023-10-26T01:30:23.784 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.775+0000 7f3396ffd640 1 -- 172.21.15.169:0/3662169647 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f339c042020 con 0x7f33980e7360 2023-10-26T01:30:23.785 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.778+0000 7f1fcb1ce640 1 Processor -- start 2023-10-26T01:30:23.786 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.778+0000 7f1fcb1ce640 1 -- start start 2023-10-26T01:30:23.786 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.780+0000 7fc3c3b4c640 1 -- 172.21.15.169:0/1707349473 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc3bc0b0830 msgr2=0x7fc3bc0cefa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:23.787 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.780+0000 7fc3c3b4c640 1 --2- 172.21.15.169:0/1707349473 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc3bc0b0830 0x7fc3bc0cefa0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc08a750 tx=0x7fc3ac02bd30 comp rx=0 tx=0).stop 2023-10-26T01:30:23.787 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.780+0000 7fd6617fa640 1 -- 172.21.15.169:0/831209740 <== mon.1 v2:172.21.15.170:3300/0 9 ==== mon_command_ack([{prefix=config set, name=debug_mds_balancer}]=0 v9)=0 v9) v1 ==== 117+0+0 (secure 0 0 0) 0x7fd65401c840 con 0x7fd664096ad0 2023-10-26T01:30:23.788 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.780+0000 7fc3c3b4c640 1 -- 172.21.15.169:0/1707349473 shutdown_connections 2023-10-26T01:30:23.789 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.780+0000 7fc3c3b4c640 1 --2- 172.21.15.169:0/1707349473 >> v2:172.21.15.169:3300/0 conn(0x7fc3bc0d0db0 0x7fc3bc065490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:23.789 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.780+0000 7fc3c3b4c640 1 --2- 172.21.15.169:0/1707349473 >> v2:172.21.15.170:3300/0 conn(0x7fc3bc0cbcc0 0x7fc3bc0d0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:23.789 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.780+0000 7fc3c3b4c640 1 --2- 172.21.15.169:0/1707349473 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc3bc0b0830 0x7fc3bc0cefa0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:23.790 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.780+0000 7fc3c3b4c640 1 -- 172.21.15.169:0/1707349473 >> 172.21.15.169:0/1707349473 conn(0x7fc3bc0d13d0 msgr2=0x7fc3bc0d17e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:23.790 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.780+0000 7fc3c3b4c640 1 -- 172.21.15.169:0/1707349473 shutdown_connections 2023-10-26T01:30:23.790 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.781+0000 7fc3c3b4c640 1 -- 172.21.15.169:0/1707349473 wait complete. 2023-10-26T01:30:23.791 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.783+0000 7f1fcb1ce640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1fc40a34b0 0x7f1fc414fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:23.792 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.784+0000 7f1fcb1ce640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1fc40d5d00 0x7f1fc4161bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:23.792 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.785+0000 7f1fcb1ce640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1fc415fa40 0x7f1fc40a1270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:23.792 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.785+0000 7f1fcb1ce640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc4162d50 con 0x7f1fc40a34b0 2023-10-26T01:30:23.793 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.786+0000 7fd66ad16640 1 -- 172.21.15.169:0/831209740 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd664096ad0 msgr2=0x7fd6641f5f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:23.793 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.786+0000 7fd66ad16640 1 --2- 172.21.15.169:0/831209740 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd664096ad0 0x7fd6641f5f80 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fd6640ce330 tx=0x7fd654009800 comp rx=0 tx=0).stop 2023-10-26T01:30:23.794 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.785+0000 7f1fc976b640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1fc415fa40 0x7f1fc40a1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:23.794 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.786+0000 7f1fc976b640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1fc415fa40 0x7f1fc40a1270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3301/0 says I am v2:172.21.15.169:48746/0 (socket says 172.21.15.169:48746) 2023-10-26T01:30:23.794 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.786+0000 7f1fc976b640 1 -- 172.21.15.169:0/2117565570 learned_addr learned my addr 172.21.15.169:0/2117565570 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:30:23.795 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.786+0000 7fd66ad16640 1 -- 172.21.15.169:0/831209740 shutdown_connections 2023-10-26T01:30:23.796 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.786+0000 7fd66ad16640 1 --2- 172.21.15.169:0/831209740 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd664096ad0 0x7fd6641f5f80 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:23.796 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.786+0000 7fd66ad16640 1 --2- 172.21.15.169:0/831209740 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd6640cc960 0x7fd664096590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:23.796 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.786+0000 7fd66ad16640 1 --2- 172.21.15.169:0/831209740 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd64c003970 0x7fd66414fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:23.797 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.786+0000 7fd66ad16640 1 -- 172.21.15.169:0/831209740 >> 172.21.15.169:0/831209740 conn(0x7fd6640b2800 msgr2=0x7fd664157e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:23.797 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.786+0000 7fd66ad16640 1 -- 172.21.15.169:0/831209740 shutdown_connections 2023-10-26T01:30:23.797 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.786+0000 7fd66ad16640 1 -- 172.21.15.169:0/831209740 wait complete. 2023-10-26T01:30:23.798 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.787+0000 7f1fc3fff640 1 --2- 172.21.15.169:0/2117565570 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1fc40d5d00 0x7f1fc4161bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:23.798 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.788+0000 7f1fc8f6a640 1 --2- 172.21.15.169:0/2117565570 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1fc40a34b0 0x7f1fc414fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:23.799 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.788+0000 7f1fcb1ce640 1 -- 172.21.15.169:0/2117565570 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc4152030 con 0x7f1fc40d5d00 2023-10-26T01:30:23.799 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.787+0000 7fc3c3b4c640 1 Processor -- start 2023-10-26T01:30:23.800 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.788+0000 7f1fcb1ce640 1 -- 172.21.15.169:0/2117565570 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc4152310 con 0x7f1fc415fa40 2023-10-26T01:30:23.800 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.790+0000 7fc3c3b4c640 1 -- start start 2023-10-26T01:30:23.801 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.789+0000 7f1fc976b640 1 -- 172.21.15.169:0/2117565570 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1fc40d5d00 msgr2=0x7f1fc4161bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:23.802 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.792+0000 7fc36c8c2640 1 -- 172.21.15.169:0/878119089 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc3680ccfb0 msgr2=0x7fc3680cd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:23.802 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.789+0000 7f1fc976b640 1 --2- 172.21.15.169:0/2117565570 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1fc40d5d00 0x7f1fc4161bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:23.803 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.792+0000 7fc36c8c2640 1 --2- 172.21.15.169:0/878119089 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc3680ccfb0 0x7fc3680cd3f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc35c005fd0 tx=0x7fc35c02bda0 comp rx=0 tx=0).stop 2023-10-26T01:30:23.803 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.789+0000 7f1fc976b640 1 -- 172.21.15.169:0/2117565570 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1fc40a34b0 msgr2=0x7f1fc414fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:23.804 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.789+0000 7f1fc976b640 1 --2- 172.21.15.169:0/2117565570 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1fc40a34b0 0x7f1fc414fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:23.804 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.789+0000 7f1fc976b640 1 -- 172.21.15.169:0/2117565570 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fb8009490 con 0x7f1fc415fa40 2023-10-26T01:30:23.804 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.790+0000 7f1fc976b640 1 --2- 172.21.15.169:0/2117565570 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1fc415fa40 0x7f1fc40a1270 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f1fb800e9b0 tx=0x7f1fb800ee70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:23.805 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.790+0000 7f1fc17fa640 1 -- 172.21.15.169:0/2117565570 <== mon.2 v2:172.21.15.170:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb800cd80 con 0x7f1fc415fa40 2023-10-26T01:30:23.805 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.791+0000 7f1fc17fa640 1 -- 172.21.15.169:0/2117565570 <== mon.2 v2:172.21.15.170:3301/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1fb8004020 con 0x7f1fc415fa40 2023-10-26T01:30:23.805 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.791+0000 7f1fc17fa640 1 -- 172.21.15.169:0/2117565570 <== mon.2 v2:172.21.15.170:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb800cd80 con 0x7f1fc415fa40 2023-10-26T01:30:23.806 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.791+0000 7f1fcb1ce640 1 -- 172.21.15.169:0/2117565570 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fb4009560 con 0x7f1fc415fa40 2023-10-26T01:30:23.806 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.792+0000 7f1fcb1ce640 1 -- 172.21.15.169:0/2117565570 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fc41528d0 con 0x7f1fc415fa40 2023-10-26T01:30:23.807 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.792+0000 7fc36c8c2640 1 -- 172.21.15.169:0/878119089 shutdown_connections 2023-10-26T01:30:23.807 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.792+0000 7fc36c8c2640 1 --2- 172.21.15.169:0/878119089 >> v2:172.21.15.169:3300/0 conn(0x7fc3681704e0 0x7fc368170970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:23.807 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.792+0000 7f1fc17fa640 1 -- 172.21.15.169:0/2117565570 <== mon.2 v2:172.21.15.170:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f1fb8004190 con 0x7f1fc415fa40 2023-10-26T01:30:23.808 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.793+0000 7fc36c8c2640 1 --2- 172.21.15.169:0/878119089 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc3680ccfb0 0x7fc3680cd3f0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:23.809 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.793+0000 7fc36c8c2640 1 --2- 172.21.15.169:0/878119089 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc3680acb60 0x7fc368092f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:23.809 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.792+0000 7f1fc17fa640 1 -- 172.21.15.169:0/2117565570 <== mon.2 v2:172.21.15.170:3301/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f1fb8014070 con 0x7f1fc415fa40 2023-10-26T01:30:23.810 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.793+0000 7fc36c8c2640 1 -- 172.21.15.169:0/878119089 >> 172.21.15.169:0/878119089 conn(0x7fc3680b2800 msgr2=0x7fc3680b2be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:23.810 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.792+0000 7fc3c3b4c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc3bc0b0830 0x7fc3bc1584e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:23.811 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.792+0000 7fc3c3b4c640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc3bc0cbcc0 0x7fc3bc1f9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:23.811 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.792+0000 7fc3c3b4c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3bc0d0db0 0x7fc3bc154850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:23.811 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.792+0000 7fc3c3b4c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3bc0c87f0 con 0x7fc3bc0d0db0 2023-10-26T01:30:23.811 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.792+0000 7fc3c3b4c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3bc0c84a0 con 0x7fc3bc0b0830 2023-10-26T01:30:23.812 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.792+0000 7fc3c3b4c640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3bc1575e0 con 0x7fc3bc0cbcc0 2023-10-26T01:30:23.813 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.793+0000 7fc36c8c2640 1 -- 172.21.15.169:0/878119089 shutdown_connections 2023-10-26T01:30:23.813 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.793+0000 7fc36c8c2640 1 -- 172.21.15.169:0/878119089 wait complete. 2023-10-26T01:30:23.813 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.793+0000 7f1fcb1ce640 1 -- 172.21.15.169:0/2117565570 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fc406ea90 con 0x7f1fc415fa40 2023-10-26T01:30:23.815 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.794+0000 7fc3c10e7640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc3bc0cbcc0 0x7fc3bc1f9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:23.816 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.794+0000 7fc3c10e7640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc3bc0cbcc0 0x7fc3bc1f9ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3301/0 says I am v2:172.21.15.169:48766/0 (socket says 172.21.15.169:48766) 2023-10-26T01:30:23.816 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.794+0000 7fc3c10e7640 1 -- 172.21.15.169:0/765466099 learned_addr learned my addr 172.21.15.169:0/765466099 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:30:23.816 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.794+0000 7fc3c20e9640 1 --2- 172.21.15.169:0/765466099 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3bc0d0db0 0x7fc3bc154850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:23.817 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.794+0000 7fc3c18e8640 1 --2- 172.21.15.169:0/765466099 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc3bc0b0830 0x7fc3bc1584e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:23.817 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.794+0000 7fc3c10e7640 1 -- 172.21.15.169:0/765466099 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc3bc0b0830 msgr2=0x7fc3bc1584e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:23.817 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.794+0000 7fc3c10e7640 1 --2- 172.21.15.169:0/765466099 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc3bc0b0830 0x7fc3bc1584e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:23.817 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.794+0000 7fc3c10e7640 1 -- 172.21.15.169:0/765466099 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3bc0d0db0 msgr2=0x7fc3bc154850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:23.818 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.795+0000 7fc3c10e7640 1 --2- 172.21.15.169:0/765466099 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3bc0d0db0 0x7fc3bc154850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:23.818 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.795+0000 7fc3c10e7640 1 -- 172.21.15.169:0/765466099 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3ac009560 con 0x7fc3bc0cbcc0 2023-10-26T01:30:23.818 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.795+0000 7fc3c20e9640 1 --2- 172.21.15.169:0/765466099 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3bc0d0db0 0x7fc3bc154850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T01:30:23.819 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.795+0000 7fc3c10e7640 1 --2- 172.21.15.169:0/765466099 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc3bc0cbcc0 0x7fc3bc1f9ed0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc3b400ef30 tx=0x7fc3b400c550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:23.821 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.796+0000 7fc36c8c2640 1 Processor -- start 2023-10-26T01:30:23.821 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.796+0000 7fc36c8c2640 1 -- start start 2023-10-26T01:30:23.822 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.797+0000 7fc36c8c2640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc3680ccfb0 0x7fc36816c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:23.822 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.797+0000 7fc36c8c2640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc3681704e0 0x7fc368202ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:23.822 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.797+0000 7fc36c8c2640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3680cc870 0x7fc3680ade40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:23.822 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.797+0000 7fc36c8c2640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc36816e110 con 0x7fc3680cc870 2023-10-26T01:30:23.823 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.797+0000 7fc36c8c2640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc36816cdb0 con 0x7fc3681704e0 2023-10-26T01:30:23.823 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.797+0000 7fc36c8c2640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc368203360 con 0x7fc3680ccfb0 2023-10-26T01:30:23.823 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.798+0000 7fc36659c640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc3680ccfb0 0x7fc36816c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:23.823 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.798+0000 7fc366d9d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3680cc870 0x7fc3680ade40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:23.823 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.798+0000 7fc366d9d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3680cc870 0x7fc3680ade40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:55516/0 (socket says 172.21.15.169:55516) 2023-10-26T01:30:23.824 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.798+0000 7fc366d9d640 1 -- 172.21.15.169:0/3036217050 learned_addr learned my addr 172.21.15.169:0/3036217050 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:30:23.824 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.798+0000 7fc365d9b640 1 --2- 172.21.15.169:0/3036217050 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc3681704e0 0x7fc368202ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:23.824 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.799+0000 7fc366d9d640 1 -- 172.21.15.169:0/3036217050 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc3680ccfb0 msgr2=0x7fc36816c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:23.824 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.799+0000 7fc366d9d640 1 --2- 172.21.15.169:0/3036217050 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc3680ccfb0 0x7fc36816c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:23.824 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.799+0000 7fc366d9d640 1 -- 172.21.15.169:0/3036217050 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc3681704e0 msgr2=0x7fc368202ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:23.825 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.799+0000 7fc366d9d640 1 --2- 172.21.15.169:0/3036217050 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc3681704e0 0x7fc368202ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:23.825 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.799+0000 7fc366d9d640 1 -- 172.21.15.169:0/3036217050 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc35c009560 con 0x7fc3680cc870 2023-10-26T01:30:23.825 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.799+0000 7fc366d9d640 1 --2- 172.21.15.169:0/3036217050 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3680cc870 0x7fc3680ade40 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc36000ea80 tx=0x7fc36000ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:23.825 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.800+0000 7f1fc17fa640 1 -- 172.21.15.169:0/2117565570 <== mon.2 v2:172.21.15.170:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f1fc406ea90 con 0x7f1fc415fa40 2023-10-26T01:30:23.826 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.800+0000 7fc3b27fc640 1 -- 172.21.15.169:0/765466099 <== mon.2 v2:172.21.15.170:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3b4018070 con 0x7fc3bc0cbcc0 2023-10-26T01:30:23.826 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.800+0000 7fc3b27fc640 1 -- 172.21.15.169:0/765466099 <== mon.2 v2:172.21.15.170:3301/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc3b400f040 con 0x7fc3bc0cbcc0 2023-10-26T01:30:23.826 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.800+0000 7fc3b27fc640 1 -- 172.21.15.169:0/765466099 <== mon.2 v2:172.21.15.170:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3b4009330 con 0x7fc3bc0cbcc0 2023-10-26T01:30:23.827 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.800+0000 7fc356ffd640 1 -- 172.21.15.169:0/3036217050 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc36000ce40 con 0x7fc3680cc870 2023-10-26T01:30:23.827 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.800+0000 7fc356ffd640 1 -- 172.21.15.169:0/3036217050 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc360004b20 con 0x7fc3680cc870 2023-10-26T01:30:23.828 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.805+0000 7fc3c3b4c640 1 -- 172.21.15.169:0/765466099 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3bc1578c0 con 0x7fc3bc0cbcc0 2023-10-26T01:30:23.828 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.800+0000 7fc356ffd640 1 -- 172.21.15.169:0/3036217050 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc360005020 con 0x7fc3680cc870 2023-10-26T01:30:23.828 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.801+0000 7fc36c8c2640 1 -- 172.21.15.169:0/3036217050 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc368203640 con 0x7fc3680cc870 2023-10-26T01:30:23.828 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.801+0000 7fc36c8c2640 1 -- 172.21.15.169:0/3036217050 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc36816dda0 con 0x7fc3680cc870 2023-10-26T01:30:23.829 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.805+0000 7fc356ffd640 1 -- 172.21.15.169:0/3036217050 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fc360005180 con 0x7fc3680cc870 2023-10-26T01:30:23.829 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.805+0000 7fc356ffd640 1 -- 172.21.15.169:0/3036217050 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fc360014070 con 0x7fc3680cc870 2023-10-26T01:30:23.829 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.805+0000 7fc3c3b4c640 1 -- 172.21.15.169:0/765466099 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3bc158a20 con 0x7fc3bc0cbcc0 2023-10-26T01:30:23.830 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.806+0000 7fc354ff9640 1 -- 172.21.15.169:0/3036217050 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc36816e280 con 0x7fc3680cc870 2023-10-26T01:30:23.830 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.806+0000 7fc3b27fc640 1 -- 172.21.15.169:0/765466099 <== mon.2 v2:172.21.15.170:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fc3b40039a0 con 0x7fc3bc0cbcc0 2023-10-26T01:30:23.830 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.807+0000 7fc3c3b4c640 1 -- 172.21.15.169:0/765466099 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3bc000940 con 0x7fc3bc0cbcc0 2023-10-26T01:30:23.831 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.811+0000 7fc3b27fc640 1 -- 172.21.15.169:0/765466099 <== mon.2 v2:172.21.15.170:3301/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fc3b4007500 con 0x7fc3bc0cbcc0 2023-10-26T01:30:23.831 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.811+0000 7fd00fc1a640 1 -- 172.21.15.169:0/1861508610 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd0080acad0 msgr2=0x7fd00815a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:23.831 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.811+0000 7fd00fc1a640 1 --2- 172.21.15.169:0/1861508610 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd0080acad0 0x7fd00815a600 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fd008071ce0 tx=0x7fd00402bda0 comp rx=0 tx=0).stop 2023-10-26T01:30:23.832 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.812+0000 7fc356ffd640 1 -- 172.21.15.169:0/3036217050 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7fc360021ad0 con 0x7fc3680cc870 2023-10-26T01:30:23.833 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.812+0000 7fd00fc1a640 1 -- 172.21.15.169:0/1861508610 shutdown_connections 2023-10-26T01:30:23.833 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.812+0000 7fd00fc1a640 1 --2- 172.21.15.169:0/1861508610 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd0080acad0 0x7fd00815a600 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:23.833 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.812+0000 7fd00fc1a640 1 --2- 172.21.15.169:0/1861508610 >> v2:172.21.15.169:3300/0 conn(0x7fd0080b3710 0x7fd0080ae1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:23.833 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.812+0000 7fd00fc1a640 1 -- 172.21.15.169:0/1861508610 >> 172.21.15.169:0/1861508610 conn(0x7fd008098a60 msgr2=0x7fd0080b4a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:23.834 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.813+0000 7fd00fc1a640 1 -- 172.21.15.169:0/1861508610 shutdown_connections 2023-10-26T01:30:23.834 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.813+0000 7fd00fc1a640 1 -- 172.21.15.169:0/1861508610 wait complete. 2023-10-26T01:30:23.835 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_ms', '1'], check_status=True, hostname='smithi169') 2023-10-26T01:30:23.835 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_debug_frag', 'True'], check_status=True, hostname='smithi169') 2023-10-26T01:30:23.835 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.816+0000 7fd00fc1a640 1 Processor -- start 2023-10-26T01:30:23.836 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.817+0000 7fc3b27fc640 1 -- 172.21.15.169:0/765466099 <== mon.2 v2:172.21.15.170:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7fc3b4004e50 con 0x7fc3bc0cbcc0 2023-10-26T01:30:23.837 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.818+0000 7fd00fc1a640 1 -- start start 2023-10-26T01:30:23.837 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.819+0000 7fd00fc1a640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd0080acad0 0x7fd00813f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:23.837 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.819+0000 7fd00fc1a640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd0080b3710 0x7fd00814e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:23.838 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.819+0000 7fd00fc1a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd0081596c0 0x7fd0080b1230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:23.838 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.819+0000 7fd00fc1a640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd00814f520 con 0x7fd0080acad0 2023-10-26T01:30:23.838 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.819+0000 7fd00fc1a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd00814ef60 con 0x7fd0081596c0 2023-10-26T01:30:23.838 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.819+0000 7fd00fc1a640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd00814f240 con 0x7fd0080b3710 2023-10-26T01:30:23.839 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.822+0000 7fd00e1b7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd0081596c0 0x7fd0080b1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:23.839 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.822+0000 7fd00e1b7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd0081596c0 0x7fd0080b1230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.169:55690/0 (socket says 172.21.15.169:55690) 2023-10-26T01:30:23.839 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.822+0000 7fd00e1b7640 1 -- 172.21.15.169:0/517532296 learned_addr learned my addr 172.21.15.169:0/517532296 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:30:23.839 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.823+0000 7fd00d9b6640 1 --2- 172.21.15.169:0/517532296 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd0080acad0 0x7fd00813f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:23.839 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.823+0000 7fd00e1b7640 1 -- 172.21.15.169:0/517532296 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd0080b3710 msgr2=0x7fd00814e4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-26T01:30:23.840 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.823+0000 7fd00e1b7640 1 --2- 172.21.15.169:0/517532296 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd0080b3710 0x7fd00814e4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:23.840 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.823+0000 7fd00e1b7640 1 -- 172.21.15.169:0/517532296 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd0080acad0 msgr2=0x7fd00813f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:23.840 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.823+0000 7fd00e1b7640 1 --2- 172.21.15.169:0/517532296 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd0080acad0 0x7fd00813f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:23.840 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.823+0000 7fd00e1b7640 1 -- 172.21.15.169:0/517532296 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd004009560 con 0x7fd0081596c0 2023-10-26T01:30:23.840 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.824+0000 7fd00d9b6640 1 --2- 172.21.15.169:0/517532296 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd0080acad0 0x7fd00813f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:30:23.841 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.824+0000 7fd00e1b7640 1 --2- 172.21.15.169:0/517532296 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd0081596c0 0x7fd0080b1230 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd004009850 tx=0x7fd004004260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:23.841 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.824+0000 7fcffe7fc640 1 -- 172.21.15.169:0/517532296 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0040037f0 con 0x7fd0081596c0 2023-10-26T01:30:23.841 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.825+0000 7fcffe7fc640 1 -- 172.21.15.169:0/517532296 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd004003a40 con 0x7fd0081596c0 2023-10-26T01:30:23.841 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.825+0000 7fcffe7fc640 1 -- 172.21.15.169:0/517532296 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd004003e30 con 0x7fd0081596c0 2023-10-26T01:30:23.841 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.827+0000 7fd00fc1a640 1 -- 172.21.15.169:0/517532296 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd008156e30 con 0x7fd0081596c0 2023-10-26T01:30:23.842 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.830+0000 7fd00fc1a640 1 -- 172.21.15.169:0/517532296 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd00814c380 con 0x7fd0081596c0 2023-10-26T01:30:23.842 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.830+0000 7fcffe7fc640 1 -- 172.21.15.169:0/517532296 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fd004049760 con 0x7fd0081596c0 2023-10-26T01:30:23.842 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.830+0000 7fcffe7fc640 1 -- 172.21.15.169:0/517532296 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fd00403f850 con 0x7fd0081596c0 2023-10-26T01:30:23.842 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.831+0000 7fd00fc1a640 1 -- 172.21.15.169:0/517532296 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0080b40b0 con 0x7fd0081596c0 2023-10-26T01:30:23.843 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.839+0000 7fcffe7fc640 1 -- 172.21.15.169:0/517532296 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7fd004039040 con 0x7fd0081596c0 2023-10-26T01:30:23.843 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.842+0000 7fb07d26f640 1 -- 172.21.15.169:0/3200498440 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command([{prefix=config set, name=mds_debug_scatterstat}] v 0) v1 -- 0x7fb0580057d0 con 0x7fb07815ba00 2023-10-26T01:30:23.852 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.850+0000 7fb05f7fe640 1 -- 172.21.15.169:0/3200498440 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_scatterstat}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7fb06c003e50 con 0x7fb07815ba00 2023-10-26T01:30:23.853 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.852+0000 7fb07d26f640 1 -- 172.21.15.169:0/3200498440 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb07815ba00 msgr2=0x7fb0780c56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:23.854 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.852+0000 7fb07d26f640 1 --2- 172.21.15.169:0/3200498440 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb07815ba00 0x7fb0780c56f0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb06c004000 tx=0x7fb06c02d900 comp rx=0 tx=0).stop 2023-10-26T01:30:23.854 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.852+0000 7fb07d26f640 1 -- 172.21.15.169:0/3200498440 shutdown_connections 2023-10-26T01:30:23.855 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.852+0000 7fb07d26f640 1 --2- 172.21.15.169:0/3200498440 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb07815ba00 0x7fb0780c56f0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:23.855 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.852+0000 7fb07d26f640 1 --2- 172.21.15.169:0/3200498440 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0780d20c0 0x7fb0780c51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:23.855 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.852+0000 7fb07d26f640 1 --2- 172.21.15.169:0/3200498440 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb0780d0fc0 0x7fb07815ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:23.855 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.852+0000 7fb07d26f640 1 -- 172.21.15.169:0/3200498440 >> 172.21.15.169:0/3200498440 conn(0x7fb07806b5c0 msgr2=0x7fb0780cc090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:23.855 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.853+0000 7fb07d26f640 1 -- 172.21.15.169:0/3200498440 shutdown_connections 2023-10-26T01:30:23.856 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.853+0000 7fb07d26f640 1 -- 172.21.15.169:0/3200498440 wait complete. 2023-10-26T01:30:23.856 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.854+0000 7f84c2e1f640 1 -- 172.21.15.169:0/3585463299 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_command([{prefix=config set, name=mds_debug_frag}] v 0) v1 -- 0x7f84a00055a0 con 0x7f84bc15b2e0 2023-10-26T01:30:23.877 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.876+0000 7f84b9ffb640 1 -- 172.21.15.169:0/3585463299 <== mon.2 v2:172.21.15.170:3301/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_frag}]=0 v11)=0 v11) v1 ==== 115+0+0 (secure 0 0 0) 0x7f84b4041020 con 0x7f84bc15b2e0 2023-10-26T01:30:23.879 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.877+0000 7f84c2e1f640 1 -- 172.21.15.169:0/3585463299 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f84bc15b2e0 msgr2=0x7f84bc0c8780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:23.879 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.877+0000 7f84c2e1f640 1 --2- 172.21.15.169:0/3585463299 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f84bc15b2e0 0x7f84bc0c8780 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f84b4033000 tx=0x7f84b4003820 comp rx=0 tx=0).stop 2023-10-26T01:30:23.881 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.880+0000 7f84c2e1f640 1 -- 172.21.15.169:0/3585463299 shutdown_connections 2023-10-26T01:30:23.882 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.880+0000 7f84c2e1f640 1 --2- 172.21.15.169:0/3585463299 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84bc0b3130 0x7f84bc0b4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:23.882 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.880+0000 7f84c2e1f640 1 --2- 172.21.15.169:0/3585463299 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f84bc15b2e0 0x7f84bc0c8780 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:23.882 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.880+0000 7f84c2e1f640 1 --2- 172.21.15.169:0/3585463299 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f84bc0b0710 0x7f84bc15e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:23.883 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.880+0000 7f84c2e1f640 1 -- 172.21.15.169:0/3585463299 >> 172.21.15.169:0/3585463299 conn(0x7f84bc0ce130 msgr2=0x7f84bc069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:23.883 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.880+0000 7f84c2e1f640 1 -- 172.21.15.169:0/3585463299 shutdown_connections 2023-10-26T01:30:23.884 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.880+0000 7f84c2e1f640 1 -- 172.21.15.169:0/3585463299 wait complete. 2023-10-26T01:30:23.901 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.900+0000 7efd567a5640 1 -- 172.21.15.169:0/2880188434 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command([{prefix=config set, name=mds_op_complaint_time}] v 0) v1 -- 0x7efd300050c0 con 0x7efd500b4770 2023-10-26T01:30:23.903 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.901+0000 7f312f939640 1 -- 172.21.15.169:0/335637154 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command([{prefix=config set, name=mds_verify_scatter}] v 0) v1 -- 0x7f31080050c0 con 0x7f3128157890 2023-10-26T01:30:23.909 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.908+0000 7efd45ffb640 1 -- 172.21.15.169:0/2880188434 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_op_complaint_time}]=0 v12)=0 v12) v1 ==== 121+0+0 (secure 0 0 0) 0x7efd4802a0c0 con 0x7efd500b4770 2023-10-26T01:30:23.911 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.910+0000 7efd567a5640 1 -- 172.21.15.169:0/2880188434 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd500b4770 msgr2=0x7efd501f21e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:23.911 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.910+0000 7efd567a5640 1 --2- 172.21.15.169:0/2880188434 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd500b4770 0x7efd501f21e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7efd4800cbc0 tx=0x7efd4800a430 comp rx=0 tx=0).stop 2023-10-26T01:30:23.912 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.910+0000 7efd567a5640 1 -- 172.21.15.169:0/2880188434 shutdown_connections 2023-10-26T01:30:23.913 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.910+0000 7efd567a5640 1 --2- 172.21.15.169:0/2880188434 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd500cc8e0 0x7efd500c6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:23.913 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.910+0000 7efd567a5640 1 --2- 172.21.15.169:0/2880188434 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7efd5015ba30 0x7efd500c9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:23.913 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.910+0000 7efd567a5640 1 --2- 172.21.15.169:0/2880188434 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd500b4770 0x7efd501f21e0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:23.913 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.910+0000 7efd567a5640 1 -- 172.21.15.169:0/2880188434 >> 172.21.15.169:0/2880188434 conn(0x7efd500d1360 msgr2=0x7efd5015e780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:23.913 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.911+0000 7efd567a5640 1 -- 172.21.15.169:0/2880188434 shutdown_connections 2023-10-26T01:30:23.914 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.911+0000 7f3125ffb640 1 -- 172.21.15.169:0/335637154 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_verify_scatter}]=0 v13)=0 v13) v1 ==== 119+0+0 (secure 0 0 0) 0x7f3120006e20 con 0x7f3128157890 2023-10-26T01:30:23.914 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.911+0000 7efd567a5640 1 -- 172.21.15.169:0/2880188434 wait complete. 2023-10-26T01:30:23.915 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.913+0000 7f312f939640 1 -- 172.21.15.169:0/335637154 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3128157890 msgr2=0x7f3128155270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:23.915 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.913+0000 7f312f939640 1 --2- 172.21.15.169:0/335637154 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3128157890 0x7f3128155270 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f312000aef0 tx=0x7f3120006590 comp rx=0 tx=0).stop 2023-10-26T01:30:23.915 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.913+0000 7f312f939640 1 -- 172.21.15.169:0/335637154 shutdown_connections 2023-10-26T01:30:23.916 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.913+0000 7f312f939640 1 --2- 172.21.15.169:0/335637154 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3128157890 0x7f3128155270 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:23.916 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.913+0000 7f312f939640 1 --2- 172.21.15.169:0/335637154 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f31281574b0 0x7f31281e3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:23.916 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.913+0000 7f312f939640 1 --2- 172.21.15.169:0/335637154 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f312806ad60 0x7f31281e3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:23.916 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.913+0000 7f312f939640 1 -- 172.21.15.169:0/335637154 >> 172.21.15.169:0/335637154 conn(0x7f312806f200 msgr2=0x7f3128157e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:23.917 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_debug_scatterstat', 'True'], check_status=True, hostname='smithi169') 2023-10-26T01:30:23.917 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_op_complaint_time', '180'], check_status=True, hostname='smithi169') 2023-10-26T01:30:23.918 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.914+0000 7f312f939640 1 -- 172.21.15.169:0/335637154 shutdown_connections 2023-10-26T01:30:23.918 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.914+0000 7f312f939640 1 -- 172.21.15.169:0/335637154 wait complete. 2023-10-26T01:30:23.937 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_verify_scatter', 'True'], check_status=True, hostname='smithi169') 2023-10-26T01:30:23.942 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.940+0000 7f33a71ae640 1 -- 172.21.15.169:0/3662169647 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command([{prefix=config set, name=mon_op_complaint_time}] v 0) v1 -- 0x7f3398005360 con 0x7f33980e7360 2023-10-26T01:30:23.943 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi169') 2023-10-26T01:30:23.945 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.944+0000 7f3396ffd640 1 -- 172.21.15.169:0/3662169647 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_op_complaint_time}]=0 v14)=0 v14) v1 ==== 121+0+0 (secure 0 0 0) 0x7f339c03ea20 con 0x7f33980e7360 2023-10-26T01:30:23.946 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.945+0000 7f1fcb1ce640 1 -- 172.21.15.169:0/2117565570 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) v1 -- 0x7f1fc4073c10 con 0x7f1fc415fa40 2023-10-26T01:30:23.947 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.945+0000 7f33a71ae640 1 -- 172.21.15.169:0/3662169647 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33980e7360 msgr2=0x7f339804c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:23.947 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.945+0000 7f33a71ae640 1 --2- 172.21.15.169:0/3662169647 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33980e7360 0x7f339804c470 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f339c036040 tx=0x7f339c02dd90 comp rx=0 tx=0).stop 2023-10-26T01:30:23.948 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.946+0000 7f33a71ae640 1 -- 172.21.15.169:0/3662169647 shutdown_connections 2023-10-26T01:30:23.948 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.946+0000 7f33a71ae640 1 --2- 172.21.15.169:0/3662169647 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33980e7360 0x7f339804c470 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:23.948 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.946+0000 7f33a71ae640 1 --2- 172.21.15.169:0/3662169647 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f33980e6f80 0x7f33980d8440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:23.948 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.946+0000 7f33a71ae640 1 --2- 172.21.15.169:0/3662169647 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33980dedf0 0x7f339804e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:23.948 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.946+0000 7f33a71ae640 1 -- 172.21.15.169:0/3662169647 >> 172.21.15.169:0/3662169647 conn(0x7f3398009f80 msgr2=0x7f33980344d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:23.949 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.946+0000 7f33a71ae640 1 -- 172.21.15.169:0/3662169647 shutdown_connections 2023-10-26T01:30:23.949 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.946+0000 7f33a71ae640 1 -- 172.21.15.169:0/3662169647 wait complete. 2023-10-26T01:30:23.954 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.953+0000 7f1fc17fa640 1 -- 172.21.15.169:0/2117565570 <== mon.2 v2:172.21.15.170:3301/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v15)=0 v15) v1 ==== 121+0+0 (secure 0 0 0) 0x7f1fb8004c00 con 0x7f1fc415fa40 2023-10-26T01:30:23.956 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.954+0000 7fc354ff9640 1 -- 172.21.15.169:0/3036217050 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) v1 -- 0x7fc36804db10 con 0x7fc3680cc870 2023-10-26T01:30:23.957 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.955+0000 7f1fcb1ce640 1 -- 172.21.15.169:0/2117565570 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1fc415fa40 msgr2=0x7f1fc40a1270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:23.957 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.955+0000 7f1fcb1ce640 1 --2- 172.21.15.169:0/2117565570 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1fc415fa40 0x7f1fc40a1270 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f1fb800e9b0 tx=0x7f1fb800ee70 comp rx=0 tx=0).stop 2023-10-26T01:30:23.958 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.955+0000 7f1fcb1ce640 1 -- 172.21.15.169:0/2117565570 shutdown_connections 2023-10-26T01:30:23.958 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.955+0000 7f1fcb1ce640 1 --2- 172.21.15.169:0/2117565570 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1fc415fa40 0x7f1fc40a1270 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:23.959 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.955+0000 7f1fcb1ce640 1 --2- 172.21.15.169:0/2117565570 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1fc40d5d00 0x7f1fc4161bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:23.959 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.955+0000 7f1fcb1ce640 1 --2- 172.21.15.169:0/2117565570 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1fc40a34b0 0x7f1fc414fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:23.959 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.955+0000 7f1fcb1ce640 1 -- 172.21.15.169:0/2117565570 >> 172.21.15.169:0/2117565570 conn(0x7f1fc4069930 msgr2=0x7f1fc40b5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:23.960 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.955+0000 7f1fcb1ce640 1 -- 172.21.15.169:0/2117565570 shutdown_connections 2023-10-26T01:30:23.960 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.955+0000 7f1fcb1ce640 1 -- 172.21.15.169:0/2117565570 wait complete. 2023-10-26T01:30:23.961 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.958+0000 7fc356ffd640 1 -- 172.21.15.169:0/3036217050 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v16)=0 v16) v1 ==== 120+0+0 (secure 0 0 0) 0x7fc360021c70 con 0x7fc3680cc870 2023-10-26T01:30:23.962 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.960+0000 7fc354ff9640 1 -- 172.21.15.169:0/3036217050 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3680cc870 msgr2=0x7fc3680ade40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:23.962 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.960+0000 7fc354ff9640 1 --2- 172.21.15.169:0/3036217050 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3680cc870 0x7fc3680ade40 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc36000ea80 tx=0x7fc36000ef40 comp rx=0 tx=0).stop 2023-10-26T01:30:23.962 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.960+0000 7fc354ff9640 1 -- 172.21.15.169:0/3036217050 shutdown_connections 2023-10-26T01:30:23.963 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.960+0000 7fc354ff9640 1 --2- 172.21.15.169:0/3036217050 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3680cc870 0x7fc3680ade40 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:23.963 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.960+0000 7fc354ff9640 1 --2- 172.21.15.169:0/3036217050 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc3681704e0 0x7fc368202ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:23.963 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.960+0000 7fc354ff9640 1 --2- 172.21.15.169:0/3036217050 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc3680ccfb0 0x7fc36816c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:23.963 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.960+0000 7fc354ff9640 1 -- 172.21.15.169:0/3036217050 >> 172.21.15.169:0/3036217050 conn(0x7fc3680b2800 msgr2=0x7fc36808fa70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:23.963 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.960+0000 7fc354ff9640 1 -- 172.21.15.169:0/3036217050 shutdown_connections 2023-10-26T01:30:23.963 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.960+0000 7fc354ff9640 1 -- 172.21.15.169:0/3036217050 wait complete. 2023-10-26T01:30:23.975 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi169') 2023-10-26T01:30:23.978 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi169') 2023-10-26T01:30:23.995 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.994+0000 7fd00fc1a640 1 -- 172.21.15.169:0/517532296 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) v1 -- 0x7fd0080b40b0 con 0x7fd0081596c0 2023-10-26T01:30:23.997 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:23.996+0000 7fc3c3b4c640 1 -- 172.21.15.169:0/765466099 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) v1 -- 0x7fc3bc0d2030 con 0x7fc3bc0cbcc0 2023-10-26T01:30:24.002 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:24.000+0000 7fcffe7fc640 1 -- 172.21.15.169:0/517532296 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v17)=0 v17) v1 ==== 121+0+0 (secure 0 0 0) 0x7fd00815ab40 con 0x7fd0081596c0 2023-10-26T01:30:24.003 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:24.001+0000 7fd00fc1a640 1 -- 172.21.15.169:0/517532296 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd0081596c0 msgr2=0x7fd0080b1230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:24.003 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:24.001+0000 7fd00fc1a640 1 --2- 172.21.15.169:0/517532296 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd0081596c0 0x7fd0080b1230 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd004009850 tx=0x7fd004004260 comp rx=0 tx=0).stop 2023-10-26T01:30:24.004 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:24.002+0000 7fd00fc1a640 1 -- 172.21.15.169:0/517532296 shutdown_connections 2023-10-26T01:30:24.004 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:24.002+0000 7fd00fc1a640 1 --2- 172.21.15.169:0/517532296 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd0081596c0 0x7fd0080b1230 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:24.004 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:24.002+0000 7fd00fc1a640 1 --2- 172.21.15.169:0/517532296 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd0080b3710 0x7fd00814e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:24.004 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:24.002+0000 7fd00fc1a640 1 --2- 172.21.15.169:0/517532296 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd0080acad0 0x7fd00813f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:24.004 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:24.002+0000 7fd00fc1a640 1 -- 172.21.15.169:0/517532296 >> 172.21.15.169:0/517532296 conn(0x7fd008098a60 msgr2=0x7fd00806d490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:24.005 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:24.002+0000 7fd00fc1a640 1 -- 172.21.15.169:0/517532296 shutdown_connections 2023-10-26T01:30:24.005 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:24.002+0000 7fd00fc1a640 1 -- 172.21.15.169:0/517532296 wait complete. 2023-10-26T01:30:24.005 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:24.004+0000 7fc3b27fc640 1 -- 172.21.15.169:0/765466099 <== mon.2 v2:172.21.15.170:3301/0 7 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v18)=0 v18) v1 ==== 120+0+0 (secure 0 0 0) 0x7fc3b4004020 con 0x7fc3bc0cbcc0 2023-10-26T01:30:24.007 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:24.005+0000 7fc3c3b4c640 1 -- 172.21.15.169:0/765466099 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc3bc0cbcc0 msgr2=0x7fc3bc1f9ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:24.007 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:24.005+0000 7fc3c3b4c640 1 --2- 172.21.15.169:0/765466099 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc3bc0cbcc0 0x7fc3bc1f9ed0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc3b400ef30 tx=0x7fc3b400c550 comp rx=0 tx=0).stop 2023-10-26T01:30:24.008 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:24.005+0000 7fc3c3b4c640 1 -- 172.21.15.169:0/765466099 shutdown_connections 2023-10-26T01:30:24.008 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:24.005+0000 7fc3c3b4c640 1 --2- 172.21.15.169:0/765466099 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3bc0d0db0 0x7fc3bc154850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:24.008 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:24.005+0000 7fc3c3b4c640 1 --2- 172.21.15.169:0/765466099 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc3bc0cbcc0 0x7fc3bc1f9ed0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:24.008 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:24.005+0000 7fc3c3b4c640 1 --2- 172.21.15.169:0/765466099 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc3bc0b0830 0x7fc3bc1584e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:24.008 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:24.005+0000 7fc3c3b4c640 1 -- 172.21.15.169:0/765466099 >> 172.21.15.169:0/765466099 conn(0x7fc3bc0d13d0 msgr2=0x7fc3bc0b28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:24.009 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:24.005+0000 7fc3c3b4c640 1 -- 172.21.15.169:0/765466099 shutdown_connections 2023-10-26T01:30:24.009 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:24.006+0000 7fc3c3b4c640 1 -- 172.21.15.169:0/765466099 wait complete. 2023-10-26T01:30:24.026 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mon', 'mon_op_complaint_time', '120'], check_status=True, hostname='smithi169') 2023-10-26T01:30:24.027 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'osd', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi169') 2023-10-26T01:30:24.027 INFO:tasks.ceph:Setting crush tunables to default 2023-10-26T01:30:24.028 DEBUG:teuthology.orchestra.run.smithi169:> sudo ceph --cluster ceph osd crush tunables default 2023-10-26T01:30:24.033 INFO:tasks.ceph.mgr.y.smithi169.stderr:2023-10-26T01:30:24.032+0000 7f33d344c040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-26T01:30:24.099 INFO:tasks.ceph.mgr.y.smithi169.stderr:2023-10-26T01:30:24.097+0000 7f33d344c040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-26T01:30:24.213 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:24.211+0000 7fbf1cc32640 1 -- 172.21.15.169:0/3009635537 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbf1815f070 msgr2=0x7fbf180ca3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:24.213 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:24.211+0000 7fbf1cc32640 1 --2- 172.21.15.169:0/3009635537 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbf1815f070 0x7fbf180ca3a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fbf10005fd0 tx=0x7fbf1002c240 comp rx=0 tx=0).stop 2023-10-26T01:30:24.213 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:24.211+0000 7fbf1cc32640 1 -- 172.21.15.169:0/3009635537 shutdown_connections 2023-10-26T01:30:24.213 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:24.211+0000 7fbf1cc32640 1 --2- 172.21.15.169:0/3009635537 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbf1815f070 0x7fbf180ca3a0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:24.214 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:24.211+0000 7fbf1cc32640 1 --2- 172.21.15.169:0/3009635537 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbf1815b530 0x7fbf1806b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:24.214 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:24.211+0000 7fbf1cc32640 1 -- 172.21.15.169:0/3009635537 >> 172.21.15.169:0/3009635537 conn(0x7fbf180cab70 msgr2=0x7fbf180caf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:24.214 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:24.211+0000 7fbf1cc32640 1 -- 172.21.15.169:0/3009635537 shutdown_connections 2023-10-26T01:30:24.214 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:24.211+0000 7fbf1cc32640 1 -- 172.21.15.169:0/3009635537 wait complete. 2023-10-26T01:30:24.217 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:24.212+0000 7fbf1cc32640 1 Processor -- start 2023-10-26T01:30:24.218 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:24.213+0000 7fbf1cc32640 1 -- start start 2023-10-26T01:30:24.218 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:24.213+0000 7fbf1cc32640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbf1806bb00 0x7fbf181507d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:24.218 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:24.214+0000 7fbf1cc32640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbf1815b530 0x7fbf180c4520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:24.218 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:24.214+0000 7fbf1cc32640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbf1815f070 0x7fbf180707c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:24.219 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:24.214+0000 7fbf1cc32640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf180b1bf0 con 0x7fbf1806bb00 2023-10-26T01:30:24.219 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:24.214+0000 7fbf1cc32640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf1815bb70 con 0x7fbf1815b530 2023-10-26T01:30:24.219 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:24.214+0000 7fbf1cc32640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf18150d10 con 0x7fbf1815f070 2023-10-26T01:30:24.219 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:24.214+0000 7fbf177fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbf1806bb00 0x7fbf181507d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:24.220 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:24.214+0000 7fbf177fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbf1806bb00 0x7fbf181507d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:54200/0 (socket says 172.21.15.169:54200) 2023-10-26T01:30:24.220 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:24.214+0000 7fbf177fe640 1 -- 172.21.15.169:0/1148520849 learned_addr learned my addr 172.21.15.169:0/1148520849 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:30:24.220 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:24.214+0000 7fbf17fff640 1 --2- 172.21.15.169:0/1148520849 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbf1815f070 0x7fbf180707c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:24.220 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:24.214+0000 7fbf0ffff640 1 --2- 172.21.15.169:0/1148520849 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbf1815b530 0x7fbf180c4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:24.220 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:24.214+0000 7fbf177fe640 1 -- 172.21.15.169:0/1148520849 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbf1815f070 msgr2=0x7fbf180707c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:24.220 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:24.214+0000 7fbf177fe640 1 --2- 172.21.15.169:0/1148520849 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbf1815f070 0x7fbf180707c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:24.221 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:24.214+0000 7fbf177fe640 1 -- 172.21.15.169:0/1148520849 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbf1815b530 msgr2=0x7fbf180c4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:24.221 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:24.214+0000 7fbf177fe640 1 --2- 172.21.15.169:0/1148520849 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbf1815b530 0x7fbf180c4520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:24.221 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:24.214+0000 7fbf177fe640 1 -- 172.21.15.169:0/1148520849 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf10009900 con 0x7fbf1806bb00 2023-10-26T01:30:24.221 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:24.215+0000 7fbf177fe640 1 --2- 172.21.15.169:0/1148520849 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbf1806bb00 0x7fbf181507d0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fbf0800d770 tx=0x7fbf0800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:24.222 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:24.215+0000 7fbf14ff9640 1 -- 172.21.15.169:0/1148520849 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf08009c30 con 0x7fbf1806bb00 2023-10-26T01:30:24.222 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:24.215+0000 7fbf14ff9640 1 -- 172.21.15.169:0/1148520849 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fbf0800de80 con 0x7fbf1806bb00 2023-10-26T01:30:24.222 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:24.215+0000 7fbf1cc32640 1 -- 172.21.15.169:0/1148520849 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf180b0f70 con 0x7fbf1806bb00 2023-10-26T01:30:24.222 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:24.215+0000 7fbf14ff9640 1 -- 172.21.15.169:0/1148520849 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf08005280 con 0x7fbf1806bb00 2023-10-26T01:30:24.223 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:24.215+0000 7fbf1cc32640 1 -- 172.21.15.169:0/1148520849 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf180b1530 con 0x7fbf1806bb00 2023-10-26T01:30:24.223 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:24.215+0000 7fbf14ff9640 1 -- 172.21.15.169:0/1148520849 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fbf080047a0 con 0x7fbf1806bb00 2023-10-26T01:30:24.223 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:24.215+0000 7fbf14ff9640 1 -- 172.21.15.169:0/1148520849 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fbf080107d0 con 0x7fbf1806bb00 2023-10-26T01:30:24.223 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:24.216+0000 7fbf1cc32640 1 -- 172.21.15.169:0/1148520849 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf18073c10 con 0x7fbf1806bb00 2023-10-26T01:30:24.224 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:24.218+0000 7fbf14ff9640 1 -- 172.21.15.169:0/1148520849 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7fbf08020060 con 0x7fbf1806bb00 2023-10-26T01:30:24.285 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:24.283+0000 7fbf1cc32640 1 -- 172.21.15.169:0/1148520849 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fbf18069fc0 con 0x7fbf1806bb00 2023-10-26T01:30:24.288 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:24.287+0000 7fbf14ff9640 1 -- 172.21.15.169:0/1148520849 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7fbf08017070 con 0x7fbf1806bb00 2023-10-26T01:30:24.288 INFO:teuthology.orchestra.run.smithi169.stderr:adjusted tunables profile to default 2023-10-26T01:30:24.290 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:24.288+0000 7fbf1cc32640 1 -- 172.21.15.169:0/1148520849 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbf1806bb00 msgr2=0x7fbf181507d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:24.290 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:24.288+0000 7fbf1cc32640 1 --2- 172.21.15.169:0/1148520849 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbf1806bb00 0x7fbf181507d0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fbf0800d770 tx=0x7fbf0800dc30 comp rx=0 tx=0).stop 2023-10-26T01:30:24.290 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:24.288+0000 7fbf1cc32640 1 -- 172.21.15.169:0/1148520849 shutdown_connections 2023-10-26T01:30:24.290 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:24.288+0000 7fbf1cc32640 1 --2- 172.21.15.169:0/1148520849 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbf1815f070 0x7fbf180707c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:24.291 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:24.288+0000 7fbf1cc32640 1 --2- 172.21.15.169:0/1148520849 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbf1815b530 0x7fbf180c4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:24.291 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:24.288+0000 7fbf1cc32640 1 --2- 172.21.15.169:0/1148520849 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbf1806bb00 0x7fbf181507d0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:24.291 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:24.288+0000 7fbf1cc32640 1 -- 172.21.15.169:0/1148520849 >> 172.21.15.169:0/1148520849 conn(0x7fbf180cab70 msgr2=0x7fbf180ae520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:24.291 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:24.288+0000 7fbf1cc32640 1 -- 172.21.15.169:0/1148520849 shutdown_connections 2023-10-26T01:30:24.291 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:24.288+0000 7fbf1cc32640 1 -- 172.21.15.169:0/1148520849 wait complete. 2023-10-26T01:30:24.307 INFO:tasks.ceph:check_enable_crimson: False 2023-10-26T01:30:24.308 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-10-26T01:30:24.308 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:30:24.309 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-10-26T01:30:24.392 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:30:24.393 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-10-26T01:30:24.397 INFO:tasks.ceph.mgr.x.smithi170.stderr:2023-10-26T01:30:24.395+0000 7f2a20213040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-26T01:30:24.458 INFO:tasks.ceph.mgr.x.smithi170.stderr:2023-10-26T01:30:24.456+0000 7f2a20213040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-26T01:30:24.508 INFO:tasks.ceph.mgr.x.smithi170.stderr:2023-10-26T01:30:24.507+0000 7f2a20213040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-26T01:30:24.517 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:30:24.517 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-10-26T01:30:24.570 INFO:tasks.ceph.mgr.x.smithi170.stderr:2023-10-26T01:30:24.569+0000 7f2a20213040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-26T01:30:24.620 INFO:tasks.ceph.mgr.x.smithi170.stderr:2023-10-26T01:30:24.619+0000 7f2a20213040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-26T01:30:24.645 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:30:24.645 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-10-26T01:30:24.725 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-26T01:30:24.725 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-10-26T01:30:24.735 INFO:tasks.ceph.mgr.x.smithi170.stderr:2023-10-26T01:30:24.732+0000 7f2a20213040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-26T01:30:24.736 INFO:tasks.ceph.mgr.y.smithi169.stderr:2023-10-26T01:30:24.735+0000 7f33d344c040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-26T01:30:24.799 INFO:tasks.ceph.mgr.y.smithi169.stderr:2023-10-26T01:30:24.798+0000 7f33d344c040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-26T01:30:24.818 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-26T01:30:24.819 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-10-26T01:30:24.820 INFO:tasks.ceph.mgr.x.smithi170.stderr:2023-10-26T01:30:24.816+0000 7f2a20213040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-26T01:30:24.851 INFO:tasks.ceph.mgr.y.smithi169.stderr:2023-10-26T01:30:24.849+0000 7f33d344c040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-26T01:30:24.875 INFO:tasks.ceph.mgr.x.smithi170.stderr:2023-10-26T01:30:24.873+0000 7f2a20213040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-26T01:30:24.903 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-26T01:30:24.903 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-10-26T01:30:24.915 INFO:tasks.ceph.mgr.y.smithi169.stderr:2023-10-26T01:30:24.913+0000 7f33d344c040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-26T01:30:24.940 INFO:tasks.ceph.mgr.x.smithi170.stderr:2023-10-26T01:30:24.938+0000 7f2a20213040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-26T01:30:24.968 INFO:tasks.ceph.mgr.y.smithi169.stderr:2023-10-26T01:30:24.966+0000 7f33d344c040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-26T01:30:24.998 INFO:tasks.ceph.mgr.x.smithi170.stderr:2023-10-26T01:30:24.997+0000 7f2a20213040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-26T01:30:25.028 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-26T01:30:25.028 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-10-26T01:30:25.080 INFO:tasks.ceph.mgr.y.smithi169.stderr:2023-10-26T01:30:25.079+0000 7f33d344c040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-26T01:30:25.114 DEBUG:teuthology.orchestra.run.smithi170:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 0 2023-10-26T01:30:25.162 INFO:tasks.ceph.mgr.y.smithi169.stderr:2023-10-26T01:30:25.160+0000 7f33d344c040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-26T01:30:25.216 INFO:tasks.ceph.mgr.y.smithi169.stderr:2023-10-26T01:30:25.214+0000 7f33d344c040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-26T01:30:25.277 INFO:tasks.ceph.mgr.y.smithi169.stderr:2023-10-26T01:30:25.275+0000 7f33d344c040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-26T01:30:25.332 INFO:tasks.ceph.mgr.y.smithi169.stderr:2023-10-26T01:30:25.331+0000 7f33d344c040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-26T01:30:25.359 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.356+0000 7f2249b39640 1 -- 172.21.15.170:0/3148681835 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f22440cd7e0 msgr2=0x7f22440cdc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:25.359 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.356+0000 7f2249b39640 1 --2- 172.21.15.170:0/3148681835 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f22440cd7e0 0x7f22440cdc10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f223c005fd0 tx=0x7f223c02c0f0 comp rx=0 tx=0).stop 2023-10-26T01:30:25.359 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.356+0000 7f2249b39640 1 -- 172.21.15.170:0/3148681835 shutdown_connections 2023-10-26T01:30:25.359 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.356+0000 7f2249b39640 1 --2- 172.21.15.170:0/3148681835 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f22440cd7e0 0x7f22440cdc10 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:25.360 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.356+0000 7f2249b39640 1 --2- 172.21.15.170:0/3148681835 >> v2:172.21.15.169:3300/0 conn(0x7f2244141cb0 0x7f224415ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:25.360 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.356+0000 7f2249b39640 1 -- 172.21.15.170:0/3148681835 >> 172.21.15.170:0/3148681835 conn(0x7f2244069930 msgr2=0x7f2244069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:25.360 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.356+0000 7f2249b39640 1 -- 172.21.15.170:0/3148681835 shutdown_connections 2023-10-26T01:30:25.360 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.356+0000 7f2249b39640 1 -- 172.21.15.170:0/3148681835 wait complete. 2023-10-26T01:30:25.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.358+0000 7f2249b39640 1 Processor -- start 2023-10-26T01:30:25.363 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.358+0000 7f2249b39640 1 -- start start 2023-10-26T01:30:25.363 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.359+0000 7f2249b39640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22440ccda0 0x7f22440caa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:25.363 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.359+0000 7f2249b39640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f22440cd7e0 0x7f22440aef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:25.363 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.359+0000 7f2249b39640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2244141cb0 0x7f22440af470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:25.364 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.359+0000 7f2249b39640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22440c67e0 con 0x7f22440ccda0 2023-10-26T01:30:25.364 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.359+0000 7f2249b39640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22440c6ac0 con 0x7f22440cd7e0 2023-10-26T01:30:25.364 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.359+0000 7f2249b39640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22440c6da0 con 0x7f2244141cb0 2023-10-26T01:30:25.364 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.359+0000 7f223bfff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f22440cd7e0 0x7f22440aef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:25.364 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.359+0000 7f2243fff640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2244141cb0 0x7f22440af470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:25.365 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.359+0000 7f223bfff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f22440cd7e0 0x7f22440aef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:60388/0 (socket says 172.21.15.170:60388) 2023-10-26T01:30:25.365 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.359+0000 7f223bfff640 1 -- 172.21.15.170:0/864152065 learned_addr learned my addr 172.21.15.170:0/864152065 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-26T01:30:25.365 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.360+0000 7f22437fe640 1 --2- 172.21.15.170:0/864152065 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22440ccda0 0x7f22440caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:25.365 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.360+0000 7f2243fff640 1 -- 172.21.15.170:0/864152065 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f22440cd7e0 msgr2=0x7f22440aef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:25.365 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.360+0000 7f2243fff640 1 --2- 172.21.15.170:0/864152065 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f22440cd7e0 0x7f22440aef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:25.366 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.360+0000 7f2243fff640 1 -- 172.21.15.170:0/864152065 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22440ccda0 msgr2=0x7f22440caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:25.366 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.360+0000 7f2243fff640 1 --2- 172.21.15.170:0/864152065 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22440ccda0 0x7f22440caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:25.366 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.360+0000 7f2243fff640 1 -- 172.21.15.170:0/864152065 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f223c009840 con 0x7f2244141cb0 2023-10-26T01:30:25.366 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.360+0000 7f2243fff640 1 --2- 172.21.15.170:0/864152065 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2244141cb0 0x7f22440af470 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f223c009000 tx=0x7f223c004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:25.367 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.360+0000 7f2240ff9640 1 -- 172.21.15.170:0/864152065 <== mon.2 v2:172.21.15.170:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f223c004670 con 0x7f2244141cb0 2023-10-26T01:30:25.368 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.360+0000 7f2240ff9640 1 -- 172.21.15.170:0/864152065 <== mon.2 v2:172.21.15.170:3301/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f223c0345a0 con 0x7f2244141cb0 2023-10-26T01:30:25.368 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.360+0000 7f2240ff9640 1 -- 172.21.15.170:0/864152065 <== mon.2 v2:172.21.15.170:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f223c0349b0 con 0x7f2244141cb0 2023-10-26T01:30:25.368 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.360+0000 7f2249b39640 1 -- 172.21.15.170:0/864152065 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22440c7080 con 0x7f2244141cb0 2023-10-26T01:30:25.368 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.361+0000 7f2249b39640 1 -- 172.21.15.170:0/864152065 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22440cafd0 con 0x7f2244141cb0 2023-10-26T01:30:25.368 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.361+0000 7f2240ff9640 1 -- 172.21.15.170:0/864152065 <== mon.2 v2:172.21.15.170:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f223c03e450 con 0x7f2244141cb0 2023-10-26T01:30:25.369 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.361+0000 7f2240ff9640 1 -- 172.21.15.170:0/864152065 <== mon.2 v2:172.21.15.170:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f223c034e10 con 0x7f2244141cb0 2023-10-26T01:30:25.369 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.362+0000 7f2249b39640 1 -- 172.21.15.170:0/864152065 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22441f3330 con 0x7f2244141cb0 2023-10-26T01:30:25.370 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.366+0000 7f2240ff9640 1 -- 172.21.15.170:0/864152065 <== mon.2 v2:172.21.15.170:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f22441f3330 con 0x7f2244141cb0 2023-10-26T01:30:25.445 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.444+0000 7f2249b39640 1 -- 172.21.15.170:0/864152065 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 0} v 0) v1 -- 0x7f22440c6ee0 con 0x7f2244141cb0 2023-10-26T01:30:25.452 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.450+0000 7f2240ff9640 1 -- 172.21.15.170:0/864152065 <== mon.2 v2:172.21.15.170:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f22440c6ee0 con 0x7f2244141cb0 2023-10-26T01:30:25.452 INFO:teuthology.orchestra.run.smithi170.stdout:0 2023-10-26T01:30:25.454 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.452+0000 7f2249b39640 1 -- 172.21.15.170:0/864152065 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2244141cb0 msgr2=0x7f22440af470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:25.454 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.452+0000 7f2249b39640 1 --2- 172.21.15.170:0/864152065 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2244141cb0 0x7f22440af470 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f223c009000 tx=0x7f223c004270 comp rx=0 tx=0).stop 2023-10-26T01:30:25.454 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.452+0000 7f2249b39640 1 -- 172.21.15.170:0/864152065 shutdown_connections 2023-10-26T01:30:25.454 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.452+0000 7f2249b39640 1 --2- 172.21.15.170:0/864152065 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2244141cb0 0x7f22440af470 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:25.455 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.452+0000 7f2249b39640 1 --2- 172.21.15.170:0/864152065 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f22440cd7e0 0x7f22440aef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:25.455 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.452+0000 7f2249b39640 1 --2- 172.21.15.170:0/864152065 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22440ccda0 0x7f22440caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:25.455 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.452+0000 7f2249b39640 1 -- 172.21.15.170:0/864152065 >> 172.21.15.170:0/864152065 conn(0x7f2244069930 msgr2=0x7f2244069d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:25.455 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.452+0000 7f2249b39640 1 -- 172.21.15.170:0/864152065 shutdown_connections 2023-10-26T01:30:25.457 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.453+0000 7f2249b39640 1 -- 172.21.15.170:0/864152065 wait complete. 2023-10-26T01:30:25.470 DEBUG:teuthology.orchestra.run.smithi170:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 1 2023-10-26T01:30:25.478 INFO:tasks.ceph.mgr.x.smithi170.stderr:2023-10-26T01:30:25.477+0000 7f2a20213040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-26T01:30:25.646 INFO:tasks.ceph.mgr.x.smithi170.stderr:2023-10-26T01:30:25.645+0000 7f2a20213040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-26T01:30:25.658 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.656+0000 7f55a4eda640 1 -- 172.21.15.170:0/1774408457 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f558c005180 msgr2=0x7f558c00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:25.658 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.656+0000 7f55a4eda640 1 --2- 172.21.15.170:0/1774408457 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f558c005180 0x7f558c00b980 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5598001d80 tx=0x7f558c0057f0 comp rx=0 tx=0).stop 2023-10-26T01:30:25.658 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.656+0000 7f55a4eda640 1 -- 172.21.15.170:0/1774408457 shutdown_connections 2023-10-26T01:30:25.658 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.656+0000 7f55a4eda640 1 --2- 172.21.15.170:0/1774408457 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f558c005180 0x7f558c00b980 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:25.658 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.656+0000 7f55a4eda640 1 -- 172.21.15.170:0/1774408457 >> 172.21.15.170:0/1774408457 conn(0x7f55a00d1360 msgr2=0x7f55a00d1740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:25.659 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.656+0000 7f55a4eda640 1 -- 172.21.15.170:0/1774408457 shutdown_connections 2023-10-26T01:30:25.659 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.656+0000 7f55a4eda640 1 -- 172.21.15.170:0/1774408457 wait complete. 2023-10-26T01:30:25.662 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.658+0000 7f55a4eda640 1 Processor -- start 2023-10-26T01:30:25.662 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.658+0000 7f55a4eda640 1 -- start start 2023-10-26T01:30:25.664 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.659+0000 7f55a4eda640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f558c005180 0x7f55a00c8430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:25.664 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.659+0000 7f55a4eda640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f55a00c9f70 0x7f55a00b3080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:25.665 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.659+0000 7f55a4eda640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55a00a7f30 0x7f55a01f5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:25.665 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.659+0000 7f55a4eda640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55a00c7450 con 0x7f55a00a7f30 2023-10-26T01:30:25.665 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.659+0000 7f55a4eda640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55a00ca3f0 con 0x7f558c005180 2023-10-26T01:30:25.665 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.659+0000 7f55a4eda640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55a00c9320 con 0x7f55a00c9f70 2023-10-26T01:30:25.665 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.659+0000 7f559f7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f558c005180 0x7f55a00c8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:25.666 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.659+0000 7f5597fff640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f55a00c9f70 0x7f55a00b3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:25.666 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.659+0000 7f559f7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f558c005180 0x7f55a00c8430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:60412/0 (socket says 172.21.15.170:60412) 2023-10-26T01:30:25.666 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.659+0000 7f559f7fe640 1 -- 172.21.15.170:0/2890225489 learned_addr learned my addr 172.21.15.170:0/2890225489 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-26T01:30:25.666 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.659+0000 7f559ffff640 1 --2- 172.21.15.170:0/2890225489 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55a00a7f30 0x7f55a01f5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:25.666 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.660+0000 7f559f7fe640 1 -- 172.21.15.170:0/2890225489 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f55a00c9f70 msgr2=0x7f55a00b3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:25.667 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.660+0000 7f559f7fe640 1 --2- 172.21.15.170:0/2890225489 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f55a00c9f70 0x7f55a00b3080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:25.667 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.660+0000 7f559f7fe640 1 -- 172.21.15.170:0/2890225489 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55a00a7f30 msgr2=0x7f55a01f5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:25.667 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.660+0000 7f559f7fe640 1 --2- 172.21.15.170:0/2890225489 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55a00a7f30 0x7f55a01f5920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:25.667 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.660+0000 7f559f7fe640 1 -- 172.21.15.170:0/2890225489 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55a00cfc60 con 0x7f558c005180 2023-10-26T01:30:25.667 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.660+0000 7f559f7fe640 1 --2- 172.21.15.170:0/2890225489 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f558c005180 0x7f55a00c8430 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f5590006630 tx=0x7f55900039c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:25.668 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.660+0000 7f559cff9640 1 -- 172.21.15.170:0/2890225489 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5590011900 con 0x7f558c005180 2023-10-26T01:30:25.668 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.660+0000 7f559cff9640 1 -- 172.21.15.170:0/2890225489 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5590011a60 con 0x7f558c005180 2023-10-26T01:30:25.668 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.660+0000 7f559cff9640 1 -- 172.21.15.170:0/2890225489 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5590011e70 con 0x7f558c005180 2023-10-26T01:30:25.668 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.660+0000 7f55a4eda640 1 -- 172.21.15.170:0/2890225489 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55a00c8a80 con 0x7f558c005180 2023-10-26T01:30:25.668 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.660+0000 7f55a4eda640 1 -- 172.21.15.170:0/2890225489 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55a00c9040 con 0x7f558c005180 2023-10-26T01:30:25.669 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.661+0000 7f559cff9640 1 -- 172.21.15.170:0/2890225489 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f559000f430 con 0x7f558c005180 2023-10-26T01:30:25.669 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.661+0000 7f559cff9640 1 -- 172.21.15.170:0/2890225489 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7f55900108e0 con 0x7f558c005180 2023-10-26T01:30:25.669 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.662+0000 7f55a4eda640 1 -- 172.21.15.170:0/2890225489 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55a0073c10 con 0x7f558c005180 2023-10-26T01:30:25.669 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.666+0000 7f559cff9640 1 -- 172.21.15.170:0/2890225489 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f559003ab40 con 0x7f558c005180 2023-10-26T01:30:25.744 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.742+0000 7f55a4eda640 1 -- 172.21.15.170:0/2890225489 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 1} v 0) v1 -- 0x7f55a00d1f20 con 0x7f558c005180 2023-10-26T01:30:25.750 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.749+0000 7f559cff9640 1 -- 172.21.15.170:0/2890225489 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f5590029020 con 0x7f558c005180 2023-10-26T01:30:25.750 INFO:teuthology.orchestra.run.smithi170.stdout:1 2023-10-26T01:30:25.752 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.750+0000 7f55a4eda640 1 -- 172.21.15.170:0/2890225489 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f558c005180 msgr2=0x7f55a00c8430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:25.752 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.750+0000 7f55a4eda640 1 --2- 172.21.15.170:0/2890225489 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f558c005180 0x7f55a00c8430 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f5590006630 tx=0x7f55900039c0 comp rx=0 tx=0).stop 2023-10-26T01:30:25.752 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.750+0000 7f55a4eda640 1 -- 172.21.15.170:0/2890225489 shutdown_connections 2023-10-26T01:30:25.752 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.750+0000 7f55a4eda640 1 --2- 172.21.15.170:0/2890225489 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55a00a7f30 0x7f55a01f5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:25.752 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.750+0000 7f55a4eda640 1 --2- 172.21.15.170:0/2890225489 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f55a00c9f70 0x7f55a00b3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:25.753 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.750+0000 7f55a4eda640 1 --2- 172.21.15.170:0/2890225489 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f558c005180 0x7f55a00c8430 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:25.753 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.750+0000 7f55a4eda640 1 -- 172.21.15.170:0/2890225489 >> 172.21.15.170:0/2890225489 conn(0x7f55a00d1360 msgr2=0x7f55a015d840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:25.753 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.750+0000 7f55a4eda640 1 -- 172.21.15.170:0/2890225489 shutdown_connections 2023-10-26T01:30:25.753 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.750+0000 7f55a4eda640 1 -- 172.21.15.170:0/2890225489 wait complete. 2023-10-26T01:30:25.767 DEBUG:teuthology.orchestra.run.smithi170:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 2 2023-10-26T01:30:25.794 INFO:tasks.ceph.mgr.y.smithi169.stderr:2023-10-26T01:30:25.793+0000 7f33d344c040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-26T01:30:25.796 INFO:tasks.ceph.mgr.x.smithi170.stderr:2023-10-26T01:30:25.795+0000 7f2a20213040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-26T01:30:25.852 INFO:tasks.ceph.mgr.x.smithi170.stderr:2023-10-26T01:30:25.851+0000 7f2a20213040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-26T01:30:25.958 INFO:tasks.ceph.mgr.y.smithi169.stderr:2023-10-26T01:30:25.956+0000 7f33d344c040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-26T01:30:25.967 INFO:tasks.ceph.mgr.x.smithi170.stderr:2023-10-26T01:30:25.965+0000 7f2a20213040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-26T01:30:25.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.989+0000 7fbf6e947640 1 -- 172.21.15.170:0/524612575 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbf680b0720 msgr2=0x7fbf680b0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:25.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.989+0000 7fbf6e947640 1 --2- 172.21.15.170:0/524612575 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbf680b0720 0x7fbf680b0170 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fbf6804e0c0 tx=0x7fbf5802bd70 comp rx=0 tx=0).stop 2023-10-26T01:30:25.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.989+0000 7fbf6e947640 1 -- 172.21.15.170:0/524612575 shutdown_connections 2023-10-26T01:30:25.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.989+0000 7fbf6e947640 1 --2- 172.21.15.170:0/524612575 >> v2:172.21.15.169:3300/0 conn(0x7fbf680d0e10 0x7fbf68065ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:25.992 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.989+0000 7fbf6e947640 1 --2- 172.21.15.170:0/524612575 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbf680b0720 0x7fbf680b0170 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:25.992 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.989+0000 7fbf6e947640 1 -- 172.21.15.170:0/524612575 >> 172.21.15.170:0/524612575 conn(0x7fbf680d1370 msgr2=0x7fbf680d1780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:25.992 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.989+0000 7fbf6e947640 1 -- 172.21.15.170:0/524612575 shutdown_connections 2023-10-26T01:30:25.992 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.989+0000 7fbf6e947640 1 -- 172.21.15.170:0/524612575 wait complete. 2023-10-26T01:30:25.993 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.991+0000 7fbf6e947640 1 Processor -- start 2023-10-26T01:30:25.993 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.991+0000 7fbf6e947640 1 -- start start 2023-10-26T01:30:25.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.992+0000 7fbf6e947640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbf680b0720 0x7fbf68150920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:25.996 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.992+0000 7fbf6e947640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbf680d0e10 0x7fbf6814b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:25.996 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.992+0000 7fbf6e947640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbf6815ece0 0x7fbf680c9a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:25.996 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.992+0000 7fbf6e947640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf68150f00 con 0x7fbf680d0e10 2023-10-26T01:30:25.996 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.992+0000 7fbf6e947640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf681511e0 con 0x7fbf680b0720 2023-10-26T01:30:25.997 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.992+0000 7fbf6e947640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf681514c0 con 0x7fbf6815ece0 2023-10-26T01:30:25.997 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.992+0000 7fbf6d945640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbf680b0720 0x7fbf68150920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:25.997 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.992+0000 7fbf6d945640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbf680b0720 0x7fbf68150920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:60438/0 (socket says 172.21.15.170:60438) 2023-10-26T01:30:25.997 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.992+0000 7fbf6d945640 1 -- 172.21.15.170:0/535218194 learned_addr learned my addr 172.21.15.170:0/535218194 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-26T01:30:25.997 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.992+0000 7fbf6e146640 1 --2- 172.21.15.170:0/535218194 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbf6815ece0 0x7fbf680c9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:25.998 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.992+0000 7fbf6d144640 1 --2- 172.21.15.170:0/535218194 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbf680d0e10 0x7fbf6814b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:25.998 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.993+0000 7fbf6d945640 1 -- 172.21.15.170:0/535218194 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbf6815ece0 msgr2=0x7fbf680c9a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:25.998 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.993+0000 7fbf6d945640 1 --2- 172.21.15.170:0/535218194 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbf6815ece0 0x7fbf680c9a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:25.998 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.993+0000 7fbf6d945640 1 -- 172.21.15.170:0/535218194 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbf680d0e10 msgr2=0x7fbf6814b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:25.998 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.993+0000 7fbf6d945640 1 --2- 172.21.15.170:0/535218194 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbf680d0e10 0x7fbf6814b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:25.999 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.993+0000 7fbf6d945640 1 -- 172.21.15.170:0/535218194 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf58009490 con 0x7fbf680b0720 2023-10-26T01:30:25.999 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.993+0000 7fbf6e146640 1 --2- 172.21.15.170:0/535218194 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbf6815ece0 0x7fbf680c9a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T01:30:25.999 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.993+0000 7fbf6d945640 1 --2- 172.21.15.170:0/535218194 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbf680b0720 0x7fbf68150920 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fbf6400c430 tx=0x7fbf6400c8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:25.999 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.993+0000 7fbf5e7fc640 1 -- 172.21.15.170:0/535218194 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf64011790 con 0x7fbf680b0720 2023-10-26T01:30:25.999 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.993+0000 7fbf5e7fc640 1 -- 172.21.15.170:0/535218194 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fbf64011a00 con 0x7fbf680b0720 2023-10-26T01:30:25.999 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.993+0000 7fbf5e7fc640 1 -- 172.21.15.170:0/535218194 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf64011e10 con 0x7fbf680b0720 2023-10-26T01:30:26.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.993+0000 7fbf6e947640 1 -- 172.21.15.170:0/535218194 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf681517a0 con 0x7fbf680b0720 2023-10-26T01:30:26.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.994+0000 7fbf6e947640 1 -- 172.21.15.170:0/535218194 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf680cbd70 con 0x7fbf680b0720 2023-10-26T01:30:26.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.994+0000 7fbf5e7fc640 1 -- 172.21.15.170:0/535218194 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fbf6401c760 con 0x7fbf680b0720 2023-10-26T01:30:26.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.994+0000 7fbf5e7fc640 1 -- 172.21.15.170:0/535218194 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7fbf64012420 con 0x7fbf680b0720 2023-10-26T01:30:26.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.995+0000 7fbf6e947640 1 -- 172.21.15.170:0/535218194 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf68073c10 con 0x7fbf680b0720 2023-10-26T01:30:26.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:25.997+0000 7fbf5e7fc640 1 -- 172.21.15.170:0/535218194 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7fbf6401ca10 con 0x7fbf680b0720 2023-10-26T01:30:26.092 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.091+0000 7fbf6e947640 1 -- 172.21.15.170:0/535218194 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 2} v 0) v1 -- 0x7fbf680d1fc0 con 0x7fbf680b0720 2023-10-26T01:30:26.096 INFO:tasks.ceph.mgr.x.smithi170.stderr:2023-10-26T01:30:26.095+0000 7f2a20213040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-26T01:30:26.098 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.097+0000 7fbf5e7fc640 1 -- 172.21.15.170:0/535218194 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7fbf6401cc90 con 0x7fbf680b0720 2023-10-26T01:30:26.098 INFO:teuthology.orchestra.run.smithi170.stdout:2 2023-10-26T01:30:26.099 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.098+0000 7fbf6e947640 1 -- 172.21.15.170:0/535218194 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbf680b0720 msgr2=0x7fbf68150920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:26.100 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.098+0000 7fbf6e947640 1 --2- 172.21.15.170:0/535218194 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbf680b0720 0x7fbf68150920 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fbf6400c430 tx=0x7fbf6400c8f0 comp rx=0 tx=0).stop 2023-10-26T01:30:26.100 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.098+0000 7fbf6e947640 1 -- 172.21.15.170:0/535218194 shutdown_connections 2023-10-26T01:30:26.101 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.098+0000 7fbf6e947640 1 --2- 172.21.15.170:0/535218194 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbf680d0e10 0x7fbf6814b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:26.101 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.098+0000 7fbf6e947640 1 --2- 172.21.15.170:0/535218194 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbf6815ece0 0x7fbf680c9a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:26.101 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.098+0000 7fbf6e947640 1 --2- 172.21.15.170:0/535218194 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbf680b0720 0x7fbf68150920 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:26.101 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.098+0000 7fbf6e947640 1 -- 172.21.15.170:0/535218194 >> 172.21.15.170:0/535218194 conn(0x7fbf680d1370 msgr2=0x7fbf680c6040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:26.101 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.099+0000 7fbf6e947640 1 -- 172.21.15.170:0/535218194 shutdown_connections 2023-10-26T01:30:26.102 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.099+0000 7fbf6e947640 1 -- 172.21.15.170:0/535218194 wait complete. 2023-10-26T01:30:26.105 INFO:tasks.ceph.mgr.y.smithi169.stderr:2023-10-26T01:30:26.103+0000 7f33d344c040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-26T01:30:26.116 DEBUG:teuthology.orchestra.run.smithi170:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 3 2023-10-26T01:30:26.159 INFO:tasks.ceph.mgr.y.smithi169.stderr:2023-10-26T01:30:26.158+0000 7f33d344c040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-26T01:30:26.267 INFO:tasks.ceph.mgr.x.smithi170.stderr:2023-10-26T01:30:26.266+0000 7f2a20213040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-26T01:30:26.270 INFO:tasks.ceph.mgr.y.smithi169.stderr:2023-10-26T01:30:26.268+0000 7f33d344c040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-26T01:30:26.315 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.314+0000 7f3bbdba5640 1 -- 172.21.15.170:0/4124134397 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3bb815ba30 msgr2=0x7f3bb80b4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:26.316 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.314+0000 7f3bbdba5640 1 --2- 172.21.15.170:0/4124134397 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3bb815ba30 0x7f3bb80b4770 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f3ba4009860 tx=0x7f3ba402bd60 comp rx=0 tx=0).stop 2023-10-26T01:30:26.316 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.314+0000 7f3bbdba5640 1 -- 172.21.15.170:0/4124134397 shutdown_connections 2023-10-26T01:30:26.316 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.314+0000 7f3bbdba5640 1 --2- 172.21.15.170:0/4124134397 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3bb815ba30 0x7f3bb80b4770 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:26.316 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.314+0000 7f3bbdba5640 1 -- 172.21.15.170:0/4124134397 >> 172.21.15.170:0/4124134397 conn(0x7f3bb80d1360 msgr2=0x7f3bb80d1740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:26.317 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.314+0000 7f3bbdba5640 1 -- 172.21.15.170:0/4124134397 shutdown_connections 2023-10-26T01:30:26.317 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.314+0000 7f3bbdba5640 1 -- 172.21.15.170:0/4124134397 wait complete. 2023-10-26T01:30:26.320 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.316+0000 7f3bbdba5640 1 Processor -- start 2023-10-26T01:30:26.320 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.316+0000 7f3bbdba5640 1 -- start start 2023-10-26T01:30:26.320 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.317+0000 7f3bbdba5640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3bb8065ea0 0x7f3bb80c4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:26.320 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.317+0000 7f3bbdba5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3bb80ccab0 0x7f3bb80c6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:26.320 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.317+0000 7f3bbdba5640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3bb815ba30 0x7f3bb81547c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:26.321 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.317+0000 7f3bbdba5640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb80c8430 con 0x7f3bb8065ea0 2023-10-26T01:30:26.321 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.317+0000 7f3bbdba5640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb80c53d0 con 0x7f3bb80ccab0 2023-10-26T01:30:26.321 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.317+0000 7f3bbdba5640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb80c7710 con 0x7f3bb815ba30 2023-10-26T01:30:26.321 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.317+0000 7f3bb7fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3bb80ccab0 0x7f3bb80c6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:26.321 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.317+0000 7f3bbd3a4640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3bb815ba30 0x7f3bb81547c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:26.322 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.317+0000 7f3bb7fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3bb80ccab0 0x7f3bb80c6ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:60444/0 (socket says 172.21.15.170:60444) 2023-10-26T01:30:26.322 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.317+0000 7f3bb7fff640 1 -- 172.21.15.170:0/3289014702 learned_addr learned my addr 172.21.15.170:0/3289014702 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-26T01:30:26.322 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.317+0000 7f3bbcba3640 1 --2- 172.21.15.170:0/3289014702 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3bb8065ea0 0x7f3bb80c4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:26.322 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.317+0000 7f3bb7fff640 1 -- 172.21.15.170:0/3289014702 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3bb815ba30 msgr2=0x7f3bb81547c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:26.322 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.317+0000 7f3bb7fff640 1 --2- 172.21.15.170:0/3289014702 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3bb815ba30 0x7f3bb81547c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:26.323 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.317+0000 7f3bb7fff640 1 -- 172.21.15.170:0/3289014702 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3bb8065ea0 msgr2=0x7f3bb80c4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:26.323 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.317+0000 7f3bb7fff640 1 --2- 172.21.15.170:0/3289014702 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3bb8065ea0 0x7f3bb80c4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:26.323 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.317+0000 7f3bb7fff640 1 -- 172.21.15.170:0/3289014702 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ba4009490 con 0x7f3bb80ccab0 2023-10-26T01:30:26.323 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.318+0000 7f3bbd3a4640 1 --2- 172.21.15.170:0/3289014702 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3bb815ba30 0x7f3bb81547c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T01:30:26.324 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.318+0000 7f3bbcba3640 1 --2- 172.21.15.170:0/3289014702 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3bb8065ea0 0x7f3bb80c4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T01:30:26.324 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.318+0000 7f3bb7fff640 1 --2- 172.21.15.170:0/3289014702 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3bb80ccab0 0x7f3bb80c6ec0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f3bac00c430 tx=0x7f3bac00c8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:26.324 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.318+0000 7f3bb57fa640 1 -- 172.21.15.170:0/3289014702 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bac011790 con 0x7f3bb80ccab0 2023-10-26T01:30:26.324 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.318+0000 7f3bb57fa640 1 -- 172.21.15.170:0/3289014702 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3bac0119e0 con 0x7f3bb80ccab0 2023-10-26T01:30:26.324 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.318+0000 7f3bb57fa640 1 -- 172.21.15.170:0/3289014702 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bac011df0 con 0x7f3bb80ccab0 2023-10-26T01:30:26.325 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.318+0000 7f3bbdba5640 1 -- 172.21.15.170:0/3289014702 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bb80c79f0 con 0x7f3bb80ccab0 2023-10-26T01:30:26.325 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.318+0000 7f3bbdba5640 1 -- 172.21.15.170:0/3289014702 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bb80c8680 con 0x7f3bb80ccab0 2023-10-26T01:30:26.325 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.318+0000 7f3bb57fa640 1 -- 172.21.15.170:0/3289014702 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f3bac01c760 con 0x7f3bb80ccab0 2023-10-26T01:30:26.325 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.319+0000 7f3bb57fa640 1 -- 172.21.15.170:0/3289014702 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f3bac012420 con 0x7f3bb80ccab0 2023-10-26T01:30:26.326 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.319+0000 7f3bbdba5640 1 -- 172.21.15.170:0/3289014702 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bb8073c10 con 0x7f3bb80ccab0 2023-10-26T01:30:26.326 INFO:tasks.ceph.mgr.x.smithi170.stderr:2023-10-26T01:30:26.320+0000 7f2a20213040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-26T01:30:26.326 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.322+0000 7f3bb57fa640 1 -- 172.21.15.170:0/3289014702 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f3bac012880 con 0x7f3bb80ccab0 2023-10-26T01:30:26.382 INFO:tasks.ceph.mgr.y.smithi169.stderr:2023-10-26T01:30:26.381+0000 7f33d344c040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-26T01:30:26.400 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.399+0000 7f3bbdba5640 1 -- 172.21.15.170:0/3289014702 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 3} v 0) v1 -- 0x7f3bb80d1f20 con 0x7f3bb80ccab0 2023-10-26T01:30:26.407 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.406+0000 7f3bb57fa640 1 -- 172.21.15.170:0/3289014702 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f3bac012b00 con 0x7f3bb80ccab0 2023-10-26T01:30:26.407 INFO:teuthology.orchestra.run.smithi170.stdout:3 2023-10-26T01:30:26.409 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.407+0000 7f3bbdba5640 1 -- 172.21.15.170:0/3289014702 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3bb80ccab0 msgr2=0x7f3bb80c6ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:26.410 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.407+0000 7f3bbdba5640 1 --2- 172.21.15.170:0/3289014702 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3bb80ccab0 0x7f3bb80c6ec0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f3bac00c430 tx=0x7f3bac00c8f0 comp rx=0 tx=0).stop 2023-10-26T01:30:26.410 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.408+0000 7f3bbdba5640 1 -- 172.21.15.170:0/3289014702 shutdown_connections 2023-10-26T01:30:26.410 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.408+0000 7f3bbdba5640 1 --2- 172.21.15.170:0/3289014702 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3bb815ba30 0x7f3bb81547c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:26.410 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.408+0000 7f3bbdba5640 1 --2- 172.21.15.170:0/3289014702 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3bb80ccab0 0x7f3bb80c6ec0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:26.411 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.408+0000 7f3bbdba5640 1 --2- 172.21.15.170:0/3289014702 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3bb8065ea0 0x7f3bb80c4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:26.411 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.408+0000 7f3bbdba5640 1 -- 172.21.15.170:0/3289014702 >> 172.21.15.170:0/3289014702 conn(0x7f3bb80d1360 msgr2=0x7f3bb80cc1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:26.411 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.408+0000 7f3bbdba5640 1 -- 172.21.15.170:0/3289014702 shutdown_connections 2023-10-26T01:30:26.411 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.408+0000 7f3bbdba5640 1 -- 172.21.15.170:0/3289014702 wait complete. 2023-10-26T01:30:26.431 DEBUG:teuthology.orchestra.run.smithi170:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 4 2023-10-26T01:30:26.493 INFO:tasks.ceph.mgr.x.smithi170.stderr:2023-10-26T01:30:26.492+0000 7f2a20213040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-26T01:30:26.546 INFO:tasks.ceph.mgr.y.smithi169.stderr:2023-10-26T01:30:26.544+0000 7f33d344c040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-26T01:30:26.596 INFO:tasks.ceph.mgr.y.smithi169.stderr:2023-10-26T01:30:26.595+0000 7f33d344c040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-26T01:30:26.635 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.634+0000 7ff1912d7640 1 -- 172.21.15.170:0/1970700994 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff18c0b11b0 msgr2=0x7ff18c15bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:26.635 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.634+0000 7ff1912d7640 1 --2- 172.21.15.170:0/1970700994 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff18c0b11b0 0x7ff18c15bae0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff18000a940 tx=0x7ff18002d090 comp rx=0 tx=0).stop 2023-10-26T01:30:26.636 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.634+0000 7ff1912d7640 1 -- 172.21.15.170:0/1970700994 shutdown_connections 2023-10-26T01:30:26.636 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.634+0000 7ff1912d7640 1 --2- 172.21.15.170:0/1970700994 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff18c0b11b0 0x7ff18c15bae0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:26.636 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.634+0000 7ff1912d7640 1 --2- 172.21.15.170:0/1970700994 >> v2:172.21.15.169:3300/0 conn(0x7ff18c15b700 0x7ff18c0712c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:26.636 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.634+0000 7ff1912d7640 1 -- 172.21.15.170:0/1970700994 >> 172.21.15.170:0/1970700994 conn(0x7ff18c0af8a0 msgr2=0x7ff18c0afcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:26.636 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.634+0000 7ff1912d7640 1 -- 172.21.15.170:0/1970700994 shutdown_connections 2023-10-26T01:30:26.637 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.634+0000 7ff1912d7640 1 -- 172.21.15.170:0/1970700994 wait complete. 2023-10-26T01:30:26.638 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.635+0000 7ff1912d7640 1 Processor -- start 2023-10-26T01:30:26.638 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.636+0000 7ff1912d7640 1 -- start start 2023-10-26T01:30:26.638 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.636+0000 7ff1912d7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff18c157fd0 0x7ff18c14f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:26.638 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.636+0000 7ff1912d7640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff18c15b700 0x7ff18c0a74c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:26.638 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.636+0000 7ff1912d7640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff18c156ed0 0x7ff18c14ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:26.639 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.636+0000 7ff1912d7640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff18c0a7a00 con 0x7ff18c156ed0 2023-10-26T01:30:26.639 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.636+0000 7ff1912d7640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff18c0a7c60 con 0x7ff18c157fd0 2023-10-26T01:30:26.639 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.636+0000 7ff1912d7640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff18c0a7f40 con 0x7ff18c15b700 2023-10-26T01:30:26.641 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.637+0000 7ff18a7fc640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff18c15b700 0x7ff18c0a74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:26.641 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.637+0000 7ff18affd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff18c157fd0 0x7ff18c14f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:26.641 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.637+0000 7ff18a7fc640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff18c15b700 0x7ff18c0a74c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3301/0 says I am v2:172.21.15.170:47346/0 (socket says 172.21.15.170:47346) 2023-10-26T01:30:26.641 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.637+0000 7ff18a7fc640 1 -- 172.21.15.170:0/745793824 learned_addr learned my addr 172.21.15.170:0/745793824 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-26T01:30:26.641 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.637+0000 7ff18b7fe640 1 --2- 172.21.15.170:0/745793824 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff18c156ed0 0x7ff18c14ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:26.642 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.637+0000 7ff18a7fc640 1 -- 172.21.15.170:0/745793824 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff18c157fd0 msgr2=0x7ff18c14f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:26.642 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.637+0000 7ff18a7fc640 1 --2- 172.21.15.170:0/745793824 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff18c157fd0 0x7ff18c14f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:26.642 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.637+0000 7ff18a7fc640 1 -- 172.21.15.170:0/745793824 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff18c156ed0 msgr2=0x7ff18c14ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:26.642 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.637+0000 7ff18a7fc640 1 --2- 172.21.15.170:0/745793824 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff18c156ed0 0x7ff18c14ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:26.642 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.637+0000 7ff18a7fc640 1 -- 172.21.15.170:0/745793824 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff18000a5a0 con 0x7ff18c15b700 2023-10-26T01:30:26.643 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.637+0000 7ff18affd640 1 --2- 172.21.15.170:0/745793824 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff18c157fd0 0x7ff18c14f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T01:30:26.643 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.637+0000 7ff18a7fc640 1 --2- 172.21.15.170:0/745793824 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff18c15b700 0x7ff18c0a74c0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff18000a890 tx=0x7ff180036730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:26.643 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.638+0000 7ff1737fe640 1 -- 172.21.15.170:0/745793824 <== mon.2 v2:172.21.15.170:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff180036d30 con 0x7ff18c15b700 2023-10-26T01:30:26.643 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.638+0000 7ff1737fe640 1 -- 172.21.15.170:0/745793824 <== mon.2 v2:172.21.15.170:3301/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff1800035f0 con 0x7ff18c15b700 2023-10-26T01:30:26.643 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.638+0000 7ff1912d7640 1 -- 172.21.15.170:0/745793824 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff18c0a8220 con 0x7ff18c15b700 2023-10-26T01:30:26.644 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.638+0000 7ff1737fe640 1 -- 172.21.15.170:0/745793824 <== mon.2 v2:172.21.15.170:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff180003a00 con 0x7ff18c15b700 2023-10-26T01:30:26.644 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.638+0000 7ff1912d7640 1 -- 172.21.15.170:0/745793824 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff18c0ab4e0 con 0x7ff18c15b700 2023-10-26T01:30:26.644 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.638+0000 7ff1737fe640 1 -- 172.21.15.170:0/745793824 <== mon.2 v2:172.21.15.170:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7ff18003c070 con 0x7ff18c15b700 2023-10-26T01:30:26.644 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.638+0000 7ff1737fe640 1 -- 172.21.15.170:0/745793824 <== mon.2 v2:172.21.15.170:3301/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7ff180040450 con 0x7ff18c15b700 2023-10-26T01:30:26.644 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.639+0000 7ff1912d7640 1 -- 172.21.15.170:0/745793824 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff18c0706b0 con 0x7ff18c15b700 2023-10-26T01:30:26.645 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.641+0000 7ff1737fe640 1 -- 172.21.15.170:0/745793824 <== mon.2 v2:172.21.15.170:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7ff180044020 con 0x7ff18c15b700 2023-10-26T01:30:26.720 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.719+0000 7ff1912d7640 1 -- 172.21.15.170:0/745793824 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 4} v 0) v1 -- 0x7ff18c0ab340 con 0x7ff18c15b700 2023-10-26T01:30:26.727 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.725+0000 7ff1737fe640 1 -- 172.21.15.170:0/745793824 <== mon.2 v2:172.21.15.170:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7ff18c0ab340 con 0x7ff18c15b700 2023-10-26T01:30:26.727 INFO:teuthology.orchestra.run.smithi170.stdout:4 2023-10-26T01:30:26.728 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.727+0000 7ff1912d7640 1 -- 172.21.15.170:0/745793824 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff18c15b700 msgr2=0x7ff18c0a74c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:26.729 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.727+0000 7ff1912d7640 1 --2- 172.21.15.170:0/745793824 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff18c15b700 0x7ff18c0a74c0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff18000a890 tx=0x7ff180036730 comp rx=0 tx=0).stop 2023-10-26T01:30:26.729 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.727+0000 7ff1912d7640 1 -- 172.21.15.170:0/745793824 shutdown_connections 2023-10-26T01:30:26.729 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.727+0000 7ff1912d7640 1 --2- 172.21.15.170:0/745793824 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff18c156ed0 0x7ff18c14ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:26.730 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.727+0000 7ff1912d7640 1 --2- 172.21.15.170:0/745793824 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff18c15b700 0x7ff18c0a74c0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:26.730 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.727+0000 7ff1912d7640 1 --2- 172.21.15.170:0/745793824 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff18c157fd0 0x7ff18c14f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:26.730 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.727+0000 7ff1912d7640 1 -- 172.21.15.170:0/745793824 >> 172.21.15.170:0/745793824 conn(0x7ff18c0af8a0 msgr2=0x7ff18c0afd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:26.731 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.727+0000 7ff1912d7640 1 -- 172.21.15.170:0/745793824 shutdown_connections 2023-10-26T01:30:26.731 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.727+0000 7ff1912d7640 1 -- 172.21.15.170:0/745793824 wait complete. 2023-10-26T01:30:26.743 DEBUG:teuthology.orchestra.run.smithi170:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 5 2023-10-26T01:30:26.758 INFO:tasks.ceph.mgr.y.smithi169.stderr:2023-10-26T01:30:26.756+0000 7f33d344c040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-26T01:30:26.937 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.935+0000 7f1a8ac5d640 1 -- 172.21.15.170:0/11197874 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1a840b7330 msgr2=0x7f1a840b3110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:26.938 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.935+0000 7f1a8ac5d640 1 --2- 172.21.15.170:0/11197874 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1a840b7330 0x7f1a840b3110 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f1a70005fd0 tx=0x7f1a7002be20 comp rx=0 tx=0).stop 2023-10-26T01:30:26.938 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.935+0000 7f1a8ac5d640 1 -- 172.21.15.170:0/11197874 shutdown_connections 2023-10-26T01:30:26.938 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.935+0000 7f1a8ac5d640 1 --2- 172.21.15.170:0/11197874 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1a840b7330 0x7f1a840b3110 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:26.938 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.935+0000 7f1a8ac5d640 1 --2- 172.21.15.170:0/11197874 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1a8415b9a0 0x7f1a8415f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:26.939 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.935+0000 7f1a8ac5d640 1 -- 172.21.15.170:0/11197874 >> 172.21.15.170:0/11197874 conn(0x7f1a840658b0 msgr2=0x7f1a84065cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:26.939 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.935+0000 7f1a8ac5d640 1 -- 172.21.15.170:0/11197874 shutdown_connections 2023-10-26T01:30:26.939 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.935+0000 7f1a8ac5d640 1 -- 172.21.15.170:0/11197874 wait complete. 2023-10-26T01:30:26.943 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.937+0000 7f1a8ac5d640 1 Processor -- start 2023-10-26T01:30:26.943 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.937+0000 7f1a8ac5d640 1 -- start start 2023-10-26T01:30:26.944 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.938+0000 7f1a8ac5d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1a840b3650 0x7f1a84146fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:26.944 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.938+0000 7f1a8ac5d640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1a840b7330 0x7f1a84157d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:26.944 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.938+0000 7f1a8ac5d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a8415b9a0 0x7f1a841f17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:26.945 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.938+0000 7f1a8ac5d640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a8414a2e0 con 0x7f1a8415b9a0 2023-10-26T01:30:26.945 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.938+0000 7f1a8ac5d640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a841584e0 con 0x7f1a840b3650 2023-10-26T01:30:26.945 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.938+0000 7f1a8ac5d640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a840ca1c0 con 0x7f1a840b7330 2023-10-26T01:30:26.945 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.938+0000 7f1a889f9640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1a840b3650 0x7f1a84146fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:26.946 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.938+0000 7f1a889f9640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1a840b3650 0x7f1a84146fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:60478/0 (socket says 172.21.15.170:60478) 2023-10-26T01:30:26.946 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.938+0000 7f1a889f9640 1 -- 172.21.15.170:0/3035774743 learned_addr learned my addr 172.21.15.170:0/3035774743 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-26T01:30:26.946 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.938+0000 7f1a83fff640 1 --2- 172.21.15.170:0/3035774743 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1a840b7330 0x7f1a84157d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:26.947 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.938+0000 7f1a891fa640 1 --2- 172.21.15.170:0/3035774743 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a8415b9a0 0x7f1a841f17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:26.947 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.938+0000 7f1a889f9640 1 -- 172.21.15.170:0/3035774743 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1a840b7330 msgr2=0x7f1a84157d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:26.948 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.938+0000 7f1a889f9640 1 --2- 172.21.15.170:0/3035774743 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1a840b7330 0x7f1a84157d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:26.948 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.938+0000 7f1a889f9640 1 -- 172.21.15.170:0/3035774743 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a8415b9a0 msgr2=0x7f1a841f17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:26.948 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.938+0000 7f1a889f9640 1 --2- 172.21.15.170:0/3035774743 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a8415b9a0 0x7f1a841f17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:26.949 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.938+0000 7f1a889f9640 1 -- 172.21.15.170:0/3035774743 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a70009560 con 0x7f1a840b3650 2023-10-26T01:30:26.949 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.938+0000 7f1a891fa640 1 --2- 172.21.15.170:0/3035774743 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a8415b9a0 0x7f1a841f17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:30:26.949 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.939+0000 7f1a889f9640 1 --2- 172.21.15.170:0/3035774743 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1a840b3650 0x7f1a84146fd0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f1a7800e7c0 tx=0x7f1a7800ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:26.950 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.939+0000 7f1a817fa640 1 -- 172.21.15.170:0/3035774743 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a78009580 con 0x7f1a840b3650 2023-10-26T01:30:26.950 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.939+0000 7f1a817fa640 1 -- 172.21.15.170:0/3035774743 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1a7800eed0 con 0x7f1a840b3650 2023-10-26T01:30:26.950 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.939+0000 7f1a8ac5d640 1 -- 172.21.15.170:0/3035774743 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a840ca4a0 con 0x7f1a840b3650 2023-10-26T01:30:26.950 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.939+0000 7f1a817fa640 1 -- 172.21.15.170:0/3035774743 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a78010430 con 0x7f1a840b3650 2023-10-26T01:30:26.950 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.939+0000 7f1a8ac5d640 1 -- 172.21.15.170:0/3035774743 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a8415aee0 con 0x7f1a840b3650 2023-10-26T01:30:26.951 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.939+0000 7f1a817fa640 1 -- 172.21.15.170:0/3035774743 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f1a780105b0 con 0x7f1a840b3650 2023-10-26T01:30:26.951 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.939+0000 7f1a817fa640 1 -- 172.21.15.170:0/3035774743 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7f1a78014070 con 0x7f1a840b3650 2023-10-26T01:30:26.951 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.940+0000 7f1a8ac5d640 1 -- 172.21.15.170:0/3035774743 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a840698d0 con 0x7f1a840b3650 2023-10-26T01:30:26.952 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:26.942+0000 7f1a817fa640 1 -- 172.21.15.170:0/3035774743 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f1a78026060 con 0x7f1a840b3650 2023-10-26T01:30:27.023 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:27.021+0000 7f1a8ac5d640 1 -- 172.21.15.170:0/3035774743 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 5} v 0) v1 -- 0x7f1a8404db10 con 0x7f1a840b3650 2023-10-26T01:30:27.030 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:27.028+0000 7f1a817fa640 1 -- 172.21.15.170:0/3035774743 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f1a78010860 con 0x7f1a840b3650 2023-10-26T01:30:27.031 INFO:teuthology.orchestra.run.smithi170.stdout:5 2023-10-26T01:30:27.032 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:27.029+0000 7f1a8ac5d640 1 -- 172.21.15.170:0/3035774743 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1a840b3650 msgr2=0x7f1a84146fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:27.032 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:27.029+0000 7f1a8ac5d640 1 --2- 172.21.15.170:0/3035774743 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1a840b3650 0x7f1a84146fd0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f1a7800e7c0 tx=0x7f1a7800ec80 comp rx=0 tx=0).stop 2023-10-26T01:30:27.032 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:27.029+0000 7f1a8ac5d640 1 -- 172.21.15.170:0/3035774743 shutdown_connections 2023-10-26T01:30:27.033 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:27.029+0000 7f1a8ac5d640 1 --2- 172.21.15.170:0/3035774743 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a8415b9a0 0x7f1a841f17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:27.033 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:27.029+0000 7f1a8ac5d640 1 --2- 172.21.15.170:0/3035774743 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1a840b7330 0x7f1a84157d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:27.033 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:27.029+0000 7f1a8ac5d640 1 --2- 172.21.15.170:0/3035774743 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1a840b3650 0x7f1a84146fd0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:27.033 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:27.029+0000 7f1a8ac5d640 1 -- 172.21.15.170:0/3035774743 >> 172.21.15.170:0/3035774743 conn(0x7f1a840658b0 msgr2=0x7f1a840b10f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:27.033 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:27.030+0000 7f1a8ac5d640 1 -- 172.21.15.170:0/3035774743 shutdown_connections 2023-10-26T01:30:27.034 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:27.030+0000 7f1a8ac5d640 1 -- 172.21.15.170:0/3035774743 wait complete. 2023-10-26T01:30:27.047 DEBUG:teuthology.orchestra.run.smithi170:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 6 2023-10-26T01:30:27.228 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:27.225+0000 7fd00a70f640 1 -- 172.21.15.170:0/2600552888 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd0040d20c0 msgr2=0x7fd0040ce090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:27.229 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:27.225+0000 7fd00a70f640 1 --2- 172.21.15.170:0/2600552888 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd0040d20c0 0x7fd0040ce090 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd0040d1a70 tx=0x7fcffc02c1c0 comp rx=0 tx=0).stop 2023-10-26T01:30:27.229 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:27.226+0000 7fd00a70f640 1 -- 172.21.15.170:0/2600552888 shutdown_connections 2023-10-26T01:30:27.229 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:27.226+0000 7fd00a70f640 1 --2- 172.21.15.170:0/2600552888 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd0040d20c0 0x7fd0040ce090 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:27.230 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:27.226+0000 7fd00a70f640 1 --2- 172.21.15.170:0/2600552888 >> v2:172.21.15.169:3300/0 conn(0x7fd0040d0fc0 0x7fd0040d1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:27.230 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:27.226+0000 7fd00a70f640 1 --2- 172.21.15.170:0/2600552888 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd00415ba00 0x7fd0040b6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:27.230 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:27.226+0000 7fd00a70f640 1 -- 172.21.15.170:0/2600552888 >> 172.21.15.170:0/2600552888 conn(0x7fd00406b5c0 msgr2=0x7fd00406b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:27.230 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:27.226+0000 7fd00a70f640 1 -- 172.21.15.170:0/2600552888 shutdown_connections 2023-10-26T01:30:27.231 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:27.226+0000 7fd00a70f640 1 -- 172.21.15.170:0/2600552888 wait complete. 2023-10-26T01:30:27.233 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:27.227+0000 7fd00a70f640 1 Processor -- start 2023-10-26T01:30:27.233 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:27.228+0000 7fd00a70f640 1 -- start start 2023-10-26T01:30:27.233 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:27.228+0000 7fd00a70f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd0040d0fc0 0x7fd004157a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:27.234 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:27.228+0000 7fd00a70f640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd0040d20c0 0x7fd004143b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:27.234 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:27.228+0000 7fd00a70f640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd00415ba00 0x7fd0041440b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:27.234 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:27.229+0000 7fd00a70f640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd004157590 con 0x7fd00415ba00 2023-10-26T01:30:27.234 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:27.229+0000 7fd00a70f640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd004156c60 con 0x7fd0040d0fc0 2023-10-26T01:30:27.234 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:27.229+0000 7fd00a70f640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd004156f40 con 0x7fd0040d20c0 2023-10-26T01:30:27.235 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:27.229+0000 7fd003fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd0040d0fc0 0x7fd004157a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:27.235 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:27.229+0000 7fd003fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd0040d0fc0 0x7fd004157a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:60492/0 (socket says 172.21.15.170:60492) 2023-10-26T01:30:27.235 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:27.229+0000 7fd003fff640 1 -- 172.21.15.170:0/1718078981 learned_addr learned my addr 172.21.15.170:0/1718078981 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-26T01:30:27.235 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:27.229+0000 7fcffbfff640 1 --2- 172.21.15.170:0/1718078981 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd0040d20c0 0x7fd004143b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:27.235 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:27.229+0000 7fd003fff640 1 -- 172.21.15.170:0/1718078981 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd0040d20c0 msgr2=0x7fd004143b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:27.236 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:27.229+0000 7fd003fff640 1 --2- 172.21.15.170:0/1718078981 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd0040d20c0 0x7fd004143b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:27.236 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:27.229+0000 7fd003fff640 1 -- 172.21.15.170:0/1718078981 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd00415ba00 msgr2=0x7fd0041440b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:27.236 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:27.229+0000 7fd003fff640 1 --2- 172.21.15.170:0/1718078981 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd00415ba00 0x7fd0041440b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:27.236 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:27.229+0000 7fd003fff640 1 -- 172.21.15.170:0/1718078981 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcff4009490 con 0x7fd0040d0fc0 2023-10-26T01:30:27.236 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:27.229+0000 7fd003fff640 1 --2- 172.21.15.170:0/1718078981 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd0040d0fc0 0x7fd004157a20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fcff400e9f0 tx=0x7fcff400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:27.236 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:27.230+0000 7fd0017fa640 1 -- 172.21.15.170:0/1718078981 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcff400cd80 con 0x7fd0040d0fc0 2023-10-26T01:30:27.237 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:27.230+0000 7fd0017fa640 1 -- 172.21.15.170:0/1718078981 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fcff4004020 con 0x7fd0040d0fc0 2023-10-26T01:30:27.237 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:27.230+0000 7fd0017fa640 1 -- 172.21.15.170:0/1718078981 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcff400cee0 con 0x7fd0040d0fc0 2023-10-26T01:30:27.237 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:27.230+0000 7fd00a70f640 1 -- 172.21.15.170:0/1718078981 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcffc009900 con 0x7fd0040d0fc0 2023-10-26T01:30:27.237 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:27.230+0000 7fd00a70f640 1 -- 172.21.15.170:0/1718078981 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd004156950 con 0x7fd0040d0fc0 2023-10-26T01:30:27.237 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:27.230+0000 7fd0017fa640 1 -- 172.21.15.170:0/1718078981 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fcff4005020 con 0x7fd0040d0fc0 2023-10-26T01:30:27.238 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:27.230+0000 7fd0017fa640 1 -- 172.21.15.170:0/1718078981 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7fcff4014070 con 0x7fd0040d0fc0 2023-10-26T01:30:27.238 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:27.231+0000 7fd00a70f640 1 -- 172.21.15.170:0/1718078981 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd00404db10 con 0x7fd0040d0fc0 2023-10-26T01:30:27.238 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:27.233+0000 7fd0017fa640 1 -- 172.21.15.170:0/1718078981 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7fcff4004430 con 0x7fd0040d0fc0 2023-10-26T01:30:27.314 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:27.311+0000 7fd00a70f640 1 -- 172.21.15.170:0/1718078981 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 6} v 0) v1 -- 0x7fd00406a940 con 0x7fd0040d0fc0 2023-10-26T01:30:27.320 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:27.318+0000 7fd0017fa640 1 -- 172.21.15.170:0/1718078981 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7fcff4004b60 con 0x7fd0040d0fc0 2023-10-26T01:30:27.320 INFO:teuthology.orchestra.run.smithi170.stdout:6 2023-10-26T01:30:27.322 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:27.319+0000 7fd00a70f640 1 -- 172.21.15.170:0/1718078981 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd0040d0fc0 msgr2=0x7fd004157a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:27.322 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:27.319+0000 7fd00a70f640 1 --2- 172.21.15.170:0/1718078981 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd0040d0fc0 0x7fd004157a20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fcff400e9f0 tx=0x7fcff400eeb0 comp rx=0 tx=0).stop 2023-10-26T01:30:27.322 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:27.319+0000 7fd00a70f640 1 -- 172.21.15.170:0/1718078981 shutdown_connections 2023-10-26T01:30:27.322 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:27.319+0000 7fd00a70f640 1 --2- 172.21.15.170:0/1718078981 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd00415ba00 0x7fd0041440b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:27.323 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:27.319+0000 7fd00a70f640 1 --2- 172.21.15.170:0/1718078981 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd0040d20c0 0x7fd004143b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:27.323 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:27.319+0000 7fd00a70f640 1 --2- 172.21.15.170:0/1718078981 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd0040d0fc0 0x7fd004157a20 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:27.323 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:27.319+0000 7fd00a70f640 1 -- 172.21.15.170:0/1718078981 >> 172.21.15.170:0/1718078981 conn(0x7fd00406b5c0 msgr2=0x7fd0040c8e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:27.323 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:27.319+0000 7fd00a70f640 1 -- 172.21.15.170:0/1718078981 shutdown_connections 2023-10-26T01:30:27.324 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:27.319+0000 7fd00a70f640 1 -- 172.21.15.170:0/1718078981 wait complete. 2023-10-26T01:30:27.338 DEBUG:teuthology.orchestra.run.smithi170:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 7 2023-10-26T01:30:27.521 INFO:tasks.ceph.mgr.x.smithi170.stderr:2023-10-26T01:30:27.519+0000 7f2a20213040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-26T01:30:27.538 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:27.535+0000 7fb8ba31b640 1 -- 172.21.15.170:0/2191456800 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb8a0003970 msgr2=0x7fb8a0003d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:27.538 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:27.535+0000 7fb8ba31b640 1 --2- 172.21.15.170:0/2191456800 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb8a0003970 0x7fb8a0003d70 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb8b0006fc0 tx=0x7fb8a00057f0 comp rx=0 tx=0).stop 2023-10-26T01:30:27.538 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:27.535+0000 7fb8ba31b640 1 -- 172.21.15.170:0/2191456800 shutdown_connections 2023-10-26T01:30:27.539 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:27.535+0000 7fb8ba31b640 1 --2- 172.21.15.170:0/2191456800 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb8a0003970 0x7fb8a0003d70 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:27.539 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:27.535+0000 7fb8ba31b640 1 -- 172.21.15.170:0/2191456800 >> 172.21.15.170:0/2191456800 conn(0x7fb8b40d1370 msgr2=0x7fb8b40d1780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:27.539 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:27.535+0000 7fb8ba31b640 1 -- 172.21.15.170:0/2191456800 shutdown_connections 2023-10-26T01:30:27.539 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:27.535+0000 7fb8ba31b640 1 -- 172.21.15.170:0/2191456800 wait complete. 2023-10-26T01:30:27.540 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:27.537+0000 7fb8ba31b640 1 Processor -- start 2023-10-26T01:30:27.540 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:27.537+0000 7fb8ba31b640 1 -- start start 2023-10-26T01:30:27.542 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:27.538+0000 7fb8ba31b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb8a0003970 0x7fb8b40c6f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:27.542 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:27.538+0000 7fb8ba31b640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb8b4157e30 0x7fb8b40caa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:27.543 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:27.538+0000 7fb8ba31b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8b41582b0 0x7fb8b40b1e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:27.543 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:27.538+0000 7fb8ba31b640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b40c7460 con 0x7fb8b41582b0 2023-10-26T01:30:27.543 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:27.538+0000 7fb8ba31b640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b4156d90 con 0x7fb8a0003970 2023-10-26T01:30:27.543 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:27.538+0000 7fb8ba31b640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b4157070 con 0x7fb8b4157e30 2023-10-26T01:30:27.543 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:27.538+0000 7fb8b9319640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb8a0003970 0x7fb8b40c6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:27.544 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:27.538+0000 7fb8b8b18640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb8b4157e30 0x7fb8b40caa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:27.544 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:27.538+0000 7fb8b9319640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb8a0003970 0x7fb8b40c6f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:60512/0 (socket says 172.21.15.170:60512) 2023-10-26T01:30:27.544 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:27.538+0000 7fb8b9319640 1 -- 172.21.15.170:0/1899327473 learned_addr learned my addr 172.21.15.170:0/1899327473 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-26T01:30:27.544 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:27.539+0000 7fb8b9b1a640 1 --2- 172.21.15.170:0/1899327473 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8b41582b0 0x7fb8b40b1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:27.544 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:27.539+0000 7fb8b8b18640 1 -- 172.21.15.170:0/1899327473 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb8a0003970 msgr2=0x7fb8b40c6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:27.545 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:27.539+0000 7fb8b8b18640 1 --2- 172.21.15.170:0/1899327473 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb8a0003970 0x7fb8b40c6f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:27.545 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:27.539+0000 7fb8b8b18640 1 -- 172.21.15.170:0/1899327473 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8b41582b0 msgr2=0x7fb8b40b1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:27.545 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:27.539+0000 7fb8b8b18640 1 --2- 172.21.15.170:0/1899327473 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8b41582b0 0x7fb8b40b1e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:27.545 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:27.539+0000 7fb8b8b18640 1 -- 172.21.15.170:0/1899327473 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8b4157350 con 0x7fb8b4157e30 2023-10-26T01:30:27.545 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:27.539+0000 7fb8b9b1a640 1 --2- 172.21.15.170:0/1899327473 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8b41582b0 0x7fb8b40b1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:30:27.546 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:27.539+0000 7fb8b8b18640 1 --2- 172.21.15.170:0/1899327473 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb8b4157e30 0x7fb8b40caa70 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb8a400d300 tx=0x7fb8a400d7c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:27.546 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:27.539+0000 7fb8a9ffb640 1 -- 172.21.15.170:0/1899327473 <== mon.2 v2:172.21.15.170:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8a4010920 con 0x7fb8b4157e30 2023-10-26T01:30:27.546 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:27.539+0000 7fb8a9ffb640 1 -- 172.21.15.170:0/1899327473 <== mon.2 v2:172.21.15.170:3301/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb8a4010b70 con 0x7fb8b4157e30 2023-10-26T01:30:27.546 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:27.539+0000 7fb8a9ffb640 1 -- 172.21.15.170:0/1899327473 <== mon.2 v2:172.21.15.170:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8a400f3b0 con 0x7fb8b4157e30 2023-10-26T01:30:27.546 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:27.539+0000 7fb8ba31b640 1 -- 172.21.15.170:0/1899327473 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8b4157630 con 0x7fb8b4157e30 2023-10-26T01:30:27.547 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:27.539+0000 7fb8ba31b640 1 -- 172.21.15.170:0/1899327473 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8b4153250 con 0x7fb8b4157e30 2023-10-26T01:30:27.547 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:27.540+0000 7fb8a9ffb640 1 -- 172.21.15.170:0/1899327473 <== mon.2 v2:172.21.15.170:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fb8a400f510 con 0x7fb8b4157e30 2023-10-26T01:30:27.547 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:27.540+0000 7fb8a9ffb640 1 -- 172.21.15.170:0/1899327473 <== mon.2 v2:172.21.15.170:3301/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7fb8a400fe20 con 0x7fb8b4157e30 2023-10-26T01:30:27.548 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:27.541+0000 7fb8ba31b640 1 -- 172.21.15.170:0/1899327473 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8b4157b50 con 0x7fb8b4157e30 2023-10-26T01:30:27.548 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:27.543+0000 7fb8a9ffb640 1 -- 172.21.15.170:0/1899327473 <== mon.2 v2:172.21.15.170:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7fb8b4157b50 con 0x7fb8b4157e30 2023-10-26T01:30:27.637 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:27.634+0000 7fb8ba31b640 1 -- 172.21.15.170:0/1899327473 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 7} v 0) v1 -- 0x7fb8b4073c10 con 0x7fb8b4157e30 2023-10-26T01:30:27.643 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:27.641+0000 7fb8a9ffb640 1 -- 172.21.15.170:0/1899327473 <== mon.2 v2:172.21.15.170:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7fb8a4026020 con 0x7fb8b4157e30 2023-10-26T01:30:27.643 INFO:teuthology.orchestra.run.smithi170.stdout:7 2023-10-26T01:30:27.644 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:27.642+0000 7fb8ba31b640 1 -- 172.21.15.170:0/1899327473 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb8b4157e30 msgr2=0x7fb8b40caa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:27.645 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:27.642+0000 7fb8ba31b640 1 --2- 172.21.15.170:0/1899327473 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb8b4157e30 0x7fb8b40caa70 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb8a400d300 tx=0x7fb8a400d7c0 comp rx=0 tx=0).stop 2023-10-26T01:30:27.645 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:27.642+0000 7fb8ba31b640 1 -- 172.21.15.170:0/1899327473 shutdown_connections 2023-10-26T01:30:27.645 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:27.642+0000 7fb8ba31b640 1 --2- 172.21.15.170:0/1899327473 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8b41582b0 0x7fb8b40b1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:27.645 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:27.642+0000 7fb8ba31b640 1 --2- 172.21.15.170:0/1899327473 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb8b4157e30 0x7fb8b40caa70 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:27.645 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:27.642+0000 7fb8ba31b640 1 --2- 172.21.15.170:0/1899327473 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb8a0003970 0x7fb8b40c6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:27.646 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:27.642+0000 7fb8ba31b640 1 -- 172.21.15.170:0/1899327473 >> 172.21.15.170:0/1899327473 conn(0x7fb8b40d1370 msgr2=0x7fb8b415c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:27.646 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:27.642+0000 7fb8ba31b640 1 -- 172.21.15.170:0/1899327473 shutdown_connections 2023-10-26T01:30:27.646 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-26T01:30:27.642+0000 7fb8ba31b640 1 -- 172.21.15.170:0/1899327473 wait complete. 2023-10-26T01:30:27.659 INFO:tasks.ceph.osd.0:Restarting daemon 2023-10-26T01:30:27.660 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-10-26T01:30:27.669 INFO:tasks.ceph.osd.0:Started 2023-10-26T01:30:27.669 INFO:tasks.ceph.osd.1:Restarting daemon 2023-10-26T01:30:27.669 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-10-26T01:30:27.677 INFO:tasks.ceph.osd.1:Started 2023-10-26T01:30:27.678 INFO:tasks.ceph.osd.2:Restarting daemon 2023-10-26T01:30:27.678 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-10-26T01:30:27.683 INFO:tasks.ceph.osd.2:Started 2023-10-26T01:30:27.684 INFO:tasks.ceph.osd.3:Restarting daemon 2023-10-26T01:30:27.684 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-10-26T01:30:27.691 INFO:tasks.ceph.osd.3:Started 2023-10-26T01:30:27.691 INFO:tasks.ceph.osd.4:Restarting daemon 2023-10-26T01:30:27.692 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-10-26T01:30:27.698 INFO:tasks.ceph.osd.4:Started 2023-10-26T01:30:27.699 INFO:tasks.ceph.osd.5:Restarting daemon 2023-10-26T01:30:27.699 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-10-26T01:30:27.707 INFO:tasks.ceph.osd.5:Started 2023-10-26T01:30:27.708 INFO:tasks.ceph.osd.6:Restarting daemon 2023-10-26T01:30:27.708 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-10-26T01:30:27.713 INFO:tasks.ceph.osd.6:Started 2023-10-26T01:30:27.713 INFO:tasks.ceph.osd.7:Restarting daemon 2023-10-26T01:30:27.714 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-10-26T01:30:27.719 INFO:tasks.ceph.osd.7:Started 2023-10-26T01:30:27.720 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:30:27.865 INFO:tasks.ceph.osd.2.smithi169.stderr:2023-10-26T01:30:27.863+0000 7fe4c9b1a500 -1 Falling back to public interface 2023-10-26T01:30:27.871 INFO:tasks.ceph.osd.6.smithi170.stderr:2023-10-26T01:30:27.869+0000 7f9a8d80d500 -1 Falling back to public interface 2023-10-26T01:30:27.872 INFO:tasks.ceph.osd.1.smithi169.stderr:2023-10-26T01:30:27.871+0000 7ff8aeea9500 -1 Falling back to public interface 2023-10-26T01:30:27.880 INFO:tasks.ceph.osd.5.smithi170.stderr:2023-10-26T01:30:27.878+0000 7fd459210500 -1 Falling back to public interface 2023-10-26T01:30:27.890 INFO:tasks.ceph.osd.2.smithi169.stderr:2023-10-26T01:30:27.888+0000 7fe4c9b1a500 -1 osd.2 0 log_to_monitors true 2023-10-26T01:30:27.892 INFO:tasks.ceph.osd.0.smithi169.stderr:2023-10-26T01:30:27.891+0000 7f61c78da500 -1 Falling back to public interface 2023-10-26T01:30:27.894 INFO:tasks.ceph.osd.6.smithi170.stderr:2023-10-26T01:30:27.891+0000 7f9a8d80d500 -1 osd.6 0 log_to_monitors true 2023-10-26T01:30:27.894 INFO:tasks.ceph.osd.1.smithi169.stderr:2023-10-26T01:30:27.892+0000 7ff8aeea9500 -1 osd.1 0 log_to_monitors true 2023-10-26T01:30:27.896 INFO:tasks.ceph.osd.4.smithi170.stderr:2023-10-26T01:30:27.894+0000 7f70dfb27500 -1 Falling back to public interface 2023-10-26T01:30:27.904 INFO:tasks.ceph.osd.5.smithi170.stderr:2023-10-26T01:30:27.901+0000 7fd459210500 -1 osd.5 0 log_to_monitors true 2023-10-26T01:30:27.904 INFO:tasks.ceph.osd.7.smithi170.stderr:2023-10-26T01:30:27.902+0000 7f4e7672b500 -1 Falling back to public interface 2023-10-26T01:30:27.907 INFO:tasks.ceph.mgr.x.smithi170.stderr:2023-10-26T01:30:27.905+0000 7f2a20213040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-26T01:30:27.911 INFO:tasks.ceph.osd.3.smithi169.stderr:2023-10-26T01:30:27.910+0000 7fee96005500 -1 Falling back to public interface 2023-10-26T01:30:27.918 INFO:tasks.ceph.osd.0.smithi169.stderr:2023-10-26T01:30:27.917+0000 7f61c78da500 -1 osd.0 0 log_to_monitors true 2023-10-26T01:30:27.929 INFO:tasks.ceph.osd.7.smithi170.stderr:2023-10-26T01:30:27.927+0000 7f4e7672b500 -1 osd.7 0 log_to_monitors true 2023-10-26T01:30:27.931 INFO:tasks.ceph.osd.4.smithi170.stderr:2023-10-26T01:30:27.929+0000 7f70dfb27500 -1 osd.4 0 log_to_monitors true 2023-10-26T01:30:27.951 INFO:tasks.ceph.osd.3.smithi169.stderr:2023-10-26T01:30:27.949+0000 7fee96005500 -1 osd.3 0 log_to_monitors true 2023-10-26T01:30:27.976 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:27.974+0000 7f39aa294640 1 -- 172.21.15.169:0/3732787972 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f39a406e050 msgr2=0x7f39a41665d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:27.976 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:27.974+0000 7f39aa294640 1 --2- 172.21.15.169:0/3732787972 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f39a406e050 0x7f39a41665d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f399c005fd0 tx=0x7f399c02c170 comp rx=0 tx=0).stop 2023-10-26T01:30:27.976 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:27.974+0000 7f39aa294640 1 -- 172.21.15.169:0/3732787972 shutdown_connections 2023-10-26T01:30:27.977 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:27.974+0000 7f39aa294640 1 --2- 172.21.15.169:0/3732787972 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f39a406e050 0x7f39a41665d0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:27.977 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:27.974+0000 7f39aa294640 1 -- 172.21.15.169:0/3732787972 >> 172.21.15.169:0/3732787972 conn(0x7f39a4098a60 msgr2=0x7f39a40b4970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:27.977 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:27.974+0000 7f39aa294640 1 -- 172.21.15.169:0/3732787972 shutdown_connections 2023-10-26T01:30:27.977 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:27.974+0000 7f39aa294640 1 -- 172.21.15.169:0/3732787972 wait complete. 2023-10-26T01:30:27.978 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:27.976+0000 7f39aa294640 1 Processor -- start 2023-10-26T01:30:27.978 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:27.976+0000 7f39aa294640 1 -- start start 2023-10-26T01:30:27.979 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:27.977+0000 7f39aa294640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f39a406e050 0x7f39a4148570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:27.979 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:27.977+0000 7f39aa294640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f39a40b2e40 0x7f39a4153520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:27.979 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:27.977+0000 7f39aa294640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f39a415ebd0 0x7f39a4156a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:27.979 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:27.977+0000 7f39aa294640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39a415c360 con 0x7f39a415ebd0 2023-10-26T01:30:27.979 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:27.977+0000 7f39aa294640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39a415d3c0 con 0x7f39a40b2e40 2023-10-26T01:30:27.979 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:27.977+0000 7f39aa294640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39a415c600 con 0x7f39a406e050 2023-10-26T01:30:27.981 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:27.977+0000 7f39a3fff640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f39a406e050 0x7f39a4148570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:27.981 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:27.977+0000 7f39a3fff640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f39a406e050 0x7f39a4148570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3301/0 says I am v2:172.21.15.169:55520/0 (socket says 172.21.15.169:55520) 2023-10-26T01:30:27.981 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:27.977+0000 7f39a3fff640 1 -- 172.21.15.169:0/2628438590 learned_addr learned my addr 172.21.15.169:0/2628438590 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:30:27.982 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:27.977+0000 7f39a37fe640 1 --2- 172.21.15.169:0/2628438590 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f39a40b2e40 0x7f39a4153520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:27.982 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:27.977+0000 7f39a8831640 1 --2- 172.21.15.169:0/2628438590 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f39a415ebd0 0x7f39a4156a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:27.982 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:27.978+0000 7f39a3fff640 1 -- 172.21.15.169:0/2628438590 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f39a40b2e40 msgr2=0x7f39a4153520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:27.982 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:27.978+0000 7f39a3fff640 1 --2- 172.21.15.169:0/2628438590 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f39a40b2e40 0x7f39a4153520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:27.982 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:27.978+0000 7f39a3fff640 1 -- 172.21.15.169:0/2628438590 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f39a415ebd0 msgr2=0x7f39a4156a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:27.983 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:27.978+0000 7f39a3fff640 1 --2- 172.21.15.169:0/2628438590 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f39a415ebd0 0x7f39a4156a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:27.983 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:27.978+0000 7f39a3fff640 1 -- 172.21.15.169:0/2628438590 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f399c009840 con 0x7f39a406e050 2023-10-26T01:30:27.983 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:27.978+0000 7f39a3fff640 1 --2- 172.21.15.169:0/2628438590 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f39a406e050 0x7f39a4148570 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f399400c390 tx=0x7f399400c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:27.983 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:27.978+0000 7f39a0ff9640 1 -- 172.21.15.169:0/2628438590 <== mon.2 v2:172.21.15.170:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3994004230 con 0x7f39a406e050 2023-10-26T01:30:27.983 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:27.978+0000 7f39a0ff9640 1 -- 172.21.15.169:0/2628438590 <== mon.2 v2:172.21.15.170:3301/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f39940044a0 con 0x7f39a406e050 2023-10-26T01:30:27.984 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:27.978+0000 7f39a0ff9640 1 -- 172.21.15.169:0/2628438590 <== mon.2 v2:172.21.15.170:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3994003850 con 0x7f39a406e050 2023-10-26T01:30:27.984 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:27.978+0000 7f39aa294640 1 -- 172.21.15.169:0/2628438590 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39a415c8e0 con 0x7f39a406e050 2023-10-26T01:30:27.984 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:27.978+0000 7f39aa294640 1 -- 172.21.15.169:0/2628438590 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39a415cea0 con 0x7f39a406e050 2023-10-26T01:30:27.984 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:27.978+0000 7f39a0ff9640 1 -- 172.21.15.169:0/2628438590 <== mon.2 v2:172.21.15.170:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f3994003cf0 con 0x7f39a406e050 2023-10-26T01:30:27.984 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:27.979+0000 7f39a0ff9640 1 -- 172.21.15.169:0/2628438590 <== mon.2 v2:172.21.15.170:3301/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f3994011c60 con 0x7f39a406e050 2023-10-26T01:30:27.984 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:27.979+0000 7f39aa294640 1 -- 172.21.15.169:0/2628438590 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39a4066450 con 0x7f39a406e050 2023-10-26T01:30:27.985 INFO:tasks.ceph.mgr.x.smithi170.stderr:2023-10-26T01:30:27.979+0000 7f2a20213040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-26T01:30:27.985 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:27.982+0000 7f39a0ff9640 1 -- 172.21.15.169:0/2628438590 <== mon.2 v2:172.21.15.170:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f39a4066450 con 0x7f39a406e050 2023-10-26T01:30:27.986 INFO:tasks.ceph.mgr.y.smithi169.stderr:2023-10-26T01:30:27.982+0000 7f33d344c040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-26T01:30:28.046 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:28.045+0000 7f39aa294640 1 -- 172.21.15.169:0/2628438590 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f39a415d0c0 con 0x7f39a406e050 2023-10-26T01:30:28.047 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:28.046+0000 7f39a0ff9640 1 -- 172.21.15.169:0/2628438590 <== mon.2 v2:172.21.15.170:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f3994017070 con 0x7f39a406e050 2023-10-26T01:30:28.048 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:30:28.048 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":10,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:30:27.637941+0000","last_up_change":"0.000000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:30:28.049 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:28.047+0000 7f39aa294640 1 -- 172.21.15.169:0/2628438590 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f39a406e050 msgr2=0x7f39a4148570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:28.049 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:28.047+0000 7f39aa294640 1 --2- 172.21.15.169:0/2628438590 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f39a406e050 0x7f39a4148570 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f399400c390 tx=0x7f399400c850 comp rx=0 tx=0).stop 2023-10-26T01:30:28.049 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:28.047+0000 7f39aa294640 1 -- 172.21.15.169:0/2628438590 shutdown_connections 2023-10-26T01:30:28.049 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:28.047+0000 7f39aa294640 1 --2- 172.21.15.169:0/2628438590 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f39a415ebd0 0x7f39a4156a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:28.050 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:28.047+0000 7f39aa294640 1 --2- 172.21.15.169:0/2628438590 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f39a40b2e40 0x7f39a4153520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:28.050 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:28.047+0000 7f39aa294640 1 --2- 172.21.15.169:0/2628438590 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f39a406e050 0x7f39a4148570 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:28.050 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:28.047+0000 7f39aa294640 1 -- 172.21.15.169:0/2628438590 >> 172.21.15.169:0/2628438590 conn(0x7f39a4098a60 msgr2=0x7f39a40afbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:28.050 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:28.047+0000 7f39aa294640 1 -- 172.21.15.169:0/2628438590 shutdown_connections 2023-10-26T01:30:28.051 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:28.047+0000 7f39aa294640 1 -- 172.21.15.169:0/2628438590 wait complete. 2023-10-26T01:30:28.064 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-10-26T01:30:28.064 INFO:tasks.ceph:Waiting for OSDs to come up 2023-10-26T01:30:28.095 INFO:tasks.ceph.mgr.x.smithi170.stderr:2023-10-26T01:30:28.093+0000 7f2a20213040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-26T01:30:28.304 INFO:tasks.ceph.mgr.y.smithi169.stderr:2023-10-26T01:30:28.302+0000 7f33d344c040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-26T01:30:28.363 INFO:tasks.ceph.mgr.y.smithi169.stderr:2023-10-26T01:30:28.362+0000 7f33d344c040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-26T01:30:28.478 INFO:tasks.ceph.mgr.y.smithi169.stderr:2023-10-26T01:30:28.477+0000 7f33d344c040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-26T01:30:28.869 DEBUG:teuthology.orchestra.run.smithi169:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-26T01:30:29.060 INFO:teuthology.misc.health.smithi169.stderr:2023-10-26T01:30:29.058+0000 7fc932af2640 1 -- 172.21.15.169:0/200811186 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc9140058d0 msgr2=0x7fc914005d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:29.061 INFO:teuthology.misc.health.smithi169.stderr:2023-10-26T01:30:29.058+0000 7fc932af2640 1 --2- 172.21.15.169:0/200811186 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc9140058d0 0x7fc914005d50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc91c0017d0 tx=0x7fc914006900 comp rx=0 tx=0).stop 2023-10-26T01:30:29.061 INFO:teuthology.misc.health.smithi169.stderr:2023-10-26T01:30:29.058+0000 7fc93108f640 1 -- 172.21.15.169:0/200811186 reap_dead start 2023-10-26T01:30:29.061 INFO:teuthology.misc.health.smithi169.stderr:2023-10-26T01:30:29.058+0000 7fc932af2640 1 -- 172.21.15.169:0/200811186 shutdown_connections 2023-10-26T01:30:29.062 INFO:teuthology.misc.health.smithi169.stderr:2023-10-26T01:30:29.058+0000 7fc932af2640 1 -- 172.21.15.169:0/200811186 >> 172.21.15.169:0/200811186 conn(0x7fc92c069930 msgr2=0x7fc92c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:29.062 INFO:teuthology.misc.health.smithi169.stderr:2023-10-26T01:30:29.058+0000 7fc932af2640 1 -- 172.21.15.169:0/200811186 shutdown_connections 2023-10-26T01:30:29.062 INFO:teuthology.misc.health.smithi169.stderr:2023-10-26T01:30:29.058+0000 7fc932af2640 1 -- 172.21.15.169:0/200811186 wait complete. 2023-10-26T01:30:29.066 INFO:teuthology.misc.health.smithi169.stderr:2023-10-26T01:30:29.061+0000 7fc932af2640 1 Processor -- start 2023-10-26T01:30:29.066 INFO:teuthology.misc.health.smithi169.stderr:2023-10-26T01:30:29.061+0000 7fc932af2640 1 -- start start 2023-10-26T01:30:29.067 INFO:teuthology.misc.health.smithi169.stderr:2023-10-26T01:30:29.062+0000 7fc932af2640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc9140058d0 0x7fc92c143cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:29.067 INFO:teuthology.misc.health.smithi169.stderr:2023-10-26T01:30:29.062+0000 7fc932af2640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc92c1513f0 0x7fc92c1e2740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:29.068 INFO:teuthology.misc.health.smithi169.stderr:2023-10-26T01:30:29.062+0000 7fc932af2640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc92c0b1160 0x7fc92c0b15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:29.068 INFO:teuthology.misc.health.smithi169.stderr:2023-10-26T01:30:29.062+0000 7fc932af2640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc92c0b1b20 con 0x7fc92c0b1160 2023-10-26T01:30:29.068 INFO:teuthology.misc.health.smithi169.stderr:2023-10-26T01:30:29.062+0000 7fc932af2640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc92c1dfe10 con 0x7fc92c1513f0 2023-10-26T01:30:29.069 INFO:teuthology.misc.health.smithi169.stderr:2023-10-26T01:30:29.062+0000 7fc932af2640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc92c1e2c80 con 0x7fc9140058d0 2023-10-26T01:30:29.069 INFO:teuthology.misc.health.smithi169.stderr:2023-10-26T01:30:29.062+0000 7fc93108f640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc92c0b1160 0x7fc92c0b15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:29.069 INFO:teuthology.misc.health.smithi169.stderr:2023-10-26T01:30:29.062+0000 7fc93108f640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc92c0b1160 0x7fc92c0b15e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:54312/0 (socket says 172.21.15.169:54312) 2023-10-26T01:30:29.069 INFO:teuthology.misc.health.smithi169.stderr:2023-10-26T01:30:29.062+0000 7fc93108f640 1 -- 172.21.15.169:0/305568507 learned_addr learned my addr 172.21.15.169:0/305568507 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:30:29.070 INFO:teuthology.misc.health.smithi169.stderr:2023-10-26T01:30:29.063+0000 7fc93088e640 1 --2- 172.21.15.169:0/305568507 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc9140058d0 0x7fc92c143cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:29.070 INFO:teuthology.misc.health.smithi169.stderr:2023-10-26T01:30:29.063+0000 7fc93108f640 1 -- 172.21.15.169:0/305568507 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc9140058d0 msgr2=0x7fc92c143cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:29.070 INFO:teuthology.misc.health.smithi169.stderr:2023-10-26T01:30:29.063+0000 7fc93108f640 1 --2- 172.21.15.169:0/305568507 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc9140058d0 0x7fc92c143cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:29.070 INFO:teuthology.misc.health.smithi169.stderr:2023-10-26T01:30:29.063+0000 7fc93108f640 1 -- 172.21.15.169:0/305568507 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc92c1513f0 msgr2=0x7fc92c1e2740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:29.071 INFO:teuthology.misc.health.smithi169.stderr:2023-10-26T01:30:29.063+0000 7fc93108f640 1 --2- 172.21.15.169:0/305568507 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc92c1513f0 0x7fc92c1e2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:29.071 INFO:teuthology.misc.health.smithi169.stderr:2023-10-26T01:30:29.063+0000 7fc93108f640 1 -- 172.21.15.169:0/305568507 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc92c1519f0 con 0x7fc92c0b1160 2023-10-26T01:30:29.071 INFO:teuthology.misc.health.smithi169.stderr:2023-10-26T01:30:29.063+0000 7fc93108f640 1 --2- 172.21.15.169:0/305568507 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc92c0b1160 0x7fc92c0b15e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc92c06ca80 tx=0x7fc92400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:29.071 INFO:teuthology.misc.health.smithi169.stderr:2023-10-26T01:30:29.063+0000 7fc9217fa640 1 -- 172.21.15.169:0/305568507 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc92400acb0 con 0x7fc92c0b1160 2023-10-26T01:30:29.071 INFO:teuthology.misc.health.smithi169.stderr:2023-10-26T01:30:29.063+0000 7fc9217fa640 1 -- 172.21.15.169:0/305568507 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc9240070d0 con 0x7fc92c0b1160 2023-10-26T01:30:29.071 INFO:teuthology.misc.health.smithi169.stderr:2023-10-26T01:30:29.063+0000 7fc9217fa640 1 -- 172.21.15.169:0/305568507 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9240074e0 con 0x7fc92c0b1160 2023-10-26T01:30:29.072 INFO:teuthology.misc.health.smithi169.stderr:2023-10-26T01:30:29.063+0000 7fc932af2640 1 -- 172.21.15.169:0/305568507 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc92c1e0060 con 0x7fc92c0b1160 2023-10-26T01:30:29.072 INFO:teuthology.misc.health.smithi169.stderr:2023-10-26T01:30:29.063+0000 7fc932af2640 1 -- 172.21.15.169:0/305568507 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc92c1e0620 con 0x7fc92c0b1160 2023-10-26T01:30:29.072 INFO:teuthology.misc.health.smithi169.stderr:2023-10-26T01:30:29.065+0000 7fc932af2640 1 -- 172.21.15.169:0/305568507 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc92c066e60 con 0x7fc92c0b1160 2023-10-26T01:30:29.073 INFO:teuthology.misc.health.smithi169.stderr:2023-10-26T01:30:29.065+0000 7fc9217fa640 1 -- 172.21.15.169:0/305568507 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 2) v1 ==== 44566+0+0 (secure 0 0 0) 0x7fc924007640 con 0x7fc92c0b1160 2023-10-26T01:30:29.073 INFO:teuthology.misc.health.smithi169.stderr:2023-10-26T01:30:29.065+0000 7fc9217fa640 1 -- 172.21.15.169:0/305568507 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(11..11 src has 1..11) v4 ==== 1883+0+0 (secure 0 0 0) 0x7fc9240494d0 con 0x7fc92c0b1160 2023-10-26T01:30:29.073 INFO:teuthology.misc.health.smithi169.stderr:2023-10-26T01:30:29.067+0000 7fc9217fa640 1 -- 172.21.15.169:0/305568507 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7fc924018af0 con 0x7fc92c0b1160 2023-10-26T01:30:29.125 INFO:teuthology.misc.health.smithi169.stderr:2023-10-26T01:30:29.124+0000 7fc9217fa640 1 -- 172.21.15.169:0/305568507 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mgrmap(e 3) v1 ==== 89067+0+0 (secure 0 0 0) 0x7fc924018d20 con 0x7fc92c0b1160 2023-10-26T01:30:29.126 INFO:teuthology.misc.health.smithi169.stderr:2023-10-26T01:30:29.125+0000 7fc9217fa640 1 --2- 172.21.15.169:0/305568507 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc91806bc70 0x7fc91806e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:29.130 INFO:teuthology.misc.health.smithi169.stderr:2023-10-26T01:30:29.126+0000 7fc93088e640 1 --2- 172.21.15.169:0/305568507 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc91806bc70 0x7fc91806e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:29.131 INFO:teuthology.misc.health.smithi169.stderr:2023-10-26T01:30:29.128+0000 7fc93088e640 1 --2- 172.21.15.169:0/305568507 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc91806bc70 0x7fc91806e060 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fc928009920 tx=0x7fc928005f90 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:29.132 INFO:teuthology.misc.health.smithi169.stderr:2023-10-26T01:30:29.131+0000 7fc932af2640 1 -- 172.21.15.169:0/305568507 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc92c1e0810 con 0x7fc92c0b1160 2023-10-26T01:30:29.133 INFO:teuthology.misc.health.smithi169.stderr:2023-10-26T01:30:29.132+0000 7fc9217fa640 1 -- 172.21.15.169:0/305568507 <== mon.0 v2:172.21.15.169:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v11) v1 ==== 74+0+7057 (secure 0 0 0) 0x7fc924017070 con 0x7fc92c0b1160 2023-10-26T01:30:29.134 INFO:teuthology.misc.health.smithi169.stdout: 2023-10-26T01:30:29.134 INFO:teuthology.misc.health.smithi169.stdout:{"epoch":11,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:30:28.863765+0000","last_up_change":"0.000000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":3,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:30:29.135 INFO:teuthology.misc.health.smithi169.stderr:2023-10-26T01:30:29.133+0000 7fc932af2640 1 -- 172.21.15.169:0/305568507 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc91806bc70 msgr2=0x7fc91806e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:29.136 INFO:teuthology.misc.health.smithi169.stderr:2023-10-26T01:30:29.133+0000 7fc932af2640 1 --2- 172.21.15.169:0/305568507 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc91806bc70 0x7fc91806e060 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fc928009920 tx=0x7fc928005f90 comp rx=0 tx=0).stop 2023-10-26T01:30:29.136 INFO:teuthology.misc.health.smithi169.stderr:2023-10-26T01:30:29.133+0000 7fc932af2640 1 -- 172.21.15.169:0/305568507 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc92c0b1160 msgr2=0x7fc92c0b15e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:29.136 INFO:teuthology.misc.health.smithi169.stderr:2023-10-26T01:30:29.133+0000 7fc932af2640 1 --2- 172.21.15.169:0/305568507 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc92c0b1160 0x7fc92c0b15e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc92c06ca80 tx=0x7fc92400a430 comp rx=0 tx=0).stop 2023-10-26T01:30:29.136 INFO:teuthology.misc.health.smithi169.stderr:2023-10-26T01:30:29.133+0000 7fc932af2640 1 -- 172.21.15.169:0/305568507 shutdown_connections 2023-10-26T01:30:29.137 INFO:teuthology.misc.health.smithi169.stderr:2023-10-26T01:30:29.133+0000 7fc932af2640 1 --2- 172.21.15.169:0/305568507 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc91806bc70 0x7fc91806e060 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:29.137 INFO:teuthology.misc.health.smithi169.stderr:2023-10-26T01:30:29.133+0000 7fc932af2640 1 --2- 172.21.15.169:0/305568507 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc92c0b1160 0x7fc92c0b15e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:29.137 INFO:teuthology.misc.health.smithi169.stderr:2023-10-26T01:30:29.133+0000 7fc932af2640 1 --2- 172.21.15.169:0/305568507 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc92c1513f0 0x7fc92c1e2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:29.137 INFO:teuthology.misc.health.smithi169.stderr:2023-10-26T01:30:29.133+0000 7fc932af2640 1 --2- 172.21.15.169:0/305568507 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc9140058d0 0x7fc92c143cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:29.137 INFO:teuthology.misc.health.smithi169.stderr:2023-10-26T01:30:29.133+0000 7fc932af2640 1 -- 172.21.15.169:0/305568507 >> 172.21.15.169:0/305568507 conn(0x7fc92c069930 msgr2=0x7fc92c14b280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:29.138 INFO:teuthology.misc.health.smithi169.stderr:2023-10-26T01:30:29.133+0000 7fc932af2640 1 -- 172.21.15.169:0/305568507 shutdown_connections 2023-10-26T01:30:29.138 INFO:teuthology.misc.health.smithi169.stderr:2023-10-26T01:30:29.134+0000 7fc932af2640 1 -- 172.21.15.169:0/305568507 wait complete. 2023-10-26T01:30:29.147 INFO:tasks.ceph.mgr.x.smithi170.stderr:2023-10-26T01:30:29.145+0000 7f29c12d7640 -1 mgr.server handle_report got status from non-daemon mon.b 2023-10-26T01:30:29.148 INFO:tasks.ceph.mgr.x.smithi170.stderr:2023-10-26T01:30:29.145+0000 7f29c12d7640 -1 mgr.server handle_report got status from non-daemon mon.c 2023-10-26T01:30:29.149 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-10-26T01:30:29.911 INFO:tasks.ceph.osd.1.smithi169.stderr:2023-10-26T01:30:29.909+0000 7ff8aae35640 -1 osd.1 0 waiting for initial osdmap 2023-10-26T01:30:29.913 INFO:tasks.ceph.osd.3.smithi169.stderr:2023-10-26T01:30:29.911+0000 7fee91f91640 -1 osd.3 0 waiting for initial osdmap 2023-10-26T01:30:29.914 INFO:tasks.ceph.osd.1.smithi169.stderr:2023-10-26T01:30:29.913+0000 7ff8a4469640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-26T01:30:29.915 INFO:tasks.ceph.osd.3.smithi169.stderr:2023-10-26T01:30:29.914+0000 7fee8adc4640 -1 osd.3 12 osdmap fullness state needs update 2023-10-26T01:30:29.915 INFO:tasks.ceph.osd.7.smithi170.stderr:2023-10-26T01:30:29.913+0000 7f4e726b9640 -1 osd.7 0 waiting for initial osdmap 2023-10-26T01:30:29.919 INFO:tasks.ceph.osd.7.smithi170.stderr:2023-10-26T01:30:29.916+0000 7f4e6bced640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-26T01:30:29.919 INFO:tasks.ceph.osd.5.smithi170.stderr:2023-10-26T01:30:29.917+0000 7fd454b07640 -1 osd.5 0 waiting for initial osdmap 2023-10-26T01:30:29.920 INFO:tasks.ceph.osd.0.smithi169.stderr:2023-10-26T01:30:29.919+0000 7f61c3868640 -1 osd.0 0 waiting for initial osdmap 2023-10-26T01:30:29.921 INFO:tasks.ceph.osd.5.smithi170.stderr:2023-10-26T01:30:29.919+0000 7fd44e13b640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-26T01:30:29.923 INFO:tasks.ceph.osd.0.smithi169.stderr:2023-10-26T01:30:29.921+0000 7f61bce9c640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-26T01:30:29.925 INFO:tasks.ceph.osd.2.smithi169.stderr:2023-10-26T01:30:29.924+0000 7fe4c62bc640 -1 osd.2 0 waiting for initial osdmap 2023-10-26T01:30:29.928 INFO:tasks.ceph.osd.2.smithi169.stderr:2023-10-26T01:30:29.927+0000 7fe4bf0da640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-26T01:30:29.928 INFO:tasks.ceph.osd.4.smithi170.stderr:2023-10-26T01:30:29.926+0000 7f70dbab3640 -1 osd.4 0 waiting for initial osdmap 2023-10-26T01:30:29.932 INFO:tasks.ceph.osd.4.smithi170.stderr:2023-10-26T01:30:29.929+0000 7f70d50e7640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-26T01:30:29.933 INFO:tasks.ceph.osd.6.smithi170.stderr:2023-10-26T01:30:29.931+0000 7f9a891da640 -1 osd.6 0 waiting for initial osdmap 2023-10-26T01:30:29.936 INFO:tasks.ceph.osd.6.smithi170.stderr:2023-10-26T01:30:29.934+0000 7f9a8280e640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-26T01:30:30.884 INFO:tasks.ceph.osd.3.smithi169.stderr:2023-10-26T01:30:30.880+0000 7fee8b5c5640 -1 osd.3 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-26T01:30:31.876 INFO:tasks.ceph.mon.a.smithi169.stderr:2023-10-26T01:30:31.875+0000 7f158b279640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 full osd(s) (OSD_FULL) 2023-10-26T01:30:33.944 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T01:30:33.954 DEBUG:teuthology.orchestra.run.smithi170:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T01:30:35.955 DEBUG:teuthology.orchestra.run.smithi169:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-26T01:30:36.145 INFO:teuthology.misc.health.smithi169.stderr:2023-10-26T01:30:36.144+0000 7fd129b0a640 1 -- 172.21.15.169:0/977517326 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd1140047c0 msgr2=0x7fd114004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:36.146 INFO:teuthology.misc.health.smithi169.stderr:2023-10-26T01:30:36.144+0000 7fd129b0a640 1 --2- 172.21.15.169:0/977517326 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd1140047c0 0x7fd114004c40 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd110001680 tx=0x7fd1140057f0 comp rx=0 tx=0).stop 2023-10-26T01:30:36.146 INFO:teuthology.misc.health.smithi169.stderr:2023-10-26T01:30:36.144+0000 7fd129b0a640 1 -- 172.21.15.169:0/977517326 shutdown_connections 2023-10-26T01:30:36.146 INFO:teuthology.misc.health.smithi169.stderr:2023-10-26T01:30:36.144+0000 7fd129b0a640 1 --2- 172.21.15.169:0/977517326 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd1140047c0 0x7fd114004c40 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:36.147 INFO:teuthology.misc.health.smithi169.stderr:2023-10-26T01:30:36.144+0000 7fd129b0a640 1 -- 172.21.15.169:0/977517326 >> 172.21.15.169:0/977517326 conn(0x7fd1240af650 msgr2=0x7fd1240afa30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:36.147 INFO:teuthology.misc.health.smithi169.stderr:2023-10-26T01:30:36.144+0000 7fd129b0a640 1 -- 172.21.15.169:0/977517326 shutdown_connections 2023-10-26T01:30:36.147 INFO:teuthology.misc.health.smithi169.stderr:2023-10-26T01:30:36.144+0000 7fd129b0a640 1 -- 172.21.15.169:0/977517326 wait complete. 2023-10-26T01:30:36.148 INFO:teuthology.misc.health.smithi169.stderr:2023-10-26T01:30:36.146+0000 7fd129b0a640 1 Processor -- start 2023-10-26T01:30:36.148 INFO:teuthology.misc.health.smithi169.stderr:2023-10-26T01:30:36.146+0000 7fd129b0a640 1 -- start start 2023-10-26T01:30:36.149 INFO:teuthology.misc.health.smithi169.stderr:2023-10-26T01:30:36.147+0000 7fd129b0a640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd1140047c0 0x7fd124151330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:36.149 INFO:teuthology.misc.health.smithi169.stderr:2023-10-26T01:30:36.147+0000 7fd129b0a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd124152b60 0x7fd1241e6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:36.149 INFO:teuthology.misc.health.smithi169.stderr:2023-10-26T01:30:36.147+0000 7fd129b0a640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd124152fe0 0x7fd1241e6540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:36.149 INFO:teuthology.misc.health.smithi169.stderr:2023-10-26T01:30:36.147+0000 7fd129b0a640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd124151870 con 0x7fd1140047c0 2023-10-26T01:30:36.150 INFO:teuthology.misc.health.smithi169.stderr:2023-10-26T01:30:36.147+0000 7fd129b0a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1240b12c0 con 0x7fd124152b60 2023-10-26T01:30:36.150 INFO:teuthology.misc.health.smithi169.stderr:2023-10-26T01:30:36.147+0000 7fd129b0a640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1240b15a0 con 0x7fd124152fe0 2023-10-26T01:30:36.150 INFO:teuthology.misc.health.smithi169.stderr:2023-10-26T01:30:36.147+0000 7fd128b08640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd1140047c0 0x7fd124151330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:36.150 INFO:teuthology.misc.health.smithi169.stderr:2023-10-26T01:30:36.147+0000 7fd128b08640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd1140047c0 0x7fd124151330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:33652/0 (socket says 172.21.15.169:33652) 2023-10-26T01:30:36.150 INFO:teuthology.misc.health.smithi169.stderr:2023-10-26T01:30:36.147+0000 7fd128b08640 1 -- 172.21.15.169:0/1614475602 learned_addr learned my addr 172.21.15.169:0/1614475602 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:30:36.152 INFO:teuthology.misc.health.smithi169.stderr:2023-10-26T01:30:36.147+0000 7fd123fff640 1 --2- 172.21.15.169:0/1614475602 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd124152b60 0x7fd1241e6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:36.152 INFO:teuthology.misc.health.smithi169.stderr:2023-10-26T01:30:36.147+0000 7fd128b08640 1 -- 172.21.15.169:0/1614475602 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd124152fe0 msgr2=0x7fd1241e6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:36.152 INFO:teuthology.misc.health.smithi169.stderr:2023-10-26T01:30:36.147+0000 7fd128b08640 1 --2- 172.21.15.169:0/1614475602 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd124152fe0 0x7fd1241e6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:36.152 INFO:teuthology.misc.health.smithi169.stderr:2023-10-26T01:30:36.147+0000 7fd128b08640 1 -- 172.21.15.169:0/1614475602 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd124152b60 msgr2=0x7fd1241e6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:36.153 INFO:teuthology.misc.health.smithi169.stderr:2023-10-26T01:30:36.147+0000 7fd128b08640 1 --2- 172.21.15.169:0/1614475602 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd124152b60 0x7fd1241e6000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:36.153 INFO:teuthology.misc.health.smithi169.stderr:2023-10-26T01:30:36.147+0000 7fd128b08640 1 -- 172.21.15.169:0/1614475602 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1240b1880 con 0x7fd1140047c0 2023-10-26T01:30:36.153 INFO:teuthology.misc.health.smithi169.stderr:2023-10-26T01:30:36.148+0000 7fd128b08640 1 --2- 172.21.15.169:0/1614475602 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd1140047c0 0x7fd124151330 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd11800c730 tx=0x7fd11800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:36.153 INFO:teuthology.misc.health.smithi169.stderr:2023-10-26T01:30:36.148+0000 7fd1217fa640 1 -- 172.21.15.169:0/1614475602 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd118005270 con 0x7fd1140047c0 2023-10-26T01:30:36.154 INFO:teuthology.misc.health.smithi169.stderr:2023-10-26T01:30:36.148+0000 7fd1217fa640 1 -- 172.21.15.169:0/1614475602 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd1180035f0 con 0x7fd1140047c0 2023-10-26T01:30:36.154 INFO:teuthology.misc.health.smithi169.stderr:2023-10-26T01:30:36.148+0000 7fd1217fa640 1 -- 172.21.15.169:0/1614475602 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd118003a40 con 0x7fd1140047c0 2023-10-26T01:30:36.154 INFO:teuthology.misc.health.smithi169.stderr:2023-10-26T01:30:36.148+0000 7fd129b0a640 1 -- 172.21.15.169:0/1614475602 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1241e3fa0 con 0x7fd1140047c0 2023-10-26T01:30:36.155 INFO:teuthology.misc.health.smithi169.stderr:2023-10-26T01:30:36.148+0000 7fd129b0a640 1 -- 172.21.15.169:0/1614475602 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1240b0ee0 con 0x7fd1140047c0 2023-10-26T01:30:36.155 INFO:teuthology.misc.health.smithi169.stderr:2023-10-26T01:30:36.149+0000 7fd129b0a640 1 -- 172.21.15.169:0/1614475602 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd124073c10 con 0x7fd1140047c0 2023-10-26T01:30:36.155 INFO:teuthology.misc.health.smithi169.stderr:2023-10-26T01:30:36.150+0000 7fd1217fa640 1 -- 172.21.15.169:0/1614475602 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 4) v1 ==== 89173+0+0 (secure 0 0 0) 0x7fd118003ba0 con 0x7fd1140047c0 2023-10-26T01:30:36.156 INFO:teuthology.misc.health.smithi169.stderr:2023-10-26T01:30:36.152+0000 7fd1217fa640 1 --2- 172.21.15.169:0/1614475602 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd10806adb0 0x7fd10806d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:36.156 INFO:teuthology.misc.health.smithi169.stderr:2023-10-26T01:30:36.152+0000 7fd1217fa640 1 -- 172.21.15.169:0/1614475602 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd118088b60 con 0x7fd1140047c0 2023-10-26T01:30:36.156 INFO:teuthology.misc.health.smithi169.stderr:2023-10-26T01:30:36.152+0000 7fd123fff640 1 --2- 172.21.15.169:0/1614475602 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd10806adb0 0x7fd10806d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:36.156 INFO:teuthology.misc.health.smithi169.stderr:2023-10-26T01:30:36.154+0000 7fd123fff640 1 --2- 172.21.15.169:0/1614475602 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd10806adb0 0x7fd10806d270 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd110034000 tx=0x7fd110039040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:36.159 INFO:teuthology.misc.health.smithi169.stderr:2023-10-26T01:30:36.158+0000 7fd1217fa640 1 -- 172.21.15.169:0/1614475602 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd118054030 con 0x7fd1140047c0 2023-10-26T01:30:36.350 INFO:teuthology.misc.health.smithi169.stderr:2023-10-26T01:30:36.349+0000 7fd129b0a640 1 -- 172.21.15.169:0/1614475602 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd1240733a0 con 0x7fd1140047c0 2023-10-26T01:30:36.352 INFO:teuthology.misc.health.smithi169.stderr:2023-10-26T01:30:36.351+0000 7fd1217fa640 1 -- 172.21.15.169:0/1614475602 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12898 (secure 0 0 0) 0x7fd118017020 con 0x7fd1140047c0 2023-10-26T01:30:36.352 INFO:teuthology.misc.health.smithi169.stdout: 2023-10-26T01:30:36.353 INFO:teuthology.misc.health.smithi169.stdout:{"epoch":17,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:30:34.918066+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:30:36.355 INFO:teuthology.misc.health.smithi169.stderr:2023-10-26T01:30:36.353+0000 7fd129b0a640 1 -- 172.21.15.169:0/1614475602 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd10806adb0 msgr2=0x7fd10806d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:36.355 INFO:teuthology.misc.health.smithi169.stderr:2023-10-26T01:30:36.353+0000 7fd129b0a640 1 --2- 172.21.15.169:0/1614475602 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd10806adb0 0x7fd10806d270 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd110034000 tx=0x7fd110039040 comp rx=0 tx=0).stop 2023-10-26T01:30:36.355 INFO:teuthology.misc.health.smithi169.stderr:2023-10-26T01:30:36.353+0000 7fd129b0a640 1 -- 172.21.15.169:0/1614475602 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd1140047c0 msgr2=0x7fd124151330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:36.355 INFO:teuthology.misc.health.smithi169.stderr:2023-10-26T01:30:36.353+0000 7fd129b0a640 1 --2- 172.21.15.169:0/1614475602 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd1140047c0 0x7fd124151330 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd11800c730 tx=0x7fd11800cbf0 comp rx=0 tx=0).stop 2023-10-26T01:30:36.355 INFO:teuthology.misc.health.smithi169.stderr:2023-10-26T01:30:36.353+0000 7fd129b0a640 1 -- 172.21.15.169:0/1614475602 shutdown_connections 2023-10-26T01:30:36.356 INFO:teuthology.misc.health.smithi169.stderr:2023-10-26T01:30:36.353+0000 7fd129b0a640 1 --2- 172.21.15.169:0/1614475602 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd10806adb0 0x7fd10806d270 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:36.356 INFO:teuthology.misc.health.smithi169.stderr:2023-10-26T01:30:36.353+0000 7fd129b0a640 1 --2- 172.21.15.169:0/1614475602 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd124152fe0 0x7fd1241e6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:36.356 INFO:teuthology.misc.health.smithi169.stderr:2023-10-26T01:30:36.353+0000 7fd129b0a640 1 --2- 172.21.15.169:0/1614475602 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd124152b60 0x7fd1241e6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:36.356 INFO:teuthology.misc.health.smithi169.stderr:2023-10-26T01:30:36.353+0000 7fd129b0a640 1 --2- 172.21.15.169:0/1614475602 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd1140047c0 0x7fd124151330 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:36.356 INFO:teuthology.misc.health.smithi169.stderr:2023-10-26T01:30:36.353+0000 7fd129b0a640 1 -- 172.21.15.169:0/1614475602 >> 172.21.15.169:0/1614475602 conn(0x7fd1240af650 msgr2=0x7fd1240cc590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:36.357 INFO:teuthology.misc.health.smithi169.stderr:2023-10-26T01:30:36.353+0000 7fd129b0a640 1 -- 172.21.15.169:0/1614475602 shutdown_connections 2023-10-26T01:30:36.357 INFO:teuthology.misc.health.smithi169.stderr:2023-10-26T01:30:36.353+0000 7fd129b0a640 1 -- 172.21.15.169:0/1614475602 wait complete. 2023-10-26T01:30:36.369 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-10-26T01:30:36.370 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-10-26T01:30:36.370 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-26T01:30:36.370 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-26T01:30:36.376 INFO:tasks.ceph.mds.a:Started 2023-10-26T01:30:36.377 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-26T01:30:36.377 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-26T01:30:36.382 INFO:tasks.ceph.mds.c:Started 2023-10-26T01:30:36.383 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-26T01:30:36.383 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-26T01:30:36.392 INFO:tasks.ceph.mds.b:Started 2023-10-26T01:30:36.392 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-26T01:30:36.393 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-26T01:30:36.401 INFO:tasks.ceph.mds.d:Started 2023-10-26T01:30:36.401 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-10-26T01:30:36.402 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:30:36.532 INFO:tasks.ceph.mds.a.smithi169.stdout:starting mds.a at 2023-10-26T01:30:36.533 INFO:tasks.ceph.mds.c.smithi169.stdout:starting mds.c at 2023-10-26T01:30:36.556 INFO:tasks.ceph.mds.b.smithi170.stdout:starting mds.b at 2023-10-26T01:30:36.557 INFO:tasks.ceph.mds.d.smithi170.stdout:starting mds.d at 2023-10-26T01:30:36.559 INFO:tasks.ceph.mds.b.smithi170.stderr:2023-10-26T01:30:36.557+0000 7fa22e4bd640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2023-10-26T01:30:36.597 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:36.595+0000 7f629f2ea640 1 -- 172.21.15.169:0/3041439867 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62980b4a30 msgr2=0x7f62980cc220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:36.597 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:36.595+0000 7f629f2ea640 1 --2- 172.21.15.169:0/3041439867 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62980b4a30 0x7f62980cc220 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6290005fd0 tx=0x7f629002c2e0 comp rx=0 tx=0).stop 2023-10-26T01:30:36.597 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:36.595+0000 7f629f2ea640 1 -- 172.21.15.169:0/3041439867 shutdown_connections 2023-10-26T01:30:36.597 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:36.595+0000 7f629f2ea640 1 --2- 172.21.15.169:0/3041439867 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62980b4a30 0x7f62980cc220 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:36.598 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:36.595+0000 7f629f2ea640 1 --2- 172.21.15.169:0/3041439867 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f62980b45d0 0x7f62980cbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:36.598 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:36.595+0000 7f629f2ea640 1 -- 172.21.15.169:0/3041439867 >> 172.21.15.169:0/3041439867 conn(0x7f62980d1360 msgr2=0x7f62980d1770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:36.598 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:36.595+0000 7f629f2ea640 1 -- 172.21.15.169:0/3041439867 shutdown_connections 2023-10-26T01:30:36.598 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:36.595+0000 7f629f2ea640 1 -- 172.21.15.169:0/3041439867 wait complete. 2023-10-26T01:30:36.599 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:36.597+0000 7f629f2ea640 1 Processor -- start 2023-10-26T01:30:36.600 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:36.597+0000 7f629f2ea640 1 -- start start 2023-10-26T01:30:36.600 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:36.598+0000 7f629f2ea640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f62980b45d0 0x7f62980aeb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:36.600 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:36.598+0000 7f629f2ea640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62980b4a30 0x7f62980b2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:36.600 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:36.598+0000 7f629f2ea640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f62980b68c0 0x7f62980b3410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:36.601 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:36.598+0000 7f629f2ea640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62980af060 con 0x7f62980b4a30 2023-10-26T01:30:36.601 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:36.598+0000 7f629f2ea640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62980ad560 con 0x7f62980b45d0 2023-10-26T01:30:36.601 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:36.598+0000 7f629f2ea640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62980ac950 con 0x7f62980b68c0 2023-10-26T01:30:36.601 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:36.598+0000 7f629dae7640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62980b4a30 0x7f62980b2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:36.601 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:36.598+0000 7f629dae7640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62980b4a30 0x7f62980b2ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:33704/0 (socket says 172.21.15.169:33704) 2023-10-26T01:30:36.602 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:36.598+0000 7f629dae7640 1 -- 172.21.15.169:0/2807349917 learned_addr learned my addr 172.21.15.169:0/2807349917 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:30:36.603 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:36.598+0000 7f629dae7640 1 -- 172.21.15.169:0/2807349917 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f62980b68c0 msgr2=0x7f62980b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:36.604 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:36.598+0000 7f629dae7640 1 --2- 172.21.15.169:0/2807349917 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f62980b68c0 0x7f62980b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:36.604 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:36.598+0000 7f629dae7640 1 -- 172.21.15.169:0/2807349917 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f62980b45d0 msgr2=0x7f62980aeb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:36.604 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:36.598+0000 7f629e2e8640 1 --2- 172.21.15.169:0/2807349917 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f62980b45d0 0x7f62980aeb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:36.604 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:36.598+0000 7f629dae7640 1 --2- 172.21.15.169:0/2807349917 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f62980b45d0 0x7f62980aeb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:36.604 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:36.599+0000 7f629dae7640 1 -- 172.21.15.169:0/2807349917 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6288009490 con 0x7f62980b4a30 2023-10-26T01:30:36.605 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:36.599+0000 7f629e2e8640 1 --2- 172.21.15.169:0/2807349917 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f62980b45d0 0x7f62980aeb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:30:36.605 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:36.599+0000 7f629dae7640 1 --2- 172.21.15.169:0/2807349917 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62980b4a30 0x7f62980b2ed0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f628800ea60 tx=0x7f628800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:36.605 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:36.599+0000 7f628effd640 1 -- 172.21.15.169:0/2807349917 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f628800cde0 con 0x7f62980b4a30 2023-10-26T01:30:36.605 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:36.599+0000 7f628effd640 1 -- 172.21.15.169:0/2807349917 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6288004020 con 0x7f62980b4a30 2023-10-26T01:30:36.605 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:36.599+0000 7f628effd640 1 -- 172.21.15.169:0/2807349917 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6288004410 con 0x7f62980b4a30 2023-10-26T01:30:36.606 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:36.599+0000 7f629f2ea640 1 -- 172.21.15.169:0/2807349917 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6290009900 con 0x7f62980b4a30 2023-10-26T01:30:36.606 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:36.599+0000 7f629f2ea640 1 -- 172.21.15.169:0/2807349917 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62980acf10 con 0x7f62980b4a30 2023-10-26T01:30:36.606 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:36.600+0000 7f629f2ea640 1 -- 172.21.15.169:0/2807349917 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6298073c10 con 0x7f62980b4a30 2023-10-26T01:30:36.606 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:36.601+0000 7f628effd640 1 -- 172.21.15.169:0/2807349917 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 4) v1 ==== 89173+0+0 (secure 0 0 0) 0x7f62880035f0 con 0x7f62980b4a30 2023-10-26T01:30:36.606 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:36.602+0000 7f628effd640 1 --2- 172.21.15.169:0/2807349917 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f627c06adb0 0x7f627c06d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:36.607 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:36.602+0000 7f628effd640 1 -- 172.21.15.169:0/2807349917 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f6288014070 con 0x7f62980b4a30 2023-10-26T01:30:36.607 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:36.603+0000 7f629e2e8640 1 --2- 172.21.15.169:0/2807349917 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f627c06adb0 0x7f627c06d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:36.608 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:36.603+0000 7f629e2e8640 1 --2- 172.21.15.169:0/2807349917 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f627c06adb0 0x7f627c06d270 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f6294001610 tx=0x7f6294009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:36.608 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:36.607+0000 7f628effd640 1 -- 172.21.15.169:0/2807349917 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f62880552f0 con 0x7f62980b4a30 2023-10-26T01:30:36.790 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:36.789+0000 7f629f2ea640 1 -- 172.21.15.169:0/2807349917 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f62980d1ee0 con 0x7f62980b4a30 2023-10-26T01:30:36.792 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:36.791+0000 7f628effd640 1 -- 172.21.15.169:0/2807349917 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12898 (secure 0 0 0) 0x7f6288058910 con 0x7f62980b4a30 2023-10-26T01:30:36.793 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:30:36.793 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":17,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:30:34.918066+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:30:36.795 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:36.793+0000 7f629f2ea640 1 -- 172.21.15.169:0/2807349917 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f627c06adb0 msgr2=0x7f627c06d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:36.795 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:36.793+0000 7f629f2ea640 1 --2- 172.21.15.169:0/2807349917 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f627c06adb0 0x7f627c06d270 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f6294001610 tx=0x7f6294009040 comp rx=0 tx=0).stop 2023-10-26T01:30:36.796 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:36.793+0000 7f629f2ea640 1 -- 172.21.15.169:0/2807349917 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62980b4a30 msgr2=0x7f62980b2ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:36.796 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:36.793+0000 7f629f2ea640 1 --2- 172.21.15.169:0/2807349917 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62980b4a30 0x7f62980b2ed0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f628800ea60 tx=0x7f628800ef20 comp rx=0 tx=0).stop 2023-10-26T01:30:36.796 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:36.794+0000 7f629f2ea640 1 -- 172.21.15.169:0/2807349917 shutdown_connections 2023-10-26T01:30:36.796 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:36.794+0000 7f629f2ea640 1 --2- 172.21.15.169:0/2807349917 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f627c06adb0 0x7f627c06d270 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:36.796 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:36.794+0000 7f629f2ea640 1 --2- 172.21.15.169:0/2807349917 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62980b4a30 0x7f62980b2ed0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:36.797 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:36.794+0000 7f629f2ea640 1 --2- 172.21.15.169:0/2807349917 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f62980b68c0 0x7f62980b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:36.797 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:36.794+0000 7f629f2ea640 1 --2- 172.21.15.169:0/2807349917 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f62980b45d0 0x7f62980aeb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:36.797 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:36.794+0000 7f629f2ea640 1 -- 172.21.15.169:0/2807349917 >> 172.21.15.169:0/2807349917 conn(0x7f62980d1360 msgr2=0x7f62980cba90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:36.797 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:36.794+0000 7f629f2ea640 1 -- 172.21.15.169:0/2807349917 shutdown_connections 2023-10-26T01:30:36.797 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:36.794+0000 7f629f2ea640 1 -- 172.21.15.169:0/2807349917 wait complete. 2023-10-26T01:30:36.811 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T01:30:32.200808+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T01:30:36.812 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T01:30:37.009 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.007+0000 7f5af8ce5640 1 -- 172.21.15.169:0/2587895129 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5af40ac910 msgr2=0x7f5af406aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:37.009 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.007+0000 7f5af8ce5640 1 --2- 172.21.15.169:0/2587895129 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5af40ac910 0x7f5af406aed0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5aec005fd0 tx=0x7f5aec02c1b0 comp rx=0 tx=0).stop 2023-10-26T01:30:37.009 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.007+0000 7f5af8ce5640 1 -- 172.21.15.169:0/2587895129 shutdown_connections 2023-10-26T01:30:37.010 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.007+0000 7f5af8ce5640 1 --2- 172.21.15.169:0/2587895129 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5af40ac910 0x7f5af406aed0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:37.010 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.007+0000 7f5af8ce5640 1 --2- 172.21.15.169:0/2587895129 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5af40cdda0 0x7f5af40ac3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:37.010 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.007+0000 7f5af8ce5640 1 -- 172.21.15.169:0/2587895129 >> 172.21.15.169:0/2587895129 conn(0x7f5af4069930 msgr2=0x7f5af4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:37.010 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.007+0000 7f5af8ce5640 1 -- 172.21.15.169:0/2587895129 shutdown_connections 2023-10-26T01:30:37.010 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.007+0000 7f5af8ce5640 1 -- 172.21.15.169:0/2587895129 wait complete. 2023-10-26T01:30:37.012 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.009+0000 7f5af8ce5640 1 Processor -- start 2023-10-26T01:30:37.012 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.009+0000 7f5af8ce5640 1 -- start start 2023-10-26T01:30:37.012 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.010+0000 7f5af8ce5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5af40ac910 0x7f5af40afa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:37.013 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.010+0000 7f5af8ce5640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5af40cdda0 0x7f5af415bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:37.013 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.010+0000 7f5af8ce5640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5af415b6a0 0x7f5af4143280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:37.013 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.010+0000 7f5af8ce5640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5af41f55c0 con 0x7f5af415b6a0 2023-10-26T01:30:37.013 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.010+0000 7f5af8ce5640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5af40cb6e0 con 0x7f5af40ac910 2023-10-26T01:30:37.013 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.010+0000 7f5af8ce5640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5af4144140 con 0x7f5af40cdda0 2023-10-26T01:30:37.014 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.010+0000 7f5af2d9d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5af415b6a0 0x7f5af4143280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:37.014 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.010+0000 7f5af2d9d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5af415b6a0 0x7f5af4143280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:33724/0 (socket says 172.21.15.169:33724) 2023-10-26T01:30:37.014 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.010+0000 7f5af2d9d640 1 -- 172.21.15.169:0/379090063 learned_addr learned my addr 172.21.15.169:0/379090063 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:30:37.014 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.010+0000 7f5aebfff640 1 --2- 172.21.15.169:0/379090063 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5af40cdda0 0x7f5af415bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:37.014 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.010+0000 7f5af2d9d640 1 -- 172.21.15.169:0/379090063 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5af40cdda0 msgr2=0x7f5af415bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:37.015 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.010+0000 7f5af2d9d640 1 --2- 172.21.15.169:0/379090063 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5af40cdda0 0x7f5af415bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:37.015 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.010+0000 7f5af2d9d640 1 -- 172.21.15.169:0/379090063 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5af40ac910 msgr2=0x7f5af40afa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:37.015 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.010+0000 7f5af2d9d640 1 --2- 172.21.15.169:0/379090063 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5af40ac910 0x7f5af40afa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:37.015 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.010+0000 7f5af2d9d640 1 -- 172.21.15.169:0/379090063 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aec009900 con 0x7f5af415b6a0 2023-10-26T01:30:37.015 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.011+0000 7f5af2d9d640 1 --2- 172.21.15.169:0/379090063 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5af415b6a0 0x7f5af4143280 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5aec006010 tx=0x7f5aec035db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:37.017 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.011+0000 7f5aeaffd640 1 -- 172.21.15.169:0/379090063 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aec034440 con 0x7f5af415b6a0 2023-10-26T01:30:37.017 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.011+0000 7f5aeaffd640 1 -- 172.21.15.169:0/379090063 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5aec004540 con 0x7f5af415b6a0 2023-10-26T01:30:37.017 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.011+0000 7f5aeaffd640 1 -- 172.21.15.169:0/379090063 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aec03f440 con 0x7f5af415b6a0 2023-10-26T01:30:37.017 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.011+0000 7f5af8ce5640 1 -- 172.21.15.169:0/379090063 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5af40cb930 con 0x7f5af415b6a0 2023-10-26T01:30:37.017 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.011+0000 7f5af8ce5640 1 -- 172.21.15.169:0/379090063 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5af40cbef0 con 0x7f5af415b6a0 2023-10-26T01:30:37.018 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.012+0000 7f5af8ce5640 1 -- 172.21.15.169:0/379090063 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5af4066af0 con 0x7f5af415b6a0 2023-10-26T01:30:37.018 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.013+0000 7f5aeaffd640 1 -- 172.21.15.169:0/379090063 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 4) v1 ==== 89173+0+0 (secure 0 0 0) 0x7f5aec03b040 con 0x7f5af415b6a0 2023-10-26T01:30:37.018 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.014+0000 7f5aeaffd640 1 --2- 172.21.15.169:0/379090063 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5ad406ade0 0x7f5ad406d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:37.018 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.014+0000 7f5aeaffd640 1 -- 172.21.15.169:0/379090063 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5aec0ac130 con 0x7f5af415b6a0 2023-10-26T01:30:37.019 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.015+0000 7f5af259c640 1 --2- 172.21.15.169:0/379090063 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5ad406ade0 0x7f5ad406d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:37.019 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.016+0000 7f5af259c640 1 --2- 172.21.15.169:0/379090063 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5ad406ade0 0x7f5ad406d2a0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f5ae4006fd0 tx=0x7f5ae4006bc0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:37.020 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.019+0000 7f5aeaffd640 1 -- 172.21.15.169:0/379090063 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5aec077610 con 0x7f5af415b6a0 2023-10-26T01:30:37.034 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.033+0000 7f5aeaffd640 1 -- 172.21.15.169:0/379090063 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f5aec00a040 con 0x7f5af415b6a0 2023-10-26T01:30:37.198 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.196+0000 7f5af8ce5640 1 -- 172.21.15.169:0/379090063 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5af404da70 con 0x7f5af415b6a0 2023-10-26T01:30:37.199 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.197+0000 7f5aeaffd640 1 -- 172.21.15.169:0/379090063 <== mon.0 v2:172.21.15.169:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5aec043020 con 0x7f5af415b6a0 2023-10-26T01:30:37.199 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 1 2023-10-26T01:30:37.202 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.200+0000 7f5af8ce5640 1 -- 172.21.15.169:0/379090063 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5ad406ade0 msgr2=0x7f5ad406d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:37.202 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.200+0000 7f5af8ce5640 1 --2- 172.21.15.169:0/379090063 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5ad406ade0 0x7f5ad406d2a0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f5ae4006fd0 tx=0x7f5ae4006bc0 comp rx=0 tx=0).stop 2023-10-26T01:30:37.202 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.200+0000 7f5af8ce5640 1 -- 172.21.15.169:0/379090063 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5af415b6a0 msgr2=0x7f5af4143280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:37.202 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.200+0000 7f5af8ce5640 1 --2- 172.21.15.169:0/379090063 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5af415b6a0 0x7f5af4143280 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5aec006010 tx=0x7f5aec035db0 comp rx=0 tx=0).stop 2023-10-26T01:30:37.203 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.200+0000 7f5af8ce5640 1 -- 172.21.15.169:0/379090063 shutdown_connections 2023-10-26T01:30:37.203 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.200+0000 7f5af8ce5640 1 --2- 172.21.15.169:0/379090063 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5ad406ade0 0x7f5ad406d2a0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:37.203 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.200+0000 7f5af8ce5640 1 --2- 172.21.15.169:0/379090063 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5af415b6a0 0x7f5af4143280 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:37.203 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.200+0000 7f5af8ce5640 1 --2- 172.21.15.169:0/379090063 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5af40cdda0 0x7f5af415bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:37.203 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.200+0000 7f5af8ce5640 1 --2- 172.21.15.169:0/379090063 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5af40ac910 0x7f5af40afa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:37.204 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.200+0000 7f5af8ce5640 1 -- 172.21.15.169:0/379090063 >> 172.21.15.169:0/379090063 conn(0x7f5af4069930 msgr2=0x7f5af40714e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:37.204 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.200+0000 7f5af8ce5640 1 -- 172.21.15.169:0/379090063 shutdown_connections 2023-10-26T01:30:37.204 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.200+0000 7f5af8ce5640 1 -- 172.21.15.169:0/379090063 wait complete. 2023-10-26T01:30:37.219 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T01:30:37.440 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.438+0000 7fc216528640 1 -- 172.21.15.169:0/737835449 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc2100c9970 msgr2=0x7fc210098a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:37.440 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.438+0000 7fc216528640 1 --2- 172.21.15.169:0/737835449 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc2100c9970 0x7fc210098a60 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc200009830 tx=0x7fc20002bd60 comp rx=0 tx=0).stop 2023-10-26T01:30:37.441 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.438+0000 7fc216528640 1 -- 172.21.15.169:0/737835449 shutdown_connections 2023-10-26T01:30:37.441 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.438+0000 7fc216528640 1 --2- 172.21.15.169:0/737835449 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc21009a5c0 0x7fc21009aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:37.441 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.438+0000 7fc216528640 1 --2- 172.21.15.169:0/737835449 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc2100c9970 0x7fc210098a60 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:37.442 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.438+0000 7fc216528640 1 -- 172.21.15.169:0/737835449 >> 172.21.15.169:0/737835449 conn(0x7fc2100aedd0 msgr2=0x7fc2100af1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:37.442 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.438+0000 7fc216528640 1 -- 172.21.15.169:0/737835449 shutdown_connections 2023-10-26T01:30:37.442 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.438+0000 7fc216528640 1 -- 172.21.15.169:0/737835449 wait complete. 2023-10-26T01:30:37.443 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.440+0000 7fc216528640 1 Processor -- start 2023-10-26T01:30:37.443 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.441+0000 7fc216528640 1 -- start start 2023-10-26T01:30:37.445 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.442+0000 7fc216528640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc21009a5c0 0x7fc2100b0f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:37.445 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.442+0000 7fc216528640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc2100ac6d0 0x7fc2100c7b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:37.445 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.442+0000 7fc216528640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc2100c9970 0x7fc2100c80b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:37.445 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.442+0000 7fc216528640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc210159ba0 con 0x7fc21009a5c0 2023-10-26T01:30:37.446 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.442+0000 7fc216528640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2101f19d0 con 0x7fc2100ac6d0 2023-10-26T01:30:37.446 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.442+0000 7fc216528640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2101f53f0 con 0x7fc2100c9970 2023-10-26T01:30:37.446 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.442+0000 7fc215526640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc21009a5c0 0x7fc2100b0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:37.446 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.442+0000 7fc215526640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc21009a5c0 0x7fc2100b0f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:33748/0 (socket says 172.21.15.169:33748) 2023-10-26T01:30:37.446 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.442+0000 7fc215526640 1 -- 172.21.15.169:0/838668391 learned_addr learned my addr 172.21.15.169:0/838668391 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:30:37.447 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.443+0000 7fc215526640 1 -- 172.21.15.169:0/838668391 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc2100c9970 msgr2=0x7fc2100c80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:37.447 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.443+0000 7fc215526640 1 --2- 172.21.15.169:0/838668391 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc2100c9970 0x7fc2100c80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:37.447 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.443+0000 7fc215526640 1 -- 172.21.15.169:0/838668391 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc2100ac6d0 msgr2=0x7fc2100c7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:37.447 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.443+0000 7fc215526640 1 --2- 172.21.15.169:0/838668391 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc2100ac6d0 0x7fc2100c7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:37.447 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.443+0000 7fc215526640 1 -- 172.21.15.169:0/838668391 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc200009490 con 0x7fc21009a5c0 2023-10-26T01:30:37.448 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.443+0000 7fc215526640 1 --2- 172.21.15.169:0/838668391 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc21009a5c0 0x7fc2100b0f00 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc20c00c700 tx=0x7fc20c00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:37.448 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.443+0000 7fc205ffb640 1 -- 172.21.15.169:0/838668391 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc20c004230 con 0x7fc21009a5c0 2023-10-26T01:30:37.448 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.443+0000 7fc205ffb640 1 -- 172.21.15.169:0/838668391 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc20c0044a0 con 0x7fc21009a5c0 2023-10-26T01:30:37.448 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.443+0000 7fc216528640 1 -- 172.21.15.169:0/838668391 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2101f56d0 con 0x7fc21009a5c0 2023-10-26T01:30:37.449 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.443+0000 7fc205ffb640 1 -- 172.21.15.169:0/838668391 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc20c00a8a0 con 0x7fc21009a5c0 2023-10-26T01:30:37.449 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.443+0000 7fc216528640 1 -- 172.21.15.169:0/838668391 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2101f5c90 con 0x7fc21009a5c0 2023-10-26T01:30:37.450 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.444+0000 7fc216528640 1 -- 172.21.15.169:0/838668391 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc210073c10 con 0x7fc21009a5c0 2023-10-26T01:30:37.450 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.445+0000 7fc205ffb640 1 -- 172.21.15.169:0/838668391 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fc20c00aa00 con 0x7fc21009a5c0 2023-10-26T01:30:37.450 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.446+0000 7fc205ffb640 1 --2- 172.21.15.169:0/838668391 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc1f406ae70 0x7fc1f406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:37.450 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.446+0000 7fc205ffb640 1 -- 172.21.15.169:0/838668391 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fc20c089560 con 0x7fc21009a5c0 2023-10-26T01:30:37.451 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.447+0000 7fc214d25640 1 --2- 172.21.15.169:0/838668391 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc1f406ae70 0x7fc1f406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:37.451 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.448+0000 7fc214d25640 1 --2- 172.21.15.169:0/838668391 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc1f406ae70 0x7fc1f406d330 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc200036000 tx=0x7fc2000046c0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:37.452 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.450+0000 7fc205ffb640 1 -- 172.21.15.169:0/838668391 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc20c0548c0 con 0x7fc21009a5c0 2023-10-26T01:30:37.677 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.675+0000 7fc216528640 1 -- 172.21.15.169:0/838668391 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc2100ca410 con 0x7fc21009a5c0 2023-10-26T01:30:37.678 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.677+0000 7fc205ffb640 1 -- 172.21.15.169:0/838668391 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3703 (secure 0 0 0) 0x7fc2100ca410 con 0x7fc21009a5c0 2023-10-26T01:30:37.679 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:30:37.679 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4298,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1809801299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":1809801299},{"type":"v1","addr":"172.21.15.169:6835","nonce":1809801299}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4304,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/2248566141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2248566141},{"type":"v1","addr":"172.21.15.170:6836","nonce":2248566141}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4317,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/2383616530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2383616530},{"type":"v1","addr":"172.21.15.169:6834","nonce":2383616530}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4338,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/590609220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":590609220},{"type":"v1","addr":"172.21.15.170:6837","nonce":590609220}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2023-10-26T01:30:37.679 INFO:teuthology.orchestra.run.smithi169.stderr:dumped fsmap epoch 3 2023-10-26T01:30:37.681 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.679+0000 7fc216528640 1 -- 172.21.15.169:0/838668391 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc1f406ae70 msgr2=0x7fc1f406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:37.681 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.679+0000 7fc216528640 1 --2- 172.21.15.169:0/838668391 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc1f406ae70 0x7fc1f406d330 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc200036000 tx=0x7fc2000046c0 comp rx=0 tx=0).stop 2023-10-26T01:30:37.682 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.679+0000 7fc216528640 1 -- 172.21.15.169:0/838668391 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc21009a5c0 msgr2=0x7fc2100b0f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:37.682 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.679+0000 7fc216528640 1 --2- 172.21.15.169:0/838668391 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc21009a5c0 0x7fc2100b0f00 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc20c00c700 tx=0x7fc20c00cbc0 comp rx=0 tx=0).stop 2023-10-26T01:30:37.682 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.679+0000 7fc216528640 1 -- 172.21.15.169:0/838668391 shutdown_connections 2023-10-26T01:30:37.682 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.679+0000 7fc216528640 1 --2- 172.21.15.169:0/838668391 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc1f406ae70 0x7fc1f406d330 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:37.682 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.679+0000 7fc216528640 1 --2- 172.21.15.169:0/838668391 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc2100c9970 0x7fc2100c80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:37.683 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.679+0000 7fc216528640 1 --2- 172.21.15.169:0/838668391 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc2100ac6d0 0x7fc2100c7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:37.683 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.679+0000 7fc216528640 1 --2- 172.21.15.169:0/838668391 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc21009a5c0 0x7fc2100b0f00 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:37.683 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.679+0000 7fc216528640 1 -- 172.21.15.169:0/838668391 >> 172.21.15.169:0/838668391 conn(0x7fc2100aedd0 msgr2=0x7fc2100af1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:37.683 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.680+0000 7fc216528640 1 -- 172.21.15.169:0/838668391 shutdown_connections 2023-10-26T01:30:37.684 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.680+0000 7fc216528640 1 -- 172.21.15.169:0/838668391 wait complete. 2023-10-26T01:30:37.699 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:30:37.909 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.907+0000 7fa234b1d640 1 -- 172.21.15.169:0/452420111 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2300cdc60 msgr2=0x7fa2300cc440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:37.909 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.907+0000 7fa234b1d640 1 --2- 172.21.15.169:0/452420111 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2300cdc60 0x7fa2300cc440 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa228005fd0 tx=0x7fa22802c050 comp rx=0 tx=0).stop 2023-10-26T01:30:37.909 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.907+0000 7fa234b1d640 1 -- 172.21.15.169:0/452420111 shutdown_connections 2023-10-26T01:30:37.910 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.907+0000 7fa234b1d640 1 --2- 172.21.15.169:0/452420111 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2300cdc60 0x7fa2300cc440 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:37.910 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.907+0000 7fa234b1d640 1 --2- 172.21.15.169:0/452420111 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa23014f6a0 0x7fa2300cd720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:37.910 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.907+0000 7fa234b1d640 1 -- 172.21.15.169:0/452420111 >> 172.21.15.169:0/452420111 conn(0x7fa230069930 msgr2=0x7fa230069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:37.910 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.907+0000 7fa234b1d640 1 -- 172.21.15.169:0/452420111 shutdown_connections 2023-10-26T01:30:37.910 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.907+0000 7fa234b1d640 1 -- 172.21.15.169:0/452420111 wait complete. 2023-10-26T01:30:37.912 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.909+0000 7fa234b1d640 1 Processor -- start 2023-10-26T01:30:37.913 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.909+0000 7fa234b1d640 1 -- start start 2023-10-26T01:30:37.913 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.910+0000 7fa234b1d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2300cdc60 0x7fa2301ef370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:37.913 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.910+0000 7fa234b1d640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa23014f2c0 0x7fa23006f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:37.913 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.910+0000 7fa234b1d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa23014f6a0 0x7fa2301503e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:37.913 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.910+0000 7fa234b1d640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa23014a3a0 con 0x7fa2300cdc60 2023-10-26T01:30:37.914 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.910+0000 7fa234b1d640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa230070830 con 0x7fa23014f6a0 2023-10-26T01:30:37.914 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.910+0000 7fa234b1d640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa23014ff50 con 0x7fa23014f2c0 2023-10-26T01:30:37.914 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.910+0000 7fa22e59c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2300cdc60 0x7fa2301ef370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:37.914 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.910+0000 7fa22e59c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2300cdc60 0x7fa2301ef370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:33774/0 (socket says 172.21.15.169:33774) 2023-10-26T01:30:37.915 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.910+0000 7fa22e59c640 1 -- 172.21.15.169:0/1968809814 learned_addr learned my addr 172.21.15.169:0/1968809814 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:30:37.915 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.910+0000 7fa22e59c640 1 -- 172.21.15.169:0/1968809814 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa23014f2c0 msgr2=0x7fa23006f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:37.915 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.910+0000 7fa22e59c640 1 --2- 172.21.15.169:0/1968809814 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa23014f2c0 0x7fa23006f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:37.915 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.910+0000 7fa22e59c640 1 -- 172.21.15.169:0/1968809814 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa23014f6a0 msgr2=0x7fa2301503e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:37.916 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.910+0000 7fa22ed9d640 1 --2- 172.21.15.169:0/1968809814 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa23014f6a0 0x7fa2301503e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:37.916 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.910+0000 7fa22e59c640 1 --2- 172.21.15.169:0/1968809814 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa23014f6a0 0x7fa2301503e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:37.916 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.910+0000 7fa22e59c640 1 -- 172.21.15.169:0/1968809814 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa228009840 con 0x7fa2300cdc60 2023-10-26T01:30:37.916 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.911+0000 7fa22e59c640 1 --2- 172.21.15.169:0/1968809814 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2300cdc60 0x7fa2301ef370 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa22000c4e0 tx=0x7fa22000c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:37.916 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.911+0000 7fa226ffd640 1 -- 172.21.15.169:0/1968809814 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa220005480 con 0x7fa2300cdc60 2023-10-26T01:30:37.916 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.911+0000 7fa226ffd640 1 -- 172.21.15.169:0/1968809814 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa2200036e0 con 0x7fa2300cdc60 2023-10-26T01:30:37.917 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.911+0000 7fa226ffd640 1 -- 172.21.15.169:0/1968809814 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa220003af0 con 0x7fa2300cdc60 2023-10-26T01:30:37.917 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.911+0000 7fa234b1d640 1 -- 172.21.15.169:0/1968809814 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa23014fc20 con 0x7fa2300cdc60 2023-10-26T01:30:37.917 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.911+0000 7fa234b1d640 1 -- 172.21.15.169:0/1968809814 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa230070550 con 0x7fa2300cdc60 2023-10-26T01:30:37.918 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.912+0000 7fa234b1d640 1 -- 172.21.15.169:0/1968809814 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2301524f0 con 0x7fa2300cdc60 2023-10-26T01:30:37.918 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.913+0000 7fa226ffd640 1 -- 172.21.15.169:0/1968809814 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fa220003c50 con 0x7fa2300cdc60 2023-10-26T01:30:37.918 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.914+0000 7fa226ffd640 1 --2- 172.21.15.169:0/1968809814 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa21006ae40 0x7fa21006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:37.919 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.914+0000 7fa226ffd640 1 -- 172.21.15.169:0/1968809814 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa220088ba0 con 0x7fa2300cdc60 2023-10-26T01:30:37.919 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.915+0000 7fa22dd9b640 1 --2- 172.21.15.169:0/1968809814 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa21006ae40 0x7fa21006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:37.919 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.916+0000 7fa22dd9b640 1 --2- 172.21.15.169:0/1968809814 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa21006ae40 0x7fa21006d300 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa2301f2fc0 tx=0x7fa21800b370 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:37.920 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:37.919+0000 7fa226ffd640 1 -- 172.21.15.169:0/1968809814 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa220053f00 con 0x7fa2300cdc60 2023-10-26T01:30:38.098 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.096+0000 7fa234b1d640 1 -- 172.21.15.169:0/1968809814 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa23004db10 con 0x7fa2300cdc60 2023-10-26T01:30:38.099 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.098+0000 7fa226ffd640 1 -- 172.21.15.169:0/1968809814 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12898 (secure 0 0 0) 0x7fa220017070 con 0x7fa2300cdc60 2023-10-26T01:30:38.100 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:30:38.100 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":17,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:30:34.918066+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:30:38.103 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.101+0000 7fa234b1d640 1 -- 172.21.15.169:0/1968809814 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa21006ae40 msgr2=0x7fa21006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:38.103 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.101+0000 7fa234b1d640 1 --2- 172.21.15.169:0/1968809814 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa21006ae40 0x7fa21006d300 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa2301f2fc0 tx=0x7fa21800b370 comp rx=0 tx=0).stop 2023-10-26T01:30:38.103 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.101+0000 7fa234b1d640 1 -- 172.21.15.169:0/1968809814 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2300cdc60 msgr2=0x7fa2301ef370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:38.103 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.101+0000 7fa234b1d640 1 --2- 172.21.15.169:0/1968809814 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2300cdc60 0x7fa2301ef370 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa22000c4e0 tx=0x7fa22000c9a0 comp rx=0 tx=0).stop 2023-10-26T01:30:38.104 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.101+0000 7fa234b1d640 1 -- 172.21.15.169:0/1968809814 shutdown_connections 2023-10-26T01:30:38.104 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.101+0000 7fa234b1d640 1 --2- 172.21.15.169:0/1968809814 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa21006ae40 0x7fa21006d300 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:38.104 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.101+0000 7fa234b1d640 1 --2- 172.21.15.169:0/1968809814 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa23014f6a0 0x7fa2301503e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:38.104 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.101+0000 7fa234b1d640 1 --2- 172.21.15.169:0/1968809814 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa23014f2c0 0x7fa23006f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:38.104 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.101+0000 7fa234b1d640 1 --2- 172.21.15.169:0/1968809814 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2300cdc60 0x7fa2301ef370 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:38.105 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.101+0000 7fa234b1d640 1 -- 172.21.15.169:0/1968809814 >> 172.21.15.169:0/1968809814 conn(0x7fa230069930 msgr2=0x7fa230159240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:38.105 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.101+0000 7fa234b1d640 1 -- 172.21.15.169:0/1968809814 shutdown_connections 2023-10-26T01:30:38.105 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.101+0000 7fa234b1d640 1 -- 172.21.15.169:0/1968809814 wait complete. 2023-10-26T01:30:38.119 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T01:30:32.200808+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T01:30:38.120 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T01:30:38.339 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.337+0000 7f5cf7802640 1 -- 172.21.15.169:0/191302837 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5cf00cd760 msgr2=0x7f5cf00cdc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:38.340 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.337+0000 7f5cf7802640 1 --2- 172.21.15.169:0/191302837 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5cf00cd760 0x7f5cf00cdc10 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5cf00b0040 tx=0x7f5ce802c7c0 comp rx=0 tx=0).stop 2023-10-26T01:30:38.340 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.338+0000 7f5cf7802640 1 -- 172.21.15.169:0/191302837 shutdown_connections 2023-10-26T01:30:38.340 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.338+0000 7f5cf7802640 1 --2- 172.21.15.169:0/191302837 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5cf00cd760 0x7f5cf00cdc10 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:38.340 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.338+0000 7f5cf7802640 1 --2- 172.21.15.169:0/191302837 >> v2:172.21.15.170:3300/0 conn(0x7f5cf00b0a80 0x7f5cf0158f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:38.341 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.338+0000 7f5cf7802640 1 --2- 172.21.15.169:0/191302837 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5cf00ab070 0x7f5cf00ac870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:38.341 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.338+0000 7f5cf7802640 1 -- 172.21.15.169:0/191302837 >> 172.21.15.169:0/191302837 conn(0x7f5cf0069930 msgr2=0x7f5cf0069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:38.341 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.338+0000 7f5cf7802640 1 -- 172.21.15.169:0/191302837 shutdown_connections 2023-10-26T01:30:38.341 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.338+0000 7f5cf7802640 1 -- 172.21.15.169:0/191302837 wait complete. 2023-10-26T01:30:38.343 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.339+0000 7f5cf7802640 1 Processor -- start 2023-10-26T01:30:38.343 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.340+0000 7f5cf7802640 1 -- start start 2023-10-26T01:30:38.343 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.340+0000 7f5cf7802640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5cf00ab070 0x7f5cf0149f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:38.344 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.340+0000 7f5cf7802640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5cf00cd760 0x7f5cf0158b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:38.344 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.340+0000 7f5cf7802640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5cf00c8960 0x7f5cf00c8e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:38.344 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.340+0000 7f5cf7802640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cf015bc90 con 0x7f5cf00c8960 2023-10-26T01:30:38.344 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.340+0000 7f5cf7802640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cf00c9520 con 0x7f5cf00ab070 2023-10-26T01:30:38.344 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.340+0000 7f5cf7802640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cf015bf50 con 0x7f5cf00cd760 2023-10-26T01:30:38.344 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.341+0000 7f5cf5d9f640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5cf00c8960 0x7f5cf00c8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:38.345 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.341+0000 7f5cf5d9f640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5cf00c8960 0x7f5cf00c8e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:33790/0 (socket says 172.21.15.169:33790) 2023-10-26T01:30:38.345 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.341+0000 7f5cf5d9f640 1 -- 172.21.15.169:0/498550586 learned_addr learned my addr 172.21.15.169:0/498550586 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:30:38.345 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.341+0000 7f5cf559e640 1 --2- 172.21.15.169:0/498550586 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5cf00ab070 0x7f5cf0149f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:38.345 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.341+0000 7f5cf4d9d640 1 --2- 172.21.15.169:0/498550586 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5cf00cd760 0x7f5cf0158b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:38.345 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.341+0000 7f5cf5d9f640 1 -- 172.21.15.169:0/498550586 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5cf00cd760 msgr2=0x7f5cf0158b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:38.346 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.341+0000 7f5cf5d9f640 1 --2- 172.21.15.169:0/498550586 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5cf00cd760 0x7f5cf0158b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:38.346 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.341+0000 7f5cf5d9f640 1 -- 172.21.15.169:0/498550586 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5cf00ab070 msgr2=0x7f5cf0149f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:38.346 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.341+0000 7f5cf5d9f640 1 --2- 172.21.15.169:0/498550586 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5cf00ab070 0x7f5cf0149f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:38.346 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.341+0000 7f5cf5d9f640 1 -- 172.21.15.169:0/498550586 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ce800c4a0 con 0x7f5cf00c8960 2023-10-26T01:30:38.346 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.342+0000 7f5cf5d9f640 1 --2- 172.21.15.169:0/498550586 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5cf00c8960 0x7f5cf00c8e10 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5ce800c790 tx=0x7f5ce8032e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:38.349 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.342+0000 7f5ce1ffb640 1 -- 172.21.15.169:0/498550586 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce8032520 con 0x7f5cf00c8960 2023-10-26T01:30:38.349 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.342+0000 7f5ce1ffb640 1 -- 172.21.15.169:0/498550586 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5ce8009570 con 0x7f5cf00c8960 2023-10-26T01:30:38.349 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.342+0000 7f5ce1ffb640 1 -- 172.21.15.169:0/498550586 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce8009980 con 0x7f5cf00c8960 2023-10-26T01:30:38.349 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.342+0000 7f5cf7802640 1 -- 172.21.15.169:0/498550586 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cf015c230 con 0x7f5cf00c8960 2023-10-26T01:30:38.349 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.342+0000 7f5cf7802640 1 -- 172.21.15.169:0/498550586 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cf00c7390 con 0x7f5cf00c8960 2023-10-26T01:30:38.350 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.343+0000 7f5cf7802640 1 -- 172.21.15.169:0/498550586 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cf0066af0 con 0x7f5cf00c8960 2023-10-26T01:30:38.350 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.344+0000 7f5ce1ffb640 1 -- 172.21.15.169:0/498550586 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f5ce803b040 con 0x7f5cf00c8960 2023-10-26T01:30:38.350 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.345+0000 7f5ce1ffb640 1 --2- 172.21.15.169:0/498550586 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5cd406ae40 0x7f5cd406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:38.350 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.345+0000 7f5ce1ffb640 1 -- 172.21.15.169:0/498550586 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5ce80ab880 con 0x7f5cf00c8960 2023-10-26T01:30:38.350 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.346+0000 7f5cf559e640 1 --2- 172.21.15.169:0/498550586 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5cd406ae40 0x7f5cd406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:38.351 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.346+0000 7f5cf559e640 1 --2- 172.21.15.169:0/498550586 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5cd406ae40 0x7f5cd406d300 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5cec005910 tx=0x7f5cec0058a0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:38.351 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.349+0000 7f5ce1ffb640 1 -- 172.21.15.169:0/498550586 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5ce8076be0 con 0x7f5cf00c8960 2023-10-26T01:30:38.537 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.536+0000 7f5cf7802640 1 -- 172.21.15.169:0/498550586 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5cf00c7520 con 0x7f5cf00c8960 2023-10-26T01:30:38.538 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.537+0000 7f5ce1ffb640 1 -- 172.21.15.169:0/498550586 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5ce807a200 con 0x7f5cf00c8960 2023-10-26T01:30:38.538 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 1 2023-10-26T01:30:38.542 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.539+0000 7f5cf7802640 1 -- 172.21.15.169:0/498550586 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5cd406ae40 msgr2=0x7f5cd406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:38.542 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.539+0000 7f5cf7802640 1 --2- 172.21.15.169:0/498550586 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5cd406ae40 0x7f5cd406d300 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5cec005910 tx=0x7f5cec0058a0 comp rx=0 tx=0).stop 2023-10-26T01:30:38.542 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.540+0000 7f5cf7802640 1 -- 172.21.15.169:0/498550586 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5cf00c8960 msgr2=0x7f5cf00c8e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:38.542 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.540+0000 7f5cf7802640 1 --2- 172.21.15.169:0/498550586 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5cf00c8960 0x7f5cf00c8e10 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5ce800c790 tx=0x7f5ce8032e80 comp rx=0 tx=0).stop 2023-10-26T01:30:38.542 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.540+0000 7f5cf7802640 1 -- 172.21.15.169:0/498550586 shutdown_connections 2023-10-26T01:30:38.543 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.540+0000 7f5cf7802640 1 --2- 172.21.15.169:0/498550586 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5cd406ae40 0x7f5cd406d300 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:38.543 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.540+0000 7f5cf7802640 1 --2- 172.21.15.169:0/498550586 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5cf00c8960 0x7f5cf00c8e10 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:38.543 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.540+0000 7f5cf7802640 1 --2- 172.21.15.169:0/498550586 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5cf00cd760 0x7f5cf0158b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:38.543 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.540+0000 7f5cf7802640 1 --2- 172.21.15.169:0/498550586 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5cf00ab070 0x7f5cf0149f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:38.543 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.540+0000 7f5cf7802640 1 -- 172.21.15.169:0/498550586 >> 172.21.15.169:0/498550586 conn(0x7f5cf0069930 msgr2=0x7f5cf00ae3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:38.544 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.540+0000 7f5cf7802640 1 -- 172.21.15.169:0/498550586 shutdown_connections 2023-10-26T01:30:38.544 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.540+0000 7f5cf7802640 1 -- 172.21.15.169:0/498550586 wait complete. 2023-10-26T01:30:38.559 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-26T01:30:38.761 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.759+0000 7fe381da4640 1 -- 172.21.15.169:0/3478860639 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe37c06bf70 msgr2=0x7fe37c0aab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:38.762 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.759+0000 7fe381da4640 1 --2- 172.21.15.169:0/3478860639 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe37c06bf70 0x7fe37c0aab10 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe37c0ac880 tx=0x7fe36402bd70 comp rx=0 tx=0).stop 2023-10-26T01:30:38.762 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.760+0000 7fe381da4640 1 -- 172.21.15.169:0/3478860639 shutdown_connections 2023-10-26T01:30:38.762 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.760+0000 7fe381da4640 1 --2- 172.21.15.169:0/3478860639 >> v2:172.21.15.170:3300/0 conn(0x7fe37c06c420 0x7fe37c0ab050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:38.762 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.760+0000 7fe381da4640 1 --2- 172.21.15.169:0/3478860639 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe37c06bf70 0x7fe37c0aab10 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:38.763 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.760+0000 7fe381da4640 1 -- 172.21.15.169:0/3478860639 >> 172.21.15.169:0/3478860639 conn(0x7fe37c069930 msgr2=0x7fe37c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:38.763 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.760+0000 7fe381da4640 1 -- 172.21.15.169:0/3478860639 shutdown_connections 2023-10-26T01:30:38.763 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.760+0000 7fe381da4640 1 -- 172.21.15.169:0/3478860639 wait complete. 2023-10-26T01:30:38.765 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.761+0000 7fe381da4640 1 Processor -- start 2023-10-26T01:30:38.765 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.761+0000 7fe381da4640 1 -- start start 2023-10-26T01:30:38.765 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.762+0000 7fe381da4640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe37c06bf70 0x7fe37c143950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:38.766 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.762+0000 7fe381da4640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe37c06c420 0x7fe37c14f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:38.766 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.762+0000 7fe381da4640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe37c14ece0 0x7fe37c14f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:38.766 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.762+0000 7fe381da4640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe37c0ada60 con 0x7fe37c06bf70 2023-10-26T01:30:38.766 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.762+0000 7fe381da4640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe37c0adcb0 con 0x7fe37c14ece0 2023-10-26T01:30:38.766 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.762+0000 7fe381da4640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe37c0adf90 con 0x7fe37c06c420 2023-10-26T01:30:38.767 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.763+0000 7fe37b7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe37c06bf70 0x7fe37c143950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:38.767 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.763+0000 7fe37b7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe37c06bf70 0x7fe37c143950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:33810/0 (socket says 172.21.15.169:33810) 2023-10-26T01:30:38.767 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.763+0000 7fe37b7fe640 1 -- 172.21.15.169:0/542439474 learned_addr learned my addr 172.21.15.169:0/542439474 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:30:38.767 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.763+0000 7fe37b7fe640 1 -- 172.21.15.169:0/542439474 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe37c06c420 msgr2=0x7fe37c14f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:38.767 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.763+0000 7fe37b7fe640 1 --2- 172.21.15.169:0/542439474 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe37c06c420 0x7fe37c14f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:38.768 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.763+0000 7fe37b7fe640 1 -- 172.21.15.169:0/542439474 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe37c14ece0 msgr2=0x7fe37c14f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:38.768 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.763+0000 7fe37b7fe640 1 --2- 172.21.15.169:0/542439474 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe37c14ece0 0x7fe37c14f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:38.768 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.763+0000 7fe37b7fe640 1 -- 172.21.15.169:0/542439474 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe364009490 con 0x7fe37c06bf70 2023-10-26T01:30:38.768 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.763+0000 7fe37b7fe640 1 --2- 172.21.15.169:0/542439474 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe37c06bf70 0x7fe37c143950 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe36c00c3c0 tx=0x7fe36c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:38.769 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.763+0000 7fe378ff9640 1 -- 172.21.15.169:0/542439474 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe36c004230 con 0x7fe37c06bf70 2023-10-26T01:30:38.769 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.763+0000 7fe378ff9640 1 -- 172.21.15.169:0/542439474 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe36c0044a0 con 0x7fe37c06bf70 2023-10-26T01:30:38.769 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.764+0000 7fe378ff9640 1 -- 172.21.15.169:0/542439474 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe36c003850 con 0x7fe37c06bf70 2023-10-26T01:30:38.769 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.764+0000 7fe381da4640 1 -- 172.21.15.169:0/542439474 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe37c0ae270 con 0x7fe37c06bf70 2023-10-26T01:30:38.769 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.764+0000 7fe381da4640 1 -- 172.21.15.169:0/542439474 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe37c143e90 con 0x7fe37c06bf70 2023-10-26T01:30:38.770 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.765+0000 7fe381da4640 1 -- 172.21.15.169:0/542439474 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe37c0ae790 con 0x7fe37c06bf70 2023-10-26T01:30:38.771 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.765+0000 7fe378ff9640 1 -- 172.21.15.169:0/542439474 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fe36c0039b0 con 0x7fe37c06bf70 2023-10-26T01:30:38.771 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.766+0000 7fe378ff9640 1 --2- 172.21.15.169:0/542439474 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe35c06ae40 0x7fe35c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:38.771 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.766+0000 7fe378ff9640 1 -- 172.21.15.169:0/542439474 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fe36c088a00 con 0x7fe37c06bf70 2023-10-26T01:30:38.771 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.767+0000 7fe372ffd640 1 --2- 172.21.15.169:0/542439474 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe35c06ae40 0x7fe35c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:38.771 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.768+0000 7fe372ffd640 1 --2- 172.21.15.169:0/542439474 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe35c06ae40 0x7fe35c06d300 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe364036000 tx=0x7fe3640046c0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:38.772 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.770+0000 7fe378ff9640 1 -- 172.21.15.169:0/542439474 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe37c0ae790 con 0x7fe37c06bf70 2023-10-26T01:30:38.963 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.962+0000 7fe381da4640 1 -- 172.21.15.169:0/542439474 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fe37c0ae790 con 0x7fe37c06bf70 2023-10-26T01:30:38.964 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.962+0000 7fe378ff9640 1 -- 172.21.15.169:0/542439474 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) v1 ==== 84+0+65 (secure 0 0 0) 0x7fe37c0ae790 con 0x7fe37c06bf70 2023-10-26T01:30:38.964 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:30:38.965 INFO:teuthology.orchestra.run.smithi169.stdout:[ 2023-10-26T01:30:38.965 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:30:38.965 INFO:teuthology.orchestra.run.smithi169.stdout: "poolnum": 1, 2023-10-26T01:30:38.965 INFO:teuthology.orchestra.run.smithi169.stdout: "poolname": ".mgr" 2023-10-26T01:30:38.965 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:30:38.966 INFO:teuthology.orchestra.run.smithi169.stdout:] 2023-10-26T01:30:38.967 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.965+0000 7fe381da4640 1 -- 172.21.15.169:0/542439474 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe35c06ae40 msgr2=0x7fe35c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:38.967 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.965+0000 7fe381da4640 1 --2- 172.21.15.169:0/542439474 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe35c06ae40 0x7fe35c06d300 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe364036000 tx=0x7fe3640046c0 comp rx=0 tx=0).stop 2023-10-26T01:30:38.967 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.965+0000 7fe381da4640 1 -- 172.21.15.169:0/542439474 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe37c06bf70 msgr2=0x7fe37c143950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:38.967 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.965+0000 7fe381da4640 1 --2- 172.21.15.169:0/542439474 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe37c06bf70 0x7fe37c143950 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe36c00c3c0 tx=0x7fe36c00c880 comp rx=0 tx=0).stop 2023-10-26T01:30:38.968 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.965+0000 7fe381da4640 1 -- 172.21.15.169:0/542439474 shutdown_connections 2023-10-26T01:30:38.968 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.965+0000 7fe381da4640 1 --2- 172.21.15.169:0/542439474 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe35c06ae40 0x7fe35c06d300 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:38.968 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.965+0000 7fe381da4640 1 --2- 172.21.15.169:0/542439474 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe37c14ece0 0x7fe37c14f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:38.968 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.965+0000 7fe381da4640 1 --2- 172.21.15.169:0/542439474 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe37c06c420 0x7fe37c14f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:38.968 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.965+0000 7fe381da4640 1 --2- 172.21.15.169:0/542439474 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe37c06bf70 0x7fe37c143950 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:38.968 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.965+0000 7fe381da4640 1 -- 172.21.15.169:0/542439474 >> 172.21.15.169:0/542439474 conn(0x7fe37c069930 msgr2=0x7fe37c1594c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:38.969 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.965+0000 7fe381da4640 1 -- 172.21.15.169:0/542439474 shutdown_connections 2023-10-26T01:30:38.969 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:38.965+0000 7fe381da4640 1 -- 172.21.15.169:0/542439474 wait complete. 2023-10-26T01:30:38.984 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-26T01:30:38.984 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-26T01:30:39.199 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:39.197+0000 7f693af11640 1 -- 172.21.15.169:0/280444229 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f69340b1870 msgr2=0x7f69340b1d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:39.199 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:39.197+0000 7f693af11640 1 --2- 172.21.15.169:0/280444229 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f69340b1870 0x7f69340b1d00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f693406ce90 tx=0x7f692402bdb0 comp rx=0 tx=0).stop 2023-10-26T01:30:39.200 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:39.197+0000 7f693af11640 1 -- 172.21.15.169:0/280444229 shutdown_connections 2023-10-26T01:30:39.200 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:39.197+0000 7f693af11640 1 --2- 172.21.15.169:0/280444229 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f69340b1870 0x7f69340b1d00 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:39.200 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:39.197+0000 7f693af11640 1 --2- 172.21.15.169:0/280444229 >> v2:172.21.15.170:3300/0 conn(0x7f6934157c30 0x7f69340aa040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:39.200 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:39.197+0000 7f693af11640 1 -- 172.21.15.169:0/280444229 >> 172.21.15.169:0/280444229 conn(0x7f69340aedd0 msgr2=0x7f69340af1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:39.201 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:39.197+0000 7f693af11640 1 -- 172.21.15.169:0/280444229 shutdown_connections 2023-10-26T01:30:39.201 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:39.197+0000 7f693af11640 1 -- 172.21.15.169:0/280444229 wait complete. 2023-10-26T01:30:39.203 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:39.199+0000 7f693af11640 1 Processor -- start 2023-10-26T01:30:39.203 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:39.199+0000 7f693af11640 1 -- start start 2023-10-26T01:30:39.204 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:39.200+0000 7f693af11640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f69340b1870 0x7f6934155370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:39.204 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:39.200+0000 7f693af11640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f69340c9ab0 0x7f693406fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:39.204 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:39.200+0000 7f693af11640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6934157c30 0x7f6934070410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:39.204 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:39.200+0000 7f693af11640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6934070a20 con 0x7f69340c9ab0 2023-10-26T01:30:39.204 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:39.200+0000 7f693af11640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6934070d00 con 0x7f69340b1870 2023-10-26T01:30:39.205 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:39.200+0000 7f693af11640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6934070fe0 con 0x7f6934157c30 2023-10-26T01:30:39.205 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:39.200+0000 7f693970e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f69340c9ab0 0x7f693406fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:39.205 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:39.200+0000 7f693970e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f69340c9ab0 0x7f693406fed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:33820/0 (socket says 172.21.15.169:33820) 2023-10-26T01:30:39.205 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:39.200+0000 7f693970e640 1 -- 172.21.15.169:0/3173230823 learned_addr learned my addr 172.21.15.169:0/3173230823 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:30:39.206 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:39.201+0000 7f693970e640 1 -- 172.21.15.169:0/3173230823 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6934157c30 msgr2=0x7f6934070410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:39.206 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:39.201+0000 7f693970e640 1 --2- 172.21.15.169:0/3173230823 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6934157c30 0x7f6934070410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:39.206 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:39.201+0000 7f693970e640 1 -- 172.21.15.169:0/3173230823 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f69340b1870 msgr2=0x7f6934155370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:39.206 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:39.201+0000 7f6939f0f640 1 --2- 172.21.15.169:0/3173230823 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f69340b1870 0x7f6934155370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:39.206 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:39.201+0000 7f693970e640 1 --2- 172.21.15.169:0/3173230823 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f69340b1870 0x7f6934155370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:39.207 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:39.201+0000 7f693970e640 1 -- 172.21.15.169:0/3173230823 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6924009560 con 0x7f69340c9ab0 2023-10-26T01:30:39.207 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:39.201+0000 7f693970e640 1 --2- 172.21.15.169:0/3173230823 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f69340c9ab0 0x7f693406fed0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6924009850 tx=0x7f692402dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:39.207 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:39.201+0000 7f692a7fc640 1 -- 172.21.15.169:0/3173230823 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6924003800 con 0x7f69340c9ab0 2023-10-26T01:30:39.207 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:39.201+0000 7f692a7fc640 1 -- 172.21.15.169:0/3173230823 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6924003a50 con 0x7f69340c9ab0 2023-10-26T01:30:39.208 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:39.201+0000 7f692a7fc640 1 -- 172.21.15.169:0/3173230823 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6924003e20 con 0x7f69340c9ab0 2023-10-26T01:30:39.208 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:39.201+0000 7f693af11640 1 -- 172.21.15.169:0/3173230823 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69340712c0 con 0x7f69340c9ab0 2023-10-26T01:30:39.208 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:39.201+0000 7f693af11640 1 -- 172.21.15.169:0/3173230823 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6934096c60 con 0x7f69340c9ab0 2023-10-26T01:30:39.209 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:39.202+0000 7f693af11640 1 -- 172.21.15.169:0/3173230823 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69340717e0 con 0x7f69340c9ab0 2023-10-26T01:30:39.209 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:39.203+0000 7f692a7fc640 1 -- 172.21.15.169:0/3173230823 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f692403e450 con 0x7f69340c9ab0 2023-10-26T01:30:39.209 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:39.204+0000 7f692a7fc640 1 --2- 172.21.15.169:0/3173230823 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f691806ae40 0x7f691806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:39.210 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:39.204+0000 7f692a7fc640 1 -- 172.21.15.169:0/3173230823 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f69240abe90 con 0x7f69340c9ab0 2023-10-26T01:30:39.210 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:39.205+0000 7f6939f0f640 1 --2- 172.21.15.169:0/3173230823 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f691806ae40 0x7f691806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:39.210 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:39.206+0000 7f6939f0f640 1 --2- 172.21.15.169:0/3173230823 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f691806ae40 0x7f691806d300 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f69300058c0 tx=0x7f6930005850 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:39.211 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:39.208+0000 7f692a7fc640 1 -- 172.21.15.169:0/3173230823 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f69340717e0 con 0x7f69340c9ab0 2023-10-26T01:30:39.385 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:39.384+0000 7f693af11640 1 -- 172.21.15.169:0/3173230823 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f69340717e0 con 0x7f69340c9ab0 2023-10-26T01:30:40.060 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:40.058+0000 7f692a7fc640 1 -- 172.21.15.169:0/3173230823 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) v1 ==== 138+0+0 (secure 0 0 0) 0x7f69340717e0 con 0x7f69340c9ab0 2023-10-26T01:30:40.061 INFO:teuthology.orchestra.run.smithi169.stderr:pool 'cephfs_metadata' created 2023-10-26T01:30:40.063 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:40.062+0000 7f693af11640 1 -- 172.21.15.169:0/3173230823 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f691806ae40 msgr2=0x7f691806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:40.064 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:40.062+0000 7f693af11640 1 --2- 172.21.15.169:0/3173230823 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f691806ae40 0x7f691806d300 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f69300058c0 tx=0x7f6930005850 comp rx=0 tx=0).stop 2023-10-26T01:30:40.064 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:40.062+0000 7f693af11640 1 -- 172.21.15.169:0/3173230823 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f69340c9ab0 msgr2=0x7f693406fed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:40.064 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:40.062+0000 7f693af11640 1 --2- 172.21.15.169:0/3173230823 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f69340c9ab0 0x7f693406fed0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6924009850 tx=0x7f692402dae0 comp rx=0 tx=0).stop 2023-10-26T01:30:40.065 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:40.062+0000 7f693af11640 1 -- 172.21.15.169:0/3173230823 shutdown_connections 2023-10-26T01:30:40.065 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:40.062+0000 7f693af11640 1 --2- 172.21.15.169:0/3173230823 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f691806ae40 0x7f691806d300 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:40.065 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:40.062+0000 7f693af11640 1 --2- 172.21.15.169:0/3173230823 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f69340c9ab0 0x7f693406fed0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:40.065 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:40.062+0000 7f693af11640 1 --2- 172.21.15.169:0/3173230823 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6934157c30 0x7f6934070410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:40.066 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:40.062+0000 7f693af11640 1 --2- 172.21.15.169:0/3173230823 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f69340b1870 0x7f6934155370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:40.066 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:40.062+0000 7f693af11640 1 -- 172.21.15.169:0/3173230823 >> 172.21.15.169:0/3173230823 conn(0x7f69340aedd0 msgr2=0x7f69341544d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:40.066 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:40.063+0000 7f693af11640 1 -- 172.21.15.169:0/3173230823 shutdown_connections 2023-10-26T01:30:40.066 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:40.063+0000 7f693af11640 1 -- 172.21.15.169:0/3173230823 wait complete. 2023-10-26T01:30:40.089 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-26T01:30:40.309 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:40.306+0000 7ffb6faf2640 1 -- 172.21.15.169:0/368851496 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffb6806d5b0 msgr2=0x7ffb6806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:40.309 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:40.306+0000 7ffb6faf2640 1 --2- 172.21.15.169:0/368851496 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffb6806d5b0 0x7ffb6806d990 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ffb68070bf0 tx=0x7ffb5c02c110 comp rx=0 tx=0).stop 2023-10-26T01:30:40.310 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:40.306+0000 7ffb6faf2640 1 -- 172.21.15.169:0/368851496 shutdown_connections 2023-10-26T01:30:40.310 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:40.306+0000 7ffb6faf2640 1 --2- 172.21.15.169:0/368851496 >> v2:172.21.15.170:3300/0 conn(0x7ffb680b2d60 0x7ffb680b3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:40.310 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:40.306+0000 7ffb6faf2640 1 --2- 172.21.15.169:0/368851496 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffb6806d5b0 0x7ffb6806d990 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:40.310 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:40.306+0000 7ffb6faf2640 1 -- 172.21.15.169:0/368851496 >> 172.21.15.169:0/368851496 conn(0x7ffb68098a60 msgr2=0x7ffb680b4a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:40.310 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:40.306+0000 7ffb6faf2640 1 -- 172.21.15.169:0/368851496 shutdown_connections 2023-10-26T01:30:40.311 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:40.306+0000 7ffb6faf2640 1 -- 172.21.15.169:0/368851496 wait complete. 2023-10-26T01:30:40.311 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:40.308+0000 7ffb6faf2640 1 Processor -- start 2023-10-26T01:30:40.311 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:40.308+0000 7ffb6faf2640 1 -- start start 2023-10-26T01:30:40.313 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:40.309+0000 7ffb6faf2640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffb6806b5f0 0x7ffb681e5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:40.313 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:40.309+0000 7ffb6faf2640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffb6806d5b0 0x7ffb681e6560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:40.314 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:40.309+0000 7ffb6faf2640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ffb680b2d60 0x7ffb681e4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:40.314 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:40.309+0000 7ffb6faf2640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb681524a0 con 0x7ffb6806b5f0 2023-10-26T01:30:40.314 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:40.309+0000 7ffb6faf2640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb68151920 con 0x7ffb6806d5b0 2023-10-26T01:30:40.314 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:40.309+0000 7ffb6faf2640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb68151c00 con 0x7ffb680b2d60 2023-10-26T01:30:40.314 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:40.309+0000 7ffb6d88e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffb6806b5f0 0x7ffb681e5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:40.315 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:40.309+0000 7ffb6d88e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffb6806b5f0 0x7ffb681e5700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:33834/0 (socket says 172.21.15.169:33834) 2023-10-26T01:30:40.315 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:40.309+0000 7ffb6d88e640 1 -- 172.21.15.169:0/3648947098 learned_addr learned my addr 172.21.15.169:0/3648947098 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:30:40.315 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:40.310+0000 7ffb6d08d640 1 --2- 172.21.15.169:0/3648947098 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffb6806d5b0 0x7ffb681e6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:40.315 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:40.310+0000 7ffb6e08f640 1 --2- 172.21.15.169:0/3648947098 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ffb680b2d60 0x7ffb681e4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:40.315 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:40.310+0000 7ffb6d88e640 1 -- 172.21.15.169:0/3648947098 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ffb680b2d60 msgr2=0x7ffb681e4df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:40.316 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:40.310+0000 7ffb6d88e640 1 --2- 172.21.15.169:0/3648947098 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ffb680b2d60 0x7ffb681e4df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:40.316 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:40.310+0000 7ffb6d88e640 1 -- 172.21.15.169:0/3648947098 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffb6806d5b0 msgr2=0x7ffb681e6560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:40.316 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:40.310+0000 7ffb6d88e640 1 --2- 172.21.15.169:0/3648947098 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffb6806d5b0 0x7ffb681e6560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:40.316 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:40.310+0000 7ffb6d88e640 1 -- 172.21.15.169:0/3648947098 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb5c009840 con 0x7ffb6806b5f0 2023-10-26T01:30:40.316 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:40.310+0000 7ffb6e08f640 1 --2- 172.21.15.169:0/3648947098 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ffb680b2d60 0x7ffb681e4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:30:40.317 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:40.310+0000 7ffb6d88e640 1 --2- 172.21.15.169:0/3648947098 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffb6806b5f0 0x7ffb681e5700 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c009ca0 tx=0x7ffb5c004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:40.317 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:40.311+0000 7ffb667fc640 1 -- 172.21.15.169:0/3648947098 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb5c004760 con 0x7ffb6806b5f0 2023-10-26T01:30:40.317 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:40.311+0000 7ffb667fc640 1 -- 172.21.15.169:0/3648947098 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ffb5c0345a0 con 0x7ffb6806b5f0 2023-10-26T01:30:40.317 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:40.311+0000 7ffb667fc640 1 -- 172.21.15.169:0/3648947098 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb5c034930 con 0x7ffb6806b5f0 2023-10-26T01:30:40.317 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:40.311+0000 7ffb6faf2640 1 -- 172.21.15.169:0/3648947098 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb68151ee0 con 0x7ffb6806b5f0 2023-10-26T01:30:40.318 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:40.311+0000 7ffb6faf2640 1 -- 172.21.15.169:0/3648947098 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb681e5c40 con 0x7ffb6806b5f0 2023-10-26T01:30:40.318 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:40.312+0000 7ffb6faf2640 1 -- 172.21.15.169:0/3648947098 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb6804db10 con 0x7ffb6806b5f0 2023-10-26T01:30:40.319 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:40.313+0000 7ffb667fc640 1 -- 172.21.15.169:0/3648947098 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ffb5c034a90 con 0x7ffb6806b5f0 2023-10-26T01:30:40.319 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:40.314+0000 7ffb667fc640 1 --2- 172.21.15.169:0/3648947098 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ffb4c06ae70 0x7ffb4c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:40.319 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:40.314+0000 7ffb667fc640 1 -- 172.21.15.169:0/3648947098 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5227+0+0 (secure 0 0 0) 0x7ffb5c0ac820 con 0x7ffb6806b5f0 2023-10-26T01:30:40.319 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:40.315+0000 7ffb6d08d640 1 --2- 172.21.15.169:0/3648947098 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ffb4c06ae70 0x7ffb4c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:40.319 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:40.316+0000 7ffb6d08d640 1 --2- 172.21.15.169:0/3648947098 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ffb4c06ae70 0x7ffb4c06d330 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ffb54005fd0 tx=0x7ffb54005bc0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:40.321 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:40.319+0000 7ffb667fc640 1 -- 172.21.15.169:0/3648947098 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ffb5c077a10 con 0x7ffb6806b5f0 2023-10-26T01:30:40.511 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:40.508+0000 7ffb6faf2640 1 -- 172.21.15.169:0/3648947098 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7ffb681e38a0 con 0x7ffb6806b5f0 2023-10-26T01:30:41.076 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:41.073+0000 7ffb667fc640 1 -- 172.21.15.169:0/3648947098 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v19) v1 ==== 170+0+0 (secure 0 0 0) 0x7ffb5c07b030 con 0x7ffb6806b5f0 2023-10-26T01:30:41.076 INFO:teuthology.orchestra.run.smithi169.stderr:pool 'cephfs_data' created 2023-10-26T01:30:41.079 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:41.076+0000 7ffb6faf2640 1 -- 172.21.15.169:0/3648947098 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ffb4c06ae70 msgr2=0x7ffb4c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:41.079 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:41.076+0000 7ffb6faf2640 1 --2- 172.21.15.169:0/3648947098 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ffb4c06ae70 0x7ffb4c06d330 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ffb54005fd0 tx=0x7ffb54005bc0 comp rx=0 tx=0).stop 2023-10-26T01:30:41.080 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:41.076+0000 7ffb6faf2640 1 -- 172.21.15.169:0/3648947098 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffb6806b5f0 msgr2=0x7ffb681e5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:41.080 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:41.076+0000 7ffb6faf2640 1 --2- 172.21.15.169:0/3648947098 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffb6806b5f0 0x7ffb681e5700 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c009ca0 tx=0x7ffb5c004360 comp rx=0 tx=0).stop 2023-10-26T01:30:41.080 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:41.077+0000 7ffb6faf2640 1 -- 172.21.15.169:0/3648947098 shutdown_connections 2023-10-26T01:30:41.080 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:41.077+0000 7ffb6faf2640 1 --2- 172.21.15.169:0/3648947098 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ffb4c06ae70 0x7ffb4c06d330 secure :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ffb54005fd0 tx=0x7ffb54005bc0 comp rx=0 tx=0).stop 2023-10-26T01:30:41.080 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:41.077+0000 7ffb6faf2640 1 --2- 172.21.15.169:0/3648947098 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ffb680b2d60 0x7ffb681e4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:41.081 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:41.077+0000 7ffb6faf2640 1 --2- 172.21.15.169:0/3648947098 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffb6806d5b0 0x7ffb681e6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:41.081 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:41.077+0000 7ffb6faf2640 1 --2- 172.21.15.169:0/3648947098 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffb6806b5f0 0x7ffb681e5700 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:41.081 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:41.077+0000 7ffb6faf2640 1 -- 172.21.15.169:0/3648947098 >> 172.21.15.169:0/3648947098 conn(0x7ffb68098a60 msgr2=0x7ffb68157f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:41.082 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:41.077+0000 7ffb6faf2640 1 -- 172.21.15.169:0/3648947098 shutdown_connections 2023-10-26T01:30:41.082 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:41.077+0000 7ffb6faf2640 1 -- 172.21.15.169:0/3648947098 wait complete. 2023-10-26T01:30:41.108 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-26T01:30:41.302 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:41.299+0000 7fda4ebd7640 1 -- 172.21.15.169:0/2554247283 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda380039a0 msgr2=0x7fda38003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:41.303 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:41.299+0000 7fda4ebd7640 1 --2- 172.21.15.169:0/2554247283 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda380039a0 0x7fda38003dc0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fda3c009b20 tx=0x7fda380057f0 comp rx=0 tx=0).stop 2023-10-26T01:30:41.303 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:41.299+0000 7fda4d174640 1 -- 172.21.15.169:0/2554247283 reap_dead start 2023-10-26T01:30:41.303 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:41.299+0000 7fda4ebd7640 1 -- 172.21.15.169:0/2554247283 shutdown_connections 2023-10-26T01:30:41.303 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:41.299+0000 7fda4ebd7640 1 -- 172.21.15.169:0/2554247283 >> 172.21.15.169:0/2554247283 conn(0x7fda48069930 msgr2=0x7fda48069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:41.303 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:41.299+0000 7fda4ebd7640 1 -- 172.21.15.169:0/2554247283 shutdown_connections 2023-10-26T01:30:41.304 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:41.300+0000 7fda4ebd7640 1 -- 172.21.15.169:0/2554247283 wait complete. 2023-10-26T01:30:41.304 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:41.301+0000 7fda4ebd7640 1 Processor -- start 2023-10-26T01:30:41.305 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:41.301+0000 7fda4ebd7640 1 -- start start 2023-10-26T01:30:41.307 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:41.302+0000 7fda4ebd7640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda380039a0 0x7fda48095d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:41.307 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:41.302+0000 7fda4ebd7640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fda4814c540 0x7fda4814d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:41.307 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:41.302+0000 7fda4ebd7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fda4814b6e0 0x7fda4814d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:41.307 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:41.302+0000 7fda4ebd7640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda48096f50 con 0x7fda380039a0 2023-10-26T01:30:41.308 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:41.302+0000 7fda4ebd7640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda480963d0 con 0x7fda4814b6e0 2023-10-26T01:30:41.308 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:41.302+0000 7fda4ebd7640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda480966b0 con 0x7fda4814c540 2023-10-26T01:30:41.308 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:41.302+0000 7fda4c973640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda380039a0 0x7fda48095d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:41.308 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:41.302+0000 7fda4c973640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda380039a0 0x7fda48095d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:33858/0 (socket says 172.21.15.169:33858) 2023-10-26T01:30:41.308 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:41.303+0000 7fda4c973640 1 -- 172.21.15.169:0/4134646210 learned_addr learned my addr 172.21.15.169:0/4134646210 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:30:41.309 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:41.303+0000 7fda4c973640 1 -- 172.21.15.169:0/4134646210 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fda4814c540 msgr2=0x7fda4814d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:41.309 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:41.303+0000 7fda4c973640 1 --2- 172.21.15.169:0/4134646210 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fda4814c540 0x7fda4814d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:41.309 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:41.303+0000 7fda4c973640 1 -- 172.21.15.169:0/4134646210 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fda4814b6e0 msgr2=0x7fda4814d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:41.309 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:41.303+0000 7fda4d174640 1 --2- 172.21.15.169:0/4134646210 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fda4814b6e0 0x7fda4814d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:41.309 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:41.303+0000 7fda4c973640 1 --2- 172.21.15.169:0/4134646210 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fda4814b6e0 0x7fda4814d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:41.310 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:41.303+0000 7fda4c973640 1 -- 172.21.15.169:0/4134646210 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda48096990 con 0x7fda380039a0 2023-10-26T01:30:41.310 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:41.303+0000 7fda4c973640 1 --2- 172.21.15.169:0/4134646210 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda380039a0 0x7fda48095d60 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fda3c009a70 tx=0x7fda3c033c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:41.310 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:41.303+0000 7fda457fa640 1 -- 172.21.15.169:0/4134646210 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda3c004220 con 0x7fda380039a0 2023-10-26T01:30:41.310 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:41.303+0000 7fda457fa640 1 -- 172.21.15.169:0/4134646210 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fda3c0043a0 con 0x7fda380039a0 2023-10-26T01:30:41.311 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:41.303+0000 7fda457fa640 1 -- 172.21.15.169:0/4134646210 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda3c03d940 con 0x7fda380039a0 2023-10-26T01:30:41.311 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:41.304+0000 7fda4ebd7640 1 -- 172.21.15.169:0/4134646210 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda48096c70 con 0x7fda380039a0 2023-10-26T01:30:41.311 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:41.304+0000 7fda4ebd7640 1 -- 172.21.15.169:0/4134646210 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda48151da0 con 0x7fda380039a0 2023-10-26T01:30:41.312 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:41.305+0000 7fda4ebd7640 1 -- 172.21.15.169:0/4134646210 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda4804db10 con 0x7fda380039a0 2023-10-26T01:30:41.312 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:41.306+0000 7fda457fa640 1 -- 172.21.15.169:0/4134646210 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fda3c03daa0 con 0x7fda380039a0 2023-10-26T01:30:41.312 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:41.307+0000 7fda457fa640 1 --2- 172.21.15.169:0/4134646210 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fda2c06ae70 0x7fda2c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:41.313 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:41.307+0000 7fda457fa640 1 -- 172.21.15.169:0/4134646210 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fda3c033e40 con 0x7fda380039a0 2023-10-26T01:30:41.313 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:41.307+0000 7fda47fff640 1 --2- 172.21.15.169:0/4134646210 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fda2c06ae70 0x7fda2c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:41.313 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:41.308+0000 7fda47fff640 1 --2- 172.21.15.169:0/4134646210 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fda2c06ae70 0x7fda2c06d330 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fda481f2d00 tx=0x7fda340014f0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:41.314 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:41.311+0000 7fda457fa640 1 -- 172.21.15.169:0/4134646210 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fda3c0773b0 con 0x7fda380039a0 2023-10-26T01:30:41.544 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:41.541+0000 7fda4ebd7640 1 -- 172.21.15.169:0/4134646210 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fda4806ffe0 con 0x7fda380039a0 2023-10-26T01:30:42.080 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.077+0000 7fda457fa640 1 -- 172.21.15.169:0/4134646210 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7fda3c07a9d0 con 0x7fda380039a0 2023-10-26T01:30:42.080 INFO:teuthology.orchestra.run.smithi169.stderr:new fs with metadata pool 2 and data pool 3 2023-10-26T01:30:42.083 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.081+0000 7fda4ebd7640 1 -- 172.21.15.169:0/4134646210 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fda2c06ae70 msgr2=0x7fda2c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:42.084 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.081+0000 7fda4ebd7640 1 --2- 172.21.15.169:0/4134646210 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fda2c06ae70 0x7fda2c06d330 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fda481f2d00 tx=0x7fda340014f0 comp rx=0 tx=0).stop 2023-10-26T01:30:42.084 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.081+0000 7fda4ebd7640 1 -- 172.21.15.169:0/4134646210 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda380039a0 msgr2=0x7fda48095d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:42.084 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.081+0000 7fda4ebd7640 1 --2- 172.21.15.169:0/4134646210 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda380039a0 0x7fda48095d60 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fda3c009a70 tx=0x7fda3c033c50 comp rx=0 tx=0).stop 2023-10-26T01:30:42.085 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.082+0000 7fda4ebd7640 1 -- 172.21.15.169:0/4134646210 shutdown_connections 2023-10-26T01:30:42.085 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.082+0000 7fda4ebd7640 1 --2- 172.21.15.169:0/4134646210 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fda2c06ae70 0x7fda2c06d330 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:42.085 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.082+0000 7fda4ebd7640 1 --2- 172.21.15.169:0/4134646210 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fda4814b6e0 0x7fda4814d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:42.086 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.082+0000 7fda4ebd7640 1 --2- 172.21.15.169:0/4134646210 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fda4814c540 0x7fda4814d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:42.086 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.082+0000 7fda4ebd7640 1 --2- 172.21.15.169:0/4134646210 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda380039a0 0x7fda48095d60 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:42.086 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.082+0000 7fda4ebd7640 1 -- 172.21.15.169:0/4134646210 >> 172.21.15.169:0/4134646210 conn(0x7fda48069930 msgr2=0x7fda48158e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:42.086 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.083+0000 7fda4ebd7640 1 -- 172.21.15.169:0/4134646210 shutdown_connections 2023-10-26T01:30:42.087 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.083+0000 7fda4ebd7640 1 -- 172.21.15.169:0/4134646210 wait complete. 2023-10-26T01:30:42.122 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:30:42.309 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.306+0000 7f5f2276e640 1 -- 172.21.15.169:0/156474113 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f1c0cdc30 msgr2=0x7f5f1c06ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:42.310 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.306+0000 7f5f2276e640 1 --2- 172.21.15.169:0/156474113 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f1c0cdc30 0x7f5f1c06ad30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c0b3440 tx=0x7f5f1402c1d0 comp rx=0 tx=0).stop 2023-10-26T01:30:42.310 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.307+0000 7f5f2276e640 1 -- 172.21.15.169:0/156474113 shutdown_connections 2023-10-26T01:30:42.310 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.307+0000 7f5f2276e640 1 --2- 172.21.15.169:0/156474113 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f1c0cdc30 0x7f5f1c06ad30 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:42.310 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.307+0000 7f5f2276e640 1 --2- 172.21.15.169:0/156474113 >> v2:172.21.15.170:3300/0 conn(0x7f5f1c0ac930 0x7f5f1c0cd6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:42.311 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.307+0000 7f5f2276e640 1 --2- 172.21.15.169:0/156474113 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5f1c1592d0 0x7f5f1c0aa3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:42.311 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.307+0000 7f5f2276e640 1 -- 172.21.15.169:0/156474113 >> 172.21.15.169:0/156474113 conn(0x7f5f1c069930 msgr2=0x7f5f1c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:42.311 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.307+0000 7f5f2276e640 1 -- 172.21.15.169:0/156474113 shutdown_connections 2023-10-26T01:30:42.311 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.307+0000 7f5f2276e640 1 -- 172.21.15.169:0/156474113 wait complete. 2023-10-26T01:30:42.313 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.308+0000 7f5f2276e640 1 Processor -- start 2023-10-26T01:30:42.313 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.309+0000 7f5f2276e640 1 -- start start 2023-10-26T01:30:42.313 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.309+0000 7f5f2276e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5f1c0ac930 0x7f5f1c14f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:42.313 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.309+0000 7f5f2276e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f1c0cdc30 0x7f5f1c15b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:42.314 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.309+0000 7f5f2276e640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5f1c1592d0 0x7f5f1c06ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:42.314 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.310+0000 7f5f2276e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f1c15bb70 con 0x7f5f1c0cdc30 2023-10-26T01:30:42.314 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.310+0000 7f5f2276e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f1c15be50 con 0x7f5f1c0ac930 2023-10-26T01:30:42.314 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.310+0000 7f5f2276e640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f1c15c130 con 0x7f5f1c1592d0 2023-10-26T01:30:42.314 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.310+0000 7f5f137fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f1c0cdc30 0x7f5f1c15b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:42.315 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.310+0000 7f5f137fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f1c0cdc30 0x7f5f1c15b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:33876/0 (socket says 172.21.15.169:33876) 2023-10-26T01:30:42.315 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.310+0000 7f5f137fe640 1 -- 172.21.15.169:0/1649480340 learned_addr learned my addr 172.21.15.169:0/1649480340 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:30:42.315 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.310+0000 7f5f20d0b640 1 --2- 172.21.15.169:0/1649480340 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5f1c1592d0 0x7f5f1c06ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:42.315 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.310+0000 7f5f1bfff640 1 --2- 172.21.15.169:0/1649480340 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5f1c0ac930 0x7f5f1c14f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:42.315 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.310+0000 7f5f137fe640 1 -- 172.21.15.169:0/1649480340 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5f1c1592d0 msgr2=0x7f5f1c06ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:42.316 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.310+0000 7f5f137fe640 1 --2- 172.21.15.169:0/1649480340 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5f1c1592d0 0x7f5f1c06ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:42.316 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.310+0000 7f5f137fe640 1 -- 172.21.15.169:0/1649480340 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5f1c0ac930 msgr2=0x7f5f1c14f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:42.316 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.310+0000 7f5f137fe640 1 --2- 172.21.15.169:0/1649480340 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5f1c0ac930 0x7f5f1c14f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:42.316 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.310+0000 7f5f137fe640 1 -- 172.21.15.169:0/1649480340 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f14009900 con 0x7f5f1c0cdc30 2023-10-26T01:30:42.316 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.311+0000 7f5f137fe640 1 --2- 172.21.15.169:0/1649480340 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f1c0cdc30 0x7f5f1c15b320 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5f0400c890 tx=0x7f5f0400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:42.318 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.311+0000 7f5f197fa640 1 -- 172.21.15.169:0/1649480340 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f04004220 con 0x7f5f1c0cdc30 2023-10-26T01:30:42.318 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.311+0000 7f5f197fa640 1 -- 172.21.15.169:0/1649480340 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5f04004470 con 0x7f5f1c0cdc30 2023-10-26T01:30:42.318 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.311+0000 7f5f2276e640 1 -- 172.21.15.169:0/1649480340 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f1c14fff0 con 0x7f5f1c0cdc30 2023-10-26T01:30:42.318 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.311+0000 7f5f197fa640 1 -- 172.21.15.169:0/1649480340 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f04007c10 con 0x7f5f1c0cdc30 2023-10-26T01:30:42.319 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.311+0000 7f5f2276e640 1 -- 172.21.15.169:0/1649480340 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f1c15b860 con 0x7f5f1c0cdc30 2023-10-26T01:30:42.319 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.312+0000 7f5f2276e640 1 -- 172.21.15.169:0/1649480340 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f1c0cd070 con 0x7f5f1c0cdc30 2023-10-26T01:30:42.319 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.313+0000 7f5f197fa640 1 -- 172.21.15.169:0/1649480340 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f5f04005020 con 0x7f5f1c0cdc30 2023-10-26T01:30:42.319 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.314+0000 7f5f197fa640 1 --2- 172.21.15.169:0/1649480340 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5efc06ae40 0x7f5efc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:42.319 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.314+0000 7f5f197fa640 1 -- 172.21.15.169:0/1649480340 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5f04089c40 con 0x7f5f1c0cdc30 2023-10-26T01:30:42.319 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.314+0000 7f5f1bfff640 1 --2- 172.21.15.169:0/1649480340 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5efc06ae40 0x7f5efc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:42.320 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.315+0000 7f5f1bfff640 1 --2- 172.21.15.169:0/1649480340 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5efc06ae40 0x7f5efc06d300 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c009960 tx=0x7f5f0c005c90 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:42.321 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.318+0000 7f5f197fa640 1 -- 172.21.15.169:0/1649480340 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5f04054c50 con 0x7f5f1c0cdc30 2023-10-26T01:30:42.505 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.502+0000 7f5f2276e640 1 -- 172.21.15.169:0/1649480340 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5f1c04db10 con 0x7f5f1c0cdc30 2023-10-26T01:30:42.508 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.505+0000 7f5f197fa640 1 -- 172.21.15.169:0/1649480340 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16566 (secure 0 0 0) 0x7f5f0401d070 con 0x7f5f1c0cdc30 2023-10-26T01:30:42.508 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:30:42.509 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":20,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:30:41.544112+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-26T01:30:39.437631+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-26T01:30:40.561774+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:30:42.511 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.508+0000 7f5f2276e640 1 -- 172.21.15.169:0/1649480340 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5efc06ae40 msgr2=0x7f5efc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:42.511 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.508+0000 7f5f2276e640 1 --2- 172.21.15.169:0/1649480340 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5efc06ae40 0x7f5efc06d300 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c009960 tx=0x7f5f0c005c90 comp rx=0 tx=0).stop 2023-10-26T01:30:42.511 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.508+0000 7f5f2276e640 1 -- 172.21.15.169:0/1649480340 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f1c0cdc30 msgr2=0x7f5f1c15b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:42.511 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.508+0000 7f5f2276e640 1 --2- 172.21.15.169:0/1649480340 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f1c0cdc30 0x7f5f1c15b320 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5f0400c890 tx=0x7f5f0400cd50 comp rx=0 tx=0).stop 2023-10-26T01:30:42.512 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.508+0000 7f5f2276e640 1 -- 172.21.15.169:0/1649480340 shutdown_connections 2023-10-26T01:30:42.512 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.508+0000 7f5f2276e640 1 --2- 172.21.15.169:0/1649480340 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5efc06ae40 0x7f5efc06d300 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:42.512 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.508+0000 7f5f2276e640 1 --2- 172.21.15.169:0/1649480340 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f1c0cdc30 0x7f5f1c15b320 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:42.512 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.508+0000 7f5f2276e640 1 --2- 172.21.15.169:0/1649480340 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5f1c1592d0 0x7f5f1c06ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:42.512 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.508+0000 7f5f2276e640 1 --2- 172.21.15.169:0/1649480340 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5f1c0ac930 0x7f5f1c14f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:42.513 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.508+0000 7f5f2276e640 1 -- 172.21.15.169:0/1649480340 >> 172.21.15.169:0/1649480340 conn(0x7f5f1c069930 msgr2=0x7f5f1c153a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:42.513 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.508+0000 7f5f2276e640 1 -- 172.21.15.169:0/1649480340 shutdown_connections 2023-10-26T01:30:42.513 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.508+0000 7f5f2276e640 1 -- 172.21.15.169:0/1649480340 wait complete. 2023-10-26T01:30:42.527 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:30:42.739 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.737+0000 7fbca0e97640 1 -- 172.21.15.169:0/912450288 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbc88005180 msgr2=0x7fbc8800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:42.740 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.737+0000 7fbca0e97640 1 --2- 172.21.15.169:0/912450288 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbc88005180 0x7fbc8800b980 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fbc9400c590 tx=0x7fbc880057f0 comp rx=0 tx=0).stop 2023-10-26T01:30:42.740 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.737+0000 7fbca0e97640 1 -- 172.21.15.169:0/912450288 shutdown_connections 2023-10-26T01:30:42.740 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.737+0000 7fbca0e97640 1 --2- 172.21.15.169:0/912450288 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbc88005180 0x7fbc8800b980 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:42.740 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.737+0000 7fbca0e97640 1 -- 172.21.15.169:0/912450288 >> 172.21.15.169:0/912450288 conn(0x7fbc9c069930 msgr2=0x7fbc9c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:42.741 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.737+0000 7fbca0e97640 1 -- 172.21.15.169:0/912450288 shutdown_connections 2023-10-26T01:30:42.741 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.737+0000 7fbca0e97640 1 -- 172.21.15.169:0/912450288 wait complete. 2023-10-26T01:30:42.741 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.738+0000 7fbca0e97640 1 Processor -- start 2023-10-26T01:30:42.742 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.739+0000 7fbca0e97640 1 -- start start 2023-10-26T01:30:42.744 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.740+0000 7fbca0e97640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbc88005180 0x7fbc9c14d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:42.744 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.740+0000 7fbca0e97640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbc9c146da0 0x7fbc9c1497d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:42.744 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.740+0000 7fbca0e97640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbc9c147220 0x7fbc9c149d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:42.744 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.740+0000 7fbca0e97640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc9c0c97b0 con 0x7fbc88005180 2023-10-26T01:30:42.745 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.740+0000 7fbca0e97640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc9c0c8b70 con 0x7fbc9c147220 2023-10-26T01:30:42.745 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.740+0000 7fbca0e97640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc9c0c8e10 con 0x7fbc9c146da0 2023-10-26T01:30:42.745 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.740+0000 7fbc9a59c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbc88005180 0x7fbc9c14d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:42.745 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.740+0000 7fbc9a59c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbc88005180 0x7fbc9c14d290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:33892/0 (socket says 172.21.15.169:33892) 2023-10-26T01:30:42.745 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.740+0000 7fbc9a59c640 1 -- 172.21.15.169:0/3046834873 learned_addr learned my addr 172.21.15.169:0/3046834873 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:30:42.746 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.740+0000 7fbc93fff640 1 --2- 172.21.15.169:0/3046834873 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbc9c146da0 0x7fbc9c1497d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:42.746 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.740+0000 7fbc9ad9d640 1 --2- 172.21.15.169:0/3046834873 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbc9c147220 0x7fbc9c149d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:42.746 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.740+0000 7fbc9a59c640 1 -- 172.21.15.169:0/3046834873 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbc9c146da0 msgr2=0x7fbc9c1497d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:42.746 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.740+0000 7fbc9a59c640 1 --2- 172.21.15.169:0/3046834873 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbc9c146da0 0x7fbc9c1497d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:42.746 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.740+0000 7fbc9a59c640 1 -- 172.21.15.169:0/3046834873 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbc9c147220 msgr2=0x7fbc9c149d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:42.747 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.740+0000 7fbc9a59c640 1 --2- 172.21.15.169:0/3046834873 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbc9c147220 0x7fbc9c149d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:42.747 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.740+0000 7fbc9a59c640 1 -- 172.21.15.169:0/3046834873 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc9c0c90f0 con 0x7fbc88005180 2023-10-26T01:30:42.747 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.741+0000 7fbc9a59c640 1 --2- 172.21.15.169:0/3046834873 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbc88005180 0x7fbc9c14d290 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c00d550 tx=0x7fbc8c00da10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:42.747 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.741+0000 7fbc92ffd640 1 -- 172.21.15.169:0/3046834873 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc8c012630 con 0x7fbc88005180 2023-10-26T01:30:42.747 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.741+0000 7fbc92ffd640 1 -- 172.21.15.169:0/3046834873 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fbc8c012790 con 0x7fbc88005180 2023-10-26T01:30:42.748 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.741+0000 7fbca0e97640 1 -- 172.21.15.169:0/3046834873 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc9c0c93d0 con 0x7fbc88005180 2023-10-26T01:30:42.748 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.741+0000 7fbc92ffd640 1 -- 172.21.15.169:0/3046834873 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc8c012b60 con 0x7fbc88005180 2023-10-26T01:30:42.748 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.741+0000 7fbca0e97640 1 -- 172.21.15.169:0/3046834873 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc9c14a450 con 0x7fbc88005180 2023-10-26T01:30:42.749 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.742+0000 7fbca0e97640 1 -- 172.21.15.169:0/3046834873 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc9c0ca1c0 con 0x7fbc88005180 2023-10-26T01:30:42.749 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.743+0000 7fbc92ffd640 1 -- 172.21.15.169:0/3046834873 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fbc8c01b400 con 0x7fbc88005180 2023-10-26T01:30:42.749 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.744+0000 7fbc92ffd640 1 --2- 172.21.15.169:0/3046834873 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbc7c06ae40 0x7fbc7c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:42.750 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.744+0000 7fbc92ffd640 1 -- 172.21.15.169:0/3046834873 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbc8c089830 con 0x7fbc88005180 2023-10-26T01:30:42.750 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.745+0000 7fbc93fff640 1 --2- 172.21.15.169:0/3046834873 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbc7c06ae40 0x7fbc7c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:42.750 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.746+0000 7fbc93fff640 1 --2- 172.21.15.169:0/3046834873 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbc7c06ae40 0x7fbc7c06d300 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c1587b0 tx=0x7fbc84005f90 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:42.750 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.748+0000 7fbc92ffd640 1 -- 172.21.15.169:0/3046834873 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbc8c058360 con 0x7fbc88005180 2023-10-26T01:30:42.936 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.933+0000 7fbca0e97640 1 -- 172.21.15.169:0/3046834873 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbc9c04db10 con 0x7fbc88005180 2023-10-26T01:30:42.939 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.936+0000 7fbc92ffd640 1 -- 172.21.15.169:0/3046834873 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16566 (secure 0 0 0) 0x7fbc8c057ef0 con 0x7fbc88005180 2023-10-26T01:30:42.940 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:30:42.940 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":20,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:30:41.544112+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-26T01:30:39.437631+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-26T01:30:40.561774+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:30:42.942 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.939+0000 7fbca0e97640 1 -- 172.21.15.169:0/3046834873 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbc7c06ae40 msgr2=0x7fbc7c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:42.942 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.939+0000 7fbca0e97640 1 --2- 172.21.15.169:0/3046834873 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbc7c06ae40 0x7fbc7c06d300 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c1587b0 tx=0x7fbc84005f90 comp rx=0 tx=0).stop 2023-10-26T01:30:42.942 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.939+0000 7fbca0e97640 1 -- 172.21.15.169:0/3046834873 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbc88005180 msgr2=0x7fbc9c14d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:42.942 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.939+0000 7fbca0e97640 1 --2- 172.21.15.169:0/3046834873 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbc88005180 0x7fbc9c14d290 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c00d550 tx=0x7fbc8c00da10 comp rx=0 tx=0).stop 2023-10-26T01:30:42.942 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.939+0000 7fbca0e97640 1 -- 172.21.15.169:0/3046834873 shutdown_connections 2023-10-26T01:30:42.943 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.939+0000 7fbca0e97640 1 --2- 172.21.15.169:0/3046834873 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbc7c06ae40 0x7fbc7c06d300 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:42.943 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.939+0000 7fbca0e97640 1 --2- 172.21.15.169:0/3046834873 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbc9c147220 0x7fbc9c149d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:42.943 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.939+0000 7fbca0e97640 1 --2- 172.21.15.169:0/3046834873 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbc9c146da0 0x7fbc9c1497d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:42.943 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.939+0000 7fbca0e97640 1 --2- 172.21.15.169:0/3046834873 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbc88005180 0x7fbc9c14d290 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:42.944 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.939+0000 7fbca0e97640 1 -- 172.21.15.169:0/3046834873 >> 172.21.15.169:0/3046834873 conn(0x7fbc9c069930 msgr2=0x7fbc9c0adf00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:42.944 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.939+0000 7fbca0e97640 1 -- 172.21.15.169:0/3046834873 shutdown_connections 2023-10-26T01:30:42.944 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:42.939+0000 7fbca0e97640 1 -- 172.21.15.169:0/3046834873 wait complete. 2023-10-26T01:30:42.958 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-26T01:30:43.170 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:43.167+0000 7f1ed3fff640 1 -- 172.21.15.169:0/114121074 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ed4158360 msgr2=0x7f1ed40aa6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:43.170 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:43.167+0000 7f1ed3fff640 1 --2- 172.21.15.169:0/114121074 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ed4158360 0x7f1ed40aa6d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc005fd0 tx=0x7f1ebc02c1b0 comp rx=0 tx=0).stop 2023-10-26T01:30:43.171 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:43.167+0000 7f1ed3fff640 1 -- 172.21.15.169:0/114121074 shutdown_connections 2023-10-26T01:30:43.171 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:43.167+0000 7f1ed3fff640 1 --2- 172.21.15.169:0/114121074 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ed4158360 0x7f1ed40aa6d0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:43.171 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:43.167+0000 7f1ed3fff640 1 --2- 172.21.15.169:0/114121074 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1ed4157f80 0x7f1ed415b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:43.171 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:43.167+0000 7f1ed3fff640 1 -- 172.21.15.169:0/114121074 >> 172.21.15.169:0/114121074 conn(0x7f1ed40af110 msgr2=0x7f1ed40af520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:43.171 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:43.167+0000 7f1ed3fff640 1 -- 172.21.15.169:0/114121074 shutdown_connections 2023-10-26T01:30:43.172 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:43.167+0000 7f1ed3fff640 1 -- 172.21.15.169:0/114121074 wait complete. 2023-10-26T01:30:43.174 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:43.169+0000 7f1ed3fff640 1 Processor -- start 2023-10-26T01:30:43.174 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:43.169+0000 7f1ed3fff640 1 -- start start 2023-10-26T01:30:43.174 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:43.170+0000 7f1ed3fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ed40aac10 0x7f1ed40c9f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:43.174 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:43.170+0000 7f1ed3fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ed4157f80 0x7f1ed40ca480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:43.174 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:43.170+0000 7f1ed3fff640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1ed4158360 0x7f1ed4140340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:43.175 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:43.170+0000 7f1ed3fff640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed4142f90 con 0x7f1ed4157f80 2023-10-26T01:30:43.175 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:43.170+0000 7f1ed3fff640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed41431e0 con 0x7f1ed40aac10 2023-10-26T01:30:43.175 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:43.170+0000 7f1ed3fff640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed41434c0 con 0x7f1ed4158360 2023-10-26T01:30:43.175 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:43.170+0000 7f1ed27fc640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ed4157f80 0x7f1ed40ca480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:43.175 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:43.170+0000 7f1ed27fc640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ed4157f80 0x7f1ed40ca480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:33906/0 (socket says 172.21.15.169:33906) 2023-10-26T01:30:43.176 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:43.170+0000 7f1ed27fc640 1 -- 172.21.15.169:0/286491301 learned_addr learned my addr 172.21.15.169:0/286491301 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:30:43.176 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:43.170+0000 7f1ed37fe640 1 --2- 172.21.15.169:0/286491301 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1ed4158360 0x7f1ed4140340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:43.176 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:43.170+0000 7f1ed2ffd640 1 --2- 172.21.15.169:0/286491301 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ed40aac10 0x7f1ed40c9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:43.176 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:43.170+0000 7f1ed27fc640 1 -- 172.21.15.169:0/286491301 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1ed4158360 msgr2=0x7f1ed4140340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:43.176 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:43.170+0000 7f1ed27fc640 1 --2- 172.21.15.169:0/286491301 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1ed4158360 0x7f1ed4140340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:43.177 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:43.170+0000 7f1ed27fc640 1 -- 172.21.15.169:0/286491301 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ed40aac10 msgr2=0x7f1ed40c9f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:43.177 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:43.170+0000 7f1ed27fc640 1 --2- 172.21.15.169:0/286491301 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ed40aac10 0x7f1ed40c9f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:43.177 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:43.170+0000 7f1ed27fc640 1 -- 172.21.15.169:0/286491301 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ebc009900 con 0x7f1ed4157f80 2023-10-26T01:30:43.177 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:43.171+0000 7f1ed27fc640 1 --2- 172.21.15.169:0/286491301 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ed4157f80 0x7f1ed40ca480 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc006010 tx=0x7f1ebc035e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:43.178 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:43.171+0000 7f1ecb7fe640 1 -- 172.21.15.169:0/286491301 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ebc034440 con 0x7f1ed4157f80 2023-10-26T01:30:43.178 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:43.171+0000 7f1ecb7fe640 1 -- 172.21.15.169:0/286491301 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1ebc004540 con 0x7f1ed4157f80 2023-10-26T01:30:43.178 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:43.171+0000 7f1ecb7fe640 1 -- 172.21.15.169:0/286491301 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ebc03f440 con 0x7f1ed4157f80 2023-10-26T01:30:43.178 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:43.171+0000 7f1ed3fff640 1 -- 172.21.15.169:0/286491301 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ed406ec30 con 0x7f1ed4157f80 2023-10-26T01:30:43.178 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:43.171+0000 7f1ed3fff640 1 -- 172.21.15.169:0/286491301 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ed4141750 con 0x7f1ed4157f80 2023-10-26T01:30:43.180 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:43.172+0000 7f1ed3fff640 1 -- 172.21.15.169:0/286491301 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ed4073c10 con 0x7f1ed4157f80 2023-10-26T01:30:43.180 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:43.173+0000 7f1ecb7fe640 1 -- 172.21.15.169:0/286491301 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f1ebc03b040 con 0x7f1ed4157f80 2023-10-26T01:30:43.180 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:43.174+0000 7f1ecb7fe640 1 --2- 172.21.15.169:0/286491301 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f1eb406ae70 0x7f1eb406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:43.180 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:43.174+0000 7f1ecb7fe640 1 -- 172.21.15.169:0/286491301 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1ebc0ac620 con 0x7f1ed4157f80 2023-10-26T01:30:43.180 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:43.175+0000 7f1ed2ffd640 1 --2- 172.21.15.169:0/286491301 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f1eb406ae70 0x7f1eb406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:43.181 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:43.176+0000 7f1ed2ffd640 1 --2- 172.21.15.169:0/286491301 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f1eb406ae70 0x7f1eb406d330 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1ec40058c0 tx=0x7f1ec4005c80 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:43.181 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:43.179+0000 7f1ecb7fe640 1 -- 172.21.15.169:0/286491301 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1ebc0775a0 con 0x7f1ed4157f80 2023-10-26T01:30:43.392 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:43.389+0000 7f1ed3fff640 1 -- 172.21.15.169:0/286491301 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f1ed4143680 con 0x7f1ed4157f80 2023-10-26T01:30:44.112 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:44.110+0000 7f1ecb7fe640 1 -- 172.21.15.169:0/286491301 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v6) v1 ==== 118+0+0 (secure 0 0 0) 0x7f1ebc07abc0 con 0x7f1ed4157f80 2023-10-26T01:30:44.116 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:44.113+0000 7f1ed3fff640 1 -- 172.21.15.169:0/286491301 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f1eb406ae70 msgr2=0x7f1eb406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:44.117 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:44.113+0000 7f1ed3fff640 1 --2- 172.21.15.169:0/286491301 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f1eb406ae70 0x7f1eb406d330 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1ec40058c0 tx=0x7f1ec4005c80 comp rx=0 tx=0).stop 2023-10-26T01:30:44.117 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:44.113+0000 7f1ed3fff640 1 -- 172.21.15.169:0/286491301 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ed4157f80 msgr2=0x7f1ed40ca480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:44.117 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:44.113+0000 7f1ed3fff640 1 --2- 172.21.15.169:0/286491301 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ed4157f80 0x7f1ed40ca480 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc006010 tx=0x7f1ebc035e30 comp rx=0 tx=0).stop 2023-10-26T01:30:44.117 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:44.113+0000 7f1ed3fff640 1 -- 172.21.15.169:0/286491301 shutdown_connections 2023-10-26T01:30:44.118 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:44.114+0000 7f1ed3fff640 1 --2- 172.21.15.169:0/286491301 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f1eb406ae70 0x7f1eb406d330 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:44.118 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:44.114+0000 7f1ed3fff640 1 --2- 172.21.15.169:0/286491301 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ed4157f80 0x7f1ed40ca480 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:44.118 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:44.114+0000 7f1ed3fff640 1 --2- 172.21.15.169:0/286491301 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1ed4158360 0x7f1ed4140340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:44.118 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:44.114+0000 7f1ed3fff640 1 --2- 172.21.15.169:0/286491301 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ed40aac10 0x7f1ed40c9f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:44.118 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:44.114+0000 7f1ed3fff640 1 -- 172.21.15.169:0/286491301 >> 172.21.15.169:0/286491301 conn(0x7f1ed40af110 msgr2=0x7f1ed4069ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:44.119 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:44.114+0000 7f1ed3fff640 1 -- 172.21.15.169:0/286491301 shutdown_connections 2023-10-26T01:30:44.119 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:44.114+0000 7f1ed3fff640 1 -- 172.21.15.169:0/286491301 wait complete. 2023-10-26T01:30:44.137 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['disabled']} 2023-10-26T01:30:44.138 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-26T01:30:44.351 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:44.348+0000 7f4aa641a640 1 -- 172.21.15.169:0/2530206164 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4aa0141cb0 msgr2=0x7f4aa00b14f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:44.351 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:44.348+0000 7f4aa641a640 1 --2- 172.21.15.169:0/2530206164 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4aa0141cb0 0x7f4aa00b14f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4aa006b140 tx=0x7f4a9402bda0 comp rx=0 tx=0).stop 2023-10-26T01:30:44.351 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:44.348+0000 7f4aa641a640 1 -- 172.21.15.169:0/2530206164 shutdown_connections 2023-10-26T01:30:44.351 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:44.348+0000 7f4aa641a640 1 --2- 172.21.15.169:0/2530206164 >> v2:172.21.15.170:3300/0 conn(0x7f4aa00cd7e0 0x7f4aa00cdbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:44.352 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:44.348+0000 7f4aa641a640 1 --2- 172.21.15.169:0/2530206164 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4aa0141cb0 0x7f4aa00b14f0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:44.352 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:44.348+0000 7f4aa641a640 1 --2- 172.21.15.169:0/2530206164 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4aa00ccda0 0x7f4aa015ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:44.352 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:44.348+0000 7f4aa641a640 1 -- 172.21.15.169:0/2530206164 >> 172.21.15.169:0/2530206164 conn(0x7f4aa0069930 msgr2=0x7f4aa0069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:44.352 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:44.348+0000 7f4aa641a640 1 -- 172.21.15.169:0/2530206164 shutdown_connections 2023-10-26T01:30:44.352 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:44.348+0000 7f4aa641a640 1 -- 172.21.15.169:0/2530206164 wait complete. 2023-10-26T01:30:44.353 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:44.350+0000 7f4aa641a640 1 Processor -- start 2023-10-26T01:30:44.353 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:44.350+0000 7f4aa641a640 1 -- start start 2023-10-26T01:30:44.356 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:44.352+0000 7f4aa641a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4aa00ccda0 0x7f4aa0156040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:44.356 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:44.352+0000 7f4aa641a640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4aa00cd7e0 0x7f4aa0097de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:44.356 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:44.352+0000 7f4aa641a640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4aa0141cb0 0x7f4aa01ef6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:44.356 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:44.352+0000 7f4aa641a640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa009b110 con 0x7f4aa0141cb0 2023-10-26T01:30:44.357 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:44.352+0000 7f4aa641a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa00986f0 con 0x7f4aa00ccda0 2023-10-26T01:30:44.357 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:44.352+0000 7f4aa641a640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa00989d0 con 0x7f4aa00cd7e0 2023-10-26T01:30:44.357 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:44.352+0000 7f4aa49b7640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4aa0141cb0 0x7f4aa01ef6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:44.357 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:44.352+0000 7f4aa49b7640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4aa0141cb0 0x7f4aa01ef6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:33910/0 (socket says 172.21.15.169:33910) 2023-10-26T01:30:44.357 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:44.352+0000 7f4aa49b7640 1 -- 172.21.15.169:0/3177936130 learned_addr learned my addr 172.21.15.169:0/3177936130 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:30:44.358 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:44.352+0000 7f4a9f7fe640 1 --2- 172.21.15.169:0/3177936130 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4aa00cd7e0 0x7f4aa0097de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:44.358 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:44.352+0000 7f4a9ffff640 1 --2- 172.21.15.169:0/3177936130 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4aa00ccda0 0x7f4aa0156040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:44.358 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:44.352+0000 7f4aa49b7640 1 -- 172.21.15.169:0/3177936130 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4aa00cd7e0 msgr2=0x7f4aa0097de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:44.358 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:44.352+0000 7f4aa49b7640 1 --2- 172.21.15.169:0/3177936130 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4aa00cd7e0 0x7f4aa0097de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:44.358 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:44.353+0000 7f4aa49b7640 1 -- 172.21.15.169:0/3177936130 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4aa00ccda0 msgr2=0x7f4aa0156040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:44.359 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:44.353+0000 7f4aa49b7640 1 --2- 172.21.15.169:0/3177936130 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4aa00ccda0 0x7f4aa0156040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:44.359 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:44.353+0000 7f4aa49b7640 1 -- 172.21.15.169:0/3177936130 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a94009560 con 0x7f4aa0141cb0 2023-10-26T01:30:44.359 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:44.353+0000 7f4aa49b7640 1 --2- 172.21.15.169:0/3177936130 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4aa0141cb0 0x7f4aa01ef6b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4a9800cce0 tx=0x7f4a9800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:44.359 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:44.353+0000 7f4a9cff9640 1 -- 172.21.15.169:0/3177936130 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a9800f450 con 0x7f4aa0141cb0 2023-10-26T01:30:44.359 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:44.353+0000 7f4a9cff9640 1 -- 172.21.15.169:0/3177936130 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4a9800f6c0 con 0x7f4aa0141cb0 2023-10-26T01:30:44.360 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:44.353+0000 7f4a9cff9640 1 -- 172.21.15.169:0/3177936130 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a9800fad0 con 0x7f4aa0141cb0 2023-10-26T01:30:44.360 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:44.353+0000 7f4aa641a640 1 -- 172.21.15.169:0/3177936130 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4aa015c5b0 con 0x7f4aa0141cb0 2023-10-26T01:30:44.361 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:44.353+0000 7f4aa641a640 1 -- 172.21.15.169:0/3177936130 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4aa0098f90 con 0x7f4aa0141cb0 2023-10-26T01:30:44.361 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:44.355+0000 7f4aa641a640 1 -- 172.21.15.169:0/3177936130 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4aa004db10 con 0x7f4aa0141cb0 2023-10-26T01:30:44.361 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:44.356+0000 7f4a9cff9640 1 -- 172.21.15.169:0/3177936130 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f4a9800fc30 con 0x7f4aa0141cb0 2023-10-26T01:30:44.361 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:44.357+0000 7f4a9cff9640 1 --2- 172.21.15.169:0/3177936130 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f4a8c06ae40 0x7f4a8c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:44.362 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:44.357+0000 7f4a9cff9640 1 -- 172.21.15.169:0/3177936130 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4a9808a120 con 0x7f4aa0141cb0 2023-10-26T01:30:44.362 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:44.357+0000 7f4a9ffff640 1 --2- 172.21.15.169:0/3177936130 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f4a8c06ae40 0x7f4a8c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:44.362 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:44.358+0000 7f4a9ffff640 1 --2- 172.21.15.169:0/3177936130 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f4a8c06ae40 0x7f4a8c06d300 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f4a90005fd0 tx=0x7f4a90005850 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:44.363 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:44.361+0000 7f4a9cff9640 1 -- 172.21.15.169:0/3177936130 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4a98055240 con 0x7f4aa0141cb0 2023-10-26T01:30:44.577 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:44.574+0000 7f4aa641a640 1 -- 172.21.15.169:0/3177936130 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f4aa00662e0 con 0x7f4aa0141cb0 2023-10-26T01:30:45.130 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:45.127+0000 7f4a9cff9640 1 -- 172.21.15.169:0/3177936130 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v7) v1 ==== 122+0+0 (secure 0 0 0) 0x7f4a98058860 con 0x7f4aa0141cb0 2023-10-26T01:30:45.133 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:45.130+0000 7f4aa641a640 1 -- 172.21.15.169:0/3177936130 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f4a8c06ae40 msgr2=0x7f4a8c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:45.134 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:45.130+0000 7f4aa641a640 1 --2- 172.21.15.169:0/3177936130 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f4a8c06ae40 0x7f4a8c06d300 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f4a90005fd0 tx=0x7f4a90005850 comp rx=0 tx=0).stop 2023-10-26T01:30:45.134 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:45.130+0000 7f4aa641a640 1 -- 172.21.15.169:0/3177936130 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4aa0141cb0 msgr2=0x7f4aa01ef6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:45.134 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:45.131+0000 7f4aa641a640 1 --2- 172.21.15.169:0/3177936130 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4aa0141cb0 0x7f4aa01ef6b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4a9800cce0 tx=0x7f4a9800a620 comp rx=0 tx=0).stop 2023-10-26T01:30:45.134 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:45.131+0000 7f4aa641a640 1 -- 172.21.15.169:0/3177936130 shutdown_connections 2023-10-26T01:30:45.135 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:45.131+0000 7f4aa641a640 1 --2- 172.21.15.169:0/3177936130 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f4a8c06ae40 0x7f4a8c06d300 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:45.135 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:45.131+0000 7f4aa641a640 1 --2- 172.21.15.169:0/3177936130 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4aa0141cb0 0x7f4aa01ef6b0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:45.135 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:45.131+0000 7f4aa641a640 1 --2- 172.21.15.169:0/3177936130 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4aa00cd7e0 0x7f4aa0097de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:45.135 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:45.131+0000 7f4aa641a640 1 --2- 172.21.15.169:0/3177936130 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4aa00ccda0 0x7f4aa0156040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:45.135 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:45.131+0000 7f4aa641a640 1 -- 172.21.15.169:0/3177936130 >> 172.21.15.169:0/3177936130 conn(0x7f4aa0069930 msgr2=0x7f4aa0158a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:45.136 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:45.131+0000 7f4aa641a640 1 -- 172.21.15.169:0/3177936130 shutdown_connections 2023-10-26T01:30:45.136 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:45.131+0000 7f4aa641a640 1 -- 172.21.15.169:0/3177936130 wait complete. 2023-10-26T01:30:45.154 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-26T01:30:45.154 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T01:30:45.363 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:45.360+0000 7fa06fc96640 1 -- 172.21.15.169:0/980238004 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa0680ac4e0 msgr2=0x7fa0680cd670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:45.363 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:45.360+0000 7fa06fc96640 1 --2- 172.21.15.169:0/980238004 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa0680ac4e0 0x7fa0680cd670 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa058005fd0 tx=0x7fa05802be10 comp rx=0 tx=0).stop 2023-10-26T01:30:45.363 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:45.360+0000 7fa06fc96640 1 -- 172.21.15.169:0/980238004 shutdown_connections 2023-10-26T01:30:45.363 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:45.360+0000 7fa06fc96640 1 --2- 172.21.15.169:0/980238004 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa0680ac4e0 0x7fa0680cd670 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:45.364 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:45.360+0000 7fa06fc96640 1 --2- 172.21.15.169:0/980238004 >> v2:172.21.15.170:3300/0 conn(0x7fa06815b610 0x7fa06806ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:45.364 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:45.360+0000 7fa06fc96640 1 -- 172.21.15.169:0/980238004 >> 172.21.15.169:0/980238004 conn(0x7fa068069930 msgr2=0x7fa068069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:45.364 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:45.360+0000 7fa06fc96640 1 -- 172.21.15.169:0/980238004 shutdown_connections 2023-10-26T01:30:45.364 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:45.360+0000 7fa06fc96640 1 -- 172.21.15.169:0/980238004 wait complete. 2023-10-26T01:30:45.366 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:45.362+0000 7fa06fc96640 1 Processor -- start 2023-10-26T01:30:45.367 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:45.362+0000 7fa06fc96640 1 -- start start 2023-10-26T01:30:45.367 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:45.363+0000 7fa06fc96640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa0680ac4e0 0x7fa068159070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:45.367 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:45.363+0000 7fa06fc96640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa0680ac940 0x7fa068149740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:45.367 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:45.363+0000 7fa06fc96640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa06815b610 0x7fa068149c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:45.368 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:45.363+0000 7fa06fc96640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa06814ab50 con 0x7fa06815b610 2023-10-26T01:30:45.368 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:45.363+0000 7fa06fc96640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa06814c3e0 con 0x7fa0680ac4e0 2023-10-26T01:30:45.368 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:45.363+0000 7fa06fc96640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa06809aef0 con 0x7fa0680ac940 2023-10-26T01:30:45.368 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:45.363+0000 7fa06e233640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa06815b610 0x7fa068149c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:45.368 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:45.363+0000 7fa06e233640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa06815b610 0x7fa068149c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:33912/0 (socket says 172.21.15.169:33912) 2023-10-26T01:30:45.368 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:45.363+0000 7fa06e233640 1 -- 172.21.15.169:0/2709612750 learned_addr learned my addr 172.21.15.169:0/2709612750 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:30:45.369 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:45.363+0000 7fa06da32640 1 --2- 172.21.15.169:0/2709612750 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa0680ac4e0 0x7fa068159070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:45.369 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:45.363+0000 7fa06d231640 1 --2- 172.21.15.169:0/2709612750 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa0680ac940 0x7fa068149740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:45.369 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:45.363+0000 7fa06e233640 1 -- 172.21.15.169:0/2709612750 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa0680ac940 msgr2=0x7fa068149740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:45.369 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:45.363+0000 7fa06e233640 1 --2- 172.21.15.169:0/2709612750 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa0680ac940 0x7fa068149740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:45.369 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:45.363+0000 7fa06e233640 1 -- 172.21.15.169:0/2709612750 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa0680ac4e0 msgr2=0x7fa068159070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:45.370 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:45.363+0000 7fa06e233640 1 --2- 172.21.15.169:0/2709612750 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa0680ac4e0 0x7fa068159070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:45.370 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:45.363+0000 7fa06e233640 1 -- 172.21.15.169:0/2709612750 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa058009560 con 0x7fa06815b610 2023-10-26T01:30:45.370 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:45.364+0000 7fa06e233640 1 --2- 172.21.15.169:0/2709612750 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa06815b610 0x7fa068149c80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa06000c7d0 tx=0x7fa06000cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:45.370 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:45.364+0000 7fa05e7fc640 1 -- 172.21.15.169:0/2709612750 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa060003b70 con 0x7fa06815b610 2023-10-26T01:30:45.370 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:45.364+0000 7fa05e7fc640 1 -- 172.21.15.169:0/2709612750 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa060003cf0 con 0x7fa06815b610 2023-10-26T01:30:45.371 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:45.364+0000 7fa06fc96640 1 -- 172.21.15.169:0/2709612750 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa06814a2b0 con 0x7fa06815b610 2023-10-26T01:30:45.371 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:45.364+0000 7fa05e7fc640 1 -- 172.21.15.169:0/2709612750 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa060003ea0 con 0x7fa06815b610 2023-10-26T01:30:45.371 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:45.364+0000 7fa06fc96640 1 -- 172.21.15.169:0/2709612750 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa06814a870 con 0x7fa06815b610 2023-10-26T01:30:45.372 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:45.365+0000 7fa06fc96640 1 -- 172.21.15.169:0/2709612750 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa06804db10 con 0x7fa06815b610 2023-10-26T01:30:45.372 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:45.366+0000 7fa05e7fc640 1 -- 172.21.15.169:0/2709612750 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fa06000aab0 con 0x7fa06815b610 2023-10-26T01:30:45.372 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:45.367+0000 7fa05e7fc640 1 --2- 172.21.15.169:0/2709612750 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa04c06ae40 0x7fa04c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:45.373 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:45.367+0000 7fa05e7fc640 1 -- 172.21.15.169:0/2709612750 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa0600895d0 con 0x7fa06815b610 2023-10-26T01:30:45.373 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:45.368+0000 7fa06da32640 1 --2- 172.21.15.169:0/2709612750 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa04c06ae40 0x7fa04c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:45.373 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:45.369+0000 7fa06da32640 1 --2- 172.21.15.169:0/2709612750 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa04c06ae40 0x7fa04c06d300 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa064009ac0 tx=0x7fa064005f90 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:45.374 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:45.372+0000 7fa05e7fc640 1 -- 172.21.15.169:0/2709612750 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa060054740 con 0x7fa06815b610 2023-10-26T01:30:45.601 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:45.598+0000 7fa06fc96640 1 -- 172.21.15.169:0/2709612750 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa06814c520 con 0x7fa06815b610 2023-10-26T01:30:45.602 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:45.600+0000 7fa05e7fc640 1 -- 172.21.15.169:0/2709612750 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4916 (secure 0 0 0) 0x7fa060057d60 con 0x7fa06815b610 2023-10-26T01:30:45.603 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:30:45.603 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4298,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1809801299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":1809801299},{"type":"v1","addr":"172.21.15.169:6835","nonce":1809801299}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4304,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/2248566141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2248566141},{"type":"v1","addr":"172.21.15.170:6836","nonce":2248566141}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4317,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/2383616530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2383616530},{"type":"v1","addr":"172.21.15.169:6834","nonce":2383616530}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T01:30:41.545068+0000","modified":"2023-10-26T01:30:44.577169+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4338},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4338":{"gid":4338,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6837/590609220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":590609220},{"type":"v1","addr":"172.21.15.170:6837","nonce":590609220}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-26T01:30:45.603 INFO:teuthology.orchestra.run.smithi169.stderr:dumped fsmap epoch 7 2023-10-26T01:30:45.605 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:45.602+0000 7fa06fc96640 1 -- 172.21.15.169:0/2709612750 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa04c06ae40 msgr2=0x7fa04c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:45.606 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:45.602+0000 7fa06fc96640 1 --2- 172.21.15.169:0/2709612750 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa04c06ae40 0x7fa04c06d300 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa064009ac0 tx=0x7fa064005f90 comp rx=0 tx=0).stop 2023-10-26T01:30:45.606 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:45.602+0000 7fa06fc96640 1 -- 172.21.15.169:0/2709612750 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa06815b610 msgr2=0x7fa068149c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:45.606 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:45.602+0000 7fa06fc96640 1 --2- 172.21.15.169:0/2709612750 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa06815b610 0x7fa068149c80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa06000c7d0 tx=0x7fa06000cc90 comp rx=0 tx=0).stop 2023-10-26T01:30:45.606 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:45.602+0000 7fa06fc96640 1 -- 172.21.15.169:0/2709612750 shutdown_connections 2023-10-26T01:30:45.607 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:45.602+0000 7fa06fc96640 1 --2- 172.21.15.169:0/2709612750 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa04c06ae40 0x7fa04c06d300 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:45.607 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:45.602+0000 7fa06fc96640 1 --2- 172.21.15.169:0/2709612750 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa06815b610 0x7fa068149c80 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:45.607 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:45.602+0000 7fa06fc96640 1 --2- 172.21.15.169:0/2709612750 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa0680ac940 0x7fa068149740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:45.607 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:45.602+0000 7fa06fc96640 1 --2- 172.21.15.169:0/2709612750 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa0680ac4e0 0x7fa068159070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:45.607 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:45.602+0000 7fa06fc96640 1 -- 172.21.15.169:0/2709612750 >> 172.21.15.169:0/2709612750 conn(0x7fa068069930 msgr2=0x7fa0680cc7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:45.608 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:45.602+0000 7fa06fc96640 1 -- 172.21.15.169:0/2709612750 shutdown_connections 2023-10-26T01:30:45.608 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:45.602+0000 7fa06fc96640 1 -- 172.21.15.169:0/2709612750 wait complete. 2023-10-26T01:30:45.624 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:30:45.841 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:45.838+0000 7f077ab1d640 1 -- 172.21.15.169:0/1752604752 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07740cd720 msgr2=0x7f07740cdb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:45.841 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:45.838+0000 7f077ab1d640 1 --2- 172.21.15.169:0/1752604752 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07740cd720 0x7f07740cdb90 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f07740acd20 tx=0x7f076c02c1d0 comp rx=0 tx=0).stop 2023-10-26T01:30:45.842 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:45.838+0000 7f077ab1d640 1 -- 172.21.15.169:0/1752604752 shutdown_connections 2023-10-26T01:30:45.842 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:45.838+0000 7f077ab1d640 1 --2- 172.21.15.169:0/1752604752 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07740cd720 0x7f07740cdb90 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:45.842 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:45.838+0000 7f077ab1d640 1 --2- 172.21.15.169:0/1752604752 >> v2:172.21.15.170:3300/0 conn(0x7f0774158850 0x7f077414e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:45.842 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:45.838+0000 7f077ab1d640 1 -- 172.21.15.169:0/1752604752 >> 172.21.15.169:0/1752604752 conn(0x7f0774069930 msgr2=0x7f0774069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:45.843 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:45.838+0000 7f077ab1d640 1 -- 172.21.15.169:0/1752604752 shutdown_connections 2023-10-26T01:30:45.843 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:45.838+0000 7f077ab1d640 1 -- 172.21.15.169:0/1752604752 wait complete. 2023-10-26T01:30:45.844 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:45.840+0000 7f077ab1d640 1 Processor -- start 2023-10-26T01:30:45.844 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:45.840+0000 7f077ab1d640 1 -- start start 2023-10-26T01:30:45.844 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:45.841+0000 7f077ab1d640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f077406ce20 0x7f07741426b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:45.845 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:45.841+0000 7f077ab1d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07740cd720 0x7f0774147450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:45.845 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:45.841+0000 7f077ab1d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0774158850 0x7f0774147990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:45.845 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:45.841+0000 7f077ab1d640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0774142bf0 con 0x7f07740cd720 2023-10-26T01:30:45.845 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:45.841+0000 7f077ab1d640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f077406ff90 con 0x7f0774158850 2023-10-26T01:30:45.845 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:45.841+0000 7f077ab1d640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0774070270 con 0x7f077406ce20 2023-10-26T01:30:45.846 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:45.841+0000 7f0773fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07740cd720 0x7f0774147450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:45.846 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:45.842+0000 7f0773fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07740cd720 0x7f0774147450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:33926/0 (socket says 172.21.15.169:33926) 2023-10-26T01:30:45.846 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:45.842+0000 7f0773fff640 1 -- 172.21.15.169:0/1159079901 learned_addr learned my addr 172.21.15.169:0/1159079901 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:30:45.847 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:45.842+0000 7f07790ba640 1 --2- 172.21.15.169:0/1159079901 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0774158850 0x7f0774147990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:45.848 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:45.842+0000 7f0773fff640 1 -- 172.21.15.169:0/1159079901 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f077406ce20 msgr2=0x7f07741426b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:45.848 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:45.842+0000 7f0773fff640 1 --2- 172.21.15.169:0/1159079901 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f077406ce20 0x7f07741426b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:45.848 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:45.842+0000 7f0773fff640 1 -- 172.21.15.169:0/1159079901 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0774158850 msgr2=0x7f0774147990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:45.848 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:45.842+0000 7f0773fff640 1 --2- 172.21.15.169:0/1159079901 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0774158850 0x7f0774147990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:45.848 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:45.842+0000 7f0773fff640 1 -- 172.21.15.169:0/1159079901 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f076c009900 con 0x7f07740cd720 2023-10-26T01:30:45.849 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:45.842+0000 7f0773fff640 1 --2- 172.21.15.169:0/1159079901 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07740cd720 0x7f0774147450 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f076000d6d0 tx=0x7f076000db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:45.849 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:45.842+0000 7f07717fa640 1 -- 172.21.15.169:0/1159079901 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0760005220 con 0x7f07740cd720 2023-10-26T01:30:45.849 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:45.842+0000 7f07717fa640 1 -- 172.21.15.169:0/1159079901 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0760005490 con 0x7f07740cd720 2023-10-26T01:30:45.849 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:45.842+0000 7f077ab1d640 1 -- 172.21.15.169:0/1159079901 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0774070550 con 0x7f07740cd720 2023-10-26T01:30:45.849 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:45.842+0000 7f07717fa640 1 -- 172.21.15.169:0/1159079901 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0760009a80 con 0x7f07740cd720 2023-10-26T01:30:45.850 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:45.843+0000 7f077ab1d640 1 -- 172.21.15.169:0/1159079901 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07740cc9f0 con 0x7f07740cd720 2023-10-26T01:30:45.850 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:45.844+0000 7f077ab1d640 1 -- 172.21.15.169:0/1159079901 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0774070a70 con 0x7f07740cd720 2023-10-26T01:30:45.850 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:45.845+0000 7f07717fa640 1 -- 172.21.15.169:0/1159079901 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f07600047a0 con 0x7f07740cd720 2023-10-26T01:30:45.851 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:45.845+0000 7f07717fa640 1 --2- 172.21.15.169:0/1159079901 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f075806ae40 0x7f075806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:45.851 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:45.846+0000 7f07717fa640 1 -- 172.21.15.169:0/1159079901 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0760089ab0 con 0x7f07740cd720 2023-10-26T01:30:45.851 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:45.846+0000 7f07788b9640 1 --2- 172.21.15.169:0/1159079901 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f075806ae40 0x7f075806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:45.851 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:45.847+0000 7f07788b9640 1 --2- 172.21.15.169:0/1159079901 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f075806ae40 0x7f075806d300 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0768001560 tx=0x7f0768007990 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:45.852 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:45.850+0000 7f07717fa640 1 -- 172.21.15.169:0/1159079901 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0774070a70 con 0x7f07740cd720 2023-10-26T01:30:46.045 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:46.042+0000 7f077ab1d640 1 -- 172.21.15.169:0/1159079901 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0774070a70 con 0x7f07740cd720 2023-10-26T01:30:46.048 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:46.045+0000 7f07717fa640 1 -- 172.21.15.169:0/1159079901 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16553 (secure 0 0 0) 0x7f0760081e60 con 0x7f07740cd720 2023-10-26T01:30:46.048 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:30:46.049 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":21,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:30:43.078706+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-26T01:30:39.437631+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-26T01:30:40.561774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:30:46.051 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:46.048+0000 7f077ab1d640 1 -- 172.21.15.169:0/1159079901 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f075806ae40 msgr2=0x7f075806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:46.051 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:46.048+0000 7f077ab1d640 1 --2- 172.21.15.169:0/1159079901 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f075806ae40 0x7f075806d300 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0768001560 tx=0x7f0768007990 comp rx=0 tx=0).stop 2023-10-26T01:30:46.051 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:46.048+0000 7f077ab1d640 1 -- 172.21.15.169:0/1159079901 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07740cd720 msgr2=0x7f0774147450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:46.052 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:46.048+0000 7f077ab1d640 1 --2- 172.21.15.169:0/1159079901 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07740cd720 0x7f0774147450 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f076000d6d0 tx=0x7f076000db90 comp rx=0 tx=0).stop 2023-10-26T01:30:46.052 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:46.048+0000 7f077ab1d640 1 -- 172.21.15.169:0/1159079901 shutdown_connections 2023-10-26T01:30:46.052 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:46.048+0000 7f077ab1d640 1 --2- 172.21.15.169:0/1159079901 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f075806ae40 0x7f075806d300 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:46.052 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:46.048+0000 7f077ab1d640 1 --2- 172.21.15.169:0/1159079901 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07740cd720 0x7f0774147450 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:46.052 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:46.048+0000 7f077ab1d640 1 --2- 172.21.15.169:0/1159079901 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0774158850 0x7f0774147990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:46.053 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:46.048+0000 7f077ab1d640 1 --2- 172.21.15.169:0/1159079901 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f077406ce20 0x7f07741426b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:46.053 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:46.048+0000 7f077ab1d640 1 -- 172.21.15.169:0/1159079901 >> 172.21.15.169:0/1159079901 conn(0x7f0774069930 msgr2=0x7f0774150620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:46.053 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:46.048+0000 7f077ab1d640 1 -- 172.21.15.169:0/1159079901 shutdown_connections 2023-10-26T01:30:46.053 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:46.048+0000 7f077ab1d640 1 -- 172.21.15.169:0/1159079901 wait complete. 2023-10-26T01:30:46.069 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-26T01:30:46.070 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-26T01:30:46.288 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:46.285+0000 7fb182e4f640 1 -- 172.21.15.169:0/1830991971 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb16c003fd0 msgr2=0x7fb16c0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:46.289 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:46.285+0000 7fb182e4f640 1 --2- 172.21.15.169:0/1830991971 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb16c003fd0 0x7fb16c0047c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb168001680 tx=0x7fb16c0057f0 comp rx=0 tx=0).stop 2023-10-26T01:30:46.289 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:46.285+0000 7fb1813ec640 1 -- 172.21.15.169:0/1830991971 reap_dead start 2023-10-26T01:30:46.289 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:46.286+0000 7fb182e4f640 1 -- 172.21.15.169:0/1830991971 shutdown_connections 2023-10-26T01:30:46.289 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:46.286+0000 7fb182e4f640 1 -- 172.21.15.169:0/1830991971 >> 172.21.15.169:0/1830991971 conn(0x7fb17c069930 msgr2=0x7fb17c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:46.290 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:46.286+0000 7fb182e4f640 1 -- 172.21.15.169:0/1830991971 shutdown_connections 2023-10-26T01:30:46.290 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:46.286+0000 7fb182e4f640 1 -- 172.21.15.169:0/1830991971 wait complete. 2023-10-26T01:30:46.291 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:46.287+0000 7fb182e4f640 1 Processor -- start 2023-10-26T01:30:46.291 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:46.287+0000 7fb182e4f640 1 -- start start 2023-10-26T01:30:46.291 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:46.288+0000 7fb182e4f640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb16c003fd0 0x7fb17c06ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:46.292 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:46.288+0000 7fb182e4f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb17c154c10 0x7fb17c0968a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:46.292 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:46.288+0000 7fb182e4f640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb17c0b1090 0x7fb17c0b1510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:46.292 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:46.288+0000 7fb182e4f640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb17c154810 con 0x7fb16c003fd0 2023-10-26T01:30:46.292 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:46.288+0000 7fb182e4f640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb17c154220 con 0x7fb17c154c10 2023-10-26T01:30:46.292 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:46.288+0000 7fb182e4f640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb17c0b1a50 con 0x7fb17c0b1090 2023-10-26T01:30:46.293 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:46.289+0000 7fb180beb640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb16c003fd0 0x7fb17c06ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:46.293 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:46.289+0000 7fb180beb640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb16c003fd0 0x7fb17c06ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:33944/0 (socket says 172.21.15.169:33944) 2023-10-26T01:30:46.293 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:46.289+0000 7fb180beb640 1 -- 172.21.15.169:0/1921432395 learned_addr learned my addr 172.21.15.169:0/1921432395 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:30:46.294 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:46.289+0000 7fb17bfff640 1 --2- 172.21.15.169:0/1921432395 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb17c154c10 0x7fb17c0968a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:46.295 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:46.289+0000 7fb180beb640 1 -- 172.21.15.169:0/1921432395 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb17c0b1090 msgr2=0x7fb17c0b1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:46.295 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:46.289+0000 7fb1813ec640 1 --2- 172.21.15.169:0/1921432395 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb17c0b1090 0x7fb17c0b1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:46.295 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:46.289+0000 7fb180beb640 1 --2- 172.21.15.169:0/1921432395 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb17c0b1090 0x7fb17c0b1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:46.295 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:46.289+0000 7fb180beb640 1 -- 172.21.15.169:0/1921432395 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb17c154c10 msgr2=0x7fb17c0968a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:46.295 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:46.289+0000 7fb180beb640 1 --2- 172.21.15.169:0/1921432395 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb17c154c10 0x7fb17c0968a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:46.296 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:46.289+0000 7fb180beb640 1 -- 172.21.15.169:0/1921432395 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb17c14eb40 con 0x7fb16c003fd0 2023-10-26T01:30:46.296 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:46.289+0000 7fb17bfff640 1 --2- 172.21.15.169:0/1921432395 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb17c154c10 0x7fb17c0968a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:30:46.296 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:46.289+0000 7fb180beb640 1 --2- 172.21.15.169:0/1921432395 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb16c003fd0 0x7fb17c06ed30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb17000e6d0 tx=0x7fb17000eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:46.296 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:46.290+0000 7fb1797fa640 1 -- 172.21.15.169:0/1921432395 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb170004250 con 0x7fb16c003fd0 2023-10-26T01:30:46.296 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:46.290+0000 7fb1797fa640 1 -- 172.21.15.169:0/1921432395 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb1700044a0 con 0x7fb16c003fd0 2023-10-26T01:30:46.296 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:46.290+0000 7fb1797fa640 1 -- 172.21.15.169:0/1921432395 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb17000caa0 con 0x7fb16c003fd0 2023-10-26T01:30:46.297 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:46.290+0000 7fb182e4f640 1 -- 172.21.15.169:0/1921432395 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb17c06f270 con 0x7fb16c003fd0 2023-10-26T01:30:46.297 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:46.290+0000 7fb182e4f640 1 -- 172.21.15.169:0/1921432395 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb17c153b70 con 0x7fb16c003fd0 2023-10-26T01:30:46.297 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:46.291+0000 7fb182e4f640 1 -- 172.21.15.169:0/1921432395 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb17c04db10 con 0x7fb16c003fd0 2023-10-26T01:30:46.297 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:46.292+0000 7fb1797fa640 1 -- 172.21.15.169:0/1921432395 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fb17000cc00 con 0x7fb16c003fd0 2023-10-26T01:30:46.298 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:46.293+0000 7fb1797fa640 1 --2- 172.21.15.169:0/1921432395 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb16006ae70 0x7fb16006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:46.298 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:46.294+0000 7fb1797fa640 1 -- 172.21.15.169:0/1921432395 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb170089b80 con 0x7fb16c003fd0 2023-10-26T01:30:46.298 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:46.294+0000 7fb17bfff640 1 --2- 172.21.15.169:0/1921432395 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb16006ae70 0x7fb16006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:46.299 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:46.295+0000 7fb17bfff640 1 --2- 172.21.15.169:0/1921432395 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb16006ae70 0x7fb16006d330 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb168034000 tx=0x7fb168039040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:46.300 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:46.298+0000 7fb1797fa640 1 -- 172.21.15.169:0/1921432395 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb170054c30 con 0x7fb16c003fd0 2023-10-26T01:30:46.485 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:46.482+0000 7fb182e4f640 1 -- 172.21.15.169:0/1921432395 --> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb17c06f270 con 0x7fb16006ae70 2023-10-26T01:30:46.510 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:46.508+0000 7fb1797fa640 1 -- 172.21.15.169:0/1921432395 <== mgr.4112 v2:172.21.15.170:6832/1187156918 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+259081 (secure 0 0 0) 0x7fb17c06f270 con 0x7fb16006ae70 2023-10-26T01:30:46.514 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:30:46.515 INFO:teuthology.orchestra.run.smithi169.stderr:dumped all 2023-10-26T01:30:46.517 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:46.513+0000 7fb182e4f640 1 -- 172.21.15.169:0/1921432395 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb16006ae70 msgr2=0x7fb16006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:46.517 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:46.513+0000 7fb182e4f640 1 --2- 172.21.15.169:0/1921432395 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb16006ae70 0x7fb16006d330 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb168034000 tx=0x7fb168039040 comp rx=0 tx=0).stop 2023-10-26T01:30:46.517 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:46.513+0000 7fb182e4f640 1 -- 172.21.15.169:0/1921432395 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb16c003fd0 msgr2=0x7fb17c06ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:46.517 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:46.513+0000 7fb182e4f640 1 --2- 172.21.15.169:0/1921432395 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb16c003fd0 0x7fb17c06ed30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb17000e6d0 tx=0x7fb17000eb90 comp rx=0 tx=0).stop 2023-10-26T01:30:46.518 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:46.513+0000 7fb182e4f640 1 -- 172.21.15.169:0/1921432395 shutdown_connections 2023-10-26T01:30:46.518 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:46.513+0000 7fb182e4f640 1 --2- 172.21.15.169:0/1921432395 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb16006ae70 0x7fb16006d330 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:46.518 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:46.513+0000 7fb182e4f640 1 --2- 172.21.15.169:0/1921432395 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb17c0b1090 0x7fb17c0b1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:46.518 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:46.513+0000 7fb182e4f640 1 --2- 172.21.15.169:0/1921432395 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb17c154c10 0x7fb17c0968a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:46.519 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:46.513+0000 7fb182e4f640 1 --2- 172.21.15.169:0/1921432395 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb16c003fd0 0x7fb17c06ed30 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:46.519 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:46.513+0000 7fb182e4f640 1 -- 172.21.15.169:0/1921432395 >> 172.21.15.169:0/1921432395 conn(0x7fb17c069930 msgr2=0x7fb17c06c030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:46.519 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:46.513+0000 7fb182e4f640 1 -- 172.21.15.169:0/1921432395 shutdown_connections 2023-10-26T01:30:46.519 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:46.513+0000 7fb182e4f640 1 -- 172.21.15.169:0/1921432395 wait complete. 2023-10-26T01:30:46.535 INFO:teuthology.orchestra.run.smithi169.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-10-26T01:30:46.128888+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":153,"ondisk_log_size":153,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":169,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":1723,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560776,"statfs":{"total":1600000000,"available":1598234860,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":58,"num_write_kb":276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001802"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.086425+0000","last_change":"2023-10-26T01:30:42.112552+0000","last_active":"2023-10-26T01:30:43.086425+0000","last_peered":"2023-10-26T01:30:43.086425+0000","last_clean":"2023-10-26T01:30:43.086425+0000","last_became_active":"2023-10-26T01:30:42.111151+0000","last_became_peered":"2023-10-26T01:30:42.111151+0000","last_unstale":"2023-10-26T01:30:43.086425+0000","last_undegraded":"2023-10-26T01:30:43.086425+0000","last_fullsized":"2023-10-26T01:30:43.086425+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:28:22.384005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.141033+0000","last_change":"2023-10-26T01:30:42.102872+0000","last_active":"2023-10-26T01:30:43.141033+0000","last_peered":"2023-10-26T01:30:43.141033+0000","last_clean":"2023-10-26T01:30:43.141033+0000","last_became_active":"2023-10-26T01:30:42.100796+0000","last_became_peered":"2023-10-26T01:30:42.100796+0000","last_unstale":"2023-10-26T01:30:43.141033+0000","last_undegraded":"2023-10-26T01:30:43.141033+0000","last_fullsized":"2023-10-26T01:30:43.141033+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:49:12.753504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.139076+0000","last_change":"2023-10-26T01:30:42.103921+0000","last_active":"2023-10-26T01:30:43.139076+0000","last_peered":"2023-10-26T01:30:43.139076+0000","last_clean":"2023-10-26T01:30:43.139076+0000","last_became_active":"2023-10-26T01:30:42.103384+0000","last_became_peered":"2023-10-26T01:30:42.103384+0000","last_unstale":"2023-10-26T01:30:43.139076+0000","last_undegraded":"2023-10-26T01:30:43.139076+0000","last_fullsized":"2023-10-26T01:30:43.139076+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:33:58.768305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.098597+0000","last_change":"2023-10-26T01:30:42.101845+0000","last_active":"2023-10-26T01:30:43.098597+0000","last_peered":"2023-10-26T01:30:43.098597+0000","last_clean":"2023-10-26T01:30:43.098597+0000","last_became_active":"2023-10-26T01:30:42.097125+0000","last_became_peered":"2023-10-26T01:30:42.097125+0000","last_unstale":"2023-10-26T01:30:43.098597+0000","last_undegraded":"2023-10-26T01:30:43.098597+0000","last_fullsized":"2023-10-26T01:30:43.098597+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:33:13.615628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.086419+0000","last_change":"2023-10-26T01:30:42.115244+0000","last_active":"2023-10-26T01:30:43.086419+0000","last_peered":"2023-10-26T01:30:43.086419+0000","last_clean":"2023-10-26T01:30:43.086419+0000","last_became_active":"2023-10-26T01:30:42.114972+0000","last_became_peered":"2023-10-26T01:30:42.114972+0000","last_unstale":"2023-10-26T01:30:43.086419+0000","last_undegraded":"2023-10-26T01:30:43.086419+0000","last_fullsized":"2023-10-26T01:30:43.086419+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:07:06.450331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.099993+0000","last_change":"2023-10-26T01:30:42.108711+0000","last_active":"2023-10-26T01:30:43.099993+0000","last_peered":"2023-10-26T01:30:43.099993+0000","last_clean":"2023-10-26T01:30:43.099993+0000","last_became_active":"2023-10-26T01:30:42.108215+0000","last_became_peered":"2023-10-26T01:30:42.108215+0000","last_unstale":"2023-10-26T01:30:43.099993+0000","last_undegraded":"2023-10-26T01:30:43.099993+0000","last_fullsized":"2023-10-26T01:30:43.099993+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:26:01.898201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.088319+0000","last_change":"2023-10-26T01:30:42.115268+0000","last_active":"2023-10-26T01:30:43.088319+0000","last_peered":"2023-10-26T01:30:43.088319+0000","last_clean":"2023-10-26T01:30:43.088319+0000","last_became_active":"2023-10-26T01:30:42.113208+0000","last_became_peered":"2023-10-26T01:30:42.113208+0000","last_unstale":"2023-10-26T01:30:43.088319+0000","last_undegraded":"2023-10-26T01:30:43.088319+0000","last_fullsized":"2023-10-26T01:30:43.088319+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:56:55.394232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-10-26T01:30:41.100533+0000","last_change":"2023-10-26T01:30:41.100533+0000","last_active":"2023-10-26T01:30:41.060332+0000","last_peered":"2023-10-26T01:30:41.060332+0000","last_clean":"2023-10-26T01:30:41.060332+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:30:41.100533+0000","last_undegraded":"2023-10-26T01:30:41.100533+0000","last_fullsized":"2023-10-26T01:30:41.100533+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.086093+0000","last_change":"2023-10-26T01:30:42.108451+0000","last_active":"2023-10-26T01:30:43.086093+0000","last_peered":"2023-10-26T01:30:43.086093+0000","last_clean":"2023-10-26T01:30:43.086093+0000","last_became_active":"2023-10-26T01:30:42.107650+0000","last_became_peered":"2023-10-26T01:30:42.107650+0000","last_unstale":"2023-10-26T01:30:43.086093+0000","last_undegraded":"2023-10-26T01:30:43.086093+0000","last_fullsized":"2023-10-26T01:30:43.086093+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:26:19.035685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.098895+0000","last_change":"2023-10-26T01:30:42.093838+0000","last_active":"2023-10-26T01:30:43.098895+0000","last_peered":"2023-10-26T01:30:43.098895+0000","last_clean":"2023-10-26T01:30:43.098895+0000","last_became_active":"2023-10-26T01:30:42.093584+0000","last_became_peered":"2023-10-26T01:30:42.093584+0000","last_unstale":"2023-10-26T01:30:43.098895+0000","last_undegraded":"2023-10-26T01:30:43.098895+0000","last_fullsized":"2023-10-26T01:30:43.098895+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:49:12.753504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.097658+0000","last_change":"2023-10-26T01:30:42.094026+0000","last_active":"2023-10-26T01:30:43.097658+0000","last_peered":"2023-10-26T01:30:43.097658+0000","last_clean":"2023-10-26T01:30:43.097658+0000","last_became_active":"2023-10-26T01:30:42.093654+0000","last_became_peered":"2023-10-26T01:30:42.093654+0000","last_unstale":"2023-10-26T01:30:43.097658+0000","last_undegraded":"2023-10-26T01:30:43.097658+0000","last_fullsized":"2023-10-26T01:30:43.097658+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:05:31.590291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.088414+0000","last_change":"2023-10-26T01:30:42.099764+0000","last_active":"2023-10-26T01:30:43.088414+0000","last_peered":"2023-10-26T01:30:43.088414+0000","last_clean":"2023-10-26T01:30:43.088414+0000","last_became_active":"2023-10-26T01:30:42.098108+0000","last_became_peered":"2023-10-26T01:30:42.098108+0000","last_unstale":"2023-10-26T01:30:43.088414+0000","last_undegraded":"2023-10-26T01:30:43.088414+0000","last_fullsized":"2023-10-26T01:30:43.088414+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:08:21.563278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-10-26T01:30:41.099897+0000","last_change":"2023-10-26T01:30:41.099897+0000","last_active":"2023-10-26T01:30:41.060332+0000","last_peered":"2023-10-26T01:30:41.060332+0000","last_clean":"2023-10-26T01:30:41.060332+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:30:41.099897+0000","last_undegraded":"2023-10-26T01:30:41.099897+0000","last_fullsized":"2023-10-26T01:30:41.099897+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.138887+0000","last_change":"2023-10-26T01:30:42.108715+0000","last_active":"2023-10-26T01:30:43.138887+0000","last_peered":"2023-10-26T01:30:43.138887+0000","last_clean":"2023-10-26T01:30:43.138887+0000","last_became_active":"2023-10-26T01:30:42.108469+0000","last_became_peered":"2023-10-26T01:30:42.108469+0000","last_unstale":"2023-10-26T01:30:43.138887+0000","last_undegraded":"2023-10-26T01:30:43.138887+0000","last_fullsized":"2023-10-26T01:30:43.138887+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:45:23.061469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.140916+0000","last_change":"2023-10-26T01:30:42.099157+0000","last_active":"2023-10-26T01:30:43.140916+0000","last_peered":"2023-10-26T01:30:43.140916+0000","last_clean":"2023-10-26T01:30:43.140916+0000","last_became_active":"2023-10-26T01:30:42.098929+0000","last_became_peered":"2023-10-26T01:30:42.098929+0000","last_unstale":"2023-10-26T01:30:43.140916+0000","last_undegraded":"2023-10-26T01:30:43.140916+0000","last_fullsized":"2023-10-26T01:30:43.140916+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:13:33.838018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.099835+0000","last_change":"2023-10-26T01:30:42.108578+0000","last_active":"2023-10-26T01:30:43.099835+0000","last_peered":"2023-10-26T01:30:43.099835+0000","last_clean":"2023-10-26T01:30:43.099835+0000","last_became_active":"2023-10-26T01:30:42.107937+0000","last_became_peered":"2023-10-26T01:30:42.107937+0000","last_unstale":"2023-10-26T01:30:43.099835+0000","last_undegraded":"2023-10-26T01:30:43.099835+0000","last_fullsized":"2023-10-26T01:30:43.099835+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:38:16.726919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.098559+0000","last_change":"2023-10-26T01:30:42.108373+0000","last_active":"2023-10-26T01:30:43.098559+0000","last_peered":"2023-10-26T01:30:43.098559+0000","last_clean":"2023-10-26T01:30:43.098559+0000","last_became_active":"2023-10-26T01:30:42.107773+0000","last_became_peered":"2023-10-26T01:30:42.107773+0000","last_unstale":"2023-10-26T01:30:43.098559+0000","last_undegraded":"2023-10-26T01:30:43.098559+0000","last_fullsized":"2023-10-26T01:30:43.098559+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:49:12.753504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.097381+0000","last_change":"2023-10-26T01:30:42.094993+0000","last_active":"2023-10-26T01:30:43.097381+0000","last_peered":"2023-10-26T01:30:43.097381+0000","last_clean":"2023-10-26T01:30:43.097381+0000","last_became_active":"2023-10-26T01:30:42.094689+0000","last_became_peered":"2023-10-26T01:30:42.094689+0000","last_unstale":"2023-10-26T01:30:43.097381+0000","last_undegraded":"2023-10-26T01:30:43.097381+0000","last_fullsized":"2023-10-26T01:30:43.097381+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:49:43.468558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.086773+0000","last_change":"2023-10-26T01:30:42.108746+0000","last_active":"2023-10-26T01:30:43.086773+0000","last_peered":"2023-10-26T01:30:43.086773+0000","last_clean":"2023-10-26T01:30:43.086773+0000","last_became_active":"2023-10-26T01:30:42.108188+0000","last_became_peered":"2023-10-26T01:30:42.108188+0000","last_unstale":"2023-10-26T01:30:43.086773+0000","last_undegraded":"2023-10-26T01:30:43.086773+0000","last_fullsized":"2023-10-26T01:30:43.086773+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:38:16.726919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.099061+0000","last_change":"2023-10-26T01:30:42.110331+0000","last_active":"2023-10-26T01:30:43.099061+0000","last_peered":"2023-10-26T01:30:43.099061+0000","last_clean":"2023-10-26T01:30:43.099061+0000","last_became_active":"2023-10-26T01:30:42.110079+0000","last_became_peered":"2023-10-26T01:30:42.110079+0000","last_unstale":"2023-10-26T01:30:43.099061+0000","last_undegraded":"2023-10-26T01:30:43.099061+0000","last_fullsized":"2023-10-26T01:30:43.099061+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:33:13.615628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.139607+0000","last_change":"2023-10-26T01:30:42.108608+0000","last_active":"2023-10-26T01:30:43.139607+0000","last_peered":"2023-10-26T01:30:43.139607+0000","last_clean":"2023-10-26T01:30:43.139607+0000","last_became_active":"2023-10-26T01:30:42.108197+0000","last_became_peered":"2023-10-26T01:30:42.108197+0000","last_unstale":"2023-10-26T01:30:43.139607+0000","last_undegraded":"2023-10-26T01:30:43.139607+0000","last_fullsized":"2023-10-26T01:30:43.139607+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:17:59.186810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.098944+0000","last_change":"2023-10-26T01:30:42.110289+0000","last_active":"2023-10-26T01:30:43.098944+0000","last_peered":"2023-10-26T01:30:43.098944+0000","last_clean":"2023-10-26T01:30:43.098944+0000","last_became_active":"2023-10-26T01:30:42.110061+0000","last_became_peered":"2023-10-26T01:30:42.110061+0000","last_unstale":"2023-10-26T01:30:43.098944+0000","last_undegraded":"2023-10-26T01:30:43.098944+0000","last_fullsized":"2023-10-26T01:30:43.098944+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:19:23.544531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-10-26T01:30:41.098174+0000","last_change":"2023-10-26T01:30:41.098174+0000","last_active":"2023-10-26T01:30:41.060332+0000","last_peered":"2023-10-26T01:30:41.060332+0000","last_clean":"2023-10-26T01:30:41.060332+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:30:41.098174+0000","last_undegraded":"2023-10-26T01:30:41.098174+0000","last_fullsized":"2023-10-26T01:30:41.098174+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.098984+0000","last_change":"2023-10-26T01:30:42.102147+0000","last_active":"2023-10-26T01:30:43.098984+0000","last_peered":"2023-10-26T01:30:43.098984+0000","last_clean":"2023-10-26T01:30:43.098984+0000","last_became_active":"2023-10-26T01:30:42.097509+0000","last_became_peered":"2023-10-26T01:30:42.097509+0000","last_unstale":"2023-10-26T01:30:43.098984+0000","last_undegraded":"2023-10-26T01:30:43.098984+0000","last_fullsized":"2023-10-26T01:30:43.098984+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:26:19.035685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-10-26T01:30:41.098206+0000","last_change":"2023-10-26T01:30:41.098206+0000","last_active":"2023-10-26T01:30:41.060332+0000","last_peered":"2023-10-26T01:30:41.060332+0000","last_clean":"2023-10-26T01:30:41.060332+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:30:41.098206+0000","last_undegraded":"2023-10-26T01:30:41.098206+0000","last_fullsized":"2023-10-26T01:30:41.098206+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.097862+0000","last_change":"2023-10-26T01:30:42.111865+0000","last_active":"2023-10-26T01:30:43.097862+0000","last_peered":"2023-10-26T01:30:43.097862+0000","last_clean":"2023-10-26T01:30:43.097862+0000","last_became_active":"2023-10-26T01:30:42.111636+0000","last_became_peered":"2023-10-26T01:30:42.111636+0000","last_unstale":"2023-10-26T01:30:43.097862+0000","last_undegraded":"2023-10-26T01:30:43.097862+0000","last_fullsized":"2023-10-26T01:30:43.097862+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:49:56.641022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.141348+0000","last_change":"2023-10-26T01:30:42.091332+0000","last_active":"2023-10-26T01:30:43.141348+0000","last_peered":"2023-10-26T01:30:43.141348+0000","last_clean":"2023-10-26T01:30:43.141348+0000","last_became_active":"2023-10-26T01:30:42.091105+0000","last_became_peered":"2023-10-26T01:30:42.091105+0000","last_unstale":"2023-10-26T01:30:43.141348+0000","last_undegraded":"2023-10-26T01:30:43.141348+0000","last_fullsized":"2023-10-26T01:30:43.141348+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:33:13.615628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-10-26T01:30:41.095233+0000","last_change":"2023-10-26T01:30:41.095233+0000","last_active":"2023-10-26T01:30:41.060332+0000","last_peered":"2023-10-26T01:30:41.060332+0000","last_clean":"2023-10-26T01:30:41.060332+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:30:41.095233+0000","last_undegraded":"2023-10-26T01:30:41.095233+0000","last_fullsized":"2023-10-26T01:30:41.095233+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.139200+0000","last_change":"2023-10-26T01:30:42.101708+0000","last_active":"2023-10-26T01:30:43.139200+0000","last_peered":"2023-10-26T01:30:43.139200+0000","last_clean":"2023-10-26T01:30:43.139200+0000","last_became_active":"2023-10-26T01:30:42.101434+0000","last_became_peered":"2023-10-26T01:30:42.101434+0000","last_unstale":"2023-10-26T01:30:43.139200+0000","last_undegraded":"2023-10-26T01:30:43.139200+0000","last_fullsized":"2023-10-26T01:30:43.139200+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:29:18.643148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.099207+0000","last_change":"2023-10-26T01:30:42.097495+0000","last_active":"2023-10-26T01:30:43.099207+0000","last_peered":"2023-10-26T01:30:43.099207+0000","last_clean":"2023-10-26T01:30:43.099207+0000","last_became_active":"2023-10-26T01:30:42.097214+0000","last_became_peered":"2023-10-26T01:30:42.097214+0000","last_unstale":"2023-10-26T01:30:43.099207+0000","last_undegraded":"2023-10-26T01:30:43.099207+0000","last_fullsized":"2023-10-26T01:30:43.099207+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T01:55:28.470913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.097481+0000","last_change":"2023-10-26T01:30:42.093157+0000","last_active":"2023-10-26T01:30:43.097481+0000","last_peered":"2023-10-26T01:30:43.097481+0000","last_clean":"2023-10-26T01:30:43.097481+0000","last_became_active":"2023-10-26T01:30:42.092613+0000","last_became_peered":"2023-10-26T01:30:42.092613+0000","last_unstale":"2023-10-26T01:30:43.097481+0000","last_undegraded":"2023-10-26T01:30:43.097481+0000","last_fullsized":"2023-10-26T01:30:43.097481+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:09:01.061004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.098226+0000","last_change":"2023-10-26T01:30:42.102704+0000","last_active":"2023-10-26T01:30:43.098226+0000","last_peered":"2023-10-26T01:30:43.098226+0000","last_clean":"2023-10-26T01:30:43.098226+0000","last_became_active":"2023-10-26T01:30:42.102350+0000","last_became_peered":"2023-10-26T01:30:42.102350+0000","last_unstale":"2023-10-26T01:30:43.098226+0000","last_undegraded":"2023-10-26T01:30:43.098226+0000","last_fullsized":"2023-10-26T01:30:43.098226+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:53:57.632838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.086493+0000","last_change":"2023-10-26T01:30:42.093818+0000","last_active":"2023-10-26T01:30:43.086493+0000","last_peered":"2023-10-26T01:30:43.086493+0000","last_clean":"2023-10-26T01:30:43.086493+0000","last_became_active":"2023-10-26T01:30:42.091310+0000","last_became_peered":"2023-10-26T01:30:42.091310+0000","last_unstale":"2023-10-26T01:30:43.086493+0000","last_undegraded":"2023-10-26T01:30:43.086493+0000","last_fullsized":"2023-10-26T01:30:43.086493+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:04:23.158155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-26T01:30:41.097736+0000","last_change":"2023-10-26T01:30:41.097666+0000","last_active":"2023-10-26T01:30:41.097736+0000","last_peered":"2023-10-26T01:30:41.097736+0000","last_clean":"2023-10-26T01:30:41.097736+0000","last_became_active":"2023-10-26T01:30:41.093986+0000","last_became_peered":"2023-10-26T01:30:41.093986+0000","last_unstale":"2023-10-26T01:30:41.097736+0000","last_undegraded":"2023-10-26T01:30:41.097736+0000","last_fullsized":"2023-10-26T01:30:41.097736+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:13:28.300112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.139516+0000","last_change":"2023-10-26T01:30:42.110839+0000","last_active":"2023-10-26T01:30:43.139516+0000","last_peered":"2023-10-26T01:30:43.139516+0000","last_clean":"2023-10-26T01:30:43.139516+0000","last_became_active":"2023-10-26T01:30:42.109385+0000","last_became_peered":"2023-10-26T01:30:42.109385+0000","last_unstale":"2023-10-26T01:30:43.139516+0000","last_undegraded":"2023-10-26T01:30:43.139516+0000","last_fullsized":"2023-10-26T01:30:43.139516+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:49:56.641022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.105272+0000","last_change":"2023-10-26T01:30:41.100248+0000","last_active":"2023-10-26T01:30:42.105272+0000","last_peered":"2023-10-26T01:30:42.105272+0000","last_clean":"2023-10-26T01:30:42.105272+0000","last_became_active":"2023-10-26T01:30:41.099957+0000","last_became_peered":"2023-10-26T01:30:41.099957+0000","last_unstale":"2023-10-26T01:30:42.105272+0000","last_undegraded":"2023-10-26T01:30:42.105272+0000","last_fullsized":"2023-10-26T01:30:42.105272+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:29:17.626922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-26T01:30:41.104334+0000","last_change":"2023-10-26T01:30:41.103843+0000","last_active":"2023-10-26T01:30:41.104334+0000","last_peered":"2023-10-26T01:30:41.104334+0000","last_clean":"2023-10-26T01:30:41.104334+0000","last_became_active":"2023-10-26T01:30:41.103356+0000","last_became_peered":"2023-10-26T01:30:41.103356+0000","last_unstale":"2023-10-26T01:30:41.104334+0000","last_undegraded":"2023-10-26T01:30:41.104334+0000","last_fullsized":"2023-10-26T01:30:41.104334+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:05:43.728770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-10-26T01:30:41.097717+0000","last_change":"2023-10-26T01:30:41.097717+0000","last_active":"2023-10-26T01:30:41.060332+0000","last_peered":"2023-10-26T01:30:41.060332+0000","last_clean":"2023-10-26T01:30:41.060332+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:30:41.097717+0000","last_undegraded":"2023-10-26T01:30:41.097717+0000","last_fullsized":"2023-10-26T01:30:41.097717+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.089693+0000","last_change":"2023-10-26T01:30:41.102989+0000","last_active":"2023-10-26T01:30:42.089693+0000","last_peered":"2023-10-26T01:30:42.089693+0000","last_clean":"2023-10-26T01:30:42.089693+0000","last_became_active":"2023-10-26T01:30:41.102681+0000","last_became_peered":"2023-10-26T01:30:41.102681+0000","last_unstale":"2023-10-26T01:30:42.089693+0000","last_undegraded":"2023-10-26T01:30:42.089693+0000","last_fullsized":"2023-10-26T01:30:42.089693+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:49:55.624796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.140341+0000","last_change":"2023-10-26T01:30:42.091359+0000","last_active":"2023-10-26T01:30:43.140341+0000","last_peered":"2023-10-26T01:30:43.140341+0000","last_clean":"2023-10-26T01:30:43.140341+0000","last_became_active":"2023-10-26T01:30:42.090949+0000","last_became_peered":"2023-10-26T01:30:42.090949+0000","last_unstale":"2023-10-26T01:30:43.140341+0000","last_undegraded":"2023-10-26T01:30:43.140341+0000","last_fullsized":"2023-10-26T01:30:43.140341+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:49:56.641022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.089836+0000","last_change":"2023-10-26T01:30:41.104058+0000","last_active":"2023-10-26T01:30:42.089836+0000","last_peered":"2023-10-26T01:30:42.089836+0000","last_clean":"2023-10-26T01:30:42.089836+0000","last_became_active":"2023-10-26T01:30:41.103821+0000","last_became_peered":"2023-10-26T01:30:41.103821+0000","last_unstale":"2023-10-26T01:30:42.089836+0000","last_undegraded":"2023-10-26T01:30:42.089836+0000","last_fullsized":"2023-10-26T01:30:42.089836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:46:05.927163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.098774+0000","last_change":"2023-10-26T01:30:42.108520+0000","last_active":"2023-10-26T01:30:43.098774+0000","last_peered":"2023-10-26T01:30:43.098774+0000","last_clean":"2023-10-26T01:30:43.098774+0000","last_became_active":"2023-10-26T01:30:42.108140+0000","last_became_peered":"2023-10-26T01:30:42.108140+0000","last_unstale":"2023-10-26T01:30:43.098774+0000","last_undegraded":"2023-10-26T01:30:43.098774+0000","last_fullsized":"2023-10-26T01:30:43.098774+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:31:08.727626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.088126+0000","last_change":"2023-10-26T01:30:41.106779+0000","last_active":"2023-10-26T01:30:42.088126+0000","last_peered":"2023-10-26T01:30:42.088126+0000","last_clean":"2023-10-26T01:30:42.088126+0000","last_became_active":"2023-10-26T01:30:41.104353+0000","last_became_peered":"2023-10-26T01:30:41.104353+0000","last_unstale":"2023-10-26T01:30:42.088126+0000","last_undegraded":"2023-10-26T01:30:42.088126+0000","last_fullsized":"2023-10-26T01:30:42.088126+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:45:22.045243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.139887+0000","last_change":"2023-10-26T01:30:42.105447+0000","last_active":"2023-10-26T01:30:43.139887+0000","last_peered":"2023-10-26T01:30:43.139887+0000","last_clean":"2023-10-26T01:30:43.139887+0000","last_became_active":"2023-10-26T01:30:42.105098+0000","last_became_peered":"2023-10-26T01:30:42.105098+0000","last_unstale":"2023-10-26T01:30:43.139887+0000","last_undegraded":"2023-10-26T01:30:43.139887+0000","last_fullsized":"2023-10-26T01:30:43.139887+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:13:33.838018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.110602+0000","last_change":"2023-10-26T01:30:41.103826+0000","last_active":"2023-10-26T01:30:42.110602+0000","last_peered":"2023-10-26T01:30:42.110602+0000","last_clean":"2023-10-26T01:30:42.110602+0000","last_became_active":"2023-10-26T01:30:41.103552+0000","last_became_peered":"2023-10-26T01:30:41.103552+0000","last_unstale":"2023-10-26T01:30:42.110602+0000","last_undegraded":"2023-10-26T01:30:42.110602+0000","last_fullsized":"2023-10-26T01:30:42.110602+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:54:04.970133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.086144+0000","last_change":"2023-10-26T01:30:42.114167+0000","last_active":"2023-10-26T01:30:43.086144+0000","last_peered":"2023-10-26T01:30:43.086144+0000","last_clean":"2023-10-26T01:30:43.086144+0000","last_became_active":"2023-10-26T01:30:42.113241+0000","last_became_peered":"2023-10-26T01:30:42.113241+0000","last_unstale":"2023-10-26T01:30:43.086144+0000","last_undegraded":"2023-10-26T01:30:43.086144+0000","last_fullsized":"2023-10-26T01:30:43.086144+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:40:16.215681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.116049+0000","last_change":"2023-10-26T01:30:41.106514+0000","last_active":"2023-10-26T01:30:42.116049+0000","last_peered":"2023-10-26T01:30:42.116049+0000","last_clean":"2023-10-26T01:30:42.116049+0000","last_became_active":"2023-10-26T01:30:41.106147+0000","last_became_peered":"2023-10-26T01:30:41.106147+0000","last_unstale":"2023-10-26T01:30:42.116049+0000","last_undegraded":"2023-10-26T01:30:42.116049+0000","last_fullsized":"2023-10-26T01:30:42.116049+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:45:22.045243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.089920+0000","last_change":"2023-10-26T01:30:42.112813+0000","last_active":"2023-10-26T01:30:43.089920+0000","last_peered":"2023-10-26T01:30:43.089920+0000","last_clean":"2023-10-26T01:30:43.089920+0000","last_became_active":"2023-10-26T01:30:42.112571+0000","last_became_peered":"2023-10-26T01:30:42.112571+0000","last_unstale":"2023-10-26T01:30:43.089920+0000","last_undegraded":"2023-10-26T01:30:43.089920+0000","last_fullsized":"2023-10-26T01:30:43.089920+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:19:16.042472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.108474+0000","last_change":"2023-10-26T01:30:41.098865+0000","last_active":"2023-10-26T01:30:42.108474+0000","last_peered":"2023-10-26T01:30:42.108474+0000","last_clean":"2023-10-26T01:30:42.108474+0000","last_became_active":"2023-10-26T01:30:41.098506+0000","last_became_peered":"2023-10-26T01:30:41.098506+0000","last_unstale":"2023-10-26T01:30:42.108474+0000","last_undegraded":"2023-10-26T01:30:42.108474+0000","last_fullsized":"2023-10-26T01:30:42.108474+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:09:00.044778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.140188+0000","last_change":"2023-10-26T01:30:42.108443+0000","last_active":"2023-10-26T01:30:43.140188+0000","last_peered":"2023-10-26T01:30:43.140188+0000","last_clean":"2023-10-26T01:30:43.140188+0000","last_became_active":"2023-10-26T01:30:42.107837+0000","last_became_peered":"2023-10-26T01:30:42.107837+0000","last_unstale":"2023-10-26T01:30:43.140188+0000","last_undegraded":"2023-10-26T01:30:43.140188+0000","last_fullsized":"2023-10-26T01:30:43.140188+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:33:13.615628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.090761+0000","last_change":"2023-10-26T01:30:41.106795+0000","last_active":"2023-10-26T01:30:42.090761+0000","last_peered":"2023-10-26T01:30:42.090761+0000","last_clean":"2023-10-26T01:30:42.090761+0000","last_became_active":"2023-10-26T01:30:41.106222+0000","last_became_peered":"2023-10-26T01:30:41.106222+0000","last_unstale":"2023-10-26T01:30:42.090761+0000","last_undegraded":"2023-10-26T01:30:42.090761+0000","last_fullsized":"2023-10-26T01:30:42.090761+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:53:56.616612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.086822+0000","last_change":"2023-10-26T01:30:42.113053+0000","last_active":"2023-10-26T01:30:43.086822+0000","last_peered":"2023-10-26T01:30:43.086822+0000","last_clean":"2023-10-26T01:30:43.086822+0000","last_became_active":"2023-10-26T01:30:42.112492+0000","last_became_peered":"2023-10-26T01:30:42.112492+0000","last_unstale":"2023-10-26T01:30:43.086822+0000","last_undegraded":"2023-10-26T01:30:43.086822+0000","last_fullsized":"2023-10-26T01:30:43.086822+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:20:47.431800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.089525+0000","last_change":"2023-10-26T01:30:41.100835+0000","last_active":"2023-10-26T01:30:42.089525+0000","last_peered":"2023-10-26T01:30:42.089525+0000","last_clean":"2023-10-26T01:30:42.089525+0000","last_became_active":"2023-10-26T01:30:41.099708+0000","last_became_peered":"2023-10-26T01:30:41.099708+0000","last_unstale":"2023-10-26T01:30:42.089525+0000","last_undegraded":"2023-10-26T01:30:42.089525+0000","last_fullsized":"2023-10-26T01:30:42.089525+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:04:22.141930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.098337+0000","last_change":"2023-10-26T01:30:42.100857+0000","last_active":"2023-10-26T01:30:43.098337+0000","last_peered":"2023-10-26T01:30:43.098337+0000","last_clean":"2023-10-26T01:30:43.098337+0000","last_became_active":"2023-10-26T01:30:42.100545+0000","last_became_peered":"2023-10-26T01:30:42.100545+0000","last_unstale":"2023-10-26T01:30:43.098337+0000","last_undegraded":"2023-10-26T01:30:43.098337+0000","last_fullsized":"2023-10-26T01:30:43.098337+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:13:33.838018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.114399+0000","last_change":"2023-10-26T01:30:41.095998+0000","last_active":"2023-10-26T01:30:42.114399+0000","last_peered":"2023-10-26T01:30:42.114399+0000","last_clean":"2023-10-26T01:30:42.114399+0000","last_became_active":"2023-10-26T01:30:41.095494+0000","last_became_peered":"2023-10-26T01:30:41.095494+0000","last_unstale":"2023-10-26T01:30:42.114399+0000","last_undegraded":"2023-10-26T01:30:42.114399+0000","last_fullsized":"2023-10-26T01:30:42.114399+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:54:04.970133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.089073+0000","last_change":"2023-10-26T01:30:42.115510+0000","last_active":"2023-10-26T01:30:43.089073+0000","last_peered":"2023-10-26T01:30:43.089073+0000","last_clean":"2023-10-26T01:30:43.089073+0000","last_became_active":"2023-10-26T01:30:42.113552+0000","last_became_peered":"2023-10-26T01:30:42.113552+0000","last_unstale":"2023-10-26T01:30:43.089073+0000","last_undegraded":"2023-10-26T01:30:43.089073+0000","last_fullsized":"2023-10-26T01:30:43.089073+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:32:08.387966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.110967+0000","last_change":"2023-10-26T01:30:41.093200+0000","last_active":"2023-10-26T01:30:42.110967+0000","last_peered":"2023-10-26T01:30:42.110967+0000","last_clean":"2023-10-26T01:30:42.110967+0000","last_became_active":"2023-10-26T01:30:41.092917+0000","last_became_peered":"2023-10-26T01:30:41.092917+0000","last_unstale":"2023-10-26T01:30:42.110967+0000","last_undegraded":"2023-10-26T01:30:42.110967+0000","last_fullsized":"2023-10-26T01:30:42.110967+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:54:04.970133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.087143+0000","last_change":"2023-10-26T01:30:42.113216+0000","last_active":"2023-10-26T01:30:43.087143+0000","last_peered":"2023-10-26T01:30:43.087143+0000","last_clean":"2023-10-26T01:30:43.087143+0000","last_became_active":"2023-10-26T01:30:42.112815+0000","last_became_peered":"2023-10-26T01:30:42.112815+0000","last_unstale":"2023-10-26T01:30:43.087143+0000","last_undegraded":"2023-10-26T01:30:43.087143+0000","last_fullsized":"2023-10-26T01:30:43.087143+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:55:51.116465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.114880+0000","last_change":"2023-10-26T01:30:41.102365+0000","last_active":"2023-10-26T01:30:42.114880+0000","last_peered":"2023-10-26T01:30:42.114880+0000","last_clean":"2023-10-26T01:30:42.114880+0000","last_became_active":"2023-10-26T01:30:41.102135+0000","last_became_peered":"2023-10-26T01:30:41.102135+0000","last_unstale":"2023-10-26T01:30:42.114880+0000","last_undegraded":"2023-10-26T01:30:42.114880+0000","last_fullsized":"2023-10-26T01:30:42.114880+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:08:20.547052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.086427+0000","last_change":"2023-10-26T01:30:42.108605+0000","last_active":"2023-10-26T01:30:43.086427+0000","last_peered":"2023-10-26T01:30:43.086427+0000","last_clean":"2023-10-26T01:30:43.086427+0000","last_became_active":"2023-10-26T01:30:42.107907+0000","last_became_peered":"2023-10-26T01:30:42.107907+0000","last_unstale":"2023-10-26T01:30:43.086427+0000","last_undegraded":"2023-10-26T01:30:43.086427+0000","last_fullsized":"2023-10-26T01:30:43.086427+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T01:55:28.470913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.105745+0000","last_change":"2023-10-26T01:30:41.093456+0000","last_active":"2023-10-26T01:30:42.105745+0000","last_peered":"2023-10-26T01:30:42.105745+0000","last_clean":"2023-10-26T01:30:42.105745+0000","last_became_active":"2023-10-26T01:30:41.092811+0000","last_became_peered":"2023-10-26T01:30:41.092811+0000","last_unstale":"2023-10-26T01:30:42.105745+0000","last_undegraded":"2023-10-26T01:30:42.105745+0000","last_fullsized":"2023-10-26T01:30:42.105745+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:04:22.141930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.099118+0000","last_change":"2023-10-26T01:30:42.097121+0000","last_active":"2023-10-26T01:30:43.099118+0000","last_peered":"2023-10-26T01:30:43.099118+0000","last_clean":"2023-10-26T01:30:43.099118+0000","last_became_active":"2023-10-26T01:30:42.096784+0000","last_became_peered":"2023-10-26T01:30:42.096784+0000","last_unstale":"2023-10-26T01:30:43.099118+0000","last_undegraded":"2023-10-26T01:30:43.099118+0000","last_fullsized":"2023-10-26T01:30:43.099118+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:28:22.384005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.083700+0000","last_change":"2023-10-26T01:30:41.106632+0000","last_active":"2023-10-26T01:30:42.083700+0000","last_peered":"2023-10-26T01:30:42.083700+0000","last_clean":"2023-10-26T01:30:42.083700+0000","last_became_active":"2023-10-26T01:30:41.106360+0000","last_became_peered":"2023-10-26T01:30:41.106360+0000","last_unstale":"2023-10-26T01:30:42.083700+0000","last_undegraded":"2023-10-26T01:30:42.083700+0000","last_fullsized":"2023-10-26T01:30:42.083700+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:40:15.199455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.086951+0000","last_change":"2023-10-26T01:30:42.109475+0000","last_active":"2023-10-26T01:30:43.086951+0000","last_peered":"2023-10-26T01:30:43.086951+0000","last_clean":"2023-10-26T01:30:43.086951+0000","last_became_active":"2023-10-26T01:30:42.109232+0000","last_became_peered":"2023-10-26T01:30:42.109232+0000","last_unstale":"2023-10-26T01:30:43.086951+0000","last_undegraded":"2023-10-26T01:30:43.086951+0000","last_fullsized":"2023-10-26T01:30:43.086951+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:26:01.898201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":165,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.096979+0000","last_change":"2023-10-26T01:30:34.883484+0000","last_active":"2023-10-26T01:30:42.096979+0000","last_peered":"2023-10-26T01:30:42.096979+0000","last_clean":"2023-10-26T01:30:42.096979+0000","last_became_active":"2023-10-26T01:30:34.883071+0000","last_became_peered":"2023-10-26T01:30:34.883071+0000","last_unstale":"2023-10-26T01:30:42.096979+0000","last_undegraded":"2023-10-26T01:30:42.096979+0000","last_fullsized":"2023-10-26T01:30:42.096979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:32.890096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:32.890096+0000","last_clean_scrub_stamp":"2023-10-26T01:30:32.890096+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:22:57.937891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.090888+0000","last_change":"2023-10-26T01:30:41.105687+0000","last_active":"2023-10-26T01:30:42.090888+0000","last_peered":"2023-10-26T01:30:42.090888+0000","last_clean":"2023-10-26T01:30:42.090888+0000","last_became_active":"2023-10-26T01:30:41.105235+0000","last_became_peered":"2023-10-26T01:30:41.105235+0000","last_unstale":"2023-10-26T01:30:42.090888+0000","last_undegraded":"2023-10-26T01:30:42.090888+0000","last_fullsized":"2023-10-26T01:30:42.090888+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:13:32.821792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-10-26T01:30:41.094673+0000","last_change":"2023-10-26T01:30:41.094673+0000","last_active":"2023-10-26T01:30:41.060332+0000","last_peered":"2023-10-26T01:30:41.060332+0000","last_clean":"2023-10-26T01:30:41.060332+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:30:41.094673+0000","last_undegraded":"2023-10-26T01:30:41.094673+0000","last_fullsized":"2023-10-26T01:30:41.094673+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.096890+0000","last_change":"2023-10-26T01:30:41.094664+0000","last_active":"2023-10-26T01:30:42.096890+0000","last_peered":"2023-10-26T01:30:42.096890+0000","last_clean":"2023-10-26T01:30:42.096890+0000","last_became_active":"2023-10-26T01:30:41.092745+0000","last_became_peered":"2023-10-26T01:30:41.092745+0000","last_unstale":"2023-10-26T01:30:42.096890+0000","last_undegraded":"2023-10-26T01:30:42.096890+0000","last_fullsized":"2023-10-26T01:30:42.096890+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:45:22.045243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.140108+0000","last_change":"2023-10-26T01:30:42.107538+0000","last_active":"2023-10-26T01:30:43.140108+0000","last_peered":"2023-10-26T01:30:43.140108+0000","last_clean":"2023-10-26T01:30:43.140108+0000","last_became_active":"2023-10-26T01:30:42.107258+0000","last_became_peered":"2023-10-26T01:30:42.107258+0000","last_unstale":"2023-10-26T01:30:43.140108+0000","last_undegraded":"2023-10-26T01:30:43.140108+0000","last_fullsized":"2023-10-26T01:30:43.140108+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:49:12.753504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.083749+0000","last_change":"2023-10-26T01:30:41.099543+0000","last_active":"2023-10-26T01:30:42.083749+0000","last_peered":"2023-10-26T01:30:42.083749+0000","last_clean":"2023-10-26T01:30:42.083749+0000","last_became_active":"2023-10-26T01:30:41.099277+0000","last_became_peered":"2023-10-26T01:30:41.099277+0000","last_unstale":"2023-10-26T01:30:42.083749+0000","last_undegraded":"2023-10-26T01:30:42.083749+0000","last_fullsized":"2023-10-26T01:30:42.083749+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:04:22.141930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.140694+0000","last_change":"2023-10-26T01:30:42.085871+0000","last_active":"2023-10-26T01:30:43.140694+0000","last_peered":"2023-10-26T01:30:43.140694+0000","last_clean":"2023-10-26T01:30:43.140694+0000","last_became_active":"2023-10-26T01:30:42.085409+0000","last_became_peered":"2023-10-26T01:30:42.085409+0000","last_unstale":"2023-10-26T01:30:43.140694+0000","last_undegraded":"2023-10-26T01:30:43.140694+0000","last_fullsized":"2023-10-26T01:30:43.140694+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:17:59.186810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.088251+0000","last_change":"2023-10-26T01:30:41.107609+0000","last_active":"2023-10-26T01:30:42.088251+0000","last_peered":"2023-10-26T01:30:42.088251+0000","last_clean":"2023-10-26T01:30:42.088251+0000","last_became_active":"2023-10-26T01:30:41.107189+0000","last_became_peered":"2023-10-26T01:30:41.107189+0000","last_unstale":"2023-10-26T01:30:42.088251+0000","last_undegraded":"2023-10-26T01:30:42.088251+0000","last_fullsized":"2023-10-26T01:30:42.088251+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:33:57.752079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.088414+0000","last_change":"2023-10-26T01:30:42.105495+0000","last_active":"2023-10-26T01:30:43.088414+0000","last_peered":"2023-10-26T01:30:43.088414+0000","last_clean":"2023-10-26T01:30:43.088414+0000","last_became_active":"2023-10-26T01:30:42.105247+0000","last_became_peered":"2023-10-26T01:30:42.105247+0000","last_unstale":"2023-10-26T01:30:43.088414+0000","last_undegraded":"2023-10-26T01:30:43.088414+0000","last_fullsized":"2023-10-26T01:30:43.088414+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:24:17.679373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.088367+0000","last_change":"2023-10-26T01:30:41.096866+0000","last_active":"2023-10-26T01:30:42.088367+0000","last_peered":"2023-10-26T01:30:42.088367+0000","last_clean":"2023-10-26T01:30:42.088367+0000","last_became_active":"2023-10-26T01:30:41.096255+0000","last_became_peered":"2023-10-26T01:30:41.096255+0000","last_unstale":"2023-10-26T01:30:42.088367+0000","last_undegraded":"2023-10-26T01:30:42.088367+0000","last_fullsized":"2023-10-26T01:30:42.088367+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:29:17.626922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.099320+0000","last_change":"2023-10-26T01:30:42.102298+0000","last_active":"2023-10-26T01:30:43.099320+0000","last_peered":"2023-10-26T01:30:43.099320+0000","last_clean":"2023-10-26T01:30:43.099320+0000","last_became_active":"2023-10-26T01:30:42.097872+0000","last_became_peered":"2023-10-26T01:30:42.097872+0000","last_unstale":"2023-10-26T01:30:43.099320+0000","last_undegraded":"2023-10-26T01:30:43.099320+0000","last_fullsized":"2023-10-26T01:30:43.099320+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:07:06.450331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.096762+0000","last_change":"2023-10-26T01:30:41.096310+0000","last_active":"2023-10-26T01:30:42.096762+0000","last_peered":"2023-10-26T01:30:42.096762+0000","last_clean":"2023-10-26T01:30:42.096762+0000","last_became_active":"2023-10-26T01:30:41.095929+0000","last_became_peered":"2023-10-26T01:30:41.095929+0000","last_unstale":"2023-10-26T01:30:42.096762+0000","last_undegraded":"2023-10-26T01:30:42.096762+0000","last_fullsized":"2023-10-26T01:30:42.096762+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:31:39.970444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.140793+0000","last_change":"2023-10-26T01:30:42.089331+0000","last_active":"2023-10-26T01:30:43.140793+0000","last_peered":"2023-10-26T01:30:43.140793+0000","last_clean":"2023-10-26T01:30:43.140793+0000","last_became_active":"2023-10-26T01:30:42.088901+0000","last_became_peered":"2023-10-26T01:30:42.088901+0000","last_unstale":"2023-10-26T01:30:43.140793+0000","last_undegraded":"2023-10-26T01:30:43.140793+0000","last_fullsized":"2023-10-26T01:30:43.140793+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:53:57.632838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.082315+0000","last_change":"2023-10-26T01:30:41.102617+0000","last_active":"2023-10-26T01:30:42.082315+0000","last_peered":"2023-10-26T01:30:42.082315+0000","last_clean":"2023-10-26T01:30:42.082315+0000","last_became_active":"2023-10-26T01:30:41.102376+0000","last_became_peered":"2023-10-26T01:30:41.102376+0000","last_unstale":"2023-10-26T01:30:42.082315+0000","last_undegraded":"2023-10-26T01:30:42.082315+0000","last_fullsized":"2023-10-26T01:30:42.082315+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:24:16.663147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-10-26T01:30:41.095125+0000","last_change":"2023-10-26T01:30:41.095125+0000","last_active":"2023-10-26T01:30:41.060332+0000","last_peered":"2023-10-26T01:30:41.060332+0000","last_clean":"2023-10-26T01:30:41.060332+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:30:41.095125+0000","last_undegraded":"2023-10-26T01:30:41.095125+0000","last_fullsized":"2023-10-26T01:30:41.095125+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.112282+0000","last_change":"2023-10-26T01:30:41.102404+0000","last_active":"2023-10-26T01:30:42.112282+0000","last_peered":"2023-10-26T01:30:42.112282+0000","last_clean":"2023-10-26T01:30:42.112282+0000","last_became_active":"2023-10-26T01:30:41.102165+0000","last_became_peered":"2023-10-26T01:30:41.102165+0000","last_unstale":"2023-10-26T01:30:42.112282+0000","last_undegraded":"2023-10-26T01:30:42.112282+0000","last_fullsized":"2023-10-26T01:30:42.112282+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:08:20.547052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.139422+0000","last_change":"2023-10-26T01:30:42.103630+0000","last_active":"2023-10-26T01:30:43.139422+0000","last_peered":"2023-10-26T01:30:43.139422+0000","last_clean":"2023-10-26T01:30:43.139422+0000","last_became_active":"2023-10-26T01:30:42.099458+0000","last_became_peered":"2023-10-26T01:30:42.099458+0000","last_unstale":"2023-10-26T01:30:43.139422+0000","last_undegraded":"2023-10-26T01:30:43.139422+0000","last_fullsized":"2023-10-26T01:30:43.139422+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:05:31.590291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.097650+0000","last_change":"2023-10-26T01:30:41.097301+0000","last_active":"2023-10-26T01:30:42.097650+0000","last_peered":"2023-10-26T01:30:42.097650+0000","last_clean":"2023-10-26T01:30:42.097650+0000","last_became_active":"2023-10-26T01:30:41.096866+0000","last_became_peered":"2023-10-26T01:30:41.096866+0000","last_unstale":"2023-10-26T01:30:42.097650+0000","last_undegraded":"2023-10-26T01:30:42.097650+0000","last_fullsized":"2023-10-26T01:30:42.097650+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:05:30.574065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.089679+0000","last_change":"2023-10-26T01:30:42.087483+0000","last_active":"2023-10-26T01:30:43.089679+0000","last_peered":"2023-10-26T01:30:43.089679+0000","last_clean":"2023-10-26T01:30:43.089679+0000","last_became_active":"2023-10-26T01:30:42.087234+0000","last_became_peered":"2023-10-26T01:30:42.087234+0000","last_unstale":"2023-10-26T01:30:43.089679+0000","last_undegraded":"2023-10-26T01:30:43.089679+0000","last_fullsized":"2023-10-26T01:30:43.089679+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:46:06.943389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.088240+0000","last_change":"2023-10-26T01:30:41.097032+0000","last_active":"2023-10-26T01:30:42.088240+0000","last_peered":"2023-10-26T01:30:42.088240+0000","last_clean":"2023-10-26T01:30:42.088240+0000","last_became_active":"2023-10-26T01:30:41.096591+0000","last_became_peered":"2023-10-26T01:30:41.096591+0000","last_unstale":"2023-10-26T01:30:42.088240+0000","last_undegraded":"2023-10-26T01:30:42.088240+0000","last_fullsized":"2023-10-26T01:30:42.088240+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:33:57.752079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-10-26T01:30:41.094665+0000","last_change":"2023-10-26T01:30:41.094665+0000","last_active":"2023-10-26T01:30:41.060332+0000","last_peered":"2023-10-26T01:30:41.060332+0000","last_clean":"2023-10-26T01:30:41.060332+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:30:41.094665+0000","last_undegraded":"2023-10-26T01:30:41.094665+0000","last_fullsized":"2023-10-26T01:30:41.094665+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.084678+0000","last_change":"2023-10-26T01:30:41.101587+0000","last_active":"2023-10-26T01:30:42.084678+0000","last_peered":"2023-10-26T01:30:42.084678+0000","last_clean":"2023-10-26T01:30:42.084678+0000","last_became_active":"2023-10-26T01:30:41.101325+0000","last_became_peered":"2023-10-26T01:30:41.101325+0000","last_unstale":"2023-10-26T01:30:42.084678+0000","last_undegraded":"2023-10-26T01:30:42.084678+0000","last_fullsized":"2023-10-26T01:30:42.084678+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:49:42.452332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-10-26T01:30:41.090145+0000","last_change":"2023-10-26T01:30:41.090145+0000","last_active":"2023-10-26T01:30:41.060332+0000","last_peered":"2023-10-26T01:30:41.060332+0000","last_clean":"2023-10-26T01:30:41.060332+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:30:41.090145+0000","last_undegraded":"2023-10-26T01:30:41.090145+0000","last_fullsized":"2023-10-26T01:30:41.090145+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.114450+0000","last_change":"2023-10-26T01:30:41.095274+0000","last_active":"2023-10-26T01:30:42.114450+0000","last_peered":"2023-10-26T01:30:42.114450+0000","last_clean":"2023-10-26T01:30:42.114450+0000","last_became_active":"2023-10-26T01:30:41.092922+0000","last_became_peered":"2023-10-26T01:30:41.092922+0000","last_unstale":"2023-10-26T01:30:42.114450+0000","last_undegraded":"2023-10-26T01:30:42.114450+0000","last_fullsized":"2023-10-26T01:30:42.114450+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:33:12.599402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.098208+0000","last_change":"2023-10-26T01:30:42.105139+0000","last_active":"2023-10-26T01:30:43.098208+0000","last_peered":"2023-10-26T01:30:43.098208+0000","last_clean":"2023-10-26T01:30:43.098208+0000","last_became_active":"2023-10-26T01:30:42.104913+0000","last_became_peered":"2023-10-26T01:30:42.104913+0000","last_unstale":"2023-10-26T01:30:43.098208+0000","last_undegraded":"2023-10-26T01:30:43.098208+0000","last_fullsized":"2023-10-26T01:30:43.098208+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:17:59.186810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.099380+0000","last_change":"2023-10-26T01:30:42.106926+0000","last_active":"2023-10-26T01:30:43.099380+0000","last_peered":"2023-10-26T01:30:43.099380+0000","last_clean":"2023-10-26T01:30:43.099380+0000","last_became_active":"2023-10-26T01:30:42.106692+0000","last_became_peered":"2023-10-26T01:30:42.106692+0000","last_unstale":"2023-10-26T01:30:43.099380+0000","last_undegraded":"2023-10-26T01:30:43.099380+0000","last_fullsized":"2023-10-26T01:30:43.099380+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:20:47.431800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.110477+0000","last_change":"2023-10-26T01:30:41.105282+0000","last_active":"2023-10-26T01:30:42.110477+0000","last_peered":"2023-10-26T01:30:42.110477+0000","last_clean":"2023-10-26T01:30:42.110477+0000","last_became_active":"2023-10-26T01:30:41.104553+0000","last_became_peered":"2023-10-26T01:30:41.104553+0000","last_unstale":"2023-10-26T01:30:42.110477+0000","last_undegraded":"2023-10-26T01:30:42.110477+0000","last_fullsized":"2023-10-26T01:30:42.110477+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:29:17.626922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.100164+0000","last_change":"2023-10-26T01:30:42.098532+0000","last_active":"2023-10-26T01:30:43.100164+0000","last_peered":"2023-10-26T01:30:43.100164+0000","last_clean":"2023-10-26T01:30:43.100164+0000","last_became_active":"2023-10-26T01:30:42.098301+0000","last_became_peered":"2023-10-26T01:30:42.098301+0000","last_unstale":"2023-10-26T01:30:43.100164+0000","last_undegraded":"2023-10-26T01:30:43.100164+0000","last_fullsized":"2023-10-26T01:30:43.100164+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:55:51.116465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.113350+0000","last_change":"2023-10-26T01:30:41.101188+0000","last_active":"2023-10-26T01:30:42.113350+0000","last_peered":"2023-10-26T01:30:42.113350+0000","last_clean":"2023-10-26T01:30:42.113350+0000","last_became_active":"2023-10-26T01:30:41.100977+0000","last_became_peered":"2023-10-26T01:30:41.100977+0000","last_unstale":"2023-10-26T01:30:42.113350+0000","last_undegraded":"2023-10-26T01:30:42.113350+0000","last_fullsized":"2023-10-26T01:30:42.113350+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:13:32.821792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.139789+0000","last_change":"2023-10-26T01:30:42.105438+0000","last_active":"2023-10-26T01:30:43.139789+0000","last_peered":"2023-10-26T01:30:43.139789+0000","last_clean":"2023-10-26T01:30:43.139789+0000","last_became_active":"2023-10-26T01:30:42.105168+0000","last_became_peered":"2023-10-26T01:30:42.105168+0000","last_unstale":"2023-10-26T01:30:43.139789+0000","last_undegraded":"2023-10-26T01:30:43.139789+0000","last_fullsized":"2023-10-26T01:30:43.139789+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:53:57.632838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-26T01:30:41.089058+0000","last_change":"2023-10-26T01:30:41.088679+0000","last_active":"2023-10-26T01:30:41.089058+0000","last_peered":"2023-10-26T01:30:41.089058+0000","last_clean":"2023-10-26T01:30:41.089058+0000","last_became_active":"2023-10-26T01:30:41.086354+0000","last_became_peered":"2023-10-26T01:30:41.086354+0000","last_unstale":"2023-10-26T01:30:41.089058+0000","last_undegraded":"2023-10-26T01:30:41.089058+0000","last_fullsized":"2023-10-26T01:30:41.089058+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:49:35.491976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-10-26T01:30:41.096096+0000","last_change":"2023-10-26T01:30:41.096096+0000","last_active":"2023-10-26T01:30:41.060332+0000","last_peered":"2023-10-26T01:30:41.060332+0000","last_clean":"2023-10-26T01:30:41.060332+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:30:41.096096+0000","last_undegraded":"2023-10-26T01:30:41.096096+0000","last_fullsized":"2023-10-26T01:30:41.096096+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.113466+0000","last_change":"2023-10-26T01:30:41.097404+0000","last_active":"2023-10-26T01:30:42.113466+0000","last_peered":"2023-10-26T01:30:42.113466+0000","last_clean":"2023-10-26T01:30:42.113466+0000","last_became_active":"2023-10-26T01:30:41.097074+0000","last_became_peered":"2023-10-26T01:30:41.097074+0000","last_unstale":"2023-10-26T01:30:42.113466+0000","last_undegraded":"2023-10-26T01:30:42.113466+0000","last_fullsized":"2023-10-26T01:30:42.113466+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:31:39.970444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":36,"ondisk_log_size":36,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":14,"seq":60129542148,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":54,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195258,"statfs":{"total":200000000,"available":199944205,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85999999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1990000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.462}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94299999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.911}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.347}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.514}]}]},{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":650,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194662,"statfs":{"total":200000000,"available":199333905,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94399999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0569999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.333}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2050000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.014}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.101}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4970000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":74,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195238,"statfs":{"total":200000000,"available":199923857,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0089999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.393}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4319999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.256}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4790000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.131}]}]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":73,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195239,"statfs":{"total":200000000,"available":199924765,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1990000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.367}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4399999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7190000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.117}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.639}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5009999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":73,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195238,"statfs":{"total":200000000,"available":199924729,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.751}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.635}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5169999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.276}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4850000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4490000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7430000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":13,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":73,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195239,"statfs":{"total":200000000,"available":199924855,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1000000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.171}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68400000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2210000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7869999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.597}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9039999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":73,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195238,"statfs":{"total":200000000,"available":199924695,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79900000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95599999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6180000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4470000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3069999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.391}]}]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":650,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194661,"statfs":{"total":200000000,"available":199333849,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.03}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.161}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.204}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2290000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0129999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.839}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3049999999999999}]}]}],"pool_statfs":[]}} 2023-10-26T01:30:46.541 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-26T01:30:46.802 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:46.799+0000 7f87f53ae640 1 -- 172.21.15.169:0/882827207 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f87f00c7ea0 msgr2=0x7f87f006a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:46.803 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:46.799+0000 7f87f53ae640 1 --2- 172.21.15.169:0/882827207 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f87f00c7ea0 0x7f87f006a990 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f87d8009830 tx=0x7f87d802bd60 comp rx=0 tx=0).stop 2023-10-26T01:30:46.803 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:46.800+0000 7f87f53ae640 1 -- 172.21.15.169:0/882827207 shutdown_connections 2023-10-26T01:30:46.803 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:46.800+0000 7f87f53ae640 1 --2- 172.21.15.169:0/882827207 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f87f00c96f0 0x7f87f00c9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:46.803 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:46.800+0000 7f87f53ae640 1 --2- 172.21.15.169:0/882827207 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f87f00c7ea0 0x7f87f006a990 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:46.804 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:46.800+0000 7f87f53ae640 1 -- 172.21.15.169:0/882827207 >> 172.21.15.169:0/882827207 conn(0x7f87f00aedd0 msgr2=0x7f87f00af1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:46.804 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:46.800+0000 7f87f53ae640 1 -- 172.21.15.169:0/882827207 shutdown_connections 2023-10-26T01:30:46.804 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:46.800+0000 7f87f53ae640 1 -- 172.21.15.169:0/882827207 wait complete. 2023-10-26T01:30:46.805 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:46.801+0000 7f87f53ae640 1 Processor -- start 2023-10-26T01:30:46.805 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:46.802+0000 7f87f53ae640 1 -- start start 2023-10-26T01:30:46.805 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:46.803+0000 7f87f53ae640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f87f006cae0 0x7f87f01f2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:46.805 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:46.803+0000 7f87f53ae640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87f00c7ea0 0x7f87f01417b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:46.806 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:46.803+0000 7f87f53ae640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f87f00c96f0 0x7f87f00aaea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:46.806 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:46.803+0000 7f87f53ae640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87f00aaa80 con 0x7f87f00c96f0 2023-10-26T01:30:46.806 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:46.803+0000 7f87f53ae640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87f00afce0 con 0x7f87f00c7ea0 2023-10-26T01:30:46.806 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:46.803+0000 7f87f53ae640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87f0141520 con 0x7f87f006cae0 2023-10-26T01:30:46.808 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:46.803+0000 7f87f4bad640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f87f00c96f0 0x7f87f00aaea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:46.808 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:46.803+0000 7f87f4bad640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f87f00c96f0 0x7f87f00aaea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:33960/0 (socket says 172.21.15.169:33960) 2023-10-26T01:30:46.808 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:46.803+0000 7f87f4bad640 1 -- 172.21.15.169:0/3578943717 learned_addr learned my addr 172.21.15.169:0/3578943717 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:30:46.809 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:46.803+0000 7f87effff640 1 --2- 172.21.15.169:0/3578943717 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f87f006cae0 0x7f87f01f2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:46.809 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:46.803+0000 7f87e7fff640 1 --2- 172.21.15.169:0/3578943717 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87f00c7ea0 0x7f87f01417b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:46.809 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:46.803+0000 7f87f4bad640 1 -- 172.21.15.169:0/3578943717 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f87f006cae0 msgr2=0x7f87f01f2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:46.809 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:46.803+0000 7f87f4bad640 1 --2- 172.21.15.169:0/3578943717 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f87f006cae0 0x7f87f01f2bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:46.810 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:46.803+0000 7f87f4bad640 1 -- 172.21.15.169:0/3578943717 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87f00c7ea0 msgr2=0x7f87f01417b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:46.810 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:46.803+0000 7f87f4bad640 1 --2- 172.21.15.169:0/3578943717 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87f00c7ea0 0x7f87f01417b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:46.810 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:46.803+0000 7f87f4bad640 1 -- 172.21.15.169:0/3578943717 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87d8009490 con 0x7f87f00c96f0 2023-10-26T01:30:46.810 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:46.804+0000 7f87f4bad640 1 --2- 172.21.15.169:0/3578943717 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f87f00c96f0 0x7f87f00aaea0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f87e800cab0 tx=0x7f87e800cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:46.810 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:46.804+0000 7f87ed7fa640 1 -- 172.21.15.169:0/3578943717 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e8007d90 con 0x7f87f00c96f0 2023-10-26T01:30:46.811 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:46.804+0000 7f87ed7fa640 1 -- 172.21.15.169:0/3578943717 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f87e800f450 con 0x7f87f00c96f0 2023-10-26T01:30:46.811 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:46.804+0000 7f87ed7fa640 1 -- 172.21.15.169:0/3578943717 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e800f840 con 0x7f87f00c96f0 2023-10-26T01:30:46.811 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:46.804+0000 7f87f53ae640 1 -- 172.21.15.169:0/3578943717 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87f00afef0 con 0x7f87f00c96f0 2023-10-26T01:30:46.811 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:46.804+0000 7f87f53ae640 1 -- 172.21.15.169:0/3578943717 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87f00b01d0 con 0x7f87f00c96f0 2023-10-26T01:30:46.811 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:46.805+0000 7f87f53ae640 1 -- 172.21.15.169:0/3578943717 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87f00aabc0 con 0x7f87f00c96f0 2023-10-26T01:30:46.812 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:46.806+0000 7f87ed7fa640 1 -- 172.21.15.169:0/3578943717 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f87e800f9a0 con 0x7f87f00c96f0 2023-10-26T01:30:46.812 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:46.807+0000 7f87ed7fa640 1 --2- 172.21.15.169:0/3578943717 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f87d006ae70 0x7f87d006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:46.813 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:46.807+0000 7f87ed7fa640 1 -- 172.21.15.169:0/3578943717 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f87e8089ff0 con 0x7f87f00c96f0 2023-10-26T01:30:46.813 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:46.808+0000 7f87effff640 1 --2- 172.21.15.169:0/3578943717 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f87d006ae70 0x7f87d006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:46.813 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:46.809+0000 7f87effff640 1 --2- 172.21.15.169:0/3578943717 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f87d006ae70 0x7f87d006d330 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f87e0000f60 tx=0x7f87e0009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:46.814 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:46.811+0000 7f87ed7fa640 1 -- 172.21.15.169:0/3578943717 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f87f00aabc0 con 0x7f87f00c96f0 2023-10-26T01:30:46.999 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:46.996+0000 7f87f53ae640 1 -- 172.21.15.169:0/3578943717 --> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f87f00cb390 con 0x7f87d006ae70 2023-10-26T01:30:47.021 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:47.019+0000 7f87ed7fa640 1 -- 172.21.15.169:0/3578943717 <== mgr.4112 v2:172.21.15.170:6832/1187156918 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+259081 (secure 0 0 0) 0x7f87e0001cf0 con 0x7f87d006ae70 2023-10-26T01:30:47.023 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:30:47.026 INFO:teuthology.orchestra.run.smithi169.stderr:dumped all 2023-10-26T01:30:47.028 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:47.025+0000 7f87f53ae640 1 -- 172.21.15.169:0/3578943717 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f87d006ae70 msgr2=0x7f87d006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:47.028 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:47.025+0000 7f87f53ae640 1 --2- 172.21.15.169:0/3578943717 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f87d006ae70 0x7f87d006d330 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f87e0000f60 tx=0x7f87e0009040 comp rx=0 tx=0).stop 2023-10-26T01:30:47.029 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:47.025+0000 7f87f53ae640 1 -- 172.21.15.169:0/3578943717 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f87f00c96f0 msgr2=0x7f87f00aaea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:47.029 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:47.025+0000 7f87f53ae640 1 --2- 172.21.15.169:0/3578943717 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f87f00c96f0 0x7f87f00aaea0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f87e800cab0 tx=0x7f87e800cf70 comp rx=0 tx=0).stop 2023-10-26T01:30:47.029 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:47.026+0000 7f87f53ae640 1 -- 172.21.15.169:0/3578943717 shutdown_connections 2023-10-26T01:30:47.029 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:47.026+0000 7f87f53ae640 1 --2- 172.21.15.169:0/3578943717 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f87d006ae70 0x7f87d006d330 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:47.029 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:47.026+0000 7f87f53ae640 1 --2- 172.21.15.169:0/3578943717 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f87f00c96f0 0x7f87f00aaea0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:47.030 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:47.026+0000 7f87f53ae640 1 --2- 172.21.15.169:0/3578943717 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87f00c7ea0 0x7f87f01417b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:47.030 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:47.026+0000 7f87f53ae640 1 --2- 172.21.15.169:0/3578943717 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f87f006cae0 0x7f87f01f2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:47.030 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:47.026+0000 7f87f53ae640 1 -- 172.21.15.169:0/3578943717 >> 172.21.15.169:0/3578943717 conn(0x7f87f00aedd0 msgr2=0x7f87f0155750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:47.030 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:47.026+0000 7f87f53ae640 1 -- 172.21.15.169:0/3578943717 shutdown_connections 2023-10-26T01:30:47.030 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:47.026+0000 7f87f53ae640 1 -- 172.21.15.169:0/3578943717 wait complete. 2023-10-26T01:30:47.047 INFO:teuthology.orchestra.run.smithi169.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-10-26T01:30:46.128888+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":153,"ondisk_log_size":153,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":169,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":1723,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560776,"statfs":{"total":1600000000,"available":1598234860,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":58,"num_write_kb":276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001802"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.086425+0000","last_change":"2023-10-26T01:30:42.112552+0000","last_active":"2023-10-26T01:30:43.086425+0000","last_peered":"2023-10-26T01:30:43.086425+0000","last_clean":"2023-10-26T01:30:43.086425+0000","last_became_active":"2023-10-26T01:30:42.111151+0000","last_became_peered":"2023-10-26T01:30:42.111151+0000","last_unstale":"2023-10-26T01:30:43.086425+0000","last_undegraded":"2023-10-26T01:30:43.086425+0000","last_fullsized":"2023-10-26T01:30:43.086425+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:28:22.384005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.141033+0000","last_change":"2023-10-26T01:30:42.102872+0000","last_active":"2023-10-26T01:30:43.141033+0000","last_peered":"2023-10-26T01:30:43.141033+0000","last_clean":"2023-10-26T01:30:43.141033+0000","last_became_active":"2023-10-26T01:30:42.100796+0000","last_became_peered":"2023-10-26T01:30:42.100796+0000","last_unstale":"2023-10-26T01:30:43.141033+0000","last_undegraded":"2023-10-26T01:30:43.141033+0000","last_fullsized":"2023-10-26T01:30:43.141033+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:49:12.753504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.139076+0000","last_change":"2023-10-26T01:30:42.103921+0000","last_active":"2023-10-26T01:30:43.139076+0000","last_peered":"2023-10-26T01:30:43.139076+0000","last_clean":"2023-10-26T01:30:43.139076+0000","last_became_active":"2023-10-26T01:30:42.103384+0000","last_became_peered":"2023-10-26T01:30:42.103384+0000","last_unstale":"2023-10-26T01:30:43.139076+0000","last_undegraded":"2023-10-26T01:30:43.139076+0000","last_fullsized":"2023-10-26T01:30:43.139076+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:33:58.768305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.098597+0000","last_change":"2023-10-26T01:30:42.101845+0000","last_active":"2023-10-26T01:30:43.098597+0000","last_peered":"2023-10-26T01:30:43.098597+0000","last_clean":"2023-10-26T01:30:43.098597+0000","last_became_active":"2023-10-26T01:30:42.097125+0000","last_became_peered":"2023-10-26T01:30:42.097125+0000","last_unstale":"2023-10-26T01:30:43.098597+0000","last_undegraded":"2023-10-26T01:30:43.098597+0000","last_fullsized":"2023-10-26T01:30:43.098597+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:33:13.615628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.086419+0000","last_change":"2023-10-26T01:30:42.115244+0000","last_active":"2023-10-26T01:30:43.086419+0000","last_peered":"2023-10-26T01:30:43.086419+0000","last_clean":"2023-10-26T01:30:43.086419+0000","last_became_active":"2023-10-26T01:30:42.114972+0000","last_became_peered":"2023-10-26T01:30:42.114972+0000","last_unstale":"2023-10-26T01:30:43.086419+0000","last_undegraded":"2023-10-26T01:30:43.086419+0000","last_fullsized":"2023-10-26T01:30:43.086419+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:07:06.450331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.099993+0000","last_change":"2023-10-26T01:30:42.108711+0000","last_active":"2023-10-26T01:30:43.099993+0000","last_peered":"2023-10-26T01:30:43.099993+0000","last_clean":"2023-10-26T01:30:43.099993+0000","last_became_active":"2023-10-26T01:30:42.108215+0000","last_became_peered":"2023-10-26T01:30:42.108215+0000","last_unstale":"2023-10-26T01:30:43.099993+0000","last_undegraded":"2023-10-26T01:30:43.099993+0000","last_fullsized":"2023-10-26T01:30:43.099993+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:26:01.898201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.088319+0000","last_change":"2023-10-26T01:30:42.115268+0000","last_active":"2023-10-26T01:30:43.088319+0000","last_peered":"2023-10-26T01:30:43.088319+0000","last_clean":"2023-10-26T01:30:43.088319+0000","last_became_active":"2023-10-26T01:30:42.113208+0000","last_became_peered":"2023-10-26T01:30:42.113208+0000","last_unstale":"2023-10-26T01:30:43.088319+0000","last_undegraded":"2023-10-26T01:30:43.088319+0000","last_fullsized":"2023-10-26T01:30:43.088319+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:56:55.394232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-10-26T01:30:41.100533+0000","last_change":"2023-10-26T01:30:41.100533+0000","last_active":"2023-10-26T01:30:41.060332+0000","last_peered":"2023-10-26T01:30:41.060332+0000","last_clean":"2023-10-26T01:30:41.060332+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:30:41.100533+0000","last_undegraded":"2023-10-26T01:30:41.100533+0000","last_fullsized":"2023-10-26T01:30:41.100533+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.086093+0000","last_change":"2023-10-26T01:30:42.108451+0000","last_active":"2023-10-26T01:30:43.086093+0000","last_peered":"2023-10-26T01:30:43.086093+0000","last_clean":"2023-10-26T01:30:43.086093+0000","last_became_active":"2023-10-26T01:30:42.107650+0000","last_became_peered":"2023-10-26T01:30:42.107650+0000","last_unstale":"2023-10-26T01:30:43.086093+0000","last_undegraded":"2023-10-26T01:30:43.086093+0000","last_fullsized":"2023-10-26T01:30:43.086093+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:26:19.035685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.098895+0000","last_change":"2023-10-26T01:30:42.093838+0000","last_active":"2023-10-26T01:30:43.098895+0000","last_peered":"2023-10-26T01:30:43.098895+0000","last_clean":"2023-10-26T01:30:43.098895+0000","last_became_active":"2023-10-26T01:30:42.093584+0000","last_became_peered":"2023-10-26T01:30:42.093584+0000","last_unstale":"2023-10-26T01:30:43.098895+0000","last_undegraded":"2023-10-26T01:30:43.098895+0000","last_fullsized":"2023-10-26T01:30:43.098895+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:49:12.753504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.097658+0000","last_change":"2023-10-26T01:30:42.094026+0000","last_active":"2023-10-26T01:30:43.097658+0000","last_peered":"2023-10-26T01:30:43.097658+0000","last_clean":"2023-10-26T01:30:43.097658+0000","last_became_active":"2023-10-26T01:30:42.093654+0000","last_became_peered":"2023-10-26T01:30:42.093654+0000","last_unstale":"2023-10-26T01:30:43.097658+0000","last_undegraded":"2023-10-26T01:30:43.097658+0000","last_fullsized":"2023-10-26T01:30:43.097658+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:05:31.590291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.088414+0000","last_change":"2023-10-26T01:30:42.099764+0000","last_active":"2023-10-26T01:30:43.088414+0000","last_peered":"2023-10-26T01:30:43.088414+0000","last_clean":"2023-10-26T01:30:43.088414+0000","last_became_active":"2023-10-26T01:30:42.098108+0000","last_became_peered":"2023-10-26T01:30:42.098108+0000","last_unstale":"2023-10-26T01:30:43.088414+0000","last_undegraded":"2023-10-26T01:30:43.088414+0000","last_fullsized":"2023-10-26T01:30:43.088414+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:08:21.563278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-10-26T01:30:41.099897+0000","last_change":"2023-10-26T01:30:41.099897+0000","last_active":"2023-10-26T01:30:41.060332+0000","last_peered":"2023-10-26T01:30:41.060332+0000","last_clean":"2023-10-26T01:30:41.060332+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:30:41.099897+0000","last_undegraded":"2023-10-26T01:30:41.099897+0000","last_fullsized":"2023-10-26T01:30:41.099897+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.138887+0000","last_change":"2023-10-26T01:30:42.108715+0000","last_active":"2023-10-26T01:30:43.138887+0000","last_peered":"2023-10-26T01:30:43.138887+0000","last_clean":"2023-10-26T01:30:43.138887+0000","last_became_active":"2023-10-26T01:30:42.108469+0000","last_became_peered":"2023-10-26T01:30:42.108469+0000","last_unstale":"2023-10-26T01:30:43.138887+0000","last_undegraded":"2023-10-26T01:30:43.138887+0000","last_fullsized":"2023-10-26T01:30:43.138887+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:45:23.061469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.140916+0000","last_change":"2023-10-26T01:30:42.099157+0000","last_active":"2023-10-26T01:30:43.140916+0000","last_peered":"2023-10-26T01:30:43.140916+0000","last_clean":"2023-10-26T01:30:43.140916+0000","last_became_active":"2023-10-26T01:30:42.098929+0000","last_became_peered":"2023-10-26T01:30:42.098929+0000","last_unstale":"2023-10-26T01:30:43.140916+0000","last_undegraded":"2023-10-26T01:30:43.140916+0000","last_fullsized":"2023-10-26T01:30:43.140916+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:13:33.838018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.099835+0000","last_change":"2023-10-26T01:30:42.108578+0000","last_active":"2023-10-26T01:30:43.099835+0000","last_peered":"2023-10-26T01:30:43.099835+0000","last_clean":"2023-10-26T01:30:43.099835+0000","last_became_active":"2023-10-26T01:30:42.107937+0000","last_became_peered":"2023-10-26T01:30:42.107937+0000","last_unstale":"2023-10-26T01:30:43.099835+0000","last_undegraded":"2023-10-26T01:30:43.099835+0000","last_fullsized":"2023-10-26T01:30:43.099835+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:38:16.726919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.098559+0000","last_change":"2023-10-26T01:30:42.108373+0000","last_active":"2023-10-26T01:30:43.098559+0000","last_peered":"2023-10-26T01:30:43.098559+0000","last_clean":"2023-10-26T01:30:43.098559+0000","last_became_active":"2023-10-26T01:30:42.107773+0000","last_became_peered":"2023-10-26T01:30:42.107773+0000","last_unstale":"2023-10-26T01:30:43.098559+0000","last_undegraded":"2023-10-26T01:30:43.098559+0000","last_fullsized":"2023-10-26T01:30:43.098559+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:49:12.753504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.097381+0000","last_change":"2023-10-26T01:30:42.094993+0000","last_active":"2023-10-26T01:30:43.097381+0000","last_peered":"2023-10-26T01:30:43.097381+0000","last_clean":"2023-10-26T01:30:43.097381+0000","last_became_active":"2023-10-26T01:30:42.094689+0000","last_became_peered":"2023-10-26T01:30:42.094689+0000","last_unstale":"2023-10-26T01:30:43.097381+0000","last_undegraded":"2023-10-26T01:30:43.097381+0000","last_fullsized":"2023-10-26T01:30:43.097381+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:49:43.468558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.086773+0000","last_change":"2023-10-26T01:30:42.108746+0000","last_active":"2023-10-26T01:30:43.086773+0000","last_peered":"2023-10-26T01:30:43.086773+0000","last_clean":"2023-10-26T01:30:43.086773+0000","last_became_active":"2023-10-26T01:30:42.108188+0000","last_became_peered":"2023-10-26T01:30:42.108188+0000","last_unstale":"2023-10-26T01:30:43.086773+0000","last_undegraded":"2023-10-26T01:30:43.086773+0000","last_fullsized":"2023-10-26T01:30:43.086773+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:38:16.726919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.099061+0000","last_change":"2023-10-26T01:30:42.110331+0000","last_active":"2023-10-26T01:30:43.099061+0000","last_peered":"2023-10-26T01:30:43.099061+0000","last_clean":"2023-10-26T01:30:43.099061+0000","last_became_active":"2023-10-26T01:30:42.110079+0000","last_became_peered":"2023-10-26T01:30:42.110079+0000","last_unstale":"2023-10-26T01:30:43.099061+0000","last_undegraded":"2023-10-26T01:30:43.099061+0000","last_fullsized":"2023-10-26T01:30:43.099061+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:33:13.615628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.139607+0000","last_change":"2023-10-26T01:30:42.108608+0000","last_active":"2023-10-26T01:30:43.139607+0000","last_peered":"2023-10-26T01:30:43.139607+0000","last_clean":"2023-10-26T01:30:43.139607+0000","last_became_active":"2023-10-26T01:30:42.108197+0000","last_became_peered":"2023-10-26T01:30:42.108197+0000","last_unstale":"2023-10-26T01:30:43.139607+0000","last_undegraded":"2023-10-26T01:30:43.139607+0000","last_fullsized":"2023-10-26T01:30:43.139607+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:17:59.186810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.098944+0000","last_change":"2023-10-26T01:30:42.110289+0000","last_active":"2023-10-26T01:30:43.098944+0000","last_peered":"2023-10-26T01:30:43.098944+0000","last_clean":"2023-10-26T01:30:43.098944+0000","last_became_active":"2023-10-26T01:30:42.110061+0000","last_became_peered":"2023-10-26T01:30:42.110061+0000","last_unstale":"2023-10-26T01:30:43.098944+0000","last_undegraded":"2023-10-26T01:30:43.098944+0000","last_fullsized":"2023-10-26T01:30:43.098944+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:19:23.544531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-10-26T01:30:41.098174+0000","last_change":"2023-10-26T01:30:41.098174+0000","last_active":"2023-10-26T01:30:41.060332+0000","last_peered":"2023-10-26T01:30:41.060332+0000","last_clean":"2023-10-26T01:30:41.060332+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:30:41.098174+0000","last_undegraded":"2023-10-26T01:30:41.098174+0000","last_fullsized":"2023-10-26T01:30:41.098174+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.098984+0000","last_change":"2023-10-26T01:30:42.102147+0000","last_active":"2023-10-26T01:30:43.098984+0000","last_peered":"2023-10-26T01:30:43.098984+0000","last_clean":"2023-10-26T01:30:43.098984+0000","last_became_active":"2023-10-26T01:30:42.097509+0000","last_became_peered":"2023-10-26T01:30:42.097509+0000","last_unstale":"2023-10-26T01:30:43.098984+0000","last_undegraded":"2023-10-26T01:30:43.098984+0000","last_fullsized":"2023-10-26T01:30:43.098984+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:26:19.035685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-10-26T01:30:41.098206+0000","last_change":"2023-10-26T01:30:41.098206+0000","last_active":"2023-10-26T01:30:41.060332+0000","last_peered":"2023-10-26T01:30:41.060332+0000","last_clean":"2023-10-26T01:30:41.060332+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:30:41.098206+0000","last_undegraded":"2023-10-26T01:30:41.098206+0000","last_fullsized":"2023-10-26T01:30:41.098206+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.097862+0000","last_change":"2023-10-26T01:30:42.111865+0000","last_active":"2023-10-26T01:30:43.097862+0000","last_peered":"2023-10-26T01:30:43.097862+0000","last_clean":"2023-10-26T01:30:43.097862+0000","last_became_active":"2023-10-26T01:30:42.111636+0000","last_became_peered":"2023-10-26T01:30:42.111636+0000","last_unstale":"2023-10-26T01:30:43.097862+0000","last_undegraded":"2023-10-26T01:30:43.097862+0000","last_fullsized":"2023-10-26T01:30:43.097862+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:49:56.641022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.141348+0000","last_change":"2023-10-26T01:30:42.091332+0000","last_active":"2023-10-26T01:30:43.141348+0000","last_peered":"2023-10-26T01:30:43.141348+0000","last_clean":"2023-10-26T01:30:43.141348+0000","last_became_active":"2023-10-26T01:30:42.091105+0000","last_became_peered":"2023-10-26T01:30:42.091105+0000","last_unstale":"2023-10-26T01:30:43.141348+0000","last_undegraded":"2023-10-26T01:30:43.141348+0000","last_fullsized":"2023-10-26T01:30:43.141348+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:33:13.615628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-10-26T01:30:41.095233+0000","last_change":"2023-10-26T01:30:41.095233+0000","last_active":"2023-10-26T01:30:41.060332+0000","last_peered":"2023-10-26T01:30:41.060332+0000","last_clean":"2023-10-26T01:30:41.060332+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:30:41.095233+0000","last_undegraded":"2023-10-26T01:30:41.095233+0000","last_fullsized":"2023-10-26T01:30:41.095233+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.139200+0000","last_change":"2023-10-26T01:30:42.101708+0000","last_active":"2023-10-26T01:30:43.139200+0000","last_peered":"2023-10-26T01:30:43.139200+0000","last_clean":"2023-10-26T01:30:43.139200+0000","last_became_active":"2023-10-26T01:30:42.101434+0000","last_became_peered":"2023-10-26T01:30:42.101434+0000","last_unstale":"2023-10-26T01:30:43.139200+0000","last_undegraded":"2023-10-26T01:30:43.139200+0000","last_fullsized":"2023-10-26T01:30:43.139200+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:29:18.643148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.099207+0000","last_change":"2023-10-26T01:30:42.097495+0000","last_active":"2023-10-26T01:30:43.099207+0000","last_peered":"2023-10-26T01:30:43.099207+0000","last_clean":"2023-10-26T01:30:43.099207+0000","last_became_active":"2023-10-26T01:30:42.097214+0000","last_became_peered":"2023-10-26T01:30:42.097214+0000","last_unstale":"2023-10-26T01:30:43.099207+0000","last_undegraded":"2023-10-26T01:30:43.099207+0000","last_fullsized":"2023-10-26T01:30:43.099207+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T01:55:28.470913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.097481+0000","last_change":"2023-10-26T01:30:42.093157+0000","last_active":"2023-10-26T01:30:43.097481+0000","last_peered":"2023-10-26T01:30:43.097481+0000","last_clean":"2023-10-26T01:30:43.097481+0000","last_became_active":"2023-10-26T01:30:42.092613+0000","last_became_peered":"2023-10-26T01:30:42.092613+0000","last_unstale":"2023-10-26T01:30:43.097481+0000","last_undegraded":"2023-10-26T01:30:43.097481+0000","last_fullsized":"2023-10-26T01:30:43.097481+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:09:01.061004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.098226+0000","last_change":"2023-10-26T01:30:42.102704+0000","last_active":"2023-10-26T01:30:43.098226+0000","last_peered":"2023-10-26T01:30:43.098226+0000","last_clean":"2023-10-26T01:30:43.098226+0000","last_became_active":"2023-10-26T01:30:42.102350+0000","last_became_peered":"2023-10-26T01:30:42.102350+0000","last_unstale":"2023-10-26T01:30:43.098226+0000","last_undegraded":"2023-10-26T01:30:43.098226+0000","last_fullsized":"2023-10-26T01:30:43.098226+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:53:57.632838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.086493+0000","last_change":"2023-10-26T01:30:42.093818+0000","last_active":"2023-10-26T01:30:43.086493+0000","last_peered":"2023-10-26T01:30:43.086493+0000","last_clean":"2023-10-26T01:30:43.086493+0000","last_became_active":"2023-10-26T01:30:42.091310+0000","last_became_peered":"2023-10-26T01:30:42.091310+0000","last_unstale":"2023-10-26T01:30:43.086493+0000","last_undegraded":"2023-10-26T01:30:43.086493+0000","last_fullsized":"2023-10-26T01:30:43.086493+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:04:23.158155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-26T01:30:41.097736+0000","last_change":"2023-10-26T01:30:41.097666+0000","last_active":"2023-10-26T01:30:41.097736+0000","last_peered":"2023-10-26T01:30:41.097736+0000","last_clean":"2023-10-26T01:30:41.097736+0000","last_became_active":"2023-10-26T01:30:41.093986+0000","last_became_peered":"2023-10-26T01:30:41.093986+0000","last_unstale":"2023-10-26T01:30:41.097736+0000","last_undegraded":"2023-10-26T01:30:41.097736+0000","last_fullsized":"2023-10-26T01:30:41.097736+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:13:28.300112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.139516+0000","last_change":"2023-10-26T01:30:42.110839+0000","last_active":"2023-10-26T01:30:43.139516+0000","last_peered":"2023-10-26T01:30:43.139516+0000","last_clean":"2023-10-26T01:30:43.139516+0000","last_became_active":"2023-10-26T01:30:42.109385+0000","last_became_peered":"2023-10-26T01:30:42.109385+0000","last_unstale":"2023-10-26T01:30:43.139516+0000","last_undegraded":"2023-10-26T01:30:43.139516+0000","last_fullsized":"2023-10-26T01:30:43.139516+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:49:56.641022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.105272+0000","last_change":"2023-10-26T01:30:41.100248+0000","last_active":"2023-10-26T01:30:42.105272+0000","last_peered":"2023-10-26T01:30:42.105272+0000","last_clean":"2023-10-26T01:30:42.105272+0000","last_became_active":"2023-10-26T01:30:41.099957+0000","last_became_peered":"2023-10-26T01:30:41.099957+0000","last_unstale":"2023-10-26T01:30:42.105272+0000","last_undegraded":"2023-10-26T01:30:42.105272+0000","last_fullsized":"2023-10-26T01:30:42.105272+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:29:17.626922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-26T01:30:41.104334+0000","last_change":"2023-10-26T01:30:41.103843+0000","last_active":"2023-10-26T01:30:41.104334+0000","last_peered":"2023-10-26T01:30:41.104334+0000","last_clean":"2023-10-26T01:30:41.104334+0000","last_became_active":"2023-10-26T01:30:41.103356+0000","last_became_peered":"2023-10-26T01:30:41.103356+0000","last_unstale":"2023-10-26T01:30:41.104334+0000","last_undegraded":"2023-10-26T01:30:41.104334+0000","last_fullsized":"2023-10-26T01:30:41.104334+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:05:43.728770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-10-26T01:30:41.097717+0000","last_change":"2023-10-26T01:30:41.097717+0000","last_active":"2023-10-26T01:30:41.060332+0000","last_peered":"2023-10-26T01:30:41.060332+0000","last_clean":"2023-10-26T01:30:41.060332+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:30:41.097717+0000","last_undegraded":"2023-10-26T01:30:41.097717+0000","last_fullsized":"2023-10-26T01:30:41.097717+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.089693+0000","last_change":"2023-10-26T01:30:41.102989+0000","last_active":"2023-10-26T01:30:42.089693+0000","last_peered":"2023-10-26T01:30:42.089693+0000","last_clean":"2023-10-26T01:30:42.089693+0000","last_became_active":"2023-10-26T01:30:41.102681+0000","last_became_peered":"2023-10-26T01:30:41.102681+0000","last_unstale":"2023-10-26T01:30:42.089693+0000","last_undegraded":"2023-10-26T01:30:42.089693+0000","last_fullsized":"2023-10-26T01:30:42.089693+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:49:55.624796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.140341+0000","last_change":"2023-10-26T01:30:42.091359+0000","last_active":"2023-10-26T01:30:43.140341+0000","last_peered":"2023-10-26T01:30:43.140341+0000","last_clean":"2023-10-26T01:30:43.140341+0000","last_became_active":"2023-10-26T01:30:42.090949+0000","last_became_peered":"2023-10-26T01:30:42.090949+0000","last_unstale":"2023-10-26T01:30:43.140341+0000","last_undegraded":"2023-10-26T01:30:43.140341+0000","last_fullsized":"2023-10-26T01:30:43.140341+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:49:56.641022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.089836+0000","last_change":"2023-10-26T01:30:41.104058+0000","last_active":"2023-10-26T01:30:42.089836+0000","last_peered":"2023-10-26T01:30:42.089836+0000","last_clean":"2023-10-26T01:30:42.089836+0000","last_became_active":"2023-10-26T01:30:41.103821+0000","last_became_peered":"2023-10-26T01:30:41.103821+0000","last_unstale":"2023-10-26T01:30:42.089836+0000","last_undegraded":"2023-10-26T01:30:42.089836+0000","last_fullsized":"2023-10-26T01:30:42.089836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:46:05.927163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.098774+0000","last_change":"2023-10-26T01:30:42.108520+0000","last_active":"2023-10-26T01:30:43.098774+0000","last_peered":"2023-10-26T01:30:43.098774+0000","last_clean":"2023-10-26T01:30:43.098774+0000","last_became_active":"2023-10-26T01:30:42.108140+0000","last_became_peered":"2023-10-26T01:30:42.108140+0000","last_unstale":"2023-10-26T01:30:43.098774+0000","last_undegraded":"2023-10-26T01:30:43.098774+0000","last_fullsized":"2023-10-26T01:30:43.098774+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:31:08.727626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.088126+0000","last_change":"2023-10-26T01:30:41.106779+0000","last_active":"2023-10-26T01:30:42.088126+0000","last_peered":"2023-10-26T01:30:42.088126+0000","last_clean":"2023-10-26T01:30:42.088126+0000","last_became_active":"2023-10-26T01:30:41.104353+0000","last_became_peered":"2023-10-26T01:30:41.104353+0000","last_unstale":"2023-10-26T01:30:42.088126+0000","last_undegraded":"2023-10-26T01:30:42.088126+0000","last_fullsized":"2023-10-26T01:30:42.088126+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:45:22.045243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.139887+0000","last_change":"2023-10-26T01:30:42.105447+0000","last_active":"2023-10-26T01:30:43.139887+0000","last_peered":"2023-10-26T01:30:43.139887+0000","last_clean":"2023-10-26T01:30:43.139887+0000","last_became_active":"2023-10-26T01:30:42.105098+0000","last_became_peered":"2023-10-26T01:30:42.105098+0000","last_unstale":"2023-10-26T01:30:43.139887+0000","last_undegraded":"2023-10-26T01:30:43.139887+0000","last_fullsized":"2023-10-26T01:30:43.139887+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:13:33.838018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.110602+0000","last_change":"2023-10-26T01:30:41.103826+0000","last_active":"2023-10-26T01:30:42.110602+0000","last_peered":"2023-10-26T01:30:42.110602+0000","last_clean":"2023-10-26T01:30:42.110602+0000","last_became_active":"2023-10-26T01:30:41.103552+0000","last_became_peered":"2023-10-26T01:30:41.103552+0000","last_unstale":"2023-10-26T01:30:42.110602+0000","last_undegraded":"2023-10-26T01:30:42.110602+0000","last_fullsized":"2023-10-26T01:30:42.110602+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:54:04.970133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.086144+0000","last_change":"2023-10-26T01:30:42.114167+0000","last_active":"2023-10-26T01:30:43.086144+0000","last_peered":"2023-10-26T01:30:43.086144+0000","last_clean":"2023-10-26T01:30:43.086144+0000","last_became_active":"2023-10-26T01:30:42.113241+0000","last_became_peered":"2023-10-26T01:30:42.113241+0000","last_unstale":"2023-10-26T01:30:43.086144+0000","last_undegraded":"2023-10-26T01:30:43.086144+0000","last_fullsized":"2023-10-26T01:30:43.086144+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:40:16.215681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.116049+0000","last_change":"2023-10-26T01:30:41.106514+0000","last_active":"2023-10-26T01:30:42.116049+0000","last_peered":"2023-10-26T01:30:42.116049+0000","last_clean":"2023-10-26T01:30:42.116049+0000","last_became_active":"2023-10-26T01:30:41.106147+0000","last_became_peered":"2023-10-26T01:30:41.106147+0000","last_unstale":"2023-10-26T01:30:42.116049+0000","last_undegraded":"2023-10-26T01:30:42.116049+0000","last_fullsized":"2023-10-26T01:30:42.116049+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:45:22.045243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.089920+0000","last_change":"2023-10-26T01:30:42.112813+0000","last_active":"2023-10-26T01:30:43.089920+0000","last_peered":"2023-10-26T01:30:43.089920+0000","last_clean":"2023-10-26T01:30:43.089920+0000","last_became_active":"2023-10-26T01:30:42.112571+0000","last_became_peered":"2023-10-26T01:30:42.112571+0000","last_unstale":"2023-10-26T01:30:43.089920+0000","last_undegraded":"2023-10-26T01:30:43.089920+0000","last_fullsized":"2023-10-26T01:30:43.089920+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:19:16.042472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.108474+0000","last_change":"2023-10-26T01:30:41.098865+0000","last_active":"2023-10-26T01:30:42.108474+0000","last_peered":"2023-10-26T01:30:42.108474+0000","last_clean":"2023-10-26T01:30:42.108474+0000","last_became_active":"2023-10-26T01:30:41.098506+0000","last_became_peered":"2023-10-26T01:30:41.098506+0000","last_unstale":"2023-10-26T01:30:42.108474+0000","last_undegraded":"2023-10-26T01:30:42.108474+0000","last_fullsized":"2023-10-26T01:30:42.108474+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:09:00.044778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.140188+0000","last_change":"2023-10-26T01:30:42.108443+0000","last_active":"2023-10-26T01:30:43.140188+0000","last_peered":"2023-10-26T01:30:43.140188+0000","last_clean":"2023-10-26T01:30:43.140188+0000","last_became_active":"2023-10-26T01:30:42.107837+0000","last_became_peered":"2023-10-26T01:30:42.107837+0000","last_unstale":"2023-10-26T01:30:43.140188+0000","last_undegraded":"2023-10-26T01:30:43.140188+0000","last_fullsized":"2023-10-26T01:30:43.140188+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:33:13.615628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.090761+0000","last_change":"2023-10-26T01:30:41.106795+0000","last_active":"2023-10-26T01:30:42.090761+0000","last_peered":"2023-10-26T01:30:42.090761+0000","last_clean":"2023-10-26T01:30:42.090761+0000","last_became_active":"2023-10-26T01:30:41.106222+0000","last_became_peered":"2023-10-26T01:30:41.106222+0000","last_unstale":"2023-10-26T01:30:42.090761+0000","last_undegraded":"2023-10-26T01:30:42.090761+0000","last_fullsized":"2023-10-26T01:30:42.090761+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:53:56.616612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.086822+0000","last_change":"2023-10-26T01:30:42.113053+0000","last_active":"2023-10-26T01:30:43.086822+0000","last_peered":"2023-10-26T01:30:43.086822+0000","last_clean":"2023-10-26T01:30:43.086822+0000","last_became_active":"2023-10-26T01:30:42.112492+0000","last_became_peered":"2023-10-26T01:30:42.112492+0000","last_unstale":"2023-10-26T01:30:43.086822+0000","last_undegraded":"2023-10-26T01:30:43.086822+0000","last_fullsized":"2023-10-26T01:30:43.086822+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:20:47.431800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.089525+0000","last_change":"2023-10-26T01:30:41.100835+0000","last_active":"2023-10-26T01:30:42.089525+0000","last_peered":"2023-10-26T01:30:42.089525+0000","last_clean":"2023-10-26T01:30:42.089525+0000","last_became_active":"2023-10-26T01:30:41.099708+0000","last_became_peered":"2023-10-26T01:30:41.099708+0000","last_unstale":"2023-10-26T01:30:42.089525+0000","last_undegraded":"2023-10-26T01:30:42.089525+0000","last_fullsized":"2023-10-26T01:30:42.089525+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:04:22.141930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.098337+0000","last_change":"2023-10-26T01:30:42.100857+0000","last_active":"2023-10-26T01:30:43.098337+0000","last_peered":"2023-10-26T01:30:43.098337+0000","last_clean":"2023-10-26T01:30:43.098337+0000","last_became_active":"2023-10-26T01:30:42.100545+0000","last_became_peered":"2023-10-26T01:30:42.100545+0000","last_unstale":"2023-10-26T01:30:43.098337+0000","last_undegraded":"2023-10-26T01:30:43.098337+0000","last_fullsized":"2023-10-26T01:30:43.098337+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:13:33.838018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.114399+0000","last_change":"2023-10-26T01:30:41.095998+0000","last_active":"2023-10-26T01:30:42.114399+0000","last_peered":"2023-10-26T01:30:42.114399+0000","last_clean":"2023-10-26T01:30:42.114399+0000","last_became_active":"2023-10-26T01:30:41.095494+0000","last_became_peered":"2023-10-26T01:30:41.095494+0000","last_unstale":"2023-10-26T01:30:42.114399+0000","last_undegraded":"2023-10-26T01:30:42.114399+0000","last_fullsized":"2023-10-26T01:30:42.114399+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:54:04.970133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.089073+0000","last_change":"2023-10-26T01:30:42.115510+0000","last_active":"2023-10-26T01:30:43.089073+0000","last_peered":"2023-10-26T01:30:43.089073+0000","last_clean":"2023-10-26T01:30:43.089073+0000","last_became_active":"2023-10-26T01:30:42.113552+0000","last_became_peered":"2023-10-26T01:30:42.113552+0000","last_unstale":"2023-10-26T01:30:43.089073+0000","last_undegraded":"2023-10-26T01:30:43.089073+0000","last_fullsized":"2023-10-26T01:30:43.089073+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:32:08.387966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.110967+0000","last_change":"2023-10-26T01:30:41.093200+0000","last_active":"2023-10-26T01:30:42.110967+0000","last_peered":"2023-10-26T01:30:42.110967+0000","last_clean":"2023-10-26T01:30:42.110967+0000","last_became_active":"2023-10-26T01:30:41.092917+0000","last_became_peered":"2023-10-26T01:30:41.092917+0000","last_unstale":"2023-10-26T01:30:42.110967+0000","last_undegraded":"2023-10-26T01:30:42.110967+0000","last_fullsized":"2023-10-26T01:30:42.110967+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:54:04.970133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.087143+0000","last_change":"2023-10-26T01:30:42.113216+0000","last_active":"2023-10-26T01:30:43.087143+0000","last_peered":"2023-10-26T01:30:43.087143+0000","last_clean":"2023-10-26T01:30:43.087143+0000","last_became_active":"2023-10-26T01:30:42.112815+0000","last_became_peered":"2023-10-26T01:30:42.112815+0000","last_unstale":"2023-10-26T01:30:43.087143+0000","last_undegraded":"2023-10-26T01:30:43.087143+0000","last_fullsized":"2023-10-26T01:30:43.087143+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:55:51.116465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.114880+0000","last_change":"2023-10-26T01:30:41.102365+0000","last_active":"2023-10-26T01:30:42.114880+0000","last_peered":"2023-10-26T01:30:42.114880+0000","last_clean":"2023-10-26T01:30:42.114880+0000","last_became_active":"2023-10-26T01:30:41.102135+0000","last_became_peered":"2023-10-26T01:30:41.102135+0000","last_unstale":"2023-10-26T01:30:42.114880+0000","last_undegraded":"2023-10-26T01:30:42.114880+0000","last_fullsized":"2023-10-26T01:30:42.114880+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:08:20.547052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.086427+0000","last_change":"2023-10-26T01:30:42.108605+0000","last_active":"2023-10-26T01:30:43.086427+0000","last_peered":"2023-10-26T01:30:43.086427+0000","last_clean":"2023-10-26T01:30:43.086427+0000","last_became_active":"2023-10-26T01:30:42.107907+0000","last_became_peered":"2023-10-26T01:30:42.107907+0000","last_unstale":"2023-10-26T01:30:43.086427+0000","last_undegraded":"2023-10-26T01:30:43.086427+0000","last_fullsized":"2023-10-26T01:30:43.086427+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T01:55:28.470913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.105745+0000","last_change":"2023-10-26T01:30:41.093456+0000","last_active":"2023-10-26T01:30:42.105745+0000","last_peered":"2023-10-26T01:30:42.105745+0000","last_clean":"2023-10-26T01:30:42.105745+0000","last_became_active":"2023-10-26T01:30:41.092811+0000","last_became_peered":"2023-10-26T01:30:41.092811+0000","last_unstale":"2023-10-26T01:30:42.105745+0000","last_undegraded":"2023-10-26T01:30:42.105745+0000","last_fullsized":"2023-10-26T01:30:42.105745+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:04:22.141930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.099118+0000","last_change":"2023-10-26T01:30:42.097121+0000","last_active":"2023-10-26T01:30:43.099118+0000","last_peered":"2023-10-26T01:30:43.099118+0000","last_clean":"2023-10-26T01:30:43.099118+0000","last_became_active":"2023-10-26T01:30:42.096784+0000","last_became_peered":"2023-10-26T01:30:42.096784+0000","last_unstale":"2023-10-26T01:30:43.099118+0000","last_undegraded":"2023-10-26T01:30:43.099118+0000","last_fullsized":"2023-10-26T01:30:43.099118+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:28:22.384005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.083700+0000","last_change":"2023-10-26T01:30:41.106632+0000","last_active":"2023-10-26T01:30:42.083700+0000","last_peered":"2023-10-26T01:30:42.083700+0000","last_clean":"2023-10-26T01:30:42.083700+0000","last_became_active":"2023-10-26T01:30:41.106360+0000","last_became_peered":"2023-10-26T01:30:41.106360+0000","last_unstale":"2023-10-26T01:30:42.083700+0000","last_undegraded":"2023-10-26T01:30:42.083700+0000","last_fullsized":"2023-10-26T01:30:42.083700+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:40:15.199455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.086951+0000","last_change":"2023-10-26T01:30:42.109475+0000","last_active":"2023-10-26T01:30:43.086951+0000","last_peered":"2023-10-26T01:30:43.086951+0000","last_clean":"2023-10-26T01:30:43.086951+0000","last_became_active":"2023-10-26T01:30:42.109232+0000","last_became_peered":"2023-10-26T01:30:42.109232+0000","last_unstale":"2023-10-26T01:30:43.086951+0000","last_undegraded":"2023-10-26T01:30:43.086951+0000","last_fullsized":"2023-10-26T01:30:43.086951+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:26:01.898201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":165,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.096979+0000","last_change":"2023-10-26T01:30:34.883484+0000","last_active":"2023-10-26T01:30:42.096979+0000","last_peered":"2023-10-26T01:30:42.096979+0000","last_clean":"2023-10-26T01:30:42.096979+0000","last_became_active":"2023-10-26T01:30:34.883071+0000","last_became_peered":"2023-10-26T01:30:34.883071+0000","last_unstale":"2023-10-26T01:30:42.096979+0000","last_undegraded":"2023-10-26T01:30:42.096979+0000","last_fullsized":"2023-10-26T01:30:42.096979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:32.890096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:32.890096+0000","last_clean_scrub_stamp":"2023-10-26T01:30:32.890096+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:22:57.937891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.090888+0000","last_change":"2023-10-26T01:30:41.105687+0000","last_active":"2023-10-26T01:30:42.090888+0000","last_peered":"2023-10-26T01:30:42.090888+0000","last_clean":"2023-10-26T01:30:42.090888+0000","last_became_active":"2023-10-26T01:30:41.105235+0000","last_became_peered":"2023-10-26T01:30:41.105235+0000","last_unstale":"2023-10-26T01:30:42.090888+0000","last_undegraded":"2023-10-26T01:30:42.090888+0000","last_fullsized":"2023-10-26T01:30:42.090888+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:13:32.821792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-10-26T01:30:41.094673+0000","last_change":"2023-10-26T01:30:41.094673+0000","last_active":"2023-10-26T01:30:41.060332+0000","last_peered":"2023-10-26T01:30:41.060332+0000","last_clean":"2023-10-26T01:30:41.060332+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:30:41.094673+0000","last_undegraded":"2023-10-26T01:30:41.094673+0000","last_fullsized":"2023-10-26T01:30:41.094673+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.096890+0000","last_change":"2023-10-26T01:30:41.094664+0000","last_active":"2023-10-26T01:30:42.096890+0000","last_peered":"2023-10-26T01:30:42.096890+0000","last_clean":"2023-10-26T01:30:42.096890+0000","last_became_active":"2023-10-26T01:30:41.092745+0000","last_became_peered":"2023-10-26T01:30:41.092745+0000","last_unstale":"2023-10-26T01:30:42.096890+0000","last_undegraded":"2023-10-26T01:30:42.096890+0000","last_fullsized":"2023-10-26T01:30:42.096890+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:45:22.045243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.140108+0000","last_change":"2023-10-26T01:30:42.107538+0000","last_active":"2023-10-26T01:30:43.140108+0000","last_peered":"2023-10-26T01:30:43.140108+0000","last_clean":"2023-10-26T01:30:43.140108+0000","last_became_active":"2023-10-26T01:30:42.107258+0000","last_became_peered":"2023-10-26T01:30:42.107258+0000","last_unstale":"2023-10-26T01:30:43.140108+0000","last_undegraded":"2023-10-26T01:30:43.140108+0000","last_fullsized":"2023-10-26T01:30:43.140108+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:49:12.753504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.083749+0000","last_change":"2023-10-26T01:30:41.099543+0000","last_active":"2023-10-26T01:30:42.083749+0000","last_peered":"2023-10-26T01:30:42.083749+0000","last_clean":"2023-10-26T01:30:42.083749+0000","last_became_active":"2023-10-26T01:30:41.099277+0000","last_became_peered":"2023-10-26T01:30:41.099277+0000","last_unstale":"2023-10-26T01:30:42.083749+0000","last_undegraded":"2023-10-26T01:30:42.083749+0000","last_fullsized":"2023-10-26T01:30:42.083749+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:04:22.141930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.140694+0000","last_change":"2023-10-26T01:30:42.085871+0000","last_active":"2023-10-26T01:30:43.140694+0000","last_peered":"2023-10-26T01:30:43.140694+0000","last_clean":"2023-10-26T01:30:43.140694+0000","last_became_active":"2023-10-26T01:30:42.085409+0000","last_became_peered":"2023-10-26T01:30:42.085409+0000","last_unstale":"2023-10-26T01:30:43.140694+0000","last_undegraded":"2023-10-26T01:30:43.140694+0000","last_fullsized":"2023-10-26T01:30:43.140694+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:17:59.186810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.088251+0000","last_change":"2023-10-26T01:30:41.107609+0000","last_active":"2023-10-26T01:30:42.088251+0000","last_peered":"2023-10-26T01:30:42.088251+0000","last_clean":"2023-10-26T01:30:42.088251+0000","last_became_active":"2023-10-26T01:30:41.107189+0000","last_became_peered":"2023-10-26T01:30:41.107189+0000","last_unstale":"2023-10-26T01:30:42.088251+0000","last_undegraded":"2023-10-26T01:30:42.088251+0000","last_fullsized":"2023-10-26T01:30:42.088251+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:33:57.752079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.088414+0000","last_change":"2023-10-26T01:30:42.105495+0000","last_active":"2023-10-26T01:30:43.088414+0000","last_peered":"2023-10-26T01:30:43.088414+0000","last_clean":"2023-10-26T01:30:43.088414+0000","last_became_active":"2023-10-26T01:30:42.105247+0000","last_became_peered":"2023-10-26T01:30:42.105247+0000","last_unstale":"2023-10-26T01:30:43.088414+0000","last_undegraded":"2023-10-26T01:30:43.088414+0000","last_fullsized":"2023-10-26T01:30:43.088414+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:24:17.679373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.088367+0000","last_change":"2023-10-26T01:30:41.096866+0000","last_active":"2023-10-26T01:30:42.088367+0000","last_peered":"2023-10-26T01:30:42.088367+0000","last_clean":"2023-10-26T01:30:42.088367+0000","last_became_active":"2023-10-26T01:30:41.096255+0000","last_became_peered":"2023-10-26T01:30:41.096255+0000","last_unstale":"2023-10-26T01:30:42.088367+0000","last_undegraded":"2023-10-26T01:30:42.088367+0000","last_fullsized":"2023-10-26T01:30:42.088367+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:29:17.626922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.099320+0000","last_change":"2023-10-26T01:30:42.102298+0000","last_active":"2023-10-26T01:30:43.099320+0000","last_peered":"2023-10-26T01:30:43.099320+0000","last_clean":"2023-10-26T01:30:43.099320+0000","last_became_active":"2023-10-26T01:30:42.097872+0000","last_became_peered":"2023-10-26T01:30:42.097872+0000","last_unstale":"2023-10-26T01:30:43.099320+0000","last_undegraded":"2023-10-26T01:30:43.099320+0000","last_fullsized":"2023-10-26T01:30:43.099320+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:07:06.450331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.096762+0000","last_change":"2023-10-26T01:30:41.096310+0000","last_active":"2023-10-26T01:30:42.096762+0000","last_peered":"2023-10-26T01:30:42.096762+0000","last_clean":"2023-10-26T01:30:42.096762+0000","last_became_active":"2023-10-26T01:30:41.095929+0000","last_became_peered":"2023-10-26T01:30:41.095929+0000","last_unstale":"2023-10-26T01:30:42.096762+0000","last_undegraded":"2023-10-26T01:30:42.096762+0000","last_fullsized":"2023-10-26T01:30:42.096762+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:31:39.970444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.140793+0000","last_change":"2023-10-26T01:30:42.089331+0000","last_active":"2023-10-26T01:30:43.140793+0000","last_peered":"2023-10-26T01:30:43.140793+0000","last_clean":"2023-10-26T01:30:43.140793+0000","last_became_active":"2023-10-26T01:30:42.088901+0000","last_became_peered":"2023-10-26T01:30:42.088901+0000","last_unstale":"2023-10-26T01:30:43.140793+0000","last_undegraded":"2023-10-26T01:30:43.140793+0000","last_fullsized":"2023-10-26T01:30:43.140793+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:53:57.632838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.082315+0000","last_change":"2023-10-26T01:30:41.102617+0000","last_active":"2023-10-26T01:30:42.082315+0000","last_peered":"2023-10-26T01:30:42.082315+0000","last_clean":"2023-10-26T01:30:42.082315+0000","last_became_active":"2023-10-26T01:30:41.102376+0000","last_became_peered":"2023-10-26T01:30:41.102376+0000","last_unstale":"2023-10-26T01:30:42.082315+0000","last_undegraded":"2023-10-26T01:30:42.082315+0000","last_fullsized":"2023-10-26T01:30:42.082315+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:24:16.663147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-10-26T01:30:41.095125+0000","last_change":"2023-10-26T01:30:41.095125+0000","last_active":"2023-10-26T01:30:41.060332+0000","last_peered":"2023-10-26T01:30:41.060332+0000","last_clean":"2023-10-26T01:30:41.060332+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:30:41.095125+0000","last_undegraded":"2023-10-26T01:30:41.095125+0000","last_fullsized":"2023-10-26T01:30:41.095125+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.112282+0000","last_change":"2023-10-26T01:30:41.102404+0000","last_active":"2023-10-26T01:30:42.112282+0000","last_peered":"2023-10-26T01:30:42.112282+0000","last_clean":"2023-10-26T01:30:42.112282+0000","last_became_active":"2023-10-26T01:30:41.102165+0000","last_became_peered":"2023-10-26T01:30:41.102165+0000","last_unstale":"2023-10-26T01:30:42.112282+0000","last_undegraded":"2023-10-26T01:30:42.112282+0000","last_fullsized":"2023-10-26T01:30:42.112282+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:08:20.547052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.139422+0000","last_change":"2023-10-26T01:30:42.103630+0000","last_active":"2023-10-26T01:30:43.139422+0000","last_peered":"2023-10-26T01:30:43.139422+0000","last_clean":"2023-10-26T01:30:43.139422+0000","last_became_active":"2023-10-26T01:30:42.099458+0000","last_became_peered":"2023-10-26T01:30:42.099458+0000","last_unstale":"2023-10-26T01:30:43.139422+0000","last_undegraded":"2023-10-26T01:30:43.139422+0000","last_fullsized":"2023-10-26T01:30:43.139422+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:05:31.590291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.097650+0000","last_change":"2023-10-26T01:30:41.097301+0000","last_active":"2023-10-26T01:30:42.097650+0000","last_peered":"2023-10-26T01:30:42.097650+0000","last_clean":"2023-10-26T01:30:42.097650+0000","last_became_active":"2023-10-26T01:30:41.096866+0000","last_became_peered":"2023-10-26T01:30:41.096866+0000","last_unstale":"2023-10-26T01:30:42.097650+0000","last_undegraded":"2023-10-26T01:30:42.097650+0000","last_fullsized":"2023-10-26T01:30:42.097650+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:05:30.574065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.089679+0000","last_change":"2023-10-26T01:30:42.087483+0000","last_active":"2023-10-26T01:30:43.089679+0000","last_peered":"2023-10-26T01:30:43.089679+0000","last_clean":"2023-10-26T01:30:43.089679+0000","last_became_active":"2023-10-26T01:30:42.087234+0000","last_became_peered":"2023-10-26T01:30:42.087234+0000","last_unstale":"2023-10-26T01:30:43.089679+0000","last_undegraded":"2023-10-26T01:30:43.089679+0000","last_fullsized":"2023-10-26T01:30:43.089679+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:46:06.943389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.088240+0000","last_change":"2023-10-26T01:30:41.097032+0000","last_active":"2023-10-26T01:30:42.088240+0000","last_peered":"2023-10-26T01:30:42.088240+0000","last_clean":"2023-10-26T01:30:42.088240+0000","last_became_active":"2023-10-26T01:30:41.096591+0000","last_became_peered":"2023-10-26T01:30:41.096591+0000","last_unstale":"2023-10-26T01:30:42.088240+0000","last_undegraded":"2023-10-26T01:30:42.088240+0000","last_fullsized":"2023-10-26T01:30:42.088240+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:33:57.752079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-10-26T01:30:41.094665+0000","last_change":"2023-10-26T01:30:41.094665+0000","last_active":"2023-10-26T01:30:41.060332+0000","last_peered":"2023-10-26T01:30:41.060332+0000","last_clean":"2023-10-26T01:30:41.060332+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:30:41.094665+0000","last_undegraded":"2023-10-26T01:30:41.094665+0000","last_fullsized":"2023-10-26T01:30:41.094665+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.084678+0000","last_change":"2023-10-26T01:30:41.101587+0000","last_active":"2023-10-26T01:30:42.084678+0000","last_peered":"2023-10-26T01:30:42.084678+0000","last_clean":"2023-10-26T01:30:42.084678+0000","last_became_active":"2023-10-26T01:30:41.101325+0000","last_became_peered":"2023-10-26T01:30:41.101325+0000","last_unstale":"2023-10-26T01:30:42.084678+0000","last_undegraded":"2023-10-26T01:30:42.084678+0000","last_fullsized":"2023-10-26T01:30:42.084678+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:49:42.452332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-10-26T01:30:41.090145+0000","last_change":"2023-10-26T01:30:41.090145+0000","last_active":"2023-10-26T01:30:41.060332+0000","last_peered":"2023-10-26T01:30:41.060332+0000","last_clean":"2023-10-26T01:30:41.060332+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:30:41.090145+0000","last_undegraded":"2023-10-26T01:30:41.090145+0000","last_fullsized":"2023-10-26T01:30:41.090145+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.114450+0000","last_change":"2023-10-26T01:30:41.095274+0000","last_active":"2023-10-26T01:30:42.114450+0000","last_peered":"2023-10-26T01:30:42.114450+0000","last_clean":"2023-10-26T01:30:42.114450+0000","last_became_active":"2023-10-26T01:30:41.092922+0000","last_became_peered":"2023-10-26T01:30:41.092922+0000","last_unstale":"2023-10-26T01:30:42.114450+0000","last_undegraded":"2023-10-26T01:30:42.114450+0000","last_fullsized":"2023-10-26T01:30:42.114450+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:33:12.599402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.098208+0000","last_change":"2023-10-26T01:30:42.105139+0000","last_active":"2023-10-26T01:30:43.098208+0000","last_peered":"2023-10-26T01:30:43.098208+0000","last_clean":"2023-10-26T01:30:43.098208+0000","last_became_active":"2023-10-26T01:30:42.104913+0000","last_became_peered":"2023-10-26T01:30:42.104913+0000","last_unstale":"2023-10-26T01:30:43.098208+0000","last_undegraded":"2023-10-26T01:30:43.098208+0000","last_fullsized":"2023-10-26T01:30:43.098208+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:17:59.186810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.099380+0000","last_change":"2023-10-26T01:30:42.106926+0000","last_active":"2023-10-26T01:30:43.099380+0000","last_peered":"2023-10-26T01:30:43.099380+0000","last_clean":"2023-10-26T01:30:43.099380+0000","last_became_active":"2023-10-26T01:30:42.106692+0000","last_became_peered":"2023-10-26T01:30:42.106692+0000","last_unstale":"2023-10-26T01:30:43.099380+0000","last_undegraded":"2023-10-26T01:30:43.099380+0000","last_fullsized":"2023-10-26T01:30:43.099380+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:20:47.431800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.110477+0000","last_change":"2023-10-26T01:30:41.105282+0000","last_active":"2023-10-26T01:30:42.110477+0000","last_peered":"2023-10-26T01:30:42.110477+0000","last_clean":"2023-10-26T01:30:42.110477+0000","last_became_active":"2023-10-26T01:30:41.104553+0000","last_became_peered":"2023-10-26T01:30:41.104553+0000","last_unstale":"2023-10-26T01:30:42.110477+0000","last_undegraded":"2023-10-26T01:30:42.110477+0000","last_fullsized":"2023-10-26T01:30:42.110477+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:29:17.626922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.100164+0000","last_change":"2023-10-26T01:30:42.098532+0000","last_active":"2023-10-26T01:30:43.100164+0000","last_peered":"2023-10-26T01:30:43.100164+0000","last_clean":"2023-10-26T01:30:43.100164+0000","last_became_active":"2023-10-26T01:30:42.098301+0000","last_became_peered":"2023-10-26T01:30:42.098301+0000","last_unstale":"2023-10-26T01:30:43.100164+0000","last_undegraded":"2023-10-26T01:30:43.100164+0000","last_fullsized":"2023-10-26T01:30:43.100164+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:55:51.116465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.113350+0000","last_change":"2023-10-26T01:30:41.101188+0000","last_active":"2023-10-26T01:30:42.113350+0000","last_peered":"2023-10-26T01:30:42.113350+0000","last_clean":"2023-10-26T01:30:42.113350+0000","last_became_active":"2023-10-26T01:30:41.100977+0000","last_became_peered":"2023-10-26T01:30:41.100977+0000","last_unstale":"2023-10-26T01:30:42.113350+0000","last_undegraded":"2023-10-26T01:30:42.113350+0000","last_fullsized":"2023-10-26T01:30:42.113350+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:13:32.821792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.139789+0000","last_change":"2023-10-26T01:30:42.105438+0000","last_active":"2023-10-26T01:30:43.139789+0000","last_peered":"2023-10-26T01:30:43.139789+0000","last_clean":"2023-10-26T01:30:43.139789+0000","last_became_active":"2023-10-26T01:30:42.105168+0000","last_became_peered":"2023-10-26T01:30:42.105168+0000","last_unstale":"2023-10-26T01:30:43.139789+0000","last_undegraded":"2023-10-26T01:30:43.139789+0000","last_fullsized":"2023-10-26T01:30:43.139789+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:53:57.632838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-26T01:30:41.089058+0000","last_change":"2023-10-26T01:30:41.088679+0000","last_active":"2023-10-26T01:30:41.089058+0000","last_peered":"2023-10-26T01:30:41.089058+0000","last_clean":"2023-10-26T01:30:41.089058+0000","last_became_active":"2023-10-26T01:30:41.086354+0000","last_became_peered":"2023-10-26T01:30:41.086354+0000","last_unstale":"2023-10-26T01:30:41.089058+0000","last_undegraded":"2023-10-26T01:30:41.089058+0000","last_fullsized":"2023-10-26T01:30:41.089058+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:49:35.491976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-10-26T01:30:41.096096+0000","last_change":"2023-10-26T01:30:41.096096+0000","last_active":"2023-10-26T01:30:41.060332+0000","last_peered":"2023-10-26T01:30:41.060332+0000","last_clean":"2023-10-26T01:30:41.060332+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:30:41.096096+0000","last_undegraded":"2023-10-26T01:30:41.096096+0000","last_fullsized":"2023-10-26T01:30:41.096096+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.113466+0000","last_change":"2023-10-26T01:30:41.097404+0000","last_active":"2023-10-26T01:30:42.113466+0000","last_peered":"2023-10-26T01:30:42.113466+0000","last_clean":"2023-10-26T01:30:42.113466+0000","last_became_active":"2023-10-26T01:30:41.097074+0000","last_became_peered":"2023-10-26T01:30:41.097074+0000","last_unstale":"2023-10-26T01:30:42.113466+0000","last_undegraded":"2023-10-26T01:30:42.113466+0000","last_fullsized":"2023-10-26T01:30:42.113466+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:31:39.970444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":36,"ondisk_log_size":36,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":14,"seq":60129542148,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":54,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195258,"statfs":{"total":200000000,"available":199944205,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85999999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1990000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.462}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94299999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.911}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.347}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.514}]}]},{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":650,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194662,"statfs":{"total":200000000,"available":199333905,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94399999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0569999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.333}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2050000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.014}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.101}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4970000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":74,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195238,"statfs":{"total":200000000,"available":199923857,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0089999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.393}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4319999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.256}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4790000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.131}]}]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":73,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195239,"statfs":{"total":200000000,"available":199924765,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1990000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.367}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4399999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7190000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.117}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.639}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5009999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":73,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195238,"statfs":{"total":200000000,"available":199924729,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.751}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.635}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5169999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.276}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4850000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4490000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7430000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":13,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":73,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195239,"statfs":{"total":200000000,"available":199924855,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1000000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.171}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68400000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2210000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7869999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.597}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9039999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":73,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195238,"statfs":{"total":200000000,"available":199924695,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79900000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95599999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6180000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4470000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3069999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.391}]}]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":650,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194661,"statfs":{"total":200000000,"available":199333849,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.03}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.161}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.204}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2290000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0129999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.839}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3049999999999999}]}]}],"pool_statfs":[]}} 2023-10-26T01:30:47.053 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-26T01:30:47.313 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:47.310+0000 7fecc4ca9640 1 -- 172.21.15.169:0/55702627 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fecc00cde00 msgr2=0x7fecc0164040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:47.313 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:47.310+0000 7fecc4ca9640 1 --2- 172.21.15.169:0/55702627 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fecc00cde00 0x7fecc0164040 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fecb8009830 tx=0x7fecb802bcd0 comp rx=0 tx=0).stop 2023-10-26T01:30:47.313 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:47.310+0000 7fecc4ca9640 1 -- 172.21.15.169:0/55702627 shutdown_connections 2023-10-26T01:30:47.313 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:47.310+0000 7fecc4ca9640 1 --2- 172.21.15.169:0/55702627 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fecc00cde00 0x7fecc0164040 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:47.314 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:47.310+0000 7fecc4ca9640 1 -- 172.21.15.169:0/55702627 >> 172.21.15.169:0/55702627 conn(0x7fecc0069930 msgr2=0x7fecc0069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:47.314 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:47.310+0000 7fecc4ca9640 1 -- 172.21.15.169:0/55702627 shutdown_connections 2023-10-26T01:30:47.314 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:47.310+0000 7fecc4ca9640 1 -- 172.21.15.169:0/55702627 wait complete. 2023-10-26T01:30:47.315 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:47.312+0000 7fecc4ca9640 1 Processor -- start 2023-10-26T01:30:47.316 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:47.312+0000 7fecc4ca9640 1 -- start start 2023-10-26T01:30:47.316 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:47.313+0000 7fecc4ca9640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fecc00ac930 0x7fecc014e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:47.316 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:47.313+0000 7fecc4ca9640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fecc00cde00 0x7fecc00b1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:47.316 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:47.313+0000 7fecc4ca9640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fecc01592d0 0x7fecc00b10d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:47.317 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:47.313+0000 7fecc4ca9640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecc00b1610 con 0x7fecc00cde00 2023-10-26T01:30:47.317 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:47.313+0000 7fecc4ca9640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecc014e8c0 con 0x7fecc01592d0 2023-10-26T01:30:47.317 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:47.313+0000 7fecc4ca9640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecc00b0330 con 0x7fecc00ac930 2023-10-26T01:30:47.317 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:47.313+0000 7fecbdd9b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fecc00cde00 0x7fecc00b1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:47.317 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:47.313+0000 7fecbdd9b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fecc00cde00 0x7fecc00b1eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:33976/0 (socket says 172.21.15.169:33976) 2023-10-26T01:30:47.318 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:47.313+0000 7fecbdd9b640 1 -- 172.21.15.169:0/1810264822 learned_addr learned my addr 172.21.15.169:0/1810264822 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:30:47.318 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:47.313+0000 7fecbdd9b640 1 -- 172.21.15.169:0/1810264822 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fecc00ac930 msgr2=0x7fecc014e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:47.318 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:47.313+0000 7fecbed9d640 1 --2- 172.21.15.169:0/1810264822 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fecc01592d0 0x7fecc00b10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:47.318 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:47.313+0000 7fecbdd9b640 1 --2- 172.21.15.169:0/1810264822 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fecc00ac930 0x7fecc014e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:47.318 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:47.313+0000 7fecbdd9b640 1 -- 172.21.15.169:0/1810264822 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fecc01592d0 msgr2=0x7fecc00b10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:47.319 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:47.313+0000 7fecbdd9b640 1 --2- 172.21.15.169:0/1810264822 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fecc01592d0 0x7fecc00b10d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:47.319 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:47.313+0000 7fecbdd9b640 1 -- 172.21.15.169:0/1810264822 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecb8009490 con 0x7fecc00cde00 2023-10-26T01:30:47.320 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:47.314+0000 7fecbdd9b640 1 --2- 172.21.15.169:0/1810264822 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fecc00cde00 0x7fecc00b1eb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7feca800c500 tx=0x7feca800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:47.320 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:47.314+0000 7fecb6ffd640 1 -- 172.21.15.169:0/1810264822 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feca8012830 con 0x7fecc00cde00 2023-10-26T01:30:47.321 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:47.314+0000 7fecb6ffd640 1 -- 172.21.15.169:0/1810264822 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7feca8012a80 con 0x7fecc00cde00 2023-10-26T01:30:47.321 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:47.314+0000 7fecb6ffd640 1 -- 172.21.15.169:0/1810264822 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feca8012e50 con 0x7fecc00cde00 2023-10-26T01:30:47.321 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:47.314+0000 7fecc4ca9640 1 -- 172.21.15.169:0/1810264822 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecc00b0610 con 0x7fecc00cde00 2023-10-26T01:30:47.321 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:47.314+0000 7fecc4ca9640 1 -- 172.21.15.169:0/1810264822 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecc00b0bd0 con 0x7fecc00cde00 2023-10-26T01:30:47.321 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:47.315+0000 7fecc4ca9640 1 -- 172.21.15.169:0/1810264822 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecc004db10 con 0x7fecc00cde00 2023-10-26T01:30:47.322 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:47.316+0000 7fecb6ffd640 1 -- 172.21.15.169:0/1810264822 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7feca801b400 con 0x7fecc00cde00 2023-10-26T01:30:47.322 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:47.317+0000 7fecb6ffd640 1 --2- 172.21.15.169:0/1810264822 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7feca006ae40 0x7feca006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:47.322 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:47.317+0000 7fecb6ffd640 1 -- 172.21.15.169:0/1810264822 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7feca8089ae0 con 0x7fecc00cde00 2023-10-26T01:30:47.322 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:47.318+0000 7fecbe59c640 1 --2- 172.21.15.169:0/1810264822 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7feca006ae40 0x7feca006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:47.323 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:47.319+0000 7fecbe59c640 1 --2- 172.21.15.169:0/1810264822 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7feca006ae40 0x7feca006d300 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fecc00ac810 tx=0x7fecb0009230 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:47.324 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:47.322+0000 7fecb6ffd640 1 -- 172.21.15.169:0/1810264822 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7feca8058590 con 0x7fecc00cde00 2023-10-26T01:30:47.503 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:47.500+0000 7fecc4ca9640 1 -- 172.21.15.169:0/1810264822 --> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fecc00b04a0 con 0x7feca006ae40 2023-10-26T01:30:47.524 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:47.522+0000 7fecb6ffd640 1 -- 172.21.15.169:0/1810264822 <== mgr.4112 v2:172.21.15.170:6832/1187156918 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+259081 (secure 0 0 0) 0x7fecc00b04a0 con 0x7feca006ae40 2023-10-26T01:30:47.528 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:30:47.529 INFO:teuthology.orchestra.run.smithi169.stderr:dumped all 2023-10-26T01:30:47.531 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:47.528+0000 7fecc4ca9640 1 -- 172.21.15.169:0/1810264822 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7feca006ae40 msgr2=0x7feca006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:47.531 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:47.528+0000 7fecc4ca9640 1 --2- 172.21.15.169:0/1810264822 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7feca006ae40 0x7feca006d300 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fecc00ac810 tx=0x7fecb0009230 comp rx=0 tx=0).stop 2023-10-26T01:30:47.531 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:47.528+0000 7fecc4ca9640 1 -- 172.21.15.169:0/1810264822 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fecc00cde00 msgr2=0x7fecc00b1eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:47.532 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:47.528+0000 7fecc4ca9640 1 --2- 172.21.15.169:0/1810264822 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fecc00cde00 0x7fecc00b1eb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7feca800c500 tx=0x7feca800c9c0 comp rx=0 tx=0).stop 2023-10-26T01:30:47.532 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:47.528+0000 7fecc4ca9640 1 -- 172.21.15.169:0/1810264822 shutdown_connections 2023-10-26T01:30:47.532 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:47.528+0000 7fecc4ca9640 1 --2- 172.21.15.169:0/1810264822 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7feca006ae40 0x7feca006d300 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:47.532 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:47.528+0000 7fecc4ca9640 1 --2- 172.21.15.169:0/1810264822 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fecc00cde00 0x7fecc00b1eb0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:47.532 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:47.528+0000 7fecc4ca9640 1 --2- 172.21.15.169:0/1810264822 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fecc01592d0 0x7fecc00b10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:47.533 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:47.528+0000 7fecc4ca9640 1 --2- 172.21.15.169:0/1810264822 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fecc00ac930 0x7fecc014e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:47.533 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:47.528+0000 7fecc4ca9640 1 -- 172.21.15.169:0/1810264822 >> 172.21.15.169:0/1810264822 conn(0x7fecc0069930 msgr2=0x7fecc0158520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:47.533 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:47.528+0000 7fecc4ca9640 1 -- 172.21.15.169:0/1810264822 shutdown_connections 2023-10-26T01:30:47.533 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:47.528+0000 7fecc4ca9640 1 -- 172.21.15.169:0/1810264822 wait complete. 2023-10-26T01:30:47.549 INFO:teuthology.orchestra.run.smithi169.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-10-26T01:30:46.128888+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":153,"ondisk_log_size":153,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":169,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":1723,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560776,"statfs":{"total":1600000000,"available":1598234860,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":58,"num_write_kb":276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001802"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.086425+0000","last_change":"2023-10-26T01:30:42.112552+0000","last_active":"2023-10-26T01:30:43.086425+0000","last_peered":"2023-10-26T01:30:43.086425+0000","last_clean":"2023-10-26T01:30:43.086425+0000","last_became_active":"2023-10-26T01:30:42.111151+0000","last_became_peered":"2023-10-26T01:30:42.111151+0000","last_unstale":"2023-10-26T01:30:43.086425+0000","last_undegraded":"2023-10-26T01:30:43.086425+0000","last_fullsized":"2023-10-26T01:30:43.086425+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:28:22.384005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.141033+0000","last_change":"2023-10-26T01:30:42.102872+0000","last_active":"2023-10-26T01:30:43.141033+0000","last_peered":"2023-10-26T01:30:43.141033+0000","last_clean":"2023-10-26T01:30:43.141033+0000","last_became_active":"2023-10-26T01:30:42.100796+0000","last_became_peered":"2023-10-26T01:30:42.100796+0000","last_unstale":"2023-10-26T01:30:43.141033+0000","last_undegraded":"2023-10-26T01:30:43.141033+0000","last_fullsized":"2023-10-26T01:30:43.141033+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:49:12.753504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.139076+0000","last_change":"2023-10-26T01:30:42.103921+0000","last_active":"2023-10-26T01:30:43.139076+0000","last_peered":"2023-10-26T01:30:43.139076+0000","last_clean":"2023-10-26T01:30:43.139076+0000","last_became_active":"2023-10-26T01:30:42.103384+0000","last_became_peered":"2023-10-26T01:30:42.103384+0000","last_unstale":"2023-10-26T01:30:43.139076+0000","last_undegraded":"2023-10-26T01:30:43.139076+0000","last_fullsized":"2023-10-26T01:30:43.139076+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:33:58.768305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.098597+0000","last_change":"2023-10-26T01:30:42.101845+0000","last_active":"2023-10-26T01:30:43.098597+0000","last_peered":"2023-10-26T01:30:43.098597+0000","last_clean":"2023-10-26T01:30:43.098597+0000","last_became_active":"2023-10-26T01:30:42.097125+0000","last_became_peered":"2023-10-26T01:30:42.097125+0000","last_unstale":"2023-10-26T01:30:43.098597+0000","last_undegraded":"2023-10-26T01:30:43.098597+0000","last_fullsized":"2023-10-26T01:30:43.098597+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:33:13.615628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.086419+0000","last_change":"2023-10-26T01:30:42.115244+0000","last_active":"2023-10-26T01:30:43.086419+0000","last_peered":"2023-10-26T01:30:43.086419+0000","last_clean":"2023-10-26T01:30:43.086419+0000","last_became_active":"2023-10-26T01:30:42.114972+0000","last_became_peered":"2023-10-26T01:30:42.114972+0000","last_unstale":"2023-10-26T01:30:43.086419+0000","last_undegraded":"2023-10-26T01:30:43.086419+0000","last_fullsized":"2023-10-26T01:30:43.086419+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:07:06.450331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.099993+0000","last_change":"2023-10-26T01:30:42.108711+0000","last_active":"2023-10-26T01:30:43.099993+0000","last_peered":"2023-10-26T01:30:43.099993+0000","last_clean":"2023-10-26T01:30:43.099993+0000","last_became_active":"2023-10-26T01:30:42.108215+0000","last_became_peered":"2023-10-26T01:30:42.108215+0000","last_unstale":"2023-10-26T01:30:43.099993+0000","last_undegraded":"2023-10-26T01:30:43.099993+0000","last_fullsized":"2023-10-26T01:30:43.099993+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:26:01.898201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.088319+0000","last_change":"2023-10-26T01:30:42.115268+0000","last_active":"2023-10-26T01:30:43.088319+0000","last_peered":"2023-10-26T01:30:43.088319+0000","last_clean":"2023-10-26T01:30:43.088319+0000","last_became_active":"2023-10-26T01:30:42.113208+0000","last_became_peered":"2023-10-26T01:30:42.113208+0000","last_unstale":"2023-10-26T01:30:43.088319+0000","last_undegraded":"2023-10-26T01:30:43.088319+0000","last_fullsized":"2023-10-26T01:30:43.088319+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:56:55.394232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-10-26T01:30:41.100533+0000","last_change":"2023-10-26T01:30:41.100533+0000","last_active":"2023-10-26T01:30:41.060332+0000","last_peered":"2023-10-26T01:30:41.060332+0000","last_clean":"2023-10-26T01:30:41.060332+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:30:41.100533+0000","last_undegraded":"2023-10-26T01:30:41.100533+0000","last_fullsized":"2023-10-26T01:30:41.100533+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.086093+0000","last_change":"2023-10-26T01:30:42.108451+0000","last_active":"2023-10-26T01:30:43.086093+0000","last_peered":"2023-10-26T01:30:43.086093+0000","last_clean":"2023-10-26T01:30:43.086093+0000","last_became_active":"2023-10-26T01:30:42.107650+0000","last_became_peered":"2023-10-26T01:30:42.107650+0000","last_unstale":"2023-10-26T01:30:43.086093+0000","last_undegraded":"2023-10-26T01:30:43.086093+0000","last_fullsized":"2023-10-26T01:30:43.086093+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:26:19.035685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.098895+0000","last_change":"2023-10-26T01:30:42.093838+0000","last_active":"2023-10-26T01:30:43.098895+0000","last_peered":"2023-10-26T01:30:43.098895+0000","last_clean":"2023-10-26T01:30:43.098895+0000","last_became_active":"2023-10-26T01:30:42.093584+0000","last_became_peered":"2023-10-26T01:30:42.093584+0000","last_unstale":"2023-10-26T01:30:43.098895+0000","last_undegraded":"2023-10-26T01:30:43.098895+0000","last_fullsized":"2023-10-26T01:30:43.098895+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:49:12.753504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.097658+0000","last_change":"2023-10-26T01:30:42.094026+0000","last_active":"2023-10-26T01:30:43.097658+0000","last_peered":"2023-10-26T01:30:43.097658+0000","last_clean":"2023-10-26T01:30:43.097658+0000","last_became_active":"2023-10-26T01:30:42.093654+0000","last_became_peered":"2023-10-26T01:30:42.093654+0000","last_unstale":"2023-10-26T01:30:43.097658+0000","last_undegraded":"2023-10-26T01:30:43.097658+0000","last_fullsized":"2023-10-26T01:30:43.097658+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:05:31.590291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.088414+0000","last_change":"2023-10-26T01:30:42.099764+0000","last_active":"2023-10-26T01:30:43.088414+0000","last_peered":"2023-10-26T01:30:43.088414+0000","last_clean":"2023-10-26T01:30:43.088414+0000","last_became_active":"2023-10-26T01:30:42.098108+0000","last_became_peered":"2023-10-26T01:30:42.098108+0000","last_unstale":"2023-10-26T01:30:43.088414+0000","last_undegraded":"2023-10-26T01:30:43.088414+0000","last_fullsized":"2023-10-26T01:30:43.088414+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:08:21.563278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-10-26T01:30:41.099897+0000","last_change":"2023-10-26T01:30:41.099897+0000","last_active":"2023-10-26T01:30:41.060332+0000","last_peered":"2023-10-26T01:30:41.060332+0000","last_clean":"2023-10-26T01:30:41.060332+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:30:41.099897+0000","last_undegraded":"2023-10-26T01:30:41.099897+0000","last_fullsized":"2023-10-26T01:30:41.099897+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.138887+0000","last_change":"2023-10-26T01:30:42.108715+0000","last_active":"2023-10-26T01:30:43.138887+0000","last_peered":"2023-10-26T01:30:43.138887+0000","last_clean":"2023-10-26T01:30:43.138887+0000","last_became_active":"2023-10-26T01:30:42.108469+0000","last_became_peered":"2023-10-26T01:30:42.108469+0000","last_unstale":"2023-10-26T01:30:43.138887+0000","last_undegraded":"2023-10-26T01:30:43.138887+0000","last_fullsized":"2023-10-26T01:30:43.138887+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:45:23.061469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.140916+0000","last_change":"2023-10-26T01:30:42.099157+0000","last_active":"2023-10-26T01:30:43.140916+0000","last_peered":"2023-10-26T01:30:43.140916+0000","last_clean":"2023-10-26T01:30:43.140916+0000","last_became_active":"2023-10-26T01:30:42.098929+0000","last_became_peered":"2023-10-26T01:30:42.098929+0000","last_unstale":"2023-10-26T01:30:43.140916+0000","last_undegraded":"2023-10-26T01:30:43.140916+0000","last_fullsized":"2023-10-26T01:30:43.140916+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:13:33.838018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.099835+0000","last_change":"2023-10-26T01:30:42.108578+0000","last_active":"2023-10-26T01:30:43.099835+0000","last_peered":"2023-10-26T01:30:43.099835+0000","last_clean":"2023-10-26T01:30:43.099835+0000","last_became_active":"2023-10-26T01:30:42.107937+0000","last_became_peered":"2023-10-26T01:30:42.107937+0000","last_unstale":"2023-10-26T01:30:43.099835+0000","last_undegraded":"2023-10-26T01:30:43.099835+0000","last_fullsized":"2023-10-26T01:30:43.099835+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:38:16.726919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.098559+0000","last_change":"2023-10-26T01:30:42.108373+0000","last_active":"2023-10-26T01:30:43.098559+0000","last_peered":"2023-10-26T01:30:43.098559+0000","last_clean":"2023-10-26T01:30:43.098559+0000","last_became_active":"2023-10-26T01:30:42.107773+0000","last_became_peered":"2023-10-26T01:30:42.107773+0000","last_unstale":"2023-10-26T01:30:43.098559+0000","last_undegraded":"2023-10-26T01:30:43.098559+0000","last_fullsized":"2023-10-26T01:30:43.098559+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:49:12.753504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.097381+0000","last_change":"2023-10-26T01:30:42.094993+0000","last_active":"2023-10-26T01:30:43.097381+0000","last_peered":"2023-10-26T01:30:43.097381+0000","last_clean":"2023-10-26T01:30:43.097381+0000","last_became_active":"2023-10-26T01:30:42.094689+0000","last_became_peered":"2023-10-26T01:30:42.094689+0000","last_unstale":"2023-10-26T01:30:43.097381+0000","last_undegraded":"2023-10-26T01:30:43.097381+0000","last_fullsized":"2023-10-26T01:30:43.097381+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:49:43.468558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.086773+0000","last_change":"2023-10-26T01:30:42.108746+0000","last_active":"2023-10-26T01:30:43.086773+0000","last_peered":"2023-10-26T01:30:43.086773+0000","last_clean":"2023-10-26T01:30:43.086773+0000","last_became_active":"2023-10-26T01:30:42.108188+0000","last_became_peered":"2023-10-26T01:30:42.108188+0000","last_unstale":"2023-10-26T01:30:43.086773+0000","last_undegraded":"2023-10-26T01:30:43.086773+0000","last_fullsized":"2023-10-26T01:30:43.086773+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:38:16.726919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.099061+0000","last_change":"2023-10-26T01:30:42.110331+0000","last_active":"2023-10-26T01:30:43.099061+0000","last_peered":"2023-10-26T01:30:43.099061+0000","last_clean":"2023-10-26T01:30:43.099061+0000","last_became_active":"2023-10-26T01:30:42.110079+0000","last_became_peered":"2023-10-26T01:30:42.110079+0000","last_unstale":"2023-10-26T01:30:43.099061+0000","last_undegraded":"2023-10-26T01:30:43.099061+0000","last_fullsized":"2023-10-26T01:30:43.099061+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:33:13.615628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.139607+0000","last_change":"2023-10-26T01:30:42.108608+0000","last_active":"2023-10-26T01:30:43.139607+0000","last_peered":"2023-10-26T01:30:43.139607+0000","last_clean":"2023-10-26T01:30:43.139607+0000","last_became_active":"2023-10-26T01:30:42.108197+0000","last_became_peered":"2023-10-26T01:30:42.108197+0000","last_unstale":"2023-10-26T01:30:43.139607+0000","last_undegraded":"2023-10-26T01:30:43.139607+0000","last_fullsized":"2023-10-26T01:30:43.139607+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:17:59.186810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.098944+0000","last_change":"2023-10-26T01:30:42.110289+0000","last_active":"2023-10-26T01:30:43.098944+0000","last_peered":"2023-10-26T01:30:43.098944+0000","last_clean":"2023-10-26T01:30:43.098944+0000","last_became_active":"2023-10-26T01:30:42.110061+0000","last_became_peered":"2023-10-26T01:30:42.110061+0000","last_unstale":"2023-10-26T01:30:43.098944+0000","last_undegraded":"2023-10-26T01:30:43.098944+0000","last_fullsized":"2023-10-26T01:30:43.098944+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:19:23.544531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-10-26T01:30:41.098174+0000","last_change":"2023-10-26T01:30:41.098174+0000","last_active":"2023-10-26T01:30:41.060332+0000","last_peered":"2023-10-26T01:30:41.060332+0000","last_clean":"2023-10-26T01:30:41.060332+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:30:41.098174+0000","last_undegraded":"2023-10-26T01:30:41.098174+0000","last_fullsized":"2023-10-26T01:30:41.098174+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.098984+0000","last_change":"2023-10-26T01:30:42.102147+0000","last_active":"2023-10-26T01:30:43.098984+0000","last_peered":"2023-10-26T01:30:43.098984+0000","last_clean":"2023-10-26T01:30:43.098984+0000","last_became_active":"2023-10-26T01:30:42.097509+0000","last_became_peered":"2023-10-26T01:30:42.097509+0000","last_unstale":"2023-10-26T01:30:43.098984+0000","last_undegraded":"2023-10-26T01:30:43.098984+0000","last_fullsized":"2023-10-26T01:30:43.098984+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:26:19.035685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-10-26T01:30:41.098206+0000","last_change":"2023-10-26T01:30:41.098206+0000","last_active":"2023-10-26T01:30:41.060332+0000","last_peered":"2023-10-26T01:30:41.060332+0000","last_clean":"2023-10-26T01:30:41.060332+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:30:41.098206+0000","last_undegraded":"2023-10-26T01:30:41.098206+0000","last_fullsized":"2023-10-26T01:30:41.098206+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.097862+0000","last_change":"2023-10-26T01:30:42.111865+0000","last_active":"2023-10-26T01:30:43.097862+0000","last_peered":"2023-10-26T01:30:43.097862+0000","last_clean":"2023-10-26T01:30:43.097862+0000","last_became_active":"2023-10-26T01:30:42.111636+0000","last_became_peered":"2023-10-26T01:30:42.111636+0000","last_unstale":"2023-10-26T01:30:43.097862+0000","last_undegraded":"2023-10-26T01:30:43.097862+0000","last_fullsized":"2023-10-26T01:30:43.097862+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:49:56.641022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.141348+0000","last_change":"2023-10-26T01:30:42.091332+0000","last_active":"2023-10-26T01:30:43.141348+0000","last_peered":"2023-10-26T01:30:43.141348+0000","last_clean":"2023-10-26T01:30:43.141348+0000","last_became_active":"2023-10-26T01:30:42.091105+0000","last_became_peered":"2023-10-26T01:30:42.091105+0000","last_unstale":"2023-10-26T01:30:43.141348+0000","last_undegraded":"2023-10-26T01:30:43.141348+0000","last_fullsized":"2023-10-26T01:30:43.141348+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:33:13.615628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-10-26T01:30:41.095233+0000","last_change":"2023-10-26T01:30:41.095233+0000","last_active":"2023-10-26T01:30:41.060332+0000","last_peered":"2023-10-26T01:30:41.060332+0000","last_clean":"2023-10-26T01:30:41.060332+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:30:41.095233+0000","last_undegraded":"2023-10-26T01:30:41.095233+0000","last_fullsized":"2023-10-26T01:30:41.095233+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.139200+0000","last_change":"2023-10-26T01:30:42.101708+0000","last_active":"2023-10-26T01:30:43.139200+0000","last_peered":"2023-10-26T01:30:43.139200+0000","last_clean":"2023-10-26T01:30:43.139200+0000","last_became_active":"2023-10-26T01:30:42.101434+0000","last_became_peered":"2023-10-26T01:30:42.101434+0000","last_unstale":"2023-10-26T01:30:43.139200+0000","last_undegraded":"2023-10-26T01:30:43.139200+0000","last_fullsized":"2023-10-26T01:30:43.139200+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:29:18.643148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.099207+0000","last_change":"2023-10-26T01:30:42.097495+0000","last_active":"2023-10-26T01:30:43.099207+0000","last_peered":"2023-10-26T01:30:43.099207+0000","last_clean":"2023-10-26T01:30:43.099207+0000","last_became_active":"2023-10-26T01:30:42.097214+0000","last_became_peered":"2023-10-26T01:30:42.097214+0000","last_unstale":"2023-10-26T01:30:43.099207+0000","last_undegraded":"2023-10-26T01:30:43.099207+0000","last_fullsized":"2023-10-26T01:30:43.099207+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T01:55:28.470913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.097481+0000","last_change":"2023-10-26T01:30:42.093157+0000","last_active":"2023-10-26T01:30:43.097481+0000","last_peered":"2023-10-26T01:30:43.097481+0000","last_clean":"2023-10-26T01:30:43.097481+0000","last_became_active":"2023-10-26T01:30:42.092613+0000","last_became_peered":"2023-10-26T01:30:42.092613+0000","last_unstale":"2023-10-26T01:30:43.097481+0000","last_undegraded":"2023-10-26T01:30:43.097481+0000","last_fullsized":"2023-10-26T01:30:43.097481+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:09:01.061004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.098226+0000","last_change":"2023-10-26T01:30:42.102704+0000","last_active":"2023-10-26T01:30:43.098226+0000","last_peered":"2023-10-26T01:30:43.098226+0000","last_clean":"2023-10-26T01:30:43.098226+0000","last_became_active":"2023-10-26T01:30:42.102350+0000","last_became_peered":"2023-10-26T01:30:42.102350+0000","last_unstale":"2023-10-26T01:30:43.098226+0000","last_undegraded":"2023-10-26T01:30:43.098226+0000","last_fullsized":"2023-10-26T01:30:43.098226+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:53:57.632838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.086493+0000","last_change":"2023-10-26T01:30:42.093818+0000","last_active":"2023-10-26T01:30:43.086493+0000","last_peered":"2023-10-26T01:30:43.086493+0000","last_clean":"2023-10-26T01:30:43.086493+0000","last_became_active":"2023-10-26T01:30:42.091310+0000","last_became_peered":"2023-10-26T01:30:42.091310+0000","last_unstale":"2023-10-26T01:30:43.086493+0000","last_undegraded":"2023-10-26T01:30:43.086493+0000","last_fullsized":"2023-10-26T01:30:43.086493+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:04:23.158155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-26T01:30:41.097736+0000","last_change":"2023-10-26T01:30:41.097666+0000","last_active":"2023-10-26T01:30:41.097736+0000","last_peered":"2023-10-26T01:30:41.097736+0000","last_clean":"2023-10-26T01:30:41.097736+0000","last_became_active":"2023-10-26T01:30:41.093986+0000","last_became_peered":"2023-10-26T01:30:41.093986+0000","last_unstale":"2023-10-26T01:30:41.097736+0000","last_undegraded":"2023-10-26T01:30:41.097736+0000","last_fullsized":"2023-10-26T01:30:41.097736+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:13:28.300112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.139516+0000","last_change":"2023-10-26T01:30:42.110839+0000","last_active":"2023-10-26T01:30:43.139516+0000","last_peered":"2023-10-26T01:30:43.139516+0000","last_clean":"2023-10-26T01:30:43.139516+0000","last_became_active":"2023-10-26T01:30:42.109385+0000","last_became_peered":"2023-10-26T01:30:42.109385+0000","last_unstale":"2023-10-26T01:30:43.139516+0000","last_undegraded":"2023-10-26T01:30:43.139516+0000","last_fullsized":"2023-10-26T01:30:43.139516+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:49:56.641022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.105272+0000","last_change":"2023-10-26T01:30:41.100248+0000","last_active":"2023-10-26T01:30:42.105272+0000","last_peered":"2023-10-26T01:30:42.105272+0000","last_clean":"2023-10-26T01:30:42.105272+0000","last_became_active":"2023-10-26T01:30:41.099957+0000","last_became_peered":"2023-10-26T01:30:41.099957+0000","last_unstale":"2023-10-26T01:30:42.105272+0000","last_undegraded":"2023-10-26T01:30:42.105272+0000","last_fullsized":"2023-10-26T01:30:42.105272+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:29:17.626922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-26T01:30:41.104334+0000","last_change":"2023-10-26T01:30:41.103843+0000","last_active":"2023-10-26T01:30:41.104334+0000","last_peered":"2023-10-26T01:30:41.104334+0000","last_clean":"2023-10-26T01:30:41.104334+0000","last_became_active":"2023-10-26T01:30:41.103356+0000","last_became_peered":"2023-10-26T01:30:41.103356+0000","last_unstale":"2023-10-26T01:30:41.104334+0000","last_undegraded":"2023-10-26T01:30:41.104334+0000","last_fullsized":"2023-10-26T01:30:41.104334+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:05:43.728770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-10-26T01:30:41.097717+0000","last_change":"2023-10-26T01:30:41.097717+0000","last_active":"2023-10-26T01:30:41.060332+0000","last_peered":"2023-10-26T01:30:41.060332+0000","last_clean":"2023-10-26T01:30:41.060332+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:30:41.097717+0000","last_undegraded":"2023-10-26T01:30:41.097717+0000","last_fullsized":"2023-10-26T01:30:41.097717+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.089693+0000","last_change":"2023-10-26T01:30:41.102989+0000","last_active":"2023-10-26T01:30:42.089693+0000","last_peered":"2023-10-26T01:30:42.089693+0000","last_clean":"2023-10-26T01:30:42.089693+0000","last_became_active":"2023-10-26T01:30:41.102681+0000","last_became_peered":"2023-10-26T01:30:41.102681+0000","last_unstale":"2023-10-26T01:30:42.089693+0000","last_undegraded":"2023-10-26T01:30:42.089693+0000","last_fullsized":"2023-10-26T01:30:42.089693+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:49:55.624796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.140341+0000","last_change":"2023-10-26T01:30:42.091359+0000","last_active":"2023-10-26T01:30:43.140341+0000","last_peered":"2023-10-26T01:30:43.140341+0000","last_clean":"2023-10-26T01:30:43.140341+0000","last_became_active":"2023-10-26T01:30:42.090949+0000","last_became_peered":"2023-10-26T01:30:42.090949+0000","last_unstale":"2023-10-26T01:30:43.140341+0000","last_undegraded":"2023-10-26T01:30:43.140341+0000","last_fullsized":"2023-10-26T01:30:43.140341+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:49:56.641022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.089836+0000","last_change":"2023-10-26T01:30:41.104058+0000","last_active":"2023-10-26T01:30:42.089836+0000","last_peered":"2023-10-26T01:30:42.089836+0000","last_clean":"2023-10-26T01:30:42.089836+0000","last_became_active":"2023-10-26T01:30:41.103821+0000","last_became_peered":"2023-10-26T01:30:41.103821+0000","last_unstale":"2023-10-26T01:30:42.089836+0000","last_undegraded":"2023-10-26T01:30:42.089836+0000","last_fullsized":"2023-10-26T01:30:42.089836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:46:05.927163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.098774+0000","last_change":"2023-10-26T01:30:42.108520+0000","last_active":"2023-10-26T01:30:43.098774+0000","last_peered":"2023-10-26T01:30:43.098774+0000","last_clean":"2023-10-26T01:30:43.098774+0000","last_became_active":"2023-10-26T01:30:42.108140+0000","last_became_peered":"2023-10-26T01:30:42.108140+0000","last_unstale":"2023-10-26T01:30:43.098774+0000","last_undegraded":"2023-10-26T01:30:43.098774+0000","last_fullsized":"2023-10-26T01:30:43.098774+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:31:08.727626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.088126+0000","last_change":"2023-10-26T01:30:41.106779+0000","last_active":"2023-10-26T01:30:42.088126+0000","last_peered":"2023-10-26T01:30:42.088126+0000","last_clean":"2023-10-26T01:30:42.088126+0000","last_became_active":"2023-10-26T01:30:41.104353+0000","last_became_peered":"2023-10-26T01:30:41.104353+0000","last_unstale":"2023-10-26T01:30:42.088126+0000","last_undegraded":"2023-10-26T01:30:42.088126+0000","last_fullsized":"2023-10-26T01:30:42.088126+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:45:22.045243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.139887+0000","last_change":"2023-10-26T01:30:42.105447+0000","last_active":"2023-10-26T01:30:43.139887+0000","last_peered":"2023-10-26T01:30:43.139887+0000","last_clean":"2023-10-26T01:30:43.139887+0000","last_became_active":"2023-10-26T01:30:42.105098+0000","last_became_peered":"2023-10-26T01:30:42.105098+0000","last_unstale":"2023-10-26T01:30:43.139887+0000","last_undegraded":"2023-10-26T01:30:43.139887+0000","last_fullsized":"2023-10-26T01:30:43.139887+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:13:33.838018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.110602+0000","last_change":"2023-10-26T01:30:41.103826+0000","last_active":"2023-10-26T01:30:42.110602+0000","last_peered":"2023-10-26T01:30:42.110602+0000","last_clean":"2023-10-26T01:30:42.110602+0000","last_became_active":"2023-10-26T01:30:41.103552+0000","last_became_peered":"2023-10-26T01:30:41.103552+0000","last_unstale":"2023-10-26T01:30:42.110602+0000","last_undegraded":"2023-10-26T01:30:42.110602+0000","last_fullsized":"2023-10-26T01:30:42.110602+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:54:04.970133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.086144+0000","last_change":"2023-10-26T01:30:42.114167+0000","last_active":"2023-10-26T01:30:43.086144+0000","last_peered":"2023-10-26T01:30:43.086144+0000","last_clean":"2023-10-26T01:30:43.086144+0000","last_became_active":"2023-10-26T01:30:42.113241+0000","last_became_peered":"2023-10-26T01:30:42.113241+0000","last_unstale":"2023-10-26T01:30:43.086144+0000","last_undegraded":"2023-10-26T01:30:43.086144+0000","last_fullsized":"2023-10-26T01:30:43.086144+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:40:16.215681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.116049+0000","last_change":"2023-10-26T01:30:41.106514+0000","last_active":"2023-10-26T01:30:42.116049+0000","last_peered":"2023-10-26T01:30:42.116049+0000","last_clean":"2023-10-26T01:30:42.116049+0000","last_became_active":"2023-10-26T01:30:41.106147+0000","last_became_peered":"2023-10-26T01:30:41.106147+0000","last_unstale":"2023-10-26T01:30:42.116049+0000","last_undegraded":"2023-10-26T01:30:42.116049+0000","last_fullsized":"2023-10-26T01:30:42.116049+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:45:22.045243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.089920+0000","last_change":"2023-10-26T01:30:42.112813+0000","last_active":"2023-10-26T01:30:43.089920+0000","last_peered":"2023-10-26T01:30:43.089920+0000","last_clean":"2023-10-26T01:30:43.089920+0000","last_became_active":"2023-10-26T01:30:42.112571+0000","last_became_peered":"2023-10-26T01:30:42.112571+0000","last_unstale":"2023-10-26T01:30:43.089920+0000","last_undegraded":"2023-10-26T01:30:43.089920+0000","last_fullsized":"2023-10-26T01:30:43.089920+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:19:16.042472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.108474+0000","last_change":"2023-10-26T01:30:41.098865+0000","last_active":"2023-10-26T01:30:42.108474+0000","last_peered":"2023-10-26T01:30:42.108474+0000","last_clean":"2023-10-26T01:30:42.108474+0000","last_became_active":"2023-10-26T01:30:41.098506+0000","last_became_peered":"2023-10-26T01:30:41.098506+0000","last_unstale":"2023-10-26T01:30:42.108474+0000","last_undegraded":"2023-10-26T01:30:42.108474+0000","last_fullsized":"2023-10-26T01:30:42.108474+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:09:00.044778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.140188+0000","last_change":"2023-10-26T01:30:42.108443+0000","last_active":"2023-10-26T01:30:43.140188+0000","last_peered":"2023-10-26T01:30:43.140188+0000","last_clean":"2023-10-26T01:30:43.140188+0000","last_became_active":"2023-10-26T01:30:42.107837+0000","last_became_peered":"2023-10-26T01:30:42.107837+0000","last_unstale":"2023-10-26T01:30:43.140188+0000","last_undegraded":"2023-10-26T01:30:43.140188+0000","last_fullsized":"2023-10-26T01:30:43.140188+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:33:13.615628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.090761+0000","last_change":"2023-10-26T01:30:41.106795+0000","last_active":"2023-10-26T01:30:42.090761+0000","last_peered":"2023-10-26T01:30:42.090761+0000","last_clean":"2023-10-26T01:30:42.090761+0000","last_became_active":"2023-10-26T01:30:41.106222+0000","last_became_peered":"2023-10-26T01:30:41.106222+0000","last_unstale":"2023-10-26T01:30:42.090761+0000","last_undegraded":"2023-10-26T01:30:42.090761+0000","last_fullsized":"2023-10-26T01:30:42.090761+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:53:56.616612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.086822+0000","last_change":"2023-10-26T01:30:42.113053+0000","last_active":"2023-10-26T01:30:43.086822+0000","last_peered":"2023-10-26T01:30:43.086822+0000","last_clean":"2023-10-26T01:30:43.086822+0000","last_became_active":"2023-10-26T01:30:42.112492+0000","last_became_peered":"2023-10-26T01:30:42.112492+0000","last_unstale":"2023-10-26T01:30:43.086822+0000","last_undegraded":"2023-10-26T01:30:43.086822+0000","last_fullsized":"2023-10-26T01:30:43.086822+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:20:47.431800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.089525+0000","last_change":"2023-10-26T01:30:41.100835+0000","last_active":"2023-10-26T01:30:42.089525+0000","last_peered":"2023-10-26T01:30:42.089525+0000","last_clean":"2023-10-26T01:30:42.089525+0000","last_became_active":"2023-10-26T01:30:41.099708+0000","last_became_peered":"2023-10-26T01:30:41.099708+0000","last_unstale":"2023-10-26T01:30:42.089525+0000","last_undegraded":"2023-10-26T01:30:42.089525+0000","last_fullsized":"2023-10-26T01:30:42.089525+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:04:22.141930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.098337+0000","last_change":"2023-10-26T01:30:42.100857+0000","last_active":"2023-10-26T01:30:43.098337+0000","last_peered":"2023-10-26T01:30:43.098337+0000","last_clean":"2023-10-26T01:30:43.098337+0000","last_became_active":"2023-10-26T01:30:42.100545+0000","last_became_peered":"2023-10-26T01:30:42.100545+0000","last_unstale":"2023-10-26T01:30:43.098337+0000","last_undegraded":"2023-10-26T01:30:43.098337+0000","last_fullsized":"2023-10-26T01:30:43.098337+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:13:33.838018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.114399+0000","last_change":"2023-10-26T01:30:41.095998+0000","last_active":"2023-10-26T01:30:42.114399+0000","last_peered":"2023-10-26T01:30:42.114399+0000","last_clean":"2023-10-26T01:30:42.114399+0000","last_became_active":"2023-10-26T01:30:41.095494+0000","last_became_peered":"2023-10-26T01:30:41.095494+0000","last_unstale":"2023-10-26T01:30:42.114399+0000","last_undegraded":"2023-10-26T01:30:42.114399+0000","last_fullsized":"2023-10-26T01:30:42.114399+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:54:04.970133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.089073+0000","last_change":"2023-10-26T01:30:42.115510+0000","last_active":"2023-10-26T01:30:43.089073+0000","last_peered":"2023-10-26T01:30:43.089073+0000","last_clean":"2023-10-26T01:30:43.089073+0000","last_became_active":"2023-10-26T01:30:42.113552+0000","last_became_peered":"2023-10-26T01:30:42.113552+0000","last_unstale":"2023-10-26T01:30:43.089073+0000","last_undegraded":"2023-10-26T01:30:43.089073+0000","last_fullsized":"2023-10-26T01:30:43.089073+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:32:08.387966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.110967+0000","last_change":"2023-10-26T01:30:41.093200+0000","last_active":"2023-10-26T01:30:42.110967+0000","last_peered":"2023-10-26T01:30:42.110967+0000","last_clean":"2023-10-26T01:30:42.110967+0000","last_became_active":"2023-10-26T01:30:41.092917+0000","last_became_peered":"2023-10-26T01:30:41.092917+0000","last_unstale":"2023-10-26T01:30:42.110967+0000","last_undegraded":"2023-10-26T01:30:42.110967+0000","last_fullsized":"2023-10-26T01:30:42.110967+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:54:04.970133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.087143+0000","last_change":"2023-10-26T01:30:42.113216+0000","last_active":"2023-10-26T01:30:43.087143+0000","last_peered":"2023-10-26T01:30:43.087143+0000","last_clean":"2023-10-26T01:30:43.087143+0000","last_became_active":"2023-10-26T01:30:42.112815+0000","last_became_peered":"2023-10-26T01:30:42.112815+0000","last_unstale":"2023-10-26T01:30:43.087143+0000","last_undegraded":"2023-10-26T01:30:43.087143+0000","last_fullsized":"2023-10-26T01:30:43.087143+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:55:51.116465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.114880+0000","last_change":"2023-10-26T01:30:41.102365+0000","last_active":"2023-10-26T01:30:42.114880+0000","last_peered":"2023-10-26T01:30:42.114880+0000","last_clean":"2023-10-26T01:30:42.114880+0000","last_became_active":"2023-10-26T01:30:41.102135+0000","last_became_peered":"2023-10-26T01:30:41.102135+0000","last_unstale":"2023-10-26T01:30:42.114880+0000","last_undegraded":"2023-10-26T01:30:42.114880+0000","last_fullsized":"2023-10-26T01:30:42.114880+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:08:20.547052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.086427+0000","last_change":"2023-10-26T01:30:42.108605+0000","last_active":"2023-10-26T01:30:43.086427+0000","last_peered":"2023-10-26T01:30:43.086427+0000","last_clean":"2023-10-26T01:30:43.086427+0000","last_became_active":"2023-10-26T01:30:42.107907+0000","last_became_peered":"2023-10-26T01:30:42.107907+0000","last_unstale":"2023-10-26T01:30:43.086427+0000","last_undegraded":"2023-10-26T01:30:43.086427+0000","last_fullsized":"2023-10-26T01:30:43.086427+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T01:55:28.470913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.105745+0000","last_change":"2023-10-26T01:30:41.093456+0000","last_active":"2023-10-26T01:30:42.105745+0000","last_peered":"2023-10-26T01:30:42.105745+0000","last_clean":"2023-10-26T01:30:42.105745+0000","last_became_active":"2023-10-26T01:30:41.092811+0000","last_became_peered":"2023-10-26T01:30:41.092811+0000","last_unstale":"2023-10-26T01:30:42.105745+0000","last_undegraded":"2023-10-26T01:30:42.105745+0000","last_fullsized":"2023-10-26T01:30:42.105745+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:04:22.141930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.099118+0000","last_change":"2023-10-26T01:30:42.097121+0000","last_active":"2023-10-26T01:30:43.099118+0000","last_peered":"2023-10-26T01:30:43.099118+0000","last_clean":"2023-10-26T01:30:43.099118+0000","last_became_active":"2023-10-26T01:30:42.096784+0000","last_became_peered":"2023-10-26T01:30:42.096784+0000","last_unstale":"2023-10-26T01:30:43.099118+0000","last_undegraded":"2023-10-26T01:30:43.099118+0000","last_fullsized":"2023-10-26T01:30:43.099118+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:28:22.384005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.083700+0000","last_change":"2023-10-26T01:30:41.106632+0000","last_active":"2023-10-26T01:30:42.083700+0000","last_peered":"2023-10-26T01:30:42.083700+0000","last_clean":"2023-10-26T01:30:42.083700+0000","last_became_active":"2023-10-26T01:30:41.106360+0000","last_became_peered":"2023-10-26T01:30:41.106360+0000","last_unstale":"2023-10-26T01:30:42.083700+0000","last_undegraded":"2023-10-26T01:30:42.083700+0000","last_fullsized":"2023-10-26T01:30:42.083700+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:40:15.199455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.086951+0000","last_change":"2023-10-26T01:30:42.109475+0000","last_active":"2023-10-26T01:30:43.086951+0000","last_peered":"2023-10-26T01:30:43.086951+0000","last_clean":"2023-10-26T01:30:43.086951+0000","last_became_active":"2023-10-26T01:30:42.109232+0000","last_became_peered":"2023-10-26T01:30:42.109232+0000","last_unstale":"2023-10-26T01:30:43.086951+0000","last_undegraded":"2023-10-26T01:30:43.086951+0000","last_fullsized":"2023-10-26T01:30:43.086951+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:26:01.898201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":165,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.096979+0000","last_change":"2023-10-26T01:30:34.883484+0000","last_active":"2023-10-26T01:30:42.096979+0000","last_peered":"2023-10-26T01:30:42.096979+0000","last_clean":"2023-10-26T01:30:42.096979+0000","last_became_active":"2023-10-26T01:30:34.883071+0000","last_became_peered":"2023-10-26T01:30:34.883071+0000","last_unstale":"2023-10-26T01:30:42.096979+0000","last_undegraded":"2023-10-26T01:30:42.096979+0000","last_fullsized":"2023-10-26T01:30:42.096979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:32.890096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:32.890096+0000","last_clean_scrub_stamp":"2023-10-26T01:30:32.890096+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:22:57.937891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.090888+0000","last_change":"2023-10-26T01:30:41.105687+0000","last_active":"2023-10-26T01:30:42.090888+0000","last_peered":"2023-10-26T01:30:42.090888+0000","last_clean":"2023-10-26T01:30:42.090888+0000","last_became_active":"2023-10-26T01:30:41.105235+0000","last_became_peered":"2023-10-26T01:30:41.105235+0000","last_unstale":"2023-10-26T01:30:42.090888+0000","last_undegraded":"2023-10-26T01:30:42.090888+0000","last_fullsized":"2023-10-26T01:30:42.090888+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:13:32.821792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-10-26T01:30:41.094673+0000","last_change":"2023-10-26T01:30:41.094673+0000","last_active":"2023-10-26T01:30:41.060332+0000","last_peered":"2023-10-26T01:30:41.060332+0000","last_clean":"2023-10-26T01:30:41.060332+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:30:41.094673+0000","last_undegraded":"2023-10-26T01:30:41.094673+0000","last_fullsized":"2023-10-26T01:30:41.094673+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.096890+0000","last_change":"2023-10-26T01:30:41.094664+0000","last_active":"2023-10-26T01:30:42.096890+0000","last_peered":"2023-10-26T01:30:42.096890+0000","last_clean":"2023-10-26T01:30:42.096890+0000","last_became_active":"2023-10-26T01:30:41.092745+0000","last_became_peered":"2023-10-26T01:30:41.092745+0000","last_unstale":"2023-10-26T01:30:42.096890+0000","last_undegraded":"2023-10-26T01:30:42.096890+0000","last_fullsized":"2023-10-26T01:30:42.096890+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:45:22.045243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.140108+0000","last_change":"2023-10-26T01:30:42.107538+0000","last_active":"2023-10-26T01:30:43.140108+0000","last_peered":"2023-10-26T01:30:43.140108+0000","last_clean":"2023-10-26T01:30:43.140108+0000","last_became_active":"2023-10-26T01:30:42.107258+0000","last_became_peered":"2023-10-26T01:30:42.107258+0000","last_unstale":"2023-10-26T01:30:43.140108+0000","last_undegraded":"2023-10-26T01:30:43.140108+0000","last_fullsized":"2023-10-26T01:30:43.140108+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:49:12.753504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.083749+0000","last_change":"2023-10-26T01:30:41.099543+0000","last_active":"2023-10-26T01:30:42.083749+0000","last_peered":"2023-10-26T01:30:42.083749+0000","last_clean":"2023-10-26T01:30:42.083749+0000","last_became_active":"2023-10-26T01:30:41.099277+0000","last_became_peered":"2023-10-26T01:30:41.099277+0000","last_unstale":"2023-10-26T01:30:42.083749+0000","last_undegraded":"2023-10-26T01:30:42.083749+0000","last_fullsized":"2023-10-26T01:30:42.083749+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:04:22.141930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.140694+0000","last_change":"2023-10-26T01:30:42.085871+0000","last_active":"2023-10-26T01:30:43.140694+0000","last_peered":"2023-10-26T01:30:43.140694+0000","last_clean":"2023-10-26T01:30:43.140694+0000","last_became_active":"2023-10-26T01:30:42.085409+0000","last_became_peered":"2023-10-26T01:30:42.085409+0000","last_unstale":"2023-10-26T01:30:43.140694+0000","last_undegraded":"2023-10-26T01:30:43.140694+0000","last_fullsized":"2023-10-26T01:30:43.140694+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:17:59.186810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.088251+0000","last_change":"2023-10-26T01:30:41.107609+0000","last_active":"2023-10-26T01:30:42.088251+0000","last_peered":"2023-10-26T01:30:42.088251+0000","last_clean":"2023-10-26T01:30:42.088251+0000","last_became_active":"2023-10-26T01:30:41.107189+0000","last_became_peered":"2023-10-26T01:30:41.107189+0000","last_unstale":"2023-10-26T01:30:42.088251+0000","last_undegraded":"2023-10-26T01:30:42.088251+0000","last_fullsized":"2023-10-26T01:30:42.088251+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:33:57.752079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.088414+0000","last_change":"2023-10-26T01:30:42.105495+0000","last_active":"2023-10-26T01:30:43.088414+0000","last_peered":"2023-10-26T01:30:43.088414+0000","last_clean":"2023-10-26T01:30:43.088414+0000","last_became_active":"2023-10-26T01:30:42.105247+0000","last_became_peered":"2023-10-26T01:30:42.105247+0000","last_unstale":"2023-10-26T01:30:43.088414+0000","last_undegraded":"2023-10-26T01:30:43.088414+0000","last_fullsized":"2023-10-26T01:30:43.088414+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:24:17.679373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.088367+0000","last_change":"2023-10-26T01:30:41.096866+0000","last_active":"2023-10-26T01:30:42.088367+0000","last_peered":"2023-10-26T01:30:42.088367+0000","last_clean":"2023-10-26T01:30:42.088367+0000","last_became_active":"2023-10-26T01:30:41.096255+0000","last_became_peered":"2023-10-26T01:30:41.096255+0000","last_unstale":"2023-10-26T01:30:42.088367+0000","last_undegraded":"2023-10-26T01:30:42.088367+0000","last_fullsized":"2023-10-26T01:30:42.088367+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:29:17.626922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.099320+0000","last_change":"2023-10-26T01:30:42.102298+0000","last_active":"2023-10-26T01:30:43.099320+0000","last_peered":"2023-10-26T01:30:43.099320+0000","last_clean":"2023-10-26T01:30:43.099320+0000","last_became_active":"2023-10-26T01:30:42.097872+0000","last_became_peered":"2023-10-26T01:30:42.097872+0000","last_unstale":"2023-10-26T01:30:43.099320+0000","last_undegraded":"2023-10-26T01:30:43.099320+0000","last_fullsized":"2023-10-26T01:30:43.099320+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:07:06.450331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.096762+0000","last_change":"2023-10-26T01:30:41.096310+0000","last_active":"2023-10-26T01:30:42.096762+0000","last_peered":"2023-10-26T01:30:42.096762+0000","last_clean":"2023-10-26T01:30:42.096762+0000","last_became_active":"2023-10-26T01:30:41.095929+0000","last_became_peered":"2023-10-26T01:30:41.095929+0000","last_unstale":"2023-10-26T01:30:42.096762+0000","last_undegraded":"2023-10-26T01:30:42.096762+0000","last_fullsized":"2023-10-26T01:30:42.096762+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:31:39.970444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.140793+0000","last_change":"2023-10-26T01:30:42.089331+0000","last_active":"2023-10-26T01:30:43.140793+0000","last_peered":"2023-10-26T01:30:43.140793+0000","last_clean":"2023-10-26T01:30:43.140793+0000","last_became_active":"2023-10-26T01:30:42.088901+0000","last_became_peered":"2023-10-26T01:30:42.088901+0000","last_unstale":"2023-10-26T01:30:43.140793+0000","last_undegraded":"2023-10-26T01:30:43.140793+0000","last_fullsized":"2023-10-26T01:30:43.140793+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:53:57.632838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.082315+0000","last_change":"2023-10-26T01:30:41.102617+0000","last_active":"2023-10-26T01:30:42.082315+0000","last_peered":"2023-10-26T01:30:42.082315+0000","last_clean":"2023-10-26T01:30:42.082315+0000","last_became_active":"2023-10-26T01:30:41.102376+0000","last_became_peered":"2023-10-26T01:30:41.102376+0000","last_unstale":"2023-10-26T01:30:42.082315+0000","last_undegraded":"2023-10-26T01:30:42.082315+0000","last_fullsized":"2023-10-26T01:30:42.082315+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:24:16.663147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-10-26T01:30:41.095125+0000","last_change":"2023-10-26T01:30:41.095125+0000","last_active":"2023-10-26T01:30:41.060332+0000","last_peered":"2023-10-26T01:30:41.060332+0000","last_clean":"2023-10-26T01:30:41.060332+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:30:41.095125+0000","last_undegraded":"2023-10-26T01:30:41.095125+0000","last_fullsized":"2023-10-26T01:30:41.095125+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.112282+0000","last_change":"2023-10-26T01:30:41.102404+0000","last_active":"2023-10-26T01:30:42.112282+0000","last_peered":"2023-10-26T01:30:42.112282+0000","last_clean":"2023-10-26T01:30:42.112282+0000","last_became_active":"2023-10-26T01:30:41.102165+0000","last_became_peered":"2023-10-26T01:30:41.102165+0000","last_unstale":"2023-10-26T01:30:42.112282+0000","last_undegraded":"2023-10-26T01:30:42.112282+0000","last_fullsized":"2023-10-26T01:30:42.112282+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:08:20.547052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.139422+0000","last_change":"2023-10-26T01:30:42.103630+0000","last_active":"2023-10-26T01:30:43.139422+0000","last_peered":"2023-10-26T01:30:43.139422+0000","last_clean":"2023-10-26T01:30:43.139422+0000","last_became_active":"2023-10-26T01:30:42.099458+0000","last_became_peered":"2023-10-26T01:30:42.099458+0000","last_unstale":"2023-10-26T01:30:43.139422+0000","last_undegraded":"2023-10-26T01:30:43.139422+0000","last_fullsized":"2023-10-26T01:30:43.139422+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:05:31.590291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.097650+0000","last_change":"2023-10-26T01:30:41.097301+0000","last_active":"2023-10-26T01:30:42.097650+0000","last_peered":"2023-10-26T01:30:42.097650+0000","last_clean":"2023-10-26T01:30:42.097650+0000","last_became_active":"2023-10-26T01:30:41.096866+0000","last_became_peered":"2023-10-26T01:30:41.096866+0000","last_unstale":"2023-10-26T01:30:42.097650+0000","last_undegraded":"2023-10-26T01:30:42.097650+0000","last_fullsized":"2023-10-26T01:30:42.097650+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:05:30.574065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.089679+0000","last_change":"2023-10-26T01:30:42.087483+0000","last_active":"2023-10-26T01:30:43.089679+0000","last_peered":"2023-10-26T01:30:43.089679+0000","last_clean":"2023-10-26T01:30:43.089679+0000","last_became_active":"2023-10-26T01:30:42.087234+0000","last_became_peered":"2023-10-26T01:30:42.087234+0000","last_unstale":"2023-10-26T01:30:43.089679+0000","last_undegraded":"2023-10-26T01:30:43.089679+0000","last_fullsized":"2023-10-26T01:30:43.089679+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:46:06.943389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.088240+0000","last_change":"2023-10-26T01:30:41.097032+0000","last_active":"2023-10-26T01:30:42.088240+0000","last_peered":"2023-10-26T01:30:42.088240+0000","last_clean":"2023-10-26T01:30:42.088240+0000","last_became_active":"2023-10-26T01:30:41.096591+0000","last_became_peered":"2023-10-26T01:30:41.096591+0000","last_unstale":"2023-10-26T01:30:42.088240+0000","last_undegraded":"2023-10-26T01:30:42.088240+0000","last_fullsized":"2023-10-26T01:30:42.088240+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:33:57.752079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-10-26T01:30:41.094665+0000","last_change":"2023-10-26T01:30:41.094665+0000","last_active":"2023-10-26T01:30:41.060332+0000","last_peered":"2023-10-26T01:30:41.060332+0000","last_clean":"2023-10-26T01:30:41.060332+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:30:41.094665+0000","last_undegraded":"2023-10-26T01:30:41.094665+0000","last_fullsized":"2023-10-26T01:30:41.094665+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.084678+0000","last_change":"2023-10-26T01:30:41.101587+0000","last_active":"2023-10-26T01:30:42.084678+0000","last_peered":"2023-10-26T01:30:42.084678+0000","last_clean":"2023-10-26T01:30:42.084678+0000","last_became_active":"2023-10-26T01:30:41.101325+0000","last_became_peered":"2023-10-26T01:30:41.101325+0000","last_unstale":"2023-10-26T01:30:42.084678+0000","last_undegraded":"2023-10-26T01:30:42.084678+0000","last_fullsized":"2023-10-26T01:30:42.084678+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:49:42.452332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-10-26T01:30:41.090145+0000","last_change":"2023-10-26T01:30:41.090145+0000","last_active":"2023-10-26T01:30:41.060332+0000","last_peered":"2023-10-26T01:30:41.060332+0000","last_clean":"2023-10-26T01:30:41.060332+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:30:41.090145+0000","last_undegraded":"2023-10-26T01:30:41.090145+0000","last_fullsized":"2023-10-26T01:30:41.090145+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.114450+0000","last_change":"2023-10-26T01:30:41.095274+0000","last_active":"2023-10-26T01:30:42.114450+0000","last_peered":"2023-10-26T01:30:42.114450+0000","last_clean":"2023-10-26T01:30:42.114450+0000","last_became_active":"2023-10-26T01:30:41.092922+0000","last_became_peered":"2023-10-26T01:30:41.092922+0000","last_unstale":"2023-10-26T01:30:42.114450+0000","last_undegraded":"2023-10-26T01:30:42.114450+0000","last_fullsized":"2023-10-26T01:30:42.114450+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:33:12.599402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.098208+0000","last_change":"2023-10-26T01:30:42.105139+0000","last_active":"2023-10-26T01:30:43.098208+0000","last_peered":"2023-10-26T01:30:43.098208+0000","last_clean":"2023-10-26T01:30:43.098208+0000","last_became_active":"2023-10-26T01:30:42.104913+0000","last_became_peered":"2023-10-26T01:30:42.104913+0000","last_unstale":"2023-10-26T01:30:43.098208+0000","last_undegraded":"2023-10-26T01:30:43.098208+0000","last_fullsized":"2023-10-26T01:30:43.098208+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:17:59.186810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.099380+0000","last_change":"2023-10-26T01:30:42.106926+0000","last_active":"2023-10-26T01:30:43.099380+0000","last_peered":"2023-10-26T01:30:43.099380+0000","last_clean":"2023-10-26T01:30:43.099380+0000","last_became_active":"2023-10-26T01:30:42.106692+0000","last_became_peered":"2023-10-26T01:30:42.106692+0000","last_unstale":"2023-10-26T01:30:43.099380+0000","last_undegraded":"2023-10-26T01:30:43.099380+0000","last_fullsized":"2023-10-26T01:30:43.099380+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:20:47.431800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.110477+0000","last_change":"2023-10-26T01:30:41.105282+0000","last_active":"2023-10-26T01:30:42.110477+0000","last_peered":"2023-10-26T01:30:42.110477+0000","last_clean":"2023-10-26T01:30:42.110477+0000","last_became_active":"2023-10-26T01:30:41.104553+0000","last_became_peered":"2023-10-26T01:30:41.104553+0000","last_unstale":"2023-10-26T01:30:42.110477+0000","last_undegraded":"2023-10-26T01:30:42.110477+0000","last_fullsized":"2023-10-26T01:30:42.110477+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:29:17.626922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.100164+0000","last_change":"2023-10-26T01:30:42.098532+0000","last_active":"2023-10-26T01:30:43.100164+0000","last_peered":"2023-10-26T01:30:43.100164+0000","last_clean":"2023-10-26T01:30:43.100164+0000","last_became_active":"2023-10-26T01:30:42.098301+0000","last_became_peered":"2023-10-26T01:30:42.098301+0000","last_unstale":"2023-10-26T01:30:43.100164+0000","last_undegraded":"2023-10-26T01:30:43.100164+0000","last_fullsized":"2023-10-26T01:30:43.100164+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:55:51.116465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.113350+0000","last_change":"2023-10-26T01:30:41.101188+0000","last_active":"2023-10-26T01:30:42.113350+0000","last_peered":"2023-10-26T01:30:42.113350+0000","last_clean":"2023-10-26T01:30:42.113350+0000","last_became_active":"2023-10-26T01:30:41.100977+0000","last_became_peered":"2023-10-26T01:30:41.100977+0000","last_unstale":"2023-10-26T01:30:42.113350+0000","last_undegraded":"2023-10-26T01:30:42.113350+0000","last_fullsized":"2023-10-26T01:30:42.113350+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:13:32.821792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.139789+0000","last_change":"2023-10-26T01:30:42.105438+0000","last_active":"2023-10-26T01:30:43.139789+0000","last_peered":"2023-10-26T01:30:43.139789+0000","last_clean":"2023-10-26T01:30:43.139789+0000","last_became_active":"2023-10-26T01:30:42.105168+0000","last_became_peered":"2023-10-26T01:30:42.105168+0000","last_unstale":"2023-10-26T01:30:43.139789+0000","last_undegraded":"2023-10-26T01:30:43.139789+0000","last_fullsized":"2023-10-26T01:30:43.139789+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:53:57.632838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-26T01:30:41.089058+0000","last_change":"2023-10-26T01:30:41.088679+0000","last_active":"2023-10-26T01:30:41.089058+0000","last_peered":"2023-10-26T01:30:41.089058+0000","last_clean":"2023-10-26T01:30:41.089058+0000","last_became_active":"2023-10-26T01:30:41.086354+0000","last_became_peered":"2023-10-26T01:30:41.086354+0000","last_unstale":"2023-10-26T01:30:41.089058+0000","last_undegraded":"2023-10-26T01:30:41.089058+0000","last_fullsized":"2023-10-26T01:30:41.089058+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:49:35.491976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2023-10-26T01:30:41.096096+0000","last_change":"2023-10-26T01:30:41.096096+0000","last_active":"2023-10-26T01:30:41.060332+0000","last_peered":"2023-10-26T01:30:41.060332+0000","last_clean":"2023-10-26T01:30:41.060332+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:30:41.096096+0000","last_undegraded":"2023-10-26T01:30:41.096096+0000","last_fullsized":"2023-10-26T01:30:41.096096+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.113466+0000","last_change":"2023-10-26T01:30:41.097404+0000","last_active":"2023-10-26T01:30:42.113466+0000","last_peered":"2023-10-26T01:30:42.113466+0000","last_clean":"2023-10-26T01:30:42.113466+0000","last_became_active":"2023-10-26T01:30:41.097074+0000","last_became_peered":"2023-10-26T01:30:41.097074+0000","last_unstale":"2023-10-26T01:30:42.113466+0000","last_undegraded":"2023-10-26T01:30:42.113466+0000","last_fullsized":"2023-10-26T01:30:42.113466+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:31:39.970444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":36,"ondisk_log_size":36,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":14,"seq":60129542148,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":54,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195258,"statfs":{"total":200000000,"available":199944205,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85999999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1990000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.462}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94299999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.911}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.347}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.514}]}]},{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":650,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194662,"statfs":{"total":200000000,"available":199333905,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94399999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0569999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.333}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2050000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.014}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.101}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4970000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":74,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195238,"statfs":{"total":200000000,"available":199923857,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0089999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.393}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4319999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.256}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4790000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.131}]}]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":73,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195239,"statfs":{"total":200000000,"available":199924765,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1990000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.367}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4399999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7190000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.117}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.639}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5009999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":73,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195238,"statfs":{"total":200000000,"available":199924729,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.751}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.635}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5169999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.276}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4850000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4490000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7430000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":13,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":73,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195239,"statfs":{"total":200000000,"available":199924855,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1000000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.171}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68400000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2210000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7869999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.597}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9039999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":73,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195238,"statfs":{"total":200000000,"available":199924695,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79900000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95599999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6180000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4470000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3069999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.391}]}]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":650,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194661,"statfs":{"total":200000000,"available":199333849,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.03}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.161}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.204}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2290000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0129999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.839}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3049999999999999}]}]}],"pool_statfs":[]}} 2023-10-26T01:30:47.678 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.38 is not active+clean 2023-10-26T01:30:47.679 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2023-10-26T01:30:41.100533+0000', 'last_change': '2023-10-26T01:30:41.100533+0000', 'last_active': '2023-10-26T01:30:41.060332+0000', 'last_peered': '2023-10-26T01:30:41.060332+0000', 'last_clean': '2023-10-26T01:30:41.060332+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T01:30:41.100533+0000', 'last_undegraded': '2023-10-26T01:30:41.100533+0000', 'last_fullsized': '2023-10-26T01:30:41.100533+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T01:30:41.060332+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T01:30:41.060332+0000', 'last_clean_scrub_stamp': '2023-10-26T01:30:41.060332+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-26T01:30:47.679 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.33 is not active+clean 2023-10-26T01:30:47.679 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2023-10-26T01:30:41.099897+0000', 'last_change': '2023-10-26T01:30:41.099897+0000', 'last_active': '2023-10-26T01:30:41.060332+0000', 'last_peered': '2023-10-26T01:30:41.060332+0000', 'last_clean': '2023-10-26T01:30:41.060332+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T01:30:41.099897+0000', 'last_undegraded': '2023-10-26T01:30:41.099897+0000', 'last_fullsized': '2023-10-26T01:30:41.099897+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T01:30:41.060332+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T01:30:41.060332+0000', 'last_clean_scrub_stamp': '2023-10-26T01:30:41.060332+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-26T01:30:47.680 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.29 is not active+clean 2023-10-26T01:30:47.680 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2023-10-26T01:30:41.098174+0000', 'last_change': '2023-10-26T01:30:41.098174+0000', 'last_active': '2023-10-26T01:30:41.060332+0000', 'last_peered': '2023-10-26T01:30:41.060332+0000', 'last_clean': '2023-10-26T01:30:41.060332+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T01:30:41.098174+0000', 'last_undegraded': '2023-10-26T01:30:41.098174+0000', 'last_fullsized': '2023-10-26T01:30:41.098174+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T01:30:41.060332+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T01:30:41.060332+0000', 'last_clean_scrub_stamp': '2023-10-26T01:30:41.060332+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-26T01:30:47.680 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.27 is not active+clean 2023-10-26T01:30:47.681 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2023-10-26T01:30:41.098206+0000', 'last_change': '2023-10-26T01:30:41.098206+0000', 'last_active': '2023-10-26T01:30:41.060332+0000', 'last_peered': '2023-10-26T01:30:41.060332+0000', 'last_clean': '2023-10-26T01:30:41.060332+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T01:30:41.098206+0000', 'last_undegraded': '2023-10-26T01:30:41.098206+0000', 'last_fullsized': '2023-10-26T01:30:41.098206+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T01:30:41.060332+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T01:30:41.060332+0000', 'last_clean_scrub_stamp': '2023-10-26T01:30:41.060332+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-26T01:30:47.681 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.24 is not active+clean 2023-10-26T01:30:47.681 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2023-10-26T01:30:41.095233+0000', 'last_change': '2023-10-26T01:30:41.095233+0000', 'last_active': '2023-10-26T01:30:41.060332+0000', 'last_peered': '2023-10-26T01:30:41.060332+0000', 'last_clean': '2023-10-26T01:30:41.060332+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T01:30:41.095233+0000', 'last_undegraded': '2023-10-26T01:30:41.095233+0000', 'last_fullsized': '2023-10-26T01:30:41.095233+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T01:30:41.060332+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T01:30:41.060332+0000', 'last_clean_scrub_stamp': '2023-10-26T01:30:41.060332+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-26T01:30:47.682 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.18 is not active+clean 2023-10-26T01:30:47.682 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2023-10-26T01:30:41.097717+0000', 'last_change': '2023-10-26T01:30:41.097717+0000', 'last_active': '2023-10-26T01:30:41.060332+0000', 'last_peered': '2023-10-26T01:30:41.060332+0000', 'last_clean': '2023-10-26T01:30:41.060332+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T01:30:41.097717+0000', 'last_undegraded': '2023-10-26T01:30:41.097717+0000', 'last_fullsized': '2023-10-26T01:30:41.097717+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T01:30:41.060332+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T01:30:41.060332+0000', 'last_clean_scrub_stamp': '2023-10-26T01:30:41.060332+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-26T01:30:47.682 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2 is not active+clean 2023-10-26T01:30:47.683 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2023-10-26T01:30:41.094673+0000', 'last_change': '2023-10-26T01:30:41.094673+0000', 'last_active': '2023-10-26T01:30:41.060332+0000', 'last_peered': '2023-10-26T01:30:41.060332+0000', 'last_clean': '2023-10-26T01:30:41.060332+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T01:30:41.094673+0000', 'last_undegraded': '2023-10-26T01:30:41.094673+0000', 'last_fullsized': '2023-10-26T01:30:41.094673+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T01:30:41.060332+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T01:30:41.060332+0000', 'last_clean_scrub_stamp': '2023-10-26T01:30:41.060332+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-26T01:30:47.683 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.7 is not active+clean 2023-10-26T01:30:47.683 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2023-10-26T01:30:41.095125+0000', 'last_change': '2023-10-26T01:30:41.095125+0000', 'last_active': '2023-10-26T01:30:41.060332+0000', 'last_peered': '2023-10-26T01:30:41.060332+0000', 'last_clean': '2023-10-26T01:30:41.060332+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T01:30:41.095125+0000', 'last_undegraded': '2023-10-26T01:30:41.095125+0000', 'last_fullsized': '2023-10-26T01:30:41.095125+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T01:30:41.060332+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T01:30:41.060332+0000', 'last_clean_scrub_stamp': '2023-10-26T01:30:41.060332+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-26T01:30:47.684 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.8 is not active+clean 2023-10-26T01:30:47.684 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2023-10-26T01:30:41.094665+0000', 'last_change': '2023-10-26T01:30:41.094665+0000', 'last_active': '2023-10-26T01:30:41.060332+0000', 'last_peered': '2023-10-26T01:30:41.060332+0000', 'last_clean': '2023-10-26T01:30:41.060332+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T01:30:41.094665+0000', 'last_undegraded': '2023-10-26T01:30:41.094665+0000', 'last_fullsized': '2023-10-26T01:30:41.094665+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T01:30:41.060332+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T01:30:41.060332+0000', 'last_clean_scrub_stamp': '2023-10-26T01:30:41.060332+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-26T01:30:47.684 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.b is not active+clean 2023-10-26T01:30:47.685 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2023-10-26T01:30:41.090145+0000', 'last_change': '2023-10-26T01:30:41.090145+0000', 'last_active': '2023-10-26T01:30:41.060332+0000', 'last_peered': '2023-10-26T01:30:41.060332+0000', 'last_clean': '2023-10-26T01:30:41.060332+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T01:30:41.090145+0000', 'last_undegraded': '2023-10-26T01:30:41.090145+0000', 'last_fullsized': '2023-10-26T01:30:41.090145+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T01:30:41.060332+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T01:30:41.060332+0000', 'last_clean_scrub_stamp': '2023-10-26T01:30:41.060332+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-26T01:30:47.685 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1e is not active+clean 2023-10-26T01:30:47.685 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2023-10-26T01:30:41.096096+0000', 'last_change': '2023-10-26T01:30:41.096096+0000', 'last_active': '2023-10-26T01:30:41.060332+0000', 'last_peered': '2023-10-26T01:30:41.060332+0000', 'last_clean': '2023-10-26T01:30:41.060332+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T01:30:41.096096+0000', 'last_undegraded': '2023-10-26T01:30:41.096096+0000', 'last_fullsized': '2023-10-26T01:30:41.096096+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T01:30:41.060332+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T01:30:41.060332+0000', 'last_clean_scrub_stamp': '2023-10-26T01:30:41.060332+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-26T01:30:47.686 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-26T01:30:47.691 INFO:tasks.ceph.mds.b.smithi170.stderr:2023-10-26T01:30:47.558+0000 7fa2284b1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-26T01:30:47.901 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:47.897+0000 7fc1522da640 1 -- 172.21.15.169:0/2519961125 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc14c1593e0 msgr2=0x7fc14c0ac200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:47.901 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:47.897+0000 7fc1522da640 1 --2- 172.21.15.169:0/2519961125 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc14c1593e0 0x7fc14c0ac200 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc13c005fd0 tx=0x7fc13c02c0e0 comp rx=0 tx=0).stop 2023-10-26T01:30:47.901 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:47.898+0000 7fc1522da640 1 -- 172.21.15.169:0/2519961125 shutdown_connections 2023-10-26T01:30:47.901 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:47.898+0000 7fc1522da640 1 --2- 172.21.15.169:0/2519961125 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc14c1593e0 0x7fc14c0ac200 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:47.901 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:47.898+0000 7fc1522da640 1 -- 172.21.15.169:0/2519961125 >> 172.21.15.169:0/2519961125 conn(0x7fc14c069930 msgr2=0x7fc14c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:47.902 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:47.898+0000 7fc1522da640 1 -- 172.21.15.169:0/2519961125 shutdown_connections 2023-10-26T01:30:47.902 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:47.898+0000 7fc1522da640 1 -- 172.21.15.169:0/2519961125 wait complete. 2023-10-26T01:30:47.902 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:47.899+0000 7fc1522da640 1 Processor -- start 2023-10-26T01:30:47.903 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:47.900+0000 7fc1522da640 1 -- start start 2023-10-26T01:30:47.905 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:47.900+0000 7fc1522da640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc14c0b0c80 0x7fc14c158440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:47.905 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:47.900+0000 7fc1522da640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc14c1593e0 0x7fc14c14d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:47.905 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:47.901+0000 7fc1522da640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc14c1597c0 0x7fc14c1e61a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:47.905 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:47.901+0000 7fc1522da640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc14c156510 con 0x7fc14c1593e0 2023-10-26T01:30:47.905 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:47.901+0000 7fc1522da640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc14c14d880 con 0x7fc14c1597c0 2023-10-26T01:30:47.906 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:47.901+0000 7fc1522da640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc14c155940 con 0x7fc14c0b0c80 2023-10-26T01:30:47.906 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:47.901+0000 7fc14b7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc14c1593e0 0x7fc14c14d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:47.906 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:47.901+0000 7fc14b7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc14c1593e0 0x7fc14c14d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:33996/0 (socket says 172.21.15.169:33996) 2023-10-26T01:30:47.906 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:47.901+0000 7fc14b7fe640 1 -- 172.21.15.169:0/517439337 learned_addr learned my addr 172.21.15.169:0/517439337 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:30:47.906 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:47.901+0000 7fc150877640 1 --2- 172.21.15.169:0/517439337 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc14c1597c0 0x7fc14c1e61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:47.907 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:47.901+0000 7fc14b7fe640 1 -- 172.21.15.169:0/517439337 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc14c0b0c80 msgr2=0x7fc14c158440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:47.907 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:47.901+0000 7fc14b7fe640 1 --2- 172.21.15.169:0/517439337 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc14c0b0c80 0x7fc14c158440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:47.907 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:47.901+0000 7fc14b7fe640 1 -- 172.21.15.169:0/517439337 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc14c1597c0 msgr2=0x7fc14c1e61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:47.907 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:47.901+0000 7fc14b7fe640 1 --2- 172.21.15.169:0/517439337 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc14c1597c0 0x7fc14c1e61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:47.907 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:47.901+0000 7fc14b7fe640 1 -- 172.21.15.169:0/517439337 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc13c009840 con 0x7fc14c1593e0 2023-10-26T01:30:47.908 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:47.901+0000 7fc150877640 1 --2- 172.21.15.169:0/517439337 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc14c1597c0 0x7fc14c1e61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:30:47.908 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:47.901+0000 7fc14b7fe640 1 --2- 172.21.15.169:0/517439337 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc14c1593e0 0x7fc14c14d340 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc13400c410 tx=0x7fc13400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:47.908 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:47.902+0000 7fc148ff9640 1 -- 172.21.15.169:0/517439337 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1340035f0 con 0x7fc14c1593e0 2023-10-26T01:30:47.908 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:47.902+0000 7fc148ff9640 1 -- 172.21.15.169:0/517439337 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc134003770 con 0x7fc14c1593e0 2023-10-26T01:30:47.908 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:47.902+0000 7fc1522da640 1 -- 172.21.15.169:0/517439337 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc14c155be0 con 0x7fc14c1593e0 2023-10-26T01:30:47.909 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:47.902+0000 7fc148ff9640 1 -- 172.21.15.169:0/517439337 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc134003b80 con 0x7fc14c1593e0 2023-10-26T01:30:47.909 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:47.902+0000 7fc1522da640 1 -- 172.21.15.169:0/517439337 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc14c1561a0 con 0x7fc14c1593e0 2023-10-26T01:30:47.910 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:47.903+0000 7fc1522da640 1 -- 172.21.15.169:0/517439337 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc14c156360 con 0x7fc14c1593e0 2023-10-26T01:30:47.910 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:47.904+0000 7fc148ff9640 1 -- 172.21.15.169:0/517439337 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fc134003ce0 con 0x7fc14c1593e0 2023-10-26T01:30:47.910 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:47.905+0000 7fc148ff9640 1 --2- 172.21.15.169:0/517439337 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc12c06ae40 0x7fc12c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:47.910 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:47.905+0000 7fc148ff9640 1 -- 172.21.15.169:0/517439337 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc134088e50 con 0x7fc14c1593e0 2023-10-26T01:30:47.911 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:47.906+0000 7fc14bfff640 1 --2- 172.21.15.169:0/517439337 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc12c06ae40 0x7fc12c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:47.911 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:47.907+0000 7fc14bfff640 1 --2- 172.21.15.169:0/517439337 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc12c06ae40 0x7fc12c06d300 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc13c035000 tx=0x7fc13c004a40 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:47.912 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:47.910+0000 7fc148ff9640 1 -- 172.21.15.169:0/517439337 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc134053e60 con 0x7fc14c1593e0 2023-10-26T01:30:48.172 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:48.170+0000 7fc1522da640 1 -- 172.21.15.169:0/517439337 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fc14c04db10 con 0x7fc14c1593e0 2023-10-26T01:30:48.173 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:48.170+0000 7fc148ff9640 1 -- 172.21.15.169:0/517439337 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1113 (secure 0 0 0) 0x7fc134017020 con 0x7fc14c1593e0 2023-10-26T01:30:48.173 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:30:48.173 INFO:teuthology.orchestra.run.smithi169.stdout:{"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":25,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":21,"num_osds":8,"num_up_osds":8,"osd_up_since":1698283831,"num_in_osds":8,"osd_in_since":1698283827,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":86},{"state_name":"creating+peering","count":11}],"num_pgs":97,"num_pools":3,"num_objects":23,"data_bytes":592778,"bytes_used":1765140,"bytes_avail":1598234860,"bytes_total":1600000000,"inactive_pgs_ratio":0.11340206116437912,"read_bytes_sec":1535,"write_bytes_sec":47089,"read_op_per_sec":1,"write_op_per_sec":9},"fsmap":{"epoch":7,"id":1,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":1,"rank":0,"name":"b","status":"up:active","gid":4338}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":1,"modified":"2023-10-26T01:30:22.898681+0000","services":{}},"progress_events":{}} 2023-10-26T01:30:48.176 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:48.173+0000 7fc1522da640 1 -- 172.21.15.169:0/517439337 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc12c06ae40 msgr2=0x7fc12c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:48.176 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:48.173+0000 7fc1522da640 1 --2- 172.21.15.169:0/517439337 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc12c06ae40 0x7fc12c06d300 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc13c035000 tx=0x7fc13c004a40 comp rx=0 tx=0).stop 2023-10-26T01:30:48.177 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:48.173+0000 7fc1522da640 1 -- 172.21.15.169:0/517439337 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc14c1593e0 msgr2=0x7fc14c14d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:48.177 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:48.173+0000 7fc1522da640 1 --2- 172.21.15.169:0/517439337 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc14c1593e0 0x7fc14c14d340 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc13400c410 tx=0x7fc13400c8d0 comp rx=0 tx=0).stop 2023-10-26T01:30:48.177 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:48.173+0000 7fc1522da640 1 -- 172.21.15.169:0/517439337 shutdown_connections 2023-10-26T01:30:48.177 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:48.173+0000 7fc1522da640 1 --2- 172.21.15.169:0/517439337 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc12c06ae40 0x7fc12c06d300 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:48.177 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:48.173+0000 7fc1522da640 1 --2- 172.21.15.169:0/517439337 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc14c1593e0 0x7fc14c14d340 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:48.178 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:48.173+0000 7fc1522da640 1 --2- 172.21.15.169:0/517439337 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc14c1597c0 0x7fc14c1e61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:48.178 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:48.173+0000 7fc1522da640 1 --2- 172.21.15.169:0/517439337 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc14c0b0c80 0x7fc14c158440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:48.178 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:48.173+0000 7fc1522da640 1 -- 172.21.15.169:0/517439337 >> 172.21.15.169:0/517439337 conn(0x7fc14c069930 msgr2=0x7fc14c0aa720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:48.178 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:48.174+0000 7fc1522da640 1 -- 172.21.15.169:0/517439337 shutdown_connections 2023-10-26T01:30:48.178 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:48.174+0000 7fc1522da640 1 -- 172.21.15.169:0/517439337 wait complete. 2023-10-26T01:30:48.193 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-10-26T01:30:48.193 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-26T01:30:48.401 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:48.398+0000 7ff50f30f640 1 -- 172.21.15.169:0/896576428 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff508159e10 msgr2=0x7ff50815a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:48.402 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:48.398+0000 7ff50f30f640 1 --2- 172.21.15.169:0/896576428 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff508159e10 0x7ff50815a280 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff500009830 tx=0x7ff50002bcd0 comp rx=0 tx=0).stop 2023-10-26T01:30:48.402 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:48.399+0000 7ff50f30f640 1 -- 172.21.15.169:0/896576428 shutdown_connections 2023-10-26T01:30:48.402 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:48.399+0000 7ff50f30f640 1 --2- 172.21.15.169:0/896576428 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff508159e10 0x7ff50815a280 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:48.402 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:48.399+0000 7ff50f30f640 1 --2- 172.21.15.169:0/896576428 >> v2:172.21.15.170:3300/0 conn(0x7ff50806ce20 0x7ff50814e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:48.403 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:48.399+0000 7ff50f30f640 1 -- 172.21.15.169:0/896576428 >> 172.21.15.169:0/896576428 conn(0x7ff508069930 msgr2=0x7ff508069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:48.403 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:48.399+0000 7ff50f30f640 1 -- 172.21.15.169:0/896576428 shutdown_connections 2023-10-26T01:30:48.403 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:48.399+0000 7ff50f30f640 1 -- 172.21.15.169:0/896576428 wait complete. 2023-10-26T01:30:48.405 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:48.400+0000 7ff50f30f640 1 Processor -- start 2023-10-26T01:30:48.405 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:48.401+0000 7ff50f30f640 1 -- start start 2023-10-26T01:30:48.405 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:48.401+0000 7ff50f30f640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff50806ce20 0x7ff508155040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:48.405 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:48.401+0000 7ff50f30f640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff508158850 0x7ff508071720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:48.405 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:48.401+0000 7ff50f30f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff508159e10 0x7ff508150d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:48.406 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:48.401+0000 7ff50f30f640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5081530b0 con 0x7ff508158850 2023-10-26T01:30:48.406 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:48.401+0000 7ff50f30f640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5080b0050 con 0x7ff508159e10 2023-10-26T01:30:48.406 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:48.401+0000 7ff50f30f640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff508155580 con 0x7ff50806ce20 2023-10-26T01:30:48.406 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:48.402+0000 7ff50c8aa640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff508158850 0x7ff508071720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:48.406 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:48.402+0000 7ff50c8aa640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff508158850 0x7ff508071720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:34020/0 (socket says 172.21.15.169:34020) 2023-10-26T01:30:48.407 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:48.402+0000 7ff50c8aa640 1 -- 172.21.15.169:0/2518864110 learned_addr learned my addr 172.21.15.169:0/2518864110 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:30:48.407 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:48.402+0000 7ff50d0ab640 1 --2- 172.21.15.169:0/2518864110 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff50806ce20 0x7ff508155040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:48.407 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:48.402+0000 7ff50d8ac640 1 --2- 172.21.15.169:0/2518864110 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff508159e10 0x7ff508150d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:48.407 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:48.402+0000 7ff50c8aa640 1 -- 172.21.15.169:0/2518864110 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff50806ce20 msgr2=0x7ff508155040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:48.407 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:48.402+0000 7ff50c8aa640 1 --2- 172.21.15.169:0/2518864110 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff50806ce20 0x7ff508155040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:48.408 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:48.402+0000 7ff50c8aa640 1 -- 172.21.15.169:0/2518864110 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff508159e10 msgr2=0x7ff508150d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:48.408 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:48.402+0000 7ff50c8aa640 1 --2- 172.21.15.169:0/2518864110 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff508159e10 0x7ff508150d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:48.408 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:48.402+0000 7ff50c8aa640 1 -- 172.21.15.169:0/2518864110 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff500009490 con 0x7ff508158850 2023-10-26T01:30:48.409 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:48.403+0000 7ff50c8aa640 1 --2- 172.21.15.169:0/2518864110 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff508158850 0x7ff508071720 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff4f800c500 tx=0x7ff4f800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:48.409 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:48.403+0000 7ff4fdffb640 1 -- 172.21.15.169:0/2518864110 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4f80117b0 con 0x7ff508158850 2023-10-26T01:30:48.410 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:48.403+0000 7ff4fdffb640 1 -- 172.21.15.169:0/2518864110 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff4f8011a00 con 0x7ff508158850 2023-10-26T01:30:48.410 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:48.403+0000 7ff4fdffb640 1 -- 172.21.15.169:0/2518864110 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4f8011e10 con 0x7ff508158850 2023-10-26T01:30:48.410 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:48.403+0000 7ff50f30f640 1 -- 172.21.15.169:0/2518864110 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5080af3d0 con 0x7ff508158850 2023-10-26T01:30:48.410 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:48.403+0000 7ff50f30f640 1 -- 172.21.15.169:0/2518864110 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5080af990 con 0x7ff508158850 2023-10-26T01:30:48.410 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:48.404+0000 7ff50f30f640 1 -- 172.21.15.169:0/2518864110 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff50806cc60 con 0x7ff508158850 2023-10-26T01:30:48.411 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:48.405+0000 7ff4fdffb640 1 -- 172.21.15.169:0/2518864110 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ff4f801c460 con 0x7ff508158850 2023-10-26T01:30:48.411 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:48.406+0000 7ff4fdffb640 1 --2- 172.21.15.169:0/2518864110 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff4ec06ae40 0x7ff4ec06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:48.411 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:48.407+0000 7ff4fdffb640 1 -- 172.21.15.169:0/2518864110 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff4f80897d0 con 0x7ff508158850 2023-10-26T01:30:48.411 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:48.407+0000 7ff50d0ab640 1 --2- 172.21.15.169:0/2518864110 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff4ec06ae40 0x7ff4ec06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:48.412 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:48.408+0000 7ff50d0ab640 1 --2- 172.21.15.169:0/2518864110 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff4ec06ae40 0x7ff4ec06d300 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff504005fd0 tx=0x7ff504009870 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:48.413 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:48.411+0000 7ff4fdffb640 1 -- 172.21.15.169:0/2518864110 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff4f8054890 con 0x7ff508158850 2023-10-26T01:30:48.597 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:48.594+0000 7ff50f30f640 1 -- 172.21.15.169:0/2518864110 --> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff5080b0190 con 0x7ff4ec06ae40 2023-10-26T01:30:48.617 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:48.614+0000 7ff4fdffb640 1 -- 172.21.15.169:0/2518864110 <== mgr.4112 v2:172.21.15.170:6832/1187156918 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+259986 (secure 0 0 0) 0x7ff5080b0190 con 0x7ff4ec06ae40 2023-10-26T01:30:48.620 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:30:48.622 INFO:teuthology.orchestra.run.smithi169.stderr:dumped all 2023-10-26T01:30:48.624 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:48.621+0000 7ff50f30f640 1 -- 172.21.15.169:0/2518864110 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff4ec06ae40 msgr2=0x7ff4ec06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:48.624 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:48.621+0000 7ff50f30f640 1 --2- 172.21.15.169:0/2518864110 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff4ec06ae40 0x7ff4ec06d300 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff504005fd0 tx=0x7ff504009870 comp rx=0 tx=0).stop 2023-10-26T01:30:48.625 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:48.621+0000 7ff50f30f640 1 -- 172.21.15.169:0/2518864110 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff508158850 msgr2=0x7ff508071720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:48.625 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:48.621+0000 7ff50f30f640 1 --2- 172.21.15.169:0/2518864110 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff508158850 0x7ff508071720 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff4f800c500 tx=0x7ff4f800c9c0 comp rx=0 tx=0).stop 2023-10-26T01:30:48.625 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:48.621+0000 7ff50f30f640 1 -- 172.21.15.169:0/2518864110 shutdown_connections 2023-10-26T01:30:48.625 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:48.621+0000 7ff50f30f640 1 --2- 172.21.15.169:0/2518864110 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff4ec06ae40 0x7ff4ec06d300 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:48.625 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:48.621+0000 7ff50f30f640 1 --2- 172.21.15.169:0/2518864110 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff508158850 0x7ff508071720 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:48.626 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:48.621+0000 7ff50f30f640 1 --2- 172.21.15.169:0/2518864110 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff508159e10 0x7ff508150d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:48.626 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:48.621+0000 7ff50f30f640 1 --2- 172.21.15.169:0/2518864110 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff50806ce20 0x7ff508155040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:48.626 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:48.621+0000 7ff50f30f640 1 -- 172.21.15.169:0/2518864110 >> 172.21.15.169:0/2518864110 conn(0x7ff508069930 msgr2=0x7ff50814d810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:48.626 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:48.621+0000 7ff50f30f640 1 -- 172.21.15.169:0/2518864110 shutdown_connections 2023-10-26T01:30:48.626 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:48.621+0000 7ff50f30f640 1 -- 172.21.15.169:0/2518864110 wait complete. 2023-10-26T01:30:48.641 INFO:teuthology.orchestra.run.smithi169.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-10-26T01:30:48.129889+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":1744,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560755,"statfs":{"total":1600000000,"available":1598213898,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.053557"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.086425+0000","last_change":"2023-10-26T01:30:42.112552+0000","last_active":"2023-10-26T01:30:43.086425+0000","last_peered":"2023-10-26T01:30:43.086425+0000","last_clean":"2023-10-26T01:30:43.086425+0000","last_became_active":"2023-10-26T01:30:42.111151+0000","last_became_peered":"2023-10-26T01:30:42.111151+0000","last_unstale":"2023-10-26T01:30:43.086425+0000","last_undegraded":"2023-10-26T01:30:43.086425+0000","last_fullsized":"2023-10-26T01:30:43.086425+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:28:22.384005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.141033+0000","last_change":"2023-10-26T01:30:42.102872+0000","last_active":"2023-10-26T01:30:43.141033+0000","last_peered":"2023-10-26T01:30:43.141033+0000","last_clean":"2023-10-26T01:30:43.141033+0000","last_became_active":"2023-10-26T01:30:42.100796+0000","last_became_peered":"2023-10-26T01:30:42.100796+0000","last_unstale":"2023-10-26T01:30:43.141033+0000","last_undegraded":"2023-10-26T01:30:43.141033+0000","last_fullsized":"2023-10-26T01:30:43.141033+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:49:12.753504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.139076+0000","last_change":"2023-10-26T01:30:42.103921+0000","last_active":"2023-10-26T01:30:43.139076+0000","last_peered":"2023-10-26T01:30:43.139076+0000","last_clean":"2023-10-26T01:30:43.139076+0000","last_became_active":"2023-10-26T01:30:42.103384+0000","last_became_peered":"2023-10-26T01:30:42.103384+0000","last_unstale":"2023-10-26T01:30:43.139076+0000","last_undegraded":"2023-10-26T01:30:43.139076+0000","last_fullsized":"2023-10-26T01:30:43.139076+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:33:58.768305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.098597+0000","last_change":"2023-10-26T01:30:42.101845+0000","last_active":"2023-10-26T01:30:43.098597+0000","last_peered":"2023-10-26T01:30:43.098597+0000","last_clean":"2023-10-26T01:30:43.098597+0000","last_became_active":"2023-10-26T01:30:42.097125+0000","last_became_peered":"2023-10-26T01:30:42.097125+0000","last_unstale":"2023-10-26T01:30:43.098597+0000","last_undegraded":"2023-10-26T01:30:43.098597+0000","last_fullsized":"2023-10-26T01:30:43.098597+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:33:13.615628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.086419+0000","last_change":"2023-10-26T01:30:42.115244+0000","last_active":"2023-10-26T01:30:43.086419+0000","last_peered":"2023-10-26T01:30:43.086419+0000","last_clean":"2023-10-26T01:30:43.086419+0000","last_became_active":"2023-10-26T01:30:42.114972+0000","last_became_peered":"2023-10-26T01:30:42.114972+0000","last_unstale":"2023-10-26T01:30:43.086419+0000","last_undegraded":"2023-10-26T01:30:43.086419+0000","last_fullsized":"2023-10-26T01:30:43.086419+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:07:06.450331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.099993+0000","last_change":"2023-10-26T01:30:42.108711+0000","last_active":"2023-10-26T01:30:43.099993+0000","last_peered":"2023-10-26T01:30:43.099993+0000","last_clean":"2023-10-26T01:30:43.099993+0000","last_became_active":"2023-10-26T01:30:42.108215+0000","last_became_peered":"2023-10-26T01:30:42.108215+0000","last_unstale":"2023-10-26T01:30:43.099993+0000","last_undegraded":"2023-10-26T01:30:43.099993+0000","last_fullsized":"2023-10-26T01:30:43.099993+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:26:01.898201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.088319+0000","last_change":"2023-10-26T01:30:42.115268+0000","last_active":"2023-10-26T01:30:43.088319+0000","last_peered":"2023-10-26T01:30:43.088319+0000","last_clean":"2023-10-26T01:30:43.088319+0000","last_became_active":"2023-10-26T01:30:42.113208+0000","last_became_peered":"2023-10-26T01:30:42.113208+0000","last_unstale":"2023-10-26T01:30:43.088319+0000","last_undegraded":"2023-10-26T01:30:43.088319+0000","last_fullsized":"2023-10-26T01:30:43.088319+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:56:55.394232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.084344+0000","last_change":"2023-10-26T01:30:42.093950+0000","last_active":"2023-10-26T01:30:43.084344+0000","last_peered":"2023-10-26T01:30:43.084344+0000","last_clean":"2023-10-26T01:30:43.084344+0000","last_became_active":"2023-10-26T01:30:42.093684+0000","last_became_peered":"2023-10-26T01:30:42.093684+0000","last_unstale":"2023-10-26T01:30:43.084344+0000","last_undegraded":"2023-10-26T01:30:43.084344+0000","last_fullsized":"2023-10-26T01:30:43.084344+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:05:31.590291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.086093+0000","last_change":"2023-10-26T01:30:42.108451+0000","last_active":"2023-10-26T01:30:43.086093+0000","last_peered":"2023-10-26T01:30:43.086093+0000","last_clean":"2023-10-26T01:30:43.086093+0000","last_became_active":"2023-10-26T01:30:42.107650+0000","last_became_peered":"2023-10-26T01:30:42.107650+0000","last_unstale":"2023-10-26T01:30:43.086093+0000","last_undegraded":"2023-10-26T01:30:43.086093+0000","last_fullsized":"2023-10-26T01:30:43.086093+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:26:19.035685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.098895+0000","last_change":"2023-10-26T01:30:42.093838+0000","last_active":"2023-10-26T01:30:43.098895+0000","last_peered":"2023-10-26T01:30:43.098895+0000","last_clean":"2023-10-26T01:30:43.098895+0000","last_became_active":"2023-10-26T01:30:42.093584+0000","last_became_peered":"2023-10-26T01:30:42.093584+0000","last_unstale":"2023-10-26T01:30:43.098895+0000","last_undegraded":"2023-10-26T01:30:43.098895+0000","last_fullsized":"2023-10-26T01:30:43.098895+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:49:12.753504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.097658+0000","last_change":"2023-10-26T01:30:42.094026+0000","last_active":"2023-10-26T01:30:43.097658+0000","last_peered":"2023-10-26T01:30:43.097658+0000","last_clean":"2023-10-26T01:30:43.097658+0000","last_became_active":"2023-10-26T01:30:42.093654+0000","last_became_peered":"2023-10-26T01:30:42.093654+0000","last_unstale":"2023-10-26T01:30:43.097658+0000","last_undegraded":"2023-10-26T01:30:43.097658+0000","last_fullsized":"2023-10-26T01:30:43.097658+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:05:31.590291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.088414+0000","last_change":"2023-10-26T01:30:42.099764+0000","last_active":"2023-10-26T01:30:43.088414+0000","last_peered":"2023-10-26T01:30:43.088414+0000","last_clean":"2023-10-26T01:30:43.088414+0000","last_became_active":"2023-10-26T01:30:42.098108+0000","last_became_peered":"2023-10-26T01:30:42.098108+0000","last_unstale":"2023-10-26T01:30:43.088414+0000","last_undegraded":"2023-10-26T01:30:43.088414+0000","last_fullsized":"2023-10-26T01:30:43.088414+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:08:21.563278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.085548+0000","last_change":"2023-10-26T01:30:42.097734+0000","last_active":"2023-10-26T01:30:43.085548+0000","last_peered":"2023-10-26T01:30:43.085548+0000","last_clean":"2023-10-26T01:30:43.085548+0000","last_became_active":"2023-10-26T01:30:42.097333+0000","last_became_peered":"2023-10-26T01:30:42.097333+0000","last_unstale":"2023-10-26T01:30:43.085548+0000","last_undegraded":"2023-10-26T01:30:43.085548+0000","last_fullsized":"2023-10-26T01:30:43.085548+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:28:22.384005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.138887+0000","last_change":"2023-10-26T01:30:42.108715+0000","last_active":"2023-10-26T01:30:43.138887+0000","last_peered":"2023-10-26T01:30:43.138887+0000","last_clean":"2023-10-26T01:30:43.138887+0000","last_became_active":"2023-10-26T01:30:42.108469+0000","last_became_peered":"2023-10-26T01:30:42.108469+0000","last_unstale":"2023-10-26T01:30:43.138887+0000","last_undegraded":"2023-10-26T01:30:43.138887+0000","last_fullsized":"2023-10-26T01:30:43.138887+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:45:23.061469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.140916+0000","last_change":"2023-10-26T01:30:42.099157+0000","last_active":"2023-10-26T01:30:43.140916+0000","last_peered":"2023-10-26T01:30:43.140916+0000","last_clean":"2023-10-26T01:30:43.140916+0000","last_became_active":"2023-10-26T01:30:42.098929+0000","last_became_peered":"2023-10-26T01:30:42.098929+0000","last_unstale":"2023-10-26T01:30:43.140916+0000","last_undegraded":"2023-10-26T01:30:43.140916+0000","last_fullsized":"2023-10-26T01:30:43.140916+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:13:33.838018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.099835+0000","last_change":"2023-10-26T01:30:42.108578+0000","last_active":"2023-10-26T01:30:43.099835+0000","last_peered":"2023-10-26T01:30:43.099835+0000","last_clean":"2023-10-26T01:30:43.099835+0000","last_became_active":"2023-10-26T01:30:42.107937+0000","last_became_peered":"2023-10-26T01:30:42.107937+0000","last_unstale":"2023-10-26T01:30:43.099835+0000","last_undegraded":"2023-10-26T01:30:43.099835+0000","last_fullsized":"2023-10-26T01:30:43.099835+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:38:16.726919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.098559+0000","last_change":"2023-10-26T01:30:42.108373+0000","last_active":"2023-10-26T01:30:43.098559+0000","last_peered":"2023-10-26T01:30:43.098559+0000","last_clean":"2023-10-26T01:30:43.098559+0000","last_became_active":"2023-10-26T01:30:42.107773+0000","last_became_peered":"2023-10-26T01:30:42.107773+0000","last_unstale":"2023-10-26T01:30:43.098559+0000","last_undegraded":"2023-10-26T01:30:43.098559+0000","last_fullsized":"2023-10-26T01:30:43.098559+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:49:12.753504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.097381+0000","last_change":"2023-10-26T01:30:42.094993+0000","last_active":"2023-10-26T01:30:43.097381+0000","last_peered":"2023-10-26T01:30:43.097381+0000","last_clean":"2023-10-26T01:30:43.097381+0000","last_became_active":"2023-10-26T01:30:42.094689+0000","last_became_peered":"2023-10-26T01:30:42.094689+0000","last_unstale":"2023-10-26T01:30:43.097381+0000","last_undegraded":"2023-10-26T01:30:43.097381+0000","last_fullsized":"2023-10-26T01:30:43.097381+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:49:43.468558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.086773+0000","last_change":"2023-10-26T01:30:42.108746+0000","last_active":"2023-10-26T01:30:43.086773+0000","last_peered":"2023-10-26T01:30:43.086773+0000","last_clean":"2023-10-26T01:30:43.086773+0000","last_became_active":"2023-10-26T01:30:42.108188+0000","last_became_peered":"2023-10-26T01:30:42.108188+0000","last_unstale":"2023-10-26T01:30:43.086773+0000","last_undegraded":"2023-10-26T01:30:43.086773+0000","last_fullsized":"2023-10-26T01:30:43.086773+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:38:16.726919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.099061+0000","last_change":"2023-10-26T01:30:42.110331+0000","last_active":"2023-10-26T01:30:43.099061+0000","last_peered":"2023-10-26T01:30:43.099061+0000","last_clean":"2023-10-26T01:30:43.099061+0000","last_became_active":"2023-10-26T01:30:42.110079+0000","last_became_peered":"2023-10-26T01:30:42.110079+0000","last_unstale":"2023-10-26T01:30:43.099061+0000","last_undegraded":"2023-10-26T01:30:43.099061+0000","last_fullsized":"2023-10-26T01:30:43.099061+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:33:13.615628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.139607+0000","last_change":"2023-10-26T01:30:42.108608+0000","last_active":"2023-10-26T01:30:43.139607+0000","last_peered":"2023-10-26T01:30:43.139607+0000","last_clean":"2023-10-26T01:30:43.139607+0000","last_became_active":"2023-10-26T01:30:42.108197+0000","last_became_peered":"2023-10-26T01:30:42.108197+0000","last_unstale":"2023-10-26T01:30:43.139607+0000","last_undegraded":"2023-10-26T01:30:43.139607+0000","last_fullsized":"2023-10-26T01:30:43.139607+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:17:59.186810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.098944+0000","last_change":"2023-10-26T01:30:42.110289+0000","last_active":"2023-10-26T01:30:43.098944+0000","last_peered":"2023-10-26T01:30:43.098944+0000","last_clean":"2023-10-26T01:30:43.098944+0000","last_became_active":"2023-10-26T01:30:42.110061+0000","last_became_peered":"2023-10-26T01:30:42.110061+0000","last_unstale":"2023-10-26T01:30:43.098944+0000","last_undegraded":"2023-10-26T01:30:43.098944+0000","last_fullsized":"2023-10-26T01:30:43.098944+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:19:23.544531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.084819+0000","last_change":"2023-10-26T01:30:42.108686+0000","last_active":"2023-10-26T01:30:43.084819+0000","last_peered":"2023-10-26T01:30:43.084819+0000","last_clean":"2023-10-26T01:30:43.084819+0000","last_became_active":"2023-10-26T01:30:42.108136+0000","last_became_peered":"2023-10-26T01:30:42.108136+0000","last_unstale":"2023-10-26T01:30:43.084819+0000","last_undegraded":"2023-10-26T01:30:43.084819+0000","last_fullsized":"2023-10-26T01:30:43.084819+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:13:33.838018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.098984+0000","last_change":"2023-10-26T01:30:42.102147+0000","last_active":"2023-10-26T01:30:43.098984+0000","last_peered":"2023-10-26T01:30:43.098984+0000","last_clean":"2023-10-26T01:30:43.098984+0000","last_became_active":"2023-10-26T01:30:42.097509+0000","last_became_peered":"2023-10-26T01:30:42.097509+0000","last_unstale":"2023-10-26T01:30:43.098984+0000","last_undegraded":"2023-10-26T01:30:43.098984+0000","last_fullsized":"2023-10-26T01:30:43.098984+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:26:19.035685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.085124+0000","last_change":"2023-10-26T01:30:42.108519+0000","last_active":"2023-10-26T01:30:43.085124+0000","last_peered":"2023-10-26T01:30:43.085124+0000","last_clean":"2023-10-26T01:30:43.085124+0000","last_became_active":"2023-10-26T01:30:42.108283+0000","last_became_peered":"2023-10-26T01:30:42.108283+0000","last_unstale":"2023-10-26T01:30:43.085124+0000","last_undegraded":"2023-10-26T01:30:43.085124+0000","last_fullsized":"2023-10-26T01:30:43.085124+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:33:13.615628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.097862+0000","last_change":"2023-10-26T01:30:42.111865+0000","last_active":"2023-10-26T01:30:43.097862+0000","last_peered":"2023-10-26T01:30:43.097862+0000","last_clean":"2023-10-26T01:30:43.097862+0000","last_became_active":"2023-10-26T01:30:42.111636+0000","last_became_peered":"2023-10-26T01:30:42.111636+0000","last_unstale":"2023-10-26T01:30:43.097862+0000","last_undegraded":"2023-10-26T01:30:43.097862+0000","last_fullsized":"2023-10-26T01:30:43.097862+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:49:56.641022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.141348+0000","last_change":"2023-10-26T01:30:42.091332+0000","last_active":"2023-10-26T01:30:43.141348+0000","last_peered":"2023-10-26T01:30:43.141348+0000","last_clean":"2023-10-26T01:30:43.141348+0000","last_became_active":"2023-10-26T01:30:42.091105+0000","last_became_peered":"2023-10-26T01:30:42.091105+0000","last_unstale":"2023-10-26T01:30:43.141348+0000","last_undegraded":"2023-10-26T01:30:43.141348+0000","last_fullsized":"2023-10-26T01:30:43.141348+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:33:13.615628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.085113+0000","last_change":"2023-10-26T01:30:42.108822+0000","last_active":"2023-10-26T01:30:43.085113+0000","last_peered":"2023-10-26T01:30:43.085113+0000","last_clean":"2023-10-26T01:30:43.085113+0000","last_became_active":"2023-10-26T01:30:42.108448+0000","last_became_peered":"2023-10-26T01:30:42.108448+0000","last_unstale":"2023-10-26T01:30:43.085113+0000","last_undegraded":"2023-10-26T01:30:43.085113+0000","last_fullsized":"2023-10-26T01:30:43.085113+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:49:12.753504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.139200+0000","last_change":"2023-10-26T01:30:42.101708+0000","last_active":"2023-10-26T01:30:43.139200+0000","last_peered":"2023-10-26T01:30:43.139200+0000","last_clean":"2023-10-26T01:30:43.139200+0000","last_became_active":"2023-10-26T01:30:42.101434+0000","last_became_peered":"2023-10-26T01:30:42.101434+0000","last_unstale":"2023-10-26T01:30:43.139200+0000","last_undegraded":"2023-10-26T01:30:43.139200+0000","last_fullsized":"2023-10-26T01:30:43.139200+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:29:18.643148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.099207+0000","last_change":"2023-10-26T01:30:42.097495+0000","last_active":"2023-10-26T01:30:43.099207+0000","last_peered":"2023-10-26T01:30:43.099207+0000","last_clean":"2023-10-26T01:30:43.099207+0000","last_became_active":"2023-10-26T01:30:42.097214+0000","last_became_peered":"2023-10-26T01:30:42.097214+0000","last_unstale":"2023-10-26T01:30:43.099207+0000","last_undegraded":"2023-10-26T01:30:43.099207+0000","last_fullsized":"2023-10-26T01:30:43.099207+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T01:55:28.470913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.097481+0000","last_change":"2023-10-26T01:30:42.093157+0000","last_active":"2023-10-26T01:30:43.097481+0000","last_peered":"2023-10-26T01:30:43.097481+0000","last_clean":"2023-10-26T01:30:43.097481+0000","last_became_active":"2023-10-26T01:30:42.092613+0000","last_became_peered":"2023-10-26T01:30:42.092613+0000","last_unstale":"2023-10-26T01:30:43.097481+0000","last_undegraded":"2023-10-26T01:30:43.097481+0000","last_fullsized":"2023-10-26T01:30:43.097481+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:09:01.061004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.098226+0000","last_change":"2023-10-26T01:30:42.102704+0000","last_active":"2023-10-26T01:30:43.098226+0000","last_peered":"2023-10-26T01:30:43.098226+0000","last_clean":"2023-10-26T01:30:43.098226+0000","last_became_active":"2023-10-26T01:30:42.102350+0000","last_became_peered":"2023-10-26T01:30:42.102350+0000","last_unstale":"2023-10-26T01:30:43.098226+0000","last_undegraded":"2023-10-26T01:30:43.098226+0000","last_fullsized":"2023-10-26T01:30:43.098226+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:53:57.632838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.086493+0000","last_change":"2023-10-26T01:30:42.093818+0000","last_active":"2023-10-26T01:30:43.086493+0000","last_peered":"2023-10-26T01:30:43.086493+0000","last_clean":"2023-10-26T01:30:43.086493+0000","last_became_active":"2023-10-26T01:30:42.091310+0000","last_became_peered":"2023-10-26T01:30:42.091310+0000","last_unstale":"2023-10-26T01:30:43.086493+0000","last_undegraded":"2023-10-26T01:30:43.086493+0000","last_fullsized":"2023-10-26T01:30:43.086493+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:04:23.158155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.114911+0000","last_change":"2023-10-26T01:30:41.097666+0000","last_active":"2023-10-26T01:30:42.114911+0000","last_peered":"2023-10-26T01:30:42.114911+0000","last_clean":"2023-10-26T01:30:42.114911+0000","last_became_active":"2023-10-26T01:30:41.093986+0000","last_became_peered":"2023-10-26T01:30:41.093986+0000","last_unstale":"2023-10-26T01:30:42.114911+0000","last_undegraded":"2023-10-26T01:30:42.114911+0000","last_fullsized":"2023-10-26T01:30:42.114911+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:49:42.452332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.139516+0000","last_change":"2023-10-26T01:30:42.110839+0000","last_active":"2023-10-26T01:30:43.139516+0000","last_peered":"2023-10-26T01:30:43.139516+0000","last_clean":"2023-10-26T01:30:43.139516+0000","last_became_active":"2023-10-26T01:30:42.109385+0000","last_became_peered":"2023-10-26T01:30:42.109385+0000","last_unstale":"2023-10-26T01:30:43.139516+0000","last_undegraded":"2023-10-26T01:30:43.139516+0000","last_fullsized":"2023-10-26T01:30:43.139516+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:49:56.641022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.105272+0000","last_change":"2023-10-26T01:30:41.100248+0000","last_active":"2023-10-26T01:30:42.105272+0000","last_peered":"2023-10-26T01:30:42.105272+0000","last_clean":"2023-10-26T01:30:42.105272+0000","last_became_active":"2023-10-26T01:30:41.099957+0000","last_became_peered":"2023-10-26T01:30:41.099957+0000","last_unstale":"2023-10-26T01:30:42.105272+0000","last_undegraded":"2023-10-26T01:30:42.105272+0000","last_fullsized":"2023-10-26T01:30:42.105272+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:29:17.626922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.088500+0000","last_change":"2023-10-26T01:30:41.103843+0000","last_active":"2023-10-26T01:30:42.088500+0000","last_peered":"2023-10-26T01:30:42.088500+0000","last_clean":"2023-10-26T01:30:42.088500+0000","last_became_active":"2023-10-26T01:30:41.103356+0000","last_became_peered":"2023-10-26T01:30:41.103356+0000","last_unstale":"2023-10-26T01:30:42.088500+0000","last_undegraded":"2023-10-26T01:30:42.088500+0000","last_fullsized":"2023-10-26T01:30:42.088500+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:40:15.199455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.085463+0000","last_change":"2023-10-26T01:30:42.102443+0000","last_active":"2023-10-26T01:30:43.085463+0000","last_peered":"2023-10-26T01:30:43.085463+0000","last_clean":"2023-10-26T01:30:43.085463+0000","last_became_active":"2023-10-26T01:30:42.102213+0000","last_became_peered":"2023-10-26T01:30:42.102213+0000","last_unstale":"2023-10-26T01:30:43.085463+0000","last_undegraded":"2023-10-26T01:30:43.085463+0000","last_fullsized":"2023-10-26T01:30:43.085463+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:26:19.035685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.089693+0000","last_change":"2023-10-26T01:30:41.102989+0000","last_active":"2023-10-26T01:30:42.089693+0000","last_peered":"2023-10-26T01:30:42.089693+0000","last_clean":"2023-10-26T01:30:42.089693+0000","last_became_active":"2023-10-26T01:30:41.102681+0000","last_became_peered":"2023-10-26T01:30:41.102681+0000","last_unstale":"2023-10-26T01:30:42.089693+0000","last_undegraded":"2023-10-26T01:30:42.089693+0000","last_fullsized":"2023-10-26T01:30:42.089693+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:49:55.624796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.140341+0000","last_change":"2023-10-26T01:30:42.091359+0000","last_active":"2023-10-26T01:30:43.140341+0000","last_peered":"2023-10-26T01:30:43.140341+0000","last_clean":"2023-10-26T01:30:43.140341+0000","last_became_active":"2023-10-26T01:30:42.090949+0000","last_became_peered":"2023-10-26T01:30:42.090949+0000","last_unstale":"2023-10-26T01:30:43.140341+0000","last_undegraded":"2023-10-26T01:30:43.140341+0000","last_fullsized":"2023-10-26T01:30:43.140341+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:49:56.641022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.089836+0000","last_change":"2023-10-26T01:30:41.104058+0000","last_active":"2023-10-26T01:30:42.089836+0000","last_peered":"2023-10-26T01:30:42.089836+0000","last_clean":"2023-10-26T01:30:42.089836+0000","last_became_active":"2023-10-26T01:30:41.103821+0000","last_became_peered":"2023-10-26T01:30:41.103821+0000","last_unstale":"2023-10-26T01:30:42.089836+0000","last_undegraded":"2023-10-26T01:30:42.089836+0000","last_fullsized":"2023-10-26T01:30:42.089836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:46:05.927163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.098774+0000","last_change":"2023-10-26T01:30:42.108520+0000","last_active":"2023-10-26T01:30:43.098774+0000","last_peered":"2023-10-26T01:30:43.098774+0000","last_clean":"2023-10-26T01:30:43.098774+0000","last_became_active":"2023-10-26T01:30:42.108140+0000","last_became_peered":"2023-10-26T01:30:42.108140+0000","last_unstale":"2023-10-26T01:30:43.098774+0000","last_undegraded":"2023-10-26T01:30:43.098774+0000","last_fullsized":"2023-10-26T01:30:43.098774+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:31:08.727626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.088126+0000","last_change":"2023-10-26T01:30:41.106779+0000","last_active":"2023-10-26T01:30:42.088126+0000","last_peered":"2023-10-26T01:30:42.088126+0000","last_clean":"2023-10-26T01:30:42.088126+0000","last_became_active":"2023-10-26T01:30:41.104353+0000","last_became_peered":"2023-10-26T01:30:41.104353+0000","last_unstale":"2023-10-26T01:30:42.088126+0000","last_undegraded":"2023-10-26T01:30:42.088126+0000","last_fullsized":"2023-10-26T01:30:42.088126+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:45:22.045243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.139887+0000","last_change":"2023-10-26T01:30:42.105447+0000","last_active":"2023-10-26T01:30:43.139887+0000","last_peered":"2023-10-26T01:30:43.139887+0000","last_clean":"2023-10-26T01:30:43.139887+0000","last_became_active":"2023-10-26T01:30:42.105098+0000","last_became_peered":"2023-10-26T01:30:42.105098+0000","last_unstale":"2023-10-26T01:30:43.139887+0000","last_undegraded":"2023-10-26T01:30:43.139887+0000","last_fullsized":"2023-10-26T01:30:43.139887+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:13:33.838018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.110602+0000","last_change":"2023-10-26T01:30:41.103826+0000","last_active":"2023-10-26T01:30:42.110602+0000","last_peered":"2023-10-26T01:30:42.110602+0000","last_clean":"2023-10-26T01:30:42.110602+0000","last_became_active":"2023-10-26T01:30:41.103552+0000","last_became_peered":"2023-10-26T01:30:41.103552+0000","last_unstale":"2023-10-26T01:30:42.110602+0000","last_undegraded":"2023-10-26T01:30:42.110602+0000","last_fullsized":"2023-10-26T01:30:42.110602+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:54:04.970133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.086144+0000","last_change":"2023-10-26T01:30:42.114167+0000","last_active":"2023-10-26T01:30:43.086144+0000","last_peered":"2023-10-26T01:30:43.086144+0000","last_clean":"2023-10-26T01:30:43.086144+0000","last_became_active":"2023-10-26T01:30:42.113241+0000","last_became_peered":"2023-10-26T01:30:42.113241+0000","last_unstale":"2023-10-26T01:30:43.086144+0000","last_undegraded":"2023-10-26T01:30:43.086144+0000","last_fullsized":"2023-10-26T01:30:43.086144+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:40:16.215681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.116049+0000","last_change":"2023-10-26T01:30:41.106514+0000","last_active":"2023-10-26T01:30:42.116049+0000","last_peered":"2023-10-26T01:30:42.116049+0000","last_clean":"2023-10-26T01:30:42.116049+0000","last_became_active":"2023-10-26T01:30:41.106147+0000","last_became_peered":"2023-10-26T01:30:41.106147+0000","last_unstale":"2023-10-26T01:30:42.116049+0000","last_undegraded":"2023-10-26T01:30:42.116049+0000","last_fullsized":"2023-10-26T01:30:42.116049+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:45:22.045243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.089920+0000","last_change":"2023-10-26T01:30:42.112813+0000","last_active":"2023-10-26T01:30:43.089920+0000","last_peered":"2023-10-26T01:30:43.089920+0000","last_clean":"2023-10-26T01:30:43.089920+0000","last_became_active":"2023-10-26T01:30:42.112571+0000","last_became_peered":"2023-10-26T01:30:42.112571+0000","last_unstale":"2023-10-26T01:30:43.089920+0000","last_undegraded":"2023-10-26T01:30:43.089920+0000","last_fullsized":"2023-10-26T01:30:43.089920+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:19:16.042472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.108474+0000","last_change":"2023-10-26T01:30:41.098865+0000","last_active":"2023-10-26T01:30:42.108474+0000","last_peered":"2023-10-26T01:30:42.108474+0000","last_clean":"2023-10-26T01:30:42.108474+0000","last_became_active":"2023-10-26T01:30:41.098506+0000","last_became_peered":"2023-10-26T01:30:41.098506+0000","last_unstale":"2023-10-26T01:30:42.108474+0000","last_undegraded":"2023-10-26T01:30:42.108474+0000","last_fullsized":"2023-10-26T01:30:42.108474+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:09:00.044778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.140188+0000","last_change":"2023-10-26T01:30:42.108443+0000","last_active":"2023-10-26T01:30:43.140188+0000","last_peered":"2023-10-26T01:30:43.140188+0000","last_clean":"2023-10-26T01:30:43.140188+0000","last_became_active":"2023-10-26T01:30:42.107837+0000","last_became_peered":"2023-10-26T01:30:42.107837+0000","last_unstale":"2023-10-26T01:30:43.140188+0000","last_undegraded":"2023-10-26T01:30:43.140188+0000","last_fullsized":"2023-10-26T01:30:43.140188+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:33:13.615628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.090761+0000","last_change":"2023-10-26T01:30:41.106795+0000","last_active":"2023-10-26T01:30:42.090761+0000","last_peered":"2023-10-26T01:30:42.090761+0000","last_clean":"2023-10-26T01:30:42.090761+0000","last_became_active":"2023-10-26T01:30:41.106222+0000","last_became_peered":"2023-10-26T01:30:41.106222+0000","last_unstale":"2023-10-26T01:30:42.090761+0000","last_undegraded":"2023-10-26T01:30:42.090761+0000","last_fullsized":"2023-10-26T01:30:42.090761+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:53:56.616612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.086822+0000","last_change":"2023-10-26T01:30:42.113053+0000","last_active":"2023-10-26T01:30:43.086822+0000","last_peered":"2023-10-26T01:30:43.086822+0000","last_clean":"2023-10-26T01:30:43.086822+0000","last_became_active":"2023-10-26T01:30:42.112492+0000","last_became_peered":"2023-10-26T01:30:42.112492+0000","last_unstale":"2023-10-26T01:30:43.086822+0000","last_undegraded":"2023-10-26T01:30:43.086822+0000","last_fullsized":"2023-10-26T01:30:43.086822+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:20:47.431800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.089525+0000","last_change":"2023-10-26T01:30:41.100835+0000","last_active":"2023-10-26T01:30:42.089525+0000","last_peered":"2023-10-26T01:30:42.089525+0000","last_clean":"2023-10-26T01:30:42.089525+0000","last_became_active":"2023-10-26T01:30:41.099708+0000","last_became_peered":"2023-10-26T01:30:41.099708+0000","last_unstale":"2023-10-26T01:30:42.089525+0000","last_undegraded":"2023-10-26T01:30:42.089525+0000","last_fullsized":"2023-10-26T01:30:42.089525+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:04:22.141930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.098337+0000","last_change":"2023-10-26T01:30:42.100857+0000","last_active":"2023-10-26T01:30:43.098337+0000","last_peered":"2023-10-26T01:30:43.098337+0000","last_clean":"2023-10-26T01:30:43.098337+0000","last_became_active":"2023-10-26T01:30:42.100545+0000","last_became_peered":"2023-10-26T01:30:42.100545+0000","last_unstale":"2023-10-26T01:30:43.098337+0000","last_undegraded":"2023-10-26T01:30:43.098337+0000","last_fullsized":"2023-10-26T01:30:43.098337+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:13:33.838018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.114399+0000","last_change":"2023-10-26T01:30:41.095998+0000","last_active":"2023-10-26T01:30:42.114399+0000","last_peered":"2023-10-26T01:30:42.114399+0000","last_clean":"2023-10-26T01:30:42.114399+0000","last_became_active":"2023-10-26T01:30:41.095494+0000","last_became_peered":"2023-10-26T01:30:41.095494+0000","last_unstale":"2023-10-26T01:30:42.114399+0000","last_undegraded":"2023-10-26T01:30:42.114399+0000","last_fullsized":"2023-10-26T01:30:42.114399+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:54:04.970133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.089073+0000","last_change":"2023-10-26T01:30:42.115510+0000","last_active":"2023-10-26T01:30:43.089073+0000","last_peered":"2023-10-26T01:30:43.089073+0000","last_clean":"2023-10-26T01:30:43.089073+0000","last_became_active":"2023-10-26T01:30:42.113552+0000","last_became_peered":"2023-10-26T01:30:42.113552+0000","last_unstale":"2023-10-26T01:30:43.089073+0000","last_undegraded":"2023-10-26T01:30:43.089073+0000","last_fullsized":"2023-10-26T01:30:43.089073+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:32:08.387966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.110967+0000","last_change":"2023-10-26T01:30:41.093200+0000","last_active":"2023-10-26T01:30:42.110967+0000","last_peered":"2023-10-26T01:30:42.110967+0000","last_clean":"2023-10-26T01:30:42.110967+0000","last_became_active":"2023-10-26T01:30:41.092917+0000","last_became_peered":"2023-10-26T01:30:41.092917+0000","last_unstale":"2023-10-26T01:30:42.110967+0000","last_undegraded":"2023-10-26T01:30:42.110967+0000","last_fullsized":"2023-10-26T01:30:42.110967+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:54:04.970133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.087143+0000","last_change":"2023-10-26T01:30:42.113216+0000","last_active":"2023-10-26T01:30:43.087143+0000","last_peered":"2023-10-26T01:30:43.087143+0000","last_clean":"2023-10-26T01:30:43.087143+0000","last_became_active":"2023-10-26T01:30:42.112815+0000","last_became_peered":"2023-10-26T01:30:42.112815+0000","last_unstale":"2023-10-26T01:30:43.087143+0000","last_undegraded":"2023-10-26T01:30:43.087143+0000","last_fullsized":"2023-10-26T01:30:43.087143+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:55:51.116465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.114880+0000","last_change":"2023-10-26T01:30:41.102365+0000","last_active":"2023-10-26T01:30:42.114880+0000","last_peered":"2023-10-26T01:30:42.114880+0000","last_clean":"2023-10-26T01:30:42.114880+0000","last_became_active":"2023-10-26T01:30:41.102135+0000","last_became_peered":"2023-10-26T01:30:41.102135+0000","last_unstale":"2023-10-26T01:30:42.114880+0000","last_undegraded":"2023-10-26T01:30:42.114880+0000","last_fullsized":"2023-10-26T01:30:42.114880+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:08:20.547052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.086427+0000","last_change":"2023-10-26T01:30:42.108605+0000","last_active":"2023-10-26T01:30:43.086427+0000","last_peered":"2023-10-26T01:30:43.086427+0000","last_clean":"2023-10-26T01:30:43.086427+0000","last_became_active":"2023-10-26T01:30:42.107907+0000","last_became_peered":"2023-10-26T01:30:42.107907+0000","last_unstale":"2023-10-26T01:30:43.086427+0000","last_undegraded":"2023-10-26T01:30:43.086427+0000","last_fullsized":"2023-10-26T01:30:43.086427+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T01:55:28.470913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.105745+0000","last_change":"2023-10-26T01:30:41.093456+0000","last_active":"2023-10-26T01:30:42.105745+0000","last_peered":"2023-10-26T01:30:42.105745+0000","last_clean":"2023-10-26T01:30:42.105745+0000","last_became_active":"2023-10-26T01:30:41.092811+0000","last_became_peered":"2023-10-26T01:30:41.092811+0000","last_unstale":"2023-10-26T01:30:42.105745+0000","last_undegraded":"2023-10-26T01:30:42.105745+0000","last_fullsized":"2023-10-26T01:30:42.105745+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:04:22.141930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.099118+0000","last_change":"2023-10-26T01:30:42.097121+0000","last_active":"2023-10-26T01:30:43.099118+0000","last_peered":"2023-10-26T01:30:43.099118+0000","last_clean":"2023-10-26T01:30:43.099118+0000","last_became_active":"2023-10-26T01:30:42.096784+0000","last_became_peered":"2023-10-26T01:30:42.096784+0000","last_unstale":"2023-10-26T01:30:43.099118+0000","last_undegraded":"2023-10-26T01:30:43.099118+0000","last_fullsized":"2023-10-26T01:30:43.099118+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:28:22.384005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.083700+0000","last_change":"2023-10-26T01:30:41.106632+0000","last_active":"2023-10-26T01:30:42.083700+0000","last_peered":"2023-10-26T01:30:42.083700+0000","last_clean":"2023-10-26T01:30:42.083700+0000","last_became_active":"2023-10-26T01:30:41.106360+0000","last_became_peered":"2023-10-26T01:30:41.106360+0000","last_unstale":"2023-10-26T01:30:42.083700+0000","last_undegraded":"2023-10-26T01:30:42.083700+0000","last_fullsized":"2023-10-26T01:30:42.083700+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:40:15.199455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.086951+0000","last_change":"2023-10-26T01:30:42.109475+0000","last_active":"2023-10-26T01:30:43.086951+0000","last_peered":"2023-10-26T01:30:43.086951+0000","last_clean":"2023-10-26T01:30:43.086951+0000","last_became_active":"2023-10-26T01:30:42.109232+0000","last_became_peered":"2023-10-26T01:30:42.109232+0000","last_unstale":"2023-10-26T01:30:43.086951+0000","last_undegraded":"2023-10-26T01:30:43.086951+0000","last_fullsized":"2023-10-26T01:30:43.086951+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:26:01.898201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":165,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.096979+0000","last_change":"2023-10-26T01:30:34.883484+0000","last_active":"2023-10-26T01:30:42.096979+0000","last_peered":"2023-10-26T01:30:42.096979+0000","last_clean":"2023-10-26T01:30:42.096979+0000","last_became_active":"2023-10-26T01:30:34.883071+0000","last_became_peered":"2023-10-26T01:30:34.883071+0000","last_unstale":"2023-10-26T01:30:42.096979+0000","last_undegraded":"2023-10-26T01:30:42.096979+0000","last_fullsized":"2023-10-26T01:30:42.096979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:32.890096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:32.890096+0000","last_clean_scrub_stamp":"2023-10-26T01:30:32.890096+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:22:57.937891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.090888+0000","last_change":"2023-10-26T01:30:41.105687+0000","last_active":"2023-10-26T01:30:42.090888+0000","last_peered":"2023-10-26T01:30:42.090888+0000","last_clean":"2023-10-26T01:30:42.090888+0000","last_became_active":"2023-10-26T01:30:41.105235+0000","last_became_peered":"2023-10-26T01:30:41.105235+0000","last_unstale":"2023-10-26T01:30:42.090888+0000","last_undegraded":"2023-10-26T01:30:42.090888+0000","last_fullsized":"2023-10-26T01:30:42.090888+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:13:32.821792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.084569+0000","last_change":"2023-10-26T01:30:42.093817+0000","last_active":"2023-10-26T01:30:43.084569+0000","last_peered":"2023-10-26T01:30:43.084569+0000","last_clean":"2023-10-26T01:30:43.084569+0000","last_became_active":"2023-10-26T01:30:42.093586+0000","last_became_peered":"2023-10-26T01:30:42.093586+0000","last_unstale":"2023-10-26T01:30:43.084569+0000","last_undegraded":"2023-10-26T01:30:43.084569+0000","last_fullsized":"2023-10-26T01:30:43.084569+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:17:59.186810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.096890+0000","last_change":"2023-10-26T01:30:41.094664+0000","last_active":"2023-10-26T01:30:42.096890+0000","last_peered":"2023-10-26T01:30:42.096890+0000","last_clean":"2023-10-26T01:30:42.096890+0000","last_became_active":"2023-10-26T01:30:41.092745+0000","last_became_peered":"2023-10-26T01:30:41.092745+0000","last_unstale":"2023-10-26T01:30:42.096890+0000","last_undegraded":"2023-10-26T01:30:42.096890+0000","last_fullsized":"2023-10-26T01:30:42.096890+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:45:22.045243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.140108+0000","last_change":"2023-10-26T01:30:42.107538+0000","last_active":"2023-10-26T01:30:43.140108+0000","last_peered":"2023-10-26T01:30:43.140108+0000","last_clean":"2023-10-26T01:30:43.140108+0000","last_became_active":"2023-10-26T01:30:42.107258+0000","last_became_peered":"2023-10-26T01:30:42.107258+0000","last_unstale":"2023-10-26T01:30:43.140108+0000","last_undegraded":"2023-10-26T01:30:43.140108+0000","last_fullsized":"2023-10-26T01:30:43.140108+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:49:12.753504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.083749+0000","last_change":"2023-10-26T01:30:41.099543+0000","last_active":"2023-10-26T01:30:42.083749+0000","last_peered":"2023-10-26T01:30:42.083749+0000","last_clean":"2023-10-26T01:30:42.083749+0000","last_became_active":"2023-10-26T01:30:41.099277+0000","last_became_peered":"2023-10-26T01:30:41.099277+0000","last_unstale":"2023-10-26T01:30:42.083749+0000","last_undegraded":"2023-10-26T01:30:42.083749+0000","last_fullsized":"2023-10-26T01:30:42.083749+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:04:22.141930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.140694+0000","last_change":"2023-10-26T01:30:42.085871+0000","last_active":"2023-10-26T01:30:43.140694+0000","last_peered":"2023-10-26T01:30:43.140694+0000","last_clean":"2023-10-26T01:30:43.140694+0000","last_became_active":"2023-10-26T01:30:42.085409+0000","last_became_peered":"2023-10-26T01:30:42.085409+0000","last_unstale":"2023-10-26T01:30:43.140694+0000","last_undegraded":"2023-10-26T01:30:43.140694+0000","last_fullsized":"2023-10-26T01:30:43.140694+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:17:59.186810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.088251+0000","last_change":"2023-10-26T01:30:41.107609+0000","last_active":"2023-10-26T01:30:42.088251+0000","last_peered":"2023-10-26T01:30:42.088251+0000","last_clean":"2023-10-26T01:30:42.088251+0000","last_became_active":"2023-10-26T01:30:41.107189+0000","last_became_peered":"2023-10-26T01:30:41.107189+0000","last_unstale":"2023-10-26T01:30:42.088251+0000","last_undegraded":"2023-10-26T01:30:42.088251+0000","last_fullsized":"2023-10-26T01:30:42.088251+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:33:57.752079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.088414+0000","last_change":"2023-10-26T01:30:42.105495+0000","last_active":"2023-10-26T01:30:43.088414+0000","last_peered":"2023-10-26T01:30:43.088414+0000","last_clean":"2023-10-26T01:30:43.088414+0000","last_became_active":"2023-10-26T01:30:42.105247+0000","last_became_peered":"2023-10-26T01:30:42.105247+0000","last_unstale":"2023-10-26T01:30:43.088414+0000","last_undegraded":"2023-10-26T01:30:43.088414+0000","last_fullsized":"2023-10-26T01:30:43.088414+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:24:17.679373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.088367+0000","last_change":"2023-10-26T01:30:41.096866+0000","last_active":"2023-10-26T01:30:42.088367+0000","last_peered":"2023-10-26T01:30:42.088367+0000","last_clean":"2023-10-26T01:30:42.088367+0000","last_became_active":"2023-10-26T01:30:41.096255+0000","last_became_peered":"2023-10-26T01:30:41.096255+0000","last_unstale":"2023-10-26T01:30:42.088367+0000","last_undegraded":"2023-10-26T01:30:42.088367+0000","last_fullsized":"2023-10-26T01:30:42.088367+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:29:17.626922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.099320+0000","last_change":"2023-10-26T01:30:42.102298+0000","last_active":"2023-10-26T01:30:43.099320+0000","last_peered":"2023-10-26T01:30:43.099320+0000","last_clean":"2023-10-26T01:30:43.099320+0000","last_became_active":"2023-10-26T01:30:42.097872+0000","last_became_peered":"2023-10-26T01:30:42.097872+0000","last_unstale":"2023-10-26T01:30:43.099320+0000","last_undegraded":"2023-10-26T01:30:43.099320+0000","last_fullsized":"2023-10-26T01:30:43.099320+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:07:06.450331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.096762+0000","last_change":"2023-10-26T01:30:41.096310+0000","last_active":"2023-10-26T01:30:42.096762+0000","last_peered":"2023-10-26T01:30:42.096762+0000","last_clean":"2023-10-26T01:30:42.096762+0000","last_became_active":"2023-10-26T01:30:41.095929+0000","last_became_peered":"2023-10-26T01:30:41.095929+0000","last_unstale":"2023-10-26T01:30:42.096762+0000","last_undegraded":"2023-10-26T01:30:42.096762+0000","last_fullsized":"2023-10-26T01:30:42.096762+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:31:39.970444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.140793+0000","last_change":"2023-10-26T01:30:42.089331+0000","last_active":"2023-10-26T01:30:43.140793+0000","last_peered":"2023-10-26T01:30:43.140793+0000","last_clean":"2023-10-26T01:30:43.140793+0000","last_became_active":"2023-10-26T01:30:42.088901+0000","last_became_peered":"2023-10-26T01:30:42.088901+0000","last_unstale":"2023-10-26T01:30:43.140793+0000","last_undegraded":"2023-10-26T01:30:43.140793+0000","last_fullsized":"2023-10-26T01:30:43.140793+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:53:57.632838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.082315+0000","last_change":"2023-10-26T01:30:41.102617+0000","last_active":"2023-10-26T01:30:42.082315+0000","last_peered":"2023-10-26T01:30:42.082315+0000","last_clean":"2023-10-26T01:30:42.082315+0000","last_became_active":"2023-10-26T01:30:41.102376+0000","last_became_peered":"2023-10-26T01:30:41.102376+0000","last_unstale":"2023-10-26T01:30:42.082315+0000","last_undegraded":"2023-10-26T01:30:42.082315+0000","last_fullsized":"2023-10-26T01:30:42.082315+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:24:16.663147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.084424+0000","last_change":"2023-10-26T01:30:42.112237+0000","last_active":"2023-10-26T01:30:43.084424+0000","last_peered":"2023-10-26T01:30:43.084424+0000","last_clean":"2023-10-26T01:30:43.084424+0000","last_became_active":"2023-10-26T01:30:42.112006+0000","last_became_peered":"2023-10-26T01:30:42.112006+0000","last_unstale":"2023-10-26T01:30:43.084424+0000","last_undegraded":"2023-10-26T01:30:43.084424+0000","last_fullsized":"2023-10-26T01:30:43.084424+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:49:56.641022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.112282+0000","last_change":"2023-10-26T01:30:41.102404+0000","last_active":"2023-10-26T01:30:42.112282+0000","last_peered":"2023-10-26T01:30:42.112282+0000","last_clean":"2023-10-26T01:30:42.112282+0000","last_became_active":"2023-10-26T01:30:41.102165+0000","last_became_peered":"2023-10-26T01:30:41.102165+0000","last_unstale":"2023-10-26T01:30:42.112282+0000","last_undegraded":"2023-10-26T01:30:42.112282+0000","last_fullsized":"2023-10-26T01:30:42.112282+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:08:20.547052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.139422+0000","last_change":"2023-10-26T01:30:42.103630+0000","last_active":"2023-10-26T01:30:43.139422+0000","last_peered":"2023-10-26T01:30:43.139422+0000","last_clean":"2023-10-26T01:30:43.139422+0000","last_became_active":"2023-10-26T01:30:42.099458+0000","last_became_peered":"2023-10-26T01:30:42.099458+0000","last_unstale":"2023-10-26T01:30:43.139422+0000","last_undegraded":"2023-10-26T01:30:43.139422+0000","last_fullsized":"2023-10-26T01:30:43.139422+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:05:31.590291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.097650+0000","last_change":"2023-10-26T01:30:41.097301+0000","last_active":"2023-10-26T01:30:42.097650+0000","last_peered":"2023-10-26T01:30:42.097650+0000","last_clean":"2023-10-26T01:30:42.097650+0000","last_became_active":"2023-10-26T01:30:41.096866+0000","last_became_peered":"2023-10-26T01:30:41.096866+0000","last_unstale":"2023-10-26T01:30:42.097650+0000","last_undegraded":"2023-10-26T01:30:42.097650+0000","last_fullsized":"2023-10-26T01:30:42.097650+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:05:30.574065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.089679+0000","last_change":"2023-10-26T01:30:42.087483+0000","last_active":"2023-10-26T01:30:43.089679+0000","last_peered":"2023-10-26T01:30:43.089679+0000","last_clean":"2023-10-26T01:30:43.089679+0000","last_became_active":"2023-10-26T01:30:42.087234+0000","last_became_peered":"2023-10-26T01:30:42.087234+0000","last_unstale":"2023-10-26T01:30:43.089679+0000","last_undegraded":"2023-10-26T01:30:43.089679+0000","last_fullsized":"2023-10-26T01:30:43.089679+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:46:06.943389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.088240+0000","last_change":"2023-10-26T01:30:41.097032+0000","last_active":"2023-10-26T01:30:42.088240+0000","last_peered":"2023-10-26T01:30:42.088240+0000","last_clean":"2023-10-26T01:30:42.088240+0000","last_became_active":"2023-10-26T01:30:41.096591+0000","last_became_peered":"2023-10-26T01:30:41.096591+0000","last_unstale":"2023-10-26T01:30:42.088240+0000","last_undegraded":"2023-10-26T01:30:42.088240+0000","last_fullsized":"2023-10-26T01:30:42.088240+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:33:57.752079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.084667+0000","last_change":"2023-10-26T01:30:42.101380+0000","last_active":"2023-10-26T01:30:43.084667+0000","last_peered":"2023-10-26T01:30:43.084667+0000","last_clean":"2023-10-26T01:30:43.084667+0000","last_became_active":"2023-10-26T01:30:42.101179+0000","last_became_peered":"2023-10-26T01:30:42.101179+0000","last_unstale":"2023-10-26T01:30:43.084667+0000","last_undegraded":"2023-10-26T01:30:43.084667+0000","last_fullsized":"2023-10-26T01:30:43.084667+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:53:57.632838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.084678+0000","last_change":"2023-10-26T01:30:41.101587+0000","last_active":"2023-10-26T01:30:42.084678+0000","last_peered":"2023-10-26T01:30:42.084678+0000","last_clean":"2023-10-26T01:30:42.084678+0000","last_became_active":"2023-10-26T01:30:41.101325+0000","last_became_peered":"2023-10-26T01:30:41.101325+0000","last_unstale":"2023-10-26T01:30:42.084678+0000","last_undegraded":"2023-10-26T01:30:42.084678+0000","last_fullsized":"2023-10-26T01:30:42.084678+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:49:42.452332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.085284+0000","last_change":"2023-10-26T01:30:42.103918+0000","last_active":"2023-10-26T01:30:43.085284+0000","last_peered":"2023-10-26T01:30:43.085284+0000","last_clean":"2023-10-26T01:30:43.085284+0000","last_became_active":"2023-10-26T01:30:42.103701+0000","last_became_peered":"2023-10-26T01:30:42.103701+0000","last_unstale":"2023-10-26T01:30:43.085284+0000","last_undegraded":"2023-10-26T01:30:43.085284+0000","last_fullsized":"2023-10-26T01:30:43.085284+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:19:23.544531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.114450+0000","last_change":"2023-10-26T01:30:41.095274+0000","last_active":"2023-10-26T01:30:42.114450+0000","last_peered":"2023-10-26T01:30:42.114450+0000","last_clean":"2023-10-26T01:30:42.114450+0000","last_became_active":"2023-10-26T01:30:41.092922+0000","last_became_peered":"2023-10-26T01:30:41.092922+0000","last_unstale":"2023-10-26T01:30:42.114450+0000","last_undegraded":"2023-10-26T01:30:42.114450+0000","last_fullsized":"2023-10-26T01:30:42.114450+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:33:12.599402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.098208+0000","last_change":"2023-10-26T01:30:42.105139+0000","last_active":"2023-10-26T01:30:43.098208+0000","last_peered":"2023-10-26T01:30:43.098208+0000","last_clean":"2023-10-26T01:30:43.098208+0000","last_became_active":"2023-10-26T01:30:42.104913+0000","last_became_peered":"2023-10-26T01:30:42.104913+0000","last_unstale":"2023-10-26T01:30:43.098208+0000","last_undegraded":"2023-10-26T01:30:43.098208+0000","last_fullsized":"2023-10-26T01:30:43.098208+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:17:59.186810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.099380+0000","last_change":"2023-10-26T01:30:42.106926+0000","last_active":"2023-10-26T01:30:43.099380+0000","last_peered":"2023-10-26T01:30:43.099380+0000","last_clean":"2023-10-26T01:30:43.099380+0000","last_became_active":"2023-10-26T01:30:42.106692+0000","last_became_peered":"2023-10-26T01:30:42.106692+0000","last_unstale":"2023-10-26T01:30:43.099380+0000","last_undegraded":"2023-10-26T01:30:43.099380+0000","last_fullsized":"2023-10-26T01:30:43.099380+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:20:47.431800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.110477+0000","last_change":"2023-10-26T01:30:41.105282+0000","last_active":"2023-10-26T01:30:42.110477+0000","last_peered":"2023-10-26T01:30:42.110477+0000","last_clean":"2023-10-26T01:30:42.110477+0000","last_became_active":"2023-10-26T01:30:41.104553+0000","last_became_peered":"2023-10-26T01:30:41.104553+0000","last_unstale":"2023-10-26T01:30:42.110477+0000","last_undegraded":"2023-10-26T01:30:42.110477+0000","last_fullsized":"2023-10-26T01:30:42.110477+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:29:17.626922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.100164+0000","last_change":"2023-10-26T01:30:42.098532+0000","last_active":"2023-10-26T01:30:43.100164+0000","last_peered":"2023-10-26T01:30:43.100164+0000","last_clean":"2023-10-26T01:30:43.100164+0000","last_became_active":"2023-10-26T01:30:42.098301+0000","last_became_peered":"2023-10-26T01:30:42.098301+0000","last_unstale":"2023-10-26T01:30:43.100164+0000","last_undegraded":"2023-10-26T01:30:43.100164+0000","last_fullsized":"2023-10-26T01:30:43.100164+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:55:51.116465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.113350+0000","last_change":"2023-10-26T01:30:41.101188+0000","last_active":"2023-10-26T01:30:42.113350+0000","last_peered":"2023-10-26T01:30:42.113350+0000","last_clean":"2023-10-26T01:30:42.113350+0000","last_became_active":"2023-10-26T01:30:41.100977+0000","last_became_peered":"2023-10-26T01:30:41.100977+0000","last_unstale":"2023-10-26T01:30:42.113350+0000","last_undegraded":"2023-10-26T01:30:42.113350+0000","last_fullsized":"2023-10-26T01:30:42.113350+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:13:32.821792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.139789+0000","last_change":"2023-10-26T01:30:42.105438+0000","last_active":"2023-10-26T01:30:43.139789+0000","last_peered":"2023-10-26T01:30:43.139789+0000","last_clean":"2023-10-26T01:30:43.139789+0000","last_became_active":"2023-10-26T01:30:42.105168+0000","last_became_peered":"2023-10-26T01:30:42.105168+0000","last_unstale":"2023-10-26T01:30:43.139789+0000","last_undegraded":"2023-10-26T01:30:43.139789+0000","last_fullsized":"2023-10-26T01:30:43.139789+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:53:57.632838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.111212+0000","last_change":"2023-10-26T01:30:41.088679+0000","last_active":"2023-10-26T01:30:42.111212+0000","last_peered":"2023-10-26T01:30:42.111212+0000","last_clean":"2023-10-26T01:30:42.111212+0000","last_became_active":"2023-10-26T01:30:41.086354+0000","last_became_peered":"2023-10-26T01:30:41.086354+0000","last_unstale":"2023-10-26T01:30:42.111212+0000","last_undegraded":"2023-10-26T01:30:42.111212+0000","last_fullsized":"2023-10-26T01:30:42.111212+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:54:04.970133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.085216+0000","last_change":"2023-10-26T01:30:42.100090+0000","last_active":"2023-10-26T01:30:43.085216+0000","last_peered":"2023-10-26T01:30:43.085216+0000","last_clean":"2023-10-26T01:30:43.085216+0000","last_became_active":"2023-10-26T01:30:42.095593+0000","last_became_peered":"2023-10-26T01:30:42.095593+0000","last_unstale":"2023-10-26T01:30:43.085216+0000","last_undegraded":"2023-10-26T01:30:43.085216+0000","last_fullsized":"2023-10-26T01:30:43.085216+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:31:08.727626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.113466+0000","last_change":"2023-10-26T01:30:41.097404+0000","last_active":"2023-10-26T01:30:42.113466+0000","last_peered":"2023-10-26T01:30:42.113466+0000","last_clean":"2023-10-26T01:30:42.113466+0000","last_became_active":"2023-10-26T01:30:41.097074+0000","last_became_peered":"2023-10-26T01:30:41.097074+0000","last_unstale":"2023-10-26T01:30:42.113466+0000","last_undegraded":"2023-10-26T01:30:42.113466+0000","last_fullsized":"2023-10-26T01:30:42.113466+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:31:39.970444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":14,"seq":60129542149,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":74,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195237,"statfs":{"total":200000000,"available":199923243,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.05}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95399999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85899999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.23}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5609999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8069999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.29}]}]},{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":650,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194662,"statfs":{"total":200000000,"available":199333905,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94399999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0569999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.333}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2050000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.014}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.101}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4970000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":74,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195238,"statfs":{"total":200000000,"available":199923857,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0089999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.393}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4319999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.256}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4790000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.131}]}]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":73,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195239,"statfs":{"total":200000000,"available":199924765,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1990000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.367}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4399999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7190000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.117}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.639}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5009999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":73,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195238,"statfs":{"total":200000000,"available":199924729,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.751}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.635}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5169999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.276}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4850000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4490000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7430000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":13,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":73,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195239,"statfs":{"total":200000000,"available":199924855,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1000000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.171}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68400000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2210000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7869999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.597}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9039999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":73,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195238,"statfs":{"total":200000000,"available":199924695,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79900000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95599999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6180000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4470000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3069999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.391}]}]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":650,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194661,"statfs":{"total":200000000,"available":199333849,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.03}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.161}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.204}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2290000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0129999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.839}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3049999999999999}]}]}],"pool_statfs":[]}} 2023-10-26T01:30:51.647 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-26T01:30:51.877 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:51.874+0000 7fd58dce7640 1 -- 172.21.15.169:0/1540697153 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5880cdc30 msgr2=0x7fd58806ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:51.878 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:51.874+0000 7fd58dce7640 1 --2- 172.21.15.169:0/1540697153 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5880cdc30 0x7fd58806ad30 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd58806be60 tx=0x7fd58002c1d0 comp rx=0 tx=0).stop 2023-10-26T01:30:51.879 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:51.874+0000 7fd58dce7640 1 -- 172.21.15.169:0/1540697153 shutdown_connections 2023-10-26T01:30:51.879 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:51.874+0000 7fd58dce7640 1 --2- 172.21.15.169:0/1540697153 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5880cdc30 0x7fd58806ad30 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:51.879 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:51.874+0000 7fd58dce7640 1 --2- 172.21.15.169:0/1540697153 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd5880ac930 0x7fd5880cd6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:51.879 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:51.874+0000 7fd58dce7640 1 --2- 172.21.15.169:0/1540697153 >> v2:172.21.15.170:3300/0 conn(0x7fd5881592d0 0x7fd5880aa3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:51.880 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:51.874+0000 7fd58dce7640 1 -- 172.21.15.169:0/1540697153 >> 172.21.15.169:0/1540697153 conn(0x7fd588069930 msgr2=0x7fd588069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:51.880 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:51.874+0000 7fd58dce7640 1 -- 172.21.15.169:0/1540697153 shutdown_connections 2023-10-26T01:30:51.881 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:51.874+0000 7fd58dce7640 1 -- 172.21.15.169:0/1540697153 wait complete. 2023-10-26T01:30:51.883 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:51.876+0000 7fd58dce7640 1 Processor -- start 2023-10-26T01:30:51.884 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:51.876+0000 7fd58dce7640 1 -- start start 2023-10-26T01:30:51.884 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:51.877+0000 7fd58dce7640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd5880ac930 0x7fd588143c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:51.884 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:51.877+0000 7fd58dce7640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5880cdc30 0x7fd58815c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:51.884 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:51.877+0000 7fd58dce7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd5881592d0 0x7fd5881f5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:51.885 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:51.877+0000 7fd58dce7640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd588144170 con 0x7fd5880cdc30 2023-10-26T01:30:51.885 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:51.877+0000 7fd58dce7640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5880b1620 con 0x7fd5881592d0 2023-10-26T01:30:51.885 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:51.877+0000 7fd58dce7640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5880b1900 con 0x7fd5880ac930 2023-10-26T01:30:51.886 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:51.877+0000 7fd57ffff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5880cdc30 0x7fd58815c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:51.886 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:51.878+0000 7fd57ffff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5880cdc30 0x7fd58815c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:34050/0 (socket says 172.21.15.169:34050) 2023-10-26T01:30:51.886 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:51.878+0000 7fd57ffff640 1 -- 172.21.15.169:0/351980036 learned_addr learned my addr 172.21.15.169:0/351980036 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:30:51.887 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:51.878+0000 7fd587fff640 1 --2- 172.21.15.169:0/351980036 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd5881592d0 0x7fd5881f5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:51.887 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:51.878+0000 7fd5877fe640 1 --2- 172.21.15.169:0/351980036 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd5880ac930 0x7fd588143c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:51.888 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:51.878+0000 7fd57ffff640 1 -- 172.21.15.169:0/351980036 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd5880ac930 msgr2=0x7fd588143c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:51.888 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:51.878+0000 7fd57ffff640 1 --2- 172.21.15.169:0/351980036 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd5880ac930 0x7fd588143c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:51.888 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:51.878+0000 7fd57ffff640 1 -- 172.21.15.169:0/351980036 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd5881592d0 msgr2=0x7fd5881f5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:51.889 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:51.878+0000 7fd57ffff640 1 --2- 172.21.15.169:0/351980036 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd5881592d0 0x7fd5881f5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:51.889 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:51.878+0000 7fd57ffff640 1 -- 172.21.15.169:0/351980036 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd580009900 con 0x7fd5880cdc30 2023-10-26T01:30:51.889 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:51.878+0000 7fd587fff640 1 --2- 172.21.15.169:0/351980036 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd5881592d0 0x7fd5881f5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:30:51.890 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:51.878+0000 7fd5877fe640 1 --2- 172.21.15.169:0/351980036 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd5880ac930 0x7fd588143c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:30:51.890 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:51.878+0000 7fd57ffff640 1 --2- 172.21.15.169:0/351980036 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5880cdc30 0x7fd58815c220 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd57000b4a0 tx=0x7fd57000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:51.890 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:51.878+0000 7fd584ff9640 1 -- 172.21.15.169:0/351980036 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd570010610 con 0x7fd5880cdc30 2023-10-26T01:30:51.891 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:51.879+0000 7fd584ff9640 1 -- 172.21.15.169:0/351980036 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd570010860 con 0x7fd5880cdc30 2023-10-26T01:30:51.891 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:51.879+0000 7fd58dce7640 1 -- 172.21.15.169:0/351980036 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5880b1be0 con 0x7fd5880cdc30 2023-10-26T01:30:51.891 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:51.879+0000 7fd584ff9640 1 -- 172.21.15.169:0/351980036 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd570010c70 con 0x7fd5880cdc30 2023-10-26T01:30:51.892 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:51.879+0000 7fd58dce7640 1 -- 172.21.15.169:0/351980036 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5881f5590 con 0x7fd5880cdc30 2023-10-26T01:30:51.892 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:51.880+0000 7fd58dce7640 1 -- 172.21.15.169:0/351980036 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd58804db10 con 0x7fd5880cdc30 2023-10-26T01:30:51.892 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:51.881+0000 7fd584ff9640 1 -- 172.21.15.169:0/351980036 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fd570017360 con 0x7fd5880cdc30 2023-10-26T01:30:51.894 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:51.881+0000 7fd584ff9640 1 --2- 172.21.15.169:0/351980036 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd56806ae70 0x7fd56806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:51.895 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:51.882+0000 7fd584ff9640 1 -- 172.21.15.169:0/351980036 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd5700894e0 con 0x7fd5880cdc30 2023-10-26T01:30:51.895 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:51.882+0000 7fd5877fe640 1 --2- 172.21.15.169:0/351980036 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd56806ae70 0x7fd56806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:51.896 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:51.883+0000 7fd5877fe640 1 --2- 172.21.15.169:0/351980036 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd56806ae70 0x7fd56806d330 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd57800a870 tx=0x7fd578008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:51.896 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:51.886+0000 7fd584ff9640 1 -- 172.21.15.169:0/351980036 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd570058010 con 0x7fd5880cdc30 2023-10-26T01:30:52.073 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:52.070+0000 7fd58dce7640 1 -- 172.21.15.169:0/351980036 --> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd5880b1a70 con 0x7fd56806ae70 2023-10-26T01:30:52.093 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:52.091+0000 7fd584ff9640 1 -- 172.21.15.169:0/351980036 <== mgr.4112 v2:172.21.15.170:6832/1187156918 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+259986 (secure 0 0 0) 0x7fd5880b1a70 con 0x7fd56806ae70 2023-10-26T01:30:52.096 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:30:52.098 INFO:teuthology.orchestra.run.smithi169.stderr:dumped all 2023-10-26T01:30:52.099 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:52.095+0000 7fd58dce7640 1 -- 172.21.15.169:0/351980036 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd56806ae70 msgr2=0x7fd56806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:52.100 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:52.095+0000 7fd58dce7640 1 --2- 172.21.15.169:0/351980036 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd56806ae70 0x7fd56806d330 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd57800a870 tx=0x7fd578008040 comp rx=0 tx=0).stop 2023-10-26T01:30:52.100 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:52.095+0000 7fd58dce7640 1 -- 172.21.15.169:0/351980036 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5880cdc30 msgr2=0x7fd58815c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:52.100 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:52.095+0000 7fd58dce7640 1 --2- 172.21.15.169:0/351980036 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5880cdc30 0x7fd58815c220 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd57000b4a0 tx=0x7fd57000b960 comp rx=0 tx=0).stop 2023-10-26T01:30:52.100 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:52.095+0000 7fd58dce7640 1 -- 172.21.15.169:0/351980036 shutdown_connections 2023-10-26T01:30:52.100 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:52.095+0000 7fd58dce7640 1 --2- 172.21.15.169:0/351980036 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd56806ae70 0x7fd56806d330 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:52.101 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:52.095+0000 7fd58dce7640 1 --2- 172.21.15.169:0/351980036 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5880cdc30 0x7fd58815c220 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:52.101 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:52.095+0000 7fd58dce7640 1 --2- 172.21.15.169:0/351980036 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd5881592d0 0x7fd5881f5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:52.101 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:52.095+0000 7fd58dce7640 1 --2- 172.21.15.169:0/351980036 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd5880ac930 0x7fd588143c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:52.101 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:52.095+0000 7fd58dce7640 1 -- 172.21.15.169:0/351980036 >> 172.21.15.169:0/351980036 conn(0x7fd588069930 msgr2=0x7fd588153c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:52.101 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:52.096+0000 7fd58dce7640 1 -- 172.21.15.169:0/351980036 shutdown_connections 2023-10-26T01:30:52.102 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:52.096+0000 7fd58dce7640 1 -- 172.21.15.169:0/351980036 wait complete. 2023-10-26T01:30:52.118 INFO:teuthology.orchestra.run.smithi169.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-10-26T01:30:50.130627+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":1744,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560755,"statfs":{"total":1600000000,"available":1598213898,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.049481"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.086425+0000","last_change":"2023-10-26T01:30:42.112552+0000","last_active":"2023-10-26T01:30:43.086425+0000","last_peered":"2023-10-26T01:30:43.086425+0000","last_clean":"2023-10-26T01:30:43.086425+0000","last_became_active":"2023-10-26T01:30:42.111151+0000","last_became_peered":"2023-10-26T01:30:42.111151+0000","last_unstale":"2023-10-26T01:30:43.086425+0000","last_undegraded":"2023-10-26T01:30:43.086425+0000","last_fullsized":"2023-10-26T01:30:43.086425+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:28:22.384005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.141033+0000","last_change":"2023-10-26T01:30:42.102872+0000","last_active":"2023-10-26T01:30:43.141033+0000","last_peered":"2023-10-26T01:30:43.141033+0000","last_clean":"2023-10-26T01:30:43.141033+0000","last_became_active":"2023-10-26T01:30:42.100796+0000","last_became_peered":"2023-10-26T01:30:42.100796+0000","last_unstale":"2023-10-26T01:30:43.141033+0000","last_undegraded":"2023-10-26T01:30:43.141033+0000","last_fullsized":"2023-10-26T01:30:43.141033+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:49:12.753504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.139076+0000","last_change":"2023-10-26T01:30:42.103921+0000","last_active":"2023-10-26T01:30:43.139076+0000","last_peered":"2023-10-26T01:30:43.139076+0000","last_clean":"2023-10-26T01:30:43.139076+0000","last_became_active":"2023-10-26T01:30:42.103384+0000","last_became_peered":"2023-10-26T01:30:42.103384+0000","last_unstale":"2023-10-26T01:30:43.139076+0000","last_undegraded":"2023-10-26T01:30:43.139076+0000","last_fullsized":"2023-10-26T01:30:43.139076+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:33:58.768305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.098597+0000","last_change":"2023-10-26T01:30:42.101845+0000","last_active":"2023-10-26T01:30:43.098597+0000","last_peered":"2023-10-26T01:30:43.098597+0000","last_clean":"2023-10-26T01:30:43.098597+0000","last_became_active":"2023-10-26T01:30:42.097125+0000","last_became_peered":"2023-10-26T01:30:42.097125+0000","last_unstale":"2023-10-26T01:30:43.098597+0000","last_undegraded":"2023-10-26T01:30:43.098597+0000","last_fullsized":"2023-10-26T01:30:43.098597+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:33:13.615628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.086419+0000","last_change":"2023-10-26T01:30:42.115244+0000","last_active":"2023-10-26T01:30:43.086419+0000","last_peered":"2023-10-26T01:30:43.086419+0000","last_clean":"2023-10-26T01:30:43.086419+0000","last_became_active":"2023-10-26T01:30:42.114972+0000","last_became_peered":"2023-10-26T01:30:42.114972+0000","last_unstale":"2023-10-26T01:30:43.086419+0000","last_undegraded":"2023-10-26T01:30:43.086419+0000","last_fullsized":"2023-10-26T01:30:43.086419+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:07:06.450331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.099993+0000","last_change":"2023-10-26T01:30:42.108711+0000","last_active":"2023-10-26T01:30:43.099993+0000","last_peered":"2023-10-26T01:30:43.099993+0000","last_clean":"2023-10-26T01:30:43.099993+0000","last_became_active":"2023-10-26T01:30:42.108215+0000","last_became_peered":"2023-10-26T01:30:42.108215+0000","last_unstale":"2023-10-26T01:30:43.099993+0000","last_undegraded":"2023-10-26T01:30:43.099993+0000","last_fullsized":"2023-10-26T01:30:43.099993+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:26:01.898201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.088319+0000","last_change":"2023-10-26T01:30:42.115268+0000","last_active":"2023-10-26T01:30:43.088319+0000","last_peered":"2023-10-26T01:30:43.088319+0000","last_clean":"2023-10-26T01:30:43.088319+0000","last_became_active":"2023-10-26T01:30:42.113208+0000","last_became_peered":"2023-10-26T01:30:42.113208+0000","last_unstale":"2023-10-26T01:30:43.088319+0000","last_undegraded":"2023-10-26T01:30:43.088319+0000","last_fullsized":"2023-10-26T01:30:43.088319+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:56:55.394232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.084344+0000","last_change":"2023-10-26T01:30:42.093950+0000","last_active":"2023-10-26T01:30:43.084344+0000","last_peered":"2023-10-26T01:30:43.084344+0000","last_clean":"2023-10-26T01:30:43.084344+0000","last_became_active":"2023-10-26T01:30:42.093684+0000","last_became_peered":"2023-10-26T01:30:42.093684+0000","last_unstale":"2023-10-26T01:30:43.084344+0000","last_undegraded":"2023-10-26T01:30:43.084344+0000","last_fullsized":"2023-10-26T01:30:43.084344+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:05:31.590291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.086093+0000","last_change":"2023-10-26T01:30:42.108451+0000","last_active":"2023-10-26T01:30:43.086093+0000","last_peered":"2023-10-26T01:30:43.086093+0000","last_clean":"2023-10-26T01:30:43.086093+0000","last_became_active":"2023-10-26T01:30:42.107650+0000","last_became_peered":"2023-10-26T01:30:42.107650+0000","last_unstale":"2023-10-26T01:30:43.086093+0000","last_undegraded":"2023-10-26T01:30:43.086093+0000","last_fullsized":"2023-10-26T01:30:43.086093+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:26:19.035685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.098895+0000","last_change":"2023-10-26T01:30:42.093838+0000","last_active":"2023-10-26T01:30:43.098895+0000","last_peered":"2023-10-26T01:30:43.098895+0000","last_clean":"2023-10-26T01:30:43.098895+0000","last_became_active":"2023-10-26T01:30:42.093584+0000","last_became_peered":"2023-10-26T01:30:42.093584+0000","last_unstale":"2023-10-26T01:30:43.098895+0000","last_undegraded":"2023-10-26T01:30:43.098895+0000","last_fullsized":"2023-10-26T01:30:43.098895+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:49:12.753504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.097658+0000","last_change":"2023-10-26T01:30:42.094026+0000","last_active":"2023-10-26T01:30:43.097658+0000","last_peered":"2023-10-26T01:30:43.097658+0000","last_clean":"2023-10-26T01:30:43.097658+0000","last_became_active":"2023-10-26T01:30:42.093654+0000","last_became_peered":"2023-10-26T01:30:42.093654+0000","last_unstale":"2023-10-26T01:30:43.097658+0000","last_undegraded":"2023-10-26T01:30:43.097658+0000","last_fullsized":"2023-10-26T01:30:43.097658+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:05:31.590291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.088414+0000","last_change":"2023-10-26T01:30:42.099764+0000","last_active":"2023-10-26T01:30:43.088414+0000","last_peered":"2023-10-26T01:30:43.088414+0000","last_clean":"2023-10-26T01:30:43.088414+0000","last_became_active":"2023-10-26T01:30:42.098108+0000","last_became_peered":"2023-10-26T01:30:42.098108+0000","last_unstale":"2023-10-26T01:30:43.088414+0000","last_undegraded":"2023-10-26T01:30:43.088414+0000","last_fullsized":"2023-10-26T01:30:43.088414+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:08:21.563278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.085548+0000","last_change":"2023-10-26T01:30:42.097734+0000","last_active":"2023-10-26T01:30:43.085548+0000","last_peered":"2023-10-26T01:30:43.085548+0000","last_clean":"2023-10-26T01:30:43.085548+0000","last_became_active":"2023-10-26T01:30:42.097333+0000","last_became_peered":"2023-10-26T01:30:42.097333+0000","last_unstale":"2023-10-26T01:30:43.085548+0000","last_undegraded":"2023-10-26T01:30:43.085548+0000","last_fullsized":"2023-10-26T01:30:43.085548+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:28:22.384005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.138887+0000","last_change":"2023-10-26T01:30:42.108715+0000","last_active":"2023-10-26T01:30:43.138887+0000","last_peered":"2023-10-26T01:30:43.138887+0000","last_clean":"2023-10-26T01:30:43.138887+0000","last_became_active":"2023-10-26T01:30:42.108469+0000","last_became_peered":"2023-10-26T01:30:42.108469+0000","last_unstale":"2023-10-26T01:30:43.138887+0000","last_undegraded":"2023-10-26T01:30:43.138887+0000","last_fullsized":"2023-10-26T01:30:43.138887+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:45:23.061469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.140916+0000","last_change":"2023-10-26T01:30:42.099157+0000","last_active":"2023-10-26T01:30:43.140916+0000","last_peered":"2023-10-26T01:30:43.140916+0000","last_clean":"2023-10-26T01:30:43.140916+0000","last_became_active":"2023-10-26T01:30:42.098929+0000","last_became_peered":"2023-10-26T01:30:42.098929+0000","last_unstale":"2023-10-26T01:30:43.140916+0000","last_undegraded":"2023-10-26T01:30:43.140916+0000","last_fullsized":"2023-10-26T01:30:43.140916+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:13:33.838018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.099835+0000","last_change":"2023-10-26T01:30:42.108578+0000","last_active":"2023-10-26T01:30:43.099835+0000","last_peered":"2023-10-26T01:30:43.099835+0000","last_clean":"2023-10-26T01:30:43.099835+0000","last_became_active":"2023-10-26T01:30:42.107937+0000","last_became_peered":"2023-10-26T01:30:42.107937+0000","last_unstale":"2023-10-26T01:30:43.099835+0000","last_undegraded":"2023-10-26T01:30:43.099835+0000","last_fullsized":"2023-10-26T01:30:43.099835+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:38:16.726919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.098559+0000","last_change":"2023-10-26T01:30:42.108373+0000","last_active":"2023-10-26T01:30:43.098559+0000","last_peered":"2023-10-26T01:30:43.098559+0000","last_clean":"2023-10-26T01:30:43.098559+0000","last_became_active":"2023-10-26T01:30:42.107773+0000","last_became_peered":"2023-10-26T01:30:42.107773+0000","last_unstale":"2023-10-26T01:30:43.098559+0000","last_undegraded":"2023-10-26T01:30:43.098559+0000","last_fullsized":"2023-10-26T01:30:43.098559+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:49:12.753504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.097381+0000","last_change":"2023-10-26T01:30:42.094993+0000","last_active":"2023-10-26T01:30:43.097381+0000","last_peered":"2023-10-26T01:30:43.097381+0000","last_clean":"2023-10-26T01:30:43.097381+0000","last_became_active":"2023-10-26T01:30:42.094689+0000","last_became_peered":"2023-10-26T01:30:42.094689+0000","last_unstale":"2023-10-26T01:30:43.097381+0000","last_undegraded":"2023-10-26T01:30:43.097381+0000","last_fullsized":"2023-10-26T01:30:43.097381+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:49:43.468558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.086773+0000","last_change":"2023-10-26T01:30:42.108746+0000","last_active":"2023-10-26T01:30:43.086773+0000","last_peered":"2023-10-26T01:30:43.086773+0000","last_clean":"2023-10-26T01:30:43.086773+0000","last_became_active":"2023-10-26T01:30:42.108188+0000","last_became_peered":"2023-10-26T01:30:42.108188+0000","last_unstale":"2023-10-26T01:30:43.086773+0000","last_undegraded":"2023-10-26T01:30:43.086773+0000","last_fullsized":"2023-10-26T01:30:43.086773+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:38:16.726919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.099061+0000","last_change":"2023-10-26T01:30:42.110331+0000","last_active":"2023-10-26T01:30:43.099061+0000","last_peered":"2023-10-26T01:30:43.099061+0000","last_clean":"2023-10-26T01:30:43.099061+0000","last_became_active":"2023-10-26T01:30:42.110079+0000","last_became_peered":"2023-10-26T01:30:42.110079+0000","last_unstale":"2023-10-26T01:30:43.099061+0000","last_undegraded":"2023-10-26T01:30:43.099061+0000","last_fullsized":"2023-10-26T01:30:43.099061+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:33:13.615628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.139607+0000","last_change":"2023-10-26T01:30:42.108608+0000","last_active":"2023-10-26T01:30:43.139607+0000","last_peered":"2023-10-26T01:30:43.139607+0000","last_clean":"2023-10-26T01:30:43.139607+0000","last_became_active":"2023-10-26T01:30:42.108197+0000","last_became_peered":"2023-10-26T01:30:42.108197+0000","last_unstale":"2023-10-26T01:30:43.139607+0000","last_undegraded":"2023-10-26T01:30:43.139607+0000","last_fullsized":"2023-10-26T01:30:43.139607+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:17:59.186810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.098944+0000","last_change":"2023-10-26T01:30:42.110289+0000","last_active":"2023-10-26T01:30:43.098944+0000","last_peered":"2023-10-26T01:30:43.098944+0000","last_clean":"2023-10-26T01:30:43.098944+0000","last_became_active":"2023-10-26T01:30:42.110061+0000","last_became_peered":"2023-10-26T01:30:42.110061+0000","last_unstale":"2023-10-26T01:30:43.098944+0000","last_undegraded":"2023-10-26T01:30:43.098944+0000","last_fullsized":"2023-10-26T01:30:43.098944+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:19:23.544531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.084819+0000","last_change":"2023-10-26T01:30:42.108686+0000","last_active":"2023-10-26T01:30:43.084819+0000","last_peered":"2023-10-26T01:30:43.084819+0000","last_clean":"2023-10-26T01:30:43.084819+0000","last_became_active":"2023-10-26T01:30:42.108136+0000","last_became_peered":"2023-10-26T01:30:42.108136+0000","last_unstale":"2023-10-26T01:30:43.084819+0000","last_undegraded":"2023-10-26T01:30:43.084819+0000","last_fullsized":"2023-10-26T01:30:43.084819+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:13:33.838018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.098984+0000","last_change":"2023-10-26T01:30:42.102147+0000","last_active":"2023-10-26T01:30:43.098984+0000","last_peered":"2023-10-26T01:30:43.098984+0000","last_clean":"2023-10-26T01:30:43.098984+0000","last_became_active":"2023-10-26T01:30:42.097509+0000","last_became_peered":"2023-10-26T01:30:42.097509+0000","last_unstale":"2023-10-26T01:30:43.098984+0000","last_undegraded":"2023-10-26T01:30:43.098984+0000","last_fullsized":"2023-10-26T01:30:43.098984+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:26:19.035685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.085124+0000","last_change":"2023-10-26T01:30:42.108519+0000","last_active":"2023-10-26T01:30:43.085124+0000","last_peered":"2023-10-26T01:30:43.085124+0000","last_clean":"2023-10-26T01:30:43.085124+0000","last_became_active":"2023-10-26T01:30:42.108283+0000","last_became_peered":"2023-10-26T01:30:42.108283+0000","last_unstale":"2023-10-26T01:30:43.085124+0000","last_undegraded":"2023-10-26T01:30:43.085124+0000","last_fullsized":"2023-10-26T01:30:43.085124+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:33:13.615628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.097862+0000","last_change":"2023-10-26T01:30:42.111865+0000","last_active":"2023-10-26T01:30:43.097862+0000","last_peered":"2023-10-26T01:30:43.097862+0000","last_clean":"2023-10-26T01:30:43.097862+0000","last_became_active":"2023-10-26T01:30:42.111636+0000","last_became_peered":"2023-10-26T01:30:42.111636+0000","last_unstale":"2023-10-26T01:30:43.097862+0000","last_undegraded":"2023-10-26T01:30:43.097862+0000","last_fullsized":"2023-10-26T01:30:43.097862+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:49:56.641022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.141348+0000","last_change":"2023-10-26T01:30:42.091332+0000","last_active":"2023-10-26T01:30:43.141348+0000","last_peered":"2023-10-26T01:30:43.141348+0000","last_clean":"2023-10-26T01:30:43.141348+0000","last_became_active":"2023-10-26T01:30:42.091105+0000","last_became_peered":"2023-10-26T01:30:42.091105+0000","last_unstale":"2023-10-26T01:30:43.141348+0000","last_undegraded":"2023-10-26T01:30:43.141348+0000","last_fullsized":"2023-10-26T01:30:43.141348+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:33:13.615628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.085113+0000","last_change":"2023-10-26T01:30:42.108822+0000","last_active":"2023-10-26T01:30:43.085113+0000","last_peered":"2023-10-26T01:30:43.085113+0000","last_clean":"2023-10-26T01:30:43.085113+0000","last_became_active":"2023-10-26T01:30:42.108448+0000","last_became_peered":"2023-10-26T01:30:42.108448+0000","last_unstale":"2023-10-26T01:30:43.085113+0000","last_undegraded":"2023-10-26T01:30:43.085113+0000","last_fullsized":"2023-10-26T01:30:43.085113+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:49:12.753504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.139200+0000","last_change":"2023-10-26T01:30:42.101708+0000","last_active":"2023-10-26T01:30:43.139200+0000","last_peered":"2023-10-26T01:30:43.139200+0000","last_clean":"2023-10-26T01:30:43.139200+0000","last_became_active":"2023-10-26T01:30:42.101434+0000","last_became_peered":"2023-10-26T01:30:42.101434+0000","last_unstale":"2023-10-26T01:30:43.139200+0000","last_undegraded":"2023-10-26T01:30:43.139200+0000","last_fullsized":"2023-10-26T01:30:43.139200+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:29:18.643148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.099207+0000","last_change":"2023-10-26T01:30:42.097495+0000","last_active":"2023-10-26T01:30:43.099207+0000","last_peered":"2023-10-26T01:30:43.099207+0000","last_clean":"2023-10-26T01:30:43.099207+0000","last_became_active":"2023-10-26T01:30:42.097214+0000","last_became_peered":"2023-10-26T01:30:42.097214+0000","last_unstale":"2023-10-26T01:30:43.099207+0000","last_undegraded":"2023-10-26T01:30:43.099207+0000","last_fullsized":"2023-10-26T01:30:43.099207+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T01:55:28.470913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.097481+0000","last_change":"2023-10-26T01:30:42.093157+0000","last_active":"2023-10-26T01:30:43.097481+0000","last_peered":"2023-10-26T01:30:43.097481+0000","last_clean":"2023-10-26T01:30:43.097481+0000","last_became_active":"2023-10-26T01:30:42.092613+0000","last_became_peered":"2023-10-26T01:30:42.092613+0000","last_unstale":"2023-10-26T01:30:43.097481+0000","last_undegraded":"2023-10-26T01:30:43.097481+0000","last_fullsized":"2023-10-26T01:30:43.097481+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:09:01.061004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.098226+0000","last_change":"2023-10-26T01:30:42.102704+0000","last_active":"2023-10-26T01:30:43.098226+0000","last_peered":"2023-10-26T01:30:43.098226+0000","last_clean":"2023-10-26T01:30:43.098226+0000","last_became_active":"2023-10-26T01:30:42.102350+0000","last_became_peered":"2023-10-26T01:30:42.102350+0000","last_unstale":"2023-10-26T01:30:43.098226+0000","last_undegraded":"2023-10-26T01:30:43.098226+0000","last_fullsized":"2023-10-26T01:30:43.098226+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:53:57.632838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.086493+0000","last_change":"2023-10-26T01:30:42.093818+0000","last_active":"2023-10-26T01:30:43.086493+0000","last_peered":"2023-10-26T01:30:43.086493+0000","last_clean":"2023-10-26T01:30:43.086493+0000","last_became_active":"2023-10-26T01:30:42.091310+0000","last_became_peered":"2023-10-26T01:30:42.091310+0000","last_unstale":"2023-10-26T01:30:43.086493+0000","last_undegraded":"2023-10-26T01:30:43.086493+0000","last_fullsized":"2023-10-26T01:30:43.086493+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:04:23.158155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.114911+0000","last_change":"2023-10-26T01:30:41.097666+0000","last_active":"2023-10-26T01:30:42.114911+0000","last_peered":"2023-10-26T01:30:42.114911+0000","last_clean":"2023-10-26T01:30:42.114911+0000","last_became_active":"2023-10-26T01:30:41.093986+0000","last_became_peered":"2023-10-26T01:30:41.093986+0000","last_unstale":"2023-10-26T01:30:42.114911+0000","last_undegraded":"2023-10-26T01:30:42.114911+0000","last_fullsized":"2023-10-26T01:30:42.114911+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:49:42.452332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.139516+0000","last_change":"2023-10-26T01:30:42.110839+0000","last_active":"2023-10-26T01:30:43.139516+0000","last_peered":"2023-10-26T01:30:43.139516+0000","last_clean":"2023-10-26T01:30:43.139516+0000","last_became_active":"2023-10-26T01:30:42.109385+0000","last_became_peered":"2023-10-26T01:30:42.109385+0000","last_unstale":"2023-10-26T01:30:43.139516+0000","last_undegraded":"2023-10-26T01:30:43.139516+0000","last_fullsized":"2023-10-26T01:30:43.139516+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:49:56.641022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.105272+0000","last_change":"2023-10-26T01:30:41.100248+0000","last_active":"2023-10-26T01:30:42.105272+0000","last_peered":"2023-10-26T01:30:42.105272+0000","last_clean":"2023-10-26T01:30:42.105272+0000","last_became_active":"2023-10-26T01:30:41.099957+0000","last_became_peered":"2023-10-26T01:30:41.099957+0000","last_unstale":"2023-10-26T01:30:42.105272+0000","last_undegraded":"2023-10-26T01:30:42.105272+0000","last_fullsized":"2023-10-26T01:30:42.105272+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:29:17.626922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.088500+0000","last_change":"2023-10-26T01:30:41.103843+0000","last_active":"2023-10-26T01:30:42.088500+0000","last_peered":"2023-10-26T01:30:42.088500+0000","last_clean":"2023-10-26T01:30:42.088500+0000","last_became_active":"2023-10-26T01:30:41.103356+0000","last_became_peered":"2023-10-26T01:30:41.103356+0000","last_unstale":"2023-10-26T01:30:42.088500+0000","last_undegraded":"2023-10-26T01:30:42.088500+0000","last_fullsized":"2023-10-26T01:30:42.088500+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:40:15.199455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.085463+0000","last_change":"2023-10-26T01:30:42.102443+0000","last_active":"2023-10-26T01:30:43.085463+0000","last_peered":"2023-10-26T01:30:43.085463+0000","last_clean":"2023-10-26T01:30:43.085463+0000","last_became_active":"2023-10-26T01:30:42.102213+0000","last_became_peered":"2023-10-26T01:30:42.102213+0000","last_unstale":"2023-10-26T01:30:43.085463+0000","last_undegraded":"2023-10-26T01:30:43.085463+0000","last_fullsized":"2023-10-26T01:30:43.085463+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:26:19.035685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.089693+0000","last_change":"2023-10-26T01:30:41.102989+0000","last_active":"2023-10-26T01:30:42.089693+0000","last_peered":"2023-10-26T01:30:42.089693+0000","last_clean":"2023-10-26T01:30:42.089693+0000","last_became_active":"2023-10-26T01:30:41.102681+0000","last_became_peered":"2023-10-26T01:30:41.102681+0000","last_unstale":"2023-10-26T01:30:42.089693+0000","last_undegraded":"2023-10-26T01:30:42.089693+0000","last_fullsized":"2023-10-26T01:30:42.089693+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:49:55.624796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.140341+0000","last_change":"2023-10-26T01:30:42.091359+0000","last_active":"2023-10-26T01:30:43.140341+0000","last_peered":"2023-10-26T01:30:43.140341+0000","last_clean":"2023-10-26T01:30:43.140341+0000","last_became_active":"2023-10-26T01:30:42.090949+0000","last_became_peered":"2023-10-26T01:30:42.090949+0000","last_unstale":"2023-10-26T01:30:43.140341+0000","last_undegraded":"2023-10-26T01:30:43.140341+0000","last_fullsized":"2023-10-26T01:30:43.140341+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:49:56.641022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.089836+0000","last_change":"2023-10-26T01:30:41.104058+0000","last_active":"2023-10-26T01:30:42.089836+0000","last_peered":"2023-10-26T01:30:42.089836+0000","last_clean":"2023-10-26T01:30:42.089836+0000","last_became_active":"2023-10-26T01:30:41.103821+0000","last_became_peered":"2023-10-26T01:30:41.103821+0000","last_unstale":"2023-10-26T01:30:42.089836+0000","last_undegraded":"2023-10-26T01:30:42.089836+0000","last_fullsized":"2023-10-26T01:30:42.089836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:46:05.927163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.098774+0000","last_change":"2023-10-26T01:30:42.108520+0000","last_active":"2023-10-26T01:30:43.098774+0000","last_peered":"2023-10-26T01:30:43.098774+0000","last_clean":"2023-10-26T01:30:43.098774+0000","last_became_active":"2023-10-26T01:30:42.108140+0000","last_became_peered":"2023-10-26T01:30:42.108140+0000","last_unstale":"2023-10-26T01:30:43.098774+0000","last_undegraded":"2023-10-26T01:30:43.098774+0000","last_fullsized":"2023-10-26T01:30:43.098774+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:31:08.727626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.088126+0000","last_change":"2023-10-26T01:30:41.106779+0000","last_active":"2023-10-26T01:30:42.088126+0000","last_peered":"2023-10-26T01:30:42.088126+0000","last_clean":"2023-10-26T01:30:42.088126+0000","last_became_active":"2023-10-26T01:30:41.104353+0000","last_became_peered":"2023-10-26T01:30:41.104353+0000","last_unstale":"2023-10-26T01:30:42.088126+0000","last_undegraded":"2023-10-26T01:30:42.088126+0000","last_fullsized":"2023-10-26T01:30:42.088126+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:45:22.045243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.139887+0000","last_change":"2023-10-26T01:30:42.105447+0000","last_active":"2023-10-26T01:30:43.139887+0000","last_peered":"2023-10-26T01:30:43.139887+0000","last_clean":"2023-10-26T01:30:43.139887+0000","last_became_active":"2023-10-26T01:30:42.105098+0000","last_became_peered":"2023-10-26T01:30:42.105098+0000","last_unstale":"2023-10-26T01:30:43.139887+0000","last_undegraded":"2023-10-26T01:30:43.139887+0000","last_fullsized":"2023-10-26T01:30:43.139887+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:13:33.838018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.110602+0000","last_change":"2023-10-26T01:30:41.103826+0000","last_active":"2023-10-26T01:30:42.110602+0000","last_peered":"2023-10-26T01:30:42.110602+0000","last_clean":"2023-10-26T01:30:42.110602+0000","last_became_active":"2023-10-26T01:30:41.103552+0000","last_became_peered":"2023-10-26T01:30:41.103552+0000","last_unstale":"2023-10-26T01:30:42.110602+0000","last_undegraded":"2023-10-26T01:30:42.110602+0000","last_fullsized":"2023-10-26T01:30:42.110602+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:54:04.970133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.086144+0000","last_change":"2023-10-26T01:30:42.114167+0000","last_active":"2023-10-26T01:30:43.086144+0000","last_peered":"2023-10-26T01:30:43.086144+0000","last_clean":"2023-10-26T01:30:43.086144+0000","last_became_active":"2023-10-26T01:30:42.113241+0000","last_became_peered":"2023-10-26T01:30:42.113241+0000","last_unstale":"2023-10-26T01:30:43.086144+0000","last_undegraded":"2023-10-26T01:30:43.086144+0000","last_fullsized":"2023-10-26T01:30:43.086144+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:40:16.215681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.116049+0000","last_change":"2023-10-26T01:30:41.106514+0000","last_active":"2023-10-26T01:30:42.116049+0000","last_peered":"2023-10-26T01:30:42.116049+0000","last_clean":"2023-10-26T01:30:42.116049+0000","last_became_active":"2023-10-26T01:30:41.106147+0000","last_became_peered":"2023-10-26T01:30:41.106147+0000","last_unstale":"2023-10-26T01:30:42.116049+0000","last_undegraded":"2023-10-26T01:30:42.116049+0000","last_fullsized":"2023-10-26T01:30:42.116049+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:45:22.045243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.089920+0000","last_change":"2023-10-26T01:30:42.112813+0000","last_active":"2023-10-26T01:30:43.089920+0000","last_peered":"2023-10-26T01:30:43.089920+0000","last_clean":"2023-10-26T01:30:43.089920+0000","last_became_active":"2023-10-26T01:30:42.112571+0000","last_became_peered":"2023-10-26T01:30:42.112571+0000","last_unstale":"2023-10-26T01:30:43.089920+0000","last_undegraded":"2023-10-26T01:30:43.089920+0000","last_fullsized":"2023-10-26T01:30:43.089920+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:19:16.042472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.108474+0000","last_change":"2023-10-26T01:30:41.098865+0000","last_active":"2023-10-26T01:30:42.108474+0000","last_peered":"2023-10-26T01:30:42.108474+0000","last_clean":"2023-10-26T01:30:42.108474+0000","last_became_active":"2023-10-26T01:30:41.098506+0000","last_became_peered":"2023-10-26T01:30:41.098506+0000","last_unstale":"2023-10-26T01:30:42.108474+0000","last_undegraded":"2023-10-26T01:30:42.108474+0000","last_fullsized":"2023-10-26T01:30:42.108474+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:09:00.044778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.140188+0000","last_change":"2023-10-26T01:30:42.108443+0000","last_active":"2023-10-26T01:30:43.140188+0000","last_peered":"2023-10-26T01:30:43.140188+0000","last_clean":"2023-10-26T01:30:43.140188+0000","last_became_active":"2023-10-26T01:30:42.107837+0000","last_became_peered":"2023-10-26T01:30:42.107837+0000","last_unstale":"2023-10-26T01:30:43.140188+0000","last_undegraded":"2023-10-26T01:30:43.140188+0000","last_fullsized":"2023-10-26T01:30:43.140188+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:33:13.615628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.090761+0000","last_change":"2023-10-26T01:30:41.106795+0000","last_active":"2023-10-26T01:30:42.090761+0000","last_peered":"2023-10-26T01:30:42.090761+0000","last_clean":"2023-10-26T01:30:42.090761+0000","last_became_active":"2023-10-26T01:30:41.106222+0000","last_became_peered":"2023-10-26T01:30:41.106222+0000","last_unstale":"2023-10-26T01:30:42.090761+0000","last_undegraded":"2023-10-26T01:30:42.090761+0000","last_fullsized":"2023-10-26T01:30:42.090761+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:53:56.616612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.086822+0000","last_change":"2023-10-26T01:30:42.113053+0000","last_active":"2023-10-26T01:30:43.086822+0000","last_peered":"2023-10-26T01:30:43.086822+0000","last_clean":"2023-10-26T01:30:43.086822+0000","last_became_active":"2023-10-26T01:30:42.112492+0000","last_became_peered":"2023-10-26T01:30:42.112492+0000","last_unstale":"2023-10-26T01:30:43.086822+0000","last_undegraded":"2023-10-26T01:30:43.086822+0000","last_fullsized":"2023-10-26T01:30:43.086822+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:20:47.431800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.089525+0000","last_change":"2023-10-26T01:30:41.100835+0000","last_active":"2023-10-26T01:30:42.089525+0000","last_peered":"2023-10-26T01:30:42.089525+0000","last_clean":"2023-10-26T01:30:42.089525+0000","last_became_active":"2023-10-26T01:30:41.099708+0000","last_became_peered":"2023-10-26T01:30:41.099708+0000","last_unstale":"2023-10-26T01:30:42.089525+0000","last_undegraded":"2023-10-26T01:30:42.089525+0000","last_fullsized":"2023-10-26T01:30:42.089525+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:04:22.141930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.098337+0000","last_change":"2023-10-26T01:30:42.100857+0000","last_active":"2023-10-26T01:30:43.098337+0000","last_peered":"2023-10-26T01:30:43.098337+0000","last_clean":"2023-10-26T01:30:43.098337+0000","last_became_active":"2023-10-26T01:30:42.100545+0000","last_became_peered":"2023-10-26T01:30:42.100545+0000","last_unstale":"2023-10-26T01:30:43.098337+0000","last_undegraded":"2023-10-26T01:30:43.098337+0000","last_fullsized":"2023-10-26T01:30:43.098337+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:13:33.838018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.114399+0000","last_change":"2023-10-26T01:30:41.095998+0000","last_active":"2023-10-26T01:30:42.114399+0000","last_peered":"2023-10-26T01:30:42.114399+0000","last_clean":"2023-10-26T01:30:42.114399+0000","last_became_active":"2023-10-26T01:30:41.095494+0000","last_became_peered":"2023-10-26T01:30:41.095494+0000","last_unstale":"2023-10-26T01:30:42.114399+0000","last_undegraded":"2023-10-26T01:30:42.114399+0000","last_fullsized":"2023-10-26T01:30:42.114399+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:54:04.970133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.089073+0000","last_change":"2023-10-26T01:30:42.115510+0000","last_active":"2023-10-26T01:30:43.089073+0000","last_peered":"2023-10-26T01:30:43.089073+0000","last_clean":"2023-10-26T01:30:43.089073+0000","last_became_active":"2023-10-26T01:30:42.113552+0000","last_became_peered":"2023-10-26T01:30:42.113552+0000","last_unstale":"2023-10-26T01:30:43.089073+0000","last_undegraded":"2023-10-26T01:30:43.089073+0000","last_fullsized":"2023-10-26T01:30:43.089073+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:32:08.387966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.110967+0000","last_change":"2023-10-26T01:30:41.093200+0000","last_active":"2023-10-26T01:30:42.110967+0000","last_peered":"2023-10-26T01:30:42.110967+0000","last_clean":"2023-10-26T01:30:42.110967+0000","last_became_active":"2023-10-26T01:30:41.092917+0000","last_became_peered":"2023-10-26T01:30:41.092917+0000","last_unstale":"2023-10-26T01:30:42.110967+0000","last_undegraded":"2023-10-26T01:30:42.110967+0000","last_fullsized":"2023-10-26T01:30:42.110967+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:54:04.970133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.087143+0000","last_change":"2023-10-26T01:30:42.113216+0000","last_active":"2023-10-26T01:30:43.087143+0000","last_peered":"2023-10-26T01:30:43.087143+0000","last_clean":"2023-10-26T01:30:43.087143+0000","last_became_active":"2023-10-26T01:30:42.112815+0000","last_became_peered":"2023-10-26T01:30:42.112815+0000","last_unstale":"2023-10-26T01:30:43.087143+0000","last_undegraded":"2023-10-26T01:30:43.087143+0000","last_fullsized":"2023-10-26T01:30:43.087143+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:55:51.116465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.114880+0000","last_change":"2023-10-26T01:30:41.102365+0000","last_active":"2023-10-26T01:30:42.114880+0000","last_peered":"2023-10-26T01:30:42.114880+0000","last_clean":"2023-10-26T01:30:42.114880+0000","last_became_active":"2023-10-26T01:30:41.102135+0000","last_became_peered":"2023-10-26T01:30:41.102135+0000","last_unstale":"2023-10-26T01:30:42.114880+0000","last_undegraded":"2023-10-26T01:30:42.114880+0000","last_fullsized":"2023-10-26T01:30:42.114880+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:08:20.547052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.086427+0000","last_change":"2023-10-26T01:30:42.108605+0000","last_active":"2023-10-26T01:30:43.086427+0000","last_peered":"2023-10-26T01:30:43.086427+0000","last_clean":"2023-10-26T01:30:43.086427+0000","last_became_active":"2023-10-26T01:30:42.107907+0000","last_became_peered":"2023-10-26T01:30:42.107907+0000","last_unstale":"2023-10-26T01:30:43.086427+0000","last_undegraded":"2023-10-26T01:30:43.086427+0000","last_fullsized":"2023-10-26T01:30:43.086427+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T01:55:28.470913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.105745+0000","last_change":"2023-10-26T01:30:41.093456+0000","last_active":"2023-10-26T01:30:42.105745+0000","last_peered":"2023-10-26T01:30:42.105745+0000","last_clean":"2023-10-26T01:30:42.105745+0000","last_became_active":"2023-10-26T01:30:41.092811+0000","last_became_peered":"2023-10-26T01:30:41.092811+0000","last_unstale":"2023-10-26T01:30:42.105745+0000","last_undegraded":"2023-10-26T01:30:42.105745+0000","last_fullsized":"2023-10-26T01:30:42.105745+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:04:22.141930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.099118+0000","last_change":"2023-10-26T01:30:42.097121+0000","last_active":"2023-10-26T01:30:43.099118+0000","last_peered":"2023-10-26T01:30:43.099118+0000","last_clean":"2023-10-26T01:30:43.099118+0000","last_became_active":"2023-10-26T01:30:42.096784+0000","last_became_peered":"2023-10-26T01:30:42.096784+0000","last_unstale":"2023-10-26T01:30:43.099118+0000","last_undegraded":"2023-10-26T01:30:43.099118+0000","last_fullsized":"2023-10-26T01:30:43.099118+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:28:22.384005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.083700+0000","last_change":"2023-10-26T01:30:41.106632+0000","last_active":"2023-10-26T01:30:42.083700+0000","last_peered":"2023-10-26T01:30:42.083700+0000","last_clean":"2023-10-26T01:30:42.083700+0000","last_became_active":"2023-10-26T01:30:41.106360+0000","last_became_peered":"2023-10-26T01:30:41.106360+0000","last_unstale":"2023-10-26T01:30:42.083700+0000","last_undegraded":"2023-10-26T01:30:42.083700+0000","last_fullsized":"2023-10-26T01:30:42.083700+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:40:15.199455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.086951+0000","last_change":"2023-10-26T01:30:42.109475+0000","last_active":"2023-10-26T01:30:43.086951+0000","last_peered":"2023-10-26T01:30:43.086951+0000","last_clean":"2023-10-26T01:30:43.086951+0000","last_became_active":"2023-10-26T01:30:42.109232+0000","last_became_peered":"2023-10-26T01:30:42.109232+0000","last_unstale":"2023-10-26T01:30:43.086951+0000","last_undegraded":"2023-10-26T01:30:43.086951+0000","last_fullsized":"2023-10-26T01:30:43.086951+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:26:01.898201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":165,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.096979+0000","last_change":"2023-10-26T01:30:34.883484+0000","last_active":"2023-10-26T01:30:42.096979+0000","last_peered":"2023-10-26T01:30:42.096979+0000","last_clean":"2023-10-26T01:30:42.096979+0000","last_became_active":"2023-10-26T01:30:34.883071+0000","last_became_peered":"2023-10-26T01:30:34.883071+0000","last_unstale":"2023-10-26T01:30:42.096979+0000","last_undegraded":"2023-10-26T01:30:42.096979+0000","last_fullsized":"2023-10-26T01:30:42.096979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:32.890096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:32.890096+0000","last_clean_scrub_stamp":"2023-10-26T01:30:32.890096+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:22:57.937891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.090888+0000","last_change":"2023-10-26T01:30:41.105687+0000","last_active":"2023-10-26T01:30:42.090888+0000","last_peered":"2023-10-26T01:30:42.090888+0000","last_clean":"2023-10-26T01:30:42.090888+0000","last_became_active":"2023-10-26T01:30:41.105235+0000","last_became_peered":"2023-10-26T01:30:41.105235+0000","last_unstale":"2023-10-26T01:30:42.090888+0000","last_undegraded":"2023-10-26T01:30:42.090888+0000","last_fullsized":"2023-10-26T01:30:42.090888+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:13:32.821792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.084569+0000","last_change":"2023-10-26T01:30:42.093817+0000","last_active":"2023-10-26T01:30:43.084569+0000","last_peered":"2023-10-26T01:30:43.084569+0000","last_clean":"2023-10-26T01:30:43.084569+0000","last_became_active":"2023-10-26T01:30:42.093586+0000","last_became_peered":"2023-10-26T01:30:42.093586+0000","last_unstale":"2023-10-26T01:30:43.084569+0000","last_undegraded":"2023-10-26T01:30:43.084569+0000","last_fullsized":"2023-10-26T01:30:43.084569+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:17:59.186810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.096890+0000","last_change":"2023-10-26T01:30:41.094664+0000","last_active":"2023-10-26T01:30:42.096890+0000","last_peered":"2023-10-26T01:30:42.096890+0000","last_clean":"2023-10-26T01:30:42.096890+0000","last_became_active":"2023-10-26T01:30:41.092745+0000","last_became_peered":"2023-10-26T01:30:41.092745+0000","last_unstale":"2023-10-26T01:30:42.096890+0000","last_undegraded":"2023-10-26T01:30:42.096890+0000","last_fullsized":"2023-10-26T01:30:42.096890+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:45:22.045243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.140108+0000","last_change":"2023-10-26T01:30:42.107538+0000","last_active":"2023-10-26T01:30:43.140108+0000","last_peered":"2023-10-26T01:30:43.140108+0000","last_clean":"2023-10-26T01:30:43.140108+0000","last_became_active":"2023-10-26T01:30:42.107258+0000","last_became_peered":"2023-10-26T01:30:42.107258+0000","last_unstale":"2023-10-26T01:30:43.140108+0000","last_undegraded":"2023-10-26T01:30:43.140108+0000","last_fullsized":"2023-10-26T01:30:43.140108+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:49:12.753504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.083749+0000","last_change":"2023-10-26T01:30:41.099543+0000","last_active":"2023-10-26T01:30:42.083749+0000","last_peered":"2023-10-26T01:30:42.083749+0000","last_clean":"2023-10-26T01:30:42.083749+0000","last_became_active":"2023-10-26T01:30:41.099277+0000","last_became_peered":"2023-10-26T01:30:41.099277+0000","last_unstale":"2023-10-26T01:30:42.083749+0000","last_undegraded":"2023-10-26T01:30:42.083749+0000","last_fullsized":"2023-10-26T01:30:42.083749+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:04:22.141930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.140694+0000","last_change":"2023-10-26T01:30:42.085871+0000","last_active":"2023-10-26T01:30:43.140694+0000","last_peered":"2023-10-26T01:30:43.140694+0000","last_clean":"2023-10-26T01:30:43.140694+0000","last_became_active":"2023-10-26T01:30:42.085409+0000","last_became_peered":"2023-10-26T01:30:42.085409+0000","last_unstale":"2023-10-26T01:30:43.140694+0000","last_undegraded":"2023-10-26T01:30:43.140694+0000","last_fullsized":"2023-10-26T01:30:43.140694+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:17:59.186810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.088251+0000","last_change":"2023-10-26T01:30:41.107609+0000","last_active":"2023-10-26T01:30:42.088251+0000","last_peered":"2023-10-26T01:30:42.088251+0000","last_clean":"2023-10-26T01:30:42.088251+0000","last_became_active":"2023-10-26T01:30:41.107189+0000","last_became_peered":"2023-10-26T01:30:41.107189+0000","last_unstale":"2023-10-26T01:30:42.088251+0000","last_undegraded":"2023-10-26T01:30:42.088251+0000","last_fullsized":"2023-10-26T01:30:42.088251+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:33:57.752079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.088414+0000","last_change":"2023-10-26T01:30:42.105495+0000","last_active":"2023-10-26T01:30:43.088414+0000","last_peered":"2023-10-26T01:30:43.088414+0000","last_clean":"2023-10-26T01:30:43.088414+0000","last_became_active":"2023-10-26T01:30:42.105247+0000","last_became_peered":"2023-10-26T01:30:42.105247+0000","last_unstale":"2023-10-26T01:30:43.088414+0000","last_undegraded":"2023-10-26T01:30:43.088414+0000","last_fullsized":"2023-10-26T01:30:43.088414+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:24:17.679373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.088367+0000","last_change":"2023-10-26T01:30:41.096866+0000","last_active":"2023-10-26T01:30:42.088367+0000","last_peered":"2023-10-26T01:30:42.088367+0000","last_clean":"2023-10-26T01:30:42.088367+0000","last_became_active":"2023-10-26T01:30:41.096255+0000","last_became_peered":"2023-10-26T01:30:41.096255+0000","last_unstale":"2023-10-26T01:30:42.088367+0000","last_undegraded":"2023-10-26T01:30:42.088367+0000","last_fullsized":"2023-10-26T01:30:42.088367+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:29:17.626922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.099320+0000","last_change":"2023-10-26T01:30:42.102298+0000","last_active":"2023-10-26T01:30:43.099320+0000","last_peered":"2023-10-26T01:30:43.099320+0000","last_clean":"2023-10-26T01:30:43.099320+0000","last_became_active":"2023-10-26T01:30:42.097872+0000","last_became_peered":"2023-10-26T01:30:42.097872+0000","last_unstale":"2023-10-26T01:30:43.099320+0000","last_undegraded":"2023-10-26T01:30:43.099320+0000","last_fullsized":"2023-10-26T01:30:43.099320+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:07:06.450331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.096762+0000","last_change":"2023-10-26T01:30:41.096310+0000","last_active":"2023-10-26T01:30:42.096762+0000","last_peered":"2023-10-26T01:30:42.096762+0000","last_clean":"2023-10-26T01:30:42.096762+0000","last_became_active":"2023-10-26T01:30:41.095929+0000","last_became_peered":"2023-10-26T01:30:41.095929+0000","last_unstale":"2023-10-26T01:30:42.096762+0000","last_undegraded":"2023-10-26T01:30:42.096762+0000","last_fullsized":"2023-10-26T01:30:42.096762+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:31:39.970444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.140793+0000","last_change":"2023-10-26T01:30:42.089331+0000","last_active":"2023-10-26T01:30:43.140793+0000","last_peered":"2023-10-26T01:30:43.140793+0000","last_clean":"2023-10-26T01:30:43.140793+0000","last_became_active":"2023-10-26T01:30:42.088901+0000","last_became_peered":"2023-10-26T01:30:42.088901+0000","last_unstale":"2023-10-26T01:30:43.140793+0000","last_undegraded":"2023-10-26T01:30:43.140793+0000","last_fullsized":"2023-10-26T01:30:43.140793+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:53:57.632838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.082315+0000","last_change":"2023-10-26T01:30:41.102617+0000","last_active":"2023-10-26T01:30:42.082315+0000","last_peered":"2023-10-26T01:30:42.082315+0000","last_clean":"2023-10-26T01:30:42.082315+0000","last_became_active":"2023-10-26T01:30:41.102376+0000","last_became_peered":"2023-10-26T01:30:41.102376+0000","last_unstale":"2023-10-26T01:30:42.082315+0000","last_undegraded":"2023-10-26T01:30:42.082315+0000","last_fullsized":"2023-10-26T01:30:42.082315+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:24:16.663147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.084424+0000","last_change":"2023-10-26T01:30:42.112237+0000","last_active":"2023-10-26T01:30:43.084424+0000","last_peered":"2023-10-26T01:30:43.084424+0000","last_clean":"2023-10-26T01:30:43.084424+0000","last_became_active":"2023-10-26T01:30:42.112006+0000","last_became_peered":"2023-10-26T01:30:42.112006+0000","last_unstale":"2023-10-26T01:30:43.084424+0000","last_undegraded":"2023-10-26T01:30:43.084424+0000","last_fullsized":"2023-10-26T01:30:43.084424+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:49:56.641022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.112282+0000","last_change":"2023-10-26T01:30:41.102404+0000","last_active":"2023-10-26T01:30:42.112282+0000","last_peered":"2023-10-26T01:30:42.112282+0000","last_clean":"2023-10-26T01:30:42.112282+0000","last_became_active":"2023-10-26T01:30:41.102165+0000","last_became_peered":"2023-10-26T01:30:41.102165+0000","last_unstale":"2023-10-26T01:30:42.112282+0000","last_undegraded":"2023-10-26T01:30:42.112282+0000","last_fullsized":"2023-10-26T01:30:42.112282+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:08:20.547052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.139422+0000","last_change":"2023-10-26T01:30:42.103630+0000","last_active":"2023-10-26T01:30:43.139422+0000","last_peered":"2023-10-26T01:30:43.139422+0000","last_clean":"2023-10-26T01:30:43.139422+0000","last_became_active":"2023-10-26T01:30:42.099458+0000","last_became_peered":"2023-10-26T01:30:42.099458+0000","last_unstale":"2023-10-26T01:30:43.139422+0000","last_undegraded":"2023-10-26T01:30:43.139422+0000","last_fullsized":"2023-10-26T01:30:43.139422+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:05:31.590291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.097650+0000","last_change":"2023-10-26T01:30:41.097301+0000","last_active":"2023-10-26T01:30:42.097650+0000","last_peered":"2023-10-26T01:30:42.097650+0000","last_clean":"2023-10-26T01:30:42.097650+0000","last_became_active":"2023-10-26T01:30:41.096866+0000","last_became_peered":"2023-10-26T01:30:41.096866+0000","last_unstale":"2023-10-26T01:30:42.097650+0000","last_undegraded":"2023-10-26T01:30:42.097650+0000","last_fullsized":"2023-10-26T01:30:42.097650+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:05:30.574065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.089679+0000","last_change":"2023-10-26T01:30:42.087483+0000","last_active":"2023-10-26T01:30:43.089679+0000","last_peered":"2023-10-26T01:30:43.089679+0000","last_clean":"2023-10-26T01:30:43.089679+0000","last_became_active":"2023-10-26T01:30:42.087234+0000","last_became_peered":"2023-10-26T01:30:42.087234+0000","last_unstale":"2023-10-26T01:30:43.089679+0000","last_undegraded":"2023-10-26T01:30:43.089679+0000","last_fullsized":"2023-10-26T01:30:43.089679+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:46:06.943389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.088240+0000","last_change":"2023-10-26T01:30:41.097032+0000","last_active":"2023-10-26T01:30:42.088240+0000","last_peered":"2023-10-26T01:30:42.088240+0000","last_clean":"2023-10-26T01:30:42.088240+0000","last_became_active":"2023-10-26T01:30:41.096591+0000","last_became_peered":"2023-10-26T01:30:41.096591+0000","last_unstale":"2023-10-26T01:30:42.088240+0000","last_undegraded":"2023-10-26T01:30:42.088240+0000","last_fullsized":"2023-10-26T01:30:42.088240+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:33:57.752079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.084667+0000","last_change":"2023-10-26T01:30:42.101380+0000","last_active":"2023-10-26T01:30:43.084667+0000","last_peered":"2023-10-26T01:30:43.084667+0000","last_clean":"2023-10-26T01:30:43.084667+0000","last_became_active":"2023-10-26T01:30:42.101179+0000","last_became_peered":"2023-10-26T01:30:42.101179+0000","last_unstale":"2023-10-26T01:30:43.084667+0000","last_undegraded":"2023-10-26T01:30:43.084667+0000","last_fullsized":"2023-10-26T01:30:43.084667+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:53:57.632838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.084678+0000","last_change":"2023-10-26T01:30:41.101587+0000","last_active":"2023-10-26T01:30:42.084678+0000","last_peered":"2023-10-26T01:30:42.084678+0000","last_clean":"2023-10-26T01:30:42.084678+0000","last_became_active":"2023-10-26T01:30:41.101325+0000","last_became_peered":"2023-10-26T01:30:41.101325+0000","last_unstale":"2023-10-26T01:30:42.084678+0000","last_undegraded":"2023-10-26T01:30:42.084678+0000","last_fullsized":"2023-10-26T01:30:42.084678+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:49:42.452332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.085284+0000","last_change":"2023-10-26T01:30:42.103918+0000","last_active":"2023-10-26T01:30:43.085284+0000","last_peered":"2023-10-26T01:30:43.085284+0000","last_clean":"2023-10-26T01:30:43.085284+0000","last_became_active":"2023-10-26T01:30:42.103701+0000","last_became_peered":"2023-10-26T01:30:42.103701+0000","last_unstale":"2023-10-26T01:30:43.085284+0000","last_undegraded":"2023-10-26T01:30:43.085284+0000","last_fullsized":"2023-10-26T01:30:43.085284+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:19:23.544531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.114450+0000","last_change":"2023-10-26T01:30:41.095274+0000","last_active":"2023-10-26T01:30:42.114450+0000","last_peered":"2023-10-26T01:30:42.114450+0000","last_clean":"2023-10-26T01:30:42.114450+0000","last_became_active":"2023-10-26T01:30:41.092922+0000","last_became_peered":"2023-10-26T01:30:41.092922+0000","last_unstale":"2023-10-26T01:30:42.114450+0000","last_undegraded":"2023-10-26T01:30:42.114450+0000","last_fullsized":"2023-10-26T01:30:42.114450+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:33:12.599402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.098208+0000","last_change":"2023-10-26T01:30:42.105139+0000","last_active":"2023-10-26T01:30:43.098208+0000","last_peered":"2023-10-26T01:30:43.098208+0000","last_clean":"2023-10-26T01:30:43.098208+0000","last_became_active":"2023-10-26T01:30:42.104913+0000","last_became_peered":"2023-10-26T01:30:42.104913+0000","last_unstale":"2023-10-26T01:30:43.098208+0000","last_undegraded":"2023-10-26T01:30:43.098208+0000","last_fullsized":"2023-10-26T01:30:43.098208+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:17:59.186810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.099380+0000","last_change":"2023-10-26T01:30:42.106926+0000","last_active":"2023-10-26T01:30:43.099380+0000","last_peered":"2023-10-26T01:30:43.099380+0000","last_clean":"2023-10-26T01:30:43.099380+0000","last_became_active":"2023-10-26T01:30:42.106692+0000","last_became_peered":"2023-10-26T01:30:42.106692+0000","last_unstale":"2023-10-26T01:30:43.099380+0000","last_undegraded":"2023-10-26T01:30:43.099380+0000","last_fullsized":"2023-10-26T01:30:43.099380+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:20:47.431800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.110477+0000","last_change":"2023-10-26T01:30:41.105282+0000","last_active":"2023-10-26T01:30:42.110477+0000","last_peered":"2023-10-26T01:30:42.110477+0000","last_clean":"2023-10-26T01:30:42.110477+0000","last_became_active":"2023-10-26T01:30:41.104553+0000","last_became_peered":"2023-10-26T01:30:41.104553+0000","last_unstale":"2023-10-26T01:30:42.110477+0000","last_undegraded":"2023-10-26T01:30:42.110477+0000","last_fullsized":"2023-10-26T01:30:42.110477+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:29:17.626922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.100164+0000","last_change":"2023-10-26T01:30:42.098532+0000","last_active":"2023-10-26T01:30:43.100164+0000","last_peered":"2023-10-26T01:30:43.100164+0000","last_clean":"2023-10-26T01:30:43.100164+0000","last_became_active":"2023-10-26T01:30:42.098301+0000","last_became_peered":"2023-10-26T01:30:42.098301+0000","last_unstale":"2023-10-26T01:30:43.100164+0000","last_undegraded":"2023-10-26T01:30:43.100164+0000","last_fullsized":"2023-10-26T01:30:43.100164+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:55:51.116465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.113350+0000","last_change":"2023-10-26T01:30:41.101188+0000","last_active":"2023-10-26T01:30:42.113350+0000","last_peered":"2023-10-26T01:30:42.113350+0000","last_clean":"2023-10-26T01:30:42.113350+0000","last_became_active":"2023-10-26T01:30:41.100977+0000","last_became_peered":"2023-10-26T01:30:41.100977+0000","last_unstale":"2023-10-26T01:30:42.113350+0000","last_undegraded":"2023-10-26T01:30:42.113350+0000","last_fullsized":"2023-10-26T01:30:42.113350+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:13:32.821792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.139789+0000","last_change":"2023-10-26T01:30:42.105438+0000","last_active":"2023-10-26T01:30:43.139789+0000","last_peered":"2023-10-26T01:30:43.139789+0000","last_clean":"2023-10-26T01:30:43.139789+0000","last_became_active":"2023-10-26T01:30:42.105168+0000","last_became_peered":"2023-10-26T01:30:42.105168+0000","last_unstale":"2023-10-26T01:30:43.139789+0000","last_undegraded":"2023-10-26T01:30:43.139789+0000","last_fullsized":"2023-10-26T01:30:43.139789+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:53:57.632838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.111212+0000","last_change":"2023-10-26T01:30:41.088679+0000","last_active":"2023-10-26T01:30:42.111212+0000","last_peered":"2023-10-26T01:30:42.111212+0000","last_clean":"2023-10-26T01:30:42.111212+0000","last_became_active":"2023-10-26T01:30:41.086354+0000","last_became_peered":"2023-10-26T01:30:41.086354+0000","last_unstale":"2023-10-26T01:30:42.111212+0000","last_undegraded":"2023-10-26T01:30:42.111212+0000","last_fullsized":"2023-10-26T01:30:42.111212+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:54:04.970133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.085216+0000","last_change":"2023-10-26T01:30:42.100090+0000","last_active":"2023-10-26T01:30:43.085216+0000","last_peered":"2023-10-26T01:30:43.085216+0000","last_clean":"2023-10-26T01:30:43.085216+0000","last_became_active":"2023-10-26T01:30:42.095593+0000","last_became_peered":"2023-10-26T01:30:42.095593+0000","last_unstale":"2023-10-26T01:30:43.085216+0000","last_undegraded":"2023-10-26T01:30:43.085216+0000","last_fullsized":"2023-10-26T01:30:43.085216+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:31:08.727626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.113466+0000","last_change":"2023-10-26T01:30:41.097404+0000","last_active":"2023-10-26T01:30:42.113466+0000","last_peered":"2023-10-26T01:30:42.113466+0000","last_clean":"2023-10-26T01:30:42.113466+0000","last_became_active":"2023-10-26T01:30:41.097074+0000","last_became_peered":"2023-10-26T01:30:41.097074+0000","last_unstale":"2023-10-26T01:30:42.113466+0000","last_undegraded":"2023-10-26T01:30:42.113466+0000","last_fullsized":"2023-10-26T01:30:42.113466+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:31:39.970444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":14,"seq":60129542149,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":74,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195237,"statfs":{"total":200000000,"available":199923243,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.05}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95399999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85899999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.23}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5609999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8069999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.29}]}]},{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":650,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194662,"statfs":{"total":200000000,"available":199333905,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94399999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0569999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.333}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2050000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.014}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.101}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4970000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":74,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195238,"statfs":{"total":200000000,"available":199923857,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0089999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.393}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4319999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.256}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4790000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.131}]}]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":73,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195239,"statfs":{"total":200000000,"available":199924765,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1990000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.367}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4399999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7190000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.117}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.639}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5009999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":73,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195238,"statfs":{"total":200000000,"available":199924729,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.751}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.635}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5169999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.276}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4850000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4490000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7430000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":13,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":73,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195239,"statfs":{"total":200000000,"available":199924855,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1000000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.171}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68400000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2210000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7869999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.597}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9039999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":73,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195238,"statfs":{"total":200000000,"available":199924695,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79900000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95599999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6180000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4470000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3069999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.391}]}]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":650,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194661,"statfs":{"total":200000000,"available":199333849,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.03}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.161}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.204}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2290000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0129999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.839}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3049999999999999}]}]}],"pool_statfs":[]}} 2023-10-26T01:30:52.123 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-26T01:30:52.124 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config log 1 --format=json 2023-10-26T01:30:52.125 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-10-26T01:30:52.379 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:52.377+0000 7f33015c4640 1 -- 172.21.15.169:0/3615026447 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32e4004d00 msgr2=0x7f32e4005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:52.380 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:52.377+0000 7f33015c4640 1 --2- 172.21.15.169:0/3615026447 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32e4004d00 0x7f32e4005180 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f32f4001d80 tx=0x7f32e40057f0 comp rx=0 tx=0).stop 2023-10-26T01:30:52.380 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:52.377+0000 7f3300dc3640 1 -- 172.21.15.169:0/3615026447 reap_dead start 2023-10-26T01:30:52.380 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:52.377+0000 7f33015c4640 1 -- 172.21.15.169:0/3615026447 shutdown_connections 2023-10-26T01:30:52.381 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:52.377+0000 7f33015c4640 1 -- 172.21.15.169:0/3615026447 >> 172.21.15.169:0/3615026447 conn(0x7f32fc0af110 msgr2=0x7f32fc0af520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:52.381 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:52.377+0000 7f33015c4640 1 -- 172.21.15.169:0/3615026447 shutdown_connections 2023-10-26T01:30:52.381 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:52.377+0000 7f33015c4640 1 -- 172.21.15.169:0/3615026447 wait complete. 2023-10-26T01:30:52.382 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:52.378+0000 7f33015c4640 1 Processor -- start 2023-10-26T01:30:52.382 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:52.379+0000 7f33015c4640 1 -- start start 2023-10-26T01:30:52.382 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:52.379+0000 7f33015c4640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32e4004d00 0x7f32fc0c6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:52.383 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:52.379+0000 7f33015c4640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f32fc151360 0x7f32fc0acb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:52.383 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:52.379+0000 7f33015c4640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f32fc0ad0a0 0x7f32fc155010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:52.383 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:52.380+0000 7f33015c4640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32fc06b100 con 0x7f32e4004d00 2023-10-26T01:30:52.383 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:52.380+0000 7f33015c4640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32fc0c7620 con 0x7f32fc151360 2023-10-26T01:30:52.383 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:52.380+0000 7f33015c4640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32fc06ab10 con 0x7f32fc0ad0a0 2023-10-26T01:30:52.384 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:52.380+0000 7f32fbfff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32e4004d00 0x7f32fc0c6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:52.384 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:52.380+0000 7f32fbfff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32e4004d00 0x7f32fc0c6370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:34058/0 (socket says 172.21.15.169:34058) 2023-10-26T01:30:52.384 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:52.380+0000 7f32fbfff640 1 -- 172.21.15.169:0/3640777804 learned_addr learned my addr 172.21.15.169:0/3640777804 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:30:52.385 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:52.380+0000 7f32fbfff640 1 -- 172.21.15.169:0/3640777804 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f32fc0ad0a0 msgr2=0x7f32fc155010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:52.386 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:52.380+0000 7f32fbfff640 1 --2- 172.21.15.169:0/3640777804 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f32fc0ad0a0 0x7f32fc155010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:52.386 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:52.380+0000 7f32fbfff640 1 -- 172.21.15.169:0/3640777804 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f32fc151360 msgr2=0x7f32fc0acb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:52.386 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:52.380+0000 7f32fbfff640 1 --2- 172.21.15.169:0/3640777804 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f32fc151360 0x7f32fc0acb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:52.386 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:52.380+0000 7f32fbfff640 1 -- 172.21.15.169:0/3640777804 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32fc0c7060 con 0x7f32e4004d00 2023-10-26T01:30:52.387 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:52.380+0000 7f32fbfff640 1 --2- 172.21.15.169:0/3640777804 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32e4004d00 0x7f32fc0c6370 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f32ec00e770 tx=0x7f32ec00ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:52.387 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:52.381+0000 7f32f8ff9640 1 -- 172.21.15.169:0/3640777804 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32ec00cbe0 con 0x7f32e4004d00 2023-10-26T01:30:52.387 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:52.381+0000 7f32f8ff9640 1 -- 172.21.15.169:0/3640777804 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f32ec00ce30 con 0x7f32e4004d00 2023-10-26T01:30:52.387 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:52.381+0000 7f33015c4640 1 -- 172.21.15.169:0/3640777804 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32fc0c7340 con 0x7f32e4004d00 2023-10-26T01:30:52.387 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:52.381+0000 7f32f8ff9640 1 -- 172.21.15.169:0/3640777804 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32ec005280 con 0x7f32e4004d00 2023-10-26T01:30:52.388 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:52.381+0000 7f33015c4640 1 -- 172.21.15.169:0/3640777804 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32fc151e60 con 0x7f32e4004d00 2023-10-26T01:30:52.388 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:52.382+0000 7f33015c4640 1 -- 172.21.15.169:0/3640777804 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32fc0ad4c0 con 0x7f32e4004d00 2023-10-26T01:30:52.388 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:52.383+0000 7f32f8ff9640 1 -- 172.21.15.169:0/3640777804 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f32ec0053e0 con 0x7f32e4004d00 2023-10-26T01:30:52.389 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:52.384+0000 7f32f8ff9640 1 --2- 172.21.15.169:0/3640777804 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f32e806ae40 0x7f32e806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:52.389 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:52.384+0000 7f32f8ff9640 1 -- 172.21.15.169:0/3640777804 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f32ec089e10 con 0x7f32e4004d00 2023-10-26T01:30:52.389 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:52.384+0000 7f32fb7fe640 1 --2- 172.21.15.169:0/3640777804 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f32e806ae40 0x7f32e806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:52.389 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:52.385+0000 7f32fb7fe640 1 --2- 172.21.15.169:0/3640777804 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f32e806ae40 0x7f32e806d300 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f32f00017a0 tx=0x7f32f0001150 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:52.393 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:52.388+0000 7f32f8ff9640 1 -- 172.21.15.169:0/3640777804 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f32ec08d030 con 0x7f32e4004d00 2023-10-26T01:30:52.582 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:52.580+0000 7f33015c4640 1 -- 172.21.15.169:0/3640777804 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "config log", "num": 1, "format": "json"} v 0) v1 -- 0x7f32fc073c10 con 0x7f32e4004d00 2023-10-26T01:30:52.584 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:52.581+0000 7f32f8ff9640 1 -- 172.21.15.169:0/3640777804 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "config log", "num": 1, "format": "json"}]=0 v18) v1 ==== 86+0+138 (secure 0 0 0) 0x7f32ec01f070 con 0x7f32e4004d00 2023-10-26T01:30:52.584 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:30:52.586 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:52.584+0000 7f33015c4640 1 -- 172.21.15.169:0/3640777804 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f32e806ae40 msgr2=0x7f32e806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:52.587 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:52.584+0000 7f33015c4640 1 --2- 172.21.15.169:0/3640777804 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f32e806ae40 0x7f32e806d300 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f32f00017a0 tx=0x7f32f0001150 comp rx=0 tx=0).stop 2023-10-26T01:30:52.587 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:52.584+0000 7f33015c4640 1 -- 172.21.15.169:0/3640777804 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32e4004d00 msgr2=0x7f32fc0c6370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:52.587 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:52.584+0000 7f33015c4640 1 --2- 172.21.15.169:0/3640777804 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32e4004d00 0x7f32fc0c6370 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f32ec00e770 tx=0x7f32ec00ec30 comp rx=0 tx=0).stop 2023-10-26T01:30:52.588 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:52.584+0000 7f33015c4640 1 -- 172.21.15.169:0/3640777804 shutdown_connections 2023-10-26T01:30:52.588 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:52.584+0000 7f33015c4640 1 --2- 172.21.15.169:0/3640777804 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f32e806ae40 0x7f32e806d300 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:52.588 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:52.584+0000 7f33015c4640 1 --2- 172.21.15.169:0/3640777804 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f32fc0ad0a0 0x7f32fc155010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:52.588 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:52.584+0000 7f33015c4640 1 --2- 172.21.15.169:0/3640777804 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f32fc151360 0x7f32fc0acb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:52.588 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:52.584+0000 7f33015c4640 1 --2- 172.21.15.169:0/3640777804 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32e4004d00 0x7f32fc0c6370 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:52.589 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:52.584+0000 7f33015c4640 1 -- 172.21.15.169:0/3640777804 >> 172.21.15.169:0/3640777804 conn(0x7f32fc0af110 msgr2=0x7f32fc0adbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:52.589 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:52.584+0000 7f33015c4640 1 -- 172.21.15.169:0/3640777804 shutdown_connections 2023-10-26T01:30:52.589 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:52.584+0000 7f33015c4640 1 -- 172.21.15.169:0/3640777804 wait complete. 2023-10-26T01:30:52.606 INFO:teuthology.orchestra.run.smithi169.stdout:[{"version":18,"timestamp":"2023-10-26T01:30:24.000012+0000","name":"","changes":[{"name":"mds/rados_osd_op_timeout","new_value":"900"}]}] 2023-10-26T01:30:52.607 INFO:tasks.ceph_manager:config epoch is 18 2023-10-26T01:30:52.607 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-10-26T01:30:52.608 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-10-26T01:30:52.608 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-10-26T01:30:52.831 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:52.828+0000 7f96a4305640 1 -- 172.21.15.169:0/4026945437 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f96880058d0 msgr2=0x7f9688005d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:52.831 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:52.828+0000 7f96a4305640 1 --2- 172.21.15.169:0/4026945437 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f96880058d0 0x7f9688005d50 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f969c0ab250 tx=0x7f9688006900 comp rx=0 tx=0).stop 2023-10-26T01:30:52.831 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:52.828+0000 7f96a28a2640 1 -- 172.21.15.169:0/4026945437 reap_dead start 2023-10-26T01:30:52.832 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:52.828+0000 7f96a4305640 1 -- 172.21.15.169:0/4026945437 shutdown_connections 2023-10-26T01:30:52.832 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:52.828+0000 7f96a4305640 1 -- 172.21.15.169:0/4026945437 >> 172.21.15.169:0/4026945437 conn(0x7f969c069930 msgr2=0x7f969c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:52.832 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:52.828+0000 7f96a4305640 1 -- 172.21.15.169:0/4026945437 shutdown_connections 2023-10-26T01:30:52.832 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:52.828+0000 7f96a4305640 1 -- 172.21.15.169:0/4026945437 wait complete. 2023-10-26T01:30:52.834 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:52.830+0000 7f96a4305640 1 Processor -- start 2023-10-26T01:30:52.834 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:52.830+0000 7f96a4305640 1 -- start start 2023-10-26T01:30:52.834 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:52.831+0000 7f96a4305640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96880058d0 0x7f969c143f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:52.834 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:52.831+0000 7f96a4305640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f969c1446a0 0x7f969c0cd6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:52.835 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:52.831+0000 7f96a4305640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f969c1e3fc0 0x7f969c0cdc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:52.835 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:52.831+0000 7f96a4305640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f969c153810 con 0x7f969c1446a0 2023-10-26T01:30:52.835 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:52.831+0000 7f96a4305640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f969c144470 con 0x7f96880058d0 2023-10-26T01:30:52.835 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:52.831+0000 7f96a4305640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f969c1e4d90 con 0x7f969c1e3fc0 2023-10-26T01:30:52.835 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:52.831+0000 7f96a18a0640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f969c1446a0 0x7f969c0cd6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:52.836 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:52.831+0000 7f96a18a0640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f969c1446a0 0x7f969c0cd6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:34074/0 (socket says 172.21.15.169:34074) 2023-10-26T01:30:52.836 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:52.831+0000 7f96a18a0640 1 -- 172.21.15.169:0/733067821 learned_addr learned my addr 172.21.15.169:0/733067821 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:30:52.836 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:52.831+0000 7f96a28a2640 1 --2- 172.21.15.169:0/733067821 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f969c1e3fc0 0x7f969c0cdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:52.836 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:52.831+0000 7f96a18a0640 1 -- 172.21.15.169:0/733067821 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f969c1e3fc0 msgr2=0x7f969c0cdc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:52.836 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:52.831+0000 7f96a20a1640 1 --2- 172.21.15.169:0/733067821 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96880058d0 0x7f969c143f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:52.837 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:52.831+0000 7f96a18a0640 1 --2- 172.21.15.169:0/733067821 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f969c1e3fc0 0x7f969c0cdc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:52.837 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:52.831+0000 7f96a18a0640 1 -- 172.21.15.169:0/733067821 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96880058d0 msgr2=0x7f969c143f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:52.837 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:52.831+0000 7f96a18a0640 1 --2- 172.21.15.169:0/733067821 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96880058d0 0x7f969c143f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:52.837 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:52.831+0000 7f96a18a0640 1 -- 172.21.15.169:0/733067821 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f969c1e5070 con 0x7f969c1446a0 2023-10-26T01:30:52.838 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:52.832+0000 7f96a18a0640 1 --2- 172.21.15.169:0/733067821 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f969c1446a0 0x7f969c0cd6f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f968c00c350 tx=0x7f968c0014b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:52.839 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:52.832+0000 7f9692ffd640 1 -- 172.21.15.169:0/733067821 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f968c006030 con 0x7f969c1446a0 2023-10-26T01:30:52.839 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:52.832+0000 7f9692ffd640 1 -- 172.21.15.169:0/733067821 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f968c006190 con 0x7f969c1446a0 2023-10-26T01:30:52.839 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:52.832+0000 7f96a4305640 1 -- 172.21.15.169:0/733067821 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f969c1e5350 con 0x7f969c1446a0 2023-10-26T01:30:52.839 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:52.832+0000 7f9692ffd640 1 -- 172.21.15.169:0/733067821 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f968c0065a0 con 0x7f969c1446a0 2023-10-26T01:30:52.839 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:52.832+0000 7f96a4305640 1 -- 172.21.15.169:0/733067821 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f969c1e4a60 con 0x7f969c1446a0 2023-10-26T01:30:52.840 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:52.833+0000 7f96a4305640 1 -- 172.21.15.169:0/733067821 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f969c04db10 con 0x7f969c1446a0 2023-10-26T01:30:52.840 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:52.834+0000 7f9692ffd640 1 -- 172.21.15.169:0/733067821 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f968c033030 con 0x7f969c1446a0 2023-10-26T01:30:52.840 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:52.835+0000 7f9692ffd640 1 --2- 172.21.15.169:0/733067821 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f968006ae70 0x7f968006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:52.840 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:52.835+0000 7f9692ffd640 1 -- 172.21.15.169:0/733067821 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f968c0aa5b0 con 0x7f969c1446a0 2023-10-26T01:30:52.840 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:52.836+0000 7f96a20a1640 1 --2- 172.21.15.169:0/733067821 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f968006ae70 0x7f968006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:52.841 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:52.837+0000 7f96a20a1640 1 --2- 172.21.15.169:0/733067821 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f968006ae70 0x7f968006d330 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f96980099e0 tx=0x7f9698005f90 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:52.842 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:52.839+0000 7f9692ffd640 1 -- 172.21.15.169:0/733067821 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f968c075650 con 0x7f969c1446a0 2023-10-26T01:30:53.083 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.081+0000 7f96a4305640 1 -- 172.21.15.169:0/733067821 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f969c069f80 con 0x7f969c1446a0 2023-10-26T01:30:53.093 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.090+0000 7f9692ffd640 1 -- 172.21.15.169:0/733067821 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+171036 (secure 0 0 0) 0x7f968c044090 con 0x7f969c1446a0 2023-10-26T01:30:53.096 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:30:53.097 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.095+0000 7f96a4305640 1 -- 172.21.15.169:0/733067821 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f968006ae70 msgr2=0x7f968006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:53.098 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.095+0000 7f96a4305640 1 --2- 172.21.15.169:0/733067821 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f968006ae70 0x7f968006d330 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f96980099e0 tx=0x7f9698005f90 comp rx=0 tx=0).stop 2023-10-26T01:30:53.098 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.095+0000 7f96a4305640 1 -- 172.21.15.169:0/733067821 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f969c1446a0 msgr2=0x7f969c0cd6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:53.098 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.095+0000 7f96a4305640 1 --2- 172.21.15.169:0/733067821 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f969c1446a0 0x7f969c0cd6f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f968c00c350 tx=0x7f968c0014b0 comp rx=0 tx=0).stop 2023-10-26T01:30:53.098 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.095+0000 7f96a4305640 1 -- 172.21.15.169:0/733067821 shutdown_connections 2023-10-26T01:30:53.098 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.095+0000 7f96a4305640 1 --2- 172.21.15.169:0/733067821 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f968006ae70 0x7f968006d330 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:53.099 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.095+0000 7f96a4305640 1 --2- 172.21.15.169:0/733067821 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f969c1446a0 0x7f969c0cd6f0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:53.099 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.095+0000 7f96a4305640 1 --2- 172.21.15.169:0/733067821 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f969c1e3fc0 0x7f969c0cdc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:53.099 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.095+0000 7f96a4305640 1 --2- 172.21.15.169:0/733067821 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96880058d0 0x7f969c143f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:53.099 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.095+0000 7f96a4305640 1 -- 172.21.15.169:0/733067821 >> 172.21.15.169:0/733067821 conn(0x7f969c069930 msgr2=0x7f969c0b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:53.099 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.095+0000 7f96a4305640 1 -- 172.21.15.169:0/733067821 shutdown_connections 2023-10-26T01:30:53.100 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.095+0000 7f96a4305640 1 -- 172.21.15.169:0/733067821 wait complete. 2023-10-26T01:30:53.115 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":5,"active_gid":4112,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1187156918},{"type":"v1","addr":"172.21.15.170:6833","nonce":1187156918}]},"active_addr":"172.21.15.170:6833/1187156918","active_change":"2023-10-26T01:30:28.099986+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4102,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.3","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.3","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.170:0","nonce":3159295850}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.170:0","nonce":1330262208}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.170:0","nonce":491280924}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.170:0","nonce":2017848700}]}]} 2023-10-26T01:30:53.120 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-10-26T01:30:53.121 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-10-26T01:30:53.121 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:30:53.334 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.331+0000 7eff781fc640 1 -- 172.21.15.169:0/3114763890 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff700cdbb0 msgr2=0x7eff700b07d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:53.334 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.331+0000 7eff781fc640 1 --2- 172.21.15.169:0/3114763890 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff700cdbb0 0x7eff700b07d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7eff68005fd0 tx=0x7eff6802c2e0 comp rx=0 tx=0).stop 2023-10-26T01:30:53.335 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.331+0000 7eff781fc640 1 -- 172.21.15.169:0/3114763890 shutdown_connections 2023-10-26T01:30:53.335 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.331+0000 7eff781fc640 1 --2- 172.21.15.169:0/3114763890 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff700cdbb0 0x7eff700b07d0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:53.335 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.331+0000 7eff781fc640 1 --2- 172.21.15.169:0/3114763890 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7eff7006ed30 0x7eff700cd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:53.336 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.331+0000 7eff781fc640 1 -- 172.21.15.169:0/3114763890 >> 172.21.15.169:0/3114763890 conn(0x7eff70069930 msgr2=0x7eff70069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:53.336 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.331+0000 7eff781fc640 1 -- 172.21.15.169:0/3114763890 shutdown_connections 2023-10-26T01:30:53.336 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.331+0000 7eff781fc640 1 -- 172.21.15.169:0/3114763890 wait complete. 2023-10-26T01:30:53.338 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.333+0000 7eff781fc640 1 Processor -- start 2023-10-26T01:30:53.339 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.333+0000 7eff781fc640 1 -- start start 2023-10-26T01:30:53.339 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.334+0000 7eff781fc640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff7006ed30 0x7eff70149640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:53.339 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.334+0000 7eff781fc640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7eff700cdbb0 0x7eff70098210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:53.339 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.334+0000 7eff781fc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff7015b610 0x7eff70149ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:53.339 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.334+0000 7eff781fc640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff70097d80 con 0x7eff7006ed30 2023-10-26T01:30:53.340 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.334+0000 7eff781fc640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff70099d00 con 0x7eff7015b610 2023-10-26T01:30:53.340 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.334+0000 7eff781fc640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff70158db0 con 0x7eff700cdbb0 2023-10-26T01:30:53.340 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.335+0000 7eff75f98640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff7006ed30 0x7eff70149640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:53.340 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.335+0000 7eff75f98640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff7006ed30 0x7eff70149640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:34094/0 (socket says 172.21.15.169:34094) 2023-10-26T01:30:53.340 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.335+0000 7eff75f98640 1 -- 172.21.15.169:0/3002373424 learned_addr learned my addr 172.21.15.169:0/3002373424 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:30:53.341 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.335+0000 7eff75797640 1 --2- 172.21.15.169:0/3002373424 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7eff700cdbb0 0x7eff70098210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:53.341 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.335+0000 7eff76799640 1 --2- 172.21.15.169:0/3002373424 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff7015b610 0x7eff70149ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:53.341 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.335+0000 7eff75f98640 1 -- 172.21.15.169:0/3002373424 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7eff700cdbb0 msgr2=0x7eff70098210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:53.341 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.335+0000 7eff75f98640 1 --2- 172.21.15.169:0/3002373424 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7eff700cdbb0 0x7eff70098210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:53.341 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.335+0000 7eff75f98640 1 -- 172.21.15.169:0/3002373424 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff7015b610 msgr2=0x7eff70149ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:53.342 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.335+0000 7eff75f98640 1 --2- 172.21.15.169:0/3002373424 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff7015b610 0x7eff70149ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:53.342 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.335+0000 7eff75f98640 1 -- 172.21.15.169:0/3002373424 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff68009900 con 0x7eff7006ed30 2023-10-26T01:30:53.342 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.335+0000 7eff76799640 1 --2- 172.21.15.169:0/3002373424 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff7015b610 0x7eff70149ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:30:53.342 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.335+0000 7eff75f98640 1 --2- 172.21.15.169:0/3002373424 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff7006ed30 0x7eff70149640 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7eff6c00c430 tx=0x7eff6c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:53.343 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.336+0000 7eff627fc640 1 -- 172.21.15.169:0/3002373424 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff6c0052d0 con 0x7eff7006ed30 2023-10-26T01:30:53.343 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.336+0000 7eff627fc640 1 -- 172.21.15.169:0/3002373424 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7eff6c0035f0 con 0x7eff7006ed30 2023-10-26T01:30:53.343 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.336+0000 7eff781fc640 1 -- 172.21.15.169:0/3002373424 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff70097450 con 0x7eff7006ed30 2023-10-26T01:30:53.343 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.336+0000 7eff627fc640 1 -- 172.21.15.169:0/3002373424 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff6c003a00 con 0x7eff7006ed30 2023-10-26T01:30:53.343 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.336+0000 7eff781fc640 1 -- 172.21.15.169:0/3002373424 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff70097a10 con 0x7eff7006ed30 2023-10-26T01:30:53.344 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.337+0000 7eff781fc640 1 -- 172.21.15.169:0/3002373424 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff7004db10 con 0x7eff7006ed30 2023-10-26T01:30:53.345 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.338+0000 7eff627fc640 1 -- 172.21.15.169:0/3002373424 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7eff6c003b60 con 0x7eff7006ed30 2023-10-26T01:30:53.345 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.339+0000 7eff627fc640 1 --2- 172.21.15.169:0/3002373424 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7eff5406ae70 0x7eff5406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:53.345 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.339+0000 7eff627fc640 1 -- 172.21.15.169:0/3002373424 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7eff6c088e40 con 0x7eff7006ed30 2023-10-26T01:30:53.345 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.340+0000 7eff75797640 1 --2- 172.21.15.169:0/3002373424 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7eff5406ae70 0x7eff5406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:53.345 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.341+0000 7eff75797640 1 --2- 172.21.15.169:0/3002373424 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7eff5406ae70 0x7eff5406d330 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7eff64005fd0 tx=0x7eff64005850 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:53.346 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.344+0000 7eff627fc640 1 -- 172.21.15.169:0/3002373424 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7eff6c053ef0 con 0x7eff7006ed30 2023-10-26T01:30:53.535 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.532+0000 7eff781fc640 1 -- 172.21.15.169:0/3002373424 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7eff7006c990 con 0x7eff7006ed30 2023-10-26T01:30:53.538 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.535+0000 7eff627fc640 1 -- 172.21.15.169:0/3002373424 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16553 (secure 0 0 0) 0x7eff6c017090 con 0x7eff7006ed30 2023-10-26T01:30:53.539 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:30:53.539 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":21,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:30:43.078706+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-26T01:30:39.437631+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-26T01:30:40.561774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:30:53.541 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.538+0000 7eff781fc640 1 -- 172.21.15.169:0/3002373424 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7eff5406ae70 msgr2=0x7eff5406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:53.541 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.538+0000 7eff781fc640 1 --2- 172.21.15.169:0/3002373424 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7eff5406ae70 0x7eff5406d330 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7eff64005fd0 tx=0x7eff64005850 comp rx=0 tx=0).stop 2023-10-26T01:30:53.541 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.538+0000 7eff781fc640 1 -- 172.21.15.169:0/3002373424 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff7006ed30 msgr2=0x7eff70149640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:53.541 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.538+0000 7eff781fc640 1 --2- 172.21.15.169:0/3002373424 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff7006ed30 0x7eff70149640 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7eff6c00c430 tx=0x7eff6c00c8f0 comp rx=0 tx=0).stop 2023-10-26T01:30:53.542 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.538+0000 7eff781fc640 1 -- 172.21.15.169:0/3002373424 shutdown_connections 2023-10-26T01:30:53.542 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.538+0000 7eff781fc640 1 --2- 172.21.15.169:0/3002373424 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7eff5406ae70 0x7eff5406d330 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:53.542 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.538+0000 7eff781fc640 1 --2- 172.21.15.169:0/3002373424 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff7015b610 0x7eff70149ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:53.542 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.538+0000 7eff781fc640 1 --2- 172.21.15.169:0/3002373424 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7eff700cdbb0 0x7eff70098210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:53.542 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.538+0000 7eff781fc640 1 --2- 172.21.15.169:0/3002373424 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff7006ed30 0x7eff70149640 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:53.543 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.538+0000 7eff781fc640 1 -- 172.21.15.169:0/3002373424 >> 172.21.15.169:0/3002373424 conn(0x7eff70069930 msgr2=0x7eff700adb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:53.543 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.538+0000 7eff781fc640 1 -- 172.21.15.169:0/3002373424 shutdown_connections 2023-10-26T01:30:53.543 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.538+0000 7eff781fc640 1 -- 172.21.15.169:0/3002373424 wait complete. 2023-10-26T01:30:53.560 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-10-26T01:30:53.561 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:30:53.781 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.778+0000 7f6c7fb1d640 1 -- 172.21.15.169:0/670576630 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6c78158850 msgr2=0x7f6c7814e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:53.781 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.778+0000 7f6c7fb1d640 1 --2- 172.21.15.169:0/670576630 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6c78158850 0x7f6c7814e790 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c005fd0 tx=0x7f6c6c02bd90 comp rx=0 tx=0).stop 2023-10-26T01:30:53.782 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.778+0000 7f6c7fb1d640 1 -- 172.21.15.169:0/670576630 shutdown_connections 2023-10-26T01:30:53.782 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.778+0000 7f6c7fb1d640 1 --2- 172.21.15.169:0/670576630 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6c78159e10 0x7f6c7815a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:53.782 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.778+0000 7f6c7fb1d640 1 --2- 172.21.15.169:0/670576630 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6c78158850 0x7f6c7814e790 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:53.782 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.778+0000 7f6c7fb1d640 1 -- 172.21.15.169:0/670576630 >> 172.21.15.169:0/670576630 conn(0x7f6c78069930 msgr2=0x7f6c78069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:53.782 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.778+0000 7f6c7fb1d640 1 -- 172.21.15.169:0/670576630 shutdown_connections 2023-10-26T01:30:53.783 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.778+0000 7f6c7fb1d640 1 -- 172.21.15.169:0/670576630 wait complete. 2023-10-26T01:30:53.784 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.780+0000 7f6c7fb1d640 1 Processor -- start 2023-10-26T01:30:53.784 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.780+0000 7f6c7fb1d640 1 -- start start 2023-10-26T01:30:53.785 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.781+0000 7f6c7fb1d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6c7806ce20 0x7f6c780b17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:53.785 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.781+0000 7f6c7fb1d640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6c78158850 0x7f6c781f24c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:53.785 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.781+0000 7f6c7fb1d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c78159e10 0x7f6c7814b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:53.785 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.781+0000 7f6c7fb1d640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c780cd0e0 con 0x7f6c78159e10 2023-10-26T01:30:53.785 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.781+0000 7f6c7fb1d640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c780cc8d0 con 0x7f6c7806ce20 2023-10-26T01:30:53.786 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.781+0000 7f6c7fb1d640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c780ad230 con 0x7f6c78158850 2023-10-26T01:30:53.786 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.781+0000 7f6c7e0ba640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c78159e10 0x7f6c7814b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:53.786 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.781+0000 7f6c7e0ba640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c78159e10 0x7f6c7814b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:34100/0 (socket says 172.21.15.169:34100) 2023-10-26T01:30:53.786 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.781+0000 7f6c7e0ba640 1 -- 172.21.15.169:0/1743765877 learned_addr learned my addr 172.21.15.169:0/1743765877 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:30:53.786 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.782+0000 7f6c7d0b8640 1 --2- 172.21.15.169:0/1743765877 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6c78158850 0x7f6c781f24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:53.787 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.782+0000 7f6c7e0ba640 1 -- 172.21.15.169:0/1743765877 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6c78158850 msgr2=0x7f6c781f24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:53.787 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.782+0000 7f6c7e0ba640 1 --2- 172.21.15.169:0/1743765877 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6c78158850 0x7f6c781f24c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:53.787 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.782+0000 7f6c7e0ba640 1 -- 172.21.15.169:0/1743765877 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6c7806ce20 msgr2=0x7f6c780b17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:53.787 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.782+0000 7f6c7e0ba640 1 --2- 172.21.15.169:0/1743765877 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6c7806ce20 0x7f6c780b17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:53.787 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.782+0000 7f6c7e0ba640 1 -- 172.21.15.169:0/1743765877 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c6c009560 con 0x7f6c78159e10 2023-10-26T01:30:53.788 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.782+0000 7f6c7e0ba640 1 --2- 172.21.15.169:0/1743765877 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c78159e10 0x7f6c7814b260 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f6c7000ebb0 tx=0x7f6c7000c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:53.789 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.782+0000 7f6c6a7fc640 1 -- 172.21.15.169:0/1743765877 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c70004d20 con 0x7f6c78159e10 2023-10-26T01:30:53.789 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.782+0000 7f6c6a7fc640 1 -- 172.21.15.169:0/1743765877 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6c70004e80 con 0x7f6c78159e10 2023-10-26T01:30:53.789 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.782+0000 7f6c7fb1d640 1 -- 172.21.15.169:0/1743765877 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c780ccb20 con 0x7f6c78159e10 2023-10-26T01:30:53.790 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.782+0000 7f6c6a7fc640 1 -- 172.21.15.169:0/1743765877 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c7000b7d0 con 0x7f6c78159e10 2023-10-26T01:30:53.790 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.782+0000 7f6c7fb1d640 1 -- 172.21.15.169:0/1743765877 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c780acec0 con 0x7f6c78159e10 2023-10-26T01:30:53.790 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.784+0000 7f6c7fb1d640 1 -- 172.21.15.169:0/1743765877 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c780acd20 con 0x7f6c78159e10 2023-10-26T01:30:53.790 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.785+0000 7f6c6a7fc640 1 -- 172.21.15.169:0/1743765877 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f6c7000b930 con 0x7f6c78159e10 2023-10-26T01:30:53.790 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.785+0000 7f6c6a7fc640 1 --2- 172.21.15.169:0/1743765877 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f6c5c06ae40 0x7f6c5c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:53.791 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.786+0000 7f6c6a7fc640 1 -- 172.21.15.169:0/1743765877 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6c7008a1c0 con 0x7f6c78159e10 2023-10-26T01:30:53.791 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.786+0000 7f6c7d8b9640 1 --2- 172.21.15.169:0/1743765877 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f6c5c06ae40 0x7f6c5c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:53.791 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.787+0000 7f6c7d8b9640 1 --2- 172.21.15.169:0/1743765877 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f6c5c06ae40 0x7f6c5c06d300 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6c74000f60 tx=0x7f6c740079d0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:53.792 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.790+0000 7f6c6a7fc640 1 -- 172.21.15.169:0/1743765877 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6c780acd20 con 0x7f6c78159e10 2023-10-26T01:30:53.981 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.978+0000 7f6c7fb1d640 1 -- 172.21.15.169:0/1743765877 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6c780acd20 con 0x7f6c78159e10 2023-10-26T01:30:53.983 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.980+0000 7f6c6a7fc640 1 -- 172.21.15.169:0/1743765877 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16553 (secure 0 0 0) 0x7f6c780acd20 con 0x7f6c78159e10 2023-10-26T01:30:53.984 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:30:53.984 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":21,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:30:43.078706+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-26T01:30:39.437631+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-26T01:30:40.561774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:30:53.986 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.983+0000 7f6c7fb1d640 1 -- 172.21.15.169:0/1743765877 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f6c5c06ae40 msgr2=0x7f6c5c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:53.986 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.983+0000 7f6c7fb1d640 1 --2- 172.21.15.169:0/1743765877 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f6c5c06ae40 0x7f6c5c06d300 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6c74000f60 tx=0x7f6c740079d0 comp rx=0 tx=0).stop 2023-10-26T01:30:53.986 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.983+0000 7f6c7fb1d640 1 -- 172.21.15.169:0/1743765877 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c78159e10 msgr2=0x7f6c7814b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:53.987 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.983+0000 7f6c7fb1d640 1 --2- 172.21.15.169:0/1743765877 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c78159e10 0x7f6c7814b260 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f6c7000ebb0 tx=0x7f6c7000c7a0 comp rx=0 tx=0).stop 2023-10-26T01:30:53.987 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.983+0000 7f6c7fb1d640 1 -- 172.21.15.169:0/1743765877 shutdown_connections 2023-10-26T01:30:53.987 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.983+0000 7f6c7fb1d640 1 --2- 172.21.15.169:0/1743765877 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f6c5c06ae40 0x7f6c5c06d300 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:53.987 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.983+0000 7f6c7fb1d640 1 --2- 172.21.15.169:0/1743765877 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c78159e10 0x7f6c7814b260 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:53.987 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.983+0000 7f6c7fb1d640 1 --2- 172.21.15.169:0/1743765877 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6c78158850 0x7f6c781f24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:53.988 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.983+0000 7f6c7fb1d640 1 --2- 172.21.15.169:0/1743765877 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6c7806ce20 0x7f6c780b17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:53.988 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.983+0000 7f6c7fb1d640 1 -- 172.21.15.169:0/1743765877 >> 172.21.15.169:0/1743765877 conn(0x7f6c78069930 msgr2=0x7f6c7814d670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:53.988 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.984+0000 7f6c7fb1d640 1 -- 172.21.15.169:0/1743765877 shutdown_connections 2023-10-26T01:30:53.988 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:53.984+0000 7f6c7fb1d640 1 -- 172.21.15.169:0/1743765877 wait complete. 2023-10-26T01:30:54.006 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-10-26T01:30:54.006 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-10-26T01:30:54.007 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-10-26T01:30:54.007 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-10-26T01:30:54.008 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-10-26T01:30:54.009 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-10-26T01:30:54.009 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-10-26T01:30:54.010 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-10-26T01:30:54.311 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.308+0000 7f6d5d15f640 1 -- 172.21.15.169:0/1409712188 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d5809b630 con 0x7f6d50033960 2023-10-26T01:30:54.312 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.309+0000 7f6d603c5640 1 -- 172.21.15.169:0/1409712188 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d50033960 msgr2=0x7f6d50054390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:54.312 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.309+0000 7f6d603c5640 1 --2- 172.21.15.169:0/1409712188 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d50033960 0x7f6d50054390 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6d58063410 tx=0x7f6d580680e0 comp rx=0 tx=0).stop 2023-10-26T01:30:54.313 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.310+0000 7f6d603c5640 1 -- 172.21.15.169:0/1409712188 shutdown_connections 2023-10-26T01:30:54.313 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.310+0000 7f6d603c5640 1 --2- 172.21.15.169:0/1409712188 >> v2:172.21.15.170:3300/0 conn(0x7f6d50038320 0x7f6d50052c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:54.313 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.310+0000 7f6d603c5640 1 --2- 172.21.15.169:0/1409712188 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d50033960 0x7f6d50054390 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:54.314 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.310+0000 7f6d603c5640 1 -- 172.21.15.169:0/1409712188 >> 172.21.15.169:0/1409712188 conn(0x7f6d50009f80 msgr2=0x7f6d5000a360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:54.314 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.311+0000 7f6d603c5640 1 -- 172.21.15.169:0/1409712188 shutdown_connections 2023-10-26T01:30:54.314 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.311+0000 7f6d603c5640 1 -- 172.21.15.169:0/1409712188 wait complete. 2023-10-26T01:30:54.317 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.315+0000 7f6d603c5640 1 Processor -- start 2023-10-26T01:30:54.317 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.315+0000 7f6d603c5640 1 -- start start 2023-10-26T01:30:54.319 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.317+0000 7f6d603c5640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d50007160 0x7f6d500efa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:54.320 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.317+0000 7f6d603c5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6d50033960 0x7f6d5004a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:54.320 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.317+0000 7f6d603c5640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6d50038320 0x7f6d50039790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:54.320 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.317+0000 7f6d603c5640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d500f9de0 con 0x7f6d50007160 2023-10-26T01:30:54.320 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.317+0000 7f6d603c5640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d500fa0c0 con 0x7f6d50033960 2023-10-26T01:30:54.320 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.317+0000 7f6d603c5640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d500fa3a0 con 0x7f6d50038320 2023-10-26T01:30:54.321 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.318+0000 7f7d6f3d7640 1 -- 172.21.15.169:0/150454397 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7d60038300 msgr2=0x7f7d60054430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:54.321 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.318+0000 7f7d6f3d7640 1 --2- 172.21.15.169:0/150454397 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7d60038300 0x7f7d60054430 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7d68063480 tx=0x7f7d68068a90 comp rx=0 tx=0).stop 2023-10-26T01:30:54.322 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.318+0000 7f6d5e161640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d50007160 0x7f6d500efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:54.322 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.318+0000 7f6d5e161640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d50007160 0x7f6d500efa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:45780/0 (socket says 172.21.15.169:45780) 2023-10-26T01:30:54.323 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.318+0000 7f6d5e161640 1 -- 172.21.15.169:0/2610389553 learned_addr learned my addr 172.21.15.169:0/2610389553 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:30:54.323 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.318+0000 7f6d5d960640 1 --2- 172.21.15.169:0/2610389553 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6d50033960 0x7f6d5004a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:54.323 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.318+0000 7f6d5e161640 1 -- 172.21.15.169:0/2610389553 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6d50038320 msgr2=0x7f6d50039790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:54.323 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.318+0000 7f6d5e161640 1 --2- 172.21.15.169:0/2610389553 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6d50038320 0x7f6d50039790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:54.323 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.318+0000 7f6d5e161640 1 -- 172.21.15.169:0/2610389553 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6d50033960 msgr2=0x7f6d5004a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:54.324 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.319+0000 7f6d5e161640 1 --2- 172.21.15.169:0/2610389553 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6d50033960 0x7f6d5004a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:54.324 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.319+0000 7f6d5e161640 1 -- 172.21.15.169:0/2610389553 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d500fa680 con 0x7f6d50007160 2023-10-26T01:30:54.324 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.319+0000 7f6d5d960640 1 --2- 172.21.15.169:0/2610389553 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6d50033960 0x7f6d5004a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:30:54.324 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.319+0000 7f6d5e161640 1 --2- 172.21.15.169:0/2610389553 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d50007160 0x7f6d500efa10 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6d580784d0 tx=0x7f6d5809a970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:54.325 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.318+0000 7f7d6f3d7640 1 -- 172.21.15.169:0/150454397 shutdown_connections 2023-10-26T01:30:54.325 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.318+0000 7f7d6f3d7640 1 --2- 172.21.15.169:0/150454397 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7d60038300 0x7f7d60054430 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:54.325 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.318+0000 7f7d6f3d7640 1 --2- 172.21.15.169:0/150454397 >> v2:172.21.15.170:3300/0 conn(0x7f7d600e31a0 0x7f7d6004f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:54.325 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.318+0000 7f7d6f3d7640 1 -- 172.21.15.169:0/150454397 >> 172.21.15.169:0/150454397 conn(0x7f7d60009f80 msgr2=0x7f7d6000a360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:54.326 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.318+0000 7f7d6f3d7640 1 -- 172.21.15.169:0/150454397 shutdown_connections 2023-10-26T01:30:54.326 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.318+0000 7f7d6f3d7640 1 -- 172.21.15.169:0/150454397 wait complete. 2023-10-26T01:30:54.326 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.321+0000 7f7d6f3d7640 1 Processor -- start 2023-10-26T01:30:54.327 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.321+0000 7f6d4affd640 1 -- 172.21.15.169:0/2610389553 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d5809b630 con 0x7f6d50007160 2023-10-26T01:30:54.327 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.321+0000 7f6d4affd640 1 -- 172.21.15.169:0/2610389553 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6d5809ade0 con 0x7f6d50007160 2023-10-26T01:30:54.327 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.321+0000 7f7d6f3d7640 1 -- start start 2023-10-26T01:30:54.328 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.321+0000 7f6d4affd640 1 -- 172.21.15.169:0/2610389553 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d580a6680 con 0x7f6d50007160 2023-10-26T01:30:54.328 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.321+0000 7f6d603c5640 1 -- 172.21.15.169:0/2610389553 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d500fa960 con 0x7f6d50007160 2023-10-26T01:30:54.328 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.322+0000 7f6d603c5640 1 -- 172.21.15.169:0/2610389553 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d500f9890 con 0x7f6d50007160 2023-10-26T01:30:54.329 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.323+0000 7f7d6f3d7640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d60038300 0x7f7d600d7b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:54.329 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.323+0000 7f6d603c5640 1 -- 172.21.15.169:0/2610389553 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6d500fa230 con 0x7f6d50007160 2023-10-26T01:30:54.331 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.323+0000 7f7d6e3d5640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d60038300 0x7f7d600d7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:54.331 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.323+0000 7f7d6e3d5640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d60038300 0x7f7d600d7b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:45812/0 (socket says 172.21.15.169:45812) 2023-10-26T01:30:54.331 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.323+0000 7f7d6f3d7640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7d60053fe0 0x7f7d600d8070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:54.332 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.323+0000 7f7d6e3d5640 1 -- 172.21.15.169:0/2883679332 learned_addr learned my addr 172.21.15.169:0/2883679332 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:30:54.332 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.323+0000 7f7d6f3d7640 1 --2- 172.21.15.169:0/2883679332 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d600e31a0 0x7f7d6004dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:54.332 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.324+0000 7f7d6f3d7640 1 -- 172.21.15.169:0/2883679332 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d6004bb60 con 0x7f7d60038300 2023-10-26T01:30:54.332 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.324+0000 7f7d6f3d7640 1 -- 172.21.15.169:0/2883679332 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d600d89a0 con 0x7f7d600e31a0 2023-10-26T01:30:54.332 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.324+0000 7f7d6f3d7640 1 -- 172.21.15.169:0/2883679332 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d600d8c80 con 0x7f7d60053fe0 2023-10-26T01:30:54.333 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.324+0000 7f7d6dbd4640 1 --2- 172.21.15.169:0/2883679332 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7d60053fe0 0x7f7d600d8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:54.333 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.324+0000 7f7d6ebd6640 1 --2- 172.21.15.169:0/2883679332 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d600e31a0 0x7f7d6004dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:54.333 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.324+0000 7f7d6e3d5640 1 -- 172.21.15.169:0/2883679332 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7d60053fe0 msgr2=0x7f7d600d8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:54.333 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.324+0000 7f7d6e3d5640 1 --2- 172.21.15.169:0/2883679332 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7d60053fe0 0x7f7d600d8070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:54.333 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.325+0000 7f7d6e3d5640 1 -- 172.21.15.169:0/2883679332 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d600e31a0 msgr2=0x7f7d6004dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:54.334 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.325+0000 7f7d6e3d5640 1 --2- 172.21.15.169:0/2883679332 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d600e31a0 0x7f7d6004dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:54.334 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.325+0000 7f7d6e3d5640 1 -- 172.21.15.169:0/2883679332 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d600d8f60 con 0x7f7d60038300 2023-10-26T01:30:54.334 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.325+0000 7f7d6e3d5640 1 --2- 172.21.15.169:0/2883679332 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d60038300 0x7f7d600d7b10 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7d6400e790 tx=0x7f7d6400ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:54.334 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.325+0000 7f7d5effd640 1 -- 172.21.15.169:0/2883679332 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d64009580 con 0x7f7d60038300 2023-10-26T01:30:54.335 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.326+0000 7f7d5effd640 1 -- 172.21.15.169:0/2883679332 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7d6400cdb0 con 0x7f7d60038300 2023-10-26T01:30:54.335 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.326+0000 7f7d5effd640 1 -- 172.21.15.169:0/2883679332 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d64010690 con 0x7f7d60038300 2023-10-26T01:30:54.335 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.326+0000 7f7d6f3d7640 1 -- 172.21.15.169:0/2883679332 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d600d9240 con 0x7f7d60038300 2023-10-26T01:30:54.335 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.326+0000 7f7d6f3d7640 1 -- 172.21.15.169:0/2883679332 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d600df640 con 0x7f7d60038300 2023-10-26T01:30:54.336 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.326+0000 7f6d4affd640 1 -- 172.21.15.169:0/2610389553 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f6d580a67e0 con 0x7f6d50007160 2023-10-26T01:30:54.336 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.327+0000 7f7d6f3d7640 1 -- 172.21.15.169:0/2883679332 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7d600d8790 con 0x7f7d60038300 2023-10-26T01:30:54.337 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.327+0000 7f89306cf640 1 -- 172.21.15.169:0/2721291895 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f89280b3380 msgr2=0x7f892815f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:54.337 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.327+0000 7f89306cf640 1 --2- 172.21.15.169:0/2721291895 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f89280b3380 0x7f892815f470 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8924005fd0 tx=0x7f892402bda0 comp rx=0 tx=0).stop 2023-10-26T01:30:54.337 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.327+0000 7f89306cf640 1 -- 172.21.15.169:0/2721291895 shutdown_connections 2023-10-26T01:30:54.337 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.327+0000 7f89306cf640 1 --2- 172.21.15.169:0/2721291895 >> v2:172.21.15.169:3300/0 conn(0x7f89280b72c0 0x7f892815b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:54.338 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.328+0000 7f6d4affd640 1 --2- 172.21.15.169:0/2610389553 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f6d3c06aec0 0x7f6d3c06d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:54.338 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.327+0000 7f89306cf640 1 --2- 172.21.15.169:0/2721291895 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f89280b3380 0x7f892815f470 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:54.338 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.327+0000 7f89306cf640 1 -- 172.21.15.169:0/2721291895 >> 172.21.15.169:0/2721291895 conn(0x7f89280658b0 msgr2=0x7f8928065cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:54.339 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.328+0000 7f6d4affd640 1 -- 172.21.15.169:0/2610389553 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6d580e3170 con 0x7f6d50007160 2023-10-26T01:30:54.339 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.328+0000 7f6d4affd640 1 --2- 172.21.15.169:0/2610389553 >> [v2:172.21.15.169:6816/4204255864,v1:172.21.15.169:6817/4204255864] conn(0x7f6d3c073dc0 0x7f6d3c076260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:54.339 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.328+0000 7f6d4affd640 1 -- 172.21.15.169:0/2610389553 --> [v2:172.21.15.169:6816/4204255864,v1:172.21.15.169:6817/4204255864] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6d3c076930 con 0x7f6d3c073dc0 2023-10-26T01:30:54.339 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.328+0000 7f6d4affd640 1 -- 172.21.15.169:0/2610389553 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6d58061190 con 0x7f6d50007160 2023-10-26T01:30:54.340 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.328+0000 7f55b4664640 1 -- 172.21.15.169:0/1654438373 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f55ac15ba00 msgr2=0x7f55ac0b6f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:54.340 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.328+0000 7f55b4664640 1 --2- 172.21.15.169:0/1654438373 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f55ac15ba00 0x7f55ac0b6f00 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f559c009830 tx=0x7f559c02bd10 comp rx=0 tx=0).stop 2023-10-26T01:30:54.341 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.328+0000 7f89306cf640 1 -- 172.21.15.169:0/2721291895 shutdown_connections 2023-10-26T01:30:54.341 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.328+0000 7f6d5d960640 1 --2- 172.21.15.169:0/2610389553 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f6d3c06aec0 0x7f6d3c06d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:54.341 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.328+0000 7f89306cf640 1 -- 172.21.15.169:0/2721291895 wait complete. 2023-10-26T01:30:54.342 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.329+0000 7f55b4664640 1 -- 172.21.15.169:0/1654438373 shutdown_connections 2023-10-26T01:30:54.342 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.329+0000 7f55b4664640 1 --2- 172.21.15.169:0/1654438373 >> v2:172.21.15.169:3300/0 conn(0x7f55ac0d0fc0 0x7f55ac0d1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:54.342 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.329+0000 7f55b4664640 1 --2- 172.21.15.169:0/1654438373 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f55ac15ba00 0x7f55ac0b6f00 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:54.343 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.329+0000 7f55b4664640 1 -- 172.21.15.169:0/1654438373 >> 172.21.15.169:0/1654438373 conn(0x7f55ac06b5c0 msgr2=0x7f55ac06b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:54.343 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.330+0000 7f55b4664640 1 -- 172.21.15.169:0/1654438373 shutdown_connections 2023-10-26T01:30:54.343 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.329+0000 7f6d5e962640 1 --2- 172.21.15.169:0/2610389553 >> [v2:172.21.15.169:6816/4204255864,v1:172.21.15.169:6817/4204255864] conn(0x7f6d3c073dc0 0x7f6d3c076260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:54.344 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.331+0000 7f6d5d960640 1 --2- 172.21.15.169:0/2610389553 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f6d3c06aec0 0x7f6d3c06d380 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c007fd0 tx=0x7f6d4c009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:54.344 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.331+0000 7f7d5effd640 1 -- 172.21.15.169:0/2883679332 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f7d640107f0 con 0x7f7d60038300 2023-10-26T01:30:54.344 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.332+0000 7f89306cf640 1 Processor -- start 2023-10-26T01:30:54.345 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.332+0000 7f6d5e962640 1 --2- 172.21.15.169:0/2610389553 >> [v2:172.21.15.169:6816/4204255864,v1:172.21.15.169:6817/4204255864] conn(0x7f6d3c073dc0 0x7f6d3c076260 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:54.345 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.332+0000 7f89306cf640 1 -- start start 2023-10-26T01:30:54.346 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.330+0000 7f55b4664640 1 -- 172.21.15.169:0/1654438373 wait complete. 2023-10-26T01:30:54.346 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.333+0000 7f55b4664640 1 Processor -- start 2023-10-26T01:30:54.346 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.333+0000 7f7d5effd640 1 --2- 172.21.15.169:0/2883679332 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7d4c06ae40 0x7f7d4c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:54.347 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.333+0000 7f7d5effd640 1 -- 172.21.15.169:0/2883679332 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7d64014070 con 0x7f7d60038300 2023-10-26T01:30:54.347 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.333+0000 7f7d5effd640 1 --2- 172.21.15.169:0/2883679332 >> [v2:172.21.15.169:6808/238206046,v1:172.21.15.169:6809/238206046] conn(0x7f7d4c072fe0 0x7f7d4c075400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:54.347 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.333+0000 7f7d5effd640 1 -- 172.21.15.169:0/2883679332 --> [v2:172.21.15.169:6808/238206046,v1:172.21.15.169:6809/238206046] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7d4c075ab0 con 0x7f7d4c072fe0 2023-10-26T01:30:54.347 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.333+0000 7f7d5effd640 1 -- 172.21.15.169:0/2883679332 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7d6408ad70 con 0x7f7d60038300 2023-10-26T01:30:54.348 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.333+0000 7f55b4664640 1 -- start start 2023-10-26T01:30:54.348 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.334+0000 7f89306cf640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f89280b3380 0x7f892815d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:54.348 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.333+0000 7f6d4affd640 1 -- 172.21.15.169:0/2610389553 <== osd.0 v2:172.21.15.169:6816/4204255864 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23595 (crc 0 0 0) 0x7f6d3c076930 con 0x7f6d3c073dc0 2023-10-26T01:30:54.349 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.334+0000 7f89306cf640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89280b3760 0x7f8928151360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:54.350 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.334+0000 7f869fccf640 1 -- 172.21.15.169:0/798902367 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8698071940 msgr2=0x7f869806afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:54.350 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.334+0000 7f869fccf640 1 --2- 172.21.15.169:0/798902367 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8698071940 0x7f869806afe0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f86980ce3c0 tx=0x7f868802bdc0 comp rx=0 tx=0).stop 2023-10-26T01:30:54.350 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.335+0000 7f869fccf640 1 -- 172.21.15.169:0/798902367 shutdown_connections 2023-10-26T01:30:54.350 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.335+0000 7f869fccf640 1 --2- 172.21.15.169:0/798902367 >> v2:172.21.15.169:3300/0 conn(0x7f86980b0cf0 0x7f8698158f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:54.350 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.335+0000 7f869fccf640 1 --2- 172.21.15.169:0/798902367 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8698071940 0x7f869806afe0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:54.351 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.335+0000 7f892e46b640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f89280b3380 0x7f892815d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:54.351 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.335+0000 7f892e46b640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f89280b3380 0x7f892815d110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3301/0 says I am v2:172.21.15.169:53746/0 (socket says 172.21.15.169:53746) 2023-10-26T01:30:54.351 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.335+0000 7f869fccf640 1 -- 172.21.15.169:0/798902367 >> 172.21.15.169:0/798902367 conn(0x7f86980b2800 msgr2=0x7f86980b2be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:54.352 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.335+0000 7f869fccf640 1 -- 172.21.15.169:0/798902367 shutdown_connections 2023-10-26T01:30:54.352 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.335+0000 7f892dc6a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89280b3760 0x7f8928151360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:54.352 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.335+0000 7f892dc6a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89280b3760 0x7f8928151360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.169:57442/0 (socket says 172.21.15.169:57442) 2023-10-26T01:30:54.353 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.335+0000 7f89306cf640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f89280b72c0 0x7f89280c7550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:54.353 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.335+0000 7f869fccf640 1 -- 172.21.15.169:0/798902367 wait complete. 2023-10-26T01:30:54.354 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.335+0000 7f892ec6c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f89280b72c0 0x7f89280c7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:54.354 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.336+0000 7f892dc6a640 1 -- 172.21.15.169:0/3429524703 learned_addr learned my addr 172.21.15.169:0/3429524703 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:30:54.354 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.336+0000 7f89306cf640 1 -- 172.21.15.169:0/3429524703 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89281518a0 con 0x7f89280b72c0 2023-10-26T01:30:54.355 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.334+0000 7f7d6ebd6640 1 --2- 172.21.15.169:0/2883679332 >> [v2:172.21.15.169:6808/238206046,v1:172.21.15.169:6809/238206046] conn(0x7f7d4c072fe0 0x7f7d4c075400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:54.355 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.335+0000 7f7d6dbd4640 1 --2- 172.21.15.169:0/2883679332 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7d4c06ae40 0x7f7d4c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:54.355 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.336+0000 7f89306cf640 1 -- 172.21.15.169:0/3429524703 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8928151ae0 con 0x7f89280b3760 2023-10-26T01:30:54.356 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.336+0000 7f89306cf640 1 -- 172.21.15.169:0/3429524703 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8928151dc0 con 0x7f89280b3380 2023-10-26T01:30:54.356 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.337+0000 7f7d6dbd4640 1 --2- 172.21.15.169:0/2883679332 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7d4c06ae40 0x7f7d4c06d300 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7d600e1730 tx=0x7f7d58009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:54.357 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.337+0000 7f869fccf640 1 Processor -- start 2023-10-26T01:30:54.357 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.337+0000 7f892ec6c640 1 -- 172.21.15.169:0/3429524703 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f89280b3380 msgr2=0x7f892815d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:54.357 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.337+0000 7f892ec6c640 1 --2- 172.21.15.169:0/3429524703 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f89280b3380 0x7f892815d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:54.357 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.337+0000 7f892ec6c640 1 -- 172.21.15.169:0/3429524703 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89280b3760 msgr2=0x7f8928151360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:54.358 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.337+0000 7f892ec6c640 1 --2- 172.21.15.169:0/3429524703 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89280b3760 0x7f8928151360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:54.358 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.337+0000 7f892ec6c640 1 -- 172.21.15.169:0/3429524703 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8920009840 con 0x7f89280b72c0 2023-10-26T01:30:54.358 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.337+0000 7f7d6ebd6640 1 --2- 172.21.15.169:0/2883679332 >> [v2:172.21.15.169:6808/238206046,v1:172.21.15.169:6809/238206046] conn(0x7f7d4c072fe0 0x7f7d4c075400 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:54.359 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.338+0000 7f869fccf640 1 -- start start 2023-10-26T01:30:54.359 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.338+0000 7f9975de7640 1 -- 172.21.15.169:0/2400195447 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99700cf530 msgr2=0x7f99700cea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:54.359 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.338+0000 7f9975de7640 1 --2- 172.21.15.169:0/2400195447 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99700cf530 0x7f99700cea80 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f99700cf250 tx=0x7f996802c2f0 comp rx=0 tx=0).stop 2023-10-26T01:30:54.360 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.338+0000 7f892ec6c640 1 --2- 172.21.15.169:0/3429524703 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f89280b72c0 0x7f89280c7550 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f892000eb50 tx=0x7f892000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:54.360 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.338+0000 7f892e46b640 1 --2- 172.21.15.169:0/3429524703 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f89280b3380 0x7f892815d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:30:54.361 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.338+0000 7f9975de7640 1 -- 172.21.15.169:0/2400195447 shutdown_connections 2023-10-26T01:30:54.361 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.338+0000 7f9975de7640 1 --2- 172.21.15.169:0/2400195447 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99700cf530 0x7f99700cea80 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:54.361 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.338+0000 7f9975de7640 1 --2- 172.21.15.169:0/2400195447 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f997015bc00 0x7f99700ce540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:54.361 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.338+0000 7f9975de7640 1 --2- 172.21.15.169:0/2400195447 >> v2:172.21.15.170:3300/0 conn(0x7f997015b820 0x7f99700cc780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:54.362 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.338+0000 7f9975de7640 1 -- 172.21.15.169:0/2400195447 >> 172.21.15.169:0/2400195447 conn(0x7f9970065670 msgr2=0x7f9970065a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:54.362 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.338+0000 7f55b4664640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f55ac0d0fc0 0x7f55ac0b3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:54.363 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.338+0000 7f891effd640 1 -- 172.21.15.169:0/3429524703 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8920003b70 con 0x7f89280b72c0 2023-10-26T01:30:54.363 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.339+0000 7f891effd640 1 -- 172.21.15.169:0/3429524703 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8920003cd0 con 0x7f89280b72c0 2023-10-26T01:30:54.363 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.339+0000 7f89306cf640 1 -- 172.21.15.169:0/3429524703 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8924009560 con 0x7f89280b72c0 2023-10-26T01:30:54.363 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.339+0000 7f89306cf640 1 -- 172.21.15.169:0/3429524703 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8928152380 con 0x7f89280b72c0 2023-10-26T01:30:54.364 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.339+0000 7f891effd640 1 -- 172.21.15.169:0/3429524703 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8920004280 con 0x7f89280b72c0 2023-10-26T01:30:54.364 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.339+0000 7f55b4664640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55ac0d20c0 0x7f55ac1f84f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:54.364 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.339+0000 7f55b4664640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f55ac15ba00 0x7f55ac0b0d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:54.365 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.339+0000 7f55b4664640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55ac15e840 con 0x7f55ac0d20c0 2023-10-26T01:30:54.365 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.339+0000 7f55b4664640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55ac0b3ff0 con 0x7f55ac0d0fc0 2023-10-26T01:30:54.365 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.339+0000 7f55b4664640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55ac0c7cd0 con 0x7f55ac15ba00 2023-10-26T01:30:54.365 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.339+0000 7f55b1bff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55ac0d20c0 0x7f55ac1f84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:54.365 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.339+0000 7f55b2400640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f55ac0d0fc0 0x7f55ac0b3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:54.366 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.339+0000 7f55b1bff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55ac0d20c0 0x7f55ac1f84f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:45854/0 (socket says 172.21.15.169:45854) 2023-10-26T01:30:54.366 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.339+0000 7f55b1bff640 1 -- 172.21.15.169:0/2944803351 learned_addr learned my addr 172.21.15.169:0/2944803351 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:30:54.366 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.339+0000 7f55b2c01640 1 --2- 172.21.15.169:0/2944803351 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f55ac15ba00 0x7f55ac0b0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:54.366 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.340+0000 7f9975de7640 1 -- 172.21.15.169:0/2400195447 shutdown_connections 2023-10-26T01:30:54.367 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.340+0000 7f9975de7640 1 -- 172.21.15.169:0/2400195447 wait complete. 2023-10-26T01:30:54.367 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.340+0000 7f869fccf640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f869806be80 0x7f86980af4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:54.367 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.340+0000 7f869fccf640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8698071940 0x7f86981421d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:54.368 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.340+0000 7f869fccf640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f86980b0cf0 0x7f8698142710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:54.368 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.340+0000 7f869fccf640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86980ccab0 con 0x7f8698071940 2023-10-26T01:30:54.368 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.340+0000 7f869fccf640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8698142c50 con 0x7f869806be80 2023-10-26T01:30:54.368 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.340+0000 7f869fccf640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86980cbe30 con 0x7f86980b0cf0 2023-10-26T01:30:54.369 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.339+0000 7f7d5effd640 1 -- 172.21.15.169:0/2883679332 <== osd.1 v2:172.21.15.169:6808/238206046 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23595 (crc 0 0 0) 0x7f7d4c075ab0 con 0x7f7d4c072fe0 2023-10-26T01:30:54.369 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.340+0000 7f55b1bff640 1 -- 172.21.15.169:0/2944803351 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f55ac15ba00 msgr2=0x7f55ac0b0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:54.369 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.340+0000 7f55b1bff640 1 --2- 172.21.15.169:0/2944803351 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f55ac15ba00 0x7f55ac0b0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:54.369 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.340+0000 7f55b1bff640 1 -- 172.21.15.169:0/2944803351 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f55ac0d0fc0 msgr2=0x7f55ac0b3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:54.370 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.340+0000 7f55b1bff640 1 --2- 172.21.15.169:0/2944803351 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f55ac0d0fc0 0x7f55ac0b3ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:54.370 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.340+0000 7f55b1bff640 1 -- 172.21.15.169:0/2944803351 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f559c009490 con 0x7f55ac0d20c0 2023-10-26T01:30:54.370 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.340+0000 7f89306cf640 1 -- 172.21.15.169:0/3429524703 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8910000fc0 con 0x7f89280b72c0 2023-10-26T01:30:54.371 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.341+0000 7f869e26c640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f86980b0cf0 0x7f8698142710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:54.371 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.341+0000 7f55b1bff640 1 --2- 172.21.15.169:0/2944803351 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55ac0d20c0 0x7f55ac1f84f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f55a800d770 tx=0x7f55a800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:54.372 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.341+0000 7f869e26c640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f86980b0cf0 0x7f8698142710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3301/0 says I am v2:172.21.15.169:53762/0 (socket says 172.21.15.169:53762) 2023-10-26T01:30:54.372 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.341+0000 7f869e26c640 1 -- 172.21.15.169:0/220458226 learned_addr learned my addr 172.21.15.169:0/220458226 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:30:54.372 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.341+0000 7f869da6b640 1 --2- 172.21.15.169:0/220458226 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f869806be80 0x7f86980af4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:54.372 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.341+0000 7f869d26a640 1 --2- 172.21.15.169:0/220458226 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8698071940 0x7f86981421d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:54.372 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.342+0000 7f869e26c640 1 -- 172.21.15.169:0/220458226 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f869806be80 msgr2=0x7f86980af4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:54.373 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.342+0000 7f869e26c640 1 --2- 172.21.15.169:0/220458226 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f869806be80 0x7f86980af4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:54.373 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.342+0000 7f869e26c640 1 -- 172.21.15.169:0/220458226 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8698071940 msgr2=0x7f86981421d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:54.373 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.342+0000 7f869e26c640 1 --2- 172.21.15.169:0/220458226 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8698071940 0x7f86981421d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:54.373 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.342+0000 7f869e26c640 1 -- 172.21.15.169:0/220458226 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8688009560 con 0x7f86980b0cf0 2023-10-26T01:30:54.374 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.342+0000 7f55a2ffd640 1 -- 172.21.15.169:0/2944803351 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55a8009c30 con 0x7f55ac0d20c0 2023-10-26T01:30:54.374 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.342+0000 7f55a2ffd640 1 -- 172.21.15.169:0/2944803351 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f55a800de80 con 0x7f55ac0d20c0 2023-10-26T01:30:54.374 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.342+0000 7f55a2ffd640 1 -- 172.21.15.169:0/2944803351 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55a80106c0 con 0x7f55ac0d20c0 2023-10-26T01:30:54.374 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.342+0000 7f9975de7640 1 Processor -- start 2023-10-26T01:30:54.375 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.342+0000 7f55b4664640 1 -- 172.21.15.169:0/2944803351 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55ac0b2fd0 con 0x7f55ac0d20c0 2023-10-26T01:30:54.375 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.343+0000 7f9975de7640 1 -- start start 2023-10-26T01:30:54.377 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.342+0000 7f869e26c640 1 --2- 172.21.15.169:0/220458226 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f86980b0cf0 0x7f8698142710 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f869000ea70 tx=0x7f869000ef30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:54.378 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.342+0000 7f55b4664640 1 -- 172.21.15.169:0/2944803351 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55ac0b3590 con 0x7f55ac0d20c0 2023-10-26T01:30:54.378 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.344+0000 7f868e7fc640 1 -- 172.21.15.169:0/220458226 <== mon.2 v2:172.21.15.170:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f869000ce40 con 0x7f86980b0cf0 2023-10-26T01:30:54.379 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.344+0000 7f868e7fc640 1 -- 172.21.15.169:0/220458226 <== mon.2 v2:172.21.15.170:3301/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8690004c10 con 0x7f86980b0cf0 2023-10-26T01:30:54.379 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.344+0000 7f55b4664640 1 -- 172.21.15.169:0/2944803351 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f55ac0d0880 con 0x7f55ac0d20c0 2023-10-26T01:30:54.380 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.344+0000 7f868e7fc640 1 -- 172.21.15.169:0/220458226 <== mon.2 v2:172.21.15.170:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8690005020 con 0x7f86980b0cf0 2023-10-26T01:30:54.380 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.344+0000 7f869fccf640 1 -- 172.21.15.169:0/220458226 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86980cc110 con 0x7f86980b0cf0 2023-10-26T01:30:54.380 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.344+0000 7f869fccf640 1 -- 172.21.15.169:0/220458226 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86980cc6d0 con 0x7f86980b0cf0 2023-10-26T01:30:54.380 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.344+0000 7f9975de7640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99700cf530 0x7f997006ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:54.381 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.344+0000 7f9975de7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f997015b820 0x7f99701ffe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:54.381 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.344+0000 7f891effd640 1 -- 172.21.15.169:0/3429524703 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f89200043e0 con 0x7f89280b72c0 2023-10-26T01:30:54.382 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.344+0000 7f9975de7640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f997015bc00 0x7f99702022a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:54.382 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.344+0000 7f9975de7640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f997006f8c0 con 0x7f99700cf530 2023-10-26T01:30:54.382 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.344+0000 7f9975de7640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99700c6470 con 0x7f997015b820 2023-10-26T01:30:54.383 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.344+0000 7f9975de7640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99700c6750 con 0x7f997015bc00 2023-10-26T01:30:54.383 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.346+0000 7f891effd640 1 --2- 172.21.15.169:0/3429524703 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f890c06af10 0x7f890c06d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:54.384 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.344+0000 7f996f7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99700cf530 0x7f997006ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:54.384 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.344+0000 7f996f7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99700cf530 0x7f997006ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:45880/0 (socket says 172.21.15.169:45880) 2023-10-26T01:30:54.384 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.344+0000 7f996f7fe640 1 -- 172.21.15.169:0/1193889390 learned_addr learned my addr 172.21.15.169:0/1193889390 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:30:54.384 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.345+0000 7f996f7fe640 1 -- 172.21.15.169:0/1193889390 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f997015bc00 msgr2=0x7f99702022a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-26T01:30:54.385 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.345+0000 7f996f7fe640 1 --2- 172.21.15.169:0/1193889390 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f997015bc00 0x7f99702022a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:54.385 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.345+0000 7f996f7fe640 1 -- 172.21.15.169:0/1193889390 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f997015b820 msgr2=0x7f99701ffe70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-26T01:30:54.385 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.345+0000 7f996f7fe640 1 --2- 172.21.15.169:0/1193889390 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f997015b820 0x7f99701ffe70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:54.385 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.345+0000 7f996f7fe640 1 -- 172.21.15.169:0/1193889390 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9968009900 con 0x7f99700cf530 2023-10-26T01:30:54.386 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.346+0000 7f891effd640 1 -- 172.21.15.169:0/3429524703 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8920014070 con 0x7f89280b72c0 2023-10-26T01:30:54.386 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.346+0000 7f55a2ffd640 1 -- 172.21.15.169:0/2944803351 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f55a8010820 con 0x7f55ac0d20c0 2023-10-26T01:30:54.386 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.346+0000 7f891effd640 1 --2- 172.21.15.169:0/3429524703 >> [v2:172.21.15.169:6824/2256701226,v1:172.21.15.169:6825/2256701226] conn(0x7f890c073130 0x7f890c075550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:54.387 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.346+0000 7f891effd640 1 -- 172.21.15.169:0/3429524703 --> [v2:172.21.15.169:6824/2256701226,v1:172.21.15.169:6825/2256701226] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f890c075c00 con 0x7f890c073130 2023-10-26T01:30:54.387 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.346+0000 7f891effd640 1 -- 172.21.15.169:0/3429524703 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f892008a750 con 0x7f89280b72c0 2023-10-26T01:30:54.387 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.347+0000 7f869fccf640 1 -- 172.21.15.169:0/220458226 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f869815cda0 con 0x7f86980b0cf0 2023-10-26T01:30:54.388 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.347+0000 7f55a2ffd640 1 --2- 172.21.15.169:0/2944803351 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f559006ae40 0x7f559006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:54.388 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.347+0000 7f892dc6a640 1 --2- 172.21.15.169:0/3429524703 >> [v2:172.21.15.169:6824/2256701226,v1:172.21.15.169:6825/2256701226] conn(0x7f890c073130 0x7f890c075550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:54.389 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.348+0000 7f996f7fe640 1 --2- 172.21.15.169:0/1193889390 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99700cf530 0x7f997006ef50 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f996000d770 tx=0x7f996000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:54.389 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.348+0000 7f892dc6a640 1 --2- 172.21.15.169:0/3429524703 >> [v2:172.21.15.169:6824/2256701226,v1:172.21.15.169:6825/2256701226] conn(0x7f890c073130 0x7f890c075550 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:54.389 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.348+0000 7f55b2400640 1 --2- 172.21.15.169:0/2944803351 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f559006ae40 0x7f559006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:54.390 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.348+0000 7f9957fff640 1 -- 172.21.15.169:0/1193889390 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9960009c30 con 0x7f99700cf530 2023-10-26T01:30:54.390 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.348+0000 7f9957fff640 1 -- 172.21.15.169:0/1193889390 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9960009e80 con 0x7f99700cf530 2023-10-26T01:30:54.390 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.348+0000 7f9957fff640 1 -- 172.21.15.169:0/1193889390 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99600106a0 con 0x7f99700cf530 2023-10-26T01:30:54.391 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.349+0000 7f892e46b640 1 --2- 172.21.15.169:0/3429524703 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f890c06af10 0x7f890c06d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:54.391 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.349+0000 7f9975de7640 1 -- 172.21.15.169:0/1193889390 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99700c6a30 con 0x7f99700cf530 2023-10-26T01:30:54.392 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.349+0000 7f9975de7640 1 -- 172.21.15.169:0/1193889390 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f997006f490 con 0x7f99700cf530 2023-10-26T01:30:54.392 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.348+0000 7f55a2ffd640 1 -- 172.21.15.169:0/2944803351 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f55a8089b60 con 0x7f55ac0d20c0 2023-10-26T01:30:54.392 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.349+0000 7f55a2ffd640 1 --2- 172.21.15.169:0/2944803351 >> [v2:172.21.15.169:6800/3215791846,v1:172.21.15.169:6801/3215791846] conn(0x7f5590072fe0 0x7f5590075400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:54.392 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.349+0000 7f55a2ffd640 1 -- 172.21.15.169:0/2944803351 --> [v2:172.21.15.169:6800/3215791846,v1:172.21.15.169:6801/3215791846] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5590075ab0 con 0x7f5590072fe0 2023-10-26T01:30:54.393 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.349+0000 7f55a2ffd640 1 -- 172.21.15.169:0/2944803351 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f55a8054c10 con 0x7f55ac0d20c0 2023-10-26T01:30:54.393 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.350+0000 7fad07807640 1 -- 172.21.15.169:0/3000808657 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fad001593e0 msgr2=0x7fad000ac200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:54.393 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.350+0000 7fad07807640 1 --2- 172.21.15.169:0/3000808657 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fad001593e0 0x7fad000ac200 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7facfc009900 tx=0x7facfc02beb0 comp rx=0 tx=0).stop 2023-10-26T01:30:54.394 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.350+0000 7f892e46b640 1 --2- 172.21.15.169:0/3429524703 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f890c06af10 0x7f890c06d3d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8924035000 tx=0x7f8924004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:54.394 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.350+0000 7f9975de7640 1 -- 172.21.15.169:0/1193889390 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9970070a60 con 0x7f99700cf530 2023-10-26T01:30:54.395 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.350+0000 7fad07807640 1 -- 172.21.15.169:0/3000808657 shutdown_connections 2023-10-26T01:30:54.395 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.350+0000 7fad07807640 1 --2- 172.21.15.169:0/3000808657 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fad001593e0 0x7fad000ac200 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:54.395 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.350+0000 7fad07807640 1 -- 172.21.15.169:0/3000808657 >> 172.21.15.169:0/3000808657 conn(0x7fad00069930 msgr2=0x7fad00069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:54.395 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.350+0000 7f55b2400640 1 --2- 172.21.15.169:0/2944803351 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f559006ae40 0x7f559006d300 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f559c036000 tx=0x7f559c004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:54.396 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.350+0000 7fad07807640 1 -- 172.21.15.169:0/3000808657 shutdown_connections 2023-10-26T01:30:54.396 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.351+0000 7f891effd640 1 -- 172.21.15.169:0/3429524703 <== osd.3 v2:172.21.15.169:6824/2256701226 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23595 (crc 0 0 0) 0x7f890c075c00 con 0x7f890c073130 2023-10-26T01:30:54.397 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.350+0000 7fad07807640 1 -- 172.21.15.169:0/3000808657 wait complete. 2023-10-26T01:30:54.397 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.351+0000 7f692a6d7640 1 -- 172.21.15.169:0/3283412492 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f69240d0fc0 msgr2=0x7f69240d1b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:54.398 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.351+0000 7f692a6d7640 1 --2- 172.21.15.169:0/3283412492 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f69240d0fc0 0x7f69240d1b80 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6914009860 tx=0x7f691402bcb0 comp rx=0 tx=0).stop 2023-10-26T01:30:54.398 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.351+0000 7f692a6d7640 1 -- 172.21.15.169:0/3283412492 shutdown_connections 2023-10-26T01:30:54.398 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.351+0000 7f692a6d7640 1 --2- 172.21.15.169:0/3283412492 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f69240d0fc0 0x7f69240d1b80 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:54.398 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.351+0000 7f692a6d7640 1 -- 172.21.15.169:0/3283412492 >> 172.21.15.169:0/3283412492 conn(0x7f692406b5c0 msgr2=0x7f692406b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:54.399 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.352+0000 7f868e7fc640 1 -- 172.21.15.169:0/220458226 <== mon.2 v2:172.21.15.170:3301/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f8690005180 con 0x7f86980b0cf0 2023-10-26T01:30:54.399 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.352+0000 7f55b2c01640 1 --2- 172.21.15.169:0/2944803351 >> [v2:172.21.15.169:6800/3215791846,v1:172.21.15.169:6801/3215791846] conn(0x7f5590072fe0 0x7f5590075400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:54.400 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.352+0000 7fad07807640 1 Processor -- start 2023-10-26T01:30:54.400 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.353+0000 7fad07807640 1 -- start start 2023-10-26T01:30:54.400 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.354+0000 7f692a6d7640 1 -- 172.21.15.169:0/3283412492 shutdown_connections 2023-10-26T01:30:54.401 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.354+0000 7f692a6d7640 1 -- 172.21.15.169:0/3283412492 wait complete. 2023-10-26T01:30:54.401 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.354+0000 7f55b2c01640 1 --2- 172.21.15.169:0/2944803351 >> [v2:172.21.15.169:6800/3215791846,v1:172.21.15.169:6801/3215791846] conn(0x7f5590072fe0 0x7f5590075400 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:54.402 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.354+0000 7fad07807640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fad000b0c80 0x7fad001572a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:54.402 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.354+0000 7fad07807640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad001593e0 0x7fad00154de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:54.402 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.354+0000 7fad07807640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fad001597c0 0x7fad000adc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:54.402 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.354+0000 7fad07807640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad00155e60 con 0x7fad001593e0 2023-10-26T01:30:54.403 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.354+0000 7fad07807640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad00156140 con 0x7fad001597c0 2023-10-26T01:30:54.403 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.354+0000 7fad07807640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad00156420 con 0x7fad000b0c80 2023-10-26T01:30:54.403 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.355+0000 7f55a2ffd640 1 -- 172.21.15.169:0/2944803351 <== osd.2 v2:172.21.15.169:6800/3215791846 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23595 (crc 0 0 0) 0x7f5590075ab0 con 0x7f5590072fe0 2023-10-26T01:30:54.404 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.355+0000 7fad055a3640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fad000b0c80 0x7fad001572a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:54.404 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.355+0000 7fad055a3640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fad000b0c80 0x7fad001572a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3301/0 says I am v2:172.21.15.169:53782/0 (socket says 172.21.15.169:53782) 2023-10-26T01:30:54.404 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.355+0000 7fad055a3640 1 -- 172.21.15.169:0/3306384886 learned_addr learned my addr 172.21.15.169:0/3306384886 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:30:54.404 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.355+0000 7fad04da2640 1 --2- 172.21.15.169:0/3306384886 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad001593e0 0x7fad00154de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:54.405 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.356+0000 7fad05da4640 1 --2- 172.21.15.169:0/3306384886 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fad001597c0 0x7fad000adc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:54.405 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.356+0000 7f692a6d7640 1 Processor -- start 2023-10-26T01:30:54.405 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.357+0000 7fad04da2640 1 -- 172.21.15.169:0/3306384886 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fad000b0c80 msgr2=0x7fad001572a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:54.406 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.357+0000 7fad04da2640 1 --2- 172.21.15.169:0/3306384886 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fad000b0c80 0x7fad001572a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:54.406 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.357+0000 7fad04da2640 1 -- 172.21.15.169:0/3306384886 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fad001597c0 msgr2=0x7fad000adc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:54.406 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.357+0000 7fad04da2640 1 --2- 172.21.15.169:0/3306384886 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fad001597c0 0x7fad000adc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:54.406 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.357+0000 7fad04da2640 1 -- 172.21.15.169:0/3306384886 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facfc009560 con 0x7fad001593e0 2023-10-26T01:30:54.407 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.357+0000 7fad05da4640 1 --2- 172.21.15.169:0/3306384886 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fad001597c0 0x7fad000adc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T01:30:54.407 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.353+0000 7f868e7fc640 1 --2- 172.21.15.169:0/220458226 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f867c06af10 0x7f867c06d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:54.407 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.353+0000 7f868e7fc640 1 -- 172.21.15.169:0/220458226 <== mon.2 v2:172.21.15.170:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8690014070 con 0x7f86980b0cf0 2023-10-26T01:30:54.408 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.353+0000 7f868e7fc640 1 --2- 172.21.15.169:0/220458226 >> [v2:172.21.15.170:6808/3335654351,v1:172.21.15.170:6809/3335654351] conn(0x7f867c073130 0x7f867c075550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:54.408 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.358+0000 7f9957fff640 1 -- 172.21.15.169:0/1193889390 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f9960010800 con 0x7f99700cf530 2023-10-26T01:30:54.409 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.353+0000 7f868e7fc640 1 -- 172.21.15.169:0/220458226 --> [v2:172.21.15.170:6808/3335654351,v1:172.21.15.170:6809/3335654351] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f867c075c00 con 0x7f867c073130 2023-10-26T01:30:54.409 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.353+0000 7f868e7fc640 1 -- 172.21.15.169:0/220458226 <== mon.2 v2:172.21.15.170:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f869008ad00 con 0x7f86980b0cf0 2023-10-26T01:30:54.409 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.354+0000 7f869d26a640 1 --2- 172.21.15.169:0/220458226 >> [v2:172.21.15.170:6808/3335654351,v1:172.21.15.170:6809/3335654351] conn(0x7f867c073130 0x7f867c075550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:54.409 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.355+0000 7f869da6b640 1 --2- 172.21.15.169:0/220458226 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f867c06af10 0x7f867c06d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:54.410 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.356+0000 7f869d26a640 1 --2- 172.21.15.169:0/220458226 >> [v2:172.21.15.170:6808/3335654351,v1:172.21.15.170:6809/3335654351] conn(0x7f867c073130 0x7f867c075550 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:54.410 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.357+0000 7f869da6b640 1 --2- 172.21.15.169:0/220458226 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f867c06af10 0x7f867c06d3d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8698065b80 tx=0x7f8694009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:54.410 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.356+0000 7f692a6d7640 1 -- start start 2023-10-26T01:30:54.411 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.358+0000 7f868e7fc640 1 -- 172.21.15.169:0/220458226 <== osd.5 v2:172.21.15.170:6808/3335654351 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23595 (crc 0 0 0) 0x7f867c075c00 con 0x7f867c073130 2023-10-26T01:30:54.411 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.358+0000 7fad04da2640 1 --2- 172.21.15.169:0/3306384886 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad001593e0 0x7fad00154de0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7facf000c390 tx=0x7facf000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:54.412 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.358+0000 7facf5ffb640 1 -- 172.21.15.169:0/3306384886 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf00052d0 con 0x7fad001593e0 2023-10-26T01:30:54.412 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.358+0000 7facf5ffb640 1 -- 172.21.15.169:0/3306384886 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7facf00035f0 con 0x7fad001593e0 2023-10-26T01:30:54.412 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.358+0000 7facf5ffb640 1 -- 172.21.15.169:0/3306384886 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf00052d0 con 0x7fad001593e0 2023-10-26T01:30:54.412 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.358+0000 7fad07807640 1 -- 172.21.15.169:0/3306384886 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad00156700 con 0x7fad001593e0 2023-10-26T01:30:54.413 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.358+0000 7f692a6d7640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f69240d0fc0 0x7f6924150c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:54.413 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.359+0000 7f9957fff640 1 --2- 172.21.15.169:0/1193889390 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f995c06ae40 0x7f995c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:54.413 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.359+0000 7f9957fff640 1 -- 172.21.15.169:0/1193889390 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f996008a110 con 0x7f99700cf530 2023-10-26T01:30:54.413 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.359+0000 7f9957fff640 1 --2- 172.21.15.169:0/1193889390 >> [v2:172.21.15.170:6816/1082279874,v1:172.21.15.170:6817/1082279874] conn(0x7f995c072fe0 0x7f995c075400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:54.414 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.359+0000 7f9957fff640 1 -- 172.21.15.169:0/1193889390 --> [v2:172.21.15.170:6816/1082279874,v1:172.21.15.170:6817/1082279874] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f995c075ab0 con 0x7f995c072fe0 2023-10-26T01:30:54.414 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.359+0000 7f9957fff640 1 -- 172.21.15.169:0/1193889390 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f996008a560 con 0x7f99700cf530 2023-10-26T01:30:54.415 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.359+0000 7f692a6d7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f69240d20c0 0x7f69241e61a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:54.415 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.359+0000 7f692a6d7640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f692415ba00 0x7f69241e66e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:54.415 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.359+0000 7f692a6d7640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6924151150 con 0x7f69240d0fc0 2023-10-26T01:30:54.415 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.359+0000 7f692a6d7640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69240c5250 con 0x7f69240d20c0 2023-10-26T01:30:54.416 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.359+0000 7f692a6d7640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69240c5530 con 0x7f692415ba00 2023-10-26T01:30:54.416 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.359+0000 7f69237fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f69240d20c0 0x7f69241e61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:54.416 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.360+0000 7f69237fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f69240d20c0 0x7f69241e61a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.169:57454/0 (socket says 172.21.15.169:57454) 2023-10-26T01:30:54.416 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.360+0000 7f69237fe640 1 -- 172.21.15.169:0/3018602487 learned_addr learned my addr 172.21.15.169:0/3018602487 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:30:54.416 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.360+0000 7f6923fff640 1 --2- 172.21.15.169:0/3018602487 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f69240d0fc0 0x7f6924150c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:54.417 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.360+0000 7f6928c74640 1 --2- 172.21.15.169:0/3018602487 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f692415ba00 0x7f69241e66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:54.417 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.362+0000 7f69237fe640 1 -- 172.21.15.169:0/3018602487 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f692415ba00 msgr2=0x7f69241e66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:54.417 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.362+0000 7f996effd640 1 --2- 172.21.15.169:0/1193889390 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f995c06ae40 0x7f995c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:54.418 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.362+0000 7f996ffff640 1 --2- 172.21.15.169:0/1193889390 >> [v2:172.21.15.170:6816/1082279874,v1:172.21.15.170:6817/1082279874] conn(0x7f995c072fe0 0x7f995c075400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:54.418 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.363+0000 7f69237fe640 1 --2- 172.21.15.169:0/3018602487 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f692415ba00 0x7f69241e66e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:54.419 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.363+0000 7facf5ffb640 1 -- 172.21.15.169:0/3306384886 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7facf00037a0 con 0x7fad001593e0 2023-10-26T01:30:54.419 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.363+0000 7fad07807640 1 -- 172.21.15.169:0/3306384886 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad001577e0 con 0x7fad001593e0 2023-10-26T01:30:54.419 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.363+0000 7f996ffff640 1 --2- 172.21.15.169:0/1193889390 >> [v2:172.21.15.170:6816/1082279874,v1:172.21.15.170:6817/1082279874] conn(0x7f995c072fe0 0x7f995c075400 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:54.420 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.363+0000 7f69237fe640 1 -- 172.21.15.169:0/3018602487 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f69240d0fc0 msgr2=0x7f6924150c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:54.420 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.363+0000 7f6d603c5640 1 -- 172.21.15.169:0/2610389553 --> [v2:172.21.15.169:6816/4204255864,v1:172.21.15.169:6817/4204255864] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f6d500f9b70 con 0x7f6d3c073dc0 2023-10-26T01:30:54.420 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.364+0000 7fad07807640 1 -- 172.21.15.169:0/3306384886 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fad000b1410 con 0x7fad001593e0 2023-10-26T01:30:54.421 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.364+0000 7f9957fff640 1 -- 172.21.15.169:0/1193889390 <== osd.4 v2:172.21.15.170:6816/1082279874 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23595 (crc 0 0 0) 0x7f995c075ab0 con 0x7f995c072fe0 2023-10-26T01:30:54.421 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.364+0000 7f6d4affd640 1 -- 172.21.15.169:0/2610389553 <== osd.0 v2:172.21.15.169:6816/4204255864 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f6d500f9b70 con 0x7f6d3c073dc0 2023-10-26T01:30:54.422 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.363+0000 7f69237fe640 1 --2- 172.21.15.169:0/3018602487 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f69240d0fc0 0x7f6924150c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:54.422 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.363+0000 7f69237fe640 1 -- 172.21.15.169:0/3018602487 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6914009490 con 0x7f69240d20c0 2023-10-26T01:30:54.422 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.363+0000 7f6923fff640 1 --2- 172.21.15.169:0/3018602487 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f69240d0fc0 0x7f6924150c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T01:30:54.423 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.363+0000 7f6928c74640 1 --2- 172.21.15.169:0/3018602487 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f692415ba00 0x7f69241e66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T01:30:54.423 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.363+0000 7f69237fe640 1 --2- 172.21.15.169:0/3018602487 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f69240d20c0 0x7f69241e61a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6914000c00 tx=0x7f69140038a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:54.423 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.364+0000 7f6920ff9640 1 -- 172.21.15.169:0/3018602487 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6914003ea0 con 0x7f69240d20c0 2023-10-26T01:30:54.423 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.364+0000 7f6920ff9640 1 -- 172.21.15.169:0/3018602487 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6914005110 con 0x7f69240d20c0 2023-10-26T01:30:54.424 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.364+0000 7f6920ff9640 1 -- 172.21.15.169:0/3018602487 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6914005520 con 0x7f69240d20c0 2023-10-26T01:30:54.424 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.364+0000 7f692a6d7640 1 -- 172.21.15.169:0/3018602487 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69240c5810 con 0x7f69240d20c0 2023-10-26T01:30:54.424 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.365+0000 7facf5ffb640 1 --2- 172.21.15.169:0/3306384886 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7face406aea0 0x7face406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:54.424 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.365+0000 7facf5ffb640 1 -- 172.21.15.169:0/3306384886 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7facf0088630 con 0x7fad001593e0 2023-10-26T01:30:54.425 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.365+0000 7facf5ffb640 1 --2- 172.21.15.169:0/3306384886 >> [v2:172.21.15.170:6800/2520460697,v1:172.21.15.170:6801/2520460697] conn(0x7face4073040 0x7face4075460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:54.425 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.365+0000 7facf5ffb640 1 -- 172.21.15.169:0/3306384886 --> [v2:172.21.15.170:6800/2520460697,v1:172.21.15.170:6801/2520460697] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7face4075b10 con 0x7face4073040 2023-10-26T01:30:54.425 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.365+0000 7facf5ffb640 1 -- 172.21.15.169:0/3306384886 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7facf00889d0 con 0x7fad001593e0 2023-10-26T01:30:54.425 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.365+0000 7f6d603c5640 1 -- 172.21.15.169:0/2610389553 >> [v2:172.21.15.169:6816/4204255864,v1:172.21.15.169:6817/4204255864] conn(0x7f6d3c073dc0 msgr2=0x7f6d3c076260 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:54.426 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.365+0000 7f6d603c5640 1 --2- 172.21.15.169:0/2610389553 >> [v2:172.21.15.169:6816/4204255864,v1:172.21.15.169:6817/4204255864] conn(0x7f6d3c073dc0 0x7f6d3c076260 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:54.426 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.365+0000 7f996effd640 1 --2- 172.21.15.169:0/1193889390 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f995c06ae40 0x7f995c06d300 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f99700c4bc0 tx=0x7f9964005ae0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:54.426 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.366+0000 7f6d603c5640 1 -- 172.21.15.169:0/2610389553 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f6d3c06aec0 msgr2=0x7f6d3c06d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:54.427 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.366+0000 7f6d603c5640 1 --2- 172.21.15.169:0/2610389553 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f6d3c06aec0 0x7f6d3c06d380 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c007fd0 tx=0x7f6d4c009040 comp rx=0 tx=0).stop 2023-10-26T01:30:54.427 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.366+0000 7f6d603c5640 1 -- 172.21.15.169:0/2610389553 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d50007160 msgr2=0x7f6d500efa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:54.427 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.366+0000 7f6d603c5640 1 --2- 172.21.15.169:0/2610389553 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d50007160 0x7f6d500efa10 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6d580784d0 tx=0x7f6d5809a970 comp rx=0 tx=0).stop 2023-10-26T01:30:54.427 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.366+0000 7fad055a3640 1 --2- 172.21.15.169:0/3306384886 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7face406aea0 0x7face406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:54.428 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.366+0000 7f6d5e962640 1 -- 172.21.15.169:0/2610389553 reap_dead start 2023-10-26T01:30:54.428 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.366+0000 7f6d603c5640 1 -- 172.21.15.169:0/2610389553 shutdown_connections 2023-10-26T01:30:54.428 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.366+0000 7f6d603c5640 1 -- 172.21.15.169:0/2610389553 >> 172.21.15.169:0/2610389553 conn(0x7f6d50009f80 msgr2=0x7f6d500f69f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:54.429 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.366+0000 7f6d603c5640 1 -- 172.21.15.169:0/2610389553 shutdown_connections 2023-10-26T01:30:54.429 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.366+0000 7f6d603c5640 1 -- 172.21.15.169:0/2610389553 wait complete. 2023-10-26T01:30:54.429 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.366+0000 7f692a6d7640 1 -- 172.21.15.169:0/3018602487 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69240c7780 con 0x7f69240d20c0 2023-10-26T01:30:54.430 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.366+0000 7fad05da4640 1 --2- 172.21.15.169:0/3306384886 >> [v2:172.21.15.170:6800/2520460697,v1:172.21.15.170:6801/2520460697] conn(0x7face4073040 0x7face4075460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:54.430 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.367+0000 7f692a6d7640 1 -- 172.21.15.169:0/3018602487 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f692404db10 con 0x7f69240d20c0 2023-10-26T01:30:54.431 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.368+0000 7fad05da4640 1 --2- 172.21.15.169:0/3306384886 >> [v2:172.21.15.170:6800/2520460697,v1:172.21.15.170:6801/2520460697] conn(0x7face4073040 0x7face4075460 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:54.431 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.369+0000 7fad055a3640 1 --2- 172.21.15.169:0/3306384886 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7face406aea0 0x7face406d360 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7facfc009900 tx=0x7facfc02e040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:54.431 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.369+0000 7facf5ffb640 1 -- 172.21.15.169:0/3306384886 <== osd.6 v2:172.21.15.170:6800/2520460697 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23595 (crc 0 0 0) 0x7face4075b10 con 0x7face4073040 2023-10-26T01:30:54.433 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.370+0000 7f6920ff9640 1 -- 172.21.15.169:0/3018602487 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f691403b040 con 0x7f69240d20c0 2023-10-26T01:30:54.433 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.372+0000 7f6920ff9640 1 --2- 172.21.15.169:0/3018602487 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f691006ae40 0x7f691006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:54.433 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.372+0000 7f6920ff9640 1 -- 172.21.15.169:0/3018602487 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f69140ac060 con 0x7f69240d20c0 2023-10-26T01:30:54.433 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.372+0000 7f6920ff9640 1 --2- 172.21.15.169:0/3018602487 >> [v2:172.21.15.170:6821/1115930619,v1:172.21.15.170:6822/1115930619] conn(0x7f6910073060 0x7f6910075480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:54.433 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.372+0000 7f6920ff9640 1 -- 172.21.15.169:0/3018602487 --> [v2:172.21.15.170:6821/1115930619,v1:172.21.15.170:6822/1115930619] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6910075b30 con 0x7f6910073060 2023-10-26T01:30:54.434 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.372+0000 7f6920ff9640 1 -- 172.21.15.169:0/3018602487 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f69140ac410 con 0x7f69240d20c0 2023-10-26T01:30:54.434 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.373+0000 7f6923fff640 1 --2- 172.21.15.169:0/3018602487 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f691006ae40 0x7f691006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:54.434 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.373+0000 7f6928c74640 1 --2- 172.21.15.169:0/3018602487 >> [v2:172.21.15.170:6821/1115930619,v1:172.21.15.170:6822/1115930619] conn(0x7f6910073060 0x7f6910075480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:54.434 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.374+0000 7f6928c74640 1 --2- 172.21.15.169:0/3018602487 >> [v2:172.21.15.170:6821/1115930619,v1:172.21.15.170:6822/1115930619] conn(0x7f6910073060 0x7f6910075480 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:54.435 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.375+0000 7f7d6f3d7640 1 -- 172.21.15.169:0/2883679332 --> [v2:172.21.15.169:6808/238206046,v1:172.21.15.169:6809/238206046] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7d60052c20 con 0x7f7d4c072fe0 2023-10-26T01:30:54.435 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.375+0000 7f6923fff640 1 --2- 172.21.15.169:0/3018602487 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f691006ae40 0x7f691006d300 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f691c00a7f0 tx=0x7f691c00d040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:54.435 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.376+0000 7f7d5effd640 1 -- 172.21.15.169:0/2883679332 <== osd.1 v2:172.21.15.169:6808/238206046 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f7d68061040 con 0x7f7d4c072fe0 2023-10-26T01:30:54.436 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.376+0000 7f6920ff9640 1 -- 172.21.15.169:0/3018602487 <== osd.7 v2:172.21.15.170:6821/1115930619 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23595 (crc 0 0 0) 0x7f6910075b30 con 0x7f6910073060 2023-10-26T01:30:54.437 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.376+0000 7f7d6f3d7640 1 -- 172.21.15.169:0/2883679332 >> [v2:172.21.15.169:6808/238206046,v1:172.21.15.169:6809/238206046] conn(0x7f7d4c072fe0 msgr2=0x7f7d4c075400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:54.437 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.376+0000 7f7d6f3d7640 1 --2- 172.21.15.169:0/2883679332 >> [v2:172.21.15.169:6808/238206046,v1:172.21.15.169:6809/238206046] conn(0x7f7d4c072fe0 0x7f7d4c075400 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:54.437 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.379+0000 7f7d6f3d7640 1 -- 172.21.15.169:0/2883679332 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7d4c06ae40 msgr2=0x7f7d4c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:54.438 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.379+0000 7f7d6f3d7640 1 --2- 172.21.15.169:0/2883679332 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7d4c06ae40 0x7f7d4c06d300 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7d600e1730 tx=0x7f7d58009040 comp rx=0 tx=0).stop 2023-10-26T01:30:54.438 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.379+0000 7f7d6f3d7640 1 -- 172.21.15.169:0/2883679332 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d60038300 msgr2=0x7f7d600d7b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:54.438 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.379+0000 7f7d6f3d7640 1 --2- 172.21.15.169:0/2883679332 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d60038300 0x7f7d600d7b10 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7d6400e790 tx=0x7f7d6400ec50 comp rx=0 tx=0).stop 2023-10-26T01:30:54.438 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.379+0000 7f7d6ebd6640 1 -- 172.21.15.169:0/2883679332 reap_dead start 2023-10-26T01:30:54.438 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.379+0000 7f7d6f3d7640 1 -- 172.21.15.169:0/2883679332 shutdown_connections 2023-10-26T01:30:54.439 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.379+0000 7f89306cf640 1 -- 172.21.15.169:0/3429524703 --> [v2:172.21.15.169:6824/2256701226,v1:172.21.15.169:6825/2256701226] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8910002cb0 con 0x7f890c073130 2023-10-26T01:30:54.439 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.379+0000 7f7d6f3d7640 1 -- 172.21.15.169:0/2883679332 >> 172.21.15.169:0/2883679332 conn(0x7f7d60009f80 msgr2=0x7f7d6016a530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:54.440 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.380+0000 7f891effd640 1 -- 172.21.15.169:0/3429524703 <== osd.3 v2:172.21.15.169:6824/2256701226 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f8910002cb0 con 0x7f890c073130 2023-10-26T01:30:54.440 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.381+0000 7f7d6f3d7640 1 -- 172.21.15.169:0/2883679332 shutdown_connections 2023-10-26T01:30:54.440 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.381+0000 7f7d6f3d7640 1 -- 172.21.15.169:0/2883679332 wait complete. 2023-10-26T01:30:54.442 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.381+0000 7f89306cf640 1 -- 172.21.15.169:0/3429524703 >> [v2:172.21.15.169:6824/2256701226,v1:172.21.15.169:6825/2256701226] conn(0x7f890c073130 msgr2=0x7f890c075550 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:54.442 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.381+0000 7f89306cf640 1 --2- 172.21.15.169:0/3429524703 >> [v2:172.21.15.169:6824/2256701226,v1:172.21.15.169:6825/2256701226] conn(0x7f890c073130 0x7f890c075550 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:54.442 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.381+0000 7f89306cf640 1 -- 172.21.15.169:0/3429524703 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f890c06af10 msgr2=0x7f890c06d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:54.443 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.381+0000 7f89306cf640 1 --2- 172.21.15.169:0/3429524703 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f890c06af10 0x7f890c06d3d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8924035000 tx=0x7f8924004730 comp rx=0 tx=0).stop 2023-10-26T01:30:54.443 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.381+0000 7f89306cf640 1 -- 172.21.15.169:0/3429524703 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f89280b72c0 msgr2=0x7f89280c7550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:54.443 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.381+0000 7f89306cf640 1 --2- 172.21.15.169:0/3429524703 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f89280b72c0 0x7f89280c7550 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f892000eb50 tx=0x7f892000c6a0 comp rx=0 tx=0).stop 2023-10-26T01:30:54.443 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.382+0000 7f892ec6c640 1 -- 172.21.15.169:0/3429524703 reap_dead start 2023-10-26T01:30:54.443 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.382+0000 7f89306cf640 1 -- 172.21.15.169:0/3429524703 shutdown_connections 2023-10-26T01:30:54.443 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.382+0000 7f89306cf640 1 -- 172.21.15.169:0/3429524703 >> 172.21.15.169:0/3429524703 conn(0x7f89280658b0 msgr2=0x7f89281538d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:54.444 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.383+0000 7f89306cf640 1 -- 172.21.15.169:0/3429524703 shutdown_connections 2023-10-26T01:30:54.444 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.383+0000 7f89306cf640 1 -- 172.21.15.169:0/3429524703 wait complete. 2023-10-26T01:30:54.444 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.385+0000 7f869fccf640 1 -- 172.21.15.169:0/220458226 --> [v2:172.21.15.170:6808/3335654351,v1:172.21.15.170:6809/3335654351] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f869804db10 con 0x7f867c073130 2023-10-26T01:30:54.445 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.387+0000 7f55b4664640 1 -- 172.21.15.169:0/2944803351 --> [v2:172.21.15.169:6800/3215791846,v1:172.21.15.169:6801/3215791846] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f55ac0b3780 con 0x7f5590072fe0 2023-10-26T01:30:54.445 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.388+0000 7f868e7fc640 1 -- 172.21.15.169:0/220458226 <== osd.5 v2:172.21.15.170:6808/3335654351 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f869804db10 con 0x7f867c073130 2023-10-26T01:30:54.446 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.388+0000 7f869fccf640 1 -- 172.21.15.169:0/220458226 >> [v2:172.21.15.170:6808/3335654351,v1:172.21.15.170:6809/3335654351] conn(0x7f867c073130 msgr2=0x7f867c075550 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:54.446 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.388+0000 7f869fccf640 1 --2- 172.21.15.169:0/220458226 >> [v2:172.21.15.170:6808/3335654351,v1:172.21.15.170:6809/3335654351] conn(0x7f867c073130 0x7f867c075550 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:54.446 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.388+0000 7f869fccf640 1 -- 172.21.15.169:0/220458226 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f867c06af10 msgr2=0x7f867c06d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:54.446 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.388+0000 7f869fccf640 1 --2- 172.21.15.169:0/220458226 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f867c06af10 0x7f867c06d3d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8698065b80 tx=0x7f8694009040 comp rx=0 tx=0).stop 2023-10-26T01:30:54.447 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.388+0000 7f869fccf640 1 -- 172.21.15.169:0/220458226 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f86980b0cf0 msgr2=0x7f8698142710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:54.447 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.388+0000 7f869fccf640 1 --2- 172.21.15.169:0/220458226 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f86980b0cf0 0x7f8698142710 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f869000ea70 tx=0x7f869000ef30 comp rx=0 tx=0).stop 2023-10-26T01:30:54.447 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.388+0000 7f55a2ffd640 1 -- 172.21.15.169:0/2944803351 <== osd.2 v2:172.21.15.169:6800/3215791846 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f55ac0b3780 con 0x7f5590072fe0 2023-10-26T01:30:54.448 INFO:teuthology.orchestra.run.smithi169.stdout:55834574854 2023-10-26T01:30:54.449 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-26T01:30:54.450 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.391+0000 7f869e26c640 1 -- 172.21.15.169:0/220458226 reap_dead start 2023-10-26T01:30:54.450 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.391+0000 7f9975de7640 1 -- 172.21.15.169:0/1193889390 --> [v2:172.21.15.170:6816/1082279874,v1:172.21.15.170:6817/1082279874] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f997004db10 con 0x7f995c072fe0 2023-10-26T01:30:54.451 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.391+0000 7f869fccf640 1 -- 172.21.15.169:0/220458226 shutdown_connections 2023-10-26T01:30:54.451 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.391+0000 7f869fccf640 1 -- 172.21.15.169:0/220458226 >> 172.21.15.169:0/220458226 conn(0x7f86980b2800 msgr2=0x7f86980aa250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:54.451 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.392+0000 7f869fccf640 1 -- 172.21.15.169:0/220458226 shutdown_connections 2023-10-26T01:30:54.451 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.392+0000 7f55b4664640 1 -- 172.21.15.169:0/2944803351 >> [v2:172.21.15.169:6800/3215791846,v1:172.21.15.169:6801/3215791846] conn(0x7f5590072fe0 msgr2=0x7f5590075400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:54.452 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.392+0000 7f869fccf640 1 -- 172.21.15.169:0/220458226 wait complete. 2023-10-26T01:30:54.452 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.392+0000 7f55b4664640 1 --2- 172.21.15.169:0/2944803351 >> [v2:172.21.15.169:6800/3215791846,v1:172.21.15.169:6801/3215791846] conn(0x7f5590072fe0 0x7f5590075400 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:54.453 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.392+0000 7f55b4664640 1 -- 172.21.15.169:0/2944803351 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f559006ae40 msgr2=0x7f559006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:54.453 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.392+0000 7f55b4664640 1 --2- 172.21.15.169:0/2944803351 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f559006ae40 0x7f559006d300 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f559c036000 tx=0x7f559c004730 comp rx=0 tx=0).stop 2023-10-26T01:30:54.453 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.392+0000 7f55b4664640 1 -- 172.21.15.169:0/2944803351 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55ac0d20c0 msgr2=0x7f55ac1f84f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:54.453 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.392+0000 7f55b4664640 1 --2- 172.21.15.169:0/2944803351 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55ac0d20c0 0x7f55ac1f84f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f55a800d770 tx=0x7f55a800dc30 comp rx=0 tx=0).stop 2023-10-26T01:30:54.454 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.392+0000 7f55b2c01640 1 -- 172.21.15.169:0/2944803351 reap_dead start 2023-10-26T01:30:54.454 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.392+0000 7f9957fff640 1 -- 172.21.15.169:0/1193889390 <== osd.4 v2:172.21.15.170:6816/1082279874 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f997004db10 con 0x7f995c072fe0 2023-10-26T01:30:54.455 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.393+0000 7f55b4664640 1 -- 172.21.15.169:0/2944803351 shutdown_connections 2023-10-26T01:30:54.455 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.393+0000 7f55b4664640 1 -- 172.21.15.169:0/2944803351 >> 172.21.15.169:0/2944803351 conn(0x7f55ac06b5c0 msgr2=0x7f55ac0c8750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:54.455 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.393+0000 7f55b4664640 1 -- 172.21.15.169:0/2944803351 shutdown_connections 2023-10-26T01:30:54.455 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.393+0000 7f55b4664640 1 -- 172.21.15.169:0/2944803351 wait complete. 2023-10-26T01:30:54.456 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.394+0000 7fad07807640 1 -- 172.21.15.169:0/3306384886 --> [v2:172.21.15.170:6800/2520460697,v1:172.21.15.170:6801/2520460697] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fad00156c20 con 0x7face4073040 2023-10-26T01:30:54.456 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.395+0000 7f9975de7640 1 -- 172.21.15.169:0/1193889390 >> [v2:172.21.15.170:6816/1082279874,v1:172.21.15.170:6817/1082279874] conn(0x7f995c072fe0 msgr2=0x7f995c075400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:54.457 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.395+0000 7f9975de7640 1 --2- 172.21.15.169:0/1193889390 >> [v2:172.21.15.170:6816/1082279874,v1:172.21.15.170:6817/1082279874] conn(0x7f995c072fe0 0x7f995c075400 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:54.457 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.395+0000 7f9975de7640 1 -- 172.21.15.169:0/1193889390 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f995c06ae40 msgr2=0x7f995c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:54.457 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.395+0000 7f9975de7640 1 --2- 172.21.15.169:0/1193889390 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f995c06ae40 0x7f995c06d300 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f99700c4bc0 tx=0x7f9964005ae0 comp rx=0 tx=0).stop 2023-10-26T01:30:54.457 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.395+0000 7f9975de7640 1 -- 172.21.15.169:0/1193889390 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99700cf530 msgr2=0x7f997006ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:54.458 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.395+0000 7f9975de7640 1 --2- 172.21.15.169:0/1193889390 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99700cf530 0x7f997006ef50 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f996000d770 tx=0x7f996000dc30 comp rx=0 tx=0).stop 2023-10-26T01:30:54.458 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.395+0000 7f996ffff640 1 -- 172.21.15.169:0/1193889390 reap_dead start 2023-10-26T01:30:54.458 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.395+0000 7facf5ffb640 1 -- 172.21.15.169:0/3306384886 <== osd.6 v2:172.21.15.170:6800/2520460697 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fad00156c20 con 0x7face4073040 2023-10-26T01:30:54.459 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.395+0000 7f9975de7640 1 -- 172.21.15.169:0/1193889390 shutdown_connections 2023-10-26T01:30:54.459 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.395+0000 7f9975de7640 1 -- 172.21.15.169:0/1193889390 >> 172.21.15.169:0/1193889390 conn(0x7f9970065670 msgr2=0x7f99700b0ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:54.460 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.396+0000 7fad07807640 1 -- 172.21.15.169:0/3306384886 >> [v2:172.21.15.170:6800/2520460697,v1:172.21.15.170:6801/2520460697] conn(0x7face4073040 msgr2=0x7face4075460 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:54.460 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.396+0000 7f9975de7640 1 -- 172.21.15.169:0/1193889390 shutdown_connections 2023-10-26T01:30:54.460 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.396+0000 7fad07807640 1 --2- 172.21.15.169:0/3306384886 >> [v2:172.21.15.170:6800/2520460697,v1:172.21.15.170:6801/2520460697] conn(0x7face4073040 0x7face4075460 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:54.461 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.396+0000 7f9975de7640 1 -- 172.21.15.169:0/1193889390 wait complete. 2023-10-26T01:30:54.462 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.396+0000 7fad07807640 1 -- 172.21.15.169:0/3306384886 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7face406aea0 msgr2=0x7face406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:54.462 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.396+0000 7fad07807640 1 --2- 172.21.15.169:0/3306384886 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7face406aea0 0x7face406d360 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7facfc009900 tx=0x7facfc02e040 comp rx=0 tx=0).stop 2023-10-26T01:30:54.462 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.396+0000 7fad07807640 1 -- 172.21.15.169:0/3306384886 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad001593e0 msgr2=0x7fad00154de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:54.462 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.396+0000 7fad07807640 1 --2- 172.21.15.169:0/3306384886 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad001593e0 0x7fad00154de0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7facf000c390 tx=0x7facf000c850 comp rx=0 tx=0).stop 2023-10-26T01:30:54.462 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.396+0000 7fad05da4640 1 -- 172.21.15.169:0/3306384886 reap_dead start 2023-10-26T01:30:54.463 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.396+0000 7fad07807640 1 -- 172.21.15.169:0/3306384886 shutdown_connections 2023-10-26T01:30:54.463 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.396+0000 7fad07807640 1 -- 172.21.15.169:0/3306384886 >> 172.21.15.169:0/3306384886 conn(0x7fad00069930 msgr2=0x7fad00152b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:54.463 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.397+0000 7fad07807640 1 -- 172.21.15.169:0/3306384886 shutdown_connections 2023-10-26T01:30:54.463 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.397+0000 7fad07807640 1 -- 172.21.15.169:0/3306384886 wait complete. 2023-10-26T01:30:54.464 INFO:teuthology.orchestra.run.smithi169.stdout:55834574854 2023-10-26T01:30:54.464 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-26T01:30:54.465 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.405+0000 7f692a6d7640 1 -- 172.21.15.169:0/3018602487 --> [v2:172.21.15.170:6821/1115930619,v1:172.21.15.170:6822/1115930619] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f69240c5d30 con 0x7f6910073060 2023-10-26T01:30:54.465 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.406+0000 7f6920ff9640 1 -- 172.21.15.169:0/3018602487 <== osd.7 v2:172.21.15.170:6821/1115930619 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f69240c5d30 con 0x7f6910073060 2023-10-26T01:30:54.466 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.406+0000 7f692a6d7640 1 -- 172.21.15.169:0/3018602487 >> [v2:172.21.15.170:6821/1115930619,v1:172.21.15.170:6822/1115930619] conn(0x7f6910073060 msgr2=0x7f6910075480 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:54.466 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.407+0000 7f692a6d7640 1 --2- 172.21.15.169:0/3018602487 >> [v2:172.21.15.170:6821/1115930619,v1:172.21.15.170:6822/1115930619] conn(0x7f6910073060 0x7f6910075480 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:54.467 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.407+0000 7f692a6d7640 1 -- 172.21.15.169:0/3018602487 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f691006ae40 msgr2=0x7f691006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:54.467 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.407+0000 7f692a6d7640 1 --2- 172.21.15.169:0/3018602487 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f691006ae40 0x7f691006d300 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f691c00a7f0 tx=0x7f691c00d040 comp rx=0 tx=0).stop 2023-10-26T01:30:54.467 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.407+0000 7f692a6d7640 1 -- 172.21.15.169:0/3018602487 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f69240d20c0 msgr2=0x7f69241e61a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:54.467 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.407+0000 7f692a6d7640 1 --2- 172.21.15.169:0/3018602487 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f69240d20c0 0x7f69241e61a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6914000c00 tx=0x7f69140038a0 comp rx=0 tx=0).stop 2023-10-26T01:30:54.467 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.407+0000 7f6928c74640 1 -- 172.21.15.169:0/3018602487 reap_dead start 2023-10-26T01:30:54.468 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.408+0000 7f692a6d7640 1 -- 172.21.15.169:0/3018602487 shutdown_connections 2023-10-26T01:30:54.468 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.408+0000 7f692a6d7640 1 -- 172.21.15.169:0/3018602487 >> 172.21.15.169:0/3018602487 conn(0x7f692406b5c0 msgr2=0x7f69240c9b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:54.468 INFO:teuthology.orchestra.run.smithi169.stdout:60129542151 2023-10-26T01:30:54.469 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.408+0000 7f692a6d7640 1 -- 172.21.15.169:0/3018602487 shutdown_connections 2023-10-26T01:30:54.470 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-26T01:30:54.470 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.408+0000 7f692a6d7640 1 -- 172.21.15.169:0/3018602487 wait complete. 2023-10-26T01:30:54.471 INFO:teuthology.orchestra.run.smithi169.stdout:55834574854 2023-10-26T01:30:54.471 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-10-26T01:30:54.472 INFO:teuthology.orchestra.run.smithi169.stdout:55834574854 2023-10-26T01:30:54.473 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-10-26T01:30:54.474 INFO:teuthology.orchestra.run.smithi169.stdout:55834574854 2023-10-26T01:30:54.474 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-10-26T01:30:54.475 INFO:teuthology.orchestra.run.smithi169.stdout:55834574854 2023-10-26T01:30:54.475 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-26T01:30:54.476 INFO:teuthology.orchestra.run.smithi169.stdout:55834574854 2023-10-26T01:30:54.477 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-10-26T01:30:54.802 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.799+0000 7f2d3a7fc640 1 -- 172.21.15.169:0/3572117708 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d30005280 con 0x7f2d34053c80 2023-10-26T01:30:54.802 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.799+0000 7f2d420bb640 1 -- 172.21.15.169:0/3572117708 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d34053c80 msgr2=0x7f2d340540d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:54.802 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.799+0000 7f2d420bb640 1 --2- 172.21.15.169:0/3572117708 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d34053c80 0x7f2d340540d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2d340388e0 tx=0x7f2d3002be10 comp rx=0 tx=0).stop 2023-10-26T01:30:54.803 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.800+0000 7f2d420bb640 1 -- 172.21.15.169:0/3572117708 shutdown_connections 2023-10-26T01:30:54.804 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.800+0000 7f2d420bb640 1 --2- 172.21.15.169:0/3572117708 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d34053c80 0x7f2d340540d0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:54.804 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.800+0000 7f2d420bb640 1 --2- 172.21.15.169:0/3572117708 >> v2:172.21.15.170:3300/0 conn(0x7f2d340e3130 0x7f2d340e7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:54.804 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.800+0000 7f2d420bb640 1 -- 172.21.15.169:0/3572117708 >> 172.21.15.169:0/3572117708 conn(0x7f2d34058eb0 msgr2=0x7f2d34059290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:54.804 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.801+0000 7f2d420bb640 1 -- 172.21.15.169:0/3572117708 shutdown_connections 2023-10-26T01:30:54.805 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.801+0000 7f2d420bb640 1 -- 172.21.15.169:0/3572117708 wait complete. 2023-10-26T01:30:54.810 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.804+0000 7f2d420bb640 1 Processor -- start 2023-10-26T01:30:54.811 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.804+0000 7f2d420bb640 1 -- start start 2023-10-26T01:30:54.811 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.806+0000 7f2d420bb640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d3403c070 0x7f2d3404d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:54.811 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.806+0000 7f2d420bb640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2d34053c80 0x7f2d340e61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:54.811 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.807+0000 7f2d420bb640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2d340e3130 0x7f2d340e49f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:54.811 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.807+0000 7f2d420bb640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d3404dd60 con 0x7f2d3403c070 2023-10-26T01:30:54.812 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.807+0000 7f2d420bb640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d340e6730 con 0x7f2d340e3130 2023-10-26T01:30:54.813 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.807+0000 7f2d420bb640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d3404da80 con 0x7f2d34053c80 2023-10-26T01:30:54.813 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.807+0000 7f2d3affd640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2d34053c80 0x7f2d340e61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:54.813 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.807+0000 7f2d3b7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d3403c070 0x7f2d3404d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:54.813 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.807+0000 7f2d3affd640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2d34053c80 0x7f2d340e61f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3301/0 says I am v2:172.21.15.169:53798/0 (socket says 172.21.15.169:53798) 2023-10-26T01:30:54.813 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.807+0000 7f2d3affd640 1 -- 172.21.15.169:0/2396313440 learned_addr learned my addr 172.21.15.169:0/2396313440 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:30:54.814 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.808+0000 7f2d3bfff640 1 --2- 172.21.15.169:0/2396313440 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2d340e3130 0x7f2d340e49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:54.814 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.808+0000 7f2d3b7fe640 1 -- 172.21.15.169:0/2396313440 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2d34053c80 msgr2=0x7f2d340e61f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:54.814 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.808+0000 7f2d3b7fe640 1 --2- 172.21.15.169:0/2396313440 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2d34053c80 0x7f2d340e61f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:54.814 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.808+0000 7f2d3b7fe640 1 -- 172.21.15.169:0/2396313440 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2d340e3130 msgr2=0x7f2d340e49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:54.814 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.808+0000 7f2d3b7fe640 1 --2- 172.21.15.169:0/2396313440 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2d340e3130 0x7f2d340e49f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:54.815 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.808+0000 7f2d3b7fe640 1 -- 172.21.15.169:0/2396313440 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d30009560 con 0x7f2d3403c070 2023-10-26T01:30:54.815 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.809+0000 7f2d3b7fe640 1 --2- 172.21.15.169:0/2396313440 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d3403c070 0x7f2d3404d3c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c068c30 tx=0x7f2d3c0669e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:54.815 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.809+0000 7f2d27fff640 1 -- 172.21.15.169:0/2396313440 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d3c069720 con 0x7f2d3403c070 2023-10-26T01:30:54.815 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.809+0000 7f2d27fff640 1 -- 172.21.15.169:0/2396313440 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2d3c069970 con 0x7f2d3403c070 2023-10-26T01:30:54.815 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.809+0000 7f2d27fff640 1 -- 172.21.15.169:0/2396313440 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d3c069720 con 0x7f2d3403c070 2023-10-26T01:30:54.816 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.810+0000 7f2d420bb640 1 -- 172.21.15.169:0/2396313440 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d3404e2b0 con 0x7f2d3403c070 2023-10-26T01:30:54.816 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.811+0000 7f2d420bb640 1 -- 172.21.15.169:0/2396313440 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d340e4f30 con 0x7f2d3403c070 2023-10-26T01:30:54.818 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.813+0000 7f2d27fff640 1 -- 172.21.15.169:0/2396313440 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f2d3c069c20 con 0x7f2d3403c070 2023-10-26T01:30:54.818 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.814+0000 7f2d420bb640 1 -- 172.21.15.169:0/2396313440 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d3404e110 con 0x7f2d3403c070 2023-10-26T01:30:54.818 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.815+0000 7f2d27fff640 1 --2- 172.21.15.169:0/2396313440 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2d2006aea0 0x7f2d2006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:54.819 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.816+0000 7f2d3affd640 1 --2- 172.21.15.169:0/2396313440 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2d2006aea0 0x7f2d2006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:54.820 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.818+0000 7f2d27fff640 1 -- 172.21.15.169:0/2396313440 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2d3c08e020 con 0x7f2d3403c070 2023-10-26T01:30:54.823 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.821+0000 7f2d3affd640 1 --2- 172.21.15.169:0/2396313440 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2d2006aea0 0x7f2d2006d360 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2d340e0e30 tx=0x7f2d28007ef0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:54.825 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.818+0000 7f539d294640 1 -- 172.21.15.169:0/3564446210 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f539815b9a0 msgr2=0x7f539815f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:54.825 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.818+0000 7f539d294640 1 --2- 172.21.15.169:0/3564446210 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f539815b9a0 0x7f539815f470 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5388009860 tx=0x7f538802bd70 comp rx=0 tx=0).stop 2023-10-26T01:30:54.825 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.818+0000 7f539d294640 1 -- 172.21.15.169:0/3564446210 shutdown_connections 2023-10-26T01:30:54.825 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.818+0000 7f539d294640 1 --2- 172.21.15.169:0/3564446210 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f53980b3820 0x7f53980d0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:54.826 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.818+0000 7f539d294640 1 --2- 172.21.15.169:0/3564446210 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f539815b9a0 0x7f539815f470 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:54.826 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.818+0000 7f539d294640 1 -- 172.21.15.169:0/3564446210 >> 172.21.15.169:0/3564446210 conn(0x7f53980658b0 msgr2=0x7f5398065cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:54.826 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.818+0000 7f539d294640 1 -- 172.21.15.169:0/3564446210 shutdown_connections 2023-10-26T01:30:54.826 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.818+0000 7f539d294640 1 -- 172.21.15.169:0/3564446210 wait complete. 2023-10-26T01:30:54.826 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.820+0000 7f539d294640 1 Processor -- start 2023-10-26T01:30:54.827 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.821+0000 7f539d294640 1 -- start start 2023-10-26T01:30:54.827 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.822+0000 7f539d294640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53980b3820 0x7f53981fe010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:54.828 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.822+0000 7f539d294640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f53980b7330 0x7f53980b23e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:54.828 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.822+0000 7f539d294640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f539815b9a0 0x7f53980b2920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:54.828 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.822+0000 7f539d294640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53980c8db0 con 0x7f53980b3820 2023-10-26T01:30:54.828 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.822+0000 7f539d294640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53981fe550 con 0x7f53980b7330 2023-10-26T01:30:54.829 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.822+0000 7f539d294640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53981e90b0 con 0x7f539815b9a0 2023-10-26T01:30:54.829 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.822+0000 7f5396ffd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53980b3820 0x7f53981fe010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:54.830 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.823+0000 7f5396ffd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53980b3820 0x7f53981fe010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:45932/0 (socket says 172.21.15.169:45932) 2023-10-26T01:30:54.830 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.823+0000 7f5396ffd640 1 -- 172.21.15.169:0/1904068956 learned_addr learned my addr 172.21.15.169:0/1904068956 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:30:54.830 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.823+0000 7f53977fe640 1 --2- 172.21.15.169:0/1904068956 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f539815b9a0 0x7f53980b2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:54.830 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.824+0000 7f53967fc640 1 --2- 172.21.15.169:0/1904068956 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f53980b7330 0x7f53980b23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:54.830 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.824+0000 7f53977fe640 1 -- 172.21.15.169:0/1904068956 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f53980b7330 msgr2=0x7f53980b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:54.831 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.824+0000 7f53977fe640 1 --2- 172.21.15.169:0/1904068956 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f53980b7330 0x7f53980b23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:54.831 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.824+0000 7f53977fe640 1 -- 172.21.15.169:0/1904068956 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53980b3820 msgr2=0x7f53981fe010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:54.831 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.824+0000 7f53977fe640 1 --2- 172.21.15.169:0/1904068956 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53980b3820 0x7f53981fe010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:54.831 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.824+0000 7f53977fe640 1 -- 172.21.15.169:0/1904068956 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5388009490 con 0x7f539815b9a0 2023-10-26T01:30:54.832 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.825+0000 7f2d27fff640 1 -- 172.21.15.169:0/2396313440 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2d3404e110 con 0x7f2d3403c070 2023-10-26T01:30:54.832 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.825+0000 7f53977fe640 1 --2- 172.21.15.169:0/1904068956 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f539815b9a0 0x7f53980b2920 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f53980cec90 tx=0x7f538000edd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:54.832 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.826+0000 7f538f7fe640 1 -- 172.21.15.169:0/1904068956 <== mon.2 v2:172.21.15.170:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5380009620 con 0x7f539815b9a0 2023-10-26T01:30:54.833 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.826+0000 7f538f7fe640 1 -- 172.21.15.169:0/1904068956 <== mon.2 v2:172.21.15.170:3301/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f538000cdb0 con 0x7f539815b9a0 2023-10-26T01:30:54.833 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.826+0000 7f538f7fe640 1 -- 172.21.15.169:0/1904068956 <== mon.2 v2:172.21.15.170:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5380004280 con 0x7f539815b9a0 2023-10-26T01:30:54.834 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.826+0000 7fbb7ec40640 1 -- 172.21.15.169:0/733224351 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb7003c0c0 msgr2=0x7fbb7000dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:54.834 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.826+0000 7fbb7ec40640 1 --2- 172.21.15.169:0/733224351 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb7003c0c0 0x7fbb7000dc30 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fbb74009830 tx=0x7fbb7402bd70 comp rx=0 tx=0).stop 2023-10-26T01:30:54.834 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.826+0000 7fbb7ec40640 1 -- 172.21.15.169:0/733224351 shutdown_connections 2023-10-26T01:30:54.834 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.826+0000 7fbb7ec40640 1 --2- 172.21.15.169:0/733224351 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb7003c0c0 0x7fbb7000dc30 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:54.835 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.826+0000 7fbb7ec40640 1 --2- 172.21.15.169:0/733224351 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbb700e3050 0x7fbb70055260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:54.835 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.826+0000 7fbb7ec40640 1 -- 172.21.15.169:0/733224351 >> 172.21.15.169:0/733224351 conn(0x7fbb70056cb0 msgr2=0x7fbb700570c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:54.835 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.827+0000 7f539d294640 1 -- 172.21.15.169:0/1904068956 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53981e9390 con 0x7f539815b9a0 2023-10-26T01:30:54.836 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.827+0000 7f539d294640 1 -- 172.21.15.169:0/1904068956 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53980c8a40 con 0x7f539815b9a0 2023-10-26T01:30:54.836 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.827+0000 7fbb7ec40640 1 -- 172.21.15.169:0/733224351 shutdown_connections 2023-10-26T01:30:54.836 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.827+0000 7fbb7ec40640 1 -- 172.21.15.169:0/733224351 wait complete. 2023-10-26T01:30:54.837 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.828+0000 7f539d294640 1 -- 172.21.15.169:0/1904068956 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53980c8f60 con 0x7f539815b9a0 2023-10-26T01:30:54.838 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.830+0000 7fbb7ec40640 1 Processor -- start 2023-10-26T01:30:54.838 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.830+0000 7fbb7ec40640 1 -- start start 2023-10-26T01:30:54.838 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.831+0000 7fbb7ec40640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb70056020 0x7fbb700e4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:54.838 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.832+0000 7fbb7ec40640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb700e3050 0x7fbb700d8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:54.838 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.832+0000 7fbb7ec40640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbb700dd350 0x7fbb700e3620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:54.839 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.832+0000 7fbb7ec40640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb700e4af0 con 0x7fbb700e3050 2023-10-26T01:30:54.839 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.832+0000 7fbb7ec40640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb700e3e70 con 0x7fbb70056020 2023-10-26T01:30:54.839 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.832+0000 7fbb7ec40640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb700e4150 con 0x7fbb700dd350 2023-10-26T01:30:54.839 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.832+0000 7fbb7d43d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb700e3050 0x7fbb700d8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:54.839 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.832+0000 7fbb7e43f640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbb700dd350 0x7fbb700e3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:54.840 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.833+0000 7fbb7e43f640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbb700dd350 0x7fbb700e3620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3301/0 says I am v2:172.21.15.169:53846/0 (socket says 172.21.15.169:53846) 2023-10-26T01:30:54.840 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.833+0000 7fbb7e43f640 1 -- 172.21.15.169:0/955559796 learned_addr learned my addr 172.21.15.169:0/955559796 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:30:54.840 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.833+0000 7fbb7d43d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb700e3050 0x7fbb700d8260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:45940/0 (socket says 172.21.15.169:45940) 2023-10-26T01:30:54.840 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.833+0000 7fbb7dc3e640 1 --2- 172.21.15.169:0/955559796 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb70056020 0x7fbb700e4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:54.841 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.833+0000 7f538f7fe640 1 -- 172.21.15.169:0/1904068956 <== mon.2 v2:172.21.15.170:3301/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f53800035f0 con 0x7f539815b9a0 2023-10-26T01:30:54.841 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.834+0000 7fbb7d43d640 1 -- 172.21.15.169:0/955559796 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbb700dd350 msgr2=0x7fbb700e3620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:54.842 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.834+0000 7fbb7d43d640 1 --2- 172.21.15.169:0/955559796 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbb700dd350 0x7fbb700e3620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:54.842 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.834+0000 7fbb7d43d640 1 -- 172.21.15.169:0/955559796 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb70056020 msgr2=0x7fbb700e4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:54.842 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.834+0000 7fbb7d43d640 1 --2- 172.21.15.169:0/955559796 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb70056020 0x7fbb700e4f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:54.842 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.834+0000 7fbb7d43d640 1 -- 172.21.15.169:0/955559796 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb74009490 con 0x7fbb700e3050 2023-10-26T01:30:54.842 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.834+0000 7fbb7e43f640 1 --2- 172.21.15.169:0/955559796 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbb700dd350 0x7fbb700e3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T01:30:54.843 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.834+0000 7fbb7d43d640 1 --2- 172.21.15.169:0/955559796 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb700e3050 0x7fbb700d8260 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fbb74033000 tx=0x7fbb74003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:54.843 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.835+0000 7f538f7fe640 1 --2- 172.21.15.169:0/1904068956 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f537806af10 0x7f537806d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:54.844 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.835+0000 7fbb6e7fc640 1 -- 172.21.15.169:0/955559796 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb74034440 con 0x7fbb700e3050 2023-10-26T01:30:54.844 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.835+0000 7fbb7ec40640 1 -- 172.21.15.169:0/955559796 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb700e4430 con 0x7fbb700e3050 2023-10-26T01:30:54.845 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.835+0000 7f538f7fe640 1 -- 172.21.15.169:0/1904068956 <== mon.2 v2:172.21.15.170:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5380014070 con 0x7f539815b9a0 2023-10-26T01:30:54.845 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.835+0000 7fbb6e7fc640 1 -- 172.21.15.169:0/955559796 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fbb740345a0 con 0x7fbb700e3050 2023-10-26T01:30:54.845 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.835+0000 7fbb6e7fc640 1 -- 172.21.15.169:0/955559796 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb740349b0 con 0x7fbb700e3050 2023-10-26T01:30:54.846 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.835+0000 7f5396ffd640 1 --2- 172.21.15.169:0/1904068956 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f537806af10 0x7f537806d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:54.846 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.837+0000 7f5396ffd640 1 --2- 172.21.15.169:0/1904068956 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f537806af10 0x7f537806d3d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5388009860 tx=0x7f5388004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:54.847 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.837+0000 7fbb7ec40640 1 -- 172.21.15.169:0/955559796 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb700e3b60 con 0x7fbb700e3050 2023-10-26T01:30:54.847 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.838+0000 7fbb7ec40640 1 -- 172.21.15.169:0/955559796 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb700073c0 con 0x7fbb700e3050 2023-10-26T01:30:54.847 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.839+0000 7fbb6e7fc640 1 -- 172.21.15.169:0/955559796 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fbb7403b040 con 0x7fbb700e3050 2023-10-26T01:30:54.847 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.841+0000 7fbb6e7fc640 1 --2- 172.21.15.169:0/955559796 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbb5c06ae70 0x7fbb5c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:54.847 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.841+0000 7fbb6e7fc640 1 -- 172.21.15.169:0/955559796 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbb740aba90 con 0x7fbb700e3050 2023-10-26T01:30:54.848 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.841+0000 7fc7a8456640 1 -- 172.21.15.169:0/3720257215 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc7a01593e0 msgr2=0x7fc7a00ac200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:54.849 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.841+0000 7fc7a8456640 1 --2- 172.21.15.169:0/3720257215 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc7a01593e0 0x7fc7a00ac200 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc7a00b1ea0 tx=0x7fc79c02bdc0 comp rx=0 tx=0).stop 2023-10-26T01:30:54.849 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.841+0000 7fc7a8456640 1 -- 172.21.15.169:0/3720257215 shutdown_connections 2023-10-26T01:30:54.849 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.841+0000 7fc7a8456640 1 --2- 172.21.15.169:0/3720257215 >> v2:172.21.15.169:3300/0 conn(0x7fc7a00b0c80 0x7fc7a00cd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:54.849 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.841+0000 7fc7a8456640 1 --2- 172.21.15.169:0/3720257215 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc7a01597c0 0x7fc7a015dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:54.849 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.841+0000 7fc7a8456640 1 --2- 172.21.15.169:0/3720257215 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc7a01593e0 0x7fc7a00ac200 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:54.850 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.841+0000 7fc7a8456640 1 -- 172.21.15.169:0/3720257215 >> 172.21.15.169:0/3720257215 conn(0x7fc7a0069930 msgr2=0x7fc7a0069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:54.850 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.842+0000 7fc7a8456640 1 -- 172.21.15.169:0/3720257215 shutdown_connections 2023-10-26T01:30:54.850 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.842+0000 7fc7a8456640 1 -- 172.21.15.169:0/3720257215 wait complete. 2023-10-26T01:30:54.851 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.842+0000 7fbb7dc3e640 1 --2- 172.21.15.169:0/955559796 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbb5c06ae70 0x7fbb5c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:54.851 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.843+0000 7fbb7dc3e640 1 --2- 172.21.15.169:0/955559796 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbb5c06ae70 0x7fbb5c06d330 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fbb78063540 tx=0x7fbb7806c040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:54.851 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.843+0000 7fc7a8456640 1 Processor -- start 2023-10-26T01:30:54.852 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.844+0000 7fc7a8456640 1 -- start start 2023-10-26T01:30:54.852 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.845+0000 7fc7a8456640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7a00b0c80 0x7fc7a006ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:54.852 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.845+0000 7fc7a8456640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc7a01593e0 0x7fc7a014e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:54.852 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.845+0000 7fc7a8456640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc7a01597c0 0x7fc7a006b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:54.853 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.845+0000 7fc7a8456640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a014ebb0 con 0x7fc7a00b0c80 2023-10-26T01:30:54.853 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.845+0000 7fc7a8456640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a00c7ab0 con 0x7fc7a01597c0 2023-10-26T01:30:54.853 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.845+0000 7fc7a8456640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a00c7d90 con 0x7fc7a01593e0 2023-10-26T01:30:54.853 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.846+0000 7fbb6e7fc640 1 -- 172.21.15.169:0/955559796 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbb74076be0 con 0x7fbb700e3050 2023-10-26T01:30:54.855 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.846+0000 7fc7a61f2640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7a00b0c80 0x7fc7a006ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:54.855 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.846+0000 7fc7a61f2640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7a00b0c80 0x7fc7a006ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:45958/0 (socket says 172.21.15.169:45958) 2023-10-26T01:30:54.855 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.846+0000 7fc7a61f2640 1 -- 172.21.15.169:0/546303034 learned_addr learned my addr 172.21.15.169:0/546303034 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:30:54.856 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.846+0000 7fc7a59f1640 1 --2- 172.21.15.169:0/546303034 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc7a01593e0 0x7fc7a014e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:54.856 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.846+0000 7f083446b640 1 -- 172.21.15.169:0/2634894665 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f082c15b820 msgr2=0x7f082c0cc780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:54.856 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.846+0000 7f083446b640 1 --2- 172.21.15.169:0/2634894665 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f082c15b820 0x7f082c0cc780 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f082c0cbd20 tx=0x7f082802bd60 comp rx=0 tx=0).stop 2023-10-26T01:30:54.857 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.846+0000 7f083446b640 1 -- 172.21.15.169:0/2634894665 shutdown_connections 2023-10-26T01:30:54.857 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.846+0000 7f083446b640 1 --2- 172.21.15.169:0/2634894665 >> v2:172.21.15.170:3300/0 conn(0x7f082c0cf530 0x7f082c0cea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:54.857 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.846+0000 7f083446b640 1 --2- 172.21.15.169:0/2634894665 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f082c15bc00 0x7f082c0ce540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:54.857 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.846+0000 7f083446b640 1 --2- 172.21.15.169:0/2634894665 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f082c15b820 0x7f082c0cc780 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:54.857 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.846+0000 7f083446b640 1 -- 172.21.15.169:0/2634894665 >> 172.21.15.169:0/2634894665 conn(0x7f082c065670 msgr2=0x7f082c065a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:54.858 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.846+0000 7fc7a69f3640 1 --2- 172.21.15.169:0/546303034 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc7a01597c0 0x7fc7a006b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:54.858 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.846+0000 7fc7a61f2640 1 -- 172.21.15.169:0/546303034 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc7a01593e0 msgr2=0x7fc7a014e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:54.858 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.846+0000 7fc7a61f2640 1 --2- 172.21.15.169:0/546303034 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc7a01593e0 0x7fc7a014e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:54.859 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.846+0000 7fc7a61f2640 1 -- 172.21.15.169:0/546303034 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc7a01597c0 msgr2=0x7fc7a006b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:54.859 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.846+0000 7fc7a61f2640 1 --2- 172.21.15.169:0/546303034 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc7a01597c0 0x7fc7a006b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:54.859 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.846+0000 7fc7a61f2640 1 -- 172.21.15.169:0/546303034 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc79c009560 con 0x7fc7a00b0c80 2023-10-26T01:30:54.860 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.847+0000 7f083446b640 1 -- 172.21.15.169:0/2634894665 shutdown_connections 2023-10-26T01:30:54.860 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.847+0000 7f083446b640 1 -- 172.21.15.169:0/2634894665 wait complete. 2023-10-26T01:30:54.861 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.847+0000 7fc7a61f2640 1 --2- 172.21.15.169:0/546303034 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7a00b0c80 0x7fc7a006ca00 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc79c036040 tx=0x7fc79c02da70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:54.861 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.848+0000 7fc796ffd640 1 -- 172.21.15.169:0/546303034 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc79c02de70 con 0x7fc7a00b0c80 2023-10-26T01:30:54.861 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.848+0000 7fc796ffd640 1 -- 172.21.15.169:0/546303034 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc79c0036e0 con 0x7fc7a00b0c80 2023-10-26T01:30:54.861 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.848+0000 7fc796ffd640 1 -- 172.21.15.169:0/546303034 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc79c003a50 con 0x7fc7a00b0c80 2023-10-26T01:30:54.861 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.848+0000 7fc7a8456640 1 -- 172.21.15.169:0/546303034 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7a00c8070 con 0x7fc7a00b0c80 2023-10-26T01:30:54.862 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.848+0000 7fc7a8456640 1 -- 172.21.15.169:0/546303034 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7a00c8630 con 0x7fc7a00b0c80 2023-10-26T01:30:54.862 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.849+0000 7f083446b640 1 Processor -- start 2023-10-26T01:30:54.862 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.850+0000 7f083446b640 1 -- start start 2023-10-26T01:30:54.863 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.850+0000 7f538f7fe640 1 -- 172.21.15.169:0/1904068956 <== mon.2 v2:172.21.15.170:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5380054cb0 con 0x7f539815b9a0 2023-10-26T01:30:54.863 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.850+0000 7fc794ff9640 1 -- 172.21.15.169:0/546303034 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc77c005250 con 0x7fc7a00b0c80 2023-10-26T01:30:54.863 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.851+0000 7fa5a95b6640 1 -- 172.21.15.169:0/1881044816 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa5a40b7330 msgr2=0x7fa5a40b3110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:54.864 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.851+0000 7fa5a95b6640 1 --2- 172.21.15.169:0/1881044816 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa5a40b7330 0x7fa5a40b3110 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa58c009830 tx=0x7fa58c02bd50 comp rx=0 tx=0).stop 2023-10-26T01:30:54.864 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.851+0000 7fa5a95b6640 1 -- 172.21.15.169:0/1881044816 shutdown_connections 2023-10-26T01:30:54.864 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.851+0000 7fa5a95b6640 1 --2- 172.21.15.169:0/1881044816 >> v2:172.21.15.170:3300/0 conn(0x7fa5a40b3650 0x7fa5a40d0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:54.864 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.851+0000 7fa5a95b6640 1 --2- 172.21.15.169:0/1881044816 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa5a40b7330 0x7fa5a40b3110 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:54.864 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.851+0000 7fa5a95b6640 1 -- 172.21.15.169:0/1881044816 >> 172.21.15.169:0/1881044816 conn(0x7fa5a40658b0 msgr2=0x7fa5a4065cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:54.865 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.851+0000 7f083446b640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f082c0cf530 0x7f082c15d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:54.865 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.851+0000 7f083446b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f082c15b820 0x7f082c1f5d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:54.866 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.851+0000 7f083446b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f082c15bc00 0x7f082c15df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:54.866 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.851+0000 7f083446b640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f082c15d350 con 0x7f082c15bc00 2023-10-26T01:30:54.866 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.851+0000 7f083446b640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f082c1f62b0 con 0x7f082c15b820 2023-10-26T01:30:54.866 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.851+0000 7f083446b640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f082c15ca40 con 0x7f082c0cf530 2023-10-26T01:30:54.867 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.851+0000 7fc796ffd640 1 -- 172.21.15.169:0/546303034 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fc79c003bb0 con 0x7fc7a00b0c80 2023-10-26T01:30:54.867 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.852+0000 7fa5a95b6640 1 -- 172.21.15.169:0/1881044816 shutdown_connections 2023-10-26T01:30:54.867 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.852+0000 7f0832207640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f082c0cf530 0x7f082c15d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:54.868 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.852+0000 7f0832207640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f082c0cf530 0x7f082c15d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3301/0 says I am v2:172.21.15.169:53894/0 (socket says 172.21.15.169:53894) 2023-10-26T01:30:54.868 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.852+0000 7f0832207640 1 -- 172.21.15.169:0/2164796419 learned_addr learned my addr 172.21.15.169:0/2164796419 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:30:54.868 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.852+0000 7f0831a06640 1 --2- 172.21.15.169:0/2164796419 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f082c15b820 0x7f082c1f5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:54.869 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.852+0000 7f1a9ea75640 1 -- 172.21.15.169:0/1411456742 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1a9815ba00 msgr2=0x7f1a980b6f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:54.869 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.853+0000 7f1a9ea75640 1 --2- 172.21.15.169:0/1411456742 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1a9815ba00 0x7f1a980b6f00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1a94009860 tx=0x7f1a9402bd70 comp rx=0 tx=0).stop 2023-10-26T01:30:54.869 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.853+0000 7f0832207640 1 -- 172.21.15.169:0/2164796419 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f082c15b820 msgr2=0x7f082c1f5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:54.870 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.853+0000 7f1a9ea75640 1 -- 172.21.15.169:0/1411456742 shutdown_connections 2023-10-26T01:30:54.870 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.853+0000 7f1a9ea75640 1 --2- 172.21.15.169:0/1411456742 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1a9815ba00 0x7f1a980b6f00 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:54.870 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.853+0000 7f1a9ea75640 1 -- 172.21.15.169:0/1411456742 >> 172.21.15.169:0/1411456742 conn(0x7f1a9806b5c0 msgr2=0x7f1a9806b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:54.871 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.853+0000 7fc796ffd640 1 --2- 172.21.15.169:0/546303034 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc78406ae40 0x7fc78406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:54.871 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.853+0000 7fc796ffd640 1 -- 172.21.15.169:0/546303034 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc79c0ac190 con 0x7fc7a00b0c80 2023-10-26T01:30:54.871 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.854+0000 7fc7a59f1640 1 --2- 172.21.15.169:0/546303034 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc78406ae40 0x7fc78406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:54.872 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.854+0000 7f5c73832640 1 -- 172.21.15.169:0/1651099628 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5c58005180 msgr2=0x7f5c5800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:54.872 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.854+0000 7f5c73832640 1 --2- 172.21.15.169:0/1651099628 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5c58005180 0x7f5c5800b980 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5c64005fd0 tx=0x7f5c580057f0 comp rx=0 tx=0).stop 2023-10-26T01:30:54.872 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.855+0000 7f5c73832640 1 -- 172.21.15.169:0/1651099628 shutdown_connections 2023-10-26T01:30:54.873 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.855+0000 7f5c73832640 1 --2- 172.21.15.169:0/1651099628 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5c58005180 0x7f5c5800b980 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:54.873 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.855+0000 7f5c73832640 1 -- 172.21.15.169:0/1651099628 >> 172.21.15.169:0/1651099628 conn(0x7f5c6c0658b0 msgr2=0x7f5c6c065cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:54.873 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.855+0000 7fc7a59f1640 1 --2- 172.21.15.169:0/546303034 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc78406ae40 0x7fc78406d300 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc7a015a440 tx=0x7fc790005f60 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:54.874 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.855+0000 7f5c73832640 1 -- 172.21.15.169:0/1651099628 shutdown_connections 2023-10-26T01:30:54.874 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.855+0000 7f5c73832640 1 -- 172.21.15.169:0/1651099628 wait complete. 2023-10-26T01:30:54.874 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.855+0000 7f0832207640 1 --2- 172.21.15.169:0/2164796419 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f082c15b820 0x7f082c1f5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:54.875 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.855+0000 7f0832a08640 1 --2- 172.21.15.169:0/2164796419 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f082c15bc00 0x7f082c15df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:54.875 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.855+0000 7f0832207640 1 -- 172.21.15.169:0/2164796419 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f082c15bc00 msgr2=0x7f082c15df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:54.875 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.852+0000 7fa5a95b6640 1 -- 172.21.15.169:0/1881044816 wait complete. 2023-10-26T01:30:54.876 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.855+0000 7f0832207640 1 --2- 172.21.15.169:0/2164796419 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f082c15bc00 0x7f082c15df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:54.876 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.855+0000 7fa5a95b6640 1 Processor -- start 2023-10-26T01:30:54.876 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.855+0000 7f0832207640 1 -- 172.21.15.169:0/2164796419 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0828009490 con 0x7f082c0cf530 2023-10-26T01:30:54.877 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.855+0000 7f0831a06640 1 --2- 172.21.15.169:0/2164796419 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f082c15b820 0x7f082c1f5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:30:54.877 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.855+0000 7f0832a08640 1 --2- 172.21.15.169:0/2164796419 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f082c15bc00 0x7f082c15df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:30:54.877 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.856+0000 7fa5a95b6640 1 -- start start 2023-10-26T01:30:54.878 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.856+0000 7f0832207640 1 --2- 172.21.15.169:0/2164796419 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f082c0cf530 0x7f082c15d760 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0828000c00 tx=0x7f08280038a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:54.878 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.857+0000 7f1a9ea75640 1 -- 172.21.15.169:0/1411456742 shutdown_connections 2023-10-26T01:30:54.878 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.857+0000 7f1a9ea75640 1 -- 172.21.15.169:0/1411456742 wait complete. 2023-10-26T01:30:54.879 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.857+0000 7f5c73832640 1 Processor -- start 2023-10-26T01:30:54.879 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.857+0000 7f5c73832640 1 -- start start 2023-10-26T01:30:54.879 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.858+0000 7fa5a95b6640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa5a40b3650 0x7fa5a406dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:54.880 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.858+0000 7fa5a95b6640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa5a40b7330 0x7fa5a415eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:54.880 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.858+0000 7f5c73832640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5c58005180 0x7f5c6c1f1b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:54.881 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.858+0000 7fa5a95b6640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa5a415b9a0 0x7fa5a415bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:54.881 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.858+0000 7f5c73832640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5c6c14d470 0x7f5c6c0b5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:54.881 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.858+0000 7f5c73832640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5c6c0b5d90 0x7f5c6c15bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:54.882 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.858+0000 7f5c73832640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c6c0b6210 con 0x7f5c6c14d470 2023-10-26T01:30:54.882 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.858+0000 7f5c73832640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c6c1f2090 con 0x7f5c58005180 2023-10-26T01:30:54.882 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.858+0000 7f5c73832640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c6c1550d0 con 0x7f5c6c0b5d90 2023-10-26T01:30:54.882 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.859+0000 7f5c715ce640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5c58005180 0x7f5c6c1f1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:54.883 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.859+0000 7f5c715ce640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5c58005180 0x7f5c6c1f1b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.169:57584/0 (socket says 172.21.15.169:57584) 2023-10-26T01:30:54.883 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.859+0000 7f5c70dcd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5c6c14d470 0x7f5c6c0b5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:54.883 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.859+0000 7f5c715ce640 1 -- 172.21.15.169:0/1793119394 learned_addr learned my addr 172.21.15.169:0/1793119394 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:30:54.883 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.859+0000 7f5c71dcf640 1 --2- 172.21.15.169:0/1793119394 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5c6c0b5d90 0x7f5c6c15bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:54.884 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.859+0000 7fa5a95b6640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a415d620 con 0x7fa5a40b7330 2023-10-26T01:30:54.884 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.859+0000 7fa5a2ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa5a40b3650 0x7fa5a406dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:54.884 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.859+0000 7f5c70dcd640 1 -- 172.21.15.169:0/1793119394 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5c6c0b5d90 msgr2=0x7f5c6c15bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:54.884 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.859+0000 7f5c70dcd640 1 --2- 172.21.15.169:0/1793119394 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5c6c0b5d90 0x7f5c6c15bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:54.885 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.860+0000 7f5c70dcd640 1 -- 172.21.15.169:0/1793119394 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5c58005180 msgr2=0x7f5c6c1f1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:54.885 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.860+0000 7f5c70dcd640 1 --2- 172.21.15.169:0/1793119394 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5c58005180 0x7f5c6c1f1b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:54.885 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.860+0000 7f5c70dcd640 1 -- 172.21.15.169:0/1793119394 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c6c1553b0 con 0x7f5c6c14d470 2023-10-26T01:30:54.886 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.860+0000 7fa5a95b6640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a415ca10 con 0x7fa5a40b3650 2023-10-26T01:30:54.886 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.860+0000 7fa5a2ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa5a40b3650 0x7fa5a406dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.169:57576/0 (socket says 172.21.15.169:57576) 2023-10-26T01:30:54.886 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.860+0000 7fa5a2ffd640 1 -- 172.21.15.169:0/1457072999 learned_addr learned my addr 172.21.15.169:0/1457072999 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:30:54.886 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.860+0000 7fa5a95b6640 1 -- 172.21.15.169:0/1457072999 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a415ccf0 con 0x7fa5a415b9a0 2023-10-26T01:30:54.887 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.860+0000 7f5c70dcd640 1 --2- 172.21.15.169:0/1793119394 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5c6c14d470 0x7f5c6c0b5850 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c068500 tx=0x7f5c5c00b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:54.887 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.860+0000 7fa5a37fe640 1 --2- 172.21.15.169:0/1457072999 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa5a415b9a0 0x7fa5a415bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:54.887 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.860+0000 7fa59bfff640 1 --2- 172.21.15.169:0/1457072999 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa5a40b7330 0x7fa5a415eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:54.888 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.861+0000 7f0822ffd640 1 -- 172.21.15.169:0/2164796419 <== mon.2 v2:172.21.15.170:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0828003ea0 con 0x7f082c0cf530 2023-10-26T01:30:54.888 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.861+0000 7f0822ffd640 1 -- 172.21.15.169:0/2164796419 <== mon.2 v2:172.21.15.170:3301/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0828034550 con 0x7f082c0cf530 2023-10-26T01:30:54.888 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.861+0000 7f0822ffd640 1 -- 172.21.15.169:0/2164796419 <== mon.2 v2:172.21.15.170:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0828034960 con 0x7f082c0cf530 2023-10-26T01:30:54.889 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.861+0000 7fa5a37fe640 1 -- 172.21.15.169:0/1457072999 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa5a40b3650 msgr2=0x7fa5a406dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:54.889 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.861+0000 7fa5a37fe640 1 --2- 172.21.15.169:0/1457072999 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa5a40b3650 0x7fa5a406dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:54.890 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.861+0000 7fa5a37fe640 1 -- 172.21.15.169:0/1457072999 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa5a40b7330 msgr2=0x7fa5a415eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:54.890 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.861+0000 7fa5a37fe640 1 --2- 172.21.15.169:0/1457072999 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa5a40b7330 0x7fa5a415eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:54.890 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.861+0000 7fa59bfff640 1 --2- 172.21.15.169:0/1457072999 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa5a40b7330 0x7fa5a415eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T01:30:54.890 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.862+0000 7fa5a37fe640 1 -- 172.21.15.169:0/1457072999 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa58c009490 con 0x7fa5a415b9a0 2023-10-26T01:30:54.890 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.862+0000 7fa5a2ffd640 1 --2- 172.21.15.169:0/1457072999 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa5a40b3650 0x7fa5a406dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T01:30:54.891 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.862+0000 7f083446b640 1 -- 172.21.15.169:0/2164796419 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f082c0cb4b0 con 0x7f082c0cf530 2023-10-26T01:30:54.891 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.862+0000 7f083446b640 1 -- 172.21.15.169:0/2164796419 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f082c15cf70 con 0x7f082c0cf530 2023-10-26T01:30:54.892 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.862+0000 7fa5a37fe640 1 --2- 172.21.15.169:0/1457072999 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa5a415b9a0 0x7fa5a415bf70 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa59c009fd0 tx=0x7fa59c00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:54.892 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.863+0000 7f083446b640 1 -- 172.21.15.169:0/2164796419 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f082c04db10 con 0x7f082c0cf530 2023-10-26T01:30:54.892 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.863+0000 7fa59b7fe640 1 -- 172.21.15.169:0/1457072999 <== mon.2 v2:172.21.15.170:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa59c00ce20 con 0x7fa5a415b9a0 2023-10-26T01:30:54.893 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.863+0000 7fa59b7fe640 1 -- 172.21.15.169:0/1457072999 <== mon.2 v2:172.21.15.170:3301/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa59c004c10 con 0x7fa5a415b9a0 2023-10-26T01:30:54.893 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.863+0000 7fa59b7fe640 1 -- 172.21.15.169:0/1457072999 <== mon.2 v2:172.21.15.170:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa59c005020 con 0x7fa5a415b9a0 2023-10-26T01:30:54.893 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.863+0000 7fa5a95b6640 1 -- 172.21.15.169:0/1457072999 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5a415cfd0 con 0x7fa5a415b9a0 2023-10-26T01:30:54.893 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.864+0000 7fa5a95b6640 1 -- 172.21.15.169:0/1457072999 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5a415acd0 con 0x7fa5a415b9a0 2023-10-26T01:30:54.894 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.864+0000 7f1a9ea75640 1 Processor -- start 2023-10-26T01:30:54.894 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.864+0000 7f1a9ea75640 1 -- start start 2023-10-26T01:30:54.894 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.865+0000 7fa5a95b6640 1 -- 172.21.15.169:0/1457072999 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5a415d790 con 0x7fa5a415b9a0 2023-10-26T01:30:54.895 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.866+0000 7f0822ffd640 1 -- 172.21.15.169:0/2164796419 <== mon.2 v2:172.21.15.170:3301/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f082803b040 con 0x7f082c0cf530 2023-10-26T01:30:54.895 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.867+0000 7f1a9ea75640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1a980d0fc0 0x7f1a98151610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:54.896 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.867+0000 7f1a9ea75640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a980d2290 0x7f1a98151b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:54.896 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.867+0000 7f1a9ea75640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1a9815ba00 0x7f1a981f29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:54.896 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.867+0000 7f1a9ea75640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a980c7ce0 con 0x7f1a980d2290 2023-10-26T01:30:54.896 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.867+0000 7f1a9ea75640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a98153910 con 0x7f1a980d0fc0 2023-10-26T01:30:54.896 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.867+0000 7f1a9ea75640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a98153bf0 con 0x7f1a9815ba00 2023-10-26T01:30:54.897 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.867+0000 7f1a8ffff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a980d2290 0x7f1a98151b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:54.897 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.867+0000 7f1a8ffff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a980d2290 0x7f1a98151b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:46000/0 (socket says 172.21.15.169:46000) 2023-10-26T01:30:54.897 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.867+0000 7f1a8ffff640 1 -- 172.21.15.169:0/1556469041 learned_addr learned my addr 172.21.15.169:0/1556469041 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:30:54.897 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.868+0000 7f5c61ffb640 1 -- 172.21.15.169:0/1793119394 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c5c006480 con 0x7f5c6c14d470 2023-10-26T01:30:54.898 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.868+0000 7f5c61ffb640 1 -- 172.21.15.169:0/1793119394 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5c5c003830 con 0x7f5c6c14d470 2023-10-26T01:30:54.898 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.868+0000 7f5c61ffb640 1 -- 172.21.15.169:0/1793119394 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c5c003c40 con 0x7f5c6c14d470 2023-10-26T01:30:54.898 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.868+0000 7f5c73832640 1 -- 172.21.15.169:0/1793119394 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c6c155690 con 0x7f5c6c14d470 2023-10-26T01:30:54.898 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.868+0000 7f0822ffd640 1 --2- 172.21.15.169:0/2164796419 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f081006af10 0x7f081006d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:54.899 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.868+0000 7fc796ffd640 1 -- 172.21.15.169:0/546303034 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc79c077530 con 0x7fc7a00b0c80 2023-10-26T01:30:54.899 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.868+0000 7f0822ffd640 1 -- 172.21.15.169:0/2164796419 <== mon.2 v2:172.21.15.170:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f08280ac020 con 0x7f082c0cf530 2023-10-26T01:30:54.900 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.868+0000 7f5c73832640 1 -- 172.21.15.169:0/1793119394 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c6c155c50 con 0x7f5c6c14d470 2023-10-26T01:30:54.900 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.868+0000 7f1a9d012640 1 --2- 172.21.15.169:0/1556469041 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1a9815ba00 0x7f1a981f29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:54.900 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.868+0000 7f1a9c811640 1 --2- 172.21.15.169:0/1556469041 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1a980d0fc0 0x7f1a98151610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:54.900 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.869+0000 7f1a8ffff640 1 -- 172.21.15.169:0/1556469041 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1a9815ba00 msgr2=0x7f1a981f29e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:54.901 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.869+0000 7f1a8ffff640 1 --2- 172.21.15.169:0/1556469041 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1a9815ba00 0x7f1a981f29e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:54.901 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.869+0000 7f1a8ffff640 1 -- 172.21.15.169:0/1556469041 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1a980d0fc0 msgr2=0x7f1a98151610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:54.901 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.869+0000 7f1a8ffff640 1 --2- 172.21.15.169:0/1556469041 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1a980d0fc0 0x7f1a98151610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:54.901 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.869+0000 7f1a8ffff640 1 -- 172.21.15.169:0/1556469041 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a94009490 con 0x7f1a980d2290 2023-10-26T01:30:54.901 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.869+0000 7f1a8ffff640 1 --2- 172.21.15.169:0/1556469041 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a980d2290 0x7f1a98151b50 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1a8800c500 tx=0x7f1a8800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:54.902 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.870+0000 7f0831a06640 1 --2- 172.21.15.169:0/2164796419 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f081006af10 0x7f081006d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:54.902 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.870+0000 7f5c73832640 1 -- 172.21.15.169:0/1793119394 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c6c15d770 con 0x7f5c6c14d470 2023-10-26T01:30:54.903 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.871+0000 7f1a8d7fa640 1 -- 172.21.15.169:0/1556469041 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a88012830 con 0x7f1a980d2290 2023-10-26T01:30:54.903 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.871+0000 7f1a9ea75640 1 -- 172.21.15.169:0/1556469041 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a98153ed0 con 0x7f1a980d2290 2023-10-26T01:30:54.903 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.871+0000 7f0831a06640 1 --2- 172.21.15.169:0/2164796419 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f081006af10 0x7f081006d3d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f082c1e8290 tx=0x7f081c005a40 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:54.904 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.871+0000 7f1a9ea75640 1 -- 172.21.15.169:0/1556469041 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a98154490 con 0x7f1a980d2290 2023-10-26T01:30:54.904 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.872+0000 7f1a9ea75640 1 -- 172.21.15.169:0/1556469041 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a981546b0 con 0x7f1a980d2290 2023-10-26T01:30:54.905 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.873+0000 7f5c61ffb640 1 -- 172.21.15.169:0/1793119394 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f5c5c022480 con 0x7f5c6c14d470 2023-10-26T01:30:54.905 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.875+0000 7f5c61ffb640 1 --2- 172.21.15.169:0/1793119394 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5c5006aec0 0x7f5c5006d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:54.905 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.876+0000 7f5c61ffb640 1 -- 172.21.15.169:0/1793119394 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5c5c089a30 con 0x7f5c6c14d470 2023-10-26T01:30:54.905 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.876+0000 7f1a8d7fa640 1 -- 172.21.15.169:0/1556469041 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1a88012990 con 0x7f1a980d2290 2023-10-26T01:30:54.906 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.876+0000 7f1a8d7fa640 1 -- 172.21.15.169:0/1556469041 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a88012d60 con 0x7f1a980d2290 2023-10-26T01:30:54.906 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.876+0000 7f5c715ce640 1 --2- 172.21.15.169:0/1793119394 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5c5006aec0 0x7f5c5006d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:54.906 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.877+0000 7f5c61ffb640 1 -- 172.21.15.169:0/1793119394 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5c5c054d00 con 0x7f5c6c14d470 2023-10-26T01:30:54.907 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.878+0000 7f5c715ce640 1 --2- 172.21.15.169:0/1793119394 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5c5006aec0 0x7f5c5006d380 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5c68001de0 tx=0x7f5c680014e0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:54.907 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.882+0000 7f0822ffd640 1 -- 172.21.15.169:0/2164796419 <== mon.2 v2:172.21.15.170:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0828077260 con 0x7f082c0cf530 2023-10-26T01:30:54.908 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.880+0000 7f1a8d7fa640 1 -- 172.21.15.169:0/1556469041 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f1a8801b400 con 0x7f1a980d2290 2023-10-26T01:30:54.908 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.882+0000 7f1a8d7fa640 1 --2- 172.21.15.169:0/1556469041 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f1a8406ae40 0x7f1a8406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:54.908 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.882+0000 7f1a8d7fa640 1 -- 172.21.15.169:0/1556469041 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1a88089f60 con 0x7f1a980d2290 2023-10-26T01:30:54.908 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.883+0000 7f1a9c811640 1 --2- 172.21.15.169:0/1556469041 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f1a8406ae40 0x7f1a8406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:54.909 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.885+0000 7f1a8d7fa640 1 -- 172.21.15.169:0/1556469041 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1a88058a90 con 0x7f1a980d2290 2023-10-26T01:30:54.909 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.885+0000 7fa59b7fe640 1 -- 172.21.15.169:0/1457072999 <== mon.2 v2:172.21.15.170:3301/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fa59c005180 con 0x7fa5a415b9a0 2023-10-26T01:30:54.909 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.886+0000 7fa59b7fe640 1 --2- 172.21.15.169:0/1457072999 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa58406ae40 0x7fa58406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:54.910 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.887+0000 7fa59b7fe640 1 -- 172.21.15.169:0/1457072999 <== mon.2 v2:172.21.15.170:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa59c014070 con 0x7fa5a415b9a0 2023-10-26T01:30:54.910 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.888+0000 7fa5a2ffd640 1 --2- 172.21.15.169:0/1457072999 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa58406ae40 0x7fa58406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:54.910 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.888+0000 7f1a9c811640 1 --2- 172.21.15.169:0/1556469041 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f1a8406ae40 0x7f1a8406d300 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1a94036000 tx=0x7f1a94004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:54.911 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.889+0000 7fa5a2ffd640 1 --2- 172.21.15.169:0/1457072999 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa58406ae40 0x7fa58406d300 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa5940011e0 tx=0x7fa594007df0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:54.911 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:54.896+0000 7fa59b7fe640 1 -- 172.21.15.169:0/1457072999 <== mon.2 v2:172.21.15.170:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa59c055a20 con 0x7fa5a415b9a0 2023-10-26T01:30:55.221 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.218+0000 7f2d420bb640 1 -- 172.21.15.169:0/2396313440 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f2d3404e110 con 0x7f2d3403c070 2023-10-26T01:30:55.222 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.219+0000 7f2d27fff640 1 -- 172.21.15.169:0/2396313440 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2d3404e110 con 0x7f2d3403c070 2023-10-26T01:30:55.223 INFO:teuthology.orchestra.run.smithi169.stdout:55834574853 2023-10-26T01:30:55.225 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.223+0000 7f2d420bb640 1 -- 172.21.15.169:0/2396313440 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2d2006aea0 msgr2=0x7f2d2006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:55.226 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.223+0000 7f2d420bb640 1 --2- 172.21.15.169:0/2396313440 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2d2006aea0 0x7f2d2006d360 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2d340e0e30 tx=0x7f2d28007ef0 comp rx=0 tx=0).stop 2023-10-26T01:30:55.226 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.223+0000 7f2d420bb640 1 -- 172.21.15.169:0/2396313440 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d3403c070 msgr2=0x7f2d3404d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:55.227 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.223+0000 7f2d420bb640 1 --2- 172.21.15.169:0/2396313440 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d3403c070 0x7f2d3404d3c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c068c30 tx=0x7f2d3c0669e0 comp rx=0 tx=0).stop 2023-10-26T01:30:55.227 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.224+0000 7f2d420bb640 1 -- 172.21.15.169:0/2396313440 shutdown_connections 2023-10-26T01:30:55.228 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.224+0000 7f2d420bb640 1 --2- 172.21.15.169:0/2396313440 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2d2006aea0 0x7f2d2006d360 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:55.228 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.224+0000 7f2d420bb640 1 --2- 172.21.15.169:0/2396313440 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2d340e3130 0x7f2d340e49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:55.228 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.224+0000 7f2d420bb640 1 --2- 172.21.15.169:0/2396313440 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2d34053c80 0x7f2d340e61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:55.228 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.224+0000 7f2d420bb640 1 --2- 172.21.15.169:0/2396313440 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d3403c070 0x7f2d3404d3c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:55.228 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.224+0000 7f2d420bb640 1 -- 172.21.15.169:0/2396313440 >> 172.21.15.169:0/2396313440 conn(0x7f2d34058eb0 msgr2=0x7f2d340e6c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:55.229 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.225+0000 7f2d420bb640 1 -- 172.21.15.169:0/2396313440 shutdown_connections 2023-10-26T01:30:55.229 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.225+0000 7f2d420bb640 1 -- 172.21.15.169:0/2396313440 wait complete. 2023-10-26T01:30:55.253 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.251+0000 7f539d294640 1 -- 172.21.15.169:0/1904068956 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f53980c8f20 con 0x7f539815b9a0 2023-10-26T01:30:55.255 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.0 2023-10-26T01:30:55.255 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.252+0000 7f538f7fe640 1 -- 172.21.15.169:0/1904068956 <== mon.2 v2:172.21.15.170:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f538001c090 con 0x7f539815b9a0 2023-10-26T01:30:55.258 INFO:teuthology.orchestra.run.smithi169.stdout:55834574853 2023-10-26T01:30:55.259 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.256+0000 7f539d294640 1 -- 172.21.15.169:0/1904068956 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f537806af10 msgr2=0x7f537806d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:55.260 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.256+0000 7f539d294640 1 --2- 172.21.15.169:0/1904068956 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f537806af10 0x7f537806d3d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5388009860 tx=0x7f5388004730 comp rx=0 tx=0).stop 2023-10-26T01:30:55.260 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.256+0000 7f539d294640 1 -- 172.21.15.169:0/1904068956 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f539815b9a0 msgr2=0x7f53980b2920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:55.260 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.256+0000 7f539d294640 1 --2- 172.21.15.169:0/1904068956 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f539815b9a0 0x7f53980b2920 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f53980cec90 tx=0x7f538000edd0 comp rx=0 tx=0).stop 2023-10-26T01:30:55.260 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.256+0000 7f539d294640 1 -- 172.21.15.169:0/1904068956 shutdown_connections 2023-10-26T01:30:55.260 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.256+0000 7f539d294640 1 --2- 172.21.15.169:0/1904068956 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f537806af10 0x7f537806d3d0 secure :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5388009860 tx=0x7f5388004730 comp rx=0 tx=0).stop 2023-10-26T01:30:55.261 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.256+0000 7f539d294640 1 --2- 172.21.15.169:0/1904068956 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f539815b9a0 0x7f53980b2920 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:55.261 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.256+0000 7f539d294640 1 --2- 172.21.15.169:0/1904068956 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f53980b7330 0x7f53980b23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:55.261 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.256+0000 7f539d294640 1 --2- 172.21.15.169:0/1904068956 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53980b3820 0x7f53981fe010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:55.261 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.256+0000 7f539d294640 1 -- 172.21.15.169:0/1904068956 >> 172.21.15.169:0/1904068956 conn(0x7f53980658b0 msgr2=0x7f53980b1300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:55.261 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.256+0000 7f539d294640 1 -- 172.21.15.169:0/1904068956 shutdown_connections 2023-10-26T01:30:55.262 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.257+0000 7f539d294640 1 -- 172.21.15.169:0/1904068956 wait complete. 2023-10-26T01:30:55.262 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.257+0000 7fbb7ec40640 1 -- 172.21.15.169:0/955559796 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fbb700e4c30 con 0x7fbb700e3050 2023-10-26T01:30:55.263 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.258+0000 7fbb6e7fc640 1 -- 172.21.15.169:0/955559796 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fbb700e4c30 con 0x7fbb700e3050 2023-10-26T01:30:55.263 INFO:teuthology.orchestra.run.smithi169.stdout:60129542150 2023-10-26T01:30:55.264 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.262+0000 7fbb7ec40640 1 -- 172.21.15.169:0/955559796 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbb5c06ae70 msgr2=0x7fbb5c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:55.265 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.262+0000 7fbb7ec40640 1 --2- 172.21.15.169:0/955559796 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbb5c06ae70 0x7fbb5c06d330 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fbb78063540 tx=0x7fbb7806c040 comp rx=0 tx=0).stop 2023-10-26T01:30:55.265 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.262+0000 7fbb7ec40640 1 -- 172.21.15.169:0/955559796 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb700e3050 msgr2=0x7fbb700d8260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:55.265 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.262+0000 7fbb7ec40640 1 --2- 172.21.15.169:0/955559796 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb700e3050 0x7fbb700d8260 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fbb74033000 tx=0x7fbb74003c10 comp rx=0 tx=0).stop 2023-10-26T01:30:55.266 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.263+0000 7fbb7ec40640 1 -- 172.21.15.169:0/955559796 shutdown_connections 2023-10-26T01:30:55.266 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.263+0000 7fbb7ec40640 1 --2- 172.21.15.169:0/955559796 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbb5c06ae70 0x7fbb5c06d330 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:55.266 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.263+0000 7fbb7ec40640 1 --2- 172.21.15.169:0/955559796 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb700e3050 0x7fbb700d8260 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:55.266 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.263+0000 7fbb7ec40640 1 --2- 172.21.15.169:0/955559796 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbb700dd350 0x7fbb700e3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:55.267 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.263+0000 7fbb7ec40640 1 --2- 172.21.15.169:0/955559796 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb70056020 0x7fbb700e4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:55.267 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.263+0000 7fbb7ec40640 1 -- 172.21.15.169:0/955559796 >> 172.21.15.169:0/955559796 conn(0x7fbb70056cb0 msgr2=0x7fbb7004c790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:55.267 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.263+0000 7fbb7ec40640 1 -- 172.21.15.169:0/955559796 shutdown_connections 2023-10-26T01:30:55.267 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.263+0000 7fbb7ec40640 1 -- 172.21.15.169:0/955559796 wait complete. 2023-10-26T01:30:55.280 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.277+0000 7f5c73832640 1 -- 172.21.15.169:0/1793119394 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f5c6c04db10 con 0x7f5c6c14d470 2023-10-26T01:30:55.281 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.278+0000 7f5c61ffb640 1 -- 172.21.15.169:0/1793119394 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5c5c01d020 con 0x7f5c6c14d470 2023-10-26T01:30:55.282 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.280+0000 7f083446b640 1 -- 172.21.15.169:0/2164796419 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f082c0b40d0 con 0x7f082c0cf530 2023-10-26T01:30:55.282 INFO:teuthology.orchestra.run.smithi169.stdout:55834574853 2023-10-26T01:30:55.283 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.280+0000 7f0822ffd640 1 -- 172.21.15.169:0/2164796419 <== mon.2 v2:172.21.15.170:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0828039020 con 0x7f082c0cf530 2023-10-26T01:30:55.283 INFO:teuthology.orchestra.run.smithi169.stdout:55834574853 2023-10-26T01:30:55.286 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.1 2023-10-26T01:30:55.286 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.283+0000 7f083446b640 1 -- 172.21.15.169:0/2164796419 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f081006af10 msgr2=0x7f081006d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:55.286 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.283+0000 7f083446b640 1 --2- 172.21.15.169:0/2164796419 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f081006af10 0x7f081006d3d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f082c1e8290 tx=0x7f081c005a40 comp rx=0 tx=0).stop 2023-10-26T01:30:55.286 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.283+0000 7f083446b640 1 -- 172.21.15.169:0/2164796419 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f082c0cf530 msgr2=0x7f082c15d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:55.287 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.283+0000 7f083446b640 1 --2- 172.21.15.169:0/2164796419 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f082c0cf530 0x7f082c15d760 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0828000c00 tx=0x7f08280038a0 comp rx=0 tx=0).stop 2023-10-26T01:30:55.287 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.283+0000 7f083446b640 1 -- 172.21.15.169:0/2164796419 shutdown_connections 2023-10-26T01:30:55.287 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.283+0000 7f083446b640 1 --2- 172.21.15.169:0/2164796419 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f081006af10 0x7f081006d3d0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:55.287 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.283+0000 7f083446b640 1 --2- 172.21.15.169:0/2164796419 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f082c15bc00 0x7f082c15df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:55.287 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.283+0000 7f083446b640 1 --2- 172.21.15.169:0/2164796419 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f082c15b820 0x7f082c1f5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:55.288 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.283+0000 7f083446b640 1 --2- 172.21.15.169:0/2164796419 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f082c0cf530 0x7f082c15d760 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:55.288 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.283+0000 7f083446b640 1 -- 172.21.15.169:0/2164796419 >> 172.21.15.169:0/2164796419 conn(0x7f082c065670 msgr2=0x7f082c0b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:55.290 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.283+0000 7f5c73832640 1 -- 172.21.15.169:0/1793119394 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5c5006aec0 msgr2=0x7f5c5006d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:55.291 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.283+0000 7f5c73832640 1 --2- 172.21.15.169:0/1793119394 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5c5006aec0 0x7f5c5006d380 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5c68001de0 tx=0x7f5c680014e0 comp rx=0 tx=0).stop 2023-10-26T01:30:55.291 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.283+0000 7f5c73832640 1 -- 172.21.15.169:0/1793119394 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5c6c14d470 msgr2=0x7f5c6c0b5850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:55.291 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.283+0000 7f5c73832640 1 --2- 172.21.15.169:0/1793119394 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5c6c14d470 0x7f5c6c0b5850 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c068500 tx=0x7f5c5c00b990 comp rx=0 tx=0).stop 2023-10-26T01:30:55.291 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.283+0000 7f5c73832640 1 -- 172.21.15.169:0/1793119394 shutdown_connections 2023-10-26T01:30:55.292 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.283+0000 7f5c73832640 1 --2- 172.21.15.169:0/1793119394 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5c5006aec0 0x7f5c5006d380 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:55.292 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.283+0000 7f5c73832640 1 --2- 172.21.15.169:0/1793119394 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5c6c14d470 0x7f5c6c0b5850 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:55.292 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.283+0000 7f5c73832640 1 --2- 172.21.15.169:0/1793119394 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5c6c0b5d90 0x7f5c6c15bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:55.292 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.283+0000 7f5c73832640 1 --2- 172.21.15.169:0/1793119394 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5c58005180 0x7f5c6c1f1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:55.292 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.283+0000 7f5c73832640 1 -- 172.21.15.169:0/1793119394 >> 172.21.15.169:0/1793119394 conn(0x7f5c6c0658b0 msgr2=0x7f5c6c0d05b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:55.293 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.284+0000 7f083446b640 1 -- 172.21.15.169:0/2164796419 shutdown_connections 2023-10-26T01:30:55.293 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.284+0000 7f083446b640 1 -- 172.21.15.169:0/2164796419 wait complete. 2023-10-26T01:30:55.294 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.284+0000 7f5c73832640 1 -- 172.21.15.169:0/1793119394 shutdown_connections 2023-10-26T01:30:55.294 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.284+0000 7f5c73832640 1 -- 172.21.15.169:0/1793119394 wait complete. 2023-10-26T01:30:55.294 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.286+0000 7fc794ff9640 1 -- 172.21.15.169:0/546303034 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fc77c0057d0 con 0x7fc7a00b0c80 2023-10-26T01:30:55.295 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.289+0000 7fc796ffd640 1 -- 172.21.15.169:0/546303034 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc79c043020 con 0x7fc7a00b0c80 2023-10-26T01:30:55.295 INFO:teuthology.orchestra.run.smithi169.stdout:55834574853 2023-10-26T01:30:55.296 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542150 for osd.3 2023-10-26T01:30:55.296 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.292+0000 7fc794ff9640 1 -- 172.21.15.169:0/546303034 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc78406ae40 msgr2=0x7fc78406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:55.297 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.292+0000 7fc794ff9640 1 --2- 172.21.15.169:0/546303034 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc78406ae40 0x7fc78406d300 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc7a015a440 tx=0x7fc790005f60 comp rx=0 tx=0).stop 2023-10-26T01:30:55.297 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.292+0000 7fc794ff9640 1 -- 172.21.15.169:0/546303034 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7a00b0c80 msgr2=0x7fc7a006ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:55.297 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.292+0000 7fc794ff9640 1 --2- 172.21.15.169:0/546303034 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7a00b0c80 0x7fc7a006ca00 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc79c036040 tx=0x7fc79c02da70 comp rx=0 tx=0).stop 2023-10-26T01:30:55.297 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.293+0000 7fa5a95b6640 1 -- 172.21.15.169:0/1457072999 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fa5a415ce30 con 0x7fa5a415b9a0 2023-10-26T01:30:55.298 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.293+0000 7fc794ff9640 1 -- 172.21.15.169:0/546303034 shutdown_connections 2023-10-26T01:30:55.298 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.293+0000 7fc794ff9640 1 --2- 172.21.15.169:0/546303034 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc78406ae40 0x7fc78406d300 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:55.299 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.293+0000 7fc794ff9640 1 --2- 172.21.15.169:0/546303034 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc7a01597c0 0x7fc7a006b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:55.299 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.293+0000 7fc794ff9640 1 --2- 172.21.15.169:0/546303034 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc7a01593e0 0x7fc7a014e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:55.299 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.293+0000 7fc794ff9640 1 --2- 172.21.15.169:0/546303034 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7a00b0c80 0x7fc7a006ca00 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:55.299 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.293+0000 7fc794ff9640 1 -- 172.21.15.169:0/546303034 >> 172.21.15.169:0/546303034 conn(0x7fc7a0069930 msgr2=0x7fc7a015b440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:55.300 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.293+0000 7fc794ff9640 1 -- 172.21.15.169:0/546303034 shutdown_connections 2023-10-26T01:30:55.300 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.293+0000 7fc794ff9640 1 -- 172.21.15.169:0/546303034 wait complete. 2023-10-26T01:30:55.300 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.293+0000 7fa59b7fe640 1 -- 172.21.15.169:0/1457072999 <== mon.2 v2:172.21.15.170:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa5a415ce30 con 0x7fa5a415b9a0 2023-10-26T01:30:55.301 INFO:teuthology.orchestra.run.smithi169.stdout:55834574853 2023-10-26T01:30:55.301 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.297+0000 7f1a9ea75640 1 -- 172.21.15.169:0/1556469041 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f1a980b7070 con 0x7f1a980d2290 2023-10-26T01:30:55.302 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.297+0000 7fa5a95b6640 1 -- 172.21.15.169:0/1457072999 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa58406ae40 msgr2=0x7fa58406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:55.302 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.297+0000 7fa5a95b6640 1 --2- 172.21.15.169:0/1457072999 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa58406ae40 0x7fa58406d300 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa5940011e0 tx=0x7fa594007df0 comp rx=0 tx=0).stop 2023-10-26T01:30:55.302 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.297+0000 7fa5a95b6640 1 -- 172.21.15.169:0/1457072999 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa5a415b9a0 msgr2=0x7fa5a415bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:55.302 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.297+0000 7fa5a95b6640 1 --2- 172.21.15.169:0/1457072999 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa5a415b9a0 0x7fa5a415bf70 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa59c009fd0 tx=0x7fa59c00ef10 comp rx=0 tx=0).stop 2023-10-26T01:30:55.303 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.297+0000 7fa5a95b6640 1 -- 172.21.15.169:0/1457072999 shutdown_connections 2023-10-26T01:30:55.303 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.297+0000 7fa5a95b6640 1 --2- 172.21.15.169:0/1457072999 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa58406ae40 0x7fa58406d300 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:55.303 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.297+0000 7fa5a95b6640 1 --2- 172.21.15.169:0/1457072999 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa5a40b7330 0x7fa5a415eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:55.303 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.297+0000 7fa5a95b6640 1 --2- 172.21.15.169:0/1457072999 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa5a415b9a0 0x7fa5a415bf70 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:55.304 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.297+0000 7fa5a95b6640 1 --2- 172.21.15.169:0/1457072999 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa5a40b3650 0x7fa5a406dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:55.304 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.297+0000 7fa5a95b6640 1 -- 172.21.15.169:0/1457072999 >> 172.21.15.169:0/1457072999 conn(0x7fa5a40658b0 msgr2=0x7fa5a40b5b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:55.304 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.298+0000 7f1a8d7fa640 1 -- 172.21.15.169:0/1556469041 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1a88058620 con 0x7f1a980d2290 2023-10-26T01:30:55.305 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.298+0000 7fa5a95b6640 1 -- 172.21.15.169:0/1457072999 shutdown_connections 2023-10-26T01:30:55.305 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.298+0000 7fa5a95b6640 1 -- 172.21.15.169:0/1457072999 wait complete. 2023-10-26T01:30:55.305 INFO:teuthology.orchestra.run.smithi169.stdout:55834574853 2023-10-26T01:30:55.306 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.300+0000 7f1a9ea75640 1 -- 172.21.15.169:0/1556469041 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f1a8406ae40 msgr2=0x7f1a8406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:55.306 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.300+0000 7f1a9ea75640 1 --2- 172.21.15.169:0/1556469041 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f1a8406ae40 0x7f1a8406d300 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1a94036000 tx=0x7f1a94004730 comp rx=0 tx=0).stop 2023-10-26T01:30:55.306 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.300+0000 7f1a9ea75640 1 -- 172.21.15.169:0/1556469041 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a980d2290 msgr2=0x7f1a98151b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:55.307 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.300+0000 7f1a9ea75640 1 --2- 172.21.15.169:0/1556469041 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a980d2290 0x7f1a98151b50 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1a8800c500 tx=0x7f1a8800c9c0 comp rx=0 tx=0).stop 2023-10-26T01:30:55.307 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.300+0000 7f1a9ea75640 1 -- 172.21.15.169:0/1556469041 shutdown_connections 2023-10-26T01:30:55.307 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.300+0000 7f1a9ea75640 1 --2- 172.21.15.169:0/1556469041 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f1a8406ae40 0x7f1a8406d300 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:55.307 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.300+0000 7f1a9ea75640 1 --2- 172.21.15.169:0/1556469041 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a980d2290 0x7f1a98151b50 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:55.307 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.300+0000 7f1a9ea75640 1 --2- 172.21.15.169:0/1556469041 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1a9815ba00 0x7f1a981f29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:55.308 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.300+0000 7f1a9ea75640 1 --2- 172.21.15.169:0/1556469041 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1a980d0fc0 0x7f1a98151610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:55.308 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.300+0000 7f1a9ea75640 1 -- 172.21.15.169:0/1556469041 >> 172.21.15.169:0/1556469041 conn(0x7f1a9806b5c0 msgr2=0x7f1a980c7fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:55.308 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.301+0000 7f1a9ea75640 1 -- 172.21.15.169:0/1556469041 shutdown_connections 2023-10-26T01:30:55.308 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:55.301+0000 7f1a9ea75640 1 -- 172.21.15.169:0/1556469041 wait complete. 2023-10-26T01:30:55.309 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.6 2023-10-26T01:30:55.311 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.7 2023-10-26T01:30:55.320 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.2 2023-10-26T01:30:55.322 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.4 2023-10-26T01:30:55.324 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.5 2023-10-26T01:30:56.255 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-26T01:30:56.287 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-26T01:30:56.297 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-26T01:30:56.310 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-10-26T01:30:56.312 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-10-26T01:30:56.321 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-26T01:30:56.323 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-10-26T01:30:56.325 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-10-26T01:30:56.543 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.540+0000 7f6df10ea640 1 -- 172.21.15.169:0/579496009 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6de4053b30 msgr2=0x7f6de404fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:56.544 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.540+0000 7f6df10ea640 1 --2- 172.21.15.169:0/579496009 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6de4053b30 0x7f6de404fcf0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6de4007a30 tx=0x7f6dd802bd10 comp rx=0 tx=0).stop 2023-10-26T01:30:56.544 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.541+0000 7f6df10ea640 1 -- 172.21.15.169:0/579496009 shutdown_connections 2023-10-26T01:30:56.545 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.541+0000 7f6df10ea640 1 --2- 172.21.15.169:0/579496009 >> v2:172.21.15.169:3300/0 conn(0x7f6de4050230 0x7f6de40e2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:56.545 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.541+0000 7f6df10ea640 1 --2- 172.21.15.169:0/579496009 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6de4053b30 0x7f6de404fcf0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:56.545 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.541+0000 7f6df10ea640 1 --2- 172.21.15.169:0/579496009 >> v2:172.21.15.170:3300/0 conn(0x7f6de40df380 0x7f6de404f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:56.545 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.541+0000 7f6df10ea640 1 -- 172.21.15.169:0/579496009 >> 172.21.15.169:0/579496009 conn(0x7f6de4009f80 msgr2=0x7f6de400a360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:56.545 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.542+0000 7f6df10ea640 1 -- 172.21.15.169:0/579496009 shutdown_connections 2023-10-26T01:30:56.546 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.542+0000 7f6df10ea640 1 -- 172.21.15.169:0/579496009 wait complete. 2023-10-26T01:30:56.548 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.546+0000 7f6df10ea640 1 Processor -- start 2023-10-26T01:30:56.548 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.546+0000 7f6df10ea640 1 -- start start 2023-10-26T01:30:56.551 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.548+0000 7f6df10ea640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6de4050230 0x7f6de404d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:56.551 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.548+0000 7f6df10ea640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6de4053b30 0x7f6de404d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:56.551 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.548+0000 7f6df10ea640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6de40df380 0x7f6de404a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:56.552 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.548+0000 7f6df10ea640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6de404e880 con 0x7f6de40df380 2023-10-26T01:30:56.552 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.548+0000 7f6df10ea640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6de404dd00 con 0x7f6de4053b30 2023-10-26T01:30:56.552 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.548+0000 7f6df10ea640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6de404dfe0 con 0x7f6de4050230 2023-10-26T01:30:56.553 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.549+0000 7f6debfff640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6de4050230 0x7f6de404d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:56.553 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.549+0000 7f6df08e9640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6de40df380 0x7f6de404a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:56.553 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.549+0000 7f6debfff640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6de4050230 0x7f6de404d140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3301/0 says I am v2:172.21.15.169:53936/0 (socket says 172.21.15.169:53936) 2023-10-26T01:30:56.553 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.549+0000 7f6debfff640 1 -- 172.21.15.169:0/27286718 learned_addr learned my addr 172.21.15.169:0/27286718 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:30:56.554 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.549+0000 7f6deb7fe640 1 --2- 172.21.15.169:0/27286718 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6de4053b30 0x7f6de404d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:56.554 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.551+0000 7f6df08e9640 1 -- 172.21.15.169:0/27286718 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6de4050230 msgr2=0x7f6de404d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:56.555 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.551+0000 7f6df08e9640 1 --2- 172.21.15.169:0/27286718 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6de4050230 0x7f6de404d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:56.555 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.551+0000 7f6df08e9640 1 -- 172.21.15.169:0/27286718 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6de4053b30 msgr2=0x7f6de404d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:56.555 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.551+0000 7f6df08e9640 1 --2- 172.21.15.169:0/27286718 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6de4053b30 0x7f6de404d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:56.555 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.551+0000 7f6df08e9640 1 -- 172.21.15.169:0/27286718 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dd8009560 con 0x7f6de40df380 2023-10-26T01:30:56.555 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.552+0000 7f6df08e9640 1 --2- 172.21.15.169:0/27286718 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6de40df380 0x7f6de404a570 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6dec074260 tx=0x7f6dec067dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:56.556 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.553+0000 7f6de8ff9640 1 -- 172.21.15.169:0/27286718 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dec069720 con 0x7f6de40df380 2023-10-26T01:30:56.556 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.553+0000 7f6de8ff9640 1 -- 172.21.15.169:0/27286718 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6dec069970 con 0x7f6de40df380 2023-10-26T01:30:56.557 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.553+0000 7f6de8ff9640 1 -- 172.21.15.169:0/27286718 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dec069de0 con 0x7f6de40df380 2023-10-26T01:30:56.557 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.554+0000 7f6df10ea640 1 -- 172.21.15.169:0/27286718 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6de404e2c0 con 0x7f6de40df380 2023-10-26T01:30:56.560 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.558+0000 7f6de8ff9640 1 -- 172.21.15.169:0/27286718 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f6dec066020 con 0x7f6de40df380 2023-10-26T01:30:56.560 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.558+0000 7f6df10ea640 1 -- 172.21.15.169:0/27286718 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6de40d6780 con 0x7f6de40df380 2023-10-26T01:30:56.565 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.560+0000 7f6df10ea640 1 -- 172.21.15.169:0/27286718 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6de4034580 con 0x7f6de40df380 2023-10-26T01:30:56.566 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.561+0000 7f6de8ff9640 1 --2- 172.21.15.169:0/27286718 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f6dd006ae40 0x7f6dd006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:56.566 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.563+0000 7f6debfff640 1 --2- 172.21.15.169:0/27286718 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f6dd006ae40 0x7f6dd006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:56.567 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.565+0000 7f6de8ff9640 1 -- 172.21.15.169:0/27286718 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6dec0cfab0 con 0x7f6de40df380 2023-10-26T01:30:56.570 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.568+0000 7f6debfff640 1 --2- 172.21.15.169:0/27286718 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f6dd006ae40 0x7f6dd006d300 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6de0005fd0 tx=0x7f6de000c040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:56.577 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.574+0000 7f6de8ff9640 1 -- 172.21.15.169:0/27286718 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6dec0cbdf0 con 0x7f6de40df380 2023-10-26T01:30:56.592 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.588+0000 7f3e3d19e640 1 -- 172.21.15.169:0/319520890 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3e3004f260 msgr2=0x7f3e30005e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:56.592 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.589+0000 7f3e3d19e640 1 --2- 172.21.15.169:0/319520890 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3e3004f260 0x7f3e30005e60 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3e28009830 tx=0x7f3e2802beb0 comp rx=0 tx=0).stop 2023-10-26T01:30:56.592 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.589+0000 7f3e3d19e640 1 -- 172.21.15.169:0/319520890 shutdown_connections 2023-10-26T01:30:56.593 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.589+0000 7f3e3d19e640 1 --2- 172.21.15.169:0/319520890 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3e3004f260 0x7f3e30005e60 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:56.593 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.589+0000 7f3e3d19e640 1 -- 172.21.15.169:0/319520890 >> 172.21.15.169:0/319520890 conn(0x7f3e30009f80 msgr2=0x7f3e3000a360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:56.593 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.589+0000 7f3e3d19e640 1 -- 172.21.15.169:0/319520890 shutdown_connections 2023-10-26T01:30:56.593 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.589+0000 7f3e3d19e640 1 -- 172.21.15.169:0/319520890 wait complete. 2023-10-26T01:30:56.595 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.592+0000 7f3e3d19e640 1 Processor -- start 2023-10-26T01:30:56.595 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.593+0000 7f3e3d19e640 1 -- start start 2023-10-26T01:30:56.597 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.594+0000 7f3e3d19e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3e300063a0 0x7f3e300deda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:56.597 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.594+0000 7f3e3d19e640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3e3004f260 0x7f3e3004dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:56.598 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.594+0000 7f3e3d19e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3e300e6f80 0x7f3e300dd620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:56.599 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.595+0000 7f3e36d9d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3e300063a0 0x7f3e300deda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:56.599 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.595+0000 7f3e36d9d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3e300063a0 0x7f3e300deda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:46022/0 (socket says 172.21.15.169:46022) 2023-10-26T01:30:56.599 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.595+0000 7f3e36d9d640 1 -- 172.21.15.169:0/2702537651 learned_addr learned my addr 172.21.15.169:0/2702537651 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:30:56.599 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.595+0000 7f3e3d19e640 1 -- 172.21.15.169:0/2702537651 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e300e3530 con 0x7f3e300063a0 2023-10-26T01:30:56.599 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.595+0000 7f3e3d19e640 1 -- 172.21.15.169:0/2702537651 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e3004e780 con 0x7f3e300e6f80 2023-10-26T01:30:56.600 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.595+0000 7f3e3d19e640 1 -- 172.21.15.169:0/2702537651 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e300e2ee0 con 0x7f3e3004f260 2023-10-26T01:30:56.600 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.595+0000 7f3e3659c640 1 --2- 172.21.15.169:0/2702537651 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3e3004f260 0x7f3e3004dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:56.600 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.595+0000 7f3e3759e640 1 --2- 172.21.15.169:0/2702537651 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3e300e6f80 0x7f3e300dd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:56.601 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.596+0000 7f3e36d9d640 1 -- 172.21.15.169:0/2702537651 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3e3004f260 msgr2=0x7f3e3004dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:56.601 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.596+0000 7f3e36d9d640 1 --2- 172.21.15.169:0/2702537651 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3e3004f260 0x7f3e3004dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:56.601 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.596+0000 7f3e36d9d640 1 -- 172.21.15.169:0/2702537651 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3e300e6f80 msgr2=0x7f3e300dd620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:56.601 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.596+0000 7f3e36d9d640 1 --2- 172.21.15.169:0/2702537651 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3e300e6f80 0x7f3e300dd620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:56.601 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.596+0000 7f3e36d9d640 1 -- 172.21.15.169:0/2702537651 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e28009490 con 0x7f3e300063a0 2023-10-26T01:30:56.602 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.597+0000 7f3e36d9d640 1 --2- 172.21.15.169:0/2702537651 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3e300063a0 0x7f3e300deda0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c00c3c0 tx=0x7f3e2c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:56.602 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.597+0000 7f3e1f7fe640 1 -- 172.21.15.169:0/2702537651 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e2c011790 con 0x7f3e300063a0 2023-10-26T01:30:56.603 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.598+0000 7faeb4c75640 1 -- 172.21.15.169:0/1572966041 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7faeb006bee0 msgr2=0x7faeb00cd240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:56.603 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.598+0000 7faeb4c75640 1 --2- 172.21.15.169:0/1572966041 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7faeb006bee0 0x7faeb00cd240 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7faea0005fd0 tx=0x7faea002c2d0 comp rx=0 tx=0).stop 2023-10-26T01:30:56.604 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.597+0000 7f3e1f7fe640 1 -- 172.21.15.169:0/2702537651 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3e2c0118f0 con 0x7f3e300063a0 2023-10-26T01:30:56.604 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.597+0000 7f3e1f7fe640 1 -- 172.21.15.169:0/2702537651 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e2c011d20 con 0x7f3e300063a0 2023-10-26T01:30:56.604 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.597+0000 7f3e3d19e640 1 -- 172.21.15.169:0/2702537651 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e300e31c0 con 0x7f3e300063a0 2023-10-26T01:30:56.604 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.597+0000 7f3e3d19e640 1 -- 172.21.15.169:0/2702537651 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e300df2e0 con 0x7f3e300063a0 2023-10-26T01:30:56.605 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.598+0000 7faeb4c75640 1 -- 172.21.15.169:0/1572966041 shutdown_connections 2023-10-26T01:30:56.605 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.598+0000 7faeb4c75640 1 --2- 172.21.15.169:0/1572966041 >> v2:172.21.15.170:3300/0 conn(0x7faeb00b1130 0x7faeb006b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:56.605 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.598+0000 7faeb4c75640 1 --2- 172.21.15.169:0/1572966041 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7faeb006bee0 0x7faeb00cd240 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:56.606 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.598+0000 7faeb4c75640 1 -- 172.21.15.169:0/1572966041 >> 172.21.15.169:0/1572966041 conn(0x7faeb00b2510 msgr2=0x7faeb00b2920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:56.606 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.599+0000 7faeb4c75640 1 -- 172.21.15.169:0/1572966041 shutdown_connections 2023-10-26T01:30:56.606 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.599+0000 7faeb4c75640 1 -- 172.21.15.169:0/1572966041 wait complete. 2023-10-26T01:30:56.606 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.599+0000 7f3e3d19e640 1 -- 172.21.15.169:0/2702537651 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e300089a0 con 0x7f3e300063a0 2023-10-26T01:30:56.607 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.602+0000 7faeb4c75640 1 Processor -- start 2023-10-26T01:30:56.607 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.602+0000 7f3e1f7fe640 1 -- 172.21.15.169:0/2702537651 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f3e2c01c460 con 0x7f3e300063a0 2023-10-26T01:30:56.608 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.603+0000 7faeb4c75640 1 -- start start 2023-10-26T01:30:56.608 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.605+0000 7faeb4c75640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faeb006bee0 0x7faeb014df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:56.608 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.605+0000 7fde4416c640 1 -- 172.21.15.169:0/1080471155 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fde3c0b12f0 msgr2=0x7fde3c0b1730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:56.609 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.605+0000 7faeb4c75640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7faeb00b1130 0x7faeb006adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:56.609 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.605+0000 7fde4416c640 1 --2- 172.21.15.169:0/1080471155 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fde3c0b12f0 0x7fde3c0b1730 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fde3c06f1a0 tx=0x7fde3402c0b0 comp rx=0 tx=0).stop 2023-10-26T01:30:56.610 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.605+0000 7faeb4c75640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faeb00cd780 0x7faeb0149690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:56.610 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.606+0000 7faeaffff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faeb00cd780 0x7faeb0149690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:56.610 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.606+0000 7faeaffff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faeb00cd780 0x7faeb0149690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:46040/0 (socket says 172.21.15.169:46040) 2023-10-26T01:30:56.611 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.606+0000 7f3e1f7fe640 1 --2- 172.21.15.169:0/2702537651 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3e2406ae90 0x7f3e2406d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:56.611 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.606+0000 7f3e1f7fe640 1 -- 172.21.15.169:0/2702537651 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3e2c0893b0 con 0x7f3e300063a0 2023-10-26T01:30:56.612 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.606+0000 7faeaffff640 1 -- 172.21.15.169:0/1359735825 learned_addr learned my addr 172.21.15.169:0/1359735825 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:30:56.612 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.607+0000 7fde4416c640 1 -- 172.21.15.169:0/1080471155 shutdown_connections 2023-10-26T01:30:56.612 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.607+0000 7fde4416c640 1 --2- 172.21.15.169:0/1080471155 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fde3c0b12f0 0x7fde3c0b1730 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:56.612 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.607+0000 7fde4416c640 1 --2- 172.21.15.169:0/1080471155 >> v2:172.21.15.170:3300/0 conn(0x7fde3c06c520 0x7fde3c15c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:56.613 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.607+0000 7fde4416c640 1 -- 172.21.15.169:0/1080471155 >> 172.21.15.169:0/1080471155 conn(0x7fde3c0b2800 msgr2=0x7fde3c0b2be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:56.613 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.607+0000 7faeaf7fe640 1 --2- 172.21.15.169:0/1359735825 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faeb006bee0 0x7faeb014df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:56.613 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.607+0000 7faeb4c75640 1 -- 172.21.15.169:0/1359735825 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeb00ac080 con 0x7faeb00cd780 2023-10-26T01:30:56.614 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.607+0000 7faeb4c75640 1 -- 172.21.15.169:0/1359735825 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeb0140360 con 0x7faeb006bee0 2023-10-26T01:30:56.614 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.607+0000 7faeb4c75640 1 -- 172.21.15.169:0/1359735825 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faeb00cc8a0 con 0x7faeb00b1130 2023-10-26T01:30:56.614 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.607+0000 7fde4416c640 1 -- 172.21.15.169:0/1080471155 shutdown_connections 2023-10-26T01:30:56.615 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.607+0000 7fde4416c640 1 -- 172.21.15.169:0/1080471155 wait complete. 2023-10-26T01:30:56.615 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.608+0000 7faeaffff640 1 -- 172.21.15.169:0/1359735825 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7faeb00b1130 msgr2=0x7faeb006adc0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-26T01:30:56.616 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.608+0000 7faeaffff640 1 --2- 172.21.15.169:0/1359735825 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7faeb00b1130 0x7faeb006adc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:56.616 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.608+0000 7faeaffff640 1 -- 172.21.15.169:0/1359735825 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faeb006bee0 msgr2=0x7faeb014df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:56.616 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.608+0000 7faeaffff640 1 --2- 172.21.15.169:0/1359735825 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faeb006bee0 0x7faeb014df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:56.616 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.608+0000 7faeaffff640 1 -- 172.21.15.169:0/1359735825 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faea0009900 con 0x7faeb00cd780 2023-10-26T01:30:56.616 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.609+0000 7faeaf7fe640 1 --2- 172.21.15.169:0/1359735825 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faeb006bee0 0x7faeb014df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:30:56.617 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.609+0000 7faeaffff640 1 --2- 172.21.15.169:0/1359735825 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faeb00cd780 0x7faeb0149690 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7faea800eab0 tx=0x7faea8009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:56.617 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.610+0000 7f3e3659c640 1 --2- 172.21.15.169:0/2702537651 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3e2406ae90 0x7f3e2406d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:56.617 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.611+0000 7fde4416c640 1 Processor -- start 2023-10-26T01:30:56.618 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.611+0000 7fde4416c640 1 -- start start 2023-10-26T01:30:56.618 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.612+0000 7f3e3659c640 1 --2- 172.21.15.169:0/2702537651 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3e2406ae90 0x7f3e2406d350 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3e28036000 tx=0x7f3e28004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:56.619 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.612+0000 7f3e1f7fe640 1 -- 172.21.15.169:0/2702537651 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3e2c0545d0 con 0x7f3e300063a0 2023-10-26T01:30:56.619 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.612+0000 7faeacff9640 1 -- 172.21.15.169:0/1359735825 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea800cc30 con 0x7faeb00cd780 2023-10-26T01:30:56.619 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.612+0000 7faeacff9640 1 -- 172.21.15.169:0/1359735825 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7faea800cd90 con 0x7faeb00cd780 2023-10-26T01:30:56.620 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.612+0000 7faeacff9640 1 -- 172.21.15.169:0/1359735825 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea8005280 con 0x7faeb00cd780 2023-10-26T01:30:56.620 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.613+0000 7faeb4c75640 1 -- 172.21.15.169:0/1359735825 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faeb00abda0 con 0x7faeb00cd780 2023-10-26T01:30:56.620 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.613+0000 7faeb4c75640 1 -- 172.21.15.169:0/1359735825 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faeb00cc5c0 con 0x7faeb00cd780 2023-10-26T01:30:56.621 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.613+0000 7fde4416c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fde3c06c520 0x7fde3c0ab650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:56.622 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.614+0000 7fde4416c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fde3c0b12f0 0x7fde3c157fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:56.622 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.614+0000 7fde4416c640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fde3c1592f0 0x7fde3c1434e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:56.622 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.614+0000 7fde4416c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde3c071aa0 con 0x7fde3c06c520 2023-10-26T01:30:56.622 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.614+0000 7fde4416c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde3c142460 con 0x7fde3c0b12f0 2023-10-26T01:30:56.622 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.614+0000 7fde4416c640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde3c142740 con 0x7fde3c1592f0 2023-10-26T01:30:56.623 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.615+0000 7fde41707640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fde3c0b12f0 0x7fde3c157fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:56.623 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.615+0000 7fde42709640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fde3c1592f0 0x7fde3c1434e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:56.623 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.615+0000 7fde42709640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fde3c1592f0 0x7fde3c1434e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3301/0 says I am v2:172.21.15.169:53976/0 (socket says 172.21.15.169:53976) 2023-10-26T01:30:56.623 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.615+0000 7fde42709640 1 -- 172.21.15.169:0/946201417 learned_addr learned my addr 172.21.15.169:0/946201417 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:30:56.623 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.615+0000 7fde41707640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fde3c0b12f0 0x7fde3c157fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.169:57670/0 (socket says 172.21.15.169:57670) 2023-10-26T01:30:56.624 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.615+0000 7fde41f08640 1 --2- 172.21.15.169:0/946201417 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fde3c06c520 0x7fde3c0ab650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:56.624 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.616+0000 7fde42709640 1 -- 172.21.15.169:0/946201417 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fde3c0b12f0 msgr2=0x7fde3c157fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:56.624 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.616+0000 7fde42709640 1 --2- 172.21.15.169:0/946201417 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fde3c0b12f0 0x7fde3c157fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:56.624 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.616+0000 7fde42709640 1 -- 172.21.15.169:0/946201417 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fde3c06c520 msgr2=0x7fde3c0ab650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:56.624 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.616+0000 7fde42709640 1 --2- 172.21.15.169:0/946201417 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fde3c06c520 0x7fde3c0ab650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:56.625 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.616+0000 7fde42709640 1 -- 172.21.15.169:0/946201417 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde34009900 con 0x7fde3c1592f0 2023-10-26T01:30:56.625 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.617+0000 7fde42709640 1 --2- 172.21.15.169:0/946201417 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fde3c1592f0 0x7fde3c1434e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fde34037040 tx=0x7fde34003f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:56.625 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.616+0000 7faea5ffb640 1 -- 172.21.15.169:0/1359735825 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae88005250 con 0x7faeb00cd780 2023-10-26T01:30:56.626 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.619+0000 7faeacff9640 1 -- 172.21.15.169:0/1359735825 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7faea80053e0 con 0x7faeb00cd780 2023-10-26T01:30:56.626 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.619+0000 7fde327fc640 1 -- 172.21.15.169:0/946201417 <== mon.2 v2:172.21.15.170:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde34004550 con 0x7fde3c1592f0 2023-10-26T01:30:56.627 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.619+0000 7fde327fc640 1 -- 172.21.15.169:0/946201417 <== mon.2 v2:172.21.15.170:3301/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fde340047a0 con 0x7fde3c1592f0 2023-10-26T01:30:56.627 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.619+0000 7fde327fc640 1 -- 172.21.15.169:0/946201417 <== mon.2 v2:172.21.15.170:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde34034810 con 0x7fde3c1592f0 2023-10-26T01:30:56.627 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.620+0000 7fde4416c640 1 -- 172.21.15.169:0/946201417 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde3c142ce0 con 0x7fde3c1592f0 2023-10-26T01:30:56.627 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.621+0000 7fde4416c640 1 -- 172.21.15.169:0/946201417 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde3c0abb90 con 0x7fde3c1592f0 2023-10-26T01:30:56.628 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.621+0000 7faeacff9640 1 --2- 172.21.15.169:0/1359735825 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fae9006ae40 0x7fae9006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:56.628 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.621+0000 7faeacff9640 1 -- 172.21.15.169:0/1359735825 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faea8014070 con 0x7faeb00cd780 2023-10-26T01:30:56.628 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.623+0000 7fde4416c640 1 -- 172.21.15.169:0/946201417 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde3c143200 con 0x7fde3c1592f0 2023-10-26T01:30:56.629 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.624+0000 7faeaf7fe640 1 --2- 172.21.15.169:0/1359735825 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fae9006ae40 0x7fae9006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:56.629 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.626+0000 7faeaf7fe640 1 --2- 172.21.15.169:0/1359735825 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fae9006ae40 0x7fae9006d300 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7faea0036000 tx=0x7faea0004ab0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:56.630 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.626+0000 7fde327fc640 1 -- 172.21.15.169:0/946201417 <== mon.2 v2:172.21.15.170:3301/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fde3403b040 con 0x7fde3c1592f0 2023-10-26T01:30:56.630 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.628+0000 7fcddd444640 1 -- 172.21.15.169:0/3579404233 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcdd80d0ea0 msgr2=0x7fcdd80c4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:56.631 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.628+0000 7faeacff9640 1 -- 172.21.15.169:0/1359735825 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7faea8055130 con 0x7faeb00cd780 2023-10-26T01:30:56.631 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.628+0000 7fcddd444640 1 --2- 172.21.15.169:0/3579404233 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcdd80d0ea0 0x7fcdd80c4ee0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fcdd8073610 tx=0x7fcdd002c1a0 comp rx=0 tx=0).stop 2023-10-26T01:30:56.632 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.628+0000 7fcddd444640 1 -- 172.21.15.169:0/3579404233 shutdown_connections 2023-10-26T01:30:56.632 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.628+0000 7fcddd444640 1 --2- 172.21.15.169:0/3579404233 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcdd80d0ea0 0x7fcdd80c4ee0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:56.632 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.628+0000 7fcddd444640 1 --2- 172.21.15.169:0/3579404233 >> v2:172.21.15.169:3300/0 conn(0x7fcdd815ec90 0x7fcdd8065490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:56.632 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.628+0000 7fde327fc640 1 --2- 172.21.15.169:0/946201417 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fde2006af10 0x7fde2006d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:56.633 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.628+0000 7fde327fc640 1 -- 172.21.15.169:0/946201417 <== mon.2 v2:172.21.15.170:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fde340ac0e0 con 0x7fde3c1592f0 2023-10-26T01:30:56.633 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.629+0000 7fcddd444640 1 -- 172.21.15.169:0/3579404233 >> 172.21.15.169:0/3579404233 conn(0x7fcdd80d1390 msgr2=0x7fcdd80d17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:56.633 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.629+0000 7fcddd444640 1 -- 172.21.15.169:0/3579404233 shutdown_connections 2023-10-26T01:30:56.633 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.629+0000 7fcddd444640 1 -- 172.21.15.169:0/3579404233 wait complete. 2023-10-26T01:30:56.634 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.629+0000 7fde41f08640 1 --2- 172.21.15.169:0/946201417 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fde2006af10 0x7fde2006d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:56.634 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.630+0000 7fde41f08640 1 --2- 172.21.15.169:0/946201417 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fde2006af10 0x7fde2006d3d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fde38000f60 tx=0x7fde38009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:56.635 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.632+0000 7fcddd444640 1 Processor -- start 2023-10-26T01:30:56.635 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.632+0000 7fcddd444640 1 -- start start 2023-10-26T01:30:56.636 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.633+0000 7fcddd444640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcdd80d0ea0 0x7fcdd814be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:56.636 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.633+0000 7fcddd444640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcdd80d20c0 0x7fcdd815db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:56.636 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.634+0000 7fcddd444640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcdd815ec90 0x7fcdd815e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:56.637 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.634+0000 7fcddd444640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd814deb0 con 0x7fcdd815ec90 2023-10-26T01:30:56.637 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.634+0000 7fcddd444640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd81f1990 con 0x7fcdd80d20c0 2023-10-26T01:30:56.637 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.634+0000 7fcddd444640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd81f5520 con 0x7fcdd80d0ea0 2023-10-26T01:30:56.638 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.634+0000 7fcdd6ffd640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcdd80d0ea0 0x7fcdd814be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:56.638 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.634+0000 7fcdd6ffd640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcdd80d0ea0 0x7fcdd814be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3301/0 says I am v2:172.21.15.169:53980/0 (socket says 172.21.15.169:53980) 2023-10-26T01:30:56.639 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.634+0000 7fcdd6ffd640 1 -- 172.21.15.169:0/3023981145 learned_addr learned my addr 172.21.15.169:0/3023981145 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:30:56.639 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.634+0000 7fcdd77fe640 1 --2- 172.21.15.169:0/3023981145 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcdd815ec90 0x7fcdd815e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:56.639 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.635+0000 7fcdd67fc640 1 --2- 172.21.15.169:0/3023981145 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcdd80d20c0 0x7fcdd815db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:56.639 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.635+0000 7fcdd77fe640 1 -- 172.21.15.169:0/3023981145 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcdd80d0ea0 msgr2=0x7fcdd814be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:56.640 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.635+0000 7fcdd77fe640 1 --2- 172.21.15.169:0/3023981145 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcdd80d0ea0 0x7fcdd814be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:56.640 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.635+0000 7fcdd77fe640 1 -- 172.21.15.169:0/3023981145 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcdd80d20c0 msgr2=0x7fcdd815db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:56.640 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.635+0000 7fcdd77fe640 1 --2- 172.21.15.169:0/3023981145 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcdd80d20c0 0x7fcdd815db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:56.640 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.635+0000 7fcdd77fe640 1 -- 172.21.15.169:0/3023981145 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdd0009840 con 0x7fcdd815ec90 2023-10-26T01:30:56.640 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.636+0000 7fcdd77fe640 1 --2- 172.21.15.169:0/3023981145 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcdd815ec90 0x7fcdd815e070 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fcdd80663e0 tx=0x7fcdd0004250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:56.641 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.637+0000 7fcdbf7fe640 1 -- 172.21.15.169:0/3023981145 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd0004890 con 0x7fcdd815ec90 2023-10-26T01:30:56.641 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.637+0000 7fcdbf7fe640 1 -- 172.21.15.169:0/3023981145 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fcdd0034690 con 0x7fcdd815ec90 2023-10-26T01:30:56.641 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.637+0000 7fcdbf7fe640 1 -- 172.21.15.169:0/3023981145 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd0034a60 con 0x7fcdd815ec90 2023-10-26T01:30:56.642 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.637+0000 7fcddd444640 1 -- 172.21.15.169:0/3023981145 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdd81f5800 con 0x7fcdd815ec90 2023-10-26T01:30:56.642 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.637+0000 7fcddd444640 1 -- 172.21.15.169:0/3023981145 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdd81f5dc0 con 0x7fcdd815ec90 2023-10-26T01:30:56.642 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.639+0000 7fde327fc640 1 -- 172.21.15.169:0/946201417 <== mon.2 v2:172.21.15.170:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fde34077220 con 0x7fde3c1592f0 2023-10-26T01:30:56.643 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.639+0000 7fcddd444640 1 -- 172.21.15.169:0/3023981145 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdd80c6f70 con 0x7fcdd815ec90 2023-10-26T01:30:56.654 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.651+0000 7fcdbf7fe640 1 -- 172.21.15.169:0/3023981145 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fcdd0034bc0 con 0x7fcdd815ec90 2023-10-26T01:30:56.656 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.653+0000 7fcdbf7fe640 1 --2- 172.21.15.169:0/3023981145 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcdc406ae40 0x7fcdc406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:56.656 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.653+0000 7fcdbf7fe640 1 -- 172.21.15.169:0/3023981145 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcdd00acae0 con 0x7fcdd815ec90 2023-10-26T01:30:56.657 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.654+0000 7fcdbf7fe640 1 -- 172.21.15.169:0/3023981145 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcdd00d8ca0 con 0x7fcdd815ec90 2023-10-26T01:30:56.657 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.654+0000 7fcdd6ffd640 1 --2- 172.21.15.169:0/3023981145 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcdc406ae40 0x7fcdc406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:56.661 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.659+0000 7fcdd6ffd640 1 --2- 172.21.15.169:0/3023981145 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcdc406ae40 0x7fcdc406d300 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fcdc80058c0 tx=0x7fcdc8009260 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:56.686 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.683+0000 7fce3558f640 1 -- 172.21.15.169:0/1519402834 <== mon.2 v2:172.21.15.170:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce2c005220 con 0x7fce30155360 2023-10-26T01:30:56.687 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.683+0000 7fce387f5640 1 -- 172.21.15.169:0/1519402834 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fce30155360 msgr2=0x7fce300cd7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:56.687 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.683+0000 7fce387f5640 1 --2- 172.21.15.169:0/1519402834 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fce30155360 0x7fce300cd7e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fce2c005fd0 tx=0x7fce2c02be40 comp rx=0 tx=0).stop 2023-10-26T01:30:56.687 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.684+0000 7fce387f5640 1 -- 172.21.15.169:0/1519402834 shutdown_connections 2023-10-26T01:30:56.687 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.684+0000 7fce387f5640 1 --2- 172.21.15.169:0/1519402834 >> v2:172.21.15.170:3300/0 conn(0x7fce3015bbb0 0x7fce300b05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:56.687 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.684+0000 7fce387f5640 1 --2- 172.21.15.169:0/1519402834 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fce30155360 0x7fce300cd7e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:56.688 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.684+0000 7fce387f5640 1 -- 172.21.15.169:0/1519402834 >> 172.21.15.169:0/1519402834 conn(0x7fce30069930 msgr2=0x7fce30069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:56.688 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.685+0000 7fce387f5640 1 -- 172.21.15.169:0/1519402834 shutdown_connections 2023-10-26T01:30:56.689 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.685+0000 7f02f2894640 1 -- 172.21.15.169:0/3672984056 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02d4003a90 msgr2=0x7f02d4003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:56.689 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.685+0000 7f95bbf25640 1 -- 172.21.15.169:0/2021423250 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f95b40d0fc0 msgr2=0x7f95b4157290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:56.689 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.685+0000 7f02f2894640 1 --2- 172.21.15.169:0/3672984056 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02d4003a90 0x7f02d4003e70 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f02e0009ce0 tx=0x7f02d40057f0 comp rx=0 tx=0).stop 2023-10-26T01:30:56.690 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.685+0000 7f95bbf25640 1 --2- 172.21.15.169:0/2021423250 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f95b40d0fc0 0x7f95b4157290 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f95a4005fd0 tx=0x7f95a402bd10 comp rx=0 tx=0).stop 2023-10-26T01:30:56.690 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.685+0000 7f02f2894640 1 -- 172.21.15.169:0/3672984056 shutdown_connections 2023-10-26T01:30:56.691 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.685+0000 7f95bbf25640 1 -- 172.21.15.169:0/2021423250 shutdown_connections 2023-10-26T01:30:56.691 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.685+0000 7f95bbf25640 1 --2- 172.21.15.169:0/2021423250 >> v2:172.21.15.169:3300/0 conn(0x7f95b40d21d0 0x7f95b41577d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:56.691 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.685+0000 7f95bbf25640 1 --2- 172.21.15.169:0/2021423250 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f95b40d0fc0 0x7f95b4157290 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:56.692 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.685+0000 7f95bbf25640 1 -- 172.21.15.169:0/2021423250 >> 172.21.15.169:0/2021423250 conn(0x7f95b406b5c0 msgr2=0x7f95b406b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:56.692 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.685+0000 7f02f2894640 1 --2- 172.21.15.169:0/3672984056 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02d4003a90 0x7f02d4003e70 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:56.692 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.685+0000 7f02f2894640 1 -- 172.21.15.169:0/3672984056 >> 172.21.15.169:0/3672984056 conn(0x7f02ec069930 msgr2=0x7f02ec069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:56.693 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.685+0000 7fce387f5640 1 -- 172.21.15.169:0/1519402834 wait complete. 2023-10-26T01:30:56.693 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.685+0000 7f95bbf25640 1 -- 172.21.15.169:0/2021423250 shutdown_connections 2023-10-26T01:30:56.693 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.685+0000 7f95bbf25640 1 -- 172.21.15.169:0/2021423250 wait complete. 2023-10-26T01:30:56.694 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.686+0000 7f02f2894640 1 -- 172.21.15.169:0/3672984056 shutdown_connections 2023-10-26T01:30:56.694 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.686+0000 7f02f2894640 1 -- 172.21.15.169:0/3672984056 wait complete. 2023-10-26T01:30:56.695 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.688+0000 7fce387f5640 1 Processor -- start 2023-10-26T01:30:56.695 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.688+0000 7f95bbf25640 1 Processor -- start 2023-10-26T01:30:56.695 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.688+0000 7f95bbf25640 1 -- start start 2023-10-26T01:30:56.696 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.690+0000 7f02f2894640 1 Processor -- start 2023-10-26T01:30:56.697 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.690+0000 7f95bbf25640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f95b40d0fc0 0x7f95b4149650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:56.697 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.690+0000 7f95bbf25640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95b40d1df0 0x7f95b414a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:56.697 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.690+0000 7f95bbf25640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f95b40d21d0 0x7f95b414a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:56.697 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.690+0000 7f95b94c0640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95b40d1df0 0x7f95b414a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:56.697 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.690+0000 7f95b94c0640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95b40d1df0 0x7f95b414a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:46096/0 (socket says 172.21.15.169:46096) 2023-10-26T01:30:56.698 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.690+0000 7f95b94c0640 1 -- 172.21.15.169:0/3430756650 learned_addr learned my addr 172.21.15.169:0/3430756650 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:30:56.698 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.690+0000 7f95bbf25640 1 -- 172.21.15.169:0/3430756650 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95b40b5220 con 0x7f95b40d1df0 2023-10-26T01:30:56.698 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.690+0000 7f95bbf25640 1 -- 172.21.15.169:0/3430756650 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95b40b5d60 con 0x7f95b40d0fc0 2023-10-26T01:30:56.698 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.691+0000 7f95bbf25640 1 -- 172.21.15.169:0/3430756650 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95b40b5430 con 0x7f95b40d21d0 2023-10-26T01:30:56.699 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.689+0000 7fce387f5640 1 -- start start 2023-10-26T01:30:56.699 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.691+0000 7f95b9cc1640 1 --2- 172.21.15.169:0/3430756650 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f95b40d0fc0 0x7f95b4149650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:56.700 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.691+0000 7f02f2894640 1 -- start start 2023-10-26T01:30:56.700 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.691+0000 7f95ba4c2640 1 --2- 172.21.15.169:0/3430756650 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f95b40d21d0 0x7f95b414a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:56.701 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.692+0000 7fce387f5640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fce300cdd20 0x7fce30153fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:56.701 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.692+0000 7f95b94c0640 1 -- 172.21.15.169:0/3430756650 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f95b40d21d0 msgr2=0x7f95b414a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:56.701 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.692+0000 7f95b94c0640 1 --2- 172.21.15.169:0/3430756650 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f95b40d21d0 0x7f95b414a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:56.702 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.692+0000 7f95b94c0640 1 -- 172.21.15.169:0/3430756650 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f95b40d0fc0 msgr2=0x7f95b4149650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:56.702 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.692+0000 7f95b94c0640 1 --2- 172.21.15.169:0/3430756650 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f95b40d0fc0 0x7f95b4149650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:56.702 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.692+0000 7f95b94c0640 1 -- 172.21.15.169:0/3430756650 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95a4009560 con 0x7f95b40d1df0 2023-10-26T01:30:56.702 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.692+0000 7f95b9cc1640 1 --2- 172.21.15.169:0/3430756650 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f95b40d0fc0 0x7f95b4149650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T01:30:56.703 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.692+0000 7f95b94c0640 1 --2- 172.21.15.169:0/3430756650 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95b40d1df0 0x7f95b414a2e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f95a4009850 tx=0x7f95a402da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:56.703 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.692+0000 7fce387f5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce30155360 0x7fce301f2780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:56.703 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.692+0000 7fce387f5640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce3015bbb0 0x7fce301f2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:56.703 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.692+0000 7fce387f5640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce300aec10 con 0x7fce3015bbb0 2023-10-26T01:30:56.704 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.692+0000 7fce387f5640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce300afa70 con 0x7fce30155360 2023-10-26T01:30:56.704 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.692+0000 7fce387f5640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce300aee60 con 0x7fce300cdd20 2023-10-26T01:30:56.704 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.693+0000 7f95aa7fc640 1 -- 172.21.15.169:0/3430756650 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a402de90 con 0x7f95b40d1df0 2023-10-26T01:30:56.705 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.693+0000 7f95bbf25640 1 -- 172.21.15.169:0/3430756650 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95b40b5710 con 0x7f95b40d1df0 2023-10-26T01:30:56.705 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.693+0000 7fce36591640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fce300cdd20 0x7fce30153fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:56.706 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.693+0000 7f95bbf25640 1 -- 172.21.15.169:0/3430756650 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95b414b140 con 0x7f95b40d1df0 2023-10-26T01:30:56.706 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.693+0000 7fce36591640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fce300cdd20 0x7fce30153fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3301/0 says I am v2:172.21.15.169:54038/0 (socket says 172.21.15.169:54038) 2023-10-26T01:30:56.706 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.693+0000 7fce36591640 1 -- 172.21.15.169:0/3980647646 learned_addr learned my addr 172.21.15.169:0/3980647646 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:30:56.707 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.693+0000 7fce36d92640 1 --2- 172.21.15.169:0/3980647646 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce3015bbb0 0x7fce301f2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:56.707 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.693+0000 7fce35d90640 1 --2- 172.21.15.169:0/3980647646 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce30155360 0x7fce301f2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:56.707 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.693+0000 7f95aa7fc640 1 -- 172.21.15.169:0/3430756650 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f95a402de90 con 0x7f95b40d1df0 2023-10-26T01:30:56.708 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.693+0000 7f95aa7fc640 1 -- 172.21.15.169:0/3430756650 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a40036e0 con 0x7f95b40d1df0 2023-10-26T01:30:56.708 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.694+0000 7fce36591640 1 -- 172.21.15.169:0/3980647646 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce30155360 msgr2=0x7fce301f2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:56.708 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.694+0000 7fce36591640 1 --2- 172.21.15.169:0/3980647646 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce30155360 0x7fce301f2780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:56.708 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.694+0000 7fce36591640 1 -- 172.21.15.169:0/3980647646 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce3015bbb0 msgr2=0x7fce301f2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:56.709 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.694+0000 7fce36591640 1 --2- 172.21.15.169:0/3980647646 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce3015bbb0 0x7fce301f2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:56.709 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.694+0000 7fce36591640 1 -- 172.21.15.169:0/3980647646 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce2c009560 con 0x7fce300cdd20 2023-10-26T01:30:56.709 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.695+0000 7fce36d92640 1 --2- 172.21.15.169:0/3980647646 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce3015bbb0 0x7fce301f2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T01:30:56.710 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.692+0000 7f02f2894640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02ec1485c0 0x7f02ec167070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:56.710 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.692+0000 7f02f2894640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f02ec0955d0 0x7f02ec1489a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:56.710 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.692+0000 7f02f2894640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f02ec06f7a0 0x7f02ec06fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:56.710 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.695+0000 7f02f2894640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02ec1596f0 con 0x7f02ec1485c0 2023-10-26T01:30:56.711 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.695+0000 7f02f2894640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02ec159260 con 0x7f02ec0955d0 2023-10-26T01:30:56.711 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.695+0000 7f02f2894640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02ec148ee0 con 0x7f02ec06f7a0 2023-10-26T01:30:56.711 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.696+0000 7fce36591640 1 --2- 172.21.15.169:0/3980647646 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fce300cdd20 0x7fce30153fb0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fce2c006010 tx=0x7fce2c004360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:56.712 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.696+0000 7f02f0e31640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f02ec06f7a0 0x7f02ec06fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:56.712 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.696+0000 7f02f0e31640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f02ec06f7a0 0x7f02ec06fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3301/0 says I am v2:172.21.15.169:54042/0 (socket says 172.21.15.169:54042) 2023-10-26T01:30:56.712 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.696+0000 7f02f0e31640 1 -- 172.21.15.169:0/4260255585 learned_addr learned my addr 172.21.15.169:0/4260255585 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:30:56.713 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.696+0000 7f02ebfff640 1 --2- 172.21.15.169:0/4260255585 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02ec1485c0 0x7f02ec167070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:56.713 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.696+0000 7f02eb7fe640 1 --2- 172.21.15.169:0/4260255585 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f02ec0955d0 0x7f02ec1489a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:56.713 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.697+0000 7fce26ffd640 1 -- 172.21.15.169:0/3980647646 <== mon.2 v2:172.21.15.170:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce2c003800 con 0x7fce300cdd20 2023-10-26T01:30:56.714 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.697+0000 7fce26ffd640 1 -- 172.21.15.169:0/3980647646 <== mon.2 v2:172.21.15.170:3301/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fce2c003a50 con 0x7fce300cdd20 2023-10-26T01:30:56.714 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.697+0000 7fce26ffd640 1 -- 172.21.15.169:0/3980647646 <== mon.2 v2:172.21.15.170:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce2c003e20 con 0x7fce300cdd20 2023-10-26T01:30:56.715 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.697+0000 7f02f0e31640 1 -- 172.21.15.169:0/4260255585 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f02ec0955d0 msgr2=0x7f02ec1489a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:56.715 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.697+0000 7f02f0e31640 1 --2- 172.21.15.169:0/4260255585 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f02ec0955d0 0x7f02ec1489a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:56.715 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.697+0000 7f02f0e31640 1 -- 172.21.15.169:0/4260255585 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02ec1485c0 msgr2=0x7f02ec167070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:56.715 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.697+0000 7f02f0e31640 1 --2- 172.21.15.169:0/4260255585 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02ec1485c0 0x7f02ec167070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:56.716 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.697+0000 7f02f0e31640 1 -- 172.21.15.169:0/4260255585 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02ec0b1500 con 0x7f02ec06f7a0 2023-10-26T01:30:56.716 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.698+0000 7fce387f5640 1 -- 172.21.15.169:0/3980647646 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce300af140 con 0x7fce300cdd20 2023-10-26T01:30:56.716 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.698+0000 7f02f0e31640 1 --2- 172.21.15.169:0/4260255585 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f02ec06f7a0 0x7f02ec06fc30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f02e4003da0 tx=0x7f02e4007660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:56.717 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.699+0000 7fce387f5640 1 -- 172.21.15.169:0/3980647646 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce300af700 con 0x7fce300cdd20 2023-10-26T01:30:56.717 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.699+0000 7f95bbf25640 1 -- 172.21.15.169:0/3430756650 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f959c005250 con 0x7f95b40d1df0 2023-10-26T01:30:56.718 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.699+0000 7f02e8ff9640 1 -- 172.21.15.169:0/4260255585 <== mon.2 v2:172.21.15.170:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02e400e9a0 con 0x7f02ec06f7a0 2023-10-26T01:30:56.718 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.700+0000 7fce387f5640 1 -- 172.21.15.169:0/3980647646 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce3006c0d0 con 0x7fce300cdd20 2023-10-26T01:30:56.718 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.700+0000 7f95aa7fc640 1 -- 172.21.15.169:0/3430756650 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f95a4003840 con 0x7f95b40d1df0 2023-10-26T01:30:56.719 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.702+0000 7f95aa7fc640 1 --2- 172.21.15.169:0/3430756650 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f959806aea0 0x7f959806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:56.719 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.702+0000 7f95aa7fc640 1 -- 172.21.15.169:0/3430756650 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f95a40abc10 con 0x7f95b40d1df0 2023-10-26T01:30:56.719 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.702+0000 7f95b9cc1640 1 --2- 172.21.15.169:0/3430756650 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f959806aea0 0x7f959806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:56.720 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.700+0000 7f02f2894640 1 -- 172.21.15.169:0/4260255585 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02ec0963e0 con 0x7f02ec06f7a0 2023-10-26T01:30:56.720 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.700+0000 7f02f2894640 1 -- 172.21.15.169:0/4260255585 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02ec06f0b0 con 0x7f02ec06f7a0 2023-10-26T01:30:56.720 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.700+0000 7f02e8ff9640 1 -- 172.21.15.169:0/4260255585 <== mon.2 v2:172.21.15.170:3301/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f02e400eb00 con 0x7f02ec06f7a0 2023-10-26T01:30:56.720 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.700+0000 7f02e8ff9640 1 -- 172.21.15.169:0/4260255585 <== mon.2 v2:172.21.15.170:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02e4011450 con 0x7f02ec06f7a0 2023-10-26T01:30:56.720 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.702+0000 7f02f2894640 1 -- 172.21.15.169:0/4260255585 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02ec04db10 con 0x7f02ec06f7a0 2023-10-26T01:30:56.721 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.704+0000 7f95b9cc1640 1 --2- 172.21.15.169:0/3430756650 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f959806aea0 0x7f959806d360 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f95b0001bc0 tx=0x7f95b0009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:56.722 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.704+0000 7fce26ffd640 1 -- 172.21.15.169:0/3980647646 <== mon.2 v2:172.21.15.170:3301/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fce2c034440 con 0x7fce300cdd20 2023-10-26T01:30:56.722 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.705+0000 7fce26ffd640 1 --2- 172.21.15.169:0/3980647646 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fce1406ae90 0x7fce1406d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:56.722 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.705+0000 7fce26ffd640 1 -- 172.21.15.169:0/3980647646 <== mon.2 v2:172.21.15.170:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fce2c0ae720 con 0x7fce300cdd20 2023-10-26T01:30:56.722 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.706+0000 7fce35d90640 1 --2- 172.21.15.169:0/3980647646 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fce1406ae90 0x7fce1406d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:56.723 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.707+0000 7f02e8ff9640 1 -- 172.21.15.169:0/4260255585 <== mon.2 v2:172.21.15.170:3301/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f02e40115b0 con 0x7f02ec06f7a0 2023-10-26T01:30:56.723 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.707+0000 7fce35d90640 1 --2- 172.21.15.169:0/3980647646 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fce1406ae90 0x7fce1406d350 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fce300b0510 tx=0x7fce20009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:56.724 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.708+0000 7f02e8ff9640 1 --2- 172.21.15.169:0/4260255585 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f02d806ae70 0x7f02d806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:56.724 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.708+0000 7f02e8ff9640 1 -- 172.21.15.169:0/4260255585 <== mon.2 v2:172.21.15.170:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f02e408a870 con 0x7f02ec06f7a0 2023-10-26T01:30:56.724 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.709+0000 7f02ebfff640 1 --2- 172.21.15.169:0/4260255585 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f02d806ae70 0x7f02d806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:56.725 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.710+0000 7f02ebfff640 1 --2- 172.21.15.169:0/4260255585 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f02d806ae70 0x7f02d806d330 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f02e0009cb0 tx=0x7f02e00046c0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:56.725 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.711+0000 7f95aa7fc640 1 -- 172.21.15.169:0/3430756650 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f95a4076e00 con 0x7f95b40d1df0 2023-10-26T01:30:56.725 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.717+0000 7fce26ffd640 1 -- 172.21.15.169:0/3980647646 <== mon.2 v2:172.21.15.170:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fce2c079860 con 0x7fce300cdd20 2023-10-26T01:30:56.732 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.730+0000 7f02e8ff9640 1 -- 172.21.15.169:0/4260255585 <== mon.2 v2:172.21.15.170:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f02e4055990 con 0x7f02ec06f7a0 2023-10-26T01:30:56.963 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.961+0000 7f6df10ea640 1 -- 172.21.15.169:0/27286718 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f6de40089a0 con 0x7f6de40df380 2023-10-26T01:30:56.964 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.961+0000 7f6de8ff9640 1 -- 172.21.15.169:0/27286718 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6dec0cf410 con 0x7f6de40df380 2023-10-26T01:30:56.965 INFO:teuthology.orchestra.run.smithi169.stdout:55834574855 2023-10-26T01:30:56.968 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.965+0000 7f6df10ea640 1 -- 172.21.15.169:0/27286718 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f6dd006ae40 msgr2=0x7f6dd006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:56.968 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.965+0000 7f6df10ea640 1 --2- 172.21.15.169:0/27286718 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f6dd006ae40 0x7f6dd006d300 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6de0005fd0 tx=0x7f6de000c040 comp rx=0 tx=0).stop 2023-10-26T01:30:56.969 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.966+0000 7f6df10ea640 1 -- 172.21.15.169:0/27286718 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6de40df380 msgr2=0x7f6de404a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:56.969 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.966+0000 7f6df10ea640 1 --2- 172.21.15.169:0/27286718 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6de40df380 0x7f6de404a570 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6dec074260 tx=0x7f6dec067dd0 comp rx=0 tx=0).stop 2023-10-26T01:30:56.970 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.967+0000 7f6df10ea640 1 -- 172.21.15.169:0/27286718 shutdown_connections 2023-10-26T01:30:56.970 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.967+0000 7f6df10ea640 1 --2- 172.21.15.169:0/27286718 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f6dd006ae40 0x7f6dd006d300 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:56.970 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.967+0000 7f6df10ea640 1 --2- 172.21.15.169:0/27286718 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6de40df380 0x7f6de404a570 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:56.970 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.967+0000 7f6df10ea640 1 --2- 172.21.15.169:0/27286718 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6de4053b30 0x7f6de404d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:56.971 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.967+0000 7f6df10ea640 1 --2- 172.21.15.169:0/27286718 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6de4050230 0x7f6de404d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:56.971 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.967+0000 7f6df10ea640 1 -- 172.21.15.169:0/27286718 >> 172.21.15.169:0/27286718 conn(0x7f6de4009f80 msgr2=0x7f6de40383d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:56.972 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.968+0000 7f6df10ea640 1 -- 172.21.15.169:0/27286718 shutdown_connections 2023-10-26T01:30:56.972 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.968+0000 7f6df10ea640 1 -- 172.21.15.169:0/27286718 wait complete. 2023-10-26T01:30:56.989 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.985+0000 7f3e3d19e640 1 -- 172.21.15.169:0/2702537651 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f3e30005360 con 0x7f3e300063a0 2023-10-26T01:30:56.989 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.986+0000 7f3e1f7fe640 1 -- 172.21.15.169:0/2702537651 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3e2c057bf0 con 0x7f3e300063a0 2023-10-26T01:30:56.991 INFO:teuthology.orchestra.run.smithi169.stdout:55834574855 2023-10-26T01:30:56.993 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.991+0000 7f3e3d19e640 1 -- 172.21.15.169:0/2702537651 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3e2406ae90 msgr2=0x7f3e2406d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:56.993 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.991+0000 7f3e3d19e640 1 --2- 172.21.15.169:0/2702537651 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3e2406ae90 0x7f3e2406d350 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3e28036000 tx=0x7f3e28004730 comp rx=0 tx=0).stop 2023-10-26T01:30:56.994 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.991+0000 7f3e3d19e640 1 -- 172.21.15.169:0/2702537651 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3e300063a0 msgr2=0x7f3e300deda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:56.994 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.991+0000 7f3e3d19e640 1 --2- 172.21.15.169:0/2702537651 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3e300063a0 0x7f3e300deda0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c00c3c0 tx=0x7f3e2c00c880 comp rx=0 tx=0).stop 2023-10-26T01:30:56.995 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.991+0000 7f3e3d19e640 1 -- 172.21.15.169:0/2702537651 shutdown_connections 2023-10-26T01:30:56.995 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.991+0000 7f3e3d19e640 1 --2- 172.21.15.169:0/2702537651 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3e2406ae90 0x7f3e2406d350 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:56.995 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.991+0000 7f3e3d19e640 1 --2- 172.21.15.169:0/2702537651 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3e300e6f80 0x7f3e300dd620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:56.995 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.991+0000 7f3e3d19e640 1 --2- 172.21.15.169:0/2702537651 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3e3004f260 0x7f3e3004dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:56.996 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.991+0000 7f3e3d19e640 1 --2- 172.21.15.169:0/2702537651 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3e300063a0 0x7f3e300deda0 secure :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c00c3c0 tx=0x7f3e2c00c880 comp rx=0 tx=0).stop 2023-10-26T01:30:56.996 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.991+0000 7f3e3d19e640 1 -- 172.21.15.169:0/2702537651 >> 172.21.15.169:0/2702537651 conn(0x7f3e30009f80 msgr2=0x7f3e30006780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:56.996 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.992+0000 7f3e3d19e640 1 -- 172.21.15.169:0/2702537651 shutdown_connections 2023-10-26T01:30:56.996 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:56.992+0000 7f3e3d19e640 1 -- 172.21.15.169:0/2702537651 wait complete. 2023-10-26T01:30:57.003 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574855 for osd.0 2023-10-26T01:30:57.003 DEBUG:teuthology.parallel:result is None 2023-10-26T01:30:57.019 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.017+0000 7faea5ffb640 1 -- 172.21.15.169:0/1359735825 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fae880050c0 con 0x7faeb00cd780 2023-10-26T01:30:57.021 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.019+0000 7faeacff9640 1 -- 172.21.15.169:0/1359735825 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7faea8058750 con 0x7faeb00cd780 2023-10-26T01:30:57.021 INFO:teuthology.orchestra.run.smithi169.stdout:60129542151 2023-10-26T01:30:57.022 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574855 for osd.1 2023-10-26T01:30:57.023 DEBUG:teuthology.parallel:result is None 2023-10-26T01:30:57.025 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.022+0000 7faea5ffb640 1 -- 172.21.15.169:0/1359735825 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fae9006ae40 msgr2=0x7fae9006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:57.025 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.022+0000 7faea5ffb640 1 --2- 172.21.15.169:0/1359735825 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fae9006ae40 0x7fae9006d300 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7faea0036000 tx=0x7faea0004ab0 comp rx=0 tx=0).stop 2023-10-26T01:30:57.025 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.022+0000 7faea5ffb640 1 -- 172.21.15.169:0/1359735825 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faeb00cd780 msgr2=0x7faeb0149690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:57.025 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.022+0000 7faea5ffb640 1 --2- 172.21.15.169:0/1359735825 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faeb00cd780 0x7faeb0149690 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7faea800eab0 tx=0x7faea8009f80 comp rx=0 tx=0).stop 2023-10-26T01:30:57.026 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.022+0000 7faea5ffb640 1 -- 172.21.15.169:0/1359735825 shutdown_connections 2023-10-26T01:30:57.026 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.022+0000 7faea5ffb640 1 --2- 172.21.15.169:0/1359735825 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fae9006ae40 0x7fae9006d300 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:57.026 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.022+0000 7faea5ffb640 1 --2- 172.21.15.169:0/1359735825 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faeb00cd780 0x7faeb0149690 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:57.026 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.022+0000 7faea5ffb640 1 --2- 172.21.15.169:0/1359735825 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7faeb00b1130 0x7faeb006adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:57.026 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.022+0000 7faea5ffb640 1 --2- 172.21.15.169:0/1359735825 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faeb006bee0 0x7faeb014df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:57.027 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.022+0000 7faea5ffb640 1 -- 172.21.15.169:0/1359735825 >> 172.21.15.169:0/1359735825 conn(0x7faeb00b2510 msgr2=0x7faeb015e3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:57.027 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.022+0000 7faea5ffb640 1 -- 172.21.15.169:0/1359735825 shutdown_connections 2023-10-26T01:30:57.027 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.022+0000 7faea5ffb640 1 -- 172.21.15.169:0/1359735825 wait complete. 2023-10-26T01:30:57.050 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542151 for osd.3 2023-10-26T01:30:57.051 DEBUG:teuthology.parallel:result is None 2023-10-26T01:30:57.057 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.055+0000 7f95bbf25640 1 -- 172.21.15.169:0/3430756650 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f959c0050c0 con 0x7f95b40d1df0 2023-10-26T01:30:57.058 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.055+0000 7f95aa7fc640 1 -- 172.21.15.169:0/3430756650 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f95a4043020 con 0x7f95b40d1df0 2023-10-26T01:30:57.058 INFO:teuthology.orchestra.run.smithi169.stdout:55834574855 2023-10-26T01:30:57.061 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.058+0000 7f95bbf25640 1 -- 172.21.15.169:0/3430756650 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f959806aea0 msgr2=0x7f959806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:57.061 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.058+0000 7f95bbf25640 1 --2- 172.21.15.169:0/3430756650 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f959806aea0 0x7f959806d360 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f95b0001bc0 tx=0x7f95b0009040 comp rx=0 tx=0).stop 2023-10-26T01:30:57.061 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.058+0000 7f95bbf25640 1 -- 172.21.15.169:0/3430756650 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95b40d1df0 msgr2=0x7f95b414a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:57.061 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.058+0000 7f95bbf25640 1 --2- 172.21.15.169:0/3430756650 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95b40d1df0 0x7f95b414a2e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f95a4009850 tx=0x7f95a402da90 comp rx=0 tx=0).stop 2023-10-26T01:30:57.062 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.058+0000 7f95bbf25640 1 -- 172.21.15.169:0/3430756650 shutdown_connections 2023-10-26T01:30:57.062 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.058+0000 7f95bbf25640 1 --2- 172.21.15.169:0/3430756650 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f959806aea0 0x7f959806d360 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:57.062 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.058+0000 7f95bbf25640 1 --2- 172.21.15.169:0/3430756650 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95b40d1df0 0x7f95b414a2e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:57.062 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.058+0000 7f95bbf25640 1 --2- 172.21.15.169:0/3430756650 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f95b40d21d0 0x7f95b414a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:57.062 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.058+0000 7f95bbf25640 1 --2- 172.21.15.169:0/3430756650 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f95b40d0fc0 0x7f95b4149650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:57.063 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.058+0000 7f95bbf25640 1 -- 172.21.15.169:0/3430756650 >> 172.21.15.169:0/3430756650 conn(0x7f95b406b5c0 msgr2=0x7f95b414fdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:57.063 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.058+0000 7f95bbf25640 1 -- 172.21.15.169:0/3430756650 shutdown_connections 2023-10-26T01:30:57.063 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.058+0000 7f95bbf25640 1 -- 172.21.15.169:0/3430756650 wait complete. 2023-10-26T01:30:57.064 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.060+0000 7fde4416c640 1 -- 172.21.15.169:0/946201417 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fde3c072060 con 0x7fde3c1592f0 2023-10-26T01:30:57.064 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.061+0000 7fde327fc640 1 -- 172.21.15.169:0/946201417 <== mon.2 v2:172.21.15.170:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fde3407a840 con 0x7fde3c1592f0 2023-10-26T01:30:57.064 INFO:teuthology.orchestra.run.smithi169.stdout:55834574855 2023-10-26T01:30:57.065 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.062+0000 7fce387f5640 1 -- 172.21.15.169:0/3980647646 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fce301558d0 con 0x7fce300cdd20 2023-10-26T01:30:57.065 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.063+0000 7fce26ffd640 1 -- 172.21.15.169:0/3980647646 <== mon.2 v2:172.21.15.170:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fce2c04d020 con 0x7fce300cdd20 2023-10-26T01:30:57.066 INFO:teuthology.orchestra.run.smithi169.stdout:55834574855 2023-10-26T01:30:57.068 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.065+0000 7fde4416c640 1 -- 172.21.15.169:0/946201417 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fde2006af10 msgr2=0x7fde2006d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:57.068 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.065+0000 7fde4416c640 1 --2- 172.21.15.169:0/946201417 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fde2006af10 0x7fde2006d3d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fde38000f60 tx=0x7fde38009040 comp rx=0 tx=0).stop 2023-10-26T01:30:57.068 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.065+0000 7fde4416c640 1 -- 172.21.15.169:0/946201417 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fde3c1592f0 msgr2=0x7fde3c1434e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:57.068 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.065+0000 7fde4416c640 1 --2- 172.21.15.169:0/946201417 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fde3c1592f0 0x7fde3c1434e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fde34037040 tx=0x7fde34003f10 comp rx=0 tx=0).stop 2023-10-26T01:30:57.068 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.065+0000 7fde4416c640 1 -- 172.21.15.169:0/946201417 shutdown_connections 2023-10-26T01:30:57.069 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.065+0000 7fde4416c640 1 --2- 172.21.15.169:0/946201417 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fde2006af10 0x7fde2006d3d0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:57.069 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.065+0000 7fde4416c640 1 --2- 172.21.15.169:0/946201417 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fde3c1592f0 0x7fde3c1434e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:57.069 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.065+0000 7fde4416c640 1 --2- 172.21.15.169:0/946201417 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fde3c0b12f0 0x7fde3c157fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:57.069 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.065+0000 7fde4416c640 1 --2- 172.21.15.169:0/946201417 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fde3c06c520 0x7fde3c0ab650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:57.069 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.065+0000 7fde4416c640 1 -- 172.21.15.169:0/946201417 >> 172.21.15.169:0/946201417 conn(0x7fde3c0b2800 msgr2=0x7fde3c15a760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:57.070 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.065+0000 7fde4416c640 1 -- 172.21.15.169:0/946201417 shutdown_connections 2023-10-26T01:30:57.070 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.065+0000 7fde4416c640 1 -- 172.21.15.169:0/946201417 wait complete. 2023-10-26T01:30:57.071 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.065+0000 7fce387f5640 1 -- 172.21.15.169:0/3980647646 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fce1406ae90 msgr2=0x7fce1406d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:57.071 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.065+0000 7fce387f5640 1 --2- 172.21.15.169:0/3980647646 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fce1406ae90 0x7fce1406d350 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fce300b0510 tx=0x7fce20009040 comp rx=0 tx=0).stop 2023-10-26T01:30:57.071 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.065+0000 7fce387f5640 1 -- 172.21.15.169:0/3980647646 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fce300cdd20 msgr2=0x7fce30153fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:57.071 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.065+0000 7fce387f5640 1 --2- 172.21.15.169:0/3980647646 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fce300cdd20 0x7fce30153fb0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fce2c006010 tx=0x7fce2c004360 comp rx=0 tx=0).stop 2023-10-26T01:30:57.071 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.066+0000 7fce387f5640 1 -- 172.21.15.169:0/3980647646 shutdown_connections 2023-10-26T01:30:57.072 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.066+0000 7fce387f5640 1 --2- 172.21.15.169:0/3980647646 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fce1406ae90 0x7fce1406d350 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:57.072 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.066+0000 7fce387f5640 1 --2- 172.21.15.169:0/3980647646 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce3015bbb0 0x7fce301f2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:57.072 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.066+0000 7fce387f5640 1 --2- 172.21.15.169:0/3980647646 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce30155360 0x7fce301f2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:57.072 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.066+0000 7fce387f5640 1 --2- 172.21.15.169:0/3980647646 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fce300cdd20 0x7fce30153fb0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:57.072 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.066+0000 7fce387f5640 1 -- 172.21.15.169:0/3980647646 >> 172.21.15.169:0/3980647646 conn(0x7fce30069930 msgr2=0x7fce30150f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:57.073 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.066+0000 7fce387f5640 1 -- 172.21.15.169:0/3980647646 shutdown_connections 2023-10-26T01:30:57.073 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.066+0000 7fce387f5640 1 -- 172.21.15.169:0/3980647646 wait complete. 2023-10-26T01:30:57.082 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.080+0000 7fcddd444640 1 -- 172.21.15.169:0/3023981145 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fcdd8073c10 con 0x7fcdd815ec90 2023-10-26T01:30:57.082 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.080+0000 7fcdbf7fe640 1 -- 172.21.15.169:0/3023981145 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fcdd004b090 con 0x7fcdd815ec90 2023-10-26T01:30:57.083 INFO:teuthology.orchestra.run.smithi169.stdout:55834574855 2023-10-26T01:30:57.086 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574855 for osd.2 2023-10-26T01:30:57.086 DEBUG:teuthology.parallel:result is None 2023-10-26T01:30:57.087 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.084+0000 7fcddd444640 1 -- 172.21.15.169:0/3023981145 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcdc406ae40 msgr2=0x7fcdc406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:57.087 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.084+0000 7fcddd444640 1 --2- 172.21.15.169:0/3023981145 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcdc406ae40 0x7fcdc406d300 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fcdc80058c0 tx=0x7fcdc8009260 comp rx=0 tx=0).stop 2023-10-26T01:30:57.087 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.084+0000 7fcddd444640 1 -- 172.21.15.169:0/3023981145 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcdd815ec90 msgr2=0x7fcdd815e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:57.088 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.084+0000 7fcddd444640 1 --2- 172.21.15.169:0/3023981145 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcdd815ec90 0x7fcdd815e070 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fcdd80663e0 tx=0x7fcdd0004250 comp rx=0 tx=0).stop 2023-10-26T01:30:57.089 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574855 for osd.5 2023-10-26T01:30:57.089 DEBUG:teuthology.parallel:result is None 2023-10-26T01:30:57.090 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.084+0000 7fcddd444640 1 -- 172.21.15.169:0/3023981145 shutdown_connections 2023-10-26T01:30:57.090 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.084+0000 7fcddd444640 1 --2- 172.21.15.169:0/3023981145 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcdc406ae40 0x7fcdc406d300 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:57.090 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.084+0000 7fcddd444640 1 --2- 172.21.15.169:0/3023981145 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcdd815ec90 0x7fcdd815e070 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:57.090 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.084+0000 7fcddd444640 1 --2- 172.21.15.169:0/3023981145 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcdd80d20c0 0x7fcdd815db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:57.091 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.084+0000 7fcddd444640 1 --2- 172.21.15.169:0/3023981145 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcdd80d0ea0 0x7fcdd814be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:57.091 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.084+0000 7fcddd444640 1 -- 172.21.15.169:0/3023981145 >> 172.21.15.169:0/3023981145 conn(0x7fcdd80d1390 msgr2=0x7fcdd80cef40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:57.091 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.084+0000 7fcddd444640 1 -- 172.21.15.169:0/3023981145 shutdown_connections 2023-10-26T01:30:57.091 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.084+0000 7fcddd444640 1 -- 172.21.15.169:0/3023981145 wait complete. 2023-10-26T01:30:57.092 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574855 for osd.6 2023-10-26T01:30:57.092 DEBUG:teuthology.parallel:result is None 2023-10-26T01:30:57.105 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.103+0000 7f02f2894640 1 -- 172.21.15.169:0/4260255585 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f02ec1593a0 con 0x7f02ec06f7a0 2023-10-26T01:30:57.107 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.104+0000 7f02e8ff9640 1 -- 172.21.15.169:0/4260255585 <== mon.2 v2:172.21.15.170:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f02e4058fb0 con 0x7f02ec06f7a0 2023-10-26T01:30:57.107 INFO:teuthology.orchestra.run.smithi169.stdout:55834574855 2023-10-26T01:30:57.108 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574855 for osd.7 2023-10-26T01:30:57.109 DEBUG:teuthology.parallel:result is None 2023-10-26T01:30:57.110 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.106+0000 7f02f2894640 1 -- 172.21.15.169:0/4260255585 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f02d806ae70 msgr2=0x7f02d806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:57.110 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.106+0000 7f02f2894640 1 --2- 172.21.15.169:0/4260255585 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f02d806ae70 0x7f02d806d330 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f02e0009cb0 tx=0x7f02e00046c0 comp rx=0 tx=0).stop 2023-10-26T01:30:57.110 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.106+0000 7f02f2894640 1 -- 172.21.15.169:0/4260255585 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f02ec06f7a0 msgr2=0x7f02ec06fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:57.110 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.106+0000 7f02f2894640 1 --2- 172.21.15.169:0/4260255585 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f02ec06f7a0 0x7f02ec06fc30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f02e4003da0 tx=0x7f02e4007660 comp rx=0 tx=0).stop 2023-10-26T01:30:57.111 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.106+0000 7f02f2894640 1 -- 172.21.15.169:0/4260255585 shutdown_connections 2023-10-26T01:30:57.111 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.106+0000 7f02f2894640 1 --2- 172.21.15.169:0/4260255585 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f02d806ae70 0x7f02d806d330 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:57.111 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.106+0000 7f02f2894640 1 --2- 172.21.15.169:0/4260255585 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f02ec06f7a0 0x7f02ec06fc30 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:57.111 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.106+0000 7f02f2894640 1 --2- 172.21.15.169:0/4260255585 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f02ec0955d0 0x7f02ec1489a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:57.111 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.106+0000 7f02f2894640 1 --2- 172.21.15.169:0/4260255585 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02ec1485c0 0x7f02ec167070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:57.112 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.106+0000 7f02f2894640 1 -- 172.21.15.169:0/4260255585 >> 172.21.15.169:0/4260255585 conn(0x7f02ec069930 msgr2=0x7f02ec1545a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:57.112 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.106+0000 7f02f2894640 1 -- 172.21.15.169:0/4260255585 shutdown_connections 2023-10-26T01:30:57.112 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.106+0000 7f02f2894640 1 -- 172.21.15.169:0/4260255585 wait complete. 2023-10-26T01:30:57.125 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574855 for osd.4 2023-10-26T01:30:57.125 DEBUG:teuthology.parallel:result is None 2023-10-26T01:30:57.125 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-10-26T01:30:57.126 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-26T01:30:57.377 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.375+0000 7f8ddda90640 1 -- 172.21.15.169:0/1447513137 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8dc40047c0 msgr2=0x7f8dc4004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:57.377 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.375+0000 7f8ddda90640 1 --2- 172.21.15.169:0/1447513137 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8dc40047c0 0x7f8dc4004c40 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8dc0009b50 tx=0x7f8dc40057f0 comp rx=0 tx=0).stop 2023-10-26T01:30:57.378 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.375+0000 7f8dd7fff640 1 -- 172.21.15.169:0/1447513137 reap_dead start 2023-10-26T01:30:57.378 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.375+0000 7f8ddda90640 1 -- 172.21.15.169:0/1447513137 shutdown_connections 2023-10-26T01:30:57.378 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.375+0000 7f8ddda90640 1 -- 172.21.15.169:0/1447513137 >> 172.21.15.169:0/1447513137 conn(0x7f8dd8069930 msgr2=0x7f8dd8069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:57.378 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.375+0000 7f8ddda90640 1 -- 172.21.15.169:0/1447513137 shutdown_connections 2023-10-26T01:30:57.378 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.375+0000 7f8ddda90640 1 -- 172.21.15.169:0/1447513137 wait complete. 2023-10-26T01:30:57.379 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.377+0000 7f8ddda90640 1 Processor -- start 2023-10-26T01:30:57.379 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.377+0000 7f8ddda90640 1 -- start start 2023-10-26T01:30:57.381 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.378+0000 7f8ddda90640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8dc40047c0 0x7f8dd8150610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:57.381 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.378+0000 7f8ddda90640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f8dd80af780 0x7f8dd80b0f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:57.382 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.378+0000 7f8ddda90640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8dd815b3a0 0x7f8dd815b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:57.382 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.378+0000 7f8ddda90640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dd80b2310 con 0x7f8dd815b3a0 2023-10-26T01:30:57.382 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.378+0000 7f8ddda90640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dd815bd90 con 0x7f8dc40047c0 2023-10-26T01:30:57.382 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.378+0000 7f8ddda90640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dd815c920 con 0x7f8dd80af780 2023-10-26T01:30:57.382 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.378+0000 7f8dd7fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8dd815b3a0 0x7f8dd815b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:57.383 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.378+0000 7f8dd7fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8dd815b3a0 0x7f8dd815b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:46138/0 (socket says 172.21.15.169:46138) 2023-10-26T01:30:57.383 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.378+0000 7f8dd7fff640 1 -- 172.21.15.169:0/2142356115 learned_addr learned my addr 172.21.15.169:0/2142356115 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:30:57.383 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.379+0000 7f8dd77fe640 1 --2- 172.21.15.169:0/2142356115 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8dc40047c0 0x7f8dd8150610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:57.383 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.379+0000 7f8dcffff640 1 --2- 172.21.15.169:0/2142356115 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f8dd80af780 0x7f8dd80b0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:57.384 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.379+0000 7f8dd7fff640 1 -- 172.21.15.169:0/2142356115 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f8dd80af780 msgr2=0x7f8dd80b0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:57.384 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.379+0000 7f8dd7fff640 1 --2- 172.21.15.169:0/2142356115 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f8dd80af780 0x7f8dd80b0f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:57.384 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.379+0000 7f8dd7fff640 1 -- 172.21.15.169:0/2142356115 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8dc40047c0 msgr2=0x7f8dd8150610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:57.384 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.379+0000 7f8dd7fff640 1 --2- 172.21.15.169:0/2142356115 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8dc40047c0 0x7f8dd8150610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:57.384 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.379+0000 7f8dd7fff640 1 -- 172.21.15.169:0/2142356115 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dd80b1790 con 0x7f8dd815b3a0 2023-10-26T01:30:57.384 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.379+0000 7f8dd77fe640 1 --2- 172.21.15.169:0/2142356115 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8dc40047c0 0x7f8dd8150610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:30:57.385 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.379+0000 7f8dd7fff640 1 --2- 172.21.15.169:0/2142356115 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8dd815b3a0 0x7f8dd815b850 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8dd80ce010 tx=0x7f8dd000dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:57.385 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.379+0000 7f8dd4ff9640 1 -- 172.21.15.169:0/2142356115 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dd0018070 con 0x7f8dd815b3a0 2023-10-26T01:30:57.385 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.379+0000 7f8dd4ff9640 1 -- 172.21.15.169:0/2142356115 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8dd0003970 con 0x7f8dd815b3a0 2023-10-26T01:30:57.385 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.379+0000 7f8dd4ff9640 1 -- 172.21.15.169:0/2142356115 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dd0003d80 con 0x7f8dd815b3a0 2023-10-26T01:30:57.385 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.379+0000 7f8ddda90640 1 -- 172.21.15.169:0/2142356115 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dd80b1a70 con 0x7f8dd815b3a0 2023-10-26T01:30:57.386 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.379+0000 7f8ddda90640 1 -- 172.21.15.169:0/2142356115 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dd80b2030 con 0x7f8dd815b3a0 2023-10-26T01:30:57.387 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.380+0000 7f8ddda90640 1 -- 172.21.15.169:0/2142356115 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8dd804db10 con 0x7f8dd815b3a0 2023-10-26T01:30:57.387 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.381+0000 7f8dd4ff9640 1 -- 172.21.15.169:0/2142356115 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f8dd000b940 con 0x7f8dd815b3a0 2023-10-26T01:30:57.387 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.382+0000 7f8dd4ff9640 1 --2- 172.21.15.169:0/2142356115 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f8db806ae70 0x7f8db806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:57.387 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.382+0000 7f8dd4ff9640 1 -- 172.21.15.169:0/2142356115 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8dd0088f70 con 0x7f8dd815b3a0 2023-10-26T01:30:57.387 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.383+0000 7f8dd77fe640 1 --2- 172.21.15.169:0/2142356115 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f8db806ae70 0x7f8db806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:57.388 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.384+0000 7f8dd77fe640 1 --2- 172.21.15.169:0/2142356115 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f8db806ae70 0x7f8db806d330 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8dd80b2480 tx=0x7f8dc8009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:57.388 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.386+0000 7f8dd4ff9640 1 -- 172.21.15.169:0/2142356115 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8dd0054010 con 0x7f8dd815b3a0 2023-10-26T01:30:57.558 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.557+0000 7f8ddda90640 1 -- 172.21.15.169:0/2142356115 --> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8dd8155020 con 0x7f8db806ae70 2023-10-26T01:30:57.580 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.578+0000 7f8dd4ff9640 1 -- 172.21.15.169:0/2142356115 <== mgr.4112 v2:172.21.15.170:6832/1187156918 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+260063 (secure 0 0 0) 0x7f8dd8155020 con 0x7f8db806ae70 2023-10-26T01:30:57.583 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:30:57.585 INFO:teuthology.orchestra.run.smithi169.stderr:dumped all 2023-10-26T01:30:57.586 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.583+0000 7f8ddda90640 1 -- 172.21.15.169:0/2142356115 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f8db806ae70 msgr2=0x7f8db806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:57.587 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.583+0000 7f8ddda90640 1 --2- 172.21.15.169:0/2142356115 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f8db806ae70 0x7f8db806d330 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8dd80b2480 tx=0x7f8dc8009040 comp rx=0 tx=0).stop 2023-10-26T01:30:57.587 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.583+0000 7f8ddda90640 1 -- 172.21.15.169:0/2142356115 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8dd815b3a0 msgr2=0x7f8dd815b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:57.587 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.583+0000 7f8ddda90640 1 --2- 172.21.15.169:0/2142356115 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8dd815b3a0 0x7f8dd815b850 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8dd80ce010 tx=0x7f8dd000dd10 comp rx=0 tx=0).stop 2023-10-26T01:30:57.587 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.583+0000 7f8ddda90640 1 -- 172.21.15.169:0/2142356115 shutdown_connections 2023-10-26T01:30:57.587 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.583+0000 7f8ddda90640 1 --2- 172.21.15.169:0/2142356115 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f8db806ae70 0x7f8db806d330 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:57.588 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.583+0000 7f8ddda90640 1 --2- 172.21.15.169:0/2142356115 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8dd815b3a0 0x7f8dd815b850 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:57.588 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.583+0000 7f8ddda90640 1 --2- 172.21.15.169:0/2142356115 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f8dd80af780 0x7f8dd80b0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:57.588 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.583+0000 7f8ddda90640 1 --2- 172.21.15.169:0/2142356115 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8dc40047c0 0x7f8dd8150610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:57.588 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.583+0000 7f8ddda90640 1 -- 172.21.15.169:0/2142356115 >> 172.21.15.169:0/2142356115 conn(0x7f8dd8069930 msgr2=0x7f8dd8154570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:57.588 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.583+0000 7f8ddda90640 1 -- 172.21.15.169:0/2142356115 shutdown_connections 2023-10-26T01:30:57.589 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.583+0000 7f8ddda90640 1 -- 172.21.15.169:0/2142356115 wait complete. 2023-10-26T01:30:57.602 INFO:teuthology.orchestra.run.smithi169.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2023-10-26T01:30:56.132776+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":1744,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560755,"statfs":{"total":1600000000,"available":1598213898,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004473"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.086425+0000","last_change":"2023-10-26T01:30:42.112552+0000","last_active":"2023-10-26T01:30:43.086425+0000","last_peered":"2023-10-26T01:30:43.086425+0000","last_clean":"2023-10-26T01:30:43.086425+0000","last_became_active":"2023-10-26T01:30:42.111151+0000","last_became_peered":"2023-10-26T01:30:42.111151+0000","last_unstale":"2023-10-26T01:30:43.086425+0000","last_undegraded":"2023-10-26T01:30:43.086425+0000","last_fullsized":"2023-10-26T01:30:43.086425+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:28:22.384005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.141033+0000","last_change":"2023-10-26T01:30:42.102872+0000","last_active":"2023-10-26T01:30:43.141033+0000","last_peered":"2023-10-26T01:30:43.141033+0000","last_clean":"2023-10-26T01:30:43.141033+0000","last_became_active":"2023-10-26T01:30:42.100796+0000","last_became_peered":"2023-10-26T01:30:42.100796+0000","last_unstale":"2023-10-26T01:30:43.141033+0000","last_undegraded":"2023-10-26T01:30:43.141033+0000","last_fullsized":"2023-10-26T01:30:43.141033+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:49:12.753504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.139076+0000","last_change":"2023-10-26T01:30:42.103921+0000","last_active":"2023-10-26T01:30:43.139076+0000","last_peered":"2023-10-26T01:30:43.139076+0000","last_clean":"2023-10-26T01:30:43.139076+0000","last_became_active":"2023-10-26T01:30:42.103384+0000","last_became_peered":"2023-10-26T01:30:42.103384+0000","last_unstale":"2023-10-26T01:30:43.139076+0000","last_undegraded":"2023-10-26T01:30:43.139076+0000","last_fullsized":"2023-10-26T01:30:43.139076+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:33:58.768305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.098597+0000","last_change":"2023-10-26T01:30:42.101845+0000","last_active":"2023-10-26T01:30:43.098597+0000","last_peered":"2023-10-26T01:30:43.098597+0000","last_clean":"2023-10-26T01:30:43.098597+0000","last_became_active":"2023-10-26T01:30:42.097125+0000","last_became_peered":"2023-10-26T01:30:42.097125+0000","last_unstale":"2023-10-26T01:30:43.098597+0000","last_undegraded":"2023-10-26T01:30:43.098597+0000","last_fullsized":"2023-10-26T01:30:43.098597+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:33:13.615628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.086419+0000","last_change":"2023-10-26T01:30:42.115244+0000","last_active":"2023-10-26T01:30:43.086419+0000","last_peered":"2023-10-26T01:30:43.086419+0000","last_clean":"2023-10-26T01:30:43.086419+0000","last_became_active":"2023-10-26T01:30:42.114972+0000","last_became_peered":"2023-10-26T01:30:42.114972+0000","last_unstale":"2023-10-26T01:30:43.086419+0000","last_undegraded":"2023-10-26T01:30:43.086419+0000","last_fullsized":"2023-10-26T01:30:43.086419+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:07:06.450331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.099993+0000","last_change":"2023-10-26T01:30:42.108711+0000","last_active":"2023-10-26T01:30:43.099993+0000","last_peered":"2023-10-26T01:30:43.099993+0000","last_clean":"2023-10-26T01:30:43.099993+0000","last_became_active":"2023-10-26T01:30:42.108215+0000","last_became_peered":"2023-10-26T01:30:42.108215+0000","last_unstale":"2023-10-26T01:30:43.099993+0000","last_undegraded":"2023-10-26T01:30:43.099993+0000","last_fullsized":"2023-10-26T01:30:43.099993+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:26:01.898201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.088319+0000","last_change":"2023-10-26T01:30:42.115268+0000","last_active":"2023-10-26T01:30:43.088319+0000","last_peered":"2023-10-26T01:30:43.088319+0000","last_clean":"2023-10-26T01:30:43.088319+0000","last_became_active":"2023-10-26T01:30:42.113208+0000","last_became_peered":"2023-10-26T01:30:42.113208+0000","last_unstale":"2023-10-26T01:30:43.088319+0000","last_undegraded":"2023-10-26T01:30:43.088319+0000","last_fullsized":"2023-10-26T01:30:43.088319+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:56:55.394232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.084344+0000","last_change":"2023-10-26T01:30:42.093950+0000","last_active":"2023-10-26T01:30:43.084344+0000","last_peered":"2023-10-26T01:30:43.084344+0000","last_clean":"2023-10-26T01:30:43.084344+0000","last_became_active":"2023-10-26T01:30:42.093684+0000","last_became_peered":"2023-10-26T01:30:42.093684+0000","last_unstale":"2023-10-26T01:30:43.084344+0000","last_undegraded":"2023-10-26T01:30:43.084344+0000","last_fullsized":"2023-10-26T01:30:43.084344+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:05:31.590291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.086093+0000","last_change":"2023-10-26T01:30:42.108451+0000","last_active":"2023-10-26T01:30:43.086093+0000","last_peered":"2023-10-26T01:30:43.086093+0000","last_clean":"2023-10-26T01:30:43.086093+0000","last_became_active":"2023-10-26T01:30:42.107650+0000","last_became_peered":"2023-10-26T01:30:42.107650+0000","last_unstale":"2023-10-26T01:30:43.086093+0000","last_undegraded":"2023-10-26T01:30:43.086093+0000","last_fullsized":"2023-10-26T01:30:43.086093+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:26:19.035685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.098895+0000","last_change":"2023-10-26T01:30:42.093838+0000","last_active":"2023-10-26T01:30:43.098895+0000","last_peered":"2023-10-26T01:30:43.098895+0000","last_clean":"2023-10-26T01:30:43.098895+0000","last_became_active":"2023-10-26T01:30:42.093584+0000","last_became_peered":"2023-10-26T01:30:42.093584+0000","last_unstale":"2023-10-26T01:30:43.098895+0000","last_undegraded":"2023-10-26T01:30:43.098895+0000","last_fullsized":"2023-10-26T01:30:43.098895+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:49:12.753504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.097658+0000","last_change":"2023-10-26T01:30:42.094026+0000","last_active":"2023-10-26T01:30:43.097658+0000","last_peered":"2023-10-26T01:30:43.097658+0000","last_clean":"2023-10-26T01:30:43.097658+0000","last_became_active":"2023-10-26T01:30:42.093654+0000","last_became_peered":"2023-10-26T01:30:42.093654+0000","last_unstale":"2023-10-26T01:30:43.097658+0000","last_undegraded":"2023-10-26T01:30:43.097658+0000","last_fullsized":"2023-10-26T01:30:43.097658+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:05:31.590291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.088414+0000","last_change":"2023-10-26T01:30:42.099764+0000","last_active":"2023-10-26T01:30:43.088414+0000","last_peered":"2023-10-26T01:30:43.088414+0000","last_clean":"2023-10-26T01:30:43.088414+0000","last_became_active":"2023-10-26T01:30:42.098108+0000","last_became_peered":"2023-10-26T01:30:42.098108+0000","last_unstale":"2023-10-26T01:30:43.088414+0000","last_undegraded":"2023-10-26T01:30:43.088414+0000","last_fullsized":"2023-10-26T01:30:43.088414+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:08:21.563278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.085548+0000","last_change":"2023-10-26T01:30:42.097734+0000","last_active":"2023-10-26T01:30:43.085548+0000","last_peered":"2023-10-26T01:30:43.085548+0000","last_clean":"2023-10-26T01:30:43.085548+0000","last_became_active":"2023-10-26T01:30:42.097333+0000","last_became_peered":"2023-10-26T01:30:42.097333+0000","last_unstale":"2023-10-26T01:30:43.085548+0000","last_undegraded":"2023-10-26T01:30:43.085548+0000","last_fullsized":"2023-10-26T01:30:43.085548+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:28:22.384005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.138887+0000","last_change":"2023-10-26T01:30:42.108715+0000","last_active":"2023-10-26T01:30:43.138887+0000","last_peered":"2023-10-26T01:30:43.138887+0000","last_clean":"2023-10-26T01:30:43.138887+0000","last_became_active":"2023-10-26T01:30:42.108469+0000","last_became_peered":"2023-10-26T01:30:42.108469+0000","last_unstale":"2023-10-26T01:30:43.138887+0000","last_undegraded":"2023-10-26T01:30:43.138887+0000","last_fullsized":"2023-10-26T01:30:43.138887+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:45:23.061469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.140916+0000","last_change":"2023-10-26T01:30:42.099157+0000","last_active":"2023-10-26T01:30:43.140916+0000","last_peered":"2023-10-26T01:30:43.140916+0000","last_clean":"2023-10-26T01:30:43.140916+0000","last_became_active":"2023-10-26T01:30:42.098929+0000","last_became_peered":"2023-10-26T01:30:42.098929+0000","last_unstale":"2023-10-26T01:30:43.140916+0000","last_undegraded":"2023-10-26T01:30:43.140916+0000","last_fullsized":"2023-10-26T01:30:43.140916+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:13:33.838018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.099835+0000","last_change":"2023-10-26T01:30:42.108578+0000","last_active":"2023-10-26T01:30:43.099835+0000","last_peered":"2023-10-26T01:30:43.099835+0000","last_clean":"2023-10-26T01:30:43.099835+0000","last_became_active":"2023-10-26T01:30:42.107937+0000","last_became_peered":"2023-10-26T01:30:42.107937+0000","last_unstale":"2023-10-26T01:30:43.099835+0000","last_undegraded":"2023-10-26T01:30:43.099835+0000","last_fullsized":"2023-10-26T01:30:43.099835+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:38:16.726919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.098559+0000","last_change":"2023-10-26T01:30:42.108373+0000","last_active":"2023-10-26T01:30:43.098559+0000","last_peered":"2023-10-26T01:30:43.098559+0000","last_clean":"2023-10-26T01:30:43.098559+0000","last_became_active":"2023-10-26T01:30:42.107773+0000","last_became_peered":"2023-10-26T01:30:42.107773+0000","last_unstale":"2023-10-26T01:30:43.098559+0000","last_undegraded":"2023-10-26T01:30:43.098559+0000","last_fullsized":"2023-10-26T01:30:43.098559+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:49:12.753504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.097381+0000","last_change":"2023-10-26T01:30:42.094993+0000","last_active":"2023-10-26T01:30:43.097381+0000","last_peered":"2023-10-26T01:30:43.097381+0000","last_clean":"2023-10-26T01:30:43.097381+0000","last_became_active":"2023-10-26T01:30:42.094689+0000","last_became_peered":"2023-10-26T01:30:42.094689+0000","last_unstale":"2023-10-26T01:30:43.097381+0000","last_undegraded":"2023-10-26T01:30:43.097381+0000","last_fullsized":"2023-10-26T01:30:43.097381+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:49:43.468558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.086773+0000","last_change":"2023-10-26T01:30:42.108746+0000","last_active":"2023-10-26T01:30:43.086773+0000","last_peered":"2023-10-26T01:30:43.086773+0000","last_clean":"2023-10-26T01:30:43.086773+0000","last_became_active":"2023-10-26T01:30:42.108188+0000","last_became_peered":"2023-10-26T01:30:42.108188+0000","last_unstale":"2023-10-26T01:30:43.086773+0000","last_undegraded":"2023-10-26T01:30:43.086773+0000","last_fullsized":"2023-10-26T01:30:43.086773+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:38:16.726919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.099061+0000","last_change":"2023-10-26T01:30:42.110331+0000","last_active":"2023-10-26T01:30:43.099061+0000","last_peered":"2023-10-26T01:30:43.099061+0000","last_clean":"2023-10-26T01:30:43.099061+0000","last_became_active":"2023-10-26T01:30:42.110079+0000","last_became_peered":"2023-10-26T01:30:42.110079+0000","last_unstale":"2023-10-26T01:30:43.099061+0000","last_undegraded":"2023-10-26T01:30:43.099061+0000","last_fullsized":"2023-10-26T01:30:43.099061+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:33:13.615628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.139607+0000","last_change":"2023-10-26T01:30:42.108608+0000","last_active":"2023-10-26T01:30:43.139607+0000","last_peered":"2023-10-26T01:30:43.139607+0000","last_clean":"2023-10-26T01:30:43.139607+0000","last_became_active":"2023-10-26T01:30:42.108197+0000","last_became_peered":"2023-10-26T01:30:42.108197+0000","last_unstale":"2023-10-26T01:30:43.139607+0000","last_undegraded":"2023-10-26T01:30:43.139607+0000","last_fullsized":"2023-10-26T01:30:43.139607+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:17:59.186810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.098944+0000","last_change":"2023-10-26T01:30:42.110289+0000","last_active":"2023-10-26T01:30:43.098944+0000","last_peered":"2023-10-26T01:30:43.098944+0000","last_clean":"2023-10-26T01:30:43.098944+0000","last_became_active":"2023-10-26T01:30:42.110061+0000","last_became_peered":"2023-10-26T01:30:42.110061+0000","last_unstale":"2023-10-26T01:30:43.098944+0000","last_undegraded":"2023-10-26T01:30:43.098944+0000","last_fullsized":"2023-10-26T01:30:43.098944+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:19:23.544531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.084819+0000","last_change":"2023-10-26T01:30:42.108686+0000","last_active":"2023-10-26T01:30:43.084819+0000","last_peered":"2023-10-26T01:30:43.084819+0000","last_clean":"2023-10-26T01:30:43.084819+0000","last_became_active":"2023-10-26T01:30:42.108136+0000","last_became_peered":"2023-10-26T01:30:42.108136+0000","last_unstale":"2023-10-26T01:30:43.084819+0000","last_undegraded":"2023-10-26T01:30:43.084819+0000","last_fullsized":"2023-10-26T01:30:43.084819+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:13:33.838018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.098984+0000","last_change":"2023-10-26T01:30:42.102147+0000","last_active":"2023-10-26T01:30:43.098984+0000","last_peered":"2023-10-26T01:30:43.098984+0000","last_clean":"2023-10-26T01:30:43.098984+0000","last_became_active":"2023-10-26T01:30:42.097509+0000","last_became_peered":"2023-10-26T01:30:42.097509+0000","last_unstale":"2023-10-26T01:30:43.098984+0000","last_undegraded":"2023-10-26T01:30:43.098984+0000","last_fullsized":"2023-10-26T01:30:43.098984+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:26:19.035685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.085124+0000","last_change":"2023-10-26T01:30:42.108519+0000","last_active":"2023-10-26T01:30:43.085124+0000","last_peered":"2023-10-26T01:30:43.085124+0000","last_clean":"2023-10-26T01:30:43.085124+0000","last_became_active":"2023-10-26T01:30:42.108283+0000","last_became_peered":"2023-10-26T01:30:42.108283+0000","last_unstale":"2023-10-26T01:30:43.085124+0000","last_undegraded":"2023-10-26T01:30:43.085124+0000","last_fullsized":"2023-10-26T01:30:43.085124+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:33:13.615628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.097862+0000","last_change":"2023-10-26T01:30:42.111865+0000","last_active":"2023-10-26T01:30:43.097862+0000","last_peered":"2023-10-26T01:30:43.097862+0000","last_clean":"2023-10-26T01:30:43.097862+0000","last_became_active":"2023-10-26T01:30:42.111636+0000","last_became_peered":"2023-10-26T01:30:42.111636+0000","last_unstale":"2023-10-26T01:30:43.097862+0000","last_undegraded":"2023-10-26T01:30:43.097862+0000","last_fullsized":"2023-10-26T01:30:43.097862+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:49:56.641022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.141348+0000","last_change":"2023-10-26T01:30:42.091332+0000","last_active":"2023-10-26T01:30:43.141348+0000","last_peered":"2023-10-26T01:30:43.141348+0000","last_clean":"2023-10-26T01:30:43.141348+0000","last_became_active":"2023-10-26T01:30:42.091105+0000","last_became_peered":"2023-10-26T01:30:42.091105+0000","last_unstale":"2023-10-26T01:30:43.141348+0000","last_undegraded":"2023-10-26T01:30:43.141348+0000","last_fullsized":"2023-10-26T01:30:43.141348+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:33:13.615628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.085113+0000","last_change":"2023-10-26T01:30:42.108822+0000","last_active":"2023-10-26T01:30:43.085113+0000","last_peered":"2023-10-26T01:30:43.085113+0000","last_clean":"2023-10-26T01:30:43.085113+0000","last_became_active":"2023-10-26T01:30:42.108448+0000","last_became_peered":"2023-10-26T01:30:42.108448+0000","last_unstale":"2023-10-26T01:30:43.085113+0000","last_undegraded":"2023-10-26T01:30:43.085113+0000","last_fullsized":"2023-10-26T01:30:43.085113+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:49:12.753504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.139200+0000","last_change":"2023-10-26T01:30:42.101708+0000","last_active":"2023-10-26T01:30:43.139200+0000","last_peered":"2023-10-26T01:30:43.139200+0000","last_clean":"2023-10-26T01:30:43.139200+0000","last_became_active":"2023-10-26T01:30:42.101434+0000","last_became_peered":"2023-10-26T01:30:42.101434+0000","last_unstale":"2023-10-26T01:30:43.139200+0000","last_undegraded":"2023-10-26T01:30:43.139200+0000","last_fullsized":"2023-10-26T01:30:43.139200+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:29:18.643148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.099207+0000","last_change":"2023-10-26T01:30:42.097495+0000","last_active":"2023-10-26T01:30:43.099207+0000","last_peered":"2023-10-26T01:30:43.099207+0000","last_clean":"2023-10-26T01:30:43.099207+0000","last_became_active":"2023-10-26T01:30:42.097214+0000","last_became_peered":"2023-10-26T01:30:42.097214+0000","last_unstale":"2023-10-26T01:30:43.099207+0000","last_undegraded":"2023-10-26T01:30:43.099207+0000","last_fullsized":"2023-10-26T01:30:43.099207+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T01:55:28.470913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.097481+0000","last_change":"2023-10-26T01:30:42.093157+0000","last_active":"2023-10-26T01:30:43.097481+0000","last_peered":"2023-10-26T01:30:43.097481+0000","last_clean":"2023-10-26T01:30:43.097481+0000","last_became_active":"2023-10-26T01:30:42.092613+0000","last_became_peered":"2023-10-26T01:30:42.092613+0000","last_unstale":"2023-10-26T01:30:43.097481+0000","last_undegraded":"2023-10-26T01:30:43.097481+0000","last_fullsized":"2023-10-26T01:30:43.097481+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:09:01.061004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.098226+0000","last_change":"2023-10-26T01:30:42.102704+0000","last_active":"2023-10-26T01:30:43.098226+0000","last_peered":"2023-10-26T01:30:43.098226+0000","last_clean":"2023-10-26T01:30:43.098226+0000","last_became_active":"2023-10-26T01:30:42.102350+0000","last_became_peered":"2023-10-26T01:30:42.102350+0000","last_unstale":"2023-10-26T01:30:43.098226+0000","last_undegraded":"2023-10-26T01:30:43.098226+0000","last_fullsized":"2023-10-26T01:30:43.098226+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:53:57.632838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.086493+0000","last_change":"2023-10-26T01:30:42.093818+0000","last_active":"2023-10-26T01:30:43.086493+0000","last_peered":"2023-10-26T01:30:43.086493+0000","last_clean":"2023-10-26T01:30:43.086493+0000","last_became_active":"2023-10-26T01:30:42.091310+0000","last_became_peered":"2023-10-26T01:30:42.091310+0000","last_unstale":"2023-10-26T01:30:43.086493+0000","last_undegraded":"2023-10-26T01:30:43.086493+0000","last_fullsized":"2023-10-26T01:30:43.086493+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:04:23.158155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.114911+0000","last_change":"2023-10-26T01:30:41.097666+0000","last_active":"2023-10-26T01:30:42.114911+0000","last_peered":"2023-10-26T01:30:42.114911+0000","last_clean":"2023-10-26T01:30:42.114911+0000","last_became_active":"2023-10-26T01:30:41.093986+0000","last_became_peered":"2023-10-26T01:30:41.093986+0000","last_unstale":"2023-10-26T01:30:42.114911+0000","last_undegraded":"2023-10-26T01:30:42.114911+0000","last_fullsized":"2023-10-26T01:30:42.114911+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:49:42.452332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.139516+0000","last_change":"2023-10-26T01:30:42.110839+0000","last_active":"2023-10-26T01:30:43.139516+0000","last_peered":"2023-10-26T01:30:43.139516+0000","last_clean":"2023-10-26T01:30:43.139516+0000","last_became_active":"2023-10-26T01:30:42.109385+0000","last_became_peered":"2023-10-26T01:30:42.109385+0000","last_unstale":"2023-10-26T01:30:43.139516+0000","last_undegraded":"2023-10-26T01:30:43.139516+0000","last_fullsized":"2023-10-26T01:30:43.139516+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:49:56.641022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.105272+0000","last_change":"2023-10-26T01:30:41.100248+0000","last_active":"2023-10-26T01:30:42.105272+0000","last_peered":"2023-10-26T01:30:42.105272+0000","last_clean":"2023-10-26T01:30:42.105272+0000","last_became_active":"2023-10-26T01:30:41.099957+0000","last_became_peered":"2023-10-26T01:30:41.099957+0000","last_unstale":"2023-10-26T01:30:42.105272+0000","last_undegraded":"2023-10-26T01:30:42.105272+0000","last_fullsized":"2023-10-26T01:30:42.105272+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:29:17.626922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.088500+0000","last_change":"2023-10-26T01:30:41.103843+0000","last_active":"2023-10-26T01:30:42.088500+0000","last_peered":"2023-10-26T01:30:42.088500+0000","last_clean":"2023-10-26T01:30:42.088500+0000","last_became_active":"2023-10-26T01:30:41.103356+0000","last_became_peered":"2023-10-26T01:30:41.103356+0000","last_unstale":"2023-10-26T01:30:42.088500+0000","last_undegraded":"2023-10-26T01:30:42.088500+0000","last_fullsized":"2023-10-26T01:30:42.088500+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:40:15.199455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.085463+0000","last_change":"2023-10-26T01:30:42.102443+0000","last_active":"2023-10-26T01:30:43.085463+0000","last_peered":"2023-10-26T01:30:43.085463+0000","last_clean":"2023-10-26T01:30:43.085463+0000","last_became_active":"2023-10-26T01:30:42.102213+0000","last_became_peered":"2023-10-26T01:30:42.102213+0000","last_unstale":"2023-10-26T01:30:43.085463+0000","last_undegraded":"2023-10-26T01:30:43.085463+0000","last_fullsized":"2023-10-26T01:30:43.085463+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:26:19.035685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.089693+0000","last_change":"2023-10-26T01:30:41.102989+0000","last_active":"2023-10-26T01:30:42.089693+0000","last_peered":"2023-10-26T01:30:42.089693+0000","last_clean":"2023-10-26T01:30:42.089693+0000","last_became_active":"2023-10-26T01:30:41.102681+0000","last_became_peered":"2023-10-26T01:30:41.102681+0000","last_unstale":"2023-10-26T01:30:42.089693+0000","last_undegraded":"2023-10-26T01:30:42.089693+0000","last_fullsized":"2023-10-26T01:30:42.089693+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:49:55.624796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.140341+0000","last_change":"2023-10-26T01:30:42.091359+0000","last_active":"2023-10-26T01:30:43.140341+0000","last_peered":"2023-10-26T01:30:43.140341+0000","last_clean":"2023-10-26T01:30:43.140341+0000","last_became_active":"2023-10-26T01:30:42.090949+0000","last_became_peered":"2023-10-26T01:30:42.090949+0000","last_unstale":"2023-10-26T01:30:43.140341+0000","last_undegraded":"2023-10-26T01:30:43.140341+0000","last_fullsized":"2023-10-26T01:30:43.140341+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:49:56.641022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.089836+0000","last_change":"2023-10-26T01:30:41.104058+0000","last_active":"2023-10-26T01:30:42.089836+0000","last_peered":"2023-10-26T01:30:42.089836+0000","last_clean":"2023-10-26T01:30:42.089836+0000","last_became_active":"2023-10-26T01:30:41.103821+0000","last_became_peered":"2023-10-26T01:30:41.103821+0000","last_unstale":"2023-10-26T01:30:42.089836+0000","last_undegraded":"2023-10-26T01:30:42.089836+0000","last_fullsized":"2023-10-26T01:30:42.089836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:46:05.927163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.098774+0000","last_change":"2023-10-26T01:30:42.108520+0000","last_active":"2023-10-26T01:30:43.098774+0000","last_peered":"2023-10-26T01:30:43.098774+0000","last_clean":"2023-10-26T01:30:43.098774+0000","last_became_active":"2023-10-26T01:30:42.108140+0000","last_became_peered":"2023-10-26T01:30:42.108140+0000","last_unstale":"2023-10-26T01:30:43.098774+0000","last_undegraded":"2023-10-26T01:30:43.098774+0000","last_fullsized":"2023-10-26T01:30:43.098774+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:31:08.727626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.088126+0000","last_change":"2023-10-26T01:30:41.106779+0000","last_active":"2023-10-26T01:30:42.088126+0000","last_peered":"2023-10-26T01:30:42.088126+0000","last_clean":"2023-10-26T01:30:42.088126+0000","last_became_active":"2023-10-26T01:30:41.104353+0000","last_became_peered":"2023-10-26T01:30:41.104353+0000","last_unstale":"2023-10-26T01:30:42.088126+0000","last_undegraded":"2023-10-26T01:30:42.088126+0000","last_fullsized":"2023-10-26T01:30:42.088126+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:45:22.045243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.139887+0000","last_change":"2023-10-26T01:30:42.105447+0000","last_active":"2023-10-26T01:30:43.139887+0000","last_peered":"2023-10-26T01:30:43.139887+0000","last_clean":"2023-10-26T01:30:43.139887+0000","last_became_active":"2023-10-26T01:30:42.105098+0000","last_became_peered":"2023-10-26T01:30:42.105098+0000","last_unstale":"2023-10-26T01:30:43.139887+0000","last_undegraded":"2023-10-26T01:30:43.139887+0000","last_fullsized":"2023-10-26T01:30:43.139887+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:13:33.838018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.110602+0000","last_change":"2023-10-26T01:30:41.103826+0000","last_active":"2023-10-26T01:30:42.110602+0000","last_peered":"2023-10-26T01:30:42.110602+0000","last_clean":"2023-10-26T01:30:42.110602+0000","last_became_active":"2023-10-26T01:30:41.103552+0000","last_became_peered":"2023-10-26T01:30:41.103552+0000","last_unstale":"2023-10-26T01:30:42.110602+0000","last_undegraded":"2023-10-26T01:30:42.110602+0000","last_fullsized":"2023-10-26T01:30:42.110602+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:54:04.970133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.086144+0000","last_change":"2023-10-26T01:30:42.114167+0000","last_active":"2023-10-26T01:30:43.086144+0000","last_peered":"2023-10-26T01:30:43.086144+0000","last_clean":"2023-10-26T01:30:43.086144+0000","last_became_active":"2023-10-26T01:30:42.113241+0000","last_became_peered":"2023-10-26T01:30:42.113241+0000","last_unstale":"2023-10-26T01:30:43.086144+0000","last_undegraded":"2023-10-26T01:30:43.086144+0000","last_fullsized":"2023-10-26T01:30:43.086144+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:40:16.215681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.116049+0000","last_change":"2023-10-26T01:30:41.106514+0000","last_active":"2023-10-26T01:30:42.116049+0000","last_peered":"2023-10-26T01:30:42.116049+0000","last_clean":"2023-10-26T01:30:42.116049+0000","last_became_active":"2023-10-26T01:30:41.106147+0000","last_became_peered":"2023-10-26T01:30:41.106147+0000","last_unstale":"2023-10-26T01:30:42.116049+0000","last_undegraded":"2023-10-26T01:30:42.116049+0000","last_fullsized":"2023-10-26T01:30:42.116049+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:45:22.045243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.089920+0000","last_change":"2023-10-26T01:30:42.112813+0000","last_active":"2023-10-26T01:30:43.089920+0000","last_peered":"2023-10-26T01:30:43.089920+0000","last_clean":"2023-10-26T01:30:43.089920+0000","last_became_active":"2023-10-26T01:30:42.112571+0000","last_became_peered":"2023-10-26T01:30:42.112571+0000","last_unstale":"2023-10-26T01:30:43.089920+0000","last_undegraded":"2023-10-26T01:30:43.089920+0000","last_fullsized":"2023-10-26T01:30:43.089920+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:19:16.042472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.108474+0000","last_change":"2023-10-26T01:30:41.098865+0000","last_active":"2023-10-26T01:30:42.108474+0000","last_peered":"2023-10-26T01:30:42.108474+0000","last_clean":"2023-10-26T01:30:42.108474+0000","last_became_active":"2023-10-26T01:30:41.098506+0000","last_became_peered":"2023-10-26T01:30:41.098506+0000","last_unstale":"2023-10-26T01:30:42.108474+0000","last_undegraded":"2023-10-26T01:30:42.108474+0000","last_fullsized":"2023-10-26T01:30:42.108474+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:09:00.044778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.140188+0000","last_change":"2023-10-26T01:30:42.108443+0000","last_active":"2023-10-26T01:30:43.140188+0000","last_peered":"2023-10-26T01:30:43.140188+0000","last_clean":"2023-10-26T01:30:43.140188+0000","last_became_active":"2023-10-26T01:30:42.107837+0000","last_became_peered":"2023-10-26T01:30:42.107837+0000","last_unstale":"2023-10-26T01:30:43.140188+0000","last_undegraded":"2023-10-26T01:30:43.140188+0000","last_fullsized":"2023-10-26T01:30:43.140188+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:33:13.615628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.090761+0000","last_change":"2023-10-26T01:30:41.106795+0000","last_active":"2023-10-26T01:30:42.090761+0000","last_peered":"2023-10-26T01:30:42.090761+0000","last_clean":"2023-10-26T01:30:42.090761+0000","last_became_active":"2023-10-26T01:30:41.106222+0000","last_became_peered":"2023-10-26T01:30:41.106222+0000","last_unstale":"2023-10-26T01:30:42.090761+0000","last_undegraded":"2023-10-26T01:30:42.090761+0000","last_fullsized":"2023-10-26T01:30:42.090761+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:53:56.616612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.086822+0000","last_change":"2023-10-26T01:30:42.113053+0000","last_active":"2023-10-26T01:30:43.086822+0000","last_peered":"2023-10-26T01:30:43.086822+0000","last_clean":"2023-10-26T01:30:43.086822+0000","last_became_active":"2023-10-26T01:30:42.112492+0000","last_became_peered":"2023-10-26T01:30:42.112492+0000","last_unstale":"2023-10-26T01:30:43.086822+0000","last_undegraded":"2023-10-26T01:30:43.086822+0000","last_fullsized":"2023-10-26T01:30:43.086822+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:20:47.431800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.089525+0000","last_change":"2023-10-26T01:30:41.100835+0000","last_active":"2023-10-26T01:30:42.089525+0000","last_peered":"2023-10-26T01:30:42.089525+0000","last_clean":"2023-10-26T01:30:42.089525+0000","last_became_active":"2023-10-26T01:30:41.099708+0000","last_became_peered":"2023-10-26T01:30:41.099708+0000","last_unstale":"2023-10-26T01:30:42.089525+0000","last_undegraded":"2023-10-26T01:30:42.089525+0000","last_fullsized":"2023-10-26T01:30:42.089525+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:04:22.141930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.098337+0000","last_change":"2023-10-26T01:30:42.100857+0000","last_active":"2023-10-26T01:30:43.098337+0000","last_peered":"2023-10-26T01:30:43.098337+0000","last_clean":"2023-10-26T01:30:43.098337+0000","last_became_active":"2023-10-26T01:30:42.100545+0000","last_became_peered":"2023-10-26T01:30:42.100545+0000","last_unstale":"2023-10-26T01:30:43.098337+0000","last_undegraded":"2023-10-26T01:30:43.098337+0000","last_fullsized":"2023-10-26T01:30:43.098337+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:13:33.838018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.114399+0000","last_change":"2023-10-26T01:30:41.095998+0000","last_active":"2023-10-26T01:30:42.114399+0000","last_peered":"2023-10-26T01:30:42.114399+0000","last_clean":"2023-10-26T01:30:42.114399+0000","last_became_active":"2023-10-26T01:30:41.095494+0000","last_became_peered":"2023-10-26T01:30:41.095494+0000","last_unstale":"2023-10-26T01:30:42.114399+0000","last_undegraded":"2023-10-26T01:30:42.114399+0000","last_fullsized":"2023-10-26T01:30:42.114399+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:54:04.970133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.089073+0000","last_change":"2023-10-26T01:30:42.115510+0000","last_active":"2023-10-26T01:30:43.089073+0000","last_peered":"2023-10-26T01:30:43.089073+0000","last_clean":"2023-10-26T01:30:43.089073+0000","last_became_active":"2023-10-26T01:30:42.113552+0000","last_became_peered":"2023-10-26T01:30:42.113552+0000","last_unstale":"2023-10-26T01:30:43.089073+0000","last_undegraded":"2023-10-26T01:30:43.089073+0000","last_fullsized":"2023-10-26T01:30:43.089073+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:32:08.387966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.110967+0000","last_change":"2023-10-26T01:30:41.093200+0000","last_active":"2023-10-26T01:30:42.110967+0000","last_peered":"2023-10-26T01:30:42.110967+0000","last_clean":"2023-10-26T01:30:42.110967+0000","last_became_active":"2023-10-26T01:30:41.092917+0000","last_became_peered":"2023-10-26T01:30:41.092917+0000","last_unstale":"2023-10-26T01:30:42.110967+0000","last_undegraded":"2023-10-26T01:30:42.110967+0000","last_fullsized":"2023-10-26T01:30:42.110967+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:54:04.970133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.087143+0000","last_change":"2023-10-26T01:30:42.113216+0000","last_active":"2023-10-26T01:30:43.087143+0000","last_peered":"2023-10-26T01:30:43.087143+0000","last_clean":"2023-10-26T01:30:43.087143+0000","last_became_active":"2023-10-26T01:30:42.112815+0000","last_became_peered":"2023-10-26T01:30:42.112815+0000","last_unstale":"2023-10-26T01:30:43.087143+0000","last_undegraded":"2023-10-26T01:30:43.087143+0000","last_fullsized":"2023-10-26T01:30:43.087143+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:55:51.116465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.114880+0000","last_change":"2023-10-26T01:30:41.102365+0000","last_active":"2023-10-26T01:30:42.114880+0000","last_peered":"2023-10-26T01:30:42.114880+0000","last_clean":"2023-10-26T01:30:42.114880+0000","last_became_active":"2023-10-26T01:30:41.102135+0000","last_became_peered":"2023-10-26T01:30:41.102135+0000","last_unstale":"2023-10-26T01:30:42.114880+0000","last_undegraded":"2023-10-26T01:30:42.114880+0000","last_fullsized":"2023-10-26T01:30:42.114880+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:08:20.547052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.086427+0000","last_change":"2023-10-26T01:30:42.108605+0000","last_active":"2023-10-26T01:30:43.086427+0000","last_peered":"2023-10-26T01:30:43.086427+0000","last_clean":"2023-10-26T01:30:43.086427+0000","last_became_active":"2023-10-26T01:30:42.107907+0000","last_became_peered":"2023-10-26T01:30:42.107907+0000","last_unstale":"2023-10-26T01:30:43.086427+0000","last_undegraded":"2023-10-26T01:30:43.086427+0000","last_fullsized":"2023-10-26T01:30:43.086427+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T01:55:28.470913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.105745+0000","last_change":"2023-10-26T01:30:41.093456+0000","last_active":"2023-10-26T01:30:42.105745+0000","last_peered":"2023-10-26T01:30:42.105745+0000","last_clean":"2023-10-26T01:30:42.105745+0000","last_became_active":"2023-10-26T01:30:41.092811+0000","last_became_peered":"2023-10-26T01:30:41.092811+0000","last_unstale":"2023-10-26T01:30:42.105745+0000","last_undegraded":"2023-10-26T01:30:42.105745+0000","last_fullsized":"2023-10-26T01:30:42.105745+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:04:22.141930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.099118+0000","last_change":"2023-10-26T01:30:42.097121+0000","last_active":"2023-10-26T01:30:43.099118+0000","last_peered":"2023-10-26T01:30:43.099118+0000","last_clean":"2023-10-26T01:30:43.099118+0000","last_became_active":"2023-10-26T01:30:42.096784+0000","last_became_peered":"2023-10-26T01:30:42.096784+0000","last_unstale":"2023-10-26T01:30:43.099118+0000","last_undegraded":"2023-10-26T01:30:43.099118+0000","last_fullsized":"2023-10-26T01:30:43.099118+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:28:22.384005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.083700+0000","last_change":"2023-10-26T01:30:41.106632+0000","last_active":"2023-10-26T01:30:42.083700+0000","last_peered":"2023-10-26T01:30:42.083700+0000","last_clean":"2023-10-26T01:30:42.083700+0000","last_became_active":"2023-10-26T01:30:41.106360+0000","last_became_peered":"2023-10-26T01:30:41.106360+0000","last_unstale":"2023-10-26T01:30:42.083700+0000","last_undegraded":"2023-10-26T01:30:42.083700+0000","last_fullsized":"2023-10-26T01:30:42.083700+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:40:15.199455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.086951+0000","last_change":"2023-10-26T01:30:42.109475+0000","last_active":"2023-10-26T01:30:43.086951+0000","last_peered":"2023-10-26T01:30:43.086951+0000","last_clean":"2023-10-26T01:30:43.086951+0000","last_became_active":"2023-10-26T01:30:42.109232+0000","last_became_peered":"2023-10-26T01:30:42.109232+0000","last_unstale":"2023-10-26T01:30:43.086951+0000","last_undegraded":"2023-10-26T01:30:43.086951+0000","last_fullsized":"2023-10-26T01:30:43.086951+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:26:01.898201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":165,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.096979+0000","last_change":"2023-10-26T01:30:34.883484+0000","last_active":"2023-10-26T01:30:42.096979+0000","last_peered":"2023-10-26T01:30:42.096979+0000","last_clean":"2023-10-26T01:30:42.096979+0000","last_became_active":"2023-10-26T01:30:34.883071+0000","last_became_peered":"2023-10-26T01:30:34.883071+0000","last_unstale":"2023-10-26T01:30:42.096979+0000","last_undegraded":"2023-10-26T01:30:42.096979+0000","last_fullsized":"2023-10-26T01:30:42.096979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:32.890096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:32.890096+0000","last_clean_scrub_stamp":"2023-10-26T01:30:32.890096+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:22:57.937891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.090888+0000","last_change":"2023-10-26T01:30:41.105687+0000","last_active":"2023-10-26T01:30:42.090888+0000","last_peered":"2023-10-26T01:30:42.090888+0000","last_clean":"2023-10-26T01:30:42.090888+0000","last_became_active":"2023-10-26T01:30:41.105235+0000","last_became_peered":"2023-10-26T01:30:41.105235+0000","last_unstale":"2023-10-26T01:30:42.090888+0000","last_undegraded":"2023-10-26T01:30:42.090888+0000","last_fullsized":"2023-10-26T01:30:42.090888+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:13:32.821792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.084569+0000","last_change":"2023-10-26T01:30:42.093817+0000","last_active":"2023-10-26T01:30:43.084569+0000","last_peered":"2023-10-26T01:30:43.084569+0000","last_clean":"2023-10-26T01:30:43.084569+0000","last_became_active":"2023-10-26T01:30:42.093586+0000","last_became_peered":"2023-10-26T01:30:42.093586+0000","last_unstale":"2023-10-26T01:30:43.084569+0000","last_undegraded":"2023-10-26T01:30:43.084569+0000","last_fullsized":"2023-10-26T01:30:43.084569+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:17:59.186810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.096890+0000","last_change":"2023-10-26T01:30:41.094664+0000","last_active":"2023-10-26T01:30:42.096890+0000","last_peered":"2023-10-26T01:30:42.096890+0000","last_clean":"2023-10-26T01:30:42.096890+0000","last_became_active":"2023-10-26T01:30:41.092745+0000","last_became_peered":"2023-10-26T01:30:41.092745+0000","last_unstale":"2023-10-26T01:30:42.096890+0000","last_undegraded":"2023-10-26T01:30:42.096890+0000","last_fullsized":"2023-10-26T01:30:42.096890+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:45:22.045243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.140108+0000","last_change":"2023-10-26T01:30:42.107538+0000","last_active":"2023-10-26T01:30:43.140108+0000","last_peered":"2023-10-26T01:30:43.140108+0000","last_clean":"2023-10-26T01:30:43.140108+0000","last_became_active":"2023-10-26T01:30:42.107258+0000","last_became_peered":"2023-10-26T01:30:42.107258+0000","last_unstale":"2023-10-26T01:30:43.140108+0000","last_undegraded":"2023-10-26T01:30:43.140108+0000","last_fullsized":"2023-10-26T01:30:43.140108+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:49:12.753504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.083749+0000","last_change":"2023-10-26T01:30:41.099543+0000","last_active":"2023-10-26T01:30:42.083749+0000","last_peered":"2023-10-26T01:30:42.083749+0000","last_clean":"2023-10-26T01:30:42.083749+0000","last_became_active":"2023-10-26T01:30:41.099277+0000","last_became_peered":"2023-10-26T01:30:41.099277+0000","last_unstale":"2023-10-26T01:30:42.083749+0000","last_undegraded":"2023-10-26T01:30:42.083749+0000","last_fullsized":"2023-10-26T01:30:42.083749+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:04:22.141930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.140694+0000","last_change":"2023-10-26T01:30:42.085871+0000","last_active":"2023-10-26T01:30:43.140694+0000","last_peered":"2023-10-26T01:30:43.140694+0000","last_clean":"2023-10-26T01:30:43.140694+0000","last_became_active":"2023-10-26T01:30:42.085409+0000","last_became_peered":"2023-10-26T01:30:42.085409+0000","last_unstale":"2023-10-26T01:30:43.140694+0000","last_undegraded":"2023-10-26T01:30:43.140694+0000","last_fullsized":"2023-10-26T01:30:43.140694+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:17:59.186810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.088251+0000","last_change":"2023-10-26T01:30:41.107609+0000","last_active":"2023-10-26T01:30:42.088251+0000","last_peered":"2023-10-26T01:30:42.088251+0000","last_clean":"2023-10-26T01:30:42.088251+0000","last_became_active":"2023-10-26T01:30:41.107189+0000","last_became_peered":"2023-10-26T01:30:41.107189+0000","last_unstale":"2023-10-26T01:30:42.088251+0000","last_undegraded":"2023-10-26T01:30:42.088251+0000","last_fullsized":"2023-10-26T01:30:42.088251+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:33:57.752079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.088414+0000","last_change":"2023-10-26T01:30:42.105495+0000","last_active":"2023-10-26T01:30:43.088414+0000","last_peered":"2023-10-26T01:30:43.088414+0000","last_clean":"2023-10-26T01:30:43.088414+0000","last_became_active":"2023-10-26T01:30:42.105247+0000","last_became_peered":"2023-10-26T01:30:42.105247+0000","last_unstale":"2023-10-26T01:30:43.088414+0000","last_undegraded":"2023-10-26T01:30:43.088414+0000","last_fullsized":"2023-10-26T01:30:43.088414+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:24:17.679373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.088367+0000","last_change":"2023-10-26T01:30:41.096866+0000","last_active":"2023-10-26T01:30:42.088367+0000","last_peered":"2023-10-26T01:30:42.088367+0000","last_clean":"2023-10-26T01:30:42.088367+0000","last_became_active":"2023-10-26T01:30:41.096255+0000","last_became_peered":"2023-10-26T01:30:41.096255+0000","last_unstale":"2023-10-26T01:30:42.088367+0000","last_undegraded":"2023-10-26T01:30:42.088367+0000","last_fullsized":"2023-10-26T01:30:42.088367+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:29:17.626922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.099320+0000","last_change":"2023-10-26T01:30:42.102298+0000","last_active":"2023-10-26T01:30:43.099320+0000","last_peered":"2023-10-26T01:30:43.099320+0000","last_clean":"2023-10-26T01:30:43.099320+0000","last_became_active":"2023-10-26T01:30:42.097872+0000","last_became_peered":"2023-10-26T01:30:42.097872+0000","last_unstale":"2023-10-26T01:30:43.099320+0000","last_undegraded":"2023-10-26T01:30:43.099320+0000","last_fullsized":"2023-10-26T01:30:43.099320+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:07:06.450331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.096762+0000","last_change":"2023-10-26T01:30:41.096310+0000","last_active":"2023-10-26T01:30:42.096762+0000","last_peered":"2023-10-26T01:30:42.096762+0000","last_clean":"2023-10-26T01:30:42.096762+0000","last_became_active":"2023-10-26T01:30:41.095929+0000","last_became_peered":"2023-10-26T01:30:41.095929+0000","last_unstale":"2023-10-26T01:30:42.096762+0000","last_undegraded":"2023-10-26T01:30:42.096762+0000","last_fullsized":"2023-10-26T01:30:42.096762+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:31:39.970444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.140793+0000","last_change":"2023-10-26T01:30:42.089331+0000","last_active":"2023-10-26T01:30:43.140793+0000","last_peered":"2023-10-26T01:30:43.140793+0000","last_clean":"2023-10-26T01:30:43.140793+0000","last_became_active":"2023-10-26T01:30:42.088901+0000","last_became_peered":"2023-10-26T01:30:42.088901+0000","last_unstale":"2023-10-26T01:30:43.140793+0000","last_undegraded":"2023-10-26T01:30:43.140793+0000","last_fullsized":"2023-10-26T01:30:43.140793+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:53:57.632838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.140717+0000","last_change":"2023-10-26T01:30:41.102617+0000","last_active":"2023-10-26T01:30:43.140717+0000","last_peered":"2023-10-26T01:30:43.140717+0000","last_clean":"2023-10-26T01:30:43.140717+0000","last_became_active":"2023-10-26T01:30:41.102376+0000","last_became_peered":"2023-10-26T01:30:41.102376+0000","last_unstale":"2023-10-26T01:30:43.140717+0000","last_undegraded":"2023-10-26T01:30:43.140717+0000","last_fullsized":"2023-10-26T01:30:43.140717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:24:16.663147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.084424+0000","last_change":"2023-10-26T01:30:42.112237+0000","last_active":"2023-10-26T01:30:43.084424+0000","last_peered":"2023-10-26T01:30:43.084424+0000","last_clean":"2023-10-26T01:30:43.084424+0000","last_became_active":"2023-10-26T01:30:42.112006+0000","last_became_peered":"2023-10-26T01:30:42.112006+0000","last_unstale":"2023-10-26T01:30:43.084424+0000","last_undegraded":"2023-10-26T01:30:43.084424+0000","last_fullsized":"2023-10-26T01:30:43.084424+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:49:56.641022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.112282+0000","last_change":"2023-10-26T01:30:41.102404+0000","last_active":"2023-10-26T01:30:42.112282+0000","last_peered":"2023-10-26T01:30:42.112282+0000","last_clean":"2023-10-26T01:30:42.112282+0000","last_became_active":"2023-10-26T01:30:41.102165+0000","last_became_peered":"2023-10-26T01:30:41.102165+0000","last_unstale":"2023-10-26T01:30:42.112282+0000","last_undegraded":"2023-10-26T01:30:42.112282+0000","last_fullsized":"2023-10-26T01:30:42.112282+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:08:20.547052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.139422+0000","last_change":"2023-10-26T01:30:42.103630+0000","last_active":"2023-10-26T01:30:43.139422+0000","last_peered":"2023-10-26T01:30:43.139422+0000","last_clean":"2023-10-26T01:30:43.139422+0000","last_became_active":"2023-10-26T01:30:42.099458+0000","last_became_peered":"2023-10-26T01:30:42.099458+0000","last_unstale":"2023-10-26T01:30:43.139422+0000","last_undegraded":"2023-10-26T01:30:43.139422+0000","last_fullsized":"2023-10-26T01:30:43.139422+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:05:31.590291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.097650+0000","last_change":"2023-10-26T01:30:41.097301+0000","last_active":"2023-10-26T01:30:42.097650+0000","last_peered":"2023-10-26T01:30:42.097650+0000","last_clean":"2023-10-26T01:30:42.097650+0000","last_became_active":"2023-10-26T01:30:41.096866+0000","last_became_peered":"2023-10-26T01:30:41.096866+0000","last_unstale":"2023-10-26T01:30:42.097650+0000","last_undegraded":"2023-10-26T01:30:42.097650+0000","last_fullsized":"2023-10-26T01:30:42.097650+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:05:30.574065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.089679+0000","last_change":"2023-10-26T01:30:42.087483+0000","last_active":"2023-10-26T01:30:43.089679+0000","last_peered":"2023-10-26T01:30:43.089679+0000","last_clean":"2023-10-26T01:30:43.089679+0000","last_became_active":"2023-10-26T01:30:42.087234+0000","last_became_peered":"2023-10-26T01:30:42.087234+0000","last_unstale":"2023-10-26T01:30:43.089679+0000","last_undegraded":"2023-10-26T01:30:43.089679+0000","last_fullsized":"2023-10-26T01:30:43.089679+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:46:06.943389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.088240+0000","last_change":"2023-10-26T01:30:41.097032+0000","last_active":"2023-10-26T01:30:42.088240+0000","last_peered":"2023-10-26T01:30:42.088240+0000","last_clean":"2023-10-26T01:30:42.088240+0000","last_became_active":"2023-10-26T01:30:41.096591+0000","last_became_peered":"2023-10-26T01:30:41.096591+0000","last_unstale":"2023-10-26T01:30:42.088240+0000","last_undegraded":"2023-10-26T01:30:42.088240+0000","last_fullsized":"2023-10-26T01:30:42.088240+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:33:57.752079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.084667+0000","last_change":"2023-10-26T01:30:42.101380+0000","last_active":"2023-10-26T01:30:43.084667+0000","last_peered":"2023-10-26T01:30:43.084667+0000","last_clean":"2023-10-26T01:30:43.084667+0000","last_became_active":"2023-10-26T01:30:42.101179+0000","last_became_peered":"2023-10-26T01:30:42.101179+0000","last_unstale":"2023-10-26T01:30:43.084667+0000","last_undegraded":"2023-10-26T01:30:43.084667+0000","last_fullsized":"2023-10-26T01:30:43.084667+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:53:57.632838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.084678+0000","last_change":"2023-10-26T01:30:41.101587+0000","last_active":"2023-10-26T01:30:42.084678+0000","last_peered":"2023-10-26T01:30:42.084678+0000","last_clean":"2023-10-26T01:30:42.084678+0000","last_became_active":"2023-10-26T01:30:41.101325+0000","last_became_peered":"2023-10-26T01:30:41.101325+0000","last_unstale":"2023-10-26T01:30:42.084678+0000","last_undegraded":"2023-10-26T01:30:42.084678+0000","last_fullsized":"2023-10-26T01:30:42.084678+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:49:42.452332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.085284+0000","last_change":"2023-10-26T01:30:42.103918+0000","last_active":"2023-10-26T01:30:43.085284+0000","last_peered":"2023-10-26T01:30:43.085284+0000","last_clean":"2023-10-26T01:30:43.085284+0000","last_became_active":"2023-10-26T01:30:42.103701+0000","last_became_peered":"2023-10-26T01:30:42.103701+0000","last_unstale":"2023-10-26T01:30:43.085284+0000","last_undegraded":"2023-10-26T01:30:43.085284+0000","last_fullsized":"2023-10-26T01:30:43.085284+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:19:23.544531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.114450+0000","last_change":"2023-10-26T01:30:41.095274+0000","last_active":"2023-10-26T01:30:42.114450+0000","last_peered":"2023-10-26T01:30:42.114450+0000","last_clean":"2023-10-26T01:30:42.114450+0000","last_became_active":"2023-10-26T01:30:41.092922+0000","last_became_peered":"2023-10-26T01:30:41.092922+0000","last_unstale":"2023-10-26T01:30:42.114450+0000","last_undegraded":"2023-10-26T01:30:42.114450+0000","last_fullsized":"2023-10-26T01:30:42.114450+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:33:12.599402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.098208+0000","last_change":"2023-10-26T01:30:42.105139+0000","last_active":"2023-10-26T01:30:43.098208+0000","last_peered":"2023-10-26T01:30:43.098208+0000","last_clean":"2023-10-26T01:30:43.098208+0000","last_became_active":"2023-10-26T01:30:42.104913+0000","last_became_peered":"2023-10-26T01:30:42.104913+0000","last_unstale":"2023-10-26T01:30:43.098208+0000","last_undegraded":"2023-10-26T01:30:43.098208+0000","last_fullsized":"2023-10-26T01:30:43.098208+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:17:59.186810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.099380+0000","last_change":"2023-10-26T01:30:42.106926+0000","last_active":"2023-10-26T01:30:43.099380+0000","last_peered":"2023-10-26T01:30:43.099380+0000","last_clean":"2023-10-26T01:30:43.099380+0000","last_became_active":"2023-10-26T01:30:42.106692+0000","last_became_peered":"2023-10-26T01:30:42.106692+0000","last_unstale":"2023-10-26T01:30:43.099380+0000","last_undegraded":"2023-10-26T01:30:43.099380+0000","last_fullsized":"2023-10-26T01:30:43.099380+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:20:47.431800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.110477+0000","last_change":"2023-10-26T01:30:41.105282+0000","last_active":"2023-10-26T01:30:42.110477+0000","last_peered":"2023-10-26T01:30:42.110477+0000","last_clean":"2023-10-26T01:30:42.110477+0000","last_became_active":"2023-10-26T01:30:41.104553+0000","last_became_peered":"2023-10-26T01:30:41.104553+0000","last_unstale":"2023-10-26T01:30:42.110477+0000","last_undegraded":"2023-10-26T01:30:42.110477+0000","last_fullsized":"2023-10-26T01:30:42.110477+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:29:17.626922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.100164+0000","last_change":"2023-10-26T01:30:42.098532+0000","last_active":"2023-10-26T01:30:43.100164+0000","last_peered":"2023-10-26T01:30:43.100164+0000","last_clean":"2023-10-26T01:30:43.100164+0000","last_became_active":"2023-10-26T01:30:42.098301+0000","last_became_peered":"2023-10-26T01:30:42.098301+0000","last_unstale":"2023-10-26T01:30:43.100164+0000","last_undegraded":"2023-10-26T01:30:43.100164+0000","last_fullsized":"2023-10-26T01:30:43.100164+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:55:51.116465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.113350+0000","last_change":"2023-10-26T01:30:41.101188+0000","last_active":"2023-10-26T01:30:42.113350+0000","last_peered":"2023-10-26T01:30:42.113350+0000","last_clean":"2023-10-26T01:30:42.113350+0000","last_became_active":"2023-10-26T01:30:41.100977+0000","last_became_peered":"2023-10-26T01:30:41.100977+0000","last_unstale":"2023-10-26T01:30:42.113350+0000","last_undegraded":"2023-10-26T01:30:42.113350+0000","last_fullsized":"2023-10-26T01:30:42.113350+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:13:32.821792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.139789+0000","last_change":"2023-10-26T01:30:42.105438+0000","last_active":"2023-10-26T01:30:43.139789+0000","last_peered":"2023-10-26T01:30:43.139789+0000","last_clean":"2023-10-26T01:30:43.139789+0000","last_became_active":"2023-10-26T01:30:42.105168+0000","last_became_peered":"2023-10-26T01:30:42.105168+0000","last_unstale":"2023-10-26T01:30:43.139789+0000","last_undegraded":"2023-10-26T01:30:43.139789+0000","last_fullsized":"2023-10-26T01:30:43.139789+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:53:57.632838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.111212+0000","last_change":"2023-10-26T01:30:41.088679+0000","last_active":"2023-10-26T01:30:42.111212+0000","last_peered":"2023-10-26T01:30:42.111212+0000","last_clean":"2023-10-26T01:30:42.111212+0000","last_became_active":"2023-10-26T01:30:41.086354+0000","last_became_peered":"2023-10-26T01:30:41.086354+0000","last_unstale":"2023-10-26T01:30:42.111212+0000","last_undegraded":"2023-10-26T01:30:42.111212+0000","last_fullsized":"2023-10-26T01:30:42.111212+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:54:04.970133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.085216+0000","last_change":"2023-10-26T01:30:42.100090+0000","last_active":"2023-10-26T01:30:43.085216+0000","last_peered":"2023-10-26T01:30:43.085216+0000","last_clean":"2023-10-26T01:30:43.085216+0000","last_became_active":"2023-10-26T01:30:42.095593+0000","last_became_peered":"2023-10-26T01:30:42.095593+0000","last_unstale":"2023-10-26T01:30:43.085216+0000","last_undegraded":"2023-10-26T01:30:43.085216+0000","last_fullsized":"2023-10-26T01:30:43.085216+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:31:08.727626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.113466+0000","last_change":"2023-10-26T01:30:41.097404+0000","last_active":"2023-10-26T01:30:42.113466+0000","last_peered":"2023-10-26T01:30:42.113466+0000","last_clean":"2023-10-26T01:30:42.113466+0000","last_became_active":"2023-10-26T01:30:41.097074+0000","last_became_peered":"2023-10-26T01:30:41.097074+0000","last_unstale":"2023-10-26T01:30:42.113466+0000","last_undegraded":"2023-10-26T01:30:42.113466+0000","last_fullsized":"2023-10-26T01:30:42.113466+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:31:39.970444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":14,"seq":60129542151,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":74,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195237,"statfs":{"total":200000000,"available":199923243,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67700000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93799999999999994}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.407}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0389999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4670000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.325}]}]},{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":650,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194662,"statfs":{"total":200000000,"available":199333905,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82599999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95199999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5649999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72099999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2629999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3500000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.411}]}]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":74,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195238,"statfs":{"total":200000000,"available":199923857,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8839999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.605}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96499999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2769999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90000000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.181}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0920000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":73,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195239,"statfs":{"total":200000000,"available":199924765,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72899999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69299999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1930000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91000000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.288}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0549999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":73,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195238,"statfs":{"total":200000000,"available":199924729,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1379999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.21}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.661}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.845}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9570000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.631}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2730000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":13,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":73,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195239,"statfs":{"total":200000000,"available":199924855,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2669999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79100000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90900000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0009999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.462}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1299999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.494}]}]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":73,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195238,"statfs":{"total":200000000,"available":199924695,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1639999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2589999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.482}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4059999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4350000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3069999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":650,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194661,"statfs":{"total":200000000,"available":199333849,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73999999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.131}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1639999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.218}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.28}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0609999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.375}]}]}],"pool_statfs":[]}} 2023-10-26T01:30:57.608 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-26T01:30:57.859 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.857+0000 7f523d397640 1 -- 172.21.15.169:0/3011951165 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5220003970 msgr2=0x7f5220003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:57.859 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.857+0000 7f523d397640 1 --2- 172.21.15.169:0/3011951165 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5220003970 0x7f5220003d90 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f5228009b50 tx=0x7f52200057f0 comp rx=0 tx=0).stop 2023-10-26T01:30:57.860 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.857+0000 7f52377fe640 1 -- 172.21.15.169:0/3011951165 reap_dead start 2023-10-26T01:30:57.860 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.857+0000 7f523d397640 1 -- 172.21.15.169:0/3011951165 shutdown_connections 2023-10-26T01:30:57.860 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.857+0000 7f523d397640 1 -- 172.21.15.169:0/3011951165 >> 172.21.15.169:0/3011951165 conn(0x7f5238069930 msgr2=0x7f5238069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:57.860 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.857+0000 7f523d397640 1 -- 172.21.15.169:0/3011951165 shutdown_connections 2023-10-26T01:30:57.860 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.857+0000 7f523d397640 1 -- 172.21.15.169:0/3011951165 wait complete. 2023-10-26T01:30:57.861 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.859+0000 7f523d397640 1 Processor -- start 2023-10-26T01:30:57.862 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.859+0000 7f523d397640 1 -- start start 2023-10-26T01:30:57.866 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.860+0000 7f523d397640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5220003970 0x7f523814a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:57.866 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.860+0000 7f523d397640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52380ad4d0 0x7f52381f1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:57.867 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.860+0000 7f523d397640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f52381f21d0 0x7f5238071200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:57.867 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.860+0000 7f523d397640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5238070160 con 0x7f52380ad4d0 2023-10-26T01:30:57.867 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.860+0000 7f523d397640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5238070440 con 0x7f52381f21d0 2023-10-26T01:30:57.867 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.860+0000 7f523d397640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5238070720 con 0x7f5220003970 2023-10-26T01:30:57.867 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.860+0000 7f52367fc640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52380ad4d0 0x7f52381f1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:57.868 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.860+0000 7f52367fc640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52380ad4d0 0x7f52381f1c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:46148/0 (socket says 172.21.15.169:46148) 2023-10-26T01:30:57.868 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.860+0000 7f52367fc640 1 -- 172.21.15.169:0/2297254100 learned_addr learned my addr 172.21.15.169:0/2297254100 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:30:57.868 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.861+0000 7f5236ffd640 1 --2- 172.21.15.169:0/2297254100 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5220003970 0x7f523814a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:57.868 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.861+0000 7f52367fc640 1 -- 172.21.15.169:0/2297254100 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5220003970 msgr2=0x7f523814a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:57.868 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.861+0000 7f52367fc640 1 --2- 172.21.15.169:0/2297254100 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5220003970 0x7f523814a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:57.869 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.861+0000 7f52377fe640 1 --2- 172.21.15.169:0/2297254100 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f52381f21d0 0x7f5238071200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:57.869 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.861+0000 7f52367fc640 1 -- 172.21.15.169:0/2297254100 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f52381f21d0 msgr2=0x7f5238071200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:57.869 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.861+0000 7f52367fc640 1 --2- 172.21.15.169:0/2297254100 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f52381f21d0 0x7f5238071200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:57.869 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.861+0000 7f52367fc640 1 -- 172.21.15.169:0/2297254100 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5238070a00 con 0x7f52380ad4d0 2023-10-26T01:30:57.869 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.861+0000 7f52377fe640 1 --2- 172.21.15.169:0/2297254100 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f52381f21d0 0x7f5238071200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:30:57.870 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.861+0000 7f52367fc640 1 --2- 172.21.15.169:0/2297254100 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52380ad4d0 0x7f52381f1c90 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f522c001010 tx=0x7f522c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:57.870 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.861+0000 7f521f7fe640 1 -- 172.21.15.169:0/2297254100 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f522c0050d0 con 0x7f52380ad4d0 2023-10-26T01:30:57.870 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.861+0000 7f521f7fe640 1 -- 172.21.15.169:0/2297254100 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f522c005230 con 0x7f52380ad4d0 2023-10-26T01:30:57.870 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.861+0000 7f521f7fe640 1 -- 172.21.15.169:0/2297254100 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f522c00b6c0 con 0x7f52380ad4d0 2023-10-26T01:30:57.870 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.861+0000 7f523d397640 1 -- 172.21.15.169:0/2297254100 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5238070ce0 con 0x7f52380ad4d0 2023-10-26T01:30:57.871 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.862+0000 7f523d397640 1 -- 172.21.15.169:0/2297254100 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f523806fd80 con 0x7f52380ad4d0 2023-10-26T01:30:57.871 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.863+0000 7f523d397640 1 -- 172.21.15.169:0/2297254100 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5238070ed0 con 0x7f52380ad4d0 2023-10-26T01:30:57.871 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.863+0000 7f521f7fe640 1 -- 172.21.15.169:0/2297254100 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f522c00b840 con 0x7f52380ad4d0 2023-10-26T01:30:57.871 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.865+0000 7f521f7fe640 1 --2- 172.21.15.169:0/2297254100 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f522406ae40 0x7f522406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:57.871 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.865+0000 7f521f7fe640 1 -- 172.21.15.169:0/2297254100 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f522c088f70 con 0x7f52380ad4d0 2023-10-26T01:30:57.872 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.865+0000 7f5236ffd640 1 --2- 172.21.15.169:0/2297254100 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f522406ae40 0x7f522406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:57.872 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.866+0000 7f5236ffd640 1 --2- 172.21.15.169:0/2297254100 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f522406ae40 0x7f522406d300 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5228036000 tx=0x7f52280046c0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:57.873 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:57.869+0000 7f521f7fe640 1 -- 172.21.15.169:0/2297254100 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f522c053f80 con 0x7f52380ad4d0 2023-10-26T01:30:58.044 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:58.043+0000 7f523d397640 1 -- 172.21.15.169:0/2297254100 --> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f52380ab220 con 0x7f522406ae40 2023-10-26T01:30:58.065 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:58.063+0000 7f521f7fe640 1 -- 172.21.15.169:0/2297254100 <== mgr.4112 v2:172.21.15.170:6832/1187156918 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+260063 (secure 0 0 0) 0x7f5228047450 con 0x7f522406ae40 2023-10-26T01:30:58.068 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:30:58.070 INFO:teuthology.orchestra.run.smithi169.stderr:dumped all 2023-10-26T01:30:58.072 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:58.069+0000 7f523d397640 1 -- 172.21.15.169:0/2297254100 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f522406ae40 msgr2=0x7f522406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:58.072 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:58.069+0000 7f523d397640 1 --2- 172.21.15.169:0/2297254100 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f522406ae40 0x7f522406d300 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5228036000 tx=0x7f52280046c0 comp rx=0 tx=0).stop 2023-10-26T01:30:58.072 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:58.069+0000 7f523d397640 1 -- 172.21.15.169:0/2297254100 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52380ad4d0 msgr2=0x7f52381f1c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:58.073 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:58.069+0000 7f523d397640 1 --2- 172.21.15.169:0/2297254100 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52380ad4d0 0x7f52381f1c90 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f522c001010 tx=0x7f522c00dc30 comp rx=0 tx=0).stop 2023-10-26T01:30:58.073 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:58.069+0000 7f523d397640 1 -- 172.21.15.169:0/2297254100 shutdown_connections 2023-10-26T01:30:58.073 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:58.069+0000 7f523d397640 1 --2- 172.21.15.169:0/2297254100 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f522406ae40 0x7f522406d300 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:58.073 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:58.069+0000 7f523d397640 1 --2- 172.21.15.169:0/2297254100 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52380ad4d0 0x7f52381f1c90 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:58.073 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:58.069+0000 7f523d397640 1 --2- 172.21.15.169:0/2297254100 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f52381f21d0 0x7f5238071200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:58.074 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:58.069+0000 7f523d397640 1 --2- 172.21.15.169:0/2297254100 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5220003970 0x7f523814a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:58.074 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:58.069+0000 7f523d397640 1 -- 172.21.15.169:0/2297254100 >> 172.21.15.169:0/2297254100 conn(0x7f5238069930 msgr2=0x7f5238159280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:58.075 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:58.069+0000 7f523d397640 1 -- 172.21.15.169:0/2297254100 shutdown_connections 2023-10-26T01:30:58.075 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:58.069+0000 7f523d397640 1 -- 172.21.15.169:0/2297254100 wait complete. 2023-10-26T01:30:58.089 INFO:teuthology.orchestra.run.smithi169.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2023-10-26T01:30:56.132776+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":1744,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560755,"statfs":{"total":1600000000,"available":1598213898,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004473"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.086425+0000","last_change":"2023-10-26T01:30:42.112552+0000","last_active":"2023-10-26T01:30:43.086425+0000","last_peered":"2023-10-26T01:30:43.086425+0000","last_clean":"2023-10-26T01:30:43.086425+0000","last_became_active":"2023-10-26T01:30:42.111151+0000","last_became_peered":"2023-10-26T01:30:42.111151+0000","last_unstale":"2023-10-26T01:30:43.086425+0000","last_undegraded":"2023-10-26T01:30:43.086425+0000","last_fullsized":"2023-10-26T01:30:43.086425+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:28:22.384005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.141033+0000","last_change":"2023-10-26T01:30:42.102872+0000","last_active":"2023-10-26T01:30:43.141033+0000","last_peered":"2023-10-26T01:30:43.141033+0000","last_clean":"2023-10-26T01:30:43.141033+0000","last_became_active":"2023-10-26T01:30:42.100796+0000","last_became_peered":"2023-10-26T01:30:42.100796+0000","last_unstale":"2023-10-26T01:30:43.141033+0000","last_undegraded":"2023-10-26T01:30:43.141033+0000","last_fullsized":"2023-10-26T01:30:43.141033+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:49:12.753504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.139076+0000","last_change":"2023-10-26T01:30:42.103921+0000","last_active":"2023-10-26T01:30:43.139076+0000","last_peered":"2023-10-26T01:30:43.139076+0000","last_clean":"2023-10-26T01:30:43.139076+0000","last_became_active":"2023-10-26T01:30:42.103384+0000","last_became_peered":"2023-10-26T01:30:42.103384+0000","last_unstale":"2023-10-26T01:30:43.139076+0000","last_undegraded":"2023-10-26T01:30:43.139076+0000","last_fullsized":"2023-10-26T01:30:43.139076+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:33:58.768305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.098597+0000","last_change":"2023-10-26T01:30:42.101845+0000","last_active":"2023-10-26T01:30:43.098597+0000","last_peered":"2023-10-26T01:30:43.098597+0000","last_clean":"2023-10-26T01:30:43.098597+0000","last_became_active":"2023-10-26T01:30:42.097125+0000","last_became_peered":"2023-10-26T01:30:42.097125+0000","last_unstale":"2023-10-26T01:30:43.098597+0000","last_undegraded":"2023-10-26T01:30:43.098597+0000","last_fullsized":"2023-10-26T01:30:43.098597+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:33:13.615628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.086419+0000","last_change":"2023-10-26T01:30:42.115244+0000","last_active":"2023-10-26T01:30:43.086419+0000","last_peered":"2023-10-26T01:30:43.086419+0000","last_clean":"2023-10-26T01:30:43.086419+0000","last_became_active":"2023-10-26T01:30:42.114972+0000","last_became_peered":"2023-10-26T01:30:42.114972+0000","last_unstale":"2023-10-26T01:30:43.086419+0000","last_undegraded":"2023-10-26T01:30:43.086419+0000","last_fullsized":"2023-10-26T01:30:43.086419+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:07:06.450331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.099993+0000","last_change":"2023-10-26T01:30:42.108711+0000","last_active":"2023-10-26T01:30:43.099993+0000","last_peered":"2023-10-26T01:30:43.099993+0000","last_clean":"2023-10-26T01:30:43.099993+0000","last_became_active":"2023-10-26T01:30:42.108215+0000","last_became_peered":"2023-10-26T01:30:42.108215+0000","last_unstale":"2023-10-26T01:30:43.099993+0000","last_undegraded":"2023-10-26T01:30:43.099993+0000","last_fullsized":"2023-10-26T01:30:43.099993+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:26:01.898201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.088319+0000","last_change":"2023-10-26T01:30:42.115268+0000","last_active":"2023-10-26T01:30:43.088319+0000","last_peered":"2023-10-26T01:30:43.088319+0000","last_clean":"2023-10-26T01:30:43.088319+0000","last_became_active":"2023-10-26T01:30:42.113208+0000","last_became_peered":"2023-10-26T01:30:42.113208+0000","last_unstale":"2023-10-26T01:30:43.088319+0000","last_undegraded":"2023-10-26T01:30:43.088319+0000","last_fullsized":"2023-10-26T01:30:43.088319+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:56:55.394232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.084344+0000","last_change":"2023-10-26T01:30:42.093950+0000","last_active":"2023-10-26T01:30:43.084344+0000","last_peered":"2023-10-26T01:30:43.084344+0000","last_clean":"2023-10-26T01:30:43.084344+0000","last_became_active":"2023-10-26T01:30:42.093684+0000","last_became_peered":"2023-10-26T01:30:42.093684+0000","last_unstale":"2023-10-26T01:30:43.084344+0000","last_undegraded":"2023-10-26T01:30:43.084344+0000","last_fullsized":"2023-10-26T01:30:43.084344+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:05:31.590291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.086093+0000","last_change":"2023-10-26T01:30:42.108451+0000","last_active":"2023-10-26T01:30:43.086093+0000","last_peered":"2023-10-26T01:30:43.086093+0000","last_clean":"2023-10-26T01:30:43.086093+0000","last_became_active":"2023-10-26T01:30:42.107650+0000","last_became_peered":"2023-10-26T01:30:42.107650+0000","last_unstale":"2023-10-26T01:30:43.086093+0000","last_undegraded":"2023-10-26T01:30:43.086093+0000","last_fullsized":"2023-10-26T01:30:43.086093+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:26:19.035685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.098895+0000","last_change":"2023-10-26T01:30:42.093838+0000","last_active":"2023-10-26T01:30:43.098895+0000","last_peered":"2023-10-26T01:30:43.098895+0000","last_clean":"2023-10-26T01:30:43.098895+0000","last_became_active":"2023-10-26T01:30:42.093584+0000","last_became_peered":"2023-10-26T01:30:42.093584+0000","last_unstale":"2023-10-26T01:30:43.098895+0000","last_undegraded":"2023-10-26T01:30:43.098895+0000","last_fullsized":"2023-10-26T01:30:43.098895+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:49:12.753504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.097658+0000","last_change":"2023-10-26T01:30:42.094026+0000","last_active":"2023-10-26T01:30:43.097658+0000","last_peered":"2023-10-26T01:30:43.097658+0000","last_clean":"2023-10-26T01:30:43.097658+0000","last_became_active":"2023-10-26T01:30:42.093654+0000","last_became_peered":"2023-10-26T01:30:42.093654+0000","last_unstale":"2023-10-26T01:30:43.097658+0000","last_undegraded":"2023-10-26T01:30:43.097658+0000","last_fullsized":"2023-10-26T01:30:43.097658+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:05:31.590291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.088414+0000","last_change":"2023-10-26T01:30:42.099764+0000","last_active":"2023-10-26T01:30:43.088414+0000","last_peered":"2023-10-26T01:30:43.088414+0000","last_clean":"2023-10-26T01:30:43.088414+0000","last_became_active":"2023-10-26T01:30:42.098108+0000","last_became_peered":"2023-10-26T01:30:42.098108+0000","last_unstale":"2023-10-26T01:30:43.088414+0000","last_undegraded":"2023-10-26T01:30:43.088414+0000","last_fullsized":"2023-10-26T01:30:43.088414+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:08:21.563278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.085548+0000","last_change":"2023-10-26T01:30:42.097734+0000","last_active":"2023-10-26T01:30:43.085548+0000","last_peered":"2023-10-26T01:30:43.085548+0000","last_clean":"2023-10-26T01:30:43.085548+0000","last_became_active":"2023-10-26T01:30:42.097333+0000","last_became_peered":"2023-10-26T01:30:42.097333+0000","last_unstale":"2023-10-26T01:30:43.085548+0000","last_undegraded":"2023-10-26T01:30:43.085548+0000","last_fullsized":"2023-10-26T01:30:43.085548+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:28:22.384005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.138887+0000","last_change":"2023-10-26T01:30:42.108715+0000","last_active":"2023-10-26T01:30:43.138887+0000","last_peered":"2023-10-26T01:30:43.138887+0000","last_clean":"2023-10-26T01:30:43.138887+0000","last_became_active":"2023-10-26T01:30:42.108469+0000","last_became_peered":"2023-10-26T01:30:42.108469+0000","last_unstale":"2023-10-26T01:30:43.138887+0000","last_undegraded":"2023-10-26T01:30:43.138887+0000","last_fullsized":"2023-10-26T01:30:43.138887+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:45:23.061469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.140916+0000","last_change":"2023-10-26T01:30:42.099157+0000","last_active":"2023-10-26T01:30:43.140916+0000","last_peered":"2023-10-26T01:30:43.140916+0000","last_clean":"2023-10-26T01:30:43.140916+0000","last_became_active":"2023-10-26T01:30:42.098929+0000","last_became_peered":"2023-10-26T01:30:42.098929+0000","last_unstale":"2023-10-26T01:30:43.140916+0000","last_undegraded":"2023-10-26T01:30:43.140916+0000","last_fullsized":"2023-10-26T01:30:43.140916+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:13:33.838018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.099835+0000","last_change":"2023-10-26T01:30:42.108578+0000","last_active":"2023-10-26T01:30:43.099835+0000","last_peered":"2023-10-26T01:30:43.099835+0000","last_clean":"2023-10-26T01:30:43.099835+0000","last_became_active":"2023-10-26T01:30:42.107937+0000","last_became_peered":"2023-10-26T01:30:42.107937+0000","last_unstale":"2023-10-26T01:30:43.099835+0000","last_undegraded":"2023-10-26T01:30:43.099835+0000","last_fullsized":"2023-10-26T01:30:43.099835+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:38:16.726919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.098559+0000","last_change":"2023-10-26T01:30:42.108373+0000","last_active":"2023-10-26T01:30:43.098559+0000","last_peered":"2023-10-26T01:30:43.098559+0000","last_clean":"2023-10-26T01:30:43.098559+0000","last_became_active":"2023-10-26T01:30:42.107773+0000","last_became_peered":"2023-10-26T01:30:42.107773+0000","last_unstale":"2023-10-26T01:30:43.098559+0000","last_undegraded":"2023-10-26T01:30:43.098559+0000","last_fullsized":"2023-10-26T01:30:43.098559+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:49:12.753504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.097381+0000","last_change":"2023-10-26T01:30:42.094993+0000","last_active":"2023-10-26T01:30:43.097381+0000","last_peered":"2023-10-26T01:30:43.097381+0000","last_clean":"2023-10-26T01:30:43.097381+0000","last_became_active":"2023-10-26T01:30:42.094689+0000","last_became_peered":"2023-10-26T01:30:42.094689+0000","last_unstale":"2023-10-26T01:30:43.097381+0000","last_undegraded":"2023-10-26T01:30:43.097381+0000","last_fullsized":"2023-10-26T01:30:43.097381+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:49:43.468558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.086773+0000","last_change":"2023-10-26T01:30:42.108746+0000","last_active":"2023-10-26T01:30:43.086773+0000","last_peered":"2023-10-26T01:30:43.086773+0000","last_clean":"2023-10-26T01:30:43.086773+0000","last_became_active":"2023-10-26T01:30:42.108188+0000","last_became_peered":"2023-10-26T01:30:42.108188+0000","last_unstale":"2023-10-26T01:30:43.086773+0000","last_undegraded":"2023-10-26T01:30:43.086773+0000","last_fullsized":"2023-10-26T01:30:43.086773+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:38:16.726919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.099061+0000","last_change":"2023-10-26T01:30:42.110331+0000","last_active":"2023-10-26T01:30:43.099061+0000","last_peered":"2023-10-26T01:30:43.099061+0000","last_clean":"2023-10-26T01:30:43.099061+0000","last_became_active":"2023-10-26T01:30:42.110079+0000","last_became_peered":"2023-10-26T01:30:42.110079+0000","last_unstale":"2023-10-26T01:30:43.099061+0000","last_undegraded":"2023-10-26T01:30:43.099061+0000","last_fullsized":"2023-10-26T01:30:43.099061+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:33:13.615628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.139607+0000","last_change":"2023-10-26T01:30:42.108608+0000","last_active":"2023-10-26T01:30:43.139607+0000","last_peered":"2023-10-26T01:30:43.139607+0000","last_clean":"2023-10-26T01:30:43.139607+0000","last_became_active":"2023-10-26T01:30:42.108197+0000","last_became_peered":"2023-10-26T01:30:42.108197+0000","last_unstale":"2023-10-26T01:30:43.139607+0000","last_undegraded":"2023-10-26T01:30:43.139607+0000","last_fullsized":"2023-10-26T01:30:43.139607+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:17:59.186810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.098944+0000","last_change":"2023-10-26T01:30:42.110289+0000","last_active":"2023-10-26T01:30:43.098944+0000","last_peered":"2023-10-26T01:30:43.098944+0000","last_clean":"2023-10-26T01:30:43.098944+0000","last_became_active":"2023-10-26T01:30:42.110061+0000","last_became_peered":"2023-10-26T01:30:42.110061+0000","last_unstale":"2023-10-26T01:30:43.098944+0000","last_undegraded":"2023-10-26T01:30:43.098944+0000","last_fullsized":"2023-10-26T01:30:43.098944+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:19:23.544531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.084819+0000","last_change":"2023-10-26T01:30:42.108686+0000","last_active":"2023-10-26T01:30:43.084819+0000","last_peered":"2023-10-26T01:30:43.084819+0000","last_clean":"2023-10-26T01:30:43.084819+0000","last_became_active":"2023-10-26T01:30:42.108136+0000","last_became_peered":"2023-10-26T01:30:42.108136+0000","last_unstale":"2023-10-26T01:30:43.084819+0000","last_undegraded":"2023-10-26T01:30:43.084819+0000","last_fullsized":"2023-10-26T01:30:43.084819+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:13:33.838018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.098984+0000","last_change":"2023-10-26T01:30:42.102147+0000","last_active":"2023-10-26T01:30:43.098984+0000","last_peered":"2023-10-26T01:30:43.098984+0000","last_clean":"2023-10-26T01:30:43.098984+0000","last_became_active":"2023-10-26T01:30:42.097509+0000","last_became_peered":"2023-10-26T01:30:42.097509+0000","last_unstale":"2023-10-26T01:30:43.098984+0000","last_undegraded":"2023-10-26T01:30:43.098984+0000","last_fullsized":"2023-10-26T01:30:43.098984+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:26:19.035685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.085124+0000","last_change":"2023-10-26T01:30:42.108519+0000","last_active":"2023-10-26T01:30:43.085124+0000","last_peered":"2023-10-26T01:30:43.085124+0000","last_clean":"2023-10-26T01:30:43.085124+0000","last_became_active":"2023-10-26T01:30:42.108283+0000","last_became_peered":"2023-10-26T01:30:42.108283+0000","last_unstale":"2023-10-26T01:30:43.085124+0000","last_undegraded":"2023-10-26T01:30:43.085124+0000","last_fullsized":"2023-10-26T01:30:43.085124+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:33:13.615628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.097862+0000","last_change":"2023-10-26T01:30:42.111865+0000","last_active":"2023-10-26T01:30:43.097862+0000","last_peered":"2023-10-26T01:30:43.097862+0000","last_clean":"2023-10-26T01:30:43.097862+0000","last_became_active":"2023-10-26T01:30:42.111636+0000","last_became_peered":"2023-10-26T01:30:42.111636+0000","last_unstale":"2023-10-26T01:30:43.097862+0000","last_undegraded":"2023-10-26T01:30:43.097862+0000","last_fullsized":"2023-10-26T01:30:43.097862+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:49:56.641022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.141348+0000","last_change":"2023-10-26T01:30:42.091332+0000","last_active":"2023-10-26T01:30:43.141348+0000","last_peered":"2023-10-26T01:30:43.141348+0000","last_clean":"2023-10-26T01:30:43.141348+0000","last_became_active":"2023-10-26T01:30:42.091105+0000","last_became_peered":"2023-10-26T01:30:42.091105+0000","last_unstale":"2023-10-26T01:30:43.141348+0000","last_undegraded":"2023-10-26T01:30:43.141348+0000","last_fullsized":"2023-10-26T01:30:43.141348+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:33:13.615628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.085113+0000","last_change":"2023-10-26T01:30:42.108822+0000","last_active":"2023-10-26T01:30:43.085113+0000","last_peered":"2023-10-26T01:30:43.085113+0000","last_clean":"2023-10-26T01:30:43.085113+0000","last_became_active":"2023-10-26T01:30:42.108448+0000","last_became_peered":"2023-10-26T01:30:42.108448+0000","last_unstale":"2023-10-26T01:30:43.085113+0000","last_undegraded":"2023-10-26T01:30:43.085113+0000","last_fullsized":"2023-10-26T01:30:43.085113+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:49:12.753504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.139200+0000","last_change":"2023-10-26T01:30:42.101708+0000","last_active":"2023-10-26T01:30:43.139200+0000","last_peered":"2023-10-26T01:30:43.139200+0000","last_clean":"2023-10-26T01:30:43.139200+0000","last_became_active":"2023-10-26T01:30:42.101434+0000","last_became_peered":"2023-10-26T01:30:42.101434+0000","last_unstale":"2023-10-26T01:30:43.139200+0000","last_undegraded":"2023-10-26T01:30:43.139200+0000","last_fullsized":"2023-10-26T01:30:43.139200+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:29:18.643148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.099207+0000","last_change":"2023-10-26T01:30:42.097495+0000","last_active":"2023-10-26T01:30:43.099207+0000","last_peered":"2023-10-26T01:30:43.099207+0000","last_clean":"2023-10-26T01:30:43.099207+0000","last_became_active":"2023-10-26T01:30:42.097214+0000","last_became_peered":"2023-10-26T01:30:42.097214+0000","last_unstale":"2023-10-26T01:30:43.099207+0000","last_undegraded":"2023-10-26T01:30:43.099207+0000","last_fullsized":"2023-10-26T01:30:43.099207+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T01:55:28.470913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.097481+0000","last_change":"2023-10-26T01:30:42.093157+0000","last_active":"2023-10-26T01:30:43.097481+0000","last_peered":"2023-10-26T01:30:43.097481+0000","last_clean":"2023-10-26T01:30:43.097481+0000","last_became_active":"2023-10-26T01:30:42.092613+0000","last_became_peered":"2023-10-26T01:30:42.092613+0000","last_unstale":"2023-10-26T01:30:43.097481+0000","last_undegraded":"2023-10-26T01:30:43.097481+0000","last_fullsized":"2023-10-26T01:30:43.097481+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:09:01.061004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.098226+0000","last_change":"2023-10-26T01:30:42.102704+0000","last_active":"2023-10-26T01:30:43.098226+0000","last_peered":"2023-10-26T01:30:43.098226+0000","last_clean":"2023-10-26T01:30:43.098226+0000","last_became_active":"2023-10-26T01:30:42.102350+0000","last_became_peered":"2023-10-26T01:30:42.102350+0000","last_unstale":"2023-10-26T01:30:43.098226+0000","last_undegraded":"2023-10-26T01:30:43.098226+0000","last_fullsized":"2023-10-26T01:30:43.098226+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:53:57.632838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.086493+0000","last_change":"2023-10-26T01:30:42.093818+0000","last_active":"2023-10-26T01:30:43.086493+0000","last_peered":"2023-10-26T01:30:43.086493+0000","last_clean":"2023-10-26T01:30:43.086493+0000","last_became_active":"2023-10-26T01:30:42.091310+0000","last_became_peered":"2023-10-26T01:30:42.091310+0000","last_unstale":"2023-10-26T01:30:43.086493+0000","last_undegraded":"2023-10-26T01:30:43.086493+0000","last_fullsized":"2023-10-26T01:30:43.086493+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:04:23.158155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.114911+0000","last_change":"2023-10-26T01:30:41.097666+0000","last_active":"2023-10-26T01:30:42.114911+0000","last_peered":"2023-10-26T01:30:42.114911+0000","last_clean":"2023-10-26T01:30:42.114911+0000","last_became_active":"2023-10-26T01:30:41.093986+0000","last_became_peered":"2023-10-26T01:30:41.093986+0000","last_unstale":"2023-10-26T01:30:42.114911+0000","last_undegraded":"2023-10-26T01:30:42.114911+0000","last_fullsized":"2023-10-26T01:30:42.114911+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:49:42.452332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.139516+0000","last_change":"2023-10-26T01:30:42.110839+0000","last_active":"2023-10-26T01:30:43.139516+0000","last_peered":"2023-10-26T01:30:43.139516+0000","last_clean":"2023-10-26T01:30:43.139516+0000","last_became_active":"2023-10-26T01:30:42.109385+0000","last_became_peered":"2023-10-26T01:30:42.109385+0000","last_unstale":"2023-10-26T01:30:43.139516+0000","last_undegraded":"2023-10-26T01:30:43.139516+0000","last_fullsized":"2023-10-26T01:30:43.139516+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:49:56.641022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.105272+0000","last_change":"2023-10-26T01:30:41.100248+0000","last_active":"2023-10-26T01:30:42.105272+0000","last_peered":"2023-10-26T01:30:42.105272+0000","last_clean":"2023-10-26T01:30:42.105272+0000","last_became_active":"2023-10-26T01:30:41.099957+0000","last_became_peered":"2023-10-26T01:30:41.099957+0000","last_unstale":"2023-10-26T01:30:42.105272+0000","last_undegraded":"2023-10-26T01:30:42.105272+0000","last_fullsized":"2023-10-26T01:30:42.105272+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:29:17.626922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.088500+0000","last_change":"2023-10-26T01:30:41.103843+0000","last_active":"2023-10-26T01:30:42.088500+0000","last_peered":"2023-10-26T01:30:42.088500+0000","last_clean":"2023-10-26T01:30:42.088500+0000","last_became_active":"2023-10-26T01:30:41.103356+0000","last_became_peered":"2023-10-26T01:30:41.103356+0000","last_unstale":"2023-10-26T01:30:42.088500+0000","last_undegraded":"2023-10-26T01:30:42.088500+0000","last_fullsized":"2023-10-26T01:30:42.088500+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:40:15.199455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.085463+0000","last_change":"2023-10-26T01:30:42.102443+0000","last_active":"2023-10-26T01:30:43.085463+0000","last_peered":"2023-10-26T01:30:43.085463+0000","last_clean":"2023-10-26T01:30:43.085463+0000","last_became_active":"2023-10-26T01:30:42.102213+0000","last_became_peered":"2023-10-26T01:30:42.102213+0000","last_unstale":"2023-10-26T01:30:43.085463+0000","last_undegraded":"2023-10-26T01:30:43.085463+0000","last_fullsized":"2023-10-26T01:30:43.085463+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:26:19.035685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.089693+0000","last_change":"2023-10-26T01:30:41.102989+0000","last_active":"2023-10-26T01:30:42.089693+0000","last_peered":"2023-10-26T01:30:42.089693+0000","last_clean":"2023-10-26T01:30:42.089693+0000","last_became_active":"2023-10-26T01:30:41.102681+0000","last_became_peered":"2023-10-26T01:30:41.102681+0000","last_unstale":"2023-10-26T01:30:42.089693+0000","last_undegraded":"2023-10-26T01:30:42.089693+0000","last_fullsized":"2023-10-26T01:30:42.089693+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:49:55.624796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.140341+0000","last_change":"2023-10-26T01:30:42.091359+0000","last_active":"2023-10-26T01:30:43.140341+0000","last_peered":"2023-10-26T01:30:43.140341+0000","last_clean":"2023-10-26T01:30:43.140341+0000","last_became_active":"2023-10-26T01:30:42.090949+0000","last_became_peered":"2023-10-26T01:30:42.090949+0000","last_unstale":"2023-10-26T01:30:43.140341+0000","last_undegraded":"2023-10-26T01:30:43.140341+0000","last_fullsized":"2023-10-26T01:30:43.140341+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:49:56.641022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.089836+0000","last_change":"2023-10-26T01:30:41.104058+0000","last_active":"2023-10-26T01:30:42.089836+0000","last_peered":"2023-10-26T01:30:42.089836+0000","last_clean":"2023-10-26T01:30:42.089836+0000","last_became_active":"2023-10-26T01:30:41.103821+0000","last_became_peered":"2023-10-26T01:30:41.103821+0000","last_unstale":"2023-10-26T01:30:42.089836+0000","last_undegraded":"2023-10-26T01:30:42.089836+0000","last_fullsized":"2023-10-26T01:30:42.089836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:46:05.927163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.098774+0000","last_change":"2023-10-26T01:30:42.108520+0000","last_active":"2023-10-26T01:30:43.098774+0000","last_peered":"2023-10-26T01:30:43.098774+0000","last_clean":"2023-10-26T01:30:43.098774+0000","last_became_active":"2023-10-26T01:30:42.108140+0000","last_became_peered":"2023-10-26T01:30:42.108140+0000","last_unstale":"2023-10-26T01:30:43.098774+0000","last_undegraded":"2023-10-26T01:30:43.098774+0000","last_fullsized":"2023-10-26T01:30:43.098774+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:31:08.727626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.088126+0000","last_change":"2023-10-26T01:30:41.106779+0000","last_active":"2023-10-26T01:30:42.088126+0000","last_peered":"2023-10-26T01:30:42.088126+0000","last_clean":"2023-10-26T01:30:42.088126+0000","last_became_active":"2023-10-26T01:30:41.104353+0000","last_became_peered":"2023-10-26T01:30:41.104353+0000","last_unstale":"2023-10-26T01:30:42.088126+0000","last_undegraded":"2023-10-26T01:30:42.088126+0000","last_fullsized":"2023-10-26T01:30:42.088126+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:45:22.045243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.139887+0000","last_change":"2023-10-26T01:30:42.105447+0000","last_active":"2023-10-26T01:30:43.139887+0000","last_peered":"2023-10-26T01:30:43.139887+0000","last_clean":"2023-10-26T01:30:43.139887+0000","last_became_active":"2023-10-26T01:30:42.105098+0000","last_became_peered":"2023-10-26T01:30:42.105098+0000","last_unstale":"2023-10-26T01:30:43.139887+0000","last_undegraded":"2023-10-26T01:30:43.139887+0000","last_fullsized":"2023-10-26T01:30:43.139887+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:13:33.838018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.110602+0000","last_change":"2023-10-26T01:30:41.103826+0000","last_active":"2023-10-26T01:30:42.110602+0000","last_peered":"2023-10-26T01:30:42.110602+0000","last_clean":"2023-10-26T01:30:42.110602+0000","last_became_active":"2023-10-26T01:30:41.103552+0000","last_became_peered":"2023-10-26T01:30:41.103552+0000","last_unstale":"2023-10-26T01:30:42.110602+0000","last_undegraded":"2023-10-26T01:30:42.110602+0000","last_fullsized":"2023-10-26T01:30:42.110602+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:54:04.970133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.086144+0000","last_change":"2023-10-26T01:30:42.114167+0000","last_active":"2023-10-26T01:30:43.086144+0000","last_peered":"2023-10-26T01:30:43.086144+0000","last_clean":"2023-10-26T01:30:43.086144+0000","last_became_active":"2023-10-26T01:30:42.113241+0000","last_became_peered":"2023-10-26T01:30:42.113241+0000","last_unstale":"2023-10-26T01:30:43.086144+0000","last_undegraded":"2023-10-26T01:30:43.086144+0000","last_fullsized":"2023-10-26T01:30:43.086144+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:40:16.215681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.116049+0000","last_change":"2023-10-26T01:30:41.106514+0000","last_active":"2023-10-26T01:30:42.116049+0000","last_peered":"2023-10-26T01:30:42.116049+0000","last_clean":"2023-10-26T01:30:42.116049+0000","last_became_active":"2023-10-26T01:30:41.106147+0000","last_became_peered":"2023-10-26T01:30:41.106147+0000","last_unstale":"2023-10-26T01:30:42.116049+0000","last_undegraded":"2023-10-26T01:30:42.116049+0000","last_fullsized":"2023-10-26T01:30:42.116049+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:45:22.045243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.089920+0000","last_change":"2023-10-26T01:30:42.112813+0000","last_active":"2023-10-26T01:30:43.089920+0000","last_peered":"2023-10-26T01:30:43.089920+0000","last_clean":"2023-10-26T01:30:43.089920+0000","last_became_active":"2023-10-26T01:30:42.112571+0000","last_became_peered":"2023-10-26T01:30:42.112571+0000","last_unstale":"2023-10-26T01:30:43.089920+0000","last_undegraded":"2023-10-26T01:30:43.089920+0000","last_fullsized":"2023-10-26T01:30:43.089920+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:19:16.042472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.108474+0000","last_change":"2023-10-26T01:30:41.098865+0000","last_active":"2023-10-26T01:30:42.108474+0000","last_peered":"2023-10-26T01:30:42.108474+0000","last_clean":"2023-10-26T01:30:42.108474+0000","last_became_active":"2023-10-26T01:30:41.098506+0000","last_became_peered":"2023-10-26T01:30:41.098506+0000","last_unstale":"2023-10-26T01:30:42.108474+0000","last_undegraded":"2023-10-26T01:30:42.108474+0000","last_fullsized":"2023-10-26T01:30:42.108474+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:09:00.044778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.140188+0000","last_change":"2023-10-26T01:30:42.108443+0000","last_active":"2023-10-26T01:30:43.140188+0000","last_peered":"2023-10-26T01:30:43.140188+0000","last_clean":"2023-10-26T01:30:43.140188+0000","last_became_active":"2023-10-26T01:30:42.107837+0000","last_became_peered":"2023-10-26T01:30:42.107837+0000","last_unstale":"2023-10-26T01:30:43.140188+0000","last_undegraded":"2023-10-26T01:30:43.140188+0000","last_fullsized":"2023-10-26T01:30:43.140188+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:33:13.615628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.090761+0000","last_change":"2023-10-26T01:30:41.106795+0000","last_active":"2023-10-26T01:30:42.090761+0000","last_peered":"2023-10-26T01:30:42.090761+0000","last_clean":"2023-10-26T01:30:42.090761+0000","last_became_active":"2023-10-26T01:30:41.106222+0000","last_became_peered":"2023-10-26T01:30:41.106222+0000","last_unstale":"2023-10-26T01:30:42.090761+0000","last_undegraded":"2023-10-26T01:30:42.090761+0000","last_fullsized":"2023-10-26T01:30:42.090761+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:53:56.616612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.086822+0000","last_change":"2023-10-26T01:30:42.113053+0000","last_active":"2023-10-26T01:30:43.086822+0000","last_peered":"2023-10-26T01:30:43.086822+0000","last_clean":"2023-10-26T01:30:43.086822+0000","last_became_active":"2023-10-26T01:30:42.112492+0000","last_became_peered":"2023-10-26T01:30:42.112492+0000","last_unstale":"2023-10-26T01:30:43.086822+0000","last_undegraded":"2023-10-26T01:30:43.086822+0000","last_fullsized":"2023-10-26T01:30:43.086822+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:20:47.431800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.089525+0000","last_change":"2023-10-26T01:30:41.100835+0000","last_active":"2023-10-26T01:30:42.089525+0000","last_peered":"2023-10-26T01:30:42.089525+0000","last_clean":"2023-10-26T01:30:42.089525+0000","last_became_active":"2023-10-26T01:30:41.099708+0000","last_became_peered":"2023-10-26T01:30:41.099708+0000","last_unstale":"2023-10-26T01:30:42.089525+0000","last_undegraded":"2023-10-26T01:30:42.089525+0000","last_fullsized":"2023-10-26T01:30:42.089525+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:04:22.141930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.098337+0000","last_change":"2023-10-26T01:30:42.100857+0000","last_active":"2023-10-26T01:30:43.098337+0000","last_peered":"2023-10-26T01:30:43.098337+0000","last_clean":"2023-10-26T01:30:43.098337+0000","last_became_active":"2023-10-26T01:30:42.100545+0000","last_became_peered":"2023-10-26T01:30:42.100545+0000","last_unstale":"2023-10-26T01:30:43.098337+0000","last_undegraded":"2023-10-26T01:30:43.098337+0000","last_fullsized":"2023-10-26T01:30:43.098337+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:13:33.838018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.114399+0000","last_change":"2023-10-26T01:30:41.095998+0000","last_active":"2023-10-26T01:30:42.114399+0000","last_peered":"2023-10-26T01:30:42.114399+0000","last_clean":"2023-10-26T01:30:42.114399+0000","last_became_active":"2023-10-26T01:30:41.095494+0000","last_became_peered":"2023-10-26T01:30:41.095494+0000","last_unstale":"2023-10-26T01:30:42.114399+0000","last_undegraded":"2023-10-26T01:30:42.114399+0000","last_fullsized":"2023-10-26T01:30:42.114399+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:54:04.970133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.089073+0000","last_change":"2023-10-26T01:30:42.115510+0000","last_active":"2023-10-26T01:30:43.089073+0000","last_peered":"2023-10-26T01:30:43.089073+0000","last_clean":"2023-10-26T01:30:43.089073+0000","last_became_active":"2023-10-26T01:30:42.113552+0000","last_became_peered":"2023-10-26T01:30:42.113552+0000","last_unstale":"2023-10-26T01:30:43.089073+0000","last_undegraded":"2023-10-26T01:30:43.089073+0000","last_fullsized":"2023-10-26T01:30:43.089073+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:32:08.387966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.110967+0000","last_change":"2023-10-26T01:30:41.093200+0000","last_active":"2023-10-26T01:30:42.110967+0000","last_peered":"2023-10-26T01:30:42.110967+0000","last_clean":"2023-10-26T01:30:42.110967+0000","last_became_active":"2023-10-26T01:30:41.092917+0000","last_became_peered":"2023-10-26T01:30:41.092917+0000","last_unstale":"2023-10-26T01:30:42.110967+0000","last_undegraded":"2023-10-26T01:30:42.110967+0000","last_fullsized":"2023-10-26T01:30:42.110967+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:54:04.970133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.087143+0000","last_change":"2023-10-26T01:30:42.113216+0000","last_active":"2023-10-26T01:30:43.087143+0000","last_peered":"2023-10-26T01:30:43.087143+0000","last_clean":"2023-10-26T01:30:43.087143+0000","last_became_active":"2023-10-26T01:30:42.112815+0000","last_became_peered":"2023-10-26T01:30:42.112815+0000","last_unstale":"2023-10-26T01:30:43.087143+0000","last_undegraded":"2023-10-26T01:30:43.087143+0000","last_fullsized":"2023-10-26T01:30:43.087143+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:55:51.116465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.114880+0000","last_change":"2023-10-26T01:30:41.102365+0000","last_active":"2023-10-26T01:30:42.114880+0000","last_peered":"2023-10-26T01:30:42.114880+0000","last_clean":"2023-10-26T01:30:42.114880+0000","last_became_active":"2023-10-26T01:30:41.102135+0000","last_became_peered":"2023-10-26T01:30:41.102135+0000","last_unstale":"2023-10-26T01:30:42.114880+0000","last_undegraded":"2023-10-26T01:30:42.114880+0000","last_fullsized":"2023-10-26T01:30:42.114880+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:08:20.547052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.086427+0000","last_change":"2023-10-26T01:30:42.108605+0000","last_active":"2023-10-26T01:30:43.086427+0000","last_peered":"2023-10-26T01:30:43.086427+0000","last_clean":"2023-10-26T01:30:43.086427+0000","last_became_active":"2023-10-26T01:30:42.107907+0000","last_became_peered":"2023-10-26T01:30:42.107907+0000","last_unstale":"2023-10-26T01:30:43.086427+0000","last_undegraded":"2023-10-26T01:30:43.086427+0000","last_fullsized":"2023-10-26T01:30:43.086427+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T01:55:28.470913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.105745+0000","last_change":"2023-10-26T01:30:41.093456+0000","last_active":"2023-10-26T01:30:42.105745+0000","last_peered":"2023-10-26T01:30:42.105745+0000","last_clean":"2023-10-26T01:30:42.105745+0000","last_became_active":"2023-10-26T01:30:41.092811+0000","last_became_peered":"2023-10-26T01:30:41.092811+0000","last_unstale":"2023-10-26T01:30:42.105745+0000","last_undegraded":"2023-10-26T01:30:42.105745+0000","last_fullsized":"2023-10-26T01:30:42.105745+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:04:22.141930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.099118+0000","last_change":"2023-10-26T01:30:42.097121+0000","last_active":"2023-10-26T01:30:43.099118+0000","last_peered":"2023-10-26T01:30:43.099118+0000","last_clean":"2023-10-26T01:30:43.099118+0000","last_became_active":"2023-10-26T01:30:42.096784+0000","last_became_peered":"2023-10-26T01:30:42.096784+0000","last_unstale":"2023-10-26T01:30:43.099118+0000","last_undegraded":"2023-10-26T01:30:43.099118+0000","last_fullsized":"2023-10-26T01:30:43.099118+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:28:22.384005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.083700+0000","last_change":"2023-10-26T01:30:41.106632+0000","last_active":"2023-10-26T01:30:42.083700+0000","last_peered":"2023-10-26T01:30:42.083700+0000","last_clean":"2023-10-26T01:30:42.083700+0000","last_became_active":"2023-10-26T01:30:41.106360+0000","last_became_peered":"2023-10-26T01:30:41.106360+0000","last_unstale":"2023-10-26T01:30:42.083700+0000","last_undegraded":"2023-10-26T01:30:42.083700+0000","last_fullsized":"2023-10-26T01:30:42.083700+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:40:15.199455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.086951+0000","last_change":"2023-10-26T01:30:42.109475+0000","last_active":"2023-10-26T01:30:43.086951+0000","last_peered":"2023-10-26T01:30:43.086951+0000","last_clean":"2023-10-26T01:30:43.086951+0000","last_became_active":"2023-10-26T01:30:42.109232+0000","last_became_peered":"2023-10-26T01:30:42.109232+0000","last_unstale":"2023-10-26T01:30:43.086951+0000","last_undegraded":"2023-10-26T01:30:43.086951+0000","last_fullsized":"2023-10-26T01:30:43.086951+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:26:01.898201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":165,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.096979+0000","last_change":"2023-10-26T01:30:34.883484+0000","last_active":"2023-10-26T01:30:42.096979+0000","last_peered":"2023-10-26T01:30:42.096979+0000","last_clean":"2023-10-26T01:30:42.096979+0000","last_became_active":"2023-10-26T01:30:34.883071+0000","last_became_peered":"2023-10-26T01:30:34.883071+0000","last_unstale":"2023-10-26T01:30:42.096979+0000","last_undegraded":"2023-10-26T01:30:42.096979+0000","last_fullsized":"2023-10-26T01:30:42.096979+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:32.890096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:32.890096+0000","last_clean_scrub_stamp":"2023-10-26T01:30:32.890096+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:22:57.937891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.090888+0000","last_change":"2023-10-26T01:30:41.105687+0000","last_active":"2023-10-26T01:30:42.090888+0000","last_peered":"2023-10-26T01:30:42.090888+0000","last_clean":"2023-10-26T01:30:42.090888+0000","last_became_active":"2023-10-26T01:30:41.105235+0000","last_became_peered":"2023-10-26T01:30:41.105235+0000","last_unstale":"2023-10-26T01:30:42.090888+0000","last_undegraded":"2023-10-26T01:30:42.090888+0000","last_fullsized":"2023-10-26T01:30:42.090888+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:13:32.821792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.084569+0000","last_change":"2023-10-26T01:30:42.093817+0000","last_active":"2023-10-26T01:30:43.084569+0000","last_peered":"2023-10-26T01:30:43.084569+0000","last_clean":"2023-10-26T01:30:43.084569+0000","last_became_active":"2023-10-26T01:30:42.093586+0000","last_became_peered":"2023-10-26T01:30:42.093586+0000","last_unstale":"2023-10-26T01:30:43.084569+0000","last_undegraded":"2023-10-26T01:30:43.084569+0000","last_fullsized":"2023-10-26T01:30:43.084569+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:17:59.186810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.096890+0000","last_change":"2023-10-26T01:30:41.094664+0000","last_active":"2023-10-26T01:30:42.096890+0000","last_peered":"2023-10-26T01:30:42.096890+0000","last_clean":"2023-10-26T01:30:42.096890+0000","last_became_active":"2023-10-26T01:30:41.092745+0000","last_became_peered":"2023-10-26T01:30:41.092745+0000","last_unstale":"2023-10-26T01:30:42.096890+0000","last_undegraded":"2023-10-26T01:30:42.096890+0000","last_fullsized":"2023-10-26T01:30:42.096890+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:45:22.045243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.140108+0000","last_change":"2023-10-26T01:30:42.107538+0000","last_active":"2023-10-26T01:30:43.140108+0000","last_peered":"2023-10-26T01:30:43.140108+0000","last_clean":"2023-10-26T01:30:43.140108+0000","last_became_active":"2023-10-26T01:30:42.107258+0000","last_became_peered":"2023-10-26T01:30:42.107258+0000","last_unstale":"2023-10-26T01:30:43.140108+0000","last_undegraded":"2023-10-26T01:30:43.140108+0000","last_fullsized":"2023-10-26T01:30:43.140108+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:49:12.753504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.083749+0000","last_change":"2023-10-26T01:30:41.099543+0000","last_active":"2023-10-26T01:30:42.083749+0000","last_peered":"2023-10-26T01:30:42.083749+0000","last_clean":"2023-10-26T01:30:42.083749+0000","last_became_active":"2023-10-26T01:30:41.099277+0000","last_became_peered":"2023-10-26T01:30:41.099277+0000","last_unstale":"2023-10-26T01:30:42.083749+0000","last_undegraded":"2023-10-26T01:30:42.083749+0000","last_fullsized":"2023-10-26T01:30:42.083749+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:04:22.141930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.140694+0000","last_change":"2023-10-26T01:30:42.085871+0000","last_active":"2023-10-26T01:30:43.140694+0000","last_peered":"2023-10-26T01:30:43.140694+0000","last_clean":"2023-10-26T01:30:43.140694+0000","last_became_active":"2023-10-26T01:30:42.085409+0000","last_became_peered":"2023-10-26T01:30:42.085409+0000","last_unstale":"2023-10-26T01:30:43.140694+0000","last_undegraded":"2023-10-26T01:30:43.140694+0000","last_fullsized":"2023-10-26T01:30:43.140694+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:17:59.186810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.088251+0000","last_change":"2023-10-26T01:30:41.107609+0000","last_active":"2023-10-26T01:30:42.088251+0000","last_peered":"2023-10-26T01:30:42.088251+0000","last_clean":"2023-10-26T01:30:42.088251+0000","last_became_active":"2023-10-26T01:30:41.107189+0000","last_became_peered":"2023-10-26T01:30:41.107189+0000","last_unstale":"2023-10-26T01:30:42.088251+0000","last_undegraded":"2023-10-26T01:30:42.088251+0000","last_fullsized":"2023-10-26T01:30:42.088251+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:33:57.752079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.088414+0000","last_change":"2023-10-26T01:30:42.105495+0000","last_active":"2023-10-26T01:30:43.088414+0000","last_peered":"2023-10-26T01:30:43.088414+0000","last_clean":"2023-10-26T01:30:43.088414+0000","last_became_active":"2023-10-26T01:30:42.105247+0000","last_became_peered":"2023-10-26T01:30:42.105247+0000","last_unstale":"2023-10-26T01:30:43.088414+0000","last_undegraded":"2023-10-26T01:30:43.088414+0000","last_fullsized":"2023-10-26T01:30:43.088414+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:24:17.679373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.088367+0000","last_change":"2023-10-26T01:30:41.096866+0000","last_active":"2023-10-26T01:30:42.088367+0000","last_peered":"2023-10-26T01:30:42.088367+0000","last_clean":"2023-10-26T01:30:42.088367+0000","last_became_active":"2023-10-26T01:30:41.096255+0000","last_became_peered":"2023-10-26T01:30:41.096255+0000","last_unstale":"2023-10-26T01:30:42.088367+0000","last_undegraded":"2023-10-26T01:30:42.088367+0000","last_fullsized":"2023-10-26T01:30:42.088367+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:29:17.626922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.099320+0000","last_change":"2023-10-26T01:30:42.102298+0000","last_active":"2023-10-26T01:30:43.099320+0000","last_peered":"2023-10-26T01:30:43.099320+0000","last_clean":"2023-10-26T01:30:43.099320+0000","last_became_active":"2023-10-26T01:30:42.097872+0000","last_became_peered":"2023-10-26T01:30:42.097872+0000","last_unstale":"2023-10-26T01:30:43.099320+0000","last_undegraded":"2023-10-26T01:30:43.099320+0000","last_fullsized":"2023-10-26T01:30:43.099320+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:07:06.450331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.096762+0000","last_change":"2023-10-26T01:30:41.096310+0000","last_active":"2023-10-26T01:30:42.096762+0000","last_peered":"2023-10-26T01:30:42.096762+0000","last_clean":"2023-10-26T01:30:42.096762+0000","last_became_active":"2023-10-26T01:30:41.095929+0000","last_became_peered":"2023-10-26T01:30:41.095929+0000","last_unstale":"2023-10-26T01:30:42.096762+0000","last_undegraded":"2023-10-26T01:30:42.096762+0000","last_fullsized":"2023-10-26T01:30:42.096762+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:31:39.970444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.140793+0000","last_change":"2023-10-26T01:30:42.089331+0000","last_active":"2023-10-26T01:30:43.140793+0000","last_peered":"2023-10-26T01:30:43.140793+0000","last_clean":"2023-10-26T01:30:43.140793+0000","last_became_active":"2023-10-26T01:30:42.088901+0000","last_became_peered":"2023-10-26T01:30:42.088901+0000","last_unstale":"2023-10-26T01:30:43.140793+0000","last_undegraded":"2023-10-26T01:30:43.140793+0000","last_fullsized":"2023-10-26T01:30:43.140793+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:53:57.632838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.140717+0000","last_change":"2023-10-26T01:30:41.102617+0000","last_active":"2023-10-26T01:30:43.140717+0000","last_peered":"2023-10-26T01:30:43.140717+0000","last_clean":"2023-10-26T01:30:43.140717+0000","last_became_active":"2023-10-26T01:30:41.102376+0000","last_became_peered":"2023-10-26T01:30:41.102376+0000","last_unstale":"2023-10-26T01:30:43.140717+0000","last_undegraded":"2023-10-26T01:30:43.140717+0000","last_fullsized":"2023-10-26T01:30:43.140717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:24:16.663147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.084424+0000","last_change":"2023-10-26T01:30:42.112237+0000","last_active":"2023-10-26T01:30:43.084424+0000","last_peered":"2023-10-26T01:30:43.084424+0000","last_clean":"2023-10-26T01:30:43.084424+0000","last_became_active":"2023-10-26T01:30:42.112006+0000","last_became_peered":"2023-10-26T01:30:42.112006+0000","last_unstale":"2023-10-26T01:30:43.084424+0000","last_undegraded":"2023-10-26T01:30:43.084424+0000","last_fullsized":"2023-10-26T01:30:43.084424+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:49:56.641022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.112282+0000","last_change":"2023-10-26T01:30:41.102404+0000","last_active":"2023-10-26T01:30:42.112282+0000","last_peered":"2023-10-26T01:30:42.112282+0000","last_clean":"2023-10-26T01:30:42.112282+0000","last_became_active":"2023-10-26T01:30:41.102165+0000","last_became_peered":"2023-10-26T01:30:41.102165+0000","last_unstale":"2023-10-26T01:30:42.112282+0000","last_undegraded":"2023-10-26T01:30:42.112282+0000","last_fullsized":"2023-10-26T01:30:42.112282+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:08:20.547052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.139422+0000","last_change":"2023-10-26T01:30:42.103630+0000","last_active":"2023-10-26T01:30:43.139422+0000","last_peered":"2023-10-26T01:30:43.139422+0000","last_clean":"2023-10-26T01:30:43.139422+0000","last_became_active":"2023-10-26T01:30:42.099458+0000","last_became_peered":"2023-10-26T01:30:42.099458+0000","last_unstale":"2023-10-26T01:30:43.139422+0000","last_undegraded":"2023-10-26T01:30:43.139422+0000","last_fullsized":"2023-10-26T01:30:43.139422+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:05:31.590291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.097650+0000","last_change":"2023-10-26T01:30:41.097301+0000","last_active":"2023-10-26T01:30:42.097650+0000","last_peered":"2023-10-26T01:30:42.097650+0000","last_clean":"2023-10-26T01:30:42.097650+0000","last_became_active":"2023-10-26T01:30:41.096866+0000","last_became_peered":"2023-10-26T01:30:41.096866+0000","last_unstale":"2023-10-26T01:30:42.097650+0000","last_undegraded":"2023-10-26T01:30:42.097650+0000","last_fullsized":"2023-10-26T01:30:42.097650+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:05:30.574065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.089679+0000","last_change":"2023-10-26T01:30:42.087483+0000","last_active":"2023-10-26T01:30:43.089679+0000","last_peered":"2023-10-26T01:30:43.089679+0000","last_clean":"2023-10-26T01:30:43.089679+0000","last_became_active":"2023-10-26T01:30:42.087234+0000","last_became_peered":"2023-10-26T01:30:42.087234+0000","last_unstale":"2023-10-26T01:30:43.089679+0000","last_undegraded":"2023-10-26T01:30:43.089679+0000","last_fullsized":"2023-10-26T01:30:43.089679+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:46:06.943389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.088240+0000","last_change":"2023-10-26T01:30:41.097032+0000","last_active":"2023-10-26T01:30:42.088240+0000","last_peered":"2023-10-26T01:30:42.088240+0000","last_clean":"2023-10-26T01:30:42.088240+0000","last_became_active":"2023-10-26T01:30:41.096591+0000","last_became_peered":"2023-10-26T01:30:41.096591+0000","last_unstale":"2023-10-26T01:30:42.088240+0000","last_undegraded":"2023-10-26T01:30:42.088240+0000","last_fullsized":"2023-10-26T01:30:42.088240+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:33:57.752079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.084667+0000","last_change":"2023-10-26T01:30:42.101380+0000","last_active":"2023-10-26T01:30:43.084667+0000","last_peered":"2023-10-26T01:30:43.084667+0000","last_clean":"2023-10-26T01:30:43.084667+0000","last_became_active":"2023-10-26T01:30:42.101179+0000","last_became_peered":"2023-10-26T01:30:42.101179+0000","last_unstale":"2023-10-26T01:30:43.084667+0000","last_undegraded":"2023-10-26T01:30:43.084667+0000","last_fullsized":"2023-10-26T01:30:43.084667+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:53:57.632838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.084678+0000","last_change":"2023-10-26T01:30:41.101587+0000","last_active":"2023-10-26T01:30:42.084678+0000","last_peered":"2023-10-26T01:30:42.084678+0000","last_clean":"2023-10-26T01:30:42.084678+0000","last_became_active":"2023-10-26T01:30:41.101325+0000","last_became_peered":"2023-10-26T01:30:41.101325+0000","last_unstale":"2023-10-26T01:30:42.084678+0000","last_undegraded":"2023-10-26T01:30:42.084678+0000","last_fullsized":"2023-10-26T01:30:42.084678+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:49:42.452332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.085284+0000","last_change":"2023-10-26T01:30:42.103918+0000","last_active":"2023-10-26T01:30:43.085284+0000","last_peered":"2023-10-26T01:30:43.085284+0000","last_clean":"2023-10-26T01:30:43.085284+0000","last_became_active":"2023-10-26T01:30:42.103701+0000","last_became_peered":"2023-10-26T01:30:42.103701+0000","last_unstale":"2023-10-26T01:30:43.085284+0000","last_undegraded":"2023-10-26T01:30:43.085284+0000","last_fullsized":"2023-10-26T01:30:43.085284+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:19:23.544531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.114450+0000","last_change":"2023-10-26T01:30:41.095274+0000","last_active":"2023-10-26T01:30:42.114450+0000","last_peered":"2023-10-26T01:30:42.114450+0000","last_clean":"2023-10-26T01:30:42.114450+0000","last_became_active":"2023-10-26T01:30:41.092922+0000","last_became_peered":"2023-10-26T01:30:41.092922+0000","last_unstale":"2023-10-26T01:30:42.114450+0000","last_undegraded":"2023-10-26T01:30:42.114450+0000","last_fullsized":"2023-10-26T01:30:42.114450+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:33:12.599402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.098208+0000","last_change":"2023-10-26T01:30:42.105139+0000","last_active":"2023-10-26T01:30:43.098208+0000","last_peered":"2023-10-26T01:30:43.098208+0000","last_clean":"2023-10-26T01:30:43.098208+0000","last_became_active":"2023-10-26T01:30:42.104913+0000","last_became_peered":"2023-10-26T01:30:42.104913+0000","last_unstale":"2023-10-26T01:30:43.098208+0000","last_undegraded":"2023-10-26T01:30:43.098208+0000","last_fullsized":"2023-10-26T01:30:43.098208+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:17:59.186810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.099380+0000","last_change":"2023-10-26T01:30:42.106926+0000","last_active":"2023-10-26T01:30:43.099380+0000","last_peered":"2023-10-26T01:30:43.099380+0000","last_clean":"2023-10-26T01:30:43.099380+0000","last_became_active":"2023-10-26T01:30:42.106692+0000","last_became_peered":"2023-10-26T01:30:42.106692+0000","last_unstale":"2023-10-26T01:30:43.099380+0000","last_undegraded":"2023-10-26T01:30:43.099380+0000","last_fullsized":"2023-10-26T01:30:43.099380+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:20:47.431800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.110477+0000","last_change":"2023-10-26T01:30:41.105282+0000","last_active":"2023-10-26T01:30:42.110477+0000","last_peered":"2023-10-26T01:30:42.110477+0000","last_clean":"2023-10-26T01:30:42.110477+0000","last_became_active":"2023-10-26T01:30:41.104553+0000","last_became_peered":"2023-10-26T01:30:41.104553+0000","last_unstale":"2023-10-26T01:30:42.110477+0000","last_undegraded":"2023-10-26T01:30:42.110477+0000","last_fullsized":"2023-10-26T01:30:42.110477+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:29:17.626922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.100164+0000","last_change":"2023-10-26T01:30:42.098532+0000","last_active":"2023-10-26T01:30:43.100164+0000","last_peered":"2023-10-26T01:30:43.100164+0000","last_clean":"2023-10-26T01:30:43.100164+0000","last_became_active":"2023-10-26T01:30:42.098301+0000","last_became_peered":"2023-10-26T01:30:42.098301+0000","last_unstale":"2023-10-26T01:30:43.100164+0000","last_undegraded":"2023-10-26T01:30:43.100164+0000","last_fullsized":"2023-10-26T01:30:43.100164+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:55:51.116465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.113350+0000","last_change":"2023-10-26T01:30:41.101188+0000","last_active":"2023-10-26T01:30:42.113350+0000","last_peered":"2023-10-26T01:30:42.113350+0000","last_clean":"2023-10-26T01:30:42.113350+0000","last_became_active":"2023-10-26T01:30:41.100977+0000","last_became_peered":"2023-10-26T01:30:41.100977+0000","last_unstale":"2023-10-26T01:30:42.113350+0000","last_undegraded":"2023-10-26T01:30:42.113350+0000","last_fullsized":"2023-10-26T01:30:42.113350+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:13:32.821792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.139789+0000","last_change":"2023-10-26T01:30:42.105438+0000","last_active":"2023-10-26T01:30:43.139789+0000","last_peered":"2023-10-26T01:30:43.139789+0000","last_clean":"2023-10-26T01:30:43.139789+0000","last_became_active":"2023-10-26T01:30:42.105168+0000","last_became_peered":"2023-10-26T01:30:42.105168+0000","last_unstale":"2023-10-26T01:30:43.139789+0000","last_undegraded":"2023-10-26T01:30:43.139789+0000","last_fullsized":"2023-10-26T01:30:43.139789+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:53:57.632838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.111212+0000","last_change":"2023-10-26T01:30:41.088679+0000","last_active":"2023-10-26T01:30:42.111212+0000","last_peered":"2023-10-26T01:30:42.111212+0000","last_clean":"2023-10-26T01:30:42.111212+0000","last_became_active":"2023-10-26T01:30:41.086354+0000","last_became_peered":"2023-10-26T01:30:41.086354+0000","last_unstale":"2023-10-26T01:30:42.111212+0000","last_undegraded":"2023-10-26T01:30:42.111212+0000","last_fullsized":"2023-10-26T01:30:42.111212+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:54:04.970133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-26T01:30:43.085216+0000","last_change":"2023-10-26T01:30:42.100090+0000","last_active":"2023-10-26T01:30:43.085216+0000","last_peered":"2023-10-26T01:30:43.085216+0000","last_clean":"2023-10-26T01:30:43.085216+0000","last_became_active":"2023-10-26T01:30:42.095593+0000","last_became_peered":"2023-10-26T01:30:42.095593+0000","last_unstale":"2023-10-26T01:30:43.085216+0000","last_undegraded":"2023-10-26T01:30:43.085216+0000","last_fullsized":"2023-10-26T01:30:43.085216+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:41.060332+0000","last_clean_scrub_stamp":"2023-10-26T01:30:41.060332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:31:08.727626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T01:30:42.113466+0000","last_change":"2023-10-26T01:30:41.097404+0000","last_active":"2023-10-26T01:30:42.113466+0000","last_peered":"2023-10-26T01:30:42.113466+0000","last_clean":"2023-10-26T01:30:42.113466+0000","last_became_active":"2023-10-26T01:30:41.097074+0000","last_became_peered":"2023-10-26T01:30:41.097074+0000","last_unstale":"2023-10-26T01:30:42.113466+0000","last_undegraded":"2023-10-26T01:30:42.113466+0000","last_fullsized":"2023-10-26T01:30:42.113466+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:40.044106+0000","last_clean_scrub_stamp":"2023-10-26T01:30:40.044106+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:31:39.970444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":14,"seq":60129542151,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":74,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195237,"statfs":{"total":200000000,"available":199923243,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67700000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93799999999999994}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.407}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0389999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4670000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.325}]}]},{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":650,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194662,"statfs":{"total":200000000,"available":199333905,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82599999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95199999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5649999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72099999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2629999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3500000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.411}]}]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":74,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195238,"statfs":{"total":200000000,"available":199923857,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8839999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.605}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96499999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2769999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90000000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.181}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0920000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":73,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195239,"statfs":{"total":200000000,"available":199924765,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72899999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69299999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1930000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91000000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.288}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0549999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":73,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195238,"statfs":{"total":200000000,"available":199924729,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1379999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.21}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.661}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.845}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9570000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.631}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2730000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":13,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":73,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195239,"statfs":{"total":200000000,"available":199924855,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2669999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79100000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90900000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0009999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.462}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1299999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.494}]}]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":73,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195238,"statfs":{"total":200000000,"available":199924695,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1639999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2589999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.482}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4059999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4350000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3069999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":650,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194661,"statfs":{"total":200000000,"available":199333849,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73999999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.131}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1639999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.218}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.28}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0609999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.375}]}]}],"pool_statfs":[]}} 2023-10-26T01:30:58.096 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-10-26T01:30:58.097 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-10-26T01:30:58.097 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-10-26T01:30:58.097 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-26T01:30:58.347 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:58.345+0000 7efe5af9a640 1 -- 172.21.15.169:0/1001729871 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe3c004d00 msgr2=0x7efe3c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:58.348 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:58.345+0000 7efe5af9a640 1 --2- 172.21.15.169:0/1001729871 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe3c004d00 0x7efe3c005180 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7efe4c00bb20 tx=0x7efe3c0057f0 comp rx=0 tx=0).stop 2023-10-26T01:30:58.348 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:58.345+0000 7efe5a799640 1 -- 172.21.15.169:0/1001729871 reap_dead start 2023-10-26T01:30:58.348 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:58.346+0000 7efe5af9a640 1 -- 172.21.15.169:0/1001729871 shutdown_connections 2023-10-26T01:30:58.349 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:58.346+0000 7efe5af9a640 1 -- 172.21.15.169:0/1001729871 >> 172.21.15.169:0/1001729871 conn(0x7efe54065490 msgr2=0x7efe540658a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:58.349 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:58.346+0000 7efe5af9a640 1 -- 172.21.15.169:0/1001729871 shutdown_connections 2023-10-26T01:30:58.349 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:58.346+0000 7efe5af9a640 1 -- 172.21.15.169:0/1001729871 wait complete. 2023-10-26T01:30:58.351 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:58.347+0000 7efe5af9a640 1 Processor -- start 2023-10-26T01:30:58.351 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:58.347+0000 7efe5af9a640 1 -- start start 2023-10-26T01:30:58.352 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:58.348+0000 7efe5af9a640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe54134d00 0x7efe540c6720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:58.352 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:58.348+0000 7efe5af9a640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7efe540c9110 0x7efe540c9590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:58.352 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:58.348+0000 7efe5af9a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe541f4e30 0x7efe541f52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:58.352 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:58.348+0000 7efe5af9a640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe54135760 con 0x7efe54134d00 2023-10-26T01:30:58.353 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:58.348+0000 7efe5af9a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe540c9ad0 con 0x7efe541f4e30 2023-10-26T01:30:58.353 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:58.348+0000 7efe5af9a640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe540c6c60 con 0x7efe540c9110 2023-10-26T01:30:58.353 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:58.349+0000 7efe59f98640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe54134d00 0x7efe540c6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:58.353 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:58.349+0000 7efe59f98640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe54134d00 0x7efe540c6720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:46168/0 (socket says 172.21.15.169:46168) 2023-10-26T01:30:58.353 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:58.349+0000 7efe59f98640 1 -- 172.21.15.169:0/2424286076 learned_addr learned my addr 172.21.15.169:0/2424286076 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:30:58.354 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:58.349+0000 7efe5a799640 1 --2- 172.21.15.169:0/2424286076 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe541f4e30 0x7efe541f52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:58.354 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:58.349+0000 7efe59797640 1 --2- 172.21.15.169:0/2424286076 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7efe540c9110 0x7efe540c9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:58.354 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:58.349+0000 7efe59f98640 1 -- 172.21.15.169:0/2424286076 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7efe540c9110 msgr2=0x7efe540c9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:58.354 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:58.349+0000 7efe59f98640 1 --2- 172.21.15.169:0/2424286076 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7efe540c9110 0x7efe540c9590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:58.355 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:58.349+0000 7efe59f98640 1 -- 172.21.15.169:0/2424286076 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe541f4e30 msgr2=0x7efe541f52e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:58.355 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:58.349+0000 7efe59f98640 1 --2- 172.21.15.169:0/2424286076 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe541f4e30 0x7efe541f52e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:58.355 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:58.349+0000 7efe59f98640 1 -- 172.21.15.169:0/2424286076 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe541e5de0 con 0x7efe54134d00 2023-10-26T01:30:58.355 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:58.349+0000 7efe59f98640 1 --2- 172.21.15.169:0/2424286076 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe54134d00 0x7efe540c6720 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7efe4400daf0 tx=0x7efe4400b0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:58.355 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:58.349+0000 7efe527fc640 1 -- 172.21.15.169:0/2424286076 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe44003c60 con 0x7efe54134d00 2023-10-26T01:30:58.356 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:58.350+0000 7efe527fc640 1 -- 172.21.15.169:0/2424286076 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7efe44003dc0 con 0x7efe54134d00 2023-10-26T01:30:58.356 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:58.350+0000 7efe527fc640 1 -- 172.21.15.169:0/2424286076 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe4400bb10 con 0x7efe54134d00 2023-10-26T01:30:58.356 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:58.350+0000 7efe5af9a640 1 -- 172.21.15.169:0/2424286076 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe541e60c0 con 0x7efe54134d00 2023-10-26T01:30:58.356 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:58.350+0000 7efe5af9a640 1 -- 172.21.15.169:0/2424286076 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe541e6680 con 0x7efe54134d00 2023-10-26T01:30:58.357 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:58.351+0000 7efe5af9a640 1 -- 172.21.15.169:0/2424286076 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe540c7140 con 0x7efe54134d00 2023-10-26T01:30:58.358 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:58.352+0000 7efe527fc640 1 -- 172.21.15.169:0/2424286076 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7efe4400bc70 con 0x7efe54134d00 2023-10-26T01:30:58.358 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:58.353+0000 7efe527fc640 1 --2- 172.21.15.169:0/2424286076 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7efe4006ae70 0x7efe4006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:58.358 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:58.353+0000 7efe527fc640 1 -- 172.21.15.169:0/2424286076 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efe440890e0 con 0x7efe54134d00 2023-10-26T01:30:58.358 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:58.353+0000 7efe59797640 1 --2- 172.21.15.169:0/2424286076 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7efe4006ae70 0x7efe4006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:58.358 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:58.355+0000 7efe59797640 1 --2- 172.21.15.169:0/2424286076 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7efe4006ae70 0x7efe4006d330 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7efe54141410 tx=0x7efe48005e50 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:58.359 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:58.358+0000 7efe527fc640 1 -- 172.21.15.169:0/2424286076 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7efe540c7140 con 0x7efe54134d00 2023-10-26T01:30:58.585 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:58.584+0000 7efe5af9a640 1 -- 172.21.15.169:0/2424286076 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7efe540cf350 con 0x7efe54134d00 2023-10-26T01:30:58.586 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:58.585+0000 7efe527fc640 1 -- 172.21.15.169:0/2424286076 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7efe540cf350 con 0x7efe54134d00 2023-10-26T01:30:58.587 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:30:58.587 INFO:teuthology.orchestra.run.smithi169.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-10-26T01:30:58.590 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:58.587+0000 7efe5af9a640 1 -- 172.21.15.169:0/2424286076 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7efe4006ae70 msgr2=0x7efe4006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:58.590 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:58.587+0000 7efe5af9a640 1 --2- 172.21.15.169:0/2424286076 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7efe4006ae70 0x7efe4006d330 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7efe54141410 tx=0x7efe48005e50 comp rx=0 tx=0).stop 2023-10-26T01:30:58.590 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:58.587+0000 7efe5af9a640 1 -- 172.21.15.169:0/2424286076 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe54134d00 msgr2=0x7efe540c6720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:58.590 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:58.587+0000 7efe5af9a640 1 --2- 172.21.15.169:0/2424286076 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe54134d00 0x7efe540c6720 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7efe4400daf0 tx=0x7efe4400b0e0 comp rx=0 tx=0).stop 2023-10-26T01:30:58.591 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:58.588+0000 7efe5af9a640 1 -- 172.21.15.169:0/2424286076 shutdown_connections 2023-10-26T01:30:58.591 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:58.588+0000 7efe5af9a640 1 --2- 172.21.15.169:0/2424286076 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7efe4006ae70 0x7efe4006d330 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:58.591 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:58.588+0000 7efe5af9a640 1 --2- 172.21.15.169:0/2424286076 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe541f4e30 0x7efe541f52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:58.591 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:58.588+0000 7efe5af9a640 1 --2- 172.21.15.169:0/2424286076 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7efe540c9110 0x7efe540c9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:58.591 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:58.588+0000 7efe5af9a640 1 --2- 172.21.15.169:0/2424286076 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe54134d00 0x7efe540c6720 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:58.591 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:58.588+0000 7efe5af9a640 1 -- 172.21.15.169:0/2424286076 >> 172.21.15.169:0/2424286076 conn(0x7efe54065490 msgr2=0x7efe54142fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:58.592 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:58.588+0000 7efe5af9a640 1 -- 172.21.15.169:0/2424286076 shutdown_connections 2023-10-26T01:30:58.592 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:58.588+0000 7efe5af9a640 1 -- 172.21.15.169:0/2424286076 wait complete. 2023-10-26T01:30:58.609 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-10-26T01:30:58.609 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:30:58.819 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:58.817+0000 7f8731541640 1 -- 172.21.15.169:0/2295577689 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8718004d00 msgr2=0x7f8718005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:58.819 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:58.817+0000 7f8731541640 1 --2- 172.21.15.169:0/2295577689 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8718004d00 0x7f8718005180 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f872400c640 tx=0x7f87180057f0 comp rx=0 tx=0).stop 2023-10-26T01:30:58.820 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:58.817+0000 7f872b7fe640 1 -- 172.21.15.169:0/2295577689 reap_dead start 2023-10-26T01:30:58.820 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:58.817+0000 7f8731541640 1 -- 172.21.15.169:0/2295577689 shutdown_connections 2023-10-26T01:30:58.820 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:58.817+0000 7f8731541640 1 -- 172.21.15.169:0/2295577689 >> 172.21.15.169:0/2295577689 conn(0x7f872c069930 msgr2=0x7f872c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:58.820 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:58.817+0000 7f8731541640 1 -- 172.21.15.169:0/2295577689 shutdown_connections 2023-10-26T01:30:58.820 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:58.817+0000 7f8731541640 1 -- 172.21.15.169:0/2295577689 wait complete. 2023-10-26T01:30:58.821 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:58.819+0000 7f8731541640 1 Processor -- start 2023-10-26T01:30:58.821 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:58.819+0000 7f8731541640 1 -- start start 2023-10-26T01:30:58.823 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:58.820+0000 7f8731541640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8718004d00 0x7f872c14ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:58.823 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:58.820+0000 7f8731541640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f872c154c70 0x7f872c1506c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:58.823 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:58.820+0000 7f8731541640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f872c150c00 0x7f872c151080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:58.823 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:58.820+0000 7f8731541640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f872c1550f0 con 0x7f8718004d00 2023-10-26T01:30:58.824 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:58.820+0000 7f8731541640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f872c145220 con 0x7f872c150c00 2023-10-26T01:30:58.824 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:58.820+0000 7f8731541640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f872c1515c0 con 0x7f872c154c70 2023-10-26T01:30:58.824 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:58.820+0000 7f872affd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8718004d00 0x7f872c14ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:58.824 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:58.820+0000 7f872affd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8718004d00 0x7f872c14ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:46182/0 (socket says 172.21.15.169:46182) 2023-10-26T01:30:58.824 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:58.820+0000 7f872affd640 1 -- 172.21.15.169:0/1593648213 learned_addr learned my addr 172.21.15.169:0/1593648213 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:30:58.825 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:58.821+0000 7f872b7fe640 1 --2- 172.21.15.169:0/1593648213 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f872c150c00 0x7f872c151080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:58.825 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:58.821+0000 7f872affd640 1 -- 172.21.15.169:0/1593648213 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f872c154c70 msgr2=0x7f872c1506c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:58.825 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:58.821+0000 7f872affd640 1 --2- 172.21.15.169:0/1593648213 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f872c154c70 0x7f872c1506c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:58.825 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:58.821+0000 7f872affd640 1 -- 172.21.15.169:0/1593648213 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f872c150c00 msgr2=0x7f872c151080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:58.825 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:58.821+0000 7f872affd640 1 --2- 172.21.15.169:0/1593648213 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f872c150c00 0x7f872c151080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:58.826 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:58.821+0000 7f872affd640 1 -- 172.21.15.169:0/1593648213 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f872c152480 con 0x7f8718004d00 2023-10-26T01:30:58.826 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:58.821+0000 7f872b7fe640 1 --2- 172.21.15.169:0/1593648213 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f872c150c00 0x7f872c151080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:30:58.826 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:58.821+0000 7f872affd640 1 --2- 172.21.15.169:0/1593648213 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8718004d00 0x7f872c14ee40 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f871c004e20 tx=0x7f871c001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:58.826 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:58.821+0000 7f8723fff640 1 -- 172.21.15.169:0/1593648213 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f871c0107d0 con 0x7f8718004d00 2023-10-26T01:30:58.826 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:58.821+0000 7f8723fff640 1 -- 172.21.15.169:0/1593648213 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f871c010a40 con 0x7f8718004d00 2023-10-26T01:30:58.827 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:58.821+0000 7f8731541640 1 -- 172.21.15.169:0/1593648213 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f872c152760 con 0x7f8718004d00 2023-10-26T01:30:58.827 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:58.821+0000 7f8723fff640 1 -- 172.21.15.169:0/1593648213 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f871c010e50 con 0x7f8718004d00 2023-10-26T01:30:58.827 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:58.821+0000 7f8731541640 1 -- 172.21.15.169:0/1593648213 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f872c153ac0 con 0x7f8718004d00 2023-10-26T01:30:58.828 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:58.822+0000 7f8731541640 1 -- 172.21.15.169:0/1593648213 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f872c157ab0 con 0x7f8718004d00 2023-10-26T01:30:58.828 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:58.823+0000 7f8723fff640 1 -- 172.21.15.169:0/1593648213 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f871c01b460 con 0x7f8718004d00 2023-10-26T01:30:58.828 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:58.824+0000 7f8723fff640 1 --2- 172.21.15.169:0/1593648213 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f870c06ae40 0x7f870c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:58.829 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:58.824+0000 7f8723fff640 1 -- 172.21.15.169:0/1593648213 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f871c08a0d0 con 0x7f8718004d00 2023-10-26T01:30:58.829 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:58.825+0000 7f87227fc640 1 --2- 172.21.15.169:0/1593648213 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f870c06ae40 0x7f870c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:58.829 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:58.826+0000 7f87227fc640 1 --2- 172.21.15.169:0/1593648213 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f870c06ae40 0x7f870c06d300 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f87140058c0 tx=0x7f8714005850 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:58.830 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:58.828+0000 7f8723fff640 1 -- 172.21.15.169:0/1593648213 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f871c055180 con 0x7f8718004d00 2023-10-26T01:30:59.010 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.008+0000 7f8731541640 1 -- 172.21.15.169:0/1593648213 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f872c04db10 con 0x7f8718004d00 2023-10-26T01:30:59.011 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.010+0000 7f8723fff640 1 -- 172.21.15.169:0/1593648213 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16553 (secure 0 0 0) 0x7f871c0587a0 con 0x7f8718004d00 2023-10-26T01:30:59.012 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:30:59.012 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":21,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:30:43.078706+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-26T01:30:39.437631+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-26T01:30:40.561774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:30:59.015 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.013+0000 7f8731541640 1 -- 172.21.15.169:0/1593648213 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f870c06ae40 msgr2=0x7f870c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:59.015 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.013+0000 7f8731541640 1 --2- 172.21.15.169:0/1593648213 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f870c06ae40 0x7f870c06d300 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f87140058c0 tx=0x7f8714005850 comp rx=0 tx=0).stop 2023-10-26T01:30:59.015 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.013+0000 7f8731541640 1 -- 172.21.15.169:0/1593648213 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8718004d00 msgr2=0x7f872c14ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:59.015 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.013+0000 7f8731541640 1 --2- 172.21.15.169:0/1593648213 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8718004d00 0x7f872c14ee40 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f871c004e20 tx=0x7f871c001630 comp rx=0 tx=0).stop 2023-10-26T01:30:59.015 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.013+0000 7f8731541640 1 -- 172.21.15.169:0/1593648213 shutdown_connections 2023-10-26T01:30:59.016 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.013+0000 7f8731541640 1 --2- 172.21.15.169:0/1593648213 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f870c06ae40 0x7f870c06d300 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:59.016 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.013+0000 7f8731541640 1 --2- 172.21.15.169:0/1593648213 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f872c150c00 0x7f872c151080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:59.016 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.013+0000 7f8731541640 1 --2- 172.21.15.169:0/1593648213 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f872c154c70 0x7f872c1506c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:59.016 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.013+0000 7f8731541640 1 --2- 172.21.15.169:0/1593648213 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8718004d00 0x7f872c14ee40 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:59.016 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.013+0000 7f8731541640 1 -- 172.21.15.169:0/1593648213 >> 172.21.15.169:0/1593648213 conn(0x7f872c069930 msgr2=0x7f872c0b10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:59.017 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.013+0000 7f8731541640 1 -- 172.21.15.169:0/1593648213 shutdown_connections 2023-10-26T01:30:59.017 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.013+0000 7f8731541640 1 -- 172.21.15.169:0/1593648213 wait complete. 2023-10-26T01:30:59.034 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T01:30:32.200808+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-26T01:30:39.437631+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-26T01:30:40.561774+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T01:30:59.035 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T01:30:59.255 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.253+0000 7f7497ee7640 1 -- 172.21.15.169:0/1717275248 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f74900b0a80 msgr2=0x7f7490158f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:59.255 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.253+0000 7f7497ee7640 1 --2- 172.21.15.169:0/1717275248 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f74900b0a80 0x7f7490158f70 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7484009830 tx=0x7f748402bd60 comp rx=0 tx=0).stop 2023-10-26T01:30:59.256 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.253+0000 7f7497ee7640 1 -- 172.21.15.169:0/1717275248 shutdown_connections 2023-10-26T01:30:59.256 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.253+0000 7f7497ee7640 1 --2- 172.21.15.169:0/1717275248 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f74900b0a80 0x7f7490158f70 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:59.256 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.253+0000 7f7497ee7640 1 --2- 172.21.15.169:0/1717275248 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f74900ab070 0x7f74900ac870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:59.256 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.253+0000 7f7497ee7640 1 -- 172.21.15.169:0/1717275248 >> 172.21.15.169:0/1717275248 conn(0x7f7490069930 msgr2=0x7f7490069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:59.256 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.253+0000 7f7497ee7640 1 -- 172.21.15.169:0/1717275248 shutdown_connections 2023-10-26T01:30:59.257 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.253+0000 7f7497ee7640 1 -- 172.21.15.169:0/1717275248 wait complete. 2023-10-26T01:30:59.259 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.255+0000 7f7497ee7640 1 Processor -- start 2023-10-26T01:30:59.259 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.255+0000 7f7497ee7640 1 -- start start 2023-10-26T01:30:59.259 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.256+0000 7f7497ee7640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f74900ab070 0x7f7490147e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:59.259 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.256+0000 7f7497ee7640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f74900cd760 0x7f749014ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:59.260 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.256+0000 7f7497ee7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f74901484b0 0x7f749014cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:59.260 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.256+0000 7f7497ee7640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f749014a880 con 0x7f74900cd760 2023-10-26T01:30:59.260 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.256+0000 7f7497ee7640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f749014b250 con 0x7f74901484b0 2023-10-26T01:30:59.260 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.256+0000 7f7497ee7640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7490149cb0 con 0x7f74900ab070 2023-10-26T01:30:59.260 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.256+0000 7f7495482640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f74900cd760 0x7f749014ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:59.261 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.256+0000 7f7495482640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f74900cd760 0x7f749014ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:46204/0 (socket says 172.21.15.169:46204) 2023-10-26T01:30:59.261 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.256+0000 7f7495482640 1 -- 172.21.15.169:0/3711959335 learned_addr learned my addr 172.21.15.169:0/3711959335 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:30:59.261 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.257+0000 7f7495c83640 1 --2- 172.21.15.169:0/3711959335 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f74900ab070 0x7f7490147e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:59.261 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.257+0000 7f7495482640 1 -- 172.21.15.169:0/3711959335 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f74900ab070 msgr2=0x7f7490147e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:59.261 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.257+0000 7f7495482640 1 --2- 172.21.15.169:0/3711959335 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f74900ab070 0x7f7490147e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:59.262 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.257+0000 7f7496484640 1 --2- 172.21.15.169:0/3711959335 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f74901484b0 0x7f749014cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:59.262 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.257+0000 7f7495482640 1 -- 172.21.15.169:0/3711959335 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f74901484b0 msgr2=0x7f749014cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:59.262 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.257+0000 7f7495482640 1 --2- 172.21.15.169:0/3711959335 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f74901484b0 0x7f749014cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:59.262 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.257+0000 7f7495482640 1 -- 172.21.15.169:0/3711959335 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7484009490 con 0x7f74900cd760 2023-10-26T01:30:59.262 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.257+0000 7f7495c83640 1 --2- 172.21.15.169:0/3711959335 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f74900ab070 0x7f7490147e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:30:59.263 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.257+0000 7f7496484640 1 --2- 172.21.15.169:0/3711959335 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f74901484b0 0x7f749014cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:30:59.263 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.257+0000 7f7495482640 1 --2- 172.21.15.169:0/3711959335 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f74900cd760 0x7f749014ad10 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f7484009950 tx=0x7f74840038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:59.263 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.257+0000 7f748e7fc640 1 -- 172.21.15.169:0/3711959335 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7484003ea0 con 0x7f74900cd760 2023-10-26T01:30:59.263 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.257+0000 7f748e7fc640 1 -- 172.21.15.169:0/3711959335 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7484034550 con 0x7f74900cd760 2023-10-26T01:30:59.263 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.257+0000 7f748e7fc640 1 -- 172.21.15.169:0/3711959335 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7484034960 con 0x7f74900cd760 2023-10-26T01:30:59.263 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.257+0000 7f7497ee7640 1 -- 172.21.15.169:0/3711959335 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7490149f50 con 0x7f74900cd760 2023-10-26T01:30:59.264 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.258+0000 7f7497ee7640 1 -- 172.21.15.169:0/3711959335 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f749014a510 con 0x7f74900cd760 2023-10-26T01:30:59.265 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.259+0000 7f7497ee7640 1 -- 172.21.15.169:0/3711959335 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7490066ad0 con 0x7f74900cd760 2023-10-26T01:30:59.265 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.260+0000 7f748e7fc640 1 -- 172.21.15.169:0/3711959335 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f748403b040 con 0x7f74900cd760 2023-10-26T01:30:59.265 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.260+0000 7f748e7fc640 1 --2- 172.21.15.169:0/3711959335 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f747c06ae40 0x7f747c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:59.265 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.261+0000 7f748e7fc640 1 -- 172.21.15.169:0/3711959335 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f74840abba0 con 0x7f74900cd760 2023-10-26T01:30:59.265 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.261+0000 7f7495c83640 1 --2- 172.21.15.169:0/3711959335 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f747c06ae40 0x7f747c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:59.266 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.262+0000 7f7495c83640 1 --2- 172.21.15.169:0/3711959335 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f747c06ae40 0x7f747c06d300 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7480005fd0 tx=0x7f74800093e0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:59.266 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.265+0000 7f748e7fc640 1 -- 172.21.15.169:0/3711959335 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7484076c50 con 0x7f74900cd760 2023-10-26T01:30:59.457 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.455+0000 7f7497ee7640 1 -- 172.21.15.169:0/3711959335 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f749004da70 con 0x7f74900cd760 2023-10-26T01:30:59.458 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.457+0000 7f748e7fc640 1 -- 172.21.15.169:0/3711959335 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f74840390a0 con 0x7f74900cd760 2023-10-26T01:30:59.458 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 1 2023-10-26T01:30:59.461 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.459+0000 7f7497ee7640 1 -- 172.21.15.169:0/3711959335 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f747c06ae40 msgr2=0x7f747c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:59.461 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.459+0000 7f7497ee7640 1 --2- 172.21.15.169:0/3711959335 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f747c06ae40 0x7f747c06d300 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7480005fd0 tx=0x7f74800093e0 comp rx=0 tx=0).stop 2023-10-26T01:30:59.461 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.459+0000 7f7497ee7640 1 -- 172.21.15.169:0/3711959335 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f74900cd760 msgr2=0x7f749014ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:59.461 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.459+0000 7f7497ee7640 1 --2- 172.21.15.169:0/3711959335 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f74900cd760 0x7f749014ad10 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f7484009950 tx=0x7f74840038a0 comp rx=0 tx=0).stop 2023-10-26T01:30:59.462 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.459+0000 7f7497ee7640 1 -- 172.21.15.169:0/3711959335 shutdown_connections 2023-10-26T01:30:59.462 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.459+0000 7f7497ee7640 1 --2- 172.21.15.169:0/3711959335 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f747c06ae40 0x7f747c06d300 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:59.462 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.459+0000 7f7497ee7640 1 --2- 172.21.15.169:0/3711959335 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f74900cd760 0x7f749014ad10 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:59.462 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.459+0000 7f7497ee7640 1 --2- 172.21.15.169:0/3711959335 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f74901484b0 0x7f749014cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:59.462 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.459+0000 7f7497ee7640 1 --2- 172.21.15.169:0/3711959335 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f74900ab070 0x7f7490147e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:59.463 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.459+0000 7f7497ee7640 1 -- 172.21.15.169:0/3711959335 >> 172.21.15.169:0/3711959335 conn(0x7f7490069930 msgr2=0x7f74900aa250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:59.463 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.459+0000 7f7497ee7640 1 -- 172.21.15.169:0/3711959335 shutdown_connections 2023-10-26T01:30:59.463 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.459+0000 7f7497ee7640 1 -- 172.21.15.169:0/3711959335 wait complete. 2023-10-26T01:30:59.480 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-26T01:30:59.701 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.699+0000 7f22f63a9640 1 -- 172.21.15.169:0/3898106545 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22f015b620 msgr2=0x7f22f00cd710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:59.702 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.699+0000 7f22f63a9640 1 --2- 172.21.15.169:0/3898106545 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22f015b620 0x7f22f00cd710 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f22f00b2140 tx=0x7f22e002c1d0 comp rx=0 tx=0).stop 2023-10-26T01:30:59.702 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.699+0000 7f22f63a9640 1 -- 172.21.15.169:0/3898106545 shutdown_connections 2023-10-26T01:30:59.702 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.699+0000 7f22f63a9640 1 --2- 172.21.15.169:0/3898106545 >> v2:172.21.15.170:3300/0 conn(0x7f22f00c9d00 0x7f22f00cdc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:59.702 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.699+0000 7f22f63a9640 1 --2- 172.21.15.169:0/3898106545 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22f015b620 0x7f22f00cd710 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:59.702 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.699+0000 7f22f63a9640 1 -- 172.21.15.169:0/3898106545 >> 172.21.15.169:0/3898106545 conn(0x7f22f0069930 msgr2=0x7f22f0069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:59.703 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.700+0000 7f22f63a9640 1 -- 172.21.15.169:0/3898106545 shutdown_connections 2023-10-26T01:30:59.703 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.700+0000 7f22f63a9640 1 -- 172.21.15.169:0/3898106545 wait complete. 2023-10-26T01:30:59.704 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.701+0000 7f22f63a9640 1 Processor -- start 2023-10-26T01:30:59.705 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.701+0000 7f22f63a9640 1 -- start start 2023-10-26T01:30:59.705 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.702+0000 7f22f63a9640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f22f00ac320 0x7f22f01f4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:59.705 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.702+0000 7f22f63a9640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22f00c9d00 0x7f22f00cbfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:59.705 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.702+0000 7f22f63a9640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f22f015b620 0x7f22f00cc4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:59.706 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.702+0000 7f22f63a9640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22f00cd0c0 con 0x7f22f00c9d00 2023-10-26T01:30:59.706 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.702+0000 7f22f63a9640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22f013f860 con 0x7f22f015b620 2023-10-26T01:30:59.706 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.702+0000 7f22f63a9640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22f00ccb00 con 0x7f22f00ac320 2023-10-26T01:30:59.706 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.703+0000 7f22ef7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22f00c9d00 0x7f22f00cbfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:59.706 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.703+0000 7f22ef7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22f00c9d00 0x7f22f00cbfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:46224/0 (socket says 172.21.15.169:46224) 2023-10-26T01:30:59.707 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.703+0000 7f22ef7fe640 1 -- 172.21.15.169:0/1328168865 learned_addr learned my addr 172.21.15.169:0/1328168865 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:30:59.707 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.703+0000 7f22effff640 1 --2- 172.21.15.169:0/1328168865 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f22f00ac320 0x7f22f01f4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:59.707 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.703+0000 7f22f4946640 1 --2- 172.21.15.169:0/1328168865 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f22f015b620 0x7f22f00cc4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:59.707 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.703+0000 7f22ef7fe640 1 -- 172.21.15.169:0/1328168865 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f22f00ac320 msgr2=0x7f22f01f4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:59.707 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.703+0000 7f22ef7fe640 1 --2- 172.21.15.169:0/1328168865 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f22f00ac320 0x7f22f01f4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:59.708 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.703+0000 7f22ef7fe640 1 -- 172.21.15.169:0/1328168865 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f22f015b620 msgr2=0x7f22f00cc4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:59.708 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.703+0000 7f22ef7fe640 1 --2- 172.21.15.169:0/1328168865 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f22f015b620 0x7f22f00cc4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:59.710 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.703+0000 7f22ef7fe640 1 -- 172.21.15.169:0/1328168865 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22e0009900 con 0x7f22f00c9d00 2023-10-26T01:30:59.711 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.703+0000 7f22ef7fe640 1 --2- 172.21.15.169:0/1328168865 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22f00c9d00 0x7f22f00cbfb0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f22e400b4a0 tx=0x7f22e400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:59.712 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.704+0000 7f22ecff9640 1 -- 172.21.15.169:0/1328168865 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e4010610 con 0x7f22f00c9d00 2023-10-26T01:30:59.712 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.704+0000 7f22ecff9640 1 -- 172.21.15.169:0/1328168865 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f22e4010860 con 0x7f22f00c9d00 2023-10-26T01:30:59.712 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.704+0000 7f22f63a9640 1 -- 172.21.15.169:0/1328168865 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22f00ccde0 con 0x7f22f00c9d00 2023-10-26T01:30:59.712 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.704+0000 7f22ecff9640 1 -- 172.21.15.169:0/1328168865 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e4010c70 con 0x7f22f00c9d00 2023-10-26T01:30:59.712 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.704+0000 7f22f63a9640 1 -- 172.21.15.169:0/1328168865 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22f01f5220 con 0x7f22f00c9d00 2023-10-26T01:30:59.713 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.705+0000 7f22f63a9640 1 -- 172.21.15.169:0/1328168865 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22f004db10 con 0x7f22f00c9d00 2023-10-26T01:30:59.713 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.706+0000 7f22ecff9640 1 -- 172.21.15.169:0/1328168865 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f22e4017360 con 0x7f22f00c9d00 2023-10-26T01:30:59.713 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.707+0000 7f22ecff9640 1 --2- 172.21.15.169:0/1328168865 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f22dc06ae70 0x7f22dc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:30:59.713 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.707+0000 7f22ecff9640 1 -- 172.21.15.169:0/1328168865 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f22e40894a0 con 0x7f22f00c9d00 2023-10-26T01:30:59.713 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.707+0000 7f22effff640 1 --2- 172.21.15.169:0/1328168865 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f22dc06ae70 0x7f22dc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:30:59.714 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.708+0000 7f22effff640 1 --2- 172.21.15.169:0/1328168865 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f22dc06ae70 0x7f22dc06d330 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f22e0036000 tx=0x7f22e0004ab0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:30:59.714 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.711+0000 7f22ecff9640 1 -- 172.21.15.169:0/1328168865 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f22e4057fd0 con 0x7f22f00c9d00 2023-10-26T01:30:59.900 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.898+0000 7f22f63a9640 1 -- 172.21.15.169:0/1328168865 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f22f0069f20 con 0x7f22f00c9d00 2023-10-26T01:30:59.901 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.899+0000 7f22ecff9640 1 -- 172.21.15.169:0/1328168865 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f22e4057b60 con 0x7f22f00c9d00 2023-10-26T01:30:59.901 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 32 2023-10-26T01:30:59.904 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.902+0000 7f22f63a9640 1 -- 172.21.15.169:0/1328168865 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f22dc06ae70 msgr2=0x7f22dc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:59.904 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.902+0000 7f22f63a9640 1 --2- 172.21.15.169:0/1328168865 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f22dc06ae70 0x7f22dc06d330 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f22e0036000 tx=0x7f22e0004ab0 comp rx=0 tx=0).stop 2023-10-26T01:30:59.904 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.902+0000 7f22f63a9640 1 -- 172.21.15.169:0/1328168865 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22f00c9d00 msgr2=0x7f22f00cbfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:30:59.904 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.902+0000 7f22f63a9640 1 --2- 172.21.15.169:0/1328168865 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22f00c9d00 0x7f22f00cbfb0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f22e400b4a0 tx=0x7f22e400b960 comp rx=0 tx=0).stop 2023-10-26T01:30:59.905 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.902+0000 7f22f63a9640 1 -- 172.21.15.169:0/1328168865 shutdown_connections 2023-10-26T01:30:59.905 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.902+0000 7f22f63a9640 1 --2- 172.21.15.169:0/1328168865 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f22dc06ae70 0x7f22dc06d330 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:59.905 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.902+0000 7f22f63a9640 1 --2- 172.21.15.169:0/1328168865 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22f00c9d00 0x7f22f00cbfb0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:59.905 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.902+0000 7f22f63a9640 1 --2- 172.21.15.169:0/1328168865 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f22f015b620 0x7f22f00cc4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:59.905 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.902+0000 7f22f63a9640 1 --2- 172.21.15.169:0/1328168865 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f22f00ac320 0x7f22f01f4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:30:59.906 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.902+0000 7f22f63a9640 1 -- 172.21.15.169:0/1328168865 >> 172.21.15.169:0/1328168865 conn(0x7f22f0069930 msgr2=0x7f22f006bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:30:59.906 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.902+0000 7f22f63a9640 1 -- 172.21.15.169:0/1328168865 shutdown_connections 2023-10-26T01:30:59.906 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:30:59.902+0000 7f22f63a9640 1 -- 172.21.15.169:0/1328168865 wait complete. 2023-10-26T01:30:59.924 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-26T01:31:00.143 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:00.141+0000 7f6fa098a640 1 -- 172.21.15.169:0/4262304145 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f9c0b0c80 msgr2=0x7f6f9c0cd710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:00.144 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:00.141+0000 7f6fa098a640 1 --2- 172.21.15.169:0/4262304145 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f9c0b0c80 0x7f6f9c0cd710 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c0b2670 tx=0x7f6f9402c260 comp rx=0 tx=0).stop 2023-10-26T01:31:00.144 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:00.141+0000 7f6fa098a640 1 -- 172.21.15.169:0/4262304145 shutdown_connections 2023-10-26T01:31:00.144 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:00.141+0000 7f6fa098a640 1 --2- 172.21.15.169:0/4262304145 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f9c0b0c80 0x7f6f9c0cd710 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:00.144 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:00.141+0000 7f6fa098a640 1 --2- 172.21.15.169:0/4262304145 >> v2:172.21.15.170:3300/0 conn(0x7f6f9c1597c0 0x7f6f9c0ac200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:00.145 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:00.141+0000 7f6fa098a640 1 -- 172.21.15.169:0/4262304145 >> 172.21.15.169:0/4262304145 conn(0x7f6f9c069930 msgr2=0x7f6f9c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:00.145 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:00.142+0000 7f6fa098a640 1 -- 172.21.15.169:0/4262304145 shutdown_connections 2023-10-26T01:31:00.145 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:00.142+0000 7f6fa098a640 1 -- 172.21.15.169:0/4262304145 wait complete. 2023-10-26T01:31:00.147 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:00.143+0000 7f6fa098a640 1 Processor -- start 2023-10-26T01:31:00.147 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:00.143+0000 7f6fa098a640 1 -- start start 2023-10-26T01:31:00.148 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:00.144+0000 7f6fa098a640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6f9c0b0c80 0x7f6f9c1498e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:00.148 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:00.144+0000 7f6fa098a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f9c1593e0 0x7f6f9c097440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:00.148 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:00.144+0000 7f6fa098a640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f9c1597c0 0x7f6f9c06f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:00.148 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:00.144+0000 7f6fa098a640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9c09abb0 con 0x7f6f9c1597c0 2023-10-26T01:31:00.149 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:00.144+0000 7f6fa098a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9c099fa0 con 0x7f6f9c1593e0 2023-10-26T01:31:00.149 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:00.144+0000 7f6fa098a640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9c09a280 con 0x7f6f9c0b0c80 2023-10-26T01:31:00.149 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:00.145+0000 7f6f9ad9d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f9c1597c0 0x7f6f9c06f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:00.149 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:00.145+0000 7f6f9ad9d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f9c1597c0 0x7f6f9c06f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:46246/0 (socket says 172.21.15.169:46246) 2023-10-26T01:31:00.149 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:00.145+0000 7f6f9ad9d640 1 -- 172.21.15.169:0/1521893234 learned_addr learned my addr 172.21.15.169:0/1521893234 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:31:00.150 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:00.145+0000 7f6f93fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f9c1593e0 0x7f6f9c097440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:00.150 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:00.145+0000 7f6f9a59c640 1 --2- 172.21.15.169:0/1521893234 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6f9c0b0c80 0x7f6f9c1498e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:00.150 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:00.145+0000 7f6f9ad9d640 1 -- 172.21.15.169:0/1521893234 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6f9c0b0c80 msgr2=0x7f6f9c1498e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:00.150 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:00.145+0000 7f6f9ad9d640 1 --2- 172.21.15.169:0/1521893234 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6f9c0b0c80 0x7f6f9c1498e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:00.150 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:00.145+0000 7f6f9ad9d640 1 -- 172.21.15.169:0/1521893234 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f9c1593e0 msgr2=0x7f6f9c097440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:00.151 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:00.145+0000 7f6f9ad9d640 1 --2- 172.21.15.169:0/1521893234 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f9c1593e0 0x7f6f9c097440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:00.151 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:00.145+0000 7f6f9ad9d640 1 -- 172.21.15.169:0/1521893234 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f94009900 con 0x7f6f9c1597c0 2023-10-26T01:31:00.151 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:00.146+0000 7f6f9ad9d640 1 --2- 172.21.15.169:0/1521893234 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f9c1597c0 0x7f6f9c06f6f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6f94037040 tx=0x7f6f94003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:00.153 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:00.146+0000 7f6f92ffd640 1 -- 172.21.15.169:0/1521893234 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f94004250 con 0x7f6f9c1597c0 2023-10-26T01:31:00.153 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:00.146+0000 7f6f92ffd640 1 -- 172.21.15.169:0/1521893234 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6f940044a0 con 0x7f6f9c1597c0 2023-10-26T01:31:00.153 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:00.146+0000 7f6f92ffd640 1 -- 172.21.15.169:0/1521893234 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f94004890 con 0x7f6f9c1597c0 2023-10-26T01:31:00.153 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:00.146+0000 7f6fa098a640 1 -- 172.21.15.169:0/1521893234 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f9c09a560 con 0x7f6f9c1597c0 2023-10-26T01:31:00.154 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:00.146+0000 7f6fa098a640 1 -- 172.21.15.169:0/1521893234 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f9c149e20 con 0x7f6f9c1597c0 2023-10-26T01:31:00.154 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:00.147+0000 7f6fa098a640 1 -- 172.21.15.169:0/1521893234 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f9c09ad60 con 0x7f6f9c1597c0 2023-10-26T01:31:00.154 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:00.148+0000 7f6f92ffd640 1 -- 172.21.15.169:0/1521893234 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f6f9403b040 con 0x7f6f9c1597c0 2023-10-26T01:31:00.154 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:00.149+0000 7f6f92ffd640 1 --2- 172.21.15.169:0/1521893234 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f6f7c06ae40 0x7f6f7c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:00.155 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:00.149+0000 7f6f92ffd640 1 -- 172.21.15.169:0/1521893234 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6f940ac500 con 0x7f6f9c1597c0 2023-10-26T01:31:00.155 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:00.150+0000 7f6f9a59c640 1 --2- 172.21.15.169:0/1521893234 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f6f7c06ae40 0x7f6f7c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:00.155 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:00.151+0000 7f6f9a59c640 1 --2- 172.21.15.169:0/1521893234 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f6f7c06ae40 0x7f6f7c06d300 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c003d90 tx=0x7f6f8c009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:00.156 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:00.153+0000 7f6f92ffd640 1 -- 172.21.15.169:0/1521893234 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6f9c09ad60 con 0x7f6f9c1597c0 2023-10-26T01:31:00.340 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:00.339+0000 7f6fa098a640 1 -- 172.21.15.169:0/1521893234 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6f9c04db10 con 0x7f6f9c1597c0 2023-10-26T01:31:00.341 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:00.339+0000 7f6f92ffd640 1 -- 172.21.15.169:0/1521893234 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6f940775a0 con 0x7f6f9c1597c0 2023-10-26T01:31:00.342 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 64 2023-10-26T01:31:00.344 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:00.342+0000 7f6fa098a640 1 -- 172.21.15.169:0/1521893234 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f6f7c06ae40 msgr2=0x7f6f7c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:00.344 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:00.342+0000 7f6fa098a640 1 --2- 172.21.15.169:0/1521893234 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f6f7c06ae40 0x7f6f7c06d300 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c003d90 tx=0x7f6f8c009040 comp rx=0 tx=0).stop 2023-10-26T01:31:00.344 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:00.342+0000 7f6fa098a640 1 -- 172.21.15.169:0/1521893234 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f9c1597c0 msgr2=0x7f6f9c06f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:00.344 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:00.342+0000 7f6fa098a640 1 --2- 172.21.15.169:0/1521893234 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f9c1597c0 0x7f6f9c06f6f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6f94037040 tx=0x7f6f94003c10 comp rx=0 tx=0).stop 2023-10-26T01:31:00.344 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:00.342+0000 7f6fa098a640 1 -- 172.21.15.169:0/1521893234 shutdown_connections 2023-10-26T01:31:00.345 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:00.342+0000 7f6fa098a640 1 --2- 172.21.15.169:0/1521893234 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f6f7c06ae40 0x7f6f7c06d300 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:00.345 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:00.342+0000 7f6fa098a640 1 --2- 172.21.15.169:0/1521893234 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f9c1597c0 0x7f6f9c06f6f0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:00.345 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:00.342+0000 7f6fa098a640 1 --2- 172.21.15.169:0/1521893234 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f9c1593e0 0x7f6f9c097440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:00.345 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:00.342+0000 7f6fa098a640 1 --2- 172.21.15.169:0/1521893234 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6f9c0b0c80 0x7f6f9c1498e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:00.345 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:00.342+0000 7f6fa098a640 1 -- 172.21.15.169:0/1521893234 >> 172.21.15.169:0/1521893234 conn(0x7f6f9c069930 msgr2=0x7f6f9c154680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:00.346 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:00.342+0000 7f6fa098a640 1 -- 172.21.15.169:0/1521893234 shutdown_connections 2023-10-26T01:31:00.346 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:00.342+0000 7f6fa098a640 1 -- 172.21.15.169:0/1521893234 wait complete. 2023-10-26T01:31:00.362 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T01:31:00.574 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:00.572+0000 7f981418e640 1 -- 172.21.15.169:0/2252237902 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f980c14e790 msgr2=0x7f980c15a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:00.575 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:00.572+0000 7f981418e640 1 --2- 172.21.15.169:0/2252237902 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f980c14e790 0x7f980c15a520 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f9804005fd0 tx=0x7f980402c170 comp rx=0 tx=0).stop 2023-10-26T01:31:00.575 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:00.573+0000 7f981418e640 1 -- 172.21.15.169:0/2252237902 shutdown_connections 2023-10-26T01:31:00.575 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:00.573+0000 7f981418e640 1 --2- 172.21.15.169:0/2252237902 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f980c14e790 0x7f980c15a520 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:00.575 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:00.573+0000 7f981418e640 1 -- 172.21.15.169:0/2252237902 >> 172.21.15.169:0/2252237902 conn(0x7f980c069930 msgr2=0x7f980c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:00.576 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:00.573+0000 7f981418e640 1 -- 172.21.15.169:0/2252237902 shutdown_connections 2023-10-26T01:31:00.576 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:00.573+0000 7f981418e640 1 -- 172.21.15.169:0/2252237902 wait complete. 2023-10-26T01:31:00.576 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:00.574+0000 7f981418e640 1 Processor -- start 2023-10-26T01:31:00.577 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:00.574+0000 7f981418e640 1 -- start start 2023-10-26T01:31:00.578 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:00.575+0000 7f981418e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f980c06ce20 0x7f980c162a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:00.578 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:00.575+0000 7f981418e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f980c14e790 0x7f980c144fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:00.579 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:00.575+0000 7f981418e640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f980c158850 0x7f980c145720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:00.579 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:00.575+0000 7f981418e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f980c156a30 con 0x7f980c14e790 2023-10-26T01:31:00.579 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:00.575+0000 7f981418e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f980c162fb0 con 0x7f980c06ce20 2023-10-26T01:31:00.579 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:00.575+0000 7f981418e640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f980c1454f0 con 0x7f980c158850 2023-10-26T01:31:00.579 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:00.576+0000 7f9811729640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f980c14e790 0x7f980c144fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:00.580 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:00.576+0000 7f9811729640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f980c14e790 0x7f980c144fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:46264/0 (socket says 172.21.15.169:46264) 2023-10-26T01:31:00.580 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:00.576+0000 7f9811729640 1 -- 172.21.15.169:0/2243094892 learned_addr learned my addr 172.21.15.169:0/2243094892 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:31:00.580 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:00.576+0000 7f9811f2a640 1 --2- 172.21.15.169:0/2243094892 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f980c06ce20 0x7f980c162a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:00.580 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:00.576+0000 7f981272b640 1 --2- 172.21.15.169:0/2243094892 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f980c158850 0x7f980c145720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:00.580 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:00.576+0000 7f9811729640 1 -- 172.21.15.169:0/2243094892 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f980c158850 msgr2=0x7f980c145720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:00.581 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:00.576+0000 7f9811729640 1 --2- 172.21.15.169:0/2243094892 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f980c158850 0x7f980c145720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:00.581 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:00.576+0000 7f9811729640 1 -- 172.21.15.169:0/2243094892 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f980c06ce20 msgr2=0x7f980c162a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:00.581 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:00.576+0000 7f9811729640 1 --2- 172.21.15.169:0/2243094892 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f980c06ce20 0x7f980c162a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:00.581 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:00.576+0000 7f9811729640 1 -- 172.21.15.169:0/2243094892 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9804009840 con 0x7f980c14e790 2023-10-26T01:31:00.581 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:00.576+0000 7f9811729640 1 --2- 172.21.15.169:0/2243094892 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f980c14e790 0x7f980c144fb0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f97fc00c400 tx=0x7f97fc00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:00.582 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:00.577+0000 7f98027fc640 1 -- 172.21.15.169:0/2243094892 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97fc0052d0 con 0x7f980c14e790 2023-10-26T01:31:00.582 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:00.577+0000 7f98027fc640 1 -- 172.21.15.169:0/2243094892 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f97fc0035f0 con 0x7f980c14e790 2023-10-26T01:31:00.582 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:00.577+0000 7f981418e640 1 -- 172.21.15.169:0/2243094892 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f980c155e20 con 0x7f980c14e790 2023-10-26T01:31:00.582 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:00.577+0000 7f98027fc640 1 -- 172.21.15.169:0/2243094892 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97fc003a00 con 0x7f980c14e790 2023-10-26T01:31:00.582 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:00.577+0000 7f981418e640 1 -- 172.21.15.169:0/2243094892 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f980c1563e0 con 0x7f980c14e790 2023-10-26T01:31:00.583 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:00.578+0000 7f981418e640 1 -- 172.21.15.169:0/2243094892 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f980c156ba0 con 0x7f980c14e790 2023-10-26T01:31:00.584 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:00.579+0000 7f98027fc640 1 -- 172.21.15.169:0/2243094892 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f97fc003b60 con 0x7f980c14e790 2023-10-26T01:31:00.584 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:00.580+0000 7f98027fc640 1 --2- 172.21.15.169:0/2243094892 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f97f006ae40 0x7f97f006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:00.584 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:00.580+0000 7f98027fc640 1 -- 172.21.15.169:0/2243094892 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f97fc088e30 con 0x7f980c14e790 2023-10-26T01:31:00.584 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:00.581+0000 7f9811f2a640 1 --2- 172.21.15.169:0/2243094892 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f97f006ae40 0x7f97f006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:00.584 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:00.582+0000 7f9811f2a640 1 --2- 172.21.15.169:0/2243094892 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f97f006ae40 0x7f97f006d300 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9808000f60 tx=0x7f9808009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:00.586 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:00.585+0000 7f98027fc640 1 -- 172.21.15.169:0/2243094892 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f97fc053ec0 con 0x7f980c14e790 2023-10-26T01:31:00.796 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:00.795+0000 7f981418e640 1 -- 172.21.15.169:0/2243094892 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f980c04db10 con 0x7f980c14e790 2023-10-26T01:31:00.797 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:00.796+0000 7f98027fc640 1 -- 172.21.15.169:0/2243094892 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4916 (secure 0 0 0) 0x7f97fc017090 con 0x7f980c14e790 2023-10-26T01:31:00.798 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:31:00.798 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4298,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1809801299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":1809801299},{"type":"v1","addr":"172.21.15.169:6835","nonce":1809801299}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4304,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/2248566141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2248566141},{"type":"v1","addr":"172.21.15.170:6836","nonce":2248566141}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4317,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/2383616530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2383616530},{"type":"v1","addr":"172.21.15.169:6834","nonce":2383616530}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T01:30:41.545068+0000","modified":"2023-10-26T01:30:44.577169+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4338},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4338":{"gid":4338,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6837/590609220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":590609220},{"type":"v1","addr":"172.21.15.170:6837","nonce":590609220}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-26T01:31:00.798 INFO:teuthology.orchestra.run.smithi169.stderr:dumped fsmap epoch 7 2023-10-26T01:31:00.800 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:00.798+0000 7f981418e640 1 -- 172.21.15.169:0/2243094892 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f97f006ae40 msgr2=0x7f97f006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:00.800 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:00.798+0000 7f981418e640 1 --2- 172.21.15.169:0/2243094892 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f97f006ae40 0x7f97f006d300 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9808000f60 tx=0x7f9808009040 comp rx=0 tx=0).stop 2023-10-26T01:31:00.800 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:00.798+0000 7f981418e640 1 -- 172.21.15.169:0/2243094892 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f980c14e790 msgr2=0x7f980c144fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:00.801 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:00.798+0000 7f981418e640 1 --2- 172.21.15.169:0/2243094892 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f980c14e790 0x7f980c144fb0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f97fc00c400 tx=0x7f97fc00c8c0 comp rx=0 tx=0).stop 2023-10-26T01:31:00.801 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:00.798+0000 7f981418e640 1 -- 172.21.15.169:0/2243094892 shutdown_connections 2023-10-26T01:31:00.801 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:00.798+0000 7f981418e640 1 --2- 172.21.15.169:0/2243094892 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f97f006ae40 0x7f97f006d300 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:00.801 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:00.798+0000 7f981418e640 1 --2- 172.21.15.169:0/2243094892 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f980c14e790 0x7f980c144fb0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:00.801 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:00.798+0000 7f981418e640 1 --2- 172.21.15.169:0/2243094892 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f980c158850 0x7f980c145720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:00.802 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:00.798+0000 7f981418e640 1 --2- 172.21.15.169:0/2243094892 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f980c06ce20 0x7f980c162a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:00.802 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:00.798+0000 7f981418e640 1 -- 172.21.15.169:0/2243094892 >> 172.21.15.169:0/2243094892 conn(0x7f980c069930 msgr2=0x7f980c14d9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:00.802 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:00.799+0000 7f981418e640 1 -- 172.21.15.169:0/2243094892 shutdown_connections 2023-10-26T01:31:00.802 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:00.799+0000 7f981418e640 1 -- 172.21.15.169:0/2243094892 wait complete. 2023-10-26T01:31:00.818 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:31:01.029 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.027+0000 7f9a19835640 1 -- 172.21.15.169:0/3973895790 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9a1406b8b0 msgr2=0x7f9a1406bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:01.029 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.027+0000 7f9a19835640 1 --2- 172.21.15.169:0/3973895790 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9a1406b8b0 0x7f9a1406bd20 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c005fd0 tx=0x7f9a0c02c0e0 comp rx=0 tx=0).stop 2023-10-26T01:31:01.030 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.027+0000 7f9a19835640 1 -- 172.21.15.169:0/3973895790 shutdown_connections 2023-10-26T01:31:01.030 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.027+0000 7f9a19835640 1 --2- 172.21.15.169:0/3973895790 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9a1406b8b0 0x7f9a1406bd20 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:01.030 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.027+0000 7f9a19835640 1 --2- 172.21.15.169:0/3973895790 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9a14158070 0x7f9a1409a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:01.030 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.027+0000 7f9a19835640 1 -- 172.21.15.169:0/3973895790 >> 172.21.15.169:0/3973895790 conn(0x7f9a140aedd0 msgr2=0x7f9a140af1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:01.030 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.027+0000 7f9a19835640 1 -- 172.21.15.169:0/3973895790 shutdown_connections 2023-10-26T01:31:01.031 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.027+0000 7f9a19835640 1 -- 172.21.15.169:0/3973895790 wait complete. 2023-10-26T01:31:01.031 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.029+0000 7f9a19835640 1 Processor -- start 2023-10-26T01:31:01.031 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.030+0000 7f9a19835640 1 -- start start 2023-10-26T01:31:01.033 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.031+0000 7f9a19835640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a1406b8b0 0x7f9a14146a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:01.033 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.031+0000 7f9a19835640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9a1409ab50 0x7f9a140ae320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:01.033 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.031+0000 7f9a19835640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9a14158070 0x7f9a1414a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:01.034 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.031+0000 7f9a19835640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a14146fc0 con 0x7f9a1406b8b0 2023-10-26T01:31:01.034 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.031+0000 7f9a19835640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a140afd80 con 0x7f9a1409ab50 2023-10-26T01:31:01.034 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.031+0000 7f9a19835640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a140b0060 con 0x7f9a14158070 2023-10-26T01:31:01.034 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.031+0000 7f9a18833640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a1406b8b0 0x7f9a14146a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:01.034 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.031+0000 7f9a18833640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a1406b8b0 0x7f9a14146a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:46276/0 (socket says 172.21.15.169:46276) 2023-10-26T01:31:01.035 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.031+0000 7f9a18833640 1 -- 172.21.15.169:0/1562495205 learned_addr learned my addr 172.21.15.169:0/1562495205 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:31:01.035 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.031+0000 7f9a13fff640 1 --2- 172.21.15.169:0/1562495205 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9a1409ab50 0x7f9a140ae320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:01.035 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.031+0000 7f9a18833640 1 -- 172.21.15.169:0/1562495205 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9a14158070 msgr2=0x7f9a1414a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:01.035 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.031+0000 7f9a19034640 1 --2- 172.21.15.169:0/1562495205 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9a14158070 0x7f9a1414a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:01.035 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.031+0000 7f9a18833640 1 --2- 172.21.15.169:0/1562495205 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9a14158070 0x7f9a1414a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:01.036 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.031+0000 7f9a18833640 1 -- 172.21.15.169:0/1562495205 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9a1409ab50 msgr2=0x7f9a140ae320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:01.036 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.031+0000 7f9a18833640 1 --2- 172.21.15.169:0/1562495205 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9a1409ab50 0x7f9a140ae320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:01.036 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.031+0000 7f9a18833640 1 -- 172.21.15.169:0/1562495205 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a0c009840 con 0x7f9a1406b8b0 2023-10-26T01:31:01.036 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.031+0000 7f9a13fff640 1 --2- 172.21.15.169:0/1562495205 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9a1409ab50 0x7f9a140ae320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:31:01.036 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.032+0000 7f9a18833640 1 --2- 172.21.15.169:0/1562495205 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a1406b8b0 0x7f9a14146a80 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9a0800e790 tx=0x7f9a0800ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:01.037 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.032+0000 7f9a117fa640 1 -- 172.21.15.169:0/1562495205 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a0800ccc0 con 0x7f9a1406b8b0 2023-10-26T01:31:01.038 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.032+0000 7f9a117fa640 1 -- 172.21.15.169:0/1562495205 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9a0800ce20 con 0x7f9a1406b8b0 2023-10-26T01:31:01.038 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.032+0000 7f9a117fa640 1 -- 172.21.15.169:0/1562495205 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a08005280 con 0x7f9a1406b8b0 2023-10-26T01:31:01.038 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.032+0000 7f9a19835640 1 -- 172.21.15.169:0/1562495205 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a140b0340 con 0x7f9a1406b8b0 2023-10-26T01:31:01.038 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.032+0000 7f9a19835640 1 -- 172.21.15.169:0/1562495205 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a140b0900 con 0x7f9a1406b8b0 2023-10-26T01:31:01.038 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.033+0000 7f9a19835640 1 -- 172.21.15.169:0/1562495205 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a14073c10 con 0x7f9a1406b8b0 2023-10-26T01:31:01.039 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.034+0000 7f9a117fa640 1 -- 172.21.15.169:0/1562495205 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f9a080053e0 con 0x7f9a1406b8b0 2023-10-26T01:31:01.039 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.035+0000 7f9a117fa640 1 --2- 172.21.15.169:0/1562495205 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f99f806ae40 0x7f99f806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:01.039 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.036+0000 7f9a117fa640 1 -- 172.21.15.169:0/1562495205 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9a08014070 con 0x7f9a1406b8b0 2023-10-26T01:31:01.039 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.036+0000 7f9a13fff640 1 --2- 172.21.15.169:0/1562495205 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f99f806ae40 0x7f99f806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:01.040 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.037+0000 7f9a13fff640 1 --2- 172.21.15.169:0/1562495205 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f99f806ae40 0x7f99f806d300 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9a140cb700 tx=0x7f9a00009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:01.041 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.040+0000 7f9a117fa640 1 -- 172.21.15.169:0/1562495205 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9a08058b00 con 0x7f9a1406b8b0 2023-10-26T01:31:01.231 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.229+0000 7f9a19835640 1 -- 172.21.15.169:0/1562495205 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9a14158570 con 0x7f9a1406b8b0 2023-10-26T01:31:01.234 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.232+0000 7f9a117fa640 1 -- 172.21.15.169:0/1562495205 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16553 (secure 0 0 0) 0x7f9a0800eea0 con 0x7f9a1406b8b0 2023-10-26T01:31:01.234 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:31:01.235 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":21,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:30:43.078706+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-26T01:30:39.437631+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-26T01:30:40.561774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:31:01.237 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.235+0000 7f9a19835640 1 -- 172.21.15.169:0/1562495205 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f99f806ae40 msgr2=0x7f99f806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:01.237 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.235+0000 7f9a19835640 1 --2- 172.21.15.169:0/1562495205 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f99f806ae40 0x7f99f806d300 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9a140cb700 tx=0x7f9a00009040 comp rx=0 tx=0).stop 2023-10-26T01:31:01.237 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.235+0000 7f9a19835640 1 -- 172.21.15.169:0/1562495205 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a1406b8b0 msgr2=0x7f9a14146a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:01.238 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.235+0000 7f9a19835640 1 --2- 172.21.15.169:0/1562495205 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a1406b8b0 0x7f9a14146a80 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9a0800e790 tx=0x7f9a0800ec50 comp rx=0 tx=0).stop 2023-10-26T01:31:01.238 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.235+0000 7f9a19835640 1 -- 172.21.15.169:0/1562495205 shutdown_connections 2023-10-26T01:31:01.238 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.235+0000 7f9a19835640 1 --2- 172.21.15.169:0/1562495205 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f99f806ae40 0x7f99f806d300 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:01.238 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.235+0000 7f9a19835640 1 --2- 172.21.15.169:0/1562495205 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9a14158070 0x7f9a1414a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:01.238 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.235+0000 7f9a19835640 1 --2- 172.21.15.169:0/1562495205 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9a1409ab50 0x7f9a140ae320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:01.239 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.235+0000 7f9a19835640 1 --2- 172.21.15.169:0/1562495205 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a1406b8b0 0x7f9a14146a80 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:01.239 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.235+0000 7f9a19835640 1 -- 172.21.15.169:0/1562495205 >> 172.21.15.169:0/1562495205 conn(0x7f9a140aedd0 msgr2=0x7f9a140af1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:01.239 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.235+0000 7f9a19835640 1 -- 172.21.15.169:0/1562495205 shutdown_connections 2023-10-26T01:31:01.239 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.236+0000 7f9a19835640 1 -- 172.21.15.169:0/1562495205 wait complete. 2023-10-26T01:31:01.257 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T01:30:32.200808+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-26T01:30:39.437631+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-26T01:30:40.561774+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T01:31:01.258 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T01:31:01.479 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.477+0000 7ff5eaa21640 1 -- 172.21.15.169:0/32142801 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff5e406cba0 msgr2=0x7ff5e40cd440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:01.479 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.477+0000 7ff5eaa21640 1 --2- 172.21.15.169:0/32142801 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff5e406cba0 0x7ff5e40cd440 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff5d4005fd0 tx=0x7ff5d402bd90 comp rx=0 tx=0).stop 2023-10-26T01:31:01.479 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.477+0000 7ff5eaa21640 1 -- 172.21.15.169:0/32142801 shutdown_connections 2023-10-26T01:31:01.480 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.477+0000 7ff5eaa21640 1 --2- 172.21.15.169:0/32142801 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff5e406cba0 0x7ff5e40cd440 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:01.480 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.477+0000 7ff5eaa21640 1 --2- 172.21.15.169:0/32142801 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff5e406b860 0x7ff5e40ccf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:01.480 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.477+0000 7ff5eaa21640 1 -- 172.21.15.169:0/32142801 >> 172.21.15.169:0/32142801 conn(0x7ff5e4069510 msgr2=0x7ff5e4069920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:01.480 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.477+0000 7ff5eaa21640 1 -- 172.21.15.169:0/32142801 shutdown_connections 2023-10-26T01:31:01.480 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.477+0000 7ff5eaa21640 1 -- 172.21.15.169:0/32142801 wait complete. 2023-10-26T01:31:01.481 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.479+0000 7ff5eaa21640 1 Processor -- start 2023-10-26T01:31:01.481 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.479+0000 7ff5eaa21640 1 -- start start 2023-10-26T01:31:01.483 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.481+0000 7ff5eaa21640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff5e406b860 0x7ff5e4151620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:01.483 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.481+0000 7ff5eaa21640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff5e406cba0 0x7ff5e406f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:01.483 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.481+0000 7ff5eaa21640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff5e4151ed0 0x7ff5e406f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:01.484 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.481+0000 7ff5eaa21640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e4143d30 con 0x7ff5e406cba0 2023-10-26T01:31:01.484 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.481+0000 7ff5eaa21640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e41486d0 con 0x7ff5e4151ed0 2023-10-26T01:31:01.484 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.481+0000 7ff5eaa21640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e4070010 con 0x7ff5e406b860 2023-10-26T01:31:01.484 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.481+0000 7ff5e921e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff5e406cba0 0x7ff5e406f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:01.484 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.481+0000 7ff5e921e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff5e406cba0 0x7ff5e406f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:46282/0 (socket says 172.21.15.169:46282) 2023-10-26T01:31:01.485 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.481+0000 7ff5e921e640 1 -- 172.21.15.169:0/3662335657 learned_addr learned my addr 172.21.15.169:0/3662335657 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:31:01.485 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.481+0000 7ff5e9a1f640 1 --2- 172.21.15.169:0/3662335657 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff5e406b860 0x7ff5e4151620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:01.485 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.481+0000 7ff5ea220640 1 --2- 172.21.15.169:0/3662335657 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff5e4151ed0 0x7ff5e406f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:01.485 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.481+0000 7ff5e921e640 1 -- 172.21.15.169:0/3662335657 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff5e406b860 msgr2=0x7ff5e4151620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:01.485 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.481+0000 7ff5e921e640 1 --2- 172.21.15.169:0/3662335657 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff5e406b860 0x7ff5e4151620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:01.485 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.481+0000 7ff5e921e640 1 -- 172.21.15.169:0/3662335657 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff5e4151ed0 msgr2=0x7ff5e406f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:01.486 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.481+0000 7ff5e921e640 1 --2- 172.21.15.169:0/3662335657 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff5e4151ed0 0x7ff5e406f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:01.486 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.481+0000 7ff5e921e640 1 -- 172.21.15.169:0/3662335657 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5e0009490 con 0x7ff5e406cba0 2023-10-26T01:31:01.486 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.481+0000 7ff5ea220640 1 --2- 172.21.15.169:0/3662335657 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff5e4151ed0 0x7ff5e406f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:31:01.486 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.482+0000 7ff5e921e640 1 --2- 172.21.15.169:0/3662335657 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff5e406cba0 0x7ff5e406f490 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff5d4036040 tx=0x7ff5d402da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:01.487 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.482+0000 7ff5da7fc640 1 -- 172.21.15.169:0/3662335657 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5d402de60 con 0x7ff5e406cba0 2023-10-26T01:31:01.488 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.482+0000 7ff5da7fc640 1 -- 172.21.15.169:0/3662335657 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff5d40036e0 con 0x7ff5e406cba0 2023-10-26T01:31:01.488 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.482+0000 7ff5da7fc640 1 -- 172.21.15.169:0/3662335657 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5d4003ab0 con 0x7ff5e406cba0 2023-10-26T01:31:01.488 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.482+0000 7ff5eaa21640 1 -- 172.21.15.169:0/3662335657 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5d4009560 con 0x7ff5e406cba0 2023-10-26T01:31:01.488 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.482+0000 7ff5eaa21640 1 -- 172.21.15.169:0/3662335657 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5e40705d0 con 0x7ff5e406cba0 2023-10-26T01:31:01.488 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.483+0000 7ff5eaa21640 1 -- 172.21.15.169:0/3662335657 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5e4061330 con 0x7ff5e406cba0 2023-10-26T01:31:01.489 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.484+0000 7ff5da7fc640 1 -- 172.21.15.169:0/3662335657 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ff5d4003c10 con 0x7ff5e406cba0 2023-10-26T01:31:01.489 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.485+0000 7ff5da7fc640 1 --2- 172.21.15.169:0/3662335657 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff5c806ae40 0x7ff5c806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:01.489 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.485+0000 7ff5da7fc640 1 -- 172.21.15.169:0/3662335657 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff5d40ac1e0 con 0x7ff5e406cba0 2023-10-26T01:31:01.489 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.486+0000 7ff5e9a1f640 1 --2- 172.21.15.169:0/3662335657 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff5c806ae40 0x7ff5c806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:01.490 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.486+0000 7ff5e9a1f640 1 --2- 172.21.15.169:0/3662335657 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff5c806ae40 0x7ff5c806d300 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff5e0009c10 tx=0x7ff5e0005f90 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:01.490 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.489+0000 7ff5da7fc640 1 -- 172.21.15.169:0/3662335657 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff5d40771f0 con 0x7ff5e406cba0 2023-10-26T01:31:01.666 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.665+0000 7ff5eaa21640 1 -- 172.21.15.169:0/3662335657 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff5e40b01d0 con 0x7ff5e406cba0 2023-10-26T01:31:01.667 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.665+0000 7ff5da7fc640 1 -- 172.21.15.169:0/3662335657 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff5d4043020 con 0x7ff5e406cba0 2023-10-26T01:31:01.667 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 1 2023-10-26T01:31:01.670 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.668+0000 7ff5eaa21640 1 -- 172.21.15.169:0/3662335657 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff5c806ae40 msgr2=0x7ff5c806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:01.670 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.668+0000 7ff5eaa21640 1 --2- 172.21.15.169:0/3662335657 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff5c806ae40 0x7ff5c806d300 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff5e0009c10 tx=0x7ff5e0005f90 comp rx=0 tx=0).stop 2023-10-26T01:31:01.670 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.668+0000 7ff5eaa21640 1 -- 172.21.15.169:0/3662335657 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff5e406cba0 msgr2=0x7ff5e406f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:01.670 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.668+0000 7ff5eaa21640 1 --2- 172.21.15.169:0/3662335657 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff5e406cba0 0x7ff5e406f490 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff5d4036040 tx=0x7ff5d402da60 comp rx=0 tx=0).stop 2023-10-26T01:31:01.671 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.668+0000 7ff5eaa21640 1 -- 172.21.15.169:0/3662335657 shutdown_connections 2023-10-26T01:31:01.671 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.668+0000 7ff5eaa21640 1 --2- 172.21.15.169:0/3662335657 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff5c806ae40 0x7ff5c806d300 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:01.671 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.668+0000 7ff5eaa21640 1 --2- 172.21.15.169:0/3662335657 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff5e406cba0 0x7ff5e406f490 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:01.671 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.668+0000 7ff5eaa21640 1 --2- 172.21.15.169:0/3662335657 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff5e4151ed0 0x7ff5e406f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:01.671 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.668+0000 7ff5eaa21640 1 --2- 172.21.15.169:0/3662335657 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff5e406b860 0x7ff5e4151620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:01.672 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.668+0000 7ff5eaa21640 1 -- 172.21.15.169:0/3662335657 >> 172.21.15.169:0/3662335657 conn(0x7ff5e4069510 msgr2=0x7ff5e40aa040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:01.672 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.668+0000 7ff5eaa21640 1 -- 172.21.15.169:0/3662335657 shutdown_connections 2023-10-26T01:31:01.672 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.668+0000 7ff5eaa21640 1 -- 172.21.15.169:0/3662335657 wait complete. 2023-10-26T01:31:01.688 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-26T01:31:01.904 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.903+0000 7fcf6dd9b640 1 -- 172.21.15.169:0/607504925 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf58034440 con 0x7fcf701597c0 2023-10-26T01:31:01.906 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.903+0000 7fcf74b83640 1 -- 172.21.15.169:0/607504925 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf701597c0 msgr2=0x7fcf7015dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:01.906 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.903+0000 7fcf74b83640 1 --2- 172.21.15.169:0/607504925 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf701597c0 0x7fcf7015dbf0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fcf700b1ea0 tx=0x7fcf5802bd70 comp rx=0 tx=0).stop 2023-10-26T01:31:01.906 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.904+0000 7fcf74b83640 1 -- 172.21.15.169:0/607504925 shutdown_connections 2023-10-26T01:31:01.906 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.904+0000 7fcf74b83640 1 --2- 172.21.15.169:0/607504925 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf701597c0 0x7fcf7015dbf0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:01.907 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.904+0000 7fcf74b83640 1 --2- 172.21.15.169:0/607504925 >> v2:172.21.15.170:3300/0 conn(0x7fcf701593e0 0x7fcf700ac200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:01.907 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.904+0000 7fcf74b83640 1 -- 172.21.15.169:0/607504925 >> 172.21.15.169:0/607504925 conn(0x7fcf70069930 msgr2=0x7fcf70069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:01.907 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.904+0000 7fcf74b83640 1 -- 172.21.15.169:0/607504925 shutdown_connections 2023-10-26T01:31:01.907 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.904+0000 7fcf74b83640 1 -- 172.21.15.169:0/607504925 wait complete. 2023-10-26T01:31:01.909 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.905+0000 7fcf74b83640 1 Processor -- start 2023-10-26T01:31:01.909 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.906+0000 7fcf74b83640 1 -- start start 2023-10-26T01:31:01.910 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.906+0000 7fcf74b83640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf700b0c80 0x7fcf700c7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:01.910 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.907+0000 7fcf74b83640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcf701593e0 0x7fcf70153af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:01.910 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.907+0000 7fcf74b83640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf701597c0 0x7fcf70154030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:01.910 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.907+0000 7fcf74b83640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7006b170 con 0x7fcf700b0c80 2023-10-26T01:31:01.910 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.907+0000 7fcf74b83640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7006b450 con 0x7fcf701597c0 2023-10-26T01:31:01.911 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.907+0000 7fcf74b83640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7006b730 con 0x7fcf701593e0 2023-10-26T01:31:01.911 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.907+0000 7fcf6e59c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf700b0c80 0x7fcf700c7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:01.911 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.907+0000 7fcf6e59c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf700b0c80 0x7fcf700c7b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:46298/0 (socket says 172.21.15.169:46298) 2023-10-26T01:31:01.911 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.907+0000 7fcf6e59c640 1 -- 172.21.15.169:0/72244099 learned_addr learned my addr 172.21.15.169:0/72244099 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:31:01.911 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.907+0000 7fcf6e59c640 1 -- 172.21.15.169:0/72244099 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcf701593e0 msgr2=0x7fcf70153af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:01.912 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.907+0000 7fcf6e59c640 1 --2- 172.21.15.169:0/72244099 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcf701593e0 0x7fcf70153af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:01.912 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.907+0000 7fcf6ed9d640 1 --2- 172.21.15.169:0/72244099 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf701597c0 0x7fcf70154030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:01.912 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.907+0000 7fcf6e59c640 1 -- 172.21.15.169:0/72244099 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf701597c0 msgr2=0x7fcf70154030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:01.912 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.907+0000 7fcf6e59c640 1 --2- 172.21.15.169:0/72244099 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf701597c0 0x7fcf70154030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:01.912 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.907+0000 7fcf6e59c640 1 -- 172.21.15.169:0/72244099 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf58009490 con 0x7fcf700b0c80 2023-10-26T01:31:01.913 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.907+0000 7fcf6e59c640 1 --2- 172.21.15.169:0/72244099 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf700b0c80 0x7fcf700c7b80 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fcf6000d950 tx=0x7fcf6000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:01.913 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.908+0000 7fcf66ffd640 1 -- 172.21.15.169:0/72244099 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf60009c30 con 0x7fcf700b0c80 2023-10-26T01:31:01.913 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.908+0000 7fcf66ffd640 1 -- 172.21.15.169:0/72244099 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fcf60009e80 con 0x7fcf700b0c80 2023-10-26T01:31:01.914 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.908+0000 7fcf66ffd640 1 -- 172.21.15.169:0/72244099 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf600106c0 con 0x7fcf700b0c80 2023-10-26T01:31:01.914 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.908+0000 7fcf74b83640 1 -- 172.21.15.169:0/72244099 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf7006ba10 con 0x7fcf700b0c80 2023-10-26T01:31:01.914 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.908+0000 7fcf74b83640 1 -- 172.21.15.169:0/72244099 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf7006ad90 con 0x7fcf700b0c80 2023-10-26T01:31:01.915 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.909+0000 7fcf74b83640 1 -- 172.21.15.169:0/72244099 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf7006af20 con 0x7fcf700b0c80 2023-10-26T01:31:01.915 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.910+0000 7fcf66ffd640 1 -- 172.21.15.169:0/72244099 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fcf60010820 con 0x7fcf700b0c80 2023-10-26T01:31:01.915 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.911+0000 7fcf66ffd640 1 --2- 172.21.15.169:0/72244099 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcf5006ae40 0x7fcf5006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:01.916 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.911+0000 7fcf66ffd640 1 -- 172.21.15.169:0/72244099 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcf60089ba0 con 0x7fcf700b0c80 2023-10-26T01:31:01.916 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.911+0000 7fcf67fff640 1 --2- 172.21.15.169:0/72244099 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcf5006ae40 0x7fcf5006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:01.916 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.912+0000 7fcf67fff640 1 --2- 172.21.15.169:0/72244099 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcf5006ae40 0x7fcf5006d300 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fcf58036000 tx=0x7fcf580046c0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:01.917 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:01.915+0000 7fcf66ffd640 1 -- 172.21.15.169:0/72244099 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcf60054bb0 con 0x7fcf700b0c80 2023-10-26T01:31:02.106 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:02.104+0000 7fcf74b83640 1 -- 172.21.15.169:0/72244099 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcf7004db10 con 0x7fcf700b0c80 2023-10-26T01:31:02.107 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:02.106+0000 7fcf66ffd640 1 -- 172.21.15.169:0/72244099 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcf60017020 con 0x7fcf700b0c80 2023-10-26T01:31:02.107 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 32 2023-10-26T01:31:02.110 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:02.108+0000 7fcf74b83640 1 -- 172.21.15.169:0/72244099 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcf5006ae40 msgr2=0x7fcf5006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:02.110 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:02.108+0000 7fcf74b83640 1 --2- 172.21.15.169:0/72244099 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcf5006ae40 0x7fcf5006d300 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fcf58036000 tx=0x7fcf580046c0 comp rx=0 tx=0).stop 2023-10-26T01:31:02.110 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:02.108+0000 7fcf74b83640 1 -- 172.21.15.169:0/72244099 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf700b0c80 msgr2=0x7fcf700c7b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:02.110 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:02.108+0000 7fcf74b83640 1 --2- 172.21.15.169:0/72244099 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf700b0c80 0x7fcf700c7b80 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fcf6000d950 tx=0x7fcf6000de10 comp rx=0 tx=0).stop 2023-10-26T01:31:02.111 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:02.108+0000 7fcf74b83640 1 -- 172.21.15.169:0/72244099 shutdown_connections 2023-10-26T01:31:02.111 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:02.108+0000 7fcf74b83640 1 --2- 172.21.15.169:0/72244099 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcf5006ae40 0x7fcf5006d300 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:02.111 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:02.108+0000 7fcf74b83640 1 --2- 172.21.15.169:0/72244099 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf701597c0 0x7fcf70154030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:02.111 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:02.108+0000 7fcf74b83640 1 --2- 172.21.15.169:0/72244099 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcf701593e0 0x7fcf70153af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:02.112 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:02.108+0000 7fcf74b83640 1 --2- 172.21.15.169:0/72244099 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf700b0c80 0x7fcf700c7b80 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:02.112 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:02.108+0000 7fcf74b83640 1 -- 172.21.15.169:0/72244099 >> 172.21.15.169:0/72244099 conn(0x7fcf70069930 msgr2=0x7fcf70154bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:02.112 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:02.108+0000 7fcf74b83640 1 -- 172.21.15.169:0/72244099 shutdown_connections 2023-10-26T01:31:02.112 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:02.108+0000 7fcf74b83640 1 -- 172.21.15.169:0/72244099 wait complete. 2023-10-26T01:31:02.128 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-26T01:31:02.349 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:02.347+0000 7fdf9332c640 1 -- 172.21.15.169:0/3904055472 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdf8c0cdef0 msgr2=0x7fdf8c15b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:02.349 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:02.347+0000 7fdf9332c640 1 --2- 172.21.15.169:0/3904055472 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdf8c0cdef0 0x7fdf8c15b4a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c00a940 tx=0x7fdf7c02d090 comp rx=0 tx=0).stop 2023-10-26T01:31:02.349 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:02.347+0000 7fdf9332c640 1 -- 172.21.15.169:0/3904055472 shutdown_connections 2023-10-26T01:31:02.350 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:02.347+0000 7fdf9332c640 1 --2- 172.21.15.169:0/3904055472 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdf8c0cdef0 0x7fdf8c15b4a0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:02.350 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:02.347+0000 7fdf9332c640 1 -- 172.21.15.169:0/3904055472 >> 172.21.15.169:0/3904055472 conn(0x7fdf8c069930 msgr2=0x7fdf8c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:02.350 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:02.347+0000 7fdf9332c640 1 -- 172.21.15.169:0/3904055472 shutdown_connections 2023-10-26T01:31:02.350 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:02.347+0000 7fdf9332c640 1 -- 172.21.15.169:0/3904055472 wait complete. 2023-10-26T01:31:02.351 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:02.349+0000 7fdf9332c640 1 Processor -- start 2023-10-26T01:31:02.351 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:02.349+0000 7fdf9332c640 1 -- start start 2023-10-26T01:31:02.352 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:02.350+0000 7fdf9332c640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdf8c0cdef0 0x7fdf8c1f2140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:02.353 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:02.350+0000 7fdf9332c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdf8c155360 0x7fdf8c14fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:02.353 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:02.350+0000 7fdf9332c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdf8c15b9e0 0x7fdf8c0aded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:02.353 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:02.350+0000 7fdf9332c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf8c1502d0 con 0x7fdf8c155360 2023-10-26T01:31:02.353 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:02.350+0000 7fdf9332c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf8c0ae720 con 0x7fdf8c15b9e0 2023-10-26T01:31:02.353 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:02.350+0000 7fdf9332c640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf8c0aea00 con 0x7fdf8c0cdef0 2023-10-26T01:31:02.354 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:02.350+0000 7fdf908c7640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdf8c155360 0x7fdf8c14fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:02.354 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:02.350+0000 7fdf908c7640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdf8c155360 0x7fdf8c14fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:46322/0 (socket says 172.21.15.169:46322) 2023-10-26T01:31:02.354 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:02.350+0000 7fdf908c7640 1 -- 172.21.15.169:0/838019885 learned_addr learned my addr 172.21.15.169:0/838019885 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:31:02.354 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:02.351+0000 7fdf918c9640 1 --2- 172.21.15.169:0/838019885 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdf8c15b9e0 0x7fdf8c0aded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:02.354 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:02.351+0000 7fdf908c7640 1 -- 172.21.15.169:0/838019885 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdf8c0cdef0 msgr2=0x7fdf8c1f2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:02.354 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:02.351+0000 7fdf908c7640 1 --2- 172.21.15.169:0/838019885 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdf8c0cdef0 0x7fdf8c1f2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:02.355 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:02.351+0000 7fdf908c7640 1 -- 172.21.15.169:0/838019885 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdf8c15b9e0 msgr2=0x7fdf8c0aded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:02.355 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:02.351+0000 7fdf908c7640 1 --2- 172.21.15.169:0/838019885 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdf8c15b9e0 0x7fdf8c0aded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:02.355 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:02.351+0000 7fdf908c7640 1 -- 172.21.15.169:0/838019885 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf7c00a5a0 con 0x7fdf8c155360 2023-10-26T01:31:02.355 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:02.351+0000 7fdf908c7640 1 --2- 172.21.15.169:0/838019885 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdf8c155360 0x7fdf8c14fd90 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c00a890 tx=0x7fdf7c036960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:02.356 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:02.351+0000 7fdf81ffb640 1 -- 172.21.15.169:0/838019885 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf7c0035f0 con 0x7fdf8c155360 2023-10-26T01:31:02.357 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:02.351+0000 7fdf81ffb640 1 -- 172.21.15.169:0/838019885 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fdf7c003860 con 0x7fdf8c155360 2023-10-26T01:31:02.357 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:02.351+0000 7fdf81ffb640 1 -- 172.21.15.169:0/838019885 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf7c003c70 con 0x7fdf8c155360 2023-10-26T01:31:02.357 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:02.351+0000 7fdf9332c640 1 -- 172.21.15.169:0/838019885 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf8c0aece0 con 0x7fdf8c155360 2023-10-26T01:31:02.357 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:02.352+0000 7fdf9332c640 1 -- 172.21.15.169:0/838019885 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf8c0af2a0 con 0x7fdf8c155360 2023-10-26T01:31:02.357 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:02.352+0000 7fdf9332c640 1 -- 172.21.15.169:0/838019885 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf8c157740 con 0x7fdf8c155360 2023-10-26T01:31:02.358 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:02.353+0000 7fdf81ffb640 1 -- 172.21.15.169:0/838019885 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fdf7c03c070 con 0x7fdf8c155360 2023-10-26T01:31:02.358 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:02.354+0000 7fdf81ffb640 1 --2- 172.21.15.169:0/838019885 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fdf7006ae40 0x7fdf7006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:02.358 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:02.354+0000 7fdf81ffb640 1 -- 172.21.15.169:0/838019885 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdf7c0ac4f0 con 0x7fdf8c155360 2023-10-26T01:31:02.358 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:02.355+0000 7fdf910c8640 1 --2- 172.21.15.169:0/838019885 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fdf7006ae40 0x7fdf7006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:02.359 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:02.356+0000 7fdf910c8640 1 --2- 172.21.15.169:0/838019885 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fdf7006ae40 0x7fdf7006d300 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c1efd30 tx=0x7fdf88007910 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:02.360 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:02.359+0000 7fdf81ffb640 1 -- 172.21.15.169:0/838019885 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdf7c0776b0 con 0x7fdf8c155360 2023-10-26T01:31:02.549 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:02.547+0000 7fdf9332c640 1 -- 172.21.15.169:0/838019885 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdf8c04db10 con 0x7fdf8c155360 2023-10-26T01:31:02.549 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:02.548+0000 7fdf81ffb640 1 -- 172.21.15.169:0/838019885 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdf7c07acd0 con 0x7fdf8c155360 2023-10-26T01:31:02.550 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 64 2023-10-26T01:31:02.552 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:02.550+0000 7fdf9332c640 1 -- 172.21.15.169:0/838019885 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fdf7006ae40 msgr2=0x7fdf7006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:02.553 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:02.550+0000 7fdf9332c640 1 --2- 172.21.15.169:0/838019885 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fdf7006ae40 0x7fdf7006d300 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c1efd30 tx=0x7fdf88007910 comp rx=0 tx=0).stop 2023-10-26T01:31:02.553 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:02.550+0000 7fdf9332c640 1 -- 172.21.15.169:0/838019885 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdf8c155360 msgr2=0x7fdf8c14fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:02.553 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:02.550+0000 7fdf9332c640 1 --2- 172.21.15.169:0/838019885 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdf8c155360 0x7fdf8c14fd90 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c00a890 tx=0x7fdf7c036960 comp rx=0 tx=0).stop 2023-10-26T01:31:02.553 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:02.551+0000 7fdf9332c640 1 -- 172.21.15.169:0/838019885 shutdown_connections 2023-10-26T01:31:02.554 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:02.551+0000 7fdf9332c640 1 --2- 172.21.15.169:0/838019885 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fdf7006ae40 0x7fdf7006d300 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:02.554 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:02.551+0000 7fdf9332c640 1 --2- 172.21.15.169:0/838019885 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdf8c155360 0x7fdf8c14fd90 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:02.554 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:02.551+0000 7fdf9332c640 1 --2- 172.21.15.169:0/838019885 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdf8c15b9e0 0x7fdf8c0aded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:02.554 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:02.551+0000 7fdf9332c640 1 --2- 172.21.15.169:0/838019885 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdf8c0cdef0 0x7fdf8c1f2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:02.554 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:02.551+0000 7fdf9332c640 1 -- 172.21.15.169:0/838019885 >> 172.21.15.169:0/838019885 conn(0x7fdf8c069930 msgr2=0x7fdf8c15b0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:02.555 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:02.551+0000 7fdf9332c640 1 -- 172.21.15.169:0/838019885 shutdown_connections 2023-10-26T01:31:02.555 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:02.551+0000 7fdf9332c640 1 -- 172.21.15.169:0/838019885 wait complete. 2023-10-26T01:31:02.573 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T01:31:02.787 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:02.784+0000 7ff84d16e640 1 -- 172.21.15.169:0/2776738688 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff84806ce20 msgr2=0x7ff84814e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:02.787 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:02.784+0000 7ff84d16e640 1 --2- 172.21.15.169:0/2776738688 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff84806ce20 0x7ff84814e790 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff838005fd0 tx=0x7ff83802bd90 comp rx=0 tx=0).stop 2023-10-26T01:31:02.787 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:02.785+0000 7ff84d16e640 1 -- 172.21.15.169:0/2776738688 shutdown_connections 2023-10-26T01:31:02.787 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:02.785+0000 7ff84d16e640 1 --2- 172.21.15.169:0/2776738688 >> v2:172.21.15.170:3300/0 conn(0x7ff8480cd720 0x7ff8480cdb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:02.788 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:02.785+0000 7ff84d16e640 1 --2- 172.21.15.169:0/2776738688 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff84806ce20 0x7ff84814e790 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:02.788 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:02.785+0000 7ff84d16e640 1 -- 172.21.15.169:0/2776738688 >> 172.21.15.169:0/2776738688 conn(0x7ff848069930 msgr2=0x7ff848069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:02.788 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:02.785+0000 7ff84d16e640 1 -- 172.21.15.169:0/2776738688 shutdown_connections 2023-10-26T01:31:02.788 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:02.785+0000 7ff84d16e640 1 -- 172.21.15.169:0/2776738688 wait complete. 2023-10-26T01:31:02.789 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:02.787+0000 7ff84d16e640 1 Processor -- start 2023-10-26T01:31:02.790 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:02.787+0000 7ff84d16e640 1 -- start start 2023-10-26T01:31:02.790 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:02.788+0000 7ff84d16e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff84806ce20 0x7ff848153cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:02.790 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:02.788+0000 7ff84d16e640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff8480cd720 0x7ff848152300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:02.790 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:02.788+0000 7ff84d16e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff848158850 0x7ff84814efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:02.791 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:02.788+0000 7ff84d16e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff848154210 con 0x7ff848158850 2023-10-26T01:31:02.791 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:02.788+0000 7ff84d16e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff848152c30 con 0x7ff84806ce20 2023-10-26T01:31:02.791 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:02.788+0000 7ff84d16e640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff848152f10 con 0x7ff8480cd720 2023-10-26T01:31:02.791 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:02.788+0000 7ff84759e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff848158850 0x7ff84814efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:02.791 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:02.788+0000 7ff84759e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff848158850 0x7ff84814efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:46328/0 (socket says 172.21.15.169:46328) 2023-10-26T01:31:02.791 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:02.788+0000 7ff84759e640 1 -- 172.21.15.169:0/3929342567 learned_addr learned my addr 172.21.15.169:0/3929342567 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:31:02.792 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:02.788+0000 7ff846d9d640 1 --2- 172.21.15.169:0/3929342567 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff84806ce20 0x7ff848153cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:02.792 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:02.788+0000 7ff84759e640 1 -- 172.21.15.169:0/3929342567 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff8480cd720 msgr2=0x7ff848152300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:02.792 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:02.788+0000 7ff84759e640 1 --2- 172.21.15.169:0/3929342567 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff8480cd720 0x7ff848152300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:02.792 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:02.788+0000 7ff84759e640 1 -- 172.21.15.169:0/3929342567 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff84806ce20 msgr2=0x7ff848153cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:02.793 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:02.788+0000 7ff84759e640 1 --2- 172.21.15.169:0/3929342567 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff84806ce20 0x7ff848153cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:02.793 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:02.788+0000 7ff84759e640 1 -- 172.21.15.169:0/3929342567 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff838009560 con 0x7ff848158850 2023-10-26T01:31:02.794 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:02.788+0000 7ff84759e640 1 --2- 172.21.15.169:0/3929342567 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff848158850 0x7ff84814efd0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff84000eaa0 tx=0x7ff84000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:02.794 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:02.789+0000 7ff82f7fe640 1 -- 172.21.15.169:0/3929342567 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff84000cdb0 con 0x7ff848158850 2023-10-26T01:31:02.794 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:02.789+0000 7ff82f7fe640 1 -- 172.21.15.169:0/3929342567 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff840004b20 con 0x7ff848158850 2023-10-26T01:31:02.795 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:02.789+0000 7ff82f7fe640 1 -- 172.21.15.169:0/3929342567 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff840005020 con 0x7ff848158850 2023-10-26T01:31:02.795 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:02.789+0000 7ff84d16e640 1 -- 172.21.15.169:0/3929342567 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8481531f0 con 0x7ff848158850 2023-10-26T01:31:02.795 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:02.789+0000 7ff84d16e640 1 -- 172.21.15.169:0/3929342567 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8481537b0 con 0x7ff848158850 2023-10-26T01:31:02.795 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:02.790+0000 7ff84d16e640 1 -- 172.21.15.169:0/3929342567 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8480cdb90 con 0x7ff848158850 2023-10-26T01:31:02.796 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:02.791+0000 7ff82f7fe640 1 -- 172.21.15.169:0/3929342567 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ff840005180 con 0x7ff848158850 2023-10-26T01:31:02.796 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:02.792+0000 7ff82f7fe640 1 --2- 172.21.15.169:0/3929342567 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff83406ae70 0x7ff83406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:02.796 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:02.792+0000 7ff82f7fe640 1 -- 172.21.15.169:0/3929342567 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff840089f10 con 0x7ff848158850 2023-10-26T01:31:02.796 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:02.793+0000 7ff846d9d640 1 --2- 172.21.15.169:0/3929342567 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff83406ae70 0x7ff83406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:02.797 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:02.794+0000 7ff846d9d640 1 --2- 172.21.15.169:0/3929342567 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff83406ae70 0x7ff83406d330 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff838035000 tx=0x7ff838004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:02.798 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:02.797+0000 7ff82f7fe640 1 -- 172.21.15.169:0/3929342567 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff840055050 con 0x7ff848158850 2023-10-26T01:31:03.028 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.027+0000 7ff84d16e640 1 -- 172.21.15.169:0/3929342567 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff848066e60 con 0x7ff848158850 2023-10-26T01:31:03.029 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.028+0000 7ff82f7fe640 1 -- 172.21.15.169:0/3929342567 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4916 (secure 0 0 0) 0x7ff84001f020 con 0x7ff848158850 2023-10-26T01:31:03.030 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:31:03.030 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4298,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1809801299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":1809801299},{"type":"v1","addr":"172.21.15.169:6835","nonce":1809801299}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4304,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/2248566141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2248566141},{"type":"v1","addr":"172.21.15.170:6836","nonce":2248566141}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4317,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/2383616530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2383616530},{"type":"v1","addr":"172.21.15.169:6834","nonce":2383616530}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T01:30:41.545068+0000","modified":"2023-10-26T01:30:44.577169+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4338},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4338":{"gid":4338,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6837/590609220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":590609220},{"type":"v1","addr":"172.21.15.170:6837","nonce":590609220}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-26T01:31:03.030 INFO:teuthology.orchestra.run.smithi169.stderr:dumped fsmap epoch 7 2023-10-26T01:31:03.032 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.030+0000 7ff84d16e640 1 -- 172.21.15.169:0/3929342567 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff83406ae70 msgr2=0x7ff83406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:03.033 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.030+0000 7ff84d16e640 1 --2- 172.21.15.169:0/3929342567 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff83406ae70 0x7ff83406d330 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff838035000 tx=0x7ff838004730 comp rx=0 tx=0).stop 2023-10-26T01:31:03.033 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.030+0000 7ff84d16e640 1 -- 172.21.15.169:0/3929342567 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff848158850 msgr2=0x7ff84814efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:03.033 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.030+0000 7ff84d16e640 1 --2- 172.21.15.169:0/3929342567 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff848158850 0x7ff84814efd0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff84000eaa0 tx=0x7ff84000ef60 comp rx=0 tx=0).stop 2023-10-26T01:31:03.033 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.031+0000 7ff84d16e640 1 -- 172.21.15.169:0/3929342567 shutdown_connections 2023-10-26T01:31:03.033 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.031+0000 7ff84d16e640 1 --2- 172.21.15.169:0/3929342567 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff83406ae70 0x7ff83406d330 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:03.034 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.031+0000 7ff84d16e640 1 --2- 172.21.15.169:0/3929342567 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff848158850 0x7ff84814efd0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:03.034 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.031+0000 7ff84d16e640 1 --2- 172.21.15.169:0/3929342567 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff8480cd720 0x7ff848152300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:03.034 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.031+0000 7ff84d16e640 1 --2- 172.21.15.169:0/3929342567 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff84806ce20 0x7ff848153cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:03.034 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.031+0000 7ff84d16e640 1 -- 172.21.15.169:0/3929342567 >> 172.21.15.169:0/3929342567 conn(0x7ff848069930 msgr2=0x7ff8480aa1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:03.035 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.031+0000 7ff84d16e640 1 -- 172.21.15.169:0/3929342567 shutdown_connections 2023-10-26T01:31:03.035 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.031+0000 7ff84d16e640 1 -- 172.21.15.169:0/3929342567 wait complete. 2023-10-26T01:31:03.051 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:31:03.275 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.273+0000 7fd90a4fc640 1 -- 172.21.15.169:0/4250545087 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd90406ce20 msgr2=0x7fd90414e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:03.275 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.273+0000 7fd90a4fc640 1 --2- 172.21.15.169:0/4250545087 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd90406ce20 0x7fd90414e790 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd8f4009830 tx=0x7fd8f402bc30 comp rx=0 tx=0).stop 2023-10-26T01:31:03.276 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.273+0000 7fd90a4fc640 1 -- 172.21.15.169:0/4250545087 shutdown_connections 2023-10-26T01:31:03.276 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.273+0000 7fd90a4fc640 1 --2- 172.21.15.169:0/4250545087 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd904158850 0x7fd9040aab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:03.276 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.273+0000 7fd90a4fc640 1 --2- 172.21.15.169:0/4250545087 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd90406ce20 0x7fd90414e790 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:03.276 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.273+0000 7fd90a4fc640 1 -- 172.21.15.169:0/4250545087 >> 172.21.15.169:0/4250545087 conn(0x7fd904069930 msgr2=0x7fd904069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:03.276 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.274+0000 7fd90a4fc640 1 -- 172.21.15.169:0/4250545087 shutdown_connections 2023-10-26T01:31:03.277 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.274+0000 7fd90a4fc640 1 -- 172.21.15.169:0/4250545087 wait complete. 2023-10-26T01:31:03.277 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.275+0000 7fd90a4fc640 1 Processor -- start 2023-10-26T01:31:03.277 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.275+0000 7fd90a4fc640 1 -- start start 2023-10-26T01:31:03.279 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.276+0000 7fd90a4fc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd90406ad30 0x7fd9041424a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:03.280 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.276+0000 7fd90a4fc640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd90406ce20 0x7fd9040ae340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:03.280 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.276+0000 7fd90a4fc640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd904158850 0x7fd9040aeec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:03.280 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.276+0000 7fd90a4fc640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd904142d60 con 0x7fd90406ce20 2023-10-26T01:31:03.280 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.276+0000 7fd90a4fc640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd904142a80 con 0x7fd90406ad30 2023-10-26T01:31:03.281 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.276+0000 7fd90a4fc640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd90406ee60 con 0x7fd904158850 2023-10-26T01:31:03.281 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.277+0000 7fd9037fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd90406ce20 0x7fd9040ae340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:03.281 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.277+0000 7fd9037fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd90406ce20 0x7fd9040ae340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:46336/0 (socket says 172.21.15.169:46336) 2023-10-26T01:31:03.281 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.277+0000 7fd9037fe640 1 -- 172.21.15.169:0/979123475 learned_addr learned my addr 172.21.15.169:0/979123475 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:31:03.281 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.277+0000 7fd903fff640 1 --2- 172.21.15.169:0/979123475 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd90406ad30 0x7fd9041424a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:03.282 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.277+0000 7fd908a99640 1 --2- 172.21.15.169:0/979123475 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd904158850 0x7fd9040aeec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:03.282 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.277+0000 7fd9037fe640 1 -- 172.21.15.169:0/979123475 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd904158850 msgr2=0x7fd9040aeec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:03.282 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.277+0000 7fd9037fe640 1 --2- 172.21.15.169:0/979123475 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd904158850 0x7fd9040aeec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:03.282 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.277+0000 7fd9037fe640 1 -- 172.21.15.169:0/979123475 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd90406ad30 msgr2=0x7fd9041424a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:03.282 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.277+0000 7fd9037fe640 1 --2- 172.21.15.169:0/979123475 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd90406ad30 0x7fd9041424a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:03.282 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.277+0000 7fd9037fe640 1 -- 172.21.15.169:0/979123475 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8f4009490 con 0x7fd90406ce20 2023-10-26T01:31:03.283 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.277+0000 7fd9037fe640 1 --2- 172.21.15.169:0/979123475 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd90406ce20 0x7fd9040ae340 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd8f800ea90 tx=0x7fd8f800ef50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:03.283 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.278+0000 7fd900ff9640 1 -- 172.21.15.169:0/979123475 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f800cc80 con 0x7fd90406ce20 2023-10-26T01:31:03.283 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.278+0000 7fd900ff9640 1 -- 172.21.15.169:0/979123475 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd8f8016410 con 0x7fd90406ce20 2023-10-26T01:31:03.283 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.278+0000 7fd900ff9640 1 -- 172.21.15.169:0/979123475 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f8016820 con 0x7fd90406ce20 2023-10-26T01:31:03.284 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.278+0000 7fd90a4fc640 1 -- 172.21.15.169:0/979123475 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd90415b030 con 0x7fd90406ce20 2023-10-26T01:31:03.284 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.278+0000 7fd90a4fc640 1 -- 172.21.15.169:0/979123475 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd90406f420 con 0x7fd90406ce20 2023-10-26T01:31:03.285 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.279+0000 7fd90a4fc640 1 -- 172.21.15.169:0/979123475 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd90406f5e0 con 0x7fd90406ce20 2023-10-26T01:31:03.285 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.280+0000 7fd900ff9640 1 -- 172.21.15.169:0/979123475 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fd8f8016980 con 0x7fd90406ce20 2023-10-26T01:31:03.285 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.281+0000 7fd900ff9640 1 --2- 172.21.15.169:0/979123475 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd8f006ae70 0x7fd8f006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:03.285 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.281+0000 7fd900ff9640 1 -- 172.21.15.169:0/979123475 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd8f8089be0 con 0x7fd90406ce20 2023-10-26T01:31:03.286 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.282+0000 7fd903fff640 1 --2- 172.21.15.169:0/979123475 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd8f006ae70 0x7fd8f006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:03.286 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.283+0000 7fd903fff640 1 --2- 172.21.15.169:0/979123475 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd8f006ae70 0x7fd8f006d330 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd8f4036000 tx=0x7fd8f4004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:03.287 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.286+0000 7fd900ff9640 1 -- 172.21.15.169:0/979123475 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd8f8054cb0 con 0x7fd90406ce20 2023-10-26T01:31:03.479 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.478+0000 7fd90a4fc640 1 -- 172.21.15.169:0/979123475 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd90404db10 con 0x7fd90406ce20 2023-10-26T01:31:03.482 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.481+0000 7fd900ff9640 1 -- 172.21.15.169:0/979123475 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16553 (secure 0 0 0) 0x7fd8f80582d0 con 0x7fd90406ce20 2023-10-26T01:31:03.483 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:31:03.483 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":21,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:30:43.078706+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-26T01:30:39.437631+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-26T01:30:40.561774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:31:03.486 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.483+0000 7fd90a4fc640 1 -- 172.21.15.169:0/979123475 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd8f006ae70 msgr2=0x7fd8f006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:03.486 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.483+0000 7fd90a4fc640 1 --2- 172.21.15.169:0/979123475 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd8f006ae70 0x7fd8f006d330 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd8f4036000 tx=0x7fd8f4004730 comp rx=0 tx=0).stop 2023-10-26T01:31:03.486 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.483+0000 7fd90a4fc640 1 -- 172.21.15.169:0/979123475 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd90406ce20 msgr2=0x7fd9040ae340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:03.486 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.483+0000 7fd90a4fc640 1 --2- 172.21.15.169:0/979123475 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd90406ce20 0x7fd9040ae340 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd8f800ea90 tx=0x7fd8f800ef50 comp rx=0 tx=0).stop 2023-10-26T01:31:03.487 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.484+0000 7fd90a4fc640 1 -- 172.21.15.169:0/979123475 shutdown_connections 2023-10-26T01:31:03.487 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.484+0000 7fd90a4fc640 1 --2- 172.21.15.169:0/979123475 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd8f006ae70 0x7fd8f006d330 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:03.487 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.484+0000 7fd90a4fc640 1 --2- 172.21.15.169:0/979123475 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd90406ce20 0x7fd9040ae340 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:03.487 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.484+0000 7fd90a4fc640 1 --2- 172.21.15.169:0/979123475 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd904158850 0x7fd9040aeec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:03.487 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.484+0000 7fd90a4fc640 1 --2- 172.21.15.169:0/979123475 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd90406ad30 0x7fd9041424a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:03.488 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.484+0000 7fd90a4fc640 1 -- 172.21.15.169:0/979123475 >> 172.21.15.169:0/979123475 conn(0x7fd904069930 msgr2=0x7fd904150b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:03.488 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.484+0000 7fd90a4fc640 1 -- 172.21.15.169:0/979123475 shutdown_connections 2023-10-26T01:31:03.488 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.484+0000 7fd90a4fc640 1 -- 172.21.15.169:0/979123475 wait complete. 2023-10-26T01:31:03.505 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T01:31:03.718 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.716+0000 7f1ccccc2640 1 -- 172.21.15.169:0/866444740 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1cb4004d00 msgr2=0x7f1cb4005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:03.718 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.716+0000 7f1ccccc2640 1 --2- 172.21.15.169:0/866444740 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1cb4004d00 0x7f1cb4005180 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1cc80b1c90 tx=0x7f1cb40057f0 comp rx=0 tx=0).stop 2023-10-26T01:31:03.718 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.716+0000 7f1cc6d9d640 1 -- 172.21.15.169:0/866444740 reap_dead start 2023-10-26T01:31:03.719 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.716+0000 7f1ccccc2640 1 -- 172.21.15.169:0/866444740 shutdown_connections 2023-10-26T01:31:03.719 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.716+0000 7f1ccccc2640 1 -- 172.21.15.169:0/866444740 >> 172.21.15.169:0/866444740 conn(0x7f1cc8069510 msgr2=0x7f1cc80698f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:03.719 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.716+0000 7f1ccccc2640 1 -- 172.21.15.169:0/866444740 shutdown_connections 2023-10-26T01:31:03.719 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.716+0000 7f1ccccc2640 1 -- 172.21.15.169:0/866444740 wait complete. 2023-10-26T01:31:03.720 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.718+0000 7f1ccccc2640 1 Processor -- start 2023-10-26T01:31:03.720 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.718+0000 7f1ccccc2640 1 -- start start 2023-10-26T01:31:03.722 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.719+0000 7f1ccccc2640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1cb4004d00 0x7f1cc81f1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:03.722 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.719+0000 7f1ccccc2640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1cc8098a60 0x7f1cc8098ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:03.722 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.719+0000 7f1ccccc2640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1cc814ca20 0x7f1cc814cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:03.723 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.719+0000 7f1ccccc2640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc81f1c90 con 0x7f1cc8098a60 2023-10-26T01:31:03.723 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.719+0000 7f1ccccc2640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc8099420 con 0x7f1cb4004d00 2023-10-26T01:31:03.723 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.719+0000 7f1ccccc2640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc814d780 con 0x7f1cc814ca20 2023-10-26T01:31:03.723 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.719+0000 7f1cbdd9b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1cc8098a60 0x7f1cc8098ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:03.723 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.719+0000 7f1cbdd9b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1cc8098a60 0x7f1cc8098ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:46344/0 (socket says 172.21.15.169:46344) 2023-10-26T01:31:03.724 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.719+0000 7f1cbdd9b640 1 -- 172.21.15.169:0/282273615 learned_addr learned my addr 172.21.15.169:0/282273615 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:31:03.724 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.720+0000 7f1cbdd9b640 1 -- 172.21.15.169:0/282273615 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1cc814ca20 msgr2=0x7f1cc814cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:03.724 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.720+0000 7f1cbdd9b640 1 --2- 172.21.15.169:0/282273615 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1cc814ca20 0x7f1cc814cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:03.724 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.720+0000 7f1cc659c640 1 --2- 172.21.15.169:0/282273615 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1cb4004d00 0x7f1cc81f1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:03.724 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.720+0000 7f1cbdd9b640 1 -- 172.21.15.169:0/282273615 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1cb4004d00 msgr2=0x7f1cc81f1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:03.724 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.720+0000 7f1cbdd9b640 1 --2- 172.21.15.169:0/282273615 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1cb4004d00 0x7f1cc81f1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:03.725 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.720+0000 7f1cbdd9b640 1 -- 172.21.15.169:0/282273615 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cc809ab10 con 0x7f1cc8098a60 2023-10-26T01:31:03.725 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.720+0000 7f1cbdd9b640 1 --2- 172.21.15.169:0/282273615 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1cc8098a60 0x7f1cc8098ee0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1cb0006fd0 tx=0x7f1cb000db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:03.725 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.720+0000 7f1cbf7fe640 1 -- 172.21.15.169:0/282273615 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cb00050d0 con 0x7f1cc8098a60 2023-10-26T01:31:03.725 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.720+0000 7f1cbf7fe640 1 -- 172.21.15.169:0/282273615 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1cb0005230 con 0x7f1cc8098a60 2023-10-26T01:31:03.725 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.720+0000 7f1cbf7fe640 1 -- 172.21.15.169:0/282273615 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cb000b920 con 0x7f1cc8098a60 2023-10-26T01:31:03.726 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.720+0000 7f1ccccc2640 1 -- 172.21.15.169:0/282273615 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cc809adf0 con 0x7f1cc8098a60 2023-10-26T01:31:03.726 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.720+0000 7f1ccccc2640 1 -- 172.21.15.169:0/282273615 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cc809b3b0 con 0x7f1cc8098a60 2023-10-26T01:31:03.727 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.721+0000 7f1ccccc2640 1 -- 172.21.15.169:0/282273615 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cc81453f0 con 0x7f1cc8098a60 2023-10-26T01:31:03.727 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.722+0000 7f1cbf7fe640 1 -- 172.21.15.169:0/282273615 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f1cb000bc90 con 0x7f1cc8098a60 2023-10-26T01:31:03.727 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.723+0000 7f1cbf7fe640 1 --2- 172.21.15.169:0/282273615 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f1ca806ae40 0x7f1ca806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:03.727 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.723+0000 7f1cbf7fe640 1 -- 172.21.15.169:0/282273615 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1cb0088f10 con 0x7f1cc8098a60 2023-10-26T01:31:03.728 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.724+0000 7f1cc659c640 1 --2- 172.21.15.169:0/282273615 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f1ca806ae40 0x7f1ca806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:03.728 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.725+0000 7f1cc659c640 1 --2- 172.21.15.169:0/282273615 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f1ca806ae40 0x7f1ca806d300 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1cb80058c0 tx=0x7f1cb8005850 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:03.729 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.728+0000 7f1cbf7fe640 1 -- 172.21.15.169:0/282273615 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1cb0053fb0 con 0x7f1cc8098a60 2023-10-26T01:31:03.954 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.952+0000 7f1ccccc2640 1 -- 172.21.15.169:0/282273615 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1cc80b2e00 con 0x7f1cc8098a60 2023-10-26T01:31:03.955 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.953+0000 7f1cbf7fe640 1 -- 172.21.15.169:0/282273615 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4916 (secure 0 0 0) 0x7f1cc80b2e00 con 0x7f1cc8098a60 2023-10-26T01:31:03.955 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:31:03.956 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4298,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1809801299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":1809801299},{"type":"v1","addr":"172.21.15.169:6835","nonce":1809801299}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4304,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/2248566141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2248566141},{"type":"v1","addr":"172.21.15.170:6836","nonce":2248566141}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4317,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/2383616530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2383616530},{"type":"v1","addr":"172.21.15.169:6834","nonce":2383616530}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T01:30:41.545068+0000","modified":"2023-10-26T01:30:44.577169+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4338},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4338":{"gid":4338,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6837/590609220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":590609220},{"type":"v1","addr":"172.21.15.170:6837","nonce":590609220}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-26T01:31:03.956 INFO:teuthology.orchestra.run.smithi169.stderr:dumped fsmap epoch 7 2023-10-26T01:31:03.958 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.956+0000 7f1ccccc2640 1 -- 172.21.15.169:0/282273615 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f1ca806ae40 msgr2=0x7f1ca806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:03.958 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.956+0000 7f1ccccc2640 1 --2- 172.21.15.169:0/282273615 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f1ca806ae40 0x7f1ca806d300 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1cb80058c0 tx=0x7f1cb8005850 comp rx=0 tx=0).stop 2023-10-26T01:31:03.958 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.956+0000 7f1ccccc2640 1 -- 172.21.15.169:0/282273615 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1cc8098a60 msgr2=0x7f1cc8098ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:03.958 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.956+0000 7f1ccccc2640 1 --2- 172.21.15.169:0/282273615 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1cc8098a60 0x7f1cc8098ee0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1cb0006fd0 tx=0x7f1cb000db90 comp rx=0 tx=0).stop 2023-10-26T01:31:03.959 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.956+0000 7f1ccccc2640 1 -- 172.21.15.169:0/282273615 shutdown_connections 2023-10-26T01:31:03.959 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.956+0000 7f1ccccc2640 1 --2- 172.21.15.169:0/282273615 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f1ca806ae40 0x7f1ca806d300 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:03.959 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.956+0000 7f1ccccc2640 1 --2- 172.21.15.169:0/282273615 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1cc8098a60 0x7f1cc8098ee0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:03.959 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.956+0000 7f1ccccc2640 1 --2- 172.21.15.169:0/282273615 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1cc814ca20 0x7f1cc814cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:03.960 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.956+0000 7f1ccccc2640 1 --2- 172.21.15.169:0/282273615 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1cb4004d00 0x7f1cc81f1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:03.960 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.956+0000 7f1ccccc2640 1 -- 172.21.15.169:0/282273615 >> 172.21.15.169:0/282273615 conn(0x7f1cc8069510 msgr2=0x7f1cc815a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:03.960 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.956+0000 7f1ccccc2640 1 -- 172.21.15.169:0/282273615 shutdown_connections 2023-10-26T01:31:03.960 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:03.956+0000 7f1ccccc2640 1 -- 172.21.15.169:0/282273615 wait complete. 2023-10-26T01:31:04.131 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T01:31:04.141 DEBUG:teuthology.orchestra.run.smithi170:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T01:31:04.378 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-26T01:30:41.545068+0000', 'modified': '2023-10-26T01:30:44.577169+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4338}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4338': {'gid': 4338, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.170:6837/590609220', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6835', 'nonce': 590609220}, {'type': 'v1', 'addr': '172.21.15.170:6837', 'nonce': 590609220}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-26T01:31:04.379 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-26T01:31:04.379 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-10-26T01:31:04.611 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.609+0000 7f2d27075640 1 -- 172.21.15.169:0/1881759354 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d200cdef0 msgr2=0x7f2d2015b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:04.611 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.609+0000 7f2d27075640 1 --2- 172.21.15.169:0/1881759354 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d200cdef0 0x7f2d2015b4a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c009830 tx=0x7f2d0c02bd60 comp rx=0 tx=0).stop 2023-10-26T01:31:04.611 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.609+0000 7f2d27075640 1 -- 172.21.15.169:0/1881759354 shutdown_connections 2023-10-26T01:31:04.612 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.609+0000 7f2d27075640 1 --2- 172.21.15.169:0/1881759354 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2d2015b9e0 0x7f2d200b05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:04.612 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.609+0000 7f2d27075640 1 --2- 172.21.15.169:0/1881759354 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d200cdef0 0x7f2d2015b4a0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:04.612 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.609+0000 7f2d27075640 1 -- 172.21.15.169:0/1881759354 >> 172.21.15.169:0/1881759354 conn(0x7f2d20069930 msgr2=0x7f2d20069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:04.612 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.609+0000 7f2d27075640 1 -- 172.21.15.169:0/1881759354 shutdown_connections 2023-10-26T01:31:04.612 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.609+0000 7f2d27075640 1 -- 172.21.15.169:0/1881759354 wait complete. 2023-10-26T01:31:04.614 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.611+0000 7f2d27075640 1 Processor -- start 2023-10-26T01:31:04.614 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.611+0000 7f2d27075640 1 -- start start 2023-10-26T01:31:04.614 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.612+0000 7f2d27075640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2d200cdef0 0x7f2d201f16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:04.614 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.612+0000 7f2d27075640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2d20155360 0x7f2d2014c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:04.614 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.612+0000 7f2d27075640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d2015b9e0 0x7f2d2014ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:04.615 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.612+0000 7f2d27075640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d200cd020 con 0x7f2d2015b9e0 2023-10-26T01:31:04.615 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.612+0000 7f2d27075640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d20070d00 con 0x7f2d200cdef0 2023-10-26T01:31:04.615 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.612+0000 7f2d27075640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d200afe30 con 0x7f2d20155360 2023-10-26T01:31:04.615 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.612+0000 7f2d25612640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d2015b9e0 0x7f2d2014ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:04.615 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.612+0000 7f2d25612640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d2015b9e0 0x7f2d2014ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:43294/0 (socket says 172.21.15.169:43294) 2023-10-26T01:31:04.616 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.612+0000 7f2d25612640 1 -- 172.21.15.169:0/3042199149 learned_addr learned my addr 172.21.15.169:0/3042199149 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:31:04.616 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.612+0000 7f2d24e11640 1 --2- 172.21.15.169:0/3042199149 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2d200cdef0 0x7f2d201f16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:04.617 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.612+0000 7f2d25612640 1 -- 172.21.15.169:0/3042199149 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2d20155360 msgr2=0x7f2d2014c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:04.617 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.613+0000 7f2d25612640 1 --2- 172.21.15.169:0/3042199149 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2d20155360 0x7f2d2014c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:04.617 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.613+0000 7f2d25612640 1 -- 172.21.15.169:0/3042199149 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2d200cdef0 msgr2=0x7f2d201f16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:04.618 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.613+0000 7f2d25612640 1 --2- 172.21.15.169:0/3042199149 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2d200cdef0 0x7f2d201f16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:04.618 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.613+0000 7f2d25612640 1 -- 172.21.15.169:0/3042199149 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d0c009490 con 0x7f2d2015b9e0 2023-10-26T01:31:04.618 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.613+0000 7f2d24e11640 1 --2- 172.21.15.169:0/3042199149 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2d200cdef0 0x7f2d201f16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:31:04.618 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.613+0000 7f2d25612640 1 --2- 172.21.15.169:0/3042199149 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d2015b9e0 0x7f2d2014ca00 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2d1800e960 tx=0x7f2d1800ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:04.619 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.613+0000 7f2d1d7fa640 1 -- 172.21.15.169:0/3042199149 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d18004d20 con 0x7f2d2015b9e0 2023-10-26T01:31:04.619 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.613+0000 7f2d1d7fa640 1 -- 172.21.15.169:0/3042199149 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2d1800cac0 con 0x7f2d2015b9e0 2023-10-26T01:31:04.619 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.613+0000 7f2d1d7fa640 1 -- 172.21.15.169:0/3042199149 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d18005020 con 0x7f2d2015b9e0 2023-10-26T01:31:04.619 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.613+0000 7f2d27075640 1 -- 172.21.15.169:0/3042199149 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d200afb50 con 0x7f2d2015b9e0 2023-10-26T01:31:04.619 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.613+0000 7f2d27075640 1 -- 172.21.15.169:0/3042199149 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d201653a0 con 0x7f2d2015b9e0 2023-10-26T01:31:04.620 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.615+0000 7f2d1d7fa640 1 -- 172.21.15.169:0/3042199149 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f2d18005180 con 0x7f2d2015b9e0 2023-10-26T01:31:04.623 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.616+0000 7f2d1d7fa640 1 --2- 172.21.15.169:0/3042199149 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2d0406ae40 0x7f2d0406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:04.623 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.616+0000 7f2d1d7fa640 1 -- 172.21.15.169:0/3042199149 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2d18089f40 con 0x7f2d2015b9e0 2023-10-26T01:31:04.623 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.617+0000 7f2d24e11640 1 --2- 172.21.15.169:0/3042199149 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2d0406ae40 0x7f2d0406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:04.623 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.617+0000 7f2d29c09740 1 Processor -- start 2023-10-26T01:31:04.624 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.617+0000 7f2d29c09740 1 -- start start 2023-10-26T01:31:04.624 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.618+0000 7f2d24e11640 1 --2- 172.21.15.169:0/3042199149 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2d0406ae40 0x7f2d0406d300 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2d140011e0 tx=0x7f2d14009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:04.624 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.618+0000 7f2d29c09740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c7aa457d0 0x555c7aa18a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:04.624 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.618+0000 7f2d29c09740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555c7aa45bb0 0x555c7aa48560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:04.624 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.618+0000 7f2d29c09740 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x555c7aa18f40 0x555c7aa48aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:04.625 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.618+0000 7f2d29c09740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c7a98b740 con 0x555c7aa45bb0 2023-10-26T01:31:04.625 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.618+0000 7f2d29c09740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c7a988e30 con 0x555c7aa457d0 2023-10-26T01:31:04.625 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.618+0000 7f2d29c09740 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x555c7a984720 con 0x555c7aa18f40 2023-10-26T01:31:04.625 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.618+0000 7f2d1ffff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555c7aa45bb0 0x555c7aa48560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:04.626 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.618+0000 7f2d1ffff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555c7aa45bb0 0x555c7aa48560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:43308/0 (socket says 172.21.15.169:43308) 2023-10-26T01:31:04.626 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.619+0000 7f2d1ffff640 1 -- 172.21.15.169:0/2541017731 learned_addr learned my addr 172.21.15.169:0/2541017731 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:31:04.626 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.619+0000 7f2d24e11640 1 --2- 172.21.15.169:0/2541017731 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c7aa457d0 0x555c7aa18a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:04.626 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.619+0000 7f2d25612640 1 --2- 172.21.15.169:0/2541017731 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x555c7aa18f40 0x555c7aa48aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:04.626 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.619+0000 7f2d1ffff640 1 -- 172.21.15.169:0/2541017731 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x555c7aa18f40 msgr2=0x555c7aa48aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T01:31:04.627 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.619+0000 7f2d1ffff640 1 --2- 172.21.15.169:0/2541017731 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x555c7aa18f40 0x555c7aa48aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:04.627 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.619+0000 7f2d1ffff640 1 -- 172.21.15.169:0/2541017731 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c7aa457d0 msgr2=0x555c7aa18a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T01:31:04.627 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.619+0000 7f2d1ffff640 1 --2- 172.21.15.169:0/2541017731 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c7aa457d0 0x555c7aa18a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:04.627 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.619+0000 7f2d1ffff640 1 -- 172.21.15.169:0/2541017731 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555c7a948910 con 0x555c7aa45bb0 2023-10-26T01:31:04.627 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.619+0000 7f2d1ffff640 1 --2- 172.21.15.169:0/2541017731 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555c7aa45bb0 0x555c7aa48560 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c009950 tx=0x7f2d0c005d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81557bc5a10e27cc server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:04.628 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.619+0000 7f2d0a7fc640 1 -- 172.21.15.169:0/2541017731 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c041070 con 0x555c7aa45bb0 2023-10-26T01:31:04.628 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.619+0000 7f2d0a7fc640 1 -- 172.21.15.169:0/2541017731 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2d0c02c8f0 con 0x555c7aa45bb0 2023-10-26T01:31:04.628 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.619+0000 7f2d0a7fc640 1 -- 172.21.15.169:0/2541017731 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c0042b0 con 0x555c7aa45bb0 2023-10-26T01:31:04.628 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.620+0000 7f2d29c09740 1 -- 172.21.15.169:0/2541017731 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555c7aa45bb0 msgr2=0x555c7aa48560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:04.628 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.620+0000 7f2d29c09740 1 --2- 172.21.15.169:0/2541017731 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555c7aa45bb0 0x555c7aa48560 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c009950 tx=0x7f2d0c005d40 comp rx=0 tx=0).stop 2023-10-26T01:31:04.629 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.620+0000 7f2d29c09740 1 -- 172.21.15.169:0/2541017731 shutdown_connections 2023-10-26T01:31:04.629 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.620+0000 7f2d29c09740 1 --2- 172.21.15.169:0/2541017731 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555c7aa45bb0 0x555c7aa48560 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:04.629 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.620+0000 7f2d29c09740 1 --2- 172.21.15.169:0/2541017731 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x555c7aa18f40 0x555c7aa48aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:04.629 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.620+0000 7f2d29c09740 1 --2- 172.21.15.169:0/2541017731 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c7aa457d0 0x555c7aa18a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:04.629 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.620+0000 7f2d29c09740 1 -- 172.21.15.169:0/2541017731 >> 172.21.15.169:0/2541017731 conn(0x555c7aa1ac70 msgr2=0x555c7aa63560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:04.630 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.620+0000 7f2d29c09740 1 -- 172.21.15.169:0/2541017731 shutdown_connections 2023-10-26T01:31:04.630 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.620+0000 7f2d29c09740 1 -- 172.21.15.169:0/2541017731 wait complete. 2023-10-26T01:31:04.630 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.621+0000 7f2d29c09740 1 -- start start 2023-10-26T01:31:04.630 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.621+0000 7f2d29c09740 1 Processor -- start 2023-10-26T01:31:04.633 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.622+0000 7f2d29c09740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555c7aa457d0 0x555c7aa4b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:04.634 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.622+0000 7f2d29c09740 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x555c7aa45bb0 0x555c7aa5db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:04.634 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.622+0000 7f2d29c09740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c7aa61ee0 0x555c7aa62360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:04.634 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.622+0000 7f2d29c09740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c7a922970 con 0x555c7aa457d0 2023-10-26T01:31:04.634 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.622+0000 7f2d29c09740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c7a922610 con 0x555c7aa61ee0 2023-10-26T01:31:04.635 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.622+0000 7f2d29c09740 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x555c7a6e8f20 con 0x555c7aa45bb0 2023-10-26T01:31:04.635 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.622+0000 7f2d29c09740 10 client.0 _ll_register_callbacks cb 0x555c7a9460c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T01:31:04.635 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.622+0000 7f2d24e11640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555c7aa457d0 0x555c7aa4b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:04.635 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.622+0000 7f2d24e11640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555c7aa457d0 0x555c7aa4b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:43322/0 (socket says 172.21.15.169:43322) 2023-10-26T01:31:04.635 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.622+0000 7f2d24e11640 1 -- 172.21.15.169:0/3869583731 learned_addr learned my addr 172.21.15.169:0/3869583731 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:31:04.636 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.623+0000 7f2d24e11640 1 -- 172.21.15.169:0/3869583731 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x555c7aa45bb0 msgr2=0x555c7aa5db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T01:31:04.636 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.623+0000 7f2d24e11640 1 --2- 172.21.15.169:0/3869583731 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x555c7aa45bb0 0x555c7aa5db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:04.636 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.623+0000 7f2d24e11640 1 -- 172.21.15.169:0/3869583731 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c7aa61ee0 msgr2=0x555c7aa62360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T01:31:04.636 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.623+0000 7f2d25612640 1 --2- 172.21.15.169:0/3869583731 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c7aa61ee0 0x555c7aa62360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:04.636 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.623+0000 7f2d24e11640 1 --2- 172.21.15.169:0/3869583731 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c7aa61ee0 0x555c7aa62360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:04.637 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.623+0000 7f2d24e11640 1 -- 172.21.15.169:0/3869583731 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d201580e0 con 0x555c7aa457d0 2023-10-26T01:31:04.637 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.623+0000 7f2d24e11640 1 --2- 172.21.15.169:0/3869583731 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555c7aa457d0 0x555c7aa4b450 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2d1400de20 tx=0x7f2d14000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe4c1eda75ce91b server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:04.637 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.623+0000 7f2cff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.169:3300/0 2023-10-26T01:31:04.637 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.623+0000 7f2cff7fe640 1 -- 172.21.15.169:0/3869583731 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d1400de50 con 0x555c7aa457d0 2023-10-26T01:31:04.637 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.623+0000 7f2cff7fe640 1 -- 172.21.15.169:0/3869583731 <== mon.0 v2:172.21.15.169:3300/0 2 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2d140125e0 con 0x555c7aa457d0 2023-10-26T01:31:04.637 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.623+0000 7f2d29c09740 1 -- 172.21.15.169:0/3869583731 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555c7a97d700 con 0x555c7aa457d0 2023-10-26T01:31:04.638 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.624+0000 7f2cff7fe640 1 -- 172.21.15.169:0/3869583731 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d140127a0 con 0x555c7aa457d0 2023-10-26T01:31:04.638 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.624+0000 7f2d29c09740 10 client.4755 fetch_fsmap learned FSMap version 7 2023-10-26T01:31:04.638 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.624+0000 7f2d29c09740 1 -- 172.21.15.169:0/3869583731 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x555c7a982da0 con 0x555c7aa457d0 2023-10-26T01:31:04.638 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.624+0000 7f2cff7fe640 1 -- 172.21.15.169:0/3869583731 <== mon.0 v2:172.21.15.169:3300/0 4 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x555c7a97d700 con 0x555c7aa457d0 2023-10-26T01:31:04.638 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.624+0000 7f2cff7fe640 1 -- 172.21.15.169:0/3869583731 <== mon.0 v2:172.21.15.169:3300/0 5 ==== fsmap(e 7) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2d14012900 con 0x555c7aa457d0 2023-10-26T01:31:04.639 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.624+0000 7f2d29c09740 10 client.4755 fetch_fsmap finished waiting for FSMap version 7 2023-10-26T01:31:04.639 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.624+0000 7f2d29c09740 10 client.4755 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-26T01:31:04.639 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.624+0000 7f2d29c09740 20 client.4755 populate_metadata read hostname 'smithi169' 2023-10-26T01:31:04.639 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.624+0000 7f2d29c09740 1 --2- 172.21.15.169:0/3869583731 >> [v2:172.21.15.170:6835/590609220,v1:172.21.15.170:6837/590609220] conn(0x555c7aa77bb0 0x555c7aa81250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:04.640 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.624+0000 7f2d29c09740 4 client.4755 mds_command: new command op to 4338 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-26T01:31:04.640 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.624+0000 7f2d29c09740 1 -- 172.21.15.169:0/3869583731 --> [v2:172.21.15.170:6835/590609220,v1:172.21.15.170:6837/590609220] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555c7a97d420 con 0x555c7aa77bb0 2023-10-26T01:31:04.640 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.625+0000 7f2d1ffff640 1 --2- 172.21.15.169:0/3869583731 >> [v2:172.21.15.170:6835/590609220,v1:172.21.15.170:6837/590609220] conn(0x555c7aa77bb0 0x555c7aa81250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:04.640 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.625+0000 7f2d1ffff640 1 --2- 172.21.15.169:0/3869583731 >> [v2:172.21.15.170:6835/590609220,v1:172.21.15.170:6837/590609220] conn(0x555c7aa77bb0 0x555c7aa81250 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=234630f91d5a3a75 server_cookie=9dedc8a9546d2b09 in_seq=0 out_seq=0 2023-10-26T01:31:04.640 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.625+0000 7f2cff7fe640 10 client.4755 ms_handle_connect on v2:172.21.15.170:6835/590609220 2023-10-26T01:31:04.641 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.627+0000 7f2cff7fe640 1 -- 172.21.15.169:0/3869583731 <== mds.0 v2:172.21.15.170:6835/590609220 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x555c7a97d420 con 0x555c7aa77bb0 2023-10-26T01:31:04.641 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.627+0000 7f2cff7fe640 10 client.4755 handle_command_reply: tid=0 2023-10-26T01:31:04.641 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.627+0000 7f2d29c09740 1 client.4755 shutdown 2023-10-26T01:31:04.641 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.627+0000 7f2d29c09740 1 -- 172.21.15.169:0/3869583731 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555c7aa457d0 msgr2=0x555c7aa4b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:04.641 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.627+0000 7f2d29c09740 1 --2- 172.21.15.169:0/3869583731 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555c7aa457d0 0x555c7aa4b450 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2d1400de20 tx=0x7f2d14000ed0 comp rx=0 tx=0).stop 2023-10-26T01:31:04.642 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.627+0000 7f2d29c09740 1 -- 172.21.15.169:0/3869583731 shutdown_connections 2023-10-26T01:31:04.642 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.627+0000 7f2d29c09740 1 --2- 172.21.15.169:0/3869583731 >> [v2:172.21.15.170:6835/590609220,v1:172.21.15.170:6837/590609220] conn(0x555c7aa77bb0 0x555c7aa81250 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:04.642 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.627+0000 7f2d29c09740 1 --2- 172.21.15.169:0/3869583731 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c7aa61ee0 0x555c7aa62360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:04.642 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.627+0000 7f2d29c09740 1 --2- 172.21.15.169:0/3869583731 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x555c7aa45bb0 0x555c7aa5db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:04.642 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.627+0000 7f2d29c09740 1 --2- 172.21.15.169:0/3869583731 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555c7aa457d0 0x555c7aa4b450 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:04.643 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.627+0000 7f2cff7fe640 0 client.4755 ms_handle_reset on v2:172.21.15.170:6835/590609220 2023-10-26T01:31:04.643 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.627+0000 7f2d29c09740 1 -- 172.21.15.169:0/3869583731 >> 172.21.15.169:0/3869583731 conn(0x555c7aa18f40 msgr2=0x555c7aa1c650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:04.643 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.628+0000 7f2d29c09740 1 -- 172.21.15.169:0/3869583731 shutdown_connections 2023-10-26T01:31:04.643 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.628+0000 7f2d29c09740 1 -- 172.21.15.169:0/3869583731 wait complete. 2023-10-26T01:31:04.643 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.628+0000 7f2d29c09740 20 client.4755 trim_cache size 0 max 16384 2023-10-26T01:31:04.645 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.644+0000 7f2d29c09740 1 Processor -- start 2023-10-26T01:31:04.646 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.644+0000 7f2d29c09740 1 -- start start 2023-10-26T01:31:04.646 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.645+0000 7f2d29c09740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c7aa457d0 0x555c7aa4a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:04.647 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.645+0000 7f2d29c09740 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x555c7aa45bb0 0x555c7aa615b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:04.647 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.645+0000 7f2d29c09740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555c7aa61ee0 0x555c7aa5d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:04.647 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.645+0000 7f2d29c09740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c7aa136a0 con 0x555c7aa61ee0 2023-10-26T01:31:04.647 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.645+0000 7f2d29c09740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c7a986600 con 0x555c7aa457d0 2023-10-26T01:31:04.647 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.645+0000 7f2d29c09740 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x555c7aa77450 con 0x555c7aa45bb0 2023-10-26T01:31:04.649 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.645+0000 7f2d25612640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555c7aa61ee0 0x555c7aa5d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:04.649 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.645+0000 7f2d1ffff640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x555c7aa45bb0 0x555c7aa615b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:04.650 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.645+0000 7f2d25612640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555c7aa61ee0 0x555c7aa5d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:43330/0 (socket says 172.21.15.169:43330) 2023-10-26T01:31:04.650 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.645+0000 7f2d25612640 1 -- 172.21.15.169:0/3245395928 learned_addr learned my addr 172.21.15.169:0/3245395928 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:31:04.650 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.645+0000 7f2d1ffff640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x555c7aa45bb0 0x555c7aa615b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3301/0 says I am v2:172.21.15.169:48420/0 (socket says 172.21.15.169:48420) 2023-10-26T01:31:04.650 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.646+0000 7f2d24e11640 1 --2- 172.21.15.169:0/3245395928 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c7aa457d0 0x555c7aa4a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:04.650 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.646+0000 7f2d25612640 1 -- 172.21.15.169:0/3245395928 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x555c7aa45bb0 msgr2=0x555c7aa615b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T01:31:04.651 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.646+0000 7f2d25612640 1 --2- 172.21.15.169:0/3245395928 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x555c7aa45bb0 0x555c7aa615b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:04.651 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.646+0000 7f2d25612640 1 -- 172.21.15.169:0/3245395928 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c7aa457d0 msgr2=0x555c7aa4a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T01:31:04.651 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.646+0000 7f2d25612640 1 --2- 172.21.15.169:0/3245395928 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c7aa457d0 0x555c7aa4a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:04.651 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.646+0000 7f2d25612640 1 -- 172.21.15.169:0/3245395928 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555c7aa73830 con 0x555c7aa61ee0 2023-10-26T01:31:04.651 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.646+0000 7f2d25612640 1 --2- 172.21.15.169:0/3245395928 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555c7aa61ee0 0x555c7aa5d800 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2d18009270 tx=0x7f2d180593a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30fc7327d7e7771c server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:04.652 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.646+0000 7f2d0affd640 1 -- 172.21.15.169:0/3245395928 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d18088170 con 0x555c7aa61ee0 2023-10-26T01:31:04.652 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.646+0000 7f2d0affd640 1 -- 172.21.15.169:0/3245395928 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2d18060fa0 con 0x555c7aa61ee0 2023-10-26T01:31:04.652 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.646+0000 7f2d0affd640 1 -- 172.21.15.169:0/3245395928 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d180553b0 con 0x555c7aa61ee0 2023-10-26T01:31:04.652 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.647+0000 7f2d29c09740 1 -- 172.21.15.169:0/3245395928 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555c7aa61ee0 msgr2=0x555c7aa5d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:04.652 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.647+0000 7f2d29c09740 1 --2- 172.21.15.169:0/3245395928 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555c7aa61ee0 0x555c7aa5d800 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2d18009270 tx=0x7f2d180593a0 comp rx=0 tx=0).stop 2023-10-26T01:31:04.653 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.647+0000 7f2d29c09740 1 -- 172.21.15.169:0/3245395928 shutdown_connections 2023-10-26T01:31:04.653 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.647+0000 7f2d29c09740 1 --2- 172.21.15.169:0/3245395928 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555c7aa61ee0 0x555c7aa5d800 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:04.653 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.647+0000 7f2d29c09740 1 --2- 172.21.15.169:0/3245395928 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x555c7aa45bb0 0x555c7aa615b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:04.653 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.647+0000 7f2d29c09740 1 --2- 172.21.15.169:0/3245395928 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c7aa457d0 0x555c7aa4a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:04.653 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.647+0000 7f2d29c09740 1 -- 172.21.15.169:0/3245395928 >> 172.21.15.169:0/3245395928 conn(0x555c7aa18f40 msgr2=0x555c7aa72c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:04.654 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.647+0000 7f2d29c09740 1 -- 172.21.15.169:0/3245395928 shutdown_connections 2023-10-26T01:31:04.654 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.647+0000 7f2d29c09740 1 -- 172.21.15.169:0/3245395928 wait complete. 2023-10-26T01:31:04.654 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.648+0000 7f2d29c09740 1 -- start start 2023-10-26T01:31:04.654 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.648+0000 7f2d29c09740 1 Processor -- start 2023-10-26T01:31:04.657 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.649+0000 7f2d29c09740 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x555c7aa457d0 0x555c7aacb380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:04.657 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.649+0000 7f2d29c09740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c7aa45bb0 0x555c7aa5d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:04.658 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.649+0000 7f2d29c09740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555c7aa61ee0 0x555c7aa5dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:04.658 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.649+0000 7f2d29c09740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c7aa61a30 con 0x555c7aa61ee0 2023-10-26T01:31:04.658 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.649+0000 7f2d29c09740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c7aa18120 con 0x555c7aa45bb0 2023-10-26T01:31:04.658 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.649+0000 7f2d29c09740 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x555c7aa61650 con 0x555c7aa457d0 2023-10-26T01:31:04.658 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.649+0000 7f2d29c09740 10 client.0 _ll_register_callbacks cb 0x7f2d0c037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T01:31:04.659 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.649+0000 7f2d25612640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555c7aa61ee0 0x555c7aa5dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:04.659 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.649+0000 7f2d25612640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555c7aa61ee0 0x555c7aa5dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:43346/0 (socket says 172.21.15.169:43346) 2023-10-26T01:31:04.659 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.649+0000 7f2d25612640 1 -- 172.21.15.169:0/1631282261 learned_addr learned my addr 172.21.15.169:0/1631282261 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:31:04.659 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.649+0000 7f2d25612640 1 -- 172.21.15.169:0/1631282261 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x555c7aa457d0 msgr2=0x555c7aacb380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T01:31:04.659 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.649+0000 7f2d25612640 1 --2- 172.21.15.169:0/1631282261 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x555c7aa457d0 0x555c7aacb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:04.660 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.649+0000 7f2d25612640 1 -- 172.21.15.169:0/1631282261 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c7aa45bb0 msgr2=0x555c7aa5d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T01:31:04.660 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.649+0000 7f2d25612640 1 --2- 172.21.15.169:0/1631282261 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c7aa45bb0 0x555c7aa5d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:04.660 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.649+0000 7f2d25612640 1 -- 172.21.15.169:0/1631282261 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d201653a0 con 0x555c7aa61ee0 2023-10-26T01:31:04.660 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.650+0000 7f2d25612640 1 --2- 172.21.15.169:0/1631282261 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555c7aa61ee0 0x555c7aa5dd00 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2d18058c30 tx=0x7f2d180550d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8fabac3460ed1f2 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:04.660 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.650+0000 7f2cff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.169:3300/0 2023-10-26T01:31:04.661 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.650+0000 7f2cff7fe640 1 -- 172.21.15.169:0/1631282261 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d18060d10 con 0x555c7aa61ee0 2023-10-26T01:31:04.661 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.650+0000 7f2cff7fe640 1 -- 172.21.15.169:0/1631282261 <== mon.0 v2:172.21.15.169:3300/0 2 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2d18088170 con 0x555c7aa61ee0 2023-10-26T01:31:04.661 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.650+0000 7f2d29c09740 1 -- 172.21.15.169:0/1631282261 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2d0c02c8f0 con 0x555c7aa61ee0 2023-10-26T01:31:04.661 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.650+0000 7f2cff7fe640 1 -- 172.21.15.169:0/1631282261 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d1805db30 con 0x555c7aa61ee0 2023-10-26T01:31:04.661 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.650+0000 7f2d29c09740 10 client.4761 fetch_fsmap learned FSMap version 7 2023-10-26T01:31:04.662 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.650+0000 7f2d29c09740 1 -- 172.21.15.169:0/1631282261 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x555c7a921b80 con 0x555c7aa61ee0 2023-10-26T01:31:04.662 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.650+0000 7f2cff7fe640 1 -- 172.21.15.169:0/1631282261 <== mon.0 v2:172.21.15.169:3300/0 4 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2d0c02c8f0 con 0x555c7aa61ee0 2023-10-26T01:31:04.662 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.650+0000 7f2cff7fe640 1 -- 172.21.15.169:0/1631282261 <== mon.0 v2:172.21.15.169:3300/0 5 ==== fsmap(e 7) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2d18018030 con 0x555c7aa61ee0 2023-10-26T01:31:04.662 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.650+0000 7f2d29c09740 10 client.4761 fetch_fsmap finished waiting for FSMap version 7 2023-10-26T01:31:04.662 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.650+0000 7f2d29c09740 10 client.4761 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-26T01:31:04.663 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.650+0000 7f2d29c09740 20 client.4761 populate_metadata read hostname 'smithi169' 2023-10-26T01:31:04.663 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.650+0000 7f2d29c09740 1 --2- 172.21.15.169:0/1631282261 >> [v2:172.21.15.170:6835/590609220,v1:172.21.15.170:6837/590609220] conn(0x555c7aa75f00 0x555c7aa762e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:04.663 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.650+0000 7f2d29c09740 4 client.4761 mds_command: new command op to 4338 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-26T01:31:04.663 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.650+0000 7f2d29c09740 1 -- 172.21.15.169:0/1631282261 --> [v2:172.21.15.170:6835/590609220,v1:172.21.15.170:6837/590609220] -- command(tid 0: {"prefix": "status"}) v1 -- 0x555c7a6866c0 con 0x555c7aa75f00 2023-10-26T01:31:04.663 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.651+0000 7f2d24e11640 1 --2- 172.21.15.169:0/1631282261 >> [v2:172.21.15.170:6835/590609220,v1:172.21.15.170:6837/590609220] conn(0x555c7aa75f00 0x555c7aa762e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:04.664 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.652+0000 7f2d24e11640 1 --2- 172.21.15.169:0/1631282261 >> [v2:172.21.15.170:6835/590609220,v1:172.21.15.170:6837/590609220] conn(0x555c7aa75f00 0x555c7aa762e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c879eaea567703c4 server_cookie=ec634c58a4c91fb1 in_seq=0 out_seq=0 2023-10-26T01:31:04.664 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.652+0000 7f2cff7fe640 10 client.4761 ms_handle_connect on v2:172.21.15.170:6835/590609220 2023-10-26T01:31:04.664 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.652+0000 7f2cff7fe640 1 -- 172.21.15.169:0/1631282261 <== mds.0 v2:172.21.15.170:6835/590609220 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x555c7a6866c0 con 0x555c7aa75f00 2023-10-26T01:31:04.664 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.652+0000 7f2cff7fe640 10 client.4761 handle_command_reply: tid=0 2023-10-26T01:31:04.664 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.652+0000 7f2d29c09740 1 client.4761 shutdown 2023-10-26T01:31:04.665 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.653+0000 7f2d29c09740 1 -- 172.21.15.169:0/1631282261 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555c7aa61ee0 msgr2=0x555c7aa5dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:04.665 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.653+0000 7f2d29c09740 1 --2- 172.21.15.169:0/1631282261 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555c7aa61ee0 0x555c7aa5dd00 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2d18058c30 tx=0x7f2d180550d0 comp rx=0 tx=0).stop 2023-10-26T01:31:04.665 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.653+0000 7f2d29c09740 1 -- 172.21.15.169:0/1631282261 shutdown_connections 2023-10-26T01:31:04.665 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.653+0000 7f2d29c09740 1 --2- 172.21.15.169:0/1631282261 >> [v2:172.21.15.170:6835/590609220,v1:172.21.15.170:6837/590609220] conn(0x555c7aa75f00 0x555c7aa762e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:04.665 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.653+0000 7f2d29c09740 1 --2- 172.21.15.169:0/1631282261 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555c7aa61ee0 0x555c7aa5dd00 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:04.666 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.653+0000 7f2d29c09740 1 --2- 172.21.15.169:0/1631282261 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c7aa45bb0 0x555c7aa5d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:04.666 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.653+0000 7f2cff7fe640 0 client.4761 ms_handle_reset on v2:172.21.15.170:6835/590609220 2023-10-26T01:31:04.666 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.653+0000 7f2d29c09740 1 --2- 172.21.15.169:0/1631282261 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x555c7aa457d0 0x555c7aacb380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:04.666 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.653+0000 7f2d29c09740 1 -- 172.21.15.169:0/1631282261 >> 172.21.15.169:0/1631282261 conn(0x555c7aa18f40 msgr2=0x555c7aa4bf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:04.666 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.653+0000 7f2d29c09740 1 -- 172.21.15.169:0/1631282261 shutdown_connections 2023-10-26T01:31:04.667 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.653+0000 7f2d29c09740 1 -- 172.21.15.169:0/1631282261 wait complete. 2023-10-26T01:31:04.667 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.653+0000 7f2d29c09740 20 client.4761 trim_cache size 0 max 16384 2023-10-26T01:31:04.667 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2023-10-26T01:31:04.667 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_fsid": "30203b28-ed8a-44e8-a9a6-a9bf25f980f1", 2023-10-26T01:31:04.668 INFO:teuthology.orchestra.run.smithi169.stdout: "whoami": 0, 2023-10-26T01:31:04.668 INFO:teuthology.orchestra.run.smithi169.stdout: "id": 4338, 2023-10-26T01:31:04.668 INFO:teuthology.orchestra.run.smithi169.stdout: "want_state": "up:active", 2023-10-26T01:31:04.668 INFO:teuthology.orchestra.run.smithi169.stdout: "state": "up:active", 2023-10-26T01:31:04.668 INFO:teuthology.orchestra.run.smithi169.stdout: "fs_name": "cephfs", 2023-10-26T01:31:04.669 INFO:teuthology.orchestra.run.smithi169.stdout: "rank_uptime": 22.572120017, 2023-10-26T01:31:04.669 INFO:teuthology.orchestra.run.smithi169.stdout: "mdsmap_epoch": 7, 2023-10-26T01:31:04.669 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_epoch": 21, 2023-10-26T01:31:04.669 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_epoch_barrier": 21, 2023-10-26T01:31:04.669 INFO:teuthology.orchestra.run.smithi169.stdout: "uptime": 28.091742887999999 2023-10-26T01:31:04.669 INFO:teuthology.orchestra.run.smithi169.stdout:} 2023-10-26T01:31:04.670 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.653+0000 7f2d27075640 1 -- 172.21.15.169:0/3042199149 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2d0406ae40 msgr2=0x7f2d0406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:04.670 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.653+0000 7f2d27075640 1 --2- 172.21.15.169:0/3042199149 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2d0406ae40 0x7f2d0406d300 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2d140011e0 tx=0x7f2d14009040 comp rx=0 tx=0).stop 2023-10-26T01:31:04.671 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.653+0000 7f2d27075640 1 -- 172.21.15.169:0/3042199149 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d2015b9e0 msgr2=0x7f2d2014ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:04.671 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.654+0000 7f2d27075640 1 --2- 172.21.15.169:0/3042199149 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d2015b9e0 0x7f2d2014ca00 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2d1800e960 tx=0x7f2d1800ee20 comp rx=0 tx=0).stop 2023-10-26T01:31:04.671 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.654+0000 7f2d27075640 1 -- 172.21.15.169:0/3042199149 shutdown_connections 2023-10-26T01:31:04.671 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.654+0000 7f2d27075640 1 --2- 172.21.15.169:0/3042199149 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2d0406ae40 0x7f2d0406d300 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:04.671 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.654+0000 7f2d27075640 1 --2- 172.21.15.169:0/3042199149 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d2015b9e0 0x7f2d2014ca00 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:04.672 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.654+0000 7f2d27075640 1 --2- 172.21.15.169:0/3042199149 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2d20155360 0x7f2d2014c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:04.672 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.654+0000 7f2d27075640 1 --2- 172.21.15.169:0/3042199149 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2d200cdef0 0x7f2d201f16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:04.672 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.654+0000 7f2d27075640 1 -- 172.21.15.169:0/3042199149 >> 172.21.15.169:0/3042199149 conn(0x7f2d20069930 msgr2=0x7f2d20157120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:04.672 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.654+0000 7f2d27075640 1 -- 172.21.15.169:0/3042199149 shutdown_connections 2023-10-26T01:31:04.672 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:04.654+0000 7f2d27075640 1 -- 172.21.15.169:0/3042199149 wait complete. 2023-10-26T01:31:04.674 INFO:teuthology.run_tasks:Running task kclient... 2023-10-26T01:31:04.696 INFO:tasks.kclient:Mounting kernel clients... 2023-10-26T01:31:04.697 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-10-26T01:31:04.697 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:31:04.698 DEBUG:teuthology.orchestra.run.smithi169:> ip netns list 2023-10-26T01:31:04.752 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:31:04.753 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link delete ceph-brx 2023-10-26T01:31:04.849 INFO:teuthology.orchestra.run.smithi169.stderr:Cannot find device "ceph-brx" 2023-10-26T01:31:04.852 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-26T01:31:04.853 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:31:04.853 DEBUG:teuthology.orchestra.run.smithi169:> ip netns list 2023-10-26T01:31:04.905 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:31:04.905 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link delete ceph-brx 2023-10-26T01:31:05.006 INFO:teuthology.orchestra.run.smithi169.stderr:Cannot find device "ceph-brx" 2023-10-26T01:31:05.009 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-26T01:31:05.010 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:31:05.010 DEBUG:teuthology.orchestra.run.smithi170:> ip netns list 2023-10-26T01:31:05.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:31:05.072 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip link delete ceph-brx 2023-10-26T01:31:05.179 INFO:teuthology.orchestra.run.smithi170.stderr:Cannot find device "ceph-brx" 2023-10-26T01:31:05.183 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-26T01:31:05.184 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:31:05.184 DEBUG:teuthology.orchestra.run.smithi170:> ip netns list 2023-10-26T01:31:05.236 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:31:05.237 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip link delete ceph-brx 2023-10-26T01:31:05.336 INFO:teuthology.orchestra.run.smithi170.stderr:Cannot find device "ceph-brx" 2023-10-26T01:31:05.339 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-26T01:31:05.340 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2023-10-26T01:31:05.340 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-10-26T01:31:05.341 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-26T01:31:05.341 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-26T01:31:05.572 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:05.570+0000 7f6257f9e640 1 -- 172.21.15.169:0/356757633 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62501593e0 msgr2=0x7f62500ac200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:05.572 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:05.570+0000 7f6257f9e640 1 --2- 172.21.15.169:0/356757633 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62501593e0 0x7f62500ac200 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f624c009830 tx=0x7f624c02bd60 comp rx=0 tx=0).stop 2023-10-26T01:31:05.572 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:05.570+0000 7f6257f9e640 1 -- 172.21.15.169:0/356757633 shutdown_connections 2023-10-26T01:31:05.572 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:05.570+0000 7f6257f9e640 1 --2- 172.21.15.169:0/356757633 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62501593e0 0x7f62500ac200 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:05.573 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:05.570+0000 7f6257f9e640 1 -- 172.21.15.169:0/356757633 >> 172.21.15.169:0/356757633 conn(0x7f6250069930 msgr2=0x7f6250069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:05.573 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:05.570+0000 7f6257f9e640 1 -- 172.21.15.169:0/356757633 shutdown_connections 2023-10-26T01:31:05.573 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:05.570+0000 7f6257f9e640 1 -- 172.21.15.169:0/356757633 wait complete. 2023-10-26T01:31:05.574 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:05.572+0000 7f6257f9e640 1 Processor -- start 2023-10-26T01:31:05.574 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:05.572+0000 7f6257f9e640 1 -- start start 2023-10-26T01:31:05.576 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:05.573+0000 7f6257f9e640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f62500b0c80 0x7f6250096930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:05.576 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:05.573+0000 7f6257f9e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f62501593e0 0x7f62500aee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:05.576 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:05.573+0000 7f6257f9e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62501597c0 0x7f62500af360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:05.576 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:05.573+0000 7f6257f9e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f625015c540 con 0x7f62501597c0 2023-10-26T01:31:05.577 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:05.573+0000 7f6257f9e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6250155a60 con 0x7f62501593e0 2023-10-26T01:31:05.577 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:05.573+0000 7f6257f9e640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f625015bca0 con 0x7f62500b0c80 2023-10-26T01:31:05.577 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:05.573+0000 7f625653b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62501597c0 0x7f62500af360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:05.577 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:05.573+0000 7f625653b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62501597c0 0x7f62500af360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:43356/0 (socket says 172.21.15.169:43356) 2023-10-26T01:31:05.577 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:05.573+0000 7f625653b640 1 -- 172.21.15.169:0/4068241535 learned_addr learned my addr 172.21.15.169:0/4068241535 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:31:05.578 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:05.573+0000 7f6255539640 1 --2- 172.21.15.169:0/4068241535 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f62501593e0 0x7f62500aee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:05.578 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:05.573+0000 7f625653b640 1 -- 172.21.15.169:0/4068241535 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f62500b0c80 msgr2=0x7f6250096930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:05.578 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:05.573+0000 7f625653b640 1 --2- 172.21.15.169:0/4068241535 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f62500b0c80 0x7f6250096930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:05.578 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:05.573+0000 7f625653b640 1 -- 172.21.15.169:0/4068241535 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f62501593e0 msgr2=0x7f62500aee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:05.578 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:05.573+0000 7f625653b640 1 --2- 172.21.15.169:0/4068241535 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f62501593e0 0x7f62500aee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:05.579 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:05.573+0000 7f625653b640 1 -- 172.21.15.169:0/4068241535 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f624c009490 con 0x7f62501597c0 2023-10-26T01:31:05.579 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:05.574+0000 7f625653b640 1 --2- 172.21.15.169:0/4068241535 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62501597c0 0x7f62500af360 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f624800c790 tx=0x7f624800cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:05.579 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:05.574+0000 7f62467fc640 1 -- 172.21.15.169:0/4068241535 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f624800cea0 con 0x7f62501597c0 2023-10-26T01:31:05.579 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:05.574+0000 7f62467fc640 1 -- 172.21.15.169:0/4068241535 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6248003da0 con 0x7f62501597c0 2023-10-26T01:31:05.579 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:05.574+0000 7f62467fc640 1 -- 172.21.15.169:0/4068241535 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62480052d0 con 0x7f62501597c0 2023-10-26T01:31:05.580 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:05.574+0000 7f6257f9e640 1 -- 172.21.15.169:0/4068241535 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f625015bf80 con 0x7f62501597c0 2023-10-26T01:31:05.580 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:05.574+0000 7f6257f9e640 1 -- 172.21.15.169:0/4068241535 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6250096e70 con 0x7f62501597c0 2023-10-26T01:31:05.581 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:05.575+0000 7f6257f9e640 1 -- 172.21.15.169:0/4068241535 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f625004db10 con 0x7f62501597c0 2023-10-26T01:31:05.581 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:05.576+0000 7f62467fc640 1 -- 172.21.15.169:0/4068241535 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f6248005430 con 0x7f62501597c0 2023-10-26T01:31:05.581 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:05.577+0000 7f62467fc640 1 --2- 172.21.15.169:0/4068241535 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f623406ae40 0x7f623406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:05.581 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:05.577+0000 7f62467fc640 1 -- 172.21.15.169:0/4068241535 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f62480895f0 con 0x7f62501597c0 2023-10-26T01:31:05.582 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:05.578+0000 7f6255d3a640 1 --2- 172.21.15.169:0/4068241535 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f623406ae40 0x7f623406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:05.582 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:05.579+0000 7f6255d3a640 1 --2- 172.21.15.169:0/4068241535 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f623406ae40 0x7f623406d300 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f624c036000 tx=0x7f624c0046c0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:05.583 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:05.581+0000 7f62467fc640 1 -- 172.21.15.169:0/4068241535 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6248054690 con 0x7f62501597c0 2023-10-26T01:31:05.815 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:05.814+0000 7f6257f9e640 1 -- 172.21.15.169:0/4068241535 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f6250155ba0 con 0x7f62501597c0 2023-10-26T01:31:05.816 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:05.815+0000 7f62467fc640 1 -- 172.21.15.169:0/4068241535 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+73 (secure 0 0 0) 0x7f6248057cb0 con 0x7f62501597c0 2023-10-26T01:31:05.817 INFO:teuthology.orchestra.run.smithi169.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-26T01:31:05.820 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:05.818+0000 7f6257f9e640 1 -- 172.21.15.169:0/4068241535 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f623406ae40 msgr2=0x7f623406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:05.820 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:05.818+0000 7f6257f9e640 1 --2- 172.21.15.169:0/4068241535 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f623406ae40 0x7f623406d300 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f624c036000 tx=0x7f624c0046c0 comp rx=0 tx=0).stop 2023-10-26T01:31:05.820 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:05.818+0000 7f6257f9e640 1 -- 172.21.15.169:0/4068241535 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62501597c0 msgr2=0x7f62500af360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:05.820 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:05.818+0000 7f6257f9e640 1 --2- 172.21.15.169:0/4068241535 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62501597c0 0x7f62500af360 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f624800c790 tx=0x7f624800cc50 comp rx=0 tx=0).stop 2023-10-26T01:31:05.820 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:05.818+0000 7f6257f9e640 1 -- 172.21.15.169:0/4068241535 shutdown_connections 2023-10-26T01:31:05.821 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:05.818+0000 7f6257f9e640 1 --2- 172.21.15.169:0/4068241535 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f623406ae40 0x7f623406d300 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:05.821 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:05.818+0000 7f6257f9e640 1 --2- 172.21.15.169:0/4068241535 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62501597c0 0x7f62500af360 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:05.821 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:05.818+0000 7f6257f9e640 1 --2- 172.21.15.169:0/4068241535 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f62501593e0 0x7f62500aee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:05.821 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:05.818+0000 7f6257f9e640 1 --2- 172.21.15.169:0/4068241535 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f62500b0c80 0x7f6250096930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:05.821 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:05.818+0000 7f6257f9e640 1 -- 172.21.15.169:0/4068241535 >> 172.21.15.169:0/4068241535 conn(0x7f6250069930 msgr2=0x7f6250154c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:05.822 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:05.818+0000 7f6257f9e640 1 -- 172.21.15.169:0/4068241535 shutdown_connections 2023-10-26T01:31:05.822 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:05.818+0000 7f6257f9e640 1 -- 172.21.15.169:0/4068241535 wait complete. 2023-10-26T01:31:05.838 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-26T01:31:05.838 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-26T01:31:05.838 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi169.front.sepia.ceph.com 2023-10-26T01:31:05.839 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-26T01:31:05.839 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-26T01:31:05.839 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-26T01:31:05.839 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-26T01:31:05.840 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-26T01:31:05.840 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-26T01:31:05.840 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:31:05.840 DEBUG:teuthology.orchestra.run.smithi169:> ip addr 2023-10-26T01:31:05.896 INFO:teuthology.orchestra.run.smithi169.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-26T01:31:05.896 INFO:teuthology.orchestra.run.smithi169.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-26T01:31:05.896 INFO:teuthology.orchestra.run.smithi169.stdout: inet 127.0.0.1/8 scope host lo 2023-10-26T01:31:05.897 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:31:05.897 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 ::1/128 scope host 2023-10-26T01:31:05.897 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:31:05.898 INFO:teuthology.orchestra.run.smithi169.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-26T01:31:05.898 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:d9:3b:b4 brd ff:ff:ff:ff:ff:ff 2023-10-26T01:31:05.898 INFO:teuthology.orchestra.run.smithi169.stdout: altname enp7s0f0 2023-10-26T01:31:05.898 INFO:teuthology.orchestra.run.smithi169.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-26T01:31:05.899 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:d9:3b:b5 brd ff:ff:ff:ff:ff:ff 2023-10-26T01:31:05.899 INFO:teuthology.orchestra.run.smithi169.stdout: altname enp7s0f1 2023-10-26T01:31:05.899 INFO:teuthology.orchestra.run.smithi169.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-26T01:31:05.899 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:88:7f:4c brd ff:ff:ff:ff:ff:ff 2023-10-26T01:31:05.899 INFO:teuthology.orchestra.run.smithi169.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-26T01:31:05.900 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:88:7f:4d brd ff:ff:ff:ff:ff:ff 2023-10-26T01:31:05.900 INFO:teuthology.orchestra.run.smithi169.stdout: inet 172.21.15.169/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-26T01:31:05.900 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft 42430sec preferred_lft 42430sec 2023-10-26T01:31:05.900 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 fe80::ec4:7aff:fe88:7f4d/64 scope link 2023-10-26T01:31:05.901 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:31:05.901 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-26T01:31:05.902 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:31:05.903 DEBUG:teuthology.orchestra.run.smithi169:> (cd / && exec bash -c ' 2023-10-26T01:31:05.904 DEBUG:teuthology.orchestra.run.smithi169:> set -e 2023-10-26T01:31:05.904 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link add name ceph-brx type bridge 2023-10-26T01:31:05.905 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip addr flush dev ceph-brx 2023-10-26T01:31:05.905 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link set ceph-brx up 2023-10-26T01:31:05.906 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-26T01:31:05.906 DEBUG:teuthology.orchestra.run.smithi169:> ') 2023-10-26T01:31:06.390 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:31:06.391 DEBUG:teuthology.orchestra.run.smithi169:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-26T01:31:06.473 INFO:teuthology.orchestra.run.smithi169.stdout:1 2023-10-26T01:31:06.476 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:31:06.477 DEBUG:teuthology.orchestra.run.smithi169:> route 2023-10-26T01:31:06.532 INFO:teuthology.orchestra.run.smithi169.stdout:Kernel IP routing table 2023-10-26T01:31:06.532 INFO:teuthology.orchestra.run.smithi169.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-26T01:31:06.532 INFO:teuthology.orchestra.run.smithi169.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-26T01:31:06.533 INFO:teuthology.orchestra.run.smithi169.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-26T01:31:06.533 INFO:teuthology.orchestra.run.smithi169.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-26T01:31:06.533 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:31:06.534 DEBUG:teuthology.orchestra.run.smithi169:> (cd / && exec bash -c ' 2023-10-26T01:31:06.534 DEBUG:teuthology.orchestra.run.smithi169:> set -e 2023-10-26T01:31:06.534 DEBUG:teuthology.orchestra.run.smithi169:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-26T01:31:06.535 DEBUG:teuthology.orchestra.run.smithi169:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-26T01:31:06.535 DEBUG:teuthology.orchestra.run.smithi169:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-26T01:31:06.535 DEBUG:teuthology.orchestra.run.smithi169:> ') 2023-10-26T01:31:07.214 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:31:07.215 DEBUG:teuthology.orchestra.run.smithi169:> ip netns list 2023-10-26T01:31:07.260 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:31:07.261 DEBUG:teuthology.orchestra.run.smithi169:> ip netns list-id 2023-10-26T01:31:07.353 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:31:07.354 DEBUG:teuthology.orchestra.run.smithi169:> (cd / && exec bash -c ' 2023-10-26T01:31:07.354 DEBUG:teuthology.orchestra.run.smithi169:> set -e 2023-10-26T01:31:07.354 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-26T01:31:07.355 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-10-26T01:31:07.355 DEBUG:teuthology.orchestra.run.smithi169:> ') 2023-10-26T01:31:07.485 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-10-26T01:31:07.485 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:31:07.485 DEBUG:teuthology.orchestra.run.smithi169:> (cd / && exec bash -c ' 2023-10-26T01:31:07.486 DEBUG:teuthology.orchestra.run.smithi169:> set -e 2023-10-26T01:31:07.486 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-10-26T01:31:07.486 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-26T01:31:07.487 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-10-26T01:31:07.487 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-10-26T01:31:07.487 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-10-26T01:31:07.488 DEBUG:teuthology.orchestra.run.smithi169:> ') 2023-10-26T01:31:07.760 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:31:07.761 DEBUG:teuthology.orchestra.run.smithi169:> (cd / && exec bash -c ' 2023-10-26T01:31:07.761 DEBUG:teuthology.orchestra.run.smithi169:> set -e 2023-10-26T01:31:07.761 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link set brx.0 up 2023-10-26T01:31:07.762 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link set dev brx.0 master ceph-brx 2023-10-26T01:31:07.762 DEBUG:teuthology.orchestra.run.smithi169:> ') 2023-10-26T01:31:07.917 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-26T01:31:07.917 DEBUG:teuthology.orchestra.run.smithi169:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-26T01:31:07.959 INFO:teuthology.orchestra.run.smithi169.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-26T01:31:07.959 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-26T01:31:07.959 DEBUG:teuthology.orchestra.run.smithi169:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-26T01:31:08.049 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-26T01:31:08.050 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:31:08.050 DEBUG:teuthology.orchestra.run.smithi169:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-26T01:31:08.188 INFO:teuthology.orchestra.run.smithi169.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-26T01:31:08.188 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-26T01:31:08.189 INFO:teuthology.orchestra.run.smithi169.stdout:invalid new device string format 2023-10-26T01:31:08.650 INFO:teuthology.orchestra.run.smithi169.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-26T01:31:08.651 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-26T01:31:08.651 INFO:teuthology.orchestra.run.smithi169.stdout:invalid new device string format 2023-10-26T01:31:08.651 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: resolved to: "172.21.15.169:3300,172.21.15.170:3300,172.21.15.170:3301" 2023-10-26T01:31:08.651 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-26T01:31:08.681 INFO:teuthology.orchestra.run.smithi169.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-26T01:31:08.681 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-26T01:31:08.681 INFO:teuthology.orchestra.run.smithi169.stdout:invalid new device string format 2023-10-26T01:31:08.682 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: resolved to: "172.21.15.169:3300,172.21.15.170:3300,172.21.15.170:3301" 2023-10-26T01:31:08.682 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-26T01:31:08.682 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: resolved to: "172.21.15.169:6789,172.21.15.170:6789,172.21.15.170:6790" 2023-10-26T01:31:08.682 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.169:6789,172.21.15.170:6789,172.21.15.170:6790:/ 2023-10-26T01:31:08.682 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=30203b28-ed8a-44e8-a9a6-a9bf25f980f1" will pass to kernel 2023-10-26T01:31:08.684 INFO:teuthology.orchestra.run.smithi169.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2023-10-26T01:31:08.684 INFO:teuthology.orchestra.run.smithi169.stdout: You just mounted a file system that supports labels which does not 2023-10-26T01:31:08.684 INFO:teuthology.orchestra.run.smithi169.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-10-26T01:31:08.685 INFO:teuthology.orchestra.run.smithi169.stdout: applications will generate AVC messages and not be allowed access to 2023-10-26T01:31:08.685 INFO:teuthology.orchestra.run.smithi169.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-10-26T01:31:08.689 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-26T01:31:08.689 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:31:08.689 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-26T01:31:08.783 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:31:08.783 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-26T01:31:08.858 INFO:teuthology.orchestra.run.smithi169.stdout:client4508 2023-10-26T01:31:08.859 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:31:08.859 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:31:08.914 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:31:08.915 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-26T01:31:08.986 INFO:teuthology.orchestra.run.smithi169.stdout:client4508 2023-10-26T01:31:08.986 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:31:08.987 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-26T01:31:09.066 INFO:teuthology.orchestra.run.smithi169.stdout:client4508 2023-10-26T01:31:09.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:31:09.067 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/30203b28-ed8a-44e8-a9a6-a9bf25f980f1.client4508/status') 2023-10-26T01:31:09.181 INFO:teuthology.orchestra.run.smithi169.stdout:instance: client.4508 (3)172.21.15.169:0/120249496 2023-10-26T01:31:09.181 INFO:teuthology.orchestra.run.smithi169.stdout:blocklisted: false 2023-10-26T01:31:09.182 INFO:teuthology.orchestra.run.smithi169.stderr:0+1 records in 2023-10-26T01:31:09.182 INFO:teuthology.orchestra.run.smithi169.stderr:0+1 records out 2023-10-26T01:31:09.182 INFO:teuthology.orchestra.run.smithi169.stderr:70 bytes copied, 8.0622e-05 s, 868 kB/s 2023-10-26T01:31:09.185 INFO:tasks.kclient:client.2 config is {'syntax': 'v1'} 2023-10-26T01:31:09.186 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-10-26T01:31:09.186 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-10-26T01:31:09.186 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-26T01:31:09.410 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:09.408+0000 7f6714290640 1 -- 172.21.15.169:0/3446227996 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f670c0c9d00 msgr2=0x7f670c0cdc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:09.410 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:09.408+0000 7f6714290640 1 --2- 172.21.15.169:0/3446227996 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f670c0c9d00 0x7f670c0cdc50 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f670c09b5a0 tx=0x7f66fc02bd90 comp rx=0 tx=0).stop 2023-10-26T01:31:09.410 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:09.408+0000 7f6714290640 1 -- 172.21.15.169:0/3446227996 shutdown_connections 2023-10-26T01:31:09.411 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:09.408+0000 7f6714290640 1 --2- 172.21.15.169:0/3446227996 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f670c0ac320 0x7f670c0ac740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:09.411 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:09.408+0000 7f6714290640 1 --2- 172.21.15.169:0/3446227996 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f670c0c9d00 0x7f670c0cdc50 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:09.411 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:09.408+0000 7f6714290640 1 --2- 172.21.15.169:0/3446227996 >> v2:172.21.15.170:3300/0 conn(0x7f670c15b620 0x7f670c0cd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:09.411 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:09.408+0000 7f6714290640 1 -- 172.21.15.169:0/3446227996 >> 172.21.15.169:0/3446227996 conn(0x7f670c069930 msgr2=0x7f670c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:09.411 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:09.408+0000 7f6714290640 1 -- 172.21.15.169:0/3446227996 shutdown_connections 2023-10-26T01:31:09.412 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:09.408+0000 7f6714290640 1 -- 172.21.15.169:0/3446227996 wait complete. 2023-10-26T01:31:09.414 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:09.410+0000 7f6714290640 1 Processor -- start 2023-10-26T01:31:09.414 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:09.410+0000 7f6714290640 1 -- start start 2023-10-26T01:31:09.414 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:09.411+0000 7f6714290640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f670c0ac320 0x7f670c146560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:09.414 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:09.411+0000 7f6714290640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f670c0c9d00 0x7f670c0c7310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:09.414 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:09.411+0000 7f6714290640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f670c15b620 0x7f670c0aa250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:09.415 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:09.411+0000 7f6714290640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f670c0cabb0 con 0x7f670c15b620 2023-10-26T01:31:09.415 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:09.411+0000 7f6714290640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f670c0c7e20 con 0x7f670c0c9d00 2023-10-26T01:31:09.415 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:09.411+0000 7f6714290640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f670c1f3150 con 0x7f670c0ac320 2023-10-26T01:31:09.415 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:09.411+0000 7f671282d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f670c15b620 0x7f670c0aa250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:09.415 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:09.411+0000 7f671282d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f670c15b620 0x7f670c0aa250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:43376/0 (socket says 172.21.15.169:43376) 2023-10-26T01:31:09.416 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:09.411+0000 7f671282d640 1 -- 172.21.15.169:0/1556480547 learned_addr learned my addr 172.21.15.169:0/1556480547 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:31:09.416 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:09.411+0000 7f671202c640 1 --2- 172.21.15.169:0/1556480547 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f670c0ac320 0x7f670c146560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:09.416 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:09.411+0000 7f671282d640 1 -- 172.21.15.169:0/1556480547 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f670c0ac320 msgr2=0x7f670c146560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:09.416 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:09.411+0000 7f671182b640 1 --2- 172.21.15.169:0/1556480547 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f670c0c9d00 0x7f670c0c7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:09.417 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:09.411+0000 7f671282d640 1 --2- 172.21.15.169:0/1556480547 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f670c0ac320 0x7f670c146560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:09.417 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:09.411+0000 7f671282d640 1 -- 172.21.15.169:0/1556480547 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f670c0c9d00 msgr2=0x7f670c0c7310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:09.417 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:09.411+0000 7f671282d640 1 --2- 172.21.15.169:0/1556480547 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f670c0c9d00 0x7f670c0c7310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:09.417 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:09.411+0000 7f671282d640 1 -- 172.21.15.169:0/1556480547 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66fc009560 con 0x7f670c15b620 2023-10-26T01:31:09.417 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:09.412+0000 7f671282d640 1 --2- 172.21.15.169:0/1556480547 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f670c15b620 0x7f670c0aa250 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f670400e980 tx=0x7f670400ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:09.418 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:09.412+0000 7f6702ffd640 1 -- 172.21.15.169:0/1556480547 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6704004d20 con 0x7f670c15b620 2023-10-26T01:31:09.418 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:09.412+0000 7f6702ffd640 1 -- 172.21.15.169:0/1556480547 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f670400cb20 con 0x7f670c15b620 2023-10-26T01:31:09.418 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:09.412+0000 7f6702ffd640 1 -- 172.21.15.169:0/1556480547 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6704005020 con 0x7f670c15b620 2023-10-26T01:31:09.418 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:09.412+0000 7f6714290640 1 -- 172.21.15.169:0/1556480547 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f670c0ca210 con 0x7f670c15b620 2023-10-26T01:31:09.418 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:09.412+0000 7f6714290640 1 -- 172.21.15.169:0/1556480547 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f670c0ca7d0 con 0x7f670c15b620 2023-10-26T01:31:09.419 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:09.413+0000 7f6714290640 1 -- 172.21.15.169:0/1556480547 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f670c0cacf0 con 0x7f670c15b620 2023-10-26T01:31:09.420 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:09.414+0000 7f6702ffd640 1 -- 172.21.15.169:0/1556480547 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f6704005180 con 0x7f670c15b620 2023-10-26T01:31:09.420 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:09.415+0000 7f6702ffd640 1 --2- 172.21.15.169:0/1556480547 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f66f006ae40 0x7f66f006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:09.420 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:09.415+0000 7f6702ffd640 1 -- 172.21.15.169:0/1556480547 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6704089f70 con 0x7f670c15b620 2023-10-26T01:31:09.420 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:09.416+0000 7f671202c640 1 --2- 172.21.15.169:0/1556480547 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f66f006ae40 0x7f66f006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:09.421 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:09.417+0000 7f671202c640 1 --2- 172.21.15.169:0/1556480547 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f66f006ae40 0x7f66f006d300 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f67080058c0 tx=0x7f6708005850 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:09.421 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:09.420+0000 7f6702ffd640 1 -- 172.21.15.169:0/1556480547 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f670c0cacf0 con 0x7f670c15b620 2023-10-26T01:31:09.648 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:09.647+0000 7f6714290640 1 -- 172.21.15.169:0/1556480547 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f670c0cacf0 con 0x7f670c15b620 2023-10-26T01:31:09.649 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:09.647+0000 7f6702ffd640 1 -- 172.21.15.169:0/1556480547 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+73 (secure 0 0 0) 0x7f670c0cacf0 con 0x7f670c15b620 2023-10-26T01:31:09.649 INFO:teuthology.orchestra.run.smithi169.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-26T01:31:09.651 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:09.650+0000 7f6714290640 1 -- 172.21.15.169:0/1556480547 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f66f006ae40 msgr2=0x7f66f006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:09.652 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:09.650+0000 7f6714290640 1 --2- 172.21.15.169:0/1556480547 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f66f006ae40 0x7f66f006d300 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f67080058c0 tx=0x7f6708005850 comp rx=0 tx=0).stop 2023-10-26T01:31:09.652 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:09.650+0000 7f6714290640 1 -- 172.21.15.169:0/1556480547 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f670c15b620 msgr2=0x7f670c0aa250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:09.652 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:09.650+0000 7f6714290640 1 --2- 172.21.15.169:0/1556480547 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f670c15b620 0x7f670c0aa250 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f670400e980 tx=0x7f670400ee40 comp rx=0 tx=0).stop 2023-10-26T01:31:09.652 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:09.650+0000 7f6714290640 1 -- 172.21.15.169:0/1556480547 shutdown_connections 2023-10-26T01:31:09.652 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:09.650+0000 7f6714290640 1 --2- 172.21.15.169:0/1556480547 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f66f006ae40 0x7f66f006d300 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:09.653 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:09.650+0000 7f6714290640 1 --2- 172.21.15.169:0/1556480547 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f670c15b620 0x7f670c0aa250 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:09.653 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:09.650+0000 7f6714290640 1 --2- 172.21.15.169:0/1556480547 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f670c0c9d00 0x7f670c0c7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:09.653 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:09.650+0000 7f6714290640 1 --2- 172.21.15.169:0/1556480547 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f670c0ac320 0x7f670c146560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:09.653 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:09.650+0000 7f6714290640 1 -- 172.21.15.169:0/1556480547 >> 172.21.15.169:0/1556480547 conn(0x7f670c069930 msgr2=0x7f670c0cd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:09.654 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:09.650+0000 7f6714290640 1 -- 172.21.15.169:0/1556480547 shutdown_connections 2023-10-26T01:31:09.654 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:09.650+0000 7f6714290640 1 -- 172.21.15.169:0/1556480547 wait complete. 2023-10-26T01:31:09.670 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-26T01:31:09.670 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-26T01:31:09.671 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi169.front.sepia.ceph.com 2023-10-26T01:31:09.671 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-10-26T01:31:09.671 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-10-26T01:31:09.671 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-26T01:31:09.672 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-26T01:31:09.672 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-26T01:31:09.672 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-10-26T01:31:09.672 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:31:09.673 DEBUG:teuthology.orchestra.run.smithi169:> ip addr 2023-10-26T01:31:09.724 INFO:teuthology.orchestra.run.smithi169.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-26T01:31:09.724 INFO:teuthology.orchestra.run.smithi169.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-26T01:31:09.724 INFO:teuthology.orchestra.run.smithi169.stdout: inet 127.0.0.1/8 scope host lo 2023-10-26T01:31:09.725 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:31:09.725 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 ::1/128 scope host 2023-10-26T01:31:09.725 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:31:09.725 INFO:teuthology.orchestra.run.smithi169.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-26T01:31:09.725 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:d9:3b:b4 brd ff:ff:ff:ff:ff:ff 2023-10-26T01:31:09.726 INFO:teuthology.orchestra.run.smithi169.stdout: altname enp7s0f0 2023-10-26T01:31:09.726 INFO:teuthology.orchestra.run.smithi169.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-26T01:31:09.726 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:d9:3b:b5 brd ff:ff:ff:ff:ff:ff 2023-10-26T01:31:09.726 INFO:teuthology.orchestra.run.smithi169.stdout: altname enp7s0f1 2023-10-26T01:31:09.726 INFO:teuthology.orchestra.run.smithi169.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-26T01:31:09.727 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:88:7f:4c brd ff:ff:ff:ff:ff:ff 2023-10-26T01:31:09.727 INFO:teuthology.orchestra.run.smithi169.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-26T01:31:09.727 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:88:7f:4d brd ff:ff:ff:ff:ff:ff 2023-10-26T01:31:09.727 INFO:teuthology.orchestra.run.smithi169.stdout: inet 172.21.15.169/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-26T01:31:09.727 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft 42426sec preferred_lft 42426sec 2023-10-26T01:31:09.728 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 fe80::ec4:7aff:fe88:7f4d/64 scope link 2023-10-26T01:31:09.728 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:31:09.728 INFO:teuthology.orchestra.run.smithi169.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-26T01:31:09.728 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 2a:80:f8:45:5f:9f brd ff:ff:ff:ff:ff:ff 2023-10-26T01:31:09.728 INFO:teuthology.orchestra.run.smithi169.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-26T01:31:09.728 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:31:09.729 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 fe80::d815:31ff:fe87:d148/64 scope link 2023-10-26T01:31:09.729 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:31:09.729 INFO:teuthology.orchestra.run.smithi169.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-26T01:31:09.729 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 2a:80:f8:45:5f:9f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-26T01:31:09.729 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 fe80::2880:f8ff:fe45:5f9f/64 scope link 2023-10-26T01:31:09.730 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:31:09.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:31:09.731 DEBUG:teuthology.orchestra.run.smithi169:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-26T01:31:09.825 INFO:teuthology.orchestra.run.smithi169.stdout:1 2023-10-26T01:31:09.829 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:31:09.829 DEBUG:teuthology.orchestra.run.smithi169:> route 2023-10-26T01:31:09.883 INFO:teuthology.orchestra.run.smithi169.stdout:Kernel IP routing table 2023-10-26T01:31:09.884 INFO:teuthology.orchestra.run.smithi169.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-26T01:31:09.884 INFO:teuthology.orchestra.run.smithi169.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-26T01:31:09.884 INFO:teuthology.orchestra.run.smithi169.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-26T01:31:09.884 INFO:teuthology.orchestra.run.smithi169.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-26T01:31:09.885 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:31:09.886 DEBUG:teuthology.orchestra.run.smithi169:> (cd / && exec bash -c ' 2023-10-26T01:31:09.886 DEBUG:teuthology.orchestra.run.smithi169:> set -e 2023-10-26T01:31:09.886 DEBUG:teuthology.orchestra.run.smithi169:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-26T01:31:09.887 DEBUG:teuthology.orchestra.run.smithi169:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-26T01:31:09.887 DEBUG:teuthology.orchestra.run.smithi169:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-26T01:31:09.887 DEBUG:teuthology.orchestra.run.smithi169:> ') 2023-10-26T01:31:10.074 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:31:10.075 DEBUG:teuthology.orchestra.run.smithi169:> ip netns list 2023-10-26T01:31:10.125 INFO:teuthology.orchestra.run.smithi169.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-26T01:31:10.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:31:10.127 DEBUG:teuthology.orchestra.run.smithi169:> ip netns list-id 2023-10-26T01:31:10.182 INFO:teuthology.orchestra.run.smithi169.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-10-26T01:31:10.183 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:31:10.184 DEBUG:teuthology.orchestra.run.smithi169:> (cd / && exec bash -c ' 2023-10-26T01:31:10.184 DEBUG:teuthology.orchestra.run.smithi169:> set -e 2023-10-26T01:31:10.184 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-26T01:31:10.185 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-10-26T01:31:10.185 DEBUG:teuthology.orchestra.run.smithi169:> ') 2023-10-26T01:31:10.322 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:31:10.323 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-10-26T01:31:10.410 INFO:teuthology.orchestra.run.smithi169.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-26T01:31:10.410 INFO:teuthology.orchestra.run.smithi169.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-26T01:31:10.410 INFO:teuthology.orchestra.run.smithi169.stdout: inet 127.0.0.1/8 scope host lo 2023-10-26T01:31:10.411 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:31:10.411 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 ::1/128 scope host 2023-10-26T01:31:10.411 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:31:10.411 INFO:teuthology.orchestra.run.smithi169.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-26T01:31:10.412 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether c6:f8:a3:60:37:1f brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-26T01:31:10.412 INFO:teuthology.orchestra.run.smithi169.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-26T01:31:10.412 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:31:10.412 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 fe80::c4f8:a3ff:fe60:371f/64 scope link 2023-10-26T01:31:10.413 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:31:10.414 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:31:10.414 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-10-26T01:31:10.510 INFO:teuthology.orchestra.run.smithi169.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-26T01:31:10.511 INFO:teuthology.orchestra.run.smithi169.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-26T01:31:10.511 INFO:teuthology.orchestra.run.smithi169.stdout: inet 127.0.0.1/8 scope host lo 2023-10-26T01:31:10.511 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:31:10.511 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 ::1/128 scope host 2023-10-26T01:31:10.512 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:31:10.512 INFO:teuthology.orchestra.run.smithi169.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-26T01:31:10.512 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether c6:f8:a3:60:37:1f brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-26T01:31:10.512 INFO:teuthology.orchestra.run.smithi169.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-26T01:31:10.513 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:31:10.513 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 fe80::c4f8:a3ff:fe60:371f/64 scope link 2023-10-26T01:31:10.513 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:31:10.514 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-10-26T01:31:10.515 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:31:10.515 DEBUG:teuthology.orchestra.run.smithi169:> (cd / && exec bash -c ' 2023-10-26T01:31:10.515 DEBUG:teuthology.orchestra.run.smithi169:> set -e 2023-10-26T01:31:10.516 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-10-26T01:31:10.516 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-10-26T01:31:10.516 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-10-26T01:31:10.517 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-10-26T01:31:10.517 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-10-26T01:31:10.517 DEBUG:teuthology.orchestra.run.smithi169:> ') 2023-10-26T01:31:10.786 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:31:10.787 DEBUG:teuthology.orchestra.run.smithi169:> (cd / && exec bash -c ' 2023-10-26T01:31:10.787 DEBUG:teuthology.orchestra.run.smithi169:> set -e 2023-10-26T01:31:10.787 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link set brx.1 up 2023-10-26T01:31:10.788 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link set dev brx.1 master ceph-brx 2023-10-26T01:31:10.788 DEBUG:teuthology.orchestra.run.smithi169:> ') 2023-10-26T01:31:10.955 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-26T01:31:10.956 DEBUG:teuthology.orchestra.run.smithi169:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2023-10-26T01:31:11.002 INFO:teuthology.orchestra.run.smithi169.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2023-10-26T01:31:11.003 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-26T01:31:11.003 DEBUG:teuthology.orchestra.run.smithi169:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2023-10-26T01:31:11.097 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-26T01:31:11.097 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:31:11.097 DEBUG:teuthology.orchestra.run.smithi169:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2023-10-26T01:31:11.230 INFO:teuthology.orchestra.run.smithi169.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2023-10-26T01:31:11.231 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2023-10-26T01:31:11.231 INFO:teuthology.orchestra.run.smithi169.stdout:invalid new device string format 2023-10-26T01:31:11.252 INFO:teuthology.orchestra.run.smithi169.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2023-10-26T01:31:11.252 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2023-10-26T01:31:11.252 INFO:teuthology.orchestra.run.smithi169.stdout:invalid new device string format 2023-10-26T01:31:11.253 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: resolved to: "172.21.15.169:3300,172.21.15.170:3300,172.21.15.170:3301" 2023-10-26T01:31:11.253 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-26T01:31:11.280 INFO:teuthology.orchestra.run.smithi169.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2023-10-26T01:31:11.281 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2023-10-26T01:31:11.281 INFO:teuthology.orchestra.run.smithi169.stdout:invalid new device string format 2023-10-26T01:31:11.281 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: resolved to: "172.21.15.169:3300,172.21.15.170:3300,172.21.15.170:3301" 2023-10-26T01:31:11.282 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-26T01:31:11.282 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: resolved to: "172.21.15.169:6789,172.21.15.170:6789,172.21.15.170:6790" 2023-10-26T01:31:11.283 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.169:6789,172.21.15.170:6789,172.21.15.170:6790:/ 2023-10-26T01:31:11.283 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs,key=2,fsid=30203b28-ed8a-44e8-a9a6-a9bf25f980f1" will pass to kernel 2023-10-26T01:31:11.285 INFO:teuthology.orchestra.run.smithi169.stdout:mount: /home/ubuntu/cephtest/mnt.2 does not contain SELinux labels. 2023-10-26T01:31:11.286 INFO:teuthology.orchestra.run.smithi169.stdout: You just mounted a file system that supports labels which does not 2023-10-26T01:31:11.286 INFO:teuthology.orchestra.run.smithi169.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-10-26T01:31:11.286 INFO:teuthology.orchestra.run.smithi169.stdout: applications will generate AVC messages and not be allowed access to 2023-10-26T01:31:11.287 INFO:teuthology.orchestra.run.smithi169.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-10-26T01:31:11.291 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-26T01:31:11.291 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:31:11.292 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-10-26T01:31:11.378 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:31:11.378 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-26T01:31:11.434 INFO:teuthology.orchestra.run.smithi169.stdout:client4779 2023-10-26T01:31:11.434 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:31:11.435 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:31:11.484 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:31:11.484 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-26T01:31:11.584 INFO:teuthology.orchestra.run.smithi169.stdout:client4779 2023-10-26T01:31:11.584 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:31:11.585 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-26T01:31:11.640 INFO:teuthology.orchestra.run.smithi169.stdout:client4779 2023-10-26T01:31:11.640 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:31:11.641 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/30203b28-ed8a-44e8-a9a6-a9bf25f980f1.client4779/status') 2023-10-26T01:31:11.735 INFO:teuthology.orchestra.run.smithi169.stdout:instance: client.4779 (3)192.168.0.2:0/4194112238 2023-10-26T01:31:11.735 INFO:teuthology.orchestra.run.smithi169.stdout:blocklisted: false 2023-10-26T01:31:11.736 INFO:teuthology.orchestra.run.smithi169.stderr:0+1 records in 2023-10-26T01:31:11.736 INFO:teuthology.orchestra.run.smithi169.stderr:0+1 records out 2023-10-26T01:31:11.736 INFO:teuthology.orchestra.run.smithi169.stderr:69 bytes copied, 7.3839e-05 s, 934 kB/s 2023-10-26T01:31:11.739 INFO:tasks.kclient:client.1 config is {'syntax': 'v1'} 2023-10-26T01:31:11.739 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-10-26T01:31:11.739 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-26T01:31:11.740 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-26T01:31:11.957 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:11.955+0000 7f8159464640 1 -- 172.21.15.169:0/4054324186 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8154159540 msgr2=0x7f81540cc770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:11.958 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:11.955+0000 7f8159464640 1 --2- 172.21.15.169:0/4054324186 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8154159540 0x7f81540cc770 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f81540ce1d0 tx=0x7f814402bdb0 comp rx=0 tx=0).stop 2023-10-26T01:31:11.958 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:11.955+0000 7f8159464640 1 -- 172.21.15.169:0/4054324186 shutdown_connections 2023-10-26T01:31:11.958 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:11.955+0000 7f8159464640 1 --2- 172.21.15.169:0/4054324186 >> v2:172.21.15.170:3300/0 conn(0x7f815406c090 0x7f81540b01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:11.958 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:11.955+0000 7f8159464640 1 --2- 172.21.15.169:0/4054324186 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8154159540 0x7f81540cc770 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:11.958 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:11.955+0000 7f8159464640 1 -- 172.21.15.169:0/4054324186 >> 172.21.15.169:0/4054324186 conn(0x7f81540aeee0 msgr2=0x7f81540af2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:11.959 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:11.955+0000 7f8159464640 1 -- 172.21.15.169:0/4054324186 shutdown_connections 2023-10-26T01:31:11.959 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:11.955+0000 7f8159464640 1 -- 172.21.15.169:0/4054324186 wait complete. 2023-10-26T01:31:11.960 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:11.957+0000 7f8159464640 1 Processor -- start 2023-10-26T01:31:11.961 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:11.957+0000 7f8159464640 1 -- start start 2023-10-26T01:31:11.961 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:11.958+0000 7f8159464640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f815406c090 0x7f815406b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:11.961 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:11.958+0000 7f8159464640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81540b0720 0x7f8154146b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:11.961 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:11.958+0000 7f8159464640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f8154159540 0x7f81540b0f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:11.961 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:11.958+0000 7f8159464640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8154147070 con 0x7f81540b0720 2023-10-26T01:31:11.962 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:11.958+0000 7f8159464640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8154147580 con 0x7f815406c090 2023-10-26T01:31:11.962 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:11.958+0000 7f8159464640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8154143860 con 0x7f8154159540 2023-10-26T01:31:11.962 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:11.959+0000 7f81527fc640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81540b0720 0x7f8154146b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:11.962 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:11.959+0000 7f81527fc640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81540b0720 0x7f8154146b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:43396/0 (socket says 172.21.15.169:43396) 2023-10-26T01:31:11.963 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:11.959+0000 7f81527fc640 1 -- 172.21.15.169:0/1098579597 learned_addr learned my addr 172.21.15.169:0/1098579597 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:31:11.963 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:11.959+0000 7f8152ffd640 1 --2- 172.21.15.169:0/1098579597 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f815406c090 0x7f815406b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:11.963 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:11.959+0000 7f81537fe640 1 --2- 172.21.15.169:0/1098579597 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f8154159540 0x7f81540b0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:11.963 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:11.959+0000 7f81527fc640 1 -- 172.21.15.169:0/1098579597 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f8154159540 msgr2=0x7f81540b0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:11.964 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:11.959+0000 7f81527fc640 1 --2- 172.21.15.169:0/1098579597 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f8154159540 0x7f81540b0f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:11.964 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:11.959+0000 7f81527fc640 1 -- 172.21.15.169:0/1098579597 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f815406c090 msgr2=0x7f815406b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:11.964 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:11.959+0000 7f81527fc640 1 --2- 172.21.15.169:0/1098579597 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f815406c090 0x7f815406b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:11.964 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:11.959+0000 7f81527fc640 1 -- 172.21.15.169:0/1098579597 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8144009560 con 0x7f81540b0720 2023-10-26T01:31:11.966 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:11.959+0000 7f81527fc640 1 --2- 172.21.15.169:0/1098579597 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81540b0720 0x7f8154146b30 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f814800d770 tx=0x7f814800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:11.966 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:11.960+0000 7f813b7fe640 1 -- 172.21.15.169:0/1098579597 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8148009bd0 con 0x7f81540b0720 2023-10-26T01:31:11.966 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:11.960+0000 7f813b7fe640 1 -- 172.21.15.169:0/1098579597 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8148009e20 con 0x7f81540b0720 2023-10-26T01:31:11.966 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:11.960+0000 7f813b7fe640 1 -- 172.21.15.169:0/1098579597 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8148005280 con 0x7f81540b0720 2023-10-26T01:31:11.966 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:11.960+0000 7f8159464640 1 -- 172.21.15.169:0/1098579597 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81541472a0 con 0x7f81540b0720 2023-10-26T01:31:11.967 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:11.960+0000 7f8159464640 1 -- 172.21.15.169:0/1098579597 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81541434f0 con 0x7f81540b0720 2023-10-26T01:31:11.967 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:11.961+0000 7f8159464640 1 -- 172.21.15.169:0/1098579597 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8154143070 con 0x7f81540b0720 2023-10-26T01:31:11.967 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:11.962+0000 7f813b7fe640 1 -- 172.21.15.169:0/1098579597 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f81480053e0 con 0x7f81540b0720 2023-10-26T01:31:11.967 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:11.963+0000 7f813b7fe640 1 --2- 172.21.15.169:0/1098579597 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f814006ae40 0x7f814006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:11.968 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:11.963+0000 7f813b7fe640 1 -- 172.21.15.169:0/1098579597 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8148089820 con 0x7f81540b0720 2023-10-26T01:31:11.968 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:11.963+0000 7f8152ffd640 1 --2- 172.21.15.169:0/1098579597 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f814006ae40 0x7f814006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:11.968 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:11.964+0000 7f8152ffd640 1 --2- 172.21.15.169:0/1098579597 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f814006ae40 0x7f814006d300 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8144035000 tx=0x7f8144004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:11.968 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:11.967+0000 7f813b7fe640 1 -- 172.21.15.169:0/1098579597 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8154143070 con 0x7f81540b0720 2023-10-26T01:31:12.201 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:12.199+0000 7f8159464640 1 -- 172.21.15.169:0/1098579597 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f8154143070 con 0x7f81540b0720 2023-10-26T01:31:12.201 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:12.200+0000 7f813b7fe640 1 -- 172.21.15.169:0/1098579597 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+73 (secure 0 0 0) 0x7f8154143070 con 0x7f81540b0720 2023-10-26T01:31:12.202 INFO:teuthology.orchestra.run.smithi169.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-26T01:31:12.205 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:12.203+0000 7f8159464640 1 -- 172.21.15.169:0/1098579597 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f814006ae40 msgr2=0x7f814006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:12.205 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:12.203+0000 7f8159464640 1 --2- 172.21.15.169:0/1098579597 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f814006ae40 0x7f814006d300 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8144035000 tx=0x7f8144004730 comp rx=0 tx=0).stop 2023-10-26T01:31:12.205 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:12.203+0000 7f8159464640 1 -- 172.21.15.169:0/1098579597 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81540b0720 msgr2=0x7f8154146b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:12.206 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:12.203+0000 7f8159464640 1 --2- 172.21.15.169:0/1098579597 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81540b0720 0x7f8154146b30 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f814800d770 tx=0x7f814800dc30 comp rx=0 tx=0).stop 2023-10-26T01:31:12.206 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:12.203+0000 7f8159464640 1 -- 172.21.15.169:0/1098579597 shutdown_connections 2023-10-26T01:31:12.206 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:12.203+0000 7f8159464640 1 --2- 172.21.15.169:0/1098579597 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f814006ae40 0x7f814006d300 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:12.206 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:12.203+0000 7f8159464640 1 --2- 172.21.15.169:0/1098579597 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81540b0720 0x7f8154146b30 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:12.206 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:12.203+0000 7f8159464640 1 --2- 172.21.15.169:0/1098579597 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f8154159540 0x7f81540b0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:12.207 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:12.203+0000 7f8159464640 1 --2- 172.21.15.169:0/1098579597 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f815406c090 0x7f815406b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:12.207 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:12.203+0000 7f8159464640 1 -- 172.21.15.169:0/1098579597 >> 172.21.15.169:0/1098579597 conn(0x7f81540aeee0 msgr2=0x7f815406c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:12.207 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:12.203+0000 7f8159464640 1 -- 172.21.15.169:0/1098579597 shutdown_connections 2023-10-26T01:31:12.207 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:12.203+0000 7f8159464640 1 -- 172.21.15.169:0/1098579597 wait complete. 2023-10-26T01:31:12.223 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-26T01:31:12.224 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-26T01:31:12.224 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi170.front.sepia.ceph.com 2023-10-26T01:31:12.224 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-26T01:31:12.225 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-26T01:31:12.225 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-26T01:31:12.225 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-26T01:31:12.225 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-26T01:31:12.225 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-26T01:31:12.226 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:31:12.226 DEBUG:teuthology.orchestra.run.smithi170:> ip addr 2023-10-26T01:31:12.287 INFO:teuthology.orchestra.run.smithi170.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-26T01:31:12.288 INFO:teuthology.orchestra.run.smithi170.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-26T01:31:12.288 INFO:teuthology.orchestra.run.smithi170.stdout: inet 127.0.0.1/8 scope host lo 2023-10-26T01:31:12.288 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:31:12.288 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 ::1/128 scope host 2023-10-26T01:31:12.289 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:31:12.289 INFO:teuthology.orchestra.run.smithi170.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-26T01:31:12.289 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:d9:3b:54 brd ff:ff:ff:ff:ff:ff 2023-10-26T01:31:12.289 INFO:teuthology.orchestra.run.smithi170.stdout: altname enp7s0f0 2023-10-26T01:31:12.289 INFO:teuthology.orchestra.run.smithi170.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-26T01:31:12.289 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:d9:3b:55 brd ff:ff:ff:ff:ff:ff 2023-10-26T01:31:12.290 INFO:teuthology.orchestra.run.smithi170.stdout: altname enp7s0f1 2023-10-26T01:31:12.290 INFO:teuthology.orchestra.run.smithi170.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-26T01:31:12.290 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:88:72:54 brd ff:ff:ff:ff:ff:ff 2023-10-26T01:31:12.290 INFO:teuthology.orchestra.run.smithi170.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-26T01:31:12.290 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:88:72:55 brd ff:ff:ff:ff:ff:ff 2023-10-26T01:31:12.291 INFO:teuthology.orchestra.run.smithi170.stdout: inet 172.21.15.170/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-26T01:31:12.291 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft 42408sec preferred_lft 42408sec 2023-10-26T01:31:12.291 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::ec4:7aff:fe88:7255/64 scope link 2023-10-26T01:31:12.291 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:31:12.292 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-26T01:31:12.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:31:12.293 DEBUG:teuthology.orchestra.run.smithi170:> (cd / && exec bash -c ' 2023-10-26T01:31:12.293 DEBUG:teuthology.orchestra.run.smithi170:> set -e 2023-10-26T01:31:12.293 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip link add name ceph-brx type bridge 2023-10-26T01:31:12.294 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip addr flush dev ceph-brx 2023-10-26T01:31:12.294 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip link set ceph-brx up 2023-10-26T01:31:12.294 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-26T01:31:12.295 DEBUG:teuthology.orchestra.run.smithi170:> ') 2023-10-26T01:31:12.778 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:31:12.778 DEBUG:teuthology.orchestra.run.smithi170:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-26T01:31:12.857 INFO:teuthology.orchestra.run.smithi170.stdout:1 2023-10-26T01:31:12.861 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:31:12.861 DEBUG:teuthology.orchestra.run.smithi170:> route 2023-10-26T01:31:12.918 INFO:teuthology.orchestra.run.smithi170.stdout:Kernel IP routing table 2023-10-26T01:31:12.919 INFO:teuthology.orchestra.run.smithi170.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-26T01:31:12.919 INFO:teuthology.orchestra.run.smithi170.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-26T01:31:12.919 INFO:teuthology.orchestra.run.smithi170.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-26T01:31:12.919 INFO:teuthology.orchestra.run.smithi170.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-26T01:31:12.921 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:31:12.921 DEBUG:teuthology.orchestra.run.smithi170:> (cd / && exec bash -c ' 2023-10-26T01:31:12.921 DEBUG:teuthology.orchestra.run.smithi170:> set -e 2023-10-26T01:31:12.922 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-26T01:31:12.922 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-26T01:31:12.922 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-26T01:31:12.922 DEBUG:teuthology.orchestra.run.smithi170:> ') 2023-10-26T01:31:13.715 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:31:13.716 DEBUG:teuthology.orchestra.run.smithi170:> ip netns list 2023-10-26T01:31:13.777 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:31:13.777 DEBUG:teuthology.orchestra.run.smithi170:> ip netns list-id 2023-10-26T01:31:13.832 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:31:13.832 DEBUG:teuthology.orchestra.run.smithi170:> (cd / && exec bash -c ' 2023-10-26T01:31:13.833 DEBUG:teuthology.orchestra.run.smithi170:> set -e 2023-10-26T01:31:13.833 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-26T01:31:13.833 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-10-26T01:31:13.833 DEBUG:teuthology.orchestra.run.smithi170:> ') 2023-10-26T01:31:13.970 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-10-26T01:31:13.970 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:31:13.970 DEBUG:teuthology.orchestra.run.smithi170:> (cd / && exec bash -c ' 2023-10-26T01:31:13.971 DEBUG:teuthology.orchestra.run.smithi170:> set -e 2023-10-26T01:31:13.971 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-10-26T01:31:13.971 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-26T01:31:13.972 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-10-26T01:31:13.972 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-10-26T01:31:13.972 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-10-26T01:31:13.973 DEBUG:teuthology.orchestra.run.smithi170:> ') 2023-10-26T01:31:14.262 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:31:14.263 DEBUG:teuthology.orchestra.run.smithi170:> (cd / && exec bash -c ' 2023-10-26T01:31:14.263 DEBUG:teuthology.orchestra.run.smithi170:> set -e 2023-10-26T01:31:14.263 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip link set brx.0 up 2023-10-26T01:31:14.264 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip link set dev brx.0 master ceph-brx 2023-10-26T01:31:14.264 DEBUG:teuthology.orchestra.run.smithi170:> ') 2023-10-26T01:31:14.420 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-26T01:31:14.421 DEBUG:teuthology.orchestra.run.smithi170:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-26T01:31:14.469 INFO:teuthology.orchestra.run.smithi170.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-26T01:31:14.470 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-26T01:31:14.470 DEBUG:teuthology.orchestra.run.smithi170:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-26T01:31:14.520 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-26T01:31:14.520 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:31:14.521 DEBUG:teuthology.orchestra.run.smithi170:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-26T01:31:14.681 INFO:teuthology.orchestra.run.smithi170.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-26T01:31:14.682 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-26T01:31:14.682 INFO:teuthology.orchestra.run.smithi170.stdout:invalid new device string format 2023-10-26T01:31:15.151 INFO:teuthology.orchestra.run.smithi170.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-26T01:31:15.151 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-26T01:31:15.151 INFO:teuthology.orchestra.run.smithi170.stdout:invalid new device string format 2023-10-26T01:31:15.151 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: resolved to: "172.21.15.169:3300,172.21.15.170:3300,172.21.15.170:3301" 2023-10-26T01:31:15.152 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-26T01:31:15.181 INFO:teuthology.orchestra.run.smithi170.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-26T01:31:15.181 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-26T01:31:15.181 INFO:teuthology.orchestra.run.smithi170.stdout:invalid new device string format 2023-10-26T01:31:15.181 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: resolved to: "172.21.15.169:3300,172.21.15.170:3300,172.21.15.170:3301" 2023-10-26T01:31:15.182 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-26T01:31:15.182 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: resolved to: "172.21.15.169:6789,172.21.15.170:6789,172.21.15.170:6790" 2023-10-26T01:31:15.182 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.169:6789,172.21.15.170:6789,172.21.15.170:6790:/ 2023-10-26T01:31:15.182 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=30203b28-ed8a-44e8-a9a6-a9bf25f980f1" will pass to kernel 2023-10-26T01:31:15.184 INFO:teuthology.orchestra.run.smithi170.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2023-10-26T01:31:15.184 INFO:teuthology.orchestra.run.smithi170.stdout: You just mounted a file system that supports labels which does not 2023-10-26T01:31:15.184 INFO:teuthology.orchestra.run.smithi170.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-10-26T01:31:15.184 INFO:teuthology.orchestra.run.smithi170.stdout: applications will generate AVC messages and not be allowed access to 2023-10-26T01:31:15.184 INFO:teuthology.orchestra.run.smithi170.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-10-26T01:31:15.188 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-26T01:31:15.188 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:31:15.189 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-26T01:31:15.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:31:15.276 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-26T01:31:15.333 INFO:teuthology.orchestra.run.smithi170.stdout:client4788 2023-10-26T01:31:15.334 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-26T01:31:15.334 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:31:15.384 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:31:15.385 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-26T01:31:15.489 INFO:teuthology.orchestra.run.smithi170.stdout:client4788 2023-10-26T01:31:15.490 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:31:15.490 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-26T01:31:15.547 INFO:teuthology.orchestra.run.smithi170.stdout:client4788 2023-10-26T01:31:15.548 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:31:15.548 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/30203b28-ed8a-44e8-a9a6-a9bf25f980f1.client4788/status') 2023-10-26T01:31:15.642 INFO:teuthology.orchestra.run.smithi170.stdout:instance: client.4788 (3)172.21.15.170:0/1262175789 2023-10-26T01:31:15.642 INFO:teuthology.orchestra.run.smithi170.stdout:blocklisted: false 2023-10-26T01:31:15.642 INFO:teuthology.orchestra.run.smithi170.stderr:0+1 records in 2023-10-26T01:31:15.643 INFO:teuthology.orchestra.run.smithi170.stderr:0+1 records out 2023-10-26T01:31:15.643 INFO:teuthology.orchestra.run.smithi170.stderr:71 bytes copied, 6.6596e-05 s, 1.1 MB/s 2023-10-26T01:31:15.646 INFO:tasks.kclient:client.3 config is {'syntax': 'v1'} 2023-10-26T01:31:15.646 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-10-26T01:31:15.646 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-10-26T01:31:15.647 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-26T01:31:15.875 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:15.873+0000 7efca898e640 1 -- 172.21.15.169:0/2145121962 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efc90003fa0 msgr2=0x7efc900047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:15.875 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:15.873+0000 7efca898e640 1 --2- 172.21.15.169:0/2145121962 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efc90003fa0 0x7efc900047c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7efc8c00bb20 tx=0x7efc900057f0 comp rx=0 tx=0).stop 2023-10-26T01:31:15.876 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:15.873+0000 7efca2d9d640 1 -- 172.21.15.169:0/2145121962 reap_dead start 2023-10-26T01:31:15.876 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:15.873+0000 7efca898e640 1 -- 172.21.15.169:0/2145121962 shutdown_connections 2023-10-26T01:31:15.876 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:15.873+0000 7efca898e640 1 -- 172.21.15.169:0/2145121962 >> 172.21.15.169:0/2145121962 conn(0x7efca4069930 msgr2=0x7efca4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:15.877 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:15.873+0000 7efca898e640 1 -- 172.21.15.169:0/2145121962 shutdown_connections 2023-10-26T01:31:15.877 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:15.873+0000 7efca898e640 1 -- 172.21.15.169:0/2145121962 wait complete. 2023-10-26T01:31:15.880 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:15.875+0000 7efca898e640 1 Processor -- start 2023-10-26T01:31:15.881 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:15.875+0000 7efca898e640 1 -- start start 2023-10-26T01:31:15.881 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:15.876+0000 7efca898e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efc90003fa0 0x7efca40c7b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:15.881 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:15.876+0000 7efca898e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efca415d600 0x7efca41f5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:15.882 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:15.876+0000 7efca898e640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7efca415da60 0x7efca41f56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:15.882 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:15.876+0000 7efca898e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efca41f5c00 con 0x7efca415d600 2023-10-26T01:31:15.883 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:15.876+0000 7efca898e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efca41f5e40 con 0x7efc90003fa0 2023-10-26T01:31:15.883 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:15.876+0000 7efca898e640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efca406eeb0 con 0x7efca415da60 2023-10-26T01:31:15.883 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:15.876+0000 7efc99d9b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efca415d600 0x7efca41f5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:15.884 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:15.877+0000 7efc99d9b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efca415d600 0x7efca41f5180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:54418/0 (socket says 172.21.15.169:54418) 2023-10-26T01:31:15.884 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:15.877+0000 7efc99d9b640 1 -- 172.21.15.169:0/4151071471 learned_addr learned my addr 172.21.15.169:0/4151071471 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:31:15.884 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:15.877+0000 7efca259c640 1 --2- 172.21.15.169:0/4151071471 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efc90003fa0 0x7efca40c7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:15.885 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:15.877+0000 7efc99d9b640 1 -- 172.21.15.169:0/4151071471 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7efca415da60 msgr2=0x7efca41f56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:15.885 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:15.877+0000 7efc99d9b640 1 --2- 172.21.15.169:0/4151071471 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7efca415da60 0x7efca41f56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:15.885 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:15.877+0000 7efc99d9b640 1 -- 172.21.15.169:0/4151071471 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efc90003fa0 msgr2=0x7efca40c7b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:15.886 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:15.877+0000 7efc99d9b640 1 --2- 172.21.15.169:0/4151071471 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efc90003fa0 0x7efca40c7b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:15.886 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:15.877+0000 7efc99d9b640 1 -- 172.21.15.169:0/4151071471 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efca406f190 con 0x7efca415d600 2023-10-26T01:31:15.886 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:15.877+0000 7efc99d9b640 1 --2- 172.21.15.169:0/4151071471 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efca415d600 0x7efca41f5180 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7efc8c038040 tx=0x7efc8c0366b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:15.887 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:15.877+0000 7efc9b7fe640 1 -- 172.21.15.169:0/4151071471 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc8c036d70 con 0x7efca415d600 2023-10-26T01:31:15.887 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:15.878+0000 7efc9b7fe640 1 -- 172.21.15.169:0/4151071471 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7efc8c004030 con 0x7efca415d600 2023-10-26T01:31:15.888 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:15.878+0000 7efc9b7fe640 1 -- 172.21.15.169:0/4151071471 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc8c004440 con 0x7efca415d600 2023-10-26T01:31:15.888 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:15.878+0000 7efca898e640 1 -- 172.21.15.169:0/4151071471 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efca406f470 con 0x7efca415d600 2023-10-26T01:31:15.888 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:15.878+0000 7efca898e640 1 -- 172.21.15.169:0/4151071471 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efca4151510 con 0x7efca415d600 2023-10-26T01:31:15.888 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:15.879+0000 7efca898e640 1 -- 172.21.15.169:0/4151071471 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efca404db10 con 0x7efca415d600 2023-10-26T01:31:15.889 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:15.880+0000 7efc9b7fe640 1 -- 172.21.15.169:0/4151071471 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7efc8c0035f0 con 0x7efca415d600 2023-10-26T01:31:15.890 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:15.881+0000 7efc9b7fe640 1 --2- 172.21.15.169:0/4151071471 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7efc8406ae40 0x7efc8406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:15.890 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:15.881+0000 7efc9b7fe640 1 -- 172.21.15.169:0/4151071471 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efc8c0ab510 con 0x7efca415d600 2023-10-26T01:31:15.890 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:15.881+0000 7efca259c640 1 --2- 172.21.15.169:0/4151071471 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7efc8406ae40 0x7efc8406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:15.890 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:15.882+0000 7efca259c640 1 --2- 172.21.15.169:0/4151071471 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7efc8406ae40 0x7efc8406d300 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7efc9400fa60 tx=0x7efc94005f30 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:15.890 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:15.885+0000 7efc9b7fe640 1 -- 172.21.15.169:0/4151071471 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7efc8c076520 con 0x7efca415d600 2023-10-26T01:31:16.113 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:16.112+0000 7efca898e640 1 -- 172.21.15.169:0/4151071471 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7efca40b2170 con 0x7efca415d600 2023-10-26T01:31:16.114 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:16.113+0000 7efc9b7fe640 1 -- 172.21.15.169:0/4151071471 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+73 (secure 0 0 0) 0x7efc8c048060 con 0x7efca415d600 2023-10-26T01:31:16.115 INFO:teuthology.orchestra.run.smithi169.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-26T01:31:16.118 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:16.116+0000 7efca898e640 1 -- 172.21.15.169:0/4151071471 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7efc8406ae40 msgr2=0x7efc8406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:16.119 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:16.116+0000 7efca898e640 1 --2- 172.21.15.169:0/4151071471 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7efc8406ae40 0x7efc8406d300 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7efc9400fa60 tx=0x7efc94005f30 comp rx=0 tx=0).stop 2023-10-26T01:31:16.119 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:16.116+0000 7efca898e640 1 -- 172.21.15.169:0/4151071471 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efca415d600 msgr2=0x7efca41f5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:16.119 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:16.116+0000 7efca898e640 1 --2- 172.21.15.169:0/4151071471 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efca415d600 0x7efca41f5180 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7efc8c038040 tx=0x7efc8c0366b0 comp rx=0 tx=0).stop 2023-10-26T01:31:16.119 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:16.116+0000 7efca898e640 1 -- 172.21.15.169:0/4151071471 shutdown_connections 2023-10-26T01:31:16.119 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:16.116+0000 7efca898e640 1 --2- 172.21.15.169:0/4151071471 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7efc8406ae40 0x7efc8406d300 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:16.120 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:16.116+0000 7efca898e640 1 --2- 172.21.15.169:0/4151071471 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efca415d600 0x7efca41f5180 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:16.120 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:16.116+0000 7efca898e640 1 --2- 172.21.15.169:0/4151071471 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7efca415da60 0x7efca41f56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:16.120 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:16.116+0000 7efca898e640 1 --2- 172.21.15.169:0/4151071471 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efc90003fa0 0x7efca40c7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:16.120 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:16.116+0000 7efca898e640 1 -- 172.21.15.169:0/4151071471 >> 172.21.15.169:0/4151071471 conn(0x7efca4069930 msgr2=0x7efca406c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:16.120 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:16.116+0000 7efca898e640 1 -- 172.21.15.169:0/4151071471 shutdown_connections 2023-10-26T01:31:16.121 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:16.116+0000 7efca898e640 1 -- 172.21.15.169:0/4151071471 wait complete. 2023-10-26T01:31:16.138 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-26T01:31:16.138 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-26T01:31:16.138 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi170.front.sepia.ceph.com 2023-10-26T01:31:16.138 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-10-26T01:31:16.139 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-10-26T01:31:16.139 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-26T01:31:16.139 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-26T01:31:16.139 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-26T01:31:16.140 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-10-26T01:31:16.140 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:31:16.141 DEBUG:teuthology.orchestra.run.smithi170:> ip addr 2023-10-26T01:31:16.200 INFO:teuthology.orchestra.run.smithi170.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-26T01:31:16.201 INFO:teuthology.orchestra.run.smithi170.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-26T01:31:16.201 INFO:teuthology.orchestra.run.smithi170.stdout: inet 127.0.0.1/8 scope host lo 2023-10-26T01:31:16.201 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:31:16.202 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 ::1/128 scope host 2023-10-26T01:31:16.202 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:31:16.202 INFO:teuthology.orchestra.run.smithi170.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-26T01:31:16.203 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:d9:3b:54 brd ff:ff:ff:ff:ff:ff 2023-10-26T01:31:16.203 INFO:teuthology.orchestra.run.smithi170.stdout: altname enp7s0f0 2023-10-26T01:31:16.203 INFO:teuthology.orchestra.run.smithi170.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-26T01:31:16.203 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:d9:3b:55 brd ff:ff:ff:ff:ff:ff 2023-10-26T01:31:16.203 INFO:teuthology.orchestra.run.smithi170.stdout: altname enp7s0f1 2023-10-26T01:31:16.204 INFO:teuthology.orchestra.run.smithi170.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-26T01:31:16.204 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:88:72:54 brd ff:ff:ff:ff:ff:ff 2023-10-26T01:31:16.204 INFO:teuthology.orchestra.run.smithi170.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-26T01:31:16.204 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:88:72:55 brd ff:ff:ff:ff:ff:ff 2023-10-26T01:31:16.204 INFO:teuthology.orchestra.run.smithi170.stdout: inet 172.21.15.170/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-26T01:31:16.205 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft 42404sec preferred_lft 42404sec 2023-10-26T01:31:16.205 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::ec4:7aff:fe88:7255/64 scope link 2023-10-26T01:31:16.205 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:31:16.206 INFO:teuthology.orchestra.run.smithi170.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-26T01:31:16.206 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 32:b5:d1:ee:10:36 brd ff:ff:ff:ff:ff:ff 2023-10-26T01:31:16.206 INFO:teuthology.orchestra.run.smithi170.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-26T01:31:16.206 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:31:16.207 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::28ed:d1ff:fe37:70d/64 scope link 2023-10-26T01:31:16.207 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:31:16.207 INFO:teuthology.orchestra.run.smithi170.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-26T01:31:16.207 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 32:b5:d1:ee:10:36 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-26T01:31:16.207 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::30b5:d1ff:feee:1036/64 scope link 2023-10-26T01:31:16.208 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:31:16.209 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:31:16.209 DEBUG:teuthology.orchestra.run.smithi170:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-26T01:31:16.301 INFO:teuthology.orchestra.run.smithi170.stdout:1 2023-10-26T01:31:16.305 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:31:16.305 DEBUG:teuthology.orchestra.run.smithi170:> route 2023-10-26T01:31:16.359 INFO:teuthology.orchestra.run.smithi170.stdout:Kernel IP routing table 2023-10-26T01:31:16.359 INFO:teuthology.orchestra.run.smithi170.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-26T01:31:16.359 INFO:teuthology.orchestra.run.smithi170.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-26T01:31:16.359 INFO:teuthology.orchestra.run.smithi170.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-26T01:31:16.360 INFO:teuthology.orchestra.run.smithi170.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-26T01:31:16.360 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:31:16.361 DEBUG:teuthology.orchestra.run.smithi170:> (cd / && exec bash -c ' 2023-10-26T01:31:16.361 DEBUG:teuthology.orchestra.run.smithi170:> set -e 2023-10-26T01:31:16.361 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-26T01:31:16.361 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-26T01:31:16.362 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-26T01:31:16.362 DEBUG:teuthology.orchestra.run.smithi170:> ') 2023-10-26T01:31:16.544 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:31:16.545 DEBUG:teuthology.orchestra.run.smithi170:> ip netns list 2023-10-26T01:31:16.596 INFO:teuthology.orchestra.run.smithi170.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-26T01:31:16.597 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:31:16.598 DEBUG:teuthology.orchestra.run.smithi170:> ip netns list-id 2023-10-26T01:31:16.648 INFO:teuthology.orchestra.run.smithi170.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-10-26T01:31:16.649 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:31:16.650 DEBUG:teuthology.orchestra.run.smithi170:> (cd / && exec bash -c ' 2023-10-26T01:31:16.650 DEBUG:teuthology.orchestra.run.smithi170:> set -e 2023-10-26T01:31:16.650 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-26T01:31:16.650 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-10-26T01:31:16.651 DEBUG:teuthology.orchestra.run.smithi170:> ') 2023-10-26T01:31:16.792 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:31:16.792 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-10-26T01:31:16.883 INFO:teuthology.orchestra.run.smithi170.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-26T01:31:16.883 INFO:teuthology.orchestra.run.smithi170.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-26T01:31:16.883 INFO:teuthology.orchestra.run.smithi170.stdout: inet 127.0.0.1/8 scope host lo 2023-10-26T01:31:16.883 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:31:16.884 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 ::1/128 scope host 2023-10-26T01:31:16.884 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:31:16.884 INFO:teuthology.orchestra.run.smithi170.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-26T01:31:16.884 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 9a:0d:81:b9:2f:02 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-26T01:31:16.884 INFO:teuthology.orchestra.run.smithi170.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-26T01:31:16.885 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:31:16.885 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::980d:81ff:feb9:2f02/64 scope link 2023-10-26T01:31:16.885 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:31:16.887 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:31:16.888 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-10-26T01:31:16.977 INFO:teuthology.orchestra.run.smithi170.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-26T01:31:16.978 INFO:teuthology.orchestra.run.smithi170.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-26T01:31:16.978 INFO:teuthology.orchestra.run.smithi170.stdout: inet 127.0.0.1/8 scope host lo 2023-10-26T01:31:16.978 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:31:16.979 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 ::1/128 scope host 2023-10-26T01:31:16.979 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:31:16.979 INFO:teuthology.orchestra.run.smithi170.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-26T01:31:16.979 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 9a:0d:81:b9:2f:02 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-26T01:31:16.979 INFO:teuthology.orchestra.run.smithi170.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-26T01:31:16.980 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:31:16.980 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::980d:81ff:feb9:2f02/64 scope link 2023-10-26T01:31:16.980 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:31:16.982 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-10-26T01:31:16.983 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:31:16.983 DEBUG:teuthology.orchestra.run.smithi170:> (cd / && exec bash -c ' 2023-10-26T01:31:16.983 DEBUG:teuthology.orchestra.run.smithi170:> set -e 2023-10-26T01:31:16.984 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-10-26T01:31:16.984 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-10-26T01:31:16.984 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-10-26T01:31:16.985 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-10-26T01:31:16.985 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-10-26T01:31:16.985 DEBUG:teuthology.orchestra.run.smithi170:> ') 2023-10-26T01:31:17.244 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:31:17.245 DEBUG:teuthology.orchestra.run.smithi170:> (cd / && exec bash -c ' 2023-10-26T01:31:17.245 DEBUG:teuthology.orchestra.run.smithi170:> set -e 2023-10-26T01:31:17.245 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip link set brx.1 up 2023-10-26T01:31:17.246 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip link set dev brx.1 master ceph-brx 2023-10-26T01:31:17.246 DEBUG:teuthology.orchestra.run.smithi170:> ') 2023-10-26T01:31:17.416 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-26T01:31:17.416 DEBUG:teuthology.orchestra.run.smithi170:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2023-10-26T01:31:17.465 INFO:teuthology.orchestra.run.smithi170.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2023-10-26T01:31:17.466 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-26T01:31:17.466 DEBUG:teuthology.orchestra.run.smithi170:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2023-10-26T01:31:17.516 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-26T01:31:17.517 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:31:17.518 DEBUG:teuthology.orchestra.run.smithi170:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2023-10-26T01:31:17.657 INFO:teuthology.orchestra.run.smithi170.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2023-10-26T01:31:17.657 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2023-10-26T01:31:17.658 INFO:teuthology.orchestra.run.smithi170.stdout:invalid new device string format 2023-10-26T01:31:17.679 INFO:teuthology.orchestra.run.smithi170.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2023-10-26T01:31:17.679 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2023-10-26T01:31:17.680 INFO:teuthology.orchestra.run.smithi170.stdout:invalid new device string format 2023-10-26T01:31:17.680 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: resolved to: "172.21.15.169:3300,172.21.15.170:3300,172.21.15.170:3301" 2023-10-26T01:31:17.680 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-26T01:31:17.703 INFO:teuthology.orchestra.run.smithi170.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2023-10-26T01:31:17.704 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2023-10-26T01:31:17.704 INFO:teuthology.orchestra.run.smithi170.stdout:invalid new device string format 2023-10-26T01:31:17.704 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: resolved to: "172.21.15.169:3300,172.21.15.170:3300,172.21.15.170:3301" 2023-10-26T01:31:17.704 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-26T01:31:17.704 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: resolved to: "172.21.15.169:6789,172.21.15.170:6789,172.21.15.170:6790" 2023-10-26T01:31:17.705 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.169:6789,172.21.15.170:6789,172.21.15.170:6790:/ 2023-10-26T01:31:17.705 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs,key=3,fsid=30203b28-ed8a-44e8-a9a6-a9bf25f980f1" will pass to kernel 2023-10-26T01:31:17.706 INFO:teuthology.orchestra.run.smithi170.stdout:mount: /home/ubuntu/cephtest/mnt.3 does not contain SELinux labels. 2023-10-26T01:31:17.706 INFO:teuthology.orchestra.run.smithi170.stdout: You just mounted a file system that supports labels which does not 2023-10-26T01:31:17.706 INFO:teuthology.orchestra.run.smithi170.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-10-26T01:31:17.707 INFO:teuthology.orchestra.run.smithi170.stdout: applications will generate AVC messages and not be allowed access to 2023-10-26T01:31:17.707 INFO:teuthology.orchestra.run.smithi170.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-10-26T01:31:17.711 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-26T01:31:17.711 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:31:17.712 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-10-26T01:31:17.803 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:31:17.803 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-26T01:31:17.861 INFO:teuthology.orchestra.run.smithi170.stdout:client4520 2023-10-26T01:31:17.862 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-26T01:31:17.862 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:31:17.915 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:31:17.915 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-26T01:31:17.971 INFO:teuthology.orchestra.run.smithi170.stdout:client4520 2023-10-26T01:31:17.971 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:31:17.971 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-26T01:31:18.024 INFO:teuthology.orchestra.run.smithi170.stdout:client4520 2023-10-26T01:31:18.025 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:31:18.025 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/30203b28-ed8a-44e8-a9a6-a9bf25f980f1.client4520/status') 2023-10-26T01:31:18.108 INFO:teuthology.orchestra.run.smithi170.stdout:instance: client.4520 (3)192.168.0.2:0/1463211435 2023-10-26T01:31:18.108 INFO:teuthology.orchestra.run.smithi170.stdout:blocklisted: false 2023-10-26T01:31:18.109 INFO:teuthology.orchestra.run.smithi170.stderr:0+1 records in 2023-10-26T01:31:18.109 INFO:teuthology.orchestra.run.smithi170.stderr:0+1 records out 2023-10-26T01:31:18.109 INFO:teuthology.orchestra.run.smithi170.stderr:69 bytes copied, 5.606e-05 s, 1.2 MB/s 2023-10-26T01:31:18.112 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-10-26T01:31:18.129 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:31:18.355 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:18.353+0000 7f1917589640 1 -- 172.21.15.169:0/1934222753 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f18fc004d00 msgr2=0x7f18fc005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:18.356 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:18.353+0000 7f1917589640 1 --2- 172.21.15.169:0/1934222753 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f18fc004d00 0x7f18fc005180 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f190800c640 tx=0x7f18fc0057f0 comp rx=0 tx=0).stop 2023-10-26T01:31:18.356 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:18.353+0000 7f1915b26640 1 -- 172.21.15.169:0/1934222753 reap_dead start 2023-10-26T01:31:18.356 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:18.353+0000 7f1917589640 1 -- 172.21.15.169:0/1934222753 shutdown_connections 2023-10-26T01:31:18.356 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:18.353+0000 7f1917589640 1 -- 172.21.15.169:0/1934222753 >> 172.21.15.169:0/1934222753 conn(0x7f1910069930 msgr2=0x7f1910069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:18.357 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:18.354+0000 7f1917589640 1 -- 172.21.15.169:0/1934222753 shutdown_connections 2023-10-26T01:31:18.357 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:18.354+0000 7f1917589640 1 -- 172.21.15.169:0/1934222753 wait complete. 2023-10-26T01:31:18.357 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:18.355+0000 7f1917589640 1 Processor -- start 2023-10-26T01:31:18.357 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:18.356+0000 7f1917589640 1 -- start start 2023-10-26T01:31:18.359 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:18.357+0000 7f1917589640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f18fc004d00 0x7f191014f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:18.359 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:18.357+0000 7f1917589640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f19100aff90 0x7f19100adcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:18.360 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:18.357+0000 7f1917589640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f19100ae210 0x7f19101f3540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:18.360 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:18.357+0000 7f1917589640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19100b1480 con 0x7f18fc004d00 2023-10-26T01:31:18.360 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:18.357+0000 7f1917589640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19100ae690 con 0x7f19100aff90 2023-10-26T01:31:18.360 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:18.357+0000 7f1917589640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19100b0800 con 0x7f19100ae210 2023-10-26T01:31:18.361 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:18.357+0000 7f1915325640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f18fc004d00 0x7f191014f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:18.361 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:18.357+0000 7f1915325640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f18fc004d00 0x7f191014f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:54434/0 (socket says 172.21.15.169:54434) 2023-10-26T01:31:18.361 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:18.357+0000 7f1915325640 1 -- 172.21.15.169:0/2455538767 learned_addr learned my addr 172.21.15.169:0/2455538767 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:31:18.361 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:18.357+0000 7f1915325640 1 -- 172.21.15.169:0/2455538767 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f19100ae210 msgr2=0x7f19101f3540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:18.361 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:18.357+0000 7f1915325640 1 --2- 172.21.15.169:0/2455538767 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f19100ae210 0x7f19101f3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:18.362 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:18.357+0000 7f1915325640 1 -- 172.21.15.169:0/2455538767 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f19100aff90 msgr2=0x7f19100adcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:18.362 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:18.357+0000 7f1915325640 1 --2- 172.21.15.169:0/2455538767 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f19100aff90 0x7f19100adcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:18.362 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:18.357+0000 7f1915325640 1 -- 172.21.15.169:0/2455538767 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19100b0ae0 con 0x7f18fc004d00 2023-10-26T01:31:18.362 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:18.358+0000 7f1915325640 1 --2- 172.21.15.169:0/2455538767 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f18fc004d00 0x7f191014f060 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f190c00e990 tx=0x7f190c00ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:18.362 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:18.358+0000 7f1905ffb640 1 -- 172.21.15.169:0/2455538767 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f190c00cb60 con 0x7f18fc004d00 2023-10-26T01:31:18.363 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:18.358+0000 7f1905ffb640 1 -- 172.21.15.169:0/2455538767 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f190c00cdb0 con 0x7f18fc004d00 2023-10-26T01:31:18.363 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:18.358+0000 7f1917589640 1 -- 172.21.15.169:0/2455538767 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19100b0dc0 con 0x7f18fc004d00 2023-10-26T01:31:18.363 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:18.358+0000 7f1905ffb640 1 -- 172.21.15.169:0/2455538767 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f190c005280 con 0x7f18fc004d00 2023-10-26T01:31:18.363 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:18.358+0000 7f1917589640 1 -- 172.21.15.169:0/2455538767 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19100b03f0 con 0x7f18fc004d00 2023-10-26T01:31:18.364 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:18.359+0000 7f1917589640 1 -- 172.21.15.169:0/2455538767 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f191004db10 con 0x7f18fc004d00 2023-10-26T01:31:18.364 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:18.360+0000 7f1905ffb640 1 -- 172.21.15.169:0/2455538767 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f190c0053e0 con 0x7f18fc004d00 2023-10-26T01:31:18.365 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:18.361+0000 7f1905ffb640 1 --2- 172.21.15.169:0/2455538767 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f18f406ae70 0x7f18f406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:18.365 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:18.361+0000 7f1905ffb640 1 -- 172.21.15.169:0/2455538767 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f190c089e20 con 0x7f18fc004d00 2023-10-26T01:31:18.365 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:18.361+0000 7f1914b24640 1 --2- 172.21.15.169:0/2455538767 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f18f406ae70 0x7f18f406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:18.365 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:18.362+0000 7f1914b24640 1 --2- 172.21.15.169:0/2455538767 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f18f406ae70 0x7f18f406d330 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1900005d80 tx=0x7f1900005c90 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:18.367 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:18.365+0000 7f1905ffb640 1 -- 172.21.15.169:0/2455538767 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f190c054ec0 con 0x7f18fc004d00 2023-10-26T01:31:18.557 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:18.556+0000 7f1917589640 1 -- 172.21.15.169:0/2455538767 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1910069ff0 con 0x7f18fc004d00 2023-10-26T01:31:18.560 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:18.559+0000 7f1905ffb640 1 -- 172.21.15.169:0/2455538767 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16553 (secure 0 0 0) 0x7f190c01f070 con 0x7f18fc004d00 2023-10-26T01:31:18.561 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:31:18.561 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":21,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:30:43.078706+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-26T01:30:39.437631+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-26T01:30:40.561774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:31:18.562 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:18.561+0000 7f1917589640 1 -- 172.21.15.169:0/2455538767 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f18f406ae70 msgr2=0x7f18f406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:18.563 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:18.561+0000 7f1917589640 1 --2- 172.21.15.169:0/2455538767 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f18f406ae70 0x7f18f406d330 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1900005d80 tx=0x7f1900005c90 comp rx=0 tx=0).stop 2023-10-26T01:31:18.563 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:18.561+0000 7f1917589640 1 -- 172.21.15.169:0/2455538767 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f18fc004d00 msgr2=0x7f191014f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:18.563 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:18.561+0000 7f1917589640 1 --2- 172.21.15.169:0/2455538767 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f18fc004d00 0x7f191014f060 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f190c00e990 tx=0x7f190c00ee50 comp rx=0 tx=0).stop 2023-10-26T01:31:18.563 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:18.561+0000 7f1917589640 1 -- 172.21.15.169:0/2455538767 shutdown_connections 2023-10-26T01:31:18.564 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:18.561+0000 7f1917589640 1 --2- 172.21.15.169:0/2455538767 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f18f406ae70 0x7f18f406d330 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:18.564 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:18.561+0000 7f1917589640 1 --2- 172.21.15.169:0/2455538767 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f19100ae210 0x7f19101f3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:18.564 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:18.561+0000 7f1917589640 1 --2- 172.21.15.169:0/2455538767 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f19100aff90 0x7f19100adcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:18.564 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:18.561+0000 7f1917589640 1 --2- 172.21.15.169:0/2455538767 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f18fc004d00 0x7f191014f060 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:18.564 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:18.561+0000 7f1917589640 1 -- 172.21.15.169:0/2455538767 >> 172.21.15.169:0/2455538767 conn(0x7f1910069930 msgr2=0x7f191014ce50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:18.565 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:18.561+0000 7f1917589640 1 -- 172.21.15.169:0/2455538767 shutdown_connections 2023-10-26T01:31:18.565 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:18.561+0000 7f1917589640 1 -- 172.21.15.169:0/2455538767 wait complete. 2023-10-26T01:31:18.581 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T01:30:32.200808+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-26T01:30:39.437631+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-26T01:30:40.561774+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T01:31:18.582 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T01:31:18.794 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:18.792+0000 7fb494d83640 1 -- 172.21.15.169:0/1520751669 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb490159480 msgr2=0x7fb4900b02e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:18.794 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:18.792+0000 7fb494d83640 1 --2- 172.21.15.169:0/1520751669 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb490159480 0x7fb4900b02e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb480009830 tx=0x7fb48002bd50 comp rx=0 tx=0).stop 2023-10-26T01:31:18.794 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:18.792+0000 7fb494d83640 1 -- 172.21.15.169:0/1520751669 shutdown_connections 2023-10-26T01:31:18.795 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:18.792+0000 7fb494d83640 1 --2- 172.21.15.169:0/1520751669 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb4900cd690 0x7fb4900cdb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:18.795 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:18.792+0000 7fb494d83640 1 --2- 172.21.15.169:0/1520751669 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb490159480 0x7fb4900b02e0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:18.795 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:18.792+0000 7fb494d83640 1 -- 172.21.15.169:0/1520751669 >> 172.21.15.169:0/1520751669 conn(0x7fb490069930 msgr2=0x7fb490069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:18.795 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:18.792+0000 7fb494d83640 1 -- 172.21.15.169:0/1520751669 shutdown_connections 2023-10-26T01:31:18.795 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:18.792+0000 7fb494d83640 1 -- 172.21.15.169:0/1520751669 wait complete. 2023-10-26T01:31:18.797 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:18.794+0000 7fb494d83640 1 Processor -- start 2023-10-26T01:31:18.797 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:18.794+0000 7fb494d83640 1 -- start start 2023-10-26T01:31:18.797 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:18.795+0000 7fb494d83640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb4900b0820 0x7fb490158780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:18.797 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:18.795+0000 7fb494d83640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb4900cd690 0x7fb4901f50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:18.797 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:18.795+0000 7fb494d83640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb490159480 0x7fb49014e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:18.798 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:18.795+0000 7fb494d83640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4901f5630 con 0x7fb4900cd690 2023-10-26T01:31:18.798 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:18.795+0000 7fb494d83640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4901f58b0 con 0x7fb4900b0820 2023-10-26T01:31:18.798 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:18.795+0000 7fb494d83640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4901f5b90 con 0x7fb490159480 2023-10-26T01:31:18.798 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:18.795+0000 7fb487fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb4900cd690 0x7fb4901f50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:18.799 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:18.795+0000 7fb487fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb4900cd690 0x7fb4901f50f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:54454/0 (socket says 172.21.15.169:54454) 2023-10-26T01:31:18.799 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:18.795+0000 7fb487fff640 1 -- 172.21.15.169:0/3364169632 learned_addr learned my addr 172.21.15.169:0/3364169632 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:31:18.800 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:18.796+0000 7fb48e59c640 1 --2- 172.21.15.169:0/3364169632 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb4900b0820 0x7fb490158780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:18.800 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:18.796+0000 7fb48ed9d640 1 --2- 172.21.15.169:0/3364169632 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb490159480 0x7fb49014e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:18.801 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:18.796+0000 7fb487fff640 1 -- 172.21.15.169:0/3364169632 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb490159480 msgr2=0x7fb49014e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:18.801 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:18.796+0000 7fb487fff640 1 --2- 172.21.15.169:0/3364169632 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb490159480 0x7fb49014e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:18.801 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:18.796+0000 7fb487fff640 1 -- 172.21.15.169:0/3364169632 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb4900b0820 msgr2=0x7fb490158780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:18.801 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:18.796+0000 7fb487fff640 1 --2- 172.21.15.169:0/3364169632 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb4900b0820 0x7fb490158780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:18.801 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:18.796+0000 7fb487fff640 1 -- 172.21.15.169:0/3364169632 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb480009490 con 0x7fb4900cd690 2023-10-26T01:31:18.802 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:18.796+0000 7fb48e59c640 1 --2- 172.21.15.169:0/3364169632 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb4900b0820 0x7fb490158780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:31:18.802 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:18.796+0000 7fb487fff640 1 --2- 172.21.15.169:0/3364169632 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb4900cd690 0x7fb4901f50f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb47800c4b0 tx=0x7fb47800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:18.802 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:18.796+0000 7fb486ffd640 1 -- 172.21.15.169:0/3364169632 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb478005480 con 0x7fb4900cd690 2023-10-26T01:31:18.802 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:18.796+0000 7fb486ffd640 1 -- 172.21.15.169:0/3364169632 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb4780036e0 con 0x7fb4900cd690 2023-10-26T01:31:18.802 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:18.796+0000 7fb494d83640 1 -- 172.21.15.169:0/3364169632 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4901f5e70 con 0x7fb4900cd690 2023-10-26T01:31:18.803 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:18.796+0000 7fb486ffd640 1 -- 172.21.15.169:0/3364169632 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb478003b30 con 0x7fb4900cd690 2023-10-26T01:31:18.803 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:18.796+0000 7fb494d83640 1 -- 172.21.15.169:0/3364169632 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb490070c70 con 0x7fb4900cd690 2023-10-26T01:31:18.803 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:18.798+0000 7fb494d83640 1 -- 172.21.15.169:0/3364169632 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb490066af0 con 0x7fb4900cd690 2023-10-26T01:31:18.804 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:18.799+0000 7fb486ffd640 1 -- 172.21.15.169:0/3364169632 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fb478003c90 con 0x7fb4900cd690 2023-10-26T01:31:18.804 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:18.799+0000 7fb486ffd640 1 --2- 172.21.15.169:0/3364169632 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb47006ae40 0x7fb47006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:18.805 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:18.799+0000 7fb486ffd640 1 -- 172.21.15.169:0/3364169632 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb478088e30 con 0x7fb4900cd690 2023-10-26T01:31:18.805 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:18.800+0000 7fb48e59c640 1 --2- 172.21.15.169:0/3364169632 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb47006ae40 0x7fb47006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:18.805 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:18.801+0000 7fb48e59c640 1 --2- 172.21.15.169:0/3364169632 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb47006ae40 0x7fb47006d300 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb480036000 tx=0x7fb480004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:18.806 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:18.804+0000 7fb486ffd640 1 -- 172.21.15.169:0/3364169632 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb478053ed0 con 0x7fb4900cd690 2023-10-26T01:31:18.996 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:18.994+0000 7fb494d83640 1 -- 172.21.15.169:0/3364169632 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb4900711c0 con 0x7fb4900cd690 2023-10-26T01:31:18.997 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:18.995+0000 7fb486ffd640 1 -- 172.21.15.169:0/3364169632 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb4900711c0 con 0x7fb4900cd690 2023-10-26T01:31:18.998 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 1 2023-10-26T01:31:19.000 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:18.998+0000 7fb494d83640 1 -- 172.21.15.169:0/3364169632 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb47006ae40 msgr2=0x7fb47006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:19.000 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:18.998+0000 7fb494d83640 1 --2- 172.21.15.169:0/3364169632 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb47006ae40 0x7fb47006d300 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb480036000 tx=0x7fb480004730 comp rx=0 tx=0).stop 2023-10-26T01:31:19.001 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:18.998+0000 7fb494d83640 1 -- 172.21.15.169:0/3364169632 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb4900cd690 msgr2=0x7fb4901f50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:19.001 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:18.998+0000 7fb494d83640 1 --2- 172.21.15.169:0/3364169632 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb4900cd690 0x7fb4901f50f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb47800c4b0 tx=0x7fb47800c970 comp rx=0 tx=0).stop 2023-10-26T01:31:19.001 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:18.998+0000 7fb494d83640 1 -- 172.21.15.169:0/3364169632 shutdown_connections 2023-10-26T01:31:19.002 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:18.998+0000 7fb494d83640 1 --2- 172.21.15.169:0/3364169632 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb47006ae40 0x7fb47006d300 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:19.002 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:18.998+0000 7fb494d83640 1 --2- 172.21.15.169:0/3364169632 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb4900cd690 0x7fb4901f50f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:19.003 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:18.998+0000 7fb494d83640 1 --2- 172.21.15.169:0/3364169632 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb490159480 0x7fb49014e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:19.003 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:18.998+0000 7fb494d83640 1 --2- 172.21.15.169:0/3364169632 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb4900b0820 0x7fb490158780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:19.003 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:18.998+0000 7fb494d83640 1 -- 172.21.15.169:0/3364169632 >> 172.21.15.169:0/3364169632 conn(0x7fb490069930 msgr2=0x7fb49014f540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:19.004 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:18.998+0000 7fb494d83640 1 -- 172.21.15.169:0/3364169632 shutdown_connections 2023-10-26T01:31:19.004 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:18.998+0000 7fb494d83640 1 -- 172.21.15.169:0/3364169632 wait complete. 2023-10-26T01:31:19.018 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-26T01:31:19.249 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:19.247+0000 7f9813697640 1 -- 172.21.15.169:0/4201789139 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f980c0ac320 msgr2=0x7f980c0ac770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:19.250 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:19.247+0000 7f9813697640 1 --2- 172.21.15.169:0/4201789139 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f980c0ac320 0x7f980c0ac770 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f980c06a790 tx=0x7f980402bd90 comp rx=0 tx=0).stop 2023-10-26T01:31:19.250 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:19.247+0000 7f9813697640 1 -- 172.21.15.169:0/4201789139 shutdown_connections 2023-10-26T01:31:19.250 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:19.247+0000 7f9813697640 1 --2- 172.21.15.169:0/4201789139 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f980c0ac320 0x7f980c0ac770 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:19.250 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:19.247+0000 7f9813697640 1 --2- 172.21.15.169:0/4201789139 >> v2:172.21.15.170:3300/0 conn(0x7f980c0c9d00 0x7f980c0cdc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:19.250 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:19.247+0000 7f9813697640 1 --2- 172.21.15.169:0/4201789139 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f980c15b620 0x7f980c0cd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:19.251 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:19.247+0000 7f9813697640 1 -- 172.21.15.169:0/4201789139 >> 172.21.15.169:0/4201789139 conn(0x7f980c069930 msgr2=0x7f980c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:19.251 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:19.247+0000 7f9813697640 1 -- 172.21.15.169:0/4201789139 shutdown_connections 2023-10-26T01:31:19.251 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:19.247+0000 7f9813697640 1 -- 172.21.15.169:0/4201789139 wait complete. 2023-10-26T01:31:19.253 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:19.249+0000 7f9813697640 1 Processor -- start 2023-10-26T01:31:19.253 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:19.249+0000 7f9813697640 1 -- start start 2023-10-26T01:31:19.253 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:19.250+0000 7f9813697640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f980c0ac320 0x7f980c150950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:19.254 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:19.250+0000 7f9813697640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f980c0c9d00 0x7f980c15bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:19.254 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:19.250+0000 7f9813697640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f980c15b620 0x7f980c0710a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:19.254 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:19.250+0000 7f9813697640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f980c150e90 con 0x7f980c0ac320 2023-10-26T01:31:19.254 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:19.250+0000 7f9813697640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f980c1524c0 con 0x7f980c0c9d00 2023-10-26T01:31:19.255 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:19.250+0000 7f9813697640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f980c1527a0 con 0x7f980c15b620 2023-10-26T01:31:19.255 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:19.250+0000 7f9811433640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f980c0ac320 0x7f980c150950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:19.255 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:19.250+0000 7f9811433640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f980c0ac320 0x7f980c150950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:54460/0 (socket says 172.21.15.169:54460) 2023-10-26T01:31:19.255 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:19.250+0000 7f9811433640 1 -- 172.21.15.169:0/1794451130 learned_addr learned my addr 172.21.15.169:0/1794451130 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:31:19.255 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:19.251+0000 7f9810c32640 1 --2- 172.21.15.169:0/1794451130 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f980c0c9d00 0x7f980c15bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:19.256 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:19.251+0000 7f9811c34640 1 --2- 172.21.15.169:0/1794451130 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f980c15b620 0x7f980c0710a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:19.256 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:19.251+0000 7f9811433640 1 -- 172.21.15.169:0/1794451130 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f980c15b620 msgr2=0x7f980c0710a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:19.256 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:19.251+0000 7f9811433640 1 --2- 172.21.15.169:0/1794451130 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f980c15b620 0x7f980c0710a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:19.256 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:19.251+0000 7f9811433640 1 -- 172.21.15.169:0/1794451130 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f980c0c9d00 msgr2=0x7f980c15bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:19.256 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:19.251+0000 7f9811433640 1 --2- 172.21.15.169:0/1794451130 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f980c0c9d00 0x7f980c15bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:19.257 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:19.251+0000 7f9811433640 1 -- 172.21.15.169:0/1794451130 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9804009560 con 0x7f980c0ac320 2023-10-26T01:31:19.257 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:19.251+0000 7f9811433640 1 --2- 172.21.15.169:0/1794451130 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f980c0ac320 0x7f980c150950 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f980000cce0 tx=0x7f9800007610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:19.257 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:19.252+0000 7f9809ffb640 1 -- 172.21.15.169:0/1794451130 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9800004b20 con 0x7f980c0ac320 2023-10-26T01:31:19.257 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:19.252+0000 7f9809ffb640 1 -- 172.21.15.169:0/1794451130 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9800004d70 con 0x7f980c0ac320 2023-10-26T01:31:19.257 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:19.252+0000 7f9809ffb640 1 -- 172.21.15.169:0/1794451130 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9800005280 con 0x7f980c0ac320 2023-10-26T01:31:19.258 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:19.252+0000 7f9813697640 1 -- 172.21.15.169:0/1794451130 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f980c152a80 con 0x7f980c0ac320 2023-10-26T01:31:19.258 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:19.252+0000 7f9813697640 1 -- 172.21.15.169:0/1794451130 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f980c15c230 con 0x7f980c0ac320 2023-10-26T01:31:19.259 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:19.253+0000 7f9813697640 1 -- 172.21.15.169:0/1794451130 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f980c04db10 con 0x7f980c0ac320 2023-10-26T01:31:19.259 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:19.254+0000 7f9809ffb640 1 -- 172.21.15.169:0/1794451130 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f98000053e0 con 0x7f980c0ac320 2023-10-26T01:31:19.259 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:19.255+0000 7f9809ffb640 1 --2- 172.21.15.169:0/1794451130 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f97f006ae40 0x7f97f006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:19.260 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:19.255+0000 7f9809ffb640 1 -- 172.21.15.169:0/1794451130 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9800089e40 con 0x7f980c0ac320 2023-10-26T01:31:19.260 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:19.255+0000 7f9810c32640 1 --2- 172.21.15.169:0/1794451130 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f97f006ae40 0x7f97f006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:19.260 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:19.256+0000 7f9810c32640 1 --2- 172.21.15.169:0/1794451130 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f97f006ae40 0x7f97f006d300 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f97f8005fd0 tx=0x7f97f800a4e0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:19.260 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:19.259+0000 7f9809ffb640 1 -- 172.21.15.169:0/1794451130 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9800054ef0 con 0x7f980c0ac320 2023-10-26T01:31:19.449 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:19.448+0000 7f9813697640 1 -- 172.21.15.169:0/1794451130 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f980c15a720 con 0x7f980c0ac320 2023-10-26T01:31:19.450 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:19.449+0000 7f9809ffb640 1 -- 172.21.15.169:0/1794451130 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f980001d020 con 0x7f980c0ac320 2023-10-26T01:31:19.451 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 32 2023-10-26T01:31:19.453 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:19.452+0000 7f9813697640 1 -- 172.21.15.169:0/1794451130 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f97f006ae40 msgr2=0x7f97f006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:19.453 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:19.452+0000 7f9813697640 1 --2- 172.21.15.169:0/1794451130 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f97f006ae40 0x7f97f006d300 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f97f8005fd0 tx=0x7f97f800a4e0 comp rx=0 tx=0).stop 2023-10-26T01:31:19.454 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:19.452+0000 7f9813697640 1 -- 172.21.15.169:0/1794451130 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f980c0ac320 msgr2=0x7f980c150950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:19.454 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:19.452+0000 7f9813697640 1 --2- 172.21.15.169:0/1794451130 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f980c0ac320 0x7f980c150950 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f980000cce0 tx=0x7f9800007610 comp rx=0 tx=0).stop 2023-10-26T01:31:19.454 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:19.452+0000 7f9813697640 1 -- 172.21.15.169:0/1794451130 shutdown_connections 2023-10-26T01:31:19.455 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:19.452+0000 7f9813697640 1 --2- 172.21.15.169:0/1794451130 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f97f006ae40 0x7f97f006d300 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:19.455 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:19.452+0000 7f9813697640 1 --2- 172.21.15.169:0/1794451130 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f980c15b620 0x7f980c0710a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:19.455 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:19.452+0000 7f9813697640 1 --2- 172.21.15.169:0/1794451130 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f980c0c9d00 0x7f980c15bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:19.455 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:19.452+0000 7f9813697640 1 --2- 172.21.15.169:0/1794451130 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f980c0ac320 0x7f980c150950 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:19.456 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:19.452+0000 7f9813697640 1 -- 172.21.15.169:0/1794451130 >> 172.21.15.169:0/1794451130 conn(0x7f980c069930 msgr2=0x7f980c0cd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:19.456 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:19.452+0000 7f9813697640 1 -- 172.21.15.169:0/1794451130 shutdown_connections 2023-10-26T01:31:19.456 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:19.452+0000 7f9813697640 1 -- 172.21.15.169:0/1794451130 wait complete. 2023-10-26T01:31:19.474 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-26T01:31:19.702 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:19.699+0000 7f37e78e5640 1 -- 172.21.15.169:0/4186408993 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37e0069c20 msgr2=0x7f37e0164490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:19.702 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:19.699+0000 7f37e78e5640 1 --2- 172.21.15.169:0/4186408993 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37e0069c20 0x7f37e0164490 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f37d8005fd0 tx=0x7f37d802c0f0 comp rx=0 tx=0).stop 2023-10-26T01:31:19.703 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:19.699+0000 7f37e78e5640 1 -- 172.21.15.169:0/4186408993 shutdown_connections 2023-10-26T01:31:19.703 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:19.699+0000 7f37e78e5640 1 --2- 172.21.15.169:0/4186408993 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37e0069c20 0x7f37e0164490 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:19.703 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:19.699+0000 7f37e78e5640 1 -- 172.21.15.169:0/4186408993 >> 172.21.15.169:0/4186408993 conn(0x7f37e006a620 msgr2=0x7f37e006aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:19.704 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:19.699+0000 7f37e78e5640 1 -- 172.21.15.169:0/4186408993 shutdown_connections 2023-10-26T01:31:19.704 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:19.699+0000 7f37e78e5640 1 -- 172.21.15.169:0/4186408993 wait complete. 2023-10-26T01:31:19.707 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:19.701+0000 7f37e78e5640 1 Processor -- start 2023-10-26T01:31:19.707 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:19.701+0000 7f37e78e5640 1 -- start start 2023-10-26T01:31:19.707 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:19.702+0000 7f37e78e5640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37e0069c20 0x7f37e0157960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:19.708 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:19.702+0000 7f37e78e5640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f37e00cbe10 0x7f37e0139d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:19.708 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:19.702+0000 7f37e78e5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f37e0143080 0x7f37e0158e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:19.708 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:19.702+0000 7f37e78e5640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37e013a790 con 0x7f37e0069c20 2023-10-26T01:31:19.708 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:19.702+0000 7f37e78e5640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37e013aa30 con 0x7f37e0143080 2023-10-26T01:31:19.708 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:19.702+0000 7f37e78e5640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37e013ad10 con 0x7f37e00cbe10 2023-10-26T01:31:19.709 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:19.702+0000 7f37e5681640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37e0069c20 0x7f37e0157960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:19.709 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:19.702+0000 7f37e5681640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37e0069c20 0x7f37e0157960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:54476/0 (socket says 172.21.15.169:54476) 2023-10-26T01:31:19.709 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:19.702+0000 7f37e5681640 1 -- 172.21.15.169:0/1517653210 learned_addr learned my addr 172.21.15.169:0/1517653210 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:31:19.709 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:19.702+0000 7f37e5681640 1 -- 172.21.15.169:0/1517653210 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f37e00cbe10 msgr2=0x7f37e0139d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:19.709 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:19.702+0000 7f37e5681640 1 --2- 172.21.15.169:0/1517653210 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f37e00cbe10 0x7f37e0139d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:19.710 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:19.702+0000 7f37e5681640 1 -- 172.21.15.169:0/1517653210 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f37e0143080 msgr2=0x7f37e0158e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:19.710 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:19.702+0000 7f37e5681640 1 --2- 172.21.15.169:0/1517653210 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f37e0143080 0x7f37e0158e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:19.710 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:19.702+0000 7f37e5681640 1 -- 172.21.15.169:0/1517653210 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37d8009840 con 0x7f37e0069c20 2023-10-26T01:31:19.710 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:19.703+0000 7f37e5681640 1 --2- 172.21.15.169:0/1517653210 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37e0069c20 0x7f37e0157960 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f37dc00b3f0 tx=0x7f37dc00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:19.711 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:19.703+0000 7f37d5ffb640 1 -- 172.21.15.169:0/1517653210 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37dc011830 con 0x7f37e0069c20 2023-10-26T01:31:19.711 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:19.703+0000 7f37d5ffb640 1 -- 172.21.15.169:0/1517653210 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f37dc011a80 con 0x7f37e0069c20 2023-10-26T01:31:19.711 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:19.703+0000 7f37d5ffb640 1 -- 172.21.15.169:0/1517653210 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37dc011e90 con 0x7f37e0069c20 2023-10-26T01:31:19.711 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:19.703+0000 7f37e78e5640 1 -- 172.21.15.169:0/1517653210 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37e013aff0 con 0x7f37e0069c20 2023-10-26T01:31:19.711 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:19.703+0000 7f37e78e5640 1 -- 172.21.15.169:0/1517653210 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37e0157ea0 con 0x7f37e0069c20 2023-10-26T01:31:19.712 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:19.704+0000 7f37e78e5640 1 -- 172.21.15.169:0/1517653210 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37e0066af0 con 0x7f37e0069c20 2023-10-26T01:31:19.712 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:19.705+0000 7f37d5ffb640 1 -- 172.21.15.169:0/1517653210 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f37dc01a400 con 0x7f37e0069c20 2023-10-26T01:31:19.713 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:19.706+0000 7f37d5ffb640 1 --2- 172.21.15.169:0/1517653210 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f37c406ae40 0x7f37c406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:19.713 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:19.706+0000 7f37d5ffb640 1 -- 172.21.15.169:0/1517653210 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f37dc089720 con 0x7f37e0069c20 2023-10-26T01:31:19.713 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:19.707+0000 7f37e4e80640 1 --2- 172.21.15.169:0/1517653210 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f37c406ae40 0x7f37c406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:19.714 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:19.708+0000 7f37e4e80640 1 --2- 172.21.15.169:0/1517653210 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f37c406ae40 0x7f37c406d300 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f37d00046b0 tx=0x7f37d0007a70 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:19.714 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:19.711+0000 7f37d5ffb640 1 -- 172.21.15.169:0/1517653210 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f37dc0547d0 con 0x7f37e0069c20 2023-10-26T01:31:19.901 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:19.899+0000 7f37e78e5640 1 -- 172.21.15.169:0/1517653210 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f37e0072060 con 0x7f37e0069c20 2023-10-26T01:31:19.902 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:19.900+0000 7f37d5ffb640 1 -- 172.21.15.169:0/1517653210 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f37dc057df0 con 0x7f37e0069c20 2023-10-26T01:31:19.902 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 64 2023-10-26T01:31:19.904 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:19.903+0000 7f37e78e5640 1 -- 172.21.15.169:0/1517653210 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f37c406ae40 msgr2=0x7f37c406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:19.905 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:19.903+0000 7f37e78e5640 1 --2- 172.21.15.169:0/1517653210 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f37c406ae40 0x7f37c406d300 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f37d00046b0 tx=0x7f37d0007a70 comp rx=0 tx=0).stop 2023-10-26T01:31:19.905 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:19.903+0000 7f37e78e5640 1 -- 172.21.15.169:0/1517653210 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37e0069c20 msgr2=0x7f37e0157960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:19.905 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:19.903+0000 7f37e78e5640 1 --2- 172.21.15.169:0/1517653210 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37e0069c20 0x7f37e0157960 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f37dc00b3f0 tx=0x7f37dc00b8b0 comp rx=0 tx=0).stop 2023-10-26T01:31:19.905 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:19.903+0000 7f37e78e5640 1 -- 172.21.15.169:0/1517653210 shutdown_connections 2023-10-26T01:31:19.905 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:19.903+0000 7f37e78e5640 1 --2- 172.21.15.169:0/1517653210 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f37c406ae40 0x7f37c406d300 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:19.906 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:19.903+0000 7f37e78e5640 1 --2- 172.21.15.169:0/1517653210 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f37e0143080 0x7f37e0158e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:19.906 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:19.903+0000 7f37e78e5640 1 --2- 172.21.15.169:0/1517653210 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f37e00cbe10 0x7f37e0139d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:19.906 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:19.903+0000 7f37e78e5640 1 --2- 172.21.15.169:0/1517653210 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37e0069c20 0x7f37e0157960 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:19.906 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:19.903+0000 7f37e78e5640 1 -- 172.21.15.169:0/1517653210 >> 172.21.15.169:0/1517653210 conn(0x7f37e006a620 msgr2=0x7f37e013fa10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:19.907 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:19.903+0000 7f37e78e5640 1 -- 172.21.15.169:0/1517653210 shutdown_connections 2023-10-26T01:31:19.907 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:19.903+0000 7f37e78e5640 1 -- 172.21.15.169:0/1517653210 wait complete. 2023-10-26T01:31:19.925 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:31:20.147 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:20.144+0000 7fdb3f20e640 1 -- 172.21.15.169:0/934245151 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdb280047c0 msgr2=0x7fdb28004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:20.147 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:20.144+0000 7fdb3f20e640 1 --2- 172.21.15.169:0/934245151 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdb280047c0 0x7fdb28004c40 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fdb24006fc0 tx=0x7fdb280057f0 comp rx=0 tx=0).stop 2023-10-26T01:31:20.147 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:20.145+0000 7fdb3f20e640 1 -- 172.21.15.169:0/934245151 shutdown_connections 2023-10-26T01:31:20.147 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:20.145+0000 7fdb3f20e640 1 --2- 172.21.15.169:0/934245151 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdb280047c0 0x7fdb28004c40 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:20.148 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:20.145+0000 7fdb3f20e640 1 -- 172.21.15.169:0/934245151 >> 172.21.15.169:0/934245151 conn(0x7fdb38069930 msgr2=0x7fdb38069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:20.148 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:20.145+0000 7fdb3f20e640 1 -- 172.21.15.169:0/934245151 shutdown_connections 2023-10-26T01:31:20.148 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:20.145+0000 7fdb3f20e640 1 -- 172.21.15.169:0/934245151 wait complete. 2023-10-26T01:31:20.150 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:20.146+0000 7fdb3f20e640 1 Processor -- start 2023-10-26T01:31:20.150 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:20.147+0000 7fdb3f20e640 1 -- start start 2023-10-26T01:31:20.150 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:20.147+0000 7fdb3f20e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdb280047c0 0x7fdb38071cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:20.150 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:20.147+0000 7fdb3f20e640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdb3809ae50 0x7fdb3814f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:20.151 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:20.147+0000 7fdb3f20e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdb381553a0 0x7fdb38155850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:20.151 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:20.147+0000 7fdb3f20e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb3815af20 con 0x7fdb381553a0 2023-10-26T01:31:20.151 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:20.147+0000 7fdb3f20e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb38155d90 con 0x7fdb280047c0 2023-10-26T01:31:20.151 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:20.147+0000 7fdb3f20e640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb3809b2d0 con 0x7fdb3809ae50 2023-10-26T01:31:20.151 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:20.148+0000 7fdb3d7ab640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdb381553a0 0x7fdb38155850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:20.151 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:20.148+0000 7fdb3d7ab640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdb381553a0 0x7fdb38155850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:54498/0 (socket says 172.21.15.169:54498) 2023-10-26T01:31:20.152 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:20.148+0000 7fdb3d7ab640 1 -- 172.21.15.169:0/3382065879 learned_addr learned my addr 172.21.15.169:0/3382065879 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:31:20.152 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:20.148+0000 7fdb3cfaa640 1 --2- 172.21.15.169:0/3382065879 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdb280047c0 0x7fdb38071cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:20.152 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:20.148+0000 7fdb37fff640 1 --2- 172.21.15.169:0/3382065879 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdb3809ae50 0x7fdb3814f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:20.152 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:20.148+0000 7fdb3d7ab640 1 -- 172.21.15.169:0/3382065879 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdb3809ae50 msgr2=0x7fdb3814f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:20.152 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:20.148+0000 7fdb3d7ab640 1 --2- 172.21.15.169:0/3382065879 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdb3809ae50 0x7fdb3814f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:20.153 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:20.148+0000 7fdb3d7ab640 1 -- 172.21.15.169:0/3382065879 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdb280047c0 msgr2=0x7fdb38071cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:20.153 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:20.148+0000 7fdb3d7ab640 1 --2- 172.21.15.169:0/3382065879 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdb280047c0 0x7fdb38071cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:20.154 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:20.148+0000 7fdb3d7ab640 1 -- 172.21.15.169:0/3382065879 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb3806f060 con 0x7fdb381553a0 2023-10-26T01:31:20.154 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:20.149+0000 7fdb3d7ab640 1 --2- 172.21.15.169:0/3382065879 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdb381553a0 0x7fdb38155850 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fdb30005fd0 tx=0x7fdb3000ed20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:20.156 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:20.149+0000 7fdb357fa640 1 -- 172.21.15.169:0/3382065879 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb30004d20 con 0x7fdb381553a0 2023-10-26T01:31:20.156 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:20.149+0000 7fdb357fa640 1 -- 172.21.15.169:0/3382065879 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fdb3000ca00 con 0x7fdb381553a0 2023-10-26T01:31:20.156 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:20.149+0000 7fdb357fa640 1 -- 172.21.15.169:0/3382065879 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb3000ce10 con 0x7fdb381553a0 2023-10-26T01:31:20.157 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:20.149+0000 7fdb3f20e640 1 -- 172.21.15.169:0/3382065879 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb3806f7b0 con 0x7fdb381553a0 2023-10-26T01:31:20.157 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:20.149+0000 7fdb3f20e640 1 -- 172.21.15.169:0/3382065879 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb3815ab40 con 0x7fdb381553a0 2023-10-26T01:31:20.157 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:20.150+0000 7fdb3f20e640 1 -- 172.21.15.169:0/3382065879 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb3806fcd0 con 0x7fdb381553a0 2023-10-26T01:31:20.157 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:20.151+0000 7fdb357fa640 1 -- 172.21.15.169:0/3382065879 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fdb30011470 con 0x7fdb381553a0 2023-10-26T01:31:20.157 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:20.152+0000 7fdb357fa640 1 --2- 172.21.15.169:0/3382065879 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fdb1c06ae70 0x7fdb1c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:20.158 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:20.152+0000 7fdb357fa640 1 -- 172.21.15.169:0/3382065879 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdb30089a10 con 0x7fdb381553a0 2023-10-26T01:31:20.158 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:20.153+0000 7fdb3cfaa640 1 --2- 172.21.15.169:0/3382065879 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fdb1c06ae70 0x7fdb1c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:20.158 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:20.154+0000 7fdb3cfaa640 1 --2- 172.21.15.169:0/3382065879 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fdb1c06ae70 0x7fdb1c06d330 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fdb3809a850 tx=0x7fdb2c006bc0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:20.159 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:20.156+0000 7fdb357fa640 1 -- 172.21.15.169:0/3382065879 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdb3806fcd0 con 0x7fdb381553a0 2023-10-26T01:31:20.349 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:20.348+0000 7fdb3f20e640 1 -- 172.21.15.169:0/3382065879 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdb3815b060 con 0x7fdb381553a0 2023-10-26T01:31:20.352 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:20.350+0000 7fdb357fa640 1 -- 172.21.15.169:0/3382065879 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16553 (secure 0 0 0) 0x7fdb3815b060 con 0x7fdb381553a0 2023-10-26T01:31:20.353 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:31:20.353 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":21,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:30:43.078706+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-26T01:30:39.437631+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-26T01:30:40.561774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:31:20.355 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:20.353+0000 7fdb3f20e640 1 -- 172.21.15.169:0/3382065879 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fdb1c06ae70 msgr2=0x7fdb1c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:20.355 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:20.353+0000 7fdb3f20e640 1 --2- 172.21.15.169:0/3382065879 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fdb1c06ae70 0x7fdb1c06d330 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fdb3809a850 tx=0x7fdb2c006bc0 comp rx=0 tx=0).stop 2023-10-26T01:31:20.355 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:20.353+0000 7fdb3f20e640 1 -- 172.21.15.169:0/3382065879 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdb381553a0 msgr2=0x7fdb38155850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:20.356 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:20.353+0000 7fdb3f20e640 1 --2- 172.21.15.169:0/3382065879 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdb381553a0 0x7fdb38155850 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fdb30005fd0 tx=0x7fdb3000ed20 comp rx=0 tx=0).stop 2023-10-26T01:31:20.356 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:20.353+0000 7fdb3f20e640 1 -- 172.21.15.169:0/3382065879 shutdown_connections 2023-10-26T01:31:20.356 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:20.353+0000 7fdb3f20e640 1 --2- 172.21.15.169:0/3382065879 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fdb1c06ae70 0x7fdb1c06d330 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:20.356 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:20.353+0000 7fdb3f20e640 1 --2- 172.21.15.169:0/3382065879 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdb381553a0 0x7fdb38155850 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:20.356 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:20.353+0000 7fdb3f20e640 1 --2- 172.21.15.169:0/3382065879 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdb3809ae50 0x7fdb3814f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:20.357 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:20.353+0000 7fdb3f20e640 1 --2- 172.21.15.169:0/3382065879 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdb280047c0 0x7fdb38071cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:20.357 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:20.354+0000 7fdb3f20e640 1 -- 172.21.15.169:0/3382065879 >> 172.21.15.169:0/3382065879 conn(0x7fdb38069930 msgr2=0x7fdb38151750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:20.357 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:20.354+0000 7fdb3f20e640 1 -- 172.21.15.169:0/3382065879 shutdown_connections 2023-10-26T01:31:20.357 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:20.354+0000 7fdb3f20e640 1 -- 172.21.15.169:0/3382065879 wait complete. 2023-10-26T01:31:20.374 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T01:30:32.200808+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-26T01:30:39.437631+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-26T01:30:40.561774+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T01:31:20.374 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T01:31:20.599 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:20.597+0000 7f9b61ec2640 1 -- 172.21.15.169:0/2288000971 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9b5c159480 msgr2=0x7f9b5c0b02e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:20.599 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:20.597+0000 7f9b61ec2640 1 --2- 172.21.15.169:0/2288000971 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9b5c159480 0x7f9b5c0b02e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9b5c0ce2c0 tx=0x7f9b4c02be40 comp rx=0 tx=0).stop 2023-10-26T01:31:20.599 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:20.597+0000 7f9b61ec2640 1 -- 172.21.15.169:0/2288000971 shutdown_connections 2023-10-26T01:31:20.600 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:20.597+0000 7f9b61ec2640 1 --2- 172.21.15.169:0/2288000971 >> v2:172.21.15.170:3300/0 conn(0x7f9b5c0b0820 0x7f9b5c0aaf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:20.600 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:20.597+0000 7f9b61ec2640 1 --2- 172.21.15.169:0/2288000971 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9b5c159480 0x7f9b5c0b02e0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:20.600 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:20.597+0000 7f9b61ec2640 1 -- 172.21.15.169:0/2288000971 >> 172.21.15.169:0/2288000971 conn(0x7f9b5c069930 msgr2=0x7f9b5c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:20.600 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:20.597+0000 7f9b61ec2640 1 -- 172.21.15.169:0/2288000971 shutdown_connections 2023-10-26T01:31:20.600 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:20.597+0000 7f9b61ec2640 1 -- 172.21.15.169:0/2288000971 wait complete. 2023-10-26T01:31:20.601 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:20.599+0000 7f9b61ec2640 1 Processor -- start 2023-10-26T01:31:20.601 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:20.599+0000 7f9b61ec2640 1 -- start start 2023-10-26T01:31:20.602 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:20.600+0000 7f9b61ec2640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9b5c0ab4c0 0x7f9b5c14d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:20.602 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:20.600+0000 7f9b61ec2640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9b5c0b0820 0x7f9b5c157350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:20.603 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:20.600+0000 7f9b61ec2640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9b5c159480 0x7f9b5c157890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:20.603 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:20.600+0000 7f9b61ec2640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b5c070610 con 0x7f9b5c159480 2023-10-26T01:31:20.603 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:20.600+0000 7f9b61ec2640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b5c09aa60 con 0x7f9b5c0ab4c0 2023-10-26T01:31:20.603 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:20.600+0000 7f9b61ec2640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b5c09ad40 con 0x7f9b5c0b0820 2023-10-26T01:31:20.603 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:20.600+0000 7f9b5bfff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9b5c159480 0x7f9b5c157890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:20.604 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:20.601+0000 7f9b5bfff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9b5c159480 0x7f9b5c157890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:54504/0 (socket says 172.21.15.169:54504) 2023-10-26T01:31:20.604 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:20.601+0000 7f9b5bfff640 1 -- 172.21.15.169:0/3309109185 learned_addr learned my addr 172.21.15.169:0/3309109185 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:31:20.604 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:20.601+0000 7f9b5b7fe640 1 --2- 172.21.15.169:0/3309109185 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9b5c0ab4c0 0x7f9b5c14d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:20.604 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:20.601+0000 7f9b53fff640 1 --2- 172.21.15.169:0/3309109185 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9b5c0b0820 0x7f9b5c157350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:20.604 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:20.601+0000 7f9b5bfff640 1 -- 172.21.15.169:0/3309109185 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9b5c0b0820 msgr2=0x7f9b5c157350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:20.605 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:20.601+0000 7f9b5bfff640 1 --2- 172.21.15.169:0/3309109185 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9b5c0b0820 0x7f9b5c157350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:20.605 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:20.601+0000 7f9b5bfff640 1 -- 172.21.15.169:0/3309109185 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9b5c0ab4c0 msgr2=0x7f9b5c14d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:20.605 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:20.601+0000 7f9b5bfff640 1 --2- 172.21.15.169:0/3309109185 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9b5c0ab4c0 0x7f9b5c14d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:20.605 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:20.601+0000 7f9b5bfff640 1 -- 172.21.15.169:0/3309109185 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b4c009560 con 0x7f9b5c159480 2023-10-26T01:31:20.606 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:20.601+0000 7f9b5bfff640 1 --2- 172.21.15.169:0/3309109185 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9b5c159480 0x7f9b5c157890 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9b5400c710 tx=0x7f9b5400cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:20.607 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:20.602+0000 7f9b58ff9640 1 -- 172.21.15.169:0/3309109185 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b54006d50 con 0x7f9b5c159480 2023-10-26T01:31:20.607 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:20.602+0000 7f9b58ff9640 1 -- 172.21.15.169:0/3309109185 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9b5400ce20 con 0x7f9b5c159480 2023-10-26T01:31:20.607 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:20.602+0000 7f9b58ff9640 1 -- 172.21.15.169:0/3309109185 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b54003bd0 con 0x7f9b5c159480 2023-10-26T01:31:20.607 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:20.602+0000 7f9b61ec2640 1 -- 172.21.15.169:0/3309109185 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b5c09b020 con 0x7f9b5c159480 2023-10-26T01:31:20.607 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:20.602+0000 7f9b61ec2640 1 -- 172.21.15.169:0/3309109185 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b5c14dde0 con 0x7f9b5c159480 2023-10-26T01:31:20.608 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:20.603+0000 7f9b61ec2640 1 -- 172.21.15.169:0/3309109185 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b5c066af0 con 0x7f9b5c159480 2023-10-26T01:31:20.608 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:20.604+0000 7f9b58ff9640 1 -- 172.21.15.169:0/3309109185 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f9b54003d30 con 0x7f9b5c159480 2023-10-26T01:31:20.608 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:20.605+0000 7f9b58ff9640 1 --2- 172.21.15.169:0/3309109185 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9b3c06f250 0x7f9b3c071710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:20.608 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:20.605+0000 7f9b58ff9640 1 -- 172.21.15.169:0/3309109185 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9b540893c0 con 0x7f9b5c159480 2023-10-26T01:31:20.609 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:20.605+0000 7f9b5b7fe640 1 --2- 172.21.15.169:0/3309109185 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9b3c06f250 0x7f9b3c071710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:20.609 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:20.606+0000 7f9b5b7fe640 1 --2- 172.21.15.169:0/3309109185 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9b3c06f250 0x7f9b3c071710 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c035000 tx=0x7f9b4c004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:20.611 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:20.609+0000 7f9b58ff9640 1 -- 172.21.15.169:0/3309109185 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9b54054500 con 0x7f9b5c159480 2023-10-26T01:31:20.799 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:20.797+0000 7f9b61ec2640 1 -- 172.21.15.169:0/3309109185 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9b5c159940 con 0x7f9b5c159480 2023-10-26T01:31:20.799 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:20.798+0000 7f9b58ff9640 1 -- 172.21.15.169:0/3309109185 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9b54057b20 con 0x7f9b5c159480 2023-10-26T01:31:20.800 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 1 2023-10-26T01:31:20.803 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:20.800+0000 7f9b61ec2640 1 -- 172.21.15.169:0/3309109185 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9b3c06f250 msgr2=0x7f9b3c071710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:20.803 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:20.800+0000 7f9b61ec2640 1 --2- 172.21.15.169:0/3309109185 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9b3c06f250 0x7f9b3c071710 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c035000 tx=0x7f9b4c004730 comp rx=0 tx=0).stop 2023-10-26T01:31:20.803 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:20.801+0000 7f9b61ec2640 1 -- 172.21.15.169:0/3309109185 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9b5c159480 msgr2=0x7f9b5c157890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:20.803 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:20.801+0000 7f9b61ec2640 1 --2- 172.21.15.169:0/3309109185 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9b5c159480 0x7f9b5c157890 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9b5400c710 tx=0x7f9b5400cbd0 comp rx=0 tx=0).stop 2023-10-26T01:31:20.803 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:20.801+0000 7f9b61ec2640 1 -- 172.21.15.169:0/3309109185 shutdown_connections 2023-10-26T01:31:20.804 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:20.801+0000 7f9b61ec2640 1 --2- 172.21.15.169:0/3309109185 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9b3c06f250 0x7f9b3c071710 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:20.804 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:20.801+0000 7f9b61ec2640 1 --2- 172.21.15.169:0/3309109185 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9b5c159480 0x7f9b5c157890 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:20.804 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:20.801+0000 7f9b61ec2640 1 --2- 172.21.15.169:0/3309109185 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9b5c0b0820 0x7f9b5c157350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:20.804 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:20.801+0000 7f9b61ec2640 1 --2- 172.21.15.169:0/3309109185 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9b5c0ab4c0 0x7f9b5c14d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:20.804 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:20.801+0000 7f9b61ec2640 1 -- 172.21.15.169:0/3309109185 >> 172.21.15.169:0/3309109185 conn(0x7f9b5c069930 msgr2=0x7f9b5c14f770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:20.805 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:20.801+0000 7f9b61ec2640 1 -- 172.21.15.169:0/3309109185 shutdown_connections 2023-10-26T01:31:20.805 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:20.801+0000 7f9b61ec2640 1 -- 172.21.15.169:0/3309109185 wait complete. 2023-10-26T01:31:20.822 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-26T01:31:21.042 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.040+0000 7f387cb05640 1 -- 172.21.15.169:0/434693999 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f38781597c0 msgr2=0x7f38780ac200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:21.042 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.040+0000 7f387cb05640 1 --2- 172.21.15.169:0/434693999 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f38781597c0 0x7f38780ac200 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f38780cd2c0 tx=0x7f386802c260 comp rx=0 tx=0).stop 2023-10-26T01:31:21.043 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.040+0000 7f387cb05640 1 -- 172.21.15.169:0/434693999 shutdown_connections 2023-10-26T01:31:21.043 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.040+0000 7f387cb05640 1 --2- 172.21.15.169:0/434693999 >> v2:172.21.15.170:3300/0 conn(0x7f38780b0c80 0x7f38780cd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:21.043 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.040+0000 7f387cb05640 1 --2- 172.21.15.169:0/434693999 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f38781597c0 0x7f38780ac200 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:21.043 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.040+0000 7f387cb05640 1 -- 172.21.15.169:0/434693999 >> 172.21.15.169:0/434693999 conn(0x7f3878069930 msgr2=0x7f3878069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:21.044 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.040+0000 7f387cb05640 1 -- 172.21.15.169:0/434693999 shutdown_connections 2023-10-26T01:31:21.044 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.040+0000 7f387cb05640 1 -- 172.21.15.169:0/434693999 wait complete. 2023-10-26T01:31:21.045 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.042+0000 7f387cb05640 1 Processor -- start 2023-10-26T01:31:21.045 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.042+0000 7f387cb05640 1 -- start start 2023-10-26T01:31:21.045 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.043+0000 7f387cb05640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f38780b0c80 0x7f38781f64e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:21.046 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.043+0000 7f387cb05640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f38781593e0 0x7f38780c8f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:21.046 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.043+0000 7f387cb05640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f38781597c0 0x7f3878149a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:21.046 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.043+0000 7f387cb05640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38781463a0 con 0x7f38781597c0 2023-10-26T01:31:21.046 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.043+0000 7f387cb05640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38781465f0 con 0x7f38781593e0 2023-10-26T01:31:21.046 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.043+0000 7f387cb05640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38781468d0 con 0x7f38780b0c80 2023-10-26T01:31:21.047 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.044+0000 7f3876d9d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f38781597c0 0x7f3878149a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:21.047 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.044+0000 7f3876d9d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f38781597c0 0x7f3878149a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:54518/0 (socket says 172.21.15.169:54518) 2023-10-26T01:31:21.047 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.044+0000 7f3876d9d640 1 -- 172.21.15.169:0/932246306 learned_addr learned my addr 172.21.15.169:0/932246306 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:31:21.048 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.044+0000 7f387659c640 1 --2- 172.21.15.169:0/932246306 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f38780b0c80 0x7f38781f64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:21.049 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.044+0000 7f3875d9b640 1 --2- 172.21.15.169:0/932246306 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f38781593e0 0x7f38780c8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:21.049 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.044+0000 7f3876d9d640 1 -- 172.21.15.169:0/932246306 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f38780b0c80 msgr2=0x7f38781f64e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:21.049 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.044+0000 7f3876d9d640 1 --2- 172.21.15.169:0/932246306 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f38780b0c80 0x7f38781f64e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:21.049 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.044+0000 7f3876d9d640 1 -- 172.21.15.169:0/932246306 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f38781593e0 msgr2=0x7f38780c8f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:21.049 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.044+0000 7f3876d9d640 1 --2- 172.21.15.169:0/932246306 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f38781593e0 0x7f38780c8f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:21.050 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.044+0000 7f3876d9d640 1 -- 172.21.15.169:0/932246306 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3868009900 con 0x7f38781597c0 2023-10-26T01:31:21.050 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.045+0000 7f3876d9d640 1 --2- 172.21.15.169:0/932246306 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f38781597c0 0x7f3878149a50 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3870009fd0 tx=0x7f387000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:21.050 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.047+0000 7f386effd640 1 -- 172.21.15.169:0/932246306 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3870011830 con 0x7f38781597c0 2023-10-26T01:31:21.050 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.047+0000 7f386effd640 1 -- 172.21.15.169:0/932246306 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3870011a80 con 0x7f38781597c0 2023-10-26T01:31:21.050 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.047+0000 7f387cb05640 1 -- 172.21.15.169:0/932246306 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3878146bb0 con 0x7f38781597c0 2023-10-26T01:31:21.051 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.047+0000 7f386effd640 1 -- 172.21.15.169:0/932246306 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3870011e70 con 0x7f38781597c0 2023-10-26T01:31:21.051 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.047+0000 7f387cb05640 1 -- 172.21.15.169:0/932246306 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38780c9440 con 0x7f38781597c0 2023-10-26T01:31:21.052 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.048+0000 7f387cb05640 1 -- 172.21.15.169:0/932246306 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38781470d0 con 0x7f38781597c0 2023-10-26T01:31:21.052 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.049+0000 7f386effd640 1 -- 172.21.15.169:0/932246306 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f387001b460 con 0x7f38781597c0 2023-10-26T01:31:21.052 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.050+0000 7f386effd640 1 --2- 172.21.15.169:0/932246306 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f385806ae40 0x7f385806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:21.052 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.050+0000 7f386effd640 1 -- 172.21.15.169:0/932246306 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3870089680 con 0x7f38781597c0 2023-10-26T01:31:21.053 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.051+0000 7f387659c640 1 --2- 172.21.15.169:0/932246306 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f385806ae40 0x7f385806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:21.053 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.052+0000 7f387659c640 1 --2- 172.21.15.169:0/932246306 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f385806ae40 0x7f385806d300 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3860004ea0 tx=0x7f3860009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:21.056 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.055+0000 7f386effd640 1 -- 172.21.15.169:0/932246306 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f38781470d0 con 0x7f38781597c0 2023-10-26T01:31:21.251 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.247+0000 7f387cb05640 1 -- 172.21.15.169:0/932246306 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f38781470d0 con 0x7f38781597c0 2023-10-26T01:31:21.251 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.248+0000 7f386effd640 1 -- 172.21.15.169:0/932246306 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f38781470d0 con 0x7f38781597c0 2023-10-26T01:31:21.253 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 32 2023-10-26T01:31:21.253 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.250+0000 7f387cb05640 1 -- 172.21.15.169:0/932246306 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f385806ae40 msgr2=0x7f385806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:21.254 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.250+0000 7f387cb05640 1 --2- 172.21.15.169:0/932246306 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f385806ae40 0x7f385806d300 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3860004ea0 tx=0x7f3860009040 comp rx=0 tx=0).stop 2023-10-26T01:31:21.254 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.250+0000 7f387cb05640 1 -- 172.21.15.169:0/932246306 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f38781597c0 msgr2=0x7f3878149a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:21.254 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.250+0000 7f387cb05640 1 --2- 172.21.15.169:0/932246306 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f38781597c0 0x7f3878149a50 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3870009fd0 tx=0x7f387000b8e0 comp rx=0 tx=0).stop 2023-10-26T01:31:21.254 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.251+0000 7f387cb05640 1 -- 172.21.15.169:0/932246306 shutdown_connections 2023-10-26T01:31:21.255 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.251+0000 7f387cb05640 1 --2- 172.21.15.169:0/932246306 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f385806ae40 0x7f385806d300 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:21.255 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.251+0000 7f387cb05640 1 --2- 172.21.15.169:0/932246306 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f38781597c0 0x7f3878149a50 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:21.255 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.251+0000 7f387cb05640 1 --2- 172.21.15.169:0/932246306 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f38781593e0 0x7f38780c8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:21.255 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.251+0000 7f387cb05640 1 --2- 172.21.15.169:0/932246306 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f38780b0c80 0x7f38781f64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:21.256 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.251+0000 7f387cb05640 1 -- 172.21.15.169:0/932246306 >> 172.21.15.169:0/932246306 conn(0x7f3878069930 msgr2=0x7f3878154940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:21.256 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.251+0000 7f387cb05640 1 -- 172.21.15.169:0/932246306 shutdown_connections 2023-10-26T01:31:21.256 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.251+0000 7f387cb05640 1 -- 172.21.15.169:0/932246306 wait complete. 2023-10-26T01:31:21.270 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-26T01:31:21.484 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.482+0000 7f9fff283640 1 -- 172.21.15.169:0/318471945 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ff8069f50 msgr2=0x7f9ff8157f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:21.485 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.482+0000 7f9fff283640 1 --2- 172.21.15.169:0/318471945 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ff8069f50 0x7f9ff8157f80 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9ff806e430 tx=0x7f9ff002c080 comp rx=0 tx=0).stop 2023-10-26T01:31:21.485 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.482+0000 7f9fff283640 1 -- 172.21.15.169:0/318471945 shutdown_connections 2023-10-26T01:31:21.485 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.482+0000 7f9fff283640 1 --2- 172.21.15.169:0/318471945 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ff8069f50 0x7f9ff8157f80 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:21.485 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.482+0000 7f9fff283640 1 --2- 172.21.15.169:0/318471945 >> v2:172.21.15.170:3300/0 conn(0x7f9ff815b140 0x7f9ff8069a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:21.486 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.482+0000 7f9fff283640 1 -- 172.21.15.169:0/318471945 >> 172.21.15.169:0/318471945 conn(0x7f9ff80af110 msgr2=0x7f9ff80af520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:21.486 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.483+0000 7f9fff283640 1 -- 172.21.15.169:0/318471945 shutdown_connections 2023-10-26T01:31:21.486 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.483+0000 7f9fff283640 1 -- 172.21.15.169:0/318471945 wait complete. 2023-10-26T01:31:21.488 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.484+0000 7f9fff283640 1 Processor -- start 2023-10-26T01:31:21.488 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.484+0000 7f9fff283640 1 -- start start 2023-10-26T01:31:21.488 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.485+0000 7f9fff283640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9ff8069f50 0x7f9ff8143300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:21.489 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.485+0000 7f9fff283640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9ff806cba0 0x7f9ff80cbbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:21.489 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.485+0000 7f9fff283640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ff815b140 0x7f9ff80cc120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:21.489 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.485+0000 7f9fff283640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff80ca5a0 con 0x7f9ff815b140 2023-10-26T01:31:21.489 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.485+0000 7f9fff283640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff80ca880 con 0x7f9ff8069f50 2023-10-26T01:31:21.489 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.485+0000 7f9fff283640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff80cab60 con 0x7f9ff806cba0 2023-10-26T01:31:21.490 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.485+0000 7f9ffea82640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ff815b140 0x7f9ff80cc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:21.490 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.486+0000 7f9ffea82640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ff815b140 0x7f9ff80cc120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:54540/0 (socket says 172.21.15.169:54540) 2023-10-26T01:31:21.490 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.486+0000 7f9ffea82640 1 -- 172.21.15.169:0/1838672461 learned_addr learned my addr 172.21.15.169:0/1838672461 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:31:21.490 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.486+0000 7f9ffea82640 1 -- 172.21.15.169:0/1838672461 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9ff806cba0 msgr2=0x7f9ff80cbbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:21.490 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.486+0000 7f9ffea82640 1 --2- 172.21.15.169:0/1838672461 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9ff806cba0 0x7f9ff80cbbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:21.491 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.486+0000 7f9ffea82640 1 -- 172.21.15.169:0/1838672461 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9ff8069f50 msgr2=0x7f9ff8143300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:21.491 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.486+0000 7f9ffea82640 1 --2- 172.21.15.169:0/1838672461 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9ff8069f50 0x7f9ff8143300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:21.491 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.486+0000 7f9ffea82640 1 -- 172.21.15.169:0/1838672461 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ff0009840 con 0x7f9ff815b140 2023-10-26T01:31:21.491 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.486+0000 7f9ffea82640 1 --2- 172.21.15.169:0/1838672461 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ff815b140 0x7f9ff80cc120 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f9ff0009ca0 tx=0x7f9ff0004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:21.491 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.486+0000 7f9feeffd640 1 -- 172.21.15.169:0/1838672461 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ff003d430 con 0x7f9ff815b140 2023-10-26T01:31:21.492 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.486+0000 7f9feeffd640 1 -- 172.21.15.169:0/1838672461 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9ff003d680 con 0x7f9ff815b140 2023-10-26T01:31:21.492 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.486+0000 7f9feeffd640 1 -- 172.21.15.169:0/1838672461 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ff003da50 con 0x7f9ff815b140 2023-10-26T01:31:21.492 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.486+0000 7f9fff283640 1 -- 172.21.15.169:0/1838672461 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ff80cae40 con 0x7f9ff815b140 2023-10-26T01:31:21.492 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.487+0000 7f9fff283640 1 -- 172.21.15.169:0/1838672461 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ff8143840 con 0x7f9ff815b140 2023-10-26T01:31:21.493 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.488+0000 7f9fff283640 1 -- 172.21.15.169:0/1838672461 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ff80cb360 con 0x7f9ff815b140 2023-10-26T01:31:21.493 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.489+0000 7f9feeffd640 1 -- 172.21.15.169:0/1838672461 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f9ff003dbb0 con 0x7f9ff815b140 2023-10-26T01:31:21.494 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.489+0000 7f9feeffd640 1 --2- 172.21.15.169:0/1838672461 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9fdc06ae40 0x7f9fdc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:21.494 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.490+0000 7f9feeffd640 1 -- 172.21.15.169:0/1838672461 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9ff00ac650 con 0x7f9ff815b140 2023-10-26T01:31:21.494 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.490+0000 7f9ffe281640 1 --2- 172.21.15.169:0/1838672461 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9fdc06ae40 0x7f9fdc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:21.494 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.491+0000 7f9ffe281640 1 --2- 172.21.15.169:0/1838672461 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9fdc06ae40 0x7f9fdc06d300 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9ff40046b0 tx=0x7f9ff4007a70 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:21.495 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.494+0000 7f9feeffd640 1 -- 172.21.15.169:0/1838672461 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9ff80cb360 con 0x7f9ff815b140 2023-10-26T01:31:21.688 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.686+0000 7f9fff283640 1 -- 172.21.15.169:0/1838672461 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9ff80cb360 con 0x7f9ff815b140 2023-10-26T01:31:21.688 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.687+0000 7f9feeffd640 1 -- 172.21.15.169:0/1838672461 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9ff80cb360 con 0x7f9ff815b140 2023-10-26T01:31:21.689 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 64 2023-10-26T01:31:21.692 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.690+0000 7f9fff283640 1 -- 172.21.15.169:0/1838672461 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9fdc06ae40 msgr2=0x7f9fdc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:21.692 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.690+0000 7f9fff283640 1 --2- 172.21.15.169:0/1838672461 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9fdc06ae40 0x7f9fdc06d300 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9ff40046b0 tx=0x7f9ff4007a70 comp rx=0 tx=0).stop 2023-10-26T01:31:21.692 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.690+0000 7f9fff283640 1 -- 172.21.15.169:0/1838672461 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ff815b140 msgr2=0x7f9ff80cc120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:21.692 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.690+0000 7f9fff283640 1 --2- 172.21.15.169:0/1838672461 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ff815b140 0x7f9ff80cc120 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f9ff0009ca0 tx=0x7f9ff0004340 comp rx=0 tx=0).stop 2023-10-26T01:31:21.692 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.690+0000 7f9fff283640 1 -- 172.21.15.169:0/1838672461 shutdown_connections 2023-10-26T01:31:21.693 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.690+0000 7f9fff283640 1 --2- 172.21.15.169:0/1838672461 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9fdc06ae40 0x7f9fdc06d300 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:21.693 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.690+0000 7f9fff283640 1 --2- 172.21.15.169:0/1838672461 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ff815b140 0x7f9ff80cc120 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:21.693 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.690+0000 7f9fff283640 1 --2- 172.21.15.169:0/1838672461 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9ff806cba0 0x7f9ff80cbbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:21.693 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.690+0000 7f9fff283640 1 --2- 172.21.15.169:0/1838672461 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9ff8069f50 0x7f9ff8143300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:21.693 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.690+0000 7f9fff283640 1 -- 172.21.15.169:0/1838672461 >> 172.21.15.169:0/1838672461 conn(0x7f9ff80af110 msgr2=0x7f9ff80b0300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:21.694 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.690+0000 7f9fff283640 1 -- 172.21.15.169:0/1838672461 shutdown_connections 2023-10-26T01:31:21.694 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.690+0000 7f9fff283640 1 -- 172.21.15.169:0/1838672461 wait complete. 2023-10-26T01:31:21.710 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:31:21.921 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.919+0000 7f8da7fff640 1 -- 172.21.15.169:0/1830229137 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8d88004200 msgr2=0x7f8d880047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:21.921 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.919+0000 7f8da7fff640 1 --2- 172.21.15.169:0/1830229137 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8d88004200 0x7f8d880047c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8d9800c260 tx=0x7f8d880057f0 comp rx=0 tx=0).stop 2023-10-26T01:31:21.921 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.919+0000 7f8da77fe640 1 -- 172.21.15.169:0/1830229137 reap_dead start 2023-10-26T01:31:21.921 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.919+0000 7f8da7fff640 1 -- 172.21.15.169:0/1830229137 shutdown_connections 2023-10-26T01:31:21.922 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.919+0000 7f8da7fff640 1 -- 172.21.15.169:0/1830229137 >> 172.21.15.169:0/1830229137 conn(0x7f8da80aedd0 msgr2=0x7f8da80af1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:21.922 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.919+0000 7f8da7fff640 1 -- 172.21.15.169:0/1830229137 shutdown_connections 2023-10-26T01:31:21.922 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.919+0000 7f8da7fff640 1 -- 172.21.15.169:0/1830229137 wait complete. 2023-10-26T01:31:21.923 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.920+0000 7f8da7fff640 1 Processor -- start 2023-10-26T01:31:21.924 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.921+0000 7f8da7fff640 1 -- start start 2023-10-26T01:31:21.924 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.921+0000 7f8da7fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d88004200 0x7f8da8154040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:21.924 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.921+0000 7f8da7fff640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f8da8153b80 0x7f8da81478f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:21.924 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.921+0000 7f8da7fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8da806e8f0 0x7f8da806ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:21.925 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.921+0000 7f8da7fff640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8da806f2b0 con 0x7f8da806e8f0 2023-10-26T01:31:21.925 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.921+0000 7f8da7fff640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8da8147e30 con 0x7f8d88004200 2023-10-26T01:31:21.925 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.921+0000 7f8da7fff640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8da80aac60 con 0x7f8da8153b80 2023-10-26T01:31:21.925 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.922+0000 7f8da77fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8da806e8f0 0x7f8da806ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:21.925 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.922+0000 7f8da77fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8da806e8f0 0x7f8da806ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:54552/0 (socket says 172.21.15.169:54552) 2023-10-26T01:31:21.926 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.922+0000 7f8da77fe640 1 -- 172.21.15.169:0/3192892966 learned_addr learned my addr 172.21.15.169:0/3192892966 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:31:21.926 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.922+0000 7f8da77fe640 1 -- 172.21.15.169:0/3192892966 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f8da8153b80 msgr2=0x7f8da81478f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:21.926 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.922+0000 7f8da77fe640 1 --2- 172.21.15.169:0/3192892966 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f8da8153b80 0x7f8da81478f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:21.926 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.922+0000 7f8da6ffd640 1 --2- 172.21.15.169:0/3192892966 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d88004200 0x7f8da8154040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:21.926 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.922+0000 7f8da77fe640 1 -- 172.21.15.169:0/3192892966 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d88004200 msgr2=0x7f8da8154040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:21.927 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.922+0000 7f8da77fe640 1 --2- 172.21.15.169:0/3192892966 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d88004200 0x7f8da8154040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:21.927 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.922+0000 7f8da77fe640 1 -- 172.21.15.169:0/3192892966 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8da80aaf40 con 0x7f8da806e8f0 2023-10-26T01:31:21.927 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.922+0000 7f8da77fe640 1 --2- 172.21.15.169:0/3192892966 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8da806e8f0 0x7f8da806ed70 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8da8071eb0 tx=0x7f8da000ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:21.928 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.922+0000 7f8da4ff9640 1 -- 172.21.15.169:0/3192892966 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da0016070 con 0x7f8da806e8f0 2023-10-26T01:31:21.929 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.923+0000 7f8da4ff9640 1 -- 172.21.15.169:0/3192892966 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8da0004b20 con 0x7f8da806e8f0 2023-10-26T01:31:21.929 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.923+0000 7f8da7fff640 1 -- 172.21.15.169:0/3192892966 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8da80ab220 con 0x7f8da806e8f0 2023-10-26T01:31:21.929 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.923+0000 7f8da4ff9640 1 -- 172.21.15.169:0/3192892966 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da0003990 con 0x7f8da806e8f0 2023-10-26T01:31:21.929 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.923+0000 7f8da7fff640 1 -- 172.21.15.169:0/3192892966 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8da80ab7e0 con 0x7f8da806e8f0 2023-10-26T01:31:21.929 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.924+0000 7f8da7fff640 1 -- 172.21.15.169:0/3192892966 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8da8073c10 con 0x7f8da806e8f0 2023-10-26T01:31:21.930 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.925+0000 7f8da4ff9640 1 -- 172.21.15.169:0/3192892966 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f8da0007980 con 0x7f8da806e8f0 2023-10-26T01:31:21.930 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.925+0000 7f8da4ff9640 1 --2- 172.21.15.169:0/3192892966 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f8d8c06ae70 0x7f8d8c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:21.930 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.926+0000 7f8da4ff9640 1 -- 172.21.15.169:0/3192892966 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8da0089150 con 0x7f8da806e8f0 2023-10-26T01:31:21.930 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.926+0000 7f8da6ffd640 1 --2- 172.21.15.169:0/3192892966 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f8d8c06ae70 0x7f8d8c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:21.930 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.927+0000 7f8da6ffd640 1 --2- 172.21.15.169:0/3192892966 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f8d8c06ae70 0x7f8d8c06d330 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8d98034000 tx=0x7f8d98039040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:21.931 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:21.930+0000 7f8da4ff9640 1 -- 172.21.15.169:0/3192892966 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8da00541f0 con 0x7f8da806e8f0 2023-10-26T01:31:22.125 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.124+0000 7f8da7fff640 1 -- 172.21.15.169:0/3192892966 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8da80ca0f0 con 0x7f8da806e8f0 2023-10-26T01:31:22.127 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.126+0000 7f8da4ff9640 1 -- 172.21.15.169:0/3192892966 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16553 (secure 0 0 0) 0x7f8da0057810 con 0x7f8da806e8f0 2023-10-26T01:31:22.128 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:31:22.128 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":21,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:30:43.078706+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-26T01:30:39.437631+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-26T01:30:40.561774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:31:22.130 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.128+0000 7f8da7fff640 1 -- 172.21.15.169:0/3192892966 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f8d8c06ae70 msgr2=0x7f8d8c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:22.130 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.128+0000 7f8da7fff640 1 --2- 172.21.15.169:0/3192892966 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f8d8c06ae70 0x7f8d8c06d330 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8d98034000 tx=0x7f8d98039040 comp rx=0 tx=0).stop 2023-10-26T01:31:22.131 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.128+0000 7f8da7fff640 1 -- 172.21.15.169:0/3192892966 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8da806e8f0 msgr2=0x7f8da806ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:22.131 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.128+0000 7f8da7fff640 1 --2- 172.21.15.169:0/3192892966 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8da806e8f0 0x7f8da806ed70 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8da8071eb0 tx=0x7f8da000ccd0 comp rx=0 tx=0).stop 2023-10-26T01:31:22.131 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.128+0000 7f8da7fff640 1 -- 172.21.15.169:0/3192892966 shutdown_connections 2023-10-26T01:31:22.131 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.128+0000 7f8da7fff640 1 --2- 172.21.15.169:0/3192892966 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f8d8c06ae70 0x7f8d8c06d330 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:22.131 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.128+0000 7f8da7fff640 1 --2- 172.21.15.169:0/3192892966 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8da806e8f0 0x7f8da806ed70 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:22.132 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.128+0000 7f8da7fff640 1 --2- 172.21.15.169:0/3192892966 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f8da8153b80 0x7f8da81478f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:22.132 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.128+0000 7f8da7fff640 1 --2- 172.21.15.169:0/3192892966 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d88004200 0x7f8da8154040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:22.132 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.128+0000 7f8da7fff640 1 -- 172.21.15.169:0/3192892966 >> 172.21.15.169:0/3192892966 conn(0x7f8da80aedd0 msgr2=0x7f8da81546c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:22.132 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.128+0000 7f8da7fff640 1 -- 172.21.15.169:0/3192892966 shutdown_connections 2023-10-26T01:31:22.132 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.128+0000 7f8da7fff640 1 -- 172.21.15.169:0/3192892966 wait complete. 2023-10-26T01:31:22.149 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T01:30:32.200808+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-26T01:30:39.437631+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-26T01:30:40.561774+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T01:31:22.150 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T01:31:22.367 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.365+0000 7f960abd7640 1 -- 172.21.15.169:0/879932542 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95ec0047c0 msgr2=0x7f95ec004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:22.367 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.365+0000 7f960abd7640 1 --2- 172.21.15.169:0/879932542 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95ec0047c0 0x7f95ec004c40 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f95f8006fd0 tx=0x7f95ec0057f0 comp rx=0 tx=0).stop 2023-10-26T01:31:22.368 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.365+0000 7f960a3d6640 1 -- 172.21.15.169:0/879932542 reap_dead start 2023-10-26T01:31:22.368 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.365+0000 7f960abd7640 1 -- 172.21.15.169:0/879932542 shutdown_connections 2023-10-26T01:31:22.368 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.365+0000 7f960abd7640 1 -- 172.21.15.169:0/879932542 >> 172.21.15.169:0/879932542 conn(0x7f9604065490 msgr2=0x7f96040658a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:22.368 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.365+0000 7f960abd7640 1 -- 172.21.15.169:0/879932542 shutdown_connections 2023-10-26T01:31:22.368 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.365+0000 7f960abd7640 1 -- 172.21.15.169:0/879932542 wait complete. 2023-10-26T01:31:22.369 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.367+0000 7f960abd7640 1 Processor -- start 2023-10-26T01:31:22.369 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.367+0000 7f960abd7640 1 -- start start 2023-10-26T01:31:22.371 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.368+0000 7f960abd7640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95ec0047c0 0x7f96041f4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:22.371 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.368+0000 7f960abd7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96040ad2f0 0x7f96040cd370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:22.371 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.368+0000 7f960abd7640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f96040ad770 0x7f96040cd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:22.372 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.368+0000 7f960abd7640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96040ae950 con 0x7f95ec0047c0 2023-10-26T01:31:22.372 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.368+0000 7f960abd7640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96040adcd0 con 0x7f96040ad2f0 2023-10-26T01:31:22.372 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.368+0000 7f960abd7640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96040adfb0 con 0x7f96040ad770 2023-10-26T01:31:22.372 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.368+0000 7f9609bd5640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95ec0047c0 0x7f96041f4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:22.372 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.369+0000 7f9609bd5640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95ec0047c0 0x7f96041f4850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:54566/0 (socket says 172.21.15.169:54566) 2023-10-26T01:31:22.373 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.369+0000 7f9609bd5640 1 -- 172.21.15.169:0/2999162712 learned_addr learned my addr 172.21.15.169:0/2999162712 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:31:22.373 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.369+0000 7f96093d4640 1 --2- 172.21.15.169:0/2999162712 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96040ad2f0 0x7f96040cd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:22.373 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.369+0000 7f9609bd5640 1 -- 172.21.15.169:0/2999162712 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f96040ad770 msgr2=0x7f96040cd8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:22.373 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.369+0000 7f9609bd5640 1 --2- 172.21.15.169:0/2999162712 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f96040ad770 0x7f96040cd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:22.373 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.369+0000 7f9609bd5640 1 -- 172.21.15.169:0/2999162712 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96040ad2f0 msgr2=0x7f96040cd370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:22.374 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.369+0000 7f9609bd5640 1 --2- 172.21.15.169:0/2999162712 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96040ad2f0 0x7f96040cd370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:22.374 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.369+0000 7f9609bd5640 1 -- 172.21.15.169:0/2999162712 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96040ae290 con 0x7f95ec0047c0 2023-10-26T01:31:22.374 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.369+0000 7f9609bd5640 1 --2- 172.21.15.169:0/2999162712 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95ec0047c0 0x7f96041f4850 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f95f400fb40 tx=0x7f95f400d2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:22.374 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.369+0000 7f96027fc640 1 -- 172.21.15.169:0/2999162712 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f4004b20 con 0x7f95ec0047c0 2023-10-26T01:31:22.374 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.369+0000 7f96027fc640 1 -- 172.21.15.169:0/2999162712 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f95f4004d70 con 0x7f95ec0047c0 2023-10-26T01:31:22.375 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.369+0000 7f96027fc640 1 -- 172.21.15.169:0/2999162712 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f4003bd0 con 0x7f95ec0047c0 2023-10-26T01:31:22.375 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.370+0000 7f960abd7640 1 -- 172.21.15.169:0/2999162712 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96040ae570 con 0x7f95ec0047c0 2023-10-26T01:31:22.375 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.370+0000 7f960abd7640 1 -- 172.21.15.169:0/2999162712 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96040ab8d0 con 0x7f95ec0047c0 2023-10-26T01:31:22.376 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.371+0000 7f960abd7640 1 -- 172.21.15.169:0/2999162712 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96041f1920 con 0x7f95ec0047c0 2023-10-26T01:31:22.376 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.372+0000 7f96027fc640 1 -- 172.21.15.169:0/2999162712 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f95f4003d30 con 0x7f95ec0047c0 2023-10-26T01:31:22.376 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.373+0000 7f96027fc640 1 --2- 172.21.15.169:0/2999162712 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f95f006ae70 0x7f95f006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:22.377 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.373+0000 7f96027fc640 1 -- 172.21.15.169:0/2999162712 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f95f4015070 con 0x7f95ec0047c0 2023-10-26T01:31:22.377 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.373+0000 7f96093d4640 1 --2- 172.21.15.169:0/2999162712 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f95f006ae70 0x7f95f006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:22.377 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.374+0000 7f96093d4640 1 --2- 172.21.15.169:0/2999162712 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f95f006ae70 0x7f95f006d330 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f95f8034000 tx=0x7f95f802e000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:22.378 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.377+0000 7f96027fc640 1 -- 172.21.15.169:0/2999162712 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f95f4055090 con 0x7f95ec0047c0 2023-10-26T01:31:22.565 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.563+0000 7f960abd7640 1 -- 172.21.15.169:0/2999162712 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9604073c10 con 0x7f95ec0047c0 2023-10-26T01:31:22.566 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.564+0000 7f96027fc640 1 -- 172.21.15.169:0/2999162712 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f95f401d020 con 0x7f95ec0047c0 2023-10-26T01:31:22.567 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 1 2023-10-26T01:31:22.569 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.567+0000 7f960abd7640 1 -- 172.21.15.169:0/2999162712 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f95f006ae70 msgr2=0x7f95f006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:22.570 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.567+0000 7f960abd7640 1 --2- 172.21.15.169:0/2999162712 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f95f006ae70 0x7f95f006d330 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f95f8034000 tx=0x7f95f802e000 comp rx=0 tx=0).stop 2023-10-26T01:31:22.570 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.567+0000 7f960abd7640 1 -- 172.21.15.169:0/2999162712 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95ec0047c0 msgr2=0x7f96041f4850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:22.570 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.567+0000 7f960abd7640 1 --2- 172.21.15.169:0/2999162712 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95ec0047c0 0x7f96041f4850 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f95f400fb40 tx=0x7f95f400d2e0 comp rx=0 tx=0).stop 2023-10-26T01:31:22.571 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.567+0000 7f960abd7640 1 -- 172.21.15.169:0/2999162712 shutdown_connections 2023-10-26T01:31:22.571 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.567+0000 7f960abd7640 1 --2- 172.21.15.169:0/2999162712 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f95f006ae70 0x7f95f006d330 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:22.571 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.567+0000 7f960abd7640 1 --2- 172.21.15.169:0/2999162712 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f96040ad770 0x7f96040cd8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:22.572 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.567+0000 7f960abd7640 1 --2- 172.21.15.169:0/2999162712 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96040ad2f0 0x7f96040cd370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:22.572 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.567+0000 7f960abd7640 1 --2- 172.21.15.169:0/2999162712 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95ec0047c0 0x7f96041f4850 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:22.573 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.567+0000 7f960abd7640 1 -- 172.21.15.169:0/2999162712 >> 172.21.15.169:0/2999162712 conn(0x7f9604065490 msgr2=0x7f960406aeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:22.573 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.567+0000 7f960abd7640 1 -- 172.21.15.169:0/2999162712 shutdown_connections 2023-10-26T01:31:22.573 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.567+0000 7f960abd7640 1 -- 172.21.15.169:0/2999162712 wait complete. 2023-10-26T01:31:22.588 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-26T01:31:22.800 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.798+0000 7f5486c60640 1 -- 172.21.15.169:0/3217824113 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f548015ece0 msgr2=0x7f54800b0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:22.800 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.798+0000 7f5486c60640 1 --2- 172.21.15.169:0/3217824113 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f548015ece0 0x7f54800b0170 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5470005fd0 tx=0x7f547002c0e0 comp rx=0 tx=0).stop 2023-10-26T01:31:22.801 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.798+0000 7f5486c60640 1 -- 172.21.15.169:0/3217824113 shutdown_connections 2023-10-26T01:31:22.801 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.798+0000 7f5486c60640 1 --2- 172.21.15.169:0/3217824113 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f54800b0720 0x7f54800d0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:22.801 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.798+0000 7f5486c60640 1 --2- 172.21.15.169:0/3217824113 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f548015ece0 0x7f54800b0170 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:22.801 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.798+0000 7f5486c60640 1 -- 172.21.15.169:0/3217824113 >> 172.21.15.169:0/3217824113 conn(0x7f54800d1370 msgr2=0x7f54800d1780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:22.801 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.798+0000 7f5486c60640 1 -- 172.21.15.169:0/3217824113 shutdown_connections 2023-10-26T01:31:22.802 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.798+0000 7f5486c60640 1 -- 172.21.15.169:0/3217824113 wait complete. 2023-10-26T01:31:22.803 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.800+0000 7f5486c60640 1 Processor -- start 2023-10-26T01:31:22.803 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.800+0000 7f5486c60640 1 -- start start 2023-10-26T01:31:22.803 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.801+0000 7f5486c60640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5480065490 0x7f54800b38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:22.803 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.801+0000 7f5486c60640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f54800b0720 0x7f5480149a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:22.804 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.801+0000 7f5486c60640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f548015ece0 0x7f5480154710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:22.804 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.801+0000 7f5486c60640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5480149f60 con 0x7f54800b0720 2023-10-26T01:31:22.804 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.801+0000 7f5486c60640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54800b2820 con 0x7f548015ece0 2023-10-26T01:31:22.804 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.801+0000 7f5486c60640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54800b2b00 con 0x7f5480065490 2023-10-26T01:31:22.804 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.802+0000 7f548545d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f54800b0720 0x7f5480149a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:22.805 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.802+0000 7f548545d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f54800b0720 0x7f5480149a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:54584/0 (socket says 172.21.15.169:54584) 2023-10-26T01:31:22.805 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.802+0000 7f548545d640 1 -- 172.21.15.169:0/2512051812 learned_addr learned my addr 172.21.15.169:0/2512051812 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:31:22.807 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.802+0000 7f5485c5e640 1 --2- 172.21.15.169:0/2512051812 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5480065490 0x7f54800b38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:22.807 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.802+0000 7f548645f640 1 --2- 172.21.15.169:0/2512051812 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f548015ece0 0x7f5480154710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:22.807 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.802+0000 7f548545d640 1 -- 172.21.15.169:0/2512051812 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5480065490 msgr2=0x7f54800b38c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:22.807 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.802+0000 7f548545d640 1 --2- 172.21.15.169:0/2512051812 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5480065490 0x7f54800b38c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:22.807 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.802+0000 7f548545d640 1 -- 172.21.15.169:0/2512051812 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f548015ece0 msgr2=0x7f5480154710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:22.808 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.802+0000 7f548545d640 1 --2- 172.21.15.169:0/2512051812 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f548015ece0 0x7f5480154710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:22.808 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.802+0000 7f548545d640 1 -- 172.21.15.169:0/2512051812 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5470009840 con 0x7f54800b0720 2023-10-26T01:31:22.808 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.802+0000 7f548645f640 1 --2- 172.21.15.169:0/2512051812 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f548015ece0 0x7f5480154710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:31:22.808 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.802+0000 7f548545d640 1 --2- 172.21.15.169:0/2512051812 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f54800b0720 0x7f5480149a20 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f547400d770 tx=0x7f547400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:22.808 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.802+0000 7f547e7fc640 1 -- 172.21.15.169:0/2512051812 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5474009c30 con 0x7f54800b0720 2023-10-26T01:31:22.809 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.803+0000 7f547e7fc640 1 -- 172.21.15.169:0/2512051812 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f547400de80 con 0x7f54800b0720 2023-10-26T01:31:22.809 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.803+0000 7f547e7fc640 1 -- 172.21.15.169:0/2512051812 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54740106c0 con 0x7f54800b0720 2023-10-26T01:31:22.809 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.803+0000 7f5486c60640 1 -- 172.21.15.169:0/2512051812 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54800b2de0 con 0x7f54800b0720 2023-10-26T01:31:22.809 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.803+0000 7f5486c60640 1 -- 172.21.15.169:0/2512051812 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54800b33a0 con 0x7f54800b0720 2023-10-26T01:31:22.809 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.804+0000 7f5486c60640 1 -- 172.21.15.169:0/2512051812 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54800b3560 con 0x7f54800b0720 2023-10-26T01:31:22.810 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.805+0000 7f547e7fc640 1 -- 172.21.15.169:0/2512051812 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f5474010820 con 0x7f54800b0720 2023-10-26T01:31:22.810 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.806+0000 7f547e7fc640 1 --2- 172.21.15.169:0/2512051812 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f546c06ae40 0x7f546c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:22.811 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.806+0000 7f547e7fc640 1 -- 172.21.15.169:0/2512051812 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5474089c50 con 0x7f54800b0720 2023-10-26T01:31:22.811 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.806+0000 7f5485c5e640 1 --2- 172.21.15.169:0/2512051812 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f546c06ae40 0x7f546c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:22.811 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.807+0000 7f5485c5e640 1 --2- 172.21.15.169:0/2512051812 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f546c06ae40 0x7f546c06d300 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5470035000 tx=0x7f5470004ab0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:22.811 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.810+0000 7f547e7fc640 1 -- 172.21.15.169:0/2512051812 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5474054c60 con 0x7f54800b0720 2023-10-26T01:31:22.997 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.995+0000 7f5486c60640 1 -- 172.21.15.169:0/2512051812 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5480073c10 con 0x7f54800b0720 2023-10-26T01:31:22.998 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.996+0000 7f547e7fc640 1 -- 172.21.15.169:0/2512051812 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5474021020 con 0x7f54800b0720 2023-10-26T01:31:22.998 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 32 2023-10-26T01:31:23.001 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.999+0000 7f5486c60640 1 -- 172.21.15.169:0/2512051812 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f546c06ae40 msgr2=0x7f546c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:23.001 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.999+0000 7f5486c60640 1 --2- 172.21.15.169:0/2512051812 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f546c06ae40 0x7f546c06d300 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5470035000 tx=0x7f5470004ab0 comp rx=0 tx=0).stop 2023-10-26T01:31:23.001 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.999+0000 7f5486c60640 1 -- 172.21.15.169:0/2512051812 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f54800b0720 msgr2=0x7f5480149a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:23.001 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.999+0000 7f5486c60640 1 --2- 172.21.15.169:0/2512051812 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f54800b0720 0x7f5480149a20 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f547400d770 tx=0x7f547400dc30 comp rx=0 tx=0).stop 2023-10-26T01:31:23.002 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.999+0000 7f5486c60640 1 -- 172.21.15.169:0/2512051812 shutdown_connections 2023-10-26T01:31:23.002 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.999+0000 7f5486c60640 1 --2- 172.21.15.169:0/2512051812 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f546c06ae40 0x7f546c06d300 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:23.002 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.999+0000 7f5486c60640 1 --2- 172.21.15.169:0/2512051812 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f54800b0720 0x7f5480149a20 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:23.003 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.999+0000 7f5486c60640 1 --2- 172.21.15.169:0/2512051812 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f548015ece0 0x7f5480154710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:23.003 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.999+0000 7f5486c60640 1 --2- 172.21.15.169:0/2512051812 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5480065490 0x7f54800b38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:23.003 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:22.999+0000 7f5486c60640 1 -- 172.21.15.169:0/2512051812 >> 172.21.15.169:0/2512051812 conn(0x7f54800d1370 msgr2=0x7f54800c64c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:23.003 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:23.000+0000 7f5486c60640 1 -- 172.21.15.169:0/2512051812 shutdown_connections 2023-10-26T01:31:23.003 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:23.000+0000 7f5486c60640 1 -- 172.21.15.169:0/2512051812 wait complete. 2023-10-26T01:31:23.019 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-26T01:31:23.236 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:23.234+0000 7f680884a640 1 -- 172.21.15.169:0/2327187178 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f67f0004d00 msgr2=0x7f67f0005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:23.236 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:23.234+0000 7f680884a640 1 --2- 172.21.15.169:0/2327187178 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f67f0004d00 0x7f67f0005180 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f67fc00af30 tx=0x7f67f00057f0 comp rx=0 tx=0).stop 2023-10-26T01:31:23.236 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:23.234+0000 7f680359e640 1 -- 172.21.15.169:0/2327187178 reap_dead start 2023-10-26T01:31:23.237 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:23.234+0000 7f680884a640 1 -- 172.21.15.169:0/2327187178 shutdown_connections 2023-10-26T01:31:23.237 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:23.234+0000 7f680884a640 1 -- 172.21.15.169:0/2327187178 >> 172.21.15.169:0/2327187178 conn(0x7f68040af110 msgr2=0x7f68040af520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:23.237 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:23.234+0000 7f680884a640 1 -- 172.21.15.169:0/2327187178 shutdown_connections 2023-10-26T01:31:23.237 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:23.234+0000 7f680884a640 1 -- 172.21.15.169:0/2327187178 wait complete. 2023-10-26T01:31:23.238 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:23.236+0000 7f680884a640 1 Processor -- start 2023-10-26T01:31:23.239 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:23.236+0000 7f680884a640 1 -- start start 2023-10-26T01:31:23.239 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:23.237+0000 7f680884a640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f67f0004d00 0x7f68041465c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:23.239 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:23.237+0000 7f680884a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f680406eab0 0x7f68040971c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:23.239 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:23.237+0000 7f680884a640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f680406ffe0 0x7f6804156620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:23.239 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:23.237+0000 7f680884a640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68040709d0 con 0x7f680406ffe0 2023-10-26T01:31:23.240 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:23.237+0000 7f680884a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f680406ef40 con 0x7f680406eab0 2023-10-26T01:31:23.240 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:23.237+0000 7f680884a640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f680406f220 con 0x7f67f0004d00 2023-10-26T01:31:23.240 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:23.237+0000 7f680359e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f680406ffe0 0x7f6804156620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:23.240 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:23.237+0000 7f680359e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f680406ffe0 0x7f6804156620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:54608/0 (socket says 172.21.15.169:54608) 2023-10-26T01:31:23.240 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:23.237+0000 7f680359e640 1 -- 172.21.15.169:0/373714075 learned_addr learned my addr 172.21.15.169:0/373714075 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:31:23.242 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:23.237+0000 7f680359e640 1 -- 172.21.15.169:0/373714075 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f67f0004d00 msgr2=0x7f68041465c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:23.242 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:23.237+0000 7f680359e640 1 --2- 172.21.15.169:0/373714075 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f67f0004d00 0x7f68041465c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:23.242 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:23.237+0000 7f680359e640 1 -- 172.21.15.169:0/373714075 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f680406eab0 msgr2=0x7f68040971c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:23.242 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:23.237+0000 7f680359e640 1 --2- 172.21.15.169:0/373714075 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f680406eab0 0x7f68040971c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:23.243 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:23.237+0000 7f680359e640 1 -- 172.21.15.169:0/373714075 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f680406f500 con 0x7f680406ffe0 2023-10-26T01:31:23.243 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:23.238+0000 7f680359e640 1 --2- 172.21.15.169:0/373714075 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f680406ffe0 0x7f6804156620 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f67fc02db90 tx=0x7f67fc035900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:23.243 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:23.238+0000 7f67fb7fe640 1 -- 172.21.15.169:0/373714075 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67fc035e80 con 0x7f680406ffe0 2023-10-26T01:31:23.243 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:23.238+0000 7f67fb7fe640 1 -- 172.21.15.169:0/373714075 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f67fc003970 con 0x7f680406ffe0 2023-10-26T01:31:23.243 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:23.238+0000 7f67fb7fe640 1 -- 172.21.15.169:0/373714075 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67fc003d60 con 0x7f680406ffe0 2023-10-26T01:31:23.244 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:23.238+0000 7f680884a640 1 -- 172.21.15.169:0/373714075 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f680406f7e0 con 0x7f680406ffe0 2023-10-26T01:31:23.244 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:23.238+0000 7f680884a640 1 -- 172.21.15.169:0/373714075 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6804097700 con 0x7f680406ffe0 2023-10-26T01:31:23.244 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:23.239+0000 7f680884a640 1 -- 172.21.15.169:0/373714075 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f680406fd00 con 0x7f680406ffe0 2023-10-26T01:31:23.244 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:23.240+0000 7f67fb7fe640 1 -- 172.21.15.169:0/373714075 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f67fc004030 con 0x7f680406ffe0 2023-10-26T01:31:23.245 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:23.241+0000 7f67fb7fe640 1 --2- 172.21.15.169:0/373714075 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f67e406ae40 0x7f67e406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:23.246 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:23.241+0000 7f67fb7fe640 1 -- 172.21.15.169:0/373714075 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f67fc0ac0a0 con 0x7f680406ffe0 2023-10-26T01:31:23.246 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:23.242+0000 7f6802d9d640 1 --2- 172.21.15.169:0/373714075 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f67e406ae40 0x7f67e406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:23.246 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:23.243+0000 7f6802d9d640 1 --2- 172.21.15.169:0/373714075 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f67e406ae40 0x7f67e406d300 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f67f4001200 tx=0x7f67f4001450 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:23.248 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:23.246+0000 7f67fb7fe640 1 -- 172.21.15.169:0/373714075 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f680406fd00 con 0x7f680406ffe0 2023-10-26T01:31:23.425 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:23.423+0000 7f680884a640 1 -- 172.21.15.169:0/373714075 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f680406fd00 con 0x7f680406ffe0 2023-10-26T01:31:23.443 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:23.424+0000 7f67fb7fe640 1 -- 172.21.15.169:0/373714075 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f680406fd00 con 0x7f680406ffe0 2023-10-26T01:31:23.444 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 64 2023-10-26T01:31:23.444 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:23.427+0000 7f680884a640 1 -- 172.21.15.169:0/373714075 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f67e406ae40 msgr2=0x7f67e406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:23.445 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:23.427+0000 7f680884a640 1 --2- 172.21.15.169:0/373714075 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f67e406ae40 0x7f67e406d300 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f67f4001200 tx=0x7f67f4001450 comp rx=0 tx=0).stop 2023-10-26T01:31:23.445 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:23.427+0000 7f680884a640 1 -- 172.21.15.169:0/373714075 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f680406ffe0 msgr2=0x7f6804156620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:23.445 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:23.427+0000 7f680884a640 1 --2- 172.21.15.169:0/373714075 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f680406ffe0 0x7f6804156620 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f67fc02db90 tx=0x7f67fc035900 comp rx=0 tx=0).stop 2023-10-26T01:31:23.445 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:23.427+0000 7f680884a640 1 -- 172.21.15.169:0/373714075 shutdown_connections 2023-10-26T01:31:23.445 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:23.427+0000 7f680884a640 1 --2- 172.21.15.169:0/373714075 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f67e406ae40 0x7f67e406d300 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:23.446 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:23.427+0000 7f680884a640 1 --2- 172.21.15.169:0/373714075 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f680406ffe0 0x7f6804156620 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:23.446 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:23.427+0000 7f680884a640 1 --2- 172.21.15.169:0/373714075 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f680406eab0 0x7f68040971c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:23.446 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:23.427+0000 7f680884a640 1 --2- 172.21.15.169:0/373714075 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f67f0004d00 0x7f68041465c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:23.446 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:23.427+0000 7f680884a640 1 -- 172.21.15.169:0/373714075 >> 172.21.15.169:0/373714075 conn(0x7f68040af110 msgr2=0x7f6804151520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:23.446 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:23.427+0000 7f680884a640 1 -- 172.21.15.169:0/373714075 shutdown_connections 2023-10-26T01:31:23.447 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:23.427+0000 7f680884a640 1 -- 172.21.15.169:0/373714075 wait complete. 2023-10-26T01:31:23.448 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:31:23.655 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:23.653+0000 7f981cb6e640 1 -- 172.21.15.169:0/2161377408 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98180c9970 msgr2=0x7f9818098a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:23.655 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:23.653+0000 7f981cb6e640 1 --2- 172.21.15.169:0/2161377408 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98180c9970 0x7f9818098a60 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9800009830 tx=0x7f980002bd60 comp rx=0 tx=0).stop 2023-10-26T01:31:23.656 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:23.653+0000 7f981cb6e640 1 -- 172.21.15.169:0/2161377408 shutdown_connections 2023-10-26T01:31:23.656 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:23.653+0000 7f981cb6e640 1 --2- 172.21.15.169:0/2161377408 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98180c9970 0x7f9818098a60 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:23.656 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:23.653+0000 7f981cb6e640 1 -- 172.21.15.169:0/2161377408 >> 172.21.15.169:0/2161377408 conn(0x7f98180aedd0 msgr2=0x7f98180af1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:23.656 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:23.653+0000 7f981cb6e640 1 -- 172.21.15.169:0/2161377408 shutdown_connections 2023-10-26T01:31:23.656 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:23.653+0000 7f981cb6e640 1 -- 172.21.15.169:0/2161377408 wait complete. 2023-10-26T01:31:23.658 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:23.655+0000 7f981cb6e640 1 Processor -- start 2023-10-26T01:31:23.658 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:23.655+0000 7f981cb6e640 1 -- start start 2023-10-26T01:31:23.658 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:23.656+0000 7f981cb6e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f981809a5c0 0x7f981806f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:23.658 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:23.656+0000 7f981cb6e640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f98180ac6d0 0x7f98181f25c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:23.658 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:23.656+0000 7f981cb6e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f98180c9970 0x7f98180a9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:23.659 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:23.656+0000 7f981cb6e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98180707b0 con 0x7f981809a5c0 2023-10-26T01:31:23.659 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:23.656+0000 7f981cb6e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9818070a90 con 0x7f98180c9970 2023-10-26T01:31:23.659 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:23.656+0000 7f981cb6e640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f981806fb30 con 0x7f98180ac6d0 2023-10-26T01:31:23.659 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:23.656+0000 7f98177fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f981809a5c0 0x7f981806f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:23.659 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:23.656+0000 7f98177fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f981809a5c0 0x7f981806f300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:54630/0 (socket says 172.21.15.169:54630) 2023-10-26T01:31:23.660 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:23.656+0000 7f98177fe640 1 -- 172.21.15.169:0/1728284817 learned_addr learned my addr 172.21.15.169:0/1728284817 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:31:23.661 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:23.657+0000 7f98177fe640 1 -- 172.21.15.169:0/1728284817 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f98180ac6d0 msgr2=0x7f98181f25c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:23.661 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:23.657+0000 7f98177fe640 1 --2- 172.21.15.169:0/1728284817 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f98180ac6d0 0x7f98181f25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:23.661 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:23.657+0000 7f98177fe640 1 -- 172.21.15.169:0/1728284817 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f98180c9970 msgr2=0x7f98180a9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:23.662 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:23.657+0000 7f98177fe640 1 --2- 172.21.15.169:0/1728284817 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f98180c9970 0x7f98180a9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:23.662 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:23.657+0000 7f98177fe640 1 -- 172.21.15.169:0/1728284817 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9800009490 con 0x7f981809a5c0 2023-10-26T01:31:23.662 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:23.657+0000 7f98177fe640 1 --2- 172.21.15.169:0/1728284817 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f981809a5c0 0x7f981806f300 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f980800c410 tx=0x7f980800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:23.662 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:23.657+0000 7f9814ff9640 1 -- 172.21.15.169:0/1728284817 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98080035f0 con 0x7f981809a5c0 2023-10-26T01:31:23.663 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:23.657+0000 7f9814ff9640 1 -- 172.21.15.169:0/1728284817 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9808003750 con 0x7f981809a5c0 2023-10-26T01:31:23.663 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:23.657+0000 7f981cb6e640 1 -- 172.21.15.169:0/1728284817 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f981806fe10 con 0x7f981809a5c0 2023-10-26T01:31:23.663 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:23.657+0000 7f9814ff9640 1 -- 172.21.15.169:0/1728284817 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9808003ba0 con 0x7f981809a5c0 2023-10-26T01:31:23.663 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:23.657+0000 7f981cb6e640 1 -- 172.21.15.169:0/1728284817 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98180703d0 con 0x7f981809a5c0 2023-10-26T01:31:23.663 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:23.658+0000 7f981cb6e640 1 -- 172.21.15.169:0/1728284817 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9818073c10 con 0x7f981809a5c0 2023-10-26T01:31:23.664 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:23.659+0000 7f9814ff9640 1 -- 172.21.15.169:0/1728284817 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f9808003d00 con 0x7f981809a5c0 2023-10-26T01:31:23.664 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:23.661+0000 7f9814ff9640 1 --2- 172.21.15.169:0/1728284817 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f97f806ae40 0x7f97f806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:23.664 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:23.661+0000 7f9814ff9640 1 -- 172.21.15.169:0/1728284817 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f98080889f0 con 0x7f981809a5c0 2023-10-26T01:31:23.665 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:23.661+0000 7f980ffff640 1 --2- 172.21.15.169:0/1728284817 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f97f806ae40 0x7f97f806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:23.665 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:23.662+0000 7f980ffff640 1 --2- 172.21.15.169:0/1728284817 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f97f806ae40 0x7f97f806d300 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f9800036000 tx=0x7f98000046c0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:23.666 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:23.665+0000 7f9814ff9640 1 -- 172.21.15.169:0/1728284817 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9808053aa0 con 0x7f981809a5c0 2023-10-26T01:31:23.854 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:23.852+0000 7f981cb6e640 1 -- 172.21.15.169:0/1728284817 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f98180cc6e0 con 0x7f981809a5c0 2023-10-26T01:31:23.856 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:23.855+0000 7f9814ff9640 1 -- 172.21.15.169:0/1728284817 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16553 (secure 0 0 0) 0x7f9808017090 con 0x7f981809a5c0 2023-10-26T01:31:23.857 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:31:23.857 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":21,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:30:43.078706+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-26T01:30:39.437631+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-26T01:30:40.561774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:31:23.859 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:23.857+0000 7f981cb6e640 1 -- 172.21.15.169:0/1728284817 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f97f806ae40 msgr2=0x7f97f806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:23.859 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:23.857+0000 7f981cb6e640 1 --2- 172.21.15.169:0/1728284817 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f97f806ae40 0x7f97f806d300 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f9800036000 tx=0x7f98000046c0 comp rx=0 tx=0).stop 2023-10-26T01:31:23.860 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:23.857+0000 7f981cb6e640 1 -- 172.21.15.169:0/1728284817 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f981809a5c0 msgr2=0x7f981806f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:23.860 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:23.857+0000 7f981cb6e640 1 --2- 172.21.15.169:0/1728284817 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f981809a5c0 0x7f981806f300 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f980800c410 tx=0x7f980800c8d0 comp rx=0 tx=0).stop 2023-10-26T01:31:23.860 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:23.857+0000 7f981cb6e640 1 -- 172.21.15.169:0/1728284817 shutdown_connections 2023-10-26T01:31:23.860 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:23.857+0000 7f981cb6e640 1 --2- 172.21.15.169:0/1728284817 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f97f806ae40 0x7f97f806d300 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:23.860 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:23.857+0000 7f981cb6e640 1 --2- 172.21.15.169:0/1728284817 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f98180c9970 0x7f98180a9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:23.861 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:23.857+0000 7f981cb6e640 1 --2- 172.21.15.169:0/1728284817 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f98180ac6d0 0x7f98181f25c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:23.861 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:23.857+0000 7f981cb6e640 1 --2- 172.21.15.169:0/1728284817 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f981809a5c0 0x7f981806f300 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:23.861 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:23.857+0000 7f981cb6e640 1 -- 172.21.15.169:0/1728284817 >> 172.21.15.169:0/1728284817 conn(0x7f98180aedd0 msgr2=0x7f9818141940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:23.861 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:23.858+0000 7f981cb6e640 1 -- 172.21.15.169:0/1728284817 shutdown_connections 2023-10-26T01:31:23.861 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:23.858+0000 7f981cb6e640 1 -- 172.21.15.169:0/1728284817 wait complete. 2023-10-26T01:31:23.879 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T01:30:32.200808+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-26T01:30:39.437631+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-26T01:30:40.561774+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T01:31:23.879 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T01:31:24.104 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.102+0000 7f3ee8626640 1 -- 172.21.15.169:0/207620985 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ee00cc1b0 msgr2=0x7f3ee00ad110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:24.104 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.102+0000 7f3ee8626640 1 --2- 172.21.15.169:0/207620985 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ee00cc1b0 0x7f3ee00ad110 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3ee00ae340 tx=0x7f3ed002be40 comp rx=0 tx=0).stop 2023-10-26T01:31:24.104 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.102+0000 7f3ee8626640 1 -- 172.21.15.169:0/207620985 shutdown_connections 2023-10-26T01:31:24.105 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.102+0000 7f3ee8626640 1 --2- 172.21.15.169:0/207620985 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ee00cc1b0 0x7f3ee00ad110 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:24.105 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.102+0000 7f3ee8626640 1 --2- 172.21.15.169:0/207620985 >> v2:172.21.15.170:3300/0 conn(0x7f3ee00cbdd0 0x7f3ee00ce090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:24.105 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.102+0000 7f3ee8626640 1 -- 172.21.15.169:0/207620985 >> 172.21.15.169:0/207620985 conn(0x7f3ee0098a60 msgr2=0x7f3ee00b10e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:24.105 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.102+0000 7f3ee8626640 1 -- 172.21.15.169:0/207620985 shutdown_connections 2023-10-26T01:31:24.105 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.102+0000 7f3ee8626640 1 -- 172.21.15.169:0/207620985 wait complete. 2023-10-26T01:31:24.106 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.104+0000 7f3ee8626640 1 Processor -- start 2023-10-26T01:31:24.107 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.104+0000 7f3ee8626640 1 -- start start 2023-10-26T01:31:24.107 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.105+0000 7f3ee8626640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3ee00ad650 0x7f3ee0146820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:24.107 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.105+0000 7f3ee8626640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ee00cbdd0 0x7f3ee00c8760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:24.107 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.105+0000 7f3ee8626640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3ee00cc1b0 0x7f3ee00c8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:24.107 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.105+0000 7f3ee8626640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee006c6a0 con 0x7f3ee00cbdd0 2023-10-26T01:31:24.107 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.105+0000 7f3ee8626640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee0146d60 con 0x7f3ee00ad650 2023-10-26T01:31:24.108 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.105+0000 7f3ee8626640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee01f5260 con 0x7f3ee00cc1b0 2023-10-26T01:31:24.109 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.105+0000 7f3ee5bc1640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ee00cbdd0 0x7f3ee00c8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:24.109 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.106+0000 7f3ee5bc1640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ee00cbdd0 0x7f3ee00c8760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:40512/0 (socket says 172.21.15.169:40512) 2023-10-26T01:31:24.110 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.106+0000 7f3ee5bc1640 1 -- 172.21.15.169:0/4173435277 learned_addr learned my addr 172.21.15.169:0/4173435277 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:31:24.110 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.106+0000 7f3ee63c2640 1 --2- 172.21.15.169:0/4173435277 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3ee00ad650 0x7f3ee0146820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:24.110 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.106+0000 7f3ee6bc3640 1 --2- 172.21.15.169:0/4173435277 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3ee00cc1b0 0x7f3ee00c8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:24.110 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.106+0000 7f3ee5bc1640 1 -- 172.21.15.169:0/4173435277 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3ee00cc1b0 msgr2=0x7f3ee00c8ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:24.110 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.106+0000 7f3ee5bc1640 1 --2- 172.21.15.169:0/4173435277 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3ee00cc1b0 0x7f3ee00c8ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:24.111 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.106+0000 7f3ee5bc1640 1 -- 172.21.15.169:0/4173435277 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3ee00ad650 msgr2=0x7f3ee0146820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:24.111 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.106+0000 7f3ee5bc1640 1 --2- 172.21.15.169:0/4173435277 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3ee00ad650 0x7f3ee0146820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:24.111 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.106+0000 7f3ee5bc1640 1 -- 172.21.15.169:0/4173435277 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ed0009560 con 0x7f3ee00cbdd0 2023-10-26T01:31:24.111 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.106+0000 7f3ee5bc1640 1 --2- 172.21.15.169:0/4173435277 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ee00cbdd0 0x7f3ee00c8760 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f3ed0036040 tx=0x7f3ed002dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:24.111 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.107+0000 7f3ed6ffd640 1 -- 172.21.15.169:0/4173435277 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ed0003800 con 0x7f3ee00cbdd0 2023-10-26T01:31:24.112 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.107+0000 7f3ed6ffd640 1 -- 172.21.15.169:0/4173435277 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3ed0003a50 con 0x7f3ee00cbdd0 2023-10-26T01:31:24.112 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.107+0000 7f3ee8626640 1 -- 172.21.15.169:0/4173435277 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ee00cc770 con 0x7f3ee00cbdd0 2023-10-26T01:31:24.112 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.107+0000 7f3ed6ffd640 1 -- 172.21.15.169:0/4173435277 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ed0003de0 con 0x7f3ee00cbdd0 2023-10-26T01:31:24.112 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.107+0000 7f3ee8626640 1 -- 172.21.15.169:0/4173435277 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ee01f54b0 con 0x7f3ee00cbdd0 2023-10-26T01:31:24.112 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.108+0000 7f3ee8626640 1 -- 172.21.15.169:0/4173435277 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ee004da70 con 0x7f3ee00cbdd0 2023-10-26T01:31:24.113 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.109+0000 7f3ed6ffd640 1 -- 172.21.15.169:0/4173435277 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f3ed003e450 con 0x7f3ee00cbdd0 2023-10-26T01:31:24.113 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.110+0000 7f3ed6ffd640 1 --2- 172.21.15.169:0/4173435277 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3ec406ae40 0x7f3ec406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:24.113 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.110+0000 7f3ed6ffd640 1 -- 172.21.15.169:0/4173435277 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3ed00ac180 con 0x7f3ee00cbdd0 2023-10-26T01:31:24.114 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.110+0000 7f3ee63c2640 1 --2- 172.21.15.169:0/4173435277 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3ec406ae40 0x7f3ec406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:24.114 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.111+0000 7f3ee63c2640 1 --2- 172.21.15.169:0/4173435277 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3ec406ae40 0x7f3ec406d300 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3edc005fd0 tx=0x7f3edc005850 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:24.115 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.114+0000 7f3ed6ffd640 1 -- 172.21.15.169:0/4173435277 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3ed0077220 con 0x7f3ee00cbdd0 2023-10-26T01:31:24.303 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.301+0000 7f3ee8626640 1 -- 172.21.15.169:0/4173435277 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3ee00b0b00 con 0x7f3ee00cbdd0 2023-10-26T01:31:24.304 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.302+0000 7f3ed6ffd640 1 -- 172.21.15.169:0/4173435277 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3ed007a840 con 0x7f3ee00cbdd0 2023-10-26T01:31:24.304 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 1 2023-10-26T01:31:24.306 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.304+0000 7f3ee8626640 1 -- 172.21.15.169:0/4173435277 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3ec406ae40 msgr2=0x7f3ec406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:24.307 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.304+0000 7f3ee8626640 1 --2- 172.21.15.169:0/4173435277 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3ec406ae40 0x7f3ec406d300 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3edc005fd0 tx=0x7f3edc005850 comp rx=0 tx=0).stop 2023-10-26T01:31:24.307 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.304+0000 7f3ee8626640 1 -- 172.21.15.169:0/4173435277 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ee00cbdd0 msgr2=0x7f3ee00c8760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:24.307 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.304+0000 7f3ee8626640 1 --2- 172.21.15.169:0/4173435277 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ee00cbdd0 0x7f3ee00c8760 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f3ed0036040 tx=0x7f3ed002dbc0 comp rx=0 tx=0).stop 2023-10-26T01:31:24.307 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.305+0000 7f3ee8626640 1 -- 172.21.15.169:0/4173435277 shutdown_connections 2023-10-26T01:31:24.308 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.305+0000 7f3ee8626640 1 --2- 172.21.15.169:0/4173435277 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3ec406ae40 0x7f3ec406d300 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:24.308 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.305+0000 7f3ee8626640 1 --2- 172.21.15.169:0/4173435277 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ee00cbdd0 0x7f3ee00c8760 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:24.308 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.305+0000 7f3ee8626640 1 --2- 172.21.15.169:0/4173435277 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3ee00cc1b0 0x7f3ee00c8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:24.308 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.305+0000 7f3ee8626640 1 --2- 172.21.15.169:0/4173435277 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3ee00ad650 0x7f3ee0146820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:24.308 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.305+0000 7f3ee8626640 1 -- 172.21.15.169:0/4173435277 >> 172.21.15.169:0/4173435277 conn(0x7f3ee0098a60 msgr2=0x7f3ee0152970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:24.309 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.305+0000 7f3ee8626640 1 -- 172.21.15.169:0/4173435277 shutdown_connections 2023-10-26T01:31:24.309 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.305+0000 7f3ee8626640 1 -- 172.21.15.169:0/4173435277 wait complete. 2023-10-26T01:31:24.325 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-26T01:31:24.543 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.541+0000 7f854da2e640 1 -- 172.21.15.169:0/4162184280 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f8548153b90 msgr2=0x7f8548069900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:24.545 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.541+0000 7f854da2e640 1 --2- 172.21.15.169:0/4162184280 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f8548153b90 0x7f8548069900 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8538005fd0 tx=0x7f853802c2e0 comp rx=0 tx=0).stop 2023-10-26T01:31:24.545 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.541+0000 7f854da2e640 1 -- 172.21.15.169:0/4162184280 shutdown_connections 2023-10-26T01:31:24.546 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.541+0000 7f854da2e640 1 --2- 172.21.15.169:0/4162184280 >> v2:172.21.15.170:3300/0 conn(0x7f854815d7e0 0x7f8548069e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:24.546 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.541+0000 7f854da2e640 1 --2- 172.21.15.169:0/4162184280 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f8548153b90 0x7f8548069900 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:24.546 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.541+0000 7f854da2e640 1 -- 172.21.15.169:0/4162184280 >> 172.21.15.169:0/4162184280 conn(0x7f85480af110 msgr2=0x7f85480af520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:24.546 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.541+0000 7f854da2e640 1 -- 172.21.15.169:0/4162184280 shutdown_connections 2023-10-26T01:31:24.546 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.541+0000 7f854da2e640 1 -- 172.21.15.169:0/4162184280 wait complete. 2023-10-26T01:31:24.548 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.543+0000 7f854da2e640 1 Processor -- start 2023-10-26T01:31:24.549 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.543+0000 7f854da2e640 1 -- start start 2023-10-26T01:31:24.549 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.544+0000 7f854da2e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f85480b0e50 0x7f85480c6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:24.549 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.544+0000 7f854da2e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8548153b90 0x7f8548152490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:24.549 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.544+0000 7f854da2e640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f854815d7e0 0x7f85481f5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:24.549 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.544+0000 7f854da2e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f854806ab10 con 0x7f85480b0e50 2023-10-26T01:31:24.550 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.544+0000 7f854da2e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f854806adb0 con 0x7f8548153b90 2023-10-26T01:31:24.550 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.544+0000 7f854da2e640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8548152fd0 con 0x7f854815d7e0 2023-10-26T01:31:24.550 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.544+0000 7f854ca2c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f85480b0e50 0x7f85480c6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:24.550 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.544+0000 7f854ca2c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f85480b0e50 0x7f85480c6370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:40516/0 (socket says 172.21.15.169:40516) 2023-10-26T01:31:24.550 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.544+0000 7f854ca2c640 1 -- 172.21.15.169:0/3144882548 learned_addr learned my addr 172.21.15.169:0/3144882548 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:31:24.551 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.545+0000 7f854ca2c640 1 -- 172.21.15.169:0/3144882548 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f854815d7e0 msgr2=0x7f85481f5230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:24.551 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.545+0000 7f854ca2c640 1 --2- 172.21.15.169:0/3144882548 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f854815d7e0 0x7f85481f5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:24.551 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.545+0000 7f8547fff640 1 --2- 172.21.15.169:0/3144882548 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8548153b90 0x7f8548152490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:24.551 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.545+0000 7f854ca2c640 1 -- 172.21.15.169:0/3144882548 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8548153b90 msgr2=0x7f8548152490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:24.551 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.545+0000 7f854ca2c640 1 --2- 172.21.15.169:0/3144882548 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8548153b90 0x7f8548152490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:24.552 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.545+0000 7f854ca2c640 1 -- 172.21.15.169:0/3144882548 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8538009900 con 0x7f85480b0e50 2023-10-26T01:31:24.552 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.545+0000 7f854ca2c640 1 --2- 172.21.15.169:0/3144882548 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f85480b0e50 0x7f85480c6370 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8538006010 tx=0x7f8538035b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:24.552 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.545+0000 7f85457fa640 1 -- 172.21.15.169:0/3144882548 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8538004240 con 0x7f85480b0e50 2023-10-26T01:31:24.552 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.545+0000 7f85457fa640 1 -- 172.21.15.169:0/3144882548 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8538004490 con 0x7f85480b0e50 2023-10-26T01:31:24.552 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.545+0000 7f854da2e640 1 -- 172.21.15.169:0/3144882548 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85481566e0 con 0x7f85480b0e50 2023-10-26T01:31:24.553 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.545+0000 7f85457fa640 1 -- 172.21.15.169:0/3144882548 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8538004840 con 0x7f85480b0e50 2023-10-26T01:31:24.553 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.545+0000 7f854da2e640 1 -- 172.21.15.169:0/3144882548 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8548156980 con 0x7f85480b0e50 2023-10-26T01:31:24.553 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.547+0000 7f854da2e640 1 -- 172.21.15.169:0/3144882548 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8548073c10 con 0x7f85480b0e50 2023-10-26T01:31:24.554 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.547+0000 7f85457fa640 1 -- 172.21.15.169:0/3144882548 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f853803b040 con 0x7f85480b0e50 2023-10-26T01:31:24.554 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.548+0000 7f85457fa640 1 --2- 172.21.15.169:0/3144882548 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f852806ae40 0x7f852806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:24.554 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.548+0000 7f85457fa640 1 -- 172.21.15.169:0/3144882548 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f85380abee0 con 0x7f85480b0e50 2023-10-26T01:31:24.555 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.549+0000 7f8547fff640 1 --2- 172.21.15.169:0/3144882548 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f852806ae40 0x7f852806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:24.555 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.550+0000 7f8547fff640 1 --2- 172.21.15.169:0/3144882548 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f852806ae40 0x7f852806d300 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8548155cc0 tx=0x7f8530009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:24.555 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.553+0000 7f85457fa640 1 -- 172.21.15.169:0/3144882548 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8538076f80 con 0x7f85480b0e50 2023-10-26T01:31:24.742 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.740+0000 7f854da2e640 1 -- 172.21.15.169:0/3144882548 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f85480b1c80 con 0x7f85480b0e50 2023-10-26T01:31:24.742 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.741+0000 7f85457fa640 1 -- 172.21.15.169:0/3144882548 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f853807a5a0 con 0x7f85480b0e50 2023-10-26T01:31:24.743 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 32 2023-10-26T01:31:24.745 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.743+0000 7f854da2e640 1 -- 172.21.15.169:0/3144882548 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f852806ae40 msgr2=0x7f852806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:24.745 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.743+0000 7f854da2e640 1 --2- 172.21.15.169:0/3144882548 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f852806ae40 0x7f852806d300 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8548155cc0 tx=0x7f8530009040 comp rx=0 tx=0).stop 2023-10-26T01:31:24.746 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.743+0000 7f854da2e640 1 -- 172.21.15.169:0/3144882548 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f85480b0e50 msgr2=0x7f85480c6370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:24.746 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.743+0000 7f854da2e640 1 --2- 172.21.15.169:0/3144882548 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f85480b0e50 0x7f85480c6370 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8538006010 tx=0x7f8538035b50 comp rx=0 tx=0).stop 2023-10-26T01:31:24.746 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.743+0000 7f854da2e640 1 -- 172.21.15.169:0/3144882548 shutdown_connections 2023-10-26T01:31:24.746 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.743+0000 7f854da2e640 1 --2- 172.21.15.169:0/3144882548 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f852806ae40 0x7f852806d300 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:24.747 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.743+0000 7f854da2e640 1 --2- 172.21.15.169:0/3144882548 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f854815d7e0 0x7f85481f5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:24.747 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.743+0000 7f854da2e640 1 --2- 172.21.15.169:0/3144882548 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8548153b90 0x7f8548152490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:24.747 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.743+0000 7f854da2e640 1 --2- 172.21.15.169:0/3144882548 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f85480b0e50 0x7f85480c6370 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:24.749 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.743+0000 7f854da2e640 1 -- 172.21.15.169:0/3144882548 >> 172.21.15.169:0/3144882548 conn(0x7f85480af110 msgr2=0x7f854815aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:24.749 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.744+0000 7f854da2e640 1 -- 172.21.15.169:0/3144882548 shutdown_connections 2023-10-26T01:31:24.749 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.744+0000 7f854da2e640 1 -- 172.21.15.169:0/3144882548 wait complete. 2023-10-26T01:31:24.763 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-26T01:31:24.978 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.976+0000 7f6309407640 1 -- 172.21.15.169:0/648857971 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63040cfb30 msgr2=0x7f63041630b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:24.979 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.976+0000 7f6309407640 1 --2- 172.21.15.169:0/648857971 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63040cfb30 0x7f63041630b0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f63040b22a0 tx=0x7f62fc02c260 comp rx=0 tx=0).stop 2023-10-26T01:31:24.979 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.976+0000 7f6309407640 1 -- 172.21.15.169:0/648857971 shutdown_connections 2023-10-26T01:31:24.979 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.976+0000 7f6309407640 1 --2- 172.21.15.169:0/648857971 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63040cfb30 0x7f63041630b0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:24.979 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.976+0000 7f6309407640 1 --2- 172.21.15.169:0/648857971 >> v2:172.21.15.170:3300/0 conn(0x7f63041594c0 0x7f630406ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:24.979 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.976+0000 7f6309407640 1 -- 172.21.15.169:0/648857971 >> 172.21.15.169:0/648857971 conn(0x7f6304098a60 msgr2=0x7f63040b4a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:24.980 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.976+0000 7f6309407640 1 -- 172.21.15.169:0/648857971 shutdown_connections 2023-10-26T01:31:24.980 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.976+0000 7f6309407640 1 -- 172.21.15.169:0/648857971 wait complete. 2023-10-26T01:31:24.981 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.978+0000 7f6309407640 1 Processor -- start 2023-10-26T01:31:24.981 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.979+0000 7f6309407640 1 -- start start 2023-10-26T01:31:24.982 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.980+0000 7f6309407640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63040b3020 0x7f630413ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:24.982 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.980+0000 7f6309407640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f63040cfb30 0x7f6304151480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:24.983 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.980+0000 7f6309407640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f63041594c0 0x7f630414ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:24.983 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.980+0000 7f6309407640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f630414c900 con 0x7f63040b3020 2023-10-26T01:31:24.983 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.980+0000 7f6309407640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f630414cbe0 con 0x7f63041594c0 2023-10-26T01:31:24.983 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.980+0000 7f6309407640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f630414cec0 con 0x7f63040cfb30 2023-10-26T01:31:24.983 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.980+0000 7f6302ffd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63040b3020 0x7f630413ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:24.984 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.980+0000 7f6302ffd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63040b3020 0x7f630413ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:40534/0 (socket says 172.21.15.169:40534) 2023-10-26T01:31:24.984 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.980+0000 7f6302ffd640 1 -- 172.21.15.169:0/1508904903 learned_addr learned my addr 172.21.15.169:0/1508904903 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:31:24.984 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.981+0000 7f62fbfff640 1 --2- 172.21.15.169:0/1508904903 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f63040cfb30 0x7f6304151480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:24.984 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.981+0000 7f6302ffd640 1 -- 172.21.15.169:0/1508904903 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f63040cfb30 msgr2=0x7f6304151480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:24.984 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.981+0000 7f6302ffd640 1 --2- 172.21.15.169:0/1508904903 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f63040cfb30 0x7f6304151480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:24.985 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.981+0000 7f63037fe640 1 --2- 172.21.15.169:0/1508904903 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f63041594c0 0x7f630414ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:24.985 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.981+0000 7f6302ffd640 1 -- 172.21.15.169:0/1508904903 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f63041594c0 msgr2=0x7f630414ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:24.985 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.981+0000 7f6302ffd640 1 --2- 172.21.15.169:0/1508904903 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f63041594c0 0x7f630414ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:24.985 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.981+0000 7f6302ffd640 1 -- 172.21.15.169:0/1508904903 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62fc009900 con 0x7f63040b3020 2023-10-26T01:31:24.986 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.981+0000 7f6302ffd640 1 --2- 172.21.15.169:0/1508904903 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63040b3020 0x7f630413ee20 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f62f400d6d0 tx=0x7f62f400db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:24.987 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.981+0000 7f62fb7fe640 1 -- 172.21.15.169:0/1508904903 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62f4005220 con 0x7f63040b3020 2023-10-26T01:31:24.987 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.981+0000 7f62fb7fe640 1 -- 172.21.15.169:0/1508904903 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f62f4005490 con 0x7f63040b3020 2023-10-26T01:31:24.987 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.981+0000 7f6309407640 1 -- 172.21.15.169:0/1508904903 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f630414d1a0 con 0x7f63040b3020 2023-10-26T01:31:24.987 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.981+0000 7f62fb7fe640 1 -- 172.21.15.169:0/1508904903 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62f4009a80 con 0x7f63040b3020 2023-10-26T01:31:24.987 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.982+0000 7f6309407640 1 -- 172.21.15.169:0/1508904903 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f630414c590 con 0x7f63040b3020 2023-10-26T01:31:24.988 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.983+0000 7f6309407640 1 -- 172.21.15.169:0/1508904903 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63041556b0 con 0x7f63040b3020 2023-10-26T01:31:24.988 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.984+0000 7f62fb7fe640 1 -- 172.21.15.169:0/1508904903 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f62f40047a0 con 0x7f63040b3020 2023-10-26T01:31:24.988 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.984+0000 7f62fb7fe640 1 --2- 172.21.15.169:0/1508904903 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f62e406ae40 0x7f62e406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:24.988 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.984+0000 7f62fb7fe640 1 -- 172.21.15.169:0/1508904903 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f62f4089a10 con 0x7f63040b3020 2023-10-26T01:31:24.988 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.985+0000 7f62fbfff640 1 --2- 172.21.15.169:0/1508904903 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f62e406ae40 0x7f62e406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:24.989 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.986+0000 7f62fbfff640 1 --2- 172.21.15.169:0/1508904903 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f62e406ae40 0x7f62e406d300 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f630406afe0 tx=0x7f62ec009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:24.990 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:24.989+0000 7f62fb7fe640 1 -- 172.21.15.169:0/1508904903 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f62f4054ac0 con 0x7f63040b3020 2023-10-26T01:31:25.179 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:25.177+0000 7f6309407640 1 -- 172.21.15.169:0/1508904903 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f63040aebc0 con 0x7f63040b3020 2023-10-26T01:31:25.180 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:25.178+0000 7f62fb7fe640 1 -- 172.21.15.169:0/1508904903 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f62f4017020 con 0x7f63040b3020 2023-10-26T01:31:25.180 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 64 2023-10-26T01:31:25.182 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:25.180+0000 7f6309407640 1 -- 172.21.15.169:0/1508904903 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f62e406ae40 msgr2=0x7f62e406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:25.183 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:25.180+0000 7f6309407640 1 --2- 172.21.15.169:0/1508904903 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f62e406ae40 0x7f62e406d300 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f630406afe0 tx=0x7f62ec009040 comp rx=0 tx=0).stop 2023-10-26T01:31:25.183 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:25.180+0000 7f6309407640 1 -- 172.21.15.169:0/1508904903 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63040b3020 msgr2=0x7f630413ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:25.183 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:25.180+0000 7f6309407640 1 --2- 172.21.15.169:0/1508904903 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63040b3020 0x7f630413ee20 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f62f400d6d0 tx=0x7f62f400db90 comp rx=0 tx=0).stop 2023-10-26T01:31:25.183 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:25.180+0000 7f6309407640 1 -- 172.21.15.169:0/1508904903 shutdown_connections 2023-10-26T01:31:25.183 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:25.180+0000 7f6309407640 1 --2- 172.21.15.169:0/1508904903 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f62e406ae40 0x7f62e406d300 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:25.184 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:25.180+0000 7f6309407640 1 --2- 172.21.15.169:0/1508904903 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f63041594c0 0x7f630414ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:25.184 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:25.180+0000 7f6309407640 1 --2- 172.21.15.169:0/1508904903 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f63040cfb30 0x7f6304151480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:25.184 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:25.180+0000 7f6309407640 1 --2- 172.21.15.169:0/1508904903 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63040b3020 0x7f630413ee20 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:25.184 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:25.180+0000 7f6309407640 1 -- 172.21.15.169:0/1508904903 >> 172.21.15.169:0/1508904903 conn(0x7f6304098a60 msgr2=0x7f630406bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:25.184 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:25.181+0000 7f6309407640 1 -- 172.21.15.169:0/1508904903 shutdown_connections 2023-10-26T01:31:25.185 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:25.181+0000 7f6309407640 1 -- 172.21.15.169:0/1508904903 wait complete. 2023-10-26T01:31:25.238 INFO:tasks.cephfs_test_runner:Starting test: test_barrier (tasks.cephfs.test_full.TestClusterFull) That when an OSD epoch barrier is set on an MDS, subsequently 2023-10-26T01:31:25.239 INFO:tasks.cephfs_test_runner:test_barrier (tasks.cephfs.test_full.TestClusterFull) 2023-10-26T01:31:25.239 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:31:25.461 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:25.459+0000 7f7116dc9640 1 -- 172.21.15.169:0/2438647273 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f711006ed30 msgr2=0x7f71100cd670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:25.461 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:25.459+0000 7f7116dc9640 1 --2- 172.21.15.169:0/2438647273 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f711006ed30 0x7f71100cd670 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f70fc009830 tx=0x7f70fc02bd60 comp rx=0 tx=0).stop 2023-10-26T01:31:25.462 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:25.459+0000 7f7116dc9640 1 -- 172.21.15.169:0/2438647273 shutdown_connections 2023-10-26T01:31:25.462 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:25.459+0000 7f7116dc9640 1 --2- 172.21.15.169:0/2438647273 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f711006ed30 0x7f71100cd670 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:25.462 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:25.459+0000 7f7116dc9640 1 -- 172.21.15.169:0/2438647273 >> 172.21.15.169:0/2438647273 conn(0x7f7110069930 msgr2=0x7f7110069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:25.462 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:25.459+0000 7f7116dc9640 1 -- 172.21.15.169:0/2438647273 shutdown_connections 2023-10-26T01:31:25.463 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:25.459+0000 7f7116dc9640 1 -- 172.21.15.169:0/2438647273 wait complete. 2023-10-26T01:31:25.464 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:25.461+0000 7f7116dc9640 1 Processor -- start 2023-10-26T01:31:25.464 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:25.461+0000 7f7116dc9640 1 -- start start 2023-10-26T01:31:25.464 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:25.462+0000 7f7116dc9640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f711006ed30 0x7f7110142a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:25.464 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:25.462+0000 7f7116dc9640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71100cdbb0 0x7f7110141070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:25.464 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:25.462+0000 7f7116dc9640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f711015b610 0x7f71101f1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:25.465 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:25.462+0000 7f7116dc9640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7110141970 con 0x7f711015b610 2023-10-26T01:31:25.465 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:25.462+0000 7f7116dc9640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7110141c50 con 0x7f71100cdbb0 2023-10-26T01:31:25.465 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:25.462+0000 7f7116dc9640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7110141f30 con 0x7f711006ed30 2023-10-26T01:31:25.465 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:25.462+0000 7f7115366640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f711015b610 0x7f71101f1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:25.465 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:25.462+0000 7f7115366640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f711015b610 0x7f71101f1c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:40538/0 (socket says 172.21.15.169:40538) 2023-10-26T01:31:25.466 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:25.462+0000 7f7115366640 1 -- 172.21.15.169:0/1839812908 learned_addr learned my addr 172.21.15.169:0/1839812908 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:31:25.467 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:25.463+0000 7f7115366640 1 -- 172.21.15.169:0/1839812908 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f711006ed30 msgr2=0x7f7110142a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:25.467 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:25.463+0000 7f7115366640 1 --2- 172.21.15.169:0/1839812908 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f711006ed30 0x7f7110142a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:25.467 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:25.463+0000 7f7115366640 1 -- 172.21.15.169:0/1839812908 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71100cdbb0 msgr2=0x7f7110141070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:25.467 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:25.463+0000 7f7115366640 1 --2- 172.21.15.169:0/1839812908 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71100cdbb0 0x7f7110141070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:25.468 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:25.463+0000 7f7115366640 1 -- 172.21.15.169:0/1839812908 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70fc009490 con 0x7f711015b610 2023-10-26T01:31:25.468 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:25.463+0000 7f7115366640 1 --2- 172.21.15.169:0/1839812908 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f711015b610 0x7f71101f1c90 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7108001560 tx=0x7f710800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:25.468 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:25.463+0000 7f710d7fa640 1 -- 172.21.15.169:0/1839812908 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7108003da0 con 0x7f711015b610 2023-10-26T01:31:25.468 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:25.463+0000 7f710d7fa640 1 -- 172.21.15.169:0/1839812908 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7108005070 con 0x7f711015b610 2023-10-26T01:31:25.468 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:25.463+0000 7f710d7fa640 1 -- 172.21.15.169:0/1839812908 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7108005480 con 0x7f711015b610 2023-10-26T01:31:25.469 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:25.463+0000 7f7116dc9640 1 -- 172.21.15.169:0/1839812908 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7110142210 con 0x7f711015b610 2023-10-26T01:31:25.469 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:25.463+0000 7f7116dc9640 1 -- 172.21.15.169:0/1839812908 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71101415b0 con 0x7f711015b610 2023-10-26T01:31:25.469 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:25.464+0000 7f7116dc9640 1 -- 172.21.15.169:0/1839812908 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7110142730 con 0x7f711015b610 2023-10-26T01:31:25.469 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:25.465+0000 7f710d7fa640 1 -- 172.21.15.169:0/1839812908 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f71080055e0 con 0x7f711015b610 2023-10-26T01:31:25.470 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:25.466+0000 7f710d7fa640 1 --2- 172.21.15.169:0/1839812908 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f70f406ae40 0x7f70f406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:25.470 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:25.466+0000 7f710d7fa640 1 -- 172.21.15.169:0/1839812908 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7108089200 con 0x7f711015b610 2023-10-26T01:31:25.471 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:25.467+0000 7f7114b65640 1 --2- 172.21.15.169:0/1839812908 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f70f406ae40 0x7f70f406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:25.471 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:25.468+0000 7f7114b65640 1 --2- 172.21.15.169:0/1839812908 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f70f406ae40 0x7f70f406d300 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7104006fd0 tx=0x7f7104006bc0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:25.472 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:25.471+0000 7f710d7fa640 1 -- 172.21.15.169:0/1839812908 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7110142730 con 0x7f711015b610 2023-10-26T01:31:25.661 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:25.659+0000 7f7116dc9640 1 -- 172.21.15.169:0/1839812908 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7110142730 con 0x7f711015b610 2023-10-26T01:31:25.662 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:25.661+0000 7f710d7fa640 1 -- 172.21.15.169:0/1839812908 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16553 (secure 0 0 0) 0x7f7110142730 con 0x7f711015b610 2023-10-26T01:31:25.663 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:31:25.663 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":21,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:30:43.078706+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-26T01:30:39.437631+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-26T01:30:40.561774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:31:25.665 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:25.663+0000 7f7116dc9640 1 -- 172.21.15.169:0/1839812908 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f70f406ae40 msgr2=0x7f70f406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:25.665 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:25.663+0000 7f7116dc9640 1 --2- 172.21.15.169:0/1839812908 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f70f406ae40 0x7f70f406d300 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7104006fd0 tx=0x7f7104006bc0 comp rx=0 tx=0).stop 2023-10-26T01:31:25.666 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:25.663+0000 7f7116dc9640 1 -- 172.21.15.169:0/1839812908 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f711015b610 msgr2=0x7f71101f1c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:25.666 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:25.663+0000 7f7116dc9640 1 --2- 172.21.15.169:0/1839812908 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f711015b610 0x7f71101f1c90 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7108001560 tx=0x7f710800cc70 comp rx=0 tx=0).stop 2023-10-26T01:31:25.666 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:25.663+0000 7f7116dc9640 1 -- 172.21.15.169:0/1839812908 shutdown_connections 2023-10-26T01:31:25.666 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:25.663+0000 7f7116dc9640 1 --2- 172.21.15.169:0/1839812908 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f70f406ae40 0x7f70f406d300 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:25.666 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:25.663+0000 7f7116dc9640 1 --2- 172.21.15.169:0/1839812908 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f711015b610 0x7f71101f1c90 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:25.667 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:25.663+0000 7f7116dc9640 1 --2- 172.21.15.169:0/1839812908 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71100cdbb0 0x7f7110141070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:25.667 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:25.663+0000 7f7116dc9640 1 --2- 172.21.15.169:0/1839812908 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f711006ed30 0x7f7110142a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:25.667 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:25.663+0000 7f7116dc9640 1 -- 172.21.15.169:0/1839812908 >> 172.21.15.169:0/1839812908 conn(0x7f7110069930 msgr2=0x7f71100aa1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:25.667 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:25.664+0000 7f7116dc9640 1 -- 172.21.15.169:0/1839812908 shutdown_connections 2023-10-26T01:31:25.667 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:25.664+0000 7f7116dc9640 1 -- 172.21.15.169:0/1839812908 wait complete. 2023-10-26T01:31:25.684 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T01:30:32.200808+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-26T01:30:39.437631+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-26T01:30:40.561774+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T01:31:25.685 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T01:31:25.904 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:25.902+0000 7fb23e64f640 1 -- 172.21.15.169:0/3896703004 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb238159480 msgr2=0x7fb2380b02e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:25.904 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:25.902+0000 7fb23e64f640 1 --2- 172.21.15.169:0/3896703004 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb238159480 0x7fb2380b02e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fb2380b20a0 tx=0x7fb22002bd90 comp rx=0 tx=0).stop 2023-10-26T01:31:25.905 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:25.902+0000 7fb23e64f640 1 -- 172.21.15.169:0/3896703004 shutdown_connections 2023-10-26T01:31:25.905 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:25.902+0000 7fb23e64f640 1 --2- 172.21.15.169:0/3896703004 >> v2:172.21.15.170:3300/0 conn(0x7fb2380ab4c0 0x7fb2380cd690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:25.905 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:25.902+0000 7fb23e64f640 1 --2- 172.21.15.169:0/3896703004 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb2380b0820 0x7fb2380aaf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:25.905 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:25.902+0000 7fb23e64f640 1 --2- 172.21.15.169:0/3896703004 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb238159480 0x7fb2380b02e0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:25.905 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:25.902+0000 7fb23e64f640 1 -- 172.21.15.169:0/3896703004 >> 172.21.15.169:0/3896703004 conn(0x7fb238069930 msgr2=0x7fb238069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:25.906 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:25.902+0000 7fb23e64f640 1 -- 172.21.15.169:0/3896703004 shutdown_connections 2023-10-26T01:31:25.906 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:25.902+0000 7fb23e64f640 1 -- 172.21.15.169:0/3896703004 wait complete. 2023-10-26T01:31:25.908 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:25.904+0000 7fb23e64f640 1 Processor -- start 2023-10-26T01:31:25.908 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:25.904+0000 7fb23e64f640 1 -- start start 2023-10-26T01:31:25.908 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:25.905+0000 7fb23e64f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb2380ab4c0 0x7fb2381527f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:25.909 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:25.905+0000 7fb23e64f640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb2380b0820 0x7fb238148ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:25.909 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:25.905+0000 7fb23e64f640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb238159480 0x7fb2381496b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:25.909 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:25.905+0000 7fb23e64f640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb238153a70 con 0x7fb238159480 2023-10-26T01:31:25.909 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:25.905+0000 7fb23e64f640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb238152e60 con 0x7fb2380ab4c0 2023-10-26T01:31:25.910 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:25.905+0000 7fb23e64f640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb238153140 con 0x7fb2380b0820 2023-10-26T01:31:25.910 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:25.905+0000 7fb23cbec640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb238159480 0x7fb2381496b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:25.910 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:25.905+0000 7fb23cbec640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb238159480 0x7fb2381496b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:40554/0 (socket says 172.21.15.169:40554) 2023-10-26T01:31:25.910 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:25.905+0000 7fb23cbec640 1 -- 172.21.15.169:0/1173678402 learned_addr learned my addr 172.21.15.169:0/1173678402 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:31:25.910 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:25.906+0000 7fb2377fe640 1 --2- 172.21.15.169:0/1173678402 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb2380b0820 0x7fb238148ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:25.911 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:25.906+0000 7fb23cbec640 1 -- 172.21.15.169:0/1173678402 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb2380b0820 msgr2=0x7fb238148ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:25.911 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:25.906+0000 7fb23cbec640 1 --2- 172.21.15.169:0/1173678402 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb2380b0820 0x7fb238148ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:25.911 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:25.906+0000 7fb237fff640 1 --2- 172.21.15.169:0/1173678402 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb2380ab4c0 0x7fb2381527f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:25.911 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:25.906+0000 7fb23cbec640 1 -- 172.21.15.169:0/1173678402 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb2380ab4c0 msgr2=0x7fb2381527f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:25.911 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:25.906+0000 7fb23cbec640 1 --2- 172.21.15.169:0/1173678402 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb2380ab4c0 0x7fb2381527f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:25.912 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:25.906+0000 7fb23cbec640 1 -- 172.21.15.169:0/1173678402 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb220009560 con 0x7fb238159480 2023-10-26T01:31:25.912 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:25.906+0000 7fb23cbec640 1 --2- 172.21.15.169:0/1173678402 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb238159480 0x7fb2381496b0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fb23000c970 tx=0x7fb23000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:25.912 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:25.906+0000 7fb234ff9640 1 -- 172.21.15.169:0/1173678402 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb230007bb0 con 0x7fb238159480 2023-10-26T01:31:25.912 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:25.907+0000 7fb234ff9640 1 -- 172.21.15.169:0/1173678402 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb230007e20 con 0x7fb238159480 2023-10-26T01:31:25.912 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:25.907+0000 7fb234ff9640 1 -- 172.21.15.169:0/1173678402 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb23000f6b0 con 0x7fb238159480 2023-10-26T01:31:25.913 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:25.907+0000 7fb23e64f640 1 -- 172.21.15.169:0/1173678402 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb238153420 con 0x7fb238159480 2023-10-26T01:31:25.913 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:25.907+0000 7fb23e64f640 1 -- 172.21.15.169:0/1173678402 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb238149020 con 0x7fb238159480 2023-10-26T01:31:25.914 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:25.908+0000 7fb23e64f640 1 -- 172.21.15.169:0/1173678402 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb238066af0 con 0x7fb238159480 2023-10-26T01:31:25.914 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:25.909+0000 7fb234ff9640 1 -- 172.21.15.169:0/1173678402 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fb23000f810 con 0x7fb238159480 2023-10-26T01:31:25.914 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:25.910+0000 7fb234ff9640 1 --2- 172.21.15.169:0/1173678402 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb21806ae70 0x7fb21806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:25.914 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:25.910+0000 7fb234ff9640 1 -- 172.21.15.169:0/1173678402 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb230089d50 con 0x7fb238159480 2023-10-26T01:31:25.915 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:25.911+0000 7fb237fff640 1 --2- 172.21.15.169:0/1173678402 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb21806ae70 0x7fb21806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:25.915 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:25.912+0000 7fb237fff640 1 --2- 172.21.15.169:0/1173678402 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb21806ae70 0x7fb21806d330 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb220035000 tx=0x7fb220004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:25.916 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:25.915+0000 7fb234ff9640 1 -- 172.21.15.169:0/1173678402 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb230054f30 con 0x7fb238159480 2023-10-26T01:31:26.105 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:26.103+0000 7fb23e64f640 1 -- 172.21.15.169:0/1173678402 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb23804da70 con 0x7fb238159480 2023-10-26T01:31:26.106 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:26.104+0000 7fb234ff9640 1 -- 172.21.15.169:0/1173678402 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb23001d020 con 0x7fb238159480 2023-10-26T01:31:26.106 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 1 2023-10-26T01:31:26.109 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:26.107+0000 7fb23e64f640 1 -- 172.21.15.169:0/1173678402 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb21806ae70 msgr2=0x7fb21806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:26.109 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:26.107+0000 7fb23e64f640 1 --2- 172.21.15.169:0/1173678402 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb21806ae70 0x7fb21806d330 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb220035000 tx=0x7fb220004730 comp rx=0 tx=0).stop 2023-10-26T01:31:26.110 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:26.107+0000 7fb23e64f640 1 -- 172.21.15.169:0/1173678402 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb238159480 msgr2=0x7fb2381496b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:26.110 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:26.107+0000 7fb23e64f640 1 --2- 172.21.15.169:0/1173678402 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb238159480 0x7fb2381496b0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fb23000c970 tx=0x7fb23000ce30 comp rx=0 tx=0).stop 2023-10-26T01:31:26.110 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:26.107+0000 7fb23e64f640 1 -- 172.21.15.169:0/1173678402 shutdown_connections 2023-10-26T01:31:26.110 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:26.107+0000 7fb23e64f640 1 --2- 172.21.15.169:0/1173678402 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb21806ae70 0x7fb21806d330 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:26.110 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:26.107+0000 7fb23e64f640 1 --2- 172.21.15.169:0/1173678402 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb238159480 0x7fb2381496b0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:26.111 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:26.107+0000 7fb23e64f640 1 --2- 172.21.15.169:0/1173678402 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb2380b0820 0x7fb238148ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:26.111 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:26.107+0000 7fb23e64f640 1 --2- 172.21.15.169:0/1173678402 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb2380ab4c0 0x7fb2381527f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:26.111 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:26.107+0000 7fb23e64f640 1 -- 172.21.15.169:0/1173678402 >> 172.21.15.169:0/1173678402 conn(0x7fb238069930 msgr2=0x7fb238070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:26.111 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:26.108+0000 7fb23e64f640 1 -- 172.21.15.169:0/1173678402 shutdown_connections 2023-10-26T01:31:26.111 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:26.108+0000 7fb23e64f640 1 -- 172.21.15.169:0/1173678402 wait complete. 2023-10-26T01:31:26.128 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-26T01:31:26.351 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:26.349+0000 7f6ac737d640 1 -- 172.21.15.169:0/4200753496 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6ac015bbb0 msgr2=0x7f6ac0155360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:26.351 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:26.349+0000 7f6ac737d640 1 --2- 172.21.15.169:0/4200753496 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6ac015bbb0 0x7f6ac0155360 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f6ac00b0520 tx=0x7f6ab002bcc0 comp rx=0 tx=0).stop 2023-10-26T01:31:26.352 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:26.349+0000 7f6ac737d640 1 -- 172.21.15.169:0/4200753496 shutdown_connections 2023-10-26T01:31:26.352 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:26.349+0000 7f6ac737d640 1 --2- 172.21.15.169:0/4200753496 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6ac00b05a0 0x7f6ac00b09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:26.352 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:26.349+0000 7f6ac737d640 1 --2- 172.21.15.169:0/4200753496 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6ac015bbb0 0x7f6ac0155360 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:26.352 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:26.349+0000 7f6ac737d640 1 -- 172.21.15.169:0/4200753496 >> 172.21.15.169:0/4200753496 conn(0x7f6ac0069930 msgr2=0x7f6ac0069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:26.353 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:26.349+0000 7f6ac737d640 1 -- 172.21.15.169:0/4200753496 shutdown_connections 2023-10-26T01:31:26.353 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:26.349+0000 7f6ac737d640 1 -- 172.21.15.169:0/4200753496 wait complete. 2023-10-26T01:31:26.354 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:26.351+0000 7f6ac737d640 1 Processor -- start 2023-10-26T01:31:26.354 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:26.351+0000 7f6ac737d640 1 -- start start 2023-10-26T01:31:26.354 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:26.352+0000 7f6ac737d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6ac006ce00 0x7f6ac01542a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:26.355 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:26.352+0000 7f6ac737d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6ac00b05a0 0x7f6ac00cbb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:26.355 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:26.352+0000 7f6ac737d640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6ac015bbb0 0x7f6ac00cc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:26.355 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:26.352+0000 7f6ac737d640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac014ad70 con 0x7f6ac006ce00 2023-10-26T01:31:26.355 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:26.352+0000 7f6ac737d640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac01f2130 con 0x7f6ac00b05a0 2023-10-26T01:31:26.356 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:26.352+0000 7f6ac737d640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac01f2410 con 0x7f6ac015bbb0 2023-10-26T01:31:26.356 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:26.352+0000 7f6ac5119640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6ac006ce00 0x7f6ac01542a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:26.356 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:26.352+0000 7f6ac5119640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6ac006ce00 0x7f6ac01542a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:40574/0 (socket says 172.21.15.169:40574) 2023-10-26T01:31:26.356 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:26.352+0000 7f6ac5119640 1 -- 172.21.15.169:0/2782669760 learned_addr learned my addr 172.21.15.169:0/2782669760 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:31:26.357 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:26.353+0000 7f6ac4918640 1 --2- 172.21.15.169:0/2782669760 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6ac00b05a0 0x7f6ac00cbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:26.358 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:26.353+0000 7f6ac591a640 1 --2- 172.21.15.169:0/2782669760 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6ac015bbb0 0x7f6ac00cc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:26.358 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:26.353+0000 7f6ac5119640 1 -- 172.21.15.169:0/2782669760 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6ac015bbb0 msgr2=0x7f6ac00cc0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:26.358 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:26.353+0000 7f6ac5119640 1 --2- 172.21.15.169:0/2782669760 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6ac015bbb0 0x7f6ac00cc0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:26.358 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:26.353+0000 7f6ac5119640 1 -- 172.21.15.169:0/2782669760 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6ac00b05a0 msgr2=0x7f6ac00cbb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:26.359 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:26.353+0000 7f6ac5119640 1 --2- 172.21.15.169:0/2782669760 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6ac00b05a0 0x7f6ac00cbb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:26.359 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:26.353+0000 7f6ac5119640 1 -- 172.21.15.169:0/2782669760 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ab0009560 con 0x7f6ac006ce00 2023-10-26T01:31:26.359 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:26.353+0000 7f6ac5119640 1 --2- 172.21.15.169:0/2782669760 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6ac006ce00 0x7f6ac01542a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f6abc00c430 tx=0x7f6abc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:26.359 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:26.353+0000 7f6ab5ffb640 1 -- 172.21.15.169:0/2782669760 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6abc011790 con 0x7f6ac006ce00 2023-10-26T01:31:26.359 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:26.353+0000 7f6ab5ffb640 1 -- 172.21.15.169:0/2782669760 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6abc0119e0 con 0x7f6ac006ce00 2023-10-26T01:31:26.360 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:26.353+0000 7f6ab5ffb640 1 -- 172.21.15.169:0/2782669760 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6abc011e30 con 0x7f6ac006ce00 2023-10-26T01:31:26.360 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:26.354+0000 7f6ac737d640 1 -- 172.21.15.169:0/2782669760 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ac01f26f0 con 0x7f6ac006ce00 2023-10-26T01:31:26.360 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:26.354+0000 7f6ac737d640 1 -- 172.21.15.169:0/2782669760 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ac01f2cb0 con 0x7f6ac006ce00 2023-10-26T01:31:26.360 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:26.355+0000 7f6ac737d640 1 -- 172.21.15.169:0/2782669760 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ac004db10 con 0x7f6ac006ce00 2023-10-26T01:31:26.360 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:26.356+0000 7f6ab5ffb640 1 -- 172.21.15.169:0/2782669760 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f6abc01c460 con 0x7f6ac006ce00 2023-10-26T01:31:26.361 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:26.357+0000 7f6ab5ffb640 1 --2- 172.21.15.169:0/2782669760 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f6aa406ae40 0x7f6aa406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:26.361 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:26.357+0000 7f6ab5ffb640 1 -- 172.21.15.169:0/2782669760 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6abc089870 con 0x7f6ac006ce00 2023-10-26T01:31:26.361 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:26.358+0000 7f6ac4918640 1 --2- 172.21.15.169:0/2782669760 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f6aa406ae40 0x7f6aa406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:26.362 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:26.359+0000 7f6ac4918640 1 --2- 172.21.15.169:0/2782669760 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f6aa406ae40 0x7f6aa406d300 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f6ab0035000 tx=0x7f6ab0004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:26.363 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:26.362+0000 7f6ab5ffb640 1 -- 172.21.15.169:0/2782669760 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6abc054910 con 0x7f6ac006ce00 2023-10-26T01:31:26.553 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:26.551+0000 7f6ac737d640 1 -- 172.21.15.169:0/2782669760 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6ac0159140 con 0x7f6ac006ce00 2023-10-26T01:31:26.554 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:26.552+0000 7f6ab5ffb640 1 -- 172.21.15.169:0/2782669760 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6abc057f30 con 0x7f6ac006ce00 2023-10-26T01:31:26.555 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 32 2023-10-26T01:31:26.557 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:26.555+0000 7f6ac737d640 1 -- 172.21.15.169:0/2782669760 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f6aa406ae40 msgr2=0x7f6aa406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:26.557 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:26.555+0000 7f6ac737d640 1 --2- 172.21.15.169:0/2782669760 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f6aa406ae40 0x7f6aa406d300 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f6ab0035000 tx=0x7f6ab0004730 comp rx=0 tx=0).stop 2023-10-26T01:31:26.558 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:26.555+0000 7f6ac737d640 1 -- 172.21.15.169:0/2782669760 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6ac006ce00 msgr2=0x7f6ac01542a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:26.558 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:26.555+0000 7f6ac737d640 1 --2- 172.21.15.169:0/2782669760 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6ac006ce00 0x7f6ac01542a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f6abc00c430 tx=0x7f6abc00c8f0 comp rx=0 tx=0).stop 2023-10-26T01:31:26.558 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:26.555+0000 7f6ac737d640 1 -- 172.21.15.169:0/2782669760 shutdown_connections 2023-10-26T01:31:26.558 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:26.555+0000 7f6ac737d640 1 --2- 172.21.15.169:0/2782669760 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f6aa406ae40 0x7f6aa406d300 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:26.559 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:26.555+0000 7f6ac737d640 1 --2- 172.21.15.169:0/2782669760 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6ac015bbb0 0x7f6ac00cc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:26.559 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:26.555+0000 7f6ac737d640 1 --2- 172.21.15.169:0/2782669760 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6ac00b05a0 0x7f6ac00cbb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:26.559 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:26.555+0000 7f6ac737d640 1 --2- 172.21.15.169:0/2782669760 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6ac006ce00 0x7f6ac01542a0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:26.559 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:26.555+0000 7f6ac737d640 1 -- 172.21.15.169:0/2782669760 >> 172.21.15.169:0/2782669760 conn(0x7f6ac0069930 msgr2=0x7f6ac00cd870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:26.560 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:26.555+0000 7f6ac737d640 1 -- 172.21.15.169:0/2782669760 shutdown_connections 2023-10-26T01:31:26.560 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:26.555+0000 7f6ac737d640 1 -- 172.21.15.169:0/2782669760 wait complete. 2023-10-26T01:31:26.576 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-26T01:31:26.799 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:26.797+0000 7efdb4d47640 1 -- 172.21.15.169:0/1561959416 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd980047c0 msgr2=0x7efd98004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:26.800 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:26.797+0000 7efdb4d47640 1 --2- 172.21.15.169:0/1561959416 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd980047c0 0x7efd98004c40 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7efda400baf0 tx=0x7efd980057f0 comp rx=0 tx=0).stop 2023-10-26T01:31:26.800 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:26.797+0000 7efdaffff640 1 -- 172.21.15.169:0/1561959416 reap_dead start 2023-10-26T01:31:26.800 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:26.797+0000 7efdb4d47640 1 -- 172.21.15.169:0/1561959416 shutdown_connections 2023-10-26T01:31:26.800 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:26.797+0000 7efdb4d47640 1 -- 172.21.15.169:0/1561959416 >> 172.21.15.169:0/1561959416 conn(0x7efdb00af110 msgr2=0x7efdb00af520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:26.801 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:26.798+0000 7efdb4d47640 1 -- 172.21.15.169:0/1561959416 shutdown_connections 2023-10-26T01:31:26.801 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:26.798+0000 7efdb4d47640 1 -- 172.21.15.169:0/1561959416 wait complete. 2023-10-26T01:31:26.802 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:26.799+0000 7efdb4d47640 1 Processor -- start 2023-10-26T01:31:26.802 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:26.800+0000 7efdb4d47640 1 -- start start 2023-10-26T01:31:26.804 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:26.800+0000 7efdb4d47640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd980047c0 0x7efdb0145a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:26.804 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:26.801+0000 7efdb4d47640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7efdb00ca180 0x7efdb01e1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:26.804 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:26.801+0000 7efdb4d47640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efdb01e2530 0x7efdb0147cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:26.805 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:26.801+0000 7efdb4d47640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdb00cca50 con 0x7efdb01e2530 2023-10-26T01:31:26.805 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:26.801+0000 7efdb4d47640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdb00cbe80 con 0x7efd980047c0 2023-10-26T01:31:26.805 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:26.801+0000 7efdb4d47640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdb0147a40 con 0x7efdb00ca180 2023-10-26T01:31:26.805 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:26.801+0000 7efdaffff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efdb01e2530 0x7efdb0147cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:26.805 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:26.801+0000 7efdaffff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efdb01e2530 0x7efdb0147cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:40600/0 (socket says 172.21.15.169:40600) 2023-10-26T01:31:26.806 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:26.801+0000 7efdaffff640 1 -- 172.21.15.169:0/533894515 learned_addr learned my addr 172.21.15.169:0/533894515 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:31:26.806 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:26.801+0000 7efdaf7fe640 1 --2- 172.21.15.169:0/533894515 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd980047c0 0x7efdb0145a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:26.806 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:26.801+0000 7efdaeffd640 1 --2- 172.21.15.169:0/533894515 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7efdb00ca180 0x7efdb01e1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:26.806 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:26.801+0000 7efdaffff640 1 -- 172.21.15.169:0/533894515 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7efdb00ca180 msgr2=0x7efdb01e1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:26.807 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:26.801+0000 7efdaffff640 1 --2- 172.21.15.169:0/533894515 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7efdb00ca180 0x7efdb01e1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:26.807 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:26.801+0000 7efdaffff640 1 -- 172.21.15.169:0/533894515 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd980047c0 msgr2=0x7efdb0145a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:26.807 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:26.801+0000 7efdaffff640 1 --2- 172.21.15.169:0/533894515 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd980047c0 0x7efdb0145a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:26.807 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:26.801+0000 7efdaffff640 1 -- 172.21.15.169:0/533894515 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdb01e29e0 con 0x7efdb01e2530 2023-10-26T01:31:26.807 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:26.801+0000 7efdaeffd640 1 --2- 172.21.15.169:0/533894515 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7efdb00ca180 0x7efdb01e1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:31:26.808 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:26.802+0000 7efdaffff640 1 --2- 172.21.15.169:0/533894515 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efdb01e2530 0x7efdb0147cf0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7efda8001370 tx=0x7efda800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:26.808 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:26.802+0000 7efd97fff640 1 -- 172.21.15.169:0/533894515 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efda800ad10 con 0x7efdb01e2530 2023-10-26T01:31:26.808 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:26.802+0000 7efd97fff640 1 -- 172.21.15.169:0/533894515 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7efda8007070 con 0x7efdb01e2530 2023-10-26T01:31:26.808 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:26.802+0000 7efdb4d47640 1 -- 172.21.15.169:0/533894515 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdb00ca960 con 0x7efdb01e2530 2023-10-26T01:31:26.809 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:26.802+0000 7efd97fff640 1 -- 172.21.15.169:0/533894515 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efda8007480 con 0x7efdb01e2530 2023-10-26T01:31:26.809 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:26.802+0000 7efdb4d47640 1 -- 172.21.15.169:0/533894515 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdb00cc0b0 con 0x7efdb01e2530 2023-10-26T01:31:26.810 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:26.803+0000 7efdb4d47640 1 -- 172.21.15.169:0/533894515 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdb00ccb90 con 0x7efdb01e2530 2023-10-26T01:31:26.810 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:26.804+0000 7efd97fff640 1 -- 172.21.15.169:0/533894515 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7efda80075e0 con 0x7efdb01e2530 2023-10-26T01:31:26.810 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:26.805+0000 7efd97fff640 1 --2- 172.21.15.169:0/533894515 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7efd9c06ae40 0x7efd9c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:26.810 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:26.805+0000 7efd97fff640 1 -- 172.21.15.169:0/533894515 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efda8089680 con 0x7efdb01e2530 2023-10-26T01:31:26.811 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:26.806+0000 7efdaf7fe640 1 --2- 172.21.15.169:0/533894515 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7efd9c06ae40 0x7efd9c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:26.811 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:26.807+0000 7efdaf7fe640 1 --2- 172.21.15.169:0/533894515 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7efd9c06ae40 0x7efd9c06d300 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7efda0005d00 tx=0x7efda0005c70 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:26.811 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:26.810+0000 7efd97fff640 1 -- 172.21.15.169:0/533894515 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7efdb00ccb90 con 0x7efdb01e2530 2023-10-26T01:31:27.001 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:26.998+0000 7efdb4d47640 1 -- 172.21.15.169:0/533894515 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7efdb006a360 con 0x7efdb01e2530 2023-10-26T01:31:27.002 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:26.999+0000 7efd97fff640 1 -- 172.21.15.169:0/533894515 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7efda8054720 con 0x7efdb01e2530 2023-10-26T01:31:27.002 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 64 2023-10-26T01:31:27.004 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:27.002+0000 7efdb4d47640 1 -- 172.21.15.169:0/533894515 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7efd9c06ae40 msgr2=0x7efd9c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:27.004 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:27.002+0000 7efdb4d47640 1 --2- 172.21.15.169:0/533894515 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7efd9c06ae40 0x7efd9c06d300 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7efda0005d00 tx=0x7efda0005c70 comp rx=0 tx=0).stop 2023-10-26T01:31:27.004 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:27.002+0000 7efdb4d47640 1 -- 172.21.15.169:0/533894515 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efdb01e2530 msgr2=0x7efdb0147cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:27.005 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:27.002+0000 7efdb4d47640 1 --2- 172.21.15.169:0/533894515 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efdb01e2530 0x7efdb0147cf0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7efda8001370 tx=0x7efda800a430 comp rx=0 tx=0).stop 2023-10-26T01:31:27.005 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:27.002+0000 7efdb4d47640 1 -- 172.21.15.169:0/533894515 shutdown_connections 2023-10-26T01:31:27.005 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:27.002+0000 7efdb4d47640 1 --2- 172.21.15.169:0/533894515 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7efd9c06ae40 0x7efd9c06d300 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:27.005 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:27.002+0000 7efdb4d47640 1 --2- 172.21.15.169:0/533894515 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efdb01e2530 0x7efdb0147cf0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:27.005 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:27.002+0000 7efdb4d47640 1 --2- 172.21.15.169:0/533894515 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7efdb00ca180 0x7efdb01e1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:27.006 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:27.002+0000 7efdb4d47640 1 --2- 172.21.15.169:0/533894515 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd980047c0 0x7efdb0145a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:27.006 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:27.002+0000 7efdb4d47640 1 -- 172.21.15.169:0/533894515 >> 172.21.15.169:0/533894515 conn(0x7efdb00af110 msgr2=0x7efdb015ac40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:27.006 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:27.002+0000 7efdb4d47640 1 -- 172.21.15.169:0/533894515 shutdown_connections 2023-10-26T01:31:27.006 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:27.002+0000 7efdb4d47640 1 -- 172.21.15.169:0/533894515 wait complete. 2023-10-26T01:31:27.023 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestClusterFull.test_barrier' 2023-10-26T01:31:27.248 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:27.245+0000 7fa8f238e640 1 -- 172.21.15.169:0/3276738324 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa8ec1592d0 msgr2=0x7fa8ec0aa3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:27.248 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:27.245+0000 7fa8f238e640 1 --2- 172.21.15.169:0/3276738324 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa8ec1592d0 0x7fa8ec0aa3d0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec0b2650 tx=0x7fa8dc02c080 comp rx=0 tx=0).stop 2023-10-26T01:31:27.248 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:27.246+0000 7fa8f238e640 1 -- 172.21.15.169:0/3276738324 shutdown_connections 2023-10-26T01:31:27.248 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:27.246+0000 7fa8f238e640 1 --2- 172.21.15.169:0/3276738324 >> v2:172.21.15.170:3300/0 conn(0x7fa8ec0ac930 0x7fa8ec0cd6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:27.249 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:27.246+0000 7fa8f238e640 1 --2- 172.21.15.169:0/3276738324 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa8ec1592d0 0x7fa8ec0aa3d0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:27.249 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:27.246+0000 7fa8f238e640 1 -- 172.21.15.169:0/3276738324 >> 172.21.15.169:0/3276738324 conn(0x7fa8ec069930 msgr2=0x7fa8ec069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:27.249 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:27.246+0000 7fa8f238e640 1 -- 172.21.15.169:0/3276738324 shutdown_connections 2023-10-26T01:31:27.249 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:27.246+0000 7fa8f238e640 1 -- 172.21.15.169:0/3276738324 wait complete. 2023-10-26T01:31:27.250 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:27.248+0000 7fa8f238e640 1 Processor -- start 2023-10-26T01:31:27.250 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:27.248+0000 7fa8f238e640 1 -- start start 2023-10-26T01:31:27.252 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:27.249+0000 7fa8f238e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa8ec0ac930 0x7fa8ec156a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:27.252 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:27.249+0000 7fa8f238e640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa8ec0cdc30 0x7fa8ec1e3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:27.252 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:27.249+0000 7fa8f238e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa8ec1592d0 0x7fa8ec154e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:27.252 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:27.249+0000 7fa8f238e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8ec155d70 con 0x7fa8ec0ac930 2023-10-26T01:31:27.253 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:27.249+0000 7fa8f238e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8ec1e26e0 con 0x7fa8ec1592d0 2023-10-26T01:31:27.253 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:27.249+0000 7fa8f238e640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8ec1e29c0 con 0x7fa8ec0cdc30 2023-10-26T01:31:27.253 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:27.250+0000 7fa8ebfff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa8ec0ac930 0x7fa8ec156a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:27.253 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:27.250+0000 7fa8ebfff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa8ec0ac930 0x7fa8ec156a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:40610/0 (socket says 172.21.15.169:40610) 2023-10-26T01:31:27.253 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:27.250+0000 7fa8ebfff640 1 -- 172.21.15.169:0/1644104734 learned_addr learned my addr 172.21.15.169:0/1644104734 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:31:27.254 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:27.250+0000 7fa8ebfff640 1 -- 172.21.15.169:0/1644104734 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa8ec0cdc30 msgr2=0x7fa8ec1e3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:27.254 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:27.250+0000 7fa8eb7fe640 1 --2- 172.21.15.169:0/1644104734 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa8ec0cdc30 0x7fa8ec1e3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:27.254 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:27.250+0000 7fa8f092b640 1 --2- 172.21.15.169:0/1644104734 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa8ec1592d0 0x7fa8ec154e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:27.255 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:27.250+0000 7fa8ebfff640 1 --2- 172.21.15.169:0/1644104734 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa8ec0cdc30 0x7fa8ec1e3780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:27.255 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:27.250+0000 7fa8ebfff640 1 -- 172.21.15.169:0/1644104734 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa8ec1592d0 msgr2=0x7fa8ec154e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:27.255 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:27.250+0000 7fa8ebfff640 1 --2- 172.21.15.169:0/1644104734 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa8ec1592d0 0x7fa8ec154e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:27.255 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:27.250+0000 7fa8ebfff640 1 -- 172.21.15.169:0/1644104734 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8dc009840 con 0x7fa8ec0ac930 2023-10-26T01:31:27.257 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:27.251+0000 7fa8ebfff640 1 --2- 172.21.15.169:0/1644104734 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa8ec0ac930 0x7fa8ec156a10 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc036040 tx=0x7fa8dc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:27.257 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:27.251+0000 7fa8e8ff9640 1 -- 172.21.15.169:0/1644104734 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8dc004870 con 0x7fa8ec0ac930 2023-10-26T01:31:27.257 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:27.251+0000 7fa8e8ff9640 1 -- 172.21.15.169:0/1644104734 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa8dc03d520 con 0x7fa8ec0ac930 2023-10-26T01:31:27.257 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:27.251+0000 7fa8e8ff9640 1 -- 172.21.15.169:0/1644104734 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8dc03d910 con 0x7fa8ec0ac930 2023-10-26T01:31:27.258 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:27.251+0000 7fa8f238e640 1 -- 172.21.15.169:0/1644104734 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8ec1e2ca0 con 0x7fa8ec0ac930 2023-10-26T01:31:27.258 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:27.251+0000 7fa8f238e640 1 -- 172.21.15.169:0/1644104734 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8ec1e3260 con 0x7fa8ec0ac930 2023-10-26T01:31:27.258 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:27.253+0000 7fa8f238e640 1 -- 172.21.15.169:0/1644104734 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8ec06c190 con 0x7fa8ec0ac930 2023-10-26T01:31:27.258 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:27.253+0000 7fa8e8ff9640 1 -- 172.21.15.169:0/1644104734 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fa8dc03da70 con 0x7fa8ec0ac930 2023-10-26T01:31:27.259 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:27.254+0000 7fa8e8ff9640 1 --2- 172.21.15.169:0/1644104734 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa8cc06ae40 0x7fa8cc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:27.259 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:27.254+0000 7fa8e8ff9640 1 -- 172.21.15.169:0/1644104734 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa8dc0ac4a0 con 0x7fa8ec0ac930 2023-10-26T01:31:27.259 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:27.255+0000 7fa8eb7fe640 1 --2- 172.21.15.169:0/1644104734 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa8cc06ae40 0x7fa8cc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:27.259 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:27.256+0000 7fa8eb7fe640 1 --2- 172.21.15.169:0/1644104734 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa8cc06ae40 0x7fa8cc06d300 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa8d4005fd0 tx=0x7fa8d4005850 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:27.260 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:27.259+0000 7fa8e8ff9640 1 -- 172.21.15.169:0/1644104734 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa8dc077550 con 0x7fa8ec0ac930 2023-10-26T01:31:27.519 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:27.517+0000 7fa8f238e640 1 -- 172.21.15.169:0/1644104734 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_barrier"]} v 0) v1 -- 0x7fa8ec04db10 con 0x7fa8ec0ac930 2023-10-26T01:31:28.089 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:28.087+0000 7fa8e8ff9640 1 -- 172.21.15.169:0/1644104734 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_barrier"]}]=0 v58) v1 ==== 133+0+0 (secure 0 0 0) 0x7fa8dc0410c0 con 0x7fa8ec0ac930 2023-10-26T01:31:28.092 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:28.090+0000 7fa8f238e640 1 -- 172.21.15.169:0/1644104734 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa8cc06ae40 msgr2=0x7fa8cc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:28.092 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:28.090+0000 7fa8f238e640 1 --2- 172.21.15.169:0/1644104734 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa8cc06ae40 0x7fa8cc06d300 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa8d4005fd0 tx=0x7fa8d4005850 comp rx=0 tx=0).stop 2023-10-26T01:31:28.093 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:28.090+0000 7fa8f238e640 1 -- 172.21.15.169:0/1644104734 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa8ec0ac930 msgr2=0x7fa8ec156a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:28.093 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:28.090+0000 7fa8f238e640 1 --2- 172.21.15.169:0/1644104734 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa8ec0ac930 0x7fa8ec156a10 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc036040 tx=0x7fa8dc004270 comp rx=0 tx=0).stop 2023-10-26T01:31:28.093 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:28.091+0000 7fa8f238e640 1 -- 172.21.15.169:0/1644104734 shutdown_connections 2023-10-26T01:31:28.093 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:28.091+0000 7fa8f238e640 1 --2- 172.21.15.169:0/1644104734 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa8cc06ae40 0x7fa8cc06d300 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:28.094 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:28.091+0000 7fa8f238e640 1 --2- 172.21.15.169:0/1644104734 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa8ec1592d0 0x7fa8ec154e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:28.094 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:28.091+0000 7fa8f238e640 1 --2- 172.21.15.169:0/1644104734 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa8ec0cdc30 0x7fa8ec1e3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:28.094 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:28.091+0000 7fa8f238e640 1 --2- 172.21.15.169:0/1644104734 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa8ec0ac930 0x7fa8ec156a10 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:28.094 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:28.091+0000 7fa8f238e640 1 -- 172.21.15.169:0/1644104734 >> 172.21.15.169:0/1644104734 conn(0x7fa8ec069930 msgr2=0x7fa8ec069d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:28.094 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:28.091+0000 7fa8f238e640 1 -- 172.21.15.169:0/1644104734 shutdown_connections 2023-10-26T01:31:28.095 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:28.091+0000 7fa8f238e640 1 -- 172.21.15.169:0/1644104734 wait complete. 2023-10-26T01:31:28.115 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_objectstore 2023-10-26T01:31:28.347 INFO:teuthology.orchestra.run.smithi169.stdout:{"osd_objectstore":"memstore"} 2023-10-26T01:31:28.347 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_objectstore": "memstore" } 2023-10-26T01:31:28.348 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2023-10-26T01:31:28.568 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:28.565+0000 7f09e538e640 1 -- 172.21.15.169:0/4150247139 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f09e0141cb0 msgr2=0x7f09e015ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:28.569 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:28.565+0000 7f09e538e640 1 --2- 172.21.15.169:0/4150247139 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f09e0141cb0 0x7f09e015ca20 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f09c8009830 tx=0x7f09c802bd60 comp rx=0 tx=0).stop 2023-10-26T01:31:28.569 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:28.565+0000 7f09e538e640 1 -- 172.21.15.169:0/4150247139 shutdown_connections 2023-10-26T01:31:28.569 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:28.565+0000 7f09e538e640 1 --2- 172.21.15.169:0/4150247139 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f09e0141cb0 0x7f09e015ca20 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:28.569 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:28.565+0000 7f09e538e640 1 -- 172.21.15.169:0/4150247139 >> 172.21.15.169:0/4150247139 conn(0x7f09e0069930 msgr2=0x7f09e0069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:28.570 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:28.565+0000 7f09e538e640 1 -- 172.21.15.169:0/4150247139 shutdown_connections 2023-10-26T01:31:28.570 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:28.566+0000 7f09e538e640 1 -- 172.21.15.169:0/4150247139 wait complete. 2023-10-26T01:31:28.571 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:28.567+0000 7f09e538e640 1 Processor -- start 2023-10-26T01:31:28.571 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:28.567+0000 7f09e538e640 1 -- start start 2023-10-26T01:31:28.572 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:28.568+0000 7f09e538e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f09e00ccda0 0x7f09e014e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:28.572 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:28.568+0000 7f09e538e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f09e00cd7e0 0x7f09e01f12c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:28.572 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:28.568+0000 7f09e538e640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f09e0141cb0 0x7f09e014b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:28.572 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:28.568+0000 7f09e538e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09e00c8830 con 0x7f09e00cd7e0 2023-10-26T01:31:28.572 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:28.568+0000 7f09e538e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09e00c7c20 con 0x7f09e00ccda0 2023-10-26T01:31:28.573 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:28.568+0000 7f09e538e640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09e00c7f00 con 0x7f09e0141cb0 2023-10-26T01:31:28.573 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:28.569+0000 7f09d7fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f09e00cd7e0 0x7f09e01f12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:28.573 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:28.569+0000 7f09d7fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f09e00cd7e0 0x7f09e01f12c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:40642/0 (socket says 172.21.15.169:40642) 2023-10-26T01:31:28.574 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:28.569+0000 7f09d7fff640 1 -- 172.21.15.169:0/829559802 learned_addr learned my addr 172.21.15.169:0/829559802 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:31:28.574 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:28.569+0000 7f09deffd640 1 --2- 172.21.15.169:0/829559802 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f09e00ccda0 0x7f09e014e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:28.576 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:28.569+0000 7f09df7fe640 1 --2- 172.21.15.169:0/829559802 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f09e0141cb0 0x7f09e014b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:28.576 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:28.569+0000 7f09d7fff640 1 -- 172.21.15.169:0/829559802 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f09e0141cb0 msgr2=0x7f09e014b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:28.576 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:28.569+0000 7f09d7fff640 1 --2- 172.21.15.169:0/829559802 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f09e0141cb0 0x7f09e014b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:28.576 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:28.569+0000 7f09d7fff640 1 -- 172.21.15.169:0/829559802 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f09e00ccda0 msgr2=0x7f09e014e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:28.577 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:28.569+0000 7f09d7fff640 1 --2- 172.21.15.169:0/829559802 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f09e00ccda0 0x7f09e014e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:28.577 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:28.569+0000 7f09d7fff640 1 -- 172.21.15.169:0/829559802 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09c8009490 con 0x7f09e00cd7e0 2023-10-26T01:31:28.577 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:28.569+0000 7f09df7fe640 1 --2- 172.21.15.169:0/829559802 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f09e0141cb0 0x7f09e014b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:31:28.577 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:28.570+0000 7f09d7fff640 1 --2- 172.21.15.169:0/829559802 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f09e00cd7e0 0x7f09e01f12c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f09c8009780 tx=0x7f09c80038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:28.577 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:28.570+0000 7f09d77fe640 1 -- 172.21.15.169:0/829559802 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c8003ea0 con 0x7f09e00cd7e0 2023-10-26T01:31:28.578 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:28.570+0000 7f09d77fe640 1 -- 172.21.15.169:0/829559802 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f09c8034440 con 0x7f09e00cd7e0 2023-10-26T01:31:28.578 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:28.570+0000 7f09d77fe640 1 -- 172.21.15.169:0/829559802 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c8034850 con 0x7f09e00cd7e0 2023-10-26T01:31:28.578 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:28.570+0000 7f09e538e640 1 -- 172.21.15.169:0/829559802 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09e00c81e0 con 0x7f09e00cd7e0 2023-10-26T01:31:28.578 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:28.570+0000 7f09e538e640 1 -- 172.21.15.169:0/829559802 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09e01f1800 con 0x7f09e00cd7e0 2023-10-26T01:31:28.579 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:28.571+0000 7f09e538e640 1 -- 172.21.15.169:0/829559802 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09e00c89e0 con 0x7f09e00cd7e0 2023-10-26T01:31:28.579 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:28.572+0000 7f09d77fe640 1 -- 172.21.15.169:0/829559802 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f09c803b040 con 0x7f09e00cd7e0 2023-10-26T01:31:28.579 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:28.573+0000 7f09d77fe640 1 --2- 172.21.15.169:0/829559802 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f09c006ae70 0x7f09c006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:28.579 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:28.573+0000 7f09d77fe640 1 -- 172.21.15.169:0/829559802 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f09c80aba20 con 0x7f09e00cd7e0 2023-10-26T01:31:28.580 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:28.574+0000 7f09deffd640 1 --2- 172.21.15.169:0/829559802 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f09c006ae70 0x7f09c006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:28.580 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:28.575+0000 7f09deffd640 1 --2- 172.21.15.169:0/829559802 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f09c006ae70 0x7f09c006d330 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f09d00011e0 tx=0x7f09d0009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:28.581 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:28.578+0000 7f09d77fe640 1 -- 172.21.15.169:0/829559802 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f09e00c89e0 con 0x7f09e00cd7e0 2023-10-26T01:31:28.770 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:28.768+0000 7f09e538e640 1 -- 172.21.15.169:0/829559802 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "config dump"} v 0) v1 -- 0x7f09e004db10 con 0x7f09e00cd7e0 2023-10-26T01:31:28.771 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:28.769+0000 7f09d77fe640 1 -- 172.21.15.169:0/829559802 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v18) v1 ==== 59+0+1116 (secure 0 0 0) 0x7f09c8039020 con 0x7f09e00cd7e0 2023-10-26T01:31:28.772 INFO:teuthology.orchestra.run.smithi169.stdout:WHO MASK LEVEL OPTION VALUE RO 2023-10-26T01:31:28.772 INFO:teuthology.orchestra.run.smithi169.stdout:mon advanced mon_op_complaint_time 120 2023-10-26T01:31:28.772 INFO:teuthology.orchestra.run.smithi169.stdout:osd advanced osd_op_complaint_time 180.000000 2023-10-26T01:31:28.772 INFO:teuthology.orchestra.run.smithi169.stdout:mds advanced debug_mds 20/20 2023-10-26T01:31:28.772 INFO:teuthology.orchestra.run.smithi169.stdout:mds advanced debug_mds_balancer 20/20 2023-10-26T01:31:28.773 INFO:teuthology.orchestra.run.smithi169.stdout:mds advanced debug_ms 1/1 2023-10-26T01:31:28.773 INFO:teuthology.orchestra.run.smithi169.stdout:mds dev mds_debug_frag true 2023-10-26T01:31:28.773 INFO:teuthology.orchestra.run.smithi169.stdout:mds dev mds_debug_scatterstat true 2023-10-26T01:31:28.773 INFO:teuthology.orchestra.run.smithi169.stdout:mds advanced mds_op_complaint_time 180.000000 2023-10-26T01:31:28.773 INFO:teuthology.orchestra.run.smithi169.stdout:mds dev mds_verify_scatter true 2023-10-26T01:31:28.774 INFO:teuthology.orchestra.run.smithi169.stdout:mds advanced osd_op_complaint_time 180.000000 2023-10-26T01:31:28.774 INFO:teuthology.orchestra.run.smithi169.stdout:mds advanced rados_mon_op_timeout 900 2023-10-26T01:31:28.774 INFO:teuthology.orchestra.run.smithi169.stdout:mds advanced rados_osd_op_timeout 900 2023-10-26T01:31:28.774 INFO:teuthology.orchestra.run.smithi169.stdout:client advanced client_mount_timeout 600 2023-10-26T01:31:28.774 INFO:teuthology.orchestra.run.smithi169.stdout:client advanced debug_client 20/20 2023-10-26T01:31:28.775 INFO:teuthology.orchestra.run.smithi169.stdout:client advanced debug_ms 1/1 2023-10-26T01:31:28.775 INFO:teuthology.orchestra.run.smithi169.stdout:client advanced rados_mon_op_timeout 900 2023-10-26T01:31:28.775 INFO:teuthology.orchestra.run.smithi169.stdout:client advanced rados_osd_op_timeout 900 2023-10-26T01:31:28.776 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:28.772+0000 7f09e538e640 1 -- 172.21.15.169:0/829559802 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f09c006ae70 msgr2=0x7f09c006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:28.777 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:28.772+0000 7f09e538e640 1 --2- 172.21.15.169:0/829559802 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f09c006ae70 0x7f09c006d330 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f09d00011e0 tx=0x7f09d0009040 comp rx=0 tx=0).stop 2023-10-26T01:31:28.777 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:28.772+0000 7f09e538e640 1 -- 172.21.15.169:0/829559802 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f09e00cd7e0 msgr2=0x7f09e01f12c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:28.777 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:28.772+0000 7f09e538e640 1 --2- 172.21.15.169:0/829559802 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f09e00cd7e0 0x7f09e01f12c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f09c8009780 tx=0x7f09c80038a0 comp rx=0 tx=0).stop 2023-10-26T01:31:28.777 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:28.772+0000 7f09e538e640 1 -- 172.21.15.169:0/829559802 shutdown_connections 2023-10-26T01:31:28.777 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:28.772+0000 7f09e538e640 1 --2- 172.21.15.169:0/829559802 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f09c006ae70 0x7f09c006d330 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:28.778 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:28.772+0000 7f09e538e640 1 --2- 172.21.15.169:0/829559802 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f09e00cd7e0 0x7f09e01f12c0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:28.778 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:28.772+0000 7f09e538e640 1 --2- 172.21.15.169:0/829559802 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f09e0141cb0 0x7f09e014b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:28.778 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:28.772+0000 7f09e538e640 1 --2- 172.21.15.169:0/829559802 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f09e00ccda0 0x7f09e014e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:28.778 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:28.772+0000 7f09e538e640 1 -- 172.21.15.169:0/829559802 >> 172.21.15.169:0/829559802 conn(0x7f09e0069930 msgr2=0x7f09e0157e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:28.778 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:28.772+0000 7f09e538e640 1 -- 172.21.15.169:0/829559802 shutdown_connections 2023-10-26T01:31:28.779 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:28.772+0000 7f09e538e640 1 -- 172.21.15.169:0/829559802 wait complete. 2023-10-26T01:31:28.793 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-26T01:31:29.011 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:29.008+0000 7f943e421640 1 -- 172.21.15.169:0/4018638333 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9424003970 msgr2=0x7f9424003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:29.012 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:29.008+0000 7f943e421640 1 --2- 172.21.15.169:0/4018638333 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9424003970 0x7f9424003d90 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f9434009b80 tx=0x7f94240057f0 comp rx=0 tx=0).stop 2023-10-26T01:31:29.012 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:29.008+0000 7f943dc20640 1 -- 172.21.15.169:0/4018638333 reap_dead start 2023-10-26T01:31:29.012 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:29.008+0000 7f943e421640 1 -- 172.21.15.169:0/4018638333 shutdown_connections 2023-10-26T01:31:29.012 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:29.008+0000 7f943e421640 1 -- 172.21.15.169:0/4018638333 >> 172.21.15.169:0/4018638333 conn(0x7f94380af110 msgr2=0x7f94380af520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:29.012 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:29.008+0000 7f943e421640 1 -- 172.21.15.169:0/4018638333 shutdown_connections 2023-10-26T01:31:29.013 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:29.008+0000 7f943e421640 1 -- 172.21.15.169:0/4018638333 wait complete. 2023-10-26T01:31:29.013 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:29.010+0000 7f943e421640 1 Processor -- start 2023-10-26T01:31:29.013 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:29.010+0000 7f943e421640 1 -- start start 2023-10-26T01:31:29.015 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:29.011+0000 7f943e421640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f94381f15b0 0x7f94380c8d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:29.016 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:29.011+0000 7f943e421640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f94381f1990 0x7f94380ca0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:29.016 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:29.011+0000 7f943e421640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f94380cb930 0x7f9438070aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:29.016 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:29.011+0000 7f943e421640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94380cb4a0 con 0x7f94380cb930 2023-10-26T01:31:29.016 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:29.011+0000 7f943e421640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94380ca610 con 0x7f94381f15b0 2023-10-26T01:31:29.016 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:29.011+0000 7f943e421640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94380ca890 con 0x7f94381f1990 2023-10-26T01:31:29.017 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:29.011+0000 7f943dc20640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f94380cb930 0x7f9438070aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:29.017 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:29.012+0000 7f943dc20640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f94380cb930 0x7f9438070aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:40652/0 (socket says 172.21.15.169:40652) 2023-10-26T01:31:29.017 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:29.012+0000 7f943dc20640 1 -- 172.21.15.169:0/2415833777 learned_addr learned my addr 172.21.15.169:0/2415833777 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:31:29.017 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:29.012+0000 7f943cc1e640 1 --2- 172.21.15.169:0/2415833777 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f94381f1990 0x7f94380ca0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:29.017 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:29.012+0000 7f943d41f640 1 --2- 172.21.15.169:0/2415833777 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f94381f15b0 0x7f94380c8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:29.018 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:29.012+0000 7f943dc20640 1 -- 172.21.15.169:0/2415833777 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f94381f1990 msgr2=0x7f94380ca0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:29.018 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:29.012+0000 7f943dc20640 1 --2- 172.21.15.169:0/2415833777 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f94381f1990 0x7f94380ca0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:29.018 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:29.012+0000 7f943dc20640 1 -- 172.21.15.169:0/2415833777 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f94381f15b0 msgr2=0x7f94380c8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:29.018 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:29.012+0000 7f943dc20640 1 --2- 172.21.15.169:0/2415833777 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f94381f15b0 0x7f94380c8d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:29.018 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:29.012+0000 7f943dc20640 1 -- 172.21.15.169:0/2415833777 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94380cab70 con 0x7f94380cb930 2023-10-26T01:31:29.019 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:29.012+0000 7f943d41f640 1 --2- 172.21.15.169:0/2415833777 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f94381f15b0 0x7f94380c8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:31:29.019 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:29.012+0000 7f943dc20640 1 --2- 172.21.15.169:0/2415833777 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f94380cb930 0x7f9438070aa0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f94300039f0 tx=0x7f943000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:29.019 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:29.012+0000 7f942dffb640 1 -- 172.21.15.169:0/2415833777 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9430015030 con 0x7f94380cb930 2023-10-26T01:31:29.019 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:29.013+0000 7f942dffb640 1 -- 172.21.15.169:0/2415833777 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f943000b8c0 con 0x7f94380cb930 2023-10-26T01:31:29.019 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:29.013+0000 7f942dffb640 1 -- 172.21.15.169:0/2415833777 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f943000bcd0 con 0x7f94380cb930 2023-10-26T01:31:29.020 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:29.013+0000 7f943e421640 1 -- 172.21.15.169:0/2415833777 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94380cae50 con 0x7f94380cb930 2023-10-26T01:31:29.020 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:29.013+0000 7f943e421640 1 -- 172.21.15.169:0/2415833777 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94380cbde0 con 0x7f94380cb930 2023-10-26T01:31:29.021 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:29.014+0000 7f943e421640 1 -- 172.21.15.169:0/2415833777 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94380cb650 con 0x7f94380cb930 2023-10-26T01:31:29.022 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:29.015+0000 7f942dffb640 1 -- 172.21.15.169:0/2415833777 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f9430018070 con 0x7f94380cb930 2023-10-26T01:31:29.022 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:29.016+0000 7f942dffb640 1 --2- 172.21.15.169:0/2415833777 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f941c06ae70 0x7f941c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:29.022 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:29.016+0000 7f942dffb640 1 -- 172.21.15.169:0/2415833777 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9430016070 con 0x7f94380cb930 2023-10-26T01:31:29.022 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:29.016+0000 7f943d41f640 1 --2- 172.21.15.169:0/2415833777 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f941c06ae70 0x7f941c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:29.022 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:29.017+0000 7f943d41f640 1 --2- 172.21.15.169:0/2415833777 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f941c06ae70 0x7f941c06d330 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9434036000 tx=0x7f94340046c0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:29.023 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:29.020+0000 7f942dffb640 1 -- 172.21.15.169:0/2415833777 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f94380cb650 con 0x7f94380cb930 2023-10-26T01:31:29.224 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:29.221+0000 7f943e421640 1 -- 172.21.15.169:0/2415833777 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f9438073c10 con 0x7f94380cb930 2023-10-26T01:31:29.230 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:29.227+0000 7f942dffb640 1 -- 172.21.15.169:0/2415833777 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v19)=0 v19) v1 ==== 122+0+0 (secure 0 0 0) 0x7f94300549d0 con 0x7f94380cb930 2023-10-26T01:31:29.232 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:29.229+0000 7f943e421640 1 -- 172.21.15.169:0/2415833777 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f941c06ae70 msgr2=0x7f941c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:29.232 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:29.229+0000 7f943e421640 1 --2- 172.21.15.169:0/2415833777 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f941c06ae70 0x7f941c06d330 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9434036000 tx=0x7f94340046c0 comp rx=0 tx=0).stop 2023-10-26T01:31:29.233 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:29.229+0000 7f943e421640 1 -- 172.21.15.169:0/2415833777 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f94380cb930 msgr2=0x7f9438070aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:29.233 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:29.229+0000 7f943e421640 1 --2- 172.21.15.169:0/2415833777 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f94380cb930 0x7f9438070aa0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f94300039f0 tx=0x7f943000de10 comp rx=0 tx=0).stop 2023-10-26T01:31:29.233 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:29.230+0000 7f943e421640 1 -- 172.21.15.169:0/2415833777 shutdown_connections 2023-10-26T01:31:29.233 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:29.230+0000 7f943e421640 1 --2- 172.21.15.169:0/2415833777 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f941c06ae70 0x7f941c06d330 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:29.234 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:29.230+0000 7f943e421640 1 --2- 172.21.15.169:0/2415833777 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f94380cb930 0x7f9438070aa0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:29.234 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:29.230+0000 7f943e421640 1 --2- 172.21.15.169:0/2415833777 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f94381f1990 0x7f94380ca0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:29.234 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:29.230+0000 7f943e421640 1 --2- 172.21.15.169:0/2415833777 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f94381f15b0 0x7f94380c8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:29.234 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:29.230+0000 7f943e421640 1 -- 172.21.15.169:0/2415833777 >> 172.21.15.169:0/2415833777 conn(0x7f94380af110 msgr2=0x7f94380ae2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:29.234 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:29.230+0000 7f943e421640 1 -- 172.21.15.169:0/2415833777 shutdown_connections 2023-10-26T01:31:29.235 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:29.230+0000 7f943e421640 1 -- 172.21.15.169:0/2415833777 wait complete. 2023-10-26T01:31:29.251 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-26T01:31:29.252 DEBUG:teuthology.orchestra.run.smithi169:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-26T01:31:29.361 DEBUG:teuthology.orchestra.run.smithi170:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-26T01:31:29.485 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-26T01:31:29.486 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:31:29.487 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:31:29.541 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:31:29.542 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:31:29.593 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:31:29.594 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:31:29.648 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:31:29.873 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:29.870+0000 7f5802459640 1 -- 172.21.15.169:0/3628417207 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f57fc0c67b0 msgr2=0x7f57fc0cb300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:29.873 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:29.870+0000 7f5802459640 1 --2- 172.21.15.169:0/3628417207 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f57fc0c67b0 0x7f57fc0cb300 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f57fc15ba00 tx=0x7f57f402c260 comp rx=0 tx=0).stop 2023-10-26T01:31:29.873 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:29.870+0000 7f5802459640 1 -- 172.21.15.169:0/3628417207 shutdown_connections 2023-10-26T01:31:29.874 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:29.870+0000 7f5802459640 1 --2- 172.21.15.169:0/3628417207 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f57fc0c67b0 0x7f57fc0cb300 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:29.874 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:29.870+0000 7f5802459640 1 --2- 172.21.15.169:0/3628417207 >> v2:172.21.15.170:3300/0 conn(0x7f57fc155c40 0x7f57fc0c6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:29.874 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:29.870+0000 7f5802459640 1 -- 172.21.15.169:0/3628417207 >> 172.21.15.169:0/3628417207 conn(0x7f57fc0aedd0 msgr2=0x7f57fc0af1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:29.874 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:29.870+0000 7f5802459640 1 -- 172.21.15.169:0/3628417207 shutdown_connections 2023-10-26T01:31:29.875 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:29.870+0000 7f5802459640 1 -- 172.21.15.169:0/3628417207 wait complete. 2023-10-26T01:31:29.880 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:29.872+0000 7f5802459640 1 Processor -- start 2023-10-26T01:31:29.880 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:29.872+0000 7f5802459640 1 -- start start 2023-10-26T01:31:29.880 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:29.873+0000 7f5802459640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f57fc0c67b0 0x7f57fc145e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:29.881 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:29.873+0000 7f5802459640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f57fc155860 0x7f57fc1490c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:29.881 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:29.873+0000 7f5802459640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f57fc071680 0x7f57fc0acaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:29.881 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:29.873+0000 7f5802459640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57fc1f27d0 con 0x7f57fc0c67b0 2023-10-26T01:31:29.881 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:29.873+0000 7f5802459640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57fc146360 con 0x7f57fc071680 2023-10-26T01:31:29.882 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:29.873+0000 7f5802459640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57fc1f1940 con 0x7f57fc155860 2023-10-26T01:31:29.882 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:29.873+0000 7f5801457640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f57fc0c67b0 0x7f57fc145e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:29.882 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:29.873+0000 7f5801457640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f57fc0c67b0 0x7f57fc145e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:40668/0 (socket says 172.21.15.169:40668) 2023-10-26T01:31:29.882 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:29.873+0000 7f5801457640 1 -- 172.21.15.169:0/900134395 learned_addr learned my addr 172.21.15.169:0/900134395 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:31:29.882 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:29.873+0000 7f5801457640 1 -- 172.21.15.169:0/900134395 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f57fc155860 msgr2=0x7f57fc1490c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:29.883 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:29.873+0000 7f5801457640 1 --2- 172.21.15.169:0/900134395 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f57fc155860 0x7f57fc1490c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:29.883 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:29.873+0000 7f5801457640 1 -- 172.21.15.169:0/900134395 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f57fc071680 msgr2=0x7f57fc0acaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:29.883 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:29.873+0000 7f5801c58640 1 --2- 172.21.15.169:0/900134395 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f57fc071680 0x7f57fc0acaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:29.883 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:29.873+0000 7f5801457640 1 --2- 172.21.15.169:0/900134395 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f57fc071680 0x7f57fc0acaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:29.884 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:29.873+0000 7f5801457640 1 -- 172.21.15.169:0/900134395 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57f4009900 con 0x7f57fc0c67b0 2023-10-26T01:31:29.884 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:29.874+0000 7f5801457640 1 --2- 172.21.15.169:0/900134395 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f57fc0c67b0 0x7f57fc145e20 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f57f800c410 tx=0x7f57f800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:29.884 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:29.874+0000 7f57f1ffb640 1 -- 172.21.15.169:0/900134395 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f80035f0 con 0x7f57fc0c67b0 2023-10-26T01:31:29.884 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:29.874+0000 7f57f1ffb640 1 -- 172.21.15.169:0/900134395 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f57f8003750 con 0x7f57fc0c67b0 2023-10-26T01:31:29.885 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:29.874+0000 7f57f1ffb640 1 -- 172.21.15.169:0/900134395 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f8003ba0 con 0x7f57fc0c67b0 2023-10-26T01:31:29.885 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:29.874+0000 7f5802459640 1 -- 172.21.15.169:0/900134395 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57fc149600 con 0x7f57fc0c67b0 2023-10-26T01:31:29.885 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:29.874+0000 7f5802459640 1 -- 172.21.15.169:0/900134395 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57fc1f1ea0 con 0x7f57fc0c67b0 2023-10-26T01:31:29.885 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:29.875+0000 7f5802459640 1 -- 172.21.15.169:0/900134395 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57fc073c10 con 0x7f57fc0c67b0 2023-10-26T01:31:29.886 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:29.876+0000 7f57f1ffb640 1 -- 172.21.15.169:0/900134395 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f57f8003d00 con 0x7f57fc0c67b0 2023-10-26T01:31:29.886 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:29.877+0000 7f57f1ffb640 1 --2- 172.21.15.169:0/900134395 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f57e006ae70 0x7f57e006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:29.886 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:29.878+0000 7f57f1ffb640 1 -- 172.21.15.169:0/900134395 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f57f8088a00 con 0x7f57fc0c67b0 2023-10-26T01:31:29.887 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:29.878+0000 7f5800c56640 1 --2- 172.21.15.169:0/900134395 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f57e006ae70 0x7f57e006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:29.887 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:29.879+0000 7f5800c56640 1 --2- 172.21.15.169:0/900134395 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f57e006ae70 0x7f57e006d330 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f57fc0cb650 tx=0x7f57ec005b70 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:29.887 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:29.882+0000 7f57f1ffb640 1 -- 172.21.15.169:0/900134395 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f57f8053a30 con 0x7f57fc0c67b0 2023-10-26T01:31:30.077 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.072+0000 7f5802459640 1 -- 172.21.15.169:0/900134395 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f57fc0c7ff0 con 0x7f57fc0c67b0 2023-10-26T01:31:30.078 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.075+0000 7f57f1ffb640 1 -- 172.21.15.169:0/900134395 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16801 (secure 0 0 0) 0x7f57f8017090 con 0x7f57fc0c67b0 2023-10-26T01:31:30.079 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:31:30.079 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":22,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:31:29.088284+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-26T01:30:39.437631+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-26T01:30:40.561774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":2}]},{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.24","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:31:30.081 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.078+0000 7f5802459640 1 -- 172.21.15.169:0/900134395 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f57e006ae70 msgr2=0x7f57e006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:30.082 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.078+0000 7f5802459640 1 --2- 172.21.15.169:0/900134395 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f57e006ae70 0x7f57e006d330 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f57fc0cb650 tx=0x7f57ec005b70 comp rx=0 tx=0).stop 2023-10-26T01:31:30.082 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.078+0000 7f5802459640 1 -- 172.21.15.169:0/900134395 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f57fc0c67b0 msgr2=0x7f57fc145e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:30.082 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.078+0000 7f5802459640 1 --2- 172.21.15.169:0/900134395 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f57fc0c67b0 0x7f57fc145e20 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f57f800c410 tx=0x7f57f800c8d0 comp rx=0 tx=0).stop 2023-10-26T01:31:30.082 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.078+0000 7f5802459640 1 -- 172.21.15.169:0/900134395 shutdown_connections 2023-10-26T01:31:30.083 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.078+0000 7f5802459640 1 --2- 172.21.15.169:0/900134395 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f57e006ae70 0x7f57e006d330 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:30.083 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.078+0000 7f5802459640 1 --2- 172.21.15.169:0/900134395 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f57fc071680 0x7f57fc0acaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:30.083 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.078+0000 7f5802459640 1 --2- 172.21.15.169:0/900134395 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f57fc155860 0x7f57fc1490c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:30.083 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.078+0000 7f5802459640 1 --2- 172.21.15.169:0/900134395 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f57fc0c67b0 0x7f57fc145e20 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:30.084 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.079+0000 7f5802459640 1 -- 172.21.15.169:0/900134395 >> 172.21.15.169:0/900134395 conn(0x7f57fc0aedd0 msgr2=0x7f57fc0af1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:30.084 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.079+0000 7f5802459640 1 -- 172.21.15.169:0/900134395 shutdown_connections 2023-10-26T01:31:30.084 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.079+0000 7f5802459640 1 -- 172.21.15.169:0/900134395 wait complete. 2023-10-26T01:31:30.102 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T01:30:32.200808+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-26T01:30:39.437631+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-26T01:30:40.561774+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T01:31:30.103 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T01:31:30.323 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.320+0000 7faf43141640 1 -- 172.21.15.169:0/1086882447 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faf3c15b6a0 msgr2=0x7faf3c0cd690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:30.323 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.320+0000 7faf43141640 1 --2- 172.21.15.169:0/1086882447 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faf3c15b6a0 0x7faf3c0cd690 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7faf2c005fd0 tx=0x7faf2c02bd10 comp rx=0 tx=0).stop 2023-10-26T01:31:30.324 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.320+0000 7faf43141640 1 -- 172.21.15.169:0/1086882447 shutdown_connections 2023-10-26T01:31:30.324 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.320+0000 7faf43141640 1 --2- 172.21.15.169:0/1086882447 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7faf3c0acae0 0x7faf3c06aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:30.324 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.320+0000 7faf43141640 1 --2- 172.21.15.169:0/1086882447 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faf3c15b6a0 0x7faf3c0cd690 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:30.324 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.320+0000 7faf43141640 1 -- 172.21.15.169:0/1086882447 >> 172.21.15.169:0/1086882447 conn(0x7faf3c069930 msgr2=0x7faf3c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:30.324 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.320+0000 7faf43141640 1 -- 172.21.15.169:0/1086882447 shutdown_connections 2023-10-26T01:31:30.325 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.320+0000 7faf43141640 1 -- 172.21.15.169:0/1086882447 wait complete. 2023-10-26T01:31:30.326 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.322+0000 7faf43141640 1 Processor -- start 2023-10-26T01:31:30.326 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.322+0000 7faf43141640 1 -- start start 2023-10-26T01:31:30.326 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.323+0000 7faf43141640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faf3c0acae0 0x7faf3c1f6020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:30.326 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.323+0000 7faf43141640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faf3c0cdbd0 0x7faf3c1f1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:30.327 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.323+0000 7faf43141640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7faf3c15b6a0 0x7faf3c155750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:30.327 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.323+0000 7faf43141640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf3c1f6560 con 0x7faf3c0acae0 2023-10-26T01:31:30.327 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.323+0000 7faf43141640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf3c1f1fc0 con 0x7faf3c0cdbd0 2023-10-26T01:31:30.327 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.323+0000 7faf43141640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf3c0ca240 con 0x7faf3c15b6a0 2023-10-26T01:31:30.327 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.323+0000 7faf40edd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faf3c0acae0 0x7faf3c1f6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:30.327 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.323+0000 7faf40edd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faf3c0acae0 0x7faf3c1f6020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:40686/0 (socket says 172.21.15.169:40686) 2023-10-26T01:31:30.328 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.323+0000 7faf40edd640 1 -- 172.21.15.169:0/316112161 learned_addr learned my addr 172.21.15.169:0/316112161 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:31:30.329 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.324+0000 7faf40edd640 1 -- 172.21.15.169:0/316112161 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7faf3c15b6a0 msgr2=0x7faf3c155750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:30.329 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.324+0000 7faf40edd640 1 --2- 172.21.15.169:0/316112161 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7faf3c15b6a0 0x7faf3c155750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:30.330 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.324+0000 7faf3bfff640 1 --2- 172.21.15.169:0/316112161 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faf3c0cdbd0 0x7faf3c1f1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:30.330 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.324+0000 7faf40edd640 1 -- 172.21.15.169:0/316112161 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faf3c0cdbd0 msgr2=0x7faf3c1f1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:30.330 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.324+0000 7faf40edd640 1 --2- 172.21.15.169:0/316112161 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faf3c0cdbd0 0x7faf3c1f1a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:30.330 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.324+0000 7faf40edd640 1 -- 172.21.15.169:0/316112161 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf2c009560 con 0x7faf3c0acae0 2023-10-26T01:31:30.330 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.324+0000 7faf40edd640 1 --2- 172.21.15.169:0/316112161 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faf3c0acae0 0x7faf3c1f6020 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7faf2c009850 tx=0x7faf2c02da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:30.331 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.324+0000 7faf397fa640 1 -- 172.21.15.169:0/316112161 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf2c02de90 con 0x7faf3c0acae0 2023-10-26T01:31:30.331 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.324+0000 7faf397fa640 1 -- 172.21.15.169:0/316112161 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7faf2c0036e0 con 0x7faf3c0acae0 2023-10-26T01:31:30.331 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.324+0000 7faf43141640 1 -- 172.21.15.169:0/316112161 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf3c0ca520 con 0x7faf3c0acae0 2023-10-26T01:31:30.331 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.324+0000 7faf397fa640 1 -- 172.21.15.169:0/316112161 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf2c003a90 con 0x7faf3c0acae0 2023-10-26T01:31:30.331 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.325+0000 7faf43141640 1 -- 172.21.15.169:0/316112161 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf3c0caae0 con 0x7faf3c0acae0 2023-10-26T01:31:30.332 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.326+0000 7faf43141640 1 -- 172.21.15.169:0/316112161 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf3c066af0 con 0x7faf3c0acae0 2023-10-26T01:31:30.332 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.327+0000 7faf397fa640 1 -- 172.21.15.169:0/316112161 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7faf2c003bf0 con 0x7faf3c0acae0 2023-10-26T01:31:30.332 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.327+0000 7faf397fa640 1 --2- 172.21.15.169:0/316112161 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7faf1c06ae40 0x7faf1c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:30.333 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.328+0000 7faf397fa640 1 -- 172.21.15.169:0/316112161 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7faf2c0ac220 con 0x7faf3c0acae0 2023-10-26T01:31:30.333 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.328+0000 7faf3bfff640 1 --2- 172.21.15.169:0/316112161 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7faf1c06ae40 0x7faf1c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:30.333 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.329+0000 7faf3bfff640 1 --2- 172.21.15.169:0/316112161 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7faf1c06ae40 0x7faf1c06d300 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7faf24007fd0 tx=0x7faf24009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:30.334 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.332+0000 7faf397fa640 1 -- 172.21.15.169:0/316112161 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7faf2c077400 con 0x7faf3c0acae0 2023-10-26T01:31:30.523 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.521+0000 7faf43141640 1 -- 172.21.15.169:0/316112161 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7faf3c04da70 con 0x7faf3c0acae0 2023-10-26T01:31:30.524 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.521+0000 7faf397fa640 1 -- 172.21.15.169:0/316112161 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7faf2c07aa20 con 0x7faf3c0acae0 2023-10-26T01:31:30.524 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 1 2023-10-26T01:31:30.528 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.525+0000 7faf43141640 1 -- 172.21.15.169:0/316112161 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7faf1c06ae40 msgr2=0x7faf1c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:30.528 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.525+0000 7faf43141640 1 --2- 172.21.15.169:0/316112161 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7faf1c06ae40 0x7faf1c06d300 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7faf24007fd0 tx=0x7faf24009040 comp rx=0 tx=0).stop 2023-10-26T01:31:30.528 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.525+0000 7faf43141640 1 -- 172.21.15.169:0/316112161 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faf3c0acae0 msgr2=0x7faf3c1f6020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:30.529 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.525+0000 7faf43141640 1 --2- 172.21.15.169:0/316112161 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faf3c0acae0 0x7faf3c1f6020 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7faf2c009850 tx=0x7faf2c02da90 comp rx=0 tx=0).stop 2023-10-26T01:31:30.529 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.525+0000 7faf43141640 1 -- 172.21.15.169:0/316112161 shutdown_connections 2023-10-26T01:31:30.529 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.525+0000 7faf43141640 1 --2- 172.21.15.169:0/316112161 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7faf1c06ae40 0x7faf1c06d300 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:30.529 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.525+0000 7faf43141640 1 --2- 172.21.15.169:0/316112161 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7faf3c15b6a0 0x7faf3c155750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:30.529 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.525+0000 7faf43141640 1 --2- 172.21.15.169:0/316112161 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faf3c0cdbd0 0x7faf3c1f1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:30.530 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.525+0000 7faf43141640 1 --2- 172.21.15.169:0/316112161 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faf3c0acae0 0x7faf3c1f6020 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:30.530 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.525+0000 7faf43141640 1 -- 172.21.15.169:0/316112161 >> 172.21.15.169:0/316112161 conn(0x7faf3c069930 msgr2=0x7faf3c0acf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:30.530 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.525+0000 7faf43141640 1 -- 172.21.15.169:0/316112161 shutdown_connections 2023-10-26T01:31:30.530 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.525+0000 7faf43141640 1 -- 172.21.15.169:0/316112161 wait complete. 2023-10-26T01:31:30.545 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-26T01:31:30.768 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.765+0000 7fd15f197640 1 -- 172.21.15.169:0/725051673 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd144004ab0 msgr2=0x7fd144004ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:30.769 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.765+0000 7fd15f197640 1 --2- 172.21.15.169:0/725051673 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd144004ab0 0x7fd144004ed0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd14800c610 tx=0x7fd144006900 comp rx=0 tx=0).stop 2023-10-26T01:31:30.769 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.765+0000 7fd15d734640 1 -- 172.21.15.169:0/725051673 reap_dead start 2023-10-26T01:31:30.769 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.766+0000 7fd15f197640 1 -- 172.21.15.169:0/725051673 shutdown_connections 2023-10-26T01:31:30.769 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.766+0000 7fd15f197640 1 -- 172.21.15.169:0/725051673 >> 172.21.15.169:0/725051673 conn(0x7fd158069930 msgr2=0x7fd158069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:30.770 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.766+0000 7fd15f197640 1 -- 172.21.15.169:0/725051673 shutdown_connections 2023-10-26T01:31:30.770 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.766+0000 7fd15f197640 1 -- 172.21.15.169:0/725051673 wait complete. 2023-10-26T01:31:30.770 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.767+0000 7fd15f197640 1 Processor -- start 2023-10-26T01:31:30.771 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.768+0000 7fd15f197640 1 -- start start 2023-10-26T01:31:30.773 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.768+0000 7fd15f197640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd144004ab0 0x7fd158153f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:30.773 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.768+0000 7fd15f197640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd15806f740 0x7fd15813b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:30.773 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.769+0000 7fd15f197640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd158071490 0x7fd158071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:30.773 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.769+0000 7fd15f197640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd158070960 con 0x7fd15806f740 2023-10-26T01:31:30.774 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.769+0000 7fd15f197640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd158154440 con 0x7fd144004ab0 2023-10-26T01:31:30.774 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.769+0000 7fd15f197640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd158071e80 con 0x7fd158071490 2023-10-26T01:31:30.774 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.769+0000 7fd157fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd15806f740 0x7fd15813b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:30.774 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.769+0000 7fd157fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd15806f740 0x7fd15813b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:40700/0 (socket says 172.21.15.169:40700) 2023-10-26T01:31:30.774 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.769+0000 7fd157fff640 1 -- 172.21.15.169:0/3165395685 learned_addr learned my addr 172.21.15.169:0/3165395685 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:31:30.775 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.769+0000 7fd15cf33640 1 --2- 172.21.15.169:0/3165395685 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd144004ab0 0x7fd158153f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:30.775 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.769+0000 7fd15d734640 1 --2- 172.21.15.169:0/3165395685 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd158071490 0x7fd158071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:30.775 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.769+0000 7fd157fff640 1 -- 172.21.15.169:0/3165395685 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd158071490 msgr2=0x7fd158071940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:30.775 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.769+0000 7fd157fff640 1 --2- 172.21.15.169:0/3165395685 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd158071490 0x7fd158071940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:30.775 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.769+0000 7fd157fff640 1 -- 172.21.15.169:0/3165395685 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd144004ab0 msgr2=0x7fd158153f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:30.775 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.769+0000 7fd157fff640 1 --2- 172.21.15.169:0/3165395685 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd144004ab0 0x7fd158153f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:30.776 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.769+0000 7fd157fff640 1 -- 172.21.15.169:0/3165395685 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd15806fce0 con 0x7fd15806f740 2023-10-26T01:31:30.776 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.769+0000 7fd15cf33640 1 --2- 172.21.15.169:0/3165395685 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd144004ab0 0x7fd158153f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:31:30.776 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.770+0000 7fd157fff640 1 --2- 172.21.15.169:0/3165395685 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd15806f740 0x7fd15813b2a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd140011a50 tx=0x7fd140011f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:30.776 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.770+0000 7fd1557fa640 1 -- 172.21.15.169:0/3165395685 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd14000fcd0 con 0x7fd15806f740 2023-10-26T01:31:30.776 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.770+0000 7fd1557fa640 1 -- 172.21.15.169:0/3165395685 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd140012450 con 0x7fd15806f740 2023-10-26T01:31:30.777 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.770+0000 7fd1557fa640 1 -- 172.21.15.169:0/3165395685 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd140012840 con 0x7fd15806f740 2023-10-26T01:31:30.777 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.770+0000 7fd15f197640 1 -- 172.21.15.169:0/3165395685 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd15806ffc0 con 0x7fd15806f740 2023-10-26T01:31:30.777 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.770+0000 7fd15f197640 1 -- 172.21.15.169:0/3165395685 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd158070580 con 0x7fd15806f740 2023-10-26T01:31:30.778 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.771+0000 7fd15f197640 1 -- 172.21.15.169:0/3165395685 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd15809b440 con 0x7fd15806f740 2023-10-26T01:31:30.778 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.772+0000 7fd1557fa640 1 -- 172.21.15.169:0/3165395685 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fd1400129a0 con 0x7fd15806f740 2023-10-26T01:31:30.778 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.773+0000 7fd1557fa640 1 --2- 172.21.15.169:0/3165395685 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd13806ae40 0x7fd13806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:30.779 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.773+0000 7fd1557fa640 1 -- 172.21.15.169:0/3165395685 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fd14008a3c0 con 0x7fd15806f740 2023-10-26T01:31:30.779 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.774+0000 7fd15cf33640 1 --2- 172.21.15.169:0/3165395685 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd13806ae40 0x7fd13806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:30.779 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.775+0000 7fd15cf33640 1 --2- 172.21.15.169:0/3165395685 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd13806ae40 0x7fd13806d300 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd148034000 tx=0x7fd14802e000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:30.780 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.777+0000 7fd1557fa640 1 -- 172.21.15.169:0/3165395685 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd140055530 con 0x7fd15806f740 2023-10-26T01:31:30.972 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.969+0000 7fd15f197640 1 -- 172.21.15.169:0/3165395685 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd15804db10 con 0x7fd15806f740 2023-10-26T01:31:30.972 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.970+0000 7fd1557fa640 1 -- 172.21.15.169:0/3165395685 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd140058b50 con 0x7fd15806f740 2023-10-26T01:31:30.973 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 32 2023-10-26T01:31:30.976 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.973+0000 7fd15f197640 1 -- 172.21.15.169:0/3165395685 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd13806ae40 msgr2=0x7fd13806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:30.976 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.973+0000 7fd15f197640 1 --2- 172.21.15.169:0/3165395685 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd13806ae40 0x7fd13806d300 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd148034000 tx=0x7fd14802e000 comp rx=0 tx=0).stop 2023-10-26T01:31:30.976 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.973+0000 7fd15f197640 1 -- 172.21.15.169:0/3165395685 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd15806f740 msgr2=0x7fd15813b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:30.976 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.973+0000 7fd15f197640 1 --2- 172.21.15.169:0/3165395685 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd15806f740 0x7fd15813b2a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd140011a50 tx=0x7fd140011f10 comp rx=0 tx=0).stop 2023-10-26T01:31:30.977 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.973+0000 7fd15f197640 1 -- 172.21.15.169:0/3165395685 shutdown_connections 2023-10-26T01:31:30.977 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.973+0000 7fd15f197640 1 --2- 172.21.15.169:0/3165395685 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd13806ae40 0x7fd13806d300 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:30.977 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.973+0000 7fd15f197640 1 --2- 172.21.15.169:0/3165395685 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd15806f740 0x7fd15813b2a0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:30.977 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.973+0000 7fd15f197640 1 --2- 172.21.15.169:0/3165395685 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd158071490 0x7fd158071940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:30.977 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.973+0000 7fd15f197640 1 --2- 172.21.15.169:0/3165395685 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd144004ab0 0x7fd158153f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:30.977 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.973+0000 7fd15f197640 1 -- 172.21.15.169:0/3165395685 >> 172.21.15.169:0/3165395685 conn(0x7fd158069930 msgr2=0x7fd1580b07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:30.978 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.973+0000 7fd15f197640 1 -- 172.21.15.169:0/3165395685 shutdown_connections 2023-10-26T01:31:30.978 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:30.973+0000 7fd15f197640 1 -- 172.21.15.169:0/3165395685 wait complete. 2023-10-26T01:31:30.995 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-26T01:31:31.224 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:31.220+0000 7fd62afb0640 1 -- 172.21.15.169:0/2983435970 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6240cde20 msgr2=0x7fd624166630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:31.224 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:31.220+0000 7fd62afb0640 1 --2- 172.21.15.169:0/2983435970 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6240cde20 0x7fd624166630 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fd61c005fd0 tx=0x7fd61c02c170 comp rx=0 tx=0).stop 2023-10-26T01:31:31.224 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:31.221+0000 7fd62afb0640 1 -- 172.21.15.169:0/2983435970 shutdown_connections 2023-10-26T01:31:31.225 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:31.221+0000 7fd62afb0640 1 --2- 172.21.15.169:0/2983435970 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6240cde20 0x7fd624166630 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:31.225 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:31.221+0000 7fd62afb0640 1 -- 172.21.15.169:0/2983435970 >> 172.21.15.169:0/2983435970 conn(0x7fd624069930 msgr2=0x7fd624069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:31.225 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:31.221+0000 7fd62afb0640 1 -- 172.21.15.169:0/2983435970 shutdown_connections 2023-10-26T01:31:31.225 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:31.221+0000 7fd62afb0640 1 -- 172.21.15.169:0/2983435970 wait complete. 2023-10-26T01:31:31.227 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:31.222+0000 7fd62afb0640 1 Processor -- start 2023-10-26T01:31:31.227 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:31.223+0000 7fd62afb0640 1 -- start start 2023-10-26T01:31:31.227 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:31.223+0000 7fd62afb0640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd6240c9d00 0x7fd624070fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:31.227 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:31.223+0000 7fd62afb0640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd6240cde20 0x7fd62414b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:31.228 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:31.223+0000 7fd62afb0640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd62415b620 0x7fd62414b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:31.228 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:31.224+0000 7fd62afb0640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd624146630 con 0x7fd62415b620 2023-10-26T01:31:31.228 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:31.224+0000 7fd62afb0640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd624146e60 con 0x7fd6240cde20 2023-10-26T01:31:31.228 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:31.224+0000 7fd62afb0640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6240719c0 con 0x7fd6240c9d00 2023-10-26T01:31:31.228 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:31.224+0000 7fd62954d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd62415b620 0x7fd62414b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:31.229 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:31.224+0000 7fd62954d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd62415b620 0x7fd62414b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:40716/0 (socket says 172.21.15.169:40716) 2023-10-26T01:31:31.229 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:31.224+0000 7fd62954d640 1 -- 172.21.15.169:0/1678531157 learned_addr learned my addr 172.21.15.169:0/1678531157 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:31:31.229 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:31.224+0000 7fd628d4c640 1 --2- 172.21.15.169:0/1678531157 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd6240c9d00 0x7fd624070fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:31.229 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:31.224+0000 7fd62954d640 1 -- 172.21.15.169:0/1678531157 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd6240c9d00 msgr2=0x7fd624070fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:31.229 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:31.224+0000 7fd623fff640 1 --2- 172.21.15.169:0/1678531157 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd6240cde20 0x7fd62414b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:31.229 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:31.224+0000 7fd62954d640 1 --2- 172.21.15.169:0/1678531157 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd6240c9d00 0x7fd624070fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:31.230 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:31.224+0000 7fd62954d640 1 -- 172.21.15.169:0/1678531157 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd6240cde20 msgr2=0x7fd62414b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:31.230 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:31.224+0000 7fd62954d640 1 --2- 172.21.15.169:0/1678531157 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd6240cde20 0x7fd62414b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:31.230 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:31.224+0000 7fd62954d640 1 -- 172.21.15.169:0/1678531157 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd61c009840 con 0x7fd62415b620 2023-10-26T01:31:31.230 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:31.225+0000 7fd62954d640 1 --2- 172.21.15.169:0/1678531157 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd62415b620 0x7fd62414b8f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fd61c009cd0 tx=0x7fd61c004250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:31.231 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:31.225+0000 7fd6217fa640 1 -- 172.21.15.169:0/1678531157 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd61c004710 con 0x7fd62415b620 2023-10-26T01:31:31.232 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:31.225+0000 7fd6217fa640 1 -- 172.21.15.169:0/1678531157 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd61c0345a0 con 0x7fd62415b620 2023-10-26T01:31:31.232 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:31.225+0000 7fd62afb0640 1 -- 172.21.15.169:0/1678531157 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd624146810 con 0x7fd62415b620 2023-10-26T01:31:31.232 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:31.225+0000 7fd6217fa640 1 -- 172.21.15.169:0/1678531157 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd61c034990 con 0x7fd62415b620 2023-10-26T01:31:31.232 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:31.225+0000 7fd62afb0640 1 -- 172.21.15.169:0/1678531157 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd624071500 con 0x7fd62415b620 2023-10-26T01:31:31.232 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:31.226+0000 7fd62afb0640 1 -- 172.21.15.169:0/1678531157 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6240acc60 con 0x7fd62415b620 2023-10-26T01:31:31.232 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:31.227+0000 7fd6217fa640 1 -- 172.21.15.169:0/1678531157 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fd61c034af0 con 0x7fd62415b620 2023-10-26T01:31:31.233 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:31.228+0000 7fd6217fa640 1 --2- 172.21.15.169:0/1678531157 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd60806ae70 0x7fd60806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:31.233 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:31.228+0000 7fd6217fa640 1 -- 172.21.15.169:0/1678531157 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fd61c0aca50 con 0x7fd62415b620 2023-10-26T01:31:31.233 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:31.229+0000 7fd628d4c640 1 --2- 172.21.15.169:0/1678531157 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd60806ae70 0x7fd60806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:31.233 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:31.230+0000 7fd628d4c640 1 --2- 172.21.15.169:0/1678531157 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd60806ae70 0x7fd60806d330 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd618000f60 tx=0x7fd618009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:31.234 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:31.232+0000 7fd6217fa640 1 -- 172.21.15.169:0/1678531157 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd6240acc60 con 0x7fd62415b620 2023-10-26T01:31:31.423 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:31.421+0000 7fd62afb0640 1 -- 172.21.15.169:0/1678531157 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd6240acc60 con 0x7fd62415b620 2023-10-26T01:31:31.424 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:31.422+0000 7fd6217fa640 1 -- 172.21.15.169:0/1678531157 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd6240acc60 con 0x7fd62415b620 2023-10-26T01:31:31.425 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 64 2023-10-26T01:31:31.428 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:31.425+0000 7fd62afb0640 1 -- 172.21.15.169:0/1678531157 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd60806ae70 msgr2=0x7fd60806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:31.428 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:31.425+0000 7fd62afb0640 1 --2- 172.21.15.169:0/1678531157 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd60806ae70 0x7fd60806d330 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd618000f60 tx=0x7fd618009040 comp rx=0 tx=0).stop 2023-10-26T01:31:31.428 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:31.425+0000 7fd62afb0640 1 -- 172.21.15.169:0/1678531157 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd62415b620 msgr2=0x7fd62414b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:31.428 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:31.425+0000 7fd62afb0640 1 --2- 172.21.15.169:0/1678531157 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd62415b620 0x7fd62414b8f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fd61c009cd0 tx=0x7fd61c004250 comp rx=0 tx=0).stop 2023-10-26T01:31:31.429 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:31.425+0000 7fd62afb0640 1 -- 172.21.15.169:0/1678531157 shutdown_connections 2023-10-26T01:31:31.429 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:31.425+0000 7fd62afb0640 1 --2- 172.21.15.169:0/1678531157 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd60806ae70 0x7fd60806d330 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:31.429 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:31.425+0000 7fd62afb0640 1 --2- 172.21.15.169:0/1678531157 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd62415b620 0x7fd62414b8f0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:31.429 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:31.425+0000 7fd62afb0640 1 --2- 172.21.15.169:0/1678531157 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd6240cde20 0x7fd62414b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:31.429 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:31.425+0000 7fd62afb0640 1 --2- 172.21.15.169:0/1678531157 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd6240c9d00 0x7fd624070fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:31.430 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:31.425+0000 7fd62afb0640 1 -- 172.21.15.169:0/1678531157 >> 172.21.15.169:0/1678531157 conn(0x7fd624069930 msgr2=0x7fd6240ac2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:31.430 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:31.425+0000 7fd62afb0640 1 -- 172.21.15.169:0/1678531157 shutdown_connections 2023-10-26T01:31:31.430 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:31.425+0000 7fd62afb0640 1 -- 172.21.15.169:0/1678531157 wait complete. 2023-10-26T01:31:31.447 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-26T01:31:31.668 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:31.665+0000 7f25e54e5640 1 -- 172.21.15.169:0/3602330595 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25e0155360 msgr2=0x7f25e00cd7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:31.668 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:31.665+0000 7f25e54e5640 1 --2- 172.21.15.169:0/3602330595 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25e0155360 0x7f25e00cd7e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f25d0005fd0 tx=0x7f25d002be10 comp rx=0 tx=0).stop 2023-10-26T01:31:31.669 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:31.665+0000 7f25e54e5640 1 -- 172.21.15.169:0/3602330595 shutdown_connections 2023-10-26T01:31:31.669 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:31.665+0000 7f25e54e5640 1 --2- 172.21.15.169:0/3602330595 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f25e015bbb0 0x7f25e00b05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:31.669 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:31.665+0000 7f25e54e5640 1 --2- 172.21.15.169:0/3602330595 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25e0155360 0x7f25e00cd7e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:31.669 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:31.665+0000 7f25e54e5640 1 -- 172.21.15.169:0/3602330595 >> 172.21.15.169:0/3602330595 conn(0x7f25e0069930 msgr2=0x7f25e0069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:31.669 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:31.665+0000 7f25e54e5640 1 -- 172.21.15.169:0/3602330595 shutdown_connections 2023-10-26T01:31:31.670 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:31.665+0000 7f25e54e5640 1 -- 172.21.15.169:0/3602330595 wait complete. 2023-10-26T01:31:31.671 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:31.667+0000 7f25e54e5640 1 Processor -- start 2023-10-26T01:31:31.671 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:31.667+0000 7f25e54e5640 1 -- start start 2023-10-26T01:31:31.672 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:31.668+0000 7f25e54e5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25e00cdd20 0x7f25e0096a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:31.672 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:31.668+0000 7f25e54e5640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f25e0155360 0x7f25e0070ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:31.672 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:31.668+0000 7f25e54e5640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25e015bbb0 0x7f25e00aef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:31.672 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:31.668+0000 7f25e54e5640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25e01520e0 con 0x7f25e015bbb0 2023-10-26T01:31:31.672 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:31.668+0000 7f25e54e5640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25e0071510 con 0x7f25e00cdd20 2023-10-26T01:31:31.673 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:31.668+0000 7f25e54e5640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25e0152330 con 0x7f25e0155360 2023-10-26T01:31:31.673 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:31.668+0000 7f25df7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25e015bbb0 0x7f25e00aef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:31.673 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:31.668+0000 7f25df7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25e015bbb0 0x7f25e00aef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:40724/0 (socket says 172.21.15.169:40724) 2023-10-26T01:31:31.673 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:31.668+0000 7f25df7fe640 1 -- 172.21.15.169:0/1762519041 learned_addr learned my addr 172.21.15.169:0/1762519041 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:31:31.673 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:31.668+0000 7f25deffd640 1 --2- 172.21.15.169:0/1762519041 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25e00cdd20 0x7f25e0096a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:31.674 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:31.669+0000 7f25de7fc640 1 --2- 172.21.15.169:0/1762519041 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f25e0155360 0x7f25e0070ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:31.674 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:31.669+0000 7f25df7fe640 1 -- 172.21.15.169:0/1762519041 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f25e0155360 msgr2=0x7f25e0070ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:31.674 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:31.669+0000 7f25df7fe640 1 --2- 172.21.15.169:0/1762519041 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f25e0155360 0x7f25e0070ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:31.674 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:31.669+0000 7f25df7fe640 1 -- 172.21.15.169:0/1762519041 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25e00cdd20 msgr2=0x7f25e0096a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:31.674 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:31.669+0000 7f25df7fe640 1 --2- 172.21.15.169:0/1762519041 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25e00cdd20 0x7f25e0096a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:31.675 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:31.669+0000 7f25df7fe640 1 -- 172.21.15.169:0/1762519041 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25d0009560 con 0x7f25e015bbb0 2023-10-26T01:31:31.675 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:31.669+0000 7f25df7fe640 1 --2- 172.21.15.169:0/1762519041 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25e015bbb0 0x7f25e00aef60 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f25d800ea80 tx=0x7f25d800ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:31.676 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:31.669+0000 7f25c77fe640 1 -- 172.21.15.169:0/1762519041 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d800cea0 con 0x7f25e015bbb0 2023-10-26T01:31:31.676 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:31.669+0000 7f25c77fe640 1 -- 172.21.15.169:0/1762519041 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f25d8004b20 con 0x7f25e015bbb0 2023-10-26T01:31:31.676 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:31.669+0000 7f25e54e5640 1 -- 172.21.15.169:0/1762519041 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25e0152610 con 0x7f25e015bbb0 2023-10-26T01:31:31.677 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:31.669+0000 7f25c77fe640 1 -- 172.21.15.169:0/1762519041 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d8005020 con 0x7f25e015bbb0 2023-10-26T01:31:31.677 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:31.669+0000 7f25e54e5640 1 -- 172.21.15.169:0/1762519041 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25e0093940 con 0x7f25e015bbb0 2023-10-26T01:31:31.677 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:31.671+0000 7f25e54e5640 1 -- 172.21.15.169:0/1762519041 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25e0152b30 con 0x7f25e015bbb0 2023-10-26T01:31:31.677 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:31.672+0000 7f25c77fe640 1 -- 172.21.15.169:0/1762519041 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f25d8005180 con 0x7f25e015bbb0 2023-10-26T01:31:31.677 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:31.673+0000 7f25c77fe640 1 --2- 172.21.15.169:0/1762519041 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f25cc06ae40 0x7f25cc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:31.678 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:31.673+0000 7f25c77fe640 1 -- 172.21.15.169:0/1762519041 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f25d8014070 con 0x7f25e015bbb0 2023-10-26T01:31:31.678 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:31.674+0000 7f25deffd640 1 --2- 172.21.15.169:0/1762519041 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f25cc06ae40 0x7f25cc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:31.678 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:31.675+0000 7f25deffd640 1 --2- 172.21.15.169:0/1762519041 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f25cc06ae40 0x7f25cc06d300 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f25d0035000 tx=0x7f25d0004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:31.680 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:31.677+0000 7f25c77fe640 1 -- 172.21.15.169:0/1762519041 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f25e0152b30 con 0x7f25e015bbb0 2023-10-26T01:31:31.883 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:31.880+0000 7f25e54e5640 1 -- 172.21.15.169:0/1762519041 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f25e0152b30 con 0x7f25e015bbb0 2023-10-26T01:31:31.883 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:31.881+0000 7f25c77fe640 1 -- 172.21.15.169:0/1762519041 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) v1 ==== 80+0+0 (secure 0 0 0) 0x7f25e0152b30 con 0x7f25e015bbb0 2023-10-26T01:31:31.884 INFO:teuthology.orchestra.run.smithi169.stderr:listed 0 entries 2023-10-26T01:31:31.886 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:31.883+0000 7f25e54e5640 1 -- 172.21.15.169:0/1762519041 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f25cc06ae40 msgr2=0x7f25cc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:31.886 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:31.883+0000 7f25e54e5640 1 --2- 172.21.15.169:0/1762519041 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f25cc06ae40 0x7f25cc06d300 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f25d0035000 tx=0x7f25d0004730 comp rx=0 tx=0).stop 2023-10-26T01:31:31.887 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:31.883+0000 7f25e54e5640 1 -- 172.21.15.169:0/1762519041 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25e015bbb0 msgr2=0x7f25e00aef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:31.887 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:31.883+0000 7f25e54e5640 1 --2- 172.21.15.169:0/1762519041 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25e015bbb0 0x7f25e00aef60 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f25d800ea80 tx=0x7f25d800ef40 comp rx=0 tx=0).stop 2023-10-26T01:31:31.887 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:31.884+0000 7f25e54e5640 1 -- 172.21.15.169:0/1762519041 shutdown_connections 2023-10-26T01:31:31.887 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:31.884+0000 7f25e54e5640 1 --2- 172.21.15.169:0/1762519041 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f25cc06ae40 0x7f25cc06d300 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:31.887 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:31.884+0000 7f25e54e5640 1 --2- 172.21.15.169:0/1762519041 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25e015bbb0 0x7f25e00aef60 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:31.888 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:31.884+0000 7f25e54e5640 1 --2- 172.21.15.169:0/1762519041 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f25e0155360 0x7f25e0070ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:31.888 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:31.884+0000 7f25e54e5640 1 --2- 172.21.15.169:0/1762519041 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25e00cdd20 0x7f25e0096a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:31.888 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:31.884+0000 7f25e54e5640 1 -- 172.21.15.169:0/1762519041 >> 172.21.15.169:0/1762519041 conn(0x7f25e0069930 msgr2=0x7f25e01597f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:31.888 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:31.884+0000 7f25e54e5640 1 -- 172.21.15.169:0/1762519041 shutdown_connections 2023-10-26T01:31:31.888 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:31.884+0000 7f25e54e5640 1 -- 172.21.15.169:0/1762519041 wait complete. 2023-10-26T01:31:31.906 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-26T01:31:31.906 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:31:31.906 DEBUG:teuthology.orchestra.run.smithi169:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2023-10-26T01:31:32.005 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi169.front.sepia.ceph.com 2023-10-26T01:31:32.006 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:31:32.006 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-26T01:31:32.062 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-26T01:31:32.062 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:31:32.120 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-26T01:31:32.121 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:31:32.174 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-26T01:31:32.175 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:31:32.228 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:31:32.456 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:32.453+0000 7fcbc4da5640 1 -- 172.21.15.169:0/2689805115 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcbc006ce20 msgr2=0x7fcbc014e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:32.456 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:32.453+0000 7fcbc4da5640 1 --2- 172.21.15.169:0/2689805115 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcbc006ce20 0x7fcbc014e790 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fcbc006eaf0 tx=0x7fcbb002bd10 comp rx=0 tx=0).stop 2023-10-26T01:31:32.456 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:32.453+0000 7fcbc4da5640 1 -- 172.21.15.169:0/2689805115 shutdown_connections 2023-10-26T01:31:32.457 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:32.453+0000 7fcbc4da5640 1 --2- 172.21.15.169:0/2689805115 >> v2:172.21.15.170:3300/0 conn(0x7fcbc0158850 0x7fcbc00aab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:32.457 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:32.453+0000 7fcbc4da5640 1 --2- 172.21.15.169:0/2689805115 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcbc006ce20 0x7fcbc014e790 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:32.457 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:32.453+0000 7fcbc4da5640 1 -- 172.21.15.169:0/2689805115 >> 172.21.15.169:0/2689805115 conn(0x7fcbc0069930 msgr2=0x7fcbc0069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:32.457 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:32.453+0000 7fcbc4da5640 1 -- 172.21.15.169:0/2689805115 shutdown_connections 2023-10-26T01:31:32.457 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:32.453+0000 7fcbc4da5640 1 -- 172.21.15.169:0/2689805115 wait complete. 2023-10-26T01:31:32.459 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:32.455+0000 7fcbc4da5640 1 Processor -- start 2023-10-26T01:31:32.459 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:32.455+0000 7fcbc4da5640 1 -- start start 2023-10-26T01:31:32.460 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:32.456+0000 7fcbc4da5640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcbc006ad30 0x7fcbc014f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:32.460 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:32.456+0000 7fcbc4da5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcbc006ce20 0x7fcbc01510c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:32.460 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:32.456+0000 7fcbc4da5640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcbc0158850 0x7fcbc01427d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:32.460 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:32.456+0000 7fcbc4da5640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc014aa50 con 0x7fcbc0158850 2023-10-26T01:31:32.460 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:32.456+0000 7fcbc4da5640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc014ad30 con 0x7fcbc006ce20 2023-10-26T01:31:32.461 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:32.456+0000 7fcbc4da5640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc014b010 con 0x7fcbc006ad30 2023-10-26T01:31:32.461 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:32.456+0000 7fcbbed9d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcbc0158850 0x7fcbc01427d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:32.461 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:32.456+0000 7fcbbed9d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcbc0158850 0x7fcbc01427d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:40744/0 (socket says 172.21.15.169:40744) 2023-10-26T01:31:32.461 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:32.456+0000 7fcbbed9d640 1 -- 172.21.15.169:0/2255314738 learned_addr learned my addr 172.21.15.169:0/2255314738 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:31:32.461 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:32.456+0000 7fcbb5d9b640 1 --2- 172.21.15.169:0/2255314738 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcbc006ce20 0x7fcbc01510c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:32.462 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:32.456+0000 7fcbbe59c640 1 --2- 172.21.15.169:0/2255314738 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcbc006ad30 0x7fcbc014f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:32.462 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:32.457+0000 7fcbbed9d640 1 -- 172.21.15.169:0/2255314738 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcbc006ad30 msgr2=0x7fcbc014f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:32.462 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:32.457+0000 7fcbbed9d640 1 --2- 172.21.15.169:0/2255314738 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcbc006ad30 0x7fcbc014f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:32.462 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:32.457+0000 7fcbbed9d640 1 -- 172.21.15.169:0/2255314738 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcbc006ce20 msgr2=0x7fcbc01510c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:32.463 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:32.457+0000 7fcbbed9d640 1 --2- 172.21.15.169:0/2255314738 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcbc006ce20 0x7fcbc01510c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:32.463 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:32.457+0000 7fcbbed9d640 1 -- 172.21.15.169:0/2255314738 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbb0009560 con 0x7fcbc0158850 2023-10-26T01:31:32.463 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:32.457+0000 7fcbbe59c640 1 --2- 172.21.15.169:0/2255314738 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcbc006ad30 0x7fcbc014f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:31:32.463 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:32.457+0000 7fcbb5d9b640 1 --2- 172.21.15.169:0/2255314738 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcbc006ce20 0x7fcbc01510c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:31:32.464 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:32.457+0000 7fcbbed9d640 1 --2- 172.21.15.169:0/2255314738 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcbc0158850 0x7fcbc01427d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fcbb800c780 tx=0x7fcbb800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:32.465 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:32.457+0000 7fcbb77fe640 1 -- 172.21.15.169:0/2255314738 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb8003d80 con 0x7fcbc0158850 2023-10-26T01:31:32.465 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:32.457+0000 7fcbb77fe640 1 -- 172.21.15.169:0/2255314738 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fcbb800ce90 con 0x7fcbc0158850 2023-10-26T01:31:32.465 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:32.457+0000 7fcbc4da5640 1 -- 172.21.15.169:0/2255314738 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbc014b2f0 con 0x7fcbc0158850 2023-10-26T01:31:32.466 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:32.457+0000 7fcbb77fe640 1 -- 172.21.15.169:0/2255314738 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb80052d0 con 0x7fcbc0158850 2023-10-26T01:31:32.466 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:32.457+0000 7fcbc4da5640 1 -- 172.21.15.169:0/2255314738 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbc0156ee0 con 0x7fcbc0158850 2023-10-26T01:31:32.466 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:32.459+0000 7fcbc4da5640 1 -- 172.21.15.169:0/2255314738 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbc004db10 con 0x7fcbc0158850 2023-10-26T01:31:32.466 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:32.459+0000 7fcbb77fe640 1 -- 172.21.15.169:0/2255314738 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fcbb8005430 con 0x7fcbc0158850 2023-10-26T01:31:32.466 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:32.460+0000 7fcbb77fe640 1 --2- 172.21.15.169:0/2255314738 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcba406ae40 0x7fcba406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:32.467 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:32.460+0000 7fcbb77fe640 1 -- 172.21.15.169:0/2255314738 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fcbb8088d40 con 0x7fcbc0158850 2023-10-26T01:31:32.467 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:32.461+0000 7fcbbe59c640 1 --2- 172.21.15.169:0/2255314738 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcba406ae40 0x7fcba406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:32.467 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:32.462+0000 7fcbbe59c640 1 --2- 172.21.15.169:0/2255314738 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcba406ae40 0x7fcba406d300 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fcbb0035000 tx=0x7fcbb0004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:32.468 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:32.465+0000 7fcbb77fe640 1 -- 172.21.15.169:0/2255314738 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcbb8053d60 con 0x7fcbc0158850 2023-10-26T01:31:32.659 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:32.656+0000 7fcbc4da5640 1 -- 172.21.15.169:0/2255314738 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcbc00adbf0 con 0x7fcbc0158850 2023-10-26T01:31:32.661 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:32.658+0000 7fcbb77fe640 1 -- 172.21.15.169:0/2255314738 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16801 (secure 0 0 0) 0x7fcbb8017090 con 0x7fcbc0158850 2023-10-26T01:31:32.662 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:31:32.662 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":23,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:31:30.100039+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-26T01:30:39.437631+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-26T01:30:40.561774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":2}]},{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.24","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:31:32.664 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:32.661+0000 7fcbc4da5640 1 -- 172.21.15.169:0/2255314738 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcba406ae40 msgr2=0x7fcba406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:32.664 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:32.661+0000 7fcbc4da5640 1 --2- 172.21.15.169:0/2255314738 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcba406ae40 0x7fcba406d300 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fcbb0035000 tx=0x7fcbb0004730 comp rx=0 tx=0).stop 2023-10-26T01:31:32.665 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:32.661+0000 7fcbc4da5640 1 -- 172.21.15.169:0/2255314738 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcbc0158850 msgr2=0x7fcbc01427d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:32.665 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:32.661+0000 7fcbc4da5640 1 --2- 172.21.15.169:0/2255314738 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcbc0158850 0x7fcbc01427d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fcbb800c780 tx=0x7fcbb800cc40 comp rx=0 tx=0).stop 2023-10-26T01:31:32.665 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:32.661+0000 7fcbc4da5640 1 -- 172.21.15.169:0/2255314738 shutdown_connections 2023-10-26T01:31:32.665 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:32.661+0000 7fcbc4da5640 1 --2- 172.21.15.169:0/2255314738 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcba406ae40 0x7fcba406d300 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:32.666 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:32.661+0000 7fcbc4da5640 1 --2- 172.21.15.169:0/2255314738 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcbc0158850 0x7fcbc01427d0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:32.666 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:32.661+0000 7fcbc4da5640 1 --2- 172.21.15.169:0/2255314738 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcbc006ce20 0x7fcbc01510c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:32.666 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:32.661+0000 7fcbc4da5640 1 --2- 172.21.15.169:0/2255314738 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcbc006ad30 0x7fcbc014f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:32.666 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:32.661+0000 7fcbc4da5640 1 -- 172.21.15.169:0/2255314738 >> 172.21.15.169:0/2255314738 conn(0x7fcbc0069930 msgr2=0x7fcbc006c030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:32.666 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:32.661+0000 7fcbc4da5640 1 -- 172.21.15.169:0/2255314738 shutdown_connections 2023-10-26T01:31:32.667 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:32.662+0000 7fcbc4da5640 1 -- 172.21.15.169:0/2255314738 wait complete. 2023-10-26T01:31:32.682 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T01:30:32.200808+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-26T01:30:39.437631+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-26T01:30:40.561774+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T01:31:32.683 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T01:31:32.906 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:32.903+0000 7f53d359e640 1 -- 172.21.15.169:0/2205534570 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53c0004d00 msgr2=0x7f53c0005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:32.906 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:32.903+0000 7f53d359e640 1 --2- 172.21.15.169:0/2205534570 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53c0004d00 0x7f53c0005180 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f53cc00c260 tx=0x7f53c00057f0 comp rx=0 tx=0).stop 2023-10-26T01:31:32.906 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:32.903+0000 7f53d2d9d640 1 -- 172.21.15.169:0/2205534570 reap_dead start 2023-10-26T01:31:32.907 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:32.903+0000 7f53d359e640 1 -- 172.21.15.169:0/2205534570 shutdown_connections 2023-10-26T01:31:32.907 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:32.903+0000 7f53d359e640 1 -- 172.21.15.169:0/2205534570 >> 172.21.15.169:0/2205534570 conn(0x7f53d4069510 msgr2=0x7f53d4069920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:32.907 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:32.903+0000 7f53d359e640 1 -- 172.21.15.169:0/2205534570 shutdown_connections 2023-10-26T01:31:32.907 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:32.903+0000 7f53d359e640 1 -- 172.21.15.169:0/2205534570 wait complete. 2023-10-26T01:31:32.908 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:32.905+0000 7f53d359e640 1 Processor -- start 2023-10-26T01:31:32.908 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:32.905+0000 7f53d359e640 1 -- start start 2023-10-26T01:31:32.910 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:32.906+0000 7f53d359e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53c0004d00 0x7f53d409ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:32.910 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:32.906+0000 7f53d359e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f53d40af4c0 0x7f53d40add90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:32.910 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:32.906+0000 7f53d359e640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f53d4143d00 0x7f53d41589a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:32.910 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:32.906+0000 7f53d359e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53d40af0a0 con 0x7f53c0004d00 2023-10-26T01:31:32.910 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:32.906+0000 7f53d359e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53d40ae420 con 0x7f53d40af4c0 2023-10-26T01:31:32.911 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:32.906+0000 7f53d359e640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53d40ae700 con 0x7f53d4143d00 2023-10-26T01:31:32.911 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:32.906+0000 7f53d259c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53c0004d00 0x7f53d409ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:32.911 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:32.906+0000 7f53d259c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53c0004d00 0x7f53d409ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:40760/0 (socket says 172.21.15.169:40760) 2023-10-26T01:31:32.911 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:32.906+0000 7f53d259c640 1 -- 172.21.15.169:0/3910741342 learned_addr learned my addr 172.21.15.169:0/3910741342 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:31:32.911 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:32.906+0000 7f53c9d9b640 1 --2- 172.21.15.169:0/3910741342 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f53d40af4c0 0x7f53d40add90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:32.912 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:32.906+0000 7f53d259c640 1 -- 172.21.15.169:0/3910741342 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f53d4143d00 msgr2=0x7f53d41589a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:32.912 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:32.906+0000 7f53d2d9d640 1 --2- 172.21.15.169:0/3910741342 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f53d4143d00 0x7f53d41589a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:32.912 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:32.907+0000 7f53d259c640 1 --2- 172.21.15.169:0/3910741342 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f53d4143d00 0x7f53d41589a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:32.912 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:32.907+0000 7f53d259c640 1 -- 172.21.15.169:0/3910741342 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f53d40af4c0 msgr2=0x7f53d40add90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:32.912 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:32.907+0000 7f53d259c640 1 --2- 172.21.15.169:0/3910741342 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f53d40af4c0 0x7f53d40add90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:32.913 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:32.907+0000 7f53d259c640 1 -- 172.21.15.169:0/3910741342 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53d40ae9e0 con 0x7f53c0004d00 2023-10-26T01:31:32.913 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:32.907+0000 7f53c9d9b640 1 --2- 172.21.15.169:0/3910741342 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f53d40af4c0 0x7f53d40add90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:31:32.913 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:32.907+0000 7f53d259c640 1 --2- 172.21.15.169:0/3910741342 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53c0004d00 0x7f53d409ad00 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f53c400eaa0 tx=0x7f53c400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:32.913 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:32.907+0000 7f53cb7fe640 1 -- 172.21.15.169:0/3910741342 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c400ce10 con 0x7f53c0004d00 2023-10-26T01:31:32.913 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:32.907+0000 7f53cb7fe640 1 -- 172.21.15.169:0/3910741342 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f53c4004b20 con 0x7f53c0004d00 2023-10-26T01:31:32.914 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:32.907+0000 7f53cb7fe640 1 -- 172.21.15.169:0/3910741342 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c4005020 con 0x7f53c0004d00 2023-10-26T01:31:32.914 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:32.907+0000 7f53d359e640 1 -- 172.21.15.169:0/3910741342 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53d40aecc0 con 0x7f53c0004d00 2023-10-26T01:31:32.914 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:32.907+0000 7f53d359e640 1 -- 172.21.15.169:0/3910741342 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53d4158ee0 con 0x7f53c0004d00 2023-10-26T01:31:32.915 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:32.908+0000 7f53d359e640 1 -- 172.21.15.169:0/3910741342 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53d4073c10 con 0x7f53c0004d00 2023-10-26T01:31:32.915 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:32.909+0000 7f53cb7fe640 1 -- 172.21.15.169:0/3910741342 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f53c4005180 con 0x7f53c0004d00 2023-10-26T01:31:32.915 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:32.911+0000 7f53cb7fe640 1 --2- 172.21.15.169:0/3910741342 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f53b406ae40 0x7f53b406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:32.915 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:32.911+0000 7f53cb7fe640 1 -- 172.21.15.169:0/3910741342 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f53c408a0c0 con 0x7f53c0004d00 2023-10-26T01:31:32.916 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:32.912+0000 7f53c9d9b640 1 --2- 172.21.15.169:0/3910741342 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f53b406ae40 0x7f53b406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:32.916 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:32.913+0000 7f53c9d9b640 1 --2- 172.21.15.169:0/3910741342 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f53b406ae40 0x7f53b406d300 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f53bc005fd0 tx=0x7f53bc005d10 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:32.918 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:32.916+0000 7f53cb7fe640 1 -- 172.21.15.169:0/3910741342 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f53c40550e0 con 0x7f53c0004d00 2023-10-26T01:31:33.108 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:33.105+0000 7f53d359e640 1 -- 172.21.15.169:0/3910741342 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f53d40ac840 con 0x7f53c0004d00 2023-10-26T01:31:33.109 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:33.106+0000 7f53cb7fe640 1 -- 172.21.15.169:0/3910741342 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f53c401f020 con 0x7f53c0004d00 2023-10-26T01:31:33.109 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 1 2023-10-26T01:31:33.112 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:33.109+0000 7f53d359e640 1 -- 172.21.15.169:0/3910741342 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f53b406ae40 msgr2=0x7f53b406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:33.112 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:33.109+0000 7f53d359e640 1 --2- 172.21.15.169:0/3910741342 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f53b406ae40 0x7f53b406d300 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f53bc005fd0 tx=0x7f53bc005d10 comp rx=0 tx=0).stop 2023-10-26T01:31:33.112 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:33.109+0000 7f53d359e640 1 -- 172.21.15.169:0/3910741342 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53c0004d00 msgr2=0x7f53d409ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:33.112 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:33.109+0000 7f53d359e640 1 --2- 172.21.15.169:0/3910741342 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53c0004d00 0x7f53d409ad00 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f53c400eaa0 tx=0x7f53c400ef60 comp rx=0 tx=0).stop 2023-10-26T01:31:33.113 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:33.109+0000 7f53d359e640 1 -- 172.21.15.169:0/3910741342 shutdown_connections 2023-10-26T01:31:33.113 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:33.109+0000 7f53d359e640 1 --2- 172.21.15.169:0/3910741342 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f53b406ae40 0x7f53b406d300 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:33.113 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:33.109+0000 7f53d359e640 1 --2- 172.21.15.169:0/3910741342 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f53d4143d00 0x7f53d41589a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:33.113 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:33.109+0000 7f53d359e640 1 --2- 172.21.15.169:0/3910741342 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f53d40af4c0 0x7f53d40add90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:33.113 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:33.109+0000 7f53d359e640 1 --2- 172.21.15.169:0/3910741342 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53c0004d00 0x7f53d409ad00 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:33.114 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:33.109+0000 7f53d359e640 1 -- 172.21.15.169:0/3910741342 >> 172.21.15.169:0/3910741342 conn(0x7f53d4069510 msgr2=0x7f53d415c340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:33.114 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:33.109+0000 7f53d359e640 1 -- 172.21.15.169:0/3910741342 shutdown_connections 2023-10-26T01:31:33.114 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:33.109+0000 7f53d359e640 1 -- 172.21.15.169:0/3910741342 wait complete. 2023-10-26T01:31:33.132 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-26T01:31:33.359 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:33.356+0000 7f9c46bb8640 1 -- 172.21.15.169:0/2431197971 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c40153b90 msgr2=0x7f9c40069900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:33.360 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:33.356+0000 7f9c46bb8640 1 --2- 172.21.15.169:0/2431197971 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c40153b90 0x7f9c40069900 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f9c400acbf0 tx=0x7f9c3c02bd70 comp rx=0 tx=0).stop 2023-10-26T01:31:33.360 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:33.356+0000 7f9c46bb8640 1 -- 172.21.15.169:0/2431197971 shutdown_connections 2023-10-26T01:31:33.360 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:33.356+0000 7f9c46bb8640 1 --2- 172.21.15.169:0/2431197971 >> v2:172.21.15.170:3300/0 conn(0x7f9c4015d7e0 0x7f9c40069e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:33.360 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:33.356+0000 7f9c46bb8640 1 --2- 172.21.15.169:0/2431197971 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c40153b90 0x7f9c40069900 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:33.360 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:33.356+0000 7f9c46bb8640 1 -- 172.21.15.169:0/2431197971 >> 172.21.15.169:0/2431197971 conn(0x7f9c400af110 msgr2=0x7f9c400af520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:33.361 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:33.356+0000 7f9c46bb8640 1 -- 172.21.15.169:0/2431197971 shutdown_connections 2023-10-26T01:31:33.361 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:33.356+0000 7f9c46bb8640 1 -- 172.21.15.169:0/2431197971 wait complete. 2023-10-26T01:31:33.361 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:33.358+0000 7f9c46bb8640 1 Processor -- start 2023-10-26T01:31:33.362 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:33.358+0000 7f9c46bb8640 1 -- start start 2023-10-26T01:31:33.363 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:33.359+0000 7f9c46bb8640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c400b0e50 0x7f9c400996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:33.364 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:33.360+0000 7f9c46bb8640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9c40153b90 0x7f9c401f25f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:33.364 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:33.360+0000 7f9c46bb8640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9c4015d7e0 0x7f9c400c7280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:33.364 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:33.360+0000 7f9c46bb8640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c40155510 con 0x7f9c400b0e50 2023-10-26T01:31:33.364 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:33.360+0000 7f9c46bb8640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c400718b0 con 0x7f9c40153b90 2023-10-26T01:31:33.364 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:33.360+0000 7f9c46bb8640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c400712f0 con 0x7f9c4015d7e0 2023-10-26T01:31:33.365 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:33.360+0000 7f9c45bb6640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c400b0e50 0x7f9c400996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:33.365 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:33.360+0000 7f9c45bb6640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c400b0e50 0x7f9c400996a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:40770/0 (socket says 172.21.15.169:40770) 2023-10-26T01:31:33.365 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:33.360+0000 7f9c45bb6640 1 -- 172.21.15.169:0/3524692457 learned_addr learned my addr 172.21.15.169:0/3524692457 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:31:33.365 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:33.360+0000 7f9c45bb6640 1 -- 172.21.15.169:0/3524692457 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9c4015d7e0 msgr2=0x7f9c400c7280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:33.365 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:33.360+0000 7f9c45bb6640 1 --2- 172.21.15.169:0/3524692457 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9c4015d7e0 0x7f9c400c7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:33.366 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:33.360+0000 7f9c45bb6640 1 -- 172.21.15.169:0/3524692457 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9c40153b90 msgr2=0x7f9c401f25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:33.366 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:33.360+0000 7f9c45bb6640 1 --2- 172.21.15.169:0/3524692457 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9c40153b90 0x7f9c401f25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:33.366 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:33.360+0000 7f9c45bb6640 1 -- 172.21.15.169:0/3524692457 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c3c009490 con 0x7f9c400b0e50 2023-10-26T01:31:33.366 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:33.361+0000 7f9c45bb6640 1 --2- 172.21.15.169:0/3524692457 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c400b0e50 0x7f9c400996a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c000c00 tx=0x7f9c3c0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:33.367 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:33.361+0000 7f9c367fc640 1 -- 172.21.15.169:0/3524692457 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c3c003ea0 con 0x7f9c400b0e50 2023-10-26T01:31:33.367 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:33.361+0000 7f9c367fc640 1 -- 172.21.15.169:0/3524692457 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9c3c034550 con 0x7f9c400b0e50 2023-10-26T01:31:33.367 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:33.361+0000 7f9c46bb8640 1 -- 172.21.15.169:0/3524692457 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c400715d0 con 0x7f9c400b0e50 2023-10-26T01:31:33.367 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:33.361+0000 7f9c367fc640 1 -- 172.21.15.169:0/3524692457 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c3c034960 con 0x7f9c400b0e50 2023-10-26T01:31:33.367 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:33.361+0000 7f9c46bb8640 1 -- 172.21.15.169:0/3524692457 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c40099f30 con 0x7f9c400b0e50 2023-10-26T01:31:33.368 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:33.362+0000 7f9c46bb8640 1 -- 172.21.15.169:0/3524692457 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c40155650 con 0x7f9c400b0e50 2023-10-26T01:31:33.369 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:33.363+0000 7f9c367fc640 1 -- 172.21.15.169:0/3524692457 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f9c3c03b040 con 0x7f9c400b0e50 2023-10-26T01:31:33.369 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:33.364+0000 7f9c367fc640 1 --2- 172.21.15.169:0/3524692457 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9c2406ae40 0x7f9c2406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:33.369 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:33.364+0000 7f9c367fc640 1 -- 172.21.15.169:0/3524692457 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f9c3c0abca0 con 0x7f9c400b0e50 2023-10-26T01:31:33.369 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:33.365+0000 7f9c453b5640 1 --2- 172.21.15.169:0/3524692457 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9c2406ae40 0x7f9c2406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:33.370 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:33.366+0000 7f9c453b5640 1 --2- 172.21.15.169:0/3524692457 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9c2406ae40 0x7f9c2406d300 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9c4006ccf0 tx=0x7f9c30007910 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:33.371 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:33.368+0000 7f9c367fc640 1 -- 172.21.15.169:0/3524692457 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9c40155650 con 0x7f9c400b0e50 2023-10-26T01:31:33.564 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:33.561+0000 7f9c46bb8640 1 -- 172.21.15.169:0/3524692457 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9c40155650 con 0x7f9c400b0e50 2023-10-26T01:31:33.565 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:33.562+0000 7f9c367fc640 1 -- 172.21.15.169:0/3524692457 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9c40155650 con 0x7f9c400b0e50 2023-10-26T01:31:33.565 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 32 2023-10-26T01:31:33.567 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:33.564+0000 7f9c46bb8640 1 -- 172.21.15.169:0/3524692457 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9c2406ae40 msgr2=0x7f9c2406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:33.568 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:33.564+0000 7f9c46bb8640 1 --2- 172.21.15.169:0/3524692457 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9c2406ae40 0x7f9c2406d300 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9c4006ccf0 tx=0x7f9c30007910 comp rx=0 tx=0).stop 2023-10-26T01:31:33.568 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:33.564+0000 7f9c46bb8640 1 -- 172.21.15.169:0/3524692457 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c400b0e50 msgr2=0x7f9c400996a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:33.568 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:33.564+0000 7f9c46bb8640 1 --2- 172.21.15.169:0/3524692457 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c400b0e50 0x7f9c400996a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c000c00 tx=0x7f9c3c0038a0 comp rx=0 tx=0).stop 2023-10-26T01:31:33.568 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:33.564+0000 7f9c46bb8640 1 -- 172.21.15.169:0/3524692457 shutdown_connections 2023-10-26T01:31:33.568 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:33.564+0000 7f9c46bb8640 1 --2- 172.21.15.169:0/3524692457 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9c2406ae40 0x7f9c2406d300 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:33.569 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:33.564+0000 7f9c46bb8640 1 --2- 172.21.15.169:0/3524692457 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9c4015d7e0 0x7f9c400c7280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:33.569 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:33.564+0000 7f9c46bb8640 1 --2- 172.21.15.169:0/3524692457 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9c40153b90 0x7f9c401f25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:33.569 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:33.564+0000 7f9c46bb8640 1 --2- 172.21.15.169:0/3524692457 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c400b0e50 0x7f9c400996a0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:33.569 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:33.564+0000 7f9c46bb8640 1 -- 172.21.15.169:0/3524692457 >> 172.21.15.169:0/3524692457 conn(0x7f9c400af110 msgr2=0x7f9c400af4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:33.569 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:33.565+0000 7f9c46bb8640 1 -- 172.21.15.169:0/3524692457 shutdown_connections 2023-10-26T01:31:33.570 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:33.565+0000 7f9c46bb8640 1 -- 172.21.15.169:0/3524692457 wait complete. 2023-10-26T01:31:33.586 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-26T01:31:33.807 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:33.804+0000 7fe6bc19e640 1 -- 172.21.15.169:0/3251580532 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe6a0003970 msgr2=0x7fe6a0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:33.807 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:33.804+0000 7fe6bc19e640 1 --2- 172.21.15.169:0/3251580532 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe6a0003970 0x7fe6a0003d90 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fe6b0006fd0 tx=0x7fe6a00057f0 comp rx=0 tx=0).stop 2023-10-26T01:31:33.808 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:33.804+0000 7fe6bc19e640 1 -- 172.21.15.169:0/3251580532 shutdown_connections 2023-10-26T01:31:33.808 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:33.804+0000 7fe6bc19e640 1 --2- 172.21.15.169:0/3251580532 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe6a0003970 0x7fe6a0003d90 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:33.808 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:33.804+0000 7fe6bc19e640 1 -- 172.21.15.169:0/3251580532 >> 172.21.15.169:0/3251580532 conn(0x7fe6b4069930 msgr2=0x7fe6b4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:33.808 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:33.804+0000 7fe6bc19e640 1 -- 172.21.15.169:0/3251580532 shutdown_connections 2023-10-26T01:31:33.809 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:33.804+0000 7fe6bc19e640 1 -- 172.21.15.169:0/3251580532 wait complete. 2023-10-26T01:31:33.811 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:33.806+0000 7fe6bc19e640 1 Processor -- start 2023-10-26T01:31:33.812 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:33.806+0000 7fe6bc19e640 1 -- start start 2023-10-26T01:31:33.812 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:33.807+0000 7fe6bc19e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe6a0003970 0x7fe6b41f22c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:33.812 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:33.807+0000 7fe6bc19e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe6b409b0d0 0x7fe6b40b0f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:33.812 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:33.807+0000 7fe6bc19e640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe6b40b1470 0x7fe6b4150460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:33.812 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:33.807+0000 7fe6bc19e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b406ed30 con 0x7fe6b409b0d0 2023-10-26T01:31:33.813 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:33.807+0000 7fe6bc19e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b406fb90 con 0x7fe6a0003970 2023-10-26T01:31:33.813 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:33.807+0000 7fe6bc19e640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b406ef10 con 0x7fe6b40b1470 2023-10-26T01:31:33.813 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:33.808+0000 7fe6b9739640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe6b409b0d0 0x7fe6b40b0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:33.813 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:33.808+0000 7fe6b9739640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe6b409b0d0 0x7fe6b40b0f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:40786/0 (socket says 172.21.15.169:40786) 2023-10-26T01:31:33.813 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:33.808+0000 7fe6b9739640 1 -- 172.21.15.169:0/4228189852 learned_addr learned my addr 172.21.15.169:0/4228189852 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:31:33.814 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:33.808+0000 7fe6b9f3a640 1 --2- 172.21.15.169:0/4228189852 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe6a0003970 0x7fe6b41f22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:33.814 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:33.808+0000 7fe6ba73b640 1 --2- 172.21.15.169:0/4228189852 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe6b40b1470 0x7fe6b4150460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:33.814 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:33.808+0000 7fe6b9739640 1 -- 172.21.15.169:0/4228189852 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe6b40b1470 msgr2=0x7fe6b4150460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:33.814 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:33.808+0000 7fe6b9739640 1 --2- 172.21.15.169:0/4228189852 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe6b40b1470 0x7fe6b4150460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:33.814 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:33.808+0000 7fe6b9739640 1 -- 172.21.15.169:0/4228189852 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe6a0003970 msgr2=0x7fe6b41f22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:33.815 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:33.808+0000 7fe6b9739640 1 --2- 172.21.15.169:0/4228189852 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe6a0003970 0x7fe6b41f22c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:33.815 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:33.808+0000 7fe6b9739640 1 -- 172.21.15.169:0/4228189852 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6b4158cd0 con 0x7fe6b409b0d0 2023-10-26T01:31:33.815 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:33.808+0000 7fe6ba73b640 1 --2- 172.21.15.169:0/4228189852 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe6b40b1470 0x7fe6b4150460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:31:33.815 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:33.808+0000 7fe6b9739640 1 --2- 172.21.15.169:0/4228189852 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe6b409b0d0 0x7fe6b40b0f30 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fe6a400fb80 tx=0x7fe6a400d7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:33.815 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:33.809+0000 7fe6aa7fc640 1 -- 172.21.15.169:0/4228189852 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a4004220 con 0x7fe6b409b0d0 2023-10-26T01:31:33.816 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:33.809+0000 7fe6aa7fc640 1 -- 172.21.15.169:0/4228189852 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe6a4004380 con 0x7fe6b409b0d0 2023-10-26T01:31:33.816 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:33.809+0000 7fe6aa7fc640 1 -- 172.21.15.169:0/4228189852 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a4003850 con 0x7fe6b409b0d0 2023-10-26T01:31:33.816 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:33.809+0000 7fe6bc19e640 1 -- 172.21.15.169:0/4228189852 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6b406f1f0 con 0x7fe6b409b0d0 2023-10-26T01:31:33.816 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:33.809+0000 7fe6bc19e640 1 -- 172.21.15.169:0/4228189852 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6b406f7b0 con 0x7fe6b409b0d0 2023-10-26T01:31:33.817 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:33.810+0000 7fe6bc19e640 1 -- 172.21.15.169:0/4228189852 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6b406f970 con 0x7fe6b409b0d0 2023-10-26T01:31:33.818 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:33.811+0000 7fe6aa7fc640 1 -- 172.21.15.169:0/4228189852 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fe6a40039b0 con 0x7fe6b409b0d0 2023-10-26T01:31:33.818 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:33.812+0000 7fe6aa7fc640 1 --2- 172.21.15.169:0/4228189852 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe69806ae70 0x7fe69806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:33.818 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:33.812+0000 7fe6aa7fc640 1 -- 172.21.15.169:0/4228189852 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fe6a408a690 con 0x7fe6b409b0d0 2023-10-26T01:31:33.818 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:33.813+0000 7fe6b9f3a640 1 --2- 172.21.15.169:0/4228189852 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe69806ae70 0x7fe69806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:33.818 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:33.814+0000 7fe6b9f3a640 1 --2- 172.21.15.169:0/4228189852 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe69806ae70 0x7fe69806d330 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe6b0034000 tx=0x7fe6b002e000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:33.819 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:33.816+0000 7fe6aa7fc640 1 -- 172.21.15.169:0/4228189852 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe6a4055620 con 0x7fe6b409b0d0 2023-10-26T01:31:34.009 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:34.007+0000 7fe6bc19e640 1 -- 172.21.15.169:0/4228189852 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe6b406fcd0 con 0x7fe6b409b0d0 2023-10-26T01:31:34.010 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:34.008+0000 7fe6aa7fc640 1 -- 172.21.15.169:0/4228189852 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe6b406fcd0 con 0x7fe6b409b0d0 2023-10-26T01:31:34.011 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 64 2023-10-26T01:31:34.013 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:34.010+0000 7fe6bc19e640 1 -- 172.21.15.169:0/4228189852 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe69806ae70 msgr2=0x7fe69806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:34.013 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:34.010+0000 7fe6bc19e640 1 --2- 172.21.15.169:0/4228189852 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe69806ae70 0x7fe69806d330 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe6b0034000 tx=0x7fe6b002e000 comp rx=0 tx=0).stop 2023-10-26T01:31:34.013 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:34.010+0000 7fe6bc19e640 1 -- 172.21.15.169:0/4228189852 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe6b409b0d0 msgr2=0x7fe6b40b0f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:34.014 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:34.010+0000 7fe6bc19e640 1 --2- 172.21.15.169:0/4228189852 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe6b409b0d0 0x7fe6b40b0f30 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fe6a400fb80 tx=0x7fe6a400d7b0 comp rx=0 tx=0).stop 2023-10-26T01:31:34.014 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:34.010+0000 7fe6bc19e640 1 -- 172.21.15.169:0/4228189852 shutdown_connections 2023-10-26T01:31:34.014 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:34.010+0000 7fe6bc19e640 1 --2- 172.21.15.169:0/4228189852 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe69806ae70 0x7fe69806d330 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:34.014 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:34.010+0000 7fe6bc19e640 1 --2- 172.21.15.169:0/4228189852 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe6b409b0d0 0x7fe6b40b0f30 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:34.014 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:34.010+0000 7fe6bc19e640 1 --2- 172.21.15.169:0/4228189852 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe6b40b1470 0x7fe6b4150460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:34.015 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:34.010+0000 7fe6bc19e640 1 --2- 172.21.15.169:0/4228189852 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe6a0003970 0x7fe6b41f22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:34.015 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:34.010+0000 7fe6bc19e640 1 -- 172.21.15.169:0/4228189852 >> 172.21.15.169:0/4228189852 conn(0x7fe6b4069930 msgr2=0x7fe6b4157180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:34.015 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:34.010+0000 7fe6bc19e640 1 -- 172.21.15.169:0/4228189852 shutdown_connections 2023-10-26T01:31:34.015 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:34.010+0000 7fe6bc19e640 1 -- 172.21.15.169:0/4228189852 wait complete. 2023-10-26T01:31:34.032 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-26T01:31:34.260 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:34.257+0000 7f6b61b95640 1 -- 172.21.15.169:0/3758492912 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b5c155360 msgr2=0x7f6b5c0cd7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:34.260 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:34.257+0000 7f6b61b95640 1 --2- 172.21.15.169:0/3758492912 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b5c155360 0x7f6b5c0cd7e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c06c2a0 tx=0x7f6b4c02bda0 comp rx=0 tx=0).stop 2023-10-26T01:31:34.260 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:34.257+0000 7f6b61b95640 1 -- 172.21.15.169:0/3758492912 shutdown_connections 2023-10-26T01:31:34.260 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:34.257+0000 7f6b61b95640 1 --2- 172.21.15.169:0/3758492912 >> v2:172.21.15.170:3300/0 conn(0x7f6b5c15b9e0 0x7f6b5c06ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:34.261 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:34.257+0000 7f6b61b95640 1 --2- 172.21.15.169:0/3758492912 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6b5c0cdd20 0x7f6b5c15b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:34.261 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:34.257+0000 7f6b61b95640 1 --2- 172.21.15.169:0/3758492912 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b5c155360 0x7f6b5c0cd7e0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:34.261 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:34.257+0000 7f6b61b95640 1 -- 172.21.15.169:0/3758492912 >> 172.21.15.169:0/3758492912 conn(0x7f6b5c069930 msgr2=0x7f6b5c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:34.261 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:34.257+0000 7f6b61b95640 1 -- 172.21.15.169:0/3758492912 shutdown_connections 2023-10-26T01:31:34.261 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:34.257+0000 7f6b61b95640 1 -- 172.21.15.169:0/3758492912 wait complete. 2023-10-26T01:31:34.262 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:34.259+0000 7f6b61b95640 1 Processor -- start 2023-10-26T01:31:34.262 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:34.259+0000 7f6b61b95640 1 -- start start 2023-10-26T01:31:34.264 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:34.260+0000 7f6b61b95640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b5c0cdd20 0x7f6b5c0715f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:34.264 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:34.260+0000 7f6b61b95640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6b5c155360 0x7f6b5c070820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:34.265 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:34.260+0000 7f6b61b95640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b5c15b9e0 0x7f6b5c153b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:34.265 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:34.260+0000 7f6b61b95640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b5c097fc0 con 0x7f6b5c15b9e0 2023-10-26T01:31:34.265 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:34.260+0000 7f6b61b95640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b5c070d60 con 0x7f6b5c0cdd20 2023-10-26T01:31:34.265 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:34.260+0000 7f6b61b95640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b5c0973d0 con 0x7f6b5c155360 2023-10-26T01:31:34.265 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:34.260+0000 7f6b5bfff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b5c15b9e0 0x7f6b5c153b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:34.265 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:34.260+0000 7f6b5bfff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b5c15b9e0 0x7f6b5c153b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:51334/0 (socket says 172.21.15.169:51334) 2023-10-26T01:31:34.266 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:34.260+0000 7f6b5bfff640 1 -- 172.21.15.169:0/2589600567 learned_addr learned my addr 172.21.15.169:0/2589600567 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:31:34.266 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:34.261+0000 7f6b53fff640 1 --2- 172.21.15.169:0/2589600567 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6b5c155360 0x7f6b5c070820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:34.266 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:34.261+0000 7f6b5b7fe640 1 --2- 172.21.15.169:0/2589600567 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b5c0cdd20 0x7f6b5c0715f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:34.266 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:34.261+0000 7f6b5bfff640 1 -- 172.21.15.169:0/2589600567 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6b5c155360 msgr2=0x7f6b5c070820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:34.266 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:34.261+0000 7f6b5bfff640 1 --2- 172.21.15.169:0/2589600567 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6b5c155360 0x7f6b5c070820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:34.267 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:34.261+0000 7f6b5bfff640 1 -- 172.21.15.169:0/2589600567 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b5c0cdd20 msgr2=0x7f6b5c0715f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:34.267 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:34.261+0000 7f6b5bfff640 1 --2- 172.21.15.169:0/2589600567 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b5c0cdd20 0x7f6b5c0715f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:34.267 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:34.261+0000 7f6b5bfff640 1 -- 172.21.15.169:0/2589600567 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b4c009df0 con 0x7f6b5c15b9e0 2023-10-26T01:31:34.267 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:34.261+0000 7f6b53fff640 1 --2- 172.21.15.169:0/2589600567 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6b5c155360 0x7f6b5c070820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:31:34.267 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:34.261+0000 7f6b5b7fe640 1 --2- 172.21.15.169:0/2589600567 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b5c0cdd20 0x7f6b5c0715f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:31:34.268 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:34.261+0000 7f6b5bfff640 1 --2- 172.21.15.169:0/2589600567 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b5c15b9e0 0x7f6b5c153b20 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f6b5400e990 tx=0x7f6b5400ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:34.268 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:34.261+0000 7f6b58ff9640 1 -- 172.21.15.169:0/2589600567 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b54004d20 con 0x7f6b5c15b9e0 2023-10-26T01:31:34.268 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:34.261+0000 7f6b58ff9640 1 -- 172.21.15.169:0/2589600567 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6b5400cb00 con 0x7f6b5c15b9e0 2023-10-26T01:31:34.268 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:34.261+0000 7f6b61b95640 1 -- 172.21.15.169:0/2589600567 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b5c097620 con 0x7f6b5c15b9e0 2023-10-26T01:31:34.268 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:34.261+0000 7f6b58ff9640 1 -- 172.21.15.169:0/2589600567 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b54005020 con 0x7f6b5c15b9e0 2023-10-26T01:31:34.269 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:34.262+0000 7f6b61b95640 1 -- 172.21.15.169:0/2589600567 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b5c097be0 con 0x7f6b5c15b9e0 2023-10-26T01:31:34.269 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:34.263+0000 7f6b61b95640 1 -- 172.21.15.169:0/2589600567 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b5c098100 con 0x7f6b5c15b9e0 2023-10-26T01:31:34.270 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:34.264+0000 7f6b58ff9640 1 -- 172.21.15.169:0/2589600567 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f6b54005180 con 0x7f6b5c15b9e0 2023-10-26T01:31:34.270 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:34.265+0000 7f6b58ff9640 1 --2- 172.21.15.169:0/2589600567 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f6b3c06ae70 0x7f6b3c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:34.270 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:34.265+0000 7f6b58ff9640 1 -- 172.21.15.169:0/2589600567 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f6b5408a120 con 0x7f6b5c15b9e0 2023-10-26T01:31:34.270 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:34.265+0000 7f6b5b7fe640 1 --2- 172.21.15.169:0/2589600567 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f6b3c06ae70 0x7f6b3c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:34.271 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:34.267+0000 7f6b5b7fe640 1 --2- 172.21.15.169:0/2589600567 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f6b3c06ae70 0x7f6b3c06d330 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c035000 tx=0x7f6b4c004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:34.271 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:34.269+0000 7f6b58ff9640 1 -- 172.21.15.169:0/2589600567 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6b5c098100 con 0x7f6b5c15b9e0 2023-10-26T01:31:34.278 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T01:31:34.284 DEBUG:teuthology.orchestra.run.smithi170:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T01:31:34.476 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:34.473+0000 7f6b61b95640 1 -- 172.21.15.169:0/2589600567 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f6b5c098100 con 0x7f6b5c15b9e0 2023-10-26T01:31:34.477 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:34.474+0000 7f6b58ff9640 1 -- 172.21.15.169:0/2589600567 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) v1 ==== 80+0+0 (secure 0 0 0) 0x7f6b5c098100 con 0x7f6b5c15b9e0 2023-10-26T01:31:34.477 INFO:teuthology.orchestra.run.smithi169.stderr:listed 0 entries 2023-10-26T01:31:34.480 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:34.477+0000 7f6b61b95640 1 -- 172.21.15.169:0/2589600567 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f6b3c06ae70 msgr2=0x7f6b3c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:34.480 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:34.477+0000 7f6b61b95640 1 --2- 172.21.15.169:0/2589600567 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f6b3c06ae70 0x7f6b3c06d330 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c035000 tx=0x7f6b4c004730 comp rx=0 tx=0).stop 2023-10-26T01:31:34.480 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:34.477+0000 7f6b61b95640 1 -- 172.21.15.169:0/2589600567 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b5c15b9e0 msgr2=0x7f6b5c153b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:34.480 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:34.477+0000 7f6b61b95640 1 --2- 172.21.15.169:0/2589600567 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b5c15b9e0 0x7f6b5c153b20 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f6b5400e990 tx=0x7f6b5400ee50 comp rx=0 tx=0).stop 2023-10-26T01:31:34.481 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:34.477+0000 7f6b61b95640 1 -- 172.21.15.169:0/2589600567 shutdown_connections 2023-10-26T01:31:34.481 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:34.477+0000 7f6b61b95640 1 --2- 172.21.15.169:0/2589600567 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f6b3c06ae70 0x7f6b3c06d330 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:34.481 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:34.477+0000 7f6b61b95640 1 --2- 172.21.15.169:0/2589600567 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b5c15b9e0 0x7f6b5c153b20 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:34.481 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:34.477+0000 7f6b61b95640 1 --2- 172.21.15.169:0/2589600567 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6b5c155360 0x7f6b5c070820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:34.482 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:34.477+0000 7f6b61b95640 1 --2- 172.21.15.169:0/2589600567 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b5c0cdd20 0x7f6b5c0715f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:34.482 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:34.477+0000 7f6b61b95640 1 -- 172.21.15.169:0/2589600567 >> 172.21.15.169:0/2589600567 conn(0x7f6b5c069930 msgr2=0x7f6b5c0cc8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:34.482 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:34.477+0000 7f6b61b95640 1 -- 172.21.15.169:0/2589600567 shutdown_connections 2023-10-26T01:31:34.482 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:34.477+0000 7f6b61b95640 1 -- 172.21.15.169:0/2589600567 wait complete. 2023-10-26T01:31:34.499 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-26T01:31:34.500 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:31:34.500 DEBUG:teuthology.orchestra.run.smithi170:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2023-10-26T01:31:34.603 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi170.front.sepia.ceph.com 2023-10-26T01:31:34.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:31:34.604 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-26T01:31:34.665 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:31:34.666 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:31:34.726 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:31:34.726 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:31:34.779 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:31:34.780 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:31:34.834 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:31:35.060 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.057+0000 7fdc3ef8e640 1 -- 172.21.15.169:0/3169765734 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdc380ac930 msgr2=0x7fdc380cd6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:35.060 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.057+0000 7fdc3ef8e640 1 --2- 172.21.15.169:0/3169765734 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdc380ac930 0x7fdc380cd6f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fdc3806be60 tx=0x7fdc2402bcf0 comp rx=0 tx=0).stop 2023-10-26T01:31:35.061 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.057+0000 7fdc3ef8e640 1 -- 172.21.15.169:0/3169765734 shutdown_connections 2023-10-26T01:31:35.061 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.057+0000 7fdc3ef8e640 1 --2- 172.21.15.169:0/3169765734 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdc380ac930 0x7fdc380cd6f0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:35.061 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.057+0000 7fdc3ef8e640 1 --2- 172.21.15.169:0/3169765734 >> v2:172.21.15.170:3300/0 conn(0x7fdc381592d0 0x7fdc380aa3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:35.061 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.057+0000 7fdc3ef8e640 1 -- 172.21.15.169:0/3169765734 >> 172.21.15.169:0/3169765734 conn(0x7fdc38069930 msgr2=0x7fdc38069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:35.061 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.057+0000 7fdc3ef8e640 1 -- 172.21.15.169:0/3169765734 shutdown_connections 2023-10-26T01:31:35.062 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.057+0000 7fdc3ef8e640 1 -- 172.21.15.169:0/3169765734 wait complete. 2023-10-26T01:31:35.064 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.059+0000 7fdc3ef8e640 1 Processor -- start 2023-10-26T01:31:35.064 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.059+0000 7fdc3ef8e640 1 -- start start 2023-10-26T01:31:35.064 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.060+0000 7fdc3ef8e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdc380ac930 0x7fdc38151730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:35.065 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.060+0000 7fdc3ef8e640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdc380cdc30 0x7fdc381f02c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:35.065 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.060+0000 7fdc3ef8e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdc381592d0 0x7fdc38159a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:35.065 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.060+0000 7fdc3ef8e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc3815c970 con 0x7fdc381592d0 2023-10-26T01:31:35.065 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.060+0000 7fdc3ef8e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc38070340 con 0x7fdc380ac930 2023-10-26T01:31:35.065 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.060+0000 7fdc3ef8e640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc381f0800 con 0x7fdc380cdc30 2023-10-26T01:31:35.066 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.060+0000 7fdc3d52b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdc381592d0 0x7fdc38159a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:35.066 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.060+0000 7fdc3d52b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdc381592d0 0x7fdc38159a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:51346/0 (socket says 172.21.15.169:51346) 2023-10-26T01:31:35.066 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.060+0000 7fdc3d52b640 1 -- 172.21.15.169:0/880870843 learned_addr learned my addr 172.21.15.169:0/880870843 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:31:35.066 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.060+0000 7fdc3cd2a640 1 --2- 172.21.15.169:0/880870843 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdc380ac930 0x7fdc38151730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:35.067 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.061+0000 7fdc37fff640 1 --2- 172.21.15.169:0/880870843 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdc380cdc30 0x7fdc381f02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:35.067 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.061+0000 7fdc3d52b640 1 -- 172.21.15.169:0/880870843 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdc380cdc30 msgr2=0x7fdc381f02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:35.067 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.061+0000 7fdc3d52b640 1 --2- 172.21.15.169:0/880870843 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdc380cdc30 0x7fdc381f02c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:35.067 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.061+0000 7fdc3d52b640 1 -- 172.21.15.169:0/880870843 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdc380ac930 msgr2=0x7fdc38151730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:35.067 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.061+0000 7fdc3d52b640 1 --2- 172.21.15.169:0/880870843 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdc380ac930 0x7fdc38151730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:35.068 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.061+0000 7fdc3d52b640 1 -- 172.21.15.169:0/880870843 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc24009490 con 0x7fdc381592d0 2023-10-26T01:31:35.068 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.061+0000 7fdc3d52b640 1 --2- 172.21.15.169:0/880870843 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdc381592d0 0x7fdc38159a20 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fdc3000cb70 tx=0x7fdc3000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:35.068 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.061+0000 7fdc357fa640 1 -- 172.21.15.169:0/880870843 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc3000ad40 con 0x7fdc381592d0 2023-10-26T01:31:35.068 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.061+0000 7fdc357fa640 1 -- 172.21.15.169:0/880870843 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fdc30012420 con 0x7fdc381592d0 2023-10-26T01:31:35.068 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.061+0000 7fdc357fa640 1 -- 172.21.15.169:0/880870843 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc30012810 con 0x7fdc381592d0 2023-10-26T01:31:35.069 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.061+0000 7fdc3ef8e640 1 -- 172.21.15.169:0/880870843 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc3806fd30 con 0x7fdc381592d0 2023-10-26T01:31:35.069 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.062+0000 7fdc3ef8e640 1 -- 172.21.15.169:0/880870843 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc3815c690 con 0x7fdc381592d0 2023-10-26T01:31:35.070 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.063+0000 7fdc3ef8e640 1 -- 172.21.15.169:0/880870843 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc380704f0 con 0x7fdc381592d0 2023-10-26T01:31:35.070 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.064+0000 7fdc357fa640 1 -- 172.21.15.169:0/880870843 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fdc3001a3b0 con 0x7fdc381592d0 2023-10-26T01:31:35.071 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.064+0000 7fdc357fa640 1 --2- 172.21.15.169:0/880870843 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fdc1c06ae40 0x7fdc1c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:35.071 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.065+0000 7fdc357fa640 1 -- 172.21.15.169:0/880870843 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fdc30089d90 con 0x7fdc381592d0 2023-10-26T01:31:35.071 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.065+0000 7fdc3cd2a640 1 --2- 172.21.15.169:0/880870843 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fdc1c06ae40 0x7fdc1c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:35.071 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.066+0000 7fdc3cd2a640 1 --2- 172.21.15.169:0/880870843 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fdc1c06ae40 0x7fdc1c06d300 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c00bb50 tx=0x7fdc2c00b2e0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:35.072 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.069+0000 7fdc357fa640 1 -- 172.21.15.169:0/880870843 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdc380704f0 con 0x7fdc381592d0 2023-10-26T01:31:35.264 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.262+0000 7fdc3ef8e640 1 -- 172.21.15.169:0/880870843 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdc380704f0 con 0x7fdc381592d0 2023-10-26T01:31:35.266 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.263+0000 7fdc357fa640 1 -- 172.21.15.169:0/880870843 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16801 (secure 0 0 0) 0x7fdc380704f0 con 0x7fdc381592d0 2023-10-26T01:31:35.267 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:31:35.267 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":23,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:31:30.100039+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-26T01:30:39.437631+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-26T01:30:40.561774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":2}]},{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.24","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:31:35.269 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.266+0000 7fdc3ef8e640 1 -- 172.21.15.169:0/880870843 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fdc1c06ae40 msgr2=0x7fdc1c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:35.270 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.266+0000 7fdc3ef8e640 1 --2- 172.21.15.169:0/880870843 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fdc1c06ae40 0x7fdc1c06d300 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c00bb50 tx=0x7fdc2c00b2e0 comp rx=0 tx=0).stop 2023-10-26T01:31:35.270 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.266+0000 7fdc3ef8e640 1 -- 172.21.15.169:0/880870843 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdc381592d0 msgr2=0x7fdc38159a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:35.270 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.266+0000 7fdc3ef8e640 1 --2- 172.21.15.169:0/880870843 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdc381592d0 0x7fdc38159a20 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fdc3000cb70 tx=0x7fdc3000a430 comp rx=0 tx=0).stop 2023-10-26T01:31:35.270 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.267+0000 7fdc3ef8e640 1 -- 172.21.15.169:0/880870843 shutdown_connections 2023-10-26T01:31:35.271 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.267+0000 7fdc3ef8e640 1 --2- 172.21.15.169:0/880870843 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fdc1c06ae40 0x7fdc1c06d300 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:35.271 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.267+0000 7fdc3ef8e640 1 --2- 172.21.15.169:0/880870843 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdc381592d0 0x7fdc38159a20 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:35.271 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.267+0000 7fdc3ef8e640 1 --2- 172.21.15.169:0/880870843 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdc380cdc30 0x7fdc381f02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:35.271 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.267+0000 7fdc3ef8e640 1 --2- 172.21.15.169:0/880870843 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdc380ac930 0x7fdc38151730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:35.272 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.267+0000 7fdc3ef8e640 1 -- 172.21.15.169:0/880870843 >> 172.21.15.169:0/880870843 conn(0x7fdc38069930 msgr2=0x7fdc3806ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:35.272 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.267+0000 7fdc3ef8e640 1 -- 172.21.15.169:0/880870843 shutdown_connections 2023-10-26T01:31:35.272 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.267+0000 7fdc3ef8e640 1 -- 172.21.15.169:0/880870843 wait complete. 2023-10-26T01:31:35.288 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T01:30:32.200808+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-26T01:30:39.437631+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-26T01:30:40.561774+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T01:31:35.289 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T01:31:35.516 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.513+0000 7f9194f73640 1 -- 172.21.15.169:0/4138691581 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9190159480 msgr2=0x7f91900b02e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:35.517 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.513+0000 7f9194f73640 1 --2- 172.21.15.169:0/4138691581 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9190159480 0x7f91900b02e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9180005fd0 tx=0x7f918002c170 comp rx=0 tx=0).stop 2023-10-26T01:31:35.517 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.513+0000 7f9194f73640 1 -- 172.21.15.169:0/4138691581 shutdown_connections 2023-10-26T01:31:35.517 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.513+0000 7f9194f73640 1 --2- 172.21.15.169:0/4138691581 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9190159480 0x7f91900b02e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:35.517 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.513+0000 7f9194f73640 1 -- 172.21.15.169:0/4138691581 >> 172.21.15.169:0/4138691581 conn(0x7f9190069930 msgr2=0x7f9190069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:35.517 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.513+0000 7f9194f73640 1 -- 172.21.15.169:0/4138691581 shutdown_connections 2023-10-26T01:31:35.518 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.513+0000 7f9194f73640 1 -- 172.21.15.169:0/4138691581 wait complete. 2023-10-26T01:31:35.518 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.515+0000 7f9194f73640 1 Processor -- start 2023-10-26T01:31:35.519 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.515+0000 7f9194f73640 1 -- start start 2023-10-26T01:31:35.521 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.516+0000 7f9194f73640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91900b0820 0x7f91901e32e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:35.521 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.516+0000 7f9194f73640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f91900cd690 0x7f91901f59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:35.521 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.516+0000 7f9194f73640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9190159480 0x7f91900ac540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:35.521 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.516+0000 7f9194f73640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91901e49a0 con 0x7f91900cd690 2023-10-26T01:31:35.521 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.516+0000 7f9194f73640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91901e3900 con 0x7f91900b0820 2023-10-26T01:31:35.522 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.517+0000 7f9194f73640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91901e3be0 con 0x7f9190159480 2023-10-26T01:31:35.522 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.517+0000 7f918dd9b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f91900cd690 0x7f91901f59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:35.522 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.517+0000 7f918dd9b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f91900cd690 0x7f91901f59c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:51354/0 (socket says 172.21.15.169:51354) 2023-10-26T01:31:35.522 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.517+0000 7f918dd9b640 1 -- 172.21.15.169:0/3183717271 learned_addr learned my addr 172.21.15.169:0/3183717271 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:31:35.522 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.517+0000 7f918e59c640 1 --2- 172.21.15.169:0/3183717271 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91900b0820 0x7f91901e32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:35.523 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.517+0000 7f918ed9d640 1 --2- 172.21.15.169:0/3183717271 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9190159480 0x7f91900ac540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:35.523 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.517+0000 7f918dd9b640 1 -- 172.21.15.169:0/3183717271 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9190159480 msgr2=0x7f91900ac540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:35.523 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.517+0000 7f918dd9b640 1 --2- 172.21.15.169:0/3183717271 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9190159480 0x7f91900ac540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:35.523 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.517+0000 7f918dd9b640 1 -- 172.21.15.169:0/3183717271 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91900b0820 msgr2=0x7f91901e32e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:35.523 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.517+0000 7f918dd9b640 1 --2- 172.21.15.169:0/3183717271 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91900b0820 0x7f91901e32e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:35.524 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.517+0000 7f918dd9b640 1 -- 172.21.15.169:0/3183717271 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9180009840 con 0x7f91900cd690 2023-10-26T01:31:35.524 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.518+0000 7f918dd9b640 1 --2- 172.21.15.169:0/3183717271 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f91900cd690 0x7f91901f59c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f917800c4d0 tx=0x7f917800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:35.524 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.518+0000 7f9186ffd640 1 -- 172.21.15.169:0/3183717271 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9178012830 con 0x7f91900cd690 2023-10-26T01:31:35.524 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.518+0000 7f9186ffd640 1 -- 172.21.15.169:0/3183717271 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9178012aa0 con 0x7f91900cd690 2023-10-26T01:31:35.524 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.518+0000 7f9186ffd640 1 -- 172.21.15.169:0/3183717271 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9178011380 con 0x7f91900cd690 2023-10-26T01:31:35.525 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.518+0000 7f9194f73640 1 -- 172.21.15.169:0/3183717271 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91901e3ec0 con 0x7f91900cd690 2023-10-26T01:31:35.525 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.518+0000 7f9194f73640 1 -- 172.21.15.169:0/3183717271 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91901e4480 con 0x7f91900cd690 2023-10-26T01:31:35.526 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.519+0000 7f9194f73640 1 -- 172.21.15.169:0/3183717271 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9190066af0 con 0x7f91900cd690 2023-10-26T01:31:35.526 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.520+0000 7f9186ffd640 1 -- 172.21.15.169:0/3183717271 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f91780114e0 con 0x7f91900cd690 2023-10-26T01:31:35.526 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.521+0000 7f9186ffd640 1 --2- 172.21.15.169:0/3183717271 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f917006ae40 0x7f917006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:35.527 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.521+0000 7f9186ffd640 1 -- 172.21.15.169:0/3183717271 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f9178089870 con 0x7f91900cd690 2023-10-26T01:31:35.527 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.521+0000 7f918e59c640 1 --2- 172.21.15.169:0/3183717271 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f917006ae40 0x7f917006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:35.527 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.522+0000 7f918e59c640 1 --2- 172.21.15.169:0/3183717271 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f917006ae40 0x7f917006d300 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9180035000 tx=0x7f9180004ab0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:35.528 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.525+0000 7f9186ffd640 1 -- 172.21.15.169:0/3183717271 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9178054890 con 0x7f91900cd690 2023-10-26T01:31:35.717 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.714+0000 7f9194f73640 1 -- 172.21.15.169:0/3183717271 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f919004da70 con 0x7f91900cd690 2023-10-26T01:31:35.718 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.715+0000 7f9186ffd640 1 -- 172.21.15.169:0/3183717271 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f91780160a0 con 0x7f91900cd690 2023-10-26T01:31:35.718 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 1 2023-10-26T01:31:35.721 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.718+0000 7f9194f73640 1 -- 172.21.15.169:0/3183717271 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f917006ae40 msgr2=0x7f917006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:35.721 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.718+0000 7f9194f73640 1 --2- 172.21.15.169:0/3183717271 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f917006ae40 0x7f917006d300 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9180035000 tx=0x7f9180004ab0 comp rx=0 tx=0).stop 2023-10-26T01:31:35.721 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.718+0000 7f9194f73640 1 -- 172.21.15.169:0/3183717271 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f91900cd690 msgr2=0x7f91901f59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:35.721 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.718+0000 7f9194f73640 1 --2- 172.21.15.169:0/3183717271 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f91900cd690 0x7f91901f59c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f917800c4d0 tx=0x7f917800c990 comp rx=0 tx=0).stop 2023-10-26T01:31:35.722 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.718+0000 7f9194f73640 1 -- 172.21.15.169:0/3183717271 shutdown_connections 2023-10-26T01:31:35.722 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.718+0000 7f9194f73640 1 --2- 172.21.15.169:0/3183717271 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f917006ae40 0x7f917006d300 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:35.722 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.718+0000 7f9194f73640 1 --2- 172.21.15.169:0/3183717271 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f91900cd690 0x7f91901f59c0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:35.722 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.718+0000 7f9194f73640 1 --2- 172.21.15.169:0/3183717271 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9190159480 0x7f91900ac540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:35.723 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.718+0000 7f9194f73640 1 --2- 172.21.15.169:0/3183717271 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91900b0820 0x7f91901e32e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:35.723 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.718+0000 7f9194f73640 1 -- 172.21.15.169:0/3183717271 >> 172.21.15.169:0/3183717271 conn(0x7f9190069930 msgr2=0x7f919014f540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:35.723 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.718+0000 7f9194f73640 1 -- 172.21.15.169:0/3183717271 shutdown_connections 2023-10-26T01:31:35.723 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.718+0000 7f9194f73640 1 -- 172.21.15.169:0/3183717271 wait complete. 2023-10-26T01:31:35.741 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-26T01:31:35.962 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.958+0000 7fcd6687c640 1 -- 172.21.15.169:0/448139975 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd6015a450 msgr2=0x7fcd600cb110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:35.962 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.958+0000 7fcd6687c640 1 --2- 172.21.15.169:0/448139975 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd6015a450 0x7fcd600cb110 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fcd6006d530 tx=0x7fcd5002c260 comp rx=0 tx=0).stop 2023-10-26T01:31:35.962 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.959+0000 7fcd6687c640 1 -- 172.21.15.169:0/448139975 shutdown_connections 2023-10-26T01:31:35.962 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.959+0000 7fcd6687c640 1 --2- 172.21.15.169:0/448139975 >> v2:172.21.15.170:3300/0 conn(0x7fcd6015a830 0x7fcd60069a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:35.963 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.959+0000 7fcd6687c640 1 --2- 172.21.15.169:0/448139975 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcd6015de30 0x7fcd60069510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:35.963 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.959+0000 7fcd6687c640 1 --2- 172.21.15.169:0/448139975 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd6015a450 0x7fcd600cb110 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:35.963 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.959+0000 7fcd6687c640 1 -- 172.21.15.169:0/448139975 >> 172.21.15.169:0/448139975 conn(0x7fcd600b0200 msgr2=0x7fcd600b05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:35.963 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.959+0000 7fcd6687c640 1 -- 172.21.15.169:0/448139975 shutdown_connections 2023-10-26T01:31:35.964 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.959+0000 7fcd6687c640 1 -- 172.21.15.169:0/448139975 wait complete. 2023-10-26T01:31:35.966 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.960+0000 7fcd6687c640 1 Processor -- start 2023-10-26T01:31:35.966 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.961+0000 7fcd6687c640 1 -- start start 2023-10-26T01:31:35.967 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.962+0000 7fcd6687c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd6015a450 0x7fcd6015b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:35.967 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.962+0000 7fcd6687c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcd6015a830 0x7fcd60151b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:35.967 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.962+0000 7fcd6687c640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcd6015de30 0x7fcd601520b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:35.967 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.962+0000 7fcd6687c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd601f5960 con 0x7fcd6015a450 2023-10-26T01:31:35.967 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.962+0000 7fcd6687c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd601f2ae0 con 0x7fcd6015a830 2023-10-26T01:31:35.968 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.962+0000 7fcd6687c640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd601f2800 con 0x7fcd6015de30 2023-10-26T01:31:35.968 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.962+0000 7fcd5ffff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd6015a450 0x7fcd6015b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:35.968 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.962+0000 7fcd5ffff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd6015a450 0x7fcd6015b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:51368/0 (socket says 172.21.15.169:51368) 2023-10-26T01:31:35.968 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.962+0000 7fcd5ffff640 1 -- 172.21.15.169:0/2953820696 learned_addr learned my addr 172.21.15.169:0/2953820696 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:31:35.968 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.962+0000 7fcd64e19640 1 --2- 172.21.15.169:0/2953820696 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcd6015de30 0x7fcd601520b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:35.969 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.962+0000 7fcd5f7fe640 1 --2- 172.21.15.169:0/2953820696 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcd6015a830 0x7fcd60151b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:35.969 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.962+0000 7fcd5ffff640 1 -- 172.21.15.169:0/2953820696 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcd6015de30 msgr2=0x7fcd601520b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:35.969 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.962+0000 7fcd5ffff640 1 --2- 172.21.15.169:0/2953820696 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcd6015de30 0x7fcd601520b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:35.969 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.962+0000 7fcd5ffff640 1 -- 172.21.15.169:0/2953820696 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcd6015a830 msgr2=0x7fcd60151b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:35.970 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.962+0000 7fcd5ffff640 1 --2- 172.21.15.169:0/2953820696 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcd6015a830 0x7fcd60151b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:35.970 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.962+0000 7fcd5ffff640 1 -- 172.21.15.169:0/2953820696 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd50009900 con 0x7fcd6015a450 2023-10-26T01:31:35.970 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.963+0000 7fcd5ffff640 1 --2- 172.21.15.169:0/2953820696 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd6015a450 0x7fcd6015b4c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fcd50000f80 tx=0x7fcd50003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:35.970 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.963+0000 7fcd5cff9640 1 -- 172.21.15.169:0/2953820696 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd50004450 con 0x7fcd6015a450 2023-10-26T01:31:35.970 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.963+0000 7fcd5cff9640 1 -- 172.21.15.169:0/2953820696 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fcd500046c0 con 0x7fcd6015a450 2023-10-26T01:31:35.971 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.963+0000 7fcd6687c640 1 -- 172.21.15.169:0/2953820696 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd601f52a0 con 0x7fcd6015a450 2023-10-26T01:31:35.971 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.963+0000 7fcd5cff9640 1 -- 172.21.15.169:0/2953820696 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd500406b0 con 0x7fcd6015a450 2023-10-26T01:31:35.971 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.963+0000 7fcd6687c640 1 -- 172.21.15.169:0/2953820696 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd6015ba00 con 0x7fcd6015a450 2023-10-26T01:31:35.972 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.964+0000 7fcd6687c640 1 -- 172.21.15.169:0/2953820696 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd6015bc50 con 0x7fcd6015a450 2023-10-26T01:31:35.972 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.965+0000 7fcd5cff9640 1 -- 172.21.15.169:0/2953820696 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fcd5000a040 con 0x7fcd6015a450 2023-10-26T01:31:35.972 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.966+0000 7fcd5cff9640 1 --2- 172.21.15.169:0/2953820696 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcd4006ae40 0x7fcd4006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:35.973 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.966+0000 7fcd5cff9640 1 -- 172.21.15.169:0/2953820696 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fcd500ac6b0 con 0x7fcd6015a450 2023-10-26T01:31:35.973 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.967+0000 7fcd5f7fe640 1 --2- 172.21.15.169:0/2953820696 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcd4006ae40 0x7fcd4006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:35.973 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.968+0000 7fcd5f7fe640 1 --2- 172.21.15.169:0/2953820696 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcd4006ae40 0x7fcd4006d300 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fcd600ca7f0 tx=0x7fcd48005f90 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:35.974 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:35.971+0000 7fcd5cff9640 1 -- 172.21.15.169:0/2953820696 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcd500777f0 con 0x7fcd6015a450 2023-10-26T01:31:36.164 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:36.161+0000 7fcd6687c640 1 -- 172.21.15.169:0/2953820696 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcd601f5100 con 0x7fcd6015a450 2023-10-26T01:31:36.166 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:36.164+0000 7fcd5cff9640 1 -- 172.21.15.169:0/2953820696 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcd601f5100 con 0x7fcd6015a450 2023-10-26T01:31:36.166 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 32 2023-10-26T01:31:36.170 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:36.167+0000 7fcd6687c640 1 -- 172.21.15.169:0/2953820696 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcd4006ae40 msgr2=0x7fcd4006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:36.170 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:36.167+0000 7fcd6687c640 1 --2- 172.21.15.169:0/2953820696 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcd4006ae40 0x7fcd4006d300 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fcd600ca7f0 tx=0x7fcd48005f90 comp rx=0 tx=0).stop 2023-10-26T01:31:36.171 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:36.167+0000 7fcd6687c640 1 -- 172.21.15.169:0/2953820696 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd6015a450 msgr2=0x7fcd6015b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:36.171 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:36.167+0000 7fcd6687c640 1 --2- 172.21.15.169:0/2953820696 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd6015a450 0x7fcd6015b4c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fcd50000f80 tx=0x7fcd50003c20 comp rx=0 tx=0).stop 2023-10-26T01:31:36.171 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:36.167+0000 7fcd6687c640 1 -- 172.21.15.169:0/2953820696 shutdown_connections 2023-10-26T01:31:36.171 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:36.167+0000 7fcd6687c640 1 --2- 172.21.15.169:0/2953820696 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcd4006ae40 0x7fcd4006d300 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:36.172 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:36.167+0000 7fcd6687c640 1 --2- 172.21.15.169:0/2953820696 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcd6015de30 0x7fcd601520b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:36.172 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:36.167+0000 7fcd6687c640 1 --2- 172.21.15.169:0/2953820696 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcd6015a830 0x7fcd60151b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:36.172 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:36.167+0000 7fcd6687c640 1 --2- 172.21.15.169:0/2953820696 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd6015a450 0x7fcd6015b4c0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:36.172 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:36.167+0000 7fcd6687c640 1 -- 172.21.15.169:0/2953820696 >> 172.21.15.169:0/2953820696 conn(0x7fcd600b0200 msgr2=0x7fcd600707f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:36.172 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:36.167+0000 7fcd6687c640 1 -- 172.21.15.169:0/2953820696 shutdown_connections 2023-10-26T01:31:36.173 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:36.167+0000 7fcd6687c640 1 -- 172.21.15.169:0/2953820696 wait complete. 2023-10-26T01:31:36.189 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-26T01:31:36.413 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:36.410+0000 7f5338d18640 1 -- 172.21.15.169:0/2194133368 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f533406ce50 msgr2=0x7f5334158bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:36.413 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:36.410+0000 7f5338d18640 1 --2- 172.21.15.169:0/2194133368 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f533406ce50 0x7f5334158bd0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f532c005fd0 tx=0x7f532c02c060 comp rx=0 tx=0).stop 2023-10-26T01:31:36.413 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:36.410+0000 7f5338d18640 1 -- 172.21.15.169:0/2194133368 shutdown_connections 2023-10-26T01:31:36.414 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:36.410+0000 7f5338d18640 1 --2- 172.21.15.169:0/2194133368 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f533406ce50 0x7f5334158bd0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:36.414 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:36.410+0000 7f5338d18640 1 --2- 172.21.15.169:0/2194133368 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f53340ab300 0x7f533406b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:36.414 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:36.410+0000 7f5338d18640 1 -- 172.21.15.169:0/2194133368 >> 172.21.15.169:0/2194133368 conn(0x7f5334069930 msgr2=0x7f5334069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:36.414 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:36.410+0000 7f5338d18640 1 -- 172.21.15.169:0/2194133368 shutdown_connections 2023-10-26T01:31:36.415 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:36.410+0000 7f5338d18640 1 -- 172.21.15.169:0/2194133368 wait complete. 2023-10-26T01:31:36.416 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:36.412+0000 7f5338d18640 1 Processor -- start 2023-10-26T01:31:36.416 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:36.412+0000 7f5338d18640 1 -- start start 2023-10-26T01:31:36.416 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:36.413+0000 7f5338d18640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f533406ce50 0x7f53340ae140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:36.416 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:36.413+0000 7f5338d18640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f53340ab300 0x7f53340af730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:36.417 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:36.413+0000 7f5338d18640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5334157be0 0x7f533414eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:36.417 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:36.413+0000 7f5338d18640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53340ace50 con 0x7f5334157be0 2023-10-26T01:31:36.417 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:36.413+0000 7f5338d18640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53340ad0a0 con 0x7f53340ab300 2023-10-26T01:31:36.417 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:36.413+0000 7f5338d18640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53340ad380 con 0x7f533406ce50 2023-10-26T01:31:36.417 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:36.413+0000 7f5332d9d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5334157be0 0x7f533414eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:36.418 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:36.413+0000 7f5332d9d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5334157be0 0x7f533414eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:51394/0 (socket says 172.21.15.169:51394) 2023-10-26T01:31:36.418 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:36.413+0000 7f5332d9d640 1 -- 172.21.15.169:0/1611711596 learned_addr learned my addr 172.21.15.169:0/1611711596 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:31:36.420 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:36.414+0000 7f533259c640 1 --2- 172.21.15.169:0/1611711596 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f533406ce50 0x7f53340ae140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:36.420 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:36.414+0000 7f5331d9b640 1 --2- 172.21.15.169:0/1611711596 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f53340ab300 0x7f53340af730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:36.420 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:36.414+0000 7f5332d9d640 1 -- 172.21.15.169:0/1611711596 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f533406ce50 msgr2=0x7f53340ae140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:36.421 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:36.414+0000 7f5332d9d640 1 --2- 172.21.15.169:0/1611711596 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f533406ce50 0x7f53340ae140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:36.421 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:36.414+0000 7f5332d9d640 1 -- 172.21.15.169:0/1611711596 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f53340ab300 msgr2=0x7f53340af730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:36.421 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:36.414+0000 7f5332d9d640 1 --2- 172.21.15.169:0/1611711596 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f53340ab300 0x7f53340af730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:36.421 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:36.414+0000 7f5332d9d640 1 -- 172.21.15.169:0/1611711596 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f532c009840 con 0x7f5334157be0 2023-10-26T01:31:36.421 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:36.414+0000 7f533259c640 1 --2- 172.21.15.169:0/1611711596 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f533406ce50 0x7f53340ae140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:31:36.422 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:36.414+0000 7f5332d9d640 1 --2- 172.21.15.169:0/1611711596 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5334157be0 0x7f533414eaf0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f532c036040 tx=0x7f532c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:36.422 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:36.414+0000 7f5322ffd640 1 -- 172.21.15.169:0/1611711596 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f532c004870 con 0x7f5334157be0 2023-10-26T01:31:36.423 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:36.414+0000 7f5322ffd640 1 -- 172.21.15.169:0/1611711596 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f532c03d430 con 0x7f5334157be0 2023-10-26T01:31:36.423 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:36.414+0000 7f5322ffd640 1 -- 172.21.15.169:0/1611711596 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f532c03d860 con 0x7f5334157be0 2023-10-26T01:31:36.423 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:36.414+0000 7f5338d18640 1 -- 172.21.15.169:0/1611711596 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53340ad660 con 0x7f5334157be0 2023-10-26T01:31:36.423 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:36.415+0000 7f5338d18640 1 -- 172.21.15.169:0/1611711596 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53340adc20 con 0x7f5334157be0 2023-10-26T01:31:36.423 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:36.416+0000 7f5338d18640 1 -- 172.21.15.169:0/1611711596 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f533404db10 con 0x7f5334157be0 2023-10-26T01:31:36.423 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:36.417+0000 7f5322ffd640 1 -- 172.21.15.169:0/1611711596 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f532c03d9c0 con 0x7f5334157be0 2023-10-26T01:31:36.424 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:36.417+0000 7f5322ffd640 1 --2- 172.21.15.169:0/1611711596 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f531406ae70 0x7f531406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:36.424 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:36.418+0000 7f5322ffd640 1 -- 172.21.15.169:0/1611711596 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f532c0ac890 con 0x7f5334157be0 2023-10-26T01:31:36.424 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:36.418+0000 7f533259c640 1 --2- 172.21.15.169:0/1611711596 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f531406ae70 0x7f531406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:36.425 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:36.419+0000 7f533259c640 1 --2- 172.21.15.169:0/1611711596 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f531406ae70 0x7f531406d330 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5324001620 tx=0x7f5324009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:36.425 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:36.422+0000 7f5322ffd640 1 -- 172.21.15.169:0/1611711596 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f532c077830 con 0x7f5334157be0 2023-10-26T01:31:36.617 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:36.615+0000 7f5338d18640 1 -- 172.21.15.169:0/1611711596 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f533406a1f0 con 0x7f5334157be0 2023-10-26T01:31:36.618 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:36.616+0000 7f5322ffd640 1 -- 172.21.15.169:0/1611711596 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f532c041020 con 0x7f5334157be0 2023-10-26T01:31:36.619 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 64 2023-10-26T01:31:36.621 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:36.618+0000 7f5338d18640 1 -- 172.21.15.169:0/1611711596 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f531406ae70 msgr2=0x7f531406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:36.622 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:36.618+0000 7f5338d18640 1 --2- 172.21.15.169:0/1611711596 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f531406ae70 0x7f531406d330 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5324001620 tx=0x7f5324009040 comp rx=0 tx=0).stop 2023-10-26T01:31:36.622 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:36.619+0000 7f5338d18640 1 -- 172.21.15.169:0/1611711596 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5334157be0 msgr2=0x7f533414eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:36.622 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:36.619+0000 7f5338d18640 1 --2- 172.21.15.169:0/1611711596 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5334157be0 0x7f533414eaf0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f532c036040 tx=0x7f532c004270 comp rx=0 tx=0).stop 2023-10-26T01:31:36.622 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:36.619+0000 7f5338d18640 1 -- 172.21.15.169:0/1611711596 shutdown_connections 2023-10-26T01:31:36.623 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:36.619+0000 7f5338d18640 1 --2- 172.21.15.169:0/1611711596 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f531406ae70 0x7f531406d330 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:36.623 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:36.619+0000 7f5338d18640 1 --2- 172.21.15.169:0/1611711596 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5334157be0 0x7f533414eaf0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:36.623 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:36.619+0000 7f5338d18640 1 --2- 172.21.15.169:0/1611711596 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f53340ab300 0x7f53340af730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:36.623 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:36.619+0000 7f5338d18640 1 --2- 172.21.15.169:0/1611711596 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f533406ce50 0x7f53340ae140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:36.623 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:36.619+0000 7f5338d18640 1 -- 172.21.15.169:0/1611711596 >> 172.21.15.169:0/1611711596 conn(0x7f5334069930 msgr2=0x7f5334153810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:36.625 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:36.619+0000 7f5338d18640 1 -- 172.21.15.169:0/1611711596 shutdown_connections 2023-10-26T01:31:36.625 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:36.619+0000 7f5338d18640 1 -- 172.21.15.169:0/1611711596 wait complete. 2023-10-26T01:31:36.641 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-26T01:31:36.864 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:36.861+0000 7efd6b55a640 1 -- 172.21.15.169:0/3399308815 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd641593e0 msgr2=0x7efd640ac200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:36.864 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:36.861+0000 7efd6b55a640 1 --2- 172.21.15.169:0/3399308815 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd641593e0 0x7efd640ac200 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7efd640c9aa0 tx=0x7efd6002c260 comp rx=0 tx=0).stop 2023-10-26T01:31:36.864 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:36.861+0000 7efd6b55a640 1 -- 172.21.15.169:0/3399308815 shutdown_connections 2023-10-26T01:31:36.865 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:36.861+0000 7efd6b55a640 1 --2- 172.21.15.169:0/3399308815 >> v2:172.21.15.170:3300/0 conn(0x7efd640b0c80 0x7efd640cd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:36.865 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:36.861+0000 7efd6b55a640 1 --2- 172.21.15.169:0/3399308815 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd641593e0 0x7efd640ac200 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:36.865 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:36.861+0000 7efd6b55a640 1 -- 172.21.15.169:0/3399308815 >> 172.21.15.169:0/3399308815 conn(0x7efd64069930 msgr2=0x7efd64069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:36.865 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:36.861+0000 7efd6b55a640 1 -- 172.21.15.169:0/3399308815 shutdown_connections 2023-10-26T01:31:36.865 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:36.861+0000 7efd6b55a640 1 -- 172.21.15.169:0/3399308815 wait complete. 2023-10-26T01:31:36.866 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:36.863+0000 7efd6b55a640 1 Processor -- start 2023-10-26T01:31:36.866 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:36.863+0000 7efd6b55a640 1 -- start start 2023-10-26T01:31:36.868 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:36.864+0000 7efd6b55a640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd640b0c80 0x7efd6406fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:36.868 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:36.864+0000 7efd6b55a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd641593e0 0x7efd64071400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:36.868 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:36.864+0000 7efd6b55a640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7efd641597c0 0x7efd64151fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:36.868 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:36.864+0000 7efd6b55a640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd64070f70 con 0x7efd640b0c80 2023-10-26T01:31:36.868 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:36.864+0000 7efd6b55a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd64070360 con 0x7efd641593e0 2023-10-26T01:31:36.869 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:36.864+0000 7efd6b55a640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd64070640 con 0x7efd641597c0 2023-10-26T01:31:36.869 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:36.864+0000 7efd692f6640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd640b0c80 0x7efd6406fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:36.869 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:36.864+0000 7efd692f6640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd640b0c80 0x7efd6406fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:51408/0 (socket says 172.21.15.169:51408) 2023-10-26T01:31:36.869 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:36.864+0000 7efd692f6640 1 -- 172.21.15.169:0/1045558384 learned_addr learned my addr 172.21.15.169:0/1045558384 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:31:36.869 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:36.865+0000 7efd692f6640 1 -- 172.21.15.169:0/1045558384 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7efd641597c0 msgr2=0x7efd64151fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:36.870 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:36.865+0000 7efd692f6640 1 --2- 172.21.15.169:0/1045558384 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7efd641597c0 0x7efd64151fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:36.870 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:36.865+0000 7efd692f6640 1 -- 172.21.15.169:0/1045558384 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd641593e0 msgr2=0x7efd64071400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:36.870 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:36.865+0000 7efd692f6640 1 --2- 172.21.15.169:0/1045558384 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd641593e0 0x7efd64071400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:36.870 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:36.865+0000 7efd692f6640 1 -- 172.21.15.169:0/1045558384 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd60009900 con 0x7efd640b0c80 2023-10-26T01:31:36.870 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:36.865+0000 7efd692f6640 1 --2- 172.21.15.169:0/1045558384 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd640b0c80 0x7efd6406fd80 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7efd60000f80 tx=0x7efd60003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:36.871 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:36.865+0000 7efd59ffb640 1 -- 172.21.15.169:0/1045558384 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd60004450 con 0x7efd640b0c80 2023-10-26T01:31:36.871 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:36.865+0000 7efd59ffb640 1 -- 172.21.15.169:0/1045558384 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7efd600046a0 con 0x7efd640b0c80 2023-10-26T01:31:36.871 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:36.865+0000 7efd59ffb640 1 -- 172.21.15.169:0/1045558384 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd6003f6a0 con 0x7efd640b0c80 2023-10-26T01:31:36.871 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:36.865+0000 7efd6b55a640 1 -- 172.21.15.169:0/1045558384 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd64070920 con 0x7efd640b0c80 2023-10-26T01:31:36.871 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:36.865+0000 7efd6b55a640 1 -- 172.21.15.169:0/1045558384 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd64151ba0 con 0x7efd640b0c80 2023-10-26T01:31:36.872 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:36.866+0000 7efd6b55a640 1 -- 172.21.15.169:0/1045558384 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd6404db10 con 0x7efd640b0c80 2023-10-26T01:31:36.872 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:36.867+0000 7efd59ffb640 1 -- 172.21.15.169:0/1045558384 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7efd6003b040 con 0x7efd640b0c80 2023-10-26T01:31:36.872 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:36.869+0000 7efd59ffb640 1 --2- 172.21.15.169:0/1045558384 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7efd4806ae40 0x7efd4806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:36.873 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:36.869+0000 7efd59ffb640 1 -- 172.21.15.169:0/1045558384 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7efd600ac610 con 0x7efd640b0c80 2023-10-26T01:31:36.873 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:36.869+0000 7efd68af5640 1 --2- 172.21.15.169:0/1045558384 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7efd4806ae40 0x7efd4806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:36.873 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:36.871+0000 7efd68af5640 1 --2- 172.21.15.169:0/1045558384 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7efd4806ae40 0x7efd4806d300 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7efd6414ec20 tx=0x7efd54006ba0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:36.876 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:36.873+0000 7efd59ffb640 1 -- 172.21.15.169:0/1045558384 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7efd600775a0 con 0x7efd640b0c80 2023-10-26T01:31:37.080 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:37.078+0000 7efd6b55a640 1 -- 172.21.15.169:0/1045558384 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7efd640b2170 con 0x7efd640b0c80 2023-10-26T01:31:37.081 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:37.079+0000 7efd59ffb640 1 -- 172.21.15.169:0/1045558384 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) v1 ==== 80+0+0 (secure 0 0 0) 0x7efd6007abc0 con 0x7efd640b0c80 2023-10-26T01:31:37.082 INFO:teuthology.orchestra.run.smithi169.stderr:listed 0 entries 2023-10-26T01:31:37.085 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:37.081+0000 7efd6b55a640 1 -- 172.21.15.169:0/1045558384 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7efd4806ae40 msgr2=0x7efd4806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:37.085 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:37.081+0000 7efd6b55a640 1 --2- 172.21.15.169:0/1045558384 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7efd4806ae40 0x7efd4806d300 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7efd6414ec20 tx=0x7efd54006ba0 comp rx=0 tx=0).stop 2023-10-26T01:31:37.085 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:37.081+0000 7efd6b55a640 1 -- 172.21.15.169:0/1045558384 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd640b0c80 msgr2=0x7efd6406fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:37.085 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:37.081+0000 7efd6b55a640 1 --2- 172.21.15.169:0/1045558384 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd640b0c80 0x7efd6406fd80 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7efd60000f80 tx=0x7efd60003c20 comp rx=0 tx=0).stop 2023-10-26T01:31:37.085 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:37.081+0000 7efd6b55a640 1 -- 172.21.15.169:0/1045558384 shutdown_connections 2023-10-26T01:31:37.086 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:37.081+0000 7efd6b55a640 1 --2- 172.21.15.169:0/1045558384 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7efd4806ae40 0x7efd4806d300 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:37.086 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:37.081+0000 7efd6b55a640 1 --2- 172.21.15.169:0/1045558384 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7efd641597c0 0x7efd64151fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:37.086 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:37.081+0000 7efd6b55a640 1 --2- 172.21.15.169:0/1045558384 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd641593e0 0x7efd64071400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:37.086 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:37.081+0000 7efd6b55a640 1 --2- 172.21.15.169:0/1045558384 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd640b0c80 0x7efd6406fd80 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:37.086 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:37.082+0000 7efd6b55a640 1 -- 172.21.15.169:0/1045558384 >> 172.21.15.169:0/1045558384 conn(0x7efd64069930 msgr2=0x7efd6415c480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:37.087 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:37.082+0000 7efd6b55a640 1 -- 172.21.15.169:0/1045558384 shutdown_connections 2023-10-26T01:31:37.087 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:37.082+0000 7efd6b55a640 1 -- 172.21.15.169:0/1045558384 wait complete. 2023-10-26T01:31:37.104 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2023-10-26T01:31:37.105 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:31:37.105 DEBUG:teuthology.orchestra.run.smithi169:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2023-10-26T01:31:37.192 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi169.front.sepia.ceph.com 2023-10-26T01:31:37.193 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:31:37.193 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-26T01:31:37.239 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-26T01:31:37.239 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:31:37.302 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-26T01:31:37.302 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:31:37.356 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-26T01:31:37.356 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:31:37.411 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:31:37.626 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:37.622+0000 7f8b46256640 1 -- 172.21.15.169:0/3087387278 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b4006ce20 msgr2=0x7f8b4014e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:37.626 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:37.622+0000 7f8b46256640 1 --2- 172.21.15.169:0/3087387278 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b4006ce20 0x7f8b4014e790 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8b40071fc0 tx=0x7f8b3002bc80 comp rx=0 tx=0).stop 2023-10-26T01:31:37.626 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:37.623+0000 7f8b46256640 1 -- 172.21.15.169:0/3087387278 shutdown_connections 2023-10-26T01:31:37.626 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:37.623+0000 7f8b46256640 1 --2- 172.21.15.169:0/3087387278 >> v2:172.21.15.170:3300/0 conn(0x7f8b4006ad30 0x7f8b4006b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:37.627 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:37.623+0000 7f8b46256640 1 --2- 172.21.15.169:0/3087387278 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f8b40158850 0x7f8b400aab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:37.627 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:37.623+0000 7f8b46256640 1 --2- 172.21.15.169:0/3087387278 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b4006ce20 0x7f8b4014e790 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:37.627 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:37.623+0000 7f8b46256640 1 -- 172.21.15.169:0/3087387278 >> 172.21.15.169:0/3087387278 conn(0x7f8b40069930 msgr2=0x7f8b40069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:37.627 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:37.623+0000 7f8b46256640 1 -- 172.21.15.169:0/3087387278 shutdown_connections 2023-10-26T01:31:37.627 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:37.623+0000 7f8b46256640 1 -- 172.21.15.169:0/3087387278 wait complete. 2023-10-26T01:31:37.629 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:37.624+0000 7f8b46256640 1 Processor -- start 2023-10-26T01:31:37.630 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:37.625+0000 7f8b46256640 1 -- start start 2023-10-26T01:31:37.630 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:37.625+0000 7f8b46256640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b4006ad30 0x7f8b4014ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:37.630 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:37.625+0000 7f8b46256640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b4006ce20 0x7f8b40149680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:37.630 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:37.625+0000 7f8b46256640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f8b40158850 0x7f8b400c82e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:37.631 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:37.625+0000 7f8b46256640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b400ca270 con 0x7f8b4006ad30 2023-10-26T01:31:37.631 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:37.625+0000 7f8b46256640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b400cbf60 con 0x7f8b4006ce20 2023-10-26T01:31:37.631 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:37.625+0000 7f8b46256640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b400ca4e0 con 0x7f8b40158850 2023-10-26T01:31:37.631 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:37.626+0000 7f8b3f7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b4006ad30 0x7f8b4014ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:37.631 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:37.626+0000 7f8b3f7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b4006ad30 0x7f8b4014ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:51440/0 (socket says 172.21.15.169:51440) 2023-10-26T01:31:37.632 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:37.626+0000 7f8b3f7fe640 1 -- 172.21.15.169:0/3580329101 learned_addr learned my addr 172.21.15.169:0/3580329101 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:31:37.632 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:37.626+0000 7f8b37fff640 1 --2- 172.21.15.169:0/3580329101 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b4006ce20 0x7f8b40149680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:37.632 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:37.626+0000 7f8b3ffff640 1 --2- 172.21.15.169:0/3580329101 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f8b40158850 0x7f8b400c82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:37.632 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:37.626+0000 7f8b3f7fe640 1 -- 172.21.15.169:0/3580329101 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f8b40158850 msgr2=0x7f8b400c82e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:37.632 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:37.626+0000 7f8b3f7fe640 1 --2- 172.21.15.169:0/3580329101 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f8b40158850 0x7f8b400c82e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:37.633 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:37.626+0000 7f8b3f7fe640 1 -- 172.21.15.169:0/3580329101 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b4006ce20 msgr2=0x7f8b40149680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:37.633 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:37.626+0000 7f8b3f7fe640 1 --2- 172.21.15.169:0/3580329101 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b4006ce20 0x7f8b40149680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:37.633 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:37.626+0000 7f8b3f7fe640 1 -- 172.21.15.169:0/3580329101 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b30009560 con 0x7f8b4006ad30 2023-10-26T01:31:37.634 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:37.626+0000 7f8b3ffff640 1 --2- 172.21.15.169:0/3580329101 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f8b40158850 0x7f8b400c82e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:31:37.634 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:37.626+0000 7f8b37fff640 1 --2- 172.21.15.169:0/3580329101 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b4006ce20 0x7f8b40149680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:31:37.634 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:37.626+0000 7f8b3f7fe640 1 --2- 172.21.15.169:0/3580329101 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b4006ad30 0x7f8b4014ed60 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8b30000c00 tx=0x7f8b3002dc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:37.635 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:37.627+0000 7f8b3cff9640 1 -- 172.21.15.169:0/3580329101 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b30003800 con 0x7f8b4006ad30 2023-10-26T01:31:37.635 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:37.627+0000 7f8b3cff9640 1 -- 172.21.15.169:0/3580329101 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8b30003a50 con 0x7f8b4006ad30 2023-10-26T01:31:37.635 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:37.627+0000 7f8b3cff9640 1 -- 172.21.15.169:0/3580329101 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b30003de0 con 0x7f8b4006ad30 2023-10-26T01:31:37.635 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:37.627+0000 7f8b46256640 1 -- 172.21.15.169:0/3580329101 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b400ca7c0 con 0x7f8b4006ad30 2023-10-26T01:31:37.636 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:37.627+0000 7f8b46256640 1 -- 172.21.15.169:0/3580329101 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b40149bc0 con 0x7f8b4006ad30 2023-10-26T01:31:37.637 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:37.628+0000 7f8b46256640 1 -- 172.21.15.169:0/3580329101 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b400ca0d0 con 0x7f8b4006ad30 2023-10-26T01:31:37.637 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:37.629+0000 7f8b3cff9640 1 -- 172.21.15.169:0/3580329101 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f8b3003e450 con 0x7f8b4006ad30 2023-10-26T01:31:37.637 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:37.630+0000 7f8b3cff9640 1 --2- 172.21.15.169:0/3580329101 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f8b2006ae40 0x7f8b2006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:37.637 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:37.630+0000 7f8b3cff9640 1 -- 172.21.15.169:0/3580329101 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f8b3002de40 con 0x7f8b4006ad30 2023-10-26T01:31:37.638 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:37.631+0000 7f8b37fff640 1 --2- 172.21.15.169:0/3580329101 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f8b2006ae40 0x7f8b2006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:37.638 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:37.632+0000 7f8b37fff640 1 --2- 172.21.15.169:0/3580329101 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f8b2006ae40 0x7f8b2006d300 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f8b401581b0 tx=0x7f8b28008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:37.638 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:37.634+0000 7f8b3cff9640 1 -- 172.21.15.169:0/3580329101 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8b300deca0 con 0x7f8b4006ad30 2023-10-26T01:31:37.827 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:37.824+0000 7f8b46256640 1 -- 172.21.15.169:0/3580329101 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8b4004db10 con 0x7f8b4006ad30 2023-10-26T01:31:37.828 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:37.825+0000 7f8b3cff9640 1 -- 172.21.15.169:0/3580329101 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16801 (secure 0 0 0) 0x7f8b30042090 con 0x7f8b4006ad30 2023-10-26T01:31:37.829 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:31:37.830 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":23,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:31:30.100039+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-26T01:30:39.437631+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-26T01:30:40.561774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":2}]},{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.24","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:31:37.831 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:37.828+0000 7f8b46256640 1 -- 172.21.15.169:0/3580329101 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f8b2006ae40 msgr2=0x7f8b2006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:37.831 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:37.828+0000 7f8b46256640 1 --2- 172.21.15.169:0/3580329101 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f8b2006ae40 0x7f8b2006d300 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f8b401581b0 tx=0x7f8b28008040 comp rx=0 tx=0).stop 2023-10-26T01:31:37.832 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:37.828+0000 7f8b46256640 1 -- 172.21.15.169:0/3580329101 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b4006ad30 msgr2=0x7f8b4014ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:37.832 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:37.828+0000 7f8b46256640 1 --2- 172.21.15.169:0/3580329101 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b4006ad30 0x7f8b4014ed60 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8b30000c00 tx=0x7f8b3002dc50 comp rx=0 tx=0).stop 2023-10-26T01:31:37.832 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:37.828+0000 7f8b46256640 1 -- 172.21.15.169:0/3580329101 shutdown_connections 2023-10-26T01:31:37.832 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:37.828+0000 7f8b46256640 1 --2- 172.21.15.169:0/3580329101 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f8b2006ae40 0x7f8b2006d300 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:37.833 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:37.828+0000 7f8b46256640 1 --2- 172.21.15.169:0/3580329101 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f8b40158850 0x7f8b400c82e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:37.833 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:37.828+0000 7f8b46256640 1 --2- 172.21.15.169:0/3580329101 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b4006ce20 0x7f8b40149680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:37.833 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:37.828+0000 7f8b46256640 1 --2- 172.21.15.169:0/3580329101 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b4006ad30 0x7f8b4014ed60 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:37.834 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:37.828+0000 7f8b46256640 1 -- 172.21.15.169:0/3580329101 >> 172.21.15.169:0/3580329101 conn(0x7f8b40069930 msgr2=0x7f8b4014d710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:37.834 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:37.828+0000 7f8b46256640 1 -- 172.21.15.169:0/3580329101 shutdown_connections 2023-10-26T01:31:37.834 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:37.828+0000 7f8b46256640 1 -- 172.21.15.169:0/3580329101 wait complete. 2023-10-26T01:31:37.850 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T01:30:32.200808+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-26T01:30:39.437631+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-26T01:30:40.561774+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T01:31:37.851 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T01:31:38.058 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.055+0000 7fe8bead7640 1 -- 172.21.15.169:0/648835438 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8b80aaf80 msgr2=0x7fe8b80ab3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:38.058 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.055+0000 7fe8bead7640 1 --2- 172.21.15.169:0/648835438 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8b80aaf80 0x7fe8b80ab3c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fe8b80ce2c0 tx=0x7fe8a402bce0 comp rx=0 tx=0).stop 2023-10-26T01:31:38.059 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.055+0000 7fe8bead7640 1 -- 172.21.15.169:0/648835438 shutdown_connections 2023-10-26T01:31:38.059 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.055+0000 7fe8bead7640 1 --2- 172.21.15.169:0/648835438 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe8b80cd690 0x7fe8b80cdb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:38.059 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.055+0000 7fe8bead7640 1 --2- 172.21.15.169:0/648835438 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8b80aaf80 0x7fe8b80ab3c0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:38.059 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.055+0000 7fe8bead7640 1 -- 172.21.15.169:0/648835438 >> 172.21.15.169:0/648835438 conn(0x7fe8b8069930 msgr2=0x7fe8b8069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:38.059 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.055+0000 7fe8bead7640 1 -- 172.21.15.169:0/648835438 shutdown_connections 2023-10-26T01:31:38.060 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.055+0000 7fe8bead7640 1 -- 172.21.15.169:0/648835438 wait complete. 2023-10-26T01:31:38.061 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.057+0000 7fe8bead7640 1 Processor -- start 2023-10-26T01:31:38.062 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.057+0000 7fe8bead7640 1 -- start start 2023-10-26T01:31:38.062 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.058+0000 7fe8bead7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe8b80aaf80 0x7fe8b806c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:38.062 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.058+0000 7fe8bead7640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8b80cd690 0x7fe8b806b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:38.062 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.058+0000 7fe8bead7640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe8b8159480 0x7fe8b8154ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:38.063 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.058+0000 7fe8bead7640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b8151630 con 0x7fe8b80cd690 2023-10-26T01:31:38.063 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.058+0000 7fe8bead7640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b8142910 con 0x7fe8b80aaf80 2023-10-26T01:31:38.063 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.058+0000 7fe8bead7640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b81517c0 con 0x7fe8b8159480 2023-10-26T01:31:38.063 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.058+0000 7fe8b7fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8b80cd690 0x7fe8b806b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:38.063 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.058+0000 7fe8b7fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8b80cd690 0x7fe8b806b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:51464/0 (socket says 172.21.15.169:51464) 2023-10-26T01:31:38.064 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.058+0000 7fe8b7fff640 1 -- 172.21.15.169:0/2522302481 learned_addr learned my addr 172.21.15.169:0/2522302481 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:31:38.064 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.058+0000 7fe8b7fff640 1 -- 172.21.15.169:0/2522302481 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe8b8159480 msgr2=0x7fe8b8154ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:38.064 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.058+0000 7fe8b7fff640 1 --2- 172.21.15.169:0/2522302481 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe8b8159480 0x7fe8b8154ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:38.064 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.058+0000 7fe8b7fff640 1 -- 172.21.15.169:0/2522302481 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe8b80aaf80 msgr2=0x7fe8b806c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:38.064 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.058+0000 7fe8bc873640 1 --2- 172.21.15.169:0/2522302481 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe8b80aaf80 0x7fe8b806c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:38.065 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.059+0000 7fe8b7fff640 1 --2- 172.21.15.169:0/2522302481 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe8b80aaf80 0x7fe8b806c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:38.065 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.059+0000 7fe8b7fff640 1 -- 172.21.15.169:0/2522302481 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8a4009560 con 0x7fe8b80cd690 2023-10-26T01:31:38.065 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.059+0000 7fe8b7fff640 1 --2- 172.21.15.169:0/2522302481 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8b80cd690 0x7fe8b806b6e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fe8a40099c0 tx=0x7fe8a402dcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:38.065 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.059+0000 7fe8b57fa640 1 -- 172.21.15.169:0/2522302481 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a402dea0 con 0x7fe8b80cd690 2023-10-26T01:31:38.065 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.059+0000 7fe8b57fa640 1 -- 172.21.15.169:0/2522302481 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe8a4003800 con 0x7fe8b80cd690 2023-10-26T01:31:38.066 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.059+0000 7fe8bead7640 1 -- 172.21.15.169:0/2522302481 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8b806bdb0 con 0x7fe8b80cd690 2023-10-26T01:31:38.066 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.059+0000 7fe8b57fa640 1 -- 172.21.15.169:0/2522302481 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a4003c10 con 0x7fe8b80cd690 2023-10-26T01:31:38.066 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.059+0000 7fe8bead7640 1 -- 172.21.15.169:0/2522302481 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8b80c8df0 con 0x7fe8b80cd690 2023-10-26T01:31:38.067 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.060+0000 7fe8bead7640 1 -- 172.21.15.169:0/2522302481 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8b8066af0 con 0x7fe8b80cd690 2023-10-26T01:31:38.067 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.061+0000 7fe8b57fa640 1 -- 172.21.15.169:0/2522302481 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fe8a4003d70 con 0x7fe8b80cd690 2023-10-26T01:31:38.067 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.063+0000 7fe8b57fa640 1 --2- 172.21.15.169:0/2522302481 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe89c06ae70 0x7fe89c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:38.068 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.063+0000 7fe8b57fa640 1 -- 172.21.15.169:0/2522302481 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fe8a40abfb0 con 0x7fe8b80cd690 2023-10-26T01:31:38.068 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.063+0000 7fe8bc873640 1 --2- 172.21.15.169:0/2522302481 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe89c06ae70 0x7fe89c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:38.068 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.065+0000 7fe8bc873640 1 --2- 172.21.15.169:0/2522302481 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe89c06ae70 0x7fe89c06d330 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac00a730 tx=0x7fe8ac008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:38.070 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.068+0000 7fe8b57fa640 1 -- 172.21.15.169:0/2522302481 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe8a40deca0 con 0x7fe8b80cd690 2023-10-26T01:31:38.267 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.264+0000 7fe8bead7640 1 -- 172.21.15.169:0/2522302481 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe8b804da70 con 0x7fe8b80cd690 2023-10-26T01:31:38.267 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.265+0000 7fe8b57fa640 1 -- 172.21.15.169:0/2522302481 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe8a40420a0 con 0x7fe8b80cd690 2023-10-26T01:31:38.268 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 1 2023-10-26T01:31:38.271 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.268+0000 7fe8bead7640 1 -- 172.21.15.169:0/2522302481 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe89c06ae70 msgr2=0x7fe89c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:38.271 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.268+0000 7fe8bead7640 1 --2- 172.21.15.169:0/2522302481 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe89c06ae70 0x7fe89c06d330 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac00a730 tx=0x7fe8ac008040 comp rx=0 tx=0).stop 2023-10-26T01:31:38.271 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.268+0000 7fe8bead7640 1 -- 172.21.15.169:0/2522302481 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8b80cd690 msgr2=0x7fe8b806b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:38.271 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.268+0000 7fe8bead7640 1 --2- 172.21.15.169:0/2522302481 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8b80cd690 0x7fe8b806b6e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fe8a40099c0 tx=0x7fe8a402dcb0 comp rx=0 tx=0).stop 2023-10-26T01:31:38.272 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.268+0000 7fe8bead7640 1 -- 172.21.15.169:0/2522302481 shutdown_connections 2023-10-26T01:31:38.272 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.268+0000 7fe8bead7640 1 --2- 172.21.15.169:0/2522302481 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe89c06ae70 0x7fe89c06d330 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:38.272 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.268+0000 7fe8bead7640 1 --2- 172.21.15.169:0/2522302481 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8b80cd690 0x7fe8b806b6e0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:38.272 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.268+0000 7fe8bead7640 1 --2- 172.21.15.169:0/2522302481 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe8b8159480 0x7fe8b8154ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:38.272 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.268+0000 7fe8bead7640 1 --2- 172.21.15.169:0/2522302481 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe8b80aaf80 0x7fe8b806c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:38.273 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.268+0000 7fe8bead7640 1 -- 172.21.15.169:0/2522302481 >> 172.21.15.169:0/2522302481 conn(0x7fe8b8069930 msgr2=0x7fe8b8069d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:38.273 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.268+0000 7fe8bead7640 1 -- 172.21.15.169:0/2522302481 shutdown_connections 2023-10-26T01:31:38.273 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.268+0000 7fe8bead7640 1 -- 172.21.15.169:0/2522302481 wait complete. 2023-10-26T01:31:38.288 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-26T01:31:38.496 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.493+0000 7f989e4ca640 1 -- 172.21.15.169:0/1086550619 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9898069ff0 msgr2=0x7f9898158f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:38.497 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.493+0000 7f989e4ca640 1 --2- 172.21.15.169:0/1086550619 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9898069ff0 0x7f9898158f30 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f989806ce30 tx=0x7f988802bdb0 comp rx=0 tx=0).stop 2023-10-26T01:31:38.497 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.493+0000 7f989e4ca640 1 -- 172.21.15.169:0/1086550619 shutdown_connections 2023-10-26T01:31:38.497 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.493+0000 7f989e4ca640 1 --2- 172.21.15.169:0/1086550619 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9898069ff0 0x7f9898158f30 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:38.497 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.493+0000 7f989e4ca640 1 --2- 172.21.15.169:0/1086550619 >> v2:172.21.15.170:3300/0 conn(0x7f9898069c10 0x7f98980b0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:38.498 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.493+0000 7f989e4ca640 1 -- 172.21.15.169:0/1086550619 >> 172.21.15.169:0/1086550619 conn(0x7f98980af110 msgr2=0x7f98980af520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:38.498 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.493+0000 7f989e4ca640 1 -- 172.21.15.169:0/1086550619 shutdown_connections 2023-10-26T01:31:38.498 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.493+0000 7f989e4ca640 1 -- 172.21.15.169:0/1086550619 wait complete. 2023-10-26T01:31:38.500 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.495+0000 7f989e4ca640 1 Processor -- start 2023-10-26T01:31:38.500 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.495+0000 7f989e4ca640 1 -- start start 2023-10-26T01:31:38.501 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.496+0000 7f989e4ca640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9898069c10 0x7f989814e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:38.501 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.496+0000 7f989e4ca640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9898069ff0 0x7f989806c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:38.501 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.496+0000 7f989e4ca640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9898159470 0x7f98981507a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:38.501 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.496+0000 7f989e4ca640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9898150070 con 0x7f9898159470 2023-10-26T01:31:38.501 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.497+0000 7f989e4ca640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f989806b4d0 con 0x7f9898069c10 2023-10-26T01:31:38.502 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.497+0000 7f989e4ca640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f989806bff0 con 0x7f9898069ff0 2023-10-26T01:31:38.502 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.497+0000 7f989dcc9640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9898159470 0x7f98981507a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:38.502 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.497+0000 7f989dcc9640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9898159470 0x7f98981507a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:51484/0 (socket says 172.21.15.169:51484) 2023-10-26T01:31:38.502 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.497+0000 7f989dcc9640 1 -- 172.21.15.169:0/288559748 learned_addr learned my addr 172.21.15.169:0/288559748 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:31:38.503 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.497+0000 7f989d4c8640 1 --2- 172.21.15.169:0/288559748 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9898069c10 0x7f989814e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:38.503 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.497+0000 7f989dcc9640 1 -- 172.21.15.169:0/288559748 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9898069ff0 msgr2=0x7f989806c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:38.503 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.497+0000 7f989dcc9640 1 --2- 172.21.15.169:0/288559748 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9898069ff0 0x7f989806c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:38.503 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.497+0000 7f989dcc9640 1 -- 172.21.15.169:0/288559748 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9898069c10 msgr2=0x7f989814e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:38.504 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.497+0000 7f989dcc9640 1 --2- 172.21.15.169:0/288559748 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9898069c10 0x7f989814e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:38.504 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.497+0000 7f989dcc9640 1 -- 172.21.15.169:0/288559748 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9888009560 con 0x7f9898159470 2023-10-26T01:31:38.504 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.497+0000 7f989d4c8640 1 --2- 172.21.15.169:0/288559748 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9898069c10 0x7f989814e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:31:38.504 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.497+0000 7f989dcc9640 1 --2- 172.21.15.169:0/288559748 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9898159470 0x7f98981507a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f989000b9a0 tx=0x7f989000be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:38.504 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.498+0000 7f988dffb640 1 -- 172.21.15.169:0/288559748 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f989000cba0 con 0x7f9898159470 2023-10-26T01:31:38.505 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.498+0000 7f988dffb640 1 -- 172.21.15.169:0/288559748 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f989000cdf0 con 0x7f9898159470 2023-10-26T01:31:38.505 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.498+0000 7f988dffb640 1 -- 172.21.15.169:0/288559748 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98900146a0 con 0x7f9898159470 2023-10-26T01:31:38.505 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.498+0000 7f989e4ca640 1 -- 172.21.15.169:0/288559748 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9898152b70 con 0x7f9898159470 2023-10-26T01:31:38.505 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.498+0000 7f989e4ca640 1 -- 172.21.15.169:0/288559748 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f989806b9a0 con 0x7f9898159470 2023-10-26T01:31:38.506 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.499+0000 7f989e4ca640 1 -- 172.21.15.169:0/288559748 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9898073c10 con 0x7f9898159470 2023-10-26T01:31:38.506 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.500+0000 7f988dffb640 1 -- 172.21.15.169:0/288559748 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f989001e440 con 0x7f9898159470 2023-10-26T01:31:38.507 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.501+0000 7f988dffb640 1 --2- 172.21.15.169:0/288559748 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f987c06ae70 0x7f987c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:38.507 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.501+0000 7f988dffb640 1 -- 172.21.15.169:0/288559748 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f9890089830 con 0x7f9898159470 2023-10-26T01:31:38.507 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.502+0000 7f989d4c8640 1 --2- 172.21.15.169:0/288559748 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f987c06ae70 0x7f987c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:38.507 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.503+0000 7f989d4c8640 1 --2- 172.21.15.169:0/288559748 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f987c06ae70 0x7f987c06d330 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f98940058c0 tx=0x7f9894005d30 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:38.508 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.505+0000 7f988dffb640 1 -- 172.21.15.169:0/288559748 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f98900547c0 con 0x7f9898159470 2023-10-26T01:31:38.696 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.693+0000 7f989e4ca640 1 -- 172.21.15.169:0/288559748 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f989806c130 con 0x7f9898159470 2023-10-26T01:31:38.696 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.694+0000 7f988dffb640 1 -- 172.21.15.169:0/288559748 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9890057de0 con 0x7f9898159470 2023-10-26T01:31:38.697 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 32 2023-10-26T01:31:38.699 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.697+0000 7f989e4ca640 1 -- 172.21.15.169:0/288559748 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f987c06ae70 msgr2=0x7f987c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:38.700 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.697+0000 7f989e4ca640 1 --2- 172.21.15.169:0/288559748 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f987c06ae70 0x7f987c06d330 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f98940058c0 tx=0x7f9894005d30 comp rx=0 tx=0).stop 2023-10-26T01:31:38.700 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.697+0000 7f989e4ca640 1 -- 172.21.15.169:0/288559748 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9898159470 msgr2=0x7f98981507a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:38.700 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.697+0000 7f989e4ca640 1 --2- 172.21.15.169:0/288559748 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9898159470 0x7f98981507a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f989000b9a0 tx=0x7f989000be60 comp rx=0 tx=0).stop 2023-10-26T01:31:38.700 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.697+0000 7f989e4ca640 1 -- 172.21.15.169:0/288559748 shutdown_connections 2023-10-26T01:31:38.701 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.697+0000 7f989e4ca640 1 --2- 172.21.15.169:0/288559748 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f987c06ae70 0x7f987c06d330 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:38.701 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.697+0000 7f989e4ca640 1 --2- 172.21.15.169:0/288559748 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9898159470 0x7f98981507a0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:38.701 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.697+0000 7f989e4ca640 1 --2- 172.21.15.169:0/288559748 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9898069ff0 0x7f989806c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:38.701 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.697+0000 7f989e4ca640 1 --2- 172.21.15.169:0/288559748 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9898069c10 0x7f989814e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:38.701 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.697+0000 7f989e4ca640 1 -- 172.21.15.169:0/288559748 >> 172.21.15.169:0/288559748 conn(0x7f98980af110 msgr2=0x7f989814f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:38.702 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.697+0000 7f989e4ca640 1 -- 172.21.15.169:0/288559748 shutdown_connections 2023-10-26T01:31:38.702 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.697+0000 7f989e4ca640 1 -- 172.21.15.169:0/288559748 wait complete. 2023-10-26T01:31:38.719 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-26T01:31:38.930 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.927+0000 7f16a5b55640 1 -- 172.21.15.169:0/3987696334 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16a00abdb0 msgr2=0x7f16a006ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:38.930 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.927+0000 7f16a5b55640 1 --2- 172.21.15.169:0/3987696334 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16a00abdb0 0x7f16a006ad20 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f16a006e2c0 tx=0x7f169802bd10 comp rx=0 tx=0).stop 2023-10-26T01:31:38.930 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.927+0000 7f16a5b55640 1 -- 172.21.15.169:0/3987696334 shutdown_connections 2023-10-26T01:31:38.931 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.927+0000 7f16a5b55640 1 --2- 172.21.15.169:0/3987696334 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16a00abdb0 0x7f16a006ad20 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:38.931 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.927+0000 7f16a5b55640 1 --2- 172.21.15.169:0/3987696334 >> v2:172.21.15.170:3300/0 conn(0x7f16a0159440 0x7f16a00cc990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:38.931 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.927+0000 7f16a5b55640 1 -- 172.21.15.169:0/3987696334 >> 172.21.15.169:0/3987696334 conn(0x7f16a00aefe0 msgr2=0x7f16a00af3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:38.931 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.927+0000 7f16a5b55640 1 -- 172.21.15.169:0/3987696334 shutdown_connections 2023-10-26T01:31:38.931 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.927+0000 7f16a5b55640 1 -- 172.21.15.169:0/3987696334 wait complete. 2023-10-26T01:31:38.933 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.928+0000 7f16a5b55640 1 Processor -- start 2023-10-26T01:31:38.934 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.929+0000 7f16a5b55640 1 -- start start 2023-10-26T01:31:38.934 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.929+0000 7f16a5b55640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16a00abdb0 0x7f16a00c6660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:38.934 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.930+0000 7f16a5b55640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f16a0159440 0x7f16a00c8610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:38.934 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.930+0000 7f16a5b55640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f16a0141e30 0x7f16a00c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:38.935 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.930+0000 7f16a5b55640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16a00c9090 con 0x7f16a00abdb0 2023-10-26T01:31:38.935 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.930+0000 7f16a5b55640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16a00b04f0 con 0x7f16a0159440 2023-10-26T01:31:38.935 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.930+0000 7f16a5b55640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16a00b07d0 con 0x7f16a0141e30 2023-10-26T01:31:38.935 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.930+0000 7f169f7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16a00abdb0 0x7f16a00c6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:38.935 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.930+0000 7f169f7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16a00abdb0 0x7f16a00c6660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:51502/0 (socket says 172.21.15.169:51502) 2023-10-26T01:31:38.936 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.930+0000 7f169f7fe640 1 -- 172.21.15.169:0/2053776681 learned_addr learned my addr 172.21.15.169:0/2053776681 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:31:38.936 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.930+0000 7f169f7fe640 1 -- 172.21.15.169:0/2053776681 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f16a0141e30 msgr2=0x7f16a00c8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:38.936 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.930+0000 7f169f7fe640 1 --2- 172.21.15.169:0/2053776681 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f16a0141e30 0x7f16a00c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:38.936 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.930+0000 7f169f7fe640 1 -- 172.21.15.169:0/2053776681 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f16a0159440 msgr2=0x7f16a00c8610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:38.936 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.930+0000 7f169f7fe640 1 --2- 172.21.15.169:0/2053776681 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f16a0159440 0x7f16a00c8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:38.937 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.930+0000 7f169f7fe640 1 -- 172.21.15.169:0/2053776681 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1698009560 con 0x7f16a00abdb0 2023-10-26T01:31:38.937 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.930+0000 7f169f7fe640 1 --2- 172.21.15.169:0/2053776681 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16a00abdb0 0x7f16a00c6660 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f169000cce0 tx=0x7f169000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:38.937 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.931+0000 7f1697fff640 1 -- 172.21.15.169:0/2053776681 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1690004b20 con 0x7f16a00abdb0 2023-10-26T01:31:38.937 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.931+0000 7f1697fff640 1 -- 172.21.15.169:0/2053776681 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1690004d70 con 0x7f16a00abdb0 2023-10-26T01:31:38.937 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.931+0000 7f16a5b55640 1 -- 172.21.15.169:0/2053776681 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16a00b0ab0 con 0x7f16a00abdb0 2023-10-26T01:31:38.938 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.931+0000 7f1697fff640 1 -- 172.21.15.169:0/2053776681 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1690005280 con 0x7f16a00abdb0 2023-10-26T01:31:38.938 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.931+0000 7f16a5b55640 1 -- 172.21.15.169:0/2053776681 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16a00c7120 con 0x7f16a00abdb0 2023-10-26T01:31:38.939 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.932+0000 7f16a5b55640 1 -- 172.21.15.169:0/2053776681 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16a0066af0 con 0x7f16a00abdb0 2023-10-26T01:31:38.939 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.933+0000 7f1697fff640 1 -- 172.21.15.169:0/2053776681 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f16900053e0 con 0x7f16a00abdb0 2023-10-26T01:31:38.939 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.934+0000 7f1697fff640 1 --2- 172.21.15.169:0/2053776681 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f168006ae40 0x7f168006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:38.939 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.934+0000 7f1697fff640 1 -- 172.21.15.169:0/2053776681 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f1690089f80 con 0x7f16a00abdb0 2023-10-26T01:31:38.940 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.934+0000 7f169effd640 1 --2- 172.21.15.169:0/2053776681 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f168006ae40 0x7f168006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:38.940 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.936+0000 7f169effd640 1 --2- 172.21.15.169:0/2053776681 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f168006ae40 0x7f168006d300 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f16a015a440 tx=0x7f1688006ba0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:38.940 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:38.938+0000 7f1697fff640 1 -- 172.21.15.169:0/2053776681 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1690054fa0 con 0x7f16a00abdb0 2023-10-26T01:31:39.126 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:39.124+0000 7f16a5b55640 1 -- 172.21.15.169:0/2053776681 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f16a00b0fd0 con 0x7f16a00abdb0 2023-10-26T01:31:39.127 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:39.125+0000 7f1697fff640 1 -- 172.21.15.169:0/2053776681 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f16a00b0fd0 con 0x7f16a00abdb0 2023-10-26T01:31:39.128 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 64 2023-10-26T01:31:39.131 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:39.127+0000 7f16a5b55640 1 -- 172.21.15.169:0/2053776681 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f168006ae40 msgr2=0x7f168006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:39.131 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:39.127+0000 7f16a5b55640 1 --2- 172.21.15.169:0/2053776681 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f168006ae40 0x7f168006d300 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f16a015a440 tx=0x7f1688006ba0 comp rx=0 tx=0).stop 2023-10-26T01:31:39.131 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:39.127+0000 7f16a5b55640 1 -- 172.21.15.169:0/2053776681 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16a00abdb0 msgr2=0x7f16a00c6660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:39.131 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:39.127+0000 7f16a5b55640 1 --2- 172.21.15.169:0/2053776681 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16a00abdb0 0x7f16a00c6660 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f169000cce0 tx=0x7f169000a620 comp rx=0 tx=0).stop 2023-10-26T01:31:39.131 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:39.128+0000 7f16a5b55640 1 -- 172.21.15.169:0/2053776681 shutdown_connections 2023-10-26T01:31:39.132 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:39.128+0000 7f16a5b55640 1 --2- 172.21.15.169:0/2053776681 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f168006ae40 0x7f168006d300 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:39.132 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:39.128+0000 7f16a5b55640 1 --2- 172.21.15.169:0/2053776681 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f16a0141e30 0x7f16a00c8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:39.132 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:39.128+0000 7f16a5b55640 1 --2- 172.21.15.169:0/2053776681 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f16a0159440 0x7f16a00c8610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:39.132 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:39.128+0000 7f16a5b55640 1 --2- 172.21.15.169:0/2053776681 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16a00abdb0 0x7f16a00c6660 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:39.132 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:39.128+0000 7f16a5b55640 1 -- 172.21.15.169:0/2053776681 >> 172.21.15.169:0/2053776681 conn(0x7f16a00aefe0 msgr2=0x7f16a00ad900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:39.133 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:39.128+0000 7f16a5b55640 1 -- 172.21.15.169:0/2053776681 shutdown_connections 2023-10-26T01:31:39.133 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:39.128+0000 7f16a5b55640 1 -- 172.21.15.169:0/2053776681 wait complete. 2023-10-26T01:31:39.148 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-26T01:31:39.358 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:39.355+0000 7f6e722c3640 1 -- 172.21.15.169:0/4069152484 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6e6c06cba0 msgr2=0x7f6e6c15b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:39.359 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:39.355+0000 7f6e722c3640 1 --2- 172.21.15.169:0/4069152484 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6e6c06cba0 0x7f6e6c15b140 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c1521a0 tx=0x7f6e6802c080 comp rx=0 tx=0).stop 2023-10-26T01:31:39.359 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:39.355+0000 7f6e722c3640 1 -- 172.21.15.169:0/4069152484 shutdown_connections 2023-10-26T01:31:39.359 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:39.355+0000 7f6e722c3640 1 --2- 172.21.15.169:0/4069152484 >> v2:172.21.15.170:3300/0 conn(0x7f6e6c0aab80 0x7f6e6c069a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:39.359 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:39.355+0000 7f6e722c3640 1 --2- 172.21.15.169:0/4069152484 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6e6c06cba0 0x7f6e6c15b140 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:39.359 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:39.355+0000 7f6e722c3640 1 -- 172.21.15.169:0/4069152484 >> 172.21.15.169:0/4069152484 conn(0x7f6e6c0af110 msgr2=0x7f6e6c0af520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:39.360 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:39.356+0000 7f6e722c3640 1 -- 172.21.15.169:0/4069152484 shutdown_connections 2023-10-26T01:31:39.360 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:39.356+0000 7f6e722c3640 1 -- 172.21.15.169:0/4069152484 wait complete. 2023-10-26T01:31:39.361 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:39.357+0000 7f6e722c3640 1 Processor -- start 2023-10-26T01:31:39.362 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:39.357+0000 7f6e722c3640 1 -- start start 2023-10-26T01:31:39.362 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:39.358+0000 7f6e722c3640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6e6c06cba0 0x7f6e6c1f2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:39.362 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:39.358+0000 7f6e722c3640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e6c0aa6d0 0x7f6e6c156de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:39.362 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:39.358+0000 7f6e722c3640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6e6c0aab80 0x7f6e6c1f3520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:39.363 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:39.358+0000 7f6e722c3640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e6c157320 con 0x7f6e6c0aab80 2023-10-26T01:31:39.363 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:39.358+0000 7f6e722c3640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e6c14bdc0 con 0x7f6e6c0aa6d0 2023-10-26T01:31:39.363 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:39.358+0000 7f6e722c3640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e6c1569b0 con 0x7f6e6c06cba0 2023-10-26T01:31:39.363 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:39.359+0000 7f6e71ac2640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6e6c0aab80 0x7f6e6c1f3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:39.363 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:39.359+0000 7f6e71ac2640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6e6c0aab80 0x7f6e6c1f3520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:51516/0 (socket says 172.21.15.169:51516) 2023-10-26T01:31:39.364 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:39.359+0000 7f6e71ac2640 1 -- 172.21.15.169:0/3674599004 learned_addr learned my addr 172.21.15.169:0/3674599004 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:31:39.364 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:39.359+0000 7f6e70ac0640 1 --2- 172.21.15.169:0/3674599004 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e6c0aa6d0 0x7f6e6c156de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:39.364 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:39.359+0000 7f6e712c1640 1 --2- 172.21.15.169:0/3674599004 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6e6c06cba0 0x7f6e6c1f2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:39.364 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:39.359+0000 7f6e71ac2640 1 -- 172.21.15.169:0/3674599004 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6e6c06cba0 msgr2=0x7f6e6c1f2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:39.365 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:39.359+0000 7f6e71ac2640 1 --2- 172.21.15.169:0/3674599004 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6e6c06cba0 0x7f6e6c1f2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:39.365 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:39.359+0000 7f6e71ac2640 1 -- 172.21.15.169:0/3674599004 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e6c0aa6d0 msgr2=0x7f6e6c156de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:39.365 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:39.359+0000 7f6e71ac2640 1 --2- 172.21.15.169:0/3674599004 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e6c0aa6d0 0x7f6e6c156de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:39.365 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:39.359+0000 7f6e71ac2640 1 -- 172.21.15.169:0/3674599004 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e68009840 con 0x7f6e6c0aab80 2023-10-26T01:31:39.367 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:39.359+0000 7f6e71ac2640 1 --2- 172.21.15.169:0/3674599004 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6e6c0aab80 0x7f6e6c1f3520 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f6e6400c9a0 tx=0x7f6e6400ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:39.367 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:39.360+0000 7f6e61ffb640 1 -- 172.21.15.169:0/3674599004 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e64007bb0 con 0x7f6e6c0aab80 2023-10-26T01:31:39.367 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:39.360+0000 7f6e61ffb640 1 -- 172.21.15.169:0/3674599004 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6e64007e00 con 0x7f6e6c0aab80 2023-10-26T01:31:39.367 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:39.360+0000 7f6e61ffb640 1 -- 172.21.15.169:0/3674599004 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e64005280 con 0x7f6e6c0aab80 2023-10-26T01:31:39.368 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:39.360+0000 7f6e722c3640 1 -- 172.21.15.169:0/3674599004 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e6c158270 con 0x7f6e6c0aab80 2023-10-26T01:31:39.368 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:39.360+0000 7f6e722c3640 1 -- 172.21.15.169:0/3674599004 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e6c1f2c90 con 0x7f6e6c0aab80 2023-10-26T01:31:39.368 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:39.361+0000 7f6e722c3640 1 -- 172.21.15.169:0/3674599004 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e6c158400 con 0x7f6e6c0aab80 2023-10-26T01:31:39.368 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:39.362+0000 7f6e61ffb640 1 -- 172.21.15.169:0/3674599004 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f6e640053e0 con 0x7f6e6c0aab80 2023-10-26T01:31:39.368 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:39.363+0000 7f6e61ffb640 1 --2- 172.21.15.169:0/3674599004 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f6e5006ae40 0x7f6e5006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:39.369 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:39.363+0000 7f6e61ffb640 1 -- 172.21.15.169:0/3674599004 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f6e64089bd0 con 0x7f6e6c0aab80 2023-10-26T01:31:39.369 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:39.363+0000 7f6e712c1640 1 --2- 172.21.15.169:0/3674599004 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f6e5006ae40 0x7f6e5006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:39.369 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:39.364+0000 7f6e712c1640 1 --2- 172.21.15.169:0/3674599004 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f6e5006ae40 0x7f6e5006d300 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f6e68035000 tx=0x7f6e68004ab0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:39.370 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:39.367+0000 7f6e61ffb640 1 -- 172.21.15.169:0/3674599004 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6e64054bf0 con 0x7f6e6c0aab80 2023-10-26T01:31:39.571 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:39.568+0000 7f6e722c3640 1 -- 172.21.15.169:0/3674599004 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f6e6c073c10 con 0x7f6e6c0aab80 2023-10-26T01:31:39.572 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:39.569+0000 7f6e61ffb640 1 -- 172.21.15.169:0/3674599004 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) v1 ==== 80+0+0 (secure 0 0 0) 0x7f6e6401d070 con 0x7f6e6c0aab80 2023-10-26T01:31:39.572 INFO:teuthology.orchestra.run.smithi169.stderr:listed 0 entries 2023-10-26T01:31:39.574 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:39.572+0000 7f6e722c3640 1 -- 172.21.15.169:0/3674599004 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f6e5006ae40 msgr2=0x7f6e5006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:39.575 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:39.572+0000 7f6e722c3640 1 --2- 172.21.15.169:0/3674599004 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f6e5006ae40 0x7f6e5006d300 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f6e68035000 tx=0x7f6e68004ab0 comp rx=0 tx=0).stop 2023-10-26T01:31:39.575 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:39.572+0000 7f6e722c3640 1 -- 172.21.15.169:0/3674599004 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6e6c0aab80 msgr2=0x7f6e6c1f3520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:39.575 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:39.572+0000 7f6e722c3640 1 --2- 172.21.15.169:0/3674599004 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6e6c0aab80 0x7f6e6c1f3520 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f6e6400c9a0 tx=0x7f6e6400ce60 comp rx=0 tx=0).stop 2023-10-26T01:31:39.575 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:39.572+0000 7f6e722c3640 1 -- 172.21.15.169:0/3674599004 shutdown_connections 2023-10-26T01:31:39.576 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:39.572+0000 7f6e722c3640 1 --2- 172.21.15.169:0/3674599004 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f6e5006ae40 0x7f6e5006d300 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:39.576 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:39.572+0000 7f6e722c3640 1 --2- 172.21.15.169:0/3674599004 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6e6c0aab80 0x7f6e6c1f3520 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:39.576 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:39.572+0000 7f6e722c3640 1 --2- 172.21.15.169:0/3674599004 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e6c0aa6d0 0x7f6e6c156de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:39.576 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:39.572+0000 7f6e722c3640 1 --2- 172.21.15.169:0/3674599004 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6e6c06cba0 0x7f6e6c1f2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:39.576 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:39.572+0000 7f6e722c3640 1 -- 172.21.15.169:0/3674599004 >> 172.21.15.169:0/3674599004 conn(0x7f6e6c0af110 msgr2=0x7f6e6c071040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:39.576 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:39.572+0000 7f6e722c3640 1 -- 172.21.15.169:0/3674599004 shutdown_connections 2023-10-26T01:31:39.577 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:39.572+0000 7f6e722c3640 1 -- 172.21.15.169:0/3674599004 wait complete. 2023-10-26T01:31:39.592 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2023-10-26T01:31:39.592 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:31:39.592 DEBUG:teuthology.orchestra.run.smithi170:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2023-10-26T01:31:39.703 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi170.front.sepia.ceph.com 2023-10-26T01:31:39.704 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:31:39.704 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-26T01:31:39.759 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-26T01:31:39.759 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T01:31:39.979 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:39.976+0000 7fa356d2c640 1 -- 172.21.15.169:0/3831099011 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa3500cdc60 msgr2=0x7fa35006ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:39.980 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:39.976+0000 7fa356d2c640 1 --2- 172.21.15.169:0/3831099011 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa3500cdc60 0x7fa35006ad30 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fa3500aff40 tx=0x7fa34802c730 comp rx=0 tx=0).stop 2023-10-26T01:31:39.980 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:39.976+0000 7fa356d2c640 1 -- 172.21.15.169:0/3831099011 shutdown_connections 2023-10-26T01:31:39.980 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:39.976+0000 7fa356d2c640 1 --2- 172.21.15.169:0/3831099011 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa3500cdc60 0x7fa35006ad30 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:39.980 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:39.976+0000 7fa356d2c640 1 --2- 172.21.15.169:0/3831099011 >> v2:172.21.15.170:3300/0 conn(0x7fa350158850 0x7fa3500cd720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:39.981 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:39.976+0000 7fa356d2c640 1 --2- 172.21.15.169:0/3831099011 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa35014e790 0x7fa3500aab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:39.981 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:39.976+0000 7fa356d2c640 1 -- 172.21.15.169:0/3831099011 >> 172.21.15.169:0/3831099011 conn(0x7fa350069930 msgr2=0x7fa350069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:39.981 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:39.976+0000 7fa356d2c640 1 -- 172.21.15.169:0/3831099011 shutdown_connections 2023-10-26T01:31:39.981 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:39.976+0000 7fa356d2c640 1 -- 172.21.15.169:0/3831099011 wait complete. 2023-10-26T01:31:39.983 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:39.978+0000 7fa356d2c640 1 Processor -- start 2023-10-26T01:31:39.984 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:39.978+0000 7fa356d2c640 1 -- start start 2023-10-26T01:31:39.984 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:39.979+0000 7fa356d2c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa3500cdc60 0x7fa350148ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:39.984 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:39.979+0000 7fa356d2c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa35014e790 0x7fa350152300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:39.984 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:39.979+0000 7fa356d2c640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa350158850 0x7fa3500c86d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:39.984 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:39.979+0000 7fa356d2c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa35014dbd0 con 0x7fa35014e790 2023-10-26T01:31:39.985 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:39.979+0000 7fa356d2c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa350154000 con 0x7fa3500cdc60 2023-10-26T01:31:39.985 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:39.979+0000 7fa356d2c640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa35014d330 con 0x7fa350158850 2023-10-26T01:31:39.985 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:39.980+0000 7fa34ffff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa35014e790 0x7fa350152300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:39.985 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:39.980+0000 7fa34ffff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa35014e790 0x7fa350152300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:51534/0 (socket says 172.21.15.169:51534) 2023-10-26T01:31:39.986 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:39.980+0000 7fa34ffff640 1 -- 172.21.15.169:0/2793687647 learned_addr learned my addr 172.21.15.169:0/2793687647 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:31:39.986 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:39.980+0000 7fa354ac8640 1 --2- 172.21.15.169:0/2793687647 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa3500cdc60 0x7fa350148ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:39.986 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:39.980+0000 7fa34ffff640 1 -- 172.21.15.169:0/2793687647 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa350158850 msgr2=0x7fa3500c86d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:39.986 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:39.980+0000 7fa34ffff640 1 --2- 172.21.15.169:0/2793687647 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa350158850 0x7fa3500c86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:39.987 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:39.980+0000 7fa34ffff640 1 -- 172.21.15.169:0/2793687647 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa3500cdc60 msgr2=0x7fa350148ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:39.987 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:39.980+0000 7fa34ffff640 1 --2- 172.21.15.169:0/2793687647 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa3500cdc60 0x7fa350148ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:39.987 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:39.980+0000 7fa34ffff640 1 -- 172.21.15.169:0/2793687647 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa34800c4a0 con 0x7fa35014e790 2023-10-26T01:31:39.987 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:39.980+0000 7fa354ac8640 1 --2- 172.21.15.169:0/2793687647 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa3500cdc60 0x7fa350148ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:31:39.987 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:39.980+0000 7fa34ffff640 1 --2- 172.21.15.169:0/2793687647 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa35014e790 0x7fa350152300 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fa33c00c970 tx=0x7fa33c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:39.988 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:39.980+0000 7fa34d7fa640 1 -- 172.21.15.169:0/2793687647 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa33c007bb0 con 0x7fa35014e790 2023-10-26T01:31:39.988 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:39.981+0000 7fa34d7fa640 1 -- 172.21.15.169:0/2793687647 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa33c007e00 con 0x7fa35014e790 2023-10-26T01:31:39.988 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:39.981+0000 7fa356d2c640 1 -- 172.21.15.169:0/2793687647 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa35014d610 con 0x7fa35014e790 2023-10-26T01:31:39.988 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:39.981+0000 7fa34d7fa640 1 -- 172.21.15.169:0/2793687647 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa33c00f6b0 con 0x7fa35014e790 2023-10-26T01:31:39.988 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:39.981+0000 7fa356d2c640 1 -- 172.21.15.169:0/2793687647 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3501542a0 con 0x7fa35014e790 2023-10-26T01:31:39.989 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:39.982+0000 7fa356d2c640 1 -- 172.21.15.169:0/2793687647 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3501544c0 con 0x7fa35014e790 2023-10-26T01:31:39.990 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:39.983+0000 7fa34d7fa640 1 -- 172.21.15.169:0/2793687647 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fa33c00f810 con 0x7fa35014e790 2023-10-26T01:31:39.990 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:39.984+0000 7fa34d7fa640 1 --2- 172.21.15.169:0/2793687647 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa33406ae40 0x7fa33406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:39.990 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:39.984+0000 7fa34d7fa640 1 -- 172.21.15.169:0/2793687647 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fa33c089f30 con 0x7fa35014e790 2023-10-26T01:31:39.990 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:39.984+0000 7fa354ac8640 1 --2- 172.21.15.169:0/2793687647 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa33406ae40 0x7fa33406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:39.990 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:39.985+0000 7fa354ac8640 1 --2- 172.21.15.169:0/2793687647 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa33406ae40 0x7fa33406d300 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa344005940 tx=0x7fa344005c90 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:39.991 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:39.988+0000 7fa34d7fa640 1 -- 172.21.15.169:0/2793687647 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa33c054f50 con 0x7fa35014e790 2023-10-26T01:31:40.218 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:40.216+0000 7fa356d2c640 1 -- 172.21.15.169:0/2793687647 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa35004db10 con 0x7fa35014e790 2023-10-26T01:31:40.220 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:40.217+0000 7fa34d7fa640 1 -- 172.21.15.169:0/2793687647 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4916 (secure 0 0 0) 0x7fa33c058570 con 0x7fa35014e790 2023-10-26T01:31:40.220 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:31:40.220 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4298,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1809801299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":1809801299},{"type":"v1","addr":"172.21.15.169:6835","nonce":1809801299}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4304,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/2248566141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2248566141},{"type":"v1","addr":"172.21.15.170:6836","nonce":2248566141}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4317,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/2383616530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2383616530},{"type":"v1","addr":"172.21.15.169:6834","nonce":2383616530}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T01:30:41.545068+0000","modified":"2023-10-26T01:30:44.577169+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4338},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4338":{"gid":4338,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6837/590609220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":590609220},{"type":"v1","addr":"172.21.15.170:6837","nonce":590609220}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-26T01:31:40.221 INFO:teuthology.orchestra.run.smithi169.stderr:dumped fsmap epoch 7 2023-10-26T01:31:40.223 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:40.220+0000 7fa356d2c640 1 -- 172.21.15.169:0/2793687647 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa33406ae40 msgr2=0x7fa33406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:40.223 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:40.220+0000 7fa356d2c640 1 --2- 172.21.15.169:0/2793687647 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa33406ae40 0x7fa33406d300 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa344005940 tx=0x7fa344005c90 comp rx=0 tx=0).stop 2023-10-26T01:31:40.223 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:40.220+0000 7fa356d2c640 1 -- 172.21.15.169:0/2793687647 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa35014e790 msgr2=0x7fa350152300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:40.223 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:40.220+0000 7fa356d2c640 1 --2- 172.21.15.169:0/2793687647 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa35014e790 0x7fa350152300 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fa33c00c970 tx=0x7fa33c00ce30 comp rx=0 tx=0).stop 2023-10-26T01:31:40.224 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:40.220+0000 7fa356d2c640 1 -- 172.21.15.169:0/2793687647 shutdown_connections 2023-10-26T01:31:40.224 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:40.220+0000 7fa356d2c640 1 --2- 172.21.15.169:0/2793687647 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa33406ae40 0x7fa33406d300 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:40.224 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:40.220+0000 7fa356d2c640 1 --2- 172.21.15.169:0/2793687647 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa35014e790 0x7fa350152300 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:40.224 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:40.220+0000 7fa356d2c640 1 --2- 172.21.15.169:0/2793687647 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa350158850 0x7fa3500c86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:40.224 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:40.220+0000 7fa356d2c640 1 --2- 172.21.15.169:0/2793687647 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa3500cdc60 0x7fa350148ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:40.225 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:40.220+0000 7fa356d2c640 1 -- 172.21.15.169:0/2793687647 >> 172.21.15.169:0/2793687647 conn(0x7fa350069930 msgr2=0x7fa350157220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:40.225 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:40.220+0000 7fa356d2c640 1 -- 172.21.15.169:0/2793687647 shutdown_connections 2023-10-26T01:31:40.225 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:40.220+0000 7fa356d2c640 1 -- 172.21.15.169:0/2793687647 wait complete. 2023-10-26T01:31:40.240 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:31:40.445 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:40.442+0000 7feddf59e640 1 -- 172.21.15.169:0/563412837 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fede00c9c30 msgr2=0x7fede00cb390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:40.446 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:40.442+0000 7feddf59e640 1 --2- 172.21.15.169:0/563412837 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fede00c9c30 0x7fede00cb390 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fede00ca5b0 tx=0x7fedd802c1c0 comp rx=0 tx=0).stop 2023-10-26T01:31:40.446 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:40.442+0000 7feddf59e640 1 -- 172.21.15.169:0/563412837 shutdown_connections 2023-10-26T01:31:40.446 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:40.442+0000 7feddf59e640 1 --2- 172.21.15.169:0/563412837 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fede00c9c30 0x7fede00cb390 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:40.446 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:40.442+0000 7feddf59e640 1 --2- 172.21.15.169:0/563412837 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fede006a990 0x7fede00c96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:40.446 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:40.442+0000 7feddf59e640 1 --2- 172.21.15.169:0/563412837 >> v2:172.21.15.170:3300/0 conn(0x7fede006cae0 0x7fede00c7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:40.447 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:40.442+0000 7feddf59e640 1 -- 172.21.15.169:0/563412837 >> 172.21.15.169:0/563412837 conn(0x7fede00aedd0 msgr2=0x7fede00af1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:40.447 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:40.442+0000 7feddf59e640 1 -- 172.21.15.169:0/563412837 shutdown_connections 2023-10-26T01:31:40.447 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:40.442+0000 7feddf59e640 1 -- 172.21.15.169:0/563412837 wait complete. 2023-10-26T01:31:40.449 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:40.444+0000 7feddf59e640 1 Processor -- start 2023-10-26T01:31:40.450 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:40.444+0000 7feddf59e640 1 -- start start 2023-10-26T01:31:40.450 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:40.445+0000 7feddf59e640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fede006a990 0x7fede0154250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:40.450 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:40.445+0000 7feddf59e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fede006cae0 0x7fede0099b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:40.450 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:40.445+0000 7feddf59e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fede00c9c30 0x7fede0154a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:40.451 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:40.445+0000 7feddf59e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fede0153f70 con 0x7fede00c9c30 2023-10-26T01:31:40.451 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:40.445+0000 7feddf59e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fede00ab4a0 con 0x7fede006cae0 2023-10-26T01:31:40.451 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:40.445+0000 7feddf59e640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fede00ab780 con 0x7fede006a990 2023-10-26T01:31:40.451 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:40.445+0000 7fedded9d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fede00c9c30 0x7fede0154a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:40.452 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:40.445+0000 7fedded9d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fede00c9c30 0x7fede0154a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:51552/0 (socket says 172.21.15.169:51552) 2023-10-26T01:31:40.452 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:40.445+0000 7fedded9d640 1 -- 172.21.15.169:0/1495874427 learned_addr learned my addr 172.21.15.169:0/1495874427 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:31:40.452 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:40.446+0000 7fedd7fff640 1 --2- 172.21.15.169:0/1495874427 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fede006cae0 0x7fede0099b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:40.452 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:40.446+0000 7fedded9d640 1 -- 172.21.15.169:0/1495874427 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fede006a990 msgr2=0x7fede0154250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:40.453 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:40.446+0000 7fedded9d640 1 --2- 172.21.15.169:0/1495874427 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fede006a990 0x7fede0154250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:40.453 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:40.446+0000 7fedded9d640 1 -- 172.21.15.169:0/1495874427 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fede006cae0 msgr2=0x7fede0099b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:40.453 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:40.446+0000 7fedded9d640 1 --2- 172.21.15.169:0/1495874427 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fede006cae0 0x7fede0099b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:40.453 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:40.446+0000 7fedded9d640 1 -- 172.21.15.169:0/1495874427 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedd8009900 con 0x7fede00c9c30 2023-10-26T01:31:40.454 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:40.446+0000 7fedded9d640 1 --2- 172.21.15.169:0/1495874427 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fede00c9c30 0x7fede0154a90 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fedd8009d60 tx=0x7fedd8035e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:40.454 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:40.446+0000 7fedd6ffd640 1 -- 172.21.15.169:0/1495874427 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedd8034440 con 0x7fede00c9c30 2023-10-26T01:31:40.454 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:40.446+0000 7fedd6ffd640 1 -- 172.21.15.169:0/1495874427 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fedd8004540 con 0x7fede00c9c30 2023-10-26T01:31:40.454 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:40.446+0000 7feddf59e640 1 -- 172.21.15.169:0/1495874427 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fede00aba60 con 0x7fede00c9c30 2023-10-26T01:31:40.454 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:40.446+0000 7fedd6ffd640 1 -- 172.21.15.169:0/1495874427 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedd8040450 con 0x7fede00c9c30 2023-10-26T01:31:40.455 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:40.447+0000 7feddf59e640 1 -- 172.21.15.169:0/1495874427 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fede009a070 con 0x7fede00c9c30 2023-10-26T01:31:40.456 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:40.447+0000 7feddf59e640 1 -- 172.21.15.169:0/1495874427 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fede009a230 con 0x7fede00c9c30 2023-10-26T01:31:40.456 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:40.449+0000 7fedd6ffd640 1 -- 172.21.15.169:0/1495874427 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fedd800a040 con 0x7fede00c9c30 2023-10-26T01:31:40.456 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:40.449+0000 7fedd6ffd640 1 --2- 172.21.15.169:0/1495874427 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fedc006ae40 0x7fedc006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:40.456 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:40.450+0000 7fedd6ffd640 1 -- 172.21.15.169:0/1495874427 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fedd80acb10 con 0x7fede00c9c30 2023-10-26T01:31:40.457 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:40.450+0000 7fedde59c640 1 --2- 172.21.15.169:0/1495874427 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fedc006ae40 0x7fedc006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:40.457 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:40.451+0000 7fedde59c640 1 --2- 172.21.15.169:0/1495874427 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fedc006ae40 0x7fedc006d300 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fedd0009710 tx=0x7fedd0006c90 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:40.457 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:40.454+0000 7fedd6ffd640 1 -- 172.21.15.169:0/1495874427 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fedd8077b30 con 0x7fede00c9c30 2023-10-26T01:31:40.635 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:40.633+0000 7feddf59e640 1 -- 172.21.15.169:0/1495874427 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fede0073c10 con 0x7fede00c9c30 2023-10-26T01:31:40.638 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:40.635+0000 7fedd6ffd640 1 -- 172.21.15.169:0/1495874427 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16801 (secure 0 0 0) 0x7fedd8044090 con 0x7fede00c9c30 2023-10-26T01:31:40.639 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:31:40.639 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":23,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:31:30.100039+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-26T01:30:39.437631+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-26T01:30:40.561774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":2}]},{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.24","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:31:40.641 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:40.638+0000 7feddf59e640 1 -- 172.21.15.169:0/1495874427 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fedc006ae40 msgr2=0x7fedc006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:40.641 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:40.638+0000 7feddf59e640 1 --2- 172.21.15.169:0/1495874427 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fedc006ae40 0x7fedc006d300 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fedd0009710 tx=0x7fedd0006c90 comp rx=0 tx=0).stop 2023-10-26T01:31:40.641 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:40.638+0000 7feddf59e640 1 -- 172.21.15.169:0/1495874427 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fede00c9c30 msgr2=0x7fede0154a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:40.641 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:40.638+0000 7feddf59e640 1 --2- 172.21.15.169:0/1495874427 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fede00c9c30 0x7fede0154a90 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fedd8009d60 tx=0x7fedd8035e30 comp rx=0 tx=0).stop 2023-10-26T01:31:40.642 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:40.638+0000 7feddf59e640 1 -- 172.21.15.169:0/1495874427 shutdown_connections 2023-10-26T01:31:40.642 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:40.638+0000 7feddf59e640 1 --2- 172.21.15.169:0/1495874427 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fedc006ae40 0x7fedc006d300 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:40.642 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:40.638+0000 7feddf59e640 1 --2- 172.21.15.169:0/1495874427 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fede00c9c30 0x7fede0154a90 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:40.642 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:40.638+0000 7feddf59e640 1 --2- 172.21.15.169:0/1495874427 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fede006cae0 0x7fede0099b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:40.642 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:40.638+0000 7feddf59e640 1 --2- 172.21.15.169:0/1495874427 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fede006a990 0x7fede0154250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:40.643 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:40.638+0000 7feddf59e640 1 -- 172.21.15.169:0/1495874427 >> 172.21.15.169:0/1495874427 conn(0x7fede00aedd0 msgr2=0x7fede0156cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:40.643 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:40.638+0000 7feddf59e640 1 -- 172.21.15.169:0/1495874427 shutdown_connections 2023-10-26T01:31:40.643 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:40.638+0000 7feddf59e640 1 -- 172.21.15.169:0/1495874427 wait complete. 2023-10-26T01:31:40.657 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T01:30:32.200808+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-26T01:30:39.437631+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-26T01:30:40.561774+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T01:31:40.658 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T01:31:40.875 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:40.872+0000 7f7fdd8c2640 1 -- 172.21.15.169:0/2618467372 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7fd80cde70 msgr2=0x7f7fd8158300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:40.876 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:40.872+0000 7f7fdd8c2640 1 --2- 172.21.15.169:0/2618467372 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7fd80cde70 0x7f7fd8158300 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f7fd0005fd0 tx=0x7f7fd002bd80 comp rx=0 tx=0).stop 2023-10-26T01:31:40.876 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:40.872+0000 7f7fdd8c2640 1 -- 172.21.15.169:0/2618467372 shutdown_connections 2023-10-26T01:31:40.876 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:40.872+0000 7f7fdd8c2640 1 --2- 172.21.15.169:0/2618467372 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7fd80cde70 0x7f7fd8158300 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:40.876 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:40.872+0000 7f7fdd8c2640 1 --2- 172.21.15.169:0/2618467372 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7fd80ab070 0x7f7fd80ac870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:40.877 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:40.872+0000 7f7fdd8c2640 1 -- 172.21.15.169:0/2618467372 >> 172.21.15.169:0/2618467372 conn(0x7f7fd8069930 msgr2=0x7f7fd8069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:40.877 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:40.872+0000 7f7fdd8c2640 1 -- 172.21.15.169:0/2618467372 shutdown_connections 2023-10-26T01:31:40.877 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:40.873+0000 7f7fdd8c2640 1 -- 172.21.15.169:0/2618467372 wait complete. 2023-10-26T01:31:40.879 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:40.874+0000 7f7fdd8c2640 1 Processor -- start 2023-10-26T01:31:40.879 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:40.874+0000 7f7fdd8c2640 1 -- start start 2023-10-26T01:31:40.880 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:40.875+0000 7f7fdd8c2640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7fd80ab070 0x7f7fd8147e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:40.880 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:40.875+0000 7f7fdd8c2640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7fd80cde70 0x7f7fd80c7010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:40.880 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:40.875+0000 7f7fdd8c2640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7fd80c7550 0x7f7fd80ae240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:40.880 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:40.875+0000 7f7fdd8c2640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fd8142450 con 0x7f7fd80ab070 2023-10-26T01:31:40.880 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:40.875+0000 7f7fdd8c2640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fd80c7a50 con 0x7f7fd80cde70 2023-10-26T01:31:40.881 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:40.875+0000 7f7fdd8c2640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fd80c7d30 con 0x7f7fd80c7550 2023-10-26T01:31:40.881 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:40.876+0000 7f7fd6ffd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7fd80ab070 0x7f7fd8147e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:40.881 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:40.876+0000 7f7fd6ffd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7fd80ab070 0x7f7fd8147e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:51576/0 (socket says 172.21.15.169:51576) 2023-10-26T01:31:40.881 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:40.876+0000 7f7fd6ffd640 1 -- 172.21.15.169:0/654922727 learned_addr learned my addr 172.21.15.169:0/654922727 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:31:40.881 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:40.876+0000 7f7fd6ffd640 1 -- 172.21.15.169:0/654922727 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7fd80c7550 msgr2=0x7f7fd80ae240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:40.882 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:40.876+0000 7f7fd6ffd640 1 --2- 172.21.15.169:0/654922727 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7fd80c7550 0x7f7fd80ae240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:40.882 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:40.876+0000 7f7fd6ffd640 1 -- 172.21.15.169:0/654922727 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7fd80cde70 msgr2=0x7f7fd80c7010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:40.882 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:40.876+0000 7f7fd67fc640 1 --2- 172.21.15.169:0/654922727 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7fd80cde70 0x7f7fd80c7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:40.882 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:40.876+0000 7f7fd6ffd640 1 --2- 172.21.15.169:0/654922727 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7fd80cde70 0x7f7fd80c7010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:40.882 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:40.876+0000 7f7fd6ffd640 1 -- 172.21.15.169:0/654922727 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fd0009560 con 0x7f7fd80ab070 2023-10-26T01:31:40.883 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:40.876+0000 7f7fd6ffd640 1 --2- 172.21.15.169:0/654922727 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7fd80ab070 0x7f7fd8147e90 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f7fc800c7a0 tx=0x7f7fc800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:40.883 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:40.877+0000 7f7fcffff640 1 -- 172.21.15.169:0/654922727 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc8003d80 con 0x7f7fd80ab070 2023-10-26T01:31:40.883 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:40.877+0000 7f7fcffff640 1 -- 172.21.15.169:0/654922727 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7fc8005070 con 0x7f7fd80ab070 2023-10-26T01:31:40.883 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:40.877+0000 7f7fcffff640 1 -- 172.21.15.169:0/654922727 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc80054e0 con 0x7f7fd80ab070 2023-10-26T01:31:40.883 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:40.877+0000 7f7fdd8c2640 1 -- 172.21.15.169:0/654922727 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fd80c8010 con 0x7f7fd80ab070 2023-10-26T01:31:40.884 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:40.877+0000 7f7fdd8c2640 1 -- 172.21.15.169:0/654922727 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fd81483d0 con 0x7f7fd80ab070 2023-10-26T01:31:40.885 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:40.878+0000 7f7fdd8c2640 1 -- 172.21.15.169:0/654922727 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fd8066af0 con 0x7f7fd80ab070 2023-10-26T01:31:40.885 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:40.879+0000 7f7fcffff640 1 -- 172.21.15.169:0/654922727 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f7fc8005640 con 0x7f7fd80ab070 2023-10-26T01:31:40.885 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:40.880+0000 7f7fcffff640 1 --2- 172.21.15.169:0/654922727 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7fb806ae40 0x7f7fb806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:40.885 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:40.880+0000 7f7fcffff640 1 -- 172.21.15.169:0/654922727 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f7fc8088dc0 con 0x7f7fd80ab070 2023-10-26T01:31:40.885 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:40.881+0000 7f7fd67fc640 1 --2- 172.21.15.169:0/654922727 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7fb806ae40 0x7f7fb806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:40.886 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:40.882+0000 7f7fd67fc640 1 --2- 172.21.15.169:0/654922727 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7fb806ae40 0x7f7fb806d300 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7fd80c9b80 tx=0x7f7fc0008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:40.887 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:40.884+0000 7f7fcffff640 1 -- 172.21.15.169:0/654922727 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7fc8053e80 con 0x7f7fd80ab070 2023-10-26T01:31:41.077 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.075+0000 7f7fdd8c2640 1 -- 172.21.15.169:0/654922727 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7fd804da70 con 0x7f7fd80ab070 2023-10-26T01:31:41.078 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.076+0000 7f7fcffff640 1 -- 172.21.15.169:0/654922727 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7fc80170a0 con 0x7f7fd80ab070 2023-10-26T01:31:41.079 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 1 2023-10-26T01:31:41.081 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.078+0000 7f7fdd8c2640 1 -- 172.21.15.169:0/654922727 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7fb806ae40 msgr2=0x7f7fb806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:41.081 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.078+0000 7f7fdd8c2640 1 --2- 172.21.15.169:0/654922727 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7fb806ae40 0x7f7fb806d300 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7fd80c9b80 tx=0x7f7fc0008040 comp rx=0 tx=0).stop 2023-10-26T01:31:41.081 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.078+0000 7f7fdd8c2640 1 -- 172.21.15.169:0/654922727 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7fd80ab070 msgr2=0x7f7fd8147e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:41.082 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.078+0000 7f7fdd8c2640 1 --2- 172.21.15.169:0/654922727 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7fd80ab070 0x7f7fd8147e90 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f7fc800c7a0 tx=0x7f7fc800cc60 comp rx=0 tx=0).stop 2023-10-26T01:31:41.082 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.078+0000 7f7fdd8c2640 1 -- 172.21.15.169:0/654922727 shutdown_connections 2023-10-26T01:31:41.082 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.078+0000 7f7fdd8c2640 1 --2- 172.21.15.169:0/654922727 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7fb806ae40 0x7f7fb806d300 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:41.082 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.078+0000 7f7fdd8c2640 1 --2- 172.21.15.169:0/654922727 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7fd80c7550 0x7f7fd80ae240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:41.082 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.078+0000 7f7fdd8c2640 1 --2- 172.21.15.169:0/654922727 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7fd80cde70 0x7f7fd80c7010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:41.083 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.078+0000 7f7fdd8c2640 1 --2- 172.21.15.169:0/654922727 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7fd80ab070 0x7f7fd8147e90 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:41.083 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.078+0000 7f7fdd8c2640 1 -- 172.21.15.169:0/654922727 >> 172.21.15.169:0/654922727 conn(0x7f7fd8069930 msgr2=0x7f7fd814d8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:41.083 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.079+0000 7f7fdd8c2640 1 -- 172.21.15.169:0/654922727 shutdown_connections 2023-10-26T01:31:41.083 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.079+0000 7f7fdd8c2640 1 -- 172.21.15.169:0/654922727 wait complete. 2023-10-26T01:31:41.100 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-26T01:31:41.323 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.319+0000 7f3eafc02640 1 -- 172.21.15.169:0/1080001649 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3e94003970 msgr2=0x7f3e94003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:41.323 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.319+0000 7f3eafc02640 1 --2- 172.21.15.169:0/1080001649 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3e94003970 0x7f3e94003d90 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f3ea400c290 tx=0x7f3e940057f0 comp rx=0 tx=0).stop 2023-10-26T01:31:41.323 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.319+0000 7f3eae19f640 1 -- 172.21.15.169:0/1080001649 reap_dead start 2023-10-26T01:31:41.323 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.320+0000 7f3eafc02640 1 -- 172.21.15.169:0/1080001649 shutdown_connections 2023-10-26T01:31:41.324 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.320+0000 7f3eafc02640 1 -- 172.21.15.169:0/1080001649 >> 172.21.15.169:0/1080001649 conn(0x7f3ea8069930 msgr2=0x7f3ea8069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:41.324 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.320+0000 7f3eafc02640 1 -- 172.21.15.169:0/1080001649 shutdown_connections 2023-10-26T01:31:41.324 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.320+0000 7f3eafc02640 1 -- 172.21.15.169:0/1080001649 wait complete. 2023-10-26T01:31:41.325 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.321+0000 7f3eafc02640 1 Processor -- start 2023-10-26T01:31:41.325 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.322+0000 7f3eafc02640 1 -- start start 2023-10-26T01:31:41.326 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.323+0000 7f3eafc02640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3e94003970 0x7f3ea81f1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:41.327 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.323+0000 7f3eafc02640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3ea80c73a0 0x7f3ea80c7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:41.327 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.323+0000 7f3eafc02640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ea806b440 0x7f3ea806b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:41.327 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.323+0000 7f3eafc02640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea814d910 con 0x7f3ea806b440 2023-10-26T01:31:41.327 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.323+0000 7f3eafc02640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea80c7d60 con 0x7f3ea80c73a0 2023-10-26T01:31:41.328 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.323+0000 7f3eafc02640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea8159d70 con 0x7f3e94003970 2023-10-26T01:31:41.328 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.323+0000 7f3eae19f640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ea806b440 0x7f3ea806b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:41.328 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.323+0000 7f3eae19f640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ea806b440 0x7f3ea806b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:51600/0 (socket says 172.21.15.169:51600) 2023-10-26T01:31:41.328 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.323+0000 7f3eae19f640 1 -- 172.21.15.169:0/923805058 learned_addr learned my addr 172.21.15.169:0/923805058 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:31:41.328 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.323+0000 7f3ead99e640 1 --2- 172.21.15.169:0/923805058 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3e94003970 0x7f3ea81f1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:41.329 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.323+0000 7f3eae19f640 1 -- 172.21.15.169:0/923805058 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3e94003970 msgr2=0x7f3ea81f1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:41.329 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.323+0000 7f3ead19d640 1 --2- 172.21.15.169:0/923805058 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3ea80c73a0 0x7f3ea80c7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:41.329 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.323+0000 7f3eae19f640 1 --2- 172.21.15.169:0/923805058 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3e94003970 0x7f3ea81f1900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:41.329 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.323+0000 7f3eae19f640 1 -- 172.21.15.169:0/923805058 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3ea80c73a0 msgr2=0x7f3ea80c7820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:41.329 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.323+0000 7f3eae19f640 1 --2- 172.21.15.169:0/923805058 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3ea80c73a0 0x7f3ea80c7820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:41.330 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.323+0000 7f3eae19f640 1 -- 172.21.15.169:0/923805058 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ea806c150 con 0x7f3ea806b440 2023-10-26T01:31:41.330 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.324+0000 7f3eae19f640 1 --2- 172.21.15.169:0/923805058 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ea806b440 0x7f3ea806b8c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f3ea80ac190 tx=0x7f3ea000ed20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:41.330 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.324+0000 7f3e9e7fc640 1 -- 172.21.15.169:0/923805058 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ea000cbb0 con 0x7f3ea806b440 2023-10-26T01:31:41.330 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.324+0000 7f3e9e7fc640 1 -- 172.21.15.169:0/923805058 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3ea000ce00 con 0x7f3ea806b440 2023-10-26T01:31:41.330 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.324+0000 7f3e9e7fc640 1 -- 172.21.15.169:0/923805058 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ea00116d0 con 0x7f3ea806b440 2023-10-26T01:31:41.331 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.324+0000 7f3eafc02640 1 -- 172.21.15.169:0/923805058 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ea806c430 con 0x7f3ea806b440 2023-10-26T01:31:41.331 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.324+0000 7f3eafc02640 1 -- 172.21.15.169:0/923805058 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ea806c9f0 con 0x7f3ea806b440 2023-10-26T01:31:41.332 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.325+0000 7f3eafc02640 1 -- 172.21.15.169:0/923805058 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ea814da50 con 0x7f3ea806b440 2023-10-26T01:31:41.332 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.326+0000 7f3e9e7fc640 1 -- 172.21.15.169:0/923805058 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f3ea0011830 con 0x7f3ea806b440 2023-10-26T01:31:41.332 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.327+0000 7f3e9e7fc640 1 --2- 172.21.15.169:0/923805058 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3e8c06ae40 0x7f3e8c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:41.332 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.327+0000 7f3e9e7fc640 1 -- 172.21.15.169:0/923805058 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f3ea0089fc0 con 0x7f3ea806b440 2023-10-26T01:31:41.333 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.328+0000 7f3ead99e640 1 --2- 172.21.15.169:0/923805058 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3e8c06ae40 0x7f3e8c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:41.333 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.329+0000 7f3ead99e640 1 --2- 172.21.15.169:0/923805058 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3e8c06ae40 0x7f3e8c06d300 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3ea4034000 tx=0x7f3ea4001a70 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:41.333 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.331+0000 7f3e9e7fc640 1 -- 172.21.15.169:0/923805058 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3ea814da50 con 0x7f3ea806b440 2023-10-26T01:31:41.524 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.521+0000 7f3eafc02640 1 -- 172.21.15.169:0/923805058 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3ea814da50 con 0x7f3ea806b440 2023-10-26T01:31:41.525 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.522+0000 7f3e9e7fc640 1 -- 172.21.15.169:0/923805058 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3ea814da50 con 0x7f3ea806b440 2023-10-26T01:31:41.525 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 32 2023-10-26T01:31:41.528 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.525+0000 7f3eafc02640 1 -- 172.21.15.169:0/923805058 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3e8c06ae40 msgr2=0x7f3e8c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:41.529 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.525+0000 7f3eafc02640 1 --2- 172.21.15.169:0/923805058 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3e8c06ae40 0x7f3e8c06d300 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3ea4034000 tx=0x7f3ea4001a70 comp rx=0 tx=0).stop 2023-10-26T01:31:41.529 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.525+0000 7f3eafc02640 1 -- 172.21.15.169:0/923805058 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ea806b440 msgr2=0x7f3ea806b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:41.529 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.525+0000 7f3eafc02640 1 --2- 172.21.15.169:0/923805058 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ea806b440 0x7f3ea806b8c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f3ea80ac190 tx=0x7f3ea000ed20 comp rx=0 tx=0).stop 2023-10-26T01:31:41.529 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.525+0000 7f3eafc02640 1 -- 172.21.15.169:0/923805058 shutdown_connections 2023-10-26T01:31:41.529 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.525+0000 7f3eafc02640 1 --2- 172.21.15.169:0/923805058 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3e8c06ae40 0x7f3e8c06d300 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:41.530 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.525+0000 7f3eafc02640 1 --2- 172.21.15.169:0/923805058 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ea806b440 0x7f3ea806b8c0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:41.530 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.525+0000 7f3eafc02640 1 --2- 172.21.15.169:0/923805058 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3ea80c73a0 0x7f3ea80c7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:41.530 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.525+0000 7f3eafc02640 1 --2- 172.21.15.169:0/923805058 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3e94003970 0x7f3ea81f1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:41.530 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.525+0000 7f3eafc02640 1 -- 172.21.15.169:0/923805058 >> 172.21.15.169:0/923805058 conn(0x7f3ea8069930 msgr2=0x7f3ea80cbc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:41.531 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.526+0000 7f3eafc02640 1 -- 172.21.15.169:0/923805058 shutdown_connections 2023-10-26T01:31:41.531 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.526+0000 7f3eafc02640 1 -- 172.21.15.169:0/923805058 wait complete. 2023-10-26T01:31:41.550 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-26T01:31:41.773 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.769+0000 7f36dae75640 1 -- 172.21.15.169:0/1017715139 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f36c4004d00 msgr2=0x7f36c4005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:41.773 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.769+0000 7f36dae75640 1 --2- 172.21.15.169:0/1017715139 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f36c4004d00 0x7f36c4005180 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f36cc009e20 tx=0x7f36c40057f0 comp rx=0 tx=0).stop 2023-10-26T01:31:41.774 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.769+0000 7f36d9412640 1 -- 172.21.15.169:0/1017715139 reap_dead start 2023-10-26T01:31:41.774 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.769+0000 7f36dae75640 1 -- 172.21.15.169:0/1017715139 shutdown_connections 2023-10-26T01:31:41.774 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.770+0000 7f36dae75640 1 -- 172.21.15.169:0/1017715139 >> 172.21.15.169:0/1017715139 conn(0x7f36d4069930 msgr2=0x7f36d4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:41.774 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.770+0000 7f36dae75640 1 -- 172.21.15.169:0/1017715139 shutdown_connections 2023-10-26T01:31:41.775 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.770+0000 7f36dae75640 1 -- 172.21.15.169:0/1017715139 wait complete. 2023-10-26T01:31:41.777 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.771+0000 7f36dae75640 1 Processor -- start 2023-10-26T01:31:41.777 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.771+0000 7f36dae75640 1 -- start start 2023-10-26T01:31:41.777 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.772+0000 7f36dae75640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f36c4004d00 0x7f36d4150d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:41.777 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.772+0000 7f36dae75640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f36d41f0f10 0x7f36d41e4530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:41.778 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.772+0000 7f36dae75640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f36d41e4a70 0x7f36d415acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:41.778 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.772+0000 7f36dae75640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36d41f1420 con 0x7f36c4004d00 2023-10-26T01:31:41.778 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.772+0000 7f36dae75640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36d41f1700 con 0x7f36d41e4a70 2023-10-26T01:31:41.778 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.772+0000 7f36dae75640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36d41f19e0 con 0x7f36d41f0f10 2023-10-26T01:31:41.778 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.773+0000 7f36d8c11640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f36c4004d00 0x7f36d4150d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:41.779 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.773+0000 7f36d8c11640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f36c4004d00 0x7f36d4150d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:51620/0 (socket says 172.21.15.169:51620) 2023-10-26T01:31:41.779 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.773+0000 7f36d8c11640 1 -- 172.21.15.169:0/4189321430 learned_addr learned my addr 172.21.15.169:0/4189321430 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:31:41.779 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.773+0000 7f36d8c11640 1 -- 172.21.15.169:0/4189321430 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f36d41f0f10 msgr2=0x7f36d41e4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:41.779 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.773+0000 7f36d8c11640 1 --2- 172.21.15.169:0/4189321430 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f36d41f0f10 0x7f36d41e4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:41.780 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.773+0000 7f36d8c11640 1 -- 172.21.15.169:0/4189321430 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f36d41e4a70 msgr2=0x7f36d415acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:41.780 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.773+0000 7f36d8c11640 1 --2- 172.21.15.169:0/4189321430 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f36d41e4a70 0x7f36d415acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:41.780 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.773+0000 7f36d8c11640 1 -- 172.21.15.169:0/4189321430 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36d41f1cc0 con 0x7f36c4004d00 2023-10-26T01:31:41.780 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.773+0000 7f36d8c11640 1 --2- 172.21.15.169:0/4189321430 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f36c4004d00 0x7f36d4150d10 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f36c800d580 tx=0x7f36c800da40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:41.780 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.774+0000 7f36d17fa640 1 -- 172.21.15.169:0/4189321430 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36c8012630 con 0x7f36c4004d00 2023-10-26T01:31:41.781 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.774+0000 7f36d17fa640 1 -- 172.21.15.169:0/4189321430 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f36c8012790 con 0x7f36c4004d00 2023-10-26T01:31:41.781 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.774+0000 7f36dae75640 1 -- 172.21.15.169:0/4189321430 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36d41f1fa0 con 0x7f36c4004d00 2023-10-26T01:31:41.781 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.774+0000 7f36d17fa640 1 -- 172.21.15.169:0/4189321430 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36c8012bc0 con 0x7f36c4004d00 2023-10-26T01:31:41.781 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.774+0000 7f36dae75640 1 -- 172.21.15.169:0/4189321430 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36d4151250 con 0x7f36c4004d00 2023-10-26T01:31:41.782 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.775+0000 7f36dae75640 1 -- 172.21.15.169:0/4189321430 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36d415cbb0 con 0x7f36c4004d00 2023-10-26T01:31:41.783 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.776+0000 7f36d17fa640 1 -- 172.21.15.169:0/4189321430 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f36c801b400 con 0x7f36c4004d00 2023-10-26T01:31:41.783 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.777+0000 7f36d17fa640 1 --2- 172.21.15.169:0/4189321430 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f36b806ae70 0x7f36b806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:41.783 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.777+0000 7f36d17fa640 1 -- 172.21.15.169:0/4189321430 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f36c80898e0 con 0x7f36c4004d00 2023-10-26T01:31:41.783 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.777+0000 7f36d3fff640 1 --2- 172.21.15.169:0/4189321430 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f36b806ae70 0x7f36b806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:41.784 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.778+0000 7f36d3fff640 1 --2- 172.21.15.169:0/4189321430 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f36b806ae70 0x7f36b806d330 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f36d41f1330 tx=0x7f36c0001580 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:41.784 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.781+0000 7f36d17fa640 1 -- 172.21.15.169:0/4189321430 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f36c80583b0 con 0x7f36c4004d00 2023-10-26T01:31:41.973 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.970+0000 7f36dae75640 1 -- 172.21.15.169:0/4189321430 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f36d404db10 con 0x7f36c4004d00 2023-10-26T01:31:41.974 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.971+0000 7f36d17fa640 1 -- 172.21.15.169:0/4189321430 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f36c8057f40 con 0x7f36c4004d00 2023-10-26T01:31:41.974 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 64 2023-10-26T01:31:41.976 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.973+0000 7f36dae75640 1 -- 172.21.15.169:0/4189321430 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f36b806ae70 msgr2=0x7f36b806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:41.976 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.973+0000 7f36dae75640 1 --2- 172.21.15.169:0/4189321430 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f36b806ae70 0x7f36b806d330 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f36d41f1330 tx=0x7f36c0001580 comp rx=0 tx=0).stop 2023-10-26T01:31:41.977 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.973+0000 7f36dae75640 1 -- 172.21.15.169:0/4189321430 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f36c4004d00 msgr2=0x7f36d4150d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:41.977 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.973+0000 7f36dae75640 1 --2- 172.21.15.169:0/4189321430 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f36c4004d00 0x7f36d4150d10 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f36c800d580 tx=0x7f36c800da40 comp rx=0 tx=0).stop 2023-10-26T01:31:41.977 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.973+0000 7f36dae75640 1 -- 172.21.15.169:0/4189321430 shutdown_connections 2023-10-26T01:31:41.977 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.973+0000 7f36dae75640 1 --2- 172.21.15.169:0/4189321430 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f36b806ae70 0x7f36b806d330 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:41.977 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.973+0000 7f36dae75640 1 --2- 172.21.15.169:0/4189321430 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f36d41e4a70 0x7f36d415acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:41.978 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.973+0000 7f36dae75640 1 --2- 172.21.15.169:0/4189321430 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f36d41f0f10 0x7f36d41e4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:41.978 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.973+0000 7f36dae75640 1 --2- 172.21.15.169:0/4189321430 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f36c4004d00 0x7f36d4150d10 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:41.978 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.973+0000 7f36dae75640 1 -- 172.21.15.169:0/4189321430 >> 172.21.15.169:0/4189321430 conn(0x7f36d4069930 msgr2=0x7f36d4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:41.978 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.973+0000 7f36dae75640 1 -- 172.21.15.169:0/4189321430 shutdown_connections 2023-10-26T01:31:41.978 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:41.973+0000 7f36dae75640 1 -- 172.21.15.169:0/4189321430 wait complete. 2023-10-26T01:31:41.995 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T01:31:42.217 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:42.214+0000 7f64ee844640 1 -- 172.21.15.169:0/1082103310 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64d4003970 msgr2=0x7f64d4003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:42.218 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:42.214+0000 7f64ee844640 1 --2- 172.21.15.169:0/1082103310 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64d4003970 0x7f64d4003d90 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f64d8009b20 tx=0x7f64d40057f0 comp rx=0 tx=0).stop 2023-10-26T01:31:42.218 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:42.214+0000 7f64ecde1640 1 -- 172.21.15.169:0/1082103310 reap_dead start 2023-10-26T01:31:42.218 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:42.215+0000 7f64ee844640 1 -- 172.21.15.169:0/1082103310 shutdown_connections 2023-10-26T01:31:42.218 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:42.215+0000 7f64ee844640 1 -- 172.21.15.169:0/1082103310 >> 172.21.15.169:0/1082103310 conn(0x7f64e8069930 msgr2=0x7f64e8069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:42.219 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:42.215+0000 7f64ee844640 1 -- 172.21.15.169:0/1082103310 shutdown_connections 2023-10-26T01:31:42.219 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:42.215+0000 7f64ee844640 1 -- 172.21.15.169:0/1082103310 wait complete. 2023-10-26T01:31:42.219 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:42.216+0000 7f64ee844640 1 Processor -- start 2023-10-26T01:31:42.220 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:42.217+0000 7f64ee844640 1 -- start start 2023-10-26T01:31:42.221 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:42.217+0000 7f64ee844640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64d4003970 0x7f64e81f5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:42.222 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:42.218+0000 7f64ee844640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f64e806f1f0 0x7f64e81f1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:42.222 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:42.218+0000 7f64ee844640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f64e81deca0 0x7f64e81df150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:42.222 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:42.218+0000 7f64ee844640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64e8070300 con 0x7f64d4003970 2023-10-26T01:31:42.222 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:42.218+0000 7f64ee844640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64e81df690 con 0x7f64e81deca0 2023-10-26T01:31:42.223 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:42.218+0000 7f64ee844640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64e81f6130 con 0x7f64e806f1f0 2023-10-26T01:31:42.223 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:42.218+0000 7f64e7fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64d4003970 0x7f64e81f5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:42.223 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:42.218+0000 7f64e7fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64d4003970 0x7f64e81f5bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:51636/0 (socket says 172.21.15.169:51636) 2023-10-26T01:31:42.223 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:42.218+0000 7f64e7fff640 1 -- 172.21.15.169:0/2864514455 learned_addr learned my addr 172.21.15.169:0/2864514455 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:31:42.223 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:42.218+0000 7f64e7fff640 1 -- 172.21.15.169:0/2864514455 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f64e806f1f0 msgr2=0x7f64e81f1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:42.224 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:42.218+0000 7f64e7fff640 1 --2- 172.21.15.169:0/2864514455 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f64e806f1f0 0x7f64e81f1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:42.224 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:42.218+0000 7f64e7fff640 1 -- 172.21.15.169:0/2864514455 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f64e81deca0 msgr2=0x7f64e81df150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:42.224 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:42.218+0000 7f64e7fff640 1 --2- 172.21.15.169:0/2864514455 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f64e81deca0 0x7f64e81df150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:42.224 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:42.218+0000 7f64e7fff640 1 -- 172.21.15.169:0/2864514455 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64e81f1ac0 con 0x7f64d4003970 2023-10-26T01:31:42.224 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:42.219+0000 7f64e7fff640 1 --2- 172.21.15.169:0/2864514455 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64d4003970 0x7f64e81f5bf0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f64d802cb60 tx=0x7f64d8033c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:42.225 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:42.219+0000 7f64e57fa640 1 -- 172.21.15.169:0/2864514455 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64d8004220 con 0x7f64d4003970 2023-10-26T01:31:42.225 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:42.219+0000 7f64e57fa640 1 -- 172.21.15.169:0/2864514455 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f64d8004380 con 0x7f64d4003970 2023-10-26T01:31:42.225 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:42.219+0000 7f64e57fa640 1 -- 172.21.15.169:0/2864514455 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64d803d880 con 0x7f64d4003970 2023-10-26T01:31:42.225 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:42.219+0000 7f64ee844640 1 -- 172.21.15.169:0/2864514455 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64e806f680 con 0x7f64d4003970 2023-10-26T01:31:42.225 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:42.219+0000 7f64ee844640 1 -- 172.21.15.169:0/2864514455 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64e806fc40 con 0x7f64d4003970 2023-10-26T01:31:42.226 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:42.220+0000 7f64ee844640 1 -- 172.21.15.169:0/2864514455 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64e80b08f0 con 0x7f64d4003970 2023-10-26T01:31:42.227 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:42.221+0000 7f64e57fa640 1 -- 172.21.15.169:0/2864514455 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f64d803d9e0 con 0x7f64d4003970 2023-10-26T01:31:42.227 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:42.222+0000 7f64e57fa640 1 --2- 172.21.15.169:0/2864514455 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f64c806ae40 0x7f64c806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:42.227 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:42.222+0000 7f64e57fa640 1 -- 172.21.15.169:0/2864514455 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f64d8033e40 con 0x7f64d4003970 2023-10-26T01:31:42.227 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:42.223+0000 7f64df7fe640 1 --2- 172.21.15.169:0/2864514455 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f64c806ae40 0x7f64c806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:42.227 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:42.224+0000 7f64df7fe640 1 --2- 172.21.15.169:0/2864514455 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f64c806ae40 0x7f64c806d300 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f64d0009460 tx=0x7f64d0009390 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:42.229 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:42.227+0000 7f64e57fa640 1 -- 172.21.15.169:0/2864514455 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f64e80b08f0 con 0x7f64d4003970 2023-10-26T01:31:42.459 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:42.456+0000 7f64ee844640 1 -- 172.21.15.169:0/2864514455 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f64e804db10 con 0x7f64d4003970 2023-10-26T01:31:42.460 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:42.458+0000 7f64e57fa640 1 -- 172.21.15.169:0/2864514455 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4916 (secure 0 0 0) 0x7f64d8077370 con 0x7f64d4003970 2023-10-26T01:31:42.461 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:31:42.461 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4298,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1809801299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":1809801299},{"type":"v1","addr":"172.21.15.169:6835","nonce":1809801299}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4304,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/2248566141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2248566141},{"type":"v1","addr":"172.21.15.170:6836","nonce":2248566141}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4317,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/2383616530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2383616530},{"type":"v1","addr":"172.21.15.169:6834","nonce":2383616530}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T01:30:41.545068+0000","modified":"2023-10-26T01:30:44.577169+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4338},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4338":{"gid":4338,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6837/590609220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":590609220},{"type":"v1","addr":"172.21.15.170:6837","nonce":590609220}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-26T01:31:42.461 INFO:teuthology.orchestra.run.smithi169.stderr:dumped fsmap epoch 7 2023-10-26T01:31:42.463 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:42.460+0000 7f64ee844640 1 -- 172.21.15.169:0/2864514455 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f64c806ae40 msgr2=0x7f64c806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:42.464 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:42.460+0000 7f64ee844640 1 --2- 172.21.15.169:0/2864514455 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f64c806ae40 0x7f64c806d300 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f64d0009460 tx=0x7f64d0009390 comp rx=0 tx=0).stop 2023-10-26T01:31:42.464 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:42.461+0000 7f64ee844640 1 -- 172.21.15.169:0/2864514455 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64d4003970 msgr2=0x7f64e81f5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:42.464 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:42.461+0000 7f64ee844640 1 --2- 172.21.15.169:0/2864514455 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64d4003970 0x7f64e81f5bf0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f64d802cb60 tx=0x7f64d8033c50 comp rx=0 tx=0).stop 2023-10-26T01:31:42.464 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:42.461+0000 7f64ee844640 1 -- 172.21.15.169:0/2864514455 shutdown_connections 2023-10-26T01:31:42.465 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:42.461+0000 7f64ee844640 1 --2- 172.21.15.169:0/2864514455 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f64c806ae40 0x7f64c806d300 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:42.465 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:42.461+0000 7f64ee844640 1 --2- 172.21.15.169:0/2864514455 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f64e81deca0 0x7f64e81df150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:42.465 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:42.461+0000 7f64ee844640 1 --2- 172.21.15.169:0/2864514455 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f64e806f1f0 0x7f64e81f1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:42.465 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:42.461+0000 7f64ee844640 1 --2- 172.21.15.169:0/2864514455 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64d4003970 0x7f64e81f5bf0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:42.465 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:42.461+0000 7f64ee844640 1 -- 172.21.15.169:0/2864514455 >> 172.21.15.169:0/2864514455 conn(0x7f64e8069930 msgr2=0x7f64e814f1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:42.465 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:42.461+0000 7f64ee844640 1 -- 172.21.15.169:0/2864514455 shutdown_connections 2023-10-26T01:31:42.466 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:42.461+0000 7f64ee844640 1 -- 172.21.15.169:0/2864514455 wait complete. 2023-10-26T01:31:42.482 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:31:42.705 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:42.702+0000 7f2a62264640 1 -- 172.21.15.169:0/2719789518 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a5c0cdc30 msgr2=0x7f2a5c06ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:42.706 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:42.702+0000 7f2a62264640 1 --2- 172.21.15.169:0/2719789518 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a5c0cdc30 0x7f2a5c06ad30 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c0b3440 tx=0x7f2a5402bd10 comp rx=0 tx=0).stop 2023-10-26T01:31:42.706 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:42.702+0000 7f2a62264640 1 -- 172.21.15.169:0/2719789518 shutdown_connections 2023-10-26T01:31:42.706 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:42.702+0000 7f2a62264640 1 --2- 172.21.15.169:0/2719789518 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a5c0cdc30 0x7f2a5c06ad30 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:42.706 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:42.702+0000 7f2a62264640 1 --2- 172.21.15.169:0/2719789518 >> v2:172.21.15.170:3300/0 conn(0x7f2a5c0ac930 0x7f2a5c0cd6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:42.706 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:42.702+0000 7f2a62264640 1 --2- 172.21.15.169:0/2719789518 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2a5c1592d0 0x7f2a5c0aa3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:42.707 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:42.702+0000 7f2a62264640 1 -- 172.21.15.169:0/2719789518 >> 172.21.15.169:0/2719789518 conn(0x7f2a5c069930 msgr2=0x7f2a5c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:42.707 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:42.703+0000 7f2a62264640 1 -- 172.21.15.169:0/2719789518 shutdown_connections 2023-10-26T01:31:42.707 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:42.703+0000 7f2a62264640 1 -- 172.21.15.169:0/2719789518 wait complete. 2023-10-26T01:31:42.708 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:42.705+0000 7f2a62264640 1 Processor -- start 2023-10-26T01:31:42.708 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:42.705+0000 7f2a62264640 1 -- start start 2023-10-26T01:31:42.709 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:42.706+0000 7f2a62264640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a5c0ac930 0x7f2a5c0c8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:42.710 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:42.706+0000 7f2a62264640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a5c0cdc30 0x7f2a5c15b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:42.710 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:42.706+0000 7f2a62264640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2a5c1592d0 0x7f2a5c157b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:42.710 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:42.706+0000 7f2a62264640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a5c15b8e0 con 0x7f2a5c0ac930 2023-10-26T01:31:42.710 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:42.706+0000 7f2a62264640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a5c0b0630 con 0x7f2a5c0cdc30 2023-10-26T01:31:42.710 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:42.706+0000 7f2a62264640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a5c0b02e0 con 0x7f2a5c1592d0 2023-10-26T01:31:42.711 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:42.706+0000 7f2a5bfff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a5c0ac930 0x7f2a5c0c8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:42.711 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:42.706+0000 7f2a5bfff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a5c0ac930 0x7f2a5c0c8bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:51654/0 (socket says 172.21.15.169:51654) 2023-10-26T01:31:42.711 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:42.706+0000 7f2a5bfff640 1 -- 172.21.15.169:0/3263666020 learned_addr learned my addr 172.21.15.169:0/3263666020 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:31:42.711 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:42.707+0000 7f2a60801640 1 --2- 172.21.15.169:0/3263666020 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2a5c1592d0 0x7f2a5c157b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:42.711 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:42.707+0000 7f2a5b7fe640 1 --2- 172.21.15.169:0/3263666020 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a5c0cdc30 0x7f2a5c15b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:42.712 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:42.707+0000 7f2a5bfff640 1 -- 172.21.15.169:0/3263666020 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2a5c1592d0 msgr2=0x7f2a5c157b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:42.712 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:42.707+0000 7f2a5bfff640 1 --2- 172.21.15.169:0/3263666020 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2a5c1592d0 0x7f2a5c157b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:42.712 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:42.707+0000 7f2a5bfff640 1 -- 172.21.15.169:0/3263666020 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a5c0cdc30 msgr2=0x7f2a5c15b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:42.712 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:42.707+0000 7f2a5bfff640 1 --2- 172.21.15.169:0/3263666020 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a5c0cdc30 0x7f2a5c15b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:42.712 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:42.707+0000 7f2a5bfff640 1 -- 172.21.15.169:0/3263666020 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a54009560 con 0x7f2a5c0ac930 2023-10-26T01:31:42.713 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:42.707+0000 7f2a5bfff640 1 --2- 172.21.15.169:0/3263666020 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a5c0ac930 0x7f2a5c0c8bb0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c00cce0 tx=0x7f2a4c0075f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:42.714 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:42.707+0000 7f2a58ff9640 1 -- 172.21.15.169:0/3263666020 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a4c004b20 con 0x7f2a5c0ac930 2023-10-26T01:31:42.714 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:42.707+0000 7f2a58ff9640 1 -- 172.21.15.169:0/3263666020 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2a4c004d70 con 0x7f2a5c0ac930 2023-10-26T01:31:42.714 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:42.707+0000 7f2a62264640 1 -- 172.21.15.169:0/3263666020 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a5c15baf0 con 0x7f2a5c0ac930 2023-10-26T01:31:42.715 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:42.708+0000 7f2a58ff9640 1 -- 172.21.15.169:0/3263666020 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a4c005280 con 0x7f2a5c0ac930 2023-10-26T01:31:42.715 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:42.708+0000 7f2a62264640 1 -- 172.21.15.169:0/3263666020 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a5c15c0b0 con 0x7f2a5c0ac930 2023-10-26T01:31:42.715 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:42.709+0000 7f2a62264640 1 -- 172.21.15.169:0/3263666020 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a5c04db10 con 0x7f2a5c0ac930 2023-10-26T01:31:42.715 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:42.710+0000 7f2a58ff9640 1 -- 172.21.15.169:0/3263666020 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f2a4c0053e0 con 0x7f2a5c0ac930 2023-10-26T01:31:42.715 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:42.711+0000 7f2a58ff9640 1 --2- 172.21.15.169:0/3263666020 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2a3c06ae40 0x7f2a3c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:42.716 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:42.711+0000 7f2a58ff9640 1 -- 172.21.15.169:0/3263666020 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f2a4c089f80 con 0x7f2a5c0ac930 2023-10-26T01:31:42.716 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:42.711+0000 7f2a5b7fe640 1 --2- 172.21.15.169:0/3263666020 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2a3c06ae40 0x7f2a3c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:42.716 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:42.713+0000 7f2a5b7fe640 1 --2- 172.21.15.169:0/3263666020 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2a3c06ae40 0x7f2a3c06d300 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2a44005fd0 tx=0x7f2a44005850 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:42.718 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:42.715+0000 7f2a58ff9640 1 -- 172.21.15.169:0/3263666020 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2a4c054ec0 con 0x7f2a5c0ac930 2023-10-26T01:31:42.909 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:42.906+0000 7f2a62264640 1 -- 172.21.15.169:0/3263666020 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2a5c0cce80 con 0x7f2a5c0ac930 2023-10-26T01:31:42.910 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:42.908+0000 7f2a58ff9640 1 -- 172.21.15.169:0/3263666020 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16801 (secure 0 0 0) 0x7f2a4c01d030 con 0x7f2a5c0ac930 2023-10-26T01:31:42.911 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:31:42.911 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":23,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:31:30.100039+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-26T01:30:39.437631+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-26T01:30:40.561774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":2}]},{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.24","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:31:42.913 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:42.910+0000 7f2a62264640 1 -- 172.21.15.169:0/3263666020 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2a3c06ae40 msgr2=0x7f2a3c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:42.914 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:42.911+0000 7f2a62264640 1 --2- 172.21.15.169:0/3263666020 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2a3c06ae40 0x7f2a3c06d300 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2a44005fd0 tx=0x7f2a44005850 comp rx=0 tx=0).stop 2023-10-26T01:31:42.914 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:42.911+0000 7f2a62264640 1 -- 172.21.15.169:0/3263666020 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a5c0ac930 msgr2=0x7f2a5c0c8bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:42.914 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:42.911+0000 7f2a62264640 1 --2- 172.21.15.169:0/3263666020 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a5c0ac930 0x7f2a5c0c8bb0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c00cce0 tx=0x7f2a4c0075f0 comp rx=0 tx=0).stop 2023-10-26T01:31:42.914 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:42.911+0000 7f2a62264640 1 -- 172.21.15.169:0/3263666020 shutdown_connections 2023-10-26T01:31:42.915 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:42.911+0000 7f2a62264640 1 --2- 172.21.15.169:0/3263666020 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2a3c06ae40 0x7f2a3c06d300 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:42.915 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:42.911+0000 7f2a62264640 1 --2- 172.21.15.169:0/3263666020 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2a5c1592d0 0x7f2a5c157b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:42.915 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:42.911+0000 7f2a62264640 1 --2- 172.21.15.169:0/3263666020 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a5c0cdc30 0x7f2a5c15b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:42.915 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:42.911+0000 7f2a62264640 1 --2- 172.21.15.169:0/3263666020 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a5c0ac930 0x7f2a5c0c8bb0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:42.915 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:42.911+0000 7f2a62264640 1 -- 172.21.15.169:0/3263666020 >> 172.21.15.169:0/3263666020 conn(0x7f2a5c069930 msgr2=0x7f2a5c158550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:42.916 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:42.911+0000 7f2a62264640 1 -- 172.21.15.169:0/3263666020 shutdown_connections 2023-10-26T01:31:42.916 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:42.911+0000 7f2a62264640 1 -- 172.21.15.169:0/3263666020 wait complete. 2023-10-26T01:31:42.933 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-26T01:31:42.934 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T01:31:43.155 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:43.152+0000 7f0a1fb4c640 1 -- 172.21.15.169:0/717136441 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0a04003b30 msgr2=0x7f0a04003f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:43.155 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:43.152+0000 7f0a1fb4c640 1 --2- 172.21.15.169:0/717136441 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0a04003b30 0x7f0a04003f30 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f0a14001800 tx=0x7f0a040057f0 comp rx=0 tx=0).stop 2023-10-26T01:31:43.156 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:43.152+0000 7f0a1e0e9640 1 -- 172.21.15.169:0/717136441 reap_dead start 2023-10-26T01:31:43.156 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:43.152+0000 7f0a1fb4c640 1 -- 172.21.15.169:0/717136441 shutdown_connections 2023-10-26T01:31:43.156 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:43.152+0000 7f0a1fb4c640 1 -- 172.21.15.169:0/717136441 >> 172.21.15.169:0/717136441 conn(0x7f0a18069930 msgr2=0x7f0a18069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:43.156 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:43.152+0000 7f0a1fb4c640 1 -- 172.21.15.169:0/717136441 shutdown_connections 2023-10-26T01:31:43.157 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:43.152+0000 7f0a1fb4c640 1 -- 172.21.15.169:0/717136441 wait complete. 2023-10-26T01:31:43.157 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:43.154+0000 7f0a1fb4c640 1 Processor -- start 2023-10-26T01:31:43.157 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:43.154+0000 7f0a1fb4c640 1 -- start start 2023-10-26T01:31:43.160 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:43.155+0000 7f0a1fb4c640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0a04003b30 0x7f0a18151ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:43.160 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:43.155+0000 7f0a1fb4c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a18157e80 0x7f0a180ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:43.160 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:43.155+0000 7f0a1fb4c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0a180af710 0x7f0a180afbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:43.160 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:43.155+0000 7f0a1fb4c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a180b0100 con 0x7f0a180af710 2023-10-26T01:31:43.161 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:43.155+0000 7f0a1fb4c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a18098b60 con 0x7f0a18157e80 2023-10-26T01:31:43.161 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:43.155+0000 7f0a1fb4c640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a180ae190 con 0x7f0a04003b30 2023-10-26T01:31:43.161 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:43.156+0000 7f0a1e0e9640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0a180af710 0x7f0a180afbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:43.161 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:43.156+0000 7f0a1e0e9640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0a180af710 0x7f0a180afbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:51668/0 (socket says 172.21.15.169:51668) 2023-10-26T01:31:43.161 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:43.156+0000 7f0a1e0e9640 1 -- 172.21.15.169:0/2776242102 learned_addr learned my addr 172.21.15.169:0/2776242102 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:31:43.162 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:43.156+0000 7f0a1d8e8640 1 --2- 172.21.15.169:0/2776242102 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0a04003b30 0x7f0a18151ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:43.162 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:43.156+0000 7f0a1d0e7640 1 --2- 172.21.15.169:0/2776242102 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a18157e80 0x7f0a180ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:43.162 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:43.156+0000 7f0a1e0e9640 1 -- 172.21.15.169:0/2776242102 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0a04003b30 msgr2=0x7f0a18151ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:43.162 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:43.156+0000 7f0a1e0e9640 1 --2- 172.21.15.169:0/2776242102 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0a04003b30 0x7f0a18151ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:43.162 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:43.156+0000 7f0a1e0e9640 1 -- 172.21.15.169:0/2776242102 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a18157e80 msgr2=0x7f0a180ad840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:43.163 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:43.156+0000 7f0a1e0e9640 1 --2- 172.21.15.169:0/2776242102 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a18157e80 0x7f0a180ad840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:43.163 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:43.156+0000 7f0a1e0e9640 1 -- 172.21.15.169:0/2776242102 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a180ae470 con 0x7f0a180af710 2023-10-26T01:31:43.163 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:43.156+0000 7f0a1d0e7640 1 --2- 172.21.15.169:0/2776242102 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a18157e80 0x7f0a180ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:31:43.163 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:43.156+0000 7f0a1e0e9640 1 --2- 172.21.15.169:0/2776242102 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0a180af710 0x7f0a180afbc0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f0a10000f80 tx=0x7f0a1000ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:43.164 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:43.157+0000 7f0a0e7fc640 1 -- 172.21.15.169:0/2776242102 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a1000cbc0 con 0x7f0a180af710 2023-10-26T01:31:43.164 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:43.157+0000 7f0a0e7fc640 1 -- 172.21.15.169:0/2776242102 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0a1000ce30 con 0x7f0a180af710 2023-10-26T01:31:43.164 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:43.157+0000 7f0a1fb4c640 1 -- 172.21.15.169:0/2776242102 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a180ae750 con 0x7f0a180af710 2023-10-26T01:31:43.164 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:43.157+0000 7f0a0e7fc640 1 -- 172.21.15.169:0/2776242102 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a100116c0 con 0x7f0a180af710 2023-10-26T01:31:43.164 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:43.157+0000 7f0a1fb4c640 1 -- 172.21.15.169:0/2776242102 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a180add80 con 0x7f0a180af710 2023-10-26T01:31:43.165 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:43.158+0000 7f0a1fb4c640 1 -- 172.21.15.169:0/2776242102 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a1804db10 con 0x7f0a180af710 2023-10-26T01:31:43.165 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:43.159+0000 7f0a0e7fc640 1 -- 172.21.15.169:0/2776242102 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f0a10011820 con 0x7f0a180af710 2023-10-26T01:31:43.166 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:43.160+0000 7f0a0e7fc640 1 --2- 172.21.15.169:0/2776242102 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f09fc06ae70 0x7f09fc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:43.166 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:43.160+0000 7f0a0e7fc640 1 -- 172.21.15.169:0/2776242102 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f0a10089d40 con 0x7f0a180af710 2023-10-26T01:31:43.166 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:43.161+0000 7f0a1d8e8640 1 --2- 172.21.15.169:0/2776242102 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f09fc06ae70 0x7f09fc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:43.166 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:43.162+0000 7f0a1d8e8640 1 --2- 172.21.15.169:0/2776242102 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f09fc06ae70 0x7f09fc06d330 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0a14034000 tx=0x7f0a14039040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:43.167 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:43.165+0000 7f0a0e7fc640 1 -- 172.21.15.169:0/2776242102 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0a10054cd0 con 0x7f0a180af710 2023-10-26T01:31:43.396 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:43.394+0000 7f0a1fb4c640 1 -- 172.21.15.169:0/2776242102 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0a180ce080 con 0x7f0a180af710 2023-10-26T01:31:43.397 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:43.395+0000 7f0a0e7fc640 1 -- 172.21.15.169:0/2776242102 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4916 (secure 0 0 0) 0x7f0a180ce080 con 0x7f0a180af710 2023-10-26T01:31:43.398 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:31:43.398 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4298,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1809801299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":1809801299},{"type":"v1","addr":"172.21.15.169:6835","nonce":1809801299}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4304,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/2248566141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2248566141},{"type":"v1","addr":"172.21.15.170:6836","nonce":2248566141}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4317,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/2383616530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2383616530},{"type":"v1","addr":"172.21.15.169:6834","nonce":2383616530}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T01:30:41.545068+0000","modified":"2023-10-26T01:30:44.577169+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4338},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4338":{"gid":4338,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6837/590609220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":590609220},{"type":"v1","addr":"172.21.15.170:6837","nonce":590609220}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-26T01:31:43.398 INFO:teuthology.orchestra.run.smithi169.stderr:dumped fsmap epoch 7 2023-10-26T01:31:43.401 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:43.398+0000 7f0a1fb4c640 1 -- 172.21.15.169:0/2776242102 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f09fc06ae70 msgr2=0x7f09fc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:43.401 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:43.398+0000 7f0a1fb4c640 1 --2- 172.21.15.169:0/2776242102 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f09fc06ae70 0x7f09fc06d330 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0a14034000 tx=0x7f0a14039040 comp rx=0 tx=0).stop 2023-10-26T01:31:43.401 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:43.398+0000 7f0a1fb4c640 1 -- 172.21.15.169:0/2776242102 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0a180af710 msgr2=0x7f0a180afbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:43.402 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:43.398+0000 7f0a1fb4c640 1 --2- 172.21.15.169:0/2776242102 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0a180af710 0x7f0a180afbc0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f0a10000f80 tx=0x7f0a1000ed90 comp rx=0 tx=0).stop 2023-10-26T01:31:43.402 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:43.398+0000 7f0a1fb4c640 1 -- 172.21.15.169:0/2776242102 shutdown_connections 2023-10-26T01:31:43.402 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:43.398+0000 7f0a1fb4c640 1 --2- 172.21.15.169:0/2776242102 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f09fc06ae70 0x7f09fc06d330 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:43.402 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:43.398+0000 7f0a1fb4c640 1 --2- 172.21.15.169:0/2776242102 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0a180af710 0x7f0a180afbc0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:43.402 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:43.398+0000 7f0a1fb4c640 1 --2- 172.21.15.169:0/2776242102 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a18157e80 0x7f0a180ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:43.403 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:43.398+0000 7f0a1fb4c640 1 --2- 172.21.15.169:0/2776242102 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0a04003b30 0x7f0a18151ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:43.403 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:43.398+0000 7f0a1fb4c640 1 -- 172.21.15.169:0/2776242102 >> 172.21.15.169:0/2776242102 conn(0x7f0a18069930 msgr2=0x7f0a18153790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:43.403 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:43.398+0000 7f0a1fb4c640 1 -- 172.21.15.169:0/2776242102 shutdown_connections 2023-10-26T01:31:43.403 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:43.398+0000 7f0a1fb4c640 1 -- 172.21.15.169:0/2776242102 wait complete. 2023-10-26T01:31:43.420 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T01:31:43.629 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:43.626+0000 7f5af1832640 1 -- 172.21.15.169:0/624803119 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5aec0ac930 msgr2=0x7f5aec0cd6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:43.630 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:43.626+0000 7f5af1832640 1 --2- 172.21.15.169:0/624803119 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5aec0ac930 0x7f5aec0cd6f0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f5aec0aab10 tx=0x7f5ae002bcf0 comp rx=0 tx=0).stop 2023-10-26T01:31:43.630 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:43.627+0000 7f5af1832640 1 -- 172.21.15.169:0/624803119 shutdown_connections 2023-10-26T01:31:43.630 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:43.627+0000 7f5af1832640 1 --2- 172.21.15.169:0/624803119 >> v2:172.21.15.170:3300/0 conn(0x7f5aec0cdc30 0x7f5aec0cbac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:43.630 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:43.627+0000 7f5af1832640 1 --2- 172.21.15.169:0/624803119 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5aec0ac930 0x7f5aec0cd6f0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:43.630 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:43.627+0000 7f5af1832640 1 -- 172.21.15.169:0/624803119 >> 172.21.15.169:0/624803119 conn(0x7f5aec069930 msgr2=0x7f5aec069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:43.631 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:43.627+0000 7f5af1832640 1 -- 172.21.15.169:0/624803119 shutdown_connections 2023-10-26T01:31:43.631 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:43.627+0000 7f5af1832640 1 -- 172.21.15.169:0/624803119 wait complete. 2023-10-26T01:31:43.631 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:43.628+0000 7f5af1832640 1 Processor -- start 2023-10-26T01:31:43.632 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:43.629+0000 7f5af1832640 1 -- start start 2023-10-26T01:31:43.633 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:43.629+0000 7f5af1832640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5aec0ac930 0x7f5aec15c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:43.633 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:43.630+0000 7f5af1832640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5aec0cdc30 0x7f5aec148610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:43.634 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:43.630+0000 7f5af1832640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5aec1592d0 0x7f5aec0af740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:43.634 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:43.630+0000 7f5af1832640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aec0b1a20 con 0x7f5aec1592d0 2023-10-26T01:31:43.634 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:43.630+0000 7f5af1832640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aec148b50 con 0x7f5aec0cdc30 2023-10-26T01:31:43.634 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:43.630+0000 7f5af1832640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5aec15b3b0 con 0x7f5aec0ac930 2023-10-26T01:31:43.634 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:43.630+0000 7f5aeb7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5aec1592d0 0x7f5aec0af740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:43.635 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:43.630+0000 7f5aeb7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5aec1592d0 0x7f5aec0af740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:51698/0 (socket says 172.21.15.169:51698) 2023-10-26T01:31:43.635 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:43.630+0000 7f5aeb7fe640 1 -- 172.21.15.169:0/515414959 learned_addr learned my addr 172.21.15.169:0/515414959 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:31:43.635 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:43.630+0000 7f5aeaffd640 1 --2- 172.21.15.169:0/515414959 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5aec0ac930 0x7f5aec15c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:43.635 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:43.630+0000 7f5aea7fc640 1 --2- 172.21.15.169:0/515414959 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5aec0cdc30 0x7f5aec148610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:43.635 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:43.630+0000 7f5aeb7fe640 1 -- 172.21.15.169:0/515414959 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5aec0ac930 msgr2=0x7f5aec15c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:43.636 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:43.630+0000 7f5aeb7fe640 1 --2- 172.21.15.169:0/515414959 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5aec0ac930 0x7f5aec15c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:43.636 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:43.630+0000 7f5aeb7fe640 1 -- 172.21.15.169:0/515414959 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5aec0cdc30 msgr2=0x7f5aec148610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:43.636 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:43.630+0000 7f5aeb7fe640 1 --2- 172.21.15.169:0/515414959 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5aec0cdc30 0x7f5aec148610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:43.636 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:43.631+0000 7f5aeb7fe640 1 -- 172.21.15.169:0/515414959 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ae0009490 con 0x7f5aec1592d0 2023-10-26T01:31:43.636 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:43.631+0000 7f5aeb7fe640 1 --2- 172.21.15.169:0/515414959 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5aec1592d0 0x7f5aec0af740 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f5ae400cce0 tx=0x7f5ae400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:43.637 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:43.631+0000 7f5ad3fff640 1 -- 172.21.15.169:0/515414959 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae400f450 con 0x7f5aec1592d0 2023-10-26T01:31:43.638 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:43.631+0000 7f5ad3fff640 1 -- 172.21.15.169:0/515414959 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5ae400f6c0 con 0x7f5aec1592d0 2023-10-26T01:31:43.638 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:43.631+0000 7f5ad3fff640 1 -- 172.21.15.169:0/515414959 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae400fad0 con 0x7f5aec1592d0 2023-10-26T01:31:43.638 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:43.631+0000 7f5af1832640 1 -- 172.21.15.169:0/515414959 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5aec15b690 con 0x7f5aec1592d0 2023-10-26T01:31:43.638 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:43.631+0000 7f5af1832640 1 -- 172.21.15.169:0/515414959 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5aec15bc50 con 0x7f5aec1592d0 2023-10-26T01:31:43.639 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:43.633+0000 7f5af1832640 1 -- 172.21.15.169:0/515414959 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5aec04db10 con 0x7f5aec1592d0 2023-10-26T01:31:43.639 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:43.633+0000 7f5ad3fff640 1 -- 172.21.15.169:0/515414959 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f5ae400fc30 con 0x7f5aec1592d0 2023-10-26T01:31:43.639 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:43.634+0000 7f5ad3fff640 1 --2- 172.21.15.169:0/515414959 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5ad806ae40 0x7f5ad806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:43.639 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:43.634+0000 7f5ad3fff640 1 -- 172.21.15.169:0/515414959 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f5ae408a240 con 0x7f5aec1592d0 2023-10-26T01:31:43.639 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:43.635+0000 7f5aeaffd640 1 --2- 172.21.15.169:0/515414959 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5ad806ae40 0x7f5ad806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:43.640 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:43.636+0000 7f5aeaffd640 1 --2- 172.21.15.169:0/515414959 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5ad806ae40 0x7f5ad806d300 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5adc001270 tx=0x7f5adc009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:43.641 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:43.638+0000 7f5ad3fff640 1 -- 172.21.15.169:0/515414959 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5ae4055300 con 0x7f5aec1592d0 2023-10-26T01:31:43.869 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:43.866+0000 7f5af1832640 1 -- 172.21.15.169:0/515414959 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5aec0662e0 con 0x7f5aec1592d0 2023-10-26T01:31:43.932 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:43.867+0000 7f5ad3fff640 1 -- 172.21.15.169:0/515414959 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4916 (secure 0 0 0) 0x7f5ae4058920 con 0x7f5aec1592d0 2023-10-26T01:31:43.933 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:31:43.933 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4298,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1809801299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":1809801299},{"type":"v1","addr":"172.21.15.169:6835","nonce":1809801299}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4304,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/2248566141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2248566141},{"type":"v1","addr":"172.21.15.170:6836","nonce":2248566141}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4317,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/2383616530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2383616530},{"type":"v1","addr":"172.21.15.169:6834","nonce":2383616530}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T01:30:41.545068+0000","modified":"2023-10-26T01:30:44.577169+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4338},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4338":{"gid":4338,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6837/590609220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":590609220},{"type":"v1","addr":"172.21.15.170:6837","nonce":590609220}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-26T01:31:43.935 INFO:teuthology.orchestra.run.smithi169.stderr:dumped fsmap epoch 7 2023-10-26T01:31:43.935 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:43.870+0000 7f5af1832640 1 -- 172.21.15.169:0/515414959 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5ad806ae40 msgr2=0x7f5ad806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:43.935 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:43.870+0000 7f5af1832640 1 --2- 172.21.15.169:0/515414959 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5ad806ae40 0x7f5ad806d300 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5adc001270 tx=0x7f5adc009040 comp rx=0 tx=0).stop 2023-10-26T01:31:43.935 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:43.870+0000 7f5af1832640 1 -- 172.21.15.169:0/515414959 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5aec1592d0 msgr2=0x7f5aec0af740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:43.936 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:43.870+0000 7f5af1832640 1 --2- 172.21.15.169:0/515414959 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5aec1592d0 0x7f5aec0af740 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f5ae400cce0 tx=0x7f5ae400a620 comp rx=0 tx=0).stop 2023-10-26T01:31:43.936 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:43.870+0000 7f5af1832640 1 -- 172.21.15.169:0/515414959 shutdown_connections 2023-10-26T01:31:43.936 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:43.870+0000 7f5af1832640 1 --2- 172.21.15.169:0/515414959 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5ad806ae40 0x7f5ad806d300 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:43.936 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:43.870+0000 7f5af1832640 1 --2- 172.21.15.169:0/515414959 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5aec1592d0 0x7f5aec0af740 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:43.936 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:43.870+0000 7f5af1832640 1 --2- 172.21.15.169:0/515414959 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5aec0cdc30 0x7f5aec148610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:43.937 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:43.870+0000 7f5af1832640 1 --2- 172.21.15.169:0/515414959 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5aec0ac930 0x7f5aec15c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:43.937 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:43.870+0000 7f5af1832640 1 -- 172.21.15.169:0/515414959 >> 172.21.15.169:0/515414959 conn(0x7f5aec069930 msgr2=0x7f5aec0cc840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:43.937 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:43.871+0000 7f5af1832640 1 -- 172.21.15.169:0/515414959 shutdown_connections 2023-10-26T01:31:43.938 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:43.871+0000 7f5af1832640 1 -- 172.21.15.169:0/515414959 wait complete. 2023-10-26T01:31:43.938 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:31:44.163 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:44.160+0000 7f970dcb4640 1 -- 172.21.15.169:0/3327514970 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f970806ce20 msgr2=0x7f970814e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:44.163 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:44.160+0000 7f970dcb4640 1 --2- 172.21.15.169:0/3327514970 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f970806ce20 0x7f970814e790 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f970806eaf0 tx=0x7f96f802bdb0 comp rx=0 tx=0).stop 2023-10-26T01:31:44.163 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:44.160+0000 7f970dcb4640 1 -- 172.21.15.169:0/3327514970 shutdown_connections 2023-10-26T01:31:44.164 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:44.160+0000 7f970dcb4640 1 --2- 172.21.15.169:0/3327514970 >> v2:172.21.15.170:3300/0 conn(0x7f9708158850 0x7f97080aab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:44.164 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:44.160+0000 7f970dcb4640 1 --2- 172.21.15.169:0/3327514970 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f970806ce20 0x7f970814e790 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:44.164 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:44.160+0000 7f970dcb4640 1 -- 172.21.15.169:0/3327514970 >> 172.21.15.169:0/3327514970 conn(0x7f9708069930 msgr2=0x7f9708069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:44.164 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:44.160+0000 7f970dcb4640 1 -- 172.21.15.169:0/3327514970 shutdown_connections 2023-10-26T01:31:44.165 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:44.160+0000 7f970dcb4640 1 -- 172.21.15.169:0/3327514970 wait complete. 2023-10-26T01:31:44.166 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:44.162+0000 7f970dcb4640 1 Processor -- start 2023-10-26T01:31:44.166 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:44.162+0000 7f970dcb4640 1 -- start start 2023-10-26T01:31:44.166 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:44.163+0000 7f970dcb4640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f970806ad30 0x7f9708071620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:44.166 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:44.163+0000 7f970dcb4640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f970806ce20 0x7f9708157150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:44.167 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:44.163+0000 7f970dcb4640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9708158850 0x7f9708152300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:44.167 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:44.163+0000 7f970dcb4640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9708071c70 con 0x7f9708158850 2023-10-26T01:31:44.167 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:44.163+0000 7f970dcb4640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9708156390 con 0x7f970806ce20 2023-10-26T01:31:44.167 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:44.163+0000 7f970dcb4640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9708156670 con 0x7f970806ad30 2023-10-26T01:31:44.168 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:44.163+0000 7f9707fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9708158850 0x7f9708152300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:44.168 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:44.163+0000 7f9707fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9708158850 0x7f9708152300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:35668/0 (socket says 172.21.15.169:35668) 2023-10-26T01:31:44.168 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:44.163+0000 7f9707fff640 1 -- 172.21.15.169:0/3611136550 learned_addr learned my addr 172.21.15.169:0/3611136550 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:31:44.170 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:44.164+0000 7f9706ffd640 1 --2- 172.21.15.169:0/3611136550 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f970806ce20 0x7f9708157150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:44.170 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:44.164+0000 7f97077fe640 1 --2- 172.21.15.169:0/3611136550 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f970806ad30 0x7f9708071620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:44.170 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:44.164+0000 7f9707fff640 1 -- 172.21.15.169:0/3611136550 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f970806ad30 msgr2=0x7f9708071620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:44.171 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:44.164+0000 7f9707fff640 1 --2- 172.21.15.169:0/3611136550 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f970806ad30 0x7f9708071620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:44.171 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:44.164+0000 7f9707fff640 1 -- 172.21.15.169:0/3611136550 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f970806ce20 msgr2=0x7f9708157150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:44.171 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:44.164+0000 7f9707fff640 1 --2- 172.21.15.169:0/3611136550 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f970806ce20 0x7f9708157150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:44.171 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:44.164+0000 7f9707fff640 1 -- 172.21.15.169:0/3611136550 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96f8009560 con 0x7f9708158850 2023-10-26T01:31:44.171 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:44.164+0000 7f9706ffd640 1 --2- 172.21.15.169:0/3611136550 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f970806ce20 0x7f9708157150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:31:44.172 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:44.164+0000 7f9707fff640 1 --2- 172.21.15.169:0/3611136550 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9708158850 0x7f9708152300 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f9700001560 tx=0x7f970000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:44.172 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:44.164+0000 7f96effff640 1 -- 172.21.15.169:0/3611136550 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9700003da0 con 0x7f9708158850 2023-10-26T01:31:44.172 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:44.164+0000 7f96effff640 1 -- 172.21.15.169:0/3611136550 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9700005070 con 0x7f9708158850 2023-10-26T01:31:44.172 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:44.164+0000 7f970dcb4640 1 -- 172.21.15.169:0/3611136550 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9708156950 con 0x7f9708158850 2023-10-26T01:31:44.172 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:44.164+0000 7f96effff640 1 -- 172.21.15.169:0/3611136550 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9700005460 con 0x7f9708158850 2023-10-26T01:31:44.173 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:44.165+0000 7f970dcb4640 1 -- 172.21.15.169:0/3611136550 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f970814f9d0 con 0x7f9708158850 2023-10-26T01:31:44.173 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:44.166+0000 7f970dcb4640 1 -- 172.21.15.169:0/3611136550 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9708156e70 con 0x7f9708158850 2023-10-26T01:31:44.173 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:44.167+0000 7f96effff640 1 -- 172.21.15.169:0/3611136550 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f97000055c0 con 0x7f9708158850 2023-10-26T01:31:44.174 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:44.167+0000 7f96effff640 1 --2- 172.21.15.169:0/3611136550 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f96f406ae40 0x7f96f406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:44.174 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:44.168+0000 7f96effff640 1 -- 172.21.15.169:0/3611136550 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f9700089230 con 0x7f9708158850 2023-10-26T01:31:44.175 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:44.168+0000 7f97077fe640 1 --2- 172.21.15.169:0/3611136550 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f96f406ae40 0x7f96f406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:44.175 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:44.169+0000 7f97077fe640 1 --2- 172.21.15.169:0/3611136550 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f96f406ae40 0x7f96f406d300 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f96f8035000 tx=0x7f96f8004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:44.176 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:44.172+0000 7f96effff640 1 -- 172.21.15.169:0/3611136550 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9708156e70 con 0x7f9708158850 2023-10-26T01:31:44.365 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:44.362+0000 7f970dcb4640 1 -- 172.21.15.169:0/3611136550 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9708156e70 con 0x7f9708158850 2023-10-26T01:31:44.367 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:44.365+0000 7f96effff640 1 -- 172.21.15.169:0/3611136550 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16801 (secure 0 0 0) 0x7f9708156e70 con 0x7f9708158850 2023-10-26T01:31:44.368 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:31:44.369 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":23,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:31:30.100039+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-26T01:30:39.437631+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-26T01:30:40.561774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":2}]},{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.24","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:31:44.371 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:44.367+0000 7f970dcb4640 1 -- 172.21.15.169:0/3611136550 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f96f406ae40 msgr2=0x7f96f406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:44.371 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:44.367+0000 7f970dcb4640 1 --2- 172.21.15.169:0/3611136550 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f96f406ae40 0x7f96f406d300 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f96f8035000 tx=0x7f96f8004730 comp rx=0 tx=0).stop 2023-10-26T01:31:44.372 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:44.367+0000 7f970dcb4640 1 -- 172.21.15.169:0/3611136550 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9708158850 msgr2=0x7f9708152300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:44.372 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:44.367+0000 7f970dcb4640 1 --2- 172.21.15.169:0/3611136550 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9708158850 0x7f9708152300 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f9700001560 tx=0x7f970000cc70 comp rx=0 tx=0).stop 2023-10-26T01:31:44.372 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:44.368+0000 7f970dcb4640 1 -- 172.21.15.169:0/3611136550 shutdown_connections 2023-10-26T01:31:44.373 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:44.368+0000 7f970dcb4640 1 --2- 172.21.15.169:0/3611136550 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f96f406ae40 0x7f96f406d300 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:44.373 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:44.368+0000 7f970dcb4640 1 --2- 172.21.15.169:0/3611136550 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9708158850 0x7f9708152300 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:44.374 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:44.368+0000 7f970dcb4640 1 --2- 172.21.15.169:0/3611136550 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f970806ce20 0x7f9708157150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:44.374 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:44.368+0000 7f970dcb4640 1 --2- 172.21.15.169:0/3611136550 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f970806ad30 0x7f9708071620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:44.374 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:44.368+0000 7f970dcb4640 1 -- 172.21.15.169:0/3611136550 >> 172.21.15.169:0/3611136550 conn(0x7f9708069930 msgr2=0x7f970814d2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:44.375 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:44.368+0000 7f970dcb4640 1 -- 172.21.15.169:0/3611136550 shutdown_connections 2023-10-26T01:31:44.375 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:44.368+0000 7f970dcb4640 1 -- 172.21.15.169:0/3611136550 wait complete. 2023-10-26T01:31:44.388 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-26T01:31:44.609 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:44.606+0000 7fe1da116640 1 -- 172.21.15.169:0/1244222170 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe1d41593e0 msgr2=0x7fe1d40ac200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:44.609 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:44.606+0000 7fe1da116640 1 --2- 172.21.15.169:0/1244222170 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe1d41593e0 0x7fe1d40ac200 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fe1d40c9aa0 tx=0x7fe1c402bda0 comp rx=0 tx=0).stop 2023-10-26T01:31:44.610 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:44.606+0000 7fe1da116640 1 -- 172.21.15.169:0/1244222170 shutdown_connections 2023-10-26T01:31:44.610 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:44.606+0000 7fe1da116640 1 --2- 172.21.15.169:0/1244222170 >> v2:172.21.15.170:3300/0 conn(0x7fe1d40b0c80 0x7fe1d40cd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:44.610 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:44.606+0000 7fe1da116640 1 --2- 172.21.15.169:0/1244222170 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe1d41593e0 0x7fe1d40ac200 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:44.610 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:44.606+0000 7fe1da116640 1 -- 172.21.15.169:0/1244222170 >> 172.21.15.169:0/1244222170 conn(0x7fe1d4069930 msgr2=0x7fe1d4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:44.610 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:44.606+0000 7fe1da116640 1 -- 172.21.15.169:0/1244222170 shutdown_connections 2023-10-26T01:31:44.611 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:44.606+0000 7fe1da116640 1 -- 172.21.15.169:0/1244222170 wait complete. 2023-10-26T01:31:44.612 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:44.608+0000 7fe1da116640 1 Processor -- start 2023-10-26T01:31:44.612 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:44.608+0000 7fe1da116640 1 -- start start 2023-10-26T01:31:44.613 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:44.609+0000 7fe1da116640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe1d40b0c80 0x7fe1d40af5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:44.613 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:44.609+0000 7fe1da116640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe1d41593e0 0x7fe1d40acf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:44.613 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:44.609+0000 7fe1da116640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe1d41597c0 0x7fe1d414d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:44.613 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:44.609+0000 7fe1da116640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d40ad440 con 0x7fe1d40b0c80 2023-10-26T01:31:44.614 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:44.609+0000 7fe1da116640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d40ae280 con 0x7fe1d41597c0 2023-10-26T01:31:44.614 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:44.609+0000 7fe1da116640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d414ddd0 con 0x7fe1d41593e0 2023-10-26T01:31:44.614 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:44.610+0000 7fe1d37fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe1d40b0c80 0x7fe1d40af5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:44.614 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:44.610+0000 7fe1d37fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe1d40b0c80 0x7fe1d40af5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:35684/0 (socket says 172.21.15.169:35684) 2023-10-26T01:31:44.614 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:44.610+0000 7fe1d37fe640 1 -- 172.21.15.169:0/239275790 learned_addr learned my addr 172.21.15.169:0/239275790 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:31:44.615 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:44.610+0000 7fe1d3fff640 1 --2- 172.21.15.169:0/239275790 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe1d41597c0 0x7fe1d414d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:44.615 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:44.610+0000 7fe1cbfff640 1 --2- 172.21.15.169:0/239275790 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe1d41593e0 0x7fe1d40acf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:44.616 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:44.610+0000 7fe1d37fe640 1 -- 172.21.15.169:0/239275790 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe1d41593e0 msgr2=0x7fe1d40acf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:44.617 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:44.610+0000 7fe1d37fe640 1 --2- 172.21.15.169:0/239275790 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe1d41593e0 0x7fe1d40acf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:44.617 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:44.610+0000 7fe1d37fe640 1 -- 172.21.15.169:0/239275790 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe1d41597c0 msgr2=0x7fe1d414d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:44.617 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:44.610+0000 7fe1d37fe640 1 --2- 172.21.15.169:0/239275790 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe1d41597c0 0x7fe1d414d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:44.617 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:44.610+0000 7fe1d37fe640 1 -- 172.21.15.169:0/239275790 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1c4009560 con 0x7fe1d40b0c80 2023-10-26T01:31:44.617 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:44.611+0000 7fe1d37fe640 1 --2- 172.21.15.169:0/239275790 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe1d40b0c80 0x7fe1d40af5a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe1c4009850 tx=0x7fe1c402dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:44.618 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:44.611+0000 7fe1d0ff9640 1 -- 172.21.15.169:0/239275790 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1c4005650 con 0x7fe1d40b0c80 2023-10-26T01:31:44.618 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:44.611+0000 7fe1d0ff9640 1 -- 172.21.15.169:0/239275790 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe1c40038f0 con 0x7fe1d40b0c80 2023-10-26T01:31:44.618 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:44.611+0000 7fe1da116640 1 -- 172.21.15.169:0/239275790 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1d40ad670 con 0x7fe1d40b0c80 2023-10-26T01:31:44.618 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:44.611+0000 7fe1d0ff9640 1 -- 172.21.15.169:0/239275790 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1c4003ce0 con 0x7fe1d40b0c80 2023-10-26T01:31:44.619 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:44.611+0000 7fe1da116640 1 -- 172.21.15.169:0/239275790 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1d40adc30 con 0x7fe1d40b0c80 2023-10-26T01:31:44.619 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:44.612+0000 7fe1da116640 1 -- 172.21.15.169:0/239275790 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1d404db10 con 0x7fe1d40b0c80 2023-10-26T01:31:44.619 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:44.613+0000 7fe1d0ff9640 1 -- 172.21.15.169:0/239275790 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fe1c403e450 con 0x7fe1d40b0c80 2023-10-26T01:31:44.619 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:44.614+0000 7fe1d0ff9640 1 --2- 172.21.15.169:0/239275790 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe1b406ae40 0x7fe1b406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:44.619 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:44.614+0000 7fe1d0ff9640 1 -- 172.21.15.169:0/239275790 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fe1c40abfe0 con 0x7fe1d40b0c80 2023-10-26T01:31:44.620 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:44.615+0000 7fe1cbfff640 1 --2- 172.21.15.169:0/239275790 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe1b406ae40 0x7fe1b406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:44.620 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:44.616+0000 7fe1cbfff640 1 --2- 172.21.15.169:0/239275790 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe1b406ae40 0x7fe1b406d300 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc006fd0 tx=0x7fe1bc008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:44.621 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:44.618+0000 7fe1d0ff9640 1 -- 172.21.15.169:0/239275790 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe1c4077040 con 0x7fe1d40b0c80 2023-10-26T01:31:44.852 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:44.849+0000 7fe1da116640 1 -- 172.21.15.169:0/239275790 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fe1d40b2170 con 0x7fe1d40b0c80 2023-10-26T01:31:44.853 INFO:tasks.ceph.mon.a.smithi169.stderr:2023-10-26T01:31:44.850+0000 7f1588a74640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-26T01:31:44.856 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:44.853+0000 7fe1d0ff9640 1 -- 172.21.15.169:0/239275790 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v8) v1 ==== 161+0+0 (secure 0 0 0) 0x7fe1c407a660 con 0x7fe1d40b0c80 2023-10-26T01:31:44.856 INFO:teuthology.orchestra.run.smithi169.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-26T01:31:44.860 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:44.857+0000 7fe1da116640 1 -- 172.21.15.169:0/239275790 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe1b406ae40 msgr2=0x7fe1b406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:44.860 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:44.857+0000 7fe1da116640 1 --2- 172.21.15.169:0/239275790 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe1b406ae40 0x7fe1b406d300 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc006fd0 tx=0x7fe1bc008040 comp rx=0 tx=0).stop 2023-10-26T01:31:44.860 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:44.857+0000 7fe1da116640 1 -- 172.21.15.169:0/239275790 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe1d40b0c80 msgr2=0x7fe1d40af5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:44.861 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:44.857+0000 7fe1da116640 1 --2- 172.21.15.169:0/239275790 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe1d40b0c80 0x7fe1d40af5a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe1c4009850 tx=0x7fe1c402dd70 comp rx=0 tx=0).stop 2023-10-26T01:31:44.861 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:44.857+0000 7fe1da116640 1 -- 172.21.15.169:0/239275790 shutdown_connections 2023-10-26T01:31:44.861 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:44.857+0000 7fe1da116640 1 --2- 172.21.15.169:0/239275790 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe1b406ae40 0x7fe1b406d300 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:44.861 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:44.857+0000 7fe1da116640 1 --2- 172.21.15.169:0/239275790 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe1d41597c0 0x7fe1d414d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:44.861 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:44.857+0000 7fe1da116640 1 --2- 172.21.15.169:0/239275790 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe1d41593e0 0x7fe1d40acf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:44.862 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:44.857+0000 7fe1da116640 1 --2- 172.21.15.169:0/239275790 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe1d40b0c80 0x7fe1d40af5a0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:44.862 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:44.857+0000 7fe1da116640 1 -- 172.21.15.169:0/239275790 >> 172.21.15.169:0/239275790 conn(0x7fe1d4069930 msgr2=0x7fe1d415a590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:44.863 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:44.857+0000 7fe1da116640 1 -- 172.21.15.169:0/239275790 shutdown_connections 2023-10-26T01:31:44.863 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:44.858+0000 7fe1da116640 1 -- 172.21.15.169:0/239275790 wait complete. 2023-10-26T01:31:44.878 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-26T01:31:44.890 INFO:tasks.ceph.mds.b.smithi170.stderr: -8751> 2023-10-26T01:30:36.557+0000 7fa22e4bd640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2023-10-26T01:31:44.897 INFO:tasks.ceph.mds.b.smithi170.stderr: -7510> 2023-10-26T01:30:47.558+0000 7fa2284b1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-26T01:31:44.975 INFO:tasks.ceph.mds.b.smithi170.stdout:starting mds.b at 2023-10-26T01:31:45.096 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:45.094+0000 7f01163ae640 1 -- 172.21.15.169:0/3350728285 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0110157f80 msgr2=0x7f011015b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:45.097 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:45.094+0000 7f01163ae640 1 --2- 172.21.15.169:0/3350728285 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0110157f80 0x7f011015b140 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f011006c280 tx=0x7f010002bd10 comp rx=0 tx=0).stop 2023-10-26T01:31:45.097 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:45.095+0000 7f01163ae640 1 -- 172.21.15.169:0/3350728285 shutdown_connections 2023-10-26T01:31:45.097 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:45.095+0000 7f01163ae640 1 --2- 172.21.15.169:0/3350728285 >> v2:172.21.15.170:3300/0 conn(0x7f01100aac10 0x7f01100b0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:45.097 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:45.095+0000 7f01163ae640 1 --2- 172.21.15.169:0/3350728285 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0110158360 0x7f01100aa6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:45.098 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:45.095+0000 7f01163ae640 1 --2- 172.21.15.169:0/3350728285 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0110157f80 0x7f011015b140 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:45.098 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:45.095+0000 7f01163ae640 1 -- 172.21.15.169:0/3350728285 >> 172.21.15.169:0/3350728285 conn(0x7f01100af110 msgr2=0x7f01100af520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:45.098 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:45.095+0000 7f01163ae640 1 -- 172.21.15.169:0/3350728285 shutdown_connections 2023-10-26T01:31:45.098 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:45.095+0000 7f01163ae640 1 -- 172.21.15.169:0/3350728285 wait complete. 2023-10-26T01:31:45.099 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:45.097+0000 7f01163ae640 1 Processor -- start 2023-10-26T01:31:45.099 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:45.097+0000 7f01163ae640 1 -- start start 2023-10-26T01:31:45.101 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:45.098+0000 7f01163ae640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f01100aac10 0x7f0110150880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:45.101 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:45.098+0000 7f01163ae640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0110157f80 0x7f0110070180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:45.101 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:45.098+0000 7f01163ae640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0110158360 0x7f01100706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:45.101 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:45.098+0000 7f01163ae640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0110150dc0 con 0x7f0110157f80 2023-10-26T01:31:45.102 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:45.098+0000 7f01163ae640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01100cb820 con 0x7f0110158360 2023-10-26T01:31:45.102 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:45.098+0000 7f01163ae640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01100cbb00 con 0x7f01100aac10 2023-10-26T01:31:45.102 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:45.098+0000 7f0114bab640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0110157f80 0x7f0110070180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:45.102 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:45.098+0000 7f0114bab640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0110157f80 0x7f0110070180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:35710/0 (socket says 172.21.15.169:35710) 2023-10-26T01:31:45.102 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:45.098+0000 7f0114bab640 1 -- 172.21.15.169:0/1759611387 learned_addr learned my addr 172.21.15.169:0/1759611387 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:31:45.103 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:45.099+0000 7f0114bab640 1 -- 172.21.15.169:0/1759611387 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f01100aac10 msgr2=0x7f0110150880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:45.103 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:45.099+0000 7f01153ac640 1 --2- 172.21.15.169:0/1759611387 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f01100aac10 0x7f0110150880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:45.103 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:45.099+0000 7f0115bad640 1 --2- 172.21.15.169:0/1759611387 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0110158360 0x7f01100706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:45.103 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:45.099+0000 7f0114bab640 1 --2- 172.21.15.169:0/1759611387 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f01100aac10 0x7f0110150880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:45.103 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:45.099+0000 7f0114bab640 1 -- 172.21.15.169:0/1759611387 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0110158360 msgr2=0x7f01100706c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:45.104 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:45.099+0000 7f0114bab640 1 --2- 172.21.15.169:0/1759611387 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0110158360 0x7f01100706c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:45.104 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:45.099+0000 7f0114bab640 1 -- 172.21.15.169:0/1759611387 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0100009df0 con 0x7f0110157f80 2023-10-26T01:31:45.104 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:45.099+0000 7f0114bab640 1 --2- 172.21.15.169:0/1759611387 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0110157f80 0x7f0110070180 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f010c00de60 tx=0x7f010c00b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:45.104 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:45.099+0000 7f0105ffb640 1 -- 172.21.15.169:0/1759611387 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f010c00e860 con 0x7f0110157f80 2023-10-26T01:31:45.104 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:45.099+0000 7f0105ffb640 1 -- 172.21.15.169:0/1759611387 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f010c00eab0 con 0x7f0110157f80 2023-10-26T01:31:45.105 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:45.099+0000 7f0105ffb640 1 -- 172.21.15.169:0/1759611387 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f010c00ee80 con 0x7f0110157f80 2023-10-26T01:31:45.105 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:45.099+0000 7f01163ae640 1 -- 172.21.15.169:0/1759611387 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01100cbde0 con 0x7f0110157f80 2023-10-26T01:31:45.105 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:45.100+0000 7f01163ae640 1 -- 172.21.15.169:0/1759611387 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01100cc3a0 con 0x7f0110157f80 2023-10-26T01:31:45.106 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:45.101+0000 7f01163ae640 1 -- 172.21.15.169:0/1759611387 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01100cc560 con 0x7f0110157f80 2023-10-26T01:31:45.106 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:45.102+0000 7f0105ffb640 1 -- 172.21.15.169:0/1759611387 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f010c016430 con 0x7f0110157f80 2023-10-26T01:31:45.106 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:45.103+0000 7f0105ffb640 1 --2- 172.21.15.169:0/1759611387 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f00fc06ae40 0x7f00fc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:45.106 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:45.103+0000 7f0105ffb640 1 -- 172.21.15.169:0/1759611387 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5911+0+0 (secure 0 0 0) 0x7f010c08a530 con 0x7f0110157f80 2023-10-26T01:31:45.107 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:45.104+0000 7f01153ac640 1 --2- 172.21.15.169:0/1759611387 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f00fc06ae40 0x7f00fc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:45.107 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:45.105+0000 7f01153ac640 1 --2- 172.21.15.169:0/1759611387 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f00fc06ae40 0x7f00fc06d300 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0100035000 tx=0x7f0100004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:45.109 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:45.108+0000 7f0105ffb640 1 -- 172.21.15.169:0/1759611387 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f010c055500 con 0x7f0110157f80 2023-10-26T01:31:45.340 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:45.338+0000 7f01163ae640 1 -- 172.21.15.169:0/1759611387 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f011006ce10 con 0x7f0110157f80 2023-10-26T01:31:45.860 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:45.858+0000 7f0105ffb640 1 -- 172.21.15.169:0/1759611387 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v9) v1 ==== 104+0+0 (secure 0 0 0) 0x7f010c058b20 con 0x7f0110157f80 2023-10-26T01:31:45.864 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:45.861+0000 7f01163ae640 1 -- 172.21.15.169:0/1759611387 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f00fc06ae40 msgr2=0x7f00fc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:45.864 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:45.861+0000 7f01163ae640 1 --2- 172.21.15.169:0/1759611387 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f00fc06ae40 0x7f00fc06d300 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0100035000 tx=0x7f0100004730 comp rx=0 tx=0).stop 2023-10-26T01:31:45.865 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:45.861+0000 7f01163ae640 1 -- 172.21.15.169:0/1759611387 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0110157f80 msgr2=0x7f0110070180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:45.865 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:45.861+0000 7f01163ae640 1 --2- 172.21.15.169:0/1759611387 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0110157f80 0x7f0110070180 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f010c00de60 tx=0x7f010c00b730 comp rx=0 tx=0).stop 2023-10-26T01:31:45.865 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:45.862+0000 7f01163ae640 1 -- 172.21.15.169:0/1759611387 shutdown_connections 2023-10-26T01:31:45.865 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:45.862+0000 7f01163ae640 1 --2- 172.21.15.169:0/1759611387 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f00fc06ae40 0x7f00fc06d300 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:45.866 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:45.862+0000 7f01163ae640 1 --2- 172.21.15.169:0/1759611387 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0110157f80 0x7f0110070180 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:45.866 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:45.862+0000 7f01163ae640 1 --2- 172.21.15.169:0/1759611387 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0110158360 0x7f01100706c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:45.866 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:45.862+0000 7f01163ae640 1 --2- 172.21.15.169:0/1759611387 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f01100aac10 0x7f0110150880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:45.867 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:45.862+0000 7f01163ae640 1 -- 172.21.15.169:0/1759611387 >> 172.21.15.169:0/1759611387 conn(0x7f01100af110 msgr2=0x7f0110151520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:45.867 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:45.862+0000 7f01163ae640 1 -- 172.21.15.169:0/1759611387 shutdown_connections 2023-10-26T01:31:45.867 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:45.862+0000 7f01163ae640 1 -- 172.21.15.169:0/1759611387 wait complete. 2023-10-26T01:31:45.884 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-26T01:31:46.103 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:46.101+0000 7f69e097c640 1 -- 172.21.15.169:0/1482198929 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f69dc06cae0 msgr2=0x7f69dc15d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:46.103 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:46.101+0000 7f69e097c640 1 --2- 172.21.15.169:0/1482198929 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f69dc06cae0 0x7f69dc15d130 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f69dc04ea40 tx=0x7f69d402c250 comp rx=0 tx=0).stop 2023-10-26T01:31:46.103 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:46.101+0000 7f69e097c640 1 -- 172.21.15.169:0/1482198929 shutdown_connections 2023-10-26T01:31:46.103 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:46.101+0000 7f69e097c640 1 --2- 172.21.15.169:0/1482198929 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f69dc06cae0 0x7f69dc15d130 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:46.104 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:46.101+0000 7f69e097c640 1 --2- 172.21.15.169:0/1482198929 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f69dc065490 0x7f69dc06c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:46.104 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:46.101+0000 7f69e097c640 1 --2- 172.21.15.169:0/1482198929 >> v2:172.21.15.170:3300/0 conn(0x7f69dc1596d0 0x7f69dc071170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:46.104 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:46.101+0000 7f69e097c640 1 -- 172.21.15.169:0/1482198929 >> 172.21.15.169:0/1482198929 conn(0x7f69dc0af650 msgr2=0x7f69dc0afa30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:46.104 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:46.101+0000 7f69e097c640 1 -- 172.21.15.169:0/1482198929 shutdown_connections 2023-10-26T01:31:46.105 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:46.101+0000 7f69e097c640 1 -- 172.21.15.169:0/1482198929 wait complete. 2023-10-26T01:31:46.106 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:46.103+0000 7f69e097c640 1 Processor -- start 2023-10-26T01:31:46.106 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:46.103+0000 7f69e097c640 1 -- start start 2023-10-26T01:31:46.106 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:46.104+0000 7f69e097c640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f69dc065490 0x7f69dc157110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:46.106 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:46.104+0000 7f69e097c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f69dc06cae0 0x7f69dc14a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:46.107 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:46.104+0000 7f69e097c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f69dc1596d0 0x7f69dc0c4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:46.107 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:46.104+0000 7f69e097c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69dc156070 con 0x7f69dc06cae0 2023-10-26T01:31:46.107 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:46.104+0000 7f69e097c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69dc156350 con 0x7f69dc1596d0 2023-10-26T01:31:46.107 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:46.104+0000 7f69e097c640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69dc156630 con 0x7f69dc065490 2023-10-26T01:31:46.107 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:46.104+0000 7f69d259c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f69dc06cae0 0x7f69dc14a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:46.108 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:46.104+0000 7f69d259c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f69dc06cae0 0x7f69dc14a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:35728/0 (socket says 172.21.15.169:35728) 2023-10-26T01:31:46.108 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:46.104+0000 7f69d259c640 1 -- 172.21.15.169:0/1518480083 learned_addr learned my addr 172.21.15.169:0/1518480083 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:31:46.109 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:46.105+0000 7f69dad9d640 1 --2- 172.21.15.169:0/1518480083 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f69dc065490 0x7f69dc157110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:46.109 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:46.105+0000 7f69db59e640 1 --2- 172.21.15.169:0/1518480083 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f69dc1596d0 0x7f69dc0c4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:46.109 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:46.105+0000 7f69d259c640 1 -- 172.21.15.169:0/1518480083 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f69dc065490 msgr2=0x7f69dc157110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:46.109 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:46.105+0000 7f69d259c640 1 --2- 172.21.15.169:0/1518480083 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f69dc065490 0x7f69dc157110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:46.110 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:46.105+0000 7f69d259c640 1 -- 172.21.15.169:0/1518480083 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f69dc1596d0 msgr2=0x7f69dc0c4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:46.110 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:46.105+0000 7f69d259c640 1 --2- 172.21.15.169:0/1518480083 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f69dc1596d0 0x7f69dc0c4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:46.110 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:46.105+0000 7f69d259c640 1 -- 172.21.15.169:0/1518480083 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69d4009900 con 0x7f69dc06cae0 2023-10-26T01:31:46.110 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:46.105+0000 7f69d259c640 1 --2- 172.21.15.169:0/1518480083 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f69dc06cae0 0x7f69dc14a120 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f69c400d770 tx=0x7f69c400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:46.110 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:46.105+0000 7f69d3fff640 1 -- 172.21.15.169:0/1518480083 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69c4009c30 con 0x7f69dc06cae0 2023-10-26T01:31:46.111 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:46.105+0000 7f69d3fff640 1 -- 172.21.15.169:0/1518480083 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f69c4009e80 con 0x7f69dc06cae0 2023-10-26T01:31:46.111 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:46.105+0000 7f69e097c640 1 -- 172.21.15.169:0/1518480083 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69dc156910 con 0x7f69dc06cae0 2023-10-26T01:31:46.111 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:46.106+0000 7f69d3fff640 1 -- 172.21.15.169:0/1518480083 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69c40106a0 con 0x7f69dc06cae0 2023-10-26T01:31:46.111 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:46.106+0000 7f69e097c640 1 -- 172.21.15.169:0/1518480083 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69dc14a660 con 0x7f69dc06cae0 2023-10-26T01:31:46.111 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:46.107+0000 7f69e097c640 1 -- 172.21.15.169:0/1518480083 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69dc04e2c0 con 0x7f69dc06cae0 2023-10-26T01:31:46.112 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:46.108+0000 7f69d3fff640 1 -- 172.21.15.169:0/1518480083 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f69c4010800 con 0x7f69dc06cae0 2023-10-26T01:31:46.113 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:46.109+0000 7f69d3fff640 1 --2- 172.21.15.169:0/1518480083 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f69bc06ae40 0x7f69bc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:46.113 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:46.109+0000 7f69d3fff640 1 -- 172.21.15.169:0/1518480083 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5911+0+0 (secure 0 0 0) 0x7f69c4089ce0 con 0x7f69dc06cae0 2023-10-26T01:31:46.113 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:46.109+0000 7f69dad9d640 1 --2- 172.21.15.169:0/1518480083 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f69bc06ae40 0x7f69bc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:46.113 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:46.111+0000 7f69dad9d640 1 --2- 172.21.15.169:0/1518480083 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f69bc06ae40 0x7f69bc06d300 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f69cc005fd0 tx=0x7f69cc005850 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:46.115 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:46.114+0000 7f69d3fff640 1 -- 172.21.15.169:0/1518480083 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f69c40586c0 con 0x7f69dc06cae0 2023-10-26T01:31:46.307 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:46.305+0000 7f69e097c640 1 -- 172.21.15.169:0/1518480083 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f69dc0ca1a0 con 0x7f69dc06cae0 2023-10-26T01:31:47.217 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:47.216+0000 7f69d3fff640 1 -- 172.21.15.169:0/1518480083 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v25) v1 ==== 181+0+0 (secure 0 0 0) 0x7f69c4021020 con 0x7f69dc06cae0 2023-10-26T01:31:47.218 INFO:teuthology.orchestra.run.smithi169.stderr:pool 'cephfs_metadata' removed 2023-10-26T01:31:47.221 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:47.219+0000 7f69e097c640 1 -- 172.21.15.169:0/1518480083 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f69bc06ae40 msgr2=0x7f69bc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:47.221 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:47.219+0000 7f69e097c640 1 --2- 172.21.15.169:0/1518480083 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f69bc06ae40 0x7f69bc06d300 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f69cc005fd0 tx=0x7f69cc005850 comp rx=0 tx=0).stop 2023-10-26T01:31:47.221 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:47.219+0000 7f69e097c640 1 -- 172.21.15.169:0/1518480083 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f69dc06cae0 msgr2=0x7f69dc14a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:47.222 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:47.219+0000 7f69e097c640 1 --2- 172.21.15.169:0/1518480083 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f69dc06cae0 0x7f69dc14a120 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f69c400d770 tx=0x7f69c400dc30 comp rx=0 tx=0).stop 2023-10-26T01:31:47.222 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:47.219+0000 7f69e097c640 1 -- 172.21.15.169:0/1518480083 shutdown_connections 2023-10-26T01:31:47.222 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:47.219+0000 7f69e097c640 1 --2- 172.21.15.169:0/1518480083 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f69bc06ae40 0x7f69bc06d300 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:47.222 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:47.219+0000 7f69e097c640 1 --2- 172.21.15.169:0/1518480083 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f69dc06cae0 0x7f69dc14a120 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:47.223 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:47.219+0000 7f69e097c640 1 --2- 172.21.15.169:0/1518480083 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f69dc1596d0 0x7f69dc0c4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:47.223 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:47.219+0000 7f69e097c640 1 --2- 172.21.15.169:0/1518480083 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f69dc065490 0x7f69dc157110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:47.223 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:47.219+0000 7f69e097c640 1 -- 172.21.15.169:0/1518480083 >> 172.21.15.169:0/1518480083 conn(0x7f69dc0af650 msgr2=0x7f69dc15c480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:47.223 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:47.219+0000 7f69e097c640 1 -- 172.21.15.169:0/1518480083 shutdown_connections 2023-10-26T01:31:47.223 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:47.219+0000 7f69e097c640 1 -- 172.21.15.169:0/1518480083 wait complete. 2023-10-26T01:31:47.240 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-26T01:31:47.459 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:47.457+0000 7f5a58432640 1 -- 172.21.15.169:0/3049680541 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a5015b2c0 msgr2=0x7f5a5015b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:47.459 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:47.457+0000 7f5a58432640 1 --2- 172.21.15.169:0/3049680541 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a5015b2c0 0x7f5a5015b750 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f5a48005fd0 tx=0x7f5a4802c1b0 comp rx=0 tx=0).stop 2023-10-26T01:31:47.459 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:47.457+0000 7f5a58432640 1 -- 172.21.15.169:0/3049680541 shutdown_connections 2023-10-26T01:31:47.460 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:47.457+0000 7f5a58432640 1 --2- 172.21.15.169:0/3049680541 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a5015b2c0 0x7f5a5015b750 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:47.460 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:47.457+0000 7f5a58432640 1 --2- 172.21.15.169:0/3049680541 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5a500aab40 0x7f5a500ac6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:47.460 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:47.457+0000 7f5a58432640 1 -- 172.21.15.169:0/3049680541 >> 172.21.15.169:0/3049680541 conn(0x7f5a500aeee0 msgr2=0x7f5a500af2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:47.460 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:47.457+0000 7f5a58432640 1 -- 172.21.15.169:0/3049680541 shutdown_connections 2023-10-26T01:31:47.461 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:47.457+0000 7f5a58432640 1 -- 172.21.15.169:0/3049680541 wait complete. 2023-10-26T01:31:47.463 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:47.459+0000 7f5a58432640 1 Processor -- start 2023-10-26T01:31:47.463 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:47.459+0000 7f5a58432640 1 -- start start 2023-10-26T01:31:47.463 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:47.460+0000 7f5a58432640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5a501577a0 0x7f5a500cdac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:47.464 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:47.460+0000 7f5a58432640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a5015b2c0 0x7f5a50150ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:47.464 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:47.460+0000 7f5a58432640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5a50151400 0x7f5a501551a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:47.464 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:47.460+0000 7f5a58432640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a500ce000 con 0x7f5a5015b2c0 2023-10-26T01:31:47.464 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:47.460+0000 7f5a58432640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a501518a0 con 0x7f5a501577a0 2023-10-26T01:31:47.464 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:47.460+0000 7f5a58432640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a50151b80 con 0x7f5a50151400 2023-10-26T01:31:47.465 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:47.460+0000 7f5a559cd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a5015b2c0 0x7f5a50150ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:47.465 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:47.460+0000 7f5a559cd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a5015b2c0 0x7f5a50150ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:35740/0 (socket says 172.21.15.169:35740) 2023-10-26T01:31:47.465 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:47.460+0000 7f5a559cd640 1 -- 172.21.15.169:0/516676648 learned_addr learned my addr 172.21.15.169:0/516676648 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:31:47.465 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:47.461+0000 7f5a561ce640 1 --2- 172.21.15.169:0/516676648 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5a501577a0 0x7f5a500cdac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:47.465 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:47.461+0000 7f5a569cf640 1 --2- 172.21.15.169:0/516676648 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5a50151400 0x7f5a501551a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:47.466 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:47.461+0000 7f5a559cd640 1 -- 172.21.15.169:0/516676648 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5a50151400 msgr2=0x7f5a501551a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:47.466 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:47.461+0000 7f5a559cd640 1 --2- 172.21.15.169:0/516676648 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5a50151400 0x7f5a501551a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:47.466 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:47.461+0000 7f5a559cd640 1 -- 172.21.15.169:0/516676648 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5a501577a0 msgr2=0x7f5a500cdac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:47.466 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:47.461+0000 7f5a559cd640 1 --2- 172.21.15.169:0/516676648 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5a501577a0 0x7f5a500cdac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:47.466 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:47.461+0000 7f5a559cd640 1 -- 172.21.15.169:0/516676648 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a48009900 con 0x7f5a5015b2c0 2023-10-26T01:31:47.467 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:47.461+0000 7f5a561ce640 1 --2- 172.21.15.169:0/516676648 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5a501577a0 0x7f5a500cdac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:31:47.467 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:47.461+0000 7f5a559cd640 1 --2- 172.21.15.169:0/516676648 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a5015b2c0 0x7f5a50150ec0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f5a4000c970 tx=0x7f5a4000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:47.467 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:47.461+0000 7f5a46ffd640 1 -- 172.21.15.169:0/516676648 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a40004440 con 0x7f5a5015b2c0 2023-10-26T01:31:47.467 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:47.461+0000 7f5a46ffd640 1 -- 172.21.15.169:0/516676648 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5a40007950 con 0x7f5a5015b2c0 2023-10-26T01:31:47.467 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:47.461+0000 7f5a46ffd640 1 -- 172.21.15.169:0/516676648 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a40007d60 con 0x7f5a5015b2c0 2023-10-26T01:31:47.468 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:47.461+0000 7f5a58432640 1 -- 172.21.15.169:0/516676648 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a50151e60 con 0x7f5a5015b2c0 2023-10-26T01:31:47.468 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:47.462+0000 7f5a58432640 1 -- 172.21.15.169:0/516676648 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a50152420 con 0x7f5a5015b2c0 2023-10-26T01:31:47.469 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:47.463+0000 7f5a58432640 1 -- 172.21.15.169:0/516676648 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a50066af0 con 0x7f5a5015b2c0 2023-10-26T01:31:47.469 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:47.464+0000 7f5a46ffd640 1 -- 172.21.15.169:0/516676648 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f5a400035f0 con 0x7f5a5015b2c0 2023-10-26T01:31:47.469 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:47.465+0000 7f5a46ffd640 1 --2- 172.21.15.169:0/516676648 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5a3406ae40 0x7f5a3406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:47.469 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:47.465+0000 7f5a46ffd640 1 -- 172.21.15.169:0/516676648 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5473+0+0 (secure 0 0 0) 0x7f5a40089c60 con 0x7f5a5015b2c0 2023-10-26T01:31:47.470 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:47.465+0000 7f5a561ce640 1 --2- 172.21.15.169:0/516676648 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5a3406ae40 0x7f5a3406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:47.470 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:47.466+0000 7f5a561ce640 1 --2- 172.21.15.169:0/516676648 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5a3406ae40 0x7f5a3406d300 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c007fd0 tx=0x7f5a4c009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:47.470 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:47.469+0000 7f5a46ffd640 1 -- 172.21.15.169:0/516676648 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5a40054de0 con 0x7f5a5015b2c0 2023-10-26T01:31:47.671 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:47.669+0000 7f5a58432640 1 -- 172.21.15.169:0/516676648 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f5a50152610 con 0x7f5a5015b2c0 2023-10-26T01:31:48.221 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:48.219+0000 7f5a46ffd640 1 -- 172.21.15.169:0/516676648 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v26) v1 ==== 169+0+0 (secure 0 0 0) 0x7f5a4001e0d0 con 0x7f5a5015b2c0 2023-10-26T01:31:48.222 INFO:teuthology.orchestra.run.smithi169.stderr:pool 'cephfs_data' removed 2023-10-26T01:31:48.225 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:48.223+0000 7f5a58432640 1 -- 172.21.15.169:0/516676648 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5a3406ae40 msgr2=0x7f5a3406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:48.225 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:48.223+0000 7f5a58432640 1 --2- 172.21.15.169:0/516676648 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5a3406ae40 0x7f5a3406d300 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c007fd0 tx=0x7f5a4c009040 comp rx=0 tx=0).stop 2023-10-26T01:31:48.225 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:48.223+0000 7f5a58432640 1 -- 172.21.15.169:0/516676648 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a5015b2c0 msgr2=0x7f5a50150ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:48.225 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:48.223+0000 7f5a58432640 1 --2- 172.21.15.169:0/516676648 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a5015b2c0 0x7f5a50150ec0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f5a4000c970 tx=0x7f5a4000ce30 comp rx=0 tx=0).stop 2023-10-26T01:31:48.225 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:48.223+0000 7f5a58432640 1 -- 172.21.15.169:0/516676648 shutdown_connections 2023-10-26T01:31:48.226 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:48.223+0000 7f5a58432640 1 --2- 172.21.15.169:0/516676648 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5a3406ae40 0x7f5a3406d300 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:48.226 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:48.223+0000 7f5a58432640 1 --2- 172.21.15.169:0/516676648 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a5015b2c0 0x7f5a50150ec0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:48.226 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:48.223+0000 7f5a58432640 1 --2- 172.21.15.169:0/516676648 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5a50151400 0x7f5a501551a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:48.226 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:48.223+0000 7f5a58432640 1 --2- 172.21.15.169:0/516676648 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5a501577a0 0x7f5a500cdac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:48.227 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:48.223+0000 7f5a58432640 1 -- 172.21.15.169:0/516676648 >> 172.21.15.169:0/516676648 conn(0x7f5a500aeee0 msgr2=0x7f5a500ae420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:48.227 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:48.223+0000 7f5a58432640 1 -- 172.21.15.169:0/516676648 shutdown_connections 2023-10-26T01:31:48.227 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:48.223+0000 7f5a58432640 1 -- 172.21.15.169:0/516676648 wait complete. 2023-10-26T01:31:48.244 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-26T01:31:48.244 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-26T01:31:48.244 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-26T01:31:48.245 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T01:31:48.245 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-26T01:31:48.245 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-26T01:31:48.245 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-26T01:31:48.246 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T01:31:48.246 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-26T01:31:48.246 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-26T01:31:48.247 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-26T01:31:48.247 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T01:31:48.247 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-26T01:31:48.247 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-26T01:31:48.248 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-26T01:31:48.248 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T01:31:48.261 INFO:tasks.ceph.mds.a:Stopped 2023-10-26T01:31:48.261 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-26T01:31:48.264 INFO:tasks.ceph.mds.c:Stopped 2023-10-26T01:31:48.265 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-26T01:31:48.267 INFO:tasks.ceph.mds.d:Stopped 2023-10-26T01:31:48.267 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-26T01:31:48.268 INFO:tasks.ceph.mds.b:Stopped 2023-10-26T01:31:48.269 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-26T01:31:48.309 INFO:tasks.ceph.mds.a:Started 2023-10-26T01:31:48.310 DEBUG:teuthology.parallel:result is None 2023-10-26T01:31:48.313 INFO:tasks.ceph.mds.c:Started 2023-10-26T01:31:48.314 DEBUG:teuthology.parallel:result is None 2023-10-26T01:31:48.317 INFO:tasks.ceph.mds.d:Started 2023-10-26T01:31:48.318 DEBUG:teuthology.parallel:result is None 2023-10-26T01:31:48.324 INFO:tasks.ceph.mds.b:Started 2023-10-26T01:31:48.325 DEBUG:teuthology.parallel:result is None 2023-10-26T01:31:48.325 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-26T01:31:48.470 INFO:tasks.ceph.mds.a.smithi169.stdout:starting mds.a at 2023-10-26T01:31:48.482 INFO:tasks.ceph.mds.d.smithi170.stdout:starting mds.d at 2023-10-26T01:31:48.483 INFO:tasks.ceph.mds.b.smithi170.stdout:starting mds.b at 2023-10-26T01:31:48.503 INFO:tasks.ceph.mds.c.smithi169.stdout:starting mds.c at 2023-10-26T01:31:48.533 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:48.531+0000 7f9f5ecde640 1 -- 172.21.15.169:0/1574284980 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f581593e0 msgr2=0x7f9f580ac200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:48.533 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:48.531+0000 7f9f5ecde640 1 --2- 172.21.15.169:0/1574284980 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f581593e0 0x7f9f580ac200 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f9f54009830 tx=0x7f9f5402bd50 comp rx=0 tx=0).stop 2023-10-26T01:31:48.533 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:48.531+0000 7f9f5ecde640 1 -- 172.21.15.169:0/1574284980 shutdown_connections 2023-10-26T01:31:48.533 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:48.531+0000 7f9f5ecde640 1 --2- 172.21.15.169:0/1574284980 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9f581597c0 0x7f9f5815dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:48.534 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:48.531+0000 7f9f5ecde640 1 --2- 172.21.15.169:0/1574284980 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f581593e0 0x7f9f580ac200 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:48.534 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:48.531+0000 7f9f5ecde640 1 -- 172.21.15.169:0/1574284980 >> 172.21.15.169:0/1574284980 conn(0x7f9f58069930 msgr2=0x7f9f58069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:48.534 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:48.531+0000 7f9f5ecde640 1 -- 172.21.15.169:0/1574284980 shutdown_connections 2023-10-26T01:31:48.534 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:48.531+0000 7f9f5ecde640 1 -- 172.21.15.169:0/1574284980 wait complete. 2023-10-26T01:31:48.537 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:48.532+0000 7f9f5ecde640 1 Processor -- start 2023-10-26T01:31:48.537 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:48.533+0000 7f9f5ecde640 1 -- start start 2023-10-26T01:31:48.537 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:48.533+0000 7f9f5ecde640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f580b0c80 0x7f9f581f19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:48.537 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:48.533+0000 7f9f5ecde640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9f581593e0 0x7f9f580afa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:48.537 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:48.534+0000 7f9f5ecde640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9f581597c0 0x7f9f580ad2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:48.538 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:48.534+0000 7f9f5ecde640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f581f1f10 con 0x7f9f580b0c80 2023-10-26T01:31:48.538 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:48.534+0000 7f9f5ecde640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f580aff70 con 0x7f9f581593e0 2023-10-26T01:31:48.538 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:48.534+0000 7f9f5ecde640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f580c70f0 con 0x7f9f581597c0 2023-10-26T01:31:48.538 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:48.534+0000 7f9f5ca7a640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f580b0c80 0x7f9f581f19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:48.539 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:48.534+0000 7f9f5ca7a640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f580b0c80 0x7f9f581f19d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:35762/0 (socket says 172.21.15.169:35762) 2023-10-26T01:31:48.539 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:48.534+0000 7f9f5ca7a640 1 -- 172.21.15.169:0/61183528 learned_addr learned my addr 172.21.15.169:0/61183528 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:31:48.539 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:48.534+0000 7f9f5d27b640 1 --2- 172.21.15.169:0/61183528 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9f581597c0 0x7f9f580ad2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:48.539 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:48.534+0000 7f9f5ca7a640 1 -- 172.21.15.169:0/61183528 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9f581597c0 msgr2=0x7f9f580ad2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:48.539 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:48.534+0000 7f9f4ffff640 1 --2- 172.21.15.169:0/61183528 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9f581593e0 0x7f9f580afa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:48.540 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:48.534+0000 7f9f5ca7a640 1 --2- 172.21.15.169:0/61183528 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9f581597c0 0x7f9f580ad2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:48.540 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:48.534+0000 7f9f5ca7a640 1 -- 172.21.15.169:0/61183528 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9f581593e0 msgr2=0x7f9f580afa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:48.540 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:48.534+0000 7f9f5ca7a640 1 --2- 172.21.15.169:0/61183528 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9f581593e0 0x7f9f580afa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:48.540 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:48.534+0000 7f9f5ca7a640 1 -- 172.21.15.169:0/61183528 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f48009490 con 0x7f9f580b0c80 2023-10-26T01:31:48.540 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:48.534+0000 7f9f5d27b640 1 --2- 172.21.15.169:0/61183528 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9f581597c0 0x7f9f580ad2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:31:48.541 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:48.534+0000 7f9f4ffff640 1 --2- 172.21.15.169:0/61183528 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9f581593e0 0x7f9f580afa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:31:48.541 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:48.535+0000 7f9f5ca7a640 1 --2- 172.21.15.169:0/61183528 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f580b0c80 0x7f9f581f19d0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f9f54009780 tx=0x7f9f540038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:48.541 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:48.535+0000 7f9f4d7fa640 1 -- 172.21.15.169:0/61183528 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f54003ea0 con 0x7f9f580b0c80 2023-10-26T01:31:48.541 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:48.535+0000 7f9f4d7fa640 1 -- 172.21.15.169:0/61183528 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9f54034440 con 0x7f9f580b0c80 2023-10-26T01:31:48.541 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:48.535+0000 7f9f4d7fa640 1 -- 172.21.15.169:0/61183528 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f54034850 con 0x7f9f580b0c80 2023-10-26T01:31:48.542 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:48.535+0000 7f9f5ecde640 1 -- 172.21.15.169:0/61183528 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f54009490 con 0x7f9f580b0c80 2023-10-26T01:31:48.542 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:48.535+0000 7f9f5ecde640 1 -- 172.21.15.169:0/61183528 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f580c76b0 con 0x7f9f580b0c80 2023-10-26T01:31:48.543 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:48.536+0000 7f9f5ecde640 1 -- 172.21.15.169:0/61183528 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f5804db10 con 0x7f9f580b0c80 2023-10-26T01:31:48.543 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:48.537+0000 7f9f4d7fa640 1 -- 172.21.15.169:0/61183528 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f9f5403b040 con 0x7f9f580b0c80 2023-10-26T01:31:48.543 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:48.538+0000 7f9f4d7fa640 1 --2- 172.21.15.169:0/61183528 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9f4406ae40 0x7f9f4406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:48.543 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:48.538+0000 7f9f4d7fa640 1 -- 172.21.15.169:0/61183528 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f9f540ab7c0 con 0x7f9f580b0c80 2023-10-26T01:31:48.544 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:48.539+0000 7f9f4ffff640 1 --2- 172.21.15.169:0/61183528 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9f4406ae40 0x7f9f4406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:48.544 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:48.539+0000 7f9f4ffff640 1 --2- 172.21.15.169:0/61183528 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9f4406ae40 0x7f9f4406d300 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9f580c9260 tx=0x7f9f48005f90 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:48.544 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:48.543+0000 7f9f4d7fa640 1 -- 172.21.15.169:0/61183528 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9f54076ad0 con 0x7f9f580b0c80 2023-10-26T01:31:48.735 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:48.733+0000 7f9f5ecde640 1 -- 172.21.15.169:0/61183528 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f9f580b2170 con 0x7f9f580b0c80 2023-10-26T01:31:49.941 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:49.939+0000 7f9f4d7fa640 1 -- 172.21.15.169:0/61183528 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v31) v1 ==== 97+0+0 (secure 0 0 0) 0x7f9f54039080 con 0x7f9f580b0c80 2023-10-26T01:31:49.941 INFO:teuthology.orchestra.run.smithi169.stderr: removed all blocklist entries 2023-10-26T01:31:49.944 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:49.943+0000 7f9f5ecde640 1 -- 172.21.15.169:0/61183528 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9f4406ae40 msgr2=0x7f9f4406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:49.945 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:49.943+0000 7f9f5ecde640 1 --2- 172.21.15.169:0/61183528 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9f4406ae40 0x7f9f4406d300 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9f580c9260 tx=0x7f9f48005f90 comp rx=0 tx=0).stop 2023-10-26T01:31:49.945 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:49.943+0000 7f9f5ecde640 1 -- 172.21.15.169:0/61183528 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f580b0c80 msgr2=0x7f9f581f19d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:49.945 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:49.943+0000 7f9f5ecde640 1 --2- 172.21.15.169:0/61183528 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f580b0c80 0x7f9f581f19d0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f9f54009780 tx=0x7f9f540038a0 comp rx=0 tx=0).stop 2023-10-26T01:31:49.945 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:49.943+0000 7f9f5ecde640 1 -- 172.21.15.169:0/61183528 shutdown_connections 2023-10-26T01:31:49.946 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:49.943+0000 7f9f5ecde640 1 --2- 172.21.15.169:0/61183528 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9f4406ae40 0x7f9f4406d300 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:49.946 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:49.943+0000 7f9f5ecde640 1 --2- 172.21.15.169:0/61183528 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9f581597c0 0x7f9f580ad2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:49.946 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:49.943+0000 7f9f5ecde640 1 --2- 172.21.15.169:0/61183528 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9f581593e0 0x7f9f580afa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:49.946 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:49.943+0000 7f9f5ecde640 1 --2- 172.21.15.169:0/61183528 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f580b0c80 0x7f9f581f19d0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:49.947 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:49.943+0000 7f9f5ecde640 1 -- 172.21.15.169:0/61183528 >> 172.21.15.169:0/61183528 conn(0x7f9f58069930 msgr2=0x7f9f58153830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:49.947 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:49.943+0000 7f9f5ecde640 1 -- 172.21.15.169:0/61183528 shutdown_connections 2023-10-26T01:31:49.947 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:49.943+0000 7f9f5ecde640 1 -- 172.21.15.169:0/61183528 wait complete. 2023-10-26T01:31:49.964 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-26T01:31:50.189 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:50.187+0000 7f1e53032640 1 -- 172.21.15.169:0/3896302779 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1e340039a0 msgr2=0x7f1e34003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:50.189 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:50.187+0000 7f1e53032640 1 --2- 172.21.15.169:0/3896302779 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1e340039a0 0x7f1e34003dc0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f1e48006fa0 tx=0x7f1e340057f0 comp rx=0 tx=0).stop 2023-10-26T01:31:50.190 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:50.187+0000 7f1e515cf640 1 -- 172.21.15.169:0/3896302779 reap_dead start 2023-10-26T01:31:50.190 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:50.187+0000 7f1e53032640 1 -- 172.21.15.169:0/3896302779 shutdown_connections 2023-10-26T01:31:50.190 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:50.187+0000 7f1e53032640 1 -- 172.21.15.169:0/3896302779 >> 172.21.15.169:0/3896302779 conn(0x7f1e4c069930 msgr2=0x7f1e4c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:50.190 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:50.187+0000 7f1e53032640 1 -- 172.21.15.169:0/3896302779 shutdown_connections 2023-10-26T01:31:50.191 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:50.187+0000 7f1e53032640 1 -- 172.21.15.169:0/3896302779 wait complete. 2023-10-26T01:31:50.193 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:50.189+0000 7f1e53032640 1 Processor -- start 2023-10-26T01:31:50.193 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:50.189+0000 7f1e53032640 1 -- start start 2023-10-26T01:31:50.194 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:50.190+0000 7f1e53032640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1e340039a0 0x7f1e4c1f0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:50.194 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:50.190+0000 7f1e53032640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1e4c0af7a0 0x7f1e4c14d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:50.194 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:50.190+0000 7f1e53032640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1e4c09a9b0 0x7f1e4c09ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:50.194 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:50.190+0000 7f1e53032640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e4c09b370 con 0x7f1e340039a0 2023-10-26T01:31:50.194 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:50.190+0000 7f1e53032640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e4c0afd60 con 0x7f1e4c0af7a0 2023-10-26T01:31:50.195 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:50.190+0000 7f1e53032640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e4c1f0fa0 con 0x7f1e4c09a9b0 2023-10-26T01:31:50.195 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:50.190+0000 7f1e50dce640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1e340039a0 0x7f1e4c1f0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:50.195 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:50.190+0000 7f1e50dce640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1e340039a0 0x7f1e4c1f0a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:35786/0 (socket says 172.21.15.169:35786) 2023-10-26T01:31:50.195 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:50.190+0000 7f1e50dce640 1 -- 172.21.15.169:0/3020352200 learned_addr learned my addr 172.21.15.169:0/3020352200 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:31:50.195 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:50.191+0000 7f1e43fff640 1 --2- 172.21.15.169:0/3020352200 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1e4c0af7a0 0x7f1e4c14d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:50.196 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:50.191+0000 7f1e515cf640 1 --2- 172.21.15.169:0/3020352200 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1e4c09a9b0 0x7f1e4c09ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:50.196 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:50.191+0000 7f1e50dce640 1 -- 172.21.15.169:0/3020352200 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1e4c09a9b0 msgr2=0x7f1e4c09ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:50.196 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:50.191+0000 7f1e50dce640 1 --2- 172.21.15.169:0/3020352200 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1e4c09a9b0 0x7f1e4c09ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:50.196 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:50.191+0000 7f1e50dce640 1 -- 172.21.15.169:0/3020352200 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1e4c0af7a0 msgr2=0x7f1e4c14d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:50.197 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:50.191+0000 7f1e50dce640 1 --2- 172.21.15.169:0/3020352200 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1e4c0af7a0 0x7f1e4c14d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:50.197 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:50.191+0000 7f1e50dce640 1 -- 172.21.15.169:0/3020352200 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e4c1f12d0 con 0x7f1e340039a0 2023-10-26T01:31:50.197 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:50.191+0000 7f1e50dce640 1 --2- 172.21.15.169:0/3020352200 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1e340039a0 0x7f1e4c1f0a60 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f1e4802c990 tx=0x7f1e480015f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:50.197 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:50.192+0000 7f1e417fa640 1 -- 172.21.15.169:0/3020352200 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e48006230 con 0x7f1e340039a0 2023-10-26T01:31:50.198 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:50.192+0000 7f1e417fa640 1 -- 172.21.15.169:0/3020352200 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1e48006390 con 0x7f1e340039a0 2023-10-26T01:31:50.198 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:50.192+0000 7f1e417fa640 1 -- 172.21.15.169:0/3020352200 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e480035f0 con 0x7f1e340039a0 2023-10-26T01:31:50.198 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:50.192+0000 7f1e53032640 1 -- 172.21.15.169:0/3020352200 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e4c1f15b0 con 0x7f1e340039a0 2023-10-26T01:31:50.198 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:50.192+0000 7f1e53032640 1 -- 172.21.15.169:0/3020352200 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e4c1f1b70 con 0x7f1e340039a0 2023-10-26T01:31:50.199 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:50.193+0000 7f1e53032640 1 -- 172.21.15.169:0/3020352200 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e4c1f1d00 con 0x7f1e340039a0 2023-10-26T01:31:50.200 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:50.194+0000 7f1e417fa640 1 -- 172.21.15.169:0/3020352200 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f1e48031030 con 0x7f1e340039a0 2023-10-26T01:31:50.200 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:50.195+0000 7f1e417fa640 1 --2- 172.21.15.169:0/3020352200 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f1e3806ae40 0x7f1e3806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:50.200 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:50.195+0000 7f1e417fa640 1 -- 172.21.15.169:0/3020352200 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f1e480aa250 con 0x7f1e340039a0 2023-10-26T01:31:50.200 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:50.196+0000 7f1e43fff640 1 --2- 172.21.15.169:0/3020352200 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f1e3806ae40 0x7f1e3806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:50.200 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:50.197+0000 7f1e43fff640 1 --2- 172.21.15.169:0/3020352200 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f1e3806ae40 0x7f1e3806d300 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c1585e0 tx=0x7f1e3c005f90 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:50.201 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:50.200+0000 7f1e417fa640 1 -- 172.21.15.169:0/3020352200 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1e480755b0 con 0x7f1e340039a0 2023-10-26T01:31:50.468 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:50.466+0000 7f1e53032640 1 -- 172.21.15.169:0/3020352200 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f1e4c04db10 con 0x7f1e340039a0 2023-10-26T01:31:50.469 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:50.468+0000 7f1e417fa640 1 -- 172.21.15.169:0/3020352200 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v4) v1 ==== 80+0+6645 (secure 0 0 0) 0x7f1e48078bd0 con 0x7f1e340039a0 2023-10-26T01:31:50.469 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:31:50.470 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2023-10-26T01:31:50.470 INFO:teuthology.orchestra.run.smithi169.stdout: "auth_dump": [ 2023-10-26T01:31:50.470 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:31:50.470 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "mds.a", 2023-10-26T01:31:50.471 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAgwTllWtUfAhAA28+12VwWEF/BUEu+ErSVyA==", 2023-10-26T01:31:50.471 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:31:50.471 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow", 2023-10-26T01:31:50.472 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow *", 2023-10-26T01:31:50.472 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow *", 2023-10-26T01:31:50.472 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:31:50.472 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:31:50.472 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:31:50.473 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:31:50.473 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "mds.b", 2023-10-26T01:31:50.473 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAgwTlleA8lHxAA+jMcMo+QnYAGllpKJ6eYgQ==", 2023-10-26T01:31:50.473 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:31:50.473 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow", 2023-10-26T01:31:50.474 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow *", 2023-10-26T01:31:50.474 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow *", 2023-10-26T01:31:50.474 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:31:50.474 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:31:50.475 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:31:50.475 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:31:50.475 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "mds.c", 2023-10-26T01:31:50.475 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAgwTllaCCqEBAAokeHvwOyp/Jv2x0yHDwysw==", 2023-10-26T01:31:50.475 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:31:50.476 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow", 2023-10-26T01:31:50.476 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow *", 2023-10-26T01:31:50.476 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow *", 2023-10-26T01:31:50.476 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:31:50.477 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:31:50.477 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:31:50.477 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:31:50.478 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "mds.d", 2023-10-26T01:31:50.478 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAgwTll51JQLhAA4Wzso4zQj+Htv6cXqvwRmg==", 2023-10-26T01:31:50.478 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:31:50.478 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow", 2023-10-26T01:31:50.479 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow *", 2023-10-26T01:31:50.479 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow *", 2023-10-26T01:31:50.479 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:31:50.479 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:31:50.479 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:31:50.480 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:31:50.480 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "osd.0", 2023-10-26T01:31:50.480 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAlwTll8RZ+HRAAt/sBK+Mi91YmTyQYbhKHLg==", 2023-10-26T01:31:50.480 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:31:50.480 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow profile osd", 2023-10-26T01:31:50.481 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow profile osd", 2023-10-26T01:31:50.481 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:31:50.481 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:31:50.481 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:31:50.482 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:31:50.482 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "osd.1", 2023-10-26T01:31:50.482 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAlwTllLXwWLRAAuHB2lJ+oDJou/IJBtsYY4Q==", 2023-10-26T01:31:50.482 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:31:50.482 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow profile osd", 2023-10-26T01:31:50.483 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow profile osd", 2023-10-26T01:31:50.483 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:31:50.483 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:31:50.483 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:31:50.483 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:31:50.484 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "osd.2", 2023-10-26T01:31:50.484 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAmwTllj1nSARAA5Jl34Q0X7a71ySd4wz17lw==", 2023-10-26T01:31:50.488 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:31:50.488 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow profile osd", 2023-10-26T01:31:50.489 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow profile osd", 2023-10-26T01:31:50.489 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:31:50.489 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:31:50.489 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:31:50.490 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:31:50.490 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "osd.3", 2023-10-26T01:31:50.490 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAmwTllC8MeEhAAfGxRVSwD+YbU/U5aeenO3w==", 2023-10-26T01:31:50.490 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:31:50.490 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow profile osd", 2023-10-26T01:31:50.491 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow profile osd", 2023-10-26T01:31:50.491 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:31:50.491 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:31:50.491 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:31:50.492 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:31:50.492 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "osd.4", 2023-10-26T01:31:50.492 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAowTll4Md6DhAAuXfadUNAdouMP28MiX/D3Q==", 2023-10-26T01:31:50.492 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:31:50.493 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow profile osd", 2023-10-26T01:31:50.493 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow profile osd", 2023-10-26T01:31:50.493 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:31:50.493 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:31:50.494 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:31:50.494 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:31:50.494 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "osd.5", 2023-10-26T01:31:50.494 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAowTllDdViHRAAec9QyhZCsjJ2rgGhhcjwpQ==", 2023-10-26T01:31:50.495 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:31:50.495 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow profile osd", 2023-10-26T01:31:50.495 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow profile osd", 2023-10-26T01:31:50.495 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:31:50.495 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:31:50.496 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:31:50.496 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:31:50.496 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "osd.6", 2023-10-26T01:31:50.496 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAowTll23X5KxAAsmQvG0yq7ilkVG5fNiaCUg==", 2023-10-26T01:31:50.496 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:31:50.497 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow profile osd", 2023-10-26T01:31:50.497 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow profile osd", 2023-10-26T01:31:50.497 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:31:50.497 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:31:50.497 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:31:50.498 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:31:50.498 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "osd.7", 2023-10-26T01:31:50.498 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAowTllhcy/OhAAY5Yjheh76ibOVREqN6WBpQ==", 2023-10-26T01:31:50.499 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:31:50.499 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow profile osd", 2023-10-26T01:31:50.499 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow profile osd", 2023-10-26T01:31:50.499 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:31:50.500 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:31:50.500 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:31:50.500 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:31:50.500 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "client.0", 2023-10-26T01:31:50.500 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAgwTllKC7gOhAAPsvxGcuUA8/p8xqbUgGteA==", 2023-10-26T01:31:50.501 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:31:50.501 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow", 2023-10-26T01:31:50.501 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow r", 2023-10-26T01:31:50.501 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow rw", 2023-10-26T01:31:50.501 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow rwx" 2023-10-26T01:31:50.502 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:31:50.502 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:31:50.502 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:31:50.502 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "client.1", 2023-10-26T01:31:50.502 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAhwTllhxb6ExAAeYTsurWpnjohWlrNIlLpsw==", 2023-10-26T01:31:50.503 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:31:50.503 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow", 2023-10-26T01:31:50.503 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow r", 2023-10-26T01:31:50.503 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow rw", 2023-10-26T01:31:50.504 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow rwx" 2023-10-26T01:31:50.504 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:31:50.504 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:31:50.504 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:31:50.504 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "client.2", 2023-10-26T01:31:50.505 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAhwTllLWMoChAAsUHzBc29GDqmEQAFnl7dHg==", 2023-10-26T01:31:50.505 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:31:50.505 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow", 2023-10-26T01:31:50.505 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow r", 2023-10-26T01:31:50.506 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow rw", 2023-10-26T01:31:50.506 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow rwx" 2023-10-26T01:31:50.506 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:31:50.506 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:31:50.506 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:31:50.507 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "client.3", 2023-10-26T01:31:50.507 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAhwTll875HHRAAGbwikoR/hDU8P/bj0OPWAw==", 2023-10-26T01:31:50.507 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:31:50.507 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow", 2023-10-26T01:31:50.507 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow r", 2023-10-26T01:31:50.508 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow rw", 2023-10-26T01:31:50.508 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow rwx" 2023-10-26T01:31:50.508 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:31:50.508 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:31:50.509 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:31:50.509 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "client.admin", 2023-10-26T01:31:50.509 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAewTll2yddMhAAO8bnXNevDGe/KEeNNa36pQ==", 2023-10-26T01:31:50.510 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:31:50.510 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow *", 2023-10-26T01:31:50.510 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow *", 2023-10-26T01:31:50.510 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow *", 2023-10-26T01:31:50.511 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:31:50.511 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:31:50.511 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:31:50.511 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:31:50.511 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "client.bootstrap-mds", 2023-10-26T01:31:50.512 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAuwTllzrSBNRAAayqWw7j+N5MBQqBYPrUnUQ==", 2023-10-26T01:31:50.512 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:31:50.512 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow profile bootstrap-mds" 2023-10-26T01:31:50.512 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:31:50.513 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:31:50.513 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:31:50.513 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "client.bootstrap-mgr", 2023-10-26T01:31:50.514 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAuwTllPceBNRAAKHwRJsW9HB6KW1/1t2hvJQ==", 2023-10-26T01:31:50.514 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:31:50.514 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow profile bootstrap-mgr" 2023-10-26T01:31:50.515 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:31:50.515 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:31:50.516 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:31:50.516 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "client.bootstrap-osd", 2023-10-26T01:31:50.516 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAuwTllediBNRAAAUtY05yVw/lrm32r/IiwSw==", 2023-10-26T01:31:50.517 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:31:50.517 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow profile bootstrap-osd" 2023-10-26T01:31:50.518 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:31:50.518 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:31:50.518 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:31:50.519 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "client.bootstrap-rbd", 2023-10-26T01:31:50.519 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAuwTll7eeBNRAAuOSs2icdiWynIHmFuEhfnA==", 2023-10-26T01:31:50.519 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:31:50.520 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow profile bootstrap-rbd" 2023-10-26T01:31:50.520 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:31:50.520 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:31:50.521 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:31:50.521 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-10-26T01:31:50.521 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAuwTllxveBNRAAr4NS1Pc/vqDa4Qaq5oRJvw==", 2023-10-26T01:31:50.522 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:31:50.522 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-10-26T01:31:50.523 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:31:50.523 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:31:50.523 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:31:50.523 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "client.bootstrap-rgw", 2023-10-26T01:31:50.524 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAuwTlleweCNRAAaxRygDap9tNuMeMKL8RLJQ==", 2023-10-26T01:31:50.524 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:31:50.524 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow profile bootstrap-rgw" 2023-10-26T01:31:50.525 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:31:50.525 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:31:50.525 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:31:50.525 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "mgr.x", 2023-10-26T01:31:50.525 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAfwTllPu3FMxAAV8d5oRFXAeeZH7e1xzPEDg==", 2023-10-26T01:31:50.526 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:31:50.526 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow *", 2023-10-26T01:31:50.526 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow profile mgr", 2023-10-26T01:31:50.526 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:31:50.526 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:31:50.527 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:31:50.527 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:31:50.527 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "mgr.y", 2023-10-26T01:31:50.527 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAfwTllZc+DJxAAsZ2TWn8H7Kw/lNTc2qyKzA==", 2023-10-26T01:31:50.527 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:31:50.528 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow *", 2023-10-26T01:31:50.528 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow profile mgr", 2023-10-26T01:31:50.528 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:31:50.528 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:31:50.528 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:31:50.528 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:31:50.529 INFO:teuthology.orchestra.run.smithi169.stdout:} 2023-10-26T01:31:50.530 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:50.470+0000 7f1e53032640 1 -- 172.21.15.169:0/3020352200 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f1e3806ae40 msgr2=0x7f1e3806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:50.530 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:50.470+0000 7f1e53032640 1 --2- 172.21.15.169:0/3020352200 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f1e3806ae40 0x7f1e3806d300 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c1585e0 tx=0x7f1e3c005f90 comp rx=0 tx=0).stop 2023-10-26T01:31:50.531 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:50.470+0000 7f1e53032640 1 -- 172.21.15.169:0/3020352200 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1e340039a0 msgr2=0x7f1e4c1f0a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:50.531 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:50.470+0000 7f1e53032640 1 --2- 172.21.15.169:0/3020352200 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1e340039a0 0x7f1e4c1f0a60 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f1e4802c990 tx=0x7f1e480015f0 comp rx=0 tx=0).stop 2023-10-26T01:31:50.531 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:50.470+0000 7f1e53032640 1 -- 172.21.15.169:0/3020352200 shutdown_connections 2023-10-26T01:31:50.531 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:50.470+0000 7f1e53032640 1 --2- 172.21.15.169:0/3020352200 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f1e3806ae40 0x7f1e3806d300 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:50.531 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:50.470+0000 7f1e53032640 1 --2- 172.21.15.169:0/3020352200 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1e4c09a9b0 0x7f1e4c09ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:50.532 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:50.470+0000 7f1e53032640 1 --2- 172.21.15.169:0/3020352200 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1e4c0af7a0 0x7f1e4c14d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:50.532 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:50.470+0000 7f1e53032640 1 --2- 172.21.15.169:0/3020352200 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1e340039a0 0x7f1e4c1f0a60 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:50.532 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:50.470+0000 7f1e53032640 1 -- 172.21.15.169:0/3020352200 >> 172.21.15.169:0/3020352200 conn(0x7f1e4c069930 msgr2=0x7f1e4c15be70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:50.532 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:50.471+0000 7f1e53032640 1 -- 172.21.15.169:0/3020352200 shutdown_connections 2023-10-26T01:31:50.532 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:50.471+0000 7f1e53032640 1 -- 172.21.15.169:0/3020352200 wait complete. 2023-10-26T01:31:50.533 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-10-26T01:31:50.763 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:50.761+0000 7f631a333640 1 -- 172.21.15.169:0/2234677099 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63140b3650 msgr2=0x7f63140699e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:50.763 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:50.761+0000 7f631a333640 1 --2- 172.21.15.169:0/2234677099 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63140b3650 0x7f63140699e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f631406ab50 tx=0x7f630c02c250 comp rx=0 tx=0).stop 2023-10-26T01:31:50.763 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:50.761+0000 7f631a333640 1 -- 172.21.15.169:0/2234677099 shutdown_connections 2023-10-26T01:31:50.763 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:50.761+0000 7f631a333640 1 --2- 172.21.15.169:0/2234677099 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63140b3650 0x7f63140699e0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:50.764 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:50.761+0000 7f631a333640 1 --2- 172.21.15.169:0/2234677099 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f63140b7330 0x7f63140b3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:50.764 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:50.761+0000 7f631a333640 1 --2- 172.21.15.169:0/2234677099 >> v2:172.21.15.170:3300/0 conn(0x7f631415b9a0 0x7f631415f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:50.764 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:50.761+0000 7f631a333640 1 -- 172.21.15.169:0/2234677099 >> 172.21.15.169:0/2234677099 conn(0x7f63140658b0 msgr2=0x7f6314065cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:50.764 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:50.761+0000 7f631a333640 1 -- 172.21.15.169:0/2234677099 shutdown_connections 2023-10-26T01:31:50.764 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:50.761+0000 7f631a333640 1 -- 172.21.15.169:0/2234677099 wait complete. 2023-10-26T01:31:50.765 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:50.762+0000 7f631a333640 1 Processor -- start 2023-10-26T01:31:50.765 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:50.763+0000 7f631a333640 1 -- start start 2023-10-26T01:31:50.766 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:50.763+0000 7f631a333640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63140b3650 0x7f6314154b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:50.766 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:50.764+0000 7f631a333640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f63140b7330 0x7f63140cad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:50.766 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:50.764+0000 7f631a333640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f631415b9a0 0x7f63140cb240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:50.766 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:50.764+0000 7f631a333640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6314147f80 con 0x7f63140b3650 2023-10-26T01:31:50.766 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:50.764+0000 7f631a333640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63141550b0 con 0x7f63140b7330 2023-10-26T01:31:50.767 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:50.764+0000 7f631a333640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6314157870 con 0x7f631415b9a0 2023-10-26T01:31:50.767 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:50.764+0000 7f6313fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63140b3650 0x7f6314154b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:50.767 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:50.764+0000 7f6313fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63140b3650 0x7f6314154b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:35812/0 (socket says 172.21.15.169:35812) 2023-10-26T01:31:50.767 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:50.764+0000 7f6313fff640 1 -- 172.21.15.169:0/1890959255 learned_addr learned my addr 172.21.15.169:0/1890959255 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:31:50.768 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:50.764+0000 7f6313fff640 1 -- 172.21.15.169:0/1890959255 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f631415b9a0 msgr2=0x7f63140cb240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:50.769 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:50.764+0000 7f6313fff640 1 --2- 172.21.15.169:0/1890959255 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f631415b9a0 0x7f63140cb240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:50.769 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:50.764+0000 7f6313fff640 1 -- 172.21.15.169:0/1890959255 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f63140b7330 msgr2=0x7f63140cad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:50.769 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:50.764+0000 7f6313fff640 1 --2- 172.21.15.169:0/1890959255 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f63140b7330 0x7f63140cad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:50.769 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:50.764+0000 7f6313fff640 1 -- 172.21.15.169:0/1890959255 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f630c009900 con 0x7f63140b3650 2023-10-26T01:31:50.769 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:50.765+0000 7f6313fff640 1 --2- 172.21.15.169:0/1890959255 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63140b3650 0x7f6314154b70 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f630400d740 tx=0x7f630400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:50.769 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:50.765+0000 7f6310ff9640 1 -- 172.21.15.169:0/1890959255 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6304009c30 con 0x7f63140b3650 2023-10-26T01:31:50.770 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:50.765+0000 7f6310ff9640 1 -- 172.21.15.169:0/1890959255 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6304009e80 con 0x7f63140b3650 2023-10-26T01:31:50.770 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:50.765+0000 7f6310ff9640 1 -- 172.21.15.169:0/1890959255 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63040106a0 con 0x7f63140b3650 2023-10-26T01:31:50.770 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:50.765+0000 7f631a333640 1 -- 172.21.15.169:0/1890959255 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6314157b50 con 0x7f63140b3650 2023-10-26T01:31:50.770 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:50.765+0000 7f631a333640 1 -- 172.21.15.169:0/1890959255 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6314147ba0 con 0x7f63140b3650 2023-10-26T01:31:50.770 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:50.766+0000 7f631a333640 1 -- 172.21.15.169:0/1890959255 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6314147d60 con 0x7f63140b3650 2023-10-26T01:31:50.771 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:50.767+0000 7f6310ff9640 1 -- 172.21.15.169:0/1890959255 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f6304010800 con 0x7f63140b3650 2023-10-26T01:31:50.771 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:50.768+0000 7f6310ff9640 1 --2- 172.21.15.169:0/1890959255 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f630006ae40 0x7f630006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:50.771 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:50.768+0000 7f6310ff9640 1 -- 172.21.15.169:0/1890959255 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f63040897b0 con 0x7f63140b3650 2023-10-26T01:31:50.772 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:50.769+0000 7f63137fe640 1 --2- 172.21.15.169:0/1890959255 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f630006ae40 0x7f630006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:50.772 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:50.770+0000 7f63137fe640 1 --2- 172.21.15.169:0/1890959255 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f630006ae40 0x7f630006d300 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6308005d80 tx=0x7f6308005cb0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:50.774 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:50.772+0000 7f6310ff9640 1 -- 172.21.15.169:0/1890959255 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6304054bb0 con 0x7f63140b3650 2023-10-26T01:31:51.034 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:51.032+0000 7f631a333640 1 -- 172.21.15.169:0/1890959255 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f631404db10 con 0x7f63140b3650 2023-10-26T01:31:51.038 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:51.037+0000 7f6310ff9640 1 -- 172.21.15.169:0/1890959255 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v5) v1 ==== 90+0+0 (secure 0 0 0) 0x7f63040170a0 con 0x7f63140b3650 2023-10-26T01:31:51.042 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:51.040+0000 7f631a333640 1 -- 172.21.15.169:0/1890959255 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f630006ae40 msgr2=0x7f630006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:51.043 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:51.040+0000 7f631a333640 1 --2- 172.21.15.169:0/1890959255 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f630006ae40 0x7f630006d300 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6308005d80 tx=0x7f6308005cb0 comp rx=0 tx=0).stop 2023-10-26T01:31:51.043 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:51.040+0000 7f631a333640 1 -- 172.21.15.169:0/1890959255 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63140b3650 msgr2=0x7f6314154b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:51.043 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:51.040+0000 7f631a333640 1 --2- 172.21.15.169:0/1890959255 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63140b3650 0x7f6314154b70 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f630400d740 tx=0x7f630400dc00 comp rx=0 tx=0).stop 2023-10-26T01:31:51.043 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:51.041+0000 7f631a333640 1 -- 172.21.15.169:0/1890959255 shutdown_connections 2023-10-26T01:31:51.043 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:51.041+0000 7f631a333640 1 --2- 172.21.15.169:0/1890959255 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f630006ae40 0x7f630006d300 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:51.044 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:51.041+0000 7f631a333640 1 --2- 172.21.15.169:0/1890959255 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f631415b9a0 0x7f63140cb240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:51.044 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:51.041+0000 7f631a333640 1 --2- 172.21.15.169:0/1890959255 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f63140b7330 0x7f63140cad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:51.044 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:51.041+0000 7f631a333640 1 --2- 172.21.15.169:0/1890959255 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63140b3650 0x7f6314154b70 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:51.044 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:51.041+0000 7f631a333640 1 -- 172.21.15.169:0/1890959255 >> 172.21.15.169:0/1890959255 conn(0x7f63140658b0 msgr2=0x7f63140b10f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:51.044 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:51.041+0000 7f631a333640 1 -- 172.21.15.169:0/1890959255 shutdown_connections 2023-10-26T01:31:51.045 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:51.041+0000 7f631a333640 1 -- 172.21.15.169:0/1890959255 wait complete. 2023-10-26T01:31:51.062 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-10-26T01:31:51.274 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:51.272+0000 7fa58759e640 1 -- 172.21.15.169:0/4035665903 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa56c004020 msgr2=0x7fa56c0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:51.275 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:51.272+0000 7fa58759e640 1 --2- 172.21.15.169:0/4035665903 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa56c004020 0x7fa56c0047c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fa570001680 tx=0x7fa56c0057f0 comp rx=0 tx=0).stop 2023-10-26T01:31:51.275 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:51.273+0000 7fa585b3b640 1 -- 172.21.15.169:0/4035665903 reap_dead start 2023-10-26T01:31:51.275 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:51.273+0000 7fa58759e640 1 -- 172.21.15.169:0/4035665903 shutdown_connections 2023-10-26T01:31:51.275 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:51.273+0000 7fa58759e640 1 -- 172.21.15.169:0/4035665903 >> 172.21.15.169:0/4035665903 conn(0x7fa580069930 msgr2=0x7fa580069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:51.276 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:51.273+0000 7fa58759e640 1 -- 172.21.15.169:0/4035665903 shutdown_connections 2023-10-26T01:31:51.276 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:51.273+0000 7fa58759e640 1 -- 172.21.15.169:0/4035665903 wait complete. 2023-10-26T01:31:51.276 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:51.274+0000 7fa58759e640 1 Processor -- start 2023-10-26T01:31:51.277 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:51.275+0000 7fa58759e640 1 -- start start 2023-10-26T01:31:51.279 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:51.276+0000 7fa58759e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa56c004020 0x7fa5800ad0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:51.279 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:51.276+0000 7fa58759e640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa58014f3e0 0x7fa580158e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:51.279 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:51.276+0000 7fa58759e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa5801593a0 0x7fa5800cb520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:51.279 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:51.276+0000 7fa58759e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa580159820 con 0x7fa56c004020 2023-10-26T01:31:51.279 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:51.276+0000 7fa58759e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa580070990 con 0x7fa5801593a0 2023-10-26T01:31:51.279 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:51.276+0000 7fa58759e640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5800ad620 con 0x7fa58014f3e0 2023-10-26T01:31:51.280 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:51.276+0000 7fa58533a640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa56c004020 0x7fa5800ad0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:51.280 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:51.276+0000 7fa58533a640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa56c004020 0x7fa5800ad0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:35816/0 (socket says 172.21.15.169:35816) 2023-10-26T01:31:51.280 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:51.276+0000 7fa58533a640 1 -- 172.21.15.169:0/1420550926 learned_addr learned my addr 172.21.15.169:0/1420550926 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:31:51.280 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:51.276+0000 7fa585b3b640 1 --2- 172.21.15.169:0/1420550926 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa5801593a0 0x7fa5800cb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:51.280 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:51.276+0000 7fa584b39640 1 --2- 172.21.15.169:0/1420550926 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa58014f3e0 0x7fa580158e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:51.281 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:51.276+0000 7fa58533a640 1 -- 172.21.15.169:0/1420550926 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa58014f3e0 msgr2=0x7fa580158e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:51.281 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:51.276+0000 7fa58533a640 1 --2- 172.21.15.169:0/1420550926 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa58014f3e0 0x7fa580158e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:51.281 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:51.276+0000 7fa58533a640 1 -- 172.21.15.169:0/1420550926 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa5801593a0 msgr2=0x7fa5800cb520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:51.281 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:51.276+0000 7fa58533a640 1 --2- 172.21.15.169:0/1420550926 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa5801593a0 0x7fa5800cb520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:51.281 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:51.276+0000 7fa58533a640 1 -- 172.21.15.169:0/1420550926 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa58006fd80 con 0x7fa56c004020 2023-10-26T01:31:51.282 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:51.277+0000 7fa58533a640 1 --2- 172.21.15.169:0/1420550926 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa56c004020 0x7fa5800ad0e0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fa57c00d950 tx=0x7fa57c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:51.282 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:51.277+0000 7fa575ffb640 1 -- 172.21.15.169:0/1420550926 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa57c0062e0 con 0x7fa56c004020 2023-10-26T01:31:51.282 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:51.277+0000 7fa575ffb640 1 -- 172.21.15.169:0/1420550926 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa57c00b830 con 0x7fa56c004020 2023-10-26T01:31:51.282 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:51.277+0000 7fa58759e640 1 -- 172.21.15.169:0/1420550926 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa580070060 con 0x7fa56c004020 2023-10-26T01:31:51.282 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:51.277+0000 7fa575ffb640 1 -- 172.21.15.169:0/1420550926 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa57c00bbc0 con 0x7fa56c004020 2023-10-26T01:31:51.283 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:51.277+0000 7fa58759e640 1 -- 172.21.15.169:0/1420550926 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa580070620 con 0x7fa56c004020 2023-10-26T01:31:51.284 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:51.278+0000 7fa58759e640 1 -- 172.21.15.169:0/1420550926 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa58015b830 con 0x7fa56c004020 2023-10-26T01:31:51.284 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:51.279+0000 7fa575ffb640 1 -- 172.21.15.169:0/1420550926 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fa57c00bd20 con 0x7fa56c004020 2023-10-26T01:31:51.284 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:51.280+0000 7fa575ffb640 1 --2- 172.21.15.169:0/1420550926 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa56406ae40 0x7fa56406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:51.284 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:51.280+0000 7fa575ffb640 1 -- 172.21.15.169:0/1420550926 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa57c089e30 con 0x7fa56c004020 2023-10-26T01:31:51.284 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:51.281+0000 7fa584b39640 1 --2- 172.21.15.169:0/1420550926 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa56406ae40 0x7fa56406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:51.285 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:51.282+0000 7fa584b39640 1 --2- 172.21.15.169:0/1420550926 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa56406ae40 0x7fa56406d300 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa570034000 tx=0x7fa570039040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:51.286 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:51.284+0000 7fa575ffb640 1 -- 172.21.15.169:0/1420550926 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa57c055190 con 0x7fa56c004020 2023-10-26T01:31:51.531 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:51.529+0000 7fa58759e640 1 -- 172.21.15.169:0/1420550926 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7fa58004db10 con 0x7fa56c004020 2023-10-26T01:31:51.535 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:51.534+0000 7fa575ffb640 1 -- 172.21.15.169:0/1420550926 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v6) v1 ==== 90+0+0 (secure 0 0 0) 0x7fa57c014020 con 0x7fa56c004020 2023-10-26T01:31:51.539 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:51.537+0000 7fa58759e640 1 -- 172.21.15.169:0/1420550926 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa56406ae40 msgr2=0x7fa56406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:51.539 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:51.537+0000 7fa58759e640 1 --2- 172.21.15.169:0/1420550926 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa56406ae40 0x7fa56406d300 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa570034000 tx=0x7fa570039040 comp rx=0 tx=0).stop 2023-10-26T01:31:51.539 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:51.537+0000 7fa58759e640 1 -- 172.21.15.169:0/1420550926 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa56c004020 msgr2=0x7fa5800ad0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:51.539 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:51.537+0000 7fa58759e640 1 --2- 172.21.15.169:0/1420550926 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa56c004020 0x7fa5800ad0e0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fa57c00d950 tx=0x7fa57c00de10 comp rx=0 tx=0).stop 2023-10-26T01:31:51.539 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:51.537+0000 7fa58759e640 1 -- 172.21.15.169:0/1420550926 shutdown_connections 2023-10-26T01:31:51.540 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:51.537+0000 7fa58759e640 1 --2- 172.21.15.169:0/1420550926 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa56406ae40 0x7fa56406d300 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:51.540 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:51.537+0000 7fa58759e640 1 --2- 172.21.15.169:0/1420550926 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa5801593a0 0x7fa5800cb520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:51.540 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:51.537+0000 7fa58759e640 1 --2- 172.21.15.169:0/1420550926 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa58014f3e0 0x7fa580158e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:51.540 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:51.537+0000 7fa58759e640 1 --2- 172.21.15.169:0/1420550926 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa56c004020 0x7fa5800ad0e0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:51.540 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:51.537+0000 7fa58759e640 1 -- 172.21.15.169:0/1420550926 >> 172.21.15.169:0/1420550926 conn(0x7fa580069930 msgr2=0x7fa5800b07c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:51.541 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:51.537+0000 7fa58759e640 1 -- 172.21.15.169:0/1420550926 shutdown_connections 2023-10-26T01:31:51.541 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:51.537+0000 7fa58759e640 1 -- 172.21.15.169:0/1420550926 wait complete. 2023-10-26T01:31:51.557 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-10-26T01:31:51.768 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:51.766+0000 7f23f69d0640 1 -- 172.21.15.169:0/4025536292 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f23dc003970 msgr2=0x7f23dc003d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:51.768 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:51.766+0000 7f23f69d0640 1 --2- 172.21.15.169:0/4025536292 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f23dc003970 0x7f23dc003d70 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f23e0006fc0 tx=0x7f23dc0057f0 comp rx=0 tx=0).stop 2023-10-26T01:31:51.768 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:51.766+0000 7f23f69d0640 1 -- 172.21.15.169:0/4025536292 shutdown_connections 2023-10-26T01:31:51.769 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:51.766+0000 7f23f69d0640 1 --2- 172.21.15.169:0/4025536292 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f23dc003970 0x7f23dc003d70 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:51.769 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:51.766+0000 7f23f69d0640 1 -- 172.21.15.169:0/4025536292 >> 172.21.15.169:0/4025536292 conn(0x7f23f0069930 msgr2=0x7f23f0069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:51.769 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:51.766+0000 7f23f69d0640 1 -- 172.21.15.169:0/4025536292 shutdown_connections 2023-10-26T01:31:51.769 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:51.766+0000 7f23f69d0640 1 -- 172.21.15.169:0/4025536292 wait complete. 2023-10-26T01:31:51.772 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:51.768+0000 7f23f69d0640 1 Processor -- start 2023-10-26T01:31:51.772 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:51.768+0000 7f23f69d0640 1 -- start start 2023-10-26T01:31:51.772 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:51.769+0000 7f23f69d0640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f23dc003970 0x7f23f01f0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:51.773 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:51.769+0000 7f23f69d0640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f23f014fd70 0x7f23f00adf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:51.773 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:51.769+0000 7f23f69d0640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f23f00ae460 0x7f23f015a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:51.773 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:51.769+0000 7f23f69d0640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23f01f0cb0 con 0x7f23dc003970 2023-10-26T01:31:51.773 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:51.769+0000 7f23f69d0640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23f00af940 con 0x7f23f00ae460 2023-10-26T01:31:51.774 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:51.769+0000 7f23f69d0640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23f00afc20 con 0x7f23f014fd70 2023-10-26T01:31:51.774 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:51.769+0000 7f23effff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f23dc003970 0x7f23f01f0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:51.774 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:51.769+0000 7f23effff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f23dc003970 0x7f23f01f0770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:35842/0 (socket says 172.21.15.169:35842) 2023-10-26T01:31:51.775 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:51.769+0000 7f23effff640 1 -- 172.21.15.169:0/1840813303 learned_addr learned my addr 172.21.15.169:0/1840813303 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:31:51.775 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:51.769+0000 7f23f4f6d640 1 --2- 172.21.15.169:0/1840813303 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f23f00ae460 0x7f23f015a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:51.775 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:51.770+0000 7f23e7fff640 1 --2- 172.21.15.169:0/1840813303 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f23f014fd70 0x7f23f00adf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:51.775 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:51.770+0000 7f23effff640 1 -- 172.21.15.169:0/1840813303 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f23f014fd70 msgr2=0x7f23f00adf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:51.776 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:51.770+0000 7f23effff640 1 --2- 172.21.15.169:0/1840813303 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f23f014fd70 0x7f23f00adf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:51.776 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:51.770+0000 7f23effff640 1 -- 172.21.15.169:0/1840813303 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f23f00ae460 msgr2=0x7f23f015a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:51.776 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:51.770+0000 7f23effff640 1 --2- 172.21.15.169:0/1840813303 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f23f00ae460 0x7f23f015a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:51.776 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:51.770+0000 7f23effff640 1 -- 172.21.15.169:0/1840813303 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23f00ad720 con 0x7f23dc003970 2023-10-26T01:31:51.777 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:51.770+0000 7f23f4f6d640 1 --2- 172.21.15.169:0/1840813303 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f23f00ae460 0x7f23f015a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:31:51.777 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:51.770+0000 7f23effff640 1 --2- 172.21.15.169:0/1840813303 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f23dc003970 0x7f23f01f0770 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f23e002c900 tx=0x7f23e0001660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:51.777 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:51.770+0000 7f23ed7fa640 1 -- 172.21.15.169:0/1840813303 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23e0006230 con 0x7f23dc003970 2023-10-26T01:31:51.777 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:51.770+0000 7f23ed7fa640 1 -- 172.21.15.169:0/1840813303 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f23e0006390 con 0x7f23dc003970 2023-10-26T01:31:51.777 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:51.770+0000 7f23ed7fa640 1 -- 172.21.15.169:0/1840813303 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23e00035f0 con 0x7f23dc003970 2023-10-26T01:31:51.778 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:51.770+0000 7f23f69d0640 1 -- 172.21.15.169:0/1840813303 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23f00ada00 con 0x7f23dc003970 2023-10-26T01:31:51.778 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:51.771+0000 7f23f69d0640 1 -- 172.21.15.169:0/1840813303 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23f00af5c0 con 0x7f23dc003970 2023-10-26T01:31:51.779 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:51.771+0000 7f23f69d0640 1 -- 172.21.15.169:0/1840813303 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23f00b1ea0 con 0x7f23dc003970 2023-10-26T01:31:51.779 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:51.772+0000 7f23ed7fa640 1 -- 172.21.15.169:0/1840813303 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f23e0031030 con 0x7f23dc003970 2023-10-26T01:31:51.779 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:51.773+0000 7f23ed7fa640 1 --2- 172.21.15.169:0/1840813303 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f23d006ae40 0x7f23d006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:51.780 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:51.773+0000 7f23ed7fa640 1 -- 172.21.15.169:0/1840813303 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f23e00aa1c0 con 0x7f23dc003970 2023-10-26T01:31:51.780 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:51.774+0000 7f23e7fff640 1 --2- 172.21.15.169:0/1840813303 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f23d006ae40 0x7f23d006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:51.780 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:51.775+0000 7f23e7fff640 1 --2- 172.21.15.169:0/1840813303 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f23d006ae40 0x7f23d006d300 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f23f015b340 tx=0x7f23d8006ac0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:51.780 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:51.778+0000 7f23ed7fa640 1 -- 172.21.15.169:0/1840813303 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f23e0075650 con 0x7f23dc003970 2023-10-26T01:31:52.041 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:52.040+0000 7f23f69d0640 1 -- 172.21.15.169:0/1840813303 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f23f004db10 con 0x7f23dc003970 2023-10-26T01:31:52.045 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:52.044+0000 7f23ed7fa640 1 -- 172.21.15.169:0/1840813303 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7f23e00a8e60 con 0x7f23dc003970 2023-10-26T01:31:52.048 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:52.046+0000 7f23f69d0640 1 -- 172.21.15.169:0/1840813303 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f23d006ae40 msgr2=0x7f23d006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:52.048 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:52.046+0000 7f23f69d0640 1 --2- 172.21.15.169:0/1840813303 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f23d006ae40 0x7f23d006d300 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f23f015b340 tx=0x7f23d8006ac0 comp rx=0 tx=0).stop 2023-10-26T01:31:52.048 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:52.046+0000 7f23f69d0640 1 -- 172.21.15.169:0/1840813303 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f23dc003970 msgr2=0x7f23f01f0770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:52.048 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:52.046+0000 7f23f69d0640 1 --2- 172.21.15.169:0/1840813303 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f23dc003970 0x7f23f01f0770 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f23e002c900 tx=0x7f23e0001660 comp rx=0 tx=0).stop 2023-10-26T01:31:52.049 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:52.046+0000 7f23f69d0640 1 -- 172.21.15.169:0/1840813303 shutdown_connections 2023-10-26T01:31:52.049 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:52.046+0000 7f23f69d0640 1 --2- 172.21.15.169:0/1840813303 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f23d006ae40 0x7f23d006d300 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:52.049 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:52.046+0000 7f23f69d0640 1 --2- 172.21.15.169:0/1840813303 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f23f00ae460 0x7f23f015a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:52.049 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:52.046+0000 7f23f69d0640 1 --2- 172.21.15.169:0/1840813303 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f23f014fd70 0x7f23f00adf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:52.049 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:52.046+0000 7f23f69d0640 1 --2- 172.21.15.169:0/1840813303 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f23dc003970 0x7f23f01f0770 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:52.050 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:52.046+0000 7f23f69d0640 1 -- 172.21.15.169:0/1840813303 >> 172.21.15.169:0/1840813303 conn(0x7f23f0069930 msgr2=0x7f23f0151910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:52.050 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:52.046+0000 7f23f69d0640 1 -- 172.21.15.169:0/1840813303 shutdown_connections 2023-10-26T01:31:52.050 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:52.046+0000 7f23f69d0640 1 -- 172.21.15.169:0/1840813303 wait complete. 2023-10-26T01:31:52.068 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-10-26T01:31:52.289 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:52.287+0000 7f5357322640 1 -- 172.21.15.169:0/2679427579 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f535006cba0 msgr2=0x7f535015b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:52.290 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:52.287+0000 7f5357322640 1 --2- 172.21.15.169:0/2679427579 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f535006cba0 0x7f535015b140 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f534c005fd0 tx=0x7f534c02bcf0 comp rx=0 tx=0).stop 2023-10-26T01:31:52.290 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:52.287+0000 7f5357322640 1 -- 172.21.15.169:0/2679427579 shutdown_connections 2023-10-26T01:31:52.290 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:52.287+0000 7f5357322640 1 --2- 172.21.15.169:0/2679427579 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f53500b0300 0x7f53500b0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:52.291 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:52.287+0000 7f5357322640 1 --2- 172.21.15.169:0/2679427579 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f535006cba0 0x7f535015b140 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:52.291 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:52.287+0000 7f5357322640 1 -- 172.21.15.169:0/2679427579 >> 172.21.15.169:0/2679427579 conn(0x7f53500af110 msgr2=0x7f53500af520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:52.291 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:52.288+0000 7f5357322640 1 -- 172.21.15.169:0/2679427579 shutdown_connections 2023-10-26T01:31:52.291 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:52.288+0000 7f5357322640 1 -- 172.21.15.169:0/2679427579 wait complete. 2023-10-26T01:31:52.293 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:52.289+0000 7f5357322640 1 Processor -- start 2023-10-26T01:31:52.294 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:52.289+0000 7f5357322640 1 -- start start 2023-10-26T01:31:52.294 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:52.290+0000 7f5357322640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5350069a10 0x7f5350143020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:52.294 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:52.290+0000 7f5357322640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f535006cba0 0x7f535015acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:52.294 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:52.290+0000 7f5357322640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f53500b0300 0x7f53501f5890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:52.295 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:52.290+0000 7f5357322640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5350143560 con 0x7f5350069a10 2023-10-26T01:31:52.295 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:52.290+0000 7f5357322640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53501e11b0 con 0x7f535006cba0 2023-10-26T01:31:52.295 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:52.290+0000 7f5357322640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f535015a820 con 0x7f53500b0300 2023-10-26T01:31:52.295 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:52.291+0000 7f5356320640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5350069a10 0x7f5350143020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:52.295 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:52.291+0000 7f5356320640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5350069a10 0x7f5350143020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:35860/0 (socket says 172.21.15.169:35860) 2023-10-26T01:31:52.296 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:52.291+0000 7f5356320640 1 -- 172.21.15.169:0/227915796 learned_addr learned my addr 172.21.15.169:0/227915796 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:31:52.296 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:52.291+0000 7f5355b1f640 1 --2- 172.21.15.169:0/227915796 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f535006cba0 0x7f535015acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:52.296 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:52.291+0000 7f5356b21640 1 --2- 172.21.15.169:0/227915796 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f53500b0300 0x7f53501f5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:52.296 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:52.291+0000 7f5356320640 1 -- 172.21.15.169:0/227915796 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f53500b0300 msgr2=0x7f53501f5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:52.297 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:52.291+0000 7f5356320640 1 --2- 172.21.15.169:0/227915796 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f53500b0300 0x7f53501f5890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:52.297 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:52.291+0000 7f5356320640 1 -- 172.21.15.169:0/227915796 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f535006cba0 msgr2=0x7f535015acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:52.297 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:52.291+0000 7f5356320640 1 --2- 172.21.15.169:0/227915796 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f535006cba0 0x7f535015acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:52.297 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:52.291+0000 7f5356320640 1 -- 172.21.15.169:0/227915796 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f534c009560 con 0x7f5350069a10 2023-10-26T01:31:52.297 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:52.292+0000 7f5356320640 1 --2- 172.21.15.169:0/227915796 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5350069a10 0x7f5350143020 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f534c036040 tx=0x7f534c02dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:52.298 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:52.292+0000 7f5346ffd640 1 -- 172.21.15.169:0/227915796 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f534c005650 con 0x7f5350069a10 2023-10-26T01:31:52.298 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:52.292+0000 7f5346ffd640 1 -- 172.21.15.169:0/227915796 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f534c0038f0 con 0x7f5350069a10 2023-10-26T01:31:52.298 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:52.292+0000 7f5346ffd640 1 -- 172.21.15.169:0/227915796 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f534c003ca0 con 0x7f5350069a10 2023-10-26T01:31:52.298 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:52.292+0000 7f5357322640 1 -- 172.21.15.169:0/227915796 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f535015b210 con 0x7f5350069a10 2023-10-26T01:31:52.299 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:52.292+0000 7f5357322640 1 -- 172.21.15.169:0/227915796 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53501e1400 con 0x7f5350069a10 2023-10-26T01:31:52.299 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:52.293+0000 7f5357322640 1 -- 172.21.15.169:0/227915796 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f535015a9f0 con 0x7f5350069a10 2023-10-26T01:31:52.300 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:52.294+0000 7f5346ffd640 1 -- 172.21.15.169:0/227915796 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f534c03e450 con 0x7f5350069a10 2023-10-26T01:31:52.300 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:52.295+0000 7f5346ffd640 1 --2- 172.21.15.169:0/227915796 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f533406ae40 0x7f533406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:52.300 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:52.295+0000 7f5346ffd640 1 -- 172.21.15.169:0/227915796 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f534c0abd20 con 0x7f5350069a10 2023-10-26T01:31:52.300 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:52.295+0000 7f5355b1f640 1 --2- 172.21.15.169:0/227915796 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f533406ae40 0x7f533406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:52.300 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:52.297+0000 7f5355b1f640 1 --2- 172.21.15.169:0/227915796 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f533406ae40 0x7f533406d300 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f53501e3ba0 tx=0x7f5340008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:52.301 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:52.300+0000 7f5346ffd640 1 -- 172.21.15.169:0/227915796 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f534c0771a0 con 0x7f5350069a10 2023-10-26T01:31:52.564 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:52.562+0000 7f5357322640 1 -- 172.21.15.169:0/227915796 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f5350073c10 con 0x7f5350069a10 2023-10-26T01:31:52.568 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:52.567+0000 7f5346ffd640 1 -- 172.21.15.169:0/227915796 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7f534c042020 con 0x7f5350069a10 2023-10-26T01:31:52.572 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:52.570+0000 7f5357322640 1 -- 172.21.15.169:0/227915796 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f533406ae40 msgr2=0x7f533406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:52.572 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:52.570+0000 7f5357322640 1 --2- 172.21.15.169:0/227915796 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f533406ae40 0x7f533406d300 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f53501e3ba0 tx=0x7f5340008040 comp rx=0 tx=0).stop 2023-10-26T01:31:52.572 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:52.570+0000 7f5357322640 1 -- 172.21.15.169:0/227915796 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5350069a10 msgr2=0x7f5350143020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:52.572 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:52.570+0000 7f5357322640 1 --2- 172.21.15.169:0/227915796 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5350069a10 0x7f5350143020 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f534c036040 tx=0x7f534c02dcc0 comp rx=0 tx=0).stop 2023-10-26T01:31:52.573 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:52.570+0000 7f5357322640 1 -- 172.21.15.169:0/227915796 shutdown_connections 2023-10-26T01:31:52.573 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:52.570+0000 7f5357322640 1 --2- 172.21.15.169:0/227915796 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f533406ae40 0x7f533406d300 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:52.573 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:52.570+0000 7f5357322640 1 --2- 172.21.15.169:0/227915796 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f53500b0300 0x7f53501f5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:52.573 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:52.570+0000 7f5357322640 1 --2- 172.21.15.169:0/227915796 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f535006cba0 0x7f535015acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:52.574 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:52.570+0000 7f5357322640 1 --2- 172.21.15.169:0/227915796 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5350069a10 0x7f5350143020 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:52.574 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:52.570+0000 7f5357322640 1 -- 172.21.15.169:0/227915796 >> 172.21.15.169:0/227915796 conn(0x7f53500af110 msgr2=0x7f53500adcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:52.574 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:52.570+0000 7f5357322640 1 -- 172.21.15.169:0/227915796 shutdown_connections 2023-10-26T01:31:52.574 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:52.570+0000 7f5357322640 1 -- 172.21.15.169:0/227915796 wait complete. 2023-10-26T01:31:52.589 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-10-26T01:31:52.803 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:52.800+0000 7f20fffff640 1 -- 172.21.15.169:0/1655363727 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f21000b0300 msgr2=0x7f21000b0780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:52.803 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:52.800+0000 7f20fffff640 1 --2- 172.21.15.169:0/1655363727 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f21000b0300 0x7f21000b0780 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f20f400a940 tx=0x7f20f402d000 comp rx=0 tx=0).stop 2023-10-26T01:31:52.803 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:52.801+0000 7f20fffff640 1 -- 172.21.15.169:0/1655363727 shutdown_connections 2023-10-26T01:31:52.804 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:52.801+0000 7f20fffff640 1 --2- 172.21.15.169:0/1655363727 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f21000b0300 0x7f21000b0780 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:52.804 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:52.801+0000 7f20fffff640 1 --2- 172.21.15.169:0/1655363727 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f210006cba0 0x7f210015b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:52.804 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:52.801+0000 7f20fffff640 1 -- 172.21.15.169:0/1655363727 >> 172.21.15.169:0/1655363727 conn(0x7f21000af110 msgr2=0x7f21000af520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:52.804 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:52.801+0000 7f20fffff640 1 -- 172.21.15.169:0/1655363727 shutdown_connections 2023-10-26T01:31:52.804 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:52.801+0000 7f20fffff640 1 -- 172.21.15.169:0/1655363727 wait complete. 2023-10-26T01:31:52.805 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:52.803+0000 7f20fffff640 1 Processor -- start 2023-10-26T01:31:52.805 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:52.803+0000 7f20fffff640 1 -- start start 2023-10-26T01:31:52.807 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:52.804+0000 7f20fffff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2100069a10 0x7f210014b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:52.807 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:52.804+0000 7f20fffff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f210006cba0 0x7f21000cba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:52.808 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:52.804+0000 7f20fffff640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f21000b0300 0x7f21000cbf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:52.808 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:52.804+0000 7f20fffff640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21000ade20 con 0x7f2100069a10 2023-10-26T01:31:52.808 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:52.804+0000 7f20fffff640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21000972d0 con 0x7f210006cba0 2023-10-26T01:31:52.808 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:52.804+0000 7f20fffff640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21000975b0 con 0x7f21000b0300 2023-10-26T01:31:52.809 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:52.804+0000 7f20feffd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2100069a10 0x7f210014b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:52.809 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:52.804+0000 7f20feffd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2100069a10 0x7f210014b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:35862/0 (socket says 172.21.15.169:35862) 2023-10-26T01:31:52.809 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:52.804+0000 7f20feffd640 1 -- 172.21.15.169:0/3771355157 learned_addr learned my addr 172.21.15.169:0/3771355157 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:31:52.809 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:52.805+0000 7f20fe7fc640 1 --2- 172.21.15.169:0/3771355157 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f210006cba0 0x7f21000cba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:52.809 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:52.805+0000 7f20ff7fe640 1 --2- 172.21.15.169:0/3771355157 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f21000b0300 0x7f21000cbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:52.810 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:52.805+0000 7f20feffd640 1 -- 172.21.15.169:0/3771355157 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f21000b0300 msgr2=0x7f21000cbf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:52.810 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:52.805+0000 7f20feffd640 1 --2- 172.21.15.169:0/3771355157 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f21000b0300 0x7f21000cbf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:52.811 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:52.805+0000 7f20feffd640 1 -- 172.21.15.169:0/3771355157 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f210006cba0 msgr2=0x7f21000cba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:52.811 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:52.805+0000 7f20feffd640 1 --2- 172.21.15.169:0/3771355157 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f210006cba0 0x7f21000cba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:52.811 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:52.805+0000 7f20feffd640 1 -- 172.21.15.169:0/3771355157 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20f400a5a0 con 0x7f2100069a10 2023-10-26T01:31:52.812 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:52.805+0000 7f20feffd640 1 --2- 172.21.15.169:0/3771355157 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2100069a10 0x7f210014b940 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f20f000e9c0 tx=0x7f20f000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:52.812 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:52.805+0000 7f20e77fe640 1 -- 172.21.15.169:0/3771355157 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f000ccc0 con 0x7f2100069a10 2023-10-26T01:31:52.812 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:52.805+0000 7f20e77fe640 1 -- 172.21.15.169:0/3771355157 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f20f000ce20 con 0x7f2100069a10 2023-10-26T01:31:52.812 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:52.805+0000 7f20fffff640 1 -- 172.21.15.169:0/3771355157 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2100097890 con 0x7f2100069a10 2023-10-26T01:31:52.812 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:52.805+0000 7f20e77fe640 1 -- 172.21.15.169:0/3771355157 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f0005280 con 0x7f2100069a10 2023-10-26T01:31:52.813 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:52.806+0000 7f20fffff640 1 -- 172.21.15.169:0/3771355157 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f210014be80 con 0x7f2100069a10 2023-10-26T01:31:52.814 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:52.807+0000 7f20fffff640 1 -- 172.21.15.169:0/3771355157 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21000ab200 con 0x7f2100069a10 2023-10-26T01:31:52.814 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:52.808+0000 7f20e77fe640 1 -- 172.21.15.169:0/3771355157 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f20f00053e0 con 0x7f2100069a10 2023-10-26T01:31:52.814 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:52.808+0000 7f20e77fe640 1 --2- 172.21.15.169:0/3771355157 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f20ec06ae70 0x7f20ec06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:52.814 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:52.809+0000 7f20e77fe640 1 -- 172.21.15.169:0/3771355157 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f20f0014070 con 0x7f2100069a10 2023-10-26T01:31:52.815 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:52.809+0000 7f20fe7fc640 1 --2- 172.21.15.169:0/3771355157 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f20ec06ae70 0x7f20ec06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:52.815 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:52.810+0000 7f20fe7fc640 1 --2- 172.21.15.169:0/3771355157 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f20ec06ae70 0x7f20ec06d330 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f20f4037000 tx=0x7f20f4004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:52.816 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:52.815+0000 7f20e77fe640 1 -- 172.21.15.169:0/3771355157 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f20f0090080 con 0x7f2100069a10 2023-10-26T01:31:53.075 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:53.073+0000 7f20fffff640 1 -- 172.21.15.169:0/3771355157 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f2100073c10 con 0x7f2100069a10 2023-10-26T01:31:53.078 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:53.077+0000 7f20e77fe640 1 -- 172.21.15.169:0/3771355157 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v9) v1 ==== 97+0+0 (secure 0 0 0) 0x7f20f0055140 con 0x7f2100069a10 2023-10-26T01:31:53.081 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:53.079+0000 7f20fffff640 1 -- 172.21.15.169:0/3771355157 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f20ec06ae70 msgr2=0x7f20ec06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:53.082 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:53.079+0000 7f20fffff640 1 --2- 172.21.15.169:0/3771355157 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f20ec06ae70 0x7f20ec06d330 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f20f4037000 tx=0x7f20f4004730 comp rx=0 tx=0).stop 2023-10-26T01:31:53.082 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:53.079+0000 7f20fffff640 1 -- 172.21.15.169:0/3771355157 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2100069a10 msgr2=0x7f210014b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:53.082 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:53.080+0000 7f20fffff640 1 --2- 172.21.15.169:0/3771355157 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2100069a10 0x7f210014b940 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f20f000e9c0 tx=0x7f20f000ee80 comp rx=0 tx=0).stop 2023-10-26T01:31:53.082 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:53.080+0000 7f20fffff640 1 -- 172.21.15.169:0/3771355157 shutdown_connections 2023-10-26T01:31:53.083 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:53.080+0000 7f20fffff640 1 --2- 172.21.15.169:0/3771355157 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f20ec06ae70 0x7f20ec06d330 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:53.083 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:53.080+0000 7f20fffff640 1 --2- 172.21.15.169:0/3771355157 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f21000b0300 0x7f21000cbf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:53.083 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:53.080+0000 7f20fffff640 1 --2- 172.21.15.169:0/3771355157 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f210006cba0 0x7f21000cba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:53.083 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:53.080+0000 7f20fffff640 1 --2- 172.21.15.169:0/3771355157 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2100069a10 0x7f210014b940 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:53.083 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:53.080+0000 7f20fffff640 1 -- 172.21.15.169:0/3771355157 >> 172.21.15.169:0/3771355157 conn(0x7f21000af110 msgr2=0x7f2100151460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:53.084 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:53.080+0000 7f20fffff640 1 -- 172.21.15.169:0/3771355157 shutdown_connections 2023-10-26T01:31:53.084 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:53.080+0000 7f20fffff640 1 -- 172.21.15.169:0/3771355157 wait complete. 2023-10-26T01:31:53.099 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-10-26T01:31:53.316 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:53.314+0000 7f581cd4c640 1 -- 172.21.15.169:0/2456144648 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f58181593e0 msgr2=0x7f58180ac200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:53.317 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:53.314+0000 7f581cd4c640 1 --2- 172.21.15.169:0/2456144648 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f58181593e0 0x7f58180ac200 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5808005fd0 tx=0x7f580802be20 comp rx=0 tx=0).stop 2023-10-26T01:31:53.317 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:53.314+0000 7f581cd4c640 1 -- 172.21.15.169:0/2456144648 shutdown_connections 2023-10-26T01:31:53.317 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:53.314+0000 7f581cd4c640 1 --2- 172.21.15.169:0/2456144648 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f58180b0c80 0x7f58180cd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:53.317 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:53.314+0000 7f581cd4c640 1 --2- 172.21.15.169:0/2456144648 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f58181593e0 0x7f58180ac200 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:53.318 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:53.314+0000 7f581cd4c640 1 -- 172.21.15.169:0/2456144648 >> 172.21.15.169:0/2456144648 conn(0x7f5818069930 msgr2=0x7f5818069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:53.318 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:53.315+0000 7f581cd4c640 1 -- 172.21.15.169:0/2456144648 shutdown_connections 2023-10-26T01:31:53.318 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:53.315+0000 7f581cd4c640 1 -- 172.21.15.169:0/2456144648 wait complete. 2023-10-26T01:31:53.319 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:53.317+0000 7f581cd4c640 1 Processor -- start 2023-10-26T01:31:53.319 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:53.317+0000 7f581cd4c640 1 -- start start 2023-10-26T01:31:53.321 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:53.318+0000 7f581cd4c640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f58180b0c80 0x7f58180ade00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:53.322 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:53.318+0000 7f581cd4c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f58181593e0 0x7f58180af7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:53.322 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:53.318+0000 7f581cd4c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58181597c0 0x7f5818071600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:53.322 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:53.318+0000 7f581cd4c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5818157210 con 0x7f58181597c0 2023-10-26T01:31:53.322 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:53.318+0000 7f581cd4c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58180ad080 con 0x7f58181593e0 2023-10-26T01:31:53.323 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:53.318+0000 7f581cd4c640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58180ad340 con 0x7f58180b0c80 2023-10-26T01:31:53.323 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:53.318+0000 7f5816d9d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58181597c0 0x7f5818071600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:53.323 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:53.319+0000 7f5816d9d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58181597c0 0x7f5818071600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:35870/0 (socket says 172.21.15.169:35870) 2023-10-26T01:31:53.323 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:53.319+0000 7f5816d9d640 1 -- 172.21.15.169:0/2298396215 learned_addr learned my addr 172.21.15.169:0/2298396215 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:31:53.323 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:53.319+0000 7f581659c640 1 --2- 172.21.15.169:0/2298396215 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f58180b0c80 0x7f58180ade00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:53.324 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:53.319+0000 7f580ffff640 1 --2- 172.21.15.169:0/2298396215 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f58181593e0 0x7f58180af7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:53.324 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:53.319+0000 7f5816d9d640 1 -- 172.21.15.169:0/2298396215 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f58180b0c80 msgr2=0x7f58180ade00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:53.324 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:53.319+0000 7f5816d9d640 1 --2- 172.21.15.169:0/2298396215 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f58180b0c80 0x7f58180ade00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:53.324 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:53.319+0000 7f5816d9d640 1 -- 172.21.15.169:0/2298396215 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f58181593e0 msgr2=0x7f58180af7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:53.325 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:53.319+0000 7f5816d9d640 1 --2- 172.21.15.169:0/2298396215 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f58181593e0 0x7f58180af7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:53.325 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:53.319+0000 7f5816d9d640 1 -- 172.21.15.169:0/2298396215 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5810009840 con 0x7f58181597c0 2023-10-26T01:31:53.325 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:53.319+0000 7f5816d9d640 1 --2- 172.21.15.169:0/2298396215 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58181597c0 0x7f5818071600 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f581000efd0 tx=0x7f581000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:53.325 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:53.320+0000 7f580effd640 1 -- 172.21.15.169:0/2298396215 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5810009b40 con 0x7f58181597c0 2023-10-26T01:31:53.325 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:53.320+0000 7f580effd640 1 -- 172.21.15.169:0/2298396215 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5810003c60 con 0x7f58181597c0 2023-10-26T01:31:53.326 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:53.320+0000 7f581cd4c640 1 -- 172.21.15.169:0/2298396215 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5808009560 con 0x7f58181597c0 2023-10-26T01:31:53.326 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:53.320+0000 7f580effd640 1 -- 172.21.15.169:0/2298396215 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5810004020 con 0x7f58181597c0 2023-10-26T01:31:53.326 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:53.320+0000 7f581cd4c640 1 -- 172.21.15.169:0/2298396215 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58180ad900 con 0x7f58181597c0 2023-10-26T01:31:53.327 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:53.321+0000 7f581cd4c640 1 -- 172.21.15.169:0/2298396215 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5818066e60 con 0x7f58181597c0 2023-10-26T01:31:53.327 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:53.322+0000 7f580effd640 1 -- 172.21.15.169:0/2298396215 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f58100041a0 con 0x7f58181597c0 2023-10-26T01:31:53.328 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:53.323+0000 7f580effd640 1 --2- 172.21.15.169:0/2298396215 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f57f806ae40 0x7f57f806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:53.328 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:53.323+0000 7f580effd640 1 -- 172.21.15.169:0/2298396215 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5810014070 con 0x7f58181597c0 2023-10-26T01:31:53.328 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:53.324+0000 7f581659c640 1 --2- 172.21.15.169:0/2298396215 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f57f806ae40 0x7f57f806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:53.328 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:53.325+0000 7f581659c640 1 --2- 172.21.15.169:0/2298396215 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f57f806ae40 0x7f57f806d300 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5808035000 tx=0x7f5808004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:53.329 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:53.327+0000 7f580effd640 1 -- 172.21.15.169:0/2298396215 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f58100557e0 con 0x7f58181597c0 2023-10-26T01:31:53.586 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:53.585+0000 7f581cd4c640 1 -- 172.21.15.169:0/2298396215 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f581804db10 con 0x7f58181597c0 2023-10-26T01:31:53.590 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:53.589+0000 7f580effd640 1 -- 172.21.15.169:0/2298396215 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7f5810058e00 con 0x7f58181597c0 2023-10-26T01:31:53.593 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:53.591+0000 7f581cd4c640 1 -- 172.21.15.169:0/2298396215 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f57f806ae40 msgr2=0x7f57f806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:53.594 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:53.591+0000 7f581cd4c640 1 --2- 172.21.15.169:0/2298396215 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f57f806ae40 0x7f57f806d300 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5808035000 tx=0x7f5808004730 comp rx=0 tx=0).stop 2023-10-26T01:31:53.594 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:53.591+0000 7f581cd4c640 1 -- 172.21.15.169:0/2298396215 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58181597c0 msgr2=0x7f5818071600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:53.594 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:53.591+0000 7f581cd4c640 1 --2- 172.21.15.169:0/2298396215 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58181597c0 0x7f5818071600 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f581000efd0 tx=0x7f581000c6a0 comp rx=0 tx=0).stop 2023-10-26T01:31:53.594 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:53.592+0000 7f581cd4c640 1 -- 172.21.15.169:0/2298396215 shutdown_connections 2023-10-26T01:31:53.594 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:53.592+0000 7f581cd4c640 1 --2- 172.21.15.169:0/2298396215 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f57f806ae40 0x7f57f806d300 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:53.595 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:53.592+0000 7f581cd4c640 1 --2- 172.21.15.169:0/2298396215 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58181597c0 0x7f5818071600 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:53.595 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:53.592+0000 7f581cd4c640 1 --2- 172.21.15.169:0/2298396215 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f58181593e0 0x7f58180af7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:53.595 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:53.592+0000 7f581cd4c640 1 --2- 172.21.15.169:0/2298396215 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f58180b0c80 0x7f58180ade00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:53.595 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:53.592+0000 7f581cd4c640 1 -- 172.21.15.169:0/2298396215 >> 172.21.15.169:0/2298396215 conn(0x7f5818069930 msgr2=0x7f5818159c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:53.595 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:53.592+0000 7f581cd4c640 1 -- 172.21.15.169:0/2298396215 shutdown_connections 2023-10-26T01:31:53.596 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:53.592+0000 7f581cd4c640 1 -- 172.21.15.169:0/2298396215 wait complete. 2023-10-26T01:31:53.610 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:31:53.819 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:53.817+0000 7f9baa3c9640 1 -- 172.21.15.169:0/3484269018 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9b8c003970 msgr2=0x7f9b8c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:53.819 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:53.817+0000 7f9baa3c9640 1 --2- 172.21.15.169:0/3484269018 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9b8c003970 0x7f9b8c003d90 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f9b94009b50 tx=0x7f9b8c0057f0 comp rx=0 tx=0).stop 2023-10-26T01:31:53.820 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:53.817+0000 7f9baa3c9640 1 -- 172.21.15.169:0/3484269018 shutdown_connections 2023-10-26T01:31:53.820 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:53.817+0000 7f9baa3c9640 1 --2- 172.21.15.169:0/3484269018 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9b8c003970 0x7f9b8c003d90 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:53.820 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:53.817+0000 7f9baa3c9640 1 -- 172.21.15.169:0/3484269018 >> 172.21.15.169:0/3484269018 conn(0x7f9ba4069930 msgr2=0x7f9ba4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:53.820 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:53.817+0000 7f9baa3c9640 1 -- 172.21.15.169:0/3484269018 shutdown_connections 2023-10-26T01:31:53.821 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:53.817+0000 7f9baa3c9640 1 -- 172.21.15.169:0/3484269018 wait complete. 2023-10-26T01:31:53.822 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:53.819+0000 7f9baa3c9640 1 Processor -- start 2023-10-26T01:31:53.822 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:53.819+0000 7f9baa3c9640 1 -- start start 2023-10-26T01:31:53.822 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:53.820+0000 7f9baa3c9640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9b8c003970 0x7f9ba41f2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:53.822 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:53.820+0000 7f9baa3c9640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9ba40706f0 0x7f9ba41f5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:53.823 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:53.820+0000 7f9baa3c9640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9ba41f5db0 0x7f9ba40cb9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:53.823 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:53.820+0000 7f9baa3c9640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba4148250 con 0x7f9b8c003970 2023-10-26T01:31:53.823 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:53.820+0000 7f9baa3c9640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba40c7380 con 0x7f9ba41f5db0 2023-10-26T01:31:53.823 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:53.820+0000 7f9baa3c9640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba41f6230 con 0x7f9ba40706f0 2023-10-26T01:31:53.823 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:53.820+0000 7f9ba3fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9b8c003970 0x7f9ba41f2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:53.824 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:53.820+0000 7f9ba3fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9b8c003970 0x7f9ba41f2e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:35902/0 (socket says 172.21.15.169:35902) 2023-10-26T01:31:53.824 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:53.820+0000 7f9ba3fff640 1 -- 172.21.15.169:0/3111384654 learned_addr learned my addr 172.21.15.169:0/3111384654 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:31:53.825 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:53.821+0000 7f9ba3fff640 1 -- 172.21.15.169:0/3111384654 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9ba40706f0 msgr2=0x7f9ba41f5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:53.825 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:53.821+0000 7f9ba3fff640 1 --2- 172.21.15.169:0/3111384654 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9ba40706f0 0x7f9ba41f5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:53.825 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:53.821+0000 7f9ba3fff640 1 -- 172.21.15.169:0/3111384654 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9ba41f5db0 msgr2=0x7f9ba40cb9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:53.826 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:53.821+0000 7f9ba3fff640 1 --2- 172.21.15.169:0/3111384654 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9ba41f5db0 0x7f9ba40cb9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:53.826 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:53.821+0000 7f9ba3fff640 1 -- 172.21.15.169:0/3111384654 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ba41486e0 con 0x7f9b8c003970 2023-10-26T01:31:53.826 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:53.821+0000 7f9ba3fff640 1 --2- 172.21.15.169:0/3111384654 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9b8c003970 0x7f9ba41f2e30 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f9b94009c70 tx=0x7f9b94033c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:53.826 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:53.821+0000 7f9ba0ff9640 1 -- 172.21.15.169:0/3111384654 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b94004220 con 0x7f9b8c003970 2023-10-26T01:31:53.827 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:53.821+0000 7f9ba0ff9640 1 -- 172.21.15.169:0/3111384654 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9b940043a0 con 0x7f9b8c003970 2023-10-26T01:31:53.827 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:53.821+0000 7f9baa3c9640 1 -- 172.21.15.169:0/3111384654 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ba4070e60 con 0x7f9b8c003970 2023-10-26T01:31:53.827 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:53.821+0000 7f9ba0ff9640 1 -- 172.21.15.169:0/3111384654 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b9403d940 con 0x7f9b8c003970 2023-10-26T01:31:53.827 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:53.821+0000 7f9baa3c9640 1 -- 172.21.15.169:0/3111384654 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ba4071420 con 0x7f9b8c003970 2023-10-26T01:31:53.828 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:53.822+0000 7f9baa3c9640 1 -- 172.21.15.169:0/3111384654 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ba4148490 con 0x7f9b8c003970 2023-10-26T01:31:53.828 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:53.823+0000 7f9ba0ff9640 1 -- 172.21.15.169:0/3111384654 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f9b9403daa0 con 0x7f9b8c003970 2023-10-26T01:31:53.829 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:53.825+0000 7f9ba0ff9640 1 --2- 172.21.15.169:0/3111384654 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9b9006ae70 0x7f9b9006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:53.829 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:53.825+0000 7f9ba0ff9640 1 -- 172.21.15.169:0/3111384654 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f9b940ac060 con 0x7f9b8c003970 2023-10-26T01:31:53.829 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:53.825+0000 7f9ba37fe640 1 --2- 172.21.15.169:0/3111384654 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9b9006ae70 0x7f9b9006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:53.829 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:53.826+0000 7f9ba37fe640 1 --2- 172.21.15.169:0/3111384654 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9b9006ae70 0x7f9b9006d330 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9ba406bea0 tx=0x7f9b980079d0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:53.831 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:53.829+0000 7f9ba0ff9640 1 -- 172.21.15.169:0/3111384654 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9b940deca0 con 0x7f9b8c003970 2023-10-26T01:31:54.021 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.019+0000 7f9baa3c9640 1 -- 172.21.15.169:0/3111384654 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9ba4071640 con 0x7f9b8c003970 2023-10-26T01:31:54.023 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.022+0000 7f9ba0ff9640 1 -- 172.21.15.169:0/3111384654 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) v1 ==== 74+0+12905 (secure 0 0 0) 0x7f9b9404a020 con 0x7f9b8c003970 2023-10-26T01:31:54.024 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:31:54.024 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":31,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:31:49.936159+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:31:54.026 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.024+0000 7f9baa3c9640 1 -- 172.21.15.169:0/3111384654 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9b9006ae70 msgr2=0x7f9b9006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:54.026 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.024+0000 7f9baa3c9640 1 --2- 172.21.15.169:0/3111384654 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9b9006ae70 0x7f9b9006d330 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9ba406bea0 tx=0x7f9b980079d0 comp rx=0 tx=0).stop 2023-10-26T01:31:54.026 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.024+0000 7f9baa3c9640 1 -- 172.21.15.169:0/3111384654 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9b8c003970 msgr2=0x7f9ba41f2e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:54.027 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.024+0000 7f9baa3c9640 1 --2- 172.21.15.169:0/3111384654 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9b8c003970 0x7f9ba41f2e30 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f9b94009c70 tx=0x7f9b94033c30 comp rx=0 tx=0).stop 2023-10-26T01:31:54.027 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.024+0000 7f9baa3c9640 1 -- 172.21.15.169:0/3111384654 shutdown_connections 2023-10-26T01:31:54.027 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.024+0000 7f9baa3c9640 1 --2- 172.21.15.169:0/3111384654 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9b9006ae70 0x7f9b9006d330 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:54.027 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.024+0000 7f9baa3c9640 1 --2- 172.21.15.169:0/3111384654 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9ba41f5db0 0x7f9ba40cb9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:54.027 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.024+0000 7f9baa3c9640 1 --2- 172.21.15.169:0/3111384654 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9ba40706f0 0x7f9ba41f5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:54.028 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.024+0000 7f9baa3c9640 1 --2- 172.21.15.169:0/3111384654 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9b8c003970 0x7f9ba41f2e30 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:54.028 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.024+0000 7f9baa3c9640 1 -- 172.21.15.169:0/3111384654 >> 172.21.15.169:0/3111384654 conn(0x7f9ba4069930 msgr2=0x7f9ba40b0740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:54.028 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.024+0000 7f9baa3c9640 1 -- 172.21.15.169:0/3111384654 shutdown_connections 2023-10-26T01:31:54.028 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.025+0000 7f9baa3c9640 1 -- 172.21.15.169:0/3111384654 wait complete. 2023-10-26T01:31:54.046 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T01:30:32.200808+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T01:31:54.046 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T01:31:54.274 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.271+0000 7f44e1e40640 1 -- 172.21.15.169:0/2768008472 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44dc0cd690 msgr2=0x7f44dc0cdae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:54.274 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.271+0000 7f44e1e40640 1 --2- 172.21.15.169:0/2768008472 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44dc0cd690 0x7f44dc0cdae0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f44dc06d040 tx=0x7f44d402c110 comp rx=0 tx=0).stop 2023-10-26T01:31:54.275 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.271+0000 7f44e1e40640 1 -- 172.21.15.169:0/2768008472 shutdown_connections 2023-10-26T01:31:54.275 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.271+0000 7f44e1e40640 1 --2- 172.21.15.169:0/2768008472 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44dc0cd690 0x7f44dc0cdae0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:54.275 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.271+0000 7f44e1e40640 1 --2- 172.21.15.169:0/2768008472 >> v2:172.21.15.170:3300/0 conn(0x7f44dc159480 0x7f44dc0b02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:54.275 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.271+0000 7f44e1e40640 1 -- 172.21.15.169:0/2768008472 >> 172.21.15.169:0/2768008472 conn(0x7f44dc069930 msgr2=0x7f44dc069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:54.275 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.272+0000 7f44e1e40640 1 -- 172.21.15.169:0/2768008472 shutdown_connections 2023-10-26T01:31:54.276 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.272+0000 7f44e1e40640 1 -- 172.21.15.169:0/2768008472 wait complete. 2023-10-26T01:31:54.278 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.273+0000 7f44e1e40640 1 Processor -- start 2023-10-26T01:31:54.278 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.273+0000 7f44e1e40640 1 -- start start 2023-10-26T01:31:54.278 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.274+0000 7f44e1e40640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f44dc0b0820 0x7f44dc1f2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:54.279 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.274+0000 7f44e1e40640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44dc0cd690 0x7f44dc0aeea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:54.279 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.274+0000 7f44e1e40640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f44dc159480 0x7f44dc1e44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:54.279 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.274+0000 7f44e1e40640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44dc1e1fa0 con 0x7f44dc0cd690 2023-10-26T01:31:54.279 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.274+0000 7f44e1e40640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44dc1e3410 con 0x7f44dc0b0820 2023-10-26T01:31:54.279 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.274+0000 7f44e1e40640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44dc1e36f0 con 0x7f44dc159480 2023-10-26T01:31:54.280 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.275+0000 7f44d3fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44dc0cd690 0x7f44dc0aeea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:54.280 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.275+0000 7f44d3fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44dc0cd690 0x7f44dc0aeea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:44262/0 (socket says 172.21.15.169:44262) 2023-10-26T01:31:54.280 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.275+0000 7f44d3fff640 1 -- 172.21.15.169:0/3500077969 learned_addr learned my addr 172.21.15.169:0/3500077969 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:31:54.280 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.275+0000 7f44db7fe640 1 --2- 172.21.15.169:0/3500077969 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f44dc0b0820 0x7f44dc1f2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:54.280 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.275+0000 7f44dbfff640 1 --2- 172.21.15.169:0/3500077969 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f44dc159480 0x7f44dc1e44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:54.281 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.275+0000 7f44d3fff640 1 -- 172.21.15.169:0/3500077969 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f44dc159480 msgr2=0x7f44dc1e44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:54.281 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.275+0000 7f44d3fff640 1 --2- 172.21.15.169:0/3500077969 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f44dc159480 0x7f44dc1e44b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:54.281 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.275+0000 7f44d3fff640 1 -- 172.21.15.169:0/3500077969 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f44dc0b0820 msgr2=0x7f44dc1f2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:54.281 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.275+0000 7f44d3fff640 1 --2- 172.21.15.169:0/3500077969 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f44dc0b0820 0x7f44dc1f2ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:54.281 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.275+0000 7f44d3fff640 1 -- 172.21.15.169:0/3500077969 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44d4009840 con 0x7f44dc0cd690 2023-10-26T01:31:54.282 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.275+0000 7f44d3fff640 1 --2- 172.21.15.169:0/3500077969 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44dc0cd690 0x7f44dc0aeea0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f44c400c750 tx=0x7f44c400cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:54.282 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.276+0000 7f44d8ff9640 1 -- 172.21.15.169:0/3500077969 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44c4005270 con 0x7f44dc0cd690 2023-10-26T01:31:54.282 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.276+0000 7f44d8ff9640 1 -- 172.21.15.169:0/3500077969 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f44c40053f0 con 0x7f44dc0cd690 2023-10-26T01:31:54.282 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.276+0000 7f44d8ff9640 1 -- 172.21.15.169:0/3500077969 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44c400a8a0 con 0x7f44dc0cd690 2023-10-26T01:31:54.282 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.276+0000 7f44e1e40640 1 -- 172.21.15.169:0/3500077969 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44dc1e39d0 con 0x7f44dc0cd690 2023-10-26T01:31:54.283 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.276+0000 7f44e1e40640 1 -- 172.21.15.169:0/3500077969 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44dc1e3f90 con 0x7f44dc0cd690 2023-10-26T01:31:54.284 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.277+0000 7f44e1e40640 1 -- 172.21.15.169:0/3500077969 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44dc066af0 con 0x7f44dc0cd690 2023-10-26T01:31:54.284 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.278+0000 7f44d8ff9640 1 -- 172.21.15.169:0/3500077969 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f44c400aa00 con 0x7f44dc0cd690 2023-10-26T01:31:54.284 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.279+0000 7f44d8ff9640 1 --2- 172.21.15.169:0/3500077969 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f44bc06ae70 0x7f44bc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:54.284 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.279+0000 7f44d8ff9640 1 -- 172.21.15.169:0/3500077969 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f44c4088960 con 0x7f44dc0cd690 2023-10-26T01:31:54.285 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.279+0000 7f44db7fe640 1 --2- 172.21.15.169:0/3500077969 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f44bc06ae70 0x7f44bc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:54.285 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.280+0000 7f44db7fe640 1 --2- 172.21.15.169:0/3500077969 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f44bc06ae70 0x7f44bc06d330 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f44dc154bf0 tx=0x7f44cc005d80 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:54.285 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.283+0000 7f44d8ff9640 1 -- 172.21.15.169:0/3500077969 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f44c4053d60 con 0x7f44dc0cd690 2023-10-26T01:31:54.472 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.470+0000 7f44e1e40640 1 -- 172.21.15.169:0/3500077969 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f44dc04da70 con 0x7f44dc0cd690 2023-10-26T01:31:54.472 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.471+0000 7f44d8ff9640 1 -- 172.21.15.169:0/3500077969 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v31) v1 ==== 93+0+10 (secure 0 0 0) 0x7f44c4017070 con 0x7f44dc0cd690 2023-10-26T01:31:54.473 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 1 2023-10-26T01:31:54.475 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.473+0000 7f44e1e40640 1 -- 172.21.15.169:0/3500077969 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f44bc06ae70 msgr2=0x7f44bc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:54.475 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.473+0000 7f44e1e40640 1 --2- 172.21.15.169:0/3500077969 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f44bc06ae70 0x7f44bc06d330 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f44dc154bf0 tx=0x7f44cc005d80 comp rx=0 tx=0).stop 2023-10-26T01:31:54.476 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.473+0000 7f44e1e40640 1 -- 172.21.15.169:0/3500077969 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44dc0cd690 msgr2=0x7f44dc0aeea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:54.476 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.473+0000 7f44e1e40640 1 --2- 172.21.15.169:0/3500077969 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44dc0cd690 0x7f44dc0aeea0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f44c400c750 tx=0x7f44c400cc10 comp rx=0 tx=0).stop 2023-10-26T01:31:54.476 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.473+0000 7f44e1e40640 1 -- 172.21.15.169:0/3500077969 shutdown_connections 2023-10-26T01:31:54.476 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.473+0000 7f44e1e40640 1 --2- 172.21.15.169:0/3500077969 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f44bc06ae70 0x7f44bc06d330 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:54.476 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.473+0000 7f44e1e40640 1 --2- 172.21.15.169:0/3500077969 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44dc0cd690 0x7f44dc0aeea0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:54.477 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.473+0000 7f44e1e40640 1 --2- 172.21.15.169:0/3500077969 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f44dc159480 0x7f44dc1e44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:54.477 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.473+0000 7f44e1e40640 1 --2- 172.21.15.169:0/3500077969 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f44dc0b0820 0x7f44dc1f2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:54.477 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.474+0000 7f44e1e40640 1 -- 172.21.15.169:0/3500077969 >> 172.21.15.169:0/3500077969 conn(0x7f44dc069930 msgr2=0x7f44dc158620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:54.477 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.474+0000 7f44e1e40640 1 -- 172.21.15.169:0/3500077969 shutdown_connections 2023-10-26T01:31:54.477 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.474+0000 7f44e1e40640 1 -- 172.21.15.169:0/3500077969 wait complete. 2023-10-26T01:31:54.495 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-26T01:31:54.709 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.707+0000 7fd69f19e640 1 -- 172.21.15.169:0/3780160160 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6840039c0 msgr2=0x7fd684003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:54.709 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.707+0000 7fd69f19e640 1 --2- 172.21.15.169:0/3780160160 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6840039c0 0x7fd684003de0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fd69400cc30 tx=0x7fd6840057f0 comp rx=0 tx=0).stop 2023-10-26T01:31:54.709 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.707+0000 7fd69f19e640 1 -- 172.21.15.169:0/3780160160 shutdown_connections 2023-10-26T01:31:54.709 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.707+0000 7fd69f19e640 1 --2- 172.21.15.169:0/3780160160 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6840039c0 0x7fd684003de0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:54.710 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.707+0000 7fd69f19e640 1 -- 172.21.15.169:0/3780160160 >> 172.21.15.169:0/3780160160 conn(0x7fd6980aedd0 msgr2=0x7fd6980af1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:54.710 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.707+0000 7fd69f19e640 1 -- 172.21.15.169:0/3780160160 shutdown_connections 2023-10-26T01:31:54.710 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.707+0000 7fd69f19e640 1 -- 172.21.15.169:0/3780160160 wait complete. 2023-10-26T01:31:54.713 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.708+0000 7fd69f19e640 1 Processor -- start 2023-10-26T01:31:54.713 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.709+0000 7fd69f19e640 1 -- start start 2023-10-26T01:31:54.713 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.709+0000 7fd69f19e640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd6840039c0 0x7fd6980b21e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:54.714 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.710+0000 7fd69f19e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6980afce0 0x7fd698154bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:54.714 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.710+0000 7fd69f19e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd6980cc350 0x7fd6980cc7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:54.714 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.710+0000 7fd69f19e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6980b2720 con 0x7fd6980afce0 2023-10-26T01:31:54.714 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.710+0000 7fd69f19e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6980ccd10 con 0x7fd6980cc350 2023-10-26T01:31:54.715 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.710+0000 7fd69f19e640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6980b0460 con 0x7fd6840039c0 2023-10-26T01:31:54.715 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.710+0000 7fd69d99b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6980afce0 0x7fd698154bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:54.715 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.710+0000 7fd69d99b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6980afce0 0x7fd698154bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:44282/0 (socket says 172.21.15.169:44282) 2023-10-26T01:31:54.715 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.710+0000 7fd69d99b640 1 -- 172.21.15.169:0/2912827721 learned_addr learned my addr 172.21.15.169:0/2912827721 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:31:54.715 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.710+0000 7fd69e19c640 1 --2- 172.21.15.169:0/2912827721 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd6840039c0 0x7fd6980b21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:54.716 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.710+0000 7fd69d99b640 1 -- 172.21.15.169:0/2912827721 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd6840039c0 msgr2=0x7fd6980b21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:54.716 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.710+0000 7fd69d99b640 1 --2- 172.21.15.169:0/2912827721 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd6840039c0 0x7fd6980b21e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:54.716 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.710+0000 7fd69e99d640 1 --2- 172.21.15.169:0/2912827721 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd6980cc350 0x7fd6980cc7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:54.716 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.710+0000 7fd69d99b640 1 -- 172.21.15.169:0/2912827721 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd6980cc350 msgr2=0x7fd6980cc7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:54.716 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.710+0000 7fd69d99b640 1 --2- 172.21.15.169:0/2912827721 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd6980cc350 0x7fd6980cc7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:54.717 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.710+0000 7fd69d99b640 1 -- 172.21.15.169:0/2912827721 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6980b0740 con 0x7fd6980afce0 2023-10-26T01:31:54.717 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.711+0000 7fd69d99b640 1 --2- 172.21.15.169:0/2912827721 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6980afce0 0x7fd698154bd0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fd68800c730 tx=0x7fd68800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:54.717 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.711+0000 7fd68effd640 1 -- 172.21.15.169:0/2912827721 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd688005270 con 0x7fd6980afce0 2023-10-26T01:31:54.717 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.711+0000 7fd68effd640 1 -- 172.21.15.169:0/2912827721 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd68800a640 con 0x7fd6980afce0 2023-10-26T01:31:54.717 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.711+0000 7fd68effd640 1 -- 172.21.15.169:0/2912827721 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd68800aa90 con 0x7fd6980afce0 2023-10-26T01:31:54.718 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.711+0000 7fd69f19e640 1 -- 172.21.15.169:0/2912827721 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6980b0a20 con 0x7fd6980afce0 2023-10-26T01:31:54.718 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.711+0000 7fd69f19e640 1 -- 172.21.15.169:0/2912827721 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6980b0160 con 0x7fd6980afce0 2023-10-26T01:31:54.719 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.712+0000 7fd69f19e640 1 -- 172.21.15.169:0/2912827721 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd698073c10 con 0x7fd6980afce0 2023-10-26T01:31:54.719 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.713+0000 7fd68effd640 1 -- 172.21.15.169:0/2912827721 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fd68800abf0 con 0x7fd6980afce0 2023-10-26T01:31:54.719 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.714+0000 7fd68effd640 1 --2- 172.21.15.169:0/2912827721 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd67c06ae40 0x7fd67c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:54.720 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.714+0000 7fd68effd640 1 -- 172.21.15.169:0/2912827721 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd688088bf0 con 0x7fd6980afce0 2023-10-26T01:31:54.720 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.715+0000 7fd69e19c640 1 --2- 172.21.15.169:0/2912827721 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd67c06ae40 0x7fd67c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:54.720 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.716+0000 7fd69e19c640 1 --2- 172.21.15.169:0/2912827721 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd67c06ae40 0x7fd67c06d300 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd694038000 tx=0x7fd69402f040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:54.721 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.718+0000 7fd68effd640 1 -- 172.21.15.169:0/2912827721 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd688053fe0 con 0x7fd6980afce0 2023-10-26T01:31:54.912 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.910+0000 7fd69f19e640 1 -- 172.21.15.169:0/2912827721 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fd69806c790 con 0x7fd6980afce0 2023-10-26T01:31:54.912 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.911+0000 7fd68effd640 1 -- 172.21.15.169:0/2912827721 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v31) v1 ==== 84+0+65 (secure 0 0 0) 0x7fd688017070 con 0x7fd6980afce0 2023-10-26T01:31:54.913 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:31:54.913 INFO:teuthology.orchestra.run.smithi169.stdout:[ 2023-10-26T01:31:54.913 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:31:54.913 INFO:teuthology.orchestra.run.smithi169.stdout: "poolnum": 1, 2023-10-26T01:31:54.913 INFO:teuthology.orchestra.run.smithi169.stdout: "poolname": ".mgr" 2023-10-26T01:31:54.914 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:31:54.914 INFO:teuthology.orchestra.run.smithi169.stdout:] 2023-10-26T01:31:54.915 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.913+0000 7fd69f19e640 1 -- 172.21.15.169:0/2912827721 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd67c06ae40 msgr2=0x7fd67c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:54.915 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.913+0000 7fd69f19e640 1 --2- 172.21.15.169:0/2912827721 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd67c06ae40 0x7fd67c06d300 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd694038000 tx=0x7fd69402f040 comp rx=0 tx=0).stop 2023-10-26T01:31:54.915 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.913+0000 7fd69f19e640 1 -- 172.21.15.169:0/2912827721 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6980afce0 msgr2=0x7fd698154bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:54.915 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.913+0000 7fd69f19e640 1 --2- 172.21.15.169:0/2912827721 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6980afce0 0x7fd698154bd0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fd68800c730 tx=0x7fd68800cbf0 comp rx=0 tx=0).stop 2023-10-26T01:31:54.916 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.913+0000 7fd69f19e640 1 -- 172.21.15.169:0/2912827721 shutdown_connections 2023-10-26T01:31:54.916 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.913+0000 7fd69f19e640 1 --2- 172.21.15.169:0/2912827721 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd67c06ae40 0x7fd67c06d300 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:54.916 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.913+0000 7fd69f19e640 1 --2- 172.21.15.169:0/2912827721 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6980afce0 0x7fd698154bd0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:54.916 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.913+0000 7fd69f19e640 1 --2- 172.21.15.169:0/2912827721 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd6980cc350 0x7fd6980cc7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:54.916 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.913+0000 7fd69f19e640 1 --2- 172.21.15.169:0/2912827721 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd6840039c0 0x7fd6980b21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:54.916 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.913+0000 7fd69f19e640 1 -- 172.21.15.169:0/2912827721 >> 172.21.15.169:0/2912827721 conn(0x7fd6980aedd0 msgr2=0x7fd6980cd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:54.917 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.913+0000 7fd69f19e640 1 -- 172.21.15.169:0/2912827721 shutdown_connections 2023-10-26T01:31:54.917 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:54.913+0000 7fd69f19e640 1 -- 172.21.15.169:0/2912827721 wait complete. 2023-10-26T01:31:54.934 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-26T01:31:54.935 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-26T01:31:55.149 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:55.146+0000 7f85cf0e0640 1 -- 172.21.15.169:0/2199589408 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f85c806bd90 msgr2=0x7f85c80cd670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:55.149 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:55.146+0000 7f85cf0e0640 1 --2- 172.21.15.169:0/2199589408 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f85c806bd90 0x7f85c80cd670 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f85b8009830 tx=0x7f85b802bd70 comp rx=0 tx=0).stop 2023-10-26T01:31:55.149 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:55.147+0000 7f85cf0e0640 1 -- 172.21.15.169:0/2199589408 shutdown_connections 2023-10-26T01:31:55.149 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:55.147+0000 7f85cf0e0640 1 --2- 172.21.15.169:0/2199589408 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f85c806bd90 0x7f85c80cd670 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:55.150 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:55.147+0000 7f85cf0e0640 1 -- 172.21.15.169:0/2199589408 >> 172.21.15.169:0/2199589408 conn(0x7f85c8069930 msgr2=0x7f85c8069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:55.150 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:55.147+0000 7f85cf0e0640 1 -- 172.21.15.169:0/2199589408 shutdown_connections 2023-10-26T01:31:55.150 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:55.147+0000 7f85cf0e0640 1 -- 172.21.15.169:0/2199589408 wait complete. 2023-10-26T01:31:55.152 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:55.148+0000 7f85cf0e0640 1 Processor -- start 2023-10-26T01:31:55.152 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:55.149+0000 7f85cf0e0640 1 -- start start 2023-10-26T01:31:55.152 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:55.149+0000 7f85cf0e0640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f85c806bd90 0x7f85c814b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:55.153 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:55.149+0000 7f85cf0e0640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f85c80cdbb0 0x7f85c80ca6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:55.153 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:55.150+0000 7f85cf0e0640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f85c8152b70 0x7f85c80cac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:55.153 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:55.150+0000 7f85cf0e0640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85c8152fa0 con 0x7f85c80cdbb0 2023-10-26T01:31:55.153 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:55.150+0000 7f85cf0e0640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85c81effe0 con 0x7f85c8152b70 2023-10-26T01:31:55.153 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:55.150+0000 7f85cf0e0640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85c80710c0 con 0x7f85c806bd90 2023-10-26T01:31:55.154 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:55.150+0000 7f85bffff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f85c80cdbb0 0x7f85c80ca6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:55.154 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:55.150+0000 7f85bffff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f85c80cdbb0 0x7f85c80ca6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:44304/0 (socket says 172.21.15.169:44304) 2023-10-26T01:31:55.154 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:55.150+0000 7f85bffff640 1 -- 172.21.15.169:0/4283247546 learned_addr learned my addr 172.21.15.169:0/4283247546 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:31:55.154 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:55.150+0000 7f85cce7c640 1 --2- 172.21.15.169:0/4283247546 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f85c806bd90 0x7f85c814b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:55.154 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:55.150+0000 7f85cd67d640 1 --2- 172.21.15.169:0/4283247546 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f85c8152b70 0x7f85c80cac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:55.155 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:55.150+0000 7f85bffff640 1 -- 172.21.15.169:0/4283247546 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f85c806bd90 msgr2=0x7f85c814b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:55.155 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:55.150+0000 7f85bffff640 1 --2- 172.21.15.169:0/4283247546 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f85c806bd90 0x7f85c814b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:55.155 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:55.150+0000 7f85bffff640 1 -- 172.21.15.169:0/4283247546 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f85c8152b70 msgr2=0x7f85c80cac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:55.155 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:55.150+0000 7f85bffff640 1 --2- 172.21.15.169:0/4283247546 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f85c8152b70 0x7f85c80cac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:55.155 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:55.150+0000 7f85bffff640 1 -- 172.21.15.169:0/4283247546 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85b8009490 con 0x7f85c80cdbb0 2023-10-26T01:31:55.156 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:55.150+0000 7f85cd67d640 1 --2- 172.21.15.169:0/4283247546 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f85c8152b70 0x7f85c80cac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:31:55.156 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:55.151+0000 7f85bffff640 1 --2- 172.21.15.169:0/4283247546 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f85c80cdbb0 0x7f85c80ca6e0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f85b8009780 tx=0x7f85b80038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:55.157 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:55.151+0000 7f85bd7fa640 1 -- 172.21.15.169:0/4283247546 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b8003ea0 con 0x7f85c80cdbb0 2023-10-26T01:31:55.157 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:55.151+0000 7f85bd7fa640 1 -- 172.21.15.169:0/4283247546 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f85b8034550 con 0x7f85c80cdbb0 2023-10-26T01:31:55.158 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:55.151+0000 7f85bd7fa640 1 -- 172.21.15.169:0/4283247546 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b8034960 con 0x7f85c80cdbb0 2023-10-26T01:31:55.158 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:55.151+0000 7f85cf0e0640 1 -- 172.21.15.169:0/4283247546 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85c80713a0 con 0x7f85c80cdbb0 2023-10-26T01:31:55.158 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:55.151+0000 7f85cf0e0640 1 -- 172.21.15.169:0/4283247546 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85c8071960 con 0x7f85c80cdbb0 2023-10-26T01:31:55.158 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:55.152+0000 7f85cf0e0640 1 -- 172.21.15.169:0/4283247546 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85c8066af0 con 0x7f85c80cdbb0 2023-10-26T01:31:55.158 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:55.153+0000 7f85bd7fa640 1 -- 172.21.15.169:0/4283247546 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f85b803b040 con 0x7f85c80cdbb0 2023-10-26T01:31:55.159 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:55.154+0000 7f85bd7fa640 1 --2- 172.21.15.169:0/4283247546 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f85b406ae40 0x7f85b406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:55.159 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:55.154+0000 7f85bd7fa640 1 -- 172.21.15.169:0/4283247546 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f85b80ab950 con 0x7f85c80cdbb0 2023-10-26T01:31:55.159 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:55.155+0000 7f85cce7c640 1 --2- 172.21.15.169:0/4283247546 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f85b406ae40 0x7f85b406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:55.159 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:55.156+0000 7f85cce7c640 1 --2- 172.21.15.169:0/4283247546 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f85b406ae40 0x7f85b406d300 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f85c4007fd0 tx=0x7f85c4009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:55.160 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:55.159+0000 7f85bd7fa640 1 -- 172.21.15.169:0/4283247546 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f85b8076d40 con 0x7f85c80cdbb0 2023-10-26T01:31:55.353 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:55.352+0000 7f85cf0e0640 1 -- 172.21.15.169:0/4283247546 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f85c8071b50 con 0x7f85c80cdbb0 2023-10-26T01:31:56.278 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:56.276+0000 7f85bd7fa640 1 -- 172.21.15.169:0/4283247546 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v32) v1 ==== 138+0+0 (secure 0 0 0) 0x7f85c8071b50 con 0x7f85c80cdbb0 2023-10-26T01:31:56.279 INFO:teuthology.orchestra.run.smithi169.stderr:pool 'cephfs_metadata' created 2023-10-26T01:31:56.282 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:56.280+0000 7f85cf0e0640 1 -- 172.21.15.169:0/4283247546 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f85b406ae40 msgr2=0x7f85b406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:56.282 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:56.280+0000 7f85cf0e0640 1 --2- 172.21.15.169:0/4283247546 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f85b406ae40 0x7f85b406d300 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f85c4007fd0 tx=0x7f85c4009040 comp rx=0 tx=0).stop 2023-10-26T01:31:56.282 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:56.280+0000 7f85cf0e0640 1 -- 172.21.15.169:0/4283247546 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f85c80cdbb0 msgr2=0x7f85c80ca6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:56.282 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:56.280+0000 7f85cf0e0640 1 --2- 172.21.15.169:0/4283247546 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f85c80cdbb0 0x7f85c80ca6e0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f85b8009780 tx=0x7f85b80038a0 comp rx=0 tx=0).stop 2023-10-26T01:31:56.283 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:56.280+0000 7f85cf0e0640 1 -- 172.21.15.169:0/4283247546 shutdown_connections 2023-10-26T01:31:56.283 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:56.280+0000 7f85cf0e0640 1 --2- 172.21.15.169:0/4283247546 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f85b406ae40 0x7f85b406d300 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:56.283 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:56.280+0000 7f85cf0e0640 1 --2- 172.21.15.169:0/4283247546 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f85c80cdbb0 0x7f85c80ca6e0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:56.283 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:56.280+0000 7f85cf0e0640 1 --2- 172.21.15.169:0/4283247546 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f85c8152b70 0x7f85c80cac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:56.284 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:56.280+0000 7f85cf0e0640 1 --2- 172.21.15.169:0/4283247546 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f85c806bd90 0x7f85c814b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:56.284 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:56.280+0000 7f85cf0e0640 1 -- 172.21.15.169:0/4283247546 >> 172.21.15.169:0/4283247546 conn(0x7f85c8069930 msgr2=0x7f85c80ab730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:56.284 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:56.281+0000 7f85cf0e0640 1 -- 172.21.15.169:0/4283247546 shutdown_connections 2023-10-26T01:31:56.285 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:56.281+0000 7f85cf0e0640 1 -- 172.21.15.169:0/4283247546 wait complete. 2023-10-26T01:31:56.305 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-26T01:31:56.523 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:56.521+0000 7f59526a9640 1 -- 172.21.15.169:0/942341603 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5938004d00 msgr2=0x7f5938005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:56.524 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:56.521+0000 7f59526a9640 1 --2- 172.21.15.169:0/942341603 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5938004d00 0x7f5938005180 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f5944001d80 tx=0x7f59380057f0 comp rx=0 tx=0).stop 2023-10-26T01:31:56.524 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:56.521+0000 7f5950c46640 1 -- 172.21.15.169:0/942341603 reap_dead start 2023-10-26T01:31:56.524 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:56.522+0000 7f59526a9640 1 -- 172.21.15.169:0/942341603 shutdown_connections 2023-10-26T01:31:56.524 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:56.522+0000 7f59526a9640 1 -- 172.21.15.169:0/942341603 >> 172.21.15.169:0/942341603 conn(0x7f594c069930 msgr2=0x7f594c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:56.524 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:56.522+0000 7f59526a9640 1 -- 172.21.15.169:0/942341603 shutdown_connections 2023-10-26T01:31:56.525 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:56.522+0000 7f59526a9640 1 -- 172.21.15.169:0/942341603 wait complete. 2023-10-26T01:31:56.525 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:56.523+0000 7f59526a9640 1 Processor -- start 2023-10-26T01:31:56.526 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:56.524+0000 7f59526a9640 1 -- start start 2023-10-26T01:31:56.528 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:56.524+0000 7f59526a9640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5938004d00 0x7f594c06c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:56.528 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:56.525+0000 7f59526a9640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f594c06ed30 0x7f594c15ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:56.528 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:56.525+0000 7f59526a9640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f594c1e4ac0 0x7f594c15b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:56.528 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:56.525+0000 7f59526a9640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f594c0c6700 con 0x7f5938004d00 2023-10-26T01:31:56.529 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:56.525+0000 7f59526a9640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f594c0c69e0 con 0x7f594c1e4ac0 2023-10-26T01:31:56.529 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:56.525+0000 7f59526a9640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f594c0c6cc0 con 0x7f594c06ed30 2023-10-26T01:31:56.529 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:56.525+0000 7f594bfff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5938004d00 0x7f594c06c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:56.529 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:56.525+0000 7f594bfff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5938004d00 0x7f594c06c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:44324/0 (socket says 172.21.15.169:44324) 2023-10-26T01:31:56.530 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:56.525+0000 7f594bfff640 1 -- 172.21.15.169:0/1663381739 learned_addr learned my addr 172.21.15.169:0/1663381739 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:31:56.530 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:56.525+0000 7f594bfff640 1 -- 172.21.15.169:0/1663381739 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f594c06ed30 msgr2=0x7f594c15ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:56.530 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:56.525+0000 7f5943fff640 1 --2- 172.21.15.169:0/1663381739 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f594c06ed30 0x7f594c15ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:56.530 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:56.525+0000 7f594bfff640 1 --2- 172.21.15.169:0/1663381739 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f594c06ed30 0x7f594c15ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:56.531 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:56.525+0000 7f594bfff640 1 -- 172.21.15.169:0/1663381739 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f594c1e4ac0 msgr2=0x7f594c15b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:56.531 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:56.525+0000 7f594bfff640 1 --2- 172.21.15.169:0/1663381739 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f594c1e4ac0 0x7f594c15b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:56.531 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:56.525+0000 7f594bfff640 1 -- 172.21.15.169:0/1663381739 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f594c0c6fa0 con 0x7f5938004d00 2023-10-26T01:31:56.531 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:56.526+0000 7f594bfff640 1 --2- 172.21.15.169:0/1663381739 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5938004d00 0x7f594c06c4c0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f593c011a50 tx=0x7f593c011f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:56.531 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:56.526+0000 7f59497fa640 1 -- 172.21.15.169:0/1663381739 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f593c00fc50 con 0x7f5938004d00 2023-10-26T01:31:56.531 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:56.526+0000 7f59497fa640 1 -- 172.21.15.169:0/1663381739 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f593c00fea0 con 0x7f5938004d00 2023-10-26T01:31:56.532 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:56.526+0000 7f59497fa640 1 -- 172.21.15.169:0/1663381739 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f593c0126b0 con 0x7f5938004d00 2023-10-26T01:31:56.532 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:56.526+0000 7f59526a9640 1 -- 172.21.15.169:0/1663381739 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f594c06ca00 con 0x7f5938004d00 2023-10-26T01:31:56.532 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:56.526+0000 7f59526a9640 1 -- 172.21.15.169:0/1663381739 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f594c14b620 con 0x7f5938004d00 2023-10-26T01:31:56.533 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:56.527+0000 7f59526a9640 1 -- 172.21.15.169:0/1663381739 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f594c0c7310 con 0x7f5938004d00 2023-10-26T01:31:56.533 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:56.528+0000 7f59497fa640 1 -- 172.21.15.169:0/1663381739 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f593c012810 con 0x7f5938004d00 2023-10-26T01:31:56.534 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:56.529+0000 7f59497fa640 1 --2- 172.21.15.169:0/1663381739 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f592c06ae70 0x7f592c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:56.534 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:56.529+0000 7f59497fa640 1 -- 172.21.15.169:0/1663381739 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f593c08a2c0 con 0x7f5938004d00 2023-10-26T01:31:56.534 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:56.530+0000 7f5943fff640 1 --2- 172.21.15.169:0/1663381739 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f592c06ae70 0x7f592c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:56.534 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:56.531+0000 7f5943fff640 1 --2- 172.21.15.169:0/1663381739 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f592c06ae70 0x7f592c06d330 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5934001eb0 tx=0x7f5934001490 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:56.536 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:56.535+0000 7f59497fa640 1 -- 172.21.15.169:0/1663381739 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f594c0c7310 con 0x7f5938004d00 2023-10-26T01:31:56.727 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:56.725+0000 7f59526a9640 1 -- 172.21.15.169:0/1663381739 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f594c0c7310 con 0x7f5938004d00 2023-10-26T01:31:57.285 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:57.283+0000 7f59497fa640 1 -- 172.21.15.169:0/1663381739 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v33) v1 ==== 170+0+0 (secure 0 0 0) 0x7f594c0c7310 con 0x7f5938004d00 2023-10-26T01:31:57.285 INFO:teuthology.orchestra.run.smithi169.stderr:pool 'cephfs_data' created 2023-10-26T01:31:57.289 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:57.287+0000 7f59526a9640 1 -- 172.21.15.169:0/1663381739 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f592c06ae70 msgr2=0x7f592c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:57.289 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:57.287+0000 7f59526a9640 1 --2- 172.21.15.169:0/1663381739 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f592c06ae70 0x7f592c06d330 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5934001eb0 tx=0x7f5934001490 comp rx=0 tx=0).stop 2023-10-26T01:31:57.290 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:57.287+0000 7f59526a9640 1 -- 172.21.15.169:0/1663381739 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5938004d00 msgr2=0x7f594c06c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:57.290 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:57.287+0000 7f59526a9640 1 --2- 172.21.15.169:0/1663381739 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5938004d00 0x7f594c06c4c0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f593c011a50 tx=0x7f593c011f10 comp rx=0 tx=0).stop 2023-10-26T01:31:57.290 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:57.287+0000 7f59526a9640 1 -- 172.21.15.169:0/1663381739 shutdown_connections 2023-10-26T01:31:57.290 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:57.287+0000 7f59526a9640 1 --2- 172.21.15.169:0/1663381739 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f592c06ae70 0x7f592c06d330 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:57.290 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:57.287+0000 7f59526a9640 1 --2- 172.21.15.169:0/1663381739 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f594c1e4ac0 0x7f594c15b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:57.291 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:57.287+0000 7f59526a9640 1 --2- 172.21.15.169:0/1663381739 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f594c06ed30 0x7f594c15ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:57.291 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:57.287+0000 7f59526a9640 1 --2- 172.21.15.169:0/1663381739 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5938004d00 0x7f594c06c4c0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:57.291 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:57.287+0000 7f59526a9640 1 -- 172.21.15.169:0/1663381739 >> 172.21.15.169:0/1663381739 conn(0x7f594c069930 msgr2=0x7f594c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:57.291 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:57.287+0000 7f59526a9640 1 -- 172.21.15.169:0/1663381739 shutdown_connections 2023-10-26T01:31:57.291 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:57.287+0000 7f59526a9640 1 -- 172.21.15.169:0/1663381739 wait complete. 2023-10-26T01:31:57.316 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-26T01:31:57.513 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:57.511+0000 7fc71d78a640 1 -- 172.21.15.169:0/2121680244 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc718157300 msgr2=0x7fc71815a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:57.513 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:57.511+0000 7fc71d78a640 1 --2- 172.21.15.169:0/2121680244 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc718157300 0x7fc71815a690 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fc708005fd0 tx=0x7fc70802c050 comp rx=0 tx=0).stop 2023-10-26T01:31:57.513 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:57.511+0000 7fc71d78a640 1 -- 172.21.15.169:0/2121680244 shutdown_connections 2023-10-26T01:31:57.514 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:57.511+0000 7fc71d78a640 1 --2- 172.21.15.169:0/2121680244 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc718157300 0x7fc71815a690 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:57.514 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:57.511+0000 7fc71d78a640 1 -- 172.21.15.169:0/2121680244 >> 172.21.15.169:0/2121680244 conn(0x7fc718098a60 msgr2=0x7fc7180b4970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:57.514 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:57.511+0000 7fc71d78a640 1 -- 172.21.15.169:0/2121680244 shutdown_connections 2023-10-26T01:31:57.514 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:57.511+0000 7fc71d78a640 1 -- 172.21.15.169:0/2121680244 wait complete. 2023-10-26T01:31:57.515 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:57.513+0000 7fc71d78a640 1 Processor -- start 2023-10-26T01:31:57.515 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:57.513+0000 7fc71d78a640 1 -- start start 2023-10-26T01:31:57.516 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:57.514+0000 7fc71d78a640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7180b2450 0x7fc7180ca700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:57.517 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:57.514+0000 7fc71d78a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc7180cfb10 0x7fc718145eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:57.517 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:57.514+0000 7fc71d78a640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc718157300 0x7fc7180adb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:57.517 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:57.514+0000 7fc71d78a640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7180ca270 con 0x7fc7180b2450 2023-10-26T01:31:57.517 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:57.514+0000 7fc71d78a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7180c9660 con 0x7fc7180cfb10 2023-10-26T01:31:57.517 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:57.514+0000 7fc71d78a640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7180c9940 con 0x7fc718157300 2023-10-26T01:31:57.518 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:57.514+0000 7fc716ffd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7180b2450 0x7fc7180ca700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:57.518 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:57.514+0000 7fc716ffd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7180b2450 0x7fc7180ca700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:44332/0 (socket says 172.21.15.169:44332) 2023-10-26T01:31:57.518 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:57.514+0000 7fc716ffd640 1 -- 172.21.15.169:0/3831072610 learned_addr learned my addr 172.21.15.169:0/3831072610 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:31:57.518 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:57.515+0000 7fc7167fc640 1 --2- 172.21.15.169:0/3831072610 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc7180cfb10 0x7fc718145eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:57.518 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:57.515+0000 7fc716ffd640 1 -- 172.21.15.169:0/3831072610 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc718157300 msgr2=0x7fc7180adb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:57.519 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:57.515+0000 7fc7177fe640 1 --2- 172.21.15.169:0/3831072610 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc718157300 0x7fc7180adb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:57.519 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:57.515+0000 7fc716ffd640 1 --2- 172.21.15.169:0/3831072610 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc718157300 0x7fc7180adb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:57.519 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:57.515+0000 7fc716ffd640 1 -- 172.21.15.169:0/3831072610 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc7180cfb10 msgr2=0x7fc718145eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:57.519 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:57.515+0000 7fc716ffd640 1 --2- 172.21.15.169:0/3831072610 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc7180cfb10 0x7fc718145eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:57.519 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:57.515+0000 7fc716ffd640 1 -- 172.21.15.169:0/3831072610 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc708009840 con 0x7fc7180b2450 2023-10-26T01:31:57.520 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:57.515+0000 7fc7167fc640 1 --2- 172.21.15.169:0/3831072610 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc7180cfb10 0x7fc718145eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:31:57.520 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:57.515+0000 7fc7177fe640 1 --2- 172.21.15.169:0/3831072610 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc718157300 0x7fc7180adb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:31:57.521 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:57.515+0000 7fc716ffd640 1 --2- 172.21.15.169:0/3831072610 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7180b2450 0x7fc7180ca700 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fc708009b30 tx=0x7fc708004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:57.521 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:57.516+0000 7fc70f7fe640 1 -- 172.21.15.169:0/3831072610 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc70803e660 con 0x7fc7180b2450 2023-10-26T01:31:57.521 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:57.516+0000 7fc70f7fe640 1 -- 172.21.15.169:0/3831072610 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc70803e8b0 con 0x7fc7180b2450 2023-10-26T01:31:57.522 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:57.516+0000 7fc71d78a640 1 -- 172.21.15.169:0/3831072610 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7180c9c20 con 0x7fc7180b2450 2023-10-26T01:31:57.522 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:57.516+0000 7fc70f7fe640 1 -- 172.21.15.169:0/3831072610 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc70803eca0 con 0x7fc7180b2450 2023-10-26T01:31:57.522 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:57.516+0000 7fc71d78a640 1 -- 172.21.15.169:0/3831072610 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7180cac40 con 0x7fc7180b2450 2023-10-26T01:31:57.522 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:57.517+0000 7fc71d78a640 1 -- 172.21.15.169:0/3831072610 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc71804db10 con 0x7fc7180b2450 2023-10-26T01:31:57.523 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:57.518+0000 7fc70f7fe640 1 -- 172.21.15.169:0/3831072610 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fc708034430 con 0x7fc7180b2450 2023-10-26T01:31:57.523 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:57.519+0000 7fc70f7fe640 1 --2- 172.21.15.169:0/3831072610 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc6f806ae40 0x7fc6f806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:57.523 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:57.519+0000 7fc70f7fe640 1 -- 172.21.15.169:0/3831072610 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fc7080ac6f0 con 0x7fc7180b2450 2023-10-26T01:31:57.523 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:57.520+0000 7fc7167fc640 1 --2- 172.21.15.169:0/3831072610 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc6f806ae40 0x7fc6f806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:57.524 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:57.521+0000 7fc7167fc640 1 --2- 172.21.15.169:0/3831072610 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc6f806ae40 0x7fc6f806d300 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc7180c90e0 tx=0x7fc700009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:57.526 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:57.524+0000 7fc70f7fe640 1 -- 172.21.15.169:0/3831072610 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc708077890 con 0x7fc7180b2450 2023-10-26T01:31:57.758 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:57.757+0000 7fc71d78a640 1 -- 172.21.15.169:0/3831072610 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fc7180cfff0 con 0x7fc7180b2450 2023-10-26T01:31:58.319 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:58.317+0000 7fc70f7fe640 1 -- 172.21.15.169:0/3831072610 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v11) v1 ==== 172+0+0 (secure 0 0 0) 0x7fc7080420c0 con 0x7fc7180b2450 2023-10-26T01:31:58.320 INFO:teuthology.orchestra.run.smithi169.stderr:new fs with metadata pool 4 and data pool 5 2023-10-26T01:31:58.323 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:58.320+0000 7fc71d78a640 1 -- 172.21.15.169:0/3831072610 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc6f806ae40 msgr2=0x7fc6f806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:58.323 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:58.320+0000 7fc71d78a640 1 --2- 172.21.15.169:0/3831072610 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc6f806ae40 0x7fc6f806d300 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc7180c90e0 tx=0x7fc700009040 comp rx=0 tx=0).stop 2023-10-26T01:31:58.324 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:58.320+0000 7fc71d78a640 1 -- 172.21.15.169:0/3831072610 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7180b2450 msgr2=0x7fc7180ca700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:58.324 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:58.320+0000 7fc71d78a640 1 --2- 172.21.15.169:0/3831072610 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7180b2450 0x7fc7180ca700 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fc708009b30 tx=0x7fc708004360 comp rx=0 tx=0).stop 2023-10-26T01:31:58.324 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:58.321+0000 7fc71d78a640 1 -- 172.21.15.169:0/3831072610 shutdown_connections 2023-10-26T01:31:58.325 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:58.321+0000 7fc71d78a640 1 --2- 172.21.15.169:0/3831072610 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc6f806ae40 0x7fc6f806d300 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:58.325 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:58.321+0000 7fc71d78a640 1 --2- 172.21.15.169:0/3831072610 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc718157300 0x7fc7180adb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:58.325 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:58.321+0000 7fc71d78a640 1 --2- 172.21.15.169:0/3831072610 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc7180cfb10 0x7fc718145eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:58.325 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:58.321+0000 7fc71d78a640 1 --2- 172.21.15.169:0/3831072610 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7180b2450 0x7fc7180ca700 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:58.325 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:58.321+0000 7fc71d78a640 1 -- 172.21.15.169:0/3831072610 >> 172.21.15.169:0/3831072610 conn(0x7fc718098a60 msgr2=0x7fc7180cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:58.326 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:58.321+0000 7fc71d78a640 1 -- 172.21.15.169:0/3831072610 shutdown_connections 2023-10-26T01:31:58.326 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:58.321+0000 7fc71d78a640 1 -- 172.21.15.169:0/3831072610 wait complete. 2023-10-26T01:31:58.346 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:31:58.538 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:58.536+0000 7f961f59e640 1 -- 172.21.15.169:0/1411096055 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f962009a5c0 msgr2=0x7f962009aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:58.539 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:58.536+0000 7f961f59e640 1 --2- 172.21.15.169:0/1411096055 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f962009a5c0 0x7f962009aa70 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9618005fd0 tx=0x7f961802c2e0 comp rx=0 tx=0).stop 2023-10-26T01:31:58.539 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:58.536+0000 7f961f59e640 1 -- 172.21.15.169:0/1411096055 shutdown_connections 2023-10-26T01:31:58.539 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:58.536+0000 7f961f59e640 1 --2- 172.21.15.169:0/1411096055 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f962009a5c0 0x7f962009aa70 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:58.540 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:58.536+0000 7f961f59e640 1 --2- 172.21.15.169:0/1411096055 >> v2:172.21.15.170:3300/0 conn(0x7f96200c9970 0x7f9620098a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:58.540 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:58.536+0000 7f961f59e640 1 -- 172.21.15.169:0/1411096055 >> 172.21.15.169:0/1411096055 conn(0x7f96200aedd0 msgr2=0x7f96200af1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:58.540 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:58.536+0000 7f961f59e640 1 -- 172.21.15.169:0/1411096055 shutdown_connections 2023-10-26T01:31:58.540 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:58.536+0000 7f961f59e640 1 -- 172.21.15.169:0/1411096055 wait complete. 2023-10-26T01:31:58.544 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:58.538+0000 7f961f59e640 1 Processor -- start 2023-10-26T01:31:58.545 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:58.539+0000 7f961f59e640 1 -- start start 2023-10-26T01:31:58.545 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:58.540+0000 7f961f59e640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f962009a5c0 0x7f9620071270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:58.546 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:58.540+0000 7f961f59e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96200ac6d0 0x7f9620158200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:58.546 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:58.540+0000 7f961f59e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f96200c9970 0x7f962006f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:58.547 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:58.540+0000 7f961f59e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9620158740 con 0x7f96200c9970 2023-10-26T01:31:58.547 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:58.540+0000 7f961f59e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f962006f840 con 0x7f96200ac6d0 2023-10-26T01:31:58.547 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:58.540+0000 7f961f59e640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9620070660 con 0x7f962009a5c0 2023-10-26T01:31:58.548 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:58.540+0000 7f961ed9d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f96200c9970 0x7f962006f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:58.548 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:58.540+0000 7f961ed9d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f96200c9970 0x7f962006f300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:44346/0 (socket says 172.21.15.169:44346) 2023-10-26T01:31:58.548 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:58.540+0000 7f961ed9d640 1 -- 172.21.15.169:0/1616590038 learned_addr learned my addr 172.21.15.169:0/1616590038 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:31:58.549 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:58.540+0000 7f961e59c640 1 --2- 172.21.15.169:0/1616590038 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f962009a5c0 0x7f9620071270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:58.549 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:58.540+0000 7f9617fff640 1 --2- 172.21.15.169:0/1616590038 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96200ac6d0 0x7f9620158200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:58.550 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:58.541+0000 7f961ed9d640 1 -- 172.21.15.169:0/1616590038 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f962009a5c0 msgr2=0x7f9620071270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:58.550 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:58.541+0000 7f961ed9d640 1 --2- 172.21.15.169:0/1616590038 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f962009a5c0 0x7f9620071270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:58.551 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:58.541+0000 7f961ed9d640 1 -- 172.21.15.169:0/1616590038 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96200ac6d0 msgr2=0x7f9620158200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:58.551 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:58.541+0000 7f961ed9d640 1 --2- 172.21.15.169:0/1616590038 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96200ac6d0 0x7f9620158200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:58.551 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:58.541+0000 7f961ed9d640 1 -- 172.21.15.169:0/1616590038 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9618009900 con 0x7f96200c9970 2023-10-26T01:31:58.552 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:58.541+0000 7f9617fff640 1 --2- 172.21.15.169:0/1616590038 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96200ac6d0 0x7f9620158200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:31:58.552 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:58.541+0000 7f961ed9d640 1 --2- 172.21.15.169:0/1616590038 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f96200c9970 0x7f962006f300 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f9618037040 tx=0x7f9618035ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:58.552 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:58.541+0000 7f9616ffd640 1 -- 172.21.15.169:0/1616590038 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9618004240 con 0x7f96200c9970 2023-10-26T01:31:58.552 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:58.541+0000 7f9616ffd640 1 -- 172.21.15.169:0/1616590038 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9618004490 con 0x7f96200c9970 2023-10-26T01:31:58.553 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:58.541+0000 7f961f59e640 1 -- 172.21.15.169:0/1616590038 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f962006fa50 con 0x7f96200c9970 2023-10-26T01:31:58.553 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:58.541+0000 7f9616ffd640 1 -- 172.21.15.169:0/1616590038 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9618004820 con 0x7f96200c9970 2023-10-26T01:31:58.553 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:58.542+0000 7f961f59e640 1 -- 172.21.15.169:0/1616590038 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9620070010 con 0x7f96200c9970 2023-10-26T01:31:58.553 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:58.543+0000 7f961f59e640 1 -- 172.21.15.169:0/1616590038 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9620073c10 con 0x7f96200c9970 2023-10-26T01:31:58.554 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:58.544+0000 7f9616ffd640 1 -- 172.21.15.169:0/1616590038 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f961803b040 con 0x7f96200c9970 2023-10-26T01:31:58.554 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:58.545+0000 7f9616ffd640 1 --2- 172.21.15.169:0/1616590038 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f960006ae40 0x7f960006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:58.555 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:58.545+0000 7f9616ffd640 1 -- 172.21.15.169:0/1616590038 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f96180ac280 con 0x7f96200c9970 2023-10-26T01:31:58.555 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:58.545+0000 7f961e59c640 1 --2- 172.21.15.169:0/1616590038 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f960006ae40 0x7f960006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:58.555 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:58.547+0000 7f961e59c640 1 --2- 172.21.15.169:0/1616590038 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f960006ae40 0x7f960006d300 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9610000f60 tx=0x7f9610009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:58.555 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:58.549+0000 7f9616ffd640 1 -- 172.21.15.169:0/1616590038 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9618077290 con 0x7f96200c9970 2023-10-26T01:31:58.741 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:58.740+0000 7f961f59e640 1 -- 172.21.15.169:0/1616590038 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9620071920 con 0x7f96200c9970 2023-10-26T01:31:58.745 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:58.743+0000 7f9616ffd640 1 -- 172.21.15.169:0/1616590038 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16630 (secure 0 0 0) 0x7f961807a8b0 con 0x7f96200c9970 2023-10-26T01:31:58.745 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:31:58.746 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":34,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:31:57.758453+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-10-26T01:31:55.401509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-10-26T01:31:56.776031+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:31:58.748 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:58.746+0000 7f961f59e640 1 -- 172.21.15.169:0/1616590038 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f960006ae40 msgr2=0x7f960006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:58.748 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:58.746+0000 7f961f59e640 1 --2- 172.21.15.169:0/1616590038 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f960006ae40 0x7f960006d300 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9610000f60 tx=0x7f9610009040 comp rx=0 tx=0).stop 2023-10-26T01:31:58.749 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:58.746+0000 7f961f59e640 1 -- 172.21.15.169:0/1616590038 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f96200c9970 msgr2=0x7f962006f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:58.749 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:58.746+0000 7f961f59e640 1 --2- 172.21.15.169:0/1616590038 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f96200c9970 0x7f962006f300 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f9618037040 tx=0x7f9618035ad0 comp rx=0 tx=0).stop 2023-10-26T01:31:58.749 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:58.746+0000 7f961f59e640 1 -- 172.21.15.169:0/1616590038 shutdown_connections 2023-10-26T01:31:58.750 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:58.746+0000 7f961f59e640 1 --2- 172.21.15.169:0/1616590038 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f960006ae40 0x7f960006d300 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:58.750 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:58.746+0000 7f961f59e640 1 --2- 172.21.15.169:0/1616590038 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f96200c9970 0x7f962006f300 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:58.750 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:58.746+0000 7f961f59e640 1 --2- 172.21.15.169:0/1616590038 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96200ac6d0 0x7f9620158200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:58.750 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:58.746+0000 7f961f59e640 1 --2- 172.21.15.169:0/1616590038 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f962009a5c0 0x7f9620071270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:58.751 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:58.746+0000 7f961f59e640 1 -- 172.21.15.169:0/1616590038 >> 172.21.15.169:0/1616590038 conn(0x7f96200aedd0 msgr2=0x7f96200ab690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:58.751 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:58.746+0000 7f961f59e640 1 -- 172.21.15.169:0/1616590038 shutdown_connections 2023-10-26T01:31:58.751 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:58.746+0000 7f961f59e640 1 -- 172.21.15.169:0/1616590038 wait complete. 2023-10-26T01:31:58.765 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:31:58.979 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:58.977+0000 7f349f3a9640 1 -- 172.21.15.169:0/625212271 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f349814e790 msgr2=0x7f34980aab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:58.979 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:58.977+0000 7f349f3a9640 1 --2- 172.21.15.169:0/625212271 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f349814e790 0x7f34980aab40 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f349806eaf0 tx=0x7f349402bd70 comp rx=0 tx=0).stop 2023-10-26T01:31:58.979 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:58.977+0000 7f349f3a9640 1 -- 172.21.15.169:0/625212271 shutdown_connections 2023-10-26T01:31:58.980 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:58.977+0000 7f349f3a9640 1 --2- 172.21.15.169:0/625212271 >> v2:172.21.15.170:3300/0 conn(0x7f3498158850 0x7f34980cd720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:58.980 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:58.977+0000 7f349f3a9640 1 --2- 172.21.15.169:0/625212271 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f349814e790 0x7f34980aab40 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:58.980 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:58.977+0000 7f349f3a9640 1 -- 172.21.15.169:0/625212271 >> 172.21.15.169:0/625212271 conn(0x7f3498069930 msgr2=0x7f3498069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:58.980 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:58.977+0000 7f349f3a9640 1 -- 172.21.15.169:0/625212271 shutdown_connections 2023-10-26T01:31:58.980 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:58.977+0000 7f349f3a9640 1 -- 172.21.15.169:0/625212271 wait complete. 2023-10-26T01:31:58.984 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:58.978+0000 7f349f3a9640 1 Processor -- start 2023-10-26T01:31:58.984 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:58.979+0000 7f349f3a9640 1 -- start start 2023-10-26T01:31:58.984 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:58.979+0000 7f349f3a9640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34980cdc60 0x7f349814bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:58.984 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:58.979+0000 7f349f3a9640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f349814e790 0x7f34981f18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:58.984 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:58.979+0000 7f349f3a9640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3498158850 0x7f3498070bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:58.985 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:58.979+0000 7f349f3a9640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34981f1de0 con 0x7f34980cdc60 2023-10-26T01:31:58.985 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:58.979+0000 7f349f3a9640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3498155b40 con 0x7f3498158850 2023-10-26T01:31:58.985 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:58.979+0000 7f349f3a9640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3498071d10 con 0x7f349814e790 2023-10-26T01:31:58.985 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:58.980+0000 7f349d145640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34980cdc60 0x7f349814bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:58.986 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:58.980+0000 7f349d145640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34980cdc60 0x7f349814bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:44366/0 (socket says 172.21.15.169:44366) 2023-10-26T01:31:58.986 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:58.980+0000 7f349d145640 1 -- 172.21.15.169:0/2901946208 learned_addr learned my addr 172.21.15.169:0/2901946208 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:31:58.986 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:58.980+0000 7f349d145640 1 -- 172.21.15.169:0/2901946208 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f349814e790 msgr2=0x7f34981f18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:58.986 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:58.980+0000 7f349d145640 1 --2- 172.21.15.169:0/2901946208 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f349814e790 0x7f34981f18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:58.987 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:58.980+0000 7f349d946640 1 --2- 172.21.15.169:0/2901946208 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3498158850 0x7f3498070bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:58.987 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:58.980+0000 7f349d145640 1 -- 172.21.15.169:0/2901946208 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3498158850 msgr2=0x7f3498070bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:58.987 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:58.980+0000 7f349d145640 1 --2- 172.21.15.169:0/2901946208 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3498158850 0x7f3498070bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:58.987 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:58.980+0000 7f349d145640 1 -- 172.21.15.169:0/2901946208 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3494009490 con 0x7f34980cdc60 2023-10-26T01:31:58.988 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:58.980+0000 7f349d145640 1 --2- 172.21.15.169:0/2901946208 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34980cdc60 0x7f349814bab0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f3494000c00 tx=0x7f34940039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:58.988 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:58.981+0000 7f348dffb640 1 -- 172.21.15.169:0/2901946208 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3494034640 con 0x7f34980cdc60 2023-10-26T01:31:58.988 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:58.981+0000 7f348dffb640 1 -- 172.21.15.169:0/2901946208 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f34940348b0 con 0x7f34980cdc60 2023-10-26T01:31:58.988 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:58.981+0000 7f349f3a9640 1 -- 172.21.15.169:0/2901946208 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3498071470 con 0x7f34980cdc60 2023-10-26T01:31:58.988 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:58.981+0000 7f348dffb640 1 -- 172.21.15.169:0/2901946208 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3494034cc0 con 0x7f34980cdc60 2023-10-26T01:31:58.989 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:58.981+0000 7f349f3a9640 1 -- 172.21.15.169:0/2901946208 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3498071a30 con 0x7f34980cdc60 2023-10-26T01:31:58.989 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:58.982+0000 7f349f3a9640 1 -- 172.21.15.169:0/2901946208 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f349804db10 con 0x7f34980cdc60 2023-10-26T01:31:58.990 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:58.983+0000 7f348dffb640 1 -- 172.21.15.169:0/2901946208 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f349403b040 con 0x7f34980cdc60 2023-10-26T01:31:58.990 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:58.984+0000 7f348dffb640 1 --2- 172.21.15.169:0/2901946208 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f347c06ae40 0x7f347c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:58.991 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:58.984+0000 7f348dffb640 1 -- 172.21.15.169:0/2901946208 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f34940abc50 con 0x7f34980cdc60 2023-10-26T01:31:58.991 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:58.984+0000 7f349c944640 1 --2- 172.21.15.169:0/2901946208 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f347c06ae40 0x7f347c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:58.991 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:58.986+0000 7f349c944640 1 --2- 172.21.15.169:0/2901946208 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f347c06ae40 0x7f347c06d300 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f349806ca80 tx=0x7f3488006c90 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:58.991 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:58.988+0000 7f348dffb640 1 -- 172.21.15.169:0/2901946208 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f34940b1020 con 0x7f34980cdc60 2023-10-26T01:31:59.183 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:59.181+0000 7f349f3a9640 1 -- 172.21.15.169:0/2901946208 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f34980aca70 con 0x7f34980cdc60 2023-10-26T01:31:59.185 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:59.184+0000 7f348dffb640 1 -- 172.21.15.169:0/2901946208 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+16617 (secure 0 0 0) 0x7f3494041090 con 0x7f34980cdc60 2023-10-26T01:31:59.186 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:31:59.186 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":35,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:31:58.899685+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-10-26T01:31:55.401509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-10-26T01:31:56.776031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:31:59.189 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:59.187+0000 7f349f3a9640 1 -- 172.21.15.169:0/2901946208 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f347c06ae40 msgr2=0x7f347c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:59.189 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:59.187+0000 7f349f3a9640 1 --2- 172.21.15.169:0/2901946208 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f347c06ae40 0x7f347c06d300 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f349806ca80 tx=0x7f3488006c90 comp rx=0 tx=0).stop 2023-10-26T01:31:59.190 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:59.187+0000 7f349f3a9640 1 -- 172.21.15.169:0/2901946208 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34980cdc60 msgr2=0x7f349814bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:59.190 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:59.187+0000 7f349f3a9640 1 --2- 172.21.15.169:0/2901946208 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34980cdc60 0x7f349814bab0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f3494000c00 tx=0x7f34940039e0 comp rx=0 tx=0).stop 2023-10-26T01:31:59.190 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:59.187+0000 7f349f3a9640 1 -- 172.21.15.169:0/2901946208 shutdown_connections 2023-10-26T01:31:59.190 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:59.187+0000 7f349f3a9640 1 --2- 172.21.15.169:0/2901946208 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f347c06ae40 0x7f347c06d300 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:59.190 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:59.187+0000 7f349f3a9640 1 --2- 172.21.15.169:0/2901946208 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3498158850 0x7f3498070bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:59.191 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:59.187+0000 7f349f3a9640 1 --2- 172.21.15.169:0/2901946208 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f349814e790 0x7f34981f18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:59.191 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:59.187+0000 7f349f3a9640 1 --2- 172.21.15.169:0/2901946208 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34980cdc60 0x7f349814bab0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:59.191 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:59.187+0000 7f349f3a9640 1 -- 172.21.15.169:0/2901946208 >> 172.21.15.169:0/2901946208 conn(0x7f3498069930 msgr2=0x7f349815a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:59.191 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:59.187+0000 7f349f3a9640 1 -- 172.21.15.169:0/2901946208 shutdown_connections 2023-10-26T01:31:59.191 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:59.187+0000 7f349f3a9640 1 -- 172.21.15.169:0/2901946208 wait complete. 2023-10-26T01:31:59.208 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-26T01:31:59.413 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:59.411+0000 7f8b9367c640 1 -- 172.21.15.169:0/2178033821 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b74005e10 msgr2=0x7f8b74006290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:59.413 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:59.411+0000 7f8b9367c640 1 --2- 172.21.15.169:0/2178033821 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b74005e10 0x7f8b74006290 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f8b8400c640 tx=0x7f8b74006900 comp rx=0 tx=0).stop 2023-10-26T01:31:59.413 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:59.411+0000 7f8b91c19640 1 -- 172.21.15.169:0/2178033821 reap_dead start 2023-10-26T01:31:59.414 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:59.411+0000 7f8b9367c640 1 -- 172.21.15.169:0/2178033821 shutdown_connections 2023-10-26T01:31:59.414 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:59.411+0000 7f8b9367c640 1 -- 172.21.15.169:0/2178033821 >> 172.21.15.169:0/2178033821 conn(0x7f8b8c069930 msgr2=0x7f8b8c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:31:59.414 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:59.411+0000 7f8b9367c640 1 -- 172.21.15.169:0/2178033821 shutdown_connections 2023-10-26T01:31:59.414 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:59.411+0000 7f8b9367c640 1 -- 172.21.15.169:0/2178033821 wait complete. 2023-10-26T01:31:59.416 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:59.413+0000 7f8b9367c640 1 Processor -- start 2023-10-26T01:31:59.417 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:59.413+0000 7f8b9367c640 1 -- start start 2023-10-26T01:31:59.417 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:59.414+0000 7f8b9367c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b74005e10 0x7f8b8c0ad760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:59.417 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:59.414+0000 7f8b9367c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b8c06edc0 0x7f8b8c1f1050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:59.417 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:59.414+0000 7f8b9367c640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f8b8c0c8830 0x7f8b8c0c8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:59.418 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:59.414+0000 7f8b9367c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b8c0c91f0 con 0x7f8b74005e10 2023-10-26T01:31:59.418 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:59.414+0000 7f8b9367c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b8c06f5b0 con 0x7f8b8c06edc0 2023-10-26T01:31:59.418 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:59.414+0000 7f8b9367c640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b8c0ae290 con 0x7f8b8c0c8830 2023-10-26T01:31:59.418 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:59.414+0000 7f8b91418640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b74005e10 0x7f8b8c0ad760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:59.418 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:59.414+0000 7f8b91418640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b74005e10 0x7f8b8c0ad760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:44378/0 (socket says 172.21.15.169:44378) 2023-10-26T01:31:59.419 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:59.414+0000 7f8b91418640 1 -- 172.21.15.169:0/3820689253 learned_addr learned my addr 172.21.15.169:0/3820689253 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:31:59.419 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:59.414+0000 7f8b91418640 1 -- 172.21.15.169:0/3820689253 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f8b8c0c8830 msgr2=0x7f8b8c0c8cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:59.419 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:59.414+0000 7f8b91418640 1 --2- 172.21.15.169:0/3820689253 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f8b8c0c8830 0x7f8b8c0c8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:59.419 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:59.414+0000 7f8b91418640 1 -- 172.21.15.169:0/3820689253 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b8c06edc0 msgr2=0x7f8b8c1f1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:31:59.420 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:59.414+0000 7f8b91418640 1 --2- 172.21.15.169:0/3820689253 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b8c06edc0 0x7f8b8c1f1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:31:59.420 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:59.414+0000 7f8b91418640 1 -- 172.21.15.169:0/3820689253 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b8c06f220 con 0x7f8b74005e10 2023-10-26T01:31:59.420 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:59.415+0000 7f8b91418640 1 --2- 172.21.15.169:0/3820689253 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b74005e10 0x7f8b8c0ad760 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c00d530 tx=0x7f8b7c00d9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:59.420 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:59.415+0000 7f8b89ffb640 1 -- 172.21.15.169:0/3820689253 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b7c012630 con 0x7f8b74005e10 2023-10-26T01:31:59.420 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:59.415+0000 7f8b89ffb640 1 -- 172.21.15.169:0/3820689253 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8b7c0127b0 con 0x7f8b74005e10 2023-10-26T01:31:59.421 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:59.415+0000 7f8b89ffb640 1 -- 172.21.15.169:0/3820689253 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b7c012bc0 con 0x7f8b74005e10 2023-10-26T01:31:59.421 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:59.415+0000 7f8b9367c640 1 -- 172.21.15.169:0/3820689253 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b8c06f800 con 0x7f8b74005e10 2023-10-26T01:31:59.421 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:59.415+0000 7f8b9367c640 1 -- 172.21.15.169:0/3820689253 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b8c0ade60 con 0x7f8b74005e10 2023-10-26T01:31:59.422 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:59.416+0000 7f8b9367c640 1 -- 172.21.15.169:0/3820689253 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b8c04db10 con 0x7f8b74005e10 2023-10-26T01:31:59.422 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:59.417+0000 7f8b89ffb640 1 -- 172.21.15.169:0/3820689253 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f8b7c01b400 con 0x7f8b74005e10 2023-10-26T01:31:59.423 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:59.418+0000 7f8b89ffb640 1 --2- 172.21.15.169:0/3820689253 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f8b7806ae40 0x7f8b7806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:31:59.423 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:59.418+0000 7f8b89ffb640 1 -- 172.21.15.169:0/3820689253 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8b7c0898b0 con 0x7f8b74005e10 2023-10-26T01:31:59.423 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:59.418+0000 7f8b90c17640 1 --2- 172.21.15.169:0/3820689253 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f8b7806ae40 0x7f8b7806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:31:59.423 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:59.420+0000 7f8b90c17640 1 --2- 172.21.15.169:0/3820689253 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f8b7806ae40 0x7f8b7806d300 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f8b80005fd0 tx=0x7f8b80005d40 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:31:59.424 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:59.423+0000 7f8b89ffb640 1 -- 172.21.15.169:0/3820689253 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8b7c0548c0 con 0x7f8b74005e10 2023-10-26T01:31:59.653 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:31:59.652+0000 7f8b9367c640 1 -- 172.21.15.169:0/3820689253 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f8b8c0ae080 con 0x7f8b74005e10 2023-10-26T01:32:00.340 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:00.338+0000 7f8b89ffb640 1 -- 172.21.15.169:0/3820689253 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v14) v1 ==== 118+0+0 (secure 0 0 0) 0x7f8b7c016020 con 0x7f8b74005e10 2023-10-26T01:32:00.344 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:00.342+0000 7f8b9367c640 1 -- 172.21.15.169:0/3820689253 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f8b7806ae40 msgr2=0x7f8b7806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:00.344 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:00.342+0000 7f8b9367c640 1 --2- 172.21.15.169:0/3820689253 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f8b7806ae40 0x7f8b7806d300 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f8b80005fd0 tx=0x7f8b80005d40 comp rx=0 tx=0).stop 2023-10-26T01:32:00.344 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:00.342+0000 7f8b9367c640 1 -- 172.21.15.169:0/3820689253 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b74005e10 msgr2=0x7f8b8c0ad760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:00.344 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:00.342+0000 7f8b9367c640 1 --2- 172.21.15.169:0/3820689253 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b74005e10 0x7f8b8c0ad760 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c00d530 tx=0x7f8b7c00d9f0 comp rx=0 tx=0).stop 2023-10-26T01:32:00.345 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:00.342+0000 7f8b9367c640 1 -- 172.21.15.169:0/3820689253 shutdown_connections 2023-10-26T01:32:00.345 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:00.342+0000 7f8b9367c640 1 --2- 172.21.15.169:0/3820689253 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f8b7806ae40 0x7f8b7806d300 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:00.345 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:00.342+0000 7f8b9367c640 1 --2- 172.21.15.169:0/3820689253 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f8b8c0c8830 0x7f8b8c0c8cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:00.345 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:00.342+0000 7f8b9367c640 1 --2- 172.21.15.169:0/3820689253 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b8c06edc0 0x7f8b8c1f1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:00.345 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:00.342+0000 7f8b9367c640 1 --2- 172.21.15.169:0/3820689253 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b74005e10 0x7f8b8c0ad760 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:00.346 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:00.342+0000 7f8b9367c640 1 -- 172.21.15.169:0/3820689253 >> 172.21.15.169:0/3820689253 conn(0x7f8b8c069930 msgr2=0x7f8b8c151b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:00.346 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:00.342+0000 7f8b9367c640 1 -- 172.21.15.169:0/3820689253 shutdown_connections 2023-10-26T01:32:00.346 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:00.342+0000 7f8b9367c640 1 -- 172.21.15.169:0/3820689253 wait complete. 2023-10-26T01:32:00.366 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-26T01:32:00.367 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-26T01:32:00.590 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:00.588+0000 7f6469768640 1 -- 172.21.15.169:0/2127393373 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6450003970 msgr2=0x7f6450003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:00.591 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:00.588+0000 7f6469768640 1 --2- 172.21.15.169:0/2127393373 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6450003970 0x7f6450003d90 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f6454009e50 tx=0x7f64500057f0 comp rx=0 tx=0).stop 2023-10-26T01:32:00.591 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:00.588+0000 7f64637fe640 1 -- 172.21.15.169:0/2127393373 reap_dead start 2023-10-26T01:32:00.591 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:00.588+0000 7f6469768640 1 -- 172.21.15.169:0/2127393373 shutdown_connections 2023-10-26T01:32:00.591 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:00.588+0000 7f6469768640 1 -- 172.21.15.169:0/2127393373 >> 172.21.15.169:0/2127393373 conn(0x7f64640af8a0 msgr2=0x7f64640afc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:00.592 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:00.588+0000 7f6469768640 1 -- 172.21.15.169:0/2127393373 shutdown_connections 2023-10-26T01:32:00.592 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:00.588+0000 7f6469768640 1 -- 172.21.15.169:0/2127393373 wait complete. 2023-10-26T01:32:00.594 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:00.590+0000 7f6469768640 1 Processor -- start 2023-10-26T01:32:00.595 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:00.590+0000 7f6469768640 1 -- start start 2023-10-26T01:32:00.595 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:00.591+0000 7f6469768640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6450003970 0x7f64640c2b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:00.595 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:00.591+0000 7f6469768640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6464148da0 0x7f64640ae8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:00.595 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:00.591+0000 7f6469768640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64640aee10 0x7f64640c5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:00.595 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:00.591+0000 7f6469768640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f646414a380 con 0x7f64640aee10 2023-10-26T01:32:00.596 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:00.591+0000 7f6469768640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64640af290 con 0x7f6464148da0 2023-10-26T01:32:00.596 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:00.591+0000 7f6469768640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64640c3050 con 0x7f6450003970 2023-10-26T01:32:00.596 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:00.592+0000 7f64637fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64640aee10 0x7f64640c5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:00.596 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:00.592+0000 7f64637fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64640aee10 0x7f64640c5450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:44396/0 (socket says 172.21.15.169:44396) 2023-10-26T01:32:00.597 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:00.592+0000 7f64637fe640 1 -- 172.21.15.169:0/1138236902 learned_addr learned my addr 172.21.15.169:0/1138236902 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:32:00.597 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:00.592+0000 7f64627fc640 1 --2- 172.21.15.169:0/1138236902 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6464148da0 0x7f64640ae8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:00.597 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:00.592+0000 7f6462ffd640 1 --2- 172.21.15.169:0/1138236902 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6450003970 0x7f64640c2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:00.597 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:00.592+0000 7f64637fe640 1 -- 172.21.15.169:0/1138236902 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6450003970 msgr2=0x7f64640c2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:00.597 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:00.592+0000 7f64637fe640 1 --2- 172.21.15.169:0/1138236902 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6450003970 0x7f64640c2b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:00.598 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:00.592+0000 7f64637fe640 1 -- 172.21.15.169:0/1138236902 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6464148da0 msgr2=0x7f64640ae8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:00.598 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:00.592+0000 7f64637fe640 1 --2- 172.21.15.169:0/1138236902 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6464148da0 0x7f64640ae8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:00.598 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:00.592+0000 7f64637fe640 1 -- 172.21.15.169:0/1138236902 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6464149700 con 0x7f64640aee10 2023-10-26T01:32:00.599 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:00.592+0000 7f64627fc640 1 --2- 172.21.15.169:0/1138236902 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6464148da0 0x7f64640ae8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:32:00.599 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:00.592+0000 7f64637fe640 1 --2- 172.21.15.169:0/1138236902 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64640aee10 0x7f64640c5450 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f6464072fd0 tx=0x7f645c00d9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:00.599 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:00.593+0000 7f645b7fe640 1 -- 172.21.15.169:0/1138236902 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f645c004230 con 0x7f64640aee10 2023-10-26T01:32:00.599 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:00.593+0000 7f645b7fe640 1 -- 172.21.15.169:0/1138236902 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f645c0044a0 con 0x7f64640aee10 2023-10-26T01:32:00.599 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:00.593+0000 7f645b7fe640 1 -- 172.21.15.169:0/1138236902 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f645c00f8f0 con 0x7f64640aee10 2023-10-26T01:32:00.600 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:00.593+0000 7f6469768640 1 -- 172.21.15.169:0/1138236902 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64641499e0 con 0x7f64640aee10 2023-10-26T01:32:00.600 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:00.593+0000 7f6469768640 1 -- 172.21.15.169:0/1138236902 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6464149fa0 con 0x7f64640aee10 2023-10-26T01:32:00.601 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:00.594+0000 7f6469768640 1 -- 172.21.15.169:0/1138236902 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f646414a4c0 con 0x7f64640aee10 2023-10-26T01:32:00.601 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:00.595+0000 7f645b7fe640 1 -- 172.21.15.169:0/1138236902 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f645c00fa50 con 0x7f64640aee10 2023-10-26T01:32:00.601 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:00.596+0000 7f645b7fe640 1 --2- 172.21.15.169:0/1138236902 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f644406ae40 0x7f644406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:00.602 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:00.596+0000 7f645b7fe640 1 -- 172.21.15.169:0/1138236902 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f645c088ed0 con 0x7f64640aee10 2023-10-26T01:32:00.602 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:00.597+0000 7f6462ffd640 1 --2- 172.21.15.169:0/1138236902 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f644406ae40 0x7f644406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:00.602 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:00.598+0000 7f6462ffd640 1 --2- 172.21.15.169:0/1138236902 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f644406ae40 0x7f644406d300 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6454036000 tx=0x7f6454035040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:00.603 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:00.600+0000 7f645b7fe640 1 -- 172.21.15.169:0/1138236902 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f646414a4c0 con 0x7f64640aee10 2023-10-26T01:32:00.831 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:00.829+0000 7f6469768640 1 -- 172.21.15.169:0/1138236902 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f646414a4c0 con 0x7f64640aee10 2023-10-26T01:32:01.361 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:01.359+0000 7f645b7fe640 1 -- 172.21.15.169:0/1138236902 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v15) v1 ==== 122+0+0 (secure 0 0 0) 0x7f646414a4c0 con 0x7f64640aee10 2023-10-26T01:32:01.364 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:01.362+0000 7f6469768640 1 -- 172.21.15.169:0/1138236902 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f644406ae40 msgr2=0x7f644406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:01.365 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:01.362+0000 7f6469768640 1 --2- 172.21.15.169:0/1138236902 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f644406ae40 0x7f644406d300 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6454036000 tx=0x7f6454035040 comp rx=0 tx=0).stop 2023-10-26T01:32:01.365 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:01.362+0000 7f6469768640 1 -- 172.21.15.169:0/1138236902 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64640aee10 msgr2=0x7f64640c5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:01.365 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:01.362+0000 7f6469768640 1 --2- 172.21.15.169:0/1138236902 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64640aee10 0x7f64640c5450 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f6464072fd0 tx=0x7f645c00d9a0 comp rx=0 tx=0).stop 2023-10-26T01:32:01.365 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:01.363+0000 7f6469768640 1 -- 172.21.15.169:0/1138236902 shutdown_connections 2023-10-26T01:32:01.365 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:01.363+0000 7f6469768640 1 --2- 172.21.15.169:0/1138236902 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f644406ae40 0x7f644406d300 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:01.366 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:01.363+0000 7f6469768640 1 --2- 172.21.15.169:0/1138236902 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64640aee10 0x7f64640c5450 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:01.366 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:01.363+0000 7f6469768640 1 --2- 172.21.15.169:0/1138236902 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6464148da0 0x7f64640ae8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:01.366 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:01.363+0000 7f6469768640 1 --2- 172.21.15.169:0/1138236902 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6450003970 0x7f64640c2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:01.366 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:01.363+0000 7f6469768640 1 -- 172.21.15.169:0/1138236902 >> 172.21.15.169:0/1138236902 conn(0x7f64640af8a0 msgr2=0x7f64640afc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:01.367 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:01.363+0000 7f6469768640 1 -- 172.21.15.169:0/1138236902 shutdown_connections 2023-10-26T01:32:01.367 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:01.363+0000 7f6469768640 1 -- 172.21.15.169:0/1138236902 wait complete. 2023-10-26T01:32:01.386 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-26T01:32:01.386 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T01:32:01.610 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:01.608+0000 7fb715e64640 1 -- 172.21.15.169:0/1078603460 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb71015b610 msgr2=0x7fb71006ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:01.611 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:01.608+0000 7fb715e64640 1 --2- 172.21.15.169:0/1078603460 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb71015b610 0x7fb71006ed30 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb700005fd0 tx=0x7fb70002c2d0 comp rx=0 tx=0).stop 2023-10-26T01:32:01.611 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:01.608+0000 7fb715e64640 1 -- 172.21.15.169:0/1078603460 shutdown_connections 2023-10-26T01:32:01.611 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:01.608+0000 7fb715e64640 1 --2- 172.21.15.169:0/1078603460 >> v2:172.21.15.170:3300/0 conn(0x7fb7100ac4e0 0x7fb7100cd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:01.611 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:01.608+0000 7fb715e64640 1 --2- 172.21.15.169:0/1078603460 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb71015b610 0x7fb71006ed30 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:01.612 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:01.608+0000 7fb715e64640 1 -- 172.21.15.169:0/1078603460 >> 172.21.15.169:0/1078603460 conn(0x7fb710069930 msgr2=0x7fb710069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:01.612 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:01.608+0000 7fb715e64640 1 -- 172.21.15.169:0/1078603460 shutdown_connections 2023-10-26T01:32:01.612 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:01.608+0000 7fb715e64640 1 -- 172.21.15.169:0/1078603460 wait complete. 2023-10-26T01:32:01.613 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:01.610+0000 7fb715e64640 1 Processor -- start 2023-10-26T01:32:01.614 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:01.610+0000 7fb715e64640 1 -- start start 2023-10-26T01:32:01.614 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:01.611+0000 7fb715e64640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb7100ac4e0 0x7fb71015bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:01.614 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:01.611+0000 7fb715e64640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb7100ac940 0x7fb71015e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:01.614 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:01.611+0000 7fb715e64640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb71015b610 0x7fb7100972d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:01.615 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:01.611+0000 7fb715e64640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb71015d070 con 0x7fb7100ac4e0 2023-10-26T01:32:01.615 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:01.611+0000 7fb715e64640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb710098600 con 0x7fb7100ac940 2023-10-26T01:32:01.615 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:01.611+0000 7fb715e64640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb71015c1e0 con 0x7fb71015b610 2023-10-26T01:32:01.615 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:01.612+0000 7fb70f7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb7100ac4e0 0x7fb71015bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:01.615 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:01.612+0000 7fb70f7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb7100ac4e0 0x7fb71015bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:44410/0 (socket says 172.21.15.169:44410) 2023-10-26T01:32:01.616 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:01.612+0000 7fb70f7fe640 1 -- 172.21.15.169:0/816786177 learned_addr learned my addr 172.21.15.169:0/816786177 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:32:01.616 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:01.612+0000 7fb70effd640 1 --2- 172.21.15.169:0/816786177 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb7100ac940 0x7fb71015e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:01.616 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:01.612+0000 7fb70ffff640 1 --2- 172.21.15.169:0/816786177 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb71015b610 0x7fb7100972d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:01.616 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:01.612+0000 7fb70f7fe640 1 -- 172.21.15.169:0/816786177 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb71015b610 msgr2=0x7fb7100972d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:01.616 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:01.612+0000 7fb70f7fe640 1 --2- 172.21.15.169:0/816786177 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb71015b610 0x7fb7100972d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:01.617 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:01.612+0000 7fb70f7fe640 1 -- 172.21.15.169:0/816786177 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb7100ac940 msgr2=0x7fb71015e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:01.617 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:01.612+0000 7fb70f7fe640 1 --2- 172.21.15.169:0/816786177 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb7100ac940 0x7fb71015e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:01.617 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:01.612+0000 7fb70f7fe640 1 -- 172.21.15.169:0/816786177 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb700009900 con 0x7fb7100ac4e0 2023-10-26T01:32:01.618 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:01.613+0000 7fb70f7fe640 1 --2- 172.21.15.169:0/816786177 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb7100ac4e0 0x7fb71015bca0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fb700006010 tx=0x7fb700003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:01.619 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:01.613+0000 7fb6f7fff640 1 -- 172.21.15.169:0/816786177 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb700004450 con 0x7fb7100ac4e0 2023-10-26T01:32:01.619 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:01.613+0000 7fb6f7fff640 1 -- 172.21.15.169:0/816786177 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb7000046a0 con 0x7fb7100ac4e0 2023-10-26T01:32:01.619 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:01.613+0000 7fb6f7fff640 1 -- 172.21.15.169:0/816786177 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb70003f680 con 0x7fb7100ac4e0 2023-10-26T01:32:01.619 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:01.613+0000 7fb715e64640 1 -- 172.21.15.169:0/816786177 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb71015c450 con 0x7fb7100ac4e0 2023-10-26T01:32:01.619 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:01.613+0000 7fb715e64640 1 -- 172.21.15.169:0/816786177 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb71015cc90 con 0x7fb7100ac4e0 2023-10-26T01:32:01.620 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:01.614+0000 7fb715e64640 1 -- 172.21.15.169:0/816786177 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb71004db10 con 0x7fb7100ac4e0 2023-10-26T01:32:01.620 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:01.615+0000 7fb6f7fff640 1 -- 172.21.15.169:0/816786177 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fb70003b040 con 0x7fb7100ac4e0 2023-10-26T01:32:01.620 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:01.616+0000 7fb6f7fff640 1 --2- 172.21.15.169:0/816786177 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb6fc06ae40 0x7fb6fc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:01.620 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:01.616+0000 7fb6f7fff640 1 -- 172.21.15.169:0/816786177 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb7000ac3d0 con 0x7fb7100ac4e0 2023-10-26T01:32:01.620 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:01.617+0000 7fb70effd640 1 --2- 172.21.15.169:0/816786177 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb6fc06ae40 0x7fb6fc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:01.621 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:01.618+0000 7fb70effd640 1 --2- 172.21.15.169:0/816786177 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb6fc06ae40 0x7fb6fc06d300 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb7100ccf80 tx=0x7fb704005f90 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:01.622 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:01.620+0000 7fb6f7fff640 1 -- 172.21.15.169:0/816786177 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb700077470 con 0x7fb7100ac4e0 2023-10-26T01:32:01.848 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:01.846+0000 7fb715e64640 1 -- 172.21.15.169:0/816786177 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb710067470 con 0x7fb7100ac4e0 2023-10-26T01:32:01.849 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:01.848+0000 7fb6f7fff640 1 -- 172.21.15.169:0/816786177 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4913 (secure 0 0 0) 0x7fb70007aa90 con 0x7fb7100ac4e0 2023-10-26T01:32:01.850 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:32:01.850 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4640,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/61096113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":61096113},{"type":"v1","addr":"172.21.15.170:6836","nonce":61096113}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4762,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2825046244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2825046244},{"type":"v1","addr":"172.21.15.169:6833","nonce":2825046244}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4777,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/258866976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":258866976},{"type":"v1","addr":"172.21.15.170:6837","nonce":258866976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T01:31:57.759251+0000","modified":"2023-10-26T01:32:00.831213+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5196},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5196":{"gid":5196,"name":"c","rank":0,"incarnation":11,"state":"up:active","state_seq":4,"addr":"172.21.15.169:6835/871978528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":871978528},{"type":"v1","addr":"172.21.15.169:6835","nonce":871978528}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-26T01:32:01.850 INFO:teuthology.orchestra.run.smithi169.stderr:dumped fsmap epoch 15 2023-10-26T01:32:01.852 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:01.850+0000 7fb715e64640 1 -- 172.21.15.169:0/816786177 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb6fc06ae40 msgr2=0x7fb6fc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:01.853 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:01.850+0000 7fb715e64640 1 --2- 172.21.15.169:0/816786177 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb6fc06ae40 0x7fb6fc06d300 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb7100ccf80 tx=0x7fb704005f90 comp rx=0 tx=0).stop 2023-10-26T01:32:01.853 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:01.850+0000 7fb715e64640 1 -- 172.21.15.169:0/816786177 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb7100ac4e0 msgr2=0x7fb71015bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:01.853 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:01.850+0000 7fb715e64640 1 --2- 172.21.15.169:0/816786177 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb7100ac4e0 0x7fb71015bca0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fb700006010 tx=0x7fb700003c20 comp rx=0 tx=0).stop 2023-10-26T01:32:01.853 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:01.851+0000 7fb715e64640 1 -- 172.21.15.169:0/816786177 shutdown_connections 2023-10-26T01:32:01.854 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:01.851+0000 7fb715e64640 1 --2- 172.21.15.169:0/816786177 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb6fc06ae40 0x7fb6fc06d300 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:01.854 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:01.851+0000 7fb715e64640 1 --2- 172.21.15.169:0/816786177 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb71015b610 0x7fb7100972d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:01.854 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:01.851+0000 7fb715e64640 1 --2- 172.21.15.169:0/816786177 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb7100ac940 0x7fb71015e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:01.854 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:01.851+0000 7fb715e64640 1 --2- 172.21.15.169:0/816786177 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb7100ac4e0 0x7fb71015bca0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:01.854 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:01.851+0000 7fb715e64640 1 -- 172.21.15.169:0/816786177 >> 172.21.15.169:0/816786177 conn(0x7fb710069930 msgr2=0x7fb7100ad910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:01.855 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:01.851+0000 7fb715e64640 1 -- 172.21.15.169:0/816786177 shutdown_connections 2023-10-26T01:32:01.855 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:01.851+0000 7fb715e64640 1 -- 172.21.15.169:0/816786177 wait complete. 2023-10-26T01:32:01.871 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:32:02.091 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:02.089+0000 7fc369052640 1 -- 172.21.15.169:0/1489587256 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3640c7ea0 msgr2=0x7fc36406a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:02.092 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:02.089+0000 7fc369052640 1 --2- 172.21.15.169:0/1489587256 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3640c7ea0 0x7fc36406a990 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fc3640ca700 tx=0x7fc35402bd70 comp rx=0 tx=0).stop 2023-10-26T01:32:02.092 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:02.090+0000 7fc369052640 1 -- 172.21.15.169:0/1489587256 shutdown_connections 2023-10-26T01:32:02.092 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:02.090+0000 7fc369052640 1 --2- 172.21.15.169:0/1489587256 >> v2:172.21.15.170:3300/0 conn(0x7fc3640cc4f0 0x7fc3640c96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:02.092 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:02.090+0000 7fc369052640 1 --2- 172.21.15.169:0/1489587256 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3640c7ea0 0x7fc36406a990 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:02.093 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:02.090+0000 7fc369052640 1 -- 172.21.15.169:0/1489587256 >> 172.21.15.169:0/1489587256 conn(0x7fc3640aedd0 msgr2=0x7fc3640af1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:02.093 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:02.090+0000 7fc369052640 1 -- 172.21.15.169:0/1489587256 shutdown_connections 2023-10-26T01:32:02.093 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:02.090+0000 7fc369052640 1 -- 172.21.15.169:0/1489587256 wait complete. 2023-10-26T01:32:02.094 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:02.091+0000 7fc369052640 1 Processor -- start 2023-10-26T01:32:02.094 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:02.092+0000 7fc369052640 1 -- start start 2023-10-26T01:32:02.095 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:02.092+0000 7fc369052640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3640c7ea0 0x7fc3640aba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:02.095 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:02.093+0000 7fc369052640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc3640cc4f0 0x7fc3640b1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:02.095 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:02.093+0000 7fc369052640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc3640cc9a0 0x7fc3641448c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:02.095 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:02.093+0000 7fc369052640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc364098e40 con 0x7fc3640c7ea0 2023-10-26T01:32:02.095 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:02.093+0000 7fc369052640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3640b2230 con 0x7fc3640cc9a0 2023-10-26T01:32:02.096 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:02.093+0000 7fc369052640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3640b2460 con 0x7fc3640cc4f0 2023-10-26T01:32:02.096 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:02.093+0000 7fc363fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3640c7ea0 0x7fc3640aba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:02.096 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:02.093+0000 7fc363fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3640c7ea0 0x7fc3640aba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:44430/0 (socket says 172.21.15.169:44430) 2023-10-26T01:32:02.096 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:02.093+0000 7fc363fff640 1 -- 172.21.15.169:0/2091355974 learned_addr learned my addr 172.21.15.169:0/2091355974 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:32:02.097 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:02.093+0000 7fc363fff640 1 -- 172.21.15.169:0/2091355974 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc3640cc4f0 msgr2=0x7fc3640b1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:02.098 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:02.093+0000 7fc363fff640 1 --2- 172.21.15.169:0/2091355974 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc3640cc4f0 0x7fc3640b1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:02.098 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:02.093+0000 7fc363fff640 1 -- 172.21.15.169:0/2091355974 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc3640cc9a0 msgr2=0x7fc3641448c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:02.098 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:02.093+0000 7fc363fff640 1 --2- 172.21.15.169:0/2091355974 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc3640cc9a0 0x7fc3641448c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:02.098 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:02.093+0000 7fc363fff640 1 -- 172.21.15.169:0/2091355974 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc354009490 con 0x7fc3640c7ea0 2023-10-26T01:32:02.098 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:02.093+0000 7fc363fff640 1 --2- 172.21.15.169:0/2091355974 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3640c7ea0 0x7fc3640aba40 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fc354000c00 tx=0x7fc3540039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:02.099 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:02.094+0000 7fc360ff9640 1 -- 172.21.15.169:0/2091355974 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc354034440 con 0x7fc3640c7ea0 2023-10-26T01:32:02.099 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:02.094+0000 7fc360ff9640 1 -- 172.21.15.169:0/2091355974 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc3540346b0 con 0x7fc3640c7ea0 2023-10-26T01:32:02.099 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:02.094+0000 7fc360ff9640 1 -- 172.21.15.169:0/2091355974 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc354034ac0 con 0x7fc3640c7ea0 2023-10-26T01:32:02.099 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:02.094+0000 7fc369052640 1 -- 172.21.15.169:0/2091355974 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3640b2740 con 0x7fc3640c7ea0 2023-10-26T01:32:02.099 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:02.094+0000 7fc369052640 1 -- 172.21.15.169:0/2091355974 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc364098a60 con 0x7fc3640c7ea0 2023-10-26T01:32:02.100 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:02.095+0000 7fc369052640 1 -- 172.21.15.169:0/2091355974 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc364098f80 con 0x7fc3640c7ea0 2023-10-26T01:32:02.100 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:02.096+0000 7fc360ff9640 1 -- 172.21.15.169:0/2091355974 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fc35403b040 con 0x7fc3640c7ea0 2023-10-26T01:32:02.100 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:02.097+0000 7fc360ff9640 1 --2- 172.21.15.169:0/2091355974 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc35006ae40 0x7fc35006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:02.101 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:02.097+0000 7fc360ff9640 1 -- 172.21.15.169:0/2091355974 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc3540abc60 con 0x7fc3640c7ea0 2023-10-26T01:32:02.101 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:02.097+0000 7fc3637fe640 1 --2- 172.21.15.169:0/2091355974 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc35006ae40 0x7fc35006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:02.101 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:02.098+0000 7fc3637fe640 1 --2- 172.21.15.169:0/2091355974 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc35006ae40 0x7fc35006d300 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc3640ac2d0 tx=0x7fc358006c90 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:02.102 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:02.101+0000 7fc360ff9640 1 -- 172.21.15.169:0/2091355974 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc364098f80 con 0x7fc3640c7ea0 2023-10-26T01:32:02.293 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:02.291+0000 7fc369052640 1 -- 172.21.15.169:0/2091355974 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc364098f80 con 0x7fc3640c7ea0 2023-10-26T01:32:02.295 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:02.294+0000 7fc360ff9640 1 -- 172.21.15.169:0/2091355974 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+16617 (secure 0 0 0) 0x7fc364098f80 con 0x7fc3640c7ea0 2023-10-26T01:32:02.296 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:32:02.297 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":35,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:31:58.899685+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-10-26T01:31:55.401509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-10-26T01:31:56.776031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:32:02.298 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:02.296+0000 7fc369052640 1 -- 172.21.15.169:0/2091355974 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc35006ae40 msgr2=0x7fc35006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:02.298 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:02.296+0000 7fc369052640 1 --2- 172.21.15.169:0/2091355974 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc35006ae40 0x7fc35006d300 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc3640ac2d0 tx=0x7fc358006c90 comp rx=0 tx=0).stop 2023-10-26T01:32:02.299 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:02.296+0000 7fc369052640 1 -- 172.21.15.169:0/2091355974 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3640c7ea0 msgr2=0x7fc3640aba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:02.299 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:02.296+0000 7fc369052640 1 --2- 172.21.15.169:0/2091355974 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3640c7ea0 0x7fc3640aba40 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fc354000c00 tx=0x7fc3540039e0 comp rx=0 tx=0).stop 2023-10-26T01:32:02.299 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:02.296+0000 7fc369052640 1 -- 172.21.15.169:0/2091355974 shutdown_connections 2023-10-26T01:32:02.299 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:02.296+0000 7fc369052640 1 --2- 172.21.15.169:0/2091355974 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc35006ae40 0x7fc35006d300 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:02.299 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:02.296+0000 7fc369052640 1 --2- 172.21.15.169:0/2091355974 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc3640cc9a0 0x7fc3641448c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:02.300 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:02.296+0000 7fc369052640 1 --2- 172.21.15.169:0/2091355974 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc3640cc4f0 0x7fc3640b1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:02.300 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:02.296+0000 7fc369052640 1 --2- 172.21.15.169:0/2091355974 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3640c7ea0 0x7fc3640aba40 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:02.300 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:02.296+0000 7fc369052640 1 -- 172.21.15.169:0/2091355974 >> 172.21.15.169:0/2091355974 conn(0x7fc3640aedd0 msgr2=0x7fc364159670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:02.300 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:02.297+0000 7fc369052640 1 -- 172.21.15.169:0/2091355974 shutdown_connections 2023-10-26T01:32:02.300 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:02.297+0000 7fc369052640 1 -- 172.21.15.169:0/2091355974 wait complete. 2023-10-26T01:32:02.319 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-26T01:32:02.319 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-26T01:32:02.541 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:02.539+0000 7fb6e1844640 1 -- 172.21.15.169:0/4178524074 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb6dc158850 msgr2=0x7fb6dc0aab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:02.541 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:02.539+0000 7fb6e1844640 1 --2- 172.21.15.169:0/4178524074 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb6dc158850 0x7fb6dc0aab40 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc0accf0 tx=0x7fb6c402bd10 comp rx=0 tx=0).stop 2023-10-26T01:32:02.541 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:02.539+0000 7fb6e1844640 1 -- 172.21.15.169:0/4178524074 shutdown_connections 2023-10-26T01:32:02.542 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:02.539+0000 7fb6e1844640 1 --2- 172.21.15.169:0/4178524074 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb6dc06ad30 0x7fb6dc06b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:02.542 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:02.539+0000 7fb6e1844640 1 --2- 172.21.15.169:0/4178524074 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb6dc158850 0x7fb6dc0aab40 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:02.542 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:02.539+0000 7fb6e1844640 1 --2- 172.21.15.169:0/4178524074 >> v2:172.21.15.170:3300/0 conn(0x7fb6dc06ce20 0x7fb6dc14e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:02.542 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:02.539+0000 7fb6e1844640 1 -- 172.21.15.169:0/4178524074 >> 172.21.15.169:0/4178524074 conn(0x7fb6dc069930 msgr2=0x7fb6dc069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:02.542 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:02.539+0000 7fb6e1844640 1 -- 172.21.15.169:0/4178524074 shutdown_connections 2023-10-26T01:32:02.543 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:02.539+0000 7fb6e1844640 1 -- 172.21.15.169:0/4178524074 wait complete. 2023-10-26T01:32:02.544 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:02.541+0000 7fb6e1844640 1 Processor -- start 2023-10-26T01:32:02.544 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:02.541+0000 7fb6e1844640 1 -- start start 2023-10-26T01:32:02.544 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:02.542+0000 7fb6e1844640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb6dc06ad30 0x7fb6dc095bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:02.545 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:02.542+0000 7fb6e1844640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb6dc06ce20 0x7fb6dc1463b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:02.545 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:02.542+0000 7fb6e1844640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb6dc158850 0x7fb6dc162270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:02.545 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:02.542+0000 7fb6e1844640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6dc0960f0 con 0x7fb6dc06ce20 2023-10-26T01:32:02.545 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:02.542+0000 7fb6e1844640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6dc157300 con 0x7fb6dc06ad30 2023-10-26T01:32:02.545 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:02.542+0000 7fb6e1844640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6dc161be0 con 0x7fb6dc158850 2023-10-26T01:32:02.546 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:02.542+0000 7fb6d27fc640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb6dc06ce20 0x7fb6dc1463b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:02.546 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:02.542+0000 7fb6d27fc640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb6dc06ce20 0x7fb6dc1463b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:44458/0 (socket says 172.21.15.169:44458) 2023-10-26T01:32:02.547 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:02.543+0000 7fb6d27fc640 1 -- 172.21.15.169:0/2506562240 learned_addr learned my addr 172.21.15.169:0/2506562240 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:32:02.547 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:02.543+0000 7fb6daffd640 1 --2- 172.21.15.169:0/2506562240 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb6dc06ad30 0x7fb6dc095bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:02.547 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:02.543+0000 7fb6db7fe640 1 --2- 172.21.15.169:0/2506562240 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb6dc158850 0x7fb6dc162270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:02.548 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:02.543+0000 7fb6d27fc640 1 -- 172.21.15.169:0/2506562240 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb6dc158850 msgr2=0x7fb6dc162270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:02.549 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:02.543+0000 7fb6d27fc640 1 --2- 172.21.15.169:0/2506562240 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb6dc158850 0x7fb6dc162270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:02.549 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:02.543+0000 7fb6d27fc640 1 -- 172.21.15.169:0/2506562240 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb6dc06ad30 msgr2=0x7fb6dc095bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:02.549 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:02.543+0000 7fb6d27fc640 1 --2- 172.21.15.169:0/2506562240 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb6dc06ad30 0x7fb6dc095bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:02.549 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:02.543+0000 7fb6d27fc640 1 -- 172.21.15.169:0/2506562240 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6c4009560 con 0x7fb6dc06ce20 2023-10-26T01:32:02.549 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:02.543+0000 7fb6daffd640 1 --2- 172.21.15.169:0/2506562240 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb6dc06ad30 0x7fb6dc095bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T01:32:02.550 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:02.543+0000 7fb6d27fc640 1 --2- 172.21.15.169:0/2506562240 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb6dc06ce20 0x7fb6dc1463b0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fb6c4009850 tx=0x7fb6c402da40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:02.550 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:02.544+0000 7fb6d8ff9640 1 -- 172.21.15.169:0/2506562240 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c4003800 con 0x7fb6dc06ce20 2023-10-26T01:32:02.550 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:02.544+0000 7fb6d8ff9640 1 -- 172.21.15.169:0/2506562240 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb6c4003a50 con 0x7fb6dc06ce20 2023-10-26T01:32:02.550 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:02.544+0000 7fb6d8ff9640 1 -- 172.21.15.169:0/2506562240 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c4003e20 con 0x7fb6dc06ce20 2023-10-26T01:32:02.550 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:02.544+0000 7fb6e1844640 1 -- 172.21.15.169:0/2506562240 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6dc1575a0 con 0x7fb6dc06ce20 2023-10-26T01:32:02.551 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:02.544+0000 7fb6e1844640 1 -- 172.21.15.169:0/2506562240 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6dc145e90 con 0x7fb6dc06ce20 2023-10-26T01:32:02.551 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:02.545+0000 7fb6e1844640 1 -- 172.21.15.169:0/2506562240 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6dc04db10 con 0x7fb6dc06ce20 2023-10-26T01:32:02.551 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:02.546+0000 7fb6d8ff9640 1 -- 172.21.15.169:0/2506562240 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fb6c403e450 con 0x7fb6dc06ce20 2023-10-26T01:32:02.552 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:02.547+0000 7fb6d8ff9640 1 --2- 172.21.15.169:0/2506562240 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb6c006ae40 0x7fb6c006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:02.552 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:02.548+0000 7fb6d8ff9640 1 -- 172.21.15.169:0/2506562240 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb6c40ac1a0 con 0x7fb6dc06ce20 2023-10-26T01:32:02.552 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:02.548+0000 7fb6daffd640 1 --2- 172.21.15.169:0/2506562240 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb6c006ae40 0x7fb6c006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:02.552 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:02.549+0000 7fb6daffd640 1 --2- 172.21.15.169:0/2506562240 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb6c006ae40 0x7fb6c006d300 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc0058c0 tx=0x7fb6cc005850 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:02.553 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:02.552+0000 7fb6d8ff9640 1 -- 172.21.15.169:0/2506562240 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb6c4077240 con 0x7fb6dc06ce20 2023-10-26T01:32:02.744 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:02.742+0000 7fb6e1844640 1 -- 172.21.15.169:0/2506562240 --> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb6dc0cd720 con 0x7fb6c006ae40 2023-10-26T01:32:02.773 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:02.771+0000 7fb6d8ff9640 1 -- 172.21.15.169:0/2506562240 <== mgr.4112 v2:172.21.15.170:6832/1187156918 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+282711 (secure 0 0 0) 0x7fb6cc00b930 con 0x7fb6c006ae40 2023-10-26T01:32:02.775 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:32:02.779 INFO:teuthology.orchestra.run.smithi169.stderr:dumped all 2023-10-26T01:32:02.781 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:02.779+0000 7fb6e1844640 1 -- 172.21.15.169:0/2506562240 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb6c006ae40 msgr2=0x7fb6c006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:02.781 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:02.779+0000 7fb6e1844640 1 --2- 172.21.15.169:0/2506562240 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb6c006ae40 0x7fb6c006d300 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc0058c0 tx=0x7fb6cc005850 comp rx=0 tx=0).stop 2023-10-26T01:32:02.781 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:02.779+0000 7fb6e1844640 1 -- 172.21.15.169:0/2506562240 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb6dc06ce20 msgr2=0x7fb6dc1463b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:02.781 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:02.779+0000 7fb6e1844640 1 --2- 172.21.15.169:0/2506562240 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb6dc06ce20 0x7fb6dc1463b0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fb6c4009850 tx=0x7fb6c402da40 comp rx=0 tx=0).stop 2023-10-26T01:32:02.781 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:02.779+0000 7fb6e1844640 1 -- 172.21.15.169:0/2506562240 shutdown_connections 2023-10-26T01:32:02.782 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:02.779+0000 7fb6e1844640 1 --2- 172.21.15.169:0/2506562240 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb6c006ae40 0x7fb6c006d300 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:02.782 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:02.779+0000 7fb6e1844640 1 --2- 172.21.15.169:0/2506562240 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb6dc06ce20 0x7fb6dc1463b0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:02.782 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:02.779+0000 7fb6e1844640 1 --2- 172.21.15.169:0/2506562240 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb6dc158850 0x7fb6dc162270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:02.782 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:02.779+0000 7fb6e1844640 1 --2- 172.21.15.169:0/2506562240 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb6dc06ad30 0x7fb6dc095bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:02.782 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:02.779+0000 7fb6e1844640 1 -- 172.21.15.169:0/2506562240 >> 172.21.15.169:0/2506562240 conn(0x7fb6dc069930 msgr2=0x7fb6dc14f000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:02.783 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:02.779+0000 7fb6e1844640 1 -- 172.21.15.169:0/2506562240 shutdown_connections 2023-10-26T01:32:02.783 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:02.779+0000 7fb6e1844640 1 -- 172.21.15.169:0/2506562240 wait complete. 2023-10-26T01:32:02.800 INFO:teuthology.orchestra.run.smithi169.stdout:{"pg_ready":true,"pg_map":{"version":74,"stamp":"2023-10-26T01:32:02.157714+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":178,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":2379,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560120,"statfs":{"total":1600000000,"available":1597563202,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.878983"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.920724+0000","last_change":"2023-10-26T01:31:57.911065+0000","last_active":"2023-10-26T01:31:58.920724+0000","last_peered":"2023-10-26T01:31:58.920724+0000","last_clean":"2023-10-26T01:31:58.920724+0000","last_became_active":"2023-10-26T01:31:57.910336+0000","last_became_peered":"2023-10-26T01:31:57.910336+0000","last_unstale":"2023-10-26T01:31:58.920724+0000","last_undegraded":"2023-10-26T01:31:58.920724+0000","last_fullsized":"2023-10-26T01:31:58.920724+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:15:19.809493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.912093+0000","last_change":"2023-10-26T01:31:57.917140+0000","last_active":"2023-10-26T01:31:58.912093+0000","last_peered":"2023-10-26T01:31:58.912093+0000","last_clean":"2023-10-26T01:31:58.912093+0000","last_became_active":"2023-10-26T01:31:57.916213+0000","last_became_peered":"2023-10-26T01:31:57.916213+0000","last_unstale":"2023-10-26T01:31:58.912093+0000","last_undegraded":"2023-10-26T01:31:58.912093+0000","last_fullsized":"2023-10-26T01:31:58.912093+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:02:28.206674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:59.260702+0000","last_change":"2023-10-26T01:31:57.919209+0000","last_active":"2023-10-26T01:31:59.260702+0000","last_peered":"2023-10-26T01:31:59.260702+0000","last_clean":"2023-10-26T01:31:59.260702+0000","last_became_active":"2023-10-26T01:31:57.917976+0000","last_became_peered":"2023-10-26T01:31:57.917976+0000","last_unstale":"2023-10-26T01:31:59.260702+0000","last_undegraded":"2023-10-26T01:31:59.260702+0000","last_fullsized":"2023-10-26T01:31:59.260702+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:35:25.011238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.919886+0000","last_change":"2023-10-26T01:31:57.928984+0000","last_active":"2023-10-26T01:31:58.919886+0000","last_peered":"2023-10-26T01:31:58.919886+0000","last_clean":"2023-10-26T01:31:58.919886+0000","last_became_active":"2023-10-26T01:31:57.928540+0000","last_became_peered":"2023-10-26T01:31:57.928540+0000","last_unstale":"2023-10-26T01:31:58.919886+0000","last_undegraded":"2023-10-26T01:31:58.919886+0000","last_fullsized":"2023-10-26T01:31:58.919886+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:14:39.347339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:59.258508+0000","last_change":"2023-10-26T01:31:57.928731+0000","last_active":"2023-10-26T01:31:59.258508+0000","last_peered":"2023-10-26T01:31:59.258508+0000","last_clean":"2023-10-26T01:31:59.258508+0000","last_became_active":"2023-10-26T01:31:57.928011+0000","last_became_peered":"2023-10-26T01:31:57.928011+0000","last_unstale":"2023-10-26T01:31:59.258508+0000","last_undegraded":"2023-10-26T01:31:59.258508+0000","last_fullsized":"2023-10-26T01:31:59.258508+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T01:48:32.945169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.911932+0000","last_change":"2023-10-26T01:31:57.931239+0000","last_active":"2023-10-26T01:31:58.911932+0000","last_peered":"2023-10-26T01:31:58.911932+0000","last_clean":"2023-10-26T01:31:58.911932+0000","last_became_active":"2023-10-26T01:31:57.930748+0000","last_became_peered":"2023-10-26T01:31:57.930748+0000","last_unstale":"2023-10-26T01:31:58.911932+0000","last_undegraded":"2023-10-26T01:31:58.911932+0000","last_fullsized":"2023-10-26T01:31:58.911932+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:21:21.898287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.916295+0000","last_change":"2023-10-26T01:31:57.923586+0000","last_active":"2023-10-26T01:31:58.916295+0000","last_peered":"2023-10-26T01:31:58.916295+0000","last_clean":"2023-10-26T01:31:58.916295+0000","last_became_active":"2023-10-26T01:31:57.923083+0000","last_became_peered":"2023-10-26T01:31:57.923083+0000","last_unstale":"2023-10-26T01:31:58.916295+0000","last_undegraded":"2023-10-26T01:31:58.916295+0000","last_fullsized":"2023-10-26T01:31:58.916295+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:51:36.984592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.914291+0000","last_change":"2023-10-26T01:31:57.927342+0000","last_active":"2023-10-26T01:31:58.914291+0000","last_peered":"2023-10-26T01:31:58.914291+0000","last_clean":"2023-10-26T01:31:58.914291+0000","last_became_active":"2023-10-26T01:31:57.927089+0000","last_became_peered":"2023-10-26T01:31:57.927089+0000","last_unstale":"2023-10-26T01:31:58.914291+0000","last_undegraded":"2023-10-26T01:31:58.914291+0000","last_fullsized":"2023-10-26T01:31:58.914291+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:43:09.209981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.913612+0000","last_change":"2023-10-26T01:31:57.928812+0000","last_active":"2023-10-26T01:31:58.913612+0000","last_peered":"2023-10-26T01:31:58.913612+0000","last_clean":"2023-10-26T01:31:58.913612+0000","last_became_active":"2023-10-26T01:31:57.928510+0000","last_became_peered":"2023-10-26T01:31:57.928510+0000","last_unstale":"2023-10-26T01:31:58.913612+0000","last_undegraded":"2023-10-26T01:31:58.913612+0000","last_fullsized":"2023-10-26T01:31:58.913612+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:42:38.754671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.920515+0000","last_change":"2023-10-26T01:31:57.924955+0000","last_active":"2023-10-26T01:31:58.920515+0000","last_peered":"2023-10-26T01:31:58.920515+0000","last_clean":"2023-10-26T01:31:58.920515+0000","last_became_active":"2023-10-26T01:31:57.924221+0000","last_became_peered":"2023-10-26T01:31:57.924221+0000","last_unstale":"2023-10-26T01:31:58.920515+0000","last_undegraded":"2023-10-26T01:31:58.920515+0000","last_fullsized":"2023-10-26T01:31:58.920515+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:24:57.262505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.917212+0000","last_change":"2023-10-26T01:31:57.924771+0000","last_active":"2023-10-26T01:31:58.917212+0000","last_peered":"2023-10-26T01:31:58.917212+0000","last_clean":"2023-10-26T01:31:58.917212+0000","last_became_active":"2023-10-26T01:31:57.924320+0000","last_became_peered":"2023-10-26T01:31:57.924320+0000","last_unstale":"2023-10-26T01:31:58.917212+0000","last_undegraded":"2023-10-26T01:31:58.917212+0000","last_fullsized":"2023-10-26T01:31:58.917212+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:21:21.898287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:59.259571+0000","last_change":"2023-10-26T01:31:57.915944+0000","last_active":"2023-10-26T01:31:59.259571+0000","last_peered":"2023-10-26T01:31:59.259571+0000","last_clean":"2023-10-26T01:31:59.259571+0000","last_became_active":"2023-10-26T01:31:57.914541+0000","last_became_peered":"2023-10-26T01:31:57.914541+0000","last_unstale":"2023-10-26T01:31:59.259571+0000","last_undegraded":"2023-10-26T01:31:59.259571+0000","last_fullsized":"2023-10-26T01:31:59.259571+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:14:39.347339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:59.258382+0000","last_change":"2023-10-26T01:31:57.931634+0000","last_active":"2023-10-26T01:31:59.258382+0000","last_peered":"2023-10-26T01:31:59.258382+0000","last_clean":"2023-10-26T01:31:59.258382+0000","last_became_active":"2023-10-26T01:31:57.930738+0000","last_became_peered":"2023-10-26T01:31:57.930738+0000","last_unstale":"2023-10-26T01:31:59.258382+0000","last_undegraded":"2023-10-26T01:31:59.258382+0000","last_fullsized":"2023-10-26T01:31:59.258382+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:29:13.336121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.912828+0000","last_change":"2023-10-26T01:31:57.929136+0000","last_active":"2023-10-26T01:31:58.912828+0000","last_peered":"2023-10-26T01:31:58.912828+0000","last_clean":"2023-10-26T01:31:58.912828+0000","last_became_active":"2023-10-26T01:31:57.928427+0000","last_became_peered":"2023-10-26T01:31:57.928427+0000","last_unstale":"2023-10-26T01:31:58.912828+0000","last_undegraded":"2023-10-26T01:31:58.912828+0000","last_fullsized":"2023-10-26T01:31:58.912828+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:52:00.679485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:59.260174+0000","last_change":"2023-10-26T01:31:57.916184+0000","last_active":"2023-10-26T01:31:59.260174+0000","last_peered":"2023-10-26T01:31:59.260174+0000","last_clean":"2023-10-26T01:31:59.260174+0000","last_became_active":"2023-10-26T01:31:57.912088+0000","last_became_peered":"2023-10-26T01:31:57.912088+0000","last_unstale":"2023-10-26T01:31:59.260174+0000","last_undegraded":"2023-10-26T01:31:59.260174+0000","last_fullsized":"2023-10-26T01:31:59.260174+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:24:57.262505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.917565+0000","last_change":"2023-10-26T01:31:57.927881+0000","last_active":"2023-10-26T01:31:58.917565+0000","last_peered":"2023-10-26T01:31:58.917565+0000","last_clean":"2023-10-26T01:31:58.917565+0000","last_became_active":"2023-10-26T01:31:57.926937+0000","last_became_peered":"2023-10-26T01:31:57.926937+0000","last_unstale":"2023-10-26T01:31:58.917565+0000","last_undegraded":"2023-10-26T01:31:58.917565+0000","last_fullsized":"2023-10-26T01:31:58.917565+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:35:25.011238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.915589+0000","last_change":"2023-10-26T01:31:57.923381+0000","last_active":"2023-10-26T01:31:58.915589+0000","last_peered":"2023-10-26T01:31:58.915589+0000","last_clean":"2023-10-26T01:31:58.915589+0000","last_became_active":"2023-10-26T01:31:57.922670+0000","last_became_peered":"2023-10-26T01:31:57.922670+0000","last_unstale":"2023-10-26T01:31:58.915589+0000","last_undegraded":"2023-10-26T01:31:58.915589+0000","last_fullsized":"2023-10-26T01:31:58.915589+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T01:34:44.444835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.913006+0000","last_change":"2023-10-26T01:31:57.923066+0000","last_active":"2023-10-26T01:31:58.913006+0000","last_peered":"2023-10-26T01:31:58.913006+0000","last_clean":"2023-10-26T01:31:58.913006+0000","last_became_active":"2023-10-26T01:31:57.922786+0000","last_became_peered":"2023-10-26T01:31:57.922786+0000","last_unstale":"2023-10-26T01:31:58.913006+0000","last_undegraded":"2023-10-26T01:31:58.913006+0000","last_fullsized":"2023-10-26T01:31:58.913006+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:01:00.051777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.911935+0000","last_change":"2023-10-26T01:31:57.928697+0000","last_active":"2023-10-26T01:31:58.911935+0000","last_peered":"2023-10-26T01:31:58.911935+0000","last_clean":"2023-10-26T01:31:58.911935+0000","last_became_active":"2023-10-26T01:31:57.925505+0000","last_became_peered":"2023-10-26T01:31:57.925505+0000","last_unstale":"2023-10-26T01:31:58.911935+0000","last_undegraded":"2023-10-26T01:31:58.911935+0000","last_fullsized":"2023-10-26T01:31:58.911935+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:24:57.262505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:59.258675+0000","last_change":"2023-10-26T01:31:57.929809+0000","last_active":"2023-10-26T01:31:59.258675+0000","last_peered":"2023-10-26T01:31:59.258675+0000","last_clean":"2023-10-26T01:31:59.258675+0000","last_became_active":"2023-10-26T01:31:57.928334+0000","last_became_peered":"2023-10-26T01:31:57.928334+0000","last_unstale":"2023-10-26T01:31:59.258675+0000","last_undegraded":"2023-10-26T01:31:59.258675+0000","last_fullsized":"2023-10-26T01:31:59.258675+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:44:10.920523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.915810+0000","last_change":"2023-10-26T01:31:57.930806+0000","last_active":"2023-10-26T01:31:58.915810+0000","last_peered":"2023-10-26T01:31:58.915810+0000","last_clean":"2023-10-26T01:31:58.915810+0000","last_became_active":"2023-10-26T01:31:57.930542+0000","last_became_peered":"2023-10-26T01:31:57.930542+0000","last_unstale":"2023-10-26T01:31:58.915810+0000","last_undegraded":"2023-10-26T01:31:58.915810+0000","last_fullsized":"2023-10-26T01:31:58.915810+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:51:09.298744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.914981+0000","last_change":"2023-10-26T01:31:57.931637+0000","last_active":"2023-10-26T01:31:58.914981+0000","last_peered":"2023-10-26T01:31:58.914981+0000","last_clean":"2023-10-26T01:31:58.914981+0000","last_became_active":"2023-10-26T01:31:57.931346+0000","last_became_peered":"2023-10-26T01:31:57.931346+0000","last_unstale":"2023-10-26T01:31:58.914981+0000","last_undegraded":"2023-10-26T01:31:58.914981+0000","last_fullsized":"2023-10-26T01:31:58.914981+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:31:13.736811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:59.259353+0000","last_change":"2023-10-26T01:31:57.924443+0000","last_active":"2023-10-26T01:31:59.259353+0000","last_peered":"2023-10-26T01:31:59.259353+0000","last_clean":"2023-10-26T01:31:59.259353+0000","last_became_active":"2023-10-26T01:31:57.921067+0000","last_became_peered":"2023-10-26T01:31:57.921067+0000","last_unstale":"2023-10-26T01:31:59.259353+0000","last_undegraded":"2023-10-26T01:31:59.259353+0000","last_fullsized":"2023-10-26T01:31:59.259353+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:13:25.337972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:59.259584+0000","last_change":"2023-10-26T01:31:57.924740+0000","last_active":"2023-10-26T01:31:59.259584+0000","last_peered":"2023-10-26T01:31:59.259584+0000","last_clean":"2023-10-26T01:31:59.259584+0000","last_became_active":"2023-10-26T01:31:57.924031+0000","last_became_peered":"2023-10-26T01:31:57.924031+0000","last_unstale":"2023-10-26T01:31:59.259584+0000","last_undegraded":"2023-10-26T01:31:59.259584+0000","last_fullsized":"2023-10-26T01:31:59.259584+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T01:34:44.444835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.919849+0000","last_change":"2023-10-26T01:31:57.925186+0000","last_active":"2023-10-26T01:31:58.919849+0000","last_peered":"2023-10-26T01:31:58.919849+0000","last_clean":"2023-10-26T01:31:58.919849+0000","last_became_active":"2023-10-26T01:31:57.924666+0000","last_became_peered":"2023-10-26T01:31:57.924666+0000","last_unstale":"2023-10-26T01:31:58.919849+0000","last_undegraded":"2023-10-26T01:31:58.919849+0000","last_fullsized":"2023-10-26T01:31:58.919849+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:43:09.209981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.912077+0000","last_change":"2023-10-26T01:31:57.922444+0000","last_active":"2023-10-26T01:31:58.912077+0000","last_peered":"2023-10-26T01:31:58.912077+0000","last_clean":"2023-10-26T01:31:58.912077+0000","last_became_active":"2023-10-26T01:31:57.922018+0000","last_became_peered":"2023-10-26T01:31:57.922018+0000","last_unstale":"2023-10-26T01:31:58.912077+0000","last_undegraded":"2023-10-26T01:31:58.912077+0000","last_fullsized":"2023-10-26T01:31:58.912077+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:06:00.266085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.919990+0000","last_change":"2023-10-26T01:31:57.930896+0000","last_active":"2023-10-26T01:31:58.919990+0000","last_peered":"2023-10-26T01:31:58.919990+0000","last_clean":"2023-10-26T01:31:58.919990+0000","last_became_active":"2023-10-26T01:31:57.930441+0000","last_became_peered":"2023-10-26T01:31:57.930441+0000","last_unstale":"2023-10-26T01:31:58.919990+0000","last_undegraded":"2023-10-26T01:31:58.919990+0000","last_fullsized":"2023-10-26T01:31:58.919990+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:56:29.283750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:59.259579+0000","last_change":"2023-10-26T01:31:57.918538+0000","last_active":"2023-10-26T01:31:59.259579+0000","last_peered":"2023-10-26T01:31:59.259579+0000","last_clean":"2023-10-26T01:31:59.259579+0000","last_became_active":"2023-10-26T01:31:57.915935+0000","last_became_peered":"2023-10-26T01:31:57.915935+0000","last_unstale":"2023-10-26T01:31:59.259579+0000","last_undegraded":"2023-10-26T01:31:59.259579+0000","last_fullsized":"2023-10-26T01:31:59.259579+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:56:29.283750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.913965+0000","last_change":"2023-10-26T01:31:57.931907+0000","last_active":"2023-10-26T01:31:58.913965+0000","last_peered":"2023-10-26T01:31:58.913965+0000","last_clean":"2023-10-26T01:31:58.913965+0000","last_became_active":"2023-10-26T01:31:57.931618+0000","last_became_peered":"2023-10-26T01:31:57.931618+0000","last_unstale":"2023-10-26T01:31:58.913965+0000","last_undegraded":"2023-10-26T01:31:58.913965+0000","last_fullsized":"2023-10-26T01:31:58.913965+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:17:24.476006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.917038+0000","last_change":"2023-10-26T01:31:57.919923+0000","last_active":"2023-10-26T01:31:58.917038+0000","last_peered":"2023-10-26T01:31:58.917038+0000","last_clean":"2023-10-26T01:31:58.917038+0000","last_became_active":"2023-10-26T01:31:57.919348+0000","last_became_peered":"2023-10-26T01:31:57.919348+0000","last_unstale":"2023-10-26T01:31:58.917038+0000","last_undegraded":"2023-10-26T01:31:58.917038+0000","last_fullsized":"2023-10-26T01:31:58.917038+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:24:57.262505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.921059+0000","last_change":"2023-10-26T01:31:57.926414+0000","last_active":"2023-10-26T01:31:58.921059+0000","last_peered":"2023-10-26T01:31:58.921059+0000","last_clean":"2023-10-26T01:31:58.921059+0000","last_became_active":"2023-10-26T01:31:57.926058+0000","last_became_peered":"2023-10-26T01:31:57.926058+0000","last_unstale":"2023-10-26T01:31:58.921059+0000","last_undegraded":"2023-10-26T01:31:58.921059+0000","last_fullsized":"2023-10-26T01:31:58.921059+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:31:13.736811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.915935+0000","last_change":"2023-10-26T01:31:57.929677+0000","last_active":"2023-10-26T01:31:58.915935+0000","last_peered":"2023-10-26T01:31:58.915935+0000","last_clean":"2023-10-26T01:31:58.915935+0000","last_became_active":"2023-10-26T01:31:57.928493+0000","last_became_peered":"2023-10-26T01:31:57.928493+0000","last_unstale":"2023-10-26T01:31:58.915935+0000","last_undegraded":"2023-10-26T01:31:58.915935+0000","last_fullsized":"2023-10-26T01:31:58.915935+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:30:35.357843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:59.259004+0000","last_change":"2023-10-26T01:31:57.923481+0000","last_active":"2023-10-26T01:31:59.259004+0000","last_peered":"2023-10-26T01:31:59.259004+0000","last_clean":"2023-10-26T01:31:59.259004+0000","last_became_active":"2023-10-26T01:31:57.923159+0000","last_became_peered":"2023-10-26T01:31:57.923159+0000","last_unstale":"2023-10-26T01:31:59.259004+0000","last_undegraded":"2023-10-26T01:31:59.259004+0000","last_fullsized":"2023-10-26T01:31:59.259004+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:52:33.901445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"34'4","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.920754+0000","last_change":"2023-10-26T01:31:57.313244+0000","last_active":"2023-10-26T01:31:58.920754+0000","last_peered":"2023-10-26T01:31:58.920754+0000","last_clean":"2023-10-26T01:31:58.920754+0000","last_became_active":"2023-10-26T01:31:57.312346+0000","last_became_peered":"2023-10-26T01:31:57.312346+0000","last_unstale":"2023-10-26T01:31:58.920754+0000","last_undegraded":"2023-10-26T01:31:58.920754+0000","last_fullsized":"2023-10-26T01:31:58.920754+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_clean_scrub_stamp":"2023-10-26T01:31:56.263810+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:27:21.313977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.920492+0000","last_change":"2023-10-26T01:31:57.929374+0000","last_active":"2023-10-26T01:31:58.920492+0000","last_peered":"2023-10-26T01:31:58.920492+0000","last_clean":"2023-10-26T01:31:58.920492+0000","last_became_active":"2023-10-26T01:31:57.929078+0000","last_became_peered":"2023-10-26T01:31:57.929078+0000","last_unstale":"2023-10-26T01:31:58.920492+0000","last_undegraded":"2023-10-26T01:31:58.920492+0000","last_fullsized":"2023-10-26T01:31:58.920492+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:42:38.754671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"34'1","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:59.258865+0000","last_change":"2023-10-26T01:31:57.315953+0000","last_active":"2023-10-26T01:31:59.258865+0000","last_peered":"2023-10-26T01:31:59.258865+0000","last_clean":"2023-10-26T01:31:59.258865+0000","last_became_active":"2023-10-26T01:31:57.315806+0000","last_became_peered":"2023-10-26T01:31:57.315806+0000","last_unstale":"2023-10-26T01:31:59.258865+0000","last_undegraded":"2023-10-26T01:31:59.258865+0000","last_fullsized":"2023-10-26T01:31:59.258865+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_clean_scrub_stamp":"2023-10-26T01:31:56.263810+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:20:35.297390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.916144+0000","last_change":"2023-10-26T01:31:57.932521+0000","last_active":"2023-10-26T01:31:58.916144+0000","last_peered":"2023-10-26T01:31:58.916144+0000","last_clean":"2023-10-26T01:31:58.916144+0000","last_became_active":"2023-10-26T01:31:57.932252+0000","last_became_peered":"2023-10-26T01:31:57.932252+0000","last_unstale":"2023-10-26T01:31:58.916144+0000","last_undegraded":"2023-10-26T01:31:58.916144+0000","last_fullsized":"2023-10-26T01:31:58.916144+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:52:33.901445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"34'2","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.332680+0000","last_change":"2023-10-26T01:31:57.307484+0000","last_active":"2023-10-26T01:31:58.332680+0000","last_peered":"2023-10-26T01:31:58.332680+0000","last_clean":"2023-10-26T01:31:58.332680+0000","last_became_active":"2023-10-26T01:31:57.307186+0000","last_became_peered":"2023-10-26T01:31:57.307186+0000","last_unstale":"2023-10-26T01:31:58.332680+0000","last_undegraded":"2023-10-26T01:31:58.332680+0000","last_fullsized":"2023-10-26T01:31:58.332680+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_clean_scrub_stamp":"2023-10-26T01:31:56.263810+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:30:34.344731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.920185+0000","last_change":"2023-10-26T01:31:57.929917+0000","last_active":"2023-10-26T01:31:58.920185+0000","last_peered":"2023-10-26T01:31:58.920185+0000","last_clean":"2023-10-26T01:31:58.920185+0000","last_became_active":"2023-10-26T01:31:57.929167+0000","last_became_peered":"2023-10-26T01:31:57.929167+0000","last_unstale":"2023-10-26T01:31:58.920185+0000","last_undegraded":"2023-10-26T01:31:58.920185+0000","last_fullsized":"2023-10-26T01:31:58.920185+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:51:36.984592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"34'2","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:59.258843+0000","last_change":"2023-10-26T01:31:57.302026+0000","last_active":"2023-10-26T01:31:59.258843+0000","last_peered":"2023-10-26T01:31:59.258843+0000","last_clean":"2023-10-26T01:31:59.258843+0000","last_became_active":"2023-10-26T01:31:57.301463+0000","last_became_peered":"2023-10-26T01:31:57.301463+0000","last_unstale":"2023-10-26T01:31:59.258843+0000","last_undegraded":"2023-10-26T01:31:59.258843+0000","last_fullsized":"2023-10-26T01:31:59.258843+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_clean_scrub_stamp":"2023-10-26T01:31:56.263810+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:46:19.545323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.914785+0000","last_change":"2023-10-26T01:31:57.930260+0000","last_active":"2023-10-26T01:31:58.914785+0000","last_peered":"2023-10-26T01:31:58.914785+0000","last_clean":"2023-10-26T01:31:58.914785+0000","last_became_active":"2023-10-26T01:31:57.929537+0000","last_became_peered":"2023-10-26T01:31:57.929537+0000","last_unstale":"2023-10-26T01:31:58.914785+0000","last_undegraded":"2023-10-26T01:31:58.914785+0000","last_fullsized":"2023-10-26T01:31:58.914785+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:14:39.347339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"34'1","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.914722+0000","last_change":"2023-10-26T01:31:57.302186+0000","last_active":"2023-10-26T01:31:58.914722+0000","last_peered":"2023-10-26T01:31:58.914722+0000","last_clean":"2023-10-26T01:31:58.914722+0000","last_became_active":"2023-10-26T01:31:57.299001+0000","last_became_peered":"2023-10-26T01:31:57.299001+0000","last_unstale":"2023-10-26T01:31:58.914722+0000","last_undegraded":"2023-10-26T01:31:58.914722+0000","last_fullsized":"2023-10-26T01:31:58.914722+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_clean_scrub_stamp":"2023-10-26T01:31:56.263810+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:29:39.990303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:59.259925+0000","last_change":"2023-10-26T01:31:57.919906+0000","last_active":"2023-10-26T01:31:59.259925+0000","last_peered":"2023-10-26T01:31:59.259925+0000","last_clean":"2023-10-26T01:31:59.259925+0000","last_became_active":"2023-10-26T01:31:57.915475+0000","last_became_peered":"2023-10-26T01:31:57.915475+0000","last_unstale":"2023-10-26T01:31:59.259925+0000","last_undegraded":"2023-10-26T01:31:59.259925+0000","last_fullsized":"2023-10-26T01:31:59.259925+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:51:36.984592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"34'1","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.913502+0000","last_change":"2023-10-26T01:31:57.295016+0000","last_active":"2023-10-26T01:31:58.913502+0000","last_peered":"2023-10-26T01:31:58.913502+0000","last_clean":"2023-10-26T01:31:58.913502+0000","last_became_active":"2023-10-26T01:31:57.294352+0000","last_became_peered":"2023-10-26T01:31:57.294352+0000","last_unstale":"2023-10-26T01:31:58.913502+0000","last_undegraded":"2023-10-26T01:31:58.913502+0000","last_fullsized":"2023-10-26T01:31:58.913502+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_clean_scrub_stamp":"2023-10-26T01:31:56.263810+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:41:28.795912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.913210+0000","last_change":"2023-10-26T01:31:57.928375+0000","last_active":"2023-10-26T01:31:58.913210+0000","last_peered":"2023-10-26T01:31:58.913210+0000","last_clean":"2023-10-26T01:31:58.913210+0000","last_became_active":"2023-10-26T01:31:57.927961+0000","last_became_peered":"2023-10-26T01:31:57.927961+0000","last_unstale":"2023-10-26T01:31:58.913210+0000","last_undegraded":"2023-10-26T01:31:58.913210+0000","last_fullsized":"2023-10-26T01:31:58.913210+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:49:05.475592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-26T01:31:57.909884+0000","last_change":"2023-10-26T01:31:57.305633+0000","last_active":"2023-10-26T01:31:57.909884+0000","last_peered":"2023-10-26T01:31:57.909884+0000","last_clean":"2023-10-26T01:31:57.909884+0000","last_became_active":"2023-10-26T01:31:57.305411+0000","last_became_peered":"2023-10-26T01:31:57.305411+0000","last_unstale":"2023-10-26T01:31:57.909884+0000","last_undegraded":"2023-10-26T01:31:57.909884+0000","last_fullsized":"2023-10-26T01:31:57.909884+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_clean_scrub_stamp":"2023-10-26T01:31:56.263810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:04:38.277788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.920072+0000","last_change":"2023-10-26T01:31:57.925433+0000","last_active":"2023-10-26T01:31:58.920072+0000","last_peered":"2023-10-26T01:31:58.920072+0000","last_clean":"2023-10-26T01:31:58.920072+0000","last_became_active":"2023-10-26T01:31:57.923390+0000","last_became_peered":"2023-10-26T01:31:57.923390+0000","last_unstale":"2023-10-26T01:31:58.920072+0000","last_undegraded":"2023-10-26T01:31:58.920072+0000","last_fullsized":"2023-10-26T01:31:58.920072+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T01:34:44.444835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.913982+0000","last_change":"2023-10-26T01:31:57.312772+0000","last_active":"2023-10-26T01:31:58.913982+0000","last_peered":"2023-10-26T01:31:58.913982+0000","last_clean":"2023-10-26T01:31:58.913982+0000","last_became_active":"2023-10-26T01:31:57.310660+0000","last_became_peered":"2023-10-26T01:31:57.310660+0000","last_unstale":"2023-10-26T01:31:58.913982+0000","last_undegraded":"2023-10-26T01:31:58.913982+0000","last_fullsized":"2023-10-26T01:31:58.913982+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_clean_scrub_stamp":"2023-10-26T01:31:56.263810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:27:03.964632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:59.260333+0000","last_change":"2023-10-26T01:31:57.920433+0000","last_active":"2023-10-26T01:31:59.260333+0000","last_peered":"2023-10-26T01:31:59.260333+0000","last_clean":"2023-10-26T01:31:59.260333+0000","last_became_active":"2023-10-26T01:31:57.919660+0000","last_became_peered":"2023-10-26T01:31:57.919660+0000","last_unstale":"2023-10-26T01:31:59.260333+0000","last_undegraded":"2023-10-26T01:31:59.260333+0000","last_fullsized":"2023-10-26T01:31:59.260333+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:06:00.266085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-26T01:31:57.910743+0000","last_change":"2023-10-26T01:31:57.317225+0000","last_active":"2023-10-26T01:31:57.910743+0000","last_peered":"2023-10-26T01:31:57.910743+0000","last_clean":"2023-10-26T01:31:57.910743+0000","last_became_active":"2023-10-26T01:31:57.316985+0000","last_became_peered":"2023-10-26T01:31:57.316985+0000","last_unstale":"2023-10-26T01:31:57.910743+0000","last_undegraded":"2023-10-26T01:31:57.910743+0000","last_fullsized":"2023-10-26T01:31:57.910743+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_clean_scrub_stamp":"2023-10-26T01:31:56.263810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:14:38.334226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.912217+0000","last_change":"2023-10-26T01:31:57.913892+0000","last_active":"2023-10-26T01:31:58.912217+0000","last_peered":"2023-10-26T01:31:58.912217+0000","last_clean":"2023-10-26T01:31:58.912217+0000","last_became_active":"2023-10-26T01:31:57.913251+0000","last_became_peered":"2023-10-26T01:31:57.913251+0000","last_unstale":"2023-10-26T01:31:58.912217+0000","last_undegraded":"2023-10-26T01:31:58.912217+0000","last_fullsized":"2023-10-26T01:31:58.912217+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:35:25.011238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:59.259184+0000","last_change":"2023-10-26T01:31:57.300118+0000","last_active":"2023-10-26T01:31:59.259184+0000","last_peered":"2023-10-26T01:31:59.259184+0000","last_clean":"2023-10-26T01:31:59.259184+0000","last_became_active":"2023-10-26T01:31:57.299790+0000","last_became_peered":"2023-10-26T01:31:57.299790+0000","last_unstale":"2023-10-26T01:31:59.259184+0000","last_undegraded":"2023-10-26T01:31:59.259184+0000","last_fullsized":"2023-10-26T01:31:59.259184+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_clean_scrub_stamp":"2023-10-26T01:31:56.263810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:27:21.313977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.913562+0000","last_change":"2023-10-26T01:31:57.922545+0000","last_active":"2023-10-26T01:31:58.913562+0000","last_peered":"2023-10-26T01:31:58.913562+0000","last_clean":"2023-10-26T01:31:58.913562+0000","last_became_active":"2023-10-26T01:31:57.922183+0000","last_became_peered":"2023-10-26T01:31:57.922183+0000","last_unstale":"2023-10-26T01:31:58.913562+0000","last_undegraded":"2023-10-26T01:31:58.913562+0000","last_fullsized":"2023-10-26T01:31:58.913562+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T01:43:15.965456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"34'3","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:59.260933+0000","last_change":"2023-10-26T01:31:57.318179+0000","last_active":"2023-10-26T01:31:59.260933+0000","last_peered":"2023-10-26T01:31:59.260933+0000","last_clean":"2023-10-26T01:31:59.260933+0000","last_became_active":"2023-10-26T01:31:57.317971+0000","last_became_peered":"2023-10-26T01:31:57.317971+0000","last_unstale":"2023-10-26T01:31:59.260933+0000","last_undegraded":"2023-10-26T01:31:59.260933+0000","last_fullsized":"2023-10-26T01:31:59.260933+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_clean_scrub_stamp":"2023-10-26T01:31:56.263810+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:04:38.277788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:59.260339+0000","last_change":"2023-10-26T01:31:57.918887+0000","last_active":"2023-10-26T01:31:59.260339+0000","last_peered":"2023-10-26T01:31:59.260339+0000","last_clean":"2023-10-26T01:31:59.260339+0000","last_became_active":"2023-10-26T01:31:57.917309+0000","last_became_peered":"2023-10-26T01:31:57.917309+0000","last_unstale":"2023-10-26T01:31:59.260339+0000","last_undegraded":"2023-10-26T01:31:59.260339+0000","last_fullsized":"2023-10-26T01:31:59.260339+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:21:21.898287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"34'6","reported_seq":18,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.338139+0000","last_change":"2023-10-26T01:31:57.299129+0000","last_active":"2023-10-26T01:31:58.338139+0000","last_peered":"2023-10-26T01:31:58.338139+0000","last_clean":"2023-10-26T01:31:58.338139+0000","last_became_active":"2023-10-26T01:31:57.298909+0000","last_became_peered":"2023-10-26T01:31:57.298909+0000","last_unstale":"2023-10-26T01:31:58.338139+0000","last_undegraded":"2023-10-26T01:31:58.338139+0000","last_fullsized":"2023-10-26T01:31:58.338139+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_clean_scrub_stamp":"2023-10-26T01:31:56.263810+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:43:08.196868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.920106+0000","last_change":"2023-10-26T01:31:57.923375+0000","last_active":"2023-10-26T01:31:58.920106+0000","last_peered":"2023-10-26T01:31:58.920106+0000","last_clean":"2023-10-26T01:31:58.920106+0000","last_became_active":"2023-10-26T01:31:57.922926+0000","last_became_peered":"2023-10-26T01:31:57.922926+0000","last_unstale":"2023-10-26T01:31:58.920106+0000","last_undegraded":"2023-10-26T01:31:58.920106+0000","last_fullsized":"2023-10-26T01:31:58.920106+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:30:35.357843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"34'2","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.914183+0000","last_change":"2023-10-26T01:31:57.307255+0000","last_active":"2023-10-26T01:31:58.914183+0000","last_peered":"2023-10-26T01:31:58.914183+0000","last_clean":"2023-10-26T01:31:58.914183+0000","last_became_active":"2023-10-26T01:31:57.306318+0000","last_became_peered":"2023-10-26T01:31:57.306318+0000","last_unstale":"2023-10-26T01:31:58.914183+0000","last_undegraded":"2023-10-26T01:31:58.914183+0000","last_fullsized":"2023-10-26T01:31:58.914183+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_clean_scrub_stamp":"2023-10-26T01:31:56.263810+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:22:09.081480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.912334+0000","last_change":"2023-10-26T01:31:57.923545+0000","last_active":"2023-10-26T01:31:58.912334+0000","last_peered":"2023-10-26T01:31:58.912334+0000","last_clean":"2023-10-26T01:31:58.912334+0000","last_became_active":"2023-10-26T01:31:57.920460+0000","last_became_peered":"2023-10-26T01:31:57.920460+0000","last_unstale":"2023-10-26T01:31:58.912334+0000","last_undegraded":"2023-10-26T01:31:58.912334+0000","last_fullsized":"2023-10-26T01:31:58.912334+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:58:30.843955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.920731+0000","last_change":"2023-10-26T01:31:57.306563+0000","last_active":"2023-10-26T01:31:58.920731+0000","last_peered":"2023-10-26T01:31:58.920731+0000","last_clean":"2023-10-26T01:31:58.920731+0000","last_became_active":"2023-10-26T01:31:57.306305+0000","last_became_peered":"2023-10-26T01:31:57.306305+0000","last_unstale":"2023-10-26T01:31:58.920731+0000","last_undegraded":"2023-10-26T01:31:58.920731+0000","last_fullsized":"2023-10-26T01:31:58.920731+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_clean_scrub_stamp":"2023-10-26T01:31:56.263810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:17:27.685445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.915062+0000","last_change":"2023-10-26T01:31:57.930623+0000","last_active":"2023-10-26T01:31:58.915062+0000","last_peered":"2023-10-26T01:31:58.915062+0000","last_clean":"2023-10-26T01:31:58.915062+0000","last_became_active":"2023-10-26T01:31:57.929835+0000","last_became_peered":"2023-10-26T01:31:57.929835+0000","last_unstale":"2023-10-26T01:31:58.915062+0000","last_undegraded":"2023-10-26T01:31:58.915062+0000","last_fullsized":"2023-10-26T01:31:58.915062+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:56:29.283750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-26T01:31:57.908174+0000","last_change":"2023-10-26T01:31:57.317377+0000","last_active":"2023-10-26T01:31:57.908174+0000","last_peered":"2023-10-26T01:31:57.908174+0000","last_clean":"2023-10-26T01:31:57.908174+0000","last_became_active":"2023-10-26T01:31:57.317103+0000","last_became_peered":"2023-10-26T01:31:57.317103+0000","last_unstale":"2023-10-26T01:31:57.908174+0000","last_undegraded":"2023-10-26T01:31:57.908174+0000","last_fullsized":"2023-10-26T01:31:57.908174+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_clean_scrub_stamp":"2023-10-26T01:31:56.263810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:51:08.285631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.920966+0000","last_change":"2023-10-26T01:31:57.928078+0000","last_active":"2023-10-26T01:31:58.920966+0000","last_peered":"2023-10-26T01:31:58.920966+0000","last_clean":"2023-10-26T01:31:58.920966+0000","last_became_active":"2023-10-26T01:31:57.927787+0000","last_became_peered":"2023-10-26T01:31:57.927787+0000","last_unstale":"2023-10-26T01:31:58.920966+0000","last_undegraded":"2023-10-26T01:31:58.920966+0000","last_fullsized":"2023-10-26T01:31:58.920966+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:17:24.476006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"34'8","reported_seq":20,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.333955+0000","last_change":"2023-10-26T01:31:57.298215+0000","last_active":"2023-10-26T01:31:58.333955+0000","last_peered":"2023-10-26T01:31:58.333955+0000","last_clean":"2023-10-26T01:31:58.333955+0000","last_became_active":"2023-10-26T01:31:57.297999+0000","last_became_peered":"2023-10-26T01:31:57.297999+0000","last_unstale":"2023-10-26T01:31:58.333955+0000","last_undegraded":"2023-10-26T01:31:58.333955+0000","last_fullsized":"2023-10-26T01:31:58.333955+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_clean_scrub_stamp":"2023-10-26T01:31:56.263810+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:15:56.135260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.913796+0000","last_change":"2023-10-26T01:31:57.928206+0000","last_active":"2023-10-26T01:31:58.913796+0000","last_peered":"2023-10-26T01:31:58.913796+0000","last_clean":"2023-10-26T01:31:58.913796+0000","last_became_active":"2023-10-26T01:31:57.927366+0000","last_became_peered":"2023-10-26T01:31:57.927366+0000","last_unstale":"2023-10-26T01:31:58.913796+0000","last_undegraded":"2023-10-26T01:31:58.913796+0000","last_fullsized":"2023-10-26T01:31:58.913796+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:15:19.809493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"34'1","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.920577+0000","last_change":"2023-10-26T01:31:57.300699+0000","last_active":"2023-10-26T01:31:58.920577+0000","last_peered":"2023-10-26T01:31:58.920577+0000","last_clean":"2023-10-26T01:31:58.920577+0000","last_became_active":"2023-10-26T01:31:57.300230+0000","last_became_peered":"2023-10-26T01:31:57.300230+0000","last_unstale":"2023-10-26T01:31:58.920577+0000","last_undegraded":"2023-10-26T01:31:58.920577+0000","last_fullsized":"2023-10-26T01:31:58.920577+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_clean_scrub_stamp":"2023-10-26T01:31:56.263810+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:15:18.796380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:59.259805+0000","last_change":"2023-10-26T01:31:57.925303+0000","last_active":"2023-10-26T01:31:59.259805+0000","last_peered":"2023-10-26T01:31:59.259805+0000","last_clean":"2023-10-26T01:31:59.259805+0000","last_became_active":"2023-10-26T01:31:57.924425+0000","last_became_peered":"2023-10-26T01:31:57.924425+0000","last_unstale":"2023-10-26T01:31:59.259805+0000","last_undegraded":"2023-10-26T01:31:59.259805+0000","last_fullsized":"2023-10-26T01:31:59.259805+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:30:35.357843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"34'2","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.914457+0000","last_change":"2023-10-26T01:31:57.306952+0000","last_active":"2023-10-26T01:31:58.914457+0000","last_peered":"2023-10-26T01:31:58.914457+0000","last_clean":"2023-10-26T01:31:58.914457+0000","last_became_active":"2023-10-26T01:31:57.305511+0000","last_became_peered":"2023-10-26T01:31:57.305511+0000","last_unstale":"2023-10-26T01:31:58.914457+0000","last_undegraded":"2023-10-26T01:31:58.914457+0000","last_fullsized":"2023-10-26T01:31:58.914457+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_clean_scrub_stamp":"2023-10-26T01:31:56.263810+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:22:42.804254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.919760+0000","last_change":"2023-10-26T01:31:57.930755+0000","last_active":"2023-10-26T01:31:58.919760+0000","last_peered":"2023-10-26T01:31:58.919760+0000","last_clean":"2023-10-26T01:31:58.919760+0000","last_became_active":"2023-10-26T01:31:57.930037+0000","last_became_peered":"2023-10-26T01:31:57.930037+0000","last_unstale":"2023-10-26T01:31:58.919760+0000","last_undegraded":"2023-10-26T01:31:58.919760+0000","last_fullsized":"2023-10-26T01:31:58.919760+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:04:39.290900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"34'1","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.916389+0000","last_change":"2023-10-26T01:31:57.295883+0000","last_active":"2023-10-26T01:31:58.916389+0000","last_peered":"2023-10-26T01:31:58.916389+0000","last_clean":"2023-10-26T01:31:58.916389+0000","last_became_active":"2023-10-26T01:31:57.295190+0000","last_became_peered":"2023-10-26T01:31:57.295190+0000","last_unstale":"2023-10-26T01:31:58.916389+0000","last_undegraded":"2023-10-26T01:31:58.916389+0000","last_fullsized":"2023-10-26T01:31:58.916389+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_clean_scrub_stamp":"2023-10-26T01:31:56.263810+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T01:48:31.932057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:59.258457+0000","last_change":"2023-10-26T01:31:57.926385+0000","last_active":"2023-10-26T01:31:59.258457+0000","last_peered":"2023-10-26T01:31:59.258457+0000","last_clean":"2023-10-26T01:31:59.258457+0000","last_became_active":"2023-10-26T01:31:57.926127+0000","last_became_peered":"2023-10-26T01:31:57.926127+0000","last_unstale":"2023-10-26T01:31:59.258457+0000","last_undegraded":"2023-10-26T01:31:59.258457+0000","last_fullsized":"2023-10-26T01:31:59.258457+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:27:59.914223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.915095+0000","last_change":"2023-10-26T01:31:57.301740+0000","last_active":"2023-10-26T01:31:58.915095+0000","last_peered":"2023-10-26T01:31:58.915095+0000","last_clean":"2023-10-26T01:31:58.915095+0000","last_became_active":"2023-10-26T01:31:57.297718+0000","last_became_peered":"2023-10-26T01:31:57.297718+0000","last_unstale":"2023-10-26T01:31:58.915095+0000","last_undegraded":"2023-10-26T01:31:58.915095+0000","last_fullsized":"2023-10-26T01:31:58.915095+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_clean_scrub_stamp":"2023-10-26T01:31:56.263810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:44:09.907410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:59.258537+0000","last_change":"2023-10-26T01:31:57.925298+0000","last_active":"2023-10-26T01:31:59.258537+0000","last_peered":"2023-10-26T01:31:59.258537+0000","last_clean":"2023-10-26T01:31:59.258537+0000","last_became_active":"2023-10-26T01:31:57.924627+0000","last_became_peered":"2023-10-26T01:31:57.924627+0000","last_unstale":"2023-10-26T01:31:59.258537+0000","last_undegraded":"2023-10-26T01:31:59.258537+0000","last_fullsized":"2023-10-26T01:31:59.258537+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:29:41.003416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"34'2","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:59.259434+0000","last_change":"2023-10-26T01:31:57.300830+0000","last_active":"2023-10-26T01:31:59.259434+0000","last_peered":"2023-10-26T01:31:59.259434+0000","last_clean":"2023-10-26T01:31:59.259434+0000","last_became_active":"2023-10-26T01:31:57.300453+0000","last_became_peered":"2023-10-26T01:31:57.300453+0000","last_unstale":"2023-10-26T01:31:59.259434+0000","last_undegraded":"2023-10-26T01:31:59.259434+0000","last_fullsized":"2023-10-26T01:31:59.259434+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_clean_scrub_stamp":"2023-10-26T01:31:56.263810+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:15:18.796380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:59.258539+0000","last_change":"2023-10-26T01:31:57.930968+0000","last_active":"2023-10-26T01:31:59.258539+0000","last_peered":"2023-10-26T01:31:59.258539+0000","last_clean":"2023-10-26T01:31:59.258539+0000","last_became_active":"2023-10-26T01:31:57.930184+0000","last_became_peered":"2023-10-26T01:31:57.930184+0000","last_unstale":"2023-10-26T01:31:59.258539+0000","last_undegraded":"2023-10-26T01:31:59.258539+0000","last_fullsized":"2023-10-26T01:31:59.258539+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:16:42.398443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.920282+0000","last_change":"2023-10-26T01:31:57.303240+0000","last_active":"2023-10-26T01:31:58.920282+0000","last_peered":"2023-10-26T01:31:58.920282+0000","last_clean":"2023-10-26T01:31:58.920282+0000","last_became_active":"2023-10-26T01:31:57.302629+0000","last_became_peered":"2023-10-26T01:31:57.302629+0000","last_unstale":"2023-10-26T01:31:58.920282+0000","last_undegraded":"2023-10-26T01:31:58.920282+0000","last_fullsized":"2023-10-26T01:31:58.920282+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_clean_scrub_stamp":"2023-10-26T01:31:56.263810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:29:39.990303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:59.258727+0000","last_change":"2023-10-26T01:31:57.929718+0000","last_active":"2023-10-26T01:31:59.258727+0000","last_peered":"2023-10-26T01:31:59.258727+0000","last_clean":"2023-10-26T01:31:59.258727+0000","last_became_active":"2023-10-26T01:31:57.929407+0000","last_became_peered":"2023-10-26T01:31:57.929407+0000","last_unstale":"2023-10-26T01:31:59.258727+0000","last_undegraded":"2023-10-26T01:31:59.258727+0000","last_fullsized":"2023-10-26T01:31:59.258727+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:41:29.809025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.914698+0000","last_change":"2023-10-26T01:31:57.302048+0000","last_active":"2023-10-26T01:31:58.914698+0000","last_peered":"2023-10-26T01:31:58.914698+0000","last_clean":"2023-10-26T01:31:58.914698+0000","last_became_active":"2023-10-26T01:31:57.301290+0000","last_became_peered":"2023-10-26T01:31:57.301290+0000","last_unstale":"2023-10-26T01:31:58.914698+0000","last_undegraded":"2023-10-26T01:31:58.914698+0000","last_fullsized":"2023-10-26T01:31:58.914698+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_clean_scrub_stamp":"2023-10-26T01:31:56.263810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:46:19.545323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.917219+0000","last_change":"2023-10-26T01:31:57.922247+0000","last_active":"2023-10-26T01:31:58.917219+0000","last_peered":"2023-10-26T01:31:58.917219+0000","last_clean":"2023-10-26T01:31:58.917219+0000","last_became_active":"2023-10-26T01:31:57.921931+0000","last_became_peered":"2023-10-26T01:31:57.921931+0000","last_unstale":"2023-10-26T01:31:58.917219+0000","last_undegraded":"2023-10-26T01:31:58.917219+0000","last_fullsized":"2023-10-26T01:31:58.917219+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:06:00.266085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:59.260138+0000","last_change":"2023-10-26T01:31:57.316497+0000","last_active":"2023-10-26T01:31:59.260138+0000","last_peered":"2023-10-26T01:31:59.260138+0000","last_clean":"2023-10-26T01:31:59.260138+0000","last_became_active":"2023-10-26T01:31:57.316154+0000","last_became_peered":"2023-10-26T01:31:57.316154+0000","last_unstale":"2023-10-26T01:31:59.260138+0000","last_undegraded":"2023-10-26T01:31:59.260138+0000","last_fullsized":"2023-10-26T01:31:59.260138+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_clean_scrub_stamp":"2023-10-26T01:31:56.263810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:49:04.462479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:59.258706+0000","last_change":"2023-10-26T01:31:57.925339+0000","last_active":"2023-10-26T01:31:59.258706+0000","last_peered":"2023-10-26T01:31:59.258706+0000","last_clean":"2023-10-26T01:31:59.258706+0000","last_became_active":"2023-10-26T01:31:57.924962+0000","last_became_peered":"2023-10-26T01:31:57.924962+0000","last_unstale":"2023-10-26T01:31:59.258706+0000","last_undegraded":"2023-10-26T01:31:59.258706+0000","last_fullsized":"2023-10-26T01:31:59.258706+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:27:22.327089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"34'2","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.919855+0000","last_change":"2023-10-26T01:31:57.307808+0000","last_active":"2023-10-26T01:31:58.919855+0000","last_peered":"2023-10-26T01:31:58.919855+0000","last_clean":"2023-10-26T01:31:58.919855+0000","last_became_active":"2023-10-26T01:31:57.307569+0000","last_became_peered":"2023-10-26T01:31:57.307569+0000","last_unstale":"2023-10-26T01:31:58.919855+0000","last_undegraded":"2023-10-26T01:31:58.919855+0000","last_fullsized":"2023-10-26T01:31:58.919855+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_clean_scrub_stamp":"2023-10-26T01:31:56.263810+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:20:35.297390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:59.260538+0000","last_change":"2023-10-26T01:31:57.920613+0000","last_active":"2023-10-26T01:31:59.260538+0000","last_peered":"2023-10-26T01:31:59.260538+0000","last_clean":"2023-10-26T01:31:59.260538+0000","last_became_active":"2023-10-26T01:31:57.920128+0000","last_became_peered":"2023-10-26T01:31:57.920128+0000","last_unstale":"2023-10-26T01:31:59.260538+0000","last_undegraded":"2023-10-26T01:31:59.260538+0000","last_fullsized":"2023-10-26T01:31:59.260538+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:02:28.206674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:59.259511+0000","last_change":"2023-10-26T01:31:57.300686+0000","last_active":"2023-10-26T01:31:59.259511+0000","last_peered":"2023-10-26T01:31:59.259511+0000","last_clean":"2023-10-26T01:31:59.259511+0000","last_became_active":"2023-10-26T01:31:57.300125+0000","last_became_peered":"2023-10-26T01:31:57.300125+0000","last_unstale":"2023-10-26T01:31:59.259511+0000","last_undegraded":"2023-10-26T01:31:59.259511+0000","last_fullsized":"2023-10-26T01:31:59.259511+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_clean_scrub_stamp":"2023-10-26T01:31:56.263810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:17:23.462894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:59.258765+0000","last_change":"2023-10-26T01:31:57.920668+0000","last_active":"2023-10-26T01:31:59.258765+0000","last_peered":"2023-10-26T01:31:59.258765+0000","last_clean":"2023-10-26T01:31:59.258765+0000","last_became_active":"2023-10-26T01:31:57.920145+0000","last_became_peered":"2023-10-26T01:31:57.920145+0000","last_unstale":"2023-10-26T01:31:59.258765+0000","last_undegraded":"2023-10-26T01:31:59.258765+0000","last_fullsized":"2023-10-26T01:31:59.258765+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:31:13.736811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.920877+0000","last_change":"2023-10-26T01:31:57.307049+0000","last_active":"2023-10-26T01:31:58.920877+0000","last_peered":"2023-10-26T01:31:58.920877+0000","last_clean":"2023-10-26T01:31:58.920877+0000","last_became_active":"2023-10-26T01:31:57.305724+0000","last_became_peered":"2023-10-26T01:31:57.305724+0000","last_unstale":"2023-10-26T01:31:58.920877+0000","last_undegraded":"2023-10-26T01:31:58.920877+0000","last_fullsized":"2023-10-26T01:31:58.920877+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_clean_scrub_stamp":"2023-10-26T01:31:56.263810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:52:32.888332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.921283+0000","last_change":"2023-10-26T01:31:57.919929+0000","last_active":"2023-10-26T01:31:58.921283+0000","last_peered":"2023-10-26T01:31:58.921283+0000","last_clean":"2023-10-26T01:31:58.921283+0000","last_became_active":"2023-10-26T01:31:57.919241+0000","last_became_peered":"2023-10-26T01:31:57.919241+0000","last_unstale":"2023-10-26T01:31:58.921283+0000","last_undegraded":"2023-10-26T01:31:58.921283+0000","last_fullsized":"2023-10-26T01:31:58.921283+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:51:09.298744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:59.259167+0000","last_change":"2023-10-26T01:31:57.313148+0000","last_active":"2023-10-26T01:31:59.259167+0000","last_peered":"2023-10-26T01:31:59.259167+0000","last_clean":"2023-10-26T01:31:59.259167+0000","last_became_active":"2023-10-26T01:31:57.312773+0000","last_became_peered":"2023-10-26T01:31:57.312773+0000","last_unstale":"2023-10-26T01:31:59.259167+0000","last_undegraded":"2023-10-26T01:31:59.259167+0000","last_fullsized":"2023-10-26T01:31:59.259167+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_clean_scrub_stamp":"2023-10-26T01:31:56.263810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:15:56.135260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:59.259003+0000","last_change":"2023-10-26T01:31:57.919722+0000","last_active":"2023-10-26T01:31:59.259003+0000","last_peered":"2023-10-26T01:31:59.259003+0000","last_clean":"2023-10-26T01:31:59.259003+0000","last_became_active":"2023-10-26T01:31:57.919048+0000","last_became_peered":"2023-10-26T01:31:57.919048+0000","last_unstale":"2023-10-26T01:31:59.259003+0000","last_undegraded":"2023-10-26T01:31:59.259003+0000","last_fullsized":"2023-10-26T01:31:59.259003+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:51:09.298744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.920076+0000","last_change":"2023-10-26T01:31:57.313119+0000","last_active":"2023-10-26T01:31:58.920076+0000","last_peered":"2023-10-26T01:31:58.920076+0000","last_clean":"2023-10-26T01:31:58.920076+0000","last_became_active":"2023-10-26T01:31:57.312885+0000","last_became_peered":"2023-10-26T01:31:57.312885+0000","last_unstale":"2023-10-26T01:31:58.920076+0000","last_undegraded":"2023-10-26T01:31:58.920076+0000","last_fullsized":"2023-10-26T01:31:58.920076+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_clean_scrub_stamp":"2023-10-26T01:31:56.263810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:42:37.741558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.912282+0000","last_change":"2023-10-26T01:31:57.928889+0000","last_active":"2023-10-26T01:31:58.912282+0000","last_peered":"2023-10-26T01:31:58.912282+0000","last_clean":"2023-10-26T01:31:58.912282+0000","last_became_active":"2023-10-26T01:31:57.926403+0000","last_became_peered":"2023-10-26T01:31:57.926403+0000","last_unstale":"2023-10-26T01:31:58.912282+0000","last_undegraded":"2023-10-26T01:31:58.912282+0000","last_fullsized":"2023-10-26T01:31:58.912282+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:03:16.325284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.913878+0000","last_change":"2023-10-26T01:31:57.312149+0000","last_active":"2023-10-26T01:31:58.913878+0000","last_peered":"2023-10-26T01:31:58.913878+0000","last_clean":"2023-10-26T01:31:58.913878+0000","last_became_active":"2023-10-26T01:31:57.311845+0000","last_became_peered":"2023-10-26T01:31:57.311845+0000","last_unstale":"2023-10-26T01:31:58.913878+0000","last_undegraded":"2023-10-26T01:31:58.913878+0000","last_fullsized":"2023-10-26T01:31:58.913878+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_clean_scrub_stamp":"2023-10-26T01:31:56.263810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:20:35.297390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.913223+0000","last_change":"2023-10-26T01:31:57.931585+0000","last_active":"2023-10-26T01:31:58.913223+0000","last_peered":"2023-10-26T01:31:58.913223+0000","last_clean":"2023-10-26T01:31:58.913223+0000","last_became_active":"2023-10-26T01:31:57.931235+0000","last_became_peered":"2023-10-26T01:31:57.931235+0000","last_unstale":"2023-10-26T01:31:58.913223+0000","last_undegraded":"2023-10-26T01:31:58.913223+0000","last_fullsized":"2023-10-26T01:31:58.913223+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:15:57.148372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-26T01:31:57.913903+0000","last_change":"2023-10-26T01:31:57.316899+0000","last_active":"2023-10-26T01:31:57.913903+0000","last_peered":"2023-10-26T01:31:57.913903+0000","last_clean":"2023-10-26T01:31:57.913903+0000","last_became_active":"2023-10-26T01:31:57.316690+0000","last_became_peered":"2023-10-26T01:31:57.316690+0000","last_unstale":"2023-10-26T01:31:57.913903+0000","last_undegraded":"2023-10-26T01:31:57.913903+0000","last_fullsized":"2023-10-26T01:31:57.913903+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_clean_scrub_stamp":"2023-10-26T01:31:56.263810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:51:35.971480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.917402+0000","last_change":"2023-10-26T01:31:57.927113+0000","last_active":"2023-10-26T01:31:58.917402+0000","last_peered":"2023-10-26T01:31:58.917402+0000","last_clean":"2023-10-26T01:31:58.917402+0000","last_became_active":"2023-10-26T01:31:57.926727+0000","last_became_peered":"2023-10-26T01:31:57.926727+0000","last_unstale":"2023-10-26T01:31:58.917402+0000","last_undegraded":"2023-10-26T01:31:58.917402+0000","last_fullsized":"2023-10-26T01:31:58.917402+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:02:28.206674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.920553+0000","last_change":"2023-10-26T01:31:57.301952+0000","last_active":"2023-10-26T01:31:58.920553+0000","last_peered":"2023-10-26T01:31:58.920553+0000","last_clean":"2023-10-26T01:31:58.920553+0000","last_became_active":"2023-10-26T01:31:57.300614+0000","last_became_peered":"2023-10-26T01:31:57.300614+0000","last_unstale":"2023-10-26T01:31:58.920553+0000","last_undegraded":"2023-10-26T01:31:58.920553+0000","last_fullsized":"2023-10-26T01:31:58.920553+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_clean_scrub_stamp":"2023-10-26T01:31:56.263810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:16:41.385331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":178,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-26T01:31:57.912740+0000","last_change":"2023-10-26T01:30:34.883484+0000","last_active":"2023-10-26T01:31:57.912740+0000","last_peered":"2023-10-26T01:31:57.912740+0000","last_clean":"2023-10-26T01:31:57.912740+0000","last_became_active":"2023-10-26T01:30:34.883071+0000","last_became_peered":"2023-10-26T01:30:34.883071+0000","last_unstale":"2023-10-26T01:31:57.912740+0000","last_undegraded":"2023-10-26T01:31:57.912740+0000","last_fullsized":"2023-10-26T01:31:57.912740+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:32.890096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:32.890096+0000","last_clean_scrub_stamp":"2023-10-26T01:30:32.890096+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:22:57.937891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":14,"seq":60129542165,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":154,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195158,"statfs":{"total":200000000,"available":199841884,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:31:32 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":2.2549999999999999,"5min":2.2549999999999999,"15min":2.2549999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.8200000000000001,"5min":1.8200000000000001,"15min":1.8200000000000001},"last":0.56699999999999995}]},{"osd":1,"last update":"Thu Oct 26 01:31:32 2023","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":0.998,"15min":0.998},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":4.4340000000000002,"5min":4.4340000000000002,"15min":4.4340000000000002},"last":1.329},{"interface":"front","average":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":2.3330000000000002,"5min":2.3330000000000002,"15min":2.3330000000000002},"last":1.288}]},{"osd":2,"last update":"Thu Oct 26 01:31:32 2023","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":4.2800000000000002,"5min":4.2800000000000002,"15min":4.2800000000000002},"last":1.02},{"interface":"front","average":{"1min":1.077,"5min":1.077,"15min":1.077},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":2.302,"5min":2.302,"15min":2.302},"last":1.373}]},{"osd":4,"last update":"Thu Oct 26 01:31:32 2023","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":2.1070000000000002,"5min":2.1070000000000002,"15min":2.1070000000000002},"last":1.0960000000000001},{"interface":"front","average":{"1min":1.4339999999999999,"5min":1.4339999999999999,"15min":1.4339999999999999},"min":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"max":{"1min":4.8250000000000002,"5min":4.8250000000000002,"15min":4.8250000000000002},"last":1.4950000000000001}]},{"osd":5,"last update":"Thu Oct 26 01:31:32 2023","interfaces":[{"interface":"back","average":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":4.9080000000000004,"5min":4.9080000000000004,"15min":4.9080000000000004},"last":1.5820000000000001},{"interface":"front","average":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"min":{"1min":0.88,"5min":0.88,"15min":0.88},"max":{"1min":2.8980000000000001,"5min":2.8980000000000001,"15min":2.8980000000000001},"last":1.177}]},{"osd":6,"last update":"Thu Oct 26 01:31:32 2023","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.321,"15min":1.321},"min":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"max":{"1min":2.8530000000000002,"5min":2.8530000000000002,"15min":2.8530000000000002},"last":1.464},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":2.181,"5min":2.181,"15min":2.181},"last":1.159}]},{"osd":7,"last update":"Thu Oct 26 01:31:32 2023","interfaces":[{"interface":"back","average":{"1min":1.286,"5min":1.286,"15min":1.286},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":2.0680000000000001,"5min":2.0680000000000001,"15min":2.0680000000000001},"last":1.2090000000000001},{"interface":"front","average":{"1min":1.45,"5min":1.45,"15min":1.45},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":4.8730000000000002,"5min":4.8730000000000002,"15min":4.8730000000000002},"last":1.6419999999999999}]}]},{"osd":7,"up_from":13,"seq":55834574868,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":729,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194583,"statfs":{"total":200000000,"available":199253060,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:31:31 2023","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.7490000000000001,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":1.246},{"interface":"front","average":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.514,"5min":1.514,"15min":1.514},"last":1.4330000000000001}]},{"osd":1,"last update":"Thu Oct 26 01:31:31 2023","interfaces":[{"interface":"back","average":{"1min":1.034,"5min":1.034,"15min":1.034},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.8520000000000001,"5min":1.8520000000000001,"15min":1.8520000000000001},"last":1.0129999999999999},{"interface":"front","average":{"1min":1.175,"5min":1.175,"15min":1.175},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.7689999999999999,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":1.123}]},{"osd":2,"last update":"Thu Oct 26 01:31:31 2023","interfaces":[{"interface":"back","average":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.5649999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.94899999999999995},{"interface":"front","average":{"1min":1.206,"5min":1.206,"15min":1.206},"min":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":1.766,"5min":1.766,"15min":1.766},"last":1.587}]},{"osd":3,"last update":"Thu Oct 26 01:31:36 2023","interfaces":[{"interface":"back","average":{"1min":1.177,"5min":1.177,"15min":1.177},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":1.169},{"interface":"front","average":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"min":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"max":{"1min":1.6180000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":1.534}]},{"osd":4,"last update":"Thu Oct 26 01:31:31 2023","interfaces":[{"interface":"back","average":{"1min":1.345,"5min":1.345,"15min":1.345},"min":{"1min":1.014,"5min":1.014,"15min":1.014},"max":{"1min":1.671,"5min":1.671,"15min":1.671},"last":1.0680000000000001},{"interface":"front","average":{"1min":1.379,"5min":1.379,"15min":1.379},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":2.105,"5min":2.105,"15min":2.105},"last":1.472}]},{"osd":5,"last update":"Thu Oct 26 01:31:31 2023","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.393,"15min":1.393},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.873,"5min":1.873,"15min":1.873},"last":1.288},{"interface":"front","average":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"min":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"max":{"1min":1.7090000000000001,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":1.3360000000000001}]},{"osd":6,"last update":"Thu Oct 26 01:31:31 2023","interfaces":[{"interface":"back","average":{"1min":2.0139999999999998,"5min":2.0139999999999998,"15min":2.0139999999999998},"min":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":11.577999999999999,"5min":11.577999999999999,"15min":11.577999999999999},"last":1.3640000000000001},{"interface":"front","average":{"1min":2.0019999999999998,"5min":2.0019999999999998,"15min":2.0019999999999998},"min":{"1min":1.159,"5min":1.159,"15min":1.159},"max":{"1min":10.704000000000001,"5min":10.704000000000001,"15min":10.704000000000001},"last":1.403}]}]},{"osd":6,"up_from":13,"seq":55834574868,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":152,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195159,"statfs":{"total":200000000,"available":199843516,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:31:32 2023","interfaces":[{"interface":"back","average":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"min":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"max":{"1min":1.8839999999999999,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.843,"5min":1.843,"15min":1.843},"last":1.0329999999999999}]},{"osd":1,"last update":"Thu Oct 26 01:31:32 2023","interfaces":[{"interface":"back","average":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.605,"5min":1.605,"15min":1.605},"last":1.1020000000000001},{"interface":"front","average":{"1min":1.226,"5min":1.226,"15min":1.226},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":2.085,"5min":2.085,"15min":2.085},"last":1.5209999999999999}]},{"osd":2,"last update":"Thu Oct 26 01:31:32 2023","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.052,"15min":1.052},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":1.6659999999999999,"5min":1.6659999999999999,"15min":1.6659999999999999},"last":0.78800000000000003},{"interface":"front","average":{"1min":1.248,"5min":1.248,"15min":1.248},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.7949999999999999,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":0.98999999999999999}]},{"osd":3,"last update":"Thu Oct 26 01:31:36 2023","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.157,"15min":1.157},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.4019999999999999,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":1.3300000000000001},{"interface":"front","average":{"1min":1.321,"5min":1.321,"15min":1.321},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.71,"5min":1.71,"15min":1.71},"last":1.21}]},{"osd":4,"last update":"Thu Oct 26 01:31:32 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.204,"15min":1.204},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":2.089,"5min":2.089,"15min":2.089},"last":1.153},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":2.4129999999999998,"5min":2.4129999999999998,"15min":2.4129999999999998},"last":1.4590000000000001}]},{"osd":5,"last update":"Thu Oct 26 01:31:32 2023","interfaces":[{"interface":"back","average":{"1min":1.391,"5min":1.391,"15min":1.391},"min":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":2.1640000000000001,"5min":2.1640000000000001,"15min":2.1640000000000001},"last":1.925},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"min":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"max":{"1min":1.7490000000000001,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":1.778}]},{"osd":7,"last update":"Thu Oct 26 01:31:32 2023","interfaces":[{"interface":"back","average":{"1min":1.355,"5min":1.355,"15min":1.355},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":2.012,"5min":2.012,"15min":2.012},"last":1.256},{"interface":"front","average":{"1min":1.4299999999999999,"5min":1.4299999999999999,"15min":1.4299999999999999},"min":{"1min":0.875,"5min":0.875,"15min":0.875},"max":{"1min":2.3239999999999998,"5min":2.3239999999999998,"15min":2.3239999999999998},"last":1.5680000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574868,"num_pgs":17,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":153,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195159,"statfs":{"total":200000000,"available":199842846,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:31:32 2023","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.7230000000000001,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":0.66200000000000003},{"interface":"front","average":{"1min":1.131,"5min":1.131,"15min":1.131},"min":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":1.52,"5min":1.52,"15min":1.52},"last":0.76600000000000001}]},{"osd":1,"last update":"Thu Oct 26 01:31:32 2023","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.722,"5min":1.722,"15min":1.722},"last":0.83899999999999997},{"interface":"front","average":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.7669999999999999,"5min":1.7669999999999999,"15min":1.7669999999999999},"last":1.1200000000000001}]},{"osd":2,"last update":"Thu Oct 26 01:31:32 2023","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.141,"15min":1.141},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.629,"5min":1.629,"15min":1.629},"last":1.2629999999999999},{"interface":"front","average":{"1min":1.161,"5min":1.161,"15min":1.161},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.5489999999999999,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":1.0720000000000001}]},{"osd":3,"last update":"Thu Oct 26 01:31:32 2023","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"min":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":1.835,"5min":1.835,"15min":1.835},"last":1.165},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":2.9249999999999998,"5min":2.9249999999999998,"15min":2.9249999999999998},"last":1.024}]},{"osd":4,"last update":"Thu Oct 26 01:31:32 2023","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.597,"5min":1.597,"15min":1.597},"last":1.474},{"interface":"front","average":{"1min":1.306,"5min":1.306,"15min":1.306},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.669,"5min":1.669,"15min":1.669},"last":1.448}]},{"osd":6,"last update":"Thu Oct 26 01:31:32 2023","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.262,"15min":1.262},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.72,"5min":1.72,"15min":1.72},"last":1.2430000000000001},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.8560000000000001,"5min":1.8560000000000001,"15min":1.8560000000000001},"last":1.319}]},{"osd":7,"last update":"Thu Oct 26 01:31:32 2023","interfaces":[{"interface":"back","average":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.7689999999999999,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":1.516},{"interface":"front","average":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.8140000000000001,"5min":1.8140000000000001,"15min":1.8140000000000001},"last":1.421}]}]},{"osd":4,"up_from":13,"seq":55834574868,"num_pgs":29,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":152,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195159,"statfs":{"total":200000000,"available":199843414,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.47,"5min":1.47,"15min":1.47},"last":1.46},{"interface":"front","average":{"1min":1.002,"5min":1.002,"15min":1.002},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.4870000000000001,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":0.621}]},{"osd":1,"last update":"Thu Oct 26 01:31:30 2023","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"min":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"max":{"1min":1.635,"5min":1.635,"15min":1.635},"last":1.163},{"interface":"front","average":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.724,"5min":1.724,"15min":1.724},"last":1.786}]},{"osd":2,"last update":"Thu Oct 26 01:31:30 2023","interfaces":[{"interface":"back","average":{"1min":1.169,"5min":1.169,"15min":1.169},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.661,"5min":1.661,"15min":1.661},"last":1.268},{"interface":"front","average":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.53,"5min":1.53,"15min":1.53},"last":0.80700000000000005}]},{"osd":3,"last update":"Thu Oct 26 01:31:32 2023","interfaces":[{"interface":"back","average":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.845,"5min":1.845,"15min":1.845},"last":1.8480000000000001},{"interface":"front","average":{"1min":1.131,"5min":1.131,"15min":1.131},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":1.5049999999999999,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":1.0309999999999999}]},{"osd":5,"last update":"Thu Oct 26 01:31:30 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.304,"15min":1.304},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.9570000000000001,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":1.907},{"interface":"front","average":{"1min":1.325,"5min":1.325,"15min":1.325},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.8120000000000001,"5min":1.8120000000000001,"15min":1.8120000000000001},"last":1.3759999999999999}]},{"osd":6,"last update":"Thu Oct 26 01:31:30 2023","interfaces":[{"interface":"back","average":{"1min":1.2909999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":2.218,"5min":2.218,"15min":2.218},"last":1.212},{"interface":"front","average":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":3.3500000000000001,"5min":3.3500000000000001,"15min":3.3500000000000001},"last":2.1320000000000001}]},{"osd":7,"last update":"Thu Oct 26 01:31:30 2023","interfaces":[{"interface":"back","average":{"1min":1.478,"5min":1.478,"15min":1.478},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":2.2730000000000001,"5min":2.2730000000000001,"15min":2.2730000000000001},"last":1.5089999999999999},{"interface":"front","average":{"1min":1.421,"5min":1.421,"15min":1.421},"min":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":2.1850000000000001,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":1.3080000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574868,"num_pgs":17,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":152,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195159,"statfs":{"total":200000000,"available":199843448,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.98999999999999999},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.5900000000000001,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":1.073}]},{"osd":1,"last update":"Thu Oct 26 01:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":1.171}]},{"osd":3,"last update":"Thu Oct 26 01:31:33 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.623,"5min":1.623,"15min":1.623},"last":1.381},{"interface":"front","average":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":1.0309999999999999}]},{"osd":4,"last update":"Thu Oct 26 01:31:31 2023","interfaces":[{"interface":"back","average":{"1min":1.22,"5min":1.22,"15min":1.22},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":2.2040000000000002,"5min":2.2040000000000002,"15min":2.2040000000000002},"last":1.546},{"interface":"front","average":{"1min":1.212,"5min":1.212,"15min":1.212},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":1.702,"5min":1.702,"15min":1.702},"last":1.502}]},{"osd":5,"last update":"Thu Oct 26 01:31:31 2023","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.252,"15min":1.252},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.7869999999999999,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":1.222},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":2.3029999999999999,"5min":2.3029999999999999,"15min":2.3029999999999999},"last":1.585}]},{"osd":6,"last update":"Thu Oct 26 01:31:31 2023","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.29,"15min":1.29},"min":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":2.9649999999999999,"5min":2.9649999999999999,"15min":2.9649999999999999},"last":1.6779999999999999},{"interface":"front","average":{"1min":1.343,"5min":1.343,"15min":1.343},"min":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"max":{"1min":1.7470000000000001,"5min":1.7470000000000001,"15min":1.7470000000000001},"last":1.6379999999999999}]},{"osd":7,"last update":"Thu Oct 26 01:31:31 2023","interfaces":[{"interface":"back","average":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":2.2450000000000001,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":1.4319999999999999},{"interface":"front","average":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"min":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"max":{"1min":2.7949999999999999,"5min":2.7949999999999999,"15min":2.7949999999999999},"last":1.74}]}]},{"osd":1,"up_from":13,"seq":55834574868,"num_pgs":32,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":154,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195158,"statfs":{"total":200000000,"available":199841884,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:31:32 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.60699999999999998}]},{"osd":2,"last update":"Thu Oct 26 01:31:32 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":1.018,"15min":1.018},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":2.4830000000000001,"5min":2.4830000000000001,"15min":2.4830000000000001},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.84599999999999997}]},{"osd":3,"last update":"Thu Oct 26 01:31:35 2023","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.6060000000000001,"5min":1.6060000000000001,"15min":1.6060000000000001},"last":0.754},{"interface":"front","average":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.6559999999999999,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":0.58099999999999996}]},{"osd":4,"last update":"Thu Oct 26 01:31:32 2023","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":2.0659999999999998,"5min":2.0659999999999998,"15min":2.0659999999999998},"last":0.98799999999999999},{"interface":"front","average":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.47,"5min":1.47,"15min":1.47},"last":0.73599999999999999}]},{"osd":5,"last update":"Thu Oct 26 01:31:32 2023","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.218,"15min":1.218},"min":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":1.6619999999999999,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":1.1220000000000001},{"interface":"front","average":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.3280000000000001},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":2.2200000000000002,"5min":2.2200000000000002,"15min":2.2200000000000002},"last":1.1459999999999999}]},{"osd":6,"last update":"Thu Oct 26 01:31:32 2023","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":2.3780000000000001,"5min":2.3780000000000001,"15min":2.3780000000000001},"last":0.82099999999999995},{"interface":"front","average":{"1min":1.226,"5min":1.226,"15min":1.226},"min":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"max":{"1min":1.984,"5min":1.984,"15min":1.984},"last":0.89600000000000002}]},{"osd":7,"last update":"Thu Oct 26 01:31:32 2023","interfaces":[{"interface":"back","average":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":2.6930000000000001,"5min":2.6930000000000001,"15min":2.6930000000000001},"last":1.169},{"interface":"front","average":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.7909999999999999,"5min":1.7909999999999999,"15min":1.7909999999999999},"last":0.91900000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574868,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":729,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194583,"statfs":{"total":200000000,"available":199253150,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Oct 26 01:31:33 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":4.5679999999999996,"5min":4.5679999999999996,"15min":4.5679999999999996},"last":0.74299999999999999},{"interface":"front","average":{"1min":1.046,"5min":1.046,"15min":1.046},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":4.8470000000000004,"5min":4.8470000000000004,"15min":4.8470000000000004},"last":0.53700000000000003}]},{"osd":2,"last update":"Thu Oct 26 01:31:33 2023","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":5.5599999999999996,"5min":5.5599999999999996,"15min":5.5599999999999996},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":4.9459999999999997,"5min":4.9459999999999997,"15min":4.9459999999999997},"last":0.82999999999999996}]},{"osd":3,"last update":"Thu Oct 26 01:31:33 2023","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":4.9800000000000004,"5min":4.9800000000000004,"15min":4.9800000000000004},"last":0.60599999999999998},{"interface":"front","average":{"1min":1.175,"5min":1.175,"15min":1.175},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":5.016,"5min":5.016,"15min":5.016},"last":0.70299999999999996}]},{"osd":4,"last update":"Thu Oct 26 01:31:33 2023","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":5.1879999999999997,"5min":5.1879999999999997,"15min":5.1879999999999997},"last":1.032},{"interface":"front","average":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":5.5229999999999997,"5min":5.5229999999999997,"15min":5.5229999999999997},"last":0.79100000000000004}]},{"osd":5,"last update":"Thu Oct 26 01:31:33 2023","interfaces":[{"interface":"back","average":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":5.6360000000000001,"5min":5.6360000000000001,"15min":5.6360000000000001},"last":1.1950000000000001},{"interface":"front","average":{"1min":1.3720000000000001,"5min":1.3720000000000001,"15min":1.3720000000000001},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":5.3399999999999999,"5min":5.3399999999999999,"15min":5.3399999999999999},"last":0.98599999999999999}]},{"osd":6,"last update":"Thu Oct 26 01:31:33 2023","interfaces":[{"interface":"back","average":{"1min":1.371,"5min":1.371,"15min":1.371},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":5.3070000000000004,"5min":5.3070000000000004,"15min":5.3070000000000004},"last":1.222},{"interface":"front","average":{"1min":1.397,"5min":1.397,"15min":1.397},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":5.5919999999999996,"5min":5.5919999999999996,"15min":5.5919999999999996},"last":1.075}]},{"osd":7,"last update":"Thu Oct 26 01:31:33 2023","interfaces":[{"interface":"back","average":{"1min":1.51,"5min":1.51,"15min":1.51},"min":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":5.6689999999999996,"5min":5.6689999999999996,"15min":5.6689999999999996},"last":1.1060000000000001},{"interface":"front","average":{"1min":1.46,"5min":1.46,"15min":1.46},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":5.3719999999999999,"5min":5.3719999999999999,"15min":5.3719999999999999},"last":0.94299999999999995}]}]}],"pool_statfs":[]}} 2023-10-26T01:32:02.807 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-26T01:32:03.069 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.067+0000 7f33d1122640 1 -- 172.21.15.169:0/1575395322 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33cc06b8b0 msgr2=0x7f33cc06bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:03.069 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.067+0000 7f33d1122640 1 --2- 172.21.15.169:0/1575395322 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33cc06b8b0 0x7f33cc06bce0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f33cc0ca6b0 tx=0x7f33c402bd70 comp rx=0 tx=0).stop 2023-10-26T01:32:03.070 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.067+0000 7f33d1122640 1 -- 172.21.15.169:0/1575395322 shutdown_connections 2023-10-26T01:32:03.070 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.067+0000 7f33d1122640 1 --2- 172.21.15.169:0/1575395322 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33cc06b8b0 0x7f33cc06bce0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:03.070 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.067+0000 7f33d1122640 1 --2- 172.21.15.169:0/1575395322 >> v2:172.21.15.170:3300/0 conn(0x7f33cc158070 0x7f33cc09a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:03.070 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.067+0000 7f33d1122640 1 -- 172.21.15.169:0/1575395322 >> 172.21.15.169:0/1575395322 conn(0x7f33cc0aedd0 msgr2=0x7f33cc0af1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:03.070 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.067+0000 7f33d1122640 1 -- 172.21.15.169:0/1575395322 shutdown_connections 2023-10-26T01:32:03.071 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.067+0000 7f33d1122640 1 -- 172.21.15.169:0/1575395322 wait complete. 2023-10-26T01:32:03.072 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.069+0000 7f33d1122640 1 Processor -- start 2023-10-26T01:32:03.073 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.069+0000 7f33d1122640 1 -- start start 2023-10-26T01:32:03.073 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.070+0000 7f33d1122640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33cc06b8b0 0x7f33cc0afce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:03.073 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.070+0000 7f33d1122640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f33cc09ab50 0x7f33cc1f3d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:03.073 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.070+0000 7f33d1122640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33cc158070 0x7f33cc147990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:03.073 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.070+0000 7f33d1122640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33cc1f4260 con 0x7f33cc158070 2023-10-26T01:32:03.074 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.070+0000 7f33d1122640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33cc06ebe0 con 0x7f33cc06b8b0 2023-10-26T01:32:03.074 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.070+0000 7f33d1122640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33cc06ee30 con 0x7f33cc09ab50 2023-10-26T01:32:03.074 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.070+0000 7f33d0921640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33cc158070 0x7f33cc147990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:03.074 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.070+0000 7f33d0921640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33cc158070 0x7f33cc147990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:44476/0 (socket says 172.21.15.169:44476) 2023-10-26T01:32:03.074 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.070+0000 7f33d0921640 1 -- 172.21.15.169:0/2905862086 learned_addr learned my addr 172.21.15.169:0/2905862086 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:32:03.075 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.071+0000 7f33cbfff640 1 --2- 172.21.15.169:0/2905862086 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33cc06b8b0 0x7f33cc0afce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:03.075 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.071+0000 7f33cb7fe640 1 --2- 172.21.15.169:0/2905862086 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f33cc09ab50 0x7f33cc1f3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:03.075 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.071+0000 7f33d0921640 1 -- 172.21.15.169:0/2905862086 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f33cc09ab50 msgr2=0x7f33cc1f3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:03.075 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.071+0000 7f33d0921640 1 --2- 172.21.15.169:0/2905862086 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f33cc09ab50 0x7f33cc1f3d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:03.076 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.071+0000 7f33d0921640 1 -- 172.21.15.169:0/2905862086 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33cc06b8b0 msgr2=0x7f33cc0afce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:03.076 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.071+0000 7f33d0921640 1 --2- 172.21.15.169:0/2905862086 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33cc06b8b0 0x7f33cc0afce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:03.076 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.071+0000 7f33d0921640 1 -- 172.21.15.169:0/2905862086 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33c4009490 con 0x7f33cc158070 2023-10-26T01:32:03.076 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.071+0000 7f33cbfff640 1 --2- 172.21.15.169:0/2905862086 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33cc06b8b0 0x7f33cc0afce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:32:03.076 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.071+0000 7f33d0921640 1 --2- 172.21.15.169:0/2905862086 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33cc158070 0x7f33cc147990 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f33c4009920 tx=0x7f33c4003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:03.078 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.071+0000 7f33c8ff9640 1 -- 172.21.15.169:0/2905862086 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33c4003e20 con 0x7f33cc158070 2023-10-26T01:32:03.078 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.071+0000 7f33c8ff9640 1 -- 172.21.15.169:0/2905862086 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f33c4034440 con 0x7f33cc158070 2023-10-26T01:32:03.078 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.071+0000 7f33c8ff9640 1 -- 172.21.15.169:0/2905862086 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33c4034830 con 0x7f33cc158070 2023-10-26T01:32:03.078 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.071+0000 7f33d1122640 1 -- 172.21.15.169:0/2905862086 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33cc06f110 con 0x7f33cc158070 2023-10-26T01:32:03.078 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.072+0000 7f33d1122640 1 -- 172.21.15.169:0/2905862086 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33cc147ed0 con 0x7f33cc158070 2023-10-26T01:32:03.079 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.073+0000 7f33d1122640 1 -- 172.21.15.169:0/2905862086 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33cc0b1cd0 con 0x7f33cc158070 2023-10-26T01:32:03.079 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.074+0000 7f33c8ff9640 1 -- 172.21.15.169:0/2905862086 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f33c403b040 con 0x7f33cc158070 2023-10-26T01:32:03.079 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.074+0000 7f33c8ff9640 1 --2- 172.21.15.169:0/2905862086 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f33ac06ae70 0x7f33ac06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:03.079 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.075+0000 7f33c8ff9640 1 -- 172.21.15.169:0/2905862086 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f33c40ab9f0 con 0x7f33cc158070 2023-10-26T01:32:03.079 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.075+0000 7f33cbfff640 1 --2- 172.21.15.169:0/2905862086 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f33ac06ae70 0x7f33ac06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:03.080 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.076+0000 7f33cbfff640 1 --2- 172.21.15.169:0/2905862086 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f33ac06ae70 0x7f33ac06d330 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f33bc005fd0 tx=0x7f33bc009340 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:03.080 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.079+0000 7f33c8ff9640 1 -- 172.21.15.169:0/2905862086 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f33cc0b1cd0 con 0x7f33cc158070 2023-10-26T01:32:03.266 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.265+0000 7f33d1122640 1 -- 172.21.15.169:0/2905862086 --> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f33cc0cbc40 con 0x7f33ac06ae70 2023-10-26T01:32:03.288 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.287+0000 7f33c8ff9640 1 -- 172.21.15.169:0/2905862086 <== mgr.4112 v2:172.21.15.170:6832/1187156918 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+282711 (secure 0 0 0) 0x7f33bc0099a0 con 0x7f33ac06ae70 2023-10-26T01:32:03.291 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:32:03.295 INFO:teuthology.orchestra.run.smithi169.stderr:dumped all 2023-10-26T01:32:03.296 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.294+0000 7f33d1122640 1 -- 172.21.15.169:0/2905862086 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f33ac06ae70 msgr2=0x7f33ac06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:03.297 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.294+0000 7f33d1122640 1 --2- 172.21.15.169:0/2905862086 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f33ac06ae70 0x7f33ac06d330 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f33bc005fd0 tx=0x7f33bc009340 comp rx=0 tx=0).stop 2023-10-26T01:32:03.297 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.294+0000 7f33d1122640 1 -- 172.21.15.169:0/2905862086 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33cc158070 msgr2=0x7f33cc147990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:03.297 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.294+0000 7f33d1122640 1 --2- 172.21.15.169:0/2905862086 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33cc158070 0x7f33cc147990 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f33c4009920 tx=0x7f33c4003820 comp rx=0 tx=0).stop 2023-10-26T01:32:03.298 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.294+0000 7f33d1122640 1 -- 172.21.15.169:0/2905862086 shutdown_connections 2023-10-26T01:32:03.298 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.294+0000 7f33d1122640 1 --2- 172.21.15.169:0/2905862086 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f33ac06ae70 0x7f33ac06d330 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:03.298 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.294+0000 7f33d1122640 1 --2- 172.21.15.169:0/2905862086 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33cc158070 0x7f33cc147990 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:03.298 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.294+0000 7f33d1122640 1 --2- 172.21.15.169:0/2905862086 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f33cc09ab50 0x7f33cc1f3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:03.298 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.294+0000 7f33d1122640 1 --2- 172.21.15.169:0/2905862086 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33cc06b8b0 0x7f33cc0afce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:03.299 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.294+0000 7f33d1122640 1 -- 172.21.15.169:0/2905862086 >> 172.21.15.169:0/2905862086 conn(0x7f33cc0aedd0 msgr2=0x7f33cc0c8c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:03.299 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.294+0000 7f33d1122640 1 -- 172.21.15.169:0/2905862086 shutdown_connections 2023-10-26T01:32:03.299 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.294+0000 7f33d1122640 1 -- 172.21.15.169:0/2905862086 wait complete. 2023-10-26T01:32:03.316 INFO:teuthology.orchestra.run.smithi169.stdout:{"pg_ready":true,"pg_map":{"version":74,"stamp":"2023-10-26T01:32:02.157714+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":178,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":2379,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560120,"statfs":{"total":1600000000,"available":1597563202,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.878983"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.920724+0000","last_change":"2023-10-26T01:31:57.911065+0000","last_active":"2023-10-26T01:31:58.920724+0000","last_peered":"2023-10-26T01:31:58.920724+0000","last_clean":"2023-10-26T01:31:58.920724+0000","last_became_active":"2023-10-26T01:31:57.910336+0000","last_became_peered":"2023-10-26T01:31:57.910336+0000","last_unstale":"2023-10-26T01:31:58.920724+0000","last_undegraded":"2023-10-26T01:31:58.920724+0000","last_fullsized":"2023-10-26T01:31:58.920724+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:15:19.809493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.912093+0000","last_change":"2023-10-26T01:31:57.917140+0000","last_active":"2023-10-26T01:31:58.912093+0000","last_peered":"2023-10-26T01:31:58.912093+0000","last_clean":"2023-10-26T01:31:58.912093+0000","last_became_active":"2023-10-26T01:31:57.916213+0000","last_became_peered":"2023-10-26T01:31:57.916213+0000","last_unstale":"2023-10-26T01:31:58.912093+0000","last_undegraded":"2023-10-26T01:31:58.912093+0000","last_fullsized":"2023-10-26T01:31:58.912093+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:02:28.206674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:59.260702+0000","last_change":"2023-10-26T01:31:57.919209+0000","last_active":"2023-10-26T01:31:59.260702+0000","last_peered":"2023-10-26T01:31:59.260702+0000","last_clean":"2023-10-26T01:31:59.260702+0000","last_became_active":"2023-10-26T01:31:57.917976+0000","last_became_peered":"2023-10-26T01:31:57.917976+0000","last_unstale":"2023-10-26T01:31:59.260702+0000","last_undegraded":"2023-10-26T01:31:59.260702+0000","last_fullsized":"2023-10-26T01:31:59.260702+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:35:25.011238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.919886+0000","last_change":"2023-10-26T01:31:57.928984+0000","last_active":"2023-10-26T01:31:58.919886+0000","last_peered":"2023-10-26T01:31:58.919886+0000","last_clean":"2023-10-26T01:31:58.919886+0000","last_became_active":"2023-10-26T01:31:57.928540+0000","last_became_peered":"2023-10-26T01:31:57.928540+0000","last_unstale":"2023-10-26T01:31:58.919886+0000","last_undegraded":"2023-10-26T01:31:58.919886+0000","last_fullsized":"2023-10-26T01:31:58.919886+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:14:39.347339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:59.258508+0000","last_change":"2023-10-26T01:31:57.928731+0000","last_active":"2023-10-26T01:31:59.258508+0000","last_peered":"2023-10-26T01:31:59.258508+0000","last_clean":"2023-10-26T01:31:59.258508+0000","last_became_active":"2023-10-26T01:31:57.928011+0000","last_became_peered":"2023-10-26T01:31:57.928011+0000","last_unstale":"2023-10-26T01:31:59.258508+0000","last_undegraded":"2023-10-26T01:31:59.258508+0000","last_fullsized":"2023-10-26T01:31:59.258508+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T01:48:32.945169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.911932+0000","last_change":"2023-10-26T01:31:57.931239+0000","last_active":"2023-10-26T01:31:58.911932+0000","last_peered":"2023-10-26T01:31:58.911932+0000","last_clean":"2023-10-26T01:31:58.911932+0000","last_became_active":"2023-10-26T01:31:57.930748+0000","last_became_peered":"2023-10-26T01:31:57.930748+0000","last_unstale":"2023-10-26T01:31:58.911932+0000","last_undegraded":"2023-10-26T01:31:58.911932+0000","last_fullsized":"2023-10-26T01:31:58.911932+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:21:21.898287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.916295+0000","last_change":"2023-10-26T01:31:57.923586+0000","last_active":"2023-10-26T01:31:58.916295+0000","last_peered":"2023-10-26T01:31:58.916295+0000","last_clean":"2023-10-26T01:31:58.916295+0000","last_became_active":"2023-10-26T01:31:57.923083+0000","last_became_peered":"2023-10-26T01:31:57.923083+0000","last_unstale":"2023-10-26T01:31:58.916295+0000","last_undegraded":"2023-10-26T01:31:58.916295+0000","last_fullsized":"2023-10-26T01:31:58.916295+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:51:36.984592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.914291+0000","last_change":"2023-10-26T01:31:57.927342+0000","last_active":"2023-10-26T01:31:58.914291+0000","last_peered":"2023-10-26T01:31:58.914291+0000","last_clean":"2023-10-26T01:31:58.914291+0000","last_became_active":"2023-10-26T01:31:57.927089+0000","last_became_peered":"2023-10-26T01:31:57.927089+0000","last_unstale":"2023-10-26T01:31:58.914291+0000","last_undegraded":"2023-10-26T01:31:58.914291+0000","last_fullsized":"2023-10-26T01:31:58.914291+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:43:09.209981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.913612+0000","last_change":"2023-10-26T01:31:57.928812+0000","last_active":"2023-10-26T01:31:58.913612+0000","last_peered":"2023-10-26T01:31:58.913612+0000","last_clean":"2023-10-26T01:31:58.913612+0000","last_became_active":"2023-10-26T01:31:57.928510+0000","last_became_peered":"2023-10-26T01:31:57.928510+0000","last_unstale":"2023-10-26T01:31:58.913612+0000","last_undegraded":"2023-10-26T01:31:58.913612+0000","last_fullsized":"2023-10-26T01:31:58.913612+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:42:38.754671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.920515+0000","last_change":"2023-10-26T01:31:57.924955+0000","last_active":"2023-10-26T01:31:58.920515+0000","last_peered":"2023-10-26T01:31:58.920515+0000","last_clean":"2023-10-26T01:31:58.920515+0000","last_became_active":"2023-10-26T01:31:57.924221+0000","last_became_peered":"2023-10-26T01:31:57.924221+0000","last_unstale":"2023-10-26T01:31:58.920515+0000","last_undegraded":"2023-10-26T01:31:58.920515+0000","last_fullsized":"2023-10-26T01:31:58.920515+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:24:57.262505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.917212+0000","last_change":"2023-10-26T01:31:57.924771+0000","last_active":"2023-10-26T01:31:58.917212+0000","last_peered":"2023-10-26T01:31:58.917212+0000","last_clean":"2023-10-26T01:31:58.917212+0000","last_became_active":"2023-10-26T01:31:57.924320+0000","last_became_peered":"2023-10-26T01:31:57.924320+0000","last_unstale":"2023-10-26T01:31:58.917212+0000","last_undegraded":"2023-10-26T01:31:58.917212+0000","last_fullsized":"2023-10-26T01:31:58.917212+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:21:21.898287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:59.259571+0000","last_change":"2023-10-26T01:31:57.915944+0000","last_active":"2023-10-26T01:31:59.259571+0000","last_peered":"2023-10-26T01:31:59.259571+0000","last_clean":"2023-10-26T01:31:59.259571+0000","last_became_active":"2023-10-26T01:31:57.914541+0000","last_became_peered":"2023-10-26T01:31:57.914541+0000","last_unstale":"2023-10-26T01:31:59.259571+0000","last_undegraded":"2023-10-26T01:31:59.259571+0000","last_fullsized":"2023-10-26T01:31:59.259571+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:14:39.347339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:59.258382+0000","last_change":"2023-10-26T01:31:57.931634+0000","last_active":"2023-10-26T01:31:59.258382+0000","last_peered":"2023-10-26T01:31:59.258382+0000","last_clean":"2023-10-26T01:31:59.258382+0000","last_became_active":"2023-10-26T01:31:57.930738+0000","last_became_peered":"2023-10-26T01:31:57.930738+0000","last_unstale":"2023-10-26T01:31:59.258382+0000","last_undegraded":"2023-10-26T01:31:59.258382+0000","last_fullsized":"2023-10-26T01:31:59.258382+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:29:13.336121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.912828+0000","last_change":"2023-10-26T01:31:57.929136+0000","last_active":"2023-10-26T01:31:58.912828+0000","last_peered":"2023-10-26T01:31:58.912828+0000","last_clean":"2023-10-26T01:31:58.912828+0000","last_became_active":"2023-10-26T01:31:57.928427+0000","last_became_peered":"2023-10-26T01:31:57.928427+0000","last_unstale":"2023-10-26T01:31:58.912828+0000","last_undegraded":"2023-10-26T01:31:58.912828+0000","last_fullsized":"2023-10-26T01:31:58.912828+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:52:00.679485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:59.260174+0000","last_change":"2023-10-26T01:31:57.916184+0000","last_active":"2023-10-26T01:31:59.260174+0000","last_peered":"2023-10-26T01:31:59.260174+0000","last_clean":"2023-10-26T01:31:59.260174+0000","last_became_active":"2023-10-26T01:31:57.912088+0000","last_became_peered":"2023-10-26T01:31:57.912088+0000","last_unstale":"2023-10-26T01:31:59.260174+0000","last_undegraded":"2023-10-26T01:31:59.260174+0000","last_fullsized":"2023-10-26T01:31:59.260174+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:24:57.262505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.917565+0000","last_change":"2023-10-26T01:31:57.927881+0000","last_active":"2023-10-26T01:31:58.917565+0000","last_peered":"2023-10-26T01:31:58.917565+0000","last_clean":"2023-10-26T01:31:58.917565+0000","last_became_active":"2023-10-26T01:31:57.926937+0000","last_became_peered":"2023-10-26T01:31:57.926937+0000","last_unstale":"2023-10-26T01:31:58.917565+0000","last_undegraded":"2023-10-26T01:31:58.917565+0000","last_fullsized":"2023-10-26T01:31:58.917565+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:35:25.011238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.915589+0000","last_change":"2023-10-26T01:31:57.923381+0000","last_active":"2023-10-26T01:31:58.915589+0000","last_peered":"2023-10-26T01:31:58.915589+0000","last_clean":"2023-10-26T01:31:58.915589+0000","last_became_active":"2023-10-26T01:31:57.922670+0000","last_became_peered":"2023-10-26T01:31:57.922670+0000","last_unstale":"2023-10-26T01:31:58.915589+0000","last_undegraded":"2023-10-26T01:31:58.915589+0000","last_fullsized":"2023-10-26T01:31:58.915589+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T01:34:44.444835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.913006+0000","last_change":"2023-10-26T01:31:57.923066+0000","last_active":"2023-10-26T01:31:58.913006+0000","last_peered":"2023-10-26T01:31:58.913006+0000","last_clean":"2023-10-26T01:31:58.913006+0000","last_became_active":"2023-10-26T01:31:57.922786+0000","last_became_peered":"2023-10-26T01:31:57.922786+0000","last_unstale":"2023-10-26T01:31:58.913006+0000","last_undegraded":"2023-10-26T01:31:58.913006+0000","last_fullsized":"2023-10-26T01:31:58.913006+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:01:00.051777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.911935+0000","last_change":"2023-10-26T01:31:57.928697+0000","last_active":"2023-10-26T01:31:58.911935+0000","last_peered":"2023-10-26T01:31:58.911935+0000","last_clean":"2023-10-26T01:31:58.911935+0000","last_became_active":"2023-10-26T01:31:57.925505+0000","last_became_peered":"2023-10-26T01:31:57.925505+0000","last_unstale":"2023-10-26T01:31:58.911935+0000","last_undegraded":"2023-10-26T01:31:58.911935+0000","last_fullsized":"2023-10-26T01:31:58.911935+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:24:57.262505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:59.258675+0000","last_change":"2023-10-26T01:31:57.929809+0000","last_active":"2023-10-26T01:31:59.258675+0000","last_peered":"2023-10-26T01:31:59.258675+0000","last_clean":"2023-10-26T01:31:59.258675+0000","last_became_active":"2023-10-26T01:31:57.928334+0000","last_became_peered":"2023-10-26T01:31:57.928334+0000","last_unstale":"2023-10-26T01:31:59.258675+0000","last_undegraded":"2023-10-26T01:31:59.258675+0000","last_fullsized":"2023-10-26T01:31:59.258675+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:44:10.920523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.915810+0000","last_change":"2023-10-26T01:31:57.930806+0000","last_active":"2023-10-26T01:31:58.915810+0000","last_peered":"2023-10-26T01:31:58.915810+0000","last_clean":"2023-10-26T01:31:58.915810+0000","last_became_active":"2023-10-26T01:31:57.930542+0000","last_became_peered":"2023-10-26T01:31:57.930542+0000","last_unstale":"2023-10-26T01:31:58.915810+0000","last_undegraded":"2023-10-26T01:31:58.915810+0000","last_fullsized":"2023-10-26T01:31:58.915810+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:51:09.298744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.914981+0000","last_change":"2023-10-26T01:31:57.931637+0000","last_active":"2023-10-26T01:31:58.914981+0000","last_peered":"2023-10-26T01:31:58.914981+0000","last_clean":"2023-10-26T01:31:58.914981+0000","last_became_active":"2023-10-26T01:31:57.931346+0000","last_became_peered":"2023-10-26T01:31:57.931346+0000","last_unstale":"2023-10-26T01:31:58.914981+0000","last_undegraded":"2023-10-26T01:31:58.914981+0000","last_fullsized":"2023-10-26T01:31:58.914981+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:31:13.736811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:59.259353+0000","last_change":"2023-10-26T01:31:57.924443+0000","last_active":"2023-10-26T01:31:59.259353+0000","last_peered":"2023-10-26T01:31:59.259353+0000","last_clean":"2023-10-26T01:31:59.259353+0000","last_became_active":"2023-10-26T01:31:57.921067+0000","last_became_peered":"2023-10-26T01:31:57.921067+0000","last_unstale":"2023-10-26T01:31:59.259353+0000","last_undegraded":"2023-10-26T01:31:59.259353+0000","last_fullsized":"2023-10-26T01:31:59.259353+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:13:25.337972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:59.259584+0000","last_change":"2023-10-26T01:31:57.924740+0000","last_active":"2023-10-26T01:31:59.259584+0000","last_peered":"2023-10-26T01:31:59.259584+0000","last_clean":"2023-10-26T01:31:59.259584+0000","last_became_active":"2023-10-26T01:31:57.924031+0000","last_became_peered":"2023-10-26T01:31:57.924031+0000","last_unstale":"2023-10-26T01:31:59.259584+0000","last_undegraded":"2023-10-26T01:31:59.259584+0000","last_fullsized":"2023-10-26T01:31:59.259584+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T01:34:44.444835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.919849+0000","last_change":"2023-10-26T01:31:57.925186+0000","last_active":"2023-10-26T01:31:58.919849+0000","last_peered":"2023-10-26T01:31:58.919849+0000","last_clean":"2023-10-26T01:31:58.919849+0000","last_became_active":"2023-10-26T01:31:57.924666+0000","last_became_peered":"2023-10-26T01:31:57.924666+0000","last_unstale":"2023-10-26T01:31:58.919849+0000","last_undegraded":"2023-10-26T01:31:58.919849+0000","last_fullsized":"2023-10-26T01:31:58.919849+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:43:09.209981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.912077+0000","last_change":"2023-10-26T01:31:57.922444+0000","last_active":"2023-10-26T01:31:58.912077+0000","last_peered":"2023-10-26T01:31:58.912077+0000","last_clean":"2023-10-26T01:31:58.912077+0000","last_became_active":"2023-10-26T01:31:57.922018+0000","last_became_peered":"2023-10-26T01:31:57.922018+0000","last_unstale":"2023-10-26T01:31:58.912077+0000","last_undegraded":"2023-10-26T01:31:58.912077+0000","last_fullsized":"2023-10-26T01:31:58.912077+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:06:00.266085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.919990+0000","last_change":"2023-10-26T01:31:57.930896+0000","last_active":"2023-10-26T01:31:58.919990+0000","last_peered":"2023-10-26T01:31:58.919990+0000","last_clean":"2023-10-26T01:31:58.919990+0000","last_became_active":"2023-10-26T01:31:57.930441+0000","last_became_peered":"2023-10-26T01:31:57.930441+0000","last_unstale":"2023-10-26T01:31:58.919990+0000","last_undegraded":"2023-10-26T01:31:58.919990+0000","last_fullsized":"2023-10-26T01:31:58.919990+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:56:29.283750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:59.259579+0000","last_change":"2023-10-26T01:31:57.918538+0000","last_active":"2023-10-26T01:31:59.259579+0000","last_peered":"2023-10-26T01:31:59.259579+0000","last_clean":"2023-10-26T01:31:59.259579+0000","last_became_active":"2023-10-26T01:31:57.915935+0000","last_became_peered":"2023-10-26T01:31:57.915935+0000","last_unstale":"2023-10-26T01:31:59.259579+0000","last_undegraded":"2023-10-26T01:31:59.259579+0000","last_fullsized":"2023-10-26T01:31:59.259579+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:56:29.283750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.913965+0000","last_change":"2023-10-26T01:31:57.931907+0000","last_active":"2023-10-26T01:31:58.913965+0000","last_peered":"2023-10-26T01:31:58.913965+0000","last_clean":"2023-10-26T01:31:58.913965+0000","last_became_active":"2023-10-26T01:31:57.931618+0000","last_became_peered":"2023-10-26T01:31:57.931618+0000","last_unstale":"2023-10-26T01:31:58.913965+0000","last_undegraded":"2023-10-26T01:31:58.913965+0000","last_fullsized":"2023-10-26T01:31:58.913965+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:17:24.476006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.917038+0000","last_change":"2023-10-26T01:31:57.919923+0000","last_active":"2023-10-26T01:31:58.917038+0000","last_peered":"2023-10-26T01:31:58.917038+0000","last_clean":"2023-10-26T01:31:58.917038+0000","last_became_active":"2023-10-26T01:31:57.919348+0000","last_became_peered":"2023-10-26T01:31:57.919348+0000","last_unstale":"2023-10-26T01:31:58.917038+0000","last_undegraded":"2023-10-26T01:31:58.917038+0000","last_fullsized":"2023-10-26T01:31:58.917038+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:24:57.262505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.921059+0000","last_change":"2023-10-26T01:31:57.926414+0000","last_active":"2023-10-26T01:31:58.921059+0000","last_peered":"2023-10-26T01:31:58.921059+0000","last_clean":"2023-10-26T01:31:58.921059+0000","last_became_active":"2023-10-26T01:31:57.926058+0000","last_became_peered":"2023-10-26T01:31:57.926058+0000","last_unstale":"2023-10-26T01:31:58.921059+0000","last_undegraded":"2023-10-26T01:31:58.921059+0000","last_fullsized":"2023-10-26T01:31:58.921059+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:31:13.736811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.915935+0000","last_change":"2023-10-26T01:31:57.929677+0000","last_active":"2023-10-26T01:31:58.915935+0000","last_peered":"2023-10-26T01:31:58.915935+0000","last_clean":"2023-10-26T01:31:58.915935+0000","last_became_active":"2023-10-26T01:31:57.928493+0000","last_became_peered":"2023-10-26T01:31:57.928493+0000","last_unstale":"2023-10-26T01:31:58.915935+0000","last_undegraded":"2023-10-26T01:31:58.915935+0000","last_fullsized":"2023-10-26T01:31:58.915935+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:30:35.357843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:59.259004+0000","last_change":"2023-10-26T01:31:57.923481+0000","last_active":"2023-10-26T01:31:59.259004+0000","last_peered":"2023-10-26T01:31:59.259004+0000","last_clean":"2023-10-26T01:31:59.259004+0000","last_became_active":"2023-10-26T01:31:57.923159+0000","last_became_peered":"2023-10-26T01:31:57.923159+0000","last_unstale":"2023-10-26T01:31:59.259004+0000","last_undegraded":"2023-10-26T01:31:59.259004+0000","last_fullsized":"2023-10-26T01:31:59.259004+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:52:33.901445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"34'4","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.920754+0000","last_change":"2023-10-26T01:31:57.313244+0000","last_active":"2023-10-26T01:31:58.920754+0000","last_peered":"2023-10-26T01:31:58.920754+0000","last_clean":"2023-10-26T01:31:58.920754+0000","last_became_active":"2023-10-26T01:31:57.312346+0000","last_became_peered":"2023-10-26T01:31:57.312346+0000","last_unstale":"2023-10-26T01:31:58.920754+0000","last_undegraded":"2023-10-26T01:31:58.920754+0000","last_fullsized":"2023-10-26T01:31:58.920754+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_clean_scrub_stamp":"2023-10-26T01:31:56.263810+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:27:21.313977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.920492+0000","last_change":"2023-10-26T01:31:57.929374+0000","last_active":"2023-10-26T01:31:58.920492+0000","last_peered":"2023-10-26T01:31:58.920492+0000","last_clean":"2023-10-26T01:31:58.920492+0000","last_became_active":"2023-10-26T01:31:57.929078+0000","last_became_peered":"2023-10-26T01:31:57.929078+0000","last_unstale":"2023-10-26T01:31:58.920492+0000","last_undegraded":"2023-10-26T01:31:58.920492+0000","last_fullsized":"2023-10-26T01:31:58.920492+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:42:38.754671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"34'1","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:59.258865+0000","last_change":"2023-10-26T01:31:57.315953+0000","last_active":"2023-10-26T01:31:59.258865+0000","last_peered":"2023-10-26T01:31:59.258865+0000","last_clean":"2023-10-26T01:31:59.258865+0000","last_became_active":"2023-10-26T01:31:57.315806+0000","last_became_peered":"2023-10-26T01:31:57.315806+0000","last_unstale":"2023-10-26T01:31:59.258865+0000","last_undegraded":"2023-10-26T01:31:59.258865+0000","last_fullsized":"2023-10-26T01:31:59.258865+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_clean_scrub_stamp":"2023-10-26T01:31:56.263810+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:20:35.297390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.916144+0000","last_change":"2023-10-26T01:31:57.932521+0000","last_active":"2023-10-26T01:31:58.916144+0000","last_peered":"2023-10-26T01:31:58.916144+0000","last_clean":"2023-10-26T01:31:58.916144+0000","last_became_active":"2023-10-26T01:31:57.932252+0000","last_became_peered":"2023-10-26T01:31:57.932252+0000","last_unstale":"2023-10-26T01:31:58.916144+0000","last_undegraded":"2023-10-26T01:31:58.916144+0000","last_fullsized":"2023-10-26T01:31:58.916144+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:52:33.901445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"34'2","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.332680+0000","last_change":"2023-10-26T01:31:57.307484+0000","last_active":"2023-10-26T01:31:58.332680+0000","last_peered":"2023-10-26T01:31:58.332680+0000","last_clean":"2023-10-26T01:31:58.332680+0000","last_became_active":"2023-10-26T01:31:57.307186+0000","last_became_peered":"2023-10-26T01:31:57.307186+0000","last_unstale":"2023-10-26T01:31:58.332680+0000","last_undegraded":"2023-10-26T01:31:58.332680+0000","last_fullsized":"2023-10-26T01:31:58.332680+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_clean_scrub_stamp":"2023-10-26T01:31:56.263810+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:30:34.344731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.920185+0000","last_change":"2023-10-26T01:31:57.929917+0000","last_active":"2023-10-26T01:31:58.920185+0000","last_peered":"2023-10-26T01:31:58.920185+0000","last_clean":"2023-10-26T01:31:58.920185+0000","last_became_active":"2023-10-26T01:31:57.929167+0000","last_became_peered":"2023-10-26T01:31:57.929167+0000","last_unstale":"2023-10-26T01:31:58.920185+0000","last_undegraded":"2023-10-26T01:31:58.920185+0000","last_fullsized":"2023-10-26T01:31:58.920185+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:51:36.984592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"34'2","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:59.258843+0000","last_change":"2023-10-26T01:31:57.302026+0000","last_active":"2023-10-26T01:31:59.258843+0000","last_peered":"2023-10-26T01:31:59.258843+0000","last_clean":"2023-10-26T01:31:59.258843+0000","last_became_active":"2023-10-26T01:31:57.301463+0000","last_became_peered":"2023-10-26T01:31:57.301463+0000","last_unstale":"2023-10-26T01:31:59.258843+0000","last_undegraded":"2023-10-26T01:31:59.258843+0000","last_fullsized":"2023-10-26T01:31:59.258843+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_clean_scrub_stamp":"2023-10-26T01:31:56.263810+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:46:19.545323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.914785+0000","last_change":"2023-10-26T01:31:57.930260+0000","last_active":"2023-10-26T01:31:58.914785+0000","last_peered":"2023-10-26T01:31:58.914785+0000","last_clean":"2023-10-26T01:31:58.914785+0000","last_became_active":"2023-10-26T01:31:57.929537+0000","last_became_peered":"2023-10-26T01:31:57.929537+0000","last_unstale":"2023-10-26T01:31:58.914785+0000","last_undegraded":"2023-10-26T01:31:58.914785+0000","last_fullsized":"2023-10-26T01:31:58.914785+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:14:39.347339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"34'1","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.914722+0000","last_change":"2023-10-26T01:31:57.302186+0000","last_active":"2023-10-26T01:31:58.914722+0000","last_peered":"2023-10-26T01:31:58.914722+0000","last_clean":"2023-10-26T01:31:58.914722+0000","last_became_active":"2023-10-26T01:31:57.299001+0000","last_became_peered":"2023-10-26T01:31:57.299001+0000","last_unstale":"2023-10-26T01:31:58.914722+0000","last_undegraded":"2023-10-26T01:31:58.914722+0000","last_fullsized":"2023-10-26T01:31:58.914722+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_clean_scrub_stamp":"2023-10-26T01:31:56.263810+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:29:39.990303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:59.259925+0000","last_change":"2023-10-26T01:31:57.919906+0000","last_active":"2023-10-26T01:31:59.259925+0000","last_peered":"2023-10-26T01:31:59.259925+0000","last_clean":"2023-10-26T01:31:59.259925+0000","last_became_active":"2023-10-26T01:31:57.915475+0000","last_became_peered":"2023-10-26T01:31:57.915475+0000","last_unstale":"2023-10-26T01:31:59.259925+0000","last_undegraded":"2023-10-26T01:31:59.259925+0000","last_fullsized":"2023-10-26T01:31:59.259925+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:51:36.984592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"34'1","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.913502+0000","last_change":"2023-10-26T01:31:57.295016+0000","last_active":"2023-10-26T01:31:58.913502+0000","last_peered":"2023-10-26T01:31:58.913502+0000","last_clean":"2023-10-26T01:31:58.913502+0000","last_became_active":"2023-10-26T01:31:57.294352+0000","last_became_peered":"2023-10-26T01:31:57.294352+0000","last_unstale":"2023-10-26T01:31:58.913502+0000","last_undegraded":"2023-10-26T01:31:58.913502+0000","last_fullsized":"2023-10-26T01:31:58.913502+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_clean_scrub_stamp":"2023-10-26T01:31:56.263810+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:41:28.795912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.913210+0000","last_change":"2023-10-26T01:31:57.928375+0000","last_active":"2023-10-26T01:31:58.913210+0000","last_peered":"2023-10-26T01:31:58.913210+0000","last_clean":"2023-10-26T01:31:58.913210+0000","last_became_active":"2023-10-26T01:31:57.927961+0000","last_became_peered":"2023-10-26T01:31:57.927961+0000","last_unstale":"2023-10-26T01:31:58.913210+0000","last_undegraded":"2023-10-26T01:31:58.913210+0000","last_fullsized":"2023-10-26T01:31:58.913210+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:49:05.475592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-26T01:31:57.909884+0000","last_change":"2023-10-26T01:31:57.305633+0000","last_active":"2023-10-26T01:31:57.909884+0000","last_peered":"2023-10-26T01:31:57.909884+0000","last_clean":"2023-10-26T01:31:57.909884+0000","last_became_active":"2023-10-26T01:31:57.305411+0000","last_became_peered":"2023-10-26T01:31:57.305411+0000","last_unstale":"2023-10-26T01:31:57.909884+0000","last_undegraded":"2023-10-26T01:31:57.909884+0000","last_fullsized":"2023-10-26T01:31:57.909884+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_clean_scrub_stamp":"2023-10-26T01:31:56.263810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:04:38.277788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.920072+0000","last_change":"2023-10-26T01:31:57.925433+0000","last_active":"2023-10-26T01:31:58.920072+0000","last_peered":"2023-10-26T01:31:58.920072+0000","last_clean":"2023-10-26T01:31:58.920072+0000","last_became_active":"2023-10-26T01:31:57.923390+0000","last_became_peered":"2023-10-26T01:31:57.923390+0000","last_unstale":"2023-10-26T01:31:58.920072+0000","last_undegraded":"2023-10-26T01:31:58.920072+0000","last_fullsized":"2023-10-26T01:31:58.920072+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T01:34:44.444835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.913982+0000","last_change":"2023-10-26T01:31:57.312772+0000","last_active":"2023-10-26T01:31:58.913982+0000","last_peered":"2023-10-26T01:31:58.913982+0000","last_clean":"2023-10-26T01:31:58.913982+0000","last_became_active":"2023-10-26T01:31:57.310660+0000","last_became_peered":"2023-10-26T01:31:57.310660+0000","last_unstale":"2023-10-26T01:31:58.913982+0000","last_undegraded":"2023-10-26T01:31:58.913982+0000","last_fullsized":"2023-10-26T01:31:58.913982+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_clean_scrub_stamp":"2023-10-26T01:31:56.263810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:27:03.964632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:59.260333+0000","last_change":"2023-10-26T01:31:57.920433+0000","last_active":"2023-10-26T01:31:59.260333+0000","last_peered":"2023-10-26T01:31:59.260333+0000","last_clean":"2023-10-26T01:31:59.260333+0000","last_became_active":"2023-10-26T01:31:57.919660+0000","last_became_peered":"2023-10-26T01:31:57.919660+0000","last_unstale":"2023-10-26T01:31:59.260333+0000","last_undegraded":"2023-10-26T01:31:59.260333+0000","last_fullsized":"2023-10-26T01:31:59.260333+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:06:00.266085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-26T01:31:57.910743+0000","last_change":"2023-10-26T01:31:57.317225+0000","last_active":"2023-10-26T01:31:57.910743+0000","last_peered":"2023-10-26T01:31:57.910743+0000","last_clean":"2023-10-26T01:31:57.910743+0000","last_became_active":"2023-10-26T01:31:57.316985+0000","last_became_peered":"2023-10-26T01:31:57.316985+0000","last_unstale":"2023-10-26T01:31:57.910743+0000","last_undegraded":"2023-10-26T01:31:57.910743+0000","last_fullsized":"2023-10-26T01:31:57.910743+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_clean_scrub_stamp":"2023-10-26T01:31:56.263810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:14:38.334226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.912217+0000","last_change":"2023-10-26T01:31:57.913892+0000","last_active":"2023-10-26T01:31:58.912217+0000","last_peered":"2023-10-26T01:31:58.912217+0000","last_clean":"2023-10-26T01:31:58.912217+0000","last_became_active":"2023-10-26T01:31:57.913251+0000","last_became_peered":"2023-10-26T01:31:57.913251+0000","last_unstale":"2023-10-26T01:31:58.912217+0000","last_undegraded":"2023-10-26T01:31:58.912217+0000","last_fullsized":"2023-10-26T01:31:58.912217+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:35:25.011238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:59.259184+0000","last_change":"2023-10-26T01:31:57.300118+0000","last_active":"2023-10-26T01:31:59.259184+0000","last_peered":"2023-10-26T01:31:59.259184+0000","last_clean":"2023-10-26T01:31:59.259184+0000","last_became_active":"2023-10-26T01:31:57.299790+0000","last_became_peered":"2023-10-26T01:31:57.299790+0000","last_unstale":"2023-10-26T01:31:59.259184+0000","last_undegraded":"2023-10-26T01:31:59.259184+0000","last_fullsized":"2023-10-26T01:31:59.259184+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_clean_scrub_stamp":"2023-10-26T01:31:56.263810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:27:21.313977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.913562+0000","last_change":"2023-10-26T01:31:57.922545+0000","last_active":"2023-10-26T01:31:58.913562+0000","last_peered":"2023-10-26T01:31:58.913562+0000","last_clean":"2023-10-26T01:31:58.913562+0000","last_became_active":"2023-10-26T01:31:57.922183+0000","last_became_peered":"2023-10-26T01:31:57.922183+0000","last_unstale":"2023-10-26T01:31:58.913562+0000","last_undegraded":"2023-10-26T01:31:58.913562+0000","last_fullsized":"2023-10-26T01:31:58.913562+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T01:43:15.965456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"34'3","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:59.260933+0000","last_change":"2023-10-26T01:31:57.318179+0000","last_active":"2023-10-26T01:31:59.260933+0000","last_peered":"2023-10-26T01:31:59.260933+0000","last_clean":"2023-10-26T01:31:59.260933+0000","last_became_active":"2023-10-26T01:31:57.317971+0000","last_became_peered":"2023-10-26T01:31:57.317971+0000","last_unstale":"2023-10-26T01:31:59.260933+0000","last_undegraded":"2023-10-26T01:31:59.260933+0000","last_fullsized":"2023-10-26T01:31:59.260933+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_clean_scrub_stamp":"2023-10-26T01:31:56.263810+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:04:38.277788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:59.260339+0000","last_change":"2023-10-26T01:31:57.918887+0000","last_active":"2023-10-26T01:31:59.260339+0000","last_peered":"2023-10-26T01:31:59.260339+0000","last_clean":"2023-10-26T01:31:59.260339+0000","last_became_active":"2023-10-26T01:31:57.917309+0000","last_became_peered":"2023-10-26T01:31:57.917309+0000","last_unstale":"2023-10-26T01:31:59.260339+0000","last_undegraded":"2023-10-26T01:31:59.260339+0000","last_fullsized":"2023-10-26T01:31:59.260339+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:21:21.898287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"34'6","reported_seq":18,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.338139+0000","last_change":"2023-10-26T01:31:57.299129+0000","last_active":"2023-10-26T01:31:58.338139+0000","last_peered":"2023-10-26T01:31:58.338139+0000","last_clean":"2023-10-26T01:31:58.338139+0000","last_became_active":"2023-10-26T01:31:57.298909+0000","last_became_peered":"2023-10-26T01:31:57.298909+0000","last_unstale":"2023-10-26T01:31:58.338139+0000","last_undegraded":"2023-10-26T01:31:58.338139+0000","last_fullsized":"2023-10-26T01:31:58.338139+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_clean_scrub_stamp":"2023-10-26T01:31:56.263810+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:43:08.196868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.920106+0000","last_change":"2023-10-26T01:31:57.923375+0000","last_active":"2023-10-26T01:31:58.920106+0000","last_peered":"2023-10-26T01:31:58.920106+0000","last_clean":"2023-10-26T01:31:58.920106+0000","last_became_active":"2023-10-26T01:31:57.922926+0000","last_became_peered":"2023-10-26T01:31:57.922926+0000","last_unstale":"2023-10-26T01:31:58.920106+0000","last_undegraded":"2023-10-26T01:31:58.920106+0000","last_fullsized":"2023-10-26T01:31:58.920106+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:30:35.357843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"34'2","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.914183+0000","last_change":"2023-10-26T01:31:57.307255+0000","last_active":"2023-10-26T01:31:58.914183+0000","last_peered":"2023-10-26T01:31:58.914183+0000","last_clean":"2023-10-26T01:31:58.914183+0000","last_became_active":"2023-10-26T01:31:57.306318+0000","last_became_peered":"2023-10-26T01:31:57.306318+0000","last_unstale":"2023-10-26T01:31:58.914183+0000","last_undegraded":"2023-10-26T01:31:58.914183+0000","last_fullsized":"2023-10-26T01:31:58.914183+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_clean_scrub_stamp":"2023-10-26T01:31:56.263810+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:22:09.081480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.912334+0000","last_change":"2023-10-26T01:31:57.923545+0000","last_active":"2023-10-26T01:31:58.912334+0000","last_peered":"2023-10-26T01:31:58.912334+0000","last_clean":"2023-10-26T01:31:58.912334+0000","last_became_active":"2023-10-26T01:31:57.920460+0000","last_became_peered":"2023-10-26T01:31:57.920460+0000","last_unstale":"2023-10-26T01:31:58.912334+0000","last_undegraded":"2023-10-26T01:31:58.912334+0000","last_fullsized":"2023-10-26T01:31:58.912334+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:58:30.843955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.920731+0000","last_change":"2023-10-26T01:31:57.306563+0000","last_active":"2023-10-26T01:31:58.920731+0000","last_peered":"2023-10-26T01:31:58.920731+0000","last_clean":"2023-10-26T01:31:58.920731+0000","last_became_active":"2023-10-26T01:31:57.306305+0000","last_became_peered":"2023-10-26T01:31:57.306305+0000","last_unstale":"2023-10-26T01:31:58.920731+0000","last_undegraded":"2023-10-26T01:31:58.920731+0000","last_fullsized":"2023-10-26T01:31:58.920731+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_clean_scrub_stamp":"2023-10-26T01:31:56.263810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:17:27.685445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.915062+0000","last_change":"2023-10-26T01:31:57.930623+0000","last_active":"2023-10-26T01:31:58.915062+0000","last_peered":"2023-10-26T01:31:58.915062+0000","last_clean":"2023-10-26T01:31:58.915062+0000","last_became_active":"2023-10-26T01:31:57.929835+0000","last_became_peered":"2023-10-26T01:31:57.929835+0000","last_unstale":"2023-10-26T01:31:58.915062+0000","last_undegraded":"2023-10-26T01:31:58.915062+0000","last_fullsized":"2023-10-26T01:31:58.915062+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:56:29.283750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-26T01:31:57.908174+0000","last_change":"2023-10-26T01:31:57.317377+0000","last_active":"2023-10-26T01:31:57.908174+0000","last_peered":"2023-10-26T01:31:57.908174+0000","last_clean":"2023-10-26T01:31:57.908174+0000","last_became_active":"2023-10-26T01:31:57.317103+0000","last_became_peered":"2023-10-26T01:31:57.317103+0000","last_unstale":"2023-10-26T01:31:57.908174+0000","last_undegraded":"2023-10-26T01:31:57.908174+0000","last_fullsized":"2023-10-26T01:31:57.908174+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_clean_scrub_stamp":"2023-10-26T01:31:56.263810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:51:08.285631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.920966+0000","last_change":"2023-10-26T01:31:57.928078+0000","last_active":"2023-10-26T01:31:58.920966+0000","last_peered":"2023-10-26T01:31:58.920966+0000","last_clean":"2023-10-26T01:31:58.920966+0000","last_became_active":"2023-10-26T01:31:57.927787+0000","last_became_peered":"2023-10-26T01:31:57.927787+0000","last_unstale":"2023-10-26T01:31:58.920966+0000","last_undegraded":"2023-10-26T01:31:58.920966+0000","last_fullsized":"2023-10-26T01:31:58.920966+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:17:24.476006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"34'8","reported_seq":20,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.333955+0000","last_change":"2023-10-26T01:31:57.298215+0000","last_active":"2023-10-26T01:31:58.333955+0000","last_peered":"2023-10-26T01:31:58.333955+0000","last_clean":"2023-10-26T01:31:58.333955+0000","last_became_active":"2023-10-26T01:31:57.297999+0000","last_became_peered":"2023-10-26T01:31:57.297999+0000","last_unstale":"2023-10-26T01:31:58.333955+0000","last_undegraded":"2023-10-26T01:31:58.333955+0000","last_fullsized":"2023-10-26T01:31:58.333955+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_clean_scrub_stamp":"2023-10-26T01:31:56.263810+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:15:56.135260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.913796+0000","last_change":"2023-10-26T01:31:57.928206+0000","last_active":"2023-10-26T01:31:58.913796+0000","last_peered":"2023-10-26T01:31:58.913796+0000","last_clean":"2023-10-26T01:31:58.913796+0000","last_became_active":"2023-10-26T01:31:57.927366+0000","last_became_peered":"2023-10-26T01:31:57.927366+0000","last_unstale":"2023-10-26T01:31:58.913796+0000","last_undegraded":"2023-10-26T01:31:58.913796+0000","last_fullsized":"2023-10-26T01:31:58.913796+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:15:19.809493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"34'1","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.920577+0000","last_change":"2023-10-26T01:31:57.300699+0000","last_active":"2023-10-26T01:31:58.920577+0000","last_peered":"2023-10-26T01:31:58.920577+0000","last_clean":"2023-10-26T01:31:58.920577+0000","last_became_active":"2023-10-26T01:31:57.300230+0000","last_became_peered":"2023-10-26T01:31:57.300230+0000","last_unstale":"2023-10-26T01:31:58.920577+0000","last_undegraded":"2023-10-26T01:31:58.920577+0000","last_fullsized":"2023-10-26T01:31:58.920577+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_clean_scrub_stamp":"2023-10-26T01:31:56.263810+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:15:18.796380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:59.259805+0000","last_change":"2023-10-26T01:31:57.925303+0000","last_active":"2023-10-26T01:31:59.259805+0000","last_peered":"2023-10-26T01:31:59.259805+0000","last_clean":"2023-10-26T01:31:59.259805+0000","last_became_active":"2023-10-26T01:31:57.924425+0000","last_became_peered":"2023-10-26T01:31:57.924425+0000","last_unstale":"2023-10-26T01:31:59.259805+0000","last_undegraded":"2023-10-26T01:31:59.259805+0000","last_fullsized":"2023-10-26T01:31:59.259805+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:30:35.357843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"34'2","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.914457+0000","last_change":"2023-10-26T01:31:57.306952+0000","last_active":"2023-10-26T01:31:58.914457+0000","last_peered":"2023-10-26T01:31:58.914457+0000","last_clean":"2023-10-26T01:31:58.914457+0000","last_became_active":"2023-10-26T01:31:57.305511+0000","last_became_peered":"2023-10-26T01:31:57.305511+0000","last_unstale":"2023-10-26T01:31:58.914457+0000","last_undegraded":"2023-10-26T01:31:58.914457+0000","last_fullsized":"2023-10-26T01:31:58.914457+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_clean_scrub_stamp":"2023-10-26T01:31:56.263810+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:22:42.804254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.919760+0000","last_change":"2023-10-26T01:31:57.930755+0000","last_active":"2023-10-26T01:31:58.919760+0000","last_peered":"2023-10-26T01:31:58.919760+0000","last_clean":"2023-10-26T01:31:58.919760+0000","last_became_active":"2023-10-26T01:31:57.930037+0000","last_became_peered":"2023-10-26T01:31:57.930037+0000","last_unstale":"2023-10-26T01:31:58.919760+0000","last_undegraded":"2023-10-26T01:31:58.919760+0000","last_fullsized":"2023-10-26T01:31:58.919760+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:04:39.290900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"34'1","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.916389+0000","last_change":"2023-10-26T01:31:57.295883+0000","last_active":"2023-10-26T01:31:58.916389+0000","last_peered":"2023-10-26T01:31:58.916389+0000","last_clean":"2023-10-26T01:31:58.916389+0000","last_became_active":"2023-10-26T01:31:57.295190+0000","last_became_peered":"2023-10-26T01:31:57.295190+0000","last_unstale":"2023-10-26T01:31:58.916389+0000","last_undegraded":"2023-10-26T01:31:58.916389+0000","last_fullsized":"2023-10-26T01:31:58.916389+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_clean_scrub_stamp":"2023-10-26T01:31:56.263810+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T01:48:31.932057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:59.258457+0000","last_change":"2023-10-26T01:31:57.926385+0000","last_active":"2023-10-26T01:31:59.258457+0000","last_peered":"2023-10-26T01:31:59.258457+0000","last_clean":"2023-10-26T01:31:59.258457+0000","last_became_active":"2023-10-26T01:31:57.926127+0000","last_became_peered":"2023-10-26T01:31:57.926127+0000","last_unstale":"2023-10-26T01:31:59.258457+0000","last_undegraded":"2023-10-26T01:31:59.258457+0000","last_fullsized":"2023-10-26T01:31:59.258457+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:27:59.914223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.915095+0000","last_change":"2023-10-26T01:31:57.301740+0000","last_active":"2023-10-26T01:31:58.915095+0000","last_peered":"2023-10-26T01:31:58.915095+0000","last_clean":"2023-10-26T01:31:58.915095+0000","last_became_active":"2023-10-26T01:31:57.297718+0000","last_became_peered":"2023-10-26T01:31:57.297718+0000","last_unstale":"2023-10-26T01:31:58.915095+0000","last_undegraded":"2023-10-26T01:31:58.915095+0000","last_fullsized":"2023-10-26T01:31:58.915095+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_clean_scrub_stamp":"2023-10-26T01:31:56.263810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:44:09.907410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:59.258537+0000","last_change":"2023-10-26T01:31:57.925298+0000","last_active":"2023-10-26T01:31:59.258537+0000","last_peered":"2023-10-26T01:31:59.258537+0000","last_clean":"2023-10-26T01:31:59.258537+0000","last_became_active":"2023-10-26T01:31:57.924627+0000","last_became_peered":"2023-10-26T01:31:57.924627+0000","last_unstale":"2023-10-26T01:31:59.258537+0000","last_undegraded":"2023-10-26T01:31:59.258537+0000","last_fullsized":"2023-10-26T01:31:59.258537+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:29:41.003416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"34'2","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:59.259434+0000","last_change":"2023-10-26T01:31:57.300830+0000","last_active":"2023-10-26T01:31:59.259434+0000","last_peered":"2023-10-26T01:31:59.259434+0000","last_clean":"2023-10-26T01:31:59.259434+0000","last_became_active":"2023-10-26T01:31:57.300453+0000","last_became_peered":"2023-10-26T01:31:57.300453+0000","last_unstale":"2023-10-26T01:31:59.259434+0000","last_undegraded":"2023-10-26T01:31:59.259434+0000","last_fullsized":"2023-10-26T01:31:59.259434+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_clean_scrub_stamp":"2023-10-26T01:31:56.263810+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:15:18.796380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:59.258539+0000","last_change":"2023-10-26T01:31:57.930968+0000","last_active":"2023-10-26T01:31:59.258539+0000","last_peered":"2023-10-26T01:31:59.258539+0000","last_clean":"2023-10-26T01:31:59.258539+0000","last_became_active":"2023-10-26T01:31:57.930184+0000","last_became_peered":"2023-10-26T01:31:57.930184+0000","last_unstale":"2023-10-26T01:31:59.258539+0000","last_undegraded":"2023-10-26T01:31:59.258539+0000","last_fullsized":"2023-10-26T01:31:59.258539+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:16:42.398443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.920282+0000","last_change":"2023-10-26T01:31:57.303240+0000","last_active":"2023-10-26T01:31:58.920282+0000","last_peered":"2023-10-26T01:31:58.920282+0000","last_clean":"2023-10-26T01:31:58.920282+0000","last_became_active":"2023-10-26T01:31:57.302629+0000","last_became_peered":"2023-10-26T01:31:57.302629+0000","last_unstale":"2023-10-26T01:31:58.920282+0000","last_undegraded":"2023-10-26T01:31:58.920282+0000","last_fullsized":"2023-10-26T01:31:58.920282+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_clean_scrub_stamp":"2023-10-26T01:31:56.263810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:29:39.990303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:59.258727+0000","last_change":"2023-10-26T01:31:57.929718+0000","last_active":"2023-10-26T01:31:59.258727+0000","last_peered":"2023-10-26T01:31:59.258727+0000","last_clean":"2023-10-26T01:31:59.258727+0000","last_became_active":"2023-10-26T01:31:57.929407+0000","last_became_peered":"2023-10-26T01:31:57.929407+0000","last_unstale":"2023-10-26T01:31:59.258727+0000","last_undegraded":"2023-10-26T01:31:59.258727+0000","last_fullsized":"2023-10-26T01:31:59.258727+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:41:29.809025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.914698+0000","last_change":"2023-10-26T01:31:57.302048+0000","last_active":"2023-10-26T01:31:58.914698+0000","last_peered":"2023-10-26T01:31:58.914698+0000","last_clean":"2023-10-26T01:31:58.914698+0000","last_became_active":"2023-10-26T01:31:57.301290+0000","last_became_peered":"2023-10-26T01:31:57.301290+0000","last_unstale":"2023-10-26T01:31:58.914698+0000","last_undegraded":"2023-10-26T01:31:58.914698+0000","last_fullsized":"2023-10-26T01:31:58.914698+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_clean_scrub_stamp":"2023-10-26T01:31:56.263810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:46:19.545323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.917219+0000","last_change":"2023-10-26T01:31:57.922247+0000","last_active":"2023-10-26T01:31:58.917219+0000","last_peered":"2023-10-26T01:31:58.917219+0000","last_clean":"2023-10-26T01:31:58.917219+0000","last_became_active":"2023-10-26T01:31:57.921931+0000","last_became_peered":"2023-10-26T01:31:57.921931+0000","last_unstale":"2023-10-26T01:31:58.917219+0000","last_undegraded":"2023-10-26T01:31:58.917219+0000","last_fullsized":"2023-10-26T01:31:58.917219+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:06:00.266085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:59.260138+0000","last_change":"2023-10-26T01:31:57.316497+0000","last_active":"2023-10-26T01:31:59.260138+0000","last_peered":"2023-10-26T01:31:59.260138+0000","last_clean":"2023-10-26T01:31:59.260138+0000","last_became_active":"2023-10-26T01:31:57.316154+0000","last_became_peered":"2023-10-26T01:31:57.316154+0000","last_unstale":"2023-10-26T01:31:59.260138+0000","last_undegraded":"2023-10-26T01:31:59.260138+0000","last_fullsized":"2023-10-26T01:31:59.260138+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_clean_scrub_stamp":"2023-10-26T01:31:56.263810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:49:04.462479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:59.258706+0000","last_change":"2023-10-26T01:31:57.925339+0000","last_active":"2023-10-26T01:31:59.258706+0000","last_peered":"2023-10-26T01:31:59.258706+0000","last_clean":"2023-10-26T01:31:59.258706+0000","last_became_active":"2023-10-26T01:31:57.924962+0000","last_became_peered":"2023-10-26T01:31:57.924962+0000","last_unstale":"2023-10-26T01:31:59.258706+0000","last_undegraded":"2023-10-26T01:31:59.258706+0000","last_fullsized":"2023-10-26T01:31:59.258706+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:27:22.327089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"34'2","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.919855+0000","last_change":"2023-10-26T01:31:57.307808+0000","last_active":"2023-10-26T01:31:58.919855+0000","last_peered":"2023-10-26T01:31:58.919855+0000","last_clean":"2023-10-26T01:31:58.919855+0000","last_became_active":"2023-10-26T01:31:57.307569+0000","last_became_peered":"2023-10-26T01:31:57.307569+0000","last_unstale":"2023-10-26T01:31:58.919855+0000","last_undegraded":"2023-10-26T01:31:58.919855+0000","last_fullsized":"2023-10-26T01:31:58.919855+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_clean_scrub_stamp":"2023-10-26T01:31:56.263810+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:20:35.297390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:59.260538+0000","last_change":"2023-10-26T01:31:57.920613+0000","last_active":"2023-10-26T01:31:59.260538+0000","last_peered":"2023-10-26T01:31:59.260538+0000","last_clean":"2023-10-26T01:31:59.260538+0000","last_became_active":"2023-10-26T01:31:57.920128+0000","last_became_peered":"2023-10-26T01:31:57.920128+0000","last_unstale":"2023-10-26T01:31:59.260538+0000","last_undegraded":"2023-10-26T01:31:59.260538+0000","last_fullsized":"2023-10-26T01:31:59.260538+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:02:28.206674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:59.259511+0000","last_change":"2023-10-26T01:31:57.300686+0000","last_active":"2023-10-26T01:31:59.259511+0000","last_peered":"2023-10-26T01:31:59.259511+0000","last_clean":"2023-10-26T01:31:59.259511+0000","last_became_active":"2023-10-26T01:31:57.300125+0000","last_became_peered":"2023-10-26T01:31:57.300125+0000","last_unstale":"2023-10-26T01:31:59.259511+0000","last_undegraded":"2023-10-26T01:31:59.259511+0000","last_fullsized":"2023-10-26T01:31:59.259511+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_clean_scrub_stamp":"2023-10-26T01:31:56.263810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:17:23.462894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:59.258765+0000","last_change":"2023-10-26T01:31:57.920668+0000","last_active":"2023-10-26T01:31:59.258765+0000","last_peered":"2023-10-26T01:31:59.258765+0000","last_clean":"2023-10-26T01:31:59.258765+0000","last_became_active":"2023-10-26T01:31:57.920145+0000","last_became_peered":"2023-10-26T01:31:57.920145+0000","last_unstale":"2023-10-26T01:31:59.258765+0000","last_undegraded":"2023-10-26T01:31:59.258765+0000","last_fullsized":"2023-10-26T01:31:59.258765+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:31:13.736811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.920877+0000","last_change":"2023-10-26T01:31:57.307049+0000","last_active":"2023-10-26T01:31:58.920877+0000","last_peered":"2023-10-26T01:31:58.920877+0000","last_clean":"2023-10-26T01:31:58.920877+0000","last_became_active":"2023-10-26T01:31:57.305724+0000","last_became_peered":"2023-10-26T01:31:57.305724+0000","last_unstale":"2023-10-26T01:31:58.920877+0000","last_undegraded":"2023-10-26T01:31:58.920877+0000","last_fullsized":"2023-10-26T01:31:58.920877+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_clean_scrub_stamp":"2023-10-26T01:31:56.263810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:52:32.888332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.921283+0000","last_change":"2023-10-26T01:31:57.919929+0000","last_active":"2023-10-26T01:31:58.921283+0000","last_peered":"2023-10-26T01:31:58.921283+0000","last_clean":"2023-10-26T01:31:58.921283+0000","last_became_active":"2023-10-26T01:31:57.919241+0000","last_became_peered":"2023-10-26T01:31:57.919241+0000","last_unstale":"2023-10-26T01:31:58.921283+0000","last_undegraded":"2023-10-26T01:31:58.921283+0000","last_fullsized":"2023-10-26T01:31:58.921283+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:51:09.298744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:59.259167+0000","last_change":"2023-10-26T01:31:57.313148+0000","last_active":"2023-10-26T01:31:59.259167+0000","last_peered":"2023-10-26T01:31:59.259167+0000","last_clean":"2023-10-26T01:31:59.259167+0000","last_became_active":"2023-10-26T01:31:57.312773+0000","last_became_peered":"2023-10-26T01:31:57.312773+0000","last_unstale":"2023-10-26T01:31:59.259167+0000","last_undegraded":"2023-10-26T01:31:59.259167+0000","last_fullsized":"2023-10-26T01:31:59.259167+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_clean_scrub_stamp":"2023-10-26T01:31:56.263810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:15:56.135260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:59.259003+0000","last_change":"2023-10-26T01:31:57.919722+0000","last_active":"2023-10-26T01:31:59.259003+0000","last_peered":"2023-10-26T01:31:59.259003+0000","last_clean":"2023-10-26T01:31:59.259003+0000","last_became_active":"2023-10-26T01:31:57.919048+0000","last_became_peered":"2023-10-26T01:31:57.919048+0000","last_unstale":"2023-10-26T01:31:59.259003+0000","last_undegraded":"2023-10-26T01:31:59.259003+0000","last_fullsized":"2023-10-26T01:31:59.259003+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:51:09.298744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.920076+0000","last_change":"2023-10-26T01:31:57.313119+0000","last_active":"2023-10-26T01:31:58.920076+0000","last_peered":"2023-10-26T01:31:58.920076+0000","last_clean":"2023-10-26T01:31:58.920076+0000","last_became_active":"2023-10-26T01:31:57.312885+0000","last_became_peered":"2023-10-26T01:31:57.312885+0000","last_unstale":"2023-10-26T01:31:58.920076+0000","last_undegraded":"2023-10-26T01:31:58.920076+0000","last_fullsized":"2023-10-26T01:31:58.920076+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_clean_scrub_stamp":"2023-10-26T01:31:56.263810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:42:37.741558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.912282+0000","last_change":"2023-10-26T01:31:57.928889+0000","last_active":"2023-10-26T01:31:58.912282+0000","last_peered":"2023-10-26T01:31:58.912282+0000","last_clean":"2023-10-26T01:31:58.912282+0000","last_became_active":"2023-10-26T01:31:57.926403+0000","last_became_peered":"2023-10-26T01:31:57.926403+0000","last_unstale":"2023-10-26T01:31:58.912282+0000","last_undegraded":"2023-10-26T01:31:58.912282+0000","last_fullsized":"2023-10-26T01:31:58.912282+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:03:16.325284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.913878+0000","last_change":"2023-10-26T01:31:57.312149+0000","last_active":"2023-10-26T01:31:58.913878+0000","last_peered":"2023-10-26T01:31:58.913878+0000","last_clean":"2023-10-26T01:31:58.913878+0000","last_became_active":"2023-10-26T01:31:57.311845+0000","last_became_peered":"2023-10-26T01:31:57.311845+0000","last_unstale":"2023-10-26T01:31:58.913878+0000","last_undegraded":"2023-10-26T01:31:58.913878+0000","last_fullsized":"2023-10-26T01:31:58.913878+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_clean_scrub_stamp":"2023-10-26T01:31:56.263810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:20:35.297390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.913223+0000","last_change":"2023-10-26T01:31:57.931585+0000","last_active":"2023-10-26T01:31:58.913223+0000","last_peered":"2023-10-26T01:31:58.913223+0000","last_clean":"2023-10-26T01:31:58.913223+0000","last_became_active":"2023-10-26T01:31:57.931235+0000","last_became_peered":"2023-10-26T01:31:57.931235+0000","last_unstale":"2023-10-26T01:31:58.913223+0000","last_undegraded":"2023-10-26T01:31:58.913223+0000","last_fullsized":"2023-10-26T01:31:58.913223+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:15:57.148372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-26T01:31:57.913903+0000","last_change":"2023-10-26T01:31:57.316899+0000","last_active":"2023-10-26T01:31:57.913903+0000","last_peered":"2023-10-26T01:31:57.913903+0000","last_clean":"2023-10-26T01:31:57.913903+0000","last_became_active":"2023-10-26T01:31:57.316690+0000","last_became_peered":"2023-10-26T01:31:57.316690+0000","last_unstale":"2023-10-26T01:31:57.913903+0000","last_undegraded":"2023-10-26T01:31:57.913903+0000","last_fullsized":"2023-10-26T01:31:57.913903+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_clean_scrub_stamp":"2023-10-26T01:31:56.263810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:51:35.971480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.917402+0000","last_change":"2023-10-26T01:31:57.927113+0000","last_active":"2023-10-26T01:31:58.917402+0000","last_peered":"2023-10-26T01:31:58.917402+0000","last_clean":"2023-10-26T01:31:58.917402+0000","last_became_active":"2023-10-26T01:31:57.926727+0000","last_became_peered":"2023-10-26T01:31:57.926727+0000","last_unstale":"2023-10-26T01:31:58.917402+0000","last_undegraded":"2023-10-26T01:31:58.917402+0000","last_fullsized":"2023-10-26T01:31:58.917402+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:57.276922+0000","last_clean_scrub_stamp":"2023-10-26T01:31:57.276922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:02:28.206674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-26T01:31:58.920553+0000","last_change":"2023-10-26T01:31:57.301952+0000","last_active":"2023-10-26T01:31:58.920553+0000","last_peered":"2023-10-26T01:31:58.920553+0000","last_clean":"2023-10-26T01:31:58.920553+0000","last_became_active":"2023-10-26T01:31:57.300614+0000","last_became_peered":"2023-10-26T01:31:57.300614+0000","last_unstale":"2023-10-26T01:31:58.920553+0000","last_undegraded":"2023-10-26T01:31:58.920553+0000","last_fullsized":"2023-10-26T01:31:58.920553+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:31:56.263810+0000","last_clean_scrub_stamp":"2023-10-26T01:31:56.263810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:16:41.385331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":178,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-26T01:31:57.912740+0000","last_change":"2023-10-26T01:30:34.883484+0000","last_active":"2023-10-26T01:31:57.912740+0000","last_peered":"2023-10-26T01:31:57.912740+0000","last_clean":"2023-10-26T01:31:57.912740+0000","last_became_active":"2023-10-26T01:30:34.883071+0000","last_became_peered":"2023-10-26T01:30:34.883071+0000","last_unstale":"2023-10-26T01:31:57.912740+0000","last_undegraded":"2023-10-26T01:31:57.912740+0000","last_fullsized":"2023-10-26T01:31:57.912740+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:32.890096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:32.890096+0000","last_clean_scrub_stamp":"2023-10-26T01:30:32.890096+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:22:57.937891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":14,"seq":60129542165,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":154,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195158,"statfs":{"total":200000000,"available":199841884,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:31:32 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":2.2549999999999999,"5min":2.2549999999999999,"15min":2.2549999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.8200000000000001,"5min":1.8200000000000001,"15min":1.8200000000000001},"last":0.56699999999999995}]},{"osd":1,"last update":"Thu Oct 26 01:31:32 2023","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":0.998,"15min":0.998},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":4.4340000000000002,"5min":4.4340000000000002,"15min":4.4340000000000002},"last":1.329},{"interface":"front","average":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":2.3330000000000002,"5min":2.3330000000000002,"15min":2.3330000000000002},"last":1.288}]},{"osd":2,"last update":"Thu Oct 26 01:31:32 2023","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":4.2800000000000002,"5min":4.2800000000000002,"15min":4.2800000000000002},"last":1.02},{"interface":"front","average":{"1min":1.077,"5min":1.077,"15min":1.077},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":2.302,"5min":2.302,"15min":2.302},"last":1.373}]},{"osd":4,"last update":"Thu Oct 26 01:31:32 2023","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":2.1070000000000002,"5min":2.1070000000000002,"15min":2.1070000000000002},"last":1.0960000000000001},{"interface":"front","average":{"1min":1.4339999999999999,"5min":1.4339999999999999,"15min":1.4339999999999999},"min":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"max":{"1min":4.8250000000000002,"5min":4.8250000000000002,"15min":4.8250000000000002},"last":1.4950000000000001}]},{"osd":5,"last update":"Thu Oct 26 01:31:32 2023","interfaces":[{"interface":"back","average":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":4.9080000000000004,"5min":4.9080000000000004,"15min":4.9080000000000004},"last":1.5820000000000001},{"interface":"front","average":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"min":{"1min":0.88,"5min":0.88,"15min":0.88},"max":{"1min":2.8980000000000001,"5min":2.8980000000000001,"15min":2.8980000000000001},"last":1.177}]},{"osd":6,"last update":"Thu Oct 26 01:31:32 2023","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.321,"15min":1.321},"min":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"max":{"1min":2.8530000000000002,"5min":2.8530000000000002,"15min":2.8530000000000002},"last":1.464},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":2.181,"5min":2.181,"15min":2.181},"last":1.159}]},{"osd":7,"last update":"Thu Oct 26 01:31:32 2023","interfaces":[{"interface":"back","average":{"1min":1.286,"5min":1.286,"15min":1.286},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":2.0680000000000001,"5min":2.0680000000000001,"15min":2.0680000000000001},"last":1.2090000000000001},{"interface":"front","average":{"1min":1.45,"5min":1.45,"15min":1.45},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":4.8730000000000002,"5min":4.8730000000000002,"15min":4.8730000000000002},"last":1.6419999999999999}]}]},{"osd":7,"up_from":13,"seq":55834574868,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":729,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194583,"statfs":{"total":200000000,"available":199253060,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:31:31 2023","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.7490000000000001,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":1.246},{"interface":"front","average":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.514,"5min":1.514,"15min":1.514},"last":1.4330000000000001}]},{"osd":1,"last update":"Thu Oct 26 01:31:31 2023","interfaces":[{"interface":"back","average":{"1min":1.034,"5min":1.034,"15min":1.034},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.8520000000000001,"5min":1.8520000000000001,"15min":1.8520000000000001},"last":1.0129999999999999},{"interface":"front","average":{"1min":1.175,"5min":1.175,"15min":1.175},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.7689999999999999,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":1.123}]},{"osd":2,"last update":"Thu Oct 26 01:31:31 2023","interfaces":[{"interface":"back","average":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.5649999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.94899999999999995},{"interface":"front","average":{"1min":1.206,"5min":1.206,"15min":1.206},"min":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":1.766,"5min":1.766,"15min":1.766},"last":1.587}]},{"osd":3,"last update":"Thu Oct 26 01:31:36 2023","interfaces":[{"interface":"back","average":{"1min":1.177,"5min":1.177,"15min":1.177},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":1.169},{"interface":"front","average":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"min":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"max":{"1min":1.6180000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":1.534}]},{"osd":4,"last update":"Thu Oct 26 01:31:31 2023","interfaces":[{"interface":"back","average":{"1min":1.345,"5min":1.345,"15min":1.345},"min":{"1min":1.014,"5min":1.014,"15min":1.014},"max":{"1min":1.671,"5min":1.671,"15min":1.671},"last":1.0680000000000001},{"interface":"front","average":{"1min":1.379,"5min":1.379,"15min":1.379},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":2.105,"5min":2.105,"15min":2.105},"last":1.472}]},{"osd":5,"last update":"Thu Oct 26 01:31:31 2023","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.393,"15min":1.393},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.873,"5min":1.873,"15min":1.873},"last":1.288},{"interface":"front","average":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"min":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"max":{"1min":1.7090000000000001,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":1.3360000000000001}]},{"osd":6,"last update":"Thu Oct 26 01:31:31 2023","interfaces":[{"interface":"back","average":{"1min":2.0139999999999998,"5min":2.0139999999999998,"15min":2.0139999999999998},"min":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":11.577999999999999,"5min":11.577999999999999,"15min":11.577999999999999},"last":1.3640000000000001},{"interface":"front","average":{"1min":2.0019999999999998,"5min":2.0019999999999998,"15min":2.0019999999999998},"min":{"1min":1.159,"5min":1.159,"15min":1.159},"max":{"1min":10.704000000000001,"5min":10.704000000000001,"15min":10.704000000000001},"last":1.403}]}]},{"osd":6,"up_from":13,"seq":55834574868,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":152,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195159,"statfs":{"total":200000000,"available":199843516,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:31:32 2023","interfaces":[{"interface":"back","average":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"min":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"max":{"1min":1.8839999999999999,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.843,"5min":1.843,"15min":1.843},"last":1.0329999999999999}]},{"osd":1,"last update":"Thu Oct 26 01:31:32 2023","interfaces":[{"interface":"back","average":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.605,"5min":1.605,"15min":1.605},"last":1.1020000000000001},{"interface":"front","average":{"1min":1.226,"5min":1.226,"15min":1.226},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":2.085,"5min":2.085,"15min":2.085},"last":1.5209999999999999}]},{"osd":2,"last update":"Thu Oct 26 01:31:32 2023","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.052,"15min":1.052},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":1.6659999999999999,"5min":1.6659999999999999,"15min":1.6659999999999999},"last":0.78800000000000003},{"interface":"front","average":{"1min":1.248,"5min":1.248,"15min":1.248},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.7949999999999999,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":0.98999999999999999}]},{"osd":3,"last update":"Thu Oct 26 01:31:36 2023","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.157,"15min":1.157},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.4019999999999999,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":1.3300000000000001},{"interface":"front","average":{"1min":1.321,"5min":1.321,"15min":1.321},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.71,"5min":1.71,"15min":1.71},"last":1.21}]},{"osd":4,"last update":"Thu Oct 26 01:31:32 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.204,"15min":1.204},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":2.089,"5min":2.089,"15min":2.089},"last":1.153},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":2.4129999999999998,"5min":2.4129999999999998,"15min":2.4129999999999998},"last":1.4590000000000001}]},{"osd":5,"last update":"Thu Oct 26 01:31:32 2023","interfaces":[{"interface":"back","average":{"1min":1.391,"5min":1.391,"15min":1.391},"min":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":2.1640000000000001,"5min":2.1640000000000001,"15min":2.1640000000000001},"last":1.925},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"min":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"max":{"1min":1.7490000000000001,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":1.778}]},{"osd":7,"last update":"Thu Oct 26 01:31:32 2023","interfaces":[{"interface":"back","average":{"1min":1.355,"5min":1.355,"15min":1.355},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":2.012,"5min":2.012,"15min":2.012},"last":1.256},{"interface":"front","average":{"1min":1.4299999999999999,"5min":1.4299999999999999,"15min":1.4299999999999999},"min":{"1min":0.875,"5min":0.875,"15min":0.875},"max":{"1min":2.3239999999999998,"5min":2.3239999999999998,"15min":2.3239999999999998},"last":1.5680000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574868,"num_pgs":17,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":153,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195159,"statfs":{"total":200000000,"available":199842846,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:31:32 2023","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.7230000000000001,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":0.66200000000000003},{"interface":"front","average":{"1min":1.131,"5min":1.131,"15min":1.131},"min":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":1.52,"5min":1.52,"15min":1.52},"last":0.76600000000000001}]},{"osd":1,"last update":"Thu Oct 26 01:31:32 2023","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.722,"5min":1.722,"15min":1.722},"last":0.83899999999999997},{"interface":"front","average":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.7669999999999999,"5min":1.7669999999999999,"15min":1.7669999999999999},"last":1.1200000000000001}]},{"osd":2,"last update":"Thu Oct 26 01:31:32 2023","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.141,"15min":1.141},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.629,"5min":1.629,"15min":1.629},"last":1.2629999999999999},{"interface":"front","average":{"1min":1.161,"5min":1.161,"15min":1.161},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.5489999999999999,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":1.0720000000000001}]},{"osd":3,"last update":"Thu Oct 26 01:31:32 2023","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"min":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":1.835,"5min":1.835,"15min":1.835},"last":1.165},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":2.9249999999999998,"5min":2.9249999999999998,"15min":2.9249999999999998},"last":1.024}]},{"osd":4,"last update":"Thu Oct 26 01:31:32 2023","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.597,"5min":1.597,"15min":1.597},"last":1.474},{"interface":"front","average":{"1min":1.306,"5min":1.306,"15min":1.306},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.669,"5min":1.669,"15min":1.669},"last":1.448}]},{"osd":6,"last update":"Thu Oct 26 01:31:32 2023","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.262,"15min":1.262},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.72,"5min":1.72,"15min":1.72},"last":1.2430000000000001},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.8560000000000001,"5min":1.8560000000000001,"15min":1.8560000000000001},"last":1.319}]},{"osd":7,"last update":"Thu Oct 26 01:31:32 2023","interfaces":[{"interface":"back","average":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.7689999999999999,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":1.516},{"interface":"front","average":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.8140000000000001,"5min":1.8140000000000001,"15min":1.8140000000000001},"last":1.421}]}]},{"osd":4,"up_from":13,"seq":55834574868,"num_pgs":29,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":152,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195159,"statfs":{"total":200000000,"available":199843414,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.47,"5min":1.47,"15min":1.47},"last":1.46},{"interface":"front","average":{"1min":1.002,"5min":1.002,"15min":1.002},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.4870000000000001,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":0.621}]},{"osd":1,"last update":"Thu Oct 26 01:31:30 2023","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"min":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"max":{"1min":1.635,"5min":1.635,"15min":1.635},"last":1.163},{"interface":"front","average":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.724,"5min":1.724,"15min":1.724},"last":1.786}]},{"osd":2,"last update":"Thu Oct 26 01:31:30 2023","interfaces":[{"interface":"back","average":{"1min":1.169,"5min":1.169,"15min":1.169},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.661,"5min":1.661,"15min":1.661},"last":1.268},{"interface":"front","average":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.53,"5min":1.53,"15min":1.53},"last":0.80700000000000005}]},{"osd":3,"last update":"Thu Oct 26 01:31:32 2023","interfaces":[{"interface":"back","average":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.845,"5min":1.845,"15min":1.845},"last":1.8480000000000001},{"interface":"front","average":{"1min":1.131,"5min":1.131,"15min":1.131},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":1.5049999999999999,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":1.0309999999999999}]},{"osd":5,"last update":"Thu Oct 26 01:31:30 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.304,"15min":1.304},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.9570000000000001,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":1.907},{"interface":"front","average":{"1min":1.325,"5min":1.325,"15min":1.325},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.8120000000000001,"5min":1.8120000000000001,"15min":1.8120000000000001},"last":1.3759999999999999}]},{"osd":6,"last update":"Thu Oct 26 01:31:30 2023","interfaces":[{"interface":"back","average":{"1min":1.2909999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":2.218,"5min":2.218,"15min":2.218},"last":1.212},{"interface":"front","average":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":3.3500000000000001,"5min":3.3500000000000001,"15min":3.3500000000000001},"last":2.1320000000000001}]},{"osd":7,"last update":"Thu Oct 26 01:31:30 2023","interfaces":[{"interface":"back","average":{"1min":1.478,"5min":1.478,"15min":1.478},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":2.2730000000000001,"5min":2.2730000000000001,"15min":2.2730000000000001},"last":1.5089999999999999},{"interface":"front","average":{"1min":1.421,"5min":1.421,"15min":1.421},"min":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":2.1850000000000001,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":1.3080000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574868,"num_pgs":17,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":152,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195159,"statfs":{"total":200000000,"available":199843448,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.98999999999999999},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.5900000000000001,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":1.073}]},{"osd":1,"last update":"Thu Oct 26 01:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":1.171}]},{"osd":3,"last update":"Thu Oct 26 01:31:33 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.623,"5min":1.623,"15min":1.623},"last":1.381},{"interface":"front","average":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":1.0309999999999999}]},{"osd":4,"last update":"Thu Oct 26 01:31:31 2023","interfaces":[{"interface":"back","average":{"1min":1.22,"5min":1.22,"15min":1.22},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":2.2040000000000002,"5min":2.2040000000000002,"15min":2.2040000000000002},"last":1.546},{"interface":"front","average":{"1min":1.212,"5min":1.212,"15min":1.212},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":1.702,"5min":1.702,"15min":1.702},"last":1.502}]},{"osd":5,"last update":"Thu Oct 26 01:31:31 2023","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.252,"15min":1.252},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.7869999999999999,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":1.222},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":2.3029999999999999,"5min":2.3029999999999999,"15min":2.3029999999999999},"last":1.585}]},{"osd":6,"last update":"Thu Oct 26 01:31:31 2023","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.29,"15min":1.29},"min":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":2.9649999999999999,"5min":2.9649999999999999,"15min":2.9649999999999999},"last":1.6779999999999999},{"interface":"front","average":{"1min":1.343,"5min":1.343,"15min":1.343},"min":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"max":{"1min":1.7470000000000001,"5min":1.7470000000000001,"15min":1.7470000000000001},"last":1.6379999999999999}]},{"osd":7,"last update":"Thu Oct 26 01:31:31 2023","interfaces":[{"interface":"back","average":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":2.2450000000000001,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":1.4319999999999999},{"interface":"front","average":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"min":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"max":{"1min":2.7949999999999999,"5min":2.7949999999999999,"15min":2.7949999999999999},"last":1.74}]}]},{"osd":1,"up_from":13,"seq":55834574868,"num_pgs":32,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":154,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195158,"statfs":{"total":200000000,"available":199841884,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:31:32 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.60699999999999998}]},{"osd":2,"last update":"Thu Oct 26 01:31:32 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":1.018,"15min":1.018},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":2.4830000000000001,"5min":2.4830000000000001,"15min":2.4830000000000001},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.84599999999999997}]},{"osd":3,"last update":"Thu Oct 26 01:31:35 2023","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.6060000000000001,"5min":1.6060000000000001,"15min":1.6060000000000001},"last":0.754},{"interface":"front","average":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.6559999999999999,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":0.58099999999999996}]},{"osd":4,"last update":"Thu Oct 26 01:31:32 2023","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":2.0659999999999998,"5min":2.0659999999999998,"15min":2.0659999999999998},"last":0.98799999999999999},{"interface":"front","average":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.47,"5min":1.47,"15min":1.47},"last":0.73599999999999999}]},{"osd":5,"last update":"Thu Oct 26 01:31:32 2023","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.218,"15min":1.218},"min":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":1.6619999999999999,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":1.1220000000000001},{"interface":"front","average":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.3280000000000001},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":2.2200000000000002,"5min":2.2200000000000002,"15min":2.2200000000000002},"last":1.1459999999999999}]},{"osd":6,"last update":"Thu Oct 26 01:31:32 2023","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":2.3780000000000001,"5min":2.3780000000000001,"15min":2.3780000000000001},"last":0.82099999999999995},{"interface":"front","average":{"1min":1.226,"5min":1.226,"15min":1.226},"min":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"max":{"1min":1.984,"5min":1.984,"15min":1.984},"last":0.89600000000000002}]},{"osd":7,"last update":"Thu Oct 26 01:31:32 2023","interfaces":[{"interface":"back","average":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":2.6930000000000001,"5min":2.6930000000000001,"15min":2.6930000000000001},"last":1.169},{"interface":"front","average":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.7909999999999999,"5min":1.7909999999999999,"15min":1.7909999999999999},"last":0.91900000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574868,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":729,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194583,"statfs":{"total":200000000,"available":199253150,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Oct 26 01:31:33 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":4.5679999999999996,"5min":4.5679999999999996,"15min":4.5679999999999996},"last":0.74299999999999999},{"interface":"front","average":{"1min":1.046,"5min":1.046,"15min":1.046},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":4.8470000000000004,"5min":4.8470000000000004,"15min":4.8470000000000004},"last":0.53700000000000003}]},{"osd":2,"last update":"Thu Oct 26 01:31:33 2023","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":5.5599999999999996,"5min":5.5599999999999996,"15min":5.5599999999999996},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":4.9459999999999997,"5min":4.9459999999999997,"15min":4.9459999999999997},"last":0.82999999999999996}]},{"osd":3,"last update":"Thu Oct 26 01:31:33 2023","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":4.9800000000000004,"5min":4.9800000000000004,"15min":4.9800000000000004},"last":0.60599999999999998},{"interface":"front","average":{"1min":1.175,"5min":1.175,"15min":1.175},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":5.016,"5min":5.016,"15min":5.016},"last":0.70299999999999996}]},{"osd":4,"last update":"Thu Oct 26 01:31:33 2023","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":5.1879999999999997,"5min":5.1879999999999997,"15min":5.1879999999999997},"last":1.032},{"interface":"front","average":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":5.5229999999999997,"5min":5.5229999999999997,"15min":5.5229999999999997},"last":0.79100000000000004}]},{"osd":5,"last update":"Thu Oct 26 01:31:33 2023","interfaces":[{"interface":"back","average":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":5.6360000000000001,"5min":5.6360000000000001,"15min":5.6360000000000001},"last":1.1950000000000001},{"interface":"front","average":{"1min":1.3720000000000001,"5min":1.3720000000000001,"15min":1.3720000000000001},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":5.3399999999999999,"5min":5.3399999999999999,"15min":5.3399999999999999},"last":0.98599999999999999}]},{"osd":6,"last update":"Thu Oct 26 01:31:33 2023","interfaces":[{"interface":"back","average":{"1min":1.371,"5min":1.371,"15min":1.371},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":5.3070000000000004,"5min":5.3070000000000004,"15min":5.3070000000000004},"last":1.222},{"interface":"front","average":{"1min":1.397,"5min":1.397,"15min":1.397},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":5.5919999999999996,"5min":5.5919999999999996,"15min":5.5919999999999996},"last":1.075}]},{"osd":7,"last update":"Thu Oct 26 01:31:33 2023","interfaces":[{"interface":"back","average":{"1min":1.51,"5min":1.51,"15min":1.51},"min":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":5.6689999999999996,"5min":5.6689999999999996,"15min":5.6689999999999996},"last":1.1060000000000001},{"interface":"front","average":{"1min":1.46,"5min":1.46,"15min":1.46},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":5.3719999999999999,"5min":5.3719999999999999,"15min":5.3719999999999999},"last":0.94299999999999995}]}]}],"pool_statfs":[]}} 2023-10-26T01:32:03.322 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-26T01:32:03.323 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-26T01:32:03.591 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.589+0000 7f88b1039640 1 -- 172.21.15.169:0/2113745317 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88ac159820 msgr2=0x7f88ac0ac200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:03.591 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.589+0000 7f88b1039640 1 --2- 172.21.15.169:0/2113745317 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88ac159820 0x7f88ac0ac200 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f8894005fd0 tx=0x7f889402bd80 comp rx=0 tx=0).stop 2023-10-26T01:32:03.591 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.589+0000 7f88b1039640 1 -- 172.21.15.169:0/2113745317 shutdown_connections 2023-10-26T01:32:03.592 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.589+0000 7f88b1039640 1 --2- 172.21.15.169:0/2113745317 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f88ac0abdb0 0x7f88ac0ac740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:03.592 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.589+0000 7f88b1039640 1 --2- 172.21.15.169:0/2113745317 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88ac159820 0x7f88ac0ac200 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:03.592 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.589+0000 7f88b1039640 1 -- 172.21.15.169:0/2113745317 >> 172.21.15.169:0/2113745317 conn(0x7f88ac0aefe0 msgr2=0x7f88ac0af3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:03.592 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.589+0000 7f88b1039640 1 -- 172.21.15.169:0/2113745317 shutdown_connections 2023-10-26T01:32:03.592 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.589+0000 7f88b1039640 1 -- 172.21.15.169:0/2113745317 wait complete. 2023-10-26T01:32:03.594 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.591+0000 7f88b1039640 1 Processor -- start 2023-10-26T01:32:03.595 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.591+0000 7f88b1039640 1 -- start start 2023-10-26T01:32:03.595 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.592+0000 7f88b1039640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f88ac0abdb0 0x7f88ac15bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:03.595 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.592+0000 7f88b1039640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f88ac159440 0x7f88ac155900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:03.595 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.592+0000 7f88b1039640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88ac1513c0 0x7f88ac09a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:03.595 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.592+0000 7f88b1039640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88ac155e40 con 0x7f88ac1513c0 2023-10-26T01:32:03.596 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.592+0000 7f88b1039640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88ac06f160 con 0x7f88ac0abdb0 2023-10-26T01:32:03.596 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.592+0000 7f88b1039640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88ac156080 con 0x7f88ac159440 2023-10-26T01:32:03.596 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.592+0000 7f88ab59e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88ac1513c0 0x7f88ac09a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:03.596 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.592+0000 7f88ab59e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88ac1513c0 0x7f88ac09a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:44486/0 (socket says 172.21.15.169:44486) 2023-10-26T01:32:03.596 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.592+0000 7f88ab59e640 1 -- 172.21.15.169:0/2197330349 learned_addr learned my addr 172.21.15.169:0/2197330349 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:32:03.597 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.593+0000 7f88a3fff640 1 --2- 172.21.15.169:0/2197330349 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f88ac159440 0x7f88ac155900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:03.597 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.593+0000 7f88aad9d640 1 --2- 172.21.15.169:0/2197330349 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f88ac0abdb0 0x7f88ac15bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:03.597 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.593+0000 7f88ab59e640 1 -- 172.21.15.169:0/2197330349 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f88ac159440 msgr2=0x7f88ac155900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:03.597 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.593+0000 7f88ab59e640 1 --2- 172.21.15.169:0/2197330349 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f88ac159440 0x7f88ac155900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:03.597 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.593+0000 7f88ab59e640 1 -- 172.21.15.169:0/2197330349 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f88ac0abdb0 msgr2=0x7f88ac15bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:03.598 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.593+0000 7f88ab59e640 1 --2- 172.21.15.169:0/2197330349 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f88ac0abdb0 0x7f88ac15bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:03.598 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.593+0000 7f88ab59e640 1 -- 172.21.15.169:0/2197330349 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8894009560 con 0x7f88ac1513c0 2023-10-26T01:32:03.598 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.593+0000 7f88a3fff640 1 --2- 172.21.15.169:0/2197330349 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f88ac159440 0x7f88ac155900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:32:03.598 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.593+0000 7f88aad9d640 1 --2- 172.21.15.169:0/2197330349 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f88ac0abdb0 0x7f88ac15bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:32:03.599 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.593+0000 7f88ab59e640 1 --2- 172.21.15.169:0/2197330349 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88ac1513c0 0x7f88ac09a8d0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f88a400ccb0 tx=0x7f88a400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:03.600 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.593+0000 7f88a37fe640 1 -- 172.21.15.169:0/2197330349 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a400f450 con 0x7f88ac1513c0 2023-10-26T01:32:03.600 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.593+0000 7f88a37fe640 1 -- 172.21.15.169:0/2197330349 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f88a400f6c0 con 0x7f88ac1513c0 2023-10-26T01:32:03.600 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.593+0000 7f88b1039640 1 -- 172.21.15.169:0/2197330349 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88ac156360 con 0x7f88ac1513c0 2023-10-26T01:32:03.601 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.593+0000 7f88a37fe640 1 -- 172.21.15.169:0/2197330349 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a400fb10 con 0x7f88ac1513c0 2023-10-26T01:32:03.601 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.593+0000 7f88b1039640 1 -- 172.21.15.169:0/2197330349 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88ac156640 con 0x7f88ac1513c0 2023-10-26T01:32:03.601 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.595+0000 7f88b1039640 1 -- 172.21.15.169:0/2197330349 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88ac066af0 con 0x7f88ac1513c0 2023-10-26T01:32:03.601 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.595+0000 7f88a37fe640 1 -- 172.21.15.169:0/2197330349 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f88a400fc70 con 0x7f88ac1513c0 2023-10-26T01:32:03.602 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.596+0000 7f88a37fe640 1 --2- 172.21.15.169:0/2197330349 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f888c06ae40 0x7f888c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:03.602 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.596+0000 7f88a37fe640 1 -- 172.21.15.169:0/2197330349 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f88a408a1b0 con 0x7f88ac1513c0 2023-10-26T01:32:03.602 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.597+0000 7f88aad9d640 1 --2- 172.21.15.169:0/2197330349 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f888c06ae40 0x7f888c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:03.602 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.598+0000 7f88aad9d640 1 --2- 172.21.15.169:0/2197330349 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f888c06ae40 0x7f888c06d300 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f889c007fd0 tx=0x7f889c009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:03.603 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.601+0000 7f88a37fe640 1 -- 172.21.15.169:0/2197330349 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f88a4055250 con 0x7f88ac1513c0 2023-10-26T01:32:03.843 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.841+0000 7f88b1039640 1 -- 172.21.15.169:0/2197330349 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f88ac04da70 con 0x7f88ac1513c0 2023-10-26T01:32:03.847 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.846+0000 7f88a37fe640 1 -- 172.21.15.169:0/2197330349 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) v1 ==== 190+0+147 (secure 0 0 0) 0x7f88a4058870 con 0x7f88ac1513c0 2023-10-26T01:32:03.848 INFO:teuthology.orchestra.run.smithi169.stdout:[client.0] 2023-10-26T01:32:03.848 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQAgwTllKC7gOhAAPsvxGcuUA8/p8xqbUgGteA== 2023-10-26T01:32:03.848 INFO:teuthology.orchestra.run.smithi169.stdout: caps mds = "allow" 2023-10-26T01:32:03.849 INFO:teuthology.orchestra.run.smithi169.stdout: caps mon = "allow r" 2023-10-26T01:32:03.849 INFO:teuthology.orchestra.run.smithi169.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-10-26T01:32:03.849 INFO:teuthology.orchestra.run.smithi169.stderr:updated caps for client.0 2023-10-26T01:32:03.851 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.849+0000 7f88b1039640 1 -- 172.21.15.169:0/2197330349 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f888c06ae40 msgr2=0x7f888c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:03.851 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.849+0000 7f88b1039640 1 --2- 172.21.15.169:0/2197330349 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f888c06ae40 0x7f888c06d300 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f889c007fd0 tx=0x7f889c009040 comp rx=0 tx=0).stop 2023-10-26T01:32:03.851 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.849+0000 7f88b1039640 1 -- 172.21.15.169:0/2197330349 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88ac1513c0 msgr2=0x7f88ac09a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:03.851 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.849+0000 7f88b1039640 1 --2- 172.21.15.169:0/2197330349 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88ac1513c0 0x7f88ac09a8d0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f88a400ccb0 tx=0x7f88a400a620 comp rx=0 tx=0).stop 2023-10-26T01:32:03.852 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.849+0000 7f88b1039640 1 -- 172.21.15.169:0/2197330349 shutdown_connections 2023-10-26T01:32:03.852 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.849+0000 7f88b1039640 1 --2- 172.21.15.169:0/2197330349 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f888c06ae40 0x7f888c06d300 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:03.852 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.849+0000 7f88b1039640 1 --2- 172.21.15.169:0/2197330349 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88ac1513c0 0x7f88ac09a8d0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:03.852 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.849+0000 7f88b1039640 1 --2- 172.21.15.169:0/2197330349 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f88ac159440 0x7f88ac155900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:03.852 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.849+0000 7f88b1039640 1 --2- 172.21.15.169:0/2197330349 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f88ac0abdb0 0x7f88ac15bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:03.853 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.849+0000 7f88b1039640 1 -- 172.21.15.169:0/2197330349 >> 172.21.15.169:0/2197330349 conn(0x7f88ac0aefe0 msgr2=0x7f88ac154660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:03.853 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.849+0000 7f88b1039640 1 -- 172.21.15.169:0/2197330349 shutdown_connections 2023-10-26T01:32:03.853 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:03.849+0000 7f88b1039640 1 -- 172.21.15.169:0/2197330349 wait complete. 2023-10-26T01:32:03.868 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T01:32:04.084 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.082+0000 7f1649bd7640 1 -- 172.21.15.169:0/604904962 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f16440ac930 msgr2=0x7f16440cd6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:04.084 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.082+0000 7f1649bd7640 1 --2- 172.21.15.169:0/604904962 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f16440ac930 0x7f16440cd6f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f162c005fd0 tx=0x7f162c02bd10 comp rx=0 tx=0).stop 2023-10-26T01:32:04.084 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.082+0000 7f1649bd7640 1 -- 172.21.15.169:0/604904962 shutdown_connections 2023-10-26T01:32:04.085 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.082+0000 7f1649bd7640 1 --2- 172.21.15.169:0/604904962 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f16440cdc30 0x7f16440cbac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:04.085 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.082+0000 7f1649bd7640 1 --2- 172.21.15.169:0/604904962 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f16440ac930 0x7f16440cd6f0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:04.085 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.082+0000 7f1649bd7640 1 -- 172.21.15.169:0/604904962 >> 172.21.15.169:0/604904962 conn(0x7f1644069930 msgr2=0x7f1644069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:04.085 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.082+0000 7f1649bd7640 1 -- 172.21.15.169:0/604904962 shutdown_connections 2023-10-26T01:32:04.085 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.082+0000 7f1649bd7640 1 -- 172.21.15.169:0/604904962 wait complete. 2023-10-26T01:32:04.086 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.084+0000 7f1649bd7640 1 Processor -- start 2023-10-26T01:32:04.086 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.085+0000 7f1649bd7640 1 -- start start 2023-10-26T01:32:04.088 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.086+0000 7f1649bd7640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f16440ac930 0x7f164406f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:04.088 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.086+0000 7f1649bd7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f16440cdc30 0x7f1644157b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:04.088 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.086+0000 7f1649bd7640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16441592d0 0x7f164415ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:04.088 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.086+0000 7f1649bd7640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f164409b400 con 0x7f16441592d0 2023-10-26T01:32:04.088 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.086+0000 7f1649bd7640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1644070090 con 0x7f16440cdc30 2023-10-26T01:32:04.089 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.086+0000 7f1649bd7640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1644070370 con 0x7f16440ac930 2023-10-26T01:32:04.089 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.086+0000 7f1643fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16441592d0 0x7f164415ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:04.089 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.086+0000 7f1643fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16441592d0 0x7f164415ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:44422/0 (socket says 172.21.15.169:44422) 2023-10-26T01:32:04.089 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.086+0000 7f1643fff640 1 -- 172.21.15.169:0/2521104639 learned_addr learned my addr 172.21.15.169:0/2521104639 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:32:04.090 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.086+0000 7f16437fe640 1 --2- 172.21.15.169:0/2521104639 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f16440ac930 0x7f164406f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:04.091 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.087+0000 7f163bfff640 1 --2- 172.21.15.169:0/2521104639 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f16440cdc30 0x7f1644157b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:04.091 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.087+0000 7f1643fff640 1 -- 172.21.15.169:0/2521104639 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f16440ac930 msgr2=0x7f164406f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:04.091 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.087+0000 7f1643fff640 1 --2- 172.21.15.169:0/2521104639 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f16440ac930 0x7f164406f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:04.091 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.087+0000 7f1643fff640 1 -- 172.21.15.169:0/2521104639 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f16440cdc30 msgr2=0x7f1644157b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:04.091 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.087+0000 7f1643fff640 1 --2- 172.21.15.169:0/2521104639 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f16440cdc30 0x7f1644157b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:04.092 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.087+0000 7f1643fff640 1 -- 172.21.15.169:0/2521104639 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f163c009840 con 0x7f16441592d0 2023-10-26T01:32:04.092 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.087+0000 7f16437fe640 1 --2- 172.21.15.169:0/2521104639 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f16440ac930 0x7f164406f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T01:32:04.092 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.087+0000 7f1643fff640 1 --2- 172.21.15.169:0/2521104639 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16441592d0 0x7f164415ba70 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f163c00efd0 tx=0x7f163c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:04.092 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.087+0000 7f1640ff9640 1 -- 172.21.15.169:0/2521104639 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f163c009b40 con 0x7f16441592d0 2023-10-26T01:32:04.092 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.088+0000 7f1640ff9640 1 -- 172.21.15.169:0/2521104639 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f163c003c60 con 0x7f16441592d0 2023-10-26T01:32:04.093 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.088+0000 7f1640ff9640 1 -- 172.21.15.169:0/2521104639 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f163c004020 con 0x7f16441592d0 2023-10-26T01:32:04.093 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.088+0000 7f1649bd7640 1 -- 172.21.15.169:0/2521104639 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f162c009560 con 0x7f16441592d0 2023-10-26T01:32:04.093 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.088+0000 7f1649bd7640 1 -- 172.21.15.169:0/2521104639 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16441f0cb0 con 0x7f16441592d0 2023-10-26T01:32:04.093 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.089+0000 7f1649bd7640 1 -- 172.21.15.169:0/2521104639 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1644070890 con 0x7f16441592d0 2023-10-26T01:32:04.094 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.090+0000 7f1640ff9640 1 -- 172.21.15.169:0/2521104639 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f163c004180 con 0x7f16441592d0 2023-10-26T01:32:04.094 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.091+0000 7f1640ff9640 1 --2- 172.21.15.169:0/2521104639 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f162406ae40 0x7f162406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:04.095 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.091+0000 7f1640ff9640 1 -- 172.21.15.169:0/2521104639 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f163c014070 con 0x7f16441592d0 2023-10-26T01:32:04.095 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.091+0000 7f16437fe640 1 --2- 172.21.15.169:0/2521104639 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f162406ae40 0x7f162406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:04.095 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.093+0000 7f16437fe640 1 --2- 172.21.15.169:0/2521104639 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f162406ae40 0x7f162406d300 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f1634001660 tx=0x7f1634009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:04.096 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.095+0000 7f1640ff9640 1 -- 172.21.15.169:0/2521104639 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1644070890 con 0x7f16441592d0 2023-10-26T01:32:04.260 INFO:tasks.ceph.mds.c.smithi169.stderr:2023-10-26T01:32:04.259+0000 7f26119d3640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-26T01:32:04.322 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.320+0000 7f1649bd7640 1 -- 172.21.15.169:0/2521104639 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f16441f0b10 con 0x7f16441592d0 2023-10-26T01:32:04.323 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.321+0000 7f1640ff9640 1 -- 172.21.15.169:0/2521104639 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4913 (secure 0 0 0) 0x7f16441f0b10 con 0x7f16441592d0 2023-10-26T01:32:04.323 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:32:04.323 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4640,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/61096113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":61096113},{"type":"v1","addr":"172.21.15.170:6836","nonce":61096113}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4762,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2825046244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2825046244},{"type":"v1","addr":"172.21.15.169:6833","nonce":2825046244}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4777,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/258866976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":258866976},{"type":"v1","addr":"172.21.15.170:6837","nonce":258866976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T01:31:57.759251+0000","modified":"2023-10-26T01:32:00.831213+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5196},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5196":{"gid":5196,"name":"c","rank":0,"incarnation":11,"state":"up:active","state_seq":4,"addr":"172.21.15.169:6835/871978528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":871978528},{"type":"v1","addr":"172.21.15.169:6835","nonce":871978528}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-26T01:32:04.324 INFO:teuthology.orchestra.run.smithi169.stderr:dumped fsmap epoch 15 2023-10-26T01:32:04.325 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.323+0000 7f1649bd7640 1 -- 172.21.15.169:0/2521104639 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f162406ae40 msgr2=0x7f162406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:04.326 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.323+0000 7f1649bd7640 1 --2- 172.21.15.169:0/2521104639 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f162406ae40 0x7f162406d300 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f1634001660 tx=0x7f1634009040 comp rx=0 tx=0).stop 2023-10-26T01:32:04.326 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.323+0000 7f1649bd7640 1 -- 172.21.15.169:0/2521104639 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16441592d0 msgr2=0x7f164415ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:04.326 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.323+0000 7f1649bd7640 1 --2- 172.21.15.169:0/2521104639 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16441592d0 0x7f164415ba70 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f163c00efd0 tx=0x7f163c00c6a0 comp rx=0 tx=0).stop 2023-10-26T01:32:04.326 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.324+0000 7f1649bd7640 1 -- 172.21.15.169:0/2521104639 shutdown_connections 2023-10-26T01:32:04.326 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.324+0000 7f1649bd7640 1 --2- 172.21.15.169:0/2521104639 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f162406ae40 0x7f162406d300 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:04.327 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.324+0000 7f1649bd7640 1 --2- 172.21.15.169:0/2521104639 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16441592d0 0x7f164415ba70 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:04.327 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.324+0000 7f1649bd7640 1 --2- 172.21.15.169:0/2521104639 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f16440cdc30 0x7f1644157b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:04.327 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.324+0000 7f1649bd7640 1 --2- 172.21.15.169:0/2521104639 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f16440ac930 0x7f164406f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:04.327 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.324+0000 7f1649bd7640 1 -- 172.21.15.169:0/2521104639 >> 172.21.15.169:0/2521104639 conn(0x7f1644069930 msgr2=0x7f1644069d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:04.327 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.324+0000 7f1649bd7640 1 -- 172.21.15.169:0/2521104639 shutdown_connections 2023-10-26T01:32:04.328 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.324+0000 7f1649bd7640 1 -- 172.21.15.169:0/2521104639 wait complete. 2023-10-26T01:32:04.420 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T01:32:04.429 DEBUG:teuthology.orchestra.run.smithi170:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T01:32:04.748 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 15, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-26T01:31:57.759251+0000', 'modified': '2023-10-26T01:32:00.831213+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5196}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5196': {'gid': 5196, 'name': 'c', 'rank': 0, 'incarnation': 11, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.169:6835/871978528', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.169:6834', 'nonce': 871978528}, {'type': 'v1', 'addr': '172.21.15.169:6835', 'nonce': 871978528}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-26T01:32:04.749 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-26T01:32:04.750 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-26T01:32:04.981 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.979+0000 7fcb6f925640 1 -- 172.21.15.169:0/2364701482 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb681597c0 msgr2=0x7fcb680ac200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:04.981 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.979+0000 7fcb6f925640 1 --2- 172.21.15.169:0/2364701482 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb681597c0 0x7fcb680ac200 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c005fd0 tx=0x7fcb5c02bd90 comp rx=0 tx=0).stop 2023-10-26T01:32:04.981 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.979+0000 7fcb6f925640 1 -- 172.21.15.169:0/2364701482 shutdown_connections 2023-10-26T01:32:04.982 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.979+0000 7fcb6f925640 1 --2- 172.21.15.169:0/2364701482 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcb680b0c80 0x7fcb680cd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:04.982 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.979+0000 7fcb6f925640 1 --2- 172.21.15.169:0/2364701482 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb681597c0 0x7fcb680ac200 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:04.982 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.979+0000 7fcb6f925640 1 -- 172.21.15.169:0/2364701482 >> 172.21.15.169:0/2364701482 conn(0x7fcb68069930 msgr2=0x7fcb68069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:04.982 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.979+0000 7fcb6f925640 1 -- 172.21.15.169:0/2364701482 shutdown_connections 2023-10-26T01:32:04.982 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.979+0000 7fcb6f925640 1 -- 172.21.15.169:0/2364701482 wait complete. 2023-10-26T01:32:04.984 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.981+0000 7fcb6f925640 1 Processor -- start 2023-10-26T01:32:04.985 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.981+0000 7fcb6f925640 1 -- start start 2023-10-26T01:32:04.985 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.982+0000 7fcb6f925640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb680b0c80 0x7fcb681467b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:04.985 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.982+0000 7fcb6f925640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcb681593e0 0x7fcb681579b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:04.985 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.982+0000 7fcb6f925640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb681597c0 0x7fcb6815b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:04.986 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.982+0000 7fcb6f925640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb68146cf0 con 0x7fcb680b0c80 2023-10-26T01:32:04.986 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.982+0000 7fcb6f925640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb68157ef0 con 0x7fcb681597c0 2023-10-26T01:32:04.986 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.982+0000 7fcb6f925640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb68156db0 con 0x7fcb681593e0 2023-10-26T01:32:04.986 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.982+0000 7fcb6d6c1640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb680b0c80 0x7fcb681467b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:04.986 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.982+0000 7fcb6d6c1640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb680b0c80 0x7fcb681467b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:44444/0 (socket says 172.21.15.169:44444) 2023-10-26T01:32:04.987 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.982+0000 7fcb6d6c1640 1 -- 172.21.15.169:0/1482777161 learned_addr learned my addr 172.21.15.169:0/1482777161 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:32:04.987 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.982+0000 7fcb6cec0640 1 --2- 172.21.15.169:0/1482777161 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcb681593e0 0x7fcb681579b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:04.987 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.982+0000 7fcb6dec2640 1 --2- 172.21.15.169:0/1482777161 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb681597c0 0x7fcb6815b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:04.987 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.982+0000 7fcb6d6c1640 1 -- 172.21.15.169:0/1482777161 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcb681593e0 msgr2=0x7fcb681579b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:04.987 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.982+0000 7fcb6d6c1640 1 --2- 172.21.15.169:0/1482777161 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcb681593e0 0x7fcb681579b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:04.988 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.982+0000 7fcb6d6c1640 1 -- 172.21.15.169:0/1482777161 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb681597c0 msgr2=0x7fcb6815b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:04.988 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.982+0000 7fcb6d6c1640 1 --2- 172.21.15.169:0/1482777161 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb681597c0 0x7fcb6815b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:04.988 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.982+0000 7fcb6d6c1640 1 -- 172.21.15.169:0/1482777161 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb5c009560 con 0x7fcb680b0c80 2023-10-26T01:32:04.988 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.983+0000 7fcb6d6c1640 1 --2- 172.21.15.169:0/1482777161 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb680b0c80 0x7fcb681467b0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fcb5800c7a0 tx=0x7fcb5800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:04.988 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.983+0000 7fcb65ffb640 1 -- 172.21.15.169:0/1482777161 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb58003d80 con 0x7fcb680b0c80 2023-10-26T01:32:04.989 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.983+0000 7fcb65ffb640 1 -- 172.21.15.169:0/1482777161 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fcb58005070 con 0x7fcb680b0c80 2023-10-26T01:32:04.989 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.983+0000 7fcb6f925640 1 -- 172.21.15.169:0/1482777161 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb68155d10 con 0x7fcb680b0c80 2023-10-26T01:32:04.989 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.983+0000 7fcb65ffb640 1 -- 172.21.15.169:0/1482777161 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb580054e0 con 0x7fcb680b0c80 2023-10-26T01:32:04.989 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.983+0000 7fcb6f925640 1 -- 172.21.15.169:0/1482777161 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb681562d0 con 0x7fcb680b0c80 2023-10-26T01:32:04.991 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.985+0000 7fcb65ffb640 1 -- 172.21.15.169:0/1482777161 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fcb58005640 con 0x7fcb680b0c80 2023-10-26T01:32:04.992 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.986+0000 7fcb65ffb640 1 --2- 172.21.15.169:0/1482777161 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcb5406ae70 0x7fcb5406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:04.992 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.986+0000 7fcb65ffb640 1 -- 172.21.15.169:0/1482777161 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcb58089120 con 0x7fcb680b0c80 2023-10-26T01:32:04.992 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.987+0000 7fcb6cec0640 1 --2- 172.21.15.169:0/1482777161 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcb5406ae70 0x7fcb5406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:04.992 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.987+0000 7fcb7258e740 1 Processor -- start 2023-10-26T01:32:04.992 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.987+0000 7fcb7258e740 1 -- start start 2023-10-26T01:32:04.993 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.988+0000 7fcb6cec0640 1 --2- 172.21.15.169:0/1482777161 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcb5406ae70 0x7fcb5406d330 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c035000 tx=0x7fcb5c004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:04.993 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.988+0000 7fcb7258e740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564b909167d0 0x564b908e9a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:04.993 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.988+0000 7fcb7258e740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564b90916bb0 0x564b90919560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:04.993 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.988+0000 7fcb7258e740 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x564b908e9f40 0x564b90919aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:04.993 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.988+0000 7fcb7258e740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b9085c740 con 0x564b90916bb0 2023-10-26T01:32:04.994 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.988+0000 7fcb7258e740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b90859e30 con 0x564b909167d0 2023-10-26T01:32:04.994 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.988+0000 7fcb7258e740 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x564b90855720 con 0x564b908e9f40 2023-10-26T01:32:04.994 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.989+0000 7fcb6cec0640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564b90916bb0 0x564b90919560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:04.994 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.989+0000 7fcb6cec0640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564b90916bb0 0x564b90919560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:44452/0 (socket says 172.21.15.169:44452) 2023-10-26T01:32:04.994 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.989+0000 7fcb6cec0640 1 -- 172.21.15.169:0/3210952010 learned_addr learned my addr 172.21.15.169:0/3210952010 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:32:04.995 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.989+0000 7fcb6dec2640 1 --2- 172.21.15.169:0/3210952010 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x564b908e9f40 0x564b90919aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:04.995 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.989+0000 7fcb6d6c1640 1 --2- 172.21.15.169:0/3210952010 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564b909167d0 0x564b908e9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:04.995 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.989+0000 7fcb6cec0640 1 -- 172.21.15.169:0/3210952010 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x564b908e9f40 msgr2=0x564b90919aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T01:32:04.995 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.989+0000 7fcb6cec0640 1 --2- 172.21.15.169:0/3210952010 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x564b908e9f40 0x564b90919aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:04.995 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.989+0000 7fcb6cec0640 1 -- 172.21.15.169:0/3210952010 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564b909167d0 msgr2=0x564b908e9a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T01:32:04.995 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.989+0000 7fcb6cec0640 1 --2- 172.21.15.169:0/3210952010 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564b909167d0 0x564b908e9a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:04.996 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.989+0000 7fcb6cec0640 1 -- 172.21.15.169:0/3210952010 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564b90819910 con 0x564b90916bb0 2023-10-26T01:32:04.996 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.989+0000 7fcb6dec2640 1 --2- 172.21.15.169:0/3210952010 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x564b908e9f40 0x564b90919aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:32:04.996 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.989+0000 7fcb6d6c1640 1 --2- 172.21.15.169:0/3210952010 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564b909167d0 0x564b908e9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:32:04.996 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.989+0000 7fcb6cec0640 1 --2- 172.21.15.169:0/3210952010 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564b90916bb0 0x564b90919560 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c0099c0 tx=0x7fcb5c02e050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed8abbb0282916ab server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:04.996 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.990+0000 7fcb4effd640 1 -- 172.21.15.169:0/3210952010 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb5c047070 con 0x564b90916bb0 2023-10-26T01:32:04.997 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.990+0000 7fcb4effd640 1 -- 172.21.15.169:0/3210952010 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fcb5c02ccf0 con 0x564b90916bb0 2023-10-26T01:32:04.997 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.990+0000 7fcb4effd640 1 -- 172.21.15.169:0/3210952010 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb5c003830 con 0x564b90916bb0 2023-10-26T01:32:04.997 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.990+0000 7fcb7258e740 1 -- 172.21.15.169:0/3210952010 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564b90916bb0 msgr2=0x564b90919560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:04.997 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.990+0000 7fcb7258e740 1 --2- 172.21.15.169:0/3210952010 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564b90916bb0 0x564b90919560 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c0099c0 tx=0x7fcb5c02e050 comp rx=0 tx=0).stop 2023-10-26T01:32:05.001 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.990+0000 7fcb7258e740 1 -- 172.21.15.169:0/3210952010 shutdown_connections 2023-10-26T01:32:05.001 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.990+0000 7fcb7258e740 1 --2- 172.21.15.169:0/3210952010 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564b90916bb0 0x564b90919560 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:05.001 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.990+0000 7fcb7258e740 1 --2- 172.21.15.169:0/3210952010 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x564b908e9f40 0x564b90919aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:05.001 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.990+0000 7fcb7258e740 1 --2- 172.21.15.169:0/3210952010 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564b909167d0 0x564b908e9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:05.002 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.990+0000 7fcb7258e740 1 -- 172.21.15.169:0/3210952010 >> 172.21.15.169:0/3210952010 conn(0x564b908ebc70 msgr2=0x564b90934560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:05.002 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.990+0000 7fcb7258e740 1 -- 172.21.15.169:0/3210952010 shutdown_connections 2023-10-26T01:32:05.002 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.991+0000 7fcb7258e740 1 -- 172.21.15.169:0/3210952010 wait complete. 2023-10-26T01:32:05.002 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.991+0000 7fcb7258e740 1 -- start start 2023-10-26T01:32:05.002 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.991+0000 7fcb7258e740 1 Processor -- start 2023-10-26T01:32:05.003 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.992+0000 7fcb7258e740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564b909167d0 0x564b909153e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:05.003 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.992+0000 7fcb7258e740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564b90916bb0 0x564b90934d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:05.003 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.992+0000 7fcb7258e740 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x564b9091bb00 0x564b9091bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:05.003 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.992+0000 7fcb7258e740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b907f3970 con 0x564b90916bb0 2023-10-26T01:32:05.003 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.992+0000 7fcb7258e740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b907f3610 con 0x564b909167d0 2023-10-26T01:32:05.004 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.992+0000 7fcb7258e740 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x564b905b9f20 con 0x564b9091bb00 2023-10-26T01:32:05.004 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.992+0000 7fcb7258e740 10 client.0 _ll_register_callbacks cb 0x564b908170c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T01:32:05.004 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.993+0000 7fcb6cec0640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564b90916bb0 0x564b90934d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:05.004 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.993+0000 7fcb6cec0640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564b90916bb0 0x564b90934d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:44454/0 (socket says 172.21.15.169:44454) 2023-10-26T01:32:05.004 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.993+0000 7fcb6cec0640 1 -- 172.21.15.169:0/1674183195 learned_addr learned my addr 172.21.15.169:0/1674183195 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:32:05.005 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.993+0000 7fcb6cec0640 1 -- 172.21.15.169:0/1674183195 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x564b9091bb00 msgr2=0x564b9091bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T01:32:05.005 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.993+0000 7fcb6cec0640 1 --2- 172.21.15.169:0/1674183195 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x564b9091bb00 0x564b9091bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:05.005 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.993+0000 7fcb6cec0640 1 -- 172.21.15.169:0/1674183195 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564b909167d0 msgr2=0x564b909153e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T01:32:05.005 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.993+0000 7fcb6d6c1640 1 --2- 172.21.15.169:0/1674183195 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564b909167d0 0x564b909153e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:05.005 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.993+0000 7fcb6cec0640 1 --2- 172.21.15.169:0/1674183195 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564b909167d0 0x564b909153e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:05.006 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.993+0000 7fcb6cec0640 1 -- 172.21.15.169:0/1674183195 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb680acae0 con 0x564b90916bb0 2023-10-26T01:32:05.006 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.993+0000 7fcb6cec0640 1 --2- 172.21.15.169:0/1674183195 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564b90916bb0 0x564b90934d40 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c036040 tx=0x7fcb5c003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec7f4658fdcda573 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:05.006 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.993+0000 7fcb47fff640 10 client.0 ms_handle_connect on v2:172.21.15.169:3300/0 2023-10-26T01:32:05.006 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.994+0000 7fcb47fff640 1 -- 172.21.15.169:0/1674183195 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb5c047040 con 0x564b90916bb0 2023-10-26T01:32:05.006 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.994+0000 7fcb47fff640 1 -- 172.21.15.169:0/1674183195 <== mon.0 v2:172.21.15.169:3300/0 2 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcb5c040880 con 0x564b90916bb0 2023-10-26T01:32:05.007 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.994+0000 7fcb7258e740 1 -- 172.21.15.169:0/1674183195 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564b9084e700 con 0x564b90916bb0 2023-10-26T01:32:05.007 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.994+0000 7fcb47fff640 1 -- 172.21.15.169:0/1674183195 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_get_version_reply(handle=1 version=15) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb5c02ce60 con 0x564b90916bb0 2023-10-26T01:32:05.007 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.994+0000 7fcb7258e740 10 client.5358 fetch_fsmap learned FSMap version 15 2023-10-26T01:32:05.007 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.994+0000 7fcb7258e740 1 -- 172.21.15.169:0/1674183195 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,fsmap=15,monmap=2+}) v3 -- 0x564b90853da0 con 0x564b90916bb0 2023-10-26T01:32:05.007 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.994+0000 7fcb47fff640 1 -- 172.21.15.169:0/1674183195 <== mon.0 v2:172.21.15.169:3300/0 4 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x564b9084e700 con 0x564b90916bb0 2023-10-26T01:32:05.008 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.994+0000 7fcb47fff640 1 -- 172.21.15.169:0/1674183195 <== mon.0 v2:172.21.15.169:3300/0 5 ==== fsmap(e 15) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fcb5c03c040 con 0x564b90916bb0 2023-10-26T01:32:05.008 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.994+0000 7fcb7258e740 10 client.5358 fetch_fsmap finished waiting for FSMap version 15 2023-10-26T01:32:05.008 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.994+0000 7fcb7258e740 10 client.5358 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-26T01:32:05.008 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.994+0000 7fcb7258e740 20 client.5358 populate_metadata read hostname 'smithi169' 2023-10-26T01:32:05.008 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.994+0000 7fcb7258e740 1 --2- 172.21.15.169:0/1674183195 >> [v2:172.21.15.169:6834/871978528,v1:172.21.15.169:6835/871978528] conn(0x564b909485a0 0x564b9092ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:05.009 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.994+0000 7fcb7258e740 4 client.5358 mds_command: new command op to 5196 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-26T01:32:05.009 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.994+0000 7fcb7258e740 1 -- 172.21.15.169:0/1674183195 --> [v2:172.21.15.169:6834/871978528,v1:172.21.15.169:6835/871978528] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564b9084e420 con 0x564b909485a0 2023-10-26T01:32:05.009 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.995+0000 7fcb6d6c1640 1 --2- 172.21.15.169:0/1674183195 >> [v2:172.21.15.169:6834/871978528,v1:172.21.15.169:6835/871978528] conn(0x564b909485a0 0x564b9092ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:05.009 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.995+0000 7fcb47fff640 10 client.5358 ms_handle_connect on v2:172.21.15.169:6834/871978528 2023-10-26T01:32:05.009 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.995+0000 7fcb6d6c1640 1 --2- 172.21.15.169:0/1674183195 >> [v2:172.21.15.169:6834/871978528,v1:172.21.15.169:6835/871978528] conn(0x564b909485a0 0x564b9092ca80 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69fe33c5b7fabc8e server_cookie=26249abf41c522a2 in_seq=0 out_seq=0 2023-10-26T01:32:05.009 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.996+0000 7fcb47fff640 1 -- 172.21.15.169:0/1674183195 <== mds.0 v2:172.21.15.169:6834/871978528 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x564b9084e420 con 0x564b909485a0 2023-10-26T01:32:05.010 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.996+0000 7fcb47fff640 10 client.5358 handle_command_reply: tid=0 2023-10-26T01:32:05.010 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.996+0000 7fcb7258e740 1 client.5358 shutdown 2023-10-26T01:32:05.010 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.996+0000 7fcb7258e740 1 -- 172.21.15.169:0/1674183195 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564b90916bb0 msgr2=0x564b90934d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:05.010 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.996+0000 7fcb7258e740 1 --2- 172.21.15.169:0/1674183195 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564b90916bb0 0x564b90934d40 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c036040 tx=0x7fcb5c003d40 comp rx=0 tx=0).stop 2023-10-26T01:32:05.010 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.996+0000 7fcb7258e740 1 -- 172.21.15.169:0/1674183195 shutdown_connections 2023-10-26T01:32:05.011 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.996+0000 7fcb7258e740 1 --2- 172.21.15.169:0/1674183195 >> [v2:172.21.15.169:6834/871978528,v1:172.21.15.169:6835/871978528] conn(0x564b909485a0 0x564b9092ca80 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:05.011 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.996+0000 7fcb7258e740 1 --2- 172.21.15.169:0/1674183195 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564b90916bb0 0x564b90934d40 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:05.011 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.996+0000 7fcb7258e740 1 --2- 172.21.15.169:0/1674183195 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x564b9091bb00 0x564b9091bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:05.011 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.996+0000 7fcb47fff640 0 client.5358 ms_handle_reset on v2:172.21.15.169:6834/871978528 2023-10-26T01:32:05.011 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.996+0000 7fcb7258e740 1 --2- 172.21.15.169:0/1674183195 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564b909167d0 0x564b909153e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:05.012 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.996+0000 7fcb7258e740 1 -- 172.21.15.169:0/1674183195 >> 172.21.15.169:0/1674183195 conn(0x564b908e9f40 msgr2=0x564b909379e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:05.012 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.996+0000 7fcb7258e740 1 -- 172.21.15.169:0/1674183195 shutdown_connections 2023-10-26T01:32:05.012 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.996+0000 7fcb7258e740 1 -- 172.21.15.169:0/1674183195 wait complete. 2023-10-26T01:32:05.012 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:04.997+0000 7fcb7258e740 20 client.5358 trim_cache size 0 max 16384 2023-10-26T01:32:05.015 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.013+0000 7fcb7258e740 1 Processor -- start 2023-10-26T01:32:05.015 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.013+0000 7fcb7258e740 1 -- start start 2023-10-26T01:32:05.016 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.014+0000 7fcb7258e740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564b909167d0 0x564b90943b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:05.017 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.014+0000 7fcb7258e740 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x564b90916bb0 0x564b90937f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:05.017 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.014+0000 7fcb7258e740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564b9091bb00 0x564b9092dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:05.017 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.014+0000 7fcb7258e740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b908ee170 con 0x564b9091bb00 2023-10-26T01:32:05.017 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.014+0000 7fcb7258e740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b9092ec30 con 0x564b909167d0 2023-10-26T01:32:05.017 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.014+0000 7fcb7258e740 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x564b90915ad0 con 0x564b90916bb0 2023-10-26T01:32:05.018 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.014+0000 7fcb6dec2640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564b9091bb00 0x564b9092dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:05.018 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.014+0000 7fcb6dec2640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564b9091bb00 0x564b9092dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:44456/0 (socket says 172.21.15.169:44456) 2023-10-26T01:32:05.018 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.014+0000 7fcb6dec2640 1 -- 172.21.15.169:0/1965875079 learned_addr learned my addr 172.21.15.169:0/1965875079 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:32:05.018 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.015+0000 7fcb6d6c1640 1 --2- 172.21.15.169:0/1965875079 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564b909167d0 0x564b90943b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:05.019 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.015+0000 7fcb6cec0640 1 --2- 172.21.15.169:0/1965875079 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x564b90916bb0 0x564b90937f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:05.019 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.015+0000 7fcb6dec2640 1 -- 172.21.15.169:0/1965875079 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x564b90916bb0 msgr2=0x564b90937f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T01:32:05.019 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.015+0000 7fcb6dec2640 1 --2- 172.21.15.169:0/1965875079 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x564b90916bb0 0x564b90937f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:05.019 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.015+0000 7fcb6dec2640 1 -- 172.21.15.169:0/1965875079 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564b909167d0 msgr2=0x564b90943b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T01:32:05.019 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.015+0000 7fcb6dec2640 1 --2- 172.21.15.169:0/1965875079 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564b909167d0 0x564b90943b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:05.020 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.015+0000 7fcb6dec2640 1 -- 172.21.15.169:0/1965875079 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564b907f2b80 con 0x564b9091bb00 2023-10-26T01:32:05.020 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.015+0000 7fcb6d6c1640 1 --2- 172.21.15.169:0/1965875079 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564b909167d0 0x564b90943b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:32:05.020 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.015+0000 7fcb6dec2640 1 --2- 172.21.15.169:0/1965875079 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564b9091bb00 0x564b9092dda0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fcb6000b770 tx=0x7fcb6000bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62f334318e04188d server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:05.024 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.015+0000 7fcb4f7fe640 1 -- 172.21.15.169:0/1965875079 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb6000dbb0 con 0x564b9091bb00 2023-10-26T01:32:05.024 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.015+0000 7fcb4f7fe640 1 -- 172.21.15.169:0/1965875079 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fcb6000c040 con 0x564b9091bb00 2023-10-26T01:32:05.024 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.015+0000 7fcb4f7fe640 1 -- 172.21.15.169:0/1965875079 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb600126c0 con 0x564b9091bb00 2023-10-26T01:32:05.025 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.016+0000 7fcb7258e740 1 -- 172.21.15.169:0/1965875079 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564b9091bb00 msgr2=0x564b9092dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:05.025 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.016+0000 7fcb7258e740 1 --2- 172.21.15.169:0/1965875079 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564b9091bb00 0x564b9092dda0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fcb6000b770 tx=0x7fcb6000bc30 comp rx=0 tx=0).stop 2023-10-26T01:32:05.025 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.016+0000 7fcb7258e740 1 -- 172.21.15.169:0/1965875079 shutdown_connections 2023-10-26T01:32:05.025 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.016+0000 7fcb7258e740 1 --2- 172.21.15.169:0/1965875079 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564b9091bb00 0x564b9092dda0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:05.025 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.016+0000 7fcb7258e740 1 --2- 172.21.15.169:0/1965875079 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x564b90916bb0 0x564b90937f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:05.026 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.016+0000 7fcb7258e740 1 --2- 172.21.15.169:0/1965875079 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564b909167d0 0x564b90943b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:05.026 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.016+0000 7fcb7258e740 1 -- 172.21.15.169:0/1965875079 >> 172.21.15.169:0/1965875079 conn(0x564b908e9f40 msgr2=0x564b9091c840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:05.026 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.016+0000 7fcb7258e740 1 -- 172.21.15.169:0/1965875079 shutdown_connections 2023-10-26T01:32:05.026 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.016+0000 7fcb7258e740 1 -- 172.21.15.169:0/1965875079 wait complete. 2023-10-26T01:32:05.026 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.017+0000 7fcb7258e740 1 -- start start 2023-10-26T01:32:05.027 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.017+0000 7fcb7258e740 1 Processor -- start 2023-10-26T01:32:05.027 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.018+0000 7fcb7258e740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564b909167d0 0x564b9099cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:05.027 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.018+0000 7fcb7258e740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564b90916bb0 0x564b908ee550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:05.027 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.018+0000 7fcb7258e740 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x564b9091bb00 0x564b908eea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:05.028 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.018+0000 7fcb7258e740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b9099ea10 con 0x564b909167d0 2023-10-26T01:32:05.028 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.018+0000 7fcb7258e740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b90937f00 con 0x564b90916bb0 2023-10-26T01:32:05.028 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.018+0000 7fcb7258e740 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x564b90938430 con 0x564b9091bb00 2023-10-26T01:32:05.028 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.018+0000 7fcb7258e740 10 client.0 _ll_register_callbacks cb 0x7fcb5c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T01:32:05.028 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.018+0000 7fcb6d6c1640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564b909167d0 0x564b9099cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:05.029 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.018+0000 7fcb6d6c1640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564b909167d0 0x564b9099cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:44470/0 (socket says 172.21.15.169:44470) 2023-10-26T01:32:05.029 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.018+0000 7fcb6d6c1640 1 -- 172.21.15.169:0/1869868665 learned_addr learned my addr 172.21.15.169:0/1869868665 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:32:05.029 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.019+0000 7fcb6d6c1640 1 -- 172.21.15.169:0/1869868665 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x564b9091bb00 msgr2=0x564b908eea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T01:32:05.029 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.019+0000 7fcb6d6c1640 1 --2- 172.21.15.169:0/1869868665 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x564b9091bb00 0x564b908eea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:05.029 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.019+0000 7fcb6d6c1640 1 -- 172.21.15.169:0/1869868665 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564b90916bb0 msgr2=0x564b908ee550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T01:32:05.030 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.019+0000 7fcb6d6c1640 1 --2- 172.21.15.169:0/1869868665 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564b90916bb0 0x564b908ee550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:05.030 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.019+0000 7fcb6d6c1640 1 -- 172.21.15.169:0/1869868665 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb6000b370 con 0x564b909167d0 2023-10-26T01:32:05.030 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.019+0000 7fcb6d6c1640 1 --2- 172.21.15.169:0/1869868665 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564b909167d0 0x564b9099cbd0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fcb5800a430 tx=0x7fcb58057d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3e5dc42802b6ea0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:05.030 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.019+0000 7fcb47fff640 10 client.0 ms_handle_connect on v2:172.21.15.169:3300/0 2023-10-26T01:32:05.030 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.019+0000 7fcb47fff640 1 -- 172.21.15.169:0/1869868665 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb58060180 con 0x564b909167d0 2023-10-26T01:32:05.031 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.019+0000 7fcb47fff640 1 -- 172.21.15.169:0/1869868665 <== mon.0 v2:172.21.15.169:3300/0 2 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcb58056220 con 0x564b909167d0 2023-10-26T01:32:05.031 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.019+0000 7fcb7258e740 1 -- 172.21.15.169:0/1869868665 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcb5c02ccf0 con 0x564b909167d0 2023-10-26T01:32:05.031 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.019+0000 7fcb47fff640 1 -- 172.21.15.169:0/1869868665 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_get_version_reply(handle=1 version=15) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb58054590 con 0x564b909167d0 2023-10-26T01:32:05.031 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.019+0000 7fcb7258e740 10 client.5364 fetch_fsmap learned FSMap version 15 2023-10-26T01:32:05.031 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.019+0000 7fcb7258e740 1 -- 172.21.15.169:0/1869868665 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,fsmap=15,monmap=2+}) v3 -- 0x564b90818170 con 0x564b909167d0 2023-10-26T01:32:05.032 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.020+0000 7fcb47fff640 1 -- 172.21.15.169:0/1869868665 <== mon.0 v2:172.21.15.169:3300/0 4 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fcb5c02ccf0 con 0x564b909167d0 2023-10-26T01:32:05.032 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.020+0000 7fcb47fff640 1 -- 172.21.15.169:0/1869868665 <== mon.0 v2:172.21.15.169:3300/0 5 ==== fsmap(e 15) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fcb5800a560 con 0x564b909167d0 2023-10-26T01:32:05.032 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.020+0000 7fcb7258e740 10 client.5364 fetch_fsmap finished waiting for FSMap version 15 2023-10-26T01:32:05.032 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.020+0000 7fcb7258e740 10 client.5364 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-26T01:32:05.032 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.020+0000 7fcb7258e740 20 client.5364 populate_metadata read hostname 'smithi169' 2023-10-26T01:32:05.033 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.020+0000 7fcb7258e740 1 --2- 172.21.15.169:0/1869868665 >> [v2:172.21.15.169:6834/871978528,v1:172.21.15.169:6835/871978528] conn(0x564b90914f50 0x564b90915330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:05.033 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.020+0000 7fcb7258e740 4 client.5364 mds_command: new command op to 5196 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-26T01:32:05.033 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.020+0000 7fcb7258e740 1 -- 172.21.15.169:0/1869868665 --> [v2:172.21.15.169:6834/871978528,v1:172.21.15.169:6835/871978528] -- command(tid 0: {"prefix": "status"}) v1 -- 0x564b905576c0 con 0x564b90914f50 2023-10-26T01:32:05.033 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.020+0000 7fcb6cec0640 1 --2- 172.21.15.169:0/1869868665 >> [v2:172.21.15.169:6834/871978528,v1:172.21.15.169:6835/871978528] conn(0x564b90914f50 0x564b90915330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:05.033 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.021+0000 7fcb6cec0640 1 --2- 172.21.15.169:0/1869868665 >> [v2:172.21.15.169:6834/871978528,v1:172.21.15.169:6835/871978528] conn(0x564b90914f50 0x564b90915330 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b6a7632074376e1 server_cookie=69125ca444b8676d in_seq=0 out_seq=0 2023-10-26T01:32:05.034 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.021+0000 7fcb47fff640 10 client.5364 ms_handle_connect on v2:172.21.15.169:6834/871978528 2023-10-26T01:32:05.034 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.021+0000 7fcb47fff640 1 -- 172.21.15.169:0/1869868665 <== mds.0 v2:172.21.15.169:6834/871978528 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x564b905576c0 con 0x564b90914f50 2023-10-26T01:32:05.034 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.021+0000 7fcb47fff640 10 client.5364 handle_command_reply: tid=0 2023-10-26T01:32:05.034 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.021+0000 7fcb7258e740 1 client.5364 shutdown 2023-10-26T01:32:05.034 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.021+0000 7fcb7258e740 1 -- 172.21.15.169:0/1869868665 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564b909167d0 msgr2=0x564b9099cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:05.035 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.021+0000 7fcb7258e740 1 --2- 172.21.15.169:0/1869868665 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564b909167d0 0x564b9099cbd0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fcb5800a430 tx=0x7fcb58057d80 comp rx=0 tx=0).stop 2023-10-26T01:32:05.035 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.021+0000 7fcb7258e740 1 -- 172.21.15.169:0/1869868665 shutdown_connections 2023-10-26T01:32:05.035 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.021+0000 7fcb7258e740 1 --2- 172.21.15.169:0/1869868665 >> [v2:172.21.15.169:6834/871978528,v1:172.21.15.169:6835/871978528] conn(0x564b90914f50 0x564b90915330 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:05.035 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.021+0000 7fcb7258e740 1 --2- 172.21.15.169:0/1869868665 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x564b9091bb00 0x564b908eea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:05.035 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.021+0000 7fcb7258e740 1 --2- 172.21.15.169:0/1869868665 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564b90916bb0 0x564b908ee550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:05.035 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.021+0000 7fcb47fff640 0 client.5364 ms_handle_reset on v2:172.21.15.169:6834/871978528 2023-10-26T01:32:05.036 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.021+0000 7fcb7258e740 1 --2- 172.21.15.169:0/1869868665 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564b909167d0 0x564b9099cbd0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:05.036 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.021+0000 7fcb7258e740 1 -- 172.21.15.169:0/1869868665 >> 172.21.15.169:0/1869868665 conn(0x564b908e9f40 msgr2=0x564b90918fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:05.036 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.021+0000 7fcb7258e740 1 -- 172.21.15.169:0/1869868665 shutdown_connections 2023-10-26T01:32:05.036 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.021+0000 7fcb7258e740 1 -- 172.21.15.169:0/1869868665 wait complete. 2023-10-26T01:32:05.036 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.021+0000 7fcb7258e740 20 client.5364 trim_cache size 0 max 16384 2023-10-26T01:32:05.037 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2023-10-26T01:32:05.037 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_fsid": "30203b28-ed8a-44e8-a9a6-a9bf25f980f1", 2023-10-26T01:32:05.037 INFO:teuthology.orchestra.run.smithi169.stdout: "whoami": 0, 2023-10-26T01:32:05.037 INFO:teuthology.orchestra.run.smithi169.stdout: "id": 5196, 2023-10-26T01:32:05.038 INFO:teuthology.orchestra.run.smithi169.stdout: "want_state": "up:active", 2023-10-26T01:32:05.038 INFO:teuthology.orchestra.run.smithi169.stdout: "state": "up:active", 2023-10-26T01:32:05.038 INFO:teuthology.orchestra.run.smithi169.stdout: "fs_name": "cephfs", 2023-10-26T01:32:05.038 INFO:teuthology.orchestra.run.smithi169.stdout: "rank_uptime": 6.7017766029999999, 2023-10-26T01:32:05.038 INFO:teuthology.orchestra.run.smithi169.stdout: "mdsmap_epoch": 15, 2023-10-26T01:32:05.039 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_epoch": 35, 2023-10-26T01:32:05.039 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_epoch_barrier": 35, 2023-10-26T01:32:05.039 INFO:teuthology.orchestra.run.smithi169.stdout: "uptime": 16.515409761000001 2023-10-26T01:32:05.039 INFO:teuthology.orchestra.run.smithi169.stdout:} 2023-10-26T01:32:05.040 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.022+0000 7fcb6f925640 1 -- 172.21.15.169:0/1482777161 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcb5406ae70 msgr2=0x7fcb5406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:05.041 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.022+0000 7fcb6f925640 1 --2- 172.21.15.169:0/1482777161 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcb5406ae70 0x7fcb5406d330 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c035000 tx=0x7fcb5c004730 comp rx=0 tx=0).stop 2023-10-26T01:32:05.041 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.022+0000 7fcb6f925640 1 -- 172.21.15.169:0/1482777161 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb680b0c80 msgr2=0x7fcb681467b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:05.041 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.022+0000 7fcb6f925640 1 --2- 172.21.15.169:0/1482777161 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb680b0c80 0x7fcb681467b0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fcb5800c7a0 tx=0x7fcb5800cc60 comp rx=0 tx=0).stop 2023-10-26T01:32:05.042 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.022+0000 7fcb6f925640 1 -- 172.21.15.169:0/1482777161 shutdown_connections 2023-10-26T01:32:05.042 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.022+0000 7fcb6f925640 1 --2- 172.21.15.169:0/1482777161 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcb5406ae70 0x7fcb5406d330 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:05.042 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.022+0000 7fcb6f925640 1 --2- 172.21.15.169:0/1482777161 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb681597c0 0x7fcb6815b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:05.042 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.022+0000 7fcb6f925640 1 --2- 172.21.15.169:0/1482777161 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcb681593e0 0x7fcb681579b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:05.042 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.022+0000 7fcb6f925640 1 --2- 172.21.15.169:0/1482777161 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb680b0c80 0x7fcb681467b0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:05.043 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.022+0000 7fcb6f925640 1 -- 172.21.15.169:0/1482777161 >> 172.21.15.169:0/1482777161 conn(0x7fcb68069930 msgr2=0x7fcb6815c2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:05.043 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.022+0000 7fcb6f925640 1 -- 172.21.15.169:0/1482777161 shutdown_connections 2023-10-26T01:32:05.043 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.022+0000 7fcb6f925640 1 -- 172.21.15.169:0/1482777161 wait complete. 2023-10-26T01:32:05.043 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-26T01:32:05.253 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.251+0000 7f8023fff640 1 -- 172.21.15.169:0/3251969209 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f800c0051d0 msgr2=0x7f800c00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:05.253 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.251+0000 7f8023fff640 1 --2- 172.21.15.169:0/3251969209 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f800c0051d0 0x7f800c00b980 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f802415c920 tx=0x7f800c0057f0 comp rx=0 tx=0).stop 2023-10-26T01:32:05.254 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.251+0000 7f8023fff640 1 -- 172.21.15.169:0/3251969209 shutdown_connections 2023-10-26T01:32:05.254 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.251+0000 7f8023fff640 1 --2- 172.21.15.169:0/3251969209 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f800c0051d0 0x7f800c00b980 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:05.254 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.251+0000 7f8023fff640 1 -- 172.21.15.169:0/3251969209 >> 172.21.15.169:0/3251969209 conn(0x7f80240af0c0 msgr2=0x7f80240af4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:05.254 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.251+0000 7f8023fff640 1 -- 172.21.15.169:0/3251969209 shutdown_connections 2023-10-26T01:32:05.254 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.251+0000 7f8023fff640 1 -- 172.21.15.169:0/3251969209 wait complete. 2023-10-26T01:32:05.255 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.253+0000 7f8023fff640 1 Processor -- start 2023-10-26T01:32:05.255 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.253+0000 7f8023fff640 1 -- start start 2023-10-26T01:32:05.257 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.254+0000 7f8023fff640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f800c0051d0 0x7f8024149c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:05.257 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.254+0000 7f8023fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8024151390 0x7f8024151e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:05.257 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.254+0000 7f8023fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f80241523d0 0x7f802415a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:05.257 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.254+0000 7f8023fff640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8024152860 con 0x7f80241523d0 2023-10-26T01:32:05.257 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.254+0000 7f8023fff640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8024147520 con 0x7f8024151390 2023-10-26T01:32:05.258 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.254+0000 7f8023fff640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8024159af0 con 0x7f800c0051d0 2023-10-26T01:32:05.258 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.255+0000 7f80237fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f80241523d0 0x7f802415a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:05.258 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.255+0000 7f80237fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f80241523d0 0x7f802415a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:44488/0 (socket says 172.21.15.169:44488) 2023-10-26T01:32:05.258 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.255+0000 7f80237fe640 1 -- 172.21.15.169:0/3533759709 learned_addr learned my addr 172.21.15.169:0/3533759709 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:32:05.258 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.255+0000 7f8022ffd640 1 --2- 172.21.15.169:0/3533759709 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f800c0051d0 0x7f8024149c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:05.259 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.255+0000 7f80227fc640 1 --2- 172.21.15.169:0/3533759709 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8024151390 0x7f8024151e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:05.259 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.255+0000 7f80237fe640 1 -- 172.21.15.169:0/3533759709 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f800c0051d0 msgr2=0x7f8024149c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:05.259 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.255+0000 7f80237fe640 1 --2- 172.21.15.169:0/3533759709 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f800c0051d0 0x7f8024149c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:05.259 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.255+0000 7f80237fe640 1 -- 172.21.15.169:0/3533759709 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8024151390 msgr2=0x7f8024151e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:05.260 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.255+0000 7f80237fe640 1 --2- 172.21.15.169:0/3533759709 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8024151390 0x7f8024151e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:05.260 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.255+0000 7f80237fe640 1 -- 172.21.15.169:0/3533759709 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8024159dd0 con 0x7f80241523d0 2023-10-26T01:32:05.261 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.256+0000 7f80237fe640 1 --2- 172.21.15.169:0/3533759709 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f80241523d0 0x7f802415a8b0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f801c00c5c0 tx=0x7f801c002160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:05.261 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.256+0000 7f800b7fe640 1 -- 172.21.15.169:0/3533759709 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f801c001ac0 con 0x7f80241523d0 2023-10-26T01:32:05.262 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.256+0000 7f800b7fe640 1 -- 172.21.15.169:0/3533759709 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f801c02ce90 con 0x7f80241523d0 2023-10-26T01:32:05.262 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.256+0000 7f800b7fe640 1 -- 172.21.15.169:0/3533759709 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f801c004da0 con 0x7f80241523d0 2023-10-26T01:32:05.262 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.256+0000 7f8023fff640 1 -- 172.21.15.169:0/3533759709 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f802415a0b0 con 0x7f80241523d0 2023-10-26T01:32:05.262 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.256+0000 7f8023fff640 1 -- 172.21.15.169:0/3533759709 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f802414a1d0 con 0x7f80241523d0 2023-10-26T01:32:05.263 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.257+0000 7f8023fff640 1 -- 172.21.15.169:0/3533759709 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f802415a5d0 con 0x7f80241523d0 2023-10-26T01:32:05.263 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.258+0000 7f800b7fe640 1 -- 172.21.15.169:0/3533759709 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f801c031030 con 0x7f80241523d0 2023-10-26T01:32:05.263 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.259+0000 7f800b7fe640 1 --2- 172.21.15.169:0/3533759709 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f801006ae40 0x7f801006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:05.263 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.260+0000 7f800b7fe640 1 -- 172.21.15.169:0/3533759709 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f801c0aa2c0 con 0x7f80241523d0 2023-10-26T01:32:05.264 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.260+0000 7f8022ffd640 1 --2- 172.21.15.169:0/3533759709 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f801006ae40 0x7f801006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:05.264 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.261+0000 7f8022ffd640 1 --2- 172.21.15.169:0/3533759709 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f801006ae40 0x7f801006d300 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f8014005fd0 tx=0x7f801400c500 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:05.265 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.264+0000 7f800b7fe640 1 -- 172.21.15.169:0/3533759709 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f802415a5d0 con 0x7f80241523d0 2023-10-26T01:32:05.500 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.499+0000 7f8023fff640 1 -- 172.21.15.169:0/3533759709 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f80240b12a0 con 0x7f80241523d0 2023-10-26T01:32:05.501 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.500+0000 7f800b7fe640 1 -- 172.21.15.169:0/3533759709 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v15) v1 ==== 53+0+73 (secure 0 0 0) 0x7f80240b12a0 con 0x7f80241523d0 2023-10-26T01:32:05.502 INFO:teuthology.orchestra.run.smithi169.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-26T01:32:05.504 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.502+0000 7f8023fff640 1 -- 172.21.15.169:0/3533759709 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f801006ae40 msgr2=0x7f801006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:05.505 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.502+0000 7f8023fff640 1 --2- 172.21.15.169:0/3533759709 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f801006ae40 0x7f801006d300 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f8014005fd0 tx=0x7f801400c500 comp rx=0 tx=0).stop 2023-10-26T01:32:05.505 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.502+0000 7f8023fff640 1 -- 172.21.15.169:0/3533759709 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f80241523d0 msgr2=0x7f802415a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:05.505 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.502+0000 7f8023fff640 1 --2- 172.21.15.169:0/3533759709 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f80241523d0 0x7f802415a8b0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f801c00c5c0 tx=0x7f801c002160 comp rx=0 tx=0).stop 2023-10-26T01:32:05.505 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.503+0000 7f8023fff640 1 -- 172.21.15.169:0/3533759709 shutdown_connections 2023-10-26T01:32:05.505 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.503+0000 7f8023fff640 1 --2- 172.21.15.169:0/3533759709 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f801006ae40 0x7f801006d300 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:05.506 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.503+0000 7f8023fff640 1 --2- 172.21.15.169:0/3533759709 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f80241523d0 0x7f802415a8b0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:05.506 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.503+0000 7f8023fff640 1 --2- 172.21.15.169:0/3533759709 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8024151390 0x7f8024151e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:05.506 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.503+0000 7f8023fff640 1 --2- 172.21.15.169:0/3533759709 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f800c0051d0 0x7f8024149c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:05.506 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.503+0000 7f8023fff640 1 -- 172.21.15.169:0/3533759709 >> 172.21.15.169:0/3533759709 conn(0x7f80240af0c0 msgr2=0x7f8024149260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:05.507 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.503+0000 7f8023fff640 1 -- 172.21.15.169:0/3533759709 shutdown_connections 2023-10-26T01:32:05.507 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:05.503+0000 7f8023fff640 1 -- 172.21.15.169:0/3533759709 wait complete. 2023-10-26T01:32:05.523 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-26T01:32:05.523 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-26T01:32:05.524 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi169.front.sepia.ceph.com 2023-10-26T01:32:05.524 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-26T01:32:05.524 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-26T01:32:05.525 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-26T01:32:05.525 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-26T01:32:05.525 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-26T01:32:05.525 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-26T01:32:05.526 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:32:05.526 DEBUG:teuthology.orchestra.run.smithi169:> ip addr 2023-10-26T01:32:05.578 INFO:teuthology.orchestra.run.smithi169.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-26T01:32:05.578 INFO:teuthology.orchestra.run.smithi169.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-26T01:32:05.578 INFO:teuthology.orchestra.run.smithi169.stdout: inet 127.0.0.1/8 scope host lo 2023-10-26T01:32:05.578 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:32:05.579 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 ::1/128 scope host 2023-10-26T01:32:05.579 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:32:05.579 INFO:teuthology.orchestra.run.smithi169.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-26T01:32:05.579 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:d9:3b:b4 brd ff:ff:ff:ff:ff:ff 2023-10-26T01:32:05.579 INFO:teuthology.orchestra.run.smithi169.stdout: altname enp7s0f0 2023-10-26T01:32:05.580 INFO:teuthology.orchestra.run.smithi169.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-26T01:32:05.580 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:d9:3b:b5 brd ff:ff:ff:ff:ff:ff 2023-10-26T01:32:05.580 INFO:teuthology.orchestra.run.smithi169.stdout: altname enp7s0f1 2023-10-26T01:32:05.580 INFO:teuthology.orchestra.run.smithi169.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-26T01:32:05.581 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:88:7f:4c brd ff:ff:ff:ff:ff:ff 2023-10-26T01:32:05.581 INFO:teuthology.orchestra.run.smithi169.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-26T01:32:05.581 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:88:7f:4d brd ff:ff:ff:ff:ff:ff 2023-10-26T01:32:05.581 INFO:teuthology.orchestra.run.smithi169.stdout: inet 172.21.15.169/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-26T01:32:05.581 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft 42370sec preferred_lft 42370sec 2023-10-26T01:32:05.582 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 fe80::ec4:7aff:fe88:7f4d/64 scope link 2023-10-26T01:32:05.582 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:32:05.582 INFO:teuthology.orchestra.run.smithi169.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-26T01:32:05.582 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 22:a0:6b:43:7b:04 brd ff:ff:ff:ff:ff:ff 2023-10-26T01:32:05.582 INFO:teuthology.orchestra.run.smithi169.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-26T01:32:05.583 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:32:05.583 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 fe80::d815:31ff:fe87:d148/64 scope link 2023-10-26T01:32:05.583 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:32:05.583 INFO:teuthology.orchestra.run.smithi169.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-26T01:32:05.583 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 2a:80:f8:45:5f:9f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-26T01:32:05.584 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 fe80::2880:f8ff:fe45:5f9f/64 scope link 2023-10-26T01:32:05.584 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:32:05.584 INFO:teuthology.orchestra.run.smithi169.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-26T01:32:05.584 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 22:a0:6b:43:7b:04 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-26T01:32:05.584 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 fe80::20a0:6bff:fe43:7b04/64 scope link 2023-10-26T01:32:05.584 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:32:05.585 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:32:05.586 DEBUG:teuthology.orchestra.run.smithi169:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-26T01:32:05.675 INFO:teuthology.orchestra.run.smithi169.stdout:1 2023-10-26T01:32:05.679 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:32:05.680 DEBUG:teuthology.orchestra.run.smithi169:> route 2023-10-26T01:32:05.736 INFO:teuthology.orchestra.run.smithi169.stdout:Kernel IP routing table 2023-10-26T01:32:05.736 INFO:teuthology.orchestra.run.smithi169.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-26T01:32:05.736 INFO:teuthology.orchestra.run.smithi169.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-26T01:32:05.736 INFO:teuthology.orchestra.run.smithi169.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-26T01:32:05.737 INFO:teuthology.orchestra.run.smithi169.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-26T01:32:05.737 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:32:05.738 DEBUG:teuthology.orchestra.run.smithi169:> (cd / && exec bash -c ' 2023-10-26T01:32:05.738 DEBUG:teuthology.orchestra.run.smithi169:> set -e 2023-10-26T01:32:05.738 DEBUG:teuthology.orchestra.run.smithi169:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-26T01:32:05.739 DEBUG:teuthology.orchestra.run.smithi169:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-26T01:32:05.739 DEBUG:teuthology.orchestra.run.smithi169:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-26T01:32:05.740 DEBUG:teuthology.orchestra.run.smithi169:> ') 2023-10-26T01:32:05.927 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:32:05.928 DEBUG:teuthology.orchestra.run.smithi169:> ip netns list 2023-10-26T01:32:05.976 INFO:teuthology.orchestra.run.smithi169.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-26T01:32:05.976 INFO:teuthology.orchestra.run.smithi169.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-26T01:32:05.977 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-26T01:32:05.978 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:32:05.978 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link set brx.0 up 2023-10-26T01:32:06.067 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-26T01:32:06.068 DEBUG:teuthology.orchestra.run.smithi169:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-26T01:32:06.119 INFO:teuthology.orchestra.run.smithi169.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-26T01:32:06.119 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-26T01:32:06.120 DEBUG:teuthology.orchestra.run.smithi169:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-26T01:32:06.168 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-26T01:32:06.169 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:32:06.169 DEBUG:teuthology.orchestra.run.smithi169:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-26T01:32:06.303 INFO:teuthology.orchestra.run.smithi169.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-26T01:32:06.304 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-26T01:32:06.304 INFO:teuthology.orchestra.run.smithi169.stdout:invalid new device string format 2023-10-26T01:32:06.325 INFO:teuthology.orchestra.run.smithi169.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-26T01:32:06.326 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-26T01:32:06.326 INFO:teuthology.orchestra.run.smithi169.stdout:invalid new device string format 2023-10-26T01:32:06.326 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: resolved to: "172.21.15.169:3300,172.21.15.170:3300,172.21.15.170:3301" 2023-10-26T01:32:06.326 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-26T01:32:06.350 INFO:teuthology.orchestra.run.smithi169.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-26T01:32:06.350 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-26T01:32:06.350 INFO:teuthology.orchestra.run.smithi169.stdout:invalid new device string format 2023-10-26T01:32:06.351 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: resolved to: "172.21.15.169:3300,172.21.15.170:3300,172.21.15.170:3301" 2023-10-26T01:32:06.351 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-26T01:32:06.351 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: resolved to: "172.21.15.169:6789,172.21.15.170:6789,172.21.15.170:6790" 2023-10-26T01:32:06.351 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.169:6789,172.21.15.170:6789,172.21.15.170:6790:/ 2023-10-26T01:32:06.351 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=30203b28-ed8a-44e8-a9a6-a9bf25f980f1" will pass to kernel 2023-10-26T01:32:06.352 INFO:teuthology.orchestra.run.smithi169.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2023-10-26T01:32:06.353 INFO:teuthology.orchestra.run.smithi169.stdout: You just mounted a file system that supports labels which does not 2023-10-26T01:32:06.353 INFO:teuthology.orchestra.run.smithi169.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-10-26T01:32:06.353 INFO:teuthology.orchestra.run.smithi169.stdout: applications will generate AVC messages and not be allowed access to 2023-10-26T01:32:06.353 INFO:teuthology.orchestra.run.smithi169.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-10-26T01:32:06.357 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-26T01:32:06.357 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:32:06.358 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-26T01:32:06.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:32:06.442 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-26T01:32:06.506 INFO:teuthology.orchestra.run.smithi169.stdout:client4712 2023-10-26T01:32:06.507 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:32:06.507 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:32:06.554 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-26T01:32:06.819 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:06.818+0000 7fc567bc9640 1 -- 172.21.15.169:0/26694948 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc56015b4a0 msgr2=0x7fc56015b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:06.820 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:06.818+0000 7fc567bc9640 1 --2- 172.21.15.169:0/26694948 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc56015b4a0 0x7fc56015b920 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc550005fd0 tx=0x7fc55002bd90 comp rx=0 tx=0).stop 2023-10-26T01:32:06.820 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:06.818+0000 7fc567bc9640 1 -- 172.21.15.169:0/26694948 shutdown_connections 2023-10-26T01:32:06.820 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:06.818+0000 7fc567bc9640 1 --2- 172.21.15.169:0/26694948 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc56015b4a0 0x7fc56015b920 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:06.821 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:06.818+0000 7fc567bc9640 1 --2- 172.21.15.169:0/26694948 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc56006ce00 0x7fc560155360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:06.821 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:06.818+0000 7fc567bc9640 1 -- 172.21.15.169:0/26694948 >> 172.21.15.169:0/26694948 conn(0x7fc560069930 msgr2=0x7fc560069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:06.821 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:06.818+0000 7fc567bc9640 1 -- 172.21.15.169:0/26694948 shutdown_connections 2023-10-26T01:32:06.821 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:06.818+0000 7fc567bc9640 1 -- 172.21.15.169:0/26694948 wait complete. 2023-10-26T01:32:06.822 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:06.820+0000 7fc567bc9640 1 Processor -- start 2023-10-26T01:32:06.822 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:06.820+0000 7fc567bc9640 1 -- start start 2023-10-26T01:32:06.824 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:06.821+0000 7fc567bc9640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc56006ad90 0x7fc560151410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:06.824 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:06.821+0000 7fc567bc9640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc56006ce00 0x7fc5600708c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:06.824 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:06.821+0000 7fc567bc9640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc56015b4a0 0x7fc560070e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:06.824 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:06.821+0000 7fc567bc9640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc56013db90 con 0x7fc56006ce00 2023-10-26T01:32:06.825 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:06.821+0000 7fc567bc9640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5601548d0 con 0x7fc56006ad90 2023-10-26T01:32:06.825 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:06.821+0000 7fc567bc9640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc56013d5d0 con 0x7fc56015b4a0 2023-10-26T01:32:06.825 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:06.821+0000 7fc565164640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc56006ce00 0x7fc5600708c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:06.825 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:06.821+0000 7fc565164640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc56006ce00 0x7fc5600708c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:44490/0 (socket says 172.21.15.169:44490) 2023-10-26T01:32:06.825 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:06.821+0000 7fc565164640 1 -- 172.21.15.169:0/2589962246 learned_addr learned my addr 172.21.15.169:0/2589962246 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:32:06.826 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:06.821+0000 7fc566166640 1 --2- 172.21.15.169:0/2589962246 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc56015b4a0 0x7fc560070e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:06.826 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:06.821+0000 7fc565965640 1 --2- 172.21.15.169:0/2589962246 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc56006ad90 0x7fc560151410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:06.826 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:06.822+0000 7fc565164640 1 -- 172.21.15.169:0/2589962246 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc56015b4a0 msgr2=0x7fc560070e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:06.826 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:06.822+0000 7fc565164640 1 --2- 172.21.15.169:0/2589962246 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc56015b4a0 0x7fc560070e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:06.827 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:06.822+0000 7fc565164640 1 -- 172.21.15.169:0/2589962246 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc56006ad90 msgr2=0x7fc560151410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:06.827 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:06.822+0000 7fc565164640 1 --2- 172.21.15.169:0/2589962246 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc56006ad90 0x7fc560151410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:06.827 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:06.822+0000 7fc565164640 1 -- 172.21.15.169:0/2589962246 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc550009560 con 0x7fc56006ce00 2023-10-26T01:32:06.827 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:06.822+0000 7fc565164640 1 --2- 172.21.15.169:0/2589962246 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc56006ce00 0x7fc5600708c0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fc550036040 tx=0x7fc55002da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:06.827 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:06.822+0000 7fc5567fc640 1 -- 172.21.15.169:0/2589962246 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc55002de60 con 0x7fc56006ce00 2023-10-26T01:32:06.828 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:06.822+0000 7fc5567fc640 1 -- 172.21.15.169:0/2589962246 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc5500036e0 con 0x7fc56006ce00 2023-10-26T01:32:06.828 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:06.822+0000 7fc567bc9640 1 -- 172.21.15.169:0/2589962246 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc56013d8b0 con 0x7fc56006ce00 2023-10-26T01:32:06.828 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:06.822+0000 7fc5567fc640 1 -- 172.21.15.169:0/2589962246 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc550003ab0 con 0x7fc56006ce00 2023-10-26T01:32:06.828 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:06.823+0000 7fc567bc9640 1 -- 172.21.15.169:0/2589962246 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5601543b0 con 0x7fc56006ce00 2023-10-26T01:32:06.829 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:06.824+0000 7fc567bc9640 1 -- 172.21.15.169:0/2589962246 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc56004db10 con 0x7fc56006ce00 2023-10-26T01:32:06.829 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:06.825+0000 7fc5567fc640 1 -- 172.21.15.169:0/2589962246 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fc550003c10 con 0x7fc56006ce00 2023-10-26T01:32:06.830 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:06.826+0000 7fc5567fc640 1 --2- 172.21.15.169:0/2589962246 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc54406ae40 0x7fc54406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:06.830 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:06.826+0000 7fc5567fc640 1 -- 172.21.15.169:0/2589962246 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc5500ac1d0 con 0x7fc56006ce00 2023-10-26T01:32:06.830 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:06.826+0000 7fc565965640 1 --2- 172.21.15.169:0/2589962246 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc54406ae40 0x7fc54406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:06.830 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:06.828+0000 7fc565965640 1 --2- 172.21.15.169:0/2589962246 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc54406ae40 0x7fc54406d300 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc55c00abd0 tx=0x7fc55c005f90 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:06.831 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:06.830+0000 7fc5567fc640 1 -- 172.21.15.169:0/2589962246 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc550077300 con 0x7fc56006ce00 2023-10-26T01:32:07.061 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:07.060+0000 7fc567bc9640 1 -- 172.21.15.169:0/2589962246 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fc56006b4b0 con 0x7fc56006ce00 2023-10-26T01:32:07.062 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:07.060+0000 7fc5567fc640 1 -- 172.21.15.169:0/2589962246 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v15) v1 ==== 53+0+73 (secure 0 0 0) 0x7fc550043060 con 0x7fc56006ce00 2023-10-26T01:32:07.062 INFO:teuthology.orchestra.run.smithi169.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-26T01:32:07.065 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:07.063+0000 7fc567bc9640 1 -- 172.21.15.169:0/2589962246 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc54406ae40 msgr2=0x7fc54406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:07.065 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:07.063+0000 7fc567bc9640 1 --2- 172.21.15.169:0/2589962246 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc54406ae40 0x7fc54406d300 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc55c00abd0 tx=0x7fc55c005f90 comp rx=0 tx=0).stop 2023-10-26T01:32:07.066 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:07.063+0000 7fc567bc9640 1 -- 172.21.15.169:0/2589962246 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc56006ce00 msgr2=0x7fc5600708c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:07.066 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:07.063+0000 7fc567bc9640 1 --2- 172.21.15.169:0/2589962246 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc56006ce00 0x7fc5600708c0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fc550036040 tx=0x7fc55002da60 comp rx=0 tx=0).stop 2023-10-26T01:32:07.066 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:07.063+0000 7fc567bc9640 1 -- 172.21.15.169:0/2589962246 shutdown_connections 2023-10-26T01:32:07.066 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:07.063+0000 7fc567bc9640 1 --2- 172.21.15.169:0/2589962246 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc54406ae40 0x7fc54406d300 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:07.066 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:07.063+0000 7fc567bc9640 1 --2- 172.21.15.169:0/2589962246 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc56006ce00 0x7fc5600708c0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:07.067 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:07.063+0000 7fc567bc9640 1 --2- 172.21.15.169:0/2589962246 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc56015b4a0 0x7fc560070e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:07.067 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:07.063+0000 7fc567bc9640 1 --2- 172.21.15.169:0/2589962246 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc56006ad90 0x7fc560151410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:07.067 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:07.063+0000 7fc567bc9640 1 -- 172.21.15.169:0/2589962246 >> 172.21.15.169:0/2589962246 conn(0x7fc560069930 msgr2=0x7fc560156080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:07.067 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:07.063+0000 7fc567bc9640 1 -- 172.21.15.169:0/2589962246 shutdown_connections 2023-10-26T01:32:07.067 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:07.063+0000 7fc567bc9640 1 -- 172.21.15.169:0/2589962246 wait complete. 2023-10-26T01:32:07.084 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-26T01:32:07.085 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-26T01:32:07.085 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi170.front.sepia.ceph.com 2023-10-26T01:32:07.085 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-26T01:32:07.086 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-26T01:32:07.086 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-26T01:32:07.086 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-26T01:32:07.087 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-26T01:32:07.087 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-26T01:32:07.088 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:32:07.088 DEBUG:teuthology.orchestra.run.smithi170:> ip addr 2023-10-26T01:32:07.152 INFO:teuthology.orchestra.run.smithi170.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-26T01:32:07.152 INFO:teuthology.orchestra.run.smithi170.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-26T01:32:07.152 INFO:teuthology.orchestra.run.smithi170.stdout: inet 127.0.0.1/8 scope host lo 2023-10-26T01:32:07.152 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:32:07.153 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 ::1/128 scope host 2023-10-26T01:32:07.153 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:32:07.153 INFO:teuthology.orchestra.run.smithi170.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-26T01:32:07.153 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:d9:3b:54 brd ff:ff:ff:ff:ff:ff 2023-10-26T01:32:07.154 INFO:teuthology.orchestra.run.smithi170.stdout: altname enp7s0f0 2023-10-26T01:32:07.154 INFO:teuthology.orchestra.run.smithi170.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-26T01:32:07.154 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:d9:3b:55 brd ff:ff:ff:ff:ff:ff 2023-10-26T01:32:07.154 INFO:teuthology.orchestra.run.smithi170.stdout: altname enp7s0f1 2023-10-26T01:32:07.154 INFO:teuthology.orchestra.run.smithi170.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-26T01:32:07.155 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:88:72:54 brd ff:ff:ff:ff:ff:ff 2023-10-26T01:32:07.155 INFO:teuthology.orchestra.run.smithi170.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-26T01:32:07.155 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:88:72:55 brd ff:ff:ff:ff:ff:ff 2023-10-26T01:32:07.155 INFO:teuthology.orchestra.run.smithi170.stdout: inet 172.21.15.170/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-26T01:32:07.155 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft 42353sec preferred_lft 42353sec 2023-10-26T01:32:07.156 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::ec4:7aff:fe88:7255/64 scope link 2023-10-26T01:32:07.156 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:32:07.156 INFO:teuthology.orchestra.run.smithi170.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-26T01:32:07.156 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 32:b5:d1:ee:10:36 brd ff:ff:ff:ff:ff:ff 2023-10-26T01:32:07.156 INFO:teuthology.orchestra.run.smithi170.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-26T01:32:07.157 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:32:07.157 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::28ed:d1ff:fe37:70d/64 scope link 2023-10-26T01:32:07.157 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:32:07.157 INFO:teuthology.orchestra.run.smithi170.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-26T01:32:07.157 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 32:b5:d1:ee:10:36 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-26T01:32:07.158 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::30b5:d1ff:feee:1036/64 scope link 2023-10-26T01:32:07.158 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:32:07.158 INFO:teuthology.orchestra.run.smithi170.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-26T01:32:07.158 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether b2:c1:53:19:69:e4 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-26T01:32:07.158 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::b0c1:53ff:fe19:69e4/64 scope link 2023-10-26T01:32:07.159 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:32:07.160 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:32:07.160 DEBUG:teuthology.orchestra.run.smithi170:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-26T01:32:07.258 INFO:teuthology.orchestra.run.smithi170.stdout:1 2023-10-26T01:32:07.262 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:32:07.262 DEBUG:teuthology.orchestra.run.smithi170:> route 2023-10-26T01:32:07.318 INFO:teuthology.orchestra.run.smithi170.stdout:Kernel IP routing table 2023-10-26T01:32:07.318 INFO:teuthology.orchestra.run.smithi170.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-26T01:32:07.318 INFO:teuthology.orchestra.run.smithi170.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-26T01:32:07.318 INFO:teuthology.orchestra.run.smithi170.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-26T01:32:07.318 INFO:teuthology.orchestra.run.smithi170.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-26T01:32:07.319 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:32:07.320 DEBUG:teuthology.orchestra.run.smithi170:> (cd / && exec bash -c ' 2023-10-26T01:32:07.320 DEBUG:teuthology.orchestra.run.smithi170:> set -e 2023-10-26T01:32:07.321 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-26T01:32:07.321 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-26T01:32:07.321 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-26T01:32:07.322 DEBUG:teuthology.orchestra.run.smithi170:> ') 2023-10-26T01:32:07.503 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:32:07.503 DEBUG:teuthology.orchestra.run.smithi170:> ip netns list 2023-10-26T01:32:07.554 INFO:teuthology.orchestra.run.smithi170.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-26T01:32:07.554 INFO:teuthology.orchestra.run.smithi170.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-26T01:32:07.555 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-26T01:32:07.556 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:32:07.556 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip link set brx.0 up 2023-10-26T01:32:07.652 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-26T01:32:07.653 DEBUG:teuthology.orchestra.run.smithi170:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-26T01:32:07.704 INFO:teuthology.orchestra.run.smithi170.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-26T01:32:07.705 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-26T01:32:07.705 DEBUG:teuthology.orchestra.run.smithi170:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-26T01:32:07.755 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-26T01:32:07.756 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:32:07.756 DEBUG:teuthology.orchestra.run.smithi170:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-26T01:32:07.896 INFO:teuthology.orchestra.run.smithi170.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-26T01:32:07.897 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-26T01:32:07.897 INFO:teuthology.orchestra.run.smithi170.stdout:invalid new device string format 2023-10-26T01:32:07.919 INFO:teuthology.orchestra.run.smithi170.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-26T01:32:07.920 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-26T01:32:07.920 INFO:teuthology.orchestra.run.smithi170.stdout:invalid new device string format 2023-10-26T01:32:07.920 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: resolved to: "172.21.15.169:3300,172.21.15.170:3300,172.21.15.170:3301" 2023-10-26T01:32:07.920 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-26T01:32:07.951 INFO:teuthology.orchestra.run.smithi170.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-26T01:32:07.952 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-26T01:32:07.952 INFO:teuthology.orchestra.run.smithi170.stdout:invalid new device string format 2023-10-26T01:32:07.952 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: resolved to: "172.21.15.169:3300,172.21.15.170:3300,172.21.15.170:3301" 2023-10-26T01:32:07.952 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-26T01:32:07.952 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: resolved to: "172.21.15.169:6789,172.21.15.170:6789,172.21.15.170:6790" 2023-10-26T01:32:07.953 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.169:6789,172.21.15.170:6789,172.21.15.170:6790:/ 2023-10-26T01:32:07.953 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=30203b28-ed8a-44e8-a9a6-a9bf25f980f1" will pass to kernel 2023-10-26T01:32:07.954 INFO:teuthology.orchestra.run.smithi170.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2023-10-26T01:32:07.954 INFO:teuthology.orchestra.run.smithi170.stdout: You just mounted a file system that supports labels which does not 2023-10-26T01:32:07.954 INFO:teuthology.orchestra.run.smithi170.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-10-26T01:32:07.955 INFO:teuthology.orchestra.run.smithi170.stdout: applications will generate AVC messages and not be allowed access to 2023-10-26T01:32:07.955 INFO:teuthology.orchestra.run.smithi170.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-10-26T01:32:07.960 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-26T01:32:07.961 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:32:07.961 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-26T01:32:08.042 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:32:08.043 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-26T01:32:08.090 INFO:teuthology.orchestra.run.smithi170.stdout:client5376 2023-10-26T01:32:08.091 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-26T01:32:08.091 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:32:08.136 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T01:32:08.366 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:08.364+0000 7f01e5525640 1 -- 172.21.15.169:0/2257773783 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f01c8003970 msgr2=0x7f01c8003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:08.366 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:08.364+0000 7f01e5525640 1 --2- 172.21.15.169:0/2257773783 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f01c8003970 0x7f01c8003d90 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f01d0006fd0 tx=0x7f01c80057f0 comp rx=0 tx=0).stop 2023-10-26T01:32:08.366 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:08.364+0000 7f01e5525640 1 -- 172.21.15.169:0/2257773783 shutdown_connections 2023-10-26T01:32:08.367 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:08.364+0000 7f01e5525640 1 --2- 172.21.15.169:0/2257773783 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f01c8003970 0x7f01c8003d90 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:08.367 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:08.364+0000 7f01e5525640 1 -- 172.21.15.169:0/2257773783 >> 172.21.15.169:0/2257773783 conn(0x7f01e0069930 msgr2=0x7f01e0069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:08.367 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:08.364+0000 7f01e5525640 1 -- 172.21.15.169:0/2257773783 shutdown_connections 2023-10-26T01:32:08.367 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:08.364+0000 7f01e5525640 1 -- 172.21.15.169:0/2257773783 wait complete. 2023-10-26T01:32:08.368 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:08.366+0000 7f01e5525640 1 Processor -- start 2023-10-26T01:32:08.368 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:08.366+0000 7f01e5525640 1 -- start start 2023-10-26T01:32:08.369 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:08.367+0000 7f01e5525640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f01c8003970 0x7f01e0149260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:08.369 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:08.367+0000 7f01e5525640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f01e0071520 0x7f01e006ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:08.370 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:08.367+0000 7f01e5525640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f01e006b270 0x7f01e0159b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:08.370 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:08.367+0000 7f01e5525640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01e014c0c0 con 0x7f01e006b270 2023-10-26T01:32:08.370 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:08.367+0000 7f01e5525640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01e015b520 con 0x7f01c8003970 2023-10-26T01:32:08.370 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:08.367+0000 7f01e5525640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01e006b6f0 con 0x7f01e0071520 2023-10-26T01:32:08.371 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:08.367+0000 7f01df7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f01e006b270 0x7f01e0159b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:08.371 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:08.367+0000 7f01df7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f01e006b270 0x7f01e0159b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:44512/0 (socket says 172.21.15.169:44512) 2023-10-26T01:32:08.371 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:08.367+0000 7f01df7fe640 1 -- 172.21.15.169:0/3130981299 learned_addr learned my addr 172.21.15.169:0/3130981299 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:32:08.371 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:08.368+0000 7f01deffd640 1 --2- 172.21.15.169:0/3130981299 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f01c8003970 0x7f01e0149260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:08.371 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:08.368+0000 7f01de7fc640 1 --2- 172.21.15.169:0/3130981299 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f01e0071520 0x7f01e006ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:08.371 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:08.368+0000 7f01df7fe640 1 -- 172.21.15.169:0/3130981299 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f01e0071520 msgr2=0x7f01e006ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:08.372 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:08.368+0000 7f01df7fe640 1 --2- 172.21.15.169:0/3130981299 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f01e0071520 0x7f01e006ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:08.372 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:08.368+0000 7f01df7fe640 1 -- 172.21.15.169:0/3130981299 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f01c8003970 msgr2=0x7f01e0149260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:08.372 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:08.368+0000 7f01df7fe640 1 --2- 172.21.15.169:0/3130981299 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f01c8003970 0x7f01e0149260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:08.372 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:08.368+0000 7f01df7fe640 1 -- 172.21.15.169:0/3130981299 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01e006f220 con 0x7f01e006b270 2023-10-26T01:32:08.372 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:08.368+0000 7f01df7fe640 1 --2- 172.21.15.169:0/3130981299 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f01e006b270 0x7f01e0159b20 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f01e00ac070 tx=0x7f01d800fef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:08.374 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:08.368+0000 7f01c77fe640 1 -- 172.21.15.169:0/3130981299 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d8003d80 con 0x7f01e006b270 2023-10-26T01:32:08.374 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:08.368+0000 7f01c77fe640 1 -- 172.21.15.169:0/3130981299 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f01d800d990 con 0x7f01e006b270 2023-10-26T01:32:08.374 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:08.368+0000 7f01c77fe640 1 -- 172.21.15.169:0/3130981299 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d800ddc0 con 0x7f01e006b270 2023-10-26T01:32:08.374 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:08.368+0000 7f01e5525640 1 -- 172.21.15.169:0/3130981299 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01e015b730 con 0x7f01e006b270 2023-10-26T01:32:08.374 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:08.369+0000 7f01e5525640 1 -- 172.21.15.169:0/3130981299 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01e006eeb0 con 0x7f01e006b270 2023-10-26T01:32:08.375 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:08.370+0000 7f01e5525640 1 -- 172.21.15.169:0/3130981299 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01e015bc50 con 0x7f01e006b270 2023-10-26T01:32:08.375 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:08.371+0000 7f01c77fe640 1 -- 172.21.15.169:0/3130981299 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f01d8004030 con 0x7f01e006b270 2023-10-26T01:32:08.375 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:08.372+0000 7f01c77fe640 1 --2- 172.21.15.169:0/3130981299 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f01cc06ae70 0x7f01cc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:08.375 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:08.372+0000 7f01c77fe640 1 -- 172.21.15.169:0/3130981299 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f01d8089fe0 con 0x7f01e006b270 2023-10-26T01:32:08.376 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:08.373+0000 7f01deffd640 1 --2- 172.21.15.169:0/3130981299 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f01cc06ae70 0x7f01cc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:08.376 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:08.374+0000 7f01deffd640 1 --2- 172.21.15.169:0/3130981299 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f01cc06ae70 0x7f01cc06d330 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f01d0036000 tx=0x7f01d0030000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:08.378 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:08.377+0000 7f01c77fe640 1 -- 172.21.15.169:0/3130981299 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f01d80550a0 con 0x7f01e006b270 2023-10-26T01:32:08.605 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:08.603+0000 7f01e5525640 1 -- 172.21.15.169:0/3130981299 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f01e006f070 con 0x7f01e006b270 2023-10-26T01:32:08.606 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:08.604+0000 7f01c77fe640 1 -- 172.21.15.169:0/3130981299 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4913 (secure 0 0 0) 0x7f01d80586c0 con 0x7f01e006b270 2023-10-26T01:32:08.606 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:32:08.606 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4640,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/61096113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":61096113},{"type":"v1","addr":"172.21.15.170:6836","nonce":61096113}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4762,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2825046244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2825046244},{"type":"v1","addr":"172.21.15.169:6833","nonce":2825046244}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4777,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/258866976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":258866976},{"type":"v1","addr":"172.21.15.170:6837","nonce":258866976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T01:31:57.759251+0000","modified":"2023-10-26T01:32:00.831213+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5196},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5196":{"gid":5196,"name":"c","rank":0,"incarnation":11,"state":"up:active","state_seq":4,"addr":"172.21.15.169:6835/871978528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":871978528},{"type":"v1","addr":"172.21.15.169:6835","nonce":871978528}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-26T01:32:08.607 INFO:teuthology.orchestra.run.smithi169.stderr:dumped fsmap epoch 15 2023-10-26T01:32:08.608 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:08.606+0000 7f01e5525640 1 -- 172.21.15.169:0/3130981299 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f01cc06ae70 msgr2=0x7f01cc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:08.609 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:08.606+0000 7f01e5525640 1 --2- 172.21.15.169:0/3130981299 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f01cc06ae70 0x7f01cc06d330 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f01d0036000 tx=0x7f01d0030000 comp rx=0 tx=0).stop 2023-10-26T01:32:08.609 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:08.606+0000 7f01e5525640 1 -- 172.21.15.169:0/3130981299 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f01e006b270 msgr2=0x7f01e0159b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:08.609 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:08.606+0000 7f01e5525640 1 --2- 172.21.15.169:0/3130981299 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f01e006b270 0x7f01e0159b20 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f01e00ac070 tx=0x7f01d800fef0 comp rx=0 tx=0).stop 2023-10-26T01:32:08.609 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:08.606+0000 7f01e5525640 1 -- 172.21.15.169:0/3130981299 shutdown_connections 2023-10-26T01:32:08.609 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:08.607+0000 7f01e5525640 1 --2- 172.21.15.169:0/3130981299 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f01cc06ae70 0x7f01cc06d330 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:08.610 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:08.607+0000 7f01e5525640 1 --2- 172.21.15.169:0/3130981299 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f01e006b270 0x7f01e0159b20 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:08.610 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:08.607+0000 7f01e5525640 1 --2- 172.21.15.169:0/3130981299 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f01e0071520 0x7f01e006ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:08.610 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:08.607+0000 7f01e5525640 1 --2- 172.21.15.169:0/3130981299 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f01c8003970 0x7f01e0149260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:08.610 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:08.607+0000 7f01e5525640 1 -- 172.21.15.169:0/3130981299 >> 172.21.15.169:0/3130981299 conn(0x7f01e0069930 msgr2=0x7f01e00ad210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:08.610 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:08.607+0000 7f01e5525640 1 -- 172.21.15.169:0/3130981299 shutdown_connections 2023-10-26T01:32:08.611 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:08.607+0000 7f01e5525640 1 -- 172.21.15.169:0/3130981299 wait complete. 2023-10-26T01:32:08.628 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2023-10-26T01:32:08.856 INFO:teuthology.orchestra.run.smithi169.stdout:{"cluster_fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","whoami":0,"id":5196,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":10.519378531999999,"mdsmap_epoch":15,"osdmap_epoch":35,"osdmap_epoch_barrier":35,"uptime":20.333019004000001} 2023-10-26T01:32:08.857 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "30203b28-ed8a-44e8-a9a6-a9bf25f980f1", "fs_name": "cephfs", "id": 5196, "mdsmap_epoch": 15, "osdmap_epoch": 35, "osdmap_epoch_barrier": 35, "rank_uptime": 10.519378532, "state": "up:active", "uptime": 20.333019004, "want_state": "up:active", "whoami": 0 } 2023-10-26T01:32:08.857 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph df --format=json-pretty 2023-10-26T01:32:09.079 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:09.077+0000 7f248e355640 1 -- 172.21.15.169:0/1361711949 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24880cd720 msgr2=0x7f24880cdbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:09.080 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:09.077+0000 7f248e355640 1 --2- 172.21.15.169:0/1361711949 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24880cd720 0x7f24880cdbb0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f24880aff40 tx=0x7f248002c140 comp rx=0 tx=0).stop 2023-10-26T01:32:09.080 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:09.077+0000 7f248e355640 1 -- 172.21.15.169:0/1361711949 shutdown_connections 2023-10-26T01:32:09.080 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:09.077+0000 7f248e355640 1 --2- 172.21.15.169:0/1361711949 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24880cd720 0x7f24880cdbb0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:09.080 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:09.077+0000 7f248e355640 1 --2- 172.21.15.169:0/1361711949 >> v2:172.21.15.170:3300/0 conn(0x7f2488158850 0x7f24880aab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:09.081 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:09.077+0000 7f248e355640 1 --2- 172.21.15.169:0/1361711949 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f248806ce20 0x7f248814e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:09.081 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:09.077+0000 7f248e355640 1 -- 172.21.15.169:0/1361711949 >> 172.21.15.169:0/1361711949 conn(0x7f2488069930 msgr2=0x7f2488069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:09.081 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:09.077+0000 7f248e355640 1 -- 172.21.15.169:0/1361711949 shutdown_connections 2023-10-26T01:32:09.082 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:09.078+0000 7f248e355640 1 -- 172.21.15.169:0/1361711949 wait complete. 2023-10-26T01:32:09.085 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:09.079+0000 7f248e355640 1 Processor -- start 2023-10-26T01:32:09.085 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:09.079+0000 7f248e355640 1 -- start start 2023-10-26T01:32:09.086 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:09.080+0000 7f248e355640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f248806ce20 0x7f24881427d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:09.086 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:09.080+0000 7f248e355640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f24880cd720 0x7f24881e42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:09.086 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:09.080+0000 7f248e355640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2488158850 0x7f24881e4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:09.087 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:09.080+0000 7f248e355640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2488154a10 con 0x7f2488158850 2023-10-26T01:32:09.087 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:09.080+0000 7f248e355640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f248814ea30 con 0x7f24880cd720 2023-10-26T01:32:09.087 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:09.080+0000 7f248e355640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2488153e00 con 0x7f248806ce20 2023-10-26T01:32:09.088 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:09.081+0000 7f248c8f2640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2488158850 0x7f24881e4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:09.088 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:09.081+0000 7f248c8f2640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2488158850 0x7f24881e4820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:44536/0 (socket says 172.21.15.169:44536) 2023-10-26T01:32:09.088 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:09.081+0000 7f248c8f2640 1 -- 172.21.15.169:0/3493723593 learned_addr learned my addr 172.21.15.169:0/3493723593 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:32:09.089 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:09.081+0000 7f247ffff640 1 --2- 172.21.15.169:0/3493723593 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f24880cd720 0x7f24881e42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:09.089 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:09.081+0000 7f248c8f2640 1 -- 172.21.15.169:0/3493723593 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f248806ce20 msgr2=0x7f24881427d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:09.089 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:09.081+0000 7f248c8f2640 1 --2- 172.21.15.169:0/3493723593 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f248806ce20 0x7f24881427d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:09.090 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:09.081+0000 7f248c8f2640 1 -- 172.21.15.169:0/3493723593 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f24880cd720 msgr2=0x7f24881e42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:09.090 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:09.081+0000 7f248c8f2640 1 --2- 172.21.15.169:0/3493723593 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f24880cd720 0x7f24881e42e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:09.091 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:09.081+0000 7f248c8f2640 1 -- 172.21.15.169:0/3493723593 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2480009900 con 0x7f2488158850 2023-10-26T01:32:09.091 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:09.081+0000 7f248c8f2640 1 --2- 172.21.15.169:0/3493723593 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2488158850 0x7f24881e4820 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f2480037040 tx=0x7f2480003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:09.091 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:09.082+0000 7f24857fa640 1 -- 172.21.15.169:0/3493723593 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2480004240 con 0x7f2488158850 2023-10-26T01:32:09.091 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:09.082+0000 7f24857fa640 1 -- 172.21.15.169:0/3493723593 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2480004490 con 0x7f2488158850 2023-10-26T01:32:09.092 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:09.082+0000 7f248e355640 1 -- 172.21.15.169:0/3493723593 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24881540e0 con 0x7f2488158850 2023-10-26T01:32:09.092 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:09.082+0000 7f24857fa640 1 -- 172.21.15.169:0/3493723593 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2480004880 con 0x7f2488158850 2023-10-26T01:32:09.092 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:09.082+0000 7f248e355640 1 -- 172.21.15.169:0/3493723593 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24881546a0 con 0x7f2488158850 2023-10-26T01:32:09.092 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:09.083+0000 7f248e355640 1 -- 172.21.15.169:0/3493723593 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f248804db10 con 0x7f2488158850 2023-10-26T01:32:09.093 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:09.084+0000 7f24857fa640 1 -- 172.21.15.169:0/3493723593 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f248003b040 con 0x7f2488158850 2023-10-26T01:32:09.094 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:09.086+0000 7f24857fa640 1 --2- 172.21.15.169:0/3493723593 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f246806ae40 0x7f246806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:09.094 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:09.086+0000 7f24857fa640 1 -- 172.21.15.169:0/3493723593 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f24800ac500 con 0x7f2488158850 2023-10-26T01:32:09.094 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:09.086+0000 7f2487fff640 1 --2- 172.21.15.169:0/3493723593 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f246806ae40 0x7f246806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:09.094 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:09.087+0000 7f2487fff640 1 --2- 172.21.15.169:0/3493723593 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f246806ae40 0x7f246806d300 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2488158620 tx=0x7f2478005b60 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:09.094 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:09.090+0000 7f24857fa640 1 -- 172.21.15.169:0/3493723593 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2480077510 con 0x7f2488158850 2023-10-26T01:32:09.350 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:09.348+0000 7f248e355640 1 -- 172.21.15.169:0/3493723593 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "df", "format": "json-pretty"} v 0) v1 -- 0x7f2488154b50 con 0x7f2488158850 2023-10-26T01:32:09.350 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:09.349+0000 7f24857fa640 1 -- 172.21.15.169:0/3493723593 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "df", "format": "json-pretty"}]=0 v0) v1 ==== 75+0+1619 (secure 0 0 0) 0x7f248003d3f0 con 0x7f2488158850 2023-10-26T01:32:09.351 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:32:09.351 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2023-10-26T01:32:09.351 INFO:teuthology.orchestra.run.smithi169.stdout: "stats": { 2023-10-26T01:32:09.351 INFO:teuthology.orchestra.run.smithi169.stdout: "total_bytes": 1600000000, 2023-10-26T01:32:09.352 INFO:teuthology.orchestra.run.smithi169.stdout: "total_avail_bytes": 1597562944, 2023-10-26T01:32:09.352 INFO:teuthology.orchestra.run.smithi169.stdout: "total_used_bytes": 2437056, 2023-10-26T01:32:09.352 INFO:teuthology.orchestra.run.smithi169.stdout: "total_used_raw_bytes": 2437056, 2023-10-26T01:32:09.352 INFO:teuthology.orchestra.run.smithi169.stdout: "total_used_raw_ratio": 0.0015231600264087319, 2023-10-26T01:32:09.352 INFO:teuthology.orchestra.run.smithi169.stdout: "num_osds": 8, 2023-10-26T01:32:09.352 INFO:teuthology.orchestra.run.smithi169.stdout: "num_per_pool_osds": 0, 2023-10-26T01:32:09.353 INFO:teuthology.orchestra.run.smithi169.stdout: "num_per_pool_omap_osds": 0 2023-10-26T01:32:09.353 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:09.353 INFO:teuthology.orchestra.run.smithi169.stdout: "stats_by_class": { 2023-10-26T01:32:09.353 INFO:teuthology.orchestra.run.smithi169.stdout: "hdd": { 2023-10-26T01:32:09.353 INFO:teuthology.orchestra.run.smithi169.stdout: "total_bytes": 1600000000, 2023-10-26T01:32:09.354 INFO:teuthology.orchestra.run.smithi169.stdout: "total_avail_bytes": 1597562944, 2023-10-26T01:32:09.354 INFO:teuthology.orchestra.run.smithi169.stdout: "total_used_bytes": 2437056, 2023-10-26T01:32:09.354 INFO:teuthology.orchestra.run.smithi169.stdout: "total_used_raw_bytes": 2437056, 2023-10-26T01:32:09.354 INFO:teuthology.orchestra.run.smithi169.stdout: "total_used_raw_ratio": 0.0015231600264087319 2023-10-26T01:32:09.354 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:32:09.355 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:09.355 INFO:teuthology.orchestra.run.smithi169.stdout: "pools": [ 2023-10-26T01:32:09.355 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:32:09.355 INFO:teuthology.orchestra.run.smithi169.stdout: "name": ".mgr", 2023-10-26T01:32:09.355 INFO:teuthology.orchestra.run.smithi169.stdout: "id": 1, 2023-10-26T01:32:09.355 INFO:teuthology.orchestra.run.smithi169.stdout: "stats": { 2023-10-26T01:32:09.356 INFO:teuthology.orchestra.run.smithi169.stdout: "stored": 590368, 2023-10-26T01:32:09.356 INFO:teuthology.orchestra.run.smithi169.stdout: "objects": 2, 2023-10-26T01:32:09.356 INFO:teuthology.orchestra.run.smithi169.stdout: "kb_used": 577, 2023-10-26T01:32:09.356 INFO:teuthology.orchestra.run.smithi169.stdout: "bytes_used": 590368, 2023-10-26T01:32:09.356 INFO:teuthology.orchestra.run.smithi169.stdout: "percent_used": 0.0005296607268974185, 2023-10-26T01:32:09.356 INFO:teuthology.orchestra.run.smithi169.stdout: "max_avail": 557012544 2023-10-26T01:32:09.357 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:32:09.357 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:09.357 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:32:09.357 INFO:teuthology.orchestra.run.smithi169.stdout: "name": "cephfs_metadata", 2023-10-26T01:32:09.357 INFO:teuthology.orchestra.run.smithi169.stdout: "id": 4, 2023-10-26T01:32:09.358 INFO:teuthology.orchestra.run.smithi169.stdout: "stats": { 2023-10-26T01:32:09.358 INFO:teuthology.orchestra.run.smithi169.stdout: "stored": 2668, 2023-10-26T01:32:09.358 INFO:teuthology.orchestra.run.smithi169.stdout: "objects": 22, 2023-10-26T01:32:09.358 INFO:teuthology.orchestra.run.smithi169.stdout: "kb_used": 3, 2023-10-26T01:32:09.358 INFO:teuthology.orchestra.run.smithi169.stdout: "bytes_used": 2668, 2023-10-26T01:32:09.359 INFO:teuthology.orchestra.run.smithi169.stdout: "percent_used": 2.3949132810230367e-06, 2023-10-26T01:32:09.359 INFO:teuthology.orchestra.run.smithi169.stdout: "max_avail": 557012544 2023-10-26T01:32:09.359 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:32:09.359 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:09.359 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:32:09.360 INFO:teuthology.orchestra.run.smithi169.stdout: "name": "cephfs_data", 2023-10-26T01:32:09.360 INFO:teuthology.orchestra.run.smithi169.stdout: "id": 5, 2023-10-26T01:32:09.360 INFO:teuthology.orchestra.run.smithi169.stdout: "stats": { 2023-10-26T01:32:09.360 INFO:teuthology.orchestra.run.smithi169.stdout: "stored": 0, 2023-10-26T01:32:09.360 INFO:teuthology.orchestra.run.smithi169.stdout: "objects": 0, 2023-10-26T01:32:09.360 INFO:teuthology.orchestra.run.smithi169.stdout: "kb_used": 0, 2023-10-26T01:32:09.361 INFO:teuthology.orchestra.run.smithi169.stdout: "bytes_used": 0, 2023-10-26T01:32:09.361 INFO:teuthology.orchestra.run.smithi169.stdout: "percent_used": 0, 2023-10-26T01:32:09.361 INFO:teuthology.orchestra.run.smithi169.stdout: "max_avail": 557012544 2023-10-26T01:32:09.361 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:32:09.361 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:32:09.362 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:32:09.362 INFO:teuthology.orchestra.run.smithi169.stdout:} 2023-10-26T01:32:09.363 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:09.351+0000 7f248e355640 1 -- 172.21.15.169:0/3493723593 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f246806ae40 msgr2=0x7f246806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:09.363 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:09.351+0000 7f248e355640 1 --2- 172.21.15.169:0/3493723593 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f246806ae40 0x7f246806d300 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2488158620 tx=0x7f2478005b60 comp rx=0 tx=0).stop 2023-10-26T01:32:09.363 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:09.351+0000 7f248e355640 1 -- 172.21.15.169:0/3493723593 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2488158850 msgr2=0x7f24881e4820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:09.363 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:09.351+0000 7f248e355640 1 --2- 172.21.15.169:0/3493723593 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2488158850 0x7f24881e4820 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f2480037040 tx=0x7f2480003ba0 comp rx=0 tx=0).stop 2023-10-26T01:32:09.364 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:09.352+0000 7f248e355640 1 -- 172.21.15.169:0/3493723593 shutdown_connections 2023-10-26T01:32:09.364 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:09.352+0000 7f248e355640 1 --2- 172.21.15.169:0/3493723593 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f246806ae40 0x7f246806d300 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:09.364 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:09.352+0000 7f248e355640 1 --2- 172.21.15.169:0/3493723593 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2488158850 0x7f24881e4820 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:09.364 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:09.352+0000 7f248e355640 1 --2- 172.21.15.169:0/3493723593 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f24880cd720 0x7f24881e42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:09.364 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:09.352+0000 7f248e355640 1 --2- 172.21.15.169:0/3493723593 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f248806ce20 0x7f24881427d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:09.365 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:09.352+0000 7f248e355640 1 -- 172.21.15.169:0/3493723593 >> 172.21.15.169:0/3493723593 conn(0x7f2488069930 msgr2=0x7f248814d5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:09.365 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:09.352+0000 7f248e355640 1 -- 172.21.15.169:0/3493723593 shutdown_connections 2023-10-26T01:32:09.365 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:09.352+0000 7f248e355640 1 -- 172.21.15.169:0/3493723593 wait complete. 2023-10-26T01:32:09.371 DEBUG:teuthology.orchestra.run.smithi169:> stdin-killer -- python3 -c ' 2023-10-26T01:32:09.372 DEBUG:teuthology.orchestra.run.smithi169:> import os 2023-10-26T01:32:09.372 DEBUG:teuthology.orchestra.run.smithi169:> fd = os.open("/home/ubuntu/cephtest/mnt.0/foo", os.O_CREAT | os.O_RDWR, 0O600) 2023-10-26T01:32:09.372 DEBUG:teuthology.orchestra.run.smithi169:> os.fchmod(fd, 0O666) 2023-10-26T01:32:09.373 DEBUG:teuthology.orchestra.run.smithi169:> os.fsync(fd) 2023-10-26T01:32:09.373 DEBUG:teuthology.orchestra.run.smithi169:> os.close(fd) 2023-10-26T01:32:09.373 DEBUG:teuthology.orchestra.run.smithi169:> ' 2023-10-26T01:32:09.526 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T01:32:09.536 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:32:09.536 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-26T01:32:09.637 INFO:teuthology.orchestra.run.smithi169.stdout:client4712 2023-10-26T01:32:09.637 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:32:09.638 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/30203b28-ed8a-44e8-a9a6-a9bf25f980f1.client4712/osdmap') 2023-10-26T01:32:09.727 INFO:teuthology.orchestra.run.smithi169.stdout:epoch 35 barrier 35 flags 0x588000 2023-10-26T01:32:09.728 INFO:teuthology.orchestra.run.smithi169.stdout:pool 1 '.mgr' type 1 size 2 min_size 1 pg_num 1 pg_num_mask 0 flags 0x1 lfor 0 read_tier -1 write_tier -1 2023-10-26T01:32:09.728 INFO:teuthology.orchestra.run.smithi169.stdout:pool 4 'cephfs_metadata' type 1 size 2 min_size 1 pg_num 32 pg_num_mask 31 flags 0x1 lfor 0 read_tier -1 write_tier -1 2023-10-26T01:32:09.728 INFO:teuthology.orchestra.run.smithi169.stdout:pool 5 'cephfs_data' type 1 size 2 min_size 1 pg_num 64 pg_num_mask 63 flags 0x1 lfor 0 read_tier -1 write_tier -1 2023-10-26T01:32:09.728 INFO:teuthology.orchestra.run.smithi169.stdout:osd0 (1)172.21.15.169:6817 100% (exists, up) 100% -1 2023-10-26T01:32:09.729 INFO:teuthology.orchestra.run.smithi169.stdout:osd1 (1)172.21.15.169:6809 100% (exists, up) 100% -1 2023-10-26T01:32:09.729 INFO:teuthology.orchestra.run.smithi169.stdout:osd2 (1)172.21.15.169:6801 100% (exists, up) 100% -1 2023-10-26T01:32:09.729 INFO:teuthology.orchestra.run.smithi169.stdout:osd3 (1)172.21.15.169:6825 100% (exists, up) 100% -1 2023-10-26T01:32:09.729 INFO:teuthology.orchestra.run.smithi169.stdout:osd4 (1)172.21.15.170:6817 100% (exists, up) 100% -1 2023-10-26T01:32:09.730 INFO:teuthology.orchestra.run.smithi169.stdout:osd5 (1)172.21.15.170:6809 100% (exists, up) 100% -1 2023-10-26T01:32:09.730 INFO:teuthology.orchestra.run.smithi169.stdout:osd6 (1)172.21.15.170:6801 100% (exists, up) 100% -1 2023-10-26T01:32:09.730 INFO:teuthology.orchestra.run.smithi169.stdout:osd7 (1)172.21.15.170:6822 100% (exists, up) 100% -1 2023-10-26T01:32:09.731 INFO:teuthology.orchestra.run.smithi169.stderr:1+1 records in 2023-10-26T01:32:09.731 INFO:teuthology.orchestra.run.smithi169.stderr:1+1 records out 2023-10-26T01:32:09.731 INFO:teuthology.orchestra.run.smithi169.stderr:799 bytes copied, 0.000155124 s, 5.2 MB/s 2023-10-26T01:32:09.732 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set pause 2023-10-26T01:32:09.960 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:09.958+0000 7f67b9754640 1 -- 172.21.15.169:0/3018548248 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67b40b0e50 msgr2=0x7f67b415d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:09.960 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:09.958+0000 7f67b9754640 1 --2- 172.21.15.169:0/3018548248 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67b40b0e50 0x7f67b415d610 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f67ac005fd0 tx=0x7f67ac02be10 comp rx=0 tx=0).stop 2023-10-26T01:32:09.961 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:09.958+0000 7f67b9754640 1 -- 172.21.15.169:0/3018548248 shutdown_connections 2023-10-26T01:32:09.961 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:09.958+0000 7f67b9754640 1 --2- 172.21.15.169:0/3018548248 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67b40b0e50 0x7f67b415d610 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:09.961 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:09.958+0000 7f67b9754640 1 --2- 172.21.15.169:0/3018548248 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f67b406a010 0x7f67b415d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:09.961 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:09.958+0000 7f67b9754640 1 -- 172.21.15.169:0/3018548248 >> 172.21.15.169:0/3018548248 conn(0x7f67b40af110 msgr2=0x7f67b40af520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:09.961 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:09.958+0000 7f67b9754640 1 -- 172.21.15.169:0/3018548248 shutdown_connections 2023-10-26T01:32:09.962 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:09.958+0000 7f67b9754640 1 -- 172.21.15.169:0/3018548248 wait complete. 2023-10-26T01:32:09.962 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:09.960+0000 7f67b9754640 1 Processor -- start 2023-10-26T01:32:09.962 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:09.960+0000 7f67b9754640 1 -- start start 2023-10-26T01:32:09.964 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:09.961+0000 7f67b9754640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f67b406a010 0x7f67b4096280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:09.965 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:09.961+0000 7f67b9754640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f67b40b0e50 0x7f67b40980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:09.965 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:09.961+0000 7f67b9754640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67b4153b90 0x7f67b40ab930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:09.965 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:09.961+0000 7f67b9754640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67b4153450 con 0x7f67b406a010 2023-10-26T01:32:09.965 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:09.961+0000 7f67b9754640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67b41f0a10 con 0x7f67b4153b90 2023-10-26T01:32:09.966 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:09.961+0000 7f67b9754640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67b41efe90 con 0x7f67b40b0e50 2023-10-26T01:32:09.966 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:09.961+0000 7f67b3fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f67b406a010 0x7f67b4096280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:09.966 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:09.961+0000 7f67b3fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f67b406a010 0x7f67b4096280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:44538/0 (socket says 172.21.15.169:44538) 2023-10-26T01:32:09.966 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:09.962+0000 7f67b3fff640 1 -- 172.21.15.169:0/385762200 learned_addr learned my addr 172.21.15.169:0/385762200 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:32:09.966 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:09.962+0000 7f67b8f53640 1 --2- 172.21.15.169:0/385762200 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67b4153b90 0x7f67b40ab930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:09.967 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:09.962+0000 7f67b37fe640 1 --2- 172.21.15.169:0/385762200 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f67b40b0e50 0x7f67b40980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:09.967 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:09.962+0000 7f67b3fff640 1 -- 172.21.15.169:0/385762200 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f67b40b0e50 msgr2=0x7f67b40980a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:09.967 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:09.962+0000 7f67b3fff640 1 --2- 172.21.15.169:0/385762200 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f67b40b0e50 0x7f67b40980a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:09.967 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:09.962+0000 7f67b3fff640 1 -- 172.21.15.169:0/385762200 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67b4153b90 msgr2=0x7f67b40ab930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:09.967 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:09.962+0000 7f67b3fff640 1 --2- 172.21.15.169:0/385762200 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67b4153b90 0x7f67b40ab930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:09.968 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:09.962+0000 7f67b3fff640 1 -- 172.21.15.169:0/385762200 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67ac009560 con 0x7f67b406a010 2023-10-26T01:32:09.968 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:09.962+0000 7f67b37fe640 1 --2- 172.21.15.169:0/385762200 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f67b40b0e50 0x7f67b40980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:32:09.968 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:09.962+0000 7f67b3fff640 1 --2- 172.21.15.169:0/385762200 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f67b406a010 0x7f67b4096280 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f679c00c400 tx=0x7f679c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:09.968 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:09.963+0000 7f67b0ff9640 1 -- 172.21.15.169:0/385762200 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f679c0052d0 con 0x7f67b406a010 2023-10-26T01:32:09.968 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:09.963+0000 7f67b0ff9640 1 -- 172.21.15.169:0/385762200 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f679c0035f0 con 0x7f67b406a010 2023-10-26T01:32:09.969 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:09.963+0000 7f67b9754640 1 -- 172.21.15.169:0/385762200 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67b41f0170 con 0x7f67b406a010 2023-10-26T01:32:09.969 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:09.963+0000 7f67b0ff9640 1 -- 172.21.15.169:0/385762200 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f679c003a00 con 0x7f67b406a010 2023-10-26T01:32:09.969 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:09.963+0000 7f67b9754640 1 -- 172.21.15.169:0/385762200 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67b41f0730 con 0x7f67b406a010 2023-10-26T01:32:09.970 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:09.964+0000 7f67b9754640 1 -- 172.21.15.169:0/385762200 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67b4073c10 con 0x7f67b406a010 2023-10-26T01:32:09.970 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:09.965+0000 7f67b0ff9640 1 -- 172.21.15.169:0/385762200 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f679c003b60 con 0x7f67b406a010 2023-10-26T01:32:09.970 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:09.967+0000 7f67b0ff9640 1 --2- 172.21.15.169:0/385762200 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f679406ae40 0x7f679406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:09.970 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:09.967+0000 7f67b0ff9640 1 -- 172.21.15.169:0/385762200 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f679c088e00 con 0x7f67b406a010 2023-10-26T01:32:09.971 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:09.967+0000 7f67b37fe640 1 --2- 172.21.15.169:0/385762200 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f679406ae40 0x7f679406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:09.971 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:09.968+0000 7f67b37fe640 1 --2- 172.21.15.169:0/385762200 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f679406ae40 0x7f679406d300 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f67b40cd7f0 tx=0x7f67a4009500 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:09.973 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:09.971+0000 7f67b0ff9640 1 -- 172.21.15.169:0/385762200 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f679c053ec0 con 0x7f67b406a010 2023-10-26T01:32:10.172 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:10.171+0000 7f67b9754640 1 -- 172.21.15.169:0/385762200 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd set", "key": "pause"} v 0) v1 -- 0x7f67b4154df0 con 0x7f67b406a010 2023-10-26T01:32:10.944 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:10.942+0000 7f67b0ff9640 1 -- 172.21.15.169:0/385762200 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "pause"}]=0 pauserd,pausewr is set v36) v1 ==== 93+0+0 (secure 0 0 0) 0x7f679c0574e0 con 0x7f67b406a010 2023-10-26T01:32:10.945 INFO:teuthology.orchestra.run.smithi169.stderr:pauserd,pausewr is set 2023-10-26T01:32:10.948 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:10.947+0000 7f67b9754640 1 -- 172.21.15.169:0/385762200 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f679406ae40 msgr2=0x7f679406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:10.949 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:10.947+0000 7f67b9754640 1 --2- 172.21.15.169:0/385762200 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f679406ae40 0x7f679406d300 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f67b40cd7f0 tx=0x7f67a4009500 comp rx=0 tx=0).stop 2023-10-26T01:32:10.949 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:10.947+0000 7f67b9754640 1 -- 172.21.15.169:0/385762200 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f67b406a010 msgr2=0x7f67b4096280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:10.949 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:10.947+0000 7f67b9754640 1 --2- 172.21.15.169:0/385762200 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f67b406a010 0x7f67b4096280 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f679c00c400 tx=0x7f679c00c8c0 comp rx=0 tx=0).stop 2023-10-26T01:32:10.949 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:10.947+0000 7f67b9754640 1 -- 172.21.15.169:0/385762200 shutdown_connections 2023-10-26T01:32:10.949 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:10.947+0000 7f67b9754640 1 --2- 172.21.15.169:0/385762200 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f679406ae40 0x7f679406d300 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:10.950 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:10.947+0000 7f67b9754640 1 --2- 172.21.15.169:0/385762200 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67b4153b90 0x7f67b40ab930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:10.950 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:10.947+0000 7f67b9754640 1 --2- 172.21.15.169:0/385762200 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f67b40b0e50 0x7f67b40980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:10.950 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:10.947+0000 7f67b9754640 1 --2- 172.21.15.169:0/385762200 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f67b406a010 0x7f67b4096280 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:10.950 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:10.947+0000 7f67b9754640 1 -- 172.21.15.169:0/385762200 >> 172.21.15.169:0/385762200 conn(0x7f67b40af110 msgr2=0x7f67b40c91e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:10.951 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:10.947+0000 7f67b9754640 1 -- 172.21.15.169:0/385762200 shutdown_connections 2023-10-26T01:32:10.951 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:10.947+0000 7f67b9754640 1 -- 172.21.15.169:0/385762200 wait complete. 2023-10-26T01:32:10.971 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset pause 2023-10-26T01:32:11.194 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:11.192+0000 7f260c27c640 1 -- 172.21.15.169:0/62632341 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26040cde20 msgr2=0x7f2604166630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:11.194 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:11.192+0000 7f260c27c640 1 --2- 172.21.15.169:0/62632341 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26040cde20 0x7f2604166630 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f25fc005fd0 tx=0x7f25fc02c0e0 comp rx=0 tx=0).stop 2023-10-26T01:32:11.194 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:11.192+0000 7f260c27c640 1 -- 172.21.15.169:0/62632341 shutdown_connections 2023-10-26T01:32:11.195 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:11.192+0000 7f260c27c640 1 --2- 172.21.15.169:0/62632341 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26040cde20 0x7f2604166630 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:11.195 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:11.192+0000 7f260c27c640 1 -- 172.21.15.169:0/62632341 >> 172.21.15.169:0/62632341 conn(0x7f2604069930 msgr2=0x7f2604069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:11.195 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:11.192+0000 7f260c27c640 1 -- 172.21.15.169:0/62632341 shutdown_connections 2023-10-26T01:32:11.195 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:11.192+0000 7f260c27c640 1 -- 172.21.15.169:0/62632341 wait complete. 2023-10-26T01:32:11.196 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:11.194+0000 7f260c27c640 1 Processor -- start 2023-10-26T01:32:11.196 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:11.194+0000 7f260c27c640 1 -- start start 2023-10-26T01:32:11.198 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:11.195+0000 7f260c27c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f26040c9d00 0x7f26041491a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:11.198 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:11.195+0000 7f260c27c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26040cde20 0x7f26041f5640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:11.198 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:11.195+0000 7f260c27c640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f260415b620 0x7f26041f5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:11.198 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:11.195+0000 7f260c27c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2604159620 con 0x7f26040cde20 2023-10-26T01:32:11.199 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:11.195+0000 7f260c27c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2604147810 con 0x7f26040c9d00 2023-10-26T01:32:11.199 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:11.195+0000 7f260c27c640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26040c67e0 con 0x7f260415b620 2023-10-26T01:32:11.199 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:11.195+0000 7f2609817640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26040cde20 0x7f26041f5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:11.199 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:11.195+0000 7f2609817640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26040cde20 0x7f26041f5640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:44552/0 (socket says 172.21.15.169:44552) 2023-10-26T01:32:11.199 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:11.195+0000 7f2609817640 1 -- 172.21.15.169:0/1789669534 learned_addr learned my addr 172.21.15.169:0/1789669534 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:32:11.200 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:11.195+0000 7f260a018640 1 --2- 172.21.15.169:0/1789669534 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f26040c9d00 0x7f26041491a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:11.200 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:11.195+0000 7f2609817640 1 -- 172.21.15.169:0/1789669534 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f260415b620 msgr2=0x7f26041f5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:11.200 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:11.195+0000 7f2609817640 1 --2- 172.21.15.169:0/1789669534 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f260415b620 0x7f26041f5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:11.200 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:11.195+0000 7f2609817640 1 -- 172.21.15.169:0/1789669534 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f26040c9d00 msgr2=0x7f26041491a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:11.200 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:11.195+0000 7f2609817640 1 --2- 172.21.15.169:0/1789669534 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f26040c9d00 0x7f26041491a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:11.200 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:11.195+0000 7f2609817640 1 -- 172.21.15.169:0/1789669534 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25fc009840 con 0x7f26040cde20 2023-10-26T01:32:11.201 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:11.196+0000 7f2609817640 1 --2- 172.21.15.169:0/1789669534 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26040cde20 0x7f26041f5640 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f25f400c430 tx=0x7f25f400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:11.201 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:11.196+0000 7f25faffd640 1 -- 172.21.15.169:0/1789669534 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f4011790 con 0x7f26040cde20 2023-10-26T01:32:11.201 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:11.196+0000 7f25faffd640 1 -- 172.21.15.169:0/1789669534 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f25f40119e0 con 0x7f26040cde20 2023-10-26T01:32:11.201 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:11.196+0000 7f25faffd640 1 -- 172.21.15.169:0/1789669534 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f4011e30 con 0x7f26040cde20 2023-10-26T01:32:11.201 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:11.196+0000 7f260c27c640 1 -- 172.21.15.169:0/1789669534 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26040c6ac0 con 0x7f26040cde20 2023-10-26T01:32:11.202 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:11.196+0000 7f260c27c640 1 -- 172.21.15.169:0/1789669534 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26041498c0 con 0x7f26040cde20 2023-10-26T01:32:11.203 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:11.197+0000 7f260c27c640 1 -- 172.21.15.169:0/1789669534 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26041597d0 con 0x7f26040cde20 2023-10-26T01:32:11.203 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:11.198+0000 7f25faffd640 1 -- 172.21.15.169:0/1789669534 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f25f401c460 con 0x7f26040cde20 2023-10-26T01:32:11.203 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:11.199+0000 7f25faffd640 1 --2- 172.21.15.169:0/1789669534 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f25e806ae40 0x7f25e806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:11.203 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:11.199+0000 7f25faffd640 1 -- 172.21.15.169:0/1789669534 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f25f4089710 con 0x7f26040cde20 2023-10-26T01:32:11.203 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:11.199+0000 7f25faffd640 1 -- 172.21.15.169:0/1789669534 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=37+}) v3 -- 0x7f25e8072970 con 0x7f26040cde20 2023-10-26T01:32:11.204 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:11.200+0000 7f260a018640 1 --2- 172.21.15.169:0/1789669534 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f25e806ae40 0x7f25e806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:11.204 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:11.201+0000 7f260a018640 1 --2- 172.21.15.169:0/1789669534 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f25e806ae40 0x7f25e806d300 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2600007fd0 tx=0x7f2600009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:11.205 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:11.203+0000 7f25faffd640 1 -- 172.21.15.169:0/1789669534 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f26041597d0 con 0x7f26040cde20 2023-10-26T01:32:11.405 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:11.403+0000 7f260c27c640 1 -- 172.21.15.169:0/1789669534 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd unset", "key": "pause"} v 0) v1 -- 0x7f26041498c0 con 0x7f26040cde20 2023-10-26T01:32:11.948 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:11.946+0000 7f25faffd640 1 -- 172.21.15.169:0/1789669534 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "pause"}]=0 pauserd,pausewr is unset v37) v1 ==== 97+0+0 (secure 0 0 0) 0x7f26041498c0 con 0x7f26040cde20 2023-10-26T01:32:11.949 INFO:teuthology.orchestra.run.smithi169.stderr:pauserd,pausewr is unset 2023-10-26T01:32:11.953 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:11.951+0000 7f25faffd640 1 -- 172.21.15.169:0/1789669534 <== mon.0 v2:172.21.15.169:3300/0 8 ==== osd_map(37..37 src has 1..37) v4 ==== 288+0+0 (secure 0 0 0) 0x7f25f4058b70 con 0x7f26040cde20 2023-10-26T01:32:11.953 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:11.951+0000 7f260c27c640 1 -- 172.21.15.169:0/1789669534 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f25e806ae40 msgr2=0x7f25e806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:11.953 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:11.951+0000 7f260c27c640 1 --2- 172.21.15.169:0/1789669534 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f25e806ae40 0x7f25e806d300 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2600007fd0 tx=0x7f2600009040 comp rx=0 tx=0).stop 2023-10-26T01:32:11.953 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:11.951+0000 7f260c27c640 1 -- 172.21.15.169:0/1789669534 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26040cde20 msgr2=0x7f26041f5640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:11.953 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:11.951+0000 7f260c27c640 1 --2- 172.21.15.169:0/1789669534 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26040cde20 0x7f26041f5640 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f25f400c430 tx=0x7f25f400c8f0 comp rx=0 tx=0).stop 2023-10-26T01:32:11.954 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:11.951+0000 7f260c27c640 1 -- 172.21.15.169:0/1789669534 shutdown_connections 2023-10-26T01:32:11.955 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:11.951+0000 7f260c27c640 1 --2- 172.21.15.169:0/1789669534 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f25e806ae40 0x7f25e806d300 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:11.955 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:11.951+0000 7f260c27c640 1 --2- 172.21.15.169:0/1789669534 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26040cde20 0x7f26041f5640 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:11.955 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:11.951+0000 7f260c27c640 1 --2- 172.21.15.169:0/1789669534 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f260415b620 0x7f26041f5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:11.955 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:11.951+0000 7f260c27c640 1 --2- 172.21.15.169:0/1789669534 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f26040c9d00 0x7f26041491a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:11.955 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:11.951+0000 7f260c27c640 1 -- 172.21.15.169:0/1789669534 >> 172.21.15.169:0/1789669534 conn(0x7f2604069930 msgr2=0x7f2604157ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:11.956 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:11.954+0000 7f260c27c640 1 -- 172.21.15.169:0/1789669534 shutdown_connections 2023-10-26T01:32:11.956 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:11.954+0000 7f260c27c640 1 -- 172.21.15.169:0/1789669534 wait complete. 2023-10-26T01:32:11.978 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:32:12.189 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:12.187+0000 7fb77ffff640 1 -- 172.21.15.169:0/1402028107 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb7800ac6d0 msgr2=0x7fb78015d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:12.190 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:12.187+0000 7fb77ffff640 1 --2- 172.21.15.169:0/1402028107 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb7800ac6d0 0x7fb78015d840 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fb770005fd0 tx=0x7fb77002c0e0 comp rx=0 tx=0).stop 2023-10-26T01:32:12.190 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:12.187+0000 7fb77ffff640 1 -- 172.21.15.169:0/1402028107 shutdown_connections 2023-10-26T01:32:12.190 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:12.187+0000 7fb77ffff640 1 --2- 172.21.15.169:0/1402028107 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb78009a5c0 0x7fb78009aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:12.190 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:12.187+0000 7fb77ffff640 1 --2- 172.21.15.169:0/1402028107 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb7800ac6d0 0x7fb78015d840 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:12.190 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:12.187+0000 7fb77ffff640 1 -- 172.21.15.169:0/1402028107 >> 172.21.15.169:0/1402028107 conn(0x7fb7800aedd0 msgr2=0x7fb7800af1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:12.191 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:12.187+0000 7fb77ffff640 1 -- 172.21.15.169:0/1402028107 shutdown_connections 2023-10-26T01:32:12.191 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:12.187+0000 7fb77ffff640 1 -- 172.21.15.169:0/1402028107 wait complete. 2023-10-26T01:32:12.191 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:12.189+0000 7fb77ffff640 1 Processor -- start 2023-10-26T01:32:12.192 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:12.189+0000 7fb77ffff640 1 -- start start 2023-10-26T01:32:12.194 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:12.190+0000 7fb77ffff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb78009a5c0 0x7fb78006e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:12.194 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:12.190+0000 7fb77ffff640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb7800ac6d0 0x7fb78015c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:12.194 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:12.190+0000 7fb77ffff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb7800c9970 0x7fb780140540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:12.195 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:12.190+0000 7fb77ffff640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb78006ee30 con 0x7fb7800c9970 2023-10-26T01:32:12.195 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:12.190+0000 7fb77ffff640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb78015b180 con 0x7fb78009a5c0 2023-10-26T01:32:12.195 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:12.190+0000 7fb77ffff640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb78015b460 con 0x7fb7800ac6d0 2023-10-26T01:32:12.195 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:12.191+0000 7fb77f7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb7800c9970 0x7fb780140540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:12.195 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:12.191+0000 7fb77f7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb7800c9970 0x7fb780140540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:44562/0 (socket says 172.21.15.169:44562) 2023-10-26T01:32:12.196 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:12.191+0000 7fb77f7fe640 1 -- 172.21.15.169:0/590705584 learned_addr learned my addr 172.21.15.169:0/590705584 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:32:12.196 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:12.191+0000 7fb77effd640 1 --2- 172.21.15.169:0/590705584 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb78009a5c0 0x7fb78006e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:12.196 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:12.191+0000 7fb7767fc640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb7800ac6d0 0x7fb78015c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:12.196 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:12.192+0000 7fb77f7fe640 1 -- 172.21.15.169:0/590705584 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb7800ac6d0 msgr2=0x7fb78015c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:12.196 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:12.192+0000 7fb77f7fe640 1 --2- 172.21.15.169:0/590705584 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb7800ac6d0 0x7fb78015c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:12.197 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:12.192+0000 7fb77f7fe640 1 -- 172.21.15.169:0/590705584 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb78009a5c0 msgr2=0x7fb78006e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:12.197 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:12.192+0000 7fb77f7fe640 1 --2- 172.21.15.169:0/590705584 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb78009a5c0 0x7fb78006e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:12.197 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:12.192+0000 7fb77f7fe640 1 -- 172.21.15.169:0/590705584 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb770009840 con 0x7fb7800c9970 2023-10-26T01:32:12.197 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:12.192+0000 7fb77f7fe640 1 --2- 172.21.15.169:0/590705584 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb7800c9970 0x7fb780140540 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fb77800d750 tx=0x7fb77800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:12.197 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:12.192+0000 7fb77cff9640 1 -- 172.21.15.169:0/590705584 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb778009bd0 con 0x7fb7800c9970 2023-10-26T01:32:12.198 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:12.192+0000 7fb77cff9640 1 -- 172.21.15.169:0/590705584 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb778009e20 con 0x7fb7800c9970 2023-10-26T01:32:12.198 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:12.192+0000 7fb77cff9640 1 -- 172.21.15.169:0/590705584 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb778005280 con 0x7fb7800c9970 2023-10-26T01:32:12.198 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:12.192+0000 7fb77ffff640 1 -- 172.21.15.169:0/590705584 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb78015b740 con 0x7fb7800c9970 2023-10-26T01:32:12.198 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:12.192+0000 7fb77ffff640 1 -- 172.21.15.169:0/590705584 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb78015bd00 con 0x7fb7800c9970 2023-10-26T01:32:12.199 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:12.193+0000 7fb77ffff640 1 -- 172.21.15.169:0/590705584 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb780073c10 con 0x7fb7800c9970 2023-10-26T01:32:12.200 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:12.194+0000 7fb77cff9640 1 -- 172.21.15.169:0/590705584 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fb7780053e0 con 0x7fb7800c9970 2023-10-26T01:32:12.200 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:12.195+0000 7fb77cff9640 1 --2- 172.21.15.169:0/590705584 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb76006ae70 0x7fb76006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:12.200 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:12.196+0000 7fb77cff9640 1 -- 172.21.15.169:0/590705584 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb778089720 con 0x7fb7800c9970 2023-10-26T01:32:12.200 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:12.196+0000 7fb77effd640 1 --2- 172.21.15.169:0/590705584 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb76006ae70 0x7fb76006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:12.200 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:12.197+0000 7fb77effd640 1 --2- 172.21.15.169:0/590705584 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb76006ae70 0x7fb76006d330 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb770035000 tx=0x7fb770004ab0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:12.201 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:12.200+0000 7fb77cff9640 1 -- 172.21.15.169:0/590705584 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb778054870 con 0x7fb7800c9970 2023-10-26T01:32:12.390 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:12.388+0000 7fb77ffff640 1 -- 172.21.15.169:0/590705584 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb7800675b0 con 0x7fb7800c9970 2023-10-26T01:32:12.391 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:12.390+0000 7fb77cff9640 1 -- 172.21.15.169:0/590705584 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+16617 (secure 0 0 0) 0x7fb778017090 con 0x7fb7800c9970 2023-10-26T01:32:12.392 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:32:12.392 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":37,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:32:11.943092+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-10-26T01:31:55.401509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-10-26T01:31:56.776031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:32:12.395 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:12.393+0000 7fb77ffff640 1 -- 172.21.15.169:0/590705584 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb76006ae70 msgr2=0x7fb76006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:12.395 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:12.393+0000 7fb77ffff640 1 --2- 172.21.15.169:0/590705584 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb76006ae70 0x7fb76006d330 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb770035000 tx=0x7fb770004ab0 comp rx=0 tx=0).stop 2023-10-26T01:32:12.395 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:12.393+0000 7fb77ffff640 1 -- 172.21.15.169:0/590705584 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb7800c9970 msgr2=0x7fb780140540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:12.396 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:12.393+0000 7fb77ffff640 1 --2- 172.21.15.169:0/590705584 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb7800c9970 0x7fb780140540 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fb77800d750 tx=0x7fb77800dc10 comp rx=0 tx=0).stop 2023-10-26T01:32:12.396 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:12.393+0000 7fb77ffff640 1 -- 172.21.15.169:0/590705584 shutdown_connections 2023-10-26T01:32:12.397 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:12.393+0000 7fb77ffff640 1 --2- 172.21.15.169:0/590705584 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb76006ae70 0x7fb76006d330 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:12.397 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:12.393+0000 7fb77ffff640 1 --2- 172.21.15.169:0/590705584 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb7800c9970 0x7fb780140540 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:12.397 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:12.393+0000 7fb77ffff640 1 --2- 172.21.15.169:0/590705584 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb7800ac6d0 0x7fb78015c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:12.398 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:12.393+0000 7fb77ffff640 1 --2- 172.21.15.169:0/590705584 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb78009a5c0 0x7fb78006e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:12.398 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:12.393+0000 7fb77ffff640 1 -- 172.21.15.169:0/590705584 >> 172.21.15.169:0/590705584 conn(0x7fb7800aedd0 msgr2=0x7fb780141a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:12.399 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:12.393+0000 7fb77ffff640 1 -- 172.21.15.169:0/590705584 shutdown_connections 2023-10-26T01:32:12.399 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:12.393+0000 7fb77ffff640 1 -- 172.21.15.169:0/590705584 wait complete. 2023-10-26T01:32:12.414 DEBUG:teuthology.orchestra.run.smithi169:> stdin-killer -- python3 -c ' 2023-10-26T01:32:12.415 DEBUG:teuthology.orchestra.run.smithi169:> import os 2023-10-26T01:32:12.415 DEBUG:teuthology.orchestra.run.smithi169:> fd = os.open("/home/ubuntu/cephtest/mnt.0/foo", os.O_CREAT | os.O_RDWR, 0O600) 2023-10-26T01:32:12.416 DEBUG:teuthology.orchestra.run.smithi169:> os.fchmod(fd, 0O666) 2023-10-26T01:32:12.416 DEBUG:teuthology.orchestra.run.smithi169:> os.fsync(fd) 2023-10-26T01:32:12.417 DEBUG:teuthology.orchestra.run.smithi169:> os.close(fd) 2023-10-26T01:32:12.417 DEBUG:teuthology.orchestra.run.smithi169:> ' 2023-10-26T01:32:12.553 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T01:32:12.562 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:32:12.563 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-26T01:32:12.666 INFO:teuthology.orchestra.run.smithi169.stdout:client4712 2023-10-26T01:32:12.667 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:32:12.667 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/30203b28-ed8a-44e8-a9a6-a9bf25f980f1.client4712/osdmap') 2023-10-26T01:32:12.759 INFO:teuthology.orchestra.run.smithi169.stdout:epoch 35 barrier 35 flags 0x588000 2023-10-26T01:32:12.759 INFO:teuthology.orchestra.run.smithi169.stdout:pool 1 '.mgr' type 1 size 2 min_size 1 pg_num 1 pg_num_mask 0 flags 0x1 lfor 0 read_tier -1 write_tier -1 2023-10-26T01:32:12.760 INFO:teuthology.orchestra.run.smithi169.stdout:pool 4 'cephfs_metadata' type 1 size 2 min_size 1 pg_num 32 pg_num_mask 31 flags 0x1 lfor 0 read_tier -1 write_tier -1 2023-10-26T01:32:12.760 INFO:teuthology.orchestra.run.smithi169.stdout:pool 5 'cephfs_data' type 1 size 2 min_size 1 pg_num 64 pg_num_mask 63 flags 0x1 lfor 0 read_tier -1 write_tier -1 2023-10-26T01:32:12.760 INFO:teuthology.orchestra.run.smithi169.stdout:osd0 (1)172.21.15.169:6817 100% (exists, up) 100% -1 2023-10-26T01:32:12.760 INFO:teuthology.orchestra.run.smithi169.stdout:osd1 (1)172.21.15.169:6809 100% (exists, up) 100% -1 2023-10-26T01:32:12.760 INFO:teuthology.orchestra.run.smithi169.stdout:osd2 (1)172.21.15.169:6801 100% (exists, up) 100% -1 2023-10-26T01:32:12.760 INFO:teuthology.orchestra.run.smithi169.stdout:osd3 (1)172.21.15.169:6825 100% (exists, up) 100% -1 2023-10-26T01:32:12.761 INFO:teuthology.orchestra.run.smithi169.stdout:osd4 (1)172.21.15.170:6817 100% (exists, up) 100% -1 2023-10-26T01:32:12.761 INFO:teuthology.orchestra.run.smithi169.stdout:osd5 (1)172.21.15.170:6809 100% (exists, up) 100% -1 2023-10-26T01:32:12.761 INFO:teuthology.orchestra.run.smithi169.stdout:osd6 (1)172.21.15.170:6801 100% (exists, up) 100% -1 2023-10-26T01:32:12.761 INFO:teuthology.orchestra.run.smithi169.stdout:osd7 (1)172.21.15.170:6822 100% (exists, up) 100% -1 2023-10-26T01:32:12.762 INFO:teuthology.orchestra.run.smithi169.stderr:1+1 records in 2023-10-26T01:32:12.762 INFO:teuthology.orchestra.run.smithi169.stderr:1+1 records out 2023-10-26T01:32:12.762 INFO:teuthology.orchestra.run.smithi169.stderr:799 bytes copied, 8.3277e-05 s, 9.6 MB/s 2023-10-26T01:32:12.763 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T01:32:12.986 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:12.984+0000 7f6845367640 1 -- 172.21.15.169:0/1234390285 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68400ac6d0 msgr2=0x7f684015d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:12.986 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:12.984+0000 7f6845367640 1 --2- 172.21.15.169:0/1234390285 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68400ac6d0 0x7f684015d840 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f68400cbd50 tx=0x7f683002be40 comp rx=0 tx=0).stop 2023-10-26T01:32:12.987 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:12.984+0000 7f6845367640 1 -- 172.21.15.169:0/1234390285 shutdown_connections 2023-10-26T01:32:12.987 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:12.984+0000 7f6845367640 1 --2- 172.21.15.169:0/1234390285 >> v2:172.21.15.170:3300/0 conn(0x7f68400c9970 0x7f6840098a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:12.987 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:12.984+0000 7f6845367640 1 --2- 172.21.15.169:0/1234390285 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68400ac6d0 0x7f684015d840 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:12.987 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:12.984+0000 7f6845367640 1 -- 172.21.15.169:0/1234390285 >> 172.21.15.169:0/1234390285 conn(0x7f68400aedd0 msgr2=0x7f68400af1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:12.987 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:12.984+0000 7f6845367640 1 -- 172.21.15.169:0/1234390285 shutdown_connections 2023-10-26T01:32:12.988 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:12.984+0000 7f6845367640 1 -- 172.21.15.169:0/1234390285 wait complete. 2023-10-26T01:32:12.989 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:12.986+0000 7f6845367640 1 Processor -- start 2023-10-26T01:32:12.989 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:12.986+0000 7f6845367640 1 -- start start 2023-10-26T01:32:12.989 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:12.987+0000 7f6845367640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f684009a5c0 0x7f6840158000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:12.989 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:12.987+0000 7f6845367640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f68400ac6d0 0x7f68400adbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:12.990 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:12.987+0000 7f6845367640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68400c9970 0x7f68400ae0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:12.990 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:12.987+0000 7f6845367640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f684015c7b0 con 0x7f68400c9970 2023-10-26T01:32:12.990 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:12.987+0000 7f6845367640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f684015b710 con 0x7f684009a5c0 2023-10-26T01:32:12.990 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:12.987+0000 7f6845367640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f684015b9f0 con 0x7f68400ac6d0 2023-10-26T01:32:12.990 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:12.987+0000 7f6844b66640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68400c9970 0x7f68400ae0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:12.991 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:12.987+0000 7f6844b66640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68400c9970 0x7f68400ae0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:44582/0 (socket says 172.21.15.169:44582) 2023-10-26T01:32:12.991 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:12.987+0000 7f6844b66640 1 -- 172.21.15.169:0/3369678666 learned_addr learned my addr 172.21.15.169:0/3369678666 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:32:12.992 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:12.988+0000 7f683ffff640 1 --2- 172.21.15.169:0/3369678666 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f684009a5c0 0x7f6840158000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:12.992 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:12.988+0000 7f6837fff640 1 --2- 172.21.15.169:0/3369678666 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f68400ac6d0 0x7f68400adbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:12.992 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:12.988+0000 7f6844b66640 1 -- 172.21.15.169:0/3369678666 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f68400ac6d0 msgr2=0x7f68400adbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:12.993 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:12.988+0000 7f6844b66640 1 --2- 172.21.15.169:0/3369678666 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f68400ac6d0 0x7f68400adbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:12.993 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:12.988+0000 7f6844b66640 1 -- 172.21.15.169:0/3369678666 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f684009a5c0 msgr2=0x7f6840158000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:12.993 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:12.988+0000 7f6844b66640 1 --2- 172.21.15.169:0/3369678666 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f684009a5c0 0x7f6840158000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:12.993 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:12.988+0000 7f6844b66640 1 -- 172.21.15.169:0/3369678666 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6830009560 con 0x7f68400c9970 2023-10-26T01:32:12.994 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:12.988+0000 7f6844b66640 1 --2- 172.21.15.169:0/3369678666 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68400c9970 0x7f68400ae0f0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f683800c7a0 tx=0x7f683800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:12.994 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:12.988+0000 7f683d7fa640 1 -- 172.21.15.169:0/3369678666 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6838003b70 con 0x7f68400c9970 2023-10-26T01:32:12.994 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:12.988+0000 7f683d7fa640 1 -- 172.21.15.169:0/3369678666 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6838003cf0 con 0x7f68400c9970 2023-10-26T01:32:12.994 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:12.989+0000 7f6845367640 1 -- 172.21.15.169:0/3369678666 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f684015bcd0 con 0x7f68400c9970 2023-10-26T01:32:12.994 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:12.989+0000 7f683d7fa640 1 -- 172.21.15.169:0/3369678666 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f683800aab0 con 0x7f68400c9970 2023-10-26T01:32:12.995 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:12.989+0000 7f6845367640 1 -- 172.21.15.169:0/3369678666 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f684015c290 con 0x7f68400c9970 2023-10-26T01:32:12.995 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:12.990+0000 7f6845367640 1 -- 172.21.15.169:0/3369678666 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6840073c10 con 0x7f68400c9970 2023-10-26T01:32:12.995 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:12.991+0000 7f683d7fa640 1 -- 172.21.15.169:0/3369678666 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f683800ac10 con 0x7f68400c9970 2023-10-26T01:32:12.996 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:12.992+0000 7f683d7fa640 1 --2- 172.21.15.169:0/3369678666 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f682006ae70 0x7f682006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:12.996 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:12.992+0000 7f683d7fa640 1 -- 172.21.15.169:0/3369678666 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6838089440 con 0x7f68400c9970 2023-10-26T01:32:12.996 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:12.993+0000 7f683ffff640 1 --2- 172.21.15.169:0/3369678666 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f682006ae70 0x7f682006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:12.996 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:12.994+0000 7f683ffff640 1 --2- 172.21.15.169:0/3369678666 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f682006ae70 0x7f682006d330 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6830035000 tx=0x7f6830004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:12.998 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:12.997+0000 7f683d7fa640 1 -- 172.21.15.169:0/3369678666 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f68380544e0 con 0x7f68400c9970 2023-10-26T01:32:13.230 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:13.229+0000 7f6845367640 1 -- 172.21.15.169:0/3369678666 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f68400ca410 con 0x7f68400c9970 2023-10-26T01:32:13.231 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:13.230+0000 7f683d7fa640 1 -- 172.21.15.169:0/3369678666 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4913 (secure 0 0 0) 0x7f68400ca410 con 0x7f68400c9970 2023-10-26T01:32:13.232 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:32:13.232 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4640,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/61096113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":61096113},{"type":"v1","addr":"172.21.15.170:6836","nonce":61096113}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4762,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2825046244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2825046244},{"type":"v1","addr":"172.21.15.169:6833","nonce":2825046244}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4777,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/258866976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":258866976},{"type":"v1","addr":"172.21.15.170:6837","nonce":258866976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T01:31:57.759251+0000","modified":"2023-10-26T01:32:00.831213+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5196},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5196":{"gid":5196,"name":"c","rank":0,"incarnation":11,"state":"up:active","state_seq":4,"addr":"172.21.15.169:6835/871978528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":871978528},{"type":"v1","addr":"172.21.15.169:6835","nonce":871978528}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-26T01:32:13.233 INFO:teuthology.orchestra.run.smithi169.stderr:dumped fsmap epoch 15 2023-10-26T01:32:13.235 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:13.233+0000 7f6845367640 1 -- 172.21.15.169:0/3369678666 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f682006ae70 msgr2=0x7f682006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:13.235 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:13.233+0000 7f6845367640 1 --2- 172.21.15.169:0/3369678666 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f682006ae70 0x7f682006d330 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6830035000 tx=0x7f6830004730 comp rx=0 tx=0).stop 2023-10-26T01:32:13.235 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:13.233+0000 7f6845367640 1 -- 172.21.15.169:0/3369678666 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68400c9970 msgr2=0x7f68400ae0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:13.236 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:13.233+0000 7f6845367640 1 --2- 172.21.15.169:0/3369678666 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68400c9970 0x7f68400ae0f0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f683800c7a0 tx=0x7f683800cc60 comp rx=0 tx=0).stop 2023-10-26T01:32:13.236 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:13.233+0000 7f6845367640 1 -- 172.21.15.169:0/3369678666 shutdown_connections 2023-10-26T01:32:13.236 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:13.233+0000 7f6845367640 1 --2- 172.21.15.169:0/3369678666 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f682006ae70 0x7f682006d330 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:13.236 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:13.233+0000 7f6845367640 1 --2- 172.21.15.169:0/3369678666 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68400c9970 0x7f68400ae0f0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:13.236 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:13.233+0000 7f6845367640 1 --2- 172.21.15.169:0/3369678666 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f68400ac6d0 0x7f68400adbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:13.236 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:13.233+0000 7f6845367640 1 --2- 172.21.15.169:0/3369678666 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f684009a5c0 0x7f6840158000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:13.237 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:13.233+0000 7f6845367640 1 -- 172.21.15.169:0/3369678666 >> 172.21.15.169:0/3369678666 conn(0x7f68400aedd0 msgr2=0x7f68400aa1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:13.237 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:13.233+0000 7f6845367640 1 -- 172.21.15.169:0/3369678666 shutdown_connections 2023-10-26T01:32:13.237 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:13.233+0000 7f6845367640 1 -- 172.21.15.169:0/3369678666 wait complete. 2023-10-26T01:32:13.255 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json osdmap barrier 37 2023-10-26T01:32:13.479 DEBUG:tasks.cephfs.filesystem:_json_asok output empty 2023-10-26T01:32:13.479 DEBUG:teuthology.orchestra.run.smithi169:> stdin-killer -- python3 -c ' 2023-10-26T01:32:13.480 DEBUG:teuthology.orchestra.run.smithi169:> import os 2023-10-26T01:32:13.480 DEBUG:teuthology.orchestra.run.smithi169:> fd = os.open("/home/ubuntu/cephtest/mnt.0/baz", os.O_CREAT | os.O_RDWR, 0O600) 2023-10-26T01:32:13.480 DEBUG:teuthology.orchestra.run.smithi169:> os.fchmod(fd, 0O666) 2023-10-26T01:32:13.481 DEBUG:teuthology.orchestra.run.smithi169:> os.fsync(fd) 2023-10-26T01:32:13.481 DEBUG:teuthology.orchestra.run.smithi169:> os.close(fd) 2023-10-26T01:32:13.481 DEBUG:teuthology.orchestra.run.smithi169:> ' 2023-10-26T01:32:13.635 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T01:32:13.645 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:32:13.646 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-26T01:32:13.748 INFO:teuthology.orchestra.run.smithi169.stdout:client4712 2023-10-26T01:32:13.749 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:32:13.749 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/30203b28-ed8a-44e8-a9a6-a9bf25f980f1.client4712/osdmap') 2023-10-26T01:32:13.839 INFO:teuthology.orchestra.run.smithi169.stdout:epoch 37 barrier 37 flags 0x588000 2023-10-26T01:32:13.840 INFO:teuthology.orchestra.run.smithi169.stdout:pool 1 '.mgr' type 1 size 2 min_size 1 pg_num 1 pg_num_mask 0 flags 0x1 lfor 0 read_tier -1 write_tier -1 2023-10-26T01:32:13.840 INFO:teuthology.orchestra.run.smithi169.stdout:pool 4 'cephfs_metadata' type 1 size 2 min_size 1 pg_num 32 pg_num_mask 31 flags 0x1 lfor 0 read_tier -1 write_tier -1 2023-10-26T01:32:13.840 INFO:teuthology.orchestra.run.smithi169.stdout:pool 5 'cephfs_data' type 1 size 2 min_size 1 pg_num 64 pg_num_mask 63 flags 0x1 lfor 0 read_tier -1 write_tier -1 2023-10-26T01:32:13.840 INFO:teuthology.orchestra.run.smithi169.stdout:osd0 (1)172.21.15.169:6817 100% (exists, up) 100% -1 2023-10-26T01:32:13.840 INFO:teuthology.orchestra.run.smithi169.stdout:osd1 (1)172.21.15.169:6809 100% (exists, up) 100% -1 2023-10-26T01:32:13.840 INFO:teuthology.orchestra.run.smithi169.stdout:osd2 (1)172.21.15.169:6801 100% (exists, up) 100% -1 2023-10-26T01:32:13.841 INFO:teuthology.orchestra.run.smithi169.stdout:osd3 (1)172.21.15.169:6825 100% (exists, up) 100% -1 2023-10-26T01:32:13.841 INFO:teuthology.orchestra.run.smithi169.stdout:osd4 (1)172.21.15.170:6817 100% (exists, up) 100% -1 2023-10-26T01:32:13.841 INFO:teuthology.orchestra.run.smithi169.stdout:osd5 (1)172.21.15.170:6809 100% (exists, up) 100% -1 2023-10-26T01:32:13.841 INFO:teuthology.orchestra.run.smithi169.stdout:osd6 (1)172.21.15.170:6801 100% (exists, up) 100% -1 2023-10-26T01:32:13.841 INFO:teuthology.orchestra.run.smithi169.stdout:osd7 (1)172.21.15.170:6822 100% (exists, up) 100% -1 2023-10-26T01:32:13.842 INFO:teuthology.orchestra.run.smithi169.stderr:1+1 records in 2023-10-26T01:32:13.842 INFO:teuthology.orchestra.run.smithi169.stderr:1+1 records out 2023-10-26T01:32:13.842 INFO:teuthology.orchestra.run.smithi169.stderr:799 bytes copied, 8.3631e-05 s, 9.6 MB/s 2023-10-26T01:32:13.843 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:32:13.844 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-26T01:32:13.899 INFO:teuthology.orchestra.run.smithi169.stdout:client4712 2023-10-26T01:32:13.899 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:32:13.900 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/30203b28-ed8a-44e8-a9a6-a9bf25f980f1.client4712/osdmap') 2023-10-26T01:32:13.987 INFO:teuthology.orchestra.run.smithi169.stdout:epoch 37 barrier 37 flags 0x588000 2023-10-26T01:32:13.987 INFO:teuthology.orchestra.run.smithi169.stdout:pool 1 '.mgr' type 1 size 2 min_size 1 pg_num 1 pg_num_mask 0 flags 0x1 lfor 0 read_tier -1 write_tier -1 2023-10-26T01:32:13.987 INFO:teuthology.orchestra.run.smithi169.stdout:pool 4 'cephfs_metadata' type 1 size 2 min_size 1 pg_num 32 pg_num_mask 31 flags 0x1 lfor 0 read_tier -1 write_tier -1 2023-10-26T01:32:13.987 INFO:teuthology.orchestra.run.smithi169.stdout:pool 5 'cephfs_data' type 1 size 2 min_size 1 pg_num 64 pg_num_mask 63 flags 0x1 lfor 0 read_tier -1 write_tier -1 2023-10-26T01:32:13.988 INFO:teuthology.orchestra.run.smithi169.stdout:osd0 (1)172.21.15.169:6817 100% (exists, up) 100% -1 2023-10-26T01:32:13.988 INFO:teuthology.orchestra.run.smithi169.stdout:osd1 (1)172.21.15.169:6809 100% (exists, up) 100% -1 2023-10-26T01:32:13.988 INFO:teuthology.orchestra.run.smithi169.stdout:osd2 (1)172.21.15.169:6801 100% (exists, up) 100% -1 2023-10-26T01:32:13.988 INFO:teuthology.orchestra.run.smithi169.stdout:osd3 (1)172.21.15.169:6825 100% (exists, up) 100% -1 2023-10-26T01:32:13.988 INFO:teuthology.orchestra.run.smithi169.stdout:osd4 (1)172.21.15.170:6817 100% (exists, up) 100% -1 2023-10-26T01:32:13.989 INFO:teuthology.orchestra.run.smithi169.stdout:osd5 (1)172.21.15.170:6809 100% (exists, up) 100% -1 2023-10-26T01:32:13.989 INFO:teuthology.orchestra.run.smithi169.stdout:osd6 (1)172.21.15.170:6801 100% (exists, up) 100% -1 2023-10-26T01:32:13.989 INFO:teuthology.orchestra.run.smithi169.stdout:osd7 (1)172.21.15.170:6822 100% (exists, up) 100% -1 2023-10-26T01:32:13.989 INFO:teuthology.orchestra.run.smithi169.stderr:1+1 records in 2023-10-26T01:32:13.990 INFO:teuthology.orchestra.run.smithi169.stderr:1+1 records out 2023-10-26T01:32:13.990 INFO:teuthology.orchestra.run.smithi169.stderr:799 bytes copied, 8.5652e-05 s, 9.3 MB/s 2023-10-26T01:32:13.991 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-26T01:32:13.991 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-26T01:32:13.992 DEBUG:teuthology.orchestra.run.smithi169:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-26T01:32:14.106 DEBUG:teuthology.orchestra.run.smithi170:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-26T01:32:14.236 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-26T01:32:14.236 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:32:14.237 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:32:14.296 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:32:14.297 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:32:14.349 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:32:14.569 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:14.567+0000 7fd5852d7640 1 -- 172.21.15.169:0/1353286097 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd5801592d0 msgr2=0x7fd5800aa3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:14.570 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:14.567+0000 7fd5852d7640 1 --2- 172.21.15.169:0/1353286097 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd5801592d0 0x7fd5800aa3d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd570005fd0 tx=0x7fd57002bd90 comp rx=0 tx=0).stop 2023-10-26T01:32:14.570 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:14.567+0000 7fd5852d7640 1 -- 172.21.15.169:0/1353286097 shutdown_connections 2023-10-26T01:32:14.570 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:14.567+0000 7fd5852d7640 1 --2- 172.21.15.169:0/1353286097 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd5800cde00 0x7fd5800cbac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:14.571 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:14.567+0000 7fd5852d7640 1 --2- 172.21.15.169:0/1353286097 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd5801592d0 0x7fd5800aa3d0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:14.571 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:14.567+0000 7fd5852d7640 1 -- 172.21.15.169:0/1353286097 >> 172.21.15.169:0/1353286097 conn(0x7fd580069930 msgr2=0x7fd580069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:14.571 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:14.567+0000 7fd5852d7640 1 -- 172.21.15.169:0/1353286097 shutdown_connections 2023-10-26T01:32:14.571 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:14.567+0000 7fd5852d7640 1 -- 172.21.15.169:0/1353286097 wait complete. 2023-10-26T01:32:14.572 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:14.569+0000 7fd5852d7640 1 Processor -- start 2023-10-26T01:32:14.572 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:14.570+0000 7fd5852d7640 1 -- start start 2023-10-26T01:32:14.574 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:14.571+0000 7fd5852d7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd5800ac930 0x7fd580146c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:14.574 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:14.571+0000 7fd5852d7640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd5800cde00 0x7fd5800c8920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:14.574 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:14.571+0000 7fd5852d7640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5801592d0 0x7fd580149420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:14.574 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:14.571+0000 7fd5852d7640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5801471a0 con 0x7fd5801592d0 2023-10-26T01:32:14.575 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:14.571+0000 7fd5852d7640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5800c92e0 con 0x7fd5800ac930 2023-10-26T01:32:14.575 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:14.571+0000 7fd5852d7640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5800c95c0 con 0x7fd5800cde00 2023-10-26T01:32:14.575 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:14.571+0000 7fd57f7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5801592d0 0x7fd580149420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:14.575 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:14.571+0000 7fd57f7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5801592d0 0x7fd580149420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:41876/0 (socket says 172.21.15.169:41876) 2023-10-26T01:32:14.575 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:14.571+0000 7fd57f7fe640 1 -- 172.21.15.169:0/3795182307 learned_addr learned my addr 172.21.15.169:0/3795182307 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:32:14.576 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:14.572+0000 7fd577fff640 1 --2- 172.21.15.169:0/3795182307 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd5800cde00 0x7fd5800c8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:14.576 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:14.572+0000 7fd57effd640 1 --2- 172.21.15.169:0/3795182307 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd5800ac930 0x7fd580146c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:14.577 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:14.572+0000 7fd57f7fe640 1 -- 172.21.15.169:0/3795182307 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd5800cde00 msgr2=0x7fd5800c8920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:14.577 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:14.572+0000 7fd57f7fe640 1 --2- 172.21.15.169:0/3795182307 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd5800cde00 0x7fd5800c8920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:14.578 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:14.572+0000 7fd57f7fe640 1 -- 172.21.15.169:0/3795182307 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd5800ac930 msgr2=0x7fd580146c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:14.578 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:14.572+0000 7fd57f7fe640 1 --2- 172.21.15.169:0/3795182307 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd5800ac930 0x7fd580146c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:14.578 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:14.572+0000 7fd57f7fe640 1 -- 172.21.15.169:0/3795182307 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd570009560 con 0x7fd5801592d0 2023-10-26T01:32:14.578 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:14.572+0000 7fd57effd640 1 --2- 172.21.15.169:0/3795182307 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd5800ac930 0x7fd580146c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:32:14.579 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:14.572+0000 7fd577fff640 1 --2- 172.21.15.169:0/3795182307 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd5800cde00 0x7fd5800c8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T01:32:14.579 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:14.572+0000 7fd57f7fe640 1 --2- 172.21.15.169:0/3795182307 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5801592d0 0x7fd580149420 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fd57800eb10 tx=0x7fd57800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:14.580 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:14.572+0000 7fd5777fe640 1 -- 172.21.15.169:0/3795182307 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd578004b20 con 0x7fd5801592d0 2023-10-26T01:32:14.581 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:14.573+0000 7fd5777fe640 1 -- 172.21.15.169:0/3795182307 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd578004ca0 con 0x7fd5801592d0 2023-10-26T01:32:14.581 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:14.573+0000 7fd5777fe640 1 -- 172.21.15.169:0/3795182307 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd578005280 con 0x7fd5801592d0 2023-10-26T01:32:14.582 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:14.573+0000 7fd5852d7640 1 -- 172.21.15.169:0/3795182307 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5800c98a0 con 0x7fd5801592d0 2023-10-26T01:32:14.582 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:14.573+0000 7fd5852d7640 1 -- 172.21.15.169:0/3795182307 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd580147490 con 0x7fd5801592d0 2023-10-26T01:32:14.582 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:14.574+0000 7fd5852d7640 1 -- 172.21.15.169:0/3795182307 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd58006a200 con 0x7fd5801592d0 2023-10-26T01:32:14.582 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:14.575+0000 7fd5777fe640 1 -- 172.21.15.169:0/3795182307 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fd5780053e0 con 0x7fd5801592d0 2023-10-26T01:32:14.583 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:14.576+0000 7fd5777fe640 1 --2- 172.21.15.169:0/3795182307 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd56006ae40 0x7fd56006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:14.583 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:14.576+0000 7fd5777fe640 1 -- 172.21.15.169:0/3795182307 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd578014070 con 0x7fd5801592d0 2023-10-26T01:32:14.583 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:14.576+0000 7fd57effd640 1 --2- 172.21.15.169:0/3795182307 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd56006ae40 0x7fd56006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:14.583 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:14.577+0000 7fd57effd640 1 --2- 172.21.15.169:0/3795182307 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd56006ae40 0x7fd56006d300 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd570035000 tx=0x7fd570004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:14.584 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:14.580+0000 7fd5777fe640 1 -- 172.21.15.169:0/3795182307 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd578055440 con 0x7fd5801592d0 2023-10-26T01:32:14.774 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:14.772+0000 7fd5852d7640 1 -- 172.21.15.169:0/3795182307 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd58004db10 con 0x7fd5801592d0 2023-10-26T01:32:14.776 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:14.774+0000 7fd5777fe640 1 -- 172.21.15.169:0/3795182307 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+16617 (secure 0 0 0) 0x7fd578058a60 con 0x7fd5801592d0 2023-10-26T01:32:14.776 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:32:14.777 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":37,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:32:11.943092+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-10-26T01:31:55.401509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-10-26T01:31:56.776031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:32:14.781 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:14.777+0000 7fd5852d7640 1 -- 172.21.15.169:0/3795182307 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd56006ae40 msgr2=0x7fd56006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:14.782 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:14.777+0000 7fd5852d7640 1 --2- 172.21.15.169:0/3795182307 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd56006ae40 0x7fd56006d300 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd570035000 tx=0x7fd570004730 comp rx=0 tx=0).stop 2023-10-26T01:32:14.782 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:14.777+0000 7fd5852d7640 1 -- 172.21.15.169:0/3795182307 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5801592d0 msgr2=0x7fd580149420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:14.782 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:14.777+0000 7fd5852d7640 1 --2- 172.21.15.169:0/3795182307 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5801592d0 0x7fd580149420 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fd57800eb10 tx=0x7fd57800c2d0 comp rx=0 tx=0).stop 2023-10-26T01:32:14.782 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:14.777+0000 7fd5852d7640 1 -- 172.21.15.169:0/3795182307 shutdown_connections 2023-10-26T01:32:14.783 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:14.777+0000 7fd5852d7640 1 --2- 172.21.15.169:0/3795182307 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd56006ae40 0x7fd56006d300 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:14.783 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:14.777+0000 7fd5852d7640 1 --2- 172.21.15.169:0/3795182307 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5801592d0 0x7fd580149420 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:14.783 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:14.777+0000 7fd5852d7640 1 --2- 172.21.15.169:0/3795182307 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd5800cde00 0x7fd5800c8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:14.783 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:14.777+0000 7fd5852d7640 1 --2- 172.21.15.169:0/3795182307 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd5800ac930 0x7fd580146c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:14.783 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:14.777+0000 7fd5852d7640 1 -- 172.21.15.169:0/3795182307 >> 172.21.15.169:0/3795182307 conn(0x7fd580069930 msgr2=0x7fd580070dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:14.784 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:14.777+0000 7fd5852d7640 1 -- 172.21.15.169:0/3795182307 shutdown_connections 2023-10-26T01:32:14.784 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:14.777+0000 7fd5852d7640 1 -- 172.21.15.169:0/3795182307 wait complete. 2023-10-26T01:32:14.798 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T01:30:32.200808+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-26T01:31:55.401509+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2023-10-26T01:31:56.776031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T01:32:14.799 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T01:32:15.025 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.023+0000 7f2bf24e1640 1 -- 172.21.15.169:0/3941276136 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2bec0cd760 msgr2=0x7f2bec0cdc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:15.026 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.023+0000 7f2bf24e1640 1 --2- 172.21.15.169:0/3941276136 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2bec0cd760 0x7f2bec0cdc10 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f2bec06c3b0 tx=0x7f2be402bd90 comp rx=0 tx=0).stop 2023-10-26T01:32:15.026 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.023+0000 7f2bf24e1640 1 -- 172.21.15.169:0/3941276136 shutdown_connections 2023-10-26T01:32:15.026 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.023+0000 7f2bf24e1640 1 --2- 172.21.15.169:0/3941276136 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2bec0cd760 0x7f2bec0cdc10 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:15.026 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.023+0000 7f2bf24e1640 1 --2- 172.21.15.169:0/3941276136 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2bec0b0a80 0x7f2bec158f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:15.027 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.023+0000 7f2bf24e1640 1 --2- 172.21.15.169:0/3941276136 >> v2:172.21.15.170:3300/0 conn(0x7f2bec0ab070 0x7f2bec0ac870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:15.027 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.023+0000 7f2bf24e1640 1 -- 172.21.15.169:0/3941276136 >> 172.21.15.169:0/3941276136 conn(0x7f2bec069930 msgr2=0x7f2bec069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:15.027 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.023+0000 7f2bf24e1640 1 -- 172.21.15.169:0/3941276136 shutdown_connections 2023-10-26T01:32:15.027 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.023+0000 7f2bf24e1640 1 -- 172.21.15.169:0/3941276136 wait complete. 2023-10-26T01:32:15.030 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.025+0000 7f2bf24e1640 1 Processor -- start 2023-10-26T01:32:15.030 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.025+0000 7f2bf24e1640 1 -- start start 2023-10-26T01:32:15.030 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.026+0000 7f2bf24e1640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2bec0ab070 0x7f2bec0c66d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:15.030 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.026+0000 7f2bf24e1640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2bec0cd760 0x7f2bec0c72c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:15.031 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.026+0000 7f2bf24e1640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2bec14d9a0 0x7f2bec14f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:15.031 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.026+0000 7f2bf24e1640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bec0c79c0 con 0x7f2bec0cd760 2023-10-26T01:32:15.031 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.026+0000 7f2bf24e1640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bec0c7ca0 con 0x7f2bec14d9a0 2023-10-26T01:32:15.031 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.026+0000 7f2bf24e1640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bec14ece0 con 0x7f2bec0ab070 2023-10-26T01:32:15.032 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.026+0000 7f2beb7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2bec0cd760 0x7f2bec0c72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:15.032 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.027+0000 7f2beb7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2bec0cd760 0x7f2bec0c72c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:41890/0 (socket says 172.21.15.169:41890) 2023-10-26T01:32:15.032 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.027+0000 7f2beb7fe640 1 -- 172.21.15.169:0/1099920603 learned_addr learned my addr 172.21.15.169:0/1099920603 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:32:15.032 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.027+0000 7f2bebfff640 1 --2- 172.21.15.169:0/1099920603 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2bec0ab070 0x7f2bec0c66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:15.033 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.027+0000 7f2bf0a7e640 1 --2- 172.21.15.169:0/1099920603 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2bec14d9a0 0x7f2bec14f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:15.033 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.027+0000 7f2beb7fe640 1 -- 172.21.15.169:0/1099920603 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2bec0ab070 msgr2=0x7f2bec0c66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:15.033 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.027+0000 7f2beb7fe640 1 --2- 172.21.15.169:0/1099920603 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2bec0ab070 0x7f2bec0c66d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:15.033 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.027+0000 7f2beb7fe640 1 -- 172.21.15.169:0/1099920603 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2bec14d9a0 msgr2=0x7f2bec14f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:15.033 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.027+0000 7f2beb7fe640 1 --2- 172.21.15.169:0/1099920603 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2bec14d9a0 0x7f2bec14f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:15.034 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.027+0000 7f2beb7fe640 1 -- 172.21.15.169:0/1099920603 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2be4009560 con 0x7f2bec0cd760 2023-10-26T01:32:15.034 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.027+0000 7f2bf0a7e640 1 --2- 172.21.15.169:0/1099920603 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2bec14d9a0 0x7f2bec14f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:32:15.034 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.027+0000 7f2beb7fe640 1 --2- 172.21.15.169:0/1099920603 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2bec0cd760 0x7f2bec0c72c0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc00ea80 tx=0x7f2bdc00ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:15.034 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.028+0000 7f2be8ff9640 1 -- 172.21.15.169:0/1099920603 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bdc00ce40 con 0x7f2bec0cd760 2023-10-26T01:32:15.034 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.028+0000 7f2be8ff9640 1 -- 172.21.15.169:0/1099920603 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2bdc004c10 con 0x7f2bec0cd760 2023-10-26T01:32:15.035 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.028+0000 7f2be8ff9640 1 -- 172.21.15.169:0/1099920603 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bdc005020 con 0x7f2bec0cd760 2023-10-26T01:32:15.035 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.028+0000 7f2bf24e1640 1 -- 172.21.15.169:0/1099920603 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bec14efc0 con 0x7f2bec0cd760 2023-10-26T01:32:15.035 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.028+0000 7f2bf24e1640 1 -- 172.21.15.169:0/1099920603 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bec14fa20 con 0x7f2bec0cd760 2023-10-26T01:32:15.036 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.029+0000 7f2bf24e1640 1 -- 172.21.15.169:0/1099920603 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bec066af0 con 0x7f2bec0cd760 2023-10-26T01:32:15.036 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.030+0000 7f2be8ff9640 1 -- 172.21.15.169:0/1099920603 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f2bdc005180 con 0x7f2bec0cd760 2023-10-26T01:32:15.036 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.031+0000 7f2be8ff9640 1 --2- 172.21.15.169:0/1099920603 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2bcc06ae40 0x7f2bcc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:15.037 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.031+0000 7f2be8ff9640 1 -- 172.21.15.169:0/1099920603 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2bdc08a1b0 con 0x7f2bec0cd760 2023-10-26T01:32:15.037 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.031+0000 7f2bebfff640 1 --2- 172.21.15.169:0/1099920603 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2bcc06ae40 0x7f2bcc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:15.037 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.032+0000 7f2bebfff640 1 --2- 172.21.15.169:0/1099920603 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2bcc06ae40 0x7f2bcc06d300 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2bd4005fd0 tx=0x7f2bd400c460 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:15.038 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.035+0000 7f2be8ff9640 1 -- 172.21.15.169:0/1099920603 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2bdc0551c0 con 0x7f2bec0cd760 2023-10-26T01:32:15.223 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.222+0000 7f2bf24e1640 1 -- 172.21.15.169:0/1099920603 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2bec14f1b0 con 0x7f2bec0cd760 2023-10-26T01:32:15.224 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.223+0000 7f2be8ff9640 1 -- 172.21.15.169:0/1099920603 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v37) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2bdc01f020 con 0x7f2bec0cd760 2023-10-26T01:32:15.225 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 1 2023-10-26T01:32:15.227 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.225+0000 7f2bf24e1640 1 -- 172.21.15.169:0/1099920603 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2bcc06ae40 msgr2=0x7f2bcc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:15.228 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.225+0000 7f2bf24e1640 1 --2- 172.21.15.169:0/1099920603 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2bcc06ae40 0x7f2bcc06d300 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2bd4005fd0 tx=0x7f2bd400c460 comp rx=0 tx=0).stop 2023-10-26T01:32:15.228 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.225+0000 7f2bf24e1640 1 -- 172.21.15.169:0/1099920603 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2bec0cd760 msgr2=0x7f2bec0c72c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:15.228 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.225+0000 7f2bf24e1640 1 --2- 172.21.15.169:0/1099920603 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2bec0cd760 0x7f2bec0c72c0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc00ea80 tx=0x7f2bdc00ef40 comp rx=0 tx=0).stop 2023-10-26T01:32:15.228 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.226+0000 7f2bf24e1640 1 -- 172.21.15.169:0/1099920603 shutdown_connections 2023-10-26T01:32:15.228 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.226+0000 7f2bf24e1640 1 --2- 172.21.15.169:0/1099920603 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2bcc06ae40 0x7f2bcc06d300 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:15.229 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.226+0000 7f2bf24e1640 1 --2- 172.21.15.169:0/1099920603 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2bec0cd760 0x7f2bec0c72c0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:15.229 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.226+0000 7f2bf24e1640 1 --2- 172.21.15.169:0/1099920603 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2bec14d9a0 0x7f2bec14f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:15.229 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.226+0000 7f2bf24e1640 1 --2- 172.21.15.169:0/1099920603 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2bec0ab070 0x7f2bec0c66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:15.229 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.226+0000 7f2bf24e1640 1 -- 172.21.15.169:0/1099920603 >> 172.21.15.169:0/1099920603 conn(0x7f2bec069930 msgr2=0x7f2bec0acf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:15.229 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.226+0000 7f2bf24e1640 1 -- 172.21.15.169:0/1099920603 shutdown_connections 2023-10-26T01:32:15.230 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.226+0000 7f2bf24e1640 1 -- 172.21.15.169:0/1099920603 wait complete. 2023-10-26T01:32:15.247 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-26T01:32:15.470 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.468+0000 7fe98af87640 1 -- 172.21.15.169:0/1472679641 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9840b0300 msgr2=0x7fe9840b0750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:15.471 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.468+0000 7fe98af87640 1 --2- 172.21.15.169:0/1472679641 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9840b0300 0x7fe9840b0750 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fe97c005fd0 tx=0x7fe97c02d390 comp rx=0 tx=0).stop 2023-10-26T01:32:15.471 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.468+0000 7fe98af87640 1 -- 172.21.15.169:0/1472679641 shutdown_connections 2023-10-26T01:32:15.471 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.468+0000 7fe98af87640 1 --2- 172.21.15.169:0/1472679641 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9840b0300 0x7fe9840b0750 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:15.471 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.468+0000 7fe98af87640 1 -- 172.21.15.169:0/1472679641 >> 172.21.15.169:0/1472679641 conn(0x7fe9840af110 msgr2=0x7fe9840af520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:15.471 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.468+0000 7fe98af87640 1 -- 172.21.15.169:0/1472679641 shutdown_connections 2023-10-26T01:32:15.472 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.468+0000 7fe98af87640 1 -- 172.21.15.169:0/1472679641 wait complete. 2023-10-26T01:32:15.472 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.470+0000 7fe98af87640 1 Processor -- start 2023-10-26T01:32:15.472 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.470+0000 7fe98af87640 1 -- start start 2023-10-26T01:32:15.474 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.471+0000 7fe98af87640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe9840b0300 0x7fe984070db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:15.474 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.471+0000 7fe98af87640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe984157f80 0x7fe984159250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:15.475 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.471+0000 7fe98af87640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe98415b140 0x7fe984159790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:15.475 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.471+0000 7fe98af87640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9841e46e0 con 0x7fe984157f80 2023-10-26T01:32:15.475 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.471+0000 7fe98af87640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe984159e10 con 0x7fe98415b140 2023-10-26T01:32:15.475 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.471+0000 7fe98af87640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe98415a0f0 con 0x7fe9840b0300 2023-10-26T01:32:15.475 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.471+0000 7fe989784640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe984157f80 0x7fe984159250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:15.476 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.471+0000 7fe989784640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe984157f80 0x7fe984159250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:41906/0 (socket says 172.21.15.169:41906) 2023-10-26T01:32:15.476 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.471+0000 7fe989784640 1 -- 172.21.15.169:0/1728727947 learned_addr learned my addr 172.21.15.169:0/1728727947 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:32:15.476 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.472+0000 7fe989784640 1 -- 172.21.15.169:0/1728727947 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe9840b0300 msgr2=0x7fe984070db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:15.476 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.472+0000 7fe989f85640 1 --2- 172.21.15.169:0/1728727947 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe9840b0300 0x7fe984070db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:15.476 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.472+0000 7fe98a786640 1 --2- 172.21.15.169:0/1728727947 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe98415b140 0x7fe984159790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:15.477 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.472+0000 7fe989784640 1 --2- 172.21.15.169:0/1728727947 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe9840b0300 0x7fe984070db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:15.477 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.472+0000 7fe989784640 1 -- 172.21.15.169:0/1728727947 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe98415b140 msgr2=0x7fe984159790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:15.477 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.472+0000 7fe989784640 1 --2- 172.21.15.169:0/1728727947 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe98415b140 0x7fe984159790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:15.477 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.472+0000 7fe989784640 1 -- 172.21.15.169:0/1728727947 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe97c00a950 con 0x7fe984157f80 2023-10-26T01:32:15.477 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.472+0000 7fe989f85640 1 --2- 172.21.15.169:0/1728727947 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe9840b0300 0x7fe984070db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:32:15.478 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.472+0000 7fe989784640 1 --2- 172.21.15.169:0/1728727947 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe984157f80 0x7fe984159250 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fe97400c500 tx=0x7fe97400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:15.478 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.472+0000 7fe97a7fc640 1 -- 172.21.15.169:0/1728727947 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe974012830 con 0x7fe984157f80 2023-10-26T01:32:15.478 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.473+0000 7fe97a7fc640 1 -- 172.21.15.169:0/1728727947 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe974012a80 con 0x7fe984157f80 2023-10-26T01:32:15.478 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.473+0000 7fe97a7fc640 1 -- 172.21.15.169:0/1728727947 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe974012e90 con 0x7fe984157f80 2023-10-26T01:32:15.478 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.473+0000 7fe98af87640 1 -- 172.21.15.169:0/1728727947 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe98415a3d0 con 0x7fe984157f80 2023-10-26T01:32:15.479 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.473+0000 7fe98af87640 1 -- 172.21.15.169:0/1728727947 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9840712f0 con 0x7fe984157f80 2023-10-26T01:32:15.480 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.474+0000 7fe98af87640 1 -- 172.21.15.169:0/1728727947 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe98415abd0 con 0x7fe984157f80 2023-10-26T01:32:15.480 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.475+0000 7fe97a7fc640 1 -- 172.21.15.169:0/1728727947 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fe97401a3b0 con 0x7fe984157f80 2023-10-26T01:32:15.480 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.476+0000 7fe97a7fc640 1 --2- 172.21.15.169:0/1728727947 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe96806ae40 0x7fe96806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:15.480 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.476+0000 7fe97a7fc640 1 -- 172.21.15.169:0/1728727947 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe974089780 con 0x7fe984157f80 2023-10-26T01:32:15.480 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.476+0000 7fe989f85640 1 --2- 172.21.15.169:0/1728727947 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe96806ae40 0x7fe96806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:15.481 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.477+0000 7fe989f85640 1 --2- 172.21.15.169:0/1728727947 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe96806ae40 0x7fe96806d300 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe980003eb0 tx=0x7fe980009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:15.481 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.480+0000 7fe97a7fc640 1 -- 172.21.15.169:0/1728727947 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe98415abd0 con 0x7fe984157f80 2023-10-26T01:32:15.669 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.667+0000 7fe98af87640 1 -- 172.21.15.169:0/1728727947 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe98415abd0 con 0x7fe984157f80 2023-10-26T01:32:15.670 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.668+0000 7fe97a7fc640 1 -- 172.21.15.169:0/1728727947 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v37) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe98415abd0 con 0x7fe984157f80 2023-10-26T01:32:15.670 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 32 2023-10-26T01:32:15.673 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.671+0000 7fe98af87640 1 -- 172.21.15.169:0/1728727947 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe96806ae40 msgr2=0x7fe96806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:15.673 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.671+0000 7fe98af87640 1 --2- 172.21.15.169:0/1728727947 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe96806ae40 0x7fe96806d300 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe980003eb0 tx=0x7fe980009040 comp rx=0 tx=0).stop 2023-10-26T01:32:15.673 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.671+0000 7fe98af87640 1 -- 172.21.15.169:0/1728727947 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe984157f80 msgr2=0x7fe984159250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:15.674 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.671+0000 7fe98af87640 1 --2- 172.21.15.169:0/1728727947 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe984157f80 0x7fe984159250 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fe97400c500 tx=0x7fe97400c9c0 comp rx=0 tx=0).stop 2023-10-26T01:32:15.674 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.671+0000 7fe98af87640 1 -- 172.21.15.169:0/1728727947 shutdown_connections 2023-10-26T01:32:15.674 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.671+0000 7fe98af87640 1 --2- 172.21.15.169:0/1728727947 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe96806ae40 0x7fe96806d300 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:15.674 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.671+0000 7fe98af87640 1 --2- 172.21.15.169:0/1728727947 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe984157f80 0x7fe984159250 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:15.675 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.671+0000 7fe98af87640 1 --2- 172.21.15.169:0/1728727947 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe98415b140 0x7fe984159790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:15.675 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.671+0000 7fe98af87640 1 --2- 172.21.15.169:0/1728727947 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe9840b0300 0x7fe984070db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:15.675 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.671+0000 7fe98af87640 1 -- 172.21.15.169:0/1728727947 >> 172.21.15.169:0/1728727947 conn(0x7fe9840af110 msgr2=0x7fe9841513f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:15.675 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.671+0000 7fe98af87640 1 -- 172.21.15.169:0/1728727947 shutdown_connections 2023-10-26T01:32:15.676 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.671+0000 7fe98af87640 1 -- 172.21.15.169:0/1728727947 wait complete. 2023-10-26T01:32:15.692 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-26T01:32:15.915 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.912+0000 7f8d4117d640 1 -- 172.21.15.169:0/2338221941 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8d28004d00 msgr2=0x7f8d28005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:15.915 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.912+0000 7f8d4117d640 1 --2- 172.21.15.169:0/2338221941 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8d28004d00 0x7f8d28005180 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f8d3400c640 tx=0x7f8d280057f0 comp rx=0 tx=0).stop 2023-10-26T01:32:15.916 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.912+0000 7f8d3b59e640 1 -- 172.21.15.169:0/2338221941 reap_dead start 2023-10-26T01:32:15.916 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.912+0000 7f8d4117d640 1 -- 172.21.15.169:0/2338221941 shutdown_connections 2023-10-26T01:32:15.916 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.912+0000 7f8d4117d640 1 -- 172.21.15.169:0/2338221941 >> 172.21.15.169:0/2338221941 conn(0x7f8d3c069930 msgr2=0x7f8d3c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:15.917 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.913+0000 7f8d4117d640 1 -- 172.21.15.169:0/2338221941 shutdown_connections 2023-10-26T01:32:15.917 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.913+0000 7f8d4117d640 1 -- 172.21.15.169:0/2338221941 wait complete. 2023-10-26T01:32:15.920 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.914+0000 7f8d4117d640 1 Processor -- start 2023-10-26T01:32:15.920 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.914+0000 7f8d4117d640 1 -- start start 2023-10-26T01:32:15.920 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.915+0000 7f8d4117d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d28004d00 0x7f8d3c14c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:15.921 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.915+0000 7f8d4117d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8d3c14ccd0 0x7f8d3c0c8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:15.921 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.915+0000 7f8d4117d640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f8d3c15bc40 0x7f8d3c0c6660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:15.921 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.915+0000 7f8d4117d640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d3c0c9420 con 0x7f8d3c14ccd0 2023-10-26T01:32:15.922 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.915+0000 7f8d4117d640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d3c0cc360 con 0x7f8d28004d00 2023-10-26T01:32:15.922 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.915+0000 7f8d4117d640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d3c0c96a0 con 0x7f8d3c15bc40 2023-10-26T01:32:15.922 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.916+0000 7f8d33fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8d3c14ccd0 0x7f8d3c0c8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:15.923 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.916+0000 7f8d33fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8d3c14ccd0 0x7f8d3c0c8ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:41922/0 (socket says 172.21.15.169:41922) 2023-10-26T01:32:15.923 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.916+0000 7f8d33fff640 1 -- 172.21.15.169:0/2086455549 learned_addr learned my addr 172.21.15.169:0/2086455549 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:32:15.923 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.916+0000 7f8d3ad9d640 1 --2- 172.21.15.169:0/2086455549 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d28004d00 0x7f8d3c14c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:15.924 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.916+0000 7f8d33fff640 1 -- 172.21.15.169:0/2086455549 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f8d3c15bc40 msgr2=0x7f8d3c0c6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:15.924 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.916+0000 7f8d33fff640 1 --2- 172.21.15.169:0/2086455549 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f8d3c15bc40 0x7f8d3c0c6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:15.924 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.916+0000 7f8d33fff640 1 -- 172.21.15.169:0/2086455549 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d28004d00 msgr2=0x7f8d3c14c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:15.924 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.916+0000 7f8d33fff640 1 --2- 172.21.15.169:0/2086455549 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d28004d00 0x7f8d3c14c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:15.924 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.916+0000 7f8d33fff640 1 -- 172.21.15.169:0/2086455549 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d3c151f60 con 0x7f8d3c14ccd0 2023-10-26T01:32:15.925 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.916+0000 7f8d33fff640 1 --2- 172.21.15.169:0/2086455549 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8d3c14ccd0 0x7f8d3c0c8ee0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f8d2400e970 tx=0x7f8d2400ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:15.925 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.917+0000 7f8d337fe640 1 -- 172.21.15.169:0/2086455549 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d2400ccc0 con 0x7f8d3c14ccd0 2023-10-26T01:32:15.925 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.917+0000 7f8d337fe640 1 -- 172.21.15.169:0/2086455549 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8d2400ce20 con 0x7f8d3c14ccd0 2023-10-26T01:32:15.925 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.917+0000 7f8d337fe640 1 -- 172.21.15.169:0/2086455549 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d24005280 con 0x7f8d3c14ccd0 2023-10-26T01:32:15.925 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.917+0000 7f8d4117d640 1 -- 172.21.15.169:0/2086455549 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d3c152240 con 0x7f8d3c14ccd0 2023-10-26T01:32:15.926 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.917+0000 7f8d4117d640 1 -- 172.21.15.169:0/2086455549 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d3c152800 con 0x7f8d3c14ccd0 2023-10-26T01:32:15.926 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.918+0000 7f8d4117d640 1 -- 172.21.15.169:0/2086455549 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d3c04db10 con 0x7f8d3c14ccd0 2023-10-26T01:32:15.927 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.919+0000 7f8d337fe640 1 -- 172.21.15.169:0/2086455549 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f8d240053e0 con 0x7f8d3c14ccd0 2023-10-26T01:32:15.927 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.920+0000 7f8d337fe640 1 --2- 172.21.15.169:0/2086455549 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f8d1c06ae70 0x7f8d1c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:15.927 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.920+0000 7f8d337fe640 1 -- 172.21.15.169:0/2086455549 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8d24014070 con 0x7f8d3c14ccd0 2023-10-26T01:32:15.927 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.921+0000 7f8d3ad9d640 1 --2- 172.21.15.169:0/2086455549 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f8d1c06ae70 0x7f8d1c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:15.928 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.922+0000 7f8d3ad9d640 1 --2- 172.21.15.169:0/2086455549 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f8d1c06ae70 0x7f8d1c06d330 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c009460 tx=0x7f8d2c009370 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:15.928 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:15.924+0000 7f8d337fe640 1 -- 172.21.15.169:0/2086455549 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8d240550a0 con 0x7f8d3c14ccd0 2023-10-26T01:32:16.117 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:16.115+0000 7f8d4117d640 1 -- 172.21.15.169:0/2086455549 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8d3c154160 con 0x7f8d3c14ccd0 2023-10-26T01:32:16.118 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:16.116+0000 7f8d337fe640 1 -- 172.21.15.169:0/2086455549 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v37) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8d240586c0 con 0x7f8d3c14ccd0 2023-10-26T01:32:16.119 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 64 2023-10-26T01:32:16.121 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:16.118+0000 7f8d4117d640 1 -- 172.21.15.169:0/2086455549 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f8d1c06ae70 msgr2=0x7f8d1c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:16.121 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:16.118+0000 7f8d4117d640 1 --2- 172.21.15.169:0/2086455549 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f8d1c06ae70 0x7f8d1c06d330 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c009460 tx=0x7f8d2c009370 comp rx=0 tx=0).stop 2023-10-26T01:32:16.121 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:16.118+0000 7f8d4117d640 1 -- 172.21.15.169:0/2086455549 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8d3c14ccd0 msgr2=0x7f8d3c0c8ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:16.122 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:16.118+0000 7f8d4117d640 1 --2- 172.21.15.169:0/2086455549 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8d3c14ccd0 0x7f8d3c0c8ee0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f8d2400e970 tx=0x7f8d2400ee30 comp rx=0 tx=0).stop 2023-10-26T01:32:16.122 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:16.118+0000 7f8d4117d640 1 -- 172.21.15.169:0/2086455549 shutdown_connections 2023-10-26T01:32:16.122 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:16.118+0000 7f8d4117d640 1 --2- 172.21.15.169:0/2086455549 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f8d1c06ae70 0x7f8d1c06d330 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:16.122 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:16.118+0000 7f8d4117d640 1 --2- 172.21.15.169:0/2086455549 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8d3c14ccd0 0x7f8d3c0c8ee0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:16.123 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:16.118+0000 7f8d4117d640 1 --2- 172.21.15.169:0/2086455549 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f8d3c15bc40 0x7f8d3c0c6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:16.123 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:16.118+0000 7f8d4117d640 1 --2- 172.21.15.169:0/2086455549 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d28004d00 0x7f8d3c14c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:16.123 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:16.118+0000 7f8d4117d640 1 -- 172.21.15.169:0/2086455549 >> 172.21.15.169:0/2086455549 conn(0x7f8d3c069930 msgr2=0x7f8d3c0caf40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:16.123 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:16.119+0000 7f8d4117d640 1 -- 172.21.15.169:0/2086455549 shutdown_connections 2023-10-26T01:32:16.123 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:16.119+0000 7f8d4117d640 1 -- 172.21.15.169:0/2086455549 wait complete. 2023-10-26T01:32:16.140 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-26T01:32:16.355 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:16.352+0000 7f25e3400640 1 -- 172.21.15.169:0/4064058203 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25dc0b0c80 msgr2=0x7f25dc0cd710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:16.355 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:16.352+0000 7f25e3400640 1 --2- 172.21.15.169:0/4064058203 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25dc0b0c80 0x7f25dc0cd710 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f25d4005fd0 tx=0x7f25d402c0e0 comp rx=0 tx=0).stop 2023-10-26T01:32:16.355 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:16.352+0000 7f25e3400640 1 -- 172.21.15.169:0/4064058203 shutdown_connections 2023-10-26T01:32:16.356 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:16.352+0000 7f25e3400640 1 --2- 172.21.15.169:0/4064058203 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25dc0b0c80 0x7f25dc0cd710 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:16.356 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:16.352+0000 7f25e3400640 1 --2- 172.21.15.169:0/4064058203 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f25dc1593e0 0x7f25dc0ac200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:16.356 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:16.352+0000 7f25e3400640 1 -- 172.21.15.169:0/4064058203 >> 172.21.15.169:0/4064058203 conn(0x7f25dc069930 msgr2=0x7f25dc069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:16.356 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:16.352+0000 7f25e3400640 1 -- 172.21.15.169:0/4064058203 shutdown_connections 2023-10-26T01:32:16.356 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:16.352+0000 7f25e3400640 1 -- 172.21.15.169:0/4064058203 wait complete. 2023-10-26T01:32:16.357 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:16.354+0000 7f25e3400640 1 Processor -- start 2023-10-26T01:32:16.357 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:16.354+0000 7f25e3400640 1 -- start start 2023-10-26T01:32:16.359 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:16.355+0000 7f25e3400640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f25dc0b0c80 0x7f25dc14a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:16.359 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:16.355+0000 7f25e3400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25dc1593e0 0x7f25dc0c8870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:16.359 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:16.355+0000 7f25e3400640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25dc1597c0 0x7f25dc0c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:16.360 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:16.355+0000 7f25e3400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25dc14ad10 con 0x7f25dc1593e0 2023-10-26T01:32:16.360 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:16.355+0000 7f25e3400640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25dc157990 con 0x7f25dc1597c0 2023-10-26T01:32:16.360 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:16.355+0000 7f25e3400640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25dc157c50 con 0x7f25dc0b0c80 2023-10-26T01:32:16.360 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:16.355+0000 7f25e099b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25dc1593e0 0x7f25dc0c8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:16.360 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:16.355+0000 7f25e099b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25dc1593e0 0x7f25dc0c8870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:41936/0 (socket says 172.21.15.169:41936) 2023-10-26T01:32:16.361 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:16.355+0000 7f25e099b640 1 -- 172.21.15.169:0/1353702380 learned_addr learned my addr 172.21.15.169:0/1353702380 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:32:16.361 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:16.356+0000 7f25e099b640 1 -- 172.21.15.169:0/1353702380 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f25dc0b0c80 msgr2=0x7f25dc14a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:16.361 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:16.356+0000 7f25e099b640 1 --2- 172.21.15.169:0/1353702380 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f25dc0b0c80 0x7f25dc14a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:16.361 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:16.356+0000 7f25e099b640 1 -- 172.21.15.169:0/1353702380 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25dc1597c0 msgr2=0x7f25dc0c8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:16.361 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:16.356+0000 7f25e099b640 1 --2- 172.21.15.169:0/1353702380 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25dc1597c0 0x7f25dc0c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:16.362 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:16.356+0000 7f25e099b640 1 -- 172.21.15.169:0/1353702380 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25d4009840 con 0x7f25dc1593e0 2023-10-26T01:32:16.362 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:16.356+0000 7f25e099b640 1 --2- 172.21.15.169:0/1353702380 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25dc1593e0 0x7f25dc0c8870 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f25cc00c410 tx=0x7f25cc00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:16.362 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:16.356+0000 7f25d1ffb640 1 -- 172.21.15.169:0/1353702380 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25cc0035f0 con 0x7f25dc1593e0 2023-10-26T01:32:16.362 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:16.356+0000 7f25d1ffb640 1 -- 172.21.15.169:0/1353702380 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f25cc003770 con 0x7f25dc1593e0 2023-10-26T01:32:16.362 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:16.356+0000 7f25d1ffb640 1 -- 172.21.15.169:0/1353702380 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25cc003b80 con 0x7f25dc1593e0 2023-10-26T01:32:16.363 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:16.356+0000 7f25e3400640 1 -- 172.21.15.169:0/1353702380 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25dc157f30 con 0x7f25dc1593e0 2023-10-26T01:32:16.363 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:16.356+0000 7f25e3400640 1 -- 172.21.15.169:0/1353702380 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25dc1584f0 con 0x7f25dc1593e0 2023-10-26T01:32:16.364 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:16.358+0000 7f25e3400640 1 -- 172.21.15.169:0/1353702380 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25dc158700 con 0x7f25dc1593e0 2023-10-26T01:32:16.364 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:16.359+0000 7f25d1ffb640 1 -- 172.21.15.169:0/1353702380 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f25cc003ce0 con 0x7f25dc1593e0 2023-10-26T01:32:16.364 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:16.359+0000 7f25d1ffb640 1 --2- 172.21.15.169:0/1353702380 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f25c006ae40 0x7f25c006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:16.364 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:16.360+0000 7f25d1ffb640 1 -- 172.21.15.169:0/1353702380 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f25cc088dc0 con 0x7f25dc1593e0 2023-10-26T01:32:16.365 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:16.360+0000 7f25e119c640 1 --2- 172.21.15.169:0/1353702380 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f25c006ae40 0x7f25c006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:16.365 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:16.361+0000 7f25e119c640 1 --2- 172.21.15.169:0/1353702380 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f25c006ae40 0x7f25c006d300 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f25d800a8f0 tx=0x7f25d8005bf0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:16.366 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:16.364+0000 7f25d1ffb640 1 -- 172.21.15.169:0/1353702380 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f25cc053e70 con 0x7f25dc1593e0 2023-10-26T01:32:16.570 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:16.568+0000 7f25e3400640 1 -- 172.21.15.169:0/1353702380 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f25dc04db10 con 0x7f25dc1593e0 2023-10-26T01:32:16.571 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:16.569+0000 7f25d1ffb640 1 -- 172.21.15.169:0/1353702380 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v37) v1 ==== 80+0+0 (secure 0 0 0) 0x7f25cc017070 con 0x7f25dc1593e0 2023-10-26T01:32:16.571 INFO:teuthology.orchestra.run.smithi169.stderr:listed 0 entries 2023-10-26T01:32:16.574 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:16.571+0000 7f25e3400640 1 -- 172.21.15.169:0/1353702380 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f25c006ae40 msgr2=0x7f25c006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:16.574 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:16.571+0000 7f25e3400640 1 --2- 172.21.15.169:0/1353702380 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f25c006ae40 0x7f25c006d300 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f25d800a8f0 tx=0x7f25d8005bf0 comp rx=0 tx=0).stop 2023-10-26T01:32:16.575 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:16.571+0000 7f25e3400640 1 -- 172.21.15.169:0/1353702380 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25dc1593e0 msgr2=0x7f25dc0c8870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:16.575 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:16.571+0000 7f25e3400640 1 --2- 172.21.15.169:0/1353702380 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25dc1593e0 0x7f25dc0c8870 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f25cc00c410 tx=0x7f25cc00c8d0 comp rx=0 tx=0).stop 2023-10-26T01:32:16.575 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:16.571+0000 7f25e3400640 1 -- 172.21.15.169:0/1353702380 shutdown_connections 2023-10-26T01:32:16.575 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:16.571+0000 7f25e3400640 1 --2- 172.21.15.169:0/1353702380 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f25c006ae40 0x7f25c006d300 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:16.576 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:16.571+0000 7f25e3400640 1 --2- 172.21.15.169:0/1353702380 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25dc1593e0 0x7f25dc0c8870 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:16.576 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:16.571+0000 7f25e3400640 1 --2- 172.21.15.169:0/1353702380 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25dc1597c0 0x7f25dc0c8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:16.576 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:16.571+0000 7f25e3400640 1 --2- 172.21.15.169:0/1353702380 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f25dc0b0c80 0x7f25dc14a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:16.576 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:16.571+0000 7f25e3400640 1 -- 172.21.15.169:0/1353702380 >> 172.21.15.169:0/1353702380 conn(0x7f25dc069930 msgr2=0x7f25dc152900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:16.576 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:16.571+0000 7f25e3400640 1 -- 172.21.15.169:0/1353702380 shutdown_connections 2023-10-26T01:32:16.577 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:16.572+0000 7f25e3400640 1 -- 172.21.15.169:0/1353702380 wait complete. 2023-10-26T01:32:16.593 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-26T01:32:16.594 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:32:16.594 DEBUG:teuthology.orchestra.run.smithi169:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-26T01:32:16.716 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi169.front.sepia.ceph.com 2023-10-26T01:32:16.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:32:16.718 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-26T01:32:16.768 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-26T01:32:16.769 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:32:16.813 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-26T01:32:16.814 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:32:16.908 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:32:17.133 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:17.130+0000 7f52474bb640 1 -- 172.21.15.169:0/4284396542 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f522c003970 msgr2=0x7f522c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:17.134 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:17.130+0000 7f52474bb640 1 --2- 172.21.15.169:0/4284396542 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f522c003970 0x7f522c003d90 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f523c00bb20 tx=0x7f522c0057f0 comp rx=0 tx=0).stop 2023-10-26T01:32:17.134 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:17.130+0000 7f5245a58640 1 -- 172.21.15.169:0/4284396542 reap_dead start 2023-10-26T01:32:17.134 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:17.131+0000 7f52474bb640 1 -- 172.21.15.169:0/4284396542 shutdown_connections 2023-10-26T01:32:17.135 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:17.131+0000 7f52474bb640 1 -- 172.21.15.169:0/4284396542 >> 172.21.15.169:0/4284396542 conn(0x7f5240069930 msgr2=0x7f5240069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:17.135 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:17.131+0000 7f52474bb640 1 -- 172.21.15.169:0/4284396542 shutdown_connections 2023-10-26T01:32:17.135 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:17.131+0000 7f52474bb640 1 -- 172.21.15.169:0/4284396542 wait complete. 2023-10-26T01:32:17.135 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:17.132+0000 7f52474bb640 1 Processor -- start 2023-10-26T01:32:17.136 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:17.133+0000 7f52474bb640 1 -- start start 2023-10-26T01:32:17.137 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:17.133+0000 7f52474bb640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f522c003970 0x7f5240141600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:17.138 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:17.134+0000 7f52474bb640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f52401e1ca0 0x7f52401e2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:17.138 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:17.134+0000 7f52474bb640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52401e3be0 0x7f52401e4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:17.138 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:17.134+0000 7f52474bb640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f524015a0e0 con 0x7f52401e3be0 2023-10-26T01:32:17.138 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:17.134+0000 7f52474bb640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52401e2660 con 0x7f522c003970 2023-10-26T01:32:17.139 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:17.134+0000 7f52474bb640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f524014a7d0 con 0x7f52401e1ca0 2023-10-26T01:32:17.139 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:17.134+0000 7f5245a58640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52401e3be0 0x7f52401e4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:17.139 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:17.134+0000 7f5245a58640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52401e3be0 0x7f52401e4060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:41956/0 (socket says 172.21.15.169:41956) 2023-10-26T01:32:17.139 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:17.134+0000 7f5245a58640 1 -- 172.21.15.169:0/2710940932 learned_addr learned my addr 172.21.15.169:0/2710940932 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:32:17.139 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:17.134+0000 7f5244a56640 1 --2- 172.21.15.169:0/2710940932 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f52401e1ca0 0x7f52401e2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:17.140 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:17.134+0000 7f5245257640 1 --2- 172.21.15.169:0/2710940932 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f522c003970 0x7f5240141600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:17.140 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:17.134+0000 7f5245a58640 1 -- 172.21.15.169:0/2710940932 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f52401e1ca0 msgr2=0x7f52401e2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:17.140 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:17.134+0000 7f5245a58640 1 --2- 172.21.15.169:0/2710940932 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f52401e1ca0 0x7f52401e2120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:17.140 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:17.134+0000 7f5245a58640 1 -- 172.21.15.169:0/2710940932 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f522c003970 msgr2=0x7f5240141600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:17.140 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:17.134+0000 7f5245a58640 1 --2- 172.21.15.169:0/2710940932 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f522c003970 0x7f5240141600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:17.141 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:17.134+0000 7f5245a58640 1 -- 172.21.15.169:0/2710940932 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52401e48c0 con 0x7f52401e3be0 2023-10-26T01:32:17.141 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:17.135+0000 7f5245a58640 1 --2- 172.21.15.169:0/2710940932 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52401e3be0 0x7f52401e4060 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f5238005fd0 tx=0x7f523800eda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:17.141 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:17.135+0000 7f5235ffb640 1 -- 172.21.15.169:0/2710940932 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f523800cc00 con 0x7f52401e3be0 2023-10-26T01:32:17.141 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:17.135+0000 7f5235ffb640 1 -- 172.21.15.169:0/2710940932 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f523800ce50 con 0x7f52401e3be0 2023-10-26T01:32:17.141 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:17.135+0000 7f5235ffb640 1 -- 172.21.15.169:0/2710940932 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52380116d0 con 0x7f52401e3be0 2023-10-26T01:32:17.142 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:17.135+0000 7f52474bb640 1 -- 172.21.15.169:0/2710940932 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5240159a20 con 0x7f52401e3be0 2023-10-26T01:32:17.142 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:17.135+0000 7f52474bb640 1 -- 172.21.15.169:0/2710940932 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52401e45a0 con 0x7f52401e3be0 2023-10-26T01:32:17.143 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:17.136+0000 7f52474bb640 1 -- 172.21.15.169:0/2710940932 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f524015a220 con 0x7f52401e3be0 2023-10-26T01:32:17.143 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:17.137+0000 7f5235ffb640 1 -- 172.21.15.169:0/2710940932 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f5238011830 con 0x7f52401e3be0 2023-10-26T01:32:17.143 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:17.139+0000 7f5235ffb640 1 --2- 172.21.15.169:0/2710940932 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f522406ae40 0x7f522406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:17.143 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:17.139+0000 7f5235ffb640 1 -- 172.21.15.169:0/2710940932 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5238089f40 con 0x7f52401e3be0 2023-10-26T01:32:17.144 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:17.139+0000 7f5245257640 1 --2- 172.21.15.169:0/2710940932 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f522406ae40 0x7f522406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:17.144 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:17.140+0000 7f5245257640 1 --2- 172.21.15.169:0/2710940932 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f522406ae40 0x7f522406d300 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f523c037000 tx=0x7f523c02e040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:17.145 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:17.143+0000 7f5235ffb640 1 -- 172.21.15.169:0/2710940932 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f524015a220 con 0x7f52401e3be0 2023-10-26T01:32:17.334 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:17.331+0000 7f52474bb640 1 -- 172.21.15.169:0/2710940932 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5240159870 con 0x7f52401e3be0 2023-10-26T01:32:17.336 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:17.334+0000 7f5235ffb640 1 -- 172.21.15.169:0/2710940932 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+16617 (secure 0 0 0) 0x7f5238016090 con 0x7f52401e3be0 2023-10-26T01:32:17.337 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:32:17.337 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":37,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:32:11.943092+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-10-26T01:31:55.401509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-10-26T01:31:56.776031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:32:17.340 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:17.337+0000 7f52474bb640 1 -- 172.21.15.169:0/2710940932 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f522406ae40 msgr2=0x7f522406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:17.340 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:17.337+0000 7f52474bb640 1 --2- 172.21.15.169:0/2710940932 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f522406ae40 0x7f522406d300 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f523c037000 tx=0x7f523c02e040 comp rx=0 tx=0).stop 2023-10-26T01:32:17.340 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:17.337+0000 7f52474bb640 1 -- 172.21.15.169:0/2710940932 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52401e3be0 msgr2=0x7f52401e4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:17.341 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:17.337+0000 7f52474bb640 1 --2- 172.21.15.169:0/2710940932 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52401e3be0 0x7f52401e4060 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f5238005fd0 tx=0x7f523800eda0 comp rx=0 tx=0).stop 2023-10-26T01:32:17.341 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:17.337+0000 7f52474bb640 1 -- 172.21.15.169:0/2710940932 shutdown_connections 2023-10-26T01:32:17.341 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:17.337+0000 7f52474bb640 1 --2- 172.21.15.169:0/2710940932 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f522406ae40 0x7f522406d300 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:17.341 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:17.337+0000 7f52474bb640 1 --2- 172.21.15.169:0/2710940932 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52401e3be0 0x7f52401e4060 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:17.341 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:17.337+0000 7f52474bb640 1 --2- 172.21.15.169:0/2710940932 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f52401e1ca0 0x7f52401e2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:17.342 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:17.337+0000 7f52474bb640 1 --2- 172.21.15.169:0/2710940932 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f522c003970 0x7f5240141600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:17.342 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:17.337+0000 7f52474bb640 1 -- 172.21.15.169:0/2710940932 >> 172.21.15.169:0/2710940932 conn(0x7f5240069930 msgr2=0x7f52400c97f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:17.342 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:17.337+0000 7f52474bb640 1 -- 172.21.15.169:0/2710940932 shutdown_connections 2023-10-26T01:32:17.342 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:17.337+0000 7f52474bb640 1 -- 172.21.15.169:0/2710940932 wait complete. 2023-10-26T01:32:17.359 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T01:30:32.200808+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-26T01:31:55.401509+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2023-10-26T01:31:56.776031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T01:32:17.360 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T01:32:17.575 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:17.571+0000 7ffa6cb3c640 1 -- 172.21.15.169:0/2551266432 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa680b0a80 msgr2=0x7ffa680ac870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:17.575 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:17.571+0000 7ffa6cb3c640 1 --2- 172.21.15.169:0/2551266432 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa680b0a80 0x7ffa680ac870 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7ffa6806eb10 tx=0x7ffa5802bd90 comp rx=0 tx=0).stop 2023-10-26T01:32:17.575 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:17.572+0000 7ffa6cb3c640 1 -- 172.21.15.169:0/2551266432 shutdown_connections 2023-10-26T01:32:17.575 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:17.572+0000 7ffa6cb3c640 1 --2- 172.21.15.169:0/2551266432 >> v2:172.21.15.170:3300/0 conn(0x7ffa6806cdb0 0x7ffa680cd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:17.576 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:17.572+0000 7ffa6cb3c640 1 --2- 172.21.15.169:0/2551266432 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa680b0a80 0x7ffa680ac870 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:17.576 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:17.572+0000 7ffa6cb3c640 1 -- 172.21.15.169:0/2551266432 >> 172.21.15.169:0/2551266432 conn(0x7ffa68069930 msgr2=0x7ffa68069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:17.576 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:17.572+0000 7ffa6cb3c640 1 -- 172.21.15.169:0/2551266432 shutdown_connections 2023-10-26T01:32:17.576 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:17.572+0000 7ffa6cb3c640 1 -- 172.21.15.169:0/2551266432 wait complete. 2023-10-26T01:32:17.577 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:17.574+0000 7ffa6cb3c640 1 Processor -- start 2023-10-26T01:32:17.577 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:17.574+0000 7ffa6cb3c640 1 -- start start 2023-10-26T01:32:17.578 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:17.575+0000 7ffa6cb3c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa6806cdb0 0x7ffa6814a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:17.578 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:17.575+0000 7ffa6cb3c640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ffa680ab070 0x7ffa68153a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:17.578 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:17.575+0000 7ffa6cb3c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffa68153fb0 0x7ffa6815a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:17.578 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:17.575+0000 7ffa6cb3c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa68154530 con 0x7ffa6806cdb0 2023-10-26T01:32:17.579 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:17.575+0000 7ffa6cb3c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa6814a7d0 con 0x7ffa68153fb0 2023-10-26T01:32:17.579 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:17.575+0000 7ffa6cb3c640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa68152610 con 0x7ffa680ab070 2023-10-26T01:32:17.579 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:17.575+0000 7ffa6659c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa6806cdb0 0x7ffa6814a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:17.579 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:17.576+0000 7ffa6659c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa6806cdb0 0x7ffa6814a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:41964/0 (socket says 172.21.15.169:41964) 2023-10-26T01:32:17.579 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:17.576+0000 7ffa6659c640 1 -- 172.21.15.169:0/2244279952 learned_addr learned my addr 172.21.15.169:0/2244279952 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:32:17.581 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:17.576+0000 7ffa5ffff640 1 --2- 172.21.15.169:0/2244279952 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ffa680ab070 0x7ffa68153a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:17.581 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:17.576+0000 7ffa66d9d640 1 --2- 172.21.15.169:0/2244279952 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffa68153fb0 0x7ffa6815a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:17.581 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:17.576+0000 7ffa6659c640 1 -- 172.21.15.169:0/2244279952 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ffa680ab070 msgr2=0x7ffa68153a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:17.581 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:17.576+0000 7ffa6659c640 1 --2- 172.21.15.169:0/2244279952 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ffa680ab070 0x7ffa68153a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:17.581 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:17.576+0000 7ffa6659c640 1 -- 172.21.15.169:0/2244279952 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffa68153fb0 msgr2=0x7ffa6815a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:17.582 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:17.576+0000 7ffa6659c640 1 --2- 172.21.15.169:0/2244279952 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffa68153fb0 0x7ffa6815a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:17.582 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:17.576+0000 7ffa6659c640 1 -- 172.21.15.169:0/2244279952 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa58009560 con 0x7ffa6806cdb0 2023-10-26T01:32:17.582 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:17.576+0000 7ffa66d9d640 1 --2- 172.21.15.169:0/2244279952 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffa68153fb0 0x7ffa6815a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:32:17.582 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:17.576+0000 7ffa6659c640 1 --2- 172.21.15.169:0/2244279952 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa6806cdb0 0x7ffa6814a290 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7ffa58009850 tx=0x7ffa5802da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:17.582 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:17.577+0000 7ffa5effd640 1 -- 172.21.15.169:0/2244279952 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa5802de60 con 0x7ffa6806cdb0 2023-10-26T01:32:17.583 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:17.577+0000 7ffa5effd640 1 -- 172.21.15.169:0/2244279952 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ffa580036e0 con 0x7ffa6806cdb0 2023-10-26T01:32:17.583 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:17.577+0000 7ffa6cb3c640 1 -- 172.21.15.169:0/2244279952 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa681528f0 con 0x7ffa6806cdb0 2023-10-26T01:32:17.583 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:17.577+0000 7ffa5effd640 1 -- 172.21.15.169:0/2244279952 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa58003a90 con 0x7ffa6806cdb0 2023-10-26T01:32:17.583 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:17.577+0000 7ffa6cb3c640 1 -- 172.21.15.169:0/2244279952 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa68152eb0 con 0x7ffa6806cdb0 2023-10-26T01:32:17.583 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:17.578+0000 7ffa6cb3c640 1 -- 172.21.15.169:0/2244279952 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa68066af0 con 0x7ffa6806cdb0 2023-10-26T01:32:17.584 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:17.579+0000 7ffa5effd640 1 -- 172.21.15.169:0/2244279952 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ffa58003bf0 con 0x7ffa6806cdb0 2023-10-26T01:32:17.584 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:17.580+0000 7ffa5effd640 1 --2- 172.21.15.169:0/2244279952 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ffa4806ae40 0x7ffa4806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:17.585 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:17.580+0000 7ffa5effd640 1 -- 172.21.15.169:0/2244279952 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ffa580ac0a0 con 0x7ffa6806cdb0 2023-10-26T01:32:17.585 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:17.581+0000 7ffa5ffff640 1 --2- 172.21.15.169:0/2244279952 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ffa4806ae40 0x7ffa4806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:17.585 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:17.582+0000 7ffa5ffff640 1 --2- 172.21.15.169:0/2244279952 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ffa4806ae40 0x7ffa4806d300 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ffa680721e0 tx=0x7ffa50009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:17.588 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:17.586+0000 7ffa5effd640 1 -- 172.21.15.169:0/2244279952 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ffa580771d0 con 0x7ffa6806cdb0 2023-10-26T01:32:17.776 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:17.773+0000 7ffa6cb3c640 1 -- 172.21.15.169:0/2244279952 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ffa680acdb0 con 0x7ffa6806cdb0 2023-10-26T01:32:17.776 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:17.774+0000 7ffa5effd640 1 -- 172.21.15.169:0/2244279952 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v37) v1 ==== 93+0+10 (secure 0 0 0) 0x7ffa680acdb0 con 0x7ffa6806cdb0 2023-10-26T01:32:17.777 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 1 2023-10-26T01:32:17.779 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:17.776+0000 7ffa6cb3c640 1 -- 172.21.15.169:0/2244279952 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ffa4806ae40 msgr2=0x7ffa4806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:17.780 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:17.776+0000 7ffa6cb3c640 1 --2- 172.21.15.169:0/2244279952 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ffa4806ae40 0x7ffa4806d300 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ffa680721e0 tx=0x7ffa50009040 comp rx=0 tx=0).stop 2023-10-26T01:32:17.780 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:17.776+0000 7ffa6cb3c640 1 -- 172.21.15.169:0/2244279952 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa6806cdb0 msgr2=0x7ffa6814a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:17.780 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:17.776+0000 7ffa6cb3c640 1 --2- 172.21.15.169:0/2244279952 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa6806cdb0 0x7ffa6814a290 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7ffa58009850 tx=0x7ffa5802da60 comp rx=0 tx=0).stop 2023-10-26T01:32:17.780 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:17.776+0000 7ffa6cb3c640 1 -- 172.21.15.169:0/2244279952 shutdown_connections 2023-10-26T01:32:17.780 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:17.776+0000 7ffa6cb3c640 1 --2- 172.21.15.169:0/2244279952 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ffa4806ae40 0x7ffa4806d300 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:17.781 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:17.776+0000 7ffa6cb3c640 1 --2- 172.21.15.169:0/2244279952 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffa68153fb0 0x7ffa6815a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:17.781 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:17.776+0000 7ffa6cb3c640 1 --2- 172.21.15.169:0/2244279952 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ffa680ab070 0x7ffa68153a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:17.781 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:17.776+0000 7ffa6cb3c640 1 --2- 172.21.15.169:0/2244279952 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa6806cdb0 0x7ffa6814a290 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:17.781 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:17.777+0000 7ffa6cb3c640 1 -- 172.21.15.169:0/2244279952 >> 172.21.15.169:0/2244279952 conn(0x7ffa68069930 msgr2=0x7ffa68154980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:17.781 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:17.777+0000 7ffa6cb3c640 1 -- 172.21.15.169:0/2244279952 shutdown_connections 2023-10-26T01:32:17.781 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:17.777+0000 7ffa6cb3c640 1 -- 172.21.15.169:0/2244279952 wait complete. 2023-10-26T01:32:17.797 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-26T01:32:18.020 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.017+0000 7f4de2097640 1 -- 172.21.15.169:0/3545522004 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ddc0b2d90 msgr2=0x7f4ddc06b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:18.020 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.017+0000 7f4de2097640 1 --2- 172.21.15.169:0/3545522004 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ddc0b2d90 0x7f4ddc06b030 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f4dc4005fd0 tx=0x7f4dc402c2d0 comp rx=0 tx=0).stop 2023-10-26T01:32:18.021 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.017+0000 7f4de2097640 1 -- 172.21.15.169:0/3545522004 shutdown_connections 2023-10-26T01:32:18.021 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.017+0000 7f4de2097640 1 --2- 172.21.15.169:0/3545522004 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4ddc06b570 0x7f4ddc15d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:18.021 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.017+0000 7f4de2097640 1 --2- 172.21.15.169:0/3545522004 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ddc0b2d90 0x7f4ddc06b030 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:18.021 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.017+0000 7f4de2097640 1 -- 172.21.15.169:0/3545522004 >> 172.21.15.169:0/3545522004 conn(0x7f4ddc098a60 msgr2=0x7f4ddc0b4a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:18.021 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.017+0000 7f4de2097640 1 -- 172.21.15.169:0/3545522004 shutdown_connections 2023-10-26T01:32:18.022 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.017+0000 7f4de2097640 1 -- 172.21.15.169:0/3545522004 wait complete. 2023-10-26T01:32:18.022 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.019+0000 7f4de2097640 1 Processor -- start 2023-10-26T01:32:18.022 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.019+0000 7f4de2097640 1 -- start start 2023-10-26T01:32:18.024 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.020+0000 7f4de2097640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ddc06b570 0x7f4ddc15b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:18.024 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.020+0000 7f4de2097640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4ddc0b2d90 0x7f4ddc09cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:18.025 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.020+0000 7f4de2097640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ddc156020 0x7f4ddc145750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:18.025 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.020+0000 7f4de2097640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ddc09a500 con 0x7f4ddc156020 2023-10-26T01:32:18.025 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.020+0000 7f4de2097640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ddc145c90 con 0x7f4ddc06b570 2023-10-26T01:32:18.025 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.020+0000 7f4de2097640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ddc15ba20 con 0x7f4ddc0b2d90 2023-10-26T01:32:18.025 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.021+0000 7f4ddbfff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ddc156020 0x7f4ddc145750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:18.026 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.021+0000 7f4ddbfff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ddc156020 0x7f4ddc145750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:41982/0 (socket says 172.21.15.169:41982) 2023-10-26T01:32:18.026 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.021+0000 7f4ddbfff640 1 -- 172.21.15.169:0/3801083171 learned_addr learned my addr 172.21.15.169:0/3801083171 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:32:18.026 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.021+0000 7f4ddb7fe640 1 --2- 172.21.15.169:0/3801083171 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ddc06b570 0x7f4ddc15b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:18.026 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.021+0000 7f4ddaffd640 1 --2- 172.21.15.169:0/3801083171 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4ddc0b2d90 0x7f4ddc09cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:18.026 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.021+0000 7f4ddbfff640 1 -- 172.21.15.169:0/3801083171 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4ddc0b2d90 msgr2=0x7f4ddc09cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:18.027 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.021+0000 7f4ddbfff640 1 --2- 172.21.15.169:0/3801083171 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4ddc0b2d90 0x7f4ddc09cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:18.027 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.021+0000 7f4ddbfff640 1 -- 172.21.15.169:0/3801083171 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ddc06b570 msgr2=0x7f4ddc15b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:18.027 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.021+0000 7f4ddbfff640 1 --2- 172.21.15.169:0/3801083171 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ddc06b570 0x7f4ddc15b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:18.027 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.021+0000 7f4ddbfff640 1 -- 172.21.15.169:0/3801083171 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dc4009900 con 0x7f4ddc156020 2023-10-26T01:32:18.027 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.021+0000 7f4ddaffd640 1 --2- 172.21.15.169:0/3801083171 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4ddc0b2d90 0x7f4ddc09cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:32:18.028 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.021+0000 7f4ddbfff640 1 --2- 172.21.15.169:0/3801083171 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ddc156020 0x7f4ddc145750 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f4dd400b450 tx=0x7f4dd400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:18.028 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.022+0000 7f4dd3fff640 1 -- 172.21.15.169:0/3801083171 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dd4011890 con 0x7f4ddc156020 2023-10-26T01:32:18.028 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.022+0000 7f4dd3fff640 1 -- 172.21.15.169:0/3801083171 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4dd4011ae0 con 0x7f4ddc156020 2023-10-26T01:32:18.028 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.022+0000 7f4dd3fff640 1 -- 172.21.15.169:0/3801083171 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dd40103e0 con 0x7f4ddc156020 2023-10-26T01:32:18.028 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.022+0000 7f4de2097640 1 -- 172.21.15.169:0/3801083171 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ddc09a750 con 0x7f4ddc156020 2023-10-26T01:32:18.029 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.022+0000 7f4de2097640 1 -- 172.21.15.169:0/3801083171 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ddc09ad10 con 0x7f4ddc156020 2023-10-26T01:32:18.030 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.023+0000 7f4de2097640 1 -- 172.21.15.169:0/3801083171 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ddc066420 con 0x7f4ddc156020 2023-10-26T01:32:18.030 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.024+0000 7f4dd3fff640 1 -- 172.21.15.169:0/3801083171 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f4dd4010540 con 0x7f4ddc156020 2023-10-26T01:32:18.030 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.025+0000 7f4dd3fff640 1 --2- 172.21.15.169:0/3801083171 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f4dbc06ae40 0x7f4dbc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:18.030 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.025+0000 7f4dd3fff640 1 -- 172.21.15.169:0/3801083171 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4dd4089d30 con 0x7f4ddc156020 2023-10-26T01:32:18.030 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.026+0000 7f4ddb7fe640 1 --2- 172.21.15.169:0/3801083171 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f4dbc06ae40 0x7f4dbc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:18.031 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.027+0000 7f4ddb7fe640 1 --2- 172.21.15.169:0/3801083171 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f4dbc06ae40 0x7f4dbc06d300 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc000f60 tx=0x7f4dcc009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:18.032 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.030+0000 7f4dd3fff640 1 -- 172.21.15.169:0/3801083171 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4dd4054d40 con 0x7f4ddc156020 2023-10-26T01:32:18.225 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.222+0000 7f4de2097640 1 -- 172.21.15.169:0/3801083171 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4ddc09af00 con 0x7f4ddc156020 2023-10-26T01:32:18.226 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.223+0000 7f4dd3fff640 1 -- 172.21.15.169:0/3801083171 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v37) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4dd4015020 con 0x7f4ddc156020 2023-10-26T01:32:18.226 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 32 2023-10-26T01:32:18.229 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.226+0000 7f4de2097640 1 -- 172.21.15.169:0/3801083171 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f4dbc06ae40 msgr2=0x7f4dbc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:18.229 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.226+0000 7f4de2097640 1 --2- 172.21.15.169:0/3801083171 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f4dbc06ae40 0x7f4dbc06d300 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc000f60 tx=0x7f4dcc009040 comp rx=0 tx=0).stop 2023-10-26T01:32:18.229 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.226+0000 7f4de2097640 1 -- 172.21.15.169:0/3801083171 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ddc156020 msgr2=0x7f4ddc145750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:18.229 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.226+0000 7f4de2097640 1 --2- 172.21.15.169:0/3801083171 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ddc156020 0x7f4ddc145750 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f4dd400b450 tx=0x7f4dd400b910 comp rx=0 tx=0).stop 2023-10-26T01:32:18.230 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.226+0000 7f4de2097640 1 -- 172.21.15.169:0/3801083171 shutdown_connections 2023-10-26T01:32:18.230 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.226+0000 7f4de2097640 1 --2- 172.21.15.169:0/3801083171 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f4dbc06ae40 0x7f4dbc06d300 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:18.230 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.226+0000 7f4de2097640 1 --2- 172.21.15.169:0/3801083171 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ddc156020 0x7f4ddc145750 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:18.230 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.226+0000 7f4de2097640 1 --2- 172.21.15.169:0/3801083171 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4ddc0b2d90 0x7f4ddc09cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:18.231 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.226+0000 7f4de2097640 1 --2- 172.21.15.169:0/3801083171 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ddc06b570 0x7f4ddc15b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:18.231 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.226+0000 7f4de2097640 1 -- 172.21.15.169:0/3801083171 >> 172.21.15.169:0/3801083171 conn(0x7f4ddc098a60 msgr2=0x7f4ddc06e310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:18.231 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.226+0000 7f4de2097640 1 -- 172.21.15.169:0/3801083171 shutdown_connections 2023-10-26T01:32:18.231 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.226+0000 7f4de2097640 1 -- 172.21.15.169:0/3801083171 wait complete. 2023-10-26T01:32:18.248 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-26T01:32:18.477 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.474+0000 7fb2ef765640 1 -- 172.21.15.169:0/2933942636 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb2e80ac320 msgr2=0x7fb2e80ac770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:18.477 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.474+0000 7fb2ef765640 1 --2- 172.21.15.169:0/2933942636 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb2e80ac320 0x7fb2e80ac770 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fb2e809b5a0 tx=0x7fb2e002c250 comp rx=0 tx=0).stop 2023-10-26T01:32:18.478 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.474+0000 7fb2ef765640 1 -- 172.21.15.169:0/2933942636 shutdown_connections 2023-10-26T01:32:18.478 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.474+0000 7fb2ef765640 1 --2- 172.21.15.169:0/2933942636 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb2e80ac320 0x7fb2e80ac770 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:18.478 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.474+0000 7fb2ef765640 1 --2- 172.21.15.169:0/2933942636 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb2e80c9d00 0x7fb2e80cdc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:18.478 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.474+0000 7fb2ef765640 1 --2- 172.21.15.169:0/2933942636 >> v2:172.21.15.170:3300/0 conn(0x7fb2e815b620 0x7fb2e80cd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:18.479 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.474+0000 7fb2ef765640 1 -- 172.21.15.169:0/2933942636 >> 172.21.15.169:0/2933942636 conn(0x7fb2e8069930 msgr2=0x7fb2e8069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:18.479 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.474+0000 7fb2ef765640 1 -- 172.21.15.169:0/2933942636 shutdown_connections 2023-10-26T01:32:18.479 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.474+0000 7fb2ef765640 1 -- 172.21.15.169:0/2933942636 wait complete. 2023-10-26T01:32:18.480 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.476+0000 7fb2ef765640 1 Processor -- start 2023-10-26T01:32:18.481 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.476+0000 7fb2ef765640 1 -- start start 2023-10-26T01:32:18.481 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.477+0000 7fb2ef765640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb2e80ac320 0x7fb2e81f4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:18.481 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.477+0000 7fb2ef765640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb2e80c9d00 0x7fb2e815a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:18.481 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.477+0000 7fb2ef765640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb2e815b620 0x7fb2e815a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:18.481 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.477+0000 7fb2ef765640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e8143550 con 0x7fb2e80c9d00 2023-10-26T01:32:18.482 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.477+0000 7fb2ef765640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e81f5420 con 0x7fb2e80ac320 2023-10-26T01:32:18.482 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.477+0000 7fb2ef765640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e8070f80 con 0x7fb2e815b620 2023-10-26T01:32:18.482 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.478+0000 7fb2ecd00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb2e80c9d00 0x7fb2e815a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:18.482 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.478+0000 7fb2ecd00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb2e80c9d00 0x7fb2e815a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:41992/0 (socket says 172.21.15.169:41992) 2023-10-26T01:32:18.482 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.478+0000 7fb2ecd00640 1 -- 172.21.15.169:0/1094592023 learned_addr learned my addr 172.21.15.169:0/1094592023 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:32:18.483 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.478+0000 7fb2ed501640 1 --2- 172.21.15.169:0/1094592023 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb2e80ac320 0x7fb2e81f4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:18.483 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.478+0000 7fb2ecd00640 1 -- 172.21.15.169:0/1094592023 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb2e815b620 msgr2=0x7fb2e815a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:18.483 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.478+0000 7fb2ecd00640 1 --2- 172.21.15.169:0/1094592023 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb2e815b620 0x7fb2e815a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:18.483 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.478+0000 7fb2ecd00640 1 -- 172.21.15.169:0/1094592023 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb2e80ac320 msgr2=0x7fb2e81f4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:18.484 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.478+0000 7fb2ecd00640 1 --2- 172.21.15.169:0/1094592023 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb2e80ac320 0x7fb2e81f4ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:18.484 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.478+0000 7fb2ecd00640 1 -- 172.21.15.169:0/1094592023 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2e0009900 con 0x7fb2e80c9d00 2023-10-26T01:32:18.485 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.478+0000 7fb2ecd00640 1 --2- 172.21.15.169:0/1094592023 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb2e80c9d00 0x7fb2e815a0d0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fb2d800d770 tx=0x7fb2d800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:18.485 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.479+0000 7fb2ddffb640 1 -- 172.21.15.169:0/1094592023 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d8009c30 con 0x7fb2e80c9d00 2023-10-26T01:32:18.485 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.479+0000 7fb2ddffb640 1 -- 172.21.15.169:0/1094592023 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb2d8009e80 con 0x7fb2e80c9d00 2023-10-26T01:32:18.486 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.479+0000 7fb2ddffb640 1 -- 172.21.15.169:0/1094592023 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d80106a0 con 0x7fb2e80c9d00 2023-10-26T01:32:18.486 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.479+0000 7fb2ef765640 1 -- 172.21.15.169:0/1094592023 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2e8071260 con 0x7fb2e80c9d00 2023-10-26T01:32:18.486 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.479+0000 7fb2ef765640 1 -- 172.21.15.169:0/1094592023 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2e8071820 con 0x7fb2e80c9d00 2023-10-26T01:32:18.486 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.480+0000 7fb2ef765640 1 -- 172.21.15.169:0/1094592023 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2e804db10 con 0x7fb2e80c9d00 2023-10-26T01:32:18.486 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.481+0000 7fb2ddffb640 1 -- 172.21.15.169:0/1094592023 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fb2d8010800 con 0x7fb2e80c9d00 2023-10-26T01:32:18.487 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.482+0000 7fb2ddffb640 1 --2- 172.21.15.169:0/1094592023 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb2cc06ae40 0x7fb2cc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:18.487 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.482+0000 7fb2ddffb640 1 -- 172.21.15.169:0/1094592023 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb2d8089b30 con 0x7fb2e80c9d00 2023-10-26T01:32:18.487 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.483+0000 7fb2ed501640 1 --2- 172.21.15.169:0/1094592023 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb2cc06ae40 0x7fb2cc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:18.487 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.484+0000 7fb2ed501640 1 --2- 172.21.15.169:0/1094592023 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb2cc06ae40 0x7fb2cc06d300 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb2e4005fd0 tx=0x7fb2e4005850 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:18.488 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.486+0000 7fb2ddffb640 1 -- 172.21.15.169:0/1094592023 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb2d8058660 con 0x7fb2e80c9d00 2023-10-26T01:32:18.677 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.674+0000 7fb2ef765640 1 -- 172.21.15.169:0/1094592023 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb2e80b2410 con 0x7fb2e80c9d00 2023-10-26T01:32:18.678 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.675+0000 7fb2ddffb640 1 -- 172.21.15.169:0/1094592023 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v37) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb2d80210a0 con 0x7fb2e80c9d00 2023-10-26T01:32:18.678 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 64 2023-10-26T01:32:18.681 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.678+0000 7fb2ef765640 1 -- 172.21.15.169:0/1094592023 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb2cc06ae40 msgr2=0x7fb2cc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:18.681 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.678+0000 7fb2ef765640 1 --2- 172.21.15.169:0/1094592023 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb2cc06ae40 0x7fb2cc06d300 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb2e4005fd0 tx=0x7fb2e4005850 comp rx=0 tx=0).stop 2023-10-26T01:32:18.681 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.678+0000 7fb2ef765640 1 -- 172.21.15.169:0/1094592023 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb2e80c9d00 msgr2=0x7fb2e815a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:18.682 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.678+0000 7fb2ef765640 1 --2- 172.21.15.169:0/1094592023 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb2e80c9d00 0x7fb2e815a0d0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fb2d800d770 tx=0x7fb2d800dc30 comp rx=0 tx=0).stop 2023-10-26T01:32:18.682 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.678+0000 7fb2ef765640 1 -- 172.21.15.169:0/1094592023 shutdown_connections 2023-10-26T01:32:18.682 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.678+0000 7fb2ef765640 1 --2- 172.21.15.169:0/1094592023 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb2cc06ae40 0x7fb2cc06d300 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:18.682 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.678+0000 7fb2ef765640 1 --2- 172.21.15.169:0/1094592023 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb2e80c9d00 0x7fb2e815a0d0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:18.682 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.678+0000 7fb2ef765640 1 --2- 172.21.15.169:0/1094592023 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb2e815b620 0x7fb2e815a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:18.683 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.678+0000 7fb2ef765640 1 --2- 172.21.15.169:0/1094592023 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb2e80ac320 0x7fb2e81f4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:18.683 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.678+0000 7fb2ef765640 1 -- 172.21.15.169:0/1094592023 >> 172.21.15.169:0/1094592023 conn(0x7fb2e8069930 msgr2=0x7fb2e8151550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:18.683 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.678+0000 7fb2ef765640 1 -- 172.21.15.169:0/1094592023 shutdown_connections 2023-10-26T01:32:18.683 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.678+0000 7fb2ef765640 1 -- 172.21.15.169:0/1094592023 wait complete. 2023-10-26T01:32:18.699 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-26T01:32:18.904 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.901+0000 7fa34bfff640 1 -- 172.21.15.169:0/2859291446 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa34c065490 msgr2=0x7fa34c0cdd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:18.904 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.901+0000 7fa34bfff640 1 --2- 172.21.15.169:0/2859291446 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa34c065490 0x7fa34c0cdd90 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fa34c06a970 tx=0x7fa33402c070 comp rx=0 tx=0).stop 2023-10-26T01:32:18.904 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.901+0000 7fa34bfff640 1 -- 172.21.15.169:0/2859291446 shutdown_connections 2023-10-26T01:32:18.905 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.901+0000 7fa34bfff640 1 --2- 172.21.15.169:0/2859291446 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa34c065490 0x7fa34c0cdd90 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:18.905 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.901+0000 7fa34bfff640 1 --2- 172.21.15.169:0/2859291446 >> v2:172.21.15.170:3300/0 conn(0x7fa34c15b500 0x7fa34c0cc5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:18.905 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.901+0000 7fa34bfff640 1 -- 172.21.15.169:0/2859291446 >> 172.21.15.169:0/2859291446 conn(0x7fa34c0b2f30 msgr2=0x7fa34c0b3340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:18.905 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.901+0000 7fa34bfff640 1 -- 172.21.15.169:0/2859291446 shutdown_connections 2023-10-26T01:32:18.905 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.901+0000 7fa34bfff640 1 -- 172.21.15.169:0/2859291446 wait complete. 2023-10-26T01:32:18.908 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.902+0000 7fa34bfff640 1 Processor -- start 2023-10-26T01:32:18.908 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.903+0000 7fa34bfff640 1 -- start start 2023-10-26T01:32:18.908 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.904+0000 7fa34bfff640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa34c065490 0x7fa34c0c4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:18.909 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.904+0000 7fa34bfff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa34c0ce2d0 0x7fa34c151660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:18.909 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.904+0000 7fa34bfff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa34c15b500 0x7fa34c0d12c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:18.909 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.904+0000 7fa34bfff640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa34c15c740 con 0x7fa34c0ce2d0 2023-10-26T01:32:18.909 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.904+0000 7fa34bfff640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa34c15bea0 con 0x7fa34c15b500 2023-10-26T01:32:18.910 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.904+0000 7fa34bfff640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa34c15c180 con 0x7fa34c065490 2023-10-26T01:32:18.910 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.904+0000 7fa34a7fc640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa34c0ce2d0 0x7fa34c151660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:18.910 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.904+0000 7fa34a7fc640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa34c0ce2d0 0x7fa34c151660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:42016/0 (socket says 172.21.15.169:42016) 2023-10-26T01:32:18.910 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.904+0000 7fa34a7fc640 1 -- 172.21.15.169:0/80566036 learned_addr learned my addr 172.21.15.169:0/80566036 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:32:18.911 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.904+0000 7fa34affd640 1 --2- 172.21.15.169:0/80566036 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa34c065490 0x7fa34c0c4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:18.911 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.904+0000 7fa34a7fc640 1 -- 172.21.15.169:0/80566036 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa34c065490 msgr2=0x7fa34c0c4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:18.911 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.904+0000 7fa34a7fc640 1 --2- 172.21.15.169:0/80566036 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa34c065490 0x7fa34c0c4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:18.911 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.904+0000 7fa34a7fc640 1 -- 172.21.15.169:0/80566036 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa34c15b500 msgr2=0x7fa34c0d12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:18.911 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.904+0000 7fa34a7fc640 1 --2- 172.21.15.169:0/80566036 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa34c15b500 0x7fa34c0d12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:18.912 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.904+0000 7fa34a7fc640 1 -- 172.21.15.169:0/80566036 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa334009840 con 0x7fa34c0ce2d0 2023-10-26T01:32:18.912 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.905+0000 7fa34a7fc640 1 --2- 172.21.15.169:0/80566036 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa34c0ce2d0 0x7fa34c151660 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fa334000f80 tx=0x7fa334004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:18.912 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.905+0000 7fa343fff640 1 -- 172.21.15.169:0/80566036 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa33403e460 con 0x7fa34c0ce2d0 2023-10-26T01:32:18.912 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.905+0000 7fa343fff640 1 -- 172.21.15.169:0/80566036 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa33403e6b0 con 0x7fa34c0ce2d0 2023-10-26T01:32:18.913 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.905+0000 7fa343fff640 1 -- 172.21.15.169:0/80566036 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa33403ea80 con 0x7fa34c0ce2d0 2023-10-26T01:32:18.913 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.905+0000 7fa34bfff640 1 -- 172.21.15.169:0/80566036 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa34c15c460 con 0x7fa34c0ce2d0 2023-10-26T01:32:18.913 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.905+0000 7fa34bfff640 1 -- 172.21.15.169:0/80566036 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa34c15bac0 con 0x7fa34c0ce2d0 2023-10-26T01:32:18.914 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.906+0000 7fa34bfff640 1 -- 172.21.15.169:0/80566036 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa34c152200 con 0x7fa34c0ce2d0 2023-10-26T01:32:18.914 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.907+0000 7fa343fff640 1 -- 172.21.15.169:0/80566036 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fa33403ebe0 con 0x7fa34c0ce2d0 2023-10-26T01:32:18.915 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.908+0000 7fa343fff640 1 --2- 172.21.15.169:0/80566036 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa32c06ae40 0x7fa32c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:18.915 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.908+0000 7fa343fff640 1 -- 172.21.15.169:0/80566036 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa3340ac720 con 0x7fa34c0ce2d0 2023-10-26T01:32:18.915 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.909+0000 7fa34affd640 1 --2- 172.21.15.169:0/80566036 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa32c06ae40 0x7fa32c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:18.915 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.910+0000 7fa34affd640 1 --2- 172.21.15.169:0/80566036 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa32c06ae40 0x7fa32c06d300 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa33c005fd0 tx=0x7fa33c005850 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:18.916 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:18.913+0000 7fa343fff640 1 -- 172.21.15.169:0/80566036 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa34c152200 con 0x7fa34c0ce2d0 2023-10-26T01:32:19.106 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:19.104+0000 7fa34bfff640 1 -- 172.21.15.169:0/80566036 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fa34c152200 con 0x7fa34c0ce2d0 2023-10-26T01:32:19.107 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:19.104+0000 7fa343fff640 1 -- 172.21.15.169:0/80566036 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v37) v1 ==== 80+0+0 (secure 0 0 0) 0x7fa34c152200 con 0x7fa34c0ce2d0 2023-10-26T01:32:19.107 INFO:teuthology.orchestra.run.smithi169.stderr:listed 0 entries 2023-10-26T01:32:19.110 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:19.107+0000 7fa34bfff640 1 -- 172.21.15.169:0/80566036 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa32c06ae40 msgr2=0x7fa32c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:19.110 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:19.107+0000 7fa34bfff640 1 --2- 172.21.15.169:0/80566036 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa32c06ae40 0x7fa32c06d300 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa33c005fd0 tx=0x7fa33c005850 comp rx=0 tx=0).stop 2023-10-26T01:32:19.110 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:19.107+0000 7fa34bfff640 1 -- 172.21.15.169:0/80566036 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa34c0ce2d0 msgr2=0x7fa34c151660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:19.111 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:19.107+0000 7fa34bfff640 1 --2- 172.21.15.169:0/80566036 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa34c0ce2d0 0x7fa34c151660 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fa334000f80 tx=0x7fa334004360 comp rx=0 tx=0).stop 2023-10-26T01:32:19.111 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:19.107+0000 7fa34bfff640 1 -- 172.21.15.169:0/80566036 shutdown_connections 2023-10-26T01:32:19.111 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:19.107+0000 7fa34bfff640 1 --2- 172.21.15.169:0/80566036 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa32c06ae40 0x7fa32c06d300 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:19.111 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:19.107+0000 7fa34bfff640 1 --2- 172.21.15.169:0/80566036 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa34c0ce2d0 0x7fa34c151660 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:19.112 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:19.107+0000 7fa34bfff640 1 --2- 172.21.15.169:0/80566036 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa34c15b500 0x7fa34c0d12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:19.112 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:19.107+0000 7fa34bfff640 1 --2- 172.21.15.169:0/80566036 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa34c065490 0x7fa34c0c4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:19.112 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:19.107+0000 7fa34bfff640 1 -- 172.21.15.169:0/80566036 >> 172.21.15.169:0/80566036 conn(0x7fa34c0b2f30 msgr2=0x7fa34c159be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:19.112 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:19.107+0000 7fa34bfff640 1 -- 172.21.15.169:0/80566036 shutdown_connections 2023-10-26T01:32:19.113 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:19.107+0000 7fa34bfff640 1 -- 172.21.15.169:0/80566036 wait complete. 2023-10-26T01:32:19.128 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-26T01:32:19.128 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:32:19.129 DEBUG:teuthology.orchestra.run.smithi170:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-26T01:32:19.263 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi170.front.sepia.ceph.com 2023-10-26T01:32:19.264 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:32:19.264 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-26T01:32:19.310 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:32:19.310 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:32:19.370 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054301,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-26T01:32:19.371 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-26T01:32:19.371 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:32:19.427 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054303,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-26T01:32:19.427 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T01:32:19.652 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:19.649+0000 7fd6ec3fc640 1 -- 172.21.15.169:0/3974648765 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6d0004d00 msgr2=0x7fd6d0005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:19.653 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:19.649+0000 7fd6ec3fc640 1 --2- 172.21.15.169:0/3974648765 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6d0004d00 0x7fd6d0005180 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc009e20 tx=0x7fd6d00057f0 comp rx=0 tx=0).stop 2023-10-26T01:32:19.653 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:19.649+0000 7fd6ea999640 1 -- 172.21.15.169:0/3974648765 reap_dead start 2023-10-26T01:32:19.653 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:19.650+0000 7fd6ec3fc640 1 -- 172.21.15.169:0/3974648765 shutdown_connections 2023-10-26T01:32:19.653 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:19.650+0000 7fd6ec3fc640 1 -- 172.21.15.169:0/3974648765 >> 172.21.15.169:0/3974648765 conn(0x7fd6e4069930 msgr2=0x7fd6e4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:19.654 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:19.650+0000 7fd6ec3fc640 1 -- 172.21.15.169:0/3974648765 shutdown_connections 2023-10-26T01:32:19.654 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:19.650+0000 7fd6ec3fc640 1 -- 172.21.15.169:0/3974648765 wait complete. 2023-10-26T01:32:19.654 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:19.651+0000 7fd6ec3fc640 1 Processor -- start 2023-10-26T01:32:19.655 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:19.652+0000 7fd6ec3fc640 1 -- start start 2023-10-26T01:32:19.656 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:19.652+0000 7fd6ec3fc640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6d0004d00 0x7fd6e414c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:19.657 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:19.652+0000 7fd6ec3fc640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd6e40c6560 0x7fd6e40b0e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:19.657 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:19.652+0000 7fd6ec3fc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd6e41e3a50 0x7fd6e41e3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:19.657 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:19.653+0000 7fd6ec3fc640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e41e4410 con 0x7fd6d0004d00 2023-10-26T01:32:19.657 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:19.653+0000 7fd6ec3fc640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e414cd00 con 0x7fd6e41e3a50 2023-10-26T01:32:19.658 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:19.653+0000 7fd6ec3fc640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e40c6ab0 con 0x7fd6e40c6560 2023-10-26T01:32:19.658 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:19.653+0000 7fd6ea198640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6d0004d00 0x7fd6e414c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:19.658 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:19.653+0000 7fd6ea198640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6d0004d00 0x7fd6e414c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:42030/0 (socket says 172.21.15.169:42030) 2023-10-26T01:32:19.658 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:19.653+0000 7fd6ea198640 1 -- 172.21.15.169:0/3056083854 learned_addr learned my addr 172.21.15.169:0/3056083854 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:32:19.658 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:19.653+0000 7fd6e9997640 1 --2- 172.21.15.169:0/3056083854 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd6e40c6560 0x7fd6e40b0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:19.659 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:19.653+0000 7fd6ea198640 1 -- 172.21.15.169:0/3056083854 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd6e40c6560 msgr2=0x7fd6e40b0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:19.659 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:19.653+0000 7fd6ea198640 1 --2- 172.21.15.169:0/3056083854 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd6e40c6560 0x7fd6e40b0e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:19.659 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:19.653+0000 7fd6ea198640 1 -- 172.21.15.169:0/3056083854 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd6e41e3a50 msgr2=0x7fd6e41e3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:19.659 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:19.653+0000 7fd6ea999640 1 --2- 172.21.15.169:0/3056083854 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd6e41e3a50 0x7fd6e41e3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:19.659 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:19.653+0000 7fd6ea198640 1 --2- 172.21.15.169:0/3056083854 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd6e41e3a50 0x7fd6e41e3ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:19.660 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:19.653+0000 7fd6ea198640 1 -- 172.21.15.169:0/3056083854 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6e40c6d90 con 0x7fd6d0004d00 2023-10-26T01:32:19.660 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:19.653+0000 7fd6ea198640 1 --2- 172.21.15.169:0/3056083854 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6d0004d00 0x7fd6e414c7c0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fd6e000e990 tx=0x7fd6e000ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:19.660 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:19.654+0000 7fd6daffd640 1 -- 172.21.15.169:0/3056083854 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e000cbe0 con 0x7fd6d0004d00 2023-10-26T01:32:19.660 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:19.654+0000 7fd6daffd640 1 -- 172.21.15.169:0/3056083854 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd6e000ce30 con 0x7fd6d0004d00 2023-10-26T01:32:19.660 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:19.654+0000 7fd6daffd640 1 -- 172.21.15.169:0/3056083854 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e0005280 con 0x7fd6d0004d00 2023-10-26T01:32:19.661 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:19.654+0000 7fd6ec3fc640 1 -- 172.21.15.169:0/3056083854 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6e40c7070 con 0x7fd6d0004d00 2023-10-26T01:32:19.661 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:19.654+0000 7fd6ec3fc640 1 -- 172.21.15.169:0/3056083854 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6e40b13a0 con 0x7fd6d0004d00 2023-10-26T01:32:19.662 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:19.655+0000 7fd6ec3fc640 1 -- 172.21.15.169:0/3056083854 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6e404db10 con 0x7fd6d0004d00 2023-10-26T01:32:19.662 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:19.656+0000 7fd6daffd640 1 -- 172.21.15.169:0/3056083854 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fd6e00053e0 con 0x7fd6d0004d00 2023-10-26T01:32:19.662 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:19.657+0000 7fd6daffd640 1 --2- 172.21.15.169:0/3056083854 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd6c806ae40 0x7fd6c806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:19.662 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:19.657+0000 7fd6daffd640 1 -- 172.21.15.169:0/3056083854 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd6e0089ff0 con 0x7fd6d0004d00 2023-10-26T01:32:19.663 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:19.658+0000 7fd6e9997640 1 --2- 172.21.15.169:0/3056083854 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd6c806ae40 0x7fd6c806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:19.663 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:19.659+0000 7fd6e9997640 1 --2- 172.21.15.169:0/3056083854 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd6c806ae40 0x7fd6c806d300 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4001820 tx=0x7fd6d4001170 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:19.664 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:19.661+0000 7fd6daffd640 1 -- 172.21.15.169:0/3056083854 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd6e0055000 con 0x7fd6d0004d00 2023-10-26T01:32:19.871 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:19.869+0000 7fd6ec3fc640 1 -- 172.21.15.169:0/3056083854 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd6e40adea0 con 0x7fd6d0004d00 2023-10-26T01:32:19.873 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:19.870+0000 7fd6daffd640 1 -- 172.21.15.169:0/3056083854 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4913 (secure 0 0 0) 0x7fd6e001f020 con 0x7fd6d0004d00 2023-10-26T01:32:19.873 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:32:19.873 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4640,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/61096113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":61096113},{"type":"v1","addr":"172.21.15.170:6836","nonce":61096113}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4762,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2825046244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2825046244},{"type":"v1","addr":"172.21.15.169:6833","nonce":2825046244}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4777,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/258866976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":258866976},{"type":"v1","addr":"172.21.15.170:6837","nonce":258866976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T01:31:57.759251+0000","modified":"2023-10-26T01:32:00.831213+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5196},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5196":{"gid":5196,"name":"c","rank":0,"incarnation":11,"state":"up:active","state_seq":4,"addr":"172.21.15.169:6835/871978528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":871978528},{"type":"v1","addr":"172.21.15.169:6835","nonce":871978528}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-26T01:32:19.874 INFO:teuthology.orchestra.run.smithi169.stderr:dumped fsmap epoch 15 2023-10-26T01:32:19.876 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:19.872+0000 7fd6ec3fc640 1 -- 172.21.15.169:0/3056083854 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd6c806ae40 msgr2=0x7fd6c806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:19.876 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:19.872+0000 7fd6ec3fc640 1 --2- 172.21.15.169:0/3056083854 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd6c806ae40 0x7fd6c806d300 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4001820 tx=0x7fd6d4001170 comp rx=0 tx=0).stop 2023-10-26T01:32:19.876 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:19.872+0000 7fd6ec3fc640 1 -- 172.21.15.169:0/3056083854 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6d0004d00 msgr2=0x7fd6e414c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:19.876 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:19.872+0000 7fd6ec3fc640 1 --2- 172.21.15.169:0/3056083854 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6d0004d00 0x7fd6e414c7c0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fd6e000e990 tx=0x7fd6e000ee50 comp rx=0 tx=0).stop 2023-10-26T01:32:19.877 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:19.873+0000 7fd6ec3fc640 1 -- 172.21.15.169:0/3056083854 shutdown_connections 2023-10-26T01:32:19.877 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:19.873+0000 7fd6ec3fc640 1 --2- 172.21.15.169:0/3056083854 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd6c806ae40 0x7fd6c806d300 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:19.877 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:19.873+0000 7fd6ec3fc640 1 --2- 172.21.15.169:0/3056083854 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd6e41e3a50 0x7fd6e41e3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:19.877 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:19.873+0000 7fd6ec3fc640 1 --2- 172.21.15.169:0/3056083854 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd6e40c6560 0x7fd6e40b0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:19.877 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:19.873+0000 7fd6ec3fc640 1 --2- 172.21.15.169:0/3056083854 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6d0004d00 0x7fd6e414c7c0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:19.878 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:19.873+0000 7fd6ec3fc640 1 -- 172.21.15.169:0/3056083854 >> 172.21.15.169:0/3056083854 conn(0x7fd6e4069930 msgr2=0x7fd6e414cf00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:19.878 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:19.873+0000 7fd6ec3fc640 1 -- 172.21.15.169:0/3056083854 shutdown_connections 2023-10-26T01:32:19.878 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:19.873+0000 7fd6ec3fc640 1 -- 172.21.15.169:0/3056083854 wait complete. 2023-10-26T01:32:19.895 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:32:20.114 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.110+0000 7f865d832640 1 -- 172.21.15.169:0/3383460046 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f865806ce20 msgr2=0x7f865814e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:20.115 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.110+0000 7f865d832640 1 --2- 172.21.15.169:0/3383460046 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f865806ce20 0x7f865814e790 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f865806eaf0 tx=0x7f864802c1d0 comp rx=0 tx=0).stop 2023-10-26T01:32:20.115 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.111+0000 7f865d832640 1 -- 172.21.15.169:0/3383460046 shutdown_connections 2023-10-26T01:32:20.116 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.111+0000 7f865d832640 1 --2- 172.21.15.169:0/3383460046 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f865806ad30 0x7f865806b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:20.116 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.111+0000 7f865d832640 1 --2- 172.21.15.169:0/3383460046 >> v2:172.21.15.170:3300/0 conn(0x7f8658158850 0x7f86580aab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:20.116 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.111+0000 7f865d832640 1 --2- 172.21.15.169:0/3383460046 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f865806ce20 0x7f865814e790 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:20.117 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.111+0000 7f865d832640 1 -- 172.21.15.169:0/3383460046 >> 172.21.15.169:0/3383460046 conn(0x7f8658069930 msgr2=0x7f8658069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:20.117 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.111+0000 7f865d832640 1 -- 172.21.15.169:0/3383460046 shutdown_connections 2023-10-26T01:32:20.118 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.111+0000 7f865d832640 1 -- 172.21.15.169:0/3383460046 wait complete. 2023-10-26T01:32:20.123 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.112+0000 7f865d832640 1 Processor -- start 2023-10-26T01:32:20.123 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.113+0000 7f865d832640 1 -- start start 2023-10-26T01:32:20.124 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.113+0000 7f865d832640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f865806ad30 0x7f86581453b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:20.124 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.113+0000 7f865d832640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f865806ce20 0x7f8658154990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:20.124 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.114+0000 7f865d832640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8658158850 0x7f8658154ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:20.125 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.114+0000 7f865d832640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8658152300 con 0x7f865806ad30 2023-10-26T01:32:20.125 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.114+0000 7f865d832640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8658152e80 con 0x7f8658158850 2023-10-26T01:32:20.125 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.114+0000 7f865d832640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8658152550 con 0x7f865806ce20 2023-10-26T01:32:20.126 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.114+0000 7f8656ffd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f865806ad30 0x7f86581453b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:20.126 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.114+0000 7f8656ffd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f865806ad30 0x7f86581453b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:42042/0 (socket says 172.21.15.169:42042) 2023-10-26T01:32:20.127 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.114+0000 7f8656ffd640 1 -- 172.21.15.169:0/1461666075 learned_addr learned my addr 172.21.15.169:0/1461666075 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:32:20.127 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.114+0000 7f8656ffd640 1 -- 172.21.15.169:0/1461666075 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f865806ce20 msgr2=0x7f8658154990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:20.127 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.114+0000 7f86577fe640 1 --2- 172.21.15.169:0/1461666075 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8658158850 0x7f8658154ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:20.127 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.114+0000 7f86567fc640 1 --2- 172.21.15.169:0/1461666075 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f865806ce20 0x7f8658154990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:20.128 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.114+0000 7f8656ffd640 1 --2- 172.21.15.169:0/1461666075 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f865806ce20 0x7f8658154990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:20.128 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.114+0000 7f8656ffd640 1 -- 172.21.15.169:0/1461666075 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8658158850 msgr2=0x7f8658154ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:20.128 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.114+0000 7f8656ffd640 1 --2- 172.21.15.169:0/1461666075 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8658158850 0x7f8658154ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:20.128 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.114+0000 7f8656ffd640 1 -- 172.21.15.169:0/1461666075 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8648009900 con 0x7f865806ad30 2023-10-26T01:32:20.128 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.115+0000 7f8656ffd640 1 --2- 172.21.15.169:0/1461666075 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f865806ad30 0x7f86581453b0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f8648009d60 tx=0x7f8648035db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:20.128 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.115+0000 7f864ffff640 1 -- 172.21.15.169:0/1461666075 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8648034440 con 0x7f865806ad30 2023-10-26T01:32:20.129 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.115+0000 7f864ffff640 1 -- 172.21.15.169:0/1461666075 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8648004560 con 0x7f865806ad30 2023-10-26T01:32:20.129 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.115+0000 7f865d832640 1 -- 172.21.15.169:0/1461666075 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8658152830 con 0x7f865806ad30 2023-10-26T01:32:20.129 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.115+0000 7f864ffff640 1 -- 172.21.15.169:0/1461666075 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8648040450 con 0x7f865806ad30 2023-10-26T01:32:20.129 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.115+0000 7f865d832640 1 -- 172.21.15.169:0/1461666075 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86581458f0 con 0x7f865806ad30 2023-10-26T01:32:20.129 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.116+0000 7f865d832640 1 -- 172.21.15.169:0/1461666075 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f865804db10 con 0x7f865806ad30 2023-10-26T01:32:20.129 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.117+0000 7f864ffff640 1 -- 172.21.15.169:0/1461666075 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f864800a040 con 0x7f865806ad30 2023-10-26T01:32:20.130 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.118+0000 7f864ffff640 1 --2- 172.21.15.169:0/1461666075 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f863806ae40 0x7f863806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:20.130 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.118+0000 7f864ffff640 1 -- 172.21.15.169:0/1461666075 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f86480ac5c0 con 0x7f865806ad30 2023-10-26T01:32:20.130 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.119+0000 7f86567fc640 1 --2- 172.21.15.169:0/1461666075 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f863806ae40 0x7f863806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:20.130 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.120+0000 7f86567fc640 1 --2- 172.21.15.169:0/1461666075 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f863806ae40 0x7f863806d300 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f865814f840 tx=0x7f8640008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:20.131 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.122+0000 7f864ffff640 1 -- 172.21.15.169:0/1461666075 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8648077670 con 0x7f865806ad30 2023-10-26T01:32:20.318 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.315+0000 7f865d832640 1 -- 172.21.15.169:0/1461666075 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8658158100 con 0x7f865806ad30 2023-10-26T01:32:20.320 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.317+0000 7f864ffff640 1 -- 172.21.15.169:0/1461666075 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+16617 (secure 0 0 0) 0x7f864807ac90 con 0x7f865806ad30 2023-10-26T01:32:20.320 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:32:20.321 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":37,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:32:11.943092+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-10-26T01:31:55.401509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-10-26T01:31:56.776031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:32:20.322 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.320+0000 7f865d832640 1 -- 172.21.15.169:0/1461666075 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f863806ae40 msgr2=0x7f863806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:20.323 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.320+0000 7f865d832640 1 --2- 172.21.15.169:0/1461666075 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f863806ae40 0x7f863806d300 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f865814f840 tx=0x7f8640008040 comp rx=0 tx=0).stop 2023-10-26T01:32:20.323 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.320+0000 7f865d832640 1 -- 172.21.15.169:0/1461666075 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f865806ad30 msgr2=0x7f86581453b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:20.323 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.320+0000 7f865d832640 1 --2- 172.21.15.169:0/1461666075 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f865806ad30 0x7f86581453b0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f8648009d60 tx=0x7f8648035db0 comp rx=0 tx=0).stop 2023-10-26T01:32:20.323 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.320+0000 7f865d832640 1 -- 172.21.15.169:0/1461666075 shutdown_connections 2023-10-26T01:32:20.324 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.320+0000 7f865d832640 1 --2- 172.21.15.169:0/1461666075 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f863806ae40 0x7f863806d300 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:20.324 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.320+0000 7f865d832640 1 --2- 172.21.15.169:0/1461666075 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8658158850 0x7f8658154ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:20.324 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.320+0000 7f865d832640 1 --2- 172.21.15.169:0/1461666075 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f865806ce20 0x7f8658154990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:20.324 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.320+0000 7f865d832640 1 --2- 172.21.15.169:0/1461666075 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f865806ad30 0x7f86581453b0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:20.324 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.320+0000 7f865d832640 1 -- 172.21.15.169:0/1461666075 >> 172.21.15.169:0/1461666075 conn(0x7f8658069930 msgr2=0x7f865814d5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:20.324 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.320+0000 7f865d832640 1 -- 172.21.15.169:0/1461666075 shutdown_connections 2023-10-26T01:32:20.325 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.320+0000 7f865d832640 1 -- 172.21.15.169:0/1461666075 wait complete. 2023-10-26T01:32:20.342 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T01:30:32.200808+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-26T01:31:55.401509+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2023-10-26T01:31:56.776031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T01:32:20.343 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T01:32:20.560 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.557+0000 7fafbb397640 1 -- 172.21.15.169:0/3939706816 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fafb415b6a0 msgr2=0x7fafb40cd690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:20.560 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.557+0000 7fafbb397640 1 --2- 172.21.15.169:0/3939706816 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fafb415b6a0 0x7fafb40cd690 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fafb0005fd0 tx=0x7fafb002bd90 comp rx=0 tx=0).stop 2023-10-26T01:32:20.560 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.557+0000 7fafbb397640 1 -- 172.21.15.169:0/3939706816 shutdown_connections 2023-10-26T01:32:20.561 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.557+0000 7fafbb397640 1 --2- 172.21.15.169:0/3939706816 >> v2:172.21.15.170:3300/0 conn(0x7fafb40acae0 0x7fafb406aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:20.561 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.557+0000 7fafbb397640 1 --2- 172.21.15.169:0/3939706816 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fafb415b6a0 0x7fafb40cd690 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:20.561 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.557+0000 7fafbb397640 1 -- 172.21.15.169:0/3939706816 >> 172.21.15.169:0/3939706816 conn(0x7fafb4069930 msgr2=0x7fafb4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:20.561 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.557+0000 7fafbb397640 1 -- 172.21.15.169:0/3939706816 shutdown_connections 2023-10-26T01:32:20.561 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.557+0000 7fafbb397640 1 -- 172.21.15.169:0/3939706816 wait complete. 2023-10-26T01:32:20.562 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.559+0000 7fafbb397640 1 Processor -- start 2023-10-26T01:32:20.562 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.559+0000 7fafbb397640 1 -- start start 2023-10-26T01:32:20.563 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.561+0000 7fafbb397640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fafb40acae0 0x7fafb41f1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:20.564 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.561+0000 7fafbb397640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fafb40cdbd0 0x7fafb40af0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:20.564 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.561+0000 7fafbb397640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fafb415b6a0 0x7fafb40af5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:20.564 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.561+0000 7fafbb397640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafb41f1f60 con 0x7fafb40cdbd0 2023-10-26T01:32:20.564 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.561+0000 7fafbb397640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafb415e530 con 0x7fafb40acae0 2023-10-26T01:32:20.565 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.561+0000 7fafbb397640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafb41522e0 con 0x7fafb415b6a0 2023-10-26T01:32:20.565 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.561+0000 7fafb8932640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fafb40cdbd0 0x7fafb40af0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:20.565 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.561+0000 7fafb8932640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fafb40cdbd0 0x7fafb40af0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:42054/0 (socket says 172.21.15.169:42054) 2023-10-26T01:32:20.565 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.561+0000 7fafb8932640 1 -- 172.21.15.169:0/2536192007 learned_addr learned my addr 172.21.15.169:0/2536192007 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:32:20.565 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.561+0000 7fafb9934640 1 --2- 172.21.15.169:0/2536192007 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fafb415b6a0 0x7fafb40af5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:20.566 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.561+0000 7fafb9133640 1 --2- 172.21.15.169:0/2536192007 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fafb40acae0 0x7fafb41f1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:20.567 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.561+0000 7fafb8932640 1 -- 172.21.15.169:0/2536192007 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fafb415b6a0 msgr2=0x7fafb40af5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:20.567 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.561+0000 7fafb8932640 1 --2- 172.21.15.169:0/2536192007 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fafb415b6a0 0x7fafb40af5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:20.567 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.561+0000 7fafb8932640 1 -- 172.21.15.169:0/2536192007 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fafb40acae0 msgr2=0x7fafb41f1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:20.567 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.561+0000 7fafb8932640 1 --2- 172.21.15.169:0/2536192007 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fafb40acae0 0x7fafb41f1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:20.568 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.562+0000 7fafb8932640 1 -- 172.21.15.169:0/2536192007 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafb0009560 con 0x7fafb40cdbd0 2023-10-26T01:32:20.568 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.562+0000 7fafb8932640 1 --2- 172.21.15.169:0/2536192007 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fafb40cdbd0 0x7fafb40af0b0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fafa800c400 tx=0x7fafa800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:20.568 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.562+0000 7fafa5ffb640 1 -- 172.21.15.169:0/2536192007 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafa80052d0 con 0x7fafb40cdbd0 2023-10-26T01:32:20.568 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.562+0000 7fafa5ffb640 1 -- 172.21.15.169:0/2536192007 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fafa8007020 con 0x7fafb40cdbd0 2023-10-26T01:32:20.569 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.562+0000 7fafbb397640 1 -- 172.21.15.169:0/2536192007 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafb41525c0 con 0x7fafb40cdbd0 2023-10-26T01:32:20.569 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.562+0000 7fafa5ffb640 1 -- 172.21.15.169:0/2536192007 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafa8007470 con 0x7fafb40cdbd0 2023-10-26T01:32:20.569 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.562+0000 7fafbb397640 1 -- 172.21.15.169:0/2536192007 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafb4152b80 con 0x7fafb40cdbd0 2023-10-26T01:32:20.569 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.564+0000 7fafbb397640 1 -- 172.21.15.169:0/2536192007 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafb4066af0 con 0x7fafb40cdbd0 2023-10-26T01:32:20.569 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.564+0000 7fafa5ffb640 1 -- 172.21.15.169:0/2536192007 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fafa80067d0 con 0x7fafb40cdbd0 2023-10-26T01:32:20.570 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.565+0000 7fafa5ffb640 1 --2- 172.21.15.169:0/2536192007 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7faf9806ae40 0x7faf9806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:20.570 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.566+0000 7fafa5ffb640 1 -- 172.21.15.169:0/2536192007 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fafa8088dc0 con 0x7fafb40cdbd0 2023-10-26T01:32:20.571 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.566+0000 7fafb9133640 1 --2- 172.21.15.169:0/2536192007 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7faf9806ae40 0x7faf9806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:20.571 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.567+0000 7fafb9133640 1 --2- 172.21.15.169:0/2536192007 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7faf9806ae40 0x7faf9806d300 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fafb0035000 tx=0x7fafb0004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:20.573 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.570+0000 7fafa5ffb640 1 -- 172.21.15.169:0/2536192007 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fafa8053d50 con 0x7fafb40cdbd0 2023-10-26T01:32:20.753 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.751+0000 7fafbb397640 1 -- 172.21.15.169:0/2536192007 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fafb404da70 con 0x7fafb40cdbd0 2023-10-26T01:32:20.754 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.752+0000 7fafa5ffb640 1 -- 172.21.15.169:0/2536192007 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v37) v1 ==== 93+0+10 (secure 0 0 0) 0x7fafa8057370 con 0x7fafb40cdbd0 2023-10-26T01:32:20.755 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 1 2023-10-26T01:32:20.757 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.754+0000 7fafbb397640 1 -- 172.21.15.169:0/2536192007 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7faf9806ae40 msgr2=0x7faf9806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:20.757 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.754+0000 7fafbb397640 1 --2- 172.21.15.169:0/2536192007 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7faf9806ae40 0x7faf9806d300 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fafb0035000 tx=0x7fafb0004730 comp rx=0 tx=0).stop 2023-10-26T01:32:20.757 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.754+0000 7fafbb397640 1 -- 172.21.15.169:0/2536192007 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fafb40cdbd0 msgr2=0x7fafb40af0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:20.757 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.754+0000 7fafbb397640 1 --2- 172.21.15.169:0/2536192007 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fafb40cdbd0 0x7fafb40af0b0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fafa800c400 tx=0x7fafa800c8c0 comp rx=0 tx=0).stop 2023-10-26T01:32:20.758 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.754+0000 7fafbb397640 1 -- 172.21.15.169:0/2536192007 shutdown_connections 2023-10-26T01:32:20.758 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.754+0000 7fafbb397640 1 --2- 172.21.15.169:0/2536192007 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7faf9806ae40 0x7faf9806d300 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:20.758 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.754+0000 7fafbb397640 1 --2- 172.21.15.169:0/2536192007 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fafb40cdbd0 0x7fafb40af0b0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:20.758 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.754+0000 7fafbb397640 1 --2- 172.21.15.169:0/2536192007 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fafb415b6a0 0x7fafb40af5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:20.759 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.754+0000 7fafbb397640 1 --2- 172.21.15.169:0/2536192007 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fafb40acae0 0x7fafb41f1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:20.759 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.754+0000 7fafbb397640 1 -- 172.21.15.169:0/2536192007 >> 172.21.15.169:0/2536192007 conn(0x7fafb4069930 msgr2=0x7fafb40acf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:20.759 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.754+0000 7fafbb397640 1 -- 172.21.15.169:0/2536192007 shutdown_connections 2023-10-26T01:32:20.759 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.754+0000 7fafbb397640 1 -- 172.21.15.169:0/2536192007 wait complete. 2023-10-26T01:32:20.775 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-26T01:32:20.997 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.994+0000 7f4648f4c640 1 -- 172.21.15.169:0/1502298133 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f46441593e0 msgr2=0x7f46440ac200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:20.997 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.994+0000 7f4648f4c640 1 --2- 172.21.15.169:0/1502298133 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f46441593e0 0x7f46440ac200 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4634005fd0 tx=0x7f463402be20 comp rx=0 tx=0).stop 2023-10-26T01:32:20.997 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.994+0000 7f4648f4c640 1 -- 172.21.15.169:0/1502298133 shutdown_connections 2023-10-26T01:32:20.998 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.994+0000 7f4648f4c640 1 --2- 172.21.15.169:0/1502298133 >> v2:172.21.15.170:3300/0 conn(0x7f46441597c0 0x7f464415dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:20.998 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.994+0000 7f4648f4c640 1 --2- 172.21.15.169:0/1502298133 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f46441593e0 0x7f46440ac200 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:20.998 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.994+0000 7f4648f4c640 1 -- 172.21.15.169:0/1502298133 >> 172.21.15.169:0/1502298133 conn(0x7f4644069930 msgr2=0x7f4644069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:20.998 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.994+0000 7f4648f4c640 1 -- 172.21.15.169:0/1502298133 shutdown_connections 2023-10-26T01:32:20.998 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.994+0000 7f4648f4c640 1 -- 172.21.15.169:0/1502298133 wait complete. 2023-10-26T01:32:21.000 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.996+0000 7f4648f4c640 1 Processor -- start 2023-10-26T01:32:21.000 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.996+0000 7f4648f4c640 1 -- start start 2023-10-26T01:32:21.000 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.997+0000 7f4648f4c640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f46440b0c80 0x7f46440ad170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:21.000 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.997+0000 7f4648f4c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f46441593e0 0x7f46441468b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:21.001 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.997+0000 7f4648f4c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f46441597c0 0x7f46441558a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:21.001 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.997+0000 7f4648f4c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f464414e9d0 con 0x7f46441597c0 2023-10-26T01:32:21.001 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.997+0000 7f4648f4c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f464414ec70 con 0x7f46441593e0 2023-10-26T01:32:21.001 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.997+0000 7f4648f4c640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f464414ef50 con 0x7f46440b0c80 2023-10-26T01:32:21.001 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.997+0000 7f4642d9d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f46441597c0 0x7f46441558a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:21.001 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.997+0000 7f4642d9d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f46441597c0 0x7f46441558a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:42058/0 (socket says 172.21.15.169:42058) 2023-10-26T01:32:21.002 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.997+0000 7f4642d9d640 1 -- 172.21.15.169:0/38684728 learned_addr learned my addr 172.21.15.169:0/38684728 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:32:21.003 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.998+0000 7f463bfff640 1 --2- 172.21.15.169:0/38684728 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f46441593e0 0x7f46441468b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:21.003 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.998+0000 7f464259c640 1 --2- 172.21.15.169:0/38684728 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f46440b0c80 0x7f46440ad170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:21.003 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.998+0000 7f4642d9d640 1 -- 172.21.15.169:0/38684728 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f46440b0c80 msgr2=0x7f46440ad170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:21.004 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.998+0000 7f4642d9d640 1 --2- 172.21.15.169:0/38684728 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f46440b0c80 0x7f46440ad170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:21.004 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.998+0000 7f4642d9d640 1 -- 172.21.15.169:0/38684728 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f46441593e0 msgr2=0x7f46441468b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:21.004 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.998+0000 7f4642d9d640 1 --2- 172.21.15.169:0/38684728 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f46441593e0 0x7f46441468b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:21.004 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.998+0000 7f4642d9d640 1 -- 172.21.15.169:0/38684728 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4634009560 con 0x7f46441597c0 2023-10-26T01:32:21.005 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.998+0000 7f4642d9d640 1 --2- 172.21.15.169:0/38684728 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f46441597c0 0x7f46441558a0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f463c00c710 tx=0x7f463c00cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:21.005 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.998+0000 7f463affd640 1 -- 172.21.15.169:0/38684728 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f463c006d50 con 0x7f46441597c0 2023-10-26T01:32:21.005 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.998+0000 7f463affd640 1 -- 172.21.15.169:0/38684728 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f463c00ce20 con 0x7f46441597c0 2023-10-26T01:32:21.005 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.998+0000 7f463affd640 1 -- 172.21.15.169:0/38684728 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f463c003bd0 con 0x7f46441597c0 2023-10-26T01:32:21.005 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.998+0000 7f4648f4c640 1 -- 172.21.15.169:0/38684728 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f464414f230 con 0x7f46441597c0 2023-10-26T01:32:21.005 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:20.999+0000 7f4648f4c640 1 -- 172.21.15.169:0/38684728 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4644158770 con 0x7f46441597c0 2023-10-26T01:32:21.006 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.000+0000 7f4648f4c640 1 -- 172.21.15.169:0/38684728 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f464414f420 con 0x7f46441597c0 2023-10-26T01:32:21.006 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.001+0000 7f463affd640 1 -- 172.21.15.169:0/38684728 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f463c003d30 con 0x7f46441597c0 2023-10-26T01:32:21.007 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.001+0000 7f463affd640 1 --2- 172.21.15.169:0/38684728 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f462406ae40 0x7f462406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:21.007 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.002+0000 7f463affd640 1 -- 172.21.15.169:0/38684728 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f463c089490 con 0x7f46441597c0 2023-10-26T01:32:21.007 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.002+0000 7f464259c640 1 --2- 172.21.15.169:0/38684728 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f462406ae40 0x7f462406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:21.007 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.003+0000 7f464259c640 1 --2- 172.21.15.169:0/38684728 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f462406ae40 0x7f462406d300 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f4634035000 tx=0x7f4634004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:21.008 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.006+0000 7f463affd640 1 -- 172.21.15.169:0/38684728 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f463c054530 con 0x7f46441597c0 2023-10-26T01:32:21.198 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.195+0000 7f4648f4c640 1 -- 172.21.15.169:0/38684728 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f464404db10 con 0x7f46441597c0 2023-10-26T01:32:21.199 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.196+0000 7f463affd640 1 -- 172.21.15.169:0/38684728 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v37) v1 ==== 104+0+11 (secure 0 0 0) 0x7f463c057b50 con 0x7f46441597c0 2023-10-26T01:32:21.199 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 32 2023-10-26T01:32:21.201 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.199+0000 7f4648f4c640 1 -- 172.21.15.169:0/38684728 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f462406ae40 msgr2=0x7f462406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:21.202 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.199+0000 7f4648f4c640 1 --2- 172.21.15.169:0/38684728 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f462406ae40 0x7f462406d300 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f4634035000 tx=0x7f4634004730 comp rx=0 tx=0).stop 2023-10-26T01:32:21.202 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.199+0000 7f4648f4c640 1 -- 172.21.15.169:0/38684728 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f46441597c0 msgr2=0x7f46441558a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:21.202 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.199+0000 7f4648f4c640 1 --2- 172.21.15.169:0/38684728 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f46441597c0 0x7f46441558a0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f463c00c710 tx=0x7f463c00cbd0 comp rx=0 tx=0).stop 2023-10-26T01:32:21.202 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.199+0000 7f4648f4c640 1 -- 172.21.15.169:0/38684728 shutdown_connections 2023-10-26T01:32:21.202 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.199+0000 7f4648f4c640 1 --2- 172.21.15.169:0/38684728 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f462406ae40 0x7f462406d300 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:21.203 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.199+0000 7f4648f4c640 1 --2- 172.21.15.169:0/38684728 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f46441597c0 0x7f46441558a0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:21.203 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.199+0000 7f4648f4c640 1 --2- 172.21.15.169:0/38684728 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f46441593e0 0x7f46441468b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:21.203 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.199+0000 7f4648f4c640 1 --2- 172.21.15.169:0/38684728 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f46440b0c80 0x7f46440ad170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:21.203 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.199+0000 7f4648f4c640 1 -- 172.21.15.169:0/38684728 >> 172.21.15.169:0/38684728 conn(0x7f4644069930 msgr2=0x7f46441519f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:21.203 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.199+0000 7f4648f4c640 1 -- 172.21.15.169:0/38684728 shutdown_connections 2023-10-26T01:32:21.204 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.199+0000 7f4648f4c640 1 -- 172.21.15.169:0/38684728 wait complete. 2023-10-26T01:32:21.220 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-26T01:32:21.441 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.438+0000 7fb97aeea640 1 -- 172.21.15.169:0/3360996152 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb974153b90 msgr2=0x7fb974069900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:21.441 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.438+0000 7fb97aeea640 1 --2- 172.21.15.169:0/3360996152 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb974153b90 0x7fb974069900 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fb970005fd0 tx=0x7fb97002bd90 comp rx=0 tx=0).stop 2023-10-26T01:32:21.441 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.438+0000 7fb97aeea640 1 -- 172.21.15.169:0/3360996152 shutdown_connections 2023-10-26T01:32:21.442 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.438+0000 7fb97aeea640 1 --2- 172.21.15.169:0/3360996152 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb9740b0e50 0x7fb974069e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:21.442 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.438+0000 7fb97aeea640 1 --2- 172.21.15.169:0/3360996152 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb974153b90 0x7fb974069900 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:21.442 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.438+0000 7fb97aeea640 1 -- 172.21.15.169:0/3360996152 >> 172.21.15.169:0/3360996152 conn(0x7fb9740af110 msgr2=0x7fb9740af520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:21.442 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.438+0000 7fb97aeea640 1 -- 172.21.15.169:0/3360996152 shutdown_connections 2023-10-26T01:32:21.442 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.438+0000 7fb97aeea640 1 -- 172.21.15.169:0/3360996152 wait complete. 2023-10-26T01:32:21.444 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.440+0000 7fb97aeea640 1 Processor -- start 2023-10-26T01:32:21.444 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.440+0000 7fb97aeea640 1 -- start start 2023-10-26T01:32:21.444 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.441+0000 7fb97aeea640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb9740b0e50 0x7fb9741f2b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:21.444 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.441+0000 7fb97aeea640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb974153b90 0x7fb97406ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:21.444 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.441+0000 7fb97aeea640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb97415d0d0 0x7fb97406b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:21.445 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.441+0000 7fb97aeea640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9741f3060 con 0x7fb974153b90 2023-10-26T01:32:21.445 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.441+0000 7fb97aeea640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb974155010 con 0x7fb97415d0d0 2023-10-26T01:32:21.445 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.441+0000 7fb97aeea640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9741552f0 con 0x7fb9740b0e50 2023-10-26T01:32:21.445 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.441+0000 7fb9796e7640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb974153b90 0x7fb97406ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:21.445 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.441+0000 7fb9796e7640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb974153b90 0x7fb97406ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:42076/0 (socket says 172.21.15.169:42076) 2023-10-26T01:32:21.446 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.441+0000 7fb9796e7640 1 -- 172.21.15.169:0/1706379647 learned_addr learned my addr 172.21.15.169:0/1706379647 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:32:21.447 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.442+0000 7fb979ee8640 1 --2- 172.21.15.169:0/1706379647 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb9740b0e50 0x7fb9741f2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:21.447 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.442+0000 7fb9796e7640 1 -- 172.21.15.169:0/1706379647 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb9740b0e50 msgr2=0x7fb9741f2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:21.447 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.442+0000 7fb9796e7640 1 --2- 172.21.15.169:0/1706379647 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb9740b0e50 0x7fb9741f2b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:21.447 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.442+0000 7fb9796e7640 1 -- 172.21.15.169:0/1706379647 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb97415d0d0 msgr2=0x7fb97406b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:21.448 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.442+0000 7fb97a6e9640 1 --2- 172.21.15.169:0/1706379647 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb97415d0d0 0x7fb97406b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:21.448 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.442+0000 7fb9796e7640 1 --2- 172.21.15.169:0/1706379647 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb97415d0d0 0x7fb97406b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:21.448 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.442+0000 7fb9796e7640 1 -- 172.21.15.169:0/1706379647 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb970009560 con 0x7fb974153b90 2023-10-26T01:32:21.448 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.442+0000 7fb9796e7640 1 --2- 172.21.15.169:0/1706379647 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb974153b90 0x7fb97406ab10 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fb96400d740 tx=0x7fb96400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:21.448 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.442+0000 7fb96a7fc640 1 -- 172.21.15.169:0/1706379647 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb964009c30 con 0x7fb974153b90 2023-10-26T01:32:21.449 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.442+0000 7fb96a7fc640 1 -- 172.21.15.169:0/1706379647 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb96400de50 con 0x7fb974153b90 2023-10-26T01:32:21.449 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.442+0000 7fb97aeea640 1 -- 172.21.15.169:0/1706379647 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9741555d0 con 0x7fb974153b90 2023-10-26T01:32:21.449 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.442+0000 7fb96a7fc640 1 -- 172.21.15.169:0/1706379647 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb964009ea0 con 0x7fb974153b90 2023-10-26T01:32:21.449 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.443+0000 7fb97aeea640 1 -- 172.21.15.169:0/1706379647 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb974155b90 con 0x7fb974153b90 2023-10-26T01:32:21.449 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.444+0000 7fb97aeea640 1 -- 172.21.15.169:0/1706379647 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb974073c10 con 0x7fb974153b90 2023-10-26T01:32:21.450 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.445+0000 7fb96a7fc640 1 -- 172.21.15.169:0/1706379647 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fb964005280 con 0x7fb974153b90 2023-10-26T01:32:21.450 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.446+0000 7fb96a7fc640 1 --2- 172.21.15.169:0/1706379647 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb95806ae70 0x7fb95806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:21.451 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.446+0000 7fb96a7fc640 1 -- 172.21.15.169:0/1706379647 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb9640897d0 con 0x7fb974153b90 2023-10-26T01:32:21.451 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.446+0000 7fb979ee8640 1 --2- 172.21.15.169:0/1706379647 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb95806ae70 0x7fb95806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:21.451 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.448+0000 7fb979ee8640 1 --2- 172.21.15.169:0/1706379647 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb95806ae70 0x7fb95806d330 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb970035000 tx=0x7fb970004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:21.453 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.451+0000 7fb96a7fc640 1 -- 172.21.15.169:0/1706379647 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb964058310 con 0x7fb974153b90 2023-10-26T01:32:21.626 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.623+0000 7fb97aeea640 1 -- 172.21.15.169:0/1706379647 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb974071950 con 0x7fb974153b90 2023-10-26T01:32:21.627 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.624+0000 7fb96a7fc640 1 -- 172.21.15.169:0/1706379647 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v37) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb964017020 con 0x7fb974153b90 2023-10-26T01:32:21.627 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 64 2023-10-26T01:32:21.629 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.626+0000 7fb97aeea640 1 -- 172.21.15.169:0/1706379647 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb95806ae70 msgr2=0x7fb95806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:21.630 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.626+0000 7fb97aeea640 1 --2- 172.21.15.169:0/1706379647 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb95806ae70 0x7fb95806d330 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb970035000 tx=0x7fb970004730 comp rx=0 tx=0).stop 2023-10-26T01:32:21.630 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.627+0000 7fb97aeea640 1 -- 172.21.15.169:0/1706379647 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb974153b90 msgr2=0x7fb97406ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:21.630 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.627+0000 7fb97aeea640 1 --2- 172.21.15.169:0/1706379647 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb974153b90 0x7fb97406ab10 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fb96400d740 tx=0x7fb96400dc00 comp rx=0 tx=0).stop 2023-10-26T01:32:21.630 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.627+0000 7fb97aeea640 1 -- 172.21.15.169:0/1706379647 shutdown_connections 2023-10-26T01:32:21.630 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.627+0000 7fb97aeea640 1 --2- 172.21.15.169:0/1706379647 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb95806ae70 0x7fb95806d330 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:21.631 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.627+0000 7fb97aeea640 1 --2- 172.21.15.169:0/1706379647 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb974153b90 0x7fb97406ab10 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:21.631 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.627+0000 7fb97aeea640 1 --2- 172.21.15.169:0/1706379647 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb97415d0d0 0x7fb97406b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:21.631 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.627+0000 7fb97aeea640 1 --2- 172.21.15.169:0/1706379647 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb9740b0e50 0x7fb9741f2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:21.631 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.627+0000 7fb97aeea640 1 -- 172.21.15.169:0/1706379647 >> 172.21.15.169:0/1706379647 conn(0x7fb9740af110 msgr2=0x7fb9740abfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:21.632 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.627+0000 7fb97aeea640 1 -- 172.21.15.169:0/1706379647 shutdown_connections 2023-10-26T01:32:21.632 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.627+0000 7fb97aeea640 1 -- 172.21.15.169:0/1706379647 wait complete. 2023-10-26T01:32:21.645 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T01:32:21.857 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.854+0000 7fcc50bb7640 1 -- 172.21.15.169:0/958641037 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc4c14e790 msgr2=0x7fcc4c0aab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:21.857 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.854+0000 7fcc50bb7640 1 --2- 172.21.15.169:0/958641037 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc4c14e790 0x7fcc4c0aab40 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c06eaf0 tx=0x7fcc3c02ca20 comp rx=0 tx=0).stop 2023-10-26T01:32:21.857 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.854+0000 7fcc50bb7640 1 -- 172.21.15.169:0/958641037 shutdown_connections 2023-10-26T01:32:21.857 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.854+0000 7fcc50bb7640 1 --2- 172.21.15.169:0/958641037 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcc4c0cdc60 0x7fcc4c06ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:21.858 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.854+0000 7fcc50bb7640 1 --2- 172.21.15.169:0/958641037 >> v2:172.21.15.170:3300/0 conn(0x7fcc4c158850 0x7fcc4c0cd720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:21.858 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.854+0000 7fcc50bb7640 1 --2- 172.21.15.169:0/958641037 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc4c14e790 0x7fcc4c0aab40 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:21.858 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.854+0000 7fcc50bb7640 1 -- 172.21.15.169:0/958641037 >> 172.21.15.169:0/958641037 conn(0x7fcc4c069930 msgr2=0x7fcc4c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:21.858 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.854+0000 7fcc50bb7640 1 -- 172.21.15.169:0/958641037 shutdown_connections 2023-10-26T01:32:21.858 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.854+0000 7fcc50bb7640 1 -- 172.21.15.169:0/958641037 wait complete. 2023-10-26T01:32:21.860 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.856+0000 7fcc50bb7640 1 Processor -- start 2023-10-26T01:32:21.860 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.856+0000 7fcc50bb7640 1 -- start start 2023-10-26T01:32:21.861 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.857+0000 7fcc50bb7640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcc4c0cdc60 0x7fcc4c096670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:21.861 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.857+0000 7fcc50bb7640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc4c14e790 0x7fcc4c0aeec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:21.861 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.857+0000 7fcc50bb7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc4c158850 0x7fcc4c0af400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:21.861 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.857+0000 7fcc50bb7640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc4c0706e0 con 0x7fcc4c14e790 2023-10-26T01:32:21.861 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.857+0000 7fcc50bb7640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc4c096bb0 con 0x7fcc4c158850 2023-10-26T01:32:21.862 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.857+0000 7fcc50bb7640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc4c06fa60 con 0x7fcc4c0cdc60 2023-10-26T01:32:21.862 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.857+0000 7fcc43fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc4c14e790 0x7fcc4c0aeec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:21.862 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.857+0000 7fcc43fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc4c14e790 0x7fcc4c0aeec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:42092/0 (socket says 172.21.15.169:42092) 2023-10-26T01:32:21.862 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.857+0000 7fcc43fff640 1 -- 172.21.15.169:0/258890165 learned_addr learned my addr 172.21.15.169:0/258890165 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:32:21.863 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.857+0000 7fcc4a59c640 1 --2- 172.21.15.169:0/258890165 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcc4c0cdc60 0x7fcc4c096670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:21.863 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.858+0000 7fcc4ad9d640 1 --2- 172.21.15.169:0/258890165 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc4c158850 0x7fcc4c0af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:21.863 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.858+0000 7fcc43fff640 1 -- 172.21.15.169:0/258890165 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcc4c0cdc60 msgr2=0x7fcc4c096670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:21.863 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.858+0000 7fcc43fff640 1 --2- 172.21.15.169:0/258890165 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcc4c0cdc60 0x7fcc4c096670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:21.863 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.858+0000 7fcc43fff640 1 -- 172.21.15.169:0/258890165 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc4c158850 msgr2=0x7fcc4c0af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:21.864 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.858+0000 7fcc43fff640 1 --2- 172.21.15.169:0/258890165 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc4c158850 0x7fcc4c0af400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:21.864 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.858+0000 7fcc43fff640 1 -- 172.21.15.169:0/258890165 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc3c0071c0 con 0x7fcc4c14e790 2023-10-26T01:32:21.864 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.858+0000 7fcc4ad9d640 1 --2- 172.21.15.169:0/258890165 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc4c158850 0x7fcc4c0af400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:32:21.864 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.858+0000 7fcc43fff640 1 --2- 172.21.15.169:0/258890165 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc4c14e790 0x7fcc4c0aeec0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fcc3400c970 tx=0x7fcc3400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:21.865 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.858+0000 7fcc42ffd640 1 -- 172.21.15.169:0/258890165 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc34007bb0 con 0x7fcc4c14e790 2023-10-26T01:32:21.866 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.858+0000 7fcc42ffd640 1 -- 172.21.15.169:0/258890165 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fcc34007e00 con 0x7fcc4c14e790 2023-10-26T01:32:21.866 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.858+0000 7fcc42ffd640 1 -- 172.21.15.169:0/258890165 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc34005280 con 0x7fcc4c14e790 2023-10-26T01:32:21.866 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.858+0000 7fcc50bb7640 1 -- 172.21.15.169:0/258890165 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc4c06fd40 con 0x7fcc4c14e790 2023-10-26T01:32:21.866 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.858+0000 7fcc50bb7640 1 -- 172.21.15.169:0/258890165 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc4c070300 con 0x7fcc4c14e790 2023-10-26T01:32:21.866 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.859+0000 7fcc50bb7640 1 -- 172.21.15.169:0/258890165 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc4c070820 con 0x7fcc4c14e790 2023-10-26T01:32:21.867 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.860+0000 7fcc42ffd640 1 -- 172.21.15.169:0/258890165 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fcc340053e0 con 0x7fcc4c14e790 2023-10-26T01:32:21.867 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.862+0000 7fcc42ffd640 1 --2- 172.21.15.169:0/258890165 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcc2c06ae70 0x7fcc2c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:21.867 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.862+0000 7fcc42ffd640 1 -- 172.21.15.169:0/258890165 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcc340899d0 con 0x7fcc4c14e790 2023-10-26T01:32:21.867 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.862+0000 7fcc4a59c640 1 --2- 172.21.15.169:0/258890165 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcc2c06ae70 0x7fcc2c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:21.868 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.863+0000 7fcc4a59c640 1 --2- 172.21.15.169:0/258890165 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcc2c06ae70 0x7fcc2c06d330 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c035000 tx=0x7fcc3c004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:21.868 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:21.866+0000 7fcc42ffd640 1 -- 172.21.15.169:0/258890165 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcc4c070820 con 0x7fcc4c14e790 2023-10-26T01:32:22.103 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.100+0000 7fcc50bb7640 1 -- 172.21.15.169:0/258890165 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcc4c070820 con 0x7fcc4c14e790 2023-10-26T01:32:22.104 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.101+0000 7fcc42ffd640 1 -- 172.21.15.169:0/258890165 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4913 (secure 0 0 0) 0x7fcc4c070820 con 0x7fcc4c14e790 2023-10-26T01:32:22.105 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:32:22.105 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4640,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/61096113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":61096113},{"type":"v1","addr":"172.21.15.170:6836","nonce":61096113}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4762,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2825046244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2825046244},{"type":"v1","addr":"172.21.15.169:6833","nonce":2825046244}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4777,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/258866976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":258866976},{"type":"v1","addr":"172.21.15.170:6837","nonce":258866976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T01:31:57.759251+0000","modified":"2023-10-26T01:32:00.831213+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5196},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5196":{"gid":5196,"name":"c","rank":0,"incarnation":11,"state":"up:active","state_seq":4,"addr":"172.21.15.169:6835/871978528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":871978528},{"type":"v1","addr":"172.21.15.169:6835","nonce":871978528}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-26T01:32:22.105 INFO:teuthology.orchestra.run.smithi169.stderr:dumped fsmap epoch 15 2023-10-26T01:32:22.107 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.104+0000 7fcc50bb7640 1 -- 172.21.15.169:0/258890165 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcc2c06ae70 msgr2=0x7fcc2c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:22.108 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.104+0000 7fcc50bb7640 1 --2- 172.21.15.169:0/258890165 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcc2c06ae70 0x7fcc2c06d330 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c035000 tx=0x7fcc3c004730 comp rx=0 tx=0).stop 2023-10-26T01:32:22.108 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.104+0000 7fcc50bb7640 1 -- 172.21.15.169:0/258890165 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc4c14e790 msgr2=0x7fcc4c0aeec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:22.108 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.104+0000 7fcc50bb7640 1 --2- 172.21.15.169:0/258890165 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc4c14e790 0x7fcc4c0aeec0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fcc3400c970 tx=0x7fcc3400ce30 comp rx=0 tx=0).stop 2023-10-26T01:32:22.108 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.104+0000 7fcc50bb7640 1 -- 172.21.15.169:0/258890165 shutdown_connections 2023-10-26T01:32:22.108 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.104+0000 7fcc50bb7640 1 --2- 172.21.15.169:0/258890165 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcc2c06ae70 0x7fcc2c06d330 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:22.109 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.104+0000 7fcc50bb7640 1 --2- 172.21.15.169:0/258890165 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc4c14e790 0x7fcc4c0aeec0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:22.109 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.104+0000 7fcc50bb7640 1 --2- 172.21.15.169:0/258890165 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc4c158850 0x7fcc4c0af400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:22.109 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.104+0000 7fcc50bb7640 1 --2- 172.21.15.169:0/258890165 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcc4c0cdc60 0x7fcc4c096670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:22.109 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.104+0000 7fcc50bb7640 1 -- 172.21.15.169:0/258890165 >> 172.21.15.169:0/258890165 conn(0x7fcc4c069930 msgr2=0x7fcc4c150d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:22.109 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.105+0000 7fcc50bb7640 1 -- 172.21.15.169:0/258890165 shutdown_connections 2023-10-26T01:32:22.110 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.105+0000 7fcc50bb7640 1 -- 172.21.15.169:0/258890165 wait complete. 2023-10-26T01:32:22.124 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:32:22.335 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.332+0000 7fe9d8ea9640 1 -- 172.21.15.169:0/3583085007 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9d40cdc30 msgr2=0x7fe9d406ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:22.335 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.332+0000 7fe9d8ea9640 1 --2- 172.21.15.169:0/3583085007 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9d40cdc30 0x7fe9d406ad30 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fe9d406be60 tx=0x7fe9cc02bd10 comp rx=0 tx=0).stop 2023-10-26T01:32:22.335 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.332+0000 7fe9d8ea9640 1 -- 172.21.15.169:0/3583085007 shutdown_connections 2023-10-26T01:32:22.335 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.332+0000 7fe9d8ea9640 1 --2- 172.21.15.169:0/3583085007 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9d40cdc30 0x7fe9d406ad30 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:22.336 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.332+0000 7fe9d8ea9640 1 --2- 172.21.15.169:0/3583085007 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe9d40ac930 0x7fe9d40cd6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:22.336 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.332+0000 7fe9d8ea9640 1 --2- 172.21.15.169:0/3583085007 >> v2:172.21.15.170:3300/0 conn(0x7fe9d41592d0 0x7fe9d40aa3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:22.336 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.332+0000 7fe9d8ea9640 1 -- 172.21.15.169:0/3583085007 >> 172.21.15.169:0/3583085007 conn(0x7fe9d4069930 msgr2=0x7fe9d4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:22.336 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.332+0000 7fe9d8ea9640 1 -- 172.21.15.169:0/3583085007 shutdown_connections 2023-10-26T01:32:22.336 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.332+0000 7fe9d8ea9640 1 -- 172.21.15.169:0/3583085007 wait complete. 2023-10-26T01:32:22.338 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.333+0000 7fe9d8ea9640 1 Processor -- start 2023-10-26T01:32:22.338 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.334+0000 7fe9d8ea9640 1 -- start start 2023-10-26T01:32:22.338 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.334+0000 7fe9d8ea9640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe9d40ac930 0x7fe9d414adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:22.338 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.335+0000 7fe9d8ea9640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe9d40cdc30 0x7fe9d41f4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:22.339 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.335+0000 7fe9d8ea9640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9d41592d0 0x7fe9d41f0f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:22.339 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.335+0000 7fe9d8ea9640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d4070000 con 0x7fe9d41592d0 2023-10-26T01:32:22.339 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.335+0000 7fe9d8ea9640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d41f5260 con 0x7fe9d40ac930 2023-10-26T01:32:22.339 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.335+0000 7fe9d8ea9640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d4147f60 con 0x7fe9d40cdc30 2023-10-26T01:32:22.339 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.335+0000 7fe9d2d9d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9d41592d0 0x7fe9d41f0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:22.340 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.335+0000 7fe9d2d9d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9d41592d0 0x7fe9d41f0f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:42114/0 (socket says 172.21.15.169:42114) 2023-10-26T01:32:22.340 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.335+0000 7fe9d2d9d640 1 -- 172.21.15.169:0/1835603198 learned_addr learned my addr 172.21.15.169:0/1835603198 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:32:22.340 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.335+0000 7fe9d259c640 1 --2- 172.21.15.169:0/1835603198 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe9d40ac930 0x7fe9d414adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:22.340 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.335+0000 7fe9d1d9b640 1 --2- 172.21.15.169:0/1835603198 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe9d40cdc30 0x7fe9d41f4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:22.340 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.335+0000 7fe9d2d9d640 1 -- 172.21.15.169:0/1835603198 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe9d40cdc30 msgr2=0x7fe9d41f4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:22.341 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.335+0000 7fe9d2d9d640 1 --2- 172.21.15.169:0/1835603198 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe9d40cdc30 0x7fe9d41f4d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:22.341 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.335+0000 7fe9d2d9d640 1 -- 172.21.15.169:0/1835603198 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe9d40ac930 msgr2=0x7fe9d414adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:22.341 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.335+0000 7fe9d2d9d640 1 --2- 172.21.15.169:0/1835603198 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe9d40ac930 0x7fe9d414adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:22.341 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.335+0000 7fe9d2d9d640 1 -- 172.21.15.169:0/1835603198 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9cc009560 con 0x7fe9d41592d0 2023-10-26T01:32:22.341 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.335+0000 7fe9d259c640 1 --2- 172.21.15.169:0/1835603198 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe9d40ac930 0x7fe9d414adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:32:22.342 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.336+0000 7fe9d2d9d640 1 --2- 172.21.15.169:0/1835603198 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9d41592d0 0x7fe9d41f0f60 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc009230 tx=0x7fe9cc02dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:22.343 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.336+0000 7fe9caffd640 1 -- 172.21.15.169:0/1835603198 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9cc005690 con 0x7fe9d41592d0 2023-10-26T01:32:22.343 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.336+0000 7fe9caffd640 1 -- 172.21.15.169:0/1835603198 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe9cc0038f0 con 0x7fe9d41592d0 2023-10-26T01:32:22.343 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.336+0000 7fe9d8ea9640 1 -- 172.21.15.169:0/1835603198 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9d41f5490 con 0x7fe9d41592d0 2023-10-26T01:32:22.343 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.336+0000 7fe9caffd640 1 -- 172.21.15.169:0/1835603198 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9cc003d00 con 0x7fe9d41592d0 2023-10-26T01:32:22.344 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.336+0000 7fe9d8ea9640 1 -- 172.21.15.169:0/1835603198 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9d41f5a50 con 0x7fe9d41592d0 2023-10-26T01:32:22.344 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.337+0000 7fe9d8ea9640 1 -- 172.21.15.169:0/1835603198 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9d4070140 con 0x7fe9d41592d0 2023-10-26T01:32:22.344 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.338+0000 7fe9caffd640 1 -- 172.21.15.169:0/1835603198 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fe9cc03e450 con 0x7fe9d41592d0 2023-10-26T01:32:22.344 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.339+0000 7fe9caffd640 1 --2- 172.21.15.169:0/1835603198 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe9b406ae40 0x7fe9b406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:22.344 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.339+0000 7fe9caffd640 1 -- 172.21.15.169:0/1835603198 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe9cc0abf70 con 0x7fe9d41592d0 2023-10-26T01:32:22.344 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.340+0000 7fe9d259c640 1 --2- 172.21.15.169:0/1835603198 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe9b406ae40 0x7fe9b406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:22.345 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.341+0000 7fe9d259c640 1 --2- 172.21.15.169:0/1835603198 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe9b406ae40 0x7fe9b406d300 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe9c400ba00 tx=0x7fe9c4005c80 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:22.346 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.343+0000 7fe9caffd640 1 -- 172.21.15.169:0/1835603198 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe9cc0deca0 con 0x7fe9d41592d0 2023-10-26T01:32:22.540 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.537+0000 7fe9d8ea9640 1 -- 172.21.15.169:0/1835603198 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe9d41f5c40 con 0x7fe9d41592d0 2023-10-26T01:32:22.542 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.540+0000 7fe9caffd640 1 -- 172.21.15.169:0/1835603198 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+16617 (secure 0 0 0) 0x7fe9cc042090 con 0x7fe9d41592d0 2023-10-26T01:32:22.543 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:32:22.543 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":37,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:32:11.943092+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-10-26T01:31:55.401509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-10-26T01:31:56.776031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:32:22.545 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.542+0000 7fe9d8ea9640 1 -- 172.21.15.169:0/1835603198 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe9b406ae40 msgr2=0x7fe9b406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:22.545 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.542+0000 7fe9d8ea9640 1 --2- 172.21.15.169:0/1835603198 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe9b406ae40 0x7fe9b406d300 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe9c400ba00 tx=0x7fe9c4005c80 comp rx=0 tx=0).stop 2023-10-26T01:32:22.545 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.542+0000 7fe9d8ea9640 1 -- 172.21.15.169:0/1835603198 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9d41592d0 msgr2=0x7fe9d41f0f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:22.546 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.542+0000 7fe9d8ea9640 1 --2- 172.21.15.169:0/1835603198 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9d41592d0 0x7fe9d41f0f60 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc009230 tx=0x7fe9cc02dc60 comp rx=0 tx=0).stop 2023-10-26T01:32:22.546 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.542+0000 7fe9d8ea9640 1 -- 172.21.15.169:0/1835603198 shutdown_connections 2023-10-26T01:32:22.546 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.542+0000 7fe9d8ea9640 1 --2- 172.21.15.169:0/1835603198 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe9b406ae40 0x7fe9b406d300 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:22.546 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.542+0000 7fe9d8ea9640 1 --2- 172.21.15.169:0/1835603198 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9d41592d0 0x7fe9d41f0f60 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:22.546 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.542+0000 7fe9d8ea9640 1 --2- 172.21.15.169:0/1835603198 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe9d40cdc30 0x7fe9d41f4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:22.547 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.542+0000 7fe9d8ea9640 1 --2- 172.21.15.169:0/1835603198 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe9d40ac930 0x7fe9d414adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:22.547 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.542+0000 7fe9d8ea9640 1 -- 172.21.15.169:0/1835603198 >> 172.21.15.169:0/1835603198 conn(0x7fe9d4069930 msgr2=0x7fe9d4158550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:22.547 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.542+0000 7fe9d8ea9640 1 -- 172.21.15.169:0/1835603198 shutdown_connections 2023-10-26T01:32:22.547 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.543+0000 7fe9d8ea9640 1 -- 172.21.15.169:0/1835603198 wait complete. 2023-10-26T01:32:22.564 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-26T01:32:22.565 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T01:32:22.776 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.773+0000 7f0ed04e5640 1 -- 172.21.15.169:0/608620661 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0ec806bf70 msgr2=0x7f0ec80aab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:22.777 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.773+0000 7f0ed04e5640 1 --2- 172.21.15.169:0/608620661 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0ec806bf70 0x7f0ec80aab10 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f0ec806cd20 tx=0x7f0ebc02bd70 comp rx=0 tx=0).stop 2023-10-26T01:32:22.777 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.774+0000 7f0ed04e5640 1 -- 172.21.15.169:0/608620661 shutdown_connections 2023-10-26T01:32:22.777 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.774+0000 7f0ed04e5640 1 --2- 172.21.15.169:0/608620661 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0ec806bf70 0x7f0ec80aab10 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:22.777 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.774+0000 7f0ed04e5640 1 --2- 172.21.15.169:0/608620661 >> v2:172.21.15.170:3300/0 conn(0x7f0ec814ece0 0x7f0ec80aa1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:22.778 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.774+0000 7f0ed04e5640 1 -- 172.21.15.169:0/608620661 >> 172.21.15.169:0/608620661 conn(0x7f0ec8069930 msgr2=0x7f0ec8069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:22.778 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.774+0000 7f0ed04e5640 1 -- 172.21.15.169:0/608620661 shutdown_connections 2023-10-26T01:32:22.778 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.774+0000 7f0ed04e5640 1 -- 172.21.15.169:0/608620661 wait complete. 2023-10-26T01:32:22.780 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.775+0000 7f0ed04e5640 1 Processor -- start 2023-10-26T01:32:22.780 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.775+0000 7f0ed04e5640 1 -- start start 2023-10-26T01:32:22.781 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.776+0000 7f0ed04e5640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0ec806bf70 0x7f0ec8145a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:22.781 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.776+0000 7f0ed04e5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0ec806c420 0x7f0ec814d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:22.781 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.776+0000 7f0ed04e5640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0ec814ece0 0x7f0ec814a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:22.781 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.776+0000 7f0ed04e5640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec814e540 con 0x7f0ec806bf70 2023-10-26T01:32:22.781 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.776+0000 7f0ed04e5640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec8145f40 con 0x7f0ec806c420 2023-10-26T01:32:22.782 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.776+0000 7f0ed04e5640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec814e260 con 0x7f0ec814ece0 2023-10-26T01:32:22.782 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.777+0000 7f0ece281640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0ec806bf70 0x7f0ec8145a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:22.782 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.777+0000 7f0ece281640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0ec806bf70 0x7f0ec8145a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:42130/0 (socket says 172.21.15.169:42130) 2023-10-26T01:32:22.782 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.777+0000 7f0ece281640 1 -- 172.21.15.169:0/248736943 learned_addr learned my addr 172.21.15.169:0/248736943 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:32:22.783 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.777+0000 7f0ecda80640 1 --2- 172.21.15.169:0/248736943 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0ec806c420 0x7f0ec814d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:22.783 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.777+0000 7f0ecea82640 1 --2- 172.21.15.169:0/248736943 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0ec814ece0 0x7f0ec814a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:22.783 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.777+0000 7f0ece281640 1 -- 172.21.15.169:0/248736943 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0ec814ece0 msgr2=0x7f0ec814a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:22.783 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.777+0000 7f0ece281640 1 --2- 172.21.15.169:0/248736943 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0ec814ece0 0x7f0ec814a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:22.783 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.777+0000 7f0ece281640 1 -- 172.21.15.169:0/248736943 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0ec806c420 msgr2=0x7f0ec814d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:22.784 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.777+0000 7f0ece281640 1 --2- 172.21.15.169:0/248736943 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0ec806c420 0x7f0ec814d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:22.784 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.777+0000 7f0ece281640 1 -- 172.21.15.169:0/248736943 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ebc009490 con 0x7f0ec806bf70 2023-10-26T01:32:22.784 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.777+0000 7f0ecda80640 1 --2- 172.21.15.169:0/248736943 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0ec806c420 0x7f0ec814d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:32:22.784 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.777+0000 7f0ece281640 1 --2- 172.21.15.169:0/248736943 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0ec806bf70 0x7f0ec8145a00 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f0eb800c920 tx=0x7f0eb800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:22.784 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.778+0000 7f0ec6ffd640 1 -- 172.21.15.169:0/248736943 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eb8007c70 con 0x7f0ec806bf70 2023-10-26T01:32:22.784 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.778+0000 7f0ec6ffd640 1 -- 172.21.15.169:0/248736943 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0eb800f450 con 0x7f0ec806bf70 2023-10-26T01:32:22.785 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.778+0000 7f0ec6ffd640 1 -- 172.21.15.169:0/248736943 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eb800f860 con 0x7f0ec806bf70 2023-10-26T01:32:22.785 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.778+0000 7f0ed04e5640 1 -- 172.21.15.169:0/248736943 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ec815a3d0 con 0x7f0ec806bf70 2023-10-26T01:32:22.785 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.778+0000 7f0ed04e5640 1 -- 172.21.15.169:0/248736943 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ec815a990 con 0x7f0ec806bf70 2023-10-26T01:32:22.786 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.779+0000 7f0ed04e5640 1 -- 172.21.15.169:0/248736943 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ec815abb0 con 0x7f0ec806bf70 2023-10-26T01:32:22.786 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.780+0000 7f0ec6ffd640 1 -- 172.21.15.169:0/248736943 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f0eb800f9c0 con 0x7f0ec806bf70 2023-10-26T01:32:22.786 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.781+0000 7f0ec6ffd640 1 --2- 172.21.15.169:0/248736943 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f0eb406ae40 0x7f0eb406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:22.787 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.781+0000 7f0ec6ffd640 1 -- 172.21.15.169:0/248736943 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0eb8089e30 con 0x7f0ec806bf70 2023-10-26T01:32:22.787 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.782+0000 7f0ecda80640 1 --2- 172.21.15.169:0/248736943 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f0eb406ae40 0x7f0eb406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:22.787 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.783+0000 7f0ecda80640 1 --2- 172.21.15.169:0/248736943 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f0eb406ae40 0x7f0eb406d300 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc036000 tx=0x7f0ebc004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:22.788 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:22.786+0000 7f0ec6ffd640 1 -- 172.21.15.169:0/248736943 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0eb8054f50 con 0x7f0ec806bf70 2023-10-26T01:32:23.017 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.014+0000 7f0ed04e5640 1 -- 172.21.15.169:0/248736943 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0ec804db10 con 0x7f0ec806bf70 2023-10-26T01:32:23.018 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.016+0000 7f0ec6ffd640 1 -- 172.21.15.169:0/248736943 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4913 (secure 0 0 0) 0x7f0eb801d020 con 0x7f0ec806bf70 2023-10-26T01:32:23.019 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:32:23.019 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4640,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/61096113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":61096113},{"type":"v1","addr":"172.21.15.170:6836","nonce":61096113}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4762,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2825046244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2825046244},{"type":"v1","addr":"172.21.15.169:6833","nonce":2825046244}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4777,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/258866976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":258866976},{"type":"v1","addr":"172.21.15.170:6837","nonce":258866976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T01:31:57.759251+0000","modified":"2023-10-26T01:32:00.831213+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5196},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5196":{"gid":5196,"name":"c","rank":0,"incarnation":11,"state":"up:active","state_seq":4,"addr":"172.21.15.169:6835/871978528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":871978528},{"type":"v1","addr":"172.21.15.169:6835","nonce":871978528}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-26T01:32:23.019 INFO:teuthology.orchestra.run.smithi169.stderr:dumped fsmap epoch 15 2023-10-26T01:32:23.021 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.018+0000 7f0ed04e5640 1 -- 172.21.15.169:0/248736943 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f0eb406ae40 msgr2=0x7f0eb406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:23.021 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.018+0000 7f0ed04e5640 1 --2- 172.21.15.169:0/248736943 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f0eb406ae40 0x7f0eb406d300 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc036000 tx=0x7f0ebc004730 comp rx=0 tx=0).stop 2023-10-26T01:32:23.021 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.018+0000 7f0ed04e5640 1 -- 172.21.15.169:0/248736943 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0ec806bf70 msgr2=0x7f0ec8145a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:23.022 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.018+0000 7f0ed04e5640 1 --2- 172.21.15.169:0/248736943 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0ec806bf70 0x7f0ec8145a00 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f0eb800c920 tx=0x7f0eb800cde0 comp rx=0 tx=0).stop 2023-10-26T01:32:23.022 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.018+0000 7f0ed04e5640 1 -- 172.21.15.169:0/248736943 shutdown_connections 2023-10-26T01:32:23.022 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.018+0000 7f0ed04e5640 1 --2- 172.21.15.169:0/248736943 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f0eb406ae40 0x7f0eb406d300 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:23.022 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.018+0000 7f0ed04e5640 1 --2- 172.21.15.169:0/248736943 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0ec814ece0 0x7f0ec814a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:23.023 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.018+0000 7f0ed04e5640 1 --2- 172.21.15.169:0/248736943 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0ec806c420 0x7f0ec814d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:23.023 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.018+0000 7f0ed04e5640 1 --2- 172.21.15.169:0/248736943 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0ec806bf70 0x7f0ec8145a00 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:23.023 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.018+0000 7f0ed04e5640 1 -- 172.21.15.169:0/248736943 >> 172.21.15.169:0/248736943 conn(0x7f0ec8069930 msgr2=0x7f0ec8069d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:23.023 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.019+0000 7f0ed04e5640 1 -- 172.21.15.169:0/248736943 shutdown_connections 2023-10-26T01:32:23.023 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.019+0000 7f0ed04e5640 1 -- 172.21.15.169:0/248736943 wait complete. 2023-10-26T01:32:23.038 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T01:32:23.257 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.254+0000 7f178c687640 1 -- 172.21.15.169:0/3491088547 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f17840cdbb0 msgr2=0x7f17840b07d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:23.257 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.254+0000 7f178c687640 1 --2- 172.21.15.169:0/3491088547 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f17840cdbb0 0x7f17840b07d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f177c005fd0 tx=0x7f177c02c0f0 comp rx=0 tx=0).stop 2023-10-26T01:32:23.258 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.254+0000 7f178c687640 1 -- 172.21.15.169:0/3491088547 shutdown_connections 2023-10-26T01:32:23.258 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.254+0000 7f178c687640 1 --2- 172.21.15.169:0/3491088547 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f17840cdbb0 0x7f17840b07d0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:23.258 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.254+0000 7f178c687640 1 --2- 172.21.15.169:0/3491088547 >> v2:172.21.15.170:3300/0 conn(0x7f178406ed30 0x7f17840cd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:23.258 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.254+0000 7f178c687640 1 -- 172.21.15.169:0/3491088547 >> 172.21.15.169:0/3491088547 conn(0x7f1784069930 msgr2=0x7f1784069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:23.258 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.254+0000 7f178c687640 1 -- 172.21.15.169:0/3491088547 shutdown_connections 2023-10-26T01:32:23.259 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.254+0000 7f178c687640 1 -- 172.21.15.169:0/3491088547 wait complete. 2023-10-26T01:32:23.259 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.256+0000 7f178c687640 1 Processor -- start 2023-10-26T01:32:23.259 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.256+0000 7f178c687640 1 -- start start 2023-10-26T01:32:23.260 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.257+0000 7f178c687640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f178406ed30 0x7f1784145cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:23.261 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.257+0000 7f178c687640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f17840cdbb0 0x7f178406fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:23.261 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.257+0000 7f178c687640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f178415b610 0x7f17840703a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:23.261 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.257+0000 7f178c687640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1784155010 con 0x7f178415b610 2023-10-26T01:32:23.262 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.257+0000 7f178c687640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17841554a0 con 0x7f178406ed30 2023-10-26T01:32:23.262 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.257+0000 7f178c687640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1784154400 con 0x7f17840cdbb0 2023-10-26T01:32:23.262 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.258+0000 7f178ac24640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f178415b610 0x7f17840703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:23.262 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.258+0000 7f178a423640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f178406ed30 0x7f1784145cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:23.262 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.258+0000 7f178ac24640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f178415b610 0x7f17840703a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:42134/0 (socket says 172.21.15.169:42134) 2023-10-26T01:32:23.263 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.258+0000 7f178ac24640 1 -- 172.21.15.169:0/3937826745 learned_addr learned my addr 172.21.15.169:0/3937826745 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:32:23.263 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.258+0000 7f1789c22640 1 --2- 172.21.15.169:0/3937826745 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f17840cdbb0 0x7f178406fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:23.263 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.258+0000 7f178ac24640 1 -- 172.21.15.169:0/3937826745 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f17840cdbb0 msgr2=0x7f178406fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:23.263 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.258+0000 7f178ac24640 1 --2- 172.21.15.169:0/3937826745 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f17840cdbb0 0x7f178406fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:23.263 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.258+0000 7f178ac24640 1 -- 172.21.15.169:0/3937826745 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f178406ed30 msgr2=0x7f1784145cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:23.264 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.258+0000 7f178ac24640 1 --2- 172.21.15.169:0/3937826745 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f178406ed30 0x7f1784145cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:23.264 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.258+0000 7f178ac24640 1 -- 172.21.15.169:0/3937826745 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1774009560 con 0x7f178415b610 2023-10-26T01:32:23.264 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.259+0000 7f178ac24640 1 --2- 172.21.15.169:0/3937826745 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f178415b610 0x7f17840703a0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f177c009cd0 tx=0x7f177c005c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:23.265 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.259+0000 7f177affd640 1 -- 172.21.15.169:0/3937826745 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f177c004680 con 0x7f178415b610 2023-10-26T01:32:23.265 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.259+0000 7f177affd640 1 -- 172.21.15.169:0/3937826745 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f177c0345a0 con 0x7f178415b610 2023-10-26T01:32:23.266 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.259+0000 7f177affd640 1 -- 172.21.15.169:0/3937826745 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f177c0349b0 con 0x7f178415b610 2023-10-26T01:32:23.266 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.259+0000 7f178c687640 1 -- 172.21.15.169:0/3937826745 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f177c009840 con 0x7f178415b610 2023-10-26T01:32:23.266 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.259+0000 7f178c687640 1 -- 172.21.15.169:0/3937826745 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17841549c0 con 0x7f178415b610 2023-10-26T01:32:23.266 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.260+0000 7f178c687640 1 -- 172.21.15.169:0/3937826745 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17841551c0 con 0x7f178415b610 2023-10-26T01:32:23.266 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.261+0000 7f177affd640 1 -- 172.21.15.169:0/3937826745 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f177c034b10 con 0x7f178415b610 2023-10-26T01:32:23.267 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.262+0000 7f177affd640 1 --2- 172.21.15.169:0/3937826745 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f176806ae40 0x7f176806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:23.267 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.262+0000 7f177affd640 1 -- 172.21.15.169:0/3937826745 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f177c0ac730 con 0x7f178415b610 2023-10-26T01:32:23.267 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.263+0000 7f178a423640 1 --2- 172.21.15.169:0/3937826745 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f176806ae40 0x7f176806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:23.267 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.264+0000 7f178a423640 1 --2- 172.21.15.169:0/3937826745 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f176806ae40 0x7f176806d300 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f17800016e0 tx=0x7f1780009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:23.270 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.267+0000 7f177affd640 1 -- 172.21.15.169:0/3937826745 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f17841551c0 con 0x7f178415b610 2023-10-26T01:32:23.501 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.498+0000 7f178c687640 1 -- 172.21.15.169:0/3937826745 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f17841551c0 con 0x7f178415b610 2023-10-26T01:32:23.502 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.499+0000 7f177affd640 1 -- 172.21.15.169:0/3937826745 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4913 (secure 0 0 0) 0x7f17841551c0 con 0x7f178415b610 2023-10-26T01:32:23.502 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:32:23.503 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4640,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/61096113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":61096113},{"type":"v1","addr":"172.21.15.170:6836","nonce":61096113}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4762,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2825046244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2825046244},{"type":"v1","addr":"172.21.15.169:6833","nonce":2825046244}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4777,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/258866976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":258866976},{"type":"v1","addr":"172.21.15.170:6837","nonce":258866976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T01:31:57.759251+0000","modified":"2023-10-26T01:32:00.831213+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5196},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5196":{"gid":5196,"name":"c","rank":0,"incarnation":11,"state":"up:active","state_seq":4,"addr":"172.21.15.169:6835/871978528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":871978528},{"type":"v1","addr":"172.21.15.169:6835","nonce":871978528}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-26T01:32:23.503 INFO:teuthology.orchestra.run.smithi169.stderr:dumped fsmap epoch 15 2023-10-26T01:32:23.505 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.502+0000 7f178c687640 1 -- 172.21.15.169:0/3937826745 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f176806ae40 msgr2=0x7f176806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:23.505 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.502+0000 7f178c687640 1 --2- 172.21.15.169:0/3937826745 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f176806ae40 0x7f176806d300 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f17800016e0 tx=0x7f1780009040 comp rx=0 tx=0).stop 2023-10-26T01:32:23.505 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.502+0000 7f178c687640 1 -- 172.21.15.169:0/3937826745 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f178415b610 msgr2=0x7f17840703a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:23.506 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.502+0000 7f178c687640 1 --2- 172.21.15.169:0/3937826745 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f178415b610 0x7f17840703a0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f177c009cd0 tx=0x7f177c005c80 comp rx=0 tx=0).stop 2023-10-26T01:32:23.506 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.502+0000 7f178c687640 1 -- 172.21.15.169:0/3937826745 shutdown_connections 2023-10-26T01:32:23.506 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.502+0000 7f178c687640 1 --2- 172.21.15.169:0/3937826745 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f176806ae40 0x7f176806d300 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:23.506 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.502+0000 7f178c687640 1 --2- 172.21.15.169:0/3937826745 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f178415b610 0x7f17840703a0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:23.507 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.502+0000 7f178c687640 1 --2- 172.21.15.169:0/3937826745 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f17840cdbb0 0x7f178406fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:23.507 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.502+0000 7f178c687640 1 --2- 172.21.15.169:0/3937826745 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f178406ed30 0x7f1784145cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:23.507 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.502+0000 7f178c687640 1 -- 172.21.15.169:0/3937826745 >> 172.21.15.169:0/3937826745 conn(0x7f1784069930 msgr2=0x7f178415a720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:23.507 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.502+0000 7f178c687640 1 -- 172.21.15.169:0/3937826745 shutdown_connections 2023-10-26T01:32:23.508 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.502+0000 7f178c687640 1 -- 172.21.15.169:0/3937826745 wait complete. 2023-10-26T01:32:23.524 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:32:23.733 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.730+0000 7fe652e1a640 1 -- 172.21.15.169:0/706605246 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe64c14f6a0 msgr2=0x7fe64c0cd720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:23.734 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.730+0000 7fe652e1a640 1 --2- 172.21.15.169:0/706605246 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe64c14f6a0 0x7fe64c0cd720 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fe638009830 tx=0x7fe63802bd70 comp rx=0 tx=0).stop 2023-10-26T01:32:23.734 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.731+0000 7fe652e1a640 1 -- 172.21.15.169:0/706605246 shutdown_connections 2023-10-26T01:32:23.734 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.731+0000 7fe652e1a640 1 --2- 172.21.15.169:0/706605246 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe64c14f6a0 0x7fe64c0cd720 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:23.734 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.731+0000 7fe652e1a640 1 --2- 172.21.15.169:0/706605246 >> v2:172.21.15.170:3300/0 conn(0x7fe64c14f2c0 0x7fe64c159240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:23.735 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.731+0000 7fe652e1a640 1 -- 172.21.15.169:0/706605246 >> 172.21.15.169:0/706605246 conn(0x7fe64c069930 msgr2=0x7fe64c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:23.735 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.731+0000 7fe652e1a640 1 -- 172.21.15.169:0/706605246 shutdown_connections 2023-10-26T01:32:23.735 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.731+0000 7fe652e1a640 1 -- 172.21.15.169:0/706605246 wait complete. 2023-10-26T01:32:23.737 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.732+0000 7fe652e1a640 1 Processor -- start 2023-10-26T01:32:23.737 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.732+0000 7fe652e1a640 1 -- start start 2023-10-26T01:32:23.737 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.733+0000 7fe652e1a640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe64c0cdc60 0x7fe64c159b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:23.737 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.733+0000 7fe652e1a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe64c14f2c0 0x7fe64c148730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:23.737 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.733+0000 7fe652e1a640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe64c14f6a0 0x7fe64c0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:23.738 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.733+0000 7fe652e1a640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe64c150440 con 0x7fe64c0cdc60 2023-10-26T01:32:23.738 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.733+0000 7fe652e1a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe64c15a370 con 0x7fe64c14f2c0 2023-10-26T01:32:23.738 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.733+0000 7fe652e1a640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe64c15a650 con 0x7fe64c14f6a0 2023-10-26T01:32:23.738 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.734+0000 7fe650bb6640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe64c0cdc60 0x7fe64c159b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:23.738 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.734+0000 7fe650bb6640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe64c0cdc60 0x7fe64c159b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:42140/0 (socket says 172.21.15.169:42140) 2023-10-26T01:32:23.739 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.734+0000 7fe650bb6640 1 -- 172.21.15.169:0/2650963094 learned_addr learned my addr 172.21.15.169:0/2650963094 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:32:23.739 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.734+0000 7fe650bb6640 1 -- 172.21.15.169:0/2650963094 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe64c14f6a0 msgr2=0x7fe64c0706c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:23.739 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.734+0000 7fe650bb6640 1 --2- 172.21.15.169:0/2650963094 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe64c14f6a0 0x7fe64c0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:23.739 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.734+0000 7fe650bb6640 1 -- 172.21.15.169:0/2650963094 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe64c14f2c0 msgr2=0x7fe64c148730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:23.739 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.734+0000 7fe650bb6640 1 --2- 172.21.15.169:0/2650963094 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe64c14f2c0 0x7fe64c148730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:23.740 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.734+0000 7fe650bb6640 1 -- 172.21.15.169:0/2650963094 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe638009490 con 0x7fe64c0cdc60 2023-10-26T01:32:23.740 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.734+0000 7fe650bb6640 1 --2- 172.21.15.169:0/2650963094 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe64c0cdc60 0x7fe64c159b20 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fe64000c920 tx=0x7fe64000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:23.741 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.735+0000 7fe6497fa640 1 -- 172.21.15.169:0/2650963094 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe640007c70 con 0x7fe64c0cdc60 2023-10-26T01:32:23.741 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.735+0000 7fe6497fa640 1 -- 172.21.15.169:0/2650963094 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe64000f450 con 0x7fe64c0cdc60 2023-10-26T01:32:23.741 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.735+0000 7fe6497fa640 1 -- 172.21.15.169:0/2650963094 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe64000f860 con 0x7fe64c0cdc60 2023-10-26T01:32:23.742 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.735+0000 7fe652e1a640 1 -- 172.21.15.169:0/2650963094 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe64c159990 con 0x7fe64c0cdc60 2023-10-26T01:32:23.742 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.735+0000 7fe652e1a640 1 -- 172.21.15.169:0/2650963094 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe64c150670 con 0x7fe64c0cdc60 2023-10-26T01:32:23.742 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.736+0000 7fe652e1a640 1 -- 172.21.15.169:0/2650963094 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe64c04db10 con 0x7fe64c0cdc60 2023-10-26T01:32:23.742 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.737+0000 7fe6497fa640 1 -- 172.21.15.169:0/2650963094 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fe64000f9c0 con 0x7fe64c0cdc60 2023-10-26T01:32:23.742 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.738+0000 7fe6497fa640 1 --2- 172.21.15.169:0/2650963094 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe63006ae40 0x7fe63006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:23.743 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.738+0000 7fe6497fa640 1 -- 172.21.15.169:0/2650963094 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe640089df0 con 0x7fe64c0cdc60 2023-10-26T01:32:23.743 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.738+0000 7fe64bfff640 1 --2- 172.21.15.169:0/2650963094 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe63006ae40 0x7fe63006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:23.743 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.739+0000 7fe64bfff640 1 --2- 172.21.15.169:0/2650963094 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe63006ae40 0x7fe63006d300 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe638036000 tx=0x7fe6380046c0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:23.744 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.742+0000 7fe6497fa640 1 -- 172.21.15.169:0/2650963094 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe640054f30 con 0x7fe64c0cdc60 2023-10-26T01:32:23.935 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.932+0000 7fe652e1a640 1 -- 172.21.15.169:0/2650963094 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe64c0abe20 con 0x7fe64c0cdc60 2023-10-26T01:32:23.937 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.935+0000 7fe6497fa640 1 -- 172.21.15.169:0/2650963094 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+16617 (secure 0 0 0) 0x7fe640058550 con 0x7fe64c0cdc60 2023-10-26T01:32:23.938 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:32:23.938 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":37,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:32:11.943092+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-10-26T01:31:55.401509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-10-26T01:31:56.776031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:32:23.940 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.937+0000 7fe652e1a640 1 -- 172.21.15.169:0/2650963094 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe63006ae40 msgr2=0x7fe63006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:23.940 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.937+0000 7fe652e1a640 1 --2- 172.21.15.169:0/2650963094 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe63006ae40 0x7fe63006d300 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe638036000 tx=0x7fe6380046c0 comp rx=0 tx=0).stop 2023-10-26T01:32:23.941 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.937+0000 7fe652e1a640 1 -- 172.21.15.169:0/2650963094 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe64c0cdc60 msgr2=0x7fe64c159b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:23.941 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.937+0000 7fe652e1a640 1 --2- 172.21.15.169:0/2650963094 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe64c0cdc60 0x7fe64c159b20 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fe64000c920 tx=0x7fe64000cde0 comp rx=0 tx=0).stop 2023-10-26T01:32:23.941 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.937+0000 7fe652e1a640 1 -- 172.21.15.169:0/2650963094 shutdown_connections 2023-10-26T01:32:23.941 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.937+0000 7fe652e1a640 1 --2- 172.21.15.169:0/2650963094 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe63006ae40 0x7fe63006d300 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:23.941 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.937+0000 7fe652e1a640 1 --2- 172.21.15.169:0/2650963094 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe64c14f6a0 0x7fe64c0706c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:23.941 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.937+0000 7fe652e1a640 1 --2- 172.21.15.169:0/2650963094 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe64c14f2c0 0x7fe64c148730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:23.942 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.937+0000 7fe652e1a640 1 --2- 172.21.15.169:0/2650963094 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe64c0cdc60 0x7fe64c159b20 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:23.942 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.937+0000 7fe652e1a640 1 -- 172.21.15.169:0/2650963094 >> 172.21.15.169:0/2650963094 conn(0x7fe64c069930 msgr2=0x7fe64c069d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:23.942 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.938+0000 7fe652e1a640 1 -- 172.21.15.169:0/2650963094 shutdown_connections 2023-10-26T01:32:23.942 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:23.938+0000 7fe652e1a640 1 -- 172.21.15.169:0/2650963094 wait complete. 2023-10-26T01:32:23.960 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-26T01:32:24.157 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:24.154+0000 7f7853467640 1 -- 172.21.15.169:0/630343592 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f784c153b90 msgr2=0x7f784c069900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:24.157 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:24.154+0000 7f7853467640 1 --2- 172.21.15.169:0/630343592 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f784c153b90 0x7f784c069900 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f7848009830 tx=0x7f784802bd70 comp rx=0 tx=0).stop 2023-10-26T01:32:24.158 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:24.154+0000 7f7853467640 1 -- 172.21.15.169:0/630343592 shutdown_connections 2023-10-26T01:32:24.158 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:24.154+0000 7f7853467640 1 --2- 172.21.15.169:0/630343592 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f784c153b90 0x7f784c069900 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:24.158 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:24.154+0000 7f7853467640 1 -- 172.21.15.169:0/630343592 >> 172.21.15.169:0/630343592 conn(0x7f784c0af110 msgr2=0x7f784c0af520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:24.158 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:24.154+0000 7f7853467640 1 -- 172.21.15.169:0/630343592 shutdown_connections 2023-10-26T01:32:24.158 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:24.154+0000 7f7853467640 1 -- 172.21.15.169:0/630343592 wait complete. 2023-10-26T01:32:24.159 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:24.156+0000 7f7853467640 1 Processor -- start 2023-10-26T01:32:24.159 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:24.156+0000 7f7853467640 1 -- start start 2023-10-26T01:32:24.161 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:24.157+0000 7f7853467640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f784c0b0e50 0x7f784c0c9a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:24.161 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:24.157+0000 7f7853467640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f784c153b90 0x7f784c13fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:24.161 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:24.157+0000 7f7853467640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f784c15d7e0 0x7f784c142170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:24.162 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:24.157+0000 7f7853467640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f784c141130 con 0x7f784c15d7e0 2023-10-26T01:32:24.162 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:24.157+0000 7f7853467640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f784c1413b0 con 0x7f784c0b0e50 2023-10-26T01:32:24.162 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:24.157+0000 7f7853467640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f784c141690 con 0x7f784c153b90 2023-10-26T01:32:24.162 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:24.157+0000 7f7852c66640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f784c15d7e0 0x7f784c142170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:24.163 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:24.157+0000 7f7852c66640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f784c15d7e0 0x7f784c142170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:33242/0 (socket says 172.21.15.169:33242) 2023-10-26T01:32:24.163 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:24.157+0000 7f7852c66640 1 -- 172.21.15.169:0/2943515789 learned_addr learned my addr 172.21.15.169:0/2943515789 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:32:24.163 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:24.158+0000 7f7852c66640 1 -- 172.21.15.169:0/2943515789 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f784c153b90 msgr2=0x7f784c13fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:24.163 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:24.158+0000 7f7851c64640 1 --2- 172.21.15.169:0/2943515789 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f784c153b90 0x7f784c13fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:24.163 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:24.158+0000 7f7852465640 1 --2- 172.21.15.169:0/2943515789 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f784c0b0e50 0x7f784c0c9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:24.163 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:24.158+0000 7f7852c66640 1 --2- 172.21.15.169:0/2943515789 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f784c153b90 0x7f784c13fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:24.164 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:24.158+0000 7f7852c66640 1 -- 172.21.15.169:0/2943515789 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f784c0b0e50 msgr2=0x7f784c0c9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:24.164 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:24.158+0000 7f7852c66640 1 --2- 172.21.15.169:0/2943515789 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f784c0b0e50 0x7f784c0c9a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:24.164 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:24.158+0000 7f7852c66640 1 -- 172.21.15.169:0/2943515789 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7848009490 con 0x7f784c15d7e0 2023-10-26T01:32:24.164 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:24.158+0000 7f7851c64640 1 --2- 172.21.15.169:0/2943515789 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f784c153b90 0x7f784c13fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:32:24.164 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:24.158+0000 7f7852c66640 1 --2- 172.21.15.169:0/2943515789 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f784c15d7e0 0x7f784c142170 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f784400b990 tx=0x7f784400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:24.165 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:24.158+0000 7f7842ffd640 1 -- 172.21.15.169:0/2943515789 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f784400cba0 con 0x7f784c15d7e0 2023-10-26T01:32:24.165 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:24.158+0000 7f7842ffd640 1 -- 172.21.15.169:0/2943515789 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f784400cdf0 con 0x7f784c15d7e0 2023-10-26T01:32:24.165 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:24.158+0000 7f7853467640 1 -- 172.21.15.169:0/2943515789 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f784c141970 con 0x7f784c15d7e0 2023-10-26T01:32:24.165 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:24.158+0000 7f7842ffd640 1 -- 172.21.15.169:0/2943515789 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78440146a0 con 0x7f784c15d7e0 2023-10-26T01:32:24.165 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:24.159+0000 7f7853467640 1 -- 172.21.15.169:0/2943515789 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f784c1f3a80 con 0x7f784c15d7e0 2023-10-26T01:32:24.166 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:24.159+0000 7f7853467640 1 -- 172.21.15.169:0/2943515789 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f784c141e90 con 0x7f784c15d7e0 2023-10-26T01:32:24.167 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:24.161+0000 7f7842ffd640 1 -- 172.21.15.169:0/2943515789 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f784401d3b0 con 0x7f784c15d7e0 2023-10-26T01:32:24.167 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:24.162+0000 7f7842ffd640 1 --2- 172.21.15.169:0/2943515789 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f783006ae40 0x7f783006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:24.167 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:24.162+0000 7f7842ffd640 1 -- 172.21.15.169:0/2943515789 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7844089800 con 0x7f784c15d7e0 2023-10-26T01:32:24.167 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:24.162+0000 7f7852465640 1 --2- 172.21.15.169:0/2943515789 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f783006ae40 0x7f783006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:24.167 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:24.163+0000 7f7852465640 1 --2- 172.21.15.169:0/2943515789 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f783006ae40 0x7f783006d300 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7848036000 tx=0x7f7848004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:24.168 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:24.166+0000 7f7842ffd640 1 -- 172.21.15.169:0/2943515789 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f784c141e90 con 0x7f784c15d7e0 2023-10-26T01:32:24.381 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:24.378+0000 7f7853467640 1 -- 172.21.15.169:0/2943515789 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f784c141e90 con 0x7f784c15d7e0 2023-10-26T01:32:24.382 INFO:tasks.ceph.mon.a.smithi169.stderr:2023-10-26T01:32:24.379+0000 7f1588a74640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-26T01:32:24.385 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:24.383+0000 7f7842ffd640 1 -- 172.21.15.169:0/2943515789 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v16) v1 ==== 161+0+0 (secure 0 0 0) 0x7f784c141e90 con 0x7f784c15d7e0 2023-10-26T01:32:24.386 INFO:teuthology.orchestra.run.smithi169.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-26T01:32:24.393 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:24.390+0000 7f7853467640 1 -- 172.21.15.169:0/2943515789 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f783006ae40 msgr2=0x7f783006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:24.393 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:24.390+0000 7f7853467640 1 --2- 172.21.15.169:0/2943515789 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f783006ae40 0x7f783006d300 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7848036000 tx=0x7f7848004730 comp rx=0 tx=0).stop 2023-10-26T01:32:24.394 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:24.390+0000 7f7853467640 1 -- 172.21.15.169:0/2943515789 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f784c15d7e0 msgr2=0x7f784c142170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:24.394 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:24.390+0000 7f7853467640 1 --2- 172.21.15.169:0/2943515789 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f784c15d7e0 0x7f784c142170 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f784400b990 tx=0x7f784400be50 comp rx=0 tx=0).stop 2023-10-26T01:32:24.394 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:24.390+0000 7f7853467640 1 -- 172.21.15.169:0/2943515789 shutdown_connections 2023-10-26T01:32:24.394 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:24.390+0000 7f7853467640 1 --2- 172.21.15.169:0/2943515789 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f783006ae40 0x7f783006d300 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:24.394 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:24.390+0000 7f7853467640 1 --2- 172.21.15.169:0/2943515789 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f784c15d7e0 0x7f784c142170 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:24.395 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:24.390+0000 7f7853467640 1 --2- 172.21.15.169:0/2943515789 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f784c153b90 0x7f784c13fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:24.395 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:24.390+0000 7f7853467640 1 --2- 172.21.15.169:0/2943515789 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f784c0b0e50 0x7f784c0c9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:24.395 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:24.390+0000 7f7853467640 1 -- 172.21.15.169:0/2943515789 >> 172.21.15.169:0/2943515789 conn(0x7f784c0af110 msgr2=0x7f784c0abfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:24.395 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:24.390+0000 7f7853467640 1 -- 172.21.15.169:0/2943515789 shutdown_connections 2023-10-26T01:32:24.396 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:24.390+0000 7f7853467640 1 -- 172.21.15.169:0/2943515789 wait complete. 2023-10-26T01:32:24.399 INFO:tasks.ceph.mds.c.smithi169.stderr: -4810> 2023-10-26T01:32:04.259+0000 7f26119d3640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-26T01:32:24.411 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-26T01:32:24.455 INFO:tasks.ceph.mds.c.smithi169.stdout:starting mds.c at 2023-10-26T01:32:24.600 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:24.597+0000 7f216ffff640 1 -- 172.21.15.169:0/560221014 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2158003fd0 msgr2=0x7f21580047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:24.600 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:24.597+0000 7f216ffff640 1 --2- 172.21.15.169:0/560221014 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2158003fd0 0x7f21580047c0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f2164001680 tx=0x7f21580057f0 comp rx=0 tx=0).stop 2023-10-26T01:32:24.601 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:24.597+0000 7f216f7fe640 1 -- 172.21.15.169:0/560221014 reap_dead start 2023-10-26T01:32:24.601 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:24.597+0000 7f216ffff640 1 -- 172.21.15.169:0/560221014 shutdown_connections 2023-10-26T01:32:24.601 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:24.597+0000 7f216ffff640 1 -- 172.21.15.169:0/560221014 >> 172.21.15.169:0/560221014 conn(0x7f21700af110 msgr2=0x7f21700af520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:24.601 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:24.598+0000 7f216ffff640 1 -- 172.21.15.169:0/560221014 shutdown_connections 2023-10-26T01:32:24.602 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:24.598+0000 7f216ffff640 1 -- 172.21.15.169:0/560221014 wait complete. 2023-10-26T01:32:24.602 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:24.599+0000 7f216ffff640 1 Processor -- start 2023-10-26T01:32:24.602 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:24.599+0000 7f216ffff640 1 -- start start 2023-10-26T01:32:24.604 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:24.600+0000 7f216ffff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2158003fd0 0x7f2170147540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:24.604 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:24.600+0000 7f216ffff640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f21700cc3a0 0x7f2170148150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:24.604 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:24.600+0000 7f216ffff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2170156620 0x7f2170156aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:24.604 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:24.600+0000 7f216ffff640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2170147a80 con 0x7f2158003fd0 2023-10-26T01:32:24.604 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:24.600+0000 7f216ffff640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2170156fe0 con 0x7f2170156620 2023-10-26T01:32:24.605 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:24.600+0000 7f216ffff640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f217014c340 con 0x7f21700cc3a0 2023-10-26T01:32:24.605 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:24.601+0000 7f216effd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2158003fd0 0x7f2170147540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:24.605 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:24.601+0000 7f216effd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2158003fd0 0x7f2170147540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:33288/0 (socket says 172.21.15.169:33288) 2023-10-26T01:32:24.605 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:24.601+0000 7f216effd640 1 -- 172.21.15.169:0/2588627714 learned_addr learned my addr 172.21.15.169:0/2588627714 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:32:24.605 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:24.601+0000 7f216effd640 1 -- 172.21.15.169:0/2588627714 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f21700cc3a0 msgr2=0x7f2170148150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:24.606 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:24.601+0000 7f216effd640 1 --2- 172.21.15.169:0/2588627714 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f21700cc3a0 0x7f2170148150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:24.606 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:24.601+0000 7f216effd640 1 -- 172.21.15.169:0/2588627714 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2170156620 msgr2=0x7f2170156aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:24.606 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:24.601+0000 7f216effd640 1 --2- 172.21.15.169:0/2588627714 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2170156620 0x7f2170156aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:24.606 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:24.601+0000 7f216effd640 1 -- 172.21.15.169:0/2588627714 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2170149240 con 0x7f2158003fd0 2023-10-26T01:32:24.606 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:24.601+0000 7f216effd640 1 --2- 172.21.15.169:0/2588627714 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2158003fd0 0x7f2170147540 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f216000fa00 tx=0x7f216000fec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:24.608 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:24.602+0000 7f21577fe640 1 -- 172.21.15.169:0/2588627714 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f216000bc50 con 0x7f2158003fd0 2023-10-26T01:32:24.608 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:24.602+0000 7f21577fe640 1 -- 172.21.15.169:0/2588627714 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2160011440 con 0x7f2158003fd0 2023-10-26T01:32:24.608 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:24.602+0000 7f21577fe640 1 -- 172.21.15.169:0/2588627714 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f216000bea0 con 0x7f2158003fd0 2023-10-26T01:32:24.609 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:24.602+0000 7f216ffff640 1 -- 172.21.15.169:0/2588627714 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21701489a0 con 0x7f2158003fd0 2023-10-26T01:32:24.609 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:24.602+0000 7f216ffff640 1 -- 172.21.15.169:0/2588627714 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2170148f60 con 0x7f2158003fd0 2023-10-26T01:32:24.609 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:24.603+0000 7f216ffff640 1 -- 172.21.15.169:0/2588627714 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2170073c10 con 0x7f2158003fd0 2023-10-26T01:32:24.609 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:24.604+0000 7f21577fe640 1 -- 172.21.15.169:0/2588627714 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f2160011830 con 0x7f2158003fd0 2023-10-26T01:32:24.609 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:24.605+0000 7f21577fe640 1 --2- 172.21.15.169:0/2588627714 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f215c06ae40 0x7f215c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:24.610 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:24.605+0000 7f21577fe640 1 -- 172.21.15.169:0/2588627714 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f216008a330 con 0x7f2158003fd0 2023-10-26T01:32:24.610 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:24.606+0000 7f216e7fc640 1 --2- 172.21.15.169:0/2588627714 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f215c06ae40 0x7f215c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:24.610 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:24.607+0000 7f216e7fc640 1 --2- 172.21.15.169:0/2588627714 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f215c06ae40 0x7f215c06d300 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f2164034000 tx=0x7f2164001120 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:24.611 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:24.609+0000 7f21577fe640 1 -- 172.21.15.169:0/2588627714 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2160055410 con 0x7f2158003fd0 2023-10-26T01:32:24.839 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:24.837+0000 7f216ffff640 1 -- 172.21.15.169:0/2588627714 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f217014c110 con 0x7f2158003fd0 2023-10-26T01:32:25.389 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:25.386+0000 7f21577fe640 1 -- 172.21.15.169:0/2588627714 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v17) v1 ==== 104+0+0 (secure 0 0 0) 0x7f2160058a30 con 0x7f2158003fd0 2023-10-26T01:32:25.392 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:25.389+0000 7f216ffff640 1 -- 172.21.15.169:0/2588627714 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f215c06ae40 msgr2=0x7f215c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:25.393 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:25.389+0000 7f216ffff640 1 --2- 172.21.15.169:0/2588627714 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f215c06ae40 0x7f215c06d300 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f2164034000 tx=0x7f2164001120 comp rx=0 tx=0).stop 2023-10-26T01:32:25.393 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:25.390+0000 7f216ffff640 1 -- 172.21.15.169:0/2588627714 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2158003fd0 msgr2=0x7f2170147540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:25.393 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:25.390+0000 7f216ffff640 1 --2- 172.21.15.169:0/2588627714 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2158003fd0 0x7f2170147540 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f216000fa00 tx=0x7f216000fec0 comp rx=0 tx=0).stop 2023-10-26T01:32:25.393 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:25.390+0000 7f216ffff640 1 -- 172.21.15.169:0/2588627714 shutdown_connections 2023-10-26T01:32:25.394 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:25.390+0000 7f216ffff640 1 --2- 172.21.15.169:0/2588627714 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f215c06ae40 0x7f215c06d300 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:25.394 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:25.390+0000 7f216ffff640 1 --2- 172.21.15.169:0/2588627714 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2170156620 0x7f2170156aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:25.394 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:25.390+0000 7f216ffff640 1 --2- 172.21.15.169:0/2588627714 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f21700cc3a0 0x7f2170148150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:25.394 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:25.390+0000 7f216ffff640 1 --2- 172.21.15.169:0/2588627714 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2158003fd0 0x7f2170147540 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:25.395 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:25.390+0000 7f216ffff640 1 -- 172.21.15.169:0/2588627714 >> 172.21.15.169:0/2588627714 conn(0x7f21700af110 msgr2=0x7f217006f690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:25.395 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:25.390+0000 7f216ffff640 1 -- 172.21.15.169:0/2588627714 shutdown_connections 2023-10-26T01:32:25.395 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:25.390+0000 7f216ffff640 1 -- 172.21.15.169:0/2588627714 wait complete. 2023-10-26T01:32:25.415 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-26T01:32:25.688 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:25.685+0000 7f2808d1b640 1 -- 172.21.15.169:0/4150674169 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27f00047c0 msgr2=0x7f27f0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:25.688 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:25.685+0000 7f2808d1b640 1 --2- 172.21.15.169:0/4150674169 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27f00047c0 0x7f27f0004c40 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f27ec006ed0 tx=0x7f27f00057f0 comp rx=0 tx=0).stop 2023-10-26T01:32:25.688 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:25.685+0000 7f2808d1b640 1 -- 172.21.15.169:0/4150674169 shutdown_connections 2023-10-26T01:32:25.689 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:25.685+0000 7f2808d1b640 1 --2- 172.21.15.169:0/4150674169 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27f00047c0 0x7f27f0004c40 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:25.689 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:25.685+0000 7f2808d1b640 1 -- 172.21.15.169:0/4150674169 >> 172.21.15.169:0/4150674169 conn(0x7f28040d13a0 msgr2=0x7f28040d17b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:25.689 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:25.685+0000 7f2808d1b640 1 -- 172.21.15.169:0/4150674169 shutdown_connections 2023-10-26T01:32:25.689 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:25.685+0000 7f2808d1b640 1 -- 172.21.15.169:0/4150674169 wait complete. 2023-10-26T01:32:25.690 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:25.687+0000 7f2808d1b640 1 Processor -- start 2023-10-26T01:32:25.690 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:25.687+0000 7f2808d1b640 1 -- start start 2023-10-26T01:32:25.692 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:25.688+0000 7f2808d1b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f27f00047c0 0x7f28040b5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:25.692 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:25.688+0000 7f2808d1b640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f28040c88a0 0x7f28040b4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:25.692 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:25.688+0000 7f2808d1b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f28040b51e0 0x7f28041f4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:25.693 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:25.688+0000 7f2808d1b640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28040b5690 con 0x7f28040b51e0 2023-10-26T01:32:25.693 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:25.688+0000 7f2808d1b640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2804148880 con 0x7f27f00047c0 2023-10-26T01:32:25.693 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:25.688+0000 7f2808d1b640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28040cb6e0 con 0x7f28040c88a0 2023-10-26T01:32:25.693 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:25.688+0000 7f2803fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f28040b51e0 0x7f28041f4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:25.693 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:25.688+0000 7f2803fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f28040b51e0 0x7f28041f4dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:33314/0 (socket says 172.21.15.169:33314) 2023-10-26T01:32:25.694 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:25.688+0000 7f2803fff640 1 -- 172.21.15.169:0/1035491392 learned_addr learned my addr 172.21.15.169:0/1035491392 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:32:25.694 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:25.689+0000 7f28037fe640 1 --2- 172.21.15.169:0/1035491392 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f27f00047c0 0x7f28040b5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:25.694 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:25.689+0000 7f2803fff640 1 -- 172.21.15.169:0/1035491392 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f28040c88a0 msgr2=0x7f28040b4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:25.694 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:25.689+0000 7f2803fff640 1 --2- 172.21.15.169:0/1035491392 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f28040c88a0 0x7f28040b4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:25.694 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:25.689+0000 7f2803fff640 1 -- 172.21.15.169:0/1035491392 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f27f00047c0 msgr2=0x7f28040b5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:25.695 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:25.689+0000 7f2803fff640 1 --2- 172.21.15.169:0/1035491392 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f27f00047c0 0x7f28040b5ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:25.695 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:25.689+0000 7f2803fff640 1 -- 172.21.15.169:0/1035491392 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28040cb9c0 con 0x7f28040b51e0 2023-10-26T01:32:25.695 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:25.689+0000 7f28037fe640 1 --2- 172.21.15.169:0/1035491392 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f27f00047c0 0x7f28040b5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:32:25.695 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:25.689+0000 7f2803fff640 1 --2- 172.21.15.169:0/1035491392 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f28040b51e0 0x7f28041f4dc0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f28040b0e30 tx=0x7f27fc010da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:25.695 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:25.689+0000 7f27fbfff640 1 -- 172.21.15.169:0/1035491392 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27fc01b070 con 0x7f28040b51e0 2023-10-26T01:32:25.695 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:25.689+0000 7f27fbfff640 1 -- 172.21.15.169:0/1035491392 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f27fc004020 con 0x7f28040b51e0 2023-10-26T01:32:25.696 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:25.689+0000 7f27fbfff640 1 -- 172.21.15.169:0/1035491392 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27fc004430 con 0x7f28040b51e0 2023-10-26T01:32:25.696 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:25.689+0000 7f2808d1b640 1 -- 172.21.15.169:0/1035491392 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28040cbca0 con 0x7f28040b51e0 2023-10-26T01:32:25.696 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:25.689+0000 7f2808d1b640 1 -- 172.21.15.169:0/1035491392 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28040cc260 con 0x7f28040b51e0 2023-10-26T01:32:25.697 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:25.690+0000 7f2808d1b640 1 -- 172.21.15.169:0/1035491392 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2804073c10 con 0x7f28040b51e0 2023-10-26T01:32:25.697 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:25.691+0000 7f27fbfff640 1 -- 172.21.15.169:0/1035491392 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f27fc00e9d0 con 0x7f28040b51e0 2023-10-26T01:32:25.697 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:25.693+0000 7f27fbfff640 1 --2- 172.21.15.169:0/1035491392 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f27e406ae40 0x7f27e406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:25.698 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:25.693+0000 7f27fbfff640 1 -- 172.21.15.169:0/1035491392 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f27fc0897a0 con 0x7f28040b51e0 2023-10-26T01:32:25.698 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:25.693+0000 7f28037fe640 1 --2- 172.21.15.169:0/1035491392 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f27e406ae40 0x7f27e406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:25.698 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:25.694+0000 7f28037fe640 1 --2- 172.21.15.169:0/1035491392 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f27e406ae40 0x7f27e406d300 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f28041f4d20 tx=0x7f27f4006d80 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:25.700 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:25.697+0000 7f27fbfff640 1 -- 172.21.15.169:0/1035491392 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f27fc054880 con 0x7f28040b51e0 2023-10-26T01:32:25.877 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:25.874+0000 7f2808d1b640 1 -- 172.21.15.169:0/1035491392 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f28040cc420 con 0x7f28040b51e0 2023-10-26T01:32:26.393 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:26.390+0000 7f27fbfff640 1 -- 172.21.15.169:0/1035491392 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v39) v1 ==== 181+0+0 (secure 0 0 0) 0x7f27fc057ea0 con 0x7f28040b51e0 2023-10-26T01:32:26.393 INFO:teuthology.orchestra.run.smithi169.stderr:pool 'cephfs_metadata' removed 2023-10-26T01:32:26.396 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:26.393+0000 7f2808d1b640 1 -- 172.21.15.169:0/1035491392 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f27e406ae40 msgr2=0x7f27e406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:26.396 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:26.393+0000 7f2808d1b640 1 --2- 172.21.15.169:0/1035491392 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f27e406ae40 0x7f27e406d300 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f28041f4d20 tx=0x7f27f4006d80 comp rx=0 tx=0).stop 2023-10-26T01:32:26.397 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:26.393+0000 7f2808d1b640 1 -- 172.21.15.169:0/1035491392 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f28040b51e0 msgr2=0x7f28041f4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:26.397 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:26.393+0000 7f2808d1b640 1 --2- 172.21.15.169:0/1035491392 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f28040b51e0 0x7f28041f4dc0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f28040b0e30 tx=0x7f27fc010da0 comp rx=0 tx=0).stop 2023-10-26T01:32:26.397 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:26.393+0000 7f2808d1b640 1 -- 172.21.15.169:0/1035491392 shutdown_connections 2023-10-26T01:32:26.397 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:26.394+0000 7f2808d1b640 1 --2- 172.21.15.169:0/1035491392 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f27e406ae40 0x7f27e406d300 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:26.397 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:26.394+0000 7f2808d1b640 1 --2- 172.21.15.169:0/1035491392 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f28040b51e0 0x7f28041f4dc0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:26.398 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:26.394+0000 7f2808d1b640 1 --2- 172.21.15.169:0/1035491392 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f28040c88a0 0x7f28040b4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:26.398 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:26.394+0000 7f2808d1b640 1 --2- 172.21.15.169:0/1035491392 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f27f00047c0 0x7f28040b5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:26.398 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:26.394+0000 7f2808d1b640 1 -- 172.21.15.169:0/1035491392 >> 172.21.15.169:0/1035491392 conn(0x7f28040d13a0 msgr2=0x7f28040b3f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:26.398 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:26.394+0000 7f2808d1b640 1 -- 172.21.15.169:0/1035491392 shutdown_connections 2023-10-26T01:32:26.398 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:26.394+0000 7f2808d1b640 1 -- 172.21.15.169:0/1035491392 wait complete. 2023-10-26T01:32:26.416 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-26T01:32:26.630 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:26.627+0000 7f31ebb3c640 1 -- 172.21.15.169:0/128885836 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f31d00063b0 msgr2=0x7f31d000ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:26.630 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:26.627+0000 7f31ebb3c640 1 --2- 172.21.15.169:0/128885836 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f31d00063b0 0x7f31d000ca90 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f31dc00c610 tx=0x7f31d0006900 comp rx=0 tx=0).stop 2023-10-26T01:32:26.631 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:26.627+0000 7f31ebb3c640 1 -- 172.21.15.169:0/128885836 shutdown_connections 2023-10-26T01:32:26.631 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:26.627+0000 7f31ebb3c640 1 --2- 172.21.15.169:0/128885836 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f31d00063b0 0x7f31d000ca90 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:26.631 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:26.627+0000 7f31ebb3c640 1 -- 172.21.15.169:0/128885836 >> 172.21.15.169:0/128885836 conn(0x7f31e40aeee0 msgr2=0x7f31e40af2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:26.631 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:26.627+0000 7f31ebb3c640 1 -- 172.21.15.169:0/128885836 shutdown_connections 2023-10-26T01:32:26.631 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:26.627+0000 7f31ebb3c640 1 -- 172.21.15.169:0/128885836 wait complete. 2023-10-26T01:32:26.633 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:26.629+0000 7f31ebb3c640 1 Processor -- start 2023-10-26T01:32:26.633 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:26.629+0000 7f31ebb3c640 1 -- start start 2023-10-26T01:32:26.633 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:26.630+0000 7f31ebb3c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f31d00063b0 0x7f31e40cbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:26.633 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:26.630+0000 7f31ebb3c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f31e40c9530 0x7f31e4154690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:26.634 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:26.630+0000 7f31ebb3c640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f31e40c9970 0x7f31e4096570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:26.634 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:26.630+0000 7f31ebb3c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31e40cb7c0 con 0x7f31e40c9530 2023-10-26T01:32:26.634 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:26.630+0000 7f31ebb3c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31e40cc580 con 0x7f31d00063b0 2023-10-26T01:32:26.634 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:26.630+0000 7f31ebb3c640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31e40cc860 con 0x7f31e40c9970 2023-10-26T01:32:26.634 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:26.630+0000 7f31e90d7640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f31e40c9530 0x7f31e4154690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:26.635 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:26.630+0000 7f31e90d7640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f31e40c9530 0x7f31e4154690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:33338/0 (socket says 172.21.15.169:33338) 2023-10-26T01:32:26.635 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:26.630+0000 7f31e90d7640 1 -- 172.21.15.169:0/3992450429 learned_addr learned my addr 172.21.15.169:0/3992450429 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:32:26.636 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:26.631+0000 7f31ea0d9640 1 --2- 172.21.15.169:0/3992450429 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f31e40c9970 0x7f31e4096570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:26.637 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:26.631+0000 7f31e98d8640 1 --2- 172.21.15.169:0/3992450429 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f31d00063b0 0x7f31e40cbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:26.637 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:26.631+0000 7f31e90d7640 1 -- 172.21.15.169:0/3992450429 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f31e40c9970 msgr2=0x7f31e4096570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:26.637 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:26.631+0000 7f31e90d7640 1 --2- 172.21.15.169:0/3992450429 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f31e40c9970 0x7f31e4096570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:26.637 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:26.631+0000 7f31e90d7640 1 -- 172.21.15.169:0/3992450429 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f31d00063b0 msgr2=0x7f31e40cbfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:26.638 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:26.631+0000 7f31e90d7640 1 --2- 172.21.15.169:0/3992450429 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f31d00063b0 0x7f31e40cbfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:26.638 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:26.631+0000 7f31e90d7640 1 -- 172.21.15.169:0/3992450429 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31e40ccb40 con 0x7f31e40c9530 2023-10-26T01:32:26.638 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:26.631+0000 7f31e90d7640 1 --2- 172.21.15.169:0/3992450429 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f31e40c9530 0x7f31e4154690 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f31d800c410 tx=0x7f31d800c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:26.638 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:26.631+0000 7f31d67fc640 1 -- 172.21.15.169:0/3992450429 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31d8011790 con 0x7f31e40c9530 2023-10-26T01:32:26.638 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:26.631+0000 7f31d67fc640 1 -- 172.21.15.169:0/3992450429 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f31d80119e0 con 0x7f31e40c9530 2023-10-26T01:32:26.639 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:26.631+0000 7f31d67fc640 1 -- 172.21.15.169:0/3992450429 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31d8011e30 con 0x7f31e40c9530 2023-10-26T01:32:26.639 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:26.631+0000 7f31ebb3c640 1 -- 172.21.15.169:0/3992450429 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31e40cce20 con 0x7f31e40c9530 2023-10-26T01:32:26.639 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:26.632+0000 7f31ebb3c640 1 -- 172.21.15.169:0/3992450429 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31e4154bd0 con 0x7f31e40c9530 2023-10-26T01:32:26.639 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:26.633+0000 7f31ebb3c640 1 -- 172.21.15.169:0/3992450429 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31e4066af0 con 0x7f31e40c9530 2023-10-26T01:32:26.640 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:26.634+0000 7f31d67fc640 1 -- 172.21.15.169:0/3992450429 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f31d801c460 con 0x7f31e40c9530 2023-10-26T01:32:26.640 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:26.634+0000 7f31d67fc640 1 --2- 172.21.15.169:0/3992450429 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f31c806ae40 0x7f31c806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:26.641 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:26.635+0000 7f31d67fc640 1 -- 172.21.15.169:0/3992450429 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5357+0+0 (secure 0 0 0) 0x7f31d80896c0 con 0x7f31e40c9530 2023-10-26T01:32:26.641 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:26.635+0000 7f31e98d8640 1 --2- 172.21.15.169:0/3992450429 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f31c806ae40 0x7f31c806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:26.641 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:26.636+0000 7f31e98d8640 1 --2- 172.21.15.169:0/3992450429 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f31c806ae40 0x7f31c806d300 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f31e0005fd0 tx=0x7f31e0005ec0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:26.641 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:26.639+0000 7f31d67fc640 1 -- 172.21.15.169:0/3992450429 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f31d8054900 con 0x7f31e40c9530 2023-10-26T01:32:26.820 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:26.818+0000 7f31ebb3c640 1 -- 172.21.15.169:0/3992450429 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f31e4154d90 con 0x7f31e40c9530 2023-10-26T01:32:27.396 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:27.393+0000 7f31d67fc640 1 -- 172.21.15.169:0/3992450429 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v40) v1 ==== 169+0+0 (secure 0 0 0) 0x7f31d8089030 con 0x7f31e40c9530 2023-10-26T01:32:27.396 INFO:teuthology.orchestra.run.smithi169.stderr:pool 'cephfs_data' removed 2023-10-26T01:32:27.401 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:27.398+0000 7f31ebb3c640 1 -- 172.21.15.169:0/3992450429 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f31c806ae40 msgr2=0x7f31c806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:27.401 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:27.398+0000 7f31ebb3c640 1 --2- 172.21.15.169:0/3992450429 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f31c806ae40 0x7f31c806d300 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f31e0005fd0 tx=0x7f31e0005ec0 comp rx=0 tx=0).stop 2023-10-26T01:32:27.401 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:27.398+0000 7f31ebb3c640 1 -- 172.21.15.169:0/3992450429 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f31e40c9530 msgr2=0x7f31e4154690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:27.401 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:27.398+0000 7f31ebb3c640 1 --2- 172.21.15.169:0/3992450429 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f31e40c9530 0x7f31e4154690 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f31d800c410 tx=0x7f31d800c810 comp rx=0 tx=0).stop 2023-10-26T01:32:27.403 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:27.400+0000 7f31ebb3c640 1 -- 172.21.15.169:0/3992450429 shutdown_connections 2023-10-26T01:32:27.403 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:27.400+0000 7f31ebb3c640 1 --2- 172.21.15.169:0/3992450429 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f31c806ae40 0x7f31c806d300 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:27.403 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:27.400+0000 7f31ebb3c640 1 --2- 172.21.15.169:0/3992450429 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f31e40c9530 0x7f31e4154690 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:27.404 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:27.400+0000 7f31ebb3c640 1 --2- 172.21.15.169:0/3992450429 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f31e40c9970 0x7f31e4096570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:27.404 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:27.400+0000 7f31ebb3c640 1 --2- 172.21.15.169:0/3992450429 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f31d00063b0 0x7f31e40cbfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:27.404 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:27.400+0000 7f31ebb3c640 1 -- 172.21.15.169:0/3992450429 >> 172.21.15.169:0/3992450429 conn(0x7f31e40aeee0 msgr2=0x7f31e40cd910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:27.404 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:27.400+0000 7f31ebb3c640 1 -- 172.21.15.169:0/3992450429 shutdown_connections 2023-10-26T01:32:27.404 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:27.400+0000 7f31ebb3c640 1 -- 172.21.15.169:0/3992450429 wait complete. 2023-10-26T01:32:27.426 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 18 2023-10-26T01:32:27.645 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:27.642+0000 7f96cc832640 1 -- 172.21.15.169:0/3368054452 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f96c8157be0 msgr2=0x7f96c806b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:27.646 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:27.642+0000 7f96cc832640 1 --2- 172.21.15.169:0/3368054452 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f96c8157be0 0x7f96c806b740 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f96b8005fd0 tx=0x7f96b802c050 comp rx=0 tx=0).stop 2023-10-26T01:32:27.646 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:27.642+0000 7f96cc832640 1 -- 172.21.15.169:0/3368054452 shutdown_connections 2023-10-26T01:32:27.646 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:27.642+0000 7f96cc832640 1 --2- 172.21.15.169:0/3368054452 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f96c8157be0 0x7f96c806b740 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:27.646 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:27.642+0000 7f96cc832640 1 -- 172.21.15.169:0/3368054452 >> 172.21.15.169:0/3368054452 conn(0x7f96c8069930 msgr2=0x7f96c8069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:27.646 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:27.642+0000 7f96cc832640 1 -- 172.21.15.169:0/3368054452 shutdown_connections 2023-10-26T01:32:27.647 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:27.643+0000 7f96cc832640 1 -- 172.21.15.169:0/3368054452 wait complete. 2023-10-26T01:32:27.648 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:27.644+0000 7f96cc832640 1 Processor -- start 2023-10-26T01:32:27.648 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:27.644+0000 7f96cc832640 1 -- start start 2023-10-26T01:32:27.648 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:27.645+0000 7f96cc832640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f96c806ce50 0x7f96c81f0220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:27.649 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:27.645+0000 7f96cc832640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f96c80ab300 0x7f96c81dfef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:27.649 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:27.645+0000 7f96cc832640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96c8157be0 0x7f96c814c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:27.649 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:27.645+0000 7f96cc832640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96c81def90 con 0x7f96c80ab300 2023-10-26T01:32:27.649 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:27.645+0000 7f96cc832640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96c814f390 con 0x7f96c8157be0 2023-10-26T01:32:27.649 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:27.645+0000 7f96cc832640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96c814f630 con 0x7f96c806ce50 2023-10-26T01:32:27.650 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:27.646+0000 7f96c5d9b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f96c80ab300 0x7f96c81dfef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:27.650 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:27.646+0000 7f96c5d9b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f96c80ab300 0x7f96c81dfef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:33358/0 (socket says 172.21.15.169:33358) 2023-10-26T01:32:27.650 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:27.646+0000 7f96c5d9b640 1 -- 172.21.15.169:0/2329244134 learned_addr learned my addr 172.21.15.169:0/2329244134 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:32:27.651 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:27.646+0000 7f96c6d9d640 1 --2- 172.21.15.169:0/2329244134 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96c8157be0 0x7f96c814c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:27.652 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:27.646+0000 7f96c5d9b640 1 -- 172.21.15.169:0/2329244134 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f96c806ce50 msgr2=0x7f96c81f0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:27.652 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:27.646+0000 7f96c5d9b640 1 --2- 172.21.15.169:0/2329244134 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f96c806ce50 0x7f96c81f0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:27.652 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:27.646+0000 7f96c5d9b640 1 -- 172.21.15.169:0/2329244134 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96c8157be0 msgr2=0x7f96c814c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:27.652 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:27.646+0000 7f96c5d9b640 1 --2- 172.21.15.169:0/2329244134 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96c8157be0 0x7f96c814c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:27.652 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:27.646+0000 7f96c5d9b640 1 -- 172.21.15.169:0/2329244134 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96b8009840 con 0x7f96c80ab300 2023-10-26T01:32:27.653 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:27.646+0000 7f96c6d9d640 1 --2- 172.21.15.169:0/2329244134 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96c8157be0 0x7f96c814c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:32:27.653 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:27.646+0000 7f96c5d9b640 1 --2- 172.21.15.169:0/2329244134 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f96c80ab300 0x7f96c81dfef0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f96b000c490 tx=0x7f96b000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:27.653 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:27.647+0000 7f96beffd640 1 -- 172.21.15.169:0/2329244134 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b0012830 con 0x7f96c80ab300 2023-10-26T01:32:27.653 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:27.647+0000 7f96beffd640 1 -- 172.21.15.169:0/2329244134 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f96b0012a80 con 0x7f96c80ab300 2023-10-26T01:32:27.653 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:27.647+0000 7f96beffd640 1 -- 172.21.15.169:0/2329244134 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b0011380 con 0x7f96c80ab300 2023-10-26T01:32:27.654 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:27.647+0000 7f96cc832640 1 -- 172.21.15.169:0/2329244134 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96c814f910 con 0x7f96c80ab300 2023-10-26T01:32:27.654 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:27.647+0000 7f96cc832640 1 -- 172.21.15.169:0/2329244134 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96c81521c0 con 0x7f96c80ab300 2023-10-26T01:32:27.654 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:27.648+0000 7f96cc832640 1 -- 172.21.15.169:0/2329244134 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96c814fe30 con 0x7f96c80ab300 2023-10-26T01:32:27.654 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:27.649+0000 7f96beffd640 1 -- 172.21.15.169:0/2329244134 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f96b00114e0 con 0x7f96c80ab300 2023-10-26T01:32:27.655 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:27.649+0000 7f96beffd640 1 --2- 172.21.15.169:0/2329244134 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f96a806ae40 0x7f96a806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:27.655 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:27.650+0000 7f96beffd640 1 -- 172.21.15.169:0/2329244134 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f96b0089800 con 0x7f96c80ab300 2023-10-26T01:32:27.656 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:27.650+0000 7f96c659c640 1 --2- 172.21.15.169:0/2329244134 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f96a806ae40 0x7f96a806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:27.656 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:27.651+0000 7f96c659c640 1 --2- 172.21.15.169:0/2329244134 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f96a806ae40 0x7f96a806d300 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f96b8035000 tx=0x7f96b8004a40 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:27.656 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:27.654+0000 7f96beffd640 1 -- 172.21.15.169:0/2329244134 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f96c814fe30 con 0x7f96c80ab300 2023-10-26T01:32:27.839 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:27.836+0000 7f96cc832640 1 -- 172.21.15.169:0/2329244134 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "config reset", "num": 18} v 0) v1 -- 0x7f96c814fe30 con 0x7f96c80ab300 2023-10-26T01:32:27.846 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:27.843+0000 7f96beffd640 1 -- 172.21.15.169:0/2329244134 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 18}]=0 v20) v1 ==== 71+0+0 (secure 0 0 0) 0x7f96c814fe30 con 0x7f96c80ab300 2023-10-26T01:32:27.850 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:27.846+0000 7f96cc832640 1 -- 172.21.15.169:0/2329244134 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f96a806ae40 msgr2=0x7f96a806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:27.850 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:27.846+0000 7f96cc832640 1 --2- 172.21.15.169:0/2329244134 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f96a806ae40 0x7f96a806d300 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f96b8035000 tx=0x7f96b8004a40 comp rx=0 tx=0).stop 2023-10-26T01:32:27.850 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:27.846+0000 7f96cc832640 1 -- 172.21.15.169:0/2329244134 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f96c80ab300 msgr2=0x7f96c81dfef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:27.850 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:27.847+0000 7f96cc832640 1 --2- 172.21.15.169:0/2329244134 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f96c80ab300 0x7f96c81dfef0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f96b000c490 tx=0x7f96b000c950 comp rx=0 tx=0).stop 2023-10-26T01:32:27.851 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:27.847+0000 7f96cc832640 1 -- 172.21.15.169:0/2329244134 shutdown_connections 2023-10-26T01:32:27.851 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:27.847+0000 7f96cc832640 1 --2- 172.21.15.169:0/2329244134 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f96a806ae40 0x7f96a806d300 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:27.851 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:27.847+0000 7f96cc832640 1 --2- 172.21.15.169:0/2329244134 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f96c80ab300 0x7f96c81dfef0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:27.851 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:27.847+0000 7f96cc832640 1 --2- 172.21.15.169:0/2329244134 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96c8157be0 0x7f96c814c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:27.851 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:27.847+0000 7f96cc832640 1 --2- 172.21.15.169:0/2329244134 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f96c806ce50 0x7f96c81f0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:27.851 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:27.847+0000 7f96cc832640 1 -- 172.21.15.169:0/2329244134 >> 172.21.15.169:0/2329244134 conn(0x7f96c8069930 msgr2=0x7f96c8097a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:27.852 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:27.847+0000 7f96cc832640 1 -- 172.21.15.169:0/2329244134 shutdown_connections 2023-10-26T01:32:27.852 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:27.847+0000 7f96cc832640 1 -- 172.21.15.169:0/2329244134 wait complete. 2023-10-26T01:32:27.867 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestClusterFull.test_barrier' 2023-10-26T01:32:28.079 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:28.076+0000 7fd74f24f640 1 -- 172.21.15.169:0/1694963103 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7481592d0 msgr2=0x7fd7480aa3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:28.079 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:28.076+0000 7fd74f24f640 1 --2- 172.21.15.169:0/1694963103 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7481592d0 0x7fd7480aa3d0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fd7480ab9b0 tx=0x7fd73802bd10 comp rx=0 tx=0).stop 2023-10-26T01:32:28.080 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:28.076+0000 7fd74f24f640 1 -- 172.21.15.169:0/1694963103 shutdown_connections 2023-10-26T01:32:28.080 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:28.076+0000 7fd74f24f640 1 --2- 172.21.15.169:0/1694963103 >> v2:172.21.15.170:3300/0 conn(0x7fd7480cdc30 0x7fd74806ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:28.080 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:28.076+0000 7fd74f24f640 1 --2- 172.21.15.169:0/1694963103 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd7480ac930 0x7fd7480cd6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:28.080 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:28.076+0000 7fd74f24f640 1 --2- 172.21.15.169:0/1694963103 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7481592d0 0x7fd7480aa3d0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:28.080 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:28.076+0000 7fd74f24f640 1 -- 172.21.15.169:0/1694963103 >> 172.21.15.169:0/1694963103 conn(0x7fd748069930 msgr2=0x7fd748069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:28.081 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:28.076+0000 7fd74f24f640 1 -- 172.21.15.169:0/1694963103 shutdown_connections 2023-10-26T01:32:28.081 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:28.076+0000 7fd74f24f640 1 -- 172.21.15.169:0/1694963103 wait complete. 2023-10-26T01:32:28.081 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:28.078+0000 7fd74f24f640 1 Processor -- start 2023-10-26T01:32:28.082 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:28.079+0000 7fd74f24f640 1 -- start start 2023-10-26T01:32:28.083 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:28.080+0000 7fd74f24f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7480ac930 0x7fd74815c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:28.083 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:28.080+0000 7fd74f24f640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd7480cdc30 0x7fd7480b1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:28.084 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:28.080+0000 7fd74f24f640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7481592d0 0x7fd748154840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:28.084 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:28.080+0000 7fd74f24f640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7481f52e0 con 0x7fd7481592d0 2023-10-26T01:32:28.084 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:28.080+0000 7fd74f24f640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd74815be40 con 0x7fd7480ac930 2023-10-26T01:32:28.084 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:28.080+0000 7fd74f24f640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd74815c620 con 0x7fd7480cdc30 2023-10-26T01:32:28.084 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:28.080+0000 7fd74d7ec640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7481592d0 0x7fd748154840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:28.085 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:28.080+0000 7fd74d7ec640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7481592d0 0x7fd748154840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:33368/0 (socket says 172.21.15.169:33368) 2023-10-26T01:32:28.085 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:28.080+0000 7fd74d7ec640 1 -- 172.21.15.169:0/3759299815 learned_addr learned my addr 172.21.15.169:0/3759299815 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:32:28.085 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:28.081+0000 7fd74cfeb640 1 --2- 172.21.15.169:0/3759299815 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7480ac930 0x7fd74815c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:28.085 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:28.081+0000 7fd747fff640 1 --2- 172.21.15.169:0/3759299815 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd7480cdc30 0x7fd7480b1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:28.085 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:28.081+0000 7fd74d7ec640 1 -- 172.21.15.169:0/3759299815 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd7480cdc30 msgr2=0x7fd7480b1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:28.086 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:28.081+0000 7fd74d7ec640 1 --2- 172.21.15.169:0/3759299815 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd7480cdc30 0x7fd7480b1ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:28.086 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:28.081+0000 7fd74d7ec640 1 -- 172.21.15.169:0/3759299815 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7480ac930 msgr2=0x7fd74815c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:28.086 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:28.081+0000 7fd74d7ec640 1 --2- 172.21.15.169:0/3759299815 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7480ac930 0x7fd74815c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:28.086 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:28.081+0000 7fd74d7ec640 1 -- 172.21.15.169:0/3759299815 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd738009560 con 0x7fd7481592d0 2023-10-26T01:32:28.087 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:28.081+0000 7fd74d7ec640 1 --2- 172.21.15.169:0/3759299815 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7481592d0 0x7fd748154840 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fd74000b970 tx=0x7fd74000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:28.088 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:28.081+0000 7fd7457fa640 1 -- 172.21.15.169:0/3759299815 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd74000cb40 con 0x7fd7481592d0 2023-10-26T01:32:28.088 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:28.081+0000 7fd7457fa640 1 -- 172.21.15.169:0/3759299815 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd74000cdb0 con 0x7fd7481592d0 2023-10-26T01:32:28.088 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:28.081+0000 7fd7457fa640 1 -- 172.21.15.169:0/3759299815 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7400146a0 con 0x7fd7481592d0 2023-10-26T01:32:28.088 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:28.081+0000 7fd74f24f640 1 -- 172.21.15.169:0/3759299815 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7480c6690 con 0x7fd7481592d0 2023-10-26T01:32:28.089 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:28.081+0000 7fd74f24f640 1 -- 172.21.15.169:0/3759299815 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd748154d80 con 0x7fd7481592d0 2023-10-26T01:32:28.089 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:28.082+0000 7fd74f24f640 1 -- 172.21.15.169:0/3759299815 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7481f5450 con 0x7fd7481592d0 2023-10-26T01:32:28.089 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:28.083+0000 7fd7457fa640 1 -- 172.21.15.169:0/3759299815 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fd74001d3b0 con 0x7fd7481592d0 2023-10-26T01:32:28.089 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:28.084+0000 7fd7457fa640 1 --2- 172.21.15.169:0/3759299815 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd73406ae40 0x7fd73406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:28.089 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:28.084+0000 7fd7457fa640 1 -- 172.21.15.169:0/3759299815 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fd7400894e0 con 0x7fd7481592d0 2023-10-26T01:32:28.090 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:28.085+0000 7fd74cfeb640 1 --2- 172.21.15.169:0/3759299815 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd73406ae40 0x7fd73406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:28.090 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:28.086+0000 7fd74cfeb640 1 --2- 172.21.15.169:0/3759299815 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd73406ae40 0x7fd73406d300 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd738035000 tx=0x7fd738004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:28.091 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:28.089+0000 7fd7457fa640 1 -- 172.21.15.169:0/3759299815 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd7481f5450 con 0x7fd7481592d0 2023-10-26T01:32:28.352 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:28.350+0000 7fd74f24f640 1 -- 172.21.15.169:0/3759299815 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_barrier"]} v 0) v1 -- 0x7fd7481f5420 con 0x7fd7481592d0 2023-10-26T01:32:28.414 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:28.412+0000 7fd7457fa640 1 -- 172.21.15.169:0/3759299815 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_barrier"]}]=0 v116) v1 ==== 130+0+0 (secure 0 0 0) 0x7fd7400190c0 con 0x7fd7481592d0 2023-10-26T01:32:28.418 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:28.414+0000 7fd74f24f640 1 -- 172.21.15.169:0/3759299815 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd73406ae40 msgr2=0x7fd73406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:28.418 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:28.415+0000 7fd74f24f640 1 --2- 172.21.15.169:0/3759299815 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd73406ae40 0x7fd73406d300 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd738035000 tx=0x7fd738004730 comp rx=0 tx=0).stop 2023-10-26T01:32:28.418 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:28.415+0000 7fd74f24f640 1 -- 172.21.15.169:0/3759299815 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7481592d0 msgr2=0x7fd748154840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:28.418 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:28.415+0000 7fd74f24f640 1 --2- 172.21.15.169:0/3759299815 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7481592d0 0x7fd748154840 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fd74000b970 tx=0x7fd74000be30 comp rx=0 tx=0).stop 2023-10-26T01:32:28.419 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:28.415+0000 7fd74f24f640 1 -- 172.21.15.169:0/3759299815 shutdown_connections 2023-10-26T01:32:28.419 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:28.415+0000 7fd74f24f640 1 --2- 172.21.15.169:0/3759299815 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd73406ae40 0x7fd73406d300 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:28.419 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:28.415+0000 7fd74f24f640 1 --2- 172.21.15.169:0/3759299815 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7481592d0 0x7fd748154840 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:28.419 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:28.415+0000 7fd74f24f640 1 --2- 172.21.15.169:0/3759299815 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd7480cdc30 0x7fd7480b1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:28.420 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:28.415+0000 7fd74f24f640 1 --2- 172.21.15.169:0/3759299815 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7480ac930 0x7fd74815c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:28.420 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:28.415+0000 7fd74f24f640 1 -- 172.21.15.169:0/3759299815 >> 172.21.15.169:0/3759299815 conn(0x7fd748069930 msgr2=0x7fd7480cbdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:28.420 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:28.415+0000 7fd74f24f640 1 -- 172.21.15.169:0/3759299815 shutdown_connections 2023-10-26T01:32:28.420 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:28.415+0000 7fd74f24f640 1 -- 172.21.15.169:0/3759299815 wait complete. 2023-10-26T01:32:28.436 INFO:tasks.cephfs_test_runner:That when an OSD epoch barrier is set on an MDS, subsequently ... ok 2023-10-26T01:32:28.437 INFO:tasks.cephfs_test_runner:Starting test: test_full_different_file (tasks.cephfs.test_full.TestClusterFull) 2023-10-26T01:32:28.437 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:32:28.657 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:28.654+0000 7ff2974fc640 1 -- 172.21.15.169:0/3916208196 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2901592d0 msgr2=0x7ff2900aa3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:28.657 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:28.654+0000 7ff2974fc640 1 --2- 172.21.15.169:0/3916208196 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2901592d0 0x7ff2900aa3d0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7ff2900acdf0 tx=0x7ff28c02bce0 comp rx=0 tx=0).stop 2023-10-26T01:32:28.657 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:28.654+0000 7ff2974fc640 1 -- 172.21.15.169:0/3916208196 shutdown_connections 2023-10-26T01:32:28.658 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:28.654+0000 7ff2974fc640 1 --2- 172.21.15.169:0/3916208196 >> v2:172.21.15.170:3300/0 conn(0x7ff2900cde00 0x7ff2900cbac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:28.658 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:28.654+0000 7ff2974fc640 1 --2- 172.21.15.169:0/3916208196 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2901592d0 0x7ff2900aa3d0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:28.658 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:28.654+0000 7ff2974fc640 1 -- 172.21.15.169:0/3916208196 >> 172.21.15.169:0/3916208196 conn(0x7ff290069930 msgr2=0x7ff290069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:28.659 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:28.654+0000 7ff2974fc640 1 -- 172.21.15.169:0/3916208196 shutdown_connections 2023-10-26T01:32:28.659 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:28.654+0000 7ff2974fc640 1 -- 172.21.15.169:0/3916208196 wait complete. 2023-10-26T01:32:28.661 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:28.656+0000 7ff2974fc640 1 Processor -- start 2023-10-26T01:32:28.661 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:28.656+0000 7ff2974fc640 1 -- start start 2023-10-26T01:32:28.661 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:28.657+0000 7ff2974fc640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2900ac930 0x7ff2901503f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:28.662 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:28.657+0000 7ff2974fc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff2900cde00 0x7ff29015a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:28.662 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:28.657+0000 7ff2974fc640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff2901592d0 0x7ff29015a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:28.662 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:28.657+0000 7ff2974fc640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff29015c910 con 0x7ff2900ac930 2023-10-26T01:32:28.662 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:28.657+0000 7ff2974fc640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff29015b590 con 0x7ff2900cde00 2023-10-26T01:32:28.663 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:28.657+0000 7ff2974fc640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff29015b870 con 0x7ff2901592d0 2023-10-26T01:32:28.663 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:28.657+0000 7ff295298640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2900ac930 0x7ff2901503f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:28.663 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:28.657+0000 7ff295298640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2900ac930 0x7ff2901503f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:33384/0 (socket says 172.21.15.169:33384) 2023-10-26T01:32:28.663 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:28.657+0000 7ff295298640 1 -- 172.21.15.169:0/1356618432 learned_addr learned my addr 172.21.15.169:0/1356618432 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:32:28.664 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:28.658+0000 7ff295298640 1 -- 172.21.15.169:0/1356618432 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff2901592d0 msgr2=0x7ff29015a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:28.664 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:28.658+0000 7ff295298640 1 --2- 172.21.15.169:0/1356618432 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff2901592d0 0x7ff29015a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:28.664 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:28.658+0000 7ff295298640 1 -- 172.21.15.169:0/1356618432 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff2900cde00 msgr2=0x7ff29015a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:28.665 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:28.658+0000 7ff295298640 1 --2- 172.21.15.169:0/1356618432 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff2900cde00 0x7ff29015a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:28.665 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:28.658+0000 7ff295298640 1 -- 172.21.15.169:0/1356618432 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff28c009490 con 0x7ff2900ac930 2023-10-26T01:32:28.665 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:28.658+0000 7ff295298640 1 --2- 172.21.15.169:0/1356618432 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2900ac930 0x7ff2901503f0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7ff28c02c2f0 tx=0x7ff28c004450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:28.665 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:28.658+0000 7ff285ffb640 1 -- 172.21.15.169:0/1356618432 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff28c003a00 con 0x7ff2900ac930 2023-10-26T01:32:28.666 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:28.658+0000 7ff285ffb640 1 -- 172.21.15.169:0/1356618432 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff28c003c50 con 0x7ff2900ac930 2023-10-26T01:32:28.666 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:28.658+0000 7ff285ffb640 1 -- 172.21.15.169:0/1356618432 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff28c034440 con 0x7ff2900ac930 2023-10-26T01:32:28.666 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:28.658+0000 7ff2974fc640 1 -- 172.21.15.169:0/1356618432 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff29015bb50 con 0x7ff2900ac930 2023-10-26T01:32:28.666 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:28.658+0000 7ff2974fc640 1 -- 172.21.15.169:0/1356618432 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff29015c110 con 0x7ff2900ac930 2023-10-26T01:32:28.667 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:28.660+0000 7ff2974fc640 1 -- 172.21.15.169:0/1356618432 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff290066e60 con 0x7ff2900ac930 2023-10-26T01:32:28.668 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:28.660+0000 7ff285ffb640 1 -- 172.21.15.169:0/1356618432 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ff28c0345a0 con 0x7ff2900ac930 2023-10-26T01:32:28.668 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:28.661+0000 7ff285ffb640 1 --2- 172.21.15.169:0/1356618432 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff27406ae40 0x7ff27406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:28.668 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:28.661+0000 7ff285ffb640 1 -- 172.21.15.169:0/1356618432 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4943+0+0 (secure 0 0 0) 0x7ff28c0abda0 con 0x7ff2900ac930 2023-10-26T01:32:28.668 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:28.662+0000 7ff294a97640 1 --2- 172.21.15.169:0/1356618432 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff27406ae40 0x7ff27406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:28.669 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:28.663+0000 7ff294a97640 1 --2- 172.21.15.169:0/1356618432 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff27406ae40 0x7ff27406d300 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff2900c9b10 tx=0x7ff280007930 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:28.669 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:28.666+0000 7ff285ffb640 1 -- 172.21.15.169:0/1356618432 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff28c0770b0 con 0x7ff2900ac930 2023-10-26T01:32:28.857 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:28.854+0000 7ff2974fc640 1 -- 172.21.15.169:0/1356618432 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff29004db10 con 0x7ff2900ac930 2023-10-26T01:32:28.858 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:28.856+0000 7ff285ffb640 1 -- 172.21.15.169:0/1356618432 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+13034 (secure 0 0 0) 0x7ff28c042090 con 0x7ff2900ac930 2023-10-26T01:32:28.859 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:32:28.859 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":40,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:32:27.391171+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.169:6835/871978528":"2023-10-27T01:32:24.381003+0000","172.21.15.169:6834/871978528":"2023-10-27T01:32:24.381003+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:32:28.861 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:28.858+0000 7ff2974fc640 1 -- 172.21.15.169:0/1356618432 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff27406ae40 msgr2=0x7ff27406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:28.861 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:28.858+0000 7ff2974fc640 1 --2- 172.21.15.169:0/1356618432 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff27406ae40 0x7ff27406d300 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff2900c9b10 tx=0x7ff280007930 comp rx=0 tx=0).stop 2023-10-26T01:32:28.861 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:28.858+0000 7ff2974fc640 1 -- 172.21.15.169:0/1356618432 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2900ac930 msgr2=0x7ff2901503f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:28.862 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:28.858+0000 7ff2974fc640 1 --2- 172.21.15.169:0/1356618432 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2900ac930 0x7ff2901503f0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7ff28c02c2f0 tx=0x7ff28c004450 comp rx=0 tx=0).stop 2023-10-26T01:32:28.862 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:28.858+0000 7ff2974fc640 1 -- 172.21.15.169:0/1356618432 shutdown_connections 2023-10-26T01:32:28.862 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:28.858+0000 7ff2974fc640 1 --2- 172.21.15.169:0/1356618432 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff27406ae40 0x7ff27406d300 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:28.862 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:28.858+0000 7ff2974fc640 1 --2- 172.21.15.169:0/1356618432 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff2901592d0 0x7ff29015a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:28.863 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:28.858+0000 7ff2974fc640 1 --2- 172.21.15.169:0/1356618432 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff2900cde00 0x7ff29015a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:28.863 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:28.858+0000 7ff2974fc640 1 --2- 172.21.15.169:0/1356618432 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2900ac930 0x7ff2901503f0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:28.863 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:28.858+0000 7ff2974fc640 1 -- 172.21.15.169:0/1356618432 >> 172.21.15.169:0/1356618432 conn(0x7ff290069930 msgr2=0x7ff290157f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:28.863 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:28.858+0000 7ff2974fc640 1 -- 172.21.15.169:0/1356618432 shutdown_connections 2023-10-26T01:32:28.863 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:28.858+0000 7ff2974fc640 1 -- 172.21.15.169:0/1356618432 wait complete. 2023-10-26T01:32:28.884 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T01:30:32.200808+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T01:32:28.885 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T01:32:29.110 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:29.107+0000 7f3d065fc640 1 -- 172.21.15.169:0/440825884 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3d00159700 msgr2=0x7f3d00069510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:29.111 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:29.107+0000 7f3d065fc640 1 --2- 172.21.15.169:0/440825884 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3d00159700 0x7f3d00069510 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f3ce8005fd0 tx=0x7f3ce802bc60 comp rx=0 tx=0).stop 2023-10-26T01:32:29.111 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:29.107+0000 7f3d065fc640 1 -- 172.21.15.169:0/440825884 shutdown_connections 2023-10-26T01:32:29.111 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:29.107+0000 7f3d065fc640 1 --2- 172.21.15.169:0/440825884 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3d000b0bf0 0x7f3d00069a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:29.111 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:29.107+0000 7f3d065fc640 1 --2- 172.21.15.169:0/440825884 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3d00159700 0x7f3d00069510 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:29.111 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:29.107+0000 7f3d065fc640 1 -- 172.21.15.169:0/440825884 >> 172.21.15.169:0/440825884 conn(0x7f3d000b00d0 msgr2=0x7f3d000b04e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:29.112 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:29.107+0000 7f3d065fc640 1 -- 172.21.15.169:0/440825884 shutdown_connections 2023-10-26T01:32:29.112 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:29.107+0000 7f3d065fc640 1 -- 172.21.15.169:0/440825884 wait complete. 2023-10-26T01:32:29.113 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:29.109+0000 7f3d065fc640 1 Processor -- start 2023-10-26T01:32:29.113 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:29.109+0000 7f3d065fc640 1 -- start start 2023-10-26T01:32:29.113 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:29.110+0000 7f3d065fc640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3d00069f90 0x7f3d0014fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:29.114 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:29.110+0000 7f3d065fc640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3d000b0bf0 0x7f3d000b11c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:29.114 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:29.110+0000 7f3d065fc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3d00159700 0x7f3d000ca8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:29.114 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:29.110+0000 7f3d065fc640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d000b1f40 con 0x7f3d00069f90 2023-10-26T01:32:29.114 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:29.110+0000 7f3d065fc640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d0015b3b0 con 0x7f3d00159700 2023-10-26T01:32:29.114 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:29.110+0000 7f3d065fc640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d0015aa70 con 0x7f3d000b0bf0 2023-10-26T01:32:29.115 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:29.110+0000 7f3cfffff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3d00069f90 0x7f3d0014fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:29.115 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:29.110+0000 7f3cfffff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3d00069f90 0x7f3d0014fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:33394/0 (socket says 172.21.15.169:33394) 2023-10-26T01:32:29.117 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:29.110+0000 7f3cfffff640 1 -- 172.21.15.169:0/1014650129 learned_addr learned my addr 172.21.15.169:0/1014650129 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:32:29.119 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:29.111+0000 7f3cfffff640 1 -- 172.21.15.169:0/1014650129 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3d000b0bf0 msgr2=0x7f3d000b11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:29.119 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:29.111+0000 7f3cff7fe640 1 --2- 172.21.15.169:0/1014650129 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3d000b0bf0 0x7f3d000b11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:29.119 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:29.111+0000 7f3d04b99640 1 --2- 172.21.15.169:0/1014650129 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3d00159700 0x7f3d000ca8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:29.119 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:29.111+0000 7f3cfffff640 1 --2- 172.21.15.169:0/1014650129 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3d000b0bf0 0x7f3d000b11c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:29.119 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:29.111+0000 7f3cfffff640 1 -- 172.21.15.169:0/1014650129 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3d00159700 msgr2=0x7f3d000ca8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:29.120 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:29.111+0000 7f3cfffff640 1 --2- 172.21.15.169:0/1014650129 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3d00159700 0x7f3d000ca8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:29.120 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:29.111+0000 7f3cfffff640 1 -- 172.21.15.169:0/1014650129 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ce8009560 con 0x7f3d00069f90 2023-10-26T01:32:29.120 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:29.111+0000 7f3cfffff640 1 --2- 172.21.15.169:0/1014650129 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3d00069f90 0x7f3d0014fbb0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f3ce8036040 tx=0x7f3ce802dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:29.120 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:29.112+0000 7f3cfcff9640 1 -- 172.21.15.169:0/1014650129 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ce8005650 con 0x7f3d00069f90 2023-10-26T01:32:29.120 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:29.112+0000 7f3cfcff9640 1 -- 172.21.15.169:0/1014650129 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3ce8003800 con 0x7f3d00069f90 2023-10-26T01:32:29.121 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:29.112+0000 7f3d065fc640 1 -- 172.21.15.169:0/1014650129 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d0015ad50 con 0x7f3d00069f90 2023-10-26T01:32:29.121 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:29.112+0000 7f3cfcff9640 1 -- 172.21.15.169:0/1014650129 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ce8003bb0 con 0x7f3d00069f90 2023-10-26T01:32:29.121 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:29.112+0000 7f3d065fc640 1 -- 172.21.15.169:0/1014650129 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d0015b600 con 0x7f3d00069f90 2023-10-26T01:32:29.121 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:29.113+0000 7f3d065fc640 1 -- 172.21.15.169:0/1014650129 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d00066af0 con 0x7f3d00069f90 2023-10-26T01:32:29.121 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:29.114+0000 7f3cfcff9640 1 -- 172.21.15.169:0/1014650129 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f3ce8003d10 con 0x7f3d00069f90 2023-10-26T01:32:29.122 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:29.115+0000 7f3cfcff9640 1 --2- 172.21.15.169:0/1014650129 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3ce006ae70 0x7f3ce006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:29.122 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:29.115+0000 7f3cfcff9640 1 -- 172.21.15.169:0/1014650129 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f3ce80abe00 con 0x7f3d00069f90 2023-10-26T01:32:29.122 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:29.116+0000 7f3cff7fe640 1 --2- 172.21.15.169:0/1014650129 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3ce006ae70 0x7f3ce006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:29.123 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:29.117+0000 7f3cff7fe640 1 --2- 172.21.15.169:0/1014650129 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3ce006ae70 0x7f3ce006d330 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3d00065a60 tx=0x7f3cf0008230 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:29.123 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:29.119+0000 7f3cfcff9640 1 -- 172.21.15.169:0/1014650129 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3ce8077240 con 0x7f3d00069f90 2023-10-26T01:32:29.312 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:29.310+0000 7f3d065fc640 1 -- 172.21.15.169:0/1014650129 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3d0015b7f0 con 0x7f3d00069f90 2023-10-26T01:32:29.313 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:29.311+0000 7f3cfcff9640 1 -- 172.21.15.169:0/1014650129 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3ce8043020 con 0x7f3d00069f90 2023-10-26T01:32:29.314 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 1 2023-10-26T01:32:29.316 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:29.313+0000 7f3d065fc640 1 -- 172.21.15.169:0/1014650129 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3ce006ae70 msgr2=0x7f3ce006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:29.316 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:29.313+0000 7f3d065fc640 1 --2- 172.21.15.169:0/1014650129 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3ce006ae70 0x7f3ce006d330 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3d00065a60 tx=0x7f3cf0008230 comp rx=0 tx=0).stop 2023-10-26T01:32:29.317 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:29.313+0000 7f3d065fc640 1 -- 172.21.15.169:0/1014650129 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3d00069f90 msgr2=0x7f3d0014fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:29.317 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:29.313+0000 7f3d065fc640 1 --2- 172.21.15.169:0/1014650129 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3d00069f90 0x7f3d0014fbb0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f3ce8036040 tx=0x7f3ce802dc30 comp rx=0 tx=0).stop 2023-10-26T01:32:29.317 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:29.313+0000 7f3d065fc640 1 -- 172.21.15.169:0/1014650129 shutdown_connections 2023-10-26T01:32:29.317 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:29.313+0000 7f3d065fc640 1 --2- 172.21.15.169:0/1014650129 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3ce006ae70 0x7f3ce006d330 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:29.317 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:29.313+0000 7f3d065fc640 1 --2- 172.21.15.169:0/1014650129 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3d00159700 0x7f3d000ca8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:29.318 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:29.313+0000 7f3d065fc640 1 --2- 172.21.15.169:0/1014650129 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3d000b0bf0 0x7f3d000b11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:29.318 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:29.313+0000 7f3d065fc640 1 --2- 172.21.15.169:0/1014650129 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3d00069f90 0x7f3d0014fbb0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:29.318 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:29.313+0000 7f3d065fc640 1 -- 172.21.15.169:0/1014650129 >> 172.21.15.169:0/1014650129 conn(0x7f3d000b00d0 msgr2=0x7f3d000c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:29.318 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:29.313+0000 7f3d065fc640 1 -- 172.21.15.169:0/1014650129 shutdown_connections 2023-10-26T01:32:29.318 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:29.313+0000 7f3d065fc640 1 -- 172.21.15.169:0/1014650129 wait complete. 2023-10-26T01:32:29.336 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestClusterFull.test_full_different_file' 2023-10-26T01:32:29.556 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:29.553+0000 7f971f1e2640 1 -- 172.21.15.169:0/3050012920 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f971806ce20 msgr2=0x7f971814e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:29.557 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:29.553+0000 7f971f1e2640 1 --2- 172.21.15.169:0/3050012920 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f971806ce20 0x7f971814e790 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f970c009860 tx=0x7f970c02bcb0 comp rx=0 tx=0).stop 2023-10-26T01:32:29.557 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:29.553+0000 7f971f1e2640 1 -- 172.21.15.169:0/3050012920 shutdown_connections 2023-10-26T01:32:29.557 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:29.553+0000 7f971f1e2640 1 --2- 172.21.15.169:0/3050012920 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9718159e10 0x7f971815a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:29.557 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:29.553+0000 7f971f1e2640 1 --2- 172.21.15.169:0/3050012920 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f971806ce20 0x7f971814e790 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:29.558 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:29.553+0000 7f971f1e2640 1 -- 172.21.15.169:0/3050012920 >> 172.21.15.169:0/3050012920 conn(0x7f9718069930 msgr2=0x7f9718069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:29.558 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:29.553+0000 7f971f1e2640 1 -- 172.21.15.169:0/3050012920 shutdown_connections 2023-10-26T01:32:29.558 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:29.553+0000 7f971f1e2640 1 -- 172.21.15.169:0/3050012920 wait complete. 2023-10-26T01:32:29.560 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:29.555+0000 7f971f1e2640 1 Processor -- start 2023-10-26T01:32:29.560 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:29.555+0000 7f971f1e2640 1 -- start start 2023-10-26T01:32:29.561 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:29.556+0000 7f971f1e2640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f971806ce20 0x7f97181508f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:29.561 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:29.556+0000 7f971f1e2640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9718158850 0x7f9718071730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:29.561 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:29.556+0000 7f971f1e2640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9718159e10 0x7f97181f4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:29.561 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:29.556+0000 7f971f1e2640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97180b1a50 con 0x7f971806ce20 2023-10-26T01:32:29.561 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:29.556+0000 7f971f1e2640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f971806f2f0 con 0x7f9718158850 2023-10-26T01:32:29.562 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:29.556+0000 7f971f1e2640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9718151510 con 0x7f9718159e10 2023-10-26T01:32:29.562 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:29.556+0000 7f971cf7e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f971806ce20 0x7f97181508f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:29.562 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:29.556+0000 7f971cf7e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f971806ce20 0x7f97181508f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:33412/0 (socket says 172.21.15.169:33412) 2023-10-26T01:32:29.562 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:29.556+0000 7f971cf7e640 1 -- 172.21.15.169:0/3300982140 learned_addr learned my addr 172.21.15.169:0/3300982140 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:32:29.562 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:29.557+0000 7f971d77f640 1 --2- 172.21.15.169:0/3300982140 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9718159e10 0x7f97181f4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:29.563 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:29.557+0000 7f9717fff640 1 --2- 172.21.15.169:0/3300982140 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9718158850 0x7f9718071730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:29.563 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:29.557+0000 7f971cf7e640 1 -- 172.21.15.169:0/3300982140 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9718159e10 msgr2=0x7f97181f4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:29.563 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:29.557+0000 7f971cf7e640 1 --2- 172.21.15.169:0/3300982140 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9718159e10 0x7f97181f4c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:29.563 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:29.557+0000 7f971cf7e640 1 -- 172.21.15.169:0/3300982140 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9718158850 msgr2=0x7f9718071730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:29.563 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:29.557+0000 7f971cf7e640 1 --2- 172.21.15.169:0/3300982140 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9718158850 0x7f9718071730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:29.564 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:29.557+0000 7f971cf7e640 1 -- 172.21.15.169:0/3300982140 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f970c009490 con 0x7f971806ce20 2023-10-26T01:32:29.564 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:29.557+0000 7f971cf7e640 1 --2- 172.21.15.169:0/3300982140 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f971806ce20 0x7f97181508f0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f970c009780 tx=0x7f970c0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:29.564 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:29.557+0000 7f97157fa640 1 -- 172.21.15.169:0/3300982140 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f970c003ea0 con 0x7f971806ce20 2023-10-26T01:32:29.564 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:29.557+0000 7f97157fa640 1 -- 172.21.15.169:0/3300982140 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f970c034440 con 0x7f971806ce20 2023-10-26T01:32:29.564 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:29.557+0000 7f97157fa640 1 -- 172.21.15.169:0/3300982140 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f970c034850 con 0x7f971806ce20 2023-10-26T01:32:29.565 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:29.557+0000 7f971f1e2640 1 -- 172.21.15.169:0/3300982140 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97180712a0 con 0x7f971806ce20 2023-10-26T01:32:29.565 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:29.558+0000 7f971f1e2640 1 -- 172.21.15.169:0/3300982140 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f971806f010 con 0x7f971806ce20 2023-10-26T01:32:29.566 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:29.559+0000 7f971f1e2640 1 -- 172.21.15.169:0/3300982140 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f971804db10 con 0x7f971806ce20 2023-10-26T01:32:29.566 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:29.560+0000 7f97157fa640 1 -- 172.21.15.169:0/3300982140 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f970c03b040 con 0x7f971806ce20 2023-10-26T01:32:29.566 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:29.561+0000 7f97157fa640 1 --2- 172.21.15.169:0/3300982140 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f970406ae40 0x7f970406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:29.566 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:29.561+0000 7f97157fa640 1 -- 172.21.15.169:0/3300982140 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f970c0ab7a0 con 0x7f971806ce20 2023-10-26T01:32:29.567 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:29.562+0000 7f9717fff640 1 --2- 172.21.15.169:0/3300982140 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f970406ae40 0x7f970406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:29.567 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:29.563+0000 7f9717fff640 1 --2- 172.21.15.169:0/3300982140 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f970406ae40 0x7f970406d300 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9708007fd0 tx=0x7f9708009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:29.568 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:29.566+0000 7f97157fa640 1 -- 172.21.15.169:0/3300982140 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f970c076b30 con 0x7f971806ce20 2023-10-26T01:32:29.826 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:29.823+0000 7f971f1e2640 1 -- 172.21.15.169:0/3300982140 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_different_file"]} v 0) v1 -- 0x7f97180aca70 con 0x7f971806ce20 2023-10-26T01:32:30.420 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:30.417+0000 7f97157fa640 1 -- 172.21.15.169:0/3300982140 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_different_file"]}]=0 v118) v1 ==== 145+0+0 (secure 0 0 0) 0x7f970c0410d0 con 0x7f971806ce20 2023-10-26T01:32:30.424 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:30.421+0000 7f971f1e2640 1 -- 172.21.15.169:0/3300982140 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f970406ae40 msgr2=0x7f970406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:30.424 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:30.421+0000 7f971f1e2640 1 --2- 172.21.15.169:0/3300982140 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f970406ae40 0x7f970406d300 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9708007fd0 tx=0x7f9708009040 comp rx=0 tx=0).stop 2023-10-26T01:32:30.424 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:30.421+0000 7f971f1e2640 1 -- 172.21.15.169:0/3300982140 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f971806ce20 msgr2=0x7f97181508f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:30.425 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:30.421+0000 7f971f1e2640 1 --2- 172.21.15.169:0/3300982140 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f971806ce20 0x7f97181508f0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f970c009780 tx=0x7f970c0038a0 comp rx=0 tx=0).stop 2023-10-26T01:32:30.425 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:30.421+0000 7f971f1e2640 1 -- 172.21.15.169:0/3300982140 shutdown_connections 2023-10-26T01:32:30.425 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:30.421+0000 7f971f1e2640 1 --2- 172.21.15.169:0/3300982140 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f970406ae40 0x7f970406d300 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:30.425 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:30.421+0000 7f971f1e2640 1 --2- 172.21.15.169:0/3300982140 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9718159e10 0x7f97181f4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:30.425 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:30.421+0000 7f971f1e2640 1 --2- 172.21.15.169:0/3300982140 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9718158850 0x7f9718071730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:30.426 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:30.421+0000 7f971f1e2640 1 --2- 172.21.15.169:0/3300982140 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f971806ce20 0x7f97181508f0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:30.426 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:30.421+0000 7f971f1e2640 1 -- 172.21.15.169:0/3300982140 >> 172.21.15.169:0/3300982140 conn(0x7f9718069930 msgr2=0x7f971814d910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:30.426 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:30.421+0000 7f971f1e2640 1 -- 172.21.15.169:0/3300982140 shutdown_connections 2023-10-26T01:32:30.426 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:30.421+0000 7f971f1e2640 1 -- 172.21.15.169:0/3300982140 wait complete. 2023-10-26T01:32:30.446 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_objectstore 2023-10-26T01:32:30.668 INFO:teuthology.orchestra.run.smithi169.stdout:{"osd_objectstore":"memstore"} 2023-10-26T01:32:30.668 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_objectstore": "memstore" } 2023-10-26T01:32:30.669 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2023-10-26T01:32:30.890 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:30.886+0000 7f7910c2e640 1 -- 172.21.15.169:0/1780243673 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f790c0cab80 msgr2=0x7f790c06bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:30.890 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:30.886+0000 7f7910c2e640 1 --2- 172.21.15.169:0/1780243673 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f790c0cab80 0x7f790c06bf10 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f78f4005fd0 tx=0x7f78f402bd10 comp rx=0 tx=0).stop 2023-10-26T01:32:30.890 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:30.887+0000 7f7910c2e640 1 -- 172.21.15.169:0/1780243673 shutdown_connections 2023-10-26T01:32:30.890 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:30.887+0000 7f7910c2e640 1 --2- 172.21.15.169:0/1780243673 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f790c0cab80 0x7f790c06bf10 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:30.891 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:30.887+0000 7f7910c2e640 1 --2- 172.21.15.169:0/1780243673 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f790c0ca7a0 0x7f790c0717a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:30.891 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:30.887+0000 7f7910c2e640 1 -- 172.21.15.169:0/1780243673 >> 172.21.15.169:0/1780243673 conn(0x7f790c0af0c0 msgr2=0x7f790c0af4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:30.891 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:30.887+0000 7f7910c2e640 1 -- 172.21.15.169:0/1780243673 shutdown_connections 2023-10-26T01:32:30.891 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:30.887+0000 7f7910c2e640 1 -- 172.21.15.169:0/1780243673 wait complete. 2023-10-26T01:32:30.892 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:30.889+0000 7f7910c2e640 1 Processor -- start 2023-10-26T01:32:30.892 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:30.889+0000 7f7910c2e640 1 -- start start 2023-10-26T01:32:30.894 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:30.890+0000 7f7910c2e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f790c0ca7a0 0x7f790c1e7db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:30.894 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:30.890+0000 7f7910c2e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f790c0cab80 0x7f790c1e82f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:30.894 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:30.890+0000 7f7910c2e640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f790c15a2e0 0x7f790c0aa1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:30.894 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:30.890+0000 7f7910c2e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f790c14e690 con 0x7f790c0cab80 2023-10-26T01:32:30.894 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:30.890+0000 7f7910c2e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f790c070ce0 con 0x7f790c0ca7a0 2023-10-26T01:32:30.895 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:30.890+0000 7f7910c2e640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f790c070a00 con 0x7f790c15a2e0 2023-10-26T01:32:30.895 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:30.890+0000 7f790affd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f790c0cab80 0x7f790c1e82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:30.895 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:30.891+0000 7f790affd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f790c0cab80 0x7f790c1e82f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:33430/0 (socket says 172.21.15.169:33430) 2023-10-26T01:32:30.895 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:30.891+0000 7f790affd640 1 -- 172.21.15.169:0/1790136191 learned_addr learned my addr 172.21.15.169:0/1790136191 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:32:30.895 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:30.891+0000 7f790affd640 1 -- 172.21.15.169:0/1790136191 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f790c15a2e0 msgr2=0x7f790c0aa1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:30.895 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:30.891+0000 7f790affd640 1 --2- 172.21.15.169:0/1790136191 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f790c15a2e0 0x7f790c0aa1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:30.896 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:30.891+0000 7f790affd640 1 -- 172.21.15.169:0/1790136191 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f790c0ca7a0 msgr2=0x7f790c1e7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:30.896 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:30.891+0000 7f790affd640 1 --2- 172.21.15.169:0/1790136191 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f790c0ca7a0 0x7f790c1e7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:30.896 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:30.891+0000 7f790affd640 1 -- 172.21.15.169:0/1790136191 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78f4009560 con 0x7f790c0cab80 2023-10-26T01:32:30.896 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:30.891+0000 7f790affd640 1 --2- 172.21.15.169:0/1790136191 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f790c0cab80 0x7f790c1e82f0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f78f4009850 tx=0x7f78f402da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:30.897 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:30.892+0000 7f7903fff640 1 -- 172.21.15.169:0/1790136191 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78f402de90 con 0x7f790c0cab80 2023-10-26T01:32:30.897 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:30.892+0000 7f7903fff640 1 -- 172.21.15.169:0/1790136191 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f78f40036e0 con 0x7f790c0cab80 2023-10-26T01:32:30.898 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:30.892+0000 7f7903fff640 1 -- 172.21.15.169:0/1790136191 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78f4003a90 con 0x7f790c0cab80 2023-10-26T01:32:30.898 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:30.892+0000 7f7910c2e640 1 -- 172.21.15.169:0/1790136191 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f790c14dbb0 con 0x7f790c0cab80 2023-10-26T01:32:30.898 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:30.892+0000 7f7910c2e640 1 -- 172.21.15.169:0/1790136191 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f790c14e170 con 0x7f790c0cab80 2023-10-26T01:32:30.898 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:30.893+0000 7f7910c2e640 1 -- 172.21.15.169:0/1790136191 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f790c14d9f0 con 0x7f790c0cab80 2023-10-26T01:32:30.898 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:30.894+0000 7f7903fff640 1 -- 172.21.15.169:0/1790136191 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f78f4003bf0 con 0x7f790c0cab80 2023-10-26T01:32:30.899 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:30.895+0000 7f7903fff640 1 --2- 172.21.15.169:0/1790136191 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f78ec06ae70 0x7f78ec06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:30.899 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:30.895+0000 7f7903fff640 1 -- 172.21.15.169:0/1790136191 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f78f40abe50 con 0x7f790c0cab80 2023-10-26T01:32:30.899 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:30.895+0000 7f790b7fe640 1 --2- 172.21.15.169:0/1790136191 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f78ec06ae70 0x7f78ec06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:30.900 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:30.897+0000 7f790b7fe640 1 --2- 172.21.15.169:0/1790136191 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f78ec06ae70 0x7f78ec06d330 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f78fc00ab90 tx=0x7f78fc009250 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:30.902 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:30.899+0000 7f7903fff640 1 -- 172.21.15.169:0/1790136191 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f78f40773b0 con 0x7f790c0cab80 2023-10-26T01:32:31.091 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:31.088+0000 7f7910c2e640 1 -- 172.21.15.169:0/1790136191 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "config dump"} v 0) v1 -- 0x7f790c09b540 con 0x7f790c0cab80 2023-10-26T01:32:31.092 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:31.089+0000 7f7903fff640 1 -- 172.21.15.169:0/1790136191 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v20) v1 ==== 59+0+1116 (secure 0 0 0) 0x7f78f4043080 con 0x7f790c0cab80 2023-10-26T01:32:31.093 INFO:teuthology.orchestra.run.smithi169.stdout:WHO MASK LEVEL OPTION VALUE RO 2023-10-26T01:32:31.093 INFO:teuthology.orchestra.run.smithi169.stdout:mon advanced mon_op_complaint_time 120 2023-10-26T01:32:31.093 INFO:teuthology.orchestra.run.smithi169.stdout:osd advanced osd_op_complaint_time 180.000000 2023-10-26T01:32:31.093 INFO:teuthology.orchestra.run.smithi169.stdout:mds advanced debug_mds 20/20 2023-10-26T01:32:31.093 INFO:teuthology.orchestra.run.smithi169.stdout:mds advanced debug_mds_balancer 20/20 2023-10-26T01:32:31.094 INFO:teuthology.orchestra.run.smithi169.stdout:mds advanced debug_ms 1/1 2023-10-26T01:32:31.094 INFO:teuthology.orchestra.run.smithi169.stdout:mds dev mds_debug_frag true 2023-10-26T01:32:31.094 INFO:teuthology.orchestra.run.smithi169.stdout:mds dev mds_debug_scatterstat true 2023-10-26T01:32:31.094 INFO:teuthology.orchestra.run.smithi169.stdout:mds advanced mds_op_complaint_time 180.000000 2023-10-26T01:32:31.094 INFO:teuthology.orchestra.run.smithi169.stdout:mds dev mds_verify_scatter true 2023-10-26T01:32:31.095 INFO:teuthology.orchestra.run.smithi169.stdout:mds advanced osd_op_complaint_time 180.000000 2023-10-26T01:32:31.095 INFO:teuthology.orchestra.run.smithi169.stdout:mds advanced rados_mon_op_timeout 900 2023-10-26T01:32:31.095 INFO:teuthology.orchestra.run.smithi169.stdout:mds advanced rados_osd_op_timeout 900 2023-10-26T01:32:31.095 INFO:teuthology.orchestra.run.smithi169.stdout:client advanced client_mount_timeout 600 2023-10-26T01:32:31.095 INFO:teuthology.orchestra.run.smithi169.stdout:client advanced debug_client 20/20 2023-10-26T01:32:31.095 INFO:teuthology.orchestra.run.smithi169.stdout:client advanced debug_ms 1/1 2023-10-26T01:32:31.096 INFO:teuthology.orchestra.run.smithi169.stdout:client advanced rados_mon_op_timeout 900 2023-10-26T01:32:31.096 INFO:teuthology.orchestra.run.smithi169.stdout:client advanced rados_osd_op_timeout 900 2023-10-26T01:32:31.096 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:31.093+0000 7f7910c2e640 1 -- 172.21.15.169:0/1790136191 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f78ec06ae70 msgr2=0x7f78ec06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:31.097 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:31.093+0000 7f7910c2e640 1 --2- 172.21.15.169:0/1790136191 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f78ec06ae70 0x7f78ec06d330 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f78fc00ab90 tx=0x7f78fc009250 comp rx=0 tx=0).stop 2023-10-26T01:32:31.097 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:31.093+0000 7f7910c2e640 1 -- 172.21.15.169:0/1790136191 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f790c0cab80 msgr2=0x7f790c1e82f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:31.097 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:31.093+0000 7f7910c2e640 1 --2- 172.21.15.169:0/1790136191 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f790c0cab80 0x7f790c1e82f0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f78f4009850 tx=0x7f78f402da90 comp rx=0 tx=0).stop 2023-10-26T01:32:31.097 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:31.093+0000 7f7910c2e640 1 -- 172.21.15.169:0/1790136191 shutdown_connections 2023-10-26T01:32:31.097 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:31.093+0000 7f7910c2e640 1 --2- 172.21.15.169:0/1790136191 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f78ec06ae70 0x7f78ec06d330 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:31.098 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:31.093+0000 7f7910c2e640 1 --2- 172.21.15.169:0/1790136191 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f790c0cab80 0x7f790c1e82f0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:31.098 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:31.093+0000 7f7910c2e640 1 --2- 172.21.15.169:0/1790136191 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f790c15a2e0 0x7f790c0aa1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:31.098 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:31.093+0000 7f7910c2e640 1 --2- 172.21.15.169:0/1790136191 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f790c0ca7a0 0x7f790c1e7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:31.098 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:31.093+0000 7f7910c2e640 1 -- 172.21.15.169:0/1790136191 >> 172.21.15.169:0/1790136191 conn(0x7f790c0af0c0 msgr2=0x7f790c06c5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:31.098 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:31.093+0000 7f7910c2e640 1 -- 172.21.15.169:0/1790136191 shutdown_connections 2023-10-26T01:32:31.099 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:31.093+0000 7f7910c2e640 1 -- 172.21.15.169:0/1790136191 wait complete. 2023-10-26T01:32:31.114 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-26T01:32:31.332 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:31.329+0000 7f6363444640 1 -- 172.21.15.169:0/2197937248 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f635c14ddf0 msgr2=0x7f635c14e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:31.333 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:31.329+0000 7f6363444640 1 --2- 172.21.15.169:0/2197937248 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f635c14ddf0 0x7f635c14e280 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f635c06e390 tx=0x7f635402c7c0 comp rx=0 tx=0).stop 2023-10-26T01:32:31.333 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:31.329+0000 7f6363444640 1 -- 172.21.15.169:0/2197937248 shutdown_connections 2023-10-26T01:32:31.333 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:31.329+0000 7f6363444640 1 --2- 172.21.15.169:0/2197937248 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f635c14ddf0 0x7f635c14e280 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:31.333 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:31.329+0000 7f6363444640 1 --2- 172.21.15.169:0/2197937248 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f635c15b320 0x7f635c15b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:31.333 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:31.329+0000 7f6363444640 1 --2- 172.21.15.169:0/2197937248 >> v2:172.21.15.170:3300/0 conn(0x7f635c06c650 0x7f635c0aac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:31.334 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:31.329+0000 7f6363444640 1 -- 172.21.15.169:0/2197937248 >> 172.21.15.169:0/2197937248 conn(0x7f635c0aefe0 msgr2=0x7f635c0af3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:31.334 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:31.329+0000 7f6363444640 1 -- 172.21.15.169:0/2197937248 shutdown_connections 2023-10-26T01:32:31.334 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:31.329+0000 7f6363444640 1 -- 172.21.15.169:0/2197937248 wait complete. 2023-10-26T01:32:31.336 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:31.331+0000 7f6363444640 1 Processor -- start 2023-10-26T01:32:31.336 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:31.331+0000 7f6363444640 1 -- start start 2023-10-26T01:32:31.336 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:31.332+0000 7f6363444640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f635c06c650 0x7f635c14d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:31.336 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:31.332+0000 7f6363444640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f635c14ddf0 0x7f635c0c9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:31.337 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:31.332+0000 7f6363444640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f635c15b320 0x7f635c151090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:31.337 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:31.332+0000 7f6363444640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f635c155100 con 0x7f635c15b320 2023-10-26T01:32:31.337 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:31.332+0000 7f6363444640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f635c14dba0 con 0x7f635c06c650 2023-10-26T01:32:31.337 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:31.332+0000 7f6363444640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f635c153410 con 0x7f635c14ddf0 2023-10-26T01:32:31.337 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:31.333+0000 7f63619e1640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f635c15b320 0x7f635c151090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:31.338 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:31.333+0000 7f63619e1640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f635c15b320 0x7f635c151090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:33448/0 (socket says 172.21.15.169:33448) 2023-10-26T01:32:31.338 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:31.333+0000 7f63619e1640 1 -- 172.21.15.169:0/2422031076 learned_addr learned my addr 172.21.15.169:0/2422031076 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:32:31.338 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:31.333+0000 7f63619e1640 1 -- 172.21.15.169:0/2422031076 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f635c14ddf0 msgr2=0x7f635c0c9cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:31.338 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:31.333+0000 7f63611e0640 1 --2- 172.21.15.169:0/2422031076 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f635c06c650 0x7f635c14d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:31.338 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:31.333+0000 7f63609df640 1 --2- 172.21.15.169:0/2422031076 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f635c14ddf0 0x7f635c0c9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:31.339 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:31.333+0000 7f63619e1640 1 --2- 172.21.15.169:0/2422031076 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f635c14ddf0 0x7f635c0c9cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:31.339 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:31.333+0000 7f63619e1640 1 -- 172.21.15.169:0/2422031076 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f635c06c650 msgr2=0x7f635c14d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:31.339 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:31.333+0000 7f63619e1640 1 --2- 172.21.15.169:0/2422031076 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f635c06c650 0x7f635c14d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:31.339 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:31.333+0000 7f63619e1640 1 -- 172.21.15.169:0/2422031076 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f635400c4a0 con 0x7f635c15b320 2023-10-26T01:32:31.339 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:31.333+0000 7f63611e0640 1 --2- 172.21.15.169:0/2422031076 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f635c06c650 0x7f635c14d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:32:31.339 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:31.334+0000 7f63619e1640 1 --2- 172.21.15.169:0/2422031076 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f635c15b320 0x7f635c151090 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f635400c790 tx=0x7f6354032e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:31.340 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:31.334+0000 7f6351ffb640 1 -- 172.21.15.169:0/2422031076 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6354032520 con 0x7f635c15b320 2023-10-26T01:32:31.341 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:31.334+0000 7f6351ffb640 1 -- 172.21.15.169:0/2422031076 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6354009570 con 0x7f635c15b320 2023-10-26T01:32:31.341 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:31.334+0000 7f6351ffb640 1 -- 172.21.15.169:0/2422031076 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6354009980 con 0x7f635c15b320 2023-10-26T01:32:31.341 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:31.334+0000 7f6363444640 1 -- 172.21.15.169:0/2422031076 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f635c155350 con 0x7f635c15b320 2023-10-26T01:32:31.341 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:31.334+0000 7f6363444640 1 -- 172.21.15.169:0/2422031076 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f635c153990 con 0x7f635c15b320 2023-10-26T01:32:31.341 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:31.335+0000 7f6363444640 1 -- 172.21.15.169:0/2422031076 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f635c066af0 con 0x7f635c15b320 2023-10-26T01:32:31.342 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:31.336+0000 7f6351ffb640 1 -- 172.21.15.169:0/2422031076 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f635403b040 con 0x7f635c15b320 2023-10-26T01:32:31.342 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:31.337+0000 7f6351ffb640 1 --2- 172.21.15.169:0/2422031076 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f634006ae40 0x7f634006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:31.342 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:31.337+0000 7f6351ffb640 1 -- 172.21.15.169:0/2422031076 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f63540ab980 con 0x7f635c15b320 2023-10-26T01:32:31.342 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:31.337+0000 7f63611e0640 1 --2- 172.21.15.169:0/2422031076 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f634006ae40 0x7f634006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:31.343 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:31.338+0000 7f63611e0640 1 --2- 172.21.15.169:0/2422031076 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f634006ae40 0x7f634006d300 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6358006fd0 tx=0x7f6358008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:31.343 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:31.341+0000 7f6351ffb640 1 -- 172.21.15.169:0/2422031076 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6354076c90 con 0x7f635c15b320 2023-10-26T01:32:31.522 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:31.519+0000 7f6363444640 1 -- 172.21.15.169:0/2422031076 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f635c04da70 con 0x7f635c15b320 2023-10-26T01:32:31.527 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:31.524+0000 7f6351ffb640 1 -- 172.21.15.169:0/2422031076 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v21)=0 v21) v1 ==== 122+0+0 (secure 0 0 0) 0x7f635407a2b0 con 0x7f635c15b320 2023-10-26T01:32:31.530 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:31.527+0000 7f6363444640 1 -- 172.21.15.169:0/2422031076 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f634006ae40 msgr2=0x7f634006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:31.530 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:31.527+0000 7f6363444640 1 --2- 172.21.15.169:0/2422031076 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f634006ae40 0x7f634006d300 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6358006fd0 tx=0x7f6358008040 comp rx=0 tx=0).stop 2023-10-26T01:32:31.530 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:31.527+0000 7f6363444640 1 -- 172.21.15.169:0/2422031076 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f635c15b320 msgr2=0x7f635c151090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:31.530 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:31.527+0000 7f6363444640 1 --2- 172.21.15.169:0/2422031076 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f635c15b320 0x7f635c151090 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f635400c790 tx=0x7f6354032e80 comp rx=0 tx=0).stop 2023-10-26T01:32:31.531 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:31.527+0000 7f6363444640 1 -- 172.21.15.169:0/2422031076 shutdown_connections 2023-10-26T01:32:31.531 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:31.527+0000 7f6363444640 1 --2- 172.21.15.169:0/2422031076 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f634006ae40 0x7f634006d300 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:31.531 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:31.527+0000 7f6363444640 1 --2- 172.21.15.169:0/2422031076 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f635c15b320 0x7f635c151090 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:31.531 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:31.527+0000 7f6363444640 1 --2- 172.21.15.169:0/2422031076 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f635c14ddf0 0x7f635c0c9cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:31.531 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:31.527+0000 7f6363444640 1 --2- 172.21.15.169:0/2422031076 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f635c06c650 0x7f635c14d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:31.531 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:31.527+0000 7f6363444640 1 -- 172.21.15.169:0/2422031076 >> 172.21.15.169:0/2422031076 conn(0x7f635c0aefe0 msgr2=0x7f635c1559c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:31.532 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:31.527+0000 7f6363444640 1 -- 172.21.15.169:0/2422031076 shutdown_connections 2023-10-26T01:32:31.532 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:31.527+0000 7f6363444640 1 -- 172.21.15.169:0/2422031076 wait complete. 2023-10-26T01:32:31.547 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-26T01:32:31.548 DEBUG:teuthology.orchestra.run.smithi169:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-26T01:32:31.644 DEBUG:teuthology.orchestra.run.smithi170:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-26T01:32:31.768 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-26T01:32:31.768 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:32:31.769 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:32:31.826 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054301,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-26T01:32:31.827 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-26T01:32:31.827 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:32:31.887 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054303,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-26T01:32:31.888 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:32:31.888 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:32:31.947 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054301,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-26T01:32:31.947 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-26T01:32:31.948 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:32:32.009 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054303,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-26T01:32:32.009 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-26T01:32:32.010 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T01:32:32.235 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:32.232+0000 7f28e759e640 1 -- 172.21.15.169:0/1599792112 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f28e80c9e00 msgr2=0x7f28e80cb390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:32.235 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:32.232+0000 7f28e759e640 1 --2- 172.21.15.169:0/1599792112 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f28e80c9e00 0x7f28e80cb390 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f28e0005fd0 tx=0x7f28e002bcf0 comp rx=0 tx=0).stop 2023-10-26T01:32:32.236 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:32.232+0000 7f28e759e640 1 -- 172.21.15.169:0/1599792112 shutdown_connections 2023-10-26T01:32:32.236 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:32.232+0000 7f28e759e640 1 --2- 172.21.15.169:0/1599792112 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f28e80c9e00 0x7f28e80cb390 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:32.236 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:32.232+0000 7f28e759e640 1 --2- 172.21.15.169:0/1599792112 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f28e806cae0 0x7f28e80c7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:32.236 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:32.232+0000 7f28e759e640 1 -- 172.21.15.169:0/1599792112 >> 172.21.15.169:0/1599792112 conn(0x7f28e80aedd0 msgr2=0x7f28e80af1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:32.237 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:32.232+0000 7f28e759e640 1 -- 172.21.15.169:0/1599792112 shutdown_connections 2023-10-26T01:32:32.237 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:32.232+0000 7f28e759e640 1 -- 172.21.15.169:0/1599792112 wait complete. 2023-10-26T01:32:32.238 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:32.234+0000 7f28e759e640 1 Processor -- start 2023-10-26T01:32:32.238 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:32.234+0000 7f28e759e640 1 -- start start 2023-10-26T01:32:32.239 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:32.235+0000 7f28e759e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f28e806a990 0x7f28e814c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:32.239 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:32.235+0000 7f28e759e640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f28e806cae0 0x7f28e809a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:32.239 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:32.236+0000 7f28e759e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f28e80c9e00 0x7f28e809a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:32.239 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:32.236+0000 7f28e759e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28e8151940 con 0x7f28e806a990 2023-10-26T01:32:32.240 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:32.236+0000 7f28e759e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28e8151c20 con 0x7f28e80c9e00 2023-10-26T01:32:32.240 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:32.236+0000 7f28e759e640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28e80996b0 con 0x7f28e806cae0 2023-10-26T01:32:32.240 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:32.236+0000 7f28e659c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f28e806a990 0x7f28e814c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:32.240 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:32.236+0000 7f28e659c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f28e806a990 0x7f28e814c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:33472/0 (socket says 172.21.15.169:33472) 2023-10-26T01:32:32.241 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:32.236+0000 7f28e659c640 1 -- 172.21.15.169:0/3027873556 learned_addr learned my addr 172.21.15.169:0/3027873556 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:32:32.242 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:32.236+0000 7f28e6d9d640 1 --2- 172.21.15.169:0/3027873556 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f28e80c9e00 0x7f28e809a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:32.243 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:32.236+0000 7f28ddd9b640 1 --2- 172.21.15.169:0/3027873556 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f28e806cae0 0x7f28e809a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:32.243 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:32.236+0000 7f28e659c640 1 -- 172.21.15.169:0/3027873556 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f28e806cae0 msgr2=0x7f28e809a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:32.243 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:32.236+0000 7f28e659c640 1 --2- 172.21.15.169:0/3027873556 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f28e806cae0 0x7f28e809a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:32.243 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:32.236+0000 7f28e659c640 1 -- 172.21.15.169:0/3027873556 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f28e80c9e00 msgr2=0x7f28e809a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:32.244 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:32.236+0000 7f28e659c640 1 --2- 172.21.15.169:0/3027873556 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f28e80c9e00 0x7f28e809a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:32.244 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:32.236+0000 7f28e659c640 1 -- 172.21.15.169:0/3027873556 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28e0009560 con 0x7f28e806a990 2023-10-26T01:32:32.244 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:32.236+0000 7f28ddd9b640 1 --2- 172.21.15.169:0/3027873556 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f28e806cae0 0x7f28e809a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:32:32.244 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:32.236+0000 7f28e6d9d640 1 --2- 172.21.15.169:0/3027873556 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f28e80c9e00 0x7f28e809a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:32:32.245 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:32.237+0000 7f28e659c640 1 --2- 172.21.15.169:0/3027873556 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f28e806a990 0x7f28e814c7b0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f28d800c9a0 tx=0x7f28d800ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:32.245 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:32.237+0000 7f28df7fe640 1 -- 172.21.15.169:0/3027873556 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28d8007bb0 con 0x7f28e806a990 2023-10-26T01:32:32.245 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:32.237+0000 7f28df7fe640 1 -- 172.21.15.169:0/3027873556 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f28d8007e20 con 0x7f28e806a990 2023-10-26T01:32:32.246 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:32.237+0000 7f28df7fe640 1 -- 172.21.15.169:0/3027873556 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28d8005280 con 0x7f28e806a990 2023-10-26T01:32:32.246 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:32.237+0000 7f28e759e640 1 -- 172.21.15.169:0/3027873556 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28e8099990 con 0x7f28e806a990 2023-10-26T01:32:32.246 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:32.237+0000 7f28e759e640 1 -- 172.21.15.169:0/3027873556 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28e814ccf0 con 0x7f28e806a990 2023-10-26T01:32:32.247 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:32.238+0000 7f28e759e640 1 -- 172.21.15.169:0/3027873556 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28e8073c10 con 0x7f28e806a990 2023-10-26T01:32:32.247 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:32.239+0000 7f28df7fe640 1 -- 172.21.15.169:0/3027873556 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f28d80053e0 con 0x7f28e806a990 2023-10-26T01:32:32.247 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:32.240+0000 7f28df7fe640 1 --2- 172.21.15.169:0/3027873556 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f28c806ae70 0x7f28c806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:32.247 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:32.240+0000 7f28df7fe640 1 -- 172.21.15.169:0/3027873556 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f28d80897e0 con 0x7f28e806a990 2023-10-26T01:32:32.248 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:32.241+0000 7f28ddd9b640 1 --2- 172.21.15.169:0/3027873556 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f28c806ae70 0x7f28c806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:32.248 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:32.242+0000 7f28ddd9b640 1 --2- 172.21.15.169:0/3027873556 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f28c806ae70 0x7f28c806d330 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f28e81f2ef0 tx=0x7f28d0009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:32.249 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:32.246+0000 7f28df7fe640 1 -- 172.21.15.169:0/3027873556 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f28d8054c20 con 0x7f28e806a990 2023-10-26T01:32:32.475 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:32.472+0000 7f28e759e640 1 -- 172.21.15.169:0/3027873556 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f28e806c790 con 0x7f28e806a990 2023-10-26T01:32:32.475 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:32.473+0000 7f28df7fe640 1 -- 172.21.15.169:0/3027873556 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+3699 (secure 0 0 0) 0x7f28d801d070 con 0x7f28e806a990 2023-10-26T01:32:32.476 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:32:32.476 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":17,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4640,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/61096113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":61096113},{"type":"v1","addr":"172.21.15.170:6836","nonce":61096113}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4762,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2825046244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2825046244},{"type":"v1","addr":"172.21.15.169:6833","nonce":2825046244}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4777,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/258866976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":258866976},{"type":"v1","addr":"172.21.15.170:6837","nonce":258866976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":5562,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/126019777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":126019777},{"type":"v1","addr":"172.21.15.169:6835","nonce":126019777}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17}],"filesystems":[]} 2023-10-26T01:32:32.476 INFO:teuthology.orchestra.run.smithi169.stderr:dumped fsmap epoch 17 2023-10-26T01:32:32.478 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:32.475+0000 7f28e759e640 1 -- 172.21.15.169:0/3027873556 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f28c806ae70 msgr2=0x7f28c806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:32.479 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:32.475+0000 7f28e759e640 1 --2- 172.21.15.169:0/3027873556 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f28c806ae70 0x7f28c806d330 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f28e81f2ef0 tx=0x7f28d0009040 comp rx=0 tx=0).stop 2023-10-26T01:32:32.479 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:32.475+0000 7f28e759e640 1 -- 172.21.15.169:0/3027873556 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f28e806a990 msgr2=0x7f28e814c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:32.479 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:32.475+0000 7f28e759e640 1 --2- 172.21.15.169:0/3027873556 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f28e806a990 0x7f28e814c7b0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f28d800c9a0 tx=0x7f28d800ce60 comp rx=0 tx=0).stop 2023-10-26T01:32:32.479 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:32.475+0000 7f28e759e640 1 -- 172.21.15.169:0/3027873556 shutdown_connections 2023-10-26T01:32:32.480 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:32.475+0000 7f28e759e640 1 --2- 172.21.15.169:0/3027873556 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f28c806ae70 0x7f28c806d330 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:32.480 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:32.475+0000 7f28e759e640 1 --2- 172.21.15.169:0/3027873556 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f28e80c9e00 0x7f28e809a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:32.480 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:32.475+0000 7f28e759e640 1 --2- 172.21.15.169:0/3027873556 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f28e806cae0 0x7f28e809a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:32.480 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:32.475+0000 7f28e759e640 1 --2- 172.21.15.169:0/3027873556 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f28e806a990 0x7f28e814c7b0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:32.480 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:32.475+0000 7f28e759e640 1 -- 172.21.15.169:0/3027873556 >> 172.21.15.169:0/3027873556 conn(0x7f28e80aedd0 msgr2=0x7f28e815b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:32.481 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:32.476+0000 7f28e759e640 1 -- 172.21.15.169:0/3027873556 shutdown_connections 2023-10-26T01:32:32.481 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:32.476+0000 7f28e759e640 1 -- 172.21.15.169:0/3027873556 wait complete. 2023-10-26T01:32:32.497 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-26T01:32:32.497 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-26T01:32:32.497 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-26T01:32:32.498 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T01:32:32.498 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-26T01:32:32.498 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-26T01:32:32.499 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-26T01:32:32.499 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T01:32:32.499 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-26T01:32:32.499 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-26T01:32:32.500 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-26T01:32:32.500 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T01:32:32.500 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-26T01:32:32.500 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-26T01:32:32.501 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-26T01:32:32.501 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T01:32:32.519 INFO:tasks.ceph.mds.c:Stopped 2023-10-26T01:32:32.519 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-26T01:32:32.522 INFO:tasks.ceph.mds.a:Stopped 2023-10-26T01:32:32.522 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-26T01:32:32.525 INFO:tasks.ceph.mds.d:Stopped 2023-10-26T01:32:32.526 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-26T01:32:32.527 INFO:tasks.ceph.mds.b:Stopped 2023-10-26T01:32:32.527 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-26T01:32:32.570 INFO:tasks.ceph.mds.c:Started 2023-10-26T01:32:32.571 DEBUG:teuthology.parallel:result is None 2023-10-26T01:32:32.573 INFO:tasks.ceph.mds.d:Started 2023-10-26T01:32:32.573 DEBUG:teuthology.parallel:result is None 2023-10-26T01:32:32.578 INFO:tasks.ceph.mds.a:Started 2023-10-26T01:32:32.578 DEBUG:teuthology.parallel:result is None 2023-10-26T01:32:32.580 INFO:tasks.ceph.mds.b:Started 2023-10-26T01:32:32.581 DEBUG:teuthology.parallel:result is None 2023-10-26T01:32:32.581 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-26T01:32:32.728 INFO:tasks.ceph.mds.c.smithi169.stdout:starting mds.c at 2023-10-26T01:32:32.731 INFO:tasks.ceph.mds.a.smithi169.stdout:starting mds.a at 2023-10-26T01:32:32.732 INFO:tasks.ceph.mds.b.smithi170.stdout:starting mds.b at 2023-10-26T01:32:32.732 INFO:tasks.ceph.mds.d.smithi170.stdout:starting mds.d at 2023-10-26T01:32:32.773 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:32.770+0000 7f558d27c640 1 -- 172.21.15.169:0/1542852675 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f55880adc70 msgr2=0x7f55880abc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:32.774 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:32.770+0000 7f558d27c640 1 --2- 172.21.15.169:0/1542852675 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f55880adc70 0x7f55880abc60 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5570009830 tx=0x7f557002bd50 comp rx=0 tx=0).stop 2023-10-26T01:32:32.774 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:32.771+0000 7f558d27c640 1 -- 172.21.15.169:0/1542852675 shutdown_connections 2023-10-26T01:32:32.774 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:32.771+0000 7f558d27c640 1 --2- 172.21.15.169:0/1542852675 >> v2:172.21.15.170:3300/0 conn(0x7f55880ac1a0 0x7f5588069510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:32.774 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:32.771+0000 7f558d27c640 1 --2- 172.21.15.169:0/1542852675 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f55880adc70 0x7f55880abc60 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:32.775 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:32.771+0000 7f558d27c640 1 -- 172.21.15.169:0/1542852675 >> 172.21.15.169:0/1542852675 conn(0x7f558806a620 msgr2=0x7f558806aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:32.775 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:32.771+0000 7f558d27c640 1 -- 172.21.15.169:0/1542852675 shutdown_connections 2023-10-26T01:32:32.775 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:32.771+0000 7f558d27c640 1 -- 172.21.15.169:0/1542852675 wait complete. 2023-10-26T01:32:32.776 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:32.772+0000 7f558d27c640 1 Processor -- start 2023-10-26T01:32:32.776 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:32.773+0000 7f558d27c640 1 -- start start 2023-10-26T01:32:32.776 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:32.773+0000 7f558d27c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55880ac1a0 0x7f55880aeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:32.776 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:32.773+0000 7f558d27c640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f55880adc70 0x7f5588148940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:32.777 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:32.773+0000 7f558d27c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f558815b990 0x7f5588148e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:32.777 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:32.773+0000 7f558d27c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55880c5740 con 0x7f55880ac1a0 2023-10-26T01:32:32.777 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:32.773+0000 7f558d27c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55880c65a0 con 0x7f558815b990 2023-10-26T01:32:32.777 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:32.773+0000 7f558d27c640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55880c5990 con 0x7f55880adc70 2023-10-26T01:32:32.779 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:32.774+0000 7f5586ffd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55880ac1a0 0x7f55880aeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:32.779 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:32.774+0000 7f5586ffd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55880ac1a0 0x7f55880aeaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:33512/0 (socket says 172.21.15.169:33512) 2023-10-26T01:32:32.779 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:32.774+0000 7f5586ffd640 1 -- 172.21.15.169:0/2751324578 learned_addr learned my addr 172.21.15.169:0/2751324578 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:32:32.779 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:32.774+0000 7f557e7fc640 1 --2- 172.21.15.169:0/2751324578 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f55880adc70 0x7f5588148940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:32.780 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:32.774+0000 7f55877fe640 1 --2- 172.21.15.169:0/2751324578 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f558815b990 0x7f5588148e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:32.780 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:32.774+0000 7f5586ffd640 1 -- 172.21.15.169:0/2751324578 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f55880adc70 msgr2=0x7f5588148940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:32.780 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:32.774+0000 7f5586ffd640 1 --2- 172.21.15.169:0/2751324578 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f55880adc70 0x7f5588148940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:32.780 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:32.774+0000 7f5586ffd640 1 -- 172.21.15.169:0/2751324578 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f558815b990 msgr2=0x7f5588148e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:32.780 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:32.774+0000 7f5586ffd640 1 --2- 172.21.15.169:0/2751324578 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f558815b990 0x7f5588148e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:32.781 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:32.774+0000 7f5586ffd640 1 -- 172.21.15.169:0/2751324578 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5570009490 con 0x7f55880ac1a0 2023-10-26T01:32:32.781 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:32.775+0000 7f5586ffd640 1 --2- 172.21.15.169:0/2751324578 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55880ac1a0 0x7f55880aeaa0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f557800cb80 tx=0x7f557800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:32.781 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:32.775+0000 7f557ffff640 1 -- 172.21.15.169:0/2751324578 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f557800ad40 con 0x7f55880ac1a0 2023-10-26T01:32:32.781 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:32.775+0000 7f557ffff640 1 -- 172.21.15.169:0/2751324578 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5578012420 con 0x7f55880ac1a0 2023-10-26T01:32:32.781 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:32.775+0000 7f557ffff640 1 -- 172.21.15.169:0/2751324578 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5578012830 con 0x7f55880ac1a0 2023-10-26T01:32:32.782 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:32.775+0000 7f558d27c640 1 -- 172.21.15.169:0/2751324578 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55880c5c70 con 0x7f55880ac1a0 2023-10-26T01:32:32.782 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:32.775+0000 7f558d27c640 1 -- 172.21.15.169:0/2751324578 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55880c6230 con 0x7f55880ac1a0 2023-10-26T01:32:32.782 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:32.776+0000 7f558d27c640 1 -- 172.21.15.169:0/2751324578 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5588066af0 con 0x7f55880ac1a0 2023-10-26T01:32:32.783 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:32.777+0000 7f557ffff640 1 -- 172.21.15.169:0/2751324578 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f557801a3b0 con 0x7f55880ac1a0 2023-10-26T01:32:32.783 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:32.778+0000 7f557ffff640 1 --2- 172.21.15.169:0/2751324578 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f556806ae40 0x7f556806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:32.783 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:32.778+0000 7f557ffff640 1 -- 172.21.15.169:0/2751324578 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f5578089a10 con 0x7f55880ac1a0 2023-10-26T01:32:32.783 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:32.779+0000 7f557e7fc640 1 --2- 172.21.15.169:0/2751324578 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f556806ae40 0x7f556806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:32.784 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:32.779+0000 7f557e7fc640 1 --2- 172.21.15.169:0/2751324578 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f556806ae40 0x7f556806d300 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5570036000 tx=0x7f5570004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:32.785 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:32.782+0000 7f557ffff640 1 -- 172.21.15.169:0/2751324578 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5578054e70 con 0x7f55880ac1a0 2023-10-26T01:32:32.992 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:32.990+0000 7f558d27c640 1 -- 172.21.15.169:0/2751324578 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f5588066e60 con 0x7f55880ac1a0 2023-10-26T01:32:33.938 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:33.936+0000 7f557ffff640 1 -- 172.21.15.169:0/2751324578 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v45) v1 ==== 97+0+0 (secure 0 0 0) 0x7f5578016080 con 0x7f55880ac1a0 2023-10-26T01:32:33.939 INFO:teuthology.orchestra.run.smithi169.stderr: removed all blocklist entries 2023-10-26T01:32:33.943 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:33.941+0000 7f558d27c640 1 -- 172.21.15.169:0/2751324578 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f556806ae40 msgr2=0x7f556806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:33.943 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:33.941+0000 7f558d27c640 1 --2- 172.21.15.169:0/2751324578 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f556806ae40 0x7f556806d300 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5570036000 tx=0x7f5570004730 comp rx=0 tx=0).stop 2023-10-26T01:32:33.943 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:33.941+0000 7f558d27c640 1 -- 172.21.15.169:0/2751324578 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55880ac1a0 msgr2=0x7f55880aeaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:33.943 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:33.941+0000 7f558d27c640 1 --2- 172.21.15.169:0/2751324578 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55880ac1a0 0x7f55880aeaa0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f557800cb80 tx=0x7f557800a430 comp rx=0 tx=0).stop 2023-10-26T01:32:33.944 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:33.941+0000 7f558d27c640 1 -- 172.21.15.169:0/2751324578 shutdown_connections 2023-10-26T01:32:33.944 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:33.941+0000 7f558d27c640 1 --2- 172.21.15.169:0/2751324578 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f556806ae40 0x7f556806d300 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:33.944 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:33.941+0000 7f558d27c640 1 --2- 172.21.15.169:0/2751324578 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f558815b990 0x7f5588148e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:33.944 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:33.941+0000 7f558d27c640 1 --2- 172.21.15.169:0/2751324578 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f55880adc70 0x7f5588148940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:33.944 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:33.941+0000 7f558d27c640 1 --2- 172.21.15.169:0/2751324578 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55880ac1a0 0x7f55880aeaa0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:33.944 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:33.941+0000 7f558d27c640 1 -- 172.21.15.169:0/2751324578 >> 172.21.15.169:0/2751324578 conn(0x7f558806a620 msgr2=0x7f558815aed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:33.945 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:33.941+0000 7f558d27c640 1 -- 172.21.15.169:0/2751324578 shutdown_connections 2023-10-26T01:32:33.945 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:33.941+0000 7f558d27c640 1 -- 172.21.15.169:0/2751324578 wait complete. 2023-10-26T01:32:33.962 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-26T01:32:34.174 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:34.172+0000 7f24f93ac640 1 -- 172.21.15.169:0/3864893957 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24f40ac6d0 msgr2=0x7f24f415d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:34.174 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:34.172+0000 7f24f93ac640 1 --2- 172.21.15.169:0/3864893957 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24f40ac6d0 0x7f24f415d840 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f24e4005fd0 tx=0x7f24e402be20 comp rx=0 tx=0).stop 2023-10-26T01:32:34.175 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:34.172+0000 7f24f93ac640 1 -- 172.21.15.169:0/3864893957 shutdown_connections 2023-10-26T01:32:34.175 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:34.172+0000 7f24f93ac640 1 --2- 172.21.15.169:0/3864893957 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f24f40c9970 0x7f24f4098a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:34.175 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:34.172+0000 7f24f93ac640 1 --2- 172.21.15.169:0/3864893957 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24f40ac6d0 0x7f24f415d840 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:34.175 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:34.172+0000 7f24f93ac640 1 -- 172.21.15.169:0/3864893957 >> 172.21.15.169:0/3864893957 conn(0x7f24f40aedd0 msgr2=0x7f24f40af1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:34.175 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:34.172+0000 7f24f93ac640 1 -- 172.21.15.169:0/3864893957 shutdown_connections 2023-10-26T01:32:34.176 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:34.172+0000 7f24f93ac640 1 -- 172.21.15.169:0/3864893957 wait complete. 2023-10-26T01:32:34.176 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:34.174+0000 7f24f93ac640 1 Processor -- start 2023-10-26T01:32:34.176 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:34.174+0000 7f24f93ac640 1 -- start start 2023-10-26T01:32:34.178 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:34.175+0000 7f24f93ac640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f24f409a5c0 0x7f24f413da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:34.178 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:34.175+0000 7f24f93ac640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f24f40ac6d0 0x7f24f40afce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:34.179 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:34.175+0000 7f24f93ac640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24f40c9970 0x7f24f4142650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:34.179 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:34.175+0000 7f24f93ac640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24f40c1e60 con 0x7f24f40c9970 2023-10-26T01:32:34.179 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:34.175+0000 7f24f93ac640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24f406fc10 con 0x7f24f409a5c0 2023-10-26T01:32:34.179 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:34.175+0000 7f24f93ac640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24f40b0dd0 con 0x7f24f40ac6d0 2023-10-26T01:32:34.179 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:34.175+0000 7f24f8bab640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24f40c9970 0x7f24f4142650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:34.180 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:34.175+0000 7f24f8bab640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24f40c9970 0x7f24f4142650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:33784/0 (socket says 172.21.15.169:33784) 2023-10-26T01:32:34.180 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:34.176+0000 7f24f8bab640 1 -- 172.21.15.169:0/1253656540 learned_addr learned my addr 172.21.15.169:0/1253656540 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:32:34.180 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:34.176+0000 7f24f37fe640 1 --2- 172.21.15.169:0/1253656540 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f24f40ac6d0 0x7f24f40afce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:34.180 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:34.176+0000 7f24f8bab640 1 -- 172.21.15.169:0/1253656540 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f24f40ac6d0 msgr2=0x7f24f40afce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:34.180 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:34.176+0000 7f24f3fff640 1 --2- 172.21.15.169:0/1253656540 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f24f409a5c0 0x7f24f413da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:34.181 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:34.176+0000 7f24f8bab640 1 --2- 172.21.15.169:0/1253656540 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f24f40ac6d0 0x7f24f40afce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:34.181 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:34.176+0000 7f24f8bab640 1 -- 172.21.15.169:0/1253656540 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f24f409a5c0 msgr2=0x7f24f413da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:34.181 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:34.176+0000 7f24f8bab640 1 --2- 172.21.15.169:0/1253656540 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f24f409a5c0 0x7f24f413da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:34.181 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:34.176+0000 7f24f8bab640 1 -- 172.21.15.169:0/1253656540 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24e4009560 con 0x7f24f40c9970 2023-10-26T01:32:34.181 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:34.176+0000 7f24f37fe640 1 --2- 172.21.15.169:0/1253656540 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f24f40ac6d0 0x7f24f40afce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:32:34.182 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:34.176+0000 7f24f8bab640 1 --2- 172.21.15.169:0/1253656540 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24f40c9970 0x7f24f4142650 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f24ec00c800 tx=0x7f24ec00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:34.182 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:34.176+0000 7f24f0ff9640 1 -- 172.21.15.169:0/1253656540 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24ec003b70 con 0x7f24f40c9970 2023-10-26T01:32:34.182 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:34.176+0000 7f24f0ff9640 1 -- 172.21.15.169:0/1253656540 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f24ec003cf0 con 0x7f24f40c9970 2023-10-26T01:32:34.182 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:34.176+0000 7f24f0ff9640 1 -- 172.21.15.169:0/1253656540 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24ec003ea0 con 0x7f24f40c9970 2023-10-26T01:32:34.182 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:34.176+0000 7f24f93ac640 1 -- 172.21.15.169:0/1253656540 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24f40b0530 con 0x7f24f40c9970 2023-10-26T01:32:34.183 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:34.177+0000 7f24f93ac640 1 -- 172.21.15.169:0/1253656540 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24f40b0af0 con 0x7f24f40c9970 2023-10-26T01:32:34.183 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:34.178+0000 7f24f93ac640 1 -- 172.21.15.169:0/1253656540 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24f406fda0 con 0x7f24f40c9970 2023-10-26T01:32:34.184 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:34.179+0000 7f24f0ff9640 1 -- 172.21.15.169:0/1253656540 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f24ec00aab0 con 0x7f24f40c9970 2023-10-26T01:32:34.184 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:34.180+0000 7f24f0ff9640 1 --2- 172.21.15.169:0/1253656540 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f24e006ae40 0x7f24e006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:34.184 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:34.180+0000 7f24f0ff9640 1 -- 172.21.15.169:0/1253656540 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f24ec0891f0 con 0x7f24f40c9970 2023-10-26T01:32:34.184 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:34.181+0000 7f24f3fff640 1 --2- 172.21.15.169:0/1253656540 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f24e006ae40 0x7f24e006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:34.184 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:34.182+0000 7f24f3fff640 1 --2- 172.21.15.169:0/1253656540 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f24e006ae40 0x7f24e006d300 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f24e4035000 tx=0x7f24e4004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:34.186 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:34.185+0000 7f24f0ff9640 1 -- 172.21.15.169:0/1253656540 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f24ec0544c0 con 0x7f24f40c9970 2023-10-26T01:32:34.448 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:34.446+0000 7f24f93ac640 1 -- 172.21.15.169:0/1253656540 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f24f40ca410 con 0x7f24f40c9970 2023-10-26T01:32:34.449 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:34.447+0000 7f24f0ff9640 1 -- 172.21.15.169:0/1253656540 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v11) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f24f40ca410 con 0x7f24f40c9970 2023-10-26T01:32:34.449 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:32:34.449 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2023-10-26T01:32:34.450 INFO:teuthology.orchestra.run.smithi169.stdout: "auth_dump": [ 2023-10-26T01:32:34.450 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:32:34.450 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "mds.a", 2023-10-26T01:32:34.450 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAgwTllWtUfAhAA28+12VwWEF/BUEu+ErSVyA==", 2023-10-26T01:32:34.451 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:32:34.451 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow", 2023-10-26T01:32:34.451 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow *", 2023-10-26T01:32:34.451 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow *", 2023-10-26T01:32:34.451 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:32:34.451 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:32:34.452 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:34.452 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:32:34.452 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "mds.b", 2023-10-26T01:32:34.452 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAgwTlleA8lHxAA+jMcMo+QnYAGllpKJ6eYgQ==", 2023-10-26T01:32:34.452 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:32:34.453 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow", 2023-10-26T01:32:34.453 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow *", 2023-10-26T01:32:34.453 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow *", 2023-10-26T01:32:34.453 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:32:34.453 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:32:34.454 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:34.454 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:32:34.454 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "mds.c", 2023-10-26T01:32:34.454 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAgwTllaCCqEBAAokeHvwOyp/Jv2x0yHDwysw==", 2023-10-26T01:32:34.454 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:32:34.455 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow", 2023-10-26T01:32:34.455 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow *", 2023-10-26T01:32:34.455 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow *", 2023-10-26T01:32:34.455 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:32:34.455 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:32:34.457 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:34.457 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:32:34.457 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "mds.d", 2023-10-26T01:32:34.457 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAgwTll51JQLhAA4Wzso4zQj+Htv6cXqvwRmg==", 2023-10-26T01:32:34.458 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:32:34.458 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow", 2023-10-26T01:32:34.458 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow *", 2023-10-26T01:32:34.458 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow *", 2023-10-26T01:32:34.458 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:32:34.459 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:32:34.459 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:34.459 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:32:34.459 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "osd.0", 2023-10-26T01:32:34.459 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAlwTll8RZ+HRAAt/sBK+Mi91YmTyQYbhKHLg==", 2023-10-26T01:32:34.460 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:32:34.460 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow profile osd", 2023-10-26T01:32:34.460 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow profile osd", 2023-10-26T01:32:34.460 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:32:34.460 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:32:34.461 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:34.461 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:32:34.461 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "osd.1", 2023-10-26T01:32:34.461 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAlwTllLXwWLRAAuHB2lJ+oDJou/IJBtsYY4Q==", 2023-10-26T01:32:34.461 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:32:34.462 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow profile osd", 2023-10-26T01:32:34.462 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow profile osd", 2023-10-26T01:32:34.462 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:32:34.462 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:32:34.462 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:34.463 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:32:34.463 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "osd.2", 2023-10-26T01:32:34.463 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAmwTllj1nSARAA5Jl34Q0X7a71ySd4wz17lw==", 2023-10-26T01:32:34.463 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:32:34.463 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow profile osd", 2023-10-26T01:32:34.463 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow profile osd", 2023-10-26T01:32:34.464 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:32:34.464 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:32:34.464 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:34.464 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:32:34.464 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "osd.3", 2023-10-26T01:32:34.465 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAmwTllC8MeEhAAfGxRVSwD+YbU/U5aeenO3w==", 2023-10-26T01:32:34.465 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:32:34.465 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow profile osd", 2023-10-26T01:32:34.465 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow profile osd", 2023-10-26T01:32:34.465 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:32:34.466 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:32:34.466 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:34.466 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:32:34.466 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "osd.4", 2023-10-26T01:32:34.466 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAowTll4Md6DhAAuXfadUNAdouMP28MiX/D3Q==", 2023-10-26T01:32:34.467 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:32:34.467 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow profile osd", 2023-10-26T01:32:34.467 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow profile osd", 2023-10-26T01:32:34.467 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:32:34.467 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:32:34.467 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:34.468 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:32:34.468 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "osd.5", 2023-10-26T01:32:34.468 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAowTllDdViHRAAec9QyhZCsjJ2rgGhhcjwpQ==", 2023-10-26T01:32:34.468 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:32:34.468 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow profile osd", 2023-10-26T01:32:34.469 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow profile osd", 2023-10-26T01:32:34.469 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:32:34.469 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:32:34.469 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:34.469 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:32:34.469 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "osd.6", 2023-10-26T01:32:34.470 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAowTll23X5KxAAsmQvG0yq7ilkVG5fNiaCUg==", 2023-10-26T01:32:34.470 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:32:34.470 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow profile osd", 2023-10-26T01:32:34.470 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow profile osd", 2023-10-26T01:32:34.470 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:32:34.471 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:32:34.471 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:34.471 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:32:34.471 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "osd.7", 2023-10-26T01:32:34.471 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAowTllhcy/OhAAY5Yjheh76ibOVREqN6WBpQ==", 2023-10-26T01:32:34.472 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:32:34.472 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow profile osd", 2023-10-26T01:32:34.472 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow profile osd", 2023-10-26T01:32:34.472 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:32:34.472 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:32:34.473 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:34.473 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:32:34.473 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "client.0", 2023-10-26T01:32:34.473 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAgwTllKC7gOhAAPsvxGcuUA8/p8xqbUgGteA==", 2023-10-26T01:32:34.473 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:32:34.473 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow", 2023-10-26T01:32:34.474 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow r", 2023-10-26T01:32:34.474 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-26T01:32:34.474 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:32:34.474 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:34.475 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:32:34.475 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "client.1", 2023-10-26T01:32:34.475 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAhwTllhxb6ExAAeYTsurWpnjohWlrNIlLpsw==", 2023-10-26T01:32:34.475 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:32:34.475 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow", 2023-10-26T01:32:34.476 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow r", 2023-10-26T01:32:34.476 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow rw", 2023-10-26T01:32:34.476 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow rwx" 2023-10-26T01:32:34.476 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:32:34.476 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:34.476 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:32:34.477 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "client.2", 2023-10-26T01:32:34.477 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAhwTllLWMoChAAsUHzBc29GDqmEQAFnl7dHg==", 2023-10-26T01:32:34.477 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:32:34.477 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow", 2023-10-26T01:32:34.477 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow r", 2023-10-26T01:32:34.478 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow rw", 2023-10-26T01:32:34.478 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow rwx" 2023-10-26T01:32:34.478 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:32:34.478 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:34.478 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:32:34.479 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "client.3", 2023-10-26T01:32:34.479 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAhwTll875HHRAAGbwikoR/hDU8P/bj0OPWAw==", 2023-10-26T01:32:34.479 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:32:34.479 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow", 2023-10-26T01:32:34.479 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow r", 2023-10-26T01:32:34.480 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow rw", 2023-10-26T01:32:34.480 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow rwx" 2023-10-26T01:32:34.480 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:32:34.480 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:34.480 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:32:34.480 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "client.admin", 2023-10-26T01:32:34.481 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAewTll2yddMhAAO8bnXNevDGe/KEeNNa36pQ==", 2023-10-26T01:32:34.481 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:32:34.481 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow *", 2023-10-26T01:32:34.481 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow *", 2023-10-26T01:32:34.481 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow *", 2023-10-26T01:32:34.482 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:32:34.482 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:32:34.482 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:34.482 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:32:34.482 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "mgr.x", 2023-10-26T01:32:34.483 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAfwTllPu3FMxAAV8d5oRFXAeeZH7e1xzPEDg==", 2023-10-26T01:32:34.483 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:32:34.483 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow *", 2023-10-26T01:32:34.483 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow profile mgr", 2023-10-26T01:32:34.483 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:32:34.484 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:32:34.484 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:34.484 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:32:34.484 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "mgr.y", 2023-10-26T01:32:34.484 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAfwTllZc+DJxAAsZ2TWn8H7Kw/lNTc2qyKzA==", 2023-10-26T01:32:34.485 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:32:34.485 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow *", 2023-10-26T01:32:34.485 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow profile mgr", 2023-10-26T01:32:34.485 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:32:34.485 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:32:34.486 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:32:34.486 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:32:34.486 INFO:teuthology.orchestra.run.smithi169.stdout:} 2023-10-26T01:32:34.487 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:34.450+0000 7f24f93ac640 1 -- 172.21.15.169:0/1253656540 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f24e006ae40 msgr2=0x7f24e006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:34.488 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:34.450+0000 7f24f93ac640 1 --2- 172.21.15.169:0/1253656540 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f24e006ae40 0x7f24e006d300 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f24e4035000 tx=0x7f24e4004730 comp rx=0 tx=0).stop 2023-10-26T01:32:34.488 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:34.450+0000 7f24f93ac640 1 -- 172.21.15.169:0/1253656540 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24f40c9970 msgr2=0x7f24f4142650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:34.488 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:34.450+0000 7f24f93ac640 1 --2- 172.21.15.169:0/1253656540 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24f40c9970 0x7f24f4142650 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f24ec00c800 tx=0x7f24ec00ccc0 comp rx=0 tx=0).stop 2023-10-26T01:32:34.488 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:34.450+0000 7f24f93ac640 1 -- 172.21.15.169:0/1253656540 shutdown_connections 2023-10-26T01:32:34.489 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:34.450+0000 7f24f93ac640 1 --2- 172.21.15.169:0/1253656540 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f24e006ae40 0x7f24e006d300 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:34.489 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:34.450+0000 7f24f93ac640 1 --2- 172.21.15.169:0/1253656540 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24f40c9970 0x7f24f4142650 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:34.489 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:34.450+0000 7f24f93ac640 1 --2- 172.21.15.169:0/1253656540 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f24f40ac6d0 0x7f24f40afce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:34.489 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:34.450+0000 7f24f93ac640 1 --2- 172.21.15.169:0/1253656540 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f24f409a5c0 0x7f24f413da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:34.489 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:34.450+0000 7f24f93ac640 1 -- 172.21.15.169:0/1253656540 >> 172.21.15.169:0/1253656540 conn(0x7f24f40aedd0 msgr2=0x7f24f40acb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:34.489 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:34.451+0000 7f24f93ac640 1 -- 172.21.15.169:0/1253656540 shutdown_connections 2023-10-26T01:32:34.490 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:34.451+0000 7f24f93ac640 1 -- 172.21.15.169:0/1253656540 wait complete. 2023-10-26T01:32:34.490 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:32:34.569 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T01:32:34.574 DEBUG:teuthology.orchestra.run.smithi170:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T01:32:34.691 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:34.690+0000 7f470e3d7640 1 -- 172.21.15.169:0/833702617 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f47080cd720 msgr2=0x7f47080cdbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:34.692 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:34.690+0000 7f470e3d7640 1 --2- 172.21.15.169:0/833702617 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f47080cd720 0x7f47080cdbd0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f4700005fd0 tx=0x7f470002c1b0 comp rx=0 tx=0).stop 2023-10-26T01:32:34.692 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:34.690+0000 7f470e3d7640 1 -- 172.21.15.169:0/833702617 shutdown_connections 2023-10-26T01:32:34.692 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:34.690+0000 7f470e3d7640 1 --2- 172.21.15.169:0/833702617 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f47080cd720 0x7f47080cdbd0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:34.692 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:34.690+0000 7f470e3d7640 1 --2- 172.21.15.169:0/833702617 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f470806ce20 0x7f470814e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:34.693 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:34.690+0000 7f470e3d7640 1 -- 172.21.15.169:0/833702617 >> 172.21.15.169:0/833702617 conn(0x7f4708069930 msgr2=0x7f4708069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:34.693 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:34.690+0000 7f470e3d7640 1 -- 172.21.15.169:0/833702617 shutdown_connections 2023-10-26T01:32:34.693 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:34.690+0000 7f470e3d7640 1 -- 172.21.15.169:0/833702617 wait complete. 2023-10-26T01:32:34.694 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:34.691+0000 7f470e3d7640 1 Processor -- start 2023-10-26T01:32:34.695 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:34.692+0000 7f470e3d7640 1 -- start start 2023-10-26T01:32:34.695 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:34.692+0000 7f470e3d7640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f470806ce20 0x7f4708071880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:34.695 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:34.692+0000 7f470e3d7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f47080cd720 0x7f4708143650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:34.695 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:34.692+0000 7f470e3d7640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4708158850 0x7f4708150d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:34.696 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:34.692+0000 7f470e3d7640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4708071dc0 con 0x7f4708158850 2023-10-26T01:32:34.696 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:34.692+0000 7f470e3d7640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47080707e0 con 0x7f47080cd720 2023-10-26T01:32:34.696 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:34.692+0000 7f470e3d7640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4708070ac0 con 0x7f470806ce20 2023-10-26T01:32:34.696 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:34.693+0000 7f470c974640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4708158850 0x7f4708150d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:34.696 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:34.693+0000 7f470c974640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4708158850 0x7f4708150d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:33806/0 (socket says 172.21.15.169:33806) 2023-10-26T01:32:34.697 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:34.693+0000 7f470c974640 1 -- 172.21.15.169:0/2194549501 learned_addr learned my addr 172.21.15.169:0/2194549501 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:32:34.697 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:34.693+0000 7f470c974640 1 -- 172.21.15.169:0/2194549501 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f470806ce20 msgr2=0x7f4708071880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:34.697 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:34.693+0000 7f470c974640 1 --2- 172.21.15.169:0/2194549501 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f470806ce20 0x7f4708071880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:34.697 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:34.693+0000 7f470c974640 1 -- 172.21.15.169:0/2194549501 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f47080cd720 msgr2=0x7f4708143650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:34.697 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:34.693+0000 7f470c974640 1 --2- 172.21.15.169:0/2194549501 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f47080cd720 0x7f4708143650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:34.698 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:34.693+0000 7f470c974640 1 -- 172.21.15.169:0/2194549501 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4700009900 con 0x7f4708158850 2023-10-26T01:32:34.699 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:34.693+0000 7f470c974640 1 --2- 172.21.15.169:0/2194549501 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4708158850 0x7f4708150d70 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f4700037040 tx=0x7f4700035db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:34.699 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:34.694+0000 7f47057fa640 1 -- 172.21.15.169:0/2194549501 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4700034440 con 0x7f4708158850 2023-10-26T01:32:34.699 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:34.694+0000 7f47057fa640 1 -- 172.21.15.169:0/2194549501 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4700004250 con 0x7f4708158850 2023-10-26T01:32:34.700 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:34.694+0000 7f47057fa640 1 -- 172.21.15.169:0/2194549501 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4700004620 con 0x7f4708158850 2023-10-26T01:32:34.700 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:34.694+0000 7f470e3d7640 1 -- 172.21.15.169:0/2194549501 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4708070da0 con 0x7f4708158850 2023-10-26T01:32:34.700 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:34.694+0000 7f470e3d7640 1 -- 172.21.15.169:0/2194549501 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4708071360 con 0x7f4708158850 2023-10-26T01:32:34.700 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:34.695+0000 7f470e3d7640 1 -- 172.21.15.169:0/2194549501 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4708071580 con 0x7f4708158850 2023-10-26T01:32:34.700 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:34.696+0000 7f47057fa640 1 -- 172.21.15.169:0/2194549501 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f470003b040 con 0x7f4708158850 2023-10-26T01:32:34.701 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:34.697+0000 7f47057fa640 1 --2- 172.21.15.169:0/2194549501 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f46ec06ae40 0x7f46ec06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:34.701 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:34.697+0000 7f47057fa640 1 -- 172.21.15.169:0/2194549501 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f47000ac0c0 con 0x7f4708158850 2023-10-26T01:32:34.701 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:34.698+0000 7f4707fff640 1 --2- 172.21.15.169:0/2194549501 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f46ec06ae40 0x7f46ec06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:34.702 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:34.699+0000 7f4707fff640 1 --2- 172.21.15.169:0/2194549501 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f46ec06ae40 0x7f46ec06d300 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f4708147720 tx=0x7f46f8005bf0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:34.702 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:34.701+0000 7f47057fa640 1 -- 172.21.15.169:0/2194549501 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4700077420 con 0x7f4708158850 2023-10-26T01:32:34.890 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:34.889+0000 7f470e3d7640 1 -- 172.21.15.169:0/2194549501 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f470815afd0 con 0x7f4708158850 2023-10-26T01:32:34.891 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:34.890+0000 7f47057fa640 1 -- 172.21.15.169:0/2194549501 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) v1 ==== 74+0+12905 (secure 0 0 0) 0x7f470007aa40 con 0x7f4708158850 2023-10-26T01:32:34.892 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:32:34.892 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":45,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:32:33.933410+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:32:34.894 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:34.892+0000 7f470e3d7640 1 -- 172.21.15.169:0/2194549501 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f46ec06ae40 msgr2=0x7f46ec06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:34.894 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:34.892+0000 7f470e3d7640 1 --2- 172.21.15.169:0/2194549501 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f46ec06ae40 0x7f46ec06d300 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f4708147720 tx=0x7f46f8005bf0 comp rx=0 tx=0).stop 2023-10-26T01:32:34.894 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:34.892+0000 7f470e3d7640 1 -- 172.21.15.169:0/2194549501 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4708158850 msgr2=0x7f4708150d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:34.895 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:34.892+0000 7f470e3d7640 1 --2- 172.21.15.169:0/2194549501 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4708158850 0x7f4708150d70 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f4700037040 tx=0x7f4700035db0 comp rx=0 tx=0).stop 2023-10-26T01:32:34.895 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:34.892+0000 7f470e3d7640 1 -- 172.21.15.169:0/2194549501 shutdown_connections 2023-10-26T01:32:34.895 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:34.892+0000 7f470e3d7640 1 --2- 172.21.15.169:0/2194549501 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f46ec06ae40 0x7f46ec06d300 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:34.895 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:34.892+0000 7f470e3d7640 1 --2- 172.21.15.169:0/2194549501 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4708158850 0x7f4708150d70 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:34.895 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:34.892+0000 7f470e3d7640 1 --2- 172.21.15.169:0/2194549501 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f47080cd720 0x7f4708143650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:34.896 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:34.892+0000 7f470e3d7640 1 --2- 172.21.15.169:0/2194549501 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f470806ce20 0x7f4708071880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:34.896 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:34.892+0000 7f470e3d7640 1 -- 172.21.15.169:0/2194549501 >> 172.21.15.169:0/2194549501 conn(0x7f4708069930 msgr2=0x7f47080acd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:34.896 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:34.892+0000 7f470e3d7640 1 -- 172.21.15.169:0/2194549501 shutdown_connections 2023-10-26T01:32:34.896 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:34.892+0000 7f470e3d7640 1 -- 172.21.15.169:0/2194549501 wait complete. 2023-10-26T01:32:34.912 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T01:30:32.200808+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T01:32:34.913 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T01:32:35.182 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:35.180+0000 7f1423607640 1 -- 172.21.15.169:0/1245859776 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f141c1589c0 msgr2=0x7f141c0cd760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:35.182 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:35.180+0000 7f1423607640 1 --2- 172.21.15.169:0/1245859776 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f141c1589c0 0x7f141c0cd760 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f141c0acef0 tx=0x7f140c02be30 comp rx=0 tx=0).stop 2023-10-26T01:32:35.182 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:35.180+0000 7f1423607640 1 -- 172.21.15.169:0/1245859776 shutdown_connections 2023-10-26T01:32:35.183 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:35.180+0000 7f1423607640 1 --2- 172.21.15.169:0/1245859776 >> v2:172.21.15.170:3300/0 conn(0x7f141c158e40 0x7f141c0cdca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:35.183 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:35.180+0000 7f1423607640 1 --2- 172.21.15.169:0/1245859776 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f141c1589c0 0x7f141c0cd760 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:35.183 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:35.180+0000 7f1423607640 1 -- 172.21.15.169:0/1245859776 >> 172.21.15.169:0/1245859776 conn(0x7f141c069930 msgr2=0x7f141c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:35.183 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:35.180+0000 7f1423607640 1 -- 172.21.15.169:0/1245859776 shutdown_connections 2023-10-26T01:32:35.183 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:35.180+0000 7f1423607640 1 -- 172.21.15.169:0/1245859776 wait complete. 2023-10-26T01:32:35.185 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:35.182+0000 7f1423607640 1 Processor -- start 2023-10-26T01:32:35.185 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:35.182+0000 7f1423607640 1 -- start start 2023-10-26T01:32:35.185 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:35.183+0000 7f1423607640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f141c0ccbe0 0x7f141c070b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:35.185 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:35.183+0000 7f1423607640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f141c1589c0 0x7f141c071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:35.186 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:35.183+0000 7f1423607640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f141c158e40 0x7f141c149390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:35.186 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:35.183+0000 7f1423607640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f141c0af1f0 con 0x7f141c0ccbe0 2023-10-26T01:32:35.186 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:35.183+0000 7f1423607640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f141c0af4d0 con 0x7f141c158e40 2023-10-26T01:32:35.186 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:35.183+0000 7f1423607640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f141c0af7b0 con 0x7f141c1589c0 2023-10-26T01:32:35.186 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:35.183+0000 7f14213a3640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f141c0ccbe0 0x7f141c070b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:35.187 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:35.183+0000 7f14213a3640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f141c0ccbe0 0x7f141c070b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:33820/0 (socket says 172.21.15.169:33820) 2023-10-26T01:32:35.187 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:35.183+0000 7f14213a3640 1 -- 172.21.15.169:0/1452311655 learned_addr learned my addr 172.21.15.169:0/1452311655 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:32:35.188 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:35.184+0000 7f14213a3640 1 -- 172.21.15.169:0/1452311655 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f141c1589c0 msgr2=0x7f141c071380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:35.188 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:35.184+0000 7f14213a3640 1 --2- 172.21.15.169:0/1452311655 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f141c1589c0 0x7f141c071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:35.188 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:35.184+0000 7f14213a3640 1 -- 172.21.15.169:0/1452311655 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f141c158e40 msgr2=0x7f141c149390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:35.189 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:35.184+0000 7f14213a3640 1 --2- 172.21.15.169:0/1452311655 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f141c158e40 0x7f141c149390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:35.189 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:35.184+0000 7f14213a3640 1 -- 172.21.15.169:0/1452311655 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f140c009560 con 0x7f141c0ccbe0 2023-10-26T01:32:35.189 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:35.184+0000 7f14213a3640 1 --2- 172.21.15.169:0/1452311655 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f141c0ccbe0 0x7f141c070b60 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f141800c870 tx=0x7f141800cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:35.189 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:35.184+0000 7f1411ffb640 1 -- 172.21.15.169:0/1452311655 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1418003b70 con 0x7f141c0ccbe0 2023-10-26T01:32:35.190 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:35.184+0000 7f1411ffb640 1 -- 172.21.15.169:0/1452311655 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1418003cf0 con 0x7f141c0ccbe0 2023-10-26T01:32:35.190 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:35.184+0000 7f1423607640 1 -- 172.21.15.169:0/1452311655 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f141c0afa90 con 0x7f141c0ccbe0 2023-10-26T01:32:35.190 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:35.184+0000 7f1411ffb640 1 -- 172.21.15.169:0/1452311655 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1418003ea0 con 0x7f141c0ccbe0 2023-10-26T01:32:35.190 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:35.184+0000 7f1423607640 1 -- 172.21.15.169:0/1452311655 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f141c0710a0 con 0x7f141c0ccbe0 2023-10-26T01:32:35.190 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:35.186+0000 7f1423607640 1 -- 172.21.15.169:0/1452311655 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f141c066af0 con 0x7f141c0ccbe0 2023-10-26T01:32:35.191 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:35.187+0000 7f1411ffb640 1 -- 172.21.15.169:0/1452311655 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f141800aab0 con 0x7f141c0ccbe0 2023-10-26T01:32:35.191 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:35.188+0000 7f1411ffb640 1 --2- 172.21.15.169:0/1452311655 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f140006ae40 0x7f140006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:35.192 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:35.188+0000 7f1411ffb640 1 -- 172.21.15.169:0/1452311655 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f1418089260 con 0x7f141c0ccbe0 2023-10-26T01:32:35.192 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:35.189+0000 7f1420ba2640 1 --2- 172.21.15.169:0/1452311655 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f140006ae40 0x7f140006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:35.192 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:35.190+0000 7f1420ba2640 1 --2- 172.21.15.169:0/1452311655 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f140006ae40 0x7f140006d300 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f140c035000 tx=0x7f140c0046c0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:35.194 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:35.192+0000 7f1411ffb640 1 -- 172.21.15.169:0/1452311655 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f14180546f0 con 0x7f141c0ccbe0 2023-10-26T01:32:35.384 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:35.382+0000 7f1423607640 1 -- 172.21.15.169:0/1452311655 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f141c04da70 con 0x7f141c0ccbe0 2023-10-26T01:32:35.384 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:35.383+0000 7f1411ffb640 1 -- 172.21.15.169:0/1452311655 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v45) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1418057d10 con 0x7f141c0ccbe0 2023-10-26T01:32:35.385 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 1 2023-10-26T01:32:35.387 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:35.385+0000 7f1423607640 1 -- 172.21.15.169:0/1452311655 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f140006ae40 msgr2=0x7f140006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:35.387 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:35.385+0000 7f1423607640 1 --2- 172.21.15.169:0/1452311655 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f140006ae40 0x7f140006d300 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f140c035000 tx=0x7f140c0046c0 comp rx=0 tx=0).stop 2023-10-26T01:32:35.388 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:35.385+0000 7f1423607640 1 -- 172.21.15.169:0/1452311655 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f141c0ccbe0 msgr2=0x7f141c070b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:35.388 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:35.385+0000 7f1423607640 1 --2- 172.21.15.169:0/1452311655 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f141c0ccbe0 0x7f141c070b60 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f141800c870 tx=0x7f141800cd30 comp rx=0 tx=0).stop 2023-10-26T01:32:35.388 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:35.385+0000 7f1423607640 1 -- 172.21.15.169:0/1452311655 shutdown_connections 2023-10-26T01:32:35.388 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:35.385+0000 7f1423607640 1 --2- 172.21.15.169:0/1452311655 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f140006ae40 0x7f140006d300 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:35.388 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:35.386+0000 7f1423607640 1 --2- 172.21.15.169:0/1452311655 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f141c158e40 0x7f141c149390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:35.389 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:35.386+0000 7f1423607640 1 --2- 172.21.15.169:0/1452311655 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f141c1589c0 0x7f141c071380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:35.389 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:35.386+0000 7f1423607640 1 --2- 172.21.15.169:0/1452311655 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f141c0ccbe0 0x7f141c070b60 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:35.389 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:35.386+0000 7f1423607640 1 -- 172.21.15.169:0/1452311655 >> 172.21.15.169:0/1452311655 conn(0x7f141c069930 msgr2=0x7f141c159ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:35.389 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:35.386+0000 7f1423607640 1 -- 172.21.15.169:0/1452311655 shutdown_connections 2023-10-26T01:32:35.389 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:35.386+0000 7f1423607640 1 -- 172.21.15.169:0/1452311655 wait complete. 2023-10-26T01:32:35.405 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-26T01:32:35.610 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:35.608+0000 7fe9042bb640 1 -- 172.21.15.169:0/2307220389 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8e8003fa0 msgr2=0x7fe8e80047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:35.611 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:35.608+0000 7fe9042bb640 1 --2- 172.21.15.169:0/2307220389 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8e8003fa0 0x7fe8e80047c0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec00bb20 tx=0x7fe8e80057f0 comp rx=0 tx=0).stop 2023-10-26T01:32:35.611 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:35.608+0000 7fe902858640 1 -- 172.21.15.169:0/2307220389 reap_dead start 2023-10-26T01:32:35.611 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:35.608+0000 7fe9042bb640 1 -- 172.21.15.169:0/2307220389 shutdown_connections 2023-10-26T01:32:35.611 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:35.608+0000 7fe9042bb640 1 -- 172.21.15.169:0/2307220389 >> 172.21.15.169:0/2307220389 conn(0x7fe8fc069930 msgr2=0x7fe8fc069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:35.612 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:35.609+0000 7fe9042bb640 1 -- 172.21.15.169:0/2307220389 shutdown_connections 2023-10-26T01:32:35.612 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:35.609+0000 7fe9042bb640 1 -- 172.21.15.169:0/2307220389 wait complete. 2023-10-26T01:32:35.614 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:35.610+0000 7fe9042bb640 1 Processor -- start 2023-10-26T01:32:35.614 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:35.610+0000 7fe9042bb640 1 -- start start 2023-10-26T01:32:35.615 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:35.611+0000 7fe9042bb640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe8e8003fa0 0x7fe8fc14a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:35.615 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:35.611+0000 7fe9042bb640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe8fc070890 0x7fe8fc070cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:35.615 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:35.611+0000 7fe9042bb640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8fc15bde0 0x7fe8fc15c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:35.615 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:35.611+0000 7fe9042bb640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8fc071230 con 0x7fe8fc15bde0 2023-10-26T01:32:35.615 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:35.611+0000 7fe9042bb640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8fc14ad70 con 0x7fe8e8003fa0 2023-10-26T01:32:35.616 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:35.611+0000 7fe9042bb640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8fc1f3190 con 0x7fe8fc070890 2023-10-26T01:32:35.616 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:35.612+0000 7fe902858640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8fc15bde0 0x7fe8fc15c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:35.616 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:35.612+0000 7fe902858640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8fc15bde0 0x7fe8fc15c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:33834/0 (socket says 172.21.15.169:33834) 2023-10-26T01:32:35.616 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:35.612+0000 7fe902858640 1 -- 172.21.15.169:0/4067514960 learned_addr learned my addr 172.21.15.169:0/4067514960 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:32:35.616 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:35.612+0000 7fe901856640 1 --2- 172.21.15.169:0/4067514960 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe8fc070890 0x7fe8fc070cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:35.617 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:35.612+0000 7fe902057640 1 --2- 172.21.15.169:0/4067514960 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe8e8003fa0 0x7fe8fc14a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:35.617 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:35.612+0000 7fe902858640 1 -- 172.21.15.169:0/4067514960 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe8fc070890 msgr2=0x7fe8fc070cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:35.617 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:35.612+0000 7fe902858640 1 --2- 172.21.15.169:0/4067514960 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe8fc070890 0x7fe8fc070cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:35.617 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:35.612+0000 7fe902858640 1 -- 172.21.15.169:0/4067514960 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe8e8003fa0 msgr2=0x7fe8fc14a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:35.617 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:35.612+0000 7fe902858640 1 --2- 172.21.15.169:0/4067514960 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe8e8003fa0 0x7fe8fc14a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:35.618 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:35.612+0000 7fe902858640 1 -- 172.21.15.169:0/4067514960 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8fc1f3470 con 0x7fe8fc15bde0 2023-10-26T01:32:35.618 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:35.612+0000 7fe902057640 1 --2- 172.21.15.169:0/4067514960 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe8e8003fa0 0x7fe8fc14a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:32:35.618 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:35.612+0000 7fe902858640 1 --2- 172.21.15.169:0/4067514960 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8fc15bde0 0x7fe8fc15c290 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fe8e8004740 tx=0x7fe8f400fb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:35.618 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:35.613+0000 7fe8f2ffd640 1 -- 172.21.15.169:0/4067514960 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f4004d20 con 0x7fe8fc15bde0 2023-10-26T01:32:35.618 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:35.613+0000 7fe8f2ffd640 1 -- 172.21.15.169:0/4067514960 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe8f4004e80 con 0x7fe8fc15bde0 2023-10-26T01:32:35.619 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:35.613+0000 7fe8f2ffd640 1 -- 172.21.15.169:0/4067514960 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f400daf0 con 0x7fe8fc15bde0 2023-10-26T01:32:35.619 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:35.613+0000 7fe9042bb640 1 -- 172.21.15.169:0/4067514960 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8fc1f3750 con 0x7fe8fc15bde0 2023-10-26T01:32:35.619 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:35.613+0000 7fe9042bb640 1 -- 172.21.15.169:0/4067514960 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8fc1f3d10 con 0x7fe8fc15bde0 2023-10-26T01:32:35.620 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:35.614+0000 7fe9042bb640 1 -- 172.21.15.169:0/4067514960 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8fc1f3f30 con 0x7fe8fc15bde0 2023-10-26T01:32:35.620 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:35.615+0000 7fe8f2ffd640 1 -- 172.21.15.169:0/4067514960 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fe8f400dc50 con 0x7fe8fc15bde0 2023-10-26T01:32:35.620 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:35.616+0000 7fe8f2ffd640 1 --2- 172.21.15.169:0/4067514960 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe8e006ae40 0x7fe8e006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:35.621 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:35.616+0000 7fe8f2ffd640 1 -- 172.21.15.169:0/4067514960 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fe8f4015070 con 0x7fe8fc15bde0 2023-10-26T01:32:35.621 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:35.617+0000 7fe902057640 1 --2- 172.21.15.169:0/4067514960 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe8e006ae40 0x7fe8e006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:35.621 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:35.618+0000 7fe902057640 1 --2- 172.21.15.169:0/4067514960 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe8e006ae40 0x7fe8e006d300 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe8f80012b0 tx=0x7fe8f8001100 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:35.622 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:35.621+0000 7fe8f2ffd640 1 -- 172.21.15.169:0/4067514960 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe8f4054e70 con 0x7fe8fc15bde0 2023-10-26T01:32:35.810 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:35.808+0000 7fe9042bb640 1 -- 172.21.15.169:0/4067514960 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fe8fc04db10 con 0x7fe8fc15bde0 2023-10-26T01:32:35.811 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:35.809+0000 7fe8f2ffd640 1 -- 172.21.15.169:0/4067514960 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v45) v1 ==== 84+0+65 (secure 0 0 0) 0x7fe8f4058490 con 0x7fe8fc15bde0 2023-10-26T01:32:35.811 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:32:35.811 INFO:teuthology.orchestra.run.smithi169.stdout:[ 2023-10-26T01:32:35.811 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:32:35.812 INFO:teuthology.orchestra.run.smithi169.stdout: "poolnum": 1, 2023-10-26T01:32:35.812 INFO:teuthology.orchestra.run.smithi169.stdout: "poolname": ".mgr" 2023-10-26T01:32:35.812 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:32:35.812 INFO:teuthology.orchestra.run.smithi169.stdout:] 2023-10-26T01:32:35.814 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:35.812+0000 7fe9042bb640 1 -- 172.21.15.169:0/4067514960 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe8e006ae40 msgr2=0x7fe8e006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:35.814 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:35.812+0000 7fe9042bb640 1 --2- 172.21.15.169:0/4067514960 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe8e006ae40 0x7fe8e006d300 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe8f80012b0 tx=0x7fe8f8001100 comp rx=0 tx=0).stop 2023-10-26T01:32:35.814 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:35.812+0000 7fe9042bb640 1 -- 172.21.15.169:0/4067514960 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8fc15bde0 msgr2=0x7fe8fc15c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:35.814 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:35.812+0000 7fe9042bb640 1 --2- 172.21.15.169:0/4067514960 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8fc15bde0 0x7fe8fc15c290 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fe8e8004740 tx=0x7fe8f400fb60 comp rx=0 tx=0).stop 2023-10-26T01:32:35.814 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:35.812+0000 7fe9042bb640 1 -- 172.21.15.169:0/4067514960 shutdown_connections 2023-10-26T01:32:35.815 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:35.812+0000 7fe9042bb640 1 --2- 172.21.15.169:0/4067514960 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe8e006ae40 0x7fe8e006d300 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:35.815 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:35.812+0000 7fe9042bb640 1 --2- 172.21.15.169:0/4067514960 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8fc15bde0 0x7fe8fc15c290 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:35.815 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:35.812+0000 7fe9042bb640 1 --2- 172.21.15.169:0/4067514960 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe8fc070890 0x7fe8fc070cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:35.815 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:35.812+0000 7fe9042bb640 1 --2- 172.21.15.169:0/4067514960 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe8e8003fa0 0x7fe8fc14a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:35.815 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:35.812+0000 7fe9042bb640 1 -- 172.21.15.169:0/4067514960 >> 172.21.15.169:0/4067514960 conn(0x7fe8fc069930 msgr2=0x7fe8fc154350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:35.816 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:35.812+0000 7fe9042bb640 1 -- 172.21.15.169:0/4067514960 shutdown_connections 2023-10-26T01:32:35.816 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:35.812+0000 7fe9042bb640 1 -- 172.21.15.169:0/4067514960 wait complete. 2023-10-26T01:32:35.832 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-26T01:32:35.833 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-26T01:32:36.051 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:36.049+0000 7fbdf491d640 1 -- 172.21.15.169:0/74059759 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbdf00b08f0 msgr2=0x7fbdf00cdb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:36.052 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:36.049+0000 7fbdf491d640 1 --2- 172.21.15.169:0/74059759 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbdf00b08f0 0x7fbdf00cdb30 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fbdf006a590 tx=0x7fbde802c2f0 comp rx=0 tx=0).stop 2023-10-26T01:32:36.052 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:36.049+0000 7fbdf491d640 1 -- 172.21.15.169:0/74059759 shutdown_connections 2023-10-26T01:32:36.052 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:36.049+0000 7fbdf491d640 1 --2- 172.21.15.169:0/74059759 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbdf00b08f0 0x7fbdf00cdb30 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:36.052 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:36.049+0000 7fbdf491d640 1 --2- 172.21.15.169:0/74059759 >> v2:172.21.15.170:3300/0 conn(0x7fbdf01539d0 0x7fbdf00cd5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:36.052 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:36.049+0000 7fbdf491d640 1 -- 172.21.15.169:0/74059759 >> 172.21.15.169:0/74059759 conn(0x7fbdf0069930 msgr2=0x7fbdf0069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:36.053 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:36.049+0000 7fbdf491d640 1 -- 172.21.15.169:0/74059759 shutdown_connections 2023-10-26T01:32:36.053 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:36.049+0000 7fbdf491d640 1 -- 172.21.15.169:0/74059759 wait complete. 2023-10-26T01:32:36.054 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:36.051+0000 7fbdf491d640 1 Processor -- start 2023-10-26T01:32:36.054 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:36.051+0000 7fbdf491d640 1 -- start start 2023-10-26T01:32:36.054 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:36.052+0000 7fbdf491d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbdf00b08f0 0x7fbdf01f25f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:36.055 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:36.052+0000 7fbdf491d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbdf01539d0 0x7fbdf006ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:36.055 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:36.052+0000 7fbdf491d640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbdf015aae0 0x7fbdf01ef990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:36.055 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:36.052+0000 7fbdf491d640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf006c010 con 0x7fbdf00b08f0 2023-10-26T01:32:36.055 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:36.052+0000 7fbdf491d640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf0158360 con 0x7fbdf01539d0 2023-10-26T01:32:36.055 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:36.052+0000 7fbdf491d640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf006b390 con 0x7fbdf015aae0 2023-10-26T01:32:36.056 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:36.052+0000 7fbdee59c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbdf00b08f0 0x7fbdf01f25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:36.056 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:36.053+0000 7fbdee59c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbdf00b08f0 0x7fbdf01f25f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:33852/0 (socket says 172.21.15.169:33852) 2023-10-26T01:32:36.056 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:36.053+0000 7fbdee59c640 1 -- 172.21.15.169:0/3231594155 learned_addr learned my addr 172.21.15.169:0/3231594155 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:32:36.058 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:36.053+0000 7fbdee59c640 1 -- 172.21.15.169:0/3231594155 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbdf015aae0 msgr2=0x7fbdf01ef990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:36.058 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:36.053+0000 7fbdee59c640 1 --2- 172.21.15.169:0/3231594155 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbdf015aae0 0x7fbdf01ef990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:36.058 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:36.053+0000 7fbdee59c640 1 -- 172.21.15.169:0/3231594155 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbdf01539d0 msgr2=0x7fbdf006ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:36.058 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:36.053+0000 7fbdedd9b640 1 --2- 172.21.15.169:0/3231594155 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbdf01539d0 0x7fbdf006ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:36.058 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:36.053+0000 7fbdee59c640 1 --2- 172.21.15.169:0/3231594155 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbdf01539d0 0x7fbdf006ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:36.059 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:36.053+0000 7fbdee59c640 1 -- 172.21.15.169:0/3231594155 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbde8009900 con 0x7fbdf00b08f0 2023-10-26T01:32:36.059 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:36.053+0000 7fbdee59c640 1 --2- 172.21.15.169:0/3231594155 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbdf00b08f0 0x7fbdf01f25f0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fbde000d950 tx=0x7fbde000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:36.059 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:36.054+0000 7fbddeffd640 1 -- 172.21.15.169:0/3231594155 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbde0009c30 con 0x7fbdf00b08f0 2023-10-26T01:32:36.059 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:36.054+0000 7fbddeffd640 1 -- 172.21.15.169:0/3231594155 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fbde0009e80 con 0x7fbdf00b08f0 2023-10-26T01:32:36.060 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:36.054+0000 7fbddeffd640 1 -- 172.21.15.169:0/3231594155 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbde00106c0 con 0x7fbdf00b08f0 2023-10-26T01:32:36.060 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:36.054+0000 7fbdf491d640 1 -- 172.21.15.169:0/3231594155 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdf006b670 con 0x7fbdf00b08f0 2023-10-26T01:32:36.060 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:36.054+0000 7fbdf491d640 1 -- 172.21.15.169:0/3231594155 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdf006bc30 con 0x7fbdf00b08f0 2023-10-26T01:32:36.060 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:36.055+0000 7fbdf491d640 1 -- 172.21.15.169:0/3231594155 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdf0066af0 con 0x7fbdf00b08f0 2023-10-26T01:32:36.060 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:36.056+0000 7fbddeffd640 1 -- 172.21.15.169:0/3231594155 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fbde0010820 con 0x7fbdf00b08f0 2023-10-26T01:32:36.061 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:36.057+0000 7fbddeffd640 1 --2- 172.21.15.169:0/3231594155 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbdd006ae40 0x7fbdd006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:36.061 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:36.057+0000 7fbddeffd640 1 -- 172.21.15.169:0/3231594155 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fbde00897b0 con 0x7fbdf00b08f0 2023-10-26T01:32:36.062 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:36.057+0000 7fbdedd9b640 1 --2- 172.21.15.169:0/3231594155 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbdd006ae40 0x7fbdd006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:36.062 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:36.058+0000 7fbdedd9b640 1 --2- 172.21.15.169:0/3231594155 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbdd006ae40 0x7fbdd006d300 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fbdf0071cc0 tx=0x7fbde4009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:36.063 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:36.061+0000 7fbddeffd640 1 -- 172.21.15.169:0/3231594155 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbde0054bb0 con 0x7fbdf00b08f0 2023-10-26T01:32:36.240 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:36.238+0000 7fbdf491d640 1 -- 172.21.15.169:0/3231594155 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fbdf0066db0 con 0x7fbdf00b08f0 2023-10-26T01:32:36.564 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:36.562+0000 7fbddeffd640 1 -- 172.21.15.169:0/3231594155 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v46) v1 ==== 138+0+0 (secure 0 0 0) 0x7fbde00200b0 con 0x7fbdf00b08f0 2023-10-26T01:32:36.564 INFO:teuthology.orchestra.run.smithi169.stderr:pool 'cephfs_metadata' created 2023-10-26T01:32:36.568 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:36.566+0000 7fbdf491d640 1 -- 172.21.15.169:0/3231594155 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbdd006ae40 msgr2=0x7fbdd006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:36.568 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:36.566+0000 7fbdf491d640 1 --2- 172.21.15.169:0/3231594155 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbdd006ae40 0x7fbdd006d300 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fbdf0071cc0 tx=0x7fbde4009040 comp rx=0 tx=0).stop 2023-10-26T01:32:36.568 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:36.566+0000 7fbdf491d640 1 -- 172.21.15.169:0/3231594155 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbdf00b08f0 msgr2=0x7fbdf01f25f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:36.568 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:36.566+0000 7fbdf491d640 1 --2- 172.21.15.169:0/3231594155 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbdf00b08f0 0x7fbdf01f25f0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fbde000d950 tx=0x7fbde000de10 comp rx=0 tx=0).stop 2023-10-26T01:32:36.568 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:36.566+0000 7fbdf491d640 1 -- 172.21.15.169:0/3231594155 shutdown_connections 2023-10-26T01:32:36.569 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:36.566+0000 7fbdf491d640 1 --2- 172.21.15.169:0/3231594155 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbdd006ae40 0x7fbdd006d300 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:36.569 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:36.566+0000 7fbdf491d640 1 --2- 172.21.15.169:0/3231594155 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbdf015aae0 0x7fbdf01ef990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:36.569 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:36.566+0000 7fbdf491d640 1 --2- 172.21.15.169:0/3231594155 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbdf01539d0 0x7fbdf006ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:36.569 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:36.566+0000 7fbdf491d640 1 --2- 172.21.15.169:0/3231594155 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbdf00b08f0 0x7fbdf01f25f0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:36.569 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:36.566+0000 7fbdf491d640 1 -- 172.21.15.169:0/3231594155 >> 172.21.15.169:0/3231594155 conn(0x7fbdf0069930 msgr2=0x7fbdf015bbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:36.570 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:36.566+0000 7fbdf491d640 1 -- 172.21.15.169:0/3231594155 shutdown_connections 2023-10-26T01:32:36.570 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:36.566+0000 7fbdf491d640 1 -- 172.21.15.169:0/3231594155 wait complete. 2023-10-26T01:32:36.590 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-26T01:32:36.804 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:36.802+0000 7f9d08ce5640 1 -- 172.21.15.169:0/285453630 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d0415b820 msgr2=0x7f9d040cc780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:36.805 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:36.802+0000 7f9d08ce5640 1 --2- 172.21.15.169:0/285453630 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d0415b820 0x7f9d040cc780 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f9d040c87a0 tx=0x7f9cf402c380 comp rx=0 tx=0).stop 2023-10-26T01:32:36.805 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:36.802+0000 7f9d08ce5640 1 -- 172.21.15.169:0/285453630 shutdown_connections 2023-10-26T01:32:36.805 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:36.802+0000 7f9d08ce5640 1 --2- 172.21.15.169:0/285453630 >> v2:172.21.15.170:3300/0 conn(0x7f9d0415bc00 0x7f9d040ce540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:36.805 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:36.802+0000 7f9d08ce5640 1 --2- 172.21.15.169:0/285453630 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d0415b820 0x7f9d040cc780 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:36.806 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:36.802+0000 7f9d08ce5640 1 -- 172.21.15.169:0/285453630 >> 172.21.15.169:0/285453630 conn(0x7f9d04065670 msgr2=0x7f9d04065a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:36.806 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:36.802+0000 7f9d08ce5640 1 -- 172.21.15.169:0/285453630 shutdown_connections 2023-10-26T01:32:36.806 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:36.803+0000 7f9d08ce5640 1 -- 172.21.15.169:0/285453630 wait complete. 2023-10-26T01:32:36.808 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:36.804+0000 7f9d08ce5640 1 Processor -- start 2023-10-26T01:32:36.808 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:36.804+0000 7f9d08ce5640 1 -- start start 2023-10-26T01:32:36.808 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:36.805+0000 7f9d08ce5640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d040cf530 0x7f9d040d1450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:36.808 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:36.805+0000 7f9d08ce5640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9d0415b820 0x7f9d041526e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:36.809 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:36.805+0000 7f9d08ce5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9d0415bc00 0x7f9d0414e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:36.809 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:36.805+0000 7f9d08ce5640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d041504f0 con 0x7f9d040cf530 2023-10-26T01:32:36.809 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:36.805+0000 7f9d08ce5640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d04154970 con 0x7f9d0415bc00 2023-10-26T01:32:36.809 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:36.805+0000 7f9d08ce5640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d04154c10 con 0x7f9d0415b820 2023-10-26T01:32:36.809 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:36.806+0000 7f9d0259c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d040cf530 0x7f9d040d1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:36.810 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:36.806+0000 7f9d0259c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d040cf530 0x7f9d040d1450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:33864/0 (socket says 172.21.15.169:33864) 2023-10-26T01:32:36.810 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:36.806+0000 7f9d0259c640 1 -- 172.21.15.169:0/1715498075 learned_addr learned my addr 172.21.15.169:0/1715498075 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:32:36.810 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:36.806+0000 7f9d01d9b640 1 --2- 172.21.15.169:0/1715498075 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9d0415b820 0x7f9d041526e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:36.810 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:36.806+0000 7f9d02d9d640 1 --2- 172.21.15.169:0/1715498075 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9d0415bc00 0x7f9d0414e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:36.810 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:36.806+0000 7f9d0259c640 1 -- 172.21.15.169:0/1715498075 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9d0415b820 msgr2=0x7f9d041526e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:36.811 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:36.806+0000 7f9d0259c640 1 --2- 172.21.15.169:0/1715498075 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9d0415b820 0x7f9d041526e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:36.811 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:36.806+0000 7f9d0259c640 1 -- 172.21.15.169:0/1715498075 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9d0415bc00 msgr2=0x7f9d0414e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:36.811 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:36.806+0000 7f9d0259c640 1 --2- 172.21.15.169:0/1715498075 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9d0415bc00 0x7f9d0414e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:36.811 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:36.806+0000 7f9d0259c640 1 -- 172.21.15.169:0/1715498075 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cf4009900 con 0x7f9d040cf530 2023-10-26T01:32:36.811 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:36.806+0000 7f9d01d9b640 1 --2- 172.21.15.169:0/1715498075 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9d0415b820 0x7f9d041526e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:32:36.812 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:36.806+0000 7f9d0259c640 1 --2- 172.21.15.169:0/1715498075 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d040cf530 0x7f9d040d1450 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f9cf4009bf0 tx=0x7f9cf40045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:36.813 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:36.807+0000 7f9cfaffd640 1 -- 172.21.15.169:0/1715498075 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf4031710 con 0x7f9d040cf530 2023-10-26T01:32:36.813 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:36.807+0000 7f9cfaffd640 1 -- 172.21.15.169:0/1715498075 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9cf4031960 con 0x7f9d040cf530 2023-10-26T01:32:36.813 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:36.807+0000 7f9cfaffd640 1 -- 172.21.15.169:0/1715498075 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf4031d10 con 0x7f9d040cf530 2023-10-26T01:32:36.814 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:36.807+0000 7f9d08ce5640 1 -- 172.21.15.169:0/1715498075 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d04154ef0 con 0x7f9d040cf530 2023-10-26T01:32:36.814 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:36.807+0000 7f9d08ce5640 1 -- 172.21.15.169:0/1715498075 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d040d1990 con 0x7f9d040cf530 2023-10-26T01:32:36.814 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:36.808+0000 7f9d08ce5640 1 -- 172.21.15.169:0/1715498075 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d0404db10 con 0x7f9d040cf530 2023-10-26T01:32:36.814 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:36.809+0000 7f9cfaffd640 1 -- 172.21.15.169:0/1715498075 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f9cf400a040 con 0x7f9d040cf530 2023-10-26T01:32:36.814 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:36.811+0000 7f9cfaffd640 1 --2- 172.21.15.169:0/1715498075 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9ce406ae40 0x7f9ce406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:36.815 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:36.811+0000 7f9cfaffd640 1 -- 172.21.15.169:0/1715498075 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f9cf40acb00 con 0x7f9d040cf530 2023-10-26T01:32:36.815 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:36.811+0000 7f9d01d9b640 1 --2- 172.21.15.169:0/1715498075 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9ce406ae40 0x7f9ce406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:36.815 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:36.812+0000 7f9d01d9b640 1 --2- 172.21.15.169:0/1715498075 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9ce406ae40 0x7f9ce406d300 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9d040b4020 tx=0x7f9cec005cf0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:36.817 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:36.816+0000 7f9cfaffd640 1 -- 172.21.15.169:0/1715498075 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9cf40aee60 con 0x7f9d040cf530 2023-10-26T01:32:37.011 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:37.009+0000 7f9d08ce5640 1 -- 172.21.15.169:0/1715498075 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f9d040b40d0 con 0x7f9d040cf530 2023-10-26T01:32:37.582 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:37.580+0000 7f9cfaffd640 1 -- 172.21.15.169:0/1715498075 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v47) v1 ==== 170+0+0 (secure 0 0 0) 0x7f9cf4077c60 con 0x7f9d040cf530 2023-10-26T01:32:37.583 INFO:teuthology.orchestra.run.smithi169.stderr:pool 'cephfs_data' created 2023-10-26T01:32:37.586 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:37.585+0000 7f9d08ce5640 1 -- 172.21.15.169:0/1715498075 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9ce406ae40 msgr2=0x7f9ce406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:37.587 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:37.585+0000 7f9d08ce5640 1 --2- 172.21.15.169:0/1715498075 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9ce406ae40 0x7f9ce406d300 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9d040b4020 tx=0x7f9cec005cf0 comp rx=0 tx=0).stop 2023-10-26T01:32:37.587 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:37.585+0000 7f9d08ce5640 1 -- 172.21.15.169:0/1715498075 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d040cf530 msgr2=0x7f9d040d1450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:37.587 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:37.585+0000 7f9d08ce5640 1 --2- 172.21.15.169:0/1715498075 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d040cf530 0x7f9d040d1450 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f9cf4009bf0 tx=0x7f9cf40045e0 comp rx=0 tx=0).stop 2023-10-26T01:32:37.587 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:37.585+0000 7f9d08ce5640 1 -- 172.21.15.169:0/1715498075 shutdown_connections 2023-10-26T01:32:37.588 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:37.585+0000 7f9d08ce5640 1 --2- 172.21.15.169:0/1715498075 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9ce406ae40 0x7f9ce406d300 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:37.588 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:37.585+0000 7f9d08ce5640 1 --2- 172.21.15.169:0/1715498075 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9d0415bc00 0x7f9d0414e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:37.588 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:37.585+0000 7f9d08ce5640 1 --2- 172.21.15.169:0/1715498075 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9d0415b820 0x7f9d041526e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:37.588 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:37.585+0000 7f9d08ce5640 1 --2- 172.21.15.169:0/1715498075 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d040cf530 0x7f9d040d1450 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:37.588 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:37.585+0000 7f9d08ce5640 1 -- 172.21.15.169:0/1715498075 >> 172.21.15.169:0/1715498075 conn(0x7f9d04065670 msgr2=0x7f9d04065a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:37.589 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:37.586+0000 7f9d08ce5640 1 -- 172.21.15.169:0/1715498075 shutdown_connections 2023-10-26T01:32:37.589 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:37.586+0000 7f9d08ce5640 1 -- 172.21.15.169:0/1715498075 wait complete. 2023-10-26T01:32:37.620 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-26T01:32:37.819 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:37.817+0000 7f36cf1a9640 1 -- 172.21.15.169:0/3469391661 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f36b8003ff0 msgr2=0x7f36b80047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:37.820 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:37.817+0000 7f36cf1a9640 1 --2- 172.21.15.169:0/3469391661 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f36b8003ff0 0x7f36b80047c0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f36b400bb20 tx=0x7f36b80057f0 comp rx=0 tx=0).stop 2023-10-26T01:32:37.820 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:37.817+0000 7f36cd746640 1 -- 172.21.15.169:0/3469391661 reap_dead start 2023-10-26T01:32:37.820 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:37.818+0000 7f36cf1a9640 1 -- 172.21.15.169:0/3469391661 shutdown_connections 2023-10-26T01:32:37.821 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:37.818+0000 7f36cf1a9640 1 -- 172.21.15.169:0/3469391661 >> 172.21.15.169:0/3469391661 conn(0x7f36c8069930 msgr2=0x7f36c8069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:37.821 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:37.818+0000 7f36cf1a9640 1 -- 172.21.15.169:0/3469391661 shutdown_connections 2023-10-26T01:32:37.821 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:37.818+0000 7f36cf1a9640 1 -- 172.21.15.169:0/3469391661 wait complete. 2023-10-26T01:32:37.822 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:37.819+0000 7f36cf1a9640 1 Processor -- start 2023-10-26T01:32:37.822 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:37.820+0000 7f36cf1a9640 1 -- start start 2023-10-26T01:32:37.822 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:37.820+0000 7f36cf1a9640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f36b8003ff0 0x7f36c81f17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:37.823 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:37.820+0000 7f36cf1a9640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f36c814b970 0x7f36c814bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:37.823 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:37.821+0000 7f36cf1a9640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f36c81504e0 0x7f36c8150960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:37.823 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:37.821+0000 7f36cf1a9640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36c8150ea0 con 0x7f36c81504e0 2023-10-26T01:32:37.823 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:37.821+0000 7f36cf1a9640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36c8089ff0 con 0x7f36c814b970 2023-10-26T01:32:37.823 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:37.821+0000 7f36cf1a9640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36c814e290 con 0x7f36b8003ff0 2023-10-26T01:32:37.825 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:37.821+0000 7f36cd746640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f36c81504e0 0x7f36c8150960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:37.825 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:37.821+0000 7f36cd746640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f36c81504e0 0x7f36c8150960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:33878/0 (socket says 172.21.15.169:33878) 2023-10-26T01:32:37.825 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:37.821+0000 7f36cd746640 1 -- 172.21.15.169:0/3310194098 learned_addr learned my addr 172.21.15.169:0/3310194098 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:32:37.825 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:37.821+0000 7f36ccf45640 1 --2- 172.21.15.169:0/3310194098 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f36b8003ff0 0x7f36c81f17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:37.825 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:37.821+0000 7f36cd746640 1 -- 172.21.15.169:0/3310194098 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f36b8003ff0 msgr2=0x7f36c81f17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:37.826 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:37.821+0000 7f36cd746640 1 --2- 172.21.15.169:0/3310194098 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f36b8003ff0 0x7f36c81f17f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:37.826 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:37.821+0000 7f36cd746640 1 -- 172.21.15.169:0/3310194098 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f36c814b970 msgr2=0x7f36c814bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:37.826 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:37.821+0000 7f36cd746640 1 --2- 172.21.15.169:0/3310194098 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f36c814b970 0x7f36c814bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:37.826 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:37.821+0000 7f36cd746640 1 -- 172.21.15.169:0/3310194098 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36c814e570 con 0x7f36c81504e0 2023-10-26T01:32:37.827 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:37.822+0000 7f36cd746640 1 --2- 172.21.15.169:0/3310194098 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f36c81504e0 0x7f36c8150960 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f36c815b880 tx=0x7f36c000dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:37.827 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:37.822+0000 7f36c57fa640 1 -- 172.21.15.169:0/3310194098 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36c0008c50 con 0x7f36c81504e0 2023-10-26T01:32:37.827 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:37.822+0000 7f36c57fa640 1 -- 172.21.15.169:0/3310194098 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f36c0017420 con 0x7f36c81504e0 2023-10-26T01:32:37.827 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:37.822+0000 7f36c57fa640 1 -- 172.21.15.169:0/3310194098 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36c0017830 con 0x7f36c81504e0 2023-10-26T01:32:37.827 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:37.822+0000 7f36cf1a9640 1 -- 172.21.15.169:0/3310194098 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36c814e850 con 0x7f36c81504e0 2023-10-26T01:32:37.828 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:37.822+0000 7f36cf1a9640 1 -- 172.21.15.169:0/3310194098 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36c81f1d30 con 0x7f36c81504e0 2023-10-26T01:32:37.828 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:37.823+0000 7f36cf1a9640 1 -- 172.21.15.169:0/3310194098 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36c804db10 con 0x7f36c81504e0 2023-10-26T01:32:37.829 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:37.824+0000 7f36c57fa640 1 -- 172.21.15.169:0/3310194098 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f36c000f420 con 0x7f36c81504e0 2023-10-26T01:32:37.829 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:37.825+0000 7f36c57fa640 1 --2- 172.21.15.169:0/3310194098 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f36ac06ae70 0x7f36ac06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:37.829 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:37.825+0000 7f36c57fa640 1 -- 172.21.15.169:0/3310194098 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f36c0013070 con 0x7f36c81504e0 2023-10-26T01:32:37.829 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:37.826+0000 7f36ccf45640 1 --2- 172.21.15.169:0/3310194098 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f36ac06ae70 0x7f36ac06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:37.830 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:37.827+0000 7f36ccf45640 1 --2- 172.21.15.169:0/3310194098 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f36ac06ae70 0x7f36ac06d330 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f36bc000c00 tx=0x7f36bc000f50 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:37.831 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:37.830+0000 7f36c57fa640 1 -- 172.21.15.169:0/3310194098 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f36c0054840 con 0x7f36c81504e0 2023-10-26T01:32:38.059 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:38.058+0000 7f36cf1a9640 1 -- 172.21.15.169:0/3310194098 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f36c8158c70 con 0x7f36c81504e0 2023-10-26T01:32:38.917 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:38.915+0000 7f36c57fa640 1 -- 172.21.15.169:0/3310194098 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v19) v1 ==== 172+0+0 (secure 0 0 0) 0x7f36c0057e60 con 0x7f36c81504e0 2023-10-26T01:32:38.918 INFO:teuthology.orchestra.run.smithi169.stderr:new fs with metadata pool 6 and data pool 7 2023-10-26T01:32:38.922 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:38.920+0000 7f36cf1a9640 1 -- 172.21.15.169:0/3310194098 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f36ac06ae70 msgr2=0x7f36ac06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:38.922 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:38.920+0000 7f36cf1a9640 1 --2- 172.21.15.169:0/3310194098 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f36ac06ae70 0x7f36ac06d330 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f36bc000c00 tx=0x7f36bc000f50 comp rx=0 tx=0).stop 2023-10-26T01:32:38.923 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:38.920+0000 7f36cf1a9640 1 -- 172.21.15.169:0/3310194098 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f36c81504e0 msgr2=0x7f36c8150960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:38.923 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:38.920+0000 7f36cf1a9640 1 --2- 172.21.15.169:0/3310194098 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f36c81504e0 0x7f36c8150960 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f36c815b880 tx=0x7f36c000dd70 comp rx=0 tx=0).stop 2023-10-26T01:32:38.923 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:38.921+0000 7f36cf1a9640 1 -- 172.21.15.169:0/3310194098 shutdown_connections 2023-10-26T01:32:38.923 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:38.921+0000 7f36cf1a9640 1 --2- 172.21.15.169:0/3310194098 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f36ac06ae70 0x7f36ac06d330 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:38.923 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:38.921+0000 7f36cf1a9640 1 --2- 172.21.15.169:0/3310194098 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f36c81504e0 0x7f36c8150960 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:38.924 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:38.921+0000 7f36cf1a9640 1 --2- 172.21.15.169:0/3310194098 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f36c814b970 0x7f36c814bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:38.924 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:38.921+0000 7f36cf1a9640 1 --2- 172.21.15.169:0/3310194098 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f36b8003ff0 0x7f36c81f17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:38.924 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:38.921+0000 7f36cf1a9640 1 -- 172.21.15.169:0/3310194098 >> 172.21.15.169:0/3310194098 conn(0x7f36c8069930 msgr2=0x7f36c8153640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:38.925 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:38.921+0000 7f36cf1a9640 1 -- 172.21.15.169:0/3310194098 shutdown_connections 2023-10-26T01:32:38.925 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:38.921+0000 7f36cf1a9640 1 -- 172.21.15.169:0/3310194098 wait complete. 2023-10-26T01:32:38.945 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:32:39.140 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:39.138+0000 7f81c3a0e640 1 -- 172.21.15.169:0/1313247566 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81bc0cdbb0 msgr2=0x7f81bc0b07d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:39.140 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:39.138+0000 7f81c3a0e640 1 --2- 172.21.15.169:0/1313247566 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81bc0cdbb0 0x7f81bc0b07d0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f81bc06c960 tx=0x7f81b402c110 comp rx=0 tx=0).stop 2023-10-26T01:32:39.140 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:39.138+0000 7f81c3a0e640 1 -- 172.21.15.169:0/1313247566 shutdown_connections 2023-10-26T01:32:39.141 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:39.138+0000 7f81c3a0e640 1 --2- 172.21.15.169:0/1313247566 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81bc0cdbb0 0x7f81bc0b07d0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:39.141 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:39.138+0000 7f81c3a0e640 1 --2- 172.21.15.169:0/1313247566 >> v2:172.21.15.170:3300/0 conn(0x7f81bc06ed30 0x7f81bc0cd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:39.141 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:39.138+0000 7f81c3a0e640 1 -- 172.21.15.169:0/1313247566 >> 172.21.15.169:0/1313247566 conn(0x7f81bc069930 msgr2=0x7f81bc069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:39.141 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:39.138+0000 7f81c3a0e640 1 -- 172.21.15.169:0/1313247566 shutdown_connections 2023-10-26T01:32:39.141 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:39.138+0000 7f81c3a0e640 1 -- 172.21.15.169:0/1313247566 wait complete. 2023-10-26T01:32:39.143 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:39.140+0000 7f81c3a0e640 1 Processor -- start 2023-10-26T01:32:39.143 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:39.140+0000 7f81c3a0e640 1 -- start start 2023-10-26T01:32:39.144 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:39.141+0000 7f81c3a0e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81bc06ed30 0x7f81bc15a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:39.144 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:39.141+0000 7f81c3a0e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81bc0cdbb0 0x7f81bc154600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:39.144 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:39.141+0000 7f81c3a0e640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f81bc15b610 0x7f81bc152840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:39.144 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:39.141+0000 7f81c3a0e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81bc1558a0 con 0x7f81bc0cdbb0 2023-10-26T01:32:39.144 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:39.141+0000 7f81c3a0e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81bc15c570 con 0x7f81bc06ed30 2023-10-26T01:32:39.145 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:39.141+0000 7f81c3a0e640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81bc155000 con 0x7f81bc15b610 2023-10-26T01:32:39.145 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:39.141+0000 7f81c0fa9640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81bc0cdbb0 0x7f81bc154600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:39.145 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:39.141+0000 7f81c0fa9640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81bc0cdbb0 0x7f81bc154600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:33898/0 (socket says 172.21.15.169:33898) 2023-10-26T01:32:39.145 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:39.141+0000 7f81c0fa9640 1 -- 172.21.15.169:0/4118046050 learned_addr learned my addr 172.21.15.169:0/4118046050 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:32:39.145 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:39.142+0000 7f81c17aa640 1 --2- 172.21.15.169:0/4118046050 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81bc06ed30 0x7f81bc15a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:39.146 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:39.142+0000 7f81c1fab640 1 --2- 172.21.15.169:0/4118046050 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f81bc15b610 0x7f81bc152840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:39.146 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:39.142+0000 7f81c0fa9640 1 -- 172.21.15.169:0/4118046050 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f81bc15b610 msgr2=0x7f81bc152840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:39.146 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:39.142+0000 7f81c0fa9640 1 --2- 172.21.15.169:0/4118046050 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f81bc15b610 0x7f81bc152840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:39.146 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:39.142+0000 7f81c0fa9640 1 -- 172.21.15.169:0/4118046050 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81bc06ed30 msgr2=0x7f81bc15a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:39.146 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:39.142+0000 7f81c0fa9640 1 --2- 172.21.15.169:0/4118046050 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81bc06ed30 0x7f81bc15a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:39.147 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:39.142+0000 7f81c0fa9640 1 -- 172.21.15.169:0/4118046050 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81b4009840 con 0x7f81bc0cdbb0 2023-10-26T01:32:39.147 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:39.142+0000 7f81c17aa640 1 --2- 172.21.15.169:0/4118046050 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81bc06ed30 0x7f81bc15a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:32:39.147 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:39.142+0000 7f81c0fa9640 1 --2- 172.21.15.169:0/4118046050 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81bc0cdbb0 0x7f81bc154600 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f81ac00d950 tx=0x7f81ac00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:39.148 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:39.142+0000 7f81b1ffb640 1 -- 172.21.15.169:0/4118046050 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81ac009c30 con 0x7f81bc0cdbb0 2023-10-26T01:32:39.149 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:39.142+0000 7f81b1ffb640 1 -- 172.21.15.169:0/4118046050 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f81ac009e80 con 0x7f81bc0cdbb0 2023-10-26T01:32:39.149 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:39.142+0000 7f81c3a0e640 1 -- 172.21.15.169:0/4118046050 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81bc1552e0 con 0x7f81bc0cdbb0 2023-10-26T01:32:39.149 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:39.142+0000 7f81b1ffb640 1 -- 172.21.15.169:0/4118046050 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81ac0106c0 con 0x7f81bc0cdbb0 2023-10-26T01:32:39.149 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:39.142+0000 7f81c3a0e640 1 -- 172.21.15.169:0/4118046050 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81bc15a6f0 con 0x7f81bc0cdbb0 2023-10-26T01:32:39.149 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:39.143+0000 7f81c3a0e640 1 -- 172.21.15.169:0/4118046050 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81bc15c3d0 con 0x7f81bc0cdbb0 2023-10-26T01:32:39.150 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:39.144+0000 7f81b1ffb640 1 -- 172.21.15.169:0/4118046050 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f81ac010820 con 0x7f81bc0cdbb0 2023-10-26T01:32:39.150 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:39.145+0000 7f81b1ffb640 1 --2- 172.21.15.169:0/4118046050 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f81a006ae70 0x7f81a006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:39.150 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:39.145+0000 7f81b1ffb640 1 -- 172.21.15.169:0/4118046050 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f81ac089a20 con 0x7f81bc0cdbb0 2023-10-26T01:32:39.150 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:39.146+0000 7f81c17aa640 1 --2- 172.21.15.169:0/4118046050 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f81a006ae70 0x7f81a006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:39.150 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:39.147+0000 7f81c17aa640 1 --2- 172.21.15.169:0/4118046050 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f81a006ae70 0x7f81a006d330 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f81b80015e0 tx=0x7f81b8009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:39.151 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:39.150+0000 7f81b1ffb640 1 -- 172.21.15.169:0/4118046050 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f81bc15c3d0 con 0x7f81bc0cdbb0 2023-10-26T01:32:39.341 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:39.339+0000 7f81c3a0e640 1 -- 172.21.15.169:0/4118046050 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f81bc15c3d0 con 0x7f81bc0cdbb0 2023-10-26T01:32:39.344 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:39.343+0000 7f81b1ffb640 1 -- 172.21.15.169:0/4118046050 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) v1 ==== 74+0+16574 (secure 0 0 0) 0x7f81bc15c3d0 con 0x7f81bc0cdbb0 2023-10-26T01:32:39.345 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:32:39.345 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":48,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:32:38.059268+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-26T01:32:36.288217+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-26T01:32:37.059775+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:32:39.347 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:39.345+0000 7f81c3a0e640 1 -- 172.21.15.169:0/4118046050 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f81a006ae70 msgr2=0x7f81a006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:39.347 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:39.345+0000 7f81c3a0e640 1 --2- 172.21.15.169:0/4118046050 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f81a006ae70 0x7f81a006d330 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f81b80015e0 tx=0x7f81b8009040 comp rx=0 tx=0).stop 2023-10-26T01:32:39.348 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:39.345+0000 7f81c3a0e640 1 -- 172.21.15.169:0/4118046050 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81bc0cdbb0 msgr2=0x7f81bc154600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:39.348 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:39.345+0000 7f81c3a0e640 1 --2- 172.21.15.169:0/4118046050 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81bc0cdbb0 0x7f81bc154600 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f81ac00d950 tx=0x7f81ac00de10 comp rx=0 tx=0).stop 2023-10-26T01:32:39.348 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:39.345+0000 7f81c3a0e640 1 -- 172.21.15.169:0/4118046050 shutdown_connections 2023-10-26T01:32:39.348 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:39.345+0000 7f81c3a0e640 1 --2- 172.21.15.169:0/4118046050 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f81a006ae70 0x7f81a006d330 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:39.348 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:39.345+0000 7f81c3a0e640 1 --2- 172.21.15.169:0/4118046050 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81bc0cdbb0 0x7f81bc154600 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:39.349 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:39.345+0000 7f81c3a0e640 1 --2- 172.21.15.169:0/4118046050 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f81bc15b610 0x7f81bc152840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:39.349 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:39.345+0000 7f81c3a0e640 1 --2- 172.21.15.169:0/4118046050 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81bc06ed30 0x7f81bc15a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:39.349 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:39.345+0000 7f81c3a0e640 1 -- 172.21.15.169:0/4118046050 >> 172.21.15.169:0/4118046050 conn(0x7f81bc069930 msgr2=0x7f81bc0b1670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:39.349 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:39.346+0000 7f81c3a0e640 1 -- 172.21.15.169:0/4118046050 shutdown_connections 2023-10-26T01:32:39.349 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:39.346+0000 7f81c3a0e640 1 -- 172.21.15.169:0/4118046050 wait complete. 2023-10-26T01:32:39.366 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:32:39.575 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:39.573+0000 7f73c1993640 1 -- 172.21.15.169:0/1962726546 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f73bc0b45d0 msgr2=0x7f73bc0cbce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:39.575 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:39.573+0000 7f73c1993640 1 --2- 172.21.15.169:0/1962726546 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f73bc0b45d0 0x7f73bc0cbce0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f73bc04e000 tx=0x7f73ac02c260 comp rx=0 tx=0).stop 2023-10-26T01:32:39.576 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:39.573+0000 7f73c1993640 1 -- 172.21.15.169:0/1962726546 shutdown_connections 2023-10-26T01:32:39.576 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:39.573+0000 7f73c1993640 1 --2- 172.21.15.169:0/1962726546 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f73bc0b45d0 0x7f73bc0cbce0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:39.576 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:39.573+0000 7f73c1993640 1 --2- 172.21.15.169:0/1962726546 >> v2:172.21.15.170:3300/0 conn(0x7f73bc0b68c0 0x7f73bc065ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:39.576 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:39.573+0000 7f73c1993640 1 -- 172.21.15.169:0/1962726546 >> 172.21.15.169:0/1962726546 conn(0x7f73bc0d1360 msgr2=0x7f73bc0d1770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:39.576 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:39.573+0000 7f73c1993640 1 -- 172.21.15.169:0/1962726546 shutdown_connections 2023-10-26T01:32:39.576 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:39.573+0000 7f73c1993640 1 -- 172.21.15.169:0/1962726546 wait complete. 2023-10-26T01:32:39.577 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:39.575+0000 7f73c1993640 1 Processor -- start 2023-10-26T01:32:39.577 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:39.575+0000 7f73c1993640 1 -- start start 2023-10-26T01:32:39.579 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:39.576+0000 7f73c1993640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f73bc0b45d0 0x7f73bc1518a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:39.579 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:39.576+0000 7f73c1993640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f73bc0b4a30 0x7f73bc1504f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:39.579 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:39.576+0000 7f73c1993640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f73bc0b68c0 0x7f73bc14ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:39.580 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:39.576+0000 7f73c1993640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73bc150a30 con 0x7f73bc0b45d0 2023-10-26T01:32:39.580 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:39.576+0000 7f73c1993640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73bc14f1a0 con 0x7f73bc0b68c0 2023-10-26T01:32:39.580 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:39.576+0000 7f73c1993640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73bc14f450 con 0x7f73bc0b4a30 2023-10-26T01:32:39.580 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:39.576+0000 7f73c0991640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f73bc0b45d0 0x7f73bc1518a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:39.580 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:39.576+0000 7f73c0991640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f73bc0b45d0 0x7f73bc1518a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:33912/0 (socket says 172.21.15.169:33912) 2023-10-26T01:32:39.581 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:39.576+0000 7f73c0991640 1 -- 172.21.15.169:0/3993065809 learned_addr learned my addr 172.21.15.169:0/3993065809 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:32:39.581 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:39.577+0000 7f73bbfff640 1 --2- 172.21.15.169:0/3993065809 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f73bc0b4a30 0x7f73bc1504f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:39.581 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:39.577+0000 7f73c0991640 1 -- 172.21.15.169:0/3993065809 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f73bc0b4a30 msgr2=0x7f73bc1504f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:39.581 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:39.577+0000 7f73c0991640 1 --2- 172.21.15.169:0/3993065809 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f73bc0b4a30 0x7f73bc1504f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:39.581 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:39.577+0000 7f73c1192640 1 --2- 172.21.15.169:0/3993065809 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f73bc0b68c0 0x7f73bc14ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:39.581 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:39.577+0000 7f73c0991640 1 -- 172.21.15.169:0/3993065809 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f73bc0b68c0 msgr2=0x7f73bc14ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:39.582 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:39.577+0000 7f73c0991640 1 --2- 172.21.15.169:0/3993065809 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f73bc0b68c0 0x7f73bc14ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:39.582 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:39.577+0000 7f73c0991640 1 -- 172.21.15.169:0/3993065809 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73ac009900 con 0x7f73bc0b45d0 2023-10-26T01:32:39.582 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:39.577+0000 7f73c0991640 1 --2- 172.21.15.169:0/3993065809 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f73bc0b45d0 0x7f73bc1518a0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f73a400c970 tx=0x7f73a400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:39.583 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:39.578+0000 7f73b97fa640 1 -- 172.21.15.169:0/3993065809 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a4007bb0 con 0x7f73bc0b45d0 2023-10-26T01:32:39.583 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:39.578+0000 7f73b97fa640 1 -- 172.21.15.169:0/3993065809 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f73a4007e00 con 0x7f73bc0b45d0 2023-10-26T01:32:39.583 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:39.578+0000 7f73b97fa640 1 -- 172.21.15.169:0/3993065809 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a400f6b0 con 0x7f73bc0b45d0 2023-10-26T01:32:39.583 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:39.578+0000 7f73c1993640 1 -- 172.21.15.169:0/3993065809 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73bc14f730 con 0x7f73bc0b45d0 2023-10-26T01:32:39.583 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:39.578+0000 7f73c1993640 1 -- 172.21.15.169:0/3993065809 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73bc14fcf0 con 0x7f73bc0b45d0 2023-10-26T01:32:39.584 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:39.579+0000 7f73c1993640 1 -- 172.21.15.169:0/3993065809 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73bc073c10 con 0x7f73bc0b45d0 2023-10-26T01:32:39.584 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:39.580+0000 7f73b97fa640 1 -- 172.21.15.169:0/3993065809 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f73a400f810 con 0x7f73bc0b45d0 2023-10-26T01:32:39.585 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:39.581+0000 7f73b97fa640 1 --2- 172.21.15.169:0/3993065809 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f739c06ae40 0x7f739c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:39.585 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:39.581+0000 7f73b97fa640 1 -- 172.21.15.169:0/3993065809 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f73a4089f20 con 0x7f73bc0b45d0 2023-10-26T01:32:39.585 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:39.581+0000 7f73bbfff640 1 --2- 172.21.15.169:0/3993065809 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f739c06ae40 0x7f739c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:39.585 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:39.582+0000 7f73bbfff640 1 --2- 172.21.15.169:0/3993065809 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f739c06ae40 0x7f739c06d300 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f73ac036000 tx=0x7f73ac004ab0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:39.587 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:39.585+0000 7f73b97fa640 1 -- 172.21.15.169:0/3993065809 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f73a4054f30 con 0x7f73bc0b45d0 2023-10-26T01:32:39.784 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:39.782+0000 7f73c1993640 1 -- 172.21.15.169:0/3993065809 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f73bc0d1ee0 con 0x7f73bc0b45d0 2023-10-26T01:32:39.786 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:39.785+0000 7f73b97fa640 1 -- 172.21.15.169:0/3993065809 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) v1 ==== 74+0+16561 (secure 0 0 0) 0x7f73a401d090 con 0x7f73bc0b45d0 2023-10-26T01:32:39.787 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:32:39.787 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":49,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:32:39.585979+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-26T01:32:36.288217+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-26T01:32:37.059775+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:32:39.790 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:39.788+0000 7f73c1993640 1 -- 172.21.15.169:0/3993065809 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f739c06ae40 msgr2=0x7f739c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:39.790 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:39.788+0000 7f73c1993640 1 --2- 172.21.15.169:0/3993065809 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f739c06ae40 0x7f739c06d300 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f73ac036000 tx=0x7f73ac004ab0 comp rx=0 tx=0).stop 2023-10-26T01:32:39.790 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:39.788+0000 7f73c1993640 1 -- 172.21.15.169:0/3993065809 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f73bc0b45d0 msgr2=0x7f73bc1518a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:39.790 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:39.788+0000 7f73c1993640 1 --2- 172.21.15.169:0/3993065809 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f73bc0b45d0 0x7f73bc1518a0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f73a400c970 tx=0x7f73a400ce30 comp rx=0 tx=0).stop 2023-10-26T01:32:39.790 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:39.788+0000 7f73c1993640 1 -- 172.21.15.169:0/3993065809 shutdown_connections 2023-10-26T01:32:39.791 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:39.788+0000 7f73c1993640 1 --2- 172.21.15.169:0/3993065809 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f739c06ae40 0x7f739c06d300 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:39.791 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:39.788+0000 7f73c1993640 1 --2- 172.21.15.169:0/3993065809 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f73bc0b68c0 0x7f73bc14ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:39.791 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:39.788+0000 7f73c1993640 1 --2- 172.21.15.169:0/3993065809 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f73bc0b4a30 0x7f73bc1504f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:39.791 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:39.788+0000 7f73c1993640 1 --2- 172.21.15.169:0/3993065809 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f73bc0b45d0 0x7f73bc1518a0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:39.791 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:39.788+0000 7f73c1993640 1 -- 172.21.15.169:0/3993065809 >> 172.21.15.169:0/3993065809 conn(0x7f73bc0d1360 msgr2=0x7f73bc0cabb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:39.792 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:39.788+0000 7f73c1993640 1 -- 172.21.15.169:0/3993065809 shutdown_connections 2023-10-26T01:32:39.792 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:39.788+0000 7f73c1993640 1 -- 172.21.15.169:0/3993065809 wait complete. 2023-10-26T01:32:39.809 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-26T01:32:40.038 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:40.036+0000 7f5edb8b4640 1 -- 172.21.15.169:0/3612397458 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ed415bbb0 msgr2=0x7f5ed4155360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:40.038 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:40.036+0000 7f5edb8b4640 1 --2- 172.21.15.169:0/3612397458 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ed415bbb0 0x7f5ed4155360 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5ec8009830 tx=0x7f5ec802bcd0 comp rx=0 tx=0).stop 2023-10-26T01:32:40.038 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:40.036+0000 7f5edb8b4640 1 -- 172.21.15.169:0/3612397458 shutdown_connections 2023-10-26T01:32:40.039 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:40.036+0000 7f5edb8b4640 1 --2- 172.21.15.169:0/3612397458 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5ed40b05a0 0x7f5ed40b09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:40.039 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:40.036+0000 7f5edb8b4640 1 --2- 172.21.15.169:0/3612397458 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ed415bbb0 0x7f5ed4155360 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:40.039 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:40.036+0000 7f5edb8b4640 1 -- 172.21.15.169:0/3612397458 >> 172.21.15.169:0/3612397458 conn(0x7f5ed4069930 msgr2=0x7f5ed4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:40.040 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:40.036+0000 7f5edb8b4640 1 -- 172.21.15.169:0/3612397458 shutdown_connections 2023-10-26T01:32:40.040 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:40.036+0000 7f5edb8b4640 1 -- 172.21.15.169:0/3612397458 wait complete. 2023-10-26T01:32:40.041 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:40.038+0000 7f5edb8b4640 1 Processor -- start 2023-10-26T01:32:40.041 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:40.038+0000 7f5edb8b4640 1 -- start start 2023-10-26T01:32:40.042 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:40.040+0000 7f5edb8b4640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5ed406ce00 0x7f5ed40c7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:40.042 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:40.040+0000 7f5edb8b4640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5ed40b05a0 0x7f5ed41f16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:40.043 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:40.040+0000 7f5edb8b4640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ed415bbb0 0x7f5ed406ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:40.043 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:40.040+0000 7f5edb8b4640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ed40c9130 con 0x7f5ed406ce00 2023-10-26T01:32:40.043 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:40.040+0000 7f5edb8b4640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ed40c8520 con 0x7f5ed415bbb0 2023-10-26T01:32:40.043 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:40.040+0000 7f5edb8b4640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ed40c8800 con 0x7f5ed40b05a0 2023-10-26T01:32:40.043 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:40.040+0000 7f5ed9650640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5ed406ce00 0x7f5ed40c7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:40.044 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:40.040+0000 7f5ed9650640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5ed406ce00 0x7f5ed40c7cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:33916/0 (socket says 172.21.15.169:33916) 2023-10-26T01:32:40.044 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:40.040+0000 7f5ed9650640 1 -- 172.21.15.169:0/3296604049 learned_addr learned my addr 172.21.15.169:0/3296604049 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:32:40.044 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:40.040+0000 7f5ed9e51640 1 --2- 172.21.15.169:0/3296604049 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ed415bbb0 0x7f5ed406ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:40.044 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:40.040+0000 7f5ed8e4f640 1 --2- 172.21.15.169:0/3296604049 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5ed40b05a0 0x7f5ed41f16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:40.044 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:40.040+0000 7f5ed9650640 1 -- 172.21.15.169:0/3296604049 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5ed40b05a0 msgr2=0x7f5ed41f16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:40.045 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:40.040+0000 7f5ed9650640 1 --2- 172.21.15.169:0/3296604049 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5ed40b05a0 0x7f5ed41f16c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:40.045 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:40.040+0000 7f5ed9650640 1 -- 172.21.15.169:0/3296604049 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ed415bbb0 msgr2=0x7f5ed406ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:40.045 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:40.040+0000 7f5ed9650640 1 --2- 172.21.15.169:0/3296604049 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ed415bbb0 0x7f5ed406ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:40.045 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:40.040+0000 7f5ed9650640 1 -- 172.21.15.169:0/3296604049 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ec8009490 con 0x7f5ed406ce00 2023-10-26T01:32:40.045 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:40.041+0000 7f5ed9650640 1 --2- 172.21.15.169:0/3296604049 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5ed406ce00 0x7f5ed40c7cb0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f5ec400cb40 tx=0x7f5ec400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:40.047 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:40.041+0000 7f5ed1ffb640 1 -- 172.21.15.169:0/3296604049 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ec400aca0 con 0x7f5ed406ce00 2023-10-26T01:32:40.047 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:40.041+0000 7f5ed1ffb640 1 -- 172.21.15.169:0/3296604049 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5ec4012420 con 0x7f5ed406ce00 2023-10-26T01:32:40.047 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:40.041+0000 7f5edb8b4640 1 -- 172.21.15.169:0/3296604049 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ed40c8ae0 con 0x7f5ed406ce00 2023-10-26T01:32:40.047 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:40.041+0000 7f5ed1ffb640 1 -- 172.21.15.169:0/3296604049 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ec4012830 con 0x7f5ed406ce00 2023-10-26T01:32:40.047 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:40.041+0000 7f5edb8b4640 1 -- 172.21.15.169:0/3296604049 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ed40c81f0 con 0x7f5ed406ce00 2023-10-26T01:32:40.048 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:40.042+0000 7f5edb8b4640 1 -- 172.21.15.169:0/3296604049 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ed406cb20 con 0x7f5ed406ce00 2023-10-26T01:32:40.048 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:40.043+0000 7f5ed1ffb640 1 -- 172.21.15.169:0/3296604049 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f5ec401b400 con 0x7f5ed406ce00 2023-10-26T01:32:40.048 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:40.044+0000 7f5ed1ffb640 1 --2- 172.21.15.169:0/3296604049 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5ec006ae40 0x7f5ec006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:40.048 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:40.044+0000 7f5ed1ffb640 1 -- 172.21.15.169:0/3296604049 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5ec408a160 con 0x7f5ed406ce00 2023-10-26T01:32:40.048 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:40.045+0000 7f5ed8e4f640 1 --2- 172.21.15.169:0/3296604049 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5ec006ae40 0x7f5ec006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:40.049 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:40.046+0000 7f5ed8e4f640 1 --2- 172.21.15.169:0/3296604049 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5ec006ae40 0x7f5ec006d300 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f5ec8036000 tx=0x7f5ec8004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:40.050 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:40.049+0000 7f5ed1ffb640 1 -- 172.21.15.169:0/3296604049 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5ec4055180 con 0x7f5ed406ce00 2023-10-26T01:32:40.281 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:40.279+0000 7f5edb8b4640 1 -- 172.21.15.169:0/3296604049 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f5ed404db10 con 0x7f5ed406ce00 2023-10-26T01:32:40.945 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:40.943+0000 7f5ed1ffb640 1 -- 172.21.15.169:0/3296604049 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v22) v1 ==== 118+0+0 (secure 0 0 0) 0x7f5ec4001c00 con 0x7f5ed406ce00 2023-10-26T01:32:40.949 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:40.947+0000 7f5edb8b4640 1 -- 172.21.15.169:0/3296604049 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5ec006ae40 msgr2=0x7f5ec006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:40.950 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:40.947+0000 7f5edb8b4640 1 --2- 172.21.15.169:0/3296604049 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5ec006ae40 0x7f5ec006d300 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f5ec8036000 tx=0x7f5ec8004730 comp rx=0 tx=0).stop 2023-10-26T01:32:40.950 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:40.947+0000 7f5edb8b4640 1 -- 172.21.15.169:0/3296604049 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5ed406ce00 msgr2=0x7f5ed40c7cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:40.950 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:40.947+0000 7f5edb8b4640 1 --2- 172.21.15.169:0/3296604049 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5ed406ce00 0x7f5ed40c7cb0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f5ec400cb40 tx=0x7f5ec400a060 comp rx=0 tx=0).stop 2023-10-26T01:32:40.950 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:40.948+0000 7f5edb8b4640 1 -- 172.21.15.169:0/3296604049 shutdown_connections 2023-10-26T01:32:40.951 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:40.948+0000 7f5edb8b4640 1 --2- 172.21.15.169:0/3296604049 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5ec006ae40 0x7f5ec006d300 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:40.951 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:40.948+0000 7f5edb8b4640 1 --2- 172.21.15.169:0/3296604049 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ed415bbb0 0x7f5ed406ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:40.951 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:40.948+0000 7f5edb8b4640 1 --2- 172.21.15.169:0/3296604049 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5ed40b05a0 0x7f5ed41f16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:40.951 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:40.948+0000 7f5edb8b4640 1 --2- 172.21.15.169:0/3296604049 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5ed406ce00 0x7f5ed40c7cb0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:40.951 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:40.948+0000 7f5edb8b4640 1 -- 172.21.15.169:0/3296604049 >> 172.21.15.169:0/3296604049 conn(0x7f5ed4069930 msgr2=0x7f5ed415a580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:40.952 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:40.948+0000 7f5edb8b4640 1 -- 172.21.15.169:0/3296604049 shutdown_connections 2023-10-26T01:32:40.952 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:40.948+0000 7f5edb8b4640 1 -- 172.21.15.169:0/3296604049 wait complete. 2023-10-26T01:32:40.970 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-26T01:32:40.970 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-26T01:32:41.188 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:41.185+0000 7fea76821640 1 -- 172.21.15.169:0/4269060166 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fea5c005180 msgr2=0x7fea5c00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:41.188 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:41.185+0000 7fea76821640 1 --2- 172.21.15.169:0/4269060166 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fea5c005180 0x7fea5c00b980 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fea68009e50 tx=0x7fea5c0057f0 comp rx=0 tx=0).stop 2023-10-26T01:32:41.188 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:41.185+0000 7fea74dbe640 1 -- 172.21.15.169:0/4269060166 reap_dead start 2023-10-26T01:32:41.188 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:41.186+0000 7fea76821640 1 -- 172.21.15.169:0/4269060166 shutdown_connections 2023-10-26T01:32:41.189 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:41.186+0000 7fea76821640 1 -- 172.21.15.169:0/4269060166 >> 172.21.15.169:0/4269060166 conn(0x7fea70069930 msgr2=0x7fea70069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:41.189 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:41.186+0000 7fea76821640 1 -- 172.21.15.169:0/4269060166 shutdown_connections 2023-10-26T01:32:41.189 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:41.186+0000 7fea76821640 1 -- 172.21.15.169:0/4269060166 wait complete. 2023-10-26T01:32:41.190 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:41.188+0000 7fea76821640 1 Processor -- start 2023-10-26T01:32:41.190 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:41.188+0000 7fea76821640 1 -- start start 2023-10-26T01:32:41.191 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:41.189+0000 7fea76821640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fea700cba30 0x7fea700cbe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:41.191 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:41.189+0000 7fea76821640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fea700c6660 0x7fea700c6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:41.191 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:41.189+0000 7fea76821640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fea7014d500 0x7fea7014d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:41.192 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:41.189+0000 7fea76821640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea7014dec0 con 0x7fea700cba30 2023-10-26T01:32:41.192 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:41.189+0000 7fea76821640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea700c7120 con 0x7fea7014d500 2023-10-26T01:32:41.192 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:41.189+0000 7fea76821640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea700c7400 con 0x7fea700c6660 2023-10-26T01:32:41.192 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:41.190+0000 7fea6ffff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fea700cba30 0x7fea700cbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:41.192 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:41.190+0000 7fea6ffff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fea700cba30 0x7fea700cbe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:33936/0 (socket says 172.21.15.169:33936) 2023-10-26T01:32:41.193 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:41.190+0000 7fea6ffff640 1 -- 172.21.15.169:0/3139365007 learned_addr learned my addr 172.21.15.169:0/3139365007 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:32:41.194 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:41.190+0000 7fea74dbe640 1 --2- 172.21.15.169:0/3139365007 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fea7014d500 0x7fea7014d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:41.194 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:41.190+0000 7fea67fff640 1 --2- 172.21.15.169:0/3139365007 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fea700c6660 0x7fea700c6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:41.194 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:41.190+0000 7fea6ffff640 1 -- 172.21.15.169:0/3139365007 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fea700c6660 msgr2=0x7fea700c6ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:41.195 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:41.190+0000 7fea6ffff640 1 --2- 172.21.15.169:0/3139365007 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fea700c6660 0x7fea700c6ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:41.195 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:41.190+0000 7fea6ffff640 1 -- 172.21.15.169:0/3139365007 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fea7014d500 msgr2=0x7fea7014d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:41.195 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:41.190+0000 7fea6ffff640 1 --2- 172.21.15.169:0/3139365007 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fea7014d500 0x7fea7014d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:41.195 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:41.190+0000 7fea6ffff640 1 -- 172.21.15.169:0/3139365007 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea700c76e0 con 0x7fea700cba30 2023-10-26T01:32:41.195 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:41.190+0000 7fea74dbe640 1 --2- 172.21.15.169:0/3139365007 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fea7014d500 0x7fea7014d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:32:41.196 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:41.190+0000 7fea6ffff640 1 --2- 172.21.15.169:0/3139365007 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fea700cba30 0x7fea700cbe10 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fea60004fd0 tx=0x7fea6000eb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:41.196 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:41.191+0000 7fea6d7fa640 1 -- 172.21.15.169:0/3139365007 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea6000ca40 con 0x7fea700cba30 2023-10-26T01:32:41.196 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:41.191+0000 7fea6d7fa640 1 -- 172.21.15.169:0/3139365007 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fea6000ccb0 con 0x7fea700cba30 2023-10-26T01:32:41.196 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:41.191+0000 7fea6d7fa640 1 -- 172.21.15.169:0/3139365007 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea600116c0 con 0x7fea700cba30 2023-10-26T01:32:41.196 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:41.191+0000 7fea76821640 1 -- 172.21.15.169:0/3139365007 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea700c79c0 con 0x7fea700cba30 2023-10-26T01:32:41.196 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:41.191+0000 7fea76821640 1 -- 172.21.15.169:0/3139365007 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea7015a710 con 0x7fea700cba30 2023-10-26T01:32:41.197 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:41.192+0000 7fea76821640 1 -- 172.21.15.169:0/3139365007 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea700c7ee0 con 0x7fea700cba30 2023-10-26T01:32:41.197 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:41.193+0000 7fea6d7fa640 1 -- 172.21.15.169:0/3139365007 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fea60011820 con 0x7fea700cba30 2023-10-26T01:32:41.198 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:41.194+0000 7fea6d7fa640 1 --2- 172.21.15.169:0/3139365007 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fea5006ae70 0x7fea5006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:41.198 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:41.194+0000 7fea6d7fa640 1 -- 172.21.15.169:0/3139365007 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fea60089fa0 con 0x7fea700cba30 2023-10-26T01:32:41.198 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:41.195+0000 7fea67fff640 1 --2- 172.21.15.169:0/3139365007 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fea5006ae70 0x7fea5006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:41.198 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:41.196+0000 7fea67fff640 1 --2- 172.21.15.169:0/3139365007 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fea5006ae70 0x7fea5006d330 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fea58005d00 tx=0x7fea58005c50 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:41.200 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:41.198+0000 7fea6d7fa640 1 -- 172.21.15.169:0/3139365007 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fea700c7ee0 con 0x7fea700cba30 2023-10-26T01:32:41.428 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:41.426+0000 7fea76821640 1 -- 172.21.15.169:0/3139365007 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fea700c7ee0 con 0x7fea700cba30 2023-10-26T01:32:41.948 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:41.946+0000 7fea6d7fa640 1 -- 172.21.15.169:0/3139365007 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v23) v1 ==== 122+0+0 (secure 0 0 0) 0x7fea700c7ee0 con 0x7fea700cba30 2023-10-26T01:32:41.951 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:41.950+0000 7fea76821640 1 -- 172.21.15.169:0/3139365007 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fea5006ae70 msgr2=0x7fea5006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:41.952 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:41.950+0000 7fea76821640 1 --2- 172.21.15.169:0/3139365007 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fea5006ae70 0x7fea5006d330 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fea58005d00 tx=0x7fea58005c50 comp rx=0 tx=0).stop 2023-10-26T01:32:41.952 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:41.950+0000 7fea76821640 1 -- 172.21.15.169:0/3139365007 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fea700cba30 msgr2=0x7fea700cbe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:41.952 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:41.950+0000 7fea76821640 1 --2- 172.21.15.169:0/3139365007 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fea700cba30 0x7fea700cbe10 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fea60004fd0 tx=0x7fea6000eb60 comp rx=0 tx=0).stop 2023-10-26T01:32:41.952 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:41.950+0000 7fea76821640 1 -- 172.21.15.169:0/3139365007 shutdown_connections 2023-10-26T01:32:41.953 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:41.950+0000 7fea76821640 1 --2- 172.21.15.169:0/3139365007 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fea5006ae70 0x7fea5006d330 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:41.953 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:41.950+0000 7fea76821640 1 --2- 172.21.15.169:0/3139365007 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fea7014d500 0x7fea7014d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:41.953 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:41.950+0000 7fea76821640 1 --2- 172.21.15.169:0/3139365007 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fea700c6660 0x7fea700c6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:41.953 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:41.950+0000 7fea76821640 1 --2- 172.21.15.169:0/3139365007 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fea700cba30 0x7fea700cbe10 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:41.953 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:41.950+0000 7fea76821640 1 -- 172.21.15.169:0/3139365007 >> 172.21.15.169:0/3139365007 conn(0x7fea70069930 msgr2=0x7fea70157d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:41.954 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:41.950+0000 7fea76821640 1 -- 172.21.15.169:0/3139365007 shutdown_connections 2023-10-26T01:32:41.954 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:41.951+0000 7fea76821640 1 -- 172.21.15.169:0/3139365007 wait complete. 2023-10-26T01:32:41.971 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-26T01:32:41.972 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T01:32:42.184 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:42.183+0000 7f276e953640 1 -- 172.21.15.169:0/2740551891 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27680ac930 msgr2=0x7f27680cd6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:42.185 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:42.183+0000 7f276e953640 1 --2- 172.21.15.169:0/2740551891 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27680ac930 0x7f27680cd6f0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f2750005fd0 tx=0x7f275002bd10 comp rx=0 tx=0).stop 2023-10-26T01:32:42.186 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:42.184+0000 7f276e953640 1 -- 172.21.15.169:0/2740551891 shutdown_connections 2023-10-26T01:32:42.186 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:42.184+0000 7f276e953640 1 --2- 172.21.15.169:0/2740551891 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27680ac930 0x7f27680cd6f0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:42.186 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:42.184+0000 7f276e953640 1 --2- 172.21.15.169:0/2740551891 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f27681592d0 0x7f27680aa3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:42.186 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:42.184+0000 7f276e953640 1 -- 172.21.15.169:0/2740551891 >> 172.21.15.169:0/2740551891 conn(0x7f2768069930 msgr2=0x7f2768069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:42.187 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:42.184+0000 7f276e953640 1 -- 172.21.15.169:0/2740551891 shutdown_connections 2023-10-26T01:32:42.187 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:42.184+0000 7f276e953640 1 -- 172.21.15.169:0/2740551891 wait complete. 2023-10-26T01:32:42.187 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:42.186+0000 7f276e953640 1 Processor -- start 2023-10-26T01:32:42.188 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:42.186+0000 7f276e953640 1 -- start start 2023-10-26T01:32:42.190 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:42.187+0000 7f276e953640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f27680ac930 0x7f27681f1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:42.190 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:42.187+0000 7f276e953640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f27680cdc30 0x7f27680c7e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:42.190 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:42.187+0000 7f276e953640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27681592d0 0x7f27680c8350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:42.191 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:42.187+0000 7f276e953640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27680c69f0 con 0x7f27681592d0 2023-10-26T01:32:42.191 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:42.187+0000 7f276e953640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27680c6cd0 con 0x7f27680cdc30 2023-10-26T01:32:42.191 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:42.187+0000 7f276e953640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27680c6fb0 con 0x7f27680ac930 2023-10-26T01:32:42.191 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:42.187+0000 7f276cef0640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27681592d0 0x7f27680c8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:42.192 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:42.187+0000 7f276cef0640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27681592d0 0x7f27680c8350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:33954/0 (socket says 172.21.15.169:33954) 2023-10-26T01:32:42.192 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:42.187+0000 7f276cef0640 1 -- 172.21.15.169:0/2796078103 learned_addr learned my addr 172.21.15.169:0/2796078103 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:32:42.192 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:42.187+0000 7f275ffff640 1 --2- 172.21.15.169:0/2796078103 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f27680cdc30 0x7f27680c7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:42.192 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:42.187+0000 7f2767fff640 1 --2- 172.21.15.169:0/2796078103 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f27680ac930 0x7f27681f1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:42.193 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:42.187+0000 7f276cef0640 1 -- 172.21.15.169:0/2796078103 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f27680ac930 msgr2=0x7f27681f1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:42.193 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:42.187+0000 7f276cef0640 1 --2- 172.21.15.169:0/2796078103 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f27680ac930 0x7f27681f1260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:42.193 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:42.187+0000 7f276cef0640 1 -- 172.21.15.169:0/2796078103 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f27680cdc30 msgr2=0x7f27680c7e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:42.194 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:42.187+0000 7f276cef0640 1 --2- 172.21.15.169:0/2796078103 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f27680cdc30 0x7f27680c7e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:42.194 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:42.188+0000 7f276cef0640 1 -- 172.21.15.169:0/2796078103 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2750009560 con 0x7f27681592d0 2023-10-26T01:32:42.194 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:42.188+0000 7f275ffff640 1 --2- 172.21.15.169:0/2796078103 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f27680cdc30 0x7f27680c7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:32:42.194 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:42.188+0000 7f276cef0640 1 --2- 172.21.15.169:0/2796078103 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27681592d0 0x7f27680c8350 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f276000c780 tx=0x7f276000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:42.195 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:42.188+0000 7f27657fa640 1 -- 172.21.15.169:0/2796078103 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2760004b60 con 0x7f27681592d0 2023-10-26T01:32:42.195 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:42.188+0000 7f27657fa640 1 -- 172.21.15.169:0/2796078103 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f276000ce90 con 0x7f27681592d0 2023-10-26T01:32:42.195 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:42.188+0000 7f276e953640 1 -- 172.21.15.169:0/2796078103 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27680c7290 con 0x7f27681592d0 2023-10-26T01:32:42.195 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:42.188+0000 7f27657fa640 1 -- 172.21.15.169:0/2796078103 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f276000aab0 con 0x7f27681592d0 2023-10-26T01:32:42.196 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:42.188+0000 7f276e953640 1 -- 172.21.15.169:0/2796078103 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27681f17a0 con 0x7f27681592d0 2023-10-26T01:32:42.197 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:42.189+0000 7f276e953640 1 -- 172.21.15.169:0/2796078103 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27680c77b0 con 0x7f27681592d0 2023-10-26T01:32:42.197 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:42.190+0000 7f27657fa640 1 -- 172.21.15.169:0/2796078103 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f276000ac10 con 0x7f27681592d0 2023-10-26T01:32:42.197 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:42.191+0000 7f27657fa640 1 --2- 172.21.15.169:0/2796078103 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f274806ae70 0x7f274806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:42.197 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:42.191+0000 7f27657fa640 1 -- 172.21.15.169:0/2796078103 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2760089b40 con 0x7f27681592d0 2023-10-26T01:32:42.198 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:42.192+0000 7f2767fff640 1 --2- 172.21.15.169:0/2796078103 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f274806ae70 0x7f274806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:42.198 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:42.193+0000 7f2767fff640 1 --2- 172.21.15.169:0/2796078103 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f274806ae70 0x7f274806d330 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f27580058c0 tx=0x7f2758005850 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:42.198 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:42.195+0000 7f27657fa640 1 -- 172.21.15.169:0/2796078103 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f27680c77b0 con 0x7f27681592d0 2023-10-26T01:32:42.423 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:42.421+0000 7f276e953640 1 -- 172.21.15.169:0/2796078103 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f27680c77b0 con 0x7f27681592d0 2023-10-26T01:32:42.424 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:42.422+0000 7f27657fa640 1 -- 172.21.15.169:0/2796078103 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4925 (secure 0 0 0) 0x7f27680c77b0 con 0x7f27681592d0 2023-10-26T01:32:42.424 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:32:42.425 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4805,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/2294107634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2294107634},{"type":"v1","addr":"172.21.15.170:6836","nonce":2294107634}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":4987,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2208026662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2208026662},{"type":"v1","addr":"172.21.15.169:6833","nonce":2208026662}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":4993,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/2630365058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":2630365058},{"type":"v1","addr":"172.21.15.170:6837","nonce":2630365058}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T01:32:38.059721+0000","modified":"2023-10-26T01:32:41.428183+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5664},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5664":{"gid":5664,"name":"a","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6835/2816266753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":2816266753},{"type":"v1","addr":"172.21.15.169:6835","nonce":2816266753}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-10-26T01:32:42.425 INFO:teuthology.orchestra.run.smithi169.stderr:dumped fsmap epoch 23 2023-10-26T01:32:42.427 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:42.425+0000 7f276e953640 1 -- 172.21.15.169:0/2796078103 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f274806ae70 msgr2=0x7f274806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:42.427 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:42.425+0000 7f276e953640 1 --2- 172.21.15.169:0/2796078103 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f274806ae70 0x7f274806d330 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f27580058c0 tx=0x7f2758005850 comp rx=0 tx=0).stop 2023-10-26T01:32:42.427 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:42.425+0000 7f276e953640 1 -- 172.21.15.169:0/2796078103 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27681592d0 msgr2=0x7f27680c8350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:42.427 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:42.425+0000 7f276e953640 1 --2- 172.21.15.169:0/2796078103 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27681592d0 0x7f27680c8350 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f276000c780 tx=0x7f276000cc40 comp rx=0 tx=0).stop 2023-10-26T01:32:42.427 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:42.425+0000 7f276e953640 1 -- 172.21.15.169:0/2796078103 shutdown_connections 2023-10-26T01:32:42.428 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:42.425+0000 7f276e953640 1 --2- 172.21.15.169:0/2796078103 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f274806ae70 0x7f274806d330 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:42.428 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:42.425+0000 7f276e953640 1 --2- 172.21.15.169:0/2796078103 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27681592d0 0x7f27680c8350 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:42.428 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:42.425+0000 7f276e953640 1 --2- 172.21.15.169:0/2796078103 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f27680cdc30 0x7f27680c7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:42.428 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:42.425+0000 7f276e953640 1 --2- 172.21.15.169:0/2796078103 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f27680ac930 0x7f27681f1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:42.428 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:42.425+0000 7f276e953640 1 -- 172.21.15.169:0/2796078103 >> 172.21.15.169:0/2796078103 conn(0x7f2768069930 msgr2=0x7f2768069d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:42.429 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:42.425+0000 7f276e953640 1 -- 172.21.15.169:0/2796078103 shutdown_connections 2023-10-26T01:32:42.429 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:42.425+0000 7f276e953640 1 -- 172.21.15.169:0/2796078103 wait complete. 2023-10-26T01:32:42.446 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:32:42.658 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:42.656+0000 7fa0b359e640 1 -- 172.21.15.169:0/2889818707 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa0b409a5c0 msgr2=0x7fa0b409aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:42.659 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:42.656+0000 7fa0b359e640 1 --2- 172.21.15.169:0/2889818707 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa0b409a5c0 0x7fa0b409aa70 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fa0b40cbd50 tx=0x7fa0ac02be30 comp rx=0 tx=0).stop 2023-10-26T01:32:42.659 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:42.657+0000 7fa0b359e640 1 -- 172.21.15.169:0/2889818707 shutdown_connections 2023-10-26T01:32:42.659 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:42.657+0000 7fa0b359e640 1 --2- 172.21.15.169:0/2889818707 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa0b409a5c0 0x7fa0b409aa70 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:42.659 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:42.657+0000 7fa0b359e640 1 --2- 172.21.15.169:0/2889818707 >> v2:172.21.15.170:3300/0 conn(0x7fa0b40c9970 0x7fa0b4098a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:42.660 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:42.657+0000 7fa0b359e640 1 -- 172.21.15.169:0/2889818707 >> 172.21.15.169:0/2889818707 conn(0x7fa0b40aedd0 msgr2=0x7fa0b40af1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:42.660 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:42.657+0000 7fa0b359e640 1 -- 172.21.15.169:0/2889818707 shutdown_connections 2023-10-26T01:32:42.660 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:42.657+0000 7fa0b359e640 1 -- 172.21.15.169:0/2889818707 wait complete. 2023-10-26T01:32:42.661 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:42.658+0000 7fa0b359e640 1 Processor -- start 2023-10-26T01:32:42.661 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:42.659+0000 7fa0b359e640 1 -- start start 2023-10-26T01:32:42.663 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:42.660+0000 7fa0b359e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa0b409a5c0 0x7fa0b415b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:42.663 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:42.660+0000 7fa0b359e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa0b40ac6d0 0x7fa0b41599a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:42.663 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:42.660+0000 7fa0b359e640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa0b40c9970 0x7fa0b41367e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:42.663 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:42.660+0000 7fa0b359e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b4159ee0 con 0x7fa0b40ac6d0 2023-10-26T01:32:42.664 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:42.660+0000 7fa0b359e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b4136d20 con 0x7fa0b409a5c0 2023-10-26T01:32:42.665 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:42.660+0000 7fa0b359e640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b415ae60 con 0x7fa0b40c9970 2023-10-26T01:32:42.665 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:42.660+0000 7fa0b1d9b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa0b40ac6d0 0x7fa0b41599a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:42.665 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:42.660+0000 7fa0b1d9b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa0b40ac6d0 0x7fa0b41599a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:33964/0 (socket says 172.21.15.169:33964) 2023-10-26T01:32:42.666 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:42.660+0000 7fa0b1d9b640 1 -- 172.21.15.169:0/4077496079 learned_addr learned my addr 172.21.15.169:0/4077496079 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:32:42.666 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:42.660+0000 7fa0b259c640 1 --2- 172.21.15.169:0/4077496079 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa0b409a5c0 0x7fa0b415b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:42.666 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:42.660+0000 7fa0b2d9d640 1 --2- 172.21.15.169:0/4077496079 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa0b40c9970 0x7fa0b41367e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:42.666 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:42.661+0000 7fa0b1d9b640 1 -- 172.21.15.169:0/4077496079 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa0b40c9970 msgr2=0x7fa0b41367e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:42.666 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:42.661+0000 7fa0b1d9b640 1 --2- 172.21.15.169:0/4077496079 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa0b40c9970 0x7fa0b41367e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:42.667 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:42.661+0000 7fa0b1d9b640 1 -- 172.21.15.169:0/4077496079 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa0b409a5c0 msgr2=0x7fa0b415b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:42.667 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:42.661+0000 7fa0b1d9b640 1 --2- 172.21.15.169:0/4077496079 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa0b409a5c0 0x7fa0b415b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:42.667 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:42.661+0000 7fa0b1d9b640 1 -- 172.21.15.169:0/4077496079 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0ac009560 con 0x7fa0b40ac6d0 2023-10-26T01:32:42.667 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:42.661+0000 7fa0b1d9b640 1 --2- 172.21.15.169:0/4077496079 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa0b40ac6d0 0x7fa0b41599a0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fa09c00c970 tx=0x7fa09c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:42.667 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:42.661+0000 7fa0aaffd640 1 -- 172.21.15.169:0/4077496079 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa09c007bb0 con 0x7fa0b40ac6d0 2023-10-26T01:32:42.668 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:42.661+0000 7fa0aaffd640 1 -- 172.21.15.169:0/4077496079 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa09c007e00 con 0x7fa0b40ac6d0 2023-10-26T01:32:42.668 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:42.661+0000 7fa0aaffd640 1 -- 172.21.15.169:0/4077496079 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa09c00f6b0 con 0x7fa0b40ac6d0 2023-10-26T01:32:42.668 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:42.661+0000 7fa0b359e640 1 -- 172.21.15.169:0/4077496079 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0b415a250 con 0x7fa0b40ac6d0 2023-10-26T01:32:42.668 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:42.661+0000 7fa0b359e640 1 -- 172.21.15.169:0/4077496079 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0b415a810 con 0x7fa0b40ac6d0 2023-10-26T01:32:42.669 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:42.663+0000 7fa0b359e640 1 -- 172.21.15.169:0/4077496079 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0b41594d0 con 0x7fa0b40ac6d0 2023-10-26T01:32:42.669 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:42.663+0000 7fa0aaffd640 1 -- 172.21.15.169:0/4077496079 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fa09c00f810 con 0x7fa0b40ac6d0 2023-10-26T01:32:42.669 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:42.664+0000 7fa0aaffd640 1 --2- 172.21.15.169:0/4077496079 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa09406ae70 0x7fa09406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:42.670 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:42.664+0000 7fa0aaffd640 1 -- 172.21.15.169:0/4077496079 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa09c089e00 con 0x7fa0b40ac6d0 2023-10-26T01:32:42.670 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:42.665+0000 7fa0b259c640 1 --2- 172.21.15.169:0/4077496079 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa09406ae70 0x7fa09406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:42.670 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:42.666+0000 7fa0b259c640 1 --2- 172.21.15.169:0/4077496079 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa09406ae70 0x7fa09406d330 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa0a40012e0 tx=0x7fa0a4009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:42.671 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:42.668+0000 7fa0aaffd640 1 -- 172.21.15.169:0/4077496079 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa09c054ea0 con 0x7fa0b40ac6d0 2023-10-26T01:32:42.864 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:42.862+0000 7fa0b359e640 1 -- 172.21.15.169:0/4077496079 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa0b4073c10 con 0x7fa0b40ac6d0 2023-10-26T01:32:42.869 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:42.865+0000 7fa0aaffd640 1 -- 172.21.15.169:0/4077496079 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) v1 ==== 74+0+16561 (secure 0 0 0) 0x7fa09c01d090 con 0x7fa0b40ac6d0 2023-10-26T01:32:42.870 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:32:42.870 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":49,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:32:39.585979+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-26T01:32:36.288217+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-26T01:32:37.059775+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:32:42.871 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:42.867+0000 7fa0b359e640 1 -- 172.21.15.169:0/4077496079 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa09406ae70 msgr2=0x7fa09406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:42.872 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:42.867+0000 7fa0b359e640 1 --2- 172.21.15.169:0/4077496079 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa09406ae70 0x7fa09406d330 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa0a40012e0 tx=0x7fa0a4009040 comp rx=0 tx=0).stop 2023-10-26T01:32:42.872 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:42.867+0000 7fa0b359e640 1 -- 172.21.15.169:0/4077496079 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa0b40ac6d0 msgr2=0x7fa0b41599a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:42.872 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:42.867+0000 7fa0b359e640 1 --2- 172.21.15.169:0/4077496079 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa0b40ac6d0 0x7fa0b41599a0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fa09c00c970 tx=0x7fa09c00ce30 comp rx=0 tx=0).stop 2023-10-26T01:32:42.872 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:42.867+0000 7fa0b359e640 1 -- 172.21.15.169:0/4077496079 shutdown_connections 2023-10-26T01:32:42.872 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:42.867+0000 7fa0b359e640 1 --2- 172.21.15.169:0/4077496079 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa09406ae70 0x7fa09406d330 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:42.873 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:42.867+0000 7fa0b359e640 1 --2- 172.21.15.169:0/4077496079 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa0b40ac6d0 0x7fa0b41599a0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:42.873 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:42.867+0000 7fa0b359e640 1 --2- 172.21.15.169:0/4077496079 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa0b40c9970 0x7fa0b41367e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:42.873 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:42.867+0000 7fa0b359e640 1 --2- 172.21.15.169:0/4077496079 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa0b409a5c0 0x7fa0b415b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:42.873 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:42.867+0000 7fa0b359e640 1 -- 172.21.15.169:0/4077496079 >> 172.21.15.169:0/4077496079 conn(0x7fa0b40aedd0 msgr2=0x7fa0b40aa780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:42.874 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:42.868+0000 7fa0b359e640 1 -- 172.21.15.169:0/4077496079 shutdown_connections 2023-10-26T01:32:42.874 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:42.868+0000 7fa0b359e640 1 -- 172.21.15.169:0/4077496079 wait complete. 2023-10-26T01:32:42.887 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-26T01:32:42.887 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-26T01:32:43.098 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:43.096+0000 7fec76847640 1 -- 172.21.15.169:0/3736220688 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec7009a5c0 msgr2=0x7fec7009aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:43.098 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:43.096+0000 7fec76847640 1 --2- 172.21.15.169:0/3736220688 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec7009a5c0 0x7fec7009aa70 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fec68005fd0 tx=0x7fec6802c2d0 comp rx=0 tx=0).stop 2023-10-26T01:32:43.098 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:43.096+0000 7fec76847640 1 -- 172.21.15.169:0/3736220688 shutdown_connections 2023-10-26T01:32:43.099 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:43.096+0000 7fec76847640 1 --2- 172.21.15.169:0/3736220688 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec7009a5c0 0x7fec7009aa70 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:43.099 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:43.096+0000 7fec76847640 1 --2- 172.21.15.169:0/3736220688 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fec700c9970 0x7fec70098a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:43.099 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:43.096+0000 7fec76847640 1 -- 172.21.15.169:0/3736220688 >> 172.21.15.169:0/3736220688 conn(0x7fec700aedd0 msgr2=0x7fec700af1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:43.099 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:43.096+0000 7fec76847640 1 -- 172.21.15.169:0/3736220688 shutdown_connections 2023-10-26T01:32:43.099 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:43.096+0000 7fec76847640 1 -- 172.21.15.169:0/3736220688 wait complete. 2023-10-26T01:32:43.102 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:43.098+0000 7fec76847640 1 Processor -- start 2023-10-26T01:32:43.102 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:43.098+0000 7fec76847640 1 -- start start 2023-10-26T01:32:43.102 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:43.099+0000 7fec76847640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fec7009a5c0 0x7fec701eefb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:43.102 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:43.099+0000 7fec76847640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fec700ac6d0 0x7fec701ef4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:43.103 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:43.099+0000 7fec76847640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec700c9970 0x7fec7006eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:43.103 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:43.099+0000 7fec76847640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec701efa30 con 0x7fec700c9970 2023-10-26T01:32:43.103 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:43.099+0000 7fec76847640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec70070840 con 0x7fec7009a5c0 2023-10-26T01:32:43.103 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:43.099+0000 7fec76847640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec70070b20 con 0x7fec700ac6d0 2023-10-26T01:32:43.103 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:43.099+0000 7fec76046640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec700c9970 0x7fec7006eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:43.104 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:43.099+0000 7fec76046640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec700c9970 0x7fec7006eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:33980/0 (socket says 172.21.15.169:33980) 2023-10-26T01:32:43.104 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:43.099+0000 7fec76046640 1 -- 172.21.15.169:0/888862556 learned_addr learned my addr 172.21.15.169:0/888862556 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:32:43.104 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:43.099+0000 7fec76046640 1 -- 172.21.15.169:0/888862556 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fec700ac6d0 msgr2=0x7fec701ef4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:43.104 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:43.099+0000 7fec76046640 1 --2- 172.21.15.169:0/888862556 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fec700ac6d0 0x7fec701ef4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:43.105 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:43.099+0000 7fec75845640 1 --2- 172.21.15.169:0/888862556 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fec7009a5c0 0x7fec701eefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:43.105 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:43.099+0000 7fec76046640 1 -- 172.21.15.169:0/888862556 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fec7009a5c0 msgr2=0x7fec701eefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:43.105 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:43.099+0000 7fec76046640 1 --2- 172.21.15.169:0/888862556 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fec7009a5c0 0x7fec701eefb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:43.105 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:43.099+0000 7fec76046640 1 -- 172.21.15.169:0/888862556 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec68009900 con 0x7fec700c9970 2023-10-26T01:32:43.105 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:43.099+0000 7fec75845640 1 --2- 172.21.15.169:0/888862556 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fec7009a5c0 0x7fec701eefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:32:43.106 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:43.100+0000 7fec76046640 1 --2- 172.21.15.169:0/888862556 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec700c9970 0x7fec7006eb60 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fec68009bf0 tx=0x7fec68003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:43.106 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:43.100+0000 7fec667fc640 1 -- 172.21.15.169:0/888862556 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec68004450 con 0x7fec700c9970 2023-10-26T01:32:43.106 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:43.100+0000 7fec667fc640 1 -- 172.21.15.169:0/888862556 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fec6802de90 con 0x7fec700c9970 2023-10-26T01:32:43.106 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:43.100+0000 7fec76847640 1 -- 172.21.15.169:0/888862556 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec70159a40 con 0x7fec700c9970 2023-10-26T01:32:43.106 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:43.100+0000 7fec667fc640 1 -- 172.21.15.169:0/888862556 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec68003e10 con 0x7fec700c9970 2023-10-26T01:32:43.107 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:43.100+0000 7fec76847640 1 -- 172.21.15.169:0/888862556 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec7015a000 con 0x7fec700c9970 2023-10-26T01:32:43.108 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:43.101+0000 7fec76847640 1 -- 172.21.15.169:0/888862556 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec70073c10 con 0x7fec700c9970 2023-10-26T01:32:43.108 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:43.102+0000 7fec667fc640 1 -- 172.21.15.169:0/888862556 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fec6800a040 con 0x7fec700c9970 2023-10-26T01:32:43.108 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:43.103+0000 7fec667fc640 1 --2- 172.21.15.169:0/888862556 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fec5406ae40 0x7fec5406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:43.108 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:43.103+0000 7fec667fc640 1 -- 172.21.15.169:0/888862556 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fec680ac680 con 0x7fec700c9970 2023-10-26T01:32:43.109 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:43.104+0000 7fec75845640 1 --2- 172.21.15.169:0/888862556 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fec5406ae40 0x7fec5406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:43.109 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:43.105+0000 7fec75845640 1 --2- 172.21.15.169:0/888862556 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fec5406ae40 0x7fec5406d300 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fec6c001680 tx=0x7fec6c009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:43.109 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:43.108+0000 7fec667fc640 1 -- 172.21.15.169:0/888862556 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fec68077840 con 0x7fec700c9970 2023-10-26T01:32:43.297 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:43.295+0000 7fec76847640 1 -- 172.21.15.169:0/888862556 --> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fec70098fa0 con 0x7fec5406ae40 2023-10-26T01:32:43.320 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:43.319+0000 7fec667fc640 1 -- 172.21.15.169:0/888862556 <== mgr.4112 v2:172.21.15.170:6832/1187156918 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+282549 (secure 0 0 0) 0x7fec6c001c60 con 0x7fec5406ae40 2023-10-26T01:32:43.325 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:32:43.326 INFO:teuthology.orchestra.run.smithi169.stderr:dumped all 2023-10-26T01:32:43.328 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:43.325+0000 7fec76847640 1 -- 172.21.15.169:0/888862556 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fec5406ae40 msgr2=0x7fec5406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:43.329 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:43.325+0000 7fec76847640 1 --2- 172.21.15.169:0/888862556 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fec5406ae40 0x7fec5406d300 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fec6c001680 tx=0x7fec6c009040 comp rx=0 tx=0).stop 2023-10-26T01:32:43.329 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:43.325+0000 7fec76847640 1 -- 172.21.15.169:0/888862556 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec700c9970 msgr2=0x7fec7006eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:43.329 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:43.325+0000 7fec76847640 1 --2- 172.21.15.169:0/888862556 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec700c9970 0x7fec7006eb60 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fec68009bf0 tx=0x7fec68003c20 comp rx=0 tx=0).stop 2023-10-26T01:32:43.329 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:43.325+0000 7fec76847640 1 -- 172.21.15.169:0/888862556 shutdown_connections 2023-10-26T01:32:43.330 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:43.325+0000 7fec76847640 1 --2- 172.21.15.169:0/888862556 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fec5406ae40 0x7fec5406d300 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:43.330 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:43.325+0000 7fec76847640 1 --2- 172.21.15.169:0/888862556 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec700c9970 0x7fec7006eb60 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:43.330 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:43.325+0000 7fec76847640 1 --2- 172.21.15.169:0/888862556 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fec700ac6d0 0x7fec701ef4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:43.330 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:43.325+0000 7fec76847640 1 --2- 172.21.15.169:0/888862556 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fec7009a5c0 0x7fec701eefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:43.330 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:43.325+0000 7fec76847640 1 -- 172.21.15.169:0/888862556 >> 172.21.15.169:0/888862556 conn(0x7fec700aedd0 msgr2=0x7fec700aac80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:43.331 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:43.325+0000 7fec76847640 1 -- 172.21.15.169:0/888862556 shutdown_connections 2023-10-26T01:32:43.331 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:43.325+0000 7fec76847640 1 -- 172.21.15.169:0/888862556 wait complete. 2023-10-26T01:32:43.346 INFO:teuthology.orchestra.run.smithi169.stdout:{"pg_ready":true,"pg_map":{"version":108,"stamp":"2023-10-26T01:32:42.170355+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":105,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":2982,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1559517,"statfs":{"total":1600000000,"available":1596945802,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.605679"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-10-26T01:32:38.615756+0000","last_change":"2023-10-26T01:32:38.615565+0000","last_active":"2023-10-26T01:32:38.615756+0000","last_peered":"2023-10-26T01:32:38.615756+0000","last_clean":"2023-10-26T01:32:38.615756+0000","last_became_active":"2023-10-26T01:32:38.614975+0000","last_became_peered":"2023-10-26T01:32:38.614975+0000","last_unstale":"2023-10-26T01:32:38.615756+0000","last_undegraded":"2023-10-26T01:32:38.615756+0000","last_fullsized":"2023-10-26T01:32:38.615756+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:30:05.266141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-10-26T01:32:38.616971+0000","last_change":"2023-10-26T01:32:38.616364+0000","last_active":"2023-10-26T01:32:38.616971+0000","last_peered":"2023-10-26T01:32:38.616971+0000","last_clean":"2023-10-26T01:32:38.616971+0000","last_became_active":"2023-10-26T01:32:38.615848+0000","last_became_peered":"2023-10-26T01:32:38.615848+0000","last_unstale":"2023-10-26T01:32:38.616971+0000","last_undegraded":"2023-10-26T01:32:38.616971+0000","last_fullsized":"2023-10-26T01:32:38.616971+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:43:37.338787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:39.592404+0000","last_change":"2023-10-26T01:32:38.595326+0000","last_active":"2023-10-26T01:32:39.592404+0000","last_peered":"2023-10-26T01:32:39.592404+0000","last_clean":"2023-10-26T01:32:39.592404+0000","last_became_active":"2023-10-26T01:32:38.594657+0000","last_became_peered":"2023-10-26T01:32:38.594657+0000","last_unstale":"2023-10-26T01:32:39.592404+0000","last_undegraded":"2023-10-26T01:32:39.592404+0000","last_fullsized":"2023-10-26T01:32:39.592404+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:16:44.879669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:39.590215+0000","last_change":"2023-10-26T01:32:38.622436+0000","last_active":"2023-10-26T01:32:39.590215+0000","last_peered":"2023-10-26T01:32:39.590215+0000","last_clean":"2023-10-26T01:32:39.590215+0000","last_became_active":"2023-10-26T01:32:38.620852+0000","last_became_peered":"2023-10-26T01:32:38.620852+0000","last_unstale":"2023-10-26T01:32:39.590215+0000","last_undegraded":"2023-10-26T01:32:39.590215+0000","last_fullsized":"2023-10-26T01:32:39.590215+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:30:05.266141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-10-26T01:32:38.618242+0000","last_change":"2023-10-26T01:32:38.617484+0000","last_active":"2023-10-26T01:32:38.618242+0000","last_peered":"2023-10-26T01:32:38.618242+0000","last_clean":"2023-10-26T01:32:38.618242+0000","last_became_active":"2023-10-26T01:32:38.617191+0000","last_became_peered":"2023-10-26T01:32:38.617191+0000","last_unstale":"2023-10-26T01:32:38.618242+0000","last_undegraded":"2023-10-26T01:32:38.618242+0000","last_fullsized":"2023-10-26T01:32:38.618242+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:54:49.271825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:40.929367+0000","last_change":"2023-10-26T01:32:38.619985+0000","last_active":"2023-10-26T01:32:40.929367+0000","last_peered":"2023-10-26T01:32:40.929367+0000","last_clean":"2023-10-26T01:32:40.929367+0000","last_became_active":"2023-10-26T01:32:38.618984+0000","last_became_peered":"2023-10-26T01:32:38.618984+0000","last_unstale":"2023-10-26T01:32:40.929367+0000","last_undegraded":"2023-10-26T01:32:40.929367+0000","last_fullsized":"2023-10-26T01:32:40.929367+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:26:27.795851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-10-26T01:32:38.623704+0000","last_change":"2023-10-26T01:32:38.623635+0000","last_active":"2023-10-26T01:32:38.623704+0000","last_peered":"2023-10-26T01:32:38.623704+0000","last_clean":"2023-10-26T01:32:38.623704+0000","last_became_active":"2023-10-26T01:32:38.623340+0000","last_became_peered":"2023-10-26T01:32:38.623340+0000","last_unstale":"2023-10-26T01:32:38.623704+0000","last_undegraded":"2023-10-26T01:32:38.623704+0000","last_fullsized":"2023-10-26T01:32:38.623704+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:16:44.879669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:39.590338+0000","last_change":"2023-10-26T01:32:38.624621+0000","last_active":"2023-10-26T01:32:39.590338+0000","last_peered":"2023-10-26T01:32:39.590338+0000","last_clean":"2023-10-26T01:32:39.590338+0000","last_became_active":"2023-10-26T01:32:38.624350+0000","last_became_peered":"2023-10-26T01:32:38.624350+0000","last_unstale":"2023-10-26T01:32:39.590338+0000","last_undegraded":"2023-10-26T01:32:39.590338+0000","last_fullsized":"2023-10-26T01:32:39.590338+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:04:47.581955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:39.590483+0000","last_change":"2023-10-26T01:32:38.621939+0000","last_active":"2023-10-26T01:32:39.590483+0000","last_peered":"2023-10-26T01:32:39.590483+0000","last_clean":"2023-10-26T01:32:39.590483+0000","last_became_active":"2023-10-26T01:32:38.619273+0000","last_became_peered":"2023-10-26T01:32:38.619273+0000","last_unstale":"2023-10-26T01:32:39.590483+0000","last_undegraded":"2023-10-26T01:32:39.590483+0000","last_fullsized":"2023-10-26T01:32:39.590483+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:08:15.316270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:39.590282+0000","last_change":"2023-10-26T01:32:38.624622+0000","last_active":"2023-10-26T01:32:39.590282+0000","last_peered":"2023-10-26T01:32:39.590282+0000","last_clean":"2023-10-26T01:32:39.590282+0000","last_became_active":"2023-10-26T01:32:38.623011+0000","last_became_peered":"2023-10-26T01:32:38.623011+0000","last_unstale":"2023-10-26T01:32:39.590282+0000","last_undegraded":"2023-10-26T01:32:39.590282+0000","last_fullsized":"2023-10-26T01:32:39.590282+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:50:36.425321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:39.590424+0000","last_change":"2023-10-26T01:32:38.624321+0000","last_active":"2023-10-26T01:32:39.590424+0000","last_peered":"2023-10-26T01:32:39.590424+0000","last_clean":"2023-10-26T01:32:39.590424+0000","last_became_active":"2023-10-26T01:32:38.623205+0000","last_became_peered":"2023-10-26T01:32:38.623205+0000","last_unstale":"2023-10-26T01:32:39.590424+0000","last_undegraded":"2023-10-26T01:32:39.590424+0000","last_fullsized":"2023-10-26T01:32:39.590424+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:04:08.255284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-10-26T01:32:38.616247+0000","last_change":"2023-10-26T01:32:38.616153+0000","last_active":"2023-10-26T01:32:38.616247+0000","last_peered":"2023-10-26T01:32:38.616247+0000","last_clean":"2023-10-26T01:32:38.616247+0000","last_became_active":"2023-10-26T01:32:38.615221+0000","last_became_peered":"2023-10-26T01:32:38.615221+0000","last_unstale":"2023-10-26T01:32:38.616247+0000","last_undegraded":"2023-10-26T01:32:38.616247+0000","last_fullsized":"2023-10-26T01:32:38.616247+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:16:44.879669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:39.591374+0000","last_change":"2023-10-26T01:32:38.605455+0000","last_active":"2023-10-26T01:32:39.591374+0000","last_peered":"2023-10-26T01:32:39.591374+0000","last_clean":"2023-10-26T01:32:39.591374+0000","last_became_active":"2023-10-26T01:32:38.604417+0000","last_became_peered":"2023-10-26T01:32:38.604417+0000","last_unstale":"2023-10-26T01:32:39.591374+0000","last_undegraded":"2023-10-26T01:32:39.591374+0000","last_fullsized":"2023-10-26T01:32:39.591374+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:03:56.612349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:39.591841+0000","last_change":"2023-10-26T01:32:38.623376+0000","last_active":"2023-10-26T01:32:39.591841+0000","last_peered":"2023-10-26T01:32:39.591841+0000","last_clean":"2023-10-26T01:32:39.591841+0000","last_became_active":"2023-10-26T01:32:38.622913+0000","last_became_peered":"2023-10-26T01:32:38.622913+0000","last_unstale":"2023-10-26T01:32:39.591841+0000","last_undegraded":"2023-10-26T01:32:39.591841+0000","last_fullsized":"2023-10-26T01:32:39.591841+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:19:50.831275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:40.929599+0000","last_change":"2023-10-26T01:32:38.614426+0000","last_active":"2023-10-26T01:32:40.929599+0000","last_peered":"2023-10-26T01:32:40.929599+0000","last_clean":"2023-10-26T01:32:40.929599+0000","last_became_active":"2023-10-26T01:32:38.612534+0000","last_became_peered":"2023-10-26T01:32:38.612534+0000","last_unstale":"2023-10-26T01:32:40.929599+0000","last_undegraded":"2023-10-26T01:32:40.929599+0000","last_fullsized":"2023-10-26T01:32:40.929599+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:59:37.874486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:40.835137+0000","last_change":"2023-10-26T01:32:38.624735+0000","last_active":"2023-10-26T01:32:40.835137+0000","last_peered":"2023-10-26T01:32:40.835137+0000","last_clean":"2023-10-26T01:32:40.835137+0000","last_became_active":"2023-10-26T01:32:38.624173+0000","last_became_peered":"2023-10-26T01:32:38.624173+0000","last_unstale":"2023-10-26T01:32:40.835137+0000","last_undegraded":"2023-10-26T01:32:40.835137+0000","last_fullsized":"2023-10-26T01:32:40.835137+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:37:33.515180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:39.592327+0000","last_change":"2023-10-26T01:32:38.613058+0000","last_active":"2023-10-26T01:32:39.592327+0000","last_peered":"2023-10-26T01:32:39.592327+0000","last_clean":"2023-10-26T01:32:39.592327+0000","last_became_active":"2023-10-26T01:32:38.609992+0000","last_became_peered":"2023-10-26T01:32:38.609992+0000","last_unstale":"2023-10-26T01:32:39.592327+0000","last_undegraded":"2023-10-26T01:32:39.592327+0000","last_fullsized":"2023-10-26T01:32:39.592327+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:54:49.271825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:39.593235+0000","last_change":"2023-10-26T01:32:38.617399+0000","last_active":"2023-10-26T01:32:39.593235+0000","last_peered":"2023-10-26T01:32:39.593235+0000","last_clean":"2023-10-26T01:32:39.593235+0000","last_became_active":"2023-10-26T01:32:38.617088+0000","last_became_peered":"2023-10-26T01:32:38.617088+0000","last_unstale":"2023-10-26T01:32:39.593235+0000","last_undegraded":"2023-10-26T01:32:39.593235+0000","last_fullsized":"2023-10-26T01:32:39.593235+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:50:36.425321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:40.836411+0000","last_change":"2023-10-26T01:32:38.619668+0000","last_active":"2023-10-26T01:32:40.836411+0000","last_peered":"2023-10-26T01:32:40.836411+0000","last_clean":"2023-10-26T01:32:40.836411+0000","last_became_active":"2023-10-26T01:32:38.619399+0000","last_became_peered":"2023-10-26T01:32:38.619399+0000","last_unstale":"2023-10-26T01:32:40.836411+0000","last_undegraded":"2023-10-26T01:32:40.836411+0000","last_fullsized":"2023-10-26T01:32:40.836411+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:10:25.558471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:39.592011+0000","last_change":"2023-10-26T01:32:38.617944+0000","last_active":"2023-10-26T01:32:39.592011+0000","last_peered":"2023-10-26T01:32:39.592011+0000","last_clean":"2023-10-26T01:32:39.592011+0000","last_became_active":"2023-10-26T01:32:38.617706+0000","last_became_peered":"2023-10-26T01:32:38.617706+0000","last_unstale":"2023-10-26T01:32:39.592011+0000","last_undegraded":"2023-10-26T01:32:39.592011+0000","last_fullsized":"2023-10-26T01:32:39.592011+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:01:40.338842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:40.929702+0000","last_change":"2023-10-26T01:32:38.615679+0000","last_active":"2023-10-26T01:32:40.929702+0000","last_peered":"2023-10-26T01:32:40.929702+0000","last_clean":"2023-10-26T01:32:40.929702+0000","last_became_active":"2023-10-26T01:32:38.615259+0000","last_became_peered":"2023-10-26T01:32:38.615259+0000","last_unstale":"2023-10-26T01:32:40.929702+0000","last_undegraded":"2023-10-26T01:32:40.929702+0000","last_fullsized":"2023-10-26T01:32:40.929702+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:19:12.202398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:39.590429+0000","last_change":"2023-10-26T01:32:38.622081+0000","last_active":"2023-10-26T01:32:39.590429+0000","last_peered":"2023-10-26T01:32:39.590429+0000","last_clean":"2023-10-26T01:32:39.590429+0000","last_became_active":"2023-10-26T01:32:38.621705+0000","last_became_peered":"2023-10-26T01:32:38.621705+0000","last_unstale":"2023-10-26T01:32:39.590429+0000","last_undegraded":"2023-10-26T01:32:39.590429+0000","last_fullsized":"2023-10-26T01:32:39.590429+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:21:07.355052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:40.835559+0000","last_change":"2023-10-26T01:32:38.616743+0000","last_active":"2023-10-26T01:32:40.835559+0000","last_peered":"2023-10-26T01:32:40.835559+0000","last_clean":"2023-10-26T01:32:40.835559+0000","last_became_active":"2023-10-26T01:32:38.616467+0000","last_became_peered":"2023-10-26T01:32:38.616467+0000","last_unstale":"2023-10-26T01:32:40.835559+0000","last_undegraded":"2023-10-26T01:32:40.835559+0000","last_fullsized":"2023-10-26T01:32:40.835559+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:13:11.537665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:40.835380+0000","last_change":"2023-10-26T01:32:38.615017+0000","last_active":"2023-10-26T01:32:40.835380+0000","last_peered":"2023-10-26T01:32:40.835380+0000","last_clean":"2023-10-26T01:32:40.835380+0000","last_became_active":"2023-10-26T01:32:38.614351+0000","last_became_peered":"2023-10-26T01:32:38.614351+0000","last_unstale":"2023-10-26T01:32:40.835380+0000","last_undegraded":"2023-10-26T01:32:40.835380+0000","last_fullsized":"2023-10-26T01:32:40.835380+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:53:59.515844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:40.835567+0000","last_change":"2023-10-26T01:32:38.625225+0000","last_active":"2023-10-26T01:32:40.835567+0000","last_peered":"2023-10-26T01:32:40.835567+0000","last_clean":"2023-10-26T01:32:40.835567+0000","last_became_active":"2023-10-26T01:32:38.624953+0000","last_became_peered":"2023-10-26T01:32:38.624953+0000","last_unstale":"2023-10-26T01:32:40.835567+0000","last_undegraded":"2023-10-26T01:32:40.835567+0000","last_fullsized":"2023-10-26T01:32:40.835567+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:43:37.338787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:39.591821+0000","last_change":"2023-10-26T01:32:38.617454+0000","last_active":"2023-10-26T01:32:39.591821+0000","last_peered":"2023-10-26T01:32:39.591821+0000","last_clean":"2023-10-26T01:32:39.591821+0000","last_became_active":"2023-10-26T01:32:38.617104+0000","last_became_peered":"2023-10-26T01:32:38.617104+0000","last_unstale":"2023-10-26T01:32:39.591821+0000","last_undegraded":"2023-10-26T01:32:39.591821+0000","last_fullsized":"2023-10-26T01:32:39.591821+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:52:40.966550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:39.592682+0000","last_change":"2023-10-26T01:32:38.613292+0000","last_active":"2023-10-26T01:32:39.592682+0000","last_peered":"2023-10-26T01:32:39.592682+0000","last_clean":"2023-10-26T01:32:39.592682+0000","last_became_active":"2023-10-26T01:32:38.611461+0000","last_became_peered":"2023-10-26T01:32:38.611461+0000","last_unstale":"2023-10-26T01:32:39.592682+0000","last_undegraded":"2023-10-26T01:32:39.592682+0000","last_fullsized":"2023-10-26T01:32:39.592682+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:57:36.439738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-10-26T01:32:38.620646+0000","last_change":"2023-10-26T01:32:38.620512+0000","last_active":"2023-10-26T01:32:38.620646+0000","last_peered":"2023-10-26T01:32:38.620646+0000","last_clean":"2023-10-26T01:32:38.620646+0000","last_became_active":"2023-10-26T01:32:38.620187+0000","last_became_peered":"2023-10-26T01:32:38.620187+0000","last_unstale":"2023-10-26T01:32:38.620646+0000","last_undegraded":"2023-10-26T01:32:38.620646+0000","last_fullsized":"2023-10-26T01:32:38.620646+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:30:05.266141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:40.835793+0000","last_change":"2023-10-26T01:32:38.624506+0000","last_active":"2023-10-26T01:32:40.835793+0000","last_peered":"2023-10-26T01:32:40.835793+0000","last_clean":"2023-10-26T01:32:40.835793+0000","last_became_active":"2023-10-26T01:32:38.623710+0000","last_became_peered":"2023-10-26T01:32:38.623710+0000","last_unstale":"2023-10-26T01:32:40.835793+0000","last_undegraded":"2023-10-26T01:32:40.835793+0000","last_fullsized":"2023-10-26T01:32:40.835793+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:34:02.809272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:39.592533+0000","last_change":"2023-10-26T01:32:38.612032+0000","last_active":"2023-10-26T01:32:39.592533+0000","last_peered":"2023-10-26T01:32:39.592533+0000","last_clean":"2023-10-26T01:32:39.592533+0000","last_became_active":"2023-10-26T01:32:38.611776+0000","last_became_peered":"2023-10-26T01:32:38.611776+0000","last_unstale":"2023-10-26T01:32:39.592533+0000","last_undegraded":"2023-10-26T01:32:39.592533+0000","last_fullsized":"2023-10-26T01:32:39.592533+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:10:25.558471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:40.929982+0000","last_change":"2023-10-26T01:32:38.619614+0000","last_active":"2023-10-26T01:32:40.929982+0000","last_peered":"2023-10-26T01:32:40.929982+0000","last_clean":"2023-10-26T01:32:40.929982+0000","last_became_active":"2023-10-26T01:32:38.617768+0000","last_became_peered":"2023-10-26T01:32:38.617768+0000","last_unstale":"2023-10-26T01:32:40.929982+0000","last_undegraded":"2023-10-26T01:32:40.929982+0000","last_fullsized":"2023-10-26T01:32:40.929982+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:19:35.058721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:39.592372+0000","last_change":"2023-10-26T01:32:38.594730+0000","last_active":"2023-10-26T01:32:39.592372+0000","last_peered":"2023-10-26T01:32:39.592372+0000","last_clean":"2023-10-26T01:32:39.592372+0000","last_became_active":"2023-10-26T01:32:38.594212+0000","last_became_peered":"2023-10-26T01:32:38.594212+0000","last_unstale":"2023-10-26T01:32:39.592372+0000","last_undegraded":"2023-10-26T01:32:39.592372+0000","last_fullsized":"2023-10-26T01:32:39.592372+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:04:54.610130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-10-26T01:32:38.622387+0000","last_change":"2023-10-26T01:32:38.622321+0000","last_active":"2023-10-26T01:32:38.622387+0000","last_peered":"2023-10-26T01:32:38.622387+0000","last_clean":"2023-10-26T01:32:38.622387+0000","last_became_active":"2023-10-26T01:32:38.622049+0000","last_became_peered":"2023-10-26T01:32:38.622049+0000","last_unstale":"2023-10-26T01:32:38.622387+0000","last_undegraded":"2023-10-26T01:32:38.622387+0000","last_fullsized":"2023-10-26T01:32:38.622387+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:23:00.714244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"48'4","reported_seq":16,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-10-26T01:32:38.932586+0000","last_change":"2023-10-26T01:32:37.607858+0000","last_active":"2023-10-26T01:32:38.932586+0000","last_peered":"2023-10-26T01:32:38.932586+0000","last_clean":"2023-10-26T01:32:38.932586+0000","last_became_active":"2023-10-26T01:32:37.607551+0000","last_became_peered":"2023-10-26T01:32:37.607551+0000","last_unstale":"2023-10-26T01:32:38.932586+0000","last_undegraded":"2023-10-26T01:32:38.932586+0000","last_fullsized":"2023-10-26T01:32:38.932586+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_clean_scrub_stamp":"2023-10-26T01:32:36.550883+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:04:07.242179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:40.835218+0000","last_change":"2023-10-26T01:32:38.612838+0000","last_active":"2023-10-26T01:32:40.835218+0000","last_peered":"2023-10-26T01:32:40.835218+0000","last_clean":"2023-10-26T01:32:40.835218+0000","last_became_active":"2023-10-26T01:32:38.612184+0000","last_became_peered":"2023-10-26T01:32:38.612184+0000","last_unstale":"2023-10-26T01:32:40.835218+0000","last_undegraded":"2023-10-26T01:32:40.835218+0000","last_fullsized":"2023-10-26T01:32:40.835218+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:53:18.288374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"48'1","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:39.591929+0000","last_change":"2023-10-26T01:32:37.609023+0000","last_active":"2023-10-26T01:32:39.591929+0000","last_peered":"2023-10-26T01:32:39.591929+0000","last_clean":"2023-10-26T01:32:39.591929+0000","last_became_active":"2023-10-26T01:32:37.608768+0000","last_became_peered":"2023-10-26T01:32:37.608768+0000","last_unstale":"2023-10-26T01:32:39.591929+0000","last_undegraded":"2023-10-26T01:32:39.591929+0000","last_fullsized":"2023-10-26T01:32:39.591929+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_clean_scrub_stamp":"2023-10-26T01:32:36.550883+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:42:35.797520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-10-26T01:32:38.615352+0000","last_change":"2023-10-26T01:32:38.615248+0000","last_active":"2023-10-26T01:32:38.615352+0000","last_peered":"2023-10-26T01:32:38.615352+0000","last_clean":"2023-10-26T01:32:38.615352+0000","last_became_active":"2023-10-26T01:32:38.614715+0000","last_became_peered":"2023-10-26T01:32:38.614715+0000","last_unstale":"2023-10-26T01:32:38.615352+0000","last_undegraded":"2023-10-26T01:32:38.615352+0000","last_fullsized":"2023-10-26T01:32:38.615352+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:57:36.439738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"48'2","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-10-26T01:32:38.929368+0000","last_change":"2023-10-26T01:32:37.604839+0000","last_active":"2023-10-26T01:32:38.929368+0000","last_peered":"2023-10-26T01:32:38.929368+0000","last_clean":"2023-10-26T01:32:38.929368+0000","last_became_active":"2023-10-26T01:32:37.604415+0000","last_became_peered":"2023-10-26T01:32:37.604415+0000","last_unstale":"2023-10-26T01:32:38.929368+0000","last_undegraded":"2023-10-26T01:32:38.929368+0000","last_fullsized":"2023-10-26T01:32:38.929368+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_clean_scrub_stamp":"2023-10-26T01:32:36.550883+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:43:36.325682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-10-26T01:32:38.624709+0000","last_change":"2023-10-26T01:32:38.624444+0000","last_active":"2023-10-26T01:32:38.624709+0000","last_peered":"2023-10-26T01:32:38.624709+0000","last_clean":"2023-10-26T01:32:38.624709+0000","last_became_active":"2023-10-26T01:32:38.624081+0000","last_became_peered":"2023-10-26T01:32:38.624081+0000","last_unstale":"2023-10-26T01:32:38.624709+0000","last_undegraded":"2023-10-26T01:32:38.624709+0000","last_fullsized":"2023-10-26T01:32:38.624709+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:52:34.748242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"48'2","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-10-26T01:32:38.936575+0000","last_change":"2023-10-26T01:32:37.604413+0000","last_active":"2023-10-26T01:32:38.936575+0000","last_peered":"2023-10-26T01:32:38.936575+0000","last_clean":"2023-10-26T01:32:38.936575+0000","last_became_active":"2023-10-26T01:32:37.604186+0000","last_became_peered":"2023-10-26T01:32:37.604186+0000","last_unstale":"2023-10-26T01:32:38.936575+0000","last_undegraded":"2023-10-26T01:32:38.936575+0000","last_fullsized":"2023-10-26T01:32:38.936575+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_clean_scrub_stamp":"2023-10-26T01:32:36.550883+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:04:46.568850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:39.592059+0000","last_change":"2023-10-26T01:32:38.610012+0000","last_active":"2023-10-26T01:32:39.592059+0000","last_peered":"2023-10-26T01:32:39.592059+0000","last_clean":"2023-10-26T01:32:39.592059+0000","last_became_active":"2023-10-26T01:32:38.609568+0000","last_became_peered":"2023-10-26T01:32:38.609568+0000","last_unstale":"2023-10-26T01:32:39.592059+0000","last_undegraded":"2023-10-26T01:32:39.592059+0000","last_fullsized":"2023-10-26T01:32:39.592059+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T01:45:53.551623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"48'1","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-10-26T01:32:38.933907+0000","last_change":"2023-10-26T01:32:37.595329+0000","last_active":"2023-10-26T01:32:38.933907+0000","last_peered":"2023-10-26T01:32:38.933907+0000","last_clean":"2023-10-26T01:32:38.933907+0000","last_became_active":"2023-10-26T01:32:37.594957+0000","last_became_peered":"2023-10-26T01:32:37.594957+0000","last_unstale":"2023-10-26T01:32:38.933907+0000","last_undegraded":"2023-10-26T01:32:38.933907+0000","last_fullsized":"2023-10-26T01:32:38.933907+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_clean_scrub_stamp":"2023-10-26T01:32:36.550883+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:50:35.412216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:39.590541+0000","last_change":"2023-10-26T01:32:38.617840+0000","last_active":"2023-10-26T01:32:39.590541+0000","last_peered":"2023-10-26T01:32:39.590541+0000","last_clean":"2023-10-26T01:32:39.590541+0000","last_became_active":"2023-10-26T01:32:38.617418+0000","last_became_peered":"2023-10-26T01:32:38.617418+0000","last_unstale":"2023-10-26T01:32:39.590541+0000","last_undegraded":"2023-10-26T01:32:39.590541+0000","last_fullsized":"2023-10-26T01:32:39.590541+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:35:27.303645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"48'1","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:39.593473+0000","last_change":"2023-10-26T01:32:37.606118+0000","last_active":"2023-10-26T01:32:39.593473+0000","last_peered":"2023-10-26T01:32:39.593473+0000","last_clean":"2023-10-26T01:32:39.593473+0000","last_became_active":"2023-10-26T01:32:37.605531+0000","last_became_peered":"2023-10-26T01:32:37.605531+0000","last_unstale":"2023-10-26T01:32:39.593473+0000","last_undegraded":"2023-10-26T01:32:39.593473+0000","last_fullsized":"2023-10-26T01:32:39.593473+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_clean_scrub_stamp":"2023-10-26T01:32:36.550883+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:34:01.796167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:39.590544+0000","last_change":"2023-10-26T01:32:38.624467+0000","last_active":"2023-10-26T01:32:39.590544+0000","last_peered":"2023-10-26T01:32:39.590544+0000","last_clean":"2023-10-26T01:32:39.590544+0000","last_became_active":"2023-10-26T01:32:38.622694+0000","last_became_peered":"2023-10-26T01:32:38.622694+0000","last_unstale":"2023-10-26T01:32:39.590544+0000","last_undegraded":"2023-10-26T01:32:39.590544+0000","last_fullsized":"2023-10-26T01:32:39.590544+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:47:40.922105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-10-26T01:32:38.602790+0000","last_change":"2023-10-26T01:32:37.605956+0000","last_active":"2023-10-26T01:32:38.602790+0000","last_peered":"2023-10-26T01:32:38.602790+0000","last_clean":"2023-10-26T01:32:38.602790+0000","last_became_active":"2023-10-26T01:32:37.605677+0000","last_became_peered":"2023-10-26T01:32:37.605677+0000","last_unstale":"2023-10-26T01:32:38.602790+0000","last_undegraded":"2023-10-26T01:32:38.602790+0000","last_fullsized":"2023-10-26T01:32:38.602790+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_clean_scrub_stamp":"2023-10-26T01:32:36.550883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:34:01.796167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:40.836112+0000","last_change":"2023-10-26T01:32:38.615178+0000","last_active":"2023-10-26T01:32:40.836112+0000","last_peered":"2023-10-26T01:32:40.836112+0000","last_clean":"2023-10-26T01:32:40.836112+0000","last_became_active":"2023-10-26T01:32:38.614723+0000","last_became_peered":"2023-10-26T01:32:38.614723+0000","last_unstale":"2023-10-26T01:32:40.836112+0000","last_undegraded":"2023-10-26T01:32:40.836112+0000","last_fullsized":"2023-10-26T01:32:40.836112+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:16:44.879669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:39.591197+0000","last_change":"2023-10-26T01:32:37.600205+0000","last_active":"2023-10-26T01:32:39.591197+0000","last_peered":"2023-10-26T01:32:39.591197+0000","last_clean":"2023-10-26T01:32:39.591197+0000","last_became_active":"2023-10-26T01:32:37.599024+0000","last_became_peered":"2023-10-26T01:32:37.599024+0000","last_unstale":"2023-10-26T01:32:39.591197+0000","last_undegraded":"2023-10-26T01:32:39.591197+0000","last_fullsized":"2023-10-26T01:32:39.591197+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_clean_scrub_stamp":"2023-10-26T01:32:36.550883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:57:35.426633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:39.592884+0000","last_change":"2023-10-26T01:32:38.618211+0000","last_active":"2023-10-26T01:32:39.592884+0000","last_peered":"2023-10-26T01:32:39.592884+0000","last_clean":"2023-10-26T01:32:39.592884+0000","last_became_active":"2023-10-26T01:32:38.617357+0000","last_became_peered":"2023-10-26T01:32:38.617357+0000","last_unstale":"2023-10-26T01:32:39.592884+0000","last_undegraded":"2023-10-26T01:32:39.592884+0000","last_fullsized":"2023-10-26T01:32:39.592884+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:30:05.266141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:39.592194+0000","last_change":"2023-10-26T01:32:37.602691+0000","last_active":"2023-10-26T01:32:39.592194+0000","last_peered":"2023-10-26T01:32:39.592194+0000","last_clean":"2023-10-26T01:32:39.592194+0000","last_became_active":"2023-10-26T01:32:37.602281+0000","last_became_peered":"2023-10-26T01:32:37.602281+0000","last_unstale":"2023-10-26T01:32:39.592194+0000","last_undegraded":"2023-10-26T01:32:39.592194+0000","last_fullsized":"2023-10-26T01:32:39.592194+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_clean_scrub_stamp":"2023-10-26T01:32:36.550883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:57:04.375905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-10-26T01:32:38.618194+0000","last_change":"2023-10-26T01:32:38.618000+0000","last_active":"2023-10-26T01:32:38.618194+0000","last_peered":"2023-10-26T01:32:38.618194+0000","last_clean":"2023-10-26T01:32:38.618194+0000","last_became_active":"2023-10-26T01:32:38.617567+0000","last_became_peered":"2023-10-26T01:32:38.617567+0000","last_unstale":"2023-10-26T01:32:38.618194+0000","last_undegraded":"2023-10-26T01:32:38.618194+0000","last_fullsized":"2023-10-26T01:32:38.618194+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:50:36.425321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:39.590685+0000","last_change":"2023-10-26T01:32:37.602681+0000","last_active":"2023-10-26T01:32:39.590685+0000","last_peered":"2023-10-26T01:32:39.590685+0000","last_clean":"2023-10-26T01:32:39.590685+0000","last_became_active":"2023-10-26T01:32:37.602418+0000","last_became_peered":"2023-10-26T01:32:37.602418+0000","last_unstale":"2023-10-26T01:32:39.590685+0000","last_undegraded":"2023-10-26T01:32:39.590685+0000","last_fullsized":"2023-10-26T01:32:39.590685+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_clean_scrub_stamp":"2023-10-26T01:32:36.550883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T01:45:52.538519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:40.928384+0000","last_change":"2023-10-26T01:32:38.619501+0000","last_active":"2023-10-26T01:32:40.928384+0000","last_peered":"2023-10-26T01:32:40.928384+0000","last_clean":"2023-10-26T01:32:40.928384+0000","last_became_active":"2023-10-26T01:32:38.619048+0000","last_became_peered":"2023-10-26T01:32:38.619048+0000","last_unstale":"2023-10-26T01:32:40.928384+0000","last_undegraded":"2023-10-26T01:32:40.928384+0000","last_fullsized":"2023-10-26T01:32:40.928384+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T01:38:57.150100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"48'3","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-10-26T01:32:38.935777+0000","last_change":"2023-10-26T01:32:37.608362+0000","last_active":"2023-10-26T01:32:38.935777+0000","last_peered":"2023-10-26T01:32:38.935777+0000","last_clean":"2023-10-26T01:32:38.935777+0000","last_became_active":"2023-10-26T01:32:37.607893+0000","last_became_peered":"2023-10-26T01:32:37.607893+0000","last_unstale":"2023-10-26T01:32:38.935777+0000","last_undegraded":"2023-10-26T01:32:38.935777+0000","last_fullsized":"2023-10-26T01:32:38.935777+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_clean_scrub_stamp":"2023-10-26T01:32:36.550883+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:54:48.258721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-10-26T01:32:38.618213+0000","last_change":"2023-10-26T01:32:38.618065+0000","last_active":"2023-10-26T01:32:38.618213+0000","last_peered":"2023-10-26T01:32:38.618213+0000","last_clean":"2023-10-26T01:32:38.618213+0000","last_became_active":"2023-10-26T01:32:38.617815+0000","last_became_peered":"2023-10-26T01:32:38.617815+0000","last_unstale":"2023-10-26T01:32:38.618213+0000","last_undegraded":"2023-10-26T01:32:38.618213+0000","last_fullsized":"2023-10-26T01:32:38.618213+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T01:45:53.551623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"48'6","reported_seq":18,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:39.591182+0000","last_change":"2023-10-26T01:32:37.607352+0000","last_active":"2023-10-26T01:32:39.591182+0000","last_peered":"2023-10-26T01:32:39.591182+0000","last_clean":"2023-10-26T01:32:39.591182+0000","last_became_active":"2023-10-26T01:32:37.606840+0000","last_became_peered":"2023-10-26T01:32:37.606840+0000","last_unstale":"2023-10-26T01:32:39.591182+0000","last_undegraded":"2023-10-26T01:32:39.591182+0000","last_fullsized":"2023-10-26T01:32:39.591182+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_clean_scrub_stamp":"2023-10-26T01:32:36.550883+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T01:35:23.718795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:40.835500+0000","last_change":"2023-10-26T01:32:38.624390+0000","last_active":"2023-10-26T01:32:40.835500+0000","last_peered":"2023-10-26T01:32:40.835500+0000","last_clean":"2023-10-26T01:32:40.835500+0000","last_became_active":"2023-10-26T01:32:38.623376+0000","last_became_peered":"2023-10-26T01:32:38.623376+0000","last_unstale":"2023-10-26T01:32:40.835500+0000","last_undegraded":"2023-10-26T01:32:40.835500+0000","last_fullsized":"2023-10-26T01:32:40.835500+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:23:00.714244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"48'2","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-10-26T01:32:38.930197+0000","last_change":"2023-10-26T01:32:37.599916+0000","last_active":"2023-10-26T01:32:38.930197+0000","last_peered":"2023-10-26T01:32:38.930197+0000","last_clean":"2023-10-26T01:32:38.930197+0000","last_became_active":"2023-10-26T01:32:37.599489+0000","last_became_peered":"2023-10-26T01:32:37.599489+0000","last_unstale":"2023-10-26T01:32:38.930197+0000","last_undegraded":"2023-10-26T01:32:38.930197+0000","last_fullsized":"2023-10-26T01:32:38.930197+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_clean_scrub_stamp":"2023-10-26T01:32:36.550883+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:19:49.818171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:39.590692+0000","last_change":"2023-10-26T01:32:38.622440+0000","last_active":"2023-10-26T01:32:39.590692+0000","last_peered":"2023-10-26T01:32:39.590692+0000","last_clean":"2023-10-26T01:32:39.590692+0000","last_became_active":"2023-10-26T01:32:38.621293+0000","last_became_peered":"2023-10-26T01:32:38.621293+0000","last_unstale":"2023-10-26T01:32:39.590692+0000","last_undegraded":"2023-10-26T01:32:39.590692+0000","last_fullsized":"2023-10-26T01:32:39.590692+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:04:30.078480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-10-26T01:32:38.608453+0000","last_change":"2023-10-26T01:32:37.599205+0000","last_active":"2023-10-26T01:32:38.608453+0000","last_peered":"2023-10-26T01:32:38.608453+0000","last_clean":"2023-10-26T01:32:38.608453+0000","last_became_active":"2023-10-26T01:32:37.598725+0000","last_became_peered":"2023-10-26T01:32:37.598725+0000","last_unstale":"2023-10-26T01:32:38.608453+0000","last_undegraded":"2023-10-26T01:32:38.608453+0000","last_fullsized":"2023-10-26T01:32:38.608453+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_clean_scrub_stamp":"2023-10-26T01:32:36.550883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:04:07.242179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:40.928174+0000","last_change":"2023-10-26T01:32:38.613120+0000","last_active":"2023-10-26T01:32:40.928174+0000","last_peered":"2023-10-26T01:32:40.928174+0000","last_clean":"2023-10-26T01:32:40.928174+0000","last_became_active":"2023-10-26T01:32:38.612583+0000","last_became_peered":"2023-10-26T01:32:38.612583+0000","last_unstale":"2023-10-26T01:32:40.928174+0000","last_undegraded":"2023-10-26T01:32:40.928174+0000","last_fullsized":"2023-10-26T01:32:40.928174+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:21:31.844678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-10-26T01:32:38.601749+0000","last_change":"2023-10-26T01:32:37.598390+0000","last_active":"2023-10-26T01:32:38.601749+0000","last_peered":"2023-10-26T01:32:38.601749+0000","last_clean":"2023-10-26T01:32:38.601749+0000","last_became_active":"2023-10-26T01:32:37.598043+0000","last_became_peered":"2023-10-26T01:32:37.598043+0000","last_unstale":"2023-10-26T01:32:38.601749+0000","last_undegraded":"2023-10-26T01:32:38.601749+0000","last_fullsized":"2023-10-26T01:32:38.601749+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_clean_scrub_stamp":"2023-10-26T01:32:36.550883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:04:46.568850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-10-26T01:32:38.613287+0000","last_change":"2023-10-26T01:32:38.612717+0000","last_active":"2023-10-26T01:32:38.613287+0000","last_peered":"2023-10-26T01:32:38.613287+0000","last_clean":"2023-10-26T01:32:38.613287+0000","last_became_active":"2023-10-26T01:32:38.612482+0000","last_became_peered":"2023-10-26T01:32:38.612482+0000","last_unstale":"2023-10-26T01:32:38.613287+0000","last_undegraded":"2023-10-26T01:32:38.613287+0000","last_fullsized":"2023-10-26T01:32:38.613287+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:57:36.439738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"48'8","reported_seq":20,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:39.590761+0000","last_change":"2023-10-26T01:32:37.604769+0000","last_active":"2023-10-26T01:32:39.590761+0000","last_peered":"2023-10-26T01:32:39.590761+0000","last_clean":"2023-10-26T01:32:39.590761+0000","last_became_active":"2023-10-26T01:32:37.604529+0000","last_became_peered":"2023-10-26T01:32:37.604529+0000","last_unstale":"2023-10-26T01:32:39.590761+0000","last_undegraded":"2023-10-26T01:32:39.590761+0000","last_fullsized":"2023-10-26T01:32:39.590761+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_clean_scrub_stamp":"2023-10-26T01:32:36.550883+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:10:24.545366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:39.592127+0000","last_change":"2023-10-26T01:32:38.609050+0000","last_active":"2023-10-26T01:32:39.592127+0000","last_peered":"2023-10-26T01:32:39.592127+0000","last_clean":"2023-10-26T01:32:39.592127+0000","last_became_active":"2023-10-26T01:32:38.606195+0000","last_became_peered":"2023-10-26T01:32:38.606195+0000","last_unstale":"2023-10-26T01:32:39.592127+0000","last_undegraded":"2023-10-26T01:32:39.592127+0000","last_fullsized":"2023-10-26T01:32:39.592127+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T01:43:56.252521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"48'1","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:39.591281+0000","last_change":"2023-10-26T01:32:37.595678+0000","last_active":"2023-10-26T01:32:39.591281+0000","last_peered":"2023-10-26T01:32:39.591281+0000","last_clean":"2023-10-26T01:32:39.591281+0000","last_became_active":"2023-10-26T01:32:37.595294+0000","last_became_peered":"2023-10-26T01:32:37.595294+0000","last_unstale":"2023-10-26T01:32:39.591281+0000","last_undegraded":"2023-10-26T01:32:39.591281+0000","last_fullsized":"2023-10-26T01:32:39.591281+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_clean_scrub_stamp":"2023-10-26T01:32:36.550883+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:31:14.631804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:39.590868+0000","last_change":"2023-10-26T01:32:38.622200+0000","last_active":"2023-10-26T01:32:39.590868+0000","last_peered":"2023-10-26T01:32:39.590868+0000","last_clean":"2023-10-26T01:32:39.590868+0000","last_became_active":"2023-10-26T01:32:38.620470+0000","last_became_peered":"2023-10-26T01:32:38.620470+0000","last_unstale":"2023-10-26T01:32:39.590868+0000","last_undegraded":"2023-10-26T01:32:39.590868+0000","last_fullsized":"2023-10-26T01:32:39.590868+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:28:57.903482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"48'2","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:39.592782+0000","last_change":"2023-10-26T01:32:37.604694+0000","last_active":"2023-10-26T01:32:39.592782+0000","last_peered":"2023-10-26T01:32:39.592782+0000","last_clean":"2023-10-26T01:32:39.592782+0000","last_became_active":"2023-10-26T01:32:37.602692+0000","last_became_peered":"2023-10-26T01:32:37.602692+0000","last_unstale":"2023-10-26T01:32:39.592782+0000","last_undegraded":"2023-10-26T01:32:39.592782+0000","last_fullsized":"2023-10-26T01:32:39.592782+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_clean_scrub_stamp":"2023-10-26T01:32:36.550883+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:53:58.502739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:40.835520+0000","last_change":"2023-10-26T01:32:38.616088+0000","last_active":"2023-10-26T01:32:40.835520+0000","last_peered":"2023-10-26T01:32:40.835520+0000","last_clean":"2023-10-26T01:32:40.835520+0000","last_became_active":"2023-10-26T01:32:38.615753+0000","last_became_peered":"2023-10-26T01:32:38.615753+0000","last_unstale":"2023-10-26T01:32:40.835520+0000","last_undegraded":"2023-10-26T01:32:40.835520+0000","last_fullsized":"2023-10-26T01:32:40.835520+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:52:34.748242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"48'1","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-10-26T01:32:38.932578+0000","last_change":"2023-10-26T01:32:37.605242+0000","last_active":"2023-10-26T01:32:38.932578+0000","last_peered":"2023-10-26T01:32:38.932578+0000","last_clean":"2023-10-26T01:32:38.932578+0000","last_became_active":"2023-10-26T01:32:37.604904+0000","last_became_peered":"2023-10-26T01:32:37.604904+0000","last_unstale":"2023-10-26T01:32:38.932578+0000","last_undegraded":"2023-10-26T01:32:38.932578+0000","last_fullsized":"2023-10-26T01:32:38.932578+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_clean_scrub_stamp":"2023-10-26T01:32:36.550883+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:15:21.947485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:40.835832+0000","last_change":"2023-10-26T01:32:38.624514+0000","last_active":"2023-10-26T01:32:40.835832+0000","last_peered":"2023-10-26T01:32:40.835832+0000","last_clean":"2023-10-26T01:32:40.835832+0000","last_became_active":"2023-10-26T01:32:38.623869+0000","last_became_peered":"2023-10-26T01:32:38.623869+0000","last_unstale":"2023-10-26T01:32:40.835832+0000","last_undegraded":"2023-10-26T01:32:40.835832+0000","last_fullsized":"2023-10-26T01:32:40.835832+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T01:45:53.551623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-10-26T01:32:38.602086+0000","last_change":"2023-10-26T01:32:37.607629+0000","last_active":"2023-10-26T01:32:38.602086+0000","last_peered":"2023-10-26T01:32:38.602086+0000","last_clean":"2023-10-26T01:32:38.602086+0000","last_became_active":"2023-10-26T01:32:37.607318+0000","last_became_peered":"2023-10-26T01:32:37.607318+0000","last_unstale":"2023-10-26T01:32:38.602086+0000","last_undegraded":"2023-10-26T01:32:38.602086+0000","last_fullsized":"2023-10-26T01:32:38.602086+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_clean_scrub_stamp":"2023-10-26T01:32:36.550883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:08:14.303165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:40.928754+0000","last_change":"2023-10-26T01:32:38.619321+0000","last_active":"2023-10-26T01:32:40.928754+0000","last_peered":"2023-10-26T01:32:40.928754+0000","last_clean":"2023-10-26T01:32:40.928754+0000","last_became_active":"2023-10-26T01:32:38.618026+0000","last_became_peered":"2023-10-26T01:32:38.618026+0000","last_unstale":"2023-10-26T01:32:40.928754+0000","last_undegraded":"2023-10-26T01:32:40.928754+0000","last_fullsized":"2023-10-26T01:32:40.928754+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:57:50.250541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"48'2","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-10-26T01:32:38.935163+0000","last_change":"2023-10-26T01:32:37.602118+0000","last_active":"2023-10-26T01:32:38.935163+0000","last_peered":"2023-10-26T01:32:38.935163+0000","last_clean":"2023-10-26T01:32:38.935163+0000","last_became_active":"2023-10-26T01:32:37.601879+0000","last_became_peered":"2023-10-26T01:32:37.601879+0000","last_unstale":"2023-10-26T01:32:38.935163+0000","last_undegraded":"2023-10-26T01:32:38.935163+0000","last_fullsized":"2023-10-26T01:32:38.935163+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_clean_scrub_stamp":"2023-10-26T01:32:36.550883+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:35:26.290541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-10-26T01:32:38.614279+0000","last_change":"2023-10-26T01:32:38.613797+0000","last_active":"2023-10-26T01:32:38.614279+0000","last_peered":"2023-10-26T01:32:38.614279+0000","last_clean":"2023-10-26T01:32:38.614279+0000","last_became_active":"2023-10-26T01:32:38.613369+0000","last_became_peered":"2023-10-26T01:32:38.613369+0000","last_unstale":"2023-10-26T01:32:38.614279+0000","last_undegraded":"2023-10-26T01:32:38.614279+0000","last_fullsized":"2023-10-26T01:32:38.614279+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:53:59.515844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-10-26T01:32:38.603133+0000","last_change":"2023-10-26T01:32:37.609602+0000","last_active":"2023-10-26T01:32:38.603133+0000","last_peered":"2023-10-26T01:32:38.603133+0000","last_clean":"2023-10-26T01:32:38.603133+0000","last_became_active":"2023-10-26T01:32:37.609351+0000","last_became_peered":"2023-10-26T01:32:37.609351+0000","last_unstale":"2023-10-26T01:32:38.603133+0000","last_undegraded":"2023-10-26T01:32:38.603133+0000","last_fullsized":"2023-10-26T01:32:38.603133+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_clean_scrub_stamp":"2023-10-26T01:32:36.550883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T01:45:52.538519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:39.591698+0000","last_change":"2023-10-26T01:32:38.611126+0000","last_active":"2023-10-26T01:32:39.591698+0000","last_peered":"2023-10-26T01:32:39.591698+0000","last_clean":"2023-10-26T01:32:39.591698+0000","last_became_active":"2023-10-26T01:32:38.610609+0000","last_became_peered":"2023-10-26T01:32:38.610609+0000","last_unstale":"2023-10-26T01:32:39.591698+0000","last_undegraded":"2023-10-26T01:32:39.591698+0000","last_fullsized":"2023-10-26T01:32:39.591698+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:59:11.131020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:39.591281+0000","last_change":"2023-10-26T01:32:37.600542+0000","last_active":"2023-10-26T01:32:39.591281+0000","last_peered":"2023-10-26T01:32:39.591281+0000","last_clean":"2023-10-26T01:32:39.591281+0000","last_became_active":"2023-10-26T01:32:37.599775+0000","last_became_peered":"2023-10-26T01:32:37.599775+0000","last_unstale":"2023-10-26T01:32:39.591281+0000","last_undegraded":"2023-10-26T01:32:39.591281+0000","last_fullsized":"2023-10-26T01:32:39.591281+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_clean_scrub_stamp":"2023-10-26T01:32:36.550883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:15:21.947485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:39.590787+0000","last_change":"2023-10-26T01:32:38.624826+0000","last_active":"2023-10-26T01:32:39.590787+0000","last_peered":"2023-10-26T01:32:39.590787+0000","last_clean":"2023-10-26T01:32:39.590787+0000","last_became_active":"2023-10-26T01:32:38.623345+0000","last_became_peered":"2023-10-26T01:32:38.623345+0000","last_unstale":"2023-10-26T01:32:39.590787+0000","last_undegraded":"2023-10-26T01:32:39.590787+0000","last_fullsized":"2023-10-26T01:32:39.590787+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:58:59.610639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-10-26T01:32:38.608627+0000","last_change":"2023-10-26T01:32:37.600656+0000","last_active":"2023-10-26T01:32:38.608627+0000","last_peered":"2023-10-26T01:32:38.608627+0000","last_clean":"2023-10-26T01:32:38.608627+0000","last_became_active":"2023-10-26T01:32:37.600267+0000","last_became_peered":"2023-10-26T01:32:37.600267+0000","last_unstale":"2023-10-26T01:32:38.608627+0000","last_undegraded":"2023-10-26T01:32:38.608627+0000","last_fullsized":"2023-10-26T01:32:38.608627+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_clean_scrub_stamp":"2023-10-26T01:32:36.550883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:21:06.341948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:40.835857+0000","last_change":"2023-10-26T01:32:38.620385+0000","last_active":"2023-10-26T01:32:40.835857+0000","last_peered":"2023-10-26T01:32:40.835857+0000","last_clean":"2023-10-26T01:32:40.835857+0000","last_became_active":"2023-10-26T01:32:38.619848+0000","last_became_peered":"2023-10-26T01:32:38.619848+0000","last_unstale":"2023-10-26T01:32:40.835857+0000","last_undegraded":"2023-10-26T01:32:40.835857+0000","last_fullsized":"2023-10-26T01:32:40.835857+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:54:49.271825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-10-26T01:32:38.602790+0000","last_change":"2023-10-26T01:32:37.597739+0000","last_active":"2023-10-26T01:32:38.602790+0000","last_peered":"2023-10-26T01:32:38.602790+0000","last_clean":"2023-10-26T01:32:38.602790+0000","last_became_active":"2023-10-26T01:32:37.597428+0000","last_became_peered":"2023-10-26T01:32:37.597428+0000","last_unstale":"2023-10-26T01:32:38.602790+0000","last_undegraded":"2023-10-26T01:32:38.602790+0000","last_fullsized":"2023-10-26T01:32:38.602790+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_clean_scrub_stamp":"2023-10-26T01:32:36.550883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:47:39.909000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:39.590854+0000","last_change":"2023-10-26T01:32:38.616317+0000","last_active":"2023-10-26T01:32:39.590854+0000","last_peered":"2023-10-26T01:32:39.590854+0000","last_clean":"2023-10-26T01:32:39.590854+0000","last_became_active":"2023-10-26T01:32:38.616016+0000","last_became_peered":"2023-10-26T01:32:38.616016+0000","last_unstale":"2023-10-26T01:32:39.590854+0000","last_undegraded":"2023-10-26T01:32:39.590854+0000","last_fullsized":"2023-10-26T01:32:39.590854+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:00:35.764976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-10-26T01:32:38.604494+0000","last_change":"2023-10-26T01:32:37.609073+0000","last_active":"2023-10-26T01:32:38.604494+0000","last_peered":"2023-10-26T01:32:38.604494+0000","last_clean":"2023-10-26T01:32:38.604494+0000","last_became_active":"2023-10-26T01:32:37.608780+0000","last_became_peered":"2023-10-26T01:32:37.608780+0000","last_unstale":"2023-10-26T01:32:38.604494+0000","last_undegraded":"2023-10-26T01:32:38.604494+0000","last_fullsized":"2023-10-26T01:32:38.604494+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_clean_scrub_stamp":"2023-10-26T01:32:36.550883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:15:21.947485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-10-26T01:32:38.618715+0000","last_change":"2023-10-26T01:32:38.618612+0000","last_active":"2023-10-26T01:32:38.618715+0000","last_peered":"2023-10-26T01:32:38.618715+0000","last_clean":"2023-10-26T01:32:38.618715+0000","last_became_active":"2023-10-26T01:32:38.618344+0000","last_became_peered":"2023-10-26T01:32:38.618344+0000","last_unstale":"2023-10-26T01:32:38.618715+0000","last_undegraded":"2023-10-26T01:32:38.618715+0000","last_fullsized":"2023-10-26T01:32:38.618715+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:10:25.558471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-10-26T01:32:38.590729+0000","last_change":"2023-10-26T01:32:37.604657+0000","last_active":"2023-10-26T01:32:38.590729+0000","last_peered":"2023-10-26T01:32:38.590729+0000","last_clean":"2023-10-26T01:32:38.590729+0000","last_became_active":"2023-10-26T01:32:37.603985+0000","last_became_peered":"2023-10-26T01:32:37.603985+0000","last_unstale":"2023-10-26T01:32:38.590729+0000","last_undegraded":"2023-10-26T01:32:38.590729+0000","last_fullsized":"2023-10-26T01:32:38.590729+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_clean_scrub_stamp":"2023-10-26T01:32:36.550883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:34:01.796167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:40.929800+0000","last_change":"2023-10-26T01:32:38.617596+0000","last_active":"2023-10-26T01:32:40.929800+0000","last_peered":"2023-10-26T01:32:40.929800+0000","last_clean":"2023-10-26T01:32:40.929800+0000","last_became_active":"2023-10-26T01:32:38.617176+0000","last_became_peered":"2023-10-26T01:32:38.617176+0000","last_unstale":"2023-10-26T01:32:40.929800+0000","last_undegraded":"2023-10-26T01:32:40.929800+0000","last_fullsized":"2023-10-26T01:32:40.929800+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:16:50.946128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-10-26T01:32:38.608786+0000","last_change":"2023-10-26T01:32:37.607098+0000","last_active":"2023-10-26T01:32:38.608786+0000","last_peered":"2023-10-26T01:32:38.608786+0000","last_clean":"2023-10-26T01:32:38.608786+0000","last_became_active":"2023-10-26T01:32:37.606861+0000","last_became_peered":"2023-10-26T01:32:37.606861+0000","last_unstale":"2023-10-26T01:32:38.608786+0000","last_undegraded":"2023-10-26T01:32:38.608786+0000","last_fullsized":"2023-10-26T01:32:38.608786+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_clean_scrub_stamp":"2023-10-26T01:32:36.550883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:08:14.303165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:40.930171+0000","last_change":"2023-10-26T01:32:38.610804+0000","last_active":"2023-10-26T01:32:40.930171+0000","last_peered":"2023-10-26T01:32:40.930171+0000","last_clean":"2023-10-26T01:32:40.930171+0000","last_became_active":"2023-10-26T01:32:38.609289+0000","last_became_peered":"2023-10-26T01:32:38.609289+0000","last_unstale":"2023-10-26T01:32:40.930171+0000","last_undegraded":"2023-10-26T01:32:40.930171+0000","last_fullsized":"2023-10-26T01:32:40.930171+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T01:59:46.176056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:39.593007+0000","last_change":"2023-10-26T01:32:37.598304+0000","last_active":"2023-10-26T01:32:39.593007+0000","last_peered":"2023-10-26T01:32:39.593007+0000","last_clean":"2023-10-26T01:32:39.593007+0000","last_became_active":"2023-10-26T01:32:37.592971+0000","last_became_peered":"2023-10-26T01:32:37.592971+0000","last_unstale":"2023-10-26T01:32:39.593007+0000","last_undegraded":"2023-10-26T01:32:39.593007+0000","last_fullsized":"2023-10-26T01:32:39.593007+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_clean_scrub_stamp":"2023-10-26T01:32:36.550883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:13:10.524561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-10-26T01:32:38.620854+0000","last_change":"2023-10-26T01:32:38.620731+0000","last_active":"2023-10-26T01:32:38.620854+0000","last_peered":"2023-10-26T01:32:38.620854+0000","last_clean":"2023-10-26T01:32:38.620854+0000","last_became_active":"2023-10-26T01:32:38.620296+0000","last_became_peered":"2023-10-26T01:32:38.620296+0000","last_unstale":"2023-10-26T01:32:38.620854+0000","last_undegraded":"2023-10-26T01:32:38.620854+0000","last_fullsized":"2023-10-26T01:32:38.620854+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:13:11.537665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-10-26T01:32:38.590016+0000","last_change":"2023-10-26T01:32:37.608145+0000","last_active":"2023-10-26T01:32:38.590016+0000","last_peered":"2023-10-26T01:32:38.590016+0000","last_clean":"2023-10-26T01:32:38.590016+0000","last_became_active":"2023-10-26T01:32:37.607909+0000","last_became_peered":"2023-10-26T01:32:37.607909+0000","last_unstale":"2023-10-26T01:32:38.590016+0000","last_undegraded":"2023-10-26T01:32:38.590016+0000","last_fullsized":"2023-10-26T01:32:38.590016+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_clean_scrub_stamp":"2023-10-26T01:32:36.550883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:13:10.524561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:39.592767+0000","last_change":"2023-10-26T01:32:38.606249+0000","last_active":"2023-10-26T01:32:39.592767+0000","last_peered":"2023-10-26T01:32:39.592767+0000","last_clean":"2023-10-26T01:32:39.592767+0000","last_became_active":"2023-10-26T01:32:38.605587+0000","last_became_peered":"2023-10-26T01:32:38.605587+0000","last_unstale":"2023-10-26T01:32:39.592767+0000","last_undegraded":"2023-10-26T01:32:39.592767+0000","last_fullsized":"2023-10-26T01:32:39.592767+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:15:22.960590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:39.591021+0000","last_change":"2023-10-26T01:32:37.598122+0000","last_active":"2023-10-26T01:32:39.591021+0000","last_peered":"2023-10-26T01:32:39.591021+0000","last_clean":"2023-10-26T01:32:39.591021+0000","last_became_active":"2023-10-26T01:32:37.597737+0000","last_became_peered":"2023-10-26T01:32:37.597737+0000","last_unstale":"2023-10-26T01:32:39.591021+0000","last_undegraded":"2023-10-26T01:32:39.591021+0000","last_fullsized":"2023-10-26T01:32:39.591021+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_clean_scrub_stamp":"2023-10-26T01:32:36.550883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:42:35.797520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:39.590575+0000","last_change":"2023-10-26T01:32:38.619436+0000","last_active":"2023-10-26T01:32:39.590575+0000","last_peered":"2023-10-26T01:32:39.590575+0000","last_clean":"2023-10-26T01:32:39.590575+0000","last_became_active":"2023-10-26T01:32:38.619179+0000","last_became_peered":"2023-10-26T01:32:38.619179+0000","last_unstale":"2023-10-26T01:32:39.590575+0000","last_undegraded":"2023-10-26T01:32:39.590575+0000","last_fullsized":"2023-10-26T01:32:39.590575+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:28:18.718833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"48'2","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-10-26T01:32:38.935995+0000","last_change":"2023-10-26T01:32:37.604369+0000","last_active":"2023-10-26T01:32:38.935995+0000","last_peered":"2023-10-26T01:32:38.935995+0000","last_clean":"2023-10-26T01:32:38.935995+0000","last_became_active":"2023-10-26T01:32:37.604012+0000","last_became_peered":"2023-10-26T01:32:37.604012+0000","last_unstale":"2023-10-26T01:32:38.935995+0000","last_undegraded":"2023-10-26T01:32:38.935995+0000","last_fullsized":"2023-10-26T01:32:38.935995+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_clean_scrub_stamp":"2023-10-26T01:32:36.550883+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:10:24.545366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":190,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T01:32:37.585675+0000","last_change":"2023-10-26T01:30:34.883484+0000","last_active":"2023-10-26T01:32:37.585675+0000","last_peered":"2023-10-26T01:32:37.585675+0000","last_clean":"2023-10-26T01:32:37.585675+0000","last_became_active":"2023-10-26T01:30:34.883071+0000","last_became_peered":"2023-10-26T01:30:34.883071+0000","last_unstale":"2023-10-26T01:32:37.585675+0000","last_undegraded":"2023-10-26T01:32:37.585675+0000","last_fullsized":"2023-10-26T01:32:37.585675+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:32.890096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:32.890096+0000","last_clean_scrub_stamp":"2023-10-26T01:30:32.890096+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:22:57.937891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":14,"seq":60129542173,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":231,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195080,"statfs":{"total":200000000,"available":199762555,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:32:38 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.78600000000000003,"15min":0.78300000000000003},"min":{"1min":0.53800000000000003,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.454,"5min":2.2549999999999999,"15min":2.2549999999999999},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.879,"15min":0.89900000000000002},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.1439999999999999,"5min":1.8200000000000001,"15min":1.8200000000000001},"last":0.745}]},{"osd":1,"last update":"Thu Oct 26 01:32:38 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.93899999999999995,"15min":0.97799999999999998},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.329,"5min":4.4340000000000002,"15min":4.4340000000000002},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.996,"15min":1.0109999999999999},"min":{"1min":0.501,"5min":0.374,"15min":0.374},"max":{"1min":1.288,"5min":2.3330000000000002,"15min":2.3330000000000002},"last":1.1659999999999999}]},{"osd":2,"last update":"Thu Oct 26 01:32:38 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":1.077,"15min":1.103},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.5720000000000001,"5min":4.2800000000000002,"15min":4.2800000000000002},"last":1.2909999999999999},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.069,"15min":1.0740000000000001},"min":{"1min":0.56899999999999995,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.542,"5min":2.302,"15min":2.302},"last":1.542}]},{"osd":4,"last update":"Thu Oct 26 01:32:38 2023","interfaces":[{"interface":"back","average":{"1min":1.123,"5min":1.1439999999999999,"15min":1.1479999999999999},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.462,"5min":2.1070000000000002,"15min":2.1070000000000002},"last":1.325},{"interface":"front","average":{"1min":1.23,"5min":1.393,"15min":1.4199999999999999},"min":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":1.7709999999999999,"5min":4.8250000000000002,"15min":4.8250000000000002},"last":1.597}]},{"osd":5,"last update":"Thu Oct 26 01:32:38 2023","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.337,"15min":1.367},"min":{"1min":0.76200000000000001,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.853,"5min":4.9080000000000004,"15min":4.9080000000000004},"last":1.4970000000000001},{"interface":"front","average":{"1min":1.2150000000000001,"5min":1.2669999999999999,"15min":1.276},"min":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.577,"5min":2.8980000000000001,"15min":2.8980000000000001},"last":1.3420000000000001}]},{"osd":6,"last update":"Thu Oct 26 01:32:38 2023","interfaces":[{"interface":"back","average":{"1min":1.3009999999999999,"5min":1.3169999999999999,"15min":1.319},"min":{"1min":0.88,"5min":0.85099999999999998,"15min":0.85099999999999998},"max":{"1min":1.7729999999999999,"5min":2.8530000000000002,"15min":2.8530000000000002},"last":1.444},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.2569999999999999,"15min":1.2709999999999999},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.623,"5min":2.181,"15min":2.181},"last":1.375}]},{"osd":7,"last update":"Thu Oct 26 01:32:38 2023","interfaces":[{"interface":"back","average":{"1min":1.228,"5min":1.274,"15min":1.282},"min":{"1min":0.871,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.8149999999999999,"5min":2.0680000000000001,"15min":2.0680000000000001},"last":1.238},{"interface":"front","average":{"1min":1.222,"5min":1.4039999999999999,"15min":1.4339999999999999},"min":{"1min":0.90900000000000003,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.679,"5min":4.8730000000000002,"15min":4.8730000000000002},"last":1.3939999999999999}]}]},{"osd":7,"up_from":13,"seq":55834574876,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":801,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194510,"statfs":{"total":200000000,"available":199179035,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:32:35 2023","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.0760000000000001,"15min":1.0680000000000001},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.5700000000000001,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":1.333},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.079,"15min":1.0589999999999999},"min":{"1min":0.76600000000000001,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.6299999999999999,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":1.373}]},{"osd":1,"last update":"Thu Oct 26 01:32:35 2023","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.0669999999999999,"15min":1.0449999999999999},"min":{"1min":0.79300000000000004,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.8640000000000001,"5min":1.8640000000000001,"15min":1.8640000000000001},"last":1.5860000000000001},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.1990000000000001,"15min":1.1830000000000001},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":2.1720000000000002,"5min":2.1720000000000002,"15min":2.1720000000000002},"last":1.28}]},{"osd":2,"last update":"Thu Oct 26 01:32:35 2023","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.091,"15min":1.0860000000000001},"min":{"1min":0.61199999999999999,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.6919999999999999,"5min":1.6919999999999999,"15min":1.6919999999999999},"last":1.4159999999999999},{"interface":"front","average":{"1min":1.254,"5min":1.2150000000000001,"15min":1.2090000000000001},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.752,"5min":1.766,"15min":1.766},"last":1.5269999999999999}]},{"osd":3,"last update":"Thu Oct 26 01:31:36 2023","interfaces":[{"interface":"back","average":{"1min":1.177,"5min":1.177,"15min":1.177},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":1.236},{"interface":"front","average":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"min":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"max":{"1min":1.6180000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":1.4590000000000001}]},{"osd":4,"last update":"Thu Oct 26 01:32:35 2023","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.3400000000000001,"15min":1.343},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":2.008,"5min":2.008,"15min":2.008},"last":1.077},{"interface":"front","average":{"1min":1.4379999999999999,"5min":1.3899999999999999,"15min":1.3819999999999999},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":2.339,"5min":2.339,"15min":2.339},"last":1.177}]},{"osd":5,"last update":"Thu Oct 26 01:32:35 2023","interfaces":[{"interface":"back","average":{"1min":1.411,"5min":1.3959999999999999,"15min":1.3939999999999999},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":2.0030000000000001,"5min":2.0030000000000001,"15min":2.0030000000000001},"last":0.97799999999999998},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.45,"15min":1.456},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.911,"5min":1.911,"15min":1.911},"last":1.494}]},{"osd":6,"last update":"Thu Oct 26 01:32:35 2023","interfaces":[{"interface":"back","average":{"1min":1.5309999999999999,"5min":1.917,"15min":1.9810000000000001},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":2.0539999999999998,"5min":11.577999999999999,"15min":11.577999999999999},"last":1.1299999999999999},{"interface":"front","average":{"1min":1.53,"5min":1.907,"15min":1.97},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":2.0870000000000002,"5min":10.704000000000001,"15min":10.704000000000001},"last":1.5509999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574876,"num_pgs":29,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":225,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195087,"statfs":{"total":200000000,"available":199769435,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:32:33 2023","interfaces":[{"interface":"back","average":{"1min":1.0249999999999999,"5min":1.1579999999999999,"15min":1.1799999999999999},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.4570000000000001,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":1.375},{"interface":"front","average":{"1min":0.92500000000000004,"5min":1.077,"15min":1.103},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.3919999999999999,"5min":1.843,"15min":1.843},"last":1.5620000000000001}]},{"osd":1,"last update":"Thu Oct 26 01:32:33 2023","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":1.149,"15min":1.167},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.7150000000000001,"5min":1.7150000000000001,"15min":1.7150000000000001},"last":1.6910000000000001},{"interface":"front","average":{"1min":1.1319999999999999,"5min":1.2070000000000001,"15min":1.2190000000000001},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.5209999999999999,"5min":2.085,"15min":2.085},"last":1.321}]},{"osd":2,"last update":"Thu Oct 26 01:32:33 2023","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.0469999999999999,"15min":1.05},"min":{"1min":0.78800000000000003,"5min":0.747,"15min":0.747},"max":{"1min":1.329,"5min":1.6659999999999999,"15min":1.6659999999999999},"last":1.6419999999999999},{"interface":"front","average":{"1min":1.232,"5min":1.244,"15min":1.246},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.9890000000000001,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":1.476}]},{"osd":3,"last update":"Thu Oct 26 01:32:37 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.167,"15min":1.1599999999999999},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.5800000000000001,"5min":1.5800000000000001,"15min":1.5800000000000001},"last":1.244},{"interface":"front","average":{"1min":1.3080000000000001,"5min":1.3180000000000001,"15min":1.3200000000000001},"min":{"1min":0.86799999999999999,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.881,"5min":1.881,"15min":1.881},"last":1.6020000000000001}]},{"osd":4,"last update":"Thu Oct 26 01:32:33 2023","interfaces":[{"interface":"back","average":{"1min":1.173,"5min":1.1970000000000001,"15min":1.2010000000000001},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.839,"5min":2.089,"15min":2.089},"last":1.006},{"interface":"front","average":{"1min":1.288,"5min":1.2609999999999999,"15min":1.2569999999999999},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.653,"5min":2.4129999999999998,"15min":2.4129999999999998},"last":1.4019999999999999}]},{"osd":5,"last update":"Thu Oct 26 01:32:33 2023","interfaces":[{"interface":"back","average":{"1min":1.458,"5min":1.4039999999999999,"15min":1.395},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.925,"5min":2.1640000000000001,"15min":2.1640000000000001},"last":1.5209999999999999},{"interface":"front","average":{"1min":1.47,"5min":1.4059999999999999,"15min":1.395},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":2.0710000000000002,"5min":2.0710000000000002,"15min":2.0710000000000002},"last":1.274}]},{"osd":7,"last update":"Thu Oct 26 01:32:33 2023","interfaces":[{"interface":"back","average":{"1min":1.357,"5min":1.355,"15min":1.355},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":2,"5min":2.012,"15min":2.012},"last":1.446},{"interface":"front","average":{"1min":1.5069999999999999,"5min":1.4450000000000001,"15min":1.4350000000000001},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":2.036,"5min":2.3239999999999998,"15min":2.3239999999999998},"last":1.0760000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574876,"num_pgs":15,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":225,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195087,"statfs":{"total":200000000,"available":199769423,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:32:38 2023","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"min":{"1min":0.66200000000000003,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.573,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":1.2110000000000001},{"interface":"front","average":{"1min":1.159,"5min":1.1359999999999999,"15min":1.1319999999999999},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.5309999999999999,"5min":1.5309999999999999,"15min":1.5309999999999999},"last":1.482}]},{"osd":1,"last update":"Thu Oct 26 01:32:38 2023","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":1.0600000000000001,"15min":1.0629999999999999},"min":{"1min":0.81499999999999995,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.3819999999999999,"5min":1.722,"15min":1.722},"last":1.1659999999999999},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.087,"15min":1.071},"min":{"1min":0.71099999999999997,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.5820000000000001,"5min":1.7669999999999999,"15min":1.7669999999999999},"last":0.97499999999999998}]},{"osd":2,"last update":"Thu Oct 26 01:32:38 2023","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":1.155,"15min":1.145},"min":{"1min":0.74399999999999999,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.6339999999999999,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":0.90700000000000003},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.1539999999999999,"15min":1.1579999999999999},"min":{"1min":0.85699999999999998,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.5269999999999999,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":1.347}]},{"osd":3,"last update":"Thu Oct 26 01:32:38 2023","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.206,"15min":1.212},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.492,"5min":1.835,"15min":1.835},"last":1.292},{"interface":"front","average":{"1min":1.25,"5min":1.276,"15min":1.28},"min":{"1min":0.81999999999999995,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.5980000000000001,"5min":2.9249999999999998,"15min":2.9249999999999998},"last":1.038}]},{"osd":4,"last update":"Thu Oct 26 01:32:38 2023","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":1.179,"15min":1.1699999999999999},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.6619999999999999,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":1.1100000000000001},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.3220000000000001,"15min":1.3109999999999999},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.736,"5min":1.736,"15min":1.736},"last":1.24}]},{"osd":6,"last update":"Thu Oct 26 01:32:38 2023","interfaces":[{"interface":"back","average":{"1min":1.3260000000000001,"5min":1.274,"15min":1.266},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.6140000000000001,"5min":1.72,"15min":1.72},"last":1.4379999999999999},{"interface":"front","average":{"1min":1.415,"5min":1.2949999999999999,"15min":1.2749999999999999},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.736,"5min":1.8560000000000001,"15min":1.8560000000000001},"last":1.573}]},{"osd":7,"last update":"Thu Oct 26 01:32:38 2023","interfaces":[{"interface":"back","average":{"1min":1.47,"5min":1.377,"15min":1.361},"min":{"1min":0.64000000000000001,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.8080000000000001,"5min":1.8080000000000001,"15min":1.8080000000000001},"last":1.389},{"interface":"front","average":{"1min":1.411,"5min":1.371,"15min":1.365},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.669,"5min":1.8140000000000001,"15min":1.8140000000000001},"last":1.5289999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574876,"num_pgs":15,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":232,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195079,"statfs":{"total":200000000,"available":199761581,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:32:32 2023","interfaces":[{"interface":"back","average":{"1min":0.98899999999999999,"5min":0.95399999999999996,"15min":0.94799999999999995},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":2.0289999999999999,"5min":2.0289999999999999,"15min":2.0289999999999999},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.99099999999999999,"15min":0.998},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.304,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":1.4690000000000001}]},{"osd":1,"last update":"Thu Oct 26 01:32:32 2023","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.089,"15min":1.0920000000000001},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":2.4249999999999998,"5min":2.4249999999999998,"15min":2.4249999999999998},"last":1.337},{"interface":"front","average":{"1min":1.1020000000000001,"5min":1.089,"15min":1.087},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":2.0760000000000001,"5min":2.0760000000000001,"15min":2.0760000000000001},"last":0.89900000000000002}]},{"osd":2,"last update":"Thu Oct 26 01:32:32 2023","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.1739999999999999,"15min":1.1699999999999999},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.714,"5min":1.714,"15min":1.714},"last":1.514},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.101,"15min":1.093},"min":{"1min":0.74299999999999999,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.673,"5min":1.673,"15min":1.673},"last":0.96899999999999997}]},{"osd":3,"last update":"Thu Oct 26 01:32:32 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.181,"15min":1.177},"min":{"1min":0.76400000000000001,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.891,"5min":1.891,"15min":1.891},"last":1.127},{"interface":"front","average":{"1min":1.155,"5min":1.135,"15min":1.1319999999999999},"min":{"1min":0.86099999999999999,"5min":0.748,"15min":0.748},"max":{"1min":1.5229999999999999,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":1.1679999999999999}]},{"osd":5,"last update":"Thu Oct 26 01:32:32 2023","interfaces":[{"interface":"back","average":{"1min":1.278,"5min":1.298,"15min":1.302},"min":{"1min":0.70499999999999996,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":2.0049999999999999,"5min":2.0049999999999999,"15min":2.0049999999999999},"last":1.038},{"interface":"front","average":{"1min":1.25,"5min":1.3100000000000001,"15min":1.3200000000000001},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.956,"5min":1.956,"15min":1.956},"last":1.413}]},{"osd":6,"last update":"Thu Oct 26 01:32:32 2023","interfaces":[{"interface":"back","average":{"1min":1.3220000000000001,"5min":1.2969999999999999,"15min":1.2929999999999999},"min":{"1min":1.04,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.6870000000000001,"5min":2.218,"15min":2.218},"last":1.095},{"interface":"front","average":{"1min":1.506,"5min":1.468,"15min":1.462},"min":{"1min":0.96899999999999997,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":2.2160000000000002,"5min":3.3500000000000001,"15min":3.3500000000000001},"last":1.208}]},{"osd":7,"last update":"Thu Oct 26 01:32:32 2023","interfaces":[{"interface":"back","average":{"1min":1.4590000000000001,"5min":1.474,"15min":1.476},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":2.5299999999999998,"5min":2.5299999999999998,"15min":2.5299999999999998},"last":1.5640000000000001},{"interface":"front","average":{"1min":1.464,"5min":1.429,"15min":1.423},"min":{"1min":1.1140000000000001,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":1.9339999999999999,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":1.2649999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574876,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":232,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195079,"statfs":{"total":200000000,"available":199761559,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:32:32 2023","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.87,"15min":0.85899999999999999},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":6.3200000000000003,"5min":6.3200000000000003,"15min":6.3200000000000003},"last":1.042},{"interface":"front","average":{"1min":0.94399999999999995,"5min":0.90300000000000002,"15min":0.89600000000000002},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":4.2619999999999996,"5min":4.2619999999999996,"15min":4.2619999999999996},"last":1.232}]},{"osd":1,"last update":"Thu Oct 26 01:32:32 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.89700000000000002,"15min":0.89300000000000002},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":5.1260000000000003,"5min":5.1260000000000003,"15min":5.1260000000000003},"last":0.82099999999999995},{"interface":"front","average":{"1min":1.0860000000000001,"5min":1.006,"15min":0.99199999999999999},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":6.1520000000000001,"5min":6.1520000000000001,"15min":6.1520000000000001},"last":1.2909999999999999}]},{"osd":3,"last update":"Thu Oct 26 01:32:36 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.90500000000000003,"15min":0.89700000000000002},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.5049999999999999,"5min":1.623,"15min":1.623},"last":1.4359999999999999},{"interface":"front","average":{"1min":1.2310000000000001,"5min":1.0009999999999999,"15min":0.96299999999999997},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":8.9019999999999992,"5min":8.9019999999999992,"15min":8.9019999999999992},"last":0.97199999999999998}]},{"osd":4,"last update":"Thu Oct 26 01:32:32 2023","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.244,"15min":1.228},"min":{"1min":0.751,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":6.2670000000000003,"5min":6.2670000000000003,"15min":6.2670000000000003},"last":1.502},{"interface":"front","average":{"1min":1.4570000000000001,"5min":1.2609999999999999,"15min":1.228},"min":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":7.343,"5min":7.343,"15min":7.343},"last":1.0700000000000001}]},{"osd":5,"last update":"Thu Oct 26 01:32:32 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.256,"15min":1.2529999999999999},"min":{"1min":0.80800000000000005,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":3.7170000000000001,"5min":3.7170000000000001,"15min":3.7170000000000001},"last":1.171},{"interface":"front","average":{"1min":1.365,"5min":1.2909999999999999,"15min":1.2789999999999999},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":6.3899999999999997,"5min":6.3899999999999997,"15min":6.3899999999999997},"last":1.1990000000000001}]},{"osd":6,"last update":"Thu Oct 26 01:32:32 2023","interfaces":[{"interface":"back","average":{"1min":1.4319999999999999,"5min":1.3180000000000001,"15min":1.2989999999999999},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":6.0490000000000004,"5min":6.0490000000000004,"15min":6.0490000000000004},"last":1.8160000000000001},{"interface":"front","average":{"1min":1.4019999999999999,"5min":1.3540000000000001,"15min":1.3460000000000001},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":5.0650000000000004,"5min":5.0650000000000004,"15min":5.0650000000000004},"last":1.115}]},{"osd":7,"last update":"Thu Oct 26 01:32:32 2023","interfaces":[{"interface":"back","average":{"1min":1.4219999999999999,"5min":1.3939999999999999,"15min":1.3899999999999999},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":6.4480000000000004,"5min":6.4480000000000004,"15min":6.4480000000000004},"last":1.143},{"interface":"front","average":{"1min":1.377,"5min":1.417,"15min":1.4239999999999999},"min":{"1min":1.0009999999999999,"5min":0.93400000000000005,"15min":0.93400000000000005},"max":{"1min":2.8439999999999999,"5min":2.8439999999999999,"15min":2.8439999999999999},"last":1.899}]}]},{"osd":1,"up_from":13,"seq":55834574876,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":225,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195087,"statfs":{"total":200000000,"available":199769423,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:32:33 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.70199999999999996,"15min":0.69399999999999995},"min":{"1min":0.36699999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":1.224},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.75600000000000001,"15min":0.76100000000000001},"min":{"1min":0.39800000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.4830000000000001,"5min":1.4830000000000001,"15min":1.4830000000000001},"last":0.82099999999999995}]},{"osd":2,"last update":"Thu Oct 26 01:32:33 2023","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":1.0049999999999999,"15min":1.0129999999999999},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.5389999999999999,"5min":2.4830000000000001,"15min":2.4830000000000001},"last":1.4450000000000001},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.84499999999999997,"15min":0.84299999999999997},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":1.167}]},{"osd":3,"last update":"Thu Oct 26 01:32:37 2023","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.94099999999999995,"15min":0.93300000000000005},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.75,"5min":1.75,"15min":1.75},"last":0.876},{"interface":"front","average":{"1min":0.98199999999999998,"5min":0.97199999999999998,"15min":0.96999999999999997},"min":{"1min":0.50900000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.802,"5min":1.802,"15min":1.802},"last":1.05}]},{"osd":4,"last update":"Thu Oct 26 01:32:33 2023","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.1899999999999999,"15min":1.1839999999999999},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.8129999999999999,"5min":2.0659999999999998,"15min":2.0659999999999998},"last":1.3400000000000001},{"interface":"front","average":{"1min":1.105,"5min":1.107,"15min":1.107},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.611,"5min":1.611,"15min":1.611},"last":1.325}]},{"osd":5,"last update":"Thu Oct 26 01:32:33 2023","interfaces":[{"interface":"back","average":{"1min":1.1459999999999999,"5min":1.2030000000000001,"15min":1.2130000000000001},"min":{"1min":0.879,"5min":0.879,"15min":0.879},"max":{"1min":1.702,"5min":1.702,"15min":1.702},"last":1.1160000000000001},{"interface":"front","average":{"1min":1.2470000000000001,"5min":1.3109999999999999,"15min":1.3220000000000001},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.782,"5min":2.2200000000000002,"15min":2.2200000000000002},"last":1.3919999999999999}]},{"osd":6,"last update":"Thu Oct 26 01:32:33 2023","interfaces":[{"interface":"back","average":{"1min":1.2230000000000001,"5min":1.2509999999999999,"15min":1.256},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.679,"5min":2.3780000000000001,"15min":2.3780000000000001},"last":1.522},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.2350000000000001,"15min":1.2290000000000001},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.905,"5min":1.984,"15min":1.984},"last":1.4830000000000001}]},{"osd":7,"last update":"Thu Oct 26 01:32:33 2023","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.2529999999999999,"15min":1.2649999999999999},"min":{"1min":0.78000000000000003,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.5620000000000001,"5min":2.6930000000000001,"15min":2.6930000000000001},"last":1.262},{"interface":"front","average":{"1min":1.3169999999999999,"5min":1.3049999999999999,"15min":1.3029999999999999},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.869,"5min":1.869,"15min":1.869},"last":1.2989999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574876,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":807,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194504,"statfs":{"total":200000000,"available":199172791,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Oct 26 01:32:37 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.85999999999999999,"15min":0.877},"min":{"1min":0.435,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.5609999999999999,"5min":4.5679999999999996,"15min":4.5679999999999996},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.85299999999999998,"5min":1.0069999999999999,"15min":1.0329999999999999},"min":{"1min":0.49399999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.252,"5min":4.8470000000000004,"15min":4.8470000000000004},"last":0.747}]},{"osd":2,"last update":"Thu Oct 26 01:32:37 2023","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":1.0980000000000001,"15min":1.127},"min":{"1min":0.61199999999999999,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.405,"5min":5.5599999999999996,"15min":5.5599999999999996},"last":0.97199999999999998},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.90300000000000002,"15min":0.91300000000000003},"min":{"1min":0.46800000000000003,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.337,"5min":4.9459999999999997,"15min":4.9459999999999997},"last":0.45200000000000001}]},{"osd":3,"last update":"Thu Oct 26 01:32:37 2023","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":1.0840000000000001,"15min":1.105},"min":{"1min":0.46999999999999997,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.867,"5min":4.9800000000000004,"15min":4.9800000000000004},"last":0.49099999999999999},{"interface":"front","average":{"1min":1.03,"5min":1.1459999999999999,"15min":1.165},"min":{"1min":0.56699999999999995,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.992,"5min":5.016,"15min":5.016},"last":0.66000000000000003}]},{"osd":4,"last update":"Thu Oct 26 01:32:37 2023","interfaces":[{"interface":"back","average":{"1min":1.133,"5min":1.2330000000000001,"15min":1.25},"min":{"1min":0.66800000000000004,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.671,"5min":5.1879999999999997,"15min":5.1879999999999997},"last":0.89900000000000002},{"interface":"front","average":{"1min":1.228,"5min":1.304,"15min":1.3169999999999999},"min":{"1min":0.79100000000000004,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":2.036,"5min":5.5229999999999997,"15min":5.5229999999999997},"last":1.417}]},{"osd":5,"last update":"Thu Oct 26 01:32:37 2023","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.4350000000000001,"15min":1.4450000000000001},"min":{"1min":1.044,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":2.2290000000000001,"5min":5.6360000000000001,"15min":5.6360000000000001},"last":0.93799999999999994},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.351,"15min":1.365},"min":{"1min":0.94799999999999995,"5min":0.626,"15min":0.626},"max":{"1min":1.74,"5min":5.3399999999999999,"15min":5.3399999999999999},"last":1.1990000000000001}]},{"osd":6,"last update":"Thu Oct 26 01:32:37 2023","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":1.3440000000000001,"15min":1.3620000000000001},"min":{"1min":0.80200000000000005,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.6100000000000001,"5min":5.3070000000000004,"15min":5.3070000000000004},"last":1.0009999999999999},{"interface":"front","average":{"1min":1.234,"5min":1.3640000000000001,"15min":1.3859999999999999},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.734,"5min":5.5919999999999996,"15min":5.5919999999999996},"last":1.4810000000000001}]},{"osd":7,"last update":"Thu Oct 26 01:32:37 2023","interfaces":[{"interface":"back","average":{"1min":1.3979999999999999,"5min":1.4870000000000001,"15min":1.502},"min":{"1min":0.90300000000000002,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":2.3330000000000002,"5min":5.6689999999999996,"15min":5.6689999999999996},"last":1.099},{"interface":"front","average":{"1min":1.345,"5min":1.4370000000000001,"15min":1.452},"min":{"1min":0.81299999999999994,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.9399999999999999,"5min":5.3719999999999999,"15min":5.3719999999999999},"last":1.272}]}]}],"pool_statfs":[]}} 2023-10-26T01:32:43.354 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-26T01:32:43.560 INFO:tasks.ceph.mds.a.smithi169.stderr:2023-10-26T01:32:43.558+0000 7fd034001640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-26T01:32:43.612 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:43.610+0000 7f47eb59e640 1 -- 172.21.15.169:0/3663221304 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f47ec06a990 msgr2=0x7f47ec0cb390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:43.612 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:43.610+0000 7f47eb59e640 1 --2- 172.21.15.169:0/3663221304 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f47ec06a990 0x7f47ec0cb390 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f47ec0ca700 tx=0x7f47e402d0c0 comp rx=0 tx=0).stop 2023-10-26T01:32:43.612 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:43.610+0000 7f47eb59e640 1 -- 172.21.15.169:0/3663221304 shutdown_connections 2023-10-26T01:32:43.612 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:43.610+0000 7f47eb59e640 1 --2- 172.21.15.169:0/3663221304 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f47ec06a990 0x7f47ec0cb390 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:43.612 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:43.610+0000 7f47eb59e640 1 --2- 172.21.15.169:0/3663221304 >> v2:172.21.15.170:3300/0 conn(0x7f47ec0c9e00 0x7f47ec0c7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:43.613 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:43.610+0000 7f47eb59e640 1 -- 172.21.15.169:0/3663221304 >> 172.21.15.169:0/3663221304 conn(0x7f47ec0aedd0 msgr2=0x7f47ec0af1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:43.613 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:43.610+0000 7f47eb59e640 1 -- 172.21.15.169:0/3663221304 shutdown_connections 2023-10-26T01:32:43.613 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:43.610+0000 7f47eb59e640 1 -- 172.21.15.169:0/3663221304 wait complete. 2023-10-26T01:32:43.614 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:43.611+0000 7f47eb59e640 1 Processor -- start 2023-10-26T01:32:43.614 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:43.612+0000 7f47eb59e640 1 -- start start 2023-10-26T01:32:43.615 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:43.612+0000 7f47eb59e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f47ec06a990 0x7f47ec149990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:43.616 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:43.613+0000 7f47eb59e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f47ec06cae0 0x7f47ec0b1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:43.616 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:43.613+0000 7f47eb59e640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f47ec0c9e00 0x7f47ec0c6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:43.616 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:43.613+0000 7f47eb59e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47ec098a60 con 0x7f47ec06cae0 2023-10-26T01:32:43.616 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:43.613+0000 7f47eb59e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47ec0b2330 con 0x7f47ec06a990 2023-10-26T01:32:43.616 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:43.613+0000 7f47eb59e640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47ec0b2610 con 0x7f47ec0c9e00 2023-10-26T01:32:43.617 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:43.613+0000 7f47e3fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f47ec06cae0 0x7f47ec0b1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:43.617 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:43.613+0000 7f47e3fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f47ec06cae0 0x7f47ec0b1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:33994/0 (socket says 172.21.15.169:33994) 2023-10-26T01:32:43.617 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:43.613+0000 7f47e3fff640 1 -- 172.21.15.169:0/3512826230 learned_addr learned my addr 172.21.15.169:0/3512826230 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:32:43.617 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:43.613+0000 7f47ead9d640 1 --2- 172.21.15.169:0/3512826230 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f47ec0c9e00 0x7f47ec0c6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:43.618 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:43.613+0000 7f47e3fff640 1 -- 172.21.15.169:0/3512826230 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f47ec0c9e00 msgr2=0x7f47ec0c6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:43.618 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:43.613+0000 7f47ea59c640 1 --2- 172.21.15.169:0/3512826230 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f47ec06a990 0x7f47ec149990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:43.618 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:43.613+0000 7f47e3fff640 1 --2- 172.21.15.169:0/3512826230 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f47ec0c9e00 0x7f47ec0c6270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:43.618 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:43.613+0000 7f47e3fff640 1 -- 172.21.15.169:0/3512826230 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f47ec06a990 msgr2=0x7f47ec149990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:43.618 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:43.613+0000 7f47e3fff640 1 --2- 172.21.15.169:0/3512826230 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f47ec06a990 0x7f47ec149990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:43.619 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:43.613+0000 7f47e3fff640 1 -- 172.21.15.169:0/3512826230 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47e400aa10 con 0x7f47ec06cae0 2023-10-26T01:32:43.619 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:43.613+0000 7f47ea59c640 1 --2- 172.21.15.169:0/3512826230 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f47ec06a990 0x7f47ec149990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:32:43.619 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:43.614+0000 7f47e3fff640 1 --2- 172.21.15.169:0/3512826230 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f47ec06cae0 0x7f47ec0b1950 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f47d400b410 tx=0x7f47d400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:43.619 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:43.614+0000 7f47e2ffd640 1 -- 172.21.15.169:0/3512826230 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47d4011830 con 0x7f47ec06cae0 2023-10-26T01:32:43.619 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:43.614+0000 7f47e2ffd640 1 -- 172.21.15.169:0/3512826230 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f47d4011aa0 con 0x7f47ec06cae0 2023-10-26T01:32:43.620 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:43.614+0000 7f47e2ffd640 1 -- 172.21.15.169:0/3512826230 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47d4010380 con 0x7f47ec06cae0 2023-10-26T01:32:43.620 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:43.614+0000 7f47eb59e640 1 -- 172.21.15.169:0/3512826230 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47ec0b28f0 con 0x7f47ec06cae0 2023-10-26T01:32:43.620 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:43.614+0000 7f47eb59e640 1 -- 172.21.15.169:0/3512826230 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47ec149ed0 con 0x7f47ec06cae0 2023-10-26T01:32:43.621 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:43.615+0000 7f47eb59e640 1 -- 172.21.15.169:0/3512826230 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47ec098ef0 con 0x7f47ec06cae0 2023-10-26T01:32:43.621 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:43.616+0000 7f47e2ffd640 1 -- 172.21.15.169:0/3512826230 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f47d40104e0 con 0x7f47ec06cae0 2023-10-26T01:32:43.621 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:43.617+0000 7f47e2ffd640 1 --2- 172.21.15.169:0/3512826230 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f47cc06ae70 0x7f47cc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:43.621 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:43.617+0000 7f47e2ffd640 1 -- 172.21.15.169:0/3512826230 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f47d4089990 con 0x7f47ec06cae0 2023-10-26T01:32:43.622 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:43.618+0000 7f47ea59c640 1 --2- 172.21.15.169:0/3512826230 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f47cc06ae70 0x7f47cc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:43.622 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:43.619+0000 7f47ea59c640 1 --2- 172.21.15.169:0/3512826230 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f47cc06ae70 0x7f47cc06d330 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f47dc007fd0 tx=0x7f47dc009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:43.623 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:43.622+0000 7f47e2ffd640 1 -- 172.21.15.169:0/3512826230 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f47ec098ef0 con 0x7f47ec06cae0 2023-10-26T01:32:43.807 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:43.805+0000 7f47eb59e640 1 -- 172.21.15.169:0/3512826230 --> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f47ec157d40 con 0x7f47cc06ae70 2023-10-26T01:32:43.829 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:43.827+0000 7f47e2ffd640 1 -- 172.21.15.169:0/3512826230 <== mgr.4112 v2:172.21.15.170:6832/1187156918 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+282549 (secure 0 0 0) 0x7f47dc001370 con 0x7f47cc06ae70 2023-10-26T01:32:43.831 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:32:43.834 INFO:teuthology.orchestra.run.smithi169.stderr:dumped all 2023-10-26T01:32:43.835 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:43.833+0000 7f47eb59e640 1 -- 172.21.15.169:0/3512826230 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f47cc06ae70 msgr2=0x7f47cc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:43.835 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:43.833+0000 7f47eb59e640 1 --2- 172.21.15.169:0/3512826230 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f47cc06ae70 0x7f47cc06d330 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f47dc007fd0 tx=0x7f47dc009040 comp rx=0 tx=0).stop 2023-10-26T01:32:43.836 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:43.833+0000 7f47eb59e640 1 -- 172.21.15.169:0/3512826230 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f47ec06cae0 msgr2=0x7f47ec0b1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:43.836 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:43.833+0000 7f47eb59e640 1 --2- 172.21.15.169:0/3512826230 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f47ec06cae0 0x7f47ec0b1950 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f47d400b410 tx=0x7f47d400b8d0 comp rx=0 tx=0).stop 2023-10-26T01:32:43.836 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:43.834+0000 7f47eb59e640 1 -- 172.21.15.169:0/3512826230 shutdown_connections 2023-10-26T01:32:43.836 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:43.834+0000 7f47eb59e640 1 --2- 172.21.15.169:0/3512826230 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f47cc06ae70 0x7f47cc06d330 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:43.836 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:43.834+0000 7f47eb59e640 1 --2- 172.21.15.169:0/3512826230 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f47ec06cae0 0x7f47ec0b1950 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:43.837 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:43.834+0000 7f47eb59e640 1 --2- 172.21.15.169:0/3512826230 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f47ec0c9e00 0x7f47ec0c6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:43.837 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:43.834+0000 7f47eb59e640 1 --2- 172.21.15.169:0/3512826230 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f47ec06a990 0x7f47ec149990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:43.837 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:43.834+0000 7f47eb59e640 1 -- 172.21.15.169:0/3512826230 >> 172.21.15.169:0/3512826230 conn(0x7f47ec0aedd0 msgr2=0x7f47ec155980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:43.837 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:43.834+0000 7f47eb59e640 1 -- 172.21.15.169:0/3512826230 shutdown_connections 2023-10-26T01:32:43.837 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:43.834+0000 7f47eb59e640 1 -- 172.21.15.169:0/3512826230 wait complete. 2023-10-26T01:32:43.854 INFO:teuthology.orchestra.run.smithi169.stdout:{"pg_ready":true,"pg_map":{"version":108,"stamp":"2023-10-26T01:32:42.170355+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":105,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":2982,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1559517,"statfs":{"total":1600000000,"available":1596945802,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.605679"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-10-26T01:32:38.615756+0000","last_change":"2023-10-26T01:32:38.615565+0000","last_active":"2023-10-26T01:32:38.615756+0000","last_peered":"2023-10-26T01:32:38.615756+0000","last_clean":"2023-10-26T01:32:38.615756+0000","last_became_active":"2023-10-26T01:32:38.614975+0000","last_became_peered":"2023-10-26T01:32:38.614975+0000","last_unstale":"2023-10-26T01:32:38.615756+0000","last_undegraded":"2023-10-26T01:32:38.615756+0000","last_fullsized":"2023-10-26T01:32:38.615756+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:30:05.266141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-10-26T01:32:38.616971+0000","last_change":"2023-10-26T01:32:38.616364+0000","last_active":"2023-10-26T01:32:38.616971+0000","last_peered":"2023-10-26T01:32:38.616971+0000","last_clean":"2023-10-26T01:32:38.616971+0000","last_became_active":"2023-10-26T01:32:38.615848+0000","last_became_peered":"2023-10-26T01:32:38.615848+0000","last_unstale":"2023-10-26T01:32:38.616971+0000","last_undegraded":"2023-10-26T01:32:38.616971+0000","last_fullsized":"2023-10-26T01:32:38.616971+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:43:37.338787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:39.592404+0000","last_change":"2023-10-26T01:32:38.595326+0000","last_active":"2023-10-26T01:32:39.592404+0000","last_peered":"2023-10-26T01:32:39.592404+0000","last_clean":"2023-10-26T01:32:39.592404+0000","last_became_active":"2023-10-26T01:32:38.594657+0000","last_became_peered":"2023-10-26T01:32:38.594657+0000","last_unstale":"2023-10-26T01:32:39.592404+0000","last_undegraded":"2023-10-26T01:32:39.592404+0000","last_fullsized":"2023-10-26T01:32:39.592404+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:16:44.879669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:39.590215+0000","last_change":"2023-10-26T01:32:38.622436+0000","last_active":"2023-10-26T01:32:39.590215+0000","last_peered":"2023-10-26T01:32:39.590215+0000","last_clean":"2023-10-26T01:32:39.590215+0000","last_became_active":"2023-10-26T01:32:38.620852+0000","last_became_peered":"2023-10-26T01:32:38.620852+0000","last_unstale":"2023-10-26T01:32:39.590215+0000","last_undegraded":"2023-10-26T01:32:39.590215+0000","last_fullsized":"2023-10-26T01:32:39.590215+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:30:05.266141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-10-26T01:32:38.618242+0000","last_change":"2023-10-26T01:32:38.617484+0000","last_active":"2023-10-26T01:32:38.618242+0000","last_peered":"2023-10-26T01:32:38.618242+0000","last_clean":"2023-10-26T01:32:38.618242+0000","last_became_active":"2023-10-26T01:32:38.617191+0000","last_became_peered":"2023-10-26T01:32:38.617191+0000","last_unstale":"2023-10-26T01:32:38.618242+0000","last_undegraded":"2023-10-26T01:32:38.618242+0000","last_fullsized":"2023-10-26T01:32:38.618242+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:54:49.271825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:40.929367+0000","last_change":"2023-10-26T01:32:38.619985+0000","last_active":"2023-10-26T01:32:40.929367+0000","last_peered":"2023-10-26T01:32:40.929367+0000","last_clean":"2023-10-26T01:32:40.929367+0000","last_became_active":"2023-10-26T01:32:38.618984+0000","last_became_peered":"2023-10-26T01:32:38.618984+0000","last_unstale":"2023-10-26T01:32:40.929367+0000","last_undegraded":"2023-10-26T01:32:40.929367+0000","last_fullsized":"2023-10-26T01:32:40.929367+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:26:27.795851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-10-26T01:32:38.623704+0000","last_change":"2023-10-26T01:32:38.623635+0000","last_active":"2023-10-26T01:32:38.623704+0000","last_peered":"2023-10-26T01:32:38.623704+0000","last_clean":"2023-10-26T01:32:38.623704+0000","last_became_active":"2023-10-26T01:32:38.623340+0000","last_became_peered":"2023-10-26T01:32:38.623340+0000","last_unstale":"2023-10-26T01:32:38.623704+0000","last_undegraded":"2023-10-26T01:32:38.623704+0000","last_fullsized":"2023-10-26T01:32:38.623704+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:16:44.879669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:39.590338+0000","last_change":"2023-10-26T01:32:38.624621+0000","last_active":"2023-10-26T01:32:39.590338+0000","last_peered":"2023-10-26T01:32:39.590338+0000","last_clean":"2023-10-26T01:32:39.590338+0000","last_became_active":"2023-10-26T01:32:38.624350+0000","last_became_peered":"2023-10-26T01:32:38.624350+0000","last_unstale":"2023-10-26T01:32:39.590338+0000","last_undegraded":"2023-10-26T01:32:39.590338+0000","last_fullsized":"2023-10-26T01:32:39.590338+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:04:47.581955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:39.590483+0000","last_change":"2023-10-26T01:32:38.621939+0000","last_active":"2023-10-26T01:32:39.590483+0000","last_peered":"2023-10-26T01:32:39.590483+0000","last_clean":"2023-10-26T01:32:39.590483+0000","last_became_active":"2023-10-26T01:32:38.619273+0000","last_became_peered":"2023-10-26T01:32:38.619273+0000","last_unstale":"2023-10-26T01:32:39.590483+0000","last_undegraded":"2023-10-26T01:32:39.590483+0000","last_fullsized":"2023-10-26T01:32:39.590483+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:08:15.316270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:39.590282+0000","last_change":"2023-10-26T01:32:38.624622+0000","last_active":"2023-10-26T01:32:39.590282+0000","last_peered":"2023-10-26T01:32:39.590282+0000","last_clean":"2023-10-26T01:32:39.590282+0000","last_became_active":"2023-10-26T01:32:38.623011+0000","last_became_peered":"2023-10-26T01:32:38.623011+0000","last_unstale":"2023-10-26T01:32:39.590282+0000","last_undegraded":"2023-10-26T01:32:39.590282+0000","last_fullsized":"2023-10-26T01:32:39.590282+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:50:36.425321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:39.590424+0000","last_change":"2023-10-26T01:32:38.624321+0000","last_active":"2023-10-26T01:32:39.590424+0000","last_peered":"2023-10-26T01:32:39.590424+0000","last_clean":"2023-10-26T01:32:39.590424+0000","last_became_active":"2023-10-26T01:32:38.623205+0000","last_became_peered":"2023-10-26T01:32:38.623205+0000","last_unstale":"2023-10-26T01:32:39.590424+0000","last_undegraded":"2023-10-26T01:32:39.590424+0000","last_fullsized":"2023-10-26T01:32:39.590424+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:04:08.255284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-10-26T01:32:38.616247+0000","last_change":"2023-10-26T01:32:38.616153+0000","last_active":"2023-10-26T01:32:38.616247+0000","last_peered":"2023-10-26T01:32:38.616247+0000","last_clean":"2023-10-26T01:32:38.616247+0000","last_became_active":"2023-10-26T01:32:38.615221+0000","last_became_peered":"2023-10-26T01:32:38.615221+0000","last_unstale":"2023-10-26T01:32:38.616247+0000","last_undegraded":"2023-10-26T01:32:38.616247+0000","last_fullsized":"2023-10-26T01:32:38.616247+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:16:44.879669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:39.591374+0000","last_change":"2023-10-26T01:32:38.605455+0000","last_active":"2023-10-26T01:32:39.591374+0000","last_peered":"2023-10-26T01:32:39.591374+0000","last_clean":"2023-10-26T01:32:39.591374+0000","last_became_active":"2023-10-26T01:32:38.604417+0000","last_became_peered":"2023-10-26T01:32:38.604417+0000","last_unstale":"2023-10-26T01:32:39.591374+0000","last_undegraded":"2023-10-26T01:32:39.591374+0000","last_fullsized":"2023-10-26T01:32:39.591374+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:03:56.612349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:39.591841+0000","last_change":"2023-10-26T01:32:38.623376+0000","last_active":"2023-10-26T01:32:39.591841+0000","last_peered":"2023-10-26T01:32:39.591841+0000","last_clean":"2023-10-26T01:32:39.591841+0000","last_became_active":"2023-10-26T01:32:38.622913+0000","last_became_peered":"2023-10-26T01:32:38.622913+0000","last_unstale":"2023-10-26T01:32:39.591841+0000","last_undegraded":"2023-10-26T01:32:39.591841+0000","last_fullsized":"2023-10-26T01:32:39.591841+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:19:50.831275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:40.929599+0000","last_change":"2023-10-26T01:32:38.614426+0000","last_active":"2023-10-26T01:32:40.929599+0000","last_peered":"2023-10-26T01:32:40.929599+0000","last_clean":"2023-10-26T01:32:40.929599+0000","last_became_active":"2023-10-26T01:32:38.612534+0000","last_became_peered":"2023-10-26T01:32:38.612534+0000","last_unstale":"2023-10-26T01:32:40.929599+0000","last_undegraded":"2023-10-26T01:32:40.929599+0000","last_fullsized":"2023-10-26T01:32:40.929599+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:59:37.874486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:40.835137+0000","last_change":"2023-10-26T01:32:38.624735+0000","last_active":"2023-10-26T01:32:40.835137+0000","last_peered":"2023-10-26T01:32:40.835137+0000","last_clean":"2023-10-26T01:32:40.835137+0000","last_became_active":"2023-10-26T01:32:38.624173+0000","last_became_peered":"2023-10-26T01:32:38.624173+0000","last_unstale":"2023-10-26T01:32:40.835137+0000","last_undegraded":"2023-10-26T01:32:40.835137+0000","last_fullsized":"2023-10-26T01:32:40.835137+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:37:33.515180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:39.592327+0000","last_change":"2023-10-26T01:32:38.613058+0000","last_active":"2023-10-26T01:32:39.592327+0000","last_peered":"2023-10-26T01:32:39.592327+0000","last_clean":"2023-10-26T01:32:39.592327+0000","last_became_active":"2023-10-26T01:32:38.609992+0000","last_became_peered":"2023-10-26T01:32:38.609992+0000","last_unstale":"2023-10-26T01:32:39.592327+0000","last_undegraded":"2023-10-26T01:32:39.592327+0000","last_fullsized":"2023-10-26T01:32:39.592327+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:54:49.271825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:39.593235+0000","last_change":"2023-10-26T01:32:38.617399+0000","last_active":"2023-10-26T01:32:39.593235+0000","last_peered":"2023-10-26T01:32:39.593235+0000","last_clean":"2023-10-26T01:32:39.593235+0000","last_became_active":"2023-10-26T01:32:38.617088+0000","last_became_peered":"2023-10-26T01:32:38.617088+0000","last_unstale":"2023-10-26T01:32:39.593235+0000","last_undegraded":"2023-10-26T01:32:39.593235+0000","last_fullsized":"2023-10-26T01:32:39.593235+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:50:36.425321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:40.836411+0000","last_change":"2023-10-26T01:32:38.619668+0000","last_active":"2023-10-26T01:32:40.836411+0000","last_peered":"2023-10-26T01:32:40.836411+0000","last_clean":"2023-10-26T01:32:40.836411+0000","last_became_active":"2023-10-26T01:32:38.619399+0000","last_became_peered":"2023-10-26T01:32:38.619399+0000","last_unstale":"2023-10-26T01:32:40.836411+0000","last_undegraded":"2023-10-26T01:32:40.836411+0000","last_fullsized":"2023-10-26T01:32:40.836411+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:10:25.558471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:39.592011+0000","last_change":"2023-10-26T01:32:38.617944+0000","last_active":"2023-10-26T01:32:39.592011+0000","last_peered":"2023-10-26T01:32:39.592011+0000","last_clean":"2023-10-26T01:32:39.592011+0000","last_became_active":"2023-10-26T01:32:38.617706+0000","last_became_peered":"2023-10-26T01:32:38.617706+0000","last_unstale":"2023-10-26T01:32:39.592011+0000","last_undegraded":"2023-10-26T01:32:39.592011+0000","last_fullsized":"2023-10-26T01:32:39.592011+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:01:40.338842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:40.929702+0000","last_change":"2023-10-26T01:32:38.615679+0000","last_active":"2023-10-26T01:32:40.929702+0000","last_peered":"2023-10-26T01:32:40.929702+0000","last_clean":"2023-10-26T01:32:40.929702+0000","last_became_active":"2023-10-26T01:32:38.615259+0000","last_became_peered":"2023-10-26T01:32:38.615259+0000","last_unstale":"2023-10-26T01:32:40.929702+0000","last_undegraded":"2023-10-26T01:32:40.929702+0000","last_fullsized":"2023-10-26T01:32:40.929702+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:19:12.202398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:39.590429+0000","last_change":"2023-10-26T01:32:38.622081+0000","last_active":"2023-10-26T01:32:39.590429+0000","last_peered":"2023-10-26T01:32:39.590429+0000","last_clean":"2023-10-26T01:32:39.590429+0000","last_became_active":"2023-10-26T01:32:38.621705+0000","last_became_peered":"2023-10-26T01:32:38.621705+0000","last_unstale":"2023-10-26T01:32:39.590429+0000","last_undegraded":"2023-10-26T01:32:39.590429+0000","last_fullsized":"2023-10-26T01:32:39.590429+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:21:07.355052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:40.835559+0000","last_change":"2023-10-26T01:32:38.616743+0000","last_active":"2023-10-26T01:32:40.835559+0000","last_peered":"2023-10-26T01:32:40.835559+0000","last_clean":"2023-10-26T01:32:40.835559+0000","last_became_active":"2023-10-26T01:32:38.616467+0000","last_became_peered":"2023-10-26T01:32:38.616467+0000","last_unstale":"2023-10-26T01:32:40.835559+0000","last_undegraded":"2023-10-26T01:32:40.835559+0000","last_fullsized":"2023-10-26T01:32:40.835559+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:13:11.537665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:40.835380+0000","last_change":"2023-10-26T01:32:38.615017+0000","last_active":"2023-10-26T01:32:40.835380+0000","last_peered":"2023-10-26T01:32:40.835380+0000","last_clean":"2023-10-26T01:32:40.835380+0000","last_became_active":"2023-10-26T01:32:38.614351+0000","last_became_peered":"2023-10-26T01:32:38.614351+0000","last_unstale":"2023-10-26T01:32:40.835380+0000","last_undegraded":"2023-10-26T01:32:40.835380+0000","last_fullsized":"2023-10-26T01:32:40.835380+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:53:59.515844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:40.835567+0000","last_change":"2023-10-26T01:32:38.625225+0000","last_active":"2023-10-26T01:32:40.835567+0000","last_peered":"2023-10-26T01:32:40.835567+0000","last_clean":"2023-10-26T01:32:40.835567+0000","last_became_active":"2023-10-26T01:32:38.624953+0000","last_became_peered":"2023-10-26T01:32:38.624953+0000","last_unstale":"2023-10-26T01:32:40.835567+0000","last_undegraded":"2023-10-26T01:32:40.835567+0000","last_fullsized":"2023-10-26T01:32:40.835567+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:43:37.338787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:39.591821+0000","last_change":"2023-10-26T01:32:38.617454+0000","last_active":"2023-10-26T01:32:39.591821+0000","last_peered":"2023-10-26T01:32:39.591821+0000","last_clean":"2023-10-26T01:32:39.591821+0000","last_became_active":"2023-10-26T01:32:38.617104+0000","last_became_peered":"2023-10-26T01:32:38.617104+0000","last_unstale":"2023-10-26T01:32:39.591821+0000","last_undegraded":"2023-10-26T01:32:39.591821+0000","last_fullsized":"2023-10-26T01:32:39.591821+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:52:40.966550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:39.592682+0000","last_change":"2023-10-26T01:32:38.613292+0000","last_active":"2023-10-26T01:32:39.592682+0000","last_peered":"2023-10-26T01:32:39.592682+0000","last_clean":"2023-10-26T01:32:39.592682+0000","last_became_active":"2023-10-26T01:32:38.611461+0000","last_became_peered":"2023-10-26T01:32:38.611461+0000","last_unstale":"2023-10-26T01:32:39.592682+0000","last_undegraded":"2023-10-26T01:32:39.592682+0000","last_fullsized":"2023-10-26T01:32:39.592682+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:57:36.439738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-10-26T01:32:38.620646+0000","last_change":"2023-10-26T01:32:38.620512+0000","last_active":"2023-10-26T01:32:38.620646+0000","last_peered":"2023-10-26T01:32:38.620646+0000","last_clean":"2023-10-26T01:32:38.620646+0000","last_became_active":"2023-10-26T01:32:38.620187+0000","last_became_peered":"2023-10-26T01:32:38.620187+0000","last_unstale":"2023-10-26T01:32:38.620646+0000","last_undegraded":"2023-10-26T01:32:38.620646+0000","last_fullsized":"2023-10-26T01:32:38.620646+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:30:05.266141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:40.835793+0000","last_change":"2023-10-26T01:32:38.624506+0000","last_active":"2023-10-26T01:32:40.835793+0000","last_peered":"2023-10-26T01:32:40.835793+0000","last_clean":"2023-10-26T01:32:40.835793+0000","last_became_active":"2023-10-26T01:32:38.623710+0000","last_became_peered":"2023-10-26T01:32:38.623710+0000","last_unstale":"2023-10-26T01:32:40.835793+0000","last_undegraded":"2023-10-26T01:32:40.835793+0000","last_fullsized":"2023-10-26T01:32:40.835793+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:34:02.809272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:39.592533+0000","last_change":"2023-10-26T01:32:38.612032+0000","last_active":"2023-10-26T01:32:39.592533+0000","last_peered":"2023-10-26T01:32:39.592533+0000","last_clean":"2023-10-26T01:32:39.592533+0000","last_became_active":"2023-10-26T01:32:38.611776+0000","last_became_peered":"2023-10-26T01:32:38.611776+0000","last_unstale":"2023-10-26T01:32:39.592533+0000","last_undegraded":"2023-10-26T01:32:39.592533+0000","last_fullsized":"2023-10-26T01:32:39.592533+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:10:25.558471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:40.929982+0000","last_change":"2023-10-26T01:32:38.619614+0000","last_active":"2023-10-26T01:32:40.929982+0000","last_peered":"2023-10-26T01:32:40.929982+0000","last_clean":"2023-10-26T01:32:40.929982+0000","last_became_active":"2023-10-26T01:32:38.617768+0000","last_became_peered":"2023-10-26T01:32:38.617768+0000","last_unstale":"2023-10-26T01:32:40.929982+0000","last_undegraded":"2023-10-26T01:32:40.929982+0000","last_fullsized":"2023-10-26T01:32:40.929982+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:19:35.058721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:39.592372+0000","last_change":"2023-10-26T01:32:38.594730+0000","last_active":"2023-10-26T01:32:39.592372+0000","last_peered":"2023-10-26T01:32:39.592372+0000","last_clean":"2023-10-26T01:32:39.592372+0000","last_became_active":"2023-10-26T01:32:38.594212+0000","last_became_peered":"2023-10-26T01:32:38.594212+0000","last_unstale":"2023-10-26T01:32:39.592372+0000","last_undegraded":"2023-10-26T01:32:39.592372+0000","last_fullsized":"2023-10-26T01:32:39.592372+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:04:54.610130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-10-26T01:32:38.622387+0000","last_change":"2023-10-26T01:32:38.622321+0000","last_active":"2023-10-26T01:32:38.622387+0000","last_peered":"2023-10-26T01:32:38.622387+0000","last_clean":"2023-10-26T01:32:38.622387+0000","last_became_active":"2023-10-26T01:32:38.622049+0000","last_became_peered":"2023-10-26T01:32:38.622049+0000","last_unstale":"2023-10-26T01:32:38.622387+0000","last_undegraded":"2023-10-26T01:32:38.622387+0000","last_fullsized":"2023-10-26T01:32:38.622387+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:23:00.714244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"48'4","reported_seq":16,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-10-26T01:32:38.932586+0000","last_change":"2023-10-26T01:32:37.607858+0000","last_active":"2023-10-26T01:32:38.932586+0000","last_peered":"2023-10-26T01:32:38.932586+0000","last_clean":"2023-10-26T01:32:38.932586+0000","last_became_active":"2023-10-26T01:32:37.607551+0000","last_became_peered":"2023-10-26T01:32:37.607551+0000","last_unstale":"2023-10-26T01:32:38.932586+0000","last_undegraded":"2023-10-26T01:32:38.932586+0000","last_fullsized":"2023-10-26T01:32:38.932586+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_clean_scrub_stamp":"2023-10-26T01:32:36.550883+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:04:07.242179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:40.835218+0000","last_change":"2023-10-26T01:32:38.612838+0000","last_active":"2023-10-26T01:32:40.835218+0000","last_peered":"2023-10-26T01:32:40.835218+0000","last_clean":"2023-10-26T01:32:40.835218+0000","last_became_active":"2023-10-26T01:32:38.612184+0000","last_became_peered":"2023-10-26T01:32:38.612184+0000","last_unstale":"2023-10-26T01:32:40.835218+0000","last_undegraded":"2023-10-26T01:32:40.835218+0000","last_fullsized":"2023-10-26T01:32:40.835218+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:53:18.288374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"48'1","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:39.591929+0000","last_change":"2023-10-26T01:32:37.609023+0000","last_active":"2023-10-26T01:32:39.591929+0000","last_peered":"2023-10-26T01:32:39.591929+0000","last_clean":"2023-10-26T01:32:39.591929+0000","last_became_active":"2023-10-26T01:32:37.608768+0000","last_became_peered":"2023-10-26T01:32:37.608768+0000","last_unstale":"2023-10-26T01:32:39.591929+0000","last_undegraded":"2023-10-26T01:32:39.591929+0000","last_fullsized":"2023-10-26T01:32:39.591929+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_clean_scrub_stamp":"2023-10-26T01:32:36.550883+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:42:35.797520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-10-26T01:32:38.615352+0000","last_change":"2023-10-26T01:32:38.615248+0000","last_active":"2023-10-26T01:32:38.615352+0000","last_peered":"2023-10-26T01:32:38.615352+0000","last_clean":"2023-10-26T01:32:38.615352+0000","last_became_active":"2023-10-26T01:32:38.614715+0000","last_became_peered":"2023-10-26T01:32:38.614715+0000","last_unstale":"2023-10-26T01:32:38.615352+0000","last_undegraded":"2023-10-26T01:32:38.615352+0000","last_fullsized":"2023-10-26T01:32:38.615352+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:57:36.439738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"48'2","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-10-26T01:32:38.929368+0000","last_change":"2023-10-26T01:32:37.604839+0000","last_active":"2023-10-26T01:32:38.929368+0000","last_peered":"2023-10-26T01:32:38.929368+0000","last_clean":"2023-10-26T01:32:38.929368+0000","last_became_active":"2023-10-26T01:32:37.604415+0000","last_became_peered":"2023-10-26T01:32:37.604415+0000","last_unstale":"2023-10-26T01:32:38.929368+0000","last_undegraded":"2023-10-26T01:32:38.929368+0000","last_fullsized":"2023-10-26T01:32:38.929368+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_clean_scrub_stamp":"2023-10-26T01:32:36.550883+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:43:36.325682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-10-26T01:32:38.624709+0000","last_change":"2023-10-26T01:32:38.624444+0000","last_active":"2023-10-26T01:32:38.624709+0000","last_peered":"2023-10-26T01:32:38.624709+0000","last_clean":"2023-10-26T01:32:38.624709+0000","last_became_active":"2023-10-26T01:32:38.624081+0000","last_became_peered":"2023-10-26T01:32:38.624081+0000","last_unstale":"2023-10-26T01:32:38.624709+0000","last_undegraded":"2023-10-26T01:32:38.624709+0000","last_fullsized":"2023-10-26T01:32:38.624709+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:52:34.748242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"48'2","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-10-26T01:32:38.936575+0000","last_change":"2023-10-26T01:32:37.604413+0000","last_active":"2023-10-26T01:32:38.936575+0000","last_peered":"2023-10-26T01:32:38.936575+0000","last_clean":"2023-10-26T01:32:38.936575+0000","last_became_active":"2023-10-26T01:32:37.604186+0000","last_became_peered":"2023-10-26T01:32:37.604186+0000","last_unstale":"2023-10-26T01:32:38.936575+0000","last_undegraded":"2023-10-26T01:32:38.936575+0000","last_fullsized":"2023-10-26T01:32:38.936575+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_clean_scrub_stamp":"2023-10-26T01:32:36.550883+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:04:46.568850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:39.592059+0000","last_change":"2023-10-26T01:32:38.610012+0000","last_active":"2023-10-26T01:32:39.592059+0000","last_peered":"2023-10-26T01:32:39.592059+0000","last_clean":"2023-10-26T01:32:39.592059+0000","last_became_active":"2023-10-26T01:32:38.609568+0000","last_became_peered":"2023-10-26T01:32:38.609568+0000","last_unstale":"2023-10-26T01:32:39.592059+0000","last_undegraded":"2023-10-26T01:32:39.592059+0000","last_fullsized":"2023-10-26T01:32:39.592059+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T01:45:53.551623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"48'1","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-10-26T01:32:38.933907+0000","last_change":"2023-10-26T01:32:37.595329+0000","last_active":"2023-10-26T01:32:38.933907+0000","last_peered":"2023-10-26T01:32:38.933907+0000","last_clean":"2023-10-26T01:32:38.933907+0000","last_became_active":"2023-10-26T01:32:37.594957+0000","last_became_peered":"2023-10-26T01:32:37.594957+0000","last_unstale":"2023-10-26T01:32:38.933907+0000","last_undegraded":"2023-10-26T01:32:38.933907+0000","last_fullsized":"2023-10-26T01:32:38.933907+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_clean_scrub_stamp":"2023-10-26T01:32:36.550883+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:50:35.412216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:39.590541+0000","last_change":"2023-10-26T01:32:38.617840+0000","last_active":"2023-10-26T01:32:39.590541+0000","last_peered":"2023-10-26T01:32:39.590541+0000","last_clean":"2023-10-26T01:32:39.590541+0000","last_became_active":"2023-10-26T01:32:38.617418+0000","last_became_peered":"2023-10-26T01:32:38.617418+0000","last_unstale":"2023-10-26T01:32:39.590541+0000","last_undegraded":"2023-10-26T01:32:39.590541+0000","last_fullsized":"2023-10-26T01:32:39.590541+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:35:27.303645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"48'1","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:39.593473+0000","last_change":"2023-10-26T01:32:37.606118+0000","last_active":"2023-10-26T01:32:39.593473+0000","last_peered":"2023-10-26T01:32:39.593473+0000","last_clean":"2023-10-26T01:32:39.593473+0000","last_became_active":"2023-10-26T01:32:37.605531+0000","last_became_peered":"2023-10-26T01:32:37.605531+0000","last_unstale":"2023-10-26T01:32:39.593473+0000","last_undegraded":"2023-10-26T01:32:39.593473+0000","last_fullsized":"2023-10-26T01:32:39.593473+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_clean_scrub_stamp":"2023-10-26T01:32:36.550883+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:34:01.796167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:39.590544+0000","last_change":"2023-10-26T01:32:38.624467+0000","last_active":"2023-10-26T01:32:39.590544+0000","last_peered":"2023-10-26T01:32:39.590544+0000","last_clean":"2023-10-26T01:32:39.590544+0000","last_became_active":"2023-10-26T01:32:38.622694+0000","last_became_peered":"2023-10-26T01:32:38.622694+0000","last_unstale":"2023-10-26T01:32:39.590544+0000","last_undegraded":"2023-10-26T01:32:39.590544+0000","last_fullsized":"2023-10-26T01:32:39.590544+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:47:40.922105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-10-26T01:32:38.602790+0000","last_change":"2023-10-26T01:32:37.605956+0000","last_active":"2023-10-26T01:32:38.602790+0000","last_peered":"2023-10-26T01:32:38.602790+0000","last_clean":"2023-10-26T01:32:38.602790+0000","last_became_active":"2023-10-26T01:32:37.605677+0000","last_became_peered":"2023-10-26T01:32:37.605677+0000","last_unstale":"2023-10-26T01:32:38.602790+0000","last_undegraded":"2023-10-26T01:32:38.602790+0000","last_fullsized":"2023-10-26T01:32:38.602790+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_clean_scrub_stamp":"2023-10-26T01:32:36.550883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:34:01.796167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:40.836112+0000","last_change":"2023-10-26T01:32:38.615178+0000","last_active":"2023-10-26T01:32:40.836112+0000","last_peered":"2023-10-26T01:32:40.836112+0000","last_clean":"2023-10-26T01:32:40.836112+0000","last_became_active":"2023-10-26T01:32:38.614723+0000","last_became_peered":"2023-10-26T01:32:38.614723+0000","last_unstale":"2023-10-26T01:32:40.836112+0000","last_undegraded":"2023-10-26T01:32:40.836112+0000","last_fullsized":"2023-10-26T01:32:40.836112+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:16:44.879669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:39.591197+0000","last_change":"2023-10-26T01:32:37.600205+0000","last_active":"2023-10-26T01:32:39.591197+0000","last_peered":"2023-10-26T01:32:39.591197+0000","last_clean":"2023-10-26T01:32:39.591197+0000","last_became_active":"2023-10-26T01:32:37.599024+0000","last_became_peered":"2023-10-26T01:32:37.599024+0000","last_unstale":"2023-10-26T01:32:39.591197+0000","last_undegraded":"2023-10-26T01:32:39.591197+0000","last_fullsized":"2023-10-26T01:32:39.591197+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_clean_scrub_stamp":"2023-10-26T01:32:36.550883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:57:35.426633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:39.592884+0000","last_change":"2023-10-26T01:32:38.618211+0000","last_active":"2023-10-26T01:32:39.592884+0000","last_peered":"2023-10-26T01:32:39.592884+0000","last_clean":"2023-10-26T01:32:39.592884+0000","last_became_active":"2023-10-26T01:32:38.617357+0000","last_became_peered":"2023-10-26T01:32:38.617357+0000","last_unstale":"2023-10-26T01:32:39.592884+0000","last_undegraded":"2023-10-26T01:32:39.592884+0000","last_fullsized":"2023-10-26T01:32:39.592884+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:30:05.266141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:39.592194+0000","last_change":"2023-10-26T01:32:37.602691+0000","last_active":"2023-10-26T01:32:39.592194+0000","last_peered":"2023-10-26T01:32:39.592194+0000","last_clean":"2023-10-26T01:32:39.592194+0000","last_became_active":"2023-10-26T01:32:37.602281+0000","last_became_peered":"2023-10-26T01:32:37.602281+0000","last_unstale":"2023-10-26T01:32:39.592194+0000","last_undegraded":"2023-10-26T01:32:39.592194+0000","last_fullsized":"2023-10-26T01:32:39.592194+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_clean_scrub_stamp":"2023-10-26T01:32:36.550883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:57:04.375905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-10-26T01:32:38.618194+0000","last_change":"2023-10-26T01:32:38.618000+0000","last_active":"2023-10-26T01:32:38.618194+0000","last_peered":"2023-10-26T01:32:38.618194+0000","last_clean":"2023-10-26T01:32:38.618194+0000","last_became_active":"2023-10-26T01:32:38.617567+0000","last_became_peered":"2023-10-26T01:32:38.617567+0000","last_unstale":"2023-10-26T01:32:38.618194+0000","last_undegraded":"2023-10-26T01:32:38.618194+0000","last_fullsized":"2023-10-26T01:32:38.618194+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:50:36.425321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:39.590685+0000","last_change":"2023-10-26T01:32:37.602681+0000","last_active":"2023-10-26T01:32:39.590685+0000","last_peered":"2023-10-26T01:32:39.590685+0000","last_clean":"2023-10-26T01:32:39.590685+0000","last_became_active":"2023-10-26T01:32:37.602418+0000","last_became_peered":"2023-10-26T01:32:37.602418+0000","last_unstale":"2023-10-26T01:32:39.590685+0000","last_undegraded":"2023-10-26T01:32:39.590685+0000","last_fullsized":"2023-10-26T01:32:39.590685+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_clean_scrub_stamp":"2023-10-26T01:32:36.550883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T01:45:52.538519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:40.928384+0000","last_change":"2023-10-26T01:32:38.619501+0000","last_active":"2023-10-26T01:32:40.928384+0000","last_peered":"2023-10-26T01:32:40.928384+0000","last_clean":"2023-10-26T01:32:40.928384+0000","last_became_active":"2023-10-26T01:32:38.619048+0000","last_became_peered":"2023-10-26T01:32:38.619048+0000","last_unstale":"2023-10-26T01:32:40.928384+0000","last_undegraded":"2023-10-26T01:32:40.928384+0000","last_fullsized":"2023-10-26T01:32:40.928384+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T01:38:57.150100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"48'3","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-10-26T01:32:38.935777+0000","last_change":"2023-10-26T01:32:37.608362+0000","last_active":"2023-10-26T01:32:38.935777+0000","last_peered":"2023-10-26T01:32:38.935777+0000","last_clean":"2023-10-26T01:32:38.935777+0000","last_became_active":"2023-10-26T01:32:37.607893+0000","last_became_peered":"2023-10-26T01:32:37.607893+0000","last_unstale":"2023-10-26T01:32:38.935777+0000","last_undegraded":"2023-10-26T01:32:38.935777+0000","last_fullsized":"2023-10-26T01:32:38.935777+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_clean_scrub_stamp":"2023-10-26T01:32:36.550883+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:54:48.258721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-10-26T01:32:38.618213+0000","last_change":"2023-10-26T01:32:38.618065+0000","last_active":"2023-10-26T01:32:38.618213+0000","last_peered":"2023-10-26T01:32:38.618213+0000","last_clean":"2023-10-26T01:32:38.618213+0000","last_became_active":"2023-10-26T01:32:38.617815+0000","last_became_peered":"2023-10-26T01:32:38.617815+0000","last_unstale":"2023-10-26T01:32:38.618213+0000","last_undegraded":"2023-10-26T01:32:38.618213+0000","last_fullsized":"2023-10-26T01:32:38.618213+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T01:45:53.551623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"48'6","reported_seq":18,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:39.591182+0000","last_change":"2023-10-26T01:32:37.607352+0000","last_active":"2023-10-26T01:32:39.591182+0000","last_peered":"2023-10-26T01:32:39.591182+0000","last_clean":"2023-10-26T01:32:39.591182+0000","last_became_active":"2023-10-26T01:32:37.606840+0000","last_became_peered":"2023-10-26T01:32:37.606840+0000","last_unstale":"2023-10-26T01:32:39.591182+0000","last_undegraded":"2023-10-26T01:32:39.591182+0000","last_fullsized":"2023-10-26T01:32:39.591182+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_clean_scrub_stamp":"2023-10-26T01:32:36.550883+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T01:35:23.718795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:40.835500+0000","last_change":"2023-10-26T01:32:38.624390+0000","last_active":"2023-10-26T01:32:40.835500+0000","last_peered":"2023-10-26T01:32:40.835500+0000","last_clean":"2023-10-26T01:32:40.835500+0000","last_became_active":"2023-10-26T01:32:38.623376+0000","last_became_peered":"2023-10-26T01:32:38.623376+0000","last_unstale":"2023-10-26T01:32:40.835500+0000","last_undegraded":"2023-10-26T01:32:40.835500+0000","last_fullsized":"2023-10-26T01:32:40.835500+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:23:00.714244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"48'2","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-10-26T01:32:38.930197+0000","last_change":"2023-10-26T01:32:37.599916+0000","last_active":"2023-10-26T01:32:38.930197+0000","last_peered":"2023-10-26T01:32:38.930197+0000","last_clean":"2023-10-26T01:32:38.930197+0000","last_became_active":"2023-10-26T01:32:37.599489+0000","last_became_peered":"2023-10-26T01:32:37.599489+0000","last_unstale":"2023-10-26T01:32:38.930197+0000","last_undegraded":"2023-10-26T01:32:38.930197+0000","last_fullsized":"2023-10-26T01:32:38.930197+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_clean_scrub_stamp":"2023-10-26T01:32:36.550883+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:19:49.818171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:39.590692+0000","last_change":"2023-10-26T01:32:38.622440+0000","last_active":"2023-10-26T01:32:39.590692+0000","last_peered":"2023-10-26T01:32:39.590692+0000","last_clean":"2023-10-26T01:32:39.590692+0000","last_became_active":"2023-10-26T01:32:38.621293+0000","last_became_peered":"2023-10-26T01:32:38.621293+0000","last_unstale":"2023-10-26T01:32:39.590692+0000","last_undegraded":"2023-10-26T01:32:39.590692+0000","last_fullsized":"2023-10-26T01:32:39.590692+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:04:30.078480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-10-26T01:32:38.608453+0000","last_change":"2023-10-26T01:32:37.599205+0000","last_active":"2023-10-26T01:32:38.608453+0000","last_peered":"2023-10-26T01:32:38.608453+0000","last_clean":"2023-10-26T01:32:38.608453+0000","last_became_active":"2023-10-26T01:32:37.598725+0000","last_became_peered":"2023-10-26T01:32:37.598725+0000","last_unstale":"2023-10-26T01:32:38.608453+0000","last_undegraded":"2023-10-26T01:32:38.608453+0000","last_fullsized":"2023-10-26T01:32:38.608453+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_clean_scrub_stamp":"2023-10-26T01:32:36.550883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:04:07.242179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:40.928174+0000","last_change":"2023-10-26T01:32:38.613120+0000","last_active":"2023-10-26T01:32:40.928174+0000","last_peered":"2023-10-26T01:32:40.928174+0000","last_clean":"2023-10-26T01:32:40.928174+0000","last_became_active":"2023-10-26T01:32:38.612583+0000","last_became_peered":"2023-10-26T01:32:38.612583+0000","last_unstale":"2023-10-26T01:32:40.928174+0000","last_undegraded":"2023-10-26T01:32:40.928174+0000","last_fullsized":"2023-10-26T01:32:40.928174+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:21:31.844678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-10-26T01:32:38.601749+0000","last_change":"2023-10-26T01:32:37.598390+0000","last_active":"2023-10-26T01:32:38.601749+0000","last_peered":"2023-10-26T01:32:38.601749+0000","last_clean":"2023-10-26T01:32:38.601749+0000","last_became_active":"2023-10-26T01:32:37.598043+0000","last_became_peered":"2023-10-26T01:32:37.598043+0000","last_unstale":"2023-10-26T01:32:38.601749+0000","last_undegraded":"2023-10-26T01:32:38.601749+0000","last_fullsized":"2023-10-26T01:32:38.601749+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_clean_scrub_stamp":"2023-10-26T01:32:36.550883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:04:46.568850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-10-26T01:32:38.613287+0000","last_change":"2023-10-26T01:32:38.612717+0000","last_active":"2023-10-26T01:32:38.613287+0000","last_peered":"2023-10-26T01:32:38.613287+0000","last_clean":"2023-10-26T01:32:38.613287+0000","last_became_active":"2023-10-26T01:32:38.612482+0000","last_became_peered":"2023-10-26T01:32:38.612482+0000","last_unstale":"2023-10-26T01:32:38.613287+0000","last_undegraded":"2023-10-26T01:32:38.613287+0000","last_fullsized":"2023-10-26T01:32:38.613287+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:57:36.439738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"48'8","reported_seq":20,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:39.590761+0000","last_change":"2023-10-26T01:32:37.604769+0000","last_active":"2023-10-26T01:32:39.590761+0000","last_peered":"2023-10-26T01:32:39.590761+0000","last_clean":"2023-10-26T01:32:39.590761+0000","last_became_active":"2023-10-26T01:32:37.604529+0000","last_became_peered":"2023-10-26T01:32:37.604529+0000","last_unstale":"2023-10-26T01:32:39.590761+0000","last_undegraded":"2023-10-26T01:32:39.590761+0000","last_fullsized":"2023-10-26T01:32:39.590761+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_clean_scrub_stamp":"2023-10-26T01:32:36.550883+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:10:24.545366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:39.592127+0000","last_change":"2023-10-26T01:32:38.609050+0000","last_active":"2023-10-26T01:32:39.592127+0000","last_peered":"2023-10-26T01:32:39.592127+0000","last_clean":"2023-10-26T01:32:39.592127+0000","last_became_active":"2023-10-26T01:32:38.606195+0000","last_became_peered":"2023-10-26T01:32:38.606195+0000","last_unstale":"2023-10-26T01:32:39.592127+0000","last_undegraded":"2023-10-26T01:32:39.592127+0000","last_fullsized":"2023-10-26T01:32:39.592127+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T01:43:56.252521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"48'1","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:39.591281+0000","last_change":"2023-10-26T01:32:37.595678+0000","last_active":"2023-10-26T01:32:39.591281+0000","last_peered":"2023-10-26T01:32:39.591281+0000","last_clean":"2023-10-26T01:32:39.591281+0000","last_became_active":"2023-10-26T01:32:37.595294+0000","last_became_peered":"2023-10-26T01:32:37.595294+0000","last_unstale":"2023-10-26T01:32:39.591281+0000","last_undegraded":"2023-10-26T01:32:39.591281+0000","last_fullsized":"2023-10-26T01:32:39.591281+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_clean_scrub_stamp":"2023-10-26T01:32:36.550883+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:31:14.631804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:39.590868+0000","last_change":"2023-10-26T01:32:38.622200+0000","last_active":"2023-10-26T01:32:39.590868+0000","last_peered":"2023-10-26T01:32:39.590868+0000","last_clean":"2023-10-26T01:32:39.590868+0000","last_became_active":"2023-10-26T01:32:38.620470+0000","last_became_peered":"2023-10-26T01:32:38.620470+0000","last_unstale":"2023-10-26T01:32:39.590868+0000","last_undegraded":"2023-10-26T01:32:39.590868+0000","last_fullsized":"2023-10-26T01:32:39.590868+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:28:57.903482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"48'2","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:39.592782+0000","last_change":"2023-10-26T01:32:37.604694+0000","last_active":"2023-10-26T01:32:39.592782+0000","last_peered":"2023-10-26T01:32:39.592782+0000","last_clean":"2023-10-26T01:32:39.592782+0000","last_became_active":"2023-10-26T01:32:37.602692+0000","last_became_peered":"2023-10-26T01:32:37.602692+0000","last_unstale":"2023-10-26T01:32:39.592782+0000","last_undegraded":"2023-10-26T01:32:39.592782+0000","last_fullsized":"2023-10-26T01:32:39.592782+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_clean_scrub_stamp":"2023-10-26T01:32:36.550883+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:53:58.502739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:40.835520+0000","last_change":"2023-10-26T01:32:38.616088+0000","last_active":"2023-10-26T01:32:40.835520+0000","last_peered":"2023-10-26T01:32:40.835520+0000","last_clean":"2023-10-26T01:32:40.835520+0000","last_became_active":"2023-10-26T01:32:38.615753+0000","last_became_peered":"2023-10-26T01:32:38.615753+0000","last_unstale":"2023-10-26T01:32:40.835520+0000","last_undegraded":"2023-10-26T01:32:40.835520+0000","last_fullsized":"2023-10-26T01:32:40.835520+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:52:34.748242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"48'1","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-10-26T01:32:38.932578+0000","last_change":"2023-10-26T01:32:37.605242+0000","last_active":"2023-10-26T01:32:38.932578+0000","last_peered":"2023-10-26T01:32:38.932578+0000","last_clean":"2023-10-26T01:32:38.932578+0000","last_became_active":"2023-10-26T01:32:37.604904+0000","last_became_peered":"2023-10-26T01:32:37.604904+0000","last_unstale":"2023-10-26T01:32:38.932578+0000","last_undegraded":"2023-10-26T01:32:38.932578+0000","last_fullsized":"2023-10-26T01:32:38.932578+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_clean_scrub_stamp":"2023-10-26T01:32:36.550883+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:15:21.947485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:40.835832+0000","last_change":"2023-10-26T01:32:38.624514+0000","last_active":"2023-10-26T01:32:40.835832+0000","last_peered":"2023-10-26T01:32:40.835832+0000","last_clean":"2023-10-26T01:32:40.835832+0000","last_became_active":"2023-10-26T01:32:38.623869+0000","last_became_peered":"2023-10-26T01:32:38.623869+0000","last_unstale":"2023-10-26T01:32:40.835832+0000","last_undegraded":"2023-10-26T01:32:40.835832+0000","last_fullsized":"2023-10-26T01:32:40.835832+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T01:45:53.551623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-10-26T01:32:38.602086+0000","last_change":"2023-10-26T01:32:37.607629+0000","last_active":"2023-10-26T01:32:38.602086+0000","last_peered":"2023-10-26T01:32:38.602086+0000","last_clean":"2023-10-26T01:32:38.602086+0000","last_became_active":"2023-10-26T01:32:37.607318+0000","last_became_peered":"2023-10-26T01:32:37.607318+0000","last_unstale":"2023-10-26T01:32:38.602086+0000","last_undegraded":"2023-10-26T01:32:38.602086+0000","last_fullsized":"2023-10-26T01:32:38.602086+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_clean_scrub_stamp":"2023-10-26T01:32:36.550883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:08:14.303165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:40.928754+0000","last_change":"2023-10-26T01:32:38.619321+0000","last_active":"2023-10-26T01:32:40.928754+0000","last_peered":"2023-10-26T01:32:40.928754+0000","last_clean":"2023-10-26T01:32:40.928754+0000","last_became_active":"2023-10-26T01:32:38.618026+0000","last_became_peered":"2023-10-26T01:32:38.618026+0000","last_unstale":"2023-10-26T01:32:40.928754+0000","last_undegraded":"2023-10-26T01:32:40.928754+0000","last_fullsized":"2023-10-26T01:32:40.928754+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:57:50.250541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"48'2","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-10-26T01:32:38.935163+0000","last_change":"2023-10-26T01:32:37.602118+0000","last_active":"2023-10-26T01:32:38.935163+0000","last_peered":"2023-10-26T01:32:38.935163+0000","last_clean":"2023-10-26T01:32:38.935163+0000","last_became_active":"2023-10-26T01:32:37.601879+0000","last_became_peered":"2023-10-26T01:32:37.601879+0000","last_unstale":"2023-10-26T01:32:38.935163+0000","last_undegraded":"2023-10-26T01:32:38.935163+0000","last_fullsized":"2023-10-26T01:32:38.935163+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_clean_scrub_stamp":"2023-10-26T01:32:36.550883+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:35:26.290541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-10-26T01:32:38.614279+0000","last_change":"2023-10-26T01:32:38.613797+0000","last_active":"2023-10-26T01:32:38.614279+0000","last_peered":"2023-10-26T01:32:38.614279+0000","last_clean":"2023-10-26T01:32:38.614279+0000","last_became_active":"2023-10-26T01:32:38.613369+0000","last_became_peered":"2023-10-26T01:32:38.613369+0000","last_unstale":"2023-10-26T01:32:38.614279+0000","last_undegraded":"2023-10-26T01:32:38.614279+0000","last_fullsized":"2023-10-26T01:32:38.614279+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:53:59.515844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-10-26T01:32:38.603133+0000","last_change":"2023-10-26T01:32:37.609602+0000","last_active":"2023-10-26T01:32:38.603133+0000","last_peered":"2023-10-26T01:32:38.603133+0000","last_clean":"2023-10-26T01:32:38.603133+0000","last_became_active":"2023-10-26T01:32:37.609351+0000","last_became_peered":"2023-10-26T01:32:37.609351+0000","last_unstale":"2023-10-26T01:32:38.603133+0000","last_undegraded":"2023-10-26T01:32:38.603133+0000","last_fullsized":"2023-10-26T01:32:38.603133+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_clean_scrub_stamp":"2023-10-26T01:32:36.550883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T01:45:52.538519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:39.591698+0000","last_change":"2023-10-26T01:32:38.611126+0000","last_active":"2023-10-26T01:32:39.591698+0000","last_peered":"2023-10-26T01:32:39.591698+0000","last_clean":"2023-10-26T01:32:39.591698+0000","last_became_active":"2023-10-26T01:32:38.610609+0000","last_became_peered":"2023-10-26T01:32:38.610609+0000","last_unstale":"2023-10-26T01:32:39.591698+0000","last_undegraded":"2023-10-26T01:32:39.591698+0000","last_fullsized":"2023-10-26T01:32:39.591698+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:59:11.131020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:39.591281+0000","last_change":"2023-10-26T01:32:37.600542+0000","last_active":"2023-10-26T01:32:39.591281+0000","last_peered":"2023-10-26T01:32:39.591281+0000","last_clean":"2023-10-26T01:32:39.591281+0000","last_became_active":"2023-10-26T01:32:37.599775+0000","last_became_peered":"2023-10-26T01:32:37.599775+0000","last_unstale":"2023-10-26T01:32:39.591281+0000","last_undegraded":"2023-10-26T01:32:39.591281+0000","last_fullsized":"2023-10-26T01:32:39.591281+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_clean_scrub_stamp":"2023-10-26T01:32:36.550883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:15:21.947485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:39.590787+0000","last_change":"2023-10-26T01:32:38.624826+0000","last_active":"2023-10-26T01:32:39.590787+0000","last_peered":"2023-10-26T01:32:39.590787+0000","last_clean":"2023-10-26T01:32:39.590787+0000","last_became_active":"2023-10-26T01:32:38.623345+0000","last_became_peered":"2023-10-26T01:32:38.623345+0000","last_unstale":"2023-10-26T01:32:39.590787+0000","last_undegraded":"2023-10-26T01:32:39.590787+0000","last_fullsized":"2023-10-26T01:32:39.590787+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:58:59.610639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-10-26T01:32:38.608627+0000","last_change":"2023-10-26T01:32:37.600656+0000","last_active":"2023-10-26T01:32:38.608627+0000","last_peered":"2023-10-26T01:32:38.608627+0000","last_clean":"2023-10-26T01:32:38.608627+0000","last_became_active":"2023-10-26T01:32:37.600267+0000","last_became_peered":"2023-10-26T01:32:37.600267+0000","last_unstale":"2023-10-26T01:32:38.608627+0000","last_undegraded":"2023-10-26T01:32:38.608627+0000","last_fullsized":"2023-10-26T01:32:38.608627+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_clean_scrub_stamp":"2023-10-26T01:32:36.550883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:21:06.341948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:40.835857+0000","last_change":"2023-10-26T01:32:38.620385+0000","last_active":"2023-10-26T01:32:40.835857+0000","last_peered":"2023-10-26T01:32:40.835857+0000","last_clean":"2023-10-26T01:32:40.835857+0000","last_became_active":"2023-10-26T01:32:38.619848+0000","last_became_peered":"2023-10-26T01:32:38.619848+0000","last_unstale":"2023-10-26T01:32:40.835857+0000","last_undegraded":"2023-10-26T01:32:40.835857+0000","last_fullsized":"2023-10-26T01:32:40.835857+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:54:49.271825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-10-26T01:32:38.602790+0000","last_change":"2023-10-26T01:32:37.597739+0000","last_active":"2023-10-26T01:32:38.602790+0000","last_peered":"2023-10-26T01:32:38.602790+0000","last_clean":"2023-10-26T01:32:38.602790+0000","last_became_active":"2023-10-26T01:32:37.597428+0000","last_became_peered":"2023-10-26T01:32:37.597428+0000","last_unstale":"2023-10-26T01:32:38.602790+0000","last_undegraded":"2023-10-26T01:32:38.602790+0000","last_fullsized":"2023-10-26T01:32:38.602790+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_clean_scrub_stamp":"2023-10-26T01:32:36.550883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:47:39.909000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:39.590854+0000","last_change":"2023-10-26T01:32:38.616317+0000","last_active":"2023-10-26T01:32:39.590854+0000","last_peered":"2023-10-26T01:32:39.590854+0000","last_clean":"2023-10-26T01:32:39.590854+0000","last_became_active":"2023-10-26T01:32:38.616016+0000","last_became_peered":"2023-10-26T01:32:38.616016+0000","last_unstale":"2023-10-26T01:32:39.590854+0000","last_undegraded":"2023-10-26T01:32:39.590854+0000","last_fullsized":"2023-10-26T01:32:39.590854+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:00:35.764976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-10-26T01:32:38.604494+0000","last_change":"2023-10-26T01:32:37.609073+0000","last_active":"2023-10-26T01:32:38.604494+0000","last_peered":"2023-10-26T01:32:38.604494+0000","last_clean":"2023-10-26T01:32:38.604494+0000","last_became_active":"2023-10-26T01:32:37.608780+0000","last_became_peered":"2023-10-26T01:32:37.608780+0000","last_unstale":"2023-10-26T01:32:38.604494+0000","last_undegraded":"2023-10-26T01:32:38.604494+0000","last_fullsized":"2023-10-26T01:32:38.604494+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_clean_scrub_stamp":"2023-10-26T01:32:36.550883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:15:21.947485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-10-26T01:32:38.618715+0000","last_change":"2023-10-26T01:32:38.618612+0000","last_active":"2023-10-26T01:32:38.618715+0000","last_peered":"2023-10-26T01:32:38.618715+0000","last_clean":"2023-10-26T01:32:38.618715+0000","last_became_active":"2023-10-26T01:32:38.618344+0000","last_became_peered":"2023-10-26T01:32:38.618344+0000","last_unstale":"2023-10-26T01:32:38.618715+0000","last_undegraded":"2023-10-26T01:32:38.618715+0000","last_fullsized":"2023-10-26T01:32:38.618715+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:10:25.558471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-10-26T01:32:38.590729+0000","last_change":"2023-10-26T01:32:37.604657+0000","last_active":"2023-10-26T01:32:38.590729+0000","last_peered":"2023-10-26T01:32:38.590729+0000","last_clean":"2023-10-26T01:32:38.590729+0000","last_became_active":"2023-10-26T01:32:37.603985+0000","last_became_peered":"2023-10-26T01:32:37.603985+0000","last_unstale":"2023-10-26T01:32:38.590729+0000","last_undegraded":"2023-10-26T01:32:38.590729+0000","last_fullsized":"2023-10-26T01:32:38.590729+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_clean_scrub_stamp":"2023-10-26T01:32:36.550883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:34:01.796167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:40.929800+0000","last_change":"2023-10-26T01:32:38.617596+0000","last_active":"2023-10-26T01:32:40.929800+0000","last_peered":"2023-10-26T01:32:40.929800+0000","last_clean":"2023-10-26T01:32:40.929800+0000","last_became_active":"2023-10-26T01:32:38.617176+0000","last_became_peered":"2023-10-26T01:32:38.617176+0000","last_unstale":"2023-10-26T01:32:40.929800+0000","last_undegraded":"2023-10-26T01:32:40.929800+0000","last_fullsized":"2023-10-26T01:32:40.929800+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:16:50.946128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-10-26T01:32:38.608786+0000","last_change":"2023-10-26T01:32:37.607098+0000","last_active":"2023-10-26T01:32:38.608786+0000","last_peered":"2023-10-26T01:32:38.608786+0000","last_clean":"2023-10-26T01:32:38.608786+0000","last_became_active":"2023-10-26T01:32:37.606861+0000","last_became_peered":"2023-10-26T01:32:37.606861+0000","last_unstale":"2023-10-26T01:32:38.608786+0000","last_undegraded":"2023-10-26T01:32:38.608786+0000","last_fullsized":"2023-10-26T01:32:38.608786+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_clean_scrub_stamp":"2023-10-26T01:32:36.550883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:08:14.303165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:40.930171+0000","last_change":"2023-10-26T01:32:38.610804+0000","last_active":"2023-10-26T01:32:40.930171+0000","last_peered":"2023-10-26T01:32:40.930171+0000","last_clean":"2023-10-26T01:32:40.930171+0000","last_became_active":"2023-10-26T01:32:38.609289+0000","last_became_peered":"2023-10-26T01:32:38.609289+0000","last_unstale":"2023-10-26T01:32:40.930171+0000","last_undegraded":"2023-10-26T01:32:40.930171+0000","last_fullsized":"2023-10-26T01:32:40.930171+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T01:59:46.176056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:39.593007+0000","last_change":"2023-10-26T01:32:37.598304+0000","last_active":"2023-10-26T01:32:39.593007+0000","last_peered":"2023-10-26T01:32:39.593007+0000","last_clean":"2023-10-26T01:32:39.593007+0000","last_became_active":"2023-10-26T01:32:37.592971+0000","last_became_peered":"2023-10-26T01:32:37.592971+0000","last_unstale":"2023-10-26T01:32:39.593007+0000","last_undegraded":"2023-10-26T01:32:39.593007+0000","last_fullsized":"2023-10-26T01:32:39.593007+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_clean_scrub_stamp":"2023-10-26T01:32:36.550883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:13:10.524561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-10-26T01:32:38.620854+0000","last_change":"2023-10-26T01:32:38.620731+0000","last_active":"2023-10-26T01:32:38.620854+0000","last_peered":"2023-10-26T01:32:38.620854+0000","last_clean":"2023-10-26T01:32:38.620854+0000","last_became_active":"2023-10-26T01:32:38.620296+0000","last_became_peered":"2023-10-26T01:32:38.620296+0000","last_unstale":"2023-10-26T01:32:38.620854+0000","last_undegraded":"2023-10-26T01:32:38.620854+0000","last_fullsized":"2023-10-26T01:32:38.620854+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:13:11.537665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-10-26T01:32:38.590016+0000","last_change":"2023-10-26T01:32:37.608145+0000","last_active":"2023-10-26T01:32:38.590016+0000","last_peered":"2023-10-26T01:32:38.590016+0000","last_clean":"2023-10-26T01:32:38.590016+0000","last_became_active":"2023-10-26T01:32:37.607909+0000","last_became_peered":"2023-10-26T01:32:37.607909+0000","last_unstale":"2023-10-26T01:32:38.590016+0000","last_undegraded":"2023-10-26T01:32:38.590016+0000","last_fullsized":"2023-10-26T01:32:38.590016+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_clean_scrub_stamp":"2023-10-26T01:32:36.550883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:13:10.524561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:39.592767+0000","last_change":"2023-10-26T01:32:38.606249+0000","last_active":"2023-10-26T01:32:39.592767+0000","last_peered":"2023-10-26T01:32:39.592767+0000","last_clean":"2023-10-26T01:32:39.592767+0000","last_became_active":"2023-10-26T01:32:38.605587+0000","last_became_peered":"2023-10-26T01:32:38.605587+0000","last_unstale":"2023-10-26T01:32:39.592767+0000","last_undegraded":"2023-10-26T01:32:39.592767+0000","last_fullsized":"2023-10-26T01:32:39.592767+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:15:22.960590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:39.591021+0000","last_change":"2023-10-26T01:32:37.598122+0000","last_active":"2023-10-26T01:32:39.591021+0000","last_peered":"2023-10-26T01:32:39.591021+0000","last_clean":"2023-10-26T01:32:39.591021+0000","last_became_active":"2023-10-26T01:32:37.597737+0000","last_became_peered":"2023-10-26T01:32:37.597737+0000","last_unstale":"2023-10-26T01:32:39.591021+0000","last_undegraded":"2023-10-26T01:32:39.591021+0000","last_fullsized":"2023-10-26T01:32:39.591021+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_clean_scrub_stamp":"2023-10-26T01:32:36.550883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:42:35.797520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-10-26T01:32:39.590575+0000","last_change":"2023-10-26T01:32:38.619436+0000","last_active":"2023-10-26T01:32:39.590575+0000","last_peered":"2023-10-26T01:32:39.590575+0000","last_clean":"2023-10-26T01:32:39.590575+0000","last_became_active":"2023-10-26T01:32:38.619179+0000","last_became_peered":"2023-10-26T01:32:38.619179+0000","last_unstale":"2023-10-26T01:32:39.590575+0000","last_undegraded":"2023-10-26T01:32:39.590575+0000","last_fullsized":"2023-10-26T01:32:39.590575+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:37.563987+0000","last_clean_scrub_stamp":"2023-10-26T01:32:37.563987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:28:18.718833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"48'2","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-10-26T01:32:38.935995+0000","last_change":"2023-10-26T01:32:37.604369+0000","last_active":"2023-10-26T01:32:38.935995+0000","last_peered":"2023-10-26T01:32:38.935995+0000","last_clean":"2023-10-26T01:32:38.935995+0000","last_became_active":"2023-10-26T01:32:37.604012+0000","last_became_peered":"2023-10-26T01:32:37.604012+0000","last_unstale":"2023-10-26T01:32:38.935995+0000","last_undegraded":"2023-10-26T01:32:38.935995+0000","last_fullsized":"2023-10-26T01:32:38.935995+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:32:36.550883+0000","last_clean_scrub_stamp":"2023-10-26T01:32:36.550883+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:10:24.545366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":190,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T01:32:37.585675+0000","last_change":"2023-10-26T01:30:34.883484+0000","last_active":"2023-10-26T01:32:37.585675+0000","last_peered":"2023-10-26T01:32:37.585675+0000","last_clean":"2023-10-26T01:32:37.585675+0000","last_became_active":"2023-10-26T01:30:34.883071+0000","last_became_peered":"2023-10-26T01:30:34.883071+0000","last_unstale":"2023-10-26T01:32:37.585675+0000","last_undegraded":"2023-10-26T01:32:37.585675+0000","last_fullsized":"2023-10-26T01:32:37.585675+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:32.890096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:32.890096+0000","last_clean_scrub_stamp":"2023-10-26T01:30:32.890096+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:22:57.937891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":14,"seq":60129542173,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":231,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195080,"statfs":{"total":200000000,"available":199762555,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:32:38 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.78600000000000003,"15min":0.78300000000000003},"min":{"1min":0.53800000000000003,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.454,"5min":2.2549999999999999,"15min":2.2549999999999999},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.879,"15min":0.89900000000000002},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.1439999999999999,"5min":1.8200000000000001,"15min":1.8200000000000001},"last":0.745}]},{"osd":1,"last update":"Thu Oct 26 01:32:38 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.93899999999999995,"15min":0.97799999999999998},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.329,"5min":4.4340000000000002,"15min":4.4340000000000002},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.996,"15min":1.0109999999999999},"min":{"1min":0.501,"5min":0.374,"15min":0.374},"max":{"1min":1.288,"5min":2.3330000000000002,"15min":2.3330000000000002},"last":1.1659999999999999}]},{"osd":2,"last update":"Thu Oct 26 01:32:38 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":1.077,"15min":1.103},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.5720000000000001,"5min":4.2800000000000002,"15min":4.2800000000000002},"last":1.2909999999999999},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.069,"15min":1.0740000000000001},"min":{"1min":0.56899999999999995,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.542,"5min":2.302,"15min":2.302},"last":1.542}]},{"osd":4,"last update":"Thu Oct 26 01:32:38 2023","interfaces":[{"interface":"back","average":{"1min":1.123,"5min":1.1439999999999999,"15min":1.1479999999999999},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.462,"5min":2.1070000000000002,"15min":2.1070000000000002},"last":1.325},{"interface":"front","average":{"1min":1.23,"5min":1.393,"15min":1.4199999999999999},"min":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":1.7709999999999999,"5min":4.8250000000000002,"15min":4.8250000000000002},"last":1.597}]},{"osd":5,"last update":"Thu Oct 26 01:32:38 2023","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.337,"15min":1.367},"min":{"1min":0.76200000000000001,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.853,"5min":4.9080000000000004,"15min":4.9080000000000004},"last":1.4970000000000001},{"interface":"front","average":{"1min":1.2150000000000001,"5min":1.2669999999999999,"15min":1.276},"min":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.577,"5min":2.8980000000000001,"15min":2.8980000000000001},"last":1.3420000000000001}]},{"osd":6,"last update":"Thu Oct 26 01:32:38 2023","interfaces":[{"interface":"back","average":{"1min":1.3009999999999999,"5min":1.3169999999999999,"15min":1.319},"min":{"1min":0.88,"5min":0.85099999999999998,"15min":0.85099999999999998},"max":{"1min":1.7729999999999999,"5min":2.8530000000000002,"15min":2.8530000000000002},"last":1.444},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.2569999999999999,"15min":1.2709999999999999},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.623,"5min":2.181,"15min":2.181},"last":1.375}]},{"osd":7,"last update":"Thu Oct 26 01:32:38 2023","interfaces":[{"interface":"back","average":{"1min":1.228,"5min":1.274,"15min":1.282},"min":{"1min":0.871,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.8149999999999999,"5min":2.0680000000000001,"15min":2.0680000000000001},"last":1.238},{"interface":"front","average":{"1min":1.222,"5min":1.4039999999999999,"15min":1.4339999999999999},"min":{"1min":0.90900000000000003,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.679,"5min":4.8730000000000002,"15min":4.8730000000000002},"last":1.3939999999999999}]}]},{"osd":7,"up_from":13,"seq":55834574876,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":801,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194510,"statfs":{"total":200000000,"available":199179035,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:32:35 2023","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.0760000000000001,"15min":1.0680000000000001},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.5700000000000001,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":1.333},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.079,"15min":1.0589999999999999},"min":{"1min":0.76600000000000001,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.6299999999999999,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":1.373}]},{"osd":1,"last update":"Thu Oct 26 01:32:35 2023","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.0669999999999999,"15min":1.0449999999999999},"min":{"1min":0.79300000000000004,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.8640000000000001,"5min":1.8640000000000001,"15min":1.8640000000000001},"last":1.5860000000000001},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.1990000000000001,"15min":1.1830000000000001},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":2.1720000000000002,"5min":2.1720000000000002,"15min":2.1720000000000002},"last":1.28}]},{"osd":2,"last update":"Thu Oct 26 01:32:35 2023","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.091,"15min":1.0860000000000001},"min":{"1min":0.61199999999999999,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.6919999999999999,"5min":1.6919999999999999,"15min":1.6919999999999999},"last":1.4159999999999999},{"interface":"front","average":{"1min":1.254,"5min":1.2150000000000001,"15min":1.2090000000000001},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.752,"5min":1.766,"15min":1.766},"last":1.5269999999999999}]},{"osd":3,"last update":"Thu Oct 26 01:31:36 2023","interfaces":[{"interface":"back","average":{"1min":1.177,"5min":1.177,"15min":1.177},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":1.236},{"interface":"front","average":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"min":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"max":{"1min":1.6180000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":1.4590000000000001}]},{"osd":4,"last update":"Thu Oct 26 01:32:35 2023","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.3400000000000001,"15min":1.343},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":2.008,"5min":2.008,"15min":2.008},"last":1.077},{"interface":"front","average":{"1min":1.4379999999999999,"5min":1.3899999999999999,"15min":1.3819999999999999},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":2.339,"5min":2.339,"15min":2.339},"last":1.177}]},{"osd":5,"last update":"Thu Oct 26 01:32:35 2023","interfaces":[{"interface":"back","average":{"1min":1.411,"5min":1.3959999999999999,"15min":1.3939999999999999},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":2.0030000000000001,"5min":2.0030000000000001,"15min":2.0030000000000001},"last":0.97799999999999998},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.45,"15min":1.456},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.911,"5min":1.911,"15min":1.911},"last":1.494}]},{"osd":6,"last update":"Thu Oct 26 01:32:35 2023","interfaces":[{"interface":"back","average":{"1min":1.5309999999999999,"5min":1.917,"15min":1.9810000000000001},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":2.0539999999999998,"5min":11.577999999999999,"15min":11.577999999999999},"last":1.1299999999999999},{"interface":"front","average":{"1min":1.53,"5min":1.907,"15min":1.97},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":2.0870000000000002,"5min":10.704000000000001,"15min":10.704000000000001},"last":1.5509999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574876,"num_pgs":29,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":225,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195087,"statfs":{"total":200000000,"available":199769435,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:32:33 2023","interfaces":[{"interface":"back","average":{"1min":1.0249999999999999,"5min":1.1579999999999999,"15min":1.1799999999999999},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.4570000000000001,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":1.375},{"interface":"front","average":{"1min":0.92500000000000004,"5min":1.077,"15min":1.103},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.3919999999999999,"5min":1.843,"15min":1.843},"last":1.5620000000000001}]},{"osd":1,"last update":"Thu Oct 26 01:32:33 2023","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":1.149,"15min":1.167},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.7150000000000001,"5min":1.7150000000000001,"15min":1.7150000000000001},"last":1.6910000000000001},{"interface":"front","average":{"1min":1.1319999999999999,"5min":1.2070000000000001,"15min":1.2190000000000001},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.5209999999999999,"5min":2.085,"15min":2.085},"last":1.321}]},{"osd":2,"last update":"Thu Oct 26 01:32:33 2023","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.0469999999999999,"15min":1.05},"min":{"1min":0.78800000000000003,"5min":0.747,"15min":0.747},"max":{"1min":1.329,"5min":1.6659999999999999,"15min":1.6659999999999999},"last":1.6419999999999999},{"interface":"front","average":{"1min":1.232,"5min":1.244,"15min":1.246},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.9890000000000001,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":1.476}]},{"osd":3,"last update":"Thu Oct 26 01:32:37 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.167,"15min":1.1599999999999999},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.5800000000000001,"5min":1.5800000000000001,"15min":1.5800000000000001},"last":1.244},{"interface":"front","average":{"1min":1.3080000000000001,"5min":1.3180000000000001,"15min":1.3200000000000001},"min":{"1min":0.86799999999999999,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.881,"5min":1.881,"15min":1.881},"last":1.6020000000000001}]},{"osd":4,"last update":"Thu Oct 26 01:32:33 2023","interfaces":[{"interface":"back","average":{"1min":1.173,"5min":1.1970000000000001,"15min":1.2010000000000001},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.839,"5min":2.089,"15min":2.089},"last":1.006},{"interface":"front","average":{"1min":1.288,"5min":1.2609999999999999,"15min":1.2569999999999999},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.653,"5min":2.4129999999999998,"15min":2.4129999999999998},"last":1.4019999999999999}]},{"osd":5,"last update":"Thu Oct 26 01:32:33 2023","interfaces":[{"interface":"back","average":{"1min":1.458,"5min":1.4039999999999999,"15min":1.395},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.925,"5min":2.1640000000000001,"15min":2.1640000000000001},"last":1.5209999999999999},{"interface":"front","average":{"1min":1.47,"5min":1.4059999999999999,"15min":1.395},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":2.0710000000000002,"5min":2.0710000000000002,"15min":2.0710000000000002},"last":1.274}]},{"osd":7,"last update":"Thu Oct 26 01:32:33 2023","interfaces":[{"interface":"back","average":{"1min":1.357,"5min":1.355,"15min":1.355},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":2,"5min":2.012,"15min":2.012},"last":1.446},{"interface":"front","average":{"1min":1.5069999999999999,"5min":1.4450000000000001,"15min":1.4350000000000001},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":2.036,"5min":2.3239999999999998,"15min":2.3239999999999998},"last":1.0760000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574876,"num_pgs":15,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":225,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195087,"statfs":{"total":200000000,"available":199769423,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:32:38 2023","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"min":{"1min":0.66200000000000003,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.573,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":1.2110000000000001},{"interface":"front","average":{"1min":1.159,"5min":1.1359999999999999,"15min":1.1319999999999999},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.5309999999999999,"5min":1.5309999999999999,"15min":1.5309999999999999},"last":1.482}]},{"osd":1,"last update":"Thu Oct 26 01:32:38 2023","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":1.0600000000000001,"15min":1.0629999999999999},"min":{"1min":0.81499999999999995,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.3819999999999999,"5min":1.722,"15min":1.722},"last":1.1659999999999999},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.087,"15min":1.071},"min":{"1min":0.71099999999999997,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.5820000000000001,"5min":1.7669999999999999,"15min":1.7669999999999999},"last":0.97499999999999998}]},{"osd":2,"last update":"Thu Oct 26 01:32:38 2023","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":1.155,"15min":1.145},"min":{"1min":0.74399999999999999,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.6339999999999999,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":0.90700000000000003},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.1539999999999999,"15min":1.1579999999999999},"min":{"1min":0.85699999999999998,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.5269999999999999,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":1.347}]},{"osd":3,"last update":"Thu Oct 26 01:32:38 2023","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.206,"15min":1.212},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.492,"5min":1.835,"15min":1.835},"last":1.292},{"interface":"front","average":{"1min":1.25,"5min":1.276,"15min":1.28},"min":{"1min":0.81999999999999995,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.5980000000000001,"5min":2.9249999999999998,"15min":2.9249999999999998},"last":1.038}]},{"osd":4,"last update":"Thu Oct 26 01:32:38 2023","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":1.179,"15min":1.1699999999999999},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.6619999999999999,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":1.1100000000000001},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.3220000000000001,"15min":1.3109999999999999},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.736,"5min":1.736,"15min":1.736},"last":1.24}]},{"osd":6,"last update":"Thu Oct 26 01:32:38 2023","interfaces":[{"interface":"back","average":{"1min":1.3260000000000001,"5min":1.274,"15min":1.266},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.6140000000000001,"5min":1.72,"15min":1.72},"last":1.4379999999999999},{"interface":"front","average":{"1min":1.415,"5min":1.2949999999999999,"15min":1.2749999999999999},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.736,"5min":1.8560000000000001,"15min":1.8560000000000001},"last":1.573}]},{"osd":7,"last update":"Thu Oct 26 01:32:38 2023","interfaces":[{"interface":"back","average":{"1min":1.47,"5min":1.377,"15min":1.361},"min":{"1min":0.64000000000000001,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.8080000000000001,"5min":1.8080000000000001,"15min":1.8080000000000001},"last":1.389},{"interface":"front","average":{"1min":1.411,"5min":1.371,"15min":1.365},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.669,"5min":1.8140000000000001,"15min":1.8140000000000001},"last":1.5289999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574876,"num_pgs":15,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":232,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195079,"statfs":{"total":200000000,"available":199761581,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:32:32 2023","interfaces":[{"interface":"back","average":{"1min":0.98899999999999999,"5min":0.95399999999999996,"15min":0.94799999999999995},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":2.0289999999999999,"5min":2.0289999999999999,"15min":2.0289999999999999},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.99099999999999999,"15min":0.998},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.304,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":1.4690000000000001}]},{"osd":1,"last update":"Thu Oct 26 01:32:32 2023","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.089,"15min":1.0920000000000001},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":2.4249999999999998,"5min":2.4249999999999998,"15min":2.4249999999999998},"last":1.337},{"interface":"front","average":{"1min":1.1020000000000001,"5min":1.089,"15min":1.087},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":2.0760000000000001,"5min":2.0760000000000001,"15min":2.0760000000000001},"last":0.89900000000000002}]},{"osd":2,"last update":"Thu Oct 26 01:32:32 2023","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.1739999999999999,"15min":1.1699999999999999},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.714,"5min":1.714,"15min":1.714},"last":1.514},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.101,"15min":1.093},"min":{"1min":0.74299999999999999,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.673,"5min":1.673,"15min":1.673},"last":0.96899999999999997}]},{"osd":3,"last update":"Thu Oct 26 01:32:32 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.181,"15min":1.177},"min":{"1min":0.76400000000000001,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.891,"5min":1.891,"15min":1.891},"last":1.127},{"interface":"front","average":{"1min":1.155,"5min":1.135,"15min":1.1319999999999999},"min":{"1min":0.86099999999999999,"5min":0.748,"15min":0.748},"max":{"1min":1.5229999999999999,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":1.1679999999999999}]},{"osd":5,"last update":"Thu Oct 26 01:32:32 2023","interfaces":[{"interface":"back","average":{"1min":1.278,"5min":1.298,"15min":1.302},"min":{"1min":0.70499999999999996,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":2.0049999999999999,"5min":2.0049999999999999,"15min":2.0049999999999999},"last":1.038},{"interface":"front","average":{"1min":1.25,"5min":1.3100000000000001,"15min":1.3200000000000001},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.956,"5min":1.956,"15min":1.956},"last":1.413}]},{"osd":6,"last update":"Thu Oct 26 01:32:32 2023","interfaces":[{"interface":"back","average":{"1min":1.3220000000000001,"5min":1.2969999999999999,"15min":1.2929999999999999},"min":{"1min":1.04,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.6870000000000001,"5min":2.218,"15min":2.218},"last":1.095},{"interface":"front","average":{"1min":1.506,"5min":1.468,"15min":1.462},"min":{"1min":0.96899999999999997,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":2.2160000000000002,"5min":3.3500000000000001,"15min":3.3500000000000001},"last":1.208}]},{"osd":7,"last update":"Thu Oct 26 01:32:32 2023","interfaces":[{"interface":"back","average":{"1min":1.4590000000000001,"5min":1.474,"15min":1.476},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":2.5299999999999998,"5min":2.5299999999999998,"15min":2.5299999999999998},"last":1.5640000000000001},{"interface":"front","average":{"1min":1.464,"5min":1.429,"15min":1.423},"min":{"1min":1.1140000000000001,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":1.9339999999999999,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":1.2649999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574876,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":232,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195079,"statfs":{"total":200000000,"available":199761559,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:32:32 2023","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.87,"15min":0.85899999999999999},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":6.3200000000000003,"5min":6.3200000000000003,"15min":6.3200000000000003},"last":1.042},{"interface":"front","average":{"1min":0.94399999999999995,"5min":0.90300000000000002,"15min":0.89600000000000002},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":4.2619999999999996,"5min":4.2619999999999996,"15min":4.2619999999999996},"last":1.232}]},{"osd":1,"last update":"Thu Oct 26 01:32:32 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.89700000000000002,"15min":0.89300000000000002},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":5.1260000000000003,"5min":5.1260000000000003,"15min":5.1260000000000003},"last":0.82099999999999995},{"interface":"front","average":{"1min":1.0860000000000001,"5min":1.006,"15min":0.99199999999999999},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":6.1520000000000001,"5min":6.1520000000000001,"15min":6.1520000000000001},"last":1.2909999999999999}]},{"osd":3,"last update":"Thu Oct 26 01:32:36 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.90500000000000003,"15min":0.89700000000000002},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.5049999999999999,"5min":1.623,"15min":1.623},"last":1.4359999999999999},{"interface":"front","average":{"1min":1.2310000000000001,"5min":1.0009999999999999,"15min":0.96299999999999997},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":8.9019999999999992,"5min":8.9019999999999992,"15min":8.9019999999999992},"last":0.97199999999999998}]},{"osd":4,"last update":"Thu Oct 26 01:32:32 2023","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.244,"15min":1.228},"min":{"1min":0.751,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":6.2670000000000003,"5min":6.2670000000000003,"15min":6.2670000000000003},"last":1.502},{"interface":"front","average":{"1min":1.4570000000000001,"5min":1.2609999999999999,"15min":1.228},"min":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":7.343,"5min":7.343,"15min":7.343},"last":1.0700000000000001}]},{"osd":5,"last update":"Thu Oct 26 01:32:32 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.256,"15min":1.2529999999999999},"min":{"1min":0.80800000000000005,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":3.7170000000000001,"5min":3.7170000000000001,"15min":3.7170000000000001},"last":1.171},{"interface":"front","average":{"1min":1.365,"5min":1.2909999999999999,"15min":1.2789999999999999},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":6.3899999999999997,"5min":6.3899999999999997,"15min":6.3899999999999997},"last":1.1990000000000001}]},{"osd":6,"last update":"Thu Oct 26 01:32:32 2023","interfaces":[{"interface":"back","average":{"1min":1.4319999999999999,"5min":1.3180000000000001,"15min":1.2989999999999999},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":6.0490000000000004,"5min":6.0490000000000004,"15min":6.0490000000000004},"last":1.8160000000000001},{"interface":"front","average":{"1min":1.4019999999999999,"5min":1.3540000000000001,"15min":1.3460000000000001},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":5.0650000000000004,"5min":5.0650000000000004,"15min":5.0650000000000004},"last":1.115}]},{"osd":7,"last update":"Thu Oct 26 01:32:32 2023","interfaces":[{"interface":"back","average":{"1min":1.4219999999999999,"5min":1.3939999999999999,"15min":1.3899999999999999},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":6.4480000000000004,"5min":6.4480000000000004,"15min":6.4480000000000004},"last":1.143},{"interface":"front","average":{"1min":1.377,"5min":1.417,"15min":1.4239999999999999},"min":{"1min":1.0009999999999999,"5min":0.93400000000000005,"15min":0.93400000000000005},"max":{"1min":2.8439999999999999,"5min":2.8439999999999999,"15min":2.8439999999999999},"last":1.899}]}]},{"osd":1,"up_from":13,"seq":55834574876,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":225,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195087,"statfs":{"total":200000000,"available":199769423,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:32:33 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.70199999999999996,"15min":0.69399999999999995},"min":{"1min":0.36699999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":1.224},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.75600000000000001,"15min":0.76100000000000001},"min":{"1min":0.39800000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.4830000000000001,"5min":1.4830000000000001,"15min":1.4830000000000001},"last":0.82099999999999995}]},{"osd":2,"last update":"Thu Oct 26 01:32:33 2023","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":1.0049999999999999,"15min":1.0129999999999999},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.5389999999999999,"5min":2.4830000000000001,"15min":2.4830000000000001},"last":1.4450000000000001},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.84499999999999997,"15min":0.84299999999999997},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":1.167}]},{"osd":3,"last update":"Thu Oct 26 01:32:37 2023","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.94099999999999995,"15min":0.93300000000000005},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.75,"5min":1.75,"15min":1.75},"last":0.876},{"interface":"front","average":{"1min":0.98199999999999998,"5min":0.97199999999999998,"15min":0.96999999999999997},"min":{"1min":0.50900000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.802,"5min":1.802,"15min":1.802},"last":1.05}]},{"osd":4,"last update":"Thu Oct 26 01:32:33 2023","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.1899999999999999,"15min":1.1839999999999999},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.8129999999999999,"5min":2.0659999999999998,"15min":2.0659999999999998},"last":1.3400000000000001},{"interface":"front","average":{"1min":1.105,"5min":1.107,"15min":1.107},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.611,"5min":1.611,"15min":1.611},"last":1.325}]},{"osd":5,"last update":"Thu Oct 26 01:32:33 2023","interfaces":[{"interface":"back","average":{"1min":1.1459999999999999,"5min":1.2030000000000001,"15min":1.2130000000000001},"min":{"1min":0.879,"5min":0.879,"15min":0.879},"max":{"1min":1.702,"5min":1.702,"15min":1.702},"last":1.1160000000000001},{"interface":"front","average":{"1min":1.2470000000000001,"5min":1.3109999999999999,"15min":1.3220000000000001},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.782,"5min":2.2200000000000002,"15min":2.2200000000000002},"last":1.3919999999999999}]},{"osd":6,"last update":"Thu Oct 26 01:32:33 2023","interfaces":[{"interface":"back","average":{"1min":1.2230000000000001,"5min":1.2509999999999999,"15min":1.256},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.679,"5min":2.3780000000000001,"15min":2.3780000000000001},"last":1.522},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.2350000000000001,"15min":1.2290000000000001},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.905,"5min":1.984,"15min":1.984},"last":1.4830000000000001}]},{"osd":7,"last update":"Thu Oct 26 01:32:33 2023","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.2529999999999999,"15min":1.2649999999999999},"min":{"1min":0.78000000000000003,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.5620000000000001,"5min":2.6930000000000001,"15min":2.6930000000000001},"last":1.262},{"interface":"front","average":{"1min":1.3169999999999999,"5min":1.3049999999999999,"15min":1.3029999999999999},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.869,"5min":1.869,"15min":1.869},"last":1.2989999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574876,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":807,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194504,"statfs":{"total":200000000,"available":199172791,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Oct 26 01:32:37 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.85999999999999999,"15min":0.877},"min":{"1min":0.435,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.5609999999999999,"5min":4.5679999999999996,"15min":4.5679999999999996},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.85299999999999998,"5min":1.0069999999999999,"15min":1.0329999999999999},"min":{"1min":0.49399999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.252,"5min":4.8470000000000004,"15min":4.8470000000000004},"last":0.747}]},{"osd":2,"last update":"Thu Oct 26 01:32:37 2023","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":1.0980000000000001,"15min":1.127},"min":{"1min":0.61199999999999999,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.405,"5min":5.5599999999999996,"15min":5.5599999999999996},"last":0.97199999999999998},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.90300000000000002,"15min":0.91300000000000003},"min":{"1min":0.46800000000000003,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.337,"5min":4.9459999999999997,"15min":4.9459999999999997},"last":0.45200000000000001}]},{"osd":3,"last update":"Thu Oct 26 01:32:37 2023","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":1.0840000000000001,"15min":1.105},"min":{"1min":0.46999999999999997,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.867,"5min":4.9800000000000004,"15min":4.9800000000000004},"last":0.49099999999999999},{"interface":"front","average":{"1min":1.03,"5min":1.1459999999999999,"15min":1.165},"min":{"1min":0.56699999999999995,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.992,"5min":5.016,"15min":5.016},"last":0.66000000000000003}]},{"osd":4,"last update":"Thu Oct 26 01:32:37 2023","interfaces":[{"interface":"back","average":{"1min":1.133,"5min":1.2330000000000001,"15min":1.25},"min":{"1min":0.66800000000000004,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.671,"5min":5.1879999999999997,"15min":5.1879999999999997},"last":0.89900000000000002},{"interface":"front","average":{"1min":1.228,"5min":1.304,"15min":1.3169999999999999},"min":{"1min":0.79100000000000004,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":2.036,"5min":5.5229999999999997,"15min":5.5229999999999997},"last":1.417}]},{"osd":5,"last update":"Thu Oct 26 01:32:37 2023","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.4350000000000001,"15min":1.4450000000000001},"min":{"1min":1.044,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":2.2290000000000001,"5min":5.6360000000000001,"15min":5.6360000000000001},"last":0.93799999999999994},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.351,"15min":1.365},"min":{"1min":0.94799999999999995,"5min":0.626,"15min":0.626},"max":{"1min":1.74,"5min":5.3399999999999999,"15min":5.3399999999999999},"last":1.1990000000000001}]},{"osd":6,"last update":"Thu Oct 26 01:32:37 2023","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":1.3440000000000001,"15min":1.3620000000000001},"min":{"1min":0.80200000000000005,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.6100000000000001,"5min":5.3070000000000004,"15min":5.3070000000000004},"last":1.0009999999999999},{"interface":"front","average":{"1min":1.234,"5min":1.3640000000000001,"15min":1.3859999999999999},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.734,"5min":5.5919999999999996,"15min":5.5919999999999996},"last":1.4810000000000001}]},{"osd":7,"last update":"Thu Oct 26 01:32:37 2023","interfaces":[{"interface":"back","average":{"1min":1.3979999999999999,"5min":1.4870000000000001,"15min":1.502},"min":{"1min":0.90300000000000002,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":2.3330000000000002,"5min":5.6689999999999996,"15min":5.6689999999999996},"last":1.099},{"interface":"front","average":{"1min":1.345,"5min":1.4370000000000001,"15min":1.452},"min":{"1min":0.81299999999999994,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.9399999999999999,"5min":5.3719999999999999,"15min":5.3719999999999999},"last":1.272}]}]}],"pool_statfs":[]}} 2023-10-26T01:32:43.859 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-26T01:32:43.860 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-26T01:32:44.131 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:44.129+0000 7ff65fe56640 1 -- 172.21.15.169:0/846158054 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff65806ce50 msgr2=0x7ff65806b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:44.131 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:44.129+0000 7ff65fe56640 1 --2- 172.21.15.169:0/846158054 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff65806ce50 0x7ff65806b740 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7ff650005fd0 tx=0x7ff65002d4b0 comp rx=0 tx=0).stop 2023-10-26T01:32:44.131 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:44.129+0000 7ff65fe56640 1 -- 172.21.15.169:0/846158054 shutdown_connections 2023-10-26T01:32:44.131 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:44.129+0000 7ff65fe56640 1 --2- 172.21.15.169:0/846158054 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff65806ce50 0x7ff65806b740 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:44.132 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:44.129+0000 7ff65fe56640 1 -- 172.21.15.169:0/846158054 >> 172.21.15.169:0/846158054 conn(0x7ff658069930 msgr2=0x7ff658069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:44.132 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:44.129+0000 7ff65fe56640 1 -- 172.21.15.169:0/846158054 shutdown_connections 2023-10-26T01:32:44.132 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:44.129+0000 7ff65fe56640 1 -- 172.21.15.169:0/846158054 wait complete. 2023-10-26T01:32:44.133 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:44.131+0000 7ff65fe56640 1 Processor -- start 2023-10-26T01:32:44.133 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:44.131+0000 7ff65fe56640 1 -- start start 2023-10-26T01:32:44.134 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:44.132+0000 7ff65fe56640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff65806ce50 0x7ff6581e2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:44.135 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:44.132+0000 7ff65fe56640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff6580ab300 0x7ff65814c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:44.135 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:44.132+0000 7ff65fe56640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff658157be0 0x7ff6580ae9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:44.135 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:44.132+0000 7ff65fe56640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff65814cab0 con 0x7ff65806ce50 2023-10-26T01:32:44.135 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:44.132+0000 7ff65fe56640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff65814cf20 con 0x7ff6580ab300 2023-10-26T01:32:44.135 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:44.132+0000 7ff65fe56640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6581e3380 con 0x7ff658157be0 2023-10-26T01:32:44.136 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:44.132+0000 7ff65dbf2640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff65806ce50 0x7ff6581e2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:44.136 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:44.132+0000 7ff65dbf2640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff65806ce50 0x7ff6581e2cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:33464/0 (socket says 172.21.15.169:33464) 2023-10-26T01:32:44.136 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:44.132+0000 7ff65dbf2640 1 -- 172.21.15.169:0/2716538906 learned_addr learned my addr 172.21.15.169:0/2716538906 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:32:44.136 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:44.132+0000 7ff65dbf2640 1 -- 172.21.15.169:0/2716538906 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff658157be0 msgr2=0x7ff6580ae9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:44.136 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:44.132+0000 7ff65dbf2640 1 --2- 172.21.15.169:0/2716538906 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff658157be0 0x7ff6580ae9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:44.137 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:44.132+0000 7ff65dbf2640 1 -- 172.21.15.169:0/2716538906 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff6580ab300 msgr2=0x7ff65814c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:44.137 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:44.132+0000 7ff65dbf2640 1 --2- 172.21.15.169:0/2716538906 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff6580ab300 0x7ff65814c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:44.137 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:44.132+0000 7ff65dbf2640 1 -- 172.21.15.169:0/2716538906 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff65000a950 con 0x7ff65806ce50 2023-10-26T01:32:44.137 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:44.133+0000 7ff65dbf2640 1 --2- 172.21.15.169:0/2716538906 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff65806ce50 0x7ff6581e2cf0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7ff65400c400 tx=0x7ff65400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:44.137 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:44.133+0000 7ff64e7fc640 1 -- 172.21.15.169:0/2716538906 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6540052d0 con 0x7ff65806ce50 2023-10-26T01:32:44.138 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:44.133+0000 7ff64e7fc640 1 -- 172.21.15.169:0/2716538906 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff6540035f0 con 0x7ff65806ce50 2023-10-26T01:32:44.138 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:44.133+0000 7ff64e7fc640 1 -- 172.21.15.169:0/2716538906 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff654003a00 con 0x7ff65806ce50 2023-10-26T01:32:44.138 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:44.133+0000 7ff65fe56640 1 -- 172.21.15.169:0/2716538906 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff65814bdb0 con 0x7ff65806ce50 2023-10-26T01:32:44.138 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:44.133+0000 7ff65fe56640 1 -- 172.21.15.169:0/2716538906 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff65814c050 con 0x7ff65806ce50 2023-10-26T01:32:44.139 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:44.134+0000 7ff65fe56640 1 -- 172.21.15.169:0/2716538906 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff65804db10 con 0x7ff65806ce50 2023-10-26T01:32:44.139 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:44.135+0000 7ff64e7fc640 1 -- 172.21.15.169:0/2716538906 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ff654003b60 con 0x7ff65806ce50 2023-10-26T01:32:44.139 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:44.137+0000 7ff64e7fc640 1 --2- 172.21.15.169:0/2716538906 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff63c06ae40 0x7ff63c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:44.140 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:44.137+0000 7ff64e7fc640 1 -- 172.21.15.169:0/2716538906 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff654088de0 con 0x7ff65806ce50 2023-10-26T01:32:44.140 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:44.138+0000 7ff65d3f1640 1 --2- 172.21.15.169:0/2716538906 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff63c06ae40 0x7ff63c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:44.140 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:44.139+0000 7ff65d3f1640 1 --2- 172.21.15.169:0/2716538906 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff63c06ae40 0x7ff63c06d300 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff648007fd0 tx=0x7ff648007920 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:44.143 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:44.141+0000 7ff64e7fc640 1 -- 172.21.15.169:0/2716538906 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff654053e80 con 0x7ff65806ce50 2023-10-26T01:32:44.402 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:44.401+0000 7ff65fe56640 1 -- 172.21.15.169:0/2716538906 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7ff6580ac470 con 0x7ff65806ce50 2023-10-26T01:32:44.408 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:44.406+0000 7ff64e7fc640 1 -- 172.21.15.169:0/2716538906 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v12) v1 ==== 190+0+147 (secure 0 0 0) 0x7ff6540170e0 con 0x7ff65806ce50 2023-10-26T01:32:44.408 INFO:teuthology.orchestra.run.smithi169.stdout:[client.0] 2023-10-26T01:32:44.409 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQAgwTllKC7gOhAAPsvxGcuUA8/p8xqbUgGteA== 2023-10-26T01:32:44.409 INFO:teuthology.orchestra.run.smithi169.stdout: caps mds = "allow" 2023-10-26T01:32:44.409 INFO:teuthology.orchestra.run.smithi169.stdout: caps mon = "allow r" 2023-10-26T01:32:44.409 INFO:teuthology.orchestra.run.smithi169.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-10-26T01:32:44.409 INFO:teuthology.orchestra.run.smithi169.stderr:updated caps for client.0 2023-10-26T01:32:44.411 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:44.409+0000 7ff65fe56640 1 -- 172.21.15.169:0/2716538906 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff63c06ae40 msgr2=0x7ff63c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:44.411 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:44.409+0000 7ff65fe56640 1 --2- 172.21.15.169:0/2716538906 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff63c06ae40 0x7ff63c06d300 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff648007fd0 tx=0x7ff648007920 comp rx=0 tx=0).stop 2023-10-26T01:32:44.411 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:44.409+0000 7ff65fe56640 1 -- 172.21.15.169:0/2716538906 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff65806ce50 msgr2=0x7ff6581e2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:44.412 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:44.409+0000 7ff65fe56640 1 --2- 172.21.15.169:0/2716538906 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff65806ce50 0x7ff6581e2cf0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7ff65400c400 tx=0x7ff65400c8c0 comp rx=0 tx=0).stop 2023-10-26T01:32:44.412 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:44.409+0000 7ff65fe56640 1 -- 172.21.15.169:0/2716538906 shutdown_connections 2023-10-26T01:32:44.412 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:44.409+0000 7ff65fe56640 1 --2- 172.21.15.169:0/2716538906 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff63c06ae40 0x7ff63c06d300 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:44.412 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:44.409+0000 7ff65fe56640 1 --2- 172.21.15.169:0/2716538906 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff658157be0 0x7ff6580ae9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:44.412 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:44.409+0000 7ff65fe56640 1 --2- 172.21.15.169:0/2716538906 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff6580ab300 0x7ff65814c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:44.413 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:44.409+0000 7ff65fe56640 1 --2- 172.21.15.169:0/2716538906 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff65806ce50 0x7ff6581e2cf0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:44.413 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:44.409+0000 7ff65fe56640 1 -- 172.21.15.169:0/2716538906 >> 172.21.15.169:0/2716538906 conn(0x7ff658069930 msgr2=0x7ff6580705e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:44.413 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:44.409+0000 7ff65fe56640 1 -- 172.21.15.169:0/2716538906 shutdown_connections 2023-10-26T01:32:44.413 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:44.410+0000 7ff65fe56640 1 -- 172.21.15.169:0/2716538906 wait complete. 2023-10-26T01:32:44.431 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T01:32:44.650 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:44.648+0000 7f98e359e640 1 -- 172.21.15.169:0/4267863068 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98d0003970 msgr2=0x7f98d0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:44.650 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:44.648+0000 7f98e359e640 1 --2- 172.21.15.169:0/4267863068 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98d0003970 0x7f98d0003d90 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f98d4006fd0 tx=0x7f98d00057f0 comp rx=0 tx=0).stop 2023-10-26T01:32:44.650 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:44.648+0000 7f98e359e640 1 -- 172.21.15.169:0/4267863068 shutdown_connections 2023-10-26T01:32:44.651 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:44.648+0000 7f98e359e640 1 --2- 172.21.15.169:0/4267863068 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98d0003970 0x7f98d0003d90 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:44.651 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:44.648+0000 7f98e359e640 1 -- 172.21.15.169:0/4267863068 >> 172.21.15.169:0/4267863068 conn(0x7f98e40aedd0 msgr2=0x7f98e40af1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:44.651 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:44.648+0000 7f98e359e640 1 -- 172.21.15.169:0/4267863068 shutdown_connections 2023-10-26T01:32:44.651 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:44.648+0000 7f98e359e640 1 -- 172.21.15.169:0/4267863068 wait complete. 2023-10-26T01:32:44.652 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:44.650+0000 7f98e359e640 1 Processor -- start 2023-10-26T01:32:44.653 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:44.650+0000 7f98e359e640 1 -- start start 2023-10-26T01:32:44.653 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:44.651+0000 7f98e359e640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f98d0003970 0x7f98e4155de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:44.653 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:44.651+0000 7f98e359e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f98e415a2e0 0x7f98e40c8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:44.653 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:44.651+0000 7f98e359e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98e40c90c0 0x7f98e40b17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:44.653 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:44.651+0000 7f98e359e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98e40c9550 con 0x7f98e40c90c0 2023-10-26T01:32:44.654 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:44.651+0000 7f98e359e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98e41f6220 con 0x7f98e415a2e0 2023-10-26T01:32:44.654 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:44.651+0000 7f98e359e640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98e41f5610 con 0x7f98d0003970 2023-10-26T01:32:44.654 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:44.651+0000 7f98e2d9d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98e40c90c0 0x7f98e40b17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:44.654 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:44.651+0000 7f98e2d9d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98e40c90c0 0x7f98e40b17e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:33492/0 (socket says 172.21.15.169:33492) 2023-10-26T01:32:44.655 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:44.651+0000 7f98e2d9d640 1 -- 172.21.15.169:0/2707366912 learned_addr learned my addr 172.21.15.169:0/2707366912 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:32:44.656 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:44.651+0000 7f98e259c640 1 --2- 172.21.15.169:0/2707366912 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f98d0003970 0x7f98e4155de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:44.656 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:44.651+0000 7f98e2d9d640 1 -- 172.21.15.169:0/2707366912 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f98d0003970 msgr2=0x7f98e4155de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:44.656 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:44.651+0000 7f98e2d9d640 1 --2- 172.21.15.169:0/2707366912 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f98d0003970 0x7f98e4155de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:44.656 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:44.651+0000 7f98e2d9d640 1 -- 172.21.15.169:0/2707366912 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f98e415a2e0 msgr2=0x7f98e40c8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:44.657 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:44.651+0000 7f98e2d9d640 1 --2- 172.21.15.169:0/2707366912 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f98e415a2e0 0x7f98e40c8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:44.657 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:44.651+0000 7f98e2d9d640 1 -- 172.21.15.169:0/2707366912 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98e41f58f0 con 0x7f98e40c90c0 2023-10-26T01:32:44.657 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:44.652+0000 7f98e2d9d640 1 --2- 172.21.15.169:0/2707366912 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98e40c90c0 0x7f98e40b17e0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f98e40c9890 tx=0x7f98dc009a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:44.657 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:44.652+0000 7f98db7fe640 1 -- 172.21.15.169:0/2707366912 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98dc017070 con 0x7f98e40c90c0 2023-10-26T01:32:44.658 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:44.652+0000 7f98db7fe640 1 -- 172.21.15.169:0/2707366912 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f98dc004c10 con 0x7f98e40c90c0 2023-10-26T01:32:44.658 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:44.652+0000 7f98db7fe640 1 -- 172.21.15.169:0/2707366912 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98dc0106a0 con 0x7f98e40c90c0 2023-10-26T01:32:44.658 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:44.652+0000 7f98e359e640 1 -- 172.21.15.169:0/2707366912 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98e41f5bd0 con 0x7f98e40c90c0 2023-10-26T01:32:44.658 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:44.652+0000 7f98e359e640 1 -- 172.21.15.169:0/2707366912 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98e40b1d20 con 0x7f98e40c90c0 2023-10-26T01:32:44.658 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:44.653+0000 7f98e359e640 1 -- 172.21.15.169:0/2707366912 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98e4073c10 con 0x7f98e40c90c0 2023-10-26T01:32:44.659 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:44.654+0000 7f98db7fe640 1 -- 172.21.15.169:0/2707366912 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f98dc010800 con 0x7f98e40c90c0 2023-10-26T01:32:44.659 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:44.655+0000 7f98db7fe640 1 --2- 172.21.15.169:0/2707366912 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f98c406ae40 0x7f98c406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:44.660 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:44.655+0000 7f98db7fe640 1 -- 172.21.15.169:0/2707366912 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f98dc089a80 con 0x7f98e40c90c0 2023-10-26T01:32:44.660 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:44.656+0000 7f98e259c640 1 --2- 172.21.15.169:0/2707366912 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f98c406ae40 0x7f98c406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:44.660 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:44.657+0000 7f98e259c640 1 --2- 172.21.15.169:0/2707366912 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f98c406ae40 0x7f98c406d300 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f98d4034000 tx=0x7f98d402e000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:44.661 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:44.660+0000 7f98db7fe640 1 -- 172.21.15.169:0/2707366912 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f98dc054a90 con 0x7f98e40c90c0 2023-10-26T01:32:44.888 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:44.887+0000 7f98e359e640 1 -- 172.21.15.169:0/2707366912 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f98e41f6360 con 0x7f98e40c90c0 2023-10-26T01:32:44.889 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:44.888+0000 7f98db7fe640 1 -- 172.21.15.169:0/2707366912 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4925 (secure 0 0 0) 0x7f98dc0580b0 con 0x7f98e40c90c0 2023-10-26T01:32:44.890 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:32:44.890 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4805,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/2294107634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2294107634},{"type":"v1","addr":"172.21.15.170:6836","nonce":2294107634}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":4987,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2208026662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2208026662},{"type":"v1","addr":"172.21.15.169:6833","nonce":2208026662}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":4993,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/2630365058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":2630365058},{"type":"v1","addr":"172.21.15.170:6837","nonce":2630365058}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T01:32:38.059721+0000","modified":"2023-10-26T01:32:41.428183+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5664},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5664":{"gid":5664,"name":"a","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6835/2816266753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":2816266753},{"type":"v1","addr":"172.21.15.169:6835","nonce":2816266753}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-10-26T01:32:44.890 INFO:teuthology.orchestra.run.smithi169.stderr:dumped fsmap epoch 23 2023-10-26T01:32:44.894 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:44.890+0000 7f98e359e640 1 -- 172.21.15.169:0/2707366912 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f98c406ae40 msgr2=0x7f98c406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:44.894 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:44.890+0000 7f98e359e640 1 --2- 172.21.15.169:0/2707366912 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f98c406ae40 0x7f98c406d300 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f98d4034000 tx=0x7f98d402e000 comp rx=0 tx=0).stop 2023-10-26T01:32:44.894 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:44.890+0000 7f98e359e640 1 -- 172.21.15.169:0/2707366912 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98e40c90c0 msgr2=0x7f98e40b17e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:44.895 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:44.890+0000 7f98e359e640 1 --2- 172.21.15.169:0/2707366912 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98e40c90c0 0x7f98e40b17e0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f98e40c9890 tx=0x7f98dc009a80 comp rx=0 tx=0).stop 2023-10-26T01:32:44.895 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:44.890+0000 7f98e359e640 1 -- 172.21.15.169:0/2707366912 shutdown_connections 2023-10-26T01:32:44.895 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:44.890+0000 7f98e359e640 1 --2- 172.21.15.169:0/2707366912 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f98c406ae40 0x7f98c406d300 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:44.895 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:44.890+0000 7f98e359e640 1 --2- 172.21.15.169:0/2707366912 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98e40c90c0 0x7f98e40b17e0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:44.895 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:44.890+0000 7f98e359e640 1 --2- 172.21.15.169:0/2707366912 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f98e415a2e0 0x7f98e40c8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:44.896 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:44.890+0000 7f98e359e640 1 --2- 172.21.15.169:0/2707366912 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f98d0003970 0x7f98e4155de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:44.896 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:44.890+0000 7f98e359e640 1 -- 172.21.15.169:0/2707366912 >> 172.21.15.169:0/2707366912 conn(0x7f98e40aedd0 msgr2=0x7f98e406b3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:44.896 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:44.890+0000 7f98e359e640 1 -- 172.21.15.169:0/2707366912 shutdown_connections 2023-10-26T01:32:44.896 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:44.891+0000 7f98e359e640 1 -- 172.21.15.169:0/2707366912 wait complete. 2023-10-26T01:32:45.314 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-26T01:32:38.059721+0000', 'modified': '2023-10-26T01:32:41.428183+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5664}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5664': {'gid': 5664, 'name': 'a', 'rank': 0, 'incarnation': 19, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.169:6835/2816266753', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.169:6834', 'nonce': 2816266753}, {'type': 'v1', 'addr': '172.21.15.169:6835', 'nonce': 2816266753}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-26T01:32:45.314 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-26T01:32:45.315 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-26T01:32:45.549 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.547+0000 7f1bfbfbb640 1 -- 172.21.15.169:0/1637682971 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1bf415b620 msgr2=0x7f1bf40cd710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:45.550 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.547+0000 7f1bfbfbb640 1 --2- 172.21.15.169:0/1637682971 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1bf415b620 0x7f1bf40cd710 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f1bf0005fd0 tx=0x7f1bf002bd90 comp rx=0 tx=0).stop 2023-10-26T01:32:45.550 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.547+0000 7f1bfbfbb640 1 -- 172.21.15.169:0/1637682971 shutdown_connections 2023-10-26T01:32:45.550 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.547+0000 7f1bfbfbb640 1 --2- 172.21.15.169:0/1637682971 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1bf40c9d00 0x7f1bf40cdc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:45.550 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.547+0000 7f1bfbfbb640 1 --2- 172.21.15.169:0/1637682971 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1bf415b620 0x7f1bf40cd710 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:45.550 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.547+0000 7f1bfbfbb640 1 -- 172.21.15.169:0/1637682971 >> 172.21.15.169:0/1637682971 conn(0x7f1bf4069930 msgr2=0x7f1bf4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:45.551 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.547+0000 7f1bfbfbb640 1 -- 172.21.15.169:0/1637682971 shutdown_connections 2023-10-26T01:32:45.551 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.548+0000 7f1bfbfbb640 1 -- 172.21.15.169:0/1637682971 wait complete. 2023-10-26T01:32:45.551 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.549+0000 7f1bfbfbb640 1 Processor -- start 2023-10-26T01:32:45.552 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.549+0000 7f1bfbfbb640 1 -- start start 2023-10-26T01:32:45.553 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.550+0000 7f1bfbfbb640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1bf40ac320 0x7f1bf41418f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:45.553 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.550+0000 7f1bfbfbb640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1bf40c9d00 0x7f1bf415e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:45.553 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.550+0000 7f1bfbfbb640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1bf415b620 0x7f1bf415e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:45.554 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.550+0000 7f1bfbfbb640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bf40cb040 con 0x7f1bf415b620 2023-10-26T01:32:45.554 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.550+0000 7f1bfbfbb640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bf40ca0e0 con 0x7f1bf40ac320 2023-10-26T01:32:45.554 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.550+0000 7f1bfbfbb640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bf40ca3c0 con 0x7f1bf40c9d00 2023-10-26T01:32:45.554 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.551+0000 7f1bfa558640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1bf415b620 0x7f1bf415e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:45.554 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.551+0000 7f1bfa558640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1bf415b620 0x7f1bf415e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:33502/0 (socket says 172.21.15.169:33502) 2023-10-26T01:32:45.555 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.551+0000 7f1bfa558640 1 -- 172.21.15.169:0/3482562271 learned_addr learned my addr 172.21.15.169:0/3482562271 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:32:45.555 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.551+0000 7f1bf9d57640 1 --2- 172.21.15.169:0/3482562271 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1bf40ac320 0x7f1bf41418f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:45.555 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.551+0000 7f1bf9556640 1 --2- 172.21.15.169:0/3482562271 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1bf40c9d00 0x7f1bf415e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:45.555 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.551+0000 7f1bfa558640 1 -- 172.21.15.169:0/3482562271 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1bf40c9d00 msgr2=0x7f1bf415e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:45.555 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.551+0000 7f1bfa558640 1 --2- 172.21.15.169:0/3482562271 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1bf40c9d00 0x7f1bf415e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:45.556 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.551+0000 7f1bfa558640 1 -- 172.21.15.169:0/3482562271 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1bf40ac320 msgr2=0x7f1bf41418f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:45.556 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.551+0000 7f1bfa558640 1 --2- 172.21.15.169:0/3482562271 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1bf40ac320 0x7f1bf41418f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:45.556 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.551+0000 7f1bfa558640 1 -- 172.21.15.169:0/3482562271 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bf0009560 con 0x7f1bf415b620 2023-10-26T01:32:45.556 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.552+0000 7f1bfa558640 1 --2- 172.21.15.169:0/3482562271 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1bf415b620 0x7f1bf415e710 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f1bec00c780 tx=0x7f1bec00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:45.557 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.552+0000 7f1bea7fc640 1 -- 172.21.15.169:0/3482562271 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bec004b60 con 0x7f1bf415b620 2023-10-26T01:32:45.558 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.552+0000 7f1bea7fc640 1 -- 172.21.15.169:0/3482562271 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1bec00ce90 con 0x7f1bf415b620 2023-10-26T01:32:45.558 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.552+0000 7f1bea7fc640 1 -- 172.21.15.169:0/3482562271 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bec00aab0 con 0x7f1bf415b620 2023-10-26T01:32:45.558 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.552+0000 7f1bfbfbb640 1 -- 172.21.15.169:0/3482562271 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bf40ca6a0 con 0x7f1bf415b620 2023-10-26T01:32:45.558 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.552+0000 7f1bfbfbb640 1 -- 172.21.15.169:0/3482562271 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bf40cac60 con 0x7f1bf415b620 2023-10-26T01:32:45.558 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.554+0000 7f1bea7fc640 1 -- 172.21.15.169:0/3482562271 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f1bec00ac10 con 0x7f1bf415b620 2023-10-26T01:32:45.559 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.555+0000 7f1bea7fc640 1 --2- 172.21.15.169:0/3482562271 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f1bd806ae40 0x7f1bd806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:45.559 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.555+0000 7f1bea7fc640 1 -- 172.21.15.169:0/3482562271 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1bec089a80 con 0x7f1bf415b620 2023-10-26T01:32:45.559 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.556+0000 7f1bfec2a740 1 Processor -- start 2023-10-26T01:32:45.559 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.556+0000 7f1bf9d57640 1 --2- 172.21.15.169:0/3482562271 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f1bd806ae40 0x7f1bd806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:45.559 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.556+0000 7f1bfec2a740 1 -- start start 2023-10-26T01:32:45.562 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.557+0000 7f1bfec2a740 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x560e7ddee7d0 0x560e7ddc1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:45.562 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.557+0000 7f1bfec2a740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560e7ddeebb0 0x560e7ddf1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:45.562 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.557+0000 7f1bfec2a740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560e7ddc1f40 0x560e7ddf1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:45.562 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.557+0000 7f1bfec2a740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e7dd34740 con 0x560e7ddeebb0 2023-10-26T01:32:45.562 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.557+0000 7f1bfec2a740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e7dd31e30 con 0x560e7ddc1f40 2023-10-26T01:32:45.563 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.557+0000 7f1bfec2a740 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x560e7dd2d720 con 0x560e7ddee7d0 2023-10-26T01:32:45.563 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.557+0000 7f1bf9d57640 1 --2- 172.21.15.169:0/3482562271 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f1bd806ae40 0x7f1bd806d300 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f1bf0035000 tx=0x7f1bf0004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:45.563 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.557+0000 7f1bf9556640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560e7ddeebb0 0x560e7ddf1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:45.563 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.557+0000 7f1bf9556640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560e7ddeebb0 0x560e7ddf1560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:33508/0 (socket says 172.21.15.169:33508) 2023-10-26T01:32:45.563 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.557+0000 7f1bf9556640 1 -- 172.21.15.169:0/1706711817 learned_addr learned my addr 172.21.15.169:0/1706711817 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:32:45.564 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.558+0000 7f1bf9d57640 1 --2- 172.21.15.169:0/1706711817 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x560e7ddee7d0 0x560e7ddc1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:45.564 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.558+0000 7f1bfa558640 1 --2- 172.21.15.169:0/1706711817 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560e7ddc1f40 0x560e7ddf1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:45.564 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.558+0000 7f1bf9556640 1 -- 172.21.15.169:0/1706711817 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x560e7ddee7d0 msgr2=0x560e7ddc1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T01:32:45.564 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.558+0000 7f1bf9556640 1 --2- 172.21.15.169:0/1706711817 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x560e7ddee7d0 0x560e7ddc1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:45.564 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.558+0000 7f1bf9556640 1 -- 172.21.15.169:0/1706711817 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560e7ddc1f40 msgr2=0x560e7ddf1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T01:32:45.565 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.558+0000 7f1bf9556640 1 --2- 172.21.15.169:0/1706711817 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560e7ddc1f40 0x560e7ddf1aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:45.565 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.558+0000 7f1bf9556640 1 -- 172.21.15.169:0/1706711817 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560e7dcf1910 con 0x560e7ddeebb0 2023-10-26T01:32:45.565 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.558+0000 7f1bf9d57640 1 --2- 172.21.15.169:0/1706711817 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x560e7ddee7d0 0x560e7ddc1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:32:45.565 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.558+0000 7f1bf9556640 1 --2- 172.21.15.169:0/1706711817 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560e7ddeebb0 0x560e7ddf1560 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f1be400b820 tx=0x7f1be400bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e2f4030c783fe8e server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:45.565 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.558+0000 7f1bd77fe640 1 -- 172.21.15.169:0/1706711817 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1be400d950 con 0x560e7ddeebb0 2023-10-26T01:32:45.565 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.558+0000 7f1bd77fe640 1 -- 172.21.15.169:0/1706711817 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1be400dba0 con 0x560e7ddeebb0 2023-10-26T01:32:45.566 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.558+0000 7f1bd77fe640 1 -- 172.21.15.169:0/1706711817 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1be4013480 con 0x560e7ddeebb0 2023-10-26T01:32:45.566 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.559+0000 7f1bfec2a740 1 -- 172.21.15.169:0/1706711817 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560e7ddeebb0 msgr2=0x560e7ddf1560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:45.566 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.559+0000 7f1bfec2a740 1 --2- 172.21.15.169:0/1706711817 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560e7ddeebb0 0x560e7ddf1560 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f1be400b820 tx=0x7f1be400bce0 comp rx=0 tx=0).stop 2023-10-26T01:32:45.566 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.559+0000 7f1bfec2a740 1 -- 172.21.15.169:0/1706711817 shutdown_connections 2023-10-26T01:32:45.566 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.559+0000 7f1bfec2a740 1 --2- 172.21.15.169:0/1706711817 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560e7ddeebb0 0x560e7ddf1560 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:45.567 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.559+0000 7f1bfec2a740 1 --2- 172.21.15.169:0/1706711817 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560e7ddc1f40 0x560e7ddf1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:45.567 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.559+0000 7f1bfec2a740 1 --2- 172.21.15.169:0/1706711817 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x560e7ddee7d0 0x560e7ddc1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:45.567 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.559+0000 7f1bfec2a740 1 -- 172.21.15.169:0/1706711817 >> 172.21.15.169:0/1706711817 conn(0x560e7ddc3c70 msgr2=0x560e7de0c560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:45.567 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.559+0000 7f1bfec2a740 1 -- 172.21.15.169:0/1706711817 shutdown_connections 2023-10-26T01:32:45.567 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.559+0000 7f1bfec2a740 1 -- 172.21.15.169:0/1706711817 wait complete. 2023-10-26T01:32:45.568 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.560+0000 7f1bfec2a740 1 -- start start 2023-10-26T01:32:45.568 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.560+0000 7f1bfec2a740 1 Processor -- start 2023-10-26T01:32:45.571 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.561+0000 7f1bfec2a740 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x560e7ddee7d0 0x560e7de0d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:45.572 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.561+0000 7f1bfec2a740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560e7ddeebb0 0x560e7ddefae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:45.572 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.561+0000 7f1bfec2a740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560e7dddba10 0x560e7ddf5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:45.572 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.561+0000 7f1bfec2a740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e7dccb970 con 0x560e7ddeebb0 2023-10-26T01:32:45.572 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.561+0000 7f1bfec2a740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e7dccb610 con 0x560e7dddba10 2023-10-26T01:32:45.572 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.561+0000 7f1bfec2a740 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x560e7da91f20 con 0x560e7ddee7d0 2023-10-26T01:32:45.573 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.561+0000 7f1bfec2a740 10 client.0 _ll_register_callbacks cb 0x560e7dcef0c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T01:32:45.573 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.562+0000 7f1bf9556640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560e7ddeebb0 0x560e7ddefae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:45.573 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.562+0000 7f1bf9556640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560e7ddeebb0 0x560e7ddefae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:33514/0 (socket says 172.21.15.169:33514) 2023-10-26T01:32:45.573 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.562+0000 7f1bf9556640 1 -- 172.21.15.169:0/1862179587 learned_addr learned my addr 172.21.15.169:0/1862179587 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:32:45.573 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.562+0000 7f1bf9d57640 1 --2- 172.21.15.169:0/1862179587 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x560e7ddee7d0 0x560e7de0d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:45.574 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.562+0000 7f1bfa558640 1 --2- 172.21.15.169:0/1862179587 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560e7dddba10 0x560e7ddf5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:45.574 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.562+0000 7f1bf9556640 1 -- 172.21.15.169:0/1862179587 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x560e7ddee7d0 msgr2=0x560e7de0d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T01:32:45.574 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.562+0000 7f1bf9556640 1 --2- 172.21.15.169:0/1862179587 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x560e7ddee7d0 0x560e7de0d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:45.574 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.562+0000 7f1bf9556640 1 -- 172.21.15.169:0/1862179587 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560e7dddba10 msgr2=0x560e7ddf5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T01:32:45.574 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.562+0000 7f1bf9556640 1 --2- 172.21.15.169:0/1862179587 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560e7dddba10 0x560e7ddf5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:45.575 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.562+0000 7f1bf9556640 1 -- 172.21.15.169:0/1862179587 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1be400e040 con 0x560e7ddeebb0 2023-10-26T01:32:45.575 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.562+0000 7f1bf9d57640 1 --2- 172.21.15.169:0/1862179587 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x560e7ddee7d0 0x560e7de0d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:32:45.575 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.562+0000 7f1bfa558640 1 --2- 172.21.15.169:0/1862179587 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560e7dddba10 0x560e7ddf5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:32:45.575 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.563+0000 7f1bf9556640 1 --2- 172.21.15.169:0/1862179587 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560e7ddeebb0 0x560e7ddefae0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f1be4015040 tx=0x7f1be400df40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ff3d14c20951874 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:45.575 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.563+0000 7f1bd4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.169:3300/0 2023-10-26T01:32:45.576 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.563+0000 7f1bd4ff9640 1 -- 172.21.15.169:0/1862179587 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1be4013690 con 0x560e7ddeebb0 2023-10-26T01:32:45.576 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.563+0000 7f1bfec2a740 1 -- 172.21.15.169:0/1862179587 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560e7dd26700 con 0x560e7ddeebb0 2023-10-26T01:32:45.576 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.563+0000 7f1bd4ff9640 1 -- 172.21.15.169:0/1862179587 <== mon.0 v2:172.21.15.169:3300/0 2 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1be40137f0 con 0x560e7ddeebb0 2023-10-26T01:32:45.576 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.563+0000 7f1bd4ff9640 1 -- 172.21.15.169:0/1862179587 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1be40139b0 con 0x560e7ddeebb0 2023-10-26T01:32:45.576 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.563+0000 7f1bfec2a740 10 client.5793 fetch_fsmap learned FSMap version 23 2023-10-26T01:32:45.576 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.563+0000 7f1bfec2a740 1 -- 172.21.15.169:0/1862179587 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x560e7dd2bda0 con 0x560e7ddeebb0 2023-10-26T01:32:45.577 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.563+0000 7f1bd4ff9640 1 -- 172.21.15.169:0/1862179587 <== mon.0 v2:172.21.15.169:3300/0 4 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x560e7dd26700 con 0x560e7ddeebb0 2023-10-26T01:32:45.577 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.564+0000 7f1bd4ff9640 1 -- 172.21.15.169:0/1862179587 <== mon.0 v2:172.21.15.169:3300/0 5 ==== fsmap(e 23) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1be4013d00 con 0x560e7ddeebb0 2023-10-26T01:32:45.577 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.564+0000 7f1bfec2a740 10 client.5793 fetch_fsmap finished waiting for FSMap version 23 2023-10-26T01:32:45.577 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.564+0000 7f1bfec2a740 10 client.5793 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-26T01:32:45.577 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.564+0000 7f1bfec2a740 20 client.5793 populate_metadata read hostname 'smithi169' 2023-10-26T01:32:45.578 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.564+0000 7f1bfec2a740 1 --2- 172.21.15.169:0/1862179587 >> [v2:172.21.15.169:6834/2816266753,v1:172.21.15.169:6835/2816266753] conn(0x560e7ddd9610 0x560e7de1f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:45.578 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.564+0000 7f1bfec2a740 4 client.5793 mds_command: new command op to 5664 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-26T01:32:45.578 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.564+0000 7f1bfec2a740 1 -- 172.21.15.169:0/1862179587 --> [v2:172.21.15.169:6834/2816266753,v1:172.21.15.169:6835/2816266753] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560e7dd26420 con 0x560e7ddd9610 2023-10-26T01:32:45.578 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.564+0000 7f1bf9d57640 1 --2- 172.21.15.169:0/1862179587 >> [v2:172.21.15.169:6834/2816266753,v1:172.21.15.169:6835/2816266753] conn(0x560e7ddd9610 0x560e7de1f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:45.578 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.565+0000 7f1bf9d57640 1 --2- 172.21.15.169:0/1862179587 >> [v2:172.21.15.169:6834/2816266753,v1:172.21.15.169:6835/2816266753] conn(0x560e7ddd9610 0x560e7de1f890 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5f9b13a5fe1f8db server_cookie=d0255f864ab2d660 in_seq=0 out_seq=0 2023-10-26T01:32:45.579 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.565+0000 7f1bd4ff9640 10 client.5793 ms_handle_connect on v2:172.21.15.169:6834/2816266753 2023-10-26T01:32:45.579 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.565+0000 7f1bd4ff9640 1 -- 172.21.15.169:0/1862179587 <== mds.0 v2:172.21.15.169:6834/2816266753 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x560e7dd26420 con 0x560e7ddd9610 2023-10-26T01:32:45.579 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.565+0000 7f1bd4ff9640 10 client.5793 handle_command_reply: tid=0 2023-10-26T01:32:45.579 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.565+0000 7f1bfec2a740 1 client.5793 shutdown 2023-10-26T01:32:45.579 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.566+0000 7f1bfec2a740 1 -- 172.21.15.169:0/1862179587 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560e7ddeebb0 msgr2=0x560e7ddefae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:45.580 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.566+0000 7f1bfec2a740 1 --2- 172.21.15.169:0/1862179587 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560e7ddeebb0 0x560e7ddefae0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f1be4015040 tx=0x7f1be400df40 comp rx=0 tx=0).stop 2023-10-26T01:32:45.580 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.566+0000 7f1bfec2a740 1 -- 172.21.15.169:0/1862179587 shutdown_connections 2023-10-26T01:32:45.580 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.566+0000 7f1bfec2a740 1 --2- 172.21.15.169:0/1862179587 >> [v2:172.21.15.169:6834/2816266753,v1:172.21.15.169:6835/2816266753] conn(0x560e7ddd9610 0x560e7de1f890 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:45.580 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.566+0000 7f1bfec2a740 1 --2- 172.21.15.169:0/1862179587 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560e7ddeebb0 0x560e7ddefae0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:45.580 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.566+0000 7f1bfec2a740 1 --2- 172.21.15.169:0/1862179587 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560e7dddba10 0x560e7ddf5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:45.580 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.566+0000 7f1bd4ff9640 0 client.5793 ms_handle_reset on v2:172.21.15.169:6834/2816266753 2023-10-26T01:32:45.581 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.566+0000 7f1bfec2a740 1 --2- 172.21.15.169:0/1862179587 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x560e7ddee7d0 0x560e7de0d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:45.581 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.566+0000 7f1bfec2a740 1 -- 172.21.15.169:0/1862179587 >> 172.21.15.169:0/1862179587 conn(0x560e7ddc1f40 msgr2=0x560e7ddc5650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:45.581 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.566+0000 7f1bfec2a740 1 -- 172.21.15.169:0/1862179587 shutdown_connections 2023-10-26T01:32:45.581 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.566+0000 7f1bfec2a740 1 -- 172.21.15.169:0/1862179587 wait complete. 2023-10-26T01:32:45.581 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.566+0000 7f1bfec2a740 20 client.5793 trim_cache size 0 max 16384 2023-10-26T01:32:45.584 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.582+0000 7f1bfec2a740 1 Processor -- start 2023-10-26T01:32:45.584 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.583+0000 7f1bfec2a740 1 -- start start 2023-10-26T01:32:45.585 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.583+0000 7f1bfec2a740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560e7dddba10 0x560e7dddb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:45.585 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.583+0000 7f1bfec2a740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560e7ddee7d0 0x560e7ddef350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:45.586 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.583+0000 7f1bfec2a740 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x560e7ddeebb0 0x560e7ddef890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:45.586 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.583+0000 7f1bfec2a740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e7ddc88c0 con 0x560e7dddba10 2023-10-26T01:32:45.586 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.583+0000 7f1bfec2a740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e7ddefdd0 con 0x560e7ddee7d0 2023-10-26T01:32:45.586 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.583+0000 7f1bfec2a740 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x560e7ddc0d70 con 0x560e7ddeebb0 2023-10-26T01:32:45.586 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.584+0000 7f1bf9d57640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560e7dddba10 0x560e7dddb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:45.587 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.584+0000 7f1bf9d57640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560e7dddba10 0x560e7dddb2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:33518/0 (socket says 172.21.15.169:33518) 2023-10-26T01:32:45.587 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.584+0000 7f1bf9d57640 1 -- 172.21.15.169:0/141282421 learned_addr learned my addr 172.21.15.169:0/141282421 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:32:45.588 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.584+0000 7f1bf9d57640 1 -- 172.21.15.169:0/141282421 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x560e7ddeebb0 msgr2=0x560e7ddef890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T01:32:45.588 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.584+0000 7f1bf9d57640 1 --2- 172.21.15.169:0/141282421 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x560e7ddeebb0 0x560e7ddef890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:45.589 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.584+0000 7f1bf9d57640 1 -- 172.21.15.169:0/141282421 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560e7ddee7d0 msgr2=0x560e7ddef350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T01:32:45.589 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.584+0000 7f1bf9d57640 1 --2- 172.21.15.169:0/141282421 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560e7ddee7d0 0x560e7ddef350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:45.589 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.584+0000 7f1bf9d57640 1 -- 172.21.15.169:0/141282421 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560e7dccab80 con 0x560e7dddba10 2023-10-26T01:32:45.589 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.584+0000 7f1bf9d57640 1 --2- 172.21.15.169:0/141282421 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560e7dddba10 0x560e7dddb2f0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f1bf0009230 tx=0x7f1bf003e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=695ec4e4f427cd86 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:45.589 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.585+0000 7f1bd7fff640 1 -- 172.21.15.169:0/141282421 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bf0047030 con 0x560e7dddba10 2023-10-26T01:32:45.590 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.585+0000 7f1bd7fff640 1 -- 172.21.15.169:0/141282421 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1bf0048040 con 0x560e7dddba10 2023-10-26T01:32:45.590 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.585+0000 7f1bd7fff640 1 -- 172.21.15.169:0/141282421 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bf004e420 con 0x560e7dddba10 2023-10-26T01:32:45.590 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.585+0000 7f1bfec2a740 1 -- 172.21.15.169:0/141282421 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560e7dddba10 msgr2=0x560e7dddb2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:45.590 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.585+0000 7f1bfec2a740 1 --2- 172.21.15.169:0/141282421 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560e7dddba10 0x560e7dddb2f0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f1bf0009230 tx=0x7f1bf003e000 comp rx=0 tx=0).stop 2023-10-26T01:32:45.590 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.585+0000 7f1bfec2a740 1 -- 172.21.15.169:0/141282421 shutdown_connections 2023-10-26T01:32:45.591 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.585+0000 7f1bfec2a740 1 --2- 172.21.15.169:0/141282421 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x560e7ddeebb0 0x560e7ddef890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:45.591 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.585+0000 7f1bfec2a740 1 --2- 172.21.15.169:0/141282421 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560e7ddee7d0 0x560e7ddef350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:45.591 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.585+0000 7f1bfec2a740 1 --2- 172.21.15.169:0/141282421 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560e7dddba10 0x560e7dddb2f0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:45.591 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.585+0000 7f1bfec2a740 1 -- 172.21.15.169:0/141282421 >> 172.21.15.169:0/141282421 conn(0x560e7ddc1f40 msgr2=0x560e7ddee070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:45.591 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.585+0000 7f1bfec2a740 1 -- 172.21.15.169:0/141282421 shutdown_connections 2023-10-26T01:32:45.592 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.585+0000 7f1bfec2a740 1 -- 172.21.15.169:0/141282421 wait complete. 2023-10-26T01:32:45.592 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.586+0000 7f1bfec2a740 1 -- start start 2023-10-26T01:32:45.592 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.586+0000 7f1bfec2a740 1 Processor -- start 2023-10-26T01:32:45.595 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.587+0000 7f1bfec2a740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560e7dddba10 0x560e7de74680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:45.595 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.587+0000 7f1bfec2a740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560e7ddee7d0 0x560e7de25a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:45.595 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.587+0000 7f1bfec2a740 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x560e7ddeebb0 0x560e7de25f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:45.595 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.587+0000 7f1bfec2a740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e7dde8290 con 0x560e7dddba10 2023-10-26T01:32:45.596 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.587+0000 7f1bfec2a740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e7ddf0140 con 0x560e7ddee7d0 2023-10-26T01:32:45.596 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.587+0000 7f1bfec2a740 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x560e7ddefb00 con 0x560e7ddeebb0 2023-10-26T01:32:45.596 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.587+0000 7f1bfec2a740 10 client.0 _ll_register_callbacks cb 0x7f1be400b220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T01:32:45.596 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.587+0000 7f1bf9d57640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560e7dddba10 0x560e7de74680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:45.596 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.587+0000 7f1bf9d57640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560e7dddba10 0x560e7de74680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:33534/0 (socket says 172.21.15.169:33534) 2023-10-26T01:32:45.597 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.587+0000 7f1bf9d57640 1 -- 172.21.15.169:0/3712974940 learned_addr learned my addr 172.21.15.169:0/3712974940 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:32:45.597 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.588+0000 7f1bf9556640 1 --2- 172.21.15.169:0/3712974940 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560e7ddee7d0 0x560e7de25a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:45.597 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.588+0000 7f1bf9d57640 1 -- 172.21.15.169:0/3712974940 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x560e7ddeebb0 msgr2=0x560e7de25f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T01:32:45.597 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.588+0000 7f1bf9d57640 1 --2- 172.21.15.169:0/3712974940 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x560e7ddeebb0 0x560e7de25f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:45.597 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.588+0000 7f1bf9d57640 1 -- 172.21.15.169:0/3712974940 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560e7ddee7d0 msgr2=0x560e7de25a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T01:32:45.598 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.588+0000 7f1bf9d57640 1 --2- 172.21.15.169:0/3712974940 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560e7ddee7d0 0x560e7de25a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:45.598 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.588+0000 7f1bf9d57640 1 -- 172.21.15.169:0/3712974940 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bf415fa00 con 0x560e7dddba10 2023-10-26T01:32:45.598 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.588+0000 7f1bf9d57640 1 --2- 172.21.15.169:0/3712974940 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560e7dddba10 0x560e7de74680 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f1bf0005770 tx=0x7f1bf0004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc3657735678c3d9 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:45.598 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.588+0000 7f1bd4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.169:3300/0 2023-10-26T01:32:45.598 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.588+0000 7f1bd4ff9640 1 -- 172.21.15.169:0/3712974940 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bf0047050 con 0x560e7dddba10 2023-10-26T01:32:45.599 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.588+0000 7f1bd4ff9640 1 -- 172.21.15.169:0/3712974940 <== mon.0 v2:172.21.15.169:3300/0 2 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1bf005fac0 con 0x560e7dddba10 2023-10-26T01:32:45.599 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.588+0000 7f1bfec2a740 1 -- 172.21.15.169:0/3712974940 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1be400dba0 con 0x560e7dddba10 2023-10-26T01:32:45.599 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.589+0000 7f1bd4ff9640 1 -- 172.21.15.169:0/3712974940 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1bf004ee50 con 0x560e7dddba10 2023-10-26T01:32:45.599 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.589+0000 7f1bfec2a740 10 client.5799 fetch_fsmap learned FSMap version 23 2023-10-26T01:32:45.599 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.589+0000 7f1bfec2a740 1 -- 172.21.15.169:0/3712974940 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x560e7dcf0170 con 0x560e7dddba10 2023-10-26T01:32:45.600 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.589+0000 7f1bd4ff9640 1 -- 172.21.15.169:0/3712974940 <== mon.0 v2:172.21.15.169:3300/0 4 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1be400dba0 con 0x560e7dddba10 2023-10-26T01:32:45.600 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.589+0000 7f1bd4ff9640 1 -- 172.21.15.169:0/3712974940 <== mon.0 v2:172.21.15.169:3300/0 5 ==== fsmap(e 23) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1bf005fc80 con 0x560e7dddba10 2023-10-26T01:32:45.600 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.589+0000 7f1bfec2a740 10 client.5799 fetch_fsmap finished waiting for FSMap version 23 2023-10-26T01:32:45.600 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.589+0000 7f1bfec2a740 10 client.5799 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-26T01:32:45.600 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.589+0000 7f1bfec2a740 20 client.5799 populate_metadata read hostname 'smithi169' 2023-10-26T01:32:45.601 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.589+0000 7f1bfec2a740 1 --2- 172.21.15.169:0/3712974940 >> [v2:172.21.15.169:6834/2816266753,v1:172.21.15.169:6835/2816266753] conn(0x560e7de1b550 0x560e7de1b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:45.601 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.589+0000 7f1bfec2a740 4 client.5799 mds_command: new command op to 5664 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-26T01:32:45.601 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.589+0000 7f1bfec2a740 1 -- 172.21.15.169:0/3712974940 --> [v2:172.21.15.169:6834/2816266753,v1:172.21.15.169:6835/2816266753] -- command(tid 0: {"prefix": "status"}) v1 -- 0x560e7da2f6c0 con 0x560e7de1b550 2023-10-26T01:32:45.601 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.589+0000 7f1bf9556640 1 --2- 172.21.15.169:0/3712974940 >> [v2:172.21.15.169:6834/2816266753,v1:172.21.15.169:6835/2816266753] conn(0x560e7de1b550 0x560e7de1b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:45.601 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.590+0000 7f1bf9556640 1 --2- 172.21.15.169:0/3712974940 >> [v2:172.21.15.169:6834/2816266753,v1:172.21.15.169:6835/2816266753] conn(0x560e7de1b550 0x560e7de1b930 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d892d28e3c333755 server_cookie=85deed66c5ee67fd in_seq=0 out_seq=0 2023-10-26T01:32:45.601 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.590+0000 7f1bd4ff9640 10 client.5799 ms_handle_connect on v2:172.21.15.169:6834/2816266753 2023-10-26T01:32:45.602 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.590+0000 7f1bd4ff9640 1 -- 172.21.15.169:0/3712974940 <== mds.0 v2:172.21.15.169:6834/2816266753 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+318 (crc 0 0 0) 0x560e7da2f6c0 con 0x560e7de1b550 2023-10-26T01:32:45.602 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.590+0000 7f1bd4ff9640 10 client.5799 handle_command_reply: tid=0 2023-10-26T01:32:45.602 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.590+0000 7f1bfec2a740 1 client.5799 shutdown 2023-10-26T01:32:45.602 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.590+0000 7f1bfec2a740 1 -- 172.21.15.169:0/3712974940 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560e7dddba10 msgr2=0x560e7de74680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:45.602 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.590+0000 7f1bfec2a740 1 --2- 172.21.15.169:0/3712974940 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560e7dddba10 0x560e7de74680 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f1bf0005770 tx=0x7f1bf0004340 comp rx=0 tx=0).stop 2023-10-26T01:32:45.603 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.590+0000 7f1bfec2a740 1 -- 172.21.15.169:0/3712974940 shutdown_connections 2023-10-26T01:32:45.603 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.590+0000 7f1bfec2a740 1 --2- 172.21.15.169:0/3712974940 >> [v2:172.21.15.169:6834/2816266753,v1:172.21.15.169:6835/2816266753] conn(0x560e7de1b550 0x560e7de1b930 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:45.603 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.590+0000 7f1bfec2a740 1 --2- 172.21.15.169:0/3712974940 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x560e7ddeebb0 0x560e7de25f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:45.603 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.590+0000 7f1bfec2a740 1 --2- 172.21.15.169:0/3712974940 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560e7ddee7d0 0x560e7de25a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:45.603 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.590+0000 7f1bfec2a740 1 --2- 172.21.15.169:0/3712974940 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560e7dddba10 0x560e7de74680 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:45.604 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.590+0000 7f1bd4ff9640 0 client.5799 ms_handle_reset on v2:172.21.15.169:6834/2816266753 2023-10-26T01:32:45.604 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.590+0000 7f1bfec2a740 1 -- 172.21.15.169:0/3712974940 >> 172.21.15.169:0/3712974940 conn(0x560e7ddc1f40 msgr2=0x560e7ddee050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:45.604 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.591+0000 7f1bfec2a740 1 -- 172.21.15.169:0/3712974940 shutdown_connections 2023-10-26T01:32:45.604 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.591+0000 7f1bfec2a740 1 -- 172.21.15.169:0/3712974940 wait complete. 2023-10-26T01:32:45.604 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.591+0000 7f1bfec2a740 20 client.5799 trim_cache size 0 max 16384 2023-10-26T01:32:45.605 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2023-10-26T01:32:45.605 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_fsid": "30203b28-ed8a-44e8-a9a6-a9bf25f980f1", 2023-10-26T01:32:45.605 INFO:teuthology.orchestra.run.smithi169.stdout: "whoami": 0, 2023-10-26T01:32:45.605 INFO:teuthology.orchestra.run.smithi169.stdout: "id": 5664, 2023-10-26T01:32:45.606 INFO:teuthology.orchestra.run.smithi169.stdout: "want_state": "up:active", 2023-10-26T01:32:45.606 INFO:teuthology.orchestra.run.smithi169.stdout: "state": "up:active", 2023-10-26T01:32:45.606 INFO:teuthology.orchestra.run.smithi169.stdout: "fs_name": "cephfs", 2023-10-26T01:32:45.606 INFO:teuthology.orchestra.run.smithi169.stdout: "rank_uptime": 6.672990553, 2023-10-26T01:32:45.606 INFO:teuthology.orchestra.run.smithi169.stdout: "mdsmap_epoch": 23, 2023-10-26T01:32:45.607 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_epoch": 49, 2023-10-26T01:32:45.607 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_epoch_barrier": 49, 2023-10-26T01:32:45.607 INFO:teuthology.orchestra.run.smithi169.stdout: "uptime": 12.855319648 2023-10-26T01:32:45.607 INFO:teuthology.orchestra.run.smithi169.stdout:} 2023-10-26T01:32:45.608 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.591+0000 7f1bfbfbb640 1 -- 172.21.15.169:0/3482562271 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f1bd806ae40 msgr2=0x7f1bd806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:45.608 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.591+0000 7f1bfbfbb640 1 --2- 172.21.15.169:0/3482562271 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f1bd806ae40 0x7f1bd806d300 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f1bf0035000 tx=0x7f1bf0004730 comp rx=0 tx=0).stop 2023-10-26T01:32:45.608 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.591+0000 7f1bfbfbb640 1 -- 172.21.15.169:0/3482562271 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1bf415b620 msgr2=0x7f1bf415e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:45.609 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.591+0000 7f1bfbfbb640 1 --2- 172.21.15.169:0/3482562271 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1bf415b620 0x7f1bf415e710 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f1bec00c780 tx=0x7f1bec00cc40 comp rx=0 tx=0).stop 2023-10-26T01:32:45.609 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.591+0000 7f1bfbfbb640 1 -- 172.21.15.169:0/3482562271 shutdown_connections 2023-10-26T01:32:45.609 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.591+0000 7f1bfbfbb640 1 --2- 172.21.15.169:0/3482562271 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f1bd806ae40 0x7f1bd806d300 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:45.609 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.591+0000 7f1bfbfbb640 1 --2- 172.21.15.169:0/3482562271 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1bf415b620 0x7f1bf415e710 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:45.609 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.591+0000 7f1bfbfbb640 1 --2- 172.21.15.169:0/3482562271 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1bf40c9d00 0x7f1bf415e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:45.610 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.591+0000 7f1bfbfbb640 1 --2- 172.21.15.169:0/3482562271 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1bf40ac320 0x7f1bf41418f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:45.610 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.591+0000 7f1bfbfbb640 1 -- 172.21.15.169:0/3482562271 >> 172.21.15.169:0/3482562271 conn(0x7f1bf4069930 msgr2=0x7f1bf406edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:45.610 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.591+0000 7f1bfbfbb640 1 -- 172.21.15.169:0/3482562271 shutdown_connections 2023-10-26T01:32:45.610 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.592+0000 7f1bfbfbb640 1 -- 172.21.15.169:0/3482562271 wait complete. 2023-10-26T01:32:45.611 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-26T01:32:45.820 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.818+0000 7f7448056640 1 -- 172.21.15.169:0/3944916626 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f74400ccda0 msgr2=0x7f744015ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:45.821 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.818+0000 7f7448056640 1 --2- 172.21.15.169:0/3944916626 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f74400ccda0 0x7f744015ca20 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f7430005fd0 tx=0x7f743002c240 comp rx=0 tx=0).stop 2023-10-26T01:32:45.821 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.819+0000 7f7448056640 1 -- 172.21.15.169:0/3944916626 shutdown_connections 2023-10-26T01:32:45.821 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.819+0000 7f7448056640 1 --2- 172.21.15.169:0/3944916626 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7440141cb0 0x7f74400b14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:45.821 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.819+0000 7f7448056640 1 --2- 172.21.15.169:0/3944916626 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f74400ccda0 0x7f744015ca20 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:45.822 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.819+0000 7f7448056640 1 -- 172.21.15.169:0/3944916626 >> 172.21.15.169:0/3944916626 conn(0x7f7440069930 msgr2=0x7f7440069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:45.822 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.819+0000 7f7448056640 1 -- 172.21.15.169:0/3944916626 shutdown_connections 2023-10-26T01:32:45.822 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.819+0000 7f7448056640 1 -- 172.21.15.169:0/3944916626 wait complete. 2023-10-26T01:32:45.824 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.820+0000 7f7448056640 1 Processor -- start 2023-10-26T01:32:45.824 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.821+0000 7f7448056640 1 -- start start 2023-10-26T01:32:45.824 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.822+0000 7f7448056640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f74400ccda0 0x7f74400aebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:45.825 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.822+0000 7f7448056640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f74400cd7e0 0x7f74401541e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:45.825 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.822+0000 7f7448056640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7440141cb0 0x7f7440154720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:45.825 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.822+0000 7f7448056640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f744006edd0 con 0x7f74400ccda0 2023-10-26T01:32:45.825 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.822+0000 7f7448056640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74400ae340 con 0x7f7440141cb0 2023-10-26T01:32:45.825 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.822+0000 7f7448056640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74400ae620 con 0x7f74400cd7e0 2023-10-26T01:32:45.826 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.822+0000 7f7445df2640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f74400ccda0 0x7f74400aebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:45.826 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.822+0000 7f7445df2640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f74400ccda0 0x7f74400aebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:33558/0 (socket says 172.21.15.169:33558) 2023-10-26T01:32:45.826 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.822+0000 7f7445df2640 1 -- 172.21.15.169:0/1953383019 learned_addr learned my addr 172.21.15.169:0/1953383019 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:32:45.826 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.822+0000 7f7445df2640 1 -- 172.21.15.169:0/1953383019 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f74400cd7e0 msgr2=0x7f74401541e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:45.826 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.822+0000 7f7445df2640 1 --2- 172.21.15.169:0/1953383019 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f74400cd7e0 0x7f74401541e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:45.827 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.822+0000 7f7445df2640 1 -- 172.21.15.169:0/1953383019 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7440141cb0 msgr2=0x7f7440154720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:45.827 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.822+0000 7f7445df2640 1 --2- 172.21.15.169:0/1953383019 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7440141cb0 0x7f7440154720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:45.827 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.822+0000 7f7445df2640 1 -- 172.21.15.169:0/1953383019 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7430009900 con 0x7f74400ccda0 2023-10-26T01:32:45.827 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.823+0000 7f7445df2640 1 --2- 172.21.15.169:0/1953383019 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f74400ccda0 0x7f74400aebf0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f7430006010 tx=0x7f7430035ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:45.829 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.823+0000 7f743e7fc640 1 -- 172.21.15.169:0/1953383019 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7430034440 con 0x7f74400ccda0 2023-10-26T01:32:45.829 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.823+0000 7f743e7fc640 1 -- 172.21.15.169:0/1953383019 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7430004330 con 0x7f74400ccda0 2023-10-26T01:32:45.829 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.823+0000 7f7448056640 1 -- 172.21.15.169:0/1953383019 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f744006f040 con 0x7f74400ccda0 2023-10-26T01:32:45.829 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.823+0000 7f743e7fc640 1 -- 172.21.15.169:0/1953383019 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7430004700 con 0x7f74400ccda0 2023-10-26T01:32:45.829 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.823+0000 7f7448056640 1 -- 172.21.15.169:0/1953383019 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74400af130 con 0x7f74400ccda0 2023-10-26T01:32:45.830 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.824+0000 7f7448056640 1 -- 172.21.15.169:0/1953383019 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f744004db10 con 0x7f74400ccda0 2023-10-26T01:32:45.830 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.825+0000 7f743e7fc640 1 -- 172.21.15.169:0/1953383019 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f743003b040 con 0x7f74400ccda0 2023-10-26T01:32:45.830 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.826+0000 7f743e7fc640 1 --2- 172.21.15.169:0/1953383019 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f742c06ae40 0x7f742c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:45.830 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.826+0000 7f743e7fc640 1 -- 172.21.15.169:0/1953383019 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f74300ac1a0 con 0x7f74400ccda0 2023-10-26T01:32:45.830 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.827+0000 7f74455f1640 1 --2- 172.21.15.169:0/1953383019 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f742c06ae40 0x7f742c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:45.831 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.828+0000 7f74455f1640 1 --2- 172.21.15.169:0/1953383019 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f742c06ae40 0x7f742c06d300 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f74340058c0 tx=0x7f7434005850 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:45.831 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:45.830+0000 7f743e7fc640 1 -- 172.21.15.169:0/1953383019 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f74300772e0 con 0x7f74400ccda0 2023-10-26T01:32:46.055 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:46.053+0000 7f7448056640 1 -- 172.21.15.169:0/1953383019 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f744006bad0 con 0x7f74400ccda0 2023-10-26T01:32:46.056 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:46.054+0000 7f743e7fc640 1 -- 172.21.15.169:0/1953383019 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v23) v1 ==== 53+0+73 (secure 0 0 0) 0x7f7430004860 con 0x7f74400ccda0 2023-10-26T01:32:46.056 INFO:teuthology.orchestra.run.smithi169.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-26T01:32:46.058 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:46.056+0000 7f7448056640 1 -- 172.21.15.169:0/1953383019 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f742c06ae40 msgr2=0x7f742c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:46.058 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:46.056+0000 7f7448056640 1 --2- 172.21.15.169:0/1953383019 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f742c06ae40 0x7f742c06d300 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f74340058c0 tx=0x7f7434005850 comp rx=0 tx=0).stop 2023-10-26T01:32:46.059 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:46.056+0000 7f7448056640 1 -- 172.21.15.169:0/1953383019 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f74400ccda0 msgr2=0x7f74400aebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:46.059 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:46.056+0000 7f7448056640 1 --2- 172.21.15.169:0/1953383019 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f74400ccda0 0x7f74400aebf0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f7430006010 tx=0x7f7430035ac0 comp rx=0 tx=0).stop 2023-10-26T01:32:46.059 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:46.057+0000 7f7448056640 1 -- 172.21.15.169:0/1953383019 shutdown_connections 2023-10-26T01:32:46.059 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:46.057+0000 7f7448056640 1 --2- 172.21.15.169:0/1953383019 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f742c06ae40 0x7f742c06d300 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:46.059 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:46.057+0000 7f7448056640 1 --2- 172.21.15.169:0/1953383019 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7440141cb0 0x7f7440154720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:46.060 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:46.057+0000 7f7448056640 1 --2- 172.21.15.169:0/1953383019 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f74400cd7e0 0x7f74401541e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:46.060 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:46.057+0000 7f7448056640 1 --2- 172.21.15.169:0/1953383019 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f74400ccda0 0x7f74400aebf0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:46.060 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:46.057+0000 7f7448056640 1 -- 172.21.15.169:0/1953383019 >> 172.21.15.169:0/1953383019 conn(0x7f7440069930 msgr2=0x7f7440157b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:46.060 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:46.057+0000 7f7448056640 1 -- 172.21.15.169:0/1953383019 shutdown_connections 2023-10-26T01:32:46.060 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:46.057+0000 7f7448056640 1 -- 172.21.15.169:0/1953383019 wait complete. 2023-10-26T01:32:46.075 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-26T01:32:46.075 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-26T01:32:46.076 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi169.front.sepia.ceph.com 2023-10-26T01:32:46.076 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-26T01:32:46.076 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-26T01:32:46.077 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-26T01:32:46.077 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-26T01:32:46.077 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-26T01:32:46.078 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-26T01:32:46.078 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:32:46.079 DEBUG:teuthology.orchestra.run.smithi169:> ip addr 2023-10-26T01:32:46.131 INFO:teuthology.orchestra.run.smithi169.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-26T01:32:46.132 INFO:teuthology.orchestra.run.smithi169.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-26T01:32:46.132 INFO:teuthology.orchestra.run.smithi169.stdout: inet 127.0.0.1/8 scope host lo 2023-10-26T01:32:46.132 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:32:46.133 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 ::1/128 scope host 2023-10-26T01:32:46.133 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:32:46.133 INFO:teuthology.orchestra.run.smithi169.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-26T01:32:46.134 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:d9:3b:b4 brd ff:ff:ff:ff:ff:ff 2023-10-26T01:32:46.134 INFO:teuthology.orchestra.run.smithi169.stdout: altname enp7s0f0 2023-10-26T01:32:46.134 INFO:teuthology.orchestra.run.smithi169.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-26T01:32:46.135 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:d9:3b:b5 brd ff:ff:ff:ff:ff:ff 2023-10-26T01:32:46.135 INFO:teuthology.orchestra.run.smithi169.stdout: altname enp7s0f1 2023-10-26T01:32:46.135 INFO:teuthology.orchestra.run.smithi169.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-26T01:32:46.135 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:88:7f:4c brd ff:ff:ff:ff:ff:ff 2023-10-26T01:32:46.136 INFO:teuthology.orchestra.run.smithi169.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-26T01:32:46.136 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:88:7f:4d brd ff:ff:ff:ff:ff:ff 2023-10-26T01:32:46.136 INFO:teuthology.orchestra.run.smithi169.stdout: inet 172.21.15.169/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-26T01:32:46.137 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft 42329sec preferred_lft 42329sec 2023-10-26T01:32:46.137 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 fe80::ec4:7aff:fe88:7f4d/64 scope link 2023-10-26T01:32:46.137 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:32:46.137 INFO:teuthology.orchestra.run.smithi169.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-26T01:32:46.138 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 22:a0:6b:43:7b:04 brd ff:ff:ff:ff:ff:ff 2023-10-26T01:32:46.138 INFO:teuthology.orchestra.run.smithi169.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-26T01:32:46.138 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:32:46.138 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 fe80::d815:31ff:fe87:d148/64 scope link 2023-10-26T01:32:46.139 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:32:46.139 INFO:teuthology.orchestra.run.smithi169.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-26T01:32:46.139 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 2a:80:f8:45:5f:9f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-26T01:32:46.140 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 fe80::2880:f8ff:fe45:5f9f/64 scope link 2023-10-26T01:32:46.140 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:32:46.140 INFO:teuthology.orchestra.run.smithi169.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-26T01:32:46.140 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 22:a0:6b:43:7b:04 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-26T01:32:46.141 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 fe80::20a0:6bff:fe43:7b04/64 scope link 2023-10-26T01:32:46.141 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:32:46.142 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:32:46.143 DEBUG:teuthology.orchestra.run.smithi169:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-26T01:32:46.235 INFO:teuthology.orchestra.run.smithi169.stdout:1 2023-10-26T01:32:46.239 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:32:46.239 DEBUG:teuthology.orchestra.run.smithi169:> route 2023-10-26T01:32:46.290 INFO:teuthology.orchestra.run.smithi169.stdout:Kernel IP routing table 2023-10-26T01:32:46.291 INFO:teuthology.orchestra.run.smithi169.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-26T01:32:46.291 INFO:teuthology.orchestra.run.smithi169.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-26T01:32:46.291 INFO:teuthology.orchestra.run.smithi169.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-26T01:32:46.291 INFO:teuthology.orchestra.run.smithi169.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-26T01:32:46.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:32:46.292 DEBUG:teuthology.orchestra.run.smithi169:> (cd / && exec bash -c ' 2023-10-26T01:32:46.293 DEBUG:teuthology.orchestra.run.smithi169:> set -e 2023-10-26T01:32:46.293 DEBUG:teuthology.orchestra.run.smithi169:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-26T01:32:46.294 DEBUG:teuthology.orchestra.run.smithi169:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-26T01:32:46.295 DEBUG:teuthology.orchestra.run.smithi169:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-26T01:32:46.295 DEBUG:teuthology.orchestra.run.smithi169:> ') 2023-10-26T01:32:46.479 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:32:46.479 DEBUG:teuthology.orchestra.run.smithi169:> ip netns list 2023-10-26T01:32:46.531 INFO:teuthology.orchestra.run.smithi169.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-26T01:32:46.531 INFO:teuthology.orchestra.run.smithi169.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-26T01:32:46.532 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-26T01:32:46.532 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:32:46.533 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link set brx.0 up 2023-10-26T01:32:46.630 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-26T01:32:46.631 DEBUG:teuthology.orchestra.run.smithi169:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-26T01:32:46.681 INFO:teuthology.orchestra.run.smithi169.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-26T01:32:46.681 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-26T01:32:46.682 DEBUG:teuthology.orchestra.run.smithi169:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-26T01:32:46.730 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-26T01:32:46.730 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:32:46.731 DEBUG:teuthology.orchestra.run.smithi169:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-26T01:32:46.868 INFO:teuthology.orchestra.run.smithi169.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-26T01:32:46.869 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-26T01:32:46.869 INFO:teuthology.orchestra.run.smithi169.stdout:invalid new device string format 2023-10-26T01:32:46.890 INFO:teuthology.orchestra.run.smithi169.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-26T01:32:46.891 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-26T01:32:46.891 INFO:teuthology.orchestra.run.smithi169.stdout:invalid new device string format 2023-10-26T01:32:46.891 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: resolved to: "172.21.15.169:3300,172.21.15.170:3300,172.21.15.170:3301" 2023-10-26T01:32:46.891 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-26T01:32:46.915 INFO:teuthology.orchestra.run.smithi169.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-26T01:32:46.916 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-26T01:32:46.916 INFO:teuthology.orchestra.run.smithi169.stdout:invalid new device string format 2023-10-26T01:32:46.916 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: resolved to: "172.21.15.169:3300,172.21.15.170:3300,172.21.15.170:3301" 2023-10-26T01:32:46.916 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-26T01:32:46.916 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: resolved to: "172.21.15.169:6789,172.21.15.170:6789,172.21.15.170:6790" 2023-10-26T01:32:46.917 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.169:6789,172.21.15.170:6789,172.21.15.170:6790:/ 2023-10-26T01:32:46.917 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=30203b28-ed8a-44e8-a9a6-a9bf25f980f1" will pass to kernel 2023-10-26T01:32:46.917 INFO:teuthology.orchestra.run.smithi169.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2023-10-26T01:32:46.917 INFO:teuthology.orchestra.run.smithi169.stdout: You just mounted a file system that supports labels which does not 2023-10-26T01:32:46.918 INFO:teuthology.orchestra.run.smithi169.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-10-26T01:32:46.918 INFO:teuthology.orchestra.run.smithi169.stdout: applications will generate AVC messages and not be allowed access to 2023-10-26T01:32:46.918 INFO:teuthology.orchestra.run.smithi169.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-10-26T01:32:46.922 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-26T01:32:46.922 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:32:46.923 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-26T01:32:47.015 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:32:47.015 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-26T01:32:47.079 INFO:teuthology.orchestra.run.smithi169.stdout:client5038 2023-10-26T01:32:47.079 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:32:47.080 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:32:47.137 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-26T01:32:47.355 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:47.353+0000 7fc7fd8a2640 1 -- 172.21.15.169:0/2863979637 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7e40047c0 msgr2=0x7fc7e4004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:47.356 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:47.353+0000 7fc7fd8a2640 1 --2- 172.21.15.169:0/2863979637 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7e40047c0 0x7fc7e4004c40 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fc7e0009b50 tx=0x7fc7e40057f0 comp rx=0 tx=0).stop 2023-10-26T01:32:47.356 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:47.353+0000 7fc7f77fe640 1 -- 172.21.15.169:0/2863979637 reap_dead start 2023-10-26T01:32:47.356 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:47.353+0000 7fc7fd8a2640 1 -- 172.21.15.169:0/2863979637 shutdown_connections 2023-10-26T01:32:47.356 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:47.353+0000 7fc7fd8a2640 1 -- 172.21.15.169:0/2863979637 >> 172.21.15.169:0/2863979637 conn(0x7fc7f8069930 msgr2=0x7fc7f8069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:47.356 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:47.354+0000 7fc7fd8a2640 1 -- 172.21.15.169:0/2863979637 shutdown_connections 2023-10-26T01:32:47.357 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:47.354+0000 7fc7fd8a2640 1 -- 172.21.15.169:0/2863979637 wait complete. 2023-10-26T01:32:47.357 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:47.355+0000 7fc7fd8a2640 1 Processor -- start 2023-10-26T01:32:47.357 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:47.355+0000 7fc7fd8a2640 1 -- start start 2023-10-26T01:32:47.359 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:47.356+0000 7fc7fd8a2640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc7e40047c0 0x7fc7f8099e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:47.359 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:47.356+0000 7fc7fd8a2640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc7f81f2e20 0x7fc7f8071220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:47.360 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:47.356+0000 7fc7fd8a2640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7f8071760 0x7fc7f81e21e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:47.360 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:47.356+0000 7fc7fd8a2640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f81e1dc0 con 0x7fc7f8071760 2023-10-26T01:32:47.360 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:47.356+0000 7fc7fd8a2640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f8071be0 con 0x7fc7f81f2e20 2023-10-26T01:32:47.360 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:47.356+0000 7fc7fd8a2640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f809a790 con 0x7fc7e40047c0 2023-10-26T01:32:47.360 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:47.357+0000 7fc7f77fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7f8071760 0x7fc7f81e21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:47.361 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:47.357+0000 7fc7f77fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7f8071760 0x7fc7f81e21e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:33586/0 (socket says 172.21.15.169:33586) 2023-10-26T01:32:47.361 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:47.357+0000 7fc7f77fe640 1 -- 172.21.15.169:0/1807681677 learned_addr learned my addr 172.21.15.169:0/1807681677 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:32:47.361 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:47.357+0000 7fc7f6ffd640 1 --2- 172.21.15.169:0/1807681677 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc7e40047c0 0x7fc7f8099e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:47.361 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:47.357+0000 7fc7f77fe640 1 -- 172.21.15.169:0/1807681677 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc7e40047c0 msgr2=0x7fc7f8099e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:47.361 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:47.357+0000 7fc7f77fe640 1 --2- 172.21.15.169:0/1807681677 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc7e40047c0 0x7fc7f8099e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:47.362 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:47.357+0000 7fc7f77fe640 1 -- 172.21.15.169:0/1807681677 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc7f81f2e20 msgr2=0x7fc7f8071220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:47.362 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:47.357+0000 7fc7f77fe640 1 --2- 172.21.15.169:0/1807681677 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc7f81f2e20 0x7fc7f8071220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:47.362 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:47.357+0000 7fc7f77fe640 1 -- 172.21.15.169:0/1807681677 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7f809aa70 con 0x7fc7f8071760 2023-10-26T01:32:47.362 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:47.357+0000 7fc7f77fe640 1 --2- 172.21.15.169:0/1807681677 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7f8071760 0x7fc7f81e21e0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fc7f806baa0 tx=0x7fc7f000ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:47.362 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:47.357+0000 7fc7f4ff9640 1 -- 172.21.15.169:0/1807681677 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7f0004d20 con 0x7fc7f8071760 2023-10-26T01:32:47.363 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:47.357+0000 7fc7f4ff9640 1 -- 172.21.15.169:0/1807681677 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc7f000ca10 con 0x7fc7f8071760 2023-10-26T01:32:47.363 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:47.357+0000 7fc7f4ff9640 1 -- 172.21.15.169:0/1807681677 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7f000ce00 con 0x7fc7f8071760 2023-10-26T01:32:47.363 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:47.358+0000 7fc7fd8a2640 1 -- 172.21.15.169:0/1807681677 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7f809ad50 con 0x7fc7f8071760 2023-10-26T01:32:47.363 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:47.358+0000 7fc7fd8a2640 1 -- 172.21.15.169:0/1807681677 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7f809a340 con 0x7fc7f8071760 2023-10-26T01:32:47.364 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:47.359+0000 7fc7fd8a2640 1 -- 172.21.15.169:0/1807681677 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7f80733c0 con 0x7fc7f8071760 2023-10-26T01:32:47.364 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:47.360+0000 7fc7f4ff9640 1 -- 172.21.15.169:0/1807681677 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fc7f0010430 con 0x7fc7f8071760 2023-10-26T01:32:47.365 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:47.361+0000 7fc7f4ff9640 1 --2- 172.21.15.169:0/1807681677 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc7d806ae70 0x7fc7d806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:47.365 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:47.361+0000 7fc7f4ff9640 1 -- 172.21.15.169:0/1807681677 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc7f0014070 con 0x7fc7f8071760 2023-10-26T01:32:47.365 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:47.361+0000 7fc7f6ffd640 1 --2- 172.21.15.169:0/1807681677 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc7d806ae70 0x7fc7d806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:47.365 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:47.362+0000 7fc7f6ffd640 1 --2- 172.21.15.169:0/1807681677 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc7d806ae70 0x7fc7d806d330 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc7e80017a0 tx=0x7fc7e8001190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:47.366 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:47.365+0000 7fc7f4ff9640 1 -- 172.21.15.169:0/1807681677 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc7f0054830 con 0x7fc7f8071760 2023-10-26T01:32:47.593 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:47.591+0000 7fc7fd8a2640 1 -- 172.21.15.169:0/1807681677 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fc7f8141010 con 0x7fc7f8071760 2023-10-26T01:32:47.594 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:47.592+0000 7fc7f4ff9640 1 -- 172.21.15.169:0/1807681677 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v23) v1 ==== 53+0+73 (secure 0 0 0) 0x7fc7f001d060 con 0x7fc7f8071760 2023-10-26T01:32:47.594 INFO:teuthology.orchestra.run.smithi169.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-26T01:32:47.597 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:47.595+0000 7fc7fd8a2640 1 -- 172.21.15.169:0/1807681677 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc7d806ae70 msgr2=0x7fc7d806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:47.597 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:47.595+0000 7fc7fd8a2640 1 --2- 172.21.15.169:0/1807681677 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc7d806ae70 0x7fc7d806d330 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc7e80017a0 tx=0x7fc7e8001190 comp rx=0 tx=0).stop 2023-10-26T01:32:47.597 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:47.595+0000 7fc7fd8a2640 1 -- 172.21.15.169:0/1807681677 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7f8071760 msgr2=0x7fc7f81e21e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:47.598 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:47.595+0000 7fc7fd8a2640 1 --2- 172.21.15.169:0/1807681677 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7f8071760 0x7fc7f81e21e0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fc7f806baa0 tx=0x7fc7f000ed80 comp rx=0 tx=0).stop 2023-10-26T01:32:47.598 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:47.595+0000 7fc7fd8a2640 1 -- 172.21.15.169:0/1807681677 shutdown_connections 2023-10-26T01:32:47.598 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:47.595+0000 7fc7fd8a2640 1 --2- 172.21.15.169:0/1807681677 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc7d806ae70 0x7fc7d806d330 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:47.598 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:47.595+0000 7fc7fd8a2640 1 --2- 172.21.15.169:0/1807681677 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7f8071760 0x7fc7f81e21e0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:47.599 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:47.595+0000 7fc7fd8a2640 1 --2- 172.21.15.169:0/1807681677 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc7f81f2e20 0x7fc7f8071220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:47.599 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:47.595+0000 7fc7fd8a2640 1 --2- 172.21.15.169:0/1807681677 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc7e40047c0 0x7fc7f8099e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:47.599 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:47.595+0000 7fc7fd8a2640 1 -- 172.21.15.169:0/1807681677 >> 172.21.15.169:0/1807681677 conn(0x7fc7f8069930 msgr2=0x7fc7f815ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:47.599 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:47.595+0000 7fc7fd8a2640 1 -- 172.21.15.169:0/1807681677 shutdown_connections 2023-10-26T01:32:47.599 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:47.596+0000 7fc7fd8a2640 1 -- 172.21.15.169:0/1807681677 wait complete. 2023-10-26T01:32:47.615 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-26T01:32:47.616 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-26T01:32:47.616 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi170.front.sepia.ceph.com 2023-10-26T01:32:47.617 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-26T01:32:47.617 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-26T01:32:47.617 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-26T01:32:47.618 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-26T01:32:47.618 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-26T01:32:47.618 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-26T01:32:47.619 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:32:47.619 DEBUG:teuthology.orchestra.run.smithi170:> ip addr 2023-10-26T01:32:47.681 INFO:teuthology.orchestra.run.smithi170.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-26T01:32:47.682 INFO:teuthology.orchestra.run.smithi170.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-26T01:32:47.682 INFO:teuthology.orchestra.run.smithi170.stdout: inet 127.0.0.1/8 scope host lo 2023-10-26T01:32:47.682 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:32:47.682 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 ::1/128 scope host 2023-10-26T01:32:47.683 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:32:47.683 INFO:teuthology.orchestra.run.smithi170.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-26T01:32:47.683 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:d9:3b:54 brd ff:ff:ff:ff:ff:ff 2023-10-26T01:32:47.683 INFO:teuthology.orchestra.run.smithi170.stdout: altname enp7s0f0 2023-10-26T01:32:47.683 INFO:teuthology.orchestra.run.smithi170.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-26T01:32:47.684 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:d9:3b:55 brd ff:ff:ff:ff:ff:ff 2023-10-26T01:32:47.684 INFO:teuthology.orchestra.run.smithi170.stdout: altname enp7s0f1 2023-10-26T01:32:47.684 INFO:teuthology.orchestra.run.smithi170.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-26T01:32:47.684 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:88:72:54 brd ff:ff:ff:ff:ff:ff 2023-10-26T01:32:47.684 INFO:teuthology.orchestra.run.smithi170.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-26T01:32:47.685 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:88:72:55 brd ff:ff:ff:ff:ff:ff 2023-10-26T01:32:47.685 INFO:teuthology.orchestra.run.smithi170.stdout: inet 172.21.15.170/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-26T01:32:47.685 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft 42312sec preferred_lft 42312sec 2023-10-26T01:32:47.685 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::ec4:7aff:fe88:7255/64 scope link 2023-10-26T01:32:47.685 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:32:47.686 INFO:teuthology.orchestra.run.smithi170.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-26T01:32:47.686 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 32:b5:d1:ee:10:36 brd ff:ff:ff:ff:ff:ff 2023-10-26T01:32:47.686 INFO:teuthology.orchestra.run.smithi170.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-26T01:32:47.686 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:32:47.686 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::28ed:d1ff:fe37:70d/64 scope link 2023-10-26T01:32:47.687 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:32:47.687 INFO:teuthology.orchestra.run.smithi170.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-26T01:32:47.687 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 32:b5:d1:ee:10:36 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-26T01:32:47.688 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::30b5:d1ff:feee:1036/64 scope link 2023-10-26T01:32:47.688 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:32:47.688 INFO:teuthology.orchestra.run.smithi170.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-26T01:32:47.689 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether b2:c1:53:19:69:e4 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-26T01:32:47.689 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::b0c1:53ff:fe19:69e4/64 scope link 2023-10-26T01:32:47.689 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:32:47.690 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:32:47.691 DEBUG:teuthology.orchestra.run.smithi170:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-26T01:32:47.791 INFO:teuthology.orchestra.run.smithi170.stdout:1 2023-10-26T01:32:47.795 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:32:47.795 DEBUG:teuthology.orchestra.run.smithi170:> route 2023-10-26T01:32:47.850 INFO:teuthology.orchestra.run.smithi170.stdout:Kernel IP routing table 2023-10-26T01:32:47.850 INFO:teuthology.orchestra.run.smithi170.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-26T01:32:47.850 INFO:teuthology.orchestra.run.smithi170.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-26T01:32:47.850 INFO:teuthology.orchestra.run.smithi170.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-26T01:32:47.851 INFO:teuthology.orchestra.run.smithi170.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-26T01:32:47.851 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:32:47.852 DEBUG:teuthology.orchestra.run.smithi170:> (cd / && exec bash -c ' 2023-10-26T01:32:47.852 DEBUG:teuthology.orchestra.run.smithi170:> set -e 2023-10-26T01:32:47.852 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-26T01:32:47.853 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-26T01:32:47.853 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-26T01:32:47.853 DEBUG:teuthology.orchestra.run.smithi170:> ') 2023-10-26T01:32:48.037 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:32:48.037 DEBUG:teuthology.orchestra.run.smithi170:> ip netns list 2023-10-26T01:32:48.089 INFO:teuthology.orchestra.run.smithi170.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-26T01:32:48.089 INFO:teuthology.orchestra.run.smithi170.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-26T01:32:48.090 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-26T01:32:48.090 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:32:48.090 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip link set brx.0 up 2023-10-26T01:32:48.182 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-26T01:32:48.183 DEBUG:teuthology.orchestra.run.smithi170:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-26T01:32:48.236 INFO:teuthology.orchestra.run.smithi170.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-26T01:32:48.236 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-26T01:32:48.237 DEBUG:teuthology.orchestra.run.smithi170:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-26T01:32:48.289 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-26T01:32:48.289 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:32:48.290 DEBUG:teuthology.orchestra.run.smithi170:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-26T01:32:48.431 INFO:teuthology.orchestra.run.smithi170.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-26T01:32:48.432 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-26T01:32:48.432 INFO:teuthology.orchestra.run.smithi170.stdout:invalid new device string format 2023-10-26T01:32:48.454 INFO:teuthology.orchestra.run.smithi170.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-26T01:32:48.454 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-26T01:32:48.454 INFO:teuthology.orchestra.run.smithi170.stdout:invalid new device string format 2023-10-26T01:32:48.455 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: resolved to: "172.21.15.169:3300,172.21.15.170:3300,172.21.15.170:3301" 2023-10-26T01:32:48.455 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-26T01:32:48.480 INFO:teuthology.orchestra.run.smithi170.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-26T01:32:48.480 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-26T01:32:48.480 INFO:teuthology.orchestra.run.smithi170.stdout:invalid new device string format 2023-10-26T01:32:48.480 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: resolved to: "172.21.15.169:3300,172.21.15.170:3300,172.21.15.170:3301" 2023-10-26T01:32:48.481 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-26T01:32:48.481 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: resolved to: "172.21.15.169:6789,172.21.15.170:6789,172.21.15.170:6790" 2023-10-26T01:32:48.481 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.169:6789,172.21.15.170:6789,172.21.15.170:6790:/ 2023-10-26T01:32:48.481 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=30203b28-ed8a-44e8-a9a6-a9bf25f980f1" will pass to kernel 2023-10-26T01:32:48.483 INFO:teuthology.orchestra.run.smithi170.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2023-10-26T01:32:48.483 INFO:teuthology.orchestra.run.smithi170.stdout: You just mounted a file system that supports labels which does not 2023-10-26T01:32:48.483 INFO:teuthology.orchestra.run.smithi170.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-10-26T01:32:48.484 INFO:teuthology.orchestra.run.smithi170.stdout: applications will generate AVC messages and not be allowed access to 2023-10-26T01:32:48.484 INFO:teuthology.orchestra.run.smithi170.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-10-26T01:32:48.488 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-26T01:32:48.488 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:32:48.489 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-26T01:32:48.579 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:32:48.579 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-26T01:32:48.639 INFO:teuthology.orchestra.run.smithi170.stdout:client5044 2023-10-26T01:32:48.640 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-26T01:32:48.640 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:32:48.698 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T01:32:48.926 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:48.924+0000 7f8f8ffff640 1 -- 172.21.15.169:0/4265100710 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f900b45d0 msgr2=0x7f8f900cbce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:48.927 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:48.924+0000 7f8f8ffff640 1 --2- 172.21.15.169:0/4265100710 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f900b45d0 0x7f8f900cbce0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f8f78005fd0 tx=0x7f8f7802be20 comp rx=0 tx=0).stop 2023-10-26T01:32:48.927 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:48.924+0000 7f8f8ffff640 1 -- 172.21.15.169:0/4265100710 shutdown_connections 2023-10-26T01:32:48.927 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:48.924+0000 7f8f8ffff640 1 --2- 172.21.15.169:0/4265100710 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f900b45d0 0x7f8f900cbce0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:48.927 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:48.924+0000 7f8f8ffff640 1 --2- 172.21.15.169:0/4265100710 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f8f900b68c0 0x7f8f90065ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:48.927 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:48.924+0000 7f8f8ffff640 1 -- 172.21.15.169:0/4265100710 >> 172.21.15.169:0/4265100710 conn(0x7f8f900d1360 msgr2=0x7f8f900d1770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:48.928 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:48.925+0000 7f8f8ffff640 1 -- 172.21.15.169:0/4265100710 shutdown_connections 2023-10-26T01:32:48.928 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:48.925+0000 7f8f8ffff640 1 -- 172.21.15.169:0/4265100710 wait complete. 2023-10-26T01:32:48.929 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:48.926+0000 7f8f8ffff640 1 Processor -- start 2023-10-26T01:32:48.929 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:48.926+0000 7f8f8ffff640 1 -- start start 2023-10-26T01:32:48.929 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:48.927+0000 7f8f8ffff640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f8f900b45d0 0x7f8f9014ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:48.930 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:48.927+0000 7f8f8ffff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8f900b4a30 0x7f8f90148c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:48.930 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:48.927+0000 7f8f8ffff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f900b68c0 0x7f8f90149140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:48.930 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:48.927+0000 7f8f8ffff640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f901487e0 con 0x7f8f900b68c0 2023-10-26T01:32:48.930 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:48.927+0000 7f8f8ffff640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f90147b60 con 0x7f8f900b4a30 2023-10-26T01:32:48.931 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:48.927+0000 7f8f8ffff640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f90147e40 con 0x7f8f900b45d0 2023-10-26T01:32:48.931 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:48.928+0000 7f8f8f7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f900b68c0 0x7f8f90149140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:48.931 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:48.928+0000 7f8f8f7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f900b68c0 0x7f8f90149140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:33606/0 (socket says 172.21.15.169:33606) 2023-10-26T01:32:48.931 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:48.928+0000 7f8f8f7fe640 1 -- 172.21.15.169:0/180667427 learned_addr learned my addr 172.21.15.169:0/180667427 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:32:48.933 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:48.928+0000 7f8f8effd640 1 --2- 172.21.15.169:0/180667427 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f8f900b45d0 0x7f8f9014ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:48.933 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:48.928+0000 7f8f867fc640 1 --2- 172.21.15.169:0/180667427 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8f900b4a30 0x7f8f90148c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:48.933 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:48.928+0000 7f8f8f7fe640 1 -- 172.21.15.169:0/180667427 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f8f900b45d0 msgr2=0x7f8f9014ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:48.933 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:48.928+0000 7f8f8f7fe640 1 --2- 172.21.15.169:0/180667427 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f8f900b45d0 0x7f8f9014ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:48.933 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:48.928+0000 7f8f8f7fe640 1 -- 172.21.15.169:0/180667427 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8f900b4a30 msgr2=0x7f8f90148c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:48.934 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:48.928+0000 7f8f8f7fe640 1 --2- 172.21.15.169:0/180667427 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8f900b4a30 0x7f8f90148c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:48.934 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:48.928+0000 7f8f8f7fe640 1 -- 172.21.15.169:0/180667427 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f78009560 con 0x7f8f900b68c0 2023-10-26T01:32:48.934 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:48.928+0000 7f8f8effd640 1 --2- 172.21.15.169:0/180667427 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f8f900b45d0 0x7f8f9014ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:32:48.934 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:48.928+0000 7f8f8f7fe640 1 --2- 172.21.15.169:0/180667427 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f900b68c0 0x7f8f90149140 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f8f8800c730 tx=0x7f8f8800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:48.935 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:48.929+0000 7f8f8cff9640 1 -- 172.21.15.169:0/180667427 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f88003b70 con 0x7f8f900b68c0 2023-10-26T01:32:48.935 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:48.929+0000 7f8f8cff9640 1 -- 172.21.15.169:0/180667427 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8f88003cd0 con 0x7f8f900b68c0 2023-10-26T01:32:48.935 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:48.929+0000 7f8f8cff9640 1 -- 172.21.15.169:0/180667427 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f8800aab0 con 0x7f8f900b68c0 2023-10-26T01:32:48.935 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:48.929+0000 7f8f8ffff640 1 -- 172.21.15.169:0/180667427 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f90148120 con 0x7f8f900b68c0 2023-10-26T01:32:48.935 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:48.929+0000 7f8f8ffff640 1 -- 172.21.15.169:0/180667427 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f90147780 con 0x7f8f900b68c0 2023-10-26T01:32:48.936 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:48.930+0000 7f8f8ffff640 1 -- 172.21.15.169:0/180667427 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f90148920 con 0x7f8f900b68c0 2023-10-26T01:32:48.936 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:48.931+0000 7f8f8cff9640 1 -- 172.21.15.169:0/180667427 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f8f8800ac10 con 0x7f8f900b68c0 2023-10-26T01:32:48.937 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:48.932+0000 7f8f8cff9640 1 --2- 172.21.15.169:0/180667427 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f8f7006ae40 0x7f8f7006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:48.937 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:48.932+0000 7f8f8cff9640 1 -- 172.21.15.169:0/180667427 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8f880895d0 con 0x7f8f900b68c0 2023-10-26T01:32:48.937 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:48.932+0000 7f8f8effd640 1 --2- 172.21.15.169:0/180667427 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f8f7006ae40 0x7f8f7006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:48.937 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:48.934+0000 7f8f8effd640 1 --2- 172.21.15.169:0/180667427 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f8f7006ae40 0x7f8f7006d300 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f8f80009aa0 tx=0x7f8f80005db0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:48.938 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:48.937+0000 7f8f8cff9640 1 -- 172.21.15.169:0/180667427 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8f90148920 con 0x7f8f900b68c0 2023-10-26T01:32:49.169 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:49.168+0000 7f8f8ffff640 1 -- 172.21.15.169:0/180667427 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8f90148920 con 0x7f8f900b68c0 2023-10-26T01:32:49.170 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:49.169+0000 7f8f8cff9640 1 -- 172.21.15.169:0/180667427 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4925 (secure 0 0 0) 0x7f8f90148920 con 0x7f8f900b68c0 2023-10-26T01:32:49.171 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:32:49.171 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4805,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/2294107634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2294107634},{"type":"v1","addr":"172.21.15.170:6836","nonce":2294107634}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":4987,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2208026662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2208026662},{"type":"v1","addr":"172.21.15.169:6833","nonce":2208026662}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":4993,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/2630365058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":2630365058},{"type":"v1","addr":"172.21.15.170:6837","nonce":2630365058}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T01:32:38.059721+0000","modified":"2023-10-26T01:32:41.428183+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5664},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5664":{"gid":5664,"name":"a","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6835/2816266753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":2816266753},{"type":"v1","addr":"172.21.15.169:6835","nonce":2816266753}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-10-26T01:32:49.171 INFO:teuthology.orchestra.run.smithi169.stderr:dumped fsmap epoch 23 2023-10-26T01:32:49.173 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:49.172+0000 7f8f8ffff640 1 -- 172.21.15.169:0/180667427 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f8f7006ae40 msgr2=0x7f8f7006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:49.174 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:49.172+0000 7f8f8ffff640 1 --2- 172.21.15.169:0/180667427 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f8f7006ae40 0x7f8f7006d300 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f8f80009aa0 tx=0x7f8f80005db0 comp rx=0 tx=0).stop 2023-10-26T01:32:49.174 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:49.172+0000 7f8f8ffff640 1 -- 172.21.15.169:0/180667427 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f900b68c0 msgr2=0x7f8f90149140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:49.174 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:49.172+0000 7f8f8ffff640 1 --2- 172.21.15.169:0/180667427 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f900b68c0 0x7f8f90149140 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f8f8800c730 tx=0x7f8f8800cbf0 comp rx=0 tx=0).stop 2023-10-26T01:32:49.174 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:49.172+0000 7f8f8ffff640 1 -- 172.21.15.169:0/180667427 shutdown_connections 2023-10-26T01:32:49.175 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:49.172+0000 7f8f8ffff640 1 --2- 172.21.15.169:0/180667427 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f8f7006ae40 0x7f8f7006d300 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:49.175 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:49.172+0000 7f8f8ffff640 1 --2- 172.21.15.169:0/180667427 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f900b68c0 0x7f8f90149140 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:49.175 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:49.172+0000 7f8f8ffff640 1 --2- 172.21.15.169:0/180667427 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8f900b4a30 0x7f8f90148c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:49.175 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:49.172+0000 7f8f8ffff640 1 --2- 172.21.15.169:0/180667427 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f8f900b45d0 0x7f8f9014ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:49.175 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:49.172+0000 7f8f8ffff640 1 -- 172.21.15.169:0/180667427 >> 172.21.15.169:0/180667427 conn(0x7f8f900d1360 msgr2=0x7f8f900b3640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:49.176 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:49.172+0000 7f8f8ffff640 1 -- 172.21.15.169:0/180667427 shutdown_connections 2023-10-26T01:32:49.176 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:49.172+0000 7f8f8ffff640 1 -- 172.21.15.169:0/180667427 wait complete. 2023-10-26T01:32:49.192 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2023-10-26T01:32:49.428 INFO:teuthology.orchestra.run.smithi169.stdout:{"cluster_fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","whoami":0,"id":5664,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":10.494017068,"mdsmap_epoch":23,"osdmap_epoch":49,"osdmap_epoch_barrier":49,"uptime":16.676361356000001} 2023-10-26T01:32:49.429 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "30203b28-ed8a-44e8-a9a6-a9bf25f980f1", "fs_name": "cephfs", "id": 5664, "mdsmap_epoch": 23, "osdmap_epoch": 49, "osdmap_epoch_barrier": 49, "rank_uptime": 10.494017068, "state": "up:active", "uptime": 16.676361356, "want_state": "up:active", "whoami": 0 } 2023-10-26T01:32:49.429 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2023-10-26T01:32:49.660 INFO:teuthology.orchestra.run.smithi169.stdout:{"osd_mon_report_interval":"5"} 2023-10-26T01:32:49.661 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2023-10-26T01:32:49.662 INFO:tasks.cephfs.test_full:Writing 531MB should fill this cluster 2023-10-26T01:32:49.663 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:32:49.663 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:32:49.718 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:32:49.719 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_a bs=1M conv=fdatasync count=265 seek=0) 2023-10-26T01:32:51.729 INFO:teuthology.orchestra.run.smithi169.stderr:265+0 records in 2023-10-26T01:32:51.729 INFO:teuthology.orchestra.run.smithi169.stderr:265+0 records out 2023-10-26T01:32:51.729 INFO:teuthology.orchestra.run.smithi169.stderr:277872640 bytes (278 MB, 265 MiB) copied, 1.9538 s, 142 MB/s 2023-10-26T01:32:51.731 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:32:51.731 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:32:51.770 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:32:51.770 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=292 seek=0) 2023-10-26T01:32:53.953 INFO:teuthology.orchestra.run.smithi169.stderr:292+0 records in 2023-10-26T01:32:53.953 INFO:teuthology.orchestra.run.smithi169.stderr:292+0 records out 2023-10-26T01:32:53.953 INFO:teuthology.orchestra.run.smithi169.stderr:306184192 bytes (306 MB, 292 MiB) copied, 2.09228 s, 146 MB/s 2023-10-26T01:32:53.955 INFO:tasks.cephfs.test_full:Writing file B succeeded (full status will happen soon) 2023-10-26T01:32:53.955 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-26T01:32:54.149 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:54.147+0000 7fd423fff640 1 -- 172.21.15.169:0/880038489 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd42409a5c0 msgr2=0x7fd42409aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:54.149 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:54.147+0000 7fd423fff640 1 --2- 172.21.15.169:0/880038489 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd42409a5c0 0x7fd42409aa70 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd41c005fd0 tx=0x7fd41c02c0f0 comp rx=0 tx=0).stop 2023-10-26T01:32:54.149 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:54.147+0000 7fd423fff640 1 -- 172.21.15.169:0/880038489 shutdown_connections 2023-10-26T01:32:54.149 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:54.147+0000 7fd423fff640 1 --2- 172.21.15.169:0/880038489 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd42409a5c0 0x7fd42409aa70 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:54.150 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:54.147+0000 7fd423fff640 1 --2- 172.21.15.169:0/880038489 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd4240ac6d0 0x7fd42415d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:54.150 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:54.147+0000 7fd423fff640 1 -- 172.21.15.169:0/880038489 >> 172.21.15.169:0/880038489 conn(0x7fd4240aedd0 msgr2=0x7fd4240af1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:54.150 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:54.147+0000 7fd423fff640 1 -- 172.21.15.169:0/880038489 shutdown_connections 2023-10-26T01:32:54.150 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:54.147+0000 7fd423fff640 1 -- 172.21.15.169:0/880038489 wait complete. 2023-10-26T01:32:54.152 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:54.149+0000 7fd423fff640 1 Processor -- start 2023-10-26T01:32:54.152 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:54.149+0000 7fd423fff640 1 -- start start 2023-10-26T01:32:54.152 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:54.150+0000 7fd423fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd42409a5c0 0x7fd4241f5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:54.153 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:54.150+0000 7fd423fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd4240ac6d0 0x7fd42415a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:54.153 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:54.150+0000 7fd423fff640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd4240c9970 0x7fd42413f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:54.153 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:54.150+0000 7fd423fff640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd42406f5a0 con 0x7fd42409a5c0 2023-10-26T01:32:54.153 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:54.150+0000 7fd423fff640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4241f5f50 con 0x7fd4240ac6d0 2023-10-26T01:32:54.153 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:54.150+0000 7fd423fff640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd42406e990 con 0x7fd4240c9970 2023-10-26T01:32:54.154 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:54.150+0000 7fd422ffd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd42409a5c0 0x7fd4241f5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:54.154 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:54.150+0000 7fd422ffd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd42409a5c0 0x7fd4241f5a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:58106/0 (socket says 172.21.15.169:58106) 2023-10-26T01:32:54.154 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:54.150+0000 7fd422ffd640 1 -- 172.21.15.169:0/3202420550 learned_addr learned my addr 172.21.15.169:0/3202420550 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:32:54.154 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:54.150+0000 7fd422ffd640 1 -- 172.21.15.169:0/3202420550 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd4240c9970 msgr2=0x7fd42413f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:54.155 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:54.150+0000 7fd422ffd640 1 --2- 172.21.15.169:0/3202420550 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd4240c9970 0x7fd42413f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:54.155 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:54.150+0000 7fd422ffd640 1 -- 172.21.15.169:0/3202420550 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd4240ac6d0 msgr2=0x7fd42415a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:54.155 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:54.150+0000 7fd422ffd640 1 --2- 172.21.15.169:0/3202420550 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd4240ac6d0 0x7fd42415a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:54.155 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:54.150+0000 7fd422ffd640 1 -- 172.21.15.169:0/3202420550 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd414009490 con 0x7fd42409a5c0 2023-10-26T01:32:54.155 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:54.151+0000 7fd422ffd640 1 --2- 172.21.15.169:0/3202420550 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd42409a5c0 0x7fd4241f5a10 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fd41400ea60 tx=0x7fd41400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:54.156 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:54.151+0000 7fd40b7fe640 1 -- 172.21.15.169:0/3202420550 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd41400cde0 con 0x7fd42409a5c0 2023-10-26T01:32:54.157 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:54.151+0000 7fd40b7fe640 1 -- 172.21.15.169:0/3202420550 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd414004020 con 0x7fd42409a5c0 2023-10-26T01:32:54.157 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:54.151+0000 7fd40b7fe640 1 -- 172.21.15.169:0/3202420550 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd414004410 con 0x7fd42409a5c0 2023-10-26T01:32:54.157 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:54.151+0000 7fd423fff640 1 -- 172.21.15.169:0/3202420550 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd41c009840 con 0x7fd42409a5c0 2023-10-26T01:32:54.157 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:54.151+0000 7fd423fff640 1 -- 172.21.15.169:0/3202420550 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd42406ef50 con 0x7fd42409a5c0 2023-10-26T01:32:54.158 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:54.152+0000 7fd423fff640 1 -- 172.21.15.169:0/3202420550 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd424073c10 con 0x7fd42409a5c0 2023-10-26T01:32:54.158 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:54.153+0000 7fd40b7fe640 1 -- 172.21.15.169:0/3202420550 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fd4140035f0 con 0x7fd42409a5c0 2023-10-26T01:32:54.158 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:54.154+0000 7fd40b7fe640 1 --2- 172.21.15.169:0/3202420550 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd41006ae40 0x7fd41006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:54.158 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:54.154+0000 7fd40b7fe640 1 -- 172.21.15.169:0/3202420550 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd414014070 con 0x7fd42409a5c0 2023-10-26T01:32:54.158 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:54.155+0000 7fd4227fc640 1 --2- 172.21.15.169:0/3202420550 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd41006ae40 0x7fd41006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:54.159 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:54.156+0000 7fd4227fc640 1 --2- 172.21.15.169:0/3202420550 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd41006ae40 0x7fd41006d300 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd424067970 tx=0x7fd418001190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:54.160 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:54.159+0000 7fd40b7fe640 1 -- 172.21.15.169:0/3202420550 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd4140552f0 con 0x7fd42409a5c0 2023-10-26T01:32:54.347 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:54.345+0000 7fd423fff640 1 -- 172.21.15.169:0/3202420550 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fd4240cc6e0 con 0x7fd42409a5c0 2023-10-26T01:32:54.349 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:54.348+0000 7fd40b7fe640 1 -- 172.21.15.169:0/3202420550 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v49) v1 ==== 81+0+33996 (secure 0 0 0) 0x7fd414058910 con 0x7fd42409a5c0 2023-10-26T01:32:54.350 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:32:54.351 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2023-10-26T01:32:54.351 INFO:teuthology.orchestra.run.smithi169.stdout: "epoch": 49, 2023-10-26T01:32:54.351 INFO:teuthology.orchestra.run.smithi169.stdout: "fsid": "30203b28-ed8a-44e8-a9a6-a9bf25f980f1", 2023-10-26T01:32:54.351 INFO:teuthology.orchestra.run.smithi169.stdout: "created": "2023-10-26T01:30:22.897155+0000", 2023-10-26T01:32:54.351 INFO:teuthology.orchestra.run.smithi169.stdout: "modified": "2023-10-26T01:32:39.585979+0000", 2023-10-26T01:32:54.352 INFO:teuthology.orchestra.run.smithi169.stdout: "last_up_change": "2023-10-26T01:30:31.875281+0000", 2023-10-26T01:32:54.352 INFO:teuthology.orchestra.run.smithi169.stdout: "last_in_change": "2023-10-26T01:30:27.637941+0000", 2023-10-26T01:32:54.352 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-26T01:32:54.352 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_num": 5799936, 2023-10-26T01:32:54.353 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_set": [ 2023-10-26T01:32:54.353 INFO:teuthology.orchestra.run.smithi169.stdout: "pglog_hardlimit", 2023-10-26T01:32:54.353 INFO:teuthology.orchestra.run.smithi169.stdout: "purged_snapdirs", 2023-10-26T01:32:54.353 INFO:teuthology.orchestra.run.smithi169.stdout: "recovery_deletes", 2023-10-26T01:32:54.353 INFO:teuthology.orchestra.run.smithi169.stdout: "sortbitwise" 2023-10-26T01:32:54.354 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:32:54.354 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_version": 4, 2023-10-26T01:32:54.354 INFO:teuthology.orchestra.run.smithi169.stdout: "full_ratio": 0.69999998807907104, 2023-10-26T01:32:54.354 INFO:teuthology.orchestra.run.smithi169.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-26T01:32:54.355 INFO:teuthology.orchestra.run.smithi169.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-26T01:32:54.355 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_snapshot": "", 2023-10-26T01:32:54.355 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_max": 7, 2023-10-26T01:32:54.355 INFO:teuthology.orchestra.run.smithi169.stdout: "max_osd": 8, 2023-10-26T01:32:54.355 INFO:teuthology.orchestra.run.smithi169.stdout: "require_min_compat_client": "luminous", 2023-10-26T01:32:54.355 INFO:teuthology.orchestra.run.smithi169.stdout: "min_compat_client": "jewel", 2023-10-26T01:32:54.358 INFO:teuthology.orchestra.run.smithi169.stdout: "require_osd_release": "reef", 2023-10-26T01:32:54.358 INFO:teuthology.orchestra.run.smithi169.stdout: "allow_crimson": false, 2023-10-26T01:32:54.358 INFO:teuthology.orchestra.run.smithi169.stdout: "pools": [ 2023-10-26T01:32:54.358 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:32:54.359 INFO:teuthology.orchestra.run.smithi169.stdout: "pool": 1, 2023-10-26T01:32:54.359 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_name": ".mgr", 2023-10-26T01:32:54.359 INFO:teuthology.orchestra.run.smithi169.stdout: "create_time": "2023-10-26T01:30:32.200808+0000", 2023-10-26T01:32:54.359 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": 1, 2023-10-26T01:32:54.359 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_names": "hashpspool", 2023-10-26T01:32:54.360 INFO:teuthology.orchestra.run.smithi169.stdout: "type": 1, 2023-10-26T01:32:54.360 INFO:teuthology.orchestra.run.smithi169.stdout: "size": 2, 2023-10-26T01:32:54.360 INFO:teuthology.orchestra.run.smithi169.stdout: "min_size": 1, 2023-10-26T01:32:54.360 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_rule": 0, 2023-10-26T01:32:54.361 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_count": 0, 2023-10-26T01:32:54.361 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_target": 0, 2023-10-26T01:32:54.361 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T01:32:54.361 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T01:32:54.361 INFO:teuthology.orchestra.run.smithi169.stdout: "object_hash": 2, 2023-10-26T01:32:54.362 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_mode": "off", 2023-10-26T01:32:54.362 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num": 1, 2023-10-26T01:32:54.362 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num": 1, 2023-10-26T01:32:54.362 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num_target": 1, 2023-10-26T01:32:54.363 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_target": 1, 2023-10-26T01:32:54.363 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_pending": 1, 2023-10-26T01:32:54.363 INFO:teuthology.orchestra.run.smithi169.stdout: "last_pg_merge_meta": { 2023-10-26T01:32:54.364 INFO:teuthology.orchestra.run.smithi169.stdout: "source_pgid": "0.0", 2023-10-26T01:32:54.364 INFO:teuthology.orchestra.run.smithi169.stdout: "ready_epoch": 0, 2023-10-26T01:32:54.364 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_started": 0, 2023-10-26T01:32:54.364 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_clean": 0, 2023-10-26T01:32:54.364 INFO:teuthology.orchestra.run.smithi169.stdout: "source_version": "0'0", 2023-10-26T01:32:54.365 INFO:teuthology.orchestra.run.smithi169.stdout: "target_version": "0'0" 2023-10-26T01:32:54.365 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:54.365 INFO:teuthology.orchestra.run.smithi169.stdout: "last_change": "17", 2023-10-26T01:32:54.365 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend": "0", 2023-10-26T01:32:54.365 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T01:32:54.366 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T01:32:54.366 INFO:teuthology.orchestra.run.smithi169.stdout: "auid": 0, 2023-10-26T01:32:54.366 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_mode": "selfmanaged", 2023-10-26T01:32:54.366 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_seq": 0, 2023-10-26T01:32:54.367 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_epoch": 0, 2023-10-26T01:32:54.367 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_snaps": [], 2023-10-26T01:32:54.367 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps": "[]", 2023-10-26T01:32:54.368 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_bytes": 0, 2023-10-26T01:32:54.368 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_objects": 0, 2023-10-26T01:32:54.368 INFO:teuthology.orchestra.run.smithi169.stdout: "tiers": [], 2023-10-26T01:32:54.368 INFO:teuthology.orchestra.run.smithi169.stdout: "tier_of": -1, 2023-10-26T01:32:54.369 INFO:teuthology.orchestra.run.smithi169.stdout: "read_tier": -1, 2023-10-26T01:32:54.369 INFO:teuthology.orchestra.run.smithi169.stdout: "write_tier": -1, 2023-10-26T01:32:54.369 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_mode": "none", 2023-10-26T01:32:54.369 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_bytes": 0, 2023-10-26T01:32:54.370 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_objects": 0, 2023-10-26T01:32:54.370 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T01:32:54.370 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T01:32:54.370 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T01:32:54.371 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_flush_age": 0, 2023-10-26T01:32:54.371 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_evict_age": 0, 2023-10-26T01:32:54.371 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profile": "", 2023-10-26T01:32:54.372 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_params": { 2023-10-26T01:32:54.372 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "none" 2023-10-26T01:32:54.372 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:54.372 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_period": 0, 2023-10-26T01:32:54.373 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_count": 0, 2023-10-26T01:32:54.373 INFO:teuthology.orchestra.run.smithi169.stdout: "use_gmt_hitset": true, 2023-10-26T01:32:54.373 INFO:teuthology.orchestra.run.smithi169.stdout: "min_read_recency_for_promote": 0, 2023-10-26T01:32:54.373 INFO:teuthology.orchestra.run.smithi169.stdout: "min_write_recency_for_promote": 0, 2023-10-26T01:32:54.374 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T01:32:54.374 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_search_last_n": 0, 2023-10-26T01:32:54.374 INFO:teuthology.orchestra.run.smithi169.stdout: "grade_table": [], 2023-10-26T01:32:54.375 INFO:teuthology.orchestra.run.smithi169.stdout: "stripe_width": 0, 2023-10-26T01:32:54.375 INFO:teuthology.orchestra.run.smithi169.stdout: "expected_num_objects": 0, 2023-10-26T01:32:54.376 INFO:teuthology.orchestra.run.smithi169.stdout: "fast_read": false, 2023-10-26T01:32:54.376 INFO:teuthology.orchestra.run.smithi169.stdout: "options": { 2023-10-26T01:32:54.376 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_max": 32, 2023-10-26T01:32:54.377 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_min": 1 2023-10-26T01:32:54.377 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:54.377 INFO:teuthology.orchestra.run.smithi169.stdout: "application_metadata": { 2023-10-26T01:32:54.378 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": {} 2023-10-26T01:32:54.378 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:54.379 INFO:teuthology.orchestra.run.smithi169.stdout: "read_balance": { 2023-10-26T01:32:54.379 INFO:teuthology.orchestra.run.smithi169.stdout: "score_acting": 8, 2023-10-26T01:32:54.379 INFO:teuthology.orchestra.run.smithi169.stdout: "score_stable": 8, 2023-10-26T01:32:54.380 INFO:teuthology.orchestra.run.smithi169.stdout: "optimal_score": 0.25, 2023-10-26T01:32:54.380 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_acting": 2, 2023-10-26T01:32:54.381 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_stable": 2, 2023-10-26T01:32:54.381 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity_weighted": 1, 2023-10-26T01:32:54.381 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity": 1, 2023-10-26T01:32:54.382 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity_weighted": 1 2023-10-26T01:32:54.382 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:32:54.382 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:54.382 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:32:54.383 INFO:teuthology.orchestra.run.smithi169.stdout: "pool": 6, 2023-10-26T01:32:54.383 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_name": "cephfs_metadata", 2023-10-26T01:32:54.383 INFO:teuthology.orchestra.run.smithi169.stdout: "create_time": "2023-10-26T01:32:36.288217+0000", 2023-10-26T01:32:54.383 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": 1, 2023-10-26T01:32:54.383 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_names": "hashpspool", 2023-10-26T01:32:54.383 INFO:teuthology.orchestra.run.smithi169.stdout: "type": 1, 2023-10-26T01:32:54.384 INFO:teuthology.orchestra.run.smithi169.stdout: "size": 2, 2023-10-26T01:32:54.384 INFO:teuthology.orchestra.run.smithi169.stdout: "min_size": 1, 2023-10-26T01:32:54.384 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_rule": 0, 2023-10-26T01:32:54.384 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_count": 0, 2023-10-26T01:32:54.385 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_target": 0, 2023-10-26T01:32:54.385 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T01:32:54.385 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T01:32:54.385 INFO:teuthology.orchestra.run.smithi169.stdout: "object_hash": 2, 2023-10-26T01:32:54.385 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_mode": "off", 2023-10-26T01:32:54.386 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num": 32, 2023-10-26T01:32:54.386 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num": 32, 2023-10-26T01:32:54.386 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num_target": 32, 2023-10-26T01:32:54.386 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_target": 32, 2023-10-26T01:32:54.386 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_pending": 32, 2023-10-26T01:32:54.387 INFO:teuthology.orchestra.run.smithi169.stdout: "last_pg_merge_meta": { 2023-10-26T01:32:54.387 INFO:teuthology.orchestra.run.smithi169.stdout: "source_pgid": "0.0", 2023-10-26T01:32:54.387 INFO:teuthology.orchestra.run.smithi169.stdout: "ready_epoch": 0, 2023-10-26T01:32:54.387 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_started": 0, 2023-10-26T01:32:54.388 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_clean": 0, 2023-10-26T01:32:54.388 INFO:teuthology.orchestra.run.smithi169.stdout: "source_version": "0'0", 2023-10-26T01:32:54.388 INFO:teuthology.orchestra.run.smithi169.stdout: "target_version": "0'0" 2023-10-26T01:32:54.388 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:54.389 INFO:teuthology.orchestra.run.smithi169.stdout: "last_change": "48", 2023-10-26T01:32:54.389 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend": "0", 2023-10-26T01:32:54.390 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T01:32:54.390 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T01:32:54.390 INFO:teuthology.orchestra.run.smithi169.stdout: "auid": 0, 2023-10-26T01:32:54.390 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_mode": "selfmanaged", 2023-10-26T01:32:54.390 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_seq": 0, 2023-10-26T01:32:54.391 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_epoch": 0, 2023-10-26T01:32:54.391 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_snaps": [], 2023-10-26T01:32:54.391 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps": "[]", 2023-10-26T01:32:54.391 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_bytes": 0, 2023-10-26T01:32:54.392 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_objects": 0, 2023-10-26T01:32:54.392 INFO:teuthology.orchestra.run.smithi169.stdout: "tiers": [], 2023-10-26T01:32:54.392 INFO:teuthology.orchestra.run.smithi169.stdout: "tier_of": -1, 2023-10-26T01:32:54.393 INFO:teuthology.orchestra.run.smithi169.stdout: "read_tier": -1, 2023-10-26T01:32:54.393 INFO:teuthology.orchestra.run.smithi169.stdout: "write_tier": -1, 2023-10-26T01:32:54.393 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_mode": "none", 2023-10-26T01:32:54.394 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_bytes": 0, 2023-10-26T01:32:54.394 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_objects": 0, 2023-10-26T01:32:54.395 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T01:32:54.395 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T01:32:54.395 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T01:32:54.395 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_flush_age": 0, 2023-10-26T01:32:54.396 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_evict_age": 0, 2023-10-26T01:32:54.396 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profile": "", 2023-10-26T01:32:54.396 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_params": { 2023-10-26T01:32:54.397 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "none" 2023-10-26T01:32:54.397 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:54.397 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_period": 0, 2023-10-26T01:32:54.397 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_count": 0, 2023-10-26T01:32:54.397 INFO:teuthology.orchestra.run.smithi169.stdout: "use_gmt_hitset": true, 2023-10-26T01:32:54.398 INFO:teuthology.orchestra.run.smithi169.stdout: "min_read_recency_for_promote": 0, 2023-10-26T01:32:54.398 INFO:teuthology.orchestra.run.smithi169.stdout: "min_write_recency_for_promote": 0, 2023-10-26T01:32:54.398 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T01:32:54.398 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_search_last_n": 0, 2023-10-26T01:32:54.399 INFO:teuthology.orchestra.run.smithi169.stdout: "grade_table": [], 2023-10-26T01:32:54.399 INFO:teuthology.orchestra.run.smithi169.stdout: "stripe_width": 0, 2023-10-26T01:32:54.399 INFO:teuthology.orchestra.run.smithi169.stdout: "expected_num_objects": 0, 2023-10-26T01:32:54.399 INFO:teuthology.orchestra.run.smithi169.stdout: "fast_read": false, 2023-10-26T01:32:54.399 INFO:teuthology.orchestra.run.smithi169.stdout: "options": { 2023-10-26T01:32:54.399 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_bias": 4, 2023-10-26T01:32:54.400 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_min": 16, 2023-10-26T01:32:54.400 INFO:teuthology.orchestra.run.smithi169.stdout: "recovery_priority": 5 2023-10-26T01:32:54.400 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:54.400 INFO:teuthology.orchestra.run.smithi169.stdout: "application_metadata": { 2023-10-26T01:32:54.400 INFO:teuthology.orchestra.run.smithi169.stdout: "cephfs": { 2023-10-26T01:32:54.401 INFO:teuthology.orchestra.run.smithi169.stdout: "metadata": "cephfs" 2023-10-26T01:32:54.401 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:32:54.401 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:54.401 INFO:teuthology.orchestra.run.smithi169.stdout: "read_balance": { 2023-10-26T01:32:54.401 INFO:teuthology.orchestra.run.smithi169.stdout: "score_acting": 1.75, 2023-10-26T01:32:54.402 INFO:teuthology.orchestra.run.smithi169.stdout: "score_stable": 1.75, 2023-10-26T01:32:54.402 INFO:teuthology.orchestra.run.smithi169.stdout: "optimal_score": 1, 2023-10-26T01:32:54.402 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_acting": 1.75, 2023-10-26T01:32:54.402 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_stable": 1.75, 2023-10-26T01:32:54.402 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity_weighted": 1, 2023-10-26T01:32:54.403 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity": 1, 2023-10-26T01:32:54.403 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity_weighted": 1 2023-10-26T01:32:54.403 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:32:54.403 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:54.404 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:32:54.404 INFO:teuthology.orchestra.run.smithi169.stdout: "pool": 7, 2023-10-26T01:32:54.404 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_name": "cephfs_data", 2023-10-26T01:32:54.404 INFO:teuthology.orchestra.run.smithi169.stdout: "create_time": "2023-10-26T01:32:37.059775+0000", 2023-10-26T01:32:54.405 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": 1, 2023-10-26T01:32:54.405 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_names": "hashpspool", 2023-10-26T01:32:54.405 INFO:teuthology.orchestra.run.smithi169.stdout: "type": 1, 2023-10-26T01:32:54.405 INFO:teuthology.orchestra.run.smithi169.stdout: "size": 2, 2023-10-26T01:32:54.405 INFO:teuthology.orchestra.run.smithi169.stdout: "min_size": 1, 2023-10-26T01:32:54.406 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_rule": 0, 2023-10-26T01:32:54.406 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_count": 0, 2023-10-26T01:32:54.406 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_target": 0, 2023-10-26T01:32:54.406 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T01:32:54.406 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T01:32:54.407 INFO:teuthology.orchestra.run.smithi169.stdout: "object_hash": 2, 2023-10-26T01:32:54.407 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_mode": "off", 2023-10-26T01:32:54.407 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num": 64, 2023-10-26T01:32:54.407 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num": 64, 2023-10-26T01:32:54.407 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num_target": 64, 2023-10-26T01:32:54.407 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_target": 64, 2023-10-26T01:32:54.408 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_pending": 64, 2023-10-26T01:32:54.408 INFO:teuthology.orchestra.run.smithi169.stdout: "last_pg_merge_meta": { 2023-10-26T01:32:54.408 INFO:teuthology.orchestra.run.smithi169.stdout: "source_pgid": "0.0", 2023-10-26T01:32:54.408 INFO:teuthology.orchestra.run.smithi169.stdout: "ready_epoch": 0, 2023-10-26T01:32:54.408 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_started": 0, 2023-10-26T01:32:54.409 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_clean": 0, 2023-10-26T01:32:54.409 INFO:teuthology.orchestra.run.smithi169.stdout: "source_version": "0'0", 2023-10-26T01:32:54.409 INFO:teuthology.orchestra.run.smithi169.stdout: "target_version": "0'0" 2023-10-26T01:32:54.410 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:54.410 INFO:teuthology.orchestra.run.smithi169.stdout: "last_change": "49", 2023-10-26T01:32:54.410 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend": "0", 2023-10-26T01:32:54.411 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T01:32:54.411 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T01:32:54.411 INFO:teuthology.orchestra.run.smithi169.stdout: "auid": 0, 2023-10-26T01:32:54.412 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_mode": "selfmanaged", 2023-10-26T01:32:54.412 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_seq": 0, 2023-10-26T01:32:54.412 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_epoch": 0, 2023-10-26T01:32:54.413 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_snaps": [], 2023-10-26T01:32:54.413 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps": "[]", 2023-10-26T01:32:54.413 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_bytes": 0, 2023-10-26T01:32:54.414 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_objects": 0, 2023-10-26T01:32:54.414 INFO:teuthology.orchestra.run.smithi169.stdout: "tiers": [], 2023-10-26T01:32:54.415 INFO:teuthology.orchestra.run.smithi169.stdout: "tier_of": -1, 2023-10-26T01:32:54.415 INFO:teuthology.orchestra.run.smithi169.stdout: "read_tier": -1, 2023-10-26T01:32:54.415 INFO:teuthology.orchestra.run.smithi169.stdout: "write_tier": -1, 2023-10-26T01:32:54.416 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_mode": "none", 2023-10-26T01:32:54.416 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_bytes": 0, 2023-10-26T01:32:54.416 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_objects": 0, 2023-10-26T01:32:54.417 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T01:32:54.417 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T01:32:54.417 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T01:32:54.418 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_flush_age": 0, 2023-10-26T01:32:54.418 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_evict_age": 0, 2023-10-26T01:32:54.419 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profile": "", 2023-10-26T01:32:54.419 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_params": { 2023-10-26T01:32:54.419 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "none" 2023-10-26T01:32:54.419 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:54.419 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_period": 0, 2023-10-26T01:32:54.420 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_count": 0, 2023-10-26T01:32:54.420 INFO:teuthology.orchestra.run.smithi169.stdout: "use_gmt_hitset": true, 2023-10-26T01:32:54.420 INFO:teuthology.orchestra.run.smithi169.stdout: "min_read_recency_for_promote": 0, 2023-10-26T01:32:54.420 INFO:teuthology.orchestra.run.smithi169.stdout: "min_write_recency_for_promote": 0, 2023-10-26T01:32:54.420 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T01:32:54.420 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_search_last_n": 0, 2023-10-26T01:32:54.421 INFO:teuthology.orchestra.run.smithi169.stdout: "grade_table": [], 2023-10-26T01:32:54.421 INFO:teuthology.orchestra.run.smithi169.stdout: "stripe_width": 0, 2023-10-26T01:32:54.421 INFO:teuthology.orchestra.run.smithi169.stdout: "expected_num_objects": 0, 2023-10-26T01:32:54.421 INFO:teuthology.orchestra.run.smithi169.stdout: "fast_read": false, 2023-10-26T01:32:54.421 INFO:teuthology.orchestra.run.smithi169.stdout: "options": { 2023-10-26T01:32:54.422 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_min": 64, 2023-10-26T01:32:54.422 INFO:teuthology.orchestra.run.smithi169.stdout: "target_size_ratio": 0.89999997615814209 2023-10-26T01:32:54.422 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:54.422 INFO:teuthology.orchestra.run.smithi169.stdout: "application_metadata": { 2023-10-26T01:32:54.422 INFO:teuthology.orchestra.run.smithi169.stdout: "cephfs": { 2023-10-26T01:32:54.423 INFO:teuthology.orchestra.run.smithi169.stdout: "data": "cephfs" 2023-10-26T01:32:54.423 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:32:54.423 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:54.423 INFO:teuthology.orchestra.run.smithi169.stdout: "read_balance": { 2023-10-26T01:32:54.423 INFO:teuthology.orchestra.run.smithi169.stdout: "score_acting": 1.75, 2023-10-26T01:32:54.424 INFO:teuthology.orchestra.run.smithi169.stdout: "score_stable": 1.75, 2023-10-26T01:32:54.424 INFO:teuthology.orchestra.run.smithi169.stdout: "optimal_score": 1, 2023-10-26T01:32:54.424 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_acting": 1.75, 2023-10-26T01:32:54.424 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_stable": 1.75, 2023-10-26T01:32:54.424 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity_weighted": 1, 2023-10-26T01:32:54.425 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity": 1, 2023-10-26T01:32:54.425 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity_weighted": 1 2023-10-26T01:32:54.425 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:32:54.425 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:32:54.425 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:32:54.425 INFO:teuthology.orchestra.run.smithi169.stdout: "osds": [ 2023-10-26T01:32:54.426 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:32:54.426 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 0, 2023-10-26T01:32:54.426 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:32:54.426 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:32:54.426 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:32:54.427 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:32:54.427 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:32:54.427 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:32:54.427 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:32:54.427 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:32:54.428 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 47, 2023-10-26T01:32:54.428 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:32:54.428 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:32:54.428 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:32:54.428 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:32:54.429 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:32:54.429 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:32:54.429 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6816", 2023-10-26T01:32:54.429 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:32:54.429 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:54.429 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:32:54.430 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:32:54.430 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6817", 2023-10-26T01:32:54.430 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:32:54.430 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:32:54.430 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:32:54.431 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:54.431 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:32:54.431 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:32:54.431 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:32:54.431 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:32:54.432 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6818", 2023-10-26T01:32:54.432 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:32:54.432 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:54.432 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:32:54.432 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:32:54.433 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6819", 2023-10-26T01:32:54.433 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:32:54.433 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:32:54.433 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:32:54.433 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:54.433 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:32:54.434 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:32:54.434 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:32:54.434 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:32:54.434 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6822", 2023-10-26T01:32:54.434 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:32:54.435 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:54.435 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:32:54.435 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:32:54.435 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6823", 2023-10-26T01:32:54.435 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:32:54.436 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:32:54.436 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:32:54.436 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:54.436 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:32:54.436 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:32:54.436 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:32:54.437 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:32:54.437 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6820", 2023-10-26T01:32:54.437 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:32:54.437 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:54.437 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:32:54.438 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:32:54.438 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6821", 2023-10-26T01:32:54.438 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:32:54.438 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:32:54.438 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:32:54.439 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:54.439 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6817/4204255864", 2023-10-26T01:32:54.439 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6819/4204255864", 2023-10-26T01:32:54.439 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6823/4204255864", 2023-10-26T01:32:54.439 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6821/4204255864", 2023-10-26T01:32:54.440 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:32:54.440 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:32:54.440 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:32:54.440 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:32:54.440 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:54.441 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:32:54.441 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 1, 2023-10-26T01:32:54.441 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:32:54.441 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:32:54.441 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:32:54.441 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:32:54.442 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:32:54.442 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:32:54.442 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:32:54.442 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:32:54.442 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 47, 2023-10-26T01:32:54.443 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:32:54.443 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:32:54.443 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:32:54.443 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:32:54.443 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:32:54.444 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:32:54.444 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6808", 2023-10-26T01:32:54.444 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:32:54.444 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:54.444 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:32:54.445 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:32:54.445 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6809", 2023-10-26T01:32:54.445 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:32:54.445 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:32:54.445 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:32:54.446 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:54.446 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:32:54.446 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:32:54.446 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:32:54.446 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:32:54.447 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6810", 2023-10-26T01:32:54.447 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:32:54.447 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:54.447 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:32:54.447 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:32:54.448 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6811", 2023-10-26T01:32:54.448 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:32:54.448 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:32:54.448 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:32:54.448 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:54.449 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:32:54.449 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:32:54.449 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:32:54.449 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:32:54.449 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6814", 2023-10-26T01:32:54.450 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:32:54.450 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:54.450 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:32:54.450 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:32:54.451 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6815", 2023-10-26T01:32:54.451 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:32:54.451 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:32:54.451 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:32:54.451 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:54.452 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:32:54.452 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:32:54.452 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:32:54.452 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:32:54.452 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6812", 2023-10-26T01:32:54.453 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:32:54.453 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:54.453 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:32:54.453 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:32:54.453 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6813", 2023-10-26T01:32:54.454 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:32:54.454 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:32:54.454 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:32:54.454 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:54.455 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6809/238206046", 2023-10-26T01:32:54.455 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6811/238206046", 2023-10-26T01:32:54.455 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6815/238206046", 2023-10-26T01:32:54.455 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6813/238206046", 2023-10-26T01:32:54.455 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:32:54.456 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:32:54.456 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:32:54.456 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:32:54.456 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:54.456 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:32:54.457 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 2, 2023-10-26T01:32:54.457 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:32:54.457 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:32:54.457 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:32:54.457 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:32:54.458 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:32:54.458 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:32:54.458 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:32:54.458 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:32:54.458 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 47, 2023-10-26T01:32:54.459 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:32:54.459 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:32:54.459 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:32:54.459 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:32:54.459 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:32:54.459 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:32:54.460 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6800", 2023-10-26T01:32:54.460 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:32:54.460 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:54.460 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:32:54.460 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:32:54.460 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6801", 2023-10-26T01:32:54.461 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:32:54.461 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:32:54.461 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:32:54.461 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:54.461 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:32:54.462 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:32:54.462 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:32:54.462 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:32:54.462 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6802", 2023-10-26T01:32:54.462 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:32:54.463 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:54.463 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:32:54.463 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:32:54.463 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6803", 2023-10-26T01:32:54.463 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:32:54.464 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:32:54.464 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:32:54.464 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:54.464 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:32:54.464 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:32:54.464 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:32:54.465 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:32:54.465 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6806", 2023-10-26T01:32:54.465 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:32:54.465 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:54.465 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:32:54.466 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:32:54.466 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6807", 2023-10-26T01:32:54.466 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:32:54.466 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:32:54.466 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:32:54.467 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:54.467 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:32:54.467 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:32:54.467 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:32:54.467 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:32:54.468 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6804", 2023-10-26T01:32:54.468 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:32:54.468 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:54.468 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:32:54.468 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:32:54.469 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6805", 2023-10-26T01:32:54.469 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:32:54.469 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:32:54.469 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:32:54.469 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:54.470 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6801/3215791846", 2023-10-26T01:32:54.470 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6803/3215791846", 2023-10-26T01:32:54.470 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6807/3215791846", 2023-10-26T01:32:54.470 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6805/3215791846", 2023-10-26T01:32:54.471 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:32:54.471 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:32:54.471 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:32:54.471 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:32:54.471 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:54.472 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:32:54.472 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 3, 2023-10-26T01:32:54.472 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:32:54.472 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:32:54.472 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:32:54.472 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:32:54.473 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:32:54.473 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:32:54.473 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:32:54.473 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 14, 2023-10-26T01:32:54.473 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 47, 2023-10-26T01:32:54.474 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:32:54.474 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:32:54.474 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:32:54.474 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:32:54.474 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:32:54.475 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:32:54.475 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6824", 2023-10-26T01:32:54.475 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:32:54.475 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:54.475 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:32:54.476 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:32:54.476 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6825", 2023-10-26T01:32:54.477 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:32:54.477 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:32:54.477 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:32:54.477 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:54.477 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:32:54.478 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:32:54.478 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:32:54.478 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:32:54.478 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6826", 2023-10-26T01:32:54.479 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:32:54.479 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:54.479 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:32:54.479 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:32:54.479 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6827", 2023-10-26T01:32:54.479 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:32:54.480 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:32:54.480 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:32:54.480 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:54.480 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:32:54.481 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:32:54.481 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:32:54.481 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:32:54.481 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6830", 2023-10-26T01:32:54.482 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:32:54.482 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:54.482 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:32:54.482 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:32:54.483 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6831", 2023-10-26T01:32:54.483 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:32:54.483 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:32:54.483 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:32:54.483 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:54.484 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:32:54.484 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:32:54.484 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:32:54.484 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:32:54.484 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6828", 2023-10-26T01:32:54.485 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:32:54.485 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:54.485 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:32:54.485 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:32:54.485 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6829", 2023-10-26T01:32:54.485 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:32:54.486 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:32:54.486 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:32:54.486 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:54.486 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6825/2256701226", 2023-10-26T01:32:54.486 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6827/2256701226", 2023-10-26T01:32:54.487 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6831/2256701226", 2023-10-26T01:32:54.487 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6829/2256701226", 2023-10-26T01:32:54.487 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:32:54.487 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:32:54.487 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:32:54.487 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:32:54.488 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:54.488 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:32:54.488 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 4, 2023-10-26T01:32:54.488 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:32:54.488 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:32:54.489 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:32:54.489 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:32:54.489 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:32:54.489 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:32:54.489 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:32:54.490 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:32:54.490 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 47, 2023-10-26T01:32:54.490 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:32:54.490 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:32:54.491 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:32:54.491 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:32:54.491 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:32:54.491 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:32:54.491 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6816", 2023-10-26T01:32:54.491 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:32:54.492 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:54.492 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:32:54.492 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:32:54.492 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6817", 2023-10-26T01:32:54.492 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:32:54.493 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:32:54.493 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:32:54.493 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:54.493 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:32:54.493 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:32:54.494 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:32:54.494 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:32:54.494 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6818", 2023-10-26T01:32:54.494 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:32:54.494 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:54.495 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:32:54.495 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:32:54.495 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6819", 2023-10-26T01:32:54.495 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:32:54.495 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:32:54.495 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:32:54.496 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:54.496 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:32:54.496 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:32:54.496 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:32:54.496 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:32:54.497 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6825", 2023-10-26T01:32:54.497 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:32:54.497 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:54.497 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:32:54.497 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:32:54.498 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6828", 2023-10-26T01:32:54.498 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:32:54.498 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:32:54.499 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:32:54.499 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:54.499 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:32:54.499 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:32:54.500 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:32:54.500 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:32:54.500 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6820", 2023-10-26T01:32:54.500 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:32:54.500 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:54.500 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:32:54.501 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:32:54.501 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6823", 2023-10-26T01:32:54.501 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:32:54.501 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:32:54.501 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:32:54.501 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:54.502 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6817/1082279874", 2023-10-26T01:32:54.502 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6819/1082279874", 2023-10-26T01:32:54.502 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6828/1082279874", 2023-10-26T01:32:54.502 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6823/1082279874", 2023-10-26T01:32:54.502 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:32:54.503 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:32:54.503 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:32:54.503 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:32:54.503 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:54.503 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:32:54.504 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 5, 2023-10-26T01:32:54.504 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:32:54.504 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:32:54.504 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:32:54.504 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:32:54.505 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:32:54.505 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:32:54.505 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:32:54.505 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:32:54.505 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 47, 2023-10-26T01:32:54.505 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:32:54.506 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:32:54.506 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:32:54.506 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:32:54.506 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:32:54.507 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:32:54.507 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6808", 2023-10-26T01:32:54.507 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:32:54.507 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:54.507 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:32:54.508 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:32:54.508 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6809", 2023-10-26T01:32:54.508 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:32:54.508 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:32:54.508 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:32:54.509 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:54.509 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:32:54.509 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:32:54.509 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:32:54.509 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:32:54.509 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6810", 2023-10-26T01:32:54.510 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:32:54.510 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:54.510 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:32:54.511 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:32:54.511 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6811", 2023-10-26T01:32:54.511 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:32:54.512 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:32:54.512 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:32:54.513 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:54.513 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:32:54.513 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:32:54.514 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:32:54.514 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:32:54.514 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6814", 2023-10-26T01:32:54.515 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:32:54.515 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:54.516 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:32:54.516 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:32:54.516 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6815", 2023-10-26T01:32:54.517 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:32:54.517 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:32:54.518 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:32:54.518 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:54.518 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:32:54.519 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:32:54.519 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:32:54.520 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:32:54.520 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6812", 2023-10-26T01:32:54.520 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:32:54.521 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:54.521 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:32:54.521 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:32:54.522 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6813", 2023-10-26T01:32:54.522 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:32:54.523 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:32:54.523 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:32:54.523 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:54.524 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6809/3335654351", 2023-10-26T01:32:54.524 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6811/3335654351", 2023-10-26T01:32:54.524 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6815/3335654351", 2023-10-26T01:32:54.525 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6813/3335654351", 2023-10-26T01:32:54.525 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:32:54.525 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:32:54.526 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:32:54.526 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:32:54.527 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:54.527 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:32:54.527 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 6, 2023-10-26T01:32:54.528 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:32:54.528 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:32:54.528 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:32:54.529 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:32:54.529 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:32:54.530 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:32:54.530 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:32:54.530 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:32:54.531 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 47, 2023-10-26T01:32:54.531 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:32:54.532 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:32:54.532 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:32:54.532 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:32:54.532 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:32:54.533 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:32:54.533 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6800", 2023-10-26T01:32:54.533 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:32:54.533 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:54.533 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:32:54.534 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:32:54.534 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6801", 2023-10-26T01:32:54.534 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:32:54.534 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:32:54.534 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:32:54.535 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:54.535 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:32:54.535 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:32:54.535 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:32:54.535 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:32:54.536 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6802", 2023-10-26T01:32:54.536 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:32:54.536 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:54.536 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:32:54.536 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:32:54.536 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6803", 2023-10-26T01:32:54.537 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:32:54.537 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:32:54.537 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:32:54.537 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:54.537 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:32:54.538 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:32:54.538 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:32:54.538 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:32:54.538 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6806", 2023-10-26T01:32:54.538 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:32:54.539 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:54.539 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:32:54.539 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:32:54.539 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6807", 2023-10-26T01:32:54.539 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:32:54.540 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:32:54.540 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:32:54.540 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:54.540 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:32:54.540 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:32:54.540 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:32:54.541 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:32:54.541 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6804", 2023-10-26T01:32:54.541 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:32:54.541 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:54.541 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:32:54.542 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:32:54.542 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6805", 2023-10-26T01:32:54.542 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:32:54.542 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:32:54.542 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:32:54.543 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:54.543 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6801/2520460697", 2023-10-26T01:32:54.543 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6803/2520460697", 2023-10-26T01:32:54.543 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6807/2520460697", 2023-10-26T01:32:54.543 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6805/2520460697", 2023-10-26T01:32:54.544 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:32:54.544 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:32:54.544 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:32:54.544 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:32:54.545 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:54.545 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:32:54.545 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 7, 2023-10-26T01:32:54.545 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:32:54.546 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:32:54.546 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:32:54.546 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:32:54.546 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:32:54.546 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:32:54.547 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:32:54.547 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:32:54.547 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 47, 2023-10-26T01:32:54.547 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:32:54.547 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:32:54.548 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:32:54.548 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:32:54.548 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:32:54.548 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:32:54.548 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6821", 2023-10-26T01:32:54.549 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:32:54.549 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:54.549 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:32:54.549 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:32:54.550 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6822", 2023-10-26T01:32:54.550 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:32:54.550 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:32:54.550 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:32:54.550 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:54.551 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:32:54.551 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:32:54.551 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:32:54.551 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:32:54.551 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6824", 2023-10-26T01:32:54.552 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:32:54.552 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:54.552 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:32:54.552 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:32:54.552 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6826", 2023-10-26T01:32:54.553 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:32:54.553 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:32:54.553 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:32:54.553 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:54.553 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:32:54.554 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:32:54.554 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:32:54.554 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:32:54.554 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6830", 2023-10-26T01:32:54.554 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:32:54.555 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:54.555 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:32:54.555 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:32:54.555 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6831", 2023-10-26T01:32:54.555 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:32:54.556 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:32:54.556 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:32:54.556 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:54.556 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:32:54.556 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:32:54.557 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:32:54.557 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:32:54.557 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6827", 2023-10-26T01:32:54.557 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:32:54.557 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:54.558 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:32:54.558 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:32:54.558 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6829", 2023-10-26T01:32:54.558 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:32:54.558 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:32:54.559 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:32:54.559 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:54.559 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6822/1115930619", 2023-10-26T01:32:54.559 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6826/1115930619", 2023-10-26T01:32:54.559 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6831/1115930619", 2023-10-26T01:32:54.561 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6829/1115930619", 2023-10-26T01:32:54.561 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:32:54.561 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:32:54.562 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:32:54.562 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:32:54.562 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:32:54.562 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:32:54.562 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_xinfo": [ 2023-10-26T01:32:54.563 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:32:54.563 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 0, 2023-10-26T01:32:54.563 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:32:54.563 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:32:54.563 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:32:54.564 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:32:54.564 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:32:54.564 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.889675+0000", 2023-10-26T01:32:54.564 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:32:54.564 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:54.565 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:32:54.565 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 1, 2023-10-26T01:32:54.565 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:32:54.565 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:32:54.565 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:32:54.566 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:32:54.566 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:32:54.566 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.864157+0000", 2023-10-26T01:32:54.566 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:32:54.566 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:54.567 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:32:54.567 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 2, 2023-10-26T01:32:54.567 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:32:54.568 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:32:54.568 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:32:54.568 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:32:54.569 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:32:54.569 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.876919+0000", 2023-10-26T01:32:54.569 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:32:54.569 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:54.570 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:32:54.570 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 3, 2023-10-26T01:32:54.570 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:32:54.570 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:32:54.571 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:32:54.571 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:32:54.571 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:32:54.571 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.974541+0000", 2023-10-26T01:32:54.571 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:32:54.572 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:54.572 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:32:54.572 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 4, 2023-10-26T01:32:54.572 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:32:54.572 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:32:54.573 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:32:54.573 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:32:54.573 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:32:54.573 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.969696+0000", 2023-10-26T01:32:54.573 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:32:54.574 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:54.574 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:32:54.574 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 5, 2023-10-26T01:32:54.574 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:32:54.575 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:32:54.575 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:32:54.575 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:32:54.575 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:32:54.575 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.859703+0000", 2023-10-26T01:32:54.575 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:32:54.576 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:54.576 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:32:54.576 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 6, 2023-10-26T01:32:54.577 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:32:54.577 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:32:54.577 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:32:54.577 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:32:54.578 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:32:54.578 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.867917+0000", 2023-10-26T01:32:54.578 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:32:54.578 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:54.579 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:32:54.579 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 7, 2023-10-26T01:32:54.579 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:32:54.579 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:32:54.580 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:32:54.580 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:32:54.581 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:32:54.581 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.898605+0000", 2023-10-26T01:32:54.581 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:32:54.581 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:32:54.582 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:32:54.582 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_upmap": [], 2023-10-26T01:32:54.582 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_upmap_items": [], 2023-10-26T01:32:54.582 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_upmap_primaries": [], 2023-10-26T01:32:54.582 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_temp": [], 2023-10-26T01:32:54.583 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_temp": [], 2023-10-26T01:32:54.583 INFO:teuthology.orchestra.run.smithi169.stdout: "blocklist": {}, 2023-10-26T01:32:54.583 INFO:teuthology.orchestra.run.smithi169.stdout: "range_blocklist": {}, 2023-10-26T01:32:54.583 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profiles": { 2023-10-26T01:32:54.584 INFO:teuthology.orchestra.run.smithi169.stdout: "default": { 2023-10-26T01:32:54.584 INFO:teuthology.orchestra.run.smithi169.stdout: "crush-failure-domain": "osd", 2023-10-26T01:32:54.584 INFO:teuthology.orchestra.run.smithi169.stdout: "k": "2", 2023-10-26T01:32:54.584 INFO:teuthology.orchestra.run.smithi169.stdout: "m": "1", 2023-10-26T01:32:54.584 INFO:teuthology.orchestra.run.smithi169.stdout: "plugin": "jerasure", 2023-10-26T01:32:54.585 INFO:teuthology.orchestra.run.smithi169.stdout: "technique": "reed_sol_van" 2023-10-26T01:32:54.585 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:32:54.585 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:32:54.585 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps_queue": [], 2023-10-26T01:32:54.586 INFO:teuthology.orchestra.run.smithi169.stdout: "new_removed_snaps": [], 2023-10-26T01:32:54.586 INFO:teuthology.orchestra.run.smithi169.stdout: "new_purged_snaps": [], 2023-10-26T01:32:54.586 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_node_flags": {}, 2023-10-26T01:32:54.586 INFO:teuthology.orchestra.run.smithi169.stdout: "device_class_flags": {}, 2023-10-26T01:32:54.586 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_mode": { 2023-10-26T01:32:54.587 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_mode_enabled": false, 2023-10-26T01:32:54.587 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_bucket_count": 0, 2023-10-26T01:32:54.587 INFO:teuthology.orchestra.run.smithi169.stdout: "degraded_stretch_mode": 0, 2023-10-26T01:32:54.587 INFO:teuthology.orchestra.run.smithi169.stdout: "recovering_stretch_mode": 0, 2023-10-26T01:32:54.588 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_mode_bucket": 0 2023-10-26T01:32:54.588 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:32:54.588 INFO:teuthology.orchestra.run.smithi169.stdout:} 2023-10-26T01:32:54.590 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:54.350+0000 7fd423fff640 1 -- 172.21.15.169:0/3202420550 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd41006ae40 msgr2=0x7fd41006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:54.591 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:54.350+0000 7fd423fff640 1 --2- 172.21.15.169:0/3202420550 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd41006ae40 0x7fd41006d300 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd424067970 tx=0x7fd418001190 comp rx=0 tx=0).stop 2023-10-26T01:32:54.591 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:54.350+0000 7fd423fff640 1 -- 172.21.15.169:0/3202420550 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd42409a5c0 msgr2=0x7fd4241f5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:54.592 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:54.350+0000 7fd423fff640 1 --2- 172.21.15.169:0/3202420550 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd42409a5c0 0x7fd4241f5a10 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fd41400ea60 tx=0x7fd41400ef20 comp rx=0 tx=0).stop 2023-10-26T01:32:54.592 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:54.350+0000 7fd423fff640 1 -- 172.21.15.169:0/3202420550 shutdown_connections 2023-10-26T01:32:54.592 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:54.350+0000 7fd423fff640 1 --2- 172.21.15.169:0/3202420550 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd41006ae40 0x7fd41006d300 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:54.592 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:54.350+0000 7fd423fff640 1 --2- 172.21.15.169:0/3202420550 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd4240c9970 0x7fd42413f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:54.593 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:54.350+0000 7fd423fff640 1 --2- 172.21.15.169:0/3202420550 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd4240ac6d0 0x7fd42415a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:54.593 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:54.350+0000 7fd423fff640 1 --2- 172.21.15.169:0/3202420550 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd42409a5c0 0x7fd4241f5a10 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:54.593 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:54.350+0000 7fd423fff640 1 -- 172.21.15.169:0/3202420550 >> 172.21.15.169:0/3202420550 conn(0x7fd4240aedd0 msgr2=0x7fd4240c6b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:54.594 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:54.351+0000 7fd423fff640 1 -- 172.21.15.169:0/3202420550 shutdown_connections 2023-10-26T01:32:54.594 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:54.351+0000 7fd423fff640 1 -- 172.21.15.169:0/3202420550 wait complete. 2023-10-26T01:32:54.595 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2023-10-26T01:32:57.018 INFO:tasks.ceph.mon.a.smithi169.stderr:2023-10-26T01:32:57.016+0000 7f158b279640 -1 log_channel(cluster) log [ERR] : Health check failed: 3 full osd(s) (OSD_FULL) 2023-10-26T01:32:59.596 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-26T01:32:59.826 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:59.824+0000 7fce9fde7640 1 -- 172.21.15.169:0/3282202740 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce980cdd80 msgr2=0x7fce980b07d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:59.826 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:59.824+0000 7fce9fde7640 1 --2- 172.21.15.169:0/3282202740 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce980cdd80 0x7fce980b07d0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fce90005fd0 tx=0x7fce9002c2e0 comp rx=0 tx=0).stop 2023-10-26T01:32:59.826 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:59.824+0000 7fce9fde7640 1 -- 172.21.15.169:0/3282202740 shutdown_connections 2023-10-26T01:32:59.827 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:59.824+0000 7fce9fde7640 1 --2- 172.21.15.169:0/3282202740 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce980cdd80 0x7fce980b07d0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:59.827 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:59.824+0000 7fce9fde7640 1 -- 172.21.15.169:0/3282202740 >> 172.21.15.169:0/3282202740 conn(0x7fce98069930 msgr2=0x7fce98069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:32:59.827 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:59.824+0000 7fce9fde7640 1 -- 172.21.15.169:0/3282202740 shutdown_connections 2023-10-26T01:32:59.827 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:59.824+0000 7fce9fde7640 1 -- 172.21.15.169:0/3282202740 wait complete. 2023-10-26T01:32:59.828 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:59.826+0000 7fce9fde7640 1 Processor -- start 2023-10-26T01:32:59.828 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:59.826+0000 7fce9fde7640 1 -- start start 2023-10-26T01:32:59.829 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:59.827+0000 7fce9fde7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce9806ed30 0x7fce9814e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:59.830 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:59.827+0000 7fce9fde7640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce980cdd80 0x7fce980ace10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:59.830 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:59.827+0000 7fce9fde7640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fce9815b610 0x7fce980ad350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:59.830 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:59.827+0000 7fce9fde7640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce98157820 con 0x7fce980cdd80 2023-10-26T01:32:59.830 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:59.827+0000 7fce9fde7640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce98157b00 con 0x7fce9806ed30 2023-10-26T01:32:59.831 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:59.827+0000 7fce9fde7640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce98157de0 con 0x7fce9815b610 2023-10-26T01:32:59.831 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:59.827+0000 7fce9d382640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce980cdd80 0x7fce980ace10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:59.831 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:59.827+0000 7fce9d382640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce980cdd80 0x7fce980ace10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:58126/0 (socket says 172.21.15.169:58126) 2023-10-26T01:32:59.831 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:59.827+0000 7fce9d382640 1 -- 172.21.15.169:0/1669530505 learned_addr learned my addr 172.21.15.169:0/1669530505 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:32:59.831 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:59.827+0000 7fce9d382640 1 -- 172.21.15.169:0/1669530505 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fce9815b610 msgr2=0x7fce980ad350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:59.832 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:59.827+0000 7fce9d382640 1 --2- 172.21.15.169:0/1669530505 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fce9815b610 0x7fce980ad350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:59.832 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:59.827+0000 7fce9d382640 1 -- 172.21.15.169:0/1669530505 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce9806ed30 msgr2=0x7fce9814e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:32:59.832 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:59.827+0000 7fce9d382640 1 --2- 172.21.15.169:0/1669530505 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce9806ed30 0x7fce9814e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:32:59.832 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:59.827+0000 7fce9d382640 1 -- 172.21.15.169:0/1669530505 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce90009900 con 0x7fce980cdd80 2023-10-26T01:32:59.832 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:59.828+0000 7fce9d382640 1 --2- 172.21.15.169:0/1669530505 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce980cdd80 0x7fce980ace10 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fce8800c450 tx=0x7fce8800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:59.832 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:59.828+0000 7fce8e7fc640 1 -- 172.21.15.169:0/1669530505 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce880052d0 con 0x7fce980cdd80 2023-10-26T01:32:59.833 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:59.828+0000 7fce8e7fc640 1 -- 172.21.15.169:0/1669530505 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fce88005430 con 0x7fce980cdd80 2023-10-26T01:32:59.833 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:59.828+0000 7fce9fde7640 1 -- 172.21.15.169:0/1669530505 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce980972d0 con 0x7fce980cdd80 2023-10-26T01:32:59.833 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:59.828+0000 7fce8e7fc640 1 -- 172.21.15.169:0/1669530505 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce88012890 con 0x7fce980cdd80 2023-10-26T01:32:59.833 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:59.828+0000 7fce9fde7640 1 -- 172.21.15.169:0/1669530505 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce9814eaf0 con 0x7fce980cdd80 2023-10-26T01:32:59.834 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:59.829+0000 7fce9fde7640 1 -- 172.21.15.169:0/1669530505 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce980977f0 con 0x7fce980cdd80 2023-10-26T01:32:59.835 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:59.830+0000 7fce8e7fc640 1 -- 172.21.15.169:0/1669530505 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fce880035f0 con 0x7fce980cdd80 2023-10-26T01:32:59.835 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:59.831+0000 7fce8e7fc640 1 --2- 172.21.15.169:0/1669530505 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fce7c06ae40 0x7fce7c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:32:59.835 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:59.831+0000 7fce8e7fc640 1 -- 172.21.15.169:0/1669530505 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fce88089360 con 0x7fce980cdd80 2023-10-26T01:32:59.835 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:59.832+0000 7fce8e7fc640 1 -- 172.21.15.169:0/1669530505 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=52}) v3 -- 0x7fce7c072970 con 0x7fce980cdd80 2023-10-26T01:32:59.835 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:59.832+0000 7fce9db83640 1 --2- 172.21.15.169:0/1669530505 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fce7c06ae40 0x7fce7c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:32:59.836 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:59.833+0000 7fce9db83640 1 --2- 172.21.15.169:0/1669530505 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fce7c06ae40 0x7fce7c06d300 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fce98158af0 tx=0x7fce94006ba0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:32:59.837 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:32:59.836+0000 7fce8e7fc640 1 -- 172.21.15.169:0/1669530505 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fce980977f0 con 0x7fce980cdd80 2023-10-26T01:33:00.028 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:00.026+0000 7fce9fde7640 1 -- 172.21.15.169:0/1669530505 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fce980977f0 con 0x7fce980cdd80 2023-10-26T01:33:00.031 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:00.030+0000 7fce8e7fc640 1 -- 172.21.15.169:0/1669530505 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v51) v1 ==== 81+0+34139 (secure 0 0 0) 0x7fce980977f0 con 0x7fce980cdd80 2023-10-26T01:33:00.032 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:33:00.033 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2023-10-26T01:33:00.033 INFO:teuthology.orchestra.run.smithi169.stdout: "epoch": 51, 2023-10-26T01:33:00.033 INFO:teuthology.orchestra.run.smithi169.stdout: "fsid": "30203b28-ed8a-44e8-a9a6-a9bf25f980f1", 2023-10-26T01:33:00.033 INFO:teuthology.orchestra.run.smithi169.stdout: "created": "2023-10-26T01:30:22.897155+0000", 2023-10-26T01:33:00.033 INFO:teuthology.orchestra.run.smithi169.stdout: "modified": "2023-10-26T01:32:59.074259+0000", 2023-10-26T01:33:00.034 INFO:teuthology.orchestra.run.smithi169.stdout: "last_up_change": "2023-10-26T01:30:31.875281+0000", 2023-10-26T01:33:00.034 INFO:teuthology.orchestra.run.smithi169.stdout: "last_in_change": "2023-10-26T01:30:27.637941+0000", 2023-10-26T01:33:00.034 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-26T01:33:00.034 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_num": 5799936, 2023-10-26T01:33:00.034 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_set": [ 2023-10-26T01:33:00.035 INFO:teuthology.orchestra.run.smithi169.stdout: "pglog_hardlimit", 2023-10-26T01:33:00.035 INFO:teuthology.orchestra.run.smithi169.stdout: "purged_snapdirs", 2023-10-26T01:33:00.035 INFO:teuthology.orchestra.run.smithi169.stdout: "recovery_deletes", 2023-10-26T01:33:00.035 INFO:teuthology.orchestra.run.smithi169.stdout: "sortbitwise" 2023-10-26T01:33:00.035 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:33:00.036 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_version": 4, 2023-10-26T01:33:00.036 INFO:teuthology.orchestra.run.smithi169.stdout: "full_ratio": 0.69999998807907104, 2023-10-26T01:33:00.036 INFO:teuthology.orchestra.run.smithi169.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-26T01:33:00.036 INFO:teuthology.orchestra.run.smithi169.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-26T01:33:00.036 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_snapshot": "", 2023-10-26T01:33:00.037 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_max": 7, 2023-10-26T01:33:00.037 INFO:teuthology.orchestra.run.smithi169.stdout: "max_osd": 8, 2023-10-26T01:33:00.037 INFO:teuthology.orchestra.run.smithi169.stdout: "require_min_compat_client": "luminous", 2023-10-26T01:33:00.037 INFO:teuthology.orchestra.run.smithi169.stdout: "min_compat_client": "jewel", 2023-10-26T01:33:00.037 INFO:teuthology.orchestra.run.smithi169.stdout: "require_osd_release": "reef", 2023-10-26T01:33:00.037 INFO:teuthology.orchestra.run.smithi169.stdout: "allow_crimson": false, 2023-10-26T01:33:00.038 INFO:teuthology.orchestra.run.smithi169.stdout: "pools": [ 2023-10-26T01:33:00.038 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:00.038 INFO:teuthology.orchestra.run.smithi169.stdout: "pool": 1, 2023-10-26T01:33:00.038 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_name": ".mgr", 2023-10-26T01:33:00.038 INFO:teuthology.orchestra.run.smithi169.stdout: "create_time": "2023-10-26T01:30:32.200808+0000", 2023-10-26T01:33:00.039 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": 3, 2023-10-26T01:33:00.039 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_names": "hashpspool,full", 2023-10-26T01:33:00.039 INFO:teuthology.orchestra.run.smithi169.stdout: "type": 1, 2023-10-26T01:33:00.039 INFO:teuthology.orchestra.run.smithi169.stdout: "size": 2, 2023-10-26T01:33:00.039 INFO:teuthology.orchestra.run.smithi169.stdout: "min_size": 1, 2023-10-26T01:33:00.040 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_rule": 0, 2023-10-26T01:33:00.040 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_count": 0, 2023-10-26T01:33:00.040 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_target": 0, 2023-10-26T01:33:00.040 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T01:33:00.040 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T01:33:00.040 INFO:teuthology.orchestra.run.smithi169.stdout: "object_hash": 2, 2023-10-26T01:33:00.041 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_mode": "off", 2023-10-26T01:33:00.041 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num": 1, 2023-10-26T01:33:00.041 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num": 1, 2023-10-26T01:33:00.041 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num_target": 1, 2023-10-26T01:33:00.041 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_target": 1, 2023-10-26T01:33:00.042 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_pending": 1, 2023-10-26T01:33:00.042 INFO:teuthology.orchestra.run.smithi169.stdout: "last_pg_merge_meta": { 2023-10-26T01:33:00.042 INFO:teuthology.orchestra.run.smithi169.stdout: "source_pgid": "0.0", 2023-10-26T01:33:00.042 INFO:teuthology.orchestra.run.smithi169.stdout: "ready_epoch": 0, 2023-10-26T01:33:00.042 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_started": 0, 2023-10-26T01:33:00.042 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_clean": 0, 2023-10-26T01:33:00.043 INFO:teuthology.orchestra.run.smithi169.stdout: "source_version": "0'0", 2023-10-26T01:33:00.043 INFO:teuthology.orchestra.run.smithi169.stdout: "target_version": "0'0" 2023-10-26T01:33:00.043 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:00.043 INFO:teuthology.orchestra.run.smithi169.stdout: "last_change": "50", 2023-10-26T01:33:00.043 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend": "0", 2023-10-26T01:33:00.044 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T01:33:00.044 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T01:33:00.044 INFO:teuthology.orchestra.run.smithi169.stdout: "auid": 0, 2023-10-26T01:33:00.044 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_mode": "selfmanaged", 2023-10-26T01:33:00.044 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_seq": 0, 2023-10-26T01:33:00.045 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_epoch": 0, 2023-10-26T01:33:00.045 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_snaps": [], 2023-10-26T01:33:00.045 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps": "[]", 2023-10-26T01:33:00.045 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_bytes": 0, 2023-10-26T01:33:00.045 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_objects": 0, 2023-10-26T01:33:00.045 INFO:teuthology.orchestra.run.smithi169.stdout: "tiers": [], 2023-10-26T01:33:00.046 INFO:teuthology.orchestra.run.smithi169.stdout: "tier_of": -1, 2023-10-26T01:33:00.046 INFO:teuthology.orchestra.run.smithi169.stdout: "read_tier": -1, 2023-10-26T01:33:00.046 INFO:teuthology.orchestra.run.smithi169.stdout: "write_tier": -1, 2023-10-26T01:33:00.046 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_mode": "none", 2023-10-26T01:33:00.046 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_bytes": 0, 2023-10-26T01:33:00.047 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_objects": 0, 2023-10-26T01:33:00.047 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T01:33:00.047 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T01:33:00.047 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T01:33:00.047 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_flush_age": 0, 2023-10-26T01:33:00.048 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_evict_age": 0, 2023-10-26T01:33:00.048 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profile": "", 2023-10-26T01:33:00.048 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_params": { 2023-10-26T01:33:00.048 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "none" 2023-10-26T01:33:00.048 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:00.048 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_period": 0, 2023-10-26T01:33:00.049 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_count": 0, 2023-10-26T01:33:00.049 INFO:teuthology.orchestra.run.smithi169.stdout: "use_gmt_hitset": true, 2023-10-26T01:33:00.049 INFO:teuthology.orchestra.run.smithi169.stdout: "min_read_recency_for_promote": 0, 2023-10-26T01:33:00.049 INFO:teuthology.orchestra.run.smithi169.stdout: "min_write_recency_for_promote": 0, 2023-10-26T01:33:00.049 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T01:33:00.050 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_search_last_n": 0, 2023-10-26T01:33:00.050 INFO:teuthology.orchestra.run.smithi169.stdout: "grade_table": [], 2023-10-26T01:33:00.050 INFO:teuthology.orchestra.run.smithi169.stdout: "stripe_width": 0, 2023-10-26T01:33:00.050 INFO:teuthology.orchestra.run.smithi169.stdout: "expected_num_objects": 0, 2023-10-26T01:33:00.050 INFO:teuthology.orchestra.run.smithi169.stdout: "fast_read": false, 2023-10-26T01:33:00.051 INFO:teuthology.orchestra.run.smithi169.stdout: "options": { 2023-10-26T01:33:00.051 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_max": 32, 2023-10-26T01:33:00.051 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_min": 1 2023-10-26T01:33:00.051 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:00.051 INFO:teuthology.orchestra.run.smithi169.stdout: "application_metadata": { 2023-10-26T01:33:00.051 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": {} 2023-10-26T01:33:00.052 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:00.052 INFO:teuthology.orchestra.run.smithi169.stdout: "read_balance": { 2023-10-26T01:33:00.052 INFO:teuthology.orchestra.run.smithi169.stdout: "score_acting": 8, 2023-10-26T01:33:00.052 INFO:teuthology.orchestra.run.smithi169.stdout: "score_stable": 8, 2023-10-26T01:33:00.052 INFO:teuthology.orchestra.run.smithi169.stdout: "optimal_score": 0.25, 2023-10-26T01:33:00.053 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_acting": 2, 2023-10-26T01:33:00.053 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_stable": 2, 2023-10-26T01:33:00.053 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity_weighted": 1, 2023-10-26T01:33:00.053 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity": 1, 2023-10-26T01:33:00.053 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity_weighted": 1 2023-10-26T01:33:00.053 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:00.054 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:00.054 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:00.054 INFO:teuthology.orchestra.run.smithi169.stdout: "pool": 6, 2023-10-26T01:33:00.054 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_name": "cephfs_metadata", 2023-10-26T01:33:00.054 INFO:teuthology.orchestra.run.smithi169.stdout: "create_time": "2023-10-26T01:32:36.288217+0000", 2023-10-26T01:33:00.055 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": 3, 2023-10-26T01:33:00.055 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_names": "hashpspool,full", 2023-10-26T01:33:00.055 INFO:teuthology.orchestra.run.smithi169.stdout: "type": 1, 2023-10-26T01:33:00.055 INFO:teuthology.orchestra.run.smithi169.stdout: "size": 2, 2023-10-26T01:33:00.056 INFO:teuthology.orchestra.run.smithi169.stdout: "min_size": 1, 2023-10-26T01:33:00.056 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_rule": 0, 2023-10-26T01:33:00.056 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_count": 0, 2023-10-26T01:33:00.056 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_target": 0, 2023-10-26T01:33:00.056 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T01:33:00.057 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T01:33:00.057 INFO:teuthology.orchestra.run.smithi169.stdout: "object_hash": 2, 2023-10-26T01:33:00.057 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_mode": "off", 2023-10-26T01:33:00.057 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num": 32, 2023-10-26T01:33:00.057 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num": 32, 2023-10-26T01:33:00.057 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num_target": 32, 2023-10-26T01:33:00.058 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_target": 32, 2023-10-26T01:33:00.058 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_pending": 32, 2023-10-26T01:33:00.058 INFO:teuthology.orchestra.run.smithi169.stdout: "last_pg_merge_meta": { 2023-10-26T01:33:00.058 INFO:teuthology.orchestra.run.smithi169.stdout: "source_pgid": "0.0", 2023-10-26T01:33:00.059 INFO:teuthology.orchestra.run.smithi169.stdout: "ready_epoch": 0, 2023-10-26T01:33:00.059 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_started": 0, 2023-10-26T01:33:00.059 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_clean": 0, 2023-10-26T01:33:00.059 INFO:teuthology.orchestra.run.smithi169.stdout: "source_version": "0'0", 2023-10-26T01:33:00.059 INFO:teuthology.orchestra.run.smithi169.stdout: "target_version": "0'0" 2023-10-26T01:33:00.059 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:00.060 INFO:teuthology.orchestra.run.smithi169.stdout: "last_change": "50", 2023-10-26T01:33:00.060 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend": "0", 2023-10-26T01:33:00.060 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T01:33:00.060 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T01:33:00.060 INFO:teuthology.orchestra.run.smithi169.stdout: "auid": 0, 2023-10-26T01:33:00.060 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_mode": "selfmanaged", 2023-10-26T01:33:00.061 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_seq": 0, 2023-10-26T01:33:00.061 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_epoch": 0, 2023-10-26T01:33:00.061 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_snaps": [], 2023-10-26T01:33:00.062 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps": "[]", 2023-10-26T01:33:00.062 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_bytes": 0, 2023-10-26T01:33:00.062 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_objects": 0, 2023-10-26T01:33:00.062 INFO:teuthology.orchestra.run.smithi169.stdout: "tiers": [], 2023-10-26T01:33:00.062 INFO:teuthology.orchestra.run.smithi169.stdout: "tier_of": -1, 2023-10-26T01:33:00.063 INFO:teuthology.orchestra.run.smithi169.stdout: "read_tier": -1, 2023-10-26T01:33:00.063 INFO:teuthology.orchestra.run.smithi169.stdout: "write_tier": -1, 2023-10-26T01:33:00.063 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_mode": "none", 2023-10-26T01:33:00.063 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_bytes": 0, 2023-10-26T01:33:00.063 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_objects": 0, 2023-10-26T01:33:00.064 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T01:33:00.064 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T01:33:00.064 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T01:33:00.064 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_flush_age": 0, 2023-10-26T01:33:00.064 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_evict_age": 0, 2023-10-26T01:33:00.064 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profile": "", 2023-10-26T01:33:00.065 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_params": { 2023-10-26T01:33:00.065 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "none" 2023-10-26T01:33:00.065 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:00.065 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_period": 0, 2023-10-26T01:33:00.065 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_count": 0, 2023-10-26T01:33:00.066 INFO:teuthology.orchestra.run.smithi169.stdout: "use_gmt_hitset": true, 2023-10-26T01:33:00.066 INFO:teuthology.orchestra.run.smithi169.stdout: "min_read_recency_for_promote": 0, 2023-10-26T01:33:00.066 INFO:teuthology.orchestra.run.smithi169.stdout: "min_write_recency_for_promote": 0, 2023-10-26T01:33:00.066 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T01:33:00.066 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_search_last_n": 0, 2023-10-26T01:33:00.067 INFO:teuthology.orchestra.run.smithi169.stdout: "grade_table": [], 2023-10-26T01:33:00.067 INFO:teuthology.orchestra.run.smithi169.stdout: "stripe_width": 0, 2023-10-26T01:33:00.067 INFO:teuthology.orchestra.run.smithi169.stdout: "expected_num_objects": 0, 2023-10-26T01:33:00.067 INFO:teuthology.orchestra.run.smithi169.stdout: "fast_read": false, 2023-10-26T01:33:00.067 INFO:teuthology.orchestra.run.smithi169.stdout: "options": { 2023-10-26T01:33:00.068 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_bias": 4, 2023-10-26T01:33:00.068 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_min": 16, 2023-10-26T01:33:00.068 INFO:teuthology.orchestra.run.smithi169.stdout: "recovery_priority": 5 2023-10-26T01:33:00.068 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:00.068 INFO:teuthology.orchestra.run.smithi169.stdout: "application_metadata": { 2023-10-26T01:33:00.069 INFO:teuthology.orchestra.run.smithi169.stdout: "cephfs": { 2023-10-26T01:33:00.069 INFO:teuthology.orchestra.run.smithi169.stdout: "metadata": "cephfs" 2023-10-26T01:33:00.069 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:00.069 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:00.069 INFO:teuthology.orchestra.run.smithi169.stdout: "read_balance": { 2023-10-26T01:33:00.070 INFO:teuthology.orchestra.run.smithi169.stdout: "score_acting": 1.75, 2023-10-26T01:33:00.070 INFO:teuthology.orchestra.run.smithi169.stdout: "score_stable": 1.75, 2023-10-26T01:33:00.070 INFO:teuthology.orchestra.run.smithi169.stdout: "optimal_score": 1, 2023-10-26T01:33:00.070 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_acting": 1.75, 2023-10-26T01:33:00.070 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_stable": 1.75, 2023-10-26T01:33:00.071 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity_weighted": 1, 2023-10-26T01:33:00.071 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity": 1, 2023-10-26T01:33:00.071 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity_weighted": 1 2023-10-26T01:33:00.071 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:00.071 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:00.071 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:00.072 INFO:teuthology.orchestra.run.smithi169.stdout: "pool": 7, 2023-10-26T01:33:00.072 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_name": "cephfs_data", 2023-10-26T01:33:00.072 INFO:teuthology.orchestra.run.smithi169.stdout: "create_time": "2023-10-26T01:32:37.059775+0000", 2023-10-26T01:33:00.072 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": 3, 2023-10-26T01:33:00.072 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_names": "hashpspool,full", 2023-10-26T01:33:00.073 INFO:teuthology.orchestra.run.smithi169.stdout: "type": 1, 2023-10-26T01:33:00.073 INFO:teuthology.orchestra.run.smithi169.stdout: "size": 2, 2023-10-26T01:33:00.073 INFO:teuthology.orchestra.run.smithi169.stdout: "min_size": 1, 2023-10-26T01:33:00.073 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_rule": 0, 2023-10-26T01:33:00.073 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_count": 0, 2023-10-26T01:33:00.073 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_target": 0, 2023-10-26T01:33:00.074 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T01:33:00.074 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T01:33:00.074 INFO:teuthology.orchestra.run.smithi169.stdout: "object_hash": 2, 2023-10-26T01:33:00.074 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_mode": "off", 2023-10-26T01:33:00.074 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num": 64, 2023-10-26T01:33:00.075 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num": 64, 2023-10-26T01:33:00.075 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num_target": 64, 2023-10-26T01:33:00.075 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_target": 64, 2023-10-26T01:33:00.075 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_pending": 64, 2023-10-26T01:33:00.075 INFO:teuthology.orchestra.run.smithi169.stdout: "last_pg_merge_meta": { 2023-10-26T01:33:00.076 INFO:teuthology.orchestra.run.smithi169.stdout: "source_pgid": "0.0", 2023-10-26T01:33:00.076 INFO:teuthology.orchestra.run.smithi169.stdout: "ready_epoch": 0, 2023-10-26T01:33:00.076 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_started": 0, 2023-10-26T01:33:00.076 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_clean": 0, 2023-10-26T01:33:00.076 INFO:teuthology.orchestra.run.smithi169.stdout: "source_version": "0'0", 2023-10-26T01:33:00.076 INFO:teuthology.orchestra.run.smithi169.stdout: "target_version": "0'0" 2023-10-26T01:33:00.077 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:00.077 INFO:teuthology.orchestra.run.smithi169.stdout: "last_change": "50", 2023-10-26T01:33:00.077 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend": "0", 2023-10-26T01:33:00.077 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T01:33:00.077 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T01:33:00.078 INFO:teuthology.orchestra.run.smithi169.stdout: "auid": 0, 2023-10-26T01:33:00.078 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_mode": "selfmanaged", 2023-10-26T01:33:00.078 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_seq": 0, 2023-10-26T01:33:00.078 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_epoch": 0, 2023-10-26T01:33:00.078 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_snaps": [], 2023-10-26T01:33:00.079 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps": "[]", 2023-10-26T01:33:00.079 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_bytes": 0, 2023-10-26T01:33:00.079 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_objects": 0, 2023-10-26T01:33:00.079 INFO:teuthology.orchestra.run.smithi169.stdout: "tiers": [], 2023-10-26T01:33:00.079 INFO:teuthology.orchestra.run.smithi169.stdout: "tier_of": -1, 2023-10-26T01:33:00.079 INFO:teuthology.orchestra.run.smithi169.stdout: "read_tier": -1, 2023-10-26T01:33:00.080 INFO:teuthology.orchestra.run.smithi169.stdout: "write_tier": -1, 2023-10-26T01:33:00.080 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_mode": "none", 2023-10-26T01:33:00.080 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_bytes": 0, 2023-10-26T01:33:00.080 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_objects": 0, 2023-10-26T01:33:00.080 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T01:33:00.081 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T01:33:00.081 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T01:33:00.081 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_flush_age": 0, 2023-10-26T01:33:00.081 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_evict_age": 0, 2023-10-26T01:33:00.081 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profile": "", 2023-10-26T01:33:00.082 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_params": { 2023-10-26T01:33:00.082 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "none" 2023-10-26T01:33:00.082 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:00.082 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_period": 0, 2023-10-26T01:33:00.082 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_count": 0, 2023-10-26T01:33:00.083 INFO:teuthology.orchestra.run.smithi169.stdout: "use_gmt_hitset": true, 2023-10-26T01:33:00.083 INFO:teuthology.orchestra.run.smithi169.stdout: "min_read_recency_for_promote": 0, 2023-10-26T01:33:00.083 INFO:teuthology.orchestra.run.smithi169.stdout: "min_write_recency_for_promote": 0, 2023-10-26T01:33:00.083 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T01:33:00.083 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_search_last_n": 0, 2023-10-26T01:33:00.084 INFO:teuthology.orchestra.run.smithi169.stdout: "grade_table": [], 2023-10-26T01:33:00.084 INFO:teuthology.orchestra.run.smithi169.stdout: "stripe_width": 0, 2023-10-26T01:33:00.084 INFO:teuthology.orchestra.run.smithi169.stdout: "expected_num_objects": 0, 2023-10-26T01:33:00.084 INFO:teuthology.orchestra.run.smithi169.stdout: "fast_read": false, 2023-10-26T01:33:00.084 INFO:teuthology.orchestra.run.smithi169.stdout: "options": { 2023-10-26T01:33:00.084 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_min": 64, 2023-10-26T01:33:00.085 INFO:teuthology.orchestra.run.smithi169.stdout: "target_size_ratio": 0.89999997615814209 2023-10-26T01:33:00.085 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:00.085 INFO:teuthology.orchestra.run.smithi169.stdout: "application_metadata": { 2023-10-26T01:33:00.085 INFO:teuthology.orchestra.run.smithi169.stdout: "cephfs": { 2023-10-26T01:33:00.085 INFO:teuthology.orchestra.run.smithi169.stdout: "data": "cephfs" 2023-10-26T01:33:00.085 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:00.086 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:00.086 INFO:teuthology.orchestra.run.smithi169.stdout: "read_balance": { 2023-10-26T01:33:00.086 INFO:teuthology.orchestra.run.smithi169.stdout: "score_acting": 1.75, 2023-10-26T01:33:00.086 INFO:teuthology.orchestra.run.smithi169.stdout: "score_stable": 1.75, 2023-10-26T01:33:00.086 INFO:teuthology.orchestra.run.smithi169.stdout: "optimal_score": 1, 2023-10-26T01:33:00.087 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_acting": 1.75, 2023-10-26T01:33:00.087 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_stable": 1.75, 2023-10-26T01:33:00.087 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity_weighted": 1, 2023-10-26T01:33:00.087 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity": 1, 2023-10-26T01:33:00.087 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity_weighted": 1 2023-10-26T01:33:00.088 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:00.088 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:00.088 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:33:00.088 INFO:teuthology.orchestra.run.smithi169.stdout: "osds": [ 2023-10-26T01:33:00.088 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:00.088 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 0, 2023-10-26T01:33:00.089 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:33:00.089 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:33:00.089 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:33:00.089 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:33:00.090 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:33:00.090 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:33:00.090 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:33:00.090 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:33:00.090 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 47, 2023-10-26T01:33:00.091 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:33:00.091 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:33:00.091 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:33:00.091 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:00.091 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:00.091 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:00.092 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6816", 2023-10-26T01:33:00.092 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:33:00.092 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:00.092 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:00.092 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:00.093 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6817", 2023-10-26T01:33:00.093 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:33:00.093 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:00.093 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:00.093 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:00.093 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:33:00.094 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:00.094 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:00.094 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:00.094 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6818", 2023-10-26T01:33:00.094 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:33:00.095 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:00.095 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:00.095 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:00.095 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6819", 2023-10-26T01:33:00.095 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:33:00.096 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:00.096 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:00.096 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:00.096 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:33:00.097 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:00.097 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:00.097 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:00.097 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6822", 2023-10-26T01:33:00.097 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:33:00.098 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:00.098 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:00.098 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:00.098 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6823", 2023-10-26T01:33:00.098 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:33:00.099 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:00.099 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:00.099 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:00.099 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:33:00.099 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:00.099 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:00.100 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:00.100 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6820", 2023-10-26T01:33:00.100 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:33:00.100 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:00.100 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:00.101 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:00.101 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6821", 2023-10-26T01:33:00.101 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:33:00.101 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:00.101 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:00.102 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:00.102 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6817/4204255864", 2023-10-26T01:33:00.102 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6819/4204255864", 2023-10-26T01:33:00.102 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6823/4204255864", 2023-10-26T01:33:00.102 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6821/4204255864", 2023-10-26T01:33:00.103 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:33:00.103 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:33:00.103 INFO:teuthology.orchestra.run.smithi169.stdout: "full", 2023-10-26T01:33:00.103 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:33:00.103 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:00.104 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:00.104 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:00.104 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 1, 2023-10-26T01:33:00.104 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:33:00.104 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:33:00.105 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:33:00.105 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:33:00.105 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:33:00.105 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:33:00.105 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:33:00.105 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:33:00.106 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 47, 2023-10-26T01:33:00.106 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:33:00.106 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:33:00.106 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:33:00.106 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:00.107 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:00.107 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:00.107 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6808", 2023-10-26T01:33:00.107 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:33:00.107 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:00.108 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:00.108 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:00.108 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6809", 2023-10-26T01:33:00.108 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:33:00.108 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:00.108 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:00.109 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:00.109 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:33:00.109 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:00.109 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:00.109 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:00.109 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6810", 2023-10-26T01:33:00.110 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:33:00.110 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:00.110 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:00.110 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:00.110 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6811", 2023-10-26T01:33:00.111 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:33:00.111 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:00.111 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:00.111 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:00.111 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:33:00.111 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:00.112 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:00.112 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:00.112 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6814", 2023-10-26T01:33:00.112 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:33:00.112 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:00.113 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:00.113 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:00.113 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6815", 2023-10-26T01:33:00.113 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:33:00.113 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:00.114 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:00.114 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:00.114 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:33:00.114 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:00.114 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:00.114 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:00.115 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6812", 2023-10-26T01:33:00.115 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:33:00.115 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:00.115 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:00.115 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:00.116 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6813", 2023-10-26T01:33:00.116 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:33:00.116 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:00.116 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:00.116 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:00.117 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6809/238206046", 2023-10-26T01:33:00.117 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6811/238206046", 2023-10-26T01:33:00.117 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6815/238206046", 2023-10-26T01:33:00.117 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6813/238206046", 2023-10-26T01:33:00.117 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:33:00.118 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:33:00.118 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:33:00.118 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:00.118 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:00.118 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:00.119 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 2, 2023-10-26T01:33:00.119 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:33:00.119 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:33:00.119 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:33:00.119 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:33:00.120 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:33:00.120 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:33:00.120 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:33:00.120 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:33:00.120 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 47, 2023-10-26T01:33:00.121 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:33:00.121 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:33:00.121 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:33:00.121 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:00.121 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:00.121 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:00.122 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6800", 2023-10-26T01:33:00.122 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:33:00.122 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:00.122 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:00.122 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:00.123 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6801", 2023-10-26T01:33:00.123 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:33:00.123 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:00.123 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:00.123 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:00.124 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:33:00.124 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:00.124 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:00.124 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:00.124 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6802", 2023-10-26T01:33:00.125 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:33:00.125 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:00.125 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:00.125 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:00.125 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6803", 2023-10-26T01:33:00.126 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:33:00.126 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:00.126 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:00.126 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:00.126 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:33:00.126 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:00.127 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:00.127 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:00.127 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6806", 2023-10-26T01:33:00.127 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:33:00.127 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:00.128 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:00.128 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:00.128 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6807", 2023-10-26T01:33:00.128 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:33:00.128 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:00.128 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:00.129 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:00.129 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:33:00.129 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:00.129 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:00.129 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:00.130 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6804", 2023-10-26T01:33:00.130 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:33:00.130 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:00.130 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:00.130 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:00.131 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6805", 2023-10-26T01:33:00.131 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:33:00.131 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:00.131 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:00.131 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:00.132 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6801/3215791846", 2023-10-26T01:33:00.132 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6803/3215791846", 2023-10-26T01:33:00.132 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6807/3215791846", 2023-10-26T01:33:00.132 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6805/3215791846", 2023-10-26T01:33:00.132 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:33:00.132 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:33:00.133 INFO:teuthology.orchestra.run.smithi169.stdout: "full", 2023-10-26T01:33:00.133 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:33:00.133 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:00.133 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:00.133 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:00.134 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 3, 2023-10-26T01:33:00.134 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:33:00.134 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:33:00.134 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:33:00.134 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:33:00.135 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:33:00.135 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:33:00.135 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:33:00.135 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 14, 2023-10-26T01:33:00.135 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 47, 2023-10-26T01:33:00.135 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:33:00.136 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:33:00.136 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:33:00.136 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:00.136 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:00.136 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:00.137 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6824", 2023-10-26T01:33:00.137 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:33:00.137 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:00.137 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:00.137 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:00.138 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6825", 2023-10-26T01:33:00.138 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:33:00.138 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:00.138 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:00.138 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:00.139 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:33:00.139 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:00.139 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:00.139 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:00.139 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6826", 2023-10-26T01:33:00.139 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:33:00.140 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:00.140 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:00.140 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:00.140 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6827", 2023-10-26T01:33:00.141 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:33:00.141 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:00.141 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:00.141 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:00.141 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:33:00.141 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:00.142 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:00.142 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:00.142 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6830", 2023-10-26T01:33:00.142 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:33:00.142 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:00.143 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:00.143 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:00.143 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6831", 2023-10-26T01:33:00.143 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:33:00.143 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:00.143 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:00.144 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:00.144 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:33:00.144 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:00.144 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:00.144 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:00.145 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6828", 2023-10-26T01:33:00.145 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:33:00.145 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:00.145 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:00.145 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:00.145 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6829", 2023-10-26T01:33:00.146 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:33:00.146 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:00.146 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:00.146 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:00.146 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6825/2256701226", 2023-10-26T01:33:00.147 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6827/2256701226", 2023-10-26T01:33:00.147 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6831/2256701226", 2023-10-26T01:33:00.147 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6829/2256701226", 2023-10-26T01:33:00.147 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:33:00.147 INFO:teuthology.orchestra.run.smithi169.stdout: "backfillfull", 2023-10-26T01:33:00.148 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:33:00.148 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:33:00.148 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:00.148 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:00.148 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:00.149 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 4, 2023-10-26T01:33:00.149 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:33:00.149 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:33:00.149 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:33:00.149 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:33:00.149 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:33:00.150 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:33:00.150 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:33:00.150 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:33:00.150 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 47, 2023-10-26T01:33:00.150 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:33:00.151 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:33:00.151 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:33:00.151 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:00.151 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:00.151 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:00.152 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6816", 2023-10-26T01:33:00.152 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:33:00.152 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:00.152 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:00.152 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:00.153 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6817", 2023-10-26T01:33:00.153 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:33:00.153 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:00.153 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:00.153 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:00.153 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:33:00.154 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:00.154 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:00.154 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:00.154 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6818", 2023-10-26T01:33:00.154 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:33:00.155 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:00.155 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:00.155 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:00.155 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6819", 2023-10-26T01:33:00.155 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:33:00.156 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:00.156 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:00.156 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:00.156 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:33:00.156 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:00.157 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:00.157 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:00.157 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6825", 2023-10-26T01:33:00.157 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:33:00.157 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:00.157 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:00.158 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:00.158 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6828", 2023-10-26T01:33:00.158 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:33:00.158 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:00.158 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:00.159 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:00.159 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:33:00.159 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:00.159 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:00.159 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:00.160 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6820", 2023-10-26T01:33:00.160 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:33:00.160 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:00.160 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:00.160 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:00.161 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6823", 2023-10-26T01:33:00.161 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:33:00.161 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:00.161 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:00.161 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:00.162 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6817/1082279874", 2023-10-26T01:33:00.162 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6819/1082279874", 2023-10-26T01:33:00.162 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6828/1082279874", 2023-10-26T01:33:00.162 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6823/1082279874", 2023-10-26T01:33:00.162 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:33:00.163 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:33:00.163 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:33:00.163 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:00.163 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:00.163 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:00.163 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 5, 2023-10-26T01:33:00.164 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:33:00.164 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:33:00.164 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:33:00.164 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:33:00.164 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:33:00.165 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:33:00.165 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:33:00.165 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:33:00.165 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 47, 2023-10-26T01:33:00.165 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:33:00.165 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:33:00.166 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:33:00.166 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:00.166 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:00.166 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:00.166 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6808", 2023-10-26T01:33:00.167 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:33:00.167 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:00.167 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:00.167 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:00.167 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6809", 2023-10-26T01:33:00.168 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:33:00.168 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:00.168 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:00.168 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:00.168 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:33:00.169 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:00.169 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:00.169 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:00.169 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6810", 2023-10-26T01:33:00.169 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:33:00.169 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:00.170 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:00.170 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:00.170 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6811", 2023-10-26T01:33:00.170 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:33:00.170 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:00.171 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:00.171 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:00.171 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:33:00.171 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:00.171 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:00.172 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:00.172 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6814", 2023-10-26T01:33:00.172 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:33:00.172 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:00.172 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:00.173 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:00.173 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6815", 2023-10-26T01:33:00.173 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:33:00.173 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:00.173 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:00.173 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:00.174 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:33:00.174 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:00.174 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:00.174 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:00.174 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6812", 2023-10-26T01:33:00.175 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:33:00.175 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:00.175 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:00.175 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:00.175 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6813", 2023-10-26T01:33:00.175 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:33:00.176 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:00.176 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:00.176 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:00.176 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6809/3335654351", 2023-10-26T01:33:00.176 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6811/3335654351", 2023-10-26T01:33:00.177 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6815/3335654351", 2023-10-26T01:33:00.177 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6813/3335654351", 2023-10-26T01:33:00.177 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:33:00.177 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:33:00.177 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:33:00.177 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:00.178 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:00.178 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:00.178 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 6, 2023-10-26T01:33:00.178 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:33:00.178 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:33:00.179 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:33:00.179 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:33:00.179 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:33:00.179 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:33:00.179 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:33:00.180 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:33:00.180 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 47, 2023-10-26T01:33:00.180 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:33:00.180 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:33:00.180 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:33:00.180 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:00.181 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:00.181 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:00.181 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6800", 2023-10-26T01:33:00.181 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:33:00.181 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:00.182 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:00.182 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:00.182 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6801", 2023-10-26T01:33:00.183 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:33:00.183 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:00.183 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:00.183 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:00.183 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:33:00.184 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:00.184 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:00.184 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:00.184 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6802", 2023-10-26T01:33:00.184 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:33:00.184 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:00.185 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:00.185 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:00.185 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6803", 2023-10-26T01:33:00.185 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:33:00.185 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:00.186 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:00.186 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:00.186 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:33:00.186 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:00.186 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:00.187 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:00.187 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6806", 2023-10-26T01:33:00.187 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:33:00.187 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:00.187 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:00.188 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:00.188 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6807", 2023-10-26T01:33:00.188 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:33:00.188 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:00.188 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:00.188 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:00.189 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:33:00.189 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:00.189 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:00.189 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:00.189 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6804", 2023-10-26T01:33:00.190 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:33:00.190 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:00.190 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:00.190 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:00.190 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6805", 2023-10-26T01:33:00.190 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:33:00.191 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:00.191 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:00.191 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:00.191 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6801/2520460697", 2023-10-26T01:33:00.191 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6803/2520460697", 2023-10-26T01:33:00.192 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6807/2520460697", 2023-10-26T01:33:00.192 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6805/2520460697", 2023-10-26T01:33:00.192 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:33:00.192 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:33:00.192 INFO:teuthology.orchestra.run.smithi169.stdout: "full", 2023-10-26T01:33:00.192 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:33:00.193 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:00.193 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:00.193 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:00.193 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 7, 2023-10-26T01:33:00.193 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:33:00.193 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:33:00.194 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:33:00.194 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:33:00.194 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:33:00.194 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:33:00.194 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:33:00.195 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:33:00.195 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 47, 2023-10-26T01:33:00.195 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:33:00.195 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:33:00.195 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:33:00.196 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:00.196 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:00.196 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:00.196 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6821", 2023-10-26T01:33:00.196 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:33:00.197 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:00.197 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:00.197 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:00.197 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6822", 2023-10-26T01:33:00.197 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:33:00.197 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:00.198 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:00.198 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:00.198 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:33:00.198 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:00.198 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:00.199 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:00.199 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6824", 2023-10-26T01:33:00.199 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:33:00.199 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:00.199 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:00.200 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:00.200 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6826", 2023-10-26T01:33:00.200 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:33:00.200 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:00.200 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:00.200 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:00.201 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:33:00.201 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:00.201 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:00.201 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:00.201 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6830", 2023-10-26T01:33:00.202 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:33:00.202 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:00.202 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:00.202 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:00.202 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6831", 2023-10-26T01:33:00.203 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:33:00.203 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:00.203 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:00.203 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:00.203 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:33:00.204 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:00.204 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:00.204 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:00.204 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6827", 2023-10-26T01:33:00.204 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:33:00.204 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:00.205 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:00.205 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:00.205 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6829", 2023-10-26T01:33:00.205 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:33:00.205 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:00.205 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:00.206 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:00.206 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6822/1115930619", 2023-10-26T01:33:00.206 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6826/1115930619", 2023-10-26T01:33:00.206 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6831/1115930619", 2023-10-26T01:33:00.206 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6829/1115930619", 2023-10-26T01:33:00.207 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:33:00.207 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:33:00.207 INFO:teuthology.orchestra.run.smithi169.stdout: "full", 2023-10-26T01:33:00.207 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:33:00.207 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:00.208 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:00.208 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:33:00.208 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_xinfo": [ 2023-10-26T01:33:00.208 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:00.208 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 0, 2023-10-26T01:33:00.208 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:33:00.209 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:33:00.209 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:33:00.209 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:33:00.209 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:33:00.209 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.889675+0000", 2023-10-26T01:33:00.210 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:33:00.210 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:00.210 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:00.210 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 1, 2023-10-26T01:33:00.210 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:33:00.211 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:33:00.211 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:33:00.211 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:33:00.211 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:33:00.211 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.864157+0000", 2023-10-26T01:33:00.211 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:33:00.212 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:00.212 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:00.212 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 2, 2023-10-26T01:33:00.212 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:33:00.212 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:33:00.212 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:33:00.213 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:33:00.213 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:33:00.213 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.876919+0000", 2023-10-26T01:33:00.213 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:33:00.213 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:00.214 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:00.214 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 3, 2023-10-26T01:33:00.214 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:33:00.214 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:33:00.214 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:33:00.215 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:33:00.215 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:33:00.215 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.974541+0000", 2023-10-26T01:33:00.215 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:33:00.215 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:00.215 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:00.216 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 4, 2023-10-26T01:33:00.216 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:33:00.216 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:33:00.216 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:33:00.216 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:33:00.216 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:33:00.217 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.969696+0000", 2023-10-26T01:33:00.217 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:33:00.217 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:00.217 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:00.217 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 5, 2023-10-26T01:33:00.218 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:33:00.218 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:33:00.218 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:33:00.218 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:33:00.218 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:33:00.219 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.859703+0000", 2023-10-26T01:33:00.219 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:33:00.219 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:00.219 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:00.219 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 6, 2023-10-26T01:33:00.220 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:33:00.220 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:33:00.220 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:33:00.220 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:33:00.220 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:33:00.220 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.867917+0000", 2023-10-26T01:33:00.221 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:33:00.221 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:00.221 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:00.221 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 7, 2023-10-26T01:33:00.221 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:33:00.222 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:33:00.222 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:33:00.222 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:33:00.222 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:33:00.222 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.898605+0000", 2023-10-26T01:33:00.222 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:33:00.223 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:00.223 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:33:00.223 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_upmap": [], 2023-10-26T01:33:00.223 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_upmap_items": [], 2023-10-26T01:33:00.223 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_upmap_primaries": [], 2023-10-26T01:33:00.224 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_temp": [], 2023-10-26T01:33:00.224 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_temp": [], 2023-10-26T01:33:00.224 INFO:teuthology.orchestra.run.smithi169.stdout: "blocklist": {}, 2023-10-26T01:33:00.224 INFO:teuthology.orchestra.run.smithi169.stdout: "range_blocklist": {}, 2023-10-26T01:33:00.224 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profiles": { 2023-10-26T01:33:00.225 INFO:teuthology.orchestra.run.smithi169.stdout: "default": { 2023-10-26T01:33:00.225 INFO:teuthology.orchestra.run.smithi169.stdout: "crush-failure-domain": "osd", 2023-10-26T01:33:00.225 INFO:teuthology.orchestra.run.smithi169.stdout: "k": "2", 2023-10-26T01:33:00.225 INFO:teuthology.orchestra.run.smithi169.stdout: "m": "1", 2023-10-26T01:33:00.225 INFO:teuthology.orchestra.run.smithi169.stdout: "plugin": "jerasure", 2023-10-26T01:33:00.225 INFO:teuthology.orchestra.run.smithi169.stdout: "technique": "reed_sol_van" 2023-10-26T01:33:00.226 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:00.226 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:00.226 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps_queue": [], 2023-10-26T01:33:00.226 INFO:teuthology.orchestra.run.smithi169.stdout: "new_removed_snaps": [], 2023-10-26T01:33:00.226 INFO:teuthology.orchestra.run.smithi169.stdout: "new_purged_snaps": [], 2023-10-26T01:33:00.227 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_node_flags": {}, 2023-10-26T01:33:00.227 INFO:teuthology.orchestra.run.smithi169.stdout: "device_class_flags": {}, 2023-10-26T01:33:00.227 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_mode": { 2023-10-26T01:33:00.227 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_mode_enabled": false, 2023-10-26T01:33:00.227 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_bucket_count": 0, 2023-10-26T01:33:00.228 INFO:teuthology.orchestra.run.smithi169.stdout: "degraded_stretch_mode": 0, 2023-10-26T01:33:00.228 INFO:teuthology.orchestra.run.smithi169.stdout: "recovering_stretch_mode": 0, 2023-10-26T01:33:00.228 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_mode_bucket": 0 2023-10-26T01:33:00.228 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:00.228 INFO:teuthology.orchestra.run.smithi169.stdout:} 2023-10-26T01:33:00.230 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:00.032+0000 7fce9fde7640 1 -- 172.21.15.169:0/1669530505 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fce7c06ae40 msgr2=0x7fce7c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:00.230 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:00.033+0000 7fce9fde7640 1 --2- 172.21.15.169:0/1669530505 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fce7c06ae40 0x7fce7c06d300 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fce98158af0 tx=0x7fce94006ba0 comp rx=0 tx=0).stop 2023-10-26T01:33:00.230 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:00.033+0000 7fce9fde7640 1 -- 172.21.15.169:0/1669530505 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce980cdd80 msgr2=0x7fce980ace10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:00.231 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:00.033+0000 7fce9fde7640 1 --2- 172.21.15.169:0/1669530505 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce980cdd80 0x7fce980ace10 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fce8800c450 tx=0x7fce8800c910 comp rx=0 tx=0).stop 2023-10-26T01:33:00.231 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:00.033+0000 7fce9fde7640 1 -- 172.21.15.169:0/1669530505 shutdown_connections 2023-10-26T01:33:00.231 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:00.033+0000 7fce9fde7640 1 --2- 172.21.15.169:0/1669530505 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fce7c06ae40 0x7fce7c06d300 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:00.231 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:00.033+0000 7fce9fde7640 1 --2- 172.21.15.169:0/1669530505 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce980cdd80 0x7fce980ace10 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:00.231 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:00.033+0000 7fce9fde7640 1 --2- 172.21.15.169:0/1669530505 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fce9815b610 0x7fce980ad350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:00.232 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:00.033+0000 7fce9fde7640 1 --2- 172.21.15.169:0/1669530505 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce9806ed30 0x7fce9814e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:00.232 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:00.033+0000 7fce9fde7640 1 -- 172.21.15.169:0/1669530505 >> 172.21.15.169:0/1669530505 conn(0x7fce98069930 msgr2=0x7fce98070ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:00.232 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:00.033+0000 7fce9fde7640 1 -- 172.21.15.169:0/1669530505 shutdown_connections 2023-10-26T01:33:00.232 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:00.033+0000 7fce9fde7640 1 -- 172.21.15.169:0/1669530505 wait complete. 2023-10-26T01:33:00.233 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-10-26T01:33:00.233 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:33:00.234 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:33:00.296 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:33:00.296 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=50 seek=265) 2023-10-26T01:33:00.748 INFO:teuthology.orchestra.run.smithi169.stderr:dd: fdatasync failed for 'large_file_b': No space left on device 2023-10-26T01:33:00.748 INFO:teuthology.orchestra.run.smithi169.stderr:50+0 records in 2023-10-26T01:33:00.748 INFO:teuthology.orchestra.run.smithi169.stderr:50+0 records out 2023-10-26T01:33:00.748 INFO:teuthology.orchestra.run.smithi169.stderr:52428800 bytes (52 MB, 50 MiB) copied, 0.381071 s, 138 MB/s 2023-10-26T01:33:00.749 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-26T01:33:00.750 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-26T01:33:00.960 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:00.958+0000 7fd2acea2640 1 -- 172.21.15.169:0/467167299 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd290003970 msgr2=0x7fd290003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:00.960 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:00.958+0000 7fd2acea2640 1 --2- 172.21.15.169:0/467167299 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd290003970 0x7fd290003d90 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fd298009b50 tx=0x7fd2900057f0 comp rx=0 tx=0).stop 2023-10-26T01:33:00.960 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:00.958+0000 7fd2acea2640 1 -- 172.21.15.169:0/467167299 shutdown_connections 2023-10-26T01:33:00.961 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:00.958+0000 7fd2acea2640 1 --2- 172.21.15.169:0/467167299 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd290003970 0x7fd290003d90 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:00.961 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:00.958+0000 7fd2acea2640 1 -- 172.21.15.169:0/467167299 >> 172.21.15.169:0/467167299 conn(0x7fd2a8069930 msgr2=0x7fd2a8069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:00.961 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:00.958+0000 7fd2acea2640 1 -- 172.21.15.169:0/467167299 shutdown_connections 2023-10-26T01:33:00.961 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:00.958+0000 7fd2acea2640 1 -- 172.21.15.169:0/467167299 wait complete. 2023-10-26T01:33:00.962 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:00.960+0000 7fd2acea2640 1 Processor -- start 2023-10-26T01:33:00.962 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:00.960+0000 7fd2acea2640 1 -- start start 2023-10-26T01:33:00.963 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:00.961+0000 7fd2acea2640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd290003970 0x7fd2a81f1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:00.964 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:00.961+0000 7fd2acea2640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd2a8152300 0x7fd2a81559a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:00.964 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:00.961+0000 7fd2acea2640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd2a80af680 0x7fd2a8155ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:00.964 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:00.961+0000 7fd2acea2640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a806feb0 con 0x7fd290003970 2023-10-26T01:33:00.964 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:00.961+0000 7fd2acea2640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a81f1d80 con 0x7fd2a8152300 2023-10-26T01:33:00.964 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:00.961+0000 7fd2acea2640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a8154f30 con 0x7fd2a80af680 2023-10-26T01:33:00.965 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:00.961+0000 7fd2a659c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd290003970 0x7fd2a81f1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:00.965 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:00.961+0000 7fd2a659c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd290003970 0x7fd2a81f1840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:58148/0 (socket says 172.21.15.169:58148) 2023-10-26T01:33:00.965 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:00.961+0000 7fd2a659c640 1 -- 172.21.15.169:0/1459065942 learned_addr learned my addr 172.21.15.169:0/1459065942 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:33:00.965 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:00.962+0000 7fd2a659c640 1 -- 172.21.15.169:0/1459065942 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd2a80af680 msgr2=0x7fd2a8155ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:00.965 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:00.962+0000 7fd2a659c640 1 --2- 172.21.15.169:0/1459065942 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd2a80af680 0x7fd2a8155ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:00.966 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:00.962+0000 7fd2a659c640 1 -- 172.21.15.169:0/1459065942 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd2a8152300 msgr2=0x7fd2a81559a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:00.966 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:00.962+0000 7fd2a659c640 1 --2- 172.21.15.169:0/1459065942 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd2a8152300 0x7fd2a81559a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:00.966 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:00.962+0000 7fd2a659c640 1 -- 172.21.15.169:0/1459065942 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2a8155210 con 0x7fd290003970 2023-10-26T01:33:00.966 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:00.962+0000 7fd2a659c640 1 --2- 172.21.15.169:0/1459065942 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd290003970 0x7fd2a81f1840 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fd298009b50 tx=0x7fd298033bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:00.968 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:00.962+0000 7fd296ffd640 1 -- 172.21.15.169:0/1459065942 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd298004230 con 0x7fd290003970 2023-10-26T01:33:00.968 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:00.962+0000 7fd296ffd640 1 -- 172.21.15.169:0/1459065942 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd298004390 con 0x7fd290003970 2023-10-26T01:33:00.968 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:00.962+0000 7fd296ffd640 1 -- 172.21.15.169:0/1459065942 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd298003850 con 0x7fd290003970 2023-10-26T01:33:00.968 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:00.962+0000 7fd2acea2640 1 -- 172.21.15.169:0/1459065942 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2a81554f0 con 0x7fd290003970 2023-10-26T01:33:00.968 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:00.963+0000 7fd2acea2640 1 -- 172.21.15.169:0/1459065942 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2a8156680 con 0x7fd290003970 2023-10-26T01:33:00.969 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:00.964+0000 7fd2acea2640 1 -- 172.21.15.169:0/1459065942 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2a804db10 con 0x7fd290003970 2023-10-26T01:33:00.969 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:00.965+0000 7fd296ffd640 1 -- 172.21.15.169:0/1459065942 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fd2980039b0 con 0x7fd290003970 2023-10-26T01:33:00.969 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:00.966+0000 7fd296ffd640 1 --2- 172.21.15.169:0/1459065942 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd28806ae40 0x7fd28806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:00.969 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:00.966+0000 7fd296ffd640 1 -- 172.21.15.169:0/1459065942 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd2980ac220 con 0x7fd290003970 2023-10-26T01:33:00.969 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:00.966+0000 7fd296ffd640 1 -- 172.21.15.169:0/1459065942 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=52}) v3 -- 0x7fd288072970 con 0x7fd290003970 2023-10-26T01:33:00.970 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:00.966+0000 7fd2a5d9b640 1 --2- 172.21.15.169:0/1459065942 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd28806ae40 0x7fd28806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:00.970 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:00.968+0000 7fd2a5d9b640 1 --2- 172.21.15.169:0/1459065942 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd28806ae40 0x7fd28806d300 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd2a8157b90 tx=0x7fd29c0079b0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:00.971 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:00.970+0000 7fd296ffd640 1 -- 172.21.15.169:0/1459065942 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd298041060 con 0x7fd290003970 2023-10-26T01:33:01.163 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:01.161+0000 7fd2acea2640 1 -- 172.21.15.169:0/1459065942 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fd2a80aca70 con 0x7fd290003970 2023-10-26T01:33:01.166 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:01.165+0000 7fd296ffd640 1 -- 172.21.15.169:0/1459065942 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v51) v1 ==== 81+0+34139 (secure 0 0 0) 0x7fd29804b070 con 0x7fd290003970 2023-10-26T01:33:01.167 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:33:01.167 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2023-10-26T01:33:01.167 INFO:teuthology.orchestra.run.smithi169.stdout: "epoch": 51, 2023-10-26T01:33:01.168 INFO:teuthology.orchestra.run.smithi169.stdout: "fsid": "30203b28-ed8a-44e8-a9a6-a9bf25f980f1", 2023-10-26T01:33:01.168 INFO:teuthology.orchestra.run.smithi169.stdout: "created": "2023-10-26T01:30:22.897155+0000", 2023-10-26T01:33:01.168 INFO:teuthology.orchestra.run.smithi169.stdout: "modified": "2023-10-26T01:32:59.074259+0000", 2023-10-26T01:33:01.168 INFO:teuthology.orchestra.run.smithi169.stdout: "last_up_change": "2023-10-26T01:30:31.875281+0000", 2023-10-26T01:33:01.168 INFO:teuthology.orchestra.run.smithi169.stdout: "last_in_change": "2023-10-26T01:30:27.637941+0000", 2023-10-26T01:33:01.169 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-26T01:33:01.169 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_num": 5799936, 2023-10-26T01:33:01.169 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_set": [ 2023-10-26T01:33:01.169 INFO:teuthology.orchestra.run.smithi169.stdout: "pglog_hardlimit", 2023-10-26T01:33:01.169 INFO:teuthology.orchestra.run.smithi169.stdout: "purged_snapdirs", 2023-10-26T01:33:01.170 INFO:teuthology.orchestra.run.smithi169.stdout: "recovery_deletes", 2023-10-26T01:33:01.170 INFO:teuthology.orchestra.run.smithi169.stdout: "sortbitwise" 2023-10-26T01:33:01.170 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:33:01.170 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_version": 4, 2023-10-26T01:33:01.170 INFO:teuthology.orchestra.run.smithi169.stdout: "full_ratio": 0.69999998807907104, 2023-10-26T01:33:01.171 INFO:teuthology.orchestra.run.smithi169.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-26T01:33:01.171 INFO:teuthology.orchestra.run.smithi169.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-26T01:33:01.171 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_snapshot": "", 2023-10-26T01:33:01.171 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_max": 7, 2023-10-26T01:33:01.171 INFO:teuthology.orchestra.run.smithi169.stdout: "max_osd": 8, 2023-10-26T01:33:01.171 INFO:teuthology.orchestra.run.smithi169.stdout: "require_min_compat_client": "luminous", 2023-10-26T01:33:01.172 INFO:teuthology.orchestra.run.smithi169.stdout: "min_compat_client": "jewel", 2023-10-26T01:33:01.172 INFO:teuthology.orchestra.run.smithi169.stdout: "require_osd_release": "reef", 2023-10-26T01:33:01.172 INFO:teuthology.orchestra.run.smithi169.stdout: "allow_crimson": false, 2023-10-26T01:33:01.172 INFO:teuthology.orchestra.run.smithi169.stdout: "pools": [ 2023-10-26T01:33:01.172 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:01.173 INFO:teuthology.orchestra.run.smithi169.stdout: "pool": 1, 2023-10-26T01:33:01.173 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_name": ".mgr", 2023-10-26T01:33:01.173 INFO:teuthology.orchestra.run.smithi169.stdout: "create_time": "2023-10-26T01:30:32.200808+0000", 2023-10-26T01:33:01.173 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": 3, 2023-10-26T01:33:01.173 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_names": "hashpspool,full", 2023-10-26T01:33:01.173 INFO:teuthology.orchestra.run.smithi169.stdout: "type": 1, 2023-10-26T01:33:01.174 INFO:teuthology.orchestra.run.smithi169.stdout: "size": 2, 2023-10-26T01:33:01.174 INFO:teuthology.orchestra.run.smithi169.stdout: "min_size": 1, 2023-10-26T01:33:01.174 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_rule": 0, 2023-10-26T01:33:01.174 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_count": 0, 2023-10-26T01:33:01.174 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_target": 0, 2023-10-26T01:33:01.175 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T01:33:01.175 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T01:33:01.175 INFO:teuthology.orchestra.run.smithi169.stdout: "object_hash": 2, 2023-10-26T01:33:01.175 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_mode": "off", 2023-10-26T01:33:01.175 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num": 1, 2023-10-26T01:33:01.175 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num": 1, 2023-10-26T01:33:01.176 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num_target": 1, 2023-10-26T01:33:01.176 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_target": 1, 2023-10-26T01:33:01.176 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_pending": 1, 2023-10-26T01:33:01.176 INFO:teuthology.orchestra.run.smithi169.stdout: "last_pg_merge_meta": { 2023-10-26T01:33:01.176 INFO:teuthology.orchestra.run.smithi169.stdout: "source_pgid": "0.0", 2023-10-26T01:33:01.177 INFO:teuthology.orchestra.run.smithi169.stdout: "ready_epoch": 0, 2023-10-26T01:33:01.177 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_started": 0, 2023-10-26T01:33:01.177 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_clean": 0, 2023-10-26T01:33:01.177 INFO:teuthology.orchestra.run.smithi169.stdout: "source_version": "0'0", 2023-10-26T01:33:01.177 INFO:teuthology.orchestra.run.smithi169.stdout: "target_version": "0'0" 2023-10-26T01:33:01.177 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:01.178 INFO:teuthology.orchestra.run.smithi169.stdout: "last_change": "50", 2023-10-26T01:33:01.178 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend": "0", 2023-10-26T01:33:01.178 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T01:33:01.178 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T01:33:01.178 INFO:teuthology.orchestra.run.smithi169.stdout: "auid": 0, 2023-10-26T01:33:01.179 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_mode": "selfmanaged", 2023-10-26T01:33:01.179 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_seq": 0, 2023-10-26T01:33:01.179 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_epoch": 0, 2023-10-26T01:33:01.179 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_snaps": [], 2023-10-26T01:33:01.179 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps": "[]", 2023-10-26T01:33:01.179 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_bytes": 0, 2023-10-26T01:33:01.180 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_objects": 0, 2023-10-26T01:33:01.180 INFO:teuthology.orchestra.run.smithi169.stdout: "tiers": [], 2023-10-26T01:33:01.180 INFO:teuthology.orchestra.run.smithi169.stdout: "tier_of": -1, 2023-10-26T01:33:01.180 INFO:teuthology.orchestra.run.smithi169.stdout: "read_tier": -1, 2023-10-26T01:33:01.180 INFO:teuthology.orchestra.run.smithi169.stdout: "write_tier": -1, 2023-10-26T01:33:01.181 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_mode": "none", 2023-10-26T01:33:01.181 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_bytes": 0, 2023-10-26T01:33:01.181 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_objects": 0, 2023-10-26T01:33:01.181 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T01:33:01.181 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T01:33:01.182 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T01:33:01.182 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_flush_age": 0, 2023-10-26T01:33:01.182 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_evict_age": 0, 2023-10-26T01:33:01.182 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profile": "", 2023-10-26T01:33:01.182 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_params": { 2023-10-26T01:33:01.182 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "none" 2023-10-26T01:33:01.183 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:01.183 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_period": 0, 2023-10-26T01:33:01.183 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_count": 0, 2023-10-26T01:33:01.183 INFO:teuthology.orchestra.run.smithi169.stdout: "use_gmt_hitset": true, 2023-10-26T01:33:01.183 INFO:teuthology.orchestra.run.smithi169.stdout: "min_read_recency_for_promote": 0, 2023-10-26T01:33:01.184 INFO:teuthology.orchestra.run.smithi169.stdout: "min_write_recency_for_promote": 0, 2023-10-26T01:33:01.184 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T01:33:01.184 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_search_last_n": 0, 2023-10-26T01:33:01.184 INFO:teuthology.orchestra.run.smithi169.stdout: "grade_table": [], 2023-10-26T01:33:01.184 INFO:teuthology.orchestra.run.smithi169.stdout: "stripe_width": 0, 2023-10-26T01:33:01.185 INFO:teuthology.orchestra.run.smithi169.stdout: "expected_num_objects": 0, 2023-10-26T01:33:01.185 INFO:teuthology.orchestra.run.smithi169.stdout: "fast_read": false, 2023-10-26T01:33:01.185 INFO:teuthology.orchestra.run.smithi169.stdout: "options": { 2023-10-26T01:33:01.185 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_max": 32, 2023-10-26T01:33:01.185 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_min": 1 2023-10-26T01:33:01.185 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:01.186 INFO:teuthology.orchestra.run.smithi169.stdout: "application_metadata": { 2023-10-26T01:33:01.186 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": {} 2023-10-26T01:33:01.186 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:01.186 INFO:teuthology.orchestra.run.smithi169.stdout: "read_balance": { 2023-10-26T01:33:01.186 INFO:teuthology.orchestra.run.smithi169.stdout: "score_acting": 8, 2023-10-26T01:33:01.187 INFO:teuthology.orchestra.run.smithi169.stdout: "score_stable": 8, 2023-10-26T01:33:01.187 INFO:teuthology.orchestra.run.smithi169.stdout: "optimal_score": 0.25, 2023-10-26T01:33:01.187 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_acting": 2, 2023-10-26T01:33:01.187 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_stable": 2, 2023-10-26T01:33:01.187 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity_weighted": 1, 2023-10-26T01:33:01.188 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity": 1, 2023-10-26T01:33:01.188 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity_weighted": 1 2023-10-26T01:33:01.188 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:01.188 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:01.188 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:01.188 INFO:teuthology.orchestra.run.smithi169.stdout: "pool": 6, 2023-10-26T01:33:01.189 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_name": "cephfs_metadata", 2023-10-26T01:33:01.189 INFO:teuthology.orchestra.run.smithi169.stdout: "create_time": "2023-10-26T01:32:36.288217+0000", 2023-10-26T01:33:01.189 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": 3, 2023-10-26T01:33:01.189 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_names": "hashpspool,full", 2023-10-26T01:33:01.189 INFO:teuthology.orchestra.run.smithi169.stdout: "type": 1, 2023-10-26T01:33:01.189 INFO:teuthology.orchestra.run.smithi169.stdout: "size": 2, 2023-10-26T01:33:01.190 INFO:teuthology.orchestra.run.smithi169.stdout: "min_size": 1, 2023-10-26T01:33:01.190 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_rule": 0, 2023-10-26T01:33:01.190 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_count": 0, 2023-10-26T01:33:01.190 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_target": 0, 2023-10-26T01:33:01.190 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T01:33:01.191 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T01:33:01.191 INFO:teuthology.orchestra.run.smithi169.stdout: "object_hash": 2, 2023-10-26T01:33:01.191 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_mode": "off", 2023-10-26T01:33:01.191 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num": 32, 2023-10-26T01:33:01.191 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num": 32, 2023-10-26T01:33:01.192 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num_target": 32, 2023-10-26T01:33:01.192 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_target": 32, 2023-10-26T01:33:01.192 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_pending": 32, 2023-10-26T01:33:01.192 INFO:teuthology.orchestra.run.smithi169.stdout: "last_pg_merge_meta": { 2023-10-26T01:33:01.192 INFO:teuthology.orchestra.run.smithi169.stdout: "source_pgid": "0.0", 2023-10-26T01:33:01.193 INFO:teuthology.orchestra.run.smithi169.stdout: "ready_epoch": 0, 2023-10-26T01:33:01.193 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_started": 0, 2023-10-26T01:33:01.193 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_clean": 0, 2023-10-26T01:33:01.193 INFO:teuthology.orchestra.run.smithi169.stdout: "source_version": "0'0", 2023-10-26T01:33:01.193 INFO:teuthology.orchestra.run.smithi169.stdout: "target_version": "0'0" 2023-10-26T01:33:01.193 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:01.194 INFO:teuthology.orchestra.run.smithi169.stdout: "last_change": "50", 2023-10-26T01:33:01.194 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend": "0", 2023-10-26T01:33:01.194 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T01:33:01.194 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T01:33:01.194 INFO:teuthology.orchestra.run.smithi169.stdout: "auid": 0, 2023-10-26T01:33:01.195 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_mode": "selfmanaged", 2023-10-26T01:33:01.195 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_seq": 0, 2023-10-26T01:33:01.195 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_epoch": 0, 2023-10-26T01:33:01.195 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_snaps": [], 2023-10-26T01:33:01.195 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps": "[]", 2023-10-26T01:33:01.196 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_bytes": 0, 2023-10-26T01:33:01.196 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_objects": 0, 2023-10-26T01:33:01.196 INFO:teuthology.orchestra.run.smithi169.stdout: "tiers": [], 2023-10-26T01:33:01.196 INFO:teuthology.orchestra.run.smithi169.stdout: "tier_of": -1, 2023-10-26T01:33:01.196 INFO:teuthology.orchestra.run.smithi169.stdout: "read_tier": -1, 2023-10-26T01:33:01.196 INFO:teuthology.orchestra.run.smithi169.stdout: "write_tier": -1, 2023-10-26T01:33:01.197 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_mode": "none", 2023-10-26T01:33:01.197 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_bytes": 0, 2023-10-26T01:33:01.197 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_objects": 0, 2023-10-26T01:33:01.197 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T01:33:01.197 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T01:33:01.198 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T01:33:01.198 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_flush_age": 0, 2023-10-26T01:33:01.198 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_evict_age": 0, 2023-10-26T01:33:01.198 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profile": "", 2023-10-26T01:33:01.198 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_params": { 2023-10-26T01:33:01.199 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "none" 2023-10-26T01:33:01.199 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:01.199 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_period": 0, 2023-10-26T01:33:01.199 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_count": 0, 2023-10-26T01:33:01.199 INFO:teuthology.orchestra.run.smithi169.stdout: "use_gmt_hitset": true, 2023-10-26T01:33:01.199 INFO:teuthology.orchestra.run.smithi169.stdout: "min_read_recency_for_promote": 0, 2023-10-26T01:33:01.200 INFO:teuthology.orchestra.run.smithi169.stdout: "min_write_recency_for_promote": 0, 2023-10-26T01:33:01.200 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T01:33:01.200 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_search_last_n": 0, 2023-10-26T01:33:01.200 INFO:teuthology.orchestra.run.smithi169.stdout: "grade_table": [], 2023-10-26T01:33:01.200 INFO:teuthology.orchestra.run.smithi169.stdout: "stripe_width": 0, 2023-10-26T01:33:01.201 INFO:teuthology.orchestra.run.smithi169.stdout: "expected_num_objects": 0, 2023-10-26T01:33:01.201 INFO:teuthology.orchestra.run.smithi169.stdout: "fast_read": false, 2023-10-26T01:33:01.201 INFO:teuthology.orchestra.run.smithi169.stdout: "options": { 2023-10-26T01:33:01.201 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_bias": 4, 2023-10-26T01:33:01.201 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_min": 16, 2023-10-26T01:33:01.201 INFO:teuthology.orchestra.run.smithi169.stdout: "recovery_priority": 5 2023-10-26T01:33:01.202 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:01.202 INFO:teuthology.orchestra.run.smithi169.stdout: "application_metadata": { 2023-10-26T01:33:01.202 INFO:teuthology.orchestra.run.smithi169.stdout: "cephfs": { 2023-10-26T01:33:01.202 INFO:teuthology.orchestra.run.smithi169.stdout: "metadata": "cephfs" 2023-10-26T01:33:01.202 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:01.202 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:01.203 INFO:teuthology.orchestra.run.smithi169.stdout: "read_balance": { 2023-10-26T01:33:01.203 INFO:teuthology.orchestra.run.smithi169.stdout: "score_acting": 1.75, 2023-10-26T01:33:01.203 INFO:teuthology.orchestra.run.smithi169.stdout: "score_stable": 1.75, 2023-10-26T01:33:01.203 INFO:teuthology.orchestra.run.smithi169.stdout: "optimal_score": 1, 2023-10-26T01:33:01.203 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_acting": 1.75, 2023-10-26T01:33:01.203 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_stable": 1.75, 2023-10-26T01:33:01.204 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity_weighted": 1, 2023-10-26T01:33:01.204 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity": 1, 2023-10-26T01:33:01.204 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity_weighted": 1 2023-10-26T01:33:01.204 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:01.204 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:01.205 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:01.205 INFO:teuthology.orchestra.run.smithi169.stdout: "pool": 7, 2023-10-26T01:33:01.205 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_name": "cephfs_data", 2023-10-26T01:33:01.205 INFO:teuthology.orchestra.run.smithi169.stdout: "create_time": "2023-10-26T01:32:37.059775+0000", 2023-10-26T01:33:01.205 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": 3, 2023-10-26T01:33:01.206 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_names": "hashpspool,full", 2023-10-26T01:33:01.206 INFO:teuthology.orchestra.run.smithi169.stdout: "type": 1, 2023-10-26T01:33:01.206 INFO:teuthology.orchestra.run.smithi169.stdout: "size": 2, 2023-10-26T01:33:01.206 INFO:teuthology.orchestra.run.smithi169.stdout: "min_size": 1, 2023-10-26T01:33:01.206 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_rule": 0, 2023-10-26T01:33:01.207 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_count": 0, 2023-10-26T01:33:01.207 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_target": 0, 2023-10-26T01:33:01.207 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T01:33:01.207 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T01:33:01.207 INFO:teuthology.orchestra.run.smithi169.stdout: "object_hash": 2, 2023-10-26T01:33:01.208 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_mode": "off", 2023-10-26T01:33:01.208 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num": 64, 2023-10-26T01:33:01.208 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num": 64, 2023-10-26T01:33:01.208 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num_target": 64, 2023-10-26T01:33:01.208 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_target": 64, 2023-10-26T01:33:01.209 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_pending": 64, 2023-10-26T01:33:01.209 INFO:teuthology.orchestra.run.smithi169.stdout: "last_pg_merge_meta": { 2023-10-26T01:33:01.209 INFO:teuthology.orchestra.run.smithi169.stdout: "source_pgid": "0.0", 2023-10-26T01:33:01.209 INFO:teuthology.orchestra.run.smithi169.stdout: "ready_epoch": 0, 2023-10-26T01:33:01.209 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_started": 0, 2023-10-26T01:33:01.210 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_clean": 0, 2023-10-26T01:33:01.210 INFO:teuthology.orchestra.run.smithi169.stdout: "source_version": "0'0", 2023-10-26T01:33:01.210 INFO:teuthology.orchestra.run.smithi169.stdout: "target_version": "0'0" 2023-10-26T01:33:01.210 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:01.210 INFO:teuthology.orchestra.run.smithi169.stdout: "last_change": "50", 2023-10-26T01:33:01.210 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend": "0", 2023-10-26T01:33:01.211 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T01:33:01.211 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T01:33:01.211 INFO:teuthology.orchestra.run.smithi169.stdout: "auid": 0, 2023-10-26T01:33:01.211 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_mode": "selfmanaged", 2023-10-26T01:33:01.211 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_seq": 0, 2023-10-26T01:33:01.212 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_epoch": 0, 2023-10-26T01:33:01.212 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_snaps": [], 2023-10-26T01:33:01.212 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps": "[]", 2023-10-26T01:33:01.212 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_bytes": 0, 2023-10-26T01:33:01.212 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_objects": 0, 2023-10-26T01:33:01.213 INFO:teuthology.orchestra.run.smithi169.stdout: "tiers": [], 2023-10-26T01:33:01.213 INFO:teuthology.orchestra.run.smithi169.stdout: "tier_of": -1, 2023-10-26T01:33:01.213 INFO:teuthology.orchestra.run.smithi169.stdout: "read_tier": -1, 2023-10-26T01:33:01.213 INFO:teuthology.orchestra.run.smithi169.stdout: "write_tier": -1, 2023-10-26T01:33:01.213 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_mode": "none", 2023-10-26T01:33:01.214 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_bytes": 0, 2023-10-26T01:33:01.214 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_objects": 0, 2023-10-26T01:33:01.214 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T01:33:01.214 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T01:33:01.214 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T01:33:01.214 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_flush_age": 0, 2023-10-26T01:33:01.215 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_evict_age": 0, 2023-10-26T01:33:01.215 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profile": "", 2023-10-26T01:33:01.215 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_params": { 2023-10-26T01:33:01.215 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "none" 2023-10-26T01:33:01.215 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:01.216 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_period": 0, 2023-10-26T01:33:01.216 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_count": 0, 2023-10-26T01:33:01.216 INFO:teuthology.orchestra.run.smithi169.stdout: "use_gmt_hitset": true, 2023-10-26T01:33:01.216 INFO:teuthology.orchestra.run.smithi169.stdout: "min_read_recency_for_promote": 0, 2023-10-26T01:33:01.216 INFO:teuthology.orchestra.run.smithi169.stdout: "min_write_recency_for_promote": 0, 2023-10-26T01:33:01.217 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T01:33:01.217 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_search_last_n": 0, 2023-10-26T01:33:01.217 INFO:teuthology.orchestra.run.smithi169.stdout: "grade_table": [], 2023-10-26T01:33:01.217 INFO:teuthology.orchestra.run.smithi169.stdout: "stripe_width": 0, 2023-10-26T01:33:01.217 INFO:teuthology.orchestra.run.smithi169.stdout: "expected_num_objects": 0, 2023-10-26T01:33:01.218 INFO:teuthology.orchestra.run.smithi169.stdout: "fast_read": false, 2023-10-26T01:33:01.218 INFO:teuthology.orchestra.run.smithi169.stdout: "options": { 2023-10-26T01:33:01.218 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_min": 64, 2023-10-26T01:33:01.218 INFO:teuthology.orchestra.run.smithi169.stdout: "target_size_ratio": 0.89999997615814209 2023-10-26T01:33:01.218 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:01.218 INFO:teuthology.orchestra.run.smithi169.stdout: "application_metadata": { 2023-10-26T01:33:01.219 INFO:teuthology.orchestra.run.smithi169.stdout: "cephfs": { 2023-10-26T01:33:01.219 INFO:teuthology.orchestra.run.smithi169.stdout: "data": "cephfs" 2023-10-26T01:33:01.219 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:01.219 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:01.219 INFO:teuthology.orchestra.run.smithi169.stdout: "read_balance": { 2023-10-26T01:33:01.220 INFO:teuthology.orchestra.run.smithi169.stdout: "score_acting": 1.75, 2023-10-26T01:33:01.220 INFO:teuthology.orchestra.run.smithi169.stdout: "score_stable": 1.75, 2023-10-26T01:33:01.220 INFO:teuthology.orchestra.run.smithi169.stdout: "optimal_score": 1, 2023-10-26T01:33:01.220 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_acting": 1.75, 2023-10-26T01:33:01.220 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_stable": 1.75, 2023-10-26T01:33:01.220 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity_weighted": 1, 2023-10-26T01:33:01.221 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity": 1, 2023-10-26T01:33:01.221 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity_weighted": 1 2023-10-26T01:33:01.221 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:01.221 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:01.221 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:33:01.222 INFO:teuthology.orchestra.run.smithi169.stdout: "osds": [ 2023-10-26T01:33:01.222 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:01.222 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 0, 2023-10-26T01:33:01.222 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:33:01.222 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:33:01.222 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:33:01.223 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:33:01.223 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:33:01.223 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:33:01.223 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:33:01.223 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:33:01.224 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 47, 2023-10-26T01:33:01.224 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:33:01.224 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:33:01.224 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:33:01.224 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:01.224 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:01.225 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:01.225 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6816", 2023-10-26T01:33:01.225 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:33:01.225 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:01.225 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:01.226 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:01.226 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6817", 2023-10-26T01:33:01.226 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:33:01.226 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:01.226 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:01.226 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:01.227 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:33:01.227 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:01.227 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:01.227 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:01.227 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6818", 2023-10-26T01:33:01.228 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:33:01.228 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:01.228 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:01.228 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:01.228 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6819", 2023-10-26T01:33:01.228 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:33:01.229 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:01.229 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:01.229 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:01.229 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:33:01.229 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:01.230 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:01.230 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:01.230 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6822", 2023-10-26T01:33:01.230 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:33:01.230 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:01.231 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:01.231 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:01.231 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6823", 2023-10-26T01:33:01.231 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:33:01.231 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:01.231 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:01.232 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:01.232 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:33:01.232 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:01.232 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:01.232 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:01.233 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6820", 2023-10-26T01:33:01.233 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:33:01.233 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:01.233 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:01.233 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:01.234 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6821", 2023-10-26T01:33:01.234 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:33:01.234 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:01.234 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:01.234 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:01.234 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6817/4204255864", 2023-10-26T01:33:01.235 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6819/4204255864", 2023-10-26T01:33:01.235 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6823/4204255864", 2023-10-26T01:33:01.235 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6821/4204255864", 2023-10-26T01:33:01.235 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:33:01.235 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:33:01.236 INFO:teuthology.orchestra.run.smithi169.stdout: "full", 2023-10-26T01:33:01.236 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:33:01.236 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:01.236 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:01.236 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:01.236 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 1, 2023-10-26T01:33:01.237 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:33:01.237 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:33:01.237 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:33:01.237 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:33:01.237 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:33:01.238 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:33:01.238 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:33:01.238 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:33:01.238 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 47, 2023-10-26T01:33:01.238 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:33:01.238 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:33:01.239 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:33:01.239 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:01.239 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:01.239 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:01.239 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6808", 2023-10-26T01:33:01.240 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:33:01.240 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:01.240 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:01.240 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:01.240 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6809", 2023-10-26T01:33:01.240 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:33:01.241 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:01.241 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:01.241 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:01.241 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:33:01.241 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:01.241 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:01.242 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:01.242 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6810", 2023-10-26T01:33:01.242 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:33:01.242 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:01.242 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:01.243 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:01.243 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6811", 2023-10-26T01:33:01.243 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:33:01.243 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:01.243 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:01.244 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:01.244 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:33:01.244 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:01.244 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:01.244 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:01.244 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6814", 2023-10-26T01:33:01.245 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:33:01.245 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:01.245 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:01.245 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:01.245 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6815", 2023-10-26T01:33:01.245 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:33:01.246 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:01.246 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:01.246 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:01.246 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:33:01.246 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:01.247 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:01.247 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:01.247 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6812", 2023-10-26T01:33:01.247 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:33:01.247 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:01.247 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:01.248 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:01.248 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6813", 2023-10-26T01:33:01.248 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:33:01.248 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:01.248 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:01.249 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:01.249 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6809/238206046", 2023-10-26T01:33:01.249 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6811/238206046", 2023-10-26T01:33:01.249 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6815/238206046", 2023-10-26T01:33:01.249 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6813/238206046", 2023-10-26T01:33:01.249 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:33:01.250 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:33:01.250 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:33:01.250 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:01.250 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:01.250 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:01.251 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 2, 2023-10-26T01:33:01.251 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:33:01.251 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:33:01.251 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:33:01.251 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:33:01.251 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:33:01.252 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:33:01.252 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:33:01.252 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:33:01.252 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 47, 2023-10-26T01:33:01.252 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:33:01.253 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:33:01.253 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:33:01.253 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:01.253 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:01.253 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:01.254 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6800", 2023-10-26T01:33:01.254 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:33:01.254 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:01.254 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:01.254 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:01.254 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6801", 2023-10-26T01:33:01.255 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:33:01.255 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:01.255 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:01.255 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:01.255 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:33:01.256 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:01.256 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:01.256 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:01.256 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6802", 2023-10-26T01:33:01.256 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:33:01.257 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:01.257 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:01.257 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:01.257 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6803", 2023-10-26T01:33:01.257 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:33:01.258 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:01.258 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:01.258 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:01.258 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:33:01.258 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:01.258 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:01.259 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:01.259 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6806", 2023-10-26T01:33:01.259 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:33:01.259 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:01.259 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:01.260 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:01.260 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6807", 2023-10-26T01:33:01.260 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:33:01.260 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:01.260 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:01.260 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:01.261 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:33:01.261 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:01.261 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:01.261 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:01.261 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6804", 2023-10-26T01:33:01.262 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:33:01.262 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:01.262 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:01.262 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:01.262 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6805", 2023-10-26T01:33:01.262 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:33:01.263 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:01.263 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:01.263 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:01.263 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6801/3215791846", 2023-10-26T01:33:01.263 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6803/3215791846", 2023-10-26T01:33:01.264 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6807/3215791846", 2023-10-26T01:33:01.264 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6805/3215791846", 2023-10-26T01:33:01.264 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:33:01.264 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:33:01.264 INFO:teuthology.orchestra.run.smithi169.stdout: "full", 2023-10-26T01:33:01.264 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:33:01.265 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:01.265 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:01.265 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:01.265 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 3, 2023-10-26T01:33:01.265 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:33:01.265 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:33:01.266 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:33:01.266 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:33:01.266 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:33:01.266 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:33:01.266 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:33:01.267 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 14, 2023-10-26T01:33:01.267 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 47, 2023-10-26T01:33:01.267 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:33:01.267 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:33:01.267 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:33:01.268 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:01.268 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:01.268 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:01.268 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6824", 2023-10-26T01:33:01.268 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:33:01.269 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:01.269 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:01.269 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:01.269 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6825", 2023-10-26T01:33:01.269 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:33:01.269 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:01.270 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:01.270 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:01.270 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:33:01.270 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:01.270 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:01.271 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:01.271 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6826", 2023-10-26T01:33:01.271 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:33:01.271 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:01.271 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:01.272 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:01.272 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6827", 2023-10-26T01:33:01.272 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:33:01.272 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:01.272 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:01.273 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:01.273 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:33:01.273 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:01.273 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:01.273 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:01.274 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6830", 2023-10-26T01:33:01.274 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:33:01.274 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:01.274 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:01.274 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:01.274 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6831", 2023-10-26T01:33:01.275 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:33:01.275 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:01.275 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:01.275 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:01.275 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:33:01.276 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:01.276 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:01.276 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:01.276 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6828", 2023-10-26T01:33:01.276 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:33:01.276 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:01.277 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:01.277 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:01.277 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6829", 2023-10-26T01:33:01.277 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:33:01.278 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:01.278 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:01.278 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:01.278 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6825/2256701226", 2023-10-26T01:33:01.278 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6827/2256701226", 2023-10-26T01:33:01.278 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6831/2256701226", 2023-10-26T01:33:01.279 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6829/2256701226", 2023-10-26T01:33:01.279 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:33:01.279 INFO:teuthology.orchestra.run.smithi169.stdout: "backfillfull", 2023-10-26T01:33:01.279 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:33:01.279 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:33:01.280 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:01.280 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:01.280 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:01.280 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 4, 2023-10-26T01:33:01.280 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:33:01.281 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:33:01.281 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:33:01.281 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:33:01.281 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:33:01.281 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:33:01.281 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:33:01.282 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:33:01.282 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 47, 2023-10-26T01:33:01.282 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:33:01.282 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:33:01.282 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:33:01.283 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:01.283 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:01.283 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:01.283 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6816", 2023-10-26T01:33:01.283 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:33:01.283 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:01.284 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:01.284 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:01.284 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6817", 2023-10-26T01:33:01.284 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:33:01.284 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:01.285 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:01.285 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:01.285 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:33:01.285 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:01.285 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:01.285 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:01.286 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6818", 2023-10-26T01:33:01.286 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:33:01.286 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:01.286 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:01.286 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:01.287 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6819", 2023-10-26T01:33:01.287 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:33:01.287 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:01.287 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:01.287 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:01.288 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:33:01.288 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:01.288 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:01.288 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:01.288 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6825", 2023-10-26T01:33:01.288 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:33:01.289 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:01.289 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:01.289 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:01.289 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6828", 2023-10-26T01:33:01.289 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:33:01.290 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:01.290 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:01.290 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:01.290 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:33:01.290 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:01.291 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:01.291 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:01.291 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6820", 2023-10-26T01:33:01.291 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:33:01.291 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:01.291 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:01.292 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:01.292 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6823", 2023-10-26T01:33:01.292 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:33:01.292 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:01.292 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:01.293 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:01.293 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6817/1082279874", 2023-10-26T01:33:01.293 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6819/1082279874", 2023-10-26T01:33:01.293 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6828/1082279874", 2023-10-26T01:33:01.293 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6823/1082279874", 2023-10-26T01:33:01.293 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:33:01.294 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:33:01.294 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:33:01.294 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:01.294 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:01.294 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:01.295 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 5, 2023-10-26T01:33:01.295 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:33:01.295 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:33:01.295 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:33:01.295 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:33:01.296 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:33:01.296 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:33:01.296 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:33:01.296 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:33:01.296 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 47, 2023-10-26T01:33:01.296 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:33:01.297 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:33:01.297 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:33:01.297 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:01.297 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:01.297 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:01.297 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6808", 2023-10-26T01:33:01.298 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:33:01.298 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:01.298 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:01.298 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:01.298 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6809", 2023-10-26T01:33:01.299 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:33:01.299 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:01.299 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:01.299 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:01.299 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:33:01.300 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:01.300 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:01.300 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:01.300 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6810", 2023-10-26T01:33:01.300 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:33:01.301 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:01.301 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:01.301 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:01.301 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6811", 2023-10-26T01:33:01.301 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:33:01.301 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:01.302 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:01.302 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:01.302 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:33:01.302 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:01.302 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:01.303 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:01.303 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6814", 2023-10-26T01:33:01.303 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:33:01.303 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:01.303 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:01.303 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:01.304 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6815", 2023-10-26T01:33:01.304 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:33:01.304 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:01.304 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:01.304 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:01.304 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:33:01.305 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:01.305 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:01.305 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:01.305 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6812", 2023-10-26T01:33:01.305 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:33:01.305 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:01.306 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:01.306 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:01.306 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6813", 2023-10-26T01:33:01.306 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:33:01.306 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:01.307 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:01.307 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:01.307 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6809/3335654351", 2023-10-26T01:33:01.307 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6811/3335654351", 2023-10-26T01:33:01.307 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6815/3335654351", 2023-10-26T01:33:01.308 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6813/3335654351", 2023-10-26T01:33:01.308 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:33:01.308 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:33:01.308 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:33:01.308 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:01.308 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:01.309 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:01.309 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 6, 2023-10-26T01:33:01.309 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:33:01.309 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:33:01.309 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:33:01.310 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:33:01.310 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:33:01.310 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:33:01.310 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:33:01.310 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:33:01.311 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 47, 2023-10-26T01:33:01.311 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:33:01.311 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:33:01.311 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:33:01.311 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:01.311 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:01.312 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:01.312 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6800", 2023-10-26T01:33:01.312 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:33:01.312 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:01.312 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:01.313 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:01.313 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6801", 2023-10-26T01:33:01.313 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:33:01.313 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:01.313 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:01.314 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:01.314 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:33:01.314 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:01.314 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:01.314 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:01.315 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6802", 2023-10-26T01:33:01.315 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:33:01.315 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:01.315 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:01.315 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:01.315 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6803", 2023-10-26T01:33:01.316 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:33:01.316 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:01.316 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:01.316 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:01.316 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:33:01.317 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:01.317 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:01.317 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:01.317 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6806", 2023-10-26T01:33:01.317 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:33:01.317 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:01.318 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:01.318 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:01.318 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6807", 2023-10-26T01:33:01.318 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:33:01.318 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:01.319 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:01.319 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:01.319 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:33:01.319 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:01.319 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:01.319 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:01.320 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6804", 2023-10-26T01:33:01.320 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:33:01.320 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:01.320 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:01.320 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:01.320 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6805", 2023-10-26T01:33:01.321 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:33:01.321 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:01.321 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:01.321 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:01.321 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6801/2520460697", 2023-10-26T01:33:01.322 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6803/2520460697", 2023-10-26T01:33:01.322 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6807/2520460697", 2023-10-26T01:33:01.322 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6805/2520460697", 2023-10-26T01:33:01.322 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:33:01.322 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:33:01.322 INFO:teuthology.orchestra.run.smithi169.stdout: "full", 2023-10-26T01:33:01.323 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:33:01.323 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:01.323 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:01.323 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:01.323 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 7, 2023-10-26T01:33:01.324 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:33:01.324 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:33:01.324 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:33:01.324 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:33:01.324 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:33:01.325 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:33:01.325 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:33:01.325 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:33:01.325 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 47, 2023-10-26T01:33:01.325 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:33:01.325 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:33:01.326 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:33:01.326 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:01.326 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:01.326 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:01.326 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6821", 2023-10-26T01:33:01.327 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:33:01.327 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:01.327 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:01.327 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:01.327 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6822", 2023-10-26T01:33:01.327 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:33:01.328 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:01.328 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:01.328 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:01.328 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:33:01.328 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:01.329 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:01.329 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:01.329 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6824", 2023-10-26T01:33:01.329 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:33:01.329 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:01.330 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:01.330 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:01.330 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6826", 2023-10-26T01:33:01.330 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:33:01.330 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:01.330 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:01.331 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:01.331 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:33:01.331 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:01.331 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:01.333 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:01.333 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6830", 2023-10-26T01:33:01.333 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:33:01.333 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:01.334 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:01.334 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:01.334 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6831", 2023-10-26T01:33:01.334 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:33:01.334 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:01.335 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:01.335 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:01.335 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:33:01.335 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:01.335 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:01.335 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:01.336 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6827", 2023-10-26T01:33:01.336 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:33:01.336 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:01.336 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:01.336 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:01.336 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6829", 2023-10-26T01:33:01.337 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:33:01.337 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:01.337 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:01.337 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:01.337 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6822/1115930619", 2023-10-26T01:33:01.337 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6826/1115930619", 2023-10-26T01:33:01.338 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6831/1115930619", 2023-10-26T01:33:01.338 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6829/1115930619", 2023-10-26T01:33:01.338 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:33:01.338 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:33:01.338 INFO:teuthology.orchestra.run.smithi169.stdout: "full", 2023-10-26T01:33:01.339 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:33:01.339 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:01.339 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:01.339 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:33:01.339 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_xinfo": [ 2023-10-26T01:33:01.340 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:01.340 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 0, 2023-10-26T01:33:01.340 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:33:01.340 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:33:01.340 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:33:01.340 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:33:01.341 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:33:01.341 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.889675+0000", 2023-10-26T01:33:01.341 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:33:01.341 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:01.341 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:01.341 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 1, 2023-10-26T01:33:01.342 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:33:01.342 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:33:01.342 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:33:01.342 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:33:01.342 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:33:01.343 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.864157+0000", 2023-10-26T01:33:01.343 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:33:01.343 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:01.343 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:01.343 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 2, 2023-10-26T01:33:01.344 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:33:01.344 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:33:01.344 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:33:01.344 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:33:01.344 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:33:01.344 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.876919+0000", 2023-10-26T01:33:01.345 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:33:01.345 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:01.345 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:01.345 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 3, 2023-10-26T01:33:01.345 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:33:01.346 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:33:01.346 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:33:01.346 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:33:01.346 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:33:01.346 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.974541+0000", 2023-10-26T01:33:01.346 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:33:01.347 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:01.347 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:01.347 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 4, 2023-10-26T01:33:01.347 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:33:01.347 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:33:01.348 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:33:01.348 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:33:01.348 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:33:01.348 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.969696+0000", 2023-10-26T01:33:01.348 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:33:01.349 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:01.349 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:01.349 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 5, 2023-10-26T01:33:01.349 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:33:01.349 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:33:01.349 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:33:01.350 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:33:01.350 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:33:01.350 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.859703+0000", 2023-10-26T01:33:01.350 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:33:01.350 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:01.351 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:01.351 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 6, 2023-10-26T01:33:01.351 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:33:01.351 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:33:01.351 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:33:01.352 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:33:01.352 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:33:01.352 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.867917+0000", 2023-10-26T01:33:01.352 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:33:01.352 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:01.353 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:01.353 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 7, 2023-10-26T01:33:01.353 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:33:01.353 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:33:01.353 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:33:01.353 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:33:01.354 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:33:01.354 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.898605+0000", 2023-10-26T01:33:01.354 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:33:01.354 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:01.354 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:33:01.355 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_upmap": [], 2023-10-26T01:33:01.355 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_upmap_items": [], 2023-10-26T01:33:01.355 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_upmap_primaries": [], 2023-10-26T01:33:01.355 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_temp": [], 2023-10-26T01:33:01.355 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_temp": [], 2023-10-26T01:33:01.356 INFO:teuthology.orchestra.run.smithi169.stdout: "blocklist": {}, 2023-10-26T01:33:01.356 INFO:teuthology.orchestra.run.smithi169.stdout: "range_blocklist": {}, 2023-10-26T01:33:01.356 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profiles": { 2023-10-26T01:33:01.356 INFO:teuthology.orchestra.run.smithi169.stdout: "default": { 2023-10-26T01:33:01.356 INFO:teuthology.orchestra.run.smithi169.stdout: "crush-failure-domain": "osd", 2023-10-26T01:33:01.356 INFO:teuthology.orchestra.run.smithi169.stdout: "k": "2", 2023-10-26T01:33:01.357 INFO:teuthology.orchestra.run.smithi169.stdout: "m": "1", 2023-10-26T01:33:01.357 INFO:teuthology.orchestra.run.smithi169.stdout: "plugin": "jerasure", 2023-10-26T01:33:01.357 INFO:teuthology.orchestra.run.smithi169.stdout: "technique": "reed_sol_van" 2023-10-26T01:33:01.357 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:01.357 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:01.358 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps_queue": [], 2023-10-26T01:33:01.358 INFO:teuthology.orchestra.run.smithi169.stdout: "new_removed_snaps": [], 2023-10-26T01:33:01.358 INFO:teuthology.orchestra.run.smithi169.stdout: "new_purged_snaps": [], 2023-10-26T01:33:01.358 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_node_flags": {}, 2023-10-26T01:33:01.358 INFO:teuthology.orchestra.run.smithi169.stdout: "device_class_flags": {}, 2023-10-26T01:33:01.358 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_mode": { 2023-10-26T01:33:01.359 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_mode_enabled": false, 2023-10-26T01:33:01.359 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_bucket_count": 0, 2023-10-26T01:33:01.359 INFO:teuthology.orchestra.run.smithi169.stdout: "degraded_stretch_mode": 0, 2023-10-26T01:33:01.359 INFO:teuthology.orchestra.run.smithi169.stdout: "recovering_stretch_mode": 0, 2023-10-26T01:33:01.359 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_mode_bucket": 0 2023-10-26T01:33:01.360 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:01.360 INFO:teuthology.orchestra.run.smithi169.stdout:} 2023-10-26T01:33:01.361 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:01.167+0000 7fd2acea2640 1 -- 172.21.15.169:0/1459065942 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd28806ae40 msgr2=0x7fd28806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:01.361 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:01.167+0000 7fd2acea2640 1 --2- 172.21.15.169:0/1459065942 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd28806ae40 0x7fd28806d300 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd2a8157b90 tx=0x7fd29c0079b0 comp rx=0 tx=0).stop 2023-10-26T01:33:01.362 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:01.167+0000 7fd2acea2640 1 -- 172.21.15.169:0/1459065942 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd290003970 msgr2=0x7fd2a81f1840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:01.362 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:01.167+0000 7fd2acea2640 1 --2- 172.21.15.169:0/1459065942 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd290003970 0x7fd2a81f1840 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fd298009b50 tx=0x7fd298033bd0 comp rx=0 tx=0).stop 2023-10-26T01:33:01.362 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:01.168+0000 7fd2acea2640 1 -- 172.21.15.169:0/1459065942 shutdown_connections 2023-10-26T01:33:01.362 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:01.168+0000 7fd2acea2640 1 --2- 172.21.15.169:0/1459065942 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd28806ae40 0x7fd28806d300 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:01.362 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:01.168+0000 7fd2acea2640 1 --2- 172.21.15.169:0/1459065942 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd2a80af680 0x7fd2a8155ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:01.363 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:01.168+0000 7fd2acea2640 1 --2- 172.21.15.169:0/1459065942 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd2a8152300 0x7fd2a81559a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:01.363 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:01.168+0000 7fd2acea2640 1 --2- 172.21.15.169:0/1459065942 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd290003970 0x7fd2a81f1840 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:01.363 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:01.168+0000 7fd2acea2640 1 -- 172.21.15.169:0/1459065942 >> 172.21.15.169:0/1459065942 conn(0x7fd2a8069930 msgr2=0x7fd2a8146810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:01.363 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:01.168+0000 7fd2acea2640 1 -- 172.21.15.169:0/1459065942 shutdown_connections 2023-10-26T01:33:01.363 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:01.168+0000 7fd2acea2640 1 -- 172.21.15.169:0/1459065942 wait complete. 2023-10-26T01:33:01.364 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T01:33:01.590 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:01.588+0000 7f3225052640 1 -- 172.21.15.169:0/2737074002 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3208004a80 msgr2=0x7f3208004ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:01.591 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:01.588+0000 7f3225052640 1 --2- 172.21.15.169:0/2737074002 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3208004a80 0x7f3208004ea0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f321000bb20 tx=0x7f3208006900 comp rx=0 tx=0).stop 2023-10-26T01:33:01.591 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:01.588+0000 7f3224851640 1 -- 172.21.15.169:0/2737074002 reap_dead start 2023-10-26T01:33:01.591 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:01.588+0000 7f3225052640 1 -- 172.21.15.169:0/2737074002 shutdown_connections 2023-10-26T01:33:01.591 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:01.588+0000 7f3225052640 1 -- 172.21.15.169:0/2737074002 >> 172.21.15.169:0/2737074002 conn(0x7f32200aedd0 msgr2=0x7f32200af1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:01.592 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:01.589+0000 7f3225052640 1 -- 172.21.15.169:0/2737074002 shutdown_connections 2023-10-26T01:33:01.592 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:01.589+0000 7f3225052640 1 -- 172.21.15.169:0/2737074002 wait complete. 2023-10-26T01:33:01.592 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:01.590+0000 7f3225052640 1 Processor -- start 2023-10-26T01:33:01.593 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:01.591+0000 7f3225052640 1 -- start start 2023-10-26T01:33:01.594 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:01.591+0000 7f3225052640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3208004a80 0x7f32200b2280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:01.595 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:01.591+0000 7f3225052640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f32200c6270 0x7f32200c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:01.595 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:01.592+0000 7f3225052640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f32201f1e00 0x7f32201f2280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:01.595 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:01.592+0000 7f3225052640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32200c6c30 con 0x7f3208004a80 2023-10-26T01:33:01.595 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:01.592+0000 7f3225052640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32201f6150 con 0x7f32200c6270 2023-10-26T01:33:01.595 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:01.592+0000 7f3225052640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32201f2bc0 con 0x7f32201f1e00 2023-10-26T01:33:01.596 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:01.592+0000 7f321ffff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3208004a80 0x7f32200b2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:01.596 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:01.592+0000 7f321ffff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3208004a80 0x7f32200b2280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:58168/0 (socket says 172.21.15.169:58168) 2023-10-26T01:33:01.596 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:01.592+0000 7f321ffff640 1 -- 172.21.15.169:0/2233186333 learned_addr learned my addr 172.21.15.169:0/2233186333 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:33:01.596 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:01.592+0000 7f321ffff640 1 -- 172.21.15.169:0/2233186333 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f32201f1e00 msgr2=0x7f32201f2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:01.596 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:01.592+0000 7f321ffff640 1 --2- 172.21.15.169:0/2233186333 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f32201f1e00 0x7f32201f2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:01.597 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:01.592+0000 7f321ffff640 1 -- 172.21.15.169:0/2233186333 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f32200c6270 msgr2=0x7f32200c66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:01.597 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:01.592+0000 7f321ffff640 1 --2- 172.21.15.169:0/2233186333 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f32200c6270 0x7f32200c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:01.597 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:01.592+0000 7f321ffff640 1 -- 172.21.15.169:0/2233186333 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32201f5390 con 0x7f3208004a80 2023-10-26T01:33:01.597 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:01.593+0000 7f321ffff640 1 --2- 172.21.15.169:0/2233186333 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3208004a80 0x7f32200b2280 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f321002c6f0 tx=0x7f3210036720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:01.597 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:01.593+0000 7f321cff9640 1 -- 172.21.15.169:0/2233186333 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3210036de0 con 0x7f3208004a80 2023-10-26T01:33:01.598 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:01.593+0000 7f321cff9640 1 -- 172.21.15.169:0/2233186333 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3210004110 con 0x7f3208004a80 2023-10-26T01:33:01.598 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:01.593+0000 7f321cff9640 1 -- 172.21.15.169:0/2233186333 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3210004500 con 0x7f3208004a80 2023-10-26T01:33:01.598 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:01.593+0000 7f3225052640 1 -- 172.21.15.169:0/2233186333 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32201f5670 con 0x7f3208004a80 2023-10-26T01:33:01.598 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:01.593+0000 7f3225052640 1 -- 172.21.15.169:0/2233186333 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32201f5c30 con 0x7f3208004a80 2023-10-26T01:33:01.599 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:01.594+0000 7f3225052640 1 -- 172.21.15.169:0/2233186333 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32201f5df0 con 0x7f3208004a80 2023-10-26T01:33:01.599 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:01.595+0000 7f321cff9640 1 -- 172.21.15.169:0/2233186333 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f32100035f0 con 0x7f3208004a80 2023-10-26T01:33:01.600 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:01.596+0000 7f321cff9640 1 --2- 172.21.15.169:0/2233186333 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f320c06ae70 0x7f320c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:01.600 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:01.596+0000 7f321cff9640 1 -- 172.21.15.169:0/2233186333 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f32100ab720 con 0x7f3208004a80 2023-10-26T01:33:01.600 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:01.596+0000 7f321cff9640 1 -- 172.21.15.169:0/2233186333 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=52}) v3 -- 0x7f320c0729a0 con 0x7f3208004a80 2023-10-26T01:33:01.600 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:01.597+0000 7f321f7fe640 1 --2- 172.21.15.169:0/2233186333 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f320c06ae70 0x7f320c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:01.601 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:01.598+0000 7f321f7fe640 1 --2- 172.21.15.169:0/2233186333 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f320c06ae70 0x7f320c06d330 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3214005ee0 tx=0x7f3214005e10 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:01.602 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:01.601+0000 7f321cff9640 1 -- 172.21.15.169:0/2233186333 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f32100767c0 con 0x7f3208004a80 2023-10-26T01:33:01.829 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:01.827+0000 7f3225052640 1 -- 172.21.15.169:0/2233186333 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f32200ca120 con 0x7f3208004a80 2023-10-26T01:33:01.830 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:01.828+0000 7f321cff9640 1 -- 172.21.15.169:0/2233186333 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4925 (secure 0 0 0) 0x7f3210045020 con 0x7f3208004a80 2023-10-26T01:33:01.830 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:33:01.830 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4805,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/2294107634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2294107634},{"type":"v1","addr":"172.21.15.170:6836","nonce":2294107634}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":4987,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2208026662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2208026662},{"type":"v1","addr":"172.21.15.169:6833","nonce":2208026662}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":4993,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/2630365058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":2630365058},{"type":"v1","addr":"172.21.15.170:6837","nonce":2630365058}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T01:32:38.059721+0000","modified":"2023-10-26T01:32:41.428183+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5664},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5664":{"gid":5664,"name":"a","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6835/2816266753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":2816266753},{"type":"v1","addr":"172.21.15.169:6835","nonce":2816266753}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-10-26T01:33:01.831 INFO:teuthology.orchestra.run.smithi169.stderr:dumped fsmap epoch 23 2023-10-26T01:33:01.834 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:01.832+0000 7f3225052640 1 -- 172.21.15.169:0/2233186333 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f320c06ae70 msgr2=0x7f320c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:01.834 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:01.832+0000 7f3225052640 1 --2- 172.21.15.169:0/2233186333 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f320c06ae70 0x7f320c06d330 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3214005ee0 tx=0x7f3214005e10 comp rx=0 tx=0).stop 2023-10-26T01:33:01.834 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:01.832+0000 7f3225052640 1 -- 172.21.15.169:0/2233186333 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3208004a80 msgr2=0x7f32200b2280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:01.835 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:01.832+0000 7f3225052640 1 --2- 172.21.15.169:0/2233186333 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3208004a80 0x7f32200b2280 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f321002c6f0 tx=0x7f3210036720 comp rx=0 tx=0).stop 2023-10-26T01:33:01.835 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:01.832+0000 7f3225052640 1 -- 172.21.15.169:0/2233186333 shutdown_connections 2023-10-26T01:33:01.835 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:01.832+0000 7f3225052640 1 --2- 172.21.15.169:0/2233186333 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f320c06ae70 0x7f320c06d330 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:01.835 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:01.832+0000 7f3225052640 1 --2- 172.21.15.169:0/2233186333 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f32201f1e00 0x7f32201f2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:01.835 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:01.832+0000 7f3225052640 1 --2- 172.21.15.169:0/2233186333 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f32200c6270 0x7f32200c66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:01.836 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:01.832+0000 7f3225052640 1 --2- 172.21.15.169:0/2233186333 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3208004a80 0x7f32200b2280 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:01.836 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:01.832+0000 7f3225052640 1 -- 172.21.15.169:0/2233186333 >> 172.21.15.169:0/2233186333 conn(0x7f32200aedd0 msgr2=0x7f32200af1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:01.836 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:01.832+0000 7f3225052640 1 -- 172.21.15.169:0/2233186333 shutdown_connections 2023-10-26T01:33:01.836 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:01.832+0000 7f3225052640 1 -- 172.21.15.169:0/2233186333 wait complete. 2023-10-26T01:33:01.852 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2023-10-26T01:33:02.083 INFO:teuthology.orchestra.run.smithi169.stdout:{"cluster_fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","whoami":0,"id":5664,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":23.148870147,"mdsmap_epoch":23,"osdmap_epoch":51,"osdmap_epoch_barrier":51,"uptime":29.331211191000001} 2023-10-26T01:33:02.084 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "30203b28-ed8a-44e8-a9a6-a9bf25f980f1", "fs_name": "cephfs", "id": 5664, "mdsmap_epoch": 23, "osdmap_epoch": 51, "osdmap_epoch_barrier": 51, "rank_uptime": 23.148870147, "state": "up:active", "uptime": 29.331211191, "want_state": "up:active", "whoami": 0 } 2023-10-26T01:33:02.084 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-26T01:33:02.085 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:33:02.085 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:33:02.135 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:33:02.135 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=small_file_1 bs=1M conv=fdatasync count=0 seek=0) 2023-10-26T01:33:02.186 INFO:teuthology.orchestra.run.smithi169.stderr:dd: failed to open 'small_file_1': Permission denied 2023-10-26T01:33:02.186 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-26T01:33:02.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:33:02.188 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_a) 2023-10-26T01:33:02.245 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:33:02.246 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_b) 2023-10-26T01:33:02.367 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-26T01:33:02.588 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:02.586+0000 7f3bc30e5640 1 -- 172.21.15.169:0/1514399634 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3bbc0cde30 msgr2=0x7f3bbc159e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:02.588 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:02.586+0000 7f3bc30e5640 1 --2- 172.21.15.169:0/1514399634 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3bbc0cde30 0x7f3bbc159e10 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc0accf0 tx=0x7f3bb402c260 comp rx=0 tx=0).stop 2023-10-26T01:33:02.589 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:02.586+0000 7f3bc30e5640 1 -- 172.21.15.169:0/1514399634 shutdown_connections 2023-10-26T01:33:02.589 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:02.586+0000 7f3bc30e5640 1 --2- 172.21.15.169:0/1514399634 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3bbc0cde30 0x7f3bbc159e10 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:02.589 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:02.586+0000 7f3bc30e5640 1 --2- 172.21.15.169:0/1514399634 >> v2:172.21.15.170:3300/0 conn(0x7f3bbc14e790 0x7f3bbc0aab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:02.589 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:02.586+0000 7f3bc30e5640 1 -- 172.21.15.169:0/1514399634 >> 172.21.15.169:0/1514399634 conn(0x7f3bbc069930 msgr2=0x7f3bbc069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:02.590 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:02.586+0000 7f3bc30e5640 1 -- 172.21.15.169:0/1514399634 shutdown_connections 2023-10-26T01:33:02.590 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:02.586+0000 7f3bc30e5640 1 -- 172.21.15.169:0/1514399634 wait complete. 2023-10-26T01:33:02.592 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:02.588+0000 7f3bc30e5640 1 Processor -- start 2023-10-26T01:33:02.592 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:02.588+0000 7f3bc30e5640 1 -- start start 2023-10-26T01:33:02.592 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:02.589+0000 7f3bc30e5640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3bbc0cde30 0x7f3bbc153820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:02.593 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:02.589+0000 7f3bc30e5640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3bbc14e790 0x7f3bbc14ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:02.593 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:02.589+0000 7f3bc30e5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3bbc158850 0x7f3bbc154b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:02.593 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:02.589+0000 7f3bc30e5640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bbc14f2a0 con 0x7f3bbc14e790 2023-10-26T01:33:02.593 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:02.589+0000 7f3bc30e5640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bbc0aef60 con 0x7f3bbc158850 2023-10-26T01:33:02.594 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:02.589+0000 7f3bc30e5640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bbc0af240 con 0x7f3bbc0cde30 2023-10-26T01:33:02.594 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:02.589+0000 7f3bbbfff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3bbc14e790 0x7f3bbc14ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:02.594 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:02.589+0000 7f3bbbfff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3bbc14e790 0x7f3bbc14ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:58182/0 (socket says 172.21.15.169:58182) 2023-10-26T01:33:02.594 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:02.589+0000 7f3bbbfff640 1 -- 172.21.15.169:0/3870057347 learned_addr learned my addr 172.21.15.169:0/3870057347 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:33:02.595 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:02.590+0000 7f3bbbfff640 1 -- 172.21.15.169:0/3870057347 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3bbc0cde30 msgr2=0x7f3bbc153820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:02.595 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:02.590+0000 7f3bbbfff640 1 --2- 172.21.15.169:0/3870057347 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3bbc0cde30 0x7f3bbc153820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:02.595 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:02.590+0000 7f3bbbfff640 1 -- 172.21.15.169:0/3870057347 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3bbc158850 msgr2=0x7f3bbc154b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:02.595 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:02.590+0000 7f3bbbfff640 1 --2- 172.21.15.169:0/3870057347 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3bbc158850 0x7f3bbc154b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:02.595 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:02.590+0000 7f3bbbfff640 1 -- 172.21.15.169:0/3870057347 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bb4009900 con 0x7f3bbc14e790 2023-10-26T01:33:02.596 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:02.590+0000 7f3bbbfff640 1 --2- 172.21.15.169:0/3870057347 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3bbc14e790 0x7f3bbc14ed60 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f3ba800b3e0 tx=0x7f3ba800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:02.596 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:02.590+0000 7f3bb97fa640 1 -- 172.21.15.169:0/3870057347 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba8011830 con 0x7f3bbc14e790 2023-10-26T01:33:02.596 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:02.590+0000 7f3bb97fa640 1 -- 172.21.15.169:0/3870057347 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3ba8011aa0 con 0x7f3bbc14e790 2023-10-26T01:33:02.596 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:02.590+0000 7f3bb97fa640 1 -- 172.21.15.169:0/3870057347 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba8010380 con 0x7f3bbc14e790 2023-10-26T01:33:02.596 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:02.590+0000 7f3bc30e5640 1 -- 172.21.15.169:0/3870057347 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bbc0af520 con 0x7f3bbc14e790 2023-10-26T01:33:02.597 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:02.590+0000 7f3bc30e5640 1 -- 172.21.15.169:0/3870057347 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bbc1550b0 con 0x7f3bbc14e790 2023-10-26T01:33:02.598 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:02.592+0000 7f3bc30e5640 1 -- 172.21.15.169:0/3870057347 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bbc0afa40 con 0x7f3bbc14e790 2023-10-26T01:33:02.598 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:02.593+0000 7f3bb97fa640 1 -- 172.21.15.169:0/3870057347 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f3ba80104e0 con 0x7f3bbc14e790 2023-10-26T01:33:02.598 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:02.593+0000 7f3bb97fa640 1 --2- 172.21.15.169:0/3870057347 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3ba006ae40 0x7f3ba006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:02.599 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:02.594+0000 7f3bb97fa640 1 -- 172.21.15.169:0/3870057347 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3ba8089a60 con 0x7f3bbc14e790 2023-10-26T01:33:02.599 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:02.594+0000 7f3bb97fa640 1 -- 172.21.15.169:0/3870057347 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=52}) v3 -- 0x7f3ba0072970 con 0x7f3bbc14e790 2023-10-26T01:33:02.599 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:02.594+0000 7f3bc0e81640 1 --2- 172.21.15.169:0/3870057347 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3ba006ae40 0x7f3ba006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:02.599 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:02.595+0000 7f3bc0e81640 1 --2- 172.21.15.169:0/3870057347 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3ba006ae40 0x7f3ba006d300 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc06c490 tx=0x7f3bb0006c90 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:02.600 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:02.599+0000 7f3bb97fa640 1 -- 172.21.15.169:0/3870057347 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3ba8015060 con 0x7f3bbc14e790 2023-10-26T01:33:02.797 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:02.795+0000 7f3bc30e5640 1 -- 172.21.15.169:0/3870057347 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f3bbc04db10 con 0x7f3bbc14e790 2023-10-26T01:33:02.799 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:02.797+0000 7f3bb97fa640 1 -- 172.21.15.169:0/3870057347 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v51) v1 ==== 81+0+34139 (secure 0 0 0) 0x7f3bbc0afa40 con 0x7f3bbc14e790 2023-10-26T01:33:02.800 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:33:02.800 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2023-10-26T01:33:02.801 INFO:teuthology.orchestra.run.smithi169.stdout: "epoch": 51, 2023-10-26T01:33:02.801 INFO:teuthology.orchestra.run.smithi169.stdout: "fsid": "30203b28-ed8a-44e8-a9a6-a9bf25f980f1", 2023-10-26T01:33:02.801 INFO:teuthology.orchestra.run.smithi169.stdout: "created": "2023-10-26T01:30:22.897155+0000", 2023-10-26T01:33:02.802 INFO:teuthology.orchestra.run.smithi169.stdout: "modified": "2023-10-26T01:32:59.074259+0000", 2023-10-26T01:33:02.802 INFO:teuthology.orchestra.run.smithi169.stdout: "last_up_change": "2023-10-26T01:30:31.875281+0000", 2023-10-26T01:33:02.802 INFO:teuthology.orchestra.run.smithi169.stdout: "last_in_change": "2023-10-26T01:30:27.637941+0000", 2023-10-26T01:33:02.802 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-26T01:33:02.803 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_num": 5799936, 2023-10-26T01:33:02.803 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_set": [ 2023-10-26T01:33:02.803 INFO:teuthology.orchestra.run.smithi169.stdout: "pglog_hardlimit", 2023-10-26T01:33:02.803 INFO:teuthology.orchestra.run.smithi169.stdout: "purged_snapdirs", 2023-10-26T01:33:02.803 INFO:teuthology.orchestra.run.smithi169.stdout: "recovery_deletes", 2023-10-26T01:33:02.804 INFO:teuthology.orchestra.run.smithi169.stdout: "sortbitwise" 2023-10-26T01:33:02.804 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:33:02.804 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_version": 4, 2023-10-26T01:33:02.804 INFO:teuthology.orchestra.run.smithi169.stdout: "full_ratio": 0.69999998807907104, 2023-10-26T01:33:02.804 INFO:teuthology.orchestra.run.smithi169.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-26T01:33:02.805 INFO:teuthology.orchestra.run.smithi169.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-26T01:33:02.805 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_snapshot": "", 2023-10-26T01:33:02.805 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_max": 7, 2023-10-26T01:33:02.805 INFO:teuthology.orchestra.run.smithi169.stdout: "max_osd": 8, 2023-10-26T01:33:02.806 INFO:teuthology.orchestra.run.smithi169.stdout: "require_min_compat_client": "luminous", 2023-10-26T01:33:02.806 INFO:teuthology.orchestra.run.smithi169.stdout: "min_compat_client": "jewel", 2023-10-26T01:33:02.806 INFO:teuthology.orchestra.run.smithi169.stdout: "require_osd_release": "reef", 2023-10-26T01:33:02.806 INFO:teuthology.orchestra.run.smithi169.stdout: "allow_crimson": false, 2023-10-26T01:33:02.807 INFO:teuthology.orchestra.run.smithi169.stdout: "pools": [ 2023-10-26T01:33:02.807 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:02.807 INFO:teuthology.orchestra.run.smithi169.stdout: "pool": 1, 2023-10-26T01:33:02.807 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_name": ".mgr", 2023-10-26T01:33:02.807 INFO:teuthology.orchestra.run.smithi169.stdout: "create_time": "2023-10-26T01:30:32.200808+0000", 2023-10-26T01:33:02.808 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": 3, 2023-10-26T01:33:02.808 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_names": "hashpspool,full", 2023-10-26T01:33:02.808 INFO:teuthology.orchestra.run.smithi169.stdout: "type": 1, 2023-10-26T01:33:02.808 INFO:teuthology.orchestra.run.smithi169.stdout: "size": 2, 2023-10-26T01:33:02.808 INFO:teuthology.orchestra.run.smithi169.stdout: "min_size": 1, 2023-10-26T01:33:02.809 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_rule": 0, 2023-10-26T01:33:02.809 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_count": 0, 2023-10-26T01:33:02.809 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_target": 0, 2023-10-26T01:33:02.809 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T01:33:02.810 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T01:33:02.810 INFO:teuthology.orchestra.run.smithi169.stdout: "object_hash": 2, 2023-10-26T01:33:02.810 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_mode": "off", 2023-10-26T01:33:02.810 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num": 1, 2023-10-26T01:33:02.811 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num": 1, 2023-10-26T01:33:02.811 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num_target": 1, 2023-10-26T01:33:02.811 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_target": 1, 2023-10-26T01:33:02.812 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_pending": 1, 2023-10-26T01:33:02.812 INFO:teuthology.orchestra.run.smithi169.stdout: "last_pg_merge_meta": { 2023-10-26T01:33:02.812 INFO:teuthology.orchestra.run.smithi169.stdout: "source_pgid": "0.0", 2023-10-26T01:33:02.813 INFO:teuthology.orchestra.run.smithi169.stdout: "ready_epoch": 0, 2023-10-26T01:33:02.813 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_started": 0, 2023-10-26T01:33:02.814 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_clean": 0, 2023-10-26T01:33:02.814 INFO:teuthology.orchestra.run.smithi169.stdout: "source_version": "0'0", 2023-10-26T01:33:02.814 INFO:teuthology.orchestra.run.smithi169.stdout: "target_version": "0'0" 2023-10-26T01:33:02.815 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:02.815 INFO:teuthology.orchestra.run.smithi169.stdout: "last_change": "50", 2023-10-26T01:33:02.815 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend": "0", 2023-10-26T01:33:02.816 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T01:33:02.816 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T01:33:02.816 INFO:teuthology.orchestra.run.smithi169.stdout: "auid": 0, 2023-10-26T01:33:02.817 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_mode": "selfmanaged", 2023-10-26T01:33:02.817 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_seq": 0, 2023-10-26T01:33:02.817 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_epoch": 0, 2023-10-26T01:33:02.818 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_snaps": [], 2023-10-26T01:33:02.818 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps": "[]", 2023-10-26T01:33:02.818 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_bytes": 0, 2023-10-26T01:33:02.819 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_objects": 0, 2023-10-26T01:33:02.819 INFO:teuthology.orchestra.run.smithi169.stdout: "tiers": [], 2023-10-26T01:33:02.819 INFO:teuthology.orchestra.run.smithi169.stdout: "tier_of": -1, 2023-10-26T01:33:02.820 INFO:teuthology.orchestra.run.smithi169.stdout: "read_tier": -1, 2023-10-26T01:33:02.820 INFO:teuthology.orchestra.run.smithi169.stdout: "write_tier": -1, 2023-10-26T01:33:02.820 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_mode": "none", 2023-10-26T01:33:02.820 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_bytes": 0, 2023-10-26T01:33:02.820 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_objects": 0, 2023-10-26T01:33:02.821 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T01:33:02.821 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T01:33:02.821 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T01:33:02.821 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_flush_age": 0, 2023-10-26T01:33:02.821 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_evict_age": 0, 2023-10-26T01:33:02.822 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profile": "", 2023-10-26T01:33:02.822 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_params": { 2023-10-26T01:33:02.822 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "none" 2023-10-26T01:33:02.822 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:02.823 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_period": 0, 2023-10-26T01:33:02.823 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_count": 0, 2023-10-26T01:33:02.823 INFO:teuthology.orchestra.run.smithi169.stdout: "use_gmt_hitset": true, 2023-10-26T01:33:02.823 INFO:teuthology.orchestra.run.smithi169.stdout: "min_read_recency_for_promote": 0, 2023-10-26T01:33:02.823 INFO:teuthology.orchestra.run.smithi169.stdout: "min_write_recency_for_promote": 0, 2023-10-26T01:33:02.824 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T01:33:02.824 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_search_last_n": 0, 2023-10-26T01:33:02.824 INFO:teuthology.orchestra.run.smithi169.stdout: "grade_table": [], 2023-10-26T01:33:02.824 INFO:teuthology.orchestra.run.smithi169.stdout: "stripe_width": 0, 2023-10-26T01:33:02.824 INFO:teuthology.orchestra.run.smithi169.stdout: "expected_num_objects": 0, 2023-10-26T01:33:02.825 INFO:teuthology.orchestra.run.smithi169.stdout: "fast_read": false, 2023-10-26T01:33:02.825 INFO:teuthology.orchestra.run.smithi169.stdout: "options": { 2023-10-26T01:33:02.825 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_max": 32, 2023-10-26T01:33:02.825 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_min": 1 2023-10-26T01:33:02.826 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:02.826 INFO:teuthology.orchestra.run.smithi169.stdout: "application_metadata": { 2023-10-26T01:33:02.826 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": {} 2023-10-26T01:33:02.826 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:02.827 INFO:teuthology.orchestra.run.smithi169.stdout: "read_balance": { 2023-10-26T01:33:02.827 INFO:teuthology.orchestra.run.smithi169.stdout: "score_acting": 8, 2023-10-26T01:33:02.827 INFO:teuthology.orchestra.run.smithi169.stdout: "score_stable": 8, 2023-10-26T01:33:02.827 INFO:teuthology.orchestra.run.smithi169.stdout: "optimal_score": 0.25, 2023-10-26T01:33:02.827 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_acting": 2, 2023-10-26T01:33:02.828 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_stable": 2, 2023-10-26T01:33:02.828 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity_weighted": 1, 2023-10-26T01:33:02.828 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity": 1, 2023-10-26T01:33:02.829 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity_weighted": 1 2023-10-26T01:33:02.829 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:02.829 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:02.829 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:02.829 INFO:teuthology.orchestra.run.smithi169.stdout: "pool": 6, 2023-10-26T01:33:02.830 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_name": "cephfs_metadata", 2023-10-26T01:33:02.830 INFO:teuthology.orchestra.run.smithi169.stdout: "create_time": "2023-10-26T01:32:36.288217+0000", 2023-10-26T01:33:02.830 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": 3, 2023-10-26T01:33:02.831 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_names": "hashpspool,full", 2023-10-26T01:33:02.831 INFO:teuthology.orchestra.run.smithi169.stdout: "type": 1, 2023-10-26T01:33:02.831 INFO:teuthology.orchestra.run.smithi169.stdout: "size": 2, 2023-10-26T01:33:02.831 INFO:teuthology.orchestra.run.smithi169.stdout: "min_size": 1, 2023-10-26T01:33:02.832 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_rule": 0, 2023-10-26T01:33:02.832 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_count": 0, 2023-10-26T01:33:02.832 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_target": 0, 2023-10-26T01:33:02.833 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T01:33:02.833 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T01:33:02.833 INFO:teuthology.orchestra.run.smithi169.stdout: "object_hash": 2, 2023-10-26T01:33:02.833 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_mode": "off", 2023-10-26T01:33:02.833 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num": 32, 2023-10-26T01:33:02.834 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num": 32, 2023-10-26T01:33:02.834 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num_target": 32, 2023-10-26T01:33:02.834 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_target": 32, 2023-10-26T01:33:02.834 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_pending": 32, 2023-10-26T01:33:02.835 INFO:teuthology.orchestra.run.smithi169.stdout: "last_pg_merge_meta": { 2023-10-26T01:33:02.835 INFO:teuthology.orchestra.run.smithi169.stdout: "source_pgid": "0.0", 2023-10-26T01:33:02.835 INFO:teuthology.orchestra.run.smithi169.stdout: "ready_epoch": 0, 2023-10-26T01:33:02.835 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_started": 0, 2023-10-26T01:33:02.836 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_clean": 0, 2023-10-26T01:33:02.836 INFO:teuthology.orchestra.run.smithi169.stdout: "source_version": "0'0", 2023-10-26T01:33:02.836 INFO:teuthology.orchestra.run.smithi169.stdout: "target_version": "0'0" 2023-10-26T01:33:02.836 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:02.836 INFO:teuthology.orchestra.run.smithi169.stdout: "last_change": "50", 2023-10-26T01:33:02.837 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend": "0", 2023-10-26T01:33:02.837 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T01:33:02.837 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T01:33:02.837 INFO:teuthology.orchestra.run.smithi169.stdout: "auid": 0, 2023-10-26T01:33:02.838 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_mode": "selfmanaged", 2023-10-26T01:33:02.838 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_seq": 0, 2023-10-26T01:33:02.838 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_epoch": 0, 2023-10-26T01:33:02.838 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_snaps": [], 2023-10-26T01:33:02.839 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps": "[]", 2023-10-26T01:33:02.839 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_bytes": 0, 2023-10-26T01:33:02.839 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_objects": 0, 2023-10-26T01:33:02.839 INFO:teuthology.orchestra.run.smithi169.stdout: "tiers": [], 2023-10-26T01:33:02.840 INFO:teuthology.orchestra.run.smithi169.stdout: "tier_of": -1, 2023-10-26T01:33:02.840 INFO:teuthology.orchestra.run.smithi169.stdout: "read_tier": -1, 2023-10-26T01:33:02.840 INFO:teuthology.orchestra.run.smithi169.stdout: "write_tier": -1, 2023-10-26T01:33:02.840 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_mode": "none", 2023-10-26T01:33:02.840 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_bytes": 0, 2023-10-26T01:33:02.841 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_objects": 0, 2023-10-26T01:33:02.841 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T01:33:02.841 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T01:33:02.841 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T01:33:02.841 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_flush_age": 0, 2023-10-26T01:33:02.842 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_evict_age": 0, 2023-10-26T01:33:02.842 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profile": "", 2023-10-26T01:33:02.842 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_params": { 2023-10-26T01:33:02.842 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "none" 2023-10-26T01:33:02.843 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:02.843 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_period": 0, 2023-10-26T01:33:02.843 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_count": 0, 2023-10-26T01:33:02.843 INFO:teuthology.orchestra.run.smithi169.stdout: "use_gmt_hitset": true, 2023-10-26T01:33:02.844 INFO:teuthology.orchestra.run.smithi169.stdout: "min_read_recency_for_promote": 0, 2023-10-26T01:33:02.844 INFO:teuthology.orchestra.run.smithi169.stdout: "min_write_recency_for_promote": 0, 2023-10-26T01:33:02.844 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T01:33:02.844 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_search_last_n": 0, 2023-10-26T01:33:02.844 INFO:teuthology.orchestra.run.smithi169.stdout: "grade_table": [], 2023-10-26T01:33:02.845 INFO:teuthology.orchestra.run.smithi169.stdout: "stripe_width": 0, 2023-10-26T01:33:02.845 INFO:teuthology.orchestra.run.smithi169.stdout: "expected_num_objects": 0, 2023-10-26T01:33:02.845 INFO:teuthology.orchestra.run.smithi169.stdout: "fast_read": false, 2023-10-26T01:33:02.845 INFO:teuthology.orchestra.run.smithi169.stdout: "options": { 2023-10-26T01:33:02.845 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_bias": 4, 2023-10-26T01:33:02.846 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_min": 16, 2023-10-26T01:33:02.846 INFO:teuthology.orchestra.run.smithi169.stdout: "recovery_priority": 5 2023-10-26T01:33:02.846 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:02.846 INFO:teuthology.orchestra.run.smithi169.stdout: "application_metadata": { 2023-10-26T01:33:02.847 INFO:teuthology.orchestra.run.smithi169.stdout: "cephfs": { 2023-10-26T01:33:02.847 INFO:teuthology.orchestra.run.smithi169.stdout: "metadata": "cephfs" 2023-10-26T01:33:02.847 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:02.847 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:02.848 INFO:teuthology.orchestra.run.smithi169.stdout: "read_balance": { 2023-10-26T01:33:02.848 INFO:teuthology.orchestra.run.smithi169.stdout: "score_acting": 1.75, 2023-10-26T01:33:02.848 INFO:teuthology.orchestra.run.smithi169.stdout: "score_stable": 1.75, 2023-10-26T01:33:02.848 INFO:teuthology.orchestra.run.smithi169.stdout: "optimal_score": 1, 2023-10-26T01:33:02.848 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_acting": 1.75, 2023-10-26T01:33:02.849 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_stable": 1.75, 2023-10-26T01:33:02.849 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity_weighted": 1, 2023-10-26T01:33:02.849 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity": 1, 2023-10-26T01:33:02.850 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity_weighted": 1 2023-10-26T01:33:02.850 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:02.850 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:02.850 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:02.851 INFO:teuthology.orchestra.run.smithi169.stdout: "pool": 7, 2023-10-26T01:33:02.851 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_name": "cephfs_data", 2023-10-26T01:33:02.851 INFO:teuthology.orchestra.run.smithi169.stdout: "create_time": "2023-10-26T01:32:37.059775+0000", 2023-10-26T01:33:02.851 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": 3, 2023-10-26T01:33:02.851 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_names": "hashpspool,full", 2023-10-26T01:33:02.852 INFO:teuthology.orchestra.run.smithi169.stdout: "type": 1, 2023-10-26T01:33:02.852 INFO:teuthology.orchestra.run.smithi169.stdout: "size": 2, 2023-10-26T01:33:02.852 INFO:teuthology.orchestra.run.smithi169.stdout: "min_size": 1, 2023-10-26T01:33:02.852 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_rule": 0, 2023-10-26T01:33:02.852 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_count": 0, 2023-10-26T01:33:02.853 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_target": 0, 2023-10-26T01:33:02.853 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T01:33:02.853 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T01:33:02.854 INFO:teuthology.orchestra.run.smithi169.stdout: "object_hash": 2, 2023-10-26T01:33:02.854 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_mode": "off", 2023-10-26T01:33:02.854 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num": 64, 2023-10-26T01:33:02.854 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num": 64, 2023-10-26T01:33:02.855 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num_target": 64, 2023-10-26T01:33:02.855 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_target": 64, 2023-10-26T01:33:02.856 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_pending": 64, 2023-10-26T01:33:02.856 INFO:teuthology.orchestra.run.smithi169.stdout: "last_pg_merge_meta": { 2023-10-26T01:33:02.856 INFO:teuthology.orchestra.run.smithi169.stdout: "source_pgid": "0.0", 2023-10-26T01:33:02.856 INFO:teuthology.orchestra.run.smithi169.stdout: "ready_epoch": 0, 2023-10-26T01:33:02.857 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_started": 0, 2023-10-26T01:33:02.857 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_clean": 0, 2023-10-26T01:33:02.857 INFO:teuthology.orchestra.run.smithi169.stdout: "source_version": "0'0", 2023-10-26T01:33:02.857 INFO:teuthology.orchestra.run.smithi169.stdout: "target_version": "0'0" 2023-10-26T01:33:02.857 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:02.858 INFO:teuthology.orchestra.run.smithi169.stdout: "last_change": "50", 2023-10-26T01:33:02.858 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend": "0", 2023-10-26T01:33:02.859 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T01:33:02.859 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T01:33:02.859 INFO:teuthology.orchestra.run.smithi169.stdout: "auid": 0, 2023-10-26T01:33:02.859 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_mode": "selfmanaged", 2023-10-26T01:33:02.860 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_seq": 0, 2023-10-26T01:33:02.860 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_epoch": 0, 2023-10-26T01:33:02.860 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_snaps": [], 2023-10-26T01:33:02.860 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps": "[]", 2023-10-26T01:33:02.861 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_bytes": 0, 2023-10-26T01:33:02.861 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_objects": 0, 2023-10-26T01:33:02.861 INFO:teuthology.orchestra.run.smithi169.stdout: "tiers": [], 2023-10-26T01:33:02.861 INFO:teuthology.orchestra.run.smithi169.stdout: "tier_of": -1, 2023-10-26T01:33:02.861 INFO:teuthology.orchestra.run.smithi169.stdout: "read_tier": -1, 2023-10-26T01:33:02.862 INFO:teuthology.orchestra.run.smithi169.stdout: "write_tier": -1, 2023-10-26T01:33:02.862 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_mode": "none", 2023-10-26T01:33:02.862 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_bytes": 0, 2023-10-26T01:33:02.862 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_objects": 0, 2023-10-26T01:33:02.863 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T01:33:02.863 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T01:33:02.863 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T01:33:02.863 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_flush_age": 0, 2023-10-26T01:33:02.864 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_evict_age": 0, 2023-10-26T01:33:02.864 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profile": "", 2023-10-26T01:33:02.864 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_params": { 2023-10-26T01:33:02.864 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "none" 2023-10-26T01:33:02.865 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:02.865 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_period": 0, 2023-10-26T01:33:02.865 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_count": 0, 2023-10-26T01:33:02.866 INFO:teuthology.orchestra.run.smithi169.stdout: "use_gmt_hitset": true, 2023-10-26T01:33:02.866 INFO:teuthology.orchestra.run.smithi169.stdout: "min_read_recency_for_promote": 0, 2023-10-26T01:33:02.866 INFO:teuthology.orchestra.run.smithi169.stdout: "min_write_recency_for_promote": 0, 2023-10-26T01:33:02.866 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T01:33:02.867 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_search_last_n": 0, 2023-10-26T01:33:02.867 INFO:teuthology.orchestra.run.smithi169.stdout: "grade_table": [], 2023-10-26T01:33:02.867 INFO:teuthology.orchestra.run.smithi169.stdout: "stripe_width": 0, 2023-10-26T01:33:02.868 INFO:teuthology.orchestra.run.smithi169.stdout: "expected_num_objects": 0, 2023-10-26T01:33:02.868 INFO:teuthology.orchestra.run.smithi169.stdout: "fast_read": false, 2023-10-26T01:33:02.868 INFO:teuthology.orchestra.run.smithi169.stdout: "options": { 2023-10-26T01:33:02.869 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_min": 64, 2023-10-26T01:33:02.869 INFO:teuthology.orchestra.run.smithi169.stdout: "target_size_ratio": 0.89999997615814209 2023-10-26T01:33:02.870 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:02.870 INFO:teuthology.orchestra.run.smithi169.stdout: "application_metadata": { 2023-10-26T01:33:02.870 INFO:teuthology.orchestra.run.smithi169.stdout: "cephfs": { 2023-10-26T01:33:02.871 INFO:teuthology.orchestra.run.smithi169.stdout: "data": "cephfs" 2023-10-26T01:33:02.871 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:02.872 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:02.872 INFO:teuthology.orchestra.run.smithi169.stdout: "read_balance": { 2023-10-26T01:33:02.872 INFO:teuthology.orchestra.run.smithi169.stdout: "score_acting": 1.75, 2023-10-26T01:33:02.873 INFO:teuthology.orchestra.run.smithi169.stdout: "score_stable": 1.75, 2023-10-26T01:33:02.873 INFO:teuthology.orchestra.run.smithi169.stdout: "optimal_score": 1, 2023-10-26T01:33:02.873 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_acting": 1.75, 2023-10-26T01:33:02.874 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_stable": 1.75, 2023-10-26T01:33:02.874 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity_weighted": 1, 2023-10-26T01:33:02.875 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity": 1, 2023-10-26T01:33:02.875 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity_weighted": 1 2023-10-26T01:33:02.875 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:02.876 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:02.876 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:33:02.876 INFO:teuthology.orchestra.run.smithi169.stdout: "osds": [ 2023-10-26T01:33:02.877 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:02.877 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 0, 2023-10-26T01:33:02.877 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:33:02.877 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:33:02.877 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:33:02.878 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:33:02.878 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:33:02.878 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:33:02.878 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:33:02.879 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:33:02.879 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 47, 2023-10-26T01:33:02.879 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:33:02.879 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:33:02.879 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:33:02.880 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:02.880 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:02.880 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:02.880 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6816", 2023-10-26T01:33:02.880 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:33:02.881 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:02.881 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:02.881 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:02.881 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6817", 2023-10-26T01:33:02.882 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:33:02.882 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:02.882 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:02.882 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:02.882 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:33:02.883 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:02.883 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:02.883 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:02.883 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6818", 2023-10-26T01:33:02.883 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:33:02.884 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:02.884 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:02.884 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:02.884 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6819", 2023-10-26T01:33:02.884 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:33:02.885 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:02.885 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:02.885 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:02.885 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:33:02.885 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:02.886 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:02.886 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:02.886 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6822", 2023-10-26T01:33:02.886 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:33:02.886 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:02.887 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:02.887 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:02.887 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6823", 2023-10-26T01:33:02.887 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:33:02.887 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:02.888 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:02.888 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:02.888 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:33:02.888 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:02.888 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:02.888 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:02.889 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6820", 2023-10-26T01:33:02.889 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:33:02.889 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:02.889 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:02.889 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:02.890 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6821", 2023-10-26T01:33:02.890 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:33:02.890 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:02.890 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:02.891 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:02.891 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6817/4204255864", 2023-10-26T01:33:02.891 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6819/4204255864", 2023-10-26T01:33:02.891 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6823/4204255864", 2023-10-26T01:33:02.891 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6821/4204255864", 2023-10-26T01:33:02.892 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:33:02.892 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:33:02.892 INFO:teuthology.orchestra.run.smithi169.stdout: "full", 2023-10-26T01:33:02.892 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:33:02.892 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:02.893 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:02.893 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:02.893 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 1, 2023-10-26T01:33:02.893 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:33:02.893 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:33:02.894 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:33:02.894 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:33:02.894 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:33:02.895 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:33:02.895 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:33:02.895 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:33:02.895 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 47, 2023-10-26T01:33:02.896 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:33:02.896 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:33:02.896 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:33:02.896 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:02.896 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:02.897 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:02.897 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6808", 2023-10-26T01:33:02.897 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:33:02.897 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:02.897 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:02.898 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:02.898 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6809", 2023-10-26T01:33:02.898 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:33:02.899 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:02.899 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:02.899 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:02.899 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:33:02.900 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:02.900 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:02.900 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:02.900 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6810", 2023-10-26T01:33:02.900 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:33:02.901 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:02.901 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:02.901 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:02.901 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6811", 2023-10-26T01:33:02.901 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:33:02.902 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:02.902 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:02.902 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:02.902 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:33:02.903 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:02.903 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:02.903 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:02.903 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6814", 2023-10-26T01:33:02.903 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:33:02.904 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:02.904 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:02.904 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:02.904 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6815", 2023-10-26T01:33:02.905 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:33:02.905 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:02.905 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:02.905 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:02.905 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:33:02.906 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:02.906 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:02.906 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:02.906 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6812", 2023-10-26T01:33:02.907 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:33:02.907 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:02.907 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:02.907 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:02.908 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6813", 2023-10-26T01:33:02.908 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:33:02.908 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:02.908 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:02.908 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:02.909 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6809/238206046", 2023-10-26T01:33:02.909 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6811/238206046", 2023-10-26T01:33:02.909 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6815/238206046", 2023-10-26T01:33:02.909 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6813/238206046", 2023-10-26T01:33:02.909 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:33:02.910 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:33:02.910 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:33:02.910 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:02.910 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:02.911 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:02.911 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 2, 2023-10-26T01:33:02.911 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:33:02.911 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:33:02.911 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:33:02.912 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:33:02.912 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:33:02.912 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:33:02.912 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:33:02.912 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:33:02.913 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 47, 2023-10-26T01:33:02.913 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:33:02.913 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:33:02.913 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:33:02.914 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:02.914 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:02.914 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:02.915 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6800", 2023-10-26T01:33:02.915 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:33:02.915 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:02.916 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:02.916 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:02.916 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6801", 2023-10-26T01:33:02.917 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:33:02.917 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:02.917 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:02.918 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:02.918 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:33:02.919 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:02.919 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:02.919 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:02.920 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6802", 2023-10-26T01:33:02.920 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:33:02.920 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:02.921 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:02.921 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:02.921 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6803", 2023-10-26T01:33:02.922 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:33:02.922 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:02.923 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:02.923 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:02.923 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:33:02.924 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:02.924 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:02.924 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:02.925 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6806", 2023-10-26T01:33:02.925 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:33:02.925 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:02.926 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:02.926 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:02.926 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6807", 2023-10-26T01:33:02.926 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:33:02.927 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:02.927 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:02.927 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:02.927 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:33:02.928 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:02.928 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:02.928 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:02.928 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6804", 2023-10-26T01:33:02.928 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:33:02.928 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:02.929 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:02.929 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:02.929 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6805", 2023-10-26T01:33:02.929 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:33:02.929 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:02.930 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:02.930 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:02.930 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6801/3215791846", 2023-10-26T01:33:02.930 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6803/3215791846", 2023-10-26T01:33:02.931 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6807/3215791846", 2023-10-26T01:33:02.931 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6805/3215791846", 2023-10-26T01:33:02.931 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:33:02.931 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:33:02.931 INFO:teuthology.orchestra.run.smithi169.stdout: "full", 2023-10-26T01:33:02.932 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:33:02.932 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:02.932 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:02.932 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:02.933 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 3, 2023-10-26T01:33:02.933 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:33:02.933 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:33:02.933 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:33:02.933 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:33:02.934 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:33:02.934 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:33:02.934 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:33:02.934 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 14, 2023-10-26T01:33:02.935 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 47, 2023-10-26T01:33:02.935 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:33:02.935 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:33:02.935 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:33:02.935 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:02.936 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:02.936 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:02.936 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6824", 2023-10-26T01:33:02.936 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:33:02.936 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:02.937 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:02.937 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:02.937 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6825", 2023-10-26T01:33:02.937 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:33:02.937 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:02.938 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:02.938 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:02.938 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:33:02.938 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:02.938 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:02.939 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:02.939 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6826", 2023-10-26T01:33:02.939 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:33:02.939 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:02.939 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:02.940 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:02.940 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6827", 2023-10-26T01:33:02.940 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:33:02.940 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:02.940 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:02.941 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:02.941 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:33:02.941 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:02.941 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:02.942 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:02.942 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6830", 2023-10-26T01:33:02.942 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:33:02.942 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:02.942 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:02.943 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:02.943 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6831", 2023-10-26T01:33:02.944 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:33:02.944 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:02.944 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:02.945 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:02.945 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:33:02.945 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:02.946 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:02.946 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:02.947 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6828", 2023-10-26T01:33:02.947 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:33:02.947 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:02.948 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:02.948 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:02.948 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6829", 2023-10-26T01:33:02.949 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:33:02.949 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:02.949 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:02.950 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:02.950 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6825/2256701226", 2023-10-26T01:33:02.950 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6827/2256701226", 2023-10-26T01:33:02.951 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6831/2256701226", 2023-10-26T01:33:02.951 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6829/2256701226", 2023-10-26T01:33:02.951 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:33:02.951 INFO:teuthology.orchestra.run.smithi169.stdout: "backfillfull", 2023-10-26T01:33:02.951 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:33:02.952 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:33:02.952 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:02.952 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:02.952 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:02.953 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 4, 2023-10-26T01:33:02.953 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:33:02.953 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:33:02.953 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:33:02.953 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:33:02.954 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:33:02.954 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:33:02.954 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:33:02.954 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:33:02.955 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 47, 2023-10-26T01:33:02.955 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:33:02.955 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:33:02.955 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:33:02.956 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:02.956 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:02.956 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:02.957 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6816", 2023-10-26T01:33:02.957 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:33:02.958 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:02.958 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:02.958 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:02.959 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6817", 2023-10-26T01:33:02.959 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:33:02.959 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:02.960 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:02.960 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:02.960 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:33:02.961 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:02.961 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:02.962 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:02.962 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6818", 2023-10-26T01:33:02.963 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:33:02.963 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:02.963 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:02.964 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:02.964 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6819", 2023-10-26T01:33:02.965 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:33:02.965 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:02.965 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:02.966 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:02.966 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:33:02.966 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:02.969 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:02.970 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:02.970 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6825", 2023-10-26T01:33:02.970 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:33:02.971 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:02.971 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:02.971 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:02.972 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6828", 2023-10-26T01:33:02.972 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:33:02.973 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:02.973 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:02.973 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:02.974 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:33:02.974 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:02.975 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:02.975 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:02.975 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6820", 2023-10-26T01:33:02.976 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:33:02.976 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:02.976 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:02.977 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:02.977 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6823", 2023-10-26T01:33:02.978 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:33:02.978 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:02.978 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:02.979 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:02.979 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6817/1082279874", 2023-10-26T01:33:02.980 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6819/1082279874", 2023-10-26T01:33:02.980 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6828/1082279874", 2023-10-26T01:33:02.980 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6823/1082279874", 2023-10-26T01:33:02.981 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:33:02.981 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:33:02.982 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:33:02.982 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:02.983 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:02.983 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:02.984 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 5, 2023-10-26T01:33:02.984 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:33:02.984 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:33:02.985 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:33:02.985 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:33:02.985 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:33:02.986 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:33:02.986 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:33:02.987 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:33:02.987 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 47, 2023-10-26T01:33:02.987 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:33:02.988 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:33:02.988 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:33:02.989 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:02.989 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:02.989 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:02.990 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6808", 2023-10-26T01:33:02.990 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:33:02.990 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:02.991 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:02.991 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:02.992 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6809", 2023-10-26T01:33:02.992 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:33:02.992 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:02.993 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:02.993 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:02.993 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:33:02.994 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:02.994 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:02.995 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:02.995 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6810", 2023-10-26T01:33:02.995 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:33:02.996 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:02.996 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:02.996 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:02.997 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6811", 2023-10-26T01:33:02.997 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:33:02.997 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:02.998 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:02.998 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:02.999 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:33:02.999 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:02.999 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:03.000 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:03.000 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6814", 2023-10-26T01:33:03.001 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:33:03.001 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:03.001 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:03.002 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:03.002 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6815", 2023-10-26T01:33:03.003 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:33:03.003 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:03.003 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:03.004 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:03.004 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:33:03.004 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:03.004 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:03.004 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:03.005 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6812", 2023-10-26T01:33:03.005 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:33:03.005 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:03.005 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:03.005 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:03.006 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6813", 2023-10-26T01:33:03.006 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:33:03.006 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:03.006 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:03.007 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:03.007 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6809/3335654351", 2023-10-26T01:33:03.007 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6811/3335654351", 2023-10-26T01:33:03.007 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6815/3335654351", 2023-10-26T01:33:03.008 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6813/3335654351", 2023-10-26T01:33:03.008 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:33:03.008 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:33:03.008 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:33:03.008 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:03.009 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:03.009 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:03.009 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 6, 2023-10-26T01:33:03.009 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:33:03.009 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:33:03.010 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:33:03.010 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:33:03.010 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:33:03.010 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:33:03.010 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:33:03.011 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:33:03.011 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 47, 2023-10-26T01:33:03.011 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:33:03.011 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:33:03.012 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:33:03.012 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:03.012 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:03.012 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:03.012 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6800", 2023-10-26T01:33:03.013 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:33:03.013 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:03.013 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:03.013 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:03.014 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6801", 2023-10-26T01:33:03.014 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:33:03.014 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:03.014 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:03.014 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:03.015 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:33:03.015 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:03.015 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:03.015 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:03.016 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6802", 2023-10-26T01:33:03.016 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:33:03.016 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:03.016 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:03.016 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:03.017 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6803", 2023-10-26T01:33:03.017 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:33:03.017 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:03.017 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:03.018 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:03.018 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:33:03.018 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:03.018 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:03.019 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:03.019 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6806", 2023-10-26T01:33:03.019 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:33:03.019 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:03.019 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:03.020 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:03.020 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6807", 2023-10-26T01:33:03.020 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:33:03.020 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:03.021 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:03.021 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:03.021 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:33:03.021 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:03.021 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:03.022 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:03.022 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6804", 2023-10-26T01:33:03.022 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:33:03.022 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:03.023 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:03.023 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:03.023 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6805", 2023-10-26T01:33:03.023 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:33:03.024 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:03.024 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:03.024 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:03.024 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6801/2520460697", 2023-10-26T01:33:03.024 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6803/2520460697", 2023-10-26T01:33:03.025 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6807/2520460697", 2023-10-26T01:33:03.025 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6805/2520460697", 2023-10-26T01:33:03.025 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:33:03.025 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:33:03.025 INFO:teuthology.orchestra.run.smithi169.stdout: "full", 2023-10-26T01:33:03.026 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:33:03.026 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:03.026 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:03.026 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:03.027 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 7, 2023-10-26T01:33:03.027 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:33:03.027 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:33:03.027 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:33:03.028 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:33:03.028 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:33:03.028 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:33:03.028 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:33:03.028 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:33:03.029 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 47, 2023-10-26T01:33:03.029 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:33:03.029 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:33:03.029 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:33:03.029 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:03.030 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:03.030 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:03.030 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6821", 2023-10-26T01:33:03.030 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:33:03.030 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:03.031 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:03.031 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:03.031 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6822", 2023-10-26T01:33:03.031 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:33:03.032 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:03.032 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:03.032 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:03.032 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:33:03.032 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:03.033 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:03.033 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:03.033 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6824", 2023-10-26T01:33:03.033 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:33:03.033 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:03.034 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:03.034 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:03.034 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6826", 2023-10-26T01:33:03.034 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:33:03.034 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:03.035 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:03.035 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:03.035 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:33:03.035 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:03.035 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:03.036 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:03.036 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6830", 2023-10-26T01:33:03.036 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:33:03.036 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:03.036 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:03.037 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:03.037 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6831", 2023-10-26T01:33:03.037 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:33:03.037 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:03.038 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:03.038 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:03.038 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:33:03.038 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:03.039 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:03.039 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:03.039 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6827", 2023-10-26T01:33:03.039 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:33:03.039 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:03.040 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:03.040 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:03.040 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6829", 2023-10-26T01:33:03.040 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:33:03.040 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:03.041 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:03.041 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:03.041 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6822/1115930619", 2023-10-26T01:33:03.041 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6826/1115930619", 2023-10-26T01:33:03.041 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6831/1115930619", 2023-10-26T01:33:03.042 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6829/1115930619", 2023-10-26T01:33:03.042 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:33:03.042 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:33:03.042 INFO:teuthology.orchestra.run.smithi169.stdout: "full", 2023-10-26T01:33:03.043 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:33:03.043 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:03.043 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:03.043 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:33:03.044 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_xinfo": [ 2023-10-26T01:33:03.044 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:03.044 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 0, 2023-10-26T01:33:03.044 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:33:03.044 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:33:03.045 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:33:03.045 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:33:03.045 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:33:03.045 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.889675+0000", 2023-10-26T01:33:03.045 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:33:03.046 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:03.046 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:03.046 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 1, 2023-10-26T01:33:03.046 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:33:03.047 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:33:03.047 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:33:03.047 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:33:03.047 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:33:03.047 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.864157+0000", 2023-10-26T01:33:03.048 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:33:03.048 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:03.048 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:03.048 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 2, 2023-10-26T01:33:03.049 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:33:03.049 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:33:03.049 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:33:03.049 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:33:03.050 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:33:03.050 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.876919+0000", 2023-10-26T01:33:03.050 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:33:03.050 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:03.050 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:03.051 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 3, 2023-10-26T01:33:03.051 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:33:03.051 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:33:03.051 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:33:03.052 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:33:03.052 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:33:03.052 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.974541+0000", 2023-10-26T01:33:03.052 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:33:03.052 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:03.053 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:03.053 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 4, 2023-10-26T01:33:03.053 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:33:03.053 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:33:03.053 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:33:03.054 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:33:03.054 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:33:03.054 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.969696+0000", 2023-10-26T01:33:03.054 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:33:03.055 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:03.055 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:03.055 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 5, 2023-10-26T01:33:03.055 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:33:03.056 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:33:03.056 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:33:03.056 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:33:03.056 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:33:03.056 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.859703+0000", 2023-10-26T01:33:03.057 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:33:03.057 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:03.057 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:03.057 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 6, 2023-10-26T01:33:03.058 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:33:03.058 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:33:03.058 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:33:03.058 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:33:03.059 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:33:03.059 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.867917+0000", 2023-10-26T01:33:03.059 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:33:03.059 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:03.059 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:03.060 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 7, 2023-10-26T01:33:03.060 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:33:03.060 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:33:03.060 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:33:03.060 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:33:03.061 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:33:03.061 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.898605+0000", 2023-10-26T01:33:03.061 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:33:03.061 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:03.061 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:33:03.062 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_upmap": [], 2023-10-26T01:33:03.062 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_upmap_items": [], 2023-10-26T01:33:03.062 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_upmap_primaries": [], 2023-10-26T01:33:03.062 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_temp": [], 2023-10-26T01:33:03.063 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_temp": [], 2023-10-26T01:33:03.063 INFO:teuthology.orchestra.run.smithi169.stdout: "blocklist": {}, 2023-10-26T01:33:03.063 INFO:teuthology.orchestra.run.smithi169.stdout: "range_blocklist": {}, 2023-10-26T01:33:03.063 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profiles": { 2023-10-26T01:33:03.063 INFO:teuthology.orchestra.run.smithi169.stdout: "default": { 2023-10-26T01:33:03.064 INFO:teuthology.orchestra.run.smithi169.stdout: "crush-failure-domain": "osd", 2023-10-26T01:33:03.064 INFO:teuthology.orchestra.run.smithi169.stdout: "k": "2", 2023-10-26T01:33:03.064 INFO:teuthology.orchestra.run.smithi169.stdout: "m": "1", 2023-10-26T01:33:03.064 INFO:teuthology.orchestra.run.smithi169.stdout: "plugin": "jerasure", 2023-10-26T01:33:03.064 INFO:teuthology.orchestra.run.smithi169.stdout: "technique": "reed_sol_van" 2023-10-26T01:33:03.065 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:03.065 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:03.065 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps_queue": [], 2023-10-26T01:33:03.065 INFO:teuthology.orchestra.run.smithi169.stdout: "new_removed_snaps": [], 2023-10-26T01:33:03.065 INFO:teuthology.orchestra.run.smithi169.stdout: "new_purged_snaps": [], 2023-10-26T01:33:03.066 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_node_flags": {}, 2023-10-26T01:33:03.066 INFO:teuthology.orchestra.run.smithi169.stdout: "device_class_flags": {}, 2023-10-26T01:33:03.066 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_mode": { 2023-10-26T01:33:03.066 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_mode_enabled": false, 2023-10-26T01:33:03.066 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_bucket_count": 0, 2023-10-26T01:33:03.067 INFO:teuthology.orchestra.run.smithi169.stdout: "degraded_stretch_mode": 0, 2023-10-26T01:33:03.067 INFO:teuthology.orchestra.run.smithi169.stdout: "recovering_stretch_mode": 0, 2023-10-26T01:33:03.067 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_mode_bucket": 0 2023-10-26T01:33:03.067 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:03.068 INFO:teuthology.orchestra.run.smithi169.stdout:} 2023-10-26T01:33:03.070 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:02.800+0000 7f3bc30e5640 1 -- 172.21.15.169:0/3870057347 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3ba006ae40 msgr2=0x7f3ba006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:03.070 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:02.800+0000 7f3bc30e5640 1 --2- 172.21.15.169:0/3870057347 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3ba006ae40 0x7f3ba006d300 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc06c490 tx=0x7f3bb0006c90 comp rx=0 tx=0).stop 2023-10-26T01:33:03.070 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:02.800+0000 7f3bc30e5640 1 -- 172.21.15.169:0/3870057347 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3bbc14e790 msgr2=0x7f3bbc14ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:03.071 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:02.800+0000 7f3bc30e5640 1 --2- 172.21.15.169:0/3870057347 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3bbc14e790 0x7f3bbc14ed60 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f3ba800b3e0 tx=0x7f3ba800b8a0 comp rx=0 tx=0).stop 2023-10-26T01:33:03.071 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:02.801+0000 7f3bc30e5640 1 -- 172.21.15.169:0/3870057347 shutdown_connections 2023-10-26T01:33:03.071 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:02.801+0000 7f3bc30e5640 1 --2- 172.21.15.169:0/3870057347 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3ba006ae40 0x7f3ba006d300 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:03.071 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:02.801+0000 7f3bc30e5640 1 --2- 172.21.15.169:0/3870057347 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3bbc14e790 0x7f3bbc14ed60 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:03.072 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:02.801+0000 7f3bc30e5640 1 --2- 172.21.15.169:0/3870057347 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3bbc158850 0x7f3bbc154b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:03.072 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:02.801+0000 7f3bc30e5640 1 --2- 172.21.15.169:0/3870057347 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3bbc0cde30 0x7f3bbc153820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:03.072 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:02.801+0000 7f3bc30e5640 1 -- 172.21.15.169:0/3870057347 >> 172.21.15.169:0/3870057347 conn(0x7f3bbc069930 msgr2=0x7f3bbc14d9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:03.072 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:02.801+0000 7f3bc30e5640 1 -- 172.21.15.169:0/3870057347 shutdown_connections 2023-10-26T01:33:03.073 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:02.801+0000 7f3bc30e5640 1 -- 172.21.15.169:0/3870057347 wait complete. 2023-10-26T01:33:03.073 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2023-10-26T01:33:03.074 INFO:tasks.ceph.mon.a.smithi169.stderr:2023-10-26T01:33:02.916+0000 7f158b279640 -1 log_channel(cluster) log [ERR] : Health check update: 4 full osd(s) (OSD_FULL) 2023-10-26T01:33:04.792 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T01:33:04.801 DEBUG:teuthology.orchestra.run.smithi170:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T01:33:07.920 INFO:tasks.ceph.mon.a.smithi169.stderr:2023-10-26T01:33:07.918+0000 7f158b279640 -1 log_channel(cluster) log [ERR] : Health check update: 1 full osd(s) (OSD_FULL) 2023-10-26T01:33:08.074 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-26T01:33:08.304 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:08.301+0000 7f814b84f640 1 -- 172.21.15.169:0/3751464782 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81300051d0 msgr2=0x7f813000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:08.305 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:08.301+0000 7f814b84f640 1 --2- 172.21.15.169:0/3751464782 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81300051d0 0x7f813000b980 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f813c005fd0 tx=0x7f81300057f0 comp rx=0 tx=0).stop 2023-10-26T01:33:08.305 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:08.301+0000 7f8149dec640 1 -- 172.21.15.169:0/3751464782 reap_dead start 2023-10-26T01:33:08.305 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:08.301+0000 7f814b84f640 1 -- 172.21.15.169:0/3751464782 shutdown_connections 2023-10-26T01:33:08.305 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:08.301+0000 7f814b84f640 1 -- 172.21.15.169:0/3751464782 >> 172.21.15.169:0/3751464782 conn(0x7f8144069930 msgr2=0x7f8144069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:08.306 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:08.302+0000 7f814b84f640 1 -- 172.21.15.169:0/3751464782 shutdown_connections 2023-10-26T01:33:08.306 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:08.302+0000 7f814b84f640 1 -- 172.21.15.169:0/3751464782 wait complete. 2023-10-26T01:33:08.307 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:08.303+0000 7f814b84f640 1 Processor -- start 2023-10-26T01:33:08.308 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:08.304+0000 7f814b84f640 1 -- start start 2023-10-26T01:33:08.308 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:08.304+0000 7f814b84f640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f81300051d0 0x7f8144147620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:08.308 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:08.304+0000 7f814b84f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8144155a10 0x7f814415b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:08.308 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:08.304+0000 7f814b84f640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f814415b830 0x7f8144153100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:08.308 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:08.304+0000 7f814b84f640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f814415bcb0 con 0x7f814415b830 2023-10-26T01:33:08.309 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:08.304+0000 7f814b84f640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81441541c0 con 0x7f8144155a10 2023-10-26T01:33:08.309 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:08.304+0000 7f814b84f640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8144147b60 con 0x7f81300051d0 2023-10-26T01:33:08.309 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:08.305+0000 7f8149dec640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f814415b830 0x7f8144153100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:08.309 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:08.305+0000 7f8149dec640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f814415b830 0x7f8144153100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:60630/0 (socket says 172.21.15.169:60630) 2023-10-26T01:33:08.309 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:08.305+0000 7f8149dec640 1 -- 172.21.15.169:0/2136255624 learned_addr learned my addr 172.21.15.169:0/2136255624 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:33:08.310 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:08.305+0000 7f81495eb640 1 --2- 172.21.15.169:0/2136255624 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f81300051d0 0x7f8144147620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:08.310 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:08.305+0000 7f8148dea640 1 --2- 172.21.15.169:0/2136255624 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8144155a10 0x7f814415b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:08.311 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:08.305+0000 7f8149dec640 1 -- 172.21.15.169:0/2136255624 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f81300051d0 msgr2=0x7f8144147620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:08.312 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:08.305+0000 7f8149dec640 1 --2- 172.21.15.169:0/2136255624 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f81300051d0 0x7f8144147620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:08.312 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:08.305+0000 7f8149dec640 1 -- 172.21.15.169:0/2136255624 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8144155a10 msgr2=0x7f814415b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:08.312 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:08.305+0000 7f8149dec640 1 --2- 172.21.15.169:0/2136255624 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8144155a10 0x7f814415b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:08.312 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:08.305+0000 7f8149dec640 1 -- 172.21.15.169:0/2136255624 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8144154460 con 0x7f814415b830 2023-10-26T01:33:08.313 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:08.305+0000 7f81495eb640 1 --2- 172.21.15.169:0/2136255624 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f81300051d0 0x7f8144147620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T01:33:08.313 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:08.306+0000 7f8149dec640 1 --2- 172.21.15.169:0/2136255624 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f814415b830 0x7f8144153100 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f813c009e40 tx=0x7f813c035b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:08.313 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:08.306+0000 7f8139ffb640 1 -- 172.21.15.169:0/2136255624 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f813c045070 con 0x7f814415b830 2023-10-26T01:33:08.313 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:08.306+0000 7f8139ffb640 1 -- 172.21.15.169:0/2136255624 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f813c004b20 con 0x7f814415b830 2023-10-26T01:33:08.314 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:08.306+0000 7f8139ffb640 1 -- 172.21.15.169:0/2136255624 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f813c004d30 con 0x7f814415b830 2023-10-26T01:33:08.314 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:08.306+0000 7f814b84f640 1 -- 172.21.15.169:0/2136255624 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8144154740 con 0x7f814415b830 2023-10-26T01:33:08.314 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:08.306+0000 7f814b84f640 1 -- 172.21.15.169:0/2136255624 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8144154d00 con 0x7f814415b830 2023-10-26T01:33:08.314 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:08.307+0000 7f814b84f640 1 -- 172.21.15.169:0/2136255624 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f814404db10 con 0x7f814415b830 2023-10-26T01:33:08.315 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:08.308+0000 7f8139ffb640 1 -- 172.21.15.169:0/2136255624 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f813c03e450 con 0x7f814415b830 2023-10-26T01:33:08.315 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:08.309+0000 7f8139ffb640 1 --2- 172.21.15.169:0/2136255624 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f812806ae40 0x7f812806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:08.315 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:08.309+0000 7f8139ffb640 1 -- 172.21.15.169:0/2136255624 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f813c0abd90 con 0x7f814415b830 2023-10-26T01:33:08.316 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:08.309+0000 7f8139ffb640 1 -- 172.21.15.169:0/2136255624 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=54}) v3 -- 0x7f8128072970 con 0x7f814415b830 2023-10-26T01:33:08.316 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:08.310+0000 7f81495eb640 1 --2- 172.21.15.169:0/2136255624 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f812806ae40 0x7f812806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:08.316 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:08.311+0000 7f81495eb640 1 --2- 172.21.15.169:0/2136255624 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f812806ae40 0x7f812806d300 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f8140006630 tx=0x7f8140006bc0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:08.317 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:08.314+0000 7f8139ffb640 1 -- 172.21.15.169:0/2136255624 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f813c076e30 con 0x7f814415b830 2023-10-26T01:33:08.506 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:08.503+0000 7f814b84f640 1 -- 172.21.15.169:0/2136255624 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f81440ccf90 con 0x7f814415b830 2023-10-26T01:33:08.509 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:08.506+0000 7f8139ffb640 1 -- 172.21.15.169:0/2136255624 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v53) v1 ==== 81+0+34035 (secure 0 0 0) 0x7f813c07a450 con 0x7f814415b830 2023-10-26T01:33:08.509 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:33:08.510 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2023-10-26T01:33:08.510 INFO:teuthology.orchestra.run.smithi169.stdout: "epoch": 53, 2023-10-26T01:33:08.510 INFO:teuthology.orchestra.run.smithi169.stdout: "fsid": "30203b28-ed8a-44e8-a9a6-a9bf25f980f1", 2023-10-26T01:33:08.510 INFO:teuthology.orchestra.run.smithi169.stdout: "created": "2023-10-26T01:30:22.897155+0000", 2023-10-26T01:33:08.511 INFO:teuthology.orchestra.run.smithi169.stdout: "modified": "2023-10-26T01:33:07.173036+0000", 2023-10-26T01:33:08.511 INFO:teuthology.orchestra.run.smithi169.stdout: "last_up_change": "2023-10-26T01:30:31.875281+0000", 2023-10-26T01:33:08.511 INFO:teuthology.orchestra.run.smithi169.stdout: "last_in_change": "2023-10-26T01:30:27.637941+0000", 2023-10-26T01:33:08.511 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-26T01:33:08.511 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_num": 5799936, 2023-10-26T01:33:08.512 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_set": [ 2023-10-26T01:33:08.512 INFO:teuthology.orchestra.run.smithi169.stdout: "pglog_hardlimit", 2023-10-26T01:33:08.512 INFO:teuthology.orchestra.run.smithi169.stdout: "purged_snapdirs", 2023-10-26T01:33:08.512 INFO:teuthology.orchestra.run.smithi169.stdout: "recovery_deletes", 2023-10-26T01:33:08.512 INFO:teuthology.orchestra.run.smithi169.stdout: "sortbitwise" 2023-10-26T01:33:08.512 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:33:08.513 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_version": 4, 2023-10-26T01:33:08.513 INFO:teuthology.orchestra.run.smithi169.stdout: "full_ratio": 0.69999998807907104, 2023-10-26T01:33:08.513 INFO:teuthology.orchestra.run.smithi169.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-26T01:33:08.513 INFO:teuthology.orchestra.run.smithi169.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-26T01:33:08.513 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_snapshot": "", 2023-10-26T01:33:08.514 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_max": 7, 2023-10-26T01:33:08.514 INFO:teuthology.orchestra.run.smithi169.stdout: "max_osd": 8, 2023-10-26T01:33:08.514 INFO:teuthology.orchestra.run.smithi169.stdout: "require_min_compat_client": "luminous", 2023-10-26T01:33:08.514 INFO:teuthology.orchestra.run.smithi169.stdout: "min_compat_client": "jewel", 2023-10-26T01:33:08.514 INFO:teuthology.orchestra.run.smithi169.stdout: "require_osd_release": "reef", 2023-10-26T01:33:08.515 INFO:teuthology.orchestra.run.smithi169.stdout: "allow_crimson": false, 2023-10-26T01:33:08.515 INFO:teuthology.orchestra.run.smithi169.stdout: "pools": [ 2023-10-26T01:33:08.515 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:08.515 INFO:teuthology.orchestra.run.smithi169.stdout: "pool": 1, 2023-10-26T01:33:08.515 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_name": ".mgr", 2023-10-26T01:33:08.516 INFO:teuthology.orchestra.run.smithi169.stdout: "create_time": "2023-10-26T01:30:32.200808+0000", 2023-10-26T01:33:08.516 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": 3, 2023-10-26T01:33:08.516 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_names": "hashpspool,full", 2023-10-26T01:33:08.516 INFO:teuthology.orchestra.run.smithi169.stdout: "type": 1, 2023-10-26T01:33:08.516 INFO:teuthology.orchestra.run.smithi169.stdout: "size": 2, 2023-10-26T01:33:08.516 INFO:teuthology.orchestra.run.smithi169.stdout: "min_size": 1, 2023-10-26T01:33:08.517 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_rule": 0, 2023-10-26T01:33:08.517 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_count": 0, 2023-10-26T01:33:08.517 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_target": 0, 2023-10-26T01:33:08.517 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T01:33:08.517 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T01:33:08.518 INFO:teuthology.orchestra.run.smithi169.stdout: "object_hash": 2, 2023-10-26T01:33:08.518 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_mode": "off", 2023-10-26T01:33:08.518 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num": 1, 2023-10-26T01:33:08.518 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num": 1, 2023-10-26T01:33:08.518 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num_target": 1, 2023-10-26T01:33:08.519 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_target": 1, 2023-10-26T01:33:08.519 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_pending": 1, 2023-10-26T01:33:08.519 INFO:teuthology.orchestra.run.smithi169.stdout: "last_pg_merge_meta": { 2023-10-26T01:33:08.519 INFO:teuthology.orchestra.run.smithi169.stdout: "source_pgid": "0.0", 2023-10-26T01:33:08.519 INFO:teuthology.orchestra.run.smithi169.stdout: "ready_epoch": 0, 2023-10-26T01:33:08.519 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_started": 0, 2023-10-26T01:33:08.520 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_clean": 0, 2023-10-26T01:33:08.520 INFO:teuthology.orchestra.run.smithi169.stdout: "source_version": "0'0", 2023-10-26T01:33:08.520 INFO:teuthology.orchestra.run.smithi169.stdout: "target_version": "0'0" 2023-10-26T01:33:08.520 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:08.520 INFO:teuthology.orchestra.run.smithi169.stdout: "last_change": "50", 2023-10-26T01:33:08.521 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend": "0", 2023-10-26T01:33:08.521 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T01:33:08.521 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T01:33:08.521 INFO:teuthology.orchestra.run.smithi169.stdout: "auid": 0, 2023-10-26T01:33:08.521 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_mode": "selfmanaged", 2023-10-26T01:33:08.522 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_seq": 0, 2023-10-26T01:33:08.522 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_epoch": 0, 2023-10-26T01:33:08.522 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_snaps": [], 2023-10-26T01:33:08.522 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps": "[]", 2023-10-26T01:33:08.522 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_bytes": 0, 2023-10-26T01:33:08.523 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_objects": 0, 2023-10-26T01:33:08.523 INFO:teuthology.orchestra.run.smithi169.stdout: "tiers": [], 2023-10-26T01:33:08.523 INFO:teuthology.orchestra.run.smithi169.stdout: "tier_of": -1, 2023-10-26T01:33:08.523 INFO:teuthology.orchestra.run.smithi169.stdout: "read_tier": -1, 2023-10-26T01:33:08.523 INFO:teuthology.orchestra.run.smithi169.stdout: "write_tier": -1, 2023-10-26T01:33:08.524 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_mode": "none", 2023-10-26T01:33:08.524 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_bytes": 0, 2023-10-26T01:33:08.524 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_objects": 0, 2023-10-26T01:33:08.524 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T01:33:08.524 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T01:33:08.525 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T01:33:08.525 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_flush_age": 0, 2023-10-26T01:33:08.525 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_evict_age": 0, 2023-10-26T01:33:08.525 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profile": "", 2023-10-26T01:33:08.525 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_params": { 2023-10-26T01:33:08.526 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "none" 2023-10-26T01:33:08.526 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:08.526 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_period": 0, 2023-10-26T01:33:08.526 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_count": 0, 2023-10-26T01:33:08.526 INFO:teuthology.orchestra.run.smithi169.stdout: "use_gmt_hitset": true, 2023-10-26T01:33:08.527 INFO:teuthology.orchestra.run.smithi169.stdout: "min_read_recency_for_promote": 0, 2023-10-26T01:33:08.527 INFO:teuthology.orchestra.run.smithi169.stdout: "min_write_recency_for_promote": 0, 2023-10-26T01:33:08.527 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T01:33:08.527 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_search_last_n": 0, 2023-10-26T01:33:08.527 INFO:teuthology.orchestra.run.smithi169.stdout: "grade_table": [], 2023-10-26T01:33:08.527 INFO:teuthology.orchestra.run.smithi169.stdout: "stripe_width": 0, 2023-10-26T01:33:08.528 INFO:teuthology.orchestra.run.smithi169.stdout: "expected_num_objects": 0, 2023-10-26T01:33:08.528 INFO:teuthology.orchestra.run.smithi169.stdout: "fast_read": false, 2023-10-26T01:33:08.528 INFO:teuthology.orchestra.run.smithi169.stdout: "options": { 2023-10-26T01:33:08.528 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_max": 32, 2023-10-26T01:33:08.528 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_min": 1 2023-10-26T01:33:08.529 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:08.529 INFO:teuthology.orchestra.run.smithi169.stdout: "application_metadata": { 2023-10-26T01:33:08.529 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": {} 2023-10-26T01:33:08.529 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:08.529 INFO:teuthology.orchestra.run.smithi169.stdout: "read_balance": { 2023-10-26T01:33:08.530 INFO:teuthology.orchestra.run.smithi169.stdout: "score_acting": 8, 2023-10-26T01:33:08.530 INFO:teuthology.orchestra.run.smithi169.stdout: "score_stable": 8, 2023-10-26T01:33:08.530 INFO:teuthology.orchestra.run.smithi169.stdout: "optimal_score": 0.25, 2023-10-26T01:33:08.530 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_acting": 2, 2023-10-26T01:33:08.530 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_stable": 2, 2023-10-26T01:33:08.530 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity_weighted": 1, 2023-10-26T01:33:08.531 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity": 1, 2023-10-26T01:33:08.531 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity_weighted": 1 2023-10-26T01:33:08.531 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:08.531 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:08.531 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:08.532 INFO:teuthology.orchestra.run.smithi169.stdout: "pool": 6, 2023-10-26T01:33:08.532 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_name": "cephfs_metadata", 2023-10-26T01:33:08.532 INFO:teuthology.orchestra.run.smithi169.stdout: "create_time": "2023-10-26T01:32:36.288217+0000", 2023-10-26T01:33:08.532 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": 3, 2023-10-26T01:33:08.532 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_names": "hashpspool,full", 2023-10-26T01:33:08.533 INFO:teuthology.orchestra.run.smithi169.stdout: "type": 1, 2023-10-26T01:33:08.533 INFO:teuthology.orchestra.run.smithi169.stdout: "size": 2, 2023-10-26T01:33:08.533 INFO:teuthology.orchestra.run.smithi169.stdout: "min_size": 1, 2023-10-26T01:33:08.533 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_rule": 0, 2023-10-26T01:33:08.533 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_count": 0, 2023-10-26T01:33:08.533 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_target": 0, 2023-10-26T01:33:08.534 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T01:33:08.534 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T01:33:08.534 INFO:teuthology.orchestra.run.smithi169.stdout: "object_hash": 2, 2023-10-26T01:33:08.534 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_mode": "off", 2023-10-26T01:33:08.534 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num": 32, 2023-10-26T01:33:08.535 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num": 32, 2023-10-26T01:33:08.535 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num_target": 32, 2023-10-26T01:33:08.535 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_target": 32, 2023-10-26T01:33:08.535 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_pending": 32, 2023-10-26T01:33:08.535 INFO:teuthology.orchestra.run.smithi169.stdout: "last_pg_merge_meta": { 2023-10-26T01:33:08.536 INFO:teuthology.orchestra.run.smithi169.stdout: "source_pgid": "0.0", 2023-10-26T01:33:08.536 INFO:teuthology.orchestra.run.smithi169.stdout: "ready_epoch": 0, 2023-10-26T01:33:08.536 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_started": 0, 2023-10-26T01:33:08.536 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_clean": 0, 2023-10-26T01:33:08.536 INFO:teuthology.orchestra.run.smithi169.stdout: "source_version": "0'0", 2023-10-26T01:33:08.536 INFO:teuthology.orchestra.run.smithi169.stdout: "target_version": "0'0" 2023-10-26T01:33:08.537 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:08.537 INFO:teuthology.orchestra.run.smithi169.stdout: "last_change": "50", 2023-10-26T01:33:08.537 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend": "0", 2023-10-26T01:33:08.537 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T01:33:08.537 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T01:33:08.538 INFO:teuthology.orchestra.run.smithi169.stdout: "auid": 0, 2023-10-26T01:33:08.538 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_mode": "selfmanaged", 2023-10-26T01:33:08.538 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_seq": 0, 2023-10-26T01:33:08.538 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_epoch": 0, 2023-10-26T01:33:08.538 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_snaps": [], 2023-10-26T01:33:08.539 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps": "[]", 2023-10-26T01:33:08.539 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_bytes": 0, 2023-10-26T01:33:08.539 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_objects": 0, 2023-10-26T01:33:08.539 INFO:teuthology.orchestra.run.smithi169.stdout: "tiers": [], 2023-10-26T01:33:08.539 INFO:teuthology.orchestra.run.smithi169.stdout: "tier_of": -1, 2023-10-26T01:33:08.539 INFO:teuthology.orchestra.run.smithi169.stdout: "read_tier": -1, 2023-10-26T01:33:08.540 INFO:teuthology.orchestra.run.smithi169.stdout: "write_tier": -1, 2023-10-26T01:33:08.540 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_mode": "none", 2023-10-26T01:33:08.540 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_bytes": 0, 2023-10-26T01:33:08.540 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_objects": 0, 2023-10-26T01:33:08.541 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T01:33:08.541 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T01:33:08.541 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T01:33:08.541 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_flush_age": 0, 2023-10-26T01:33:08.541 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_evict_age": 0, 2023-10-26T01:33:08.542 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profile": "", 2023-10-26T01:33:08.542 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_params": { 2023-10-26T01:33:08.542 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "none" 2023-10-26T01:33:08.542 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:08.542 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_period": 0, 2023-10-26T01:33:08.543 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_count": 0, 2023-10-26T01:33:08.543 INFO:teuthology.orchestra.run.smithi169.stdout: "use_gmt_hitset": true, 2023-10-26T01:33:08.543 INFO:teuthology.orchestra.run.smithi169.stdout: "min_read_recency_for_promote": 0, 2023-10-26T01:33:08.543 INFO:teuthology.orchestra.run.smithi169.stdout: "min_write_recency_for_promote": 0, 2023-10-26T01:33:08.543 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T01:33:08.544 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_search_last_n": 0, 2023-10-26T01:33:08.544 INFO:teuthology.orchestra.run.smithi169.stdout: "grade_table": [], 2023-10-26T01:33:08.544 INFO:teuthology.orchestra.run.smithi169.stdout: "stripe_width": 0, 2023-10-26T01:33:08.544 INFO:teuthology.orchestra.run.smithi169.stdout: "expected_num_objects": 0, 2023-10-26T01:33:08.544 INFO:teuthology.orchestra.run.smithi169.stdout: "fast_read": false, 2023-10-26T01:33:08.545 INFO:teuthology.orchestra.run.smithi169.stdout: "options": { 2023-10-26T01:33:08.545 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_bias": 4, 2023-10-26T01:33:08.545 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_min": 16, 2023-10-26T01:33:08.545 INFO:teuthology.orchestra.run.smithi169.stdout: "recovery_priority": 5 2023-10-26T01:33:08.545 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:08.546 INFO:teuthology.orchestra.run.smithi169.stdout: "application_metadata": { 2023-10-26T01:33:08.546 INFO:teuthology.orchestra.run.smithi169.stdout: "cephfs": { 2023-10-26T01:33:08.546 INFO:teuthology.orchestra.run.smithi169.stdout: "metadata": "cephfs" 2023-10-26T01:33:08.546 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:08.546 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:08.547 INFO:teuthology.orchestra.run.smithi169.stdout: "read_balance": { 2023-10-26T01:33:08.547 INFO:teuthology.orchestra.run.smithi169.stdout: "score_acting": 1.75, 2023-10-26T01:33:08.547 INFO:teuthology.orchestra.run.smithi169.stdout: "score_stable": 1.75, 2023-10-26T01:33:08.547 INFO:teuthology.orchestra.run.smithi169.stdout: "optimal_score": 1, 2023-10-26T01:33:08.547 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_acting": 1.75, 2023-10-26T01:33:08.547 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_stable": 1.75, 2023-10-26T01:33:08.548 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity_weighted": 1, 2023-10-26T01:33:08.548 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity": 1, 2023-10-26T01:33:08.548 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity_weighted": 1 2023-10-26T01:33:08.548 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:08.548 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:08.549 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:08.549 INFO:teuthology.orchestra.run.smithi169.stdout: "pool": 7, 2023-10-26T01:33:08.549 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_name": "cephfs_data", 2023-10-26T01:33:08.549 INFO:teuthology.orchestra.run.smithi169.stdout: "create_time": "2023-10-26T01:32:37.059775+0000", 2023-10-26T01:33:08.550 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": 3, 2023-10-26T01:33:08.550 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_names": "hashpspool,full", 2023-10-26T01:33:08.550 INFO:teuthology.orchestra.run.smithi169.stdout: "type": 1, 2023-10-26T01:33:08.550 INFO:teuthology.orchestra.run.smithi169.stdout: "size": 2, 2023-10-26T01:33:08.550 INFO:teuthology.orchestra.run.smithi169.stdout: "min_size": 1, 2023-10-26T01:33:08.551 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_rule": 0, 2023-10-26T01:33:08.551 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_count": 0, 2023-10-26T01:33:08.551 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_target": 0, 2023-10-26T01:33:08.551 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T01:33:08.551 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T01:33:08.551 INFO:teuthology.orchestra.run.smithi169.stdout: "object_hash": 2, 2023-10-26T01:33:08.552 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_mode": "off", 2023-10-26T01:33:08.552 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num": 64, 2023-10-26T01:33:08.552 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num": 64, 2023-10-26T01:33:08.552 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num_target": 64, 2023-10-26T01:33:08.552 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_target": 64, 2023-10-26T01:33:08.553 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_pending": 64, 2023-10-26T01:33:08.553 INFO:teuthology.orchestra.run.smithi169.stdout: "last_pg_merge_meta": { 2023-10-26T01:33:08.553 INFO:teuthology.orchestra.run.smithi169.stdout: "source_pgid": "0.0", 2023-10-26T01:33:08.553 INFO:teuthology.orchestra.run.smithi169.stdout: "ready_epoch": 0, 2023-10-26T01:33:08.553 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_started": 0, 2023-10-26T01:33:08.554 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_clean": 0, 2023-10-26T01:33:08.554 INFO:teuthology.orchestra.run.smithi169.stdout: "source_version": "0'0", 2023-10-26T01:33:08.554 INFO:teuthology.orchestra.run.smithi169.stdout: "target_version": "0'0" 2023-10-26T01:33:08.554 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:08.554 INFO:teuthology.orchestra.run.smithi169.stdout: "last_change": "50", 2023-10-26T01:33:08.554 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend": "0", 2023-10-26T01:33:08.555 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T01:33:08.555 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T01:33:08.555 INFO:teuthology.orchestra.run.smithi169.stdout: "auid": 0, 2023-10-26T01:33:08.555 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_mode": "selfmanaged", 2023-10-26T01:33:08.555 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_seq": 0, 2023-10-26T01:33:08.556 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_epoch": 0, 2023-10-26T01:33:08.556 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_snaps": [], 2023-10-26T01:33:08.556 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps": "[]", 2023-10-26T01:33:08.556 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_bytes": 0, 2023-10-26T01:33:08.556 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_objects": 0, 2023-10-26T01:33:08.557 INFO:teuthology.orchestra.run.smithi169.stdout: "tiers": [], 2023-10-26T01:33:08.557 INFO:teuthology.orchestra.run.smithi169.stdout: "tier_of": -1, 2023-10-26T01:33:08.557 INFO:teuthology.orchestra.run.smithi169.stdout: "read_tier": -1, 2023-10-26T01:33:08.557 INFO:teuthology.orchestra.run.smithi169.stdout: "write_tier": -1, 2023-10-26T01:33:08.557 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_mode": "none", 2023-10-26T01:33:08.558 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_bytes": 0, 2023-10-26T01:33:08.558 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_objects": 0, 2023-10-26T01:33:08.558 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T01:33:08.558 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T01:33:08.558 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T01:33:08.558 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_flush_age": 0, 2023-10-26T01:33:08.559 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_evict_age": 0, 2023-10-26T01:33:08.559 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profile": "", 2023-10-26T01:33:08.559 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_params": { 2023-10-26T01:33:08.559 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "none" 2023-10-26T01:33:08.559 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:08.560 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_period": 0, 2023-10-26T01:33:08.560 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_count": 0, 2023-10-26T01:33:08.560 INFO:teuthology.orchestra.run.smithi169.stdout: "use_gmt_hitset": true, 2023-10-26T01:33:08.560 INFO:teuthology.orchestra.run.smithi169.stdout: "min_read_recency_for_promote": 0, 2023-10-26T01:33:08.560 INFO:teuthology.orchestra.run.smithi169.stdout: "min_write_recency_for_promote": 0, 2023-10-26T01:33:08.561 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T01:33:08.561 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_search_last_n": 0, 2023-10-26T01:33:08.561 INFO:teuthology.orchestra.run.smithi169.stdout: "grade_table": [], 2023-10-26T01:33:08.561 INFO:teuthology.orchestra.run.smithi169.stdout: "stripe_width": 0, 2023-10-26T01:33:08.561 INFO:teuthology.orchestra.run.smithi169.stdout: "expected_num_objects": 0, 2023-10-26T01:33:08.562 INFO:teuthology.orchestra.run.smithi169.stdout: "fast_read": false, 2023-10-26T01:33:08.562 INFO:teuthology.orchestra.run.smithi169.stdout: "options": { 2023-10-26T01:33:08.562 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_min": 64, 2023-10-26T01:33:08.562 INFO:teuthology.orchestra.run.smithi169.stdout: "target_size_ratio": 0.89999997615814209 2023-10-26T01:33:08.562 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:08.563 INFO:teuthology.orchestra.run.smithi169.stdout: "application_metadata": { 2023-10-26T01:33:08.563 INFO:teuthology.orchestra.run.smithi169.stdout: "cephfs": { 2023-10-26T01:33:08.563 INFO:teuthology.orchestra.run.smithi169.stdout: "data": "cephfs" 2023-10-26T01:33:08.563 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:08.563 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:08.564 INFO:teuthology.orchestra.run.smithi169.stdout: "read_balance": { 2023-10-26T01:33:08.564 INFO:teuthology.orchestra.run.smithi169.stdout: "score_acting": 1.75, 2023-10-26T01:33:08.564 INFO:teuthology.orchestra.run.smithi169.stdout: "score_stable": 1.75, 2023-10-26T01:33:08.564 INFO:teuthology.orchestra.run.smithi169.stdout: "optimal_score": 1, 2023-10-26T01:33:08.564 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_acting": 1.75, 2023-10-26T01:33:08.565 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_stable": 1.75, 2023-10-26T01:33:08.565 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity_weighted": 1, 2023-10-26T01:33:08.565 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity": 1, 2023-10-26T01:33:08.565 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity_weighted": 1 2023-10-26T01:33:08.565 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:08.566 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:08.566 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:33:08.566 INFO:teuthology.orchestra.run.smithi169.stdout: "osds": [ 2023-10-26T01:33:08.566 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:08.566 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 0, 2023-10-26T01:33:08.566 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:33:08.567 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:33:08.567 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:33:08.567 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:33:08.567 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:33:08.567 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:33:08.568 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:33:08.568 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:33:08.568 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 47, 2023-10-26T01:33:08.568 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:33:08.568 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:33:08.569 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:33:08.569 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:08.569 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:08.569 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:08.569 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6816", 2023-10-26T01:33:08.570 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:33:08.570 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:08.570 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:08.570 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:08.570 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6817", 2023-10-26T01:33:08.571 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:33:08.571 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:08.571 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:08.571 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:08.571 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:33:08.572 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:08.572 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:08.572 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:08.572 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6818", 2023-10-26T01:33:08.572 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:33:08.573 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:08.573 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:08.573 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:08.573 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6819", 2023-10-26T01:33:08.573 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:33:08.574 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:08.574 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:08.574 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:08.574 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:33:08.574 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:08.575 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:08.575 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:08.575 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6822", 2023-10-26T01:33:08.575 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:33:08.575 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:08.575 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:08.576 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:08.576 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6823", 2023-10-26T01:33:08.576 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:33:08.576 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:08.577 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:08.577 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:08.577 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:33:08.577 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:08.577 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:08.578 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:08.578 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6820", 2023-10-26T01:33:08.578 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:33:08.578 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:08.578 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:08.579 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:08.579 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6821", 2023-10-26T01:33:08.579 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:33:08.579 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:08.579 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:08.580 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:08.580 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6817/4204255864", 2023-10-26T01:33:08.580 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6819/4204255864", 2023-10-26T01:33:08.580 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6823/4204255864", 2023-10-26T01:33:08.580 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6821/4204255864", 2023-10-26T01:33:08.581 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:33:08.581 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:33:08.581 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:33:08.581 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:08.581 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:08.581 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:08.582 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 1, 2023-10-26T01:33:08.582 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:33:08.582 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:33:08.582 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:33:08.583 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:33:08.583 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:33:08.583 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:33:08.583 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:33:08.583 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:33:08.583 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 47, 2023-10-26T01:33:08.584 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:33:08.584 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:33:08.584 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:33:08.584 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:08.584 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:08.585 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:08.585 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6808", 2023-10-26T01:33:08.585 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:33:08.585 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:08.585 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:08.586 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:08.586 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6809", 2023-10-26T01:33:08.586 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:33:08.586 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:08.587 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:08.587 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:08.587 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:33:08.587 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:08.587 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:08.587 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:08.588 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6810", 2023-10-26T01:33:08.588 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:33:08.588 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:08.588 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:08.588 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:08.589 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6811", 2023-10-26T01:33:08.589 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:33:08.589 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:08.589 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:08.589 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:08.590 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:33:08.590 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:08.590 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:08.590 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:08.590 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6814", 2023-10-26T01:33:08.591 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:33:08.591 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:08.591 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:08.591 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:08.591 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6815", 2023-10-26T01:33:08.591 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:33:08.592 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:08.592 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:08.592 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:08.592 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:33:08.592 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:08.593 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:08.593 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:08.593 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6812", 2023-10-26T01:33:08.593 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:33:08.593 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:08.594 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:08.594 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:08.594 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6813", 2023-10-26T01:33:08.594 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:33:08.594 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:08.594 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:08.595 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:08.595 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6809/238206046", 2023-10-26T01:33:08.595 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6811/238206046", 2023-10-26T01:33:08.595 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6815/238206046", 2023-10-26T01:33:08.595 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6813/238206046", 2023-10-26T01:33:08.596 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:33:08.596 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:33:08.596 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:33:08.596 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:08.596 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:08.596 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:08.597 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 2, 2023-10-26T01:33:08.597 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:33:08.597 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:33:08.597 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:33:08.597 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:33:08.598 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:33:08.598 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:33:08.598 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:33:08.598 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:33:08.598 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 47, 2023-10-26T01:33:08.599 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:33:08.599 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:33:08.599 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:33:08.599 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:08.599 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:08.600 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:08.600 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6800", 2023-10-26T01:33:08.600 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:33:08.600 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:08.600 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:08.601 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:08.601 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6801", 2023-10-26T01:33:08.601 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:33:08.601 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:08.601 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:08.602 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:08.602 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:33:08.602 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:08.602 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:08.602 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:08.602 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6802", 2023-10-26T01:33:08.603 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:33:08.603 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:08.603 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:08.603 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:08.603 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6803", 2023-10-26T01:33:08.604 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:33:08.604 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:08.604 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:08.604 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:08.604 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:33:08.605 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:08.605 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:08.605 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:08.605 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6806", 2023-10-26T01:33:08.605 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:33:08.606 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:08.606 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:08.606 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:08.606 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6807", 2023-10-26T01:33:08.606 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:33:08.607 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:08.607 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:08.607 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:08.607 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:33:08.607 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:08.608 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:08.608 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:08.608 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6804", 2023-10-26T01:33:08.608 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:33:08.608 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:08.609 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:08.609 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:08.609 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6805", 2023-10-26T01:33:08.609 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:33:08.609 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:08.610 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:08.610 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:08.610 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6801/3215791846", 2023-10-26T01:33:08.610 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6803/3215791846", 2023-10-26T01:33:08.610 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6807/3215791846", 2023-10-26T01:33:08.610 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6805/3215791846", 2023-10-26T01:33:08.611 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:33:08.611 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:33:08.611 INFO:teuthology.orchestra.run.smithi169.stdout: "full", 2023-10-26T01:33:08.611 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:33:08.611 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:08.612 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:08.612 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:08.612 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 3, 2023-10-26T01:33:08.612 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:33:08.612 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:33:08.613 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:33:08.613 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:33:08.613 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:33:08.613 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:33:08.613 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:33:08.613 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 14, 2023-10-26T01:33:08.614 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 47, 2023-10-26T01:33:08.614 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:33:08.614 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:33:08.614 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:33:08.614 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:08.615 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:08.615 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:08.615 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6824", 2023-10-26T01:33:08.615 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:33:08.615 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:08.616 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:08.616 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:08.616 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6825", 2023-10-26T01:33:08.616 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:33:08.616 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:08.616 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:08.617 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:08.617 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:33:08.617 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:08.617 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:08.617 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:08.618 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6826", 2023-10-26T01:33:08.618 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:33:08.618 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:08.618 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:08.618 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:08.619 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6827", 2023-10-26T01:33:08.619 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:33:08.619 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:08.619 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:08.619 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:08.620 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:33:08.620 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:08.620 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:08.620 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:08.620 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6830", 2023-10-26T01:33:08.620 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:33:08.621 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:08.621 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:08.621 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:08.621 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6831", 2023-10-26T01:33:08.621 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:33:08.622 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:08.622 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:08.622 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:08.622 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:33:08.622 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:08.622 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:08.623 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:08.623 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6828", 2023-10-26T01:33:08.623 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:33:08.623 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:08.623 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:08.624 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:08.624 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6829", 2023-10-26T01:33:08.624 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:33:08.624 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:08.624 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:08.624 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:08.625 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6825/2256701226", 2023-10-26T01:33:08.625 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6827/2256701226", 2023-10-26T01:33:08.625 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6831/2256701226", 2023-10-26T01:33:08.625 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6829/2256701226", 2023-10-26T01:33:08.625 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:33:08.626 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:33:08.626 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:33:08.626 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:08.626 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:08.626 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:08.627 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 4, 2023-10-26T01:33:08.627 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:33:08.627 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:33:08.627 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:33:08.627 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:33:08.627 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:33:08.628 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:33:08.628 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:33:08.628 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:33:08.628 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 47, 2023-10-26T01:33:08.628 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:33:08.629 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:33:08.629 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:33:08.629 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:08.629 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:08.629 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:08.630 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6816", 2023-10-26T01:33:08.630 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:33:08.630 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:08.630 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:08.630 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:08.631 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6817", 2023-10-26T01:33:08.631 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:33:08.631 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:08.631 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:08.631 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:08.631 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:33:08.632 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:08.632 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:08.632 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:08.632 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6818", 2023-10-26T01:33:08.632 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:33:08.633 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:08.633 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:08.633 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:08.633 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6819", 2023-10-26T01:33:08.633 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:33:08.634 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:08.634 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:08.634 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:08.634 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:33:08.634 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:08.635 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:08.635 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:08.635 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6825", 2023-10-26T01:33:08.635 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:33:08.635 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:08.635 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:08.636 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:08.636 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6828", 2023-10-26T01:33:08.636 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:33:08.636 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:08.636 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:08.636 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:08.637 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:33:08.637 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:08.637 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:08.637 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:08.637 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6820", 2023-10-26T01:33:08.637 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:33:08.638 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:08.638 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:08.638 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:08.638 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6823", 2023-10-26T01:33:08.638 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:33:08.639 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:08.639 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:08.639 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:08.639 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6817/1082279874", 2023-10-26T01:33:08.639 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6819/1082279874", 2023-10-26T01:33:08.639 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6828/1082279874", 2023-10-26T01:33:08.640 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6823/1082279874", 2023-10-26T01:33:08.640 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:33:08.640 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:33:08.640 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:33:08.640 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:08.641 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:08.641 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:08.641 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 5, 2023-10-26T01:33:08.641 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:33:08.641 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:33:08.641 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:33:08.642 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:33:08.642 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:33:08.642 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:33:08.642 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:33:08.642 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:33:08.643 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 47, 2023-10-26T01:33:08.643 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:33:08.643 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:33:08.643 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:33:08.643 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:08.643 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:08.644 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:08.644 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6808", 2023-10-26T01:33:08.644 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:33:08.644 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:08.644 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:08.645 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:08.645 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6809", 2023-10-26T01:33:08.645 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:33:08.645 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:08.645 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:08.645 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:08.646 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:33:08.646 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:08.646 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:08.646 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:08.646 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6810", 2023-10-26T01:33:08.647 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:33:08.647 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:08.647 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:08.647 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:08.647 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6811", 2023-10-26T01:33:08.647 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:33:08.648 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:08.648 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:08.648 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:08.648 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:33:08.648 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:08.648 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:08.649 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:08.649 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6814", 2023-10-26T01:33:08.649 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:33:08.649 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:08.649 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:08.649 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:08.650 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6815", 2023-10-26T01:33:08.650 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:33:08.650 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:08.650 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:08.650 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:08.651 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:33:08.651 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:08.651 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:08.651 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:08.651 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6812", 2023-10-26T01:33:08.652 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:33:08.652 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:08.652 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:08.652 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:08.652 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6813", 2023-10-26T01:33:08.653 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:33:08.653 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:08.653 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:08.653 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:08.653 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6809/3335654351", 2023-10-26T01:33:08.653 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6811/3335654351", 2023-10-26T01:33:08.654 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6815/3335654351", 2023-10-26T01:33:08.654 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6813/3335654351", 2023-10-26T01:33:08.654 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:33:08.654 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:33:08.654 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:33:08.655 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:08.655 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:08.655 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:08.655 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 6, 2023-10-26T01:33:08.655 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:33:08.656 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:33:08.656 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:33:08.656 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:33:08.656 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:33:08.656 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:33:08.657 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:33:08.657 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:33:08.657 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 47, 2023-10-26T01:33:08.657 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:33:08.657 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:33:08.657 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:33:08.658 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:08.658 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:08.658 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:08.658 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6800", 2023-10-26T01:33:08.658 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:33:08.659 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:08.659 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:08.659 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:08.659 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6801", 2023-10-26T01:33:08.659 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:33:08.659 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:08.660 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:08.660 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:08.660 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:33:08.660 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:08.660 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:08.660 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:08.661 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6802", 2023-10-26T01:33:08.661 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:33:08.661 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:08.661 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:08.661 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:08.661 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6803", 2023-10-26T01:33:08.662 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:33:08.662 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:08.662 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:08.662 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:08.662 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:33:08.663 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:08.663 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:08.663 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:08.663 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6806", 2023-10-26T01:33:08.663 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:33:08.664 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:08.664 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:08.664 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:08.664 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6807", 2023-10-26T01:33:08.664 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:33:08.664 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:08.665 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:08.665 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:08.665 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:33:08.665 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:08.665 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:08.666 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:08.666 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6804", 2023-10-26T01:33:08.666 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:33:08.666 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:08.666 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:08.666 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:08.667 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6805", 2023-10-26T01:33:08.667 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:33:08.667 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:08.667 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:08.667 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:08.668 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6801/2520460697", 2023-10-26T01:33:08.668 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6803/2520460697", 2023-10-26T01:33:08.668 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6807/2520460697", 2023-10-26T01:33:08.668 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6805/2520460697", 2023-10-26T01:33:08.668 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:33:08.668 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:33:08.669 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:33:08.669 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:08.669 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:08.669 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:08.669 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 7, 2023-10-26T01:33:08.669 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:33:08.670 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:33:08.670 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:33:08.670 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:33:08.670 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:33:08.670 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:33:08.670 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:33:08.671 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:33:08.671 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 47, 2023-10-26T01:33:08.671 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:33:08.671 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:33:08.671 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:33:08.672 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:08.672 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:08.672 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:08.672 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6821", 2023-10-26T01:33:08.672 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:33:08.672 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:08.673 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:08.673 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:08.673 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6822", 2023-10-26T01:33:08.673 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:33:08.673 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:08.673 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:08.674 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:08.674 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:33:08.674 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:08.674 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:08.674 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:08.675 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6824", 2023-10-26T01:33:08.675 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:33:08.675 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:08.675 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:08.675 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:08.675 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6826", 2023-10-26T01:33:08.675 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:33:08.676 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:08.676 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:08.676 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:08.676 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:33:08.676 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:08.677 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:08.677 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:08.677 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6830", 2023-10-26T01:33:08.677 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:33:08.677 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:08.677 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:08.678 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:08.678 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6831", 2023-10-26T01:33:08.678 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:33:08.678 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:08.678 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:08.679 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:08.679 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:33:08.679 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:08.679 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:08.679 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:08.679 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6827", 2023-10-26T01:33:08.680 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:33:08.680 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:08.680 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:08.680 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:08.680 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6829", 2023-10-26T01:33:08.681 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:33:08.681 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:08.681 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:08.681 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:08.681 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6822/1115930619", 2023-10-26T01:33:08.681 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6826/1115930619", 2023-10-26T01:33:08.682 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6831/1115930619", 2023-10-26T01:33:08.682 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6829/1115930619", 2023-10-26T01:33:08.682 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:33:08.682 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:33:08.682 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:33:08.683 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:08.683 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:08.683 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:33:08.683 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_xinfo": [ 2023-10-26T01:33:08.683 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:08.684 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 0, 2023-10-26T01:33:08.684 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:33:08.684 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:33:08.684 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:33:08.684 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:33:08.684 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:33:08.685 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.889675+0000", 2023-10-26T01:33:08.685 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:33:08.685 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:08.685 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:08.685 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 1, 2023-10-26T01:33:08.686 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:33:08.686 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:33:08.686 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:33:08.686 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:33:08.686 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:33:08.687 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.864157+0000", 2023-10-26T01:33:08.687 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:33:08.687 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:08.687 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:08.687 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 2, 2023-10-26T01:33:08.688 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:33:08.688 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:33:08.688 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:33:08.688 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:33:08.688 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:33:08.688 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.876919+0000", 2023-10-26T01:33:08.689 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:33:08.689 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:08.689 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:08.689 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 3, 2023-10-26T01:33:08.689 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:33:08.689 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:33:08.690 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:33:08.690 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:33:08.690 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:33:08.690 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.974541+0000", 2023-10-26T01:33:08.690 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:33:08.691 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:08.691 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:08.691 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 4, 2023-10-26T01:33:08.691 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:33:08.691 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:33:08.691 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:33:08.692 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:33:08.692 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:33:08.692 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.969696+0000", 2023-10-26T01:33:08.692 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:33:08.692 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:08.692 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:08.693 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 5, 2023-10-26T01:33:08.693 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:33:08.693 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:33:08.693 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:33:08.693 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:33:08.693 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:33:08.694 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.859703+0000", 2023-10-26T01:33:08.694 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:33:08.694 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:08.694 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:08.694 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 6, 2023-10-26T01:33:08.695 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:33:08.695 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:33:08.695 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:33:08.695 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:33:08.695 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:33:08.696 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.867917+0000", 2023-10-26T01:33:08.696 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:33:08.696 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:08.696 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:08.696 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 7, 2023-10-26T01:33:08.697 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:33:08.697 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:33:08.697 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:33:08.697 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:33:08.697 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:33:08.697 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.898605+0000", 2023-10-26T01:33:08.698 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:33:08.698 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:08.698 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:33:08.698 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_upmap": [], 2023-10-26T01:33:08.698 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_upmap_items": [], 2023-10-26T01:33:08.699 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_upmap_primaries": [], 2023-10-26T01:33:08.699 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_temp": [], 2023-10-26T01:33:08.699 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_temp": [], 2023-10-26T01:33:08.699 INFO:teuthology.orchestra.run.smithi169.stdout: "blocklist": {}, 2023-10-26T01:33:08.699 INFO:teuthology.orchestra.run.smithi169.stdout: "range_blocklist": {}, 2023-10-26T01:33:08.700 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profiles": { 2023-10-26T01:33:08.700 INFO:teuthology.orchestra.run.smithi169.stdout: "default": { 2023-10-26T01:33:08.700 INFO:teuthology.orchestra.run.smithi169.stdout: "crush-failure-domain": "osd", 2023-10-26T01:33:08.700 INFO:teuthology.orchestra.run.smithi169.stdout: "k": "2", 2023-10-26T01:33:08.700 INFO:teuthology.orchestra.run.smithi169.stdout: "m": "1", 2023-10-26T01:33:08.701 INFO:teuthology.orchestra.run.smithi169.stdout: "plugin": "jerasure", 2023-10-26T01:33:08.701 INFO:teuthology.orchestra.run.smithi169.stdout: "technique": "reed_sol_van" 2023-10-26T01:33:08.701 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:08.701 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:08.701 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps_queue": [], 2023-10-26T01:33:08.702 INFO:teuthology.orchestra.run.smithi169.stdout: "new_removed_snaps": [], 2023-10-26T01:33:08.702 INFO:teuthology.orchestra.run.smithi169.stdout: "new_purged_snaps": [], 2023-10-26T01:33:08.702 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_node_flags": {}, 2023-10-26T01:33:08.702 INFO:teuthology.orchestra.run.smithi169.stdout: "device_class_flags": {}, 2023-10-26T01:33:08.702 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_mode": { 2023-10-26T01:33:08.702 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_mode_enabled": false, 2023-10-26T01:33:08.703 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_bucket_count": 0, 2023-10-26T01:33:08.703 INFO:teuthology.orchestra.run.smithi169.stdout: "degraded_stretch_mode": 0, 2023-10-26T01:33:08.703 INFO:teuthology.orchestra.run.smithi169.stdout: "recovering_stretch_mode": 0, 2023-10-26T01:33:08.703 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_mode_bucket": 0 2023-10-26T01:33:08.704 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:08.705 INFO:teuthology.orchestra.run.smithi169.stdout:} 2023-10-26T01:33:08.707 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:08.509+0000 7f814b84f640 1 -- 172.21.15.169:0/2136255624 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f812806ae40 msgr2=0x7f812806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:08.707 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:08.509+0000 7f814b84f640 1 --2- 172.21.15.169:0/2136255624 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f812806ae40 0x7f812806d300 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f8140006630 tx=0x7f8140006bc0 comp rx=0 tx=0).stop 2023-10-26T01:33:08.707 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:08.509+0000 7f814b84f640 1 -- 172.21.15.169:0/2136255624 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f814415b830 msgr2=0x7f8144153100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:08.708 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:08.509+0000 7f814b84f640 1 --2- 172.21.15.169:0/2136255624 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f814415b830 0x7f8144153100 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f813c009e40 tx=0x7f813c035b70 comp rx=0 tx=0).stop 2023-10-26T01:33:08.708 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:08.509+0000 7f814b84f640 1 -- 172.21.15.169:0/2136255624 shutdown_connections 2023-10-26T01:33:08.708 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:08.509+0000 7f814b84f640 1 --2- 172.21.15.169:0/2136255624 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f812806ae40 0x7f812806d300 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:08.708 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:08.509+0000 7f814b84f640 1 --2- 172.21.15.169:0/2136255624 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f814415b830 0x7f8144153100 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:08.708 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:08.509+0000 7f814b84f640 1 --2- 172.21.15.169:0/2136255624 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8144155a10 0x7f814415b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:08.709 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:08.509+0000 7f814b84f640 1 --2- 172.21.15.169:0/2136255624 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f81300051d0 0x7f8144147620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:08.709 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:08.509+0000 7f814b84f640 1 -- 172.21.15.169:0/2136255624 >> 172.21.15.169:0/2136255624 conn(0x7f8144069930 msgr2=0x7f81441613a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:08.709 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:08.509+0000 7f814b84f640 1 -- 172.21.15.169:0/2136255624 shutdown_connections 2023-10-26T01:33:08.709 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:08.510+0000 7f814b84f640 1 -- 172.21.15.169:0/2136255624 wait complete. 2023-10-26T01:33:08.710 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2023-10-26T01:33:13.711 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-26T01:33:13.943 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:13.940+0000 7fe4c1f54640 1 -- 172.21.15.169:0/1389840160 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe4bc06a990 msgr2=0x7fe4bc0c96f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:13.943 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:13.940+0000 7fe4c1f54640 1 --2- 172.21.15.169:0/1389840160 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe4bc06a990 0x7fe4bc0c96f0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fe4a8005fd0 tx=0x7fe4a802bd90 comp rx=0 tx=0).stop 2023-10-26T01:33:13.943 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:13.940+0000 7fe4c1f54640 1 -- 172.21.15.169:0/1389840160 shutdown_connections 2023-10-26T01:33:13.944 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:13.940+0000 7fe4c1f54640 1 --2- 172.21.15.169:0/1389840160 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe4bc0c9c30 0x7fe4bc0cb390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:13.944 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:13.940+0000 7fe4c1f54640 1 --2- 172.21.15.169:0/1389840160 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe4bc06a990 0x7fe4bc0c96f0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:13.944 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:13.940+0000 7fe4c1f54640 1 -- 172.21.15.169:0/1389840160 >> 172.21.15.169:0/1389840160 conn(0x7fe4bc0aedd0 msgr2=0x7fe4bc0af1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:13.944 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:13.940+0000 7fe4c1f54640 1 -- 172.21.15.169:0/1389840160 shutdown_connections 2023-10-26T01:33:13.944 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:13.940+0000 7fe4c1f54640 1 -- 172.21.15.169:0/1389840160 wait complete. 2023-10-26T01:33:13.945 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:13.942+0000 7fe4c1f54640 1 Processor -- start 2023-10-26T01:33:13.945 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:13.942+0000 7fe4c1f54640 1 -- start start 2023-10-26T01:33:13.946 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:13.943+0000 7fe4c1f54640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe4bc06a990 0x7fe4bc0b1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:13.947 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:13.943+0000 7fe4c1f54640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe4bc0c7ea0 0x7fe4bc0afce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:13.947 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:13.943+0000 7fe4c1f54640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe4bc0c9c30 0x7fe4bc154970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:13.947 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:13.943+0000 7fe4c1f54640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4bc1639d0 con 0x7fe4bc0c7ea0 2023-10-26T01:33:13.947 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:13.943+0000 7fe4c1f54640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4bc14cfe0 con 0x7fe4bc06a990 2023-10-26T01:33:13.947 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:13.943+0000 7fe4c1f54640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4bc14d2c0 con 0x7fe4bc0c9c30 2023-10-26T01:33:13.948 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:13.943+0000 7fe4bbfff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe4bc0c7ea0 0x7fe4bc0afce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:13.948 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:13.943+0000 7fe4bbfff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe4bc0c7ea0 0x7fe4bc0afce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:60642/0 (socket says 172.21.15.169:60642) 2023-10-26T01:33:13.948 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:13.943+0000 7fe4bbfff640 1 -- 172.21.15.169:0/3451348275 learned_addr learned my addr 172.21.15.169:0/3451348275 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:33:13.948 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:13.944+0000 7fe4c0f52640 1 --2- 172.21.15.169:0/3451348275 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe4bc06a990 0x7fe4bc0b1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:13.948 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:13.944+0000 7fe4c1753640 1 --2- 172.21.15.169:0/3451348275 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe4bc0c9c30 0x7fe4bc154970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:13.949 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:13.944+0000 7fe4bbfff640 1 -- 172.21.15.169:0/3451348275 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe4bc0c9c30 msgr2=0x7fe4bc154970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:13.949 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:13.944+0000 7fe4bbfff640 1 --2- 172.21.15.169:0/3451348275 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe4bc0c9c30 0x7fe4bc154970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:13.949 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:13.944+0000 7fe4bbfff640 1 -- 172.21.15.169:0/3451348275 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe4bc06a990 msgr2=0x7fe4bc0b1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:13.949 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:13.944+0000 7fe4bbfff640 1 --2- 172.21.15.169:0/3451348275 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe4bc06a990 0x7fe4bc0b1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:13.949 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:13.944+0000 7fe4bbfff640 1 -- 172.21.15.169:0/3451348275 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4a8009560 con 0x7fe4bc0c7ea0 2023-10-26T01:33:13.950 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:13.944+0000 7fe4c1753640 1 --2- 172.21.15.169:0/3451348275 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe4bc0c9c30 0x7fe4bc154970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:33:13.950 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:13.944+0000 7fe4bbfff640 1 --2- 172.21.15.169:0/3451348275 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe4bc0c7ea0 0x7fe4bc0afce0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fe4a8009850 tx=0x7fe4a802db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:13.951 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:13.944+0000 7fe4b97fa640 1 -- 172.21.15.169:0/3451348275 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4a80035f0 con 0x7fe4bc0c7ea0 2023-10-26T01:33:13.951 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:13.944+0000 7fe4b97fa640 1 -- 172.21.15.169:0/3451348275 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe4a8003750 con 0x7fe4bc0c7ea0 2023-10-26T01:33:13.951 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:13.945+0000 7fe4b97fa640 1 -- 172.21.15.169:0/3451348275 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4a8003b00 con 0x7fe4bc0c7ea0 2023-10-26T01:33:13.952 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:13.945+0000 7fe4c1f54640 1 -- 172.21.15.169:0/3451348275 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4bc14d5a0 con 0x7fe4bc0c7ea0 2023-10-26T01:33:13.952 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:13.945+0000 7fe4c1f54640 1 -- 172.21.15.169:0/3451348275 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4bc14db60 con 0x7fe4bc0c7ea0 2023-10-26T01:33:13.952 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:13.946+0000 7fe4c1f54640 1 -- 172.21.15.169:0/3451348275 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4bc0cca20 con 0x7fe4bc0c7ea0 2023-10-26T01:33:13.952 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:13.947+0000 7fe4b97fa640 1 -- 172.21.15.169:0/3451348275 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fe4a8003c60 con 0x7fe4bc0c7ea0 2023-10-26T01:33:13.952 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:13.948+0000 7fe4b97fa640 1 --2- 172.21.15.169:0/3451348275 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe4a006ae40 0x7fe4a006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:13.953 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:13.948+0000 7fe4b97fa640 1 -- 172.21.15.169:0/3451348275 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe4a80ac240 con 0x7fe4bc0c7ea0 2023-10-26T01:33:13.953 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:13.948+0000 7fe4c0f52640 1 --2- 172.21.15.169:0/3451348275 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe4a006ae40 0x7fe4a006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:13.953 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:13.950+0000 7fe4c0f52640 1 --2- 172.21.15.169:0/3451348275 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe4a006ae40 0x7fe4a006d300 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe4b0004ea0 tx=0x7fe4b0009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:13.955 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:13.953+0000 7fe4b97fa640 1 -- 172.21.15.169:0/3451348275 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe4a8077380 con 0x7fe4bc0c7ea0 2023-10-26T01:33:14.147 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:14.145+0000 7fe4c1f54640 1 -- 172.21.15.169:0/3451348275 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fe4bc0ca150 con 0x7fe4bc0c7ea0 2023-10-26T01:33:14.150 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:14.148+0000 7fe4b97fa640 1 -- 172.21.15.169:0/3451348275 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v54) v1 ==== 81+0+33996 (secure 0 0 0) 0x7fe4a807a9a0 con 0x7fe4bc0c7ea0 2023-10-26T01:33:14.151 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:33:14.152 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2023-10-26T01:33:14.152 INFO:teuthology.orchestra.run.smithi169.stdout: "epoch": 54, 2023-10-26T01:33:14.152 INFO:teuthology.orchestra.run.smithi169.stdout: "fsid": "30203b28-ed8a-44e8-a9a6-a9bf25f980f1", 2023-10-26T01:33:14.152 INFO:teuthology.orchestra.run.smithi169.stdout: "created": "2023-10-26T01:30:22.897155+0000", 2023-10-26T01:33:14.152 INFO:teuthology.orchestra.run.smithi169.stdout: "modified": "2023-10-26T01:33:10.190714+0000", 2023-10-26T01:33:14.153 INFO:teuthology.orchestra.run.smithi169.stdout: "last_up_change": "2023-10-26T01:30:31.875281+0000", 2023-10-26T01:33:14.153 INFO:teuthology.orchestra.run.smithi169.stdout: "last_in_change": "2023-10-26T01:30:27.637941+0000", 2023-10-26T01:33:14.153 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-26T01:33:14.153 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_num": 5799936, 2023-10-26T01:33:14.153 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_set": [ 2023-10-26T01:33:14.154 INFO:teuthology.orchestra.run.smithi169.stdout: "pglog_hardlimit", 2023-10-26T01:33:14.154 INFO:teuthology.orchestra.run.smithi169.stdout: "purged_snapdirs", 2023-10-26T01:33:14.154 INFO:teuthology.orchestra.run.smithi169.stdout: "recovery_deletes", 2023-10-26T01:33:14.154 INFO:teuthology.orchestra.run.smithi169.stdout: "sortbitwise" 2023-10-26T01:33:14.154 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:33:14.155 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_version": 4, 2023-10-26T01:33:14.155 INFO:teuthology.orchestra.run.smithi169.stdout: "full_ratio": 0.69999998807907104, 2023-10-26T01:33:14.155 INFO:teuthology.orchestra.run.smithi169.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-26T01:33:14.155 INFO:teuthology.orchestra.run.smithi169.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-26T01:33:14.155 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_snapshot": "", 2023-10-26T01:33:14.156 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_max": 7, 2023-10-26T01:33:14.156 INFO:teuthology.orchestra.run.smithi169.stdout: "max_osd": 8, 2023-10-26T01:33:14.156 INFO:teuthology.orchestra.run.smithi169.stdout: "require_min_compat_client": "luminous", 2023-10-26T01:33:14.156 INFO:teuthology.orchestra.run.smithi169.stdout: "min_compat_client": "jewel", 2023-10-26T01:33:14.156 INFO:teuthology.orchestra.run.smithi169.stdout: "require_osd_release": "reef", 2023-10-26T01:33:14.157 INFO:teuthology.orchestra.run.smithi169.stdout: "allow_crimson": false, 2023-10-26T01:33:14.157 INFO:teuthology.orchestra.run.smithi169.stdout: "pools": [ 2023-10-26T01:33:14.157 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.157 INFO:teuthology.orchestra.run.smithi169.stdout: "pool": 1, 2023-10-26T01:33:14.157 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_name": ".mgr", 2023-10-26T01:33:14.157 INFO:teuthology.orchestra.run.smithi169.stdout: "create_time": "2023-10-26T01:30:32.200808+0000", 2023-10-26T01:33:14.158 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": 1, 2023-10-26T01:33:14.158 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_names": "hashpspool", 2023-10-26T01:33:14.158 INFO:teuthology.orchestra.run.smithi169.stdout: "type": 1, 2023-10-26T01:33:14.158 INFO:teuthology.orchestra.run.smithi169.stdout: "size": 2, 2023-10-26T01:33:14.158 INFO:teuthology.orchestra.run.smithi169.stdout: "min_size": 1, 2023-10-26T01:33:14.159 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_rule": 0, 2023-10-26T01:33:14.159 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_count": 0, 2023-10-26T01:33:14.159 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_target": 0, 2023-10-26T01:33:14.159 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T01:33:14.159 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T01:33:14.160 INFO:teuthology.orchestra.run.smithi169.stdout: "object_hash": 2, 2023-10-26T01:33:14.160 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_mode": "off", 2023-10-26T01:33:14.160 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num": 1, 2023-10-26T01:33:14.160 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num": 1, 2023-10-26T01:33:14.160 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num_target": 1, 2023-10-26T01:33:14.160 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_target": 1, 2023-10-26T01:33:14.161 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_pending": 1, 2023-10-26T01:33:14.161 INFO:teuthology.orchestra.run.smithi169.stdout: "last_pg_merge_meta": { 2023-10-26T01:33:14.161 INFO:teuthology.orchestra.run.smithi169.stdout: "source_pgid": "0.0", 2023-10-26T01:33:14.161 INFO:teuthology.orchestra.run.smithi169.stdout: "ready_epoch": 0, 2023-10-26T01:33:14.161 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_started": 0, 2023-10-26T01:33:14.162 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_clean": 0, 2023-10-26T01:33:14.162 INFO:teuthology.orchestra.run.smithi169.stdout: "source_version": "0'0", 2023-10-26T01:33:14.162 INFO:teuthology.orchestra.run.smithi169.stdout: "target_version": "0'0" 2023-10-26T01:33:14.162 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.162 INFO:teuthology.orchestra.run.smithi169.stdout: "last_change": "54", 2023-10-26T01:33:14.162 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend": "0", 2023-10-26T01:33:14.163 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T01:33:14.163 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T01:33:14.163 INFO:teuthology.orchestra.run.smithi169.stdout: "auid": 0, 2023-10-26T01:33:14.163 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_mode": "selfmanaged", 2023-10-26T01:33:14.163 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_seq": 0, 2023-10-26T01:33:14.164 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_epoch": 0, 2023-10-26T01:33:14.164 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_snaps": [], 2023-10-26T01:33:14.164 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps": "[]", 2023-10-26T01:33:14.164 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_bytes": 0, 2023-10-26T01:33:14.164 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_objects": 0, 2023-10-26T01:33:14.165 INFO:teuthology.orchestra.run.smithi169.stdout: "tiers": [], 2023-10-26T01:33:14.165 INFO:teuthology.orchestra.run.smithi169.stdout: "tier_of": -1, 2023-10-26T01:33:14.165 INFO:teuthology.orchestra.run.smithi169.stdout: "read_tier": -1, 2023-10-26T01:33:14.165 INFO:teuthology.orchestra.run.smithi169.stdout: "write_tier": -1, 2023-10-26T01:33:14.165 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_mode": "none", 2023-10-26T01:33:14.165 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_bytes": 0, 2023-10-26T01:33:14.166 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_objects": 0, 2023-10-26T01:33:14.166 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T01:33:14.166 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T01:33:14.166 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T01:33:14.166 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_flush_age": 0, 2023-10-26T01:33:14.167 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_evict_age": 0, 2023-10-26T01:33:14.167 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profile": "", 2023-10-26T01:33:14.167 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_params": { 2023-10-26T01:33:14.167 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "none" 2023-10-26T01:33:14.167 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.168 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_period": 0, 2023-10-26T01:33:14.168 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_count": 0, 2023-10-26T01:33:14.168 INFO:teuthology.orchestra.run.smithi169.stdout: "use_gmt_hitset": true, 2023-10-26T01:33:14.168 INFO:teuthology.orchestra.run.smithi169.stdout: "min_read_recency_for_promote": 0, 2023-10-26T01:33:14.168 INFO:teuthology.orchestra.run.smithi169.stdout: "min_write_recency_for_promote": 0, 2023-10-26T01:33:14.169 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T01:33:14.169 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_search_last_n": 0, 2023-10-26T01:33:14.169 INFO:teuthology.orchestra.run.smithi169.stdout: "grade_table": [], 2023-10-26T01:33:14.169 INFO:teuthology.orchestra.run.smithi169.stdout: "stripe_width": 0, 2023-10-26T01:33:14.169 INFO:teuthology.orchestra.run.smithi169.stdout: "expected_num_objects": 0, 2023-10-26T01:33:14.170 INFO:teuthology.orchestra.run.smithi169.stdout: "fast_read": false, 2023-10-26T01:33:14.170 INFO:teuthology.orchestra.run.smithi169.stdout: "options": { 2023-10-26T01:33:14.170 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_max": 32, 2023-10-26T01:33:14.170 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_min": 1 2023-10-26T01:33:14.170 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.171 INFO:teuthology.orchestra.run.smithi169.stdout: "application_metadata": { 2023-10-26T01:33:14.171 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": {} 2023-10-26T01:33:14.171 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.171 INFO:teuthology.orchestra.run.smithi169.stdout: "read_balance": { 2023-10-26T01:33:14.171 INFO:teuthology.orchestra.run.smithi169.stdout: "score_acting": 8, 2023-10-26T01:33:14.172 INFO:teuthology.orchestra.run.smithi169.stdout: "score_stable": 8, 2023-10-26T01:33:14.172 INFO:teuthology.orchestra.run.smithi169.stdout: "optimal_score": 0.25, 2023-10-26T01:33:14.172 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_acting": 2, 2023-10-26T01:33:14.172 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_stable": 2, 2023-10-26T01:33:14.172 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity_weighted": 1, 2023-10-26T01:33:14.172 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity": 1, 2023-10-26T01:33:14.173 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity_weighted": 1 2023-10-26T01:33:14.173 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:14.173 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.173 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.174 INFO:teuthology.orchestra.run.smithi169.stdout: "pool": 6, 2023-10-26T01:33:14.174 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_name": "cephfs_metadata", 2023-10-26T01:33:14.174 INFO:teuthology.orchestra.run.smithi169.stdout: "create_time": "2023-10-26T01:32:36.288217+0000", 2023-10-26T01:33:14.174 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": 1, 2023-10-26T01:33:14.174 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_names": "hashpspool", 2023-10-26T01:33:14.174 INFO:teuthology.orchestra.run.smithi169.stdout: "type": 1, 2023-10-26T01:33:14.175 INFO:teuthology.orchestra.run.smithi169.stdout: "size": 2, 2023-10-26T01:33:14.175 INFO:teuthology.orchestra.run.smithi169.stdout: "min_size": 1, 2023-10-26T01:33:14.175 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_rule": 0, 2023-10-26T01:33:14.175 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_count": 0, 2023-10-26T01:33:14.175 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_target": 0, 2023-10-26T01:33:14.176 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T01:33:14.176 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T01:33:14.176 INFO:teuthology.orchestra.run.smithi169.stdout: "object_hash": 2, 2023-10-26T01:33:14.176 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_mode": "off", 2023-10-26T01:33:14.176 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num": 32, 2023-10-26T01:33:14.176 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num": 32, 2023-10-26T01:33:14.177 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num_target": 32, 2023-10-26T01:33:14.177 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_target": 32, 2023-10-26T01:33:14.177 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_pending": 32, 2023-10-26T01:33:14.177 INFO:teuthology.orchestra.run.smithi169.stdout: "last_pg_merge_meta": { 2023-10-26T01:33:14.177 INFO:teuthology.orchestra.run.smithi169.stdout: "source_pgid": "0.0", 2023-10-26T01:33:14.178 INFO:teuthology.orchestra.run.smithi169.stdout: "ready_epoch": 0, 2023-10-26T01:33:14.178 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_started": 0, 2023-10-26T01:33:14.178 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_clean": 0, 2023-10-26T01:33:14.178 INFO:teuthology.orchestra.run.smithi169.stdout: "source_version": "0'0", 2023-10-26T01:33:14.178 INFO:teuthology.orchestra.run.smithi169.stdout: "target_version": "0'0" 2023-10-26T01:33:14.179 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.179 INFO:teuthology.orchestra.run.smithi169.stdout: "last_change": "54", 2023-10-26T01:33:14.179 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend": "0", 2023-10-26T01:33:14.179 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T01:33:14.179 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T01:33:14.180 INFO:teuthology.orchestra.run.smithi169.stdout: "auid": 0, 2023-10-26T01:33:14.180 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_mode": "selfmanaged", 2023-10-26T01:33:14.180 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_seq": 0, 2023-10-26T01:33:14.180 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_epoch": 0, 2023-10-26T01:33:14.180 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_snaps": [], 2023-10-26T01:33:14.181 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps": "[]", 2023-10-26T01:33:14.181 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_bytes": 0, 2023-10-26T01:33:14.181 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_objects": 0, 2023-10-26T01:33:14.181 INFO:teuthology.orchestra.run.smithi169.stdout: "tiers": [], 2023-10-26T01:33:14.181 INFO:teuthology.orchestra.run.smithi169.stdout: "tier_of": -1, 2023-10-26T01:33:14.182 INFO:teuthology.orchestra.run.smithi169.stdout: "read_tier": -1, 2023-10-26T01:33:14.182 INFO:teuthology.orchestra.run.smithi169.stdout: "write_tier": -1, 2023-10-26T01:33:14.182 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_mode": "none", 2023-10-26T01:33:14.182 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_bytes": 0, 2023-10-26T01:33:14.182 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_objects": 0, 2023-10-26T01:33:14.183 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T01:33:14.183 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T01:33:14.183 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T01:33:14.183 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_flush_age": 0, 2023-10-26T01:33:14.183 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_evict_age": 0, 2023-10-26T01:33:14.184 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profile": "", 2023-10-26T01:33:14.184 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_params": { 2023-10-26T01:33:14.184 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "none" 2023-10-26T01:33:14.184 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.184 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_period": 0, 2023-10-26T01:33:14.185 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_count": 0, 2023-10-26T01:33:14.185 INFO:teuthology.orchestra.run.smithi169.stdout: "use_gmt_hitset": true, 2023-10-26T01:33:14.185 INFO:teuthology.orchestra.run.smithi169.stdout: "min_read_recency_for_promote": 0, 2023-10-26T01:33:14.185 INFO:teuthology.orchestra.run.smithi169.stdout: "min_write_recency_for_promote": 0, 2023-10-26T01:33:14.185 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T01:33:14.186 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_search_last_n": 0, 2023-10-26T01:33:14.186 INFO:teuthology.orchestra.run.smithi169.stdout: "grade_table": [], 2023-10-26T01:33:14.186 INFO:teuthology.orchestra.run.smithi169.stdout: "stripe_width": 0, 2023-10-26T01:33:14.186 INFO:teuthology.orchestra.run.smithi169.stdout: "expected_num_objects": 0, 2023-10-26T01:33:14.186 INFO:teuthology.orchestra.run.smithi169.stdout: "fast_read": false, 2023-10-26T01:33:14.187 INFO:teuthology.orchestra.run.smithi169.stdout: "options": { 2023-10-26T01:33:14.187 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_bias": 4, 2023-10-26T01:33:14.187 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_min": 16, 2023-10-26T01:33:14.187 INFO:teuthology.orchestra.run.smithi169.stdout: "recovery_priority": 5 2023-10-26T01:33:14.187 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.187 INFO:teuthology.orchestra.run.smithi169.stdout: "application_metadata": { 2023-10-26T01:33:14.188 INFO:teuthology.orchestra.run.smithi169.stdout: "cephfs": { 2023-10-26T01:33:14.188 INFO:teuthology.orchestra.run.smithi169.stdout: "metadata": "cephfs" 2023-10-26T01:33:14.188 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:14.188 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.188 INFO:teuthology.orchestra.run.smithi169.stdout: "read_balance": { 2023-10-26T01:33:14.189 INFO:teuthology.orchestra.run.smithi169.stdout: "score_acting": 1.75, 2023-10-26T01:33:14.189 INFO:teuthology.orchestra.run.smithi169.stdout: "score_stable": 1.75, 2023-10-26T01:33:14.189 INFO:teuthology.orchestra.run.smithi169.stdout: "optimal_score": 1, 2023-10-26T01:33:14.189 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_acting": 1.75, 2023-10-26T01:33:14.189 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_stable": 1.75, 2023-10-26T01:33:14.190 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity_weighted": 1, 2023-10-26T01:33:14.190 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity": 1, 2023-10-26T01:33:14.190 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity_weighted": 1 2023-10-26T01:33:14.190 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:14.190 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.191 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.191 INFO:teuthology.orchestra.run.smithi169.stdout: "pool": 7, 2023-10-26T01:33:14.191 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_name": "cephfs_data", 2023-10-26T01:33:14.191 INFO:teuthology.orchestra.run.smithi169.stdout: "create_time": "2023-10-26T01:32:37.059775+0000", 2023-10-26T01:33:14.191 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": 1, 2023-10-26T01:33:14.192 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_names": "hashpspool", 2023-10-26T01:33:14.192 INFO:teuthology.orchestra.run.smithi169.stdout: "type": 1, 2023-10-26T01:33:14.192 INFO:teuthology.orchestra.run.smithi169.stdout: "size": 2, 2023-10-26T01:33:14.192 INFO:teuthology.orchestra.run.smithi169.stdout: "min_size": 1, 2023-10-26T01:33:14.192 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_rule": 0, 2023-10-26T01:33:14.193 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_count": 0, 2023-10-26T01:33:14.193 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_target": 0, 2023-10-26T01:33:14.193 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T01:33:14.193 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T01:33:14.193 INFO:teuthology.orchestra.run.smithi169.stdout: "object_hash": 2, 2023-10-26T01:33:14.194 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_mode": "off", 2023-10-26T01:33:14.194 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num": 64, 2023-10-26T01:33:14.194 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num": 64, 2023-10-26T01:33:14.194 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num_target": 64, 2023-10-26T01:33:14.194 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_target": 64, 2023-10-26T01:33:14.194 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_pending": 64, 2023-10-26T01:33:14.195 INFO:teuthology.orchestra.run.smithi169.stdout: "last_pg_merge_meta": { 2023-10-26T01:33:14.195 INFO:teuthology.orchestra.run.smithi169.stdout: "source_pgid": "0.0", 2023-10-26T01:33:14.195 INFO:teuthology.orchestra.run.smithi169.stdout: "ready_epoch": 0, 2023-10-26T01:33:14.195 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_started": 0, 2023-10-26T01:33:14.195 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_clean": 0, 2023-10-26T01:33:14.196 INFO:teuthology.orchestra.run.smithi169.stdout: "source_version": "0'0", 2023-10-26T01:33:14.196 INFO:teuthology.orchestra.run.smithi169.stdout: "target_version": "0'0" 2023-10-26T01:33:14.196 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.196 INFO:teuthology.orchestra.run.smithi169.stdout: "last_change": "54", 2023-10-26T01:33:14.196 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend": "0", 2023-10-26T01:33:14.197 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T01:33:14.197 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T01:33:14.197 INFO:teuthology.orchestra.run.smithi169.stdout: "auid": 0, 2023-10-26T01:33:14.197 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_mode": "selfmanaged", 2023-10-26T01:33:14.197 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_seq": 0, 2023-10-26T01:33:14.198 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_epoch": 0, 2023-10-26T01:33:14.198 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_snaps": [], 2023-10-26T01:33:14.198 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps": "[]", 2023-10-26T01:33:14.198 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_bytes": 0, 2023-10-26T01:33:14.198 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_objects": 0, 2023-10-26T01:33:14.199 INFO:teuthology.orchestra.run.smithi169.stdout: "tiers": [], 2023-10-26T01:33:14.199 INFO:teuthology.orchestra.run.smithi169.stdout: "tier_of": -1, 2023-10-26T01:33:14.199 INFO:teuthology.orchestra.run.smithi169.stdout: "read_tier": -1, 2023-10-26T01:33:14.199 INFO:teuthology.orchestra.run.smithi169.stdout: "write_tier": -1, 2023-10-26T01:33:14.199 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_mode": "none", 2023-10-26T01:33:14.200 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_bytes": 0, 2023-10-26T01:33:14.200 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_objects": 0, 2023-10-26T01:33:14.200 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T01:33:14.200 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T01:33:14.200 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T01:33:14.200 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_flush_age": 0, 2023-10-26T01:33:14.201 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_evict_age": 0, 2023-10-26T01:33:14.201 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profile": "", 2023-10-26T01:33:14.201 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_params": { 2023-10-26T01:33:14.201 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "none" 2023-10-26T01:33:14.201 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.202 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_period": 0, 2023-10-26T01:33:14.202 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_count": 0, 2023-10-26T01:33:14.202 INFO:teuthology.orchestra.run.smithi169.stdout: "use_gmt_hitset": true, 2023-10-26T01:33:14.202 INFO:teuthology.orchestra.run.smithi169.stdout: "min_read_recency_for_promote": 0, 2023-10-26T01:33:14.202 INFO:teuthology.orchestra.run.smithi169.stdout: "min_write_recency_for_promote": 0, 2023-10-26T01:33:14.203 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T01:33:14.203 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_search_last_n": 0, 2023-10-26T01:33:14.203 INFO:teuthology.orchestra.run.smithi169.stdout: "grade_table": [], 2023-10-26T01:33:14.203 INFO:teuthology.orchestra.run.smithi169.stdout: "stripe_width": 0, 2023-10-26T01:33:14.203 INFO:teuthology.orchestra.run.smithi169.stdout: "expected_num_objects": 0, 2023-10-26T01:33:14.204 INFO:teuthology.orchestra.run.smithi169.stdout: "fast_read": false, 2023-10-26T01:33:14.204 INFO:teuthology.orchestra.run.smithi169.stdout: "options": { 2023-10-26T01:33:14.204 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_min": 64, 2023-10-26T01:33:14.204 INFO:teuthology.orchestra.run.smithi169.stdout: "target_size_ratio": 0.89999997615814209 2023-10-26T01:33:14.204 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.205 INFO:teuthology.orchestra.run.smithi169.stdout: "application_metadata": { 2023-10-26T01:33:14.205 INFO:teuthology.orchestra.run.smithi169.stdout: "cephfs": { 2023-10-26T01:33:14.205 INFO:teuthology.orchestra.run.smithi169.stdout: "data": "cephfs" 2023-10-26T01:33:14.205 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:14.205 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.206 INFO:teuthology.orchestra.run.smithi169.stdout: "read_balance": { 2023-10-26T01:33:14.206 INFO:teuthology.orchestra.run.smithi169.stdout: "score_acting": 1.75, 2023-10-26T01:33:14.206 INFO:teuthology.orchestra.run.smithi169.stdout: "score_stable": 1.75, 2023-10-26T01:33:14.206 INFO:teuthology.orchestra.run.smithi169.stdout: "optimal_score": 1, 2023-10-26T01:33:14.206 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_acting": 1.75, 2023-10-26T01:33:14.207 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_stable": 1.75, 2023-10-26T01:33:14.207 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity_weighted": 1, 2023-10-26T01:33:14.207 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity": 1, 2023-10-26T01:33:14.207 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity_weighted": 1 2023-10-26T01:33:14.207 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:14.207 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:14.208 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:33:14.208 INFO:teuthology.orchestra.run.smithi169.stdout: "osds": [ 2023-10-26T01:33:14.208 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.208 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 0, 2023-10-26T01:33:14.208 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:33:14.209 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:33:14.209 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:33:14.209 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:33:14.209 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:33:14.209 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:33:14.209 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:33:14.210 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:33:14.210 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 47, 2023-10-26T01:33:14.210 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:33:14.210 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:33:14.210 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:33:14.211 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:14.211 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.211 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:14.211 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6816", 2023-10-26T01:33:14.211 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:33:14.212 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.212 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.212 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:14.212 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6817", 2023-10-26T01:33:14.212 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:33:14.213 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:14.213 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:14.213 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.213 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:33:14.213 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:14.214 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.214 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:14.214 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6818", 2023-10-26T01:33:14.214 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:33:14.214 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.214 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.215 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:14.215 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6819", 2023-10-26T01:33:14.215 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:33:14.215 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:14.215 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:14.216 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.216 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:33:14.216 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:14.216 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.216 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:14.216 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6822", 2023-10-26T01:33:14.217 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:33:14.217 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.217 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.217 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:14.217 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6823", 2023-10-26T01:33:14.218 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:33:14.218 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:14.218 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:14.218 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.218 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:33:14.219 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:14.219 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.219 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:14.219 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6820", 2023-10-26T01:33:14.219 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:33:14.219 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.220 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.220 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:14.220 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6821", 2023-10-26T01:33:14.220 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:33:14.220 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:14.221 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:14.221 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.221 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6817/4204255864", 2023-10-26T01:33:14.221 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6819/4204255864", 2023-10-26T01:33:14.221 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6823/4204255864", 2023-10-26T01:33:14.222 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6821/4204255864", 2023-10-26T01:33:14.222 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:33:14.222 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:33:14.222 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:33:14.222 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:14.223 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.223 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.223 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 1, 2023-10-26T01:33:14.223 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:33:14.223 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:33:14.223 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:33:14.224 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:33:14.224 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:33:14.224 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:33:14.224 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:33:14.224 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:33:14.225 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 47, 2023-10-26T01:33:14.225 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:33:14.225 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:33:14.225 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:33:14.225 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:14.225 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.226 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:14.226 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6808", 2023-10-26T01:33:14.226 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:33:14.226 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.226 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.227 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:14.227 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6809", 2023-10-26T01:33:14.227 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:33:14.227 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:14.227 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:14.228 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.228 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:33:14.228 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:14.228 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.228 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:14.228 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6810", 2023-10-26T01:33:14.229 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:33:14.229 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.229 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.229 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:14.229 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6811", 2023-10-26T01:33:14.230 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:33:14.230 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:14.230 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:14.230 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.230 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:33:14.231 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:14.231 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.231 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:14.231 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6814", 2023-10-26T01:33:14.231 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:33:14.232 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.232 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.232 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:14.232 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6815", 2023-10-26T01:33:14.232 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:33:14.232 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:14.233 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:14.233 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.233 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:33:14.233 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:14.233 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.234 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:14.234 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6812", 2023-10-26T01:33:14.234 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:33:14.234 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.234 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.234 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:14.235 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6813", 2023-10-26T01:33:14.235 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:33:14.235 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:14.235 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:14.235 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.236 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6809/238206046", 2023-10-26T01:33:14.236 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6811/238206046", 2023-10-26T01:33:14.236 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6815/238206046", 2023-10-26T01:33:14.236 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6813/238206046", 2023-10-26T01:33:14.236 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:33:14.236 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:33:14.237 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:33:14.237 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:14.237 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.237 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.237 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 2, 2023-10-26T01:33:14.238 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:33:14.238 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:33:14.238 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:33:14.238 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:33:14.238 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:33:14.238 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:33:14.239 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:33:14.239 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:33:14.239 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 47, 2023-10-26T01:33:14.239 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:33:14.239 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:33:14.240 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:33:14.240 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:14.240 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.240 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:14.240 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6800", 2023-10-26T01:33:14.240 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:33:14.241 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.241 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.241 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:14.241 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6801", 2023-10-26T01:33:14.241 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:33:14.242 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:14.242 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:14.242 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.242 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:33:14.242 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:14.243 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.243 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:14.243 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6802", 2023-10-26T01:33:14.243 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:33:14.243 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.243 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.244 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:14.245 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6803", 2023-10-26T01:33:14.245 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:33:14.245 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:14.246 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:14.246 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.246 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:33:14.246 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:14.246 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.247 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:14.247 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6806", 2023-10-26T01:33:14.247 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:33:14.247 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.247 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.248 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:14.248 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6807", 2023-10-26T01:33:14.248 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:33:14.248 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:14.248 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:14.249 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.249 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:33:14.249 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:14.249 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.249 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:14.249 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6804", 2023-10-26T01:33:14.250 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:33:14.250 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.250 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.250 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:14.250 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6805", 2023-10-26T01:33:14.251 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:33:14.251 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:14.251 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:14.251 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.251 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6801/3215791846", 2023-10-26T01:33:14.252 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6803/3215791846", 2023-10-26T01:33:14.252 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6807/3215791846", 2023-10-26T01:33:14.252 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6805/3215791846", 2023-10-26T01:33:14.252 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:33:14.252 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:33:14.253 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:33:14.253 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:14.253 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.253 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.253 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 3, 2023-10-26T01:33:14.254 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:33:14.254 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:33:14.254 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:33:14.254 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:33:14.254 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:33:14.255 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:33:14.255 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:33:14.255 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 14, 2023-10-26T01:33:14.255 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 47, 2023-10-26T01:33:14.255 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:33:14.256 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:33:14.256 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:33:14.256 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:14.256 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.256 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:14.257 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6824", 2023-10-26T01:33:14.257 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:33:14.257 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.257 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.258 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:14.258 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6825", 2023-10-26T01:33:14.258 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:33:14.258 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:14.259 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:14.259 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.259 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:33:14.259 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:14.259 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.259 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:14.260 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6826", 2023-10-26T01:33:14.260 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:33:14.260 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.260 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.260 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:14.261 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6827", 2023-10-26T01:33:14.261 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:33:14.261 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:14.261 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:14.261 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.261 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:33:14.262 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:14.262 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.262 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:14.262 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6830", 2023-10-26T01:33:14.262 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:33:14.263 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.263 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.263 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:14.263 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6831", 2023-10-26T01:33:14.263 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:33:14.264 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:14.264 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:14.264 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.264 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:33:14.264 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:14.265 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.265 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:14.265 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6828", 2023-10-26T01:33:14.265 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:33:14.265 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.265 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.266 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:14.266 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6829", 2023-10-26T01:33:14.266 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:33:14.266 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:14.266 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:14.267 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.267 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6825/2256701226", 2023-10-26T01:33:14.267 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6827/2256701226", 2023-10-26T01:33:14.267 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6831/2256701226", 2023-10-26T01:33:14.267 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6829/2256701226", 2023-10-26T01:33:14.268 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:33:14.268 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:33:14.268 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:33:14.268 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:14.268 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.268 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.269 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 4, 2023-10-26T01:33:14.269 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:33:14.269 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:33:14.269 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:33:14.269 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:33:14.270 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:33:14.270 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:33:14.270 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:33:14.270 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:33:14.270 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 47, 2023-10-26T01:33:14.271 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:33:14.271 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:33:14.271 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:33:14.271 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:14.271 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.271 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:14.272 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6816", 2023-10-26T01:33:14.272 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:33:14.272 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.272 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.272 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:14.273 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6817", 2023-10-26T01:33:14.273 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:33:14.273 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:14.273 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:14.273 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.274 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:33:14.274 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:14.274 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.274 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:14.274 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6818", 2023-10-26T01:33:14.274 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:33:14.275 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.275 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.275 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:14.275 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6819", 2023-10-26T01:33:14.275 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:33:14.276 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:14.276 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:14.276 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.276 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:33:14.276 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:14.277 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.277 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:14.277 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6825", 2023-10-26T01:33:14.277 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:33:14.277 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.278 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.278 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:14.278 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6828", 2023-10-26T01:33:14.278 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:33:14.278 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:14.278 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:14.279 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.279 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:33:14.279 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:14.279 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.279 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:14.280 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6820", 2023-10-26T01:33:14.280 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:33:14.280 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.280 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.281 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:14.281 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6823", 2023-10-26T01:33:14.281 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:33:14.281 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:14.281 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:14.281 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.282 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6817/1082279874", 2023-10-26T01:33:14.282 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6819/1082279874", 2023-10-26T01:33:14.282 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6828/1082279874", 2023-10-26T01:33:14.282 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6823/1082279874", 2023-10-26T01:33:14.282 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:33:14.283 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:33:14.283 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:33:14.283 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:14.283 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.283 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.284 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 5, 2023-10-26T01:33:14.284 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:33:14.284 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:33:14.284 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:33:14.284 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:33:14.285 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:33:14.285 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:33:14.285 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:33:14.285 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:33:14.285 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 47, 2023-10-26T01:33:14.285 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:33:14.286 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:33:14.286 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:33:14.286 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:14.286 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.286 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:14.287 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6808", 2023-10-26T01:33:14.287 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:33:14.287 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.287 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.287 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:14.288 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6809", 2023-10-26T01:33:14.288 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:33:14.288 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:14.288 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:14.288 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.289 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:33:14.289 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:14.289 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.289 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:14.289 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6810", 2023-10-26T01:33:14.290 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:33:14.290 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.290 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.290 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:14.290 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6811", 2023-10-26T01:33:14.291 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:33:14.291 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:14.291 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:14.291 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.291 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:33:14.292 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:14.292 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.292 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:14.292 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6814", 2023-10-26T01:33:14.292 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:33:14.292 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.293 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.293 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:14.293 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6815", 2023-10-26T01:33:14.293 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:33:14.293 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:14.294 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:14.294 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.294 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:33:14.294 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:14.294 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.295 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:14.295 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6812", 2023-10-26T01:33:14.295 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:33:14.295 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.295 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.296 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:14.296 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6813", 2023-10-26T01:33:14.296 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:33:14.296 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:14.296 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:14.296 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.297 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6809/3335654351", 2023-10-26T01:33:14.297 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6811/3335654351", 2023-10-26T01:33:14.297 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6815/3335654351", 2023-10-26T01:33:14.297 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6813/3335654351", 2023-10-26T01:33:14.297 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:33:14.298 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:33:14.298 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:33:14.298 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:14.298 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.298 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.299 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 6, 2023-10-26T01:33:14.299 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:33:14.299 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:33:14.299 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:33:14.299 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:33:14.300 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:33:14.300 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:33:14.300 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:33:14.300 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:33:14.300 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 47, 2023-10-26T01:33:14.300 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:33:14.301 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:33:14.301 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:33:14.301 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:14.301 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.301 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:14.302 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6800", 2023-10-26T01:33:14.302 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:33:14.302 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.302 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.302 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:14.303 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6801", 2023-10-26T01:33:14.303 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:33:14.303 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:14.303 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:14.303 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.303 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:33:14.304 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:14.304 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.304 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:14.304 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6802", 2023-10-26T01:33:14.304 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:33:14.305 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.305 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.305 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:14.305 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6803", 2023-10-26T01:33:14.305 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:33:14.306 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:14.306 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:14.306 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.306 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:33:14.306 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:14.306 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.307 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:14.307 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6806", 2023-10-26T01:33:14.307 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:33:14.307 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.307 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.308 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:14.308 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6807", 2023-10-26T01:33:14.308 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:33:14.308 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:14.308 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:14.309 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.309 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:33:14.309 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:14.309 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.309 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:14.310 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6804", 2023-10-26T01:33:14.310 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:33:14.310 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.310 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.310 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:14.311 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6805", 2023-10-26T01:33:14.311 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:33:14.311 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:14.311 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:14.311 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.312 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6801/2520460697", 2023-10-26T01:33:14.312 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6803/2520460697", 2023-10-26T01:33:14.312 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6807/2520460697", 2023-10-26T01:33:14.312 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6805/2520460697", 2023-10-26T01:33:14.312 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:33:14.312 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:33:14.313 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:33:14.313 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:14.313 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.313 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.313 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 7, 2023-10-26T01:33:14.314 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:33:14.314 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:33:14.314 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:33:14.314 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:33:14.314 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:33:14.315 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:33:14.315 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:33:14.315 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:33:14.315 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 47, 2023-10-26T01:33:14.315 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:33:14.315 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:33:14.316 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:33:14.316 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:14.316 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.316 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:14.316 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6821", 2023-10-26T01:33:14.317 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:33:14.317 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.317 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.317 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:14.317 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6822", 2023-10-26T01:33:14.318 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:33:14.318 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:14.318 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:14.318 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.318 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:33:14.318 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:14.319 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.319 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:14.319 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6824", 2023-10-26T01:33:14.319 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:33:14.319 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.320 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.320 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:14.320 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6826", 2023-10-26T01:33:14.320 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:33:14.320 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:14.321 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:14.321 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.321 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:33:14.321 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:14.321 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.321 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:14.322 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6830", 2023-10-26T01:33:14.322 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:33:14.322 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.322 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.322 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:14.323 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6831", 2023-10-26T01:33:14.323 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:33:14.323 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:14.323 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:14.323 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.323 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:33:14.324 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:14.324 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.324 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:14.324 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6827", 2023-10-26T01:33:14.324 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:33:14.325 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.325 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.325 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:14.325 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6829", 2023-10-26T01:33:14.325 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:33:14.325 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:14.326 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:14.326 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.326 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6822/1115930619", 2023-10-26T01:33:14.326 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6826/1115930619", 2023-10-26T01:33:14.326 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6831/1115930619", 2023-10-26T01:33:14.327 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6829/1115930619", 2023-10-26T01:33:14.327 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:33:14.327 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:33:14.327 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:33:14.327 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:14.327 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:14.328 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:33:14.328 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_xinfo": [ 2023-10-26T01:33:14.328 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.328 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 0, 2023-10-26T01:33:14.328 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:33:14.329 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:33:14.329 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:33:14.329 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:33:14.329 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:33:14.329 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.889675+0000", 2023-10-26T01:33:14.329 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:33:14.330 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.330 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.330 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 1, 2023-10-26T01:33:14.330 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:33:14.330 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:33:14.331 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:33:14.331 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:33:14.331 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:33:14.331 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.864157+0000", 2023-10-26T01:33:14.331 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:33:14.331 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.332 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.332 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 2, 2023-10-26T01:33:14.332 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:33:14.332 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:33:14.332 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:33:14.332 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:33:14.333 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:33:14.333 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.876919+0000", 2023-10-26T01:33:14.333 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:33:14.333 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.333 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.334 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 3, 2023-10-26T01:33:14.334 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:33:14.334 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:33:14.334 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:33:14.334 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:33:14.335 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:33:14.335 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.974541+0000", 2023-10-26T01:33:14.335 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:33:14.335 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.335 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.335 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 4, 2023-10-26T01:33:14.336 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:33:14.336 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:33:14.336 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:33:14.336 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:33:14.336 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:33:14.337 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.969696+0000", 2023-10-26T01:33:14.337 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:33:14.337 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.337 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.337 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 5, 2023-10-26T01:33:14.338 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:33:14.338 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:33:14.338 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:33:14.338 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:33:14.338 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:33:14.338 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.859703+0000", 2023-10-26T01:33:14.339 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:33:14.339 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.339 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.339 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 6, 2023-10-26T01:33:14.339 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:33:14.340 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:33:14.340 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:33:14.340 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:33:14.340 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:33:14.340 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.867917+0000", 2023-10-26T01:33:14.341 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:33:14.341 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.341 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.341 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 7, 2023-10-26T01:33:14.341 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:33:14.341 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:33:14.342 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:33:14.342 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:33:14.342 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:33:14.342 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.898605+0000", 2023-10-26T01:33:14.342 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:33:14.343 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:14.343 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:33:14.343 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_upmap": [], 2023-10-26T01:33:14.343 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_upmap_items": [], 2023-10-26T01:33:14.343 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_upmap_primaries": [], 2023-10-26T01:33:14.344 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_temp": [], 2023-10-26T01:33:14.344 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_temp": [], 2023-10-26T01:33:14.344 INFO:teuthology.orchestra.run.smithi169.stdout: "blocklist": {}, 2023-10-26T01:33:14.344 INFO:teuthology.orchestra.run.smithi169.stdout: "range_blocklist": {}, 2023-10-26T01:33:14.344 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profiles": { 2023-10-26T01:33:14.344 INFO:teuthology.orchestra.run.smithi169.stdout: "default": { 2023-10-26T01:33:14.345 INFO:teuthology.orchestra.run.smithi169.stdout: "crush-failure-domain": "osd", 2023-10-26T01:33:14.345 INFO:teuthology.orchestra.run.smithi169.stdout: "k": "2", 2023-10-26T01:33:14.345 INFO:teuthology.orchestra.run.smithi169.stdout: "m": "1", 2023-10-26T01:33:14.345 INFO:teuthology.orchestra.run.smithi169.stdout: "plugin": "jerasure", 2023-10-26T01:33:14.345 INFO:teuthology.orchestra.run.smithi169.stdout: "technique": "reed_sol_van" 2023-10-26T01:33:14.346 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:14.346 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.346 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps_queue": [], 2023-10-26T01:33:14.346 INFO:teuthology.orchestra.run.smithi169.stdout: "new_removed_snaps": [], 2023-10-26T01:33:14.346 INFO:teuthology.orchestra.run.smithi169.stdout: "new_purged_snaps": [], 2023-10-26T01:33:14.347 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_node_flags": {}, 2023-10-26T01:33:14.347 INFO:teuthology.orchestra.run.smithi169.stdout: "device_class_flags": {}, 2023-10-26T01:33:14.347 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_mode": { 2023-10-26T01:33:14.347 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_mode_enabled": false, 2023-10-26T01:33:14.347 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_bucket_count": 0, 2023-10-26T01:33:14.347 INFO:teuthology.orchestra.run.smithi169.stdout: "degraded_stretch_mode": 0, 2023-10-26T01:33:14.348 INFO:teuthology.orchestra.run.smithi169.stdout: "recovering_stretch_mode": 0, 2023-10-26T01:33:14.348 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_mode_bucket": 0 2023-10-26T01:33:14.348 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:14.348 INFO:teuthology.orchestra.run.smithi169.stdout:} 2023-10-26T01:33:14.350 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:14.151+0000 7fe4c1f54640 1 -- 172.21.15.169:0/3451348275 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe4a006ae40 msgr2=0x7fe4a006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:14.350 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:14.151+0000 7fe4c1f54640 1 --2- 172.21.15.169:0/3451348275 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe4a006ae40 0x7fe4a006d300 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe4b0004ea0 tx=0x7fe4b0009040 comp rx=0 tx=0).stop 2023-10-26T01:33:14.350 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:14.151+0000 7fe4c1f54640 1 -- 172.21.15.169:0/3451348275 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe4bc0c7ea0 msgr2=0x7fe4bc0afce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:14.351 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:14.151+0000 7fe4c1f54640 1 --2- 172.21.15.169:0/3451348275 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe4bc0c7ea0 0x7fe4bc0afce0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fe4a8009850 tx=0x7fe4a802db10 comp rx=0 tx=0).stop 2023-10-26T01:33:14.351 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:14.151+0000 7fe4c1f54640 1 -- 172.21.15.169:0/3451348275 shutdown_connections 2023-10-26T01:33:14.351 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:14.151+0000 7fe4c1f54640 1 --2- 172.21.15.169:0/3451348275 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe4a006ae40 0x7fe4a006d300 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:14.351 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:14.151+0000 7fe4c1f54640 1 --2- 172.21.15.169:0/3451348275 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe4bc0c7ea0 0x7fe4bc0afce0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:14.351 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:14.151+0000 7fe4c1f54640 1 --2- 172.21.15.169:0/3451348275 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe4bc0c9c30 0x7fe4bc154970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:14.352 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:14.151+0000 7fe4c1f54640 1 --2- 172.21.15.169:0/3451348275 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe4bc06a990 0x7fe4bc0b1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:14.352 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:14.151+0000 7fe4c1f54640 1 -- 172.21.15.169:0/3451348275 >> 172.21.15.169:0/3451348275 conn(0x7fe4bc0aedd0 msgr2=0x7fe4bc155cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:14.352 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:14.151+0000 7fe4c1f54640 1 -- 172.21.15.169:0/3451348275 shutdown_connections 2023-10-26T01:33:14.352 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:14.151+0000 7fe4c1f54640 1 -- 172.21.15.169:0/3451348275 wait complete. 2023-10-26T01:33:14.353 DEBUG:tasks.ceph_test_case:wait_until_true: success in 10s and 0 retries 2023-10-26T01:33:14.353 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-26T01:33:14.580 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:14.577+0000 7f3b65e1a640 1 -- 172.21.15.169:0/1188189847 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b600cde00 msgr2=0x7f3b600cbac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:14.581 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:14.577+0000 7f3b65e1a640 1 --2- 172.21.15.169:0/1188189847 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b600cde00 0x7f3b600cbac0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f3b58005fd0 tx=0x7f3b5802c240 comp rx=0 tx=0).stop 2023-10-26T01:33:14.581 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:14.577+0000 7f3b65e1a640 1 -- 172.21.15.169:0/1188189847 shutdown_connections 2023-10-26T01:33:14.581 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:14.577+0000 7f3b65e1a640 1 --2- 172.21.15.169:0/1188189847 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b600cde00 0x7f3b600cbac0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:14.581 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:14.577+0000 7f3b65e1a640 1 --2- 172.21.15.169:0/1188189847 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3b601592d0 0x7f3b600aa3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:14.581 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:14.577+0000 7f3b65e1a640 1 -- 172.21.15.169:0/1188189847 >> 172.21.15.169:0/1188189847 conn(0x7f3b60069930 msgr2=0x7f3b60069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:14.582 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:14.578+0000 7f3b65e1a640 1 -- 172.21.15.169:0/1188189847 shutdown_connections 2023-10-26T01:33:14.582 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:14.578+0000 7f3b65e1a640 1 -- 172.21.15.169:0/1188189847 wait complete. 2023-10-26T01:33:14.582 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:14.579+0000 7f3b65e1a640 1 Processor -- start 2023-10-26T01:33:14.583 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:14.579+0000 7f3b65e1a640 1 -- start start 2023-10-26T01:33:14.585 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:14.580+0000 7f3b65e1a640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3b600ac930 0x7f3b601f19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:14.585 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:14.580+0000 7f3b65e1a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3b600cde00 0x7f3b6014a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:14.585 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:14.580+0000 7f3b65e1a640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b601592d0 0x7f3b600c7980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:14.585 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:14.581+0000 7f3b65e1a640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b600c7ec0 con 0x7f3b601592d0 2023-10-26T01:33:14.585 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:14.581+0000 7f3b65e1a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b601f1fa0 con 0x7f3b600cde00 2023-10-26T01:33:14.586 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:14.581+0000 7f3b65e1a640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b601f2280 con 0x7f3b600ac930 2023-10-26T01:33:14.586 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:14.581+0000 7f3b5ffff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b601592d0 0x7f3b600c7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:14.586 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:14.581+0000 7f3b5ffff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b601592d0 0x7f3b600c7980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:47406/0 (socket says 172.21.15.169:47406) 2023-10-26T01:33:14.586 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:14.581+0000 7f3b5ffff640 1 -- 172.21.15.169:0/3516870202 learned_addr learned my addr 172.21.15.169:0/3516870202 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:33:14.587 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:14.581+0000 7f3b5f7fe640 1 --2- 172.21.15.169:0/3516870202 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3b600ac930 0x7f3b601f19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:14.587 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:14.581+0000 7f3b5ffff640 1 -- 172.21.15.169:0/3516870202 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3b600ac930 msgr2=0x7f3b601f19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:14.587 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:14.581+0000 7f3b5ffff640 1 --2- 172.21.15.169:0/3516870202 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3b600ac930 0x7f3b601f19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:14.587 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:14.581+0000 7f3b5ffff640 1 -- 172.21.15.169:0/3516870202 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3b600cde00 msgr2=0x7f3b6014a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:14.587 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:14.581+0000 7f3b57fff640 1 --2- 172.21.15.169:0/3516870202 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3b600cde00 0x7f3b6014a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:14.588 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:14.581+0000 7f3b5ffff640 1 --2- 172.21.15.169:0/3516870202 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3b600cde00 0x7f3b6014a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:14.588 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:14.581+0000 7f3b5ffff640 1 -- 172.21.15.169:0/3516870202 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b58009900 con 0x7f3b601592d0 2023-10-26T01:33:14.588 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:14.582+0000 7f3b5ffff640 1 --2- 172.21.15.169:0/3516870202 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b601592d0 0x7f3b600c7980 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f3b58037040 tx=0x7f3b58035a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:14.589 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:14.582+0000 7f3b5cff9640 1 -- 172.21.15.169:0/3516870202 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b58034440 con 0x7f3b601592d0 2023-10-26T01:33:14.589 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:14.582+0000 7f3b5cff9640 1 -- 172.21.15.169:0/3516870202 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3b58004330 con 0x7f3b601592d0 2023-10-26T01:33:14.589 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:14.582+0000 7f3b5cff9640 1 -- 172.21.15.169:0/3516870202 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b580046c0 con 0x7f3b601592d0 2023-10-26T01:33:14.589 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:14.582+0000 7f3b65e1a640 1 -- 172.21.15.169:0/3516870202 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b601f2560 con 0x7f3b601592d0 2023-10-26T01:33:14.589 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:14.582+0000 7f3b65e1a640 1 -- 172.21.15.169:0/3516870202 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b601530b0 con 0x7f3b601592d0 2023-10-26T01:33:14.590 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:14.583+0000 7f3b65e1a640 1 -- 172.21.15.169:0/3516870202 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b601532a0 con 0x7f3b601592d0 2023-10-26T01:33:14.591 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:14.584+0000 7f3b5cff9640 1 -- 172.21.15.169:0/3516870202 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f3b5803b040 con 0x7f3b601592d0 2023-10-26T01:33:14.591 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:14.585+0000 7f3b5cff9640 1 --2- 172.21.15.169:0/3516870202 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3b4006ae40 0x7f3b4006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:14.591 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:14.585+0000 7f3b5cff9640 1 -- 172.21.15.169:0/3516870202 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3b580ac280 con 0x7f3b601592d0 2023-10-26T01:33:14.591 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:14.586+0000 7f3b5f7fe640 1 --2- 172.21.15.169:0/3516870202 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3b4006ae40 0x7f3b4006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:14.592 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:14.587+0000 7f3b5f7fe640 1 --2- 172.21.15.169:0/3516870202 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3b4006ae40 0x7f3b4006d300 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f3b50005fd0 tx=0x7f3b50005850 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:14.592 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:14.589+0000 7f3b5cff9640 1 -- 172.21.15.169:0/3516870202 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3b58077290 con 0x7f3b601592d0 2023-10-26T01:33:14.781 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:14.778+0000 7f3b65e1a640 1 -- 172.21.15.169:0/3516870202 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f3b6004db10 con 0x7f3b601592d0 2023-10-26T01:33:14.783 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:14.780+0000 7f3b5cff9640 1 -- 172.21.15.169:0/3516870202 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v54) v1 ==== 81+0+33996 (secure 0 0 0) 0x7f3b5807a8b0 con 0x7f3b601592d0 2023-10-26T01:33:14.784 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:33:14.784 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2023-10-26T01:33:14.784 INFO:teuthology.orchestra.run.smithi169.stdout: "epoch": 54, 2023-10-26T01:33:14.784 INFO:teuthology.orchestra.run.smithi169.stdout: "fsid": "30203b28-ed8a-44e8-a9a6-a9bf25f980f1", 2023-10-26T01:33:14.784 INFO:teuthology.orchestra.run.smithi169.stdout: "created": "2023-10-26T01:30:22.897155+0000", 2023-10-26T01:33:14.785 INFO:teuthology.orchestra.run.smithi169.stdout: "modified": "2023-10-26T01:33:10.190714+0000", 2023-10-26T01:33:14.785 INFO:teuthology.orchestra.run.smithi169.stdout: "last_up_change": "2023-10-26T01:30:31.875281+0000", 2023-10-26T01:33:14.785 INFO:teuthology.orchestra.run.smithi169.stdout: "last_in_change": "2023-10-26T01:30:27.637941+0000", 2023-10-26T01:33:14.785 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-26T01:33:14.785 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_num": 5799936, 2023-10-26T01:33:14.786 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_set": [ 2023-10-26T01:33:14.786 INFO:teuthology.orchestra.run.smithi169.stdout: "pglog_hardlimit", 2023-10-26T01:33:14.786 INFO:teuthology.orchestra.run.smithi169.stdout: "purged_snapdirs", 2023-10-26T01:33:14.786 INFO:teuthology.orchestra.run.smithi169.stdout: "recovery_deletes", 2023-10-26T01:33:14.786 INFO:teuthology.orchestra.run.smithi169.stdout: "sortbitwise" 2023-10-26T01:33:14.787 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:33:14.787 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_version": 4, 2023-10-26T01:33:14.787 INFO:teuthology.orchestra.run.smithi169.stdout: "full_ratio": 0.69999998807907104, 2023-10-26T01:33:14.787 INFO:teuthology.orchestra.run.smithi169.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-26T01:33:14.787 INFO:teuthology.orchestra.run.smithi169.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-26T01:33:14.788 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_snapshot": "", 2023-10-26T01:33:14.788 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_max": 7, 2023-10-26T01:33:14.788 INFO:teuthology.orchestra.run.smithi169.stdout: "max_osd": 8, 2023-10-26T01:33:14.788 INFO:teuthology.orchestra.run.smithi169.stdout: "require_min_compat_client": "luminous", 2023-10-26T01:33:14.788 INFO:teuthology.orchestra.run.smithi169.stdout: "min_compat_client": "jewel", 2023-10-26T01:33:14.788 INFO:teuthology.orchestra.run.smithi169.stdout: "require_osd_release": "reef", 2023-10-26T01:33:14.789 INFO:teuthology.orchestra.run.smithi169.stdout: "allow_crimson": false, 2023-10-26T01:33:14.789 INFO:teuthology.orchestra.run.smithi169.stdout: "pools": [ 2023-10-26T01:33:14.789 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.789 INFO:teuthology.orchestra.run.smithi169.stdout: "pool": 1, 2023-10-26T01:33:14.789 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_name": ".mgr", 2023-10-26T01:33:14.790 INFO:teuthology.orchestra.run.smithi169.stdout: "create_time": "2023-10-26T01:30:32.200808+0000", 2023-10-26T01:33:14.790 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": 1, 2023-10-26T01:33:14.790 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_names": "hashpspool", 2023-10-26T01:33:14.790 INFO:teuthology.orchestra.run.smithi169.stdout: "type": 1, 2023-10-26T01:33:14.790 INFO:teuthology.orchestra.run.smithi169.stdout: "size": 2, 2023-10-26T01:33:14.790 INFO:teuthology.orchestra.run.smithi169.stdout: "min_size": 1, 2023-10-26T01:33:14.791 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_rule": 0, 2023-10-26T01:33:14.791 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_count": 0, 2023-10-26T01:33:14.791 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_target": 0, 2023-10-26T01:33:14.791 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T01:33:14.791 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T01:33:14.792 INFO:teuthology.orchestra.run.smithi169.stdout: "object_hash": 2, 2023-10-26T01:33:14.792 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_mode": "off", 2023-10-26T01:33:14.792 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num": 1, 2023-10-26T01:33:14.792 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num": 1, 2023-10-26T01:33:14.792 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num_target": 1, 2023-10-26T01:33:14.793 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_target": 1, 2023-10-26T01:33:14.793 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_pending": 1, 2023-10-26T01:33:14.793 INFO:teuthology.orchestra.run.smithi169.stdout: "last_pg_merge_meta": { 2023-10-26T01:33:14.793 INFO:teuthology.orchestra.run.smithi169.stdout: "source_pgid": "0.0", 2023-10-26T01:33:14.793 INFO:teuthology.orchestra.run.smithi169.stdout: "ready_epoch": 0, 2023-10-26T01:33:14.793 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_started": 0, 2023-10-26T01:33:14.794 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_clean": 0, 2023-10-26T01:33:14.794 INFO:teuthology.orchestra.run.smithi169.stdout: "source_version": "0'0", 2023-10-26T01:33:14.794 INFO:teuthology.orchestra.run.smithi169.stdout: "target_version": "0'0" 2023-10-26T01:33:14.794 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.794 INFO:teuthology.orchestra.run.smithi169.stdout: "last_change": "54", 2023-10-26T01:33:14.795 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend": "0", 2023-10-26T01:33:14.795 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T01:33:14.795 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T01:33:14.795 INFO:teuthology.orchestra.run.smithi169.stdout: "auid": 0, 2023-10-26T01:33:14.795 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_mode": "selfmanaged", 2023-10-26T01:33:14.795 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_seq": 0, 2023-10-26T01:33:14.796 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_epoch": 0, 2023-10-26T01:33:14.796 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_snaps": [], 2023-10-26T01:33:14.796 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps": "[]", 2023-10-26T01:33:14.796 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_bytes": 0, 2023-10-26T01:33:14.796 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_objects": 0, 2023-10-26T01:33:14.797 INFO:teuthology.orchestra.run.smithi169.stdout: "tiers": [], 2023-10-26T01:33:14.797 INFO:teuthology.orchestra.run.smithi169.stdout: "tier_of": -1, 2023-10-26T01:33:14.797 INFO:teuthology.orchestra.run.smithi169.stdout: "read_tier": -1, 2023-10-26T01:33:14.797 INFO:teuthology.orchestra.run.smithi169.stdout: "write_tier": -1, 2023-10-26T01:33:14.797 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_mode": "none", 2023-10-26T01:33:14.797 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_bytes": 0, 2023-10-26T01:33:14.798 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_objects": 0, 2023-10-26T01:33:14.798 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T01:33:14.798 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T01:33:14.798 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T01:33:14.798 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_flush_age": 0, 2023-10-26T01:33:14.799 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_evict_age": 0, 2023-10-26T01:33:14.799 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profile": "", 2023-10-26T01:33:14.799 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_params": { 2023-10-26T01:33:14.799 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "none" 2023-10-26T01:33:14.799 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.800 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_period": 0, 2023-10-26T01:33:14.800 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_count": 0, 2023-10-26T01:33:14.800 INFO:teuthology.orchestra.run.smithi169.stdout: "use_gmt_hitset": true, 2023-10-26T01:33:14.800 INFO:teuthology.orchestra.run.smithi169.stdout: "min_read_recency_for_promote": 0, 2023-10-26T01:33:14.800 INFO:teuthology.orchestra.run.smithi169.stdout: "min_write_recency_for_promote": 0, 2023-10-26T01:33:14.800 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T01:33:14.801 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_search_last_n": 0, 2023-10-26T01:33:14.801 INFO:teuthology.orchestra.run.smithi169.stdout: "grade_table": [], 2023-10-26T01:33:14.801 INFO:teuthology.orchestra.run.smithi169.stdout: "stripe_width": 0, 2023-10-26T01:33:14.801 INFO:teuthology.orchestra.run.smithi169.stdout: "expected_num_objects": 0, 2023-10-26T01:33:14.801 INFO:teuthology.orchestra.run.smithi169.stdout: "fast_read": false, 2023-10-26T01:33:14.802 INFO:teuthology.orchestra.run.smithi169.stdout: "options": { 2023-10-26T01:33:14.802 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_max": 32, 2023-10-26T01:33:14.802 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_min": 1 2023-10-26T01:33:14.802 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.802 INFO:teuthology.orchestra.run.smithi169.stdout: "application_metadata": { 2023-10-26T01:33:14.803 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": {} 2023-10-26T01:33:14.803 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.803 INFO:teuthology.orchestra.run.smithi169.stdout: "read_balance": { 2023-10-26T01:33:14.803 INFO:teuthology.orchestra.run.smithi169.stdout: "score_acting": 8, 2023-10-26T01:33:14.803 INFO:teuthology.orchestra.run.smithi169.stdout: "score_stable": 8, 2023-10-26T01:33:14.804 INFO:teuthology.orchestra.run.smithi169.stdout: "optimal_score": 0.25, 2023-10-26T01:33:14.804 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_acting": 2, 2023-10-26T01:33:14.804 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_stable": 2, 2023-10-26T01:33:14.804 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity_weighted": 1, 2023-10-26T01:33:14.804 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity": 1, 2023-10-26T01:33:14.804 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity_weighted": 1 2023-10-26T01:33:14.805 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:14.805 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.805 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.805 INFO:teuthology.orchestra.run.smithi169.stdout: "pool": 6, 2023-10-26T01:33:14.805 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_name": "cephfs_metadata", 2023-10-26T01:33:14.806 INFO:teuthology.orchestra.run.smithi169.stdout: "create_time": "2023-10-26T01:32:36.288217+0000", 2023-10-26T01:33:14.806 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": 1, 2023-10-26T01:33:14.806 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_names": "hashpspool", 2023-10-26T01:33:14.806 INFO:teuthology.orchestra.run.smithi169.stdout: "type": 1, 2023-10-26T01:33:14.806 INFO:teuthology.orchestra.run.smithi169.stdout: "size": 2, 2023-10-26T01:33:14.806 INFO:teuthology.orchestra.run.smithi169.stdout: "min_size": 1, 2023-10-26T01:33:14.807 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_rule": 0, 2023-10-26T01:33:14.807 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_count": 0, 2023-10-26T01:33:14.807 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_target": 0, 2023-10-26T01:33:14.807 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T01:33:14.807 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T01:33:14.808 INFO:teuthology.orchestra.run.smithi169.stdout: "object_hash": 2, 2023-10-26T01:33:14.808 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_mode": "off", 2023-10-26T01:33:14.808 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num": 32, 2023-10-26T01:33:14.808 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num": 32, 2023-10-26T01:33:14.808 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num_target": 32, 2023-10-26T01:33:14.808 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_target": 32, 2023-10-26T01:33:14.809 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_pending": 32, 2023-10-26T01:33:14.809 INFO:teuthology.orchestra.run.smithi169.stdout: "last_pg_merge_meta": { 2023-10-26T01:33:14.809 INFO:teuthology.orchestra.run.smithi169.stdout: "source_pgid": "0.0", 2023-10-26T01:33:14.809 INFO:teuthology.orchestra.run.smithi169.stdout: "ready_epoch": 0, 2023-10-26T01:33:14.809 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_started": 0, 2023-10-26T01:33:14.810 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_clean": 0, 2023-10-26T01:33:14.810 INFO:teuthology.orchestra.run.smithi169.stdout: "source_version": "0'0", 2023-10-26T01:33:14.810 INFO:teuthology.orchestra.run.smithi169.stdout: "target_version": "0'0" 2023-10-26T01:33:14.810 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.810 INFO:teuthology.orchestra.run.smithi169.stdout: "last_change": "54", 2023-10-26T01:33:14.810 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend": "0", 2023-10-26T01:33:14.811 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T01:33:14.811 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T01:33:14.811 INFO:teuthology.orchestra.run.smithi169.stdout: "auid": 0, 2023-10-26T01:33:14.811 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_mode": "selfmanaged", 2023-10-26T01:33:14.811 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_seq": 0, 2023-10-26T01:33:14.812 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_epoch": 0, 2023-10-26T01:33:14.812 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_snaps": [], 2023-10-26T01:33:14.812 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps": "[]", 2023-10-26T01:33:14.812 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_bytes": 0, 2023-10-26T01:33:14.812 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_objects": 0, 2023-10-26T01:33:14.812 INFO:teuthology.orchestra.run.smithi169.stdout: "tiers": [], 2023-10-26T01:33:14.813 INFO:teuthology.orchestra.run.smithi169.stdout: "tier_of": -1, 2023-10-26T01:33:14.813 INFO:teuthology.orchestra.run.smithi169.stdout: "read_tier": -1, 2023-10-26T01:33:14.813 INFO:teuthology.orchestra.run.smithi169.stdout: "write_tier": -1, 2023-10-26T01:33:14.813 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_mode": "none", 2023-10-26T01:33:14.813 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_bytes": 0, 2023-10-26T01:33:14.814 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_objects": 0, 2023-10-26T01:33:14.814 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T01:33:14.814 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T01:33:14.814 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T01:33:14.814 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_flush_age": 0, 2023-10-26T01:33:14.815 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_evict_age": 0, 2023-10-26T01:33:14.815 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profile": "", 2023-10-26T01:33:14.815 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_params": { 2023-10-26T01:33:14.815 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "none" 2023-10-26T01:33:14.815 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.815 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_period": 0, 2023-10-26T01:33:14.816 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_count": 0, 2023-10-26T01:33:14.816 INFO:teuthology.orchestra.run.smithi169.stdout: "use_gmt_hitset": true, 2023-10-26T01:33:14.816 INFO:teuthology.orchestra.run.smithi169.stdout: "min_read_recency_for_promote": 0, 2023-10-26T01:33:14.816 INFO:teuthology.orchestra.run.smithi169.stdout: "min_write_recency_for_promote": 0, 2023-10-26T01:33:14.816 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T01:33:14.817 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_search_last_n": 0, 2023-10-26T01:33:14.817 INFO:teuthology.orchestra.run.smithi169.stdout: "grade_table": [], 2023-10-26T01:33:14.817 INFO:teuthology.orchestra.run.smithi169.stdout: "stripe_width": 0, 2023-10-26T01:33:14.817 INFO:teuthology.orchestra.run.smithi169.stdout: "expected_num_objects": 0, 2023-10-26T01:33:14.817 INFO:teuthology.orchestra.run.smithi169.stdout: "fast_read": false, 2023-10-26T01:33:14.817 INFO:teuthology.orchestra.run.smithi169.stdout: "options": { 2023-10-26T01:33:14.818 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_bias": 4, 2023-10-26T01:33:14.818 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_min": 16, 2023-10-26T01:33:14.818 INFO:teuthology.orchestra.run.smithi169.stdout: "recovery_priority": 5 2023-10-26T01:33:14.818 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.818 INFO:teuthology.orchestra.run.smithi169.stdout: "application_metadata": { 2023-10-26T01:33:14.819 INFO:teuthology.orchestra.run.smithi169.stdout: "cephfs": { 2023-10-26T01:33:14.819 INFO:teuthology.orchestra.run.smithi169.stdout: "metadata": "cephfs" 2023-10-26T01:33:14.819 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:14.819 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.819 INFO:teuthology.orchestra.run.smithi169.stdout: "read_balance": { 2023-10-26T01:33:14.819 INFO:teuthology.orchestra.run.smithi169.stdout: "score_acting": 1.75, 2023-10-26T01:33:14.820 INFO:teuthology.orchestra.run.smithi169.stdout: "score_stable": 1.75, 2023-10-26T01:33:14.820 INFO:teuthology.orchestra.run.smithi169.stdout: "optimal_score": 1, 2023-10-26T01:33:14.820 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_acting": 1.75, 2023-10-26T01:33:14.820 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_stable": 1.75, 2023-10-26T01:33:14.820 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity_weighted": 1, 2023-10-26T01:33:14.821 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity": 1, 2023-10-26T01:33:14.821 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity_weighted": 1 2023-10-26T01:33:14.821 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:14.821 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.821 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.821 INFO:teuthology.orchestra.run.smithi169.stdout: "pool": 7, 2023-10-26T01:33:14.822 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_name": "cephfs_data", 2023-10-26T01:33:14.822 INFO:teuthology.orchestra.run.smithi169.stdout: "create_time": "2023-10-26T01:32:37.059775+0000", 2023-10-26T01:33:14.822 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": 1, 2023-10-26T01:33:14.822 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_names": "hashpspool", 2023-10-26T01:33:14.822 INFO:teuthology.orchestra.run.smithi169.stdout: "type": 1, 2023-10-26T01:33:14.823 INFO:teuthology.orchestra.run.smithi169.stdout: "size": 2, 2023-10-26T01:33:14.823 INFO:teuthology.orchestra.run.smithi169.stdout: "min_size": 1, 2023-10-26T01:33:14.823 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_rule": 0, 2023-10-26T01:33:14.823 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_count": 0, 2023-10-26T01:33:14.823 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_target": 0, 2023-10-26T01:33:14.823 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T01:33:14.824 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T01:33:14.824 INFO:teuthology.orchestra.run.smithi169.stdout: "object_hash": 2, 2023-10-26T01:33:14.824 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_mode": "off", 2023-10-26T01:33:14.824 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num": 64, 2023-10-26T01:33:14.824 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num": 64, 2023-10-26T01:33:14.825 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num_target": 64, 2023-10-26T01:33:14.825 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_target": 64, 2023-10-26T01:33:14.825 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_pending": 64, 2023-10-26T01:33:14.825 INFO:teuthology.orchestra.run.smithi169.stdout: "last_pg_merge_meta": { 2023-10-26T01:33:14.825 INFO:teuthology.orchestra.run.smithi169.stdout: "source_pgid": "0.0", 2023-10-26T01:33:14.826 INFO:teuthology.orchestra.run.smithi169.stdout: "ready_epoch": 0, 2023-10-26T01:33:14.826 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_started": 0, 2023-10-26T01:33:14.826 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_clean": 0, 2023-10-26T01:33:14.826 INFO:teuthology.orchestra.run.smithi169.stdout: "source_version": "0'0", 2023-10-26T01:33:14.826 INFO:teuthology.orchestra.run.smithi169.stdout: "target_version": "0'0" 2023-10-26T01:33:14.827 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.827 INFO:teuthology.orchestra.run.smithi169.stdout: "last_change": "54", 2023-10-26T01:33:14.827 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend": "0", 2023-10-26T01:33:14.827 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T01:33:14.827 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T01:33:14.828 INFO:teuthology.orchestra.run.smithi169.stdout: "auid": 0, 2023-10-26T01:33:14.828 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_mode": "selfmanaged", 2023-10-26T01:33:14.828 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_seq": 0, 2023-10-26T01:33:14.828 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_epoch": 0, 2023-10-26T01:33:14.828 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_snaps": [], 2023-10-26T01:33:14.828 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps": "[]", 2023-10-26T01:33:14.829 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_bytes": 0, 2023-10-26T01:33:14.829 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_objects": 0, 2023-10-26T01:33:14.829 INFO:teuthology.orchestra.run.smithi169.stdout: "tiers": [], 2023-10-26T01:33:14.829 INFO:teuthology.orchestra.run.smithi169.stdout: "tier_of": -1, 2023-10-26T01:33:14.829 INFO:teuthology.orchestra.run.smithi169.stdout: "read_tier": -1, 2023-10-26T01:33:14.830 INFO:teuthology.orchestra.run.smithi169.stdout: "write_tier": -1, 2023-10-26T01:33:14.830 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_mode": "none", 2023-10-26T01:33:14.830 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_bytes": 0, 2023-10-26T01:33:14.830 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_objects": 0, 2023-10-26T01:33:14.830 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T01:33:14.831 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T01:33:14.831 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T01:33:14.831 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_flush_age": 0, 2023-10-26T01:33:14.831 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_evict_age": 0, 2023-10-26T01:33:14.831 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profile": "", 2023-10-26T01:33:14.831 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_params": { 2023-10-26T01:33:14.832 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "none" 2023-10-26T01:33:14.832 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.832 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_period": 0, 2023-10-26T01:33:14.832 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_count": 0, 2023-10-26T01:33:14.832 INFO:teuthology.orchestra.run.smithi169.stdout: "use_gmt_hitset": true, 2023-10-26T01:33:14.833 INFO:teuthology.orchestra.run.smithi169.stdout: "min_read_recency_for_promote": 0, 2023-10-26T01:33:14.833 INFO:teuthology.orchestra.run.smithi169.stdout: "min_write_recency_for_promote": 0, 2023-10-26T01:33:14.833 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T01:33:14.833 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_search_last_n": 0, 2023-10-26T01:33:14.833 INFO:teuthology.orchestra.run.smithi169.stdout: "grade_table": [], 2023-10-26T01:33:14.834 INFO:teuthology.orchestra.run.smithi169.stdout: "stripe_width": 0, 2023-10-26T01:33:14.834 INFO:teuthology.orchestra.run.smithi169.stdout: "expected_num_objects": 0, 2023-10-26T01:33:14.834 INFO:teuthology.orchestra.run.smithi169.stdout: "fast_read": false, 2023-10-26T01:33:14.834 INFO:teuthology.orchestra.run.smithi169.stdout: "options": { 2023-10-26T01:33:14.834 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_min": 64, 2023-10-26T01:33:14.834 INFO:teuthology.orchestra.run.smithi169.stdout: "target_size_ratio": 0.89999997615814209 2023-10-26T01:33:14.835 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.835 INFO:teuthology.orchestra.run.smithi169.stdout: "application_metadata": { 2023-10-26T01:33:14.835 INFO:teuthology.orchestra.run.smithi169.stdout: "cephfs": { 2023-10-26T01:33:14.835 INFO:teuthology.orchestra.run.smithi169.stdout: "data": "cephfs" 2023-10-26T01:33:14.835 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:14.835 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.836 INFO:teuthology.orchestra.run.smithi169.stdout: "read_balance": { 2023-10-26T01:33:14.836 INFO:teuthology.orchestra.run.smithi169.stdout: "score_acting": 1.75, 2023-10-26T01:33:14.836 INFO:teuthology.orchestra.run.smithi169.stdout: "score_stable": 1.75, 2023-10-26T01:33:14.836 INFO:teuthology.orchestra.run.smithi169.stdout: "optimal_score": 1, 2023-10-26T01:33:14.836 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_acting": 1.75, 2023-10-26T01:33:14.836 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_stable": 1.75, 2023-10-26T01:33:14.837 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity_weighted": 1, 2023-10-26T01:33:14.837 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity": 1, 2023-10-26T01:33:14.837 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity_weighted": 1 2023-10-26T01:33:14.837 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:14.837 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:14.838 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:33:14.838 INFO:teuthology.orchestra.run.smithi169.stdout: "osds": [ 2023-10-26T01:33:14.838 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.838 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 0, 2023-10-26T01:33:14.838 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:33:14.838 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:33:14.839 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:33:14.839 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:33:14.839 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:33:14.839 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:33:14.839 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:33:14.840 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:33:14.840 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 47, 2023-10-26T01:33:14.840 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:33:14.840 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:33:14.840 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:33:14.840 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:14.841 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.841 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:14.841 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6816", 2023-10-26T01:33:14.841 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:33:14.841 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.842 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.842 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:14.842 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6817", 2023-10-26T01:33:14.842 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:33:14.842 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:14.843 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:14.843 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.843 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:33:14.843 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:14.843 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.843 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:14.844 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6818", 2023-10-26T01:33:14.844 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:33:14.844 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.844 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.844 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:14.845 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6819", 2023-10-26T01:33:14.845 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:33:14.845 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:14.845 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:14.845 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.845 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:33:14.846 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:14.846 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.846 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:14.846 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6822", 2023-10-26T01:33:14.846 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:33:14.847 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.847 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.847 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:14.847 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6823", 2023-10-26T01:33:14.847 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:33:14.847 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:14.848 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:14.848 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.848 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:33:14.848 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:14.848 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.849 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:14.849 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6820", 2023-10-26T01:33:14.849 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:33:14.849 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.849 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.849 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:14.850 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6821", 2023-10-26T01:33:14.850 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:33:14.850 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:14.850 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:14.850 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.851 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6817/4204255864", 2023-10-26T01:33:14.851 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6819/4204255864", 2023-10-26T01:33:14.851 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6823/4204255864", 2023-10-26T01:33:14.851 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6821/4204255864", 2023-10-26T01:33:14.851 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:33:14.851 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:33:14.852 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:33:14.852 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:14.852 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.852 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.852 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 1, 2023-10-26T01:33:14.853 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:33:14.853 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:33:14.853 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:33:14.853 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:33:14.853 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:33:14.853 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:33:14.854 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:33:14.854 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:33:14.854 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 47, 2023-10-26T01:33:14.854 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:33:14.854 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:33:14.855 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:33:14.855 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:14.855 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.855 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:14.855 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6808", 2023-10-26T01:33:14.855 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:33:14.857 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.857 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.857 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:14.858 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6809", 2023-10-26T01:33:14.858 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:33:14.858 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:14.858 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:14.858 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.858 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:33:14.859 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:14.859 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.859 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:14.859 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6810", 2023-10-26T01:33:14.859 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:33:14.860 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.860 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.860 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:14.860 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6811", 2023-10-26T01:33:14.860 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:33:14.861 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:14.861 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:14.861 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.861 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:33:14.861 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:14.861 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.862 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:14.862 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6814", 2023-10-26T01:33:14.862 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:33:14.862 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.862 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.863 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:14.863 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6815", 2023-10-26T01:33:14.863 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:33:14.863 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:14.863 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:14.864 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.864 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:33:14.864 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:14.864 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.864 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:14.865 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6812", 2023-10-26T01:33:14.865 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:33:14.865 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.865 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.865 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:14.865 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6813", 2023-10-26T01:33:14.866 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:33:14.866 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:14.866 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:14.866 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.866 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6809/238206046", 2023-10-26T01:33:14.867 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6811/238206046", 2023-10-26T01:33:14.867 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6815/238206046", 2023-10-26T01:33:14.867 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6813/238206046", 2023-10-26T01:33:14.867 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:33:14.867 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:33:14.868 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:33:14.868 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:14.868 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.868 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.868 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 2, 2023-10-26T01:33:14.869 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:33:14.869 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:33:14.869 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:33:14.869 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:33:14.869 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:33:14.869 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:33:14.870 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:33:14.870 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:33:14.870 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 47, 2023-10-26T01:33:14.870 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:33:14.870 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:33:14.871 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:33:14.871 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:14.871 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.871 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:14.871 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6800", 2023-10-26T01:33:14.872 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:33:14.872 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.872 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.872 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:14.872 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6801", 2023-10-26T01:33:14.873 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:33:14.873 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:14.873 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:14.873 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.873 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:33:14.873 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:14.874 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.874 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:14.874 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6802", 2023-10-26T01:33:14.874 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:33:14.874 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.875 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.875 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:14.875 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6803", 2023-10-26T01:33:14.875 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:33:14.875 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:14.876 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:14.876 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.876 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:33:14.876 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:14.876 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.876 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:14.877 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6806", 2023-10-26T01:33:14.877 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:33:14.877 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.877 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.877 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:14.878 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6807", 2023-10-26T01:33:14.878 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:33:14.878 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:14.878 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:14.878 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.878 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:33:14.879 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:14.879 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.879 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:14.879 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6804", 2023-10-26T01:33:14.879 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:33:14.880 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.880 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.880 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:14.880 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6805", 2023-10-26T01:33:14.880 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:33:14.880 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:14.881 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:14.881 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.881 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6801/3215791846", 2023-10-26T01:33:14.881 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6803/3215791846", 2023-10-26T01:33:14.881 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6807/3215791846", 2023-10-26T01:33:14.882 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6805/3215791846", 2023-10-26T01:33:14.882 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:33:14.882 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:33:14.882 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:33:14.882 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:14.882 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.883 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.883 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 3, 2023-10-26T01:33:14.883 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:33:14.883 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:33:14.883 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:33:14.884 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:33:14.884 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:33:14.884 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:33:14.884 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:33:14.884 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 14, 2023-10-26T01:33:14.884 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 47, 2023-10-26T01:33:14.885 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:33:14.885 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:33:14.885 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:33:14.885 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:14.885 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.886 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:14.886 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6824", 2023-10-26T01:33:14.886 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:33:14.886 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.886 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.886 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:14.887 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6825", 2023-10-26T01:33:14.887 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:33:14.887 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:14.887 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:14.887 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.888 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:33:14.888 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:14.888 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.888 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:14.888 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6826", 2023-10-26T01:33:14.889 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:33:14.889 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.889 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.889 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:14.889 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6827", 2023-10-26T01:33:14.889 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:33:14.890 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:14.890 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:14.890 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.890 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:33:14.890 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:14.891 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.891 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:14.891 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6830", 2023-10-26T01:33:14.891 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:33:14.891 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.892 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.892 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:14.892 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6831", 2023-10-26T01:33:14.892 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:33:14.892 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:14.893 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:14.893 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.893 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:33:14.893 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:14.893 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.894 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:14.894 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6828", 2023-10-26T01:33:14.894 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:33:14.894 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.894 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.894 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:14.895 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6829", 2023-10-26T01:33:14.895 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:33:14.895 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:14.895 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:14.895 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.896 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6825/2256701226", 2023-10-26T01:33:14.896 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6827/2256701226", 2023-10-26T01:33:14.896 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6831/2256701226", 2023-10-26T01:33:14.896 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6829/2256701226", 2023-10-26T01:33:14.896 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:33:14.896 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:33:14.897 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:33:14.897 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:14.897 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.897 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.897 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 4, 2023-10-26T01:33:14.898 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:33:14.898 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:33:14.898 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:33:14.898 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:33:14.898 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:33:14.898 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:33:14.899 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:33:14.899 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:33:14.899 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 47, 2023-10-26T01:33:14.899 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:33:14.899 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:33:14.900 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:33:14.900 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:14.900 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.900 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:14.900 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6816", 2023-10-26T01:33:14.900 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:33:14.901 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.901 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.901 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:14.901 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6817", 2023-10-26T01:33:14.901 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:33:14.902 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:14.902 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:14.902 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.902 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:33:14.902 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:14.903 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.903 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:14.903 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6818", 2023-10-26T01:33:14.903 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:33:14.903 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.903 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.904 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:14.904 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6819", 2023-10-26T01:33:14.904 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:33:14.904 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:14.904 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:14.904 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.905 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:33:14.905 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:14.905 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.905 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:14.905 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6825", 2023-10-26T01:33:14.906 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:33:14.906 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.906 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.906 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:14.906 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6828", 2023-10-26T01:33:14.907 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:33:14.907 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:14.907 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:14.907 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.907 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:33:14.907 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:14.908 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.908 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:14.908 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6820", 2023-10-26T01:33:14.908 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:33:14.908 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.909 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.909 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:14.909 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6823", 2023-10-26T01:33:14.909 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:33:14.909 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:14.909 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:14.910 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.910 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6817/1082279874", 2023-10-26T01:33:14.910 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6819/1082279874", 2023-10-26T01:33:14.910 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6828/1082279874", 2023-10-26T01:33:14.910 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6823/1082279874", 2023-10-26T01:33:14.911 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:33:14.911 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:33:14.911 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:33:14.911 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:14.911 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.912 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.912 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 5, 2023-10-26T01:33:14.912 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:33:14.912 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:33:14.912 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:33:14.912 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:33:14.913 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:33:14.913 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:33:14.913 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:33:14.913 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:33:14.913 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 47, 2023-10-26T01:33:14.914 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:33:14.914 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:33:14.914 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:33:14.914 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:14.914 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.915 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:14.915 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6808", 2023-10-26T01:33:14.915 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:33:14.915 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.915 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.916 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:14.916 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6809", 2023-10-26T01:33:14.916 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:33:14.916 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:14.916 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:14.916 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.917 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:33:14.917 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:14.917 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.917 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:14.917 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6810", 2023-10-26T01:33:14.918 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:33:14.918 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.918 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.918 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:14.918 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6811", 2023-10-26T01:33:14.918 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:33:14.919 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:14.919 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:14.919 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.919 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:33:14.919 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:14.920 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.920 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:14.920 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6814", 2023-10-26T01:33:14.920 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:33:14.920 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.920 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.921 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:14.921 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6815", 2023-10-26T01:33:14.921 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:33:14.921 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:14.921 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:14.922 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.922 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:33:14.922 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:14.922 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.922 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:14.923 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6812", 2023-10-26T01:33:14.923 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:33:14.923 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.923 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.923 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:14.923 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6813", 2023-10-26T01:33:14.924 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:33:14.924 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:14.924 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:14.924 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.924 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6809/3335654351", 2023-10-26T01:33:14.925 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6811/3335654351", 2023-10-26T01:33:14.925 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6815/3335654351", 2023-10-26T01:33:14.925 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6813/3335654351", 2023-10-26T01:33:14.925 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:33:14.925 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:33:14.925 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:33:14.926 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:14.926 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.926 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.926 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 6, 2023-10-26T01:33:14.926 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:33:14.927 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:33:14.927 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:33:14.927 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:33:14.927 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:33:14.927 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:33:14.928 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:33:14.928 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:33:14.928 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 47, 2023-10-26T01:33:14.928 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:33:14.928 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:33:14.928 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:33:14.929 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:14.929 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.929 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:14.929 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6800", 2023-10-26T01:33:14.929 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:33:14.930 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.930 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.930 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:14.930 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6801", 2023-10-26T01:33:14.930 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:33:14.931 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:14.931 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:14.931 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.931 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:33:14.931 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:14.931 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.932 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:14.932 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6802", 2023-10-26T01:33:14.932 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:33:14.932 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.932 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.933 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:14.933 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6803", 2023-10-26T01:33:14.933 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:33:14.933 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:14.933 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:14.933 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.934 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:33:14.934 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:14.934 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.934 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:14.934 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6806", 2023-10-26T01:33:14.935 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:33:14.935 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.935 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.935 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:14.935 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6807", 2023-10-26T01:33:14.935 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:33:14.936 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:14.936 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:14.936 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.936 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:33:14.936 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:14.937 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.937 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:14.937 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6804", 2023-10-26T01:33:14.937 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:33:14.937 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.938 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.938 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:14.938 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6805", 2023-10-26T01:33:14.938 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:33:14.938 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:14.938 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:14.939 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.939 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6801/2520460697", 2023-10-26T01:33:14.939 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6803/2520460697", 2023-10-26T01:33:14.939 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6807/2520460697", 2023-10-26T01:33:14.939 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6805/2520460697", 2023-10-26T01:33:14.939 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:33:14.940 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:33:14.940 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:33:14.940 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:14.940 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.940 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.941 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 7, 2023-10-26T01:33:14.941 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:33:14.941 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:33:14.941 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:33:14.941 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:33:14.941 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:33:14.942 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:33:14.942 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:33:14.942 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:33:14.942 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 47, 2023-10-26T01:33:14.942 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:33:14.943 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:33:14.943 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:33:14.943 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:14.943 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.943 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:14.944 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6821", 2023-10-26T01:33:14.944 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:33:14.944 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.944 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.944 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:14.945 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6822", 2023-10-26T01:33:14.945 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:33:14.945 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:14.945 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:14.945 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.945 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:33:14.946 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:14.946 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.946 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:14.946 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6824", 2023-10-26T01:33:14.946 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:33:14.947 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.947 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.947 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:14.947 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6826", 2023-10-26T01:33:14.947 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:33:14.948 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:14.948 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:14.948 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.948 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:33:14.948 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:14.949 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.949 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:14.949 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6830", 2023-10-26T01:33:14.949 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:33:14.949 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.949 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.950 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:14.950 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6831", 2023-10-26T01:33:14.950 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:33:14.950 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:14.950 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:14.951 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.951 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:33:14.951 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:33:14.951 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.951 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:33:14.952 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6827", 2023-10-26T01:33:14.952 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:33:14.952 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.952 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.952 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:33:14.953 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6829", 2023-10-26T01:33:14.953 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:33:14.953 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:14.953 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:14.953 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.953 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6822/1115930619", 2023-10-26T01:33:14.954 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6826/1115930619", 2023-10-26T01:33:14.954 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6831/1115930619", 2023-10-26T01:33:14.954 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6829/1115930619", 2023-10-26T01:33:14.954 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:33:14.954 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:33:14.954 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:33:14.955 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:14.955 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:14.955 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:33:14.955 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_xinfo": [ 2023-10-26T01:33:14.955 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.956 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 0, 2023-10-26T01:33:14.956 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:33:14.956 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:33:14.956 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:33:14.956 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:33:14.956 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:33:14.957 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.889675+0000", 2023-10-26T01:33:14.957 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:33:14.957 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.957 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.957 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 1, 2023-10-26T01:33:14.958 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:33:14.958 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:33:14.958 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:33:14.958 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:33:14.958 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:33:14.959 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.864157+0000", 2023-10-26T01:33:14.959 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:33:14.959 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.959 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.959 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 2, 2023-10-26T01:33:14.960 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:33:14.960 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:33:14.960 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:33:14.960 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:33:14.960 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:33:14.961 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.876919+0000", 2023-10-26T01:33:14.961 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:33:14.961 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.961 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.961 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 3, 2023-10-26T01:33:14.961 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:33:14.962 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:33:14.962 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:33:14.962 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:33:14.962 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:33:14.962 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.974541+0000", 2023-10-26T01:33:14.963 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:33:14.963 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.963 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.963 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 4, 2023-10-26T01:33:14.963 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:33:14.964 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:33:14.964 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:33:14.964 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:33:14.964 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:33:14.964 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.969696+0000", 2023-10-26T01:33:14.964 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:33:14.965 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.965 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.965 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 5, 2023-10-26T01:33:14.965 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:33:14.965 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:33:14.966 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:33:14.966 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:33:14.966 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:33:14.966 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.859703+0000", 2023-10-26T01:33:14.966 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:33:14.967 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.967 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.967 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 6, 2023-10-26T01:33:14.967 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:33:14.967 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:33:14.967 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:33:14.968 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:33:14.968 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:33:14.968 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.867917+0000", 2023-10-26T01:33:14.968 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:33:14.968 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.969 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:14.969 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 7, 2023-10-26T01:33:14.969 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:33:14.969 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:33:14.969 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:33:14.969 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:33:14.970 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:33:14.970 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.898605+0000", 2023-10-26T01:33:14.970 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:33:14.970 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:14.970 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:33:14.971 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_upmap": [], 2023-10-26T01:33:14.971 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_upmap_items": [], 2023-10-26T01:33:14.971 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_upmap_primaries": [], 2023-10-26T01:33:14.971 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_temp": [], 2023-10-26T01:33:14.971 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_temp": [], 2023-10-26T01:33:14.971 INFO:teuthology.orchestra.run.smithi169.stdout: "blocklist": {}, 2023-10-26T01:33:14.972 INFO:teuthology.orchestra.run.smithi169.stdout: "range_blocklist": {}, 2023-10-26T01:33:14.972 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profiles": { 2023-10-26T01:33:14.972 INFO:teuthology.orchestra.run.smithi169.stdout: "default": { 2023-10-26T01:33:14.972 INFO:teuthology.orchestra.run.smithi169.stdout: "crush-failure-domain": "osd", 2023-10-26T01:33:14.972 INFO:teuthology.orchestra.run.smithi169.stdout: "k": "2", 2023-10-26T01:33:14.973 INFO:teuthology.orchestra.run.smithi169.stdout: "m": "1", 2023-10-26T01:33:14.973 INFO:teuthology.orchestra.run.smithi169.stdout: "plugin": "jerasure", 2023-10-26T01:33:14.973 INFO:teuthology.orchestra.run.smithi169.stdout: "technique": "reed_sol_van" 2023-10-26T01:33:14.973 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:14.973 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:14.974 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps_queue": [], 2023-10-26T01:33:14.974 INFO:teuthology.orchestra.run.smithi169.stdout: "new_removed_snaps": [], 2023-10-26T01:33:14.974 INFO:teuthology.orchestra.run.smithi169.stdout: "new_purged_snaps": [], 2023-10-26T01:33:14.974 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_node_flags": {}, 2023-10-26T01:33:14.974 INFO:teuthology.orchestra.run.smithi169.stdout: "device_class_flags": {}, 2023-10-26T01:33:14.975 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_mode": { 2023-10-26T01:33:14.975 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_mode_enabled": false, 2023-10-26T01:33:14.975 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_bucket_count": 0, 2023-10-26T01:33:14.975 INFO:teuthology.orchestra.run.smithi169.stdout: "degraded_stretch_mode": 0, 2023-10-26T01:33:14.975 INFO:teuthology.orchestra.run.smithi169.stdout: "recovering_stretch_mode": 0, 2023-10-26T01:33:14.975 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_mode_bucket": 0 2023-10-26T01:33:14.976 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:14.976 INFO:teuthology.orchestra.run.smithi169.stdout:} 2023-10-26T01:33:14.978 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:14.783+0000 7f3b65e1a640 1 -- 172.21.15.169:0/3516870202 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3b4006ae40 msgr2=0x7f3b4006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:14.978 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:14.783+0000 7f3b65e1a640 1 --2- 172.21.15.169:0/3516870202 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3b4006ae40 0x7f3b4006d300 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f3b50005fd0 tx=0x7f3b50005850 comp rx=0 tx=0).stop 2023-10-26T01:33:14.978 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:14.783+0000 7f3b65e1a640 1 -- 172.21.15.169:0/3516870202 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b601592d0 msgr2=0x7f3b600c7980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:14.978 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:14.783+0000 7f3b65e1a640 1 --2- 172.21.15.169:0/3516870202 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b601592d0 0x7f3b600c7980 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f3b58037040 tx=0x7f3b58035a40 comp rx=0 tx=0).stop 2023-10-26T01:33:14.978 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:14.783+0000 7f3b65e1a640 1 -- 172.21.15.169:0/3516870202 shutdown_connections 2023-10-26T01:33:14.979 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:14.783+0000 7f3b65e1a640 1 --2- 172.21.15.169:0/3516870202 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3b4006ae40 0x7f3b4006d300 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:14.979 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:14.783+0000 7f3b65e1a640 1 --2- 172.21.15.169:0/3516870202 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b601592d0 0x7f3b600c7980 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:14.979 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:14.783+0000 7f3b65e1a640 1 --2- 172.21.15.169:0/3516870202 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3b600cde00 0x7f3b6014a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:14.979 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:14.783+0000 7f3b65e1a640 1 --2- 172.21.15.169:0/3516870202 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3b600ac930 0x7f3b601f19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:14.979 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:14.783+0000 7f3b65e1a640 1 -- 172.21.15.169:0/3516870202 >> 172.21.15.169:0/3516870202 conn(0x7f3b60069930 msgr2=0x7f3b60157f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:14.980 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:14.783+0000 7f3b65e1a640 1 -- 172.21.15.169:0/3516870202 shutdown_connections 2023-10-26T01:33:14.980 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:14.783+0000 7f3b65e1a640 1 -- 172.21.15.169:0/3516870202 wait complete. 2023-10-26T01:33:14.981 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T01:33:15.207 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:15.204+0000 7f7c69610640 1 -- 172.21.15.169:0/1120148749 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c640cdc30 msgr2=0x7f7c6406ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:15.208 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:15.204+0000 7f7c69610640 1 --2- 172.21.15.169:0/1120148749 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c640cdc30 0x7f7c6406ad30 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f7c640b3440 tx=0x7f7c5c02c1c0 comp rx=0 tx=0).stop 2023-10-26T01:33:15.208 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:15.204+0000 7f7c69610640 1 -- 172.21.15.169:0/1120148749 shutdown_connections 2023-10-26T01:33:15.208 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:15.204+0000 7f7c69610640 1 --2- 172.21.15.169:0/1120148749 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c640cdc30 0x7f7c6406ad30 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:15.208 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:15.204+0000 7f7c69610640 1 --2- 172.21.15.169:0/1120148749 >> v2:172.21.15.170:3300/0 conn(0x7f7c640ac930 0x7f7c640cd6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:15.208 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:15.204+0000 7f7c69610640 1 --2- 172.21.15.169:0/1120148749 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7c641592d0 0x7f7c640aa3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:15.209 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:15.204+0000 7f7c69610640 1 -- 172.21.15.169:0/1120148749 >> 172.21.15.169:0/1120148749 conn(0x7f7c64069930 msgr2=0x7f7c64069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:15.209 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:15.205+0000 7f7c69610640 1 -- 172.21.15.169:0/1120148749 shutdown_connections 2023-10-26T01:33:15.209 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:15.205+0000 7f7c69610640 1 -- 172.21.15.169:0/1120148749 wait complete. 2023-10-26T01:33:15.211 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:15.206+0000 7f7c69610640 1 Processor -- start 2023-10-26T01:33:15.212 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:15.207+0000 7f7c69610640 1 -- start start 2023-10-26T01:33:15.212 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:15.207+0000 7f7c69610640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c640ac930 0x7f7c641518c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:15.212 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:15.207+0000 7f7c69610640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c640cdc30 0x7f7c64070ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:15.212 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:15.207+0000 7f7c69610640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7c641592d0 0x7f7c6415b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:15.213 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:15.207+0000 7f7c69610640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c64070630 con 0x7f7c640cdc30 2023-10-26T01:33:15.213 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:15.208+0000 7f7c69610640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c6406f740 con 0x7f7c640ac930 2023-10-26T01:33:15.213 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:15.208+0000 7f7c69610640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c6406fa20 con 0x7f7c641592d0 2023-10-26T01:33:15.213 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:15.208+0000 7f7c5a7fc640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c640cdc30 0x7f7c64070ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:15.213 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:15.208+0000 7f7c5a7fc640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c640cdc30 0x7f7c64070ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:47416/0 (socket says 172.21.15.169:47416) 2023-10-26T01:33:15.214 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:15.208+0000 7f7c5a7fc640 1 -- 172.21.15.169:0/274217422 learned_addr learned my addr 172.21.15.169:0/274217422 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:33:15.214 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:15.208+0000 7f7c62ffd640 1 --2- 172.21.15.169:0/274217422 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c640ac930 0x7f7c641518c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:15.214 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:15.208+0000 7f7c5a7fc640 1 -- 172.21.15.169:0/274217422 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7c641592d0 msgr2=0x7f7c6415b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:15.214 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:15.208+0000 7f7c637fe640 1 --2- 172.21.15.169:0/274217422 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7c641592d0 0x7f7c6415b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:15.214 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:15.208+0000 7f7c5a7fc640 1 --2- 172.21.15.169:0/274217422 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7c641592d0 0x7f7c6415b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:15.215 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:15.208+0000 7f7c5a7fc640 1 -- 172.21.15.169:0/274217422 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c640ac930 msgr2=0x7f7c641518c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:15.215 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:15.208+0000 7f7c5a7fc640 1 --2- 172.21.15.169:0/274217422 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c640ac930 0x7f7c641518c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:15.215 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:15.208+0000 7f7c5a7fc640 1 -- 172.21.15.169:0/274217422 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c5c009900 con 0x7f7c640cdc30 2023-10-26T01:33:15.215 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:15.208+0000 7f7c62ffd640 1 --2- 172.21.15.169:0/274217422 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c640ac930 0x7f7c641518c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:33:15.215 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:15.209+0000 7f7c5a7fc640 1 --2- 172.21.15.169:0/274217422 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c640cdc30 0x7f7c64070ac0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c00c920 tx=0x7f7c4c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:15.216 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:15.209+0000 7f7c5bfff640 1 -- 172.21.15.169:0/274217422 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c4c007c70 con 0x7f7c640cdc30 2023-10-26T01:33:15.216 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:15.209+0000 7f7c5bfff640 1 -- 172.21.15.169:0/274217422 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7c4c00f450 con 0x7f7c640cdc30 2023-10-26T01:33:15.216 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:15.209+0000 7f7c69610640 1 -- 172.21.15.169:0/274217422 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c6406fd00 con 0x7f7c640cdc30 2023-10-26T01:33:15.216 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:15.209+0000 7f7c5bfff640 1 -- 172.21.15.169:0/274217422 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c4c00f860 con 0x7f7c640cdc30 2023-10-26T01:33:15.216 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:15.209+0000 7f7c69610640 1 -- 172.21.15.169:0/274217422 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c640702c0 con 0x7f7c640cdc30 2023-10-26T01:33:15.217 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:15.210+0000 7f7c69610640 1 -- 172.21.15.169:0/274217422 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c6404db10 con 0x7f7c640cdc30 2023-10-26T01:33:15.217 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:15.211+0000 7f7c5bfff640 1 -- 172.21.15.169:0/274217422 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f7c4c00f9c0 con 0x7f7c640cdc30 2023-10-26T01:33:15.218 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:15.212+0000 7f7c5bfff640 1 --2- 172.21.15.169:0/274217422 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7c4406ae40 0x7f7c4406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:15.218 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:15.212+0000 7f7c5bfff640 1 -- 172.21.15.169:0/274217422 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7c4c089f90 con 0x7f7c640cdc30 2023-10-26T01:33:15.218 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:15.213+0000 7f7c62ffd640 1 --2- 172.21.15.169:0/274217422 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7c4406ae40 0x7f7c4406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:15.218 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:15.214+0000 7f7c62ffd640 1 --2- 172.21.15.169:0/274217422 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7c4406ae40 0x7f7c4406d300 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f7c540058c0 tx=0x7f7c54005c30 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:15.219 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:15.217+0000 7f7c5bfff640 1 -- 172.21.15.169:0/274217422 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7c4c054fa0 con 0x7f7c640cdc30 2023-10-26T01:33:15.447 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:15.444+0000 7f7c69610640 1 -- 172.21.15.169:0/274217422 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7c640cd010 con 0x7f7c640cdc30 2023-10-26T01:33:15.448 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:15.446+0000 7f7c5bfff640 1 -- 172.21.15.169:0/274217422 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4925 (secure 0 0 0) 0x7f7c4c01d020 con 0x7f7c640cdc30 2023-10-26T01:33:15.448 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:33:15.449 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4805,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/2294107634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2294107634},{"type":"v1","addr":"172.21.15.170:6836","nonce":2294107634}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":4987,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2208026662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2208026662},{"type":"v1","addr":"172.21.15.169:6833","nonce":2208026662}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":4993,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/2630365058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":2630365058},{"type":"v1","addr":"172.21.15.170:6837","nonce":2630365058}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T01:32:38.059721+0000","modified":"2023-10-26T01:32:41.428183+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5664},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5664":{"gid":5664,"name":"a","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6835/2816266753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":2816266753},{"type":"v1","addr":"172.21.15.169:6835","nonce":2816266753}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-10-26T01:33:15.449 INFO:teuthology.orchestra.run.smithi169.stderr:dumped fsmap epoch 23 2023-10-26T01:33:15.451 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:15.448+0000 7f7c69610640 1 -- 172.21.15.169:0/274217422 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7c4406ae40 msgr2=0x7f7c4406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:15.451 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:15.448+0000 7f7c69610640 1 --2- 172.21.15.169:0/274217422 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7c4406ae40 0x7f7c4406d300 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f7c540058c0 tx=0x7f7c54005c30 comp rx=0 tx=0).stop 2023-10-26T01:33:15.452 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:15.448+0000 7f7c69610640 1 -- 172.21.15.169:0/274217422 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c640cdc30 msgr2=0x7f7c64070ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:15.452 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:15.448+0000 7f7c69610640 1 --2- 172.21.15.169:0/274217422 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c640cdc30 0x7f7c64070ac0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c00c920 tx=0x7f7c4c00cde0 comp rx=0 tx=0).stop 2023-10-26T01:33:15.452 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:15.448+0000 7f7c69610640 1 -- 172.21.15.169:0/274217422 shutdown_connections 2023-10-26T01:33:15.452 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:15.448+0000 7f7c69610640 1 --2- 172.21.15.169:0/274217422 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7c4406ae40 0x7f7c4406d300 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:15.452 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:15.448+0000 7f7c69610640 1 --2- 172.21.15.169:0/274217422 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c640cdc30 0x7f7c64070ac0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:15.453 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:15.448+0000 7f7c69610640 1 --2- 172.21.15.169:0/274217422 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7c641592d0 0x7f7c6415b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:15.453 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:15.448+0000 7f7c69610640 1 --2- 172.21.15.169:0/274217422 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c640ac930 0x7f7c641518c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:15.453 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:15.448+0000 7f7c69610640 1 -- 172.21.15.169:0/274217422 >> 172.21.15.169:0/274217422 conn(0x7f7c64069930 msgr2=0x7f7c640addc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:15.453 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:15.448+0000 7f7c69610640 1 -- 172.21.15.169:0/274217422 shutdown_connections 2023-10-26T01:33:15.453 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:15.448+0000 7f7c69610640 1 -- 172.21.15.169:0/274217422 wait complete. 2023-10-26T01:33:15.471 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2023-10-26T01:33:15.696 INFO:teuthology.orchestra.run.smithi169.stdout:{"cluster_fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","whoami":0,"id":5664,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":36.761669099000002,"mdsmap_epoch":23,"osdmap_epoch":54,"osdmap_epoch_barrier":51,"uptime":42.944011955000001} 2023-10-26T01:33:15.696 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "30203b28-ed8a-44e8-a9a6-a9bf25f980f1", "fs_name": "cephfs", "id": 5664, "mdsmap_epoch": 23, "osdmap_epoch": 54, "osdmap_epoch_barrier": 51, "rank_uptime": 36.761669099, "state": "up:active", "uptime": 42.944011955, "want_state": "up:active", "whoami": 0 } 2023-10-26T01:33:15.697 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-26T01:33:15.697 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:33:15.697 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:33:15.748 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:33:15.749 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file bs=1M conv=fdatasync count=50 seek=0) 2023-10-26T01:33:16.345 INFO:teuthology.orchestra.run.smithi169.stderr:50+0 records in 2023-10-26T01:33:16.345 INFO:teuthology.orchestra.run.smithi169.stderr:50+0 records out 2023-10-26T01:33:16.345 INFO:teuthology.orchestra.run.smithi169.stderr:52428800 bytes (52 MB, 50 MiB) copied, 0.492501 s, 106 MB/s 2023-10-26T01:33:16.346 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-26T01:33:16.347 DEBUG:teuthology.orchestra.run.smithi169:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-26T01:33:16.453 DEBUG:teuthology.orchestra.run.smithi170:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-26T01:33:16.567 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-26T01:33:16.568 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:33:16.568 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:33:16.626 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:33:16.627 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:33:16.683 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:33:16.904 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:16.900+0000 7f7b729fb640 1 -- 172.21.15.169:0/528171025 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b6c06cae0 msgr2=0x7f7b6c0c7ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:16.904 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:16.900+0000 7f7b729fb640 1 --2- 172.21.15.169:0/528171025 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b6c06cae0 0x7f7b6c0c7ea0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c0ca700 tx=0x7f7b6802c1d0 comp rx=0 tx=0).stop 2023-10-26T01:33:16.904 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:16.901+0000 7f7b729fb640 1 -- 172.21.15.169:0/528171025 shutdown_connections 2023-10-26T01:33:16.905 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:16.901+0000 7f7b729fb640 1 --2- 172.21.15.169:0/528171025 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7b6c0c9c30 0x7f7b6c0cb390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:16.905 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:16.901+0000 7f7b729fb640 1 --2- 172.21.15.169:0/528171025 >> v2:172.21.15.170:3300/0 conn(0x7f7b6c06a990 0x7f7b6c0c96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:16.905 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:16.901+0000 7f7b729fb640 1 --2- 172.21.15.169:0/528171025 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b6c06cae0 0x7f7b6c0c7ea0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:16.905 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:16.901+0000 7f7b729fb640 1 -- 172.21.15.169:0/528171025 >> 172.21.15.169:0/528171025 conn(0x7f7b6c0aedd0 msgr2=0x7f7b6c0af1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:16.906 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:16.901+0000 7f7b729fb640 1 -- 172.21.15.169:0/528171025 shutdown_connections 2023-10-26T01:33:16.906 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:16.901+0000 7f7b729fb640 1 -- 172.21.15.169:0/528171025 wait complete. 2023-10-26T01:33:16.908 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:16.902+0000 7f7b729fb640 1 Processor -- start 2023-10-26T01:33:16.908 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:16.903+0000 7f7b729fb640 1 -- start start 2023-10-26T01:33:16.908 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:16.904+0000 7f7b729fb640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b6c06a990 0x7f7b6c1538d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:16.909 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:16.904+0000 7f7b729fb640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b6c06cae0 0x7f7b6c0ad380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:16.909 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:16.904+0000 7f7b729fb640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7b6c0c9c30 0x7f7b6c0ad8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:16.909 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:16.904+0000 7f7b729fb640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b6c154320 con 0x7f7b6c06a990 2023-10-26T01:33:16.909 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:16.904+0000 7f7b729fb640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b6c154600 con 0x7f7b6c06cae0 2023-10-26T01:33:16.909 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:16.904+0000 7f7b729fb640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b6c1548e0 con 0x7f7b6c0c9c30 2023-10-26T01:33:16.910 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:16.904+0000 7f7b719f9640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b6c06a990 0x7f7b6c1538d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:16.910 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:16.904+0000 7f7b719f9640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b6c06a990 0x7f7b6c1538d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:47434/0 (socket says 172.21.15.169:47434) 2023-10-26T01:33:16.910 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:16.904+0000 7f7b719f9640 1 -- 172.21.15.169:0/4215838394 learned_addr learned my addr 172.21.15.169:0/4215838394 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:33:16.910 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:16.904+0000 7f7b719f9640 1 -- 172.21.15.169:0/4215838394 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7b6c0c9c30 msgr2=0x7f7b6c0ad8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:16.911 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:16.904+0000 7f7b719f9640 1 --2- 172.21.15.169:0/4215838394 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7b6c0c9c30 0x7f7b6c0ad8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:16.911 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:16.904+0000 7f7b719f9640 1 -- 172.21.15.169:0/4215838394 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b6c06cae0 msgr2=0x7f7b6c0ad380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:16.911 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:16.904+0000 7f7b719f9640 1 --2- 172.21.15.169:0/4215838394 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b6c06cae0 0x7f7b6c0ad380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:16.911 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:16.904+0000 7f7b719f9640 1 -- 172.21.15.169:0/4215838394 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b68009900 con 0x7f7b6c06a990 2023-10-26T01:33:16.912 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:16.905+0000 7f7b719f9640 1 --2- 172.21.15.169:0/4215838394 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b6c06a990 0x7f7b6c1538d0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f7b68000f80 tx=0x7f7b68035e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:16.912 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:16.905+0000 7f7b627fc640 1 -- 172.21.15.169:0/4215838394 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b68034440 con 0x7f7b6c06a990 2023-10-26T01:33:16.912 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:16.905+0000 7f7b627fc640 1 -- 172.21.15.169:0/4215838394 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7b68004540 con 0x7f7b6c06a990 2023-10-26T01:33:16.912 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:16.905+0000 7f7b729fb640 1 -- 172.21.15.169:0/4215838394 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b6c138750 con 0x7f7b6c06a990 2023-10-26T01:33:16.912 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:16.905+0000 7f7b627fc640 1 -- 172.21.15.169:0/4215838394 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b6803f440 con 0x7f7b6c06a990 2023-10-26T01:33:16.913 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:16.905+0000 7f7b729fb640 1 -- 172.21.15.169:0/4215838394 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b6c1382b0 con 0x7f7b6c06a990 2023-10-26T01:33:16.914 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:16.906+0000 7f7b729fb640 1 -- 172.21.15.169:0/4215838394 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b6c073c10 con 0x7f7b6c06a990 2023-10-26T01:33:16.914 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:16.907+0000 7f7b627fc640 1 -- 172.21.15.169:0/4215838394 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f7b6803b040 con 0x7f7b6c06a990 2023-10-26T01:33:16.914 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:16.908+0000 7f7b627fc640 1 --2- 172.21.15.169:0/4215838394 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7b5006ae40 0x7f7b5006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:16.914 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:16.908+0000 7f7b627fc640 1 -- 172.21.15.169:0/4215838394 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7b680ac4e0 con 0x7f7b6c06a990 2023-10-26T01:33:16.914 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:16.909+0000 7f7b711f8640 1 --2- 172.21.15.169:0/4215838394 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7b5006ae40 0x7f7b5006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:16.915 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:16.910+0000 7f7b711f8640 1 --2- 172.21.15.169:0/4215838394 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7b5006ae40 0x7f7b5006d300 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c1f4e80 tx=0x7f7b5c005b60 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:16.915 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:16.912+0000 7f7b627fc640 1 -- 172.21.15.169:0/4215838394 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7b680774f0 con 0x7f7b6c06a990 2023-10-26T01:33:17.107 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:17.104+0000 7f7b729fb640 1 -- 172.21.15.169:0/4215838394 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7b6c154df0 con 0x7f7b6c06a990 2023-10-26T01:33:17.108 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:17.106+0000 7f7b627fc640 1 -- 172.21.15.169:0/4215838394 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+16561 (secure 0 0 0) 0x7f7b6807ab10 con 0x7f7b6c06a990 2023-10-26T01:33:17.109 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:33:17.109 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":54,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:33:10.190714+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-26T01:32:36.288217+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-26T01:32:37.059775+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:33:17.112 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:17.109+0000 7f7b729fb640 1 -- 172.21.15.169:0/4215838394 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7b5006ae40 msgr2=0x7f7b5006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:17.113 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:17.109+0000 7f7b729fb640 1 --2- 172.21.15.169:0/4215838394 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7b5006ae40 0x7f7b5006d300 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c1f4e80 tx=0x7f7b5c005b60 comp rx=0 tx=0).stop 2023-10-26T01:33:17.113 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:17.109+0000 7f7b729fb640 1 -- 172.21.15.169:0/4215838394 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b6c06a990 msgr2=0x7f7b6c1538d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:17.113 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:17.109+0000 7f7b729fb640 1 --2- 172.21.15.169:0/4215838394 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b6c06a990 0x7f7b6c1538d0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f7b68000f80 tx=0x7f7b68035e30 comp rx=0 tx=0).stop 2023-10-26T01:33:17.114 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:17.109+0000 7f7b729fb640 1 -- 172.21.15.169:0/4215838394 shutdown_connections 2023-10-26T01:33:17.114 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:17.109+0000 7f7b729fb640 1 --2- 172.21.15.169:0/4215838394 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7b5006ae40 0x7f7b5006d300 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:17.114 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:17.109+0000 7f7b729fb640 1 --2- 172.21.15.169:0/4215838394 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7b6c0c9c30 0x7f7b6c0ad8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:17.115 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:17.109+0000 7f7b729fb640 1 --2- 172.21.15.169:0/4215838394 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b6c06cae0 0x7f7b6c0ad380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:17.115 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:17.109+0000 7f7b729fb640 1 --2- 172.21.15.169:0/4215838394 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b6c06a990 0x7f7b6c1538d0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:17.115 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:17.109+0000 7f7b729fb640 1 -- 172.21.15.169:0/4215838394 >> 172.21.15.169:0/4215838394 conn(0x7f7b6c0aedd0 msgr2=0x7f7b6c159600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:17.115 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:17.109+0000 7f7b729fb640 1 -- 172.21.15.169:0/4215838394 shutdown_connections 2023-10-26T01:33:17.115 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:17.109+0000 7f7b729fb640 1 -- 172.21.15.169:0/4215838394 wait complete. 2023-10-26T01:33:17.131 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T01:30:32.200808+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-26T01:32:36.288217+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-10-26T01:32:37.059775+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T01:33:17.132 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T01:33:17.360 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:17.357+0000 7f6679d16640 1 -- 172.21.15.169:0/1430727821 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f665c003f20 msgr2=0x7f665c0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:17.361 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:17.357+0000 7f6679d16640 1 --2- 172.21.15.169:0/1430727821 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f665c003f20 0x7f665c0047c0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f6668006f90 tx=0x7f665c0057f0 comp rx=0 tx=0).stop 2023-10-26T01:33:17.361 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:17.357+0000 7f6673fff640 1 -- 172.21.15.169:0/1430727821 reap_dead start 2023-10-26T01:33:17.361 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:17.357+0000 7f6679d16640 1 -- 172.21.15.169:0/1430727821 shutdown_connections 2023-10-26T01:33:17.361 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:17.357+0000 7f6679d16640 1 -- 172.21.15.169:0/1430727821 >> 172.21.15.169:0/1430727821 conn(0x7f6674069930 msgr2=0x7f6674069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:17.361 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:17.357+0000 7f6679d16640 1 -- 172.21.15.169:0/1430727821 shutdown_connections 2023-10-26T01:33:17.362 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:17.357+0000 7f6679d16640 1 -- 172.21.15.169:0/1430727821 wait complete. 2023-10-26T01:33:17.362 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:17.359+0000 7f6679d16640 1 Processor -- start 2023-10-26T01:33:17.362 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:17.359+0000 7f6679d16640 1 -- start start 2023-10-26T01:33:17.364 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:17.360+0000 7f6679d16640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f665c003f20 0x7f667414e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:17.365 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:17.360+0000 7f6679d16640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f667414e7a0 0x7f66741ef130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:17.365 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:17.360+0000 7f6679d16640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66741ef670 0x7f66741f3250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:17.365 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:17.360+0000 7f6679d16640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f667414ec20 con 0x7f667414e7a0 2023-10-26T01:33:17.365 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:17.360+0000 7f6679d16640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6674142bb0 con 0x7f66741ef670 2023-10-26T01:33:17.366 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:17.360+0000 7f6679d16640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66741515b0 con 0x7f665c003f20 2023-10-26T01:33:17.366 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:17.361+0000 7f6672ffd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f667414e7a0 0x7f66741ef130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:17.366 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:17.361+0000 7f6672ffd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f667414e7a0 0x7f66741ef130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:47448/0 (socket says 172.21.15.169:47448) 2023-10-26T01:33:17.366 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:17.361+0000 7f6672ffd640 1 -- 172.21.15.169:0/4217798142 learned_addr learned my addr 172.21.15.169:0/4217798142 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:33:17.367 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:17.361+0000 7f6673fff640 1 --2- 172.21.15.169:0/4217798142 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66741ef670 0x7f66741f3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:17.367 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:17.361+0000 7f66737fe640 1 --2- 172.21.15.169:0/4217798142 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f665c003f20 0x7f667414e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:17.367 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:17.361+0000 7f6672ffd640 1 -- 172.21.15.169:0/4217798142 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f665c003f20 msgr2=0x7f667414e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:17.367 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:17.361+0000 7f6672ffd640 1 --2- 172.21.15.169:0/4217798142 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f665c003f20 0x7f667414e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:17.368 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:17.361+0000 7f6672ffd640 1 -- 172.21.15.169:0/4217798142 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66741ef670 msgr2=0x7f66741f3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:17.368 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:17.361+0000 7f6672ffd640 1 --2- 172.21.15.169:0/4217798142 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66741ef670 0x7f66741f3250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:17.368 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:17.361+0000 7f6672ffd640 1 -- 172.21.15.169:0/4217798142 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6674150510 con 0x7f667414e7a0 2023-10-26T01:33:17.368 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:17.361+0000 7f66737fe640 1 --2- 172.21.15.169:0/4217798142 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f665c003f20 0x7f667414e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:33:17.368 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:17.362+0000 7f6672ffd640 1 --2- 172.21.15.169:0/4217798142 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f667414e7a0 0x7f66741ef130 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f666800c350 tx=0x7f66680017c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:17.369 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:17.362+0000 7f665bfff640 1 -- 172.21.15.169:0/4217798142 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6668006230 con 0x7f667414e7a0 2023-10-26T01:33:17.369 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:17.362+0000 7f665bfff640 1 -- 172.21.15.169:0/4217798142 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f66680063b0 con 0x7f667414e7a0 2023-10-26T01:33:17.369 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:17.362+0000 7f665bfff640 1 -- 172.21.15.169:0/4217798142 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66680035f0 con 0x7f667414e7a0 2023-10-26T01:33:17.369 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:17.362+0000 7f6679d16640 1 -- 172.21.15.169:0/4217798142 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66741507f0 con 0x7f667414e7a0 2023-10-26T01:33:17.370 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:17.362+0000 7f6679d16640 1 -- 172.21.15.169:0/4217798142 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6674150db0 con 0x7f667414e7a0 2023-10-26T01:33:17.371 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:17.363+0000 7f6679d16640 1 -- 172.21.15.169:0/4217798142 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6674066af0 con 0x7f667414e7a0 2023-10-26T01:33:17.371 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:17.364+0000 7f665bfff640 1 -- 172.21.15.169:0/4217798142 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f6668031030 con 0x7f667414e7a0 2023-10-26T01:33:17.371 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:17.365+0000 7f665bfff640 1 --2- 172.21.15.169:0/4217798142 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f666006ae40 0x7f666006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:17.371 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:17.365+0000 7f665bfff640 1 -- 172.21.15.169:0/4217798142 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f66680aa490 con 0x7f667414e7a0 2023-10-26T01:33:17.372 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:17.366+0000 7f66737fe640 1 --2- 172.21.15.169:0/4217798142 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f666006ae40 0x7f666006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:17.372 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:17.367+0000 7f66737fe640 1 --2- 172.21.15.169:0/4217798142 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f666006ae40 0x7f666006d300 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f666400ab40 tx=0x7f6664005f90 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:17.372 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:17.369+0000 7f665bfff640 1 -- 172.21.15.169:0/4217798142 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f66680754a0 con 0x7f667414e7a0 2023-10-26T01:33:17.561 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:17.559+0000 7f6679d16640 1 -- 172.21.15.169:0/4217798142 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f66740acdb0 con 0x7f667414e7a0 2023-10-26T01:33:17.562 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:17.560+0000 7f665bfff640 1 -- 172.21.15.169:0/4217798142 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v54) v1 ==== 93+0+10 (secure 0 0 0) 0x7f66740acdb0 con 0x7f667414e7a0 2023-10-26T01:33:17.563 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 1 2023-10-26T01:33:17.565 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:17.562+0000 7f6679d16640 1 -- 172.21.15.169:0/4217798142 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f666006ae40 msgr2=0x7f666006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:17.565 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:17.562+0000 7f6679d16640 1 --2- 172.21.15.169:0/4217798142 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f666006ae40 0x7f666006d300 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f666400ab40 tx=0x7f6664005f90 comp rx=0 tx=0).stop 2023-10-26T01:33:17.565 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:17.562+0000 7f6679d16640 1 -- 172.21.15.169:0/4217798142 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f667414e7a0 msgr2=0x7f66741ef130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:17.566 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:17.562+0000 7f6679d16640 1 --2- 172.21.15.169:0/4217798142 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f667414e7a0 0x7f66741ef130 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f666800c350 tx=0x7f66680017c0 comp rx=0 tx=0).stop 2023-10-26T01:33:17.566 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:17.562+0000 7f6679d16640 1 -- 172.21.15.169:0/4217798142 shutdown_connections 2023-10-26T01:33:17.566 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:17.562+0000 7f6679d16640 1 --2- 172.21.15.169:0/4217798142 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f666006ae40 0x7f666006d300 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:17.566 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:17.562+0000 7f6679d16640 1 --2- 172.21.15.169:0/4217798142 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f667414e7a0 0x7f66741ef130 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:17.567 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:17.562+0000 7f6679d16640 1 --2- 172.21.15.169:0/4217798142 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66741ef670 0x7f66741f3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:17.567 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:17.562+0000 7f6679d16640 1 --2- 172.21.15.169:0/4217798142 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f665c003f20 0x7f667414e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:17.567 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:17.562+0000 7f6679d16640 1 -- 172.21.15.169:0/4217798142 >> 172.21.15.169:0/4217798142 conn(0x7f6674069930 msgr2=0x7f66740aa250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:17.567 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:17.563+0000 7f6679d16640 1 -- 172.21.15.169:0/4217798142 shutdown_connections 2023-10-26T01:33:17.567 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:17.563+0000 7f6679d16640 1 -- 172.21.15.169:0/4217798142 wait complete. 2023-10-26T01:33:17.583 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-26T01:33:17.804 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:17.801+0000 7f7d095d7640 1 -- 172.21.15.169:0/4063491651 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7d040cdef0 msgr2=0x7f7d04155360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:17.805 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:17.801+0000 7f7d095d7640 1 --2- 172.21.15.169:0/4063491651 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7d040cdef0 0x7f7d04155360 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7cec005fd0 tx=0x7f7cec02bd90 comp rx=0 tx=0).stop 2023-10-26T01:33:17.805 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:17.802+0000 7f7d095d7640 1 -- 172.21.15.169:0/4063491651 shutdown_connections 2023-10-26T01:33:17.805 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:17.802+0000 7f7d095d7640 1 --2- 172.21.15.169:0/4063491651 >> v2:172.21.15.170:3300/0 conn(0x7f7d0415b4a0 0x7f7d0415b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:17.805 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:17.802+0000 7f7d095d7640 1 --2- 172.21.15.169:0/4063491651 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7d040cdef0 0x7f7d04155360 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:17.805 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:17.802+0000 7f7d095d7640 1 -- 172.21.15.169:0/4063491651 >> 172.21.15.169:0/4063491651 conn(0x7f7d04069930 msgr2=0x7f7d04069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:17.806 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:17.802+0000 7f7d095d7640 1 -- 172.21.15.169:0/4063491651 shutdown_connections 2023-10-26T01:33:17.806 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:17.802+0000 7f7d095d7640 1 -- 172.21.15.169:0/4063491651 wait complete. 2023-10-26T01:33:17.806 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:17.804+0000 7f7d095d7640 1 Processor -- start 2023-10-26T01:33:17.807 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:17.804+0000 7f7d095d7640 1 -- start start 2023-10-26T01:33:17.808 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:17.805+0000 7f7d095d7640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d0406ce00 0x7f7d0414ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:17.808 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:17.805+0000 7f7d095d7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d040cdef0 0x7f7d04160820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:17.809 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:17.805+0000 7f7d095d7640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7d0415b4a0 0x7f7d04070d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:17.809 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:17.805+0000 7f7d095d7640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d04094330 con 0x7f7d0406ce00 2023-10-26T01:33:17.809 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:17.805+0000 7f7d095d7640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d0414cfb0 con 0x7f7d040cdef0 2023-10-26T01:33:17.809 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:17.805+0000 7f7d095d7640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d04161120 con 0x7f7d0415b4a0 2023-10-26T01:33:17.809 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:17.805+0000 7f7d02ffd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d0406ce00 0x7f7d0414ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:17.809 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:17.806+0000 7f7d02ffd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d0406ce00 0x7f7d0414ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:47450/0 (socket says 172.21.15.169:47450) 2023-10-26T01:33:17.810 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:17.806+0000 7f7d02ffd640 1 -- 172.21.15.169:0/3774154470 learned_addr learned my addr 172.21.15.169:0/3774154470 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:33:17.810 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:17.806+0000 7f7cfa7fc640 1 --2- 172.21.15.169:0/3774154470 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d040cdef0 0x7f7d04160820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:17.810 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:17.806+0000 7f7d037fe640 1 --2- 172.21.15.169:0/3774154470 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7d0415b4a0 0x7f7d04070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:17.811 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:17.806+0000 7f7d02ffd640 1 -- 172.21.15.169:0/3774154470 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7d0415b4a0 msgr2=0x7f7d04070d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:17.811 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:17.806+0000 7f7d02ffd640 1 --2- 172.21.15.169:0/3774154470 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7d0415b4a0 0x7f7d04070d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:17.812 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:17.806+0000 7f7d02ffd640 1 -- 172.21.15.169:0/3774154470 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d040cdef0 msgr2=0x7f7d04160820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:17.812 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:17.806+0000 7f7d02ffd640 1 --2- 172.21.15.169:0/3774154470 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d040cdef0 0x7f7d04160820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:17.812 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:17.806+0000 7f7d02ffd640 1 -- 172.21.15.169:0/3774154470 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cec009560 con 0x7f7d0406ce00 2023-10-26T01:33:17.812 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:17.806+0000 7f7d02ffd640 1 --2- 172.21.15.169:0/3774154470 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d0406ce00 0x7f7d0414ca70 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f7cf400c450 tx=0x7f7cf400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:17.813 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:17.807+0000 7f7cfbfff640 1 -- 172.21.15.169:0/3774154470 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf40052d0 con 0x7f7d0406ce00 2023-10-26T01:33:17.813 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:17.807+0000 7f7cfbfff640 1 -- 172.21.15.169:0/3774154470 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7cf4005430 con 0x7f7d0406ce00 2023-10-26T01:33:17.813 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:17.807+0000 7f7cfbfff640 1 -- 172.21.15.169:0/3774154470 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf4012890 con 0x7f7d0406ce00 2023-10-26T01:33:17.813 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:17.807+0000 7f7d095d7640 1 -- 172.21.15.169:0/3774154470 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d04158cd0 con 0x7f7d0406ce00 2023-10-26T01:33:17.814 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:17.807+0000 7f7d095d7640 1 -- 172.21.15.169:0/3774154470 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d041616e0 con 0x7f7d0406ce00 2023-10-26T01:33:17.814 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:17.808+0000 7f7d095d7640 1 -- 172.21.15.169:0/3774154470 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d04094470 con 0x7f7d0406ce00 2023-10-26T01:33:17.815 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:17.809+0000 7f7cfbfff640 1 -- 172.21.15.169:0/3774154470 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f7cf40035f0 con 0x7f7d0406ce00 2023-10-26T01:33:17.815 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:17.810+0000 7f7cfbfff640 1 --2- 172.21.15.169:0/3774154470 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7ce406ae40 0x7f7ce406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:17.815 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:17.810+0000 7f7cfbfff640 1 -- 172.21.15.169:0/3774154470 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7cf4089200 con 0x7f7d0406ce00 2023-10-26T01:33:17.815 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:17.811+0000 7f7cfa7fc640 1 --2- 172.21.15.169:0/3774154470 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7ce406ae40 0x7f7ce406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:17.815 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:17.812+0000 7f7cfa7fc640 1 --2- 172.21.15.169:0/3774154470 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7ce406ae40 0x7f7ce406d300 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f7cec035000 tx=0x7f7cec004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:17.817 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:17.815+0000 7f7cfbfff640 1 -- 172.21.15.169:0/3774154470 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7d04094470 con 0x7f7d0406ce00 2023-10-26T01:33:18.010 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.007+0000 7f7d095d7640 1 -- 172.21.15.169:0/3774154470 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7d04094470 con 0x7f7d0406ce00 2023-10-26T01:33:18.011 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.008+0000 7f7cfbfff640 1 -- 172.21.15.169:0/3774154470 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v54) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7d04094470 con 0x7f7d0406ce00 2023-10-26T01:33:18.011 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 32 2023-10-26T01:33:18.014 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.011+0000 7f7d095d7640 1 -- 172.21.15.169:0/3774154470 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7ce406ae40 msgr2=0x7f7ce406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:18.015 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.011+0000 7f7d095d7640 1 --2- 172.21.15.169:0/3774154470 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7ce406ae40 0x7f7ce406d300 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f7cec035000 tx=0x7f7cec004730 comp rx=0 tx=0).stop 2023-10-26T01:33:18.015 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.011+0000 7f7d095d7640 1 -- 172.21.15.169:0/3774154470 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d0406ce00 msgr2=0x7f7d0414ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:18.015 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.011+0000 7f7d095d7640 1 --2- 172.21.15.169:0/3774154470 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d0406ce00 0x7f7d0414ca70 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f7cf400c450 tx=0x7f7cf400c910 comp rx=0 tx=0).stop 2023-10-26T01:33:18.015 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.011+0000 7f7d095d7640 1 -- 172.21.15.169:0/3774154470 shutdown_connections 2023-10-26T01:33:18.016 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.011+0000 7f7d095d7640 1 --2- 172.21.15.169:0/3774154470 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7ce406ae40 0x7f7ce406d300 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:18.016 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.011+0000 7f7d095d7640 1 --2- 172.21.15.169:0/3774154470 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7d0415b4a0 0x7f7d04070d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:18.016 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.011+0000 7f7d095d7640 1 --2- 172.21.15.169:0/3774154470 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d040cdef0 0x7f7d04160820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:18.017 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.011+0000 7f7d095d7640 1 --2- 172.21.15.169:0/3774154470 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d0406ce00 0x7f7d0414ca70 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:18.017 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.011+0000 7f7d095d7640 1 -- 172.21.15.169:0/3774154470 >> 172.21.15.169:0/3774154470 conn(0x7f7d04069930 msgr2=0x7f7d0406ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:18.017 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.011+0000 7f7d095d7640 1 -- 172.21.15.169:0/3774154470 shutdown_connections 2023-10-26T01:33:18.017 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.011+0000 7f7d095d7640 1 -- 172.21.15.169:0/3774154470 wait complete. 2023-10-26T01:33:18.033 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-26T01:33:18.263 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.260+0000 7f04d8e7c640 1 -- 172.21.15.169:0/1280790599 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04bc004d00 msgr2=0x7f04bc005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:18.263 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.260+0000 7f04d8e7c640 1 --2- 172.21.15.169:0/1280790599 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04bc004d00 0x7f04bc005180 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f04cc006f00 tx=0x7f04bc0057f0 comp rx=0 tx=0).stop 2023-10-26T01:33:18.264 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.260+0000 7f04d2d9d640 1 -- 172.21.15.169:0/1280790599 reap_dead start 2023-10-26T01:33:18.264 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.260+0000 7f04d8e7c640 1 -- 172.21.15.169:0/1280790599 shutdown_connections 2023-10-26T01:33:18.264 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.260+0000 7f04d8e7c640 1 -- 172.21.15.169:0/1280790599 >> 172.21.15.169:0/1280790599 conn(0x7f04d4069930 msgr2=0x7f04d4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:18.264 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.260+0000 7f04d8e7c640 1 -- 172.21.15.169:0/1280790599 shutdown_connections 2023-10-26T01:33:18.264 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.260+0000 7f04d8e7c640 1 -- 172.21.15.169:0/1280790599 wait complete. 2023-10-26T01:33:18.267 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.262+0000 7f04d8e7c640 1 Processor -- start 2023-10-26T01:33:18.267 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.262+0000 7f04d8e7c640 1 -- start start 2023-10-26T01:33:18.267 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.263+0000 7f04d8e7c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04bc004d00 0x7f04d41558a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:18.267 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.263+0000 7f04d8e7c640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f04d41ef720 0x7f04d4070d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:18.268 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.263+0000 7f04d8e7c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f04d41efba0 0x7f04d4071290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:18.268 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.263+0000 7f04d8e7c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04d4155de0 con 0x7f04bc004d00 2023-10-26T01:33:18.268 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.263+0000 7f04d8e7c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04d4157e20 con 0x7f04d41efba0 2023-10-26T01:33:18.268 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.263+0000 7f04d8e7c640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04d4158100 con 0x7f04d41ef720 2023-10-26T01:33:18.269 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.263+0000 7f04d259c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04bc004d00 0x7f04d41558a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:18.269 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.263+0000 7f04d259c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04bc004d00 0x7f04d41558a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:47468/0 (socket says 172.21.15.169:47468) 2023-10-26T01:33:18.269 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.263+0000 7f04d259c640 1 -- 172.21.15.169:0/1954437890 learned_addr learned my addr 172.21.15.169:0/1954437890 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:33:18.269 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.263+0000 7f04d259c640 1 -- 172.21.15.169:0/1954437890 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f04d41ef720 msgr2=0x7f04d4070d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:18.269 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.263+0000 7f04d259c640 1 --2- 172.21.15.169:0/1954437890 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f04d41ef720 0x7f04d4070d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:18.270 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.263+0000 7f04d259c640 1 -- 172.21.15.169:0/1954437890 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f04d41efba0 msgr2=0x7f04d4071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:18.270 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.263+0000 7f04d259c640 1 --2- 172.21.15.169:0/1954437890 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f04d41efba0 0x7f04d4071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:18.270 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.263+0000 7f04d259c640 1 -- 172.21.15.169:0/1954437890 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04d41583e0 con 0x7f04bc004d00 2023-10-26T01:33:18.271 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.264+0000 7f04d259c640 1 --2- 172.21.15.169:0/1954437890 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04bc004d00 0x7f04d41558a0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f04c400fc80 tx=0x7f04c400d6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:18.271 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.264+0000 7f04c2ffd640 1 -- 172.21.15.169:0/1954437890 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c4004d20 con 0x7f04bc004d00 2023-10-26T01:33:18.271 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.264+0000 7f04c2ffd640 1 -- 172.21.15.169:0/1954437890 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f04c400fe90 con 0x7f04bc004d00 2023-10-26T01:33:18.271 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.264+0000 7f04c2ffd640 1 -- 172.21.15.169:0/1954437890 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c4005260 con 0x7f04bc004d00 2023-10-26T01:33:18.271 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.264+0000 7f04d8e7c640 1 -- 172.21.15.169:0/1954437890 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04d41586c0 con 0x7f04bc004d00 2023-10-26T01:33:18.272 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.264+0000 7f04d8e7c640 1 -- 172.21.15.169:0/1954437890 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04d40719b0 con 0x7f04bc004d00 2023-10-26T01:33:18.273 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.265+0000 7f04d8e7c640 1 -- 172.21.15.169:0/1954437890 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04d41e2ed0 con 0x7f04bc004d00 2023-10-26T01:33:18.273 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.266+0000 7f04c2ffd640 1 -- 172.21.15.169:0/1954437890 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f04c40053c0 con 0x7f04bc004d00 2023-10-26T01:33:18.273 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.268+0000 7f04c2ffd640 1 --2- 172.21.15.169:0/1954437890 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f04b406ae40 0x7f04b406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:18.274 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.268+0000 7f04c2ffd640 1 -- 172.21.15.169:0/1954437890 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f04c408a5f0 con 0x7f04bc004d00 2023-10-26T01:33:18.274 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.268+0000 7f04d1d9b640 1 --2- 172.21.15.169:0/1954437890 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f04b406ae40 0x7f04b406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:18.274 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.269+0000 7f04d1d9b640 1 --2- 172.21.15.169:0/1954437890 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f04b406ae40 0x7f04b406d300 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f04d415be50 tx=0x7f04c8006900 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:18.275 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.273+0000 7f04c2ffd640 1 -- 172.21.15.169:0/1954437890 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f04c4055580 con 0x7f04bc004d00 2023-10-26T01:33:18.464 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.462+0000 7f04d8e7c640 1 -- 172.21.15.169:0/1954437890 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f04d404db10 con 0x7f04bc004d00 2023-10-26T01:33:18.465 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.463+0000 7f04c2ffd640 1 -- 172.21.15.169:0/1954437890 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v54) v1 ==== 100+0+11 (secure 0 0 0) 0x7f04c4020020 con 0x7f04bc004d00 2023-10-26T01:33:18.465 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 64 2023-10-26T01:33:18.468 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.465+0000 7f04d8e7c640 1 -- 172.21.15.169:0/1954437890 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f04b406ae40 msgr2=0x7f04b406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:18.468 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.465+0000 7f04d8e7c640 1 --2- 172.21.15.169:0/1954437890 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f04b406ae40 0x7f04b406d300 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f04d415be50 tx=0x7f04c8006900 comp rx=0 tx=0).stop 2023-10-26T01:33:18.468 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.465+0000 7f04d8e7c640 1 -- 172.21.15.169:0/1954437890 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04bc004d00 msgr2=0x7f04d41558a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:18.469 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.465+0000 7f04d8e7c640 1 --2- 172.21.15.169:0/1954437890 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04bc004d00 0x7f04d41558a0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f04c400fc80 tx=0x7f04c400d6b0 comp rx=0 tx=0).stop 2023-10-26T01:33:18.469 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.465+0000 7f04d8e7c640 1 -- 172.21.15.169:0/1954437890 shutdown_connections 2023-10-26T01:33:18.469 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.465+0000 7f04d8e7c640 1 --2- 172.21.15.169:0/1954437890 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f04b406ae40 0x7f04b406d300 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:18.469 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.465+0000 7f04d8e7c640 1 --2- 172.21.15.169:0/1954437890 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f04d41efba0 0x7f04d4071290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:18.469 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.465+0000 7f04d8e7c640 1 --2- 172.21.15.169:0/1954437890 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f04d41ef720 0x7f04d4070d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:18.469 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.465+0000 7f04d8e7c640 1 --2- 172.21.15.169:0/1954437890 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04bc004d00 0x7f04d41558a0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:18.470 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.465+0000 7f04d8e7c640 1 -- 172.21.15.169:0/1954437890 >> 172.21.15.169:0/1954437890 conn(0x7f04d4069930 msgr2=0x7f04d415c650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:18.470 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.466+0000 7f04d8e7c640 1 -- 172.21.15.169:0/1954437890 shutdown_connections 2023-10-26T01:33:18.470 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.466+0000 7f04d8e7c640 1 -- 172.21.15.169:0/1954437890 wait complete. 2023-10-26T01:33:18.488 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-26T01:33:18.701 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.698+0000 7f851cfc0640 1 -- 172.21.15.169:0/2644843564 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8504004d00 msgr2=0x7f8504005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:18.701 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.698+0000 7f851cfc0640 1 --2- 172.21.15.169:0/2644843564 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8504004d00 0x7f8504005180 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f8508006f00 tx=0x7f85040057f0 comp rx=0 tx=0).stop 2023-10-26T01:33:18.701 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.698+0000 7f8516d9d640 1 -- 172.21.15.169:0/2644843564 reap_dead start 2023-10-26T01:33:18.702 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.698+0000 7f851cfc0640 1 -- 172.21.15.169:0/2644843564 shutdown_connections 2023-10-26T01:33:18.702 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.698+0000 7f851cfc0640 1 -- 172.21.15.169:0/2644843564 >> 172.21.15.169:0/2644843564 conn(0x7f851806a620 msgr2=0x7f851806aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:18.702 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.698+0000 7f851cfc0640 1 -- 172.21.15.169:0/2644843564 shutdown_connections 2023-10-26T01:33:18.702 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.698+0000 7f851cfc0640 1 -- 172.21.15.169:0/2644843564 wait complete. 2023-10-26T01:33:18.703 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.700+0000 7f851cfc0640 1 Processor -- start 2023-10-26T01:33:18.703 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.700+0000 7f851cfc0640 1 -- start start 2023-10-26T01:33:18.705 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.701+0000 7f851cfc0640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8504004d00 0x7f85180afdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:18.705 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.701+0000 7f851cfc0640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8518159030 0x7f85181594b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:18.705 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.701+0000 7f851cfc0640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f85180c7730 0x7f85180c7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:18.705 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.701+0000 7f851cfc0640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85181599f0 con 0x7f8504004d00 2023-10-26T01:33:18.705 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.701+0000 7f851cfc0640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85180c3710 con 0x7f8518159030 2023-10-26T01:33:18.706 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.701+0000 7f851cfc0640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85180c39b0 con 0x7f85180c7730 2023-10-26T01:33:18.706 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.701+0000 7f851659c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8504004d00 0x7f85180afdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:18.706 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.701+0000 7f851659c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8504004d00 0x7f85180afdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:47488/0 (socket says 172.21.15.169:47488) 2023-10-26T01:33:18.706 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.701+0000 7f851659c640 1 -- 172.21.15.169:0/3764980087 learned_addr learned my addr 172.21.15.169:0/3764980087 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:33:18.707 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.702+0000 7f851659c640 1 -- 172.21.15.169:0/3764980087 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f85180c7730 msgr2=0x7f85180c7bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:18.707 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.702+0000 7f851659c640 1 --2- 172.21.15.169:0/3764980087 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f85180c7730 0x7f85180c7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:18.707 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.702+0000 7f851659c640 1 -- 172.21.15.169:0/3764980087 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8518159030 msgr2=0x7f85181594b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:18.707 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.702+0000 7f851659c640 1 --2- 172.21.15.169:0/3764980087 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8518159030 0x7f85181594b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:18.707 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.702+0000 7f851659c640 1 -- 172.21.15.169:0/3764980087 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85180c3c90 con 0x7f8504004d00 2023-10-26T01:33:18.708 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.702+0000 7f851659c640 1 --2- 172.21.15.169:0/3764980087 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8504004d00 0x7f85180afdf0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f8510010bf0 tx=0x7f851000e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:18.708 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.702+0000 7f850effd640 1 -- 172.21.15.169:0/3764980087 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8510003b70 con 0x7f8504004d00 2023-10-26T01:33:18.708 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.702+0000 7f850effd640 1 -- 172.21.15.169:0/3764980087 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8510015070 con 0x7f8504004d00 2023-10-26T01:33:18.708 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.702+0000 7f850effd640 1 -- 172.21.15.169:0/3764980087 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8510004050 con 0x7f8504004d00 2023-10-26T01:33:18.708 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.702+0000 7f851cfc0640 1 -- 172.21.15.169:0/3764980087 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85180c3f70 con 0x7f8504004d00 2023-10-26T01:33:18.708 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.702+0000 7f851cfc0640 1 -- 172.21.15.169:0/3764980087 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85180c80f0 con 0x7f8504004d00 2023-10-26T01:33:18.709 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.704+0000 7f851cfc0640 1 -- 172.21.15.169:0/3764980087 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8518066af0 con 0x7f8504004d00 2023-10-26T01:33:18.709 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.705+0000 7f850effd640 1 -- 172.21.15.169:0/3764980087 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f85100041b0 con 0x7f8504004d00 2023-10-26T01:33:18.710 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.706+0000 7f850effd640 1 --2- 172.21.15.169:0/3764980087 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f84f806ae40 0x7f84f806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:18.710 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.706+0000 7f850effd640 1 -- 172.21.15.169:0/3764980087 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8510017070 con 0x7f8504004d00 2023-10-26T01:33:18.710 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.707+0000 7f8515d9b640 1 --2- 172.21.15.169:0/3764980087 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f84f806ae40 0x7f84f806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:18.711 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.708+0000 7f8515d9b640 1 --2- 172.21.15.169:0/3764980087 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f84f806ae40 0x7f84f806d300 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f85181411b0 tx=0x7f8500006d50 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:18.713 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.710+0000 7f850effd640 1 -- 172.21.15.169:0/3764980087 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8510055950 con 0x7f8504004d00 2023-10-26T01:33:18.900 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.898+0000 7f851cfc0640 1 -- 172.21.15.169:0/3764980087 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f8518072060 con 0x7f8504004d00 2023-10-26T01:33:18.901 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.899+0000 7f850effd640 1 -- 172.21.15.169:0/3764980087 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v54) v1 ==== 80+0+0 (secure 0 0 0) 0x7f8510021080 con 0x7f8504004d00 2023-10-26T01:33:18.901 INFO:teuthology.orchestra.run.smithi169.stderr:listed 0 entries 2023-10-26T01:33:18.904 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.901+0000 7f851cfc0640 1 -- 172.21.15.169:0/3764980087 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f84f806ae40 msgr2=0x7f84f806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:18.905 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.901+0000 7f851cfc0640 1 --2- 172.21.15.169:0/3764980087 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f84f806ae40 0x7f84f806d300 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f85181411b0 tx=0x7f8500006d50 comp rx=0 tx=0).stop 2023-10-26T01:33:18.905 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.901+0000 7f851cfc0640 1 -- 172.21.15.169:0/3764980087 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8504004d00 msgr2=0x7f85180afdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:18.905 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.901+0000 7f851cfc0640 1 --2- 172.21.15.169:0/3764980087 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8504004d00 0x7f85180afdf0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f8510010bf0 tx=0x7f851000e720 comp rx=0 tx=0).stop 2023-10-26T01:33:18.905 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.901+0000 7f851cfc0640 1 -- 172.21.15.169:0/3764980087 shutdown_connections 2023-10-26T01:33:18.905 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.901+0000 7f851cfc0640 1 --2- 172.21.15.169:0/3764980087 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f84f806ae40 0x7f84f806d300 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:18.906 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.901+0000 7f851cfc0640 1 --2- 172.21.15.169:0/3764980087 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f85180c7730 0x7f85180c7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:18.906 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.902+0000 7f851cfc0640 1 --2- 172.21.15.169:0/3764980087 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8518159030 0x7f85181594b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:18.906 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.902+0000 7f851cfc0640 1 --2- 172.21.15.169:0/3764980087 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8504004d00 0x7f85180afdf0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:18.906 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.902+0000 7f851cfc0640 1 -- 172.21.15.169:0/3764980087 >> 172.21.15.169:0/3764980087 conn(0x7f851806a620 msgr2=0x7f8518141b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:18.906 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.902+0000 7f851cfc0640 1 -- 172.21.15.169:0/3764980087 shutdown_connections 2023-10-26T01:33:18.907 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:18.902+0000 7f851cfc0640 1 -- 172.21.15.169:0/3764980087 wait complete. 2023-10-26T01:33:18.921 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-26T01:33:18.922 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:33:18.922 DEBUG:teuthology.orchestra.run.smithi169:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-26T01:33:19.075 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi169.front.sepia.ceph.com 2023-10-26T01:33:19.075 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:33:19.076 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-26T01:33:19.124 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-26T01:33:19.125 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:33:19.171 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-26T01:33:19.172 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:33:19.264 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:33:19.487 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:19.484+0000 7f3b74955640 1 -- 172.21.15.169:0/2840971786 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b580039a0 msgr2=0x7f3b58003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:19.487 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:19.484+0000 7f3b74955640 1 --2- 172.21.15.169:0/2840971786 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b580039a0 0x7f3b58003dc0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f3b6000bb20 tx=0x7f3b580057f0 comp rx=0 tx=0).stop 2023-10-26T01:33:19.487 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:19.484+0000 7f3b6ed9d640 1 -- 172.21.15.169:0/2840971786 reap_dead start 2023-10-26T01:33:19.488 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:19.484+0000 7f3b74955640 1 -- 172.21.15.169:0/2840971786 shutdown_connections 2023-10-26T01:33:19.488 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:19.484+0000 7f3b74955640 1 -- 172.21.15.169:0/2840971786 >> 172.21.15.169:0/2840971786 conn(0x7f3b70069930 msgr2=0x7f3b70069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:19.488 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:19.484+0000 7f3b74955640 1 -- 172.21.15.169:0/2840971786 shutdown_connections 2023-10-26T01:33:19.488 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:19.484+0000 7f3b74955640 1 -- 172.21.15.169:0/2840971786 wait complete. 2023-10-26T01:33:19.489 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:19.486+0000 7f3b74955640 1 Processor -- start 2023-10-26T01:33:19.489 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:19.486+0000 7f3b74955640 1 -- start start 2023-10-26T01:33:19.490 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:19.487+0000 7f3b74955640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b701527b0 0x7f3b70070aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:19.490 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:19.487+0000 7f3b74955640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3b70070fe0 0x7f3b7015b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:19.491 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:19.487+0000 7f3b74955640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3b7015b590 0x7f3b700b02e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:19.491 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:19.487+0000 7f3b74955640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b7015ba10 con 0x7f3b701527b0 2023-10-26T01:33:19.491 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:19.487+0000 7f3b74955640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b701f1f90 con 0x7f3b7015b590 2023-10-26T01:33:19.491 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:19.487+0000 7f3b74955640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b701f2270 con 0x7f3b70070fe0 2023-10-26T01:33:19.492 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:19.487+0000 7f3b6e59c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b701527b0 0x7f3b70070aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:19.492 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:19.487+0000 7f3b6e59c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b701527b0 0x7f3b70070aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:47496/0 (socket says 172.21.15.169:47496) 2023-10-26T01:33:19.492 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:19.487+0000 7f3b6e59c640 1 -- 172.21.15.169:0/898274669 learned_addr learned my addr 172.21.15.169:0/898274669 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:33:19.492 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:19.488+0000 7f3b6e59c640 1 -- 172.21.15.169:0/898274669 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3b70070fe0 msgr2=0x7f3b7015b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:19.492 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:19.488+0000 7f3b6e59c640 1 --2- 172.21.15.169:0/898274669 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3b70070fe0 0x7f3b7015b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:19.493 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:19.488+0000 7f3b6e59c640 1 -- 172.21.15.169:0/898274669 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3b7015b590 msgr2=0x7f3b700b02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:19.493 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:19.488+0000 7f3b6e59c640 1 --2- 172.21.15.169:0/898274669 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3b7015b590 0x7f3b700b02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:19.493 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:19.488+0000 7f3b6e59c640 1 -- 172.21.15.169:0/898274669 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b701f2550 con 0x7f3b701527b0 2023-10-26T01:33:19.493 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:19.488+0000 7f3b6e59c640 1 --2- 172.21.15.169:0/898274669 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b701527b0 0x7f3b70070aa0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f3b60038040 tx=0x7f3b600367d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:19.494 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:19.488+0000 7f3b5effd640 1 -- 172.21.15.169:0/898274669 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b60036dd0 con 0x7f3b701527b0 2023-10-26T01:33:19.495 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:19.488+0000 7f3b5effd640 1 -- 172.21.15.169:0/898274669 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3b60004120 con 0x7f3b701527b0 2023-10-26T01:33:19.495 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:19.488+0000 7f3b5effd640 1 -- 172.21.15.169:0/898274669 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b600035f0 con 0x7f3b701527b0 2023-10-26T01:33:19.495 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:19.488+0000 7f3b74955640 1 -- 172.21.15.169:0/898274669 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b701f2830 con 0x7f3b701527b0 2023-10-26T01:33:19.495 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:19.488+0000 7f3b74955640 1 -- 172.21.15.169:0/898274669 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b701f1b60 con 0x7f3b701527b0 2023-10-26T01:33:19.496 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:19.489+0000 7f3b74955640 1 -- 172.21.15.169:0/898274669 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b7004db10 con 0x7f3b701527b0 2023-10-26T01:33:19.496 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:19.490+0000 7f3b5effd640 1 -- 172.21.15.169:0/898274669 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f3b60003750 con 0x7f3b701527b0 2023-10-26T01:33:19.496 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:19.491+0000 7f3b5effd640 1 --2- 172.21.15.169:0/898274669 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3b5006ae70 0x7f3b5006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:19.496 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:19.491+0000 7f3b5effd640 1 -- 172.21.15.169:0/898274669 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3b600ab3c0 con 0x7f3b701527b0 2023-10-26T01:33:19.496 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:19.492+0000 7f3b6dd9b640 1 --2- 172.21.15.169:0/898274669 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3b5006ae70 0x7f3b5006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:19.497 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:19.493+0000 7f3b6dd9b640 1 --2- 172.21.15.169:0/898274669 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3b5006ae70 0x7f3b5006d330 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f3b70153a20 tx=0x7f3b64000f70 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:19.498 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:19.496+0000 7f3b5effd640 1 -- 172.21.15.169:0/898274669 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3b600763d0 con 0x7f3b701527b0 2023-10-26T01:33:19.688 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:19.685+0000 7f3b74955640 1 -- 172.21.15.169:0/898274669 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3b70159870 con 0x7f3b701527b0 2023-10-26T01:33:19.690 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:19.687+0000 7f3b5effd640 1 -- 172.21.15.169:0/898274669 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+16561 (secure 0 0 0) 0x7f3b60005ab0 con 0x7f3b701527b0 2023-10-26T01:33:19.691 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:33:19.691 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":54,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:33:10.190714+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-26T01:32:36.288217+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-26T01:32:37.059775+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:33:19.693 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:19.690+0000 7f3b74955640 1 -- 172.21.15.169:0/898274669 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3b5006ae70 msgr2=0x7f3b5006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:19.693 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:19.690+0000 7f3b74955640 1 --2- 172.21.15.169:0/898274669 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3b5006ae70 0x7f3b5006d330 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f3b70153a20 tx=0x7f3b64000f70 comp rx=0 tx=0).stop 2023-10-26T01:33:19.693 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:19.690+0000 7f3b74955640 1 -- 172.21.15.169:0/898274669 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b701527b0 msgr2=0x7f3b70070aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:19.693 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:19.690+0000 7f3b74955640 1 --2- 172.21.15.169:0/898274669 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b701527b0 0x7f3b70070aa0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f3b60038040 tx=0x7f3b600367d0 comp rx=0 tx=0).stop 2023-10-26T01:33:19.693 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:19.690+0000 7f3b74955640 1 -- 172.21.15.169:0/898274669 shutdown_connections 2023-10-26T01:33:19.694 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:19.690+0000 7f3b74955640 1 --2- 172.21.15.169:0/898274669 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3b5006ae70 0x7f3b5006d330 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:19.694 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:19.690+0000 7f3b74955640 1 --2- 172.21.15.169:0/898274669 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3b7015b590 0x7f3b700b02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:19.694 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:19.690+0000 7f3b74955640 1 --2- 172.21.15.169:0/898274669 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3b70070fe0 0x7f3b7015b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:19.694 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:19.690+0000 7f3b74955640 1 --2- 172.21.15.169:0/898274669 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b701527b0 0x7f3b70070aa0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:19.695 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:19.690+0000 7f3b74955640 1 -- 172.21.15.169:0/898274669 >> 172.21.15.169:0/898274669 conn(0x7f3b70069930 msgr2=0x7f3b700ae010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:19.695 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:19.690+0000 7f3b74955640 1 -- 172.21.15.169:0/898274669 shutdown_connections 2023-10-26T01:33:19.695 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:19.690+0000 7f3b74955640 1 -- 172.21.15.169:0/898274669 wait complete. 2023-10-26T01:33:19.710 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T01:30:32.200808+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-26T01:32:36.288217+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-10-26T01:32:37.059775+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T01:33:19.711 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T01:33:19.920 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:19.917+0000 7fd491b0a640 1 -- 172.21.15.169:0/2430242534 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd47c004020 msgr2=0x7fd47c0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:19.920 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:19.917+0000 7fd491b0a640 1 --2- 172.21.15.169:0/2430242534 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd47c004020 0x7fd47c0047c0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fd47800bb20 tx=0x7fd47c0057f0 comp rx=0 tx=0).stop 2023-10-26T01:33:19.921 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:19.917+0000 7fd491309640 1 -- 172.21.15.169:0/2430242534 reap_dead start 2023-10-26T01:33:19.921 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:19.917+0000 7fd491b0a640 1 -- 172.21.15.169:0/2430242534 shutdown_connections 2023-10-26T01:33:19.921 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:19.917+0000 7fd491b0a640 1 -- 172.21.15.169:0/2430242534 >> 172.21.15.169:0/2430242534 conn(0x7fd48c069510 msgr2=0x7fd48c069920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:19.921 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:19.917+0000 7fd491b0a640 1 -- 172.21.15.169:0/2430242534 shutdown_connections 2023-10-26T01:33:19.921 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:19.917+0000 7fd491b0a640 1 -- 172.21.15.169:0/2430242534 wait complete. 2023-10-26T01:33:19.922 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:19.919+0000 7fd491b0a640 1 Processor -- start 2023-10-26T01:33:19.923 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:19.919+0000 7fd491b0a640 1 -- start start 2023-10-26T01:33:19.923 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:19.920+0000 7fd491b0a640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd47c004020 0x7fd48c151fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:19.923 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:19.920+0000 7fd491b0a640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd48c0971c0 0x7fd48c150a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:19.923 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:19.920+0000 7fd491b0a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd48c150f70 0x7fd48c1ef260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:19.924 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:19.920+0000 7fd491b0a640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd48c0affb0 con 0x7fd47c004020 2023-10-26T01:33:19.924 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:19.920+0000 7fd491b0a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd48c1514e0 con 0x7fd48c150f70 2023-10-26T01:33:19.924 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:19.920+0000 7fd491b0a640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd48c1517c0 con 0x7fd48c0971c0 2023-10-26T01:33:19.924 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:19.920+0000 7fd490b08640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd47c004020 0x7fd48c151fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:19.924 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:19.920+0000 7fd490b08640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd47c004020 0x7fd48c151fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:47506/0 (socket says 172.21.15.169:47506) 2023-10-26T01:33:19.924 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:19.920+0000 7fd490b08640 1 -- 172.21.15.169:0/3395309434 learned_addr learned my addr 172.21.15.169:0/3395309434 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:33:19.926 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:19.920+0000 7fd490b08640 1 -- 172.21.15.169:0/3395309434 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd48c0971c0 msgr2=0x7fd48c150a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:19.926 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:19.920+0000 7fd490b08640 1 --2- 172.21.15.169:0/3395309434 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd48c0971c0 0x7fd48c150a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:19.926 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:19.920+0000 7fd490b08640 1 -- 172.21.15.169:0/3395309434 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd48c150f70 msgr2=0x7fd48c1ef260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:19.926 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:19.920+0000 7fd490b08640 1 --2- 172.21.15.169:0/3395309434 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd48c150f70 0x7fd48c1ef260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:19.927 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:19.921+0000 7fd490b08640 1 -- 172.21.15.169:0/3395309434 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd48c1594c0 con 0x7fd47c004020 2023-10-26T01:33:19.927 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:19.921+0000 7fd490b08640 1 --2- 172.21.15.169:0/3395309434 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd47c004020 0x7fd48c151fc0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fd48000c700 tx=0x7fd48000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:19.927 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:19.921+0000 7fd4897fa640 1 -- 172.21.15.169:0/3395309434 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd480005480 con 0x7fd47c004020 2023-10-26T01:33:19.927 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:19.921+0000 7fd4897fa640 1 -- 172.21.15.169:0/3395309434 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd48000a640 con 0x7fd47c004020 2023-10-26T01:33:19.928 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:19.921+0000 7fd4897fa640 1 -- 172.21.15.169:0/3395309434 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd48000aa50 con 0x7fd47c004020 2023-10-26T01:33:19.928 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:19.921+0000 7fd491b0a640 1 -- 172.21.15.169:0/3395309434 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd48c151aa0 con 0x7fd47c004020 2023-10-26T01:33:19.928 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:19.921+0000 7fd491b0a640 1 -- 172.21.15.169:0/3395309434 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd48c0b0250 con 0x7fd47c004020 2023-10-26T01:33:19.928 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:19.922+0000 7fd491b0a640 1 -- 172.21.15.169:0/3395309434 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd48c151ce0 con 0x7fd47c004020 2023-10-26T01:33:19.928 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:19.923+0000 7fd4897fa640 1 -- 172.21.15.169:0/3395309434 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fd48000abb0 con 0x7fd47c004020 2023-10-26T01:33:19.929 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:19.924+0000 7fd4897fa640 1 --2- 172.21.15.169:0/3395309434 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd47006ae40 0x7fd47006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:19.929 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:19.924+0000 7fd4897fa640 1 -- 172.21.15.169:0/3395309434 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd480089190 con 0x7fd47c004020 2023-10-26T01:33:19.930 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:19.925+0000 7fd48bfff640 1 --2- 172.21.15.169:0/3395309434 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd47006ae40 0x7fd47006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:19.930 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:19.926+0000 7fd48bfff640 1 --2- 172.21.15.169:0/3395309434 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd47006ae40 0x7fd47006d300 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd478037000 tx=0x7fd47802e040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:19.931 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:19.928+0000 7fd4897fa640 1 -- 172.21.15.169:0/3395309434 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd48c151ce0 con 0x7fd47c004020 2023-10-26T01:33:20.121 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.118+0000 7fd491b0a640 1 -- 172.21.15.169:0/3395309434 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd48c151ce0 con 0x7fd47c004020 2023-10-26T01:33:20.121 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.119+0000 7fd4897fa640 1 -- 172.21.15.169:0/3395309434 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v54) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd480017020 con 0x7fd47c004020 2023-10-26T01:33:20.122 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 1 2023-10-26T01:33:20.124 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.121+0000 7fd491b0a640 1 -- 172.21.15.169:0/3395309434 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd47006ae40 msgr2=0x7fd47006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:20.124 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.121+0000 7fd491b0a640 1 --2- 172.21.15.169:0/3395309434 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd47006ae40 0x7fd47006d300 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd478037000 tx=0x7fd47802e040 comp rx=0 tx=0).stop 2023-10-26T01:33:20.125 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.121+0000 7fd491b0a640 1 -- 172.21.15.169:0/3395309434 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd47c004020 msgr2=0x7fd48c151fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:20.125 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.121+0000 7fd491b0a640 1 --2- 172.21.15.169:0/3395309434 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd47c004020 0x7fd48c151fc0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fd48000c700 tx=0x7fd48000cbc0 comp rx=0 tx=0).stop 2023-10-26T01:33:20.125 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.121+0000 7fd491b0a640 1 -- 172.21.15.169:0/3395309434 shutdown_connections 2023-10-26T01:33:20.125 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.121+0000 7fd491b0a640 1 --2- 172.21.15.169:0/3395309434 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd47006ae40 0x7fd47006d300 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:20.125 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.121+0000 7fd491b0a640 1 --2- 172.21.15.169:0/3395309434 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd48c150f70 0x7fd48c1ef260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:20.126 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.121+0000 7fd491b0a640 1 --2- 172.21.15.169:0/3395309434 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd48c0971c0 0x7fd48c150a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:20.126 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.121+0000 7fd491b0a640 1 --2- 172.21.15.169:0/3395309434 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd47c004020 0x7fd48c151fc0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:20.126 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.121+0000 7fd491b0a640 1 -- 172.21.15.169:0/3395309434 >> 172.21.15.169:0/3395309434 conn(0x7fd48c069510 msgr2=0x7fd48c14de90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:20.126 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.121+0000 7fd491b0a640 1 -- 172.21.15.169:0/3395309434 shutdown_connections 2023-10-26T01:33:20.126 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.121+0000 7fd491b0a640 1 -- 172.21.15.169:0/3395309434 wait complete. 2023-10-26T01:33:20.141 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-26T01:33:20.357 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.354+0000 7f5f25ba9640 1 -- 172.21.15.169:0/2834096953 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f0c004d00 msgr2=0x7f5f0c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:20.357 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.354+0000 7f5f25ba9640 1 --2- 172.21.15.169:0/2834096953 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f0c004d00 0x7f5f0c005180 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f5f18005fd0 tx=0x7f5f0c0057f0 comp rx=0 tx=0).stop 2023-10-26T01:33:20.358 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.354+0000 7f5f1ffff640 1 -- 172.21.15.169:0/2834096953 reap_dead start 2023-10-26T01:33:20.358 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.354+0000 7f5f25ba9640 1 -- 172.21.15.169:0/2834096953 shutdown_connections 2023-10-26T01:33:20.358 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.354+0000 7f5f25ba9640 1 -- 172.21.15.169:0/2834096953 >> 172.21.15.169:0/2834096953 conn(0x7f5f200af570 msgr2=0x7f5f200af980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:20.358 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.354+0000 7f5f25ba9640 1 -- 172.21.15.169:0/2834096953 shutdown_connections 2023-10-26T01:33:20.358 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.354+0000 7f5f25ba9640 1 -- 172.21.15.169:0/2834096953 wait complete. 2023-10-26T01:33:20.359 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.356+0000 7f5f25ba9640 1 Processor -- start 2023-10-26T01:33:20.359 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.356+0000 7f5f25ba9640 1 -- start start 2023-10-26T01:33:20.361 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.357+0000 7f5f25ba9640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f0c004d00 0x7f5f20163ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:20.361 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.357+0000 7f5f25ba9640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5f20160e80 0x7f5f2014bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:20.362 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.357+0000 7f5f25ba9640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5f200ad860 0x7f5f200add10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:20.362 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.357+0000 7f5f25ba9640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f200b19e0 con 0x7f5f0c004d00 2023-10-26T01:33:20.362 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.357+0000 7f5f25ba9640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f200ae250 con 0x7f5f200ad860 2023-10-26T01:33:20.362 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.357+0000 7f5f25ba9640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f201613f0 con 0x7f5f20160e80 2023-10-26T01:33:20.362 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.357+0000 7f5f1f7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f0c004d00 0x7f5f20163ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:20.363 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.357+0000 7f5f1f7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f0c004d00 0x7f5f20163ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:47520/0 (socket says 172.21.15.169:47520) 2023-10-26T01:33:20.363 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.357+0000 7f5f1f7fe640 1 -- 172.21.15.169:0/2081871825 learned_addr learned my addr 172.21.15.169:0/2081871825 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:33:20.363 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.358+0000 7f5f1ffff640 1 --2- 172.21.15.169:0/2081871825 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5f200ad860 0x7f5f200add10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:20.363 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.358+0000 7f5f16ffd640 1 --2- 172.21.15.169:0/2081871825 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5f20160e80 0x7f5f2014bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:20.363 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.358+0000 7f5f1f7fe640 1 -- 172.21.15.169:0/2081871825 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5f20160e80 msgr2=0x7f5f2014bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:20.364 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.358+0000 7f5f1f7fe640 1 --2- 172.21.15.169:0/2081871825 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5f20160e80 0x7f5f2014bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:20.364 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.358+0000 7f5f1f7fe640 1 -- 172.21.15.169:0/2081871825 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5f200ad860 msgr2=0x7f5f200add10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:20.364 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.358+0000 7f5f1f7fe640 1 --2- 172.21.15.169:0/2081871825 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5f200ad860 0x7f5f200add10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:20.364 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.358+0000 7f5f1f7fe640 1 -- 172.21.15.169:0/2081871825 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f201616d0 con 0x7f5f0c004d00 2023-10-26T01:33:20.364 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.358+0000 7f5f1f7fe640 1 --2- 172.21.15.169:0/2081871825 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f0c004d00 0x7f5f20163ec0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f5f1000d720 tx=0x7f5f1000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:20.364 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.358+0000 7f5f1cff9640 1 -- 172.21.15.169:0/2081871825 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f1000de30 con 0x7f5f0c004d00 2023-10-26T01:33:20.365 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.358+0000 7f5f1cff9640 1 -- 172.21.15.169:0/2081871825 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5f100063d0 con 0x7f5f0c004d00 2023-10-26T01:33:20.365 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.358+0000 7f5f1cff9640 1 -- 172.21.15.169:0/2081871825 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f1000ba30 con 0x7f5f0c004d00 2023-10-26T01:33:20.365 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.359+0000 7f5f25ba9640 1 -- 172.21.15.169:0/2081871825 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f201619b0 con 0x7f5f0c004d00 2023-10-26T01:33:20.365 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.359+0000 7f5f25ba9640 1 -- 172.21.15.169:0/2081871825 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f20160690 con 0x7f5f0c004d00 2023-10-26T01:33:20.366 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.360+0000 7f5f25ba9640 1 -- 172.21.15.169:0/2081871825 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f2004db10 con 0x7f5f0c004d00 2023-10-26T01:33:20.366 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.361+0000 7f5f1cff9640 1 -- 172.21.15.169:0/2081871825 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f5f1000bb90 con 0x7f5f0c004d00 2023-10-26T01:33:20.367 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.362+0000 7f5f1cff9640 1 --2- 172.21.15.169:0/2081871825 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5f0006ae70 0x7f5f0006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:20.367 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.362+0000 7f5f1cff9640 1 -- 172.21.15.169:0/2081871825 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5f10089f90 con 0x7f5f0c004d00 2023-10-26T01:33:20.367 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.362+0000 7f5f16ffd640 1 --2- 172.21.15.169:0/2081871825 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5f0006ae70 0x7f5f0006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:20.367 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.364+0000 7f5f16ffd640 1 --2- 172.21.15.169:0/2081871825 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5f0006ae70 0x7f5f0006d330 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f5f080011e0 tx=0x7f5f08009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:20.369 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.367+0000 7f5f1cff9640 1 -- 172.21.15.169:0/2081871825 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5f10054fa0 con 0x7f5f0c004d00 2023-10-26T01:33:20.556 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.554+0000 7f5f25ba9640 1 -- 172.21.15.169:0/2081871825 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5f20159c20 con 0x7f5f0c004d00 2023-10-26T01:33:20.557 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.555+0000 7f5f1cff9640 1 -- 172.21.15.169:0/2081871825 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v54) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5f100140a0 con 0x7f5f0c004d00 2023-10-26T01:33:20.558 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 32 2023-10-26T01:33:20.560 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.557+0000 7f5f25ba9640 1 -- 172.21.15.169:0/2081871825 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5f0006ae70 msgr2=0x7f5f0006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:20.561 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.557+0000 7f5f25ba9640 1 --2- 172.21.15.169:0/2081871825 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5f0006ae70 0x7f5f0006d330 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f5f080011e0 tx=0x7f5f08009040 comp rx=0 tx=0).stop 2023-10-26T01:33:20.561 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.557+0000 7f5f25ba9640 1 -- 172.21.15.169:0/2081871825 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f0c004d00 msgr2=0x7f5f20163ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:20.561 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.557+0000 7f5f25ba9640 1 --2- 172.21.15.169:0/2081871825 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f0c004d00 0x7f5f20163ec0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f5f1000d720 tx=0x7f5f1000dbe0 comp rx=0 tx=0).stop 2023-10-26T01:33:20.561 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.557+0000 7f5f25ba9640 1 -- 172.21.15.169:0/2081871825 shutdown_connections 2023-10-26T01:33:20.561 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.557+0000 7f5f25ba9640 1 --2- 172.21.15.169:0/2081871825 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5f0006ae70 0x7f5f0006d330 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:20.562 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.557+0000 7f5f25ba9640 1 --2- 172.21.15.169:0/2081871825 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5f200ad860 0x7f5f200add10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:20.562 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.557+0000 7f5f25ba9640 1 --2- 172.21.15.169:0/2081871825 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5f20160e80 0x7f5f2014bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:20.562 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.557+0000 7f5f25ba9640 1 --2- 172.21.15.169:0/2081871825 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f0c004d00 0x7f5f20163ec0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:20.562 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.558+0000 7f5f25ba9640 1 -- 172.21.15.169:0/2081871825 >> 172.21.15.169:0/2081871825 conn(0x7f5f200af570 msgr2=0x7f5f2006af50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:20.562 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.558+0000 7f5f25ba9640 1 -- 172.21.15.169:0/2081871825 shutdown_connections 2023-10-26T01:33:20.563 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.558+0000 7f5f25ba9640 1 -- 172.21.15.169:0/2081871825 wait complete. 2023-10-26T01:33:20.579 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-26T01:33:20.799 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.796+0000 7f3ba7296640 1 -- 172.21.15.169:0/1329811252 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ba006ce00 msgr2=0x7f3ba0155360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:20.799 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.796+0000 7f3ba7296640 1 --2- 172.21.15.169:0/1329811252 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ba006ce00 0x7f3ba0155360 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f3ba00ce3f0 tx=0x7f3b9c02bd30 comp rx=0 tx=0).stop 2023-10-26T01:33:20.799 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.796+0000 7f3ba7296640 1 -- 172.21.15.169:0/1329811252 shutdown_connections 2023-10-26T01:33:20.800 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.796+0000 7f3ba7296640 1 --2- 172.21.15.169:0/1329811252 >> v2:172.21.15.170:3300/0 conn(0x7f3ba015bbb0 0x7f3ba00cdc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:20.800 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.796+0000 7f3ba7296640 1 --2- 172.21.15.169:0/1329811252 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ba006ce00 0x7f3ba0155360 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:20.800 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.796+0000 7f3ba7296640 1 -- 172.21.15.169:0/1329811252 >> 172.21.15.169:0/1329811252 conn(0x7f3ba0069930 msgr2=0x7f3ba0069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:20.800 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.796+0000 7f3ba7296640 1 -- 172.21.15.169:0/1329811252 shutdown_connections 2023-10-26T01:33:20.800 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.796+0000 7f3ba7296640 1 -- 172.21.15.169:0/1329811252 wait complete. 2023-10-26T01:33:20.802 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.797+0000 7f3ba7296640 1 Processor -- start 2023-10-26T01:33:20.802 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.798+0000 7f3ba7296640 1 -- start start 2023-10-26T01:33:20.802 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.798+0000 7f3ba7296640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3ba006ce00 0x7f3ba014f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:20.803 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.798+0000 7f3ba7296640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3ba00cd7e0 0x7f3ba014e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:20.803 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.799+0000 7f3ba7296640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ba015bbb0 0x7f3ba0160820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:20.803 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.799+0000 7f3ba7296640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba0153a90 con 0x7f3ba015bbb0 2023-10-26T01:33:20.803 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.799+0000 7f3ba7296640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba0153f20 con 0x7f3ba006ce00 2023-10-26T01:33:20.804 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.799+0000 7f3ba7296640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba0152e80 con 0x7f3ba00cd7e0 2023-10-26T01:33:20.804 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.799+0000 7f3ba5833640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ba015bbb0 0x7f3ba0160820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:20.804 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.799+0000 7f3ba5833640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ba015bbb0 0x7f3ba0160820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:47550/0 (socket says 172.21.15.169:47550) 2023-10-26T01:33:20.804 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.799+0000 7f3ba5833640 1 -- 172.21.15.169:0/4278664882 learned_addr learned my addr 172.21.15.169:0/4278664882 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:33:20.804 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.799+0000 7f3ba5032640 1 --2- 172.21.15.169:0/4278664882 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3ba006ce00 0x7f3ba014f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:20.805 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.799+0000 7f3ba4831640 1 --2- 172.21.15.169:0/4278664882 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3ba00cd7e0 0x7f3ba014e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:20.805 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.799+0000 7f3ba5833640 1 -- 172.21.15.169:0/4278664882 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3ba00cd7e0 msgr2=0x7f3ba014e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:20.805 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.799+0000 7f3ba5833640 1 --2- 172.21.15.169:0/4278664882 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3ba00cd7e0 0x7f3ba014e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:20.805 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.799+0000 7f3ba5833640 1 -- 172.21.15.169:0/4278664882 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3ba006ce00 msgr2=0x7f3ba014f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:20.805 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.799+0000 7f3ba5833640 1 --2- 172.21.15.169:0/4278664882 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3ba006ce00 0x7f3ba014f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:20.806 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.799+0000 7f3ba5833640 1 -- 172.21.15.169:0/4278664882 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b9c009560 con 0x7f3ba015bbb0 2023-10-26T01:33:20.806 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.800+0000 7f3ba5833640 1 --2- 172.21.15.169:0/4278664882 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ba015bbb0 0x7f3ba0160820 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f3b9800b970 tx=0x7f3b9800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:20.807 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.800+0000 7f3b95ffb640 1 -- 172.21.15.169:0/4278664882 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b9800cb40 con 0x7f3ba015bbb0 2023-10-26T01:33:20.808 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.800+0000 7f3b95ffb640 1 -- 172.21.15.169:0/4278664882 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3b9800cdb0 con 0x7f3ba015bbb0 2023-10-26T01:33:20.808 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.800+0000 7f3ba7296640 1 -- 172.21.15.169:0/4278664882 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ba0153160 con 0x7f3ba015bbb0 2023-10-26T01:33:20.808 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.800+0000 7f3b95ffb640 1 -- 172.21.15.169:0/4278664882 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b980146a0 con 0x7f3ba015bbb0 2023-10-26T01:33:20.808 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.800+0000 7f3ba7296640 1 -- 172.21.15.169:0/4278664882 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ba0153720 con 0x7f3ba015bbb0 2023-10-26T01:33:20.809 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.801+0000 7f3ba7296640 1 -- 172.21.15.169:0/4278664882 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ba01538e0 con 0x7f3ba015bbb0 2023-10-26T01:33:20.809 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.802+0000 7f3b95ffb640 1 -- 172.21.15.169:0/4278664882 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f3b9801d3b0 con 0x7f3ba015bbb0 2023-10-26T01:33:20.809 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.804+0000 7f3b95ffb640 1 --2- 172.21.15.169:0/4278664882 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3b8406ae40 0x7f3b8406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:20.809 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.804+0000 7f3b95ffb640 1 -- 172.21.15.169:0/4278664882 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3b98089860 con 0x7f3ba015bbb0 2023-10-26T01:33:20.809 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.805+0000 7f3ba5032640 1 --2- 172.21.15.169:0/4278664882 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3b8406ae40 0x7f3b8406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:20.810 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.806+0000 7f3ba5032640 1 --2- 172.21.15.169:0/4278664882 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3b8406ae40 0x7f3b8406d300 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c035000 tx=0x7f3b9c004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:20.810 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.807+0000 7f3b95ffb640 1 -- 172.21.15.169:0/4278664882 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3b98054870 con 0x7f3ba015bbb0 2023-10-26T01:33:20.988 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.985+0000 7f3ba7296640 1 -- 172.21.15.169:0/4278664882 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3ba0153c20 con 0x7f3ba015bbb0 2023-10-26T01:33:20.989 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.986+0000 7f3b95ffb640 1 -- 172.21.15.169:0/4278664882 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v54) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3b980190a0 con 0x7f3ba015bbb0 2023-10-26T01:33:20.989 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 64 2023-10-26T01:33:20.992 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.988+0000 7f3ba7296640 1 -- 172.21.15.169:0/4278664882 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3b8406ae40 msgr2=0x7f3b8406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:20.992 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.988+0000 7f3ba7296640 1 --2- 172.21.15.169:0/4278664882 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3b8406ae40 0x7f3b8406d300 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c035000 tx=0x7f3b9c004730 comp rx=0 tx=0).stop 2023-10-26T01:33:20.992 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.989+0000 7f3ba7296640 1 -- 172.21.15.169:0/4278664882 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ba015bbb0 msgr2=0x7f3ba0160820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:20.993 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.989+0000 7f3ba7296640 1 --2- 172.21.15.169:0/4278664882 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ba015bbb0 0x7f3ba0160820 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f3b9800b970 tx=0x7f3b9800be30 comp rx=0 tx=0).stop 2023-10-26T01:33:20.993 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.989+0000 7f3ba7296640 1 -- 172.21.15.169:0/4278664882 shutdown_connections 2023-10-26T01:33:20.993 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.989+0000 7f3ba7296640 1 --2- 172.21.15.169:0/4278664882 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3b8406ae40 0x7f3b8406d300 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:20.993 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.989+0000 7f3ba7296640 1 --2- 172.21.15.169:0/4278664882 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ba015bbb0 0x7f3ba0160820 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:20.993 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.989+0000 7f3ba7296640 1 --2- 172.21.15.169:0/4278664882 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3ba00cd7e0 0x7f3ba014e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:20.994 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.989+0000 7f3ba7296640 1 --2- 172.21.15.169:0/4278664882 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3ba006ce00 0x7f3ba014f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:20.994 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.989+0000 7f3ba7296640 1 -- 172.21.15.169:0/4278664882 >> 172.21.15.169:0/4278664882 conn(0x7f3ba0069930 msgr2=0x7f3ba0159260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:20.994 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.989+0000 7f3ba7296640 1 -- 172.21.15.169:0/4278664882 shutdown_connections 2023-10-26T01:33:20.994 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:20.989+0000 7f3ba7296640 1 -- 172.21.15.169:0/4278664882 wait complete. 2023-10-26T01:33:21.010 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-26T01:33:21.219 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:21.217+0000 7f030059e640 1 -- 172.21.15.169:0/2575279224 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02f80b05a0 msgr2=0x7f02f80b0a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:21.219 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:21.217+0000 7f030059e640 1 --2- 172.21.15.169:0/2575279224 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02f80b05a0 0x7f02f80b0a10 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f02f0005fd0 tx=0x7f02f002bd90 comp rx=0 tx=0).stop 2023-10-26T01:33:21.220 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:21.217+0000 7f030059e640 1 -- 172.21.15.169:0/2575279224 shutdown_connections 2023-10-26T01:33:21.220 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:21.217+0000 7f030059e640 1 --2- 172.21.15.169:0/2575279224 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02f80b05a0 0x7f02f80b0a10 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:21.220 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:21.217+0000 7f030059e640 1 --2- 172.21.15.169:0/2575279224 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f02f806ce00 0x7f02f8155360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:21.220 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:21.217+0000 7f030059e640 1 -- 172.21.15.169:0/2575279224 >> 172.21.15.169:0/2575279224 conn(0x7f02f8069930 msgr2=0x7f02f8069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:21.221 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:21.217+0000 7f030059e640 1 -- 172.21.15.169:0/2575279224 shutdown_connections 2023-10-26T01:33:21.221 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:21.217+0000 7f030059e640 1 -- 172.21.15.169:0/2575279224 wait complete. 2023-10-26T01:33:21.223 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:21.219+0000 7f030059e640 1 Processor -- start 2023-10-26T01:33:21.224 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:21.219+0000 7f030059e640 1 -- start start 2023-10-26T01:33:21.224 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:21.220+0000 7f030059e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f02f806ce00 0x7f02f8152950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:21.224 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:21.220+0000 7f030059e640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f02f80b05a0 0x7f02f8154490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:21.224 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:21.220+0000 7f030059e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02f815b4a0 0x7f02f81e57a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:21.224 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:21.220+0000 7f030059e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02f81549d0 con 0x7f02f815b4a0 2023-10-26T01:33:21.225 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:21.220+0000 7f030059e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02f8152f30 con 0x7f02f806ce00 2023-10-26T01:33:21.225 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:21.220+0000 7f030059e640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02f8153210 con 0x7f02f80b05a0 2023-10-26T01:33:21.225 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:21.220+0000 7f02feb3b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02f815b4a0 0x7f02f81e57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:21.225 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:21.220+0000 7f02feb3b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02f815b4a0 0x7f02f81e57a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:47574/0 (socket says 172.21.15.169:47574) 2023-10-26T01:33:21.225 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:21.220+0000 7f02feb3b640 1 -- 172.21.15.169:0/556334019 learned_addr learned my addr 172.21.15.169:0/556334019 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:33:21.226 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:21.221+0000 7f02fe33a640 1 --2- 172.21.15.169:0/556334019 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f02f806ce00 0x7f02f8152950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:21.226 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:21.221+0000 7f02fdb39640 1 --2- 172.21.15.169:0/556334019 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f02f80b05a0 0x7f02f8154490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:21.226 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:21.221+0000 7f02feb3b640 1 -- 172.21.15.169:0/556334019 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f02f80b05a0 msgr2=0x7f02f8154490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:21.226 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:21.221+0000 7f02feb3b640 1 --2- 172.21.15.169:0/556334019 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f02f80b05a0 0x7f02f8154490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:21.227 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:21.221+0000 7f02feb3b640 1 -- 172.21.15.169:0/556334019 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f02f806ce00 msgr2=0x7f02f8152950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:21.227 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:21.221+0000 7f02feb3b640 1 --2- 172.21.15.169:0/556334019 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f02f806ce00 0x7f02f8152950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:21.227 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:21.221+0000 7f02feb3b640 1 -- 172.21.15.169:0/556334019 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02f0009560 con 0x7f02f815b4a0 2023-10-26T01:33:21.227 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:21.221+0000 7f02fdb39640 1 --2- 172.21.15.169:0/556334019 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f02f80b05a0 0x7f02f8154490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:33:21.227 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:21.221+0000 7f02feb3b640 1 --2- 172.21.15.169:0/556334019 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02f815b4a0 0x7f02f81e57a0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f02f0006010 tx=0x7f02f002dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:21.228 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:21.221+0000 7f02f6ffd640 1 -- 172.21.15.169:0/556334019 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02f00035f0 con 0x7f02f815b4a0 2023-10-26T01:33:21.228 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:21.221+0000 7f02f6ffd640 1 -- 172.21.15.169:0/556334019 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f02f0003840 con 0x7f02f815b4a0 2023-10-26T01:33:21.228 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:21.221+0000 7f02f6ffd640 1 -- 172.21.15.169:0/556334019 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02f0003bf0 con 0x7f02f815b4a0 2023-10-26T01:33:21.229 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:21.221+0000 7f030059e640 1 -- 172.21.15.169:0/556334019 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02f81534f0 con 0x7f02f815b4a0 2023-10-26T01:33:21.229 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:21.222+0000 7f030059e640 1 -- 172.21.15.169:0/556334019 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02f8153ab0 con 0x7f02f815b4a0 2023-10-26T01:33:21.230 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:21.223+0000 7f030059e640 1 -- 172.21.15.169:0/556334019 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02f804db10 con 0x7f02f815b4a0 2023-10-26T01:33:21.230 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:21.224+0000 7f02f6ffd640 1 -- 172.21.15.169:0/556334019 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f02f0003d50 con 0x7f02f815b4a0 2023-10-26T01:33:21.230 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:21.224+0000 7f02f6ffd640 1 --2- 172.21.15.169:0/556334019 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f02dc06ae40 0x7f02dc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:21.230 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:21.224+0000 7f02f6ffd640 1 -- 172.21.15.169:0/556334019 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f02f00ac2e0 con 0x7f02f815b4a0 2023-10-26T01:33:21.231 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:21.225+0000 7f02fe33a640 1 --2- 172.21.15.169:0/556334019 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f02dc06ae40 0x7f02dc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:21.231 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:21.226+0000 7f02fe33a640 1 --2- 172.21.15.169:0/556334019 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f02dc06ae40 0x7f02dc06d300 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f02ec005c40 tx=0x7f02ec005bd0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:21.231 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:21.229+0000 7f02f6ffd640 1 -- 172.21.15.169:0/556334019 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f02f0077380 con 0x7f02f815b4a0 2023-10-26T01:33:21.436 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:21.434+0000 7f030059e640 1 -- 172.21.15.169:0/556334019 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f02f80aa840 con 0x7f02f815b4a0 2023-10-26T01:33:21.436 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:21.435+0000 7f02f6ffd640 1 -- 172.21.15.169:0/556334019 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v54) v1 ==== 80+0+0 (secure 0 0 0) 0x7f02f0043080 con 0x7f02f815b4a0 2023-10-26T01:33:21.437 INFO:teuthology.orchestra.run.smithi169.stderr:listed 0 entries 2023-10-26T01:33:21.439 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:21.437+0000 7f030059e640 1 -- 172.21.15.169:0/556334019 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f02dc06ae40 msgr2=0x7f02dc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:21.440 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:21.437+0000 7f030059e640 1 --2- 172.21.15.169:0/556334019 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f02dc06ae40 0x7f02dc06d300 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f02ec005c40 tx=0x7f02ec005bd0 comp rx=0 tx=0).stop 2023-10-26T01:33:21.440 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:21.437+0000 7f030059e640 1 -- 172.21.15.169:0/556334019 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02f815b4a0 msgr2=0x7f02f81e57a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:21.440 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:21.437+0000 7f030059e640 1 --2- 172.21.15.169:0/556334019 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02f815b4a0 0x7f02f81e57a0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f02f0006010 tx=0x7f02f002dab0 comp rx=0 tx=0).stop 2023-10-26T01:33:21.440 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:21.438+0000 7f030059e640 1 -- 172.21.15.169:0/556334019 shutdown_connections 2023-10-26T01:33:21.440 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:21.438+0000 7f030059e640 1 --2- 172.21.15.169:0/556334019 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f02dc06ae40 0x7f02dc06d300 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:21.441 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:21.438+0000 7f030059e640 1 --2- 172.21.15.169:0/556334019 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02f815b4a0 0x7f02f81e57a0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:21.441 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:21.438+0000 7f030059e640 1 --2- 172.21.15.169:0/556334019 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f02f80b05a0 0x7f02f8154490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:21.441 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:21.438+0000 7f030059e640 1 --2- 172.21.15.169:0/556334019 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f02f806ce00 0x7f02f8152950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:21.441 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:21.438+0000 7f030059e640 1 -- 172.21.15.169:0/556334019 >> 172.21.15.169:0/556334019 conn(0x7f02f8069930 msgr2=0x7f02f8071350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:21.441 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:21.438+0000 7f030059e640 1 -- 172.21.15.169:0/556334019 shutdown_connections 2023-10-26T01:33:21.442 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:21.438+0000 7f030059e640 1 -- 172.21.15.169:0/556334019 wait complete. 2023-10-26T01:33:21.457 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-26T01:33:21.458 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:33:21.458 DEBUG:teuthology.orchestra.run.smithi170:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-26T01:33:21.573 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi170.front.sepia.ceph.com 2023-10-26T01:33:21.573 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:33:21.574 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-26T01:33:21.626 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:33:21.627 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:33:21.687 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054301,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-26T01:33:21.687 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-26T01:33:21.688 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:33:21.748 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054303,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-26T01:33:21.748 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T01:33:21.978 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:21.976+0000 7f7105890640 1 -- 172.21.15.169:0/1717133020 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71000b45d0 msgr2=0x7f71000cbce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:21.979 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:21.976+0000 7f7105890640 1 --2- 172.21.15.169:0/1717133020 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71000b45d0 0x7f71000cbce0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f70ec009830 tx=0x7f70ec02bd60 comp rx=0 tx=0).stop 2023-10-26T01:33:21.979 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:21.976+0000 7f7105890640 1 -- 172.21.15.169:0/1717133020 shutdown_connections 2023-10-26T01:33:21.979 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:21.976+0000 7f7105890640 1 --2- 172.21.15.169:0/1717133020 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71000b45d0 0x7f71000cbce0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:21.980 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:21.976+0000 7f7105890640 1 --2- 172.21.15.169:0/1717133020 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f71000b68c0 0x7f7100065ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:21.980 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:21.976+0000 7f7105890640 1 -- 172.21.15.169:0/1717133020 >> 172.21.15.169:0/1717133020 conn(0x7f71000d1360 msgr2=0x7f71000d1770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:21.980 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:21.976+0000 7f7105890640 1 -- 172.21.15.169:0/1717133020 shutdown_connections 2023-10-26T01:33:21.980 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:21.977+0000 7f7105890640 1 -- 172.21.15.169:0/1717133020 wait complete. 2023-10-26T01:33:21.983 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:21.978+0000 7f7105890640 1 Processor -- start 2023-10-26T01:33:21.983 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:21.978+0000 7f7105890640 1 -- start start 2023-10-26T01:33:21.983 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:21.979+0000 7f7105890640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f71000b45d0 0x7f71001f1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:21.983 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:21.979+0000 7f7105890640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71000b4a30 0x7f71000aed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:21.983 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:21.979+0000 7f7105890640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71000b68c0 0x7f71000af280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:21.984 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:21.979+0000 7f7105890640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f710014e330 con 0x7f71000b4a30 2023-10-26T01:33:21.984 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:21.979+0000 7f7105890640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f710014ef20 con 0x7f71000b68c0 2023-10-26T01:33:21.984 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:21.979+0000 7f7105890640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f710014e660 con 0x7f71000b45d0 2023-10-26T01:33:21.984 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:21.980+0000 7f70fffff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71000b4a30 0x7f71000aed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:21.985 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:21.980+0000 7f70fffff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71000b4a30 0x7f71000aed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:47586/0 (socket says 172.21.15.169:47586) 2023-10-26T01:33:21.985 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:21.980+0000 7f70fffff640 1 -- 172.21.15.169:0/1206916885 learned_addr learned my addr 172.21.15.169:0/1206916885 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:33:21.985 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:21.980+0000 7f710488e640 1 --2- 172.21.15.169:0/1206916885 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f71000b45d0 0x7f71001f1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:21.985 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:21.980+0000 7f710508f640 1 --2- 172.21.15.169:0/1206916885 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71000b68c0 0x7f71000af280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:21.985 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:21.980+0000 7f70fffff640 1 -- 172.21.15.169:0/1206916885 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f71000b45d0 msgr2=0x7f71001f1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:21.986 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:21.980+0000 7f70fffff640 1 --2- 172.21.15.169:0/1206916885 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f71000b45d0 0x7f71001f1800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:21.986 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:21.980+0000 7f70fffff640 1 -- 172.21.15.169:0/1206916885 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71000b68c0 msgr2=0x7f71000af280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:21.986 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:21.980+0000 7f70fffff640 1 --2- 172.21.15.169:0/1206916885 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71000b68c0 0x7f71000af280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:21.987 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:21.980+0000 7f70fffff640 1 -- 172.21.15.169:0/1206916885 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70ec009490 con 0x7f71000b4a30 2023-10-26T01:33:21.987 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:21.980+0000 7f710508f640 1 --2- 172.21.15.169:0/1206916885 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71000b68c0 0x7f71000af280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:33:21.987 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:21.980+0000 7f70fffff640 1 --2- 172.21.15.169:0/1206916885 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71000b4a30 0x7f71000aed40 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f70ec009950 tx=0x7f70ec0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:21.987 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:21.981+0000 7f70fd7fa640 1 -- 172.21.15.169:0/1206916885 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70ec003ea0 con 0x7f71000b4a30 2023-10-26T01:33:21.988 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:21.981+0000 7f70fd7fa640 1 -- 172.21.15.169:0/1206916885 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f70ec034550 con 0x7f71000b4a30 2023-10-26T01:33:21.988 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:21.981+0000 7f7105890640 1 -- 172.21.15.169:0/1206916885 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f710014e8d0 con 0x7f71000b4a30 2023-10-26T01:33:21.988 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:21.981+0000 7f70fd7fa640 1 -- 172.21.15.169:0/1206916885 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70ec034960 con 0x7f71000b4a30 2023-10-26T01:33:21.988 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:21.981+0000 7f7105890640 1 -- 172.21.15.169:0/1206916885 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71001f1d40 con 0x7f71000b4a30 2023-10-26T01:33:21.989 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:21.982+0000 7f7105890640 1 -- 172.21.15.169:0/1206916885 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f710014f0b0 con 0x7f71000b4a30 2023-10-26T01:33:21.989 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:21.983+0000 7f70fd7fa640 1 -- 172.21.15.169:0/1206916885 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f70ec03b040 con 0x7f71000b4a30 2023-10-26T01:33:21.990 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:21.984+0000 7f70fd7fa640 1 --2- 172.21.15.169:0/1206916885 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f70e406ae40 0x7f70e406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:21.990 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:21.984+0000 7f70fd7fa640 1 -- 172.21.15.169:0/1206916885 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f70ec0abce0 con 0x7f71000b4a30 2023-10-26T01:33:21.990 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:21.984+0000 7f710488e640 1 --2- 172.21.15.169:0/1206916885 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f70e406ae40 0x7f70e406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:21.991 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:21.986+0000 7f710488e640 1 --2- 172.21.15.169:0/1206916885 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f70e406ae40 0x7f70e406d300 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f70f4005d80 tx=0x7f70f4005c70 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:21.991 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:21.988+0000 7f70fd7fa640 1 -- 172.21.15.169:0/1206916885 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f70ec076c60 con 0x7f71000b4a30 2023-10-26T01:33:22.218 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:22.216+0000 7f7105890640 1 -- 172.21.15.169:0/1206916885 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7100073c10 con 0x7f71000b4a30 2023-10-26T01:33:22.219 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:22.218+0000 7f70fd7fa640 1 -- 172.21.15.169:0/1206916885 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4925 (secure 0 0 0) 0x7f70ec039020 con 0x7f71000b4a30 2023-10-26T01:33:22.220 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:33:22.220 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4805,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/2294107634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2294107634},{"type":"v1","addr":"172.21.15.170:6836","nonce":2294107634}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":4987,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2208026662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2208026662},{"type":"v1","addr":"172.21.15.169:6833","nonce":2208026662}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":4993,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/2630365058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":2630365058},{"type":"v1","addr":"172.21.15.170:6837","nonce":2630365058}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T01:32:38.059721+0000","modified":"2023-10-26T01:32:41.428183+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5664},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5664":{"gid":5664,"name":"a","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6835/2816266753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":2816266753},{"type":"v1","addr":"172.21.15.169:6835","nonce":2816266753}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-10-26T01:33:22.220 INFO:teuthology.orchestra.run.smithi169.stderr:dumped fsmap epoch 23 2023-10-26T01:33:22.223 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:22.221+0000 7f7105890640 1 -- 172.21.15.169:0/1206916885 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f70e406ae40 msgr2=0x7f70e406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:22.223 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:22.221+0000 7f7105890640 1 --2- 172.21.15.169:0/1206916885 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f70e406ae40 0x7f70e406d300 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f70f4005d80 tx=0x7f70f4005c70 comp rx=0 tx=0).stop 2023-10-26T01:33:22.223 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:22.221+0000 7f7105890640 1 -- 172.21.15.169:0/1206916885 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71000b4a30 msgr2=0x7f71000aed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:22.223 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:22.221+0000 7f7105890640 1 --2- 172.21.15.169:0/1206916885 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71000b4a30 0x7f71000aed40 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f70ec009950 tx=0x7f70ec0038a0 comp rx=0 tx=0).stop 2023-10-26T01:33:22.224 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:22.221+0000 7f7105890640 1 -- 172.21.15.169:0/1206916885 shutdown_connections 2023-10-26T01:33:22.224 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:22.221+0000 7f7105890640 1 --2- 172.21.15.169:0/1206916885 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f70e406ae40 0x7f70e406d300 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:22.224 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:22.221+0000 7f7105890640 1 --2- 172.21.15.169:0/1206916885 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71000b4a30 0x7f71000aed40 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:22.224 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:22.221+0000 7f7105890640 1 --2- 172.21.15.169:0/1206916885 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71000b68c0 0x7f71000af280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:22.224 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:22.221+0000 7f7105890640 1 --2- 172.21.15.169:0/1206916885 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f71000b45d0 0x7f71001f1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:22.225 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:22.221+0000 7f7105890640 1 -- 172.21.15.169:0/1206916885 >> 172.21.15.169:0/1206916885 conn(0x7f71000d1360 msgr2=0x7f71000cac20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:22.225 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:22.221+0000 7f7105890640 1 -- 172.21.15.169:0/1206916885 shutdown_connections 2023-10-26T01:33:22.225 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:22.221+0000 7f7105890640 1 -- 172.21.15.169:0/1206916885 wait complete. 2023-10-26T01:33:22.242 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:33:22.443 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:22.441+0000 7fd77a95e640 1 -- 172.21.15.169:0/1541573088 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7740ac6d0 msgr2=0x7fd77415d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:22.443 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:22.441+0000 7fd77a95e640 1 --2- 172.21.15.169:0/1541573088 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7740ac6d0 0x7fd77415d840 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fd770005fd0 tx=0x7fd77002be20 comp rx=0 tx=0).stop 2023-10-26T01:33:22.443 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:22.441+0000 7fd77a95e640 1 -- 172.21.15.169:0/1541573088 shutdown_connections 2023-10-26T01:33:22.444 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:22.441+0000 7fd77a95e640 1 --2- 172.21.15.169:0/1541573088 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd7740c9970 0x7fd774098a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:22.444 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:22.441+0000 7fd77a95e640 1 --2- 172.21.15.169:0/1541573088 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7740ac6d0 0x7fd77415d840 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:22.444 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:22.441+0000 7fd77a95e640 1 -- 172.21.15.169:0/1541573088 >> 172.21.15.169:0/1541573088 conn(0x7fd7740aedd0 msgr2=0x7fd7740af1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:22.444 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:22.441+0000 7fd77a95e640 1 -- 172.21.15.169:0/1541573088 shutdown_connections 2023-10-26T01:33:22.444 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:22.441+0000 7fd77a95e640 1 -- 172.21.15.169:0/1541573088 wait complete. 2023-10-26T01:33:22.446 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:22.443+0000 7fd77a95e640 1 Processor -- start 2023-10-26T01:33:22.446 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:22.443+0000 7fd77a95e640 1 -- start start 2023-10-26T01:33:22.446 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:22.444+0000 7fd77a95e640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd77409a5c0 0x7fd774158ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:22.447 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:22.444+0000 7fd77a95e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7740ac6d0 0x7fd77406e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:22.447 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:22.444+0000 7fd77a95e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7740c9970 0x7fd774135220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:22.447 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:22.444+0000 7fd77a95e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7740c8570 con 0x7fd7740c9970 2023-10-26T01:33:22.447 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:22.444+0000 7fd77a95e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7741593e0 con 0x7fd7740ac6d0 2023-10-26T01:33:22.447 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:22.444+0000 7fd77a95e640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7740719d0 con 0x7fd77409a5c0 2023-10-26T01:33:22.447 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:22.444+0000 7fd77a15d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7740c9970 0x7fd774135220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:22.448 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:22.444+0000 7fd77a15d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7740c9970 0x7fd774135220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:47598/0 (socket says 172.21.15.169:47598) 2023-10-26T01:33:22.448 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:22.444+0000 7fd77a15d640 1 -- 172.21.15.169:0/669586221 learned_addr learned my addr 172.21.15.169:0/669586221 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:33:22.448 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:22.444+0000 7fd77995c640 1 --2- 172.21.15.169:0/669586221 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd77409a5c0 0x7fd774158ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:22.448 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:22.444+0000 7fd77a15d640 1 -- 172.21.15.169:0/669586221 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd77409a5c0 msgr2=0x7fd774158ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:22.448 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:22.444+0000 7fd77a15d640 1 --2- 172.21.15.169:0/669586221 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd77409a5c0 0x7fd774158ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:22.449 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:22.444+0000 7fd77a15d640 1 -- 172.21.15.169:0/669586221 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7740ac6d0 msgr2=0x7fd77406e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:22.449 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:22.444+0000 7fd77a15d640 1 --2- 172.21.15.169:0/669586221 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7740ac6d0 0x7fd77406e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:22.449 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:22.444+0000 7fd77a15d640 1 -- 172.21.15.169:0/669586221 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd770009560 con 0x7fd7740c9970 2023-10-26T01:33:22.449 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:22.445+0000 7fd77a15d640 1 --2- 172.21.15.169:0/669586221 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7740c9970 0x7fd774135220 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fd76c00c7d0 tx=0x7fd76c00cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:22.450 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:22.445+0000 7fd76a7fc640 1 -- 172.21.15.169:0/669586221 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd76c003b70 con 0x7fd7740c9970 2023-10-26T01:33:22.451 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:22.445+0000 7fd76a7fc640 1 -- 172.21.15.169:0/669586221 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd76c003cd0 con 0x7fd7740c9970 2023-10-26T01:33:22.451 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:22.445+0000 7fd76a7fc640 1 -- 172.21.15.169:0/669586221 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd76c00aab0 con 0x7fd7740c9970 2023-10-26T01:33:22.451 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:22.445+0000 7fd77a95e640 1 -- 172.21.15.169:0/669586221 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd77415d3d0 con 0x7fd7740c9970 2023-10-26T01:33:22.451 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:22.445+0000 7fd77a95e640 1 -- 172.21.15.169:0/669586221 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd77406ee30 con 0x7fd7740c9970 2023-10-26T01:33:22.451 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:22.446+0000 7fd77a95e640 1 -- 172.21.15.169:0/669586221 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7740c86b0 con 0x7fd7740c9970 2023-10-26T01:33:22.452 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:22.447+0000 7fd76a7fc640 1 -- 172.21.15.169:0/669586221 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fd76c00ac10 con 0x7fd7740c9970 2023-10-26T01:33:22.452 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:22.448+0000 7fd76a7fc640 1 --2- 172.21.15.169:0/669586221 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd75806ae40 0x7fd75806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:22.452 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:22.448+0000 7fd76a7fc640 1 -- 172.21.15.169:0/669586221 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd76c089480 con 0x7fd7740c9970 2023-10-26T01:33:22.452 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:22.449+0000 7fd77995c640 1 --2- 172.21.15.169:0/669586221 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd75806ae40 0x7fd75806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:22.453 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:22.450+0000 7fd77995c640 1 --2- 172.21.15.169:0/669586221 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd75806ae40 0x7fd75806d300 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fd770035000 tx=0x7fd770004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:22.453 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:22.452+0000 7fd76a7fc640 1 -- 172.21.15.169:0/669586221 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd7740c86b0 con 0x7fd7740c9970 2023-10-26T01:33:22.632 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:22.630+0000 7fd77a95e640 1 -- 172.21.15.169:0/669586221 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd7740ca410 con 0x7fd7740c9970 2023-10-26T01:33:22.634 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:22.632+0000 7fd76a7fc640 1 -- 172.21.15.169:0/669586221 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+16561 (secure 0 0 0) 0x7fd7740ca410 con 0x7fd7740c9970 2023-10-26T01:33:22.635 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:33:22.635 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":54,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:33:10.190714+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-26T01:32:36.288217+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-26T01:32:37.059775+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:33:22.637 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:22.635+0000 7fd77a95e640 1 -- 172.21.15.169:0/669586221 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd75806ae40 msgr2=0x7fd75806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:22.637 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:22.635+0000 7fd77a95e640 1 --2- 172.21.15.169:0/669586221 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd75806ae40 0x7fd75806d300 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fd770035000 tx=0x7fd770004730 comp rx=0 tx=0).stop 2023-10-26T01:33:22.637 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:22.635+0000 7fd77a95e640 1 -- 172.21.15.169:0/669586221 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7740c9970 msgr2=0x7fd774135220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:22.637 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:22.635+0000 7fd77a95e640 1 --2- 172.21.15.169:0/669586221 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7740c9970 0x7fd774135220 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fd76c00c7d0 tx=0x7fd76c00cc90 comp rx=0 tx=0).stop 2023-10-26T01:33:22.638 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:22.635+0000 7fd77a95e640 1 -- 172.21.15.169:0/669586221 shutdown_connections 2023-10-26T01:33:22.638 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:22.635+0000 7fd77a95e640 1 --2- 172.21.15.169:0/669586221 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd75806ae40 0x7fd75806d300 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:22.638 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:22.635+0000 7fd77a95e640 1 --2- 172.21.15.169:0/669586221 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7740c9970 0x7fd774135220 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:22.641 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:22.635+0000 7fd77a95e640 1 --2- 172.21.15.169:0/669586221 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7740ac6d0 0x7fd77406e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:22.642 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:22.635+0000 7fd77a95e640 1 --2- 172.21.15.169:0/669586221 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd77409a5c0 0x7fd774158ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:22.642 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:22.635+0000 7fd77a95e640 1 -- 172.21.15.169:0/669586221 >> 172.21.15.169:0/669586221 conn(0x7fd7740aedd0 msgr2=0x7fd774141740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:22.642 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:22.636+0000 7fd77a95e640 1 -- 172.21.15.169:0/669586221 shutdown_connections 2023-10-26T01:33:22.642 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:22.636+0000 7fd77a95e640 1 -- 172.21.15.169:0/669586221 wait complete. 2023-10-26T01:33:22.655 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T01:30:32.200808+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-26T01:32:36.288217+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-10-26T01:32:37.059775+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T01:33:22.656 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T01:33:22.873 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:22.871+0000 7f99fd6f2640 1 -- 172.21.15.169:0/2380041896 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99f80cdbd0 msgr2=0x7f99f80ac3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:22.874 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:22.871+0000 7f99fd6f2640 1 --2- 172.21.15.169:0/2380041896 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99f80cdbd0 0x7f99f80ac3d0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f99e0005fd0 tx=0x7f99e002bd90 comp rx=0 tx=0).stop 2023-10-26T01:33:22.874 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:22.872+0000 7f99fd6f2640 1 -- 172.21.15.169:0/2380041896 shutdown_connections 2023-10-26T01:33:22.874 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:22.872+0000 7f99fd6f2640 1 --2- 172.21.15.169:0/2380041896 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99f80cdbd0 0x7f99f80ac3d0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:22.875 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:22.872+0000 7f99fd6f2640 1 --2- 172.21.15.169:0/2380041896 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f99f815b6a0 0x7f99f80cd690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:22.875 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:22.872+0000 7f99fd6f2640 1 -- 172.21.15.169:0/2380041896 >> 172.21.15.169:0/2380041896 conn(0x7f99f8069930 msgr2=0x7f99f8069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:22.875 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:22.872+0000 7f99fd6f2640 1 -- 172.21.15.169:0/2380041896 shutdown_connections 2023-10-26T01:33:22.875 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:22.872+0000 7f99fd6f2640 1 -- 172.21.15.169:0/2380041896 wait complete. 2023-10-26T01:33:22.878 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:22.873+0000 7f99fd6f2640 1 Processor -- start 2023-10-26T01:33:22.878 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:22.874+0000 7f99fd6f2640 1 -- start start 2023-10-26T01:33:22.878 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:22.874+0000 7f99fd6f2640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f99f80ac910 0x7f99f81f18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:22.879 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:22.874+0000 7f99fd6f2640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99f80cdbd0 0x7f99f8153af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:22.879 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:22.874+0000 7f99fd6f2640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f99f815b6a0 0x7f99f8151bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:22.879 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:22.875+0000 7f99fd6f2640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99f8158bb0 con 0x7f99f80cdbd0 2023-10-26T01:33:22.879 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:22.875+0000 7f99fd6f2640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99f81f1e30 con 0x7f99f80ac910 2023-10-26T01:33:22.879 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:22.875+0000 7f99fd6f2640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99f81544f0 con 0x7f99f815b6a0 2023-10-26T01:33:22.880 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:22.875+0000 7f99f67fc640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99f80cdbd0 0x7f99f8153af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:22.880 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:22.875+0000 7f99f67fc640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99f80cdbd0 0x7f99f8153af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:47618/0 (socket says 172.21.15.169:47618) 2023-10-26T01:33:22.880 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:22.875+0000 7f99f67fc640 1 -- 172.21.15.169:0/555853928 learned_addr learned my addr 172.21.15.169:0/555853928 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:33:22.880 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:22.875+0000 7f99f6ffd640 1 --2- 172.21.15.169:0/555853928 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f99f80ac910 0x7f99f81f18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:22.880 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:22.875+0000 7f99f67fc640 1 -- 172.21.15.169:0/555853928 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f99f815b6a0 msgr2=0x7f99f8151bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:22.881 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:22.875+0000 7f99f67fc640 1 --2- 172.21.15.169:0/555853928 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f99f815b6a0 0x7f99f8151bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:22.881 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:22.875+0000 7f99f67fc640 1 -- 172.21.15.169:0/555853928 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f99f80ac910 msgr2=0x7f99f81f18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:22.881 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:22.875+0000 7f99f67fc640 1 --2- 172.21.15.169:0/555853928 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f99f80ac910 0x7f99f81f18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:22.881 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:22.875+0000 7f99f67fc640 1 -- 172.21.15.169:0/555853928 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99e0009560 con 0x7f99f80cdbd0 2023-10-26T01:33:22.882 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:22.875+0000 7f99f6ffd640 1 --2- 172.21.15.169:0/555853928 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f99f80ac910 0x7f99f81f18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:33:22.882 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:22.876+0000 7f99f67fc640 1 --2- 172.21.15.169:0/555853928 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99f80cdbd0 0x7f99f8153af0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f99e0036040 tx=0x7f99e002dd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:22.882 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:22.876+0000 7f99ef7fe640 1 -- 172.21.15.169:0/555853928 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e0005650 con 0x7f99f80cdbd0 2023-10-26T01:33:22.882 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:22.876+0000 7f99ef7fe640 1 -- 172.21.15.169:0/555853928 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f99e00038f0 con 0x7f99f80cdbd0 2023-10-26T01:33:22.883 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:22.876+0000 7f99fd6f2640 1 -- 172.21.15.169:0/555853928 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99f81547d0 con 0x7f99f80cdbd0 2023-10-26T01:33:22.883 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:22.876+0000 7f99ef7fe640 1 -- 172.21.15.169:0/555853928 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e0003ca0 con 0x7f99f80cdbd0 2023-10-26T01:33:22.883 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:22.876+0000 7f99fd6f2640 1 -- 172.21.15.169:0/555853928 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99f8154d90 con 0x7f99f80cdbd0 2023-10-26T01:33:22.884 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:22.877+0000 7f99fd6f2640 1 -- 172.21.15.169:0/555853928 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99f8066af0 con 0x7f99f80cdbd0 2023-10-26T01:33:22.885 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:22.878+0000 7f99ef7fe640 1 -- 172.21.15.169:0/555853928 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f99e003e450 con 0x7f99f80cdbd0 2023-10-26T01:33:22.885 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:22.879+0000 7f99ef7fe640 1 --2- 172.21.15.169:0/555853928 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f99d806ae40 0x7f99d806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:22.885 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:22.879+0000 7f99ef7fe640 1 -- 172.21.15.169:0/555853928 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f99e00ac360 con 0x7f99f80cdbd0 2023-10-26T01:33:22.885 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:22.880+0000 7f99f6ffd640 1 --2- 172.21.15.169:0/555853928 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f99d806ae40 0x7f99d806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:22.885 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:22.881+0000 7f99f6ffd640 1 --2- 172.21.15.169:0/555853928 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f99d806ae40 0x7f99d806d300 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f99e8005fd0 tx=0x7f99e80082b0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:22.886 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:22.883+0000 7f99ef7fe640 1 -- 172.21.15.169:0/555853928 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f99e0077540 con 0x7f99f80cdbd0 2023-10-26T01:33:23.075 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.073+0000 7f99fd6f2640 1 -- 172.21.15.169:0/555853928 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f99f804da70 con 0x7f99f80cdbd0 2023-10-26T01:33:23.075 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.074+0000 7f99ef7fe640 1 -- 172.21.15.169:0/555853928 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v54) v1 ==== 93+0+10 (secure 0 0 0) 0x7f99e007ab60 con 0x7f99f80cdbd0 2023-10-26T01:33:23.076 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 1 2023-10-26T01:33:23.078 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.076+0000 7f99fd6f2640 1 -- 172.21.15.169:0/555853928 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f99d806ae40 msgr2=0x7f99d806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:23.079 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.076+0000 7f99fd6f2640 1 --2- 172.21.15.169:0/555853928 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f99d806ae40 0x7f99d806d300 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f99e8005fd0 tx=0x7f99e80082b0 comp rx=0 tx=0).stop 2023-10-26T01:33:23.079 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.076+0000 7f99fd6f2640 1 -- 172.21.15.169:0/555853928 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99f80cdbd0 msgr2=0x7f99f8153af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:23.079 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.076+0000 7f99fd6f2640 1 --2- 172.21.15.169:0/555853928 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99f80cdbd0 0x7f99f8153af0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f99e0036040 tx=0x7f99e002dd60 comp rx=0 tx=0).stop 2023-10-26T01:33:23.079 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.076+0000 7f99fd6f2640 1 -- 172.21.15.169:0/555853928 shutdown_connections 2023-10-26T01:33:23.079 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.077+0000 7f99fd6f2640 1 --2- 172.21.15.169:0/555853928 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f99d806ae40 0x7f99d806d300 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:23.080 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.077+0000 7f99fd6f2640 1 --2- 172.21.15.169:0/555853928 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99f80cdbd0 0x7f99f8153af0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:23.080 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.077+0000 7f99fd6f2640 1 --2- 172.21.15.169:0/555853928 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f99f815b6a0 0x7f99f8151bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:23.080 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.077+0000 7f99fd6f2640 1 --2- 172.21.15.169:0/555853928 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f99f80ac910 0x7f99f81f18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:23.080 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.077+0000 7f99fd6f2640 1 -- 172.21.15.169:0/555853928 >> 172.21.15.169:0/555853928 conn(0x7f99f8069930 msgr2=0x7f99f806c500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:23.080 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.077+0000 7f99fd6f2640 1 -- 172.21.15.169:0/555853928 shutdown_connections 2023-10-26T01:33:23.081 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.077+0000 7f99fd6f2640 1 -- 172.21.15.169:0/555853928 wait complete. 2023-10-26T01:33:23.097 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-26T01:33:23.320 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.317+0000 7f85c8c1a640 1 -- 172.21.15.169:0/2779591954 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f85b0003970 msgr2=0x7f85b0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:23.320 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.317+0000 7f85c8c1a640 1 --2- 172.21.15.169:0/2779591954 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f85b0003970 0x7f85b0003d90 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f85b400c670 tx=0x7f85b00057f0 comp rx=0 tx=0).stop 2023-10-26T01:33:23.320 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.317+0000 7f85c2d9d640 1 -- 172.21.15.169:0/2779591954 reap_dead start 2023-10-26T01:33:23.320 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.318+0000 7f85c8c1a640 1 -- 172.21.15.169:0/2779591954 shutdown_connections 2023-10-26T01:33:23.321 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.318+0000 7f85c8c1a640 1 -- 172.21.15.169:0/2779591954 >> 172.21.15.169:0/2779591954 conn(0x7f85c4069930 msgr2=0x7f85c4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:23.321 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.318+0000 7f85c8c1a640 1 -- 172.21.15.169:0/2779591954 shutdown_connections 2023-10-26T01:33:23.321 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.318+0000 7f85c8c1a640 1 -- 172.21.15.169:0/2779591954 wait complete. 2023-10-26T01:33:23.323 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.319+0000 7f85c8c1a640 1 Processor -- start 2023-10-26T01:33:23.324 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.320+0000 7f85c8c1a640 1 -- start start 2023-10-26T01:33:23.324 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.320+0000 7f85c8c1a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f85b0003970 0x7f85c40cb250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:23.324 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.321+0000 7f85c8c1a640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f85c414e9c0 0x7f85c40af210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:23.324 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.321+0000 7f85c8c1a640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f85c40af750 0x7f85c40ca0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:23.324 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.321+0000 7f85c8c1a640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85c40afbe0 con 0x7f85c414e9c0 2023-10-26T01:33:23.325 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.321+0000 7f85c8c1a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85c41f3150 con 0x7f85b0003970 2023-10-26T01:33:23.325 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.321+0000 7f85c8c1a640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85c41f2540 con 0x7f85c40af750 2023-10-26T01:33:23.325 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.321+0000 7f85c1d9b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f85c414e9c0 0x7f85c40af210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:23.325 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.321+0000 7f85c1d9b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f85c414e9c0 0x7f85c40af210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:47648/0 (socket says 172.21.15.169:47648) 2023-10-26T01:33:23.325 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.321+0000 7f85c1d9b640 1 -- 172.21.15.169:0/1468137341 learned_addr learned my addr 172.21.15.169:0/1468137341 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:33:23.326 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.321+0000 7f85c259c640 1 --2- 172.21.15.169:0/1468137341 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f85b0003970 0x7f85c40cb250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:23.326 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.321+0000 7f85c1d9b640 1 -- 172.21.15.169:0/1468137341 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f85c40af750 msgr2=0x7f85c40ca0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:23.326 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.321+0000 7f85c2d9d640 1 --2- 172.21.15.169:0/1468137341 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f85c40af750 0x7f85c40ca0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:23.326 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.321+0000 7f85c1d9b640 1 --2- 172.21.15.169:0/1468137341 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f85c40af750 0x7f85c40ca0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:23.326 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.321+0000 7f85c1d9b640 1 -- 172.21.15.169:0/1468137341 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f85b0003970 msgr2=0x7f85c40cb250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:23.327 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.321+0000 7f85c1d9b640 1 --2- 172.21.15.169:0/1468137341 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f85b0003970 0x7f85c40cb250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:23.327 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.321+0000 7f85c1d9b640 1 -- 172.21.15.169:0/1468137341 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85c41f2820 con 0x7f85c414e9c0 2023-10-26T01:33:23.327 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.322+0000 7f85c1d9b640 1 --2- 172.21.15.169:0/1468137341 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f85c414e9c0 0x7f85c40af210 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f85ac00e740 tx=0x7f85ac00ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:23.327 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.322+0000 7f85baffd640 1 -- 172.21.15.169:0/1468137341 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85ac00cc60 con 0x7f85c414e9c0 2023-10-26T01:33:23.327 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.322+0000 7f85baffd640 1 -- 172.21.15.169:0/1468137341 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f85ac00ee50 con 0x7f85c414e9c0 2023-10-26T01:33:23.328 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.322+0000 7f85c8c1a640 1 -- 172.21.15.169:0/1468137341 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85c41f2b00 con 0x7f85c414e9c0 2023-10-26T01:33:23.328 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.322+0000 7f85baffd640 1 -- 172.21.15.169:0/1468137341 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85ac005280 con 0x7f85c414e9c0 2023-10-26T01:33:23.328 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.322+0000 7f85c8c1a640 1 -- 172.21.15.169:0/1468137341 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85c415a830 con 0x7f85c414e9c0 2023-10-26T01:33:23.329 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.323+0000 7f85c8c1a640 1 -- 172.21.15.169:0/1468137341 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85c40cba80 con 0x7f85c414e9c0 2023-10-26T01:33:23.329 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.324+0000 7f85baffd640 1 -- 172.21.15.169:0/1468137341 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f85ac0053e0 con 0x7f85c414e9c0 2023-10-26T01:33:23.329 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.325+0000 7f85baffd640 1 --2- 172.21.15.169:0/1468137341 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f85a406ae70 0x7f85a406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:23.330 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.325+0000 7f85baffd640 1 -- 172.21.15.169:0/1468137341 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f85ac089ed0 con 0x7f85c414e9c0 2023-10-26T01:33:23.330 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.326+0000 7f85c259c640 1 --2- 172.21.15.169:0/1468137341 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f85a406ae70 0x7f85a406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:23.330 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.327+0000 7f85c259c640 1 --2- 172.21.15.169:0/1468137341 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f85a406ae70 0x7f85a406d330 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f85b4035000 tx=0x7f85b4001df0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:23.331 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.330+0000 7f85baffd640 1 -- 172.21.15.169:0/1468137341 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f85c40cba80 con 0x7f85c414e9c0 2023-10-26T01:33:23.523 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.522+0000 7f85c8c1a640 1 -- 172.21.15.169:0/1468137341 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f85c40cba60 con 0x7f85c414e9c0 2023-10-26T01:33:23.524 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.522+0000 7f85baffd640 1 -- 172.21.15.169:0/1468137341 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v54) v1 ==== 104+0+11 (secure 0 0 0) 0x7f85ac01f020 con 0x7f85c414e9c0 2023-10-26T01:33:23.524 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 32 2023-10-26T01:33:23.527 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.525+0000 7f85c8c1a640 1 -- 172.21.15.169:0/1468137341 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f85a406ae70 msgr2=0x7f85a406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:23.527 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.525+0000 7f85c8c1a640 1 --2- 172.21.15.169:0/1468137341 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f85a406ae70 0x7f85a406d330 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f85b4035000 tx=0x7f85b4001df0 comp rx=0 tx=0).stop 2023-10-26T01:33:23.528 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.525+0000 7f85c8c1a640 1 -- 172.21.15.169:0/1468137341 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f85c414e9c0 msgr2=0x7f85c40af210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:23.528 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.525+0000 7f85c8c1a640 1 --2- 172.21.15.169:0/1468137341 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f85c414e9c0 0x7f85c40af210 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f85ac00e740 tx=0x7f85ac00ec00 comp rx=0 tx=0).stop 2023-10-26T01:33:23.528 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.525+0000 7f85c8c1a640 1 -- 172.21.15.169:0/1468137341 shutdown_connections 2023-10-26T01:33:23.528 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.525+0000 7f85c8c1a640 1 --2- 172.21.15.169:0/1468137341 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f85a406ae70 0x7f85a406d330 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:23.528 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.525+0000 7f85c8c1a640 1 --2- 172.21.15.169:0/1468137341 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f85c414e9c0 0x7f85c40af210 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:23.529 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.525+0000 7f85c8c1a640 1 --2- 172.21.15.169:0/1468137341 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f85c40af750 0x7f85c40ca0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:23.529 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.525+0000 7f85c8c1a640 1 --2- 172.21.15.169:0/1468137341 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f85b0003970 0x7f85c40cb250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:23.529 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.525+0000 7f85c8c1a640 1 -- 172.21.15.169:0/1468137341 >> 172.21.15.169:0/1468137341 conn(0x7f85c4069930 msgr2=0x7f85c40ad160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:23.529 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.525+0000 7f85c8c1a640 1 -- 172.21.15.169:0/1468137341 shutdown_connections 2023-10-26T01:33:23.529 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.526+0000 7f85c8c1a640 1 -- 172.21.15.169:0/1468137341 wait complete. 2023-10-26T01:33:23.545 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-26T01:33:23.756 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.753+0000 7fbad513c640 1 -- 172.21.15.169:0/579213727 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbad006cba0 msgr2=0x7fbad015b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:23.756 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.753+0000 7fbad513c640 1 --2- 172.21.15.169:0/579213727 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbad006cba0 0x7fbad015b140 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fbad006e430 tx=0x7fbac002bd10 comp rx=0 tx=0).stop 2023-10-26T01:33:23.756 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.754+0000 7fbad513c640 1 -- 172.21.15.169:0/579213727 shutdown_connections 2023-10-26T01:33:23.756 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.754+0000 7fbad513c640 1 --2- 172.21.15.169:0/579213727 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbad00b0300 0x7fbad00b0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:23.757 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.754+0000 7fbad513c640 1 --2- 172.21.15.169:0/579213727 >> v2:172.21.15.170:3300/0 conn(0x7fbad00aa6d0 0x7fbad00aab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:23.757 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.754+0000 7fbad513c640 1 --2- 172.21.15.169:0/579213727 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbad006cba0 0x7fbad015b140 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:23.757 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.754+0000 7fbad513c640 1 -- 172.21.15.169:0/579213727 >> 172.21.15.169:0/579213727 conn(0x7fbad00af110 msgr2=0x7fbad00af520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:23.757 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.754+0000 7fbad513c640 1 -- 172.21.15.169:0/579213727 shutdown_connections 2023-10-26T01:33:23.757 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.754+0000 7fbad513c640 1 -- 172.21.15.169:0/579213727 wait complete. 2023-10-26T01:33:23.760 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.755+0000 7fbad513c640 1 Processor -- start 2023-10-26T01:33:23.760 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.756+0000 7fbad513c640 1 -- start start 2023-10-26T01:33:23.760 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.757+0000 7fbad513c640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbad006cba0 0x7fbad015a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:23.760 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.757+0000 7fbad513c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbad00aa6d0 0x7fbad0150fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:23.760 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.757+0000 7fbad513c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbad00b0300 0x7fbad01514f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:23.761 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.757+0000 7fbad513c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbad006f8d0 con 0x7fbad00b0300 2023-10-26T01:33:23.761 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.757+0000 7fbad513c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbad006ec50 con 0x7fbad00aa6d0 2023-10-26T01:33:23.761 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.757+0000 7fbad513c640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbad006ef30 con 0x7fbad006cba0 2023-10-26T01:33:23.761 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.757+0000 7fbad493b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbad00b0300 0x7fbad01514f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:23.762 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.757+0000 7fbad493b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbad00b0300 0x7fbad01514f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:47656/0 (socket says 172.21.15.169:47656) 2023-10-26T01:33:23.762 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.757+0000 7fbad493b640 1 -- 172.21.15.169:0/2119033421 learned_addr learned my addr 172.21.15.169:0/2119033421 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:33:23.762 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.757+0000 7fbacffff640 1 --2- 172.21.15.169:0/2119033421 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbad006cba0 0x7fbad015a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:23.762 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.757+0000 7fbac7fff640 1 --2- 172.21.15.169:0/2119033421 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbad00aa6d0 0x7fbad0150fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:23.762 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.757+0000 7fbad493b640 1 -- 172.21.15.169:0/2119033421 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbad006cba0 msgr2=0x7fbad015a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:23.763 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.757+0000 7fbad493b640 1 --2- 172.21.15.169:0/2119033421 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbad006cba0 0x7fbad015a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:23.763 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.757+0000 7fbad493b640 1 -- 172.21.15.169:0/2119033421 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbad00aa6d0 msgr2=0x7fbad0150fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:23.763 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.757+0000 7fbad493b640 1 --2- 172.21.15.169:0/2119033421 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbad00aa6d0 0x7fbad0150fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:23.763 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.757+0000 7fbad493b640 1 -- 172.21.15.169:0/2119033421 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbac0009560 con 0x7fbad00b0300 2023-10-26T01:33:23.763 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.758+0000 7fbad493b640 1 --2- 172.21.15.169:0/2119033421 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbad00b0300 0x7fbad01514f0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fbac800cad0 tx=0x7fbac800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:23.764 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.758+0000 7fbacd7fa640 1 -- 172.21.15.169:0/2119033421 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbac800ae20 con 0x7fbad00b0300 2023-10-26T01:33:23.764 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.758+0000 7fbacd7fa640 1 -- 172.21.15.169:0/2119033421 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fbac8004b20 con 0x7fbad00b0300 2023-10-26T01:33:23.764 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.758+0000 7fbad513c640 1 -- 172.21.15.169:0/2119033421 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbad006f210 con 0x7fbad00b0300 2023-10-26T01:33:23.764 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.758+0000 7fbacd7fa640 1 -- 172.21.15.169:0/2119033421 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbac8005020 con 0x7fbad00b0300 2023-10-26T01:33:23.764 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.758+0000 7fbad513c640 1 -- 172.21.15.169:0/2119033421 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbad01e3750 con 0x7fbad00b0300 2023-10-26T01:33:23.765 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.759+0000 7fbad513c640 1 -- 172.21.15.169:0/2119033421 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbad006fa10 con 0x7fbad00b0300 2023-10-26T01:33:23.766 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.760+0000 7fbacd7fa640 1 -- 172.21.15.169:0/2119033421 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fbac8005180 con 0x7fbad00b0300 2023-10-26T01:33:23.766 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.761+0000 7fbacd7fa640 1 --2- 172.21.15.169:0/2119033421 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbab006ae90 0x7fbab006d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:23.766 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.761+0000 7fbacd7fa640 1 -- 172.21.15.169:0/2119033421 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbac8089ea0 con 0x7fbad00b0300 2023-10-26T01:33:23.766 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.762+0000 7fbacffff640 1 --2- 172.21.15.169:0/2119033421 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbab006ae90 0x7fbab006d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:23.766 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.763+0000 7fbacffff640 1 --2- 172.21.15.169:0/2119033421 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbab006ae90 0x7fbab006d350 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fbac0035000 tx=0x7fbac0004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:23.767 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.765+0000 7fbacd7fa640 1 -- 172.21.15.169:0/2119033421 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbad006fa10 con 0x7fbad00b0300 2023-10-26T01:33:23.950 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.948+0000 7fbad513c640 1 -- 172.21.15.169:0/2119033421 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbad006fa10 con 0x7fbad00b0300 2023-10-26T01:33:23.951 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.949+0000 7fbacd7fa640 1 -- 172.21.15.169:0/2119033421 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v54) v1 ==== 100+0+11 (secure 0 0 0) 0x7fbad006fa10 con 0x7fbad00b0300 2023-10-26T01:33:23.951 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 64 2023-10-26T01:33:23.954 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.952+0000 7fbad513c640 1 -- 172.21.15.169:0/2119033421 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbab006ae90 msgr2=0x7fbab006d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:23.954 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.952+0000 7fbad513c640 1 --2- 172.21.15.169:0/2119033421 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbab006ae90 0x7fbab006d350 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fbac0035000 tx=0x7fbac0004730 comp rx=0 tx=0).stop 2023-10-26T01:33:23.954 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.952+0000 7fbad513c640 1 -- 172.21.15.169:0/2119033421 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbad00b0300 msgr2=0x7fbad01514f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:23.954 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.952+0000 7fbad513c640 1 --2- 172.21.15.169:0/2119033421 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbad00b0300 0x7fbad01514f0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fbac800cad0 tx=0x7fbac800cf90 comp rx=0 tx=0).stop 2023-10-26T01:33:23.955 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.952+0000 7fbad513c640 1 -- 172.21.15.169:0/2119033421 shutdown_connections 2023-10-26T01:33:23.955 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.952+0000 7fbad513c640 1 --2- 172.21.15.169:0/2119033421 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbab006ae90 0x7fbab006d350 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:23.955 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.952+0000 7fbad513c640 1 --2- 172.21.15.169:0/2119033421 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbad00b0300 0x7fbad01514f0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:23.956 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.952+0000 7fbad513c640 1 --2- 172.21.15.169:0/2119033421 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbad00aa6d0 0x7fbad0150fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:23.956 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.952+0000 7fbad513c640 1 --2- 172.21.15.169:0/2119033421 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbad006cba0 0x7fbad015a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:23.956 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.952+0000 7fbad513c640 1 -- 172.21.15.169:0/2119033421 >> 172.21.15.169:0/2119033421 conn(0x7fbad00af110 msgr2=0x7fbad00ae570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:23.956 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.952+0000 7fbad513c640 1 -- 172.21.15.169:0/2119033421 shutdown_connections 2023-10-26T01:33:23.956 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:23.952+0000 7fbad513c640 1 -- 172.21.15.169:0/2119033421 wait complete. 2023-10-26T01:33:23.972 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T01:33:24.180 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:24.178+0000 7fd011597640 1 -- 172.21.15.169:0/2545582245 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd00c0cde30 msgr2=0x7fd00c0cc440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:24.181 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:24.178+0000 7fd011597640 1 --2- 172.21.15.169:0/2545582245 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd00c0cde30 0x7fd00c0cc440 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fd004009be0 tx=0x7fd00402c170 comp rx=0 tx=0).stop 2023-10-26T01:33:24.181 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:24.178+0000 7fd011597640 1 -- 172.21.15.169:0/2545582245 shutdown_connections 2023-10-26T01:33:24.181 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:24.178+0000 7fd011597640 1 --2- 172.21.15.169:0/2545582245 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd00c0cde30 0x7fd00c0cc440 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:24.181 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:24.178+0000 7fd011597640 1 -- 172.21.15.169:0/2545582245 >> 172.21.15.169:0/2545582245 conn(0x7fd00c069930 msgr2=0x7fd00c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:24.182 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:24.179+0000 7fd011597640 1 -- 172.21.15.169:0/2545582245 shutdown_connections 2023-10-26T01:33:24.182 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:24.179+0000 7fd011597640 1 -- 172.21.15.169:0/2545582245 wait complete. 2023-10-26T01:33:24.184 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:24.180+0000 7fd011597640 1 Processor -- start 2023-10-26T01:33:24.184 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:24.180+0000 7fd011597640 1 -- start start 2023-10-26T01:33:24.184 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:24.181+0000 7fd011597640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd00c0cde30 0x7fd00c06fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:24.185 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:24.181+0000 7fd011597640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd00c14f2c0 0x7fd00c071470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:24.185 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:24.181+0000 7fd011597640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd00c14f6a0 0x7fd00c14d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:24.185 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:24.181+0000 7fd011597640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd00c0703d0 con 0x7fd00c14f2c0 2023-10-26T01:33:24.185 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:24.181+0000 7fd011597640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd00c0706b0 con 0x7fd00c0cde30 2023-10-26T01:33:24.185 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:24.181+0000 7fd011597640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd00c070990 con 0x7fd00c14f6a0 2023-10-26T01:33:24.186 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:24.181+0000 7fd0027fc640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd00c14f2c0 0x7fd00c071470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:24.186 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:24.181+0000 7fd0027fc640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd00c14f2c0 0x7fd00c071470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:42648/0 (socket says 172.21.15.169:42648) 2023-10-26T01:33:24.186 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:24.181+0000 7fd0027fc640 1 -- 172.21.15.169:0/2448915286 learned_addr learned my addr 172.21.15.169:0/2448915286 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:33:24.186 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:24.182+0000 7fd0027fc640 1 -- 172.21.15.169:0/2448915286 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd00c14f6a0 msgr2=0x7fd00c14d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:24.186 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:24.182+0000 7fd0027fc640 1 --2- 172.21.15.169:0/2448915286 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd00c14f6a0 0x7fd00c14d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:24.187 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:24.182+0000 7fd0027fc640 1 -- 172.21.15.169:0/2448915286 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd00c0cde30 msgr2=0x7fd00c06fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:24.187 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:24.182+0000 7fd00affd640 1 --2- 172.21.15.169:0/2448915286 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd00c0cde30 0x7fd00c06fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:24.187 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:24.182+0000 7fd0027fc640 1 --2- 172.21.15.169:0/2448915286 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd00c0cde30 0x7fd00c06fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:24.187 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:24.182+0000 7fd0027fc640 1 -- 172.21.15.169:0/2448915286 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd004009840 con 0x7fd00c14f2c0 2023-10-26T01:33:24.187 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:24.182+0000 7fd00affd640 1 --2- 172.21.15.169:0/2448915286 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd00c0cde30 0x7fd00c06fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:33:24.188 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:24.182+0000 7fd0027fc640 1 --2- 172.21.15.169:0/2448915286 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd00c14f2c0 0x7fd00c071470 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fcff400c3c0 tx=0x7fcff400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:24.188 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:24.182+0000 7fd003fff640 1 -- 172.21.15.169:0/2448915286 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcff40052d0 con 0x7fd00c14f2c0 2023-10-26T01:33:24.188 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:24.182+0000 7fd003fff640 1 -- 172.21.15.169:0/2448915286 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fcff40035f0 con 0x7fd00c14f2c0 2023-10-26T01:33:24.188 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:24.182+0000 7fd003fff640 1 -- 172.21.15.169:0/2448915286 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcff4003a00 con 0x7fd00c14f2c0 2023-10-26T01:33:24.188 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:24.182+0000 7fd011597640 1 -- 172.21.15.169:0/2448915286 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd00c070c70 con 0x7fd00c14f2c0 2023-10-26T01:33:24.189 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:24.183+0000 7fd011597640 1 -- 172.21.15.169:0/2448915286 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd00c0719b0 con 0x7fd00c14f2c0 2023-10-26T01:33:24.190 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:24.184+0000 7fd011597640 1 -- 172.21.15.169:0/2448915286 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd00c071190 con 0x7fd00c14f2c0 2023-10-26T01:33:24.190 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:24.185+0000 7fd003fff640 1 -- 172.21.15.169:0/2448915286 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fcff4003b60 con 0x7fd00c14f2c0 2023-10-26T01:33:24.190 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:24.185+0000 7fd003fff640 1 --2- 172.21.15.169:0/2448915286 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcfec06ae40 0x7fcfec06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:24.191 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:24.186+0000 7fd003fff640 1 -- 172.21.15.169:0/2448915286 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcff4088dd0 con 0x7fd00c14f2c0 2023-10-26T01:33:24.191 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:24.186+0000 7fd00affd640 1 --2- 172.21.15.169:0/2448915286 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcfec06ae40 0x7fcfec06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:24.191 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:24.187+0000 7fd00affd640 1 --2- 172.21.15.169:0/2448915286 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcfec06ae40 0x7fcfec06d300 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fcffc000f60 tx=0x7fcffc009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:24.191 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:24.190+0000 7fd003fff640 1 -- 172.21.15.169:0/2448915286 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd00c071190 con 0x7fd00c14f2c0 2023-10-26T01:33:24.398 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:24.396+0000 7fd011597640 1 -- 172.21.15.169:0/2448915286 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd00c071190 con 0x7fd00c14f2c0 2023-10-26T01:33:24.399 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:24.398+0000 7fd003fff640 1 -- 172.21.15.169:0/2448915286 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4925 (secure 0 0 0) 0x7fd00c071190 con 0x7fd00c14f2c0 2023-10-26T01:33:24.400 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:33:24.400 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4805,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/2294107634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2294107634},{"type":"v1","addr":"172.21.15.170:6836","nonce":2294107634}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":4987,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2208026662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2208026662},{"type":"v1","addr":"172.21.15.169:6833","nonce":2208026662}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":4993,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/2630365058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":2630365058},{"type":"v1","addr":"172.21.15.170:6837","nonce":2630365058}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T01:32:38.059721+0000","modified":"2023-10-26T01:32:41.428183+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5664},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5664":{"gid":5664,"name":"a","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6835/2816266753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":2816266753},{"type":"v1","addr":"172.21.15.169:6835","nonce":2816266753}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-10-26T01:33:24.400 INFO:teuthology.orchestra.run.smithi169.stderr:dumped fsmap epoch 23 2023-10-26T01:33:24.402 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:24.400+0000 7fd011597640 1 -- 172.21.15.169:0/2448915286 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcfec06ae40 msgr2=0x7fcfec06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:24.402 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:24.400+0000 7fd011597640 1 --2- 172.21.15.169:0/2448915286 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcfec06ae40 0x7fcfec06d300 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fcffc000f60 tx=0x7fcffc009040 comp rx=0 tx=0).stop 2023-10-26T01:33:24.403 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:24.400+0000 7fd011597640 1 -- 172.21.15.169:0/2448915286 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd00c14f2c0 msgr2=0x7fd00c071470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:24.403 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:24.400+0000 7fd011597640 1 --2- 172.21.15.169:0/2448915286 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd00c14f2c0 0x7fd00c071470 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fcff400c3c0 tx=0x7fcff400c880 comp rx=0 tx=0).stop 2023-10-26T01:33:24.403 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:24.400+0000 7fd011597640 1 -- 172.21.15.169:0/2448915286 shutdown_connections 2023-10-26T01:33:24.403 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:24.400+0000 7fd011597640 1 --2- 172.21.15.169:0/2448915286 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcfec06ae40 0x7fcfec06d300 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:24.404 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:24.400+0000 7fd011597640 1 --2- 172.21.15.169:0/2448915286 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd00c14f2c0 0x7fd00c071470 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:24.404 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:24.400+0000 7fd011597640 1 --2- 172.21.15.169:0/2448915286 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd00c14f6a0 0x7fd00c14d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:24.404 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:24.400+0000 7fd011597640 1 --2- 172.21.15.169:0/2448915286 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd00c0cde30 0x7fd00c06fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:24.404 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:24.400+0000 7fd011597640 1 -- 172.21.15.169:0/2448915286 >> 172.21.15.169:0/2448915286 conn(0x7fd00c069930 msgr2=0x7fd00c159240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:24.404 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:24.401+0000 7fd011597640 1 -- 172.21.15.169:0/2448915286 shutdown_connections 2023-10-26T01:33:24.404 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:24.401+0000 7fd011597640 1 -- 172.21.15.169:0/2448915286 wait complete. 2023-10-26T01:33:24.419 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:33:24.637 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:24.635+0000 7f69284e5640 1 -- 172.21.15.169:0/2516397577 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f69200ac4e0 msgr2=0x7f69200cd670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:24.638 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:24.635+0000 7f69284e5640 1 --2- 172.21.15.169:0/2516397577 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f69200ac4e0 0x7f69200cd670 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6910005fd0 tx=0x7f691002be10 comp rx=0 tx=0).stop 2023-10-26T01:33:24.638 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:24.635+0000 7f69284e5640 1 -- 172.21.15.169:0/2516397577 shutdown_connections 2023-10-26T01:33:24.638 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:24.635+0000 7f69284e5640 1 --2- 172.21.15.169:0/2516397577 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f69200ac4e0 0x7f69200cd670 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:24.638 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:24.635+0000 7f69284e5640 1 --2- 172.21.15.169:0/2516397577 >> v2:172.21.15.170:3300/0 conn(0x7f692015b610 0x7f692006ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:24.639 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:24.635+0000 7f69284e5640 1 -- 172.21.15.169:0/2516397577 >> 172.21.15.169:0/2516397577 conn(0x7f6920069930 msgr2=0x7f6920069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:24.639 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:24.635+0000 7f69284e5640 1 -- 172.21.15.169:0/2516397577 shutdown_connections 2023-10-26T01:33:24.639 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:24.635+0000 7f69284e5640 1 -- 172.21.15.169:0/2516397577 wait complete. 2023-10-26T01:33:24.641 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:24.637+0000 7f69284e5640 1 Processor -- start 2023-10-26T01:33:24.641 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:24.637+0000 7f69284e5640 1 -- start start 2023-10-26T01:33:24.641 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:24.638+0000 7f69284e5640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f69200ac4e0 0x7f69201f1f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:24.642 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:24.638+0000 7f69284e5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f69200ac940 0x7f69200c66b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:24.642 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:24.638+0000 7f69284e5640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f692015b610 0x7f692015e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:24.642 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:24.638+0000 7f69284e5640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69201f2560 con 0x7f692015b610 2023-10-26T01:33:24.642 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:24.638+0000 7f69284e5640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69201f2840 con 0x7f69200ac940 2023-10-26T01:33:24.642 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:24.638+0000 7f69284e5640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69201f2b20 con 0x7f69200ac4e0 2023-10-26T01:33:24.643 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:24.638+0000 7f6926a82640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f692015b610 0x7f692015e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:24.643 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:24.638+0000 7f6926a82640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f692015b610 0x7f692015e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:42654/0 (socket says 172.21.15.169:42654) 2023-10-26T01:33:24.643 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:24.638+0000 7f6926a82640 1 -- 172.21.15.169:0/3019907487 learned_addr learned my addr 172.21.15.169:0/3019907487 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:33:24.643 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:24.639+0000 7f6926281640 1 --2- 172.21.15.169:0/3019907487 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f69200ac4e0 0x7f69201f1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:24.644 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:24.639+0000 7f6925a80640 1 --2- 172.21.15.169:0/3019907487 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f69200ac940 0x7f69200c66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:24.644 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:24.639+0000 7f6926a82640 1 -- 172.21.15.169:0/3019907487 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f69200ac4e0 msgr2=0x7f69201f1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:24.644 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:24.639+0000 7f6926a82640 1 --2- 172.21.15.169:0/3019907487 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f69200ac4e0 0x7f69201f1f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:24.644 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:24.639+0000 7f6926a82640 1 -- 172.21.15.169:0/3019907487 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f69200ac940 msgr2=0x7f69200c66b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:24.644 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:24.639+0000 7f6926a82640 1 --2- 172.21.15.169:0/3019907487 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f69200ac940 0x7f69200c66b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:24.645 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:24.639+0000 7f6926a82640 1 -- 172.21.15.169:0/3019907487 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6910009560 con 0x7f692015b610 2023-10-26T01:33:24.645 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:24.639+0000 7f6926a82640 1 --2- 172.21.15.169:0/3019907487 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f692015b610 0x7f692015e380 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f691800c7a0 tx=0x7f691800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:24.645 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:24.639+0000 7f6916ffd640 1 -- 172.21.15.169:0/3019907487 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6918003b70 con 0x7f692015b610 2023-10-26T01:33:24.645 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:24.639+0000 7f6916ffd640 1 -- 172.21.15.169:0/3019907487 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6918003cf0 con 0x7f692015b610 2023-10-26T01:33:24.645 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:24.639+0000 7f6916ffd640 1 -- 172.21.15.169:0/3019907487 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f691800aab0 con 0x7f692015b610 2023-10-26T01:33:24.646 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:24.639+0000 7f69284e5640 1 -- 172.21.15.169:0/3019907487 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69201f2e00 con 0x7f692015b610 2023-10-26T01:33:24.646 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:24.640+0000 7f69284e5640 1 -- 172.21.15.169:0/3019907487 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69200c6bf0 con 0x7f692015b610 2023-10-26T01:33:24.647 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:24.641+0000 7f69284e5640 1 -- 172.21.15.169:0/3019907487 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f692004db10 con 0x7f692015b610 2023-10-26T01:33:24.648 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:24.642+0000 7f6916ffd640 1 -- 172.21.15.169:0/3019907487 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f691800ac10 con 0x7f692015b610 2023-10-26T01:33:24.648 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:24.643+0000 7f6916ffd640 1 --2- 172.21.15.169:0/3019907487 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f690406ae40 0x7f690406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:24.649 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:24.643+0000 7f6916ffd640 1 -- 172.21.15.169:0/3019907487 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f69180896d0 con 0x7f692015b610 2023-10-26T01:33:24.649 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:24.644+0000 7f6926281640 1 --2- 172.21.15.169:0/3019907487 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f690406ae40 0x7f690406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:24.649 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:24.645+0000 7f6926281640 1 --2- 172.21.15.169:0/3019907487 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f690406ae40 0x7f690406d300 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f69201f5480 tx=0x7f691c005f90 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:24.650 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:24.647+0000 7f6916ffd640 1 -- 172.21.15.169:0/3019907487 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f69180546e0 con 0x7f692015b610 2023-10-26T01:33:24.840 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:24.838+0000 7f69284e5640 1 -- 172.21.15.169:0/3019907487 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f692006c990 con 0x7f692015b610 2023-10-26T01:33:24.843 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:24.841+0000 7f6916ffd640 1 -- 172.21.15.169:0/3019907487 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+16561 (secure 0 0 0) 0x7f6918057d00 con 0x7f692015b610 2023-10-26T01:33:24.843 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:33:24.843 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":54,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:33:10.190714+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-26T01:32:36.288217+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-26T01:32:37.059775+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:33:24.845 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:24.843+0000 7f69284e5640 1 -- 172.21.15.169:0/3019907487 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f690406ae40 msgr2=0x7f690406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:24.846 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:24.843+0000 7f69284e5640 1 --2- 172.21.15.169:0/3019907487 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f690406ae40 0x7f690406d300 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f69201f5480 tx=0x7f691c005f90 comp rx=0 tx=0).stop 2023-10-26T01:33:24.846 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:24.843+0000 7f69284e5640 1 -- 172.21.15.169:0/3019907487 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f692015b610 msgr2=0x7f692015e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:24.846 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:24.843+0000 7f69284e5640 1 --2- 172.21.15.169:0/3019907487 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f692015b610 0x7f692015e380 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f691800c7a0 tx=0x7f691800cc60 comp rx=0 tx=0).stop 2023-10-26T01:33:24.846 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:24.843+0000 7f69284e5640 1 -- 172.21.15.169:0/3019907487 shutdown_connections 2023-10-26T01:33:24.846 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:24.844+0000 7f69284e5640 1 --2- 172.21.15.169:0/3019907487 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f690406ae40 0x7f690406d300 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:24.847 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:24.844+0000 7f69284e5640 1 --2- 172.21.15.169:0/3019907487 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f692015b610 0x7f692015e380 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:24.847 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:24.844+0000 7f69284e5640 1 --2- 172.21.15.169:0/3019907487 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f69200ac940 0x7f69200c66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:24.847 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:24.844+0000 7f69284e5640 1 --2- 172.21.15.169:0/3019907487 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f69200ac4e0 0x7f69201f1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:24.847 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:24.844+0000 7f69284e5640 1 -- 172.21.15.169:0/3019907487 >> 172.21.15.169:0/3019907487 conn(0x7f6920069930 msgr2=0x7f69201598b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:24.847 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:24.844+0000 7f69284e5640 1 -- 172.21.15.169:0/3019907487 shutdown_connections 2023-10-26T01:33:24.848 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:24.844+0000 7f69284e5640 1 -- 172.21.15.169:0/3019907487 wait complete. 2023-10-26T01:33:24.864 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-26T01:33:24.864 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T01:33:25.085 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.083+0000 7f2203244640 1 -- 172.21.15.169:0/3596741651 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f21fc0ac930 msgr2=0x7f21fc0cd6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:25.085 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.083+0000 7f2203244640 1 --2- 172.21.15.169:0/3596741651 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f21fc0ac930 0x7f21fc0cd6f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f21e8005fd0 tx=0x7f21e802bd90 comp rx=0 tx=0).stop 2023-10-26T01:33:25.085 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.083+0000 7f2203244640 1 -- 172.21.15.169:0/3596741651 shutdown_connections 2023-10-26T01:33:25.086 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.083+0000 7f2203244640 1 --2- 172.21.15.169:0/3596741651 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f21fc0ac930 0x7f21fc0cd6f0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:25.086 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.083+0000 7f2203244640 1 --2- 172.21.15.169:0/3596741651 >> v2:172.21.15.170:3300/0 conn(0x7f21fc1592d0 0x7f21fc0aa3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:25.086 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.083+0000 7f2203244640 1 -- 172.21.15.169:0/3596741651 >> 172.21.15.169:0/3596741651 conn(0x7f21fc069930 msgr2=0x7f21fc069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:25.086 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.083+0000 7f2203244640 1 -- 172.21.15.169:0/3596741651 shutdown_connections 2023-10-26T01:33:25.087 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.083+0000 7f2203244640 1 -- 172.21.15.169:0/3596741651 wait complete. 2023-10-26T01:33:25.087 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.085+0000 7f2203244640 1 Processor -- start 2023-10-26T01:33:25.087 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.085+0000 7f2203244640 1 -- start start 2023-10-26T01:33:25.090 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.086+0000 7f2203244640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21fc0ac930 0x7f21fc1f07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:25.090 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.087+0000 7f2203244640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f21fc0cdc30 0x7f21fc070840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:25.091 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.087+0000 7f2203244640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f21fc1592d0 0x7f21fc150e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:25.093 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.087+0000 7f2203244640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21fc070d80 con 0x7f21fc0ac930 2023-10-26T01:33:25.093 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.087+0000 7f2203244640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21fc06ed30 con 0x7f21fc0cdc30 2023-10-26T01:33:25.094 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.087+0000 7f2203244640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21fc06ef80 con 0x7f21fc1592d0 2023-10-26T01:33:25.094 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.087+0000 7f2200fe0640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21fc0ac930 0x7f21fc1f07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:25.094 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.087+0000 7f2200fe0640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21fc0ac930 0x7f21fc1f07b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:42670/0 (socket says 172.21.15.169:42670) 2023-10-26T01:33:25.094 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.087+0000 7f2200fe0640 1 -- 172.21.15.169:0/1170249979 learned_addr learned my addr 172.21.15.169:0/1170249979 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:33:25.094 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.087+0000 7f21fbfff640 1 --2- 172.21.15.169:0/1170249979 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f21fc0cdc30 0x7f21fc070840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:25.095 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.087+0000 7f22017e1640 1 --2- 172.21.15.169:0/1170249979 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f21fc1592d0 0x7f21fc150e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:25.095 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.087+0000 7f2200fe0640 1 -- 172.21.15.169:0/1170249979 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f21fc1592d0 msgr2=0x7f21fc150e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:25.095 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.087+0000 7f2200fe0640 1 --2- 172.21.15.169:0/1170249979 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f21fc1592d0 0x7f21fc150e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:25.095 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.087+0000 7f2200fe0640 1 -- 172.21.15.169:0/1170249979 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f21fc0cdc30 msgr2=0x7f21fc070840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:25.095 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.087+0000 7f2200fe0640 1 --2- 172.21.15.169:0/1170249979 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f21fc0cdc30 0x7f21fc070840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:25.096 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.087+0000 7f2200fe0640 1 -- 172.21.15.169:0/1170249979 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21e8009560 con 0x7f21fc0ac930 2023-10-26T01:33:25.096 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.087+0000 7f21fbfff640 1 --2- 172.21.15.169:0/1170249979 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f21fc0cdc30 0x7f21fc070840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:33:25.096 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.087+0000 7f22017e1640 1 --2- 172.21.15.169:0/1170249979 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f21fc1592d0 0x7f21fc150e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:33:25.096 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.088+0000 7f2200fe0640 1 --2- 172.21.15.169:0/1170249979 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21fc0ac930 0x7f21fc1f07b0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f21f000e9f0 tx=0x7f21f000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:25.097 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.088+0000 7f21f97fa640 1 -- 172.21.15.169:0/1170249979 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f0009700 con 0x7f21fc0ac930 2023-10-26T01:33:25.097 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.088+0000 7f21f97fa640 1 -- 172.21.15.169:0/1170249979 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f21f000ce70 con 0x7f21fc0ac930 2023-10-26T01:33:25.097 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.088+0000 7f21f97fa640 1 -- 172.21.15.169:0/1170249979 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f0010670 con 0x7f21fc0ac930 2023-10-26T01:33:25.097 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.088+0000 7f2203244640 1 -- 172.21.15.169:0/1170249979 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21fc06f260 con 0x7f21fc0ac930 2023-10-26T01:33:25.097 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.088+0000 7f2203244640 1 -- 172.21.15.169:0/1170249979 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21fc06f820 con 0x7f21fc0ac930 2023-10-26T01:33:25.098 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.089+0000 7f2203244640 1 -- 172.21.15.169:0/1170249979 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21fc06f9e0 con 0x7f21fc0ac930 2023-10-26T01:33:25.099 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.090+0000 7f21f97fa640 1 -- 172.21.15.169:0/1170249979 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f21f00107d0 con 0x7f21fc0ac930 2023-10-26T01:33:25.099 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.091+0000 7f21f97fa640 1 --2- 172.21.15.169:0/1170249979 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f21e006ae40 0x7f21e006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:25.099 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.091+0000 7f21f97fa640 1 -- 172.21.15.169:0/1170249979 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f21f0014070 con 0x7f21fc0ac930 2023-10-26T01:33:25.099 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.092+0000 7f21fbfff640 1 --2- 172.21.15.169:0/1170249979 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f21e006ae40 0x7f21e006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:25.099 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.093+0000 7f21fbfff640 1 --2- 172.21.15.169:0/1170249979 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f21e006ae40 0x7f21e006d300 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f21e8035000 tx=0x7f21e8004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:25.100 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.095+0000 7f21f97fa640 1 -- 172.21.15.169:0/1170249979 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f21f0051ca0 con 0x7f21fc0ac930 2023-10-26T01:33:25.324 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.323+0000 7f2203244640 1 -- 172.21.15.169:0/1170249979 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f21fc04db10 con 0x7f21fc0ac930 2023-10-26T01:33:25.325 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.324+0000 7f21f97fa640 1 -- 172.21.15.169:0/1170249979 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4925 (secure 0 0 0) 0x7f21f0058e40 con 0x7f21fc0ac930 2023-10-26T01:33:25.326 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:33:25.326 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4805,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/2294107634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2294107634},{"type":"v1","addr":"172.21.15.170:6836","nonce":2294107634}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":4987,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2208026662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2208026662},{"type":"v1","addr":"172.21.15.169:6833","nonce":2208026662}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":4993,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/2630365058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":2630365058},{"type":"v1","addr":"172.21.15.170:6837","nonce":2630365058}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T01:32:38.059721+0000","modified":"2023-10-26T01:32:41.428183+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5664},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5664":{"gid":5664,"name":"a","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6835/2816266753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":2816266753},{"type":"v1","addr":"172.21.15.169:6835","nonce":2816266753}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-10-26T01:33:25.327 INFO:teuthology.orchestra.run.smithi169.stderr:dumped fsmap epoch 23 2023-10-26T01:33:25.328 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.326+0000 7f2203244640 1 -- 172.21.15.169:0/1170249979 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f21e006ae40 msgr2=0x7f21e006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:25.329 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.326+0000 7f2203244640 1 --2- 172.21.15.169:0/1170249979 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f21e006ae40 0x7f21e006d300 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f21e8035000 tx=0x7f21e8004730 comp rx=0 tx=0).stop 2023-10-26T01:33:25.329 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.326+0000 7f2203244640 1 -- 172.21.15.169:0/1170249979 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21fc0ac930 msgr2=0x7f21fc1f07b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:25.329 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.326+0000 7f2203244640 1 --2- 172.21.15.169:0/1170249979 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21fc0ac930 0x7f21fc1f07b0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f21f000e9f0 tx=0x7f21f000eeb0 comp rx=0 tx=0).stop 2023-10-26T01:33:25.329 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.326+0000 7f2203244640 1 -- 172.21.15.169:0/1170249979 shutdown_connections 2023-10-26T01:33:25.329 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.326+0000 7f2203244640 1 --2- 172.21.15.169:0/1170249979 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f21e006ae40 0x7f21e006d300 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:25.330 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.326+0000 7f2203244640 1 --2- 172.21.15.169:0/1170249979 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f21fc1592d0 0x7f21fc150e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:25.330 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.326+0000 7f2203244640 1 --2- 172.21.15.169:0/1170249979 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f21fc0cdc30 0x7f21fc070840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:25.330 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.326+0000 7f2203244640 1 --2- 172.21.15.169:0/1170249979 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21fc0ac930 0x7f21fc1f07b0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:25.330 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.326+0000 7f2203244640 1 -- 172.21.15.169:0/1170249979 >> 172.21.15.169:0/1170249979 conn(0x7f21fc069930 msgr2=0x7f21fc0afaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:25.330 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.327+0000 7f2203244640 1 -- 172.21.15.169:0/1170249979 shutdown_connections 2023-10-26T01:33:25.331 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.327+0000 7f2203244640 1 -- 172.21.15.169:0/1170249979 wait complete. 2023-10-26T01:33:25.348 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T01:33:25.562 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.560+0000 7f84b116c640 1 -- 172.21.15.169:0/1311894291 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84ac15b610 msgr2=0x7f84ac06ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:25.563 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.560+0000 7f84b116c640 1 --2- 172.21.15.169:0/1311894291 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84ac15b610 0x7f84ac06ed30 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f849c005fd0 tx=0x7f849c02be20 comp rx=0 tx=0).stop 2023-10-26T01:33:25.563 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.560+0000 7f84b116c640 1 -- 172.21.15.169:0/1311894291 shutdown_connections 2023-10-26T01:33:25.563 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.560+0000 7f84b116c640 1 --2- 172.21.15.169:0/1311894291 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f84ac0ac4e0 0x7f84ac0cd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:25.564 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.560+0000 7f84b116c640 1 --2- 172.21.15.169:0/1311894291 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84ac15b610 0x7f84ac06ed30 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:25.564 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.560+0000 7f84b116c640 1 -- 172.21.15.169:0/1311894291 >> 172.21.15.169:0/1311894291 conn(0x7f84ac069930 msgr2=0x7f84ac069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:25.564 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.561+0000 7f84b116c640 1 -- 172.21.15.169:0/1311894291 shutdown_connections 2023-10-26T01:33:25.564 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.561+0000 7f84b116c640 1 -- 172.21.15.169:0/1311894291 wait complete. 2023-10-26T01:33:25.566 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.562+0000 7f84b116c640 1 Processor -- start 2023-10-26T01:33:25.567 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.562+0000 7f84b116c640 1 -- start start 2023-10-26T01:33:25.567 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.563+0000 7f84b116c640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f84ac0ac4e0 0x7f84ac1f1db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:25.567 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.563+0000 7f84b116c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84ac0ac940 0x7f84ac15cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:25.567 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.563+0000 7f84b116c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f84ac15b610 0x7f84ac097c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:25.567 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.563+0000 7f84b116c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84ac0701e0 con 0x7f84ac0ac940 2023-10-26T01:33:25.568 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.563+0000 7f84b116c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84ac09b320 con 0x7f84ac15b610 2023-10-26T01:33:25.568 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.563+0000 7f84b116c640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84ac06f840 con 0x7f84ac0ac4e0 2023-10-26T01:33:25.568 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.564+0000 7f84aa59c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84ac0ac940 0x7f84ac15cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:25.568 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.564+0000 7f84aa59c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84ac0ac940 0x7f84ac15cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:42684/0 (socket says 172.21.15.169:42684) 2023-10-26T01:33:25.569 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.564+0000 7f84aa59c640 1 -- 172.21.15.169:0/1276817980 learned_addr learned my addr 172.21.15.169:0/1276817980 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:33:25.569 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.564+0000 7f84aad9d640 1 --2- 172.21.15.169:0/1276817980 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f84ac0ac4e0 0x7f84ac1f1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:25.569 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.564+0000 7f84ab59e640 1 --2- 172.21.15.169:0/1276817980 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f84ac15b610 0x7f84ac097c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:25.569 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.564+0000 7f84aa59c640 1 -- 172.21.15.169:0/1276817980 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f84ac0ac4e0 msgr2=0x7f84ac1f1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:25.569 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.564+0000 7f84aa59c640 1 --2- 172.21.15.169:0/1276817980 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f84ac0ac4e0 0x7f84ac1f1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:25.569 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.564+0000 7f84aa59c640 1 -- 172.21.15.169:0/1276817980 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f84ac15b610 msgr2=0x7f84ac097c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:25.570 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.564+0000 7f84aa59c640 1 --2- 172.21.15.169:0/1276817980 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f84ac15b610 0x7f84ac097c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:25.570 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.564+0000 7f84aa59c640 1 -- 172.21.15.169:0/1276817980 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f849c009560 con 0x7f84ac0ac940 2023-10-26T01:33:25.570 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.564+0000 7f84aad9d640 1 --2- 172.21.15.169:0/1276817980 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f84ac0ac4e0 0x7f84ac1f1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:33:25.570 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.564+0000 7f84ab59e640 1 --2- 172.21.15.169:0/1276817980 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f84ac15b610 0x7f84ac097c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:33:25.570 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.565+0000 7f84aa59c640 1 --2- 172.21.15.169:0/1276817980 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84ac0ac940 0x7f84ac15cc90 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f84a000b4a0 tx=0x7f84a000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:25.571 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.565+0000 7f84937fe640 1 -- 172.21.15.169:0/1276817980 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a0010610 con 0x7f84ac0ac940 2023-10-26T01:33:25.571 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.565+0000 7f84937fe640 1 -- 172.21.15.169:0/1276817980 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f84a0010860 con 0x7f84ac0ac940 2023-10-26T01:33:25.571 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.565+0000 7f84b116c640 1 -- 172.21.15.169:0/1276817980 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84ac06fb20 con 0x7f84ac0ac940 2023-10-26T01:33:25.571 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.565+0000 7f84937fe640 1 -- 172.21.15.169:0/1276817980 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a0010c70 con 0x7f84ac0ac940 2023-10-26T01:33:25.571 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.565+0000 7f84b116c640 1 -- 172.21.15.169:0/1276817980 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84ac1f22f0 con 0x7f84ac0ac940 2023-10-26T01:33:25.572 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.566+0000 7f84b116c640 1 -- 172.21.15.169:0/1276817980 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84ac04db10 con 0x7f84ac0ac940 2023-10-26T01:33:25.573 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.567+0000 7f84937fe640 1 -- 172.21.15.169:0/1276817980 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f84a0017360 con 0x7f84ac0ac940 2023-10-26T01:33:25.573 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.568+0000 7f84937fe640 1 --2- 172.21.15.169:0/1276817980 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f849806ae40 0x7f849806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:25.573 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.568+0000 7f84937fe640 1 -- 172.21.15.169:0/1276817980 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f84a0089500 con 0x7f84ac0ac940 2023-10-26T01:33:25.573 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.569+0000 7f84aad9d640 1 --2- 172.21.15.169:0/1276817980 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f849806ae40 0x7f849806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:25.573 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.570+0000 7f84aad9d640 1 --2- 172.21.15.169:0/1276817980 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f849806ae40 0x7f849806d300 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f849c035000 tx=0x7f849c004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:25.574 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.572+0000 7f84937fe640 1 -- 172.21.15.169:0/1276817980 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f84a00545a0 con 0x7f84ac0ac940 2023-10-26T01:33:25.796 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.795+0000 7f84b116c640 1 -- 172.21.15.169:0/1276817980 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f84ac06c990 con 0x7f84ac0ac940 2023-10-26T01:33:25.797 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.795+0000 7f84937fe640 1 -- 172.21.15.169:0/1276817980 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4925 (secure 0 0 0) 0x7f84a0057bc0 con 0x7f84ac0ac940 2023-10-26T01:33:25.798 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:33:25.798 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4805,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/2294107634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2294107634},{"type":"v1","addr":"172.21.15.170:6836","nonce":2294107634}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":4987,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2208026662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2208026662},{"type":"v1","addr":"172.21.15.169:6833","nonce":2208026662}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":4993,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/2630365058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":2630365058},{"type":"v1","addr":"172.21.15.170:6837","nonce":2630365058}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T01:32:38.059721+0000","modified":"2023-10-26T01:32:41.428183+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5664},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5664":{"gid":5664,"name":"a","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6835/2816266753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":2816266753},{"type":"v1","addr":"172.21.15.169:6835","nonce":2816266753}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-10-26T01:33:25.798 INFO:teuthology.orchestra.run.smithi169.stderr:dumped fsmap epoch 23 2023-10-26T01:33:25.799 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.798+0000 7f84b116c640 1 -- 172.21.15.169:0/1276817980 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f849806ae40 msgr2=0x7f849806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:25.800 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.798+0000 7f84b116c640 1 --2- 172.21.15.169:0/1276817980 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f849806ae40 0x7f849806d300 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f849c035000 tx=0x7f849c004730 comp rx=0 tx=0).stop 2023-10-26T01:33:25.800 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.798+0000 7f84b116c640 1 -- 172.21.15.169:0/1276817980 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84ac0ac940 msgr2=0x7f84ac15cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:25.800 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.798+0000 7f84b116c640 1 --2- 172.21.15.169:0/1276817980 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84ac0ac940 0x7f84ac15cc90 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f84a000b4a0 tx=0x7f84a000b960 comp rx=0 tx=0).stop 2023-10-26T01:33:25.801 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.798+0000 7f84b116c640 1 -- 172.21.15.169:0/1276817980 shutdown_connections 2023-10-26T01:33:25.801 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.798+0000 7f84b116c640 1 --2- 172.21.15.169:0/1276817980 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f849806ae40 0x7f849806d300 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:25.801 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.798+0000 7f84b116c640 1 --2- 172.21.15.169:0/1276817980 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84ac0ac940 0x7f84ac15cc90 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:25.801 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.798+0000 7f84b116c640 1 --2- 172.21.15.169:0/1276817980 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f84ac15b610 0x7f84ac097c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:25.801 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.798+0000 7f84b116c640 1 --2- 172.21.15.169:0/1276817980 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f84ac0ac4e0 0x7f84ac1f1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:25.802 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.798+0000 7f84b116c640 1 -- 172.21.15.169:0/1276817980 >> 172.21.15.169:0/1276817980 conn(0x7f84ac069930 msgr2=0x7f84ac06be50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:25.802 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.798+0000 7f84b116c640 1 -- 172.21.15.169:0/1276817980 shutdown_connections 2023-10-26T01:33:25.802 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:25.798+0000 7f84b116c640 1 -- 172.21.15.169:0/1276817980 wait complete. 2023-10-26T01:33:25.818 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:33:26.035 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.033+0000 7fa2148a8640 1 -- 172.21.15.169:0/1278701317 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa21006a990 msgr2=0x7fa2100c96f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:26.036 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.033+0000 7fa2148a8640 1 --2- 172.21.15.169:0/1278701317 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa21006a990 0x7fa2100c96f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa1f8007350 tx=0x7fa1f802c9d0 comp rx=0 tx=0).stop 2023-10-26T01:33:26.036 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.034+0000 7fa2148a8640 1 -- 172.21.15.169:0/1278701317 shutdown_connections 2023-10-26T01:33:26.036 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.034+0000 7fa2148a8640 1 --2- 172.21.15.169:0/1278701317 >> v2:172.21.15.170:3300/0 conn(0x7fa2100c9c30 0x7fa2100cb390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:26.037 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.034+0000 7fa2148a8640 1 --2- 172.21.15.169:0/1278701317 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa21006a990 0x7fa2100c96f0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:26.037 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.034+0000 7fa2148a8640 1 -- 172.21.15.169:0/1278701317 >> 172.21.15.169:0/1278701317 conn(0x7fa2100aedd0 msgr2=0x7fa2100af1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:26.037 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.034+0000 7fa2148a8640 1 -- 172.21.15.169:0/1278701317 shutdown_connections 2023-10-26T01:33:26.037 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.034+0000 7fa2148a8640 1 -- 172.21.15.169:0/1278701317 wait complete. 2023-10-26T01:33:26.038 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.036+0000 7fa2148a8640 1 Processor -- start 2023-10-26T01:33:26.038 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.036+0000 7fa2148a8640 1 -- start start 2023-10-26T01:33:26.039 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.037+0000 7fa2148a8640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa21006a990 0x7fa2101f18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:26.039 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.037+0000 7fa2148a8640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2100c7ea0 0x7fa210147990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:26.039 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.037+0000 7fa2148a8640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa2100c9c30 0x7fa2100b23d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:26.040 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.037+0000 7fa2148a8640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa210154480 con 0x7fa2100c7ea0 2023-10-26T01:33:26.040 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.037+0000 7fa2148a8640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa210147ed0 con 0x7fa21006a990 2023-10-26T01:33:26.040 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.037+0000 7fa2148a8640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2100b2910 con 0x7fa2100c9c30 2023-10-26T01:33:26.040 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.037+0000 7fa207fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2100c7ea0 0x7fa210147990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:26.040 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.038+0000 7fa207fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2100c7ea0 0x7fa210147990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:42696/0 (socket says 172.21.15.169:42696) 2023-10-26T01:33:26.041 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.038+0000 7fa207fff640 1 -- 172.21.15.169:0/2816080727 learned_addr learned my addr 172.21.15.169:0/2816080727 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:33:26.042 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.038+0000 7fa20f59e640 1 --2- 172.21.15.169:0/2816080727 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa2100c9c30 0x7fa2100b23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:26.042 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.038+0000 7fa20ed9d640 1 --2- 172.21.15.169:0/2816080727 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa21006a990 0x7fa2101f18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:26.042 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.038+0000 7fa207fff640 1 -- 172.21.15.169:0/2816080727 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa2100c9c30 msgr2=0x7fa2100b23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:26.043 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.038+0000 7fa207fff640 1 --2- 172.21.15.169:0/2816080727 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa2100c9c30 0x7fa2100b23d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:26.043 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.038+0000 7fa207fff640 1 -- 172.21.15.169:0/2816080727 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa21006a990 msgr2=0x7fa2101f18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:26.043 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.038+0000 7fa207fff640 1 --2- 172.21.15.169:0/2816080727 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa21006a990 0x7fa2101f18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:26.043 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.038+0000 7fa207fff640 1 -- 172.21.15.169:0/2816080727 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa208009840 con 0x7fa2100c7ea0 2023-10-26T01:33:26.043 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.038+0000 7fa20f59e640 1 --2- 172.21.15.169:0/2816080727 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa2100c9c30 0x7fa2100b23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:33:26.044 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.038+0000 7fa207fff640 1 --2- 172.21.15.169:0/2816080727 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2100c7ea0 0x7fa210147990 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fa1f8007470 tx=0x7fa1f800a740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:26.044 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.039+0000 7fa2077fe640 1 -- 172.21.15.169:0/2816080727 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f800a0d0 con 0x7fa2100c7ea0 2023-10-26T01:33:26.044 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.039+0000 7fa2077fe640 1 -- 172.21.15.169:0/2816080727 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa1f803f070 con 0x7fa2100c7ea0 2023-10-26T01:33:26.044 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.039+0000 7fa2148a8640 1 -- 172.21.15.169:0/2816080727 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1f800ae30 con 0x7fa2100c7ea0 2023-10-26T01:33:26.044 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.039+0000 7fa2077fe640 1 -- 172.21.15.169:0/2816080727 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f8007cb0 con 0x7fa2100c7ea0 2023-10-26T01:33:26.044 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.039+0000 7fa2148a8640 1 -- 172.21.15.169:0/2816080727 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2100aaeb0 con 0x7fa2100c7ea0 2023-10-26T01:33:26.045 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.040+0000 7fa2148a8640 1 -- 172.21.15.169:0/2816080727 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa21006ca60 con 0x7fa2100c7ea0 2023-10-26T01:33:26.045 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.041+0000 7fa2077fe640 1 -- 172.21.15.169:0/2816080727 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fa1f8041030 con 0x7fa2100c7ea0 2023-10-26T01:33:26.046 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.042+0000 7fa2077fe640 1 --2- 172.21.15.169:0/2816080727 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa1f006ae40 0x7fa1f006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:26.046 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.042+0000 7fa2077fe640 1 -- 172.21.15.169:0/2816080727 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa1f80ab960 con 0x7fa2100c7ea0 2023-10-26T01:33:26.046 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.042+0000 7fa20ed9d640 1 --2- 172.21.15.169:0/2816080727 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa1f006ae40 0x7fa1f006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:26.046 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.044+0000 7fa20ed9d640 1 --2- 172.21.15.169:0/2816080727 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa1f006ae40 0x7fa1f006d300 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fa200001370 tx=0x7fa200009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:26.047 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.046+0000 7fa2077fe640 1 -- 172.21.15.169:0/2816080727 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa1f8076a10 con 0x7fa2100c7ea0 2023-10-26T01:33:26.231 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.229+0000 7fa2148a8640 1 -- 172.21.15.169:0/2816080727 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa210073c10 con 0x7fa2100c7ea0 2023-10-26T01:33:26.232 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.231+0000 7fa2077fe640 1 -- 172.21.15.169:0/2816080727 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+16561 (secure 0 0 0) 0x7fa1f8035020 con 0x7fa2100c7ea0 2023-10-26T01:33:26.233 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:33:26.233 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":54,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:33:10.190714+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-26T01:32:36.288217+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-26T01:32:37.059775+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:33:26.238 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.235+0000 7fa2148a8640 1 -- 172.21.15.169:0/2816080727 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa1f006ae40 msgr2=0x7fa1f006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:26.238 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.235+0000 7fa2148a8640 1 --2- 172.21.15.169:0/2816080727 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa1f006ae40 0x7fa1f006d300 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fa200001370 tx=0x7fa200009040 comp rx=0 tx=0).stop 2023-10-26T01:33:26.238 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.235+0000 7fa2148a8640 1 -- 172.21.15.169:0/2816080727 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2100c7ea0 msgr2=0x7fa210147990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:26.238 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.235+0000 7fa2148a8640 1 --2- 172.21.15.169:0/2816080727 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2100c7ea0 0x7fa210147990 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fa1f8007470 tx=0x7fa1f800a740 comp rx=0 tx=0).stop 2023-10-26T01:33:26.239 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.236+0000 7fa2148a8640 1 -- 172.21.15.169:0/2816080727 shutdown_connections 2023-10-26T01:33:26.239 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.236+0000 7fa2148a8640 1 --2- 172.21.15.169:0/2816080727 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa1f006ae40 0x7fa1f006d300 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:26.239 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.236+0000 7fa2148a8640 1 --2- 172.21.15.169:0/2816080727 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2100c7ea0 0x7fa210147990 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:26.239 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.236+0000 7fa2148a8640 1 --2- 172.21.15.169:0/2816080727 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa2100c9c30 0x7fa2100b23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:26.239 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.236+0000 7fa2148a8640 1 --2- 172.21.15.169:0/2816080727 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa21006a990 0x7fa2101f18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:26.240 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.236+0000 7fa2148a8640 1 -- 172.21.15.169:0/2816080727 >> 172.21.15.169:0/2816080727 conn(0x7fa2100aedd0 msgr2=0x7fa210155750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:26.240 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.236+0000 7fa2148a8640 1 -- 172.21.15.169:0/2816080727 shutdown_connections 2023-10-26T01:33:26.240 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.236+0000 7fa2148a8640 1 -- 172.21.15.169:0/2816080727 wait complete. 2023-10-26T01:33:26.257 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-26T01:33:26.468 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.467+0000 7f5601ff0640 1 -- 172.21.15.169:0/3872882346 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55fc0cbff0 msgr2=0x7f55fc15d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:26.469 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.467+0000 7f5601ff0640 1 --2- 172.21.15.169:0/3872882346 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55fc0cbff0 0x7f55fc15d990 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f55f4005fd0 tx=0x7f55f402c0e0 comp rx=0 tx=0).stop 2023-10-26T01:33:26.469 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.467+0000 7f5601ff0640 1 -- 172.21.15.169:0/3872882346 shutdown_connections 2023-10-26T01:33:26.469 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.467+0000 7f5601ff0640 1 --2- 172.21.15.169:0/3872882346 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55fc0cbff0 0x7f55fc15d990 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:26.469 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.467+0000 7f5601ff0640 1 -- 172.21.15.169:0/3872882346 >> 172.21.15.169:0/3872882346 conn(0x7f55fc0af8a0 msgr2=0x7f55fc0afc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:26.470 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.467+0000 7f5601ff0640 1 -- 172.21.15.169:0/3872882346 shutdown_connections 2023-10-26T01:33:26.470 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.467+0000 7f5601ff0640 1 -- 172.21.15.169:0/3872882346 wait complete. 2023-10-26T01:33:26.470 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.468+0000 7f5601ff0640 1 Processor -- start 2023-10-26T01:33:26.471 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.469+0000 7f5601ff0640 1 -- start start 2023-10-26T01:33:26.472 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.469+0000 7f5601ff0640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55fc0713d0 0x7f55fc1480d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:26.473 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.469+0000 7f5601ff0640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f55fc0cbff0 0x7f55fc1f18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:26.473 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.470+0000 7f5601ff0640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f55fc15ca70 0x7f55fc15b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:26.473 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.470+0000 7f5601ff0640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55fc151e10 con 0x7f55fc0713d0 2023-10-26T01:33:26.473 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.470+0000 7f5601ff0640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55fc1f1e00 con 0x7f55fc15ca70 2023-10-26T01:33:26.473 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.470+0000 7f5601ff0640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55fc1511d0 con 0x7f55fc0cbff0 2023-10-26T01:33:26.474 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.470+0000 7f55fb7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55fc0713d0 0x7f55fc1480d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:26.474 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.470+0000 7f55fb7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55fc0713d0 0x7f55fc1480d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:42714/0 (socket says 172.21.15.169:42714) 2023-10-26T01:33:26.474 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.470+0000 7f55fb7fe640 1 -- 172.21.15.169:0/1681696597 learned_addr learned my addr 172.21.15.169:0/1681696597 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:33:26.474 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.470+0000 7f55fb7fe640 1 -- 172.21.15.169:0/1681696597 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f55fc0cbff0 msgr2=0x7f55fc1f18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:26.474 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.470+0000 7f55fb7fe640 1 --2- 172.21.15.169:0/1681696597 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f55fc0cbff0 0x7f55fc1f18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:26.475 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.470+0000 7f55fb7fe640 1 -- 172.21.15.169:0/1681696597 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f55fc15ca70 msgr2=0x7f55fc15b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:26.475 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.470+0000 7f55fb7fe640 1 --2- 172.21.15.169:0/1681696597 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f55fc15ca70 0x7f55fc15b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:26.475 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.470+0000 7f55fb7fe640 1 -- 172.21.15.169:0/1681696597 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55f4009840 con 0x7f55fc0713d0 2023-10-26T01:33:26.475 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.470+0000 7f55fb7fe640 1 --2- 172.21.15.169:0/1681696597 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55fc0713d0 0x7f55fc1480d0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f55ec00c3c0 tx=0x7f55ec00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:26.475 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.471+0000 7f55f8ff9640 1 -- 172.21.15.169:0/1681696597 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55ec004290 con 0x7f55fc0713d0 2023-10-26T01:33:26.476 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.471+0000 7f55f8ff9640 1 -- 172.21.15.169:0/1681696597 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f55ec0035f0 con 0x7f55fc0713d0 2023-10-26T01:33:26.476 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.471+0000 7f5601ff0640 1 -- 172.21.15.169:0/1681696597 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55fc15bed0 con 0x7f55fc0713d0 2023-10-26T01:33:26.476 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.471+0000 7f55f8ff9640 1 -- 172.21.15.169:0/1681696597 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55ec004500 con 0x7f55fc0713d0 2023-10-26T01:33:26.476 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.471+0000 7f5601ff0640 1 -- 172.21.15.169:0/1681696597 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55fc151750 con 0x7f55fc0713d0 2023-10-26T01:33:26.477 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.472+0000 7f5601ff0640 1 -- 172.21.15.169:0/1681696597 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55fc04db10 con 0x7f55fc0713d0 2023-10-26T01:33:26.477 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.473+0000 7f55f8ff9640 1 -- 172.21.15.169:0/1681696597 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f55ec003a00 con 0x7f55fc0713d0 2023-10-26T01:33:26.477 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.474+0000 7f55f8ff9640 1 --2- 172.21.15.169:0/1681696597 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f55dc06ae40 0x7f55dc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:26.478 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.474+0000 7f55f8ff9640 1 -- 172.21.15.169:0/1681696597 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f55ec088de0 con 0x7f55fc0713d0 2023-10-26T01:33:26.478 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.475+0000 7f55f2ffd640 1 --2- 172.21.15.169:0/1681696597 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f55dc06ae40 0x7f55dc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:26.478 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.476+0000 7f55f2ffd640 1 --2- 172.21.15.169:0/1681696597 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f55dc06ae40 0x7f55dc06d300 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f55e4003cc0 tx=0x7f55e40079b0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:26.480 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.479+0000 7f55f8ff9640 1 -- 172.21.15.169:0/1681696597 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f55ec0579e0 con 0x7f55fc0713d0 2023-10-26T01:33:26.695 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.693+0000 7f5601ff0640 1 -- 172.21.15.169:0/1681696597 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f55fc0b00f0 con 0x7f55fc0713d0 2023-10-26T01:33:26.695 INFO:tasks.ceph.mon.a.smithi169.stderr:2023-10-26T01:33:26.694+0000 7f1588a74640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-26T01:33:26.699 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.697+0000 7f55f8ff9640 1 -- 172.21.15.169:0/1681696597 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v24) v1 ==== 161+0+0 (secure 0 0 0) 0x7f55ec017020 con 0x7f55fc0713d0 2023-10-26T01:33:26.699 INFO:teuthology.orchestra.run.smithi169.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-26T01:33:26.710 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.706+0000 7f5601ff0640 1 -- 172.21.15.169:0/1681696597 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f55dc06ae40 msgr2=0x7f55dc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:26.710 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.706+0000 7f5601ff0640 1 --2- 172.21.15.169:0/1681696597 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f55dc06ae40 0x7f55dc06d300 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f55e4003cc0 tx=0x7f55e40079b0 comp rx=0 tx=0).stop 2023-10-26T01:33:26.710 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.706+0000 7f5601ff0640 1 -- 172.21.15.169:0/1681696597 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55fc0713d0 msgr2=0x7f55fc1480d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:26.710 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.706+0000 7f5601ff0640 1 --2- 172.21.15.169:0/1681696597 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55fc0713d0 0x7f55fc1480d0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f55ec00c3c0 tx=0x7f55ec00c880 comp rx=0 tx=0).stop 2023-10-26T01:33:26.711 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.707+0000 7f5601ff0640 1 -- 172.21.15.169:0/1681696597 shutdown_connections 2023-10-26T01:33:26.711 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.707+0000 7f5601ff0640 1 --2- 172.21.15.169:0/1681696597 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f55dc06ae40 0x7f55dc06d300 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:26.711 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.707+0000 7f5601ff0640 1 --2- 172.21.15.169:0/1681696597 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f55fc15ca70 0x7f55fc15b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:26.711 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.707+0000 7f5601ff0640 1 --2- 172.21.15.169:0/1681696597 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f55fc0cbff0 0x7f55fc1f18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:26.711 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.707+0000 7f5601ff0640 1 --2- 172.21.15.169:0/1681696597 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55fc0713d0 0x7f55fc1480d0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:26.712 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.707+0000 7f5601ff0640 1 -- 172.21.15.169:0/1681696597 >> 172.21.15.169:0/1681696597 conn(0x7f55fc0af8a0 msgr2=0x7f55fc1586c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:26.712 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.707+0000 7f5601ff0640 1 -- 172.21.15.169:0/1681696597 shutdown_connections 2023-10-26T01:33:26.712 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.707+0000 7f5601ff0640 1 -- 172.21.15.169:0/1681696597 wait complete. 2023-10-26T01:33:26.726 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-26T01:33:26.772 INFO:tasks.ceph.mds.a.smithi169.stdout:starting mds.a at 2023-10-26T01:33:26.912 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.909+0000 7f77d4bf0640 1 -- 172.21.15.169:0/1487575194 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f77d01597c0 msgr2=0x7f77d015dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:26.912 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.909+0000 7f77d4bf0640 1 --2- 172.21.15.169:0/1487575194 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f77d01597c0 0x7f77d015dbf0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f77d00b1ea0 tx=0x7f77b802bda0 comp rx=0 tx=0).stop 2023-10-26T01:33:26.912 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.910+0000 7f77d4bf0640 1 -- 172.21.15.169:0/1487575194 shutdown_connections 2023-10-26T01:33:26.913 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.910+0000 7f77d4bf0640 1 --2- 172.21.15.169:0/1487575194 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f77d00b0c80 0x7f77d00cd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:26.913 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.910+0000 7f77d4bf0640 1 --2- 172.21.15.169:0/1487575194 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f77d01597c0 0x7f77d015dbf0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:26.913 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.910+0000 7f77d4bf0640 1 --2- 172.21.15.169:0/1487575194 >> v2:172.21.15.170:3300/0 conn(0x7f77d01593e0 0x7f77d00ac200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:26.913 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.910+0000 7f77d4bf0640 1 -- 172.21.15.169:0/1487575194 >> 172.21.15.169:0/1487575194 conn(0x7f77d0069930 msgr2=0x7f77d0069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:26.914 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.910+0000 7f77d4bf0640 1 -- 172.21.15.169:0/1487575194 shutdown_connections 2023-10-26T01:33:26.914 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.910+0000 7f77d4bf0640 1 -- 172.21.15.169:0/1487575194 wait complete. 2023-10-26T01:33:26.916 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.912+0000 7f77d4bf0640 1 Processor -- start 2023-10-26T01:33:26.917 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.912+0000 7f77d4bf0640 1 -- start start 2023-10-26T01:33:26.917 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.913+0000 7f77d4bf0640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f77d00b0c80 0x7f77d014f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:26.917 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.913+0000 7f77d4bf0640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f77d01593e0 0x7f77d00ad930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:26.917 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.913+0000 7f77d4bf0640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f77d01597c0 0x7f77d00af620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:26.917 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.913+0000 7f77d4bf0640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d014fd60 con 0x7f77d00b0c80 2023-10-26T01:33:26.918 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.913+0000 7f77d4bf0640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d006fb80 con 0x7f77d01597c0 2023-10-26T01:33:26.918 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.913+0000 7f77d4bf0640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77d006fe20 con 0x7f77d01593e0 2023-10-26T01:33:26.918 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.914+0000 7f77ce59c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f77d00b0c80 0x7f77d014f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:26.918 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.914+0000 7f77ce59c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f77d00b0c80 0x7f77d014f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:42762/0 (socket says 172.21.15.169:42762) 2023-10-26T01:33:26.919 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.914+0000 7f77ce59c640 1 -- 172.21.15.169:0/3188783639 learned_addr learned my addr 172.21.15.169:0/3188783639 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:33:26.919 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.914+0000 7f77ced9d640 1 --2- 172.21.15.169:0/3188783639 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f77d01597c0 0x7f77d00af620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:26.919 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.914+0000 7f77c5d9b640 1 --2- 172.21.15.169:0/3188783639 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f77d01593e0 0x7f77d00ad930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:26.919 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.914+0000 7f77ce59c640 1 -- 172.21.15.169:0/3188783639 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f77d01593e0 msgr2=0x7f77d00ad930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:26.919 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.914+0000 7f77ce59c640 1 --2- 172.21.15.169:0/3188783639 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f77d01593e0 0x7f77d00ad930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:26.920 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.914+0000 7f77ce59c640 1 -- 172.21.15.169:0/3188783639 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f77d01597c0 msgr2=0x7f77d00af620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:26.920 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.914+0000 7f77ce59c640 1 --2- 172.21.15.169:0/3188783639 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f77d01597c0 0x7f77d00af620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:26.920 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.914+0000 7f77ce59c640 1 -- 172.21.15.169:0/3188783639 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77b8009560 con 0x7f77d00b0c80 2023-10-26T01:33:26.920 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.914+0000 7f77ced9d640 1 --2- 172.21.15.169:0/3188783639 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f77d01597c0 0x7f77d00af620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:33:26.921 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.914+0000 7f77ce59c640 1 --2- 172.21.15.169:0/3188783639 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f77d00b0c80 0x7f77d014f820 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f77c000d950 tx=0x7f77c000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:26.921 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.915+0000 7f77c77fe640 1 -- 172.21.15.169:0/3188783639 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c0009c30 con 0x7f77d00b0c80 2023-10-26T01:33:26.921 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.915+0000 7f77c77fe640 1 -- 172.21.15.169:0/3188783639 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f77c0009e80 con 0x7f77d00b0c80 2023-10-26T01:33:26.921 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.915+0000 7f77d4bf0640 1 -- 172.21.15.169:0/3188783639 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77d0070100 con 0x7f77d00b0c80 2023-10-26T01:33:26.921 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.915+0000 7f77c77fe640 1 -- 172.21.15.169:0/3188783639 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c00106c0 con 0x7f77d00b0c80 2023-10-26T01:33:26.922 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.915+0000 7f77d4bf0640 1 -- 172.21.15.169:0/3188783639 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77d00706c0 con 0x7f77d00b0c80 2023-10-26T01:33:26.923 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.916+0000 7f77d4bf0640 1 -- 172.21.15.169:0/3188783639 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77d0070850 con 0x7f77d00b0c80 2023-10-26T01:33:26.923 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.917+0000 7f77c77fe640 1 -- 172.21.15.169:0/3188783639 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f77c0010820 con 0x7f77d00b0c80 2023-10-26T01:33:26.924 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.918+0000 7f77c77fe640 1 --2- 172.21.15.169:0/3188783639 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f77b006ae40 0x7f77b006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:26.924 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.918+0000 7f77c77fe640 1 -- 172.21.15.169:0/3188783639 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f77c0089b70 con 0x7f77d00b0c80 2023-10-26T01:33:26.924 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.919+0000 7f77c5d9b640 1 --2- 172.21.15.169:0/3188783639 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f77b006ae40 0x7f77b006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:26.924 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.920+0000 7f77c5d9b640 1 --2- 172.21.15.169:0/3188783639 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f77b006ae40 0x7f77b006d300 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f77b8035000 tx=0x7f77b8004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:26.925 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:26.922+0000 7f77c77fe640 1 -- 172.21.15.169:0/3188783639 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f77c0054bb0 con 0x7f77d00b0c80 2023-10-26T01:33:27.154 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:27.152+0000 7f77d4bf0640 1 -- 172.21.15.169:0/3188783639 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f77d00b1390 con 0x7f77d00b0c80 2023-10-26T01:33:27.702 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:27.700+0000 7f77c77fe640 1 -- 172.21.15.169:0/3188783639 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v25) v1 ==== 104+0+0 (secure 0 0 0) 0x7f77c00200a0 con 0x7f77d00b0c80 2023-10-26T01:33:27.706 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:27.704+0000 7f77d4bf0640 1 -- 172.21.15.169:0/3188783639 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f77b006ae40 msgr2=0x7f77b006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:27.706 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:27.704+0000 7f77d4bf0640 1 --2- 172.21.15.169:0/3188783639 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f77b006ae40 0x7f77b006d300 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f77b8035000 tx=0x7f77b8004730 comp rx=0 tx=0).stop 2023-10-26T01:33:27.706 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:27.704+0000 7f77d4bf0640 1 -- 172.21.15.169:0/3188783639 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f77d00b0c80 msgr2=0x7f77d014f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:27.707 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:27.704+0000 7f77d4bf0640 1 --2- 172.21.15.169:0/3188783639 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f77d00b0c80 0x7f77d014f820 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f77c000d950 tx=0x7f77c000de10 comp rx=0 tx=0).stop 2023-10-26T01:33:27.707 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:27.704+0000 7f77d4bf0640 1 -- 172.21.15.169:0/3188783639 shutdown_connections 2023-10-26T01:33:27.707 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:27.704+0000 7f77d4bf0640 1 --2- 172.21.15.169:0/3188783639 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f77b006ae40 0x7f77b006d300 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:27.708 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:27.704+0000 7f77d4bf0640 1 --2- 172.21.15.169:0/3188783639 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f77d01597c0 0x7f77d00af620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:27.708 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:27.704+0000 7f77d4bf0640 1 --2- 172.21.15.169:0/3188783639 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f77d01593e0 0x7f77d00ad930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:27.708 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:27.704+0000 7f77d4bf0640 1 --2- 172.21.15.169:0/3188783639 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f77d00b0c80 0x7f77d014f820 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:27.708 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:27.704+0000 7f77d4bf0640 1 -- 172.21.15.169:0/3188783639 >> 172.21.15.169:0/3188783639 conn(0x7f77d0069930 msgr2=0x7f77d01516e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:27.708 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:27.704+0000 7f77d4bf0640 1 -- 172.21.15.169:0/3188783639 shutdown_connections 2023-10-26T01:33:27.709 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:27.704+0000 7f77d4bf0640 1 -- 172.21.15.169:0/3188783639 wait complete. 2023-10-26T01:33:27.728 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-26T01:33:27.999 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:27.997+0000 7f219710a640 1 -- 172.21.15.169:0/4150689842 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f219015a430 msgr2=0x7f21900b0e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:28.000 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:27.997+0000 7f219710a640 1 --2- 172.21.15.169:0/4150689842 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f219015a430 0x7f21900b0e90 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f2188005fd0 tx=0x7f218802bfc0 comp rx=0 tx=0).stop 2023-10-26T01:33:28.000 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:27.997+0000 7f219710a640 1 -- 172.21.15.169:0/4150689842 shutdown_connections 2023-10-26T01:33:28.001 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:27.997+0000 7f219710a640 1 --2- 172.21.15.169:0/4150689842 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f219015a430 0x7f21900b0e90 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:28.001 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:27.997+0000 7f219710a640 1 --2- 172.21.15.169:0/4150689842 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f21900717c0 0x7f219009a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:28.001 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:27.997+0000 7f219710a640 1 -- 172.21.15.169:0/4150689842 >> 172.21.15.169:0/4150689842 conn(0x7f2190069b40 msgr2=0x7f21900aeb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:28.001 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:27.997+0000 7f219710a640 1 -- 172.21.15.169:0/4150689842 shutdown_connections 2023-10-26T01:33:28.001 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:27.997+0000 7f219710a640 1 -- 172.21.15.169:0/4150689842 wait complete. 2023-10-26T01:33:28.004 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:27.999+0000 7f219710a640 1 Processor -- start 2023-10-26T01:33:28.004 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:27.999+0000 7f219710a640 1 -- start start 2023-10-26T01:33:28.004 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:28.000+0000 7f219710a640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f21900717c0 0x7f219014fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:28.004 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:28.000+0000 7f219710a640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21901538c0 0x7f219006f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:28.004 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:28.000+0000 7f219710a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f219015a430 0x7f21901ef750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:28.005 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:28.000+0000 7f219710a640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21901f0cd0 con 0x7f21901538c0 2023-10-26T01:33:28.005 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:28.000+0000 7f219710a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21901f00c0 con 0x7f219015a430 2023-10-26T01:33:28.005 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:28.000+0000 7f219710a640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21901f03a0 con 0x7f21900717c0 2023-10-26T01:33:28.005 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:28.001+0000 7f2195907640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21901538c0 0x7f219006f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:28.005 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:28.001+0000 7f2195907640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21901538c0 0x7f219006f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:42778/0 (socket says 172.21.15.169:42778) 2023-10-26T01:33:28.006 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:28.001+0000 7f2195907640 1 -- 172.21.15.169:0/133104749 learned_addr learned my addr 172.21.15.169:0/133104749 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:33:28.006 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:28.001+0000 7f2196108640 1 --2- 172.21.15.169:0/133104749 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f21900717c0 0x7f219014fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:28.006 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:28.001+0000 7f2195907640 1 -- 172.21.15.169:0/133104749 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f21900717c0 msgr2=0x7f219014fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:28.006 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:28.001+0000 7f2195907640 1 --2- 172.21.15.169:0/133104749 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f21900717c0 0x7f219014fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:28.006 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:28.001+0000 7f2195907640 1 -- 172.21.15.169:0/133104749 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f219015a430 msgr2=0x7f21901ef750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:28.007 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:28.001+0000 7f2196909640 1 --2- 172.21.15.169:0/133104749 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f219015a430 0x7f21901ef750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:28.007 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:28.001+0000 7f2195907640 1 --2- 172.21.15.169:0/133104749 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f219015a430 0x7f21901ef750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:28.007 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:28.001+0000 7f2195907640 1 -- 172.21.15.169:0/133104749 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2188009840 con 0x7f21901538c0 2023-10-26T01:33:28.007 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:28.001+0000 7f2195907640 1 --2- 172.21.15.169:0/133104749 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21901538c0 0x7f219006f670 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f218400d950 tx=0x7f218400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:28.007 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:28.002+0000 7f2182ffd640 1 -- 172.21.15.169:0/133104749 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2184009c30 con 0x7f21901538c0 2023-10-26T01:33:28.008 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:28.002+0000 7f2182ffd640 1 -- 172.21.15.169:0/133104749 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2184009e80 con 0x7f21901538c0 2023-10-26T01:33:28.008 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:28.002+0000 7f219710a640 1 -- 172.21.15.169:0/133104749 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21901f0680 con 0x7f21901538c0 2023-10-26T01:33:28.008 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:28.002+0000 7f2182ffd640 1 -- 172.21.15.169:0/133104749 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21840106c0 con 0x7f21901538c0 2023-10-26T01:33:28.008 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:28.002+0000 7f219710a640 1 -- 172.21.15.169:0/133104749 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2190151e50 con 0x7f21901538c0 2023-10-26T01:33:28.009 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:28.003+0000 7f219710a640 1 -- 172.21.15.169:0/133104749 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2190073c10 con 0x7f21901538c0 2023-10-26T01:33:28.009 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:28.004+0000 7f2182ffd640 1 -- 172.21.15.169:0/133104749 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f2184010820 con 0x7f21901538c0 2023-10-26T01:33:28.009 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:28.005+0000 7f2182ffd640 1 --2- 172.21.15.169:0/133104749 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f217406ae70 0x7f217406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:28.010 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:28.005+0000 7f2182ffd640 1 -- 172.21.15.169:0/133104749 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f2184089bb0 con 0x7f21901538c0 2023-10-26T01:33:28.010 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:28.005+0000 7f2196108640 1 --2- 172.21.15.169:0/133104749 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f217406ae70 0x7f217406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:28.010 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:28.006+0000 7f2196108640 1 --2- 172.21.15.169:0/133104749 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f217406ae70 0x7f217406d330 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f218c001640 tx=0x7f218c009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:28.011 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:28.009+0000 7f2182ffd640 1 -- 172.21.15.169:0/133104749 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2184054b60 con 0x7f21901538c0 2023-10-26T01:33:28.207 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:28.206+0000 7f219710a640 1 -- 172.21.15.169:0/133104749 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f21900b1fb0 con 0x7f21901538c0 2023-10-26T01:33:29.025 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:29.023+0000 7f2182ffd640 1 -- 172.21.15.169:0/133104749 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v56) v1 ==== 181+0+0 (secure 0 0 0) 0x7f21840170d0 con 0x7f21901538c0 2023-10-26T01:33:29.026 INFO:teuthology.orchestra.run.smithi169.stderr:pool 'cephfs_metadata' removed 2023-10-26T01:33:29.028 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:29.027+0000 7f219710a640 1 -- 172.21.15.169:0/133104749 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f217406ae70 msgr2=0x7f217406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:29.029 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:29.027+0000 7f219710a640 1 --2- 172.21.15.169:0/133104749 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f217406ae70 0x7f217406d330 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f218c001640 tx=0x7f218c009040 comp rx=0 tx=0).stop 2023-10-26T01:33:29.029 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:29.027+0000 7f219710a640 1 -- 172.21.15.169:0/133104749 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21901538c0 msgr2=0x7f219006f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:29.029 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:29.027+0000 7f219710a640 1 --2- 172.21.15.169:0/133104749 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21901538c0 0x7f219006f670 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f218400d950 tx=0x7f218400de10 comp rx=0 tx=0).stop 2023-10-26T01:33:29.030 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:29.027+0000 7f219710a640 1 -- 172.21.15.169:0/133104749 shutdown_connections 2023-10-26T01:33:29.030 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:29.027+0000 7f219710a640 1 --2- 172.21.15.169:0/133104749 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f217406ae70 0x7f217406d330 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:29.030 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:29.027+0000 7f219710a640 1 --2- 172.21.15.169:0/133104749 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21901538c0 0x7f219006f670 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:29.030 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:29.027+0000 7f219710a640 1 --2- 172.21.15.169:0/133104749 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f219015a430 0x7f21901ef750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:29.031 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:29.027+0000 7f219710a640 1 --2- 172.21.15.169:0/133104749 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f21900717c0 0x7f219014fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:29.031 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:29.027+0000 7f219710a640 1 -- 172.21.15.169:0/133104749 >> 172.21.15.169:0/133104749 conn(0x7f2190069b40 msgr2=0x7f2190152940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:29.031 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:29.027+0000 7f219710a640 1 -- 172.21.15.169:0/133104749 shutdown_connections 2023-10-26T01:33:29.031 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:29.027+0000 7f219710a640 1 -- 172.21.15.169:0/133104749 wait complete. 2023-10-26T01:33:29.051 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-26T01:33:29.272 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:29.270+0000 7f8a16ca9640 1 -- 172.21.15.169:0/536751248 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a000039a0 msgr2=0x7f8a00003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:29.273 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:29.270+0000 7f8a16ca9640 1 --2- 172.21.15.169:0/536751248 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a000039a0 0x7f8a00003da0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f8a04006f20 tx=0x7f8a000057f0 comp rx=0 tx=0).stop 2023-10-26T01:33:29.273 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:29.270+0000 7f8a15246640 1 -- 172.21.15.169:0/536751248 reap_dead start 2023-10-26T01:33:29.273 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:29.270+0000 7f8a16ca9640 1 -- 172.21.15.169:0/536751248 shutdown_connections 2023-10-26T01:33:29.273 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:29.270+0000 7f8a16ca9640 1 -- 172.21.15.169:0/536751248 >> 172.21.15.169:0/536751248 conn(0x7f8a100aeee0 msgr2=0x7f8a100af2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:29.273 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:29.271+0000 7f8a16ca9640 1 -- 172.21.15.169:0/536751248 shutdown_connections 2023-10-26T01:33:29.274 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:29.271+0000 7f8a16ca9640 1 -- 172.21.15.169:0/536751248 wait complete. 2023-10-26T01:33:29.274 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:29.272+0000 7f8a16ca9640 1 Processor -- start 2023-10-26T01:33:29.274 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:29.272+0000 7f8a16ca9640 1 -- start start 2023-10-26T01:33:29.276 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:29.273+0000 7f8a16ca9640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f8a000039a0 0x7f8a1014f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:29.276 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:29.273+0000 7f8a16ca9640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a101f57e0 0x7f8a1015a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:29.276 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:29.273+0000 7f8a16ca9640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a10153f80 0x7f8a10154430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:29.277 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:29.273+0000 7f8a16ca9640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a101f6780 con 0x7f8a10153f80 2023-10-26T01:33:29.277 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:29.273+0000 7f8a16ca9640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a10154970 con 0x7f8a101f57e0 2023-10-26T01:33:29.277 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:29.273+0000 7f8a16ca9640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a1015ac40 con 0x7f8a000039a0 2023-10-26T01:33:29.277 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:29.274+0000 7f8a15246640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a10153f80 0x7f8a10154430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:29.277 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:29.274+0000 7f8a15246640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a10153f80 0x7f8a10154430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:42792/0 (socket says 172.21.15.169:42792) 2023-10-26T01:33:29.278 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:29.274+0000 7f8a15246640 1 -- 172.21.15.169:0/2373295161 learned_addr learned my addr 172.21.15.169:0/2373295161 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:33:29.278 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:29.274+0000 7f8a15246640 1 -- 172.21.15.169:0/2373295161 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f8a000039a0 msgr2=0x7f8a1014f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:29.278 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:29.274+0000 7f8a15246640 1 --2- 172.21.15.169:0/2373295161 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f8a000039a0 0x7f8a1014f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:29.278 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:29.274+0000 7f8a15246640 1 -- 172.21.15.169:0/2373295161 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a101f57e0 msgr2=0x7f8a1015a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:29.278 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:29.274+0000 7f8a15246640 1 --2- 172.21.15.169:0/2373295161 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a101f57e0 0x7f8a1015a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:29.279 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:29.274+0000 7f8a15246640 1 -- 172.21.15.169:0/2373295161 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a101f5c60 con 0x7f8a10153f80 2023-10-26T01:33:29.279 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:29.274+0000 7f8a15246640 1 --2- 172.21.15.169:0/2373295161 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a10153f80 0x7f8a10154430 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f8a10072210 tx=0x7f8a0800cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:29.279 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:29.274+0000 7f8a0d7fa640 1 -- 172.21.15.169:0/2373295161 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a08005d40 con 0x7f8a10153f80 2023-10-26T01:33:29.279 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:29.274+0000 7f8a0d7fa640 1 -- 172.21.15.169:0/2373295161 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8a0800ced0 con 0x7f8a10153f80 2023-10-26T01:33:29.279 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:29.274+0000 7f8a0d7fa640 1 -- 172.21.15.169:0/2373295161 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a0800a810 con 0x7f8a10153f80 2023-10-26T01:33:29.280 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:29.274+0000 7f8a16ca9640 1 -- 172.21.15.169:0/2373295161 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a101f5ee0 con 0x7f8a10153f80 2023-10-26T01:33:29.280 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:29.275+0000 7f8a16ca9640 1 -- 172.21.15.169:0/2373295161 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a101f64a0 con 0x7f8a10153f80 2023-10-26T01:33:29.281 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:29.276+0000 7f8a16ca9640 1 -- 172.21.15.169:0/2373295161 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a10066af0 con 0x7f8a10153f80 2023-10-26T01:33:29.281 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:29.277+0000 7f8a0d7fa640 1 -- 172.21.15.169:0/2373295161 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f8a0800a970 con 0x7f8a10153f80 2023-10-26T01:33:29.281 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:29.277+0000 7f8a0d7fa640 1 --2- 172.21.15.169:0/2373295161 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f89f406ae70 0x7f89f406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:29.281 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:29.278+0000 7f8a0d7fa640 1 -- 172.21.15.169:0/2373295161 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5357+0+0 (secure 0 0 0) 0x7f8a08088be0 con 0x7f8a10153f80 2023-10-26T01:33:29.281 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:29.278+0000 7f8a14a45640 1 --2- 172.21.15.169:0/2373295161 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f89f406ae70 0x7f89f406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:29.282 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:29.279+0000 7f8a14a45640 1 --2- 172.21.15.169:0/2373295161 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f89f406ae70 0x7f89f406d330 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f8a04034000 tx=0x7f8a04039040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:29.283 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:29.282+0000 7f8a0d7fa640 1 -- 172.21.15.169:0/2373295161 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8a08053d50 con 0x7f8a10153f80 2023-10-26T01:33:29.457 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:29.455+0000 7f8a16ca9640 1 -- 172.21.15.169:0/2373295161 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8a1004da70 con 0x7f8a10153f80 2023-10-26T01:33:30.037 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:30.035+0000 7f8a0d7fa640 1 -- 172.21.15.169:0/2373295161 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v57) v1 ==== 169+0+0 (secure 0 0 0) 0x7f8a08017020 con 0x7f8a10153f80 2023-10-26T01:33:30.037 INFO:teuthology.orchestra.run.smithi169.stderr:pool 'cephfs_data' removed 2023-10-26T01:33:30.044 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:30.042+0000 7f8a16ca9640 1 -- 172.21.15.169:0/2373295161 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f89f406ae70 msgr2=0x7f89f406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:30.044 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:30.042+0000 7f8a16ca9640 1 --2- 172.21.15.169:0/2373295161 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f89f406ae70 0x7f89f406d330 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f8a04034000 tx=0x7f8a04039040 comp rx=0 tx=0).stop 2023-10-26T01:33:30.044 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:30.042+0000 7f8a16ca9640 1 -- 172.21.15.169:0/2373295161 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a10153f80 msgr2=0x7f8a10154430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:30.044 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:30.042+0000 7f8a16ca9640 1 --2- 172.21.15.169:0/2373295161 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a10153f80 0x7f8a10154430 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f8a10072210 tx=0x7f8a0800cb90 comp rx=0 tx=0).stop 2023-10-26T01:33:30.046 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:30.044+0000 7f8a16ca9640 1 -- 172.21.15.169:0/2373295161 shutdown_connections 2023-10-26T01:33:30.046 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:30.044+0000 7f8a16ca9640 1 --2- 172.21.15.169:0/2373295161 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f89f406ae70 0x7f89f406d330 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:30.046 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:30.044+0000 7f8a16ca9640 1 --2- 172.21.15.169:0/2373295161 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a10153f80 0x7f8a10154430 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:30.049 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:30.044+0000 7f8a16ca9640 1 --2- 172.21.15.169:0/2373295161 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a101f57e0 0x7f8a1015a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:30.049 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:30.044+0000 7f8a16ca9640 1 --2- 172.21.15.169:0/2373295161 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f8a000039a0 0x7f8a1014f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:30.049 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:30.044+0000 7f8a16ca9640 1 -- 172.21.15.169:0/2373295161 >> 172.21.15.169:0/2373295161 conn(0x7f8a100aeee0 msgr2=0x7f8a1015d700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:30.050 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:30.044+0000 7f8a16ca9640 1 -- 172.21.15.169:0/2373295161 shutdown_connections 2023-10-26T01:33:30.050 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:30.044+0000 7f8a16ca9640 1 -- 172.21.15.169:0/2373295161 wait complete. 2023-10-26T01:33:30.069 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 18 2023-10-26T01:33:30.289 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:30.287+0000 7f684d452640 1 -- 172.21.15.169:0/745996198 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f684815ece0 msgr2=0x7f68480b0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:30.290 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:30.287+0000 7f684d452640 1 --2- 172.21.15.169:0/745996198 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f684815ece0 0x7f68480b0170 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f6838009830 tx=0x7f683802bd70 comp rx=0 tx=0).stop 2023-10-26T01:33:30.290 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:30.288+0000 7f684d452640 1 -- 172.21.15.169:0/745996198 shutdown_connections 2023-10-26T01:33:30.290 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:30.288+0000 7f684d452640 1 --2- 172.21.15.169:0/745996198 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f684815ece0 0x7f68480b0170 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:30.290 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:30.288+0000 7f684d452640 1 -- 172.21.15.169:0/745996198 >> 172.21.15.169:0/745996198 conn(0x7f68480d1370 msgr2=0x7f68480d1780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:30.291 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:30.288+0000 7f684d452640 1 -- 172.21.15.169:0/745996198 shutdown_connections 2023-10-26T01:33:30.291 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:30.288+0000 7f684d452640 1 -- 172.21.15.169:0/745996198 wait complete. 2023-10-26T01:33:30.291 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:30.289+0000 7f684d452640 1 Processor -- start 2023-10-26T01:33:30.291 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:30.290+0000 7f684d452640 1 -- start start 2023-10-26T01:33:30.293 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:30.290+0000 7f684d452640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f68480b0720 0x7f68480c9490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:30.293 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:30.290+0000 7f684d452640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f68480d0e10 0x7f68481f29d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:30.294 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:30.291+0000 7f684d452640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f684815ece0 0x7f68481537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:30.294 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:30.291+0000 7f684d452640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f684814f930 con 0x7f684815ece0 2023-10-26T01:33:30.294 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:30.291+0000 7f684d452640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68481f2f10 con 0x7f68480b0720 2023-10-26T01:33:30.294 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:30.291+0000 7f684d452640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6848153d20 con 0x7f68480d0e10 2023-10-26T01:33:30.294 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:30.291+0000 7f684cc51640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f684815ece0 0x7f68481537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:30.295 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:30.291+0000 7f684cc51640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f684815ece0 0x7f68481537e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:42818/0 (socket says 172.21.15.169:42818) 2023-10-26T01:33:30.295 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:30.291+0000 7f684cc51640 1 -- 172.21.15.169:0/2077873592 learned_addr learned my addr 172.21.15.169:0/2077873592 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:33:30.295 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:30.291+0000 7f6847fff640 1 --2- 172.21.15.169:0/2077873592 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f68480b0720 0x7f68480c9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:30.295 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:30.291+0000 7f683ffff640 1 --2- 172.21.15.169:0/2077873592 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f68480d0e10 0x7f68481f29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:30.295 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:30.291+0000 7f684cc51640 1 -- 172.21.15.169:0/2077873592 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f68480d0e10 msgr2=0x7f68481f29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:30.296 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:30.291+0000 7f684cc51640 1 --2- 172.21.15.169:0/2077873592 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f68480d0e10 0x7f68481f29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:30.296 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:30.291+0000 7f684cc51640 1 -- 172.21.15.169:0/2077873592 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f68480b0720 msgr2=0x7f68480c9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:30.296 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:30.291+0000 7f684cc51640 1 --2- 172.21.15.169:0/2077873592 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f68480b0720 0x7f68480c9490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:30.296 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:30.291+0000 7f684cc51640 1 -- 172.21.15.169:0/2077873592 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6838009490 con 0x7f684815ece0 2023-10-26T01:33:30.296 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:30.291+0000 7f6847fff640 1 --2- 172.21.15.169:0/2077873592 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f68480b0720 0x7f68480c9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T01:33:30.297 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:30.292+0000 7f684cc51640 1 --2- 172.21.15.169:0/2077873592 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f684815ece0 0x7f68481537e0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f684000c7b0 tx=0x7f684000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:30.298 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:30.292+0000 7f68457fa640 1 -- 172.21.15.169:0/2077873592 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6840004b60 con 0x7f684815ece0 2023-10-26T01:33:30.298 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:30.292+0000 7f68457fa640 1 -- 172.21.15.169:0/2077873592 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6840004db0 con 0x7f684815ece0 2023-10-26T01:33:30.298 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:30.292+0000 7f68457fa640 1 -- 172.21.15.169:0/2077873592 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f684000aab0 con 0x7f684815ece0 2023-10-26T01:33:30.298 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:30.292+0000 7f684d452640 1 -- 172.21.15.169:0/2077873592 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f684814d5c0 con 0x7f684815ece0 2023-10-26T01:33:30.299 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:30.292+0000 7f684d452640 1 -- 172.21.15.169:0/2077873592 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68480c9e70 con 0x7f684815ece0 2023-10-26T01:33:30.299 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:30.293+0000 7f684d452640 1 -- 172.21.15.169:0/2077873592 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f684814d7c0 con 0x7f684815ece0 2023-10-26T01:33:30.299 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:30.294+0000 7f68457fa640 1 -- 172.21.15.169:0/2077873592 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f684000ac10 con 0x7f684815ece0 2023-10-26T01:33:30.299 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:30.295+0000 7f68457fa640 1 --2- 172.21.15.169:0/2077873592 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f682806ae40 0x7f682806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:30.299 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:30.295+0000 7f68457fa640 1 -- 172.21.15.169:0/2077873592 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f6840089690 con 0x7f684815ece0 2023-10-26T01:33:30.299 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:30.296+0000 7f6847fff640 1 --2- 172.21.15.169:0/2077873592 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f682806ae40 0x7f682806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:30.300 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:30.297+0000 7f6847fff640 1 --2- 172.21.15.169:0/2077873592 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f682806ae40 0x7f682806d300 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f6838036000 tx=0x7f6838004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:30.301 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:30.300+0000 7f68457fa640 1 -- 172.21.15.169:0/2077873592 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f684814d7c0 con 0x7f684815ece0 2023-10-26T01:33:30.496 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:30.494+0000 7f684d452640 1 -- 172.21.15.169:0/2077873592 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "config reset", "num": 18} v 0) v1 -- 0x7f68480d07a0 con 0x7f684815ece0 2023-10-26T01:33:30.503 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:30.502+0000 7f68457fa640 1 -- 172.21.15.169:0/2077873592 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 18}]=0 v22) v1 ==== 71+0+0 (secure 0 0 0) 0x7f6840054b80 con 0x7f684815ece0 2023-10-26T01:33:30.507 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:30.505+0000 7f684d452640 1 -- 172.21.15.169:0/2077873592 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f682806ae40 msgr2=0x7f682806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:30.507 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:30.505+0000 7f684d452640 1 --2- 172.21.15.169:0/2077873592 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f682806ae40 0x7f682806d300 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f6838036000 tx=0x7f6838004730 comp rx=0 tx=0).stop 2023-10-26T01:33:30.508 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:30.505+0000 7f684d452640 1 -- 172.21.15.169:0/2077873592 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f684815ece0 msgr2=0x7f68481537e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:30.508 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:30.505+0000 7f684d452640 1 --2- 172.21.15.169:0/2077873592 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f684815ece0 0x7f68481537e0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f684000c7b0 tx=0x7f684000cc70 comp rx=0 tx=0).stop 2023-10-26T01:33:30.508 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:30.505+0000 7f684d452640 1 -- 172.21.15.169:0/2077873592 shutdown_connections 2023-10-26T01:33:30.508 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:30.505+0000 7f684d452640 1 --2- 172.21.15.169:0/2077873592 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f682806ae40 0x7f682806d300 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:30.508 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:30.505+0000 7f684d452640 1 --2- 172.21.15.169:0/2077873592 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f684815ece0 0x7f68481537e0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:30.509 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:30.505+0000 7f684d452640 1 --2- 172.21.15.169:0/2077873592 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f68480d0e10 0x7f68481f29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:30.509 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:30.505+0000 7f684d452640 1 --2- 172.21.15.169:0/2077873592 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f68480b0720 0x7f68480c9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:30.509 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:30.505+0000 7f684d452640 1 -- 172.21.15.169:0/2077873592 >> 172.21.15.169:0/2077873592 conn(0x7f68480d1370 msgr2=0x7f68480c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:30.509 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:30.505+0000 7f684d452640 1 -- 172.21.15.169:0/2077873592 shutdown_connections 2023-10-26T01:33:30.509 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:30.506+0000 7f684d452640 1 -- 172.21.15.169:0/2077873592 wait complete. 2023-10-26T01:33:30.525 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestClusterFull.test_full_different_file' 2023-10-26T01:33:30.727 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:30.725+0000 7fd92869b640 1 -- 172.21.15.169:0/3012822701 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd92014f2c0 msgr2=0x7fd920159240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:30.728 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:30.725+0000 7fd92869b640 1 --2- 172.21.15.169:0/3012822701 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd92014f2c0 0x7fd920159240 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fd910009830 tx=0x7fd91002bd60 comp rx=0 tx=0).stop 2023-10-26T01:33:30.728 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:30.725+0000 7fd92869b640 1 -- 172.21.15.169:0/3012822701 shutdown_connections 2023-10-26T01:33:30.728 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:30.725+0000 7fd92869b640 1 --2- 172.21.15.169:0/3012822701 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd92014f6a0 0x7fd9200cd720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:30.728 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:30.725+0000 7fd92869b640 1 --2- 172.21.15.169:0/3012822701 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd92014f2c0 0x7fd920159240 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:30.729 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:30.725+0000 7fd92869b640 1 -- 172.21.15.169:0/3012822701 >> 172.21.15.169:0/3012822701 conn(0x7fd920069930 msgr2=0x7fd920069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:30.729 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:30.726+0000 7fd92869b640 1 -- 172.21.15.169:0/3012822701 shutdown_connections 2023-10-26T01:33:30.729 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:30.726+0000 7fd92869b640 1 -- 172.21.15.169:0/3012822701 wait complete. 2023-10-26T01:33:30.730 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:30.727+0000 7fd92869b640 1 Processor -- start 2023-10-26T01:33:30.730 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:30.727+0000 7fd92869b640 1 -- start start 2023-10-26T01:33:30.730 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:30.728+0000 7fd92869b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd9200cdc60 0x7fd92009a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:30.731 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:30.728+0000 7fd92869b640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd92014f2c0 0x7fd920150610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:30.731 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:30.728+0000 7fd92869b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd92014f6a0 0x7fd920157a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:30.731 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:30.728+0000 7fd92869b640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd92014fa80 con 0x7fd92014f6a0 2023-10-26T01:33:30.731 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:30.728+0000 7fd92869b640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd92014ff10 con 0x7fd9200cdc60 2023-10-26T01:33:30.732 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:30.728+0000 7fd92869b640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9200ae8f0 con 0x7fd92014f2c0 2023-10-26T01:33:30.732 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:30.729+0000 7fd926c38640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd92014f6a0 0x7fd920157a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:30.732 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:30.729+0000 7fd926c38640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd92014f6a0 0x7fd920157a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:42834/0 (socket says 172.21.15.169:42834) 2023-10-26T01:33:30.732 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:30.729+0000 7fd926c38640 1 -- 172.21.15.169:0/500216289 learned_addr learned my addr 172.21.15.169:0/500216289 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:33:30.733 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:30.729+0000 7fd925c36640 1 --2- 172.21.15.169:0/500216289 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd92014f2c0 0x7fd920150610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:30.734 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:30.729+0000 7fd926c38640 1 -- 172.21.15.169:0/500216289 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd92014f2c0 msgr2=0x7fd920150610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:30.734 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:30.729+0000 7fd926c38640 1 --2- 172.21.15.169:0/500216289 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd92014f2c0 0x7fd920150610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:30.734 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:30.729+0000 7fd926c38640 1 -- 172.21.15.169:0/500216289 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd9200cdc60 msgr2=0x7fd92009a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:30.734 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:30.729+0000 7fd926c38640 1 --2- 172.21.15.169:0/500216289 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd9200cdc60 0x7fd92009a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:30.734 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:30.729+0000 7fd926c38640 1 -- 172.21.15.169:0/500216289 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd910009490 con 0x7fd92014f6a0 2023-10-26T01:33:30.735 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:30.729+0000 7fd926c38640 1 --2- 172.21.15.169:0/500216289 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd92014f6a0 0x7fd920157a10 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fd91800c430 tx=0x7fd91800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:30.735 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:30.730+0000 7fd916ffd640 1 -- 172.21.15.169:0/500216289 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9180035f0 con 0x7fd92014f6a0 2023-10-26T01:33:30.735 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:30.730+0000 7fd916ffd640 1 -- 172.21.15.169:0/500216289 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd918003770 con 0x7fd92014f6a0 2023-10-26T01:33:30.735 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:30.730+0000 7fd916ffd640 1 -- 172.21.15.169:0/500216289 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd918003b80 con 0x7fd92014f6a0 2023-10-26T01:33:30.736 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:30.730+0000 7fd92869b640 1 -- 172.21.15.169:0/500216289 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9200ae0f0 con 0x7fd92014f6a0 2023-10-26T01:33:30.736 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:30.730+0000 7fd92869b640 1 -- 172.21.15.169:0/500216289 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9200aebd0 con 0x7fd92014f6a0 2023-10-26T01:33:30.736 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:30.731+0000 7fd92869b640 1 -- 172.21.15.169:0/500216289 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd92014fbf0 con 0x7fd92014f6a0 2023-10-26T01:33:30.736 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:30.732+0000 7fd916ffd640 1 -- 172.21.15.169:0/500216289 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fd918003ce0 con 0x7fd92014f6a0 2023-10-26T01:33:30.737 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:30.733+0000 7fd916ffd640 1 --2- 172.21.15.169:0/500216289 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd90c06ae40 0x7fd90c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:30.737 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:30.733+0000 7fd916ffd640 1 -- 172.21.15.169:0/500216289 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fd918088a60 con 0x7fd92014f6a0 2023-10-26T01:33:30.737 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:30.734+0000 7fd926437640 1 --2- 172.21.15.169:0/500216289 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd90c06ae40 0x7fd90c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:30.738 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:30.735+0000 7fd926437640 1 --2- 172.21.15.169:0/500216289 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd90c06ae40 0x7fd90c06d300 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd910036000 tx=0x7fd9100046c0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:30.739 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:30.737+0000 7fd916ffd640 1 -- 172.21.15.169:0/500216289 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd918053ea0 con 0x7fd92014f6a0 2023-10-26T01:33:30.984 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:30.983+0000 7fd92869b640 1 -- 172.21.15.169:0/500216289 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_different_file"]} v 0) v1 -- 0x7fd92004db10 con 0x7fd92014f6a0 2023-10-26T01:33:31.049 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.046+0000 7fd916ffd640 1 -- 172.21.15.169:0/500216289 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_different_file"]}]=0 v170) v1 ==== 142+0+0 (secure 0 0 0) 0x7fd918017070 con 0x7fd92014f6a0 2023-10-26T01:33:31.051 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.049+0000 7fd92869b640 1 -- 172.21.15.169:0/500216289 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd90c06ae40 msgr2=0x7fd90c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:31.051 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.049+0000 7fd92869b640 1 --2- 172.21.15.169:0/500216289 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd90c06ae40 0x7fd90c06d300 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd910036000 tx=0x7fd9100046c0 comp rx=0 tx=0).stop 2023-10-26T01:33:31.052 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.049+0000 7fd92869b640 1 -- 172.21.15.169:0/500216289 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd92014f6a0 msgr2=0x7fd920157a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:31.052 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.049+0000 7fd92869b640 1 --2- 172.21.15.169:0/500216289 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd92014f6a0 0x7fd920157a10 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fd91800c430 tx=0x7fd91800c8f0 comp rx=0 tx=0).stop 2023-10-26T01:33:31.052 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.049+0000 7fd92869b640 1 -- 172.21.15.169:0/500216289 shutdown_connections 2023-10-26T01:33:31.052 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.049+0000 7fd92869b640 1 --2- 172.21.15.169:0/500216289 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd90c06ae40 0x7fd90c06d300 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:31.053 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.049+0000 7fd92869b640 1 --2- 172.21.15.169:0/500216289 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd92014f6a0 0x7fd920157a10 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:31.053 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.049+0000 7fd92869b640 1 --2- 172.21.15.169:0/500216289 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd92014f2c0 0x7fd920150610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:31.053 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.050+0000 7fd92869b640 1 --2- 172.21.15.169:0/500216289 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd9200cdc60 0x7fd92009a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:31.053 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.050+0000 7fd92869b640 1 -- 172.21.15.169:0/500216289 >> 172.21.15.169:0/500216289 conn(0x7fd920069930 msgr2=0x7fd9200cc3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:31.053 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.050+0000 7fd92869b640 1 -- 172.21.15.169:0/500216289 shutdown_connections 2023-10-26T01:33:31.053 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.050+0000 7fd92869b640 1 -- 172.21.15.169:0/500216289 wait complete. 2023-10-26T01:33:31.073 INFO:tasks.cephfs_test_runner:test_full_different_file (tasks.cephfs.test_full.TestClusterFull) ... ok 2023-10-26T01:33:31.074 INFO:tasks.cephfs_test_runner:Starting test: test_full_fclose (tasks.cephfs.test_full.TestClusterFull) 2023-10-26T01:33:31.074 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:33:31.291 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.289+0000 7f5028610640 1 -- 172.21.15.169:0/2743978984 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f50200cdd80 msgr2=0x7f50200b07d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:31.292 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.289+0000 7f5028610640 1 --2- 172.21.15.169:0/2743978984 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f50200cdd80 0x7f50200b07d0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f502006bff0 tx=0x7f501802c110 comp rx=0 tx=0).stop 2023-10-26T01:33:31.292 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.289+0000 7f5028610640 1 -- 172.21.15.169:0/2743978984 shutdown_connections 2023-10-26T01:33:31.292 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.289+0000 7f5028610640 1 --2- 172.21.15.169:0/2743978984 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f50200cdd80 0x7f50200b07d0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:31.293 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.289+0000 7f5028610640 1 --2- 172.21.15.169:0/2743978984 >> v2:172.21.15.170:3300/0 conn(0x7f502015b610 0x7f502006ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:31.293 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.289+0000 7f5028610640 1 -- 172.21.15.169:0/2743978984 >> 172.21.15.169:0/2743978984 conn(0x7f5020069930 msgr2=0x7f5020069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:31.293 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.290+0000 7f5028610640 1 -- 172.21.15.169:0/2743978984 shutdown_connections 2023-10-26T01:33:31.293 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.290+0000 7f5028610640 1 -- 172.21.15.169:0/2743978984 wait complete. 2023-10-26T01:33:31.295 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.291+0000 7f5028610640 1 Processor -- start 2023-10-26T01:33:31.295 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.291+0000 7f5028610640 1 -- start start 2023-10-26T01:33:31.295 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.292+0000 7f5028610640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f50200ac4e0 0x7f5020099aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:31.295 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.292+0000 7f5028610640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f50200cdd80 0x7f5020152b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:31.295 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.292+0000 7f5028610640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f502015b610 0x7f50201584b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:31.296 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.292+0000 7f5028610640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5020099fe0 con 0x7f502015b610 2023-10-26T01:33:31.296 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.292+0000 7f5028610640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50201530a0 con 0x7f50200ac4e0 2023-10-26T01:33:31.296 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.292+0000 7f5028610640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f502009b260 con 0x7f50200cdd80 2023-10-26T01:33:31.296 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.293+0000 7f5026bad640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f502015b610 0x7f50201584b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:31.296 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.293+0000 7f5026bad640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f502015b610 0x7f50201584b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:42858/0 (socket says 172.21.15.169:42858) 2023-10-26T01:33:31.297 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.293+0000 7f5026bad640 1 -- 172.21.15.169:0/74955684 learned_addr learned my addr 172.21.15.169:0/74955684 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:33:31.297 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.293+0000 7f50263ac640 1 --2- 172.21.15.169:0/74955684 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f50200ac4e0 0x7f5020099aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:31.297 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.293+0000 7f5026bad640 1 -- 172.21.15.169:0/74955684 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f50200cdd80 msgr2=0x7f5020152b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:31.297 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.293+0000 7f5026bad640 1 --2- 172.21.15.169:0/74955684 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f50200cdd80 0x7f5020152b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:31.297 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.293+0000 7f5026bad640 1 -- 172.21.15.169:0/74955684 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f50200ac4e0 msgr2=0x7f5020099aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:31.298 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.293+0000 7f5026bad640 1 --2- 172.21.15.169:0/74955684 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f50200ac4e0 0x7f5020099aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:31.298 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.293+0000 7f5026bad640 1 -- 172.21.15.169:0/74955684 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5018009840 con 0x7f502015b610 2023-10-26T01:33:31.298 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.293+0000 7f50263ac640 1 --2- 172.21.15.169:0/74955684 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f50200ac4e0 0x7f5020099aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:33:31.298 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.293+0000 7f5026bad640 1 --2- 172.21.15.169:0/74955684 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f502015b610 0x7f50201584b0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f501802c720 tx=0x7f50180042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:31.299 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.294+0000 7f5016ffd640 1 -- 172.21.15.169:0/74955684 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50180046f0 con 0x7f502015b610 2023-10-26T01:33:31.299 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.294+0000 7f5016ffd640 1 -- 172.21.15.169:0/74955684 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5018004890 con 0x7f502015b610 2023-10-26T01:33:31.300 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.294+0000 7f5016ffd640 1 -- 172.21.15.169:0/74955684 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50180347e0 con 0x7f502015b610 2023-10-26T01:33:31.300 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.294+0000 7f5028610640 1 -- 172.21.15.169:0/74955684 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50201589f0 con 0x7f502015b610 2023-10-26T01:33:31.300 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.294+0000 7f5028610640 1 -- 172.21.15.169:0/74955684 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f502009a780 con 0x7f502015b610 2023-10-26T01:33:31.300 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.295+0000 7f5028610640 1 -- 172.21.15.169:0/74955684 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f502004db10 con 0x7f502015b610 2023-10-26T01:33:31.300 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.296+0000 7f5016ffd640 1 -- 172.21.15.169:0/74955684 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f5018034940 con 0x7f502015b610 2023-10-26T01:33:31.300 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.297+0000 7f5016ffd640 1 --2- 172.21.15.169:0/74955684 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f500406ae40 0x7f500406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:31.301 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.297+0000 7f5016ffd640 1 -- 172.21.15.169:0/74955684 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f50180ac270 con 0x7f502015b610 2023-10-26T01:33:31.301 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.297+0000 7f50263ac640 1 --2- 172.21.15.169:0/74955684 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f500406ae40 0x7f500406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:31.301 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.298+0000 7f50263ac640 1 --2- 172.21.15.169:0/74955684 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f500406ae40 0x7f500406d300 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f501c009aa0 tx=0x7f501c005db0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:31.302 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.301+0000 7f5016ffd640 1 -- 172.21.15.169:0/74955684 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5018077620 con 0x7f502015b610 2023-10-26T01:33:31.482 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.478+0000 7f5028610640 1 -- 172.21.15.169:0/74955684 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f502006c990 con 0x7f502015b610 2023-10-26T01:33:31.482 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.479+0000 7f5016ffd640 1 -- 172.21.15.169:0/74955684 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) v1 ==== 74+0+13036 (secure 0 0 0) 0x7f501807ac40 con 0x7f502015b610 2023-10-26T01:33:31.482 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:33:31.483 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":57,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:33:30.024718+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.169:6835/2816266753":"2023-10-27T01:33:26.694538+0000","172.21.15.169:6834/2816266753":"2023-10-27T01:33:26.694538+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:33:31.484 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.482+0000 7f5028610640 1 -- 172.21.15.169:0/74955684 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f500406ae40 msgr2=0x7f500406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:31.484 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.482+0000 7f5028610640 1 --2- 172.21.15.169:0/74955684 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f500406ae40 0x7f500406d300 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f501c009aa0 tx=0x7f501c005db0 comp rx=0 tx=0).stop 2023-10-26T01:33:31.485 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.482+0000 7f5028610640 1 -- 172.21.15.169:0/74955684 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f502015b610 msgr2=0x7f50201584b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:31.485 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.482+0000 7f5028610640 1 --2- 172.21.15.169:0/74955684 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f502015b610 0x7f50201584b0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f501802c720 tx=0x7f50180042f0 comp rx=0 tx=0).stop 2023-10-26T01:33:31.485 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.482+0000 7f5028610640 1 -- 172.21.15.169:0/74955684 shutdown_connections 2023-10-26T01:33:31.485 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.482+0000 7f5028610640 1 --2- 172.21.15.169:0/74955684 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f500406ae40 0x7f500406d300 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:31.485 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.482+0000 7f5028610640 1 --2- 172.21.15.169:0/74955684 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f502015b610 0x7f50201584b0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:31.486 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.482+0000 7f5028610640 1 --2- 172.21.15.169:0/74955684 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f50200cdd80 0x7f5020152b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:31.486 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.482+0000 7f5028610640 1 --2- 172.21.15.169:0/74955684 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f50200ac4e0 0x7f5020099aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:31.486 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.482+0000 7f5028610640 1 -- 172.21.15.169:0/74955684 >> 172.21.15.169:0/74955684 conn(0x7f5020069930 msgr2=0x7f50200ade00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:31.486 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.483+0000 7f5028610640 1 -- 172.21.15.169:0/74955684 shutdown_connections 2023-10-26T01:33:31.487 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.483+0000 7f5028610640 1 -- 172.21.15.169:0/74955684 wait complete. 2023-10-26T01:33:31.502 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T01:30:32.200808+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T01:33:31.503 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T01:33:31.711 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.709+0000 7fe81619e640 1 -- 172.21.15.169:0/2501337813 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8100b1360 msgr2=0x7fe81015be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:31.712 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.709+0000 7fe81619e640 1 --2- 172.21.15.169:0/2501337813 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8100b1360 0x7fe81015be60 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fe81006a480 tx=0x7fe80802bc80 comp rx=0 tx=0).stop 2023-10-26T01:33:31.712 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.709+0000 7fe81619e640 1 -- 172.21.15.169:0/2501337813 shutdown_connections 2023-10-26T01:33:31.712 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.709+0000 7fe81619e640 1 --2- 172.21.15.169:0/2501337813 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8100b1360 0x7fe81015be60 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:31.712 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.709+0000 7fe81619e640 1 --2- 172.21.15.169:0/2501337813 >> v2:172.21.15.170:3300/0 conn(0x7fe8100aaa40 0x7fe8100aae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:31.713 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.709+0000 7fe81619e640 1 -- 172.21.15.169:0/2501337813 >> 172.21.15.169:0/2501337813 conn(0x7fe810069930 msgr2=0x7fe810069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:31.713 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.710+0000 7fe81619e640 1 -- 172.21.15.169:0/2501337813 shutdown_connections 2023-10-26T01:33:31.713 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.710+0000 7fe81619e640 1 -- 172.21.15.169:0/2501337813 wait complete. 2023-10-26T01:33:31.715 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.711+0000 7fe81619e640 1 Processor -- start 2023-10-26T01:33:31.715 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.711+0000 7fe81619e640 1 -- start start 2023-10-26T01:33:31.715 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.712+0000 7fe81619e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe81006b6d0 0x7fe8101572d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:31.715 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.712+0000 7fe81619e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe8100aaa40 0x7fe810154d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:31.716 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.712+0000 7fe81619e640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe8100b1360 0x7fe8100af420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:31.716 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.712+0000 7fe81619e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe810155280 con 0x7fe81006b6d0 2023-10-26T01:33:31.716 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.712+0000 7fe81619e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe810154280 con 0x7fe8100aaa40 2023-10-26T01:33:31.716 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.712+0000 7fe81619e640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe81015ace0 con 0x7fe8100b1360 2023-10-26T01:33:31.716 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.712+0000 7fe80f7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe81006b6d0 0x7fe8101572d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:31.717 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.712+0000 7fe80f7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe81006b6d0 0x7fe8101572d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:42874/0 (socket says 172.21.15.169:42874) 2023-10-26T01:33:31.717 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.712+0000 7fe80f7fe640 1 -- 172.21.15.169:0/3289966987 learned_addr learned my addr 172.21.15.169:0/3289966987 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:33:31.717 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.713+0000 7fe80f7fe640 1 -- 172.21.15.169:0/3289966987 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe8100b1360 msgr2=0x7fe8100af420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:31.717 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.713+0000 7fe80f7fe640 1 --2- 172.21.15.169:0/3289966987 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe8100b1360 0x7fe8100af420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:31.717 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.713+0000 7fe80f7fe640 1 -- 172.21.15.169:0/3289966987 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe8100aaa40 msgr2=0x7fe810154d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:31.717 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.713+0000 7fe80f7fe640 1 --2- 172.21.15.169:0/3289966987 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe8100aaa40 0x7fe810154d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:31.718 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.713+0000 7fe80f7fe640 1 -- 172.21.15.169:0/3289966987 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe808009560 con 0x7fe81006b6d0 2023-10-26T01:33:31.718 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.713+0000 7fe80f7fe640 1 --2- 172.21.15.169:0/3289966987 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe81006b6d0 0x7fe8101572d0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fe80000ba50 tx=0x7fe80000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:31.718 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.713+0000 7fe807fff640 1 -- 172.21.15.169:0/3289966987 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe80000cbe0 con 0x7fe81006b6d0 2023-10-26T01:33:31.718 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.713+0000 7fe807fff640 1 -- 172.21.15.169:0/3289966987 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe80000ce50 con 0x7fe81006b6d0 2023-10-26T01:33:31.718 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.713+0000 7fe81619e640 1 -- 172.21.15.169:0/3289966987 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe81015afc0 con 0x7fe81006b6d0 2023-10-26T01:33:31.719 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.713+0000 7fe807fff640 1 -- 172.21.15.169:0/3289966987 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe800014680 con 0x7fe81006b6d0 2023-10-26T01:33:31.719 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.714+0000 7fe81619e640 1 -- 172.21.15.169:0/3289966987 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe810154820 con 0x7fe81006b6d0 2023-10-26T01:33:31.720 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.714+0000 7fe81619e640 1 -- 172.21.15.169:0/3289966987 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe810066af0 con 0x7fe81006b6d0 2023-10-26T01:33:31.720 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.715+0000 7fe807fff640 1 -- 172.21.15.169:0/3289966987 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fe80001d3b0 con 0x7fe81006b6d0 2023-10-26T01:33:31.720 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.716+0000 7fe807fff640 1 --2- 172.21.15.169:0/3289966987 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe7f006ae40 0x7fe7f006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:31.720 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.716+0000 7fe807fff640 1 -- 172.21.15.169:0/3289966987 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fe8000895e0 con 0x7fe81006b6d0 2023-10-26T01:33:31.721 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.717+0000 7fe80effd640 1 --2- 172.21.15.169:0/3289966987 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe7f006ae40 0x7fe7f006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:31.721 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.718+0000 7fe80effd640 1 --2- 172.21.15.169:0/3289966987 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe7f006ae40 0x7fe7f006d300 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fe81014be50 tx=0x7fe7f8007990 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:31.722 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.721+0000 7fe807fff640 1 -- 172.21.15.169:0/3289966987 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe8000548f0 con 0x7fe81006b6d0 2023-10-26T01:33:31.912 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.911+0000 7fe81619e640 1 -- 172.21.15.169:0/3289966987 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe810154a40 con 0x7fe81006b6d0 2023-10-26T01:33:31.913 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.912+0000 7fe807fff640 1 -- 172.21.15.169:0/3289966987 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v57) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe8000190a0 con 0x7fe81006b6d0 2023-10-26T01:33:31.913 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 1 2023-10-26T01:33:31.916 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.914+0000 7fe81619e640 1 -- 172.21.15.169:0/3289966987 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe7f006ae40 msgr2=0x7fe7f006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:31.916 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.914+0000 7fe81619e640 1 --2- 172.21.15.169:0/3289966987 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe7f006ae40 0x7fe7f006d300 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fe81014be50 tx=0x7fe7f8007990 comp rx=0 tx=0).stop 2023-10-26T01:33:31.917 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.914+0000 7fe81619e640 1 -- 172.21.15.169:0/3289966987 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe81006b6d0 msgr2=0x7fe8101572d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:31.917 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.914+0000 7fe81619e640 1 --2- 172.21.15.169:0/3289966987 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe81006b6d0 0x7fe8101572d0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fe80000ba50 tx=0x7fe80000bf10 comp rx=0 tx=0).stop 2023-10-26T01:33:31.917 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.914+0000 7fe81619e640 1 -- 172.21.15.169:0/3289966987 shutdown_connections 2023-10-26T01:33:31.917 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.914+0000 7fe81619e640 1 --2- 172.21.15.169:0/3289966987 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe7f006ae40 0x7fe7f006d300 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:31.917 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.914+0000 7fe81619e640 1 --2- 172.21.15.169:0/3289966987 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe8100b1360 0x7fe8100af420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:31.917 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.914+0000 7fe81619e640 1 --2- 172.21.15.169:0/3289966987 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe8100aaa40 0x7fe810154d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:31.918 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.914+0000 7fe81619e640 1 --2- 172.21.15.169:0/3289966987 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe81006b6d0 0x7fe8101572d0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:31.918 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.914+0000 7fe81619e640 1 -- 172.21.15.169:0/3289966987 >> 172.21.15.169:0/3289966987 conn(0x7fe810069930 msgr2=0x7fe81014b350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:31.918 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.915+0000 7fe81619e640 1 -- 172.21.15.169:0/3289966987 shutdown_connections 2023-10-26T01:33:31.918 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:31.915+0000 7fe81619e640 1 -- 172.21.15.169:0/3289966987 wait complete. 2023-10-26T01:33:31.935 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestClusterFull.test_full_fclose' 2023-10-26T01:33:32.157 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:32.155+0000 7f5980f97640 1 -- 172.21.15.169:0/2624188512 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f59680039a0 msgr2=0x7f5968003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:32.157 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:32.155+0000 7f5980f97640 1 --2- 172.21.15.169:0/2624188512 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f59680039a0 0x7f5968003dc0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f596c00bb20 tx=0x7f59680057f0 comp rx=0 tx=0).stop 2023-10-26T01:33:32.158 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:32.155+0000 7f597ad9d640 1 -- 172.21.15.169:0/2624188512 reap_dead start 2023-10-26T01:33:32.158 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:32.155+0000 7f5980f97640 1 -- 172.21.15.169:0/2624188512 shutdown_connections 2023-10-26T01:33:32.158 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:32.155+0000 7f5980f97640 1 -- 172.21.15.169:0/2624188512 >> 172.21.15.169:0/2624188512 conn(0x7f597c069510 msgr2=0x7f597c0698f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:32.158 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:32.155+0000 7f5980f97640 1 -- 172.21.15.169:0/2624188512 shutdown_connections 2023-10-26T01:33:32.159 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:32.155+0000 7f5980f97640 1 -- 172.21.15.169:0/2624188512 wait complete. 2023-10-26T01:33:32.159 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:32.157+0000 7f5980f97640 1 Processor -- start 2023-10-26T01:33:32.159 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:32.157+0000 7f5980f97640 1 -- start start 2023-10-26T01:33:32.161 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:32.158+0000 7f5980f97640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f59680039a0 0x7f597c0b1980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:32.161 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:32.158+0000 7f5980f97640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f597c13cf30 0x7f597c13d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:32.161 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:32.158+0000 7f5980f97640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f597c098a60 0x7f597c098ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:32.162 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:32.158+0000 7f5980f97640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f597c1f5d80 con 0x7f59680039a0 2023-10-26T01:33:32.162 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:32.158+0000 7f5980f97640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f597c1f6060 con 0x7f597c13cf30 2023-10-26T01:33:32.162 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:32.158+0000 7f5980f97640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f597c1f6340 con 0x7f597c098a60 2023-10-26T01:33:32.162 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:32.158+0000 7f597a59c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f59680039a0 0x7f597c0b1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:32.163 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:32.158+0000 7f597a59c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f59680039a0 0x7f597c0b1980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:42890/0 (socket says 172.21.15.169:42890) 2023-10-26T01:33:32.163 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:32.158+0000 7f597a59c640 1 -- 172.21.15.169:0/553235643 learned_addr learned my addr 172.21.15.169:0/553235643 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:33:32.163 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:32.159+0000 7f597a59c640 1 -- 172.21.15.169:0/553235643 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f597c098a60 msgr2=0x7f597c098ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:32.163 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:32.159+0000 7f5973fff640 1 --2- 172.21.15.169:0/553235643 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f597c13cf30 0x7f597c13d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:32.164 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:32.159+0000 7f597a59c640 1 --2- 172.21.15.169:0/553235643 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f597c098a60 0x7f597c098ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:32.164 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:32.159+0000 7f597a59c640 1 -- 172.21.15.169:0/553235643 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f597c13cf30 msgr2=0x7f597c13d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:32.164 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:32.159+0000 7f597a59c640 1 --2- 172.21.15.169:0/553235643 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f597c13cf30 0x7f597c13d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:32.164 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:32.159+0000 7f597a59c640 1 -- 172.21.15.169:0/553235643 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f597c1f6620 con 0x7f59680039a0 2023-10-26T01:33:32.164 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:32.159+0000 7f5973fff640 1 --2- 172.21.15.169:0/553235643 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f597c13cf30 0x7f597c13d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:33:32.165 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:32.159+0000 7f597a59c640 1 --2- 172.21.15.169:0/553235643 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f59680039a0 0x7f597c0b1980 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f596c00ba40 tx=0x7f596c036720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:32.165 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:32.159+0000 7f5972ffd640 1 -- 172.21.15.169:0/553235643 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f596c036de0 con 0x7f59680039a0 2023-10-26T01:33:32.165 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:32.159+0000 7f5972ffd640 1 -- 172.21.15.169:0/553235643 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f596c004120 con 0x7f59680039a0 2023-10-26T01:33:32.165 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:32.159+0000 7f5972ffd640 1 -- 172.21.15.169:0/553235643 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f596c0035f0 con 0x7f59680039a0 2023-10-26T01:33:32.165 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:32.159+0000 7f5980f97640 1 -- 172.21.15.169:0/553235643 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f597c1f6900 con 0x7f59680039a0 2023-10-26T01:33:32.166 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:32.160+0000 7f5980f97640 1 -- 172.21.15.169:0/553235643 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f597c0b1ec0 con 0x7f59680039a0 2023-10-26T01:33:32.167 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:32.161+0000 7f5980f97640 1 -- 172.21.15.169:0/553235643 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f597c0b20b0 con 0x7f59680039a0 2023-10-26T01:33:32.167 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:32.162+0000 7f5972ffd640 1 -- 172.21.15.169:0/553235643 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f596c003750 con 0x7f59680039a0 2023-10-26T01:33:32.167 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:32.163+0000 7f5972ffd640 1 --2- 172.21.15.169:0/553235643 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f595c06ae40 0x7f595c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:32.167 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:32.163+0000 7f5972ffd640 1 -- 172.21.15.169:0/553235643 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f596c0ab170 con 0x7f59680039a0 2023-10-26T01:33:32.168 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:32.164+0000 7f5973fff640 1 --2- 172.21.15.169:0/553235643 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f595c06ae40 0x7f595c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:32.168 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:32.165+0000 7f5973fff640 1 --2- 172.21.15.169:0/553235643 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f595c06ae40 0x7f595c06d300 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f5964000c00 tx=0x7f5964006a00 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:32.169 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:32.168+0000 7f5972ffd640 1 -- 172.21.15.169:0/553235643 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f596c076480 con 0x7f59680039a0 2023-10-26T01:33:32.409 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:32.408+0000 7f5980f97640 1 -- 172.21.15.169:0/553235643 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_fclose"]} v 0) v1 -- 0x7f597c04db10 con 0x7f59680039a0 2023-10-26T01:33:33.060 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:33.058+0000 7f5972ffd640 1 -- 172.21.15.169:0/553235643 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_fclose"]}]=0 v172) v1 ==== 137+0+0 (secure 0 0 0) 0x7f596c079aa0 con 0x7f59680039a0 2023-10-26T01:33:33.063 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:33.062+0000 7f5980f97640 1 -- 172.21.15.169:0/553235643 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f595c06ae40 msgr2=0x7f595c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:33.064 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:33.062+0000 7f5980f97640 1 --2- 172.21.15.169:0/553235643 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f595c06ae40 0x7f595c06d300 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f5964000c00 tx=0x7f5964006a00 comp rx=0 tx=0).stop 2023-10-26T01:33:33.064 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:33.062+0000 7f5980f97640 1 -- 172.21.15.169:0/553235643 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f59680039a0 msgr2=0x7f597c0b1980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:33.064 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:33.062+0000 7f5980f97640 1 --2- 172.21.15.169:0/553235643 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f59680039a0 0x7f597c0b1980 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f596c00ba40 tx=0x7f596c036720 comp rx=0 tx=0).stop 2023-10-26T01:33:33.064 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:33.062+0000 7f5980f97640 1 -- 172.21.15.169:0/553235643 shutdown_connections 2023-10-26T01:33:33.065 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:33.062+0000 7f5980f97640 1 --2- 172.21.15.169:0/553235643 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f595c06ae40 0x7f595c06d300 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:33.065 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:33.062+0000 7f5980f97640 1 --2- 172.21.15.169:0/553235643 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f597c098a60 0x7f597c098ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:33.065 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:33.062+0000 7f5980f97640 1 --2- 172.21.15.169:0/553235643 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f597c13cf30 0x7f597c13d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:33.065 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:33.062+0000 7f5980f97640 1 --2- 172.21.15.169:0/553235643 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f59680039a0 0x7f597c0b1980 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:33.065 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:33.062+0000 7f5980f97640 1 -- 172.21.15.169:0/553235643 >> 172.21.15.169:0/553235643 conn(0x7f597c069510 msgr2=0x7f597c0aad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:33.066 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:33.062+0000 7f5980f97640 1 -- 172.21.15.169:0/553235643 shutdown_connections 2023-10-26T01:33:33.066 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:33.062+0000 7f5980f97640 1 -- 172.21.15.169:0/553235643 wait complete. 2023-10-26T01:33:33.086 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_objectstore 2023-10-26T01:33:33.315 INFO:teuthology.orchestra.run.smithi169.stdout:{"osd_objectstore":"memstore"} 2023-10-26T01:33:33.315 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_objectstore": "memstore" } 2023-10-26T01:33:33.316 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2023-10-26T01:33:33.536 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:33.534+0000 7f17d7b55640 1 -- 172.21.15.169:0/2085056286 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17d01593e0 msgr2=0x7f17d00ac200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:33.536 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:33.534+0000 7f17d7b55640 1 --2- 172.21.15.169:0/2085056286 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17d01593e0 0x7f17d00ac200 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f17d00b2670 tx=0x7f17cc02bd90 comp rx=0 tx=0).stop 2023-10-26T01:33:33.536 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:33.534+0000 7f17d7b55640 1 -- 172.21.15.169:0/2085056286 shutdown_connections 2023-10-26T01:33:33.537 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:33.534+0000 7f17d7b55640 1 --2- 172.21.15.169:0/2085056286 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f17d00b0c80 0x7f17d00cd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:33.537 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:33.534+0000 7f17d7b55640 1 --2- 172.21.15.169:0/2085056286 >> v2:172.21.15.170:3300/0 conn(0x7f17d01597c0 0x7f17d015dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:33.537 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:33.534+0000 7f17d7b55640 1 --2- 172.21.15.169:0/2085056286 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17d01593e0 0x7f17d00ac200 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:33.537 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:33.534+0000 7f17d7b55640 1 -- 172.21.15.169:0/2085056286 >> 172.21.15.169:0/2085056286 conn(0x7f17d0069930 msgr2=0x7f17d0069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:33.537 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:33.534+0000 7f17d7b55640 1 -- 172.21.15.169:0/2085056286 shutdown_connections 2023-10-26T01:33:33.538 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:33.534+0000 7f17d7b55640 1 -- 172.21.15.169:0/2085056286 wait complete. 2023-10-26T01:33:33.539 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:33.536+0000 7f17d7b55640 1 Processor -- start 2023-10-26T01:33:33.540 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:33.536+0000 7f17d7b55640 1 -- start start 2023-10-26T01:33:33.540 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:33.537+0000 7f17d7b55640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f17d00b0c80 0x7f17d01f3c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:33.540 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:33.537+0000 7f17d7b55640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17d01593e0 0x7f17d00adc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:33.540 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:33.537+0000 7f17d7b55640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f17d01597c0 0x7f17d006ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:33.541 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:33.537+0000 7f17d7b55640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17d006f480 con 0x7f17d01593e0 2023-10-26T01:33:33.541 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:33.537+0000 7f17d7b55640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17d006ff50 con 0x7f17d01597c0 2023-10-26T01:33:33.541 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:33.537+0000 7f17d7b55640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17d006f6b0 con 0x7f17d00b0c80 2023-10-26T01:33:33.541 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:33.537+0000 7f17d50f0640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17d01593e0 0x7f17d00adc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:33.541 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:33.537+0000 7f17d50f0640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17d01593e0 0x7f17d00adc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:42916/0 (socket says 172.21.15.169:42916) 2023-10-26T01:33:33.542 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:33.537+0000 7f17d50f0640 1 -- 172.21.15.169:0/1001051239 learned_addr learned my addr 172.21.15.169:0/1001051239 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:33:33.542 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:33.537+0000 7f17d58f1640 1 --2- 172.21.15.169:0/1001051239 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f17d00b0c80 0x7f17d01f3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:33.542 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:33.537+0000 7f17d50f0640 1 -- 172.21.15.169:0/1001051239 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f17d00b0c80 msgr2=0x7f17d01f3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:33.542 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:33.537+0000 7f17d50f0640 1 --2- 172.21.15.169:0/1001051239 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f17d00b0c80 0x7f17d01f3c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:33.543 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:33.537+0000 7f17d50f0640 1 -- 172.21.15.169:0/1001051239 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f17d01597c0 msgr2=0x7f17d006ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:33.543 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:33.537+0000 7f17d50f0640 1 --2- 172.21.15.169:0/1001051239 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f17d01597c0 0x7f17d006ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:33.543 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:33.537+0000 7f17d50f0640 1 -- 172.21.15.169:0/1001051239 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17cc009560 con 0x7f17d01593e0 2023-10-26T01:33:33.543 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:33.538+0000 7f17d50f0640 1 --2- 172.21.15.169:0/1001051239 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17d01593e0 0x7f17d00adc20 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f17c000b410 tx=0x7f17c000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:33.543 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:33.538+0000 7f17c67fc640 1 -- 172.21.15.169:0/1001051239 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c0011830 con 0x7f17d01593e0 2023-10-26T01:33:33.544 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:33.538+0000 7f17c67fc640 1 -- 172.21.15.169:0/1001051239 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f17c0011aa0 con 0x7f17d01593e0 2023-10-26T01:33:33.544 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:33.538+0000 7f17d7b55640 1 -- 172.21.15.169:0/1001051239 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17d006f990 con 0x7f17d01593e0 2023-10-26T01:33:33.544 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:33.538+0000 7f17c67fc640 1 -- 172.21.15.169:0/1001051239 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c0010380 con 0x7f17d01593e0 2023-10-26T01:33:33.544 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:33.538+0000 7f17d7b55640 1 -- 172.21.15.169:0/1001051239 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17d014e9f0 con 0x7f17d01593e0 2023-10-26T01:33:33.545 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:33.539+0000 7f17d7b55640 1 -- 172.21.15.169:0/1001051239 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17d014e850 con 0x7f17d01593e0 2023-10-26T01:33:33.545 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:33.541+0000 7f17c67fc640 1 -- 172.21.15.169:0/1001051239 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f17c00104e0 con 0x7f17d01593e0 2023-10-26T01:33:33.545 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:33.541+0000 7f17c67fc640 1 --2- 172.21.15.169:0/1001051239 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f17b406ae40 0x7f17b406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:33.546 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:33.542+0000 7f17c67fc640 1 -- 172.21.15.169:0/1001051239 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f17c0089730 con 0x7f17d01593e0 2023-10-26T01:33:33.546 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:33.542+0000 7f17d58f1640 1 --2- 172.21.15.169:0/1001051239 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f17b406ae40 0x7f17b406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:33.546 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:33.543+0000 7f17d58f1640 1 --2- 172.21.15.169:0/1001051239 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f17b406ae40 0x7f17b406d300 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f17cc035000 tx=0x7f17cc004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:33.548 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:33.546+0000 7f17c67fc640 1 -- 172.21.15.169:0/1001051239 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f17d014e850 con 0x7f17d01593e0 2023-10-26T01:33:33.739 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:33.737+0000 7f17d7b55640 1 -- 172.21.15.169:0/1001051239 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "config dump"} v 0) v1 -- 0x7f17d014e850 con 0x7f17d01593e0 2023-10-26T01:33:33.740 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:33.738+0000 7f17c67fc640 1 -- 172.21.15.169:0/1001051239 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v22) v1 ==== 59+0+1116 (secure 0 0 0) 0x7f17c0015470 con 0x7f17d01593e0 2023-10-26T01:33:33.740 INFO:teuthology.orchestra.run.smithi169.stdout:WHO MASK LEVEL OPTION VALUE RO 2023-10-26T01:33:33.740 INFO:teuthology.orchestra.run.smithi169.stdout:mon advanced mon_op_complaint_time 120 2023-10-26T01:33:33.740 INFO:teuthology.orchestra.run.smithi169.stdout:osd advanced osd_op_complaint_time 180.000000 2023-10-26T01:33:33.741 INFO:teuthology.orchestra.run.smithi169.stdout:mds advanced debug_mds 20/20 2023-10-26T01:33:33.741 INFO:teuthology.orchestra.run.smithi169.stdout:mds advanced debug_mds_balancer 20/20 2023-10-26T01:33:33.741 INFO:teuthology.orchestra.run.smithi169.stdout:mds advanced debug_ms 1/1 2023-10-26T01:33:33.741 INFO:teuthology.orchestra.run.smithi169.stdout:mds dev mds_debug_frag true 2023-10-26T01:33:33.741 INFO:teuthology.orchestra.run.smithi169.stdout:mds dev mds_debug_scatterstat true 2023-10-26T01:33:33.742 INFO:teuthology.orchestra.run.smithi169.stdout:mds advanced mds_op_complaint_time 180.000000 2023-10-26T01:33:33.742 INFO:teuthology.orchestra.run.smithi169.stdout:mds dev mds_verify_scatter true 2023-10-26T01:33:33.742 INFO:teuthology.orchestra.run.smithi169.stdout:mds advanced osd_op_complaint_time 180.000000 2023-10-26T01:33:33.742 INFO:teuthology.orchestra.run.smithi169.stdout:mds advanced rados_mon_op_timeout 900 2023-10-26T01:33:33.742 INFO:teuthology.orchestra.run.smithi169.stdout:mds advanced rados_osd_op_timeout 900 2023-10-26T01:33:33.743 INFO:teuthology.orchestra.run.smithi169.stdout:client advanced client_mount_timeout 600 2023-10-26T01:33:33.743 INFO:teuthology.orchestra.run.smithi169.stdout:client advanced debug_client 20/20 2023-10-26T01:33:33.743 INFO:teuthology.orchestra.run.smithi169.stdout:client advanced debug_ms 1/1 2023-10-26T01:33:33.743 INFO:teuthology.orchestra.run.smithi169.stdout:client advanced rados_mon_op_timeout 900 2023-10-26T01:33:33.743 INFO:teuthology.orchestra.run.smithi169.stdout:client advanced rados_osd_op_timeout 900 2023-10-26T01:33:33.744 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:33.741+0000 7f17d7b55640 1 -- 172.21.15.169:0/1001051239 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f17b406ae40 msgr2=0x7f17b406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:33.745 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:33.741+0000 7f17d7b55640 1 --2- 172.21.15.169:0/1001051239 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f17b406ae40 0x7f17b406d300 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f17cc035000 tx=0x7f17cc004730 comp rx=0 tx=0).stop 2023-10-26T01:33:33.745 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:33.741+0000 7f17d7b55640 1 -- 172.21.15.169:0/1001051239 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17d01593e0 msgr2=0x7f17d00adc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:33.745 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:33.741+0000 7f17d7b55640 1 --2- 172.21.15.169:0/1001051239 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17d01593e0 0x7f17d00adc20 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f17c000b410 tx=0x7f17c000b8d0 comp rx=0 tx=0).stop 2023-10-26T01:33:33.745 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:33.741+0000 7f17d7b55640 1 -- 172.21.15.169:0/1001051239 shutdown_connections 2023-10-26T01:33:33.746 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:33.741+0000 7f17d7b55640 1 --2- 172.21.15.169:0/1001051239 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f17b406ae40 0x7f17b406d300 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:33.746 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:33.741+0000 7f17d7b55640 1 --2- 172.21.15.169:0/1001051239 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17d01593e0 0x7f17d00adc20 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:33.746 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:33.741+0000 7f17d7b55640 1 --2- 172.21.15.169:0/1001051239 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f17d01597c0 0x7f17d006ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:33.746 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:33.741+0000 7f17d7b55640 1 --2- 172.21.15.169:0/1001051239 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f17d00b0c80 0x7f17d01f3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:33.746 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:33.741+0000 7f17d7b55640 1 -- 172.21.15.169:0/1001051239 >> 172.21.15.169:0/1001051239 conn(0x7f17d0069930 msgr2=0x7f17d00cb760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:33.747 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:33.741+0000 7f17d7b55640 1 -- 172.21.15.169:0/1001051239 shutdown_connections 2023-10-26T01:33:33.747 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:33.741+0000 7f17d7b55640 1 -- 172.21.15.169:0/1001051239 wait complete. 2023-10-26T01:33:33.761 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-26T01:33:33.984 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:33.982+0000 7fb2df59e640 1 -- 172.21.15.169:0/4196745025 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb2e0069a10 msgr2=0x7fb2e0069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:33.985 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:33.982+0000 7fb2df59e640 1 --2- 172.21.15.169:0/4196745025 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb2e0069a10 0x7fb2e0069ea0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb2d8005fd0 tx=0x7fb2d802c1b0 comp rx=0 tx=0).stop 2023-10-26T01:33:33.985 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:33.982+0000 7fb2df59e640 1 -- 172.21.15.169:0/4196745025 shutdown_connections 2023-10-26T01:33:33.985 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:33.982+0000 7fb2df59e640 1 --2- 172.21.15.169:0/4196745025 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb2e0069a10 0x7fb2e0069ea0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:33.985 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:33.982+0000 7fb2df59e640 1 --2- 172.21.15.169:0/4196745025 >> v2:172.21.15.170:3300/0 conn(0x7fb2e0158200 0x7fb2e006f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:33.986 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:33.982+0000 7fb2df59e640 1 -- 172.21.15.169:0/4196745025 >> 172.21.15.169:0/4196745025 conn(0x7fb2e00af110 msgr2=0x7fb2e00af520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:33.986 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:33.983+0000 7fb2df59e640 1 -- 172.21.15.169:0/4196745025 shutdown_connections 2023-10-26T01:33:33.986 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:33.983+0000 7fb2df59e640 1 -- 172.21.15.169:0/4196745025 wait complete. 2023-10-26T01:33:33.987 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:33.984+0000 7fb2df59e640 1 Processor -- start 2023-10-26T01:33:33.987 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:33.984+0000 7fb2df59e640 1 -- start start 2023-10-26T01:33:33.988 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:33.985+0000 7fb2df59e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb2e0069a10 0x7fb2e01f5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:33.988 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:33.985+0000 7fb2df59e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb2e006bbd0 0x7fb2e014d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:33.988 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:33.985+0000 7fb2df59e640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb2e0158200 0x7fb2e014ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:33.988 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:33.985+0000 7fb2df59e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e0152b70 con 0x7fb2e0069a10 2023-10-26T01:33:33.988 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:33.985+0000 7fb2df59e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e01f5fd0 con 0x7fb2e006bbd0 2023-10-26T01:33:33.989 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:33.985+0000 7fb2df59e640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e01f5a10 con 0x7fb2e0158200 2023-10-26T01:33:33.989 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:33.986+0000 7fb2de59c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb2e0069a10 0x7fb2e01f5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:33.989 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:33.986+0000 7fb2de59c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb2e0069a10 0x7fb2e01f5110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:53884/0 (socket says 172.21.15.169:53884) 2023-10-26T01:33:33.989 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:33.986+0000 7fb2de59c640 1 -- 172.21.15.169:0/1638758704 learned_addr learned my addr 172.21.15.169:0/1638758704 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:33:33.990 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:33.986+0000 7fb2de59c640 1 -- 172.21.15.169:0/1638758704 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb2e0158200 msgr2=0x7fb2e014ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:33.991 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:33.986+0000 7fb2de59c640 1 --2- 172.21.15.169:0/1638758704 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb2e0158200 0x7fb2e014ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:33.991 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:33.986+0000 7fb2de59c640 1 -- 172.21.15.169:0/1638758704 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb2e006bbd0 msgr2=0x7fb2e014d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:33.991 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:33.986+0000 7fb2de59c640 1 --2- 172.21.15.169:0/1638758704 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb2e006bbd0 0x7fb2e014d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:33.991 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:33.986+0000 7fb2de59c640 1 -- 172.21.15.169:0/1638758704 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2d8009900 con 0x7fb2e0069a10 2023-10-26T01:33:33.991 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:33.986+0000 7fb2de59c640 1 --2- 172.21.15.169:0/1638758704 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb2e0069a10 0x7fb2e01f5110 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fb2d000e790 tx=0x7fb2d000ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:33.992 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:33.987+0000 7fb2d77fe640 1 -- 172.21.15.169:0/1638758704 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d0009580 con 0x7fb2e0069a10 2023-10-26T01:33:33.992 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:33.987+0000 7fb2d77fe640 1 -- 172.21.15.169:0/1638758704 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb2d000cdb0 con 0x7fb2e0069a10 2023-10-26T01:33:33.992 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:33.987+0000 7fb2df59e640 1 -- 172.21.15.169:0/1638758704 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2e01f5cf0 con 0x7fb2e0069a10 2023-10-26T01:33:33.992 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:33.987+0000 7fb2d77fe640 1 -- 172.21.15.169:0/1638758704 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d0010690 con 0x7fb2e0069a10 2023-10-26T01:33:33.992 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:33.987+0000 7fb2df59e640 1 -- 172.21.15.169:0/1638758704 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2e0153060 con 0x7fb2e0069a10 2023-10-26T01:33:33.993 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:33.988+0000 7fb2df59e640 1 -- 172.21.15.169:0/1638758704 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2e00afdc0 con 0x7fb2e0069a10 2023-10-26T01:33:33.993 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:33.989+0000 7fb2d77fe640 1 -- 172.21.15.169:0/1638758704 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fb2d00107f0 con 0x7fb2e0069a10 2023-10-26T01:33:33.993 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:33.990+0000 7fb2d77fe640 1 --2- 172.21.15.169:0/1638758704 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb2c006ae40 0x7fb2c006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:33.994 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:33.990+0000 7fb2d77fe640 1 -- 172.21.15.169:0/1638758704 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fb2d0014070 con 0x7fb2e0069a10 2023-10-26T01:33:33.994 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:33.991+0000 7fb2d5d9b640 1 --2- 172.21.15.169:0/1638758704 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb2c006ae40 0x7fb2c006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:33.994 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:33.992+0000 7fb2d5d9b640 1 --2- 172.21.15.169:0/1638758704 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb2c006ae40 0x7fb2c006d300 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fb2e01f5070 tx=0x7fb2c8007910 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:33.996 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:33.994+0000 7fb2d77fe640 1 -- 172.21.15.169:0/1638758704 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb2d0051ca0 con 0x7fb2e0069a10 2023-10-26T01:33:34.185 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:34.183+0000 7fb2df59e640 1 -- 172.21.15.169:0/1638758704 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fb2e0159870 con 0x7fb2e0069a10 2023-10-26T01:33:34.191 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:34.189+0000 7fb2d77fe640 1 -- 172.21.15.169:0/1638758704 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v23)=0 v23) v1 ==== 122+0+0 (secure 0 0 0) 0x7fb2d0055140 con 0x7fb2e0069a10 2023-10-26T01:33:34.194 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:34.192+0000 7fb2df59e640 1 -- 172.21.15.169:0/1638758704 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb2c006ae40 msgr2=0x7fb2c006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:34.194 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:34.192+0000 7fb2df59e640 1 --2- 172.21.15.169:0/1638758704 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb2c006ae40 0x7fb2c006d300 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fb2e01f5070 tx=0x7fb2c8007910 comp rx=0 tx=0).stop 2023-10-26T01:33:34.195 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:34.192+0000 7fb2df59e640 1 -- 172.21.15.169:0/1638758704 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb2e0069a10 msgr2=0x7fb2e01f5110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:34.195 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:34.192+0000 7fb2df59e640 1 --2- 172.21.15.169:0/1638758704 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb2e0069a10 0x7fb2e01f5110 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fb2d000e790 tx=0x7fb2d000ec50 comp rx=0 tx=0).stop 2023-10-26T01:33:34.195 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:34.192+0000 7fb2df59e640 1 -- 172.21.15.169:0/1638758704 shutdown_connections 2023-10-26T01:33:34.195 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:34.192+0000 7fb2df59e640 1 --2- 172.21.15.169:0/1638758704 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb2c006ae40 0x7fb2c006d300 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:34.195 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:34.192+0000 7fb2df59e640 1 --2- 172.21.15.169:0/1638758704 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb2e0158200 0x7fb2e014ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:34.196 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:34.192+0000 7fb2df59e640 1 --2- 172.21.15.169:0/1638758704 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb2e006bbd0 0x7fb2e014d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:34.196 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:34.192+0000 7fb2df59e640 1 --2- 172.21.15.169:0/1638758704 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb2e0069a10 0x7fb2e01f5110 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:34.196 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:34.192+0000 7fb2df59e640 1 -- 172.21.15.169:0/1638758704 >> 172.21.15.169:0/1638758704 conn(0x7fb2e00af110 msgr2=0x7fb2e00acc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:34.196 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:34.192+0000 7fb2df59e640 1 -- 172.21.15.169:0/1638758704 shutdown_connections 2023-10-26T01:33:34.196 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:34.193+0000 7fb2df59e640 1 -- 172.21.15.169:0/1638758704 wait complete. 2023-10-26T01:33:34.214 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-26T01:33:34.215 DEBUG:teuthology.orchestra.run.smithi169:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-26T01:33:34.343 DEBUG:teuthology.orchestra.run.smithi170:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-26T01:33:34.475 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-26T01:33:34.476 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:33:34.476 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:33:34.538 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054301,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-26T01:33:34.539 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-26T01:33:34.539 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:33:34.598 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054303,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-26T01:33:34.599 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:33:34.599 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:33:34.659 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054301,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-26T01:33:34.660 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-26T01:33:34.660 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:33:34.720 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054303,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-26T01:33:34.721 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-26T01:33:34.721 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T01:33:34.932 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T01:33:34.941 DEBUG:teuthology.orchestra.run.smithi170:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T01:33:34.947 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:34.945+0000 7f3746fc4640 1 -- 172.21.15.169:0/2125244320 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f374009a5c0 msgr2=0x7f374009aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:34.948 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:34.945+0000 7f3746fc4640 1 --2- 172.21.15.169:0/2125244320 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f374009a5c0 0x7f374009aa70 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f37400cbd50 tx=0x7f373802c260 comp rx=0 tx=0).stop 2023-10-26T01:33:34.948 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:34.945+0000 7f3746fc4640 1 -- 172.21.15.169:0/2125244320 shutdown_connections 2023-10-26T01:33:34.949 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:34.945+0000 7f3746fc4640 1 --2- 172.21.15.169:0/2125244320 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f374009a5c0 0x7f374009aa70 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:34.949 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:34.945+0000 7f3746fc4640 1 --2- 172.21.15.169:0/2125244320 >> v2:172.21.15.170:3300/0 conn(0x7f37400c9970 0x7f3740098a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:34.950 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:34.945+0000 7f3746fc4640 1 -- 172.21.15.169:0/2125244320 >> 172.21.15.169:0/2125244320 conn(0x7f37400aedd0 msgr2=0x7f37400af1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:34.950 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:34.946+0000 7f3746fc4640 1 -- 172.21.15.169:0/2125244320 shutdown_connections 2023-10-26T01:33:34.954 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:34.946+0000 7f3746fc4640 1 -- 172.21.15.169:0/2125244320 wait complete. 2023-10-26T01:33:34.958 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:34.948+0000 7f3746fc4640 1 Processor -- start 2023-10-26T01:33:34.958 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:34.948+0000 7f3746fc4640 1 -- start start 2023-10-26T01:33:34.958 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:34.949+0000 7f3746fc4640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f374009a5c0 0x7f374006c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:34.959 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:34.949+0000 7f3746fc4640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37400ac6d0 0x7f37401e6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:34.959 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:34.949+0000 7f3746fc4640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f37400c9970 0x7f37401e6660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:34.959 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:34.949+0000 7f3746fc4640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37400717a0 con 0x7f37400ac6d0 2023-10-26T01:33:34.960 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:34.949+0000 7f3746fc4640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37401e6ba0 con 0x7f37400c9970 2023-10-26T01:33:34.960 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:34.949+0000 7f3746fc4640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37400714c0 con 0x7f374009a5c0 2023-10-26T01:33:34.960 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:34.949+0000 7f37457c1640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37400ac6d0 0x7f37401e6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:34.961 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:34.949+0000 7f37457c1640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37400ac6d0 0x7f37401e6120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:53902/0 (socket says 172.21.15.169:53902) 2023-10-26T01:33:34.961 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:34.949+0000 7f37457c1640 1 -- 172.21.15.169:0/1241867869 learned_addr learned my addr 172.21.15.169:0/1241867869 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:33:34.961 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:34.950+0000 7f37467c3640 1 --2- 172.21.15.169:0/1241867869 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f37400c9970 0x7f37401e6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:34.961 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:34.950+0000 7f3745fc2640 1 --2- 172.21.15.169:0/1241867869 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f374009a5c0 0x7f374006c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:34.961 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:34.950+0000 7f37457c1640 1 -- 172.21.15.169:0/1241867869 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f374009a5c0 msgr2=0x7f374006c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:34.962 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:34.950+0000 7f37457c1640 1 --2- 172.21.15.169:0/1241867869 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f374009a5c0 0x7f374006c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:34.962 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:34.950+0000 7f37457c1640 1 -- 172.21.15.169:0/1241867869 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f37400c9970 msgr2=0x7f37401e6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:34.962 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:34.950+0000 7f37457c1640 1 --2- 172.21.15.169:0/1241867869 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f37400c9970 0x7f37401e6660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:34.962 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:34.950+0000 7f37457c1640 1 -- 172.21.15.169:0/1241867869 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3738009900 con 0x7f37400ac6d0 2023-10-26T01:33:34.962 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:34.950+0000 7f37467c3640 1 --2- 172.21.15.169:0/1241867869 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f37400c9970 0x7f37401e6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T01:33:34.963 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:34.950+0000 7f37457c1640 1 --2- 172.21.15.169:0/1241867869 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37400ac6d0 0x7f37401e6120 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f373000b440 tx=0x7f373000b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:34.963 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:34.951+0000 7f37367fc640 1 -- 172.21.15.169:0/1241867869 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3730011890 con 0x7f37400ac6d0 2023-10-26T01:33:34.963 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:34.951+0000 7f37367fc640 1 -- 172.21.15.169:0/1241867869 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3730011ae0 con 0x7f37400ac6d0 2023-10-26T01:33:34.963 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:34.951+0000 7f37367fc640 1 -- 172.21.15.169:0/1241867869 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37300103e0 con 0x7f37400ac6d0 2023-10-26T01:33:34.963 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:34.951+0000 7f3746fc4640 1 -- 172.21.15.169:0/1241867869 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3740130620 con 0x7f37400ac6d0 2023-10-26T01:33:34.964 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:34.951+0000 7f3746fc4640 1 -- 172.21.15.169:0/1241867869 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3740130be0 con 0x7f37400ac6d0 2023-10-26T01:33:34.964 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:34.952+0000 7f3746fc4640 1 -- 172.21.15.169:0/1241867869 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3740130480 con 0x7f37400ac6d0 2023-10-26T01:33:34.964 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:34.953+0000 7f37367fc640 1 -- 172.21.15.169:0/1241867869 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f3730010540 con 0x7f37400ac6d0 2023-10-26T01:33:34.964 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:34.954+0000 7f37367fc640 1 --2- 172.21.15.169:0/1241867869 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f372406ae40 0x7f372406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:34.964 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:34.954+0000 7f37367fc640 1 -- 172.21.15.169:0/1241867869 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f3730089920 con 0x7f37400ac6d0 2023-10-26T01:33:34.965 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:34.955+0000 7f3745fc2640 1 --2- 172.21.15.169:0/1241867869 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f372406ae40 0x7f372406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:34.965 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:34.956+0000 7f3745fc2640 1 --2- 172.21.15.169:0/1241867869 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f372406ae40 0x7f372406d300 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f373c007fd0 tx=0x7f373c009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:34.965 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:34.959+0000 7f37367fc640 1 -- 172.21.15.169:0/1241867869 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3740130480 con 0x7f37400ac6d0 2023-10-26T01:33:35.195 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:35.193+0000 7f3746fc4640 1 -- 172.21.15.169:0/1241867869 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3740130480 con 0x7f37400ac6d0 2023-10-26T01:33:35.196 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:35.195+0000 7f37367fc640 1 -- 172.21.15.169:0/1241867869 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+3711 (secure 0 0 0) 0x7f3740130480 con 0x7f37400ac6d0 2023-10-26T01:33:35.197 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:33:35.197 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":25,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4805,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/2294107634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2294107634},{"type":"v1","addr":"172.21.15.170:6836","nonce":2294107634}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":4987,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2208026662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2208026662},{"type":"v1","addr":"172.21.15.169:6833","nonce":2208026662}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":4993,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/2630365058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":2630365058},{"type":"v1","addr":"172.21.15.170:6837","nonce":2630365058}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":6003,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/3498699566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":3498699566},{"type":"v1","addr":"172.21.15.169:6835","nonce":3498699566}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25}],"filesystems":[]} 2023-10-26T01:33:35.197 INFO:teuthology.orchestra.run.smithi169.stderr:dumped fsmap epoch 25 2023-10-26T01:33:35.199 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:35.197+0000 7f3746fc4640 1 -- 172.21.15.169:0/1241867869 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f372406ae40 msgr2=0x7f372406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:35.200 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:35.198+0000 7f3746fc4640 1 --2- 172.21.15.169:0/1241867869 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f372406ae40 0x7f372406d300 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f373c007fd0 tx=0x7f373c009040 comp rx=0 tx=0).stop 2023-10-26T01:33:35.200 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:35.198+0000 7f3746fc4640 1 -- 172.21.15.169:0/1241867869 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37400ac6d0 msgr2=0x7f37401e6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:35.200 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:35.198+0000 7f3746fc4640 1 --2- 172.21.15.169:0/1241867869 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37400ac6d0 0x7f37401e6120 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f373000b440 tx=0x7f373000b900 comp rx=0 tx=0).stop 2023-10-26T01:33:35.200 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:35.198+0000 7f3746fc4640 1 -- 172.21.15.169:0/1241867869 shutdown_connections 2023-10-26T01:33:35.201 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:35.198+0000 7f3746fc4640 1 --2- 172.21.15.169:0/1241867869 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f372406ae40 0x7f372406d300 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:35.201 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:35.198+0000 7f3746fc4640 1 --2- 172.21.15.169:0/1241867869 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37400ac6d0 0x7f37401e6120 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:35.201 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:35.198+0000 7f3746fc4640 1 --2- 172.21.15.169:0/1241867869 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f37400c9970 0x7f37401e6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:35.201 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:35.198+0000 7f3746fc4640 1 --2- 172.21.15.169:0/1241867869 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f374009a5c0 0x7f374006c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:35.201 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:35.198+0000 7f3746fc4640 1 -- 172.21.15.169:0/1241867869 >> 172.21.15.169:0/1241867869 conn(0x7f37400aedd0 msgr2=0x7f37400c6270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:35.202 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:35.198+0000 7f3746fc4640 1 -- 172.21.15.169:0/1241867869 shutdown_connections 2023-10-26T01:33:35.202 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:35.198+0000 7f3746fc4640 1 -- 172.21.15.169:0/1241867869 wait complete. 2023-10-26T01:33:35.220 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-26T01:33:35.220 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-26T01:33:35.221 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-26T01:33:35.221 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T01:33:35.221 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-26T01:33:35.221 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-26T01:33:35.222 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-26T01:33:35.222 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T01:33:35.222 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-26T01:33:35.223 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-26T01:33:35.223 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-26T01:33:35.223 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T01:33:35.223 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-26T01:33:35.224 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-26T01:33:35.224 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-26T01:33:35.224 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T01:33:35.241 INFO:tasks.ceph.mds.c:Stopped 2023-10-26T01:33:35.241 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-26T01:33:35.244 INFO:tasks.ceph.mds.a:Stopped 2023-10-26T01:33:35.244 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-26T01:33:35.251 INFO:tasks.ceph.mds.d:Stopped 2023-10-26T01:33:35.251 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-26T01:33:35.252 INFO:tasks.ceph.mds.b:Stopped 2023-10-26T01:33:35.252 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-26T01:33:35.294 INFO:tasks.ceph.mds.c:Started 2023-10-26T01:33:35.295 DEBUG:teuthology.parallel:result is None 2023-10-26T01:33:35.301 INFO:tasks.ceph.mds.d:Started 2023-10-26T01:33:35.301 DEBUG:teuthology.parallel:result is None 2023-10-26T01:33:35.302 INFO:tasks.ceph.mds.a:Started 2023-10-26T01:33:35.302 DEBUG:teuthology.parallel:result is None 2023-10-26T01:33:35.308 INFO:tasks.ceph.mds.b:Started 2023-10-26T01:33:35.309 DEBUG:teuthology.parallel:result is None 2023-10-26T01:33:35.309 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-26T01:33:35.450 INFO:tasks.ceph.mds.c.smithi169.stdout:starting mds.c at 2023-10-26T01:33:35.455 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2023-10-26T01:33:35.455 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2023-10-26T01:33:35.461 INFO:tasks.ceph.mds.a.smithi169.stdout:starting mds.a at 2023-10-26T01:33:35.465 INFO:tasks.ceph.mds.d.smithi170.stdout:starting mds.d at 2023-10-26T01:33:35.472 INFO:tasks.ceph.mds.b.smithi170.stdout:starting mds.b at 2023-10-26T01:33:35.500 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:35.498+0000 7f2af1239640 1 -- 172.21.15.169:0/2288434538 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2aec06cba0 msgr2=0x7f2aec15b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:35.500 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:35.498+0000 7f2af1239640 1 --2- 172.21.15.169:0/2288434538 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2aec06cba0 0x7f2aec15b140 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f2aec06c280 tx=0x7f2adc02bd10 comp rx=0 tx=0).stop 2023-10-26T01:33:35.500 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:35.498+0000 7f2af1239640 1 -- 172.21.15.169:0/2288434538 shutdown_connections 2023-10-26T01:33:35.500 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:35.498+0000 7f2af1239640 1 --2- 172.21.15.169:0/2288434538 >> v2:172.21.15.170:3300/0 conn(0x7f2aec069a10 0x7f2aec069e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:35.500 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:35.498+0000 7f2af1239640 1 --2- 172.21.15.169:0/2288434538 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2aec0b0300 0x7f2aec0b0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:35.501 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:35.498+0000 7f2af1239640 1 --2- 172.21.15.169:0/2288434538 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2aec06cba0 0x7f2aec15b140 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:35.501 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:35.498+0000 7f2af1239640 1 -- 172.21.15.169:0/2288434538 >> 172.21.15.169:0/2288434538 conn(0x7f2aec0af110 msgr2=0x7f2aec0af520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:35.501 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:35.498+0000 7f2af1239640 1 -- 172.21.15.169:0/2288434538 shutdown_connections 2023-10-26T01:33:35.501 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:35.498+0000 7f2af1239640 1 -- 172.21.15.169:0/2288434538 wait complete. 2023-10-26T01:33:35.502 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:35.499+0000 7f2af1239640 1 Processor -- start 2023-10-26T01:33:35.502 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:35.500+0000 7f2af1239640 1 -- start start 2023-10-26T01:33:35.503 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:35.500+0000 7f2af1239640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2aec069a10 0x7f2aec15adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:35.503 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:35.500+0000 7f2af1239640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2aec06cba0 0x7f2aec14c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:35.503 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:35.500+0000 7f2af1239640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2aec0b0300 0x7f2aec14eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:35.503 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:35.500+0000 7f2af1239640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aec150e30 con 0x7f2aec069a10 2023-10-26T01:33:35.504 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:35.500+0000 7f2af1239640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aec14d390 con 0x7f2aec0b0300 2023-10-26T01:33:35.504 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:35.500+0000 7f2af1239640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2aec150b50 con 0x7f2aec06cba0 2023-10-26T01:33:35.504 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:35.501+0000 7f2aebfff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2aec069a10 0x7f2aec15adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:35.504 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:35.501+0000 7f2aeb7fe640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2aec06cba0 0x7f2aec14c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:35.504 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:35.501+0000 7f2aebfff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2aec069a10 0x7f2aec15adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:53950/0 (socket says 172.21.15.169:53950) 2023-10-26T01:33:35.505 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:35.501+0000 7f2aebfff640 1 -- 172.21.15.169:0/1925264980 learned_addr learned my addr 172.21.15.169:0/1925264980 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:33:35.505 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:35.501+0000 7f2af0a38640 1 --2- 172.21.15.169:0/1925264980 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2aec0b0300 0x7f2aec14eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:35.506 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:35.501+0000 7f2aebfff640 1 -- 172.21.15.169:0/1925264980 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2aec06cba0 msgr2=0x7f2aec14c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:35.506 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:35.501+0000 7f2aebfff640 1 --2- 172.21.15.169:0/1925264980 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2aec06cba0 0x7f2aec14c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:35.506 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:35.501+0000 7f2aebfff640 1 -- 172.21.15.169:0/1925264980 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2aec0b0300 msgr2=0x7f2aec14eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:35.507 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:35.501+0000 7f2aebfff640 1 --2- 172.21.15.169:0/1925264980 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2aec0b0300 0x7f2aec14eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:35.507 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:35.501+0000 7f2aebfff640 1 -- 172.21.15.169:0/1925264980 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2adc009560 con 0x7f2aec069a10 2023-10-26T01:33:35.507 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:35.501+0000 7f2af0a38640 1 --2- 172.21.15.169:0/1925264980 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2aec0b0300 0x7f2aec14eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T01:33:35.507 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:35.501+0000 7f2aeb7fe640 1 --2- 172.21.15.169:0/1925264980 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2aec06cba0 0x7f2aec14c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T01:33:35.507 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:35.502+0000 7f2aebfff640 1 --2- 172.21.15.169:0/1925264980 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2aec069a10 0x7f2aec15adb0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f2adc009850 tx=0x7f2adc02dce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:35.508 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:35.502+0000 7f2ae8ff9640 1 -- 172.21.15.169:0/1925264980 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2adc005650 con 0x7f2aec069a10 2023-10-26T01:33:35.508 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:35.502+0000 7f2ae8ff9640 1 -- 172.21.15.169:0/1925264980 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2adc0038f0 con 0x7f2aec069a10 2023-10-26T01:33:35.508 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:35.502+0000 7f2ae8ff9640 1 -- 172.21.15.169:0/1925264980 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2adc003ce0 con 0x7f2aec069a10 2023-10-26T01:33:35.508 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:35.502+0000 7f2af1239640 1 -- 172.21.15.169:0/1925264980 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2aec14e010 con 0x7f2aec069a10 2023-10-26T01:33:35.508 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:35.502+0000 7f2af1239640 1 -- 172.21.15.169:0/1925264980 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2aec14e5d0 con 0x7f2aec069a10 2023-10-26T01:33:35.509 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:35.503+0000 7f2af1239640 1 -- 172.21.15.169:0/1925264980 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2aec14de70 con 0x7f2aec069a10 2023-10-26T01:33:35.509 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:35.504+0000 7f2ae8ff9640 1 -- 172.21.15.169:0/1925264980 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f2adc03e450 con 0x7f2aec069a10 2023-10-26T01:33:35.509 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:35.506+0000 7f2ae8ff9640 1 --2- 172.21.15.169:0/1925264980 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2ad806ae40 0x7f2ad806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:35.510 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:35.506+0000 7f2ae8ff9640 1 -- 172.21.15.169:0/1925264980 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f2adc0abc70 con 0x7f2aec069a10 2023-10-26T01:33:35.510 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:35.506+0000 7f2aeb7fe640 1 --2- 172.21.15.169:0/1925264980 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2ad806ae40 0x7f2ad806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:35.510 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:35.507+0000 7f2aeb7fe640 1 --2- 172.21.15.169:0/1925264980 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2ad806ae40 0x7f2ad806d300 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f2aec071ab0 tx=0x7f2ae0009110 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:35.512 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:35.511+0000 7f2ae8ff9640 1 -- 172.21.15.169:0/1925264980 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2aec14de70 con 0x7f2aec069a10 2023-10-26T01:33:35.706 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:35.704+0000 7f2af1239640 1 -- 172.21.15.169:0/1925264980 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f2aec14de70 con 0x7f2aec069a10 2023-10-26T01:33:36.222 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:36.220+0000 7f2ae8ff9640 1 -- 172.21.15.169:0/1925264980 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v59) v1 ==== 97+0+0 (secure 0 0 0) 0x7f2aec14de70 con 0x7f2aec069a10 2023-10-26T01:33:36.223 INFO:teuthology.orchestra.run.smithi169.stderr: removed all blocklist entries 2023-10-26T01:33:36.226 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:36.224+0000 7f2af1239640 1 -- 172.21.15.169:0/1925264980 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2ad806ae40 msgr2=0x7f2ad806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:36.226 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:36.224+0000 7f2af1239640 1 --2- 172.21.15.169:0/1925264980 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2ad806ae40 0x7f2ad806d300 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f2aec071ab0 tx=0x7f2ae0009110 comp rx=0 tx=0).stop 2023-10-26T01:33:36.226 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:36.224+0000 7f2af1239640 1 -- 172.21.15.169:0/1925264980 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2aec069a10 msgr2=0x7f2aec15adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:36.226 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:36.224+0000 7f2af1239640 1 --2- 172.21.15.169:0/1925264980 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2aec069a10 0x7f2aec15adb0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f2adc009850 tx=0x7f2adc02dce0 comp rx=0 tx=0).stop 2023-10-26T01:33:36.227 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:36.224+0000 7f2af1239640 1 -- 172.21.15.169:0/1925264980 shutdown_connections 2023-10-26T01:33:36.227 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:36.224+0000 7f2af1239640 1 --2- 172.21.15.169:0/1925264980 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2ad806ae40 0x7f2ad806d300 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:36.227 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:36.224+0000 7f2af1239640 1 --2- 172.21.15.169:0/1925264980 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2aec0b0300 0x7f2aec14eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:36.227 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:36.224+0000 7f2af1239640 1 --2- 172.21.15.169:0/1925264980 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2aec06cba0 0x7f2aec14c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:36.227 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:36.224+0000 7f2af1239640 1 --2- 172.21.15.169:0/1925264980 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2aec069a10 0x7f2aec15adb0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:36.228 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:36.224+0000 7f2af1239640 1 -- 172.21.15.169:0/1925264980 >> 172.21.15.169:0/1925264980 conn(0x7f2aec0af110 msgr2=0x7f2aec06eb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:36.228 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:36.224+0000 7f2af1239640 1 -- 172.21.15.169:0/1925264980 shutdown_connections 2023-10-26T01:33:36.228 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:36.224+0000 7f2af1239640 1 -- 172.21.15.169:0/1925264980 wait complete. 2023-10-26T01:33:36.249 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-26T01:33:36.469 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:36.467+0000 7f66b17e2640 1 -- 172.21.15.169:0/2849081376 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66ac15b610 msgr2=0x7f66ac06ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:36.470 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:36.467+0000 7f66b17e2640 1 --2- 172.21.15.169:0/2849081376 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66ac15b610 0x7f66ac06ed30 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f66ac06c960 tx=0x7f669c02bd90 comp rx=0 tx=0).stop 2023-10-26T01:33:36.470 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:36.468+0000 7f66b17e2640 1 -- 172.21.15.169:0/2849081376 shutdown_connections 2023-10-26T01:33:36.470 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:36.468+0000 7f66b17e2640 1 --2- 172.21.15.169:0/2849081376 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f66ac0ac940 0x7f66ac0cdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:36.470 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:36.468+0000 7f66b17e2640 1 --2- 172.21.15.169:0/2849081376 >> v2:172.21.15.170:3300/0 conn(0x7f66ac0ac4e0 0x7f66ac0cd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:36.471 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:36.468+0000 7f66b17e2640 1 --2- 172.21.15.169:0/2849081376 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66ac15b610 0x7f66ac06ed30 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:36.471 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:36.468+0000 7f66b17e2640 1 -- 172.21.15.169:0/2849081376 >> 172.21.15.169:0/2849081376 conn(0x7f66ac069930 msgr2=0x7f66ac069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:36.471 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:36.468+0000 7f66b17e2640 1 -- 172.21.15.169:0/2849081376 shutdown_connections 2023-10-26T01:33:36.471 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:36.468+0000 7f66b17e2640 1 -- 172.21.15.169:0/2849081376 wait complete. 2023-10-26T01:33:36.473 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:36.469+0000 7f66b17e2640 1 Processor -- start 2023-10-26T01:33:36.473 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:36.470+0000 7f66b17e2640 1 -- start start 2023-10-26T01:33:36.473 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:36.470+0000 7f66b17e2640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66ac0ac4e0 0x7f66ac15bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:36.473 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:36.470+0000 7f66b17e2640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f66ac0ac940 0x7f66ac1f1da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:36.474 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:36.471+0000 7f66b17e2640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66ac15b610 0x7f66ac14f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:36.474 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:36.471+0000 7f66b17e2640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66ac153720 con 0x7f66ac15b610 2023-10-26T01:33:36.474 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:36.471+0000 7f66b17e2640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66ac1f23f0 con 0x7f66ac0ac4e0 2023-10-26T01:33:36.474 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:36.471+0000 7f66b17e2640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66ac1f26d0 con 0x7f66ac0ac940 2023-10-26T01:33:36.474 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:36.471+0000 7f66ab7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66ac15b610 0x7f66ac14f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:36.475 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:36.471+0000 7f66ab7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66ac15b610 0x7f66ac14f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:53964/0 (socket says 172.21.15.169:53964) 2023-10-26T01:33:36.475 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:36.471+0000 7f66ab7fe640 1 -- 172.21.15.169:0/3304368911 learned_addr learned my addr 172.21.15.169:0/3304368911 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:33:36.475 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:36.471+0000 7f66aa7fc640 1 --2- 172.21.15.169:0/3304368911 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f66ac0ac940 0x7f66ac1f1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:36.475 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:36.471+0000 7f66aaffd640 1 --2- 172.21.15.169:0/3304368911 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66ac0ac4e0 0x7f66ac15bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:36.475 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:36.471+0000 7f66ab7fe640 1 -- 172.21.15.169:0/3304368911 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f66ac0ac940 msgr2=0x7f66ac1f1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:36.476 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:36.471+0000 7f66ab7fe640 1 --2- 172.21.15.169:0/3304368911 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f66ac0ac940 0x7f66ac1f1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:36.476 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:36.471+0000 7f66ab7fe640 1 -- 172.21.15.169:0/3304368911 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66ac0ac4e0 msgr2=0x7f66ac15bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:36.476 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:36.471+0000 7f66ab7fe640 1 --2- 172.21.15.169:0/3304368911 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66ac0ac4e0 0x7f66ac15bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:36.476 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:36.471+0000 7f66ab7fe640 1 -- 172.21.15.169:0/3304368911 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f669c009560 con 0x7f66ac15b610 2023-10-26T01:33:36.476 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:36.471+0000 7f66aaffd640 1 --2- 172.21.15.169:0/3304368911 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66ac0ac4e0 0x7f66ac15bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:33:36.478 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:36.472+0000 7f66ab7fe640 1 --2- 172.21.15.169:0/3304368911 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66ac15b610 0x7f66ac14f8a0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f66a400cce0 tx=0x7f66a400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:36.478 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:36.472+0000 7f66937fe640 1 -- 172.21.15.169:0/3304368911 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a400f450 con 0x7f66ac15b610 2023-10-26T01:33:36.478 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:36.472+0000 7f66937fe640 1 -- 172.21.15.169:0/3304368911 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f66a400f6c0 con 0x7f66ac15b610 2023-10-26T01:33:36.478 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:36.472+0000 7f66937fe640 1 -- 172.21.15.169:0/3304368911 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a400fad0 con 0x7f66ac15b610 2023-10-26T01:33:36.478 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:36.472+0000 7f66b17e2640 1 -- 172.21.15.169:0/3304368911 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66ac1f29b0 con 0x7f66ac15b610 2023-10-26T01:33:36.479 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:36.472+0000 7f66b17e2640 1 -- 172.21.15.169:0/3304368911 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66ac1f2c90 con 0x7f66ac15b610 2023-10-26T01:33:36.479 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:36.473+0000 7f66b17e2640 1 -- 172.21.15.169:0/3304368911 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66ac1538d0 con 0x7f66ac15b610 2023-10-26T01:33:36.479 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:36.474+0000 7f66937fe640 1 -- 172.21.15.169:0/3304368911 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f66a400fc30 con 0x7f66ac15b610 2023-10-26T01:33:36.479 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:36.475+0000 7f66937fe640 1 --2- 172.21.15.169:0/3304368911 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f669806ae40 0x7f669806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:36.479 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:36.476+0000 7f66937fe640 1 -- 172.21.15.169:0/3304368911 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f66a4089eb0 con 0x7f66ac15b610 2023-10-26T01:33:36.480 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:36.476+0000 7f66aaffd640 1 --2- 172.21.15.169:0/3304368911 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f669806ae40 0x7f669806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:36.480 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:36.477+0000 7f66aaffd640 1 --2- 172.21.15.169:0/3304368911 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f669806ae40 0x7f669806d300 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f669c035000 tx=0x7f669c004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:36.481 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:36.480+0000 7f66937fe640 1 -- 172.21.15.169:0/3304368911 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f66ac1538d0 con 0x7f66ac15b610 2023-10-26T01:33:36.736 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:36.735+0000 7f66b17e2640 1 -- 172.21.15.169:0/3304368911 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f66ac04db10 con 0x7f66ac15b610 2023-10-26T01:33:36.737 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:36.736+0000 7f66937fe640 1 -- 172.21.15.169:0/3304368911 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v12) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f66a401d090 con 0x7f66ac15b610 2023-10-26T01:33:36.738 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:33:36.738 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2023-10-26T01:33:36.738 INFO:teuthology.orchestra.run.smithi169.stdout: "auth_dump": [ 2023-10-26T01:33:36.738 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:36.739 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "mds.a", 2023-10-26T01:33:36.739 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAgwTllWtUfAhAA28+12VwWEF/BUEu+ErSVyA==", 2023-10-26T01:33:36.739 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:33:36.739 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow", 2023-10-26T01:33:36.739 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow *", 2023-10-26T01:33:36.740 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow *", 2023-10-26T01:33:36.740 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:33:36.740 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:36.740 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:36.740 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:36.740 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "mds.b", 2023-10-26T01:33:36.741 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAgwTlleA8lHxAA+jMcMo+QnYAGllpKJ6eYgQ==", 2023-10-26T01:33:36.741 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:33:36.741 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow", 2023-10-26T01:33:36.741 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow *", 2023-10-26T01:33:36.741 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow *", 2023-10-26T01:33:36.742 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:33:36.742 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:36.742 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:36.742 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:36.742 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "mds.c", 2023-10-26T01:33:36.743 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAgwTllaCCqEBAAokeHvwOyp/Jv2x0yHDwysw==", 2023-10-26T01:33:36.743 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:33:36.743 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow", 2023-10-26T01:33:36.743 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow *", 2023-10-26T01:33:36.743 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow *", 2023-10-26T01:33:36.744 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:33:36.744 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:36.744 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:36.744 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:36.744 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "mds.d", 2023-10-26T01:33:36.745 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAgwTll51JQLhAA4Wzso4zQj+Htv6cXqvwRmg==", 2023-10-26T01:33:36.745 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:33:36.745 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow", 2023-10-26T01:33:36.745 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow *", 2023-10-26T01:33:36.745 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow *", 2023-10-26T01:33:36.746 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:33:36.746 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:36.746 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:36.746 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:36.746 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "osd.0", 2023-10-26T01:33:36.746 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAlwTll8RZ+HRAAt/sBK+Mi91YmTyQYbhKHLg==", 2023-10-26T01:33:36.747 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:33:36.747 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow profile osd", 2023-10-26T01:33:36.747 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow profile osd", 2023-10-26T01:33:36.747 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:33:36.747 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:36.748 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:36.748 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:36.748 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "osd.1", 2023-10-26T01:33:36.748 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAlwTllLXwWLRAAuHB2lJ+oDJou/IJBtsYY4Q==", 2023-10-26T01:33:36.748 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:33:36.749 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow profile osd", 2023-10-26T01:33:36.749 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow profile osd", 2023-10-26T01:33:36.749 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:33:36.749 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:36.749 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:36.750 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:36.750 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "osd.2", 2023-10-26T01:33:36.750 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAmwTllj1nSARAA5Jl34Q0X7a71ySd4wz17lw==", 2023-10-26T01:33:36.750 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:33:36.750 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow profile osd", 2023-10-26T01:33:36.750 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow profile osd", 2023-10-26T01:33:36.751 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:33:36.751 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:36.751 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:36.751 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:36.751 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "osd.3", 2023-10-26T01:33:36.752 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAmwTllC8MeEhAAfGxRVSwD+YbU/U5aeenO3w==", 2023-10-26T01:33:36.752 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:33:36.752 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow profile osd", 2023-10-26T01:33:36.752 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow profile osd", 2023-10-26T01:33:36.752 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:33:36.752 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:36.753 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:36.753 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:36.753 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "osd.4", 2023-10-26T01:33:36.753 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAowTll4Md6DhAAuXfadUNAdouMP28MiX/D3Q==", 2023-10-26T01:33:36.753 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:33:36.754 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow profile osd", 2023-10-26T01:33:36.754 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow profile osd", 2023-10-26T01:33:36.754 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:33:36.754 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:36.754 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:36.755 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:36.755 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "osd.5", 2023-10-26T01:33:36.755 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAowTllDdViHRAAec9QyhZCsjJ2rgGhhcjwpQ==", 2023-10-26T01:33:36.755 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:33:36.755 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow profile osd", 2023-10-26T01:33:36.756 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow profile osd", 2023-10-26T01:33:36.756 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:33:36.756 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:36.756 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:36.756 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:36.756 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "osd.6", 2023-10-26T01:33:36.757 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAowTll23X5KxAAsmQvG0yq7ilkVG5fNiaCUg==", 2023-10-26T01:33:36.757 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:33:36.757 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow profile osd", 2023-10-26T01:33:36.757 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow profile osd", 2023-10-26T01:33:36.757 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:33:36.758 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:36.758 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:36.758 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:36.758 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "osd.7", 2023-10-26T01:33:36.758 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAowTllhcy/OhAAY5Yjheh76ibOVREqN6WBpQ==", 2023-10-26T01:33:36.759 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:33:36.759 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow profile osd", 2023-10-26T01:33:36.759 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow profile osd", 2023-10-26T01:33:36.759 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:33:36.759 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:36.759 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:36.760 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:36.760 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "client.0", 2023-10-26T01:33:36.760 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAgwTllKC7gOhAAPsvxGcuUA8/p8xqbUgGteA==", 2023-10-26T01:33:36.760 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:33:36.760 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow", 2023-10-26T01:33:36.761 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow r", 2023-10-26T01:33:36.761 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-26T01:33:36.761 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:36.761 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:36.761 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:36.762 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "client.1", 2023-10-26T01:33:36.762 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAhwTllhxb6ExAAeYTsurWpnjohWlrNIlLpsw==", 2023-10-26T01:33:36.762 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:33:36.762 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow", 2023-10-26T01:33:36.762 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow r", 2023-10-26T01:33:36.763 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow rw", 2023-10-26T01:33:36.763 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow rwx" 2023-10-26T01:33:36.763 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:36.763 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:36.763 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:36.764 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "client.2", 2023-10-26T01:33:36.764 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAhwTllLWMoChAAsUHzBc29GDqmEQAFnl7dHg==", 2023-10-26T01:33:36.764 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:33:36.764 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow", 2023-10-26T01:33:36.764 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow r", 2023-10-26T01:33:36.764 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow rw", 2023-10-26T01:33:36.765 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow rwx" 2023-10-26T01:33:36.765 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:36.765 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:36.765 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:36.765 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "client.3", 2023-10-26T01:33:36.766 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAhwTll875HHRAAGbwikoR/hDU8P/bj0OPWAw==", 2023-10-26T01:33:36.766 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:33:36.766 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow", 2023-10-26T01:33:36.766 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow r", 2023-10-26T01:33:36.766 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow rw", 2023-10-26T01:33:36.767 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow rwx" 2023-10-26T01:33:36.767 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:36.767 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:36.767 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:36.767 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "client.admin", 2023-10-26T01:33:36.768 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAewTll2yddMhAAO8bnXNevDGe/KEeNNa36pQ==", 2023-10-26T01:33:36.768 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:33:36.768 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow *", 2023-10-26T01:33:36.768 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow *", 2023-10-26T01:33:36.768 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow *", 2023-10-26T01:33:36.768 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:33:36.769 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:36.769 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:36.769 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:36.769 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "mgr.x", 2023-10-26T01:33:36.769 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAfwTllPu3FMxAAV8d5oRFXAeeZH7e1xzPEDg==", 2023-10-26T01:33:36.770 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:33:36.770 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow *", 2023-10-26T01:33:36.770 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow profile mgr", 2023-10-26T01:33:36.770 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:33:36.770 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:36.771 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:33:36.771 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:36.771 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "mgr.y", 2023-10-26T01:33:36.771 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAfwTllZc+DJxAAsZ2TWn8H7Kw/lNTc2qyKzA==", 2023-10-26T01:33:36.771 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:33:36.772 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow *", 2023-10-26T01:33:36.772 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow profile mgr", 2023-10-26T01:33:36.772 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:33:36.772 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:36.772 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:36.772 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:33:36.773 INFO:teuthology.orchestra.run.smithi169.stdout:} 2023-10-26T01:33:36.774 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:36.738+0000 7f66b17e2640 1 -- 172.21.15.169:0/3304368911 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f669806ae40 msgr2=0x7f669806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:36.775 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:36.738+0000 7f66b17e2640 1 --2- 172.21.15.169:0/3304368911 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f669806ae40 0x7f669806d300 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f669c035000 tx=0x7f669c004730 comp rx=0 tx=0).stop 2023-10-26T01:33:36.775 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:36.738+0000 7f66b17e2640 1 -- 172.21.15.169:0/3304368911 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66ac15b610 msgr2=0x7f66ac14f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:36.775 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:36.738+0000 7f66b17e2640 1 --2- 172.21.15.169:0/3304368911 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66ac15b610 0x7f66ac14f8a0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f66a400cce0 tx=0x7f66a400a620 comp rx=0 tx=0).stop 2023-10-26T01:33:36.775 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:36.739+0000 7f66b17e2640 1 -- 172.21.15.169:0/3304368911 shutdown_connections 2023-10-26T01:33:36.776 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:36.739+0000 7f66b17e2640 1 --2- 172.21.15.169:0/3304368911 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f669806ae40 0x7f669806d300 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:36.776 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:36.739+0000 7f66b17e2640 1 --2- 172.21.15.169:0/3304368911 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66ac15b610 0x7f66ac14f8a0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:36.776 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:36.739+0000 7f66b17e2640 1 --2- 172.21.15.169:0/3304368911 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f66ac0ac940 0x7f66ac1f1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:36.776 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:36.739+0000 7f66b17e2640 1 --2- 172.21.15.169:0/3304368911 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66ac0ac4e0 0x7f66ac15bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:36.776 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:36.739+0000 7f66b17e2640 1 -- 172.21.15.169:0/3304368911 >> 172.21.15.169:0/3304368911 conn(0x7f66ac069930 msgr2=0x7f66ac0cc980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:36.777 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:36.739+0000 7f66b17e2640 1 -- 172.21.15.169:0/3304368911 shutdown_connections 2023-10-26T01:33:36.777 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:36.739+0000 7f66b17e2640 1 -- 172.21.15.169:0/3304368911 wait complete. 2023-10-26T01:33:36.777 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:33:36.994 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:36.992+0000 7ff91da9b640 1 -- 172.21.15.169:0/1476742383 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff91814e790 msgr2=0x7ff9180aab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:36.994 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:36.992+0000 7ff91da9b640 1 --2- 172.21.15.169:0/1476742383 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff91814e790 0x7ff9180aab40 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7ff900005fd0 tx=0x7ff90002d150 comp rx=0 tx=0).stop 2023-10-26T01:33:36.995 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:36.992+0000 7ff91da9b640 1 -- 172.21.15.169:0/1476742383 shutdown_connections 2023-10-26T01:33:36.995 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:36.992+0000 7ff91da9b640 1 --2- 172.21.15.169:0/1476742383 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff91814e790 0x7ff9180aab40 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:36.995 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:36.992+0000 7ff91da9b640 1 -- 172.21.15.169:0/1476742383 >> 172.21.15.169:0/1476742383 conn(0x7ff918069930 msgr2=0x7ff918069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:36.995 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:36.992+0000 7ff91da9b640 1 -- 172.21.15.169:0/1476742383 shutdown_connections 2023-10-26T01:33:36.995 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:36.992+0000 7ff91da9b640 1 -- 172.21.15.169:0/1476742383 wait complete. 2023-10-26T01:33:36.997 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:36.994+0000 7ff91da9b640 1 Processor -- start 2023-10-26T01:33:36.997 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:36.994+0000 7ff91da9b640 1 -- start start 2023-10-26T01:33:36.998 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:36.995+0000 7ff91da9b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9180cde30 0x7ff9181523e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:36.998 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:36.995+0000 7ff91da9b640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff91814e790 0x7ff9181547e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:36.998 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:36.995+0000 7ff91da9b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff918158850 0x7ff9181efbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:36.998 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:36.995+0000 7ff91da9b640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff918096d80 con 0x7ff9180cde30 2023-10-26T01:33:36.999 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:36.995+0000 7ff91da9b640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9180af390 con 0x7ff918158850 2023-10-26T01:33:36.999 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:36.995+0000 7ff91da9b640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff918154d20 con 0x7ff91814e790 2023-10-26T01:33:36.999 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:36.995+0000 7ff9177fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9180cde30 0x7ff9181523e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:36.999 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:36.995+0000 7ff9177fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9180cde30 0x7ff9181523e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:53980/0 (socket says 172.21.15.169:53980) 2023-10-26T01:33:36.999 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:36.995+0000 7ff9177fe640 1 -- 172.21.15.169:0/1657205254 learned_addr learned my addr 172.21.15.169:0/1657205254 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:33:37.000 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:36.995+0000 7ff9177fe640 1 -- 172.21.15.169:0/1657205254 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff91814e790 msgr2=0x7ff9181547e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:37.000 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:36.995+0000 7ff9177fe640 1 --2- 172.21.15.169:0/1657205254 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff91814e790 0x7ff9181547e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:37.000 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:36.995+0000 7ff9177fe640 1 -- 172.21.15.169:0/1657205254 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff918158850 msgr2=0x7ff9181efbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:37.000 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:36.995+0000 7ff9177fe640 1 --2- 172.21.15.169:0/1657205254 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff918158850 0x7ff9181efbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:37.000 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:36.995+0000 7ff9177fe640 1 -- 172.21.15.169:0/1657205254 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff90000a670 con 0x7ff9180cde30 2023-10-26T01:33:37.001 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:36.996+0000 7ff9177fe640 1 --2- 172.21.15.169:0/1657205254 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9180cde30 0x7ff9181523e0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7ff90000a960 tx=0x7ff9000049d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:37.002 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:36.996+0000 7ff90ffff640 1 -- 172.21.15.169:0/1657205254 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff900004fd0 con 0x7ff9180cde30 2023-10-26T01:33:37.002 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:36.996+0000 7ff90ffff640 1 -- 172.21.15.169:0/1657205254 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff9000036e0 con 0x7ff9180cde30 2023-10-26T01:33:37.003 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:36.996+0000 7ff90ffff640 1 -- 172.21.15.169:0/1657205254 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff900003a90 con 0x7ff9180cde30 2023-10-26T01:33:37.003 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:36.996+0000 7ff91da9b640 1 -- 172.21.15.169:0/1657205254 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9180af040 con 0x7ff9180cde30 2023-10-26T01:33:37.003 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:36.996+0000 7ff91da9b640 1 -- 172.21.15.169:0/1657205254 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff918155260 con 0x7ff9180cde30 2023-10-26T01:33:37.003 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:36.997+0000 7ff91da9b640 1 -- 172.21.15.169:0/1657205254 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff91804db10 con 0x7ff9180cde30 2023-10-26T01:33:37.004 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:36.998+0000 7ff90ffff640 1 -- 172.21.15.169:0/1657205254 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ff900003bf0 con 0x7ff9180cde30 2023-10-26T01:33:37.004 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:37.000+0000 7ff90ffff640 1 --2- 172.21.15.169:0/1657205254 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff8f806ae40 0x7ff8f806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:37.004 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:37.000+0000 7ff90ffff640 1 -- 172.21.15.169:0/1657205254 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ff9000ac750 con 0x7ff9180cde30 2023-10-26T01:33:37.004 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:37.000+0000 7ff916ffd640 1 --2- 172.21.15.169:0/1657205254 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff8f806ae40 0x7ff8f806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:37.005 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:37.001+0000 7ff916ffd640 1 --2- 172.21.15.169:0/1657205254 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff8f806ae40 0x7ff8f806d300 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ff9181490c0 tx=0x7ff908009230 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:37.005 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:37.004+0000 7ff90ffff640 1 -- 172.21.15.169:0/1657205254 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff900077ab0 con 0x7ff9180cde30 2023-10-26T01:33:37.196 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:37.194+0000 7ff91da9b640 1 -- 172.21.15.169:0/1657205254 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff9180aca70 con 0x7ff9180cde30 2023-10-26T01:33:37.198 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:37.196+0000 7ff90ffff640 1 -- 172.21.15.169:0/1657205254 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) v1 ==== 74+0+12905 (secure 0 0 0) 0x7ff90007b0d0 con 0x7ff9180cde30 2023-10-26T01:33:37.198 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:33:37.199 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":61,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:33:36.231819+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:33:37.200 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:37.199+0000 7ff91da9b640 1 -- 172.21.15.169:0/1657205254 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff8f806ae40 msgr2=0x7ff8f806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:37.201 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:37.199+0000 7ff91da9b640 1 --2- 172.21.15.169:0/1657205254 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff8f806ae40 0x7ff8f806d300 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ff9181490c0 tx=0x7ff908009230 comp rx=0 tx=0).stop 2023-10-26T01:33:37.201 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:37.199+0000 7ff91da9b640 1 -- 172.21.15.169:0/1657205254 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9180cde30 msgr2=0x7ff9181523e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:37.201 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:37.199+0000 7ff91da9b640 1 --2- 172.21.15.169:0/1657205254 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9180cde30 0x7ff9181523e0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7ff90000a960 tx=0x7ff9000049d0 comp rx=0 tx=0).stop 2023-10-26T01:33:37.201 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:37.199+0000 7ff91da9b640 1 -- 172.21.15.169:0/1657205254 shutdown_connections 2023-10-26T01:33:37.202 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:37.199+0000 7ff91da9b640 1 --2- 172.21.15.169:0/1657205254 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff8f806ae40 0x7ff8f806d300 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:37.202 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:37.199+0000 7ff91da9b640 1 --2- 172.21.15.169:0/1657205254 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff918158850 0x7ff9181efbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:37.202 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:37.199+0000 7ff91da9b640 1 --2- 172.21.15.169:0/1657205254 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff91814e790 0x7ff9181547e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:37.202 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:37.199+0000 7ff91da9b640 1 --2- 172.21.15.169:0/1657205254 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9180cde30 0x7ff9181523e0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:37.202 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:37.199+0000 7ff91da9b640 1 -- 172.21.15.169:0/1657205254 >> 172.21.15.169:0/1657205254 conn(0x7ff918069930 msgr2=0x7ff918157f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:37.203 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:37.199+0000 7ff91da9b640 1 -- 172.21.15.169:0/1657205254 shutdown_connections 2023-10-26T01:33:37.203 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:37.199+0000 7ff91da9b640 1 -- 172.21.15.169:0/1657205254 wait complete. 2023-10-26T01:33:37.221 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T01:30:32.200808+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T01:33:37.222 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T01:33:37.438 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:37.436+0000 7f1306dd5640 1 -- 172.21.15.169:0/1369493835 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1300159480 msgr2=0x7f13000b02e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:37.438 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:37.436+0000 7f1306dd5640 1 --2- 172.21.15.169:0/1369493835 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1300159480 0x7f13000b02e0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f12e8005fd0 tx=0x7f12e802be20 comp rx=0 tx=0).stop 2023-10-26T01:33:37.438 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:37.436+0000 7f1306dd5640 1 -- 172.21.15.169:0/1369493835 shutdown_connections 2023-10-26T01:33:37.439 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:37.436+0000 7f1306dd5640 1 --2- 172.21.15.169:0/1369493835 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1300159480 0x7f13000b02e0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:37.439 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:37.436+0000 7f1306dd5640 1 -- 172.21.15.169:0/1369493835 >> 172.21.15.169:0/1369493835 conn(0x7f1300069930 msgr2=0x7f1300069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:37.439 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:37.436+0000 7f1306dd5640 1 -- 172.21.15.169:0/1369493835 shutdown_connections 2023-10-26T01:33:37.439 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:37.436+0000 7f1306dd5640 1 -- 172.21.15.169:0/1369493835 wait complete. 2023-10-26T01:33:37.440 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:37.438+0000 7f1306dd5640 1 Processor -- start 2023-10-26T01:33:37.440 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:37.438+0000 7f1306dd5640 1 -- start start 2023-10-26T01:33:37.442 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:37.439+0000 7f1306dd5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f13000b0820 0x7f13001e1d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:37.442 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:37.439+0000 7f1306dd5640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f13000cd690 0x7f13001545f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:37.442 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:37.439+0000 7f1306dd5640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1300159480 0x7f130014e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:37.443 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:37.439+0000 7f1306dd5640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13001e16e0 con 0x7f13000cd690 2023-10-26T01:33:37.443 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:37.439+0000 7f1306dd5640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13001e23a0 con 0x7f13000b0820 2023-10-26T01:33:37.444 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:37.439+0000 7f1306dd5640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f130006adc0 con 0x7f1300159480 2023-10-26T01:33:37.444 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:37.439+0000 7f12fffff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f13000cd690 0x7f13001545f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:37.444 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:37.439+0000 7f12fffff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f13000cd690 0x7f13001545f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:54000/0 (socket says 172.21.15.169:54000) 2023-10-26T01:33:37.444 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:37.439+0000 7f12fffff640 1 -- 172.21.15.169:0/2717656650 learned_addr learned my addr 172.21.15.169:0/2717656650 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:33:37.444 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:37.439+0000 7f12fffff640 1 -- 172.21.15.169:0/2717656650 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1300159480 msgr2=0x7f130014e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:37.445 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:37.439+0000 7f12fffff640 1 --2- 172.21.15.169:0/2717656650 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1300159480 0x7f130014e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:37.445 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:37.439+0000 7f12fffff640 1 -- 172.21.15.169:0/2717656650 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f13000b0820 msgr2=0x7f13001e1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:37.445 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:37.439+0000 7f12fffff640 1 --2- 172.21.15.169:0/2717656650 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f13000b0820 0x7f13001e1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:37.445 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:37.439+0000 7f12fffff640 1 -- 172.21.15.169:0/2717656650 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12e8009560 con 0x7f13000cd690 2023-10-26T01:33:37.445 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:37.440+0000 7f12fffff640 1 --2- 172.21.15.169:0/2717656650 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f13000cd690 0x7f13001545f0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f12f000c7c0 tx=0x7f12f000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:37.446 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:37.440+0000 7f12fd7fa640 1 -- 172.21.15.169:0/2717656650 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f000aa50 con 0x7f13000cd690 2023-10-26T01:33:37.446 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:37.440+0000 7f12fd7fa640 1 -- 172.21.15.169:0/2717656650 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f12f000aca0 con 0x7f13000cd690 2023-10-26T01:33:37.446 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:37.440+0000 7f1306dd5640 1 -- 172.21.15.169:0/2717656650 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f130006b0a0 con 0x7f13000cd690 2023-10-26T01:33:37.446 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:37.440+0000 7f12fd7fa640 1 -- 172.21.15.169:0/2717656650 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f0012680 con 0x7f13000cd690 2023-10-26T01:33:37.446 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:37.440+0000 7f1306dd5640 1 -- 172.21.15.169:0/2717656650 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f130006b660 con 0x7f13000cd690 2023-10-26T01:33:37.447 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:37.441+0000 7f1306dd5640 1 -- 172.21.15.169:0/2717656650 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1300066af0 con 0x7f13000cd690 2023-10-26T01:33:37.448 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:37.442+0000 7f12fd7fa640 1 -- 172.21.15.169:0/2717656650 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f12f001a3b0 con 0x7f13000cd690 2023-10-26T01:33:37.448 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:37.443+0000 7f12fd7fa640 1 --2- 172.21.15.169:0/2717656650 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f12e006ae40 0x7f12e006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:37.448 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:37.443+0000 7f12fd7fa640 1 -- 172.21.15.169:0/2717656650 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f12f0089520 con 0x7f13000cd690 2023-10-26T01:33:37.448 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:37.444+0000 7f1304b71640 1 --2- 172.21.15.169:0/2717656650 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f12e006ae40 0x7f12e006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:37.448 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:37.445+0000 7f1304b71640 1 --2- 172.21.15.169:0/2717656650 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f12e006ae40 0x7f12e006d300 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f12e8035000 tx=0x7f12e80046c0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:37.449 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:37.447+0000 7f12fd7fa640 1 -- 172.21.15.169:0/2717656650 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f12f0054880 con 0x7f13000cd690 2023-10-26T01:33:37.636 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:37.634+0000 7f1306dd5640 1 -- 172.21.15.169:0/2717656650 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f130004da70 con 0x7f13000cd690 2023-10-26T01:33:37.637 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:37.635+0000 7f12fd7fa640 1 -- 172.21.15.169:0/2717656650 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v61) v1 ==== 93+0+10 (secure 0 0 0) 0x7f12f0057ea0 con 0x7f13000cd690 2023-10-26T01:33:37.637 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 1 2023-10-26T01:33:37.639 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:37.637+0000 7f1306dd5640 1 -- 172.21.15.169:0/2717656650 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f12e006ae40 msgr2=0x7f12e006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:37.640 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:37.637+0000 7f1306dd5640 1 --2- 172.21.15.169:0/2717656650 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f12e006ae40 0x7f12e006d300 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f12e8035000 tx=0x7f12e80046c0 comp rx=0 tx=0).stop 2023-10-26T01:33:37.640 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:37.637+0000 7f1306dd5640 1 -- 172.21.15.169:0/2717656650 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f13000cd690 msgr2=0x7f13001545f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:37.640 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:37.637+0000 7f1306dd5640 1 --2- 172.21.15.169:0/2717656650 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f13000cd690 0x7f13001545f0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f12f000c7c0 tx=0x7f12f000cc80 comp rx=0 tx=0).stop 2023-10-26T01:33:37.640 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:37.638+0000 7f1306dd5640 1 -- 172.21.15.169:0/2717656650 shutdown_connections 2023-10-26T01:33:37.640 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:37.638+0000 7f1306dd5640 1 --2- 172.21.15.169:0/2717656650 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f12e006ae40 0x7f12e006d300 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:37.641 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:37.638+0000 7f1306dd5640 1 --2- 172.21.15.169:0/2717656650 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f13000cd690 0x7f13001545f0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:37.641 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:37.638+0000 7f1306dd5640 1 --2- 172.21.15.169:0/2717656650 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1300159480 0x7f130014e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:37.641 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:37.638+0000 7f1306dd5640 1 --2- 172.21.15.169:0/2717656650 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f13000b0820 0x7f13001e1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:37.641 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:37.638+0000 7f1306dd5640 1 -- 172.21.15.169:0/2717656650 >> 172.21.15.169:0/2717656650 conn(0x7f1300069930 msgr2=0x7f1300158540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:37.641 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:37.638+0000 7f1306dd5640 1 -- 172.21.15.169:0/2717656650 shutdown_connections 2023-10-26T01:33:37.642 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:37.638+0000 7f1306dd5640 1 -- 172.21.15.169:0/2717656650 wait complete. 2023-10-26T01:33:37.659 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-26T01:33:37.878 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:37.876+0000 7f9d6696c640 1 -- 172.21.15.169:0/33076555 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d600ac6d0 msgr2=0x7f9d6015d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:37.878 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:37.876+0000 7f9d6696c640 1 --2- 172.21.15.169:0/33076555 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d600ac6d0 0x7f9d6015d840 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c009830 tx=0x7f9d5c02bdf0 comp rx=0 tx=0).stop 2023-10-26T01:33:37.879 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:37.876+0000 7f9d6696c640 1 -- 172.21.15.169:0/33076555 shutdown_connections 2023-10-26T01:33:37.879 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:37.876+0000 7f9d6696c640 1 --2- 172.21.15.169:0/33076555 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d600ac6d0 0x7f9d6015d840 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:37.879 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:37.876+0000 7f9d6696c640 1 -- 172.21.15.169:0/33076555 >> 172.21.15.169:0/33076555 conn(0x7f9d600aedd0 msgr2=0x7f9d600af1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:37.879 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:37.876+0000 7f9d6696c640 1 -- 172.21.15.169:0/33076555 shutdown_connections 2023-10-26T01:33:37.879 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:37.876+0000 7f9d6696c640 1 -- 172.21.15.169:0/33076555 wait complete. 2023-10-26T01:33:37.880 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:37.878+0000 7f9d6696c640 1 Processor -- start 2023-10-26T01:33:37.880 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:37.878+0000 7f9d6696c640 1 -- start start 2023-10-26T01:33:37.882 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:37.879+0000 7f9d6696c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d6009a5c0 0x7f9d6006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:37.882 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:37.879+0000 7f9d6696c640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9d600ac6d0 0x7f9d6015a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:37.882 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:37.879+0000 7f9d6696c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9d600c9970 0x7f9d60156af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:37.882 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:37.879+0000 7f9d6696c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d600719b0 con 0x7f9d6009a5c0 2023-10-26T01:33:37.882 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:37.879+0000 7f9d6696c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d600c7730 con 0x7f9d600c9970 2023-10-26T01:33:37.883 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:37.879+0000 7f9d6696c640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d600c7a10 con 0x7f9d600ac6d0 2023-10-26T01:33:37.883 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:37.880+0000 7f9d6596a640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d6009a5c0 0x7f9d6006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:37.883 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:37.880+0000 7f9d6596a640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d6009a5c0 0x7f9d6006ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:54008/0 (socket says 172.21.15.169:54008) 2023-10-26T01:33:37.883 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:37.880+0000 7f9d6596a640 1 -- 172.21.15.169:0/3048143085 learned_addr learned my addr 172.21.15.169:0/3048143085 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:33:37.883 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:37.880+0000 7f9d65169640 1 --2- 172.21.15.169:0/3048143085 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9d600ac6d0 0x7f9d6015a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:37.884 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:37.880+0000 7f9d6616b640 1 --2- 172.21.15.169:0/3048143085 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9d600c9970 0x7f9d60156af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:37.884 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:37.880+0000 7f9d6596a640 1 -- 172.21.15.169:0/3048143085 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9d600ac6d0 msgr2=0x7f9d6015a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:37.884 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:37.880+0000 7f9d6596a640 1 --2- 172.21.15.169:0/3048143085 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9d600ac6d0 0x7f9d6015a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:37.884 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:37.880+0000 7f9d6596a640 1 -- 172.21.15.169:0/3048143085 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9d600c9970 msgr2=0x7f9d60156af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:37.884 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:37.880+0000 7f9d6596a640 1 --2- 172.21.15.169:0/3048143085 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9d600c9970 0x7f9d60156af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:37.885 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:37.880+0000 7f9d6596a640 1 -- 172.21.15.169:0/3048143085 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d5c009490 con 0x7f9d6009a5c0 2023-10-26T01:33:37.885 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:37.880+0000 7f9d6616b640 1 --2- 172.21.15.169:0/3048143085 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9d600c9970 0x7f9d60156af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:33:37.885 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:37.880+0000 7f9d6596a640 1 --2- 172.21.15.169:0/3048143085 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d6009a5c0 0x7f9d6006ee00 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c009780 tx=0x7f9d5c0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:37.886 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:37.881+0000 7f9d567fc640 1 -- 172.21.15.169:0/3048143085 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d5c003ea0 con 0x7f9d6009a5c0 2023-10-26T01:33:37.886 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:37.881+0000 7f9d567fc640 1 -- 172.21.15.169:0/3048143085 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9d5c034440 con 0x7f9d6009a5c0 2023-10-26T01:33:37.886 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:37.881+0000 7f9d567fc640 1 -- 172.21.15.169:0/3048143085 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d5c034850 con 0x7f9d6009a5c0 2023-10-26T01:33:37.887 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:37.881+0000 7f9d6696c640 1 -- 172.21.15.169:0/3048143085 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d600c7cf0 con 0x7f9d6009a5c0 2023-10-26T01:33:37.887 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:37.881+0000 7f9d6696c640 1 -- 172.21.15.169:0/3048143085 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d6006f340 con 0x7f9d6009a5c0 2023-10-26T01:33:37.887 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:37.882+0000 7f9d6696c640 1 -- 172.21.15.169:0/3048143085 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d600c8210 con 0x7f9d6009a5c0 2023-10-26T01:33:37.887 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:37.883+0000 7f9d567fc640 1 -- 172.21.15.169:0/3048143085 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f9d5c03b040 con 0x7f9d6009a5c0 2023-10-26T01:33:37.887 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:37.884+0000 7f9d567fc640 1 --2- 172.21.15.169:0/3048143085 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9d4406ae40 0x7f9d4406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:37.887 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:37.884+0000 7f9d567fc640 1 -- 172.21.15.169:0/3048143085 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f9d5c0ab7c0 con 0x7f9d6009a5c0 2023-10-26T01:33:37.888 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:37.885+0000 7f9d65169640 1 --2- 172.21.15.169:0/3048143085 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9d4406ae40 0x7f9d4406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:37.888 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:37.886+0000 7f9d65169640 1 --2- 172.21.15.169:0/3048143085 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9d4406ae40 0x7f9d4406d300 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f9d6015a2e0 tx=0x7f9d50009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:37.890 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:37.888+0000 7f9d567fc640 1 -- 172.21.15.169:0/3048143085 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9d600c8210 con 0x7f9d6009a5c0 2023-10-26T01:33:38.083 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:38.081+0000 7f9d6696c640 1 -- 172.21.15.169:0/3048143085 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f9d600c8210 con 0x7f9d6009a5c0 2023-10-26T01:33:38.084 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:38.082+0000 7f9d567fc640 1 -- 172.21.15.169:0/3048143085 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v61) v1 ==== 84+0+65 (secure 0 0 0) 0x7f9d5c039060 con 0x7f9d6009a5c0 2023-10-26T01:33:38.084 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:33:38.085 INFO:teuthology.orchestra.run.smithi169.stdout:[ 2023-10-26T01:33:38.085 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:33:38.085 INFO:teuthology.orchestra.run.smithi169.stdout: "poolnum": 1, 2023-10-26T01:33:38.085 INFO:teuthology.orchestra.run.smithi169.stdout: "poolname": ".mgr" 2023-10-26T01:33:38.085 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:33:38.086 INFO:teuthology.orchestra.run.smithi169.stdout:] 2023-10-26T01:33:38.088 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:38.086+0000 7f9d6696c640 1 -- 172.21.15.169:0/3048143085 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9d4406ae40 msgr2=0x7f9d4406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:38.088 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:38.086+0000 7f9d6696c640 1 --2- 172.21.15.169:0/3048143085 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9d4406ae40 0x7f9d4406d300 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f9d6015a2e0 tx=0x7f9d50009040 comp rx=0 tx=0).stop 2023-10-26T01:33:38.088 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:38.086+0000 7f9d6696c640 1 -- 172.21.15.169:0/3048143085 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d6009a5c0 msgr2=0x7f9d6006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:38.088 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:38.086+0000 7f9d6696c640 1 --2- 172.21.15.169:0/3048143085 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d6009a5c0 0x7f9d6006ee00 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c009780 tx=0x7f9d5c0038a0 comp rx=0 tx=0).stop 2023-10-26T01:33:38.089 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:38.086+0000 7f9d6696c640 1 -- 172.21.15.169:0/3048143085 shutdown_connections 2023-10-26T01:33:38.089 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:38.086+0000 7f9d6696c640 1 --2- 172.21.15.169:0/3048143085 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9d4406ae40 0x7f9d4406d300 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:38.089 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:38.086+0000 7f9d6696c640 1 --2- 172.21.15.169:0/3048143085 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9d600c9970 0x7f9d60156af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:38.089 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:38.086+0000 7f9d6696c640 1 --2- 172.21.15.169:0/3048143085 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9d600ac6d0 0x7f9d6015a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:38.089 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:38.086+0000 7f9d6696c640 1 --2- 172.21.15.169:0/3048143085 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d6009a5c0 0x7f9d6006ee00 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:38.089 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:38.086+0000 7f9d6696c640 1 -- 172.21.15.169:0/3048143085 >> 172.21.15.169:0/3048143085 conn(0x7f9d600aedd0 msgr2=0x7f9d600caaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:38.090 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:38.086+0000 7f9d6696c640 1 -- 172.21.15.169:0/3048143085 shutdown_connections 2023-10-26T01:33:38.090 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:38.086+0000 7f9d6696c640 1 -- 172.21.15.169:0/3048143085 wait complete. 2023-10-26T01:33:38.107 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-26T01:33:38.107 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-26T01:33:38.337 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:38.335+0000 7fb93a96e640 1 -- 172.21.15.169:0/1848843772 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb9340cd6a0 msgr2=0x7fb9340cdb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:38.337 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:38.335+0000 7fb93a96e640 1 --2- 172.21.15.169:0/1848843772 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb9340cd6a0 0x7fb9340cdb00 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb91c005fd0 tx=0x7fb91c02be10 comp rx=0 tx=0).stop 2023-10-26T01:33:38.337 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:38.335+0000 7fb93a96e640 1 -- 172.21.15.169:0/1848843772 shutdown_connections 2023-10-26T01:33:38.338 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:38.335+0000 7fb93a96e640 1 --2- 172.21.15.169:0/1848843772 >> v2:172.21.15.170:3300/0 conn(0x7fb9340b0990 0x7fb9340b0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:38.338 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:38.335+0000 7fb93a96e640 1 --2- 172.21.15.169:0/1848843772 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb9340cd6a0 0x7fb9340cdb00 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:38.338 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:38.335+0000 7fb93a96e640 1 -- 172.21.15.169:0/1848843772 >> 172.21.15.169:0/1848843772 conn(0x7fb934069930 msgr2=0x7fb934069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:38.338 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:38.335+0000 7fb93a96e640 1 -- 172.21.15.169:0/1848843772 shutdown_connections 2023-10-26T01:33:38.339 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:38.335+0000 7fb93a96e640 1 -- 172.21.15.169:0/1848843772 wait complete. 2023-10-26T01:33:38.341 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:38.337+0000 7fb93a96e640 1 Processor -- start 2023-10-26T01:33:38.341 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:38.337+0000 7fb93a96e640 1 -- start start 2023-10-26T01:33:38.342 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:38.338+0000 7fb93a96e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9340b0990 0x7fb934149a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:38.342 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:38.338+0000 7fb93a96e640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb9340cd6a0 0x7fb9341565c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:38.342 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:38.338+0000 7fb93a96e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb93414e780 0x7fb934156b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:38.342 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:38.338+0000 7fb93a96e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9341f6690 con 0x7fb9340b0990 2023-10-26T01:33:38.342 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:38.338+0000 7fb93a96e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb93406f780 con 0x7fb93414e780 2023-10-26T01:33:38.343 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:38.338+0000 7fb93a96e640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9341f63b0 con 0x7fb9340cd6a0 2023-10-26T01:33:38.343 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:38.338+0000 7fb933fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9340b0990 0x7fb934149a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:38.343 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:38.338+0000 7fb933fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9340b0990 0x7fb934149a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:54016/0 (socket says 172.21.15.169:54016) 2023-10-26T01:33:38.343 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:38.338+0000 7fb933fff640 1 -- 172.21.15.169:0/1639656574 learned_addr learned my addr 172.21.15.169:0/1639656574 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:33:38.344 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:38.339+0000 7fb92bfff640 1 --2- 172.21.15.169:0/1639656574 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb9340cd6a0 0x7fb9341565c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:38.344 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:38.339+0000 7fb938f0b640 1 --2- 172.21.15.169:0/1639656574 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb93414e780 0x7fb934156b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:38.344 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:38.339+0000 7fb933fff640 1 -- 172.21.15.169:0/1639656574 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb9340cd6a0 msgr2=0x7fb9341565c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:38.344 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:38.339+0000 7fb933fff640 1 --2- 172.21.15.169:0/1639656574 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb9340cd6a0 0x7fb9341565c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:38.344 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:38.339+0000 7fb933fff640 1 -- 172.21.15.169:0/1639656574 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb93414e780 msgr2=0x7fb934156b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:38.345 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:38.339+0000 7fb933fff640 1 --2- 172.21.15.169:0/1639656574 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb93414e780 0x7fb934156b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:38.345 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:38.339+0000 7fb933fff640 1 -- 172.21.15.169:0/1639656574 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb91c009560 con 0x7fb9340b0990 2023-10-26T01:33:38.345 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:38.339+0000 7fb92bfff640 1 --2- 172.21.15.169:0/1639656574 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb9340cd6a0 0x7fb9341565c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:33:38.345 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:38.339+0000 7fb938f0b640 1 --2- 172.21.15.169:0/1639656574 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb93414e780 0x7fb934156b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:33:38.346 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:38.339+0000 7fb933fff640 1 --2- 172.21.15.169:0/1639656574 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9340b0990 0x7fb934149a50 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fb92400c3c0 tx=0x7fb92400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:38.346 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:38.339+0000 7fb9317fa640 1 -- 172.21.15.169:0/1639656574 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9240052d0 con 0x7fb9340b0990 2023-10-26T01:33:38.346 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:38.339+0000 7fb9317fa640 1 -- 172.21.15.169:0/1639656574 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb924007020 con 0x7fb9340b0990 2023-10-26T01:33:38.346 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:38.339+0000 7fb93a96e640 1 -- 172.21.15.169:0/1639656574 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb93406f130 con 0x7fb9340b0990 2023-10-26T01:33:38.346 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:38.339+0000 7fb9317fa640 1 -- 172.21.15.169:0/1639656574 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb924007470 con 0x7fb9340b0990 2023-10-26T01:33:38.347 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:38.339+0000 7fb93a96e640 1 -- 172.21.15.169:0/1639656574 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9341f2150 con 0x7fb9340b0990 2023-10-26T01:33:38.348 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:38.340+0000 7fb93a96e640 1 -- 172.21.15.169:0/1639656574 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb934066af0 con 0x7fb9340b0990 2023-10-26T01:33:38.348 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:38.341+0000 7fb9317fa640 1 -- 172.21.15.169:0/1639656574 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fb9240067d0 con 0x7fb9340b0990 2023-10-26T01:33:38.348 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:38.342+0000 7fb9317fa640 1 --2- 172.21.15.169:0/1639656574 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb91406ae40 0x7fb91406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:38.348 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:38.342+0000 7fb9317fa640 1 -- 172.21.15.169:0/1639656574 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fb924088980 con 0x7fb9340b0990 2023-10-26T01:33:38.349 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:38.343+0000 7fb92bfff640 1 --2- 172.21.15.169:0/1639656574 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb91406ae40 0x7fb91406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:38.349 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:38.344+0000 7fb92bfff640 1 --2- 172.21.15.169:0/1639656574 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb91406ae40 0x7fb91406d300 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fb91c035000 tx=0x7fb91c004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:38.349 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:38.347+0000 7fb9317fa640 1 -- 172.21.15.169:0/1639656574 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb924053ce0 con 0x7fb9340b0990 2023-10-26T01:33:38.537 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:38.535+0000 7fb93a96e640 1 -- 172.21.15.169:0/1639656574 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fb93406ef90 con 0x7fb9340b0990 2023-10-26T01:33:39.237 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:39.235+0000 7fb9317fa640 1 -- 172.21.15.169:0/1639656574 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v62) v1 ==== 138+0+0 (secure 0 0 0) 0x7fb93406ef90 con 0x7fb9340b0990 2023-10-26T01:33:39.237 INFO:teuthology.orchestra.run.smithi169.stderr:pool 'cephfs_metadata' created 2023-10-26T01:33:39.243 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:39.241+0000 7fb93a96e640 1 -- 172.21.15.169:0/1639656574 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb91406ae40 msgr2=0x7fb91406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:39.243 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:39.241+0000 7fb93a96e640 1 --2- 172.21.15.169:0/1639656574 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb91406ae40 0x7fb91406d300 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fb91c035000 tx=0x7fb91c004730 comp rx=0 tx=0).stop 2023-10-26T01:33:39.243 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:39.241+0000 7fb93a96e640 1 -- 172.21.15.169:0/1639656574 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9340b0990 msgr2=0x7fb934149a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:39.244 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:39.241+0000 7fb93a96e640 1 --2- 172.21.15.169:0/1639656574 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9340b0990 0x7fb934149a50 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fb92400c3c0 tx=0x7fb92400c880 comp rx=0 tx=0).stop 2023-10-26T01:33:39.244 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:39.242+0000 7fb93a96e640 1 -- 172.21.15.169:0/1639656574 shutdown_connections 2023-10-26T01:33:39.245 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:39.242+0000 7fb93a96e640 1 --2- 172.21.15.169:0/1639656574 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb91406ae40 0x7fb91406d300 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:39.245 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:39.242+0000 7fb93a96e640 1 --2- 172.21.15.169:0/1639656574 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb93414e780 0x7fb934156b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:39.245 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:39.242+0000 7fb93a96e640 1 --2- 172.21.15.169:0/1639656574 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb9340cd6a0 0x7fb9341565c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:39.245 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:39.242+0000 7fb93a96e640 1 --2- 172.21.15.169:0/1639656574 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9340b0990 0x7fb934149a50 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:39.245 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:39.242+0000 7fb93a96e640 1 -- 172.21.15.169:0/1639656574 >> 172.21.15.169:0/1639656574 conn(0x7fb934069930 msgr2=0x7fb93414d9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:39.246 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:39.242+0000 7fb93a96e640 1 -- 172.21.15.169:0/1639656574 shutdown_connections 2023-10-26T01:33:39.246 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:39.242+0000 7fb93a96e640 1 -- 172.21.15.169:0/1639656574 wait complete. 2023-10-26T01:33:39.270 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-26T01:33:39.496 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:39.493+0000 7f618ffff640 1 -- 172.21.15.169:0/1876815264 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61900cef30 msgr2=0x7f61900b45d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:39.496 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:39.493+0000 7f618ffff640 1 --2- 172.21.15.169:0/1876815264 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61900cef30 0x7f61900b45d0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f6178005fd0 tx=0x7f617802bd10 comp rx=0 tx=0).stop 2023-10-26T01:33:39.497 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:39.494+0000 7f618ffff640 1 -- 172.21.15.169:0/1876815264 shutdown_connections 2023-10-26T01:33:39.497 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:39.494+0000 7f618ffff640 1 --2- 172.21.15.169:0/1876815264 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f61900b4b10 0x7f61900d0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:39.497 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:39.494+0000 7f618ffff640 1 --2- 172.21.15.169:0/1876815264 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61900cef30 0x7f61900b45d0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:39.497 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:39.494+0000 7f618ffff640 1 -- 172.21.15.169:0/1876815264 >> 172.21.15.169:0/1876815264 conn(0x7f61900d1360 msgr2=0x7f61900d1770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:39.497 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:39.494+0000 7f618ffff640 1 -- 172.21.15.169:0/1876815264 shutdown_connections 2023-10-26T01:33:39.498 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:39.494+0000 7f618ffff640 1 -- 172.21.15.169:0/1876815264 wait complete. 2023-10-26T01:33:39.500 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:39.496+0000 7f618ffff640 1 Processor -- start 2023-10-26T01:33:39.500 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:39.496+0000 7f618ffff640 1 -- start start 2023-10-26T01:33:39.501 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:39.497+0000 7f618ffff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61900b4b10 0x7f61900a8f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:39.501 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:39.497+0000 7f618ffff640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f61900b68d0 0x7f61901506c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:39.501 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:39.497+0000 7f618ffff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f61900cef30 0x7f61900ad610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:39.501 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:39.497+0000 7f618ffff640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61900aa370 con 0x7f61900b4b10 2023-10-26T01:33:39.502 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:39.497+0000 7f618ffff640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61900a9760 con 0x7f61900cef30 2023-10-26T01:33:39.502 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:39.497+0000 7f618ffff640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61900a9a40 con 0x7f61900b68d0 2023-10-26T01:33:39.502 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:39.497+0000 7f618effd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61900b4b10 0x7f61900a8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:39.503 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:39.497+0000 7f618effd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61900b4b10 0x7f61900a8f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:54046/0 (socket says 172.21.15.169:54046) 2023-10-26T01:33:39.503 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:39.497+0000 7f618effd640 1 -- 172.21.15.169:0/2074553163 learned_addr learned my addr 172.21.15.169:0/2074553163 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:33:39.503 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:39.497+0000 7f618f7fe640 1 --2- 172.21.15.169:0/2074553163 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f61900cef30 0x7f61900ad610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:39.503 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:39.497+0000 7f6187fff640 1 --2- 172.21.15.169:0/2074553163 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f61900b68d0 0x7f61901506c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:39.503 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:39.498+0000 7f618effd640 1 -- 172.21.15.169:0/2074553163 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f61900b68d0 msgr2=0x7f61901506c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:39.503 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:39.498+0000 7f618effd640 1 --2- 172.21.15.169:0/2074553163 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f61900b68d0 0x7f61901506c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:39.504 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:39.498+0000 7f618effd640 1 -- 172.21.15.169:0/2074553163 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f61900cef30 msgr2=0x7f61900ad610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:39.504 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:39.498+0000 7f618effd640 1 --2- 172.21.15.169:0/2074553163 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f61900cef30 0x7f61900ad610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:39.504 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:39.498+0000 7f618effd640 1 -- 172.21.15.169:0/2074553163 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6178009560 con 0x7f61900b4b10 2023-10-26T01:33:39.504 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:39.498+0000 7f618f7fe640 1 --2- 172.21.15.169:0/2074553163 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f61900cef30 0x7f61900ad610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:33:39.505 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:39.498+0000 7f6187fff640 1 --2- 172.21.15.169:0/2074553163 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f61900b68d0 0x7f61901506c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:33:39.505 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:39.498+0000 7f618effd640 1 --2- 172.21.15.169:0/2074553163 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61900b4b10 0x7f61900a8f10 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f618000c430 tx=0x7f618000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:39.505 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:39.498+0000 7f61877fe640 1 -- 172.21.15.169:0/2074553163 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6180011790 con 0x7f61900b4b10 2023-10-26T01:33:39.505 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:39.498+0000 7f61877fe640 1 -- 172.21.15.169:0/2074553163 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f61800119e0 con 0x7f61900b4b10 2023-10-26T01:33:39.505 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:39.498+0000 7f61877fe640 1 -- 172.21.15.169:0/2074553163 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6180011e30 con 0x7f61900b4b10 2023-10-26T01:33:39.506 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:39.498+0000 7f618ffff640 1 -- 172.21.15.169:0/2074553163 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61900a9d20 con 0x7f61900b4b10 2023-10-26T01:33:39.506 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:39.498+0000 7f618ffff640 1 -- 172.21.15.169:0/2074553163 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61900a9450 con 0x7f61900b4b10 2023-10-26T01:33:39.507 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:39.499+0000 7f618ffff640 1 -- 172.21.15.169:0/2074553163 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6190073c10 con 0x7f61900b4b10 2023-10-26T01:33:39.507 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:39.501+0000 7f61877fe640 1 -- 172.21.15.169:0/2074553163 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f618001c460 con 0x7f61900b4b10 2023-10-26T01:33:39.507 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:39.502+0000 7f61877fe640 1 --2- 172.21.15.169:0/2074553163 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f617006ae40 0x7f617006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:39.507 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:39.502+0000 7f61877fe640 1 -- 172.21.15.169:0/2074553163 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f61800896e0 con 0x7f61900b4b10 2023-10-26T01:33:39.508 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:39.502+0000 7f6187fff640 1 --2- 172.21.15.169:0/2074553163 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f617006ae40 0x7f617006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:39.508 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:39.503+0000 7f6187fff640 1 --2- 172.21.15.169:0/2074553163 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f617006ae40 0x7f617006d300 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f6178035000 tx=0x7f6178004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:39.508 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:39.507+0000 7f61877fe640 1 -- 172.21.15.169:0/2074553163 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f61800548d0 con 0x7f61900b4b10 2023-10-26T01:33:39.700 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:39.698+0000 7f618ffff640 1 -- 172.21.15.169:0/2074553163 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f61900b4190 con 0x7f61900b4b10 2023-10-26T01:33:40.244 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:40.242+0000 7f61877fe640 1 -- 172.21.15.169:0/2074553163 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v63) v1 ==== 170+0+0 (secure 0 0 0) 0x7f6180089030 con 0x7f61900b4b10 2023-10-26T01:33:40.244 INFO:teuthology.orchestra.run.smithi169.stderr:pool 'cephfs_data' created 2023-10-26T01:33:40.247 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:40.245+0000 7f618ffff640 1 -- 172.21.15.169:0/2074553163 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f617006ae40 msgr2=0x7f617006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:40.247 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:40.245+0000 7f618ffff640 1 --2- 172.21.15.169:0/2074553163 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f617006ae40 0x7f617006d300 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f6178035000 tx=0x7f6178004730 comp rx=0 tx=0).stop 2023-10-26T01:33:40.248 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:40.245+0000 7f618ffff640 1 -- 172.21.15.169:0/2074553163 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61900b4b10 msgr2=0x7f61900a8f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:40.248 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:40.245+0000 7f618ffff640 1 --2- 172.21.15.169:0/2074553163 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61900b4b10 0x7f61900a8f10 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f618000c430 tx=0x7f618000c8f0 comp rx=0 tx=0).stop 2023-10-26T01:33:40.248 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:40.245+0000 7f618ffff640 1 -- 172.21.15.169:0/2074553163 shutdown_connections 2023-10-26T01:33:40.248 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:40.245+0000 7f618ffff640 1 --2- 172.21.15.169:0/2074553163 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f617006ae40 0x7f617006d300 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:40.249 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:40.245+0000 7f618ffff640 1 --2- 172.21.15.169:0/2074553163 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f61900cef30 0x7f61900ad610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:40.249 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:40.245+0000 7f618ffff640 1 --2- 172.21.15.169:0/2074553163 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f61900b68d0 0x7f61901506c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:40.249 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:40.245+0000 7f618ffff640 1 --2- 172.21.15.169:0/2074553163 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61900b4b10 0x7f61900a8f10 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:40.249 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:40.245+0000 7f618ffff640 1 -- 172.21.15.169:0/2074553163 >> 172.21.15.169:0/2074553163 conn(0x7f61900d1360 msgr2=0x7f61900d1740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:40.249 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:40.246+0000 7f618ffff640 1 -- 172.21.15.169:0/2074553163 shutdown_connections 2023-10-26T01:33:40.250 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:40.246+0000 7f618ffff640 1 -- 172.21.15.169:0/2074553163 wait complete. 2023-10-26T01:33:40.276 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-26T01:33:40.476 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:40.474+0000 7f88d48b3640 1 -- 172.21.15.169:0/2120217237 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88bc0047c0 msgr2=0x7f88bc004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:40.476 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:40.474+0000 7f88d48b3640 1 --2- 172.21.15.169:0/2120217237 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88bc0047c0 0x7f88bc004c40 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f88d0159ad0 tx=0x7f88bc0057f0 comp rx=0 tx=0).stop 2023-10-26T01:33:40.477 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:40.474+0000 7f88cf59e640 1 -- 172.21.15.169:0/2120217237 reap_dead start 2023-10-26T01:33:40.477 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:40.474+0000 7f88d48b3640 1 -- 172.21.15.169:0/2120217237 shutdown_connections 2023-10-26T01:33:40.477 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:40.474+0000 7f88d48b3640 1 -- 172.21.15.169:0/2120217237 >> 172.21.15.169:0/2120217237 conn(0x7f88d00af110 msgr2=0x7f88d00af520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:40.477 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:40.474+0000 7f88d48b3640 1 -- 172.21.15.169:0/2120217237 shutdown_connections 2023-10-26T01:33:40.477 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:40.474+0000 7f88d48b3640 1 -- 172.21.15.169:0/2120217237 wait complete. 2023-10-26T01:33:40.478 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:40.476+0000 7f88d48b3640 1 Processor -- start 2023-10-26T01:33:40.478 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:40.476+0000 7f88d48b3640 1 -- start start 2023-10-26T01:33:40.480 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:40.477+0000 7f88d48b3640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f88bc0047c0 0x7f88d0144250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:40.481 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:40.477+0000 7f88d48b3640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88d0157f30 0x7f88d01583b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:40.481 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:40.477+0000 7f88d48b3640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f88d01e6240 0x7f88d01e66c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:40.481 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:40.477+0000 7f88d48b3640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88d01555a0 con 0x7f88d0157f30 2023-10-26T01:33:40.481 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:40.477+0000 7f88d48b3640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88d01e6c00 con 0x7f88bc0047c0 2023-10-26T01:33:40.481 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:40.477+0000 7f88d48b3640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88d01540a0 con 0x7f88d01e6240 2023-10-26T01:33:40.482 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:40.477+0000 7f88c7fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88d0157f30 0x7f88d01583b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:40.482 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:40.478+0000 7f88c7fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88d0157f30 0x7f88d01583b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:54062/0 (socket says 172.21.15.169:54062) 2023-10-26T01:33:40.482 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:40.478+0000 7f88c7fff640 1 -- 172.21.15.169:0/2571849763 learned_addr learned my addr 172.21.15.169:0/2571849763 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:33:40.482 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:40.478+0000 7f88c7fff640 1 -- 172.21.15.169:0/2571849763 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f88d01e6240 msgr2=0x7f88d01e66c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:40.482 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:40.478+0000 7f88cf59e640 1 --2- 172.21.15.169:0/2571849763 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f88d01e6240 0x7f88d01e66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:40.483 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:40.478+0000 7f88ced9d640 1 --2- 172.21.15.169:0/2571849763 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f88bc0047c0 0x7f88d0144250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:40.483 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:40.478+0000 7f88c7fff640 1 --2- 172.21.15.169:0/2571849763 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f88d01e6240 0x7f88d01e66c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:40.483 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:40.478+0000 7f88c7fff640 1 -- 172.21.15.169:0/2571849763 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f88bc0047c0 msgr2=0x7f88d0144250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:40.483 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:40.478+0000 7f88c7fff640 1 --2- 172.21.15.169:0/2571849763 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f88bc0047c0 0x7f88d0144250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:40.483 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:40.478+0000 7f88c7fff640 1 -- 172.21.15.169:0/2571849763 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88d0154380 con 0x7f88d0157f30 2023-10-26T01:33:40.484 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:40.478+0000 7f88cf59e640 1 --2- 172.21.15.169:0/2571849763 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f88d01e6240 0x7f88d01e66c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:33:40.484 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:40.478+0000 7f88ced9d640 1 --2- 172.21.15.169:0/2571849763 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f88bc0047c0 0x7f88d0144250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:33:40.484 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:40.478+0000 7f88c7fff640 1 --2- 172.21.15.169:0/2571849763 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88d0157f30 0x7f88d01583b0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f88b8001740 tx=0x7f88b8003de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:40.484 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:40.479+0000 7f88c77fe640 1 -- 172.21.15.169:0/2571849763 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b803b070 con 0x7f88d0157f30 2023-10-26T01:33:40.484 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:40.479+0000 7f88c77fe640 1 -- 172.21.15.169:0/2571849763 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f88b8006260 con 0x7f88d0157f30 2023-10-26T01:33:40.485 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:40.479+0000 7f88d48b3640 1 -- 172.21.15.169:0/2571849763 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88d0154660 con 0x7f88d0157f30 2023-10-26T01:33:40.485 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:40.479+0000 7f88c77fe640 1 -- 172.21.15.169:0/2571849763 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b802d710 con 0x7f88d0157f30 2023-10-26T01:33:40.485 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:40.479+0000 7f88d48b3640 1 -- 172.21.15.169:0/2571849763 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88d0153db0 con 0x7f88d0157f30 2023-10-26T01:33:40.486 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:40.480+0000 7f88d48b3640 1 -- 172.21.15.169:0/2571849763 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88d00afdb0 con 0x7f88d0157f30 2023-10-26T01:33:40.486 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:40.481+0000 7f88c77fe640 1 -- 172.21.15.169:0/2571849763 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f88b802d870 con 0x7f88d0157f30 2023-10-26T01:33:40.486 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:40.482+0000 7f88c77fe640 1 --2- 172.21.15.169:0/2571849763 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f88b006ae70 0x7f88b006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:40.487 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:40.482+0000 7f88c77fe640 1 -- 172.21.15.169:0/2571849763 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f88b80ab430 con 0x7f88d0157f30 2023-10-26T01:33:40.487 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:40.483+0000 7f88ced9d640 1 --2- 172.21.15.169:0/2571849763 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f88b006ae70 0x7f88b006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:40.487 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:40.484+0000 7f88ced9d640 1 --2- 172.21.15.169:0/2571849763 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f88b006ae70 0x7f88b006d330 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f88c0001200 tx=0x7f88c0001c10 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:40.488 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:40.487+0000 7f88c77fe640 1 -- 172.21.15.169:0/2571849763 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f88b8076420 con 0x7f88d0157f30 2023-10-26T01:33:40.716 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:40.715+0000 7f88d48b3640 1 -- 172.21.15.169:0/2571849763 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f88d0073c10 con 0x7f88d0157f30 2023-10-26T01:33:41.300 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:41.297+0000 7f88c77fe640 1 -- 172.21.15.169:0/2571849763 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v28) v1 ==== 172+0+0 (secure 0 0 0) 0x7f88b8079a40 con 0x7f88d0157f30 2023-10-26T01:33:41.300 INFO:teuthology.orchestra.run.smithi169.stderr:new fs with metadata pool 8 and data pool 9 2023-10-26T01:33:41.307 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:41.305+0000 7f88d48b3640 1 -- 172.21.15.169:0/2571849763 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f88b006ae70 msgr2=0x7f88b006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:41.307 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:41.305+0000 7f88d48b3640 1 --2- 172.21.15.169:0/2571849763 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f88b006ae70 0x7f88b006d330 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f88c0001200 tx=0x7f88c0001c10 comp rx=0 tx=0).stop 2023-10-26T01:33:41.307 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:41.305+0000 7f88d48b3640 1 -- 172.21.15.169:0/2571849763 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88d0157f30 msgr2=0x7f88d01583b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:41.307 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:41.305+0000 7f88d48b3640 1 --2- 172.21.15.169:0/2571849763 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88d0157f30 0x7f88d01583b0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f88b8001740 tx=0x7f88b8003de0 comp rx=0 tx=0).stop 2023-10-26T01:33:41.308 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:41.305+0000 7f88d48b3640 1 -- 172.21.15.169:0/2571849763 shutdown_connections 2023-10-26T01:33:41.308 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:41.305+0000 7f88d48b3640 1 --2- 172.21.15.169:0/2571849763 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f88b006ae70 0x7f88b006d330 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:41.308 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:41.305+0000 7f88d48b3640 1 --2- 172.21.15.169:0/2571849763 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88d0157f30 0x7f88d01583b0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:41.309 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:41.305+0000 7f88d48b3640 1 --2- 172.21.15.169:0/2571849763 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f88d01e6240 0x7f88d01e66c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:41.309 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:41.305+0000 7f88d48b3640 1 --2- 172.21.15.169:0/2571849763 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f88bc0047c0 0x7f88d0144250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:41.309 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:41.305+0000 7f88d48b3640 1 -- 172.21.15.169:0/2571849763 >> 172.21.15.169:0/2571849763 conn(0x7f88d00af110 msgr2=0x7f88d00acc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:41.309 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:41.306+0000 7f88d48b3640 1 -- 172.21.15.169:0/2571849763 shutdown_connections 2023-10-26T01:33:41.309 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:41.306+0000 7f88d48b3640 1 -- 172.21.15.169:0/2571849763 wait complete. 2023-10-26T01:33:41.333 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:33:41.526 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:41.524+0000 7f251baa9640 1 -- 172.21.15.169:0/131128373 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f251406ce20 msgr2=0x7f251414e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:41.526 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:41.524+0000 7f251baa9640 1 --2- 172.21.15.169:0/131128373 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f251406ce20 0x7f251414e790 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2510005fd0 tx=0x7f251002bd10 comp rx=0 tx=0).stop 2023-10-26T01:33:41.526 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:41.524+0000 7f251baa9640 1 -- 172.21.15.169:0/131128373 shutdown_connections 2023-10-26T01:33:41.527 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:41.524+0000 7f251baa9640 1 --2- 172.21.15.169:0/131128373 >> v2:172.21.15.170:3300/0 conn(0x7f2514158850 0x7f25140aab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:41.527 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:41.524+0000 7f251baa9640 1 --2- 172.21.15.169:0/131128373 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f251406ce20 0x7f251414e790 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:41.527 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:41.524+0000 7f251baa9640 1 -- 172.21.15.169:0/131128373 >> 172.21.15.169:0/131128373 conn(0x7f2514069930 msgr2=0x7f2514069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:41.527 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:41.524+0000 7f251baa9640 1 -- 172.21.15.169:0/131128373 shutdown_connections 2023-10-26T01:33:41.527 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:41.524+0000 7f251baa9640 1 -- 172.21.15.169:0/131128373 wait complete. 2023-10-26T01:33:41.528 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:41.526+0000 7f251baa9640 1 Processor -- start 2023-10-26T01:33:41.528 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:41.526+0000 7f251baa9640 1 -- start start 2023-10-26T01:33:41.530 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:41.527+0000 7f251baa9640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f251406ce20 0x7f2514142720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:41.531 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:41.527+0000 7f251baa9640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25140cd720 0x7f25140700a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:41.531 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:41.527+0000 7f251baa9640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2514158850 0x7f25140705e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:41.531 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:41.527+0000 7f251baa9640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25141526b0 con 0x7f251406ce20 2023-10-26T01:33:41.531 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:41.527+0000 7f251baa9640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25141534d0 con 0x7f25140cd720 2023-10-26T01:33:41.531 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:41.527+0000 7f251baa9640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f251415ab70 con 0x7f2514158850 2023-10-26T01:33:41.532 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:41.527+0000 7f2519845640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f251406ce20 0x7f2514142720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:41.532 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:41.527+0000 7f2519845640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f251406ce20 0x7f2514142720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:54066/0 (socket says 172.21.15.169:54066) 2023-10-26T01:33:41.532 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:41.527+0000 7f2519845640 1 -- 172.21.15.169:0/3713273239 learned_addr learned my addr 172.21.15.169:0/3713273239 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:33:41.532 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:41.528+0000 7f2519044640 1 --2- 172.21.15.169:0/3713273239 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25140cd720 0x7f25140700a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:41.532 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:41.528+0000 7f251a046640 1 --2- 172.21.15.169:0/3713273239 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2514158850 0x7f25140705e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:41.533 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:41.528+0000 7f2519845640 1 -- 172.21.15.169:0/3713273239 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2514158850 msgr2=0x7f25140705e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:41.533 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:41.528+0000 7f2519845640 1 --2- 172.21.15.169:0/3713273239 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2514158850 0x7f25140705e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:41.533 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:41.528+0000 7f2519845640 1 -- 172.21.15.169:0/3713273239 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25140cd720 msgr2=0x7f25140700a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:41.533 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:41.528+0000 7f2519845640 1 --2- 172.21.15.169:0/3713273239 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25140cd720 0x7f25140700a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:41.533 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:41.528+0000 7f2519845640 1 -- 172.21.15.169:0/3713273239 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2510009560 con 0x7f251406ce20 2023-10-26T01:33:41.534 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:41.528+0000 7f251a046640 1 --2- 172.21.15.169:0/3713273239 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2514158850 0x7f25140705e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:33:41.534 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:41.528+0000 7f2519845640 1 --2- 172.21.15.169:0/3713273239 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f251406ce20 0x7f2514142720 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f2510036040 tx=0x7f251002dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:41.534 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:41.529+0000 7f250a7fc640 1 -- 172.21.15.169:0/3713273239 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25100035f0 con 0x7f251406ce20 2023-10-26T01:33:41.534 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:41.529+0000 7f250a7fc640 1 -- 172.21.15.169:0/3713273239 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2510003840 con 0x7f251406ce20 2023-10-26T01:33:41.534 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:41.529+0000 7f251baa9640 1 -- 172.21.15.169:0/3713273239 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f251415a890 con 0x7f251406ce20 2023-10-26T01:33:41.534 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:41.529+0000 7f250a7fc640 1 -- 172.21.15.169:0/3713273239 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2510003bf0 con 0x7f251406ce20 2023-10-26T01:33:41.535 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:41.529+0000 7f251baa9640 1 -- 172.21.15.169:0/3713273239 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25140af2d0 con 0x7f251406ce20 2023-10-26T01:33:41.536 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:41.530+0000 7f251baa9640 1 -- 172.21.15.169:0/3713273239 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f251404db10 con 0x7f251406ce20 2023-10-26T01:33:41.536 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:41.531+0000 7f250a7fc640 1 -- 172.21.15.169:0/3713273239 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f2510003d50 con 0x7f251406ce20 2023-10-26T01:33:41.536 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:41.532+0000 7f250a7fc640 1 --2- 172.21.15.169:0/3713273239 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f24f806ae70 0x7f24f806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:41.536 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:41.532+0000 7f250a7fc640 1 -- 172.21.15.169:0/3713273239 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f25100ac160 con 0x7f251406ce20 2023-10-26T01:33:41.536 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:41.533+0000 7f2519044640 1 --2- 172.21.15.169:0/3713273239 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f24f806ae70 0x7f24f806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:41.537 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:41.534+0000 7f2519044640 1 --2- 172.21.15.169:0/3713273239 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f24f806ae70 0x7f24f806d330 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f25140b20e0 tx=0x7f2504005f90 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:41.538 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:41.537+0000 7f250a7fc640 1 -- 172.21.15.169:0/3713273239 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f25100772b0 con 0x7f251406ce20 2023-10-26T01:33:41.729 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:41.727+0000 7f251baa9640 1 -- 172.21.15.169:0/3713273239 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f251414f550 con 0x7f251406ce20 2023-10-26T01:33:41.731 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:41.730+0000 7f250a7fc640 1 -- 172.21.15.169:0/3713273239 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) v1 ==== 74+0+16626 (secure 0 0 0) 0x7f251007a8d0 con 0x7f251406ce20 2023-10-26T01:33:41.732 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:33:41.732 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":64,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:33:40.716737+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-10-26T01:33:38.593237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-10-26T01:33:39.753692+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:33:41.734 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:41.732+0000 7f251baa9640 1 -- 172.21.15.169:0/3713273239 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f24f806ae70 msgr2=0x7f24f806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:41.735 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:41.732+0000 7f251baa9640 1 --2- 172.21.15.169:0/3713273239 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f24f806ae70 0x7f24f806d330 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f25140b20e0 tx=0x7f2504005f90 comp rx=0 tx=0).stop 2023-10-26T01:33:41.735 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:41.732+0000 7f251baa9640 1 -- 172.21.15.169:0/3713273239 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f251406ce20 msgr2=0x7f2514142720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:41.735 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:41.732+0000 7f251baa9640 1 --2- 172.21.15.169:0/3713273239 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f251406ce20 0x7f2514142720 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f2510036040 tx=0x7f251002dab0 comp rx=0 tx=0).stop 2023-10-26T01:33:41.735 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:41.733+0000 7f251baa9640 1 -- 172.21.15.169:0/3713273239 shutdown_connections 2023-10-26T01:33:41.735 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:41.733+0000 7f251baa9640 1 --2- 172.21.15.169:0/3713273239 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f24f806ae70 0x7f24f806d330 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:41.736 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:41.733+0000 7f251baa9640 1 --2- 172.21.15.169:0/3713273239 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2514158850 0x7f25140705e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:41.736 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:41.733+0000 7f251baa9640 1 --2- 172.21.15.169:0/3713273239 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25140cd720 0x7f25140700a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:41.736 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:41.733+0000 7f251baa9640 1 --2- 172.21.15.169:0/3713273239 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f251406ce20 0x7f2514142720 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:41.736 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:41.733+0000 7f251baa9640 1 -- 172.21.15.169:0/3713273239 >> 172.21.15.169:0/3713273239 conn(0x7f2514069930 msgr2=0x7f251414ed60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:41.736 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:41.733+0000 7f251baa9640 1 -- 172.21.15.169:0/3713273239 shutdown_connections 2023-10-26T01:33:41.736 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:41.733+0000 7f251baa9640 1 -- 172.21.15.169:0/3713273239 wait complete. 2023-10-26T01:33:41.753 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:33:41.971 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:41.969+0000 7f3017b4c640 1 -- 172.21.15.169:0/2585586167 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2ffc0039a0 msgr2=0x7f2ffc003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:41.971 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:41.969+0000 7f3017b4c640 1 --2- 172.21.15.169:0/2585586167 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2ffc0039a0 0x7f2ffc003dc0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f300c001d80 tx=0x7f2ffc0057f0 comp rx=0 tx=0).stop 2023-10-26T01:33:41.972 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:41.969+0000 7f30160e9640 1 -- 172.21.15.169:0/2585586167 reap_dead start 2023-10-26T01:33:41.972 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:41.969+0000 7f3017b4c640 1 -- 172.21.15.169:0/2585586167 shutdown_connections 2023-10-26T01:33:41.972 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:41.969+0000 7f3017b4c640 1 -- 172.21.15.169:0/2585586167 >> 172.21.15.169:0/2585586167 conn(0x7f3010069930 msgr2=0x7f3010069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:41.972 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:41.969+0000 7f3017b4c640 1 -- 172.21.15.169:0/2585586167 shutdown_connections 2023-10-26T01:33:41.973 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:41.969+0000 7f3017b4c640 1 -- 172.21.15.169:0/2585586167 wait complete. 2023-10-26T01:33:41.975 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:41.971+0000 7f3017b4c640 1 Processor -- start 2023-10-26T01:33:41.975 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:41.971+0000 7f3017b4c640 1 -- start start 2023-10-26T01:33:41.975 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:41.972+0000 7f3017b4c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ffc0039a0 0x7f3010149760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:41.976 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:41.972+0000 7f3017b4c640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3010155980 0x7f3010147820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:41.976 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:41.972+0000 7f3017b4c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3010157040 0x7f30101e4280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:41.976 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:41.972+0000 7f3017b4c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3010152300 con 0x7f3010157040 2023-10-26T01:33:41.976 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:41.972+0000 7f3017b4c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3010155ef0 con 0x7f2ffc0039a0 2023-10-26T01:33:41.976 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:41.972+0000 7f3017b4c640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30101561d0 con 0x7f3010155980 2023-10-26T01:33:41.977 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:41.972+0000 7f30160e9640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3010157040 0x7f30101e4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:41.977 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:41.972+0000 7f30160e9640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3010157040 0x7f30101e4280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:54094/0 (socket says 172.21.15.169:54094) 2023-10-26T01:33:41.977 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:41.972+0000 7f30160e9640 1 -- 172.21.15.169:0/2722677158 learned_addr learned my addr 172.21.15.169:0/2722677158 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:33:41.977 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:41.973+0000 7f30158e8640 1 --2- 172.21.15.169:0/2722677158 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ffc0039a0 0x7f3010149760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:41.977 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:41.973+0000 7f30150e7640 1 --2- 172.21.15.169:0/2722677158 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3010155980 0x7f3010147820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:41.978 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:41.973+0000 7f30160e9640 1 -- 172.21.15.169:0/2722677158 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3010155980 msgr2=0x7f3010147820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:41.978 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:41.973+0000 7f30160e9640 1 --2- 172.21.15.169:0/2722677158 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3010155980 0x7f3010147820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:41.978 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:41.973+0000 7f30160e9640 1 -- 172.21.15.169:0/2722677158 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ffc0039a0 msgr2=0x7f3010149760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:41.978 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:41.973+0000 7f30160e9640 1 --2- 172.21.15.169:0/2722677158 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ffc0039a0 0x7f3010149760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:41.979 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:41.973+0000 7f30160e9640 1 -- 172.21.15.169:0/2722677158 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30101564b0 con 0x7f3010157040 2023-10-26T01:33:41.979 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:41.973+0000 7f30160e9640 1 --2- 172.21.15.169:0/2722677158 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3010157040 0x7f30101e4280 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f301015a1f0 tx=0x7f300800c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:41.979 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:41.973+0000 7f30067fc640 1 -- 172.21.15.169:0/2722677158 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3008005d50 con 0x7f3010157040 2023-10-26T01:33:41.979 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:41.973+0000 7f30067fc640 1 -- 172.21.15.169:0/2722677158 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f30080070d0 con 0x7f3010157040 2023-10-26T01:33:41.980 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:41.973+0000 7f30067fc640 1 -- 172.21.15.169:0/2722677158 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30080074e0 con 0x7f3010157040 2023-10-26T01:33:41.980 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:41.973+0000 7f3017b4c640 1 -- 172.21.15.169:0/2722677158 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3010156790 con 0x7f3010157040 2023-10-26T01:33:41.980 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:41.974+0000 7f3017b4c640 1 -- 172.21.15.169:0/2722677158 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30101576a0 con 0x7f3010157040 2023-10-26T01:33:41.981 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:41.975+0000 7f3017b4c640 1 -- 172.21.15.169:0/2722677158 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f301004db10 con 0x7f3010157040 2023-10-26T01:33:41.981 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:41.976+0000 7f30067fc640 1 -- 172.21.15.169:0/2722677158 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f300801a360 con 0x7f3010157040 2023-10-26T01:33:41.982 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:41.977+0000 7f30067fc640 1 --2- 172.21.15.169:0/2722677158 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2ff406ae40 0x7f2ff406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:41.982 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:41.977+0000 7f30067fc640 1 -- 172.21.15.169:0/2722677158 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f30080887c0 con 0x7f3010157040 2023-10-26T01:33:41.982 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:41.977+0000 7f30158e8640 1 --2- 172.21.15.169:0/2722677158 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2ff406ae40 0x7f2ff406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:41.982 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:41.978+0000 7f30158e8640 1 --2- 172.21.15.169:0/2722677158 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2ff406ae40 0x7f2ff406d300 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f300c034000 tx=0x7f300c039040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:41.983 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:41.981+0000 7f30067fc640 1 -- 172.21.15.169:0/2722677158 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3008053900 con 0x7f3010157040 2023-10-26T01:33:42.168 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2023-10-26T01:33:42.168 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2023-10-26T01:33:42.178 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:42.176+0000 7f3017b4c640 1 -- 172.21.15.169:0/2722677158 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f30100aca70 con 0x7f3010157040 2023-10-26T01:33:42.180 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:42.178+0000 7f30067fc640 1 -- 172.21.15.169:0/2722677158 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) v1 ==== 74+0+16626 (secure 0 0 0) 0x7f3008017020 con 0x7f3010157040 2023-10-26T01:33:42.180 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:33:42.181 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":64,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:33:40.716737+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-10-26T01:33:38.593237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-10-26T01:33:39.753692+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:33:42.183 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:42.181+0000 7f3017b4c640 1 -- 172.21.15.169:0/2722677158 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2ff406ae40 msgr2=0x7f2ff406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:42.183 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:42.181+0000 7f3017b4c640 1 --2- 172.21.15.169:0/2722677158 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2ff406ae40 0x7f2ff406d300 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f300c034000 tx=0x7f300c039040 comp rx=0 tx=0).stop 2023-10-26T01:33:42.183 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:42.181+0000 7f3017b4c640 1 -- 172.21.15.169:0/2722677158 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3010157040 msgr2=0x7f30101e4280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:42.183 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:42.181+0000 7f3017b4c640 1 --2- 172.21.15.169:0/2722677158 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3010157040 0x7f30101e4280 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f301015a1f0 tx=0x7f300800c960 comp rx=0 tx=0).stop 2023-10-26T01:33:42.184 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:42.181+0000 7f3017b4c640 1 -- 172.21.15.169:0/2722677158 shutdown_connections 2023-10-26T01:33:42.184 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:42.181+0000 7f3017b4c640 1 --2- 172.21.15.169:0/2722677158 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2ff406ae40 0x7f2ff406d300 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:42.184 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:42.181+0000 7f3017b4c640 1 --2- 172.21.15.169:0/2722677158 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3010157040 0x7f30101e4280 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:42.184 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:42.181+0000 7f3017b4c640 1 --2- 172.21.15.169:0/2722677158 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3010155980 0x7f3010147820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:42.184 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:42.181+0000 7f3017b4c640 1 --2- 172.21.15.169:0/2722677158 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ffc0039a0 0x7f3010149760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:42.185 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:42.181+0000 7f3017b4c640 1 -- 172.21.15.169:0/2722677158 >> 172.21.15.169:0/2722677158 conn(0x7f3010069930 msgr2=0x7f30100cd720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:42.185 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:42.181+0000 7f3017b4c640 1 -- 172.21.15.169:0/2722677158 shutdown_connections 2023-10-26T01:33:42.185 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:42.181+0000 7f3017b4c640 1 -- 172.21.15.169:0/2722677158 wait complete. 2023-10-26T01:33:42.203 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-26T01:33:42.414 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:42.412+0000 7f97fc840640 1 -- 172.21.15.169:0/221599369 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97e4003970 msgr2=0x7f97e4003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:42.414 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:42.412+0000 7f97fc840640 1 --2- 172.21.15.169:0/221599369 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97e4003970 0x7f97e4003d90 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f97e800bb50 tx=0x7f97e40057f0 comp rx=0 tx=0).stop 2023-10-26T01:33:42.415 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:42.412+0000 7f97f7fff640 1 -- 172.21.15.169:0/221599369 reap_dead start 2023-10-26T01:33:42.415 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:42.412+0000 7f97fc840640 1 -- 172.21.15.169:0/221599369 shutdown_connections 2023-10-26T01:33:42.415 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:42.412+0000 7f97fc840640 1 -- 172.21.15.169:0/221599369 >> 172.21.15.169:0/221599369 conn(0x7f97f80af110 msgr2=0x7f97f80af520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:42.415 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:42.412+0000 7f97fc840640 1 -- 172.21.15.169:0/221599369 shutdown_connections 2023-10-26T01:33:42.415 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:42.412+0000 7f97fc840640 1 -- 172.21.15.169:0/221599369 wait complete. 2023-10-26T01:33:42.416 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:42.414+0000 7f97fc840640 1 Processor -- start 2023-10-26T01:33:42.416 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:42.414+0000 7f97fc840640 1 -- start start 2023-10-26T01:33:42.417 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:42.416+0000 7f97fc840640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97e4003970 0x7f97f8099940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:42.418 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:42.416+0000 7f97fc840640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f97f8155a40 0x7f97f8153fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:42.418 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:42.416+0000 7f97fc840640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f97f8154500 0x7f97f809a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:42.418 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:42.416+0000 7f97fc840640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97f81549a0 con 0x7f97e4003970 2023-10-26T01:33:42.418 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:42.416+0000 7f97fc840640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97f8154c80 con 0x7f97f8155a40 2023-10-26T01:33:42.419 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:42.416+0000 7f97fc840640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97f8154f60 con 0x7f97f8154500 2023-10-26T01:33:42.419 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:42.416+0000 7f97f77fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97e4003970 0x7f97f8099940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:42.419 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:42.416+0000 7f97f77fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97e4003970 0x7f97f8099940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:54112/0 (socket says 172.21.15.169:54112) 2023-10-26T01:33:42.419 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:42.416+0000 7f97f77fe640 1 -- 172.21.15.169:0/1741637498 learned_addr learned my addr 172.21.15.169:0/1741637498 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:33:42.419 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:42.416+0000 7f97f77fe640 1 -- 172.21.15.169:0/1741637498 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f97f8154500 msgr2=0x7f97f809a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:42.420 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:42.416+0000 7f97f77fe640 1 --2- 172.21.15.169:0/1741637498 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f97f8154500 0x7f97f809a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:42.420 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:42.416+0000 7f97f77fe640 1 -- 172.21.15.169:0/1741637498 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f97f8155a40 msgr2=0x7f97f8153fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:42.420 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:42.416+0000 7f97f77fe640 1 --2- 172.21.15.169:0/1741637498 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f97f8155a40 0x7f97f8153fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:42.420 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:42.416+0000 7f97f77fe640 1 -- 172.21.15.169:0/1741637498 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97f8155240 con 0x7f97e4003970 2023-10-26T01:33:42.422 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:42.417+0000 7f97f77fe640 1 --2- 172.21.15.169:0/1741637498 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97e4003970 0x7f97f8099940 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f97e800baf0 tx=0x7f97e8036720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:42.422 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:42.417+0000 7f97f4ff9640 1 -- 172.21.15.169:0/1741637498 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e8036de0 con 0x7f97e4003970 2023-10-26T01:33:42.422 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:42.417+0000 7f97f4ff9640 1 -- 172.21.15.169:0/1741637498 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f97e8004120 con 0x7f97e4003970 2023-10-26T01:33:42.422 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:42.417+0000 7f97f4ff9640 1 -- 172.21.15.169:0/1741637498 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e80044f0 con 0x7f97e4003970 2023-10-26T01:33:42.423 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:42.417+0000 7f97fc840640 1 -- 172.21.15.169:0/1741637498 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97f8155520 con 0x7f97e4003970 2023-10-26T01:33:42.423 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:42.417+0000 7f97fc840640 1 -- 172.21.15.169:0/1741637498 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97f806c7e0 con 0x7f97e4003970 2023-10-26T01:33:42.423 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:42.418+0000 7f97fc840640 1 -- 172.21.15.169:0/1741637498 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97f81556b0 con 0x7f97e4003970 2023-10-26T01:33:42.423 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:42.419+0000 7f97f4ff9640 1 -- 172.21.15.169:0/1741637498 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f97e80035f0 con 0x7f97e4003970 2023-10-26T01:33:42.423 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:42.420+0000 7f97f4ff9640 1 --2- 172.21.15.169:0/1741637498 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f97d806ae40 0x7f97d806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:42.424 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:42.420+0000 7f97f4ff9640 1 -- 172.21.15.169:0/1741637498 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f97e80ab430 con 0x7f97e4003970 2023-10-26T01:33:42.424 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:42.421+0000 7f97effff640 1 --2- 172.21.15.169:0/1741637498 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f97d806ae40 0x7f97d806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:42.424 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:42.422+0000 7f97effff640 1 --2- 172.21.15.169:0/1741637498 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f97d806ae40 0x7f97d806d300 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f97f809b2d0 tx=0x7f97e00014b0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:42.426 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:42.425+0000 7f97f4ff9640 1 -- 172.21.15.169:0/1741637498 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f97e8076440 con 0x7f97e4003970 2023-10-26T01:33:42.638 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:42.636+0000 7f97fc840640 1 -- 172.21.15.169:0/1741637498 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f97f8073c10 con 0x7f97e4003970 2023-10-26T01:33:43.317 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:43.315+0000 7f97f4ff9640 1 -- 172.21.15.169:0/1741637498 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v31) v1 ==== 118+0+0 (secure 0 0 0) 0x7f97e8079a60 con 0x7f97e4003970 2023-10-26T01:33:43.321 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:43.319+0000 7f97fc840640 1 -- 172.21.15.169:0/1741637498 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f97d806ae40 msgr2=0x7f97d806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:43.321 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:43.319+0000 7f97fc840640 1 --2- 172.21.15.169:0/1741637498 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f97d806ae40 0x7f97d806d300 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f97f809b2d0 tx=0x7f97e00014b0 comp rx=0 tx=0).stop 2023-10-26T01:33:43.322 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:43.319+0000 7f97fc840640 1 -- 172.21.15.169:0/1741637498 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97e4003970 msgr2=0x7f97f8099940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:43.322 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:43.319+0000 7f97fc840640 1 --2- 172.21.15.169:0/1741637498 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97e4003970 0x7f97f8099940 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f97e800baf0 tx=0x7f97e8036720 comp rx=0 tx=0).stop 2023-10-26T01:33:43.322 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:43.319+0000 7f97fc840640 1 -- 172.21.15.169:0/1741637498 shutdown_connections 2023-10-26T01:33:43.322 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:43.319+0000 7f97fc840640 1 --2- 172.21.15.169:0/1741637498 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f97d806ae40 0x7f97d806d300 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:43.322 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:43.319+0000 7f97fc840640 1 --2- 172.21.15.169:0/1741637498 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f97f8154500 0x7f97f809a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:43.323 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:43.319+0000 7f97fc840640 1 --2- 172.21.15.169:0/1741637498 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f97f8155a40 0x7f97f8153fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:43.323 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:43.319+0000 7f97fc840640 1 --2- 172.21.15.169:0/1741637498 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97e4003970 0x7f97f8099940 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:43.323 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:43.319+0000 7f97fc840640 1 -- 172.21.15.169:0/1741637498 >> 172.21.15.169:0/1741637498 conn(0x7f97f80af110 msgr2=0x7f97f815c8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:43.323 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:43.319+0000 7f97fc840640 1 -- 172.21.15.169:0/1741637498 shutdown_connections 2023-10-26T01:33:43.323 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:43.320+0000 7f97fc840640 1 -- 172.21.15.169:0/1741637498 wait complete. 2023-10-26T01:33:43.342 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-26T01:33:43.342 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-26T01:33:43.551 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:43.549+0000 7f356490f640 1 -- 172.21.15.169:0/237114 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f356006ce50 msgr2=0x7f3560158bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:43.552 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:43.549+0000 7f356490f640 1 --2- 172.21.15.169:0/237114 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f356006ce50 0x7f3560158bd0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f35600b2790 tx=0x7f354802bdc0 comp rx=0 tx=0).stop 2023-10-26T01:33:43.552 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:43.549+0000 7f356490f640 1 -- 172.21.15.169:0/237114 shutdown_connections 2023-10-26T01:33:43.552 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:43.549+0000 7f356490f640 1 --2- 172.21.15.169:0/237114 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f356006ce50 0x7f3560158bd0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:43.552 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:43.549+0000 7f356490f640 1 --2- 172.21.15.169:0/237114 >> v2:172.21.15.170:3300/0 conn(0x7f3560157be0 0x7f356006b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:43.552 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:43.549+0000 7f356490f640 1 -- 172.21.15.169:0/237114 >> 172.21.15.169:0/237114 conn(0x7f3560069930 msgr2=0x7f3560069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:43.553 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:43.549+0000 7f356490f640 1 -- 172.21.15.169:0/237114 shutdown_connections 2023-10-26T01:33:43.553 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:43.549+0000 7f356490f640 1 -- 172.21.15.169:0/237114 wait complete. 2023-10-26T01:33:43.554 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:43.551+0000 7f356490f640 1 Processor -- start 2023-10-26T01:33:43.554 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:43.551+0000 7f356490f640 1 -- start start 2023-10-26T01:33:43.554 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:43.552+0000 7f356490f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f356006ce50 0x7f3560154080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:43.555 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:43.552+0000 7f356490f640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f35600aabf0 0x7f35601545c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:43.555 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:43.552+0000 7f356490f640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3560157be0 0x7f35600973d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:43.555 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:43.552+0000 7f356490f640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35600ace50 con 0x7f3560157be0 2023-10-26T01:33:43.555 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:43.552+0000 7f356490f640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35600ad060 con 0x7f356006ce50 2023-10-26T01:33:43.555 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:43.552+0000 7f356490f640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35600ad340 con 0x7f35600aabf0 2023-10-26T01:33:43.556 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:43.553+0000 7f355ed9d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3560157be0 0x7f35600973d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:43.556 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:43.553+0000 7f355ed9d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3560157be0 0x7f35600973d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:54134/0 (socket says 172.21.15.169:54134) 2023-10-26T01:33:43.556 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:43.553+0000 7f355ed9d640 1 -- 172.21.15.169:0/3817147389 learned_addr learned my addr 172.21.15.169:0/3817147389 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:33:43.558 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:43.553+0000 7f355e59c640 1 --2- 172.21.15.169:0/3817147389 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f356006ce50 0x7f3560154080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:43.558 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:43.553+0000 7f3557fff640 1 --2- 172.21.15.169:0/3817147389 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f35600aabf0 0x7f35601545c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:43.558 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:43.553+0000 7f355ed9d640 1 -- 172.21.15.169:0/3817147389 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f35600aabf0 msgr2=0x7f35601545c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:43.558 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:43.553+0000 7f355ed9d640 1 --2- 172.21.15.169:0/3817147389 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f35600aabf0 0x7f35601545c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:43.559 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:43.553+0000 7f355ed9d640 1 -- 172.21.15.169:0/3817147389 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f356006ce50 msgr2=0x7f3560154080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:43.559 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:43.553+0000 7f355ed9d640 1 --2- 172.21.15.169:0/3817147389 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f356006ce50 0x7f3560154080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:43.559 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:43.553+0000 7f355ed9d640 1 -- 172.21.15.169:0/3817147389 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3548009560 con 0x7f3560157be0 2023-10-26T01:33:43.559 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:43.553+0000 7f3557fff640 1 --2- 172.21.15.169:0/3817147389 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f35600aabf0 0x7f35601545c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:33:43.559 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:43.553+0000 7f355ed9d640 1 --2- 172.21.15.169:0/3817147389 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3560157be0 0x7f35600973d0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f35580016f0 tx=0x7f355800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:43.560 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:43.554+0000 7f3556ffd640 1 -- 172.21.15.169:0/3817147389 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3558003da0 con 0x7f3560157be0 2023-10-26T01:33:43.560 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:43.554+0000 7f3556ffd640 1 -- 172.21.15.169:0/3817147389 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3558005070 con 0x7f3560157be0 2023-10-26T01:33:43.560 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:43.554+0000 7f356490f640 1 -- 172.21.15.169:0/3817147389 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35600ad620 con 0x7f3560157be0 2023-10-26T01:33:43.560 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:43.554+0000 7f3556ffd640 1 -- 172.21.15.169:0/3817147389 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3558005500 con 0x7f3560157be0 2023-10-26T01:33:43.561 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:43.554+0000 7f356490f640 1 -- 172.21.15.169:0/3817147389 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35600ade20 con 0x7f3560157be0 2023-10-26T01:33:43.561 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:43.555+0000 7f356490f640 1 -- 172.21.15.169:0/3817147389 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35600adb40 con 0x7f3560157be0 2023-10-26T01:33:43.561 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:43.556+0000 7f3556ffd640 1 -- 172.21.15.169:0/3817147389 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f3558005660 con 0x7f3560157be0 2023-10-26T01:33:43.562 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:43.557+0000 7f3556ffd640 1 --2- 172.21.15.169:0/3817147389 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f354406ae40 0x7f354406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:43.562 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:43.557+0000 7f3556ffd640 1 -- 172.21.15.169:0/3817147389 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3558088f10 con 0x7f3560157be0 2023-10-26T01:33:43.562 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:43.557+0000 7f355e59c640 1 --2- 172.21.15.169:0/3817147389 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f354406ae40 0x7f354406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:43.562 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:43.558+0000 7f355e59c640 1 --2- 172.21.15.169:0/3817147389 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f354406ae40 0x7f354406d300 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f3550005d70 tx=0x7f3550005d00 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:43.563 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:43.561+0000 7f3556ffd640 1 -- 172.21.15.169:0/3817147389 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f35600adb40 con 0x7f3560157be0 2023-10-26T01:33:43.796 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:43.794+0000 7f356490f640 1 -- 172.21.15.169:0/3817147389 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f3560159110 con 0x7f3560157be0 2023-10-26T01:33:44.329 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:44.327+0000 7f3556ffd640 1 -- 172.21.15.169:0/3817147389 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v32) v1 ==== 122+0+0 (secure 0 0 0) 0x7f3560159110 con 0x7f3560157be0 2023-10-26T01:33:44.332 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:44.330+0000 7f356490f640 1 -- 172.21.15.169:0/3817147389 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f354406ae40 msgr2=0x7f354406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:44.333 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:44.330+0000 7f356490f640 1 --2- 172.21.15.169:0/3817147389 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f354406ae40 0x7f354406d300 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f3550005d70 tx=0x7f3550005d00 comp rx=0 tx=0).stop 2023-10-26T01:33:44.333 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:44.331+0000 7f356490f640 1 -- 172.21.15.169:0/3817147389 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3560157be0 msgr2=0x7f35600973d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:44.333 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:44.331+0000 7f356490f640 1 --2- 172.21.15.169:0/3817147389 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3560157be0 0x7f35600973d0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f35580016f0 tx=0x7f355800cc70 comp rx=0 tx=0).stop 2023-10-26T01:33:44.333 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:44.331+0000 7f356490f640 1 -- 172.21.15.169:0/3817147389 shutdown_connections 2023-10-26T01:33:44.334 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:44.331+0000 7f356490f640 1 --2- 172.21.15.169:0/3817147389 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f354406ae40 0x7f354406d300 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:44.334 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:44.331+0000 7f356490f640 1 --2- 172.21.15.169:0/3817147389 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3560157be0 0x7f35600973d0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:44.334 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:44.331+0000 7f356490f640 1 --2- 172.21.15.169:0/3817147389 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f35600aabf0 0x7f35601545c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:44.334 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:44.331+0000 7f356490f640 1 --2- 172.21.15.169:0/3817147389 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f356006ce50 0x7f3560154080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:44.334 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:44.331+0000 7f356490f640 1 -- 172.21.15.169:0/3817147389 >> 172.21.15.169:0/3817147389 conn(0x7f3560069930 msgr2=0x7f35601592b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:44.335 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:44.331+0000 7f356490f640 1 -- 172.21.15.169:0/3817147389 shutdown_connections 2023-10-26T01:33:44.335 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:44.331+0000 7f356490f640 1 -- 172.21.15.169:0/3817147389 wait complete. 2023-10-26T01:33:44.353 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-26T01:33:44.354 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T01:33:44.577 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:44.575+0000 7f14c5f1d640 1 -- 172.21.15.169:0/1723918216 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14ac004e20 msgr2=0x7f14ac00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:44.577 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:44.575+0000 7f14c5f1d640 1 --2- 172.21.15.169:0/1723918216 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14ac004e20 0x7f14ac00b980 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f14b8006fd0 tx=0x7f14ac0057f0 comp rx=0 tx=0).stop 2023-10-26T01:33:44.578 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:44.575+0000 7f14bffff640 1 -- 172.21.15.169:0/1723918216 reap_dead start 2023-10-26T01:33:44.578 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:44.575+0000 7f14c5f1d640 1 -- 172.21.15.169:0/1723918216 shutdown_connections 2023-10-26T01:33:44.578 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:44.575+0000 7f14c5f1d640 1 -- 172.21.15.169:0/1723918216 >> 172.21.15.169:0/1723918216 conn(0x7f14c0098a60 msgr2=0x7f14c00b4970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:44.578 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:44.575+0000 7f14c5f1d640 1 -- 172.21.15.169:0/1723918216 shutdown_connections 2023-10-26T01:33:44.578 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:44.575+0000 7f14c5f1d640 1 -- 172.21.15.169:0/1723918216 wait complete. 2023-10-26T01:33:44.579 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:44.577+0000 7f14c5f1d640 1 Processor -- start 2023-10-26T01:33:44.580 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:44.577+0000 7f14c5f1d640 1 -- start start 2023-10-26T01:33:44.580 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:44.578+0000 7f14c5f1d640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f14ac004e20 0x7f14c01e20f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:44.580 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:44.578+0000 7f14c5f1d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f14c01e2630 0x7f14c0154e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:44.580 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:44.578+0000 7f14c5f1d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14c01553a0 0x7f14c01e3f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:44.581 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:44.578+0000 7f14c5f1d640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14c0155830 con 0x7f14c01553a0 2023-10-26T01:33:44.581 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:44.578+0000 7f14c5f1d640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14c012c320 con 0x7f14c01e2630 2023-10-26T01:33:44.581 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:44.578+0000 7f14c5f1d640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14c01e4dc0 con 0x7f14ac004e20 2023-10-26T01:33:44.581 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:44.578+0000 7f14bffff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14c01553a0 0x7f14c01e3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:44.581 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:44.578+0000 7f14bffff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14c01553a0 0x7f14c01e3f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:59312/0 (socket says 172.21.15.169:59312) 2023-10-26T01:33:44.582 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:44.578+0000 7f14bffff640 1 -- 172.21.15.169:0/3135524507 learned_addr learned my addr 172.21.15.169:0/3135524507 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:33:44.583 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:44.578+0000 7f14bffff640 1 -- 172.21.15.169:0/3135524507 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f14ac004e20 msgr2=0x7f14c01e20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:44.583 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:44.578+0000 7f14bffff640 1 --2- 172.21.15.169:0/3135524507 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f14ac004e20 0x7f14c01e20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:44.583 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:44.578+0000 7f14bffff640 1 -- 172.21.15.169:0/3135524507 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f14c01e2630 msgr2=0x7f14c0154e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:44.583 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:44.578+0000 7f14bffff640 1 --2- 172.21.15.169:0/3135524507 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f14c01e2630 0x7f14c0154e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:44.584 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:44.578+0000 7f14bffff640 1 -- 172.21.15.169:0/3135524507 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14c012b9f0 con 0x7f14c01553a0 2023-10-26T01:33:44.584 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:44.579+0000 7f14bffff640 1 --2- 172.21.15.169:0/3135524507 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14c01553a0 0x7f14c01e3f90 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f14b802c9d0 tx=0x7f14b8000f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:44.584 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:44.579+0000 7f14b7fff640 1 -- 172.21.15.169:0/3135524507 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14b8001570 con 0x7f14c01553a0 2023-10-26T01:33:44.584 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:44.579+0000 7f14b7fff640 1 -- 172.21.15.169:0/3135524507 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f14b8006070 con 0x7f14c01553a0 2023-10-26T01:33:44.584 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:44.579+0000 7f14b7fff640 1 -- 172.21.15.169:0/3135524507 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14b8006440 con 0x7f14c01553a0 2023-10-26T01:33:44.585 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:44.579+0000 7f14c5f1d640 1 -- 172.21.15.169:0/3135524507 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14c012bcd0 con 0x7f14c01553a0 2023-10-26T01:33:44.585 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:44.579+0000 7f14c5f1d640 1 -- 172.21.15.169:0/3135524507 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14c01559c0 con 0x7f14c01553a0 2023-10-26T01:33:44.585 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:44.580+0000 7f14c5f1d640 1 -- 172.21.15.169:0/3135524507 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14c00b0f20 con 0x7f14c01553a0 2023-10-26T01:33:44.585 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:44.581+0000 7f14b7fff640 1 -- 172.21.15.169:0/3135524507 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f14b8031030 con 0x7f14c01553a0 2023-10-26T01:33:44.586 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:44.583+0000 7f14b7fff640 1 --2- 172.21.15.169:0/3135524507 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f14a006ae70 0x7f14a006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:44.586 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:44.583+0000 7f14b7fff640 1 -- 172.21.15.169:0/3135524507 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f14b80a9fd0 con 0x7f14c01553a0 2023-10-26T01:33:44.586 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:44.583+0000 7f14bf7fe640 1 --2- 172.21.15.169:0/3135524507 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f14a006ae70 0x7f14a006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:44.587 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:44.584+0000 7f14bf7fe640 1 --2- 172.21.15.169:0/3135524507 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f14a006ae70 0x7f14a006d330 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f14b0001d30 tx=0x7f14b0006ca0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:44.588 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:44.587+0000 7f14b7fff640 1 -- 172.21.15.169:0/3135524507 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f14b8075070 con 0x7f14c01553a0 2023-10-26T01:33:44.814 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:44.813+0000 7f14c5f1d640 1 -- 172.21.15.169:0/3135524507 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f14c0155b80 con 0x7f14c01553a0 2023-10-26T01:33:44.815 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:44.814+0000 7f14b7fff640 1 -- 172.21.15.169:0/3135524507 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4922 (secure 0 0 0) 0x7f14b8042020 con 0x7f14c01553a0 2023-10-26T01:33:44.816 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:33:44.816 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4976,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/2977083370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2977083370},{"type":"v1","addr":"172.21.15.170:6837","nonce":2977083370}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":5176,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/1598214042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1598214042},{"type":"v1","addr":"172.21.15.170:6835","nonce":1598214042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":6087,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/406582181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":406582181},{"type":"v1","addr":"172.21.15.169:6833","nonce":406582181}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T01:33:40.717881+0000","modified":"2023-10-26T01:33:43.796080+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6093},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6093":{"gid":6093,"name":"a","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6835/2338833513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":2338833513},{"type":"v1","addr":"172.21.15.169:6835","nonce":2338833513}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-10-26T01:33:44.817 INFO:teuthology.orchestra.run.smithi169.stderr:dumped fsmap epoch 32 2023-10-26T01:33:44.818 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:44.816+0000 7f14c5f1d640 1 -- 172.21.15.169:0/3135524507 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f14a006ae70 msgr2=0x7f14a006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:44.819 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:44.816+0000 7f14c5f1d640 1 --2- 172.21.15.169:0/3135524507 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f14a006ae70 0x7f14a006d330 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f14b0001d30 tx=0x7f14b0006ca0 comp rx=0 tx=0).stop 2023-10-26T01:33:44.819 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:44.816+0000 7f14c5f1d640 1 -- 172.21.15.169:0/3135524507 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14c01553a0 msgr2=0x7f14c01e3f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:44.819 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:44.816+0000 7f14c5f1d640 1 --2- 172.21.15.169:0/3135524507 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14c01553a0 0x7f14c01e3f90 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f14b802c9d0 tx=0x7f14b8000f60 comp rx=0 tx=0).stop 2023-10-26T01:33:44.820 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:44.817+0000 7f14c5f1d640 1 -- 172.21.15.169:0/3135524507 shutdown_connections 2023-10-26T01:33:44.820 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:44.817+0000 7f14c5f1d640 1 --2- 172.21.15.169:0/3135524507 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f14a006ae70 0x7f14a006d330 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:44.820 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:44.817+0000 7f14c5f1d640 1 --2- 172.21.15.169:0/3135524507 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14c01553a0 0x7f14c01e3f90 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:44.820 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:44.817+0000 7f14c5f1d640 1 --2- 172.21.15.169:0/3135524507 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f14c01e2630 0x7f14c0154e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:44.821 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:44.817+0000 7f14c5f1d640 1 --2- 172.21.15.169:0/3135524507 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f14ac004e20 0x7f14c01e20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:44.821 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:44.817+0000 7f14c5f1d640 1 -- 172.21.15.169:0/3135524507 >> 172.21.15.169:0/3135524507 conn(0x7f14c0098a60 msgr2=0x7f14c015a4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:44.821 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:44.817+0000 7f14c5f1d640 1 -- 172.21.15.169:0/3135524507 shutdown_connections 2023-10-26T01:33:44.821 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:44.817+0000 7f14c5f1d640 1 -- 172.21.15.169:0/3135524507 wait complete. 2023-10-26T01:33:44.838 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:33:45.063 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:45.061+0000 7fdb53b25640 1 -- 172.21.15.169:0/2476476142 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdb38003970 msgr2=0x7fdb38003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:45.064 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:45.061+0000 7fdb53b25640 1 --2- 172.21.15.169:0/2476476142 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdb38003970 0x7fdb38003d90 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fdb4800baf0 tx=0x7fdb380057f0 comp rx=0 tx=0).stop 2023-10-26T01:33:45.064 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:45.061+0000 7fdb53b25640 1 -- 172.21.15.169:0/2476476142 shutdown_connections 2023-10-26T01:33:45.064 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:45.061+0000 7fdb53b25640 1 --2- 172.21.15.169:0/2476476142 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdb38003970 0x7fdb38003d90 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:45.064 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:45.061+0000 7fdb53b25640 1 -- 172.21.15.169:0/2476476142 >> 172.21.15.169:0/2476476142 conn(0x7fdb4c069930 msgr2=0x7fdb4c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:45.064 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:45.061+0000 7fdb53b25640 1 -- 172.21.15.169:0/2476476142 shutdown_connections 2023-10-26T01:33:45.065 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:45.061+0000 7fdb53b25640 1 -- 172.21.15.169:0/2476476142 wait complete. 2023-10-26T01:33:45.065 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:45.063+0000 7fdb53b25640 1 Processor -- start 2023-10-26T01:33:45.065 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:45.063+0000 7fdb53b25640 1 -- start start 2023-10-26T01:33:45.067 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:45.064+0000 7fdb53b25640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdb38003970 0x7fdb4c06ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:45.067 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:45.064+0000 7fdb53b25640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdb4c0c8a20 0x7fdb4c1f4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:45.067 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:45.064+0000 7fdb53b25640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdb4c1f5520 0x7fdb4c1f0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:45.068 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:45.064+0000 7fdb53b25640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb4c1f59a0 con 0x7fdb4c1f5520 2023-10-26T01:33:45.068 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:45.064+0000 7fdb53b25640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb4c071d20 con 0x7fdb38003970 2023-10-26T01:33:45.068 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:45.064+0000 7fdb53b25640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb4c071070 con 0x7fdb4c0c8a20 2023-10-26T01:33:45.068 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:45.065+0000 7fdb520c2640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdb4c1f5520 0x7fdb4c1f0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:45.069 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:45.065+0000 7fdb520c2640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdb4c1f5520 0x7fdb4c1f0ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:59338/0 (socket says 172.21.15.169:59338) 2023-10-26T01:33:45.069 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:45.065+0000 7fdb520c2640 1 -- 172.21.15.169:0/1921966961 learned_addr learned my addr 172.21.15.169:0/1921966961 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:33:45.069 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:45.065+0000 7fdb518c1640 1 --2- 172.21.15.169:0/1921966961 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdb38003970 0x7fdb4c06ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:45.069 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:45.065+0000 7fdb510c0640 1 --2- 172.21.15.169:0/1921966961 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdb4c0c8a20 0x7fdb4c1f4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:45.070 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:45.065+0000 7fdb520c2640 1 -- 172.21.15.169:0/1921966961 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdb4c0c8a20 msgr2=0x7fdb4c1f4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:45.070 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:45.065+0000 7fdb520c2640 1 --2- 172.21.15.169:0/1921966961 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdb4c0c8a20 0x7fdb4c1f4fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:45.070 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:45.065+0000 7fdb520c2640 1 -- 172.21.15.169:0/1921966961 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdb38003970 msgr2=0x7fdb4c06ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:45.070 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:45.065+0000 7fdb520c2640 1 --2- 172.21.15.169:0/1921966961 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdb38003970 0x7fdb4c06ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:45.071 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:45.065+0000 7fdb520c2640 1 -- 172.21.15.169:0/1921966961 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb4c0712b0 con 0x7fdb4c1f5520 2023-10-26T01:33:45.071 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:45.066+0000 7fdb520c2640 1 --2- 172.21.15.169:0/1921966961 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdb4c1f5520 0x7fdb4c1f0ac0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c0ca120 tx=0x7fdb4400da80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:45.072 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:45.066+0000 7fdb427fc640 1 -- 172.21.15.169:0/1921966961 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb44015030 con 0x7fdb4c1f5520 2023-10-26T01:33:45.073 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:45.066+0000 7fdb427fc640 1 -- 172.21.15.169:0/1921966961 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fdb44003e70 con 0x7fdb4c1f5520 2023-10-26T01:33:45.073 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:45.066+0000 7fdb427fc640 1 -- 172.21.15.169:0/1921966961 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb440062b0 con 0x7fdb4c1f5520 2023-10-26T01:33:45.073 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:45.066+0000 7fdb53b25640 1 -- 172.21.15.169:0/1921966961 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb4c06f270 con 0x7fdb4c1f5520 2023-10-26T01:33:45.073 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:45.066+0000 7fdb53b25640 1 -- 172.21.15.169:0/1921966961 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb4c071520 con 0x7fdb4c1f5520 2023-10-26T01:33:45.074 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:45.067+0000 7fdb53b25640 1 -- 172.21.15.169:0/1921966961 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb4c04db10 con 0x7fdb4c1f5520 2023-10-26T01:33:45.074 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:45.068+0000 7fdb427fc640 1 -- 172.21.15.169:0/1921966961 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fdb44018070 con 0x7fdb4c1f5520 2023-10-26T01:33:45.074 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:45.069+0000 7fdb427fc640 1 --2- 172.21.15.169:0/1921966961 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fdb3006ae40 0x7fdb3006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:45.074 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:45.069+0000 7fdb427fc640 1 -- 172.21.15.169:0/1921966961 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdb44016070 con 0x7fdb4c1f5520 2023-10-26T01:33:45.074 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:45.070+0000 7fdb518c1640 1 --2- 172.21.15.169:0/1921966961 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fdb3006ae40 0x7fdb3006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:45.075 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:45.071+0000 7fdb518c1640 1 --2- 172.21.15.169:0/1921966961 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fdb3006ae40 0x7fdb3006d300 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fdb48037000 tx=0x7fdb4802e040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:45.075 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:45.073+0000 7fdb427fc640 1 -- 172.21.15.169:0/1921966961 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdb44058110 con 0x7fdb4c1f5520 2023-10-26T01:33:45.265 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:45.264+0000 7fdb53b25640 1 -- 172.21.15.169:0/1921966961 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdb4c069f70 con 0x7fdb4c1f5520 2023-10-26T01:33:45.268 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:45.267+0000 7fdb427fc640 1 -- 172.21.15.169:0/1921966961 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+16613 (secure 0 0 0) 0x7fdb440014a0 con 0x7fdb4c1f5520 2023-10-26T01:33:45.269 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:33:45.269 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":65,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:33:42.269670+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-10-26T01:33:38.593237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-10-26T01:33:39.753692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:33:45.271 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:45.269+0000 7fdb53b25640 1 -- 172.21.15.169:0/1921966961 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fdb3006ae40 msgr2=0x7fdb3006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:45.271 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:45.269+0000 7fdb53b25640 1 --2- 172.21.15.169:0/1921966961 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fdb3006ae40 0x7fdb3006d300 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fdb48037000 tx=0x7fdb4802e040 comp rx=0 tx=0).stop 2023-10-26T01:33:45.272 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:45.269+0000 7fdb53b25640 1 -- 172.21.15.169:0/1921966961 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdb4c1f5520 msgr2=0x7fdb4c1f0ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:45.272 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:45.269+0000 7fdb53b25640 1 --2- 172.21.15.169:0/1921966961 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdb4c1f5520 0x7fdb4c1f0ac0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c0ca120 tx=0x7fdb4400da80 comp rx=0 tx=0).stop 2023-10-26T01:33:45.272 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:45.269+0000 7fdb53b25640 1 -- 172.21.15.169:0/1921966961 shutdown_connections 2023-10-26T01:33:45.272 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:45.270+0000 7fdb53b25640 1 --2- 172.21.15.169:0/1921966961 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fdb3006ae40 0x7fdb3006d300 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:45.272 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:45.270+0000 7fdb53b25640 1 --2- 172.21.15.169:0/1921966961 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdb4c1f5520 0x7fdb4c1f0ac0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:45.273 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:45.270+0000 7fdb53b25640 1 --2- 172.21.15.169:0/1921966961 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdb4c0c8a20 0x7fdb4c1f4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:45.273 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:45.270+0000 7fdb53b25640 1 --2- 172.21.15.169:0/1921966961 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdb38003970 0x7fdb4c06ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:45.273 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:45.270+0000 7fdb53b25640 1 -- 172.21.15.169:0/1921966961 >> 172.21.15.169:0/1921966961 conn(0x7fdb4c069930 msgr2=0x7fdb4c0ad540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:45.273 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:45.270+0000 7fdb53b25640 1 -- 172.21.15.169:0/1921966961 shutdown_connections 2023-10-26T01:33:45.273 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:45.270+0000 7fdb53b25640 1 -- 172.21.15.169:0/1921966961 wait complete. 2023-10-26T01:33:45.291 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-26T01:33:45.291 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-26T01:33:45.509 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:45.507+0000 7f746487c640 1 -- 172.21.15.169:0/3822766227 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f744c003970 msgr2=0x7f744c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:45.510 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:45.507+0000 7f746487c640 1 --2- 172.21.15.169:0/3822766227 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f744c003970 0x7f744c003d90 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f745000bb20 tx=0x7f744c0057f0 comp rx=0 tx=0).stop 2023-10-26T01:33:45.510 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:45.507+0000 7f745ed9d640 1 -- 172.21.15.169:0/3822766227 reap_dead start 2023-10-26T01:33:45.510 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:45.508+0000 7f746487c640 1 -- 172.21.15.169:0/3822766227 shutdown_connections 2023-10-26T01:33:45.510 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:45.508+0000 7f746487c640 1 -- 172.21.15.169:0/3822766227 >> 172.21.15.169:0/3822766227 conn(0x7f7460069930 msgr2=0x7f7460069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:45.511 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:45.508+0000 7f746487c640 1 -- 172.21.15.169:0/3822766227 shutdown_connections 2023-10-26T01:33:45.511 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:45.508+0000 7f746487c640 1 -- 172.21.15.169:0/3822766227 wait complete. 2023-10-26T01:33:45.511 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:45.509+0000 7f746487c640 1 Processor -- start 2023-10-26T01:33:45.512 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:45.510+0000 7f746487c640 1 -- start start 2023-10-26T01:33:45.513 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:45.510+0000 7f746487c640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f744c003970 0x7f746013c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:45.513 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:45.511+0000 7f746487c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f746013cb40 0x7f746015b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:45.513 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:45.511+0000 7f746487c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f746015bdc0 0x7f74601e3de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:45.513 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:45.511+0000 7f746487c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f746013cfc0 con 0x7f746013cb40 2023-10-26T01:33:45.514 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:45.511+0000 7f746487c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f746015a400 con 0x7f746015bdc0 2023-10-26T01:33:45.514 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:45.511+0000 7f746487c640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f746014a220 con 0x7f744c003970 2023-10-26T01:33:45.514 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:45.511+0000 7f7457fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f746013cb40 0x7f746015b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:45.514 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:45.511+0000 7f7457fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f746013cb40 0x7f746015b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:59356/0 (socket says 172.21.15.169:59356) 2023-10-26T01:33:45.514 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:45.511+0000 7f7457fff640 1 -- 172.21.15.169:0/2924465914 learned_addr learned my addr 172.21.15.169:0/2924465914 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:33:45.515 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:45.511+0000 7f745ed9d640 1 --2- 172.21.15.169:0/2924465914 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f746015bdc0 0x7f74601e3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:45.515 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:45.511+0000 7f745e59c640 1 --2- 172.21.15.169:0/2924465914 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f744c003970 0x7f746013c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:45.515 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:45.511+0000 7f7457fff640 1 -- 172.21.15.169:0/2924465914 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f744c003970 msgr2=0x7f746013c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:45.515 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:45.511+0000 7f7457fff640 1 --2- 172.21.15.169:0/2924465914 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f744c003970 0x7f746013c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:45.515 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:45.511+0000 7f7457fff640 1 -- 172.21.15.169:0/2924465914 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f746015bdc0 msgr2=0x7f74601e3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:45.516 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:45.511+0000 7f7457fff640 1 --2- 172.21.15.169:0/2924465914 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f746015bdc0 0x7f74601e3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:45.516 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:45.511+0000 7f7457fff640 1 -- 172.21.15.169:0/2924465914 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f746014a500 con 0x7f746013cb40 2023-10-26T01:33:45.516 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:45.511+0000 7f745ed9d640 1 --2- 172.21.15.169:0/2924465914 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f746015bdc0 0x7f74601e3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:33:45.517 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:45.512+0000 7f7457fff640 1 --2- 172.21.15.169:0/2924465914 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f746013cb40 0x7f746015b880 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f744800c690 tx=0x7f744800cb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:45.518 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:45.512+0000 7f7456ffd640 1 -- 172.21.15.169:0/2924465914 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7448005270 con 0x7f746013cb40 2023-10-26T01:33:45.518 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:45.512+0000 7f7456ffd640 1 -- 172.21.15.169:0/2924465914 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f74480053d0 con 0x7f746013cb40 2023-10-26T01:33:45.518 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:45.512+0000 7f7456ffd640 1 -- 172.21.15.169:0/2924465914 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f744800a8a0 con 0x7f746013cb40 2023-10-26T01:33:45.518 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:45.512+0000 7f746487c640 1 -- 172.21.15.169:0/2924465914 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f746014a7e0 con 0x7f746013cb40 2023-10-26T01:33:45.518 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:45.512+0000 7f746487c640 1 -- 172.21.15.169:0/2924465914 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f746014ada0 con 0x7f746013cb40 2023-10-26T01:33:45.519 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:45.513+0000 7f746487c640 1 -- 172.21.15.169:0/2924465914 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74601370f0 con 0x7f746013cb40 2023-10-26T01:33:45.519 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:45.514+0000 7f7456ffd640 1 -- 172.21.15.169:0/2924465914 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f744800aa00 con 0x7f746013cb40 2023-10-26T01:33:45.519 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:45.515+0000 7f7456ffd640 1 --2- 172.21.15.169:0/2924465914 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f744006ae70 0x7f744006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:45.519 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:45.515+0000 7f7456ffd640 1 -- 172.21.15.169:0/2924465914 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f74480895d0 con 0x7f746013cb40 2023-10-26T01:33:45.519 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:45.516+0000 7f745e59c640 1 --2- 172.21.15.169:0/2924465914 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f744006ae70 0x7f744006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:45.520 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:45.517+0000 7f745e59c640 1 --2- 172.21.15.169:0/2924465914 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f744006ae70 0x7f744006d330 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f7450037000 tx=0x7f745002e040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:45.521 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:45.519+0000 7f7456ffd640 1 -- 172.21.15.169:0/2924465914 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f74601370f0 con 0x7f746013cb40 2023-10-26T01:33:45.706 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:45.704+0000 7f746487c640 1 -- 172.21.15.169:0/2924465914 --> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f74600aa620 con 0x7f744006ae70 2023-10-26T01:33:45.733 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:45.731+0000 7f7456ffd640 1 -- 172.21.15.169:0/2924465914 <== mgr.4112 v2:172.21.15.170:6832/1187156918 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+278498 (secure 0 0 0) 0x7f7450041440 con 0x7f744006ae70 2023-10-26T01:33:45.739 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:33:45.741 INFO:teuthology.orchestra.run.smithi169.stderr:dumped all 2023-10-26T01:33:45.745 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:45.737+0000 7f746487c640 1 -- 172.21.15.169:0/2924465914 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f744006ae70 msgr2=0x7f744006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:45.746 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:45.737+0000 7f746487c640 1 --2- 172.21.15.169:0/2924465914 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f744006ae70 0x7f744006d330 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f7450037000 tx=0x7f745002e040 comp rx=0 tx=0).stop 2023-10-26T01:33:45.746 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:45.737+0000 7f746487c640 1 -- 172.21.15.169:0/2924465914 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f746013cb40 msgr2=0x7f746015b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:45.746 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:45.737+0000 7f746487c640 1 --2- 172.21.15.169:0/2924465914 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f746013cb40 0x7f746015b880 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f744800c690 tx=0x7f744800cb50 comp rx=0 tx=0).stop 2023-10-26T01:33:45.747 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:45.738+0000 7f746487c640 1 -- 172.21.15.169:0/2924465914 shutdown_connections 2023-10-26T01:33:45.747 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:45.738+0000 7f746487c640 1 --2- 172.21.15.169:0/2924465914 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f744006ae70 0x7f744006d330 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:45.748 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:45.738+0000 7f746487c640 1 --2- 172.21.15.169:0/2924465914 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f746013cb40 0x7f746015b880 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:45.748 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:45.738+0000 7f746487c640 1 --2- 172.21.15.169:0/2924465914 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f746015bdc0 0x7f74601e3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:45.748 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:45.738+0000 7f746487c640 1 --2- 172.21.15.169:0/2924465914 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f744c003970 0x7f746013c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:45.749 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:45.738+0000 7f746487c640 1 -- 172.21.15.169:0/2924465914 >> 172.21.15.169:0/2924465914 conn(0x7f7460069930 msgr2=0x7f7460069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:45.749 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:45.738+0000 7f746487c640 1 -- 172.21.15.169:0/2924465914 shutdown_connections 2023-10-26T01:33:45.749 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:45.738+0000 7f746487c640 1 -- 172.21.15.169:0/2924465914 wait complete. 2023-10-26T01:33:45.760 INFO:teuthology.orchestra.run.smithi169.stdout:{"pg_ready":true,"pg_map":{"version":155,"stamp":"2023-10-26T01:33:44.192644+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590424,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":164,"num_write_kb":2424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":123,"ondisk_log_size":123,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":55,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":3638,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1558861,"statfs":{"total":1600000000,"available":1596274437,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":56,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.955338"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-10-26T01:33:40.267104+0000","last_change":"2023-10-26T01:33:40.267104+0000","last_active":"2023-10-26T01:33:40.235547+0000","last_peered":"2023-10-26T01:33:40.235547+0000","last_clean":"2023-10-26T01:33:40.235547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:33:40.267104+0000","last_undegraded":"2023-10-26T01:33:40.267104+0000","last_fullsized":"2023-10-26T01:33:40.267104+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-26T01:33:41.301683+0000","last_change":"2023-10-26T01:33:41.300524+0000","last_active":"2023-10-26T01:33:41.301683+0000","last_peered":"2023-10-26T01:33:41.301683+0000","last_clean":"2023-10-26T01:33:41.301683+0000","last_became_active":"2023-10-26T01:33:41.295476+0000","last_became_peered":"2023-10-26T01:33:41.295476+0000","last_unstale":"2023-10-26T01:33:41.301683+0000","last_undegraded":"2023-10-26T01:33:41.301683+0000","last_fullsized":"2023-10-26T01:33:41.301683+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:35:24.220074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-10-26T01:33:40.260952+0000","last_change":"2023-10-26T01:33:40.260952+0000","last_active":"2023-10-26T01:33:40.235547+0000","last_peered":"2023-10-26T01:33:40.235547+0000","last_clean":"2023-10-26T01:33:40.235547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:33:40.260952+0000","last_undegraded":"2023-10-26T01:33:40.260952+0000","last_fullsized":"2023-10-26T01:33:40.260952+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-10-26T01:33:40.260762+0000","last_change":"2023-10-26T01:33:40.260762+0000","last_active":"2023-10-26T01:33:40.235547+0000","last_peered":"2023-10-26T01:33:40.235547+0000","last_clean":"2023-10-26T01:33:40.235547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:33:40.260762+0000","last_undegraded":"2023-10-26T01:33:40.260762+0000","last_fullsized":"2023-10-26T01:33:40.260762+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-10-26T01:33:40.260939+0000","last_change":"2023-10-26T01:33:40.260939+0000","last_active":"2023-10-26T01:33:40.235547+0000","last_peered":"2023-10-26T01:33:40.235547+0000","last_clean":"2023-10-26T01:33:40.235547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:33:40.260939+0000","last_undegraded":"2023-10-26T01:33:40.260939+0000","last_fullsized":"2023-10-26T01:33:40.260939+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-10-26T01:33:40.258524+0000","last_change":"2023-10-26T01:33:40.258524+0000","last_active":"2023-10-26T01:33:40.235547+0000","last_peered":"2023-10-26T01:33:40.235547+0000","last_clean":"2023-10-26T01:33:40.235547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:33:40.258524+0000","last_undegraded":"2023-10-26T01:33:40.258524+0000","last_fullsized":"2023-10-26T01:33:40.258524+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-10-26T01:33:40.253141+0000","last_change":"2023-10-26T01:33:40.253141+0000","last_active":"2023-10-26T01:33:40.235547+0000","last_peered":"2023-10-26T01:33:40.235547+0000","last_clean":"2023-10-26T01:33:40.235547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:33:40.253141+0000","last_undegraded":"2023-10-26T01:33:40.253141+0000","last_fullsized":"2023-10-26T01:33:40.253141+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-26T01:33:41.299557+0000","last_change":"2023-10-26T01:33:41.299209+0000","last_active":"2023-10-26T01:33:41.299557+0000","last_peered":"2023-10-26T01:33:41.299557+0000","last_clean":"2023-10-26T01:33:41.299557+0000","last_became_active":"2023-10-26T01:33:41.298665+0000","last_became_peered":"2023-10-26T01:33:41.298665+0000","last_unstale":"2023-10-26T01:33:41.299557+0000","last_undegraded":"2023-10-26T01:33:41.299557+0000","last_fullsized":"2023-10-26T01:33:41.299557+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:09:19.054540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-10-26T01:33:40.261308+0000","last_change":"2023-10-26T01:33:40.261308+0000","last_active":"2023-10-26T01:33:40.235547+0000","last_peered":"2023-10-26T01:33:40.235547+0000","last_clean":"2023-10-26T01:33:40.235547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:33:40.261308+0000","last_undegraded":"2023-10-26T01:33:40.261308+0000","last_fullsized":"2023-10-26T01:33:40.261308+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-10-26T01:33:40.260818+0000","last_change":"2023-10-26T01:33:40.260818+0000","last_active":"2023-10-26T01:33:40.235547+0000","last_peered":"2023-10-26T01:33:40.235547+0000","last_clean":"2023-10-26T01:33:40.235547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:33:40.260818+0000","last_undegraded":"2023-10-26T01:33:40.260818+0000","last_fullsized":"2023-10-26T01:33:40.260818+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-10-26T01:33:40.260003+0000","last_change":"2023-10-26T01:33:40.260003+0000","last_active":"2023-10-26T01:33:40.235547+0000","last_peered":"2023-10-26T01:33:40.235547+0000","last_clean":"2023-10-26T01:33:40.235547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:33:40.260003+0000","last_undegraded":"2023-10-26T01:33:40.260003+0000","last_fullsized":"2023-10-26T01:33:40.260003+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-10-26T01:33:40.261563+0000","last_change":"2023-10-26T01:33:40.261563+0000","last_active":"2023-10-26T01:33:40.235547+0000","last_peered":"2023-10-26T01:33:40.235547+0000","last_clean":"2023-10-26T01:33:40.235547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:33:40.261563+0000","last_undegraded":"2023-10-26T01:33:40.261563+0000","last_fullsized":"2023-10-26T01:33:40.261563+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-10-26T01:33:40.260805+0000","last_change":"2023-10-26T01:33:40.260805+0000","last_active":"2023-10-26T01:33:40.235547+0000","last_peered":"2023-10-26T01:33:40.235547+0000","last_clean":"2023-10-26T01:33:40.235547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:33:40.260805+0000","last_undegraded":"2023-10-26T01:33:40.260805+0000","last_fullsized":"2023-10-26T01:33:40.260805+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-10-26T01:33:40.260879+0000","last_change":"2023-10-26T01:33:40.260879+0000","last_active":"2023-10-26T01:33:40.235547+0000","last_peered":"2023-10-26T01:33:40.235547+0000","last_clean":"2023-10-26T01:33:40.235547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:33:40.260879+0000","last_undegraded":"2023-10-26T01:33:40.260879+0000","last_fullsized":"2023-10-26T01:33:40.260879+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-10-26T01:33:40.252014+0000","last_change":"2023-10-26T01:33:40.252014+0000","last_active":"2023-10-26T01:33:40.235547+0000","last_peered":"2023-10-26T01:33:40.235547+0000","last_clean":"2023-10-26T01:33:40.235547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:33:40.252014+0000","last_undegraded":"2023-10-26T01:33:40.252014+0000","last_fullsized":"2023-10-26T01:33:40.252014+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-10-26T01:33:40.258110+0000","last_change":"2023-10-26T01:33:40.258110+0000","last_active":"2023-10-26T01:33:40.235547+0000","last_peered":"2023-10-26T01:33:40.235547+0000","last_clean":"2023-10-26T01:33:40.235547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:33:40.258110+0000","last_undegraded":"2023-10-26T01:33:40.258110+0000","last_fullsized":"2023-10-26T01:33:40.258110+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-26T01:33:41.301717+0000","last_change":"2023-10-26T01:33:41.301569+0000","last_active":"2023-10-26T01:33:41.301717+0000","last_peered":"2023-10-26T01:33:41.301717+0000","last_clean":"2023-10-26T01:33:41.301717+0000","last_became_active":"2023-10-26T01:33:41.301183+0000","last_became_peered":"2023-10-26T01:33:41.301183+0000","last_unstale":"2023-10-26T01:33:41.301717+0000","last_undegraded":"2023-10-26T01:33:41.301717+0000","last_fullsized":"2023-10-26T01:33:41.301717+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:23:14.596280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-10-26T01:33:40.260310+0000","last_change":"2023-10-26T01:33:40.260310+0000","last_active":"2023-10-26T01:33:40.235547+0000","last_peered":"2023-10-26T01:33:40.235547+0000","last_clean":"2023-10-26T01:33:40.235547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:33:40.260310+0000","last_undegraded":"2023-10-26T01:33:40.260310+0000","last_fullsized":"2023-10-26T01:33:40.260310+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-10-26T01:33:40.265318+0000","last_change":"2023-10-26T01:33:40.265318+0000","last_active":"2023-10-26T01:33:40.235547+0000","last_peered":"2023-10-26T01:33:40.235547+0000","last_clean":"2023-10-26T01:33:40.235547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:33:40.265318+0000","last_undegraded":"2023-10-26T01:33:40.265318+0000","last_fullsized":"2023-10-26T01:33:40.265318+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-26T01:33:41.295374+0000","last_change":"2023-10-26T01:33:41.295080+0000","last_active":"2023-10-26T01:33:41.295374+0000","last_peered":"2023-10-26T01:33:41.295374+0000","last_clean":"2023-10-26T01:33:41.295374+0000","last_became_active":"2023-10-26T01:33:41.294605+0000","last_became_peered":"2023-10-26T01:33:41.294605+0000","last_unstale":"2023-10-26T01:33:41.295374+0000","last_undegraded":"2023-10-26T01:33:41.295374+0000","last_fullsized":"2023-10-26T01:33:41.295374+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:03:47.019520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-10-26T01:33:40.258980+0000","last_change":"2023-10-26T01:33:40.258980+0000","last_active":"2023-10-26T01:33:40.235547+0000","last_peered":"2023-10-26T01:33:40.235547+0000","last_clean":"2023-10-26T01:33:40.235547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:33:40.258980+0000","last_undegraded":"2023-10-26T01:33:40.258980+0000","last_fullsized":"2023-10-26T01:33:40.258980+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-26T01:33:41.310784+0000","last_change":"2023-10-26T01:33:41.309460+0000","last_active":"2023-10-26T01:33:41.310784+0000","last_peered":"2023-10-26T01:33:41.310784+0000","last_clean":"2023-10-26T01:33:41.310784+0000","last_became_active":"2023-10-26T01:33:41.309191+0000","last_became_peered":"2023-10-26T01:33:41.309191+0000","last_unstale":"2023-10-26T01:33:41.310784+0000","last_undegraded":"2023-10-26T01:33:41.310784+0000","last_fullsized":"2023-10-26T01:33:41.310784+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:22:38.364628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-10-26T01:33:40.259518+0000","last_change":"2023-10-26T01:33:40.259518+0000","last_active":"2023-10-26T01:33:40.235547+0000","last_peered":"2023-10-26T01:33:40.235547+0000","last_clean":"2023-10-26T01:33:40.235547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:33:40.259518+0000","last_undegraded":"2023-10-26T01:33:40.259518+0000","last_fullsized":"2023-10-26T01:33:40.259518+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-10-26T01:33:40.256989+0000","last_change":"2023-10-26T01:33:40.256989+0000","last_active":"2023-10-26T01:33:40.235547+0000","last_peered":"2023-10-26T01:33:40.235547+0000","last_clean":"2023-10-26T01:33:40.235547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:33:40.256989+0000","last_undegraded":"2023-10-26T01:33:40.256989+0000","last_fullsized":"2023-10-26T01:33:40.256989+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-10-26T01:33:40.260440+0000","last_change":"2023-10-26T01:33:40.260440+0000","last_active":"2023-10-26T01:33:40.235547+0000","last_peered":"2023-10-26T01:33:40.235547+0000","last_clean":"2023-10-26T01:33:40.235547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:33:40.260440+0000","last_undegraded":"2023-10-26T01:33:40.260440+0000","last_fullsized":"2023-10-26T01:33:40.260440+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-10-26T01:33:40.259217+0000","last_change":"2023-10-26T01:33:40.259217+0000","last_active":"2023-10-26T01:33:40.235547+0000","last_peered":"2023-10-26T01:33:40.235547+0000","last_clean":"2023-10-26T01:33:40.235547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:33:40.259217+0000","last_undegraded":"2023-10-26T01:33:40.259217+0000","last_fullsized":"2023-10-26T01:33:40.259217+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-10-26T01:33:40.262101+0000","last_change":"2023-10-26T01:33:40.262101+0000","last_active":"2023-10-26T01:33:40.235547+0000","last_peered":"2023-10-26T01:33:40.235547+0000","last_clean":"2023-10-26T01:33:40.235547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:33:40.262101+0000","last_undegraded":"2023-10-26T01:33:40.262101+0000","last_fullsized":"2023-10-26T01:33:40.262101+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-10-26T01:33:40.262480+0000","last_change":"2023-10-26T01:33:40.262480+0000","last_active":"2023-10-26T01:33:40.235547+0000","last_peered":"2023-10-26T01:33:40.235547+0000","last_clean":"2023-10-26T01:33:40.235547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:33:40.262480+0000","last_undegraded":"2023-10-26T01:33:40.262480+0000","last_fullsized":"2023-10-26T01:33:40.262480+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-26T01:33:41.302333+0000","last_change":"2023-10-26T01:33:41.301837+0000","last_active":"2023-10-26T01:33:41.302333+0000","last_peered":"2023-10-26T01:33:41.302333+0000","last_clean":"2023-10-26T01:33:41.302333+0000","last_became_active":"2023-10-26T01:33:41.301551+0000","last_became_peered":"2023-10-26T01:33:41.301551+0000","last_unstale":"2023-10-26T01:33:41.302333+0000","last_undegraded":"2023-10-26T01:33:41.302333+0000","last_fullsized":"2023-10-26T01:33:41.302333+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:06:05.830226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-10-26T01:33:40.253465+0000","last_change":"2023-10-26T01:33:40.253465+0000","last_active":"2023-10-26T01:33:40.235547+0000","last_peered":"2023-10-26T01:33:40.235547+0000","last_clean":"2023-10-26T01:33:40.235547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:33:40.253465+0000","last_undegraded":"2023-10-26T01:33:40.253465+0000","last_fullsized":"2023-10-26T01:33:40.253465+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-10-26T01:33:40.265449+0000","last_change":"2023-10-26T01:33:40.265449+0000","last_active":"2023-10-26T01:33:40.235547+0000","last_peered":"2023-10-26T01:33:40.235547+0000","last_clean":"2023-10-26T01:33:40.235547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:33:40.265449+0000","last_undegraded":"2023-10-26T01:33:40.265449+0000","last_fullsized":"2023-10-26T01:33:40.265449+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-10-26T01:33:40.261788+0000","last_change":"2023-10-26T01:33:40.261788+0000","last_active":"2023-10-26T01:33:40.235547+0000","last_peered":"2023-10-26T01:33:40.235547+0000","last_clean":"2023-10-26T01:33:40.235547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:33:40.261788+0000","last_undegraded":"2023-10-26T01:33:40.261788+0000","last_fullsized":"2023-10-26T01:33:40.261788+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-10-26T01:33:40.256294+0000","last_change":"2023-10-26T01:33:40.256294+0000","last_active":"2023-10-26T01:33:40.235547+0000","last_peered":"2023-10-26T01:33:40.235547+0000","last_clean":"2023-10-26T01:33:40.235547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:33:40.256294+0000","last_undegraded":"2023-10-26T01:33:40.256294+0000","last_fullsized":"2023-10-26T01:33:40.256294+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-26T01:33:40.271804+0000","last_change":"2023-10-26T01:33:40.271579+0000","last_active":"2023-10-26T01:33:40.271804+0000","last_peered":"2023-10-26T01:33:40.271804+0000","last_clean":"2023-10-26T01:33:40.271804+0000","last_became_active":"2023-10-26T01:33:40.271194+0000","last_became_peered":"2023-10-26T01:33:40.271194+0000","last_unstale":"2023-10-26T01:33:40.271804+0000","last_undegraded":"2023-10-26T01:33:40.271804+0000","last_fullsized":"2023-10-26T01:33:40.271804+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_clean_scrub_stamp":"2023-10-26T01:33:39.224951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:35:32.205712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-10-26T01:33:40.260319+0000","last_change":"2023-10-26T01:33:40.260319+0000","last_active":"2023-10-26T01:33:40.235547+0000","last_peered":"2023-10-26T01:33:40.235547+0000","last_clean":"2023-10-26T01:33:40.235547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:33:40.260319+0000","last_undegraded":"2023-10-26T01:33:40.260319+0000","last_fullsized":"2023-10-26T01:33:40.260319+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"64'1","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-26T01:33:41.317285+0000","last_change":"2023-10-26T01:33:40.266886+0000","last_active":"2023-10-26T01:33:41.317285+0000","last_peered":"2023-10-26T01:33:41.317285+0000","last_clean":"2023-10-26T01:33:41.317285+0000","last_became_active":"2023-10-26T01:33:40.266621+0000","last_became_peered":"2023-10-26T01:33:40.266621+0000","last_unstale":"2023-10-26T01:33:41.317285+0000","last_undegraded":"2023-10-26T01:33:41.317285+0000","last_fullsized":"2023-10-26T01:33:41.317285+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_clean_scrub_stamp":"2023-10-26T01:33:39.224951+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:22:58.516924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-10-26T01:33:40.255529+0000","last_change":"2023-10-26T01:33:40.255529+0000","last_active":"2023-10-26T01:33:40.235547+0000","last_peered":"2023-10-26T01:33:40.235547+0000","last_clean":"2023-10-26T01:33:40.235547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:33:40.255529+0000","last_undegraded":"2023-10-26T01:33:40.255529+0000","last_fullsized":"2023-10-26T01:33:40.255529+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-26T01:33:40.269176+0000","last_change":"2023-10-26T01:33:40.266751+0000","last_active":"2023-10-26T01:33:40.269176+0000","last_peered":"2023-10-26T01:33:40.269176+0000","last_clean":"2023-10-26T01:33:40.269176+0000","last_became_active":"2023-10-26T01:33:40.266391+0000","last_became_peered":"2023-10-26T01:33:40.266391+0000","last_unstale":"2023-10-26T01:33:40.269176+0000","last_undegraded":"2023-10-26T01:33:40.269176+0000","last_fullsized":"2023-10-26T01:33:40.269176+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_clean_scrub_stamp":"2023-10-26T01:33:39.224951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:43:28.673846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-10-26T01:33:40.263884+0000","last_change":"2023-10-26T01:33:40.263884+0000","last_active":"2023-10-26T01:33:40.235547+0000","last_peered":"2023-10-26T01:33:40.235547+0000","last_clean":"2023-10-26T01:33:40.235547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:33:40.263884+0000","last_undegraded":"2023-10-26T01:33:40.263884+0000","last_fullsized":"2023-10-26T01:33:40.263884+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-26T01:33:40.268879+0000","last_change":"2023-10-26T01:33:40.267788+0000","last_active":"2023-10-26T01:33:40.268879+0000","last_peered":"2023-10-26T01:33:40.268879+0000","last_clean":"2023-10-26T01:33:40.268879+0000","last_became_active":"2023-10-26T01:33:40.266846+0000","last_became_peered":"2023-10-26T01:33:40.266846+0000","last_unstale":"2023-10-26T01:33:40.268879+0000","last_undegraded":"2023-10-26T01:33:40.268879+0000","last_fullsized":"2023-10-26T01:33:40.268879+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_clean_scrub_stamp":"2023-10-26T01:33:39.224951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:25:12.702847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-26T01:33:41.311604+0000","last_change":"2023-10-26T01:33:41.311443+0000","last_active":"2023-10-26T01:33:41.311604+0000","last_peered":"2023-10-26T01:33:41.311604+0000","last_clean":"2023-10-26T01:33:41.311604+0000","last_became_active":"2023-10-26T01:33:41.311197+0000","last_became_peered":"2023-10-26T01:33:41.311197+0000","last_unstale":"2023-10-26T01:33:41.311604+0000","last_undegraded":"2023-10-26T01:33:41.311604+0000","last_fullsized":"2023-10-26T01:33:41.311604+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:56:20.883631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-26T01:33:40.269614+0000","last_change":"2023-10-26T01:33:40.269465+0000","last_active":"2023-10-26T01:33:40.269614+0000","last_peered":"2023-10-26T01:33:40.269614+0000","last_clean":"2023-10-26T01:33:40.269614+0000","last_became_active":"2023-10-26T01:33:40.266610+0000","last_became_peered":"2023-10-26T01:33:40.266610+0000","last_unstale":"2023-10-26T01:33:40.269614+0000","last_undegraded":"2023-10-26T01:33:40.269614+0000","last_fullsized":"2023-10-26T01:33:40.269614+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_clean_scrub_stamp":"2023-10-26T01:33:39.224951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:58:12.392348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-26T01:33:41.301702+0000","last_change":"2023-10-26T01:33:41.301423+0000","last_active":"2023-10-26T01:33:41.301702+0000","last_peered":"2023-10-26T01:33:41.301702+0000","last_clean":"2023-10-26T01:33:41.301702+0000","last_became_active":"2023-10-26T01:33:41.300804+0000","last_became_peered":"2023-10-26T01:33:41.300804+0000","last_unstale":"2023-10-26T01:33:41.301702+0000","last_undegraded":"2023-10-26T01:33:41.301702+0000","last_fullsized":"2023-10-26T01:33:41.301702+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:06:53.887209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-26T01:33:40.269407+0000","last_change":"2023-10-26T01:33:40.268904+0000","last_active":"2023-10-26T01:33:40.269407+0000","last_peered":"2023-10-26T01:33:40.269407+0000","last_clean":"2023-10-26T01:33:40.269407+0000","last_became_active":"2023-10-26T01:33:40.268191+0000","last_became_peered":"2023-10-26T01:33:40.268191+0000","last_unstale":"2023-10-26T01:33:40.269407+0000","last_undegraded":"2023-10-26T01:33:40.269407+0000","last_fullsized":"2023-10-26T01:33:40.269407+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_clean_scrub_stamp":"2023-10-26T01:33:39.224951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:05:50.003308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-10-26T01:33:40.256870+0000","last_change":"2023-10-26T01:33:40.256870+0000","last_active":"2023-10-26T01:33:40.235547+0000","last_peered":"2023-10-26T01:33:40.235547+0000","last_clean":"2023-10-26T01:33:40.235547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:33:40.256870+0000","last_undegraded":"2023-10-26T01:33:40.256870+0000","last_fullsized":"2023-10-26T01:33:40.256870+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-26T01:33:40.268855+0000","last_change":"2023-10-26T01:33:40.267785+0000","last_active":"2023-10-26T01:33:40.268855+0000","last_peered":"2023-10-26T01:33:40.268855+0000","last_clean":"2023-10-26T01:33:40.268855+0000","last_became_active":"2023-10-26T01:33:40.267136+0000","last_became_peered":"2023-10-26T01:33:40.267136+0000","last_unstale":"2023-10-26T01:33:40.268855+0000","last_undegraded":"2023-10-26T01:33:40.268855+0000","last_fullsized":"2023-10-26T01:33:40.268855+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_clean_scrub_stamp":"2023-10-26T01:33:39.224951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:58:12.392348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-26T01:33:41.295407+0000","last_change":"2023-10-26T01:33:41.295216+0000","last_active":"2023-10-26T01:33:41.295407+0000","last_peered":"2023-10-26T01:33:41.295407+0000","last_clean":"2023-10-26T01:33:41.295407+0000","last_became_active":"2023-10-26T01:33:41.294775+0000","last_became_peered":"2023-10-26T01:33:41.294775+0000","last_unstale":"2023-10-26T01:33:41.295407+0000","last_undegraded":"2023-10-26T01:33:41.295407+0000","last_fullsized":"2023-10-26T01:33:41.295407+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:17:49.911001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-26T01:33:40.269341+0000","last_change":"2023-10-26T01:33:40.267785+0000","last_active":"2023-10-26T01:33:40.269341+0000","last_peered":"2023-10-26T01:33:40.269341+0000","last_clean":"2023-10-26T01:33:40.269341+0000","last_became_active":"2023-10-26T01:33:40.267331+0000","last_became_peered":"2023-10-26T01:33:40.267331+0000","last_unstale":"2023-10-26T01:33:40.269341+0000","last_undegraded":"2023-10-26T01:33:40.269341+0000","last_fullsized":"2023-10-26T01:33:40.269341+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_clean_scrub_stamp":"2023-10-26T01:33:39.224951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:03:06.315831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-10-26T01:33:40.252327+0000","last_change":"2023-10-26T01:33:40.252327+0000","last_active":"2023-10-26T01:33:40.235547+0000","last_peered":"2023-10-26T01:33:40.235547+0000","last_clean":"2023-10-26T01:33:40.235547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:33:40.252327+0000","last_undegraded":"2023-10-26T01:33:40.252327+0000","last_fullsized":"2023-10-26T01:33:40.252327+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-26T01:33:40.270363+0000","last_change":"2023-10-26T01:33:40.270251+0000","last_active":"2023-10-26T01:33:40.270363+0000","last_peered":"2023-10-26T01:33:40.270363+0000","last_clean":"2023-10-26T01:33:40.270363+0000","last_became_active":"2023-10-26T01:33:40.269663+0000","last_became_peered":"2023-10-26T01:33:40.269663+0000","last_unstale":"2023-10-26T01:33:40.270363+0000","last_undegraded":"2023-10-26T01:33:40.270363+0000","last_fullsized":"2023-10-26T01:33:40.270363+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_clean_scrub_stamp":"2023-10-26T01:33:39.224951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:33:02.993299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-10-26T01:33:40.256370+0000","last_change":"2023-10-26T01:33:40.256370+0000","last_active":"2023-10-26T01:33:40.235547+0000","last_peered":"2023-10-26T01:33:40.235547+0000","last_clean":"2023-10-26T01:33:40.235547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:33:40.256370+0000","last_undegraded":"2023-10-26T01:33:40.256370+0000","last_fullsized":"2023-10-26T01:33:40.256370+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-26T01:33:41.294795+0000","last_change":"2023-10-26T01:33:40.272855+0000","last_active":"2023-10-26T01:33:41.294795+0000","last_peered":"2023-10-26T01:33:41.294795+0000","last_clean":"2023-10-26T01:33:41.294795+0000","last_became_active":"2023-10-26T01:33:40.271480+0000","last_became_peered":"2023-10-26T01:33:40.271480+0000","last_unstale":"2023-10-26T01:33:41.294795+0000","last_undegraded":"2023-10-26T01:33:41.294795+0000","last_fullsized":"2023-10-26T01:33:41.294795+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_clean_scrub_stamp":"2023-10-26T01:33:39.224951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:12:25.980965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-10-26T01:33:40.261198+0000","last_change":"2023-10-26T01:33:40.261198+0000","last_active":"2023-10-26T01:33:40.235547+0000","last_peered":"2023-10-26T01:33:40.235547+0000","last_clean":"2023-10-26T01:33:40.235547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:33:40.261198+0000","last_undegraded":"2023-10-26T01:33:40.261198+0000","last_fullsized":"2023-10-26T01:33:40.261198+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-26T01:33:40.265057+0000","last_change":"2023-10-26T01:33:40.264901+0000","last_active":"2023-10-26T01:33:40.265057+0000","last_peered":"2023-10-26T01:33:40.265057+0000","last_clean":"2023-10-26T01:33:40.265057+0000","last_became_active":"2023-10-26T01:33:40.263006+0000","last_became_peered":"2023-10-26T01:33:40.263006+0000","last_unstale":"2023-10-26T01:33:40.265057+0000","last_undegraded":"2023-10-26T01:33:40.265057+0000","last_fullsized":"2023-10-26T01:33:40.265057+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_clean_scrub_stamp":"2023-10-26T01:33:39.224951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:10:30.826612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-10-26T01:33:40.252794+0000","last_change":"2023-10-26T01:33:40.252794+0000","last_active":"2023-10-26T01:33:40.235547+0000","last_peered":"2023-10-26T01:33:40.235547+0000","last_clean":"2023-10-26T01:33:40.235547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:33:40.252794+0000","last_undegraded":"2023-10-26T01:33:40.252794+0000","last_fullsized":"2023-10-26T01:33:40.252794+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-26T01:33:40.262217+0000","last_change":"2023-10-26T01:33:40.262090+0000","last_active":"2023-10-26T01:33:40.262217+0000","last_peered":"2023-10-26T01:33:40.262217+0000","last_clean":"2023-10-26T01:33:40.262217+0000","last_became_active":"2023-10-26T01:33:40.261736+0000","last_became_peered":"2023-10-26T01:33:40.261736+0000","last_unstale":"2023-10-26T01:33:40.262217+0000","last_undegraded":"2023-10-26T01:33:40.262217+0000","last_fullsized":"2023-10-26T01:33:40.262217+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_clean_scrub_stamp":"2023-10-26T01:33:39.224951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:16:22.976409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-10-26T01:33:40.257239+0000","last_change":"2023-10-26T01:33:40.257239+0000","last_active":"2023-10-26T01:33:40.235547+0000","last_peered":"2023-10-26T01:33:40.235547+0000","last_clean":"2023-10-26T01:33:40.235547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:33:40.257239+0000","last_undegraded":"2023-10-26T01:33:40.257239+0000","last_fullsized":"2023-10-26T01:33:40.257239+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"64'2","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-26T01:33:41.318445+0000","last_change":"2023-10-26T01:33:40.269670+0000","last_active":"2023-10-26T01:33:41.318445+0000","last_peered":"2023-10-26T01:33:41.318445+0000","last_clean":"2023-10-26T01:33:41.318445+0000","last_became_active":"2023-10-26T01:33:40.267307+0000","last_became_peered":"2023-10-26T01:33:40.267307+0000","last_unstale":"2023-10-26T01:33:41.318445+0000","last_undegraded":"2023-10-26T01:33:41.318445+0000","last_fullsized":"2023-10-26T01:33:41.318445+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_clean_scrub_stamp":"2023-10-26T01:33:39.224951+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:37:16.574662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-10-26T01:33:40.252601+0000","last_change":"2023-10-26T01:33:40.252601+0000","last_active":"2023-10-26T01:33:40.235547+0000","last_peered":"2023-10-26T01:33:40.235547+0000","last_clean":"2023-10-26T01:33:40.235547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:33:40.252601+0000","last_undegraded":"2023-10-26T01:33:40.252601+0000","last_fullsized":"2023-10-26T01:33:40.252601+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-26T01:33:40.265704+0000","last_change":"2023-10-26T01:33:40.265574+0000","last_active":"2023-10-26T01:33:40.265704+0000","last_peered":"2023-10-26T01:33:40.265704+0000","last_clean":"2023-10-26T01:33:40.265704+0000","last_became_active":"2023-10-26T01:33:40.265261+0000","last_became_peered":"2023-10-26T01:33:40.265261+0000","last_unstale":"2023-10-26T01:33:40.265704+0000","last_undegraded":"2023-10-26T01:33:40.265704+0000","last_fullsized":"2023-10-26T01:33:40.265704+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_clean_scrub_stamp":"2023-10-26T01:33:39.224951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:55:32.466720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-10-26T01:33:40.248748+0000","last_change":"2023-10-26T01:33:40.248748+0000","last_active":"2023-10-26T01:33:40.235547+0000","last_peered":"2023-10-26T01:33:40.235547+0000","last_clean":"2023-10-26T01:33:40.235547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:33:40.248748+0000","last_undegraded":"2023-10-26T01:33:40.248748+0000","last_fullsized":"2023-10-26T01:33:40.248748+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-26T01:33:40.269068+0000","last_change":"2023-10-26T01:33:40.268967+0000","last_active":"2023-10-26T01:33:40.269068+0000","last_peered":"2023-10-26T01:33:40.269068+0000","last_clean":"2023-10-26T01:33:40.269068+0000","last_became_active":"2023-10-26T01:33:40.268346+0000","last_became_peered":"2023-10-26T01:33:40.268346+0000","last_unstale":"2023-10-26T01:33:40.269068+0000","last_undegraded":"2023-10-26T01:33:40.269068+0000","last_fullsized":"2023-10-26T01:33:40.269068+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_clean_scrub_stamp":"2023-10-26T01:33:39.224951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:16:22.976409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-10-26T01:33:40.257430+0000","last_change":"2023-10-26T01:33:40.257430+0000","last_active":"2023-10-26T01:33:40.235547+0000","last_peered":"2023-10-26T01:33:40.235547+0000","last_clean":"2023-10-26T01:33:40.235547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:33:40.257430+0000","last_undegraded":"2023-10-26T01:33:40.257430+0000","last_fullsized":"2023-10-26T01:33:40.257430+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-26T01:33:40.270743+0000","last_change":"2023-10-26T01:33:40.270644+0000","last_active":"2023-10-26T01:33:40.270743+0000","last_peered":"2023-10-26T01:33:40.270743+0000","last_clean":"2023-10-26T01:33:40.270743+0000","last_became_active":"2023-10-26T01:33:40.270292+0000","last_became_peered":"2023-10-26T01:33:40.270292+0000","last_unstale":"2023-10-26T01:33:40.270743+0000","last_undegraded":"2023-10-26T01:33:40.270743+0000","last_fullsized":"2023-10-26T01:33:40.270743+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_clean_scrub_stamp":"2023-10-26T01:33:39.224951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:59:57.719590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-10-26T01:33:40.251169+0000","last_change":"2023-10-26T01:33:40.251169+0000","last_active":"2023-10-26T01:33:40.235547+0000","last_peered":"2023-10-26T01:33:40.235547+0000","last_clean":"2023-10-26T01:33:40.235547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:33:40.251169+0000","last_undegraded":"2023-10-26T01:33:40.251169+0000","last_fullsized":"2023-10-26T01:33:40.251169+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"64'1","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-26T01:33:41.318330+0000","last_change":"2023-10-26T01:33:40.274622+0000","last_active":"2023-10-26T01:33:41.318330+0000","last_peered":"2023-10-26T01:33:41.318330+0000","last_clean":"2023-10-26T01:33:41.318330+0000","last_became_active":"2023-10-26T01:33:40.271512+0000","last_became_peered":"2023-10-26T01:33:40.271512+0000","last_unstale":"2023-10-26T01:33:41.318330+0000","last_undegraded":"2023-10-26T01:33:41.318330+0000","last_fullsized":"2023-10-26T01:33:41.318330+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_clean_scrub_stamp":"2023-10-26T01:33:39.224951+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:31:50.211414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-10-26T01:33:40.260100+0000","last_change":"2023-10-26T01:33:40.260100+0000","last_active":"2023-10-26T01:33:40.235547+0000","last_peered":"2023-10-26T01:33:40.235547+0000","last_clean":"2023-10-26T01:33:40.235547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:33:40.260100+0000","last_undegraded":"2023-10-26T01:33:40.260100+0000","last_fullsized":"2023-10-26T01:33:40.260100+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-26T01:33:40.271559+0000","last_change":"2023-10-26T01:33:40.271394+0000","last_active":"2023-10-26T01:33:40.271559+0000","last_peered":"2023-10-26T01:33:40.271559+0000","last_clean":"2023-10-26T01:33:40.271559+0000","last_became_active":"2023-10-26T01:33:40.271150+0000","last_became_peered":"2023-10-26T01:33:40.271150+0000","last_unstale":"2023-10-26T01:33:40.271559+0000","last_undegraded":"2023-10-26T01:33:40.271559+0000","last_fullsized":"2023-10-26T01:33:40.271559+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_clean_scrub_stamp":"2023-10-26T01:33:39.224951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:01:42.255111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-26T01:33:41.308095+0000","last_change":"2023-10-26T01:33:41.305337+0000","last_active":"2023-10-26T01:33:41.308095+0000","last_peered":"2023-10-26T01:33:41.308095+0000","last_clean":"2023-10-26T01:33:41.308095+0000","last_became_active":"2023-10-26T01:33:41.305082+0000","last_became_peered":"2023-10-26T01:33:41.305082+0000","last_unstale":"2023-10-26T01:33:41.308095+0000","last_undegraded":"2023-10-26T01:33:41.308095+0000","last_fullsized":"2023-10-26T01:33:41.308095+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:15:58.724118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-26T01:33:40.268901+0000","last_change":"2023-10-26T01:33:40.267963+0000","last_active":"2023-10-26T01:33:40.268901+0000","last_peered":"2023-10-26T01:33:40.268901+0000","last_clean":"2023-10-26T01:33:40.268901+0000","last_became_active":"2023-10-26T01:33:40.267288+0000","last_became_peered":"2023-10-26T01:33:40.267288+0000","last_unstale":"2023-10-26T01:33:40.268901+0000","last_undegraded":"2023-10-26T01:33:40.268901+0000","last_fullsized":"2023-10-26T01:33:40.268901+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_clean_scrub_stamp":"2023-10-26T01:33:39.224951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:02:57.614799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-10-26T01:33:40.255649+0000","last_change":"2023-10-26T01:33:40.255649+0000","last_active":"2023-10-26T01:33:40.235547+0000","last_peered":"2023-10-26T01:33:40.235547+0000","last_clean":"2023-10-26T01:33:40.235547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:33:40.255649+0000","last_undegraded":"2023-10-26T01:33:40.255649+0000","last_fullsized":"2023-10-26T01:33:40.255649+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-26T01:33:40.269911+0000","last_change":"2023-10-26T01:33:40.269822+0000","last_active":"2023-10-26T01:33:40.269911+0000","last_peered":"2023-10-26T01:33:40.269911+0000","last_clean":"2023-10-26T01:33:40.269911+0000","last_became_active":"2023-10-26T01:33:40.267425+0000","last_became_peered":"2023-10-26T01:33:40.267425+0000","last_unstale":"2023-10-26T01:33:40.269911+0000","last_undegraded":"2023-10-26T01:33:40.269911+0000","last_fullsized":"2023-10-26T01:33:40.269911+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_clean_scrub_stamp":"2023-10-26T01:33:39.224951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:17:31.515720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-10-26T01:33:40.256697+0000","last_change":"2023-10-26T01:33:40.256697+0000","last_active":"2023-10-26T01:33:40.235547+0000","last_peered":"2023-10-26T01:33:40.235547+0000","last_clean":"2023-10-26T01:33:40.235547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:33:40.256697+0000","last_undegraded":"2023-10-26T01:33:40.256697+0000","last_fullsized":"2023-10-26T01:33:40.256697+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"64'2","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-26T01:33:41.317710+0000","last_change":"2023-10-26T01:33:40.262402+0000","last_active":"2023-10-26T01:33:41.317710+0000","last_peered":"2023-10-26T01:33:41.317710+0000","last_clean":"2023-10-26T01:33:41.317710+0000","last_became_active":"2023-10-26T01:33:40.262031+0000","last_became_peered":"2023-10-26T01:33:40.262031+0000","last_unstale":"2023-10-26T01:33:41.317710+0000","last_undegraded":"2023-10-26T01:33:41.317710+0000","last_fullsized":"2023-10-26T01:33:41.317710+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_clean_scrub_stamp":"2023-10-26T01:33:39.224951+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:58:10.641484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-10-26T01:33:40.264963+0000","last_change":"2023-10-26T01:33:40.264963+0000","last_active":"2023-10-26T01:33:40.235547+0000","last_peered":"2023-10-26T01:33:40.235547+0000","last_clean":"2023-10-26T01:33:40.235547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:33:40.264963+0000","last_undegraded":"2023-10-26T01:33:40.264963+0000","last_fullsized":"2023-10-26T01:33:40.264963+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-26T01:33:40.270496+0000","last_change":"2023-10-26T01:33:40.270401+0000","last_active":"2023-10-26T01:33:40.270496+0000","last_peered":"2023-10-26T01:33:40.270496+0000","last_clean":"2023-10-26T01:33:40.270496+0000","last_became_active":"2023-10-26T01:33:40.270199+0000","last_became_peered":"2023-10-26T01:33:40.270199+0000","last_unstale":"2023-10-26T01:33:40.270496+0000","last_undegraded":"2023-10-26T01:33:40.270496+0000","last_fullsized":"2023-10-26T01:33:40.270496+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_clean_scrub_stamp":"2023-10-26T01:33:39.224951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:48:54.677253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-10-26T01:33:40.263514+0000","last_change":"2023-10-26T01:33:40.263514+0000","last_active":"2023-10-26T01:33:40.235547+0000","last_peered":"2023-10-26T01:33:40.235547+0000","last_clean":"2023-10-26T01:33:40.235547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:33:40.263514+0000","last_undegraded":"2023-10-26T01:33:40.263514+0000","last_fullsized":"2023-10-26T01:33:40.263514+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-26T01:33:40.265300+0000","last_change":"2023-10-26T01:33:40.263474+0000","last_active":"2023-10-26T01:33:40.265300+0000","last_peered":"2023-10-26T01:33:40.265300+0000","last_clean":"2023-10-26T01:33:40.265300+0000","last_became_active":"2023-10-26T01:33:40.263130+0000","last_became_peered":"2023-10-26T01:33:40.263130+0000","last_unstale":"2023-10-26T01:33:40.265300+0000","last_undegraded":"2023-10-26T01:33:40.265300+0000","last_fullsized":"2023-10-26T01:33:40.265300+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_clean_scrub_stamp":"2023-10-26T01:33:39.224951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:47:19.697952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-10-26T01:33:40.258947+0000","last_change":"2023-10-26T01:33:40.258947+0000","last_active":"2023-10-26T01:33:40.235547+0000","last_peered":"2023-10-26T01:33:40.235547+0000","last_clean":"2023-10-26T01:33:40.235547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:33:40.258947+0000","last_undegraded":"2023-10-26T01:33:40.258947+0000","last_fullsized":"2023-10-26T01:33:40.258947+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-26T01:33:40.269333+0000","last_change":"2023-10-26T01:33:40.268525+0000","last_active":"2023-10-26T01:33:40.269333+0000","last_peered":"2023-10-26T01:33:40.269333+0000","last_clean":"2023-10-26T01:33:40.269333+0000","last_became_active":"2023-10-26T01:33:40.267115+0000","last_became_peered":"2023-10-26T01:33:40.267115+0000","last_unstale":"2023-10-26T01:33:40.269333+0000","last_undegraded":"2023-10-26T01:33:40.269333+0000","last_fullsized":"2023-10-26T01:33:40.269333+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_clean_scrub_stamp":"2023-10-26T01:33:39.224951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:16:02.965440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-10-26T01:33:40.260947+0000","last_change":"2023-10-26T01:33:40.260947+0000","last_active":"2023-10-26T01:33:40.235547+0000","last_peered":"2023-10-26T01:33:40.235547+0000","last_clean":"2023-10-26T01:33:40.235547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:33:40.260947+0000","last_undegraded":"2023-10-26T01:33:40.260947+0000","last_fullsized":"2023-10-26T01:33:40.260947+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-26T01:33:40.269425+0000","last_change":"2023-10-26T01:33:40.269043+0000","last_active":"2023-10-26T01:33:40.269425+0000","last_peered":"2023-10-26T01:33:40.269425+0000","last_clean":"2023-10-26T01:33:40.269425+0000","last_became_active":"2023-10-26T01:33:40.268478+0000","last_became_peered":"2023-10-26T01:33:40.268478+0000","last_unstale":"2023-10-26T01:33:40.269425+0000","last_undegraded":"2023-10-26T01:33:40.269425+0000","last_fullsized":"2023-10-26T01:33:40.269425+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_clean_scrub_stamp":"2023-10-26T01:33:39.224951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:04:14.094026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-10-26T01:33:40.257636+0000","last_change":"2023-10-26T01:33:40.257636+0000","last_active":"2023-10-26T01:33:40.235547+0000","last_peered":"2023-10-26T01:33:40.235547+0000","last_clean":"2023-10-26T01:33:40.235547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:33:40.257636+0000","last_undegraded":"2023-10-26T01:33:40.257636+0000","last_fullsized":"2023-10-26T01:33:40.257636+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-26T01:33:40.264621+0000","last_change":"2023-10-26T01:33:40.264477+0000","last_active":"2023-10-26T01:33:40.264621+0000","last_peered":"2023-10-26T01:33:40.264621+0000","last_clean":"2023-10-26T01:33:40.264621+0000","last_became_active":"2023-10-26T01:33:40.264087+0000","last_became_peered":"2023-10-26T01:33:40.264087+0000","last_unstale":"2023-10-26T01:33:40.264621+0000","last_undegraded":"2023-10-26T01:33:40.264621+0000","last_fullsized":"2023-10-26T01:33:40.264621+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_clean_scrub_stamp":"2023-10-26T01:33:39.224951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:20:21.803277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-10-26T01:33:40.255835+0000","last_change":"2023-10-26T01:33:40.255835+0000","last_active":"2023-10-26T01:33:40.235547+0000","last_peered":"2023-10-26T01:33:40.235547+0000","last_clean":"2023-10-26T01:33:40.235547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:33:40.255835+0000","last_undegraded":"2023-10-26T01:33:40.255835+0000","last_fullsized":"2023-10-26T01:33:40.255835+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-26T01:33:40.265270+0000","last_change":"2023-10-26T01:33:40.262786+0000","last_active":"2023-10-26T01:33:40.265270+0000","last_peered":"2023-10-26T01:33:40.265270+0000","last_clean":"2023-10-26T01:33:40.265270+0000","last_became_active":"2023-10-26T01:33:40.262305+0000","last_became_peered":"2023-10-26T01:33:40.262305+0000","last_unstale":"2023-10-26T01:33:40.265270+0000","last_undegraded":"2023-10-26T01:33:40.265270+0000","last_fullsized":"2023-10-26T01:33:40.265270+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_clean_scrub_stamp":"2023-10-26T01:33:39.224951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:03:06.315831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-26T01:33:41.309299+0000","last_change":"2023-10-26T01:33:41.308913+0000","last_active":"2023-10-26T01:33:41.309299+0000","last_peered":"2023-10-26T01:33:41.309299+0000","last_clean":"2023-10-26T01:33:41.309299+0000","last_became_active":"2023-10-26T01:33:41.307785+0000","last_became_peered":"2023-10-26T01:33:41.307785+0000","last_unstale":"2023-10-26T01:33:41.309299+0000","last_undegraded":"2023-10-26T01:33:41.309299+0000","last_fullsized":"2023-10-26T01:33:41.309299+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T01:39:22.276139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-26T01:33:40.268445+0000","last_change":"2023-10-26T01:33:40.266077+0000","last_active":"2023-10-26T01:33:40.268445+0000","last_peered":"2023-10-26T01:33:40.268445+0000","last_clean":"2023-10-26T01:33:40.268445+0000","last_became_active":"2023-10-26T01:33:40.265759+0000","last_became_peered":"2023-10-26T01:33:40.265759+0000","last_unstale":"2023-10-26T01:33:40.268445+0000","last_undegraded":"2023-10-26T01:33:40.268445+0000","last_fullsized":"2023-10-26T01:33:40.268445+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_clean_scrub_stamp":"2023-10-26T01:33:39.224951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:49:19.521293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-10-26T01:33:40.254843+0000","last_change":"2023-10-26T01:33:40.254843+0000","last_active":"2023-10-26T01:33:40.235547+0000","last_peered":"2023-10-26T01:33:40.235547+0000","last_clean":"2023-10-26T01:33:40.235547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:33:40.254843+0000","last_undegraded":"2023-10-26T01:33:40.254843+0000","last_fullsized":"2023-10-26T01:33:40.254843+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-26T01:33:40.266008+0000","last_change":"2023-10-26T01:33:40.264127+0000","last_active":"2023-10-26T01:33:40.266008+0000","last_peered":"2023-10-26T01:33:40.266008+0000","last_clean":"2023-10-26T01:33:40.266008+0000","last_became_active":"2023-10-26T01:33:40.263771+0000","last_became_peered":"2023-10-26T01:33:40.263771+0000","last_unstale":"2023-10-26T01:33:40.266008+0000","last_undegraded":"2023-10-26T01:33:40.266008+0000","last_fullsized":"2023-10-26T01:33:40.266008+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_clean_scrub_stamp":"2023-10-26T01:33:39.224951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:25:12.702847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-10-26T01:33:40.256343+0000","last_change":"2023-10-26T01:33:40.256343+0000","last_active":"2023-10-26T01:33:40.235547+0000","last_peered":"2023-10-26T01:33:40.235547+0000","last_clean":"2023-10-26T01:33:40.235547+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:33:40.256343+0000","last_undegraded":"2023-10-26T01:33:40.256343+0000","last_fullsized":"2023-10-26T01:33:40.256343+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-26T01:33:40.270181+0000","last_change":"2023-10-26T01:33:40.270109+0000","last_active":"2023-10-26T01:33:40.270181+0000","last_peered":"2023-10-26T01:33:40.270181+0000","last_clean":"2023-10-26T01:33:40.270181+0000","last_became_active":"2023-10-26T01:33:40.269898+0000","last_became_peered":"2023-10-26T01:33:40.269898+0000","last_unstale":"2023-10-26T01:33:40.270181+0000","last_undegraded":"2023-10-26T01:33:40.270181+0000","last_fullsized":"2023-10-26T01:33:40.270181+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_clean_scrub_stamp":"2023-10-26T01:33:39.224951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:34:01.826753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-26T01:33:41.299531+0000","last_change":"2023-10-26T01:33:41.299046+0000","last_active":"2023-10-26T01:33:41.299531+0000","last_peered":"2023-10-26T01:33:41.299531+0000","last_clean":"2023-10-26T01:33:41.299531+0000","last_became_active":"2023-10-26T01:33:41.298329+0000","last_became_peered":"2023-10-26T01:33:41.298329+0000","last_unstale":"2023-10-26T01:33:41.299531+0000","last_undegraded":"2023-10-26T01:33:41.299531+0000","last_fullsized":"2023-10-26T01:33:41.299531+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:08:42.823671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-26T01:33:41.285826+0000","last_change":"2023-10-26T01:33:40.274518+0000","last_active":"2023-10-26T01:33:41.285826+0000","last_peered":"2023-10-26T01:33:41.285826+0000","last_clean":"2023-10-26T01:33:41.285826+0000","last_became_active":"2023-10-26T01:33:40.271103+0000","last_became_peered":"2023-10-26T01:33:40.271103+0000","last_unstale":"2023-10-26T01:33:41.285826+0000","last_undegraded":"2023-10-26T01:33:41.285826+0000","last_fullsized":"2023-10-26T01:33:41.285826+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_clean_scrub_stamp":"2023-10-26T01:33:39.224951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:31:52.146845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-26T01:33:41.308053+0000","last_change":"2023-10-26T01:33:41.305817+0000","last_active":"2023-10-26T01:33:41.308053+0000","last_peered":"2023-10-26T01:33:41.308053+0000","last_clean":"2023-10-26T01:33:41.308053+0000","last_became_active":"2023-10-26T01:33:41.305585+0000","last_became_peered":"2023-10-26T01:33:41.305585+0000","last_unstale":"2023-10-26T01:33:41.308053+0000","last_undegraded":"2023-10-26T01:33:41.308053+0000","last_fullsized":"2023-10-26T01:33:41.308053+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:14:31.511712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-26T01:33:40.268818+0000","last_change":"2023-10-26T01:33:40.268720+0000","last_active":"2023-10-26T01:33:40.268818+0000","last_peered":"2023-10-26T01:33:40.268818+0000","last_clean":"2023-10-26T01:33:40.268818+0000","last_became_active":"2023-10-26T01:33:40.268390+0000","last_became_peered":"2023-10-26T01:33:40.268390+0000","last_unstale":"2023-10-26T01:33:40.268818+0000","last_undegraded":"2023-10-26T01:33:40.268818+0000","last_fullsized":"2023-10-26T01:33:40.268818+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_clean_scrub_stamp":"2023-10-26T01:33:39.224951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:03:57.022567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":207,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-26T01:33:39.239234+0000","last_change":"2023-10-26T01:30:34.883484+0000","last_active":"2023-10-26T01:33:39.239234+0000","last_peered":"2023-10-26T01:33:39.239234+0000","last_clean":"2023-10-26T01:33:39.239234+0000","last_became_active":"2023-10-26T01:30:34.883071+0000","last_became_peered":"2023-10-26T01:30:34.883071+0000","last_unstale":"2023-10-26T01:33:39.239234+0000","last_undegraded":"2023-10-26T01:33:39.239234+0000","last_fullsized":"2023-10-26T01:33:39.239234+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:32.890096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:32.890096+0000","last_clean_scrub_stamp":"2023-10-26T01:30:32.890096+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:00:50.687712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":56,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":14,"seq":60129542185,"num_pgs":23,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":318,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194994,"statfs":{"total":200000000,"available":199674276,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:33:40 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.79000000000000004,"15min":0.78400000000000003},"min":{"1min":0.52700000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.1579999999999999,"5min":2.2549999999999999,"15min":2.2549999999999999},"last":1.046},{"interface":"front","average":{"1min":0.748,"5min":0.84699999999999998,"15min":0.88800000000000001},"min":{"1min":0.441,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.2050000000000001,"5min":1.8200000000000001,"15min":1.8200000000000001},"last":0.76600000000000001}]},{"osd":1,"last update":"Thu Oct 26 01:33:40 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.90300000000000002,"15min":0.96599999999999997},"min":{"1min":0.48599999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.2450000000000001,"5min":4.4340000000000002,"15min":4.4340000000000002},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.97599999999999998,"15min":1.004},"min":{"1min":0.56599999999999995,"5min":0.374,"15min":0.374},"max":{"1min":1.4570000000000001,"5min":2.3330000000000002,"15min":2.3330000000000002},"last":0.876}]},{"osd":2,"last update":"Thu Oct 26 01:33:40 2023","interfaces":[{"interface":"back","average":{"1min":1.0669999999999999,"5min":1.0680000000000001,"15min":1.1000000000000001},"min":{"1min":0.64700000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.5509999999999999,"5min":4.2800000000000002,"15min":4.2800000000000002},"last":1.1619999999999999},{"interface":"front","average":{"1min":1.087,"5min":1.071,"15min":1.075},"min":{"1min":0.59599999999999997,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.502,"5min":2.302,"15min":2.302},"last":0.97599999999999998}]},{"osd":4,"last update":"Thu Oct 26 01:33:40 2023","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.145,"15min":1.1479999999999999},"min":{"1min":0.77100000000000002,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.5660000000000001,"5min":2.1070000000000002,"15min":2.1070000000000002},"last":1.121},{"interface":"front","average":{"1min":1.3280000000000001,"5min":1.3720000000000001,"15min":1.413},"min":{"1min":1.0309999999999999,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":1.8160000000000001,"5min":4.8250000000000002,"15min":4.8250000000000002},"last":1.4139999999999999}]},{"osd":5,"last update":"Thu Oct 26 01:33:40 2023","interfaces":[{"interface":"back","average":{"1min":1.248,"5min":1.3100000000000001,"15min":1.3580000000000001},"min":{"1min":0.83199999999999996,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.631,"5min":4.9080000000000004,"15min":4.9080000000000004},"last":1.3500000000000001},{"interface":"front","average":{"1min":1.246,"5min":1.26,"15min":1.274},"min":{"1min":0.80900000000000005,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.8919999999999999,"5min":2.8980000000000001,"15min":2.8980000000000001},"last":1.2090000000000001}]},{"osd":6,"last update":"Thu Oct 26 01:33:40 2023","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.3149999999999999,"15min":1.319},"min":{"1min":0.96499999999999997,"5min":0.85099999999999998,"15min":0.85099999999999998},"max":{"1min":1.823,"5min":2.8530000000000002,"15min":2.8530000000000002},"last":1.3839999999999999},{"interface":"front","average":{"1min":1.177,"5min":1.236,"15min":1.264},"min":{"1min":0.81499999999999995,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.784,"5min":2.181,"15min":2.181},"last":1.1950000000000001}]},{"osd":7,"last update":"Thu Oct 26 01:33:40 2023","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.2689999999999999,"15min":1.28},"min":{"1min":0.85199999999999998,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.891,"5min":2.0680000000000001,"15min":2.0680000000000001},"last":1.3120000000000001},{"interface":"front","average":{"1min":1.381,"5min":1.3899999999999999,"15min":1.4299999999999999},"min":{"1min":0.89500000000000002,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.698,"5min":4.8730000000000002,"15min":4.8730000000000002},"last":1.2829999999999999}]}]},{"osd":7,"up_from":13,"seq":55834574888,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":887,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194424,"statfs":{"total":200000000,"available":199090739,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:33:40 2023","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.0900000000000001,"15min":1.073},"min":{"1min":0.72799999999999998,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.9410000000000001,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":1.3149999999999999},{"interface":"front","average":{"1min":1.123,"5min":1.093,"15min":1.0629999999999999},"min":{"1min":0.73599999999999999,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.6220000000000001,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":1.4419999999999999}]},{"osd":1,"last update":"Thu Oct 26 01:33:40 2023","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":1.103,"15min":1.0569999999999999},"min":{"1min":0.78200000000000003,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":2.0510000000000002,"5min":2.0510000000000002,"15min":2.0510000000000002},"last":0.92300000000000004},{"interface":"front","average":{"1min":1.2270000000000001,"5min":1.2090000000000001,"15min":1.1859999999999999},"min":{"1min":0.751,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.681,"5min":2.1720000000000002,"15min":2.1720000000000002},"last":1.0669999999999999}]},{"osd":2,"last update":"Thu Oct 26 01:33:40 2023","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":1.095,"15min":1.087},"min":{"1min":0.65700000000000003,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.53,"5min":1.6919999999999999,"15min":1.6919999999999999},"last":1.482},{"interface":"front","average":{"1min":1.2709999999999999,"5min":1.228,"15min":1.2130000000000001},"min":{"1min":0.81699999999999995,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":2.181,"5min":2.181,"15min":2.181},"last":1.1359999999999999}]},{"osd":3,"last update":"Thu Oct 26 01:32:42 2023","interfaces":[{"interface":"back","average":{"1min":1.2050000000000001,"5min":1.1819999999999999,"15min":1.1779999999999999},"min":{"1min":0.85999999999999999,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.8100000000000001,"5min":1.8100000000000001,"15min":1.8100000000000001},"last":1.609},{"interface":"front","average":{"1min":1.3520000000000001,"5min":1.355,"15min":1.355},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.9319999999999999,"5min":1.9319999999999999,"15min":1.9319999999999999},"last":1.2509999999999999}]},{"osd":4,"last update":"Thu Oct 26 01:33:40 2023","interfaces":[{"interface":"back","average":{"1min":1.331,"5min":1.337,"15min":1.3420000000000001},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.5880000000000001,"5min":2.008,"15min":2.008},"last":1.387},{"interface":"front","average":{"1min":1.47,"5min":1.409,"15min":1.389},"min":{"1min":0.96399999999999997,"5min":0.496,"15min":0.496},"max":{"1min":2.6619999999999999,"5min":2.6619999999999999,"15min":2.6619999999999999},"last":1.52}]},{"osd":5,"last update":"Thu Oct 26 01:33:40 2023","interfaces":[{"interface":"back","average":{"1min":1.5389999999999999,"5min":1.425,"15min":1.403},"min":{"1min":0.97799999999999998,"5min":0.441,"15min":0.441},"max":{"1min":2.492,"5min":2.492,"15min":2.492},"last":1.361},{"interface":"front","average":{"1min":1.506,"5min":1.46,"15min":1.4590000000000001},"min":{"1min":1.0920000000000001,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.7450000000000001,"5min":1.911,"15min":1.911},"last":1.73}]},{"osd":6,"last update":"Thu Oct 26 01:33:40 2023","interfaces":[{"interface":"back","average":{"1min":1.5940000000000001,"5min":1.833,"15min":1.9530000000000001},"min":{"1min":1.1299999999999999,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":2.0099999999999998,"5min":11.577999999999999,"15min":11.577999999999999},"last":1.5680000000000001},{"interface":"front","average":{"1min":1.641,"5min":1.835,"15min":1.946},"min":{"1min":1.2789999999999999,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":2.4489999999999998,"5min":10.704000000000001,"15min":10.704000000000001},"last":1.6659999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574888,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":310,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195001,"statfs":{"total":200000000,"available":199681721,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:33:37 2023","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":1.1220000000000001,"15min":1.1679999999999999},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.5069999999999999,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":2.395},{"interface":"front","average":{"1min":1.133,"5min":1.081,"15min":1.1040000000000001},"min":{"1min":0.68600000000000005,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.5620000000000001,"5min":1.843,"15min":1.843},"last":0.93600000000000005}]},{"osd":1,"last update":"Thu Oct 26 01:33:37 2023","interfaces":[{"interface":"back","average":{"1min":1.081,"5min":1.1299999999999999,"15min":1.1599999999999999},"min":{"1min":0.71599999999999997,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.6910000000000001,"5min":1.7150000000000001,"15min":1.7150000000000001},"last":2.0529999999999999},{"interface":"front","average":{"1min":1.024,"5min":1.1659999999999999,"15min":1.206},"min":{"1min":0.67700000000000005,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.4870000000000001,"5min":2.085,"15min":2.085},"last":0.82299999999999995}]},{"osd":2,"last update":"Thu Oct 26 01:33:37 2023","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.0680000000000001,"15min":1.0569999999999999},"min":{"1min":0.84099999999999997,"5min":0.747,"15min":0.747},"max":{"1min":1.6419999999999999,"5min":1.6659999999999999,"15min":1.6659999999999999},"last":3.7879999999999998},{"interface":"front","average":{"1min":1.222,"5min":1.2390000000000001,"15min":1.2450000000000001},"min":{"1min":0.84499999999999997,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.8899999999999999,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":2.2890000000000001}]},{"osd":3,"last update":"Thu Oct 26 01:33:37 2023","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.173,"15min":1.1619999999999999},"min":{"1min":0.74099999999999999,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.764,"5min":1.764,"15min":1.764},"last":3.907},{"interface":"front","average":{"1min":1.2989999999999999,"5min":1.3140000000000001,"15min":1.3180000000000001},"min":{"1min":1.0589999999999999,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.8029999999999999,"5min":1.881,"15min":1.881},"last":2.5019999999999998}]},{"osd":4,"last update":"Thu Oct 26 01:33:37 2023","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.216,"15min":1.208},"min":{"1min":0.92900000000000005,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.845,"5min":2.089,"15min":2.089},"last":2.9489999999999998},{"interface":"front","average":{"1min":1.3640000000000001,"5min":1.2829999999999999,"15min":1.264},"min":{"1min":0.92300000000000004,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.704,"5min":2.4129999999999998,"15min":2.4129999999999998},"last":5.5469999999999997}]},{"osd":5,"last update":"Thu Oct 26 01:33:37 2023","interfaces":[{"interface":"back","average":{"1min":1.508,"5min":1.427,"15min":1.403},"min":{"1min":1.1559999999999999,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.843,"5min":2.1640000000000001,"15min":2.1640000000000001},"last":5.4370000000000003},{"interface":"front","average":{"1min":1.4339999999999999,"5min":1.4139999999999999,"15min":1.3979999999999999},"min":{"1min":0.97199999999999998,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.9470000000000001,"5min":2.0710000000000002,"15min":2.0710000000000002},"last":2.621}]},{"osd":7,"last update":"Thu Oct 26 01:33:37 2023","interfaces":[{"interface":"back","average":{"1min":1.4670000000000001,"5min":1.377,"15min":1.3620000000000001},"min":{"1min":1.0009999999999999,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.7629999999999999,"5min":2.012,"15min":2.012},"last":11.804},{"interface":"front","average":{"1min":1.5409999999999999,"5min":1.4670000000000001,"15min":1.4419999999999999},"min":{"1min":1.0760000000000001,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.9199999999999999,"5min":2.3239999999999998,"15min":2.3239999999999998},"last":11.541}]}]},{"osd":5,"up_from":13,"seq":55834574888,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":305,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195007,"statfs":{"total":200000000,"available":199687374,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:32:38 2023","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"min":{"1min":0.66200000000000003,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.573,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":1.387},{"interface":"front","average":{"1min":1.159,"5min":1.1359999999999999,"15min":1.1319999999999999},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.5309999999999999,"5min":1.5309999999999999,"15min":1.5309999999999999},"last":1.3660000000000001}]},{"osd":1,"last update":"Thu Oct 26 01:32:38 2023","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":1.0600000000000001,"15min":1.0629999999999999},"min":{"1min":0.81499999999999995,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.3819999999999999,"5min":1.722,"15min":1.722},"last":1.337},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.087,"15min":1.071},"min":{"1min":0.71099999999999997,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.5820000000000001,"5min":1.7669999999999999,"15min":1.7669999999999999},"last":1.482}]},{"osd":2,"last update":"Thu Oct 26 01:32:38 2023","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":1.155,"15min":1.145},"min":{"1min":0.74399999999999999,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.6339999999999999,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":1.147},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.1539999999999999,"15min":1.1579999999999999},"min":{"1min":0.85699999999999998,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.5269999999999999,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":1.579}]},{"osd":3,"last update":"Thu Oct 26 01:32:38 2023","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.206,"15min":1.212},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.492,"5min":1.835,"15min":1.835},"last":1.4339999999999999},{"interface":"front","average":{"1min":1.25,"5min":1.276,"15min":1.28},"min":{"1min":0.81999999999999995,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.5980000000000001,"5min":2.9249999999999998,"15min":2.9249999999999998},"last":1.2949999999999999}]},{"osd":4,"last update":"Thu Oct 26 01:32:38 2023","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":1.179,"15min":1.1699999999999999},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.6619999999999999,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.92300000000000004},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.3220000000000001,"15min":1.3109999999999999},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.736,"5min":1.736,"15min":1.736},"last":1.5269999999999999}]},{"osd":6,"last update":"Thu Oct 26 01:32:38 2023","interfaces":[{"interface":"back","average":{"1min":1.3260000000000001,"5min":1.274,"15min":1.266},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.6140000000000001,"5min":1.72,"15min":1.72},"last":1.649},{"interface":"front","average":{"1min":1.415,"5min":1.2949999999999999,"15min":1.2749999999999999},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.736,"5min":1.8560000000000001,"15min":1.8560000000000001},"last":1.6240000000000001}]},{"osd":7,"last update":"Thu Oct 26 01:32:38 2023","interfaces":[{"interface":"back","average":{"1min":1.47,"5min":1.377,"15min":1.361},"min":{"1min":0.64000000000000001,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.8080000000000001,"5min":1.8080000000000001,"15min":1.8080000000000001},"last":1.2410000000000001},{"interface":"front","average":{"1min":1.411,"5min":1.371,"15min":1.365},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.669,"5min":1.8140000000000001,"15min":1.8140000000000001},"last":1.7010000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574888,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":311,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195001,"statfs":{"total":200000000,"available":199681107,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:33:33 2023","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":0.97999999999999998,"15min":0.95699999999999996},"min":{"1min":0.58099999999999996,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.885,"5min":2.0289999999999999,"15min":2.0289999999999999},"last":1.1539999999999999},{"interface":"front","average":{"1min":1.145,"5min":1.0189999999999999,"15min":1.0069999999999999},"min":{"1min":0.67600000000000005,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":2.6829999999999998,"5min":2.6829999999999998,"15min":2.6829999999999998},"last":0.80000000000000004}]},{"osd":1,"last update":"Thu Oct 26 01:33:33 2023","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.123,"15min":1.103},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":4.0519999999999996,"5min":4.0519999999999996,"15min":4.0519999999999996},"last":1.2529999999999999},{"interface":"front","average":{"1min":1.143,"5min":1.1000000000000001,"15min":1.0900000000000001},"min":{"1min":0.627,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.546,"5min":2.0760000000000001,"15min":2.0760000000000001},"last":0.93799999999999994}]},{"osd":2,"last update":"Thu Oct 26 01:33:33 2023","interfaces":[{"interface":"back","average":{"1min":1.579,"5min":1.256,"15min":1.198},"min":{"1min":0.68500000000000005,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":5.6749999999999998,"5min":5.6749999999999998,"15min":5.6749999999999998},"last":1.0580000000000001},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.1200000000000001,"15min":1.1000000000000001},"min":{"1min":0.70899999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.6060000000000001,"5min":1.673,"15min":1.673},"last":1.1020000000000001}]},{"osd":3,"last update":"Thu Oct 26 01:33:33 2023","interfaces":[{"interface":"back","average":{"1min":1.135,"5min":1.173,"15min":1.175},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.661,"5min":1.891,"15min":1.891},"last":1.476},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.151,"15min":1.137},"min":{"1min":0.751,"5min":0.748,"15min":0.748},"max":{"1min":1.7210000000000001,"5min":1.7210000000000001,"15min":1.7210000000000001},"last":1.0129999999999999}]},{"osd":5,"last update":"Thu Oct 26 01:33:33 2023","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.282,"15min":1.296},"min":{"1min":0.86099999999999999,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.782,"5min":2.0049999999999999,"15min":2.0049999999999999},"last":1.3049999999999999},{"interface":"front","average":{"1min":1.5720000000000001,"5min":1.359,"15min":1.3360000000000001},"min":{"1min":0.86799999999999999,"5min":0.626,"15min":0.626},"max":{"1min":5.8579999999999997,"5min":5.8579999999999997,"15min":5.8579999999999997},"last":1.351}]},{"osd":6,"last update":"Thu Oct 26 01:33:33 2023","interfaces":[{"interface":"back","average":{"1min":1.917,"5min":1.4219999999999999,"15min":1.3340000000000001},"min":{"1min":0.91900000000000004,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":12.343999999999999,"5min":12.343999999999999,"15min":12.343999999999999},"last":1.2310000000000001},{"interface":"front","average":{"1min":2.0539999999999998,"5min":1.587,"15min":1.5009999999999999},"min":{"1min":1.0209999999999999,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":12.605,"5min":12.605,"15min":12.605},"last":1.581}]},{"osd":7,"last update":"Thu Oct 26 01:33:33 2023","interfaces":[{"interface":"back","average":{"1min":1.7370000000000001,"5min":1.526,"15min":1.494},"min":{"1min":1.099,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":5.774,"5min":5.774,"15min":5.774},"last":1.4359999999999999},{"interface":"front","average":{"1min":1.631,"5min":1.4710000000000001,"15min":1.4370000000000001},"min":{"1min":1.0069999999999999,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":4.6520000000000001,"5min":4.6520000000000001,"15min":4.6520000000000001},"last":1.395}]}]},{"osd":2,"up_from":13,"seq":55834574888,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":311,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195001,"statfs":{"total":200000000,"available":199681107,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:33:33 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.85999999999999999,"15min":0.85599999999999998},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":1.208,"5min":6.3200000000000003,"15min":6.3200000000000003},"last":0.85799999999999998},{"interface":"front","average":{"1min":1.036,"5min":0.93100000000000005,"15min":0.90500000000000003},"min":{"1min":0.47199999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":2.1400000000000001,"5min":4.2619999999999996,"15min":4.2619999999999996},"last":1.2430000000000001}]},{"osd":1,"last update":"Thu Oct 26 01:33:33 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.90300000000000002,"15min":0.89500000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.569,"5min":5.1260000000000003,"15min":5.1260000000000003},"last":0.93500000000000005},{"interface":"front","average":{"1min":0.97899999999999998,"5min":1.004,"15min":0.99199999999999999},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.74,"5min":6.1520000000000001,"15min":6.1520000000000001},"last":1.161}]},{"osd":3,"last update":"Thu Oct 26 01:33:39 2023","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":0.93600000000000005,"15min":0.90800000000000003},"min":{"1min":0.50900000000000001,"5min":0.443,"15min":0.443},"max":{"1min":1.7250000000000001,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":1.0660000000000001},{"interface":"front","average":{"1min":1.161,"5min":1.044,"15min":0.97699999999999998},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.8660000000000001,"5min":8.9019999999999992,"15min":8.9019999999999992},"last":1.544}]},{"osd":4,"last update":"Thu Oct 26 01:33:33 2023","interfaces":[{"interface":"back","average":{"1min":1.2969999999999999,"5min":1.2589999999999999,"15min":1.2330000000000001},"min":{"1min":0.76000000000000001,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.7929999999999999,"5min":6.2670000000000003,"15min":6.2670000000000003},"last":1.514},{"interface":"front","average":{"1min":1.351,"5min":1.288,"15min":1.2370000000000001},"min":{"1min":0.89300000000000002,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":2.3530000000000002,"5min":7.343,"15min":7.343},"last":1.1080000000000001}]},{"osd":5,"last update":"Thu Oct 26 01:33:33 2023","interfaces":[{"interface":"back","average":{"1min":1.3149999999999999,"5min":1.2689999999999999,"15min":1.2569999999999999},"min":{"1min":0.86099999999999999,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":2.0089999999999999,"5min":3.7170000000000001,"15min":3.7170000000000001},"last":1.3029999999999999},{"interface":"front","average":{"1min":1.2889999999999999,"5min":1.294,"15min":1.28},"min":{"1min":0.70999999999999996,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":2.0099999999999998,"5min":6.3899999999999997,"15min":6.3899999999999997},"last":1.3700000000000001}]},{"osd":6,"last update":"Thu Oct 26 01:33:33 2023","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":1.3320000000000001,"15min":1.304},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.9359999999999999,"5min":6.0490000000000004,"15min":6.0490000000000004},"last":1.4179999999999999},{"interface":"front","average":{"1min":1.401,"5min":1.3660000000000001,"15min":1.3500000000000001},"min":{"1min":0.88400000000000001,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":2.1840000000000002,"5min":5.0650000000000004,"15min":5.0650000000000004},"last":1.48}]},{"osd":7,"last update":"Thu Oct 26 01:33:33 2023","interfaces":[{"interface":"back","average":{"1min":1.3959999999999999,"5min":1.3959999999999999,"15min":1.3899999999999999},"min":{"1min":0.96899999999999997,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":2.0899999999999999,"5min":6.4480000000000004,"15min":6.4480000000000004},"last":1.452},{"interface":"front","average":{"1min":1.4970000000000001,"5min":1.431,"15min":1.429},"min":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"max":{"1min":2.21,"5min":2.8439999999999999,"15min":2.8439999999999999},"last":1.6040000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574888,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":305,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195007,"statfs":{"total":200000000,"available":199687374,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:33:37 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.72399999999999998,"15min":0.70199999999999996},"min":{"1min":0.45600000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.224,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":1.155},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.77100000000000002,"15min":0.76600000000000001},"min":{"1min":0.40500000000000003,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.3879999999999999,"5min":1.4830000000000001,"15min":1.4830000000000001},"last":0.84499999999999997}]},{"osd":2,"last update":"Thu Oct 26 01:33:37 2023","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.98499999999999999,"15min":1.0069999999999999},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.4450000000000001,"5min":2.4830000000000001,"15min":2.4830000000000001},"last":0.94899999999999995},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.84799999999999998,"15min":0.84399999999999997},"min":{"1min":0.47299999999999998,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.68899999999999995}]},{"osd":3,"last update":"Thu Oct 26 01:33:38 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.93999999999999995,"15min":0.93200000000000005},"min":{"1min":0.51700000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.3640000000000001,"5min":1.75,"15min":1.75},"last":0.51800000000000002},{"interface":"front","average":{"1min":1,"5min":0.97799999999999998,"15min":0.97199999999999998},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.361,"5min":1.802,"15min":1.802},"last":1.0960000000000001}]},{"osd":4,"last update":"Thu Oct 26 01:33:37 2023","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.1950000000000001,"15min":1.1859999999999999},"min":{"1min":0.82399999999999995,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.6220000000000001,"5min":2.0659999999999998,"15min":2.0659999999999998},"last":0.79200000000000004},{"interface":"front","average":{"1min":1.2,"5min":1.125,"15min":1.113},"min":{"1min":0.78100000000000003,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.486,"5min":1.611,"15min":1.611},"last":1.0229999999999999}]},{"osd":5,"last update":"Thu Oct 26 01:33:37 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.2070000000000001,"15min":1.214},"min":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":1.5369999999999999,"5min":1.702,"15min":1.702},"last":1.1910000000000001},{"interface":"front","average":{"1min":1.347,"5min":1.3149999999999999,"15min":1.323},"min":{"1min":0.92500000000000004,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.619,"5min":2.2200000000000002,"15min":2.2200000000000002},"last":1.29}]},{"osd":6,"last update":"Thu Oct 26 01:33:37 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.25,"15min":1.256},"min":{"1min":0.83199999999999996,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.694,"5min":2.3780000000000001,"15min":2.3780000000000001},"last":1.2330000000000001},{"interface":"front","average":{"1min":1.2609999999999999,"5min":1.242,"15min":1.2310000000000001},"min":{"1min":0.71299999999999997,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.609,"5min":1.984,"15min":1.984},"last":0.98199999999999998}]},{"osd":7,"last update":"Thu Oct 26 01:33:37 2023","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.242,"15min":1.2609999999999999},"min":{"1min":0.84099999999999997,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.7010000000000001,"5min":2.6930000000000001,"15min":2.6930000000000001},"last":1.333},{"interface":"front","average":{"1min":1.3460000000000001,"5min":1.3140000000000001,"15min":1.306},"min":{"1min":1.0249999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.8240000000000001,"5min":1.869,"15min":1.869},"last":1.5049999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574888,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":887,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194424,"statfs":{"total":200000000,"available":199090739,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Oct 26 01:33:38 2023","interfaces":[{"interface":"back","average":{"1min":1.833,"5min":1.0489999999999999,"15min":0.93999999999999995},"min":{"1min":0.60499999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":19.425000000000001,"5min":19.425000000000001,"15min":19.425000000000001},"last":0.90800000000000003},{"interface":"front","average":{"1min":1.6319999999999999,"5min":1.1240000000000001,"15min":1.0720000000000001},"min":{"1min":0.56799999999999995,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":15.162000000000001,"5min":15.162000000000001,"15min":15.162000000000001},"last":0.98399999999999999}]},{"osd":2,"last update":"Thu Oct 26 01:33:38 2023","interfaces":[{"interface":"back","average":{"1min":2.4940000000000002,"5min":1.3680000000000001,"15min":1.2170000000000001},"min":{"1min":0.57999999999999996,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":28.439,"5min":28.439,"15min":28.439},"last":1.1259999999999999},{"interface":"front","average":{"1min":1.0129999999999999,"5min":0.92200000000000004,"15min":0.91900000000000004},"min":{"1min":0.45200000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.502,"5min":4.9459999999999997,"15min":4.9459999999999997},"last":0.82299999999999995}]},{"osd":3,"last update":"Thu Oct 26 01:33:38 2023","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.075,"15min":1.1020000000000001},"min":{"1min":0.49099999999999999,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.3919999999999999,"5min":4.9800000000000004,"15min":4.9800000000000004},"last":0.94199999999999995},{"interface":"front","average":{"1min":1.21,"5min":1.153,"15min":1.167},"min":{"1min":0.66000000000000003,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.659,"5min":5.016,"15min":5.016},"last":1.0820000000000001}]},{"osd":4,"last update":"Thu Oct 26 01:33:38 2023","interfaces":[{"interface":"back","average":{"1min":2.145,"5min":1.411,"15min":1.3089999999999999},"min":{"1min":0.79100000000000004,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":19.329000000000001,"5min":19.329000000000001,"15min":19.329000000000001},"last":1.4339999999999999},{"interface":"front","average":{"1min":1.2609999999999999,"5min":1.292,"15min":1.3129999999999999},"min":{"1min":0.90300000000000002,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.575,"5min":5.5229999999999997,"15min":5.5229999999999997},"last":1.1799999999999999}]},{"osd":5,"last update":"Thu Oct 26 01:33:38 2023","interfaces":[{"interface":"back","average":{"1min":1.4359999999999999,"5min":1.4319999999999999,"15min":1.444},"min":{"1min":0.93799999999999994,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.915,"5min":5.6360000000000001,"15min":5.6360000000000001},"last":1.4830000000000001},{"interface":"front","average":{"1min":2.3130000000000002,"5min":1.5389999999999999,"15min":1.427},"min":{"1min":0.86299999999999999,"5min":0.626,"15min":0.626},"max":{"1min":19.384,"5min":19.384,"15min":19.384},"last":1.3360000000000001}]},{"osd":6,"last update":"Thu Oct 26 01:33:38 2023","interfaces":[{"interface":"back","average":{"1min":1.7270000000000001,"5min":1.415,"15min":1.385},"min":{"1min":0.82499999999999996,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":8.2609999999999992,"5min":8.2609999999999992,"15min":8.2609999999999992},"last":1.216},{"interface":"front","average":{"1min":1.923,"5min":1.4690000000000001,"15min":1.421},"min":{"1min":1.0049999999999999,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":11.849,"5min":11.849,"15min":11.849},"last":1.2649999999999999}]},{"osd":7,"last update":"Thu Oct 26 01:33:38 2023","interfaces":[{"interface":"back","average":{"1min":1.524,"5min":1.49,"15min":1.5029999999999999},"min":{"1min":1.099,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":2.4060000000000001,"5min":5.6689999999999996,"15min":5.6689999999999996},"last":1.3779999999999999},{"interface":"front","average":{"1min":2.387,"5min":1.6220000000000001,"15min":1.514},"min":{"1min":1.0529999999999999,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":19.245999999999999,"5min":19.245999999999999,"15min":19.245999999999999},"last":1.536}]}]}],"pool_statfs":[]}} 2023-10-26T01:33:45.772 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-26T01:33:46.024 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:46.022+0000 7f63aa6fc640 1 -- 172.21.15.169:0/2866174847 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63a41592d0 msgr2=0x7f63a40aa3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:46.024 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:46.022+0000 7f63aa6fc640 1 --2- 172.21.15.169:0/2866174847 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63a41592d0 0x7f63a40aa3d0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f6394005fd0 tx=0x7f639402bcf0 comp rx=0 tx=0).stop 2023-10-26T01:33:46.025 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:46.022+0000 7f63aa6fc640 1 -- 172.21.15.169:0/2866174847 shutdown_connections 2023-10-26T01:33:46.025 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:46.022+0000 7f63aa6fc640 1 --2- 172.21.15.169:0/2866174847 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f63a40cde00 0x7f63a40cbac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:46.025 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:46.022+0000 7f63aa6fc640 1 --2- 172.21.15.169:0/2866174847 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63a41592d0 0x7f63a40aa3d0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:46.025 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:46.022+0000 7f63aa6fc640 1 -- 172.21.15.169:0/2866174847 >> 172.21.15.169:0/2866174847 conn(0x7f63a4069930 msgr2=0x7f63a4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:46.026 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:46.022+0000 7f63aa6fc640 1 -- 172.21.15.169:0/2866174847 shutdown_connections 2023-10-26T01:33:46.026 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:46.022+0000 7f63aa6fc640 1 -- 172.21.15.169:0/2866174847 wait complete. 2023-10-26T01:33:46.028 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:46.024+0000 7f63aa6fc640 1 Processor -- start 2023-10-26T01:33:46.028 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:46.024+0000 7f63aa6fc640 1 -- start start 2023-10-26T01:33:46.028 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:46.025+0000 7f63aa6fc640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63a40ac930 0x7f63a4154b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:46.029 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:46.025+0000 7f63aa6fc640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f63a40cde00 0x7f63a40c6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:46.029 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:46.025+0000 7f63aa6fc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f63a41592d0 0x7f63a40ae270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:46.029 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:46.025+0000 7f63aa6fc640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63a41f5080 con 0x7f63a40ac930 2023-10-26T01:33:46.029 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:46.025+0000 7f63aa6fc640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63a41f5360 con 0x7f63a41592d0 2023-10-26T01:33:46.030 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:46.025+0000 7f63aa6fc640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63a41f5640 con 0x7f63a40cde00 2023-10-26T01:33:46.030 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:46.025+0000 7f63a3fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63a40ac930 0x7f63a4154b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:46.030 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:46.025+0000 7f63a3fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63a40ac930 0x7f63a4154b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:59378/0 (socket says 172.21.15.169:59378) 2023-10-26T01:33:46.031 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:46.025+0000 7f63a3fff640 1 -- 172.21.15.169:0/956699270 learned_addr learned my addr 172.21.15.169:0/956699270 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:33:46.031 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:46.026+0000 7f63a3fff640 1 -- 172.21.15.169:0/956699270 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f63a40cde00 msgr2=0x7f63a40c6bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:46.031 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:46.026+0000 7f63a37fe640 1 --2- 172.21.15.169:0/956699270 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f63a40cde00 0x7f63a40c6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:46.031 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:46.026+0000 7f63a3fff640 1 --2- 172.21.15.169:0/956699270 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f63a40cde00 0x7f63a40c6bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:46.032 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:46.026+0000 7f63a3fff640 1 -- 172.21.15.169:0/956699270 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f63a41592d0 msgr2=0x7f63a40ae270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:46.034 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:46.026+0000 7f63a3fff640 1 --2- 172.21.15.169:0/956699270 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f63a41592d0 0x7f63a40ae270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:46.034 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:46.026+0000 7f63a3fff640 1 -- 172.21.15.169:0/956699270 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6394009560 con 0x7f63a40ac930 2023-10-26T01:33:46.034 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:46.026+0000 7f63a3fff640 1 --2- 172.21.15.169:0/956699270 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63a40ac930 0x7f63a4154b50 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f6394009850 tx=0x7f639402da70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:46.034 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:46.026+0000 7f63a0ff9640 1 -- 172.21.15.169:0/956699270 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f639402de70 con 0x7f63a40ac930 2023-10-26T01:33:46.035 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:46.026+0000 7f63aa6fc640 1 -- 172.21.15.169:0/956699270 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63a41f5920 con 0x7f63a40ac930 2023-10-26T01:33:46.035 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:46.026+0000 7f63a0ff9640 1 -- 172.21.15.169:0/956699270 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f63940036e0 con 0x7f63a40ac930 2023-10-26T01:33:46.035 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:46.026+0000 7f63a0ff9640 1 -- 172.21.15.169:0/956699270 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6394003a90 con 0x7f63a40ac930 2023-10-26T01:33:46.035 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:46.027+0000 7f63aa6fc640 1 -- 172.21.15.169:0/956699270 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63a40c6730 con 0x7f63a40ac930 2023-10-26T01:33:46.037 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:46.028+0000 7f63aa6fc640 1 -- 172.21.15.169:0/956699270 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63a41f5e40 con 0x7f63a40ac930 2023-10-26T01:33:46.037 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:46.029+0000 7f63a0ff9640 1 -- 172.21.15.169:0/956699270 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f6394003bf0 con 0x7f63a40ac930 2023-10-26T01:33:46.037 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:46.029+0000 7f63a0ff9640 1 --2- 172.21.15.169:0/956699270 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f639006ae70 0x7f639006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:46.037 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:46.030+0000 7f63a0ff9640 1 -- 172.21.15.169:0/956699270 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f63940ac240 con 0x7f63a40ac930 2023-10-26T01:33:46.037 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:46.030+0000 7f63a37fe640 1 --2- 172.21.15.169:0/956699270 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f639006ae70 0x7f639006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:46.038 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:46.031+0000 7f63a37fe640 1 --2- 172.21.15.169:0/956699270 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f639006ae70 0x7f639006d330 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f6398007fd0 tx=0x7f6398009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:46.038 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:46.034+0000 7f63a0ff9640 1 -- 172.21.15.169:0/956699270 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f63a41f5e40 con 0x7f63a40ac930 2023-10-26T01:33:46.220 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:46.219+0000 7f63aa6fc640 1 -- 172.21.15.169:0/956699270 --> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f63a41f54d0 con 0x7f639006ae70 2023-10-26T01:33:46.243 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:46.241+0000 7f63a0ff9640 1 -- 172.21.15.169:0/956699270 <== mgr.4112 v2:172.21.15.170:6832/1187156918 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+282467 (secure 0 0 0) 0x7f63a41f54d0 con 0x7f639006ae70 2023-10-26T01:33:46.245 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:33:46.248 INFO:teuthology.orchestra.run.smithi169.stderr:dumped all 2023-10-26T01:33:46.249 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:46.247+0000 7f63aa6fc640 1 -- 172.21.15.169:0/956699270 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f639006ae70 msgr2=0x7f639006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:46.250 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:46.247+0000 7f63aa6fc640 1 --2- 172.21.15.169:0/956699270 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f639006ae70 0x7f639006d330 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f6398007fd0 tx=0x7f6398009040 comp rx=0 tx=0).stop 2023-10-26T01:33:46.250 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:46.247+0000 7f63aa6fc640 1 -- 172.21.15.169:0/956699270 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63a40ac930 msgr2=0x7f63a4154b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:46.250 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:46.247+0000 7f63aa6fc640 1 --2- 172.21.15.169:0/956699270 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63a40ac930 0x7f63a4154b50 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f6394009850 tx=0x7f639402da70 comp rx=0 tx=0).stop 2023-10-26T01:33:46.250 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:46.247+0000 7f63aa6fc640 1 -- 172.21.15.169:0/956699270 shutdown_connections 2023-10-26T01:33:46.250 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:46.247+0000 7f63aa6fc640 1 --2- 172.21.15.169:0/956699270 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f639006ae70 0x7f639006d330 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:46.251 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:46.247+0000 7f63aa6fc640 1 --2- 172.21.15.169:0/956699270 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f63a41592d0 0x7f63a40ae270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:46.251 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:46.247+0000 7f63aa6fc640 1 --2- 172.21.15.169:0/956699270 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f63a40cde00 0x7f63a40c6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:46.251 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:46.247+0000 7f63aa6fc640 1 --2- 172.21.15.169:0/956699270 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63a40ac930 0x7f63a4154b50 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:46.251 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:46.247+0000 7f63aa6fc640 1 -- 172.21.15.169:0/956699270 >> 172.21.15.169:0/956699270 conn(0x7f63a4069930 msgr2=0x7f63a4155f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:46.251 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:46.247+0000 7f63aa6fc640 1 -- 172.21.15.169:0/956699270 shutdown_connections 2023-10-26T01:33:46.252 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:46.247+0000 7f63aa6fc640 1 -- 172.21.15.169:0/956699270 wait complete. 2023-10-26T01:33:46.267 INFO:teuthology.orchestra.run.smithi169.stdout:{"pg_ready":true,"pg_map":{"version":156,"stamp":"2023-10-26T01:33:46.193236+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":187,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":3744,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1558755,"statfs":{"total":1600000000,"available":1596165614,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001866"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-26T01:33:43.741972+0000","last_change":"2023-10-26T01:33:41.312540+0000","last_active":"2023-10-26T01:33:43.741972+0000","last_peered":"2023-10-26T01:33:43.741972+0000","last_clean":"2023-10-26T01:33:43.741972+0000","last_became_active":"2023-10-26T01:33:41.312271+0000","last_became_peered":"2023-10-26T01:33:41.312271+0000","last_unstale":"2023-10-26T01:33:43.741972+0000","last_undegraded":"2023-10-26T01:33:43.741972+0000","last_fullsized":"2023-10-26T01:33:43.741972+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:14:04.835272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-26T01:33:41.301683+0000","last_change":"2023-10-26T01:33:41.300524+0000","last_active":"2023-10-26T01:33:41.301683+0000","last_peered":"2023-10-26T01:33:41.301683+0000","last_clean":"2023-10-26T01:33:41.301683+0000","last_became_active":"2023-10-26T01:33:41.295476+0000","last_became_peered":"2023-10-26T01:33:41.295476+0000","last_unstale":"2023-10-26T01:33:41.301683+0000","last_undegraded":"2023-10-26T01:33:41.301683+0000","last_fullsized":"2023-10-26T01:33:41.301683+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:35:24.220074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-26T01:33:42.277173+0000","last_change":"2023-10-26T01:33:41.292909+0000","last_active":"2023-10-26T01:33:42.277173+0000","last_peered":"2023-10-26T01:33:42.277173+0000","last_clean":"2023-10-26T01:33:42.277173+0000","last_became_active":"2023-10-26T01:33:41.289535+0000","last_became_peered":"2023-10-26T01:33:41.289535+0000","last_unstale":"2023-10-26T01:33:42.277173+0000","last_undegraded":"2023-10-26T01:33:42.277173+0000","last_fullsized":"2023-10-26T01:33:42.277173+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:23:22.078788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-26T01:33:42.573031+0000","last_change":"2023-10-26T01:33:41.308831+0000","last_active":"2023-10-26T01:33:42.573031+0000","last_peered":"2023-10-26T01:33:42.573031+0000","last_clean":"2023-10-26T01:33:42.573031+0000","last_became_active":"2023-10-26T01:33:41.308674+0000","last_became_peered":"2023-10-26T01:33:41.308674+0000","last_unstale":"2023-10-26T01:33:42.573031+0000","last_undegraded":"2023-10-26T01:33:42.573031+0000","last_fullsized":"2023-10-26T01:33:42.573031+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:35:33.216308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-26T01:33:43.742060+0000","last_change":"2023-10-26T01:33:41.307247+0000","last_active":"2023-10-26T01:33:43.742060+0000","last_peered":"2023-10-26T01:33:43.742060+0000","last_clean":"2023-10-26T01:33:43.742060+0000","last_became_active":"2023-10-26T01:33:41.307024+0000","last_became_peered":"2023-10-26T01:33:41.307024+0000","last_unstale":"2023-10-26T01:33:43.742060+0000","last_undegraded":"2023-10-26T01:33:43.742060+0000","last_fullsized":"2023-10-26T01:33:43.742060+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:06:10.291267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-26T01:33:42.571897+0000","last_change":"2023-10-26T01:33:41.302576+0000","last_active":"2023-10-26T01:33:42.571897+0000","last_peered":"2023-10-26T01:33:42.571897+0000","last_clean":"2023-10-26T01:33:42.571897+0000","last_became_active":"2023-10-26T01:33:41.302298+0000","last_became_peered":"2023-10-26T01:33:41.302298+0000","last_unstale":"2023-10-26T01:33:42.571897+0000","last_undegraded":"2023-10-26T01:33:42.571897+0000","last_fullsized":"2023-10-26T01:33:42.571897+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:13:55.953915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-26T01:33:42.279035+0000","last_change":"2023-10-26T01:33:41.297658+0000","last_active":"2023-10-26T01:33:42.279035+0000","last_peered":"2023-10-26T01:33:42.279035+0000","last_clean":"2023-10-26T01:33:42.279035+0000","last_became_active":"2023-10-26T01:33:41.296664+0000","last_became_peered":"2023-10-26T01:33:41.296664+0000","last_unstale":"2023-10-26T01:33:42.279035+0000","last_undegraded":"2023-10-26T01:33:42.279035+0000","last_fullsized":"2023-10-26T01:33:42.279035+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:42:45.072477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-26T01:33:41.299557+0000","last_change":"2023-10-26T01:33:41.299209+0000","last_active":"2023-10-26T01:33:41.299557+0000","last_peered":"2023-10-26T01:33:41.299557+0000","last_clean":"2023-10-26T01:33:41.299557+0000","last_became_active":"2023-10-26T01:33:41.298665+0000","last_became_peered":"2023-10-26T01:33:41.298665+0000","last_unstale":"2023-10-26T01:33:41.299557+0000","last_undegraded":"2023-10-26T01:33:41.299557+0000","last_fullsized":"2023-10-26T01:33:41.299557+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:09:19.054540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-26T01:33:42.276846+0000","last_change":"2023-10-26T01:33:41.285776+0000","last_active":"2023-10-26T01:33:42.276846+0000","last_peered":"2023-10-26T01:33:42.276846+0000","last_clean":"2023-10-26T01:33:42.276846+0000","last_became_active":"2023-10-26T01:33:41.285226+0000","last_became_peered":"2023-10-26T01:33:41.285226+0000","last_unstale":"2023-10-26T01:33:42.276846+0000","last_undegraded":"2023-10-26T01:33:42.276846+0000","last_fullsized":"2023-10-26T01:33:42.276846+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T01:42:22.470655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-26T01:33:42.275126+0000","last_change":"2023-10-26T01:33:41.305158+0000","last_active":"2023-10-26T01:33:42.275126+0000","last_peered":"2023-10-26T01:33:42.275126+0000","last_clean":"2023-10-26T01:33:42.275126+0000","last_became_active":"2023-10-26T01:33:41.304938+0000","last_became_peered":"2023-10-26T01:33:41.304938+0000","last_unstale":"2023-10-26T01:33:42.275126+0000","last_undegraded":"2023-10-26T01:33:42.275126+0000","last_fullsized":"2023-10-26T01:33:42.275126+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:16:23.987005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-26T01:33:42.275256+0000","last_change":"2023-10-26T01:33:41.300361+0000","last_active":"2023-10-26T01:33:42.275256+0000","last_peered":"2023-10-26T01:33:42.275256+0000","last_clean":"2023-10-26T01:33:42.275256+0000","last_became_active":"2023-10-26T01:33:41.297751+0000","last_became_peered":"2023-10-26T01:33:41.297751+0000","last_unstale":"2023-10-26T01:33:42.275256+0000","last_undegraded":"2023-10-26T01:33:42.275256+0000","last_fullsized":"2023-10-26T01:33:42.275256+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:03:07.326428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-26T01:33:42.573138+0000","last_change":"2023-10-26T01:33:41.298238+0000","last_active":"2023-10-26T01:33:42.573138+0000","last_peered":"2023-10-26T01:33:42.573138+0000","last_clean":"2023-10-26T01:33:42.573138+0000","last_became_active":"2023-10-26T01:33:41.297144+0000","last_became_peered":"2023-10-26T01:33:41.297144+0000","last_unstale":"2023-10-26T01:33:42.573138+0000","last_undegraded":"2023-10-26T01:33:42.573138+0000","last_fullsized":"2023-10-26T01:33:42.573138+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:17:32.526317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-26T01:33:43.743513+0000","last_change":"2023-10-26T01:33:41.310485+0000","last_active":"2023-10-26T01:33:43.743513+0000","last_peered":"2023-10-26T01:33:43.743513+0000","last_clean":"2023-10-26T01:33:43.743513+0000","last_became_active":"2023-10-26T01:33:41.310319+0000","last_became_peered":"2023-10-26T01:33:41.310319+0000","last_unstale":"2023-10-26T01:33:43.743513+0000","last_undegraded":"2023-10-26T01:33:43.743513+0000","last_fullsized":"2023-10-26T01:33:43.743513+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:47:20.708548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-26T01:33:42.277187+0000","last_change":"2023-10-26T01:33:41.306287+0000","last_active":"2023-10-26T01:33:42.277187+0000","last_peered":"2023-10-26T01:33:42.277187+0000","last_clean":"2023-10-26T01:33:42.277187+0000","last_became_active":"2023-10-26T01:33:41.306033+0000","last_became_peered":"2023-10-26T01:33:41.306033+0000","last_unstale":"2023-10-26T01:33:42.277187+0000","last_undegraded":"2023-10-26T01:33:42.277187+0000","last_fullsized":"2023-10-26T01:33:42.277187+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:48:19.456198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-26T01:33:42.280223+0000","last_change":"2023-10-26T01:33:41.292740+0000","last_active":"2023-10-26T01:33:42.280223+0000","last_peered":"2023-10-26T01:33:42.280223+0000","last_clean":"2023-10-26T01:33:42.280223+0000","last_became_active":"2023-10-26T01:33:41.292365+0000","last_became_peered":"2023-10-26T01:33:41.292365+0000","last_unstale":"2023-10-26T01:33:42.280223+0000","last_undegraded":"2023-10-26T01:33:42.280223+0000","last_fullsized":"2023-10-26T01:33:42.280223+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:31:35.715377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-26T01:33:42.572401+0000","last_change":"2023-10-26T01:33:41.301951+0000","last_active":"2023-10-26T01:33:42.572401+0000","last_peered":"2023-10-26T01:33:42.572401+0000","last_clean":"2023-10-26T01:33:42.572401+0000","last_became_active":"2023-10-26T01:33:41.297300+0000","last_became_peered":"2023-10-26T01:33:41.297300+0000","last_unstale":"2023-10-26T01:33:42.572401+0000","last_undegraded":"2023-10-26T01:33:42.572401+0000","last_fullsized":"2023-10-26T01:33:42.572401+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:48:22.480340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-26T01:33:41.301717+0000","last_change":"2023-10-26T01:33:41.301569+0000","last_active":"2023-10-26T01:33:41.301717+0000","last_peered":"2023-10-26T01:33:41.301717+0000","last_clean":"2023-10-26T01:33:41.301717+0000","last_became_active":"2023-10-26T01:33:41.301183+0000","last_became_peered":"2023-10-26T01:33:41.301183+0000","last_unstale":"2023-10-26T01:33:41.301717+0000","last_undegraded":"2023-10-26T01:33:41.301717+0000","last_fullsized":"2023-10-26T01:33:41.301717+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:23:14.596280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-26T01:33:42.277602+0000","last_change":"2023-10-26T01:33:41.292598+0000","last_active":"2023-10-26T01:33:42.277602+0000","last_peered":"2023-10-26T01:33:42.277602+0000","last_clean":"2023-10-26T01:33:42.277602+0000","last_became_active":"2023-10-26T01:33:41.288796+0000","last_became_peered":"2023-10-26T01:33:41.288796+0000","last_unstale":"2023-10-26T01:33:42.277602+0000","last_undegraded":"2023-10-26T01:33:42.277602+0000","last_fullsized":"2023-10-26T01:33:42.277602+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:06:05.830226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-26T01:33:43.741860+0000","last_change":"2023-10-26T01:33:41.310034+0000","last_active":"2023-10-26T01:33:43.741860+0000","last_peered":"2023-10-26T01:33:43.741860+0000","last_clean":"2023-10-26T01:33:43.741860+0000","last_became_active":"2023-10-26T01:33:41.309672+0000","last_became_peered":"2023-10-26T01:33:41.309672+0000","last_unstale":"2023-10-26T01:33:43.741860+0000","last_undegraded":"2023-10-26T01:33:43.741860+0000","last_fullsized":"2023-10-26T01:33:43.741860+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:22:02.975566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-26T01:33:41.295374+0000","last_change":"2023-10-26T01:33:41.295080+0000","last_active":"2023-10-26T01:33:41.295374+0000","last_peered":"2023-10-26T01:33:41.295374+0000","last_clean":"2023-10-26T01:33:41.295374+0000","last_became_active":"2023-10-26T01:33:41.294605+0000","last_became_peered":"2023-10-26T01:33:41.294605+0000","last_unstale":"2023-10-26T01:33:41.295374+0000","last_undegraded":"2023-10-26T01:33:41.295374+0000","last_fullsized":"2023-10-26T01:33:41.295374+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:03:47.019520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-26T01:33:42.277976+0000","last_change":"2023-10-26T01:33:41.297595+0000","last_active":"2023-10-26T01:33:42.277976+0000","last_peered":"2023-10-26T01:33:42.277976+0000","last_clean":"2023-10-26T01:33:42.277976+0000","last_became_active":"2023-10-26T01:33:41.296285+0000","last_became_peered":"2023-10-26T01:33:41.296285+0000","last_unstale":"2023-10-26T01:33:42.277976+0000","last_undegraded":"2023-10-26T01:33:42.277976+0000","last_fullsized":"2023-10-26T01:33:42.277976+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:35:24.220074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-26T01:33:41.310784+0000","last_change":"2023-10-26T01:33:41.309460+0000","last_active":"2023-10-26T01:33:41.310784+0000","last_peered":"2023-10-26T01:33:41.310784+0000","last_clean":"2023-10-26T01:33:41.310784+0000","last_became_active":"2023-10-26T01:33:41.309191+0000","last_became_peered":"2023-10-26T01:33:41.309191+0000","last_unstale":"2023-10-26T01:33:41.310784+0000","last_undegraded":"2023-10-26T01:33:41.310784+0000","last_fullsized":"2023-10-26T01:33:41.310784+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:22:38.364628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-26T01:33:42.277910+0000","last_change":"2023-10-26T01:33:41.293192+0000","last_active":"2023-10-26T01:33:42.277910+0000","last_peered":"2023-10-26T01:33:42.277910+0000","last_clean":"2023-10-26T01:33:42.277910+0000","last_became_active":"2023-10-26T01:33:41.290393+0000","last_became_peered":"2023-10-26T01:33:41.290393+0000","last_unstale":"2023-10-26T01:33:42.277910+0000","last_undegraded":"2023-10-26T01:33:42.277910+0000","last_fullsized":"2023-10-26T01:33:42.277910+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:09:19.054540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-26T01:33:43.743182+0000","last_change":"2023-10-26T01:33:41.303610+0000","last_active":"2023-10-26T01:33:43.743182+0000","last_peered":"2023-10-26T01:33:43.743182+0000","last_clean":"2023-10-26T01:33:43.743182+0000","last_became_active":"2023-10-26T01:33:41.303332+0000","last_became_peered":"2023-10-26T01:33:41.303332+0000","last_unstale":"2023-10-26T01:33:43.743182+0000","last_undegraded":"2023-10-26T01:33:43.743182+0000","last_fullsized":"2023-10-26T01:33:43.743182+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:02:58.625395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-26T01:33:43.742353+0000","last_change":"2023-10-26T01:33:41.308148+0000","last_active":"2023-10-26T01:33:43.742353+0000","last_peered":"2023-10-26T01:33:43.742353+0000","last_clean":"2023-10-26T01:33:43.742353+0000","last_became_active":"2023-10-26T01:33:41.307934+0000","last_became_peered":"2023-10-26T01:33:41.307934+0000","last_unstale":"2023-10-26T01:33:43.742353+0000","last_undegraded":"2023-10-26T01:33:43.742353+0000","last_fullsized":"2023-10-26T01:33:43.742353+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:16:03.976036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-26T01:33:42.573037+0000","last_change":"2023-10-26T01:33:41.296489+0000","last_active":"2023-10-26T01:33:42.573037+0000","last_peered":"2023-10-26T01:33:42.573037+0000","last_clean":"2023-10-26T01:33:42.573037+0000","last_became_active":"2023-10-26T01:33:41.296090+0000","last_became_peered":"2023-10-26T01:33:41.296090+0000","last_unstale":"2023-10-26T01:33:42.573037+0000","last_undegraded":"2023-10-26T01:33:42.573037+0000","last_fullsized":"2023-10-26T01:33:42.573037+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:21:36.442653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-26T01:33:42.571890+0000","last_change":"2023-10-26T01:33:41.298931+0000","last_active":"2023-10-26T01:33:42.571890+0000","last_peered":"2023-10-26T01:33:42.571890+0000","last_clean":"2023-10-26T01:33:42.571890+0000","last_became_active":"2023-10-26T01:33:41.296311+0000","last_became_peered":"2023-10-26T01:33:41.296311+0000","last_unstale":"2023-10-26T01:33:42.571890+0000","last_undegraded":"2023-10-26T01:33:42.571890+0000","last_fullsized":"2023-10-26T01:33:42.571890+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:12:26.991561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-26T01:33:43.742042+0000","last_change":"2023-10-26T01:33:41.305926+0000","last_active":"2023-10-26T01:33:43.742042+0000","last_peered":"2023-10-26T01:33:43.742042+0000","last_clean":"2023-10-26T01:33:43.742042+0000","last_became_active":"2023-10-26T01:33:41.305534+0000","last_became_peered":"2023-10-26T01:33:41.305534+0000","last_unstale":"2023-10-26T01:33:43.742042+0000","last_undegraded":"2023-10-26T01:33:43.742042+0000","last_fullsized":"2023-10-26T01:33:43.742042+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:51:05.100286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-26T01:33:41.302333+0000","last_change":"2023-10-26T01:33:41.301837+0000","last_active":"2023-10-26T01:33:41.302333+0000","last_peered":"2023-10-26T01:33:41.302333+0000","last_clean":"2023-10-26T01:33:41.302333+0000","last_became_active":"2023-10-26T01:33:41.301551+0000","last_became_peered":"2023-10-26T01:33:41.301551+0000","last_unstale":"2023-10-26T01:33:41.302333+0000","last_undegraded":"2023-10-26T01:33:41.302333+0000","last_fullsized":"2023-10-26T01:33:41.302333+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:06:05.830226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-26T01:33:42.277381+0000","last_change":"2023-10-26T01:33:41.296729+0000","last_active":"2023-10-26T01:33:42.277381+0000","last_peered":"2023-10-26T01:33:42.277381+0000","last_clean":"2023-10-26T01:33:42.277381+0000","last_became_active":"2023-10-26T01:33:41.294622+0000","last_became_peered":"2023-10-26T01:33:41.294622+0000","last_unstale":"2023-10-26T01:33:42.277381+0000","last_undegraded":"2023-10-26T01:33:42.277381+0000","last_fullsized":"2023-10-26T01:33:42.277381+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:26:16.433105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-26T01:33:43.742301+0000","last_change":"2023-10-26T01:33:41.309172+0000","last_active":"2023-10-26T01:33:43.742301+0000","last_peered":"2023-10-26T01:33:43.742301+0000","last_clean":"2023-10-26T01:33:43.742301+0000","last_became_active":"2023-10-26T01:33:41.308946+0000","last_became_peered":"2023-10-26T01:33:41.308946+0000","last_unstale":"2023-10-26T01:33:43.742301+0000","last_undegraded":"2023-10-26T01:33:43.742301+0000","last_fullsized":"2023-10-26T01:33:43.742301+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:51:52.829756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-26T01:33:42.572248+0000","last_change":"2023-10-26T01:33:41.299111+0000","last_active":"2023-10-26T01:33:42.572248+0000","last_peered":"2023-10-26T01:33:42.572248+0000","last_clean":"2023-10-26T01:33:42.572248+0000","last_became_active":"2023-10-26T01:33:41.298666+0000","last_became_peered":"2023-10-26T01:33:41.298666+0000","last_unstale":"2023-10-26T01:33:42.572248+0000","last_undegraded":"2023-10-26T01:33:42.572248+0000","last_fullsized":"2023-10-26T01:33:42.572248+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:15:48.934656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-26T01:33:42.571824+0000","last_change":"2023-10-26T01:33:41.278068+0000","last_active":"2023-10-26T01:33:42.571824+0000","last_peered":"2023-10-26T01:33:42.571824+0000","last_clean":"2023-10-26T01:33:42.571824+0000","last_became_active":"2023-10-26T01:33:41.277428+0000","last_became_peered":"2023-10-26T01:33:41.277428+0000","last_unstale":"2023-10-26T01:33:42.571824+0000","last_undegraded":"2023-10-26T01:33:42.571824+0000","last_fullsized":"2023-10-26T01:33:42.571824+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T01:39:22.276139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"64'4","reported_seq":16,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-26T01:33:41.320304+0000","last_change":"2023-10-26T01:33:40.271579+0000","last_active":"2023-10-26T01:33:41.320304+0000","last_peered":"2023-10-26T01:33:41.320304+0000","last_clean":"2023-10-26T01:33:41.320304+0000","last_became_active":"2023-10-26T01:33:40.271194+0000","last_became_peered":"2023-10-26T01:33:40.271194+0000","last_unstale":"2023-10-26T01:33:41.320304+0000","last_undegraded":"2023-10-26T01:33:41.320304+0000","last_fullsized":"2023-10-26T01:33:41.320304+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_clean_scrub_stamp":"2023-10-26T01:33:39.224951+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:35:23.209477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-26T01:33:42.274793+0000","last_change":"2023-10-26T01:33:41.304577+0000","last_active":"2023-10-26T01:33:42.274793+0000","last_peered":"2023-10-26T01:33:42.274793+0000","last_clean":"2023-10-26T01:33:42.274793+0000","last_became_active":"2023-10-26T01:33:41.304290+0000","last_became_peered":"2023-10-26T01:33:41.304290+0000","last_unstale":"2023-10-26T01:33:42.274793+0000","last_undegraded":"2023-10-26T01:33:42.274793+0000","last_fullsized":"2023-10-26T01:33:42.274793+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:09:17.094564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"64'1","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-26T01:33:41.317285+0000","last_change":"2023-10-26T01:33:40.266886+0000","last_active":"2023-10-26T01:33:41.317285+0000","last_peered":"2023-10-26T01:33:41.317285+0000","last_clean":"2023-10-26T01:33:41.317285+0000","last_became_active":"2023-10-26T01:33:40.266621+0000","last_became_peered":"2023-10-26T01:33:40.266621+0000","last_unstale":"2023-10-26T01:33:41.317285+0000","last_undegraded":"2023-10-26T01:33:41.317285+0000","last_fullsized":"2023-10-26T01:33:41.317285+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_clean_scrub_stamp":"2023-10-26T01:33:39.224951+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:22:58.516924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-26T01:33:42.572701+0000","last_change":"2023-10-26T01:33:41.290812+0000","last_active":"2023-10-26T01:33:42.572701+0000","last_peered":"2023-10-26T01:33:42.572701+0000","last_clean":"2023-10-26T01:33:42.572701+0000","last_became_active":"2023-10-26T01:33:41.289119+0000","last_became_peered":"2023-10-26T01:33:41.289119+0000","last_unstale":"2023-10-26T01:33:42.572701+0000","last_undegraded":"2023-10-26T01:33:42.572701+0000","last_fullsized":"2023-10-26T01:33:42.572701+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:10:12.647542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"64'2","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-26T01:33:41.320115+0000","last_change":"2023-10-26T01:33:40.266751+0000","last_active":"2023-10-26T01:33:41.320115+0000","last_peered":"2023-10-26T01:33:41.320115+0000","last_clean":"2023-10-26T01:33:41.320115+0000","last_became_active":"2023-10-26T01:33:40.266391+0000","last_became_peered":"2023-10-26T01:33:40.266391+0000","last_unstale":"2023-10-26T01:33:41.320115+0000","last_undegraded":"2023-10-26T01:33:41.320115+0000","last_fullsized":"2023-10-26T01:33:41.320115+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_clean_scrub_stamp":"2023-10-26T01:33:39.224951+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:22:37.354032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-26T01:33:42.571677+0000","last_change":"2023-10-26T01:33:41.297096+0000","last_active":"2023-10-26T01:33:42.571677+0000","last_peered":"2023-10-26T01:33:42.571677+0000","last_clean":"2023-10-26T01:33:42.571677+0000","last_became_active":"2023-10-26T01:33:41.296775+0000","last_became_peered":"2023-10-26T01:33:41.296775+0000","last_unstale":"2023-10-26T01:33:42.571677+0000","last_undegraded":"2023-10-26T01:33:42.571677+0000","last_fullsized":"2023-10-26T01:33:42.571677+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:14:31.511712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"64'2","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-26T01:33:41.317367+0000","last_change":"2023-10-26T01:33:40.267788+0000","last_active":"2023-10-26T01:33:41.317367+0000","last_peered":"2023-10-26T01:33:41.317367+0000","last_clean":"2023-10-26T01:33:41.317367+0000","last_became_active":"2023-10-26T01:33:40.266846+0000","last_became_peered":"2023-10-26T01:33:40.266846+0000","last_unstale":"2023-10-26T01:33:41.317367+0000","last_undegraded":"2023-10-26T01:33:41.317367+0000","last_fullsized":"2023-10-26T01:33:41.317367+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_clean_scrub_stamp":"2023-10-26T01:33:39.224951+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:28:43.420820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-26T01:33:41.311604+0000","last_change":"2023-10-26T01:33:41.311443+0000","last_active":"2023-10-26T01:33:41.311604+0000","last_peered":"2023-10-26T01:33:41.311604+0000","last_clean":"2023-10-26T01:33:41.311604+0000","last_became_active":"2023-10-26T01:33:41.311197+0000","last_became_peered":"2023-10-26T01:33:41.311197+0000","last_unstale":"2023-10-26T01:33:41.311604+0000","last_undegraded":"2023-10-26T01:33:41.311604+0000","last_fullsized":"2023-10-26T01:33:41.311604+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:56:20.883631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"64'1","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-26T01:33:41.317776+0000","last_change":"2023-10-26T01:33:40.269465+0000","last_active":"2023-10-26T01:33:41.317776+0000","last_peered":"2023-10-26T01:33:41.317776+0000","last_clean":"2023-10-26T01:33:41.317776+0000","last_became_active":"2023-10-26T01:33:40.266610+0000","last_became_peered":"2023-10-26T01:33:40.266610+0000","last_unstale":"2023-10-26T01:33:41.317776+0000","last_undegraded":"2023-10-26T01:33:41.317776+0000","last_fullsized":"2023-10-26T01:33:41.317776+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_clean_scrub_stamp":"2023-10-26T01:33:39.224951+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:10:30.826612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-26T01:33:41.301702+0000","last_change":"2023-10-26T01:33:41.301423+0000","last_active":"2023-10-26T01:33:41.301702+0000","last_peered":"2023-10-26T01:33:41.301702+0000","last_clean":"2023-10-26T01:33:41.301702+0000","last_became_active":"2023-10-26T01:33:41.300804+0000","last_became_peered":"2023-10-26T01:33:41.300804+0000","last_unstale":"2023-10-26T01:33:41.301702+0000","last_undegraded":"2023-10-26T01:33:41.301702+0000","last_fullsized":"2023-10-26T01:33:41.301702+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:06:53.887209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"64'1","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-26T01:33:41.320404+0000","last_change":"2023-10-26T01:33:40.268904+0000","last_active":"2023-10-26T01:33:41.320404+0000","last_peered":"2023-10-26T01:33:41.320404+0000","last_clean":"2023-10-26T01:33:41.320404+0000","last_became_active":"2023-10-26T01:33:40.268191+0000","last_became_peered":"2023-10-26T01:33:40.268191+0000","last_unstale":"2023-10-26T01:33:41.320404+0000","last_undegraded":"2023-10-26T01:33:41.320404+0000","last_fullsized":"2023-10-26T01:33:41.320404+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_clean_scrub_stamp":"2023-10-26T01:33:39.224951+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:31:50.211414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-26T01:33:43.743143+0000","last_change":"2023-10-26T01:33:41.299481+0000","last_active":"2023-10-26T01:33:43.743143+0000","last_peered":"2023-10-26T01:33:43.743143+0000","last_clean":"2023-10-26T01:33:43.743143+0000","last_became_active":"2023-10-26T01:33:41.299142+0000","last_became_peered":"2023-10-26T01:33:41.299142+0000","last_unstale":"2023-10-26T01:33:43.743143+0000","last_undegraded":"2023-10-26T01:33:43.743143+0000","last_fullsized":"2023-10-26T01:33:43.743143+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:25:13.713444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-26T01:33:41.294238+0000","last_change":"2023-10-26T01:33:40.267785+0000","last_active":"2023-10-26T01:33:41.294238+0000","last_peered":"2023-10-26T01:33:41.294238+0000","last_clean":"2023-10-26T01:33:41.294238+0000","last_became_active":"2023-10-26T01:33:40.267136+0000","last_became_peered":"2023-10-26T01:33:40.267136+0000","last_unstale":"2023-10-26T01:33:41.294238+0000","last_undegraded":"2023-10-26T01:33:41.294238+0000","last_fullsized":"2023-10-26T01:33:41.294238+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_clean_scrub_stamp":"2023-10-26T01:33:39.224951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:31:34.704780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-26T01:33:41.295407+0000","last_change":"2023-10-26T01:33:41.295216+0000","last_active":"2023-10-26T01:33:41.295407+0000","last_peered":"2023-10-26T01:33:41.295407+0000","last_clean":"2023-10-26T01:33:41.295407+0000","last_became_active":"2023-10-26T01:33:41.294775+0000","last_became_peered":"2023-10-26T01:33:41.294775+0000","last_unstale":"2023-10-26T01:33:41.295407+0000","last_undegraded":"2023-10-26T01:33:41.295407+0000","last_fullsized":"2023-10-26T01:33:41.295407+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:17:49.911001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-26T01:33:41.280859+0000","last_change":"2023-10-26T01:33:40.267785+0000","last_active":"2023-10-26T01:33:41.280859+0000","last_peered":"2023-10-26T01:33:41.280859+0000","last_clean":"2023-10-26T01:33:41.280859+0000","last_became_active":"2023-10-26T01:33:40.267331+0000","last_became_peered":"2023-10-26T01:33:40.267331+0000","last_unstale":"2023-10-26T01:33:41.280859+0000","last_undegraded":"2023-10-26T01:33:41.280859+0000","last_fullsized":"2023-10-26T01:33:41.280859+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_clean_scrub_stamp":"2023-10-26T01:33:39.224951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:28:43.420820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-26T01:33:42.279600+0000","last_change":"2023-10-26T01:33:41.292961+0000","last_active":"2023-10-26T01:33:42.279600+0000","last_peered":"2023-10-26T01:33:42.279600+0000","last_clean":"2023-10-26T01:33:42.279600+0000","last_became_active":"2023-10-26T01:33:41.292428+0000","last_became_peered":"2023-10-26T01:33:41.292428+0000","last_unstale":"2023-10-26T01:33:42.279600+0000","last_undegraded":"2023-10-26T01:33:42.279600+0000","last_fullsized":"2023-10-26T01:33:42.279600+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:35:44.926447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-26T01:33:41.280006+0000","last_change":"2023-10-26T01:33:40.270251+0000","last_active":"2023-10-26T01:33:41.280006+0000","last_peered":"2023-10-26T01:33:41.280006+0000","last_clean":"2023-10-26T01:33:41.280006+0000","last_became_active":"2023-10-26T01:33:40.269663+0000","last_became_peered":"2023-10-26T01:33:40.269663+0000","last_unstale":"2023-10-26T01:33:41.280006+0000","last_undegraded":"2023-10-26T01:33:41.280006+0000","last_fullsized":"2023-10-26T01:33:41.280006+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_clean_scrub_stamp":"2023-10-26T01:33:39.224951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:21:35.432057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-26T01:33:42.278306+0000","last_change":"2023-10-26T01:33:41.290801+0000","last_active":"2023-10-26T01:33:42.278306+0000","last_peered":"2023-10-26T01:33:42.278306+0000","last_clean":"2023-10-26T01:33:42.278306+0000","last_became_active":"2023-10-26T01:33:41.290225+0000","last_became_peered":"2023-10-26T01:33:41.290225+0000","last_unstale":"2023-10-26T01:33:42.278306+0000","last_undegraded":"2023-10-26T01:33:42.278306+0000","last_fullsized":"2023-10-26T01:33:42.278306+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:22:59.527521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-26T01:33:41.294795+0000","last_change":"2023-10-26T01:33:40.272855+0000","last_active":"2023-10-26T01:33:41.294795+0000","last_peered":"2023-10-26T01:33:41.294795+0000","last_clean":"2023-10-26T01:33:41.294795+0000","last_became_active":"2023-10-26T01:33:40.271480+0000","last_became_peered":"2023-10-26T01:33:40.271480+0000","last_unstale":"2023-10-26T01:33:41.294795+0000","last_undegraded":"2023-10-26T01:33:41.294795+0000","last_fullsized":"2023-10-26T01:33:41.294795+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_clean_scrub_stamp":"2023-10-26T01:33:39.224951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:12:25.980965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-26T01:33:42.274683+0000","last_change":"2023-10-26T01:33:41.302824+0000","last_active":"2023-10-26T01:33:42.274683+0000","last_peered":"2023-10-26T01:33:42.274683+0000","last_clean":"2023-10-26T01:33:42.274683+0000","last_became_active":"2023-10-26T01:33:41.302050+0000","last_became_peered":"2023-10-26T01:33:41.302050+0000","last_unstale":"2023-10-26T01:33:42.274683+0000","last_undegraded":"2023-10-26T01:33:42.274683+0000","last_fullsized":"2023-10-26T01:33:42.274683+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:51:11.300431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"64'3","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-26T01:33:41.322754+0000","last_change":"2023-10-26T01:33:40.264901+0000","last_active":"2023-10-26T01:33:41.322754+0000","last_peered":"2023-10-26T01:33:41.322754+0000","last_clean":"2023-10-26T01:33:41.322754+0000","last_became_active":"2023-10-26T01:33:40.263006+0000","last_became_peered":"2023-10-26T01:33:40.263006+0000","last_unstale":"2023-10-26T01:33:41.322754+0000","last_undegraded":"2023-10-26T01:33:41.322754+0000","last_fullsized":"2023-10-26T01:33:41.322754+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_clean_scrub_stamp":"2023-10-26T01:33:39.224951+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:05:50.003308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-26T01:33:42.278231+0000","last_change":"2023-10-26T01:33:41.293589+0000","last_active":"2023-10-26T01:33:42.278231+0000","last_peered":"2023-10-26T01:33:42.278231+0000","last_clean":"2023-10-26T01:33:42.278231+0000","last_became_active":"2023-10-26T01:33:41.292046+0000","last_became_peered":"2023-10-26T01:33:41.292046+0000","last_unstale":"2023-10-26T01:33:42.278231+0000","last_undegraded":"2023-10-26T01:33:42.278231+0000","last_fullsized":"2023-10-26T01:33:42.278231+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:56:20.883631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"64'6","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-26T01:33:41.321825+0000","last_change":"2023-10-26T01:33:40.262090+0000","last_active":"2023-10-26T01:33:41.321825+0000","last_peered":"2023-10-26T01:33:41.321825+0000","last_clean":"2023-10-26T01:33:41.321825+0000","last_became_active":"2023-10-26T01:33:40.261736+0000","last_became_peered":"2023-10-26T01:33:40.261736+0000","last_unstale":"2023-10-26T01:33:41.321825+0000","last_undegraded":"2023-10-26T01:33:41.321825+0000","last_fullsized":"2023-10-26T01:33:41.321825+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_clean_scrub_stamp":"2023-10-26T01:33:39.224951+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:30:06.226887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-26T01:33:42.573386+0000","last_change":"2023-10-26T01:33:41.308486+0000","last_active":"2023-10-26T01:33:42.573386+0000","last_peered":"2023-10-26T01:33:42.573386+0000","last_clean":"2023-10-26T01:33:42.573386+0000","last_became_active":"2023-10-26T01:33:41.308202+0000","last_became_peered":"2023-10-26T01:33:41.308202+0000","last_unstale":"2023-10-26T01:33:42.573386+0000","last_undegraded":"2023-10-26T01:33:42.573386+0000","last_fullsized":"2023-10-26T01:33:42.573386+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:04:15.104623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"64'2","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-26T01:33:41.318445+0000","last_change":"2023-10-26T01:33:40.269670+0000","last_active":"2023-10-26T01:33:41.318445+0000","last_peered":"2023-10-26T01:33:41.318445+0000","last_clean":"2023-10-26T01:33:41.318445+0000","last_became_active":"2023-10-26T01:33:40.267307+0000","last_became_peered":"2023-10-26T01:33:40.267307+0000","last_unstale":"2023-10-26T01:33:41.318445+0000","last_undegraded":"2023-10-26T01:33:41.318445+0000","last_fullsized":"2023-10-26T01:33:41.318445+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_clean_scrub_stamp":"2023-10-26T01:33:39.224951+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:37:16.574662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-26T01:33:42.277958+0000","last_change":"2023-10-26T01:33:41.296317+0000","last_active":"2023-10-26T01:33:42.277958+0000","last_peered":"2023-10-26T01:33:42.277958+0000","last_clean":"2023-10-26T01:33:42.277958+0000","last_became_active":"2023-10-26T01:33:41.293890+0000","last_became_peered":"2023-10-26T01:33:41.293890+0000","last_unstale":"2023-10-26T01:33:42.277958+0000","last_undegraded":"2023-10-26T01:33:42.277958+0000","last_fullsized":"2023-10-26T01:33:42.277958+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:17:49.911001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-26T01:33:41.289710+0000","last_change":"2023-10-26T01:33:40.265574+0000","last_active":"2023-10-26T01:33:41.289710+0000","last_peered":"2023-10-26T01:33:41.289710+0000","last_clean":"2023-10-26T01:33:41.289710+0000","last_became_active":"2023-10-26T01:33:40.265261+0000","last_became_peered":"2023-10-26T01:33:40.265261+0000","last_unstale":"2023-10-26T01:33:41.289710+0000","last_undegraded":"2023-10-26T01:33:41.289710+0000","last_fullsized":"2023-10-26T01:33:41.289710+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_clean_scrub_stamp":"2023-10-26T01:33:39.224951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:03:06.315831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-26T01:33:42.277534+0000","last_change":"2023-10-26T01:33:41.295276+0000","last_active":"2023-10-26T01:33:42.277534+0000","last_peered":"2023-10-26T01:33:42.277534+0000","last_clean":"2023-10-26T01:33:42.277534+0000","last_became_active":"2023-10-26T01:33:41.294456+0000","last_became_peered":"2023-10-26T01:33:41.294456+0000","last_unstale":"2023-10-26T01:33:42.277534+0000","last_undegraded":"2023-10-26T01:33:42.277534+0000","last_fullsized":"2023-10-26T01:33:42.277534+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:44:24.727759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-26T01:33:41.289129+0000","last_change":"2023-10-26T01:33:40.268967+0000","last_active":"2023-10-26T01:33:41.289129+0000","last_peered":"2023-10-26T01:33:41.289129+0000","last_clean":"2023-10-26T01:33:41.289129+0000","last_became_active":"2023-10-26T01:33:40.268346+0000","last_became_peered":"2023-10-26T01:33:40.268346+0000","last_unstale":"2023-10-26T01:33:41.289129+0000","last_undegraded":"2023-10-26T01:33:41.289129+0000","last_fullsized":"2023-10-26T01:33:41.289129+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_clean_scrub_stamp":"2023-10-26T01:33:39.224951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:59:57.719590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-26T01:33:42.277329+0000","last_change":"2023-10-26T01:33:41.300326+0000","last_active":"2023-10-26T01:33:42.277329+0000","last_peered":"2023-10-26T01:33:42.277329+0000","last_clean":"2023-10-26T01:33:42.277329+0000","last_became_active":"2023-10-26T01:33:41.297883+0000","last_became_peered":"2023-10-26T01:33:41.297883+0000","last_unstale":"2023-10-26T01:33:42.277329+0000","last_undegraded":"2023-10-26T01:33:42.277329+0000","last_fullsized":"2023-10-26T01:33:42.277329+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:39:51.963639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"64'8","reported_seq":20,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-26T01:33:41.321999+0000","last_change":"2023-10-26T01:33:40.270644+0000","last_active":"2023-10-26T01:33:41.321999+0000","last_peered":"2023-10-26T01:33:41.321999+0000","last_clean":"2023-10-26T01:33:41.321999+0000","last_became_active":"2023-10-26T01:33:40.270292+0000","last_became_peered":"2023-10-26T01:33:40.270292+0000","last_unstale":"2023-10-26T01:33:41.321999+0000","last_undegraded":"2023-10-26T01:33:41.321999+0000","last_fullsized":"2023-10-26T01:33:41.321999+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_clean_scrub_stamp":"2023-10-26T01:33:39.224951+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:04:14.094026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-26T01:33:42.278322+0000","last_change":"2023-10-26T01:33:41.297108+0000","last_active":"2023-10-26T01:33:42.278322+0000","last_peered":"2023-10-26T01:33:42.278322+0000","last_clean":"2023-10-26T01:33:42.278322+0000","last_became_active":"2023-10-26T01:33:41.295739+0000","last_became_peered":"2023-10-26T01:33:41.295739+0000","last_unstale":"2023-10-26T01:33:42.278322+0000","last_undegraded":"2023-10-26T01:33:42.278322+0000","last_fullsized":"2023-10-26T01:33:42.278322+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:31:51.222010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"64'1","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-26T01:33:41.318330+0000","last_change":"2023-10-26T01:33:40.274622+0000","last_active":"2023-10-26T01:33:41.318330+0000","last_peered":"2023-10-26T01:33:41.318330+0000","last_clean":"2023-10-26T01:33:41.318330+0000","last_became_active":"2023-10-26T01:33:40.271512+0000","last_became_peered":"2023-10-26T01:33:40.271512+0000","last_unstale":"2023-10-26T01:33:41.318330+0000","last_undegraded":"2023-10-26T01:33:41.318330+0000","last_fullsized":"2023-10-26T01:33:41.318330+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_clean_scrub_stamp":"2023-10-26T01:33:39.224951+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:31:50.211414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-26T01:33:42.573424+0000","last_change":"2023-10-26T01:33:41.306746+0000","last_active":"2023-10-26T01:33:42.573424+0000","last_peered":"2023-10-26T01:33:42.573424+0000","last_clean":"2023-10-26T01:33:42.573424+0000","last_became_active":"2023-10-26T01:33:41.306453+0000","last_became_peered":"2023-10-26T01:33:41.306453+0000","last_unstale":"2023-10-26T01:33:42.573424+0000","last_undegraded":"2023-10-26T01:33:42.573424+0000","last_fullsized":"2023-10-26T01:33:42.573424+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:05:51.013904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"64'2","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-26T01:33:41.319317+0000","last_change":"2023-10-26T01:33:40.271394+0000","last_active":"2023-10-26T01:33:41.319317+0000","last_peered":"2023-10-26T01:33:41.319317+0000","last_clean":"2023-10-26T01:33:41.319317+0000","last_became_active":"2023-10-26T01:33:40.271150+0000","last_became_peered":"2023-10-26T01:33:40.271150+0000","last_unstale":"2023-10-26T01:33:41.319317+0000","last_undegraded":"2023-10-26T01:33:41.319317+0000","last_fullsized":"2023-10-26T01:33:41.319317+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_clean_scrub_stamp":"2023-10-26T01:33:39.224951+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:06:52.876612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-26T01:33:41.308095+0000","last_change":"2023-10-26T01:33:41.305337+0000","last_active":"2023-10-26T01:33:41.308095+0000","last_peered":"2023-10-26T01:33:41.308095+0000","last_clean":"2023-10-26T01:33:41.308095+0000","last_became_active":"2023-10-26T01:33:41.305082+0000","last_became_peered":"2023-10-26T01:33:41.305082+0000","last_unstale":"2023-10-26T01:33:41.308095+0000","last_undegraded":"2023-10-26T01:33:41.308095+0000","last_fullsized":"2023-10-26T01:33:41.308095+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:15:58.724118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"64'1","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-26T01:33:41.317885+0000","last_change":"2023-10-26T01:33:40.267963+0000","last_active":"2023-10-26T01:33:41.317885+0000","last_peered":"2023-10-26T01:33:41.317885+0000","last_clean":"2023-10-26T01:33:41.317885+0000","last_became_active":"2023-10-26T01:33:40.267288+0000","last_became_peered":"2023-10-26T01:33:40.267288+0000","last_unstale":"2023-10-26T01:33:41.317885+0000","last_undegraded":"2023-10-26T01:33:41.317885+0000","last_fullsized":"2023-10-26T01:33:41.317885+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_clean_scrub_stamp":"2023-10-26T01:33:39.224951+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T01:42:21.460059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-26T01:33:42.278763+0000","last_change":"2023-10-26T01:33:41.290595+0000","last_active":"2023-10-26T01:33:42.278763+0000","last_peered":"2023-10-26T01:33:42.278763+0000","last_clean":"2023-10-26T01:33:42.278763+0000","last_became_active":"2023-10-26T01:33:41.289746+0000","last_became_peered":"2023-10-26T01:33:41.289746+0000","last_unstale":"2023-10-26T01:33:42.278763+0000","last_undegraded":"2023-10-26T01:33:42.278763+0000","last_fullsized":"2023-10-26T01:33:42.278763+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:31:53.157442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-26T01:33:41.274231+0000","last_change":"2023-10-26T01:33:40.269822+0000","last_active":"2023-10-26T01:33:41.274231+0000","last_peered":"2023-10-26T01:33:41.274231+0000","last_clean":"2023-10-26T01:33:41.274231+0000","last_became_active":"2023-10-26T01:33:40.267425+0000","last_became_peered":"2023-10-26T01:33:40.267425+0000","last_unstale":"2023-10-26T01:33:41.274231+0000","last_undegraded":"2023-10-26T01:33:41.274231+0000","last_fullsized":"2023-10-26T01:33:41.274231+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_clean_scrub_stamp":"2023-10-26T01:33:39.224951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:08:41.813075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-26T01:33:42.573967+0000","last_change":"2023-10-26T01:33:41.311766+0000","last_active":"2023-10-26T01:33:42.573967+0000","last_peered":"2023-10-26T01:33:42.573967+0000","last_clean":"2023-10-26T01:33:42.573967+0000","last_became_active":"2023-10-26T01:33:41.311518+0000","last_became_peered":"2023-10-26T01:33:41.311518+0000","last_unstale":"2023-10-26T01:33:42.573967+0000","last_undegraded":"2023-10-26T01:33:42.573967+0000","last_fullsized":"2023-10-26T01:33:42.573967+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:01:43.265707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"64'2","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-26T01:33:41.317710+0000","last_change":"2023-10-26T01:33:40.262402+0000","last_active":"2023-10-26T01:33:41.317710+0000","last_peered":"2023-10-26T01:33:41.317710+0000","last_clean":"2023-10-26T01:33:41.317710+0000","last_became_active":"2023-10-26T01:33:40.262031+0000","last_became_peered":"2023-10-26T01:33:40.262031+0000","last_unstale":"2023-10-26T01:33:41.317710+0000","last_undegraded":"2023-10-26T01:33:41.317710+0000","last_fullsized":"2023-10-26T01:33:41.317710+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_clean_scrub_stamp":"2023-10-26T01:33:39.224951+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:58:10.641484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-26T01:33:42.571978+0000","last_change":"2023-10-26T01:33:41.295412+0000","last_active":"2023-10-26T01:33:42.571978+0000","last_peered":"2023-10-26T01:33:42.571978+0000","last_clean":"2023-10-26T01:33:42.571978+0000","last_became_active":"2023-10-26T01:33:41.294924+0000","last_became_peered":"2023-10-26T01:33:41.294924+0000","last_unstale":"2023-10-26T01:33:42.571978+0000","last_undegraded":"2023-10-26T01:33:42.571978+0000","last_fullsized":"2023-10-26T01:33:42.571978+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:04:30.189571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-26T01:33:41.290794+0000","last_change":"2023-10-26T01:33:40.270401+0000","last_active":"2023-10-26T01:33:41.290794+0000","last_peered":"2023-10-26T01:33:41.290794+0000","last_clean":"2023-10-26T01:33:41.290794+0000","last_became_active":"2023-10-26T01:33:40.270199+0000","last_became_peered":"2023-10-26T01:33:40.270199+0000","last_unstale":"2023-10-26T01:33:41.290794+0000","last_undegraded":"2023-10-26T01:33:41.290794+0000","last_fullsized":"2023-10-26T01:33:41.290794+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_clean_scrub_stamp":"2023-10-26T01:33:39.224951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:20:21.803277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-26T01:33:42.571815+0000","last_change":"2023-10-26T01:33:41.284431+0000","last_active":"2023-10-26T01:33:42.571815+0000","last_peered":"2023-10-26T01:33:42.571815+0000","last_clean":"2023-10-26T01:33:42.571815+0000","last_became_active":"2023-10-26T01:33:41.283694+0000","last_became_peered":"2023-10-26T01:33:41.283694+0000","last_unstale":"2023-10-26T01:33:42.571815+0000","last_undegraded":"2023-10-26T01:33:42.571815+0000","last_fullsized":"2023-10-26T01:33:42.571815+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:15:58.724118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-26T01:33:41.292217+0000","last_change":"2023-10-26T01:33:40.263474+0000","last_active":"2023-10-26T01:33:41.292217+0000","last_peered":"2023-10-26T01:33:41.292217+0000","last_clean":"2023-10-26T01:33:41.292217+0000","last_became_active":"2023-10-26T01:33:40.263130+0000","last_became_peered":"2023-10-26T01:33:40.263130+0000","last_unstale":"2023-10-26T01:33:41.292217+0000","last_undegraded":"2023-10-26T01:33:41.292217+0000","last_fullsized":"2023-10-26T01:33:41.292217+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_clean_scrub_stamp":"2023-10-26T01:33:39.224951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:48:18.445601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-26T01:33:42.573269+0000","last_change":"2023-10-26T01:33:41.307472+0000","last_active":"2023-10-26T01:33:42.573269+0000","last_peered":"2023-10-26T01:33:42.573269+0000","last_clean":"2023-10-26T01:33:42.573269+0000","last_became_active":"2023-10-26T01:33:41.307159+0000","last_became_peered":"2023-10-26T01:33:41.307159+0000","last_unstale":"2023-10-26T01:33:42.573269+0000","last_undegraded":"2023-10-26T01:33:42.573269+0000","last_fullsized":"2023-10-26T01:33:42.573269+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:43:29.684443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"64'2","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-26T01:33:41.318358+0000","last_change":"2023-10-26T01:33:40.268525+0000","last_active":"2023-10-26T01:33:41.318358+0000","last_peered":"2023-10-26T01:33:41.318358+0000","last_clean":"2023-10-26T01:33:41.318358+0000","last_became_active":"2023-10-26T01:33:40.267115+0000","last_became_peered":"2023-10-26T01:33:40.267115+0000","last_unstale":"2023-10-26T01:33:41.318358+0000","last_undegraded":"2023-10-26T01:33:41.318358+0000","last_fullsized":"2023-10-26T01:33:41.318358+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_clean_scrub_stamp":"2023-10-26T01:33:39.224951+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:21:04.695200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-26T01:33:42.572750+0000","last_change":"2023-10-26T01:33:41.297371+0000","last_active":"2023-10-26T01:33:42.572750+0000","last_peered":"2023-10-26T01:33:42.572750+0000","last_clean":"2023-10-26T01:33:42.572750+0000","last_became_active":"2023-10-26T01:33:41.296828+0000","last_became_peered":"2023-10-26T01:33:41.296828+0000","last_unstale":"2023-10-26T01:33:42.572750+0000","last_undegraded":"2023-10-26T01:33:42.572750+0000","last_fullsized":"2023-10-26T01:33:42.572750+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:58:48.148959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"65'1","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-26T01:33:42.572877+0000","last_change":"2023-10-26T01:33:40.269043+0000","last_active":"2023-10-26T01:33:42.572877+0000","last_peered":"2023-10-26T01:33:42.572877+0000","last_clean":"2023-10-26T01:33:42.572877+0000","last_became_active":"2023-10-26T01:33:40.268478+0000","last_became_peered":"2023-10-26T01:33:40.268478+0000","last_unstale":"2023-10-26T01:33:42.572877+0000","last_undegraded":"2023-10-26T01:33:42.572877+0000","last_fullsized":"2023-10-26T01:33:42.572877+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_clean_scrub_stamp":"2023-10-26T01:33:39.224951+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:58:10.641484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-26T01:33:42.275306+0000","last_change":"2023-10-26T01:33:41.296176+0000","last_active":"2023-10-26T01:33:42.275306+0000","last_peered":"2023-10-26T01:33:42.275306+0000","last_clean":"2023-10-26T01:33:42.275306+0000","last_became_active":"2023-10-26T01:33:41.295812+0000","last_became_peered":"2023-10-26T01:33:41.295812+0000","last_unstale":"2023-10-26T01:33:42.275306+0000","last_undegraded":"2023-10-26T01:33:42.275306+0000","last_fullsized":"2023-10-26T01:33:42.275306+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:25:13.713444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-26T01:33:41.287917+0000","last_change":"2023-10-26T01:33:40.264477+0000","last_active":"2023-10-26T01:33:41.287917+0000","last_peered":"2023-10-26T01:33:41.287917+0000","last_clean":"2023-10-26T01:33:41.287917+0000","last_became_active":"2023-10-26T01:33:40.264087+0000","last_became_peered":"2023-10-26T01:33:40.264087+0000","last_unstale":"2023-10-26T01:33:41.287917+0000","last_undegraded":"2023-10-26T01:33:41.287917+0000","last_fullsized":"2023-10-26T01:33:41.287917+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_clean_scrub_stamp":"2023-10-26T01:33:39.224951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:58:12.392348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-26T01:33:42.572197+0000","last_change":"2023-10-26T01:33:41.303134+0000","last_active":"2023-10-26T01:33:42.572197+0000","last_peered":"2023-10-26T01:33:42.572197+0000","last_clean":"2023-10-26T01:33:42.572197+0000","last_became_active":"2023-10-26T01:33:41.302907+0000","last_became_peered":"2023-10-26T01:33:41.302907+0000","last_unstale":"2023-10-26T01:33:42.572197+0000","last_undegraded":"2023-10-26T01:33:42.572197+0000","last_fullsized":"2023-10-26T01:33:42.572197+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:00:23.198647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-26T01:33:41.284782+0000","last_change":"2023-10-26T01:33:40.262786+0000","last_active":"2023-10-26T01:33:41.284782+0000","last_peered":"2023-10-26T01:33:41.284782+0000","last_clean":"2023-10-26T01:33:41.284782+0000","last_became_active":"2023-10-26T01:33:40.262305+0000","last_became_peered":"2023-10-26T01:33:40.262305+0000","last_unstale":"2023-10-26T01:33:41.284782+0000","last_undegraded":"2023-10-26T01:33:41.284782+0000","last_fullsized":"2023-10-26T01:33:41.284782+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_clean_scrub_stamp":"2023-10-26T01:33:39.224951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:12:10.197581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-26T01:33:41.309299+0000","last_change":"2023-10-26T01:33:41.308913+0000","last_active":"2023-10-26T01:33:41.309299+0000","last_peered":"2023-10-26T01:33:41.309299+0000","last_clean":"2023-10-26T01:33:41.309299+0000","last_became_active":"2023-10-26T01:33:41.307785+0000","last_became_peered":"2023-10-26T01:33:41.307785+0000","last_unstale":"2023-10-26T01:33:41.309299+0000","last_undegraded":"2023-10-26T01:33:41.309299+0000","last_fullsized":"2023-10-26T01:33:41.309299+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T01:39:22.276139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-26T01:33:41.286571+0000","last_change":"2023-10-26T01:33:40.266077+0000","last_active":"2023-10-26T01:33:41.286571+0000","last_peered":"2023-10-26T01:33:41.286571+0000","last_clean":"2023-10-26T01:33:41.286571+0000","last_became_active":"2023-10-26T01:33:40.265759+0000","last_became_peered":"2023-10-26T01:33:40.265759+0000","last_unstale":"2023-10-26T01:33:41.286571+0000","last_undegraded":"2023-10-26T01:33:41.286571+0000","last_fullsized":"2023-10-26T01:33:41.286571+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_clean_scrub_stamp":"2023-10-26T01:33:39.224951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T01:37:34.975915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-26T01:33:42.279072+0000","last_change":"2023-10-26T01:33:41.300235+0000","last_active":"2023-10-26T01:33:42.279072+0000","last_peered":"2023-10-26T01:33:42.279072+0000","last_clean":"2023-10-26T01:33:42.279072+0000","last_became_active":"2023-10-26T01:33:41.297898+0000","last_became_peered":"2023-10-26T01:33:41.297898+0000","last_unstale":"2023-10-26T01:33:42.279072+0000","last_undegraded":"2023-10-26T01:33:42.279072+0000","last_fullsized":"2023-10-26T01:33:42.279072+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:23:14.596280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-26T01:33:41.279096+0000","last_change":"2023-10-26T01:33:40.264127+0000","last_active":"2023-10-26T01:33:41.279096+0000","last_peered":"2023-10-26T01:33:41.279096+0000","last_clean":"2023-10-26T01:33:41.279096+0000","last_became_active":"2023-10-26T01:33:40.263771+0000","last_became_peered":"2023-10-26T01:33:40.263771+0000","last_unstale":"2023-10-26T01:33:41.279096+0000","last_undegraded":"2023-10-26T01:33:41.279096+0000","last_fullsized":"2023-10-26T01:33:41.279096+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_clean_scrub_stamp":"2023-10-26T01:33:39.224951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:59:57.719590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-26T01:33:42.274831+0000","last_change":"2023-10-26T01:33:41.311096+0000","last_active":"2023-10-26T01:33:42.274831+0000","last_peered":"2023-10-26T01:33:42.274831+0000","last_clean":"2023-10-26T01:33:42.274831+0000","last_became_active":"2023-10-26T01:33:41.309654+0000","last_became_peered":"2023-10-26T01:33:41.309654+0000","last_unstale":"2023-10-26T01:33:42.274831+0000","last_undegraded":"2023-10-26T01:33:42.274831+0000","last_fullsized":"2023-10-26T01:33:42.274831+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:58:13.402945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-26T01:33:41.283733+0000","last_change":"2023-10-26T01:33:40.270109+0000","last_active":"2023-10-26T01:33:41.283733+0000","last_peered":"2023-10-26T01:33:41.283733+0000","last_clean":"2023-10-26T01:33:41.283733+0000","last_became_active":"2023-10-26T01:33:40.269898+0000","last_became_peered":"2023-10-26T01:33:40.269898+0000","last_unstale":"2023-10-26T01:33:41.283733+0000","last_undegraded":"2023-10-26T01:33:41.283733+0000","last_fullsized":"2023-10-26T01:33:41.283733+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_clean_scrub_stamp":"2023-10-26T01:33:39.224951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:22:31.617818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-26T01:33:41.299531+0000","last_change":"2023-10-26T01:33:41.299046+0000","last_active":"2023-10-26T01:33:41.299531+0000","last_peered":"2023-10-26T01:33:41.299531+0000","last_clean":"2023-10-26T01:33:41.299531+0000","last_became_active":"2023-10-26T01:33:41.298329+0000","last_became_peered":"2023-10-26T01:33:41.298329+0000","last_unstale":"2023-10-26T01:33:41.299531+0000","last_undegraded":"2023-10-26T01:33:41.299531+0000","last_fullsized":"2023-10-26T01:33:41.299531+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:08:42.823671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-26T01:33:41.285826+0000","last_change":"2023-10-26T01:33:40.274518+0000","last_active":"2023-10-26T01:33:41.285826+0000","last_peered":"2023-10-26T01:33:41.285826+0000","last_clean":"2023-10-26T01:33:41.285826+0000","last_became_active":"2023-10-26T01:33:40.271103+0000","last_became_peered":"2023-10-26T01:33:40.271103+0000","last_unstale":"2023-10-26T01:33:41.285826+0000","last_undegraded":"2023-10-26T01:33:41.285826+0000","last_fullsized":"2023-10-26T01:33:41.285826+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_clean_scrub_stamp":"2023-10-26T01:33:39.224951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:31:52.146845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-26T01:33:41.308053+0000","last_change":"2023-10-26T01:33:41.305817+0000","last_active":"2023-10-26T01:33:41.308053+0000","last_peered":"2023-10-26T01:33:41.308053+0000","last_clean":"2023-10-26T01:33:41.308053+0000","last_became_active":"2023-10-26T01:33:41.305585+0000","last_became_peered":"2023-10-26T01:33:41.305585+0000","last_unstale":"2023-10-26T01:33:41.308053+0000","last_undegraded":"2023-10-26T01:33:41.308053+0000","last_fullsized":"2023-10-26T01:33:41.308053+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:40.235547+0000","last_clean_scrub_stamp":"2023-10-26T01:33:40.235547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:14:31.511712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-26T01:33:41.289216+0000","last_change":"2023-10-26T01:33:40.268720+0000","last_active":"2023-10-26T01:33:41.289216+0000","last_peered":"2023-10-26T01:33:41.289216+0000","last_clean":"2023-10-26T01:33:41.289216+0000","last_became_active":"2023-10-26T01:33:40.268390+0000","last_became_peered":"2023-10-26T01:33:40.268390+0000","last_unstale":"2023-10-26T01:33:41.289216+0000","last_undegraded":"2023-10-26T01:33:41.289216+0000","last_fullsized":"2023-10-26T01:33:41.289216+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:33:39.224951+0000","last_clean_scrub_stamp":"2023-10-26T01:33:39.224951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:16:22.976409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":209,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-26T01:33:41.275139+0000","last_change":"2023-10-26T01:30:34.883484+0000","last_active":"2023-10-26T01:33:41.275139+0000","last_peered":"2023-10-26T01:33:41.275139+0000","last_clean":"2023-10-26T01:33:41.275139+0000","last_became_active":"2023-10-26T01:30:34.883071+0000","last_became_peered":"2023-10-26T01:30:34.883071+0000","last_unstale":"2023-10-26T01:33:41.275139+0000","last_undegraded":"2023-10-26T01:33:41.275139+0000","last_fullsized":"2023-10-26T01:33:41.275139+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:32.890096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:32.890096+0000","last_clean_scrub_stamp":"2023-10-26T01:30:32.890096+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:00:50.687712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":14,"seq":60129542185,"num_pgs":23,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":318,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194994,"statfs":{"total":200000000,"available":199674276,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:33:40 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.79000000000000004,"15min":0.78400000000000003},"min":{"1min":0.52700000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.1579999999999999,"5min":2.2549999999999999,"15min":2.2549999999999999},"last":1.046},{"interface":"front","average":{"1min":0.748,"5min":0.84699999999999998,"15min":0.88800000000000001},"min":{"1min":0.441,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.2050000000000001,"5min":1.8200000000000001,"15min":1.8200000000000001},"last":0.76600000000000001}]},{"osd":1,"last update":"Thu Oct 26 01:33:40 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.90300000000000002,"15min":0.96599999999999997},"min":{"1min":0.48599999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.2450000000000001,"5min":4.4340000000000002,"15min":4.4340000000000002},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.97599999999999998,"15min":1.004},"min":{"1min":0.56599999999999995,"5min":0.374,"15min":0.374},"max":{"1min":1.4570000000000001,"5min":2.3330000000000002,"15min":2.3330000000000002},"last":0.876}]},{"osd":2,"last update":"Thu Oct 26 01:33:40 2023","interfaces":[{"interface":"back","average":{"1min":1.0669999999999999,"5min":1.0680000000000001,"15min":1.1000000000000001},"min":{"1min":0.64700000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.5509999999999999,"5min":4.2800000000000002,"15min":4.2800000000000002},"last":1.1619999999999999},{"interface":"front","average":{"1min":1.087,"5min":1.071,"15min":1.075},"min":{"1min":0.59599999999999997,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.502,"5min":2.302,"15min":2.302},"last":0.97599999999999998}]},{"osd":4,"last update":"Thu Oct 26 01:33:40 2023","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.145,"15min":1.1479999999999999},"min":{"1min":0.77100000000000002,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.5660000000000001,"5min":2.1070000000000002,"15min":2.1070000000000002},"last":1.121},{"interface":"front","average":{"1min":1.3280000000000001,"5min":1.3720000000000001,"15min":1.413},"min":{"1min":1.0309999999999999,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":1.8160000000000001,"5min":4.8250000000000002,"15min":4.8250000000000002},"last":1.4139999999999999}]},{"osd":5,"last update":"Thu Oct 26 01:33:40 2023","interfaces":[{"interface":"back","average":{"1min":1.248,"5min":1.3100000000000001,"15min":1.3580000000000001},"min":{"1min":0.83199999999999996,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.631,"5min":4.9080000000000004,"15min":4.9080000000000004},"last":1.3500000000000001},{"interface":"front","average":{"1min":1.246,"5min":1.26,"15min":1.274},"min":{"1min":0.80900000000000005,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.8919999999999999,"5min":2.8980000000000001,"15min":2.8980000000000001},"last":1.2090000000000001}]},{"osd":6,"last update":"Thu Oct 26 01:33:40 2023","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.3149999999999999,"15min":1.319},"min":{"1min":0.96499999999999997,"5min":0.85099999999999998,"15min":0.85099999999999998},"max":{"1min":1.823,"5min":2.8530000000000002,"15min":2.8530000000000002},"last":1.3839999999999999},{"interface":"front","average":{"1min":1.177,"5min":1.236,"15min":1.264},"min":{"1min":0.81499999999999995,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.784,"5min":2.181,"15min":2.181},"last":1.1950000000000001}]},{"osd":7,"last update":"Thu Oct 26 01:33:40 2023","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.2689999999999999,"15min":1.28},"min":{"1min":0.85199999999999998,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.891,"5min":2.0680000000000001,"15min":2.0680000000000001},"last":1.3120000000000001},{"interface":"front","average":{"1min":1.381,"5min":1.3899999999999999,"15min":1.4299999999999999},"min":{"1min":0.89500000000000002,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.698,"5min":4.8730000000000002,"15min":4.8730000000000002},"last":1.2829999999999999}]}]},{"osd":7,"up_from":13,"seq":55834574889,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":900,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194411,"statfs":{"total":200000000,"available":199077504,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:33:40 2023","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.0900000000000001,"15min":1.073},"min":{"1min":0.72799999999999998,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.9410000000000001,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":0.94599999999999995},{"interface":"front","average":{"1min":1.123,"5min":1.093,"15min":1.0629999999999999},"min":{"1min":0.73599999999999999,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.6220000000000001,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":1.0029999999999999}]},{"osd":1,"last update":"Thu Oct 26 01:33:40 2023","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":1.103,"15min":1.0569999999999999},"min":{"1min":0.78200000000000003,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":2.0510000000000002,"5min":2.0510000000000002,"15min":2.0510000000000002},"last":1.0740000000000001},{"interface":"front","average":{"1min":1.2270000000000001,"5min":1.2090000000000001,"15min":1.1859999999999999},"min":{"1min":0.751,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.681,"5min":2.1720000000000002,"15min":2.1720000000000002},"last":1.1970000000000001}]},{"osd":2,"last update":"Thu Oct 26 01:33:40 2023","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":1.095,"15min":1.087},"min":{"1min":0.65700000000000003,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.53,"5min":1.6919999999999999,"15min":1.6919999999999999},"last":1.238},{"interface":"front","average":{"1min":1.2709999999999999,"5min":1.228,"15min":1.2130000000000001},"min":{"1min":0.81699999999999995,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":2.181,"5min":2.181,"15min":2.181},"last":1.2549999999999999}]},{"osd":3,"last update":"Thu Oct 26 01:33:44 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.1970000000000001,"15min":1.1830000000000001},"min":{"1min":0.871,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.609,"5min":1.8100000000000001,"15min":1.8100000000000001},"last":1.129},{"interface":"front","average":{"1min":1.3939999999999999,"5min":1.3620000000000001,"15min":1.3580000000000001},"min":{"1min":1.0169999999999999,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.7250000000000001,"5min":1.9319999999999999,"15min":1.9319999999999999},"last":1.448}]},{"osd":4,"last update":"Thu Oct 26 01:33:40 2023","interfaces":[{"interface":"back","average":{"1min":1.331,"5min":1.337,"15min":1.3420000000000001},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.5880000000000001,"5min":2.008,"15min":2.008},"last":1.292},{"interface":"front","average":{"1min":1.47,"5min":1.409,"15min":1.389},"min":{"1min":0.96399999999999997,"5min":0.496,"15min":0.496},"max":{"1min":2.6619999999999999,"5min":2.6619999999999999,"15min":2.6619999999999999},"last":1.6639999999999999}]},{"osd":5,"last update":"Thu Oct 26 01:33:40 2023","interfaces":[{"interface":"back","average":{"1min":1.5389999999999999,"5min":1.425,"15min":1.403},"min":{"1min":0.97799999999999998,"5min":0.441,"15min":0.441},"max":{"1min":2.492,"5min":2.492,"15min":2.492},"last":1.3580000000000001},{"interface":"front","average":{"1min":1.506,"5min":1.46,"15min":1.4590000000000001},"min":{"1min":1.0920000000000001,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.7450000000000001,"5min":1.911,"15min":1.911},"last":1.401}]},{"osd":6,"last update":"Thu Oct 26 01:33:40 2023","interfaces":[{"interface":"back","average":{"1min":1.5940000000000001,"5min":1.833,"15min":1.9530000000000001},"min":{"1min":1.1299999999999999,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":2.0099999999999998,"5min":11.577999999999999,"15min":11.577999999999999},"last":1.5580000000000001},{"interface":"front","average":{"1min":1.641,"5min":1.835,"15min":1.946},"min":{"1min":1.2789999999999999,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":2.4489999999999998,"5min":10.704000000000001,"15min":10.704000000000001},"last":1.514}]}]},{"osd":6,"up_from":13,"seq":55834574889,"num_pgs":17,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":323,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194988,"statfs":{"total":200000000,"available":199668646,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:33:37 2023","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":1.1220000000000001,"15min":1.1679999999999999},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.5069999999999999,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":0.64500000000000002},{"interface":"front","average":{"1min":1.133,"5min":1.081,"15min":1.1040000000000001},"min":{"1min":0.68600000000000005,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.5620000000000001,"5min":1.843,"15min":1.843},"last":0.86099999999999999}]},{"osd":1,"last update":"Thu Oct 26 01:33:37 2023","interfaces":[{"interface":"back","average":{"1min":1.081,"5min":1.1299999999999999,"15min":1.1599999999999999},"min":{"1min":0.71599999999999997,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.6910000000000001,"5min":1.7150000000000001,"15min":1.7150000000000001},"last":0.77900000000000003},{"interface":"front","average":{"1min":1.024,"5min":1.1659999999999999,"15min":1.206},"min":{"1min":0.67700000000000005,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.4870000000000001,"5min":2.085,"15min":2.085},"last":0.98199999999999998}]},{"osd":2,"last update":"Thu Oct 26 01:33:37 2023","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.0680000000000001,"15min":1.0569999999999999},"min":{"1min":0.84099999999999997,"5min":0.747,"15min":0.747},"max":{"1min":1.6419999999999999,"5min":1.6659999999999999,"15min":1.6659999999999999},"last":0.93300000000000005},{"interface":"front","average":{"1min":1.222,"5min":1.2390000000000001,"15min":1.2450000000000001},"min":{"1min":0.84499999999999997,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.8899999999999999,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":1.1739999999999999}]},{"osd":3,"last update":"Thu Oct 26 01:33:37 2023","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.173,"15min":1.1619999999999999},"min":{"1min":0.74099999999999999,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.764,"5min":1.764,"15min":1.764},"last":1.1419999999999999},{"interface":"front","average":{"1min":1.2989999999999999,"5min":1.3140000000000001,"15min":1.3180000000000001},"min":{"1min":1.0589999999999999,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.8029999999999999,"5min":1.881,"15min":1.881},"last":1.032}]},{"osd":4,"last update":"Thu Oct 26 01:33:37 2023","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.216,"15min":1.208},"min":{"1min":0.92900000000000005,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.845,"5min":2.089,"15min":2.089},"last":1.504},{"interface":"front","average":{"1min":1.3640000000000001,"5min":1.2829999999999999,"15min":1.264},"min":{"1min":0.92300000000000004,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.704,"5min":2.4129999999999998,"15min":2.4129999999999998},"last":1.2250000000000001}]},{"osd":5,"last update":"Thu Oct 26 01:33:37 2023","interfaces":[{"interface":"back","average":{"1min":1.508,"5min":1.427,"15min":1.403},"min":{"1min":1.1559999999999999,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.843,"5min":2.1640000000000001,"15min":2.1640000000000001},"last":1.2709999999999999},{"interface":"front","average":{"1min":1.4339999999999999,"5min":1.4139999999999999,"15min":1.3979999999999999},"min":{"1min":0.97199999999999998,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.9470000000000001,"5min":2.0710000000000002,"15min":2.0710000000000002},"last":1.294}]},{"osd":7,"last update":"Thu Oct 26 01:33:37 2023","interfaces":[{"interface":"back","average":{"1min":1.4670000000000001,"5min":1.377,"15min":1.3620000000000001},"min":{"1min":1.0009999999999999,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.7629999999999999,"5min":2.012,"15min":2.012},"last":1.095},{"interface":"front","average":{"1min":1.5409999999999999,"5min":1.4670000000000001,"15min":1.4419999999999999},"min":{"1min":1.0760000000000001,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.9199999999999999,"5min":2.3239999999999998,"15min":2.3239999999999998},"last":1.341}]}]},{"osd":5,"up_from":13,"seq":55834574889,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":324,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194987,"statfs":{"total":200000000,"available":199667380,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:33:41 2023","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.1120000000000001,"15min":1.1040000000000001},"min":{"1min":0.71899999999999997,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.7390000000000001,"5min":1.7390000000000001,"15min":1.7390000000000001},"last":0.88100000000000001},{"interface":"front","average":{"1min":1.234,"5min":1.157,"15min":1.139},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.641,"5min":1.641,"15min":1.641},"last":1.355}]},{"osd":1,"last update":"Thu Oct 26 01:33:41 2023","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":1.087,"15min":1.0720000000000001},"min":{"1min":0.67600000000000005,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.7729999999999999,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":0.80000000000000004},{"interface":"front","average":{"1min":1.1259999999999999,"5min":1.099,"15min":1.075},"min":{"1min":0.68200000000000005,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.585,"5min":1.7669999999999999,"15min":1.7669999999999999},"last":0.96599999999999997}]},{"osd":2,"last update":"Thu Oct 26 01:33:41 2023","interfaces":[{"interface":"back","average":{"1min":1.159,"5min":1.1579999999999999,"15min":1.1459999999999999},"min":{"1min":0.85699999999999998,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.788,"5min":1.788,"15min":1.788},"last":0.84099999999999997},{"interface":"front","average":{"1min":1.232,"5min":1.1679999999999999,"15min":1.163},"min":{"1min":0.91600000000000004,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.579,"5min":1.579,"15min":1.579},"last":1.133}]},{"osd":3,"last update":"Thu Oct 26 01:33:41 2023","interfaces":[{"interface":"back","average":{"1min":1.446,"5min":1.252,"15min":1.2270000000000001},"min":{"1min":1.1140000000000001,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.7649999999999999,"5min":1.835,"15min":1.835},"last":1.28},{"interface":"front","average":{"1min":1.343,"5min":1.288,"15min":1.284},"min":{"1min":1.0009999999999999,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.752,"5min":2.9249999999999998,"15min":2.9249999999999998},"last":1.048}]},{"osd":4,"last update":"Thu Oct 26 01:33:41 2023","interfaces":[{"interface":"back","average":{"1min":1.3799999999999999,"5min":1.222,"15min":1.1839999999999999},"min":{"1min":0.92300000000000004,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.6879999999999999,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":1.218},{"interface":"front","average":{"1min":1.5069999999999999,"5min":1.363,"15min":1.325},"min":{"1min":1.238,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.867,"5min":1.867,"15min":1.867},"last":1.4590000000000001}]},{"osd":6,"last update":"Thu Oct 26 01:33:41 2023","interfaces":[{"interface":"back","average":{"1min":1.5720000000000001,"5min":1.3360000000000001,"15min":1.286},"min":{"1min":1.2869999999999999,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.8999999999999999,"5min":1.8999999999999999,"15min":1.8999999999999999},"last":1.423},{"interface":"front","average":{"1min":1.53,"5min":1.3480000000000001,"15min":1.292},"min":{"1min":0.99299999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.8460000000000001,"5min":1.8560000000000001,"15min":1.8560000000000001},"last":1.593}]},{"osd":7,"last update":"Thu Oct 26 01:33:41 2023","interfaces":[{"interface":"back","average":{"1min":1.607,"5min":1.427,"15min":1.3779999999999999},"min":{"1min":1.1000000000000001,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.97,"5min":1.97,"15min":1.97},"last":1.544},{"interface":"front","average":{"1min":1.698,"5min":1.4390000000000001,"15min":1.387},"min":{"1min":1.3440000000000001,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":2.0680000000000001,"5min":2.0680000000000001,"15min":2.0680000000000001},"last":1.512}]}]},{"osd":4,"up_from":13,"seq":55834574889,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":325,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194986,"statfs":{"total":200000000,"available":199666420,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:33:33 2023","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":0.97999999999999998,"15min":0.95699999999999996},"min":{"1min":0.58099999999999996,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.885,"5min":2.0289999999999999,"15min":2.0289999999999999},"last":1.1879999999999999},{"interface":"front","average":{"1min":1.145,"5min":1.0189999999999999,"15min":1.0069999999999999},"min":{"1min":0.67600000000000005,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":2.6829999999999998,"5min":2.6829999999999998,"15min":2.6829999999999998},"last":1.0429999999999999}]},{"osd":1,"last update":"Thu Oct 26 01:33:33 2023","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.123,"15min":1.103},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":4.0519999999999996,"5min":4.0519999999999996,"15min":4.0519999999999996},"last":0.92500000000000004},{"interface":"front","average":{"1min":1.143,"5min":1.1000000000000001,"15min":1.0900000000000001},"min":{"1min":0.627,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.546,"5min":2.0760000000000001,"15min":2.0760000000000001},"last":1.0880000000000001}]},{"osd":2,"last update":"Thu Oct 26 01:33:33 2023","interfaces":[{"interface":"back","average":{"1min":1.579,"5min":1.256,"15min":1.198},"min":{"1min":0.68500000000000005,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":5.6749999999999998,"5min":5.6749999999999998,"15min":5.6749999999999998},"last":1.236},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.1200000000000001,"15min":1.1000000000000001},"min":{"1min":0.70899999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.6060000000000001,"5min":1.673,"15min":1.673},"last":0.98499999999999999}]},{"osd":3,"last update":"Thu Oct 26 01:33:33 2023","interfaces":[{"interface":"back","average":{"1min":1.135,"5min":1.173,"15min":1.175},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.661,"5min":1.891,"15min":1.891},"last":0.87},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.151,"15min":1.137},"min":{"1min":0.751,"5min":0.748,"15min":0.748},"max":{"1min":1.7210000000000001,"5min":1.7210000000000001,"15min":1.7210000000000001},"last":1.1379999999999999}]},{"osd":5,"last update":"Thu Oct 26 01:33:33 2023","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.282,"15min":1.296},"min":{"1min":0.86099999999999999,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.782,"5min":2.0049999999999999,"15min":2.0049999999999999},"last":1.2849999999999999},{"interface":"front","average":{"1min":1.5720000000000001,"5min":1.359,"15min":1.3360000000000001},"min":{"1min":0.86799999999999999,"5min":0.626,"15min":0.626},"max":{"1min":5.8579999999999997,"5min":5.8579999999999997,"15min":5.8579999999999997},"last":1.3300000000000001}]},{"osd":6,"last update":"Thu Oct 26 01:33:33 2023","interfaces":[{"interface":"back","average":{"1min":1.917,"5min":1.4219999999999999,"15min":1.3340000000000001},"min":{"1min":0.91900000000000004,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":12.343999999999999,"5min":12.343999999999999,"15min":12.343999999999999},"last":1.407},{"interface":"front","average":{"1min":2.0539999999999998,"5min":1.587,"15min":1.5009999999999999},"min":{"1min":1.0209999999999999,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":12.605,"5min":12.605,"15min":12.605},"last":1.365}]},{"osd":7,"last update":"Thu Oct 26 01:33:33 2023","interfaces":[{"interface":"back","average":{"1min":1.7370000000000001,"5min":1.526,"15min":1.494},"min":{"1min":1.099,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":5.774,"5min":5.774,"15min":5.774},"last":1.468},{"interface":"front","average":{"1min":1.631,"5min":1.4710000000000001,"15min":1.4370000000000001},"min":{"1min":1.0069999999999999,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":4.6520000000000001,"5min":4.6520000000000001,"15min":4.6520000000000001},"last":1.6539999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574889,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":324,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194988,"statfs":{"total":200000000,"available":199667976,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:33:33 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.85999999999999999,"15min":0.85599999999999998},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":1.208,"5min":6.3200000000000003,"15min":6.3200000000000003},"last":0.97199999999999998},{"interface":"front","average":{"1min":1.036,"5min":0.93100000000000005,"15min":0.90500000000000003},"min":{"1min":0.47199999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":2.1400000000000001,"5min":4.2619999999999996,"15min":4.2619999999999996},"last":1.3320000000000001}]},{"osd":1,"last update":"Thu Oct 26 01:33:33 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.90300000000000002,"15min":0.89500000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.569,"5min":5.1260000000000003,"15min":5.1260000000000003},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.97899999999999998,"5min":1.004,"15min":0.99199999999999999},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.74,"5min":6.1520000000000001,"15min":6.1520000000000001},"last":1.173}]},{"osd":3,"last update":"Thu Oct 26 01:33:39 2023","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":0.93600000000000005,"15min":0.90800000000000003},"min":{"1min":0.50900000000000001,"5min":0.443,"15min":0.443},"max":{"1min":1.7250000000000001,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":1.036},{"interface":"front","average":{"1min":1.161,"5min":1.044,"15min":0.97699999999999998},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.8660000000000001,"5min":8.9019999999999992,"15min":8.9019999999999992},"last":1.246}]},{"osd":4,"last update":"Thu Oct 26 01:33:33 2023","interfaces":[{"interface":"back","average":{"1min":1.2969999999999999,"5min":1.2589999999999999,"15min":1.2330000000000001},"min":{"1min":0.76000000000000001,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.7929999999999999,"5min":6.2670000000000003,"15min":6.2670000000000003},"last":1.696},{"interface":"front","average":{"1min":1.351,"5min":1.288,"15min":1.2370000000000001},"min":{"1min":0.89300000000000002,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":2.3530000000000002,"5min":7.343,"15min":7.343},"last":1.5609999999999999}]},{"osd":5,"last update":"Thu Oct 26 01:33:33 2023","interfaces":[{"interface":"back","average":{"1min":1.3149999999999999,"5min":1.2689999999999999,"15min":1.2569999999999999},"min":{"1min":0.86099999999999999,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":2.0089999999999999,"5min":3.7170000000000001,"15min":3.7170000000000001},"last":1.659},{"interface":"front","average":{"1min":1.2889999999999999,"5min":1.294,"15min":1.28},"min":{"1min":0.70999999999999996,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":2.0099999999999998,"5min":6.3899999999999997,"15min":6.3899999999999997},"last":1.3049999999999999}]},{"osd":6,"last update":"Thu Oct 26 01:33:33 2023","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":1.3320000000000001,"15min":1.304},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.9359999999999999,"5min":6.0490000000000004,"15min":6.0490000000000004},"last":1.905},{"interface":"front","average":{"1min":1.401,"5min":1.3660000000000001,"15min":1.3500000000000001},"min":{"1min":0.88400000000000001,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":2.1840000000000002,"5min":5.0650000000000004,"15min":5.0650000000000004},"last":1.7509999999999999}]},{"osd":7,"last update":"Thu Oct 26 01:33:33 2023","interfaces":[{"interface":"back","average":{"1min":1.3959999999999999,"5min":1.3959999999999999,"15min":1.3899999999999999},"min":{"1min":0.96899999999999997,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":2.0899999999999999,"5min":6.4480000000000004,"15min":6.4480000000000004},"last":1.3620000000000001},{"interface":"front","average":{"1min":1.4970000000000001,"5min":1.431,"15min":1.429},"min":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"max":{"1min":2.21,"5min":2.8439999999999999,"15min":2.8439999999999999},"last":2.0070000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574889,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":324,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194988,"statfs":{"total":200000000,"available":199667942,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:33:37 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.72399999999999998,"15min":0.70199999999999996},"min":{"1min":0.45600000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.224,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.77100000000000002,"15min":0.76600000000000001},"min":{"1min":0.40500000000000003,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.3879999999999999,"5min":1.4830000000000001,"15min":1.4830000000000001},"last":0.79900000000000004}]},{"osd":2,"last update":"Thu Oct 26 01:33:37 2023","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.98499999999999999,"15min":1.0069999999999999},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.4450000000000001,"5min":2.4830000000000001,"15min":2.4830000000000001},"last":1.002},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.84799999999999998,"15min":0.84399999999999997},"min":{"1min":0.47299999999999998,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.89600000000000002}]},{"osd":3,"last update":"Thu Oct 26 01:33:38 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.93999999999999995,"15min":0.93200000000000005},"min":{"1min":0.51700000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.3640000000000001,"5min":1.75,"15min":1.75},"last":0.94099999999999995},{"interface":"front","average":{"1min":1,"5min":0.97799999999999998,"15min":0.97199999999999998},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.361,"5min":1.802,"15min":1.802},"last":0.67100000000000004}]},{"osd":4,"last update":"Thu Oct 26 01:33:37 2023","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.1950000000000001,"15min":1.1859999999999999},"min":{"1min":0.82399999999999995,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.6220000000000001,"5min":2.0659999999999998,"15min":2.0659999999999998},"last":1.101},{"interface":"front","average":{"1min":1.2,"5min":1.125,"15min":1.113},"min":{"1min":0.78100000000000003,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.486,"5min":1.611,"15min":1.611},"last":1.0549999999999999}]},{"osd":5,"last update":"Thu Oct 26 01:33:37 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.2070000000000001,"15min":1.214},"min":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":1.5369999999999999,"5min":1.702,"15min":1.702},"last":1.167},{"interface":"front","average":{"1min":1.347,"5min":1.3149999999999999,"15min":1.323},"min":{"1min":0.92500000000000004,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.619,"5min":2.2200000000000002,"15min":2.2200000000000002},"last":1.468}]},{"osd":6,"last update":"Thu Oct 26 01:33:37 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.25,"15min":1.256},"min":{"1min":0.83199999999999996,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.694,"5min":2.3780000000000001,"15min":2.3780000000000001},"last":1.4139999999999999},{"interface":"front","average":{"1min":1.2609999999999999,"5min":1.242,"15min":1.2310000000000001},"min":{"1min":0.71299999999999997,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.609,"5min":1.984,"15min":1.984},"last":0.98599999999999999}]},{"osd":7,"last update":"Thu Oct 26 01:33:37 2023","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.242,"15min":1.2609999999999999},"min":{"1min":0.84099999999999997,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.7010000000000001,"5min":2.6930000000000001,"15min":2.6930000000000001},"last":1.431},{"interface":"front","average":{"1min":1.3460000000000001,"5min":1.3140000000000001,"15min":1.306},"min":{"1min":1.0249999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.8240000000000001,"5min":1.869,"15min":1.869},"last":1.367}]}]},{"osd":0,"up_from":13,"seq":55834574889,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":902,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194409,"statfs":{"total":200000000,"available":199075470,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Oct 26 01:33:38 2023","interfaces":[{"interface":"back","average":{"1min":1.833,"5min":1.0489999999999999,"15min":0.93999999999999995},"min":{"1min":0.60499999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":19.425000000000001,"5min":19.425000000000001,"15min":19.425000000000001},"last":0.63600000000000001},{"interface":"front","average":{"1min":1.6319999999999999,"5min":1.1240000000000001,"15min":1.0720000000000001},"min":{"1min":0.56799999999999995,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":15.162000000000001,"5min":15.162000000000001,"15min":15.162000000000001},"last":1.038}]},{"osd":2,"last update":"Thu Oct 26 01:33:38 2023","interfaces":[{"interface":"back","average":{"1min":2.4940000000000002,"5min":1.3680000000000001,"15min":1.2170000000000001},"min":{"1min":0.57999999999999996,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":28.439,"5min":28.439,"15min":28.439},"last":0.99299999999999999},{"interface":"front","average":{"1min":1.0129999999999999,"5min":0.92200000000000004,"15min":0.91900000000000004},"min":{"1min":0.45200000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.502,"5min":4.9459999999999997,"15min":4.9459999999999997},"last":0.95999999999999996}]},{"osd":3,"last update":"Thu Oct 26 01:33:38 2023","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.075,"15min":1.1020000000000001},"min":{"1min":0.49099999999999999,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.3919999999999999,"5min":4.9800000000000004,"15min":4.9800000000000004},"last":1.071},{"interface":"front","average":{"1min":1.21,"5min":1.153,"15min":1.167},"min":{"1min":0.66000000000000003,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.659,"5min":5.016,"15min":5.016},"last":1.2170000000000001}]},{"osd":4,"last update":"Thu Oct 26 01:33:38 2023","interfaces":[{"interface":"back","average":{"1min":2.145,"5min":1.411,"15min":1.3089999999999999},"min":{"1min":0.79100000000000004,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":19.329000000000001,"5min":19.329000000000001,"15min":19.329000000000001},"last":1.4359999999999999},{"interface":"front","average":{"1min":1.2609999999999999,"5min":1.292,"15min":1.3129999999999999},"min":{"1min":0.90300000000000002,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.575,"5min":5.5229999999999997,"15min":5.5229999999999997},"last":1.6739999999999999}]},{"osd":5,"last update":"Thu Oct 26 01:33:38 2023","interfaces":[{"interface":"back","average":{"1min":1.4359999999999999,"5min":1.4319999999999999,"15min":1.444},"min":{"1min":0.93799999999999994,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.915,"5min":5.6360000000000001,"15min":5.6360000000000001},"last":1.5149999999999999},{"interface":"front","average":{"1min":2.3130000000000002,"5min":1.5389999999999999,"15min":1.427},"min":{"1min":0.86299999999999999,"5min":0.626,"15min":0.626},"max":{"1min":19.384,"5min":19.384,"15min":19.384},"last":1.7330000000000001}]},{"osd":6,"last update":"Thu Oct 26 01:33:38 2023","interfaces":[{"interface":"back","average":{"1min":1.7270000000000001,"5min":1.415,"15min":1.385},"min":{"1min":0.82499999999999996,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":8.2609999999999992,"5min":8.2609999999999992,"15min":8.2609999999999992},"last":1.1639999999999999},{"interface":"front","average":{"1min":1.923,"5min":1.4690000000000001,"15min":1.421},"min":{"1min":1.0049999999999999,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":11.849,"5min":11.849,"15min":11.849},"last":1.264}]},{"osd":7,"last update":"Thu Oct 26 01:33:38 2023","interfaces":[{"interface":"back","average":{"1min":1.524,"5min":1.49,"15min":1.5029999999999999},"min":{"1min":1.099,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":2.4060000000000001,"5min":5.6689999999999996,"15min":5.6689999999999996},"last":1.6040000000000001},{"interface":"front","average":{"1min":2.387,"5min":1.6220000000000001,"15min":1.514},"min":{"1min":1.0529999999999999,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":19.245999999999999,"5min":19.245999999999999,"15min":19.245999999999999},"last":1.5449999999999999}]}]}],"pool_statfs":[]}} 2023-10-26T01:33:46.273 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-26T01:33:46.274 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-26T01:33:46.535 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:46.533+0000 7f0402c7c640 1 -- 172.21.15.169:0/643728811 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03ec003970 msgr2=0x7f03ec003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:46.535 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:46.533+0000 7f0402c7c640 1 --2- 172.21.15.169:0/643728811 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03ec003970 0x7f03ec003d90 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f03f0009b50 tx=0x7f03ec0057f0 comp rx=0 tx=0).stop 2023-10-26T01:33:46.535 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:46.533+0000 7f0401219640 1 -- 172.21.15.169:0/643728811 reap_dead start 2023-10-26T01:33:46.536 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:46.533+0000 7f0402c7c640 1 -- 172.21.15.169:0/643728811 shutdown_connections 2023-10-26T01:33:46.536 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:46.533+0000 7f0402c7c640 1 -- 172.21.15.169:0/643728811 >> 172.21.15.169:0/643728811 conn(0x7f03fc069930 msgr2=0x7f03fc069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:46.536 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:46.533+0000 7f0402c7c640 1 -- 172.21.15.169:0/643728811 shutdown_connections 2023-10-26T01:33:46.536 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:46.533+0000 7f0402c7c640 1 -- 172.21.15.169:0/643728811 wait complete. 2023-10-26T01:33:46.537 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:46.535+0000 7f0402c7c640 1 Processor -- start 2023-10-26T01:33:46.537 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:46.535+0000 7f0402c7c640 1 -- start start 2023-10-26T01:33:46.538 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:46.536+0000 7f0402c7c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03ec003970 0x7f03fc0c72c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:46.538 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:46.536+0000 7f0402c7c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f03fc06b050 0x7f03fc0c8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:46.538 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:46.536+0000 7f0402c7c640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f03fc0c9420 0x7f03fc14f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:46.539 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:46.536+0000 7f0402c7c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03fc1f1f80 con 0x7f03ec003970 2023-10-26T01:33:46.539 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:46.536+0000 7f0402c7c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03fc1f2220 con 0x7f03fc06b050 2023-10-26T01:33:46.539 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:46.536+0000 7f0402c7c640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03fc1f2500 con 0x7f03fc0c9420 2023-10-26T01:33:46.539 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:46.536+0000 7f0400a18640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03ec003970 0x7f03fc0c72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:46.539 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:46.536+0000 7f0400a18640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03ec003970 0x7f03fc0c72c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:59404/0 (socket says 172.21.15.169:59404) 2023-10-26T01:33:46.540 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:46.536+0000 7f0400a18640 1 -- 172.21.15.169:0/2379945997 learned_addr learned my addr 172.21.15.169:0/2379945997 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:33:46.541 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:46.537+0000 7f03fbfff640 1 --2- 172.21.15.169:0/2379945997 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f03fc06b050 0x7f03fc0c8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:46.541 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:46.537+0000 7f0400a18640 1 -- 172.21.15.169:0/2379945997 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f03fc0c9420 msgr2=0x7f03fc14f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:46.541 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:46.537+0000 7f0400a18640 1 --2- 172.21.15.169:0/2379945997 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f03fc0c9420 0x7f03fc14f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:46.541 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:46.537+0000 7f0400a18640 1 -- 172.21.15.169:0/2379945997 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f03fc06b050 msgr2=0x7f03fc0c8ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:46.541 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:46.537+0000 7f0400a18640 1 --2- 172.21.15.169:0/2379945997 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f03fc06b050 0x7f03fc0c8ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:46.542 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:46.537+0000 7f0400a18640 1 -- 172.21.15.169:0/2379945997 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03fc1f27e0 con 0x7f03ec003970 2023-10-26T01:33:46.542 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:46.537+0000 7f0400a18640 1 --2- 172.21.15.169:0/2379945997 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03ec003970 0x7f03fc0c72c0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f03f0009a70 tx=0x7f03f0033c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:46.542 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:46.537+0000 7f03f97fa640 1 -- 172.21.15.169:0/2379945997 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03f0004220 con 0x7f03ec003970 2023-10-26T01:33:46.542 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:46.537+0000 7f03f97fa640 1 -- 172.21.15.169:0/2379945997 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f03f0004380 con 0x7f03ec003970 2023-10-26T01:33:46.543 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:46.537+0000 7f0402c7c640 1 -- 172.21.15.169:0/2379945997 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03fc1f2ac0 con 0x7f03ec003970 2023-10-26T01:33:46.543 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:46.538+0000 7f03f97fa640 1 -- 172.21.15.169:0/2379945997 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03f003d940 con 0x7f03ec003970 2023-10-26T01:33:46.543 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:46.538+0000 7f0402c7c640 1 -- 172.21.15.169:0/2379945997 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03fc0c79e0 con 0x7f03ec003970 2023-10-26T01:33:46.543 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:46.539+0000 7f0402c7c640 1 -- 172.21.15.169:0/2379945997 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03fc04db10 con 0x7f03ec003970 2023-10-26T01:33:46.544 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:46.540+0000 7f03f97fa640 1 -- 172.21.15.169:0/2379945997 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f03f003daa0 con 0x7f03ec003970 2023-10-26T01:33:46.544 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:46.541+0000 7f03f97fa640 1 --2- 172.21.15.169:0/2379945997 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f03e006ae40 0x7f03e006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:46.544 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:46.541+0000 7f03f97fa640 1 -- 172.21.15.169:0/2379945997 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f03f0033e40 con 0x7f03ec003970 2023-10-26T01:33:46.544 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:46.541+0000 7f03fbfff640 1 --2- 172.21.15.169:0/2379945997 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f03e006ae40 0x7f03e006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:46.545 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:46.542+0000 7f03fbfff640 1 --2- 172.21.15.169:0/2379945997 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f03e006ae40 0x7f03e006d300 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f03fc1f53c0 tx=0x7f03e8005f90 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:46.546 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:46.545+0000 7f03f97fa640 1 -- 172.21.15.169:0/2379945997 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f03f0077330 con 0x7f03ec003970 2023-10-26T01:33:46.803 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:46.801+0000 7f0402c7c640 1 -- 172.21.15.169:0/2379945997 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f03fc0c79e0 con 0x7f03ec003970 2023-10-26T01:33:46.808 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:46.806+0000 7f03f97fa640 1 -- 172.21.15.169:0/2379945997 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 190+0+147 (secure 0 0 0) 0x7f03f004a0b0 con 0x7f03ec003970 2023-10-26T01:33:46.808 INFO:teuthology.orchestra.run.smithi169.stdout:[client.0] 2023-10-26T01:33:46.808 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQAgwTllKC7gOhAAPsvxGcuUA8/p8xqbUgGteA== 2023-10-26T01:33:46.809 INFO:teuthology.orchestra.run.smithi169.stdout: caps mds = "allow" 2023-10-26T01:33:46.809 INFO:teuthology.orchestra.run.smithi169.stdout: caps mon = "allow r" 2023-10-26T01:33:46.809 INFO:teuthology.orchestra.run.smithi169.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-10-26T01:33:46.809 INFO:teuthology.orchestra.run.smithi169.stderr:updated caps for client.0 2023-10-26T01:33:46.811 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:46.808+0000 7f0402c7c640 1 -- 172.21.15.169:0/2379945997 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f03e006ae40 msgr2=0x7f03e006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:46.811 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:46.808+0000 7f0402c7c640 1 --2- 172.21.15.169:0/2379945997 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f03e006ae40 0x7f03e006d300 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f03fc1f53c0 tx=0x7f03e8005f90 comp rx=0 tx=0).stop 2023-10-26T01:33:46.811 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:46.808+0000 7f0402c7c640 1 -- 172.21.15.169:0/2379945997 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03ec003970 msgr2=0x7f03fc0c72c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:46.811 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:46.808+0000 7f0402c7c640 1 --2- 172.21.15.169:0/2379945997 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03ec003970 0x7f03fc0c72c0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f03f0009a70 tx=0x7f03f0033c50 comp rx=0 tx=0).stop 2023-10-26T01:33:46.812 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:46.809+0000 7f0402c7c640 1 -- 172.21.15.169:0/2379945997 shutdown_connections 2023-10-26T01:33:46.812 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:46.809+0000 7f0402c7c640 1 --2- 172.21.15.169:0/2379945997 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f03e006ae40 0x7f03e006d300 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:46.812 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:46.809+0000 7f0402c7c640 1 --2- 172.21.15.169:0/2379945997 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f03fc0c9420 0x7f03fc14f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:46.812 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:46.809+0000 7f0402c7c640 1 --2- 172.21.15.169:0/2379945997 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f03fc06b050 0x7f03fc0c8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:46.813 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:46.809+0000 7f0402c7c640 1 --2- 172.21.15.169:0/2379945997 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03ec003970 0x7f03fc0c72c0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:46.813 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:46.809+0000 7f0402c7c640 1 -- 172.21.15.169:0/2379945997 >> 172.21.15.169:0/2379945997 conn(0x7f03fc069930 msgr2=0x7f03fc151750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:46.813 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:46.809+0000 7f0402c7c640 1 -- 172.21.15.169:0/2379945997 shutdown_connections 2023-10-26T01:33:46.813 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:46.809+0000 7f0402c7c640 1 -- 172.21.15.169:0/2379945997 wait complete. 2023-10-26T01:33:46.830 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T01:33:47.053 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.051+0000 7f86ba096640 1 -- 172.21.15.169:0/1042268847 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86a00047c0 msgr2=0x7f86a0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:47.053 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.051+0000 7f86ba096640 1 --2- 172.21.15.169:0/1042268847 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86a00047c0 0x7f86a0004c40 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f869c009b50 tx=0x7f86a00057f0 comp rx=0 tx=0).stop 2023-10-26T01:33:47.053 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.051+0000 7f86b3fff640 1 -- 172.21.15.169:0/1042268847 reap_dead start 2023-10-26T01:33:47.053 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.051+0000 7f86ba096640 1 -- 172.21.15.169:0/1042268847 shutdown_connections 2023-10-26T01:33:47.054 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.051+0000 7f86ba096640 1 -- 172.21.15.169:0/1042268847 >> 172.21.15.169:0/1042268847 conn(0x7f86b4069930 msgr2=0x7f86b4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:47.054 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.051+0000 7f86ba096640 1 -- 172.21.15.169:0/1042268847 shutdown_connections 2023-10-26T01:33:47.054 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.051+0000 7f86ba096640 1 -- 172.21.15.169:0/1042268847 wait complete. 2023-10-26T01:33:47.055 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.053+0000 7f86ba096640 1 Processor -- start 2023-10-26T01:33:47.055 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.053+0000 7f86ba096640 1 -- start start 2023-10-26T01:33:47.057 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.054+0000 7f86ba096640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f86a00047c0 0x7f86b414b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:47.057 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.054+0000 7f86ba096640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f86b414a600 0x7f86b41f16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:47.057 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.054+0000 7f86ba096640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86b414aa60 0x7f86b41f1be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:47.057 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.054+0000 7f86ba096640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86b40c71f0 con 0x7f86b414aa60 2023-10-26T01:33:47.057 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.054+0000 7f86ba096640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86b41483a0 con 0x7f86b414a600 2023-10-26T01:33:47.058 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.054+0000 7f86ba096640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86b4148680 con 0x7f86a00047c0 2023-10-26T01:33:47.058 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.054+0000 7f86b3fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86b414aa60 0x7f86b41f1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:47.058 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.054+0000 7f86b3fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86b414aa60 0x7f86b41f1be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:59424/0 (socket says 172.21.15.169:59424) 2023-10-26T01:33:47.058 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.054+0000 7f86b3fff640 1 -- 172.21.15.169:0/2948079052 learned_addr learned my addr 172.21.15.169:0/2948079052 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:33:47.058 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.054+0000 7f86b3fff640 1 -- 172.21.15.169:0/2948079052 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f86a00047c0 msgr2=0x7f86b414b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:47.059 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.054+0000 7f86b3fff640 1 --2- 172.21.15.169:0/2948079052 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f86a00047c0 0x7f86b414b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:47.059 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.054+0000 7f86b3fff640 1 -- 172.21.15.169:0/2948079052 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f86b414a600 msgr2=0x7f86b41f16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:47.059 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.054+0000 7f86b3fff640 1 --2- 172.21.15.169:0/2948079052 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f86b414a600 0x7f86b41f16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:47.059 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.054+0000 7f86b3fff640 1 -- 172.21.15.169:0/2948079052 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86b4148960 con 0x7f86b414aa60 2023-10-26T01:33:47.059 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.055+0000 7f86b3fff640 1 --2- 172.21.15.169:0/2948079052 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86b414aa60 0x7f86b41f1be0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f86ac004a50 tx=0x7f86ac00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:47.061 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.055+0000 7f86b0ff9640 1 -- 172.21.15.169:0/2948079052 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86ac0020d0 con 0x7f86b414aa60 2023-10-26T01:33:47.061 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.055+0000 7f86b0ff9640 1 -- 172.21.15.169:0/2948079052 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f86ac007070 con 0x7f86b414aa60 2023-10-26T01:33:47.061 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.055+0000 7f86ba096640 1 -- 172.21.15.169:0/2948079052 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86b4148c40 con 0x7f86b414aa60 2023-10-26T01:33:47.061 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.055+0000 7f86b0ff9640 1 -- 172.21.15.169:0/2948079052 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86ac007480 con 0x7f86b414aa60 2023-10-26T01:33:47.062 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.055+0000 7f86ba096640 1 -- 172.21.15.169:0/2948079052 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86b41e5f00 con 0x7f86b414aa60 2023-10-26T01:33:47.063 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.056+0000 7f86ba096640 1 -- 172.21.15.169:0/2948079052 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86b4149160 con 0x7f86b414aa60 2023-10-26T01:33:47.063 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.057+0000 7f86b0ff9640 1 -- 172.21.15.169:0/2948079052 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f86ac0075e0 con 0x7f86b414aa60 2023-10-26T01:33:47.063 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.058+0000 7f86b0ff9640 1 --2- 172.21.15.169:0/2948079052 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f869406ae40 0x7f869406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:47.063 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.058+0000 7f86b0ff9640 1 -- 172.21.15.169:0/2948079052 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f86ac0897b0 con 0x7f86b414aa60 2023-10-26T01:33:47.064 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.059+0000 7f86b37fe640 1 --2- 172.21.15.169:0/2948079052 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f869406ae40 0x7f869406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:47.064 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.060+0000 7f86b37fe640 1 --2- 172.21.15.169:0/2948079052 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f869406ae40 0x7f869406d300 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f86a40058c0 tx=0x7f86a4005780 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:47.064 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.063+0000 7f86b0ff9640 1 -- 172.21.15.169:0/2948079052 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f86b4149160 con 0x7f86b414aa60 2023-10-26T01:33:47.227 INFO:tasks.ceph.mds.a.smithi169.stderr:2023-10-26T01:33:47.225+0000 7f60c2948640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-26T01:33:47.284 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.283+0000 7f86ba096640 1 -- 172.21.15.169:0/2948079052 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f86b4149160 con 0x7f86b414aa60 2023-10-26T01:33:47.285 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.284+0000 7f86b0ff9640 1 -- 172.21.15.169:0/2948079052 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4922 (secure 0 0 0) 0x7f86b4149160 con 0x7f86b414aa60 2023-10-26T01:33:47.286 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:33:47.286 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4976,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/2977083370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2977083370},{"type":"v1","addr":"172.21.15.170:6837","nonce":2977083370}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":5176,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/1598214042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1598214042},{"type":"v1","addr":"172.21.15.170:6835","nonce":1598214042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":6087,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/406582181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":406582181},{"type":"v1","addr":"172.21.15.169:6833","nonce":406582181}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T01:33:40.717881+0000","modified":"2023-10-26T01:33:43.796080+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6093},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6093":{"gid":6093,"name":"a","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6835/2338833513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":2338833513},{"type":"v1","addr":"172.21.15.169:6835","nonce":2338833513}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-10-26T01:33:47.286 INFO:teuthology.orchestra.run.smithi169.stderr:dumped fsmap epoch 32 2023-10-26T01:33:47.288 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.286+0000 7f86ba096640 1 -- 172.21.15.169:0/2948079052 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f869406ae40 msgr2=0x7f869406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:47.288 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.286+0000 7f86ba096640 1 --2- 172.21.15.169:0/2948079052 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f869406ae40 0x7f869406d300 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f86a40058c0 tx=0x7f86a4005780 comp rx=0 tx=0).stop 2023-10-26T01:33:47.289 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.286+0000 7f86ba096640 1 -- 172.21.15.169:0/2948079052 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86b414aa60 msgr2=0x7f86b41f1be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:47.289 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.286+0000 7f86ba096640 1 --2- 172.21.15.169:0/2948079052 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86b414aa60 0x7f86b41f1be0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f86ac004a50 tx=0x7f86ac00a430 comp rx=0 tx=0).stop 2023-10-26T01:33:47.289 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.286+0000 7f86ba096640 1 -- 172.21.15.169:0/2948079052 shutdown_connections 2023-10-26T01:33:47.289 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.286+0000 7f86ba096640 1 --2- 172.21.15.169:0/2948079052 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f869406ae40 0x7f869406d300 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:47.289 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.286+0000 7f86ba096640 1 --2- 172.21.15.169:0/2948079052 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86b414aa60 0x7f86b41f1be0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:47.290 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.286+0000 7f86ba096640 1 --2- 172.21.15.169:0/2948079052 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f86b414a600 0x7f86b41f16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:47.290 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.286+0000 7f86ba096640 1 --2- 172.21.15.169:0/2948079052 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f86a00047c0 0x7f86b414b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:47.290 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.286+0000 7f86ba096640 1 -- 172.21.15.169:0/2948079052 >> 172.21.15.169:0/2948079052 conn(0x7f86b4069930 msgr2=0x7f86b406be40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:47.290 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.287+0000 7f86ba096640 1 -- 172.21.15.169:0/2948079052 shutdown_connections 2023-10-26T01:33:47.290 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.287+0000 7f86ba096640 1 -- 172.21.15.169:0/2948079052 wait complete. 2023-10-26T01:33:47.707 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 32, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-26T01:33:40.717881+0000', 'modified': '2023-10-26T01:33:43.796080+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6093}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6093': {'gid': 6093, 'name': 'a', 'rank': 0, 'incarnation': 28, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.169:6835/2338833513', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.169:6834', 'nonce': 2338833513}, {'type': 'v1', 'addr': '172.21.15.169:6835', 'nonce': 2338833513}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-26T01:33:47.708 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-26T01:33:47.708 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-26T01:33:47.942 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.940+0000 7f75cf983640 1 -- 172.21.15.169:0/950936283 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75c815bbb0 msgr2=0x7f75c8155360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:47.942 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.940+0000 7f75cf983640 1 --2- 172.21.15.169:0/950936283 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75c815bbb0 0x7f75c8155360 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f75b8005fd0 tx=0x7f75b802bd90 comp rx=0 tx=0).stop 2023-10-26T01:33:47.942 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.940+0000 7f75cf983640 1 -- 172.21.15.169:0/950936283 shutdown_connections 2023-10-26T01:33:47.943 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.940+0000 7f75cf983640 1 --2- 172.21.15.169:0/950936283 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75c815bbb0 0x7f75c8155360 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:47.943 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.940+0000 7f75cf983640 1 -- 172.21.15.169:0/950936283 >> 172.21.15.169:0/950936283 conn(0x7f75c8069930 msgr2=0x7f75c8069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:47.943 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.940+0000 7f75cf983640 1 -- 172.21.15.169:0/950936283 shutdown_connections 2023-10-26T01:33:47.943 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.940+0000 7f75cf983640 1 -- 172.21.15.169:0/950936283 wait complete. 2023-10-26T01:33:47.944 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.942+0000 7f75cf983640 1 Processor -- start 2023-10-26T01:33:47.944 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.942+0000 7f75cf983640 1 -- start start 2023-10-26T01:33:47.946 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.943+0000 7f75cf983640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f75c806ce00 0x7f75c8096910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:47.946 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.943+0000 7f75cf983640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f75c80b05a0 0x7f75c81e5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:47.946 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.943+0000 7f75cf983640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75c815bbb0 0x7f75c81e5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:47.946 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.943+0000 7f75cf983640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75c81535d0 con 0x7f75c815bbb0 2023-10-26T01:33:47.947 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.943+0000 7f75cf983640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75c80ca830 con 0x7f75c80b05a0 2023-10-26T01:33:47.947 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.943+0000 7f75cf983640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75c8152f80 con 0x7f75c806ce00 2023-10-26T01:33:47.947 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.943+0000 7f75cdf20640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75c815bbb0 0x7f75c81e5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:47.947 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.943+0000 7f75cdf20640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75c815bbb0 0x7f75c81e5a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:59444/0 (socket says 172.21.15.169:59444) 2023-10-26T01:33:47.947 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.943+0000 7f75cdf20640 1 -- 172.21.15.169:0/2606884186 learned_addr learned my addr 172.21.15.169:0/2606884186 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:33:47.948 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.944+0000 7f75cd71f640 1 --2- 172.21.15.169:0/2606884186 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f75c806ce00 0x7f75c8096910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:47.948 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.944+0000 7f75ccf1e640 1 --2- 172.21.15.169:0/2606884186 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f75c80b05a0 0x7f75c81e5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:47.948 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.944+0000 7f75cdf20640 1 -- 172.21.15.169:0/2606884186 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f75c806ce00 msgr2=0x7f75c8096910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:47.948 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.944+0000 7f75cdf20640 1 --2- 172.21.15.169:0/2606884186 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f75c806ce00 0x7f75c8096910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:47.948 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.944+0000 7f75cdf20640 1 -- 172.21.15.169:0/2606884186 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f75c80b05a0 msgr2=0x7f75c81e5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:47.949 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.944+0000 7f75cdf20640 1 --2- 172.21.15.169:0/2606884186 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f75c80b05a0 0x7f75c81e5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:47.949 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.944+0000 7f75cdf20640 1 -- 172.21.15.169:0/2606884186 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75b8009560 con 0x7f75c815bbb0 2023-10-26T01:33:47.949 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.944+0000 7f75ccf1e640 1 --2- 172.21.15.169:0/2606884186 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f75c80b05a0 0x7f75c81e5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:33:47.949 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.944+0000 7f75cdf20640 1 --2- 172.21.15.169:0/2606884186 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75c815bbb0 0x7f75c81e5a70 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f75c000c780 tx=0x7f75c000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:47.949 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.944+0000 7f75bdffb640 1 -- 172.21.15.169:0/2606884186 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75c0004b60 con 0x7f75c815bbb0 2023-10-26T01:33:47.950 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.944+0000 7f75bdffb640 1 -- 172.21.15.169:0/2606884186 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f75c000ce90 con 0x7f75c815bbb0 2023-10-26T01:33:47.950 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.944+0000 7f75bdffb640 1 -- 172.21.15.169:0/2606884186 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75c000aab0 con 0x7f75c815bbb0 2023-10-26T01:33:47.950 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.944+0000 7f75cf983640 1 -- 172.21.15.169:0/2606884186 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75c8153260 con 0x7f75c815bbb0 2023-10-26T01:33:47.950 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.944+0000 7f75cf983640 1 -- 172.21.15.169:0/2606884186 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75c80cab10 con 0x7f75c815bbb0 2023-10-26T01:33:47.951 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.947+0000 7f75bdffb640 1 -- 172.21.15.169:0/2606884186 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f75c000ac10 con 0x7f75c815bbb0 2023-10-26T01:33:47.951 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.948+0000 7f75bdffb640 1 --2- 172.21.15.169:0/2606884186 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f75ac06ae40 0x7f75ac06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:47.951 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.948+0000 7f75bdffb640 1 -- 172.21.15.169:0/2606884186 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f75c0089ab0 con 0x7f75c815bbb0 2023-10-26T01:33:47.952 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.949+0000 7f75d25ef740 1 Processor -- start 2023-10-26T01:33:47.952 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.949+0000 7f75cd71f640 1 --2- 172.21.15.169:0/2606884186 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f75ac06ae40 0x7f75ac06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:47.952 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.949+0000 7f75d25ef740 1 -- start start 2023-10-26T01:33:47.954 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.950+0000 7f75cd71f640 1 --2- 172.21.15.169:0/2606884186 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f75ac06ae40 0x7f75ac06d300 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f75c4000f60 tx=0x7f75c4009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:47.955 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.950+0000 7f75d25ef740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a289bba7d0 0x55a289b8da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:47.955 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.950+0000 7f75d25ef740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a289bbabb0 0x55a289bbd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:47.955 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.950+0000 7f75d25ef740 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x55a289b8df40 0x55a289bbdaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:47.955 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.950+0000 7f75d25ef740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a289b00740 con 0x55a289bba7d0 2023-10-26T01:33:47.955 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.950+0000 7f75d25ef740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a289afde30 con 0x55a289bbabb0 2023-10-26T01:33:47.956 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.950+0000 7f75d25ef740 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a289af9720 con 0x55a289b8df40 2023-10-26T01:33:47.956 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.950+0000 7f75cd71f640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a289bba7d0 0x55a289b8da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:47.956 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.950+0000 7f75cd71f640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a289bba7d0 0x55a289b8da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:59454/0 (socket says 172.21.15.169:59454) 2023-10-26T01:33:47.956 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.950+0000 7f75cd71f640 1 -- 172.21.15.169:0/1804890909 learned_addr learned my addr 172.21.15.169:0/1804890909 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:33:47.956 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.951+0000 7f75ccf1e640 1 --2- 172.21.15.169:0/1804890909 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a289bbabb0 0x55a289bbd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:47.957 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.951+0000 7f75cdf20640 1 --2- 172.21.15.169:0/1804890909 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x55a289b8df40 0x55a289bbdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:47.957 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.951+0000 7f75cd71f640 1 -- 172.21.15.169:0/1804890909 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x55a289b8df40 msgr2=0x55a289bbdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T01:33:47.957 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.951+0000 7f75cd71f640 1 --2- 172.21.15.169:0/1804890909 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x55a289b8df40 0x55a289bbdaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:47.957 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.951+0000 7f75cd71f640 1 -- 172.21.15.169:0/1804890909 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a289bbabb0 msgr2=0x55a289bbd560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T01:33:47.957 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.951+0000 7f75cd71f640 1 --2- 172.21.15.169:0/1804890909 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a289bbabb0 0x55a289bbd560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:47.958 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.951+0000 7f75cd71f640 1 -- 172.21.15.169:0/1804890909 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a289abd910 con 0x55a289bba7d0 2023-10-26T01:33:47.958 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.951+0000 7f75cd71f640 1 --2- 172.21.15.169:0/1804890909 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a289bba7d0 0x55a289b8da00 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f75c4005130 tx=0x7f75c4010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1295d3c40aacd75f server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:47.958 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.951+0000 7f75aaffd640 1 -- 172.21.15.169:0/1804890909 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75c401c070 con 0x55a289bba7d0 2023-10-26T01:33:47.958 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.951+0000 7f75aaffd640 1 -- 172.21.15.169:0/1804890909 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f75c4003610 con 0x55a289bba7d0 2023-10-26T01:33:47.958 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.951+0000 7f75aaffd640 1 -- 172.21.15.169:0/1804890909 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75c40039c0 con 0x55a289bba7d0 2023-10-26T01:33:47.959 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.952+0000 7f75d25ef740 1 -- 172.21.15.169:0/1804890909 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a289bba7d0 msgr2=0x55a289b8da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:47.959 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.952+0000 7f75d25ef740 1 --2- 172.21.15.169:0/1804890909 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a289bba7d0 0x55a289b8da00 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f75c4005130 tx=0x7f75c4010000 comp rx=0 tx=0).stop 2023-10-26T01:33:47.959 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.952+0000 7f75d25ef740 1 -- 172.21.15.169:0/1804890909 shutdown_connections 2023-10-26T01:33:47.959 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.952+0000 7f75d25ef740 1 --2- 172.21.15.169:0/1804890909 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x55a289b8df40 0x55a289bbdaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:47.959 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.952+0000 7f75d25ef740 1 --2- 172.21.15.169:0/1804890909 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a289bbabb0 0x55a289bbd560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:47.960 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.952+0000 7f75d25ef740 1 --2- 172.21.15.169:0/1804890909 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a289bba7d0 0x55a289b8da00 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:47.960 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.952+0000 7f75d25ef740 1 -- 172.21.15.169:0/1804890909 >> 172.21.15.169:0/1804890909 conn(0x55a289b8fc70 msgr2=0x55a289bd8560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:47.960 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.952+0000 7f75d25ef740 1 -- 172.21.15.169:0/1804890909 shutdown_connections 2023-10-26T01:33:47.960 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.952+0000 7f75d25ef740 1 -- 172.21.15.169:0/1804890909 wait complete. 2023-10-26T01:33:47.960 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.953+0000 7f75d25ef740 1 -- start start 2023-10-26T01:33:47.960 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.953+0000 7f75d25ef740 1 Processor -- start 2023-10-26T01:33:47.964 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.954+0000 7f75d25ef740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a289bba7d0 0x55a289ba8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:47.964 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.954+0000 7f75d25ef740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a289bbabb0 0x55a289bbfa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:47.964 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.954+0000 7f75d25ef740 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x55a289bbff80 0x55a289bd2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:47.964 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.954+0000 7f75d25ef740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a289a97970 con 0x55a289bbabb0 2023-10-26T01:33:47.965 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.954+0000 7f75d25ef740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a289a97610 con 0x55a289bba7d0 2023-10-26T01:33:47.965 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.954+0000 7f75d25ef740 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a28985df20 con 0x55a289bbff80 2023-10-26T01:33:47.965 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.954+0000 7f75d25ef740 10 client.0 _ll_register_callbacks cb 0x55a289abb0c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T01:33:47.965 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.954+0000 7f75ccf1e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a289bbabb0 0x55a289bbfa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:47.965 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.954+0000 7f75ccf1e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a289bbabb0 0x55a289bbfa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:59458/0 (socket says 172.21.15.169:59458) 2023-10-26T01:33:47.966 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.954+0000 7f75ccf1e640 1 -- 172.21.15.169:0/476023015 learned_addr learned my addr 172.21.15.169:0/476023015 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:33:47.966 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.955+0000 7f75cdf20640 1 --2- 172.21.15.169:0/476023015 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x55a289bbff80 0x55a289bd2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:47.966 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.955+0000 7f75cd71f640 1 --2- 172.21.15.169:0/476023015 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a289bba7d0 0x55a289ba8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:47.966 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.955+0000 7f75ccf1e640 1 -- 172.21.15.169:0/476023015 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x55a289bbff80 msgr2=0x55a289bd2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T01:33:47.966 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.955+0000 7f75ccf1e640 1 --2- 172.21.15.169:0/476023015 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x55a289bbff80 0x55a289bd2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:47.967 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.955+0000 7f75ccf1e640 1 -- 172.21.15.169:0/476023015 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a289bba7d0 msgr2=0x55a289ba8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T01:33:47.967 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.955+0000 7f75ccf1e640 1 --2- 172.21.15.169:0/476023015 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a289bba7d0 0x55a289ba8710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:47.967 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.955+0000 7f75ccf1e640 1 -- 172.21.15.169:0/476023015 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75c815c1d0 con 0x55a289bbabb0 2023-10-26T01:33:47.967 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.955+0000 7f75ccf1e640 1 --2- 172.21.15.169:0/476023015 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a289bbabb0 0x55a289bbfa40 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f75b80099f0 tx=0x7f75b80045c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cb9e0438b7e0f30 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:47.967 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.955+0000 7f75a3fff640 10 client.0 ms_handle_connect on v2:172.21.15.169:3300/0 2023-10-26T01:33:47.968 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.955+0000 7f75a3fff640 1 -- 172.21.15.169:0/476023015 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75b802ccc0 con 0x55a289bbabb0 2023-10-26T01:33:47.968 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.955+0000 7f75d25ef740 1 -- 172.21.15.169:0/476023015 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a289af2700 con 0x55a289bbabb0 2023-10-26T01:33:47.968 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.955+0000 7f75a3fff640 1 -- 172.21.15.169:0/476023015 <== mon.0 v2:172.21.15.169:3300/0 2 ==== osd_map(65..65 src has 1..65) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f75b802ce20 con 0x55a289bbabb0 2023-10-26T01:33:47.968 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.956+0000 7f75a3fff640 1 -- 172.21.15.169:0/476023015 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f75b80035f0 con 0x55a289bbabb0 2023-10-26T01:33:47.968 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.956+0000 7f75d25ef740 10 client.6231 fetch_fsmap learned FSMap version 32 2023-10-26T01:33:47.969 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.956+0000 7f75d25ef740 1 -- 172.21.15.169:0/476023015 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) v3 -- 0x55a289af7da0 con 0x55a289bbabb0 2023-10-26T01:33:47.969 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.956+0000 7f75a3fff640 1 -- 172.21.15.169:0/476023015 <== mon.0 v2:172.21.15.169:3300/0 4 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x55a289af2700 con 0x55a289bbabb0 2023-10-26T01:33:47.969 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.956+0000 7f75a3fff640 1 -- 172.21.15.169:0/476023015 <== mon.0 v2:172.21.15.169:3300/0 5 ==== fsmap(e 32) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f75b803c050 con 0x55a289bbabb0 2023-10-26T01:33:47.969 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.956+0000 7f75d25ef740 10 client.6231 fetch_fsmap finished waiting for FSMap version 32 2023-10-26T01:33:47.969 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.956+0000 7f75d25ef740 10 client.6231 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-26T01:33:47.969 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.956+0000 7f75d25ef740 20 client.6231 populate_metadata read hostname 'smithi169' 2023-10-26T01:33:47.970 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.956+0000 7f75d25ef740 1 --2- 172.21.15.169:0/476023015 >> [v2:172.21.15.169:6834/2338833513,v1:172.21.15.169:6835/2338833513] conn(0x55a289baae70 0x55a289ba92b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:47.970 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.956+0000 7f75d25ef740 4 client.6231 mds_command: new command op to 6093 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-26T01:33:47.970 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.956+0000 7f75d25ef740 1 -- 172.21.15.169:0/476023015 --> [v2:172.21.15.169:6834/2338833513,v1:172.21.15.169:6835/2338833513] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a289af2420 con 0x55a289baae70 2023-10-26T01:33:47.970 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.957+0000 7f75cd71f640 1 --2- 172.21.15.169:0/476023015 >> [v2:172.21.15.169:6834/2338833513,v1:172.21.15.169:6835/2338833513] conn(0x55a289baae70 0x55a289ba92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:47.970 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.957+0000 7f75cd71f640 1 --2- 172.21.15.169:0/476023015 >> [v2:172.21.15.169:6834/2338833513,v1:172.21.15.169:6835/2338833513] conn(0x55a289baae70 0x55a289ba92b0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=acb11b108c0fce3 server_cookie=4d985c93656cdba in_seq=0 out_seq=0 2023-10-26T01:33:47.971 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.957+0000 7f75a3fff640 10 client.6231 ms_handle_connect on v2:172.21.15.169:6834/2338833513 2023-10-26T01:33:47.971 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.958+0000 7f75a3fff640 1 -- 172.21.15.169:0/476023015 <== mds.0 v2:172.21.15.169:6834/2338833513 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a289af2420 con 0x55a289baae70 2023-10-26T01:33:47.971 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.958+0000 7f75a3fff640 10 client.6231 handle_command_reply: tid=0 2023-10-26T01:33:47.971 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.958+0000 7f75d25ef740 1 client.6231 shutdown 2023-10-26T01:33:47.971 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.958+0000 7f75d25ef740 1 -- 172.21.15.169:0/476023015 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a289bbabb0 msgr2=0x55a289bbfa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:47.972 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.958+0000 7f75d25ef740 1 --2- 172.21.15.169:0/476023015 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a289bbabb0 0x55a289bbfa40 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f75b80099f0 tx=0x7f75b80045c0 comp rx=0 tx=0).stop 2023-10-26T01:33:47.972 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.958+0000 7f75d25ef740 1 -- 172.21.15.169:0/476023015 shutdown_connections 2023-10-26T01:33:47.972 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.958+0000 7f75d25ef740 1 --2- 172.21.15.169:0/476023015 >> [v2:172.21.15.169:6834/2338833513,v1:172.21.15.169:6835/2338833513] conn(0x55a289baae70 0x55a289ba92b0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:47.972 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.958+0000 7f75d25ef740 1 --2- 172.21.15.169:0/476023015 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a289bbabb0 0x55a289bbfa40 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:47.972 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.958+0000 7f75d25ef740 1 --2- 172.21.15.169:0/476023015 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x55a289bbff80 0x55a289bd2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:47.973 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.958+0000 7f75a3fff640 0 client.6231 ms_handle_reset on v2:172.21.15.169:6834/2338833513 2023-10-26T01:33:47.973 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.958+0000 7f75d25ef740 1 --2- 172.21.15.169:0/476023015 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a289bba7d0 0x55a289ba8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:47.973 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.958+0000 7f75d25ef740 1 -- 172.21.15.169:0/476023015 >> 172.21.15.169:0/476023015 conn(0x55a289b8df40 msgr2=0x55a289bbd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:47.973 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.959+0000 7f75d25ef740 1 -- 172.21.15.169:0/476023015 shutdown_connections 2023-10-26T01:33:47.973 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.959+0000 7f75d25ef740 1 -- 172.21.15.169:0/476023015 wait complete. 2023-10-26T01:33:47.974 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.959+0000 7f75d25ef740 20 client.6231 trim_cache size 0 max 16384 2023-10-26T01:33:47.977 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.976+0000 7f75d25ef740 1 Processor -- start 2023-10-26T01:33:47.977 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.976+0000 7f75d25ef740 1 -- start start 2023-10-26T01:33:47.979 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.977+0000 7f75d25ef740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a289bba7d0 0x55a289bbd670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:47.979 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.977+0000 7f75d25ef740 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x55a289bbabb0 0x55a289bd1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:47.979 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.977+0000 7f75d25ef740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a289bbff80 0x55a289b916c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:47.980 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.977+0000 7f75d25ef740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a289bd44d0 con 0x55a289bbff80 2023-10-26T01:33:47.980 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.977+0000 7f75d25ef740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a289bc05a0 con 0x55a289bba7d0 2023-10-26T01:33:47.980 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.977+0000 7f75d25ef740 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a289baa990 con 0x55a289bbabb0 2023-10-26T01:33:47.980 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.977+0000 7f75cdf20640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a289bbff80 0x55a289b916c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:47.980 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.977+0000 7f75cdf20640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a289bbff80 0x55a289b916c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:59466/0 (socket says 172.21.15.169:59466) 2023-10-26T01:33:47.981 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.977+0000 7f75cdf20640 1 -- 172.21.15.169:0/3897069697 learned_addr learned my addr 172.21.15.169:0/3897069697 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:33:47.981 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.977+0000 7f75cd71f640 1 --2- 172.21.15.169:0/3897069697 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a289bba7d0 0x55a289bbd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:47.981 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.977+0000 7f75cdf20640 1 -- 172.21.15.169:0/3897069697 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x55a289bbabb0 msgr2=0x55a289bd1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T01:33:47.981 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.977+0000 7f75cdf20640 1 --2- 172.21.15.169:0/3897069697 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x55a289bbabb0 0x55a289bd1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:47.981 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.977+0000 7f75cdf20640 1 -- 172.21.15.169:0/3897069697 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a289bba7d0 msgr2=0x55a289bbd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T01:33:47.981 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.977+0000 7f75cdf20640 1 --2- 172.21.15.169:0/3897069697 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a289bba7d0 0x55a289bbd670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:47.982 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.977+0000 7f75cdf20640 1 -- 172.21.15.169:0/3897069697 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a289a96b80 con 0x55a289bbff80 2023-10-26T01:33:47.982 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.978+0000 7f75cdf20640 1 --2- 172.21.15.169:0/3897069697 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a289bbff80 0x55a289b916c0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f75c00605f0 tx=0x7f75c000c710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9476a2099f720ba server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:47.986 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.978+0000 7f75ab7fe640 1 -- 172.21.15.169:0/3897069697 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75c0056bc0 con 0x55a289bbff80 2023-10-26T01:33:47.986 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.978+0000 7f75ab7fe640 1 -- 172.21.15.169:0/3897069697 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f75c0054f30 con 0x55a289bbff80 2023-10-26T01:33:47.986 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.978+0000 7f75ab7fe640 1 -- 172.21.15.169:0/3897069697 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75c0058ce0 con 0x55a289bbff80 2023-10-26T01:33:47.986 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.978+0000 7f75d25ef740 1 -- 172.21.15.169:0/3897069697 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a289bbff80 msgr2=0x55a289b916c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:47.987 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.978+0000 7f75d25ef740 1 --2- 172.21.15.169:0/3897069697 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a289bbff80 0x55a289b916c0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f75c00605f0 tx=0x7f75c000c710 comp rx=0 tx=0).stop 2023-10-26T01:33:47.987 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.978+0000 7f75d25ef740 1 -- 172.21.15.169:0/3897069697 shutdown_connections 2023-10-26T01:33:47.987 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.978+0000 7f75d25ef740 1 --2- 172.21.15.169:0/3897069697 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a289bbff80 0x55a289b916c0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:47.987 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.978+0000 7f75d25ef740 1 --2- 172.21.15.169:0/3897069697 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x55a289bbabb0 0x55a289bd1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:47.987 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.978+0000 7f75d25ef740 1 --2- 172.21.15.169:0/3897069697 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a289bba7d0 0x55a289bbd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:47.988 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.978+0000 7f75d25ef740 1 -- 172.21.15.169:0/3897069697 >> 172.21.15.169:0/3897069697 conn(0x55a289b8df40 msgr2=0x55a289bb9e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:47.988 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.978+0000 7f75d25ef740 1 -- 172.21.15.169:0/3897069697 shutdown_connections 2023-10-26T01:33:47.988 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.979+0000 7f75d25ef740 1 -- 172.21.15.169:0/3897069697 wait complete. 2023-10-26T01:33:47.988 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.979+0000 7f75d25ef740 1 -- start start 2023-10-26T01:33:47.988 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.979+0000 7f75d25ef740 1 Processor -- start 2023-10-26T01:33:47.988 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.980+0000 7f75d25ef740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a289bba7d0 0x55a289b931d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:47.989 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.980+0000 7f75d25ef740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a289bbabb0 0x55a289bd2440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:47.989 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.980+0000 7f75d25ef740 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x55a289bbff80 0x55a289bd2980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:47.989 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.980+0000 7f75d25ef740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a289bd6970 con 0x55a289bba7d0 2023-10-26T01:33:47.989 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.980+0000 7f75d25ef740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a289b93710 con 0x55a289bbabb0 2023-10-26T01:33:47.989 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.980+0000 7f75d25ef740 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a289bd2ec0 con 0x55a289bbff80 2023-10-26T01:33:47.990 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.980+0000 7f75d25ef740 10 client.0 _ll_register_callbacks cb 0x7f75c40013e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T01:33:47.990 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.981+0000 7f75cd71f640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a289bba7d0 0x55a289b931d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:47.990 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.981+0000 7f75cd71f640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a289bba7d0 0x55a289b931d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:59480/0 (socket says 172.21.15.169:59480) 2023-10-26T01:33:47.990 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.981+0000 7f75cd71f640 1 -- 172.21.15.169:0/1613553821 learned_addr learned my addr 172.21.15.169:0/1613553821 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:33:47.990 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.981+0000 7f75cdf20640 1 --2- 172.21.15.169:0/1613553821 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x55a289bbff80 0x55a289bd2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:47.991 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.981+0000 7f75cd71f640 1 -- 172.21.15.169:0/1613553821 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x55a289bbff80 msgr2=0x55a289bd2980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T01:33:47.991 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.981+0000 7f75cd71f640 1 --2- 172.21.15.169:0/1613553821 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x55a289bbff80 0x55a289bd2980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:47.991 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.981+0000 7f75cd71f640 1 -- 172.21.15.169:0/1613553821 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a289bbabb0 msgr2=0x55a289bd2440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T01:33:47.991 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.981+0000 7f75cd71f640 1 --2- 172.21.15.169:0/1613553821 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a289bbabb0 0x55a289bd2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:47.991 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.981+0000 7f75cd71f640 1 -- 172.21.15.169:0/1613553821 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75c80cab10 con 0x55a289bba7d0 2023-10-26T01:33:47.992 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.981+0000 7f75cd71f640 1 --2- 172.21.15.169:0/1613553821 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a289bba7d0 0x55a289b931d0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f75c400db40 tx=0x7f75c4001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15360ba4c995241b server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:47.992 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.981+0000 7f75a3fff640 10 client.0 ms_handle_connect on v2:172.21.15.169:3300/0 2023-10-26T01:33:47.992 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.981+0000 7f75a3fff640 1 -- 172.21.15.169:0/1613553821 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75c401c040 con 0x55a289bba7d0 2023-10-26T01:33:47.992 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.982+0000 7f75a3fff640 1 -- 172.21.15.169:0/1613553821 <== mon.0 v2:172.21.15.169:3300/0 2 ==== osd_map(65..65 src has 1..65) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f75c4029810 con 0x55a289bba7d0 2023-10-26T01:33:47.992 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.982+0000 7f75d25ef740 1 -- 172.21.15.169:0/1613553821 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f75c4003610 con 0x55a289bba7d0 2023-10-26T01:33:47.993 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.982+0000 7f75a3fff640 1 -- 172.21.15.169:0/1613553821 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f75c40295d0 con 0x55a289bba7d0 2023-10-26T01:33:47.993 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.982+0000 7f75d25ef740 10 client.6237 fetch_fsmap learned FSMap version 32 2023-10-26T01:33:47.993 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.982+0000 7f75d25ef740 1 -- 172.21.15.169:0/1613553821 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) v3 -- 0x55a289abc170 con 0x55a289bba7d0 2023-10-26T01:33:47.993 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.982+0000 7f75a3fff640 1 -- 172.21.15.169:0/1613553821 <== mon.0 v2:172.21.15.169:3300/0 4 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f75c4003610 con 0x55a289bba7d0 2023-10-26T01:33:47.993 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.982+0000 7f75a3fff640 1 -- 172.21.15.169:0/1613553821 <== mon.0 v2:172.21.15.169:3300/0 5 ==== fsmap(e 32) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f75c40299d0 con 0x55a289bba7d0 2023-10-26T01:33:47.994 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.982+0000 7f75d25ef740 10 client.6237 fetch_fsmap finished waiting for FSMap version 32 2023-10-26T01:33:47.994 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.982+0000 7f75d25ef740 10 client.6237 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-26T01:33:47.994 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.982+0000 7f75d25ef740 20 client.6237 populate_metadata read hostname 'smithi169' 2023-10-26T01:33:47.994 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.982+0000 7f75d25ef740 1 --2- 172.21.15.169:0/1613553821 >> [v2:172.21.15.169:6834/2338833513,v1:172.21.15.169:6835/2338833513] conn(0x55a289bce380 0x55a289bd97b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:47.994 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.982+0000 7f75d25ef740 4 client.6237 mds_command: new command op to 6093 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-26T01:33:47.995 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.982+0000 7f75d25ef740 1 -- 172.21.15.169:0/1613553821 --> [v2:172.21.15.169:6834/2338833513,v1:172.21.15.169:6835/2338833513] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a2897fb6c0 con 0x55a289bce380 2023-10-26T01:33:47.995 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.982+0000 7f75ccf1e640 1 --2- 172.21.15.169:0/1613553821 >> [v2:172.21.15.169:6834/2338833513,v1:172.21.15.169:6835/2338833513] conn(0x55a289bce380 0x55a289bd97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:47.995 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.983+0000 7f75a3fff640 10 client.6237 ms_handle_connect on v2:172.21.15.169:6834/2338833513 2023-10-26T01:33:47.995 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.983+0000 7f75ccf1e640 1 --2- 172.21.15.169:0/1613553821 >> [v2:172.21.15.169:6834/2338833513,v1:172.21.15.169:6835/2338833513] conn(0x55a289bce380 0x55a289bd97b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15ac607af9332c70 server_cookie=38fa63ad196314c0 in_seq=0 out_seq=0 2023-10-26T01:33:47.995 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.983+0000 7f75a3fff640 1 -- 172.21.15.169:0/1613553821 <== mds.0 v2:172.21.15.169:6834/2338833513 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55a2897fb6c0 con 0x55a289bce380 2023-10-26T01:33:47.996 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.983+0000 7f75a3fff640 10 client.6237 handle_command_reply: tid=0 2023-10-26T01:33:47.996 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.983+0000 7f75d25ef740 1 client.6237 shutdown 2023-10-26T01:33:47.996 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.983+0000 7f75d25ef740 1 -- 172.21.15.169:0/1613553821 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a289bba7d0 msgr2=0x55a289b931d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:47.996 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.983+0000 7f75d25ef740 1 --2- 172.21.15.169:0/1613553821 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a289bba7d0 0x55a289b931d0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f75c400db40 tx=0x7f75c4001120 comp rx=0 tx=0).stop 2023-10-26T01:33:47.996 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.983+0000 7f75d25ef740 1 -- 172.21.15.169:0/1613553821 shutdown_connections 2023-10-26T01:33:47.996 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.983+0000 7f75d25ef740 1 --2- 172.21.15.169:0/1613553821 >> [v2:172.21.15.169:6834/2338833513,v1:172.21.15.169:6835/2338833513] conn(0x55a289bce380 0x55a289bd97b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:47.997 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.983+0000 7f75d25ef740 1 --2- 172.21.15.169:0/1613553821 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x55a289bbff80 0x55a289bd2980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:47.997 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.983+0000 7f75a3fff640 0 client.6237 ms_handle_reset on v2:172.21.15.169:6834/2338833513 2023-10-26T01:33:47.997 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.983+0000 7f75d25ef740 1 --2- 172.21.15.169:0/1613553821 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a289bbabb0 0x55a289bd2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:47.997 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.983+0000 7f75d25ef740 1 --2- 172.21.15.169:0/1613553821 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a289bba7d0 0x55a289b931d0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:47.997 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.983+0000 7f75d25ef740 1 -- 172.21.15.169:0/1613553821 >> 172.21.15.169:0/1613553821 conn(0x55a289b8df40 msgr2=0x55a289bb8d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:47.998 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.984+0000 7f75d25ef740 1 -- 172.21.15.169:0/1613553821 shutdown_connections 2023-10-26T01:33:47.998 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.984+0000 7f75d25ef740 1 -- 172.21.15.169:0/1613553821 wait complete. 2023-10-26T01:33:47.998 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.984+0000 7f75d25ef740 20 client.6237 trim_cache size 0 max 16384 2023-10-26T01:33:47.998 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2023-10-26T01:33:47.999 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_fsid": "30203b28-ed8a-44e8-a9a6-a9bf25f980f1", 2023-10-26T01:33:47.999 INFO:teuthology.orchestra.run.smithi169.stdout: "whoami": 0, 2023-10-26T01:33:47.999 INFO:teuthology.orchestra.run.smithi169.stdout: "id": 6093, 2023-10-26T01:33:47.999 INFO:teuthology.orchestra.run.smithi169.stdout: "want_state": "up:active", 2023-10-26T01:33:47.999 INFO:teuthology.orchestra.run.smithi169.stdout: "state": "up:active", 2023-10-26T01:33:48.000 INFO:teuthology.orchestra.run.smithi169.stdout: "fs_name": "cephfs", 2023-10-26T01:33:48.000 INFO:teuthology.orchestra.run.smithi169.stdout: "rank_uptime": 6.6835354999999996, 2023-10-26T01:33:48.000 INFO:teuthology.orchestra.run.smithi169.stdout: "mdsmap_epoch": 32, 2023-10-26T01:33:48.000 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_epoch": 65, 2023-10-26T01:33:48.000 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_epoch_barrier": 65, 2023-10-26T01:33:48.001 INFO:teuthology.orchestra.run.smithi169.stdout: "uptime": 12.520075694999999 2023-10-26T01:33:48.001 INFO:teuthology.orchestra.run.smithi169.stdout:} 2023-10-26T01:33:48.002 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.984+0000 7f75cf983640 1 -- 172.21.15.169:0/2606884186 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f75ac06ae40 msgr2=0x7f75ac06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:48.002 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.984+0000 7f75cf983640 1 --2- 172.21.15.169:0/2606884186 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f75ac06ae40 0x7f75ac06d300 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f75c4000f60 tx=0x7f75c4009040 comp rx=0 tx=0).stop 2023-10-26T01:33:48.003 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.984+0000 7f75cf983640 1 -- 172.21.15.169:0/2606884186 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75c815bbb0 msgr2=0x7f75c81e5a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:48.003 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.984+0000 7f75cf983640 1 --2- 172.21.15.169:0/2606884186 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75c815bbb0 0x7f75c81e5a70 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f75c000c780 tx=0x7f75c000cc40 comp rx=0 tx=0).stop 2023-10-26T01:33:48.003 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.984+0000 7f75cf983640 1 -- 172.21.15.169:0/2606884186 shutdown_connections 2023-10-26T01:33:48.003 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.984+0000 7f75cf983640 1 --2- 172.21.15.169:0/2606884186 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f75ac06ae40 0x7f75ac06d300 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:48.003 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.984+0000 7f75cf983640 1 --2- 172.21.15.169:0/2606884186 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75c815bbb0 0x7f75c81e5a70 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:48.004 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.984+0000 7f75cf983640 1 --2- 172.21.15.169:0/2606884186 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f75c80b05a0 0x7f75c81e5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:48.004 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.984+0000 7f75cf983640 1 --2- 172.21.15.169:0/2606884186 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f75c806ce00 0x7f75c8096910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:48.004 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.984+0000 7f75cf983640 1 -- 172.21.15.169:0/2606884186 >> 172.21.15.169:0/2606884186 conn(0x7f75c8069930 msgr2=0x7f75c8157120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:48.004 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.984+0000 7f75cf983640 1 -- 172.21.15.169:0/2606884186 shutdown_connections 2023-10-26T01:33:48.004 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:47.984+0000 7f75cf983640 1 -- 172.21.15.169:0/2606884186 wait complete. 2023-10-26T01:33:48.005 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-26T01:33:48.213 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:48.211+0000 7f070a8c2640 1 -- 172.21.15.169:0/3065327229 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07040713d0 msgr2=0x7f0704070730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:48.214 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:48.211+0000 7f070a8c2640 1 --2- 172.21.15.169:0/3065327229 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07040713d0 0x7f0704070730 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f070406a560 tx=0x7f06ec02be40 comp rx=0 tx=0).stop 2023-10-26T01:33:48.214 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:48.211+0000 7f070a8c2640 1 -- 172.21.15.169:0/3065327229 shutdown_connections 2023-10-26T01:33:48.214 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:48.211+0000 7f070a8c2640 1 --2- 172.21.15.169:0/3065327229 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07040713d0 0x7f0704070730 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:48.214 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:48.211+0000 7f070a8c2640 1 --2- 172.21.15.169:0/3065327229 >> v2:172.21.15.170:3300/0 conn(0x7f070415ca70 0x7f070415d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:48.214 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:48.211+0000 7f070a8c2640 1 -- 172.21.15.169:0/3065327229 >> 172.21.15.169:0/3065327229 conn(0x7f07040af8a0 msgr2=0x7f07040afc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:48.215 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:48.211+0000 7f070a8c2640 1 -- 172.21.15.169:0/3065327229 shutdown_connections 2023-10-26T01:33:48.215 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:48.212+0000 7f070a8c2640 1 -- 172.21.15.169:0/3065327229 wait complete. 2023-10-26T01:33:48.217 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:48.213+0000 7f070a8c2640 1 Processor -- start 2023-10-26T01:33:48.217 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:48.213+0000 7f070a8c2640 1 -- start start 2023-10-26T01:33:48.217 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:48.214+0000 7f070a8c2640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0704070c70 0x7f07040b2830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:48.218 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:48.214+0000 7f070a8c2640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f07040713d0 0x7f070415b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:48.218 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:48.214+0000 7f070a8c2640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f070415ca70 0x7f07040adf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:48.218 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:48.214+0000 7f070a8c2640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0704146e50 con 0x7f0704070c70 2023-10-26T01:33:48.218 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:48.214+0000 7f070a8c2640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f070415bbb0 con 0x7f070415ca70 2023-10-26T01:33:48.218 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:48.214+0000 7f070a8c2640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07040ae460 con 0x7f07040713d0 2023-10-26T01:33:48.219 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:48.214+0000 7f0703fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0704070c70 0x7f07040b2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:48.219 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:48.215+0000 7f0703fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0704070c70 0x7f07040b2830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:59502/0 (socket says 172.21.15.169:59502) 2023-10-26T01:33:48.219 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:48.215+0000 7f0703fff640 1 -- 172.21.15.169:0/214508896 learned_addr learned my addr 172.21.15.169:0/214508896 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:33:48.219 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:48.215+0000 7f06fb7fe640 1 --2- 172.21.15.169:0/214508896 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f07040713d0 0x7f070415b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:48.219 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:48.215+0000 7f0708e5f640 1 --2- 172.21.15.169:0/214508896 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f070415ca70 0x7f07040adf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:48.219 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:48.215+0000 7f0703fff640 1 -- 172.21.15.169:0/214508896 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f07040713d0 msgr2=0x7f070415b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:48.220 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:48.215+0000 7f0703fff640 1 --2- 172.21.15.169:0/214508896 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f07040713d0 0x7f070415b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:48.220 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:48.215+0000 7f0703fff640 1 -- 172.21.15.169:0/214508896 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f070415ca70 msgr2=0x7f07040adf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:48.220 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:48.215+0000 7f0703fff640 1 --2- 172.21.15.169:0/214508896 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f070415ca70 0x7f07040adf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:48.220 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:48.215+0000 7f0703fff640 1 -- 172.21.15.169:0/214508896 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06ec009560 con 0x7f0704070c70 2023-10-26T01:33:48.220 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:48.215+0000 7f0708e5f640 1 --2- 172.21.15.169:0/214508896 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f070415ca70 0x7f07040adf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:33:48.220 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:48.215+0000 7f06fb7fe640 1 --2- 172.21.15.169:0/214508896 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f07040713d0 0x7f070415b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:33:48.221 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:48.215+0000 7f0703fff640 1 --2- 172.21.15.169:0/214508896 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0704070c70 0x7f07040b2830 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f06f400d770 tx=0x7f06f400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:48.221 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:48.215+0000 7f07017fa640 1 -- 172.21.15.169:0/214508896 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06f4009c30 con 0x7f0704070c70 2023-10-26T01:33:48.221 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:48.215+0000 7f07017fa640 1 -- 172.21.15.169:0/214508896 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f06f4009e80 con 0x7f0704070c70 2023-10-26T01:33:48.221 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:48.216+0000 7f07017fa640 1 -- 172.21.15.169:0/214508896 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06f40106c0 con 0x7f0704070c70 2023-10-26T01:33:48.221 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:48.216+0000 7f070a8c2640 1 -- 172.21.15.169:0/214508896 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f070415be00 con 0x7f0704070c70 2023-10-26T01:33:48.222 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:48.216+0000 7f070a8c2640 1 -- 172.21.15.169:0/214508896 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07040b0ca0 con 0x7f0704070c70 2023-10-26T01:33:48.222 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:48.217+0000 7f070a8c2640 1 -- 172.21.15.169:0/214508896 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f070404db10 con 0x7f0704070c70 2023-10-26T01:33:48.223 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:48.218+0000 7f07017fa640 1 -- 172.21.15.169:0/214508896 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f06f4010820 con 0x7f0704070c70 2023-10-26T01:33:48.223 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:48.218+0000 7f07017fa640 1 --2- 172.21.15.169:0/214508896 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f06e406ae70 0x7f06e406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:48.223 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:48.218+0000 7f07017fa640 1 -- 172.21.15.169:0/214508896 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f06f4089ba0 con 0x7f0704070c70 2023-10-26T01:33:48.223 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:48.219+0000 7f06fb7fe640 1 --2- 172.21.15.169:0/214508896 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f06e406ae70 0x7f06e406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:48.223 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:48.220+0000 7f06fb7fe640 1 --2- 172.21.15.169:0/214508896 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f06e406ae70 0x7f06e406d330 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f06ec035000 tx=0x7f06ec004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:48.224 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:48.223+0000 7f07017fa640 1 -- 172.21.15.169:0/214508896 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f06f4054bb0 con 0x7f0704070c70 2023-10-26T01:33:48.455 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:48.453+0000 7f070a8c2640 1 -- 172.21.15.169:0/214508896 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f07041e7b90 con 0x7f0704070c70 2023-10-26T01:33:48.456 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:48.454+0000 7f07017fa640 1 -- 172.21.15.169:0/214508896 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v32) v1 ==== 53+0+73 (secure 0 0 0) 0x7f06f4021070 con 0x7f0704070c70 2023-10-26T01:33:48.456 INFO:teuthology.orchestra.run.smithi169.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-26T01:33:48.459 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:48.457+0000 7f070a8c2640 1 -- 172.21.15.169:0/214508896 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f06e406ae70 msgr2=0x7f06e406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:48.459 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:48.457+0000 7f070a8c2640 1 --2- 172.21.15.169:0/214508896 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f06e406ae70 0x7f06e406d330 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f06ec035000 tx=0x7f06ec004730 comp rx=0 tx=0).stop 2023-10-26T01:33:48.459 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:48.457+0000 7f070a8c2640 1 -- 172.21.15.169:0/214508896 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0704070c70 msgr2=0x7f07040b2830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:48.460 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:48.457+0000 7f070a8c2640 1 --2- 172.21.15.169:0/214508896 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0704070c70 0x7f07040b2830 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f06f400d770 tx=0x7f06f400dc30 comp rx=0 tx=0).stop 2023-10-26T01:33:48.460 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:48.457+0000 7f070a8c2640 1 -- 172.21.15.169:0/214508896 shutdown_connections 2023-10-26T01:33:48.460 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:48.457+0000 7f070a8c2640 1 --2- 172.21.15.169:0/214508896 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f06e406ae70 0x7f06e406d330 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:48.460 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:48.457+0000 7f070a8c2640 1 --2- 172.21.15.169:0/214508896 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f070415ca70 0x7f07040adf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:48.460 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:48.457+0000 7f070a8c2640 1 --2- 172.21.15.169:0/214508896 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f07040713d0 0x7f070415b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:48.461 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:48.457+0000 7f070a8c2640 1 --2- 172.21.15.169:0/214508896 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0704070c70 0x7f07040b2830 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:48.461 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:48.457+0000 7f070a8c2640 1 -- 172.21.15.169:0/214508896 >> 172.21.15.169:0/214508896 conn(0x7f07040af8a0 msgr2=0x7f070414f520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:48.461 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:48.457+0000 7f070a8c2640 1 -- 172.21.15.169:0/214508896 shutdown_connections 2023-10-26T01:33:48.461 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:48.458+0000 7f070a8c2640 1 -- 172.21.15.169:0/214508896 wait complete. 2023-10-26T01:33:48.476 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-26T01:33:48.477 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-26T01:33:48.477 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi169.front.sepia.ceph.com 2023-10-26T01:33:48.477 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-26T01:33:48.478 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-26T01:33:48.478 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-26T01:33:48.478 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-26T01:33:48.479 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-26T01:33:48.479 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-26T01:33:48.479 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:33:48.480 DEBUG:teuthology.orchestra.run.smithi169:> ip addr 2023-10-26T01:33:48.531 INFO:teuthology.orchestra.run.smithi169.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-26T01:33:48.531 INFO:teuthology.orchestra.run.smithi169.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-26T01:33:48.531 INFO:teuthology.orchestra.run.smithi169.stdout: inet 127.0.0.1/8 scope host lo 2023-10-26T01:33:48.532 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:33:48.532 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 ::1/128 scope host 2023-10-26T01:33:48.532 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:33:48.532 INFO:teuthology.orchestra.run.smithi169.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-26T01:33:48.532 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:d9:3b:b4 brd ff:ff:ff:ff:ff:ff 2023-10-26T01:33:48.533 INFO:teuthology.orchestra.run.smithi169.stdout: altname enp7s0f0 2023-10-26T01:33:48.533 INFO:teuthology.orchestra.run.smithi169.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-26T01:33:48.533 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:d9:3b:b5 brd ff:ff:ff:ff:ff:ff 2023-10-26T01:33:48.533 INFO:teuthology.orchestra.run.smithi169.stdout: altname enp7s0f1 2023-10-26T01:33:48.533 INFO:teuthology.orchestra.run.smithi169.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-26T01:33:48.534 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:88:7f:4c brd ff:ff:ff:ff:ff:ff 2023-10-26T01:33:48.534 INFO:teuthology.orchestra.run.smithi169.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-26T01:33:48.534 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:88:7f:4d brd ff:ff:ff:ff:ff:ff 2023-10-26T01:33:48.534 INFO:teuthology.orchestra.run.smithi169.stdout: inet 172.21.15.169/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-26T01:33:48.534 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft 42267sec preferred_lft 42267sec 2023-10-26T01:33:48.535 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 fe80::ec4:7aff:fe88:7f4d/64 scope link 2023-10-26T01:33:48.535 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:33:48.535 INFO:teuthology.orchestra.run.smithi169.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-26T01:33:48.535 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 22:a0:6b:43:7b:04 brd ff:ff:ff:ff:ff:ff 2023-10-26T01:33:48.535 INFO:teuthology.orchestra.run.smithi169.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-26T01:33:48.536 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:33:48.536 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 fe80::d815:31ff:fe87:d148/64 scope link 2023-10-26T01:33:48.536 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:33:48.536 INFO:teuthology.orchestra.run.smithi169.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-26T01:33:48.536 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 2a:80:f8:45:5f:9f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-26T01:33:48.536 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 fe80::2880:f8ff:fe45:5f9f/64 scope link 2023-10-26T01:33:48.537 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:33:48.537 INFO:teuthology.orchestra.run.smithi169.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-26T01:33:48.537 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 22:a0:6b:43:7b:04 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-26T01:33:48.537 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 fe80::20a0:6bff:fe43:7b04/64 scope link 2023-10-26T01:33:48.538 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:33:48.538 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:33:48.538 DEBUG:teuthology.orchestra.run.smithi169:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-26T01:33:48.622 INFO:teuthology.orchestra.run.smithi169.stdout:1 2023-10-26T01:33:48.626 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:33:48.626 DEBUG:teuthology.orchestra.run.smithi169:> route 2023-10-26T01:33:48.679 INFO:teuthology.orchestra.run.smithi169.stdout:Kernel IP routing table 2023-10-26T01:33:48.679 INFO:teuthology.orchestra.run.smithi169.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-26T01:33:48.679 INFO:teuthology.orchestra.run.smithi169.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-26T01:33:48.679 INFO:teuthology.orchestra.run.smithi169.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-26T01:33:48.680 INFO:teuthology.orchestra.run.smithi169.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-26T01:33:48.680 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:33:48.680 DEBUG:teuthology.orchestra.run.smithi169:> (cd / && exec bash -c ' 2023-10-26T01:33:48.681 DEBUG:teuthology.orchestra.run.smithi169:> set -e 2023-10-26T01:33:48.681 DEBUG:teuthology.orchestra.run.smithi169:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-26T01:33:48.681 DEBUG:teuthology.orchestra.run.smithi169:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-26T01:33:48.682 DEBUG:teuthology.orchestra.run.smithi169:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-26T01:33:48.682 DEBUG:teuthology.orchestra.run.smithi169:> ') 2023-10-26T01:33:48.862 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:33:48.863 DEBUG:teuthology.orchestra.run.smithi169:> ip netns list 2023-10-26T01:33:48.917 INFO:teuthology.orchestra.run.smithi169.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-26T01:33:48.917 INFO:teuthology.orchestra.run.smithi169.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-26T01:33:48.917 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-26T01:33:48.918 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:33:48.918 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link set brx.0 up 2023-10-26T01:33:49.016 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-26T01:33:49.016 DEBUG:teuthology.orchestra.run.smithi169:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-26T01:33:49.069 INFO:teuthology.orchestra.run.smithi169.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-26T01:33:49.069 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-26T01:33:49.070 DEBUG:teuthology.orchestra.run.smithi169:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-26T01:33:49.123 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-26T01:33:49.123 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:33:49.124 DEBUG:teuthology.orchestra.run.smithi169:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-26T01:33:49.270 INFO:teuthology.orchestra.run.smithi169.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-26T01:33:49.270 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-26T01:33:49.271 INFO:teuthology.orchestra.run.smithi169.stdout:invalid new device string format 2023-10-26T01:33:49.292 INFO:teuthology.orchestra.run.smithi169.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-26T01:33:49.292 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-26T01:33:49.292 INFO:teuthology.orchestra.run.smithi169.stdout:invalid new device string format 2023-10-26T01:33:49.293 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: resolved to: "172.21.15.169:3300,172.21.15.170:3300,172.21.15.170:3301" 2023-10-26T01:33:49.293 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-26T01:33:49.315 INFO:teuthology.orchestra.run.smithi169.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-26T01:33:49.315 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-26T01:33:49.315 INFO:teuthology.orchestra.run.smithi169.stdout:invalid new device string format 2023-10-26T01:33:49.316 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: resolved to: "172.21.15.169:3300,172.21.15.170:3300,172.21.15.170:3301" 2023-10-26T01:33:49.316 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-26T01:33:49.316 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: resolved to: "172.21.15.169:6789,172.21.15.170:6789,172.21.15.170:6790" 2023-10-26T01:33:49.316 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.169:6789,172.21.15.170:6789,172.21.15.170:6790:/ 2023-10-26T01:33:49.316 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=30203b28-ed8a-44e8-a9a6-a9bf25f980f1" will pass to kernel 2023-10-26T01:33:49.317 INFO:teuthology.orchestra.run.smithi169.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2023-10-26T01:33:49.317 INFO:teuthology.orchestra.run.smithi169.stdout: You just mounted a file system that supports labels which does not 2023-10-26T01:33:49.317 INFO:teuthology.orchestra.run.smithi169.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-10-26T01:33:49.317 INFO:teuthology.orchestra.run.smithi169.stdout: applications will generate AVC messages and not be allowed access to 2023-10-26T01:33:49.318 INFO:teuthology.orchestra.run.smithi169.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-10-26T01:33:49.320 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-26T01:33:49.321 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:33:49.321 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-26T01:33:49.396 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:33:49.396 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-26T01:33:49.453 INFO:teuthology.orchestra.run.smithi169.stdout:client6249 2023-10-26T01:33:49.454 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:33:49.454 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:33:49.505 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-26T01:33:49.726 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:49.722+0000 7fe50da4b640 1 -- 172.21.15.169:0/780356153 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe4f0003970 msgr2=0x7fe4f0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:49.727 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:49.722+0000 7fe50da4b640 1 --2- 172.21.15.169:0/780356153 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe4f0003970 0x7fe4f0003d90 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fe4f8006fd0 tx=0x7fe4f00057f0 comp rx=0 tx=0).stop 2023-10-26T01:33:49.727 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:49.723+0000 7fe50da4b640 1 -- 172.21.15.169:0/780356153 shutdown_connections 2023-10-26T01:33:49.728 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:49.723+0000 7fe50da4b640 1 --2- 172.21.15.169:0/780356153 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe4f0003970 0x7fe4f0003d90 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:49.728 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:49.723+0000 7fe50da4b640 1 -- 172.21.15.169:0/780356153 >> 172.21.15.169:0/780356153 conn(0x7fe508069930 msgr2=0x7fe508069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:49.728 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:49.723+0000 7fe50da4b640 1 -- 172.21.15.169:0/780356153 shutdown_connections 2023-10-26T01:33:49.728 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:49.723+0000 7fe50da4b640 1 -- 172.21.15.169:0/780356153 wait complete. 2023-10-26T01:33:49.728 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:49.724+0000 7fe50da4b640 1 Processor -- start 2023-10-26T01:33:49.729 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:49.724+0000 7fe50da4b640 1 -- start start 2023-10-26T01:33:49.730 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:49.725+0000 7fe50da4b640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe4f0003970 0x7fe5081f3160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:49.731 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:49.725+0000 7fe50da4b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe508071a40 0x7fe5081f36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:49.731 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:49.725+0000 7fe50da4b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe508157000 0x7fe5081efbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:49.731 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:49.725+0000 7fe50da4b640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe508156940 con 0x7fe508157000 2023-10-26T01:33:49.731 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:49.725+0000 7fe50da4b640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5081f4010 con 0x7fe508071a40 2023-10-26T01:33:49.731 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:49.725+0000 7fe50da4b640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5081f3d30 con 0x7fe4f0003970 2023-10-26T01:33:49.731 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:49.726+0000 7fe5077fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe508157000 0x7fe5081efbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:49.732 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:49.726+0000 7fe5077fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe508157000 0x7fe5081efbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:59526/0 (socket says 172.21.15.169:59526) 2023-10-26T01:33:49.732 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:49.726+0000 7fe5077fe640 1 -- 172.21.15.169:0/3510694736 learned_addr learned my addr 172.21.15.169:0/3510694736 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:33:49.732 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:49.726+0000 7fe506ffd640 1 --2- 172.21.15.169:0/3510694736 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe4f0003970 0x7fe5081f3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:49.732 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:49.726+0000 7fe5067fc640 1 --2- 172.21.15.169:0/3510694736 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe508071a40 0x7fe5081f36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:49.733 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:49.726+0000 7fe5077fe640 1 -- 172.21.15.169:0/3510694736 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe4f0003970 msgr2=0x7fe5081f3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:49.733 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:49.726+0000 7fe5077fe640 1 --2- 172.21.15.169:0/3510694736 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe4f0003970 0x7fe5081f3160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:49.733 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:49.726+0000 7fe5077fe640 1 -- 172.21.15.169:0/3510694736 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe508071a40 msgr2=0x7fe5081f36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:49.733 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:49.726+0000 7fe5077fe640 1 --2- 172.21.15.169:0/3510694736 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe508071a40 0x7fe5081f36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:49.733 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:49.726+0000 7fe5077fe640 1 -- 172.21.15.169:0/3510694736 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe508156660 con 0x7fe508157000 2023-10-26T01:33:49.734 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:49.726+0000 7fe5067fc640 1 --2- 172.21.15.169:0/3510694736 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe508071a40 0x7fe5081f36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:33:49.734 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:49.726+0000 7fe506ffd640 1 --2- 172.21.15.169:0/3510694736 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe4f0003970 0x7fe5081f3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:33:49.734 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:49.726+0000 7fe5077fe640 1 --2- 172.21.15.169:0/3510694736 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe508157000 0x7fe5081efbc0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fe50806baa0 tx=0x7fe50000eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:49.734 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:49.727+0000 7fe4effff640 1 -- 172.21.15.169:0/3510694736 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe50000ca20 con 0x7fe508157000 2023-10-26T01:33:49.735 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:49.727+0000 7fe4effff640 1 -- 172.21.15.169:0/3510694736 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe50000cc90 con 0x7fe508157000 2023-10-26T01:33:49.735 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:49.727+0000 7fe50da4b640 1 -- 172.21.15.169:0/3510694736 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe508097570 con 0x7fe508157000 2023-10-26T01:33:49.735 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:49.727+0000 7fe4effff640 1 -- 172.21.15.169:0/3510694736 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe500011470 con 0x7fe508157000 2023-10-26T01:33:49.735 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:49.727+0000 7fe50da4b640 1 -- 172.21.15.169:0/3510694736 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5080cac10 con 0x7fe508157000 2023-10-26T01:33:49.735 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:49.728+0000 7fe50da4b640 1 -- 172.21.15.169:0/3510694736 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5081564c0 con 0x7fe508157000 2023-10-26T01:33:49.736 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:49.729+0000 7fe4effff640 1 -- 172.21.15.169:0/3510694736 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fe5000115d0 con 0x7fe508157000 2023-10-26T01:33:49.737 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:49.730+0000 7fe4effff640 1 --2- 172.21.15.169:0/3510694736 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe4f406ae70 0x7fe4f406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:49.738 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:49.730+0000 7fe4effff640 1 -- 172.21.15.169:0/3510694736 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe500089240 con 0x7fe508157000 2023-10-26T01:33:49.738 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:49.730+0000 7fe506ffd640 1 --2- 172.21.15.169:0/3510694736 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe4f406ae70 0x7fe4f406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:49.738 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:49.732+0000 7fe506ffd640 1 --2- 172.21.15.169:0/3510694736 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe4f406ae70 0x7fe4f406d330 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fe4f8034000 tx=0x7fe4f802e000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:49.739 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:49.734+0000 7fe4effff640 1 -- 172.21.15.169:0/3510694736 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe5081564c0 con 0x7fe508157000 2023-10-26T01:33:49.966 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:49.965+0000 7fe50da4b640 1 -- 172.21.15.169:0/3510694736 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe5080973d0 con 0x7fe508157000 2023-10-26T01:33:49.967 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:49.966+0000 7fe4effff640 1 -- 172.21.15.169:0/3510694736 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v32) v1 ==== 53+0+73 (secure 0 0 0) 0x7fe5080973d0 con 0x7fe508157000 2023-10-26T01:33:49.968 INFO:teuthology.orchestra.run.smithi169.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-26T01:33:49.971 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:49.968+0000 7fe50da4b640 1 -- 172.21.15.169:0/3510694736 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe4f406ae70 msgr2=0x7fe4f406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:49.971 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:49.968+0000 7fe50da4b640 1 --2- 172.21.15.169:0/3510694736 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe4f406ae70 0x7fe4f406d330 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fe4f8034000 tx=0x7fe4f802e000 comp rx=0 tx=0).stop 2023-10-26T01:33:49.971 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:49.969+0000 7fe50da4b640 1 -- 172.21.15.169:0/3510694736 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe508157000 msgr2=0x7fe5081efbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:49.971 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:49.969+0000 7fe50da4b640 1 --2- 172.21.15.169:0/3510694736 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe508157000 0x7fe5081efbc0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fe50806baa0 tx=0x7fe50000eb90 comp rx=0 tx=0).stop 2023-10-26T01:33:49.971 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:49.969+0000 7fe50da4b640 1 -- 172.21.15.169:0/3510694736 shutdown_connections 2023-10-26T01:33:49.972 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:49.969+0000 7fe50da4b640 1 --2- 172.21.15.169:0/3510694736 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe4f406ae70 0x7fe4f406d330 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:49.972 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:49.969+0000 7fe50da4b640 1 --2- 172.21.15.169:0/3510694736 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe508157000 0x7fe5081efbc0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:49.972 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:49.969+0000 7fe50da4b640 1 --2- 172.21.15.169:0/3510694736 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe508071a40 0x7fe5081f36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:49.972 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:49.969+0000 7fe50da4b640 1 --2- 172.21.15.169:0/3510694736 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe4f0003970 0x7fe5081f3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:49.972 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:49.969+0000 7fe50da4b640 1 -- 172.21.15.169:0/3510694736 >> 172.21.15.169:0/3510694736 conn(0x7fe508069930 msgr2=0x7fe508157cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:49.973 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:49.969+0000 7fe50da4b640 1 -- 172.21.15.169:0/3510694736 shutdown_connections 2023-10-26T01:33:49.973 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:49.969+0000 7fe50da4b640 1 -- 172.21.15.169:0/3510694736 wait complete. 2023-10-26T01:33:49.989 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-26T01:33:49.989 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-26T01:33:49.990 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi170.front.sepia.ceph.com 2023-10-26T01:33:49.990 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-26T01:33:49.990 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-26T01:33:49.991 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-26T01:33:49.991 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-26T01:33:49.991 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-26T01:33:49.992 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-26T01:33:49.992 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:33:49.993 DEBUG:teuthology.orchestra.run.smithi170:> ip addr 2023-10-26T01:33:50.052 INFO:teuthology.orchestra.run.smithi170.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-26T01:33:50.053 INFO:teuthology.orchestra.run.smithi170.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-26T01:33:50.053 INFO:teuthology.orchestra.run.smithi170.stdout: inet 127.0.0.1/8 scope host lo 2023-10-26T01:33:50.053 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:33:50.054 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 ::1/128 scope host 2023-10-26T01:33:50.054 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:33:50.054 INFO:teuthology.orchestra.run.smithi170.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-26T01:33:50.054 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:d9:3b:54 brd ff:ff:ff:ff:ff:ff 2023-10-26T01:33:50.054 INFO:teuthology.orchestra.run.smithi170.stdout: altname enp7s0f0 2023-10-26T01:33:50.055 INFO:teuthology.orchestra.run.smithi170.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-26T01:33:50.055 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:d9:3b:55 brd ff:ff:ff:ff:ff:ff 2023-10-26T01:33:50.055 INFO:teuthology.orchestra.run.smithi170.stdout: altname enp7s0f1 2023-10-26T01:33:50.055 INFO:teuthology.orchestra.run.smithi170.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-26T01:33:50.056 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:88:72:54 brd ff:ff:ff:ff:ff:ff 2023-10-26T01:33:50.056 INFO:teuthology.orchestra.run.smithi170.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-26T01:33:50.056 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:88:72:55 brd ff:ff:ff:ff:ff:ff 2023-10-26T01:33:50.056 INFO:teuthology.orchestra.run.smithi170.stdout: inet 172.21.15.170/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-26T01:33:50.056 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft 42250sec preferred_lft 42250sec 2023-10-26T01:33:50.057 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::ec4:7aff:fe88:7255/64 scope link 2023-10-26T01:33:50.057 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:33:50.057 INFO:teuthology.orchestra.run.smithi170.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-26T01:33:50.057 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 32:b5:d1:ee:10:36 brd ff:ff:ff:ff:ff:ff 2023-10-26T01:33:50.057 INFO:teuthology.orchestra.run.smithi170.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-26T01:33:50.058 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:33:50.058 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::28ed:d1ff:fe37:70d/64 scope link 2023-10-26T01:33:50.058 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:33:50.058 INFO:teuthology.orchestra.run.smithi170.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-26T01:33:50.058 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 32:b5:d1:ee:10:36 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-26T01:33:50.059 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::30b5:d1ff:feee:1036/64 scope link 2023-10-26T01:33:50.059 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:33:50.059 INFO:teuthology.orchestra.run.smithi170.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-26T01:33:50.059 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether b2:c1:53:19:69:e4 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-26T01:33:50.059 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::b0c1:53ff:fe19:69e4/64 scope link 2023-10-26T01:33:50.059 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:33:50.060 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:33:50.060 DEBUG:teuthology.orchestra.run.smithi170:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-26T01:33:50.154 INFO:teuthology.orchestra.run.smithi170.stdout:1 2023-10-26T01:33:50.159 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:33:50.159 DEBUG:teuthology.orchestra.run.smithi170:> route 2023-10-26T01:33:50.213 INFO:teuthology.orchestra.run.smithi170.stdout:Kernel IP routing table 2023-10-26T01:33:50.213 INFO:teuthology.orchestra.run.smithi170.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-26T01:33:50.213 INFO:teuthology.orchestra.run.smithi170.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-26T01:33:50.214 INFO:teuthology.orchestra.run.smithi170.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-26T01:33:50.214 INFO:teuthology.orchestra.run.smithi170.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-26T01:33:50.214 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:33:50.215 DEBUG:teuthology.orchestra.run.smithi170:> (cd / && exec bash -c ' 2023-10-26T01:33:50.215 DEBUG:teuthology.orchestra.run.smithi170:> set -e 2023-10-26T01:33:50.215 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-26T01:33:50.216 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-26T01:33:50.216 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-26T01:33:50.217 DEBUG:teuthology.orchestra.run.smithi170:> ') 2023-10-26T01:33:50.403 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:33:50.403 DEBUG:teuthology.orchestra.run.smithi170:> ip netns list 2023-10-26T01:33:50.454 INFO:teuthology.orchestra.run.smithi170.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-26T01:33:50.454 INFO:teuthology.orchestra.run.smithi170.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-26T01:33:50.455 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-26T01:33:50.456 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:33:50.456 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip link set brx.0 up 2023-10-26T01:33:50.548 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-26T01:33:50.549 DEBUG:teuthology.orchestra.run.smithi170:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-26T01:33:50.604 INFO:teuthology.orchestra.run.smithi170.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-26T01:33:50.605 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-26T01:33:50.605 DEBUG:teuthology.orchestra.run.smithi170:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-26T01:33:50.653 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-26T01:33:50.654 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:33:50.654 DEBUG:teuthology.orchestra.run.smithi170:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-26T01:33:50.793 INFO:teuthology.orchestra.run.smithi170.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-26T01:33:50.793 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-26T01:33:50.793 INFO:teuthology.orchestra.run.smithi170.stdout:invalid new device string format 2023-10-26T01:33:50.814 INFO:teuthology.orchestra.run.smithi170.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-26T01:33:50.814 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-26T01:33:50.815 INFO:teuthology.orchestra.run.smithi170.stdout:invalid new device string format 2023-10-26T01:33:50.815 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: resolved to: "172.21.15.169:3300,172.21.15.170:3300,172.21.15.170:3301" 2023-10-26T01:33:50.815 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-26T01:33:50.839 INFO:teuthology.orchestra.run.smithi170.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-26T01:33:50.839 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-26T01:33:50.839 INFO:teuthology.orchestra.run.smithi170.stdout:invalid new device string format 2023-10-26T01:33:50.840 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: resolved to: "172.21.15.169:3300,172.21.15.170:3300,172.21.15.170:3301" 2023-10-26T01:33:50.840 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-26T01:33:50.840 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: resolved to: "172.21.15.169:6789,172.21.15.170:6789,172.21.15.170:6790" 2023-10-26T01:33:50.840 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.169:6789,172.21.15.170:6789,172.21.15.170:6790:/ 2023-10-26T01:33:50.840 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=30203b28-ed8a-44e8-a9a6-a9bf25f980f1" will pass to kernel 2023-10-26T01:33:50.841 INFO:teuthology.orchestra.run.smithi170.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2023-10-26T01:33:50.841 INFO:teuthology.orchestra.run.smithi170.stdout: You just mounted a file system that supports labels which does not 2023-10-26T01:33:50.842 INFO:teuthology.orchestra.run.smithi170.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-10-26T01:33:50.842 INFO:teuthology.orchestra.run.smithi170.stdout: applications will generate AVC messages and not be allowed access to 2023-10-26T01:33:50.842 INFO:teuthology.orchestra.run.smithi170.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-10-26T01:33:50.846 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-26T01:33:50.847 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:33:50.847 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-26T01:33:50.926 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:33:50.927 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-26T01:33:50.980 INFO:teuthology.orchestra.run.smithi170.stdout:client5245 2023-10-26T01:33:50.980 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-26T01:33:50.981 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:33:51.029 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T01:33:51.259 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:51.256+0000 7f983c7d7640 1 -- 172.21.15.169:0/631741314 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98340ac930 msgr2=0x7f98340cd6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:51.259 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:51.256+0000 7f983c7d7640 1 --2- 172.21.15.169:0/631741314 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98340ac930 0x7f98340cd6f0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f98340ab9b0 tx=0x7f982402bd10 comp rx=0 tx=0).stop 2023-10-26T01:33:51.260 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:51.256+0000 7f983c7d7640 1 -- 172.21.15.169:0/631741314 shutdown_connections 2023-10-26T01:33:51.260 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:51.256+0000 7f983c7d7640 1 --2- 172.21.15.169:0/631741314 >> v2:172.21.15.170:3300/0 conn(0x7f98340cdc30 0x7f983406ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:51.260 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:51.256+0000 7f983c7d7640 1 --2- 172.21.15.169:0/631741314 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98340ac930 0x7f98340cd6f0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:51.261 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:51.256+0000 7f983c7d7640 1 --2- 172.21.15.169:0/631741314 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f98341592d0 0x7f98340aa3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:51.261 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:51.256+0000 7f983c7d7640 1 -- 172.21.15.169:0/631741314 >> 172.21.15.169:0/631741314 conn(0x7f9834069930 msgr2=0x7f9834069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:51.261 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:51.256+0000 7f983c7d7640 1 -- 172.21.15.169:0/631741314 shutdown_connections 2023-10-26T01:33:51.262 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:51.256+0000 7f983c7d7640 1 -- 172.21.15.169:0/631741314 wait complete. 2023-10-26T01:33:51.267 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:51.258+0000 7f983c7d7640 1 Processor -- start 2023-10-26T01:33:51.267 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:51.258+0000 7f983c7d7640 1 -- start start 2023-10-26T01:33:51.268 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:51.259+0000 7f983c7d7640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f98340ac930 0x7f983414c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:51.268 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:51.259+0000 7f983c7d7640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98340cdc30 0x7f98340b1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:51.268 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:51.259+0000 7f983c7d7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f98341592d0 0x7f9834070830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:51.269 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:51.259+0000 7f983c7d7640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f983415aa30 con 0x7f98340cdc30 2023-10-26T01:33:51.269 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:51.259+0000 7f983c7d7640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f983415aec0 con 0x7f98341592d0 2023-10-26T01:33:51.270 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:51.259+0000 7f983c7d7640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9834159e20 con 0x7f98340ac930 2023-10-26T01:33:51.270 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:51.259+0000 7f9839d72640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98340cdc30 0x7f98340b1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:51.270 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:51.260+0000 7f9839d72640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98340cdc30 0x7f98340b1bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:59546/0 (socket says 172.21.15.169:59546) 2023-10-26T01:33:51.271 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:51.260+0000 7f9839d72640 1 -- 172.21.15.169:0/3430976671 learned_addr learned my addr 172.21.15.169:0/3430976671 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:33:51.271 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:51.260+0000 7f983a573640 1 --2- 172.21.15.169:0/3430976671 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f98340ac930 0x7f983414c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:51.271 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:51.260+0000 7f9839d72640 1 -- 172.21.15.169:0/3430976671 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f98340ac930 msgr2=0x7f983414c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:51.272 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:51.260+0000 7f9839d72640 1 --2- 172.21.15.169:0/3430976671 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f98340ac930 0x7f983414c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:51.272 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:51.260+0000 7f9839d72640 1 -- 172.21.15.169:0/3430976671 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f98341592d0 msgr2=0x7f9834070830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:51.272 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:51.260+0000 7f983ad74640 1 --2- 172.21.15.169:0/3430976671 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f98341592d0 0x7f9834070830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:51.273 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:51.260+0000 7f9839d72640 1 --2- 172.21.15.169:0/3430976671 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f98341592d0 0x7f9834070830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:51.273 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:51.260+0000 7f9839d72640 1 -- 172.21.15.169:0/3430976671 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9824009560 con 0x7f98340cdc30 2023-10-26T01:33:51.273 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:51.260+0000 7f983ad74640 1 --2- 172.21.15.169:0/3430976671 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f98341592d0 0x7f9834070830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:33:51.273 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:51.260+0000 7f9839d72640 1 --2- 172.21.15.169:0/3430976671 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98340cdc30 0x7f98340b1bd0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f9824009850 tx=0x7f982402dce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:51.274 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:51.261+0000 7f982affd640 1 -- 172.21.15.169:0/3430976671 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9824005650 con 0x7f98340cdc30 2023-10-26T01:33:51.274 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:51.261+0000 7f982affd640 1 -- 172.21.15.169:0/3430976671 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f98240038f0 con 0x7f98340cdc30 2023-10-26T01:33:51.274 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:51.261+0000 7f982affd640 1 -- 172.21.15.169:0/3430976671 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9824003ce0 con 0x7f98340cdc30 2023-10-26T01:33:51.275 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:51.261+0000 7f983c7d7640 1 -- 172.21.15.169:0/3430976671 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f983415a100 con 0x7f98340cdc30 2023-10-26T01:33:51.275 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:51.261+0000 7f983c7d7640 1 -- 172.21.15.169:0/3430976671 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f983415a6c0 con 0x7f98340cdc30 2023-10-26T01:33:51.275 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:51.262+0000 7f983c7d7640 1 -- 172.21.15.169:0/3430976671 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f983404db10 con 0x7f98340cdc30 2023-10-26T01:33:51.276 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:51.263+0000 7f982affd640 1 -- 172.21.15.169:0/3430976671 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f982403e450 con 0x7f98340cdc30 2023-10-26T01:33:51.276 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:51.264+0000 7f982affd640 1 --2- 172.21.15.169:0/3430976671 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f981806ae40 0x7f981806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:33:51.276 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:51.264+0000 7f982affd640 1 -- 172.21.15.169:0/3430976671 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f98240ac060 con 0x7f98340cdc30 2023-10-26T01:33:51.276 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:51.264+0000 7f983a573640 1 --2- 172.21.15.169:0/3430976671 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f981806ae40 0x7f981806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:33:51.276 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:51.265+0000 7f983a573640 1 --2- 172.21.15.169:0/3430976671 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f981806ae40 0x7f981806d300 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f9830007da0 tx=0x7f98300072d0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:33:51.277 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:51.268+0000 7f982affd640 1 -- 172.21.15.169:0/3430976671 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f98240deca0 con 0x7f98340cdc30 2023-10-26T01:33:51.496 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:51.494+0000 7f983c7d7640 1 -- 172.21.15.169:0/3430976671 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9834158a30 con 0x7f98340cdc30 2023-10-26T01:33:51.496 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:51.495+0000 7f982affd640 1 -- 172.21.15.169:0/3430976671 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4922 (secure 0 0 0) 0x7f9824077070 con 0x7f98340cdc30 2023-10-26T01:33:51.497 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:33:51.497 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4976,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/2977083370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2977083370},{"type":"v1","addr":"172.21.15.170:6837","nonce":2977083370}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":5176,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/1598214042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1598214042},{"type":"v1","addr":"172.21.15.170:6835","nonce":1598214042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":6087,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/406582181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":406582181},{"type":"v1","addr":"172.21.15.169:6833","nonce":406582181}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T01:33:40.717881+0000","modified":"2023-10-26T01:33:43.796080+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6093},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6093":{"gid":6093,"name":"a","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6835/2338833513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":2338833513},{"type":"v1","addr":"172.21.15.169:6835","nonce":2338833513}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-10-26T01:33:51.498 INFO:teuthology.orchestra.run.smithi169.stderr:dumped fsmap epoch 32 2023-10-26T01:33:51.499 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:51.497+0000 7f983c7d7640 1 -- 172.21.15.169:0/3430976671 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f981806ae40 msgr2=0x7f981806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:51.500 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:51.497+0000 7f983c7d7640 1 --2- 172.21.15.169:0/3430976671 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f981806ae40 0x7f981806d300 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f9830007da0 tx=0x7f98300072d0 comp rx=0 tx=0).stop 2023-10-26T01:33:51.500 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:51.497+0000 7f983c7d7640 1 -- 172.21.15.169:0/3430976671 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98340cdc30 msgr2=0x7f98340b1bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:33:51.500 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:51.497+0000 7f983c7d7640 1 --2- 172.21.15.169:0/3430976671 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98340cdc30 0x7f98340b1bd0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f9824009850 tx=0x7f982402dce0 comp rx=0 tx=0).stop 2023-10-26T01:33:51.500 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:51.498+0000 7f983c7d7640 1 -- 172.21.15.169:0/3430976671 shutdown_connections 2023-10-26T01:33:51.500 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:51.498+0000 7f983c7d7640 1 --2- 172.21.15.169:0/3430976671 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f981806ae40 0x7f981806d300 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:51.501 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:51.498+0000 7f983c7d7640 1 --2- 172.21.15.169:0/3430976671 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98340cdc30 0x7f98340b1bd0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:51.501 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:51.498+0000 7f983c7d7640 1 --2- 172.21.15.169:0/3430976671 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f98341592d0 0x7f9834070830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:51.501 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:51.498+0000 7f983c7d7640 1 --2- 172.21.15.169:0/3430976671 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f98340ac930 0x7f983414c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:33:51.501 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:51.498+0000 7f983c7d7640 1 -- 172.21.15.169:0/3430976671 >> 172.21.15.169:0/3430976671 conn(0x7f9834069930 msgr2=0x7f98340cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:33:51.501 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:51.498+0000 7f983c7d7640 1 -- 172.21.15.169:0/3430976671 shutdown_connections 2023-10-26T01:33:51.502 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:33:51.498+0000 7f983c7d7640 1 -- 172.21.15.169:0/3430976671 wait complete. 2023-10-26T01:33:51.519 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2023-10-26T01:33:51.733 INFO:teuthology.orchestra.run.smithi169.stdout:{"cluster_fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","whoami":0,"id":6093,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":10.416725413,"mdsmap_epoch":32,"osdmap_epoch":65,"osdmap_epoch_barrier":65,"uptime":16.253279643999999} 2023-10-26T01:33:51.734 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "30203b28-ed8a-44e8-a9a6-a9bf25f980f1", "fs_name": "cephfs", "id": 6093, "mdsmap_epoch": 32, "osdmap_epoch": 65, "osdmap_epoch_barrier": 65, "rank_uptime": 10.416725413, "state": "up:active", "uptime": 16.253279644, "want_state": "up:active", "whoami": 0 } 2023-10-26T01:33:51.735 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2023-10-26T01:33:51.967 INFO:teuthology.orchestra.run.smithi169.stdout:{"osd_mon_report_interval":"5"} 2023-10-26T01:33:51.968 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2023-10-26T01:33:51.969 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mon_tick_interval 2023-10-26T01:33:52.188 INFO:teuthology.orchestra.run.smithi169.stdout:{"mon_tick_interval":"5"} 2023-10-26T01:33:52.188 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mon_tick_interval": "5" } 2023-10-26T01:33:52.189 INFO:tasks.cephfs.test_full:pool capacity 557012544, 531MB should be enough to fill it 2023-10-26T01:33:52.189 DEBUG:teuthology.orchestra.run.smithi169:> stdin-killer -- python3 -c ' 2023-10-26T01:33:52.190 DEBUG:teuthology.orchestra.run.smithi169:> import time 2023-10-26T01:33:52.190 DEBUG:teuthology.orchestra.run.smithi169:> import datetime 2023-10-26T01:33:52.190 DEBUG:teuthology.orchestra.run.smithi169:> import subprocess 2023-10-26T01:33:52.191 DEBUG:teuthology.orchestra.run.smithi169:> import os 2023-10-26T01:33:52.191 DEBUG:teuthology.orchestra.run.smithi169:> 2023-10-26T01:33:52.192 DEBUG:teuthology.orchestra.run.smithi169:> # Write some buffered data through before going full, all should be well 2023-10-26T01:33:52.192 DEBUG:teuthology.orchestra.run.smithi169:> print("writing some data through which we expect to succeed") 2023-10-26T01:33:52.192 DEBUG:teuthology.orchestra.run.smithi169:> bytes = 0 2023-10-26T01:33:52.193 DEBUG:teuthology.orchestra.run.smithi169:> f = os.open("/home/ubuntu/cephtest/mnt.0/full_test_file", os.O_WRONLY | os.O_CREAT) 2023-10-26T01:33:52.193 DEBUG:teuthology.orchestra.run.smithi169:> os.setxattr("/home/ubuntu/cephtest/mnt.0/full_test_file", '"'"'ceph.file.layout'"'"', b'"'"'stripe_unit=1048576 stripe_count=1 object_size=1048576'"'"') 2023-10-26T01:33:52.194 DEBUG:teuthology.orchestra.run.smithi169:> bytes += os.write(f, b'"'"'a'"'"' * 512 * 1024) 2023-10-26T01:33:52.194 DEBUG:teuthology.orchestra.run.smithi169:> os.fsync(f) 2023-10-26T01:33:52.195 DEBUG:teuthology.orchestra.run.smithi169:> print("fsync'"'"'ed data successfully, will now attempt to fill fs") 2023-10-26T01:33:52.195 DEBUG:teuthology.orchestra.run.smithi169:> 2023-10-26T01:33:52.195 DEBUG:teuthology.orchestra.run.smithi169:> # Okay, now we'"'"'re going to fill up the filesystem, and then keep 2023-10-26T01:33:52.196 DEBUG:teuthology.orchestra.run.smithi169:> # writing until we see an error from fsync. As long as we'"'"'re doing 2023-10-26T01:33:52.196 DEBUG:teuthology.orchestra.run.smithi169:> # buffered IO, the error should always only appear from fsync and not 2023-10-26T01:33:52.196 DEBUG:teuthology.orchestra.run.smithi169:> # from write 2023-10-26T01:33:52.197 DEBUG:teuthology.orchestra.run.smithi169:> full = False 2023-10-26T01:33:52.197 DEBUG:teuthology.orchestra.run.smithi169:> 2023-10-26T01:33:52.197 DEBUG:teuthology.orchestra.run.smithi169:> for n in range(0, int(531 * 0.9)): 2023-10-26T01:33:52.198 DEBUG:teuthology.orchestra.run.smithi169:> bytes += os.write(f, b'"'"'x'"'"' * 1024 * 1024) 2023-10-26T01:33:52.198 DEBUG:teuthology.orchestra.run.smithi169:> print("wrote {0} bytes via buffered write, may repeat".format(bytes)) 2023-10-26T01:33:52.198 DEBUG:teuthology.orchestra.run.smithi169:> print("done writing {0} bytes".format(bytes)) 2023-10-26T01:33:52.199 DEBUG:teuthology.orchestra.run.smithi169:> 2023-10-26T01:33:52.199 DEBUG:teuthology.orchestra.run.smithi169:> # OK, now we should sneak in under the full condition 2023-10-26T01:33:52.199 DEBUG:teuthology.orchestra.run.smithi169:> # due to the time it takes the OSDs to report to the 2023-10-26T01:33:52.200 DEBUG:teuthology.orchestra.run.smithi169:> # mons, and get a successful fsync on our full-making data 2023-10-26T01:33:52.200 DEBUG:teuthology.orchestra.run.smithi169:> os.fsync(f) 2023-10-26T01:33:52.200 DEBUG:teuthology.orchestra.run.smithi169:> print("successfully fsync'"'"'ed prior to getting full state reported") 2023-10-26T01:33:52.201 DEBUG:teuthology.orchestra.run.smithi169:> 2023-10-26T01:33:52.201 DEBUG:teuthology.orchestra.run.smithi169:> # buffered write, add more dirty data to the buffer 2023-10-26T01:33:52.202 DEBUG:teuthology.orchestra.run.smithi169:> print("starting buffered write") 2023-10-26T01:33:52.202 DEBUG:teuthology.orchestra.run.smithi169:> try: 2023-10-26T01:33:52.202 DEBUG:teuthology.orchestra.run.smithi169:> for n in range(0, int(531 * 0.2)): 2023-10-26T01:33:52.203 DEBUG:teuthology.orchestra.run.smithi169:> bytes += os.write(f, b'"'"'x'"'"' * 1024 * 1024) 2023-10-26T01:33:52.203 DEBUG:teuthology.orchestra.run.smithi169:> print("sleeping a bit as we'"'"'ve exceeded 90% of our expected full ratio") 2023-10-26T01:33:52.203 DEBUG:teuthology.orchestra.run.smithi169:> time.sleep(15.0) 2023-10-26T01:33:52.204 DEBUG:teuthology.orchestra.run.smithi169:> except OSError: 2023-10-26T01:33:52.204 DEBUG:teuthology.orchestra.run.smithi169:> pass; 2023-10-26T01:33:52.204 DEBUG:teuthology.orchestra.run.smithi169:> 2023-10-26T01:33:52.205 DEBUG:teuthology.orchestra.run.smithi169:> print("wrote, now waiting 30s and then doing a close we expect to fail") 2023-10-26T01:33:52.205 DEBUG:teuthology.orchestra.run.smithi169:> 2023-10-26T01:33:52.206 DEBUG:teuthology.orchestra.run.smithi169:> # Wait long enough for a background flush that should fail 2023-10-26T01:33:52.206 DEBUG:teuthology.orchestra.run.smithi169:> time.sleep(30) 2023-10-26T01:33:52.206 DEBUG:teuthology.orchestra.run.smithi169:> 2023-10-26T01:33:52.207 DEBUG:teuthology.orchestra.run.smithi169:> if False: 2023-10-26T01:33:52.207 DEBUG:teuthology.orchestra.run.smithi169:> # ...and check that the failed background flush is reflected in fclose 2023-10-26T01:33:52.207 DEBUG:teuthology.orchestra.run.smithi169:> try: 2023-10-26T01:33:52.208 DEBUG:teuthology.orchestra.run.smithi169:> os.close(f) 2023-10-26T01:33:52.208 DEBUG:teuthology.orchestra.run.smithi169:> except OSError: 2023-10-26T01:33:52.208 DEBUG:teuthology.orchestra.run.smithi169:> print("close() returned an error as expected") 2023-10-26T01:33:52.209 DEBUG:teuthology.orchestra.run.smithi169:> else: 2023-10-26T01:33:52.209 DEBUG:teuthology.orchestra.run.smithi169:> raise RuntimeError("close() failed to raise error") 2023-10-26T01:33:52.210 DEBUG:teuthology.orchestra.run.smithi169:> else: 2023-10-26T01:33:52.210 DEBUG:teuthology.orchestra.run.smithi169:> # The kernel cephfs client does not raise errors on fclose 2023-10-26T01:33:52.210 DEBUG:teuthology.orchestra.run.smithi169:> os.close(f) 2023-10-26T01:33:52.211 DEBUG:teuthology.orchestra.run.smithi169:> 2023-10-26T01:33:52.211 DEBUG:teuthology.orchestra.run.smithi169:> os.unlink("/home/ubuntu/cephtest/mnt.0/full_test_file") 2023-10-26T01:33:52.211 DEBUG:teuthology.orchestra.run.smithi169:> ' 2023-10-26T01:33:55.692 INFO:teuthology.orchestra.run.smithi169.stdout:writing some data through which we expect to succeed 2023-10-26T01:33:55.693 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed data successfully, will now attempt to fill fs 2023-10-26T01:33:55.693 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 1572864 bytes via buffered write, may repeat 2023-10-26T01:33:55.693 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 2621440 bytes via buffered write, may repeat 2023-10-26T01:33:55.693 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 3670016 bytes via buffered write, may repeat 2023-10-26T01:33:55.693 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 4718592 bytes via buffered write, may repeat 2023-10-26T01:33:55.694 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 5767168 bytes via buffered write, may repeat 2023-10-26T01:33:55.694 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 6815744 bytes via buffered write, may repeat 2023-10-26T01:33:55.694 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 7864320 bytes via buffered write, may repeat 2023-10-26T01:33:55.694 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 8912896 bytes via buffered write, may repeat 2023-10-26T01:33:55.695 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 9961472 bytes via buffered write, may repeat 2023-10-26T01:33:55.695 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 11010048 bytes via buffered write, may repeat 2023-10-26T01:33:55.695 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 12058624 bytes via buffered write, may repeat 2023-10-26T01:33:55.695 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 13107200 bytes via buffered write, may repeat 2023-10-26T01:33:55.695 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 14155776 bytes via buffered write, may repeat 2023-10-26T01:33:55.695 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 15204352 bytes via buffered write, may repeat 2023-10-26T01:33:55.696 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 16252928 bytes via buffered write, may repeat 2023-10-26T01:33:55.696 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 17301504 bytes via buffered write, may repeat 2023-10-26T01:33:55.696 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 18350080 bytes via buffered write, may repeat 2023-10-26T01:33:55.696 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 19398656 bytes via buffered write, may repeat 2023-10-26T01:33:55.696 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 20447232 bytes via buffered write, may repeat 2023-10-26T01:33:55.697 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 21495808 bytes via buffered write, may repeat 2023-10-26T01:33:55.697 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 22544384 bytes via buffered write, may repeat 2023-10-26T01:33:55.697 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 23592960 bytes via buffered write, may repeat 2023-10-26T01:33:55.697 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 24641536 bytes via buffered write, may repeat 2023-10-26T01:33:55.697 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 25690112 bytes via buffered write, may repeat 2023-10-26T01:33:55.697 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 26738688 bytes via buffered write, may repeat 2023-10-26T01:33:55.698 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 27787264 bytes via buffered write, may repeat 2023-10-26T01:33:55.698 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 28835840 bytes via buffered write, may repeat 2023-10-26T01:33:55.698 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 29884416 bytes via buffered write, may repeat 2023-10-26T01:33:55.698 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 30932992 bytes via buffered write, may repeat 2023-10-26T01:33:55.698 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 31981568 bytes via buffered write, may repeat 2023-10-26T01:33:55.699 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 33030144 bytes via buffered write, may repeat 2023-10-26T01:33:55.699 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 34078720 bytes via buffered write, may repeat 2023-10-26T01:33:55.699 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 35127296 bytes via buffered write, may repeat 2023-10-26T01:33:55.699 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 36175872 bytes via buffered write, may repeat 2023-10-26T01:33:55.699 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 37224448 bytes via buffered write, may repeat 2023-10-26T01:33:55.700 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 38273024 bytes via buffered write, may repeat 2023-10-26T01:33:55.700 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 39321600 bytes via buffered write, may repeat 2023-10-26T01:33:55.700 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 40370176 bytes via buffered write, may repeat 2023-10-26T01:33:55.700 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 41418752 bytes via buffered write, may repeat 2023-10-26T01:33:55.700 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 42467328 bytes via buffered write, may repeat 2023-10-26T01:33:55.701 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 43515904 bytes via buffered write, may repeat 2023-10-26T01:33:55.701 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 44564480 bytes via buffered write, may repeat 2023-10-26T01:33:55.701 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 45613056 bytes via buffered write, may repeat 2023-10-26T01:33:55.701 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 46661632 bytes via buffered write, may repeat 2023-10-26T01:33:55.701 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 47710208 bytes via buffered write, may repeat 2023-10-26T01:33:55.701 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 48758784 bytes via buffered write, may repeat 2023-10-26T01:33:55.702 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 49807360 bytes via buffered write, may repeat 2023-10-26T01:33:55.702 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 50855936 bytes via buffered write, may repeat 2023-10-26T01:33:55.702 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 51904512 bytes via buffered write, may repeat 2023-10-26T01:33:55.702 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 52953088 bytes via buffered write, may repeat 2023-10-26T01:33:55.702 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 54001664 bytes via buffered write, may repeat 2023-10-26T01:33:55.703 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 55050240 bytes via buffered write, may repeat 2023-10-26T01:33:55.703 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 56098816 bytes via buffered write, may repeat 2023-10-26T01:33:55.703 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 57147392 bytes via buffered write, may repeat 2023-10-26T01:33:55.703 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 58195968 bytes via buffered write, may repeat 2023-10-26T01:33:55.703 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 59244544 bytes via buffered write, may repeat 2023-10-26T01:33:55.703 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 60293120 bytes via buffered write, may repeat 2023-10-26T01:33:55.704 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 61341696 bytes via buffered write, may repeat 2023-10-26T01:33:55.704 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 62390272 bytes via buffered write, may repeat 2023-10-26T01:33:55.704 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 63438848 bytes via buffered write, may repeat 2023-10-26T01:33:55.704 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 64487424 bytes via buffered write, may repeat 2023-10-26T01:33:55.704 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 65536000 bytes via buffered write, may repeat 2023-10-26T01:33:55.705 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 66584576 bytes via buffered write, may repeat 2023-10-26T01:33:55.705 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 67633152 bytes via buffered write, may repeat 2023-10-26T01:33:55.705 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 68681728 bytes via buffered write, may repeat 2023-10-26T01:33:55.705 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 69730304 bytes via buffered write, may repeat 2023-10-26T01:33:55.705 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 70778880 bytes via buffered write, may repeat 2023-10-26T01:33:55.706 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 71827456 bytes via buffered write, may repeat 2023-10-26T01:33:55.706 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 72876032 bytes via buffered write, may repeat 2023-10-26T01:33:55.706 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 73924608 bytes via buffered write, may repeat 2023-10-26T01:33:55.706 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 74973184 bytes via buffered write, may repeat 2023-10-26T01:33:55.706 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 76021760 bytes via buffered write, may repeat 2023-10-26T01:33:55.707 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 77070336 bytes via buffered write, may repeat 2023-10-26T01:33:55.707 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 78118912 bytes via buffered write, may repeat 2023-10-26T01:33:55.707 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 79167488 bytes via buffered write, may repeat 2023-10-26T01:33:55.707 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 80216064 bytes via buffered write, may repeat 2023-10-26T01:33:55.707 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 81264640 bytes via buffered write, may repeat 2023-10-26T01:33:55.707 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 82313216 bytes via buffered write, may repeat 2023-10-26T01:33:55.708 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 83361792 bytes via buffered write, may repeat 2023-10-26T01:33:55.708 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 84410368 bytes via buffered write, may repeat 2023-10-26T01:33:55.708 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 85458944 bytes via buffered write, may repeat 2023-10-26T01:33:55.708 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 86507520 bytes via buffered write, may repeat 2023-10-26T01:33:55.708 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 87556096 bytes via buffered write, may repeat 2023-10-26T01:33:55.709 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 88604672 bytes via buffered write, may repeat 2023-10-26T01:33:55.709 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 89653248 bytes via buffered write, may repeat 2023-10-26T01:33:55.709 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 90701824 bytes via buffered write, may repeat 2023-10-26T01:33:55.709 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 91750400 bytes via buffered write, may repeat 2023-10-26T01:33:55.709 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 92798976 bytes via buffered write, may repeat 2023-10-26T01:33:55.709 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 93847552 bytes via buffered write, may repeat 2023-10-26T01:33:55.710 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 94896128 bytes via buffered write, may repeat 2023-10-26T01:33:55.710 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 95944704 bytes via buffered write, may repeat 2023-10-26T01:33:55.710 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 96993280 bytes via buffered write, may repeat 2023-10-26T01:33:55.710 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 98041856 bytes via buffered write, may repeat 2023-10-26T01:33:55.710 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 99090432 bytes via buffered write, may repeat 2023-10-26T01:33:55.711 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 100139008 bytes via buffered write, may repeat 2023-10-26T01:33:55.711 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 101187584 bytes via buffered write, may repeat 2023-10-26T01:33:55.711 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 102236160 bytes via buffered write, may repeat 2023-10-26T01:33:55.711 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 103284736 bytes via buffered write, may repeat 2023-10-26T01:33:55.711 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 104333312 bytes via buffered write, may repeat 2023-10-26T01:33:55.712 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 105381888 bytes via buffered write, may repeat 2023-10-26T01:33:55.712 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 106430464 bytes via buffered write, may repeat 2023-10-26T01:33:55.712 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 107479040 bytes via buffered write, may repeat 2023-10-26T01:33:55.712 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 108527616 bytes via buffered write, may repeat 2023-10-26T01:33:55.712 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 109576192 bytes via buffered write, may repeat 2023-10-26T01:33:55.712 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 110624768 bytes via buffered write, may repeat 2023-10-26T01:33:55.713 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 111673344 bytes via buffered write, may repeat 2023-10-26T01:33:55.713 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 112721920 bytes via buffered write, may repeat 2023-10-26T01:33:55.713 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 113770496 bytes via buffered write, may repeat 2023-10-26T01:33:55.713 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 114819072 bytes via buffered write, may repeat 2023-10-26T01:33:55.713 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 115867648 bytes via buffered write, may repeat 2023-10-26T01:33:55.714 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 116916224 bytes via buffered write, may repeat 2023-10-26T01:33:55.714 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 117964800 bytes via buffered write, may repeat 2023-10-26T01:33:55.714 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 119013376 bytes via buffered write, may repeat 2023-10-26T01:33:55.714 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 120061952 bytes via buffered write, may repeat 2023-10-26T01:33:55.714 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 121110528 bytes via buffered write, may repeat 2023-10-26T01:33:55.715 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 122159104 bytes via buffered write, may repeat 2023-10-26T01:33:55.715 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 123207680 bytes via buffered write, may repeat 2023-10-26T01:33:55.715 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 124256256 bytes via buffered write, may repeat 2023-10-26T01:33:55.715 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 125304832 bytes via buffered write, may repeat 2023-10-26T01:33:55.715 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 126353408 bytes via buffered write, may repeat 2023-10-26T01:33:55.715 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 127401984 bytes via buffered write, may repeat 2023-10-26T01:33:55.716 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 128450560 bytes via buffered write, may repeat 2023-10-26T01:33:55.716 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 129499136 bytes via buffered write, may repeat 2023-10-26T01:33:55.716 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 130547712 bytes via buffered write, may repeat 2023-10-26T01:33:55.716 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 131596288 bytes via buffered write, may repeat 2023-10-26T01:33:55.716 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 132644864 bytes via buffered write, may repeat 2023-10-26T01:33:55.717 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 133693440 bytes via buffered write, may repeat 2023-10-26T01:33:55.717 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 134742016 bytes via buffered write, may repeat 2023-10-26T01:33:55.717 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 135790592 bytes via buffered write, may repeat 2023-10-26T01:33:55.717 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 136839168 bytes via buffered write, may repeat 2023-10-26T01:33:55.717 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 137887744 bytes via buffered write, may repeat 2023-10-26T01:33:55.717 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 138936320 bytes via buffered write, may repeat 2023-10-26T01:33:55.718 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 139984896 bytes via buffered write, may repeat 2023-10-26T01:33:55.718 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 141033472 bytes via buffered write, may repeat 2023-10-26T01:33:55.718 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 142082048 bytes via buffered write, may repeat 2023-10-26T01:33:55.718 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 143130624 bytes via buffered write, may repeat 2023-10-26T01:33:55.718 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 144179200 bytes via buffered write, may repeat 2023-10-26T01:33:55.719 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 145227776 bytes via buffered write, may repeat 2023-10-26T01:33:55.719 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 146276352 bytes via buffered write, may repeat 2023-10-26T01:33:55.719 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 147324928 bytes via buffered write, may repeat 2023-10-26T01:33:55.719 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 148373504 bytes via buffered write, may repeat 2023-10-26T01:33:55.719 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 149422080 bytes via buffered write, may repeat 2023-10-26T01:33:55.719 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 150470656 bytes via buffered write, may repeat 2023-10-26T01:33:55.720 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 151519232 bytes via buffered write, may repeat 2023-10-26T01:33:55.720 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 152567808 bytes via buffered write, may repeat 2023-10-26T01:33:55.720 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 153616384 bytes via buffered write, may repeat 2023-10-26T01:33:55.720 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 154664960 bytes via buffered write, may repeat 2023-10-26T01:33:55.720 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 155713536 bytes via buffered write, may repeat 2023-10-26T01:33:55.721 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 156762112 bytes via buffered write, may repeat 2023-10-26T01:33:55.721 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 157810688 bytes via buffered write, may repeat 2023-10-26T01:33:55.721 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 158859264 bytes via buffered write, may repeat 2023-10-26T01:33:55.721 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 159907840 bytes via buffered write, may repeat 2023-10-26T01:33:55.721 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 160956416 bytes via buffered write, may repeat 2023-10-26T01:33:55.721 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 162004992 bytes via buffered write, may repeat 2023-10-26T01:33:55.871 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 163053568 bytes via buffered write, may repeat 2023-10-26T01:33:55.872 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 164102144 bytes via buffered write, may repeat 2023-10-26T01:33:55.872 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 165150720 bytes via buffered write, may repeat 2023-10-26T01:33:55.872 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 166199296 bytes via buffered write, may repeat 2023-10-26T01:33:55.872 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 167247872 bytes via buffered write, may repeat 2023-10-26T01:33:55.873 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 168296448 bytes via buffered write, may repeat 2023-10-26T01:33:55.873 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 169345024 bytes via buffered write, may repeat 2023-10-26T01:33:55.873 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 170393600 bytes via buffered write, may repeat 2023-10-26T01:33:55.873 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 171442176 bytes via buffered write, may repeat 2023-10-26T01:33:55.873 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 172490752 bytes via buffered write, may repeat 2023-10-26T01:33:55.874 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 173539328 bytes via buffered write, may repeat 2023-10-26T01:33:55.874 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 174587904 bytes via buffered write, may repeat 2023-10-26T01:33:55.874 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 175636480 bytes via buffered write, may repeat 2023-10-26T01:33:55.874 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 176685056 bytes via buffered write, may repeat 2023-10-26T01:33:55.874 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 177733632 bytes via buffered write, may repeat 2023-10-26T01:33:55.875 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 178782208 bytes via buffered write, may repeat 2023-10-26T01:33:55.875 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 179830784 bytes via buffered write, may repeat 2023-10-26T01:33:55.875 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 180879360 bytes via buffered write, may repeat 2023-10-26T01:33:55.875 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 181927936 bytes via buffered write, may repeat 2023-10-26T01:33:55.875 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 182976512 bytes via buffered write, may repeat 2023-10-26T01:33:55.876 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 184025088 bytes via buffered write, may repeat 2023-10-26T01:33:55.876 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 185073664 bytes via buffered write, may repeat 2023-10-26T01:33:55.876 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 186122240 bytes via buffered write, may repeat 2023-10-26T01:33:55.876 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 187170816 bytes via buffered write, may repeat 2023-10-26T01:33:55.877 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 188219392 bytes via buffered write, may repeat 2023-10-26T01:33:55.877 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 189267968 bytes via buffered write, may repeat 2023-10-26T01:33:55.877 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 190316544 bytes via buffered write, may repeat 2023-10-26T01:33:55.877 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 191365120 bytes via buffered write, may repeat 2023-10-26T01:33:55.877 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 192413696 bytes via buffered write, may repeat 2023-10-26T01:33:55.878 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 193462272 bytes via buffered write, may repeat 2023-10-26T01:33:55.878 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 194510848 bytes via buffered write, may repeat 2023-10-26T01:33:55.878 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 195559424 bytes via buffered write, may repeat 2023-10-26T01:33:55.878 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 196608000 bytes via buffered write, may repeat 2023-10-26T01:33:55.878 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 197656576 bytes via buffered write, may repeat 2023-10-26T01:33:55.879 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 198705152 bytes via buffered write, may repeat 2023-10-26T01:33:55.879 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 199753728 bytes via buffered write, may repeat 2023-10-26T01:33:55.879 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 200802304 bytes via buffered write, may repeat 2023-10-26T01:33:55.879 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 201850880 bytes via buffered write, may repeat 2023-10-26T01:33:55.879 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 202899456 bytes via buffered write, may repeat 2023-10-26T01:33:55.880 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 203948032 bytes via buffered write, may repeat 2023-10-26T01:33:55.880 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 204996608 bytes via buffered write, may repeat 2023-10-26T01:33:55.880 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 206045184 bytes via buffered write, may repeat 2023-10-26T01:33:55.880 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 207093760 bytes via buffered write, may repeat 2023-10-26T01:33:55.880 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 208142336 bytes via buffered write, may repeat 2023-10-26T01:33:55.881 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 209190912 bytes via buffered write, may repeat 2023-10-26T01:33:55.881 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 210239488 bytes via buffered write, may repeat 2023-10-26T01:33:55.881 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 211288064 bytes via buffered write, may repeat 2023-10-26T01:33:55.881 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 212336640 bytes via buffered write, may repeat 2023-10-26T01:33:55.882 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 213385216 bytes via buffered write, may repeat 2023-10-26T01:33:55.882 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 214433792 bytes via buffered write, may repeat 2023-10-26T01:33:55.882 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 215482368 bytes via buffered write, may repeat 2023-10-26T01:33:55.882 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 216530944 bytes via buffered write, may repeat 2023-10-26T01:33:55.882 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 217579520 bytes via buffered write, may repeat 2023-10-26T01:33:55.883 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 218628096 bytes via buffered write, may repeat 2023-10-26T01:33:55.883 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 219676672 bytes via buffered write, may repeat 2023-10-26T01:33:55.883 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 220725248 bytes via buffered write, may repeat 2023-10-26T01:33:55.883 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 221773824 bytes via buffered write, may repeat 2023-10-26T01:33:55.883 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 222822400 bytes via buffered write, may repeat 2023-10-26T01:33:55.884 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 223870976 bytes via buffered write, may repeat 2023-10-26T01:33:55.884 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 224919552 bytes via buffered write, may repeat 2023-10-26T01:33:55.884 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 225968128 bytes via buffered write, may repeat 2023-10-26T01:33:55.884 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 227016704 bytes via buffered write, may repeat 2023-10-26T01:33:55.885 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 228065280 bytes via buffered write, may repeat 2023-10-26T01:33:55.885 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 229113856 bytes via buffered write, may repeat 2023-10-26T01:33:55.885 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 230162432 bytes via buffered write, may repeat 2023-10-26T01:33:55.885 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 231211008 bytes via buffered write, may repeat 2023-10-26T01:33:55.886 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 232259584 bytes via buffered write, may repeat 2023-10-26T01:33:55.886 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 233308160 bytes via buffered write, may repeat 2023-10-26T01:33:55.886 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 234356736 bytes via buffered write, may repeat 2023-10-26T01:33:55.886 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 235405312 bytes via buffered write, may repeat 2023-10-26T01:33:55.887 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 236453888 bytes via buffered write, may repeat 2023-10-26T01:33:55.887 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 237502464 bytes via buffered write, may repeat 2023-10-26T01:33:55.887 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 238551040 bytes via buffered write, may repeat 2023-10-26T01:33:55.887 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 239599616 bytes via buffered write, may repeat 2023-10-26T01:33:55.887 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 240648192 bytes via buffered write, may repeat 2023-10-26T01:33:55.888 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 241696768 bytes via buffered write, may repeat 2023-10-26T01:33:55.888 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 242745344 bytes via buffered write, may repeat 2023-10-26T01:33:55.888 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 243793920 bytes via buffered write, may repeat 2023-10-26T01:33:55.888 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 244842496 bytes via buffered write, may repeat 2023-10-26T01:33:55.888 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 245891072 bytes via buffered write, may repeat 2023-10-26T01:33:55.889 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 246939648 bytes via buffered write, may repeat 2023-10-26T01:33:55.889 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 247988224 bytes via buffered write, may repeat 2023-10-26T01:33:55.889 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 249036800 bytes via buffered write, may repeat 2023-10-26T01:33:55.889 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 250085376 bytes via buffered write, may repeat 2023-10-26T01:33:55.889 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 251133952 bytes via buffered write, may repeat 2023-10-26T01:33:55.889 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 252182528 bytes via buffered write, may repeat 2023-10-26T01:33:55.890 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 253231104 bytes via buffered write, may repeat 2023-10-26T01:33:55.890 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 254279680 bytes via buffered write, may repeat 2023-10-26T01:33:55.890 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 255328256 bytes via buffered write, may repeat 2023-10-26T01:33:55.890 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 256376832 bytes via buffered write, may repeat 2023-10-26T01:33:55.891 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 257425408 bytes via buffered write, may repeat 2023-10-26T01:33:55.891 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 258473984 bytes via buffered write, may repeat 2023-10-26T01:33:55.891 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 259522560 bytes via buffered write, may repeat 2023-10-26T01:33:55.891 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 260571136 bytes via buffered write, may repeat 2023-10-26T01:33:55.891 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 261619712 bytes via buffered write, may repeat 2023-10-26T01:33:55.892 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 262668288 bytes via buffered write, may repeat 2023-10-26T01:33:55.892 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 263716864 bytes via buffered write, may repeat 2023-10-26T01:33:55.892 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 264765440 bytes via buffered write, may repeat 2023-10-26T01:33:55.892 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 265814016 bytes via buffered write, may repeat 2023-10-26T01:33:55.893 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 266862592 bytes via buffered write, may repeat 2023-10-26T01:33:55.893 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 267911168 bytes via buffered write, may repeat 2023-10-26T01:33:55.893 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 268959744 bytes via buffered write, may repeat 2023-10-26T01:33:55.893 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 270008320 bytes via buffered write, may repeat 2023-10-26T01:33:55.894 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 271056896 bytes via buffered write, may repeat 2023-10-26T01:33:55.894 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 272105472 bytes via buffered write, may repeat 2023-10-26T01:33:55.894 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 273154048 bytes via buffered write, may repeat 2023-10-26T01:33:55.894 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 274202624 bytes via buffered write, may repeat 2023-10-26T01:33:55.895 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 275251200 bytes via buffered write, may repeat 2023-10-26T01:33:55.895 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 276299776 bytes via buffered write, may repeat 2023-10-26T01:33:55.895 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 277348352 bytes via buffered write, may repeat 2023-10-26T01:33:55.895 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 278396928 bytes via buffered write, may repeat 2023-10-26T01:33:55.896 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 279445504 bytes via buffered write, may repeat 2023-10-26T01:33:55.896 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 280494080 bytes via buffered write, may repeat 2023-10-26T01:33:55.896 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 281542656 bytes via buffered write, may repeat 2023-10-26T01:33:55.896 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 282591232 bytes via buffered write, may repeat 2023-10-26T01:33:55.896 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 283639808 bytes via buffered write, may repeat 2023-10-26T01:33:55.897 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 284688384 bytes via buffered write, may repeat 2023-10-26T01:33:55.897 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 285736960 bytes via buffered write, may repeat 2023-10-26T01:33:55.897 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 286785536 bytes via buffered write, may repeat 2023-10-26T01:33:55.897 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 287834112 bytes via buffered write, may repeat 2023-10-26T01:33:55.898 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 288882688 bytes via buffered write, may repeat 2023-10-26T01:33:55.898 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 289931264 bytes via buffered write, may repeat 2023-10-26T01:33:55.898 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 290979840 bytes via buffered write, may repeat 2023-10-26T01:33:55.898 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 292028416 bytes via buffered write, may repeat 2023-10-26T01:33:55.899 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 293076992 bytes via buffered write, may repeat 2023-10-26T01:33:55.899 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 294125568 bytes via buffered write, may repeat 2023-10-26T01:33:55.899 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 295174144 bytes via buffered write, may repeat 2023-10-26T01:33:55.899 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 296222720 bytes via buffered write, may repeat 2023-10-26T01:33:55.900 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 297271296 bytes via buffered write, may repeat 2023-10-26T01:33:55.900 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 298319872 bytes via buffered write, may repeat 2023-10-26T01:33:55.900 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 299368448 bytes via buffered write, may repeat 2023-10-26T01:33:55.900 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 300417024 bytes via buffered write, may repeat 2023-10-26T01:33:55.901 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 301465600 bytes via buffered write, may repeat 2023-10-26T01:33:55.901 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 302514176 bytes via buffered write, may repeat 2023-10-26T01:33:55.901 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 303562752 bytes via buffered write, may repeat 2023-10-26T01:33:55.901 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 304611328 bytes via buffered write, may repeat 2023-10-26T01:33:55.901 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 305659904 bytes via buffered write, may repeat 2023-10-26T01:33:55.902 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 306708480 bytes via buffered write, may repeat 2023-10-26T01:33:55.902 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 307757056 bytes via buffered write, may repeat 2023-10-26T01:33:55.902 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 308805632 bytes via buffered write, may repeat 2023-10-26T01:33:55.902 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 309854208 bytes via buffered write, may repeat 2023-10-26T01:33:55.903 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 310902784 bytes via buffered write, may repeat 2023-10-26T01:33:55.903 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 311951360 bytes via buffered write, may repeat 2023-10-26T01:33:55.903 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 312999936 bytes via buffered write, may repeat 2023-10-26T01:33:55.903 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 314048512 bytes via buffered write, may repeat 2023-10-26T01:33:55.904 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 315097088 bytes via buffered write, may repeat 2023-10-26T01:33:55.904 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 316145664 bytes via buffered write, may repeat 2023-10-26T01:33:55.904 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 317194240 bytes via buffered write, may repeat 2023-10-26T01:33:55.904 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 318242816 bytes via buffered write, may repeat 2023-10-26T01:33:55.905 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 319291392 bytes via buffered write, may repeat 2023-10-26T01:33:55.905 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 320339968 bytes via buffered write, may repeat 2023-10-26T01:33:55.905 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 321388544 bytes via buffered write, may repeat 2023-10-26T01:33:55.905 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 322437120 bytes via buffered write, may repeat 2023-10-26T01:33:55.905 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 323485696 bytes via buffered write, may repeat 2023-10-26T01:33:56.044 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 324534272 bytes via buffered write, may repeat 2023-10-26T01:33:56.044 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 325582848 bytes via buffered write, may repeat 2023-10-26T01:33:56.044 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 326631424 bytes via buffered write, may repeat 2023-10-26T01:33:56.045 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 327680000 bytes via buffered write, may repeat 2023-10-26T01:33:56.045 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 328728576 bytes via buffered write, may repeat 2023-10-26T01:33:56.045 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 329777152 bytes via buffered write, may repeat 2023-10-26T01:33:56.045 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 330825728 bytes via buffered write, may repeat 2023-10-26T01:33:56.046 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 331874304 bytes via buffered write, may repeat 2023-10-26T01:33:56.046 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 332922880 bytes via buffered write, may repeat 2023-10-26T01:33:56.046 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 333971456 bytes via buffered write, may repeat 2023-10-26T01:33:56.046 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 335020032 bytes via buffered write, may repeat 2023-10-26T01:33:56.046 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 336068608 bytes via buffered write, may repeat 2023-10-26T01:33:56.047 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 337117184 bytes via buffered write, may repeat 2023-10-26T01:33:56.047 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 338165760 bytes via buffered write, may repeat 2023-10-26T01:33:56.047 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 339214336 bytes via buffered write, may repeat 2023-10-26T01:33:56.047 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 340262912 bytes via buffered write, may repeat 2023-10-26T01:33:56.047 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 341311488 bytes via buffered write, may repeat 2023-10-26T01:33:56.048 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 342360064 bytes via buffered write, may repeat 2023-10-26T01:33:56.048 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 343408640 bytes via buffered write, may repeat 2023-10-26T01:33:56.048 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 344457216 bytes via buffered write, may repeat 2023-10-26T01:33:56.048 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 345505792 bytes via buffered write, may repeat 2023-10-26T01:33:56.049 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 346554368 bytes via buffered write, may repeat 2023-10-26T01:33:56.049 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 347602944 bytes via buffered write, may repeat 2023-10-26T01:33:56.049 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 348651520 bytes via buffered write, may repeat 2023-10-26T01:33:56.049 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 349700096 bytes via buffered write, may repeat 2023-10-26T01:33:56.049 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 350748672 bytes via buffered write, may repeat 2023-10-26T01:33:56.050 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 351797248 bytes via buffered write, may repeat 2023-10-26T01:33:56.050 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 352845824 bytes via buffered write, may repeat 2023-10-26T01:33:56.050 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 353894400 bytes via buffered write, may repeat 2023-10-26T01:33:56.050 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 354942976 bytes via buffered write, may repeat 2023-10-26T01:33:56.051 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 355991552 bytes via buffered write, may repeat 2023-10-26T01:33:56.051 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 357040128 bytes via buffered write, may repeat 2023-10-26T01:33:56.051 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 358088704 bytes via buffered write, may repeat 2023-10-26T01:33:56.051 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 359137280 bytes via buffered write, may repeat 2023-10-26T01:33:56.051 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 360185856 bytes via buffered write, may repeat 2023-10-26T01:33:56.052 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 361234432 bytes via buffered write, may repeat 2023-10-26T01:33:56.052 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 362283008 bytes via buffered write, may repeat 2023-10-26T01:33:56.052 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 363331584 bytes via buffered write, may repeat 2023-10-26T01:33:56.052 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 364380160 bytes via buffered write, may repeat 2023-10-26T01:33:56.052 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 365428736 bytes via buffered write, may repeat 2023-10-26T01:33:56.053 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 366477312 bytes via buffered write, may repeat 2023-10-26T01:33:56.053 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 367525888 bytes via buffered write, may repeat 2023-10-26T01:33:56.053 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 368574464 bytes via buffered write, may repeat 2023-10-26T01:33:56.053 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 369623040 bytes via buffered write, may repeat 2023-10-26T01:33:56.053 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 370671616 bytes via buffered write, may repeat 2023-10-26T01:33:56.054 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 371720192 bytes via buffered write, may repeat 2023-10-26T01:33:56.054 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 372768768 bytes via buffered write, may repeat 2023-10-26T01:33:56.054 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 373817344 bytes via buffered write, may repeat 2023-10-26T01:33:56.054 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 374865920 bytes via buffered write, may repeat 2023-10-26T01:33:56.055 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 375914496 bytes via buffered write, may repeat 2023-10-26T01:33:56.055 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 376963072 bytes via buffered write, may repeat 2023-10-26T01:33:56.055 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 378011648 bytes via buffered write, may repeat 2023-10-26T01:33:56.056 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 379060224 bytes via buffered write, may repeat 2023-10-26T01:33:56.056 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 380108800 bytes via buffered write, may repeat 2023-10-26T01:33:56.056 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 381157376 bytes via buffered write, may repeat 2023-10-26T01:33:56.057 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 382205952 bytes via buffered write, may repeat 2023-10-26T01:33:56.057 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 383254528 bytes via buffered write, may repeat 2023-10-26T01:33:56.058 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 384303104 bytes via buffered write, may repeat 2023-10-26T01:33:56.058 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 385351680 bytes via buffered write, may repeat 2023-10-26T01:33:56.058 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 386400256 bytes via buffered write, may repeat 2023-10-26T01:33:56.059 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 387448832 bytes via buffered write, may repeat 2023-10-26T01:33:56.059 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 388497408 bytes via buffered write, may repeat 2023-10-26T01:33:56.059 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 389545984 bytes via buffered write, may repeat 2023-10-26T01:33:56.060 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 390594560 bytes via buffered write, may repeat 2023-10-26T01:33:56.060 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 391643136 bytes via buffered write, may repeat 2023-10-26T01:33:56.060 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 392691712 bytes via buffered write, may repeat 2023-10-26T01:33:56.061 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 393740288 bytes via buffered write, may repeat 2023-10-26T01:33:56.061 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 394788864 bytes via buffered write, may repeat 2023-10-26T01:33:56.061 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 395837440 bytes via buffered write, may repeat 2023-10-26T01:33:56.061 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 396886016 bytes via buffered write, may repeat 2023-10-26T01:33:56.062 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 397934592 bytes via buffered write, may repeat 2023-10-26T01:33:56.062 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 398983168 bytes via buffered write, may repeat 2023-10-26T01:33:56.062 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 400031744 bytes via buffered write, may repeat 2023-10-26T01:33:56.063 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 401080320 bytes via buffered write, may repeat 2023-10-26T01:33:56.063 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 402128896 bytes via buffered write, may repeat 2023-10-26T01:33:56.063 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 403177472 bytes via buffered write, may repeat 2023-10-26T01:33:56.064 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 404226048 bytes via buffered write, may repeat 2023-10-26T01:33:56.064 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 405274624 bytes via buffered write, may repeat 2023-10-26T01:33:56.064 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 406323200 bytes via buffered write, may repeat 2023-10-26T01:33:56.064 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 407371776 bytes via buffered write, may repeat 2023-10-26T01:33:56.065 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 408420352 bytes via buffered write, may repeat 2023-10-26T01:33:56.065 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 409468928 bytes via buffered write, may repeat 2023-10-26T01:33:56.065 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 410517504 bytes via buffered write, may repeat 2023-10-26T01:33:56.065 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 411566080 bytes via buffered write, may repeat 2023-10-26T01:33:56.066 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 412614656 bytes via buffered write, may repeat 2023-10-26T01:33:56.066 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 413663232 bytes via buffered write, may repeat 2023-10-26T01:33:56.066 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 414711808 bytes via buffered write, may repeat 2023-10-26T01:33:56.067 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 415760384 bytes via buffered write, may repeat 2023-10-26T01:33:56.067 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 416808960 bytes via buffered write, may repeat 2023-10-26T01:33:56.067 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 417857536 bytes via buffered write, may repeat 2023-10-26T01:33:56.067 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 418906112 bytes via buffered write, may repeat 2023-10-26T01:33:56.068 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 419954688 bytes via buffered write, may repeat 2023-10-26T01:33:56.068 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 421003264 bytes via buffered write, may repeat 2023-10-26T01:33:56.068 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 422051840 bytes via buffered write, may repeat 2023-10-26T01:33:56.069 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 423100416 bytes via buffered write, may repeat 2023-10-26T01:33:56.069 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 424148992 bytes via buffered write, may repeat 2023-10-26T01:33:56.069 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 425197568 bytes via buffered write, may repeat 2023-10-26T01:33:56.070 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 426246144 bytes via buffered write, may repeat 2023-10-26T01:33:56.070 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 427294720 bytes via buffered write, may repeat 2023-10-26T01:33:56.073 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 428343296 bytes via buffered write, may repeat 2023-10-26T01:33:56.073 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 429391872 bytes via buffered write, may repeat 2023-10-26T01:33:56.074 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 430440448 bytes via buffered write, may repeat 2023-10-26T01:33:56.074 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 431489024 bytes via buffered write, may repeat 2023-10-26T01:33:56.074 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 432537600 bytes via buffered write, may repeat 2023-10-26T01:33:56.075 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 433586176 bytes via buffered write, may repeat 2023-10-26T01:33:56.075 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 434634752 bytes via buffered write, may repeat 2023-10-26T01:33:56.075 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 435683328 bytes via buffered write, may repeat 2023-10-26T01:33:56.076 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 436731904 bytes via buffered write, may repeat 2023-10-26T01:33:56.076 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 437780480 bytes via buffered write, may repeat 2023-10-26T01:33:56.077 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 438829056 bytes via buffered write, may repeat 2023-10-26T01:33:56.077 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 439877632 bytes via buffered write, may repeat 2023-10-26T01:33:56.077 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 440926208 bytes via buffered write, may repeat 2023-10-26T01:33:56.078 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 441974784 bytes via buffered write, may repeat 2023-10-26T01:33:56.078 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 443023360 bytes via buffered write, may repeat 2023-10-26T01:33:56.078 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 444071936 bytes via buffered write, may repeat 2023-10-26T01:33:56.079 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 445120512 bytes via buffered write, may repeat 2023-10-26T01:33:56.079 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 446169088 bytes via buffered write, may repeat 2023-10-26T01:33:56.079 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 447217664 bytes via buffered write, may repeat 2023-10-26T01:33:56.079 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 448266240 bytes via buffered write, may repeat 2023-10-26T01:33:56.079 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 449314816 bytes via buffered write, may repeat 2023-10-26T01:33:56.080 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 450363392 bytes via buffered write, may repeat 2023-10-26T01:33:56.080 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 451411968 bytes via buffered write, may repeat 2023-10-26T01:33:56.080 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 452460544 bytes via buffered write, may repeat 2023-10-26T01:33:56.080 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 453509120 bytes via buffered write, may repeat 2023-10-26T01:33:56.080 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 454557696 bytes via buffered write, may repeat 2023-10-26T01:33:56.081 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 455606272 bytes via buffered write, may repeat 2023-10-26T01:33:56.081 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 456654848 bytes via buffered write, may repeat 2023-10-26T01:33:56.081 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 457703424 bytes via buffered write, may repeat 2023-10-26T01:33:56.081 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 458752000 bytes via buffered write, may repeat 2023-10-26T01:33:56.081 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 459800576 bytes via buffered write, may repeat 2023-10-26T01:33:56.082 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 460849152 bytes via buffered write, may repeat 2023-10-26T01:33:56.082 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 461897728 bytes via buffered write, may repeat 2023-10-26T01:33:56.082 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 462946304 bytes via buffered write, may repeat 2023-10-26T01:33:56.082 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 463994880 bytes via buffered write, may repeat 2023-10-26T01:33:56.083 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 465043456 bytes via buffered write, may repeat 2023-10-26T01:33:56.083 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 466092032 bytes via buffered write, may repeat 2023-10-26T01:33:56.083 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 467140608 bytes via buffered write, may repeat 2023-10-26T01:33:56.083 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 468189184 bytes via buffered write, may repeat 2023-10-26T01:33:56.083 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 469237760 bytes via buffered write, may repeat 2023-10-26T01:33:56.084 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 470286336 bytes via buffered write, may repeat 2023-10-26T01:33:56.084 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 471334912 bytes via buffered write, may repeat 2023-10-26T01:33:56.084 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 472383488 bytes via buffered write, may repeat 2023-10-26T01:33:56.084 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 473432064 bytes via buffered write, may repeat 2023-10-26T01:33:56.084 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 474480640 bytes via buffered write, may repeat 2023-10-26T01:33:56.085 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 475529216 bytes via buffered write, may repeat 2023-10-26T01:33:56.085 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 476577792 bytes via buffered write, may repeat 2023-10-26T01:33:56.085 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 477626368 bytes via buffered write, may repeat 2023-10-26T01:33:56.085 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 478674944 bytes via buffered write, may repeat 2023-10-26T01:33:56.085 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 479723520 bytes via buffered write, may repeat 2023-10-26T01:33:56.086 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 480772096 bytes via buffered write, may repeat 2023-10-26T01:33:56.086 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 481820672 bytes via buffered write, may repeat 2023-10-26T01:33:56.086 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 482869248 bytes via buffered write, may repeat 2023-10-26T01:33:56.086 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 483917824 bytes via buffered write, may repeat 2023-10-26T01:33:56.087 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 484966400 bytes via buffered write, may repeat 2023-10-26T01:33:59.984 INFO:tasks.ceph.mon.a.smithi169.stderr:2023-10-26T01:33:59.982+0000 7f158b279640 -1 log_channel(cluster) log [ERR] : Health check failed: 2 full osd(s) (OSD_FULL) 2023-10-26T01:34:05.130 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T01:34:05.140 DEBUG:teuthology.orchestra.run.smithi170:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T01:34:07.943 INFO:tasks.ceph.mon.a.smithi169.stderr:2023-10-26T01:34:07.941+0000 7f158b279640 -1 log_channel(cluster) log [ERR] : Health check update: 4 full osd(s) (OSD_FULL) 2023-10-26T01:34:35.273 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T01:34:35.285 DEBUG:teuthology.orchestra.run.smithi170:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T01:34:57.475 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 486014976 bytes via buffered write, may repeat 2023-10-26T01:34:57.475 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 487063552 bytes via buffered write, may repeat 2023-10-26T01:34:57.475 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 488112128 bytes via buffered write, may repeat 2023-10-26T01:34:57.476 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 489160704 bytes via buffered write, may repeat 2023-10-26T01:34:57.476 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 490209280 bytes via buffered write, may repeat 2023-10-26T01:34:57.476 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 491257856 bytes via buffered write, may repeat 2023-10-26T01:34:57.476 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 492306432 bytes via buffered write, may repeat 2023-10-26T01:34:57.476 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 493355008 bytes via buffered write, may repeat 2023-10-26T01:34:57.477 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 494403584 bytes via buffered write, may repeat 2023-10-26T01:34:57.477 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 495452160 bytes via buffered write, may repeat 2023-10-26T01:34:57.477 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 496500736 bytes via buffered write, may repeat 2023-10-26T01:34:57.477 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 497549312 bytes via buffered write, may repeat 2023-10-26T01:34:57.477 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 498597888 bytes via buffered write, may repeat 2023-10-26T01:34:57.478 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 499646464 bytes via buffered write, may repeat 2023-10-26T01:34:57.478 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 500695040 bytes via buffered write, may repeat 2023-10-26T01:34:57.478 INFO:teuthology.orchestra.run.smithi169.stdout:done writing 500695040 bytes 2023-10-26T01:34:57.478 INFO:teuthology.orchestra.run.smithi169.stdout:successfully fsync'ed prior to getting full state reported 2023-10-26T01:34:57.478 INFO:teuthology.orchestra.run.smithi169.stdout:starting buffered write 2023-10-26T01:34:57.479 INFO:teuthology.orchestra.run.smithi169.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2023-10-26T01:34:57.479 INFO:teuthology.orchestra.run.smithi169.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2023-10-26T01:34:57.479 INFO:teuthology.orchestra.run.smithi169.stdout:wrote, now waiting 30s and then doing a close we expect to fail 2023-10-26T01:34:57.482 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T01:34:57.502 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-26T01:34:57.503 DEBUG:teuthology.orchestra.run.smithi169:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-26T01:34:57.630 DEBUG:teuthology.orchestra.run.smithi170:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-26T01:34:57.758 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-26T01:34:57.758 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:34:57.759 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:34:57.817 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:34:57.818 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:34:57.874 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:34:58.098 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.096+0000 7fa07114c640 1 -- 172.21.15.169:0/1208176756 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa06c0cdc30 msgr2=0x7fa06c0cbac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:34:58.098 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.096+0000 7fa07114c640 1 --2- 172.21.15.169:0/1208176756 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa06c0cdc30 0x7fa06c0cbac0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fa06c0b2650 tx=0x7fa06402c260 comp rx=0 tx=0).stop 2023-10-26T01:34:58.098 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.096+0000 7fa07114c640 1 -- 172.21.15.169:0/1208176756 shutdown_connections 2023-10-26T01:34:58.099 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.096+0000 7fa07114c640 1 --2- 172.21.15.169:0/1208176756 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa06c0cdc30 0x7fa06c0cbac0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:34:58.099 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.096+0000 7fa07114c640 1 --2- 172.21.15.169:0/1208176756 >> v2:172.21.15.170:3300/0 conn(0x7fa06c0ac930 0x7fa06c0cd6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:34:58.099 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.096+0000 7fa07114c640 1 -- 172.21.15.169:0/1208176756 >> 172.21.15.169:0/1208176756 conn(0x7fa06c069930 msgr2=0x7fa06c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:34:58.099 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.096+0000 7fa07114c640 1 -- 172.21.15.169:0/1208176756 shutdown_connections 2023-10-26T01:34:58.099 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.096+0000 7fa07114c640 1 -- 172.21.15.169:0/1208176756 wait complete. 2023-10-26T01:34:58.100 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.098+0000 7fa07114c640 1 Processor -- start 2023-10-26T01:34:58.100 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.098+0000 7fa07114c640 1 -- start start 2023-10-26T01:34:58.101 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.099+0000 7fa07114c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa06c0ac930 0x7fa06c14e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:34:58.101 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.099+0000 7fa07114c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa06c0cdc30 0x7fa06c0c8de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:34:58.101 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.099+0000 7fa07114c640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa06c1592d0 0x7fa06c0c9320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:34:58.101 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.099+0000 7fa07114c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa06c14e9a0 con 0x7fa06c0cdc30 2023-10-26T01:34:58.101 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.099+0000 7fa07114c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa06c0af1f0 con 0x7fa06c0ac930 2023-10-26T01:34:58.102 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.099+0000 7fa07114c640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa06c0af4d0 con 0x7fa06c1592d0 2023-10-26T01:34:58.102 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.099+0000 7fa06a59c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa06c0cdc30 0x7fa06c0c8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:34:58.102 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.099+0000 7fa06a59c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa06c0cdc30 0x7fa06c0c8de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:47158/0 (socket says 172.21.15.169:47158) 2023-10-26T01:34:58.102 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.099+0000 7fa06a59c640 1 -- 172.21.15.169:0/3070637102 learned_addr learned my addr 172.21.15.169:0/3070637102 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:34:58.104 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.099+0000 7fa06ad9d640 1 --2- 172.21.15.169:0/3070637102 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa06c0ac930 0x7fa06c14e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:34:58.104 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.099+0000 7fa06b59e640 1 --2- 172.21.15.169:0/3070637102 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa06c1592d0 0x7fa06c0c9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:34:58.104 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.099+0000 7fa06a59c640 1 -- 172.21.15.169:0/3070637102 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa06c1592d0 msgr2=0x7fa06c0c9320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:34:58.104 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.099+0000 7fa06a59c640 1 --2- 172.21.15.169:0/3070637102 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa06c1592d0 0x7fa06c0c9320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:34:58.105 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.099+0000 7fa06a59c640 1 -- 172.21.15.169:0/3070637102 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa06c0ac930 msgr2=0x7fa06c14e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:34:58.105 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.099+0000 7fa06a59c640 1 --2- 172.21.15.169:0/3070637102 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa06c0ac930 0x7fa06c14e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:34:58.105 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.099+0000 7fa06a59c640 1 -- 172.21.15.169:0/3070637102 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa064009900 con 0x7fa06c0cdc30 2023-10-26T01:34:58.105 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.100+0000 7fa06a59c640 1 --2- 172.21.15.169:0/3070637102 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa06c0cdc30 0x7fa06c0c8de0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fa06000c890 tx=0x7fa06000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:34:58.106 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.100+0000 7fa0537fe640 1 -- 172.21.15.169:0/3070637102 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa060004220 con 0x7fa06c0cdc30 2023-10-26T01:34:58.106 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.100+0000 7fa0537fe640 1 -- 172.21.15.169:0/3070637102 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa060004470 con 0x7fa06c0cdc30 2023-10-26T01:34:58.106 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.100+0000 7fa0537fe640 1 -- 172.21.15.169:0/3070637102 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa060007c10 con 0x7fa06c0cdc30 2023-10-26T01:34:58.106 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.100+0000 7fa07114c640 1 -- 172.21.15.169:0/3070637102 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa06c0af7b0 con 0x7fa06c0cdc30 2023-10-26T01:34:58.106 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.100+0000 7fa07114c640 1 -- 172.21.15.169:0/3070637102 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa06c0afd70 con 0x7fa06c0cdc30 2023-10-26T01:34:58.106 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.101+0000 7fa07114c640 1 -- 172.21.15.169:0/3070637102 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa06c0aff90 con 0x7fa06c0cdc30 2023-10-26T01:34:58.107 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.102+0000 7fa0537fe640 1 -- 172.21.15.169:0/3070637102 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fa060005020 con 0x7fa06c0cdc30 2023-10-26T01:34:58.107 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.104+0000 7fa0537fe640 1 --2- 172.21.15.169:0/3070637102 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa05806ae40 0x7fa05806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:34:58.108 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.104+0000 7fa0537fe640 1 -- 172.21.15.169:0/3070637102 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa060089c40 con 0x7fa06c0cdc30 2023-10-26T01:34:58.108 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.104+0000 7fa0537fe640 1 -- 172.21.15.169:0/3070637102 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=70}) v3 -- 0x7fa058072970 con 0x7fa06c0cdc30 2023-10-26T01:34:58.108 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.104+0000 7fa06ad9d640 1 --2- 172.21.15.169:0/3070637102 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa05806ae40 0x7fa05806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:34:58.108 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.106+0000 7fa06ad9d640 1 --2- 172.21.15.169:0/3070637102 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa05806ae40 0x7fa05806d300 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fa05c000f60 tx=0x7fa05c009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:34:58.110 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.109+0000 7fa0537fe640 1 -- 172.21.15.169:0/3070637102 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa06001d030 con 0x7fa06c0cdc30 2023-10-26T01:34:58.304 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.302+0000 7fa07114c640 1 -- 172.21.15.169:0/3070637102 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa06c04db10 con 0x7fa06c0cdc30 2023-10-26T01:34:58.305 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.304+0000 7fa0537fe640 1 -- 172.21.15.169:0/3070637102 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) v1 ==== 74+0+16671 (secure 0 0 0) 0x7fa06001d030 con 0x7fa06c0cdc30 2023-10-26T01:34:58.306 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:34:58.306 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":69,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:34:04.037343+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-10-26T01:33:38.593237+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-10-26T01:33:39.753692+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","full","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","full","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","full","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["backfillfull","exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","full","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:34:58.308 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.306+0000 7fa07114c640 1 -- 172.21.15.169:0/3070637102 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa05806ae40 msgr2=0x7fa05806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:34:58.308 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.306+0000 7fa07114c640 1 --2- 172.21.15.169:0/3070637102 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa05806ae40 0x7fa05806d300 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fa05c000f60 tx=0x7fa05c009040 comp rx=0 tx=0).stop 2023-10-26T01:34:58.309 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.306+0000 7fa07114c640 1 -- 172.21.15.169:0/3070637102 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa06c0cdc30 msgr2=0x7fa06c0c8de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:34:58.309 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.306+0000 7fa07114c640 1 --2- 172.21.15.169:0/3070637102 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa06c0cdc30 0x7fa06c0c8de0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fa06000c890 tx=0x7fa06000cd50 comp rx=0 tx=0).stop 2023-10-26T01:34:58.309 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.306+0000 7fa07114c640 1 -- 172.21.15.169:0/3070637102 shutdown_connections 2023-10-26T01:34:58.309 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.306+0000 7fa07114c640 1 --2- 172.21.15.169:0/3070637102 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa05806ae40 0x7fa05806d300 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:34:58.309 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.306+0000 7fa07114c640 1 --2- 172.21.15.169:0/3070637102 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa06c0cdc30 0x7fa06c0c8de0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:34:58.310 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.306+0000 7fa07114c640 1 --2- 172.21.15.169:0/3070637102 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa06c1592d0 0x7fa06c0c9320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:34:58.310 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.306+0000 7fa07114c640 1 --2- 172.21.15.169:0/3070637102 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa06c0ac930 0x7fa06c14e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:34:58.310 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.306+0000 7fa07114c640 1 -- 172.21.15.169:0/3070637102 >> 172.21.15.169:0/3070637102 conn(0x7fa06c069930 msgr2=0x7fa06c0adf00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:34:58.310 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.307+0000 7fa07114c640 1 -- 172.21.15.169:0/3070637102 shutdown_connections 2023-10-26T01:34:58.310 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.307+0000 7fa07114c640 1 -- 172.21.15.169:0/3070637102 wait complete. 2023-10-26T01:34:58.327 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T01:30:32.200808+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-26T01:33:38.593237+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2023-10-26T01:33:39.753692+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T01:34:58.328 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T01:34:58.552 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.550+0000 7efe5a0d7640 1 -- 172.21.15.169:0/3872208214 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe54159480 msgr2=0x7efe540b02e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:34:58.553 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.550+0000 7efe5a0d7640 1 --2- 172.21.15.169:0/3872208214 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe54159480 0x7efe540b02e0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7efe44005fd0 tx=0x7efe4402bd90 comp rx=0 tx=0).stop 2023-10-26T01:34:58.553 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.551+0000 7efe5a0d7640 1 -- 172.21.15.169:0/3872208214 shutdown_connections 2023-10-26T01:34:58.553 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.551+0000 7efe5a0d7640 1 --2- 172.21.15.169:0/3872208214 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7efe540b0820 0x7efe540aaf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:34:58.553 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.551+0000 7efe5a0d7640 1 --2- 172.21.15.169:0/3872208214 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe54159480 0x7efe540b02e0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:34:58.554 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.551+0000 7efe5a0d7640 1 -- 172.21.15.169:0/3872208214 >> 172.21.15.169:0/3872208214 conn(0x7efe54069930 msgr2=0x7efe54069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:34:58.554 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.551+0000 7efe5a0d7640 1 -- 172.21.15.169:0/3872208214 shutdown_connections 2023-10-26T01:34:58.554 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.551+0000 7efe5a0d7640 1 -- 172.21.15.169:0/3872208214 wait complete. 2023-10-26T01:34:58.554 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.552+0000 7efe5a0d7640 1 Processor -- start 2023-10-26T01:34:58.555 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.553+0000 7efe5a0d7640 1 -- start start 2023-10-26T01:34:58.557 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.553+0000 7efe5a0d7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe540ab4c0 0x7efe54071100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:34:58.557 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.553+0000 7efe5a0d7640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7efe540b0820 0x7efe5414d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:34:58.557 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.553+0000 7efe5a0d7640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe54159480 0x7efe5414dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:34:58.557 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.554+0000 7efe5a0d7640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe541f5910 con 0x7efe54159480 2023-10-26T01:34:58.557 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.554+0000 7efe5a0d7640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe5414e3d0 con 0x7efe540ab4c0 2023-10-26T01:34:58.558 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.554+0000 7efe5a0d7640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe541f4f70 con 0x7efe540b0820 2023-10-26T01:34:58.558 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.554+0000 7efe53fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe54159480 0x7efe5414dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:34:58.558 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.554+0000 7efe53fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe54159480 0x7efe5414dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:47184/0 (socket says 172.21.15.169:47184) 2023-10-26T01:34:58.558 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.554+0000 7efe53fff640 1 -- 172.21.15.169:0/1218159378 learned_addr learned my addr 172.21.15.169:0/1218159378 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:34:58.558 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.554+0000 7efe4bfff640 1 --2- 172.21.15.169:0/1218159378 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7efe540b0820 0x7efe5414d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:34:58.559 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.554+0000 7efe537fe640 1 --2- 172.21.15.169:0/1218159378 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe540ab4c0 0x7efe54071100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:34:58.559 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.554+0000 7efe53fff640 1 -- 172.21.15.169:0/1218159378 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7efe540b0820 msgr2=0x7efe5414d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:34:58.559 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.554+0000 7efe53fff640 1 --2- 172.21.15.169:0/1218159378 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7efe540b0820 0x7efe5414d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:34:58.559 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.554+0000 7efe53fff640 1 -- 172.21.15.169:0/1218159378 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe540ab4c0 msgr2=0x7efe54071100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:34:58.559 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.554+0000 7efe53fff640 1 --2- 172.21.15.169:0/1218159378 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe540ab4c0 0x7efe54071100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:34:58.559 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.554+0000 7efe53fff640 1 -- 172.21.15.169:0/1218159378 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe44009560 con 0x7efe54159480 2023-10-26T01:34:58.560 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.555+0000 7efe53fff640 1 --2- 172.21.15.169:0/1218159378 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe54159480 0x7efe5414dde0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7efe4c00c780 tx=0x7efe4c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:34:58.560 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.555+0000 7efe50ff9640 1 -- 172.21.15.169:0/1218159378 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe4c004b60 con 0x7efe54159480 2023-10-26T01:34:58.560 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.555+0000 7efe50ff9640 1 -- 172.21.15.169:0/1218159378 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7efe4c00ce90 con 0x7efe54159480 2023-10-26T01:34:58.560 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.555+0000 7efe5a0d7640 1 -- 172.21.15.169:0/1218159378 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe541f5250 con 0x7efe54159480 2023-10-26T01:34:58.560 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.555+0000 7efe50ff9640 1 -- 172.21.15.169:0/1218159378 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe4c00aab0 con 0x7efe54159480 2023-10-26T01:34:58.561 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.555+0000 7efe5a0d7640 1 -- 172.21.15.169:0/1218159378 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe541f5d30 con 0x7efe54159480 2023-10-26T01:34:58.561 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.556+0000 7efe5a0d7640 1 -- 172.21.15.169:0/1218159378 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe54066af0 con 0x7efe54159480 2023-10-26T01:34:58.562 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.557+0000 7efe50ff9640 1 -- 172.21.15.169:0/1218159378 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7efe4c00ac10 con 0x7efe54159480 2023-10-26T01:34:58.562 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.558+0000 7efe50ff9640 1 --2- 172.21.15.169:0/1218159378 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7efe3406ae40 0x7efe3406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:34:58.562 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.558+0000 7efe50ff9640 1 -- 172.21.15.169:0/1218159378 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efe4c089b00 con 0x7efe54159480 2023-10-26T01:34:58.562 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.558+0000 7efe50ff9640 1 -- 172.21.15.169:0/1218159378 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=70}) v3 -- 0x7efe34072970 con 0x7efe54159480 2023-10-26T01:34:58.562 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.559+0000 7efe537fe640 1 --2- 172.21.15.169:0/1218159378 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7efe3406ae40 0x7efe3406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:34:58.563 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.560+0000 7efe537fe640 1 --2- 172.21.15.169:0/1218159378 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7efe3406ae40 0x7efe3406d300 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7efe44035000 tx=0x7efe44004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:34:58.564 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.562+0000 7efe50ff9640 1 -- 172.21.15.169:0/1218159378 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7efe4c08be60 con 0x7efe54159480 2023-10-26T01:34:58.757 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.756+0000 7efe5a0d7640 1 -- 172.21.15.169:0/1218159378 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efe541f5a50 con 0x7efe54159480 2023-10-26T01:34:58.758 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.756+0000 7efe50ff9640 1 -- 172.21.15.169:0/1218159378 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v69) v1 ==== 93+0+10 (secure 0 0 0) 0x7efe541f5a50 con 0x7efe54159480 2023-10-26T01:34:58.758 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 1 2023-10-26T01:34:58.761 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.759+0000 7efe5a0d7640 1 -- 172.21.15.169:0/1218159378 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7efe3406ae40 msgr2=0x7efe3406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:34:58.761 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.759+0000 7efe5a0d7640 1 --2- 172.21.15.169:0/1218159378 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7efe3406ae40 0x7efe3406d300 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7efe44035000 tx=0x7efe44004730 comp rx=0 tx=0).stop 2023-10-26T01:34:58.761 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.759+0000 7efe5a0d7640 1 -- 172.21.15.169:0/1218159378 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe54159480 msgr2=0x7efe5414dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:34:58.762 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.759+0000 7efe5a0d7640 1 --2- 172.21.15.169:0/1218159378 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe54159480 0x7efe5414dde0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7efe4c00c780 tx=0x7efe4c00cc40 comp rx=0 tx=0).stop 2023-10-26T01:34:58.762 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.759+0000 7efe5a0d7640 1 -- 172.21.15.169:0/1218159378 shutdown_connections 2023-10-26T01:34:58.762 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.759+0000 7efe5a0d7640 1 --2- 172.21.15.169:0/1218159378 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7efe3406ae40 0x7efe3406d300 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:34:58.762 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.759+0000 7efe5a0d7640 1 --2- 172.21.15.169:0/1218159378 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe54159480 0x7efe5414dde0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:34:58.762 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.759+0000 7efe5a0d7640 1 --2- 172.21.15.169:0/1218159378 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7efe540b0820 0x7efe5414d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:34:58.763 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.759+0000 7efe5a0d7640 1 --2- 172.21.15.169:0/1218159378 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe540ab4c0 0x7efe54071100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:34:58.763 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.759+0000 7efe5a0d7640 1 -- 172.21.15.169:0/1218159378 >> 172.21.15.169:0/1218159378 conn(0x7efe54069930 msgr2=0x7efe54069d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:34:58.763 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.760+0000 7efe5a0d7640 1 -- 172.21.15.169:0/1218159378 shutdown_connections 2023-10-26T01:34:58.763 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.760+0000 7efe5a0d7640 1 -- 172.21.15.169:0/1218159378 wait complete. 2023-10-26T01:34:58.780 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-26T01:34:58.996 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.994+0000 7f0e51c10640 1 -- 172.21.15.169:0/1358090198 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e4c15b820 msgr2=0x7f0e4c0cc780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:34:58.996 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.994+0000 7f0e51c10640 1 --2- 172.21.15.169:0/1358090198 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e4c15b820 0x7f0e4c0cc780 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f0e4c0c87a0 tx=0x7f0e3c02be00 comp rx=0 tx=0).stop 2023-10-26T01:34:58.996 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.994+0000 7f0e51c10640 1 -- 172.21.15.169:0/1358090198 shutdown_connections 2023-10-26T01:34:58.996 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.994+0000 7f0e51c10640 1 --2- 172.21.15.169:0/1358090198 >> v2:172.21.15.170:3300/0 conn(0x7f0e4c15bc00 0x7f0e4c0ce540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:34:58.997 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.994+0000 7f0e51c10640 1 --2- 172.21.15.169:0/1358090198 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e4c15b820 0x7f0e4c0cc780 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:34:58.997 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.994+0000 7f0e51c10640 1 -- 172.21.15.169:0/1358090198 >> 172.21.15.169:0/1358090198 conn(0x7f0e4c065670 msgr2=0x7f0e4c065a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:34:58.997 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.994+0000 7f0e51c10640 1 -- 172.21.15.169:0/1358090198 shutdown_connections 2023-10-26T01:34:58.997 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.994+0000 7f0e51c10640 1 -- 172.21.15.169:0/1358090198 wait complete. 2023-10-26T01:34:59.000 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.996+0000 7f0e51c10640 1 Processor -- start 2023-10-26T01:34:59.000 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.996+0000 7f0e51c10640 1 -- start start 2023-10-26T01:34:59.000 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.997+0000 7f0e51c10640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0e4c0cf530 0x7f0e4c1f2db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:34:59.000 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.997+0000 7f0e51c10640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e4c15b820 0x7f0e4c1f4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:34:59.001 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.997+0000 7f0e51c10640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0e4c15bc00 0x7f0e4c0cab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:34:59.001 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.997+0000 7f0e51c10640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e4c1f32f0 con 0x7f0e4c15b820 2023-10-26T01:34:59.001 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.997+0000 7f0e51c10640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e4c1f5080 con 0x7f0e4c0cf530 2023-10-26T01:34:59.001 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.997+0000 7f0e51c10640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e4c15d250 con 0x7f0e4c15bc00 2023-10-26T01:34:59.001 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.997+0000 7f0e42ffd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e4c15b820 0x7f0e4c1f4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:34:59.002 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.997+0000 7f0e42ffd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e4c15b820 0x7f0e4c1f4b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:47198/0 (socket says 172.21.15.169:47198) 2023-10-26T01:34:59.002 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.997+0000 7f0e42ffd640 1 -- 172.21.15.169:0/2257503456 learned_addr learned my addr 172.21.15.169:0/2257503456 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:34:59.002 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.997+0000 7f0e4b7fe640 1 --2- 172.21.15.169:0/2257503456 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0e4c0cf530 0x7f0e4c1f2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:34:59.002 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.997+0000 7f0e42ffd640 1 -- 172.21.15.169:0/2257503456 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0e4c15bc00 msgr2=0x7f0e4c0cab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:34:59.002 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.997+0000 7f0e42ffd640 1 --2- 172.21.15.169:0/2257503456 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0e4c15bc00 0x7f0e4c0cab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:34:59.003 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.997+0000 7f0e42ffd640 1 -- 172.21.15.169:0/2257503456 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0e4c0cf530 msgr2=0x7f0e4c1f2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:34:59.003 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.997+0000 7f0e42ffd640 1 --2- 172.21.15.169:0/2257503456 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0e4c0cf530 0x7f0e4c1f2db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:34:59.003 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.997+0000 7f0e42ffd640 1 -- 172.21.15.169:0/2257503456 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e3c009490 con 0x7f0e4c15b820 2023-10-26T01:34:59.003 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.998+0000 7f0e42ffd640 1 --2- 172.21.15.169:0/2257503456 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e4c15b820 0x7f0e4c1f4b40 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f0e3400d740 tx=0x7f0e3400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:34:59.003 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.998+0000 7f0e48ff9640 1 -- 172.21.15.169:0/2257503456 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e34009c30 con 0x7f0e4c15b820 2023-10-26T01:34:59.004 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.998+0000 7f0e48ff9640 1 -- 172.21.15.169:0/2257503456 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0e34009e80 con 0x7f0e4c15b820 2023-10-26T01:34:59.004 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.998+0000 7f0e48ff9640 1 -- 172.21.15.169:0/2257503456 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e340106c0 con 0x7f0e4c15b820 2023-10-26T01:34:59.004 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.998+0000 7f0e51c10640 1 -- 172.21.15.169:0/2257503456 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e4c15d530 con 0x7f0e4c15b820 2023-10-26T01:34:59.004 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.998+0000 7f0e51c10640 1 -- 172.21.15.169:0/2257503456 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e4c15daf0 con 0x7f0e4c15b820 2023-10-26T01:34:59.005 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:58.999+0000 7f0e51c10640 1 -- 172.21.15.169:0/2257503456 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e4c04db10 con 0x7f0e4c15b820 2023-10-26T01:34:59.005 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.000+0000 7f0e48ff9640 1 -- 172.21.15.169:0/2257503456 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f0e34010820 con 0x7f0e4c15b820 2023-10-26T01:34:59.006 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.001+0000 7f0e48ff9640 1 --2- 172.21.15.169:0/2257503456 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f0e2c06ae40 0x7f0e2c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:34:59.006 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.001+0000 7f0e48ff9640 1 -- 172.21.15.169:0/2257503456 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0e34089b40 con 0x7f0e4c15b820 2023-10-26T01:34:59.006 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.001+0000 7f0e48ff9640 1 -- 172.21.15.169:0/2257503456 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=70}) v3 -- 0x7f0e2c072970 con 0x7f0e4c15b820 2023-10-26T01:34:59.006 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.002+0000 7f0e4b7fe640 1 --2- 172.21.15.169:0/2257503456 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f0e2c06ae40 0x7f0e2c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:34:59.007 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.003+0000 7f0e4b7fe640 1 --2- 172.21.15.169:0/2257503456 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f0e2c06ae40 0x7f0e2c06d300 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c036000 tx=0x7f0e3c004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:34:59.007 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.005+0000 7f0e48ff9640 1 -- 172.21.15.169:0/2257503456 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0e34054b70 con 0x7f0e4c15b820 2023-10-26T01:34:59.201 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.199+0000 7f0e51c10640 1 -- 172.21.15.169:0/2257503456 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0e4c0b40d0 con 0x7f0e4c15b820 2023-10-26T01:34:59.202 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.200+0000 7f0e48ff9640 1 -- 172.21.15.169:0/2257503456 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v69) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0e340210a0 con 0x7f0e4c15b820 2023-10-26T01:34:59.202 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 32 2023-10-26T01:34:59.204 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.202+0000 7f0e51c10640 1 -- 172.21.15.169:0/2257503456 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f0e2c06ae40 msgr2=0x7f0e2c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:34:59.205 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.202+0000 7f0e51c10640 1 --2- 172.21.15.169:0/2257503456 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f0e2c06ae40 0x7f0e2c06d300 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c036000 tx=0x7f0e3c004730 comp rx=0 tx=0).stop 2023-10-26T01:34:59.205 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.202+0000 7f0e51c10640 1 -- 172.21.15.169:0/2257503456 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e4c15b820 msgr2=0x7f0e4c1f4b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:34:59.205 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.202+0000 7f0e51c10640 1 --2- 172.21.15.169:0/2257503456 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e4c15b820 0x7f0e4c1f4b40 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f0e3400d740 tx=0x7f0e3400dc00 comp rx=0 tx=0).stop 2023-10-26T01:34:59.205 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.203+0000 7f0e51c10640 1 -- 172.21.15.169:0/2257503456 shutdown_connections 2023-10-26T01:34:59.205 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.203+0000 7f0e51c10640 1 --2- 172.21.15.169:0/2257503456 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f0e2c06ae40 0x7f0e2c06d300 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:34:59.206 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.203+0000 7f0e51c10640 1 --2- 172.21.15.169:0/2257503456 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e4c15b820 0x7f0e4c1f4b40 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:34:59.206 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.203+0000 7f0e51c10640 1 --2- 172.21.15.169:0/2257503456 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0e4c15bc00 0x7f0e4c0cab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:34:59.206 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.203+0000 7f0e51c10640 1 --2- 172.21.15.169:0/2257503456 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0e4c0cf530 0x7f0e4c1f2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:34:59.206 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.203+0000 7f0e51c10640 1 -- 172.21.15.169:0/2257503456 >> 172.21.15.169:0/2257503456 conn(0x7f0e4c065670 msgr2=0x7f0e4c0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:34:59.207 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.203+0000 7f0e51c10640 1 -- 172.21.15.169:0/2257503456 shutdown_connections 2023-10-26T01:34:59.207 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.203+0000 7f0e51c10640 1 -- 172.21.15.169:0/2257503456 wait complete. 2023-10-26T01:34:59.223 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-26T01:34:59.436 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.434+0000 7f5ad0002640 1 -- 172.21.15.169:0/2814490364 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5ac81597c0 msgr2=0x7f5ac80ac200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:34:59.437 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.434+0000 7f5ad0002640 1 --2- 172.21.15.169:0/2814490364 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5ac81597c0 0x7f5ac80ac200 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f5ac80cd2c0 tx=0x7f5ab802bdb0 comp rx=0 tx=0).stop 2023-10-26T01:34:59.437 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.434+0000 7f5ad0002640 1 -- 172.21.15.169:0/2814490364 shutdown_connections 2023-10-26T01:34:59.437 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.434+0000 7f5ad0002640 1 --2- 172.21.15.169:0/2814490364 >> v2:172.21.15.170:3300/0 conn(0x7f5ac80b0c80 0x7f5ac80cd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:34:59.437 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.434+0000 7f5ad0002640 1 --2- 172.21.15.169:0/2814490364 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5ac81597c0 0x7f5ac80ac200 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:34:59.437 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.434+0000 7f5ad0002640 1 -- 172.21.15.169:0/2814490364 >> 172.21.15.169:0/2814490364 conn(0x7f5ac8069930 msgr2=0x7f5ac8069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:34:59.438 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.434+0000 7f5ad0002640 1 -- 172.21.15.169:0/2814490364 shutdown_connections 2023-10-26T01:34:59.438 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.434+0000 7f5ad0002640 1 -- 172.21.15.169:0/2814490364 wait complete. 2023-10-26T01:34:59.439 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.436+0000 7f5ad0002640 1 Processor -- start 2023-10-26T01:34:59.439 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.436+0000 7f5ad0002640 1 -- start start 2023-10-26T01:34:59.439 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.437+0000 7f5ad0002640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5ac80b0c80 0x7f5ac81ef730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:34:59.439 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.437+0000 7f5ad0002640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5ac81593e0 0x7f5ac8158250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:34:59.440 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.437+0000 7f5ad0002640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ac81597c0 0x7f5ac81ecc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:34:59.440 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.437+0000 7f5ad0002640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ac8157f30 con 0x7f5ac81593e0 2023-10-26T01:34:59.440 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.437+0000 7f5ad0002640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ac806edf0 con 0x7f5ac81597c0 2023-10-26T01:34:59.440 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.437+0000 7f5ad0002640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ac81ed170 con 0x7f5ac80b0c80 2023-10-26T01:34:59.440 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.437+0000 7f5acd59d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5ac81593e0 0x7f5ac8158250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:34:59.441 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.437+0000 7f5acd59d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5ac81593e0 0x7f5ac8158250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:47214/0 (socket says 172.21.15.169:47214) 2023-10-26T01:34:59.441 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.437+0000 7f5acd59d640 1 -- 172.21.15.169:0/3852211364 learned_addr learned my addr 172.21.15.169:0/3852211364 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:34:59.442 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.438+0000 7f5acdd9e640 1 --2- 172.21.15.169:0/3852211364 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5ac80b0c80 0x7f5ac81ef730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:34:59.442 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.438+0000 7f5acd59d640 1 -- 172.21.15.169:0/3852211364 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5ac80b0c80 msgr2=0x7f5ac81ef730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:34:59.443 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.438+0000 7f5ace59f640 1 --2- 172.21.15.169:0/3852211364 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ac81597c0 0x7f5ac81ecc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:34:59.443 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.438+0000 7f5acd59d640 1 --2- 172.21.15.169:0/3852211364 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5ac80b0c80 0x7f5ac81ef730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:34:59.443 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.438+0000 7f5acd59d640 1 -- 172.21.15.169:0/3852211364 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ac81597c0 msgr2=0x7f5ac81ecc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:34:59.443 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.438+0000 7f5acd59d640 1 --2- 172.21.15.169:0/3852211364 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ac81597c0 0x7f5ac81ecc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:34:59.443 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.438+0000 7f5acd59d640 1 -- 172.21.15.169:0/3852211364 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ab8009560 con 0x7f5ac81593e0 2023-10-26T01:34:59.444 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.438+0000 7f5acdd9e640 1 --2- 172.21.15.169:0/3852211364 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5ac80b0c80 0x7f5ac81ef730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:34:59.444 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.438+0000 7f5acd59d640 1 --2- 172.21.15.169:0/3852211364 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5ac81593e0 0x7f5ac8158250 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f5ab8000c00 tx=0x7f5ab802dd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:34:59.444 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.438+0000 7f5abe7fc640 1 -- 172.21.15.169:0/3852211364 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab8005670 con 0x7f5ac81593e0 2023-10-26T01:34:59.444 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.438+0000 7f5abe7fc640 1 -- 172.21.15.169:0/3852211364 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5ab80038f0 con 0x7f5ac81593e0 2023-10-26T01:34:59.444 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.439+0000 7f5ad0002640 1 -- 172.21.15.169:0/3852211364 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ac81efd80 con 0x7f5ac81593e0 2023-10-26T01:34:59.445 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.439+0000 7f5abe7fc640 1 -- 172.21.15.169:0/3852211364 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab8003ce0 con 0x7f5ac81593e0 2023-10-26T01:34:59.445 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.439+0000 7f5ad0002640 1 -- 172.21.15.169:0/3852211364 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ac81f0340 con 0x7f5ac81593e0 2023-10-26T01:34:59.445 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.440+0000 7f5ad0002640 1 -- 172.21.15.169:0/3852211364 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ac806ef80 con 0x7f5ac81593e0 2023-10-26T01:34:59.445 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.441+0000 7f5abe7fc640 1 -- 172.21.15.169:0/3852211364 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f5ab803e450 con 0x7f5ac81593e0 2023-10-26T01:34:59.446 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.442+0000 7f5abe7fc640 1 --2- 172.21.15.169:0/3852211364 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5aac06ae40 0x7f5aac06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:34:59.446 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.442+0000 7f5abe7fc640 1 -- 172.21.15.169:0/3852211364 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5ab80abf20 con 0x7f5ac81593e0 2023-10-26T01:34:59.446 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.442+0000 7f5abe7fc640 1 -- 172.21.15.169:0/3852211364 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=70}) v3 -- 0x7f5aac072970 con 0x7f5ac81593e0 2023-10-26T01:34:59.447 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.443+0000 7f5acdd9e640 1 --2- 172.21.15.169:0/3852211364 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5aac06ae40 0x7f5aac06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:34:59.447 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.444+0000 7f5acdd9e640 1 --2- 172.21.15.169:0/3852211364 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5aac06ae40 0x7f5aac06d300 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f5ac81588a0 tx=0x7f5ac4009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:34:59.448 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.446+0000 7f5abe7fc640 1 -- 172.21.15.169:0/3852211364 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5ab8077180 con 0x7f5ac81593e0 2023-10-26T01:34:59.637 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.635+0000 7f5ad0002640 1 -- 172.21.15.169:0/3852211364 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5ac804db10 con 0x7f5ac81593e0 2023-10-26T01:34:59.638 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.636+0000 7f5abe7fc640 1 -- 172.21.15.169:0/3852211364 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v69) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5ab8042070 con 0x7f5ac81593e0 2023-10-26T01:34:59.638 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 64 2023-10-26T01:34:59.640 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.638+0000 7f5ad0002640 1 -- 172.21.15.169:0/3852211364 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5aac06ae40 msgr2=0x7f5aac06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:34:59.641 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.638+0000 7f5ad0002640 1 --2- 172.21.15.169:0/3852211364 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5aac06ae40 0x7f5aac06d300 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f5ac81588a0 tx=0x7f5ac4009040 comp rx=0 tx=0).stop 2023-10-26T01:34:59.641 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.638+0000 7f5ad0002640 1 -- 172.21.15.169:0/3852211364 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5ac81593e0 msgr2=0x7f5ac8158250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:34:59.641 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.638+0000 7f5ad0002640 1 --2- 172.21.15.169:0/3852211364 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5ac81593e0 0x7f5ac8158250 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f5ab8000c00 tx=0x7f5ab802dd80 comp rx=0 tx=0).stop 2023-10-26T01:34:59.641 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.639+0000 7f5ad0002640 1 -- 172.21.15.169:0/3852211364 shutdown_connections 2023-10-26T01:34:59.641 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.639+0000 7f5ad0002640 1 --2- 172.21.15.169:0/3852211364 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5aac06ae40 0x7f5aac06d300 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:34:59.642 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.639+0000 7f5ad0002640 1 --2- 172.21.15.169:0/3852211364 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5ac81593e0 0x7f5ac8158250 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:34:59.642 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.639+0000 7f5ad0002640 1 --2- 172.21.15.169:0/3852211364 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ac81597c0 0x7f5ac81ecc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:34:59.642 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.639+0000 7f5ad0002640 1 --2- 172.21.15.169:0/3852211364 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5ac80b0c80 0x7f5ac81ef730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:34:59.642 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.639+0000 7f5ad0002640 1 -- 172.21.15.169:0/3852211364 >> 172.21.15.169:0/3852211364 conn(0x7f5ac8069930 msgr2=0x7f5ac815bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:34:59.642 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.639+0000 7f5ad0002640 1 -- 172.21.15.169:0/3852211364 shutdown_connections 2023-10-26T01:34:59.643 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.639+0000 7f5ad0002640 1 -- 172.21.15.169:0/3852211364 wait complete. 2023-10-26T01:34:59.659 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-26T01:34:59.876 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.874+0000 7f7ad98ea640 1 -- 172.21.15.169:0/218631103 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7ad40b0720 msgr2=0x7f7ad40d0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:34:59.876 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.874+0000 7f7ad98ea640 1 --2- 172.21.15.169:0/218631103 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7ad40b0720 0x7f7ad40d0e10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7ac0005fd0 tx=0x7f7ac002be10 comp rx=0 tx=0).stop 2023-10-26T01:34:59.876 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.874+0000 7f7ad98ea640 1 -- 172.21.15.169:0/218631103 shutdown_connections 2023-10-26T01:34:59.877 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.874+0000 7f7ad98ea640 1 --2- 172.21.15.169:0/218631103 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7ad40b0720 0x7f7ad40d0e10 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:34:59.877 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.874+0000 7f7ad98ea640 1 --2- 172.21.15.169:0/218631103 >> v2:172.21.15.170:3300/0 conn(0x7f7ad415ece0 0x7f7ad40b0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:34:59.877 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.874+0000 7f7ad98ea640 1 -- 172.21.15.169:0/218631103 >> 172.21.15.169:0/218631103 conn(0x7f7ad40d1370 msgr2=0x7f7ad40d1780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:34:59.877 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.874+0000 7f7ad98ea640 1 -- 172.21.15.169:0/218631103 shutdown_connections 2023-10-26T01:34:59.878 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.874+0000 7f7ad98ea640 1 -- 172.21.15.169:0/218631103 wait complete. 2023-10-26T01:34:59.878 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.876+0000 7f7ad98ea640 1 Processor -- start 2023-10-26T01:34:59.879 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.876+0000 7f7ad98ea640 1 -- start start 2023-10-26T01:34:59.879 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.877+0000 7f7ad98ea640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ad4065490 0x7f7ad40b12e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:34:59.879 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.877+0000 7f7ad98ea640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7ad40b0720 0x7f7ad41f51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:34:59.879 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.877+0000 7f7ad98ea640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ad415ece0 0x7f7ad41f5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:34:59.880 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.877+0000 7f7ad98ea640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ad40c9eb0 con 0x7f7ad415ece0 2023-10-26T01:34:59.880 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.877+0000 7f7ad98ea640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ad415a2c0 con 0x7f7ad4065490 2023-10-26T01:34:59.880 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.877+0000 7f7ad98ea640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ad40ca340 con 0x7f7ad40b0720 2023-10-26T01:34:59.881 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.877+0000 7f7ad90e9640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ad415ece0 0x7f7ad41f5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:34:59.882 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.877+0000 7f7ad90e9640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ad415ece0 0x7f7ad41f5700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:47220/0 (socket says 172.21.15.169:47220) 2023-10-26T01:34:59.882 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.877+0000 7f7ad90e9640 1 -- 172.21.15.169:0/1101572894 learned_addr learned my addr 172.21.15.169:0/1101572894 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:34:59.882 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.878+0000 7f7ad3fff640 1 --2- 172.21.15.169:0/1101572894 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7ad40b0720 0x7f7ad41f51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:34:59.882 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.878+0000 7f7ad88e8640 1 --2- 172.21.15.169:0/1101572894 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ad4065490 0x7f7ad40b12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:34:59.882 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.878+0000 7f7ad90e9640 1 -- 172.21.15.169:0/1101572894 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7ad40b0720 msgr2=0x7f7ad41f51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:34:59.883 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.878+0000 7f7ad90e9640 1 --2- 172.21.15.169:0/1101572894 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7ad40b0720 0x7f7ad41f51c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:34:59.883 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.878+0000 7f7ad90e9640 1 -- 172.21.15.169:0/1101572894 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ad4065490 msgr2=0x7f7ad40b12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:34:59.883 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.878+0000 7f7ad90e9640 1 --2- 172.21.15.169:0/1101572894 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ad4065490 0x7f7ad40b12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:34:59.883 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.878+0000 7f7ad90e9640 1 -- 172.21.15.169:0/1101572894 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ac0009560 con 0x7f7ad415ece0 2023-10-26T01:34:59.883 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.878+0000 7f7ad88e8640 1 --2- 172.21.15.169:0/1101572894 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ad4065490 0x7f7ad40b12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:34:59.884 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.878+0000 7f7ad90e9640 1 --2- 172.21.15.169:0/1101572894 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ad415ece0 0x7f7ad41f5700 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f7acc00c710 tx=0x7f7acc00cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:34:59.884 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.878+0000 7f7ad17fa640 1 -- 172.21.15.169:0/1101572894 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7acc006d50 con 0x7f7ad415ece0 2023-10-26T01:34:59.884 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.878+0000 7f7ad17fa640 1 -- 172.21.15.169:0/1101572894 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7acc00ce20 con 0x7f7ad415ece0 2023-10-26T01:34:59.884 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.878+0000 7f7ad98ea640 1 -- 172.21.15.169:0/1101572894 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ad40c9580 con 0x7f7ad415ece0 2023-10-26T01:34:59.884 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.879+0000 7f7ad17fa640 1 -- 172.21.15.169:0/1101572894 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7acc003bd0 con 0x7f7ad415ece0 2023-10-26T01:34:59.885 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.879+0000 7f7ad98ea640 1 -- 172.21.15.169:0/1101572894 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ad40c9b40 con 0x7f7ad415ece0 2023-10-26T01:34:59.885 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.880+0000 7f7ad98ea640 1 -- 172.21.15.169:0/1101572894 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ad40ca060 con 0x7f7ad415ece0 2023-10-26T01:34:59.886 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.881+0000 7f7ad17fa640 1 -- 172.21.15.169:0/1101572894 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f7acc003d30 con 0x7f7ad415ece0 2023-10-26T01:34:59.886 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.882+0000 7f7ad17fa640 1 --2- 172.21.15.169:0/1101572894 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7ab806ae40 0x7f7ab806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:34:59.886 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.882+0000 7f7ad17fa640 1 -- 172.21.15.169:0/1101572894 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7acc089510 con 0x7f7ad415ece0 2023-10-26T01:34:59.886 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.882+0000 7f7ad17fa640 1 -- 172.21.15.169:0/1101572894 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=70}) v3 -- 0x7f7ab8072970 con 0x7f7ad415ece0 2023-10-26T01:34:59.886 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.882+0000 7f7ad88e8640 1 --2- 172.21.15.169:0/1101572894 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7ab806ae40 0x7f7ab806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:34:59.887 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.883+0000 7f7ad88e8640 1 --2- 172.21.15.169:0/1101572894 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7ab806ae40 0x7f7ab806d300 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f7ac8009ab0 tx=0x7f7ac8005f90 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:34:59.887 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:34:59.886+0000 7f7ad17fa640 1 -- 172.21.15.169:0/1101572894 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7ad40ca060 con 0x7f7ad415ece0 2023-10-26T01:35:00.090 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:00.088+0000 7f7ad98ea640 1 -- 172.21.15.169:0/1101572894 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f7ad40ca060 con 0x7f7ad415ece0 2023-10-26T01:35:00.091 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:00.089+0000 7f7ad17fa640 1 -- 172.21.15.169:0/1101572894 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v69) v1 ==== 80+0+0 (secure 0 0 0) 0x7f7ad40ca060 con 0x7f7ad415ece0 2023-10-26T01:35:00.091 INFO:teuthology.orchestra.run.smithi169.stderr:listed 0 entries 2023-10-26T01:35:00.094 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:00.092+0000 7f7ad98ea640 1 -- 172.21.15.169:0/1101572894 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7ab806ae40 msgr2=0x7f7ab806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:00.094 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:00.092+0000 7f7ad98ea640 1 --2- 172.21.15.169:0/1101572894 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7ab806ae40 0x7f7ab806d300 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f7ac8009ab0 tx=0x7f7ac8005f90 comp rx=0 tx=0).stop 2023-10-26T01:35:00.094 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:00.092+0000 7f7ad98ea640 1 -- 172.21.15.169:0/1101572894 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ad415ece0 msgr2=0x7f7ad41f5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:00.094 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:00.092+0000 7f7ad98ea640 1 --2- 172.21.15.169:0/1101572894 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ad415ece0 0x7f7ad41f5700 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f7acc00c710 tx=0x7f7acc00cbd0 comp rx=0 tx=0).stop 2023-10-26T01:35:00.094 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:00.092+0000 7f7ad98ea640 1 -- 172.21.15.169:0/1101572894 shutdown_connections 2023-10-26T01:35:00.095 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:00.092+0000 7f7ad98ea640 1 --2- 172.21.15.169:0/1101572894 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7ab806ae40 0x7f7ab806d300 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:00.095 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:00.092+0000 7f7ad98ea640 1 --2- 172.21.15.169:0/1101572894 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ad415ece0 0x7f7ad41f5700 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:00.095 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:00.092+0000 7f7ad98ea640 1 --2- 172.21.15.169:0/1101572894 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7ad40b0720 0x7f7ad41f51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:00.095 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:00.092+0000 7f7ad98ea640 1 --2- 172.21.15.169:0/1101572894 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ad4065490 0x7f7ad40b12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:00.095 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:00.092+0000 7f7ad98ea640 1 -- 172.21.15.169:0/1101572894 >> 172.21.15.169:0/1101572894 conn(0x7f7ad40d1370 msgr2=0x7f7ad4152550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:35:00.096 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:00.092+0000 7f7ad98ea640 1 -- 172.21.15.169:0/1101572894 shutdown_connections 2023-10-26T01:35:00.096 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:00.092+0000 7f7ad98ea640 1 -- 172.21.15.169:0/1101572894 wait complete. 2023-10-26T01:35:00.113 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-26T01:35:00.113 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:35:00.114 DEBUG:teuthology.orchestra.run.smithi169:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-26T01:35:00.219 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi169.front.sepia.ceph.com 2023-10-26T01:35:00.220 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:35:00.220 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-26T01:35:00.272 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-26T01:35:00.272 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:35:00.330 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-26T01:35:00.331 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:35:00.385 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:35:00.675 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:00.673+0000 7f1766a4b640 1 -- 172.21.15.169:0/3862533410 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1760070ce0 msgr2=0x7f176015da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:00.676 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:00.673+0000 7f1766a4b640 1 --2- 172.21.15.169:0/3862533410 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1760070ce0 0x7f176015da30 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f17600ab350 tx=0x7f175002bd90 comp rx=0 tx=0).stop 2023-10-26T01:35:00.676 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:00.673+0000 7f1766a4b640 1 -- 172.21.15.169:0/3862533410 shutdown_connections 2023-10-26T01:35:00.676 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:00.673+0000 7f1766a4b640 1 --2- 172.21.15.169:0/3862533410 >> v2:172.21.15.170:3300/0 conn(0x7f176013f310 0x7f17600cdab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:00.677 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:00.673+0000 7f1766a4b640 1 --2- 172.21.15.169:0/3862533410 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f17600710c0 0x7f176013edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:00.677 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:00.673+0000 7f1766a4b640 1 --2- 172.21.15.169:0/3862533410 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1760070ce0 0x7f176015da30 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:00.677 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:00.673+0000 7f1766a4b640 1 -- 172.21.15.169:0/3862533410 >> 172.21.15.169:0/3862533410 conn(0x7f1760069510 msgr2=0x7f17600698f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:35:00.677 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:00.673+0000 7f1766a4b640 1 -- 172.21.15.169:0/3862533410 shutdown_connections 2023-10-26T01:35:00.678 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:00.673+0000 7f1766a4b640 1 -- 172.21.15.169:0/3862533410 wait complete. 2023-10-26T01:35:00.680 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:00.675+0000 7f1766a4b640 1 Processor -- start 2023-10-26T01:35:00.680 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:00.675+0000 7f1766a4b640 1 -- start start 2023-10-26T01:35:00.681 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:00.676+0000 7f1766a4b640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1760070ce0 0x7f17601f17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:00.681 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:00.676+0000 7f1766a4b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17600710c0 0x7f17600aeed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:00.681 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:00.676+0000 7f1766a4b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f176013f310 0x7f176006bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:00.681 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:00.676+0000 7f1766a4b640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f176006b650 con 0x7f17600710c0 2023-10-26T01:35:00.681 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:00.676+0000 7f1766a4b640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f176006b370 con 0x7f176013f310 2023-10-26T01:35:00.682 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:00.676+0000 7f1766a4b640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f176013b240 con 0x7f1760070ce0 2023-10-26T01:35:00.682 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:00.676+0000 7f175f7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17600710c0 0x7f17600aeed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:00.682 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:00.676+0000 7f175f7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17600710c0 0x7f17600aeed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:47244/0 (socket says 172.21.15.169:47244) 2023-10-26T01:35:00.682 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:00.677+0000 7f175f7fe640 1 -- 172.21.15.169:0/3141539060 learned_addr learned my addr 172.21.15.169:0/3141539060 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:35:00.683 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:00.677+0000 7f175ffff640 1 --2- 172.21.15.169:0/3141539060 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1760070ce0 0x7f17601f17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:00.683 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:00.677+0000 7f1764fe8640 1 --2- 172.21.15.169:0/3141539060 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f176013f310 0x7f176006bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:00.683 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:00.677+0000 7f175f7fe640 1 -- 172.21.15.169:0/3141539060 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1760070ce0 msgr2=0x7f17601f17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:00.683 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:00.677+0000 7f175f7fe640 1 --2- 172.21.15.169:0/3141539060 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1760070ce0 0x7f17601f17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:00.683 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:00.677+0000 7f175f7fe640 1 -- 172.21.15.169:0/3141539060 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f176013f310 msgr2=0x7f176006bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:00.684 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:00.677+0000 7f175f7fe640 1 --2- 172.21.15.169:0/3141539060 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f176013f310 0x7f176006bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:00.684 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:00.677+0000 7f175f7fe640 1 -- 172.21.15.169:0/3141539060 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1750009560 con 0x7f17600710c0 2023-10-26T01:35:00.684 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:00.677+0000 7f1764fe8640 1 --2- 172.21.15.169:0/3141539060 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f176013f310 0x7f176006bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:35:00.684 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:00.677+0000 7f175f7fe640 1 --2- 172.21.15.169:0/3141539060 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17600710c0 0x7f17600aeed0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f175400b4a0 tx=0x7f175400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:35:00.684 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:00.677+0000 7f175cff9640 1 -- 172.21.15.169:0/3141539060 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1754010610 con 0x7f17600710c0 2023-10-26T01:35:00.685 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:00.678+0000 7f175cff9640 1 -- 172.21.15.169:0/3141539060 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1754010860 con 0x7f17600710c0 2023-10-26T01:35:00.685 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:00.678+0000 7f1766a4b640 1 -- 172.21.15.169:0/3141539060 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f176013b520 con 0x7f17600710c0 2023-10-26T01:35:00.685 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:00.678+0000 7f175cff9640 1 -- 172.21.15.169:0/3141539060 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1754010c70 con 0x7f17600710c0 2023-10-26T01:35:00.685 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:00.678+0000 7f1766a4b640 1 -- 172.21.15.169:0/3141539060 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17601f1ce0 con 0x7f17600710c0 2023-10-26T01:35:00.686 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:00.679+0000 7f1766a4b640 1 -- 172.21.15.169:0/3141539060 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f176006b800 con 0x7f17600710c0 2023-10-26T01:35:00.687 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:00.680+0000 7f175cff9640 1 -- 172.21.15.169:0/3141539060 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f1754017360 con 0x7f17600710c0 2023-10-26T01:35:00.687 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:00.681+0000 7f175cff9640 1 --2- 172.21.15.169:0/3141539060 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f174c06ae70 0x7f174c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:00.687 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:00.681+0000 7f175cff9640 1 -- 172.21.15.169:0/3141539060 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1754089590 con 0x7f17600710c0 2023-10-26T01:35:00.687 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:00.681+0000 7f175cff9640 1 -- 172.21.15.169:0/3141539060 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=70}) v3 -- 0x7f174c0729a0 con 0x7f17600710c0 2023-10-26T01:35:00.687 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:00.681+0000 7f175ffff640 1 --2- 172.21.15.169:0/3141539060 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f174c06ae70 0x7f174c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:00.688 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:00.682+0000 7f175ffff640 1 --2- 172.21.15.169:0/3141539060 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f174c06ae70 0x7f174c06d330 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f1750035000 tx=0x7f1750004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:35:00.688 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:00.685+0000 7f175cff9640 1 -- 172.21.15.169:0/3141539060 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f17540545a0 con 0x7f17600710c0 2023-10-26T01:35:00.875 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:00.873+0000 7f1766a4b640 1 -- 172.21.15.169:0/3141539060 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f176004db10 con 0x7f17600710c0 2023-10-26T01:35:00.878 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:00.876+0000 7f175cff9640 1 -- 172.21.15.169:0/3141539060 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) v1 ==== 74+0+16671 (secure 0 0 0) 0x7f1754014090 con 0x7f17600710c0 2023-10-26T01:35:00.878 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:35:00.879 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":69,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:34:04.037343+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-10-26T01:33:38.593237+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-10-26T01:33:39.753692+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","full","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","full","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","full","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["backfillfull","exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","full","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:35:00.881 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:00.879+0000 7f1766a4b640 1 -- 172.21.15.169:0/3141539060 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f174c06ae70 msgr2=0x7f174c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:00.881 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:00.879+0000 7f1766a4b640 1 --2- 172.21.15.169:0/3141539060 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f174c06ae70 0x7f174c06d330 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f1750035000 tx=0x7f1750004730 comp rx=0 tx=0).stop 2023-10-26T01:35:00.881 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:00.879+0000 7f1766a4b640 1 -- 172.21.15.169:0/3141539060 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17600710c0 msgr2=0x7f17600aeed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:00.881 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:00.879+0000 7f1766a4b640 1 --2- 172.21.15.169:0/3141539060 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17600710c0 0x7f17600aeed0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f175400b4a0 tx=0x7f175400b960 comp rx=0 tx=0).stop 2023-10-26T01:35:00.881 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:00.879+0000 7f1766a4b640 1 -- 172.21.15.169:0/3141539060 shutdown_connections 2023-10-26T01:35:00.882 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:00.879+0000 7f1766a4b640 1 --2- 172.21.15.169:0/3141539060 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f174c06ae70 0x7f174c06d330 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:00.882 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:00.879+0000 7f1766a4b640 1 --2- 172.21.15.169:0/3141539060 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17600710c0 0x7f17600aeed0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:00.882 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:00.879+0000 7f1766a4b640 1 --2- 172.21.15.169:0/3141539060 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f176013f310 0x7f176006bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:00.882 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:00.879+0000 7f1766a4b640 1 --2- 172.21.15.169:0/3141539060 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1760070ce0 0x7f17601f17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:00.883 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:00.879+0000 7f1766a4b640 1 -- 172.21.15.169:0/3141539060 >> 172.21.15.169:0/3141539060 conn(0x7f1760069510 msgr2=0x7f17600698f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:35:00.883 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:00.879+0000 7f1766a4b640 1 -- 172.21.15.169:0/3141539060 shutdown_connections 2023-10-26T01:35:00.883 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:00.879+0000 7f1766a4b640 1 -- 172.21.15.169:0/3141539060 wait complete. 2023-10-26T01:35:00.898 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T01:30:32.200808+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-26T01:33:38.593237+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2023-10-26T01:33:39.753692+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T01:35:00.899 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T01:35:01.120 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:01.118+0000 7f03fa3a4640 1 -- 172.21.15.169:0/2467001156 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03e0004d00 msgr2=0x7f03e0005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:01.120 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:01.118+0000 7f03fa3a4640 1 --2- 172.21.15.169:0/2467001156 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03e0004d00 0x7f03e0005180 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f03ec005fd0 tx=0x7f03e00057f0 comp rx=0 tx=0).stop 2023-10-26T01:35:01.120 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:01.118+0000 7f03f8941640 1 -- 172.21.15.169:0/2467001156 reap_dead start 2023-10-26T01:35:01.121 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:01.118+0000 7f03fa3a4640 1 -- 172.21.15.169:0/2467001156 shutdown_connections 2023-10-26T01:35:01.121 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:01.118+0000 7f03fa3a4640 1 -- 172.21.15.169:0/2467001156 >> 172.21.15.169:0/2467001156 conn(0x7f03f4069930 msgr2=0x7f03f4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:35:01.121 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:01.118+0000 7f03fa3a4640 1 -- 172.21.15.169:0/2467001156 shutdown_connections 2023-10-26T01:35:01.121 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:01.118+0000 7f03fa3a4640 1 -- 172.21.15.169:0/2467001156 wait complete. 2023-10-26T01:35:01.123 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:01.120+0000 7f03fa3a4640 1 Processor -- start 2023-10-26T01:35:01.123 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:01.120+0000 7f03fa3a4640 1 -- start start 2023-10-26T01:35:01.123 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:01.121+0000 7f03fa3a4640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f03e0004d00 0x7f03f4154ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:01.123 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:01.121+0000 7f03fa3a4640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f03f4070d60 0x7f03f4143fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:01.123 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:01.121+0000 7f03fa3a4640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03f41485d0 0x7f03f4148a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:01.124 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:01.121+0000 7f03fa3a4640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03f4148f90 con 0x7f03f41485d0 2023-10-26T01:35:01.124 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:01.121+0000 7f03fa3a4640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03f40711e0 con 0x7f03f4070d60 2023-10-26T01:35:01.125 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:01.121+0000 7f03fa3a4640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03f41444e0 con 0x7f03e0004d00 2023-10-26T01:35:01.125 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:01.121+0000 7f03f8941640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03f41485d0 0x7f03f4148a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:01.125 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:01.121+0000 7f03f8941640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03f41485d0 0x7f03f4148a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:47260/0 (socket says 172.21.15.169:47260) 2023-10-26T01:35:01.126 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:01.121+0000 7f03f8941640 1 -- 172.21.15.169:0/2245542428 learned_addr learned my addr 172.21.15.169:0/2245542428 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:35:01.127 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:01.121+0000 7f03f3fff640 1 --2- 172.21.15.169:0/2245542428 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f03e0004d00 0x7f03f4154ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:01.128 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:01.121+0000 7f03ebfff640 1 --2- 172.21.15.169:0/2245542428 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f03f4070d60 0x7f03f4143fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:01.128 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:01.121+0000 7f03f8941640 1 -- 172.21.15.169:0/2245542428 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f03e0004d00 msgr2=0x7f03f4154ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:01.128 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:01.121+0000 7f03f8941640 1 --2- 172.21.15.169:0/2245542428 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f03e0004d00 0x7f03f4154ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:01.128 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:01.121+0000 7f03f8941640 1 -- 172.21.15.169:0/2245542428 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f03f4070d60 msgr2=0x7f03f4143fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:01.128 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:01.122+0000 7f03f8941640 1 --2- 172.21.15.169:0/2245542428 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f03f4070d60 0x7f03f4143fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:01.129 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:01.122+0000 7f03f8941640 1 -- 172.21.15.169:0/2245542428 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03f41449e0 con 0x7f03f41485d0 2023-10-26T01:35:01.129 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:01.122+0000 7f03f8941640 1 --2- 172.21.15.169:0/2245542428 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03f41485d0 0x7f03f4148a50 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f03ec009f90 tx=0x7f03ec031730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:35:01.129 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:01.122+0000 7f03f17fa640 1 -- 172.21.15.169:0/2245542428 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03ec03f070 con 0x7f03f41485d0 2023-10-26T01:35:01.129 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:01.122+0000 7f03f17fa640 1 -- 172.21.15.169:0/2245542428 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f03ec003970 con 0x7f03f41485d0 2023-10-26T01:35:01.129 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:01.122+0000 7f03f17fa640 1 -- 172.21.15.169:0/2245542428 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03ec003d60 con 0x7f03f41485d0 2023-10-26T01:35:01.130 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:01.122+0000 7f03fa3a4640 1 -- 172.21.15.169:0/2245542428 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03f4136750 con 0x7f03f41485d0 2023-10-26T01:35:01.130 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:01.122+0000 7f03fa3a4640 1 -- 172.21.15.169:0/2245542428 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03f4071700 con 0x7f03f41485d0 2023-10-26T01:35:01.130 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:01.123+0000 7f03fa3a4640 1 -- 172.21.15.169:0/2245542428 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03f4066af0 con 0x7f03f41485d0 2023-10-26T01:35:01.130 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:01.124+0000 7f03f17fa640 1 -- 172.21.15.169:0/2245542428 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f03ec02f070 con 0x7f03f41485d0 2023-10-26T01:35:01.130 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:01.125+0000 7f03f17fa640 1 --2- 172.21.15.169:0/2245542428 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f03d406ae70 0x7f03d406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:01.131 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:01.125+0000 7f03f17fa640 1 -- 172.21.15.169:0/2245542428 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f03ec0abb30 con 0x7f03f41485d0 2023-10-26T01:35:01.131 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:01.125+0000 7f03f17fa640 1 -- 172.21.15.169:0/2245542428 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=70}) v3 -- 0x7f03d40729a0 con 0x7f03f41485d0 2023-10-26T01:35:01.131 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:01.126+0000 7f03f3fff640 1 --2- 172.21.15.169:0/2245542428 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f03d406ae70 0x7f03d406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:01.132 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:01.127+0000 7f03f3fff640 1 --2- 172.21.15.169:0/2245542428 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f03d406ae70 0x7f03d406d330 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f03e4001660 tx=0x7f03e40069a0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:35:01.132 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:01.130+0000 7f03f17fa640 1 -- 172.21.15.169:0/2245542428 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f03ec035030 con 0x7f03f41485d0 2023-10-26T01:35:01.327 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:01.325+0000 7f03fa3a4640 1 -- 172.21.15.169:0/2245542428 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f03f404da70 con 0x7f03f41485d0 2023-10-26T01:35:01.327 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:01.326+0000 7f03f17fa640 1 -- 172.21.15.169:0/2245542428 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v69) v1 ==== 93+0+10 (secure 0 0 0) 0x7f03ec035030 con 0x7f03f41485d0 2023-10-26T01:35:01.328 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 1 2023-10-26T01:35:01.330 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:01.328+0000 7f03fa3a4640 1 -- 172.21.15.169:0/2245542428 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f03d406ae70 msgr2=0x7f03d406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:01.331 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:01.328+0000 7f03fa3a4640 1 --2- 172.21.15.169:0/2245542428 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f03d406ae70 0x7f03d406d330 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f03e4001660 tx=0x7f03e40069a0 comp rx=0 tx=0).stop 2023-10-26T01:35:01.331 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:01.328+0000 7f03fa3a4640 1 -- 172.21.15.169:0/2245542428 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03f41485d0 msgr2=0x7f03f4148a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:01.331 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:01.328+0000 7f03fa3a4640 1 --2- 172.21.15.169:0/2245542428 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03f41485d0 0x7f03f4148a50 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f03ec009f90 tx=0x7f03ec031730 comp rx=0 tx=0).stop 2023-10-26T01:35:01.331 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:01.329+0000 7f03fa3a4640 1 -- 172.21.15.169:0/2245542428 shutdown_connections 2023-10-26T01:35:01.332 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:01.329+0000 7f03fa3a4640 1 --2- 172.21.15.169:0/2245542428 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f03d406ae70 0x7f03d406d330 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:01.332 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:01.329+0000 7f03fa3a4640 1 --2- 172.21.15.169:0/2245542428 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03f41485d0 0x7f03f4148a50 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:01.332 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:01.329+0000 7f03fa3a4640 1 --2- 172.21.15.169:0/2245542428 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f03f4070d60 0x7f03f4143fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:01.332 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:01.329+0000 7f03fa3a4640 1 --2- 172.21.15.169:0/2245542428 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f03e0004d00 0x7f03f4154ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:01.332 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:01.329+0000 7f03fa3a4640 1 -- 172.21.15.169:0/2245542428 >> 172.21.15.169:0/2245542428 conn(0x7f03f4069930 msgr2=0x7f03f4156b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:35:01.333 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:01.329+0000 7f03fa3a4640 1 -- 172.21.15.169:0/2245542428 shutdown_connections 2023-10-26T01:35:01.333 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:01.329+0000 7f03fa3a4640 1 -- 172.21.15.169:0/2245542428 wait complete. 2023-10-26T01:35:01.350 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-26T01:35:01.567 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:01.565+0000 7f1ff9e26640 1 -- 172.21.15.169:0/2336688843 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1fe0003970 msgr2=0x7f1fe0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:01.568 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:01.565+0000 7f1ff9e26640 1 --2- 172.21.15.169:0/2336688843 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1fe0003970 0x7f1fe0003d90 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f1fe4006f10 tx=0x7f1fe00057f0 comp rx=0 tx=0).stop 2023-10-26T01:35:01.568 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:01.565+0000 7f1ff3fff640 1 -- 172.21.15.169:0/2336688843 reap_dead start 2023-10-26T01:35:01.568 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:01.565+0000 7f1ff9e26640 1 -- 172.21.15.169:0/2336688843 shutdown_connections 2023-10-26T01:35:01.568 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:01.565+0000 7f1ff9e26640 1 -- 172.21.15.169:0/2336688843 >> 172.21.15.169:0/2336688843 conn(0x7f1ff4069930 msgr2=0x7f1ff4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:35:01.569 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:01.566+0000 7f1ff9e26640 1 -- 172.21.15.169:0/2336688843 shutdown_connections 2023-10-26T01:35:01.569 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:01.566+0000 7f1ff9e26640 1 -- 172.21.15.169:0/2336688843 wait complete. 2023-10-26T01:35:01.569 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:01.567+0000 7f1ff9e26640 1 Processor -- start 2023-10-26T01:35:01.570 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:01.568+0000 7f1ff9e26640 1 -- start start 2023-10-26T01:35:01.571 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:01.568+0000 7f1ff9e26640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1fe0003970 0x7f1ff4156c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:01.572 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:01.569+0000 7f1ff9e26640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ff41f2fd0 0x7f1ff40ca430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:01.572 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:01.569+0000 7f1ff9e26640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1ff40ca970 0x7f1ff4159f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:01.572 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:01.569+0000 7f1ff9e26640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff4157150 con 0x7f1ff41f2fd0 2023-10-26T01:35:01.572 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:01.569+0000 7f1ff9e26640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff40cadf0 con 0x7f1fe0003970 2023-10-26T01:35:01.573 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:01.569+0000 7f1ff9e26640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff41591d0 con 0x7f1ff40ca970 2023-10-26T01:35:01.573 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:01.569+0000 7f1febfff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ff41f2fd0 0x7f1ff40ca430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:01.573 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:01.569+0000 7f1febfff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ff41f2fd0 0x7f1ff40ca430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:47284/0 (socket says 172.21.15.169:47284) 2023-10-26T01:35:01.573 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:01.569+0000 7f1febfff640 1 -- 172.21.15.169:0/3245735861 learned_addr learned my addr 172.21.15.169:0/3245735861 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:35:01.573 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:01.569+0000 7f1ff37fe640 1 --2- 172.21.15.169:0/3245735861 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1fe0003970 0x7f1ff4156c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:01.574 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:01.569+0000 7f1ff3fff640 1 --2- 172.21.15.169:0/3245735861 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1ff40ca970 0x7f1ff4159f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:01.574 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:01.569+0000 7f1febfff640 1 -- 172.21.15.169:0/3245735861 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1ff40ca970 msgr2=0x7f1ff4159f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:01.574 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:01.569+0000 7f1febfff640 1 --2- 172.21.15.169:0/3245735861 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1ff40ca970 0x7f1ff4159f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:01.574 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:01.569+0000 7f1febfff640 1 -- 172.21.15.169:0/3245735861 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1fe0003970 msgr2=0x7f1ff4156c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:01.574 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:01.569+0000 7f1febfff640 1 --2- 172.21.15.169:0/3245735861 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1fe0003970 0x7f1ff4156c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:01.575 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:01.569+0000 7f1febfff640 1 -- 172.21.15.169:0/3245735861 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ff41f2cb0 con 0x7f1ff41f2fd0 2023-10-26T01:35:01.575 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:01.570+0000 7f1febfff640 1 --2- 172.21.15.169:0/3245735861 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ff41f2fd0 0x7f1ff40ca430 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc010a10 tx=0x7f1fdc010ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:35:01.575 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:01.570+0000 7f1ff0ff9640 1 -- 172.21.15.169:0/3245735861 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fdc00ece0 con 0x7f1ff41f2fd0 2023-10-26T01:35:01.575 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:01.570+0000 7f1ff0ff9640 1 -- 172.21.15.169:0/3245735861 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1fdc00ee40 con 0x7f1ff41f2fd0 2023-10-26T01:35:01.575 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:01.570+0000 7f1ff0ff9640 1 -- 172.21.15.169:0/3245735861 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fdc005280 con 0x7f1ff41f2fd0 2023-10-26T01:35:01.575 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:01.570+0000 7f1ff9e26640 1 -- 172.21.15.169:0/3245735861 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ff4159490 con 0x7f1ff41f2fd0 2023-10-26T01:35:01.576 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:01.570+0000 7f1ff9e26640 1 -- 172.21.15.169:0/3245735861 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ff4159a50 con 0x7f1ff41f2fd0 2023-10-26T01:35:01.577 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:01.571+0000 7f1ff9e26640 1 -- 172.21.15.169:0/3245735861 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ff404db10 con 0x7f1ff41f2fd0 2023-10-26T01:35:01.577 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:01.572+0000 7f1ff0ff9640 1 -- 172.21.15.169:0/3245735861 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f1fdc0053e0 con 0x7f1ff41f2fd0 2023-10-26T01:35:01.577 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:01.573+0000 7f1ff0ff9640 1 --2- 172.21.15.169:0/3245735861 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f1fd406ae40 0x7f1fd406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:01.577 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:01.574+0000 7f1ff0ff9640 1 -- 172.21.15.169:0/3245735861 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1fdc016070 con 0x7f1ff41f2fd0 2023-10-26T01:35:01.578 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:01.574+0000 7f1ff0ff9640 1 -- 172.21.15.169:0/3245735861 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=70}) v3 -- 0x7f1fd4072970 con 0x7f1ff41f2fd0 2023-10-26T01:35:01.578 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:01.574+0000 7f1ff37fe640 1 --2- 172.21.15.169:0/3245735861 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f1fd406ae40 0x7f1fd406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:01.578 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:01.575+0000 7f1ff37fe640 1 --2- 172.21.15.169:0/3245735861 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f1fd406ae40 0x7f1fd406d300 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f1fe4034000 tx=0x7f1fe402e040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:35:01.579 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:01.578+0000 7f1ff0ff9640 1 -- 172.21.15.169:0/3245735861 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1fdc0554c0 con 0x7f1ff41f2fd0 2023-10-26T01:35:01.768 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:01.767+0000 7f1ff9e26640 1 -- 172.21.15.169:0/3245735861 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1ff40cc910 con 0x7f1ff41f2fd0 2023-10-26T01:35:01.769 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:01.768+0000 7f1ff0ff9640 1 -- 172.21.15.169:0/3245735861 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v69) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1fdc058ae0 con 0x7f1ff41f2fd0 2023-10-26T01:35:01.769 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 32 2023-10-26T01:35:01.772 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:01.770+0000 7f1ff9e26640 1 -- 172.21.15.169:0/3245735861 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f1fd406ae40 msgr2=0x7f1fd406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:01.772 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:01.770+0000 7f1ff9e26640 1 --2- 172.21.15.169:0/3245735861 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f1fd406ae40 0x7f1fd406d300 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f1fe4034000 tx=0x7f1fe402e040 comp rx=0 tx=0).stop 2023-10-26T01:35:01.773 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:01.770+0000 7f1ff9e26640 1 -- 172.21.15.169:0/3245735861 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ff41f2fd0 msgr2=0x7f1ff40ca430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:01.773 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:01.770+0000 7f1ff9e26640 1 --2- 172.21.15.169:0/3245735861 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ff41f2fd0 0x7f1ff40ca430 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc010a10 tx=0x7f1fdc010ed0 comp rx=0 tx=0).stop 2023-10-26T01:35:01.773 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:01.771+0000 7f1ff9e26640 1 -- 172.21.15.169:0/3245735861 shutdown_connections 2023-10-26T01:35:01.773 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:01.771+0000 7f1ff9e26640 1 --2- 172.21.15.169:0/3245735861 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f1fd406ae40 0x7f1fd406d300 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:01.773 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:01.771+0000 7f1ff9e26640 1 --2- 172.21.15.169:0/3245735861 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ff41f2fd0 0x7f1ff40ca430 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:01.774 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:01.771+0000 7f1ff9e26640 1 --2- 172.21.15.169:0/3245735861 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1ff40ca970 0x7f1ff4159f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:01.774 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:01.771+0000 7f1ff9e26640 1 --2- 172.21.15.169:0/3245735861 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1fe0003970 0x7f1ff4156c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:01.774 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:01.771+0000 7f1ff9e26640 1 -- 172.21.15.169:0/3245735861 >> 172.21.15.169:0/3245735861 conn(0x7f1ff4069930 msgr2=0x7f1ff406ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:35:01.774 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:01.771+0000 7f1ff9e26640 1 -- 172.21.15.169:0/3245735861 shutdown_connections 2023-10-26T01:35:01.775 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:01.771+0000 7f1ff9e26640 1 -- 172.21.15.169:0/3245735861 wait complete. 2023-10-26T01:35:01.791 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-26T01:35:02.009 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.007+0000 7f352de7c640 1 -- 172.21.15.169:0/2119859217 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3514005000 msgr2=0x7f35140058d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:02.009 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.007+0000 7f352de7c640 1 --2- 172.21.15.169:0/2119859217 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3514005000 0x7f35140058d0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f3510009b80 tx=0x7f3514006900 comp rx=0 tx=0).stop 2023-10-26T01:35:02.010 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.007+0000 7f3527fff640 1 -- 172.21.15.169:0/2119859217 reap_dead start 2023-10-26T01:35:02.010 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.007+0000 7f352de7c640 1 -- 172.21.15.169:0/2119859217 shutdown_connections 2023-10-26T01:35:02.010 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.007+0000 7f352de7c640 1 -- 172.21.15.169:0/2119859217 >> 172.21.15.169:0/2119859217 conn(0x7f3528069930 msgr2=0x7f3528069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:35:02.010 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.007+0000 7f352de7c640 1 -- 172.21.15.169:0/2119859217 shutdown_connections 2023-10-26T01:35:02.011 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.007+0000 7f352de7c640 1 -- 172.21.15.169:0/2119859217 wait complete. 2023-10-26T01:35:02.011 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.009+0000 7f352de7c640 1 Processor -- start 2023-10-26T01:35:02.011 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.009+0000 7f352de7c640 1 -- start start 2023-10-26T01:35:02.013 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.010+0000 7f352de7c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3514005000 0x7f35280aef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:02.014 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.010+0000 7f352de7c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3528071170 0x7f35281e45a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:02.014 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.010+0000 7f352de7c640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f35280715f0 0x7f35281e4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:02.014 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.010+0000 7f352de7c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35280addb0 con 0x7f3528071170 2023-10-26T01:35:02.014 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.010+0000 7f352de7c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35281e5020 con 0x7f3514005000 2023-10-26T01:35:02.014 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.010+0000 7f352de7c640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35280aeb00 con 0x7f35280715f0 2023-10-26T01:35:02.015 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.011+0000 7f351ffff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3528071170 0x7f35281e45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:02.015 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.011+0000 7f351ffff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3528071170 0x7f35281e45a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:47300/0 (socket says 172.21.15.169:47300) 2023-10-26T01:35:02.015 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.011+0000 7f351ffff640 1 -- 172.21.15.169:0/864533736 learned_addr learned my addr 172.21.15.169:0/864533736 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:35:02.015 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.011+0000 7f35277fe640 1 --2- 172.21.15.169:0/864533736 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3514005000 0x7f35280aef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:02.015 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.011+0000 7f3527fff640 1 --2- 172.21.15.169:0/864533736 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f35280715f0 0x7f35281e4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:02.016 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.011+0000 7f351ffff640 1 -- 172.21.15.169:0/864533736 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f35280715f0 msgr2=0x7f35281e4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:02.016 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.011+0000 7f351ffff640 1 --2- 172.21.15.169:0/864533736 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f35280715f0 0x7f35281e4ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:02.016 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.011+0000 7f351ffff640 1 -- 172.21.15.169:0/864533736 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3514005000 msgr2=0x7f35280aef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:02.016 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.011+0000 7f351ffff640 1 --2- 172.21.15.169:0/864533736 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3514005000 0x7f35280aef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:02.016 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.011+0000 7f351ffff640 1 -- 172.21.15.169:0/864533736 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35280ad130 con 0x7f3528071170 2023-10-26T01:35:02.017 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.011+0000 7f35277fe640 1 --2- 172.21.15.169:0/864533736 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3514005000 0x7f35280aef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:35:02.017 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.011+0000 7f351ffff640 1 --2- 172.21.15.169:0/864533736 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3528071170 0x7f35281e45a0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f351002cba0 tx=0x7f3510033c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:35:02.017 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.012+0000 7f3524ff9640 1 -- 172.21.15.169:0/864533736 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f351003d8e0 con 0x7f3528071170 2023-10-26T01:35:02.017 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.012+0000 7f3524ff9640 1 -- 172.21.15.169:0/864533736 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f351003da40 con 0x7f3528071170 2023-10-26T01:35:02.017 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.012+0000 7f3524ff9640 1 -- 172.21.15.169:0/864533736 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f351003de10 con 0x7f3528071170 2023-10-26T01:35:02.018 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.012+0000 7f352de7c640 1 -- 172.21.15.169:0/864533736 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35280ad410 con 0x7f3528071170 2023-10-26T01:35:02.018 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.012+0000 7f352de7c640 1 -- 172.21.15.169:0/864533736 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35280ad9d0 con 0x7f3528071170 2023-10-26T01:35:02.019 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.013+0000 7f352de7c640 1 -- 172.21.15.169:0/864533736 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f352804db10 con 0x7f3528071170 2023-10-26T01:35:02.019 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.014+0000 7f3524ff9640 1 -- 172.21.15.169:0/864533736 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f3510045400 con 0x7f3528071170 2023-10-26T01:35:02.019 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.015+0000 7f3524ff9640 1 --2- 172.21.15.169:0/864533736 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f350806ae40 0x7f350806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:02.019 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.015+0000 7f3524ff9640 1 -- 172.21.15.169:0/864533736 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f35100ac3f0 con 0x7f3528071170 2023-10-26T01:35:02.020 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.015+0000 7f3524ff9640 1 -- 172.21.15.169:0/864533736 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=70}) v3 -- 0x7f3508072970 con 0x7f3528071170 2023-10-26T01:35:02.020 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.016+0000 7f35277fe640 1 --2- 172.21.15.169:0/864533736 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f350806ae40 0x7f350806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:02.020 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.017+0000 7f35277fe640 1 --2- 172.21.15.169:0/864533736 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f350806ae40 0x7f350806d300 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f35180099a0 tx=0x7f3518005f90 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:35:02.021 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.019+0000 7f3524ff9640 1 -- 172.21.15.169:0/864533736 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3510033e60 con 0x7f3528071170 2023-10-26T01:35:02.210 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.209+0000 7f352de7c640 1 -- 172.21.15.169:0/864533736 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3528069f60 con 0x7f3528071170 2023-10-26T01:35:02.211 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.210+0000 7f3524ff9640 1 -- 172.21.15.169:0/864533736 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v69) v1 ==== 100+0+11 (secure 0 0 0) 0x7f351004a070 con 0x7f3528071170 2023-10-26T01:35:02.211 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 64 2023-10-26T01:35:02.214 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.212+0000 7f352de7c640 1 -- 172.21.15.169:0/864533736 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f350806ae40 msgr2=0x7f350806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:02.214 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.212+0000 7f352de7c640 1 --2- 172.21.15.169:0/864533736 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f350806ae40 0x7f350806d300 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f35180099a0 tx=0x7f3518005f90 comp rx=0 tx=0).stop 2023-10-26T01:35:02.215 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.212+0000 7f352de7c640 1 -- 172.21.15.169:0/864533736 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3528071170 msgr2=0x7f35281e45a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:02.215 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.212+0000 7f352de7c640 1 --2- 172.21.15.169:0/864533736 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3528071170 0x7f35281e45a0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f351002cba0 tx=0x7f3510033c70 comp rx=0 tx=0).stop 2023-10-26T01:35:02.215 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.212+0000 7f352de7c640 1 -- 172.21.15.169:0/864533736 shutdown_connections 2023-10-26T01:35:02.215 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.212+0000 7f352de7c640 1 --2- 172.21.15.169:0/864533736 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f350806ae40 0x7f350806d300 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:02.215 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.212+0000 7f352de7c640 1 --2- 172.21.15.169:0/864533736 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3528071170 0x7f35281e45a0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:02.216 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.212+0000 7f352de7c640 1 --2- 172.21.15.169:0/864533736 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f35280715f0 0x7f35281e4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:02.216 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.212+0000 7f352de7c640 1 --2- 172.21.15.169:0/864533736 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3514005000 0x7f35280aef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:02.216 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.212+0000 7f352de7c640 1 -- 172.21.15.169:0/864533736 >> 172.21.15.169:0/864533736 conn(0x7f3528069930 msgr2=0x7f3528155910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:35:02.216 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.213+0000 7f352de7c640 1 -- 172.21.15.169:0/864533736 shutdown_connections 2023-10-26T01:35:02.216 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.213+0000 7f352de7c640 1 -- 172.21.15.169:0/864533736 wait complete. 2023-10-26T01:35:02.233 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-26T01:35:02.453 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.451+0000 7f1573f1d640 1 -- 172.21.15.169:0/1357918415 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f156c0cdef0 msgr2=0x7f156c15b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:02.454 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.451+0000 7f1573f1d640 1 --2- 172.21.15.169:0/1357918415 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f156c0cdef0 0x7f156c15b4a0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f156c13e070 tx=0x7f155c02bdb0 comp rx=0 tx=0).stop 2023-10-26T01:35:02.454 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.451+0000 7f1573f1d640 1 -- 172.21.15.169:0/1357918415 shutdown_connections 2023-10-26T01:35:02.454 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.451+0000 7f1573f1d640 1 --2- 172.21.15.169:0/1357918415 >> v2:172.21.15.170:3300/0 conn(0x7f156c15b9e0 0x7f156c0b05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:02.454 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.451+0000 7f1573f1d640 1 --2- 172.21.15.169:0/1357918415 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f156c0cdef0 0x7f156c15b4a0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:02.455 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.451+0000 7f1573f1d640 1 -- 172.21.15.169:0/1357918415 >> 172.21.15.169:0/1357918415 conn(0x7f156c069930 msgr2=0x7f156c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:35:02.455 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.452+0000 7f1573f1d640 1 -- 172.21.15.169:0/1357918415 shutdown_connections 2023-10-26T01:35:02.455 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.452+0000 7f1573f1d640 1 -- 172.21.15.169:0/1357918415 wait complete. 2023-10-26T01:35:02.457 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.453+0000 7f1573f1d640 1 Processor -- start 2023-10-26T01:35:02.458 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.453+0000 7f1573f1d640 1 -- start start 2023-10-26T01:35:02.458 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.454+0000 7f1573f1d640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f156c0cdef0 0x7f156c14aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:02.458 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.454+0000 7f1573f1d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f156c155360 0x7f156c14ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:02.458 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.454+0000 7f1573f1d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f156c15b9e0 0x7f156c14f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:02.459 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.454+0000 7f1573f1d640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f156c14b0e0 con 0x7f156c155360 2023-10-26T01:35:02.459 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.454+0000 7f1573f1d640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f156c152740 con 0x7f156c15b9e0 2023-10-26T01:35:02.459 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.454+0000 7f1573f1d640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f156c0af6e0 con 0x7f156c0cdef0 2023-10-26T01:35:02.459 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.455+0000 7f15714b8640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f156c155360 0x7f156c14ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:02.459 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.455+0000 7f15714b8640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f156c155360 0x7f156c14ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:47322/0 (socket says 172.21.15.169:47322) 2023-10-26T01:35:02.459 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.455+0000 7f15714b8640 1 -- 172.21.15.169:0/283296498 learned_addr learned my addr 172.21.15.169:0/283296498 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:35:02.460 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.455+0000 7f15724ba640 1 --2- 172.21.15.169:0/283296498 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f156c15b9e0 0x7f156c14f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:02.460 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.455+0000 7f1571cb9640 1 --2- 172.21.15.169:0/283296498 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f156c0cdef0 0x7f156c14aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:02.460 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.455+0000 7f15714b8640 1 -- 172.21.15.169:0/283296498 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f156c0cdef0 msgr2=0x7f156c14aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:02.460 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.455+0000 7f15714b8640 1 --2- 172.21.15.169:0/283296498 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f156c0cdef0 0x7f156c14aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:02.460 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.455+0000 7f15714b8640 1 -- 172.21.15.169:0/283296498 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f156c15b9e0 msgr2=0x7f156c14f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:02.461 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.455+0000 7f15714b8640 1 --2- 172.21.15.169:0/283296498 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f156c15b9e0 0x7f156c14f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:02.461 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.455+0000 7f15714b8640 1 -- 172.21.15.169:0/283296498 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f155c009560 con 0x7f156c155360 2023-10-26T01:35:02.461 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.455+0000 7f15724ba640 1 --2- 172.21.15.169:0/283296498 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f156c15b9e0 0x7f156c14f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:35:02.461 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.455+0000 7f15714b8640 1 --2- 172.21.15.169:0/283296498 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f156c155360 0x7f156c14ecc0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f155c000c00 tx=0x7f155c02db50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:35:02.461 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.456+0000 7f15627fc640 1 -- 172.21.15.169:0/283296498 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f155c003800 con 0x7f156c155360 2023-10-26T01:35:02.461 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.456+0000 7f15627fc640 1 -- 172.21.15.169:0/283296498 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f155c003a50 con 0x7f156c155360 2023-10-26T01:35:02.462 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.456+0000 7f15627fc640 1 -- 172.21.15.169:0/283296498 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f155c003e60 con 0x7f156c155360 2023-10-26T01:35:02.462 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.456+0000 7f1573f1d640 1 -- 172.21.15.169:0/283296498 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f156c14c670 con 0x7f156c155360 2023-10-26T01:35:02.462 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.456+0000 7f1573f1d640 1 -- 172.21.15.169:0/283296498 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f156c0af980 con 0x7f156c155360 2023-10-26T01:35:02.463 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.457+0000 7f1573f1d640 1 -- 172.21.15.169:0/283296498 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f156c04db10 con 0x7f156c155360 2023-10-26T01:35:02.463 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.458+0000 7f15627fc640 1 -- 172.21.15.169:0/283296498 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f155c03e450 con 0x7f156c155360 2023-10-26T01:35:02.464 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.459+0000 7f15627fc640 1 --2- 172.21.15.169:0/283296498 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f155006ae40 0x7f155006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:02.464 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.459+0000 7f15627fc640 1 -- 172.21.15.169:0/283296498 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f155c0ac0d0 con 0x7f156c155360 2023-10-26T01:35:02.464 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.459+0000 7f15627fc640 1 -- 172.21.15.169:0/283296498 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=70}) v3 -- 0x7f1550072970 con 0x7f156c155360 2023-10-26T01:35:02.464 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.460+0000 7f1571cb9640 1 --2- 172.21.15.169:0/283296498 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f155006ae40 0x7f155006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:02.464 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.461+0000 7f1571cb9640 1 --2- 172.21.15.169:0/283296498 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f155006ae40 0x7f155006d300 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f156c06d880 tx=0x7f1568009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:35:02.465 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.463+0000 7f15627fc640 1 -- 172.21.15.169:0/283296498 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f155c042060 con 0x7f156c155360 2023-10-26T01:35:02.667 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.665+0000 7f1573f1d640 1 -- 172.21.15.169:0/283296498 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f156c0aab00 con 0x7f156c155360 2023-10-26T01:35:02.668 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.666+0000 7f15627fc640 1 -- 172.21.15.169:0/283296498 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v69) v1 ==== 80+0+0 (secure 0 0 0) 0x7f155c077290 con 0x7f156c155360 2023-10-26T01:35:02.668 INFO:teuthology.orchestra.run.smithi169.stderr:listed 0 entries 2023-10-26T01:35:02.671 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.668+0000 7f1573f1d640 1 -- 172.21.15.169:0/283296498 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f155006ae40 msgr2=0x7f155006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:02.671 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.669+0000 7f1573f1d640 1 --2- 172.21.15.169:0/283296498 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f155006ae40 0x7f155006d300 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f156c06d880 tx=0x7f1568009040 comp rx=0 tx=0).stop 2023-10-26T01:35:02.671 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.669+0000 7f1573f1d640 1 -- 172.21.15.169:0/283296498 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f156c155360 msgr2=0x7f156c14ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:02.671 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.669+0000 7f1573f1d640 1 --2- 172.21.15.169:0/283296498 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f156c155360 0x7f156c14ecc0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f155c000c00 tx=0x7f155c02db50 comp rx=0 tx=0).stop 2023-10-26T01:35:02.671 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.669+0000 7f1573f1d640 1 -- 172.21.15.169:0/283296498 shutdown_connections 2023-10-26T01:35:02.672 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.669+0000 7f1573f1d640 1 --2- 172.21.15.169:0/283296498 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f155006ae40 0x7f155006d300 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:02.672 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.669+0000 7f1573f1d640 1 --2- 172.21.15.169:0/283296498 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f156c155360 0x7f156c14ecc0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:02.672 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.669+0000 7f1573f1d640 1 --2- 172.21.15.169:0/283296498 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f156c15b9e0 0x7f156c14f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:02.672 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.669+0000 7f1573f1d640 1 --2- 172.21.15.169:0/283296498 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f156c0cdef0 0x7f156c14aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:02.672 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.669+0000 7f1573f1d640 1 -- 172.21.15.169:0/283296498 >> 172.21.15.169:0/283296498 conn(0x7f156c069930 msgr2=0x7f156c14b540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:35:02.673 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.669+0000 7f1573f1d640 1 -- 172.21.15.169:0/283296498 shutdown_connections 2023-10-26T01:35:02.673 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:02.669+0000 7f1573f1d640 1 -- 172.21.15.169:0/283296498 wait complete. 2023-10-26T01:35:02.689 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-26T01:35:02.690 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:35:02.690 DEBUG:teuthology.orchestra.run.smithi170:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-26T01:35:02.802 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi170.front.sepia.ceph.com 2023-10-26T01:35:02.802 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:35:02.803 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-26T01:35:02.854 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:35:02.855 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:35:02.915 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054301,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-26T01:35:02.916 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-26T01:35:02.916 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:35:02.977 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054303,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-26T01:35:02.977 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T01:35:03.204 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.202+0000 7f05120bb640 1 -- 172.21.15.169:0/3552647339 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f050c06ed30 msgr2=0x7f050c0cd670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:03.204 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.202+0000 7f05120bb640 1 --2- 172.21.15.169:0/3552647339 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f050c06ed30 0x7f050c0cd670 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f050c0cd170 tx=0x7f04f402bd70 comp rx=0 tx=0).stop 2023-10-26T01:35:03.204 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.202+0000 7f05120bb640 1 -- 172.21.15.169:0/3552647339 shutdown_connections 2023-10-26T01:35:03.205 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.202+0000 7f05120bb640 1 --2- 172.21.15.169:0/3552647339 >> v2:172.21.15.170:3300/0 conn(0x7f050c0cdbb0 0x7f050c0b07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:03.205 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.202+0000 7f05120bb640 1 --2- 172.21.15.169:0/3552647339 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f050c06ed30 0x7f050c0cd670 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:03.205 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.202+0000 7f05120bb640 1 -- 172.21.15.169:0/3552647339 >> 172.21.15.169:0/3552647339 conn(0x7f050c069930 msgr2=0x7f050c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:35:03.205 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.202+0000 7f05120bb640 1 -- 172.21.15.169:0/3552647339 shutdown_connections 2023-10-26T01:35:03.205 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.202+0000 7f05120bb640 1 -- 172.21.15.169:0/3552647339 wait complete. 2023-10-26T01:35:03.207 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.204+0000 7f05120bb640 1 Processor -- start 2023-10-26T01:35:03.207 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.204+0000 7f05120bb640 1 -- start start 2023-10-26T01:35:03.208 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.205+0000 7f05120bb640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f050c06ed30 0x7f050c070760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:03.208 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.205+0000 7f05120bb640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f050c0cdbb0 0x7f050c143850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:03.208 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.205+0000 7f05120bb640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f050c15b610 0x7f050c143d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:03.208 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.205+0000 7f05120bb640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f050c0702d0 con 0x7f050c06ed30 2023-10-26T01:35:03.208 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.205+0000 7f05120bb640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f050c06ffa0 con 0x7f050c15b610 2023-10-26T01:35:03.209 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.205+0000 7f05120bb640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f050c151b10 con 0x7f050c0cdbb0 2023-10-26T01:35:03.209 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.205+0000 7f050b7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f050c06ed30 0x7f050c070760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:03.209 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.205+0000 7f050b7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f050c06ed30 0x7f050c070760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:47340/0 (socket says 172.21.15.169:47340) 2023-10-26T01:35:03.209 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.205+0000 7f050b7fe640 1 -- 172.21.15.169:0/1924881087 learned_addr learned my addr 172.21.15.169:0/1924881087 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:35:03.209 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.205+0000 7f0503fff640 1 --2- 172.21.15.169:0/1924881087 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f050c0cdbb0 0x7f050c143850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:03.210 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.205+0000 7f050bfff640 1 --2- 172.21.15.169:0/1924881087 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f050c15b610 0x7f050c143d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:03.210 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.205+0000 7f050b7fe640 1 -- 172.21.15.169:0/1924881087 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f050c0cdbb0 msgr2=0x7f050c143850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:03.210 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.205+0000 7f050b7fe640 1 --2- 172.21.15.169:0/1924881087 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f050c0cdbb0 0x7f050c143850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:03.210 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.205+0000 7f050b7fe640 1 -- 172.21.15.169:0/1924881087 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f050c15b610 msgr2=0x7f050c143d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:03.210 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.205+0000 7f050b7fe640 1 --2- 172.21.15.169:0/1924881087 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f050c15b610 0x7f050c143d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:03.211 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.205+0000 7f050b7fe640 1 -- 172.21.15.169:0/1924881087 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04f4009490 con 0x7f050c06ed30 2023-10-26T01:35:03.211 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.205+0000 7f050bfff640 1 --2- 172.21.15.169:0/1924881087 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f050c15b610 0x7f050c143d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:35:03.211 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.206+0000 7f050b7fe640 1 --2- 172.21.15.169:0/1924881087 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f050c06ed30 0x7f050c070760 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f04fc00c3c0 tx=0x7f04fc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:35:03.212 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.206+0000 7f0508ff9640 1 -- 172.21.15.169:0/1924881087 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04fc004230 con 0x7f050c06ed30 2023-10-26T01:35:03.213 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.206+0000 7f0508ff9640 1 -- 172.21.15.169:0/1924881087 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f04fc0044a0 con 0x7f050c06ed30 2023-10-26T01:35:03.213 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.206+0000 7f05120bb640 1 -- 172.21.15.169:0/1924881087 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f050c151df0 con 0x7f050c06ed30 2023-10-26T01:35:03.213 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.206+0000 7f0508ff9640 1 -- 172.21.15.169:0/1924881087 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04fc003850 con 0x7f050c06ed30 2023-10-26T01:35:03.213 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.206+0000 7f05120bb640 1 -- 172.21.15.169:0/1924881087 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f050c070ca0 con 0x7f050c06ed30 2023-10-26T01:35:03.213 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.207+0000 7f05120bb640 1 -- 172.21.15.169:0/1924881087 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f050c04db10 con 0x7f050c06ed30 2023-10-26T01:35:03.214 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.208+0000 7f0508ff9640 1 -- 172.21.15.169:0/1924881087 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f04fc0039b0 con 0x7f050c06ed30 2023-10-26T01:35:03.214 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.209+0000 7f0508ff9640 1 --2- 172.21.15.169:0/1924881087 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f04ec06ae40 0x7f04ec06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:03.214 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.209+0000 7f0508ff9640 1 -- 172.21.15.169:0/1924881087 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f04fc088e40 con 0x7f050c06ed30 2023-10-26T01:35:03.214 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.210+0000 7f0508ff9640 1 -- 172.21.15.169:0/1924881087 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=70}) v3 -- 0x7f04ec072970 con 0x7f050c06ed30 2023-10-26T01:35:03.214 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.210+0000 7f0503fff640 1 --2- 172.21.15.169:0/1924881087 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f04ec06ae40 0x7f04ec06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:03.215 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.211+0000 7f0503fff640 1 --2- 172.21.15.169:0/1924881087 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f04ec06ae40 0x7f04ec06d300 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f04f4036000 tx=0x7f04f4004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:35:03.216 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.214+0000 7f0508ff9640 1 -- 172.21.15.169:0/1924881087 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f04fc017070 con 0x7f050c06ed30 2023-10-26T01:35:03.443 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.441+0000 7f05120bb640 1 -- 172.21.15.169:0/1924881087 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f050c0acc70 con 0x7f050c06ed30 2023-10-26T01:35:03.444 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.443+0000 7f0508ff9640 1 -- 172.21.15.169:0/1924881087 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4922 (secure 0 0 0) 0x7f04fc0578e0 con 0x7f050c06ed30 2023-10-26T01:35:03.445 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:35:03.445 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4976,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/2977083370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2977083370},{"type":"v1","addr":"172.21.15.170:6837","nonce":2977083370}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":5176,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/1598214042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1598214042},{"type":"v1","addr":"172.21.15.170:6835","nonce":1598214042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":6087,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/406582181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":406582181},{"type":"v1","addr":"172.21.15.169:6833","nonce":406582181}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T01:33:40.717881+0000","modified":"2023-10-26T01:33:43.796080+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6093},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6093":{"gid":6093,"name":"a","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6835/2338833513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":2338833513},{"type":"v1","addr":"172.21.15.169:6835","nonce":2338833513}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-10-26T01:35:03.445 INFO:teuthology.orchestra.run.smithi169.stderr:dumped fsmap epoch 32 2023-10-26T01:35:03.447 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.445+0000 7f05120bb640 1 -- 172.21.15.169:0/1924881087 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f04ec06ae40 msgr2=0x7f04ec06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:03.447 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.445+0000 7f05120bb640 1 --2- 172.21.15.169:0/1924881087 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f04ec06ae40 0x7f04ec06d300 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f04f4036000 tx=0x7f04f4004730 comp rx=0 tx=0).stop 2023-10-26T01:35:03.447 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.445+0000 7f05120bb640 1 -- 172.21.15.169:0/1924881087 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f050c06ed30 msgr2=0x7f050c070760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:03.447 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.445+0000 7f05120bb640 1 --2- 172.21.15.169:0/1924881087 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f050c06ed30 0x7f050c070760 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f04fc00c3c0 tx=0x7f04fc00c880 comp rx=0 tx=0).stop 2023-10-26T01:35:03.448 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.445+0000 7f05120bb640 1 -- 172.21.15.169:0/1924881087 shutdown_connections 2023-10-26T01:35:03.448 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.445+0000 7f05120bb640 1 --2- 172.21.15.169:0/1924881087 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f04ec06ae40 0x7f04ec06d300 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:03.448 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.445+0000 7f05120bb640 1 --2- 172.21.15.169:0/1924881087 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f050c15b610 0x7f050c143d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:03.448 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.445+0000 7f05120bb640 1 --2- 172.21.15.169:0/1924881087 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f050c0cdbb0 0x7f050c143850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:03.448 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.445+0000 7f05120bb640 1 --2- 172.21.15.169:0/1924881087 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f050c06ed30 0x7f050c070760 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:03.448 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.445+0000 7f05120bb640 1 -- 172.21.15.169:0/1924881087 >> 172.21.15.169:0/1924881087 conn(0x7f050c069930 msgr2=0x7f050c0aa3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:35:03.449 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.445+0000 7f05120bb640 1 -- 172.21.15.169:0/1924881087 shutdown_connections 2023-10-26T01:35:03.449 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.446+0000 7f05120bb640 1 -- 172.21.15.169:0/1924881087 wait complete. 2023-10-26T01:35:03.466 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:35:03.687 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.685+0000 7fcbedae5640 1 -- 172.21.15.169:0/1362610709 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcbd4003970 msgr2=0x7fcbd4003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:03.687 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.685+0000 7fcbedae5640 1 --2- 172.21.15.169:0/1362610709 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcbd4003970 0x7fcbd4003d90 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fcbd8001d50 tx=0x7fcbd40057f0 comp rx=0 tx=0).stop 2023-10-26T01:35:03.687 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.685+0000 7fcbedae5640 1 -- 172.21.15.169:0/1362610709 shutdown_connections 2023-10-26T01:35:03.688 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.685+0000 7fcbedae5640 1 --2- 172.21.15.169:0/1362610709 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcbd4003970 0x7fcbd4003d90 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:03.688 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.685+0000 7fcbedae5640 1 -- 172.21.15.169:0/1362610709 >> 172.21.15.169:0/1362610709 conn(0x7fcbe8069930 msgr2=0x7fcbe8069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:35:03.688 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.685+0000 7fcbedae5640 1 -- 172.21.15.169:0/1362610709 shutdown_connections 2023-10-26T01:35:03.688 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.685+0000 7fcbedae5640 1 -- 172.21.15.169:0/1362610709 wait complete. 2023-10-26T01:35:03.689 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.687+0000 7fcbedae5640 1 Processor -- start 2023-10-26T01:35:03.689 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.687+0000 7fcbedae5640 1 -- start start 2023-10-26T01:35:03.691 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.688+0000 7fcbedae5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcbd4003970 0x7fcbe814c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:03.691 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.688+0000 7fcbedae5640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcbe80b0cf0 0x7fcbe80ade60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:03.691 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.688+0000 7fcbedae5640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcbe80b1150 0x7fcbe80ae3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:03.691 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.688+0000 7fcbedae5640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbe814cc90 con 0x7fcbe80b0cf0 2023-10-26T01:35:03.692 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.688+0000 7fcbedae5640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbe80b22a0 con 0x7fcbd4003970 2023-10-26T01:35:03.692 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.688+0000 7fcbedae5640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbe80b1720 con 0x7fcbe80b1150 2023-10-26T01:35:03.692 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.688+0000 7fcbe6ffd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcbe80b0cf0 0x7fcbe80ade60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:03.692 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.688+0000 7fcbe6ffd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcbe80b0cf0 0x7fcbe80ade60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:47356/0 (socket says 172.21.15.169:47356) 2023-10-26T01:35:03.693 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.688+0000 7fcbe6ffd640 1 -- 172.21.15.169:0/3196935546 learned_addr learned my addr 172.21.15.169:0/3196935546 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:35:03.693 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.689+0000 7fcbe7fff640 1 --2- 172.21.15.169:0/3196935546 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcbe80b1150 0x7fcbe80ae3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:03.693 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.689+0000 7fcbe77fe640 1 --2- 172.21.15.169:0/3196935546 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcbd4003970 0x7fcbe814c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:03.693 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.689+0000 7fcbe6ffd640 1 -- 172.21.15.169:0/3196935546 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcbe80b1150 msgr2=0x7fcbe80ae3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:03.693 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.689+0000 7fcbe6ffd640 1 --2- 172.21.15.169:0/3196935546 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcbe80b1150 0x7fcbe80ae3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:03.694 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.689+0000 7fcbe6ffd640 1 -- 172.21.15.169:0/3196935546 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcbd4003970 msgr2=0x7fcbe814c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:03.694 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.689+0000 7fcbe6ffd640 1 --2- 172.21.15.169:0/3196935546 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcbd4003970 0x7fcbe814c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:03.694 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.689+0000 7fcbe6ffd640 1 -- 172.21.15.169:0/3196935546 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbe80b1a00 con 0x7fcbe80b0cf0 2023-10-26T01:35:03.694 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.689+0000 7fcbe7fff640 1 --2- 172.21.15.169:0/3196935546 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcbe80b1150 0x7fcbe80ae3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:35:03.694 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.689+0000 7fcbe6ffd640 1 --2- 172.21.15.169:0/3196935546 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcbe80b0cf0 0x7fcbe80ade60 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fcbd0005bc0 tx=0x7fcbd000db80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:35:03.695 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.689+0000 7fcbdffff640 1 -- 172.21.15.169:0/3196935546 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbd0004220 con 0x7fcbe80b0cf0 2023-10-26T01:35:03.695 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.689+0000 7fcbdffff640 1 -- 172.21.15.169:0/3196935546 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fcbd0004470 con 0x7fcbe80b0cf0 2023-10-26T01:35:03.695 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.689+0000 7fcbedae5640 1 -- 172.21.15.169:0/3196935546 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbe80b1ce0 con 0x7fcbe80b0cf0 2023-10-26T01:35:03.695 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.689+0000 7fcbdffff640 1 -- 172.21.15.169:0/3196935546 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbd0009a80 con 0x7fcbe80b0cf0 2023-10-26T01:35:03.695 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.690+0000 7fcbedae5640 1 -- 172.21.15.169:0/3196935546 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbe80c7bf0 con 0x7fcbe80b0cf0 2023-10-26T01:35:03.696 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.691+0000 7fcbedae5640 1 -- 172.21.15.169:0/3196935546 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbe804db10 con 0x7fcbe80b0cf0 2023-10-26T01:35:03.697 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.692+0000 7fcbdffff640 1 -- 172.21.15.169:0/3196935546 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fcbd0009be0 con 0x7fcbe80b0cf0 2023-10-26T01:35:03.697 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.693+0000 7fcbdffff640 1 --2- 172.21.15.169:0/3196935546 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcbc806ae70 0x7fcbc806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:03.697 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.693+0000 7fcbdffff640 1 -- 172.21.15.169:0/3196935546 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcbd0089660 con 0x7fcbe80b0cf0 2023-10-26T01:35:03.697 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.693+0000 7fcbdffff640 1 -- 172.21.15.169:0/3196935546 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=70}) v3 -- 0x7fcbc80729a0 con 0x7fcbe80b0cf0 2023-10-26T01:35:03.698 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.694+0000 7fcbe77fe640 1 --2- 172.21.15.169:0/3196935546 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcbc806ae70 0x7fcbc806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:03.698 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.695+0000 7fcbe77fe640 1 --2- 172.21.15.169:0/3196935546 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcbc806ae70 0x7fcbc806d330 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fcbd8034000 tx=0x7fcbd8039040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:35:03.699 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.698+0000 7fcbdffff640 1 -- 172.21.15.169:0/3196935546 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcbd0020060 con 0x7fcbe80b0cf0 2023-10-26T01:35:03.892 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.890+0000 7fcbedae5640 1 -- 172.21.15.169:0/3196935546 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcbe80c7d80 con 0x7fcbe80b0cf0 2023-10-26T01:35:03.894 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.893+0000 7fcbdffff640 1 -- 172.21.15.169:0/3196935546 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) v1 ==== 74+0+16671 (secure 0 0 0) 0x7fcbd0017070 con 0x7fcbe80b0cf0 2023-10-26T01:35:03.895 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:35:03.895 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":69,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:34:04.037343+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-10-26T01:33:38.593237+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-10-26T01:33:39.753692+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","full","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","full","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","full","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["backfillfull","exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","full","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:35:03.897 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.896+0000 7fcbedae5640 1 -- 172.21.15.169:0/3196935546 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcbc806ae70 msgr2=0x7fcbc806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:03.898 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.896+0000 7fcbedae5640 1 --2- 172.21.15.169:0/3196935546 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcbc806ae70 0x7fcbc806d330 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fcbd8034000 tx=0x7fcbd8039040 comp rx=0 tx=0).stop 2023-10-26T01:35:03.898 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.896+0000 7fcbedae5640 1 -- 172.21.15.169:0/3196935546 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcbe80b0cf0 msgr2=0x7fcbe80ade60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:03.898 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.896+0000 7fcbedae5640 1 --2- 172.21.15.169:0/3196935546 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcbe80b0cf0 0x7fcbe80ade60 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fcbd0005bc0 tx=0x7fcbd000db80 comp rx=0 tx=0).stop 2023-10-26T01:35:03.898 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.896+0000 7fcbedae5640 1 -- 172.21.15.169:0/3196935546 shutdown_connections 2023-10-26T01:35:03.898 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.896+0000 7fcbedae5640 1 --2- 172.21.15.169:0/3196935546 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcbc806ae70 0x7fcbc806d330 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:03.899 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.896+0000 7fcbedae5640 1 --2- 172.21.15.169:0/3196935546 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcbe80b0cf0 0x7fcbe80ade60 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:03.899 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.896+0000 7fcbedae5640 1 --2- 172.21.15.169:0/3196935546 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcbe80b1150 0x7fcbe80ae3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:03.899 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.896+0000 7fcbedae5640 1 --2- 172.21.15.169:0/3196935546 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcbd4003970 0x7fcbe814c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:03.900 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.896+0000 7fcbedae5640 1 -- 172.21.15.169:0/3196935546 >> 172.21.15.169:0/3196935546 conn(0x7fcbe8069930 msgr2=0x7fcbe80aeec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:35:03.900 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.896+0000 7fcbedae5640 1 -- 172.21.15.169:0/3196935546 shutdown_connections 2023-10-26T01:35:03.900 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:03.896+0000 7fcbedae5640 1 -- 172.21.15.169:0/3196935546 wait complete. 2023-10-26T01:35:03.916 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T01:30:32.200808+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-26T01:33:38.593237+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2023-10-26T01:33:39.753692+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T01:35:03.917 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T01:35:03.967 INFO:tasks.ceph.mon.a.smithi169.stderr:2023-10-26T01:35:03.966+0000 7f158b279640 -1 log_channel(cluster) log [ERR] : Health check update: 2 full osd(s) (OSD_FULL) 2023-10-26T01:35:04.140 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:04.138+0000 7f04ea10f640 1 -- 172.21.15.169:0/1495701984 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04d0003970 msgr2=0x7f04d0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:04.140 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:04.138+0000 7f04ea10f640 1 --2- 172.21.15.169:0/1495701984 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04d0003970 0x7f04d0003d90 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f04d4001bb0 tx=0x7f04d00057f0 comp rx=0 tx=0).stop 2023-10-26T01:35:04.140 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:04.138+0000 7f04ea10f640 1 -- 172.21.15.169:0/1495701984 shutdown_connections 2023-10-26T01:35:04.141 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:04.138+0000 7f04ea10f640 1 --2- 172.21.15.169:0/1495701984 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04d0003970 0x7f04d0003d90 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:04.141 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:04.138+0000 7f04ea10f640 1 -- 172.21.15.169:0/1495701984 >> 172.21.15.169:0/1495701984 conn(0x7f04e4069930 msgr2=0x7f04e4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:35:04.141 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:04.138+0000 7f04ea10f640 1 -- 172.21.15.169:0/1495701984 shutdown_connections 2023-10-26T01:35:04.141 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:04.138+0000 7f04ea10f640 1 -- 172.21.15.169:0/1495701984 wait complete. 2023-10-26T01:35:04.142 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:04.140+0000 7f04ea10f640 1 Processor -- start 2023-10-26T01:35:04.142 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:04.140+0000 7f04ea10f640 1 -- start start 2023-10-26T01:35:04.144 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:04.141+0000 7f04ea10f640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f04d0003970 0x7f04e414c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:04.144 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:04.141+0000 7f04ea10f640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04e4151500 0x7f04e4151960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:04.144 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:04.141+0000 7f04ea10f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f04e406fd10 0x7f04e4070190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:04.144 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:04.141+0000 7f04ea10f640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04e40706d0 con 0x7f04e4151500 2023-10-26T01:35:04.145 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:04.141+0000 7f04ea10f640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04e4151fc0 con 0x7f04e406fd10 2023-10-26T01:35:04.145 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:04.141+0000 7f04ea10f640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04e41522a0 con 0x7f04d0003970 2023-10-26T01:35:04.145 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:04.141+0000 7f04daffd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04e4151500 0x7f04e4151960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:04.145 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:04.141+0000 7f04daffd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04e4151500 0x7f04e4151960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:48458/0 (socket says 172.21.15.169:48458) 2023-10-26T01:35:04.145 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:04.141+0000 7f04daffd640 1 -- 172.21.15.169:0/4283024078 learned_addr learned my addr 172.21.15.169:0/4283024078 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:35:04.146 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:04.141+0000 7f04e37fe640 1 --2- 172.21.15.169:0/4283024078 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f04d0003970 0x7f04e414c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:04.146 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:04.142+0000 7f04daffd640 1 -- 172.21.15.169:0/4283024078 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f04d0003970 msgr2=0x7f04e414c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:04.146 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:04.142+0000 7f04daffd640 1 --2- 172.21.15.169:0/4283024078 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f04d0003970 0x7f04e414c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:04.146 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:04.142+0000 7f04daffd640 1 -- 172.21.15.169:0/4283024078 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f04e406fd10 msgr2=0x7f04e4070190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:04.146 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:04.142+0000 7f04daffd640 1 --2- 172.21.15.169:0/4283024078 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f04e406fd10 0x7f04e4070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:04.147 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:04.142+0000 7f04daffd640 1 -- 172.21.15.169:0/4283024078 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04e4152580 con 0x7f04e4151500 2023-10-26T01:35:04.147 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:04.142+0000 7f04daffd640 1 --2- 172.21.15.169:0/4283024078 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04e4151500 0x7f04e4151960 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f04cc005bc0 tx=0x7f04cc00db80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:35:04.147 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:04.142+0000 7f04e0ff9640 1 -- 172.21.15.169:0/4283024078 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04cc004220 con 0x7f04e4151500 2023-10-26T01:35:04.147 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:04.142+0000 7f04e0ff9640 1 -- 172.21.15.169:0/4283024078 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f04cc004470 con 0x7f04e4151500 2023-10-26T01:35:04.147 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:04.142+0000 7f04e0ff9640 1 -- 172.21.15.169:0/4283024078 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04cc009a80 con 0x7f04e4151500 2023-10-26T01:35:04.148 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:04.142+0000 7f04ea10f640 1 -- 172.21.15.169:0/4283024078 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04e414ed90 con 0x7f04e4151500 2023-10-26T01:35:04.148 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:04.142+0000 7f04ea10f640 1 -- 172.21.15.169:0/4283024078 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04e414cc90 con 0x7f04e4151500 2023-10-26T01:35:04.149 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:04.143+0000 7f04ea10f640 1 -- 172.21.15.169:0/4283024078 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04e4066af0 con 0x7f04e4151500 2023-10-26T01:35:04.149 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:04.144+0000 7f04e0ff9640 1 -- 172.21.15.169:0/4283024078 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f04cc009be0 con 0x7f04e4151500 2023-10-26T01:35:04.149 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:04.145+0000 7f04e0ff9640 1 --2- 172.21.15.169:0/4283024078 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f04c406ae70 0x7f04c406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:04.149 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:04.145+0000 7f04e0ff9640 1 -- 172.21.15.169:0/4283024078 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f04cc089700 con 0x7f04e4151500 2023-10-26T01:35:04.150 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:04.146+0000 7f04e0ff9640 1 -- 172.21.15.169:0/4283024078 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=71}) v3 -- 0x7f04c40729a0 con 0x7f04e4151500 2023-10-26T01:35:04.150 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:04.146+0000 7f04e37fe640 1 --2- 172.21.15.169:0/4283024078 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f04c406ae70 0x7f04c406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:04.150 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:04.147+0000 7f04e37fe640 1 --2- 172.21.15.169:0/4283024078 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f04c406ae70 0x7f04c406d330 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f04d4034000 tx=0x7f04d4001510 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:35:04.151 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:04.149+0000 7f04e0ff9640 1 -- 172.21.15.169:0/4283024078 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f04cc020060 con 0x7f04e4151500 2023-10-26T01:35:04.339 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:04.337+0000 7f04ea10f640 1 -- 172.21.15.169:0/4283024078 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f04e404da70 con 0x7f04e4151500 2023-10-26T01:35:04.340 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:04.338+0000 7f04e0ff9640 1 -- 172.21.15.169:0/4283024078 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v70) v1 ==== 93+0+10 (secure 0 0 0) 0x7f04cc017070 con 0x7f04e4151500 2023-10-26T01:35:04.340 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 1 2023-10-26T01:35:04.342 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:04.340+0000 7f04ea10f640 1 -- 172.21.15.169:0/4283024078 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f04c406ae70 msgr2=0x7f04c406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:04.343 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:04.340+0000 7f04ea10f640 1 --2- 172.21.15.169:0/4283024078 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f04c406ae70 0x7f04c406d330 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f04d4034000 tx=0x7f04d4001510 comp rx=0 tx=0).stop 2023-10-26T01:35:04.343 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:04.341+0000 7f04ea10f640 1 -- 172.21.15.169:0/4283024078 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04e4151500 msgr2=0x7f04e4151960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:04.343 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:04.341+0000 7f04ea10f640 1 --2- 172.21.15.169:0/4283024078 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04e4151500 0x7f04e4151960 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f04cc005bc0 tx=0x7f04cc00db80 comp rx=0 tx=0).stop 2023-10-26T01:35:04.343 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:04.341+0000 7f04ea10f640 1 -- 172.21.15.169:0/4283024078 shutdown_connections 2023-10-26T01:35:04.344 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:04.341+0000 7f04ea10f640 1 --2- 172.21.15.169:0/4283024078 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f04c406ae70 0x7f04c406d330 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:04.344 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:04.341+0000 7f04ea10f640 1 --2- 172.21.15.169:0/4283024078 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04e4151500 0x7f04e4151960 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:04.344 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:04.341+0000 7f04ea10f640 1 --2- 172.21.15.169:0/4283024078 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f04e406fd10 0x7f04e4070190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:04.344 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:04.341+0000 7f04ea10f640 1 --2- 172.21.15.169:0/4283024078 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f04d0003970 0x7f04e414c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:04.344 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:04.341+0000 7f04ea10f640 1 -- 172.21.15.169:0/4283024078 >> 172.21.15.169:0/4283024078 conn(0x7f04e4069930 msgr2=0x7f04e4071400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:35:04.345 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:04.341+0000 7f04ea10f640 1 -- 172.21.15.169:0/4283024078 shutdown_connections 2023-10-26T01:35:04.345 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:04.341+0000 7f04ea10f640 1 -- 172.21.15.169:0/4283024078 wait complete. 2023-10-26T01:35:04.361 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-26T01:35:04.579 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:04.577+0000 7f7f714c3640 1 -- 172.21.15.169:0/519903356 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f58003f40 msgr2=0x7f7f580047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:04.579 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:04.577+0000 7f7f714c3640 1 --2- 172.21.15.169:0/519903356 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f58003f40 0x7f7f580047c0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f7f5400baf0 tx=0x7f7f580057f0 comp rx=0 tx=0).stop 2023-10-26T01:35:04.579 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:04.577+0000 7f7f70cc2640 1 -- 172.21.15.169:0/519903356 reap_dead start 2023-10-26T01:35:04.579 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:04.577+0000 7f7f714c3640 1 -- 172.21.15.169:0/519903356 shutdown_connections 2023-10-26T01:35:04.580 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:04.577+0000 7f7f714c3640 1 -- 172.21.15.169:0/519903356 >> 172.21.15.169:0/519903356 conn(0x7f7f6c0d1370 msgr2=0x7f7f6c0d1780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:35:04.580 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:04.577+0000 7f7f714c3640 1 -- 172.21.15.169:0/519903356 shutdown_connections 2023-10-26T01:35:04.580 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:04.577+0000 7f7f714c3640 1 -- 172.21.15.169:0/519903356 wait complete. 2023-10-26T01:35:04.581 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:04.579+0000 7f7f714c3640 1 Processor -- start 2023-10-26T01:35:04.581 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:04.579+0000 7f7f714c3640 1 -- start start 2023-10-26T01:35:04.583 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:04.580+0000 7f7f714c3640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7f6c1e65d0 0x7f7f6c1f52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:04.583 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:04.580+0000 7f7f714c3640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f6c0c7a00 0x7f7f6c0b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:04.584 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:04.580+0000 7f7f714c3640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7f6c0b2730 0x7f7f6c0ca310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:04.584 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:04.580+0000 7f7f714c3640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f6c0b2bb0 con 0x7f7f6c0c7a00 2023-10-26T01:35:04.584 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:04.580+0000 7f7f714c3640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f6c1f5bd0 con 0x7f7f6c1e65d0 2023-10-26T01:35:04.584 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:04.580+0000 7f7f714c3640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f6c1f5eb0 con 0x7f7f6c0b2730 2023-10-26T01:35:04.584 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:04.580+0000 7f7f637fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f6c0c7a00 0x7f7f6c0b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:04.585 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:04.580+0000 7f7f637fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f6c0c7a00 0x7f7f6c0b21f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:48464/0 (socket says 172.21.15.169:48464) 2023-10-26T01:35:04.585 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:04.580+0000 7f7f637fe640 1 -- 172.21.15.169:0/3336852638 learned_addr learned my addr 172.21.15.169:0/3336852638 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:35:04.585 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:04.580+0000 7f7f6bfff640 1 --2- 172.21.15.169:0/3336852638 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7f6c1e65d0 0x7f7f6c1f52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:04.585 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:04.580+0000 7f7f70cc2640 1 --2- 172.21.15.169:0/3336852638 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7f6c0b2730 0x7f7f6c0ca310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:04.585 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:04.580+0000 7f7f637fe640 1 -- 172.21.15.169:0/3336852638 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7f6c0b2730 msgr2=0x7f7f6c0ca310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:04.586 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:04.581+0000 7f7f637fe640 1 --2- 172.21.15.169:0/3336852638 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7f6c0b2730 0x7f7f6c0ca310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:04.586 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:04.581+0000 7f7f637fe640 1 -- 172.21.15.169:0/3336852638 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7f6c1e65d0 msgr2=0x7f7f6c1f52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:04.586 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:04.581+0000 7f7f637fe640 1 --2- 172.21.15.169:0/3336852638 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7f6c1e65d0 0x7f7f6c1f52f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:04.586 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:04.581+0000 7f7f637fe640 1 -- 172.21.15.169:0/3336852638 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f6c1f6190 con 0x7f7f6c0c7a00 2023-10-26T01:35:04.586 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:04.581+0000 7f7f6bfff640 1 --2- 172.21.15.169:0/3336852638 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7f6c1e65d0 0x7f7f6c1f52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:35:04.587 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:04.581+0000 7f7f637fe640 1 --2- 172.21.15.169:0/3336852638 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f6c0c7a00 0x7f7f6c0b21f0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f7f5400ba40 tx=0x7f7f54036700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:35:04.587 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:04.581+0000 7f7f697fa640 1 -- 172.21.15.169:0/3336852638 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f54036d00 con 0x7f7f6c0c7a00 2023-10-26T01:35:04.587 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:04.581+0000 7f7f697fa640 1 -- 172.21.15.169:0/3336852638 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7f54004020 con 0x7f7f6c0c7a00 2023-10-26T01:35:04.587 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:04.581+0000 7f7f714c3640 1 -- 172.21.15.169:0/3336852638 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f6c1e4110 con 0x7f7f6c0c7a00 2023-10-26T01:35:04.587 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:04.581+0000 7f7f697fa640 1 -- 172.21.15.169:0/3336852638 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f54004430 con 0x7f7f6c0c7a00 2023-10-26T01:35:04.588 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:04.581+0000 7f7f714c3640 1 -- 172.21.15.169:0/3336852638 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f6c1f5830 con 0x7f7f6c0c7a00 2023-10-26T01:35:04.589 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:04.582+0000 7f7f714c3640 1 -- 172.21.15.169:0/3336852638 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f6c073c10 con 0x7f7f6c0c7a00 2023-10-26T01:35:04.589 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:04.583+0000 7f7f697fa640 1 -- 172.21.15.169:0/3336852638 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f7f540035f0 con 0x7f7f6c0c7a00 2023-10-26T01:35:04.589 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:04.584+0000 7f7f697fa640 1 --2- 172.21.15.169:0/3336852638 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7f4c06ae70 0x7f7f4c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:04.589 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:04.584+0000 7f7f697fa640 1 -- 172.21.15.169:0/3336852638 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7f540ab4d0 con 0x7f7f6c0c7a00 2023-10-26T01:35:04.589 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:04.585+0000 7f7f697fa640 1 -- 172.21.15.169:0/3336852638 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=71}) v3 -- 0x7f7f4c0729a0 con 0x7f7f6c0c7a00 2023-10-26T01:35:04.590 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:04.585+0000 7f7f6bfff640 1 --2- 172.21.15.169:0/3336852638 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7f4c06ae70 0x7f7f4c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:04.590 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:04.586+0000 7f7f6bfff640 1 --2- 172.21.15.169:0/3336852638 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7f4c06ae70 0x7f7f4c06d330 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c005fd0 tx=0x7f7f5c009500 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:35:04.590 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:04.589+0000 7f7f697fa640 1 -- 172.21.15.169:0/3336852638 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7f54032060 con 0x7f7f6c0c7a00 2023-10-26T01:35:04.781 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:04.779+0000 7f7f714c3640 1 -- 172.21.15.169:0/3336852638 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7f6c06ae50 con 0x7f7f6c0c7a00 2023-10-26T01:35:04.782 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:04.780+0000 7f7f697fa640 1 -- 172.21.15.169:0/3336852638 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v70) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7f540764e0 con 0x7f7f6c0c7a00 2023-10-26T01:35:04.782 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 32 2023-10-26T01:35:04.785 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:04.783+0000 7f7f714c3640 1 -- 172.21.15.169:0/3336852638 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7f4c06ae70 msgr2=0x7f7f4c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:04.785 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:04.783+0000 7f7f714c3640 1 --2- 172.21.15.169:0/3336852638 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7f4c06ae70 0x7f7f4c06d330 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c005fd0 tx=0x7f7f5c009500 comp rx=0 tx=0).stop 2023-10-26T01:35:04.785 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:04.783+0000 7f7f714c3640 1 -- 172.21.15.169:0/3336852638 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f6c0c7a00 msgr2=0x7f7f6c0b21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:04.786 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:04.783+0000 7f7f714c3640 1 --2- 172.21.15.169:0/3336852638 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f6c0c7a00 0x7f7f6c0b21f0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f7f5400ba40 tx=0x7f7f54036700 comp rx=0 tx=0).stop 2023-10-26T01:35:04.786 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:04.783+0000 7f7f714c3640 1 -- 172.21.15.169:0/3336852638 shutdown_connections 2023-10-26T01:35:04.786 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:04.783+0000 7f7f714c3640 1 --2- 172.21.15.169:0/3336852638 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7f4c06ae70 0x7f7f4c06d330 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:04.786 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:04.783+0000 7f7f714c3640 1 --2- 172.21.15.169:0/3336852638 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f6c0c7a00 0x7f7f6c0b21f0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:04.786 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:04.783+0000 7f7f714c3640 1 --2- 172.21.15.169:0/3336852638 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7f6c0b2730 0x7f7f6c0ca310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:04.786 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:04.783+0000 7f7f714c3640 1 --2- 172.21.15.169:0/3336852638 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7f6c1e65d0 0x7f7f6c1f52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:04.787 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:04.783+0000 7f7f714c3640 1 -- 172.21.15.169:0/3336852638 >> 172.21.15.169:0/3336852638 conn(0x7f7f6c0d1370 msgr2=0x7f7f6c0d0210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:35:04.787 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:04.783+0000 7f7f714c3640 1 -- 172.21.15.169:0/3336852638 shutdown_connections 2023-10-26T01:35:04.787 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:04.783+0000 7f7f714c3640 1 -- 172.21.15.169:0/3336852638 wait complete. 2023-10-26T01:35:04.803 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-26T01:35:05.025 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.023+0000 7fd021f54640 1 -- 172.21.15.169:0/1895046894 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd01c153b90 msgr2=0x7fd01c069900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:05.025 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.023+0000 7fd021f54640 1 --2- 172.21.15.169:0/1895046894 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd01c153b90 0x7fd01c069900 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fd010005fd0 tx=0x7fd01002bd80 comp rx=0 tx=0).stop 2023-10-26T01:35:05.025 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.023+0000 7fd021f54640 1 -- 172.21.15.169:0/1895046894 shutdown_connections 2023-10-26T01:35:05.026 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.023+0000 7fd021f54640 1 --2- 172.21.15.169:0/1895046894 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd01c0b0e50 0x7fd01c069e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:05.026 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.023+0000 7fd021f54640 1 --2- 172.21.15.169:0/1895046894 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd01c153b90 0x7fd01c069900 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:05.026 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.023+0000 7fd021f54640 1 -- 172.21.15.169:0/1895046894 >> 172.21.15.169:0/1895046894 conn(0x7fd01c0af110 msgr2=0x7fd01c0af520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:35:05.026 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.023+0000 7fd021f54640 1 -- 172.21.15.169:0/1895046894 shutdown_connections 2023-10-26T01:35:05.026 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.023+0000 7fd021f54640 1 -- 172.21.15.169:0/1895046894 wait complete. 2023-10-26T01:35:05.027 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.025+0000 7fd021f54640 1 Processor -- start 2023-10-26T01:35:05.027 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.025+0000 7fd021f54640 1 -- start start 2023-10-26T01:35:05.029 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.026+0000 7fd021f54640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd01c0b0e50 0x7fd01c1f4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:05.030 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.026+0000 7fd021f54640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd01c153b90 0x7fd01c0c8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:05.030 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.026+0000 7fd021f54640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd01c15d0d0 0x7fd01c06ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:05.030 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.026+0000 7fd021f54640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd01c071610 con 0x7fd01c15d0d0 2023-10-26T01:35:05.030 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.026+0000 7fd021f54640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd01c0c7b70 con 0x7fd01c0b0e50 2023-10-26T01:35:05.030 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.026+0000 7fd021f54640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd01c0c7e50 con 0x7fd01c153b90 2023-10-26T01:35:05.031 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.026+0000 7fd021753640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd01c15d0d0 0x7fd01c06ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:05.031 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.026+0000 7fd021753640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd01c15d0d0 0x7fd01c06ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:48476/0 (socket says 172.21.15.169:48476) 2023-10-26T01:35:05.031 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.026+0000 7fd021753640 1 -- 172.21.15.169:0/1345032854 learned_addr learned my addr 172.21.15.169:0/1345032854 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:35:05.031 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.027+0000 7fd020f52640 1 --2- 172.21.15.169:0/1345032854 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd01c0b0e50 0x7fd01c1f4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:05.031 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.027+0000 7fd01bfff640 1 --2- 172.21.15.169:0/1345032854 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd01c153b90 0x7fd01c0c8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:05.032 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.027+0000 7fd021753640 1 -- 172.21.15.169:0/1345032854 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd01c153b90 msgr2=0x7fd01c0c8c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:05.032 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.027+0000 7fd021753640 1 --2- 172.21.15.169:0/1345032854 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd01c153b90 0x7fd01c0c8c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:05.032 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.027+0000 7fd021753640 1 -- 172.21.15.169:0/1345032854 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd01c0b0e50 msgr2=0x7fd01c1f4e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:05.032 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.027+0000 7fd021753640 1 --2- 172.21.15.169:0/1345032854 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd01c0b0e50 0x7fd01c1f4e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:05.032 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.027+0000 7fd021753640 1 -- 172.21.15.169:0/1345032854 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd010009560 con 0x7fd01c15d0d0 2023-10-26T01:35:05.033 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.027+0000 7fd01bfff640 1 --2- 172.21.15.169:0/1345032854 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd01c153b90 0x7fd01c0c8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:35:05.033 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.027+0000 7fd021753640 1 --2- 172.21.15.169:0/1345032854 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd01c15d0d0 0x7fd01c06ab10 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fd01400b990 tx=0x7fd01400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:35:05.033 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.027+0000 7fd0197fa640 1 -- 172.21.15.169:0/1345032854 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd01400cba0 con 0x7fd01c15d0d0 2023-10-26T01:35:05.033 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.027+0000 7fd0197fa640 1 -- 172.21.15.169:0/1345032854 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd01400cdf0 con 0x7fd01c15d0d0 2023-10-26T01:35:05.033 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.027+0000 7fd0197fa640 1 -- 172.21.15.169:0/1345032854 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0140146a0 con 0x7fd01c15d0d0 2023-10-26T01:35:05.034 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.028+0000 7fd021f54640 1 -- 172.21.15.169:0/1345032854 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd01c0c8130 con 0x7fd01c15d0d0 2023-10-26T01:35:05.034 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.028+0000 7fd021f54640 1 -- 172.21.15.169:0/1345032854 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd01c0c86f0 con 0x7fd01c15d0d0 2023-10-26T01:35:05.035 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.029+0000 7fd021f54640 1 -- 172.21.15.169:0/1345032854 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd01c0c8910 con 0x7fd01c15d0d0 2023-10-26T01:35:05.035 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.030+0000 7fd0197fa640 1 -- 172.21.15.169:0/1345032854 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fd01401d3b0 con 0x7fd01c15d0d0 2023-10-26T01:35:05.035 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.031+0000 7fd0197fa640 1 --2- 172.21.15.169:0/1345032854 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd00006ae40 0x7fd00006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:05.035 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.031+0000 7fd0197fa640 1 -- 172.21.15.169:0/1345032854 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd0140898c0 con 0x7fd01c15d0d0 2023-10-26T01:35:05.036 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.031+0000 7fd0197fa640 1 -- 172.21.15.169:0/1345032854 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=72}) v3 -- 0x7fd000072970 con 0x7fd01c15d0d0 2023-10-26T01:35:05.036 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.031+0000 7fd020f52640 1 --2- 172.21.15.169:0/1345032854 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd00006ae40 0x7fd00006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:05.036 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.033+0000 7fd020f52640 1 --2- 172.21.15.169:0/1345032854 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd00006ae40 0x7fd00006d300 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fd010035000 tx=0x7fd010004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:35:05.037 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.035+0000 7fd0197fa640 1 -- 172.21.15.169:0/1345032854 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd014019070 con 0x7fd01c15d0d0 2023-10-26T01:35:05.230 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.228+0000 7fd021f54640 1 -- 172.21.15.169:0/1345032854 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd01c09a3f0 con 0x7fd01c15d0d0 2023-10-26T01:35:05.230 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.229+0000 7fd0197fa640 1 -- 172.21.15.169:0/1345032854 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v71) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd014089030 con 0x7fd01c15d0d0 2023-10-26T01:35:05.231 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 64 2023-10-26T01:35:05.233 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.231+0000 7fd021f54640 1 -- 172.21.15.169:0/1345032854 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd00006ae40 msgr2=0x7fd00006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:05.233 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.231+0000 7fd021f54640 1 --2- 172.21.15.169:0/1345032854 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd00006ae40 0x7fd00006d300 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fd010035000 tx=0x7fd010004730 comp rx=0 tx=0).stop 2023-10-26T01:35:05.233 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.231+0000 7fd021f54640 1 -- 172.21.15.169:0/1345032854 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd01c15d0d0 msgr2=0x7fd01c06ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:05.233 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.231+0000 7fd021f54640 1 --2- 172.21.15.169:0/1345032854 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd01c15d0d0 0x7fd01c06ab10 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fd01400b990 tx=0x7fd01400be50 comp rx=0 tx=0).stop 2023-10-26T01:35:05.234 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.232+0000 7fd021f54640 1 -- 172.21.15.169:0/1345032854 shutdown_connections 2023-10-26T01:35:05.234 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.232+0000 7fd021f54640 1 --2- 172.21.15.169:0/1345032854 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd00006ae40 0x7fd00006d300 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:05.235 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.232+0000 7fd021f54640 1 --2- 172.21.15.169:0/1345032854 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd01c15d0d0 0x7fd01c06ab10 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:05.235 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.232+0000 7fd021f54640 1 --2- 172.21.15.169:0/1345032854 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd01c153b90 0x7fd01c0c8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:05.235 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.232+0000 7fd021f54640 1 --2- 172.21.15.169:0/1345032854 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd01c0b0e50 0x7fd01c1f4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:05.235 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.232+0000 7fd021f54640 1 -- 172.21.15.169:0/1345032854 >> 172.21.15.169:0/1345032854 conn(0x7fd01c0af110 msgr2=0x7fd01c0ae4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:35:05.235 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.232+0000 7fd021f54640 1 -- 172.21.15.169:0/1345032854 shutdown_connections 2023-10-26T01:35:05.236 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.232+0000 7fd021f54640 1 -- 172.21.15.169:0/1345032854 wait complete. 2023-10-26T01:35:05.251 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T01:35:05.424 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T01:35:05.431 DEBUG:teuthology.orchestra.run.smithi170:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T01:35:05.463 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.460+0000 7f3b7359e640 1 -- 172.21.15.169:0/2815802656 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b740b45d0 msgr2=0x7f3b740cbce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:05.463 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.460+0000 7f3b7359e640 1 --2- 172.21.15.169:0/2815802656 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b740b45d0 0x7f3b740cbce0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f3b74071a90 tx=0x7f3b6802bda0 comp rx=0 tx=0).stop 2023-10-26T01:35:05.463 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.461+0000 7f3b7359e640 1 -- 172.21.15.169:0/2815802656 shutdown_connections 2023-10-26T01:35:05.463 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.461+0000 7f3b7359e640 1 --2- 172.21.15.169:0/2815802656 >> v2:172.21.15.170:3300/0 conn(0x7f3b740b4a30 0x7f3b740cc220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:05.463 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.461+0000 7f3b7359e640 1 --2- 172.21.15.169:0/2815802656 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b740b45d0 0x7f3b740cbce0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:05.464 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.461+0000 7f3b7359e640 1 --2- 172.21.15.169:0/2815802656 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3b740b68c0 0x7f3b74065ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:05.464 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.461+0000 7f3b7359e640 1 -- 172.21.15.169:0/2815802656 >> 172.21.15.169:0/2815802656 conn(0x7f3b740d1360 msgr2=0x7f3b740d1770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:35:05.464 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.461+0000 7f3b7359e640 1 -- 172.21.15.169:0/2815802656 shutdown_connections 2023-10-26T01:35:05.464 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.461+0000 7f3b7359e640 1 -- 172.21.15.169:0/2815802656 wait complete. 2023-10-26T01:35:05.466 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.463+0000 7f3b7359e640 1 Processor -- start 2023-10-26T01:35:05.466 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.463+0000 7f3b7359e640 1 -- start start 2023-10-26T01:35:05.467 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.464+0000 7f3b7359e640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3b740b45d0 0x7f3b741f1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:05.467 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.464+0000 7f3b7359e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b740b4a30 0x7f3b74155680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:05.467 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.464+0000 7f3b7359e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3b740b68c0 0x7f3b741586b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:05.467 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.464+0000 7f3b7359e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b740b0640 con 0x7f3b740b4a30 2023-10-26T01:35:05.467 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.464+0000 7f3b7359e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b741f20e0 con 0x7f3b740b68c0 2023-10-26T01:35:05.468 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.464+0000 7f3b7359e640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b741572f0 con 0x7f3b740b45d0 2023-10-26T01:35:05.468 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.464+0000 7f3b71d9b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b740b4a30 0x7f3b74155680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:05.468 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.464+0000 7f3b71d9b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b740b4a30 0x7f3b74155680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:48488/0 (socket says 172.21.15.169:48488) 2023-10-26T01:35:05.468 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.464+0000 7f3b71d9b640 1 -- 172.21.15.169:0/3844920184 learned_addr learned my addr 172.21.15.169:0/3844920184 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:35:05.468 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.464+0000 7f3b7259c640 1 --2- 172.21.15.169:0/3844920184 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3b740b45d0 0x7f3b741f1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:05.469 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.464+0000 7f3b72d9d640 1 --2- 172.21.15.169:0/3844920184 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3b740b68c0 0x7f3b741586b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:05.469 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.464+0000 7f3b71d9b640 1 -- 172.21.15.169:0/3844920184 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3b740b45d0 msgr2=0x7f3b741f1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:05.469 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.464+0000 7f3b71d9b640 1 --2- 172.21.15.169:0/3844920184 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3b740b45d0 0x7f3b741f1ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:05.469 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.464+0000 7f3b71d9b640 1 -- 172.21.15.169:0/3844920184 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3b740b68c0 msgr2=0x7f3b741586b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:05.469 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.464+0000 7f3b71d9b640 1 --2- 172.21.15.169:0/3844920184 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3b740b68c0 0x7f3b741586b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:05.470 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.464+0000 7f3b71d9b640 1 -- 172.21.15.169:0/3844920184 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b68009560 con 0x7f3b740b4a30 2023-10-26T01:35:05.470 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.464+0000 7f3b72d9d640 1 --2- 172.21.15.169:0/3844920184 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3b740b68c0 0x7f3b741586b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:35:05.470 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.465+0000 7f3b71d9b640 1 --2- 172.21.15.169:0/3844920184 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b740b4a30 0x7f3b74155680 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f3b68036040 tx=0x7f3b6802dad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:35:05.471 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.465+0000 7f3b62ffd640 1 -- 172.21.15.169:0/3844920184 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b68003800 con 0x7f3b740b4a30 2023-10-26T01:35:05.472 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.465+0000 7f3b62ffd640 1 -- 172.21.15.169:0/3844920184 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3b68003a50 con 0x7f3b740b4a30 2023-10-26T01:35:05.472 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.465+0000 7f3b62ffd640 1 -- 172.21.15.169:0/3844920184 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b68003e20 con 0x7f3b740b4a30 2023-10-26T01:35:05.472 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.465+0000 7f3b7359e640 1 -- 172.21.15.169:0/3844920184 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b741566e0 con 0x7f3b740b4a30 2023-10-26T01:35:05.472 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.465+0000 7f3b7359e640 1 -- 172.21.15.169:0/3844920184 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b74156ca0 con 0x7f3b740b4a30 2023-10-26T01:35:05.472 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.466+0000 7f3b7359e640 1 -- 172.21.15.169:0/3844920184 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b74073c10 con 0x7f3b740b4a30 2023-10-26T01:35:05.473 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.467+0000 7f3b62ffd640 1 -- 172.21.15.169:0/3844920184 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f3b6803e450 con 0x7f3b740b4a30 2023-10-26T01:35:05.473 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.468+0000 7f3b62ffd640 1 --2- 172.21.15.169:0/3844920184 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3b5406ae40 0x7f3b5406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:05.473 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.469+0000 7f3b62ffd640 1 -- 172.21.15.169:0/3844920184 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3b680ac170 con 0x7f3b740b4a30 2023-10-26T01:35:05.473 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.469+0000 7f3b62ffd640 1 -- 172.21.15.169:0/3844920184 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=72}) v3 -- 0x7f3b54072970 con 0x7f3b740b4a30 2023-10-26T01:35:05.473 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.469+0000 7f3b7259c640 1 --2- 172.21.15.169:0/3844920184 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3b5406ae40 0x7f3b5406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:05.474 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.470+0000 7f3b7259c640 1 --2- 172.21.15.169:0/3844920184 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3b5406ae40 0x7f3b5406d300 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f3b640058c0 tx=0x7f3b64005850 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:35:05.475 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.474+0000 7f3b62ffd640 1 -- 172.21.15.169:0/3844920184 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3b68042060 con 0x7f3b740b4a30 2023-10-26T01:35:05.706 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.705+0000 7f3b7359e640 1 -- 172.21.15.169:0/3844920184 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3b74157430 con 0x7f3b740b4a30 2023-10-26T01:35:05.707 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.706+0000 7f3b62ffd640 1 -- 172.21.15.169:0/3844920184 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4922 (secure 0 0 0) 0x7f3b6807e350 con 0x7f3b740b4a30 2023-10-26T01:35:05.708 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:35:05.708 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4976,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/2977083370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2977083370},{"type":"v1","addr":"172.21.15.170:6837","nonce":2977083370}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":5176,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/1598214042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1598214042},{"type":"v1","addr":"172.21.15.170:6835","nonce":1598214042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":6087,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/406582181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":406582181},{"type":"v1","addr":"172.21.15.169:6833","nonce":406582181}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T01:33:40.717881+0000","modified":"2023-10-26T01:33:43.796080+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6093},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6093":{"gid":6093,"name":"a","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6835/2338833513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":2338833513},{"type":"v1","addr":"172.21.15.169:6835","nonce":2338833513}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-10-26T01:35:05.708 INFO:teuthology.orchestra.run.smithi169.stderr:dumped fsmap epoch 32 2023-10-26T01:35:05.710 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.708+0000 7f3b7359e640 1 -- 172.21.15.169:0/3844920184 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3b5406ae40 msgr2=0x7f3b5406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:05.710 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.708+0000 7f3b7359e640 1 --2- 172.21.15.169:0/3844920184 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3b5406ae40 0x7f3b5406d300 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f3b640058c0 tx=0x7f3b64005850 comp rx=0 tx=0).stop 2023-10-26T01:35:05.711 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.708+0000 7f3b7359e640 1 -- 172.21.15.169:0/3844920184 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b740b4a30 msgr2=0x7f3b74155680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:05.711 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.708+0000 7f3b7359e640 1 --2- 172.21.15.169:0/3844920184 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b740b4a30 0x7f3b74155680 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f3b68036040 tx=0x7f3b6802dad0 comp rx=0 tx=0).stop 2023-10-26T01:35:05.711 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.708+0000 7f3b7359e640 1 -- 172.21.15.169:0/3844920184 shutdown_connections 2023-10-26T01:35:05.711 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.708+0000 7f3b7359e640 1 --2- 172.21.15.169:0/3844920184 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3b5406ae40 0x7f3b5406d300 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:05.711 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.708+0000 7f3b7359e640 1 --2- 172.21.15.169:0/3844920184 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b740b4a30 0x7f3b74155680 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:05.712 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.708+0000 7f3b7359e640 1 --2- 172.21.15.169:0/3844920184 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3b740b68c0 0x7f3b741586b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:05.712 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.708+0000 7f3b7359e640 1 --2- 172.21.15.169:0/3844920184 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3b740b45d0 0x7f3b741f1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:05.713 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.708+0000 7f3b7359e640 1 -- 172.21.15.169:0/3844920184 >> 172.21.15.169:0/3844920184 conn(0x7f3b740d1360 msgr2=0x7f3b740c8430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:35:05.713 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.709+0000 7f3b7359e640 1 -- 172.21.15.169:0/3844920184 shutdown_connections 2023-10-26T01:35:05.713 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.709+0000 7f3b7359e640 1 -- 172.21.15.169:0/3844920184 wait complete. 2023-10-26T01:35:05.729 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:35:05.946 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.944+0000 7fd848875640 1 -- 172.21.15.169:0/3687386423 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd8300047c0 msgr2=0x7fd830004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:05.946 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.944+0000 7fd848875640 1 --2- 172.21.15.169:0/3687386423 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd8300047c0 0x7fd830004c40 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fd82c005fd0 tx=0x7fd8300057f0 comp rx=0 tx=0).stop 2023-10-26T01:35:05.947 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.944+0000 7fd848875640 1 -- 172.21.15.169:0/3687386423 shutdown_connections 2023-10-26T01:35:05.947 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.944+0000 7fd848875640 1 --2- 172.21.15.169:0/3687386423 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd8300047c0 0x7fd830004c40 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:05.947 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.944+0000 7fd848875640 1 -- 172.21.15.169:0/3687386423 >> 172.21.15.169:0/3687386423 conn(0x7fd844069930 msgr2=0x7fd844069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:35:05.947 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.944+0000 7fd848875640 1 -- 172.21.15.169:0/3687386423 shutdown_connections 2023-10-26T01:35:05.947 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.944+0000 7fd848875640 1 -- 172.21.15.169:0/3687386423 wait complete. 2023-10-26T01:35:05.948 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.946+0000 7fd848875640 1 Processor -- start 2023-10-26T01:35:05.948 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.946+0000 7fd848875640 1 -- start start 2023-10-26T01:35:05.951 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.947+0000 7fd848875640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd8300047c0 0x7fd844151760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:05.951 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.947+0000 7fd848875640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd8440b0bd0 0x7fd8440b1050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:05.951 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.947+0000 7fd848875640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd8440c8bb0 0x7fd8440c9030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:05.952 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.947+0000 7fd848875640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8440c9570 con 0x7fd8440b0bd0 2023-10-26T01:35:05.952 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.947+0000 7fd848875640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8440b1720 con 0x7fd8440c8bb0 2023-10-26T01:35:05.952 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.947+0000 7fd848875640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd84406ed30 con 0x7fd8300047c0 2023-10-26T01:35:05.952 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.947+0000 7fd839d9b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd8440b0bd0 0x7fd8440b1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:05.952 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.947+0000 7fd839d9b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd8440b0bd0 0x7fd8440b1050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:48508/0 (socket says 172.21.15.169:48508) 2023-10-26T01:35:05.953 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.947+0000 7fd839d9b640 1 -- 172.21.15.169:0/399614051 learned_addr learned my addr 172.21.15.169:0/399614051 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:35:05.953 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.948+0000 7fd839d9b640 1 -- 172.21.15.169:0/399614051 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd8300047c0 msgr2=0x7fd844151760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:05.953 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.948+0000 7fd84259c640 1 --2- 172.21.15.169:0/399614051 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd8300047c0 0x7fd844151760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:05.953 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.948+0000 7fd842d9d640 1 --2- 172.21.15.169:0/399614051 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd8440c8bb0 0x7fd8440c9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:05.954 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.948+0000 7fd839d9b640 1 --2- 172.21.15.169:0/399614051 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd8300047c0 0x7fd844151760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:05.954 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.948+0000 7fd839d9b640 1 -- 172.21.15.169:0/399614051 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd8440c8bb0 msgr2=0x7fd8440c9030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:05.954 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.948+0000 7fd839d9b640 1 --2- 172.21.15.169:0/399614051 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd8440c8bb0 0x7fd8440c9030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:05.954 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.948+0000 7fd839d9b640 1 -- 172.21.15.169:0/399614051 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd84406f010 con 0x7fd8440b0bd0 2023-10-26T01:35:05.954 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.948+0000 7fd842d9d640 1 --2- 172.21.15.169:0/399614051 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd8440c8bb0 0x7fd8440c9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:35:05.955 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.948+0000 7fd839d9b640 1 --2- 172.21.15.169:0/399614051 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd8440b0bd0 0x7fd8440b1050 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fd82c02ca30 tx=0x7fd82c033d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:35:05.955 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.948+0000 7fd83b7fe640 1 -- 172.21.15.169:0/399614051 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd82c03d820 con 0x7fd8440b0bd0 2023-10-26T01:35:05.955 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.948+0000 7fd83b7fe640 1 -- 172.21.15.169:0/399614051 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd82c03d980 con 0x7fd8440b0bd0 2023-10-26T01:35:05.955 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.948+0000 7fd848875640 1 -- 172.21.15.169:0/399614051 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd84406f2f0 con 0x7fd8440b0bd0 2023-10-26T01:35:05.955 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.948+0000 7fd83b7fe640 1 -- 172.21.15.169:0/399614051 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd82c03dcf0 con 0x7fd8440b0bd0 2023-10-26T01:35:05.956 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.949+0000 7fd848875640 1 -- 172.21.15.169:0/399614051 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8440c7590 con 0x7fd8440b0bd0 2023-10-26T01:35:05.957 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.950+0000 7fd848875640 1 -- 172.21.15.169:0/399614051 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd84404db10 con 0x7fd8440b0bd0 2023-10-26T01:35:05.957 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.951+0000 7fd83b7fe640 1 -- 172.21.15.169:0/399614051 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fd82c04e3f0 con 0x7fd8440b0bd0 2023-10-26T01:35:05.957 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.951+0000 7fd83b7fe640 1 --2- 172.21.15.169:0/399614051 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd82406ae40 0x7fd82406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:05.957 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.952+0000 7fd83b7fe640 1 -- 172.21.15.169:0/399614051 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd82c0ac5b0 con 0x7fd8440b0bd0 2023-10-26T01:35:05.958 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.952+0000 7fd83b7fe640 1 -- 172.21.15.169:0/399614051 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=72}) v3 -- 0x7fd824072970 con 0x7fd8440b0bd0 2023-10-26T01:35:05.958 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.952+0000 7fd84259c640 1 --2- 172.21.15.169:0/399614051 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd82406ae40 0x7fd82406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:05.958 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.953+0000 7fd84259c640 1 --2- 172.21.15.169:0/399614051 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd82406ae40 0x7fd82406d300 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fd8440b24c0 tx=0x7fd834009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:35:05.958 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:05.956+0000 7fd83b7fe640 1 -- 172.21.15.169:0/399614051 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd82c0aae60 con 0x7fd8440b0bd0 2023-10-26T01:35:06.148 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.146+0000 7fd848875640 1 -- 172.21.15.169:0/399614051 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd844157ed0 con 0x7fd8440b0bd0 2023-10-26T01:35:06.150 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.148+0000 7fd83b7fe640 1 -- 172.21.15.169:0/399614051 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) v1 ==== 74+0+16635 (secure 0 0 0) 0x7fd82c077650 con 0x7fd8440b0bd0 2023-10-26T01:35:06.150 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:35:06.151 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":71,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:35:04.969603+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-10-26T01:33:38.593237+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-10-26T01:33:39.753692+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","full","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:35:06.153 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.151+0000 7fd848875640 1 -- 172.21.15.169:0/399614051 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd82406ae40 msgr2=0x7fd82406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:06.153 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.151+0000 7fd848875640 1 --2- 172.21.15.169:0/399614051 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd82406ae40 0x7fd82406d300 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fd8440b24c0 tx=0x7fd834009040 comp rx=0 tx=0).stop 2023-10-26T01:35:06.153 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.151+0000 7fd848875640 1 -- 172.21.15.169:0/399614051 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd8440b0bd0 msgr2=0x7fd8440b1050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:06.154 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.151+0000 7fd848875640 1 --2- 172.21.15.169:0/399614051 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd8440b0bd0 0x7fd8440b1050 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fd82c02ca30 tx=0x7fd82c033d50 comp rx=0 tx=0).stop 2023-10-26T01:35:06.154 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.151+0000 7fd848875640 1 -- 172.21.15.169:0/399614051 shutdown_connections 2023-10-26T01:35:06.154 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.151+0000 7fd848875640 1 --2- 172.21.15.169:0/399614051 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd82406ae40 0x7fd82406d300 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:06.154 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.151+0000 7fd848875640 1 --2- 172.21.15.169:0/399614051 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd8440b0bd0 0x7fd8440b1050 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:06.154 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.151+0000 7fd848875640 1 --2- 172.21.15.169:0/399614051 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd8440c8bb0 0x7fd8440c9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:06.155 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.151+0000 7fd848875640 1 --2- 172.21.15.169:0/399614051 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd8300047c0 0x7fd844151760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:06.155 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.151+0000 7fd848875640 1 -- 172.21.15.169:0/399614051 >> 172.21.15.169:0/399614051 conn(0x7fd844069930 msgr2=0x7fd844153ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:35:06.155 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.151+0000 7fd848875640 1 -- 172.21.15.169:0/399614051 shutdown_connections 2023-10-26T01:35:06.155 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.152+0000 7fd848875640 1 -- 172.21.15.169:0/399614051 wait complete. 2023-10-26T01:35:06.172 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-26T01:35:06.172 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T01:35:06.380 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.378+0000 7f22ab7ce640 1 -- 172.21.15.169:0/199519349 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22a41592d0 msgr2=0x7f22a40aa3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:06.380 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.378+0000 7f22ab7ce640 1 --2- 172.21.15.169:0/199519349 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22a41592d0 0x7f22a40aa3d0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f22a0009830 tx=0x7f22a002bcd0 comp rx=0 tx=0).stop 2023-10-26T01:35:06.380 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.378+0000 7f22ab7ce640 1 -- 172.21.15.169:0/199519349 shutdown_connections 2023-10-26T01:35:06.381 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.378+0000 7f22ab7ce640 1 --2- 172.21.15.169:0/199519349 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22a41592d0 0x7f22a40aa3d0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:06.381 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.378+0000 7f22ab7ce640 1 -- 172.21.15.169:0/199519349 >> 172.21.15.169:0/199519349 conn(0x7f22a4069930 msgr2=0x7f22a4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:35:06.381 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.378+0000 7f22ab7ce640 1 -- 172.21.15.169:0/199519349 shutdown_connections 2023-10-26T01:35:06.381 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.378+0000 7f22ab7ce640 1 -- 172.21.15.169:0/199519349 wait complete. 2023-10-26T01:35:06.382 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.380+0000 7f22ab7ce640 1 Processor -- start 2023-10-26T01:35:06.382 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.380+0000 7f22ab7ce640 1 -- start start 2023-10-26T01:35:06.383 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.381+0000 7f22ab7ce640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f22a40ac930 0x7f22a40c8f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:06.384 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.381+0000 7f22ab7ce640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22a40cde00 0x7f22a40b1540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:06.384 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.381+0000 7f22ab7ce640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f22a41592d0 0x7f22a40c75a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:06.385 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.381+0000 7f22ab7ce640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22a40c7ef0 con 0x7f22a40cde00 2023-10-26T01:35:06.385 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.381+0000 7f22ab7ce640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22a40c81d0 con 0x7f22a40ac930 2023-10-26T01:35:06.385 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.381+0000 7f22ab7ce640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22a40c84b0 con 0x7f22a41592d0 2023-10-26T01:35:06.386 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.381+0000 7f22a8d69640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22a40cde00 0x7f22a40b1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:06.386 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.381+0000 7f22a8d69640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22a40cde00 0x7f22a40b1540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:48522/0 (socket says 172.21.15.169:48522) 2023-10-26T01:35:06.386 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.381+0000 7f22a8d69640 1 -- 172.21.15.169:0/1531517184 learned_addr learned my addr 172.21.15.169:0/1531517184 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:35:06.386 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.382+0000 7f22a956a640 1 --2- 172.21.15.169:0/1531517184 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f22a40ac930 0x7f22a40c8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:06.386 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.382+0000 7f22a8d69640 1 -- 172.21.15.169:0/1531517184 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f22a41592d0 msgr2=0x7f22a40c75a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:06.387 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.382+0000 7f22a8d69640 1 --2- 172.21.15.169:0/1531517184 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f22a41592d0 0x7f22a40c75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:06.387 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.382+0000 7f22a8d69640 1 -- 172.21.15.169:0/1531517184 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f22a40ac930 msgr2=0x7f22a40c8f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:06.387 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.382+0000 7f22a8d69640 1 --2- 172.21.15.169:0/1531517184 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f22a40ac930 0x7f22a40c8f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:06.387 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.382+0000 7f22a8d69640 1 -- 172.21.15.169:0/1531517184 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22a0009490 con 0x7f22a40cde00 2023-10-26T01:35:06.387 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.382+0000 7f22a956a640 1 --2- 172.21.15.169:0/1531517184 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f22a40ac930 0x7f22a40c8f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:35:06.388 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.382+0000 7f22a8d69640 1 --2- 172.21.15.169:0/1531517184 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22a40cde00 0x7f22a40b1540 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f229400c4b0 tx=0x7f229400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:35:06.389 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.382+0000 7f2299ffb640 1 -- 172.21.15.169:0/1531517184 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2294005480 con 0x7f22a40cde00 2023-10-26T01:35:06.389 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.382+0000 7f2299ffb640 1 -- 172.21.15.169:0/1531517184 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f22940036e0 con 0x7f22a40cde00 2023-10-26T01:35:06.389 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.382+0000 7f22ab7ce640 1 -- 172.21.15.169:0/1531517184 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22a40c8790 con 0x7f22a40cde00 2023-10-26T01:35:06.390 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.382+0000 7f2299ffb640 1 -- 172.21.15.169:0/1531517184 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2294003b30 con 0x7f22a40cde00 2023-10-26T01:35:06.390 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.383+0000 7f22ab7ce640 1 -- 172.21.15.169:0/1531517184 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22a40b1a80 con 0x7f22a40cde00 2023-10-26T01:35:06.390 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.384+0000 7f22ab7ce640 1 -- 172.21.15.169:0/1531517184 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22a4072e40 con 0x7f22a40cde00 2023-10-26T01:35:06.390 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.385+0000 7f2299ffb640 1 -- 172.21.15.169:0/1531517184 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f2294003c90 con 0x7f22a40cde00 2023-10-26T01:35:06.390 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.385+0000 7f2299ffb640 1 --2- 172.21.15.169:0/1531517184 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f228806ae40 0x7f228806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:06.391 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.386+0000 7f2299ffb640 1 -- 172.21.15.169:0/1531517184 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2294088e20 con 0x7f22a40cde00 2023-10-26T01:35:06.391 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.386+0000 7f2299ffb640 1 -- 172.21.15.169:0/1531517184 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=72}) v3 -- 0x7f2288072970 con 0x7f22a40cde00 2023-10-26T01:35:06.391 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.386+0000 7f22a956a640 1 --2- 172.21.15.169:0/1531517184 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f228806ae40 0x7f228806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:06.391 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.387+0000 7f22a956a640 1 --2- 172.21.15.169:0/1531517184 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f228806ae40 0x7f228806d300 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f22a0036000 tx=0x7f22a0004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:35:06.392 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.390+0000 7f2299ffb640 1 -- 172.21.15.169:0/1531517184 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2294017070 con 0x7f22a40cde00 2023-10-26T01:35:06.620 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.618+0000 7f22ab7ce640 1 -- 172.21.15.169:0/1531517184 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f22a404db10 con 0x7f22a40cde00 2023-10-26T01:35:06.621 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.619+0000 7f2299ffb640 1 -- 172.21.15.169:0/1531517184 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4922 (secure 0 0 0) 0x7f2294053ed0 con 0x7f22a40cde00 2023-10-26T01:35:06.621 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:35:06.622 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4976,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/2977083370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2977083370},{"type":"v1","addr":"172.21.15.170:6837","nonce":2977083370}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":5176,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/1598214042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1598214042},{"type":"v1","addr":"172.21.15.170:6835","nonce":1598214042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":6087,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/406582181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":406582181},{"type":"v1","addr":"172.21.15.169:6833","nonce":406582181}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T01:33:40.717881+0000","modified":"2023-10-26T01:33:43.796080+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6093},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6093":{"gid":6093,"name":"a","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6835/2338833513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":2338833513},{"type":"v1","addr":"172.21.15.169:6835","nonce":2338833513}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-10-26T01:35:06.622 INFO:teuthology.orchestra.run.smithi169.stderr:dumped fsmap epoch 32 2023-10-26T01:35:06.624 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.622+0000 7f22ab7ce640 1 -- 172.21.15.169:0/1531517184 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f228806ae40 msgr2=0x7f228806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:06.624 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.622+0000 7f22ab7ce640 1 --2- 172.21.15.169:0/1531517184 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f228806ae40 0x7f228806d300 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f22a0036000 tx=0x7f22a0004730 comp rx=0 tx=0).stop 2023-10-26T01:35:06.624 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.622+0000 7f22ab7ce640 1 -- 172.21.15.169:0/1531517184 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22a40cde00 msgr2=0x7f22a40b1540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:06.624 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.622+0000 7f22ab7ce640 1 --2- 172.21.15.169:0/1531517184 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22a40cde00 0x7f22a40b1540 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f229400c4b0 tx=0x7f229400c970 comp rx=0 tx=0).stop 2023-10-26T01:35:06.625 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.622+0000 7f22ab7ce640 1 -- 172.21.15.169:0/1531517184 shutdown_connections 2023-10-26T01:35:06.625 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.622+0000 7f22ab7ce640 1 --2- 172.21.15.169:0/1531517184 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f228806ae40 0x7f228806d300 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:06.625 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.622+0000 7f22ab7ce640 1 --2- 172.21.15.169:0/1531517184 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22a40cde00 0x7f22a40b1540 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:06.625 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.622+0000 7f22ab7ce640 1 --2- 172.21.15.169:0/1531517184 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f22a41592d0 0x7f22a40c75a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:06.625 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.622+0000 7f22ab7ce640 1 --2- 172.21.15.169:0/1531517184 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f22a40ac930 0x7f22a40c8f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:06.626 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.622+0000 7f22ab7ce640 1 -- 172.21.15.169:0/1531517184 >> 172.21.15.169:0/1531517184 conn(0x7f22a4069930 msgr2=0x7f22a40ae3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:35:06.626 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.622+0000 7f22ab7ce640 1 -- 172.21.15.169:0/1531517184 shutdown_connections 2023-10-26T01:35:06.626 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.622+0000 7f22ab7ce640 1 -- 172.21.15.169:0/1531517184 wait complete. 2023-10-26T01:35:06.641 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T01:35:06.847 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.845+0000 7f4e9b59e640 1 -- 172.21.15.169:0/2781187311 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e9c0c9c30 msgr2=0x7f4e9c0cb390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:06.848 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.845+0000 7f4e9b59e640 1 --2- 172.21.15.169:0/2781187311 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e9c0c9c30 0x7f4e9c0cb390 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c0ca610 tx=0x7f4e9402c1c0 comp rx=0 tx=0).stop 2023-10-26T01:35:06.848 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.845+0000 7f4e9b59e640 1 -- 172.21.15.169:0/2781187311 shutdown_connections 2023-10-26T01:35:06.848 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.845+0000 7f4e9b59e640 1 --2- 172.21.15.169:0/2781187311 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e9c0c9c30 0x7f4e9c0cb390 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:06.848 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.845+0000 7f4e9b59e640 1 --2- 172.21.15.169:0/2781187311 >> v2:172.21.15.170:3300/0 conn(0x7f4e9c06a990 0x7f4e9c0c96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:06.849 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.845+0000 7f4e9b59e640 1 --2- 172.21.15.169:0/2781187311 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4e9c06cae0 0x7f4e9c0c7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:06.849 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.845+0000 7f4e9b59e640 1 -- 172.21.15.169:0/2781187311 >> 172.21.15.169:0/2781187311 conn(0x7f4e9c0aedd0 msgr2=0x7f4e9c0af1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:35:06.849 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.846+0000 7f4e9b59e640 1 -- 172.21.15.169:0/2781187311 shutdown_connections 2023-10-26T01:35:06.849 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.846+0000 7f4e9b59e640 1 -- 172.21.15.169:0/2781187311 wait complete. 2023-10-26T01:35:06.851 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.847+0000 7f4e9b59e640 1 Processor -- start 2023-10-26T01:35:06.851 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.847+0000 7f4e9b59e640 1 -- start start 2023-10-26T01:35:06.851 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.848+0000 7f4e9b59e640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4e9c06a990 0x7f4e9c149880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:06.851 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.848+0000 7f4e9b59e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e9c06cae0 0x7f4e9c14f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:06.851 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.848+0000 7f4e9b59e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4e9c0c9c30 0x7f4e9c14fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:06.852 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.848+0000 7f4e9b59e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9c0ac000 con 0x7f4e9c06cae0 2023-10-26T01:35:06.852 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.848+0000 7f4e9b59e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9c0ab3f0 con 0x7f4e9c0c9c30 2023-10-26T01:35:06.852 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.848+0000 7f4e9b59e640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9c0ab6d0 con 0x7f4e9c06a990 2023-10-26T01:35:06.852 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.849+0000 7f4e93fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e9c06cae0 0x7f4e9c14f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:06.852 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.849+0000 7f4e93fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e9c06cae0 0x7f4e9c14f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:48538/0 (socket says 172.21.15.169:48538) 2023-10-26T01:35:06.853 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.849+0000 7f4e93fff640 1 -- 172.21.15.169:0/1458702208 learned_addr learned my addr 172.21.15.169:0/1458702208 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:35:06.853 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.849+0000 7f4e9a59c640 1 --2- 172.21.15.169:0/1458702208 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4e9c06a990 0x7f4e9c149880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:06.853 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.849+0000 7f4e93fff640 1 -- 172.21.15.169:0/1458702208 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4e9c06a990 msgr2=0x7f4e9c149880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:06.853 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.849+0000 7f4e93fff640 1 --2- 172.21.15.169:0/1458702208 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4e9c06a990 0x7f4e9c149880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:06.853 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.849+0000 7f4e93fff640 1 -- 172.21.15.169:0/1458702208 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4e9c0c9c30 msgr2=0x7f4e9c14fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:06.853 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.849+0000 7f4e9ad9d640 1 --2- 172.21.15.169:0/1458702208 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4e9c0c9c30 0x7f4e9c14fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:06.854 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.849+0000 7f4e93fff640 1 --2- 172.21.15.169:0/1458702208 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4e9c0c9c30 0x7f4e9c14fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:06.854 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.849+0000 7f4e93fff640 1 -- 172.21.15.169:0/1458702208 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e94009900 con 0x7f4e9c06cae0 2023-10-26T01:35:06.854 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.849+0000 7f4e9ad9d640 1 --2- 172.21.15.169:0/1458702208 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4e9c0c9c30 0x7f4e9c14fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:35:06.854 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.849+0000 7f4e93fff640 1 --2- 172.21.15.169:0/1458702208 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e9c06cae0 0x7f4e9c14f7f0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f4e8400c920 tx=0x7f4e8400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:35:06.856 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.850+0000 7f4e92ffd640 1 -- 172.21.15.169:0/1458702208 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e84007c70 con 0x7f4e9c06cae0 2023-10-26T01:35:06.856 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.850+0000 7f4e92ffd640 1 -- 172.21.15.169:0/1458702208 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4e8400f450 con 0x7f4e9c06cae0 2023-10-26T01:35:06.856 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.850+0000 7f4e92ffd640 1 -- 172.21.15.169:0/1458702208 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e8400f860 con 0x7f4e9c06cae0 2023-10-26T01:35:06.856 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.850+0000 7f4e9b59e640 1 -- 172.21.15.169:0/1458702208 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e9c0ab9b0 con 0x7f4e9c06cae0 2023-10-26T01:35:06.856 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.850+0000 7f4e9b59e640 1 -- 172.21.15.169:0/1458702208 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e9c149dc0 con 0x7f4e9c06cae0 2023-10-26T01:35:06.857 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.851+0000 7f4e9b59e640 1 -- 172.21.15.169:0/1458702208 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e9c073c10 con 0x7f4e9c06cae0 2023-10-26T01:35:06.857 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.852+0000 7f4e92ffd640 1 -- 172.21.15.169:0/1458702208 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f4e8400f9c0 con 0x7f4e9c06cae0 2023-10-26T01:35:06.857 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.853+0000 7f4e92ffd640 1 --2- 172.21.15.169:0/1458702208 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f4e7c06ae40 0x7f4e7c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:06.857 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.853+0000 7f4e92ffd640 1 -- 172.21.15.169:0/1458702208 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4e84089f50 con 0x7f4e9c06cae0 2023-10-26T01:35:06.857 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.854+0000 7f4e92ffd640 1 -- 172.21.15.169:0/1458702208 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=72}) v3 -- 0x7f4e7c072970 con 0x7f4e9c06cae0 2023-10-26T01:35:06.858 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.854+0000 7f4e9a59c640 1 --2- 172.21.15.169:0/1458702208 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f4e7c06ae40 0x7f4e7c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:06.858 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.855+0000 7f4e9a59c640 1 --2- 172.21.15.169:0/1458702208 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f4e7c06ae40 0x7f4e7c06d300 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c14d820 tx=0x7f4e8c008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:35:06.859 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:06.858+0000 7f4e92ffd640 1 -- 172.21.15.169:0/1458702208 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4e8401d060 con 0x7f4e9c06cae0 2023-10-26T01:35:07.076 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.074+0000 7f4e9b59e640 1 -- 172.21.15.169:0/1458702208 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4e9c14a520 con 0x7f4e9c06cae0 2023-10-26T01:35:07.077 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.076+0000 7f4e92ffd640 1 -- 172.21.15.169:0/1458702208 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4922 (secure 0 0 0) 0x7f4e84054f60 con 0x7f4e9c06cae0 2023-10-26T01:35:07.078 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:35:07.079 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4976,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/2977083370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2977083370},{"type":"v1","addr":"172.21.15.170:6837","nonce":2977083370}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":5176,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/1598214042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1598214042},{"type":"v1","addr":"172.21.15.170:6835","nonce":1598214042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":6087,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/406582181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":406582181},{"type":"v1","addr":"172.21.15.169:6833","nonce":406582181}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T01:33:40.717881+0000","modified":"2023-10-26T01:33:43.796080+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6093},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6093":{"gid":6093,"name":"a","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6835/2338833513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":2338833513},{"type":"v1","addr":"172.21.15.169:6835","nonce":2338833513}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-10-26T01:35:07.079 INFO:teuthology.orchestra.run.smithi169.stderr:dumped fsmap epoch 32 2023-10-26T01:35:07.081 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.078+0000 7f4e9b59e640 1 -- 172.21.15.169:0/1458702208 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f4e7c06ae40 msgr2=0x7f4e7c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:07.082 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.078+0000 7f4e9b59e640 1 --2- 172.21.15.169:0/1458702208 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f4e7c06ae40 0x7f4e7c06d300 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c14d820 tx=0x7f4e8c008040 comp rx=0 tx=0).stop 2023-10-26T01:35:07.082 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.078+0000 7f4e9b59e640 1 -- 172.21.15.169:0/1458702208 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e9c06cae0 msgr2=0x7f4e9c14f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:07.083 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.078+0000 7f4e9b59e640 1 --2- 172.21.15.169:0/1458702208 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e9c06cae0 0x7f4e9c14f7f0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f4e8400c920 tx=0x7f4e8400cde0 comp rx=0 tx=0).stop 2023-10-26T01:35:07.083 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.078+0000 7f4e9b59e640 1 -- 172.21.15.169:0/1458702208 shutdown_connections 2023-10-26T01:35:07.084 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.078+0000 7f4e9b59e640 1 --2- 172.21.15.169:0/1458702208 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f4e7c06ae40 0x7f4e7c06d300 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:07.084 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.078+0000 7f4e9b59e640 1 --2- 172.21.15.169:0/1458702208 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e9c06cae0 0x7f4e9c14f7f0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:07.084 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.078+0000 7f4e9b59e640 1 --2- 172.21.15.169:0/1458702208 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4e9c0c9c30 0x7f4e9c14fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:07.085 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.078+0000 7f4e9b59e640 1 --2- 172.21.15.169:0/1458702208 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4e9c06a990 0x7f4e9c149880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:07.085 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.078+0000 7f4e9b59e640 1 -- 172.21.15.169:0/1458702208 >> 172.21.15.169:0/1458702208 conn(0x7f4e9c0aedd0 msgr2=0x7f4e9c156a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:35:07.085 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.078+0000 7f4e9b59e640 1 -- 172.21.15.169:0/1458702208 shutdown_connections 2023-10-26T01:35:07.086 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.079+0000 7f4e9b59e640 1 -- 172.21.15.169:0/1458702208 wait complete. 2023-10-26T01:35:07.098 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:35:07.299 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.297+0000 7f6c85725640 1 -- 172.21.15.169:0/1052061486 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c800ac940 msgr2=0x7f6c800cdbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:07.300 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.297+0000 7f6c85725640 1 --2- 172.21.15.169:0/1052061486 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c800ac940 0x7f6c800cdbb0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f6c8006bff0 tx=0x7f6c7802c260 comp rx=0 tx=0).stop 2023-10-26T01:35:07.300 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.297+0000 7f6c85725640 1 -- 172.21.15.169:0/1052061486 shutdown_connections 2023-10-26T01:35:07.300 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.297+0000 7f6c85725640 1 --2- 172.21.15.169:0/1052061486 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c800ac940 0x7f6c800cdbb0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:07.300 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.297+0000 7f6c85725640 1 --2- 172.21.15.169:0/1052061486 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6c800ac4e0 0x7f6c800cd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:07.301 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.297+0000 7f6c85725640 1 --2- 172.21.15.169:0/1052061486 >> v2:172.21.15.170:3300/0 conn(0x7f6c8015b610 0x7f6c8006ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:07.301 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.297+0000 7f6c85725640 1 -- 172.21.15.169:0/1052061486 >> 172.21.15.169:0/1052061486 conn(0x7f6c80069930 msgr2=0x7f6c80069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:35:07.301 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.298+0000 7f6c85725640 1 -- 172.21.15.169:0/1052061486 shutdown_connections 2023-10-26T01:35:07.301 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.298+0000 7f6c85725640 1 -- 172.21.15.169:0/1052061486 wait complete. 2023-10-26T01:35:07.303 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.299+0000 7f6c85725640 1 Processor -- start 2023-10-26T01:35:07.303 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.299+0000 7f6c85725640 1 -- start start 2023-10-26T01:35:07.304 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.300+0000 7f6c85725640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6c800ac4e0 0x7f6c8014f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:07.304 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.300+0000 7f6c85725640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6c800ac940 0x7f6c800ada50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:07.304 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.300+0000 7f6c85725640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c8015b610 0x7f6c800adf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:07.304 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.300+0000 7f6c85725640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c8015ea00 con 0x7f6c8015b610 2023-10-26T01:35:07.304 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.300+0000 7f6c85725640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c8015e570 con 0x7f6c800ac940 2023-10-26T01:35:07.305 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.300+0000 7f6c85725640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c801f6290 con 0x7f6c800ac4e0 2023-10-26T01:35:07.305 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.300+0000 7f6c7f7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c8015b610 0x7f6c800adf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:07.305 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.300+0000 7f6c7f7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c8015b610 0x7f6c800adf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:48554/0 (socket says 172.21.15.169:48554) 2023-10-26T01:35:07.305 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.300+0000 7f6c7f7fe640 1 -- 172.21.15.169:0/639021429 learned_addr learned my addr 172.21.15.169:0/639021429 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:35:07.305 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.301+0000 7f6c7effd640 1 --2- 172.21.15.169:0/639021429 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6c800ac4e0 0x7f6c8014f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:07.306 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.301+0000 7f6c7f7fe640 1 -- 172.21.15.169:0/639021429 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6c800ac4e0 msgr2=0x7f6c8014f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:07.306 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.301+0000 7f6c7f7fe640 1 --2- 172.21.15.169:0/639021429 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6c800ac4e0 0x7f6c8014f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:07.306 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.301+0000 7f6c7f7fe640 1 -- 172.21.15.169:0/639021429 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6c800ac940 msgr2=0x7f6c800ada50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:07.306 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.301+0000 7f6c7f7fe640 1 --2- 172.21.15.169:0/639021429 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6c800ac940 0x7f6c800ada50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:07.306 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.301+0000 7f6c7f7fe640 1 -- 172.21.15.169:0/639021429 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c78009900 con 0x7f6c8015b610 2023-10-26T01:35:07.307 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.301+0000 7f6c7f7fe640 1 --2- 172.21.15.169:0/639021429 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c8015b610 0x7f6c800adf90 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f6c78037040 tx=0x7f6c78003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:35:07.307 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.301+0000 7f6c77fff640 1 -- 172.21.15.169:0/639021429 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c78004240 con 0x7f6c8015b610 2023-10-26T01:35:07.307 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.301+0000 7f6c77fff640 1 -- 172.21.15.169:0/639021429 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6c78004490 con 0x7f6c8015b610 2023-10-26T01:35:07.307 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.301+0000 7f6c77fff640 1 -- 172.21.15.169:0/639021429 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c78004880 con 0x7f6c8015b610 2023-10-26T01:35:07.308 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.301+0000 7f6c85725640 1 -- 172.21.15.169:0/639021429 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c8015dc40 con 0x7f6c8015b610 2023-10-26T01:35:07.308 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.302+0000 7f6c85725640 1 -- 172.21.15.169:0/639021429 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c8015e200 con 0x7f6c8015b610 2023-10-26T01:35:07.309 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.303+0000 7f6c85725640 1 -- 172.21.15.169:0/639021429 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c8004db10 con 0x7f6c8015b610 2023-10-26T01:35:07.309 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.304+0000 7f6c77fff640 1 -- 172.21.15.169:0/639021429 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f6c7803b040 con 0x7f6c8015b610 2023-10-26T01:35:07.309 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.305+0000 7f6c77fff640 1 --2- 172.21.15.169:0/639021429 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f6c6006ae40 0x7f6c6006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:07.310 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.305+0000 7f6c77fff640 1 -- 172.21.15.169:0/639021429 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6c780ac400 con 0x7f6c8015b610 2023-10-26T01:35:07.310 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.306+0000 7f6c7effd640 1 --2- 172.21.15.169:0/639021429 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f6c6006ae40 0x7f6c6006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:07.310 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.307+0000 7f6c7effd640 1 --2- 172.21.15.169:0/639021429 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f6c6006ae40 0x7f6c6006d300 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f6c700058c0 tx=0x7f6c70005be0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:35:07.311 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.309+0000 7f6c77fff640 1 -- 172.21.15.169:0/639021429 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6c780774b0 con 0x7f6c8015b610 2023-10-26T01:35:07.488 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.486+0000 7f6c85725640 1 -- 172.21.15.169:0/639021429 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6c8015e6b0 con 0x7f6c8015b610 2023-10-26T01:35:07.490 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.488+0000 7f6c77fff640 1 -- 172.21.15.169:0/639021429 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) v1 ==== 74+0+16613 (secure 0 0 0) 0x7f6c7807aad0 con 0x7f6c8015b610 2023-10-26T01:35:07.490 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:35:07.491 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":72,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:35:07.099169+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"72","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-10-26T01:33:38.593237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"72","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-10-26T01:33:39.753692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"72","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:35:07.493 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.491+0000 7f6c85725640 1 -- 172.21.15.169:0/639021429 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f6c6006ae40 msgr2=0x7f6c6006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:07.493 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.491+0000 7f6c85725640 1 --2- 172.21.15.169:0/639021429 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f6c6006ae40 0x7f6c6006d300 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f6c700058c0 tx=0x7f6c70005be0 comp rx=0 tx=0).stop 2023-10-26T01:35:07.493 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.491+0000 7f6c85725640 1 -- 172.21.15.169:0/639021429 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c8015b610 msgr2=0x7f6c800adf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:07.494 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.491+0000 7f6c85725640 1 --2- 172.21.15.169:0/639021429 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c8015b610 0x7f6c800adf90 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f6c78037040 tx=0x7f6c78003ba0 comp rx=0 tx=0).stop 2023-10-26T01:35:07.494 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.491+0000 7f6c85725640 1 -- 172.21.15.169:0/639021429 shutdown_connections 2023-10-26T01:35:07.494 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.491+0000 7f6c85725640 1 --2- 172.21.15.169:0/639021429 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f6c6006ae40 0x7f6c6006d300 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:07.494 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.491+0000 7f6c85725640 1 --2- 172.21.15.169:0/639021429 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c8015b610 0x7f6c800adf90 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:07.495 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.491+0000 7f6c85725640 1 --2- 172.21.15.169:0/639021429 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6c800ac940 0x7f6c800ada50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:07.495 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.491+0000 7f6c85725640 1 --2- 172.21.15.169:0/639021429 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6c800ac4e0 0x7f6c8014f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:07.495 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.491+0000 7f6c85725640 1 -- 172.21.15.169:0/639021429 >> 172.21.15.169:0/639021429 conn(0x7f6c80069930 msgr2=0x7f6c800cc980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:35:07.495 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.491+0000 7f6c85725640 1 -- 172.21.15.169:0/639021429 shutdown_connections 2023-10-26T01:35:07.496 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.491+0000 7f6c85725640 1 -- 172.21.15.169:0/639021429 wait complete. 2023-10-26T01:35:07.512 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-26T01:35:07.732 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.730+0000 7f24dd105640 1 -- 172.21.15.169:0/79739648 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24c4003970 msgr2=0x7f24c4003d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:07.732 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.730+0000 7f24dd105640 1 --2- 172.21.15.169:0/79739648 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24c4003970 0x7f24c4003d70 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f24c8006fc0 tx=0x7f24c40057f0 comp rx=0 tx=0).stop 2023-10-26T01:35:07.732 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.730+0000 7f24dd105640 1 -- 172.21.15.169:0/79739648 shutdown_connections 2023-10-26T01:35:07.733 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.730+0000 7f24dd105640 1 --2- 172.21.15.169:0/79739648 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24c4003970 0x7f24c4003d70 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:07.733 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.730+0000 7f24dd105640 1 -- 172.21.15.169:0/79739648 >> 172.21.15.169:0/79739648 conn(0x7f24d8069930 msgr2=0x7f24d8069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:35:07.733 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.730+0000 7f24dd105640 1 -- 172.21.15.169:0/79739648 shutdown_connections 2023-10-26T01:35:07.733 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.730+0000 7f24dd105640 1 -- 172.21.15.169:0/79739648 wait complete. 2023-10-26T01:35:07.734 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.732+0000 7f24dd105640 1 Processor -- start 2023-10-26T01:35:07.734 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.732+0000 7f24dd105640 1 -- start start 2023-10-26T01:35:07.736 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.733+0000 7f24dd105640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f24c4003970 0x7f24d806ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:07.736 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.733+0000 7f24dd105640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f24d80aeb30 0x7f24d814e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:07.736 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.733+0000 7f24dd105640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24d80cc480 0x7f24d80cc900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:07.737 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.733+0000 7f24dd105640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24d806bef0 con 0x7f24d80cc480 2023-10-26T01:35:07.737 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.733+0000 7f24dd105640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24d80cce40 con 0x7f24d80aeb30 2023-10-26T01:35:07.737 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.733+0000 7f24dd105640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24d806b370 con 0x7f24c4003970 2023-10-26T01:35:07.737 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.733+0000 7f24d759e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24d80cc480 0x7f24d80cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:07.737 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.733+0000 7f24d759e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24d80cc480 0x7f24d80cc900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:48570/0 (socket says 172.21.15.169:48570) 2023-10-26T01:35:07.738 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.733+0000 7f24d759e640 1 -- 172.21.15.169:0/4105224953 learned_addr learned my addr 172.21.15.169:0/4105224953 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:35:07.738 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.733+0000 7f24cffff640 1 --2- 172.21.15.169:0/4105224953 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f24d80aeb30 0x7f24d814e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:07.738 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.733+0000 7f24d6d9d640 1 --2- 172.21.15.169:0/4105224953 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f24c4003970 0x7f24d806ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:07.738 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.734+0000 7f24d759e640 1 -- 172.21.15.169:0/4105224953 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f24c4003970 msgr2=0x7f24d806ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:07.738 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.734+0000 7f24d759e640 1 --2- 172.21.15.169:0/4105224953 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f24c4003970 0x7f24d806ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:07.739 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.734+0000 7f24d759e640 1 -- 172.21.15.169:0/4105224953 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f24d80aeb30 msgr2=0x7f24d814e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:07.739 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.734+0000 7f24d759e640 1 --2- 172.21.15.169:0/4105224953 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f24d80aeb30 0x7f24d814e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:07.739 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.734+0000 7f24d759e640 1 -- 172.21.15.169:0/4105224953 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24d806b650 con 0x7f24d80cc480 2023-10-26T01:35:07.739 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.734+0000 7f24d6d9d640 1 --2- 172.21.15.169:0/4105224953 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f24c4003970 0x7f24d806ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:35:07.739 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.734+0000 7f24d759e640 1 --2- 172.21.15.169:0/4105224953 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24d80cc480 0x7f24d80cc900 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f24d80b35a0 tx=0x7f24d000d9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:35:07.740 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.734+0000 7f24cf7fe640 1 -- 172.21.15.169:0/4105224953 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24d0012630 con 0x7f24d80cc480 2023-10-26T01:35:07.740 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.734+0000 7f24cf7fe640 1 -- 172.21.15.169:0/4105224953 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f24d0012790 con 0x7f24d80cc480 2023-10-26T01:35:07.740 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.734+0000 7f24dd105640 1 -- 172.21.15.169:0/4105224953 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24d806b930 con 0x7f24d80cc480 2023-10-26T01:35:07.740 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.734+0000 7f24cf7fe640 1 -- 172.21.15.169:0/4105224953 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24d0012ba0 con 0x7f24d80cc480 2023-10-26T01:35:07.740 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.734+0000 7f24dd105640 1 -- 172.21.15.169:0/4105224953 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24d80aefb0 con 0x7f24d80cc480 2023-10-26T01:35:07.741 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.736+0000 7f24dd105640 1 -- 172.21.15.169:0/4105224953 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24d804db10 con 0x7f24d80cc480 2023-10-26T01:35:07.742 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.736+0000 7f24cf7fe640 1 -- 172.21.15.169:0/4105224953 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f24d001d460 con 0x7f24d80cc480 2023-10-26T01:35:07.742 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.737+0000 7f24cf7fe640 1 --2- 172.21.15.169:0/4105224953 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f24b806ae40 0x7f24b806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:07.742 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.737+0000 7f24cf7fe640 1 -- 172.21.15.169:0/4105224953 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f24d00891c0 con 0x7f24d80cc480 2023-10-26T01:35:07.742 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.738+0000 7f24d6d9d640 1 --2- 172.21.15.169:0/4105224953 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f24b806ae40 0x7f24b806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:07.742 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.739+0000 7f24d6d9d640 1 --2- 172.21.15.169:0/4105224953 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f24b806ae40 0x7f24b806d300 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f24c8034000 tx=0x7f24c802e000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:35:07.743 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.741+0000 7f24cf7fe640 1 -- 172.21.15.169:0/4105224953 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f24d0054260 con 0x7f24d80cc480 2023-10-26T01:35:07.969 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.968+0000 7f24dd105640 1 -- 172.21.15.169:0/4105224953 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f24d80b2170 con 0x7f24d80cc480 2023-10-26T01:35:07.974 INFO:tasks.ceph.mon.a.smithi169.stderr:2023-10-26T01:35:07.973+0000 7f1588a74640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-26T01:35:07.978 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.976+0000 7f24cf7fe640 1 -- 172.21.15.169:0/4105224953 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v33) v1 ==== 161+0+0 (secure 0 0 0) 0x7f24d0057880 con 0x7f24d80cc480 2023-10-26T01:35:07.978 INFO:teuthology.orchestra.run.smithi169.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-26T01:35:07.982 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.980+0000 7f24dd105640 1 -- 172.21.15.169:0/4105224953 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f24b806ae40 msgr2=0x7f24b806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:07.982 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.980+0000 7f24dd105640 1 --2- 172.21.15.169:0/4105224953 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f24b806ae40 0x7f24b806d300 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f24c8034000 tx=0x7f24c802e000 comp rx=0 tx=0).stop 2023-10-26T01:35:07.983 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.981+0000 7f24dd105640 1 -- 172.21.15.169:0/4105224953 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24d80cc480 msgr2=0x7f24d80cc900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:07.983 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.981+0000 7f24dd105640 1 --2- 172.21.15.169:0/4105224953 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24d80cc480 0x7f24d80cc900 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f24d80b35a0 tx=0x7f24d000d9e0 comp rx=0 tx=0).stop 2023-10-26T01:35:07.983 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.981+0000 7f24dd105640 1 -- 172.21.15.169:0/4105224953 shutdown_connections 2023-10-26T01:35:07.983 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.981+0000 7f24dd105640 1 --2- 172.21.15.169:0/4105224953 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f24b806ae40 0x7f24b806d300 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:07.983 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.981+0000 7f24dd105640 1 --2- 172.21.15.169:0/4105224953 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24d80cc480 0x7f24d80cc900 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:07.984 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.981+0000 7f24dd105640 1 --2- 172.21.15.169:0/4105224953 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f24d80aeb30 0x7f24d814e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:07.984 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.981+0000 7f24dd105640 1 --2- 172.21.15.169:0/4105224953 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f24c4003970 0x7f24d806ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:07.984 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.981+0000 7f24dd105640 1 -- 172.21.15.169:0/4105224953 >> 172.21.15.169:0/4105224953 conn(0x7f24d8069930 msgr2=0x7f24d815b740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:35:07.986 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.985+0000 7f24dd105640 1 -- 172.21.15.169:0/4105224953 shutdown_connections 2023-10-26T01:35:07.987 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:07.985+0000 7f24dd105640 1 -- 172.21.15.169:0/4105224953 wait complete. 2023-10-26T01:35:08.010 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-26T01:35:08.099 INFO:tasks.ceph.mds.a.smithi169.stdout:starting mds.a at 2023-10-26T01:35:08.215 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:08.212+0000 7f2f13fff640 1 -- 172.21.15.169:0/2305811241 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2f140ac130 msgr2=0x7f2f1406aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:08.215 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:08.212+0000 7f2f13fff640 1 --2- 172.21.15.169:0/2305811241 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2f140ac130 0x7f2f1406aef0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2f04009830 tx=0x7f2f0402bd60 comp rx=0 tx=0).stop 2023-10-26T01:35:08.215 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:08.212+0000 7f2f13fff640 1 -- 172.21.15.169:0/2305811241 shutdown_connections 2023-10-26T01:35:08.215 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:08.212+0000 7f2f13fff640 1 --2- 172.21.15.169:0/2305811241 >> v2:172.21.15.170:3300/0 conn(0x7f2f1415f450 0x7f2f1415f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:08.216 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:08.212+0000 7f2f13fff640 1 --2- 172.21.15.169:0/2305811241 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2f140ac130 0x7f2f1406aef0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:08.216 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:08.212+0000 7f2f13fff640 1 -- 172.21.15.169:0/2305811241 >> 172.21.15.169:0/2305811241 conn(0x7f2f140af110 msgr2=0x7f2f140af520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:35:08.216 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:08.213+0000 7f2f13fff640 1 -- 172.21.15.169:0/2305811241 shutdown_connections 2023-10-26T01:35:08.216 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:08.213+0000 7f2f13fff640 1 -- 172.21.15.169:0/2305811241 wait complete. 2023-10-26T01:35:08.218 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:08.214+0000 7f2f13fff640 1 Processor -- start 2023-10-26T01:35:08.218 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:08.214+0000 7f2f13fff640 1 -- start start 2023-10-26T01:35:08.218 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:08.215+0000 7f2f13fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f140ac130 0x7f2f14148a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:08.218 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:08.215+0000 7f2f13fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2f140ac510 0x7f2f141f2960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:08.219 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:08.215+0000 7f2f13fff640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2f1415f450 0x7f2f1415d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:08.219 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:08.215+0000 7f2f13fff640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f14156530 con 0x7f2f140ac130 2023-10-26T01:35:08.219 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:08.215+0000 7f2f13fff640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f14148f70 con 0x7f2f140ac510 2023-10-26T01:35:08.219 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:08.215+0000 7f2f13fff640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f14142dc0 con 0x7f2f1415f450 2023-10-26T01:35:08.220 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:08.216+0000 7f2f12ffd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f140ac130 0x7f2f14148a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:08.220 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:08.216+0000 7f2f12ffd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f140ac130 0x7f2f14148a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:48602/0 (socket says 172.21.15.169:48602) 2023-10-26T01:35:08.220 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:08.216+0000 7f2f12ffd640 1 -- 172.21.15.169:0/2657013549 learned_addr learned my addr 172.21.15.169:0/2657013549 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:35:08.220 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:08.216+0000 7f2f0bfff640 1 --2- 172.21.15.169:0/2657013549 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2f140ac510 0x7f2f141f2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:08.220 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:08.216+0000 7f2f137fe640 1 --2- 172.21.15.169:0/2657013549 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2f1415f450 0x7f2f1415d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:08.221 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:08.216+0000 7f2f12ffd640 1 -- 172.21.15.169:0/2657013549 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2f1415f450 msgr2=0x7f2f1415d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:08.221 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:08.216+0000 7f2f12ffd640 1 --2- 172.21.15.169:0/2657013549 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2f1415f450 0x7f2f1415d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:08.221 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:08.216+0000 7f2f12ffd640 1 -- 172.21.15.169:0/2657013549 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2f140ac510 msgr2=0x7f2f141f2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:08.221 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:08.216+0000 7f2f12ffd640 1 --2- 172.21.15.169:0/2657013549 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2f140ac510 0x7f2f141f2960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:08.221 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:08.216+0000 7f2f12ffd640 1 -- 172.21.15.169:0/2657013549 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f04009490 con 0x7f2f140ac130 2023-10-26T01:35:08.222 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:08.216+0000 7f2f12ffd640 1 --2- 172.21.15.169:0/2657013549 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f140ac130 0x7f2f14148a30 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f2f04009780 tx=0x7f2f040038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:35:08.223 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:08.217+0000 7f2f10ff9640 1 -- 172.21.15.169:0/2657013549 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f04003ea0 con 0x7f2f140ac130 2023-10-26T01:35:08.223 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:08.217+0000 7f2f10ff9640 1 -- 172.21.15.169:0/2657013549 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2f04034440 con 0x7f2f140ac130 2023-10-26T01:35:08.223 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:08.217+0000 7f2f13fff640 1 -- 172.21.15.169:0/2657013549 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f1406a2f0 con 0x7f2f140ac130 2023-10-26T01:35:08.223 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:08.217+0000 7f2f10ff9640 1 -- 172.21.15.169:0/2657013549 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f04034850 con 0x7f2f140ac130 2023-10-26T01:35:08.224 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:08.217+0000 7f2f13fff640 1 -- 172.21.15.169:0/2657013549 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f141433c0 con 0x7f2f140ac130 2023-10-26T01:35:08.224 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:08.218+0000 7f2f13fff640 1 -- 172.21.15.169:0/2657013549 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f1415ff90 con 0x7f2f140ac130 2023-10-26T01:35:08.224 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:08.219+0000 7f2f10ff9640 1 -- 172.21.15.169:0/2657013549 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f2f0403b040 con 0x7f2f140ac130 2023-10-26T01:35:08.224 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:08.220+0000 7f2f10ff9640 1 --2- 172.21.15.169:0/2657013549 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2ef406ae40 0x7f2ef406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:08.224 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:08.221+0000 7f2f10ff9640 1 -- 172.21.15.169:0/2657013549 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f2f040abb30 con 0x7f2f140ac130 2023-10-26T01:35:08.225 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:08.221+0000 7f2f0bfff640 1 --2- 172.21.15.169:0/2657013549 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2ef406ae40 0x7f2ef406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:08.225 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:08.222+0000 7f2f0bfff640 1 --2- 172.21.15.169:0/2657013549 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2ef406ae40 0x7f2ef406d300 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f2f1409ab90 tx=0x7f2efc009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:35:08.227 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:08.225+0000 7f2f10ff9640 1 -- 172.21.15.169:0/2657013549 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2f04076c10 con 0x7f2f140ac130 2023-10-26T01:35:08.459 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:08.457+0000 7f2f13fff640 1 -- 172.21.15.169:0/2657013549 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f2f14073c10 con 0x7f2f140ac130 2023-10-26T01:35:09.109 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:09.107+0000 7f2f10ff9640 1 -- 172.21.15.169:0/2657013549 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v34) v1 ==== 104+0+0 (secure 0 0 0) 0x7f2f040410a0 con 0x7f2f140ac130 2023-10-26T01:35:09.112 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:09.110+0000 7f2f13fff640 1 -- 172.21.15.169:0/2657013549 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2ef406ae40 msgr2=0x7f2ef406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:09.113 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:09.110+0000 7f2f13fff640 1 --2- 172.21.15.169:0/2657013549 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2ef406ae40 0x7f2ef406d300 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f2f1409ab90 tx=0x7f2efc009040 comp rx=0 tx=0).stop 2023-10-26T01:35:09.113 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:09.110+0000 7f2f13fff640 1 -- 172.21.15.169:0/2657013549 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f140ac130 msgr2=0x7f2f14148a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:09.113 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:09.110+0000 7f2f13fff640 1 --2- 172.21.15.169:0/2657013549 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f140ac130 0x7f2f14148a30 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f2f04009780 tx=0x7f2f040038a0 comp rx=0 tx=0).stop 2023-10-26T01:35:09.113 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:09.111+0000 7f2f13fff640 1 -- 172.21.15.169:0/2657013549 shutdown_connections 2023-10-26T01:35:09.113 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:09.111+0000 7f2f13fff640 1 --2- 172.21.15.169:0/2657013549 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2ef406ae40 0x7f2ef406d300 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:09.114 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:09.111+0000 7f2f13fff640 1 --2- 172.21.15.169:0/2657013549 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2f1415f450 0x7f2f1415d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:09.114 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:09.111+0000 7f2f13fff640 1 --2- 172.21.15.169:0/2657013549 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2f140ac510 0x7f2f141f2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:09.114 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:09.111+0000 7f2f13fff640 1 --2- 172.21.15.169:0/2657013549 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f140ac130 0x7f2f14148a30 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:09.114 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:09.111+0000 7f2f13fff640 1 -- 172.21.15.169:0/2657013549 >> 172.21.15.169:0/2657013549 conn(0x7f2f140af110 msgr2=0x7f2f1415e190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:35:09.114 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:09.111+0000 7f2f13fff640 1 -- 172.21.15.169:0/2657013549 shutdown_connections 2023-10-26T01:35:09.115 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:09.111+0000 7f2f13fff640 1 -- 172.21.15.169:0/2657013549 wait complete. 2023-10-26T01:35:09.134 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-26T01:35:09.356 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:09.354+0000 7f689eef2640 1 -- 172.21.15.169:0/1214276736 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68980ac2f0 msgr2=0x7f68980ac7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:09.357 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:09.354+0000 7f689eef2640 1 --2- 172.21.15.169:0/1214276736 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68980ac2f0 0x7f68980ac7a0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f6890005fd0 tx=0x7f689002c1b0 comp rx=0 tx=0).stop 2023-10-26T01:35:09.357 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:09.354+0000 7f689eef2640 1 -- 172.21.15.169:0/1214276736 shutdown_connections 2023-10-26T01:35:09.357 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:09.354+0000 7f689eef2640 1 --2- 172.21.15.169:0/1214276736 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68980ac2f0 0x7f68980ac7a0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:09.358 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:09.354+0000 7f689eef2640 1 --2- 172.21.15.169:0/1214276736 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f68981595e0 0x7f68980c9bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:09.358 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:09.354+0000 7f689eef2640 1 -- 172.21.15.169:0/1214276736 >> 172.21.15.169:0/1214276736 conn(0x7f68980aeee0 msgr2=0x7f68980af2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:35:09.358 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:09.354+0000 7f689eef2640 1 -- 172.21.15.169:0/1214276736 shutdown_connections 2023-10-26T01:35:09.358 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:09.355+0000 7f689eef2640 1 -- 172.21.15.169:0/1214276736 wait complete. 2023-10-26T01:35:09.360 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:09.356+0000 7f689eef2640 1 Processor -- start 2023-10-26T01:35:09.361 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:09.356+0000 7f689eef2640 1 -- start start 2023-10-26T01:35:09.361 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:09.357+0000 7f689eef2640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f68980ac2f0 0x7f68980715b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:09.361 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:09.357+0000 7f689eef2640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68981595e0 0x7f6898142f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:09.361 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:09.357+0000 7f689eef2640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f689815d120 0x7f68981544e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:09.361 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:09.357+0000 7f689eef2640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6898071120 con 0x7f68981595e0 2023-10-26T01:35:09.362 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:09.357+0000 7f689eef2640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6898070b30 con 0x7f68980ac2f0 2023-10-26T01:35:09.362 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:09.357+0000 7f689eef2640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68981439d0 con 0x7f689815d120 2023-10-26T01:35:09.362 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:09.358+0000 7f6897fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68981595e0 0x7f6898142f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:09.362 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:09.358+0000 7f6897fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68981595e0 0x7f6898142f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:48616/0 (socket says 172.21.15.169:48616) 2023-10-26T01:35:09.362 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:09.358+0000 7f6897fff640 1 -- 172.21.15.169:0/58092559 learned_addr learned my addr 172.21.15.169:0/58092559 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:35:09.363 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:09.358+0000 7f689d48f640 1 --2- 172.21.15.169:0/58092559 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f689815d120 0x7f68981544e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:09.363 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:09.358+0000 7f689cc8e640 1 --2- 172.21.15.169:0/58092559 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f68980ac2f0 0x7f68980715b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:09.363 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:09.358+0000 7f6897fff640 1 -- 172.21.15.169:0/58092559 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f689815d120 msgr2=0x7f68981544e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:09.363 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:09.358+0000 7f6897fff640 1 --2- 172.21.15.169:0/58092559 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f689815d120 0x7f68981544e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:09.363 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:09.358+0000 7f6897fff640 1 -- 172.21.15.169:0/58092559 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f68980ac2f0 msgr2=0x7f68980715b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:09.364 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:09.358+0000 7f6897fff640 1 --2- 172.21.15.169:0/58092559 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f68980ac2f0 0x7f68980715b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:09.364 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:09.358+0000 7f6897fff640 1 -- 172.21.15.169:0/58092559 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6890009900 con 0x7f68981595e0 2023-10-26T01:35:09.364 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:09.358+0000 7f689cc8e640 1 --2- 172.21.15.169:0/58092559 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f68980ac2f0 0x7f68980715b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:35:09.364 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:09.358+0000 7f6897fff640 1 --2- 172.21.15.169:0/58092559 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68981595e0 0x7f6898142f80 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f688400b410 tx=0x7f688400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:35:09.364 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:09.358+0000 7f68957fa640 1 -- 172.21.15.169:0/58092559 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6884011830 con 0x7f68981595e0 2023-10-26T01:35:09.365 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:09.359+0000 7f68957fa640 1 -- 172.21.15.169:0/58092559 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6884011a80 con 0x7f68981595e0 2023-10-26T01:35:09.365 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:09.359+0000 7f68957fa640 1 -- 172.21.15.169:0/58092559 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6884011e90 con 0x7f68981595e0 2023-10-26T01:35:09.365 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:09.359+0000 7f689eef2640 1 -- 172.21.15.169:0/58092559 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6898143cb0 con 0x7f68981595e0 2023-10-26T01:35:09.365 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:09.359+0000 7f689eef2640 1 -- 172.21.15.169:0/58092559 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f689815c3d0 con 0x7f68981595e0 2023-10-26T01:35:09.366 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:09.360+0000 7f689eef2640 1 -- 172.21.15.169:0/58092559 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6898066af0 con 0x7f68981595e0 2023-10-26T01:35:09.367 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:09.361+0000 7f68957fa640 1 -- 172.21.15.169:0/58092559 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f688401a400 con 0x7f68981595e0 2023-10-26T01:35:09.367 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:09.362+0000 7f68957fa640 1 --2- 172.21.15.169:0/58092559 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f687c06ae40 0x7f687c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:09.367 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:09.362+0000 7f68957fa640 1 -- 172.21.15.169:0/58092559 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f6884089ab0 con 0x7f68981595e0 2023-10-26T01:35:09.367 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:09.362+0000 7f689cc8e640 1 --2- 172.21.15.169:0/58092559 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f687c06ae40 0x7f687c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:09.368 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:09.363+0000 7f689cc8e640 1 --2- 172.21.15.169:0/58092559 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f687c06ae40 0x7f687c06d300 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f688c0058c0 tx=0x7f688c005d30 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:35:09.368 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:09.366+0000 7f68957fa640 1 -- 172.21.15.169:0/58092559 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6884054b00 con 0x7f68981595e0 2023-10-26T01:35:09.558 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:09.556+0000 7f689eef2640 1 -- 172.21.15.169:0/58092559 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f689804da70 con 0x7f68981595e0 2023-10-26T01:35:10.120 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:10.118+0000 7f68957fa640 1 -- 172.21.15.169:0/58092559 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v74) v1 ==== 181+0+0 (secure 0 0 0) 0x7f6884015020 con 0x7f68981595e0 2023-10-26T01:35:10.120 INFO:teuthology.orchestra.run.smithi169.stderr:pool 'cephfs_metadata' removed 2023-10-26T01:35:10.123 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:10.121+0000 7f689eef2640 1 -- 172.21.15.169:0/58092559 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f687c06ae40 msgr2=0x7f687c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:10.124 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:10.121+0000 7f689eef2640 1 --2- 172.21.15.169:0/58092559 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f687c06ae40 0x7f687c06d300 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f688c0058c0 tx=0x7f688c005d30 comp rx=0 tx=0).stop 2023-10-26T01:35:10.124 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:10.121+0000 7f689eef2640 1 -- 172.21.15.169:0/58092559 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68981595e0 msgr2=0x7f6898142f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:10.124 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:10.121+0000 7f689eef2640 1 --2- 172.21.15.169:0/58092559 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68981595e0 0x7f6898142f80 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f688400b410 tx=0x7f688400b8d0 comp rx=0 tx=0).stop 2023-10-26T01:35:10.124 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:10.122+0000 7f689eef2640 1 -- 172.21.15.169:0/58092559 shutdown_connections 2023-10-26T01:35:10.125 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:10.122+0000 7f689eef2640 1 --2- 172.21.15.169:0/58092559 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f687c06ae40 0x7f687c06d300 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:10.125 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:10.122+0000 7f689eef2640 1 --2- 172.21.15.169:0/58092559 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68981595e0 0x7f6898142f80 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:10.125 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:10.122+0000 7f689eef2640 1 --2- 172.21.15.169:0/58092559 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f689815d120 0x7f68981544e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:10.125 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:10.122+0000 7f689eef2640 1 --2- 172.21.15.169:0/58092559 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f68980ac2f0 0x7f68980715b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:10.125 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:10.122+0000 7f689eef2640 1 -- 172.21.15.169:0/58092559 >> 172.21.15.169:0/58092559 conn(0x7f68980aeee0 msgr2=0x7f689814efb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:35:10.126 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:10.122+0000 7f689eef2640 1 -- 172.21.15.169:0/58092559 shutdown_connections 2023-10-26T01:35:10.126 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:10.122+0000 7f689eef2640 1 -- 172.21.15.169:0/58092559 wait complete. 2023-10-26T01:35:10.144 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-26T01:35:10.368 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:10.365+0000 7f48df21a640 1 -- 172.21.15.169:0/3467608626 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f48c40039a0 msgr2=0x7f48c4003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:10.368 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:10.365+0000 7f48df21a640 1 --2- 172.21.15.169:0/3467608626 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f48c40039a0 0x7f48c4003dc0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f48d4001650 tx=0x7f48c40057f0 comp rx=0 tx=0).stop 2023-10-26T01:35:10.368 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:10.366+0000 7f48dea19640 1 -- 172.21.15.169:0/3467608626 reap_dead start 2023-10-26T01:35:10.368 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:10.366+0000 7f48df21a640 1 -- 172.21.15.169:0/3467608626 shutdown_connections 2023-10-26T01:35:10.369 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:10.366+0000 7f48df21a640 1 -- 172.21.15.169:0/3467608626 >> 172.21.15.169:0/3467608626 conn(0x7f48d8069b40 msgr2=0x7f48d80aeb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:35:10.369 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:10.366+0000 7f48df21a640 1 -- 172.21.15.169:0/3467608626 shutdown_connections 2023-10-26T01:35:10.369 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:10.366+0000 7f48df21a640 1 -- 172.21.15.169:0/3467608626 wait complete. 2023-10-26T01:35:10.369 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:10.368+0000 7f48df21a640 1 Processor -- start 2023-10-26T01:35:10.370 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:10.368+0000 7f48df21a640 1 -- start start 2023-10-26T01:35:10.372 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:10.369+0000 7f48df21a640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f48c40039a0 0x7f48d81538b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:10.372 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:10.369+0000 7f48df21a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48d8150e30 0x7f48d81f4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:10.372 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:10.369+0000 7f48df21a640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f48d80ae150 0x7f48d81f5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:10.372 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:10.369+0000 7f48df21a640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48d81e1e80 con 0x7f48c40039a0 2023-10-26T01:35:10.372 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:10.369+0000 7f48df21a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48d8071200 con 0x7f48d8150e30 2023-10-26T01:35:10.373 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:10.369+0000 7f48df21a640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48d81e2140 con 0x7f48d80ae150 2023-10-26T01:35:10.373 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:10.369+0000 7f48de218640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f48c40039a0 0x7f48d81538b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:10.373 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:10.369+0000 7f48de218640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f48c40039a0 0x7f48d81538b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:48640/0 (socket says 172.21.15.169:48640) 2023-10-26T01:35:10.373 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:10.369+0000 7f48de218640 1 -- 172.21.15.169:0/2944752513 learned_addr learned my addr 172.21.15.169:0/2944752513 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:35:10.373 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:10.369+0000 7f48dea19640 1 --2- 172.21.15.169:0/2944752513 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f48d80ae150 0x7f48d81f5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:10.374 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:10.369+0000 7f48dda17640 1 --2- 172.21.15.169:0/2944752513 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48d8150e30 0x7f48d81f4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:10.374 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:10.369+0000 7f48de218640 1 -- 172.21.15.169:0/2944752513 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f48d80ae150 msgr2=0x7f48d81f5510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:10.374 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:10.369+0000 7f48de218640 1 --2- 172.21.15.169:0/2944752513 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f48d80ae150 0x7f48d81f5510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:10.374 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:10.369+0000 7f48de218640 1 -- 172.21.15.169:0/2944752513 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48d8150e30 msgr2=0x7f48d81f4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:10.374 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:10.369+0000 7f48de218640 1 --2- 172.21.15.169:0/2944752513 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48d8150e30 0x7f48d81f4fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:10.375 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:10.369+0000 7f48de218640 1 -- 172.21.15.169:0/2944752513 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48d81e2420 con 0x7f48c40039a0 2023-10-26T01:35:10.375 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:10.370+0000 7f48de218640 1 --2- 172.21.15.169:0/2944752513 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f48c40039a0 0x7f48d81538b0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f48d402c0b0 tx=0x7f48d40035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:35:10.375 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:10.370+0000 7f48ceffd640 1 -- 172.21.15.169:0/2944752513 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d4003bf0 con 0x7f48c40039a0 2023-10-26T01:35:10.375 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:10.370+0000 7f48ceffd640 1 -- 172.21.15.169:0/2944752513 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f48d4003e40 con 0x7f48c40039a0 2023-10-26T01:35:10.375 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:10.370+0000 7f48ceffd640 1 -- 172.21.15.169:0/2944752513 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d402c770 con 0x7f48c40039a0 2023-10-26T01:35:10.376 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:10.370+0000 7f48df21a640 1 -- 172.21.15.169:0/2944752513 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48d81e2700 con 0x7f48c40039a0 2023-10-26T01:35:10.376 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:10.370+0000 7f48df21a640 1 -- 172.21.15.169:0/2944752513 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48d8153fd0 con 0x7f48c40039a0 2023-10-26T01:35:10.377 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:10.371+0000 7f48df21a640 1 -- 172.21.15.169:0/2944752513 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48d81e2890 con 0x7f48c40039a0 2023-10-26T01:35:10.377 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:10.372+0000 7f48ceffd640 1 -- 172.21.15.169:0/2944752513 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f48d402c8d0 con 0x7f48c40039a0 2023-10-26T01:35:10.377 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:10.373+0000 7f48ceffd640 1 --2- 172.21.15.169:0/2944752513 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f48bc06ae70 0x7f48bc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:10.377 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:10.373+0000 7f48ceffd640 1 -- 172.21.15.169:0/2944752513 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5357+0+0 (secure 0 0 0) 0x7f48d40aa750 con 0x7f48c40039a0 2023-10-26T01:35:10.378 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:10.374+0000 7f48dda17640 1 --2- 172.21.15.169:0/2944752513 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f48bc06ae70 0x7f48bc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:10.378 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:10.375+0000 7f48dda17640 1 --2- 172.21.15.169:0/2944752513 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f48bc06ae70 0x7f48bc06d330 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f48d8151210 tx=0x7f48c8005f90 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:35:10.379 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:10.378+0000 7f48ceffd640 1 -- 172.21.15.169:0/2944752513 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f48d40758c0 con 0x7f48c40039a0 2023-10-26T01:35:10.576 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:10.574+0000 7f48df21a640 1 -- 172.21.15.169:0/2944752513 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f48d8153fd0 con 0x7f48c40039a0 2023-10-26T01:35:11.145 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:11.143+0000 7f48ceffd640 1 -- 172.21.15.169:0/2944752513 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v75) v1 ==== 169+0+0 (secure 0 0 0) 0x7f48d8153fd0 con 0x7f48c40039a0 2023-10-26T01:35:11.146 INFO:teuthology.orchestra.run.smithi169.stderr:pool 'cephfs_data' removed 2023-10-26T01:35:11.150 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:11.148+0000 7f48df21a640 1 -- 172.21.15.169:0/2944752513 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f48bc06ae70 msgr2=0x7f48bc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:11.151 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:11.148+0000 7f48df21a640 1 --2- 172.21.15.169:0/2944752513 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f48bc06ae70 0x7f48bc06d330 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f48d8151210 tx=0x7f48c8005f90 comp rx=0 tx=0).stop 2023-10-26T01:35:11.151 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:11.148+0000 7f48df21a640 1 -- 172.21.15.169:0/2944752513 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f48c40039a0 msgr2=0x7f48d81538b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:11.151 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:11.148+0000 7f48df21a640 1 --2- 172.21.15.169:0/2944752513 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f48c40039a0 0x7f48d81538b0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f48d402c0b0 tx=0x7f48d40035f0 comp rx=0 tx=0).stop 2023-10-26T01:35:11.151 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:11.148+0000 7f48df21a640 1 -- 172.21.15.169:0/2944752513 shutdown_connections 2023-10-26T01:35:11.152 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:11.148+0000 7f48df21a640 1 --2- 172.21.15.169:0/2944752513 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f48bc06ae70 0x7f48bc06d330 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:11.152 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:11.148+0000 7f48df21a640 1 --2- 172.21.15.169:0/2944752513 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f48d80ae150 0x7f48d81f5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:11.152 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:11.148+0000 7f48df21a640 1 --2- 172.21.15.169:0/2944752513 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48d8150e30 0x7f48d81f4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:11.152 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:11.148+0000 7f48df21a640 1 --2- 172.21.15.169:0/2944752513 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f48c40039a0 0x7f48d81538b0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:11.152 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:11.148+0000 7f48df21a640 1 -- 172.21.15.169:0/2944752513 >> 172.21.15.169:0/2944752513 conn(0x7f48d8069b40 msgr2=0x7f48d80abe70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:35:11.153 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:11.148+0000 7f48df21a640 1 -- 172.21.15.169:0/2944752513 shutdown_connections 2023-10-26T01:35:11.153 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:11.148+0000 7f48df21a640 1 -- 172.21.15.169:0/2944752513 wait complete. 2023-10-26T01:35:11.173 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 18 2023-10-26T01:35:11.413 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:11.411+0000 7f9a5634c640 1 -- 172.21.15.169:0/3705986226 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a500cdd20 msgr2=0x7f9a5015b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:11.413 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:11.411+0000 7f9a5634c640 1 --2- 172.21.15.169:0/3705986226 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a500cdd20 0x7f9a5015b4a0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f9a38005fd0 tx=0x7f9a3802be20 comp rx=0 tx=0).stop 2023-10-26T01:35:11.413 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:11.411+0000 7f9a5634c640 1 -- 172.21.15.169:0/3705986226 shutdown_connections 2023-10-26T01:35:11.414 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:11.411+0000 7f9a5634c640 1 --2- 172.21.15.169:0/3705986226 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a500cdd20 0x7f9a5015b4a0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:11.414 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:11.411+0000 7f9a5634c640 1 --2- 172.21.15.169:0/3705986226 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9a50155360 0x7f9a500cd7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:11.414 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:11.411+0000 7f9a5634c640 1 -- 172.21.15.169:0/3705986226 >> 172.21.15.169:0/3705986226 conn(0x7f9a50069930 msgr2=0x7f9a50069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:35:11.414 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:11.411+0000 7f9a5634c640 1 -- 172.21.15.169:0/3705986226 shutdown_connections 2023-10-26T01:35:11.415 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:11.411+0000 7f9a5634c640 1 -- 172.21.15.169:0/3705986226 wait complete. 2023-10-26T01:35:11.417 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:11.413+0000 7f9a5634c640 1 Processor -- start 2023-10-26T01:35:11.417 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:11.413+0000 7f9a5634c640 1 -- start start 2023-10-26T01:35:11.417 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:11.414+0000 7f9a5634c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a500cdd20 0x7f9a5014ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:11.417 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:11.414+0000 7f9a5634c640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9a50155360 0x7f9a5006ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:11.418 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:11.414+0000 7f9a5634c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9a5015b9e0 0x7f9a5006f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:11.418 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:11.414+0000 7f9a5634c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a500acd50 con 0x7f9a500cdd20 2023-10-26T01:35:11.418 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:11.414+0000 7f9a5634c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a501534e0 con 0x7f9a5015b9e0 2023-10-26T01:35:11.418 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:11.414+0000 7f9a5634c640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a501537c0 con 0x7f9a50155360 2023-10-26T01:35:11.419 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:11.414+0000 7f9a4ffff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a500cdd20 0x7f9a5014ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:11.419 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:11.414+0000 7f9a4ffff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a500cdd20 0x7f9a5014ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:48658/0 (socket says 172.21.15.169:48658) 2023-10-26T01:35:11.419 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:11.414+0000 7f9a4ffff640 1 -- 172.21.15.169:0/172376935 learned_addr learned my addr 172.21.15.169:0/172376935 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:35:11.420 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:11.414+0000 7f9a47fff640 1 --2- 172.21.15.169:0/172376935 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9a50155360 0x7f9a5006ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:11.420 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:11.414+0000 7f9a4ffff640 1 -- 172.21.15.169:0/172376935 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9a50155360 msgr2=0x7f9a5006ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:11.420 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:11.414+0000 7f9a4ffff640 1 --2- 172.21.15.169:0/172376935 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9a50155360 0x7f9a5006ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:11.421 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:11.414+0000 7f9a4ffff640 1 -- 172.21.15.169:0/172376935 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9a5015b9e0 msgr2=0x7f9a5006f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:11.421 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:11.414+0000 7f9a548e9640 1 --2- 172.21.15.169:0/172376935 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9a5015b9e0 0x7f9a5006f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:11.422 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:11.415+0000 7f9a4ffff640 1 --2- 172.21.15.169:0/172376935 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9a5015b9e0 0x7f9a5006f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:11.422 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:11.415+0000 7f9a4ffff640 1 -- 172.21.15.169:0/172376935 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a38009560 con 0x7f9a500cdd20 2023-10-26T01:35:11.422 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:11.415+0000 7f9a548e9640 1 --2- 172.21.15.169:0/172376935 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9a5015b9e0 0x7f9a5006f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:35:11.422 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:11.415+0000 7f9a4ffff640 1 --2- 172.21.15.169:0/172376935 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a500cdd20 0x7f9a5014ad00 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f9a4000c920 tx=0x7f9a4000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:35:11.423 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:11.415+0000 7f9a4d7fa640 1 -- 172.21.15.169:0/172376935 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a40007c70 con 0x7f9a500cdd20 2023-10-26T01:35:11.423 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:11.415+0000 7f9a4d7fa640 1 -- 172.21.15.169:0/172376935 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9a4000f450 con 0x7f9a500cdd20 2023-10-26T01:35:11.423 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:11.415+0000 7f9a4d7fa640 1 -- 172.21.15.169:0/172376935 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a4000f860 con 0x7f9a500cdd20 2023-10-26T01:35:11.424 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:11.415+0000 7f9a5634c640 1 -- 172.21.15.169:0/172376935 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a50153aa0 con 0x7f9a500cdd20 2023-10-26T01:35:11.424 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:11.415+0000 7f9a5634c640 1 -- 172.21.15.169:0/172376935 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a50070490 con 0x7f9a500cdd20 2023-10-26T01:35:11.426 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:11.416+0000 7f9a5634c640 1 -- 172.21.15.169:0/172376935 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a5015b4a0 con 0x7f9a500cdd20 2023-10-26T01:35:11.426 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:11.417+0000 7f9a4d7fa640 1 -- 172.21.15.169:0/172376935 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f9a4000f9c0 con 0x7f9a500cdd20 2023-10-26T01:35:11.427 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:11.418+0000 7f9a4d7fa640 1 --2- 172.21.15.169:0/172376935 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9a3006ae40 0x7f9a3006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:11.427 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:11.418+0000 7f9a4d7fa640 1 -- 172.21.15.169:0/172376935 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f9a40089b40 con 0x7f9a500cdd20 2023-10-26T01:35:11.427 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:11.419+0000 7f9a47fff640 1 --2- 172.21.15.169:0/172376935 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9a3006ae40 0x7f9a3006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:11.427 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:11.420+0000 7f9a47fff640 1 --2- 172.21.15.169:0/172376935 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9a3006ae40 0x7f9a3006d300 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f9a38035000 tx=0x7f9a38004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:35:11.428 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:11.422+0000 7f9a4d7fa640 1 -- 172.21.15.169:0/172376935 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9a40054f60 con 0x7f9a500cdd20 2023-10-26T01:35:11.614 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:11.612+0000 7f9a5634c640 1 -- 172.21.15.169:0/172376935 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "config reset", "num": 18} v 0) v1 -- 0x7f9a5006ce00 con 0x7f9a500cdd20 2023-10-26T01:35:11.619 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:11.618+0000 7f9a4d7fa640 1 -- 172.21.15.169:0/172376935 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 18}]=0 v24) v1 ==== 71+0+0 (secure 0 0 0) 0x7f9a5006ce00 con 0x7f9a500cdd20 2023-10-26T01:35:11.623 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:11.621+0000 7f9a5634c640 1 -- 172.21.15.169:0/172376935 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9a3006ae40 msgr2=0x7f9a3006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:11.623 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:11.621+0000 7f9a5634c640 1 --2- 172.21.15.169:0/172376935 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9a3006ae40 0x7f9a3006d300 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f9a38035000 tx=0x7f9a38004730 comp rx=0 tx=0).stop 2023-10-26T01:35:11.623 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:11.621+0000 7f9a5634c640 1 -- 172.21.15.169:0/172376935 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a500cdd20 msgr2=0x7f9a5014ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:11.624 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:11.621+0000 7f9a5634c640 1 --2- 172.21.15.169:0/172376935 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a500cdd20 0x7f9a5014ad00 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f9a4000c920 tx=0x7f9a4000cde0 comp rx=0 tx=0).stop 2023-10-26T01:35:11.624 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:11.621+0000 7f9a5634c640 1 -- 172.21.15.169:0/172376935 shutdown_connections 2023-10-26T01:35:11.624 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:11.621+0000 7f9a5634c640 1 --2- 172.21.15.169:0/172376935 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9a3006ae40 0x7f9a3006d300 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:11.624 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:11.621+0000 7f9a5634c640 1 --2- 172.21.15.169:0/172376935 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9a5015b9e0 0x7f9a5006f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:11.624 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:11.621+0000 7f9a5634c640 1 --2- 172.21.15.169:0/172376935 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9a50155360 0x7f9a5006ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:11.625 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:11.621+0000 7f9a5634c640 1 --2- 172.21.15.169:0/172376935 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a500cdd20 0x7f9a5014ad00 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:11.625 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:11.621+0000 7f9a5634c640 1 -- 172.21.15.169:0/172376935 >> 172.21.15.169:0/172376935 conn(0x7f9a50069930 msgr2=0x7f9a5015a580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:35:11.625 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:11.621+0000 7f9a5634c640 1 -- 172.21.15.169:0/172376935 shutdown_connections 2023-10-26T01:35:11.625 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:11.621+0000 7f9a5634c640 1 -- 172.21.15.169:0/172376935 wait complete. 2023-10-26T01:35:11.641 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestClusterFull.test_full_fclose' 2023-10-26T01:35:11.860 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:11.858+0000 7f819b2af640 1 -- 172.21.15.169:0/1750537892 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8180005180 msgr2=0x7f818000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:11.860 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:11.858+0000 7f819b2af640 1 --2- 172.21.15.169:0/1750537892 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8180005180 0x7f818000b980 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f818c009f50 tx=0x7f81800057f0 comp rx=0 tx=0).stop 2023-10-26T01:35:11.860 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:11.858+0000 7f819984c640 1 -- 172.21.15.169:0/1750537892 reap_dead start 2023-10-26T01:35:11.861 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:11.858+0000 7f819b2af640 1 -- 172.21.15.169:0/1750537892 shutdown_connections 2023-10-26T01:35:11.861 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:11.858+0000 7f819b2af640 1 -- 172.21.15.169:0/1750537892 >> 172.21.15.169:0/1750537892 conn(0x7f8194069930 msgr2=0x7f8194069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:35:11.861 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:11.858+0000 7f819b2af640 1 -- 172.21.15.169:0/1750537892 shutdown_connections 2023-10-26T01:35:11.861 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:11.858+0000 7f819b2af640 1 -- 172.21.15.169:0/1750537892 wait complete. 2023-10-26T01:35:11.862 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:11.860+0000 7f819b2af640 1 Processor -- start 2023-10-26T01:35:11.862 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:11.860+0000 7f819b2af640 1 -- start start 2023-10-26T01:35:11.864 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:11.861+0000 7f819b2af640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f8180005180 0x7f819415bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:11.865 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:11.861+0000 7f819b2af640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f819415c3f0 0x7f819414e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:11.865 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:11.861+0000 7f819b2af640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f819415a290 0x7f819415a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:11.865 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:11.861+0000 7f819b2af640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f819415ac50 con 0x7f819415a290 2023-10-26T01:35:11.865 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:11.861+0000 7f819b2af640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f819414e7f0 con 0x7f819415c3f0 2023-10-26T01:35:11.866 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:11.861+0000 7f819b2af640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f819415ebf0 con 0x7f8180005180 2023-10-26T01:35:11.866 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:11.861+0000 7f819984c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f819415a290 0x7f819415a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:11.866 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:11.861+0000 7f819984c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f819415a290 0x7f819415a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:48678/0 (socket says 172.21.15.169:48678) 2023-10-26T01:35:11.867 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:11.861+0000 7f819984c640 1 -- 172.21.15.169:0/737061617 learned_addr learned my addr 172.21.15.169:0/737061617 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:35:11.867 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:11.862+0000 7f819904b640 1 --2- 172.21.15.169:0/737061617 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f8180005180 0x7f819415bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:11.868 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:11.862+0000 7f819884a640 1 --2- 172.21.15.169:0/737061617 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f819415c3f0 0x7f819414e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:11.868 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:11.862+0000 7f819984c640 1 -- 172.21.15.169:0/737061617 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f8180005180 msgr2=0x7f819415bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:11.868 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:11.862+0000 7f819984c640 1 --2- 172.21.15.169:0/737061617 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f8180005180 0x7f819415bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:11.869 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:11.862+0000 7f819984c640 1 -- 172.21.15.169:0/737061617 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f819415c3f0 msgr2=0x7f819414e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:11.869 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:11.862+0000 7f819984c640 1 --2- 172.21.15.169:0/737061617 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f819415c3f0 0x7f819414e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:11.869 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:11.862+0000 7f819984c640 1 -- 172.21.15.169:0/737061617 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f819415d210 con 0x7f819415a290 2023-10-26T01:35:11.870 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:11.862+0000 7f819884a640 1 --2- 172.21.15.169:0/737061617 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f819415c3f0 0x7f819414e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:35:11.870 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:11.862+0000 7f819984c640 1 --2- 172.21.15.169:0/737061617 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f819415a290 0x7f819415a710 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f818c009f50 tx=0x7f818c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:35:11.870 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:11.862+0000 7f8189ffb640 1 -- 172.21.15.169:0/737061617 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f818c003ec0 con 0x7f819415a290 2023-10-26T01:35:11.870 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:11.862+0000 7f8189ffb640 1 -- 172.21.15.169:0/737061617 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f818c004660 con 0x7f819415a290 2023-10-26T01:35:11.871 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:11.862+0000 7f819b2af640 1 -- 172.21.15.169:0/737061617 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81941e6570 con 0x7f819415a290 2023-10-26T01:35:11.871 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:11.863+0000 7f8189ffb640 1 -- 172.21.15.169:0/737061617 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f818c004b20 con 0x7f819415a290 2023-10-26T01:35:11.871 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:11.863+0000 7f819b2af640 1 -- 172.21.15.169:0/737061617 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f819415cc50 con 0x7f819415a290 2023-10-26T01:35:11.873 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:11.864+0000 7f819b2af640 1 -- 172.21.15.169:0/737061617 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f819404db10 con 0x7f819415a290 2023-10-26T01:35:11.873 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:11.865+0000 7f8189ffb640 1 -- 172.21.15.169:0/737061617 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f818c03c070 con 0x7f819415a290 2023-10-26T01:35:11.873 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:11.866+0000 7f8189ffb640 1 --2- 172.21.15.169:0/737061617 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f817806ae40 0x7f817806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:11.874 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:11.866+0000 7f8189ffb640 1 -- 172.21.15.169:0/737061617 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f818c02ce40 con 0x7f819415a290 2023-10-26T01:35:11.874 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:11.866+0000 7f819904b640 1 --2- 172.21.15.169:0/737061617 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f817806ae40 0x7f817806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:11.874 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:11.868+0000 7f819904b640 1 --2- 172.21.15.169:0/737061617 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f817806ae40 0x7f817806d300 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f8190006a30 tx=0x7f8190001430 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:35:11.874 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:11.870+0000 7f8189ffb640 1 -- 172.21.15.169:0/737061617 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f818c0764d0 con 0x7f819415a290 2023-10-26T01:35:12.130 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:12.128+0000 7f819b2af640 1 -- 172.21.15.169:0/737061617 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_fclose"]} v 0) v1 -- 0x7f81940ad460 con 0x7f819415a290 2023-10-26T01:35:12.148 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:12.147+0000 7f8189ffb640 1 -- 172.21.15.169:0/737061617 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_fclose"]}]=0 v240) v1 ==== 134+0+0 (secure 0 0 0) 0x7f818c079af0 con 0x7f819415a290 2023-10-26T01:35:12.152 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:12.150+0000 7f819b2af640 1 -- 172.21.15.169:0/737061617 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f817806ae40 msgr2=0x7f817806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:12.152 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:12.150+0000 7f819b2af640 1 --2- 172.21.15.169:0/737061617 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f817806ae40 0x7f817806d300 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f8190006a30 tx=0x7f8190001430 comp rx=0 tx=0).stop 2023-10-26T01:35:12.153 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:12.150+0000 7f819b2af640 1 -- 172.21.15.169:0/737061617 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f819415a290 msgr2=0x7f819415a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:12.153 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:12.150+0000 7f819b2af640 1 --2- 172.21.15.169:0/737061617 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f819415a290 0x7f819415a710 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f818c009f50 tx=0x7f818c004270 comp rx=0 tx=0).stop 2023-10-26T01:35:12.153 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:12.150+0000 7f819b2af640 1 -- 172.21.15.169:0/737061617 shutdown_connections 2023-10-26T01:35:12.153 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:12.150+0000 7f819b2af640 1 --2- 172.21.15.169:0/737061617 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f817806ae40 0x7f817806d300 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:12.153 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:12.150+0000 7f819b2af640 1 --2- 172.21.15.169:0/737061617 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f819415a290 0x7f819415a710 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:12.154 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:12.150+0000 7f819b2af640 1 --2- 172.21.15.169:0/737061617 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f819415c3f0 0x7f819414e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:12.154 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:12.150+0000 7f819b2af640 1 --2- 172.21.15.169:0/737061617 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f8180005180 0x7f819415bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:12.154 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:12.150+0000 7f819b2af640 1 -- 172.21.15.169:0/737061617 >> 172.21.15.169:0/737061617 conn(0x7f8194069930 msgr2=0x7f8194069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:35:12.154 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:12.150+0000 7f819b2af640 1 -- 172.21.15.169:0/737061617 shutdown_connections 2023-10-26T01:35:12.155 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:12.150+0000 7f819b2af640 1 -- 172.21.15.169:0/737061617 wait complete. 2023-10-26T01:35:12.172 INFO:tasks.cephfs_test_runner:test_full_fclose (tasks.cephfs.test_full.TestClusterFull) ... ok 2023-10-26T01:35:12.172 INFO:tasks.cephfs_test_runner:Starting test: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) That when the full flag is encountered during asynchronous 2023-10-26T01:35:12.173 INFO:tasks.cephfs_test_runner:test_full_fsync (tasks.cephfs.test_full.TestClusterFull) 2023-10-26T01:35:12.173 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:35:12.393 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:12.391+0000 7fcb3ad47640 1 -- 172.21.15.169:0/795473119 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcb340c9970 msgr2=0x7fcb34098a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:12.393 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:12.391+0000 7fcb3ad47640 1 --2- 172.21.15.169:0/795473119 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcb340c9970 0x7fcb34098a60 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fcb24005fd0 tx=0x7fcb2402be10 comp rx=0 tx=0).stop 2023-10-26T01:35:12.393 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:12.391+0000 7fcb3ad47640 1 -- 172.21.15.169:0/795473119 shutdown_connections 2023-10-26T01:35:12.394 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:12.391+0000 7fcb3ad47640 1 --2- 172.21.15.169:0/795473119 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcb340c9970 0x7fcb34098a60 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:12.394 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:12.391+0000 7fcb3ad47640 1 --2- 172.21.15.169:0/795473119 >> v2:172.21.15.170:3300/0 conn(0x7fcb340ac6d0 0x7fcb3415d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:12.394 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:12.391+0000 7fcb3ad47640 1 -- 172.21.15.169:0/795473119 >> 172.21.15.169:0/795473119 conn(0x7fcb340aedd0 msgr2=0x7fcb340af1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:35:12.394 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:12.391+0000 7fcb3ad47640 1 -- 172.21.15.169:0/795473119 shutdown_connections 2023-10-26T01:35:12.394 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:12.391+0000 7fcb3ad47640 1 -- 172.21.15.169:0/795473119 wait complete. 2023-10-26T01:35:12.395 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:12.393+0000 7fcb3ad47640 1 Processor -- start 2023-10-26T01:35:12.395 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:12.393+0000 7fcb3ad47640 1 -- start start 2023-10-26T01:35:12.397 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:12.395+0000 7fcb3ad47640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb3409a5c0 0x7fcb341565e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:12.397 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:12.395+0000 7fcb3ad47640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb340ac6d0 0x7fcb3406eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:12.397 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:12.395+0000 7fcb3ad47640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcb340c9970 0x7fcb3406f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:12.397 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:12.395+0000 7fcb3ad47640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb340c8930 con 0x7fcb340ac6d0 2023-10-26T01:35:12.397 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:12.395+0000 7fcb3ad47640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb34071760 con 0x7fcb3409a5c0 2023-10-26T01:35:12.398 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:12.395+0000 7fcb3ad47640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb34156b20 con 0x7fcb340c9970 2023-10-26T01:35:12.398 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:12.395+0000 7fcb39544640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb340ac6d0 0x7fcb3406eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:12.398 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:12.395+0000 7fcb39544640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb340ac6d0 0x7fcb3406eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:48694/0 (socket says 172.21.15.169:48694) 2023-10-26T01:35:12.398 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:12.395+0000 7fcb39544640 1 -- 172.21.15.169:0/4012084351 learned_addr learned my addr 172.21.15.169:0/4012084351 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:35:12.400 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:12.395+0000 7fcb39d45640 1 --2- 172.21.15.169:0/4012084351 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb3409a5c0 0x7fcb341565e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:12.400 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:12.396+0000 7fcb3a546640 1 --2- 172.21.15.169:0/4012084351 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcb340c9970 0x7fcb3406f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:12.400 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:12.396+0000 7fcb39544640 1 -- 172.21.15.169:0/4012084351 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcb340c9970 msgr2=0x7fcb3406f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:12.401 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:12.396+0000 7fcb39544640 1 --2- 172.21.15.169:0/4012084351 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcb340c9970 0x7fcb3406f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:12.401 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:12.396+0000 7fcb39544640 1 -- 172.21.15.169:0/4012084351 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb3409a5c0 msgr2=0x7fcb341565e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:12.401 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:12.396+0000 7fcb39544640 1 --2- 172.21.15.169:0/4012084351 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb3409a5c0 0x7fcb341565e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:12.401 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:12.396+0000 7fcb39544640 1 -- 172.21.15.169:0/4012084351 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb24009560 con 0x7fcb340ac6d0 2023-10-26T01:35:12.401 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:12.396+0000 7fcb39d45640 1 --2- 172.21.15.169:0/4012084351 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb3409a5c0 0x7fcb341565e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:35:12.402 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:12.396+0000 7fcb3a546640 1 --2- 172.21.15.169:0/4012084351 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcb340c9970 0x7fcb3406f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:35:12.402 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:12.396+0000 7fcb39544640 1 --2- 172.21.15.169:0/4012084351 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb340ac6d0 0x7fcb3406eb70 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fcb24006010 tx=0x7fcb2402da70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:35:12.402 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:12.396+0000 7fcb2a7fc640 1 -- 172.21.15.169:0/4012084351 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb2402de70 con 0x7fcb340ac6d0 2023-10-26T01:35:12.402 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:12.396+0000 7fcb2a7fc640 1 -- 172.21.15.169:0/4012084351 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fcb240036e0 con 0x7fcb340ac6d0 2023-10-26T01:35:12.402 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:12.396+0000 7fcb2a7fc640 1 -- 172.21.15.169:0/4012084351 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb24003af0 con 0x7fcb340ac6d0 2023-10-26T01:35:12.403 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:12.396+0000 7fcb3ad47640 1 -- 172.21.15.169:0/4012084351 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb340c75b0 con 0x7fcb340ac6d0 2023-10-26T01:35:12.403 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:12.397+0000 7fcb3ad47640 1 -- 172.21.15.169:0/4012084351 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb340c7b70 con 0x7fcb340ac6d0 2023-10-26T01:35:12.403 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:12.398+0000 7fcb3ad47640 1 -- 172.21.15.169:0/4012084351 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb34073c10 con 0x7fcb340ac6d0 2023-10-26T01:35:12.404 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:12.399+0000 7fcb2a7fc640 1 -- 172.21.15.169:0/4012084351 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fcb24003c50 con 0x7fcb340ac6d0 2023-10-26T01:35:12.404 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:12.400+0000 7fcb2a7fc640 1 --2- 172.21.15.169:0/4012084351 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcb1806ae40 0x7fcb1806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:12.404 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:12.400+0000 7fcb2a7fc640 1 -- 172.21.15.169:0/4012084351 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fcb240abff0 con 0x7fcb340ac6d0 2023-10-26T01:35:12.404 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:12.401+0000 7fcb39d45640 1 --2- 172.21.15.169:0/4012084351 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcb1806ae40 0x7fcb1806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:12.405 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:12.402+0000 7fcb39d45640 1 --2- 172.21.15.169:0/4012084351 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcb1806ae40 0x7fcb1806d300 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fcb30009b50 tx=0x7fcb30005f90 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:35:12.406 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:12.405+0000 7fcb2a7fc640 1 -- 172.21.15.169:0/4012084351 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcb240deca0 con 0x7fcb340ac6d0 2023-10-26T01:35:12.595 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:12.594+0000 7fcb3ad47640 1 -- 172.21.15.169:0/4012084351 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcb34071920 con 0x7fcb340ac6d0 2023-10-26T01:35:12.597 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:12.596+0000 7fcb2a7fc640 1 -- 172.21.15.169:0/4012084351 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) v1 ==== 74+0+13036 (secure 0 0 0) 0x7fcb24043090 con 0x7fcb340ac6d0 2023-10-26T01:35:12.598 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:35:12.598 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":75,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:35:11.134483+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"72","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.169:6835/2338833513":"2023-10-27T01:35:07.973450+0000","172.21.15.169:6834/2338833513":"2023-10-27T01:35:07.973450+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:35:12.600 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:12.598+0000 7fcb3ad47640 1 -- 172.21.15.169:0/4012084351 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcb1806ae40 msgr2=0x7fcb1806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:12.600 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:12.598+0000 7fcb3ad47640 1 --2- 172.21.15.169:0/4012084351 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcb1806ae40 0x7fcb1806d300 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fcb30009b50 tx=0x7fcb30005f90 comp rx=0 tx=0).stop 2023-10-26T01:35:12.600 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:12.598+0000 7fcb3ad47640 1 -- 172.21.15.169:0/4012084351 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb340ac6d0 msgr2=0x7fcb3406eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:12.601 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:12.598+0000 7fcb3ad47640 1 --2- 172.21.15.169:0/4012084351 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb340ac6d0 0x7fcb3406eb70 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fcb24006010 tx=0x7fcb2402da70 comp rx=0 tx=0).stop 2023-10-26T01:35:12.601 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:12.598+0000 7fcb3ad47640 1 -- 172.21.15.169:0/4012084351 shutdown_connections 2023-10-26T01:35:12.601 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:12.598+0000 7fcb3ad47640 1 --2- 172.21.15.169:0/4012084351 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcb1806ae40 0x7fcb1806d300 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:12.601 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:12.598+0000 7fcb3ad47640 1 --2- 172.21.15.169:0/4012084351 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb340ac6d0 0x7fcb3406eb70 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:12.601 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:12.598+0000 7fcb3ad47640 1 --2- 172.21.15.169:0/4012084351 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcb340c9970 0x7fcb3406f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:12.602 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:12.598+0000 7fcb3ad47640 1 --2- 172.21.15.169:0/4012084351 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb3409a5c0 0x7fcb341565e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:12.602 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:12.598+0000 7fcb3ad47640 1 -- 172.21.15.169:0/4012084351 >> 172.21.15.169:0/4012084351 conn(0x7fcb340aedd0 msgr2=0x7fcb340aadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:35:12.602 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:12.598+0000 7fcb3ad47640 1 -- 172.21.15.169:0/4012084351 shutdown_connections 2023-10-26T01:35:12.602 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:12.599+0000 7fcb3ad47640 1 -- 172.21.15.169:0/4012084351 wait complete. 2023-10-26T01:35:12.619 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T01:30:32.200808+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '72', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T01:35:12.620 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T01:35:12.839 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:12.837+0000 7f2144afc640 1 -- 172.21.15.169:0/1623252276 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f212c0047c0 msgr2=0x7f212c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:12.839 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:12.837+0000 7f2144afc640 1 --2- 172.21.15.169:0/1623252276 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f212c0047c0 0x7f212c004c40 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f2128006fd0 tx=0x7f212c0057f0 comp rx=0 tx=0).stop 2023-10-26T01:35:12.840 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:12.837+0000 7f2144afc640 1 -- 172.21.15.169:0/1623252276 shutdown_connections 2023-10-26T01:35:12.840 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:12.837+0000 7f2144afc640 1 --2- 172.21.15.169:0/1623252276 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f212c0047c0 0x7f212c004c40 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:12.840 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:12.837+0000 7f2144afc640 1 -- 172.21.15.169:0/1623252276 >> 172.21.15.169:0/1623252276 conn(0x7f2140069930 msgr2=0x7f2140069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:35:12.841 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:12.837+0000 7f2144afc640 1 -- 172.21.15.169:0/1623252276 shutdown_connections 2023-10-26T01:35:12.841 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:12.837+0000 7f2144afc640 1 -- 172.21.15.169:0/1623252276 wait complete. 2023-10-26T01:35:12.844 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:12.839+0000 7f2144afc640 1 Processor -- start 2023-10-26T01:35:12.844 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:12.839+0000 7f2144afc640 1 -- start start 2023-10-26T01:35:12.844 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:12.840+0000 7f2144afc640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f212c0047c0 0x7f214014c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:12.844 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:12.840+0000 7f2144afc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2140070e90 0x7f2140071310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:12.844 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:12.840+0000 7f2144afc640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f214014fc70 0x7f21401500d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:12.845 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:12.840+0000 7f2144afc640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2140071850 con 0x7f212c0047c0 2023-10-26T01:35:12.845 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:12.840+0000 7f2144afc640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21400c8630 con 0x7f2140070e90 2023-10-26T01:35:12.845 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:12.840+0000 7f2144afc640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2140150610 con 0x7f214014fc70 2023-10-26T01:35:12.845 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:12.840+0000 7f213e59c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f212c0047c0 0x7f214014c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:12.845 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:12.840+0000 7f213e59c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f212c0047c0 0x7f214014c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:48712/0 (socket says 172.21.15.169:48712) 2023-10-26T01:35:12.846 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:12.840+0000 7f213e59c640 1 -- 172.21.15.169:0/2647362860 learned_addr learned my addr 172.21.15.169:0/2647362860 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:35:12.846 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:12.841+0000 7f213e59c640 1 -- 172.21.15.169:0/2647362860 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f214014fc70 msgr2=0x7f21401500d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:12.846 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:12.841+0000 7f213e59c640 1 --2- 172.21.15.169:0/2647362860 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f214014fc70 0x7f21401500d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:12.846 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:12.841+0000 7f213e59c640 1 -- 172.21.15.169:0/2647362860 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2140070e90 msgr2=0x7f2140071310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:12.846 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:12.841+0000 7f213e59c640 1 --2- 172.21.15.169:0/2647362860 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2140070e90 0x7f2140071310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:12.847 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:12.841+0000 7f213e59c640 1 -- 172.21.15.169:0/2647362860 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21400c7a20 con 0x7f212c0047c0 2023-10-26T01:35:12.847 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:12.841+0000 7f213e59c640 1 --2- 172.21.15.169:0/2647362860 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f212c0047c0 0x7f214014c770 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f213000d5b0 tx=0x7f213000da70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:35:12.847 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:12.841+0000 7f2136ffd640 1 -- 172.21.15.169:0/2647362860 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21300062e0 con 0x7f212c0047c0 2023-10-26T01:35:12.847 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:12.841+0000 7f2136ffd640 1 -- 172.21.15.169:0/2647362860 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2130006530 con 0x7f212c0047c0 2023-10-26T01:35:12.847 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:12.841+0000 7f2136ffd640 1 -- 172.21.15.169:0/2647362860 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2130003870 con 0x7f212c0047c0 2023-10-26T01:35:12.848 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:12.841+0000 7f2144afc640 1 -- 172.21.15.169:0/2647362860 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21400c7d00 con 0x7f212c0047c0 2023-10-26T01:35:12.848 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:12.842+0000 7f2144afc640 1 -- 172.21.15.169:0/2647362860 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21400c82c0 con 0x7f212c0047c0 2023-10-26T01:35:12.849 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:12.843+0000 7f2144afc640 1 -- 172.21.15.169:0/2647362860 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2140066af0 con 0x7f212c0047c0 2023-10-26T01:35:12.849 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:12.844+0000 7f2136ffd640 1 -- 172.21.15.169:0/2647362860 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f21300039d0 con 0x7f212c0047c0 2023-10-26T01:35:12.849 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:12.845+0000 7f2136ffd640 1 --2- 172.21.15.169:0/2647362860 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f212006ae40 0x7f212006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:12.849 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:12.845+0000 7f2136ffd640 1 -- 172.21.15.169:0/2647362860 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f2130089a80 con 0x7f212c0047c0 2023-10-26T01:35:12.850 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:12.845+0000 7f2137fff640 1 --2- 172.21.15.169:0/2647362860 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f212006ae40 0x7f212006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:12.850 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:12.846+0000 7f2137fff640 1 --2- 172.21.15.169:0/2647362860 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f212006ae40 0x7f212006d300 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f2128034000 tx=0x7f2128039040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:35:12.851 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:12.849+0000 7f2136ffd640 1 -- 172.21.15.169:0/2647362860 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2130054d90 con 0x7f212c0047c0 2023-10-26T01:35:13.043 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:13.041+0000 7f2144afc640 1 -- 172.21.15.169:0/2647362860 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f214014d750 con 0x7f212c0047c0 2023-10-26T01:35:13.043 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:13.042+0000 7f2136ffd640 1 -- 172.21.15.169:0/2647362860 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v75) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2130014020 con 0x7f212c0047c0 2023-10-26T01:35:13.044 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 1 2023-10-26T01:35:13.046 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:13.044+0000 7f2144afc640 1 -- 172.21.15.169:0/2647362860 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f212006ae40 msgr2=0x7f212006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:13.046 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:13.044+0000 7f2144afc640 1 --2- 172.21.15.169:0/2647362860 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f212006ae40 0x7f212006d300 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f2128034000 tx=0x7f2128039040 comp rx=0 tx=0).stop 2023-10-26T01:35:13.047 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:13.044+0000 7f2144afc640 1 -- 172.21.15.169:0/2647362860 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f212c0047c0 msgr2=0x7f214014c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:13.047 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:13.044+0000 7f2144afc640 1 --2- 172.21.15.169:0/2647362860 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f212c0047c0 0x7f214014c770 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f213000d5b0 tx=0x7f213000da70 comp rx=0 tx=0).stop 2023-10-26T01:35:13.047 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:13.044+0000 7f2144afc640 1 -- 172.21.15.169:0/2647362860 shutdown_connections 2023-10-26T01:35:13.047 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:13.044+0000 7f2144afc640 1 --2- 172.21.15.169:0/2647362860 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f212006ae40 0x7f212006d300 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:13.047 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:13.044+0000 7f2144afc640 1 --2- 172.21.15.169:0/2647362860 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f214014fc70 0x7f21401500d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:13.048 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:13.044+0000 7f2144afc640 1 --2- 172.21.15.169:0/2647362860 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2140070e90 0x7f2140071310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:13.048 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:13.044+0000 7f2144afc640 1 --2- 172.21.15.169:0/2647362860 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f212c0047c0 0x7f214014c770 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:13.048 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:13.044+0000 7f2144afc640 1 -- 172.21.15.169:0/2647362860 >> 172.21.15.169:0/2647362860 conn(0x7f2140069930 msgr2=0x7f214014d040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:35:13.048 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:13.044+0000 7f2144afc640 1 -- 172.21.15.169:0/2647362860 shutdown_connections 2023-10-26T01:35:13.048 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:13.045+0000 7f2144afc640 1 -- 172.21.15.169:0/2647362860 wait complete. 2023-10-26T01:35:13.064 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestClusterFull.test_full_fsync' 2023-10-26T01:35:13.286 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:13.284+0000 7f7aa450c640 1 -- 172.21.15.169:0/4213569498 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7a880047c0 msgr2=0x7f7a88004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:13.286 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:13.284+0000 7f7aa450c640 1 --2- 172.21.15.169:0/4213569498 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7a880047c0 0x7f7a88004c40 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f7a8c00c260 tx=0x7f7a880057f0 comp rx=0 tx=0).stop 2023-10-26T01:35:13.286 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:13.284+0000 7f7aa2aa9640 1 -- 172.21.15.169:0/4213569498 reap_dead start 2023-10-26T01:35:13.287 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:13.284+0000 7f7aa450c640 1 -- 172.21.15.169:0/4213569498 shutdown_connections 2023-10-26T01:35:13.287 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:13.284+0000 7f7aa450c640 1 -- 172.21.15.169:0/4213569498 >> 172.21.15.169:0/4213569498 conn(0x7f7a9c069930 msgr2=0x7f7a9c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:35:13.287 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:13.284+0000 7f7aa450c640 1 -- 172.21.15.169:0/4213569498 shutdown_connections 2023-10-26T01:35:13.287 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:13.284+0000 7f7aa450c640 1 -- 172.21.15.169:0/4213569498 wait complete. 2023-10-26T01:35:13.288 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:13.286+0000 7f7aa450c640 1 Processor -- start 2023-10-26T01:35:13.288 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:13.286+0000 7f7aa450c640 1 -- start start 2023-10-26T01:35:13.290 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:13.287+0000 7f7aa450c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7a880047c0 0x7f7a9c15bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:13.290 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:13.287+0000 7f7aa450c640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7a9c15d4f0 0x7f7a9c15e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:13.290 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:13.287+0000 7f7aa450c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7a9c14f430 0x7f7a9c15db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:13.290 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:13.287+0000 7f7aa450c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a9c15ecd0 con 0x7f7a880047c0 2023-10-26T01:35:13.291 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:13.287+0000 7f7aa450c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a9c15c450 con 0x7f7a9c14f430 2023-10-26T01:35:13.291 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:13.287+0000 7f7aa450c640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a9c15c730 con 0x7f7a9c15d4f0 2023-10-26T01:35:13.291 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:13.287+0000 7f7aa22a8640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7a880047c0 0x7f7a9c15bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:13.291 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:13.287+0000 7f7aa22a8640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7a880047c0 0x7f7a9c15bd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:48724/0 (socket says 172.21.15.169:48724) 2023-10-26T01:35:13.291 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:13.287+0000 7f7aa22a8640 1 -- 172.21.15.169:0/1848207001 learned_addr learned my addr 172.21.15.169:0/1848207001 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:35:13.291 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:13.287+0000 7f7aa1aa7640 1 --2- 172.21.15.169:0/1848207001 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7a9c15d4f0 0x7f7a9c15e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:13.292 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:13.287+0000 7f7aa22a8640 1 -- 172.21.15.169:0/1848207001 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7a9c15d4f0 msgr2=0x7f7a9c15e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:13.292 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:13.287+0000 7f7aa22a8640 1 --2- 172.21.15.169:0/1848207001 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7a9c15d4f0 0x7f7a9c15e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:13.292 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:13.287+0000 7f7aa22a8640 1 -- 172.21.15.169:0/1848207001 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7a9c14f430 msgr2=0x7f7a9c15db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:13.292 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:13.287+0000 7f7aa22a8640 1 --2- 172.21.15.169:0/1848207001 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7a9c14f430 0x7f7a9c15db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:13.292 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:13.287+0000 7f7aa22a8640 1 -- 172.21.15.169:0/1848207001 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a9c15ca10 con 0x7f7a880047c0 2023-10-26T01:35:13.293 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:13.288+0000 7f7aa22a8640 1 --2- 172.21.15.169:0/1848207001 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7a880047c0 0x7f7a9c15bd70 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f7a9800e990 tx=0x7f7a9800ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:35:13.293 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:13.288+0000 7f7a92ffd640 1 -- 172.21.15.169:0/1848207001 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a9800cb60 con 0x7f7a880047c0 2023-10-26T01:35:13.293 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:13.288+0000 7f7a92ffd640 1 -- 172.21.15.169:0/1848207001 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7a9800cdb0 con 0x7f7a880047c0 2023-10-26T01:35:13.293 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:13.288+0000 7f7a92ffd640 1 -- 172.21.15.169:0/1848207001 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a98005280 con 0x7f7a880047c0 2023-10-26T01:35:13.293 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:13.288+0000 7f7aa450c640 1 -- 172.21.15.169:0/1848207001 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a9c15ccf0 con 0x7f7a880047c0 2023-10-26T01:35:13.294 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:13.288+0000 7f7aa450c640 1 -- 172.21.15.169:0/1848207001 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a9c15e0c0 con 0x7f7a880047c0 2023-10-26T01:35:13.295 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:13.289+0000 7f7aa450c640 1 -- 172.21.15.169:0/1848207001 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a9c04db10 con 0x7f7a880047c0 2023-10-26T01:35:13.295 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:13.290+0000 7f7a92ffd640 1 -- 172.21.15.169:0/1848207001 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f7a980053e0 con 0x7f7a880047c0 2023-10-26T01:35:13.295 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:13.291+0000 7f7a92ffd640 1 --2- 172.21.15.169:0/1848207001 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7a8006ae70 0x7f7a8006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:13.295 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:13.292+0000 7f7a92ffd640 1 -- 172.21.15.169:0/1848207001 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f7a98089bd0 con 0x7f7a880047c0 2023-10-26T01:35:13.295 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:13.292+0000 7f7aa1aa7640 1 --2- 172.21.15.169:0/1848207001 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7a8006ae70 0x7f7a8006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:13.296 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:13.293+0000 7f7aa1aa7640 1 --2- 172.21.15.169:0/1848207001 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7a8006ae70 0x7f7a8006d330 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f7a8c034000 tx=0x7f7a8c039040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:35:13.298 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:13.296+0000 7f7a92ffd640 1 -- 172.21.15.169:0/1848207001 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7a98054ee0 con 0x7f7a880047c0 2023-10-26T01:35:13.548 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:13.547+0000 7f7aa450c640 1 -- 172.21.15.169:0/1848207001 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_fsync"]} v 0) v1 -- 0x7f7a9c06c990 con 0x7f7a880047c0 2023-10-26T01:35:14.162 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:14.160+0000 7f7a92ffd640 1 -- 172.21.15.169:0/1848207001 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_fsync"]}]=0 v242) v1 ==== 136+0+0 (secure 0 0 0) 0x7f7a9801f020 con 0x7f7a880047c0 2023-10-26T01:35:14.166 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:14.163+0000 7f7aa450c640 1 -- 172.21.15.169:0/1848207001 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7a8006ae70 msgr2=0x7f7a8006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:14.166 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:14.164+0000 7f7aa450c640 1 --2- 172.21.15.169:0/1848207001 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7a8006ae70 0x7f7a8006d330 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f7a8c034000 tx=0x7f7a8c039040 comp rx=0 tx=0).stop 2023-10-26T01:35:14.166 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:14.164+0000 7f7aa450c640 1 -- 172.21.15.169:0/1848207001 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7a880047c0 msgr2=0x7f7a9c15bd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:14.166 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:14.164+0000 7f7aa450c640 1 --2- 172.21.15.169:0/1848207001 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7a880047c0 0x7f7a9c15bd70 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f7a9800e990 tx=0x7f7a9800ee50 comp rx=0 tx=0).stop 2023-10-26T01:35:14.167 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:14.164+0000 7f7aa450c640 1 -- 172.21.15.169:0/1848207001 shutdown_connections 2023-10-26T01:35:14.167 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:14.164+0000 7f7aa450c640 1 --2- 172.21.15.169:0/1848207001 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7a8006ae70 0x7f7a8006d330 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:14.167 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:14.164+0000 7f7aa450c640 1 --2- 172.21.15.169:0/1848207001 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7a9c14f430 0x7f7a9c15db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:14.167 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:14.164+0000 7f7aa450c640 1 --2- 172.21.15.169:0/1848207001 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7a9c15d4f0 0x7f7a9c15e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:14.167 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:14.164+0000 7f7aa450c640 1 --2- 172.21.15.169:0/1848207001 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7a880047c0 0x7f7a9c15bd70 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:14.168 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:14.164+0000 7f7aa450c640 1 -- 172.21.15.169:0/1848207001 >> 172.21.15.169:0/1848207001 conn(0x7f7a9c069930 msgr2=0x7f7a9c0ae540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:35:14.168 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:14.164+0000 7f7aa450c640 1 -- 172.21.15.169:0/1848207001 shutdown_connections 2023-10-26T01:35:14.168 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:14.164+0000 7f7aa450c640 1 -- 172.21.15.169:0/1848207001 wait complete. 2023-10-26T01:35:14.187 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_objectstore 2023-10-26T01:35:14.412 INFO:teuthology.orchestra.run.smithi169.stdout:{"osd_objectstore":"memstore"} 2023-10-26T01:35:14.412 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_objectstore": "memstore" } 2023-10-26T01:35:14.413 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2023-10-26T01:35:14.621 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:14.619+0000 7f9762e1a640 1 -- 172.21.15.169:0/3703461726 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97480047c0 msgr2=0x7f9748004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:14.622 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:14.619+0000 7f9762e1a640 1 --2- 172.21.15.169:0/3703461726 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97480047c0 0x7f9748004c40 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f974c00c6e0 tx=0x7f97480057f0 comp rx=0 tx=0).stop 2023-10-26T01:35:14.622 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:14.619+0000 7f9762e1a640 1 -- 172.21.15.169:0/3703461726 shutdown_connections 2023-10-26T01:35:14.622 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:14.619+0000 7f9762e1a640 1 --2- 172.21.15.169:0/3703461726 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97480047c0 0x7f9748004c40 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:14.622 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:14.619+0000 7f9762e1a640 1 -- 172.21.15.169:0/3703461726 >> 172.21.15.169:0/3703461726 conn(0x7f975c0af0c0 msgr2=0x7f975c0af4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:35:14.623 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:14.619+0000 7f9762e1a640 1 -- 172.21.15.169:0/3703461726 shutdown_connections 2023-10-26T01:35:14.623 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:14.619+0000 7f9762e1a640 1 -- 172.21.15.169:0/3703461726 wait complete. 2023-10-26T01:35:14.623 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:14.621+0000 7f9762e1a640 1 Processor -- start 2023-10-26T01:35:14.624 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:14.622+0000 7f9762e1a640 1 -- start start 2023-10-26T01:35:14.625 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:14.623+0000 7f9762e1a640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97480047c0 0x7f975c1f2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:14.626 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:14.623+0000 7f9762e1a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f975c0cbc50 0x7f975c0cc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:14.626 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:14.623+0000 7f9762e1a640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f975c06bb30 0x7f975c06bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:14.626 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:14.623+0000 7f9762e1a640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f975c0cc5f0 con 0x7f97480047c0 2023-10-26T01:35:14.626 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:14.623+0000 7f9762e1a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f975c06c4f0 con 0x7f975c0cbc50 2023-10-26T01:35:14.627 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:14.623+0000 7f9762e1a640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f975c1f6820 con 0x7f975c06bb30 2023-10-26T01:35:14.627 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:14.623+0000 7f9761e18640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97480047c0 0x7f975c1f2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:14.627 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:14.623+0000 7f9761e18640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97480047c0 0x7f975c1f2ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:52032/0 (socket says 172.21.15.169:52032) 2023-10-26T01:35:14.627 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:14.623+0000 7f9761e18640 1 -- 172.21.15.169:0/3101799401 learned_addr learned my addr 172.21.15.169:0/3101799401 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:35:14.627 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:14.623+0000 7f9761e18640 1 -- 172.21.15.169:0/3101799401 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f975c06bb30 msgr2=0x7f975c06bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:14.628 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:14.623+0000 7f9761e18640 1 --2- 172.21.15.169:0/3101799401 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f975c06bb30 0x7f975c06bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:14.628 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:14.623+0000 7f9761e18640 1 -- 172.21.15.169:0/3101799401 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f975c0cbc50 msgr2=0x7f975c0cc0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:14.628 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:14.623+0000 7f9761e18640 1 --2- 172.21.15.169:0/3101799401 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f975c0cbc50 0x7f975c0cc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:14.628 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:14.623+0000 7f9761e18640 1 -- 172.21.15.169:0/3101799401 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f975c1f5c10 con 0x7f97480047c0 2023-10-26T01:35:14.628 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:14.624+0000 7f9761e18640 1 --2- 172.21.15.169:0/3101799401 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97480047c0 0x7f975c1f2ce0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f975800cd10 tx=0x7f975800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:35:14.629 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:14.624+0000 7f97527fc640 1 -- 172.21.15.169:0/3101799401 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9758004b20 con 0x7f97480047c0 2023-10-26T01:35:14.629 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:14.624+0000 7f97527fc640 1 -- 172.21.15.169:0/3101799401 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9758004d90 con 0x7f97480047c0 2023-10-26T01:35:14.629 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:14.624+0000 7f9762e1a640 1 -- 172.21.15.169:0/3101799401 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f975c1f5ef0 con 0x7f97480047c0 2023-10-26T01:35:14.629 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:14.624+0000 7f97527fc640 1 -- 172.21.15.169:0/3101799401 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9758005280 con 0x7f97480047c0 2023-10-26T01:35:14.629 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:14.624+0000 7f9762e1a640 1 -- 172.21.15.169:0/3101799401 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f975c1f64b0 con 0x7f97480047c0 2023-10-26T01:35:14.631 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:14.625+0000 7f9762e1a640 1 -- 172.21.15.169:0/3101799401 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f975c0affa0 con 0x7f97480047c0 2023-10-26T01:35:14.631 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:14.626+0000 7f97527fc640 1 -- 172.21.15.169:0/3101799401 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f97580053e0 con 0x7f97480047c0 2023-10-26T01:35:14.631 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:14.628+0000 7f97527fc640 1 --2- 172.21.15.169:0/3101799401 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f974006ae70 0x7f974006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:14.632 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:14.628+0000 7f97527fc640 1 -- 172.21.15.169:0/3101799401 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f9758089bc0 con 0x7f97480047c0 2023-10-26T01:35:14.632 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:14.628+0000 7f9761617640 1 --2- 172.21.15.169:0/3101799401 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f974006ae70 0x7f974006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:14.632 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:14.630+0000 7f9761617640 1 --2- 172.21.15.169:0/3101799401 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f974006ae70 0x7f974006d330 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f974c034000 tx=0x7f974c0036f0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:35:14.634 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:14.632+0000 7f97527fc640 1 -- 172.21.15.169:0/3101799401 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9758054ed0 con 0x7f97480047c0 2023-10-26T01:35:14.825 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:14.823+0000 7f9762e1a640 1 -- 172.21.15.169:0/3101799401 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "config dump"} v 0) v1 -- 0x7f975c1f6640 con 0x7f97480047c0 2023-10-26T01:35:14.826 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:14.824+0000 7f97527fc640 1 -- 172.21.15.169:0/3101799401 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v24) v1 ==== 59+0+1116 (secure 0 0 0) 0x7f975c1f6640 con 0x7f97480047c0 2023-10-26T01:35:14.826 INFO:teuthology.orchestra.run.smithi169.stdout:WHO MASK LEVEL OPTION VALUE RO 2023-10-26T01:35:14.826 INFO:teuthology.orchestra.run.smithi169.stdout:mon advanced mon_op_complaint_time 120 2023-10-26T01:35:14.827 INFO:teuthology.orchestra.run.smithi169.stdout:osd advanced osd_op_complaint_time 180.000000 2023-10-26T01:35:14.827 INFO:teuthology.orchestra.run.smithi169.stdout:mds advanced debug_mds 20/20 2023-10-26T01:35:14.827 INFO:teuthology.orchestra.run.smithi169.stdout:mds advanced debug_mds_balancer 20/20 2023-10-26T01:35:14.827 INFO:teuthology.orchestra.run.smithi169.stdout:mds advanced debug_ms 1/1 2023-10-26T01:35:14.828 INFO:teuthology.orchestra.run.smithi169.stdout:mds dev mds_debug_frag true 2023-10-26T01:35:14.828 INFO:teuthology.orchestra.run.smithi169.stdout:mds dev mds_debug_scatterstat true 2023-10-26T01:35:14.828 INFO:teuthology.orchestra.run.smithi169.stdout:mds advanced mds_op_complaint_time 180.000000 2023-10-26T01:35:14.828 INFO:teuthology.orchestra.run.smithi169.stdout:mds dev mds_verify_scatter true 2023-10-26T01:35:14.828 INFO:teuthology.orchestra.run.smithi169.stdout:mds advanced osd_op_complaint_time 180.000000 2023-10-26T01:35:14.829 INFO:teuthology.orchestra.run.smithi169.stdout:mds advanced rados_mon_op_timeout 900 2023-10-26T01:35:14.829 INFO:teuthology.orchestra.run.smithi169.stdout:mds advanced rados_osd_op_timeout 900 2023-10-26T01:35:14.829 INFO:teuthology.orchestra.run.smithi169.stdout:client advanced client_mount_timeout 600 2023-10-26T01:35:14.829 INFO:teuthology.orchestra.run.smithi169.stdout:client advanced debug_client 20/20 2023-10-26T01:35:14.829 INFO:teuthology.orchestra.run.smithi169.stdout:client advanced debug_ms 1/1 2023-10-26T01:35:14.830 INFO:teuthology.orchestra.run.smithi169.stdout:client advanced rados_mon_op_timeout 900 2023-10-26T01:35:14.830 INFO:teuthology.orchestra.run.smithi169.stdout:client advanced rados_osd_op_timeout 900 2023-10-26T01:35:14.831 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:14.827+0000 7f9762e1a640 1 -- 172.21.15.169:0/3101799401 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f974006ae70 msgr2=0x7f974006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:14.831 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:14.827+0000 7f9762e1a640 1 --2- 172.21.15.169:0/3101799401 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f974006ae70 0x7f974006d330 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f974c034000 tx=0x7f974c0036f0 comp rx=0 tx=0).stop 2023-10-26T01:35:14.831 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:14.827+0000 7f9762e1a640 1 -- 172.21.15.169:0/3101799401 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97480047c0 msgr2=0x7f975c1f2ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:14.831 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:14.827+0000 7f9762e1a640 1 --2- 172.21.15.169:0/3101799401 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97480047c0 0x7f975c1f2ce0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f975800cd10 tx=0x7f975800a620 comp rx=0 tx=0).stop 2023-10-26T01:35:14.832 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:14.827+0000 7f9762e1a640 1 -- 172.21.15.169:0/3101799401 shutdown_connections 2023-10-26T01:35:14.832 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:14.827+0000 7f9762e1a640 1 --2- 172.21.15.169:0/3101799401 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f974006ae70 0x7f974006d330 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:14.832 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:14.827+0000 7f9762e1a640 1 --2- 172.21.15.169:0/3101799401 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f975c06bb30 0x7f975c06bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:14.832 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:14.827+0000 7f9762e1a640 1 --2- 172.21.15.169:0/3101799401 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f975c0cbc50 0x7f975c0cc0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:14.832 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:14.827+0000 7f9762e1a640 1 --2- 172.21.15.169:0/3101799401 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97480047c0 0x7f975c1f2ce0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:14.833 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:14.827+0000 7f9762e1a640 1 -- 172.21.15.169:0/3101799401 >> 172.21.15.169:0/3101799401 conn(0x7f975c0af0c0 msgr2=0x7f975c096720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:35:14.833 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:14.827+0000 7f9762e1a640 1 -- 172.21.15.169:0/3101799401 shutdown_connections 2023-10-26T01:35:14.833 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:14.827+0000 7f9762e1a640 1 -- 172.21.15.169:0/3101799401 wait complete. 2023-10-26T01:35:14.847 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-26T01:35:15.073 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:15.070+0000 7f7ae2847640 1 -- 172.21.15.169:0/3148781647 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7adc06b9b0 msgr2=0x7f7adc0afff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:15.073 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:15.071+0000 7f7ae2847640 1 --2- 172.21.15.169:0/3148781647 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7adc06b9b0 0x7f7adc0afff0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f7ad4005fd0 tx=0x7f7ad402d400 comp rx=0 tx=0).stop 2023-10-26T01:35:15.073 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:15.071+0000 7f7ae2847640 1 -- 172.21.15.169:0/3148781647 shutdown_connections 2023-10-26T01:35:15.073 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:15.071+0000 7f7ae2847640 1 --2- 172.21.15.169:0/3148781647 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7adc06b9b0 0x7f7adc0afff0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:15.074 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:15.071+0000 7f7ae2847640 1 --2- 172.21.15.169:0/3148781647 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7adc1579d0 0x7f7adc15b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:15.074 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:15.071+0000 7f7ae2847640 1 -- 172.21.15.169:0/3148781647 >> 172.21.15.169:0/3148781647 conn(0x7f7adc0af110 msgr2=0x7f7adc0af520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:35:15.074 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:15.071+0000 7f7ae2847640 1 -- 172.21.15.169:0/3148781647 shutdown_connections 2023-10-26T01:35:15.074 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:15.071+0000 7f7ae2847640 1 -- 172.21.15.169:0/3148781647 wait complete. 2023-10-26T01:35:15.075 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:15.073+0000 7f7ae2847640 1 Processor -- start 2023-10-26T01:35:15.075 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:15.073+0000 7f7ae2847640 1 -- start start 2023-10-26T01:35:15.076 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:15.074+0000 7f7ae2847640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7adc06b9b0 0x7f7adc14f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:15.077 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:15.074+0000 7f7ae2847640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7adc1579d0 0x7f7adc150fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:15.077 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:15.074+0000 7f7ae2847640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7adc15bb90 0x7f7adc09a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:15.077 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:15.074+0000 7f7ae2847640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7adc0c9890 con 0x7f7adc06b9b0 2023-10-26T01:35:15.077 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:15.074+0000 7f7ae2847640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7adc14fbf0 con 0x7f7adc1579d0 2023-10-26T01:35:15.077 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:15.074+0000 7f7ae2847640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7adc0c9260 con 0x7f7adc15bb90 2023-10-26T01:35:15.078 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:15.074+0000 7f7ae1845640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7adc06b9b0 0x7f7adc14f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:15.078 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:15.074+0000 7f7ae1845640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7adc06b9b0 0x7f7adc14f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:52046/0 (socket says 172.21.15.169:52046) 2023-10-26T01:35:15.078 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:15.074+0000 7f7ae1845640 1 -- 172.21.15.169:0/3369750913 learned_addr learned my addr 172.21.15.169:0/3369750913 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:35:15.078 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:15.075+0000 7f7ae1845640 1 -- 172.21.15.169:0/3369750913 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7adc15bb90 msgr2=0x7f7adc09a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:15.078 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:15.075+0000 7f7ae1845640 1 --2- 172.21.15.169:0/3369750913 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7adc15bb90 0x7f7adc09a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:15.079 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:15.075+0000 7f7ae1845640 1 -- 172.21.15.169:0/3369750913 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7adc1579d0 msgr2=0x7f7adc150fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:15.079 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:15.075+0000 7f7ae1845640 1 --2- 172.21.15.169:0/3369750913 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7adc1579d0 0x7f7adc150fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:15.079 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:15.075+0000 7f7ae1845640 1 -- 172.21.15.169:0/3369750913 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ad8009490 con 0x7f7adc06b9b0 2023-10-26T01:35:15.080 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:15.075+0000 7f7ae1845640 1 --2- 172.21.15.169:0/3369750913 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7adc06b9b0 0x7f7adc14f6b0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f7ad800ea90 tx=0x7f7ad800ef50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:35:15.080 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:15.075+0000 7f7ad27fc640 1 -- 172.21.15.169:0/3369750913 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad800ce40 con 0x7f7adc06b9b0 2023-10-26T01:35:15.081 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:15.075+0000 7f7ad27fc640 1 -- 172.21.15.169:0/3369750913 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7ad8004020 con 0x7f7adc06b9b0 2023-10-26T01:35:15.081 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:15.075+0000 7f7ae2847640 1 -- 172.21.15.169:0/3369750913 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ad400a950 con 0x7f7adc06b9b0 2023-10-26T01:35:15.081 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:15.076+0000 7f7ad27fc640 1 -- 172.21.15.169:0/3369750913 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad8004430 con 0x7f7adc06b9b0 2023-10-26T01:35:15.081 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:15.076+0000 7f7ae2847640 1 -- 172.21.15.169:0/3369750913 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7adc150100 con 0x7f7adc06b9b0 2023-10-26T01:35:15.081 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:15.077+0000 7f7ae2847640 1 -- 172.21.15.169:0/3369750913 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7adc073c10 con 0x7f7adc06b9b0 2023-10-26T01:35:15.081 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:15.078+0000 7f7ad27fc640 1 -- 172.21.15.169:0/3369750913 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f7ad80035f0 con 0x7f7adc06b9b0 2023-10-26T01:35:15.082 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:15.079+0000 7f7ad27fc640 1 --2- 172.21.15.169:0/3369750913 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7ac006ae40 0x7f7ac006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:15.082 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:15.079+0000 7f7ad27fc640 1 -- 172.21.15.169:0/3369750913 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f7ad8014070 con 0x7f7adc06b9b0 2023-10-26T01:35:15.082 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:15.079+0000 7f7ae1044640 1 --2- 172.21.15.169:0/3369750913 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7ac006ae40 0x7f7ac006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:15.083 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:15.080+0000 7f7ae1044640 1 --2- 172.21.15.169:0/3369750913 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7ac006ae40 0x7f7ac006d300 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f7acc000f60 tx=0x7f7acc007a30 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:35:15.086 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:15.083+0000 7f7ad27fc640 1 -- 172.21.15.169:0/3369750913 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7ad8055330 con 0x7f7adc06b9b0 2023-10-26T01:35:15.276 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:15.275+0000 7f7ae2847640 1 -- 172.21.15.169:0/3369750913 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f7adc06b080 con 0x7f7adc06b9b0 2023-10-26T01:35:15.282 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:15.280+0000 7f7ad27fc640 1 -- 172.21.15.169:0/3369750913 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v25)=0 v25) v1 ==== 122+0+0 (secure 0 0 0) 0x7f7ad8058950 con 0x7f7adc06b9b0 2023-10-26T01:35:15.285 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:15.282+0000 7f7ae2847640 1 -- 172.21.15.169:0/3369750913 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7ac006ae40 msgr2=0x7f7ac006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:15.285 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:15.283+0000 7f7ae2847640 1 --2- 172.21.15.169:0/3369750913 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7ac006ae40 0x7f7ac006d300 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f7acc000f60 tx=0x7f7acc007a30 comp rx=0 tx=0).stop 2023-10-26T01:35:15.285 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:15.283+0000 7f7ae2847640 1 -- 172.21.15.169:0/3369750913 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7adc06b9b0 msgr2=0x7f7adc14f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:15.286 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:15.283+0000 7f7ae2847640 1 --2- 172.21.15.169:0/3369750913 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7adc06b9b0 0x7f7adc14f6b0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f7ad800ea90 tx=0x7f7ad800ef50 comp rx=0 tx=0).stop 2023-10-26T01:35:15.286 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:15.283+0000 7f7ae2847640 1 -- 172.21.15.169:0/3369750913 shutdown_connections 2023-10-26T01:35:15.286 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:15.283+0000 7f7ae2847640 1 --2- 172.21.15.169:0/3369750913 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7ac006ae40 0x7f7ac006d300 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:15.286 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:15.283+0000 7f7ae2847640 1 --2- 172.21.15.169:0/3369750913 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7adc15bb90 0x7f7adc09a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:15.286 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:15.283+0000 7f7ae2847640 1 --2- 172.21.15.169:0/3369750913 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7adc1579d0 0x7f7adc150fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:15.287 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:15.283+0000 7f7ae2847640 1 --2- 172.21.15.169:0/3369750913 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7adc06b9b0 0x7f7adc14f6b0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:15.287 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:15.283+0000 7f7ae2847640 1 -- 172.21.15.169:0/3369750913 >> 172.21.15.169:0/3369750913 conn(0x7f7adc0af110 msgr2=0x7f7adc0b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:35:15.287 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:15.283+0000 7f7ae2847640 1 -- 172.21.15.169:0/3369750913 shutdown_connections 2023-10-26T01:35:15.290 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:15.283+0000 7f7ae2847640 1 -- 172.21.15.169:0/3369750913 wait complete. 2023-10-26T01:35:15.302 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-26T01:35:15.303 DEBUG:teuthology.orchestra.run.smithi169:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-26T01:35:15.402 DEBUG:teuthology.orchestra.run.smithi170:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-26T01:35:15.527 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-26T01:35:15.528 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:35:15.528 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:35:15.590 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054301,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-26T01:35:15.591 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-26T01:35:15.591 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:35:15.653 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054303,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-26T01:35:15.653 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:35:15.654 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:35:15.714 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054301,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-26T01:35:15.715 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-26T01:35:15.715 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:35:15.773 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054303,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-26T01:35:15.774 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-26T01:35:15.774 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T01:35:16.007 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:16.004+0000 7fdf6ca25640 1 -- 172.21.15.169:0/2635796409 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdf680c9970 msgr2=0x7fdf68098a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:16.007 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:16.004+0000 7fdf6ca25640 1 --2- 172.21.15.169:0/2635796409 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdf680c9970 0x7fdf68098a60 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fdf50005fd0 tx=0x7fdf5002bd90 comp rx=0 tx=0).stop 2023-10-26T01:35:16.007 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:16.005+0000 7fdf6ca25640 1 -- 172.21.15.169:0/2635796409 shutdown_connections 2023-10-26T01:35:16.007 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:16.005+0000 7fdf6ca25640 1 --2- 172.21.15.169:0/2635796409 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdf6809a5c0 0x7fdf6809aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:16.008 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:16.005+0000 7fdf6ca25640 1 --2- 172.21.15.169:0/2635796409 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdf680c9970 0x7fdf68098a60 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:16.008 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:16.005+0000 7fdf6ca25640 1 -- 172.21.15.169:0/2635796409 >> 172.21.15.169:0/2635796409 conn(0x7fdf680aedd0 msgr2=0x7fdf680af1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:35:16.008 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:16.005+0000 7fdf6ca25640 1 -- 172.21.15.169:0/2635796409 shutdown_connections 2023-10-26T01:35:16.008 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:16.005+0000 7fdf6ca25640 1 -- 172.21.15.169:0/2635796409 wait complete. 2023-10-26T01:35:16.009 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:16.006+0000 7fdf6ca25640 1 Processor -- start 2023-10-26T01:35:16.009 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:16.007+0000 7fdf6ca25640 1 -- start start 2023-10-26T01:35:16.010 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:16.008+0000 7fdf6ca25640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdf6809a5c0 0x7fdf68140f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:16.010 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:16.008+0000 7fdf6ca25640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdf680ac6d0 0x7fdf680c6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:16.010 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:16.008+0000 7fdf6ca25640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdf680c9970 0x7fdf680c68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:16.010 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:16.008+0000 7fdf6ca25640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf680b1ed0 con 0x7fdf680ac6d0 2023-10-26T01:35:16.011 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:16.008+0000 7fdf6ca25640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf680b21b0 con 0x7fdf680c9970 2023-10-26T01:35:16.011 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:16.008+0000 7fdf6ca25640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf680b2490 con 0x7fdf6809a5c0 2023-10-26T01:35:16.012 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:16.008+0000 7fdf5ffff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdf680ac6d0 0x7fdf680c6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:16.012 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:16.008+0000 7fdf5ffff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdf680ac6d0 0x7fdf680c6370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:52072/0 (socket says 172.21.15.169:52072) 2023-10-26T01:35:16.012 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:16.008+0000 7fdf5ffff640 1 -- 172.21.15.169:0/3888318152 learned_addr learned my addr 172.21.15.169:0/3888318152 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:35:16.016 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:16.008+0000 7fdf677fe640 1 --2- 172.21.15.169:0/3888318152 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdf6809a5c0 0x7fdf68140f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:16.016 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:16.008+0000 7fdf5ffff640 1 -- 172.21.15.169:0/3888318152 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdf6809a5c0 msgr2=0x7fdf68140f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:16.017 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:16.008+0000 7fdf5ffff640 1 --2- 172.21.15.169:0/3888318152 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdf6809a5c0 0x7fdf68140f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:16.017 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:16.008+0000 7fdf5ffff640 1 -- 172.21.15.169:0/3888318152 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdf680c9970 msgr2=0x7fdf680c68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:16.017 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:16.008+0000 7fdf5ffff640 1 --2- 172.21.15.169:0/3888318152 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdf680c9970 0x7fdf680c68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:16.018 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:16.008+0000 7fdf5ffff640 1 -- 172.21.15.169:0/3888318152 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf50009560 con 0x7fdf680ac6d0 2023-10-26T01:35:16.018 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:16.009+0000 7fdf5ffff640 1 --2- 172.21.15.169:0/3888318152 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdf680ac6d0 0x7fdf680c6370 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fdf50009850 tx=0x7fdf5002db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:35:16.018 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:16.009+0000 7fdf64ff9640 1 -- 172.21.15.169:0/3888318152 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf500035f0 con 0x7fdf680ac6d0 2023-10-26T01:35:16.018 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:16.009+0000 7fdf64ff9640 1 -- 172.21.15.169:0/3888318152 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fdf50003750 con 0x7fdf680ac6d0 2023-10-26T01:35:16.018 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:16.009+0000 7fdf64ff9640 1 -- 172.21.15.169:0/3888318152 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf50003b00 con 0x7fdf680ac6d0 2023-10-26T01:35:16.019 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:16.009+0000 7fdf6ca25640 1 -- 172.21.15.169:0/3888318152 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf680b2770 con 0x7fdf680ac6d0 2023-10-26T01:35:16.019 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:16.009+0000 7fdf6ca25640 1 -- 172.21.15.169:0/3888318152 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf680b1150 con 0x7fdf680ac6d0 2023-10-26T01:35:16.019 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:16.010+0000 7fdf6ca25640 1 -- 172.21.15.169:0/3888318152 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf681599a0 con 0x7fdf680ac6d0 2023-10-26T01:35:16.019 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:16.011+0000 7fdf64ff9640 1 -- 172.21.15.169:0/3888318152 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fdf50003c60 con 0x7fdf680ac6d0 2023-10-26T01:35:16.019 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:16.012+0000 7fdf64ff9640 1 --2- 172.21.15.169:0/3888318152 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fdf4806ae40 0x7fdf4806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:16.020 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:16.012+0000 7fdf64ff9640 1 -- 172.21.15.169:0/3888318152 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fdf500abe30 con 0x7fdf680ac6d0 2023-10-26T01:35:16.020 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:16.013+0000 7fdf677fe640 1 --2- 172.21.15.169:0/3888318152 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fdf4806ae40 0x7fdf4806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:16.020 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:16.014+0000 7fdf677fe640 1 --2- 172.21.15.169:0/3888318152 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fdf4806ae40 0x7fdf4806d300 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fdf58003d90 tx=0x7fdf58009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:35:16.020 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:16.017+0000 7fdf64ff9640 1 -- 172.21.15.169:0/3888318152 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdf681599a0 con 0x7fdf680ac6d0 2023-10-26T01:35:16.246 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:16.244+0000 7fdf6ca25640 1 -- 172.21.15.169:0/3888318152 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdf680ca410 con 0x7fdf680ac6d0 2023-10-26T01:35:16.247 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:16.245+0000 7fdf64ff9640 1 -- 172.21.15.169:0/3888318152 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+3708 (secure 0 0 0) 0x7fdf680ca410 con 0x7fdf680ac6d0 2023-10-26T01:35:16.248 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:35:16.248 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":34,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4976,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/2977083370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2977083370},{"type":"v1","addr":"172.21.15.170:6837","nonce":2977083370}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":5176,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/1598214042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1598214042},{"type":"v1","addr":"172.21.15.170:6835","nonce":1598214042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":6087,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/406582181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":406582181},{"type":"v1","addr":"172.21.15.169:6833","nonce":406582181}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26},{"gid":6402,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1761593386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1761593386},{"type":"v1","addr":"172.21.15.169:6835","nonce":1761593386}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":34}],"filesystems":[]} 2023-10-26T01:35:16.248 INFO:teuthology.orchestra.run.smithi169.stderr:dumped fsmap epoch 34 2023-10-26T01:35:16.250 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:16.248+0000 7fdf6ca25640 1 -- 172.21.15.169:0/3888318152 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fdf4806ae40 msgr2=0x7fdf4806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:16.250 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:16.248+0000 7fdf6ca25640 1 --2- 172.21.15.169:0/3888318152 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fdf4806ae40 0x7fdf4806d300 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fdf58003d90 tx=0x7fdf58009040 comp rx=0 tx=0).stop 2023-10-26T01:35:16.250 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:16.248+0000 7fdf6ca25640 1 -- 172.21.15.169:0/3888318152 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdf680ac6d0 msgr2=0x7fdf680c6370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:16.250 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:16.248+0000 7fdf6ca25640 1 --2- 172.21.15.169:0/3888318152 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdf680ac6d0 0x7fdf680c6370 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fdf50009850 tx=0x7fdf5002db10 comp rx=0 tx=0).stop 2023-10-26T01:35:16.251 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:16.248+0000 7fdf6ca25640 1 -- 172.21.15.169:0/3888318152 shutdown_connections 2023-10-26T01:35:16.251 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:16.248+0000 7fdf6ca25640 1 --2- 172.21.15.169:0/3888318152 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fdf4806ae40 0x7fdf4806d300 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:16.251 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:16.248+0000 7fdf6ca25640 1 --2- 172.21.15.169:0/3888318152 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdf680ac6d0 0x7fdf680c6370 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:16.251 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:16.248+0000 7fdf6ca25640 1 --2- 172.21.15.169:0/3888318152 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdf680c9970 0x7fdf680c68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:16.251 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:16.248+0000 7fdf6ca25640 1 --2- 172.21.15.169:0/3888318152 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdf6809a5c0 0x7fdf68140f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:16.252 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:16.248+0000 7fdf6ca25640 1 -- 172.21.15.169:0/3888318152 >> 172.21.15.169:0/3888318152 conn(0x7fdf680aedd0 msgr2=0x7fdf680aa0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:35:16.252 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:16.248+0000 7fdf6ca25640 1 -- 172.21.15.169:0/3888318152 shutdown_connections 2023-10-26T01:35:16.252 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:16.248+0000 7fdf6ca25640 1 -- 172.21.15.169:0/3888318152 wait complete. 2023-10-26T01:35:16.269 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-26T01:35:16.270 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-26T01:35:16.270 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-26T01:35:16.270 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T01:35:16.271 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-26T01:35:16.271 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-26T01:35:16.271 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-26T01:35:16.271 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T01:35:16.272 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-26T01:35:16.272 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-26T01:35:16.272 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-26T01:35:16.272 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T01:35:16.273 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-26T01:35:16.273 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-26T01:35:16.273 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-26T01:35:16.274 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T01:35:16.294 INFO:tasks.ceph.mds.c:Stopped 2023-10-26T01:35:16.294 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-26T01:35:16.295 INFO:tasks.ceph.mds.d:Stopped 2023-10-26T01:35:16.296 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-26T01:35:16.302 INFO:tasks.ceph.mds.b:Stopped 2023-10-26T01:35:16.302 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-26T01:35:16.343 INFO:tasks.ceph.mds.c:Started 2023-10-26T01:35:16.343 DEBUG:teuthology.parallel:result is None 2023-10-26T01:35:16.350 INFO:tasks.ceph.mds.d:Started 2023-10-26T01:35:16.351 DEBUG:teuthology.parallel:result is None 2023-10-26T01:35:16.358 INFO:tasks.ceph.mds.b:Started 2023-10-26T01:35:16.359 DEBUG:teuthology.parallel:result is None 2023-10-26T01:35:16.526 INFO:tasks.ceph.mds.c.smithi169.stdout:starting mds.c at 2023-10-26T01:35:16.565 INFO:tasks.ceph.mds.d.smithi170.stdout:starting mds.d at 2023-10-26T01:35:16.565 INFO:tasks.ceph.mds.b.smithi170.stdout:starting mds.b at 2023-10-26T01:35:22.293 INFO:tasks.ceph.mds.a:Stopped 2023-10-26T01:35:22.294 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-26T01:35:22.303 INFO:tasks.ceph.mds.a:Started 2023-10-26T01:35:22.304 DEBUG:teuthology.parallel:result is None 2023-10-26T01:35:22.304 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-26T01:35:22.469 INFO:tasks.ceph.mds.a.smithi169.stdout:starting mds.a at 2023-10-26T01:35:22.505 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:22.503+0000 7ff1a404f640 1 -- 172.21.15.169:0/2028960979 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff19c15bbb0 msgr2=0x7ff19c0b05a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:22.505 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:22.503+0000 7ff1a404f640 1 --2- 172.21.15.169:0/2028960979 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff19c15bbb0 0x7ff19c0b05a0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7ff19c0b0570 tx=0x7ff19402c260 comp rx=0 tx=0).stop 2023-10-26T01:35:22.505 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:22.503+0000 7ff1a404f640 1 -- 172.21.15.169:0/2028960979 shutdown_connections 2023-10-26T01:35:22.506 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:22.503+0000 7ff1a404f640 1 --2- 172.21.15.169:0/2028960979 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff19c15bbb0 0x7ff19c0b05a0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:22.506 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:22.503+0000 7ff1a404f640 1 --2- 172.21.15.169:0/2028960979 >> v2:172.21.15.170:3300/0 conn(0x7ff19c155360 0x7ff19c0cd7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:22.506 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:22.503+0000 7ff1a404f640 1 -- 172.21.15.169:0/2028960979 >> 172.21.15.169:0/2028960979 conn(0x7ff19c069930 msgr2=0x7ff19c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:35:22.506 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:22.503+0000 7ff1a404f640 1 -- 172.21.15.169:0/2028960979 shutdown_connections 2023-10-26T01:35:22.506 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:22.503+0000 7ff1a404f640 1 -- 172.21.15.169:0/2028960979 wait complete. 2023-10-26T01:35:22.507 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:22.505+0000 7ff1a404f640 1 Processor -- start 2023-10-26T01:35:22.508 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:22.505+0000 7ff1a404f640 1 -- start start 2023-10-26T01:35:22.508 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:22.506+0000 7ff1a404f640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff19c0cdd20 0x7ff19c096070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:22.508 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:22.506+0000 7ff1a404f640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff19c155360 0x7ff19c0cc360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:22.508 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:22.506+0000 7ff1a404f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff19c15bbb0 0x7ff19c0cc8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:22.509 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:22.506+0000 7ff1a404f640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff19c0ad850 con 0x7ff19c155360 2023-10-26T01:35:22.509 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:22.506+0000 7ff1a404f640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff19c0adaf0 con 0x7ff19c15bbb0 2023-10-26T01:35:22.509 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:22.506+0000 7ff1a404f640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff19c0addd0 con 0x7ff19c0cdd20 2023-10-26T01:35:22.509 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:22.506+0000 7ff1a15ea640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff19c155360 0x7ff19c0cc360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:22.509 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:22.506+0000 7ff1a15ea640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff19c155360 0x7ff19c0cc360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:52142/0 (socket says 172.21.15.169:52142) 2023-10-26T01:35:22.510 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:22.506+0000 7ff1a15ea640 1 -- 172.21.15.169:0/1985059188 learned_addr learned my addr 172.21.15.169:0/1985059188 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:35:22.511 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:22.506+0000 7ff1a15ea640 1 -- 172.21.15.169:0/1985059188 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff19c0cdd20 msgr2=0x7ff19c096070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:22.511 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:22.506+0000 7ff1a1deb640 1 --2- 172.21.15.169:0/1985059188 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff19c0cdd20 0x7ff19c096070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:22.511 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:22.506+0000 7ff1a15ea640 1 --2- 172.21.15.169:0/1985059188 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff19c0cdd20 0x7ff19c096070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:22.511 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:22.506+0000 7ff1a15ea640 1 -- 172.21.15.169:0/1985059188 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff19c15bbb0 msgr2=0x7ff19c0cc8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:22.512 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:22.506+0000 7ff1a15ea640 1 --2- 172.21.15.169:0/1985059188 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff19c15bbb0 0x7ff19c0cc8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:22.512 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:22.507+0000 7ff1a15ea640 1 -- 172.21.15.169:0/1985059188 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff194009900 con 0x7ff19c155360 2023-10-26T01:35:22.512 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:22.507+0000 7ff1a15ea640 1 --2- 172.21.15.169:0/1985059188 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff19c155360 0x7ff19c0cc360 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7ff18c00c410 tx=0x7ff18c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:35:22.512 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:22.507+0000 7ff1927fc640 1 -- 172.21.15.169:0/1985059188 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff18c0035f0 con 0x7ff19c155360 2023-10-26T01:35:22.512 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:22.507+0000 7ff1927fc640 1 -- 172.21.15.169:0/1985059188 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff18c003750 con 0x7ff19c155360 2023-10-26T01:35:22.513 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:22.507+0000 7ff1927fc640 1 -- 172.21.15.169:0/1985059188 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff18c003ba0 con 0x7ff19c155360 2023-10-26T01:35:22.513 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:22.507+0000 7ff1a404f640 1 -- 172.21.15.169:0/1985059188 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff19c0ae0b0 con 0x7ff19c155360 2023-10-26T01:35:22.513 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:22.507+0000 7ff1a404f640 1 -- 172.21.15.169:0/1985059188 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff19c096bb0 con 0x7ff19c155360 2023-10-26T01:35:22.513 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:22.508+0000 7ff1a404f640 1 -- 172.21.15.169:0/1985059188 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff19c0ae5d0 con 0x7ff19c155360 2023-10-26T01:35:22.513 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:22.509+0000 7ff1927fc640 1 -- 172.21.15.169:0/1985059188 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ff18c003d00 con 0x7ff19c155360 2023-10-26T01:35:22.514 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:22.510+0000 7ff1927fc640 1 --2- 172.21.15.169:0/1985059188 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff18006ae40 0x7ff18006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:22.514 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:22.510+0000 7ff1927fc640 1 -- 172.21.15.169:0/1985059188 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 4943+0+0 (secure 0 0 0) 0x7ff18c088700 con 0x7ff19c155360 2023-10-26T01:35:22.515 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:22.511+0000 7ff1a1deb640 1 --2- 172.21.15.169:0/1985059188 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff18006ae40 0x7ff18006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:22.515 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:22.512+0000 7ff1a1deb640 1 --2- 172.21.15.169:0/1985059188 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff18006ae40 0x7ff18006d300 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7ff19c06d8b0 tx=0x7ff198008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:35:22.517 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:22.515+0000 7ff1927fc640 1 -- 172.21.15.169:0/1985059188 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff19c0ae5d0 con 0x7ff19c155360 2023-10-26T01:35:22.718 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:22.716+0000 7ff1a404f640 1 -- 172.21.15.169:0/1985059188 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7ff19c0ae5d0 con 0x7ff19c155360 2023-10-26T01:35:22.763 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2023-10-26T01:35:23.981 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:23.979+0000 7ff1927fc640 1 -- 172.21.15.169:0/1985059188 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v80) v1 ==== 97+0+0 (secure 0 0 0) 0x7ff19c0ae5d0 con 0x7ff19c155360 2023-10-26T01:35:23.981 INFO:teuthology.orchestra.run.smithi169.stderr: removed all blocklist entries 2023-10-26T01:35:23.984 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:23.982+0000 7ff1a404f640 1 -- 172.21.15.169:0/1985059188 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff18006ae40 msgr2=0x7ff18006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:23.985 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:23.982+0000 7ff1a404f640 1 --2- 172.21.15.169:0/1985059188 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff18006ae40 0x7ff18006d300 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7ff19c06d8b0 tx=0x7ff198008040 comp rx=0 tx=0).stop 2023-10-26T01:35:23.985 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:23.982+0000 7ff1a404f640 1 -- 172.21.15.169:0/1985059188 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff19c155360 msgr2=0x7ff19c0cc360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:23.985 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:23.982+0000 7ff1a404f640 1 --2- 172.21.15.169:0/1985059188 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff19c155360 0x7ff19c0cc360 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7ff18c00c410 tx=0x7ff18c00c8d0 comp rx=0 tx=0).stop 2023-10-26T01:35:23.985 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:23.983+0000 7ff1a404f640 1 -- 172.21.15.169:0/1985059188 shutdown_connections 2023-10-26T01:35:23.986 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:23.983+0000 7ff1a404f640 1 --2- 172.21.15.169:0/1985059188 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff18006ae40 0x7ff18006d300 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:23.986 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:23.983+0000 7ff1a404f640 1 --2- 172.21.15.169:0/1985059188 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff19c155360 0x7ff19c0cc360 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:23.986 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:23.983+0000 7ff1a404f640 1 --2- 172.21.15.169:0/1985059188 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff19c15bbb0 0x7ff19c0cc8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:23.986 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:23.983+0000 7ff1a404f640 1 --2- 172.21.15.169:0/1985059188 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff19c0cdd20 0x7ff19c096070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:23.986 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:23.983+0000 7ff1a404f640 1 -- 172.21.15.169:0/1985059188 >> 172.21.15.169:0/1985059188 conn(0x7ff19c069930 msgr2=0x7ff19c15a670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:35:23.986 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:23.983+0000 7ff1a404f640 1 -- 172.21.15.169:0/1985059188 shutdown_connections 2023-10-26T01:35:23.987 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:23.983+0000 7ff1a404f640 1 -- 172.21.15.169:0/1985059188 wait complete. 2023-10-26T01:35:24.004 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-26T01:35:24.228 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:24.226+0000 7f53e9b5e640 1 -- 172.21.15.169:0/3094980838 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53e40cdd80 msgr2=0x7f53e40b07d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:24.229 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:24.226+0000 7f53e9b5e640 1 --2- 172.21.15.169:0/3094980838 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53e40cdd80 0x7f53e40b07d0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f53e406bff0 tx=0x7f53dc02be40 comp rx=0 tx=0).stop 2023-10-26T01:35:24.229 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:24.226+0000 7f53e9b5e640 1 -- 172.21.15.169:0/3094980838 shutdown_connections 2023-10-26T01:35:24.229 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:24.226+0000 7f53e9b5e640 1 --2- 172.21.15.169:0/3094980838 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53e40cdd80 0x7f53e40b07d0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:24.229 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:24.226+0000 7f53e9b5e640 1 --2- 172.21.15.169:0/3094980838 >> v2:172.21.15.170:3300/0 conn(0x7f53e415b610 0x7f53e406ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:24.229 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:24.226+0000 7f53e9b5e640 1 -- 172.21.15.169:0/3094980838 >> 172.21.15.169:0/3094980838 conn(0x7f53e4069930 msgr2=0x7f53e4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:35:24.229 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:24.227+0000 7f53e9b5e640 1 -- 172.21.15.169:0/3094980838 shutdown_connections 2023-10-26T01:35:24.230 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:24.227+0000 7f53e9b5e640 1 -- 172.21.15.169:0/3094980838 wait complete. 2023-10-26T01:35:24.230 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:24.228+0000 7f53e9b5e640 1 Processor -- start 2023-10-26T01:35:24.230 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:24.228+0000 7f53e9b5e640 1 -- start start 2023-10-26T01:35:24.232 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:24.229+0000 7f53e9b5e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53e40ac4e0 0x7f53e41436c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:24.232 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:24.229+0000 7f53e9b5e640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f53e40cdd80 0x7f53e4156170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:24.232 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:24.229+0000 7f53e9b5e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f53e415b610 0x7f53e41566b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:24.232 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:24.229+0000 7f53e9b5e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e4143c00 con 0x7f53e40ac4e0 2023-10-26T01:35:24.233 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:24.229+0000 7f53e9b5e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e4152700 con 0x7f53e415b610 2023-10-26T01:35:24.233 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:24.229+0000 7f53e9b5e640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53e41529e0 con 0x7f53e40cdd80 2023-10-26T01:35:24.233 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:24.230+0000 7f53e37fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53e40ac4e0 0x7f53e41436c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:24.233 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:24.230+0000 7f53e37fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53e40ac4e0 0x7f53e41436c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:56800/0 (socket says 172.21.15.169:56800) 2023-10-26T01:35:24.233 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:24.230+0000 7f53e37fe640 1 -- 172.21.15.169:0/1660505857 learned_addr learned my addr 172.21.15.169:0/1660505857 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:35:24.233 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:24.230+0000 7f53e3fff640 1 --2- 172.21.15.169:0/1660505857 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f53e415b610 0x7f53e41566b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:24.234 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:24.230+0000 7f53e37fe640 1 -- 172.21.15.169:0/1660505857 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f53e40cdd80 msgr2=0x7f53e4156170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:24.234 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:24.230+0000 7f53daffd640 1 --2- 172.21.15.169:0/1660505857 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f53e40cdd80 0x7f53e4156170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:24.234 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:24.230+0000 7f53e37fe640 1 --2- 172.21.15.169:0/1660505857 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f53e40cdd80 0x7f53e4156170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:24.234 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:24.230+0000 7f53e37fe640 1 -- 172.21.15.169:0/1660505857 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f53e415b610 msgr2=0x7f53e41566b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:24.234 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:24.230+0000 7f53e37fe640 1 --2- 172.21.15.169:0/1660505857 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f53e415b610 0x7f53e41566b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:24.235 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:24.230+0000 7f53e37fe640 1 -- 172.21.15.169:0/1660505857 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53dc009560 con 0x7f53e40ac4e0 2023-10-26T01:35:24.235 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:24.230+0000 7f53e3fff640 1 --2- 172.21.15.169:0/1660505857 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f53e415b610 0x7f53e41566b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:35:24.235 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:24.230+0000 7f53e37fe640 1 --2- 172.21.15.169:0/1660505857 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53e40ac4e0 0x7f53e41436c0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f53d400cce0 tx=0x7f53d400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:35:24.236 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:24.231+0000 7f53e0ff9640 1 -- 172.21.15.169:0/1660505857 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d400f450 con 0x7f53e40ac4e0 2023-10-26T01:35:24.236 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:24.231+0000 7f53e0ff9640 1 -- 172.21.15.169:0/1660505857 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f53d400f6a0 con 0x7f53e40ac4e0 2023-10-26T01:35:24.236 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:24.231+0000 7f53e0ff9640 1 -- 172.21.15.169:0/1660505857 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d400fab0 con 0x7f53e40ac4e0 2023-10-26T01:35:24.236 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:24.231+0000 7f53e9b5e640 1 -- 172.21.15.169:0/1660505857 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53e4152cc0 con 0x7f53e40ac4e0 2023-10-26T01:35:24.237 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:24.231+0000 7f53e9b5e640 1 -- 172.21.15.169:0/1660505857 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53e41519d0 con 0x7f53e40ac4e0 2023-10-26T01:35:24.237 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:24.232+0000 7f53e9b5e640 1 -- 172.21.15.169:0/1660505857 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53e404db10 con 0x7f53e40ac4e0 2023-10-26T01:35:24.237 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:24.233+0000 7f53e0ff9640 1 -- 172.21.15.169:0/1660505857 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f53d400fc10 con 0x7f53e40ac4e0 2023-10-26T01:35:24.237 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:24.234+0000 7f53e0ff9640 1 --2- 172.21.15.169:0/1660505857 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f53c406ae40 0x7f53c406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:24.237 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:24.234+0000 7f53e0ff9640 1 -- 172.21.15.169:0/1660505857 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f53d4089e10 con 0x7f53e40ac4e0 2023-10-26T01:35:24.238 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:24.234+0000 7f53daffd640 1 --2- 172.21.15.169:0/1660505857 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f53c406ae40 0x7f53c406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:24.238 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:24.236+0000 7f53daffd640 1 --2- 172.21.15.169:0/1660505857 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f53c406ae40 0x7f53c406d300 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f53e40b1b20 tx=0x7f53cc009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:35:24.240 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:24.238+0000 7f53e0ff9640 1 -- 172.21.15.169:0/1660505857 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f53d408d030 con 0x7f53e40ac4e0 2023-10-26T01:35:24.492 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:24.491+0000 7f53e9b5e640 1 -- 172.21.15.169:0/1660505857 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f53e406c990 con 0x7f53e40ac4e0 2023-10-26T01:35:24.493 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:24.492+0000 7f53e0ff9640 1 -- 172.21.15.169:0/1660505857 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f53d401d090 con 0x7f53e40ac4e0 2023-10-26T01:35:24.493 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:35:24.494 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2023-10-26T01:35:24.494 INFO:teuthology.orchestra.run.smithi169.stdout: "auth_dump": [ 2023-10-26T01:35:24.494 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:35:24.494 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "mds.a", 2023-10-26T01:35:24.494 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAgwTllWtUfAhAA28+12VwWEF/BUEu+ErSVyA==", 2023-10-26T01:35:24.495 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:35:24.495 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow", 2023-10-26T01:35:24.495 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow *", 2023-10-26T01:35:24.495 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow *", 2023-10-26T01:35:24.495 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:35:24.496 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:35:24.496 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:35:24.496 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:35:24.496 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "mds.b", 2023-10-26T01:35:24.496 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAgwTlleA8lHxAA+jMcMo+QnYAGllpKJ6eYgQ==", 2023-10-26T01:35:24.497 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:35:24.497 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow", 2023-10-26T01:35:24.497 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow *", 2023-10-26T01:35:24.497 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow *", 2023-10-26T01:35:24.497 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:35:24.497 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:35:24.498 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:35:24.498 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:35:24.498 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "mds.c", 2023-10-26T01:35:24.498 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAgwTllaCCqEBAAokeHvwOyp/Jv2x0yHDwysw==", 2023-10-26T01:35:24.498 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:35:24.499 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow", 2023-10-26T01:35:24.499 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow *", 2023-10-26T01:35:24.499 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow *", 2023-10-26T01:35:24.499 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:35:24.499 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:35:24.500 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:35:24.500 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:35:24.500 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "mds.d", 2023-10-26T01:35:24.500 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAgwTll51JQLhAA4Wzso4zQj+Htv6cXqvwRmg==", 2023-10-26T01:35:24.500 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:35:24.501 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow", 2023-10-26T01:35:24.501 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow *", 2023-10-26T01:35:24.501 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow *", 2023-10-26T01:35:24.501 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:35:24.501 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:35:24.502 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:35:24.502 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:35:24.502 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "osd.0", 2023-10-26T01:35:24.502 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAlwTll8RZ+HRAAt/sBK+Mi91YmTyQYbhKHLg==", 2023-10-26T01:35:24.502 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:35:24.502 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow profile osd", 2023-10-26T01:35:24.503 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow profile osd", 2023-10-26T01:35:24.503 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:35:24.503 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:35:24.503 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:35:24.503 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:35:24.504 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "osd.1", 2023-10-26T01:35:24.504 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAlwTllLXwWLRAAuHB2lJ+oDJou/IJBtsYY4Q==", 2023-10-26T01:35:24.504 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:35:24.504 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow profile osd", 2023-10-26T01:35:24.504 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow profile osd", 2023-10-26T01:35:24.505 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:35:24.505 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:35:24.505 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:35:24.505 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:35:24.505 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "osd.2", 2023-10-26T01:35:24.505 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAmwTllj1nSARAA5Jl34Q0X7a71ySd4wz17lw==", 2023-10-26T01:35:24.506 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:35:24.506 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow profile osd", 2023-10-26T01:35:24.506 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow profile osd", 2023-10-26T01:35:24.506 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:35:24.506 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:35:24.507 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:35:24.509 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:35:24.509 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "osd.3", 2023-10-26T01:35:24.509 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAmwTllC8MeEhAAfGxRVSwD+YbU/U5aeenO3w==", 2023-10-26T01:35:24.509 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:35:24.510 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow profile osd", 2023-10-26T01:35:24.510 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow profile osd", 2023-10-26T01:35:24.510 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:35:24.510 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:35:24.510 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:35:24.511 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:35:24.511 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "osd.4", 2023-10-26T01:35:24.511 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAowTll4Md6DhAAuXfadUNAdouMP28MiX/D3Q==", 2023-10-26T01:35:24.511 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:35:24.511 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow profile osd", 2023-10-26T01:35:24.511 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow profile osd", 2023-10-26T01:35:24.512 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:35:24.512 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:35:24.512 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:35:24.512 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:35:24.512 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "osd.5", 2023-10-26T01:35:24.512 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAowTllDdViHRAAec9QyhZCsjJ2rgGhhcjwpQ==", 2023-10-26T01:35:24.513 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:35:24.513 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow profile osd", 2023-10-26T01:35:24.513 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow profile osd", 2023-10-26T01:35:24.513 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:35:24.513 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:35:24.514 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:35:24.514 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:35:24.514 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "osd.6", 2023-10-26T01:35:24.514 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAowTll23X5KxAAsmQvG0yq7ilkVG5fNiaCUg==", 2023-10-26T01:35:24.514 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:35:24.515 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow profile osd", 2023-10-26T01:35:24.515 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow profile osd", 2023-10-26T01:35:24.515 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:35:24.515 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:35:24.515 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:35:24.515 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:35:24.516 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "osd.7", 2023-10-26T01:35:24.516 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAowTllhcy/OhAAY5Yjheh76ibOVREqN6WBpQ==", 2023-10-26T01:35:24.516 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:35:24.516 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow profile osd", 2023-10-26T01:35:24.516 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow profile osd", 2023-10-26T01:35:24.516 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:35:24.517 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:35:24.517 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:35:24.517 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:35:24.517 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "client.0", 2023-10-26T01:35:24.517 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAgwTllKC7gOhAAPsvxGcuUA8/p8xqbUgGteA==", 2023-10-26T01:35:24.517 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:35:24.518 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow", 2023-10-26T01:35:24.518 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow r", 2023-10-26T01:35:24.518 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-26T01:35:24.518 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:35:24.518 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:35:24.519 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:35:24.519 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "client.1", 2023-10-26T01:35:24.519 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAhwTllhxb6ExAAeYTsurWpnjohWlrNIlLpsw==", 2023-10-26T01:35:24.519 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:35:24.519 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow", 2023-10-26T01:35:24.519 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow r", 2023-10-26T01:35:24.520 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow rw", 2023-10-26T01:35:24.520 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow rwx" 2023-10-26T01:35:24.520 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:35:24.520 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:35:24.520 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:35:24.520 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "client.2", 2023-10-26T01:35:24.521 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAhwTllLWMoChAAsUHzBc29GDqmEQAFnl7dHg==", 2023-10-26T01:35:24.521 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:35:24.521 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow", 2023-10-26T01:35:24.521 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow r", 2023-10-26T01:35:24.521 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow rw", 2023-10-26T01:35:24.522 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow rwx" 2023-10-26T01:35:24.522 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:35:24.522 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:35:24.522 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:35:24.522 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "client.3", 2023-10-26T01:35:24.522 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAhwTll875HHRAAGbwikoR/hDU8P/bj0OPWAw==", 2023-10-26T01:35:24.523 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:35:24.523 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow", 2023-10-26T01:35:24.523 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow r", 2023-10-26T01:35:24.523 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow rw", 2023-10-26T01:35:24.523 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow rwx" 2023-10-26T01:35:24.524 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:35:24.524 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:35:24.524 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:35:24.524 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "client.admin", 2023-10-26T01:35:24.524 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAewTll2yddMhAAO8bnXNevDGe/KEeNNa36pQ==", 2023-10-26T01:35:24.525 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:35:24.525 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow *", 2023-10-26T01:35:24.525 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow *", 2023-10-26T01:35:24.525 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow *", 2023-10-26T01:35:24.525 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:35:24.525 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:35:24.526 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:35:24.526 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:35:24.526 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "mgr.x", 2023-10-26T01:35:24.526 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAfwTllPu3FMxAAV8d5oRFXAeeZH7e1xzPEDg==", 2023-10-26T01:35:24.526 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:35:24.527 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow *", 2023-10-26T01:35:24.527 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow profile mgr", 2023-10-26T01:35:24.527 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:35:24.527 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:35:24.527 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:35:24.527 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:35:24.528 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "mgr.y", 2023-10-26T01:35:24.528 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAfwTllZc+DJxAAsZ2TWn8H7Kw/lNTc2qyKzA==", 2023-10-26T01:35:24.528 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:35:24.528 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow *", 2023-10-26T01:35:24.528 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow profile mgr", 2023-10-26T01:35:24.528 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:35:24.529 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:35:24.529 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:35:24.529 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:35:24.529 INFO:teuthology.orchestra.run.smithi169.stdout:} 2023-10-26T01:35:24.530 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:24.494+0000 7f53e9b5e640 1 -- 172.21.15.169:0/1660505857 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f53c406ae40 msgr2=0x7f53c406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:24.531 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:24.494+0000 7f53e9b5e640 1 --2- 172.21.15.169:0/1660505857 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f53c406ae40 0x7f53c406d300 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f53e40b1b20 tx=0x7f53cc009040 comp rx=0 tx=0).stop 2023-10-26T01:35:24.531 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:24.494+0000 7f53e9b5e640 1 -- 172.21.15.169:0/1660505857 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53e40ac4e0 msgr2=0x7f53e41436c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:24.531 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:24.494+0000 7f53e9b5e640 1 --2- 172.21.15.169:0/1660505857 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53e40ac4e0 0x7f53e41436c0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f53d400cce0 tx=0x7f53d400a620 comp rx=0 tx=0).stop 2023-10-26T01:35:24.531 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:24.494+0000 7f53e9b5e640 1 -- 172.21.15.169:0/1660505857 shutdown_connections 2023-10-26T01:35:24.531 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:24.494+0000 7f53e9b5e640 1 --2- 172.21.15.169:0/1660505857 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f53c406ae40 0x7f53c406d300 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:24.532 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:24.494+0000 7f53e9b5e640 1 --2- 172.21.15.169:0/1660505857 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f53e415b610 0x7f53e41566b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:24.532 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:24.494+0000 7f53e9b5e640 1 --2- 172.21.15.169:0/1660505857 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f53e40cdd80 0x7f53e4156170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:24.532 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:24.494+0000 7f53e9b5e640 1 --2- 172.21.15.169:0/1660505857 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53e40ac4e0 0x7f53e41436c0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:24.532 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:24.494+0000 7f53e9b5e640 1 -- 172.21.15.169:0/1660505857 >> 172.21.15.169:0/1660505857 conn(0x7f53e4069930 msgr2=0x7f53e40aa7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:35:24.532 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:24.494+0000 7f53e9b5e640 1 -- 172.21.15.169:0/1660505857 shutdown_connections 2023-10-26T01:35:24.533 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:24.494+0000 7f53e9b5e640 1 -- 172.21.15.169:0/1660505857 wait complete. 2023-10-26T01:35:24.533 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:35:24.751 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:24.749+0000 7fc9872d1640 1 -- 172.21.15.169:0/3510185117 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc96c0039a0 msgr2=0x7fc96c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:24.751 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:24.749+0000 7fc9872d1640 1 --2- 172.21.15.169:0/3510185117 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc96c0039a0 0x7fc96c003dc0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fc97c001680 tx=0x7fc96c0057f0 comp rx=0 tx=0).stop 2023-10-26T01:35:24.751 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:24.749+0000 7fc986ad0640 1 -- 172.21.15.169:0/3510185117 reap_dead start 2023-10-26T01:35:24.752 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:24.749+0000 7fc9872d1640 1 -- 172.21.15.169:0/3510185117 shutdown_connections 2023-10-26T01:35:24.752 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:24.749+0000 7fc9872d1640 1 -- 172.21.15.169:0/3510185117 >> 172.21.15.169:0/3510185117 conn(0x7fc9800aedd0 msgr2=0x7fc9800af1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:35:24.752 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:24.749+0000 7fc9872d1640 1 -- 172.21.15.169:0/3510185117 shutdown_connections 2023-10-26T01:35:24.752 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:24.749+0000 7fc9872d1640 1 -- 172.21.15.169:0/3510185117 wait complete. 2023-10-26T01:35:24.753 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:24.751+0000 7fc9872d1640 1 Processor -- start 2023-10-26T01:35:24.753 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:24.751+0000 7fc9872d1640 1 -- start start 2023-10-26T01:35:24.754 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:24.752+0000 7fc9872d1640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc96c0039a0 0x7fc98006f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:24.754 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:24.752+0000 7fc9872d1640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc9800c6830 0x7fc9800c98e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:24.754 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:24.752+0000 7fc9872d1640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc9800b0bb0 0x7fc9800b1010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:24.755 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:24.752+0000 7fc9872d1640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9800b0720 con 0x7fc96c0039a0 2023-10-26T01:35:24.755 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:24.752+0000 7fc9872d1640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc98006fbc0 con 0x7fc9800c6830 2023-10-26T01:35:24.755 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:24.752+0000 7fc9872d1640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9800afdf0 con 0x7fc9800b0bb0 2023-10-26T01:35:24.755 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:24.752+0000 7fc9862cf640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc96c0039a0 0x7fc98006f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:24.755 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:24.752+0000 7fc9862cf640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc96c0039a0 0x7fc98006f300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:56816/0 (socket says 172.21.15.169:56816) 2023-10-26T01:35:24.756 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:24.752+0000 7fc9862cf640 1 -- 172.21.15.169:0/2588384062 learned_addr learned my addr 172.21.15.169:0/2588384062 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:35:24.756 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:24.753+0000 7fc9862cf640 1 -- 172.21.15.169:0/2588384062 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc9800b0bb0 msgr2=0x7fc9800b1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:24.756 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:24.753+0000 7fc9862cf640 1 --2- 172.21.15.169:0/2588384062 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc9800b0bb0 0x7fc9800b1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:24.756 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:24.753+0000 7fc985ace640 1 --2- 172.21.15.169:0/2588384062 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc9800c6830 0x7fc9800c98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:24.756 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:24.753+0000 7fc9862cf640 1 -- 172.21.15.169:0/2588384062 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc9800c6830 msgr2=0x7fc9800c98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:24.757 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:24.753+0000 7fc9862cf640 1 --2- 172.21.15.169:0/2588384062 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc9800c6830 0x7fc9800c98e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:24.757 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:24.753+0000 7fc9862cf640 1 -- 172.21.15.169:0/2588384062 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9800b00d0 con 0x7fc96c0039a0 2023-10-26T01:35:24.758 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:24.753+0000 7fc9862cf640 1 --2- 172.21.15.169:0/2588384062 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc96c0039a0 0x7fc98006f300 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fc97c0017a0 tx=0x7fc97c0035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:35:24.758 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:24.753+0000 7fc976ffd640 1 -- 172.21.15.169:0/2588384062 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc97c003bf0 con 0x7fc96c0039a0 2023-10-26T01:35:24.758 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:24.753+0000 7fc976ffd640 1 -- 172.21.15.169:0/2588384062 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc97c003e40 con 0x7fc96c0039a0 2023-10-26T01:35:24.759 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:24.753+0000 7fc9872d1640 1 -- 172.21.15.169:0/2588384062 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9800b03b0 con 0x7fc96c0039a0 2023-10-26T01:35:24.759 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:24.753+0000 7fc976ffd640 1 -- 172.21.15.169:0/2588384062 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc97c02c790 con 0x7fc96c0039a0 2023-10-26T01:35:24.759 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:24.754+0000 7fc9872d1640 1 -- 172.21.15.169:0/2588384062 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9800c9e20 con 0x7fc96c0039a0 2023-10-26T01:35:24.759 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:24.755+0000 7fc9872d1640 1 -- 172.21.15.169:0/2588384062 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc980073c10 con 0x7fc96c0039a0 2023-10-26T01:35:24.759 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:24.756+0000 7fc976ffd640 1 -- 172.21.15.169:0/2588384062 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fc97c02c8f0 con 0x7fc96c0039a0 2023-10-26T01:35:24.760 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:24.756+0000 7fc976ffd640 1 --2- 172.21.15.169:0/2588384062 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc96406ae40 0x7fc96406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:24.760 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:24.756+0000 7fc976ffd640 1 -- 172.21.15.169:0/2588384062 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fc97c0aa2c0 con 0x7fc96c0039a0 2023-10-26T01:35:24.760 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:24.757+0000 7fc985ace640 1 --2- 172.21.15.169:0/2588384062 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc96406ae40 0x7fc96406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:24.761 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:24.758+0000 7fc985ace640 1 --2- 172.21.15.169:0/2588384062 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc96406ae40 0x7fc96406d300 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fc980153d00 tx=0x7fc970000ed0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:35:24.762 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:24.761+0000 7fc976ffd640 1 -- 172.21.15.169:0/2588384062 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc97c0756c0 con 0x7fc96c0039a0 2023-10-26T01:35:24.954 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:24.952+0000 7fc9872d1640 1 -- 172.21.15.169:0/2588384062 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc9800cb2f0 con 0x7fc96c0039a0 2023-10-26T01:35:24.954 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:24.953+0000 7fc976ffd640 1 -- 172.21.15.169:0/2588384062 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+12905 (secure 0 0 0) 0x7fc97c043090 con 0x7fc96c0039a0 2023-10-26T01:35:24.955 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:35:24.955 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":80,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:35:23.975892+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"72","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:35:24.957 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:24.955+0000 7fc9872d1640 1 -- 172.21.15.169:0/2588384062 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc96406ae40 msgr2=0x7fc96406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:24.958 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:24.955+0000 7fc9872d1640 1 --2- 172.21.15.169:0/2588384062 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc96406ae40 0x7fc96406d300 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fc980153d00 tx=0x7fc970000ed0 comp rx=0 tx=0).stop 2023-10-26T01:35:24.958 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:24.955+0000 7fc9872d1640 1 -- 172.21.15.169:0/2588384062 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc96c0039a0 msgr2=0x7fc98006f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:24.958 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:24.955+0000 7fc9872d1640 1 --2- 172.21.15.169:0/2588384062 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc96c0039a0 0x7fc98006f300 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fc97c0017a0 tx=0x7fc97c0035f0 comp rx=0 tx=0).stop 2023-10-26T01:35:24.958 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:24.956+0000 7fc9872d1640 1 -- 172.21.15.169:0/2588384062 shutdown_connections 2023-10-26T01:35:24.958 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:24.956+0000 7fc9872d1640 1 --2- 172.21.15.169:0/2588384062 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc96406ae40 0x7fc96406d300 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:24.959 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:24.956+0000 7fc9872d1640 1 --2- 172.21.15.169:0/2588384062 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc9800b0bb0 0x7fc9800b1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:24.959 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:24.956+0000 7fc9872d1640 1 --2- 172.21.15.169:0/2588384062 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc9800c6830 0x7fc9800c98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:24.959 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:24.956+0000 7fc9872d1640 1 --2- 172.21.15.169:0/2588384062 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc96c0039a0 0x7fc98006f300 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:24.959 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:24.956+0000 7fc9872d1640 1 -- 172.21.15.169:0/2588384062 >> 172.21.15.169:0/2588384062 conn(0x7fc9800aedd0 msgr2=0x7fc98006b7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:35:24.959 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:24.956+0000 7fc9872d1640 1 -- 172.21.15.169:0/2588384062 shutdown_connections 2023-10-26T01:35:24.960 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:24.956+0000 7fc9872d1640 1 -- 172.21.15.169:0/2588384062 wait complete. 2023-10-26T01:35:24.976 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T01:30:32.200808+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '72', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T01:35:24.977 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T01:35:25.195 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:25.193+0000 7f6de7697640 1 -- 172.21.15.169:0/2054800027 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6dcc0047c0 msgr2=0x7f6dcc004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:25.196 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:25.193+0000 7f6de7697640 1 --2- 172.21.15.169:0/2054800027 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6dcc0047c0 0x7f6dcc004c40 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f6dd0009b50 tx=0x7f6dcc0057f0 comp rx=0 tx=0).stop 2023-10-26T01:35:25.196 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:25.193+0000 7f6de5c34640 1 -- 172.21.15.169:0/2054800027 reap_dead start 2023-10-26T01:35:25.196 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:25.193+0000 7f6de7697640 1 -- 172.21.15.169:0/2054800027 shutdown_connections 2023-10-26T01:35:25.196 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:25.193+0000 7f6de7697640 1 -- 172.21.15.169:0/2054800027 >> 172.21.15.169:0/2054800027 conn(0x7f6de0069930 msgr2=0x7f6de0069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:35:25.196 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:25.193+0000 7f6de7697640 1 -- 172.21.15.169:0/2054800027 shutdown_connections 2023-10-26T01:35:25.197 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:25.193+0000 7f6de7697640 1 -- 172.21.15.169:0/2054800027 wait complete. 2023-10-26T01:35:25.197 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:25.195+0000 7f6de7697640 1 Processor -- start 2023-10-26T01:35:25.197 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:25.195+0000 7f6de7697640 1 -- start start 2023-10-26T01:35:25.199 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:25.196+0000 7f6de7697640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6dcc0047c0 0x7f6de00c7e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:25.200 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:25.196+0000 7f6de7697640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6de0157250 0x7f6de01f1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:25.200 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:25.196+0000 7f6de7697640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6de01596b0 0x7f6de0159b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:25.200 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:25.196+0000 7f6de7697640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6de01f3010 con 0x7f6de0157250 2023-10-26T01:35:25.200 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:25.196+0000 7f6de7697640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6de01576d0 con 0x7f6de01596b0 2023-10-26T01:35:25.200 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:25.196+0000 7f6de7697640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6de01f2510 con 0x7f6dcc0047c0 2023-10-26T01:35:25.201 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:25.196+0000 7f6de4c32640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6de0157250 0x7f6de01f1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:25.201 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:25.196+0000 7f6de4c32640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6de0157250 0x7f6de01f1fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:56836/0 (socket says 172.21.15.169:56836) 2023-10-26T01:35:25.201 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:25.197+0000 7f6de4c32640 1 -- 172.21.15.169:0/1914792540 learned_addr learned my addr 172.21.15.169:0/1914792540 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:35:25.201 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:25.197+0000 7f6de4c32640 1 -- 172.21.15.169:0/1914792540 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6dcc0047c0 msgr2=0x7f6de00c7e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:25.201 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:25.197+0000 7f6de5433640 1 --2- 172.21.15.169:0/1914792540 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6dcc0047c0 0x7f6de00c7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:25.201 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:25.197+0000 7f6de4c32640 1 --2- 172.21.15.169:0/1914792540 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6dcc0047c0 0x7f6de00c7e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:25.202 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:25.197+0000 7f6de5c34640 1 --2- 172.21.15.169:0/1914792540 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6de01596b0 0x7f6de0159b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:25.202 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:25.197+0000 7f6de4c32640 1 -- 172.21.15.169:0/1914792540 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6de01596b0 msgr2=0x7f6de0159b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:25.202 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:25.197+0000 7f6de4c32640 1 --2- 172.21.15.169:0/1914792540 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6de01596b0 0x7f6de0159b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:25.202 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:25.197+0000 7f6de4c32640 1 -- 172.21.15.169:0/1914792540 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6de01f2770 con 0x7f6de0157250 2023-10-26T01:35:25.203 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:25.197+0000 7f6de5433640 1 --2- 172.21.15.169:0/1914792540 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6dcc0047c0 0x7f6de00c7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:35:25.203 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:25.197+0000 7f6de4c32640 1 --2- 172.21.15.169:0/1914792540 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6de0157250 0x7f6de01f1fd0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f6dd0009a70 tx=0x7f6dd0033c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:35:25.203 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:25.198+0000 7f6dd5ffb640 1 -- 172.21.15.169:0/1914792540 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd0004220 con 0x7f6de0157250 2023-10-26T01:35:25.203 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:25.198+0000 7f6dd5ffb640 1 -- 172.21.15.169:0/1914792540 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6dd0004380 con 0x7f6de0157250 2023-10-26T01:35:25.203 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:25.198+0000 7f6dd5ffb640 1 -- 172.21.15.169:0/1914792540 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd003d940 con 0x7f6de0157250 2023-10-26T01:35:25.204 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:25.198+0000 7f6de7697640 1 -- 172.21.15.169:0/1914792540 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6de01f2a50 con 0x7f6de0157250 2023-10-26T01:35:25.204 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:25.198+0000 7f6de7697640 1 -- 172.21.15.169:0/1914792540 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6de01558e0 con 0x7f6de0157250 2023-10-26T01:35:25.205 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:25.199+0000 7f6de7697640 1 -- 172.21.15.169:0/1914792540 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6de0066af0 con 0x7f6de0157250 2023-10-26T01:35:25.205 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:25.200+0000 7f6dd5ffb640 1 -- 172.21.15.169:0/1914792540 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f6dd003daa0 con 0x7f6de0157250 2023-10-26T01:35:25.205 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:25.201+0000 7f6dd5ffb640 1 --2- 172.21.15.169:0/1914792540 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f6dc406ae40 0x7f6dc406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:25.205 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:25.201+0000 7f6dd5ffb640 1 -- 172.21.15.169:0/1914792540 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f6dd0033e40 con 0x7f6de0157250 2023-10-26T01:35:25.205 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:25.201+0000 7f6de5433640 1 --2- 172.21.15.169:0/1914792540 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f6dc406ae40 0x7f6dc406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:25.206 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:25.203+0000 7f6de5433640 1 --2- 172.21.15.169:0/1914792540 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f6dc406ae40 0x7f6dc406d300 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc006fd0 tx=0x7f6ddc006a30 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:35:25.207 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:25.206+0000 7f6dd5ffb640 1 -- 172.21.15.169:0/1914792540 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6dd00774f0 con 0x7f6de0157250 2023-10-26T01:35:25.396 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:25.394+0000 7f6de7697640 1 -- 172.21.15.169:0/1914792540 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6de01f2f70 con 0x7f6de0157250 2023-10-26T01:35:25.397 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:25.395+0000 7f6dd5ffb640 1 -- 172.21.15.169:0/1914792540 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6dd004a020 con 0x7f6de0157250 2023-10-26T01:35:25.397 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 1 2023-10-26T01:35:25.399 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:25.397+0000 7f6de7697640 1 -- 172.21.15.169:0/1914792540 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f6dc406ae40 msgr2=0x7f6dc406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:25.400 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:25.398+0000 7f6de7697640 1 --2- 172.21.15.169:0/1914792540 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f6dc406ae40 0x7f6dc406d300 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc006fd0 tx=0x7f6ddc006a30 comp rx=0 tx=0).stop 2023-10-26T01:35:25.400 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:25.398+0000 7f6de7697640 1 -- 172.21.15.169:0/1914792540 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6de0157250 msgr2=0x7f6de01f1fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:25.400 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:25.398+0000 7f6de7697640 1 --2- 172.21.15.169:0/1914792540 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6de0157250 0x7f6de01f1fd0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f6dd0009a70 tx=0x7f6dd0033c50 comp rx=0 tx=0).stop 2023-10-26T01:35:25.400 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:25.398+0000 7f6de7697640 1 -- 172.21.15.169:0/1914792540 shutdown_connections 2023-10-26T01:35:25.400 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:25.398+0000 7f6de7697640 1 --2- 172.21.15.169:0/1914792540 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f6dc406ae40 0x7f6dc406d300 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:25.401 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:25.398+0000 7f6de7697640 1 --2- 172.21.15.169:0/1914792540 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6de0157250 0x7f6de01f1fd0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:25.401 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:25.398+0000 7f6de7697640 1 --2- 172.21.15.169:0/1914792540 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6de01596b0 0x7f6de0159b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:25.401 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:25.398+0000 7f6de7697640 1 --2- 172.21.15.169:0/1914792540 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6dcc0047c0 0x7f6de00c7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:25.401 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:25.398+0000 7f6de7697640 1 -- 172.21.15.169:0/1914792540 >> 172.21.15.169:0/1914792540 conn(0x7f6de0069930 msgr2=0x7f6de00caea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:35:25.401 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:25.398+0000 7f6de7697640 1 -- 172.21.15.169:0/1914792540 shutdown_connections 2023-10-26T01:35:25.401 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:25.398+0000 7f6de7697640 1 -- 172.21.15.169:0/1914792540 wait complete. 2023-10-26T01:35:25.418 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-26T01:35:25.626 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:25.624+0000 7f4309002640 1 -- 172.21.15.169:0/3917627031 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f43040ac930 msgr2=0x7f43040cd6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:25.627 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:25.624+0000 7f4309002640 1 --2- 172.21.15.169:0/3917627031 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f43040ac930 0x7f43040cd6f0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f42ec005fd0 tx=0x7f42ec02bd90 comp rx=0 tx=0).stop 2023-10-26T01:35:25.627 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:25.624+0000 7f4309002640 1 -- 172.21.15.169:0/3917627031 shutdown_connections 2023-10-26T01:35:25.627 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:25.624+0000 7f4309002640 1 --2- 172.21.15.169:0/3917627031 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f43040ac930 0x7f43040cd6f0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:25.627 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:25.624+0000 7f4309002640 1 --2- 172.21.15.169:0/3917627031 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f43041592d0 0x7f43040aa3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:25.627 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:25.624+0000 7f4309002640 1 -- 172.21.15.169:0/3917627031 >> 172.21.15.169:0/3917627031 conn(0x7f4304069930 msgr2=0x7f4304069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:35:25.628 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:25.624+0000 7f4309002640 1 -- 172.21.15.169:0/3917627031 shutdown_connections 2023-10-26T01:35:25.628 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:25.624+0000 7f4309002640 1 -- 172.21.15.169:0/3917627031 wait complete. 2023-10-26T01:35:25.629 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:25.626+0000 7f4309002640 1 Processor -- start 2023-10-26T01:35:25.630 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:25.626+0000 7f4309002640 1 -- start start 2023-10-26T01:35:25.630 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:25.627+0000 7f4309002640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f43040ac930 0x7f43041509b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:25.630 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:25.627+0000 7f4309002640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f43040cdc30 0x7f43041f2c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:25.630 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:25.627+0000 7f4309002640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f43041592d0 0x7f430415bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:25.631 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:25.627+0000 7f4309002640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43041556d0 con 0x7f43040cdc30 2023-10-26T01:35:25.631 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:25.627+0000 7f4309002640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4304070720 con 0x7f43040ac930 2023-10-26T01:35:25.631 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:25.627+0000 7f4309002640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4304070a00 con 0x7f43041592d0 2023-10-26T01:35:25.632 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:25.627+0000 7f42fa59c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f43040cdc30 0x7f43041f2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:25.632 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:25.627+0000 7f42fa59c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f43040cdc30 0x7f43041f2c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:56850/0 (socket says 172.21.15.169:56850) 2023-10-26T01:35:25.632 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:25.627+0000 7f42fa59c640 1 -- 172.21.15.169:0/3719249395 learned_addr learned my addr 172.21.15.169:0/3719249395 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:35:25.633 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:25.628+0000 7f4302d9d640 1 --2- 172.21.15.169:0/3719249395 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f43040ac930 0x7f43041509b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:25.633 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:25.628+0000 7f42fa59c640 1 -- 172.21.15.169:0/3719249395 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f43041592d0 msgr2=0x7f430415bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:25.633 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:25.628+0000 7f42fa59c640 1 --2- 172.21.15.169:0/3719249395 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f43041592d0 0x7f430415bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:25.634 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:25.628+0000 7f42fa59c640 1 -- 172.21.15.169:0/3719249395 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f43040ac930 msgr2=0x7f43041509b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:25.634 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:25.628+0000 7f42fa59c640 1 --2- 172.21.15.169:0/3719249395 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f43040ac930 0x7f43041509b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:25.634 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:25.628+0000 7f42fa59c640 1 -- 172.21.15.169:0/3719249395 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42ec009560 con 0x7f43040cdc30 2023-10-26T01:35:25.634 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:25.628+0000 7f4302d9d640 1 --2- 172.21.15.169:0/3719249395 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f43040ac930 0x7f43041509b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:35:25.635 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:25.628+0000 7f42fa59c640 1 --2- 172.21.15.169:0/3719249395 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f43040cdc30 0x7f43041f2c90 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f42ec009850 tx=0x7f42ec02db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:35:25.636 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:25.628+0000 7f42fbfff640 1 -- 172.21.15.169:0/3719249395 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42ec0035f0 con 0x7f43040cdc30 2023-10-26T01:35:25.636 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:25.628+0000 7f42fbfff640 1 -- 172.21.15.169:0/3719249395 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f42ec003750 con 0x7f43040cdc30 2023-10-26T01:35:25.637 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:25.628+0000 7f42fbfff640 1 -- 172.21.15.169:0/3719249395 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42ec003b00 con 0x7f43040cdc30 2023-10-26T01:35:25.637 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:25.629+0000 7f4309002640 1 -- 172.21.15.169:0/3719249395 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4304070ce0 con 0x7f43040cdc30 2023-10-26T01:35:25.637 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:25.629+0000 7f4309002640 1 -- 172.21.15.169:0/3719249395 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43041f31d0 con 0x7f43040cdc30 2023-10-26T01:35:25.637 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:25.630+0000 7f4309002640 1 -- 172.21.15.169:0/3719249395 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43041f3360 con 0x7f43040cdc30 2023-10-26T01:35:25.637 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:25.631+0000 7f42fbfff640 1 -- 172.21.15.169:0/3719249395 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f42ec003c60 con 0x7f43040cdc30 2023-10-26T01:35:25.638 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:25.632+0000 7f42fbfff640 1 --2- 172.21.15.169:0/3719249395 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f42e406ae40 0x7f42e406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:25.638 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:25.632+0000 7f42fbfff640 1 -- 172.21.15.169:0/3719249395 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f42ec0abfe0 con 0x7f43040cdc30 2023-10-26T01:35:25.638 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:25.632+0000 7f4302d9d640 1 --2- 172.21.15.169:0/3719249395 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f42e406ae40 0x7f42e406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:25.638 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:25.633+0000 7f4302d9d640 1 --2- 172.21.15.169:0/3719249395 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f42e406ae40 0x7f42e406d300 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f42f4005fd0 tx=0x7f42f40082b0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:35:25.639 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:25.637+0000 7f42fbfff640 1 -- 172.21.15.169:0/3719249395 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f42ec077380 con 0x7f43040cdc30 2023-10-26T01:35:25.830 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:25.829+0000 7f4309002640 1 -- 172.21.15.169:0/3719249395 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f43040711a0 con 0x7f43040cdc30 2023-10-26T01:35:25.831 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:25.830+0000 7f42fbfff640 1 -- 172.21.15.169:0/3719249395 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v80) v1 ==== 84+0+65 (secure 0 0 0) 0x7f42ec043090 con 0x7f43040cdc30 2023-10-26T01:35:25.831 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:35:25.832 INFO:teuthology.orchestra.run.smithi169.stdout:[ 2023-10-26T01:35:25.832 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:35:25.832 INFO:teuthology.orchestra.run.smithi169.stdout: "poolnum": 1, 2023-10-26T01:35:25.832 INFO:teuthology.orchestra.run.smithi169.stdout: "poolname": ".mgr" 2023-10-26T01:35:25.833 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:35:25.833 INFO:teuthology.orchestra.run.smithi169.stdout:] 2023-10-26T01:35:25.834 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:25.832+0000 7f4309002640 1 -- 172.21.15.169:0/3719249395 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f42e406ae40 msgr2=0x7f42e406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:25.834 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:25.832+0000 7f4309002640 1 --2- 172.21.15.169:0/3719249395 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f42e406ae40 0x7f42e406d300 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f42f4005fd0 tx=0x7f42f40082b0 comp rx=0 tx=0).stop 2023-10-26T01:35:25.834 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:25.832+0000 7f4309002640 1 -- 172.21.15.169:0/3719249395 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f43040cdc30 msgr2=0x7f43041f2c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:25.834 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:25.832+0000 7f4309002640 1 --2- 172.21.15.169:0/3719249395 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f43040cdc30 0x7f43041f2c90 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f42ec009850 tx=0x7f42ec02db10 comp rx=0 tx=0).stop 2023-10-26T01:35:25.835 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:25.832+0000 7f4309002640 1 -- 172.21.15.169:0/3719249395 shutdown_connections 2023-10-26T01:35:25.835 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:25.832+0000 7f4309002640 1 --2- 172.21.15.169:0/3719249395 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f42e406ae40 0x7f42e406d300 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:25.835 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:25.832+0000 7f4309002640 1 --2- 172.21.15.169:0/3719249395 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f43040cdc30 0x7f43041f2c90 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:25.835 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:25.832+0000 7f4309002640 1 --2- 172.21.15.169:0/3719249395 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f43041592d0 0x7f430415bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:25.835 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:25.832+0000 7f4309002640 1 --2- 172.21.15.169:0/3719249395 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f43040ac930 0x7f43041509b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:25.836 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:25.832+0000 7f4309002640 1 -- 172.21.15.169:0/3719249395 >> 172.21.15.169:0/3719249395 conn(0x7f4304069930 msgr2=0x7f4304069d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:35:25.837 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:25.832+0000 7f4309002640 1 -- 172.21.15.169:0/3719249395 shutdown_connections 2023-10-26T01:35:25.837 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:25.832+0000 7f4309002640 1 -- 172.21.15.169:0/3719249395 wait complete. 2023-10-26T01:35:25.852 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-26T01:35:25.853 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-26T01:35:26.067 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:26.064+0000 7f694ffff640 1 -- 172.21.15.169:0/2141103141 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f695015a5a0 msgr2=0x7f69500b17a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:26.067 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:26.064+0000 7f694ffff640 1 --2- 172.21.15.169:0/2141103141 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f695015a5a0 0x7f69500b17a0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f6938005fd0 tx=0x7f693802be20 comp rx=0 tx=0).stop 2023-10-26T01:35:26.067 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:26.065+0000 7f694ffff640 1 -- 172.21.15.169:0/2141103141 shutdown_connections 2023-10-26T01:35:26.067 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:26.065+0000 7f694ffff640 1 --2- 172.21.15.169:0/2141103141 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6950069900 0x7f6950069d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:26.068 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:26.065+0000 7f694ffff640 1 --2- 172.21.15.169:0/2141103141 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f695015a5a0 0x7f69500b17a0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:26.068 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:26.065+0000 7f694ffff640 1 -- 172.21.15.169:0/2141103141 >> 172.21.15.169:0/2141103141 conn(0x7f69500af290 msgr2=0x7f69500af6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:35:26.068 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:26.065+0000 7f694ffff640 1 -- 172.21.15.169:0/2141103141 shutdown_connections 2023-10-26T01:35:26.068 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:26.065+0000 7f694ffff640 1 -- 172.21.15.169:0/2141103141 wait complete. 2023-10-26T01:35:26.070 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:26.066+0000 7f694ffff640 1 Processor -- start 2023-10-26T01:35:26.070 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:26.066+0000 7f694ffff640 1 -- start start 2023-10-26T01:35:26.070 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:26.067+0000 7f694ffff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6950069900 0x7f69501ef6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:26.070 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:26.067+0000 7f694ffff640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f69500ac760 0x7f695015c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:26.071 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:26.067+0000 7f694ffff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f695015a5a0 0x7f69500ad6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:26.071 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:26.067+0000 7f694ffff640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6950097980 con 0x7f6950069900 2023-10-26T01:35:26.071 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:26.067+0000 7f694ffff640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6950097370 con 0x7f695015a5a0 2023-10-26T01:35:26.071 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:26.067+0000 7f694ffff640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69500ae200 con 0x7f69500ac760 2023-10-26T01:35:26.072 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:26.068+0000 7f694effd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6950069900 0x7f69501ef6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:26.072 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:26.068+0000 7f694effd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6950069900 0x7f69501ef6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:56874/0 (socket says 172.21.15.169:56874) 2023-10-26T01:35:26.072 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:26.068+0000 7f694effd640 1 -- 172.21.15.169:0/2681100706 learned_addr learned my addr 172.21.15.169:0/2681100706 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:35:26.072 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:26.068+0000 7f6947fff640 1 --2- 172.21.15.169:0/2681100706 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f69500ac760 0x7f695015c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:26.072 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:26.068+0000 7f694f7fe640 1 --2- 172.21.15.169:0/2681100706 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f695015a5a0 0x7f69500ad6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:26.073 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:26.068+0000 7f694effd640 1 -- 172.21.15.169:0/2681100706 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f69500ac760 msgr2=0x7f695015c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:26.073 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:26.068+0000 7f694effd640 1 --2- 172.21.15.169:0/2681100706 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f69500ac760 0x7f695015c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:26.073 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:26.068+0000 7f694effd640 1 -- 172.21.15.169:0/2681100706 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f695015a5a0 msgr2=0x7f69500ad6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:26.073 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:26.068+0000 7f694effd640 1 --2- 172.21.15.169:0/2681100706 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f695015a5a0 0x7f69500ad6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:26.073 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:26.068+0000 7f694effd640 1 -- 172.21.15.169:0/2681100706 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6938009560 con 0x7f6950069900 2023-10-26T01:35:26.075 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:26.069+0000 7f694effd640 1 --2- 172.21.15.169:0/2681100706 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6950069900 0x7f69501ef6d0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f694000c450 tx=0x7f694000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:35:26.075 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:26.069+0000 7f69477fe640 1 -- 172.21.15.169:0/2681100706 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69400052d0 con 0x7f6950069900 2023-10-26T01:35:26.075 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:26.069+0000 7f69477fe640 1 -- 172.21.15.169:0/2681100706 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6940005430 con 0x7f6950069900 2023-10-26T01:35:26.075 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:26.069+0000 7f69477fe640 1 -- 172.21.15.169:0/2681100706 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6940012890 con 0x7f6950069900 2023-10-26T01:35:26.076 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:26.069+0000 7f694ffff640 1 -- 172.21.15.169:0/2681100706 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69500adf20 con 0x7f6950069900 2023-10-26T01:35:26.076 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:26.069+0000 7f694ffff640 1 -- 172.21.15.169:0/2681100706 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69500adc10 con 0x7f6950069900 2023-10-26T01:35:26.076 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:26.070+0000 7f694ffff640 1 -- 172.21.15.169:0/2681100706 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6950066af0 con 0x7f6950069900 2023-10-26T01:35:26.076 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:26.071+0000 7f69477fe640 1 -- 172.21.15.169:0/2681100706 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f69400035f0 con 0x7f6950069900 2023-10-26T01:35:26.076 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:26.072+0000 7f69477fe640 1 --2- 172.21.15.169:0/2681100706 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f693006ae40 0x7f693006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:26.077 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:26.072+0000 7f69477fe640 1 -- 172.21.15.169:0/2681100706 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f6940088f80 con 0x7f6950069900 2023-10-26T01:35:26.077 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:26.073+0000 7f6947fff640 1 --2- 172.21.15.169:0/2681100706 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f693006ae40 0x7f693006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:26.077 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:26.074+0000 7f6947fff640 1 --2- 172.21.15.169:0/2681100706 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f693006ae40 0x7f693006d300 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f6938035000 tx=0x7f6938004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:35:26.078 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:26.077+0000 7f69477fe640 1 -- 172.21.15.169:0/2681100706 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6940054370 con 0x7f6950069900 2023-10-26T01:35:26.267 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:26.265+0000 7f694ffff640 1 -- 172.21.15.169:0/2681100706 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f695004da70 con 0x7f6950069900 2023-10-26T01:35:26.353 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:26.349+0000 7f69477fe640 1 -- 172.21.15.169:0/2681100706 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v81) v1 ==== 138+0+0 (secure 0 0 0) 0x7f6940057990 con 0x7f6950069900 2023-10-26T01:35:26.353 INFO:teuthology.orchestra.run.smithi169.stderr:pool 'cephfs_metadata' created 2023-10-26T01:35:26.355 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:26.354+0000 7f694ffff640 1 -- 172.21.15.169:0/2681100706 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f693006ae40 msgr2=0x7f693006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:26.356 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:26.354+0000 7f694ffff640 1 --2- 172.21.15.169:0/2681100706 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f693006ae40 0x7f693006d300 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f6938035000 tx=0x7f6938004730 comp rx=0 tx=0).stop 2023-10-26T01:35:26.356 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:26.354+0000 7f694ffff640 1 -- 172.21.15.169:0/2681100706 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6950069900 msgr2=0x7f69501ef6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:26.356 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:26.354+0000 7f694ffff640 1 --2- 172.21.15.169:0/2681100706 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6950069900 0x7f69501ef6d0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f694000c450 tx=0x7f694000c910 comp rx=0 tx=0).stop 2023-10-26T01:35:26.357 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:26.354+0000 7f694ffff640 1 -- 172.21.15.169:0/2681100706 shutdown_connections 2023-10-26T01:35:26.357 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:26.354+0000 7f694ffff640 1 --2- 172.21.15.169:0/2681100706 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f693006ae40 0x7f693006d300 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:26.357 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:26.354+0000 7f694ffff640 1 --2- 172.21.15.169:0/2681100706 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f695015a5a0 0x7f69500ad6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:26.357 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:26.354+0000 7f694ffff640 1 --2- 172.21.15.169:0/2681100706 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f69500ac760 0x7f695015c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:26.358 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:26.354+0000 7f694ffff640 1 --2- 172.21.15.169:0/2681100706 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6950069900 0x7f69501ef6d0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:26.358 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:26.354+0000 7f694ffff640 1 -- 172.21.15.169:0/2681100706 >> 172.21.15.169:0/2681100706 conn(0x7f69500af290 msgr2=0x7f69500ca0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:35:26.358 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:26.355+0000 7f694ffff640 1 -- 172.21.15.169:0/2681100706 shutdown_connections 2023-10-26T01:35:26.358 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:26.355+0000 7f694ffff640 1 -- 172.21.15.169:0/2681100706 wait complete. 2023-10-26T01:35:26.377 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-26T01:35:26.597 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:26.595+0000 7f93221c0640 1 -- 172.21.15.169:0/1652246272 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f931c1597c0 msgr2=0x7f931c15dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:26.597 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:26.595+0000 7f93221c0640 1 --2- 172.21.15.169:0/1652246272 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f931c1597c0 0x7f931c15dbf0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9310009830 tx=0x7f931002bd50 comp rx=0 tx=0).stop 2023-10-26T01:35:26.598 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:26.595+0000 7f93221c0640 1 -- 172.21.15.169:0/1652246272 shutdown_connections 2023-10-26T01:35:26.598 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:26.595+0000 7f93221c0640 1 --2- 172.21.15.169:0/1652246272 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f931c1597c0 0x7f931c15dbf0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:26.598 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:26.595+0000 7f93221c0640 1 --2- 172.21.15.169:0/1652246272 >> v2:172.21.15.170:3300/0 conn(0x7f931c1593e0 0x7f931c0ac200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:26.598 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:26.595+0000 7f93221c0640 1 -- 172.21.15.169:0/1652246272 >> 172.21.15.169:0/1652246272 conn(0x7f931c069930 msgr2=0x7f931c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:35:26.598 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:26.595+0000 7f93221c0640 1 -- 172.21.15.169:0/1652246272 shutdown_connections 2023-10-26T01:35:26.599 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:26.595+0000 7f93221c0640 1 -- 172.21.15.169:0/1652246272 wait complete. 2023-10-26T01:35:26.599 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:26.597+0000 7f93221c0640 1 Processor -- start 2023-10-26T01:35:26.599 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:26.597+0000 7f93221c0640 1 -- start start 2023-10-26T01:35:26.601 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:26.598+0000 7f93221c0640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f931c0b0c80 0x7f931c070a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:26.601 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:26.598+0000 7f93221c0640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f931c1593e0 0x7f931c153610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:26.601 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:26.598+0000 7f93221c0640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f931c1597c0 0x7f931c153b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:26.601 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:26.598+0000 7f93221c0640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f931c156680 con 0x7f931c0b0c80 2023-10-26T01:35:26.602 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:26.598+0000 7f93221c0640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f931c157800 con 0x7f931c1593e0 2023-10-26T01:35:26.602 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:26.598+0000 7f93221c0640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f931c157ae0 con 0x7f931c1597c0 2023-10-26T01:35:26.602 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:26.599+0000 7f931b7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f931c0b0c80 0x7f931c070a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:26.602 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:26.599+0000 7f931b7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f931c0b0c80 0x7f931c070a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:56878/0 (socket says 172.21.15.169:56878) 2023-10-26T01:35:26.602 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:26.599+0000 7f931b7fe640 1 -- 172.21.15.169:0/309343488 learned_addr learned my addr 172.21.15.169:0/309343488 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:35:26.603 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:26.599+0000 7f931affd640 1 --2- 172.21.15.169:0/309343488 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f931c1593e0 0x7f931c153610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:26.603 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:26.599+0000 7f931bfff640 1 --2- 172.21.15.169:0/309343488 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f931c1597c0 0x7f931c153b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:26.603 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:26.599+0000 7f931b7fe640 1 -- 172.21.15.169:0/309343488 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f931c1597c0 msgr2=0x7f931c153b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:26.603 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:26.599+0000 7f931b7fe640 1 --2- 172.21.15.169:0/309343488 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f931c1597c0 0x7f931c153b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:26.603 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:26.599+0000 7f931b7fe640 1 -- 172.21.15.169:0/309343488 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f931c1593e0 msgr2=0x7f931c153610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:26.603 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:26.599+0000 7f931b7fe640 1 --2- 172.21.15.169:0/309343488 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f931c1593e0 0x7f931c153610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:26.604 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:26.599+0000 7f931b7fe640 1 -- 172.21.15.169:0/309343488 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f930c009490 con 0x7f931c0b0c80 2023-10-26T01:35:26.604 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:26.599+0000 7f931bfff640 1 --2- 172.21.15.169:0/309343488 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f931c1597c0 0x7f931c153b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T01:35:26.605 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:26.600+0000 7f931b7fe640 1 --2- 172.21.15.169:0/309343488 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f931c0b0c80 0x7f931c070a60 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f930c00ea60 tx=0x7f930c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:35:26.605 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:26.600+0000 7f9303fff640 1 -- 172.21.15.169:0/309343488 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f930c00cde0 con 0x7f931c0b0c80 2023-10-26T01:35:26.605 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:26.600+0000 7f9303fff640 1 -- 172.21.15.169:0/309343488 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f930c004020 con 0x7f931c0b0c80 2023-10-26T01:35:26.606 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:26.600+0000 7f9303fff640 1 -- 172.21.15.169:0/309343488 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f930c004410 con 0x7f931c0b0c80 2023-10-26T01:35:26.606 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:26.600+0000 7f93221c0640 1 -- 172.21.15.169:0/309343488 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9310009490 con 0x7f931c0b0c80 2023-10-26T01:35:26.606 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:26.600+0000 7f93221c0640 1 -- 172.21.15.169:0/309343488 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f931c1580a0 con 0x7f931c0b0c80 2023-10-26T01:35:26.606 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:26.601+0000 7f93221c0640 1 -- 172.21.15.169:0/309343488 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f931c158230 con 0x7f931c0b0c80 2023-10-26T01:35:26.606 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:26.603+0000 7f9303fff640 1 -- 172.21.15.169:0/309343488 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f930c0035f0 con 0x7f931c0b0c80 2023-10-26T01:35:26.607 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:26.604+0000 7f9303fff640 1 --2- 172.21.15.169:0/309343488 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f930806ae40 0x7f930806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:26.607 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:26.604+0000 7f9303fff640 1 -- 172.21.15.169:0/309343488 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f930c014070 con 0x7f931c0b0c80 2023-10-26T01:35:26.608 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:26.605+0000 7f931affd640 1 --2- 172.21.15.169:0/309343488 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f930806ae40 0x7f930806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:26.608 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:26.606+0000 7f931affd640 1 --2- 172.21.15.169:0/309343488 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f930806ae40 0x7f930806d300 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f9310036000 tx=0x7f9310004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:35:26.610 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:26.609+0000 7f9303fff640 1 -- 172.21.15.169:0/309343488 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f930c055250 con 0x7f931c0b0c80 2023-10-26T01:35:26.803 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:26.801+0000 7f93221c0640 1 -- 172.21.15.169:0/309343488 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f931c15df50 con 0x7f931c0b0c80 2023-10-26T01:35:27.358 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:27.356+0000 7f9303fff640 1 -- 172.21.15.169:0/309343488 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v82) v1 ==== 170+0+0 (secure 0 0 0) 0x7f930c058870 con 0x7f931c0b0c80 2023-10-26T01:35:27.359 INFO:teuthology.orchestra.run.smithi169.stderr:pool 'cephfs_data' created 2023-10-26T01:35:27.363 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:27.361+0000 7f93221c0640 1 -- 172.21.15.169:0/309343488 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f930806ae40 msgr2=0x7f930806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:27.363 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:27.361+0000 7f93221c0640 1 --2- 172.21.15.169:0/309343488 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f930806ae40 0x7f930806d300 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f9310036000 tx=0x7f9310004730 comp rx=0 tx=0).stop 2023-10-26T01:35:27.363 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:27.361+0000 7f93221c0640 1 -- 172.21.15.169:0/309343488 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f931c0b0c80 msgr2=0x7f931c070a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:27.364 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:27.361+0000 7f93221c0640 1 --2- 172.21.15.169:0/309343488 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f931c0b0c80 0x7f931c070a60 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f930c00ea60 tx=0x7f930c00ef20 comp rx=0 tx=0).stop 2023-10-26T01:35:27.364 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:27.361+0000 7f93221c0640 1 -- 172.21.15.169:0/309343488 shutdown_connections 2023-10-26T01:35:27.364 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:27.361+0000 7f93221c0640 1 --2- 172.21.15.169:0/309343488 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f930806ae40 0x7f930806d300 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:27.364 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:27.361+0000 7f93221c0640 1 --2- 172.21.15.169:0/309343488 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f931c1597c0 0x7f931c153b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:27.364 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:27.361+0000 7f93221c0640 1 --2- 172.21.15.169:0/309343488 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f931c1593e0 0x7f931c153610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:27.365 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:27.361+0000 7f93221c0640 1 --2- 172.21.15.169:0/309343488 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f931c0b0c80 0x7f931c070a60 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:27.365 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:27.361+0000 7f93221c0640 1 -- 172.21.15.169:0/309343488 >> 172.21.15.169:0/309343488 conn(0x7f931c069930 msgr2=0x7f931c151980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:35:27.365 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:27.362+0000 7f93221c0640 1 -- 172.21.15.169:0/309343488 shutdown_connections 2023-10-26T01:35:27.365 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:27.362+0000 7f93221c0640 1 -- 172.21.15.169:0/309343488 wait complete. 2023-10-26T01:35:27.392 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-26T01:35:27.591 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:27.589+0000 7f07dc5b7640 1 -- 172.21.15.169:0/2775368174 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07c0003970 msgr2=0x7f07c0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:27.591 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:27.589+0000 7f07dc5b7640 1 --2- 172.21.15.169:0/2775368174 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07c0003970 0x7f07c0003d90 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f07d0001680 tx=0x7f07c00057f0 comp rx=0 tx=0).stop 2023-10-26T01:35:27.592 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:27.589+0000 7f07dc5b7640 1 -- 172.21.15.169:0/2775368174 shutdown_connections 2023-10-26T01:35:27.592 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:27.589+0000 7f07dc5b7640 1 --2- 172.21.15.169:0/2775368174 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07c0003970 0x7f07c0003d90 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:27.592 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:27.589+0000 7f07dc5b7640 1 -- 172.21.15.169:0/2775368174 >> 172.21.15.169:0/2775368174 conn(0x7f07d4069930 msgr2=0x7f07d4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:35:27.592 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:27.589+0000 7f07dc5b7640 1 -- 172.21.15.169:0/2775368174 shutdown_connections 2023-10-26T01:35:27.592 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:27.589+0000 7f07dc5b7640 1 -- 172.21.15.169:0/2775368174 wait complete. 2023-10-26T01:35:27.593 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:27.591+0000 7f07dc5b7640 1 Processor -- start 2023-10-26T01:35:27.593 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:27.591+0000 7f07dc5b7640 1 -- start start 2023-10-26T01:35:27.595 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:27.592+0000 7f07dc5b7640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07c0003970 0x7f07d40c7180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:27.595 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:27.592+0000 7f07dc5b7640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f07d41f1cc0 0x7f07d40ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:27.595 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:27.592+0000 7f07dc5b7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f07d40ca520 0x7f07d40aee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:27.596 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:27.592+0000 7f07dc5b7640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07d40c7f00 con 0x7f07c0003970 2023-10-26T01:35:27.596 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:27.592+0000 7f07dc5b7640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07d414b010 con 0x7f07d40ca520 2023-10-26T01:35:27.596 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:27.592+0000 7f07dc5b7640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07d40ae450 con 0x7f07d41f1cc0 2023-10-26T01:35:27.596 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:27.592+0000 7f07da353640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07c0003970 0x7f07d40c7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:27.596 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:27.592+0000 7f07da353640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07c0003970 0x7f07d40c7180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:56892/0 (socket says 172.21.15.169:56892) 2023-10-26T01:35:27.597 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:27.592+0000 7f07da353640 1 -- 172.21.15.169:0/2272671366 learned_addr learned my addr 172.21.15.169:0/2272671366 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:35:27.597 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:27.593+0000 7f07d9b52640 1 --2- 172.21.15.169:0/2272671366 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f07d41f1cc0 0x7f07d40ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:27.597 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:27.593+0000 7f07da353640 1 -- 172.21.15.169:0/2272671366 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f07d41f1cc0 msgr2=0x7f07d40ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:27.597 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:27.593+0000 7f07da353640 1 --2- 172.21.15.169:0/2272671366 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f07d41f1cc0 0x7f07d40ae8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:27.597 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:27.593+0000 7f07da353640 1 -- 172.21.15.169:0/2272671366 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f07d40ca520 msgr2=0x7f07d40aee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:27.598 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:27.593+0000 7f07da353640 1 --2- 172.21.15.169:0/2272671366 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f07d40ca520 0x7f07d40aee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:27.598 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:27.593+0000 7f07da353640 1 -- 172.21.15.169:0/2272671366 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07d40c76c0 con 0x7f07c0003970 2023-10-26T01:35:27.598 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:27.593+0000 7f07da353640 1 --2- 172.21.15.169:0/2272671366 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07c0003970 0x7f07d40c7180 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f07d00017a0 tx=0x7f07d00062f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:35:27.598 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:27.593+0000 7f07caffd640 1 -- 172.21.15.169:0/2272671366 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d0003b80 con 0x7f07c0003970 2023-10-26T01:35:27.598 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:27.593+0000 7f07caffd640 1 -- 172.21.15.169:0/2272671366 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f07d0003dd0 con 0x7f07c0003970 2023-10-26T01:35:27.599 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:27.593+0000 7f07caffd640 1 -- 172.21.15.169:0/2272671366 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d002c7b0 con 0x7f07c0003970 2023-10-26T01:35:27.599 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:27.593+0000 7f07dc5b7640 1 -- 172.21.15.169:0/2272671366 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07d40ade80 con 0x7f07c0003970 2023-10-26T01:35:27.599 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:27.594+0000 7f07dc5b7640 1 -- 172.21.15.169:0/2272671366 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07d40c7940 con 0x7f07c0003970 2023-10-26T01:35:27.600 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:27.595+0000 7f07dc5b7640 1 -- 172.21.15.169:0/2272671366 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07d404db10 con 0x7f07c0003970 2023-10-26T01:35:27.601 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:27.596+0000 7f07caffd640 1 -- 172.21.15.169:0/2272671366 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f07d002c910 con 0x7f07c0003970 2023-10-26T01:35:27.601 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:27.597+0000 7f07caffd640 1 --2- 172.21.15.169:0/2272671366 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f07b806ae40 0x7f07b806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:27.601 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:27.597+0000 7f07caffd640 1 -- 172.21.15.169:0/2272671366 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f07d00aa840 con 0x7f07c0003970 2023-10-26T01:35:27.601 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:27.598+0000 7f07d9b52640 1 --2- 172.21.15.169:0/2272671366 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f07b806ae40 0x7f07b806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:27.601 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:27.599+0000 7f07d9b52640 1 --2- 172.21.15.169:0/2272671366 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f07b806ae40 0x7f07b806d300 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f07d40c9860 tx=0x7f07c4005f90 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:35:27.604 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:27.602+0000 7f07caffd640 1 -- 172.21.15.169:0/2272671366 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f07d0075940 con 0x7f07c0003970 2023-10-26T01:35:27.833 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:27.831+0000 7f07dc5b7640 1 -- 172.21.15.169:0/2272671366 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f07d41e83e0 con 0x7f07c0003970 2023-10-26T01:35:28.391 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:28.387+0000 7f07caffd640 1 -- 172.21.15.169:0/2272671366 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 10 and data pool 11 v38) v1 ==== 174+0+0 (secure 0 0 0) 0x7f07d0044020 con 0x7f07c0003970 2023-10-26T01:35:28.391 INFO:teuthology.orchestra.run.smithi169.stderr:new fs with metadata pool 10 and data pool 11 2023-10-26T01:35:28.394 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:28.391+0000 7f07dc5b7640 1 -- 172.21.15.169:0/2272671366 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f07b806ae40 msgr2=0x7f07b806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:28.394 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:28.391+0000 7f07dc5b7640 1 --2- 172.21.15.169:0/2272671366 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f07b806ae40 0x7f07b806d300 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f07d40c9860 tx=0x7f07c4005f90 comp rx=0 tx=0).stop 2023-10-26T01:35:28.395 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:28.391+0000 7f07dc5b7640 1 -- 172.21.15.169:0/2272671366 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07c0003970 msgr2=0x7f07d40c7180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:28.395 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:28.391+0000 7f07dc5b7640 1 --2- 172.21.15.169:0/2272671366 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07c0003970 0x7f07d40c7180 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f07d00017a0 tx=0x7f07d00062f0 comp rx=0 tx=0).stop 2023-10-26T01:35:28.395 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:28.391+0000 7f07dc5b7640 1 -- 172.21.15.169:0/2272671366 shutdown_connections 2023-10-26T01:35:28.395 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:28.391+0000 7f07dc5b7640 1 --2- 172.21.15.169:0/2272671366 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f07b806ae40 0x7f07b806d300 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:28.395 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:28.391+0000 7f07dc5b7640 1 --2- 172.21.15.169:0/2272671366 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f07d40ca520 0x7f07d40aee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:28.395 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:28.391+0000 7f07dc5b7640 1 --2- 172.21.15.169:0/2272671366 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f07d41f1cc0 0x7f07d40ae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:28.396 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:28.391+0000 7f07dc5b7640 1 --2- 172.21.15.169:0/2272671366 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07c0003970 0x7f07d40c7180 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:28.396 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:28.391+0000 7f07dc5b7640 1 -- 172.21.15.169:0/2272671366 >> 172.21.15.169:0/2272671366 conn(0x7f07d4069930 msgr2=0x7f07d415dbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:35:28.396 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:28.391+0000 7f07dc5b7640 1 -- 172.21.15.169:0/2272671366 shutdown_connections 2023-10-26T01:35:28.396 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:28.392+0000 7f07dc5b7640 1 -- 172.21.15.169:0/2272671366 wait complete. 2023-10-26T01:35:28.418 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:35:28.619 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:28.616+0000 7f297c51d640 1 -- 172.21.15.169:0/481287912 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29600039a0 msgr2=0x7f2960003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:28.620 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:28.616+0000 7f297c51d640 1 --2- 172.21.15.169:0/481287912 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29600039a0 0x7f2960003dc0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f2970006fd0 tx=0x7f29600057f0 comp rx=0 tx=0).stop 2023-10-26T01:35:28.620 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:28.616+0000 7f297aaba640 1 -- 172.21.15.169:0/481287912 reap_dead start 2023-10-26T01:35:28.620 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:28.616+0000 7f297c51d640 1 -- 172.21.15.169:0/481287912 shutdown_connections 2023-10-26T01:35:28.620 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:28.616+0000 7f297c51d640 1 -- 172.21.15.169:0/481287912 >> 172.21.15.169:0/481287912 conn(0x7f2974069930 msgr2=0x7f2974069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:35:28.621 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:28.617+0000 7f297c51d640 1 -- 172.21.15.169:0/481287912 shutdown_connections 2023-10-26T01:35:28.621 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:28.617+0000 7f297c51d640 1 -- 172.21.15.169:0/481287912 wait complete. 2023-10-26T01:35:28.623 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:28.618+0000 7f297c51d640 1 Processor -- start 2023-10-26T01:35:28.623 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:28.618+0000 7f297c51d640 1 -- start start 2023-10-26T01:35:28.623 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:28.619+0000 7f297c51d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f29600039a0 0x7f2974145900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:28.623 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:28.619+0000 7f297c51d640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f29740ace30 0x7f29740ae390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:28.624 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:28.619+0000 7f297c51d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29740af220 0x7f29741eecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:28.624 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:28.619+0000 7f297c51d640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2974148d00 con 0x7f29740af220 2023-10-26T01:35:28.624 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:28.619+0000 7f297c51d640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2974145e40 con 0x7f29600039a0 2023-10-26T01:35:28.624 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:28.619+0000 7f297c51d640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29740ad2f0 con 0x7f29740ace30 2023-10-26T01:35:28.624 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:28.620+0000 7f297aaba640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29740af220 0x7f29741eecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:28.625 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:28.620+0000 7f297aaba640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29740af220 0x7f29741eecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:56908/0 (socket says 172.21.15.169:56908) 2023-10-26T01:35:28.625 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:28.620+0000 7f297aaba640 1 -- 172.21.15.169:0/2800585305 learned_addr learned my addr 172.21.15.169:0/2800585305 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:35:28.625 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:28.620+0000 7f297a2b9640 1 --2- 172.21.15.169:0/2800585305 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f29600039a0 0x7f2974145900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:28.625 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:28.620+0000 7f2979ab8640 1 --2- 172.21.15.169:0/2800585305 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f29740ace30 0x7f29740ae390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:28.625 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:28.620+0000 7f297aaba640 1 -- 172.21.15.169:0/2800585305 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f29740ace30 msgr2=0x7f29740ae390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:28.626 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:28.620+0000 7f297aaba640 1 --2- 172.21.15.169:0/2800585305 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f29740ace30 0x7f29740ae390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:28.626 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:28.620+0000 7f297aaba640 1 -- 172.21.15.169:0/2800585305 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f29600039a0 msgr2=0x7f2974145900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:28.626 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:28.620+0000 7f297aaba640 1 --2- 172.21.15.169:0/2800585305 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f29600039a0 0x7f2974145900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:28.626 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:28.620+0000 7f297aaba640 1 -- 172.21.15.169:0/2800585305 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29740ad5d0 con 0x7f29740af220 2023-10-26T01:35:28.626 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:28.620+0000 7f297aaba640 1 --2- 172.21.15.169:0/2800585305 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29740af220 0x7f29741eecd0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f29740b3410 tx=0x7f296c00cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:35:28.628 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:28.621+0000 7f296affd640 1 -- 172.21.15.169:0/2800585305 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f296c005d40 con 0x7f29740af220 2023-10-26T01:35:28.628 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:28.621+0000 7f296affd640 1 -- 172.21.15.169:0/2800585305 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f296c00ced0 con 0x7f29740af220 2023-10-26T01:35:28.628 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:28.621+0000 7f297c51d640 1 -- 172.21.15.169:0/2800585305 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29740ad8b0 con 0x7f29740af220 2023-10-26T01:35:28.629 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:28.621+0000 7f296affd640 1 -- 172.21.15.169:0/2800585305 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f296c00a870 con 0x7f29740af220 2023-10-26T01:35:28.629 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:28.621+0000 7f297c51d640 1 -- 172.21.15.169:0/2800585305 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29740ade70 con 0x7f29740af220 2023-10-26T01:35:28.629 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:28.622+0000 7f297c51d640 1 -- 172.21.15.169:0/2800585305 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29740cd960 con 0x7f29740af220 2023-10-26T01:35:28.629 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:28.624+0000 7f296affd640 1 -- 172.21.15.169:0/2800585305 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f296c00a9d0 con 0x7f29740af220 2023-10-26T01:35:28.630 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:28.624+0000 7f296affd640 1 --2- 172.21.15.169:0/2800585305 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f295806ae40 0x7f295806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:28.630 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:28.625+0000 7f296affd640 1 -- 172.21.15.169:0/2800585305 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f296c088a20 con 0x7f29740af220 2023-10-26T01:35:28.630 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:28.625+0000 7f297a2b9640 1 --2- 172.21.15.169:0/2800585305 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f295806ae40 0x7f295806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:28.630 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:28.626+0000 7f297a2b9640 1 --2- 172.21.15.169:0/2800585305 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f295806ae40 0x7f295806d300 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f2970034000 tx=0x7f297002e000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:35:28.632 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:28.629+0000 7f296affd640 1 -- 172.21.15.169:0/2800585305 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f29740cd960 con 0x7f29740af220 2023-10-26T01:35:28.823 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:28.820+0000 7f297c51d640 1 -- 172.21.15.169:0/2800585305 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f29740cd960 con 0x7f29740af220 2023-10-26T01:35:28.824 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:28.822+0000 7f296affd640 1 -- 172.21.15.169:0/2800585305 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+16629 (secure 0 0 0) 0x7f29740cd960 con 0x7f29740af220 2023-10-26T01:35:28.825 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:35:28.825 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":83,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:35:27.833324+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"72","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-10-26T01:35:26.320950+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-10-26T01:35:26.855916+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:35:28.828 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:28.825+0000 7f297c51d640 1 -- 172.21.15.169:0/2800585305 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f295806ae40 msgr2=0x7f295806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:28.828 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:28.825+0000 7f297c51d640 1 --2- 172.21.15.169:0/2800585305 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f295806ae40 0x7f295806d300 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f2970034000 tx=0x7f297002e000 comp rx=0 tx=0).stop 2023-10-26T01:35:28.828 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:28.825+0000 7f297c51d640 1 -- 172.21.15.169:0/2800585305 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29740af220 msgr2=0x7f29741eecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:28.828 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:28.825+0000 7f297c51d640 1 --2- 172.21.15.169:0/2800585305 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29740af220 0x7f29741eecd0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f29740b3410 tx=0x7f296c00cb90 comp rx=0 tx=0).stop 2023-10-26T01:35:28.828 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:28.825+0000 7f297c51d640 1 -- 172.21.15.169:0/2800585305 shutdown_connections 2023-10-26T01:35:28.829 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:28.825+0000 7f297c51d640 1 --2- 172.21.15.169:0/2800585305 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f295806ae40 0x7f295806d300 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:28.829 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:28.825+0000 7f297c51d640 1 --2- 172.21.15.169:0/2800585305 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29740af220 0x7f29741eecd0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:28.829 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:28.825+0000 7f297c51d640 1 --2- 172.21.15.169:0/2800585305 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f29740ace30 0x7f29740ae390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:28.829 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:28.825+0000 7f297c51d640 1 --2- 172.21.15.169:0/2800585305 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f29600039a0 0x7f2974145900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:28.829 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:28.825+0000 7f297c51d640 1 -- 172.21.15.169:0/2800585305 >> 172.21.15.169:0/2800585305 conn(0x7f2974069930 msgr2=0x7f2974153bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:35:28.830 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:28.825+0000 7f297c51d640 1 -- 172.21.15.169:0/2800585305 shutdown_connections 2023-10-26T01:35:28.830 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:28.825+0000 7f297c51d640 1 -- 172.21.15.169:0/2800585305 wait complete. 2023-10-26T01:35:28.847 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:35:29.069 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:29.066+0000 7facf83b0640 1 -- 172.21.15.169:0/3592840880 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7facf015c6b0 msgr2=0x7facf015bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:29.069 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:29.066+0000 7facf83b0640 1 --2- 172.21.15.169:0/3592840880 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7facf015c6b0 0x7facf015bbc0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7facf0160250 tx=0x7facec02bcf0 comp rx=0 tx=0).stop 2023-10-26T01:35:29.070 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:29.066+0000 7facf83b0640 1 -- 172.21.15.169:0/3592840880 shutdown_connections 2023-10-26T01:35:29.070 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:29.066+0000 7facf83b0640 1 --2- 172.21.15.169:0/3592840880 >> v2:172.21.15.170:3300/0 conn(0x7facf00ab8d0 0x7facf006f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:29.070 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:29.066+0000 7facf83b0640 1 --2- 172.21.15.169:0/3592840880 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7facf015c6b0 0x7facf015bbc0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:29.070 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:29.066+0000 7facf83b0640 1 -- 172.21.15.169:0/3592840880 >> 172.21.15.169:0/3592840880 conn(0x7facf00af7c0 msgr2=0x7facf00afba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:35:29.070 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:29.066+0000 7facf83b0640 1 -- 172.21.15.169:0/3592840880 shutdown_connections 2023-10-26T01:35:29.071 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:29.066+0000 7facf83b0640 1 -- 172.21.15.169:0/3592840880 wait complete. 2023-10-26T01:35:29.072 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:29.068+0000 7facf83b0640 1 Processor -- start 2023-10-26T01:35:29.072 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:29.068+0000 7facf83b0640 1 -- start start 2023-10-26T01:35:29.072 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:29.069+0000 7facf83b0640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7facf006fb10 0x7facf013a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:29.072 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:29.069+0000 7facf83b0640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7facf00ab8d0 0x7facf00ccc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:29.073 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:29.069+0000 7facf83b0640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7facf015c6b0 0x7facf01f1b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:29.073 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:29.069+0000 7facf83b0640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facf013e270 con 0x7facf015c6b0 2023-10-26T01:35:29.073 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:29.069+0000 7facf83b0640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facf013a600 con 0x7facf00ab8d0 2023-10-26T01:35:29.073 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:29.069+0000 7facf83b0640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facf01e7820 con 0x7facf006fb10 2023-10-26T01:35:29.073 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:29.069+0000 7facf694d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7facf015c6b0 0x7facf01f1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:29.074 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:29.069+0000 7facf694d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7facf015c6b0 0x7facf01f1b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:56920/0 (socket says 172.21.15.169:56920) 2023-10-26T01:35:29.074 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:29.069+0000 7facf694d640 1 -- 172.21.15.169:0/941992536 learned_addr learned my addr 172.21.15.169:0/941992536 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:35:29.076 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:29.070+0000 7facf614c640 1 --2- 172.21.15.169:0/941992536 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7facf006fb10 0x7facf013a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:29.076 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:29.070+0000 7facf594b640 1 --2- 172.21.15.169:0/941992536 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7facf00ab8d0 0x7facf00ccc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:29.076 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:29.070+0000 7facf694d640 1 -- 172.21.15.169:0/941992536 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7facf006fb10 msgr2=0x7facf013a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:29.076 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:29.070+0000 7facf694d640 1 --2- 172.21.15.169:0/941992536 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7facf006fb10 0x7facf013a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:29.077 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:29.070+0000 7facf694d640 1 -- 172.21.15.169:0/941992536 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7facf00ab8d0 msgr2=0x7facf00ccc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:29.077 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:29.070+0000 7facf694d640 1 --2- 172.21.15.169:0/941992536 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7facf00ab8d0 0x7facf00ccc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:29.077 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:29.070+0000 7facf694d640 1 -- 172.21.15.169:0/941992536 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facec009490 con 0x7facf015c6b0 2023-10-26T01:35:29.077 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:29.070+0000 7facf594b640 1 --2- 172.21.15.169:0/941992536 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7facf00ab8d0 0x7facf00ccc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:35:29.077 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:29.070+0000 7facf694d640 1 --2- 172.21.15.169:0/941992536 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7facf015c6b0 0x7facf01f1b70 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7face800cbc0 tx=0x7face800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:35:29.078 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:29.070+0000 7face6ffd640 1 -- 172.21.15.169:0/941992536 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face800aca0 con 0x7facf015c6b0 2023-10-26T01:35:29.078 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:29.071+0000 7face6ffd640 1 -- 172.21.15.169:0/941992536 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7face8003970 con 0x7facf015c6b0 2023-10-26T01:35:29.078 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:29.071+0000 7face6ffd640 1 -- 172.21.15.169:0/941992536 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face8003d80 con 0x7facf015c6b0 2023-10-26T01:35:29.078 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:29.071+0000 7facf83b0640 1 -- 172.21.15.169:0/941992536 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facf01e7b00 con 0x7facf015c6b0 2023-10-26T01:35:29.078 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:29.071+0000 7facf83b0640 1 -- 172.21.15.169:0/941992536 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facf01f2980 con 0x7facf015c6b0 2023-10-26T01:35:29.079 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:29.072+0000 7facf83b0640 1 -- 172.21.15.169:0/941992536 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facf004db10 con 0x7facf015c6b0 2023-10-26T01:35:29.079 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:29.073+0000 7face6ffd640 1 -- 172.21.15.169:0/941992536 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7face8004030 con 0x7facf015c6b0 2023-10-26T01:35:29.079 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:29.074+0000 7face6ffd640 1 --2- 172.21.15.169:0/941992536 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7facd406ae40 0x7facd406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:29.080 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:29.074+0000 7face6ffd640 1 -- 172.21.15.169:0/941992536 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7face808a1a0 con 0x7facf015c6b0 2023-10-26T01:35:29.080 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:29.075+0000 7facf614c640 1 --2- 172.21.15.169:0/941992536 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7facd406ae40 0x7facd406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:29.080 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:29.076+0000 7facf614c640 1 --2- 172.21.15.169:0/941992536 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7facd406ae40 0x7facd406d300 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7facec036000 tx=0x7facec004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:35:29.081 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:29.079+0000 7face6ffd640 1 -- 172.21.15.169:0/941992536 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7face80551b0 con 0x7facf015c6b0 2023-10-26T01:35:29.273 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:29.271+0000 7facf83b0640 1 -- 172.21.15.169:0/941992536 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7facf0066a60 con 0x7facf015c6b0 2023-10-26T01:35:29.275 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:29.273+0000 7face6ffd640 1 -- 172.21.15.169:0/941992536 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+16616 (secure 0 0 0) 0x7face80587d0 con 0x7facf015c6b0 2023-10-26T01:35:29.276 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:35:29.276 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":84,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:35:28.979183+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"72","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-10-26T01:35:26.320950+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-10-26T01:35:26.855916+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:35:29.279 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:29.276+0000 7facf83b0640 1 -- 172.21.15.169:0/941992536 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7facd406ae40 msgr2=0x7facd406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:29.279 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:29.276+0000 7facf83b0640 1 --2- 172.21.15.169:0/941992536 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7facd406ae40 0x7facd406d300 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7facec036000 tx=0x7facec004730 comp rx=0 tx=0).stop 2023-10-26T01:35:29.279 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:29.276+0000 7facf83b0640 1 -- 172.21.15.169:0/941992536 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7facf015c6b0 msgr2=0x7facf01f1b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:29.279 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:29.276+0000 7facf83b0640 1 --2- 172.21.15.169:0/941992536 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7facf015c6b0 0x7facf01f1b70 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7face800cbc0 tx=0x7face800a060 comp rx=0 tx=0).stop 2023-10-26T01:35:29.280 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:29.276+0000 7facf83b0640 1 -- 172.21.15.169:0/941992536 shutdown_connections 2023-10-26T01:35:29.280 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:29.276+0000 7facf83b0640 1 --2- 172.21.15.169:0/941992536 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7facd406ae40 0x7facd406d300 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:29.280 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:29.276+0000 7facf83b0640 1 --2- 172.21.15.169:0/941992536 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7facf015c6b0 0x7facf01f1b70 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:29.280 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:29.276+0000 7facf83b0640 1 --2- 172.21.15.169:0/941992536 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7facf00ab8d0 0x7facf00ccc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:29.280 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:29.276+0000 7facf83b0640 1 --2- 172.21.15.169:0/941992536 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7facf006fb10 0x7facf013a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:29.281 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:29.276+0000 7facf83b0640 1 -- 172.21.15.169:0/941992536 >> 172.21.15.169:0/941992536 conn(0x7facf00af7c0 msgr2=0x7facf0155a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:35:29.281 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:29.276+0000 7facf83b0640 1 -- 172.21.15.169:0/941992536 shutdown_connections 2023-10-26T01:35:29.281 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:29.276+0000 7facf83b0640 1 -- 172.21.15.169:0/941992536 wait complete. 2023-10-26T01:35:29.297 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-26T01:35:29.477 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2023-10-26T01:35:29.519 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:29.516+0000 7f73db24f640 1 -- 172.21.15.169:0/575001268 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f73d406ce00 msgr2=0x7f73d4155360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:29.520 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:29.516+0000 7f73db24f640 1 --2- 172.21.15.169:0/575001268 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f73d406ce00 0x7f73d4155360 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f73d40b0570 tx=0x7f73c802bcc0 comp rx=0 tx=0).stop 2023-10-26T01:35:29.520 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:29.516+0000 7f73db24f640 1 -- 172.21.15.169:0/575001268 shutdown_connections 2023-10-26T01:35:29.520 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:29.516+0000 7f73db24f640 1 --2- 172.21.15.169:0/575001268 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f73d40b05a0 0x7f73d40b09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:29.520 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:29.516+0000 7f73db24f640 1 --2- 172.21.15.169:0/575001268 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f73d406ce00 0x7f73d4155360 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:29.520 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:29.516+0000 7f73db24f640 1 -- 172.21.15.169:0/575001268 >> 172.21.15.169:0/575001268 conn(0x7f73d4069930 msgr2=0x7f73d4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:35:29.521 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:29.517+0000 7f73db24f640 1 -- 172.21.15.169:0/575001268 shutdown_connections 2023-10-26T01:35:29.521 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:29.517+0000 7f73db24f640 1 -- 172.21.15.169:0/575001268 wait complete. 2023-10-26T01:35:29.522 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:29.518+0000 7f73db24f640 1 Processor -- start 2023-10-26T01:35:29.522 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:29.518+0000 7f73db24f640 1 -- start start 2023-10-26T01:35:29.522 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:29.519+0000 7f73db24f640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f73d406ce00 0x7f73d4152d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:29.523 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:29.519+0000 7f73db24f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f73d40b05a0 0x7f73d414fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:29.523 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:29.519+0000 7f73db24f640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f73d415b4a0 0x7f73d41503b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:29.523 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:29.519+0000 7f73db24f640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73d41597f0 con 0x7f73d406ce00 2023-10-26T01:35:29.523 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:29.519+0000 7f73db24f640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73d40cc700 con 0x7f73d40b05a0 2023-10-26T01:35:29.523 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:29.519+0000 7f73db24f640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73d40cc9e0 con 0x7f73d415b4a0 2023-10-26T01:35:29.524 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:29.520+0000 7f73d8feb640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f73d406ce00 0x7f73d4152d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:29.524 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:29.520+0000 7f73d8feb640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f73d406ce00 0x7f73d4152d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:56940/0 (socket says 172.21.15.169:56940) 2023-10-26T01:35:29.524 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:29.520+0000 7f73d8feb640 1 -- 172.21.15.169:0/3846611676 learned_addr learned my addr 172.21.15.169:0/3846611676 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:35:29.525 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:29.520+0000 7f73d97ec640 1 --2- 172.21.15.169:0/3846611676 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f73d415b4a0 0x7f73d41503b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:29.526 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:29.520+0000 7f73d8feb640 1 -- 172.21.15.169:0/3846611676 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f73d415b4a0 msgr2=0x7f73d41503b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:29.526 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:29.520+0000 7f73d8feb640 1 --2- 172.21.15.169:0/3846611676 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f73d415b4a0 0x7f73d41503b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:29.526 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:29.520+0000 7f73d8feb640 1 -- 172.21.15.169:0/3846611676 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f73d40b05a0 msgr2=0x7f73d414fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:29.526 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:29.520+0000 7f73d3fff640 1 --2- 172.21.15.169:0/3846611676 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f73d40b05a0 0x7f73d414fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:29.527 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:29.520+0000 7f73d8feb640 1 --2- 172.21.15.169:0/3846611676 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f73d40b05a0 0x7f73d414fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:29.527 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:29.520+0000 7f73d8feb640 1 -- 172.21.15.169:0/3846611676 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73c8009560 con 0x7f73d406ce00 2023-10-26T01:35:29.527 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:29.520+0000 7f73d97ec640 1 --2- 172.21.15.169:0/3846611676 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f73d415b4a0 0x7f73d41503b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:35:29.527 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:29.520+0000 7f73d3fff640 1 --2- 172.21.15.169:0/3846611676 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f73d40b05a0 0x7f73d414fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:35:29.527 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:29.520+0000 7f73d8feb640 1 --2- 172.21.15.169:0/3846611676 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f73d406ce00 0x7f73d4152d20 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f73c8036040 tx=0x7f73c802dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:35:29.528 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:29.521+0000 7f73d17fa640 1 -- 172.21.15.169:0/3846611676 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73c8003800 con 0x7f73d406ce00 2023-10-26T01:35:29.528 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:29.521+0000 7f73d17fa640 1 -- 172.21.15.169:0/3846611676 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f73c8003a50 con 0x7f73d406ce00 2023-10-26T01:35:29.528 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:29.521+0000 7f73db24f640 1 -- 172.21.15.169:0/3846611676 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73d4159550 con 0x7f73d406ce00 2023-10-26T01:35:29.528 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:29.521+0000 7f73d17fa640 1 -- 172.21.15.169:0/3846611676 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73c8003de0 con 0x7f73d406ce00 2023-10-26T01:35:29.528 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:29.521+0000 7f73db24f640 1 -- 172.21.15.169:0/3846611676 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73d41555a0 con 0x7f73d406ce00 2023-10-26T01:35:29.529 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:29.522+0000 7f73db24f640 1 -- 172.21.15.169:0/3846611676 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73d40cc1b0 con 0x7f73d406ce00 2023-10-26T01:35:29.529 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:29.523+0000 7f73d17fa640 1 -- 172.21.15.169:0/3846611676 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f73c803e450 con 0x7f73d406ce00 2023-10-26T01:35:29.529 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:29.524+0000 7f73d17fa640 1 --2- 172.21.15.169:0/3846611676 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f73c006ae40 0x7f73c006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:29.530 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:29.524+0000 7f73d17fa640 1 -- 172.21.15.169:0/3846611676 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f73c80abe80 con 0x7f73d406ce00 2023-10-26T01:35:29.530 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:29.525+0000 7f73d3fff640 1 --2- 172.21.15.169:0/3846611676 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f73c006ae40 0x7f73c006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:29.530 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:29.526+0000 7f73d3fff640 1 --2- 172.21.15.169:0/3846611676 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f73c006ae40 0x7f73c006d300 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f73d40721b0 tx=0x7f73c4008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:35:29.531 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:29.528+0000 7f73d17fa640 1 -- 172.21.15.169:0/3846611676 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f73d40cc1b0 con 0x7f73d406ce00 2023-10-26T01:35:29.761 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:29.758+0000 7f73db24f640 1 -- 172.21.15.169:0/3846611676 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f73d40cc1b0 con 0x7f73d406ce00 2023-10-26T01:35:30.416 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:30.413+0000 7f73d17fa640 1 -- 172.21.15.169:0/3846611676 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v41) v1 ==== 118+0+0 (secure 0 0 0) 0x7f73d40cc1b0 con 0x7f73d406ce00 2023-10-26T01:35:30.422 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:30.419+0000 7f73db24f640 1 -- 172.21.15.169:0/3846611676 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f73c006ae40 msgr2=0x7f73c006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:30.422 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:30.419+0000 7f73db24f640 1 --2- 172.21.15.169:0/3846611676 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f73c006ae40 0x7f73c006d300 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f73d40721b0 tx=0x7f73c4008040 comp rx=0 tx=0).stop 2023-10-26T01:35:30.422 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:30.419+0000 7f73db24f640 1 -- 172.21.15.169:0/3846611676 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f73d406ce00 msgr2=0x7f73d4152d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:30.422 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:30.419+0000 7f73db24f640 1 --2- 172.21.15.169:0/3846611676 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f73d406ce00 0x7f73d4152d20 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f73c8036040 tx=0x7f73c802dc90 comp rx=0 tx=0).stop 2023-10-26T01:35:30.423 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:30.419+0000 7f73db24f640 1 -- 172.21.15.169:0/3846611676 shutdown_connections 2023-10-26T01:35:30.423 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:30.419+0000 7f73db24f640 1 --2- 172.21.15.169:0/3846611676 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f73c006ae40 0x7f73c006d300 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:30.423 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:30.419+0000 7f73db24f640 1 --2- 172.21.15.169:0/3846611676 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f73d415b4a0 0x7f73d41503b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:30.423 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:30.419+0000 7f73db24f640 1 --2- 172.21.15.169:0/3846611676 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f73d40b05a0 0x7f73d414fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:30.424 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:30.419+0000 7f73db24f640 1 --2- 172.21.15.169:0/3846611676 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f73d406ce00 0x7f73d4152d20 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:30.424 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:30.419+0000 7f73db24f640 1 -- 172.21.15.169:0/3846611676 >> 172.21.15.169:0/3846611676 conn(0x7f73d4069930 msgr2=0x7f73d4156de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:35:30.424 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:30.419+0000 7f73db24f640 1 -- 172.21.15.169:0/3846611676 shutdown_connections 2023-10-26T01:35:30.424 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:30.419+0000 7f73db24f640 1 -- 172.21.15.169:0/3846611676 wait complete. 2023-10-26T01:35:30.447 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-26T01:35:30.447 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-26T01:35:30.674 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:30.670+0000 7f4f2e875640 1 -- 172.21.15.169:0/1056844306 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4f281593e0 msgr2=0x7f4f280ac200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:30.674 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:30.670+0000 7f4f2e875640 1 --2- 172.21.15.169:0/1056844306 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4f281593e0 0x7f4f280ac200 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f4f18005fd0 tx=0x7f4f1802bd80 comp rx=0 tx=0).stop 2023-10-26T01:35:30.674 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:30.670+0000 7f4f2e875640 1 -- 172.21.15.169:0/1056844306 shutdown_connections 2023-10-26T01:35:30.674 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:30.670+0000 7f4f2e875640 1 --2- 172.21.15.169:0/1056844306 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4f280b0c80 0x7f4f280cd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:30.675 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:30.670+0000 7f4f2e875640 1 --2- 172.21.15.169:0/1056844306 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4f281593e0 0x7f4f280ac200 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:30.675 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:30.670+0000 7f4f2e875640 1 -- 172.21.15.169:0/1056844306 >> 172.21.15.169:0/1056844306 conn(0x7f4f28069930 msgr2=0x7f4f28069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:35:30.675 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:30.670+0000 7f4f2e875640 1 -- 172.21.15.169:0/1056844306 shutdown_connections 2023-10-26T01:35:30.675 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:30.670+0000 7f4f2e875640 1 -- 172.21.15.169:0/1056844306 wait complete. 2023-10-26T01:35:30.677 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:30.672+0000 7f4f2e875640 1 Processor -- start 2023-10-26T01:35:30.677 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:30.672+0000 7f4f2e875640 1 -- start start 2023-10-26T01:35:30.678 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:30.673+0000 7f4f2e875640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4f280b0c80 0x7f4f2814d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:30.678 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:30.673+0000 7f4f2e875640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4f281593e0 0x7f4f280c7b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:30.678 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:30.673+0000 7f4f2e875640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4f281597c0 0x7f4f2806b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:30.678 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:30.673+0000 7f4f2e875640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f28156c40 con 0x7f4f280b0c80 2023-10-26T01:35:30.678 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:30.673+0000 7f4f2e875640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f281568f0 con 0x7f4f281593e0 2023-10-26T01:35:30.679 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:30.673+0000 7f4f2e875640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f280ccfc0 con 0x7f4f281597c0 2023-10-26T01:35:30.679 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:30.673+0000 7f4f27fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4f280b0c80 0x7f4f2814d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:30.680 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:30.673+0000 7f4f27fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4f280b0c80 0x7f4f2814d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:56966/0 (socket says 172.21.15.169:56966) 2023-10-26T01:35:30.680 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:30.673+0000 7f4f27fff640 1 -- 172.21.15.169:0/3365299458 learned_addr learned my addr 172.21.15.169:0/3365299458 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:35:30.680 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:30.674+0000 7f4f27fff640 1 -- 172.21.15.169:0/3365299458 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4f281597c0 msgr2=0x7f4f2806b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:30.680 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:30.674+0000 7f4f27fff640 1 --2- 172.21.15.169:0/3365299458 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4f281597c0 0x7f4f2806b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:30.680 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:30.674+0000 7f4f27fff640 1 -- 172.21.15.169:0/3365299458 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4f281593e0 msgr2=0x7f4f280c7b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:30.681 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:30.674+0000 7f4f27fff640 1 --2- 172.21.15.169:0/3365299458 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4f281593e0 0x7f4f280c7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:30.681 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:30.674+0000 7f4f27fff640 1 -- 172.21.15.169:0/3365299458 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f18009560 con 0x7f4f280b0c80 2023-10-26T01:35:30.681 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:30.674+0000 7f4f27fff640 1 --2- 172.21.15.169:0/3365299458 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4f280b0c80 0x7f4f2814d900 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f4f18009850 tx=0x7f4f1802db00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:35:30.681 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:30.674+0000 7f4f24ff9640 1 -- 172.21.15.169:0/3365299458 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f180035f0 con 0x7f4f280b0c80 2023-10-26T01:35:30.681 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:30.674+0000 7f4f24ff9640 1 -- 172.21.15.169:0/3365299458 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4f18003840 con 0x7f4f280b0c80 2023-10-26T01:35:30.682 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:30.674+0000 7f4f24ff9640 1 -- 172.21.15.169:0/3365299458 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f18003bf0 con 0x7f4f280b0c80 2023-10-26T01:35:30.682 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:30.674+0000 7f4f2e875640 1 -- 172.21.15.169:0/3365299458 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f2815bc70 con 0x7f4f280b0c80 2023-10-26T01:35:30.682 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:30.674+0000 7f4f2e875640 1 -- 172.21.15.169:0/3365299458 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f2815c230 con 0x7f4f280b0c80 2023-10-26T01:35:30.683 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:30.676+0000 7f4f2e875640 1 -- 172.21.15.169:0/3365299458 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f2804db10 con 0x7f4f280b0c80 2023-10-26T01:35:30.683 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:30.676+0000 7f4f24ff9640 1 -- 172.21.15.169:0/3365299458 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f4f18003d50 con 0x7f4f280b0c80 2023-10-26T01:35:30.683 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:30.678+0000 7f4f24ff9640 1 --2- 172.21.15.169:0/3365299458 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f4f1406ae70 0x7f4f1406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:30.684 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:30.678+0000 7f4f24ff9640 1 -- 172.21.15.169:0/3365299458 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4f180ac230 con 0x7f4f280b0c80 2023-10-26T01:35:30.684 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:30.679+0000 7f4f277fe640 1 --2- 172.21.15.169:0/3365299458 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f4f1406ae70 0x7f4f1406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:30.684 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:30.680+0000 7f4f277fe640 1 --2- 172.21.15.169:0/3365299458 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f4f1406ae70 0x7f4f1406d330 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f4f281582e0 tx=0x7f4f1c006ba0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:35:30.685 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:30.682+0000 7f4f24ff9640 1 -- 172.21.15.169:0/3365299458 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4f180773f0 con 0x7f4f280b0c80 2023-10-26T01:35:30.917 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:30.914+0000 7f4f2e875640 1 -- 172.21.15.169:0/3365299458 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f4f280b2170 con 0x7f4f280b0c80 2023-10-26T01:35:31.429 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:31.426+0000 7f4f24ff9640 1 -- 172.21.15.169:0/3365299458 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v42) v1 ==== 122+0+0 (secure 0 0 0) 0x7f4f1807aa10 con 0x7f4f280b0c80 2023-10-26T01:35:31.434 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:31.430+0000 7f4f2e875640 1 -- 172.21.15.169:0/3365299458 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f4f1406ae70 msgr2=0x7f4f1406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:31.434 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:31.430+0000 7f4f2e875640 1 --2- 172.21.15.169:0/3365299458 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f4f1406ae70 0x7f4f1406d330 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f4f281582e0 tx=0x7f4f1c006ba0 comp rx=0 tx=0).stop 2023-10-26T01:35:31.435 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:31.430+0000 7f4f2e875640 1 -- 172.21.15.169:0/3365299458 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4f280b0c80 msgr2=0x7f4f2814d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:31.435 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:31.430+0000 7f4f2e875640 1 --2- 172.21.15.169:0/3365299458 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4f280b0c80 0x7f4f2814d900 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f4f18009850 tx=0x7f4f1802db00 comp rx=0 tx=0).stop 2023-10-26T01:35:31.435 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:31.430+0000 7f4f2e875640 1 -- 172.21.15.169:0/3365299458 shutdown_connections 2023-10-26T01:35:31.436 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:31.430+0000 7f4f2e875640 1 --2- 172.21.15.169:0/3365299458 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f4f1406ae70 0x7f4f1406d330 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:31.436 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:31.430+0000 7f4f2e875640 1 --2- 172.21.15.169:0/3365299458 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4f281597c0 0x7f4f2806b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:31.436 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:31.430+0000 7f4f2e875640 1 --2- 172.21.15.169:0/3365299458 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4f281593e0 0x7f4f280c7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:31.436 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:31.430+0000 7f4f2e875640 1 --2- 172.21.15.169:0/3365299458 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4f280b0c80 0x7f4f2814d900 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:31.436 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:31.430+0000 7f4f2e875640 1 -- 172.21.15.169:0/3365299458 >> 172.21.15.169:0/3365299458 conn(0x7f4f28069930 msgr2=0x7f4f280cc1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:35:31.437 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:31.431+0000 7f4f2e875640 1 -- 172.21.15.169:0/3365299458 shutdown_connections 2023-10-26T01:35:31.437 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:31.431+0000 7f4f2e875640 1 -- 172.21.15.169:0/3365299458 wait complete. 2023-10-26T01:35:31.455 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-26T01:35:31.456 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T01:35:31.658 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:31.655+0000 7f46f4d8a640 1 -- 172.21.15.169:0/1373678389 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f46f01592d0 msgr2=0x7f46f00aa3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:31.658 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:31.655+0000 7f46f4d8a640 1 --2- 172.21.15.169:0/1373678389 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f46f01592d0 0x7f46f00aa3d0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f46e0005fd0 tx=0x7f46e002bd80 comp rx=0 tx=0).stop 2023-10-26T01:35:31.659 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:31.655+0000 7f46f4d8a640 1 -- 172.21.15.169:0/1373678389 shutdown_connections 2023-10-26T01:35:31.659 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:31.655+0000 7f46f4d8a640 1 --2- 172.21.15.169:0/1373678389 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f46f00cde00 0x7f46f00cbac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:31.659 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:31.655+0000 7f46f4d8a640 1 --2- 172.21.15.169:0/1373678389 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f46f01592d0 0x7f46f00aa3d0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:31.659 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:31.655+0000 7f46f4d8a640 1 -- 172.21.15.169:0/1373678389 >> 172.21.15.169:0/1373678389 conn(0x7f46f0069930 msgr2=0x7f46f0069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:35:31.659 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:31.655+0000 7f46f4d8a640 1 -- 172.21.15.169:0/1373678389 shutdown_connections 2023-10-26T01:35:31.660 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:31.655+0000 7f46f4d8a640 1 -- 172.21.15.169:0/1373678389 wait complete. 2023-10-26T01:35:31.661 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:31.657+0000 7f46f4d8a640 1 Processor -- start 2023-10-26T01:35:31.661 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:31.657+0000 7f46f4d8a640 1 -- start start 2023-10-26T01:35:31.662 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:31.658+0000 7f46f4d8a640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f46f00ac930 0x7f46f0155600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:31.662 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:31.658+0000 7f46f4d8a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f46f00cde00 0x7f46f00c6560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:31.662 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:31.658+0000 7f46f4d8a640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f46f01592d0 0x7f46f00c6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:31.662 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:31.658+0000 7f46f4d8a640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46f00aeec0 con 0x7f46f00ac930 2023-10-26T01:35:31.663 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:31.658+0000 7f46f4d8a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46f00b1100 con 0x7f46f00cde00 2023-10-26T01:35:31.663 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:31.658+0000 7f46f4d8a640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46f00c70d0 con 0x7f46f01592d0 2023-10-26T01:35:31.663 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:31.658+0000 7f46ee59c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f46f00ac930 0x7f46f0155600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:31.663 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:31.658+0000 7f46ee59c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f46f00ac930 0x7f46f0155600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:56986/0 (socket says 172.21.15.169:56986) 2023-10-26T01:35:31.663 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:31.658+0000 7f46ee59c640 1 -- 172.21.15.169:0/4106528384 learned_addr learned my addr 172.21.15.169:0/4106528384 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:35:31.664 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:31.659+0000 7f46ee59c640 1 -- 172.21.15.169:0/4106528384 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f46f01592d0 msgr2=0x7f46f00c6aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:31.664 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:31.659+0000 7f46ee59c640 1 --2- 172.21.15.169:0/4106528384 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f46f01592d0 0x7f46f00c6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:31.664 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:31.659+0000 7f46ee59c640 1 -- 172.21.15.169:0/4106528384 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f46f00cde00 msgr2=0x7f46f00c6560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:31.664 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:31.659+0000 7f46ee59c640 1 --2- 172.21.15.169:0/4106528384 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f46f00cde00 0x7f46f00c6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:31.664 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:31.659+0000 7f46ee59c640 1 -- 172.21.15.169:0/4106528384 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46e0009560 con 0x7f46f00ac930 2023-10-26T01:35:31.666 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:31.659+0000 7f46ee59c640 1 --2- 172.21.15.169:0/4106528384 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f46f00ac930 0x7f46f0155600 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f46e0009850 tx=0x7f46e002db00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:35:31.666 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:31.659+0000 7f46e6ffd640 1 -- 172.21.15.169:0/4106528384 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46e00035f0 con 0x7f46f00ac930 2023-10-26T01:35:31.666 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:31.659+0000 7f46e6ffd640 1 -- 172.21.15.169:0/4106528384 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f46e0003840 con 0x7f46f00ac930 2023-10-26T01:35:31.667 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:31.659+0000 7f46f4d8a640 1 -- 172.21.15.169:0/4106528384 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46f00c73b0 con 0x7f46f00ac930 2023-10-26T01:35:31.667 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:31.659+0000 7f46e6ffd640 1 -- 172.21.15.169:0/4106528384 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46e0003bf0 con 0x7f46f00ac930 2023-10-26T01:35:31.667 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:31.659+0000 7f46f4d8a640 1 -- 172.21.15.169:0/4106528384 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46f00c7970 con 0x7f46f00ac930 2023-10-26T01:35:31.668 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:31.661+0000 7f46f4d8a640 1 -- 172.21.15.169:0/4106528384 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46f00c7b90 con 0x7f46f00ac930 2023-10-26T01:35:31.668 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:31.661+0000 7f46e6ffd640 1 -- 172.21.15.169:0/4106528384 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f46e0003d50 con 0x7f46f00ac930 2023-10-26T01:35:31.668 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:31.663+0000 7f46e6ffd640 1 --2- 172.21.15.169:0/4106528384 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f46d006ae40 0x7f46d006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:31.669 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:31.663+0000 7f46e6ffd640 1 -- 172.21.15.169:0/4106528384 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f46e00ac1d0 con 0x7f46f00ac930 2023-10-26T01:35:31.669 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:31.663+0000 7f46e7fff640 1 --2- 172.21.15.169:0/4106528384 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f46d006ae40 0x7f46d006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:31.670 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:31.665+0000 7f46e7fff640 1 --2- 172.21.15.169:0/4106528384 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f46d006ae40 0x7f46d006d300 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f46d8000f60 tx=0x7f46d80079b0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:35:31.670 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:31.667+0000 7f46e6ffd640 1 -- 172.21.15.169:0/4106528384 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f46e0077430 con 0x7f46f00ac930 2023-10-26T01:35:31.894 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:31.891+0000 7f46f4d8a640 1 -- 172.21.15.169:0/4106528384 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f46f004db10 con 0x7f46f00ac930 2023-10-26T01:35:31.895 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:31.893+0000 7f46e6ffd640 1 -- 172.21.15.169:0/4106528384 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+4927 (secure 0 0 0) 0x7f46e007aa50 con 0x7f46f00ac930 2023-10-26T01:35:31.896 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:35:31.896 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":42,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5347,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/2629782429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2629782429},{"type":"v1","addr":"172.21.15.170:6836","nonce":2629782429}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":6474,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2561200417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2561200417},{"type":"v1","addr":"172.21.15.169:6833","nonce":2561200417}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":6480,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/2834878721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":2834878721},{"type":"v1","addr":"172.21.15.170:6837","nonce":2834878721}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T01:35:27.834145+0000","modified":"2023-10-26T01:35:30.917014+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6486},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6486":{"gid":6486,"name":"a","rank":0,"incarnation":38,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6835/4004306080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":4004306080},{"type":"v1","addr":"172.21.15.169:6835","nonce":4004306080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-10-26T01:35:31.896 INFO:teuthology.orchestra.run.smithi169.stderr:dumped fsmap epoch 42 2023-10-26T01:35:31.898 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:31.895+0000 7f46f4d8a640 1 -- 172.21.15.169:0/4106528384 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f46d006ae40 msgr2=0x7f46d006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:31.899 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:31.895+0000 7f46f4d8a640 1 --2- 172.21.15.169:0/4106528384 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f46d006ae40 0x7f46d006d300 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f46d8000f60 tx=0x7f46d80079b0 comp rx=0 tx=0).stop 2023-10-26T01:35:31.899 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:31.895+0000 7f46f4d8a640 1 -- 172.21.15.169:0/4106528384 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f46f00ac930 msgr2=0x7f46f0155600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:31.899 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:31.895+0000 7f46f4d8a640 1 --2- 172.21.15.169:0/4106528384 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f46f00ac930 0x7f46f0155600 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f46e0009850 tx=0x7f46e002db00 comp rx=0 tx=0).stop 2023-10-26T01:35:31.901 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:31.896+0000 7f46f4d8a640 1 -- 172.21.15.169:0/4106528384 shutdown_connections 2023-10-26T01:35:31.901 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:31.896+0000 7f46f4d8a640 1 --2- 172.21.15.169:0/4106528384 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f46d006ae40 0x7f46d006d300 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:31.901 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:31.896+0000 7f46f4d8a640 1 --2- 172.21.15.169:0/4106528384 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f46f01592d0 0x7f46f00c6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:31.902 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:31.896+0000 7f46f4d8a640 1 --2- 172.21.15.169:0/4106528384 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f46f00cde00 0x7f46f00c6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:31.902 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:31.896+0000 7f46f4d8a640 1 --2- 172.21.15.169:0/4106528384 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f46f00ac930 0x7f46f0155600 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:31.902 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:31.896+0000 7f46f4d8a640 1 -- 172.21.15.169:0/4106528384 >> 172.21.15.169:0/4106528384 conn(0x7f46f0069930 msgr2=0x7f46f006ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:35:31.902 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:31.896+0000 7f46f4d8a640 1 -- 172.21.15.169:0/4106528384 shutdown_connections 2023-10-26T01:35:31.902 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:31.896+0000 7f46f4d8a640 1 -- 172.21.15.169:0/4106528384 wait complete. 2023-10-26T01:35:31.916 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:35:32.133 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:32.130+0000 7efd53f4c640 1 -- 172.21.15.169:0/731220708 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd4c159e10 msgr2=0x7efd4c15a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:32.133 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:32.130+0000 7efd53f4c640 1 --2- 172.21.15.169:0/731220708 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd4c159e10 0x7efd4c15a2c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7efd44005fd0 tx=0x7efd4402d2e0 comp rx=0 tx=0).stop 2023-10-26T01:35:32.133 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:32.130+0000 7efd53f4c640 1 -- 172.21.15.169:0/731220708 shutdown_connections 2023-10-26T01:35:32.133 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:32.130+0000 7efd53f4c640 1 --2- 172.21.15.169:0/731220708 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd4c159e10 0x7efd4c15a2c0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:32.134 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:32.130+0000 7efd53f4c640 1 --2- 172.21.15.169:0/731220708 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7efd4c06ce20 0x7efd4c14e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:32.134 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:32.130+0000 7efd53f4c640 1 -- 172.21.15.169:0/731220708 >> 172.21.15.169:0/731220708 conn(0x7efd4c069930 msgr2=0x7efd4c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:35:32.134 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:32.130+0000 7efd53f4c640 1 -- 172.21.15.169:0/731220708 shutdown_connections 2023-10-26T01:35:32.134 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:32.130+0000 7efd53f4c640 1 -- 172.21.15.169:0/731220708 wait complete. 2023-10-26T01:35:32.135 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:32.132+0000 7efd53f4c640 1 Processor -- start 2023-10-26T01:35:32.135 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:32.132+0000 7efd53f4c640 1 -- start start 2023-10-26T01:35:32.136 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:32.134+0000 7efd53f4c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd4c06ce20 0x7efd4c1539c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:32.137 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:32.134+0000 7efd53f4c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd4c158850 0x7efd4c145600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:32.137 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:32.134+0000 7efd53f4c640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7efd4c159e10 0x7efd4c156a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:32.137 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:32.134+0000 7efd53f4c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd4c06fe20 con 0x7efd4c06ce20 2023-10-26T01:35:32.137 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:32.134+0000 7efd53f4c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd4c0700c0 con 0x7efd4c158850 2023-10-26T01:35:32.137 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:32.134+0000 7efd53f4c640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd4c0703a0 con 0x7efd4c159e10 2023-10-26T01:35:32.138 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:32.134+0000 7efd51ce8640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd4c06ce20 0x7efd4c1539c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:32.138 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:32.134+0000 7efd51ce8640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd4c06ce20 0x7efd4c1539c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:56992/0 (socket says 172.21.15.169:56992) 2023-10-26T01:35:32.138 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:32.134+0000 7efd51ce8640 1 -- 172.21.15.169:0/2856025433 learned_addr learned my addr 172.21.15.169:0/2856025433 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:35:32.139 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:32.134+0000 7efd514e7640 1 --2- 172.21.15.169:0/2856025433 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd4c158850 0x7efd4c145600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:32.140 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:32.134+0000 7efd524e9640 1 --2- 172.21.15.169:0/2856025433 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7efd4c159e10 0x7efd4c156a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:32.140 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:32.134+0000 7efd51ce8640 1 -- 172.21.15.169:0/2856025433 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7efd4c159e10 msgr2=0x7efd4c156a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:32.140 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:32.134+0000 7efd51ce8640 1 --2- 172.21.15.169:0/2856025433 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7efd4c159e10 0x7efd4c156a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:32.140 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:32.134+0000 7efd51ce8640 1 -- 172.21.15.169:0/2856025433 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd4c158850 msgr2=0x7efd4c145600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:32.140 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:32.134+0000 7efd51ce8640 1 --2- 172.21.15.169:0/2856025433 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd4c158850 0x7efd4c145600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:32.141 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:32.134+0000 7efd51ce8640 1 -- 172.21.15.169:0/2856025433 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd48009560 con 0x7efd4c06ce20 2023-10-26T01:35:32.141 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:32.134+0000 7efd514e7640 1 --2- 172.21.15.169:0/2856025433 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd4c158850 0x7efd4c145600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:35:32.141 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:32.135+0000 7efd51ce8640 1 --2- 172.21.15.169:0/2856025433 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd4c06ce20 0x7efd4c1539c0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7efd4800e9c0 tx=0x7efd4800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:35:32.141 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:32.135+0000 7efd427fc640 1 -- 172.21.15.169:0/2856025433 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd4800cde0 con 0x7efd4c06ce20 2023-10-26T01:35:32.141 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:32.135+0000 7efd427fc640 1 -- 172.21.15.169:0/2856025433 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7efd48004020 con 0x7efd4c06ce20 2023-10-26T01:35:32.142 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:32.135+0000 7efd53f4c640 1 -- 172.21.15.169:0/2856025433 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd4400a950 con 0x7efd4c06ce20 2023-10-26T01:35:32.142 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:32.135+0000 7efd427fc640 1 -- 172.21.15.169:0/2856025433 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd48004410 con 0x7efd4c06ce20 2023-10-26T01:35:32.142 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:32.135+0000 7efd53f4c640 1 -- 172.21.15.169:0/2856025433 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd4c070960 con 0x7efd4c06ce20 2023-10-26T01:35:32.142 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:32.136+0000 7efd53f4c640 1 -- 172.21.15.169:0/2856025433 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd4c04db10 con 0x7efd4c06ce20 2023-10-26T01:35:32.143 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:32.137+0000 7efd427fc640 1 -- 172.21.15.169:0/2856025433 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7efd480035f0 con 0x7efd4c06ce20 2023-10-26T01:35:32.143 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:32.138+0000 7efd427fc640 1 --2- 172.21.15.169:0/2856025433 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7efd3006ae40 0x7efd3006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:32.143 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:32.138+0000 7efd427fc640 1 -- 172.21.15.169:0/2856025433 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efd48014070 con 0x7efd4c06ce20 2023-10-26T01:35:32.144 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:32.139+0000 7efd514e7640 1 --2- 172.21.15.169:0/2856025433 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7efd3006ae40 0x7efd3006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:32.144 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:32.140+0000 7efd514e7640 1 --2- 172.21.15.169:0/2856025433 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7efd3006ae40 0x7efd3006d300 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7efd4c1f50d0 tx=0x7efd3c009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:35:32.145 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:32.142+0000 7efd427fc640 1 -- 172.21.15.169:0/2856025433 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7efd48055360 con 0x7efd4c06ce20 2023-10-26T01:35:32.339 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:32.336+0000 7efd53f4c640 1 -- 172.21.15.169:0/2856025433 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efd4c0acdc0 con 0x7efd4c06ce20 2023-10-26T01:35:32.341 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:32.338+0000 7efd427fc640 1 -- 172.21.15.169:0/2856025433 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+16616 (secure 0 0 0) 0x7efd48058980 con 0x7efd4c06ce20 2023-10-26T01:35:32.341 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:35:32.342 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":84,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:35:28.979183+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"72","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-10-26T01:35:26.320950+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-10-26T01:35:26.855916+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:35:32.344 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:32.341+0000 7efd53f4c640 1 -- 172.21.15.169:0/2856025433 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7efd3006ae40 msgr2=0x7efd3006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:32.344 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:32.341+0000 7efd53f4c640 1 --2- 172.21.15.169:0/2856025433 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7efd3006ae40 0x7efd3006d300 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7efd4c1f50d0 tx=0x7efd3c009040 comp rx=0 tx=0).stop 2023-10-26T01:35:32.344 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:32.341+0000 7efd53f4c640 1 -- 172.21.15.169:0/2856025433 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd4c06ce20 msgr2=0x7efd4c1539c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:32.344 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:32.341+0000 7efd53f4c640 1 --2- 172.21.15.169:0/2856025433 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd4c06ce20 0x7efd4c1539c0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7efd4800e9c0 tx=0x7efd4800ee80 comp rx=0 tx=0).stop 2023-10-26T01:35:32.344 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:32.341+0000 7efd53f4c640 1 -- 172.21.15.169:0/2856025433 shutdown_connections 2023-10-26T01:35:32.345 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:32.341+0000 7efd53f4c640 1 --2- 172.21.15.169:0/2856025433 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7efd3006ae40 0x7efd3006d300 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:32.345 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:32.341+0000 7efd53f4c640 1 --2- 172.21.15.169:0/2856025433 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7efd4c159e10 0x7efd4c156a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:32.345 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:32.341+0000 7efd53f4c640 1 --2- 172.21.15.169:0/2856025433 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd4c158850 0x7efd4c145600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:32.346 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:32.341+0000 7efd53f4c640 1 --2- 172.21.15.169:0/2856025433 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd4c06ce20 0x7efd4c1539c0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:32.346 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:32.341+0000 7efd53f4c640 1 -- 172.21.15.169:0/2856025433 >> 172.21.15.169:0/2856025433 conn(0x7efd4c069930 msgr2=0x7efd4c0cd900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:35:32.346 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:32.341+0000 7efd53f4c640 1 -- 172.21.15.169:0/2856025433 shutdown_connections 2023-10-26T01:35:32.346 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:32.341+0000 7efd53f4c640 1 -- 172.21.15.169:0/2856025433 wait complete. 2023-10-26T01:35:32.364 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-26T01:35:32.364 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-26T01:35:32.579 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:32.576+0000 7f110ca56640 1 -- 172.21.15.169:0/1562642601 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11080cdc60 msgr2=0x7f110815c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:32.579 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:32.576+0000 7f110ca56640 1 --2- 172.21.15.169:0/1562642601 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11080cdc60 0x7f110815c6d0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f10f8005fd0 tx=0x7f10f802be20 comp rx=0 tx=0).stop 2023-10-26T01:35:32.580 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:32.576+0000 7f110ca56640 1 -- 172.21.15.169:0/1562642601 shutdown_connections 2023-10-26T01:35:32.580 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:32.576+0000 7f110ca56640 1 --2- 172.21.15.169:0/1562642601 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11080cdc60 0x7f110815c6d0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:32.580 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:32.576+0000 7f110ca56640 1 -- 172.21.15.169:0/1562642601 >> 172.21.15.169:0/1562642601 conn(0x7f110806a620 msgr2=0x7f110806aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:35:32.580 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:32.576+0000 7f110ca56640 1 -- 172.21.15.169:0/1562642601 shutdown_connections 2023-10-26T01:35:32.580 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:32.577+0000 7f110ca56640 1 -- 172.21.15.169:0/1562642601 wait complete. 2023-10-26T01:35:32.581 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:32.579+0000 7f110ca56640 1 Processor -- start 2023-10-26T01:35:32.581 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:32.579+0000 7f110ca56640 1 -- start start 2023-10-26T01:35:32.584 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:32.580+0000 7f110ca56640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1108069c20 0x7f11080c5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:32.584 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:32.580+0000 7f110ca56640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f110806d7b0 0x7f11080c5650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:32.584 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:32.580+0000 7f110ca56640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11080cdc60 0x7f11081f52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:32.585 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:32.580+0000 7f110ca56640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1108158480 con 0x7f11080cdc60 2023-10-26T01:35:32.585 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:32.580+0000 7f110ca56640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11080c4380 con 0x7f1108069c20 2023-10-26T01:35:32.585 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:32.580+0000 7f110ca56640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11080c4660 con 0x7f110806d7b0 2023-10-26T01:35:32.585 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:32.580+0000 7f1106d9d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11080cdc60 0x7f11081f52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:32.585 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:32.581+0000 7f1106d9d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11080cdc60 0x7f11081f52f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:57014/0 (socket says 172.21.15.169:57014) 2023-10-26T01:35:32.586 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:32.581+0000 7f1106d9d640 1 -- 172.21.15.169:0/2585230705 learned_addr learned my addr 172.21.15.169:0/2585230705 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:35:32.586 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:32.581+0000 7f110659c640 1 --2- 172.21.15.169:0/2585230705 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1108069c20 0x7f11080c5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:32.586 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:32.581+0000 7f1106d9d640 1 -- 172.21.15.169:0/2585230705 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f110806d7b0 msgr2=0x7f11080c5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:32.586 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:32.581+0000 7f10fffff640 1 --2- 172.21.15.169:0/2585230705 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f110806d7b0 0x7f11080c5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:32.587 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:32.581+0000 7f1106d9d640 1 --2- 172.21.15.169:0/2585230705 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f110806d7b0 0x7f11080c5650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:32.587 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:32.581+0000 7f1106d9d640 1 -- 172.21.15.169:0/2585230705 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1108069c20 msgr2=0x7f11080c5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:32.587 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:32.581+0000 7f1106d9d640 1 --2- 172.21.15.169:0/2585230705 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1108069c20 0x7f11080c5110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:32.587 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:32.581+0000 7f1106d9d640 1 -- 172.21.15.169:0/2585230705 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10f8009560 con 0x7f11080cdc60 2023-10-26T01:35:32.587 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:32.581+0000 7f1106d9d640 1 --2- 172.21.15.169:0/2585230705 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11080cdc60 0x7f11081f52f0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f110000c780 tx=0x7f110000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:35:32.588 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:32.581+0000 7f10feffd640 1 -- 172.21.15.169:0/2585230705 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1100004b60 con 0x7f11080cdc60 2023-10-26T01:35:32.588 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:32.581+0000 7f10feffd640 1 -- 172.21.15.169:0/2585230705 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f110000ce90 con 0x7f11080cdc60 2023-10-26T01:35:32.588 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:32.581+0000 7f110ca56640 1 -- 172.21.15.169:0/2585230705 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11080c4940 con 0x7f11080cdc60 2023-10-26T01:35:32.588 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:32.581+0000 7f10feffd640 1 -- 172.21.15.169:0/2585230705 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f110000aab0 con 0x7f11080cdc60 2023-10-26T01:35:32.588 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:32.582+0000 7f110ca56640 1 -- 172.21.15.169:0/2585230705 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f110815ae20 con 0x7f11080cdc60 2023-10-26T01:35:32.589 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:32.583+0000 7f110ca56640 1 -- 172.21.15.169:0/2585230705 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1108158630 con 0x7f11080cdc60 2023-10-26T01:35:32.590 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:32.584+0000 7f10feffd640 1 -- 172.21.15.169:0/2585230705 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f110000ac10 con 0x7f11080cdc60 2023-10-26T01:35:32.590 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:32.585+0000 7f10feffd640 1 --2- 172.21.15.169:0/2585230705 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f10e806ae40 0x7f10e806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:32.590 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:32.585+0000 7f10feffd640 1 -- 172.21.15.169:0/2585230705 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1100089a20 con 0x7f11080cdc60 2023-10-26T01:35:32.590 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:32.585+0000 7f110659c640 1 --2- 172.21.15.169:0/2585230705 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f10e806ae40 0x7f10e806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:32.591 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:32.587+0000 7f110659c640 1 --2- 172.21.15.169:0/2585230705 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f10e806ae40 0x7f10e806d300 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f10f8035000 tx=0x7f10f8004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:35:32.591 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:32.589+0000 7f10feffd640 1 -- 172.21.15.169:0/2585230705 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1108158630 con 0x7f11080cdc60 2023-10-26T01:35:32.782 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:32.780+0000 7f110ca56640 1 -- 172.21.15.169:0/2585230705 --> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f11080c47d0 con 0x7f10e806ae40 2023-10-26T01:35:32.808 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:32.805+0000 7f10feffd640 1 -- 172.21.15.169:0/2585230705 <== mgr.4112 v2:172.21.15.170:6832/1187156918 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+282622 (secure 0 0 0) 0x7f11080c47d0 con 0x7f10e806ae40 2023-10-26T01:35:32.813 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:35:32.814 INFO:teuthology.orchestra.run.smithi169.stderr:dumped all 2023-10-26T01:35:32.816 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:32.812+0000 7f110ca56640 1 -- 172.21.15.169:0/2585230705 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f10e806ae40 msgr2=0x7f10e806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:32.816 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:32.812+0000 7f110ca56640 1 --2- 172.21.15.169:0/2585230705 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f10e806ae40 0x7f10e806d300 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f10f8035000 tx=0x7f10f8004730 comp rx=0 tx=0).stop 2023-10-26T01:35:32.817 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:32.812+0000 7f110ca56640 1 -- 172.21.15.169:0/2585230705 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11080cdc60 msgr2=0x7f11081f52f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:32.817 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:32.812+0000 7f110ca56640 1 --2- 172.21.15.169:0/2585230705 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11080cdc60 0x7f11081f52f0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f110000c780 tx=0x7f110000cc40 comp rx=0 tx=0).stop 2023-10-26T01:35:32.817 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:32.812+0000 7f110ca56640 1 -- 172.21.15.169:0/2585230705 shutdown_connections 2023-10-26T01:35:32.817 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:32.812+0000 7f110ca56640 1 --2- 172.21.15.169:0/2585230705 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f10e806ae40 0x7f10e806d300 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:32.817 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:32.812+0000 7f110ca56640 1 --2- 172.21.15.169:0/2585230705 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11080cdc60 0x7f11081f52f0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:32.818 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:32.812+0000 7f110ca56640 1 --2- 172.21.15.169:0/2585230705 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f110806d7b0 0x7f11080c5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:32.818 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:32.812+0000 7f110ca56640 1 --2- 172.21.15.169:0/2585230705 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1108069c20 0x7f11080c5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:32.818 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:32.812+0000 7f110ca56640 1 -- 172.21.15.169:0/2585230705 >> 172.21.15.169:0/2585230705 conn(0x7f110806a620 msgr2=0x7f11080af9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:35:32.818 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:32.812+0000 7f110ca56640 1 -- 172.21.15.169:0/2585230705 shutdown_connections 2023-10-26T01:35:32.818 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:32.812+0000 7f110ca56640 1 -- 172.21.15.169:0/2585230705 wait complete. 2023-10-26T01:35:32.835 INFO:teuthology.orchestra.run.smithi169.stdout:{"pg_ready":true,"pg_map":{"version":228,"stamp":"2023-10-26T01:35:32.233368+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":169,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":4628,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1557871,"statfs":{"total":1600000000,"available":1595260898,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.881757"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:28.997503+0000","last_change":"2023-10-26T01:35:28.008835+0000","last_active":"2023-10-26T01:35:28.997503+0000","last_peered":"2023-10-26T01:35:28.997503+0000","last_clean":"2023-10-26T01:35:28.997503+0000","last_became_active":"2023-10-26T01:35:28.007977+0000","last_became_peered":"2023-10-26T01:35:28.007977+0000","last_unstale":"2023-10-26T01:35:28.997503+0000","last_undegraded":"2023-10-26T01:35:28.997503+0000","last_fullsized":"2023-10-26T01:35:28.997503+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:05:52.032045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.214564+0000","last_change":"2023-10-26T01:35:28.011509+0000","last_active":"2023-10-26T01:35:29.214564+0000","last_peered":"2023-10-26T01:35:29.214564+0000","last_clean":"2023-10-26T01:35:29.214564+0000","last_became_active":"2023-10-26T01:35:28.010898+0000","last_became_peered":"2023-10-26T01:35:28.010898+0000","last_unstale":"2023-10-26T01:35:29.214564+0000","last_undegraded":"2023-10-26T01:35:29.214564+0000","last_fullsized":"2023-10-26T01:35:29.214564+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:10:26.499805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:28.995305+0000","last_change":"2023-10-26T01:35:28.004530+0000","last_active":"2023-10-26T01:35:28.995305+0000","last_peered":"2023-10-26T01:35:28.995305+0000","last_clean":"2023-10-26T01:35:28.995305+0000","last_became_active":"2023-10-26T01:35:28.004099+0000","last_became_peered":"2023-10-26T01:35:28.004099+0000","last_unstale":"2023-10-26T01:35:28.995305+0000","last_undegraded":"2023-10-26T01:35:28.995305+0000","last_fullsized":"2023-10-26T01:35:28.995305+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:44:24.881425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.216331+0000","last_change":"2023-10-26T01:35:28.004622+0000","last_active":"2023-10-26T01:35:29.216331+0000","last_peered":"2023-10-26T01:35:29.216331+0000","last_clean":"2023-10-26T01:35:29.216331+0000","last_became_active":"2023-10-26T01:35:28.004090+0000","last_became_peered":"2023-10-26T01:35:28.004090+0000","last_unstale":"2023-10-26T01:35:29.216331+0000","last_undegraded":"2023-10-26T01:35:29.216331+0000","last_fullsized":"2023-10-26T01:35:29.216331+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:46:51.824429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.216205+0000","last_change":"2023-10-26T01:35:28.009726+0000","last_active":"2023-10-26T01:35:29.216205+0000","last_peered":"2023-10-26T01:35:29.216205+0000","last_clean":"2023-10-26T01:35:29.216205+0000","last_became_active":"2023-10-26T01:35:28.009274+0000","last_became_peered":"2023-10-26T01:35:28.009274+0000","last_unstale":"2023-10-26T01:35:29.216205+0000","last_undegraded":"2023-10-26T01:35:29.216205+0000","last_fullsized":"2023-10-26T01:35:29.216205+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:14:14.105927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.216080+0000","last_change":"2023-10-26T01:35:28.009483+0000","last_active":"2023-10-26T01:35:29.216080+0000","last_peered":"2023-10-26T01:35:29.216080+0000","last_clean":"2023-10-26T01:35:29.216080+0000","last_became_active":"2023-10-26T01:35:28.009021+0000","last_became_peered":"2023-10-26T01:35:28.009021+0000","last_unstale":"2023-10-26T01:35:29.216080+0000","last_undegraded":"2023-10-26T01:35:29.216080+0000","last_fullsized":"2023-10-26T01:35:29.216080+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:06:54.676358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.214239+0000","last_change":"2023-10-26T01:35:28.010589+0000","last_active":"2023-10-26T01:35:29.214239+0000","last_peered":"2023-10-26T01:35:29.214239+0000","last_clean":"2023-10-26T01:35:29.214239+0000","last_became_active":"2023-10-26T01:35:28.009970+0000","last_became_peered":"2023-10-26T01:35:28.009970+0000","last_unstale":"2023-10-26T01:35:29.214239+0000","last_undegraded":"2023-10-26T01:35:29.214239+0000","last_fullsized":"2023-10-26T01:35:29.214239+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:54:51.431715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.215619+0000","last_change":"2023-10-26T01:35:28.012565+0000","last_active":"2023-10-26T01:35:29.215619+0000","last_peered":"2023-10-26T01:35:29.215619+0000","last_clean":"2023-10-26T01:35:29.215619+0000","last_became_active":"2023-10-26T01:35:28.011885+0000","last_became_peered":"2023-10-26T01:35:28.011885+0000","last_unstale":"2023-10-26T01:35:29.215619+0000","last_undegraded":"2023-10-26T01:35:29.215619+0000","last_fullsized":"2023-10-26T01:35:29.215619+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:12:53.392033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.213687+0000","last_change":"2023-10-26T01:35:28.009370+0000","last_active":"2023-10-26T01:35:29.213687+0000","last_peered":"2023-10-26T01:35:29.213687+0000","last_clean":"2023-10-26T01:35:29.213687+0000","last_became_active":"2023-10-26T01:35:28.009002+0000","last_became_peered":"2023-10-26T01:35:28.009002+0000","last_unstale":"2023-10-26T01:35:29.213687+0000","last_undegraded":"2023-10-26T01:35:29.213687+0000","last_fullsized":"2023-10-26T01:35:29.213687+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:59:58.766446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:28.995924+0000","last_change":"2023-10-26T01:35:28.010322+0000","last_active":"2023-10-26T01:35:28.995924+0000","last_peered":"2023-10-26T01:35:28.995924+0000","last_clean":"2023-10-26T01:35:28.995924+0000","last_became_active":"2023-10-26T01:35:28.010056+0000","last_became_peered":"2023-10-26T01:35:28.010056+0000","last_unstale":"2023-10-26T01:35:28.995924+0000","last_undegraded":"2023-10-26T01:35:28.995924+0000","last_fullsized":"2023-10-26T01:35:28.995924+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:49:39.347860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:28.997955+0000","last_change":"2023-10-26T01:35:28.005984+0000","last_active":"2023-10-26T01:35:28.997955+0000","last_peered":"2023-10-26T01:35:28.997955+0000","last_clean":"2023-10-26T01:35:28.997955+0000","last_became_active":"2023-10-26T01:35:28.005409+0000","last_became_peered":"2023-10-26T01:35:28.005409+0000","last_unstale":"2023-10-26T01:35:28.997955+0000","last_undegraded":"2023-10-26T01:35:28.997955+0000","last_fullsized":"2023-10-26T01:35:28.997955+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:45:12.037351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.213966+0000","last_change":"2023-10-26T01:35:28.014813+0000","last_active":"2023-10-26T01:35:29.213966+0000","last_peered":"2023-10-26T01:35:29.213966+0000","last_clean":"2023-10-26T01:35:29.213966+0000","last_became_active":"2023-10-26T01:35:28.014182+0000","last_became_peered":"2023-10-26T01:35:28.014182+0000","last_unstale":"2023-10-26T01:35:29.213966+0000","last_undegraded":"2023-10-26T01:35:29.213966+0000","last_fullsized":"2023-10-26T01:35:29.213966+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:50:07.224712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.214308+0000","last_change":"2023-10-26T01:35:28.013781+0000","last_active":"2023-10-26T01:35:29.214308+0000","last_peered":"2023-10-26T01:35:29.214308+0000","last_clean":"2023-10-26T01:35:29.214308+0000","last_became_active":"2023-10-26T01:35:28.013069+0000","last_became_peered":"2023-10-26T01:35:28.013069+0000","last_unstale":"2023-10-26T01:35:29.214308+0000","last_undegraded":"2023-10-26T01:35:29.214308+0000","last_fullsized":"2023-10-26T01:35:29.214308+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:59:03.753748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.214011+0000","last_change":"2023-10-26T01:35:28.007382+0000","last_active":"2023-10-26T01:35:29.214011+0000","last_peered":"2023-10-26T01:35:29.214011+0000","last_clean":"2023-10-26T01:35:29.214011+0000","last_became_active":"2023-10-26T01:35:28.006605+0000","last_became_peered":"2023-10-26T01:35:28.006605+0000","last_unstale":"2023-10-26T01:35:29.214011+0000","last_undegraded":"2023-10-26T01:35:29.214011+0000","last_fullsized":"2023-10-26T01:35:29.214011+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:24:25.478994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.216357+0000","last_change":"2023-10-26T01:35:28.009743+0000","last_active":"2023-10-26T01:35:29.216357+0000","last_peered":"2023-10-26T01:35:29.216357+0000","last_clean":"2023-10-26T01:35:29.216357+0000","last_became_active":"2023-10-26T01:35:28.009410+0000","last_became_peered":"2023-10-26T01:35:28.009410+0000","last_unstale":"2023-10-26T01:35:29.216357+0000","last_undegraded":"2023-10-26T01:35:29.216357+0000","last_fullsized":"2023-10-26T01:35:29.216357+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:40:50.370204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:28.995621+0000","last_change":"2023-10-26T01:35:28.007395+0000","last_active":"2023-10-26T01:35:28.995621+0000","last_peered":"2023-10-26T01:35:28.995621+0000","last_clean":"2023-10-26T01:35:28.995621+0000","last_became_active":"2023-10-26T01:35:28.006533+0000","last_became_peered":"2023-10-26T01:35:28.006533+0000","last_unstale":"2023-10-26T01:35:28.995621+0000","last_undegraded":"2023-10-26T01:35:28.995621+0000","last_fullsized":"2023-10-26T01:35:28.995621+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:44:27.457808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.215118+0000","last_change":"2023-10-26T01:35:28.013728+0000","last_active":"2023-10-26T01:35:29.215118+0000","last_peered":"2023-10-26T01:35:29.215118+0000","last_clean":"2023-10-26T01:35:29.215118+0000","last_became_active":"2023-10-26T01:35:28.013257+0000","last_became_peered":"2023-10-26T01:35:28.013257+0000","last_unstale":"2023-10-26T01:35:29.215118+0000","last_undegraded":"2023-10-26T01:35:29.215118+0000","last_fullsized":"2023-10-26T01:35:29.215118+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:39:04.699624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.213605+0000","last_change":"2023-10-26T01:35:28.014854+0000","last_active":"2023-10-26T01:35:29.213605+0000","last_peered":"2023-10-26T01:35:29.213605+0000","last_clean":"2023-10-26T01:35:29.213605+0000","last_became_active":"2023-10-26T01:35:28.014435+0000","last_became_peered":"2023-10-26T01:35:28.014435+0000","last_unstale":"2023-10-26T01:35:29.213605+0000","last_undegraded":"2023-10-26T01:35:29.213605+0000","last_fullsized":"2023-10-26T01:35:29.213605+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:11:49.236285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.214583+0000","last_change":"2023-10-26T01:35:28.013054+0000","last_active":"2023-10-26T01:35:29.214583+0000","last_peered":"2023-10-26T01:35:29.214583+0000","last_clean":"2023-10-26T01:35:29.214583+0000","last_became_active":"2023-10-26T01:35:28.011914+0000","last_became_peered":"2023-10-26T01:35:28.011914+0000","last_unstale":"2023-10-26T01:35:29.214583+0000","last_undegraded":"2023-10-26T01:35:29.214583+0000","last_fullsized":"2023-10-26T01:35:29.214583+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:58:11.147240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.216710+0000","last_change":"2023-10-26T01:35:28.004658+0000","last_active":"2023-10-26T01:35:29.216710+0000","last_peered":"2023-10-26T01:35:29.216710+0000","last_clean":"2023-10-26T01:35:29.216710+0000","last_became_active":"2023-10-26T01:35:28.003773+0000","last_became_peered":"2023-10-26T01:35:28.003773+0000","last_unstale":"2023-10-26T01:35:29.216710+0000","last_undegraded":"2023-10-26T01:35:29.216710+0000","last_fullsized":"2023-10-26T01:35:29.216710+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:17:46.328218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:28.996098+0000","last_change":"2023-10-26T01:35:28.002723+0000","last_active":"2023-10-26T01:35:28.996098+0000","last_peered":"2023-10-26T01:35:28.996098+0000","last_clean":"2023-10-26T01:35:28.996098+0000","last_became_active":"2023-10-26T01:35:28.002231+0000","last_became_peered":"2023-10-26T01:35:28.002231+0000","last_unstale":"2023-10-26T01:35:28.996098+0000","last_undegraded":"2023-10-26T01:35:28.996098+0000","last_fullsized":"2023-10-26T01:35:28.996098+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:47:22.912447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:28.995662+0000","last_change":"2023-10-26T01:35:28.011690+0000","last_active":"2023-10-26T01:35:28.995662+0000","last_peered":"2023-10-26T01:35:28.995662+0000","last_clean":"2023-10-26T01:35:28.995662+0000","last_became_active":"2023-10-26T01:35:28.011424+0000","last_became_peered":"2023-10-26T01:35:28.011424+0000","last_unstale":"2023-10-26T01:35:28.995662+0000","last_undegraded":"2023-10-26T01:35:28.995662+0000","last_fullsized":"2023-10-26T01:35:28.995662+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:10:55.997083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:28.996782+0000","last_change":"2023-10-26T01:35:27.999889+0000","last_active":"2023-10-26T01:35:28.996782+0000","last_peered":"2023-10-26T01:35:28.996782+0000","last_clean":"2023-10-26T01:35:28.996782+0000","last_became_active":"2023-10-26T01:35:27.999521+0000","last_became_peered":"2023-10-26T01:35:27.999521+0000","last_unstale":"2023-10-26T01:35:28.996782+0000","last_undegraded":"2023-10-26T01:35:28.996782+0000","last_fullsized":"2023-10-26T01:35:28.996782+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:04:11.794309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:28.994867+0000","last_change":"2023-10-26T01:35:28.014763+0000","last_active":"2023-10-26T01:35:28.994867+0000","last_peered":"2023-10-26T01:35:28.994867+0000","last_clean":"2023-10-26T01:35:28.994867+0000","last_became_active":"2023-10-26T01:35:28.014538+0000","last_became_peered":"2023-10-26T01:35:28.014538+0000","last_unstale":"2023-10-26T01:35:28.994867+0000","last_undegraded":"2023-10-26T01:35:28.994867+0000","last_fullsized":"2023-10-26T01:35:28.994867+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:41:28.638341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:28.996239+0000","last_change":"2023-10-26T01:35:27.998285+0000","last_active":"2023-10-26T01:35:28.996239+0000","last_peered":"2023-10-26T01:35:28.996239+0000","last_clean":"2023-10-26T01:35:28.996239+0000","last_became_active":"2023-10-26T01:35:27.997577+0000","last_became_peered":"2023-10-26T01:35:27.997577+0000","last_unstale":"2023-10-26T01:35:28.996239+0000","last_undegraded":"2023-10-26T01:35:28.996239+0000","last_fullsized":"2023-10-26T01:35:28.996239+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:16:47.008792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.216023+0000","last_change":"2023-10-26T01:35:28.012518+0000","last_active":"2023-10-26T01:35:29.216023+0000","last_peered":"2023-10-26T01:35:29.216023+0000","last_clean":"2023-10-26T01:35:29.216023+0000","last_became_active":"2023-10-26T01:35:28.012254+0000","last_became_peered":"2023-10-26T01:35:28.012254+0000","last_unstale":"2023-10-26T01:35:29.216023+0000","last_undegraded":"2023-10-26T01:35:29.216023+0000","last_fullsized":"2023-10-26T01:35:29.216023+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:41:18.798894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:28.994945+0000","last_change":"2023-10-26T01:35:27.999138+0000","last_active":"2023-10-26T01:35:28.994945+0000","last_peered":"2023-10-26T01:35:28.994945+0000","last_clean":"2023-10-26T01:35:28.994945+0000","last_became_active":"2023-10-26T01:35:27.998526+0000","last_became_peered":"2023-10-26T01:35:27.998526+0000","last_unstale":"2023-10-26T01:35:28.994945+0000","last_undegraded":"2023-10-26T01:35:28.994945+0000","last_fullsized":"2023-10-26T01:35:28.994945+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:21:07.307330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:28.994315+0000","last_change":"2023-10-26T01:35:28.012297+0000","last_active":"2023-10-26T01:35:28.994315+0000","last_peered":"2023-10-26T01:35:28.994315+0000","last_clean":"2023-10-26T01:35:28.994315+0000","last_became_active":"2023-10-26T01:35:28.011776+0000","last_became_peered":"2023-10-26T01:35:28.011776+0000","last_unstale":"2023-10-26T01:35:28.994315+0000","last_undegraded":"2023-10-26T01:35:28.994315+0000","last_fullsized":"2023-10-26T01:35:28.994315+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:58:10.505901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:28.995265+0000","last_change":"2023-10-26T01:35:28.003936+0000","last_active":"2023-10-26T01:35:28.995265+0000","last_peered":"2023-10-26T01:35:28.995265+0000","last_clean":"2023-10-26T01:35:28.995265+0000","last_became_active":"2023-10-26T01:35:28.003324+0000","last_became_peered":"2023-10-26T01:35:28.003324+0000","last_unstale":"2023-10-26T01:35:28.995265+0000","last_undegraded":"2023-10-26T01:35:28.995265+0000","last_fullsized":"2023-10-26T01:35:28.995265+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:04:03.749653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.214530+0000","last_change":"2023-10-26T01:35:28.007835+0000","last_active":"2023-10-26T01:35:29.214530+0000","last_peered":"2023-10-26T01:35:29.214530+0000","last_clean":"2023-10-26T01:35:29.214530+0000","last_became_active":"2023-10-26T01:35:28.006302+0000","last_became_peered":"2023-10-26T01:35:28.006302+0000","last_unstale":"2023-10-26T01:35:29.214530+0000","last_undegraded":"2023-10-26T01:35:29.214530+0000","last_fullsized":"2023-10-26T01:35:29.214530+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:39:30.070804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.216345+0000","last_change":"2023-10-26T01:35:28.011170+0000","last_active":"2023-10-26T01:35:29.216345+0000","last_peered":"2023-10-26T01:35:29.216345+0000","last_clean":"2023-10-26T01:35:29.216345+0000","last_became_active":"2023-10-26T01:35:28.010863+0000","last_became_peered":"2023-10-26T01:35:28.010863+0000","last_unstale":"2023-10-26T01:35:29.216345+0000","last_undegraded":"2023-10-26T01:35:29.216345+0000","last_fullsized":"2023-10-26T01:35:29.216345+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:18:23.257036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.214921+0000","last_change":"2023-10-26T01:35:28.011689+0000","last_active":"2023-10-26T01:35:29.214921+0000","last_peered":"2023-10-26T01:35:29.214921+0000","last_clean":"2023-10-26T01:35:29.214921+0000","last_became_active":"2023-10-26T01:35:28.011266+0000","last_became_peered":"2023-10-26T01:35:28.011266+0000","last_unstale":"2023-10-26T01:35:29.214921+0000","last_undegraded":"2023-10-26T01:35:29.214921+0000","last_fullsized":"2023-10-26T01:35:29.214921+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:52:35.198913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:28.995195+0000","last_change":"2023-10-26T01:35:28.014275+0000","last_active":"2023-10-26T01:35:28.995195+0000","last_peered":"2023-10-26T01:35:28.995195+0000","last_clean":"2023-10-26T01:35:28.995195+0000","last_became_active":"2023-10-26T01:35:28.014053+0000","last_became_peered":"2023-10-26T01:35:28.014053+0000","last_unstale":"2023-10-26T01:35:28.995195+0000","last_undegraded":"2023-10-26T01:35:28.995195+0000","last_fullsized":"2023-10-26T01:35:28.995195+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:50:40.176544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"83'4","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:28.997021+0000","last_change":"2023-10-26T01:35:27.378062+0000","last_active":"2023-10-26T01:35:28.997021+0000","last_peered":"2023-10-26T01:35:28.997021+0000","last_clean":"2023-10-26T01:35:28.997021+0000","last_became_active":"2023-10-26T01:35:27.377281+0000","last_became_peered":"2023-10-26T01:35:27.377281+0000","last_unstale":"2023-10-26T01:35:28.997021+0000","last_undegraded":"2023-10-26T01:35:28.997021+0000","last_fullsized":"2023-10-26T01:35:28.997021+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_clean_scrub_stamp":"2023-10-26T01:35:26.337183+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:26:19.488470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.214119+0000","last_change":"2023-10-26T01:35:28.008559+0000","last_active":"2023-10-26T01:35:29.214119+0000","last_peered":"2023-10-26T01:35:29.214119+0000","last_clean":"2023-10-26T01:35:29.214119+0000","last_became_active":"2023-10-26T01:35:28.007917+0000","last_became_peered":"2023-10-26T01:35:28.007917+0000","last_unstale":"2023-10-26T01:35:29.214119+0000","last_undegraded":"2023-10-26T01:35:29.214119+0000","last_fullsized":"2023-10-26T01:35:29.214119+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:05:34.133886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"83'1","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.216930+0000","last_change":"2023-10-26T01:35:27.378724+0000","last_active":"2023-10-26T01:35:29.216930+0000","last_peered":"2023-10-26T01:35:29.216930+0000","last_clean":"2023-10-26T01:35:29.216930+0000","last_became_active":"2023-10-26T01:35:27.378341+0000","last_became_peered":"2023-10-26T01:35:27.378341+0000","last_unstale":"2023-10-26T01:35:29.216930+0000","last_undegraded":"2023-10-26T01:35:29.216930+0000","last_fullsized":"2023-10-26T01:35:29.216930+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_clean_scrub_stamp":"2023-10-26T01:35:26.337183+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:57:29.050189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.216392+0000","last_change":"2023-10-26T01:35:28.012857+0000","last_active":"2023-10-26T01:35:29.216392+0000","last_peered":"2023-10-26T01:35:29.216392+0000","last_clean":"2023-10-26T01:35:29.216392+0000","last_became_active":"2023-10-26T01:35:28.012160+0000","last_became_peered":"2023-10-26T01:35:28.012160+0000","last_unstale":"2023-10-26T01:35:29.216392+0000","last_undegraded":"2023-10-26T01:35:29.216392+0000","last_fullsized":"2023-10-26T01:35:29.216392+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:58:10.505901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"83'2","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:28.996748+0000","last_change":"2023-10-26T01:35:27.376547+0000","last_active":"2023-10-26T01:35:28.996748+0000","last_peered":"2023-10-26T01:35:28.996748+0000","last_clean":"2023-10-26T01:35:28.996748+0000","last_became_active":"2023-10-26T01:35:27.375661+0000","last_became_peered":"2023-10-26T01:35:27.375661+0000","last_unstale":"2023-10-26T01:35:28.996748+0000","last_undegraded":"2023-10-26T01:35:28.996748+0000","last_fullsized":"2023-10-26T01:35:28.996748+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_clean_scrub_stamp":"2023-10-26T01:35:26.337183+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:12:52.379303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:28.996769+0000","last_change":"2023-10-26T01:35:27.999847+0000","last_active":"2023-10-26T01:35:28.996769+0000","last_peered":"2023-10-26T01:35:28.996769+0000","last_clean":"2023-10-26T01:35:28.996769+0000","last_became_active":"2023-10-26T01:35:27.999354+0000","last_became_peered":"2023-10-26T01:35:27.999354+0000","last_unstale":"2023-10-26T01:35:28.996769+0000","last_undegraded":"2023-10-26T01:35:28.996769+0000","last_fullsized":"2023-10-26T01:35:28.996769+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:57:47.827646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"83'2","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.216832+0000","last_change":"2023-10-26T01:35:27.388698+0000","last_active":"2023-10-26T01:35:29.216832+0000","last_peered":"2023-10-26T01:35:29.216832+0000","last_clean":"2023-10-26T01:35:29.216832+0000","last_became_active":"2023-10-26T01:35:27.387595+0000","last_became_peered":"2023-10-26T01:35:27.387595+0000","last_unstale":"2023-10-26T01:35:29.216832+0000","last_undegraded":"2023-10-26T01:35:29.216832+0000","last_fullsized":"2023-10-26T01:35:29.216832+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_clean_scrub_stamp":"2023-10-26T01:35:26.337183+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:16:57.595672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.214172+0000","last_change":"2023-10-26T01:35:28.009258+0000","last_active":"2023-10-26T01:35:29.214172+0000","last_peered":"2023-10-26T01:35:29.214172+0000","last_clean":"2023-10-26T01:35:29.214172+0000","last_became_active":"2023-10-26T01:35:28.008895+0000","last_became_peered":"2023-10-26T01:35:28.008895+0000","last_unstale":"2023-10-26T01:35:29.214172+0000","last_undegraded":"2023-10-26T01:35:29.214172+0000","last_fullsized":"2023-10-26T01:35:29.214172+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:03:13.152836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"83'1","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T01:35:28.407749+0000","last_change":"2023-10-26T01:35:27.384835+0000","last_active":"2023-10-26T01:35:28.407749+0000","last_peered":"2023-10-26T01:35:28.407749+0000","last_clean":"2023-10-26T01:35:28.407749+0000","last_became_active":"2023-10-26T01:35:27.384560+0000","last_became_peered":"2023-10-26T01:35:27.384560+0000","last_unstale":"2023-10-26T01:35:28.407749+0000","last_undegraded":"2023-10-26T01:35:28.407749+0000","last_fullsized":"2023-10-26T01:35:28.407749+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_clean_scrub_stamp":"2023-10-26T01:35:26.337183+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:46:20.561796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:28.996010+0000","last_change":"2023-10-26T01:35:28.000055+0000","last_active":"2023-10-26T01:35:28.996010+0000","last_peered":"2023-10-26T01:35:28.996010+0000","last_clean":"2023-10-26T01:35:28.996010+0000","last_became_active":"2023-10-26T01:35:27.996858+0000","last_became_peered":"2023-10-26T01:35:27.996858+0000","last_unstale":"2023-10-26T01:35:28.996010+0000","last_undegraded":"2023-10-26T01:35:28.996010+0000","last_fullsized":"2023-10-26T01:35:28.996010+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:41:39.097519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"83'1","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.216164+0000","last_change":"2023-10-26T01:35:27.386727+0000","last_active":"2023-10-26T01:35:29.216164+0000","last_peered":"2023-10-26T01:35:29.216164+0000","last_clean":"2023-10-26T01:35:29.216164+0000","last_became_active":"2023-10-26T01:35:27.386318+0000","last_became_peered":"2023-10-26T01:35:27.386318+0000","last_unstale":"2023-10-26T01:35:29.216164+0000","last_undegraded":"2023-10-26T01:35:29.216164+0000","last_fullsized":"2023-10-26T01:35:29.216164+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_clean_scrub_stamp":"2023-10-26T01:35:26.337183+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:30:09.768728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.216708+0000","last_change":"2023-10-26T01:35:28.006993+0000","last_active":"2023-10-26T01:35:29.216708+0000","last_peered":"2023-10-26T01:35:29.216708+0000","last_clean":"2023-10-26T01:35:29.216708+0000","last_became_active":"2023-10-26T01:35:28.006476+0000","last_became_peered":"2023-10-26T01:35:28.006476+0000","last_unstale":"2023-10-26T01:35:29.216708+0000","last_undegraded":"2023-10-26T01:35:29.216708+0000","last_fullsized":"2023-10-26T01:35:29.216708+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:15:43.068280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.216889+0000","last_change":"2023-10-26T01:35:27.376474+0000","last_active":"2023-10-26T01:35:29.216889+0000","last_peered":"2023-10-26T01:35:29.216889+0000","last_clean":"2023-10-26T01:35:29.216889+0000","last_became_active":"2023-10-26T01:35:27.376103+0000","last_became_peered":"2023-10-26T01:35:27.376103+0000","last_unstale":"2023-10-26T01:35:29.216889+0000","last_undegraded":"2023-10-26T01:35:29.216889+0000","last_fullsized":"2023-10-26T01:35:29.216889+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_clean_scrub_stamp":"2023-10-26T01:35:26.337183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:32:37.976894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.216684+0000","last_change":"2023-10-26T01:35:27.999618+0000","last_active":"2023-10-26T01:35:29.216684+0000","last_peered":"2023-10-26T01:35:29.216684+0000","last_clean":"2023-10-26T01:35:29.216684+0000","last_became_active":"2023-10-26T01:35:27.996868+0000","last_became_peered":"2023-10-26T01:35:27.996868+0000","last_unstale":"2023-10-26T01:35:29.216684+0000","last_undegraded":"2023-10-26T01:35:29.216684+0000","last_fullsized":"2023-10-26T01:35:29.216684+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:04:23.200706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.217061+0000","last_change":"2023-10-26T01:35:27.377997+0000","last_active":"2023-10-26T01:35:29.217061+0000","last_peered":"2023-10-26T01:35:29.217061+0000","last_clean":"2023-10-26T01:35:29.217061+0000","last_became_active":"2023-10-26T01:35:27.376180+0000","last_became_peered":"2023-10-26T01:35:27.376180+0000","last_unstale":"2023-10-26T01:35:29.217061+0000","last_undegraded":"2023-10-26T01:35:29.217061+0000","last_fullsized":"2023-10-26T01:35:29.217061+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_clean_scrub_stamp":"2023-10-26T01:35:26.337183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:10:28.925306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.214645+0000","last_change":"2023-10-26T01:35:28.005939+0000","last_active":"2023-10-26T01:35:29.214645+0000","last_peered":"2023-10-26T01:35:29.214645+0000","last_clean":"2023-10-26T01:35:29.214645+0000","last_became_active":"2023-10-26T01:35:28.005566+0000","last_became_peered":"2023-10-26T01:35:28.005566+0000","last_unstale":"2023-10-26T01:35:29.214645+0000","last_undegraded":"2023-10-26T01:35:29.214645+0000","last_fullsized":"2023-10-26T01:35:29.214645+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T01:49:24.955549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:28.996318+0000","last_change":"2023-10-26T01:35:27.382676+0000","last_active":"2023-10-26T01:35:28.996318+0000","last_peered":"2023-10-26T01:35:28.996318+0000","last_clean":"2023-10-26T01:35:28.996318+0000","last_became_active":"2023-10-26T01:35:27.378175+0000","last_became_peered":"2023-10-26T01:35:27.378175+0000","last_unstale":"2023-10-26T01:35:28.996318+0000","last_undegraded":"2023-10-26T01:35:28.996318+0000","last_fullsized":"2023-10-26T01:35:28.996318+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_clean_scrub_stamp":"2023-10-26T01:35:26.337183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:06:53.663628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:28.998119+0000","last_change":"2023-10-26T01:35:28.015414+0000","last_active":"2023-10-26T01:35:28.998119+0000","last_peered":"2023-10-26T01:35:28.998119+0000","last_clean":"2023-10-26T01:35:28.998119+0000","last_became_active":"2023-10-26T01:35:28.014670+0000","last_became_peered":"2023-10-26T01:35:28.014670+0000","last_unstale":"2023-10-26T01:35:28.998119+0000","last_undegraded":"2023-10-26T01:35:28.998119+0000","last_fullsized":"2023-10-26T01:35:28.998119+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:59:43.451135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.215162+0000","last_change":"2023-10-26T01:35:27.386158+0000","last_active":"2023-10-26T01:35:29.215162+0000","last_peered":"2023-10-26T01:35:29.215162+0000","last_clean":"2023-10-26T01:35:29.215162+0000","last_became_active":"2023-10-26T01:35:27.385865+0000","last_became_peered":"2023-10-26T01:35:27.385865+0000","last_unstale":"2023-10-26T01:35:29.215162+0000","last_undegraded":"2023-10-26T01:35:29.215162+0000","last_fullsized":"2023-10-26T01:35:29.215162+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_clean_scrub_stamp":"2023-10-26T01:35:26.337183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:11:48.223555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.214799+0000","last_change":"2023-10-26T01:35:28.012681+0000","last_active":"2023-10-26T01:35:29.214799+0000","last_peered":"2023-10-26T01:35:29.214799+0000","last_clean":"2023-10-26T01:35:29.214799+0000","last_became_active":"2023-10-26T01:35:28.010972+0000","last_became_peered":"2023-10-26T01:35:28.010972+0000","last_unstale":"2023-10-26T01:35:29.214799+0000","last_undegraded":"2023-10-26T01:35:29.214799+0000","last_fullsized":"2023-10-26T01:35:29.214799+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:03:19.603679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"83'3","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T01:35:28.412662+0000","last_change":"2023-10-26T01:35:27.384617+0000","last_active":"2023-10-26T01:35:28.412662+0000","last_peered":"2023-10-26T01:35:28.412662+0000","last_clean":"2023-10-26T01:35:28.412662+0000","last_became_active":"2023-10-26T01:35:27.384299+0000","last_became_peered":"2023-10-26T01:35:27.384299+0000","last_unstale":"2023-10-26T01:35:28.412662+0000","last_undegraded":"2023-10-26T01:35:28.412662+0000","last_fullsized":"2023-10-26T01:35:28.412662+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_clean_scrub_stamp":"2023-10-26T01:35:26.337183+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:52:21.850204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.214446+0000","last_change":"2023-10-26T01:35:28.008261+0000","last_active":"2023-10-26T01:35:29.214446+0000","last_peered":"2023-10-26T01:35:29.214446+0000","last_clean":"2023-10-26T01:35:29.214446+0000","last_became_active":"2023-10-26T01:35:28.007031+0000","last_became_peered":"2023-10-26T01:35:28.007031+0000","last_unstale":"2023-10-26T01:35:29.214446+0000","last_undegraded":"2023-10-26T01:35:29.214446+0000","last_fullsized":"2023-10-26T01:35:29.214446+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:08:41.001574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"83'6","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.217269+0000","last_change":"2023-10-26T01:35:27.374334+0000","last_active":"2023-10-26T01:35:29.217269+0000","last_peered":"2023-10-26T01:35:29.217269+0000","last_clean":"2023-10-26T01:35:29.217269+0000","last_became_active":"2023-10-26T01:35:27.372877+0000","last_became_peered":"2023-10-26T01:35:27.372877+0000","last_unstale":"2023-10-26T01:35:29.217269+0000","last_undegraded":"2023-10-26T01:35:29.217269+0000","last_fullsized":"2023-10-26T01:35:29.217269+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_clean_scrub_stamp":"2023-10-26T01:35:26.337183+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:02:33.998135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:28.998277+0000","last_change":"2023-10-26T01:35:28.009106+0000","last_active":"2023-10-26T01:35:28.998277+0000","last_peered":"2023-10-26T01:35:28.998277+0000","last_clean":"2023-10-26T01:35:28.998277+0000","last_became_active":"2023-10-26T01:35:28.008430+0000","last_became_peered":"2023-10-26T01:35:28.008430+0000","last_unstale":"2023-10-26T01:35:28.998277+0000","last_undegraded":"2023-10-26T01:35:28.998277+0000","last_fullsized":"2023-10-26T01:35:28.998277+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:53:25.094797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"83'2","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.214403+0000","last_change":"2023-10-26T01:35:27.378502+0000","last_active":"2023-10-26T01:35:29.214403+0000","last_peered":"2023-10-26T01:35:29.214403+0000","last_clean":"2023-10-26T01:35:29.214403+0000","last_became_active":"2023-10-26T01:35:27.378247+0000","last_became_peered":"2023-10-26T01:35:27.378247+0000","last_unstale":"2023-10-26T01:35:29.214403+0000","last_undegraded":"2023-10-26T01:35:29.214403+0000","last_fullsized":"2023-10-26T01:35:29.214403+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_clean_scrub_stamp":"2023-10-26T01:35:26.337183+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T01:51:53.290088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.214851+0000","last_change":"2023-10-26T01:35:28.008044+0000","last_active":"2023-10-26T01:35:29.214851+0000","last_peered":"2023-10-26T01:35:29.214851+0000","last_clean":"2023-10-26T01:35:29.214851+0000","last_became_active":"2023-10-26T01:35:28.007421+0000","last_became_peered":"2023-10-26T01:35:28.007421+0000","last_unstale":"2023-10-26T01:35:29.214851+0000","last_undegraded":"2023-10-26T01:35:29.214851+0000","last_fullsized":"2023-10-26T01:35:29.214851+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:17:13.966369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.216938+0000","last_change":"2023-10-26T01:35:27.388474+0000","last_active":"2023-10-26T01:35:29.216938+0000","last_peered":"2023-10-26T01:35:29.216938+0000","last_clean":"2023-10-26T01:35:29.216938+0000","last_became_active":"2023-10-26T01:35:27.387040+0000","last_became_peered":"2023-10-26T01:35:27.387040+0000","last_unstale":"2023-10-26T01:35:29.216938+0000","last_undegraded":"2023-10-26T01:35:29.216938+0000","last_fullsized":"2023-10-26T01:35:29.216938+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_clean_scrub_stamp":"2023-10-26T01:35:26.337183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:47:51.280157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.214625+0000","last_change":"2023-10-26T01:35:28.014358+0000","last_active":"2023-10-26T01:35:29.214625+0000","last_peered":"2023-10-26T01:35:29.214625+0000","last_clean":"2023-10-26T01:35:29.214625+0000","last_became_active":"2023-10-26T01:35:28.014074+0000","last_became_peered":"2023-10-26T01:35:28.014074+0000","last_unstale":"2023-10-26T01:35:29.214625+0000","last_undegraded":"2023-10-26T01:35:29.214625+0000","last_fullsized":"2023-10-26T01:35:29.214625+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:05:37.048290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.215191+0000","last_change":"2023-10-26T01:35:27.369340+0000","last_active":"2023-10-26T01:35:29.215191+0000","last_peered":"2023-10-26T01:35:29.215191+0000","last_clean":"2023-10-26T01:35:29.215191+0000","last_became_active":"2023-10-26T01:35:27.368595+0000","last_became_peered":"2023-10-26T01:35:27.368595+0000","last_unstale":"2023-10-26T01:35:29.215191+0000","last_undegraded":"2023-10-26T01:35:29.215191+0000","last_fullsized":"2023-10-26T01:35:29.215191+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_clean_scrub_stamp":"2023-10-26T01:35:26.337183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:27:04.115099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.215423+0000","last_change":"2023-10-26T01:35:28.012065+0000","last_active":"2023-10-26T01:35:29.215423+0000","last_peered":"2023-10-26T01:35:29.215423+0000","last_clean":"2023-10-26T01:35:29.215423+0000","last_became_active":"2023-10-26T01:35:28.011702+0000","last_became_peered":"2023-10-26T01:35:28.011702+0000","last_unstale":"2023-10-26T01:35:29.215423+0000","last_undegraded":"2023-10-26T01:35:29.215423+0000","last_fullsized":"2023-10-26T01:35:29.215423+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:16:18.626077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"83'8","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:28.994596+0000","last_change":"2023-10-26T01:35:27.382105+0000","last_active":"2023-10-26T01:35:28.994596+0000","last_peered":"2023-10-26T01:35:28.994596+0000","last_clean":"2023-10-26T01:35:28.994596+0000","last_became_active":"2023-10-26T01:35:27.380841+0000","last_became_peered":"2023-10-26T01:35:27.380841+0000","last_unstale":"2023-10-26T01:35:28.994596+0000","last_undegraded":"2023-10-26T01:35:28.994596+0000","last_fullsized":"2023-10-26T01:35:28.994596+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_clean_scrub_stamp":"2023-10-26T01:35:26.337183+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:57:48.871856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:28.996552+0000","last_change":"2023-10-26T01:35:28.004115+0000","last_active":"2023-10-26T01:35:28.996552+0000","last_peered":"2023-10-26T01:35:28.996552+0000","last_clean":"2023-10-26T01:35:28.996552+0000","last_became_active":"2023-10-26T01:35:28.003137+0000","last_became_peered":"2023-10-26T01:35:28.003137+0000","last_unstale":"2023-10-26T01:35:28.996552+0000","last_undegraded":"2023-10-26T01:35:28.996552+0000","last_fullsized":"2023-10-26T01:35:28.996552+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:23:00.412684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"83'1","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.215017+0000","last_change":"2023-10-26T01:35:27.386064+0000","last_active":"2023-10-26T01:35:29.215017+0000","last_peered":"2023-10-26T01:35:29.215017+0000","last_clean":"2023-10-26T01:35:29.215017+0000","last_became_active":"2023-10-26T01:35:27.385758+0000","last_became_peered":"2023-10-26T01:35:27.385758+0000","last_unstale":"2023-10-26T01:35:29.215017+0000","last_undegraded":"2023-10-26T01:35:29.215017+0000","last_fullsized":"2023-10-26T01:35:29.215017+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_clean_scrub_stamp":"2023-10-26T01:35:26.337183+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:50:06.211982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:28.996804+0000","last_change":"2023-10-26T01:35:27.997556+0000","last_active":"2023-10-26T01:35:28.996804+0000","last_peered":"2023-10-26T01:35:28.996804+0000","last_clean":"2023-10-26T01:35:28.996804+0000","last_became_active":"2023-10-26T01:35:27.996932+0000","last_became_peered":"2023-10-26T01:35:27.996932+0000","last_unstale":"2023-10-26T01:35:28.996804+0000","last_undegraded":"2023-10-26T01:35:28.996804+0000","last_fullsized":"2023-10-26T01:35:28.996804+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T01:59:40.284419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"83'2","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:28.996943+0000","last_change":"2023-10-26T01:35:27.376227+0000","last_active":"2023-10-26T01:35:28.996943+0000","last_peered":"2023-10-26T01:35:28.996943+0000","last_clean":"2023-10-26T01:35:28.996943+0000","last_became_active":"2023-10-26T01:35:27.375091+0000","last_became_peered":"2023-10-26T01:35:27.375091+0000","last_unstale":"2023-10-26T01:35:28.996943+0000","last_undegraded":"2023-10-26T01:35:28.996943+0000","last_fullsized":"2023-10-26T01:35:28.996943+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_clean_scrub_stamp":"2023-10-26T01:35:26.337183+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:39:51.002944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.214088+0000","last_change":"2023-10-26T01:35:28.015000+0000","last_active":"2023-10-26T01:35:29.214088+0000","last_peered":"2023-10-26T01:35:29.214088+0000","last_clean":"2023-10-26T01:35:29.214088+0000","last_became_active":"2023-10-26T01:35:28.014563+0000","last_became_peered":"2023-10-26T01:35:28.014563+0000","last_unstale":"2023-10-26T01:35:29.214088+0000","last_undegraded":"2023-10-26T01:35:29.214088+0000","last_fullsized":"2023-10-26T01:35:29.214088+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:29:03.673854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"83'1","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.217147+0000","last_change":"2023-10-26T01:35:27.388852+0000","last_active":"2023-10-26T01:35:29.217147+0000","last_peered":"2023-10-26T01:35:29.217147+0000","last_clean":"2023-10-26T01:35:29.217147+0000","last_became_active":"2023-10-26T01:35:27.388231+0000","last_became_peered":"2023-10-26T01:35:27.388231+0000","last_unstale":"2023-10-26T01:35:29.217147+0000","last_undegraded":"2023-10-26T01:35:29.217147+0000","last_fullsized":"2023-10-26T01:35:29.217147+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_clean_scrub_stamp":"2023-10-26T01:35:26.337183+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:21:38.139916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.215707+0000","last_change":"2023-10-26T01:35:28.006049+0000","last_active":"2023-10-26T01:35:29.215707+0000","last_peered":"2023-10-26T01:35:29.215707+0000","last_clean":"2023-10-26T01:35:29.215707+0000","last_became_active":"2023-10-26T01:35:28.005583+0000","last_became_peered":"2023-10-26T01:35:28.005583+0000","last_unstale":"2023-10-26T01:35:29.215707+0000","last_undegraded":"2023-10-26T01:35:29.215707+0000","last_fullsized":"2023-10-26T01:35:29.215707+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:17:45.838484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:28.996675+0000","last_change":"2023-10-26T01:35:27.384945+0000","last_active":"2023-10-26T01:35:28.996675+0000","last_peered":"2023-10-26T01:35:28.996675+0000","last_clean":"2023-10-26T01:35:28.996675+0000","last_became_active":"2023-10-26T01:35:27.384581+0000","last_became_peered":"2023-10-26T01:35:27.384581+0000","last_unstale":"2023-10-26T01:35:28.996675+0000","last_undegraded":"2023-10-26T01:35:28.996675+0000","last_fullsized":"2023-10-26T01:35:28.996675+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_clean_scrub_stamp":"2023-10-26T01:35:26.337183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:54:50.418985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:28.995605+0000","last_change":"2023-10-26T01:35:27.997899+0000","last_active":"2023-10-26T01:35:28.995605+0000","last_peered":"2023-10-26T01:35:28.995605+0000","last_clean":"2023-10-26T01:35:28.995605+0000","last_became_active":"2023-10-26T01:35:27.997225+0000","last_became_peered":"2023-10-26T01:35:27.997225+0000","last_unstale":"2023-10-26T01:35:28.995605+0000","last_undegraded":"2023-10-26T01:35:28.995605+0000","last_fullsized":"2023-10-26T01:35:28.995605+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:28:19.850875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.215140+0000","last_change":"2023-10-26T01:35:27.385581+0000","last_active":"2023-10-26T01:35:29.215140+0000","last_peered":"2023-10-26T01:35:29.215140+0000","last_clean":"2023-10-26T01:35:29.215140+0000","last_became_active":"2023-10-26T01:35:27.382991+0000","last_became_peered":"2023-10-26T01:35:27.382991+0000","last_unstale":"2023-10-26T01:35:29.215140+0000","last_undegraded":"2023-10-26T01:35:29.215140+0000","last_fullsized":"2023-10-26T01:35:29.215140+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_clean_scrub_stamp":"2023-10-26T01:35:26.337183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:29:02.661124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:28.995222+0000","last_change":"2023-10-26T01:35:28.014715+0000","last_active":"2023-10-26T01:35:28.995222+0000","last_peered":"2023-10-26T01:35:28.995222+0000","last_clean":"2023-10-26T01:35:28.995222+0000","last_became_active":"2023-10-26T01:35:28.014454+0000","last_became_peered":"2023-10-26T01:35:28.014454+0000","last_unstale":"2023-10-26T01:35:28.995222+0000","last_undegraded":"2023-10-26T01:35:28.995222+0000","last_fullsized":"2023-10-26T01:35:28.995222+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:49:11.256897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.215076+0000","last_change":"2023-10-26T01:35:27.384315+0000","last_active":"2023-10-26T01:35:29.215076+0000","last_peered":"2023-10-26T01:35:29.215076+0000","last_clean":"2023-10-26T01:35:29.215076+0000","last_became_active":"2023-10-26T01:35:27.383731+0000","last_became_peered":"2023-10-26T01:35:27.383731+0000","last_unstale":"2023-10-26T01:35:29.215076+0000","last_undegraded":"2023-10-26T01:35:29.215076+0000","last_fullsized":"2023-10-26T01:35:29.215076+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_clean_scrub_stamp":"2023-10-26T01:35:26.337183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:01:54.686020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:28.995027+0000","last_change":"2023-10-26T01:35:28.009584+0000","last_active":"2023-10-26T01:35:28.995027+0000","last_peered":"2023-10-26T01:35:28.995027+0000","last_clean":"2023-10-26T01:35:28.995027+0000","last_became_active":"2023-10-26T01:35:28.008557+0000","last_became_peered":"2023-10-26T01:35:28.008557+0000","last_unstale":"2023-10-26T01:35:28.995027+0000","last_undegraded":"2023-10-26T01:35:28.995027+0000","last_fullsized":"2023-10-26T01:35:28.995027+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:14:00.199313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.215763+0000","last_change":"2023-10-26T01:35:27.377085+0000","last_active":"2023-10-26T01:35:29.215763+0000","last_peered":"2023-10-26T01:35:29.215763+0000","last_clean":"2023-10-26T01:35:29.215763+0000","last_became_active":"2023-10-26T01:35:27.376815+0000","last_became_peered":"2023-10-26T01:35:27.376815+0000","last_unstale":"2023-10-26T01:35:29.215763+0000","last_undegraded":"2023-10-26T01:35:29.215763+0000","last_fullsized":"2023-10-26T01:35:29.215763+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_clean_scrub_stamp":"2023-10-26T01:35:26.337183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:44:59.108027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.214974+0000","last_change":"2023-10-26T01:35:28.011211+0000","last_active":"2023-10-26T01:35:29.214974+0000","last_peered":"2023-10-26T01:35:29.214974+0000","last_clean":"2023-10-26T01:35:29.214974+0000","last_became_active":"2023-10-26T01:35:28.010899+0000","last_became_peered":"2023-10-26T01:35:28.010899+0000","last_unstale":"2023-10-26T01:35:29.214974+0000","last_undegraded":"2023-10-26T01:35:29.214974+0000","last_fullsized":"2023-10-26T01:35:29.214974+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:11:54.879258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"83'2","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.215414+0000","last_change":"2023-10-26T01:35:27.382652+0000","last_active":"2023-10-26T01:35:29.215414+0000","last_peered":"2023-10-26T01:35:29.215414+0000","last_clean":"2023-10-26T01:35:29.215414+0000","last_became_active":"2023-10-26T01:35:27.382362+0000","last_became_peered":"2023-10-26T01:35:27.382362+0000","last_unstale":"2023-10-26T01:35:29.215414+0000","last_undegraded":"2023-10-26T01:35:29.215414+0000","last_fullsized":"2023-10-26T01:35:29.215414+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_clean_scrub_stamp":"2023-10-26T01:35:26.337183+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:57:48.871856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.214979+0000","last_change":"2023-10-26T01:35:28.010351+0000","last_active":"2023-10-26T01:35:29.214979+0000","last_peered":"2023-10-26T01:35:29.214979+0000","last_clean":"2023-10-26T01:35:29.214979+0000","last_became_active":"2023-10-26T01:35:28.009508+0000","last_became_peered":"2023-10-26T01:35:28.009508+0000","last_unstale":"2023-10-26T01:35:29.214979+0000","last_undegraded":"2023-10-26T01:35:29.214979+0000","last_fullsized":"2023-10-26T01:35:29.214979+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:34:13.502328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.214585+0000","last_change":"2023-10-26T01:35:27.382069+0000","last_active":"2023-10-26T01:35:29.214585+0000","last_peered":"2023-10-26T01:35:29.214585+0000","last_clean":"2023-10-26T01:35:29.214585+0000","last_became_active":"2023-10-26T01:35:27.371481+0000","last_became_peered":"2023-10-26T01:35:27.371481+0000","last_unstale":"2023-10-26T01:35:29.214585+0000","last_undegraded":"2023-10-26T01:35:29.214585+0000","last_fullsized":"2023-10-26T01:35:29.214585+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_clean_scrub_stamp":"2023-10-26T01:35:26.337183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:25:32.799627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:28.997111+0000","last_change":"2023-10-26T01:35:28.005065+0000","last_active":"2023-10-26T01:35:28.997111+0000","last_peered":"2023-10-26T01:35:28.997111+0000","last_clean":"2023-10-26T01:35:28.997111+0000","last_became_active":"2023-10-26T01:35:28.004636+0000","last_became_peered":"2023-10-26T01:35:28.004636+0000","last_unstale":"2023-10-26T01:35:28.997111+0000","last_undegraded":"2023-10-26T01:35:28.997111+0000","last_fullsized":"2023-10-26T01:35:28.997111+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:33:16.474796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.214213+0000","last_change":"2023-10-26T01:35:27.380061+0000","last_active":"2023-10-26T01:35:29.214213+0000","last_peered":"2023-10-26T01:35:29.214213+0000","last_clean":"2023-10-26T01:35:29.214213+0000","last_became_active":"2023-10-26T01:35:27.379816+0000","last_became_peered":"2023-10-26T01:35:27.379816+0000","last_unstale":"2023-10-26T01:35:29.214213+0000","last_undegraded":"2023-10-26T01:35:29.214213+0000","last_fullsized":"2023-10-26T01:35:29.214213+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_clean_scrub_stamp":"2023-10-26T01:35:26.337183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:19:36.012636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:28.997157+0000","last_change":"2023-10-26T01:35:27.995695+0000","last_active":"2023-10-26T01:35:28.997157+0000","last_peered":"2023-10-26T01:35:28.997157+0000","last_clean":"2023-10-26T01:35:28.997157+0000","last_became_active":"2023-10-26T01:35:27.995082+0000","last_became_peered":"2023-10-26T01:35:27.995082+0000","last_unstale":"2023-10-26T01:35:28.997157+0000","last_undegraded":"2023-10-26T01:35:28.997157+0000","last_fullsized":"2023-10-26T01:35:28.997157+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:18:23.792256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.216952+0000","last_change":"2023-10-26T01:35:27.374935+0000","last_active":"2023-10-26T01:35:29.216952+0000","last_peered":"2023-10-26T01:35:29.216952+0000","last_clean":"2023-10-26T01:35:29.216952+0000","last_became_active":"2023-10-26T01:35:27.374477+0000","last_became_peered":"2023-10-26T01:35:27.374477+0000","last_unstale":"2023-10-26T01:35:29.216952+0000","last_undegraded":"2023-10-26T01:35:29.216952+0000","last_fullsized":"2023-10-26T01:35:29.216952+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_clean_scrub_stamp":"2023-10-26T01:35:26.337183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:30:10.457948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.215771+0000","last_change":"2023-10-26T01:35:28.010475+0000","last_active":"2023-10-26T01:35:29.215771+0000","last_peered":"2023-10-26T01:35:29.215771+0000","last_clean":"2023-10-26T01:35:29.215771+0000","last_became_active":"2023-10-26T01:35:28.010174+0000","last_became_peered":"2023-10-26T01:35:28.010174+0000","last_unstale":"2023-10-26T01:35:29.215771+0000","last_undegraded":"2023-10-26T01:35:29.215771+0000","last_fullsized":"2023-10-26T01:35:29.215771+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T01:41:09.390504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.215935+0000","last_change":"2023-10-26T01:35:27.380771+0000","last_active":"2023-10-26T01:35:29.215935+0000","last_peered":"2023-10-26T01:35:29.215935+0000","last_clean":"2023-10-26T01:35:29.215935+0000","last_became_active":"2023-10-26T01:35:27.380526+0000","last_became_peered":"2023-10-26T01:35:27.380526+0000","last_unstale":"2023-10-26T01:35:29.215935+0000","last_undegraded":"2023-10-26T01:35:29.215935+0000","last_fullsized":"2023-10-26T01:35:29.215935+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_clean_scrub_stamp":"2023-10-26T01:35:26.337183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:25:00.697915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.216709+0000","last_change":"2023-10-26T01:35:28.012709+0000","last_active":"2023-10-26T01:35:29.216709+0000","last_peered":"2023-10-26T01:35:29.216709+0000","last_clean":"2023-10-26T01:35:29.216709+0000","last_became_active":"2023-10-26T01:35:28.010687+0000","last_became_peered":"2023-10-26T01:35:28.010687+0000","last_unstale":"2023-10-26T01:35:29.216709+0000","last_undegraded":"2023-10-26T01:35:29.216709+0000","last_fullsized":"2023-10-26T01:35:29.216709+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:44:27.457808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.214166+0000","last_change":"2023-10-26T01:35:27.382781+0000","last_active":"2023-10-26T01:35:29.214166+0000","last_peered":"2023-10-26T01:35:29.214166+0000","last_clean":"2023-10-26T01:35:29.214166+0000","last_became_active":"2023-10-26T01:35:27.382235+0000","last_became_peered":"2023-10-26T01:35:27.382235+0000","last_unstale":"2023-10-26T01:35:29.214166+0000","last_undegraded":"2023-10-26T01:35:29.214166+0000","last_fullsized":"2023-10-26T01:35:29.214166+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_clean_scrub_stamp":"2023-10-26T01:35:26.337183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:13:37.237113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.216675+0000","last_change":"2023-10-26T01:35:28.001107+0000","last_active":"2023-10-26T01:35:29.216675+0000","last_peered":"2023-10-26T01:35:29.216675+0000","last_clean":"2023-10-26T01:35:29.216675+0000","last_became_active":"2023-10-26T01:35:28.000115+0000","last_became_peered":"2023-10-26T01:35:28.000115+0000","last_unstale":"2023-10-26T01:35:29.216675+0000","last_undegraded":"2023-10-26T01:35:29.216675+0000","last_fullsized":"2023-10-26T01:35:29.216675+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:14:38.641246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:28.994817+0000","last_change":"2023-10-26T01:35:27.381156+0000","last_active":"2023-10-26T01:35:28.994817+0000","last_peered":"2023-10-26T01:35:28.994817+0000","last_clean":"2023-10-26T01:35:28.994817+0000","last_became_active":"2023-10-26T01:35:27.380359+0000","last_became_peered":"2023-10-26T01:35:27.380359+0000","last_unstale":"2023-10-26T01:35:28.994817+0000","last_undegraded":"2023-10-26T01:35:28.994817+0000","last_fullsized":"2023-10-26T01:35:28.994817+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_clean_scrub_stamp":"2023-10-26T01:35:26.337183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:30:09.768728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.215930+0000","last_change":"2023-10-26T01:35:28.013975+0000","last_active":"2023-10-26T01:35:29.215930+0000","last_peered":"2023-10-26T01:35:29.215930+0000","last_clean":"2023-10-26T01:35:29.215930+0000","last_became_active":"2023-10-26T01:35:28.013716+0000","last_became_peered":"2023-10-26T01:35:28.013716+0000","last_unstale":"2023-10-26T01:35:29.215930+0000","last_undegraded":"2023-10-26T01:35:29.215930+0000","last_fullsized":"2023-10-26T01:35:29.215930+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:39:52.015674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.215256+0000","last_change":"2023-10-26T01:35:27.385396+0000","last_active":"2023-10-26T01:35:29.215256+0000","last_peered":"2023-10-26T01:35:29.215256+0000","last_clean":"2023-10-26T01:35:29.215256+0000","last_became_active":"2023-10-26T01:35:27.382472+0000","last_became_peered":"2023-10-26T01:35:27.382472+0000","last_unstale":"2023-10-26T01:35:29.215256+0000","last_undegraded":"2023-10-26T01:35:29.215256+0000","last_fullsized":"2023-10-26T01:35:29.215256+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_clean_scrub_stamp":"2023-10-26T01:35:26.337183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:46:50.811699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:28.995569+0000","last_change":"2023-10-26T01:35:28.008065+0000","last_active":"2023-10-26T01:35:28.995569+0000","last_peered":"2023-10-26T01:35:28.995569+0000","last_clean":"2023-10-26T01:35:28.995569+0000","last_became_active":"2023-10-26T01:35:28.007721+0000","last_became_peered":"2023-10-26T01:35:28.007721+0000","last_unstale":"2023-10-26T01:35:28.995569+0000","last_undegraded":"2023-10-26T01:35:28.995569+0000","last_fullsized":"2023-10-26T01:35:28.995569+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:58:55.944315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"83'2","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.216590+0000","last_change":"2023-10-26T01:35:27.386574+0000","last_active":"2023-10-26T01:35:29.216590+0000","last_peered":"2023-10-26T01:35:29.216590+0000","last_clean":"2023-10-26T01:35:29.216590+0000","last_became_active":"2023-10-26T01:35:27.385743+0000","last_became_peered":"2023-10-26T01:35:27.385743+0000","last_unstale":"2023-10-26T01:35:29.216590+0000","last_undegraded":"2023-10-26T01:35:29.216590+0000","last_fullsized":"2023-10-26T01:35:29.216590+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_clean_scrub_stamp":"2023-10-26T01:35:26.337183+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:27:20.180507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":229,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T01:35:27.987210+0000","last_change":"2023-10-26T01:30:34.883484+0000","last_active":"2023-10-26T01:35:27.987210+0000","last_peered":"2023-10-26T01:35:27.987210+0000","last_clean":"2023-10-26T01:35:27.987210+0000","last_became_active":"2023-10-26T01:30:34.883071+0000","last_became_peered":"2023-10-26T01:30:34.883071+0000","last_unstale":"2023-10-26T01:35:27.987210+0000","last_undegraded":"2023-10-26T01:35:27.987210+0000","last_fullsized":"2023-10-26T01:35:27.987210+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:32.890096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:32.890096+0000","last_clean_scrub_stamp":"2023-10-26T01:30:32.890096+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:25:16.321641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":14,"seq":60129542207,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":433,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194878,"statfs":{"total":200000000,"available":199555618,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:34:43 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.80500000000000005,"15min":0.78900000000000003},"min":{"1min":0.628,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.365,"5min":2.2549999999999999,"15min":2.2549999999999999},"last":0.82599999999999996},{"interface":"front","average":{"1min":1.004,"5min":0.86599999999999999,"15min":0.89400000000000002},"min":{"1min":0.67500000000000004,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.5800000000000001,"5min":1.8200000000000001,"15min":1.8200000000000001},"last":0.94399999999999995}]},{"osd":1,"last update":"Thu Oct 26 01:34:43 2023","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":0.89800000000000002,"15min":0.96399999999999997},"min":{"1min":0.52000000000000002,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.444,"5min":4.4340000000000002,"15min":4.4340000000000002},"last":1.0820000000000001},{"interface":"front","average":{"1min":1.048,"5min":0.98199999999999998,"15min":1.006},"min":{"1min":0.81399999999999995,"5min":0.374,"15min":0.374},"max":{"1min":1.538,"5min":2.3330000000000002,"15min":2.3330000000000002},"last":1.119}]},{"osd":2,"last update":"Thu Oct 26 01:34:43 2023","interfaces":[{"interface":"back","average":{"1min":1.071,"5min":1.0589999999999999,"15min":1.097},"min":{"1min":0.59099999999999997,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.3779999999999999,"5min":4.2800000000000002,"15min":4.2800000000000002},"last":0.71799999999999997},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.1080000000000001,"15min":1.087},"min":{"1min":0.746,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.659,"5min":2.302,"15min":2.302},"last":1.3080000000000001}]},{"osd":4,"last update":"Thu Oct 26 01:34:43 2023","interfaces":[{"interface":"back","average":{"1min":1.2749999999999999,"5min":1.1699999999999999,"15min":1.1559999999999999},"min":{"1min":0.96099999999999997,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.5960000000000001,"5min":2.1070000000000002,"15min":2.1070000000000002},"last":1.252},{"interface":"front","average":{"1min":1.405,"5min":1.3660000000000001,"15min":1.411},"min":{"1min":0.92200000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":1.5940000000000001,"5min":4.8250000000000002,"15min":4.8250000000000002},"last":1.379}]},{"osd":5,"last update":"Thu Oct 26 01:34:43 2023","interfaces":[{"interface":"back","average":{"1min":1.343,"5min":1.302,"15min":1.355},"min":{"1min":0.85799999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.9530000000000001,"5min":4.9080000000000004,"15min":4.9080000000000004},"last":1.484},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.264,"15min":1.2749999999999999},"min":{"1min":0.98299999999999998,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.706,"5min":2.8980000000000001,"15min":2.8980000000000001},"last":1.196}]},{"osd":6,"last update":"Thu Oct 26 01:34:43 2023","interfaces":[{"interface":"back","average":{"1min":1.3959999999999999,"5min":1.3300000000000001,"15min":1.3240000000000001},"min":{"1min":1.0109999999999999,"5min":0.85099999999999998,"15min":0.85099999999999998},"max":{"1min":1.6499999999999999,"5min":2.8530000000000002,"15min":2.8530000000000002},"last":1.4139999999999999},{"interface":"front","average":{"1min":1.3180000000000001,"5min":1.244,"15min":1.2669999999999999},"min":{"1min":0.83699999999999997,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.694,"5min":2.181,"15min":2.181},"last":1.6160000000000001}]},{"osd":7,"last update":"Thu Oct 26 01:34:43 2023","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.284,"15min":1.2849999999999999},"min":{"1min":0.89300000000000002,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.778,"5min":2.0680000000000001,"15min":2.0680000000000001},"last":1.6879999999999999},{"interface":"front","average":{"1min":1.4590000000000001,"5min":1.3919999999999999,"15min":1.4299999999999999},"min":{"1min":0.90900000000000003,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.887,"5min":4.8730000000000002,"15min":4.8730000000000002},"last":1.577}]}]},{"osd":7,"up_from":13,"seq":55834574910,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1010,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194301,"statfs":{"total":200000000,"available":198964746,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:34:44 2023","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.0980000000000001,"15min":1.0760000000000001},"min":{"1min":0.752,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.5740000000000001,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":1.609},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.1279999999999999,"15min":1.075},"min":{"1min":0.80300000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":2.6779999999999999,"5min":2.6779999999999999,"15min":2.6779999999999999},"last":1.2589999999999999}]},{"osd":1,"last update":"Thu Oct 26 01:34:44 2023","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.155,"15min":1.0740000000000001},"min":{"1min":0.83299999999999996,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":2.8260000000000001,"5min":2.8260000000000001,"15min":2.8260000000000001},"last":1.2190000000000001},{"interface":"front","average":{"1min":1.3300000000000001,"5min":1.24,"15min":1.196},"min":{"1min":0.86399999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":3.2730000000000001,"5min":3.2730000000000001,"15min":3.2730000000000001},"last":0.84699999999999998}]},{"osd":2,"last update":"Thu Oct 26 01:34:44 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.1259999999999999,"15min":1.0980000000000001},"min":{"1min":0.85799999999999998,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":2.218,"5min":2.218,"15min":2.218},"last":1.54},{"interface":"front","average":{"1min":1.4279999999999999,"5min":1.2729999999999999,"15min":1.228},"min":{"1min":1.036,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":3.0510000000000002,"5min":3.0510000000000002,"15min":3.0510000000000002},"last":0.99399999999999999}]},{"osd":3,"last update":"Thu Oct 26 01:34:44 2023","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.2290000000000001,"15min":1.194},"min":{"1min":0.79000000000000004,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.9830000000000001,"5min":1.9830000000000001,"15min":1.9830000000000001},"last":1.133},{"interface":"front","average":{"1min":1.7170000000000001,"5min":1.4350000000000001,"15min":1.3819999999999999},"min":{"1min":0.89100000000000001,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":5.5039999999999996,"5min":5.5039999999999996,"15min":5.5039999999999996},"last":1.444}]},{"osd":4,"last update":"Thu Oct 26 01:34:44 2023","interfaces":[{"interface":"back","average":{"1min":1.425,"5min":1.353,"15min":1.347},"min":{"1min":0.93400000000000005,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":2.77,"5min":2.77,"15min":2.77},"last":1.3680000000000001},{"interface":"front","average":{"1min":1.546,"5min":1.4419999999999999,"15min":1.3999999999999999},"min":{"1min":1.0640000000000001,"5min":0.496,"15min":0.496},"max":{"1min":3.1419999999999999,"5min":3.1419999999999999,"15min":3.1419999999999999},"last":1.4850000000000001}]},{"osd":5,"last update":"Thu Oct 26 01:34:44 2023","interfaces":[{"interface":"back","average":{"1min":1.722,"5min":1.4910000000000001,"15min":1.425},"min":{"1min":1.0649999999999999,"5min":0.441,"15min":0.441},"max":{"1min":5.734,"5min":5.734,"15min":5.734},"last":1.639},{"interface":"front","average":{"1min":1.6040000000000001,"5min":1.4890000000000001,"15min":1.4690000000000001},"min":{"1min":1.113,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":2.8980000000000001,"5min":2.8980000000000001,"15min":2.8980000000000001},"last":1.883}]},{"osd":6,"last update":"Thu Oct 26 01:34:44 2023","interfaces":[{"interface":"back","average":{"1min":1.8129999999999999,"5min":1.7929999999999999,"15min":1.9399999999999999},"min":{"1min":1.175,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":5.6340000000000003,"5min":11.577999999999999,"15min":11.577999999999999},"last":1.8300000000000001},{"interface":"front","average":{"1min":1.863,"5min":1.8069999999999999,"15min":1.9370000000000001},"min":{"1min":1.1559999999999999,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":5.0919999999999996,"5min":10.704000000000001,"15min":10.704000000000001},"last":1.706}]}]},{"osd":6,"up_from":13,"seq":55834574910,"num_pgs":35,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":433,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194879,"statfs":{"total":200000000,"available":199556252,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:34:37 2023","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.135,"15min":1.173},"min":{"1min":0.64500000000000002,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":2.395,"5min":2.395,"15min":2.395},"last":1.462},{"interface":"front","average":{"1min":1.222,"5min":1.1020000000000001,"15min":1.111},"min":{"1min":0.86099999999999999,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.6910000000000001,"5min":1.843,"15min":1.843},"last":0.96499999999999997}]},{"osd":1,"last update":"Thu Oct 26 01:34:37 2023","interfaces":[{"interface":"back","average":{"1min":1.306,"5min":1.1559999999999999,"15min":1.169},"min":{"1min":0.77300000000000002,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":2.0529999999999999,"5min":2.0529999999999999,"15min":2.0529999999999999},"last":1.369},{"interface":"front","average":{"1min":1.3129999999999999,"5min":1.1839999999999999,"15min":1.212},"min":{"1min":0.82299999999999995,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.581,"5min":2.085,"15min":2.085},"last":1.331}]},{"osd":2,"last update":"Thu Oct 26 01:34:37 2023","interfaces":[{"interface":"back","average":{"1min":1.373,"5min":1.1319999999999999,"15min":1.0780000000000001},"min":{"1min":0.93000000000000005,"5min":0.747,"15min":0.747},"max":{"1min":3.7879999999999998,"5min":3.7879999999999998,"15min":3.7879999999999998},"last":1.4079999999999999},{"interface":"front","average":{"1min":1.3740000000000001,"5min":1.264,"15min":1.2529999999999999},"min":{"1min":0.79100000000000004,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":2.2890000000000001,"5min":2.2890000000000001,"15min":2.2890000000000001},"last":1.0269999999999999}]},{"osd":3,"last update":"Thu Oct 26 01:34:37 2023","interfaces":[{"interface":"back","average":{"1min":1.5269999999999999,"5min":1.2470000000000001,"15min":1.1870000000000001},"min":{"1min":0.92700000000000005,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":4.4560000000000004,"5min":4.4560000000000004,"15min":4.4560000000000004},"last":1.2370000000000001},{"interface":"front","average":{"1min":1.3979999999999999,"5min":1.329,"15min":1.323},"min":{"1min":0.82299999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":2.5019999999999998,"5min":2.5019999999999998,"15min":2.5019999999999998},"last":1.5569999999999999}]},{"osd":4,"last update":"Thu Oct 26 01:34:37 2023","interfaces":[{"interface":"back","average":{"1min":1.2869999999999999,"5min":1.2330000000000001,"15min":1.2130000000000001},"min":{"1min":0.63900000000000001,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":2.9489999999999998,"5min":2.9489999999999998,"15min":2.9489999999999998},"last":1.208},{"interface":"front","average":{"1min":1.7629999999999999,"5min":1.385,"15min":1.298},"min":{"1min":0.85899999999999999,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":6.609,"5min":6.609,"15min":6.609},"last":1.5269999999999999}]},{"osd":5,"last update":"Thu Oct 26 01:34:37 2023","interfaces":[{"interface":"back","average":{"1min":1.647,"5min":1.4790000000000001,"15min":1.4199999999999999},"min":{"1min":1.1739999999999999,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":5.4370000000000003,"5min":5.4370000000000003,"15min":5.4370000000000003},"last":1.589},{"interface":"front","average":{"1min":1.3919999999999999,"5min":1.415,"15min":1.3979999999999999},"min":{"1min":0.61399999999999999,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":2.621,"5min":2.621,"15min":2.621},"last":1.6160000000000001}]},{"osd":7,"last update":"Thu Oct 26 01:34:37 2023","interfaces":[{"interface":"back","average":{"1min":1.9199999999999999,"5min":1.49,"15min":1.3999999999999999},"min":{"1min":1.0029999999999999,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":11.804,"5min":11.804,"15min":11.804},"last":1.494},{"interface":"front","average":{"1min":2.0539999999999998,"5min":1.5920000000000001,"15min":1.484},"min":{"1min":1.1639999999999999,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":11.541,"5min":11.541,"15min":11.541},"last":1.9419999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574910,"num_pgs":14,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":433,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194878,"statfs":{"total":200000000,"available":199555640,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:34:43 2023","interfaces":[{"interface":"back","average":{"1min":1.3169999999999999,"5min":1.155,"15min":1.1180000000000001},"min":{"1min":0.88100000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.617,"5min":1.7390000000000001,"15min":1.7390000000000001},"last":0.70899999999999996},{"interface":"front","average":{"1min":1.4139999999999999,"5min":1.2130000000000001,"15min":1.1579999999999999},"min":{"1min":1.0309999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.6659999999999999,"5min":1.6659999999999999,"15min":1.6659999999999999},"last":1.248}]},{"osd":1,"last update":"Thu Oct 26 01:34:43 2023","interfaces":[{"interface":"back","average":{"1min":1.194,"5min":1.113,"15min":1.081},"min":{"1min":0.80000000000000004,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.504,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":1.333},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.1319999999999999,"15min":1.0860000000000001},"min":{"1min":0.80500000000000005,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.6000000000000001,"5min":1.7669999999999999,"15min":1.7669999999999999},"last":0.83399999999999996}]},{"osd":2,"last update":"Thu Oct 26 01:34:43 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.1719999999999999,"15min":1.151},"min":{"1min":0.84099999999999997,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.599,"5min":1.788,"15min":1.788},"last":1.097},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.1719999999999999,"15min":1.1639999999999999},"min":{"1min":0.85499999999999998,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.474,"5min":1.579,"15min":1.579},"last":0.98799999999999999}]},{"osd":3,"last update":"Thu Oct 26 01:34:43 2023","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.268,"15min":1.232},"min":{"1min":0.99099999999999999,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.536,"5min":1.835,"15min":1.835},"last":1.1759999999999999},{"interface":"front","average":{"1min":1.3560000000000001,"5min":1.3029999999999999,"15min":1.2889999999999999},"min":{"1min":1.048,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.7509999999999999,"5min":2.9249999999999998,"15min":2.9249999999999998},"last":1.381}]},{"osd":4,"last update":"Thu Oct 26 01:34:43 2023","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.242,"15min":1.1910000000000001},"min":{"1min":0.84799999999999998,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.579,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":1.0329999999999999},{"interface":"front","average":{"1min":1.371,"5min":1.3759999999999999,"15min":1.329},"min":{"1min":0.96699999999999997,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.764,"5min":1.867,"15min":1.867},"last":1.5700000000000001}]},{"osd":6,"last update":"Thu Oct 26 01:34:43 2023","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.3440000000000001,"15min":1.2889999999999999},"min":{"1min":0.97699999999999998,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.6439999999999999,"5min":1.8999999999999999,"15min":1.8999999999999999},"last":1.4390000000000001},{"interface":"front","average":{"1min":1.401,"5min":1.375,"15min":1.3009999999999999},"min":{"1min":1.0740000000000001,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.6850000000000001,"5min":1.8560000000000001,"15min":1.8560000000000001},"last":0.93700000000000006}]},{"osd":7,"last update":"Thu Oct 26 01:34:43 2023","interfaces":[{"interface":"back","average":{"1min":1.5269999999999999,"5min":1.462,"15min":1.3899999999999999},"min":{"1min":1.234,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.8999999999999999,"5min":1.97,"15min":1.97},"last":1.4850000000000001},{"interface":"front","average":{"1min":1.4850000000000001,"5min":1.4630000000000001,"15min":1.395},"min":{"1min":1.0600000000000001,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.901,"5min":2.0680000000000001,"15min":2.0680000000000001},"last":1.524}]}]},{"osd":4,"up_from":13,"seq":55834574910,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":433,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194878,"statfs":{"total":200000000,"available":199555660,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:34:34 2023","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":0.996,"15min":0.96199999999999997},"min":{"1min":0.71799999999999997,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.3080000000000001,"5min":2.0289999999999999,"15min":2.0289999999999999},"last":1.3620000000000001},{"interface":"front","average":{"1min":1.016,"5min":1.022,"15min":1.008},"min":{"1min":0.71099999999999997,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.363,"5min":2.6829999999999998,"15min":2.6829999999999998},"last":1.036}]},{"osd":1,"last update":"Thu Oct 26 01:34:34 2023","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":1.1299999999999999,"15min":1.1060000000000001},"min":{"1min":0.67800000000000005,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.5900000000000001,"5min":4.0519999999999996,"15min":4.0519999999999996},"last":1.1739999999999999},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.121,"15min":1.097},"min":{"1min":0.83799999999999997,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.6419999999999999,"5min":2.0760000000000001,"15min":2.0760000000000001},"last":1.202}]},{"osd":2,"last update":"Thu Oct 26 01:34:34 2023","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":1.2829999999999999,"15min":1.2070000000000001},"min":{"1min":0.91200000000000003,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.633,"5min":5.6749999999999998,"15min":5.6749999999999998},"last":1.2989999999999999},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.151,"15min":1.1100000000000001},"min":{"1min":0.751,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.849,"5min":1.849,"15min":1.849},"last":0.94499999999999995}]},{"osd":3,"last update":"Thu Oct 26 01:34:34 2023","interfaces":[{"interface":"back","average":{"1min":1.24,"5min":1.1859999999999999,"15min":1.179},"min":{"1min":0.87,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.5469999999999999,"5min":1.891,"15min":1.891},"last":0.83699999999999997},{"interface":"front","average":{"1min":1.208,"5min":1.167,"15min":1.143},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.681,"5min":1.7210000000000001,"15min":1.7210000000000001},"last":1.139}]},{"osd":5,"last update":"Thu Oct 26 01:34:34 2023","interfaces":[{"interface":"back","average":{"1min":1.2250000000000001,"5min":1.266,"15min":1.2909999999999999},"min":{"1min":0.73699999999999999,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.6140000000000001,"5min":2.0049999999999999,"15min":2.0049999999999999},"last":0.996},{"interface":"front","average":{"1min":1.2769999999999999,"5min":1.349,"15min":1.333},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.9810000000000001,"5min":5.8579999999999997,"15min":5.8579999999999997},"last":1.52}]},{"osd":6,"last update":"Thu Oct 26 01:34:34 2023","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":1.4339999999999999,"15min":1.3380000000000001},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.919,"5min":12.343999999999999,"15min":12.343999999999999},"last":1.2549999999999999},{"interface":"front","average":{"1min":1.5069999999999999,"5min":1.597,"15min":1.5049999999999999},"min":{"1min":0.78500000000000003,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.929,"5min":12.605,"15min":12.605},"last":1.097}]},{"osd":7,"last update":"Thu Oct 26 01:34:34 2023","interfaces":[{"interface":"back","average":{"1min":1.4770000000000001,"5min":1.5249999999999999,"15min":1.4930000000000001},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":2.089,"5min":5.774,"15min":5.774},"last":1.45},{"interface":"front","average":{"1min":1.5449999999999999,"5min":1.496,"15min":1.446},"min":{"1min":0.80600000000000005,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":1.972,"5min":4.6520000000000001,"15min":4.6520000000000001},"last":1.417}]}]},{"osd":2,"up_from":13,"seq":55834574910,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":435,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194877,"statfs":{"total":200000000,"available":199554118,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:34:35 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.85599999999999998,"15min":0.85399999999999998},"min":{"1min":0.67000000000000004,"5min":0.308,"15min":0.308},"max":{"1min":1.105,"5min":6.3200000000000003,"15min":6.3200000000000003},"last":0.79200000000000004},{"interface":"front","average":{"1min":1.0980000000000001,"5min":0.97199999999999998,"15min":0.91900000000000004},"min":{"1min":0.63700000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.361,"5min":4.2619999999999996,"15min":4.2619999999999996},"last":0.877}]},{"osd":1,"last update":"Thu Oct 26 01:34:35 2023","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.90800000000000003,"15min":0.89700000000000002},"min":{"1min":0.73799999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.5309999999999999,"5min":5.1260000000000003,"15min":5.1260000000000003},"last":1.1100000000000001},{"interface":"front","average":{"1min":1.103,"5min":1.028,"15min":1},"min":{"1min":0.80900000000000005,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.4259999999999999,"5min":6.1520000000000001,"15min":6.1520000000000001},"last":1.1299999999999999}]},{"osd":3,"last update":"Thu Oct 26 01:34:42 2023","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":0.97899999999999998,"15min":0.92200000000000004},"min":{"1min":0.63100000000000001,"5min":0.443,"15min":0.443},"max":{"1min":1.4830000000000001,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":1.004},{"interface":"front","average":{"1min":1.1319999999999999,"5min":1.0820000000000001,"15min":0.98999999999999999},"min":{"1min":0.89500000000000002,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.673,"5min":8.9019999999999992,"15min":8.9019999999999992},"last":1.0389999999999999}]},{"osd":4,"last update":"Thu Oct 26 01:34:35 2023","interfaces":[{"interface":"back","average":{"1min":1.419,"5min":1.2989999999999999,"15min":1.246},"min":{"1min":1.0760000000000001,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.774,"5min":6.2670000000000003,"15min":6.2670000000000003},"last":1.3300000000000001},{"interface":"front","average":{"1min":1.258,"5min":1.298,"15min":1.24},"min":{"1min":1.032,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":1.591,"5min":7.343,"15min":7.343},"last":1.2310000000000001}]},{"osd":5,"last update":"Thu Oct 26 01:34:35 2023","interfaces":[{"interface":"back","average":{"1min":1.389,"5min":1.296,"15min":1.266},"min":{"1min":1.1060000000000001,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.659,"5min":3.7170000000000001,"15min":3.7170000000000001},"last":1.1799999999999999},{"interface":"front","average":{"1min":1.363,"5min":1.3120000000000001,"15min":1.286},"min":{"1min":0.98399999999999999,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.5920000000000001,"5min":6.3899999999999997,"15min":6.3899999999999997},"last":1.2649999999999999}]},{"osd":6,"last update":"Thu Oct 26 01:34:35 2023","interfaces":[{"interface":"back","average":{"1min":1.4910000000000001,"5min":1.3720000000000001,"15min":1.3169999999999999},"min":{"1min":1,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.905,"5min":6.0490000000000004,"15min":6.0490000000000004},"last":1.0660000000000001},{"interface":"front","average":{"1min":1.411,"5min":1.3799999999999999,"15min":1.355},"min":{"1min":1.141,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.7509999999999999,"5min":5.0650000000000004,"15min":5.0650000000000004},"last":1.3069999999999999}]},{"osd":7,"last update":"Thu Oct 26 01:34:35 2023","interfaces":[{"interface":"back","average":{"1min":1.4850000000000001,"5min":1.415,"15min":1.397},"min":{"1min":1.1579999999999999,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.8839999999999999,"5min":6.4480000000000004,"15min":6.4480000000000004},"last":1.532},{"interface":"front","average":{"1min":1.5740000000000001,"5min":1.46,"15min":1.4379999999999999},"min":{"1min":1.1930000000000001,"5min":0.91800000000000004,"15min":0.91800000000000004},"max":{"1min":2.0070000000000001,"5min":2.8439999999999999,"15min":2.8439999999999999},"last":1.4399999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574910,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":436,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194876,"statfs":{"total":200000000,"available":199553502,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:34:38 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.751,"15min":0.71099999999999997},"min":{"1min":0.379,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.80000000000000004,"15min":0.77600000000000002},"min":{"1min":0.45000000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.405,"5min":1.4830000000000001,"15min":1.4830000000000001},"last":0.67500000000000004}]},{"osd":2,"last update":"Thu Oct 26 01:34:38 2023","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":0.97699999999999998,"15min":1.004},"min":{"1min":0.39700000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.4239999999999999,"5min":2.4830000000000001,"15min":2.4830000000000001},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.91400000000000003,"5min":0.86199999999999999,"15min":0.84899999999999998},"min":{"1min":0.57299999999999995,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.3500000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":1.0980000000000001}]},{"osd":3,"last update":"Thu Oct 26 01:34:42 2023","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":0.97399999999999998,"15min":0.94399999999999995},"min":{"1min":0.621,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.611,"5min":1.75,"15min":1.75},"last":1.1850000000000001},{"interface":"front","average":{"1min":1.05,"5min":0.99399999999999999,"15min":0.97799999999999998},"min":{"1min":0.47799999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.5389999999999999,"5min":1.802,"15min":1.802},"last":0.48799999999999999}]},{"osd":4,"last update":"Thu Oct 26 01:34:38 2023","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.214,"15min":1.1919999999999999},"min":{"1min":0.79200000000000004,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.605,"5min":2.0659999999999998,"15min":2.0659999999999998},"last":0.92600000000000005},{"interface":"front","average":{"1min":1.2470000000000001,"5min":1.153,"15min":1.123},"min":{"1min":0.95399999999999996,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.637,"5min":1.637,"15min":1.637},"last":0.76600000000000001}]},{"osd":5,"last update":"Thu Oct 26 01:34:38 2023","interfaces":[{"interface":"back","average":{"1min":1.3009999999999999,"5min":1.224,"15min":1.22},"min":{"1min":1.077,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":1.5620000000000001,"5min":1.702,"15min":1.702},"last":1.0549999999999999},{"interface":"front","average":{"1min":1.4079999999999999,"5min":1.331,"15min":1.329},"min":{"1min":0.84799999999999998,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.673,"5min":2.2200000000000002,"15min":2.2200000000000002},"last":1.085}]},{"osd":6,"last update":"Thu Oct 26 01:34:38 2023","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":1.278,"15min":1.2649999999999999},"min":{"1min":0.80900000000000005,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":2.0910000000000002,"5min":2.3780000000000001,"15min":2.3780000000000001},"last":0.90000000000000002},{"interface":"front","average":{"1min":1.274,"5min":1.252,"15min":1.234},"min":{"1min":0.91700000000000004,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.6679999999999999,"5min":1.984,"15min":1.984},"last":1.2390000000000001}]},{"osd":7,"last update":"Thu Oct 26 01:34:38 2023","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":1.25,"15min":1.264},"min":{"1min":0.81299999999999994,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.5700000000000001,"5min":2.6930000000000001,"15min":2.6930000000000001},"last":0.84199999999999997},{"interface":"front","average":{"1min":1.4350000000000001,"5min":1.3400000000000001,"15min":1.3149999999999999},"min":{"1min":0.89700000000000002,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.7709999999999999,"5min":1.869,"15min":1.869},"last":1.0329999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574910,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1010,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194302,"statfs":{"total":200000000,"available":198965362,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Oct 26 01:34:38 2023","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":1.046,"15min":0.93899999999999995},"min":{"1min":0.63600000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.5049999999999999,"5min":19.425000000000001,"15min":19.425000000000001},"last":0.39200000000000002},{"interface":"front","average":{"1min":1.05,"5min":1.125,"15min":1.0720000000000001},"min":{"1min":0.69599999999999995,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.452,"5min":15.162000000000001,"15min":15.162000000000001},"last":0.47299999999999998}]},{"osd":2,"last update":"Thu Oct 26 01:34:38 2023","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.3700000000000001,"15min":1.218},"min":{"1min":0.878,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.577,"5min":28.439,"15min":28.439},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.91800000000000004,"15min":0.91800000000000004},"min":{"1min":0.58099999999999996,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.3220000000000001,"5min":4.9459999999999997,"15min":4.9459999999999997},"last":0.67200000000000004}]},{"osd":3,"last update":"Thu Oct 26 01:34:38 2023","interfaces":[{"interface":"back","average":{"1min":1.167,"5min":1.085,"15min":1.105},"min":{"1min":0.86599999999999999,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.6479999999999999,"5min":4.9800000000000004,"15min":4.9800000000000004},"last":0.50800000000000001},{"interface":"front","average":{"1min":1.226,"5min":1.163,"15min":1.171},"min":{"1min":0.61099999999999999,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.6559999999999999,"5min":5.016,"15min":5.016},"last":0.72499999999999998}]},{"osd":4,"last update":"Thu Oct 26 01:34:38 2023","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.4159999999999999,"15min":1.3109999999999999},"min":{"1min":0.94499999999999995,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.6439999999999999,"5min":19.329000000000001,"15min":19.329000000000001},"last":1.2010000000000001},{"interface":"front","average":{"1min":1.3460000000000001,"5min":1.296,"15min":1.3140000000000001},"min":{"1min":1.054,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.6739999999999999,"5min":5.5229999999999997,"15min":5.5229999999999997},"last":0.997}]},{"osd":5,"last update":"Thu Oct 26 01:34:38 2023","interfaces":[{"interface":"back","average":{"1min":1.3979999999999999,"5min":1.4219999999999999,"15min":1.4410000000000001},"min":{"1min":1.1839999999999999,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.605,"5min":5.6360000000000001,"15min":5.6360000000000001},"last":0.97899999999999998},{"interface":"front","average":{"1min":1.349,"5min":1.534,"15min":1.4259999999999999},"min":{"1min":0.95799999999999996,"5min":0.626,"15min":0.626},"max":{"1min":1.7390000000000001,"5min":19.384,"15min":19.384},"last":0.78400000000000003}]},{"osd":6,"last update":"Thu Oct 26 01:34:38 2023","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.4159999999999999,"15min":1.3859999999999999},"min":{"1min":1.1639999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.653,"5min":8.2609999999999992,"15min":8.2609999999999992},"last":0.84499999999999997},{"interface":"front","average":{"1min":1.411,"5min":1.472,"15min":1.4219999999999999},"min":{"1min":1.143,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.702,"5min":11.849,"15min":11.849},"last":0.93899999999999995}]},{"osd":7,"last update":"Thu Oct 26 01:34:38 2023","interfaces":[{"interface":"back","average":{"1min":1.4650000000000001,"5min":1.4810000000000001,"15min":1.5},"min":{"1min":1.2430000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":1.7769999999999999,"5min":5.6689999999999996,"15min":5.6689999999999996},"last":1.0880000000000001},{"interface":"front","average":{"1min":1.4970000000000001,"5min":1.629,"15min":1.516},"min":{"1min":1.155,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.8169999999999999,"5min":19.245999999999999,"15min":19.245999999999999},"last":0.879}]}]}],"pool_statfs":[]}} 2023-10-26T01:35:32.842 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-26T01:35:33.101 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:33.098+0000 7fa592087640 1 -- 172.21.15.169:0/482472658 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa58c0ceb30 msgr2=0x7fa58c0b3110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:33.102 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:33.098+0000 7fa592087640 1 --2- 172.21.15.169:0/482472658 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa58c0ceb30 0x7fa58c0b3110 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fa58c0b4280 tx=0x7fa58402bd90 comp rx=0 tx=0).stop 2023-10-26T01:35:33.102 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:33.099+0000 7fa592087640 1 -- 172.21.15.169:0/482472658 shutdown_connections 2023-10-26T01:35:33.102 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:33.099+0000 7fa592087640 1 --2- 172.21.15.169:0/482472658 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa58c0ceb30 0x7fa58c0b3110 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:33.102 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:33.099+0000 7fa592087640 1 --2- 172.21.15.169:0/482472658 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa58c0d0fb0 0x7fa58c0ce5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:33.103 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:33.099+0000 7fa592087640 1 --2- 172.21.15.169:0/482472658 >> v2:172.21.15.170:3300/0 conn(0x7fa58c15baa0 0x7fa58c0b6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:33.103 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:33.099+0000 7fa592087640 1 -- 172.21.15.169:0/482472658 >> 172.21.15.169:0/482472658 conn(0x7fa58c0658b0 msgr2=0x7fa58c065cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:35:33.103 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:33.099+0000 7fa592087640 1 -- 172.21.15.169:0/482472658 shutdown_connections 2023-10-26T01:35:33.103 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:33.099+0000 7fa592087640 1 -- 172.21.15.169:0/482472658 wait complete. 2023-10-26T01:35:33.105 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:33.100+0000 7fa592087640 1 Processor -- start 2023-10-26T01:35:33.106 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:33.101+0000 7fa592087640 1 -- start start 2023-10-26T01:35:33.106 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:33.101+0000 7fa592087640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa58c0ceb30 0x7fa58c1f3610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:33.106 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:33.102+0000 7fa592087640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa58c0d0fb0 0x7fa58c14eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:33.106 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:33.102+0000 7fa592087640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa58c0b4a80 0x7fa58c0b4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:33.107 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:33.102+0000 7fa592087640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa58c0b5450 con 0x7fa58c0d0fb0 2023-10-26T01:35:33.107 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:33.102+0000 7fa592087640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa58c15e620 con 0x7fa58c0b4a80 2023-10-26T01:35:33.107 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:33.102+0000 7fa592087640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa58c0c5340 con 0x7fa58c0ceb30 2023-10-26T01:35:33.107 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:33.102+0000 7fa58affd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa58c0d0fb0 0x7fa58c14eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:33.107 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:33.102+0000 7fa58affd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa58c0d0fb0 0x7fa58c14eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:57026/0 (socket says 172.21.15.169:57026) 2023-10-26T01:35:33.108 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:33.102+0000 7fa58affd640 1 -- 172.21.15.169:0/1475693708 learned_addr learned my addr 172.21.15.169:0/1475693708 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:35:33.108 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:33.102+0000 7fa58b7fe640 1 --2- 172.21.15.169:0/1475693708 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa58c0ceb30 0x7fa58c1f3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:33.108 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:33.102+0000 7fa58bfff640 1 --2- 172.21.15.169:0/1475693708 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa58c0b4a80 0x7fa58c0b4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:33.108 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:33.102+0000 7fa58affd640 1 -- 172.21.15.169:0/1475693708 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa58c0ceb30 msgr2=0x7fa58c1f3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:33.108 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:33.102+0000 7fa58affd640 1 --2- 172.21.15.169:0/1475693708 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa58c0ceb30 0x7fa58c1f3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:33.109 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:33.102+0000 7fa58affd640 1 -- 172.21.15.169:0/1475693708 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa58c0b4a80 msgr2=0x7fa58c0b4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:33.109 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:33.102+0000 7fa58affd640 1 --2- 172.21.15.169:0/1475693708 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa58c0b4a80 0x7fa58c0b4f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:33.109 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:33.102+0000 7fa58affd640 1 -- 172.21.15.169:0/1475693708 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa584009560 con 0x7fa58c0d0fb0 2023-10-26T01:35:33.109 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:33.102+0000 7fa58b7fe640 1 --2- 172.21.15.169:0/1475693708 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa58c0ceb30 0x7fa58c1f3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:35:33.110 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:33.102+0000 7fa58bfff640 1 --2- 172.21.15.169:0/1475693708 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa58c0b4a80 0x7fa58c0b4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:35:33.110 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:33.103+0000 7fa58affd640 1 --2- 172.21.15.169:0/1475693708 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa58c0d0fb0 0x7fa58c14eb00 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fa57400d750 tx=0x7fa57400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:35:33.110 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:33.103+0000 7fa583fff640 1 -- 172.21.15.169:0/1475693708 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5740099d0 con 0x7fa58c0d0fb0 2023-10-26T01:35:33.110 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:33.103+0000 7fa583fff640 1 -- 172.21.15.169:0/1475693708 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa574009c20 con 0x7fa58c0d0fb0 2023-10-26T01:35:33.110 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:33.103+0000 7fa583fff640 1 -- 172.21.15.169:0/1475693708 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa574005020 con 0x7fa58c0d0fb0 2023-10-26T01:35:33.111 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:33.103+0000 7fa592087640 1 -- 172.21.15.169:0/1475693708 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa58c0c5620 con 0x7fa58c0d0fb0 2023-10-26T01:35:33.111 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:33.103+0000 7fa592087640 1 -- 172.21.15.169:0/1475693708 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa58c0c5be0 con 0x7fa58c0d0fb0 2023-10-26T01:35:33.112 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:33.104+0000 7fa592087640 1 -- 172.21.15.169:0/1475693708 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa58c04db10 con 0x7fa58c0d0fb0 2023-10-26T01:35:33.112 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:33.105+0000 7fa583fff640 1 -- 172.21.15.169:0/1475693708 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fa574005180 con 0x7fa58c0d0fb0 2023-10-26T01:35:33.112 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:33.106+0000 7fa583fff640 1 --2- 172.21.15.169:0/1475693708 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa56c06ae40 0x7fa56c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:33.112 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:33.106+0000 7fa583fff640 1 -- 172.21.15.169:0/1475693708 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa5740898c0 con 0x7fa58c0d0fb0 2023-10-26T01:35:33.112 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:33.107+0000 7fa58b7fe640 1 --2- 172.21.15.169:0/1475693708 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa56c06ae40 0x7fa56c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:33.113 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:33.108+0000 7fa58b7fe640 1 --2- 172.21.15.169:0/1475693708 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa56c06ae40 0x7fa56c06d300 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fa57c005fd0 tx=0x7fa57c009340 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:35:33.113 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:33.111+0000 7fa583fff640 1 -- 172.21.15.169:0/1475693708 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa574054840 con 0x7fa58c0d0fb0 2023-10-26T01:35:33.301 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:33.298+0000 7fa592087640 1 -- 172.21.15.169:0/1475693708 --> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa58c0c8260 con 0x7fa56c06ae40 2023-10-26T01:35:33.324 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:33.321+0000 7fa583fff640 1 -- 172.21.15.169:0/1475693708 <== mgr.4112 v2:172.21.15.170:6832/1187156918 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+282622 (secure 0 0 0) 0x7fa58c0c8260 con 0x7fa56c06ae40 2023-10-26T01:35:33.328 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:35:33.329 INFO:teuthology.orchestra.run.smithi169.stderr:dumped all 2023-10-26T01:35:33.331 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:33.327+0000 7fa592087640 1 -- 172.21.15.169:0/1475693708 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa56c06ae40 msgr2=0x7fa56c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:33.331 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:33.327+0000 7fa592087640 1 --2- 172.21.15.169:0/1475693708 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa56c06ae40 0x7fa56c06d300 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fa57c005fd0 tx=0x7fa57c009340 comp rx=0 tx=0).stop 2023-10-26T01:35:33.331 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:33.327+0000 7fa592087640 1 -- 172.21.15.169:0/1475693708 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa58c0d0fb0 msgr2=0x7fa58c14eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:33.333 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:33.327+0000 7fa592087640 1 --2- 172.21.15.169:0/1475693708 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa58c0d0fb0 0x7fa58c14eb00 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fa57400d750 tx=0x7fa57400dc10 comp rx=0 tx=0).stop 2023-10-26T01:35:33.333 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:33.327+0000 7fa592087640 1 -- 172.21.15.169:0/1475693708 shutdown_connections 2023-10-26T01:35:33.334 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:33.327+0000 7fa592087640 1 --2- 172.21.15.169:0/1475693708 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa56c06ae40 0x7fa56c06d300 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:33.334 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:33.327+0000 7fa592087640 1 --2- 172.21.15.169:0/1475693708 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa58c0d0fb0 0x7fa58c14eb00 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:33.334 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:33.327+0000 7fa592087640 1 --2- 172.21.15.169:0/1475693708 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa58c0b4a80 0x7fa58c0b4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:33.334 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:33.327+0000 7fa592087640 1 --2- 172.21.15.169:0/1475693708 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa58c0ceb30 0x7fa58c1f3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:33.334 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:33.327+0000 7fa592087640 1 -- 172.21.15.169:0/1475693708 >> 172.21.15.169:0/1475693708 conn(0x7fa58c0658b0 msgr2=0x7fa58c0b2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:35:33.335 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:33.327+0000 7fa592087640 1 -- 172.21.15.169:0/1475693708 shutdown_connections 2023-10-26T01:35:33.335 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:33.327+0000 7fa592087640 1 -- 172.21.15.169:0/1475693708 wait complete. 2023-10-26T01:35:33.346 INFO:tasks.ceph.mds.a.smithi169.stderr:2023-10-26T01:35:33.343+0000 7fc4a0fdf640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-26T01:35:33.350 INFO:teuthology.orchestra.run.smithi169.stdout:{"pg_ready":true,"pg_map":{"version":228,"stamp":"2023-10-26T01:35:32.233368+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":169,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":4628,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1557871,"statfs":{"total":1600000000,"available":1595260898,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.881757"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:28.997503+0000","last_change":"2023-10-26T01:35:28.008835+0000","last_active":"2023-10-26T01:35:28.997503+0000","last_peered":"2023-10-26T01:35:28.997503+0000","last_clean":"2023-10-26T01:35:28.997503+0000","last_became_active":"2023-10-26T01:35:28.007977+0000","last_became_peered":"2023-10-26T01:35:28.007977+0000","last_unstale":"2023-10-26T01:35:28.997503+0000","last_undegraded":"2023-10-26T01:35:28.997503+0000","last_fullsized":"2023-10-26T01:35:28.997503+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:05:52.032045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.214564+0000","last_change":"2023-10-26T01:35:28.011509+0000","last_active":"2023-10-26T01:35:29.214564+0000","last_peered":"2023-10-26T01:35:29.214564+0000","last_clean":"2023-10-26T01:35:29.214564+0000","last_became_active":"2023-10-26T01:35:28.010898+0000","last_became_peered":"2023-10-26T01:35:28.010898+0000","last_unstale":"2023-10-26T01:35:29.214564+0000","last_undegraded":"2023-10-26T01:35:29.214564+0000","last_fullsized":"2023-10-26T01:35:29.214564+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:10:26.499805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:28.995305+0000","last_change":"2023-10-26T01:35:28.004530+0000","last_active":"2023-10-26T01:35:28.995305+0000","last_peered":"2023-10-26T01:35:28.995305+0000","last_clean":"2023-10-26T01:35:28.995305+0000","last_became_active":"2023-10-26T01:35:28.004099+0000","last_became_peered":"2023-10-26T01:35:28.004099+0000","last_unstale":"2023-10-26T01:35:28.995305+0000","last_undegraded":"2023-10-26T01:35:28.995305+0000","last_fullsized":"2023-10-26T01:35:28.995305+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:44:24.881425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.216331+0000","last_change":"2023-10-26T01:35:28.004622+0000","last_active":"2023-10-26T01:35:29.216331+0000","last_peered":"2023-10-26T01:35:29.216331+0000","last_clean":"2023-10-26T01:35:29.216331+0000","last_became_active":"2023-10-26T01:35:28.004090+0000","last_became_peered":"2023-10-26T01:35:28.004090+0000","last_unstale":"2023-10-26T01:35:29.216331+0000","last_undegraded":"2023-10-26T01:35:29.216331+0000","last_fullsized":"2023-10-26T01:35:29.216331+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:46:51.824429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.216205+0000","last_change":"2023-10-26T01:35:28.009726+0000","last_active":"2023-10-26T01:35:29.216205+0000","last_peered":"2023-10-26T01:35:29.216205+0000","last_clean":"2023-10-26T01:35:29.216205+0000","last_became_active":"2023-10-26T01:35:28.009274+0000","last_became_peered":"2023-10-26T01:35:28.009274+0000","last_unstale":"2023-10-26T01:35:29.216205+0000","last_undegraded":"2023-10-26T01:35:29.216205+0000","last_fullsized":"2023-10-26T01:35:29.216205+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:14:14.105927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.216080+0000","last_change":"2023-10-26T01:35:28.009483+0000","last_active":"2023-10-26T01:35:29.216080+0000","last_peered":"2023-10-26T01:35:29.216080+0000","last_clean":"2023-10-26T01:35:29.216080+0000","last_became_active":"2023-10-26T01:35:28.009021+0000","last_became_peered":"2023-10-26T01:35:28.009021+0000","last_unstale":"2023-10-26T01:35:29.216080+0000","last_undegraded":"2023-10-26T01:35:29.216080+0000","last_fullsized":"2023-10-26T01:35:29.216080+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:06:54.676358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.214239+0000","last_change":"2023-10-26T01:35:28.010589+0000","last_active":"2023-10-26T01:35:29.214239+0000","last_peered":"2023-10-26T01:35:29.214239+0000","last_clean":"2023-10-26T01:35:29.214239+0000","last_became_active":"2023-10-26T01:35:28.009970+0000","last_became_peered":"2023-10-26T01:35:28.009970+0000","last_unstale":"2023-10-26T01:35:29.214239+0000","last_undegraded":"2023-10-26T01:35:29.214239+0000","last_fullsized":"2023-10-26T01:35:29.214239+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:54:51.431715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.215619+0000","last_change":"2023-10-26T01:35:28.012565+0000","last_active":"2023-10-26T01:35:29.215619+0000","last_peered":"2023-10-26T01:35:29.215619+0000","last_clean":"2023-10-26T01:35:29.215619+0000","last_became_active":"2023-10-26T01:35:28.011885+0000","last_became_peered":"2023-10-26T01:35:28.011885+0000","last_unstale":"2023-10-26T01:35:29.215619+0000","last_undegraded":"2023-10-26T01:35:29.215619+0000","last_fullsized":"2023-10-26T01:35:29.215619+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:12:53.392033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.213687+0000","last_change":"2023-10-26T01:35:28.009370+0000","last_active":"2023-10-26T01:35:29.213687+0000","last_peered":"2023-10-26T01:35:29.213687+0000","last_clean":"2023-10-26T01:35:29.213687+0000","last_became_active":"2023-10-26T01:35:28.009002+0000","last_became_peered":"2023-10-26T01:35:28.009002+0000","last_unstale":"2023-10-26T01:35:29.213687+0000","last_undegraded":"2023-10-26T01:35:29.213687+0000","last_fullsized":"2023-10-26T01:35:29.213687+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:59:58.766446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:28.995924+0000","last_change":"2023-10-26T01:35:28.010322+0000","last_active":"2023-10-26T01:35:28.995924+0000","last_peered":"2023-10-26T01:35:28.995924+0000","last_clean":"2023-10-26T01:35:28.995924+0000","last_became_active":"2023-10-26T01:35:28.010056+0000","last_became_peered":"2023-10-26T01:35:28.010056+0000","last_unstale":"2023-10-26T01:35:28.995924+0000","last_undegraded":"2023-10-26T01:35:28.995924+0000","last_fullsized":"2023-10-26T01:35:28.995924+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:49:39.347860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:28.997955+0000","last_change":"2023-10-26T01:35:28.005984+0000","last_active":"2023-10-26T01:35:28.997955+0000","last_peered":"2023-10-26T01:35:28.997955+0000","last_clean":"2023-10-26T01:35:28.997955+0000","last_became_active":"2023-10-26T01:35:28.005409+0000","last_became_peered":"2023-10-26T01:35:28.005409+0000","last_unstale":"2023-10-26T01:35:28.997955+0000","last_undegraded":"2023-10-26T01:35:28.997955+0000","last_fullsized":"2023-10-26T01:35:28.997955+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:45:12.037351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.213966+0000","last_change":"2023-10-26T01:35:28.014813+0000","last_active":"2023-10-26T01:35:29.213966+0000","last_peered":"2023-10-26T01:35:29.213966+0000","last_clean":"2023-10-26T01:35:29.213966+0000","last_became_active":"2023-10-26T01:35:28.014182+0000","last_became_peered":"2023-10-26T01:35:28.014182+0000","last_unstale":"2023-10-26T01:35:29.213966+0000","last_undegraded":"2023-10-26T01:35:29.213966+0000","last_fullsized":"2023-10-26T01:35:29.213966+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:50:07.224712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.214308+0000","last_change":"2023-10-26T01:35:28.013781+0000","last_active":"2023-10-26T01:35:29.214308+0000","last_peered":"2023-10-26T01:35:29.214308+0000","last_clean":"2023-10-26T01:35:29.214308+0000","last_became_active":"2023-10-26T01:35:28.013069+0000","last_became_peered":"2023-10-26T01:35:28.013069+0000","last_unstale":"2023-10-26T01:35:29.214308+0000","last_undegraded":"2023-10-26T01:35:29.214308+0000","last_fullsized":"2023-10-26T01:35:29.214308+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:59:03.753748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.214011+0000","last_change":"2023-10-26T01:35:28.007382+0000","last_active":"2023-10-26T01:35:29.214011+0000","last_peered":"2023-10-26T01:35:29.214011+0000","last_clean":"2023-10-26T01:35:29.214011+0000","last_became_active":"2023-10-26T01:35:28.006605+0000","last_became_peered":"2023-10-26T01:35:28.006605+0000","last_unstale":"2023-10-26T01:35:29.214011+0000","last_undegraded":"2023-10-26T01:35:29.214011+0000","last_fullsized":"2023-10-26T01:35:29.214011+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:24:25.478994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.216357+0000","last_change":"2023-10-26T01:35:28.009743+0000","last_active":"2023-10-26T01:35:29.216357+0000","last_peered":"2023-10-26T01:35:29.216357+0000","last_clean":"2023-10-26T01:35:29.216357+0000","last_became_active":"2023-10-26T01:35:28.009410+0000","last_became_peered":"2023-10-26T01:35:28.009410+0000","last_unstale":"2023-10-26T01:35:29.216357+0000","last_undegraded":"2023-10-26T01:35:29.216357+0000","last_fullsized":"2023-10-26T01:35:29.216357+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:40:50.370204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:28.995621+0000","last_change":"2023-10-26T01:35:28.007395+0000","last_active":"2023-10-26T01:35:28.995621+0000","last_peered":"2023-10-26T01:35:28.995621+0000","last_clean":"2023-10-26T01:35:28.995621+0000","last_became_active":"2023-10-26T01:35:28.006533+0000","last_became_peered":"2023-10-26T01:35:28.006533+0000","last_unstale":"2023-10-26T01:35:28.995621+0000","last_undegraded":"2023-10-26T01:35:28.995621+0000","last_fullsized":"2023-10-26T01:35:28.995621+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:44:27.457808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.215118+0000","last_change":"2023-10-26T01:35:28.013728+0000","last_active":"2023-10-26T01:35:29.215118+0000","last_peered":"2023-10-26T01:35:29.215118+0000","last_clean":"2023-10-26T01:35:29.215118+0000","last_became_active":"2023-10-26T01:35:28.013257+0000","last_became_peered":"2023-10-26T01:35:28.013257+0000","last_unstale":"2023-10-26T01:35:29.215118+0000","last_undegraded":"2023-10-26T01:35:29.215118+0000","last_fullsized":"2023-10-26T01:35:29.215118+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:39:04.699624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.213605+0000","last_change":"2023-10-26T01:35:28.014854+0000","last_active":"2023-10-26T01:35:29.213605+0000","last_peered":"2023-10-26T01:35:29.213605+0000","last_clean":"2023-10-26T01:35:29.213605+0000","last_became_active":"2023-10-26T01:35:28.014435+0000","last_became_peered":"2023-10-26T01:35:28.014435+0000","last_unstale":"2023-10-26T01:35:29.213605+0000","last_undegraded":"2023-10-26T01:35:29.213605+0000","last_fullsized":"2023-10-26T01:35:29.213605+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:11:49.236285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.214583+0000","last_change":"2023-10-26T01:35:28.013054+0000","last_active":"2023-10-26T01:35:29.214583+0000","last_peered":"2023-10-26T01:35:29.214583+0000","last_clean":"2023-10-26T01:35:29.214583+0000","last_became_active":"2023-10-26T01:35:28.011914+0000","last_became_peered":"2023-10-26T01:35:28.011914+0000","last_unstale":"2023-10-26T01:35:29.214583+0000","last_undegraded":"2023-10-26T01:35:29.214583+0000","last_fullsized":"2023-10-26T01:35:29.214583+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:58:11.147240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.216710+0000","last_change":"2023-10-26T01:35:28.004658+0000","last_active":"2023-10-26T01:35:29.216710+0000","last_peered":"2023-10-26T01:35:29.216710+0000","last_clean":"2023-10-26T01:35:29.216710+0000","last_became_active":"2023-10-26T01:35:28.003773+0000","last_became_peered":"2023-10-26T01:35:28.003773+0000","last_unstale":"2023-10-26T01:35:29.216710+0000","last_undegraded":"2023-10-26T01:35:29.216710+0000","last_fullsized":"2023-10-26T01:35:29.216710+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:17:46.328218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:28.996098+0000","last_change":"2023-10-26T01:35:28.002723+0000","last_active":"2023-10-26T01:35:28.996098+0000","last_peered":"2023-10-26T01:35:28.996098+0000","last_clean":"2023-10-26T01:35:28.996098+0000","last_became_active":"2023-10-26T01:35:28.002231+0000","last_became_peered":"2023-10-26T01:35:28.002231+0000","last_unstale":"2023-10-26T01:35:28.996098+0000","last_undegraded":"2023-10-26T01:35:28.996098+0000","last_fullsized":"2023-10-26T01:35:28.996098+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:47:22.912447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:28.995662+0000","last_change":"2023-10-26T01:35:28.011690+0000","last_active":"2023-10-26T01:35:28.995662+0000","last_peered":"2023-10-26T01:35:28.995662+0000","last_clean":"2023-10-26T01:35:28.995662+0000","last_became_active":"2023-10-26T01:35:28.011424+0000","last_became_peered":"2023-10-26T01:35:28.011424+0000","last_unstale":"2023-10-26T01:35:28.995662+0000","last_undegraded":"2023-10-26T01:35:28.995662+0000","last_fullsized":"2023-10-26T01:35:28.995662+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:10:55.997083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:28.996782+0000","last_change":"2023-10-26T01:35:27.999889+0000","last_active":"2023-10-26T01:35:28.996782+0000","last_peered":"2023-10-26T01:35:28.996782+0000","last_clean":"2023-10-26T01:35:28.996782+0000","last_became_active":"2023-10-26T01:35:27.999521+0000","last_became_peered":"2023-10-26T01:35:27.999521+0000","last_unstale":"2023-10-26T01:35:28.996782+0000","last_undegraded":"2023-10-26T01:35:28.996782+0000","last_fullsized":"2023-10-26T01:35:28.996782+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:04:11.794309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:28.994867+0000","last_change":"2023-10-26T01:35:28.014763+0000","last_active":"2023-10-26T01:35:28.994867+0000","last_peered":"2023-10-26T01:35:28.994867+0000","last_clean":"2023-10-26T01:35:28.994867+0000","last_became_active":"2023-10-26T01:35:28.014538+0000","last_became_peered":"2023-10-26T01:35:28.014538+0000","last_unstale":"2023-10-26T01:35:28.994867+0000","last_undegraded":"2023-10-26T01:35:28.994867+0000","last_fullsized":"2023-10-26T01:35:28.994867+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:41:28.638341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:28.996239+0000","last_change":"2023-10-26T01:35:27.998285+0000","last_active":"2023-10-26T01:35:28.996239+0000","last_peered":"2023-10-26T01:35:28.996239+0000","last_clean":"2023-10-26T01:35:28.996239+0000","last_became_active":"2023-10-26T01:35:27.997577+0000","last_became_peered":"2023-10-26T01:35:27.997577+0000","last_unstale":"2023-10-26T01:35:28.996239+0000","last_undegraded":"2023-10-26T01:35:28.996239+0000","last_fullsized":"2023-10-26T01:35:28.996239+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:16:47.008792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.216023+0000","last_change":"2023-10-26T01:35:28.012518+0000","last_active":"2023-10-26T01:35:29.216023+0000","last_peered":"2023-10-26T01:35:29.216023+0000","last_clean":"2023-10-26T01:35:29.216023+0000","last_became_active":"2023-10-26T01:35:28.012254+0000","last_became_peered":"2023-10-26T01:35:28.012254+0000","last_unstale":"2023-10-26T01:35:29.216023+0000","last_undegraded":"2023-10-26T01:35:29.216023+0000","last_fullsized":"2023-10-26T01:35:29.216023+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:41:18.798894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:28.994945+0000","last_change":"2023-10-26T01:35:27.999138+0000","last_active":"2023-10-26T01:35:28.994945+0000","last_peered":"2023-10-26T01:35:28.994945+0000","last_clean":"2023-10-26T01:35:28.994945+0000","last_became_active":"2023-10-26T01:35:27.998526+0000","last_became_peered":"2023-10-26T01:35:27.998526+0000","last_unstale":"2023-10-26T01:35:28.994945+0000","last_undegraded":"2023-10-26T01:35:28.994945+0000","last_fullsized":"2023-10-26T01:35:28.994945+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:21:07.307330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:28.994315+0000","last_change":"2023-10-26T01:35:28.012297+0000","last_active":"2023-10-26T01:35:28.994315+0000","last_peered":"2023-10-26T01:35:28.994315+0000","last_clean":"2023-10-26T01:35:28.994315+0000","last_became_active":"2023-10-26T01:35:28.011776+0000","last_became_peered":"2023-10-26T01:35:28.011776+0000","last_unstale":"2023-10-26T01:35:28.994315+0000","last_undegraded":"2023-10-26T01:35:28.994315+0000","last_fullsized":"2023-10-26T01:35:28.994315+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:58:10.505901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:28.995265+0000","last_change":"2023-10-26T01:35:28.003936+0000","last_active":"2023-10-26T01:35:28.995265+0000","last_peered":"2023-10-26T01:35:28.995265+0000","last_clean":"2023-10-26T01:35:28.995265+0000","last_became_active":"2023-10-26T01:35:28.003324+0000","last_became_peered":"2023-10-26T01:35:28.003324+0000","last_unstale":"2023-10-26T01:35:28.995265+0000","last_undegraded":"2023-10-26T01:35:28.995265+0000","last_fullsized":"2023-10-26T01:35:28.995265+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:04:03.749653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.214530+0000","last_change":"2023-10-26T01:35:28.007835+0000","last_active":"2023-10-26T01:35:29.214530+0000","last_peered":"2023-10-26T01:35:29.214530+0000","last_clean":"2023-10-26T01:35:29.214530+0000","last_became_active":"2023-10-26T01:35:28.006302+0000","last_became_peered":"2023-10-26T01:35:28.006302+0000","last_unstale":"2023-10-26T01:35:29.214530+0000","last_undegraded":"2023-10-26T01:35:29.214530+0000","last_fullsized":"2023-10-26T01:35:29.214530+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:39:30.070804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.216345+0000","last_change":"2023-10-26T01:35:28.011170+0000","last_active":"2023-10-26T01:35:29.216345+0000","last_peered":"2023-10-26T01:35:29.216345+0000","last_clean":"2023-10-26T01:35:29.216345+0000","last_became_active":"2023-10-26T01:35:28.010863+0000","last_became_peered":"2023-10-26T01:35:28.010863+0000","last_unstale":"2023-10-26T01:35:29.216345+0000","last_undegraded":"2023-10-26T01:35:29.216345+0000","last_fullsized":"2023-10-26T01:35:29.216345+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:18:23.257036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.214921+0000","last_change":"2023-10-26T01:35:28.011689+0000","last_active":"2023-10-26T01:35:29.214921+0000","last_peered":"2023-10-26T01:35:29.214921+0000","last_clean":"2023-10-26T01:35:29.214921+0000","last_became_active":"2023-10-26T01:35:28.011266+0000","last_became_peered":"2023-10-26T01:35:28.011266+0000","last_unstale":"2023-10-26T01:35:29.214921+0000","last_undegraded":"2023-10-26T01:35:29.214921+0000","last_fullsized":"2023-10-26T01:35:29.214921+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:52:35.198913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:28.995195+0000","last_change":"2023-10-26T01:35:28.014275+0000","last_active":"2023-10-26T01:35:28.995195+0000","last_peered":"2023-10-26T01:35:28.995195+0000","last_clean":"2023-10-26T01:35:28.995195+0000","last_became_active":"2023-10-26T01:35:28.014053+0000","last_became_peered":"2023-10-26T01:35:28.014053+0000","last_unstale":"2023-10-26T01:35:28.995195+0000","last_undegraded":"2023-10-26T01:35:28.995195+0000","last_fullsized":"2023-10-26T01:35:28.995195+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:50:40.176544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"83'4","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:28.997021+0000","last_change":"2023-10-26T01:35:27.378062+0000","last_active":"2023-10-26T01:35:28.997021+0000","last_peered":"2023-10-26T01:35:28.997021+0000","last_clean":"2023-10-26T01:35:28.997021+0000","last_became_active":"2023-10-26T01:35:27.377281+0000","last_became_peered":"2023-10-26T01:35:27.377281+0000","last_unstale":"2023-10-26T01:35:28.997021+0000","last_undegraded":"2023-10-26T01:35:28.997021+0000","last_fullsized":"2023-10-26T01:35:28.997021+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_clean_scrub_stamp":"2023-10-26T01:35:26.337183+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:26:19.488470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.214119+0000","last_change":"2023-10-26T01:35:28.008559+0000","last_active":"2023-10-26T01:35:29.214119+0000","last_peered":"2023-10-26T01:35:29.214119+0000","last_clean":"2023-10-26T01:35:29.214119+0000","last_became_active":"2023-10-26T01:35:28.007917+0000","last_became_peered":"2023-10-26T01:35:28.007917+0000","last_unstale":"2023-10-26T01:35:29.214119+0000","last_undegraded":"2023-10-26T01:35:29.214119+0000","last_fullsized":"2023-10-26T01:35:29.214119+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:05:34.133886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"83'1","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.216930+0000","last_change":"2023-10-26T01:35:27.378724+0000","last_active":"2023-10-26T01:35:29.216930+0000","last_peered":"2023-10-26T01:35:29.216930+0000","last_clean":"2023-10-26T01:35:29.216930+0000","last_became_active":"2023-10-26T01:35:27.378341+0000","last_became_peered":"2023-10-26T01:35:27.378341+0000","last_unstale":"2023-10-26T01:35:29.216930+0000","last_undegraded":"2023-10-26T01:35:29.216930+0000","last_fullsized":"2023-10-26T01:35:29.216930+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_clean_scrub_stamp":"2023-10-26T01:35:26.337183+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:57:29.050189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.216392+0000","last_change":"2023-10-26T01:35:28.012857+0000","last_active":"2023-10-26T01:35:29.216392+0000","last_peered":"2023-10-26T01:35:29.216392+0000","last_clean":"2023-10-26T01:35:29.216392+0000","last_became_active":"2023-10-26T01:35:28.012160+0000","last_became_peered":"2023-10-26T01:35:28.012160+0000","last_unstale":"2023-10-26T01:35:29.216392+0000","last_undegraded":"2023-10-26T01:35:29.216392+0000","last_fullsized":"2023-10-26T01:35:29.216392+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:58:10.505901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"83'2","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:28.996748+0000","last_change":"2023-10-26T01:35:27.376547+0000","last_active":"2023-10-26T01:35:28.996748+0000","last_peered":"2023-10-26T01:35:28.996748+0000","last_clean":"2023-10-26T01:35:28.996748+0000","last_became_active":"2023-10-26T01:35:27.375661+0000","last_became_peered":"2023-10-26T01:35:27.375661+0000","last_unstale":"2023-10-26T01:35:28.996748+0000","last_undegraded":"2023-10-26T01:35:28.996748+0000","last_fullsized":"2023-10-26T01:35:28.996748+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_clean_scrub_stamp":"2023-10-26T01:35:26.337183+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:12:52.379303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:28.996769+0000","last_change":"2023-10-26T01:35:27.999847+0000","last_active":"2023-10-26T01:35:28.996769+0000","last_peered":"2023-10-26T01:35:28.996769+0000","last_clean":"2023-10-26T01:35:28.996769+0000","last_became_active":"2023-10-26T01:35:27.999354+0000","last_became_peered":"2023-10-26T01:35:27.999354+0000","last_unstale":"2023-10-26T01:35:28.996769+0000","last_undegraded":"2023-10-26T01:35:28.996769+0000","last_fullsized":"2023-10-26T01:35:28.996769+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:57:47.827646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"83'2","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.216832+0000","last_change":"2023-10-26T01:35:27.388698+0000","last_active":"2023-10-26T01:35:29.216832+0000","last_peered":"2023-10-26T01:35:29.216832+0000","last_clean":"2023-10-26T01:35:29.216832+0000","last_became_active":"2023-10-26T01:35:27.387595+0000","last_became_peered":"2023-10-26T01:35:27.387595+0000","last_unstale":"2023-10-26T01:35:29.216832+0000","last_undegraded":"2023-10-26T01:35:29.216832+0000","last_fullsized":"2023-10-26T01:35:29.216832+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_clean_scrub_stamp":"2023-10-26T01:35:26.337183+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:16:57.595672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.214172+0000","last_change":"2023-10-26T01:35:28.009258+0000","last_active":"2023-10-26T01:35:29.214172+0000","last_peered":"2023-10-26T01:35:29.214172+0000","last_clean":"2023-10-26T01:35:29.214172+0000","last_became_active":"2023-10-26T01:35:28.008895+0000","last_became_peered":"2023-10-26T01:35:28.008895+0000","last_unstale":"2023-10-26T01:35:29.214172+0000","last_undegraded":"2023-10-26T01:35:29.214172+0000","last_fullsized":"2023-10-26T01:35:29.214172+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:03:13.152836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"83'1","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T01:35:28.407749+0000","last_change":"2023-10-26T01:35:27.384835+0000","last_active":"2023-10-26T01:35:28.407749+0000","last_peered":"2023-10-26T01:35:28.407749+0000","last_clean":"2023-10-26T01:35:28.407749+0000","last_became_active":"2023-10-26T01:35:27.384560+0000","last_became_peered":"2023-10-26T01:35:27.384560+0000","last_unstale":"2023-10-26T01:35:28.407749+0000","last_undegraded":"2023-10-26T01:35:28.407749+0000","last_fullsized":"2023-10-26T01:35:28.407749+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_clean_scrub_stamp":"2023-10-26T01:35:26.337183+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:46:20.561796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:28.996010+0000","last_change":"2023-10-26T01:35:28.000055+0000","last_active":"2023-10-26T01:35:28.996010+0000","last_peered":"2023-10-26T01:35:28.996010+0000","last_clean":"2023-10-26T01:35:28.996010+0000","last_became_active":"2023-10-26T01:35:27.996858+0000","last_became_peered":"2023-10-26T01:35:27.996858+0000","last_unstale":"2023-10-26T01:35:28.996010+0000","last_undegraded":"2023-10-26T01:35:28.996010+0000","last_fullsized":"2023-10-26T01:35:28.996010+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:41:39.097519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"83'1","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.216164+0000","last_change":"2023-10-26T01:35:27.386727+0000","last_active":"2023-10-26T01:35:29.216164+0000","last_peered":"2023-10-26T01:35:29.216164+0000","last_clean":"2023-10-26T01:35:29.216164+0000","last_became_active":"2023-10-26T01:35:27.386318+0000","last_became_peered":"2023-10-26T01:35:27.386318+0000","last_unstale":"2023-10-26T01:35:29.216164+0000","last_undegraded":"2023-10-26T01:35:29.216164+0000","last_fullsized":"2023-10-26T01:35:29.216164+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_clean_scrub_stamp":"2023-10-26T01:35:26.337183+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:30:09.768728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.216708+0000","last_change":"2023-10-26T01:35:28.006993+0000","last_active":"2023-10-26T01:35:29.216708+0000","last_peered":"2023-10-26T01:35:29.216708+0000","last_clean":"2023-10-26T01:35:29.216708+0000","last_became_active":"2023-10-26T01:35:28.006476+0000","last_became_peered":"2023-10-26T01:35:28.006476+0000","last_unstale":"2023-10-26T01:35:29.216708+0000","last_undegraded":"2023-10-26T01:35:29.216708+0000","last_fullsized":"2023-10-26T01:35:29.216708+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:15:43.068280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.216889+0000","last_change":"2023-10-26T01:35:27.376474+0000","last_active":"2023-10-26T01:35:29.216889+0000","last_peered":"2023-10-26T01:35:29.216889+0000","last_clean":"2023-10-26T01:35:29.216889+0000","last_became_active":"2023-10-26T01:35:27.376103+0000","last_became_peered":"2023-10-26T01:35:27.376103+0000","last_unstale":"2023-10-26T01:35:29.216889+0000","last_undegraded":"2023-10-26T01:35:29.216889+0000","last_fullsized":"2023-10-26T01:35:29.216889+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_clean_scrub_stamp":"2023-10-26T01:35:26.337183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:32:37.976894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.216684+0000","last_change":"2023-10-26T01:35:27.999618+0000","last_active":"2023-10-26T01:35:29.216684+0000","last_peered":"2023-10-26T01:35:29.216684+0000","last_clean":"2023-10-26T01:35:29.216684+0000","last_became_active":"2023-10-26T01:35:27.996868+0000","last_became_peered":"2023-10-26T01:35:27.996868+0000","last_unstale":"2023-10-26T01:35:29.216684+0000","last_undegraded":"2023-10-26T01:35:29.216684+0000","last_fullsized":"2023-10-26T01:35:29.216684+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:04:23.200706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.217061+0000","last_change":"2023-10-26T01:35:27.377997+0000","last_active":"2023-10-26T01:35:29.217061+0000","last_peered":"2023-10-26T01:35:29.217061+0000","last_clean":"2023-10-26T01:35:29.217061+0000","last_became_active":"2023-10-26T01:35:27.376180+0000","last_became_peered":"2023-10-26T01:35:27.376180+0000","last_unstale":"2023-10-26T01:35:29.217061+0000","last_undegraded":"2023-10-26T01:35:29.217061+0000","last_fullsized":"2023-10-26T01:35:29.217061+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_clean_scrub_stamp":"2023-10-26T01:35:26.337183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:10:28.925306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.214645+0000","last_change":"2023-10-26T01:35:28.005939+0000","last_active":"2023-10-26T01:35:29.214645+0000","last_peered":"2023-10-26T01:35:29.214645+0000","last_clean":"2023-10-26T01:35:29.214645+0000","last_became_active":"2023-10-26T01:35:28.005566+0000","last_became_peered":"2023-10-26T01:35:28.005566+0000","last_unstale":"2023-10-26T01:35:29.214645+0000","last_undegraded":"2023-10-26T01:35:29.214645+0000","last_fullsized":"2023-10-26T01:35:29.214645+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T01:49:24.955549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:28.996318+0000","last_change":"2023-10-26T01:35:27.382676+0000","last_active":"2023-10-26T01:35:28.996318+0000","last_peered":"2023-10-26T01:35:28.996318+0000","last_clean":"2023-10-26T01:35:28.996318+0000","last_became_active":"2023-10-26T01:35:27.378175+0000","last_became_peered":"2023-10-26T01:35:27.378175+0000","last_unstale":"2023-10-26T01:35:28.996318+0000","last_undegraded":"2023-10-26T01:35:28.996318+0000","last_fullsized":"2023-10-26T01:35:28.996318+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_clean_scrub_stamp":"2023-10-26T01:35:26.337183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:06:53.663628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:28.998119+0000","last_change":"2023-10-26T01:35:28.015414+0000","last_active":"2023-10-26T01:35:28.998119+0000","last_peered":"2023-10-26T01:35:28.998119+0000","last_clean":"2023-10-26T01:35:28.998119+0000","last_became_active":"2023-10-26T01:35:28.014670+0000","last_became_peered":"2023-10-26T01:35:28.014670+0000","last_unstale":"2023-10-26T01:35:28.998119+0000","last_undegraded":"2023-10-26T01:35:28.998119+0000","last_fullsized":"2023-10-26T01:35:28.998119+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:59:43.451135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.215162+0000","last_change":"2023-10-26T01:35:27.386158+0000","last_active":"2023-10-26T01:35:29.215162+0000","last_peered":"2023-10-26T01:35:29.215162+0000","last_clean":"2023-10-26T01:35:29.215162+0000","last_became_active":"2023-10-26T01:35:27.385865+0000","last_became_peered":"2023-10-26T01:35:27.385865+0000","last_unstale":"2023-10-26T01:35:29.215162+0000","last_undegraded":"2023-10-26T01:35:29.215162+0000","last_fullsized":"2023-10-26T01:35:29.215162+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_clean_scrub_stamp":"2023-10-26T01:35:26.337183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:11:48.223555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.214799+0000","last_change":"2023-10-26T01:35:28.012681+0000","last_active":"2023-10-26T01:35:29.214799+0000","last_peered":"2023-10-26T01:35:29.214799+0000","last_clean":"2023-10-26T01:35:29.214799+0000","last_became_active":"2023-10-26T01:35:28.010972+0000","last_became_peered":"2023-10-26T01:35:28.010972+0000","last_unstale":"2023-10-26T01:35:29.214799+0000","last_undegraded":"2023-10-26T01:35:29.214799+0000","last_fullsized":"2023-10-26T01:35:29.214799+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:03:19.603679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"83'3","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T01:35:28.412662+0000","last_change":"2023-10-26T01:35:27.384617+0000","last_active":"2023-10-26T01:35:28.412662+0000","last_peered":"2023-10-26T01:35:28.412662+0000","last_clean":"2023-10-26T01:35:28.412662+0000","last_became_active":"2023-10-26T01:35:27.384299+0000","last_became_peered":"2023-10-26T01:35:27.384299+0000","last_unstale":"2023-10-26T01:35:28.412662+0000","last_undegraded":"2023-10-26T01:35:28.412662+0000","last_fullsized":"2023-10-26T01:35:28.412662+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_clean_scrub_stamp":"2023-10-26T01:35:26.337183+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:52:21.850204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.214446+0000","last_change":"2023-10-26T01:35:28.008261+0000","last_active":"2023-10-26T01:35:29.214446+0000","last_peered":"2023-10-26T01:35:29.214446+0000","last_clean":"2023-10-26T01:35:29.214446+0000","last_became_active":"2023-10-26T01:35:28.007031+0000","last_became_peered":"2023-10-26T01:35:28.007031+0000","last_unstale":"2023-10-26T01:35:29.214446+0000","last_undegraded":"2023-10-26T01:35:29.214446+0000","last_fullsized":"2023-10-26T01:35:29.214446+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:08:41.001574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"83'6","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.217269+0000","last_change":"2023-10-26T01:35:27.374334+0000","last_active":"2023-10-26T01:35:29.217269+0000","last_peered":"2023-10-26T01:35:29.217269+0000","last_clean":"2023-10-26T01:35:29.217269+0000","last_became_active":"2023-10-26T01:35:27.372877+0000","last_became_peered":"2023-10-26T01:35:27.372877+0000","last_unstale":"2023-10-26T01:35:29.217269+0000","last_undegraded":"2023-10-26T01:35:29.217269+0000","last_fullsized":"2023-10-26T01:35:29.217269+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_clean_scrub_stamp":"2023-10-26T01:35:26.337183+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:02:33.998135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:28.998277+0000","last_change":"2023-10-26T01:35:28.009106+0000","last_active":"2023-10-26T01:35:28.998277+0000","last_peered":"2023-10-26T01:35:28.998277+0000","last_clean":"2023-10-26T01:35:28.998277+0000","last_became_active":"2023-10-26T01:35:28.008430+0000","last_became_peered":"2023-10-26T01:35:28.008430+0000","last_unstale":"2023-10-26T01:35:28.998277+0000","last_undegraded":"2023-10-26T01:35:28.998277+0000","last_fullsized":"2023-10-26T01:35:28.998277+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:53:25.094797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"83'2","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.214403+0000","last_change":"2023-10-26T01:35:27.378502+0000","last_active":"2023-10-26T01:35:29.214403+0000","last_peered":"2023-10-26T01:35:29.214403+0000","last_clean":"2023-10-26T01:35:29.214403+0000","last_became_active":"2023-10-26T01:35:27.378247+0000","last_became_peered":"2023-10-26T01:35:27.378247+0000","last_unstale":"2023-10-26T01:35:29.214403+0000","last_undegraded":"2023-10-26T01:35:29.214403+0000","last_fullsized":"2023-10-26T01:35:29.214403+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_clean_scrub_stamp":"2023-10-26T01:35:26.337183+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T01:51:53.290088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.214851+0000","last_change":"2023-10-26T01:35:28.008044+0000","last_active":"2023-10-26T01:35:29.214851+0000","last_peered":"2023-10-26T01:35:29.214851+0000","last_clean":"2023-10-26T01:35:29.214851+0000","last_became_active":"2023-10-26T01:35:28.007421+0000","last_became_peered":"2023-10-26T01:35:28.007421+0000","last_unstale":"2023-10-26T01:35:29.214851+0000","last_undegraded":"2023-10-26T01:35:29.214851+0000","last_fullsized":"2023-10-26T01:35:29.214851+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:17:13.966369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.216938+0000","last_change":"2023-10-26T01:35:27.388474+0000","last_active":"2023-10-26T01:35:29.216938+0000","last_peered":"2023-10-26T01:35:29.216938+0000","last_clean":"2023-10-26T01:35:29.216938+0000","last_became_active":"2023-10-26T01:35:27.387040+0000","last_became_peered":"2023-10-26T01:35:27.387040+0000","last_unstale":"2023-10-26T01:35:29.216938+0000","last_undegraded":"2023-10-26T01:35:29.216938+0000","last_fullsized":"2023-10-26T01:35:29.216938+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_clean_scrub_stamp":"2023-10-26T01:35:26.337183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:47:51.280157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.214625+0000","last_change":"2023-10-26T01:35:28.014358+0000","last_active":"2023-10-26T01:35:29.214625+0000","last_peered":"2023-10-26T01:35:29.214625+0000","last_clean":"2023-10-26T01:35:29.214625+0000","last_became_active":"2023-10-26T01:35:28.014074+0000","last_became_peered":"2023-10-26T01:35:28.014074+0000","last_unstale":"2023-10-26T01:35:29.214625+0000","last_undegraded":"2023-10-26T01:35:29.214625+0000","last_fullsized":"2023-10-26T01:35:29.214625+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:05:37.048290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.215191+0000","last_change":"2023-10-26T01:35:27.369340+0000","last_active":"2023-10-26T01:35:29.215191+0000","last_peered":"2023-10-26T01:35:29.215191+0000","last_clean":"2023-10-26T01:35:29.215191+0000","last_became_active":"2023-10-26T01:35:27.368595+0000","last_became_peered":"2023-10-26T01:35:27.368595+0000","last_unstale":"2023-10-26T01:35:29.215191+0000","last_undegraded":"2023-10-26T01:35:29.215191+0000","last_fullsized":"2023-10-26T01:35:29.215191+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_clean_scrub_stamp":"2023-10-26T01:35:26.337183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:27:04.115099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.215423+0000","last_change":"2023-10-26T01:35:28.012065+0000","last_active":"2023-10-26T01:35:29.215423+0000","last_peered":"2023-10-26T01:35:29.215423+0000","last_clean":"2023-10-26T01:35:29.215423+0000","last_became_active":"2023-10-26T01:35:28.011702+0000","last_became_peered":"2023-10-26T01:35:28.011702+0000","last_unstale":"2023-10-26T01:35:29.215423+0000","last_undegraded":"2023-10-26T01:35:29.215423+0000","last_fullsized":"2023-10-26T01:35:29.215423+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:16:18.626077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"83'8","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:28.994596+0000","last_change":"2023-10-26T01:35:27.382105+0000","last_active":"2023-10-26T01:35:28.994596+0000","last_peered":"2023-10-26T01:35:28.994596+0000","last_clean":"2023-10-26T01:35:28.994596+0000","last_became_active":"2023-10-26T01:35:27.380841+0000","last_became_peered":"2023-10-26T01:35:27.380841+0000","last_unstale":"2023-10-26T01:35:28.994596+0000","last_undegraded":"2023-10-26T01:35:28.994596+0000","last_fullsized":"2023-10-26T01:35:28.994596+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_clean_scrub_stamp":"2023-10-26T01:35:26.337183+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:57:48.871856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:28.996552+0000","last_change":"2023-10-26T01:35:28.004115+0000","last_active":"2023-10-26T01:35:28.996552+0000","last_peered":"2023-10-26T01:35:28.996552+0000","last_clean":"2023-10-26T01:35:28.996552+0000","last_became_active":"2023-10-26T01:35:28.003137+0000","last_became_peered":"2023-10-26T01:35:28.003137+0000","last_unstale":"2023-10-26T01:35:28.996552+0000","last_undegraded":"2023-10-26T01:35:28.996552+0000","last_fullsized":"2023-10-26T01:35:28.996552+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:23:00.412684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"83'1","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.215017+0000","last_change":"2023-10-26T01:35:27.386064+0000","last_active":"2023-10-26T01:35:29.215017+0000","last_peered":"2023-10-26T01:35:29.215017+0000","last_clean":"2023-10-26T01:35:29.215017+0000","last_became_active":"2023-10-26T01:35:27.385758+0000","last_became_peered":"2023-10-26T01:35:27.385758+0000","last_unstale":"2023-10-26T01:35:29.215017+0000","last_undegraded":"2023-10-26T01:35:29.215017+0000","last_fullsized":"2023-10-26T01:35:29.215017+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_clean_scrub_stamp":"2023-10-26T01:35:26.337183+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:50:06.211982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:28.996804+0000","last_change":"2023-10-26T01:35:27.997556+0000","last_active":"2023-10-26T01:35:28.996804+0000","last_peered":"2023-10-26T01:35:28.996804+0000","last_clean":"2023-10-26T01:35:28.996804+0000","last_became_active":"2023-10-26T01:35:27.996932+0000","last_became_peered":"2023-10-26T01:35:27.996932+0000","last_unstale":"2023-10-26T01:35:28.996804+0000","last_undegraded":"2023-10-26T01:35:28.996804+0000","last_fullsized":"2023-10-26T01:35:28.996804+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T01:59:40.284419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"83'2","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:28.996943+0000","last_change":"2023-10-26T01:35:27.376227+0000","last_active":"2023-10-26T01:35:28.996943+0000","last_peered":"2023-10-26T01:35:28.996943+0000","last_clean":"2023-10-26T01:35:28.996943+0000","last_became_active":"2023-10-26T01:35:27.375091+0000","last_became_peered":"2023-10-26T01:35:27.375091+0000","last_unstale":"2023-10-26T01:35:28.996943+0000","last_undegraded":"2023-10-26T01:35:28.996943+0000","last_fullsized":"2023-10-26T01:35:28.996943+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_clean_scrub_stamp":"2023-10-26T01:35:26.337183+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:39:51.002944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.214088+0000","last_change":"2023-10-26T01:35:28.015000+0000","last_active":"2023-10-26T01:35:29.214088+0000","last_peered":"2023-10-26T01:35:29.214088+0000","last_clean":"2023-10-26T01:35:29.214088+0000","last_became_active":"2023-10-26T01:35:28.014563+0000","last_became_peered":"2023-10-26T01:35:28.014563+0000","last_unstale":"2023-10-26T01:35:29.214088+0000","last_undegraded":"2023-10-26T01:35:29.214088+0000","last_fullsized":"2023-10-26T01:35:29.214088+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:29:03.673854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"83'1","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.217147+0000","last_change":"2023-10-26T01:35:27.388852+0000","last_active":"2023-10-26T01:35:29.217147+0000","last_peered":"2023-10-26T01:35:29.217147+0000","last_clean":"2023-10-26T01:35:29.217147+0000","last_became_active":"2023-10-26T01:35:27.388231+0000","last_became_peered":"2023-10-26T01:35:27.388231+0000","last_unstale":"2023-10-26T01:35:29.217147+0000","last_undegraded":"2023-10-26T01:35:29.217147+0000","last_fullsized":"2023-10-26T01:35:29.217147+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_clean_scrub_stamp":"2023-10-26T01:35:26.337183+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:21:38.139916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.215707+0000","last_change":"2023-10-26T01:35:28.006049+0000","last_active":"2023-10-26T01:35:29.215707+0000","last_peered":"2023-10-26T01:35:29.215707+0000","last_clean":"2023-10-26T01:35:29.215707+0000","last_became_active":"2023-10-26T01:35:28.005583+0000","last_became_peered":"2023-10-26T01:35:28.005583+0000","last_unstale":"2023-10-26T01:35:29.215707+0000","last_undegraded":"2023-10-26T01:35:29.215707+0000","last_fullsized":"2023-10-26T01:35:29.215707+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:17:45.838484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:28.996675+0000","last_change":"2023-10-26T01:35:27.384945+0000","last_active":"2023-10-26T01:35:28.996675+0000","last_peered":"2023-10-26T01:35:28.996675+0000","last_clean":"2023-10-26T01:35:28.996675+0000","last_became_active":"2023-10-26T01:35:27.384581+0000","last_became_peered":"2023-10-26T01:35:27.384581+0000","last_unstale":"2023-10-26T01:35:28.996675+0000","last_undegraded":"2023-10-26T01:35:28.996675+0000","last_fullsized":"2023-10-26T01:35:28.996675+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_clean_scrub_stamp":"2023-10-26T01:35:26.337183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:54:50.418985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:28.995605+0000","last_change":"2023-10-26T01:35:27.997899+0000","last_active":"2023-10-26T01:35:28.995605+0000","last_peered":"2023-10-26T01:35:28.995605+0000","last_clean":"2023-10-26T01:35:28.995605+0000","last_became_active":"2023-10-26T01:35:27.997225+0000","last_became_peered":"2023-10-26T01:35:27.997225+0000","last_unstale":"2023-10-26T01:35:28.995605+0000","last_undegraded":"2023-10-26T01:35:28.995605+0000","last_fullsized":"2023-10-26T01:35:28.995605+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:28:19.850875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.215140+0000","last_change":"2023-10-26T01:35:27.385581+0000","last_active":"2023-10-26T01:35:29.215140+0000","last_peered":"2023-10-26T01:35:29.215140+0000","last_clean":"2023-10-26T01:35:29.215140+0000","last_became_active":"2023-10-26T01:35:27.382991+0000","last_became_peered":"2023-10-26T01:35:27.382991+0000","last_unstale":"2023-10-26T01:35:29.215140+0000","last_undegraded":"2023-10-26T01:35:29.215140+0000","last_fullsized":"2023-10-26T01:35:29.215140+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_clean_scrub_stamp":"2023-10-26T01:35:26.337183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:29:02.661124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:28.995222+0000","last_change":"2023-10-26T01:35:28.014715+0000","last_active":"2023-10-26T01:35:28.995222+0000","last_peered":"2023-10-26T01:35:28.995222+0000","last_clean":"2023-10-26T01:35:28.995222+0000","last_became_active":"2023-10-26T01:35:28.014454+0000","last_became_peered":"2023-10-26T01:35:28.014454+0000","last_unstale":"2023-10-26T01:35:28.995222+0000","last_undegraded":"2023-10-26T01:35:28.995222+0000","last_fullsized":"2023-10-26T01:35:28.995222+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:49:11.256897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.215076+0000","last_change":"2023-10-26T01:35:27.384315+0000","last_active":"2023-10-26T01:35:29.215076+0000","last_peered":"2023-10-26T01:35:29.215076+0000","last_clean":"2023-10-26T01:35:29.215076+0000","last_became_active":"2023-10-26T01:35:27.383731+0000","last_became_peered":"2023-10-26T01:35:27.383731+0000","last_unstale":"2023-10-26T01:35:29.215076+0000","last_undegraded":"2023-10-26T01:35:29.215076+0000","last_fullsized":"2023-10-26T01:35:29.215076+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_clean_scrub_stamp":"2023-10-26T01:35:26.337183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:01:54.686020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:28.995027+0000","last_change":"2023-10-26T01:35:28.009584+0000","last_active":"2023-10-26T01:35:28.995027+0000","last_peered":"2023-10-26T01:35:28.995027+0000","last_clean":"2023-10-26T01:35:28.995027+0000","last_became_active":"2023-10-26T01:35:28.008557+0000","last_became_peered":"2023-10-26T01:35:28.008557+0000","last_unstale":"2023-10-26T01:35:28.995027+0000","last_undegraded":"2023-10-26T01:35:28.995027+0000","last_fullsized":"2023-10-26T01:35:28.995027+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:14:00.199313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.215763+0000","last_change":"2023-10-26T01:35:27.377085+0000","last_active":"2023-10-26T01:35:29.215763+0000","last_peered":"2023-10-26T01:35:29.215763+0000","last_clean":"2023-10-26T01:35:29.215763+0000","last_became_active":"2023-10-26T01:35:27.376815+0000","last_became_peered":"2023-10-26T01:35:27.376815+0000","last_unstale":"2023-10-26T01:35:29.215763+0000","last_undegraded":"2023-10-26T01:35:29.215763+0000","last_fullsized":"2023-10-26T01:35:29.215763+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_clean_scrub_stamp":"2023-10-26T01:35:26.337183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:44:59.108027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.214974+0000","last_change":"2023-10-26T01:35:28.011211+0000","last_active":"2023-10-26T01:35:29.214974+0000","last_peered":"2023-10-26T01:35:29.214974+0000","last_clean":"2023-10-26T01:35:29.214974+0000","last_became_active":"2023-10-26T01:35:28.010899+0000","last_became_peered":"2023-10-26T01:35:28.010899+0000","last_unstale":"2023-10-26T01:35:29.214974+0000","last_undegraded":"2023-10-26T01:35:29.214974+0000","last_fullsized":"2023-10-26T01:35:29.214974+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:11:54.879258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"83'2","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.215414+0000","last_change":"2023-10-26T01:35:27.382652+0000","last_active":"2023-10-26T01:35:29.215414+0000","last_peered":"2023-10-26T01:35:29.215414+0000","last_clean":"2023-10-26T01:35:29.215414+0000","last_became_active":"2023-10-26T01:35:27.382362+0000","last_became_peered":"2023-10-26T01:35:27.382362+0000","last_unstale":"2023-10-26T01:35:29.215414+0000","last_undegraded":"2023-10-26T01:35:29.215414+0000","last_fullsized":"2023-10-26T01:35:29.215414+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_clean_scrub_stamp":"2023-10-26T01:35:26.337183+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:57:48.871856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.214979+0000","last_change":"2023-10-26T01:35:28.010351+0000","last_active":"2023-10-26T01:35:29.214979+0000","last_peered":"2023-10-26T01:35:29.214979+0000","last_clean":"2023-10-26T01:35:29.214979+0000","last_became_active":"2023-10-26T01:35:28.009508+0000","last_became_peered":"2023-10-26T01:35:28.009508+0000","last_unstale":"2023-10-26T01:35:29.214979+0000","last_undegraded":"2023-10-26T01:35:29.214979+0000","last_fullsized":"2023-10-26T01:35:29.214979+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:34:13.502328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.214585+0000","last_change":"2023-10-26T01:35:27.382069+0000","last_active":"2023-10-26T01:35:29.214585+0000","last_peered":"2023-10-26T01:35:29.214585+0000","last_clean":"2023-10-26T01:35:29.214585+0000","last_became_active":"2023-10-26T01:35:27.371481+0000","last_became_peered":"2023-10-26T01:35:27.371481+0000","last_unstale":"2023-10-26T01:35:29.214585+0000","last_undegraded":"2023-10-26T01:35:29.214585+0000","last_fullsized":"2023-10-26T01:35:29.214585+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_clean_scrub_stamp":"2023-10-26T01:35:26.337183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:25:32.799627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:28.997111+0000","last_change":"2023-10-26T01:35:28.005065+0000","last_active":"2023-10-26T01:35:28.997111+0000","last_peered":"2023-10-26T01:35:28.997111+0000","last_clean":"2023-10-26T01:35:28.997111+0000","last_became_active":"2023-10-26T01:35:28.004636+0000","last_became_peered":"2023-10-26T01:35:28.004636+0000","last_unstale":"2023-10-26T01:35:28.997111+0000","last_undegraded":"2023-10-26T01:35:28.997111+0000","last_fullsized":"2023-10-26T01:35:28.997111+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:33:16.474796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.214213+0000","last_change":"2023-10-26T01:35:27.380061+0000","last_active":"2023-10-26T01:35:29.214213+0000","last_peered":"2023-10-26T01:35:29.214213+0000","last_clean":"2023-10-26T01:35:29.214213+0000","last_became_active":"2023-10-26T01:35:27.379816+0000","last_became_peered":"2023-10-26T01:35:27.379816+0000","last_unstale":"2023-10-26T01:35:29.214213+0000","last_undegraded":"2023-10-26T01:35:29.214213+0000","last_fullsized":"2023-10-26T01:35:29.214213+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_clean_scrub_stamp":"2023-10-26T01:35:26.337183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:19:36.012636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:28.997157+0000","last_change":"2023-10-26T01:35:27.995695+0000","last_active":"2023-10-26T01:35:28.997157+0000","last_peered":"2023-10-26T01:35:28.997157+0000","last_clean":"2023-10-26T01:35:28.997157+0000","last_became_active":"2023-10-26T01:35:27.995082+0000","last_became_peered":"2023-10-26T01:35:27.995082+0000","last_unstale":"2023-10-26T01:35:28.997157+0000","last_undegraded":"2023-10-26T01:35:28.997157+0000","last_fullsized":"2023-10-26T01:35:28.997157+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:18:23.792256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.216952+0000","last_change":"2023-10-26T01:35:27.374935+0000","last_active":"2023-10-26T01:35:29.216952+0000","last_peered":"2023-10-26T01:35:29.216952+0000","last_clean":"2023-10-26T01:35:29.216952+0000","last_became_active":"2023-10-26T01:35:27.374477+0000","last_became_peered":"2023-10-26T01:35:27.374477+0000","last_unstale":"2023-10-26T01:35:29.216952+0000","last_undegraded":"2023-10-26T01:35:29.216952+0000","last_fullsized":"2023-10-26T01:35:29.216952+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_clean_scrub_stamp":"2023-10-26T01:35:26.337183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:30:10.457948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.215771+0000","last_change":"2023-10-26T01:35:28.010475+0000","last_active":"2023-10-26T01:35:29.215771+0000","last_peered":"2023-10-26T01:35:29.215771+0000","last_clean":"2023-10-26T01:35:29.215771+0000","last_became_active":"2023-10-26T01:35:28.010174+0000","last_became_peered":"2023-10-26T01:35:28.010174+0000","last_unstale":"2023-10-26T01:35:29.215771+0000","last_undegraded":"2023-10-26T01:35:29.215771+0000","last_fullsized":"2023-10-26T01:35:29.215771+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T01:41:09.390504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.215935+0000","last_change":"2023-10-26T01:35:27.380771+0000","last_active":"2023-10-26T01:35:29.215935+0000","last_peered":"2023-10-26T01:35:29.215935+0000","last_clean":"2023-10-26T01:35:29.215935+0000","last_became_active":"2023-10-26T01:35:27.380526+0000","last_became_peered":"2023-10-26T01:35:27.380526+0000","last_unstale":"2023-10-26T01:35:29.215935+0000","last_undegraded":"2023-10-26T01:35:29.215935+0000","last_fullsized":"2023-10-26T01:35:29.215935+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_clean_scrub_stamp":"2023-10-26T01:35:26.337183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:25:00.697915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.216709+0000","last_change":"2023-10-26T01:35:28.012709+0000","last_active":"2023-10-26T01:35:29.216709+0000","last_peered":"2023-10-26T01:35:29.216709+0000","last_clean":"2023-10-26T01:35:29.216709+0000","last_became_active":"2023-10-26T01:35:28.010687+0000","last_became_peered":"2023-10-26T01:35:28.010687+0000","last_unstale":"2023-10-26T01:35:29.216709+0000","last_undegraded":"2023-10-26T01:35:29.216709+0000","last_fullsized":"2023-10-26T01:35:29.216709+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:44:27.457808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.214166+0000","last_change":"2023-10-26T01:35:27.382781+0000","last_active":"2023-10-26T01:35:29.214166+0000","last_peered":"2023-10-26T01:35:29.214166+0000","last_clean":"2023-10-26T01:35:29.214166+0000","last_became_active":"2023-10-26T01:35:27.382235+0000","last_became_peered":"2023-10-26T01:35:27.382235+0000","last_unstale":"2023-10-26T01:35:29.214166+0000","last_undegraded":"2023-10-26T01:35:29.214166+0000","last_fullsized":"2023-10-26T01:35:29.214166+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_clean_scrub_stamp":"2023-10-26T01:35:26.337183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:13:37.237113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.216675+0000","last_change":"2023-10-26T01:35:28.001107+0000","last_active":"2023-10-26T01:35:29.216675+0000","last_peered":"2023-10-26T01:35:29.216675+0000","last_clean":"2023-10-26T01:35:29.216675+0000","last_became_active":"2023-10-26T01:35:28.000115+0000","last_became_peered":"2023-10-26T01:35:28.000115+0000","last_unstale":"2023-10-26T01:35:29.216675+0000","last_undegraded":"2023-10-26T01:35:29.216675+0000","last_fullsized":"2023-10-26T01:35:29.216675+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:14:38.641246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:28.994817+0000","last_change":"2023-10-26T01:35:27.381156+0000","last_active":"2023-10-26T01:35:28.994817+0000","last_peered":"2023-10-26T01:35:28.994817+0000","last_clean":"2023-10-26T01:35:28.994817+0000","last_became_active":"2023-10-26T01:35:27.380359+0000","last_became_peered":"2023-10-26T01:35:27.380359+0000","last_unstale":"2023-10-26T01:35:28.994817+0000","last_undegraded":"2023-10-26T01:35:28.994817+0000","last_fullsized":"2023-10-26T01:35:28.994817+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_clean_scrub_stamp":"2023-10-26T01:35:26.337183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:30:09.768728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.215930+0000","last_change":"2023-10-26T01:35:28.013975+0000","last_active":"2023-10-26T01:35:29.215930+0000","last_peered":"2023-10-26T01:35:29.215930+0000","last_clean":"2023-10-26T01:35:29.215930+0000","last_became_active":"2023-10-26T01:35:28.013716+0000","last_became_peered":"2023-10-26T01:35:28.013716+0000","last_unstale":"2023-10-26T01:35:29.215930+0000","last_undegraded":"2023-10-26T01:35:29.215930+0000","last_fullsized":"2023-10-26T01:35:29.215930+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:39:52.015674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.215256+0000","last_change":"2023-10-26T01:35:27.385396+0000","last_active":"2023-10-26T01:35:29.215256+0000","last_peered":"2023-10-26T01:35:29.215256+0000","last_clean":"2023-10-26T01:35:29.215256+0000","last_became_active":"2023-10-26T01:35:27.382472+0000","last_became_peered":"2023-10-26T01:35:27.382472+0000","last_unstale":"2023-10-26T01:35:29.215256+0000","last_undegraded":"2023-10-26T01:35:29.215256+0000","last_fullsized":"2023-10-26T01:35:29.215256+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_clean_scrub_stamp":"2023-10-26T01:35:26.337183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:46:50.811699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:28.995569+0000","last_change":"2023-10-26T01:35:28.008065+0000","last_active":"2023-10-26T01:35:28.995569+0000","last_peered":"2023-10-26T01:35:28.995569+0000","last_clean":"2023-10-26T01:35:28.995569+0000","last_became_active":"2023-10-26T01:35:28.007721+0000","last_became_peered":"2023-10-26T01:35:28.007721+0000","last_unstale":"2023-10-26T01:35:28.995569+0000","last_undegraded":"2023-10-26T01:35:28.995569+0000","last_fullsized":"2023-10-26T01:35:28.995569+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:27.349913+0000","last_clean_scrub_stamp":"2023-10-26T01:35:27.349913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:58:55.944315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"83'2","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-26T01:35:29.216590+0000","last_change":"2023-10-26T01:35:27.386574+0000","last_active":"2023-10-26T01:35:29.216590+0000","last_peered":"2023-10-26T01:35:29.216590+0000","last_clean":"2023-10-26T01:35:29.216590+0000","last_became_active":"2023-10-26T01:35:27.385743+0000","last_became_peered":"2023-10-26T01:35:27.385743+0000","last_unstale":"2023-10-26T01:35:29.216590+0000","last_undegraded":"2023-10-26T01:35:29.216590+0000","last_fullsized":"2023-10-26T01:35:29.216590+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:35:26.337183+0000","last_clean_scrub_stamp":"2023-10-26T01:35:26.337183+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:27:20.180507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":229,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-10-26T01:35:27.987210+0000","last_change":"2023-10-26T01:30:34.883484+0000","last_active":"2023-10-26T01:35:27.987210+0000","last_peered":"2023-10-26T01:35:27.987210+0000","last_clean":"2023-10-26T01:35:27.987210+0000","last_became_active":"2023-10-26T01:30:34.883071+0000","last_became_peered":"2023-10-26T01:30:34.883071+0000","last_unstale":"2023-10-26T01:35:27.987210+0000","last_undegraded":"2023-10-26T01:35:27.987210+0000","last_fullsized":"2023-10-26T01:35:27.987210+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:32.890096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:32.890096+0000","last_clean_scrub_stamp":"2023-10-26T01:30:32.890096+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:25:16.321641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":14,"seq":60129542207,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":433,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194878,"statfs":{"total":200000000,"available":199555618,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:34:43 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.80500000000000005,"15min":0.78900000000000003},"min":{"1min":0.628,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.365,"5min":2.2549999999999999,"15min":2.2549999999999999},"last":0.82599999999999996},{"interface":"front","average":{"1min":1.004,"5min":0.86599999999999999,"15min":0.89400000000000002},"min":{"1min":0.67500000000000004,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.5800000000000001,"5min":1.8200000000000001,"15min":1.8200000000000001},"last":0.94399999999999995}]},{"osd":1,"last update":"Thu Oct 26 01:34:43 2023","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":0.89800000000000002,"15min":0.96399999999999997},"min":{"1min":0.52000000000000002,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.444,"5min":4.4340000000000002,"15min":4.4340000000000002},"last":1.0820000000000001},{"interface":"front","average":{"1min":1.048,"5min":0.98199999999999998,"15min":1.006},"min":{"1min":0.81399999999999995,"5min":0.374,"15min":0.374},"max":{"1min":1.538,"5min":2.3330000000000002,"15min":2.3330000000000002},"last":1.119}]},{"osd":2,"last update":"Thu Oct 26 01:34:43 2023","interfaces":[{"interface":"back","average":{"1min":1.071,"5min":1.0589999999999999,"15min":1.097},"min":{"1min":0.59099999999999997,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.3779999999999999,"5min":4.2800000000000002,"15min":4.2800000000000002},"last":0.71799999999999997},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.1080000000000001,"15min":1.087},"min":{"1min":0.746,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.659,"5min":2.302,"15min":2.302},"last":1.3080000000000001}]},{"osd":4,"last update":"Thu Oct 26 01:34:43 2023","interfaces":[{"interface":"back","average":{"1min":1.2749999999999999,"5min":1.1699999999999999,"15min":1.1559999999999999},"min":{"1min":0.96099999999999997,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.5960000000000001,"5min":2.1070000000000002,"15min":2.1070000000000002},"last":1.252},{"interface":"front","average":{"1min":1.405,"5min":1.3660000000000001,"15min":1.411},"min":{"1min":0.92200000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":1.5940000000000001,"5min":4.8250000000000002,"15min":4.8250000000000002},"last":1.379}]},{"osd":5,"last update":"Thu Oct 26 01:34:43 2023","interfaces":[{"interface":"back","average":{"1min":1.343,"5min":1.302,"15min":1.355},"min":{"1min":0.85799999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.9530000000000001,"5min":4.9080000000000004,"15min":4.9080000000000004},"last":1.484},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.264,"15min":1.2749999999999999},"min":{"1min":0.98299999999999998,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.706,"5min":2.8980000000000001,"15min":2.8980000000000001},"last":1.196}]},{"osd":6,"last update":"Thu Oct 26 01:34:43 2023","interfaces":[{"interface":"back","average":{"1min":1.3959999999999999,"5min":1.3300000000000001,"15min":1.3240000000000001},"min":{"1min":1.0109999999999999,"5min":0.85099999999999998,"15min":0.85099999999999998},"max":{"1min":1.6499999999999999,"5min":2.8530000000000002,"15min":2.8530000000000002},"last":1.4139999999999999},{"interface":"front","average":{"1min":1.3180000000000001,"5min":1.244,"15min":1.2669999999999999},"min":{"1min":0.83699999999999997,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.694,"5min":2.181,"15min":2.181},"last":1.6160000000000001}]},{"osd":7,"last update":"Thu Oct 26 01:34:43 2023","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.284,"15min":1.2849999999999999},"min":{"1min":0.89300000000000002,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.778,"5min":2.0680000000000001,"15min":2.0680000000000001},"last":1.6879999999999999},{"interface":"front","average":{"1min":1.4590000000000001,"5min":1.3919999999999999,"15min":1.4299999999999999},"min":{"1min":0.90900000000000003,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.887,"5min":4.8730000000000002,"15min":4.8730000000000002},"last":1.577}]}]},{"osd":7,"up_from":13,"seq":55834574910,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1010,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194301,"statfs":{"total":200000000,"available":198964746,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:34:44 2023","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.0980000000000001,"15min":1.0760000000000001},"min":{"1min":0.752,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.5740000000000001,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":1.609},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.1279999999999999,"15min":1.075},"min":{"1min":0.80300000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":2.6779999999999999,"5min":2.6779999999999999,"15min":2.6779999999999999},"last":1.2589999999999999}]},{"osd":1,"last update":"Thu Oct 26 01:34:44 2023","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.155,"15min":1.0740000000000001},"min":{"1min":0.83299999999999996,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":2.8260000000000001,"5min":2.8260000000000001,"15min":2.8260000000000001},"last":1.2190000000000001},{"interface":"front","average":{"1min":1.3300000000000001,"5min":1.24,"15min":1.196},"min":{"1min":0.86399999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":3.2730000000000001,"5min":3.2730000000000001,"15min":3.2730000000000001},"last":0.84699999999999998}]},{"osd":2,"last update":"Thu Oct 26 01:34:44 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.1259999999999999,"15min":1.0980000000000001},"min":{"1min":0.85799999999999998,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":2.218,"5min":2.218,"15min":2.218},"last":1.54},{"interface":"front","average":{"1min":1.4279999999999999,"5min":1.2729999999999999,"15min":1.228},"min":{"1min":1.036,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":3.0510000000000002,"5min":3.0510000000000002,"15min":3.0510000000000002},"last":0.99399999999999999}]},{"osd":3,"last update":"Thu Oct 26 01:34:44 2023","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.2290000000000001,"15min":1.194},"min":{"1min":0.79000000000000004,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.9830000000000001,"5min":1.9830000000000001,"15min":1.9830000000000001},"last":1.133},{"interface":"front","average":{"1min":1.7170000000000001,"5min":1.4350000000000001,"15min":1.3819999999999999},"min":{"1min":0.89100000000000001,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":5.5039999999999996,"5min":5.5039999999999996,"15min":5.5039999999999996},"last":1.444}]},{"osd":4,"last update":"Thu Oct 26 01:34:44 2023","interfaces":[{"interface":"back","average":{"1min":1.425,"5min":1.353,"15min":1.347},"min":{"1min":0.93400000000000005,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":2.77,"5min":2.77,"15min":2.77},"last":1.3680000000000001},{"interface":"front","average":{"1min":1.546,"5min":1.4419999999999999,"15min":1.3999999999999999},"min":{"1min":1.0640000000000001,"5min":0.496,"15min":0.496},"max":{"1min":3.1419999999999999,"5min":3.1419999999999999,"15min":3.1419999999999999},"last":1.4850000000000001}]},{"osd":5,"last update":"Thu Oct 26 01:34:44 2023","interfaces":[{"interface":"back","average":{"1min":1.722,"5min":1.4910000000000001,"15min":1.425},"min":{"1min":1.0649999999999999,"5min":0.441,"15min":0.441},"max":{"1min":5.734,"5min":5.734,"15min":5.734},"last":1.639},{"interface":"front","average":{"1min":1.6040000000000001,"5min":1.4890000000000001,"15min":1.4690000000000001},"min":{"1min":1.113,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":2.8980000000000001,"5min":2.8980000000000001,"15min":2.8980000000000001},"last":1.883}]},{"osd":6,"last update":"Thu Oct 26 01:34:44 2023","interfaces":[{"interface":"back","average":{"1min":1.8129999999999999,"5min":1.7929999999999999,"15min":1.9399999999999999},"min":{"1min":1.175,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":5.6340000000000003,"5min":11.577999999999999,"15min":11.577999999999999},"last":1.8300000000000001},{"interface":"front","average":{"1min":1.863,"5min":1.8069999999999999,"15min":1.9370000000000001},"min":{"1min":1.1559999999999999,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":5.0919999999999996,"5min":10.704000000000001,"15min":10.704000000000001},"last":1.706}]}]},{"osd":6,"up_from":13,"seq":55834574910,"num_pgs":35,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":433,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194879,"statfs":{"total":200000000,"available":199556252,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:34:37 2023","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.135,"15min":1.173},"min":{"1min":0.64500000000000002,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":2.395,"5min":2.395,"15min":2.395},"last":1.462},{"interface":"front","average":{"1min":1.222,"5min":1.1020000000000001,"15min":1.111},"min":{"1min":0.86099999999999999,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.6910000000000001,"5min":1.843,"15min":1.843},"last":0.96499999999999997}]},{"osd":1,"last update":"Thu Oct 26 01:34:37 2023","interfaces":[{"interface":"back","average":{"1min":1.306,"5min":1.1559999999999999,"15min":1.169},"min":{"1min":0.77300000000000002,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":2.0529999999999999,"5min":2.0529999999999999,"15min":2.0529999999999999},"last":1.369},{"interface":"front","average":{"1min":1.3129999999999999,"5min":1.1839999999999999,"15min":1.212},"min":{"1min":0.82299999999999995,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.581,"5min":2.085,"15min":2.085},"last":1.331}]},{"osd":2,"last update":"Thu Oct 26 01:34:37 2023","interfaces":[{"interface":"back","average":{"1min":1.373,"5min":1.1319999999999999,"15min":1.0780000000000001},"min":{"1min":0.93000000000000005,"5min":0.747,"15min":0.747},"max":{"1min":3.7879999999999998,"5min":3.7879999999999998,"15min":3.7879999999999998},"last":1.4079999999999999},{"interface":"front","average":{"1min":1.3740000000000001,"5min":1.264,"15min":1.2529999999999999},"min":{"1min":0.79100000000000004,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":2.2890000000000001,"5min":2.2890000000000001,"15min":2.2890000000000001},"last":1.0269999999999999}]},{"osd":3,"last update":"Thu Oct 26 01:34:37 2023","interfaces":[{"interface":"back","average":{"1min":1.5269999999999999,"5min":1.2470000000000001,"15min":1.1870000000000001},"min":{"1min":0.92700000000000005,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":4.4560000000000004,"5min":4.4560000000000004,"15min":4.4560000000000004},"last":1.2370000000000001},{"interface":"front","average":{"1min":1.3979999999999999,"5min":1.329,"15min":1.323},"min":{"1min":0.82299999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":2.5019999999999998,"5min":2.5019999999999998,"15min":2.5019999999999998},"last":1.5569999999999999}]},{"osd":4,"last update":"Thu Oct 26 01:34:37 2023","interfaces":[{"interface":"back","average":{"1min":1.2869999999999999,"5min":1.2330000000000001,"15min":1.2130000000000001},"min":{"1min":0.63900000000000001,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":2.9489999999999998,"5min":2.9489999999999998,"15min":2.9489999999999998},"last":1.208},{"interface":"front","average":{"1min":1.7629999999999999,"5min":1.385,"15min":1.298},"min":{"1min":0.85899999999999999,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":6.609,"5min":6.609,"15min":6.609},"last":1.5269999999999999}]},{"osd":5,"last update":"Thu Oct 26 01:34:37 2023","interfaces":[{"interface":"back","average":{"1min":1.647,"5min":1.4790000000000001,"15min":1.4199999999999999},"min":{"1min":1.1739999999999999,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":5.4370000000000003,"5min":5.4370000000000003,"15min":5.4370000000000003},"last":1.589},{"interface":"front","average":{"1min":1.3919999999999999,"5min":1.415,"15min":1.3979999999999999},"min":{"1min":0.61399999999999999,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":2.621,"5min":2.621,"15min":2.621},"last":1.6160000000000001}]},{"osd":7,"last update":"Thu Oct 26 01:34:37 2023","interfaces":[{"interface":"back","average":{"1min":1.9199999999999999,"5min":1.49,"15min":1.3999999999999999},"min":{"1min":1.0029999999999999,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":11.804,"5min":11.804,"15min":11.804},"last":1.494},{"interface":"front","average":{"1min":2.0539999999999998,"5min":1.5920000000000001,"15min":1.484},"min":{"1min":1.1639999999999999,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":11.541,"5min":11.541,"15min":11.541},"last":1.9419999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574910,"num_pgs":14,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":433,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194878,"statfs":{"total":200000000,"available":199555640,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:34:43 2023","interfaces":[{"interface":"back","average":{"1min":1.3169999999999999,"5min":1.155,"15min":1.1180000000000001},"min":{"1min":0.88100000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.617,"5min":1.7390000000000001,"15min":1.7390000000000001},"last":0.70899999999999996},{"interface":"front","average":{"1min":1.4139999999999999,"5min":1.2130000000000001,"15min":1.1579999999999999},"min":{"1min":1.0309999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.6659999999999999,"5min":1.6659999999999999,"15min":1.6659999999999999},"last":1.248}]},{"osd":1,"last update":"Thu Oct 26 01:34:43 2023","interfaces":[{"interface":"back","average":{"1min":1.194,"5min":1.113,"15min":1.081},"min":{"1min":0.80000000000000004,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.504,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":1.333},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.1319999999999999,"15min":1.0860000000000001},"min":{"1min":0.80500000000000005,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.6000000000000001,"5min":1.7669999999999999,"15min":1.7669999999999999},"last":0.83399999999999996}]},{"osd":2,"last update":"Thu Oct 26 01:34:43 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.1719999999999999,"15min":1.151},"min":{"1min":0.84099999999999997,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.599,"5min":1.788,"15min":1.788},"last":1.097},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.1719999999999999,"15min":1.1639999999999999},"min":{"1min":0.85499999999999998,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.474,"5min":1.579,"15min":1.579},"last":0.98799999999999999}]},{"osd":3,"last update":"Thu Oct 26 01:34:43 2023","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.268,"15min":1.232},"min":{"1min":0.99099999999999999,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.536,"5min":1.835,"15min":1.835},"last":1.1759999999999999},{"interface":"front","average":{"1min":1.3560000000000001,"5min":1.3029999999999999,"15min":1.2889999999999999},"min":{"1min":1.048,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.7509999999999999,"5min":2.9249999999999998,"15min":2.9249999999999998},"last":1.381}]},{"osd":4,"last update":"Thu Oct 26 01:34:43 2023","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.242,"15min":1.1910000000000001},"min":{"1min":0.84799999999999998,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.579,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":1.0329999999999999},{"interface":"front","average":{"1min":1.371,"5min":1.3759999999999999,"15min":1.329},"min":{"1min":0.96699999999999997,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.764,"5min":1.867,"15min":1.867},"last":1.5700000000000001}]},{"osd":6,"last update":"Thu Oct 26 01:34:43 2023","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.3440000000000001,"15min":1.2889999999999999},"min":{"1min":0.97699999999999998,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.6439999999999999,"5min":1.8999999999999999,"15min":1.8999999999999999},"last":1.4390000000000001},{"interface":"front","average":{"1min":1.401,"5min":1.375,"15min":1.3009999999999999},"min":{"1min":1.0740000000000001,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.6850000000000001,"5min":1.8560000000000001,"15min":1.8560000000000001},"last":0.93700000000000006}]},{"osd":7,"last update":"Thu Oct 26 01:34:43 2023","interfaces":[{"interface":"back","average":{"1min":1.5269999999999999,"5min":1.462,"15min":1.3899999999999999},"min":{"1min":1.234,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.8999999999999999,"5min":1.97,"15min":1.97},"last":1.4850000000000001},{"interface":"front","average":{"1min":1.4850000000000001,"5min":1.4630000000000001,"15min":1.395},"min":{"1min":1.0600000000000001,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.901,"5min":2.0680000000000001,"15min":2.0680000000000001},"last":1.524}]}]},{"osd":4,"up_from":13,"seq":55834574910,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":433,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194878,"statfs":{"total":200000000,"available":199555660,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:34:34 2023","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":0.996,"15min":0.96199999999999997},"min":{"1min":0.71799999999999997,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.3080000000000001,"5min":2.0289999999999999,"15min":2.0289999999999999},"last":1.3620000000000001},{"interface":"front","average":{"1min":1.016,"5min":1.022,"15min":1.008},"min":{"1min":0.71099999999999997,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.363,"5min":2.6829999999999998,"15min":2.6829999999999998},"last":1.036}]},{"osd":1,"last update":"Thu Oct 26 01:34:34 2023","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":1.1299999999999999,"15min":1.1060000000000001},"min":{"1min":0.67800000000000005,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.5900000000000001,"5min":4.0519999999999996,"15min":4.0519999999999996},"last":1.1739999999999999},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.121,"15min":1.097},"min":{"1min":0.83799999999999997,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.6419999999999999,"5min":2.0760000000000001,"15min":2.0760000000000001},"last":1.202}]},{"osd":2,"last update":"Thu Oct 26 01:34:34 2023","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":1.2829999999999999,"15min":1.2070000000000001},"min":{"1min":0.91200000000000003,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.633,"5min":5.6749999999999998,"15min":5.6749999999999998},"last":1.2989999999999999},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.151,"15min":1.1100000000000001},"min":{"1min":0.751,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.849,"5min":1.849,"15min":1.849},"last":0.94499999999999995}]},{"osd":3,"last update":"Thu Oct 26 01:34:34 2023","interfaces":[{"interface":"back","average":{"1min":1.24,"5min":1.1859999999999999,"15min":1.179},"min":{"1min":0.87,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.5469999999999999,"5min":1.891,"15min":1.891},"last":0.83699999999999997},{"interface":"front","average":{"1min":1.208,"5min":1.167,"15min":1.143},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.681,"5min":1.7210000000000001,"15min":1.7210000000000001},"last":1.139}]},{"osd":5,"last update":"Thu Oct 26 01:34:34 2023","interfaces":[{"interface":"back","average":{"1min":1.2250000000000001,"5min":1.266,"15min":1.2909999999999999},"min":{"1min":0.73699999999999999,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.6140000000000001,"5min":2.0049999999999999,"15min":2.0049999999999999},"last":0.996},{"interface":"front","average":{"1min":1.2769999999999999,"5min":1.349,"15min":1.333},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.9810000000000001,"5min":5.8579999999999997,"15min":5.8579999999999997},"last":1.52}]},{"osd":6,"last update":"Thu Oct 26 01:34:34 2023","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":1.4339999999999999,"15min":1.3380000000000001},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.919,"5min":12.343999999999999,"15min":12.343999999999999},"last":1.2549999999999999},{"interface":"front","average":{"1min":1.5069999999999999,"5min":1.597,"15min":1.5049999999999999},"min":{"1min":0.78500000000000003,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.929,"5min":12.605,"15min":12.605},"last":1.097}]},{"osd":7,"last update":"Thu Oct 26 01:34:34 2023","interfaces":[{"interface":"back","average":{"1min":1.4770000000000001,"5min":1.5249999999999999,"15min":1.4930000000000001},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":2.089,"5min":5.774,"15min":5.774},"last":1.45},{"interface":"front","average":{"1min":1.5449999999999999,"5min":1.496,"15min":1.446},"min":{"1min":0.80600000000000005,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":1.972,"5min":4.6520000000000001,"15min":4.6520000000000001},"last":1.417}]}]},{"osd":2,"up_from":13,"seq":55834574910,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":435,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194877,"statfs":{"total":200000000,"available":199554118,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:34:35 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.85599999999999998,"15min":0.85399999999999998},"min":{"1min":0.67000000000000004,"5min":0.308,"15min":0.308},"max":{"1min":1.105,"5min":6.3200000000000003,"15min":6.3200000000000003},"last":0.79200000000000004},{"interface":"front","average":{"1min":1.0980000000000001,"5min":0.97199999999999998,"15min":0.91900000000000004},"min":{"1min":0.63700000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.361,"5min":4.2619999999999996,"15min":4.2619999999999996},"last":0.877}]},{"osd":1,"last update":"Thu Oct 26 01:34:35 2023","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.90800000000000003,"15min":0.89700000000000002},"min":{"1min":0.73799999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.5309999999999999,"5min":5.1260000000000003,"15min":5.1260000000000003},"last":1.1100000000000001},{"interface":"front","average":{"1min":1.103,"5min":1.028,"15min":1},"min":{"1min":0.80900000000000005,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.4259999999999999,"5min":6.1520000000000001,"15min":6.1520000000000001},"last":1.1299999999999999}]},{"osd":3,"last update":"Thu Oct 26 01:34:42 2023","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":0.97899999999999998,"15min":0.92200000000000004},"min":{"1min":0.63100000000000001,"5min":0.443,"15min":0.443},"max":{"1min":1.4830000000000001,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":1.004},{"interface":"front","average":{"1min":1.1319999999999999,"5min":1.0820000000000001,"15min":0.98999999999999999},"min":{"1min":0.89500000000000002,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.673,"5min":8.9019999999999992,"15min":8.9019999999999992},"last":1.0389999999999999}]},{"osd":4,"last update":"Thu Oct 26 01:34:35 2023","interfaces":[{"interface":"back","average":{"1min":1.419,"5min":1.2989999999999999,"15min":1.246},"min":{"1min":1.0760000000000001,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.774,"5min":6.2670000000000003,"15min":6.2670000000000003},"last":1.3300000000000001},{"interface":"front","average":{"1min":1.258,"5min":1.298,"15min":1.24},"min":{"1min":1.032,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":1.591,"5min":7.343,"15min":7.343},"last":1.2310000000000001}]},{"osd":5,"last update":"Thu Oct 26 01:34:35 2023","interfaces":[{"interface":"back","average":{"1min":1.389,"5min":1.296,"15min":1.266},"min":{"1min":1.1060000000000001,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.659,"5min":3.7170000000000001,"15min":3.7170000000000001},"last":1.1799999999999999},{"interface":"front","average":{"1min":1.363,"5min":1.3120000000000001,"15min":1.286},"min":{"1min":0.98399999999999999,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.5920000000000001,"5min":6.3899999999999997,"15min":6.3899999999999997},"last":1.2649999999999999}]},{"osd":6,"last update":"Thu Oct 26 01:34:35 2023","interfaces":[{"interface":"back","average":{"1min":1.4910000000000001,"5min":1.3720000000000001,"15min":1.3169999999999999},"min":{"1min":1,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.905,"5min":6.0490000000000004,"15min":6.0490000000000004},"last":1.0660000000000001},{"interface":"front","average":{"1min":1.411,"5min":1.3799999999999999,"15min":1.355},"min":{"1min":1.141,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.7509999999999999,"5min":5.0650000000000004,"15min":5.0650000000000004},"last":1.3069999999999999}]},{"osd":7,"last update":"Thu Oct 26 01:34:35 2023","interfaces":[{"interface":"back","average":{"1min":1.4850000000000001,"5min":1.415,"15min":1.397},"min":{"1min":1.1579999999999999,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.8839999999999999,"5min":6.4480000000000004,"15min":6.4480000000000004},"last":1.532},{"interface":"front","average":{"1min":1.5740000000000001,"5min":1.46,"15min":1.4379999999999999},"min":{"1min":1.1930000000000001,"5min":0.91800000000000004,"15min":0.91800000000000004},"max":{"1min":2.0070000000000001,"5min":2.8439999999999999,"15min":2.8439999999999999},"last":1.4399999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574910,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":436,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194876,"statfs":{"total":200000000,"available":199553502,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:34:38 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.751,"15min":0.71099999999999997},"min":{"1min":0.379,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.80000000000000004,"15min":0.77600000000000002},"min":{"1min":0.45000000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.405,"5min":1.4830000000000001,"15min":1.4830000000000001},"last":0.67500000000000004}]},{"osd":2,"last update":"Thu Oct 26 01:34:38 2023","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":0.97699999999999998,"15min":1.004},"min":{"1min":0.39700000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.4239999999999999,"5min":2.4830000000000001,"15min":2.4830000000000001},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.91400000000000003,"5min":0.86199999999999999,"15min":0.84899999999999998},"min":{"1min":0.57299999999999995,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.3500000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":1.0980000000000001}]},{"osd":3,"last update":"Thu Oct 26 01:34:42 2023","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":0.97399999999999998,"15min":0.94399999999999995},"min":{"1min":0.621,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.611,"5min":1.75,"15min":1.75},"last":1.1850000000000001},{"interface":"front","average":{"1min":1.05,"5min":0.99399999999999999,"15min":0.97799999999999998},"min":{"1min":0.47799999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.5389999999999999,"5min":1.802,"15min":1.802},"last":0.48799999999999999}]},{"osd":4,"last update":"Thu Oct 26 01:34:38 2023","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.214,"15min":1.1919999999999999},"min":{"1min":0.79200000000000004,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.605,"5min":2.0659999999999998,"15min":2.0659999999999998},"last":0.92600000000000005},{"interface":"front","average":{"1min":1.2470000000000001,"5min":1.153,"15min":1.123},"min":{"1min":0.95399999999999996,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.637,"5min":1.637,"15min":1.637},"last":0.76600000000000001}]},{"osd":5,"last update":"Thu Oct 26 01:34:38 2023","interfaces":[{"interface":"back","average":{"1min":1.3009999999999999,"5min":1.224,"15min":1.22},"min":{"1min":1.077,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":1.5620000000000001,"5min":1.702,"15min":1.702},"last":1.0549999999999999},{"interface":"front","average":{"1min":1.4079999999999999,"5min":1.331,"15min":1.329},"min":{"1min":0.84799999999999998,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.673,"5min":2.2200000000000002,"15min":2.2200000000000002},"last":1.085}]},{"osd":6,"last update":"Thu Oct 26 01:34:38 2023","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":1.278,"15min":1.2649999999999999},"min":{"1min":0.80900000000000005,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":2.0910000000000002,"5min":2.3780000000000001,"15min":2.3780000000000001},"last":0.90000000000000002},{"interface":"front","average":{"1min":1.274,"5min":1.252,"15min":1.234},"min":{"1min":0.91700000000000004,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.6679999999999999,"5min":1.984,"15min":1.984},"last":1.2390000000000001}]},{"osd":7,"last update":"Thu Oct 26 01:34:38 2023","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":1.25,"15min":1.264},"min":{"1min":0.81299999999999994,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.5700000000000001,"5min":2.6930000000000001,"15min":2.6930000000000001},"last":0.84199999999999997},{"interface":"front","average":{"1min":1.4350000000000001,"5min":1.3400000000000001,"15min":1.3149999999999999},"min":{"1min":0.89700000000000002,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.7709999999999999,"5min":1.869,"15min":1.869},"last":1.0329999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574910,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1010,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194302,"statfs":{"total":200000000,"available":198965362,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Oct 26 01:34:38 2023","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":1.046,"15min":0.93899999999999995},"min":{"1min":0.63600000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.5049999999999999,"5min":19.425000000000001,"15min":19.425000000000001},"last":0.39200000000000002},{"interface":"front","average":{"1min":1.05,"5min":1.125,"15min":1.0720000000000001},"min":{"1min":0.69599999999999995,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.452,"5min":15.162000000000001,"15min":15.162000000000001},"last":0.47299999999999998}]},{"osd":2,"last update":"Thu Oct 26 01:34:38 2023","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.3700000000000001,"15min":1.218},"min":{"1min":0.878,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.577,"5min":28.439,"15min":28.439},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.91800000000000004,"15min":0.91800000000000004},"min":{"1min":0.58099999999999996,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.3220000000000001,"5min":4.9459999999999997,"15min":4.9459999999999997},"last":0.67200000000000004}]},{"osd":3,"last update":"Thu Oct 26 01:34:38 2023","interfaces":[{"interface":"back","average":{"1min":1.167,"5min":1.085,"15min":1.105},"min":{"1min":0.86599999999999999,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.6479999999999999,"5min":4.9800000000000004,"15min":4.9800000000000004},"last":0.50800000000000001},{"interface":"front","average":{"1min":1.226,"5min":1.163,"15min":1.171},"min":{"1min":0.61099999999999999,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.6559999999999999,"5min":5.016,"15min":5.016},"last":0.72499999999999998}]},{"osd":4,"last update":"Thu Oct 26 01:34:38 2023","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.4159999999999999,"15min":1.3109999999999999},"min":{"1min":0.94499999999999995,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.6439999999999999,"5min":19.329000000000001,"15min":19.329000000000001},"last":1.2010000000000001},{"interface":"front","average":{"1min":1.3460000000000001,"5min":1.296,"15min":1.3140000000000001},"min":{"1min":1.054,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.6739999999999999,"5min":5.5229999999999997,"15min":5.5229999999999997},"last":0.997}]},{"osd":5,"last update":"Thu Oct 26 01:34:38 2023","interfaces":[{"interface":"back","average":{"1min":1.3979999999999999,"5min":1.4219999999999999,"15min":1.4410000000000001},"min":{"1min":1.1839999999999999,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.605,"5min":5.6360000000000001,"15min":5.6360000000000001},"last":0.97899999999999998},{"interface":"front","average":{"1min":1.349,"5min":1.534,"15min":1.4259999999999999},"min":{"1min":0.95799999999999996,"5min":0.626,"15min":0.626},"max":{"1min":1.7390000000000001,"5min":19.384,"15min":19.384},"last":0.78400000000000003}]},{"osd":6,"last update":"Thu Oct 26 01:34:38 2023","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.4159999999999999,"15min":1.3859999999999999},"min":{"1min":1.1639999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.653,"5min":8.2609999999999992,"15min":8.2609999999999992},"last":0.84499999999999997},{"interface":"front","average":{"1min":1.411,"5min":1.472,"15min":1.4219999999999999},"min":{"1min":1.143,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.702,"5min":11.849,"15min":11.849},"last":0.93899999999999995}]},{"osd":7,"last update":"Thu Oct 26 01:34:38 2023","interfaces":[{"interface":"back","average":{"1min":1.4650000000000001,"5min":1.4810000000000001,"15min":1.5},"min":{"1min":1.2430000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":1.7769999999999999,"5min":5.6689999999999996,"15min":5.6689999999999996},"last":1.0880000000000001},{"interface":"front","average":{"1min":1.4970000000000001,"5min":1.629,"15min":1.516},"min":{"1min":1.155,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.8169999999999999,"5min":19.245999999999999,"15min":19.245999999999999},"last":0.879}]}]}],"pool_statfs":[]}} 2023-10-26T01:35:33.356 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-26T01:35:33.356 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-26T01:35:33.625 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:33.622+0000 7f78baae5640 1 -- 172.21.15.169:0/3382990409 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f78b40b0c80 msgr2=0x7f78b40cd710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:33.625 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:33.622+0000 7f78baae5640 1 --2- 172.21.15.169:0/3382990409 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f78b40b0c80 0x7f78b40cd710 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f78ac005fd0 tx=0x7f78ac02be20 comp rx=0 tx=0).stop 2023-10-26T01:35:33.626 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:33.622+0000 7f78baae5640 1 -- 172.21.15.169:0/3382990409 shutdown_connections 2023-10-26T01:35:33.626 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:33.622+0000 7f78baae5640 1 --2- 172.21.15.169:0/3382990409 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f78b40b0c80 0x7f78b40cd710 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:33.626 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:33.622+0000 7f78baae5640 1 --2- 172.21.15.169:0/3382990409 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f78b41597c0 0x7f78b40ac200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:33.626 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:33.622+0000 7f78baae5640 1 -- 172.21.15.169:0/3382990409 >> 172.21.15.169:0/3382990409 conn(0x7f78b4069930 msgr2=0x7f78b4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:35:33.626 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:33.622+0000 7f78baae5640 1 -- 172.21.15.169:0/3382990409 shutdown_connections 2023-10-26T01:35:33.627 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:33.622+0000 7f78baae5640 1 -- 172.21.15.169:0/3382990409 wait complete. 2023-10-26T01:35:33.627 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:33.624+0000 7f78baae5640 1 Processor -- start 2023-10-26T01:35:33.627 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:33.624+0000 7f78baae5640 1 -- start start 2023-10-26T01:35:33.628 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:33.626+0000 7f78baae5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f78b40b0c80 0x7f78b41f1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:33.629 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:33.626+0000 7f78baae5640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f78b41593e0 0x7f78b40ae1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:33.629 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:33.626+0000 7f78baae5640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f78b41597c0 0x7f78b40ae730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:33.629 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:33.626+0000 7f78baae5640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78b4149c40 con 0x7f78b41597c0 2023-10-26T01:35:33.629 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:33.626+0000 7f78baae5640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78b40acf00 con 0x7f78b40b0c80 2023-10-26T01:35:33.629 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:33.626+0000 7f78baae5640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78b40add60 con 0x7f78b41593e0 2023-10-26T01:35:33.630 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:33.626+0000 7f78b9082640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f78b41597c0 0x7f78b40ae730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:33.630 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:33.626+0000 7f78b9082640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f78b41597c0 0x7f78b40ae730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:57038/0 (socket says 172.21.15.169:57038) 2023-10-26T01:35:33.630 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:33.626+0000 7f78b9082640 1 -- 172.21.15.169:0/4016908233 learned_addr learned my addr 172.21.15.169:0/4016908233 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:35:33.631 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:33.626+0000 7f78b3fff640 1 --2- 172.21.15.169:0/4016908233 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f78b41593e0 0x7f78b40ae1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:33.631 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:33.626+0000 7f78b9082640 1 -- 172.21.15.169:0/4016908233 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f78b41593e0 msgr2=0x7f78b40ae1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:33.632 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:33.626+0000 7f78b9082640 1 --2- 172.21.15.169:0/4016908233 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f78b41593e0 0x7f78b40ae1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:33.632 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:33.626+0000 7f78b9082640 1 -- 172.21.15.169:0/4016908233 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f78b40b0c80 msgr2=0x7f78b41f1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:33.632 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:33.626+0000 7f78b9082640 1 --2- 172.21.15.169:0/4016908233 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f78b40b0c80 0x7f78b41f1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:33.632 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:33.626+0000 7f78b9082640 1 -- 172.21.15.169:0/4016908233 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78ac009560 con 0x7f78b41597c0 2023-10-26T01:35:33.632 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:33.627+0000 7f78b9082640 1 --2- 172.21.15.169:0/4016908233 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f78b41597c0 0x7f78b40ae730 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f78ac009230 tx=0x7f78ac02db60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:35:33.633 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:33.627+0000 7f78b17fa640 1 -- 172.21.15.169:0/4016908233 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78ac003800 con 0x7f78b41597c0 2023-10-26T01:35:33.633 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:33.627+0000 7f78b17fa640 1 -- 172.21.15.169:0/4016908233 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f78ac003a50 con 0x7f78b41597c0 2023-10-26T01:35:33.633 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:33.627+0000 7f78baae5640 1 -- 172.21.15.169:0/4016908233 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78b40ad150 con 0x7f78b41597c0 2023-10-26T01:35:33.633 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:33.627+0000 7f78b17fa640 1 -- 172.21.15.169:0/4016908233 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78ac003e60 con 0x7f78b41597c0 2023-10-26T01:35:33.633 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:33.627+0000 7f78baae5640 1 -- 172.21.15.169:0/4016908233 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78b40ad710 con 0x7f78b41597c0 2023-10-26T01:35:33.634 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:33.628+0000 7f78baae5640 1 -- 172.21.15.169:0/4016908233 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78b4066e60 con 0x7f78b41597c0 2023-10-26T01:35:33.634 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:33.629+0000 7f78b17fa640 1 -- 172.21.15.169:0/4016908233 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f78ac02dd50 con 0x7f78b41597c0 2023-10-26T01:35:33.635 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:33.630+0000 7f78b17fa640 1 --2- 172.21.15.169:0/4016908233 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f789406ae40 0x7f789406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:33.635 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:33.630+0000 7f78b17fa640 1 -- 172.21.15.169:0/4016908233 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f78ac0abf20 con 0x7f78b41597c0 2023-10-26T01:35:33.635 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:33.631+0000 7f78b8881640 1 --2- 172.21.15.169:0/4016908233 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f789406ae40 0x7f789406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:33.635 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:33.632+0000 7f78b8881640 1 --2- 172.21.15.169:0/4016908233 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f789406ae40 0x7f789406d300 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f78a40012e0 tx=0x7f78a4007d30 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:35:33.637 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:33.635+0000 7f78b17fa640 1 -- 172.21.15.169:0/4016908233 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f78ac0770e0 con 0x7f78b41597c0 2023-10-26T01:35:33.902 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:33.899+0000 7f78baae5640 1 -- 172.21.15.169:0/4016908233 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f78b404db10 con 0x7f78b41597c0 2023-10-26T01:35:33.908 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:33.905+0000 7f78b17fa640 1 -- 172.21.15.169:0/4016908233 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v14) v1 ==== 190+0+147 (secure 0 0 0) 0x7f78ac042020 con 0x7f78b41597c0 2023-10-26T01:35:33.908 INFO:teuthology.orchestra.run.smithi169.stdout:[client.0] 2023-10-26T01:35:33.908 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQAgwTllKC7gOhAAPsvxGcuUA8/p8xqbUgGteA== 2023-10-26T01:35:33.908 INFO:teuthology.orchestra.run.smithi169.stdout: caps mds = "allow" 2023-10-26T01:35:33.909 INFO:teuthology.orchestra.run.smithi169.stdout: caps mon = "allow r" 2023-10-26T01:35:33.909 INFO:teuthology.orchestra.run.smithi169.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-10-26T01:35:33.909 INFO:teuthology.orchestra.run.smithi169.stderr:updated caps for client.0 2023-10-26T01:35:33.910 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:33.907+0000 7f78baae5640 1 -- 172.21.15.169:0/4016908233 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f789406ae40 msgr2=0x7f789406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:33.911 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:33.907+0000 7f78baae5640 1 --2- 172.21.15.169:0/4016908233 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f789406ae40 0x7f789406d300 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f78a40012e0 tx=0x7f78a4007d30 comp rx=0 tx=0).stop 2023-10-26T01:35:33.911 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:33.908+0000 7f78baae5640 1 -- 172.21.15.169:0/4016908233 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f78b41597c0 msgr2=0x7f78b40ae730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:33.911 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:33.908+0000 7f78baae5640 1 --2- 172.21.15.169:0/4016908233 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f78b41597c0 0x7f78b40ae730 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f78ac009230 tx=0x7f78ac02db60 comp rx=0 tx=0).stop 2023-10-26T01:35:33.911 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:33.908+0000 7f78baae5640 1 -- 172.21.15.169:0/4016908233 shutdown_connections 2023-10-26T01:35:33.912 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:33.908+0000 7f78baae5640 1 --2- 172.21.15.169:0/4016908233 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f789406ae40 0x7f789406d300 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:33.912 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:33.908+0000 7f78baae5640 1 --2- 172.21.15.169:0/4016908233 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f78b41597c0 0x7f78b40ae730 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:33.912 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:33.908+0000 7f78baae5640 1 --2- 172.21.15.169:0/4016908233 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f78b41593e0 0x7f78b40ae1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:33.912 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:33.908+0000 7f78baae5640 1 --2- 172.21.15.169:0/4016908233 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f78b40b0c80 0x7f78b41f1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:33.912 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:33.908+0000 7f78baae5640 1 -- 172.21.15.169:0/4016908233 >> 172.21.15.169:0/4016908233 conn(0x7f78b4069930 msgr2=0x7f78b40cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:35:33.912 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:33.908+0000 7f78baae5640 1 -- 172.21.15.169:0/4016908233 shutdown_connections 2023-10-26T01:35:33.913 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:33.908+0000 7f78baae5640 1 -- 172.21.15.169:0/4016908233 wait complete. 2023-10-26T01:35:33.928 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T01:35:34.146 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:34.142+0000 7eff8c256640 1 -- 172.21.15.169:0/712459143 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff8406ed30 msgr2=0x7eff840cd670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:34.146 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:34.142+0000 7eff8c256640 1 --2- 172.21.15.169:0/712459143 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff8406ed30 0x7eff840cd670 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7eff74005fd0 tx=0x7eff7402be20 comp rx=0 tx=0).stop 2023-10-26T01:35:34.146 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:34.142+0000 7eff8c256640 1 -- 172.21.15.169:0/712459143 shutdown_connections 2023-10-26T01:35:34.147 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:34.142+0000 7eff8c256640 1 --2- 172.21.15.169:0/712459143 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff8406ed30 0x7eff840cd670 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:34.147 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:34.142+0000 7eff8c256640 1 -- 172.21.15.169:0/712459143 >> 172.21.15.169:0/712459143 conn(0x7eff84069930 msgr2=0x7eff84069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:35:34.147 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:34.143+0000 7eff8c256640 1 -- 172.21.15.169:0/712459143 shutdown_connections 2023-10-26T01:35:34.147 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:34.143+0000 7eff8c256640 1 -- 172.21.15.169:0/712459143 wait complete. 2023-10-26T01:35:34.149 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:34.144+0000 7eff8c256640 1 Processor -- start 2023-10-26T01:35:34.150 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:34.144+0000 7eff8c256640 1 -- start start 2023-10-26T01:35:34.150 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:34.145+0000 7eff8c256640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7eff8406ed30 0x7eff84153670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:34.150 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:34.145+0000 7eff8c256640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff840cdbb0 0x7eff84157b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:34.151 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:34.145+0000 7eff8c256640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff8415b610 0x7eff84143a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:34.151 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:34.145+0000 7eff8c256640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff84154c00 con 0x7eff8415b610 2023-10-26T01:35:34.151 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:34.145+0000 7eff8c256640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff84153f80 con 0x7eff840cdbb0 2023-10-26T01:35:34.151 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:34.145+0000 7eff8c256640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff84154260 con 0x7eff8406ed30 2023-10-26T01:35:34.152 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:34.146+0000 7eff8a7f3640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff8415b610 0x7eff84143a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:34.152 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:34.146+0000 7eff8a7f3640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff8415b610 0x7eff84143a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:54520/0 (socket says 172.21.15.169:54520) 2023-10-26T01:35:34.152 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:34.146+0000 7eff8a7f3640 1 -- 172.21.15.169:0/3787474654 learned_addr learned my addr 172.21.15.169:0/3787474654 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:35:34.152 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:34.146+0000 7eff89ff2640 1 --2- 172.21.15.169:0/3787474654 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7eff8406ed30 0x7eff84153670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:34.153 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:34.146+0000 7eff897f1640 1 --2- 172.21.15.169:0/3787474654 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff840cdbb0 0x7eff84157b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:34.153 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:34.146+0000 7eff8a7f3640 1 -- 172.21.15.169:0/3787474654 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7eff8406ed30 msgr2=0x7eff84153670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:34.153 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:34.146+0000 7eff8a7f3640 1 --2- 172.21.15.169:0/3787474654 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7eff8406ed30 0x7eff84153670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:34.153 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:34.146+0000 7eff8a7f3640 1 -- 172.21.15.169:0/3787474654 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff840cdbb0 msgr2=0x7eff84157b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:34.153 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:34.146+0000 7eff8a7f3640 1 --2- 172.21.15.169:0/3787474654 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff840cdbb0 0x7eff84157b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:34.154 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:34.146+0000 7eff8a7f3640 1 -- 172.21.15.169:0/3787474654 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff74009560 con 0x7eff8415b610 2023-10-26T01:35:34.154 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:34.147+0000 7eff8a7f3640 1 --2- 172.21.15.169:0/3787474654 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff8415b610 0x7eff84143a90 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7eff7c00c7a0 tx=0x7eff7c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:35:34.154 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:34.147+0000 7eff7a7fc640 1 -- 172.21.15.169:0/3787474654 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff7c003b70 con 0x7eff8415b610 2023-10-26T01:35:34.154 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:34.147+0000 7eff7a7fc640 1 -- 172.21.15.169:0/3787474654 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7eff7c003cf0 con 0x7eff8415b610 2023-10-26T01:35:34.154 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:34.147+0000 7eff7a7fc640 1 -- 172.21.15.169:0/3787474654 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff7c00aab0 con 0x7eff8415b610 2023-10-26T01:35:34.155 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:34.147+0000 7eff8c256640 1 -- 172.21.15.169:0/3787474654 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff84154540 con 0x7eff8415b610 2023-10-26T01:35:34.155 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:34.147+0000 7eff8c256640 1 -- 172.21.15.169:0/3787474654 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff84153bb0 con 0x7eff8415b610 2023-10-26T01:35:34.156 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:34.148+0000 7eff8c256640 1 -- 172.21.15.169:0/3787474654 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff8404db10 con 0x7eff8415b610 2023-10-26T01:35:34.156 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:34.149+0000 7eff7a7fc640 1 -- 172.21.15.169:0/3787474654 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7eff7c00ac10 con 0x7eff8415b610 2023-10-26T01:35:34.156 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:34.150+0000 7eff7a7fc640 1 --2- 172.21.15.169:0/3787474654 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7eff6806ae40 0x7eff6806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:34.157 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:34.151+0000 7eff7a7fc640 1 -- 172.21.15.169:0/3787474654 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7eff7c0895a0 con 0x7eff8415b610 2023-10-26T01:35:34.157 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:34.151+0000 7eff89ff2640 1 --2- 172.21.15.169:0/3787474654 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7eff6806ae40 0x7eff6806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:34.157 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:34.152+0000 7eff89ff2640 1 --2- 172.21.15.169:0/3787474654 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7eff6806ae40 0x7eff6806d300 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7eff80006fd0 tx=0x7eff80008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:35:34.158 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:34.155+0000 7eff7a7fc640 1 -- 172.21.15.169:0/3787474654 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7eff7c0546e0 con 0x7eff8415b610 2023-10-26T01:35:34.384 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:34.381+0000 7eff8c256640 1 -- 172.21.15.169:0/3787474654 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7eff8406c990 con 0x7eff8415b610 2023-10-26T01:35:34.385 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:34.382+0000 7eff7a7fc640 1 -- 172.21.15.169:0/3787474654 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+4927 (secure 0 0 0) 0x7eff7c057d00 con 0x7eff8415b610 2023-10-26T01:35:34.385 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:35:34.385 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":42,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5347,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/2629782429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2629782429},{"type":"v1","addr":"172.21.15.170:6836","nonce":2629782429}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":6474,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2561200417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2561200417},{"type":"v1","addr":"172.21.15.169:6833","nonce":2561200417}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":6480,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/2834878721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":2834878721},{"type":"v1","addr":"172.21.15.170:6837","nonce":2834878721}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T01:35:27.834145+0000","modified":"2023-10-26T01:35:30.917014+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6486},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6486":{"gid":6486,"name":"a","rank":0,"incarnation":38,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6835/4004306080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":4004306080},{"type":"v1","addr":"172.21.15.169:6835","nonce":4004306080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-10-26T01:35:34.386 INFO:teuthology.orchestra.run.smithi169.stderr:dumped fsmap epoch 42 2023-10-26T01:35:34.388 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:34.385+0000 7eff8c256640 1 -- 172.21.15.169:0/3787474654 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7eff6806ae40 msgr2=0x7eff6806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:34.388 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:34.385+0000 7eff8c256640 1 --2- 172.21.15.169:0/3787474654 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7eff6806ae40 0x7eff6806d300 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7eff80006fd0 tx=0x7eff80008040 comp rx=0 tx=0).stop 2023-10-26T01:35:34.388 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:34.385+0000 7eff8c256640 1 -- 172.21.15.169:0/3787474654 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff8415b610 msgr2=0x7eff84143a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:34.389 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:34.385+0000 7eff8c256640 1 --2- 172.21.15.169:0/3787474654 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff8415b610 0x7eff84143a90 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7eff7c00c7a0 tx=0x7eff7c00cc60 comp rx=0 tx=0).stop 2023-10-26T01:35:34.389 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:34.385+0000 7eff8c256640 1 -- 172.21.15.169:0/3787474654 shutdown_connections 2023-10-26T01:35:34.389 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:34.385+0000 7eff8c256640 1 --2- 172.21.15.169:0/3787474654 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7eff6806ae40 0x7eff6806d300 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:34.389 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:34.385+0000 7eff8c256640 1 --2- 172.21.15.169:0/3787474654 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff8415b610 0x7eff84143a90 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:34.389 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:34.385+0000 7eff8c256640 1 --2- 172.21.15.169:0/3787474654 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff840cdbb0 0x7eff84157b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:34.390 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:34.385+0000 7eff8c256640 1 --2- 172.21.15.169:0/3787474654 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7eff8406ed30 0x7eff84153670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:34.390 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:34.385+0000 7eff8c256640 1 -- 172.21.15.169:0/3787474654 >> 172.21.15.169:0/3787474654 conn(0x7eff84069930 msgr2=0x7eff8415a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:35:34.390 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:34.385+0000 7eff8c256640 1 -- 172.21.15.169:0/3787474654 shutdown_connections 2023-10-26T01:35:34.390 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:34.385+0000 7eff8c256640 1 -- 172.21.15.169:0/3787474654 wait complete. 2023-10-26T01:35:34.810 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 42, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-26T01:35:27.834145+0000', 'modified': '2023-10-26T01:35:30.917014+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6486}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6486': {'gid': 6486, 'name': 'a', 'rank': 0, 'incarnation': 38, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.169:6835/4004306080', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.169:6834', 'nonce': 4004306080}, {'type': 'v1', 'addr': '172.21.15.169:6835', 'nonce': 4004306080}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-26T01:35:34.811 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-26T01:35:34.811 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-26T01:35:35.040 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.037+0000 7fe5a2d83640 1 -- 172.21.15.169:0/2582010857 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe58c004d00 msgr2=0x7fe58c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:35.040 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.037+0000 7fe5a2d83640 1 --2- 172.21.15.169:0/2582010857 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe58c004d00 0x7fe58c005180 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fe594009e20 tx=0x7fe58c0057f0 comp rx=0 tx=0).stop 2023-10-26T01:35:35.040 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.037+0000 7fe5a1320640 1 -- 172.21.15.169:0/2582010857 reap_dead start 2023-10-26T01:35:35.040 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.037+0000 7fe5a2d83640 1 -- 172.21.15.169:0/2582010857 shutdown_connections 2023-10-26T01:35:35.041 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.037+0000 7fe5a2d83640 1 -- 172.21.15.169:0/2582010857 >> 172.21.15.169:0/2582010857 conn(0x7fe59c065670 msgr2=0x7fe59c065a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:35:35.041 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.037+0000 7fe5a2d83640 1 -- 172.21.15.169:0/2582010857 shutdown_connections 2023-10-26T01:35:35.041 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.037+0000 7fe5a2d83640 1 -- 172.21.15.169:0/2582010857 wait complete. 2023-10-26T01:35:35.042 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.039+0000 7fe5a2d83640 1 Processor -- start 2023-10-26T01:35:35.042 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.039+0000 7fe5a2d83640 1 -- start start 2023-10-26T01:35:35.044 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.040+0000 7fe5a2d83640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe58c004d00 0x7fe59c15e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:35.044 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.040+0000 7fe5a2d83640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe59c14cb80 0x7fe59c1e5d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:35.044 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.040+0000 7fe5a2d83640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe59c1e6250 0x7fe59c1f1b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:35.044 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.040+0000 7fe5a2d83640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe59c15dbc0 con 0x7fe59c14cb80 2023-10-26T01:35:35.045 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.040+0000 7fe5a2d83640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe59c1e66b0 con 0x7fe58c004d00 2023-10-26T01:35:35.045 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.040+0000 7fe5a2d83640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe59c15d730 con 0x7fe59c1e6250 2023-10-26T01:35:35.045 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.040+0000 7fe59bfff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe59c14cb80 0x7fe59c1e5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:35.045 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.040+0000 7fe59bfff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe59c14cb80 0x7fe59c1e5d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:54544/0 (socket says 172.21.15.169:54544) 2023-10-26T01:35:35.045 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.040+0000 7fe59bfff640 1 -- 172.21.15.169:0/2679507482 learned_addr learned my addr 172.21.15.169:0/2679507482 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:35:35.046 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.040+0000 7fe5a1320640 1 --2- 172.21.15.169:0/2679507482 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe59c1e6250 0x7fe59c1f1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:35.046 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.040+0000 7fe5a0b1f640 1 --2- 172.21.15.169:0/2679507482 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe58c004d00 0x7fe59c15e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:35.046 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.041+0000 7fe59bfff640 1 -- 172.21.15.169:0/2679507482 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe59c1e6250 msgr2=0x7fe59c1f1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:35.046 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.041+0000 7fe59bfff640 1 --2- 172.21.15.169:0/2679507482 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe59c1e6250 0x7fe59c1f1b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:35.046 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.041+0000 7fe59bfff640 1 -- 172.21.15.169:0/2679507482 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe58c004d00 msgr2=0x7fe59c15e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:35.047 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.041+0000 7fe59bfff640 1 --2- 172.21.15.169:0/2679507482 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe58c004d00 0x7fe59c15e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:35.047 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.041+0000 7fe59bfff640 1 -- 172.21.15.169:0/2679507482 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe59c15cb20 con 0x7fe59c14cb80 2023-10-26T01:35:35.047 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.041+0000 7fe5a1320640 1 --2- 172.21.15.169:0/2679507482 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe59c1e6250 0x7fe59c1f1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:35:35.047 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.041+0000 7fe5a0b1f640 1 --2- 172.21.15.169:0/2679507482 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe58c004d00 0x7fe59c15e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:35:35.047 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.041+0000 7fe59bfff640 1 --2- 172.21.15.169:0/2679507482 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe59c14cb80 0x7fe59c1e5d10 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fe58800d640 tx=0x7fe58800db00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:35:35.048 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.041+0000 7fe5997fa640 1 -- 172.21.15.169:0/2679507482 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5880062e0 con 0x7fe59c14cb80 2023-10-26T01:35:35.048 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.041+0000 7fe5997fa640 1 -- 172.21.15.169:0/2679507482 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe588006530 con 0x7fe59c14cb80 2023-10-26T01:35:35.048 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.041+0000 7fe5997fa640 1 -- 172.21.15.169:0/2679507482 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe588003850 con 0x7fe59c14cb80 2023-10-26T01:35:35.048 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.041+0000 7fe5a2d83640 1 -- 172.21.15.169:0/2679507482 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe59c15ce00 con 0x7fe59c14cb80 2023-10-26T01:35:35.048 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.041+0000 7fe5a2d83640 1 -- 172.21.15.169:0/2679507482 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe59c15d3c0 con 0x7fe59c14cb80 2023-10-26T01:35:35.051 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.043+0000 7fe5997fa640 1 -- 172.21.15.169:0/2679507482 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fe5880039b0 con 0x7fe59c14cb80 2023-10-26T01:35:35.051 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.044+0000 7fe5997fa640 1 --2- 172.21.15.169:0/2679507482 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe58006ae40 0x7fe58006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:35.052 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.044+0000 7fe5997fa640 1 -- 172.21.15.169:0/2679507482 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe588089fb0 con 0x7fe59c14cb80 2023-10-26T01:35:35.052 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.045+0000 7fe5a0b1f640 1 --2- 172.21.15.169:0/2679507482 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe58006ae40 0x7fe58006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:35.052 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.045+0000 7fe5a59ee740 1 Processor -- start 2023-10-26T01:35:35.052 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.045+0000 7fe5a59ee740 1 -- start start 2023-10-26T01:35:35.052 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.046+0000 7fe5a0b1f640 1 --2- 172.21.15.169:0/2679507482 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe58006ae40 0x7fe58006d300 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fe590001eb0 tx=0x7fe5900015d0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:35:35.053 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.046+0000 7fe5a59ee740 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x56010c6457d0 0x56010c618a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:35.053 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.046+0000 7fe5a59ee740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56010c645bb0 0x56010c648560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:35.053 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.046+0000 7fe5a59ee740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56010c618f40 0x56010c648aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:35.053 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.046+0000 7fe5a59ee740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x56010c58b740 con 0x56010c618f40 2023-10-26T01:35:35.053 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.046+0000 7fe5a59ee740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56010c588e30 con 0x56010c645bb0 2023-10-26T01:35:35.054 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.046+0000 7fe5a59ee740 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x56010c584720 con 0x56010c6457d0 2023-10-26T01:35:35.054 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.047+0000 7fe5a1320640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56010c618f40 0x56010c648aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:35.054 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.047+0000 7fe5a1320640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56010c618f40 0x56010c648aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:54546/0 (socket says 172.21.15.169:54546) 2023-10-26T01:35:35.054 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.047+0000 7fe5a1320640 1 -- 172.21.15.169:0/783900046 learned_addr learned my addr 172.21.15.169:0/783900046 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:35:35.054 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.047+0000 7fe5a0b1f640 1 --2- 172.21.15.169:0/783900046 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x56010c6457d0 0x56010c618a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:35.055 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.047+0000 7fe59bfff640 1 --2- 172.21.15.169:0/783900046 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56010c645bb0 0x56010c648560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:35.055 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.047+0000 7fe5a1320640 1 -- 172.21.15.169:0/783900046 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x56010c6457d0 msgr2=0x56010c618a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T01:35:35.055 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.047+0000 7fe5a1320640 1 --2- 172.21.15.169:0/783900046 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x56010c6457d0 0x56010c618a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:35.055 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.047+0000 7fe5a1320640 1 -- 172.21.15.169:0/783900046 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56010c645bb0 msgr2=0x56010c648560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T01:35:35.055 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.047+0000 7fe5a1320640 1 --2- 172.21.15.169:0/783900046 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56010c645bb0 0x56010c648560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:35.056 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.047+0000 7fe5a1320640 1 -- 172.21.15.169:0/783900046 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56010c548910 con 0x56010c618f40 2023-10-26T01:35:35.056 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.047+0000 7fe59bfff640 1 --2- 172.21.15.169:0/783900046 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56010c645bb0 0x56010c648560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:35:35.056 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.048+0000 7fe5a1320640 1 --2- 172.21.15.169:0/783900046 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56010c618f40 0x56010c648aa0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fe594009f40 tx=0x7fe594004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad26e05ff210ba8d server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:35:35.056 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.048+0000 7fe5867fc640 1 -- 172.21.15.169:0/783900046 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe59403d6d0 con 0x56010c618f40 2023-10-26T01:35:35.057 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.048+0000 7fe5867fc640 1 -- 172.21.15.169:0/783900046 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe59403d830 con 0x56010c618f40 2023-10-26T01:35:35.057 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.048+0000 7fe5867fc640 1 -- 172.21.15.169:0/783900046 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe59403dc00 con 0x56010c618f40 2023-10-26T01:35:35.057 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.048+0000 7fe5a59ee740 1 -- 172.21.15.169:0/783900046 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56010c618f40 msgr2=0x56010c648aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:35.057 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.048+0000 7fe5a59ee740 1 --2- 172.21.15.169:0/783900046 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56010c618f40 0x56010c648aa0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fe594009f40 tx=0x7fe594004a40 comp rx=0 tx=0).stop 2023-10-26T01:35:35.057 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.049+0000 7fe5a59ee740 1 -- 172.21.15.169:0/783900046 shutdown_connections 2023-10-26T01:35:35.058 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.049+0000 7fe5a59ee740 1 --2- 172.21.15.169:0/783900046 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56010c618f40 0x56010c648aa0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:35.058 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.049+0000 7fe5a59ee740 1 --2- 172.21.15.169:0/783900046 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56010c645bb0 0x56010c648560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:35.058 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.049+0000 7fe5a59ee740 1 --2- 172.21.15.169:0/783900046 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x56010c6457d0 0x56010c618a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:35.058 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.049+0000 7fe5a59ee740 1 -- 172.21.15.169:0/783900046 >> 172.21.15.169:0/783900046 conn(0x56010c61ac70 msgr2=0x56010c663560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:35:35.058 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.049+0000 7fe5a59ee740 1 -- 172.21.15.169:0/783900046 shutdown_connections 2023-10-26T01:35:35.059 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.049+0000 7fe5a59ee740 1 -- 172.21.15.169:0/783900046 wait complete. 2023-10-26T01:35:35.062 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.049+0000 7fe5a59ee740 1 -- start start 2023-10-26T01:35:35.062 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.050+0000 7fe5a59ee740 1 Processor -- start 2023-10-26T01:35:35.062 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.051+0000 7fe5a59ee740 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x56010c6457d0 0x56010c64afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:35.062 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.051+0000 7fe5a59ee740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56010c645bb0 0x56010c639b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:35.063 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.051+0000 7fe5a59ee740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56010c644a30 0x56010c660d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:35.063 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.051+0000 7fe5a59ee740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x56010c522970 con 0x56010c644a30 2023-10-26T01:35:35.063 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.051+0000 7fe5a59ee740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56010c522610 con 0x56010c645bb0 2023-10-26T01:35:35.063 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.051+0000 7fe5a59ee740 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x56010c2e8f20 con 0x56010c6457d0 2023-10-26T01:35:35.063 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.051+0000 7fe5a59ee740 10 client.0 _ll_register_callbacks cb 0x56010c5460c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T01:35:35.064 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.051+0000 7fe5a1320640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56010c644a30 0x56010c660d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:35.064 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.051+0000 7fe5a1320640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56010c644a30 0x56010c660d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:54548/0 (socket says 172.21.15.169:54548) 2023-10-26T01:35:35.064 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.051+0000 7fe5a1320640 1 -- 172.21.15.169:0/2680181699 learned_addr learned my addr 172.21.15.169:0/2680181699 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:35:35.064 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.051+0000 7fe5a1320640 1 -- 172.21.15.169:0/2680181699 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x56010c6457d0 msgr2=0x56010c64afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T01:35:35.064 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.051+0000 7fe5a1320640 1 --2- 172.21.15.169:0/2680181699 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x56010c6457d0 0x56010c64afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:35.065 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.051+0000 7fe5a1320640 1 -- 172.21.15.169:0/2680181699 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56010c645bb0 msgr2=0x56010c639b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T01:35:35.065 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.051+0000 7fe5a1320640 1 --2- 172.21.15.169:0/2680181699 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56010c645bb0 0x56010c639b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:35.065 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.051+0000 7fe5a1320640 1 -- 172.21.15.169:0/2680181699 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe58c003520 con 0x56010c644a30 2023-10-26T01:35:35.065 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.052+0000 7fe5a1320640 1 --2- 172.21.15.169:0/2680181699 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56010c644a30 0x56010c660d50 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fe594009f40 tx=0x7fe59403d9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1761b063bfbed32 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:35:35.065 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.052+0000 7fe57b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.169:3300/0 2023-10-26T01:35:35.066 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.052+0000 7fe57b7fe640 1 -- 172.21.15.169:0/2680181699 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe594047400 con 0x56010c644a30 2023-10-26T01:35:35.066 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.052+0000 7fe57b7fe640 1 -- 172.21.15.169:0/2680181699 <== mon.0 v2:172.21.15.169:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe594047560 con 0x56010c644a30 2023-10-26T01:35:35.066 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.052+0000 7fe5a59ee740 1 -- 172.21.15.169:0/2680181699 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56010c57d700 con 0x56010c644a30 2023-10-26T01:35:35.066 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.052+0000 7fe57b7fe640 1 -- 172.21.15.169:0/2680181699 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe594004b40 con 0x56010c644a30 2023-10-26T01:35:35.066 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.052+0000 7fe5a59ee740 10 client.6621 fetch_fsmap learned FSMap version 42 2023-10-26T01:35:35.067 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.052+0000 7fe5a59ee740 1 -- 172.21.15.169:0/2680181699 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=2+}) v3 -- 0x56010c582da0 con 0x56010c644a30 2023-10-26T01:35:35.067 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.052+0000 7fe57b7fe640 1 -- 172.21.15.169:0/2680181699 <== mon.0 v2:172.21.15.169:3300/0 4 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x56010c57d700 con 0x56010c644a30 2023-10-26T01:35:35.067 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.053+0000 7fe57b7fe640 1 -- 172.21.15.169:0/2680181699 <== mon.0 v2:172.21.15.169:3300/0 5 ==== fsmap(e 42) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe594045030 con 0x56010c644a30 2023-10-26T01:35:35.067 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.053+0000 7fe5a59ee740 10 client.6621 fetch_fsmap finished waiting for FSMap version 42 2023-10-26T01:35:35.067 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.053+0000 7fe5a59ee740 10 client.6621 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-26T01:35:35.068 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.053+0000 7fe5a59ee740 20 client.6621 populate_metadata read hostname 'smithi169' 2023-10-26T01:35:35.068 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.053+0000 7fe5a59ee740 1 --2- 172.21.15.169:0/2680181699 >> [v2:172.21.15.169:6834/4004306080,v1:172.21.15.169:6835/4004306080] conn(0x56010c663640 0x56010c680960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:35.068 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.053+0000 7fe5a59ee740 4 client.6621 mds_command: new command op to 6486 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-26T01:35:35.068 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.053+0000 7fe5a59ee740 1 -- 172.21.15.169:0/2680181699 --> [v2:172.21.15.169:6834/4004306080,v1:172.21.15.169:6835/4004306080] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56010c57d420 con 0x56010c663640 2023-10-26T01:35:35.068 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.053+0000 7fe5a0b1f640 1 --2- 172.21.15.169:0/2680181699 >> [v2:172.21.15.169:6834/4004306080,v1:172.21.15.169:6835/4004306080] conn(0x56010c663640 0x56010c680960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:35.069 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.053+0000 7fe5a0b1f640 1 --2- 172.21.15.169:0/2680181699 >> [v2:172.21.15.169:6834/4004306080,v1:172.21.15.169:6835/4004306080] conn(0x56010c663640 0x56010c680960 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=882fb71c48809954 server_cookie=3e2ed013ee04aed in_seq=0 out_seq=0 2023-10-26T01:35:35.069 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.053+0000 7fe57b7fe640 10 client.6621 ms_handle_connect on v2:172.21.15.169:6834/4004306080 2023-10-26T01:35:35.069 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.054+0000 7fe57b7fe640 1 -- 172.21.15.169:0/2680181699 <== mds.0 v2:172.21.15.169:6834/4004306080 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56010c57d420 con 0x56010c663640 2023-10-26T01:35:35.069 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.054+0000 7fe57b7fe640 10 client.6621 handle_command_reply: tid=0 2023-10-26T01:35:35.069 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.054+0000 7fe5a59ee740 1 client.6621 shutdown 2023-10-26T01:35:35.069 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.054+0000 7fe5a59ee740 1 -- 172.21.15.169:0/2680181699 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56010c644a30 msgr2=0x56010c660d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:35.070 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.054+0000 7fe5a59ee740 1 --2- 172.21.15.169:0/2680181699 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56010c644a30 0x56010c660d50 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fe594009f40 tx=0x7fe59403d9a0 comp rx=0 tx=0).stop 2023-10-26T01:35:35.070 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.055+0000 7fe5a59ee740 1 -- 172.21.15.169:0/2680181699 shutdown_connections 2023-10-26T01:35:35.070 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.055+0000 7fe5a59ee740 1 --2- 172.21.15.169:0/2680181699 >> [v2:172.21.15.169:6834/4004306080,v1:172.21.15.169:6835/4004306080] conn(0x56010c663640 0x56010c680960 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:35.070 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.055+0000 7fe5a59ee740 1 --2- 172.21.15.169:0/2680181699 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56010c644a30 0x56010c660d50 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:35.070 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.055+0000 7fe5a59ee740 1 --2- 172.21.15.169:0/2680181699 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56010c645bb0 0x56010c639b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:35.071 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.055+0000 7fe5a59ee740 1 --2- 172.21.15.169:0/2680181699 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x56010c6457d0 0x56010c64afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:35.071 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.055+0000 7fe5a59ee740 1 -- 172.21.15.169:0/2680181699 >> 172.21.15.169:0/2680181699 conn(0x56010c618f40 msgr2=0x56010c648550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:35:35.071 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.055+0000 7fe57b7fe640 0 client.6621 ms_handle_reset on v2:172.21.15.169:6834/4004306080 2023-10-26T01:35:35.071 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.055+0000 7fe5a59ee740 1 -- 172.21.15.169:0/2680181699 shutdown_connections 2023-10-26T01:35:35.071 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.055+0000 7fe5a59ee740 1 -- 172.21.15.169:0/2680181699 wait complete. 2023-10-26T01:35:35.072 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.055+0000 7fe5a59ee740 20 client.6621 trim_cache size 0 max 16384 2023-10-26T01:35:35.074 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.071+0000 7fe5a59ee740 1 Processor -- start 2023-10-26T01:35:35.074 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.072+0000 7fe5a59ee740 1 -- start start 2023-10-26T01:35:35.075 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.072+0000 7fe5a59ee740 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x56010c644a30 0x56010c614120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:35.075 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.072+0000 7fe5a59ee740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56010c6457d0 0x56010c648740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:35.075 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.072+0000 7fe5a59ee740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56010c645bb0 0x56010c664090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:35.075 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.072+0000 7fe5a59ee740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x56010c685260 con 0x56010c645bb0 2023-10-26T01:35:35.076 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.072+0000 7fe5a59ee740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56010c649dc0 con 0x56010c6457d0 2023-10-26T01:35:35.076 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.072+0000 7fe5a59ee740 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x56010c614660 con 0x56010c644a30 2023-10-26T01:35:35.078 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.073+0000 7fe5a1320640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56010c645bb0 0x56010c664090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:35.078 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.073+0000 7fe5a1320640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56010c645bb0 0x56010c664090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:54556/0 (socket says 172.21.15.169:54556) 2023-10-26T01:35:35.078 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.073+0000 7fe5a1320640 1 -- 172.21.15.169:0/2511018248 learned_addr learned my addr 172.21.15.169:0/2511018248 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:35:35.079 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.073+0000 7fe5a0b1f640 1 --2- 172.21.15.169:0/2511018248 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x56010c644a30 0x56010c614120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:35.079 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.073+0000 7fe59bfff640 1 --2- 172.21.15.169:0/2511018248 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56010c6457d0 0x56010c648740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:35.079 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.073+0000 7fe5a1320640 1 -- 172.21.15.169:0/2511018248 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x56010c644a30 msgr2=0x56010c614120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T01:35:35.079 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.073+0000 7fe5a1320640 1 --2- 172.21.15.169:0/2511018248 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x56010c644a30 0x56010c614120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:35.079 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.073+0000 7fe5a1320640 1 -- 172.21.15.169:0/2511018248 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56010c6457d0 msgr2=0x56010c648740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T01:35:35.080 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.073+0000 7fe5a1320640 1 --2- 172.21.15.169:0/2511018248 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56010c6457d0 0x56010c648740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:35.080 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.073+0000 7fe5a1320640 1 -- 172.21.15.169:0/2511018248 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56010c521b80 con 0x56010c645bb0 2023-10-26T01:35:35.080 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.073+0000 7fe5a0b1f640 1 --2- 172.21.15.169:0/2511018248 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x56010c644a30 0x56010c614120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:35:35.080 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.073+0000 7fe59bfff640 1 --2- 172.21.15.169:0/2511018248 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56010c6457d0 0x56010c648740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:35:35.080 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.073+0000 7fe5a1320640 1 --2- 172.21.15.169:0/2511018248 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56010c645bb0 0x56010c664090 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fe594009d70 tx=0x7fe594035040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de741d83b8ae88d4 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:35:35.081 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.074+0000 7fe586ffd640 1 -- 172.21.15.169:0/2511018248 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe594047b60 con 0x56010c645bb0 2023-10-26T01:35:35.081 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.074+0000 7fe586ffd640 1 -- 172.21.15.169:0/2511018248 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe594047db0 con 0x56010c645bb0 2023-10-26T01:35:35.081 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.074+0000 7fe586ffd640 1 -- 172.21.15.169:0/2511018248 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe594004da0 con 0x56010c645bb0 2023-10-26T01:35:35.081 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.074+0000 7fe5a59ee740 1 -- 172.21.15.169:0/2511018248 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56010c645bb0 msgr2=0x56010c664090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:35.081 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.074+0000 7fe5a59ee740 1 --2- 172.21.15.169:0/2511018248 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56010c645bb0 0x56010c664090 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fe594009d70 tx=0x7fe594035040 comp rx=0 tx=0).stop 2023-10-26T01:35:35.082 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.074+0000 7fe5a59ee740 1 -- 172.21.15.169:0/2511018248 shutdown_connections 2023-10-26T01:35:35.082 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.074+0000 7fe5a59ee740 1 --2- 172.21.15.169:0/2511018248 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56010c645bb0 0x56010c664090 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:35.082 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.074+0000 7fe5a59ee740 1 --2- 172.21.15.169:0/2511018248 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56010c6457d0 0x56010c648740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:35.082 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.074+0000 7fe5a59ee740 1 --2- 172.21.15.169:0/2511018248 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x56010c644a30 0x56010c614120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:35.082 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.074+0000 7fe5a59ee740 1 -- 172.21.15.169:0/2511018248 >> 172.21.15.169:0/2511018248 conn(0x56010c618f40 msgr2=0x56010c64a780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:35:35.083 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.074+0000 7fe5a59ee740 1 -- 172.21.15.169:0/2511018248 shutdown_connections 2023-10-26T01:35:35.083 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.074+0000 7fe5a59ee740 1 -- 172.21.15.169:0/2511018248 wait complete. 2023-10-26T01:35:35.083 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.075+0000 7fe5a59ee740 1 -- start start 2023-10-26T01:35:35.083 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.075+0000 7fe5a59ee740 1 Processor -- start 2023-10-26T01:35:35.087 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.076+0000 7fe5a59ee740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56010c644a30 0x56010c648740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:35.087 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.076+0000 7fe5a59ee740 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x56010c6457d0 0x56010c648c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:35.087 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.076+0000 7fe5a59ee740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56010c645bb0 0x56010c6721a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:35.087 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.076+0000 7fe5a59ee740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x56010c680cb0 con 0x56010c644a30 2023-10-26T01:35:35.088 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.076+0000 7fe5a59ee740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56010c649520 con 0x56010c645bb0 2023-10-26T01:35:35.088 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.076+0000 7fe5a59ee740 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x56010c64cab0 con 0x56010c6457d0 2023-10-26T01:35:35.088 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.076+0000 7fe5a59ee740 10 client.0 _ll_register_callbacks cb 0x7fe58c0036b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T01:35:35.088 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.076+0000 7fe5a0b1f640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56010c644a30 0x56010c648740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:35.088 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.076+0000 7fe5a0b1f640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56010c644a30 0x56010c648740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:54560/0 (socket says 172.21.15.169:54560) 2023-10-26T01:35:35.089 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.076+0000 7fe5a0b1f640 1 -- 172.21.15.169:0/2733328515 learned_addr learned my addr 172.21.15.169:0/2733328515 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:35:35.089 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.077+0000 7fe5a0b1f640 1 -- 172.21.15.169:0/2733328515 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x56010c6457d0 msgr2=0x56010c648c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T01:35:35.089 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.077+0000 7fe59bfff640 1 --2- 172.21.15.169:0/2733328515 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x56010c6457d0 0x56010c648c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:35.089 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.077+0000 7fe5a0b1f640 1 --2- 172.21.15.169:0/2733328515 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x56010c6457d0 0x56010c648c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:35.089 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.077+0000 7fe5a0b1f640 1 -- 172.21.15.169:0/2733328515 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56010c645bb0 msgr2=0x56010c6721a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T01:35:35.089 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.077+0000 7fe5a0b1f640 1 --2- 172.21.15.169:0/2733328515 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56010c645bb0 0x56010c6721a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:35.090 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.077+0000 7fe5a0b1f640 1 -- 172.21.15.169:0/2733328515 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56010c582da0 con 0x56010c644a30 2023-10-26T01:35:35.090 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.077+0000 7fe5a0b1f640 1 --2- 172.21.15.169:0/2733328515 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56010c644a30 0x56010c648740 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fe59000ea30 tx=0x7fe590000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e92f7068ae3fb908 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:35:35.090 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.077+0000 7fe57b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.169:3300/0 2023-10-26T01:35:35.090 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.077+0000 7fe57b7fe640 1 -- 172.21.15.169:0/2733328515 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe590023d20 con 0x56010c644a30 2023-10-26T01:35:35.090 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.077+0000 7fe57b7fe640 1 -- 172.21.15.169:0/2733328515 <== mon.0 v2:172.21.15.169:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe59001b040 con 0x56010c644a30 2023-10-26T01:35:35.091 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.077+0000 7fe5a59ee740 1 -- 172.21.15.169:0/2733328515 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe59403d830 con 0x56010c644a30 2023-10-26T01:35:35.091 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.078+0000 7fe57b7fe640 1 -- 172.21.15.169:0/2733328515 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe590023e80 con 0x56010c644a30 2023-10-26T01:35:35.091 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.078+0000 7fe5a59ee740 10 client.6627 fetch_fsmap learned FSMap version 42 2023-10-26T01:35:35.091 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.078+0000 7fe5a59ee740 1 -- 172.21.15.169:0/2733328515 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=2+}) v3 -- 0x56010c547170 con 0x56010c644a30 2023-10-26T01:35:35.091 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.078+0000 7fe57b7fe640 1 -- 172.21.15.169:0/2733328515 <== mon.0 v2:172.21.15.169:3300/0 4 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe59403d830 con 0x56010c644a30 2023-10-26T01:35:35.092 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.078+0000 7fe57b7fe640 1 -- 172.21.15.169:0/2733328515 <== mon.0 v2:172.21.15.169:3300/0 5 ==== fsmap(e 42) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe590011070 con 0x56010c644a30 2023-10-26T01:35:35.092 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.078+0000 7fe5a59ee740 10 client.6627 fetch_fsmap finished waiting for FSMap version 42 2023-10-26T01:35:35.092 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.078+0000 7fe5a59ee740 10 client.6627 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-26T01:35:35.092 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.078+0000 7fe5a59ee740 20 client.6627 populate_metadata read hostname 'smithi169' 2023-10-26T01:35:35.092 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.078+0000 7fe5a59ee740 1 --2- 172.21.15.169:0/2733328515 >> [v2:172.21.15.169:6834/4004306080,v1:172.21.15.169:6835/4004306080] conn(0x56010c6727b0 0x56010c672b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:35.093 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.078+0000 7fe5a59ee740 4 client.6627 mds_command: new command op to 6486 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-26T01:35:35.093 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.078+0000 7fe5a59ee740 1 -- 172.21.15.169:0/2733328515 --> [v2:172.21.15.169:6834/4004306080,v1:172.21.15.169:6835/4004306080] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56010c2866c0 con 0x56010c6727b0 2023-10-26T01:35:35.093 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.078+0000 7fe59bfff640 1 --2- 172.21.15.169:0/2733328515 >> [v2:172.21.15.169:6834/4004306080,v1:172.21.15.169:6835/4004306080] conn(0x56010c6727b0 0x56010c672b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:35.093 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.079+0000 7fe59bfff640 1 --2- 172.21.15.169:0/2733328515 >> [v2:172.21.15.169:6834/4004306080,v1:172.21.15.169:6835/4004306080] conn(0x56010c6727b0 0x56010c672b90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65ebe71b2d754f98 server_cookie=19a2db4dbf4d6f8 in_seq=0 out_seq=0 2023-10-26T01:35:35.093 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.079+0000 7fe57b7fe640 10 client.6627 ms_handle_connect on v2:172.21.15.169:6834/4004306080 2023-10-26T01:35:35.094 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.079+0000 7fe57b7fe640 1 -- 172.21.15.169:0/2733328515 <== mds.0 v2:172.21.15.169:6834/4004306080 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x56010c2866c0 con 0x56010c6727b0 2023-10-26T01:35:35.094 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.079+0000 7fe57b7fe640 10 client.6627 handle_command_reply: tid=0 2023-10-26T01:35:35.094 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.079+0000 7fe5a59ee740 1 client.6627 shutdown 2023-10-26T01:35:35.094 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.079+0000 7fe5a59ee740 1 -- 172.21.15.169:0/2733328515 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56010c644a30 msgr2=0x56010c648740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:35.094 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.079+0000 7fe5a59ee740 1 --2- 172.21.15.169:0/2733328515 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56010c644a30 0x56010c648740 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fe59000ea30 tx=0x7fe590000ed0 comp rx=0 tx=0).stop 2023-10-26T01:35:35.095 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.079+0000 7fe5a59ee740 1 -- 172.21.15.169:0/2733328515 shutdown_connections 2023-10-26T01:35:35.095 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.079+0000 7fe5a59ee740 1 --2- 172.21.15.169:0/2733328515 >> [v2:172.21.15.169:6834/4004306080,v1:172.21.15.169:6835/4004306080] conn(0x56010c6727b0 0x56010c672b90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:35.095 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.079+0000 7fe5a59ee740 1 --2- 172.21.15.169:0/2733328515 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56010c645bb0 0x56010c6721a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:35.095 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.079+0000 7fe5a59ee740 1 --2- 172.21.15.169:0/2733328515 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x56010c6457d0 0x56010c648c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:35.095 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.079+0000 7fe57b7fe640 0 client.6627 ms_handle_reset on v2:172.21.15.169:6834/4004306080 2023-10-26T01:35:35.096 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.079+0000 7fe5a59ee740 1 --2- 172.21.15.169:0/2733328515 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56010c644a30 0x56010c648740 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:35.096 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.079+0000 7fe5a59ee740 1 -- 172.21.15.169:0/2733328515 >> 172.21.15.169:0/2733328515 conn(0x56010c618f40 msgr2=0x56010c64afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:35:35.096 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.080+0000 7fe5a59ee740 1 -- 172.21.15.169:0/2733328515 shutdown_connections 2023-10-26T01:35:35.096 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.080+0000 7fe5a59ee740 1 -- 172.21.15.169:0/2733328515 wait complete. 2023-10-26T01:35:35.096 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.080+0000 7fe5a59ee740 20 client.6627 trim_cache size 0 max 16384 2023-10-26T01:35:35.097 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2023-10-26T01:35:35.097 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_fsid": "30203b28-ed8a-44e8-a9a6-a9bf25f980f1", 2023-10-26T01:35:35.097 INFO:teuthology.orchestra.run.smithi169.stdout: "whoami": 0, 2023-10-26T01:35:35.097 INFO:teuthology.orchestra.run.smithi169.stdout: "id": 6486, 2023-10-26T01:35:35.098 INFO:teuthology.orchestra.run.smithi169.stdout: "want_state": "up:active", 2023-10-26T01:35:35.098 INFO:teuthology.orchestra.run.smithi169.stdout: "state": "up:active", 2023-10-26T01:35:35.098 INFO:teuthology.orchestra.run.smithi169.stdout: "fs_name": "cephfs", 2023-10-26T01:35:35.098 INFO:teuthology.orchestra.run.smithi169.stdout: "rank_uptime": 6.6900659490000001, 2023-10-26T01:35:35.098 INFO:teuthology.orchestra.run.smithi169.stdout: "mdsmap_epoch": 42, 2023-10-26T01:35:35.099 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_epoch": 84, 2023-10-26T01:35:35.099 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_epoch_barrier": 84, 2023-10-26T01:35:35.099 INFO:teuthology.orchestra.run.smithi169.stdout: "uptime": 12.606688034999999 2023-10-26T01:35:35.099 INFO:teuthology.orchestra.run.smithi169.stdout:} 2023-10-26T01:35:35.100 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.080+0000 7fe5a2d83640 1 -- 172.21.15.169:0/2679507482 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe58006ae40 msgr2=0x7fe58006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:35.101 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.080+0000 7fe5a2d83640 1 --2- 172.21.15.169:0/2679507482 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe58006ae40 0x7fe58006d300 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fe590001eb0 tx=0x7fe5900015d0 comp rx=0 tx=0).stop 2023-10-26T01:35:35.101 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.080+0000 7fe5a2d83640 1 -- 172.21.15.169:0/2679507482 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe59c14cb80 msgr2=0x7fe59c1e5d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:35.101 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.080+0000 7fe5a2d83640 1 --2- 172.21.15.169:0/2679507482 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe59c14cb80 0x7fe59c1e5d10 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fe58800d640 tx=0x7fe58800db00 comp rx=0 tx=0).stop 2023-10-26T01:35:35.101 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.080+0000 7fe5a2d83640 1 -- 172.21.15.169:0/2679507482 shutdown_connections 2023-10-26T01:35:35.101 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.080+0000 7fe5a2d83640 1 --2- 172.21.15.169:0/2679507482 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe58006ae40 0x7fe58006d300 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:35.102 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.080+0000 7fe5a2d83640 1 --2- 172.21.15.169:0/2679507482 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe59c14cb80 0x7fe59c1e5d10 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:35.102 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.080+0000 7fe5a2d83640 1 --2- 172.21.15.169:0/2679507482 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe59c1e6250 0x7fe59c1f1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:35.102 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.080+0000 7fe5a2d83640 1 --2- 172.21.15.169:0/2679507482 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe58c004d00 0x7fe59c15e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:35.102 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.080+0000 7fe5a2d83640 1 -- 172.21.15.169:0/2679507482 >> 172.21.15.169:0/2679507482 conn(0x7fe59c065670 msgr2=0x7fe59c065a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:35:35.102 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.081+0000 7fe5a2d83640 1 -- 172.21.15.169:0/2679507482 shutdown_connections 2023-10-26T01:35:35.103 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.081+0000 7fe5a2d83640 1 -- 172.21.15.169:0/2679507482 wait complete. 2023-10-26T01:35:35.103 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-26T01:35:35.319 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.315+0000 7f2b4d3c4640 1 -- 172.21.15.169:0/1428358180 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2b480cab80 msgr2=0x7f2b480717a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:35.319 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.315+0000 7f2b4d3c4640 1 --2- 172.21.15.169:0/1428358180 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2b480cab80 0x7f2b480717a0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f2b30005fd0 tx=0x7f2b3002bd10 comp rx=0 tx=0).stop 2023-10-26T01:35:35.319 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.316+0000 7f2b4d3c4640 1 -- 172.21.15.169:0/1428358180 shutdown_connections 2023-10-26T01:35:35.319 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.316+0000 7f2b4d3c4640 1 --2- 172.21.15.169:0/1428358180 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2b480cab80 0x7f2b480717a0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:35.320 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.316+0000 7f2b4d3c4640 1 -- 172.21.15.169:0/1428358180 >> 172.21.15.169:0/1428358180 conn(0x7f2b480af0c0 msgr2=0x7f2b480af4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:35:35.320 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.316+0000 7f2b4d3c4640 1 -- 172.21.15.169:0/1428358180 shutdown_connections 2023-10-26T01:35:35.320 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.316+0000 7f2b4d3c4640 1 -- 172.21.15.169:0/1428358180 wait complete. 2023-10-26T01:35:35.321 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.317+0000 7f2b4d3c4640 1 Processor -- start 2023-10-26T01:35:35.321 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.318+0000 7f2b4d3c4640 1 -- start start 2023-10-26T01:35:35.323 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.318+0000 7f2b4d3c4640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2b480ca7a0 0x7f2b480abc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:35.323 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.318+0000 7f2b4d3c4640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2b480cab80 0x7f2b48150910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:35.323 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.318+0000 7f2b4d3c4640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2b480cc980 0x7f2b48144370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:35.323 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.318+0000 7f2b4d3c4640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b48151160 con 0x7f2b480ca7a0 2023-10-26T01:35:35.323 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.319+0000 7f2b4d3c4640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b48151440 con 0x7f2b480cc980 2023-10-26T01:35:35.324 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.319+0000 7f2b4d3c4640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b48151720 con 0x7f2b480cab80 2023-10-26T01:35:35.324 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.319+0000 7f2b47fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2b480ca7a0 0x7f2b480abc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:35.324 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.319+0000 7f2b47fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2b480ca7a0 0x7f2b480abc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:54566/0 (socket says 172.21.15.169:54566) 2023-10-26T01:35:35.324 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.319+0000 7f2b47fff640 1 -- 172.21.15.169:0/2117294497 learned_addr learned my addr 172.21.15.169:0/2117294497 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:35:35.324 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.319+0000 7f2b3ffff640 1 --2- 172.21.15.169:0/2117294497 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2b480cab80 0x7f2b48150910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:35.325 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.319+0000 7f2b4cbc3640 1 --2- 172.21.15.169:0/2117294497 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2b480cc980 0x7f2b48144370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:35.325 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.319+0000 7f2b47fff640 1 -- 172.21.15.169:0/2117294497 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2b480cab80 msgr2=0x7f2b48150910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:35.325 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.319+0000 7f2b47fff640 1 --2- 172.21.15.169:0/2117294497 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2b480cab80 0x7f2b48150910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:35.326 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.319+0000 7f2b47fff640 1 -- 172.21.15.169:0/2117294497 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2b480cc980 msgr2=0x7f2b48144370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:35.326 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.319+0000 7f2b47fff640 1 --2- 172.21.15.169:0/2117294497 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2b480cc980 0x7f2b48144370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:35.326 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.319+0000 7f2b47fff640 1 -- 172.21.15.169:0/2117294497 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b30009560 con 0x7f2b480ca7a0 2023-10-26T01:35:35.326 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.319+0000 7f2b4cbc3640 1 --2- 172.21.15.169:0/2117294497 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2b480cc980 0x7f2b48144370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:35:35.326 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.320+0000 7f2b47fff640 1 --2- 172.21.15.169:0/2117294497 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2b480ca7a0 0x7f2b480abc50 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f2b3800c3c0 tx=0x7f2b3800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:35:35.327 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.320+0000 7f2b457fa640 1 -- 172.21.15.169:0/2117294497 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b380052d0 con 0x7f2b480ca7a0 2023-10-26T01:35:35.327 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.320+0000 7f2b457fa640 1 -- 172.21.15.169:0/2117294497 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2b38007020 con 0x7f2b480ca7a0 2023-10-26T01:35:35.327 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.320+0000 7f2b457fa640 1 -- 172.21.15.169:0/2117294497 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b38007470 con 0x7f2b480ca7a0 2023-10-26T01:35:35.327 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.320+0000 7f2b4d3c4640 1 -- 172.21.15.169:0/2117294497 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b48151a00 con 0x7f2b480ca7a0 2023-10-26T01:35:35.327 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.320+0000 7f2b4d3c4640 1 -- 172.21.15.169:0/2117294497 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b48150e50 con 0x7f2b480ca7a0 2023-10-26T01:35:35.328 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.321+0000 7f2b4d3c4640 1 -- 172.21.15.169:0/2117294497 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b48151f20 con 0x7f2b480ca7a0 2023-10-26T01:35:35.329 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.322+0000 7f2b457fa640 1 -- 172.21.15.169:0/2117294497 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f2b380067d0 con 0x7f2b480ca7a0 2023-10-26T01:35:35.329 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.323+0000 7f2b457fa640 1 --2- 172.21.15.169:0/2117294497 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2b2806ae40 0x7f2b2806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:35.329 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.323+0000 7f2b457fa640 1 -- 172.21.15.169:0/2117294497 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2b38088ba0 con 0x7f2b480ca7a0 2023-10-26T01:35:35.329 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.324+0000 7f2b3ffff640 1 --2- 172.21.15.169:0/2117294497 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2b2806ae40 0x7f2b2806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:35.329 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.325+0000 7f2b3ffff640 1 --2- 172.21.15.169:0/2117294497 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2b2806ae40 0x7f2b2806d300 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f2b30035000 tx=0x7f2b30004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:35:35.330 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.328+0000 7f2b457fa640 1 -- 172.21.15.169:0/2117294497 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2b38053ce0 con 0x7f2b480ca7a0 2023-10-26T01:35:35.552 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T01:35:35.561 DEBUG:teuthology.orchestra.run.smithi170:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T01:35:35.562 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.559+0000 7f2b4d3c4640 1 -- 172.21.15.169:0/2117294497 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f2b4809b540 con 0x7f2b480ca7a0 2023-10-26T01:35:35.563 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.560+0000 7f2b457fa640 1 -- 172.21.15.169:0/2117294497 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v42) v1 ==== 53+0+73 (secure 0 0 0) 0x7f2b38057300 con 0x7f2b480ca7a0 2023-10-26T01:35:35.563 INFO:teuthology.orchestra.run.smithi169.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-26T01:35:35.565 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.562+0000 7f2b4d3c4640 1 -- 172.21.15.169:0/2117294497 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2b2806ae40 msgr2=0x7f2b2806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:35.565 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.562+0000 7f2b4d3c4640 1 --2- 172.21.15.169:0/2117294497 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2b2806ae40 0x7f2b2806d300 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f2b30035000 tx=0x7f2b30004730 comp rx=0 tx=0).stop 2023-10-26T01:35:35.565 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.562+0000 7f2b4d3c4640 1 -- 172.21.15.169:0/2117294497 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2b480ca7a0 msgr2=0x7f2b480abc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:35.566 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.562+0000 7f2b4d3c4640 1 --2- 172.21.15.169:0/2117294497 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2b480ca7a0 0x7f2b480abc50 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f2b3800c3c0 tx=0x7f2b3800c880 comp rx=0 tx=0).stop 2023-10-26T01:35:35.566 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.562+0000 7f2b4d3c4640 1 -- 172.21.15.169:0/2117294497 shutdown_connections 2023-10-26T01:35:35.566 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.562+0000 7f2b4d3c4640 1 --2- 172.21.15.169:0/2117294497 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2b2806ae40 0x7f2b2806d300 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:35.566 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.562+0000 7f2b4d3c4640 1 --2- 172.21.15.169:0/2117294497 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2b480cc980 0x7f2b48144370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:35.567 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.562+0000 7f2b4d3c4640 1 --2- 172.21.15.169:0/2117294497 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2b480cab80 0x7f2b48150910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:35.567 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.562+0000 7f2b4d3c4640 1 --2- 172.21.15.169:0/2117294497 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2b480ca7a0 0x7f2b480abc50 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:35.567 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.562+0000 7f2b4d3c4640 1 -- 172.21.15.169:0/2117294497 >> 172.21.15.169:0/2117294497 conn(0x7f2b480af0c0 msgr2=0x7f2b480af4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:35:35.567 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.563+0000 7f2b4d3c4640 1 -- 172.21.15.169:0/2117294497 shutdown_connections 2023-10-26T01:35:35.567 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:35.563+0000 7f2b4d3c4640 1 -- 172.21.15.169:0/2117294497 wait complete. 2023-10-26T01:35:35.582 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-26T01:35:35.582 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-26T01:35:35.583 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi169.front.sepia.ceph.com 2023-10-26T01:35:35.583 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-26T01:35:35.583 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-26T01:35:35.584 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-26T01:35:35.584 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-26T01:35:35.584 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-26T01:35:35.585 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-26T01:35:35.585 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:35:35.586 DEBUG:teuthology.orchestra.run.smithi169:> ip addr 2023-10-26T01:35:35.675 INFO:teuthology.orchestra.run.smithi169.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-26T01:35:35.676 INFO:teuthology.orchestra.run.smithi169.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-26T01:35:35.676 INFO:teuthology.orchestra.run.smithi169.stdout: inet 127.0.0.1/8 scope host lo 2023-10-26T01:35:35.676 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:35:35.676 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 ::1/128 scope host 2023-10-26T01:35:35.676 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:35:35.677 INFO:teuthology.orchestra.run.smithi169.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-26T01:35:35.677 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:d9:3b:b4 brd ff:ff:ff:ff:ff:ff 2023-10-26T01:35:35.677 INFO:teuthology.orchestra.run.smithi169.stdout: altname enp7s0f0 2023-10-26T01:35:35.677 INFO:teuthology.orchestra.run.smithi169.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-26T01:35:35.677 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:d9:3b:b5 brd ff:ff:ff:ff:ff:ff 2023-10-26T01:35:35.678 INFO:teuthology.orchestra.run.smithi169.stdout: altname enp7s0f1 2023-10-26T01:35:35.678 INFO:teuthology.orchestra.run.smithi169.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-26T01:35:35.678 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:88:7f:4c brd ff:ff:ff:ff:ff:ff 2023-10-26T01:35:35.678 INFO:teuthology.orchestra.run.smithi169.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-26T01:35:35.678 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:88:7f:4d brd ff:ff:ff:ff:ff:ff 2023-10-26T01:35:35.679 INFO:teuthology.orchestra.run.smithi169.stdout: inet 172.21.15.169/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-26T01:35:35.679 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft 42160sec preferred_lft 42160sec 2023-10-26T01:35:35.679 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 fe80::ec4:7aff:fe88:7f4d/64 scope link 2023-10-26T01:35:35.679 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:35:35.679 INFO:teuthology.orchestra.run.smithi169.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-26T01:35:35.680 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 22:a0:6b:43:7b:04 brd ff:ff:ff:ff:ff:ff 2023-10-26T01:35:35.680 INFO:teuthology.orchestra.run.smithi169.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-26T01:35:35.680 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:35:35.680 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 fe80::d815:31ff:fe87:d148/64 scope link 2023-10-26T01:35:35.680 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:35:35.680 INFO:teuthology.orchestra.run.smithi169.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-26T01:35:35.681 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 2a:80:f8:45:5f:9f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-26T01:35:35.681 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 fe80::2880:f8ff:fe45:5f9f/64 scope link 2023-10-26T01:35:35.681 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:35:35.681 INFO:teuthology.orchestra.run.smithi169.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-26T01:35:35.681 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 22:a0:6b:43:7b:04 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-26T01:35:35.682 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 fe80::20a0:6bff:fe43:7b04/64 scope link 2023-10-26T01:35:35.682 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:35:35.682 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:35:35.683 DEBUG:teuthology.orchestra.run.smithi169:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-26T01:35:35.767 INFO:teuthology.orchestra.run.smithi169.stdout:1 2023-10-26T01:35:35.771 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:35:35.771 DEBUG:teuthology.orchestra.run.smithi169:> route 2023-10-26T01:35:35.824 INFO:teuthology.orchestra.run.smithi169.stdout:Kernel IP routing table 2023-10-26T01:35:35.824 INFO:teuthology.orchestra.run.smithi169.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-26T01:35:35.825 INFO:teuthology.orchestra.run.smithi169.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-26T01:35:35.825 INFO:teuthology.orchestra.run.smithi169.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-26T01:35:35.825 INFO:teuthology.orchestra.run.smithi169.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-26T01:35:35.825 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:35:35.826 DEBUG:teuthology.orchestra.run.smithi169:> (cd / && exec bash -c ' 2023-10-26T01:35:35.826 DEBUG:teuthology.orchestra.run.smithi169:> set -e 2023-10-26T01:35:35.827 DEBUG:teuthology.orchestra.run.smithi169:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-26T01:35:35.827 DEBUG:teuthology.orchestra.run.smithi169:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-26T01:35:35.828 DEBUG:teuthology.orchestra.run.smithi169:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-26T01:35:35.828 DEBUG:teuthology.orchestra.run.smithi169:> ') 2023-10-26T01:35:36.015 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:35:36.016 DEBUG:teuthology.orchestra.run.smithi169:> ip netns list 2023-10-26T01:35:36.069 INFO:teuthology.orchestra.run.smithi169.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-26T01:35:36.069 INFO:teuthology.orchestra.run.smithi169.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-26T01:35:36.070 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-26T01:35:36.070 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:35:36.071 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link set brx.0 up 2023-10-26T01:35:36.161 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-26T01:35:36.162 DEBUG:teuthology.orchestra.run.smithi169:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-26T01:35:36.216 INFO:teuthology.orchestra.run.smithi169.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-26T01:35:36.217 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-26T01:35:36.217 DEBUG:teuthology.orchestra.run.smithi169:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-26T01:35:36.270 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-26T01:35:36.271 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:35:36.271 DEBUG:teuthology.orchestra.run.smithi169:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-26T01:35:36.419 INFO:teuthology.orchestra.run.smithi169.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-26T01:35:36.419 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-26T01:35:36.420 INFO:teuthology.orchestra.run.smithi169.stdout:invalid new device string format 2023-10-26T01:35:36.441 INFO:teuthology.orchestra.run.smithi169.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-26T01:35:36.442 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-26T01:35:36.442 INFO:teuthology.orchestra.run.smithi169.stdout:invalid new device string format 2023-10-26T01:35:36.442 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: resolved to: "172.21.15.169:3300,172.21.15.170:3300,172.21.15.170:3301" 2023-10-26T01:35:36.442 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-26T01:35:36.465 INFO:teuthology.orchestra.run.smithi169.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-26T01:35:36.465 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-26T01:35:36.465 INFO:teuthology.orchestra.run.smithi169.stdout:invalid new device string format 2023-10-26T01:35:36.465 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: resolved to: "172.21.15.169:3300,172.21.15.170:3300,172.21.15.170:3301" 2023-10-26T01:35:36.466 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-26T01:35:36.466 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: resolved to: "172.21.15.169:6789,172.21.15.170:6789,172.21.15.170:6790" 2023-10-26T01:35:36.466 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.169:6789,172.21.15.170:6789,172.21.15.170:6790:/ 2023-10-26T01:35:36.466 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=30203b28-ed8a-44e8-a9a6-a9bf25f980f1" will pass to kernel 2023-10-26T01:35:36.467 INFO:teuthology.orchestra.run.smithi169.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2023-10-26T01:35:36.467 INFO:teuthology.orchestra.run.smithi169.stdout: You just mounted a file system that supports labels which does not 2023-10-26T01:35:36.467 INFO:teuthology.orchestra.run.smithi169.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-10-26T01:35:36.467 INFO:teuthology.orchestra.run.smithi169.stdout: applications will generate AVC messages and not be allowed access to 2023-10-26T01:35:36.467 INFO:teuthology.orchestra.run.smithi169.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-10-26T01:35:36.470 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-26T01:35:36.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:35:36.471 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-26T01:35:36.551 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:35:36.552 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-26T01:35:36.614 INFO:teuthology.orchestra.run.smithi169.stdout:client5141 2023-10-26T01:35:36.614 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:35:36.615 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:35:36.662 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-26T01:35:36.927 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:36.924+0000 7f7afcde2640 1 -- 172.21.15.169:0/4083523453 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7af80c9d00 msgr2=0x7f7af80cdc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:36.927 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:36.924+0000 7f7afcde2640 1 --2- 172.21.15.169:0/4083523453 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7af80c9d00 0x7f7af80cdc50 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7ae8009830 tx=0x7f7ae802bd50 comp rx=0 tx=0).stop 2023-10-26T01:35:36.928 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:36.924+0000 7f7afcde2640 1 -- 172.21.15.169:0/4083523453 shutdown_connections 2023-10-26T01:35:36.928 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:36.924+0000 7f7afcde2640 1 --2- 172.21.15.169:0/4083523453 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7af80c9d00 0x7f7af80cdc50 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:36.928 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:36.924+0000 7f7afcde2640 1 --2- 172.21.15.169:0/4083523453 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7af815b620 0x7f7af80cd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:36.928 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:36.924+0000 7f7afcde2640 1 -- 172.21.15.169:0/4083523453 >> 172.21.15.169:0/4083523453 conn(0x7f7af8069930 msgr2=0x7f7af8069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:35:36.928 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:36.924+0000 7f7afcde2640 1 -- 172.21.15.169:0/4083523453 shutdown_connections 2023-10-26T01:35:36.929 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:36.924+0000 7f7afcde2640 1 -- 172.21.15.169:0/4083523453 wait complete. 2023-10-26T01:35:36.931 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:36.926+0000 7f7afcde2640 1 Processor -- start 2023-10-26T01:35:36.931 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:36.926+0000 7f7afcde2640 1 -- start start 2023-10-26T01:35:36.931 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:36.927+0000 7f7afcde2640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7af80ac320 0x7f7af815bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:36.931 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:36.927+0000 7f7afcde2640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7af80c9d00 0x7f7af8149e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:36.932 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:36.927+0000 7f7afcde2640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7af815b620 0x7f7af80b0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:36.932 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:36.927+0000 7f7afcde2640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7af80c9430 con 0x7f7af80c9d00 2023-10-26T01:35:36.932 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:36.927+0000 7f7afcde2640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7af80b0e70 con 0x7f7af80ac320 2023-10-26T01:35:36.932 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:36.927+0000 7f7afcde2640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7af80ad640 con 0x7f7af815b620 2023-10-26T01:35:36.932 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:36.927+0000 7f7af5d9b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7af80c9d00 0x7f7af8149e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:36.933 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:36.927+0000 7f7af5d9b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7af80c9d00 0x7f7af8149e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:54572/0 (socket says 172.21.15.169:54572) 2023-10-26T01:35:36.933 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:36.927+0000 7f7af5d9b640 1 -- 172.21.15.169:0/687035554 learned_addr learned my addr 172.21.15.169:0/687035554 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:35:36.933 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:36.927+0000 7f7af659c640 1 --2- 172.21.15.169:0/687035554 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7af80ac320 0x7f7af815bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:36.933 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:36.927+0000 7f7af6d9d640 1 --2- 172.21.15.169:0/687035554 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7af815b620 0x7f7af80b0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:36.933 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:36.927+0000 7f7af5d9b640 1 -- 172.21.15.169:0/687035554 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7af815b620 msgr2=0x7f7af80b0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:36.934 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:36.928+0000 7f7af5d9b640 1 --2- 172.21.15.169:0/687035554 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7af815b620 0x7f7af80b0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:36.934 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:36.928+0000 7f7af5d9b640 1 -- 172.21.15.169:0/687035554 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7af80ac320 msgr2=0x7f7af815bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:36.934 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:36.928+0000 7f7af5d9b640 1 --2- 172.21.15.169:0/687035554 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7af80ac320 0x7f7af815bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:36.935 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:36.928+0000 7f7af5d9b640 1 -- 172.21.15.169:0/687035554 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ae0009490 con 0x7f7af80c9d00 2023-10-26T01:35:36.935 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:36.928+0000 7f7af659c640 1 --2- 172.21.15.169:0/687035554 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7af80ac320 0x7f7af815bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:35:36.935 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:36.928+0000 7f7af6d9d640 1 --2- 172.21.15.169:0/687035554 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7af815b620 0x7f7af80b0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:35:36.935 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:36.928+0000 7f7af5d9b640 1 --2- 172.21.15.169:0/687035554 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7af80c9d00 0x7f7af8149e70 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f7ae8009780 tx=0x7f7ae80038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:35:36.936 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:36.928+0000 7f7aeeffd640 1 -- 172.21.15.169:0/687035554 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae8003ea0 con 0x7f7af80c9d00 2023-10-26T01:35:36.936 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:36.928+0000 7f7aeeffd640 1 -- 172.21.15.169:0/687035554 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7ae8034440 con 0x7f7af80c9d00 2023-10-26T01:35:36.936 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:36.928+0000 7f7afcde2640 1 -- 172.21.15.169:0/687035554 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ae8009490 con 0x7f7af80c9d00 2023-10-26T01:35:36.936 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:36.928+0000 7f7aeeffd640 1 -- 172.21.15.169:0/687035554 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae8034850 con 0x7f7af80c9d00 2023-10-26T01:35:36.936 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:36.928+0000 7f7afcde2640 1 -- 172.21.15.169:0/687035554 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7af80c8960 con 0x7f7af80c9d00 2023-10-26T01:35:36.937 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:36.929+0000 7f7afcde2640 1 -- 172.21.15.169:0/687035554 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7af80ad7d0 con 0x7f7af80c9d00 2023-10-26T01:35:36.937 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:36.930+0000 7f7aeeffd640 1 -- 172.21.15.169:0/687035554 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f7ae803b040 con 0x7f7af80c9d00 2023-10-26T01:35:36.938 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:36.931+0000 7f7aeeffd640 1 --2- 172.21.15.169:0/687035554 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7ad806ae40 0x7f7ad806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:36.938 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:36.931+0000 7f7aeeffd640 1 -- 172.21.15.169:0/687035554 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7ae80abc00 con 0x7f7af80c9d00 2023-10-26T01:35:36.938 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:36.932+0000 7f7af659c640 1 --2- 172.21.15.169:0/687035554 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7ad806ae40 0x7f7ad806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:36.938 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:36.933+0000 7f7af659c640 1 --2- 172.21.15.169:0/687035554 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7ad806ae40 0x7f7ad806d300 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f7ae0009c40 tx=0x7f7ae0005f90 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:35:36.939 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:36.936+0000 7f7aeeffd640 1 -- 172.21.15.169:0/687035554 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7ae8076c10 con 0x7f7af80c9d00 2023-10-26T01:35:37.166 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:37.163+0000 7f7afcde2640 1 -- 172.21.15.169:0/687035554 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f7af804db10 con 0x7f7af80c9d00 2023-10-26T01:35:37.166 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:37.164+0000 7f7aeeffd640 1 -- 172.21.15.169:0/687035554 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v42) v1 ==== 53+0+73 (secure 0 0 0) 0x7f7ae8039070 con 0x7f7af80c9d00 2023-10-26T01:35:37.167 INFO:teuthology.orchestra.run.smithi169.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-26T01:35:37.169 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:37.166+0000 7f7afcde2640 1 -- 172.21.15.169:0/687035554 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7ad806ae40 msgr2=0x7f7ad806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:37.169 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:37.166+0000 7f7afcde2640 1 --2- 172.21.15.169:0/687035554 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7ad806ae40 0x7f7ad806d300 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f7ae0009c40 tx=0x7f7ae0005f90 comp rx=0 tx=0).stop 2023-10-26T01:35:37.170 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:37.166+0000 7f7afcde2640 1 -- 172.21.15.169:0/687035554 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7af80c9d00 msgr2=0x7f7af8149e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:37.170 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:37.166+0000 7f7afcde2640 1 --2- 172.21.15.169:0/687035554 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7af80c9d00 0x7f7af8149e70 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f7ae8009780 tx=0x7f7ae80038a0 comp rx=0 tx=0).stop 2023-10-26T01:35:37.170 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:37.167+0000 7f7afcde2640 1 -- 172.21.15.169:0/687035554 shutdown_connections 2023-10-26T01:35:37.170 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:37.167+0000 7f7afcde2640 1 --2- 172.21.15.169:0/687035554 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7ad806ae40 0x7f7ad806d300 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:37.171 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:37.167+0000 7f7afcde2640 1 --2- 172.21.15.169:0/687035554 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7af80c9d00 0x7f7af8149e70 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:37.171 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:37.167+0000 7f7afcde2640 1 --2- 172.21.15.169:0/687035554 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7af815b620 0x7f7af80b0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:37.171 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:37.167+0000 7f7afcde2640 1 --2- 172.21.15.169:0/687035554 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7af80ac320 0x7f7af815bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:37.171 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:37.167+0000 7f7afcde2640 1 -- 172.21.15.169:0/687035554 >> 172.21.15.169:0/687035554 conn(0x7f7af8069930 msgr2=0x7f7af8154b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:35:37.171 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:37.167+0000 7f7afcde2640 1 -- 172.21.15.169:0/687035554 shutdown_connections 2023-10-26T01:35:37.172 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:37.167+0000 7f7afcde2640 1 -- 172.21.15.169:0/687035554 wait complete. 2023-10-26T01:35:37.189 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-26T01:35:37.189 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-26T01:35:37.189 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi170.front.sepia.ceph.com 2023-10-26T01:35:37.190 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-26T01:35:37.190 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-26T01:35:37.190 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-26T01:35:37.191 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-26T01:35:37.191 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-26T01:35:37.191 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-26T01:35:37.192 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:35:37.192 DEBUG:teuthology.orchestra.run.smithi170:> ip addr 2023-10-26T01:35:37.252 INFO:teuthology.orchestra.run.smithi170.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-26T01:35:37.253 INFO:teuthology.orchestra.run.smithi170.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-26T01:35:37.253 INFO:teuthology.orchestra.run.smithi170.stdout: inet 127.0.0.1/8 scope host lo 2023-10-26T01:35:37.253 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:35:37.253 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 ::1/128 scope host 2023-10-26T01:35:37.254 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:35:37.254 INFO:teuthology.orchestra.run.smithi170.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-26T01:35:37.254 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:d9:3b:54 brd ff:ff:ff:ff:ff:ff 2023-10-26T01:35:37.254 INFO:teuthology.orchestra.run.smithi170.stdout: altname enp7s0f0 2023-10-26T01:35:37.254 INFO:teuthology.orchestra.run.smithi170.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-26T01:35:37.255 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:d9:3b:55 brd ff:ff:ff:ff:ff:ff 2023-10-26T01:35:37.255 INFO:teuthology.orchestra.run.smithi170.stdout: altname enp7s0f1 2023-10-26T01:35:37.255 INFO:teuthology.orchestra.run.smithi170.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-26T01:35:37.255 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:88:72:54 brd ff:ff:ff:ff:ff:ff 2023-10-26T01:35:37.255 INFO:teuthology.orchestra.run.smithi170.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-26T01:35:37.255 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:88:72:55 brd ff:ff:ff:ff:ff:ff 2023-10-26T01:35:37.256 INFO:teuthology.orchestra.run.smithi170.stdout: inet 172.21.15.170/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-26T01:35:37.256 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft 42143sec preferred_lft 42143sec 2023-10-26T01:35:37.256 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::ec4:7aff:fe88:7255/64 scope link 2023-10-26T01:35:37.256 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:35:37.256 INFO:teuthology.orchestra.run.smithi170.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-26T01:35:37.257 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 32:b5:d1:ee:10:36 brd ff:ff:ff:ff:ff:ff 2023-10-26T01:35:37.257 INFO:teuthology.orchestra.run.smithi170.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-26T01:35:37.257 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:35:37.257 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::28ed:d1ff:fe37:70d/64 scope link 2023-10-26T01:35:37.258 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:35:37.258 INFO:teuthology.orchestra.run.smithi170.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-26T01:35:37.258 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 32:b5:d1:ee:10:36 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-26T01:35:37.258 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::30b5:d1ff:feee:1036/64 scope link 2023-10-26T01:35:37.258 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:35:37.258 INFO:teuthology.orchestra.run.smithi170.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-26T01:35:37.259 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether b2:c1:53:19:69:e4 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-26T01:35:37.259 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::b0c1:53ff:fe19:69e4/64 scope link 2023-10-26T01:35:37.259 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:35:37.260 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:35:37.260 DEBUG:teuthology.orchestra.run.smithi170:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-26T01:35:37.356 INFO:teuthology.orchestra.run.smithi170.stdout:1 2023-10-26T01:35:37.360 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:35:37.361 DEBUG:teuthology.orchestra.run.smithi170:> route 2023-10-26T01:35:37.416 INFO:teuthology.orchestra.run.smithi170.stdout:Kernel IP routing table 2023-10-26T01:35:37.416 INFO:teuthology.orchestra.run.smithi170.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-26T01:35:37.416 INFO:teuthology.orchestra.run.smithi170.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-26T01:35:37.416 INFO:teuthology.orchestra.run.smithi170.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-26T01:35:37.417 INFO:teuthology.orchestra.run.smithi170.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-26T01:35:37.418 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:35:37.418 DEBUG:teuthology.orchestra.run.smithi170:> (cd / && exec bash -c ' 2023-10-26T01:35:37.419 DEBUG:teuthology.orchestra.run.smithi170:> set -e 2023-10-26T01:35:37.419 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-26T01:35:37.419 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-26T01:35:37.420 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-26T01:35:37.420 DEBUG:teuthology.orchestra.run.smithi170:> ') 2023-10-26T01:35:37.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:35:37.604 DEBUG:teuthology.orchestra.run.smithi170:> ip netns list 2023-10-26T01:35:37.656 INFO:teuthology.orchestra.run.smithi170.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-26T01:35:37.656 INFO:teuthology.orchestra.run.smithi170.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-26T01:35:37.657 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-26T01:35:37.657 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:35:37.658 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip link set brx.0 up 2023-10-26T01:35:37.753 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-26T01:35:37.754 DEBUG:teuthology.orchestra.run.smithi170:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-26T01:35:37.811 INFO:teuthology.orchestra.run.smithi170.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-26T01:35:37.812 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-26T01:35:37.812 DEBUG:teuthology.orchestra.run.smithi170:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-26T01:35:37.862 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-26T01:35:37.862 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:35:37.863 DEBUG:teuthology.orchestra.run.smithi170:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-26T01:35:37.999 INFO:teuthology.orchestra.run.smithi170.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-26T01:35:37.999 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-26T01:35:37.999 INFO:teuthology.orchestra.run.smithi170.stdout:invalid new device string format 2023-10-26T01:35:38.021 INFO:teuthology.orchestra.run.smithi170.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-26T01:35:38.021 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-26T01:35:38.021 INFO:teuthology.orchestra.run.smithi170.stdout:invalid new device string format 2023-10-26T01:35:38.021 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: resolved to: "172.21.15.169:3300,172.21.15.170:3300,172.21.15.170:3301" 2023-10-26T01:35:38.021 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-26T01:35:38.046 INFO:teuthology.orchestra.run.smithi170.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-26T01:35:38.047 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-26T01:35:38.047 INFO:teuthology.orchestra.run.smithi170.stdout:invalid new device string format 2023-10-26T01:35:38.047 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: resolved to: "172.21.15.169:3300,172.21.15.170:3300,172.21.15.170:3301" 2023-10-26T01:35:38.047 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-26T01:35:38.047 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: resolved to: "172.21.15.169:6789,172.21.15.170:6789,172.21.15.170:6790" 2023-10-26T01:35:38.048 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.169:6789,172.21.15.170:6789,172.21.15.170:6790:/ 2023-10-26T01:35:38.048 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=30203b28-ed8a-44e8-a9a6-a9bf25f980f1" will pass to kernel 2023-10-26T01:35:38.048 INFO:teuthology.orchestra.run.smithi170.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2023-10-26T01:35:38.048 INFO:teuthology.orchestra.run.smithi170.stdout: You just mounted a file system that supports labels which does not 2023-10-26T01:35:38.049 INFO:teuthology.orchestra.run.smithi170.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-10-26T01:35:38.049 INFO:teuthology.orchestra.run.smithi170.stdout: applications will generate AVC messages and not be allowed access to 2023-10-26T01:35:38.049 INFO:teuthology.orchestra.run.smithi170.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-10-26T01:35:38.052 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-26T01:35:38.053 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:35:38.053 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-26T01:35:38.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:35:38.141 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-26T01:35:38.200 INFO:teuthology.orchestra.run.smithi170.stdout:client5147 2023-10-26T01:35:38.200 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-26T01:35:38.201 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:35:38.253 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T01:35:38.485 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:38.481+0000 7f35cb9bb640 1 -- 172.21.15.169:0/4163797031 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35ac003f70 msgr2=0x7f35ac0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:38.485 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:38.481+0000 7f35cb9bb640 1 --2- 172.21.15.169:0/4163797031 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35ac003f70 0x7f35ac0047c0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f35b8006f00 tx=0x7f35ac0057f0 comp rx=0 tx=0).stop 2023-10-26T01:35:38.485 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:38.481+0000 7f35c9f58640 1 -- 172.21.15.169:0/4163797031 reap_dead start 2023-10-26T01:35:38.485 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:38.482+0000 7f35cb9bb640 1 -- 172.21.15.169:0/4163797031 shutdown_connections 2023-10-26T01:35:38.486 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:38.482+0000 7f35cb9bb640 1 -- 172.21.15.169:0/4163797031 >> 172.21.15.169:0/4163797031 conn(0x7f35c4069930 msgr2=0x7f35c4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:35:38.486 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:38.482+0000 7f35cb9bb640 1 -- 172.21.15.169:0/4163797031 shutdown_connections 2023-10-26T01:35:38.486 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:38.482+0000 7f35cb9bb640 1 -- 172.21.15.169:0/4163797031 wait complete. 2023-10-26T01:35:38.487 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:38.483+0000 7f35cb9bb640 1 Processor -- start 2023-10-26T01:35:38.487 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:38.484+0000 7f35cb9bb640 1 -- start start 2023-10-26T01:35:38.488 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:38.484+0000 7f35cb9bb640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f35ac003f70 0x7f35c41e46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:38.488 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:38.484+0000 7f35cb9bb640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f35c41535c0 0x7f35c414e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:38.488 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:38.484+0000 7f35cb9bb640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35c415ba10 0x7f35c415be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:38.488 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:38.484+0000 7f35cb9bb640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35c415c3d0 con 0x7f35c415ba10 2023-10-26T01:35:38.488 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:38.484+0000 7f35cb9bb640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35c4153b30 con 0x7f35c41535c0 2023-10-26T01:35:38.488 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:38.484+0000 7f35cb9bb640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35c4153e10 con 0x7f35ac003f70 2023-10-26T01:35:38.489 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:38.485+0000 7f35c9f58640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35c415ba10 0x7f35c415be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:38.489 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:38.485+0000 7f35c9f58640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35c415ba10 0x7f35c415be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:54588/0 (socket says 172.21.15.169:54588) 2023-10-26T01:35:38.489 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:38.485+0000 7f35c9f58640 1 -- 172.21.15.169:0/1925566536 learned_addr learned my addr 172.21.15.169:0/1925566536 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:35:38.491 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:38.485+0000 7f35c9757640 1 --2- 172.21.15.169:0/1925566536 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f35ac003f70 0x7f35c41e46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:38.491 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:38.485+0000 7f35c8f56640 1 --2- 172.21.15.169:0/1925566536 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f35c41535c0 0x7f35c414e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:38.491 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:38.485+0000 7f35c9f58640 1 -- 172.21.15.169:0/1925566536 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f35ac003f70 msgr2=0x7f35c41e46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:38.491 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:38.485+0000 7f35c9f58640 1 --2- 172.21.15.169:0/1925566536 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f35ac003f70 0x7f35c41e46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:38.491 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:38.485+0000 7f35c9f58640 1 -- 172.21.15.169:0/1925566536 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f35c41535c0 msgr2=0x7f35c414e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:38.492 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:38.485+0000 7f35c9f58640 1 --2- 172.21.15.169:0/1925566536 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f35c41535c0 0x7f35c414e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:38.492 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:38.485+0000 7f35c9f58640 1 -- 172.21.15.169:0/1925566536 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35c41540f0 con 0x7f35c415ba10 2023-10-26T01:35:38.492 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:38.485+0000 7f35c9757640 1 --2- 172.21.15.169:0/1925566536 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f35ac003f70 0x7f35c41e46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:35:38.492 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:38.485+0000 7f35c9f58640 1 --2- 172.21.15.169:0/1925566536 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35c415ba10 0x7f35c415be90 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f35c40b2680 tx=0x7f35bc00e680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:35:38.492 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:38.486+0000 7f35c1ffb640 1 -- 172.21.15.169:0/1925566536 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35bc00c070 con 0x7f35c415ba10 2023-10-26T01:35:38.493 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:38.486+0000 7f35c1ffb640 1 -- 172.21.15.169:0/1925566536 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f35bc013610 con 0x7f35c415ba10 2023-10-26T01:35:38.493 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:38.486+0000 7f35c1ffb640 1 -- 172.21.15.169:0/1925566536 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35bc013a20 con 0x7f35c415ba10 2023-10-26T01:35:38.493 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:38.486+0000 7f35cb9bb640 1 -- 172.21.15.169:0/1925566536 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35c41543d0 con 0x7f35c415ba10 2023-10-26T01:35:38.493 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:38.486+0000 7f35cb9bb640 1 -- 172.21.15.169:0/1925566536 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35c41e4c30 con 0x7f35c415ba10 2023-10-26T01:35:38.493 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:38.487+0000 7f35cb9bb640 1 -- 172.21.15.169:0/1925566536 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35c404db10 con 0x7f35c415ba10 2023-10-26T01:35:38.494 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:38.488+0000 7f35c1ffb640 1 -- 172.21.15.169:0/1925566536 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f35bc01c3b0 con 0x7f35c415ba10 2023-10-26T01:35:38.494 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:38.489+0000 7f35c1ffb640 1 --2- 172.21.15.169:0/1925566536 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f35b006ae40 0x7f35b006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:35:38.495 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:38.489+0000 7f35c1ffb640 1 -- 172.21.15.169:0/1925566536 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f35bc0886e0 con 0x7f35c415ba10 2023-10-26T01:35:38.495 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:38.489+0000 7f35c9757640 1 --2- 172.21.15.169:0/1925566536 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f35b006ae40 0x7f35b006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:35:38.495 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:38.490+0000 7f35c9757640 1 --2- 172.21.15.169:0/1925566536 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f35b006ae40 0x7f35b006d300 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f35b4005fd0 tx=0x7f35b4005ac0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:35:38.496 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:38.494+0000 7f35c1ffb640 1 -- 172.21.15.169:0/1925566536 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f35bc053790 con 0x7f35c415ba10 2023-10-26T01:35:38.724 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:38.721+0000 7f35cb9bb640 1 -- 172.21.15.169:0/1925566536 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f35c40b22d0 con 0x7f35c415ba10 2023-10-26T01:35:38.725 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:38.722+0000 7f35c1ffb640 1 -- 172.21.15.169:0/1925566536 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+4927 (secure 0 0 0) 0x7f35bc018090 con 0x7f35c415ba10 2023-10-26T01:35:38.725 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:35:38.726 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":42,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5347,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/2629782429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2629782429},{"type":"v1","addr":"172.21.15.170:6836","nonce":2629782429}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":6474,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2561200417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2561200417},{"type":"v1","addr":"172.21.15.169:6833","nonce":2561200417}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":6480,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/2834878721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":2834878721},{"type":"v1","addr":"172.21.15.170:6837","nonce":2834878721}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T01:35:27.834145+0000","modified":"2023-10-26T01:35:30.917014+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6486},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6486":{"gid":6486,"name":"a","rank":0,"incarnation":38,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6835/4004306080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":4004306080},{"type":"v1","addr":"172.21.15.169:6835","nonce":4004306080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-10-26T01:35:38.726 INFO:teuthology.orchestra.run.smithi169.stderr:dumped fsmap epoch 42 2023-10-26T01:35:38.728 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:38.725+0000 7f35cb9bb640 1 -- 172.21.15.169:0/1925566536 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f35b006ae40 msgr2=0x7f35b006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:38.728 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:38.725+0000 7f35cb9bb640 1 --2- 172.21.15.169:0/1925566536 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f35b006ae40 0x7f35b006d300 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f35b4005fd0 tx=0x7f35b4005ac0 comp rx=0 tx=0).stop 2023-10-26T01:35:38.729 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:38.725+0000 7f35cb9bb640 1 -- 172.21.15.169:0/1925566536 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35c415ba10 msgr2=0x7f35c415be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:35:38.729 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:38.725+0000 7f35cb9bb640 1 --2- 172.21.15.169:0/1925566536 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35c415ba10 0x7f35c415be90 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f35c40b2680 tx=0x7f35bc00e680 comp rx=0 tx=0).stop 2023-10-26T01:35:38.729 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:38.726+0000 7f35cb9bb640 1 -- 172.21.15.169:0/1925566536 shutdown_connections 2023-10-26T01:35:38.729 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:38.726+0000 7f35cb9bb640 1 --2- 172.21.15.169:0/1925566536 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f35b006ae40 0x7f35b006d300 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:38.729 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:38.726+0000 7f35cb9bb640 1 --2- 172.21.15.169:0/1925566536 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35c415ba10 0x7f35c415be90 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:38.730 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:38.726+0000 7f35cb9bb640 1 --2- 172.21.15.169:0/1925566536 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f35c41535c0 0x7f35c414e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:38.730 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:38.726+0000 7f35cb9bb640 1 --2- 172.21.15.169:0/1925566536 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f35ac003f70 0x7f35c41e46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:35:38.730 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:38.726+0000 7f35cb9bb640 1 -- 172.21.15.169:0/1925566536 >> 172.21.15.169:0/1925566536 conn(0x7f35c4069930 msgr2=0x7f35c414f140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:35:38.730 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:38.726+0000 7f35cb9bb640 1 -- 172.21.15.169:0/1925566536 shutdown_connections 2023-10-26T01:35:38.730 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:35:38.726+0000 7f35cb9bb640 1 -- 172.21.15.169:0/1925566536 wait complete. 2023-10-26T01:35:38.748 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2023-10-26T01:35:38.967 INFO:teuthology.orchestra.run.smithi169.stdout:{"cluster_fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","whoami":0,"id":6486,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":10.560649638999999,"mdsmap_epoch":42,"osdmap_epoch":84,"osdmap_epoch_barrier":84,"uptime":16.477284801} 2023-10-26T01:35:38.968 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "30203b28-ed8a-44e8-a9a6-a9bf25f980f1", "fs_name": "cephfs", "id": 6486, "mdsmap_epoch": 42, "osdmap_epoch": 84, "osdmap_epoch_barrier": 84, "rank_uptime": 10.560649639, "state": "up:active", "uptime": 16.477284801, "want_state": "up:active", "whoami": 0 } 2023-10-26T01:35:38.969 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2023-10-26T01:35:39.186 INFO:teuthology.orchestra.run.smithi169.stdout:{"osd_mon_report_interval":"5"} 2023-10-26T01:35:39.186 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2023-10-26T01:35:39.187 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mon_tick_interval 2023-10-26T01:35:39.409 INFO:teuthology.orchestra.run.smithi169.stdout:{"mon_tick_interval":"5"} 2023-10-26T01:35:39.410 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mon_tick_interval": "5" } 2023-10-26T01:35:39.410 INFO:tasks.cephfs.test_full:pool capacity 557012544, 531MB should be enough to fill it 2023-10-26T01:35:39.411 DEBUG:teuthology.orchestra.run.smithi169:> stdin-killer -- python3 -c ' 2023-10-26T01:35:39.411 DEBUG:teuthology.orchestra.run.smithi169:> import time 2023-10-26T01:35:39.411 DEBUG:teuthology.orchestra.run.smithi169:> import datetime 2023-10-26T01:35:39.412 DEBUG:teuthology.orchestra.run.smithi169:> import subprocess 2023-10-26T01:35:39.412 DEBUG:teuthology.orchestra.run.smithi169:> import os 2023-10-26T01:35:39.412 DEBUG:teuthology.orchestra.run.smithi169:> 2023-10-26T01:35:39.413 DEBUG:teuthology.orchestra.run.smithi169:> # Write some buffered data through before going full, all should be well 2023-10-26T01:35:39.413 DEBUG:teuthology.orchestra.run.smithi169:> print("writing some data through which we expect to succeed") 2023-10-26T01:35:39.413 DEBUG:teuthology.orchestra.run.smithi169:> bytes = 0 2023-10-26T01:35:39.414 DEBUG:teuthology.orchestra.run.smithi169:> f = os.open("/home/ubuntu/cephtest/mnt.0/full_test_file", os.O_WRONLY | os.O_CREAT) 2023-10-26T01:35:39.414 DEBUG:teuthology.orchestra.run.smithi169:> os.setxattr("/home/ubuntu/cephtest/mnt.0/full_test_file", '"'"'ceph.file.layout'"'"', b'"'"'stripe_unit=1048576 stripe_count=1 object_size=1048576'"'"') 2023-10-26T01:35:39.414 DEBUG:teuthology.orchestra.run.smithi169:> bytes += os.write(f, b'"'"'a'"'"' * 4096) 2023-10-26T01:35:39.415 DEBUG:teuthology.orchestra.run.smithi169:> os.fsync(f) 2023-10-26T01:35:39.415 DEBUG:teuthology.orchestra.run.smithi169:> print("fsync'"'"'ed data successfully, will now attempt to fill fs") 2023-10-26T01:35:39.415 DEBUG:teuthology.orchestra.run.smithi169:> 2023-10-26T01:35:39.416 DEBUG:teuthology.orchestra.run.smithi169:> # Okay, now we'"'"'re going to fill up the filesystem, and then keep 2023-10-26T01:35:39.416 DEBUG:teuthology.orchestra.run.smithi169:> # writing until we see an error from fsync. As long as we'"'"'re doing 2023-10-26T01:35:39.416 DEBUG:teuthology.orchestra.run.smithi169:> # buffered IO, the error should always only appear from fsync and not 2023-10-26T01:35:39.417 DEBUG:teuthology.orchestra.run.smithi169:> # from write 2023-10-26T01:35:39.417 DEBUG:teuthology.orchestra.run.smithi169:> full = False 2023-10-26T01:35:39.417 DEBUG:teuthology.orchestra.run.smithi169:> 2023-10-26T01:35:39.418 DEBUG:teuthology.orchestra.run.smithi169:> for n in range(0, int(531 * 1.1)): 2023-10-26T01:35:39.418 DEBUG:teuthology.orchestra.run.smithi169:> try: 2023-10-26T01:35:39.418 DEBUG:teuthology.orchestra.run.smithi169:> bytes += os.write(f, b'"'"'x'"'"' * 1024 * 1024) 2023-10-26T01:35:39.418 DEBUG:teuthology.orchestra.run.smithi169:> print("wrote bytes via buffered write, moving on to fsync") 2023-10-26T01:35:39.419 DEBUG:teuthology.orchestra.run.smithi169:> except OSError as e: 2023-10-26T01:35:39.419 DEBUG:teuthology.orchestra.run.smithi169:> if False: 2023-10-26T01:35:39.420 DEBUG:teuthology.orchestra.run.smithi169:> print("Unexpected error %s from write() instead of fsync()" % e) 2023-10-26T01:35:39.420 DEBUG:teuthology.orchestra.run.smithi169:> raise 2023-10-26T01:35:39.420 DEBUG:teuthology.orchestra.run.smithi169:> else: 2023-10-26T01:35:39.420 DEBUG:teuthology.orchestra.run.smithi169:> print("Reached fullness after %.2f MB" % (bytes / (1024.0 * 1024.0))) 2023-10-26T01:35:39.421 DEBUG:teuthology.orchestra.run.smithi169:> full = True 2023-10-26T01:35:39.421 DEBUG:teuthology.orchestra.run.smithi169:> break 2023-10-26T01:35:39.421 DEBUG:teuthology.orchestra.run.smithi169:> 2023-10-26T01:35:39.422 DEBUG:teuthology.orchestra.run.smithi169:> try: 2023-10-26T01:35:39.422 DEBUG:teuthology.orchestra.run.smithi169:> os.fsync(f) 2023-10-26T01:35:39.422 DEBUG:teuthology.orchestra.run.smithi169:> print("fsync'"'"'ed successfully") 2023-10-26T01:35:39.423 DEBUG:teuthology.orchestra.run.smithi169:> except OSError as e: 2023-10-26T01:35:39.423 DEBUG:teuthology.orchestra.run.smithi169:> print("Reached fullness after %.2f MB" % (bytes / (1024.0 * 1024.0))) 2023-10-26T01:35:39.423 DEBUG:teuthology.orchestra.run.smithi169:> full = True 2023-10-26T01:35:39.424 DEBUG:teuthology.orchestra.run.smithi169:> break 2023-10-26T01:35:39.424 DEBUG:teuthology.orchestra.run.smithi169:> else: 2023-10-26T01:35:39.424 DEBUG:teuthology.orchestra.run.smithi169:> print("Not full yet after %.2f MB" % (bytes / (1024.0 * 1024.0))) 2023-10-26T01:35:39.425 DEBUG:teuthology.orchestra.run.smithi169:> 2023-10-26T01:35:39.425 DEBUG:teuthology.orchestra.run.smithi169:> if n > 531 * 0.9: 2023-10-26T01:35:39.425 DEBUG:teuthology.orchestra.run.smithi169:> # Be cautious in the last region where we expect to hit 2023-10-26T01:35:39.426 DEBUG:teuthology.orchestra.run.smithi169:> # the full condition, so that we don'"'"'t overshoot too dramatically 2023-10-26T01:35:39.426 DEBUG:teuthology.orchestra.run.smithi169:> print("sleeping a bit as we'"'"'ve exceeded 90% of our expected full ratio") 2023-10-26T01:35:39.426 DEBUG:teuthology.orchestra.run.smithi169:> time.sleep(15.0) 2023-10-26T01:35:39.427 DEBUG:teuthology.orchestra.run.smithi169:> 2023-10-26T01:35:39.427 DEBUG:teuthology.orchestra.run.smithi169:> if not full: 2023-10-26T01:35:39.427 DEBUG:teuthology.orchestra.run.smithi169:> raise RuntimeError("Failed to reach fullness after writing %d bytes" % bytes) 2023-10-26T01:35:39.428 DEBUG:teuthology.orchestra.run.smithi169:> 2023-10-26T01:35:39.428 DEBUG:teuthology.orchestra.run.smithi169:> # close() should not raise an error because we already caught it in 2023-10-26T01:35:39.428 DEBUG:teuthology.orchestra.run.smithi169:> # fsync. There shouldn'"'"'t have been any more writeback errors 2023-10-26T01:35:39.428 DEBUG:teuthology.orchestra.run.smithi169:> # since then because all IOs got cancelled on the full flag. 2023-10-26T01:35:39.429 DEBUG:teuthology.orchestra.run.smithi169:> print("calling close") 2023-10-26T01:35:39.429 DEBUG:teuthology.orchestra.run.smithi169:> os.close(f) 2023-10-26T01:35:39.429 DEBUG:teuthology.orchestra.run.smithi169:> print("close() did not raise error") 2023-10-26T01:35:39.430 DEBUG:teuthology.orchestra.run.smithi169:> 2023-10-26T01:35:39.430 DEBUG:teuthology.orchestra.run.smithi169:> os.unlink("/home/ubuntu/cephtest/mnt.0/full_test_file") 2023-10-26T01:35:39.430 DEBUG:teuthology.orchestra.run.smithi169:> ' 2023-10-26T01:35:43.518 INFO:teuthology.orchestra.run.smithi169.stdout:writing some data through which we expect to succeed 2023-10-26T01:35:43.518 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed data successfully, will now attempt to fill fs 2023-10-26T01:35:43.519 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:43.519 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:43.519 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 1.00 MB 2023-10-26T01:35:43.519 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:43.519 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:43.520 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 2.00 MB 2023-10-26T01:35:43.520 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:43.520 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:43.520 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 3.00 MB 2023-10-26T01:35:43.520 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:43.520 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:43.521 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 4.00 MB 2023-10-26T01:35:43.521 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:43.521 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:43.521 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 5.00 MB 2023-10-26T01:35:43.521 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:43.522 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:43.522 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 6.00 MB 2023-10-26T01:35:43.522 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:43.522 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:43.522 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 7.00 MB 2023-10-26T01:35:43.523 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:43.523 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:43.523 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 8.00 MB 2023-10-26T01:35:43.523 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:43.523 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:43.523 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 9.00 MB 2023-10-26T01:35:43.524 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:43.524 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:43.524 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 10.00 MB 2023-10-26T01:35:43.524 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:43.524 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:43.525 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 11.00 MB 2023-10-26T01:35:43.525 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:43.525 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:43.525 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 12.00 MB 2023-10-26T01:35:43.525 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:43.526 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:43.526 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 13.00 MB 2023-10-26T01:35:43.526 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:43.526 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:43.526 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 14.00 MB 2023-10-26T01:35:43.527 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:43.527 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:43.527 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 15.00 MB 2023-10-26T01:35:43.527 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:43.527 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:43.527 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 16.00 MB 2023-10-26T01:35:43.528 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:43.528 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:43.528 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 17.00 MB 2023-10-26T01:35:43.528 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:43.528 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:43.529 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 18.00 MB 2023-10-26T01:35:43.529 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:43.529 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:43.529 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 19.00 MB 2023-10-26T01:35:43.529 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:43.529 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:43.530 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 20.00 MB 2023-10-26T01:35:43.530 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:43.530 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:43.530 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 21.00 MB 2023-10-26T01:35:43.530 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:43.531 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:43.531 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 22.00 MB 2023-10-26T01:35:43.531 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:43.531 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:43.531 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 23.00 MB 2023-10-26T01:35:43.531 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:43.532 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:43.532 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 24.00 MB 2023-10-26T01:35:43.532 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:43.532 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:43.532 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 25.00 MB 2023-10-26T01:35:43.533 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:43.533 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:43.533 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 26.00 MB 2023-10-26T01:35:43.533 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:43.533 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:43.533 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 27.00 MB 2023-10-26T01:35:43.534 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:43.534 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:43.534 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 28.00 MB 2023-10-26T01:35:43.534 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:43.534 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:43.535 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 29.00 MB 2023-10-26T01:35:43.535 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:43.535 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:43.535 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 30.00 MB 2023-10-26T01:35:43.535 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:43.536 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:43.536 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 31.00 MB 2023-10-26T01:35:43.536 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:43.536 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:43.536 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 32.00 MB 2023-10-26T01:35:43.537 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:43.537 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:43.537 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 33.00 MB 2023-10-26T01:35:43.537 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:43.537 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:43.538 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 34.00 MB 2023-10-26T01:35:43.538 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:43.538 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:43.538 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 35.00 MB 2023-10-26T01:35:43.538 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:43.538 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:43.539 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 36.00 MB 2023-10-26T01:35:43.539 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:43.539 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:43.539 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 37.00 MB 2023-10-26T01:35:43.539 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:43.540 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:43.540 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 38.00 MB 2023-10-26T01:35:43.540 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:43.540 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:43.540 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 39.00 MB 2023-10-26T01:35:43.540 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:43.541 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:43.541 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 40.00 MB 2023-10-26T01:35:43.541 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:43.541 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:43.541 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 41.00 MB 2023-10-26T01:35:43.542 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:43.542 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:43.542 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 42.00 MB 2023-10-26T01:35:43.542 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:43.542 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:43.542 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 43.00 MB 2023-10-26T01:35:43.543 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:43.543 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:43.543 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 44.00 MB 2023-10-26T01:35:43.543 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:43.543 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:43.544 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 45.00 MB 2023-10-26T01:35:43.544 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:43.544 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:43.544 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 46.00 MB 2023-10-26T01:35:43.544 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:43.544 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:43.545 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 47.00 MB 2023-10-26T01:35:43.545 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:43.545 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:43.545 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 48.00 MB 2023-10-26T01:35:43.545 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:43.546 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:43.546 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 49.00 MB 2023-10-26T01:35:43.546 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:43.546 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:43.546 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 50.00 MB 2023-10-26T01:35:43.547 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:43.547 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:43.547 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 51.00 MB 2023-10-26T01:35:43.547 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:43.547 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:43.547 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 52.00 MB 2023-10-26T01:35:43.548 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:43.548 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:43.548 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 53.00 MB 2023-10-26T01:35:43.548 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:43.548 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:43.549 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 54.00 MB 2023-10-26T01:35:43.549 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:43.549 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:43.549 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 55.00 MB 2023-10-26T01:35:43.549 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:43.549 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:43.550 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 56.00 MB 2023-10-26T01:35:43.550 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:43.550 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:43.550 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 57.00 MB 2023-10-26T01:35:43.550 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:43.551 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:43.551 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 58.00 MB 2023-10-26T01:35:43.551 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:43.551 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:43.551 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 59.00 MB 2023-10-26T01:35:43.551 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:43.552 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:43.552 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 60.00 MB 2023-10-26T01:35:43.552 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:43.552 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:43.552 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 61.00 MB 2023-10-26T01:35:43.553 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:43.553 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:43.553 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 62.00 MB 2023-10-26T01:35:43.553 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:43.553 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:43.553 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 63.00 MB 2023-10-26T01:35:43.554 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:43.554 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:43.554 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 64.00 MB 2023-10-26T01:35:43.554 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:43.554 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:43.555 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 65.00 MB 2023-10-26T01:35:43.555 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:43.555 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:43.555 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 66.00 MB 2023-10-26T01:35:43.555 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:43.555 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:43.556 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 67.00 MB 2023-10-26T01:35:43.556 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:43.556 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:43.556 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 68.00 MB 2023-10-26T01:35:43.556 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:43.557 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:43.557 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 69.00 MB 2023-10-26T01:35:43.557 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:43.557 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:43.557 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 70.00 MB 2023-10-26T01:35:43.557 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:43.558 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:43.558 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 71.00 MB 2023-10-26T01:35:43.558 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:43.558 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:43.558 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 72.00 MB 2023-10-26T01:35:43.559 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:43.559 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:43.559 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 73.00 MB 2023-10-26T01:35:43.559 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:43.559 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:43.560 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 74.00 MB 2023-10-26T01:35:43.560 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:43.560 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:43.560 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 75.00 MB 2023-10-26T01:35:43.560 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:43.560 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:43.561 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 76.00 MB 2023-10-26T01:35:43.561 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:43.561 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:43.561 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 77.00 MB 2023-10-26T01:35:43.561 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:43.562 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:43.562 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 78.00 MB 2023-10-26T01:35:43.562 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:43.562 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:43.562 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 79.00 MB 2023-10-26T01:35:43.563 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:43.563 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:43.563 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 80.00 MB 2023-10-26T01:35:44.453 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:44.454 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:44.454 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 81.00 MB 2023-10-26T01:35:44.454 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:44.454 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:44.455 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 82.00 MB 2023-10-26T01:35:44.455 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:44.455 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:44.455 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 83.00 MB 2023-10-26T01:35:44.455 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:44.456 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:44.456 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 84.00 MB 2023-10-26T01:35:44.456 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:44.456 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:44.456 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 85.00 MB 2023-10-26T01:35:44.457 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:44.457 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:44.457 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 86.00 MB 2023-10-26T01:35:44.457 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:44.457 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:44.457 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 87.00 MB 2023-10-26T01:35:44.458 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:44.458 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:44.458 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 88.00 MB 2023-10-26T01:35:44.458 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:44.458 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:44.459 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 89.00 MB 2023-10-26T01:35:44.459 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:44.459 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:44.459 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 90.00 MB 2023-10-26T01:35:44.459 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:44.460 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:44.460 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 91.00 MB 2023-10-26T01:35:44.460 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:44.460 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:44.460 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 92.00 MB 2023-10-26T01:35:44.460 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:44.461 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:44.461 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 93.00 MB 2023-10-26T01:35:44.461 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:44.461 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:44.461 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 94.00 MB 2023-10-26T01:35:44.462 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:44.462 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:44.462 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 95.00 MB 2023-10-26T01:35:44.462 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:44.462 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:44.463 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 96.00 MB 2023-10-26T01:35:44.463 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:44.463 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:44.463 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 97.00 MB 2023-10-26T01:35:44.463 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:44.463 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:44.464 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 98.00 MB 2023-10-26T01:35:44.464 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:44.464 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:44.464 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 99.00 MB 2023-10-26T01:35:44.464 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:44.465 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:44.465 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 100.00 MB 2023-10-26T01:35:44.465 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:44.465 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:44.465 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 101.00 MB 2023-10-26T01:35:44.465 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:44.466 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:44.466 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 102.00 MB 2023-10-26T01:35:44.466 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:44.466 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:44.466 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 103.00 MB 2023-10-26T01:35:44.467 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:44.467 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:44.467 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 104.00 MB 2023-10-26T01:35:44.467 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:44.467 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:44.468 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 105.00 MB 2023-10-26T01:35:44.468 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:44.468 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:44.468 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 106.00 MB 2023-10-26T01:35:44.468 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:44.468 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:44.469 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 107.00 MB 2023-10-26T01:35:44.469 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:44.469 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:44.469 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 108.00 MB 2023-10-26T01:35:44.469 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:44.470 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:44.470 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 109.00 MB 2023-10-26T01:35:44.470 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:44.470 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:44.470 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 110.00 MB 2023-10-26T01:35:44.471 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:44.471 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:44.471 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 111.00 MB 2023-10-26T01:35:44.471 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:44.471 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:44.471 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 112.00 MB 2023-10-26T01:35:44.472 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:44.472 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:44.472 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 113.00 MB 2023-10-26T01:35:44.472 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:44.472 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:44.473 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 114.00 MB 2023-10-26T01:35:44.473 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:44.473 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:44.473 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 115.00 MB 2023-10-26T01:35:44.473 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:44.474 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:44.474 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 116.00 MB 2023-10-26T01:35:44.474 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:44.474 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:44.474 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 117.00 MB 2023-10-26T01:35:44.475 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:44.475 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:44.475 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 118.00 MB 2023-10-26T01:35:44.475 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:44.475 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:44.475 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 119.00 MB 2023-10-26T01:35:44.476 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:44.476 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:44.476 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 120.00 MB 2023-10-26T01:35:44.476 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:44.476 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:44.477 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 121.00 MB 2023-10-26T01:35:44.477 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:44.477 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:44.477 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 122.00 MB 2023-10-26T01:35:44.477 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:44.477 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:44.478 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 123.00 MB 2023-10-26T01:35:44.478 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:44.478 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:44.478 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 124.00 MB 2023-10-26T01:35:44.478 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:44.479 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:44.479 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 125.00 MB 2023-10-26T01:35:44.479 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:44.479 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:44.479 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 126.00 MB 2023-10-26T01:35:44.479 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:44.480 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:44.480 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 127.00 MB 2023-10-26T01:35:44.480 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:44.480 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:44.480 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 128.00 MB 2023-10-26T01:35:44.481 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:44.481 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:44.481 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 129.00 MB 2023-10-26T01:35:44.481 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:44.481 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:44.481 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 130.00 MB 2023-10-26T01:35:44.482 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:44.482 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:44.482 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 131.00 MB 2023-10-26T01:35:44.482 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:44.482 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:44.483 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 132.00 MB 2023-10-26T01:35:44.483 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:44.483 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:44.483 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 133.00 MB 2023-10-26T01:35:44.483 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:44.484 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:44.484 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 134.00 MB 2023-10-26T01:35:44.484 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:44.484 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:44.484 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 135.00 MB 2023-10-26T01:35:44.484 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:44.485 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:44.485 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 136.00 MB 2023-10-26T01:35:44.485 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:44.485 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:44.485 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 137.00 MB 2023-10-26T01:35:44.486 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:44.486 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:44.486 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 138.00 MB 2023-10-26T01:35:44.486 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:44.486 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:44.487 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 139.00 MB 2023-10-26T01:35:44.487 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:44.487 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:44.487 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 140.00 MB 2023-10-26T01:35:44.487 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:44.487 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:44.488 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 141.00 MB 2023-10-26T01:35:44.488 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:44.488 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:44.488 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 142.00 MB 2023-10-26T01:35:44.488 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:44.489 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:44.489 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 143.00 MB 2023-10-26T01:35:44.489 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:44.489 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:44.489 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 144.00 MB 2023-10-26T01:35:44.489 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:44.490 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:44.490 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 145.00 MB 2023-10-26T01:35:44.490 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:44.490 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:44.490 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 146.00 MB 2023-10-26T01:35:44.491 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:44.491 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:44.491 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 147.00 MB 2023-10-26T01:35:44.491 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:44.491 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:44.491 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 148.00 MB 2023-10-26T01:35:44.492 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:44.492 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:44.492 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 149.00 MB 2023-10-26T01:35:44.492 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:44.492 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:44.493 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 150.00 MB 2023-10-26T01:35:44.493 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:44.493 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:44.493 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 151.00 MB 2023-10-26T01:35:44.493 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:44.493 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:44.494 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 152.00 MB 2023-10-26T01:35:44.494 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:44.494 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:44.494 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 153.00 MB 2023-10-26T01:35:44.494 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:44.495 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:44.495 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 154.00 MB 2023-10-26T01:35:44.495 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:44.495 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:44.495 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 155.00 MB 2023-10-26T01:35:44.496 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:44.496 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:44.496 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 156.00 MB 2023-10-26T01:35:44.496 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:44.496 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:44.497 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 157.00 MB 2023-10-26T01:35:44.497 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:44.497 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:44.497 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 158.00 MB 2023-10-26T01:35:44.497 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:44.498 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:44.498 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 159.00 MB 2023-10-26T01:35:44.498 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:44.498 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:44.498 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 160.00 MB 2023-10-26T01:35:44.499 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:45.390 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:45.391 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 161.00 MB 2023-10-26T01:35:45.391 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:45.391 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:45.391 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 162.00 MB 2023-10-26T01:35:45.391 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:45.392 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:45.392 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 163.00 MB 2023-10-26T01:35:45.392 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:45.392 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:45.393 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 164.00 MB 2023-10-26T01:35:45.393 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:45.393 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:45.393 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 165.00 MB 2023-10-26T01:35:45.393 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:45.394 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:45.394 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 166.00 MB 2023-10-26T01:35:45.394 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:45.394 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:45.394 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 167.00 MB 2023-10-26T01:35:45.395 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:45.395 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:45.395 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 168.00 MB 2023-10-26T01:35:45.395 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:45.395 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:45.396 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 169.00 MB 2023-10-26T01:35:45.396 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:45.396 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:45.396 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 170.00 MB 2023-10-26T01:35:45.396 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:45.397 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:45.397 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 171.00 MB 2023-10-26T01:35:45.397 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:45.398 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:45.398 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 172.00 MB 2023-10-26T01:35:45.398 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:45.398 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:45.399 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 173.00 MB 2023-10-26T01:35:45.399 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:45.399 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:45.399 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 174.00 MB 2023-10-26T01:35:45.399 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:45.399 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:45.400 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 175.00 MB 2023-10-26T01:35:45.400 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:45.400 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:45.400 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 176.00 MB 2023-10-26T01:35:45.401 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:45.401 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:45.401 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 177.00 MB 2023-10-26T01:35:45.401 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:45.401 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:45.401 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 178.00 MB 2023-10-26T01:35:45.402 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:45.402 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:45.402 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 179.00 MB 2023-10-26T01:35:45.402 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:45.403 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:45.403 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 180.00 MB 2023-10-26T01:35:45.403 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:45.403 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:45.403 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 181.00 MB 2023-10-26T01:35:45.403 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:45.404 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:45.404 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 182.00 MB 2023-10-26T01:35:45.404 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:45.404 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:45.404 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 183.00 MB 2023-10-26T01:35:45.405 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:45.405 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:45.405 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 184.00 MB 2023-10-26T01:35:45.405 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:45.405 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:45.406 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 185.00 MB 2023-10-26T01:35:45.406 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:45.406 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:45.406 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 186.00 MB 2023-10-26T01:35:45.406 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:45.407 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:45.407 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 187.00 MB 2023-10-26T01:35:45.407 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:45.407 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:45.407 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 188.00 MB 2023-10-26T01:35:45.408 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:45.408 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:45.408 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 189.00 MB 2023-10-26T01:35:45.408 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:45.408 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:45.409 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 190.00 MB 2023-10-26T01:35:45.409 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:45.409 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:45.409 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 191.00 MB 2023-10-26T01:35:45.409 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:45.410 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:45.410 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 192.00 MB 2023-10-26T01:35:45.410 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:45.410 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:45.410 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 193.00 MB 2023-10-26T01:35:45.411 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:45.411 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:45.411 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 194.00 MB 2023-10-26T01:35:45.411 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:45.411 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:45.412 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 195.00 MB 2023-10-26T01:35:45.412 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:45.412 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:45.412 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 196.00 MB 2023-10-26T01:35:45.413 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:45.413 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:45.413 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 197.00 MB 2023-10-26T01:35:45.413 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:45.414 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:45.414 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 198.00 MB 2023-10-26T01:35:45.414 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:45.414 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:45.415 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 199.00 MB 2023-10-26T01:35:45.415 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:45.415 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:45.415 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 200.00 MB 2023-10-26T01:35:45.415 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:45.416 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:45.416 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 201.00 MB 2023-10-26T01:35:45.416 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:45.416 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:45.416 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 202.00 MB 2023-10-26T01:35:45.416 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:45.417 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:45.417 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 203.00 MB 2023-10-26T01:35:45.417 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:45.417 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:45.417 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 204.00 MB 2023-10-26T01:35:45.418 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:45.418 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:45.418 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 205.00 MB 2023-10-26T01:35:45.418 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:45.418 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:45.419 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 206.00 MB 2023-10-26T01:35:45.419 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:45.419 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:45.419 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 207.00 MB 2023-10-26T01:35:45.419 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:45.419 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:45.420 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 208.00 MB 2023-10-26T01:35:45.420 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:45.420 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:45.420 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 209.00 MB 2023-10-26T01:35:45.420 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:45.421 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:45.421 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 210.00 MB 2023-10-26T01:35:45.421 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:45.421 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:45.421 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 211.00 MB 2023-10-26T01:35:45.422 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:45.422 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:45.422 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 212.00 MB 2023-10-26T01:35:45.422 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:45.422 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:45.423 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 213.00 MB 2023-10-26T01:35:45.423 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:45.423 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:45.423 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 214.00 MB 2023-10-26T01:35:45.423 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:45.423 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:45.424 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 215.00 MB 2023-10-26T01:35:45.424 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:45.424 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:45.424 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 216.00 MB 2023-10-26T01:35:45.424 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:45.425 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:45.425 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 217.00 MB 2023-10-26T01:35:45.425 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:45.425 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:45.425 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 218.00 MB 2023-10-26T01:35:45.426 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:45.426 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:45.426 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 219.00 MB 2023-10-26T01:35:45.426 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:45.426 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:45.427 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 220.00 MB 2023-10-26T01:35:45.427 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:45.427 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:45.427 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 221.00 MB 2023-10-26T01:35:45.427 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:45.427 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:45.428 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 222.00 MB 2023-10-26T01:35:45.428 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:45.428 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:45.428 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 223.00 MB 2023-10-26T01:35:45.428 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:45.429 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:45.429 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 224.00 MB 2023-10-26T01:35:45.429 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:45.429 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:45.430 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 225.00 MB 2023-10-26T01:35:45.430 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:45.430 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:45.430 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 226.00 MB 2023-10-26T01:35:45.431 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:45.431 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:45.431 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 227.00 MB 2023-10-26T01:35:45.431 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:45.431 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:45.433 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 228.00 MB 2023-10-26T01:35:45.433 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:45.433 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:45.433 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 229.00 MB 2023-10-26T01:35:45.434 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:45.434 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:45.434 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 230.00 MB 2023-10-26T01:35:45.435 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:45.435 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:45.435 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 231.00 MB 2023-10-26T01:35:45.436 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:45.436 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:45.436 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 232.00 MB 2023-10-26T01:35:45.437 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:45.437 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:45.437 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 233.00 MB 2023-10-26T01:35:45.438 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:45.438 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:45.439 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 234.00 MB 2023-10-26T01:35:45.439 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:45.439 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:45.439 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 235.00 MB 2023-10-26T01:35:45.440 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:45.440 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:45.440 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 236.00 MB 2023-10-26T01:35:45.441 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:45.441 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:45.441 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 237.00 MB 2023-10-26T01:35:45.442 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:45.442 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:45.443 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 238.00 MB 2023-10-26T01:35:45.443 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:45.443 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:45.443 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 239.00 MB 2023-10-26T01:35:45.444 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:45.444 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:45.444 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 240.00 MB 2023-10-26T01:35:45.444 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:45.444 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:46.344 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 241.00 MB 2023-10-26T01:35:46.344 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:46.344 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:46.344 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 242.00 MB 2023-10-26T01:35:46.345 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:46.345 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:46.345 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 243.00 MB 2023-10-26T01:35:46.345 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:46.345 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:46.345 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 244.00 MB 2023-10-26T01:35:46.346 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:46.346 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:46.346 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 245.00 MB 2023-10-26T01:35:46.346 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:46.346 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:46.347 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 246.00 MB 2023-10-26T01:35:46.347 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:46.347 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:46.347 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 247.00 MB 2023-10-26T01:35:46.347 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:46.348 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:46.348 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 248.00 MB 2023-10-26T01:35:46.348 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:46.348 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:46.348 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 249.00 MB 2023-10-26T01:35:46.348 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:46.349 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:46.349 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 250.00 MB 2023-10-26T01:35:46.349 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:46.349 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:46.349 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 251.00 MB 2023-10-26T01:35:46.349 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:46.350 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:46.350 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 252.00 MB 2023-10-26T01:35:46.350 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:46.350 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:46.351 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 253.00 MB 2023-10-26T01:35:46.351 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:46.351 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:46.351 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 254.00 MB 2023-10-26T01:35:46.351 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:46.352 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:46.352 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 255.00 MB 2023-10-26T01:35:46.352 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:46.352 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:46.352 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 256.00 MB 2023-10-26T01:35:46.352 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:46.353 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:46.353 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 257.00 MB 2023-10-26T01:35:46.353 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:46.353 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:46.354 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 258.00 MB 2023-10-26T01:35:46.354 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:46.354 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:46.354 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 259.00 MB 2023-10-26T01:35:46.354 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:46.355 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:46.355 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 260.00 MB 2023-10-26T01:35:46.355 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:46.355 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:46.355 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 261.00 MB 2023-10-26T01:35:46.355 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:46.356 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:46.356 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 262.00 MB 2023-10-26T01:35:46.356 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:46.356 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:46.356 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 263.00 MB 2023-10-26T01:35:46.356 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:46.357 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:46.357 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 264.00 MB 2023-10-26T01:35:46.357 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:46.357 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:46.357 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 265.00 MB 2023-10-26T01:35:46.358 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:46.358 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:46.358 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 266.00 MB 2023-10-26T01:35:46.358 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:46.358 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:46.358 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 267.00 MB 2023-10-26T01:35:46.359 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:46.359 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:46.359 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 268.00 MB 2023-10-26T01:35:46.359 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:46.359 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:46.359 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 269.00 MB 2023-10-26T01:35:46.360 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:46.360 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:46.360 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 270.00 MB 2023-10-26T01:35:46.360 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:46.360 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:46.360 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 271.00 MB 2023-10-26T01:35:46.361 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:46.361 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:46.361 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 272.00 MB 2023-10-26T01:35:46.361 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:46.362 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:46.362 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 273.00 MB 2023-10-26T01:35:46.362 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:46.362 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:46.362 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 274.00 MB 2023-10-26T01:35:46.362 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:46.363 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:46.363 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 275.00 MB 2023-10-26T01:35:46.363 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:46.363 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:46.363 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 276.00 MB 2023-10-26T01:35:46.363 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:46.364 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:46.364 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 277.00 MB 2023-10-26T01:35:46.364 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:46.364 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:46.365 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 278.00 MB 2023-10-26T01:35:46.365 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:46.365 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:46.365 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 279.00 MB 2023-10-26T01:35:46.365 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:46.365 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:46.366 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 280.00 MB 2023-10-26T01:35:46.366 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:46.366 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:46.366 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 281.00 MB 2023-10-26T01:35:46.366 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:46.366 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:46.367 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 282.00 MB 2023-10-26T01:35:46.367 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:46.367 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:46.367 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 283.00 MB 2023-10-26T01:35:46.368 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:46.368 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:46.368 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 284.00 MB 2023-10-26T01:35:46.368 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:46.368 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:46.368 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 285.00 MB 2023-10-26T01:35:46.369 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:46.369 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:46.369 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 286.00 MB 2023-10-26T01:35:46.369 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:46.369 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:46.369 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 287.00 MB 2023-10-26T01:35:46.370 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:46.370 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:46.370 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 288.00 MB 2023-10-26T01:35:46.370 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:46.370 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:46.370 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 289.00 MB 2023-10-26T01:35:46.371 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:46.371 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:46.371 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 290.00 MB 2023-10-26T01:35:46.371 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:46.371 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:46.371 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 291.00 MB 2023-10-26T01:35:46.372 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:46.372 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:46.372 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 292.00 MB 2023-10-26T01:35:46.372 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:46.372 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:46.373 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 293.00 MB 2023-10-26T01:35:46.373 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:46.373 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:46.373 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 294.00 MB 2023-10-26T01:35:46.373 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:46.373 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:46.374 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 295.00 MB 2023-10-26T01:35:46.374 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:46.374 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:46.374 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 296.00 MB 2023-10-26T01:35:46.375 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:46.375 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:46.375 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 297.00 MB 2023-10-26T01:35:46.375 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:46.375 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:46.375 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 298.00 MB 2023-10-26T01:35:46.376 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:46.376 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:46.376 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 299.00 MB 2023-10-26T01:35:46.376 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:46.376 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:46.376 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 300.00 MB 2023-10-26T01:35:46.377 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:46.377 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:46.377 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 301.00 MB 2023-10-26T01:35:46.377 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:46.377 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:46.378 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 302.00 MB 2023-10-26T01:35:46.378 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:46.378 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:46.378 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 303.00 MB 2023-10-26T01:35:46.378 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:46.379 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:46.379 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 304.00 MB 2023-10-26T01:35:46.379 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:46.379 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:46.380 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 305.00 MB 2023-10-26T01:35:46.380 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:46.380 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:46.380 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 306.00 MB 2023-10-26T01:35:46.380 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:46.381 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:46.381 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 307.00 MB 2023-10-26T01:35:46.381 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:46.381 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:46.381 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 308.00 MB 2023-10-26T01:35:46.382 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:46.382 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:46.383 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 309.00 MB 2023-10-26T01:35:46.383 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:46.383 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:46.384 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 310.00 MB 2023-10-26T01:35:46.384 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:46.385 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:46.385 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 311.00 MB 2023-10-26T01:35:46.385 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:46.385 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:46.385 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 312.00 MB 2023-10-26T01:35:46.386 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:46.386 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:46.386 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 313.00 MB 2023-10-26T01:35:46.387 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:46.387 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:46.387 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 314.00 MB 2023-10-26T01:35:46.387 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:46.387 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:46.388 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 315.00 MB 2023-10-26T01:35:46.388 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:46.388 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:46.388 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 316.00 MB 2023-10-26T01:35:46.388 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:46.389 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:46.389 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 317.00 MB 2023-10-26T01:35:46.389 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:46.389 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:46.389 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 318.00 MB 2023-10-26T01:35:46.389 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:46.390 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:46.390 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 319.00 MB 2023-10-26T01:35:46.390 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:46.390 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:46.391 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 320.00 MB 2023-10-26T01:35:46.391 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:46.391 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:46.391 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 321.00 MB 2023-10-26T01:35:47.344 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:47.344 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:47.345 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 322.00 MB 2023-10-26T01:35:47.345 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:47.345 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:47.345 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 323.00 MB 2023-10-26T01:35:47.345 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:47.346 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:47.346 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 324.00 MB 2023-10-26T01:35:47.346 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:47.346 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:47.346 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 325.00 MB 2023-10-26T01:35:47.347 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:47.347 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:47.347 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 326.00 MB 2023-10-26T01:35:47.347 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:47.347 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:47.347 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 327.00 MB 2023-10-26T01:35:47.348 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:47.348 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:47.348 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 328.00 MB 2023-10-26T01:35:47.348 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:47.348 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:47.349 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 329.00 MB 2023-10-26T01:35:47.349 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:47.349 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:47.349 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 330.00 MB 2023-10-26T01:35:47.349 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:47.349 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:47.350 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 331.00 MB 2023-10-26T01:35:47.350 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:47.350 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:47.350 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 332.00 MB 2023-10-26T01:35:47.350 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:47.350 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:47.351 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 333.00 MB 2023-10-26T01:35:47.351 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:47.351 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:47.351 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 334.00 MB 2023-10-26T01:35:47.351 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:47.351 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:47.352 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 335.00 MB 2023-10-26T01:35:47.352 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:47.352 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:47.352 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 336.00 MB 2023-10-26T01:35:47.352 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:47.353 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:47.353 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 337.00 MB 2023-10-26T01:35:47.353 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:47.353 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:47.353 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 338.00 MB 2023-10-26T01:35:47.354 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:47.354 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:47.354 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 339.00 MB 2023-10-26T01:35:47.354 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:47.354 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:47.354 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 340.00 MB 2023-10-26T01:35:47.355 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:47.355 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:47.355 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 341.00 MB 2023-10-26T01:35:47.355 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:47.355 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:47.356 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 342.00 MB 2023-10-26T01:35:47.356 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:47.356 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:47.356 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 343.00 MB 2023-10-26T01:35:47.356 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:47.357 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:47.357 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 344.00 MB 2023-10-26T01:35:47.357 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:47.357 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:47.357 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 345.00 MB 2023-10-26T01:35:47.358 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:47.358 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:47.358 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 346.00 MB 2023-10-26T01:35:47.358 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:47.358 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:47.359 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 347.00 MB 2023-10-26T01:35:47.359 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:47.359 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:47.359 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 348.00 MB 2023-10-26T01:35:47.359 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:47.360 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:47.360 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 349.00 MB 2023-10-26T01:35:47.360 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:47.360 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:47.360 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 350.00 MB 2023-10-26T01:35:47.361 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:47.361 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:47.361 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 351.00 MB 2023-10-26T01:35:47.361 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:47.361 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:47.362 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 352.00 MB 2023-10-26T01:35:47.362 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:47.362 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:47.362 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 353.00 MB 2023-10-26T01:35:47.362 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:47.363 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:47.363 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 354.00 MB 2023-10-26T01:35:47.363 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:47.363 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:47.363 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 355.00 MB 2023-10-26T01:35:47.364 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:47.364 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:47.364 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 356.00 MB 2023-10-26T01:35:47.364 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:47.364 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:47.365 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 357.00 MB 2023-10-26T01:35:47.365 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:47.365 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:47.365 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 358.00 MB 2023-10-26T01:35:47.365 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:47.365 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:47.366 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 359.00 MB 2023-10-26T01:35:47.366 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:47.366 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:47.366 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 360.00 MB 2023-10-26T01:35:47.366 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:47.367 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:47.367 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 361.00 MB 2023-10-26T01:35:47.367 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:47.367 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:47.367 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 362.00 MB 2023-10-26T01:35:47.368 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:47.368 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:47.368 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 363.00 MB 2023-10-26T01:35:47.368 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:47.368 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:47.368 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 364.00 MB 2023-10-26T01:35:47.369 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:47.369 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:47.369 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 365.00 MB 2023-10-26T01:35:47.369 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:47.369 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:47.370 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 366.00 MB 2023-10-26T01:35:47.370 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:47.370 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:47.370 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 367.00 MB 2023-10-26T01:35:47.370 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:47.371 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:47.371 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 368.00 MB 2023-10-26T01:35:47.371 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:47.371 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:47.371 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 369.00 MB 2023-10-26T01:35:47.371 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:47.372 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:47.372 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 370.00 MB 2023-10-26T01:35:47.372 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:47.372 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:47.372 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 371.00 MB 2023-10-26T01:35:47.373 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:47.373 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:47.373 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 372.00 MB 2023-10-26T01:35:47.373 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:47.373 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:47.374 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 373.00 MB 2023-10-26T01:35:47.374 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:47.374 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:47.374 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 374.00 MB 2023-10-26T01:35:47.374 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:47.375 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:47.375 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 375.00 MB 2023-10-26T01:35:47.375 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:47.375 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:47.375 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 376.00 MB 2023-10-26T01:35:47.376 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:47.376 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:47.376 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 377.00 MB 2023-10-26T01:35:47.376 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:47.376 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:47.376 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 378.00 MB 2023-10-26T01:35:47.377 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:47.377 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:47.377 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 379.00 MB 2023-10-26T01:35:47.377 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:47.377 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:47.378 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 380.00 MB 2023-10-26T01:35:47.378 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:47.378 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:47.378 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 381.00 MB 2023-10-26T01:35:47.378 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:47.378 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:47.379 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 382.00 MB 2023-10-26T01:35:47.379 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:47.379 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:47.379 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 383.00 MB 2023-10-26T01:35:47.379 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:47.380 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:47.380 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 384.00 MB 2023-10-26T01:35:47.380 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:47.380 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:47.380 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 385.00 MB 2023-10-26T01:35:47.380 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:47.381 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:47.381 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 386.00 MB 2023-10-26T01:35:47.381 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:47.381 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:47.381 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 387.00 MB 2023-10-26T01:35:47.382 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:47.382 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:47.382 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 388.00 MB 2023-10-26T01:35:47.382 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:47.382 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:47.382 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 389.00 MB 2023-10-26T01:35:47.383 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:47.383 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:47.383 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 390.00 MB 2023-10-26T01:35:47.383 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:47.383 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:47.384 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 391.00 MB 2023-10-26T01:35:47.384 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:47.384 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:47.384 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 392.00 MB 2023-10-26T01:35:47.384 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:47.384 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:47.385 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 393.00 MB 2023-10-26T01:35:47.385 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:47.385 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:47.385 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 394.00 MB 2023-10-26T01:35:47.385 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:47.386 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:47.386 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 395.00 MB 2023-10-26T01:35:47.386 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:47.386 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:47.386 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 396.00 MB 2023-10-26T01:35:47.386 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:47.387 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:47.387 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 397.00 MB 2023-10-26T01:35:47.387 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:47.387 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:47.387 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 398.00 MB 2023-10-26T01:35:47.388 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:47.388 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:47.388 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 399.00 MB 2023-10-26T01:35:47.388 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:47.388 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:47.388 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 400.00 MB 2023-10-26T01:35:47.389 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:35:47.389 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:35:47.389 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 401.00 MB 2023-10-26T01:35:50.095 INFO:tasks.ceph.mon.a.smithi169.stderr:2023-10-26T01:35:50.093+0000 7f158b279640 -1 log_channel(cluster) log [ERR] : Health check failed: 2 full osd(s) (OSD_FULL) 2023-10-26T01:35:57.986 INFO:tasks.ceph.mon.a.smithi169.stderr:2023-10-26T01:35:57.984+0000 7f158b279640 -1 log_channel(cluster) log [ERR] : Health check update: 4 full osd(s) (OSD_FULL) 2023-10-26T01:36:05.701 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T01:36:05.711 DEBUG:teuthology.orchestra.run.smithi170:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T01:36:18.295 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:36:18.295 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:36:18.296 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 402.00 MB 2023-10-26T01:36:18.296 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:36:18.296 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:36:18.296 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 403.00 MB 2023-10-26T01:36:18.296 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:36:18.297 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:36:18.297 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 404.00 MB 2023-10-26T01:36:18.297 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:36:18.297 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:36:18.297 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 405.00 MB 2023-10-26T01:36:18.297 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:36:18.298 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:36:18.298 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 406.00 MB 2023-10-26T01:36:18.298 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:36:18.298 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:36:18.299 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 407.00 MB 2023-10-26T01:36:18.299 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:36:18.299 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:36:18.299 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 408.00 MB 2023-10-26T01:36:18.299 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:36:18.300 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:36:18.300 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 409.00 MB 2023-10-26T01:36:18.300 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:36:18.300 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:36:18.300 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 410.00 MB 2023-10-26T01:36:18.300 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:36:18.301 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:36:18.301 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 411.00 MB 2023-10-26T01:36:18.301 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:36:18.301 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:36:18.301 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 412.00 MB 2023-10-26T01:36:18.302 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:36:18.302 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:36:18.302 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 413.00 MB 2023-10-26T01:36:18.302 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:36:18.302 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:36:18.303 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 414.00 MB 2023-10-26T01:36:18.303 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:36:18.303 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:36:18.303 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 415.00 MB 2023-10-26T01:36:18.303 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:36:18.303 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:36:18.304 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 416.00 MB 2023-10-26T01:36:18.304 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:36:18.304 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:36:18.304 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 417.00 MB 2023-10-26T01:36:18.304 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:36:18.305 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:36:18.305 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 418.00 MB 2023-10-26T01:36:18.305 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:36:18.305 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:36:18.305 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 419.00 MB 2023-10-26T01:36:18.306 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:36:18.306 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:36:18.306 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 420.00 MB 2023-10-26T01:36:18.306 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:36:18.307 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:36:18.307 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 421.00 MB 2023-10-26T01:36:18.307 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:36:18.307 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:36:18.307 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 422.00 MB 2023-10-26T01:36:18.308 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:36:18.308 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:36:18.308 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 423.00 MB 2023-10-26T01:36:18.308 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:36:18.308 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:36:18.309 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 424.00 MB 2023-10-26T01:36:18.309 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:36:18.309 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:36:18.309 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 425.00 MB 2023-10-26T01:36:18.309 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:36:18.310 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:36:18.310 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 426.00 MB 2023-10-26T01:36:18.310 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:36:18.310 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:36:18.310 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 427.00 MB 2023-10-26T01:36:18.311 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:36:18.311 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:36:18.311 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 428.00 MB 2023-10-26T01:36:18.311 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:36:18.311 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:36:18.312 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 429.00 MB 2023-10-26T01:36:18.312 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:36:18.312 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:36:18.312 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 430.00 MB 2023-10-26T01:36:18.312 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:36:18.313 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:36:18.313 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 431.00 MB 2023-10-26T01:36:18.313 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:36:18.313 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:36:18.313 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 432.00 MB 2023-10-26T01:36:18.313 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:36:18.314 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:36:18.314 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 433.00 MB 2023-10-26T01:36:18.314 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:36:18.314 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:36:18.315 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 434.00 MB 2023-10-26T01:36:18.315 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:36:18.315 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:36:18.315 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 435.00 MB 2023-10-26T01:36:18.315 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:36:18.315 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:36:18.316 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 436.00 MB 2023-10-26T01:36:18.316 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:36:18.316 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:36:18.316 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 437.00 MB 2023-10-26T01:36:18.316 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:36:18.316 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:36:18.317 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 438.00 MB 2023-10-26T01:36:18.317 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:36:18.317 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:36:18.317 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 439.00 MB 2023-10-26T01:36:18.317 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:36:18.318 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:36:18.318 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 440.00 MB 2023-10-26T01:36:18.318 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:36:18.318 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:36:18.318 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 441.00 MB 2023-10-26T01:36:18.319 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:36:18.319 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:36:18.319 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 442.00 MB 2023-10-26T01:36:18.319 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:36:18.319 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:36:18.320 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 443.00 MB 2023-10-26T01:36:18.320 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:36:18.320 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:36:18.320 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 444.00 MB 2023-10-26T01:36:18.320 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:36:18.320 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:36:18.321 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 445.00 MB 2023-10-26T01:36:18.321 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:36:18.321 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:36:18.321 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 446.00 MB 2023-10-26T01:36:18.321 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:36:18.322 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:36:18.322 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 447.00 MB 2023-10-26T01:36:18.322 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:36:18.322 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:36:18.322 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 448.00 MB 2023-10-26T01:36:18.323 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:36:18.323 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:36:18.323 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 449.00 MB 2023-10-26T01:36:18.323 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:36:18.323 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:36:18.324 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 450.00 MB 2023-10-26T01:36:18.324 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:36:18.324 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:36:18.324 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 451.00 MB 2023-10-26T01:36:18.324 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:36:18.325 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:36:18.325 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 452.00 MB 2023-10-26T01:36:18.325 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:36:18.325 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:36:18.325 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 453.00 MB 2023-10-26T01:36:18.325 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:36:18.326 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:36:18.326 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 454.00 MB 2023-10-26T01:36:18.326 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:36:18.326 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:36:18.326 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 455.00 MB 2023-10-26T01:36:18.327 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:36:18.327 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:36:18.327 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 456.00 MB 2023-10-26T01:36:18.327 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:36:18.327 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:36:18.328 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 457.00 MB 2023-10-26T01:36:18.328 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:36:18.328 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:36:18.328 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 458.00 MB 2023-10-26T01:36:18.328 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:36:18.329 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:36:18.329 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 459.00 MB 2023-10-26T01:36:18.329 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:36:18.329 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:36:18.329 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 460.00 MB 2023-10-26T01:36:18.329 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:36:18.330 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:36:18.330 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 461.00 MB 2023-10-26T01:36:18.330 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:36:18.330 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:36:18.330 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 462.00 MB 2023-10-26T01:36:18.331 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:36:18.331 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:36:18.331 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 463.00 MB 2023-10-26T01:36:18.331 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:36:18.331 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:36:18.332 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 464.00 MB 2023-10-26T01:36:18.332 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:36:18.332 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:36:18.332 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 465.00 MB 2023-10-26T01:36:18.332 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:36:18.332 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:36:18.333 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 466.00 MB 2023-10-26T01:36:18.333 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:36:18.333 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:36:18.333 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 467.00 MB 2023-10-26T01:36:18.333 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:36:18.334 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:36:18.334 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 468.00 MB 2023-10-26T01:36:18.334 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:36:18.334 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:36:18.334 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 469.00 MB 2023-10-26T01:36:18.335 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:36:18.335 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:36:18.335 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 470.00 MB 2023-10-26T01:36:18.335 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:36:18.335 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:36:18.335 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 471.00 MB 2023-10-26T01:36:18.336 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:36:18.336 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:36:18.336 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 472.00 MB 2023-10-26T01:36:18.336 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:36:18.336 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:36:18.337 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 473.00 MB 2023-10-26T01:36:18.337 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:36:18.337 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:36:18.337 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 474.00 MB 2023-10-26T01:36:18.337 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:36:18.337 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:36:18.338 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 475.00 MB 2023-10-26T01:36:18.338 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:36:18.338 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:36:18.338 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 476.00 MB 2023-10-26T01:36:18.338 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:36:18.339 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:36:18.339 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 477.00 MB 2023-10-26T01:36:18.339 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:36:18.339 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:36:18.339 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 478.00 MB 2023-10-26T01:36:18.339 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:36:18.340 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:36:18.340 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 479.00 MB 2023-10-26T01:36:18.340 INFO:teuthology.orchestra.run.smithi169.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2023-10-26T01:36:18.340 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:36:18.340 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:36:18.341 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 480.00 MB 2023-10-26T01:36:18.341 INFO:teuthology.orchestra.run.smithi169.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2023-10-26T01:36:18.429 INFO:teuthology.orchestra.run.smithi169.stdout:Reached fullness after 480.00 MB 2023-10-26T01:36:18.429 INFO:teuthology.orchestra.run.smithi169.stdout:calling close 2023-10-26T01:36:18.429 INFO:teuthology.orchestra.run.smithi169.stdout:close() did not raise error 2023-10-26T01:36:18.436 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T01:36:18.456 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-26T01:36:18.457 DEBUG:teuthology.orchestra.run.smithi169:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-26T01:36:18.577 DEBUG:teuthology.orchestra.run.smithi170:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-26T01:36:18.697 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-26T01:36:18.698 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:36:18.698 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:36:18.760 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:36:18.761 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:36:18.814 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:36:19.032 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.029+0000 7f98e9de7640 1 -- 172.21.15.169:0/3410239577 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98e4158850 msgr2=0x7f98e414e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:19.033 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.029+0000 7f98e9de7640 1 --2- 172.21.15.169:0/3410239577 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98e4158850 0x7f98e414e790 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f98d8005fd0 tx=0x7f98d802bd90 comp rx=0 tx=0).stop 2023-10-26T01:36:19.033 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.029+0000 7f98e9de7640 1 -- 172.21.15.169:0/3410239577 shutdown_connections 2023-10-26T01:36:19.033 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.029+0000 7f98e9de7640 1 --2- 172.21.15.169:0/3410239577 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f98e40cd720 0x7f98e40cdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:19.033 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.029+0000 7f98e9de7640 1 --2- 172.21.15.169:0/3410239577 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98e4158850 0x7f98e414e790 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:19.033 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.029+0000 7f98e9de7640 1 -- 172.21.15.169:0/3410239577 >> 172.21.15.169:0/3410239577 conn(0x7f98e4069930 msgr2=0x7f98e4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:19.034 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.029+0000 7f98e9de7640 1 -- 172.21.15.169:0/3410239577 shutdown_connections 2023-10-26T01:36:19.034 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.029+0000 7f98e9de7640 1 -- 172.21.15.169:0/3410239577 wait complete. 2023-10-26T01:36:19.035 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.031+0000 7f98e9de7640 1 Processor -- start 2023-10-26T01:36:19.035 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.031+0000 7f98e9de7640 1 -- start start 2023-10-26T01:36:19.035 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.032+0000 7f98e9de7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f98e406ce20 0x7f98e41618a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:19.035 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.032+0000 7f98e9de7640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f98e40cd720 0x7f98e4095c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:19.036 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.032+0000 7f98e9de7640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98e4158850 0x7f98e40961b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:19.036 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.032+0000 7f98e9de7640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98e4155a40 con 0x7f98e4158850 2023-10-26T01:36:19.036 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.032+0000 7f98e9de7640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98e4155cb0 con 0x7f98e406ce20 2023-10-26T01:36:19.036 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.032+0000 7f98e9de7640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98e4155f90 con 0x7f98e40cd720 2023-10-26T01:36:19.036 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.032+0000 7f98e3fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98e4158850 0x7f98e40961b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:19.037 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.032+0000 7f98e3fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98e4158850 0x7f98e40961b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:50232/0 (socket says 172.21.15.169:50232) 2023-10-26T01:36:19.037 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.032+0000 7f98e3fff640 1 -- 172.21.15.169:0/2163072219 learned_addr learned my addr 172.21.15.169:0/2163072219 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:36:19.038 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.033+0000 7f98e2ffd640 1 --2- 172.21.15.169:0/2163072219 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f98e40cd720 0x7f98e4095c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:19.038 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.033+0000 7f98e37fe640 1 --2- 172.21.15.169:0/2163072219 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f98e406ce20 0x7f98e41618a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:19.038 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.033+0000 7f98e3fff640 1 -- 172.21.15.169:0/2163072219 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f98e40cd720 msgr2=0x7f98e4095c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:19.039 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.033+0000 7f98e3fff640 1 --2- 172.21.15.169:0/2163072219 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f98e40cd720 0x7f98e4095c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:19.039 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.033+0000 7f98e3fff640 1 -- 172.21.15.169:0/2163072219 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f98e406ce20 msgr2=0x7f98e41618a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:19.039 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.033+0000 7f98e3fff640 1 --2- 172.21.15.169:0/2163072219 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f98e406ce20 0x7f98e41618a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:19.039 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.033+0000 7f98e3fff640 1 -- 172.21.15.169:0/2163072219 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98d8009560 con 0x7f98e4158850 2023-10-26T01:36:19.039 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.033+0000 7f98e2ffd640 1 --2- 172.21.15.169:0/2163072219 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f98e40cd720 0x7f98e4095c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:36:19.040 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.033+0000 7f98e3fff640 1 --2- 172.21.15.169:0/2163072219 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98e4158850 0x7f98e40961b0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f98dc00b970 tx=0x7f98dc00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:19.040 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.033+0000 7f98cbfff640 1 -- 172.21.15.169:0/2163072219 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98dc00cb40 con 0x7f98e4158850 2023-10-26T01:36:19.040 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.033+0000 7f98cbfff640 1 -- 172.21.15.169:0/2163072219 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f98dc00cdb0 con 0x7f98e4158850 2023-10-26T01:36:19.040 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.033+0000 7f98cbfff640 1 -- 172.21.15.169:0/2163072219 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98dc0146a0 con 0x7f98e4158850 2023-10-26T01:36:19.040 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.033+0000 7f98e9de7640 1 -- 172.21.15.169:0/2163072219 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98e4156270 con 0x7f98e4158850 2023-10-26T01:36:19.041 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.034+0000 7f98e9de7640 1 -- 172.21.15.169:0/2163072219 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98e4152300 con 0x7f98e4158850 2023-10-26T01:36:19.041 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.035+0000 7f98e9de7640 1 -- 172.21.15.169:0/2163072219 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98e41524f0 con 0x7f98e4158850 2023-10-26T01:36:19.041 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.036+0000 7f98cbfff640 1 -- 172.21.15.169:0/2163072219 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f98dc01d3b0 con 0x7f98e4158850 2023-10-26T01:36:19.042 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.036+0000 7f98cbfff640 1 --2- 172.21.15.169:0/2163072219 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f98d006ae40 0x7f98d006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:19.042 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.037+0000 7f98cbfff640 1 -- 172.21.15.169:0/2163072219 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f98dc0898c0 con 0x7f98e4158850 2023-10-26T01:36:19.042 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.037+0000 7f98cbfff640 1 -- 172.21.15.169:0/2163072219 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=89}) v3 -- 0x7f98d0072970 con 0x7f98e4158850 2023-10-26T01:36:19.045 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.037+0000 7f98e37fe640 1 --2- 172.21.15.169:0/2163072219 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f98d006ae40 0x7f98d006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:19.045 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.038+0000 7f98e37fe640 1 --2- 172.21.15.169:0/2163072219 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f98d006ae40 0x7f98d006d300 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f98d4000f60 tx=0x7f98d4009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:19.046 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.042+0000 7f98cbfff640 1 -- 172.21.15.169:0/2163072219 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f98dc019020 con 0x7f98e4158850 2023-10-26T01:36:19.235 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.233+0000 7f98e9de7640 1 -- 172.21.15.169:0/2163072219 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f98e4156790 con 0x7f98e4158850 2023-10-26T01:36:19.237 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.235+0000 7f98cbfff640 1 -- 172.21.15.169:0/2163072219 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+16659 (secure 0 0 0) 0x7f98dc089070 con 0x7f98e4158850 2023-10-26T01:36:19.238 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:36:19.238 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":88,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:35:54.125183+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-10-26T01:35:26.320950+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-10-26T01:35:26.855916+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","full","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","full","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","full","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","full","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:36:19.240 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.237+0000 7f98e9de7640 1 -- 172.21.15.169:0/2163072219 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f98d006ae40 msgr2=0x7f98d006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:19.241 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.237+0000 7f98e9de7640 1 --2- 172.21.15.169:0/2163072219 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f98d006ae40 0x7f98d006d300 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f98d4000f60 tx=0x7f98d4009040 comp rx=0 tx=0).stop 2023-10-26T01:36:19.241 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.237+0000 7f98e9de7640 1 -- 172.21.15.169:0/2163072219 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98e4158850 msgr2=0x7f98e40961b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:19.241 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.237+0000 7f98e9de7640 1 --2- 172.21.15.169:0/2163072219 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98e4158850 0x7f98e40961b0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f98dc00b970 tx=0x7f98dc00be30 comp rx=0 tx=0).stop 2023-10-26T01:36:19.241 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.237+0000 7f98e9de7640 1 -- 172.21.15.169:0/2163072219 shutdown_connections 2023-10-26T01:36:19.241 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.237+0000 7f98e9de7640 1 --2- 172.21.15.169:0/2163072219 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f98d006ae40 0x7f98d006d300 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:19.242 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.237+0000 7f98e9de7640 1 --2- 172.21.15.169:0/2163072219 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98e4158850 0x7f98e40961b0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:19.242 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.237+0000 7f98e9de7640 1 --2- 172.21.15.169:0/2163072219 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f98e40cd720 0x7f98e4095c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:19.242 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.237+0000 7f98e9de7640 1 --2- 172.21.15.169:0/2163072219 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f98e406ce20 0x7f98e41618a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:19.242 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.237+0000 7f98e9de7640 1 -- 172.21.15.169:0/2163072219 >> 172.21.15.169:0/2163072219 conn(0x7f98e4069930 msgr2=0x7f98e414f240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:19.243 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.238+0000 7f98e9de7640 1 -- 172.21.15.169:0/2163072219 shutdown_connections 2023-10-26T01:36:19.243 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.238+0000 7f98e9de7640 1 -- 172.21.15.169:0/2163072219 wait complete. 2023-10-26T01:36:19.259 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T01:30:32.200808+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-26T01:35:26.320950+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2023-10-26T01:35:26.855916+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T01:36:19.260 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T01:36:19.475 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.472+0000 7f0a4036e640 1 -- 172.21.15.169:0/1217202504 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0a380b0a80 msgr2=0x7f0a38158f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:19.476 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.472+0000 7f0a4036e640 1 --2- 172.21.15.169:0/1217202504 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0a380b0a80 0x7f0a38158f70 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f0a3806c3b0 tx=0x7f0a2802bdb0 comp rx=0 tx=0).stop 2023-10-26T01:36:19.476 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.472+0000 7f0a4036e640 1 -- 172.21.15.169:0/1217202504 shutdown_connections 2023-10-26T01:36:19.476 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.472+0000 7f0a4036e640 1 --2- 172.21.15.169:0/1217202504 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0a380b0a80 0x7f0a38158f70 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:19.476 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.472+0000 7f0a4036e640 1 --2- 172.21.15.169:0/1217202504 >> v2:172.21.15.170:3300/0 conn(0x7f0a380ab070 0x7f0a380ac870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:19.477 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.472+0000 7f0a4036e640 1 -- 172.21.15.169:0/1217202504 >> 172.21.15.169:0/1217202504 conn(0x7f0a38069930 msgr2=0x7f0a38069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:19.477 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.472+0000 7f0a4036e640 1 -- 172.21.15.169:0/1217202504 shutdown_connections 2023-10-26T01:36:19.477 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.472+0000 7f0a4036e640 1 -- 172.21.15.169:0/1217202504 wait complete. 2023-10-26T01:36:19.479 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.474+0000 7f0a4036e640 1 Processor -- start 2023-10-26T01:36:19.479 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.474+0000 7f0a4036e640 1 -- start start 2023-10-26T01:36:19.480 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.475+0000 7f0a4036e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0a380ab070 0x7f0a3814f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:19.480 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.475+0000 7f0a4036e640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0a380cd760 0x7f0a380cc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:19.480 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.475+0000 7f0a4036e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a380c9460 0x7f0a380c9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:19.480 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.475+0000 7f0a4036e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a3806fa70 con 0x7f0a380ab070 2023-10-26T01:36:19.480 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.475+0000 7f0a4036e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a3806edf0 con 0x7f0a380c9460 2023-10-26T01:36:19.481 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.475+0000 7f0a4036e640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a3806f0d0 con 0x7f0a380cd760 2023-10-26T01:36:19.481 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.475+0000 7f0a3e10a640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0a380ab070 0x7f0a3814f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:19.481 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.475+0000 7f0a3e10a640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0a380ab070 0x7f0a3814f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:50264/0 (socket says 172.21.15.169:50264) 2023-10-26T01:36:19.481 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.475+0000 7f0a3e10a640 1 -- 172.21.15.169:0/1932440396 learned_addr learned my addr 172.21.15.169:0/1932440396 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:36:19.482 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.476+0000 7f0a3e10a640 1 -- 172.21.15.169:0/1932440396 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0a380cd760 msgr2=0x7f0a380cc4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:19.482 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.476+0000 7f0a3e10a640 1 --2- 172.21.15.169:0/1932440396 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0a380cd760 0x7f0a380cc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:19.482 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.476+0000 7f0a3e10a640 1 -- 172.21.15.169:0/1932440396 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a380c9460 msgr2=0x7f0a380c9cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:19.483 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.476+0000 7f0a3e10a640 1 --2- 172.21.15.169:0/1932440396 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a380c9460 0x7f0a380c9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:19.483 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.476+0000 7f0a3e10a640 1 -- 172.21.15.169:0/1932440396 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a28009560 con 0x7f0a380ab070 2023-10-26T01:36:19.483 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.476+0000 7f0a3e10a640 1 --2- 172.21.15.169:0/1932440396 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0a380ab070 0x7f0a3814f9f0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f0a3400b4a0 tx=0x7f0a3400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:19.483 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.476+0000 7f0a2effd640 1 -- 172.21.15.169:0/1932440396 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a34010610 con 0x7f0a380ab070 2023-10-26T01:36:19.484 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.476+0000 7f0a2effd640 1 -- 172.21.15.169:0/1932440396 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0a34010860 con 0x7f0a380ab070 2023-10-26T01:36:19.484 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.476+0000 7f0a2effd640 1 -- 172.21.15.169:0/1932440396 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a34010c70 con 0x7f0a380ab070 2023-10-26T01:36:19.484 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.476+0000 7f0a4036e640 1 -- 172.21.15.169:0/1932440396 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a38159670 con 0x7f0a380ab070 2023-10-26T01:36:19.484 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.476+0000 7f0a4036e640 1 -- 172.21.15.169:0/1932440396 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a3806f690 con 0x7f0a380ab070 2023-10-26T01:36:19.485 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.478+0000 7f0a4036e640 1 -- 172.21.15.169:0/1932440396 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a38066af0 con 0x7f0a380ab070 2023-10-26T01:36:19.485 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.479+0000 7f0a2effd640 1 -- 172.21.15.169:0/1932440396 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f0a34017360 con 0x7f0a380ab070 2023-10-26T01:36:19.486 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.479+0000 7f0a2effd640 1 --2- 172.21.15.169:0/1932440396 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f0a1c06ae40 0x7f0a1c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:19.486 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.479+0000 7f0a2effd640 1 -- 172.21.15.169:0/1932440396 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0a34089510 con 0x7f0a380ab070 2023-10-26T01:36:19.486 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.480+0000 7f0a2effd640 1 -- 172.21.15.169:0/1932440396 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=89}) v3 -- 0x7f0a1c072970 con 0x7f0a380ab070 2023-10-26T01:36:19.486 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.480+0000 7f0a3d909640 1 --2- 172.21.15.169:0/1932440396 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f0a1c06ae40 0x7f0a1c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:19.486 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.481+0000 7f0a3d909640 1 --2- 172.21.15.169:0/1932440396 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f0a1c06ae40 0x7f0a1c06d300 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f0a28035000 tx=0x7f0a280046c0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:19.487 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.484+0000 7f0a2effd640 1 -- 172.21.15.169:0/1932440396 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0a340545b0 con 0x7f0a380ab070 2023-10-26T01:36:19.677 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.674+0000 7f0a4036e640 1 -- 172.21.15.169:0/1932440396 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0a3806cd30 con 0x7f0a380ab070 2023-10-26T01:36:19.678 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.675+0000 7f0a2effd640 1 -- 172.21.15.169:0/1932440396 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0a340140a0 con 0x7f0a380ab070 2023-10-26T01:36:19.678 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 1 2023-10-26T01:36:19.680 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.677+0000 7f0a4036e640 1 -- 172.21.15.169:0/1932440396 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f0a1c06ae40 msgr2=0x7f0a1c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:19.681 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.677+0000 7f0a4036e640 1 --2- 172.21.15.169:0/1932440396 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f0a1c06ae40 0x7f0a1c06d300 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f0a28035000 tx=0x7f0a280046c0 comp rx=0 tx=0).stop 2023-10-26T01:36:19.681 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.677+0000 7f0a4036e640 1 -- 172.21.15.169:0/1932440396 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0a380ab070 msgr2=0x7f0a3814f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:19.681 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.677+0000 7f0a4036e640 1 --2- 172.21.15.169:0/1932440396 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0a380ab070 0x7f0a3814f9f0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f0a3400b4a0 tx=0x7f0a3400b960 comp rx=0 tx=0).stop 2023-10-26T01:36:19.681 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.677+0000 7f0a4036e640 1 -- 172.21.15.169:0/1932440396 shutdown_connections 2023-10-26T01:36:19.681 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.677+0000 7f0a4036e640 1 --2- 172.21.15.169:0/1932440396 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f0a1c06ae40 0x7f0a1c06d300 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:19.682 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.677+0000 7f0a4036e640 1 --2- 172.21.15.169:0/1932440396 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a380c9460 0x7f0a380c9cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:19.682 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.678+0000 7f0a4036e640 1 --2- 172.21.15.169:0/1932440396 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0a380cd760 0x7f0a380cc4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:19.682 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.678+0000 7f0a4036e640 1 --2- 172.21.15.169:0/1932440396 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0a380ab070 0x7f0a3814f9f0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:19.682 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.678+0000 7f0a4036e640 1 -- 172.21.15.169:0/1932440396 >> 172.21.15.169:0/1932440396 conn(0x7f0a38069930 msgr2=0x7f0a380aa250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:19.682 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.678+0000 7f0a4036e640 1 -- 172.21.15.169:0/1932440396 shutdown_connections 2023-10-26T01:36:19.683 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.678+0000 7f0a4036e640 1 -- 172.21.15.169:0/1932440396 wait complete. 2023-10-26T01:36:19.698 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-26T01:36:19.928 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.924+0000 7f8a54b33640 1 -- 172.21.15.169:0/3867448136 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a500c9d00 msgr2=0x7f8a500ac320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:19.928 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.924+0000 7f8a54b33640 1 --2- 172.21.15.169:0/3867448136 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a500c9d00 0x7f8a500ac320 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f8a500abb80 tx=0x7f8a3802bd70 comp rx=0 tx=0).stop 2023-10-26T01:36:19.928 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.924+0000 7f8a54b33640 1 -- 172.21.15.169:0/3867448136 shutdown_connections 2023-10-26T01:36:19.929 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.924+0000 7f8a54b33640 1 --2- 172.21.15.169:0/3867448136 >> v2:172.21.15.170:3300/0 conn(0x7f8a500cde20 0x7f8a500ac860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:19.929 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.924+0000 7f8a54b33640 1 --2- 172.21.15.169:0/3867448136 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a500c9d00 0x7f8a500ac320 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:19.929 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.924+0000 7f8a54b33640 1 -- 172.21.15.169:0/3867448136 >> 172.21.15.169:0/3867448136 conn(0x7f8a50069930 msgr2=0x7f8a50069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:19.929 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.924+0000 7f8a54b33640 1 -- 172.21.15.169:0/3867448136 shutdown_connections 2023-10-26T01:36:19.929 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.924+0000 7f8a54b33640 1 -- 172.21.15.169:0/3867448136 wait complete. 2023-10-26T01:36:19.931 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.926+0000 7f8a54b33640 1 Processor -- start 2023-10-26T01:36:19.932 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.926+0000 7f8a54b33640 1 -- start start 2023-10-26T01:36:19.932 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.927+0000 7f8a54b33640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a500c9d00 0x7f8a5014c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:19.932 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.927+0000 7f8a54b33640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f8a500cde20 0x7f8a50153ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:19.932 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.927+0000 7f8a54b33640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a5015b620 0x7f8a5006bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:19.932 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.927+0000 7f8a54b33640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a501f6140 con 0x7f8a5015b620 2023-10-26T01:36:19.933 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.927+0000 7f8a54b33640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a5014c5b0 con 0x7f8a500c9d00 2023-10-26T01:36:19.933 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.927+0000 7f8a54b33640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a501f5500 con 0x7f8a500cde20 2023-10-26T01:36:19.933 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.928+0000 7f8a4ed9d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a5015b620 0x7f8a5006bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:19.933 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.928+0000 7f8a4ed9d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a5015b620 0x7f8a5006bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:50270/0 (socket says 172.21.15.169:50270) 2023-10-26T01:36:19.934 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.928+0000 7f8a4ed9d640 1 -- 172.21.15.169:0/3356951037 learned_addr learned my addr 172.21.15.169:0/3356951037 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:36:19.934 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.928+0000 7f8a4e59c640 1 --2- 172.21.15.169:0/3356951037 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a500c9d00 0x7f8a5014c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:19.934 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.928+0000 7f8a4ed9d640 1 -- 172.21.15.169:0/3356951037 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f8a500cde20 msgr2=0x7f8a50153ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:19.934 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.928+0000 7f8a4ed9d640 1 --2- 172.21.15.169:0/3356951037 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f8a500cde20 0x7f8a50153ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:19.935 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.928+0000 7f8a4ed9d640 1 -- 172.21.15.169:0/3356951037 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a500c9d00 msgr2=0x7f8a5014c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:19.935 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.928+0000 7f8a4ed9d640 1 --2- 172.21.15.169:0/3356951037 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a500c9d00 0x7f8a5014c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:19.935 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.928+0000 7f8a4ed9d640 1 -- 172.21.15.169:0/3356951037 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a38009490 con 0x7f8a5015b620 2023-10-26T01:36:19.936 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.928+0000 7f8a4e59c640 1 --2- 172.21.15.169:0/3356951037 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a500c9d00 0x7f8a5014c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:36:19.936 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.928+0000 7f8a4ed9d640 1 --2- 172.21.15.169:0/3356951037 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a5015b620 0x7f8a5006bca0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f8a4800cce0 tx=0x7f8a4800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:19.936 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.929+0000 7f8a46ffd640 1 -- 172.21.15.169:0/3356951037 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a4800f450 con 0x7f8a5015b620 2023-10-26T01:36:19.936 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.929+0000 7f8a46ffd640 1 -- 172.21.15.169:0/3356951037 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8a4800f6c0 con 0x7f8a5015b620 2023-10-26T01:36:19.937 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.929+0000 7f8a54b33640 1 -- 172.21.15.169:0/3356951037 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a501f57a0 con 0x7f8a5015b620 2023-10-26T01:36:19.937 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.929+0000 7f8a46ffd640 1 -- 172.21.15.169:0/3356951037 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a4800fad0 con 0x7f8a5015b620 2023-10-26T01:36:19.937 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.929+0000 7f8a54b33640 1 -- 172.21.15.169:0/3356951037 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a501f5d60 con 0x7f8a5015b620 2023-10-26T01:36:19.939 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.930+0000 7f8a54b33640 1 -- 172.21.15.169:0/3356951037 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a501f6280 con 0x7f8a5015b620 2023-10-26T01:36:19.940 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.931+0000 7f8a46ffd640 1 -- 172.21.15.169:0/3356951037 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f8a4800fc30 con 0x7f8a5015b620 2023-10-26T01:36:19.940 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.932+0000 7f8a46ffd640 1 --2- 172.21.15.169:0/3356951037 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f8a3006ae40 0x7f8a3006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:19.940 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.932+0000 7f8a46ffd640 1 -- 172.21.15.169:0/3356951037 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8a4808a260 con 0x7f8a5015b620 2023-10-26T01:36:19.940 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.932+0000 7f8a46ffd640 1 -- 172.21.15.169:0/3356951037 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=89}) v3 -- 0x7f8a30072970 con 0x7f8a5015b620 2023-10-26T01:36:19.941 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.932+0000 7f8a4e59c640 1 --2- 172.21.15.169:0/3356951037 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f8a3006ae40 0x7f8a3006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:19.941 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.934+0000 7f8a4e59c640 1 --2- 172.21.15.169:0/3356951037 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f8a3006ae40 0x7f8a3006d300 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f8a400046b0 tx=0x7f8a40009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:19.941 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:19.936+0000 7f8a46ffd640 1 -- 172.21.15.169:0/3356951037 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8a48055300 con 0x7f8a5015b620 2023-10-26T01:36:20.131 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.128+0000 7f8a54b33640 1 -- 172.21.15.169:0/3356951037 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8a5004db10 con 0x7f8a5015b620 2023-10-26T01:36:20.132 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.129+0000 7f8a46ffd640 1 -- 172.21.15.169:0/3356951037 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8a4801d020 con 0x7f8a5015b620 2023-10-26T01:36:20.132 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 32 2023-10-26T01:36:20.135 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.132+0000 7f8a54b33640 1 -- 172.21.15.169:0/3356951037 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f8a3006ae40 msgr2=0x7f8a3006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:20.135 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.132+0000 7f8a54b33640 1 --2- 172.21.15.169:0/3356951037 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f8a3006ae40 0x7f8a3006d300 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f8a400046b0 tx=0x7f8a40009040 comp rx=0 tx=0).stop 2023-10-26T01:36:20.135 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.132+0000 7f8a54b33640 1 -- 172.21.15.169:0/3356951037 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a5015b620 msgr2=0x7f8a5006bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:20.135 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.132+0000 7f8a54b33640 1 --2- 172.21.15.169:0/3356951037 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a5015b620 0x7f8a5006bca0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f8a4800cce0 tx=0x7f8a4800a620 comp rx=0 tx=0).stop 2023-10-26T01:36:20.136 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.132+0000 7f8a54b33640 1 -- 172.21.15.169:0/3356951037 shutdown_connections 2023-10-26T01:36:20.136 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.132+0000 7f8a54b33640 1 --2- 172.21.15.169:0/3356951037 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f8a3006ae40 0x7f8a3006d300 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:20.136 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.132+0000 7f8a54b33640 1 --2- 172.21.15.169:0/3356951037 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a5015b620 0x7f8a5006bca0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:20.136 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.132+0000 7f8a54b33640 1 --2- 172.21.15.169:0/3356951037 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f8a500cde20 0x7f8a50153ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:20.136 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.132+0000 7f8a54b33640 1 --2- 172.21.15.169:0/3356951037 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a500c9d00 0x7f8a5014c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:20.137 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.132+0000 7f8a54b33640 1 -- 172.21.15.169:0/3356951037 >> 172.21.15.169:0/3356951037 conn(0x7f8a50069930 msgr2=0x7f8a5015a8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:20.137 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.132+0000 7f8a54b33640 1 -- 172.21.15.169:0/3356951037 shutdown_connections 2023-10-26T01:36:20.137 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.132+0000 7f8a54b33640 1 -- 172.21.15.169:0/3356951037 wait complete. 2023-10-26T01:36:20.154 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-26T01:36:20.377 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.374+0000 7f8267b4c640 1 -- 172.21.15.169:0/3640803110 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f824c003ef0 msgr2=0x7f824c0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:20.377 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.374+0000 7f8267b4c640 1 --2- 172.21.15.169:0/3640803110 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f824c003ef0 0x7f824c0047c0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f8250009b50 tx=0x7f824c0057f0 comp rx=0 tx=0).stop 2023-10-26T01:36:20.377 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.374+0000 7f82660e9640 1 -- 172.21.15.169:0/3640803110 reap_dead start 2023-10-26T01:36:20.377 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.374+0000 7f8267b4c640 1 -- 172.21.15.169:0/3640803110 shutdown_connections 2023-10-26T01:36:20.378 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.374+0000 7f8267b4c640 1 -- 172.21.15.169:0/3640803110 >> 172.21.15.169:0/3640803110 conn(0x7f8260069930 msgr2=0x7f8260069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:20.378 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.374+0000 7f8267b4c640 1 -- 172.21.15.169:0/3640803110 shutdown_connections 2023-10-26T01:36:20.378 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.374+0000 7f8267b4c640 1 -- 172.21.15.169:0/3640803110 wait complete. 2023-10-26T01:36:20.379 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.376+0000 7f8267b4c640 1 Processor -- start 2023-10-26T01:36:20.379 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.376+0000 7f8267b4c640 1 -- start start 2023-10-26T01:36:20.381 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.377+0000 7f8267b4c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f824c003ef0 0x7f82600addb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:20.381 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.377+0000 7f8267b4c640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f82601f1910 0x7f82601565b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:20.381 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.377+0000 7f8267b4c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f826014e490 0x7f82600af430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:20.381 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.377+0000 7f8267b4c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f826014e300 con 0x7f826014e490 2023-10-26T01:36:20.382 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.377+0000 7f8267b4c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82600ae390 con 0x7f824c003ef0 2023-10-26T01:36:20.382 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.377+0000 7f8267b4c640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82600ae670 con 0x7f82601f1910 2023-10-26T01:36:20.382 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.377+0000 7f82660e9640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f826014e490 0x7f82600af430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:20.382 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.377+0000 7f82660e9640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f826014e490 0x7f82600af430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:50290/0 (socket says 172.21.15.169:50290) 2023-10-26T01:36:20.383 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.377+0000 7f82660e9640 1 -- 172.21.15.169:0/1960577706 learned_addr learned my addr 172.21.15.169:0/1960577706 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:36:20.383 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.377+0000 7f82650e7640 1 --2- 172.21.15.169:0/1960577706 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f82601f1910 0x7f82601565b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:20.383 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.377+0000 7f82658e8640 1 --2- 172.21.15.169:0/1960577706 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f824c003ef0 0x7f82600addb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:20.383 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.378+0000 7f82660e9640 1 -- 172.21.15.169:0/1960577706 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f82601f1910 msgr2=0x7f82601565b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:20.383 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.378+0000 7f82660e9640 1 --2- 172.21.15.169:0/1960577706 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f82601f1910 0x7f82601565b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:20.384 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.378+0000 7f82660e9640 1 -- 172.21.15.169:0/1960577706 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f824c003ef0 msgr2=0x7f82600addb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:20.384 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.378+0000 7f82660e9640 1 --2- 172.21.15.169:0/1960577706 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f824c003ef0 0x7f82600addb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:20.384 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.378+0000 7f82660e9640 1 -- 172.21.15.169:0/1960577706 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82600ae950 con 0x7f826014e490 2023-10-26T01:36:20.384 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.378+0000 7f82658e8640 1 --2- 172.21.15.169:0/1960577706 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f824c003ef0 0x7f82600addb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:36:20.384 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.378+0000 7f82660e9640 1 --2- 172.21.15.169:0/1960577706 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f826014e490 0x7f82600af430 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f8258004a50 tx=0x7f825800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:20.384 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.378+0000 7f82567fc640 1 -- 172.21.15.169:0/1960577706 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f825800aeb0 con 0x7f826014e490 2023-10-26T01:36:20.385 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.378+0000 7f82567fc640 1 -- 172.21.15.169:0/1960577706 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8258007070 con 0x7f826014e490 2023-10-26T01:36:20.385 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.378+0000 7f82567fc640 1 -- 172.21.15.169:0/1960577706 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8258007480 con 0x7f826014e490 2023-10-26T01:36:20.385 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.378+0000 7f8267b4c640 1 -- 172.21.15.169:0/1960577706 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82600aec30 con 0x7f826014e490 2023-10-26T01:36:20.385 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.378+0000 7f8267b4c640 1 -- 172.21.15.169:0/1960577706 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82601f1d90 con 0x7f826014e490 2023-10-26T01:36:20.386 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.380+0000 7f8267b4c640 1 -- 172.21.15.169:0/1960577706 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f826004db10 con 0x7f826014e490 2023-10-26T01:36:20.387 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.380+0000 7f82567fc640 1 -- 172.21.15.169:0/1960577706 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f82580075e0 con 0x7f826014e490 2023-10-26T01:36:20.387 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.381+0000 7f82567fc640 1 --2- 172.21.15.169:0/1960577706 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f824406ae70 0x7f824406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:20.387 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.381+0000 7f82567fc640 1 -- 172.21.15.169:0/1960577706 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f82580895b0 con 0x7f826014e490 2023-10-26T01:36:20.387 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.382+0000 7f82567fc640 1 -- 172.21.15.169:0/1960577706 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=89}) v3 -- 0x7f82440729a0 con 0x7f826014e490 2023-10-26T01:36:20.387 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.382+0000 7f82658e8640 1 --2- 172.21.15.169:0/1960577706 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f824406ae70 0x7f824406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:20.388 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.383+0000 7f82658e8640 1 --2- 172.21.15.169:0/1960577706 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f824406ae70 0x7f824406d330 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f825c0099f0 tx=0x7f825c005740 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:20.388 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.386+0000 7f82567fc640 1 -- 172.21.15.169:0/1960577706 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8258017030 con 0x7f826014e490 2023-10-26T01:36:20.580 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.577+0000 7f8267b4c640 1 -- 172.21.15.169:0/1960577706 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f82600b00c0 con 0x7f826014e490 2023-10-26T01:36:20.580 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.578+0000 7f82567fc640 1 -- 172.21.15.169:0/1960577706 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8258017020 con 0x7f826014e490 2023-10-26T01:36:20.581 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 64 2023-10-26T01:36:20.584 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.581+0000 7f8267b4c640 1 -- 172.21.15.169:0/1960577706 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f824406ae70 msgr2=0x7f824406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:20.584 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.581+0000 7f8267b4c640 1 --2- 172.21.15.169:0/1960577706 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f824406ae70 0x7f824406d330 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f825c0099f0 tx=0x7f825c005740 comp rx=0 tx=0).stop 2023-10-26T01:36:20.584 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.581+0000 7f8267b4c640 1 -- 172.21.15.169:0/1960577706 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f826014e490 msgr2=0x7f82600af430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:20.584 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.581+0000 7f8267b4c640 1 --2- 172.21.15.169:0/1960577706 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f826014e490 0x7f82600af430 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f8258004a50 tx=0x7f825800cf90 comp rx=0 tx=0).stop 2023-10-26T01:36:20.585 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.581+0000 7f8267b4c640 1 -- 172.21.15.169:0/1960577706 shutdown_connections 2023-10-26T01:36:20.585 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.581+0000 7f8267b4c640 1 --2- 172.21.15.169:0/1960577706 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f824406ae70 0x7f824406d330 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:20.585 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.581+0000 7f8267b4c640 1 --2- 172.21.15.169:0/1960577706 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f826014e490 0x7f82600af430 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:20.585 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.581+0000 7f8267b4c640 1 --2- 172.21.15.169:0/1960577706 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f82601f1910 0x7f82601565b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:20.585 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.581+0000 7f8267b4c640 1 --2- 172.21.15.169:0/1960577706 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f824c003ef0 0x7f82600addb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:20.585 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.581+0000 7f8267b4c640 1 -- 172.21.15.169:0/1960577706 >> 172.21.15.169:0/1960577706 conn(0x7f8260069930 msgr2=0x7f82600ab5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:20.586 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.581+0000 7f8267b4c640 1 -- 172.21.15.169:0/1960577706 shutdown_connections 2023-10-26T01:36:20.586 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.581+0000 7f8267b4c640 1 -- 172.21.15.169:0/1960577706 wait complete. 2023-10-26T01:36:20.602 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-26T01:36:20.830 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.827+0000 7fa539173640 1 -- 172.21.15.169:0/2352351653 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa5341597c0 msgr2=0x7fa53415dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:20.831 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.827+0000 7fa539173640 1 --2- 172.21.15.169:0/2352351653 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa5341597c0 0x7fa53415dbf0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fa5340cd2c0 tx=0x7fa51c02bda0 comp rx=0 tx=0).stop 2023-10-26T01:36:20.831 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.827+0000 7fa539173640 1 -- 172.21.15.169:0/2352351653 shutdown_connections 2023-10-26T01:36:20.832 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.827+0000 7fa539173640 1 --2- 172.21.15.169:0/2352351653 >> v2:172.21.15.170:3300/0 conn(0x7fa5340b0c80 0x7fa5340cd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:20.832 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.827+0000 7fa539173640 1 --2- 172.21.15.169:0/2352351653 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa5341597c0 0x7fa53415dbf0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:20.832 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.827+0000 7fa539173640 1 --2- 172.21.15.169:0/2352351653 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa5341593e0 0x7fa5340ac200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:20.833 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.827+0000 7fa539173640 1 -- 172.21.15.169:0/2352351653 >> 172.21.15.169:0/2352351653 conn(0x7fa534069930 msgr2=0x7fa534069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:20.833 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.827+0000 7fa539173640 1 -- 172.21.15.169:0/2352351653 shutdown_connections 2023-10-26T01:36:20.833 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.827+0000 7fa539173640 1 -- 172.21.15.169:0/2352351653 wait complete. 2023-10-26T01:36:20.835 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.829+0000 7fa539173640 1 Processor -- start 2023-10-26T01:36:20.835 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.829+0000 7fa539173640 1 -- start start 2023-10-26T01:36:20.835 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.830+0000 7fa539173640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa5340b0c80 0x7fa53406c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:20.836 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.830+0000 7fa539173640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa5341593e0 0x7fa534150500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:20.836 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.830+0000 7fa539173640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa5341597c0 0x7fa5341f1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:20.836 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.830+0000 7fa539173640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5341f29c0 con 0x7fa5340b0c80 2023-10-26T01:36:20.836 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.830+0000 7fa539173640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5341f2400 con 0x7fa5341597c0 2023-10-26T01:36:20.836 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.830+0000 7fa539173640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5341f26e0 con 0x7fa5341593e0 2023-10-26T01:36:20.837 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.830+0000 7fa532d9d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa5340b0c80 0x7fa53406c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:20.837 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.830+0000 7fa532d9d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa5340b0c80 0x7fa53406c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:50306/0 (socket says 172.21.15.169:50306) 2023-10-26T01:36:20.837 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.830+0000 7fa532d9d640 1 -- 172.21.15.169:0/841484482 learned_addr learned my addr 172.21.15.169:0/841484482 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:36:20.837 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.831+0000 7fa53359e640 1 --2- 172.21.15.169:0/841484482 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa5341597c0 0x7fa5341f1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:20.837 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.831+0000 7fa52bfff640 1 --2- 172.21.15.169:0/841484482 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa5341593e0 0x7fa534150500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:20.838 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.831+0000 7fa532d9d640 1 -- 172.21.15.169:0/841484482 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa5341593e0 msgr2=0x7fa534150500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:20.838 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.831+0000 7fa532d9d640 1 --2- 172.21.15.169:0/841484482 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa5341593e0 0x7fa534150500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:20.839 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.831+0000 7fa532d9d640 1 -- 172.21.15.169:0/841484482 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa5341597c0 msgr2=0x7fa5341f1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:20.839 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.831+0000 7fa532d9d640 1 --2- 172.21.15.169:0/841484482 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa5341597c0 0x7fa5341f1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:20.839 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.831+0000 7fa532d9d640 1 -- 172.21.15.169:0/841484482 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa51c009560 con 0x7fa5340b0c80 2023-10-26T01:36:20.840 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.831+0000 7fa52bfff640 1 --2- 172.21.15.169:0/841484482 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa5341593e0 0x7fa534150500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:36:20.840 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.831+0000 7fa532d9d640 1 --2- 172.21.15.169:0/841484482 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa5340b0c80 0x7fa53406c7f0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fa52400c920 tx=0x7fa52400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:20.840 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.831+0000 7fa52b7fe640 1 -- 172.21.15.169:0/841484482 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa524007c70 con 0x7fa5340b0c80 2023-10-26T01:36:20.841 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.831+0000 7fa52b7fe640 1 -- 172.21.15.169:0/841484482 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa52400f450 con 0x7fa5340b0c80 2023-10-26T01:36:20.841 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.832+0000 7fa52b7fe640 1 -- 172.21.15.169:0/841484482 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa52400f860 con 0x7fa5340b0c80 2023-10-26T01:36:20.841 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.832+0000 7fa539173640 1 -- 172.21.15.169:0/841484482 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa53406bff0 con 0x7fa5340b0c80 2023-10-26T01:36:20.842 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.832+0000 7fa539173640 1 -- 172.21.15.169:0/841484482 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa53406cd30 con 0x7fa5340b0c80 2023-10-26T01:36:20.843 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.833+0000 7fa539173640 1 -- 172.21.15.169:0/841484482 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa53404db10 con 0x7fa5340b0c80 2023-10-26T01:36:20.843 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.834+0000 7fa52b7fe640 1 -- 172.21.15.169:0/841484482 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fa52400f9c0 con 0x7fa5340b0c80 2023-10-26T01:36:20.843 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.835+0000 7fa52b7fe640 1 --2- 172.21.15.169:0/841484482 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa51406ae40 0x7fa51406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:20.844 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.835+0000 7fa52b7fe640 1 -- 172.21.15.169:0/841484482 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa524089e20 con 0x7fa5340b0c80 2023-10-26T01:36:20.844 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.835+0000 7fa52b7fe640 1 -- 172.21.15.169:0/841484482 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=89}) v3 -- 0x7fa514072970 con 0x7fa5340b0c80 2023-10-26T01:36:20.844 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.835+0000 7fa52bfff640 1 --2- 172.21.15.169:0/841484482 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa51406ae40 0x7fa51406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:20.844 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.836+0000 7fa52bfff640 1 --2- 172.21.15.169:0/841484482 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa51406ae40 0x7fa51406d300 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fa51c035000 tx=0x7fa51c004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:20.845 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:20.841+0000 7fa52b7fe640 1 -- 172.21.15.169:0/841484482 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa52401d060 con 0x7fa5340b0c80 2023-10-26T01:36:21.041 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:21.039+0000 7fa539173640 1 -- 172.21.15.169:0/841484482 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fa5340b2170 con 0x7fa5340b0c80 2023-10-26T01:36:21.042 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:21.039+0000 7fa52b7fe640 1 -- 172.21.15.169:0/841484482 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v88) v1 ==== 80+0+0 (secure 0 0 0) 0x7fa524054f60 con 0x7fa5340b0c80 2023-10-26T01:36:21.042 INFO:teuthology.orchestra.run.smithi169.stderr:listed 0 entries 2023-10-26T01:36:21.045 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:21.041+0000 7fa539173640 1 -- 172.21.15.169:0/841484482 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa51406ae40 msgr2=0x7fa51406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:21.045 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:21.041+0000 7fa539173640 1 --2- 172.21.15.169:0/841484482 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa51406ae40 0x7fa51406d300 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fa51c035000 tx=0x7fa51c004730 comp rx=0 tx=0).stop 2023-10-26T01:36:21.045 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:21.042+0000 7fa539173640 1 -- 172.21.15.169:0/841484482 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa5340b0c80 msgr2=0x7fa53406c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:21.045 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:21.042+0000 7fa539173640 1 --2- 172.21.15.169:0/841484482 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa5340b0c80 0x7fa53406c7f0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fa52400c920 tx=0x7fa52400cde0 comp rx=0 tx=0).stop 2023-10-26T01:36:21.045 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:21.042+0000 7fa539173640 1 -- 172.21.15.169:0/841484482 shutdown_connections 2023-10-26T01:36:21.046 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:21.042+0000 7fa539173640 1 --2- 172.21.15.169:0/841484482 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa51406ae40 0x7fa51406d300 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:21.046 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:21.042+0000 7fa539173640 1 --2- 172.21.15.169:0/841484482 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa5341597c0 0x7fa5341f1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:21.046 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:21.042+0000 7fa539173640 1 --2- 172.21.15.169:0/841484482 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa5341593e0 0x7fa534150500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:21.046 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:21.042+0000 7fa539173640 1 --2- 172.21.15.169:0/841484482 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa5340b0c80 0x7fa53406c7f0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:21.046 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:21.042+0000 7fa539173640 1 -- 172.21.15.169:0/841484482 >> 172.21.15.169:0/841484482 conn(0x7fa534069930 msgr2=0x7fa5340aa720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:21.047 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:21.042+0000 7fa539173640 1 -- 172.21.15.169:0/841484482 shutdown_connections 2023-10-26T01:36:21.047 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:21.042+0000 7fa539173640 1 -- 172.21.15.169:0/841484482 wait complete. 2023-10-26T01:36:21.063 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-26T01:36:21.063 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:36:21.064 DEBUG:teuthology.orchestra.run.smithi169:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-26T01:36:21.166 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi169.front.sepia.ceph.com 2023-10-26T01:36:21.166 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:36:21.166 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-26T01:36:21.217 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-26T01:36:21.217 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:36:21.276 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-26T01:36:21.277 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:36:21.331 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:36:21.586 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:21.583+0000 7efcf26fc640 1 -- 172.21.15.169:0/932467658 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efcd80047c0 msgr2=0x7efcd8004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:21.586 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:21.583+0000 7efcf26fc640 1 --2- 172.21.15.169:0/932467658 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efcd80047c0 0x7efcd8004c40 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7efcdc0015b0 tx=0x7efcd80057f0 comp rx=0 tx=0).stop 2023-10-26T01:36:21.586 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:21.583+0000 7efcf26fc640 1 -- 172.21.15.169:0/932467658 shutdown_connections 2023-10-26T01:36:21.587 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:21.583+0000 7efcf26fc640 1 --2- 172.21.15.169:0/932467658 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efcd80047c0 0x7efcd8004c40 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:21.587 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:21.583+0000 7efcf26fc640 1 -- 172.21.15.169:0/932467658 >> 172.21.15.169:0/932467658 conn(0x7efcec0aedd0 msgr2=0x7efcec0af1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:21.587 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:21.583+0000 7efcf26fc640 1 -- 172.21.15.169:0/932467658 shutdown_connections 2023-10-26T01:36:21.587 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:21.583+0000 7efcf26fc640 1 -- 172.21.15.169:0/932467658 wait complete. 2023-10-26T01:36:21.588 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:21.586+0000 7efcf26fc640 1 Processor -- start 2023-10-26T01:36:21.589 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:21.586+0000 7efcf26fc640 1 -- start start 2023-10-26T01:36:21.591 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:21.588+0000 7efcf26fc640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efcd80047c0 0x7efcec1445b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:21.591 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:21.589+0000 7efcf26fc640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7efcec070e10 0x7efcec140580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:21.592 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:21.589+0000 7efcf26fc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efcec140ac0 0x7efcec140f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:21.592 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:21.589+0000 7efcf26fc640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcec144af0 con 0x7efcd80047c0 2023-10-26T01:36:21.592 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:21.589+0000 7efcf26fc640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcec1545d0 con 0x7efcec140ac0 2023-10-26T01:36:21.592 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:21.589+0000 7efcf26fc640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcec154870 con 0x7efcec070e10 2023-10-26T01:36:21.593 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:21.589+0000 7efcf16fa640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efcd80047c0 0x7efcec1445b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:21.593 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:21.589+0000 7efcf16fa640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efcd80047c0 0x7efcec1445b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:50310/0 (socket says 172.21.15.169:50310) 2023-10-26T01:36:21.593 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:21.589+0000 7efcf16fa640 1 -- 172.21.15.169:0/2918623885 learned_addr learned my addr 172.21.15.169:0/2918623885 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:36:21.593 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:21.590+0000 7efcf1efb640 1 --2- 172.21.15.169:0/2918623885 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efcec140ac0 0x7efcec140f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:21.595 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:21.590+0000 7efcf0ef9640 1 --2- 172.21.15.169:0/2918623885 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7efcec070e10 0x7efcec140580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:21.595 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:21.590+0000 7efcf16fa640 1 -- 172.21.15.169:0/2918623885 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7efcec070e10 msgr2=0x7efcec140580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:21.595 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:21.590+0000 7efcf16fa640 1 --2- 172.21.15.169:0/2918623885 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7efcec070e10 0x7efcec140580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:21.595 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:21.590+0000 7efcf16fa640 1 -- 172.21.15.169:0/2918623885 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efcec140ac0 msgr2=0x7efcec140f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:21.595 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:21.590+0000 7efcf16fa640 1 --2- 172.21.15.169:0/2918623885 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efcec140ac0 0x7efcec140f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:21.595 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:21.590+0000 7efcf16fa640 1 -- 172.21.15.169:0/2918623885 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcec156c60 con 0x7efcd80047c0 2023-10-26T01:36:21.596 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:21.591+0000 7efcf16fa640 1 --2- 172.21.15.169:0/2918623885 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efcd80047c0 0x7efcec1445b0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7efce8011a40 tx=0x7efce8011f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:21.596 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:21.591+0000 7efce1ffb640 1 -- 172.21.15.169:0/2918623885 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efce8004460 con 0x7efcd80047c0 2023-10-26T01:36:21.596 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:21.591+0000 7efce1ffb640 1 -- 172.21.15.169:0/2918623885 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7efce800f9a0 con 0x7efcd80047c0 2023-10-26T01:36:21.596 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:21.592+0000 7efce1ffb640 1 -- 172.21.15.169:0/2918623885 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efce800fd90 con 0x7efcd80047c0 2023-10-26T01:36:21.596 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:21.592+0000 7efcf26fc640 1 -- 172.21.15.169:0/2918623885 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcec156f40 con 0x7efcd80047c0 2023-10-26T01:36:21.597 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:21.592+0000 7efcf26fc640 1 -- 172.21.15.169:0/2918623885 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcec157460 con 0x7efcd80047c0 2023-10-26T01:36:21.597 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:21.593+0000 7efcf26fc640 1 -- 172.21.15.169:0/2918623885 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcec1549b0 con 0x7efcd80047c0 2023-10-26T01:36:21.598 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:21.596+0000 7efce1ffb640 1 -- 172.21.15.169:0/2918623885 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7efce80035f0 con 0x7efcd80047c0 2023-10-26T01:36:21.600 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:21.597+0000 7efce1ffb640 1 --2- 172.21.15.169:0/2918623885 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7efcd006ae90 0x7efcd006d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:21.600 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:21.597+0000 7efce1ffb640 1 -- 172.21.15.169:0/2918623885 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efce808a2c0 con 0x7efcd80047c0 2023-10-26T01:36:21.600 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:21.597+0000 7efce1ffb640 1 -- 172.21.15.169:0/2918623885 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=89}) v3 -- 0x7efcd00729c0 con 0x7efcd80047c0 2023-10-26T01:36:21.601 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:21.598+0000 7efcf0ef9640 1 --2- 172.21.15.169:0/2918623885 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7efcd006ae90 0x7efcd006d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:21.602 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:21.599+0000 7efcf0ef9640 1 --2- 172.21.15.169:0/2918623885 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7efcd006ae90 0x7efcd006d350 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7efcdc034000 tx=0x7efcdc030040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:21.606 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:21.603+0000 7efce1ffb640 1 -- 172.21.15.169:0/2918623885 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7efcec1549b0 con 0x7efcd80047c0 2023-10-26T01:36:21.841 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:21.838+0000 7efcf26fc640 1 -- 172.21.15.169:0/2918623885 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efcec156ac0 con 0x7efcd80047c0 2023-10-26T01:36:21.844 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:21.841+0000 7efce1ffb640 1 -- 172.21.15.169:0/2918623885 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+16659 (secure 0 0 0) 0x7efcec156ac0 con 0x7efcd80047c0 2023-10-26T01:36:21.844 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:36:21.845 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":88,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:35:54.125183+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-10-26T01:35:26.320950+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-10-26T01:35:26.855916+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","full","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","full","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","full","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","full","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:36:21.847 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:21.844+0000 7efcf26fc640 1 -- 172.21.15.169:0/2918623885 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7efcd006ae90 msgr2=0x7efcd006d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:21.847 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:21.844+0000 7efcf26fc640 1 --2- 172.21.15.169:0/2918623885 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7efcd006ae90 0x7efcd006d350 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7efcdc034000 tx=0x7efcdc030040 comp rx=0 tx=0).stop 2023-10-26T01:36:21.847 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:21.844+0000 7efcf26fc640 1 -- 172.21.15.169:0/2918623885 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efcd80047c0 msgr2=0x7efcec1445b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:21.848 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:21.844+0000 7efcf26fc640 1 --2- 172.21.15.169:0/2918623885 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efcd80047c0 0x7efcec1445b0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7efce8011a40 tx=0x7efce8011f00 comp rx=0 tx=0).stop 2023-10-26T01:36:21.848 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:21.844+0000 7efcf26fc640 1 -- 172.21.15.169:0/2918623885 shutdown_connections 2023-10-26T01:36:21.848 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:21.844+0000 7efcf26fc640 1 --2- 172.21.15.169:0/2918623885 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7efcd006ae90 0x7efcd006d350 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:21.848 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:21.844+0000 7efcf26fc640 1 --2- 172.21.15.169:0/2918623885 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efcec140ac0 0x7efcec140f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:21.848 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:21.844+0000 7efcf26fc640 1 --2- 172.21.15.169:0/2918623885 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7efcec070e10 0x7efcec140580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:21.848 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:21.844+0000 7efcf26fc640 1 --2- 172.21.15.169:0/2918623885 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efcd80047c0 0x7efcec1445b0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:21.849 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:21.844+0000 7efcf26fc640 1 -- 172.21.15.169:0/2918623885 >> 172.21.15.169:0/2918623885 conn(0x7efcec0aedd0 msgr2=0x7efcec0c9a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:21.849 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:21.844+0000 7efcf26fc640 1 -- 172.21.15.169:0/2918623885 shutdown_connections 2023-10-26T01:36:21.849 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:21.844+0000 7efcf26fc640 1 -- 172.21.15.169:0/2918623885 wait complete. 2023-10-26T01:36:21.865 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T01:30:32.200808+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-26T01:35:26.320950+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2023-10-26T01:35:26.855916+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T01:36:21.866 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T01:36:22.082 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.079+0000 7f6715078640 1 -- 172.21.15.169:0/103622226 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66fc004d00 msgr2=0x7f66fc005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:22.082 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.079+0000 7f6715078640 1 --2- 172.21.15.169:0/103622226 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66fc004d00 0x7f66fc005180 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f6708006f10 tx=0x7f66fc0057f0 comp rx=0 tx=0).stop 2023-10-26T01:36:22.083 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.079+0000 7f6714877640 1 -- 172.21.15.169:0/103622226 reap_dead start 2023-10-26T01:36:22.083 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.079+0000 7f6715078640 1 -- 172.21.15.169:0/103622226 shutdown_connections 2023-10-26T01:36:22.083 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.079+0000 7f6715078640 1 -- 172.21.15.169:0/103622226 >> 172.21.15.169:0/103622226 conn(0x7f67100b1050 msgr2=0x7f67100b1430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:22.083 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.079+0000 7f6715078640 1 -- 172.21.15.169:0/103622226 shutdown_connections 2023-10-26T01:36:22.083 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.079+0000 7f6715078640 1 -- 172.21.15.169:0/103622226 wait complete. 2023-10-26T01:36:22.084 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.081+0000 7f6715078640 1 Processor -- start 2023-10-26T01:36:22.084 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.081+0000 7f6715078640 1 -- start start 2023-10-26T01:36:22.085 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.082+0000 7f6715078640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f67100cdda0 0x7f67100ad290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:22.085 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.082+0000 7f6715078640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f67100ad7d0 0x7f67100c43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:22.085 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.082+0000 7f6715078640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67100c4920 0x7f6710146af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:22.086 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.082+0000 7f6715078640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67100c4da0 con 0x7f67100ad7d0 2023-10-26T01:36:22.086 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.082+0000 7f6715078640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6710144f60 con 0x7f67100c4920 2023-10-26T01:36:22.086 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.082+0000 7f6715078640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6710144c10 con 0x7f67100cdda0 2023-10-26T01:36:22.086 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.082+0000 7f670f7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f67100ad7d0 0x7f67100c43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:22.087 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.082+0000 7f670f7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f67100ad7d0 0x7f67100c43e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:50332/0 (socket says 172.21.15.169:50332) 2023-10-26T01:36:22.087 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.083+0000 7f670f7fe640 1 -- 172.21.15.169:0/923545114 learned_addr learned my addr 172.21.15.169:0/923545114 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:36:22.087 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.083+0000 7f670ffff640 1 --2- 172.21.15.169:0/923545114 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f67100cdda0 0x7f67100ad290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:22.089 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.083+0000 7f670f7fe640 1 -- 172.21.15.169:0/923545114 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f67100cdda0 msgr2=0x7f67100ad290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:22.090 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.083+0000 7f670f7fe640 1 --2- 172.21.15.169:0/923545114 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f67100cdda0 0x7f67100ad290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:22.090 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.083+0000 7f670f7fe640 1 -- 172.21.15.169:0/923545114 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67100c4920 msgr2=0x7f6710146af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:22.091 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.083+0000 7f6714877640 1 --2- 172.21.15.169:0/923545114 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67100c4920 0x7f6710146af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:22.091 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.083+0000 7f670f7fe640 1 --2- 172.21.15.169:0/923545114 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67100c4920 0x7f6710146af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:22.091 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.083+0000 7f670f7fe640 1 -- 172.21.15.169:0/923545114 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67100add30 con 0x7f67100ad7d0 2023-10-26T01:36:22.091 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.083+0000 7f670f7fe640 1 --2- 172.21.15.169:0/923545114 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f67100ad7d0 0x7f67100c43e0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f66f800ec20 tx=0x7f66f800c640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:22.092 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.084+0000 7f670cff9640 1 -- 172.21.15.169:0/923545114 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f8011670 con 0x7f67100ad7d0 2023-10-26T01:36:22.092 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.084+0000 7f670cff9640 1 -- 172.21.15.169:0/923545114 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f66f80118c0 con 0x7f67100ad7d0 2023-10-26T01:36:22.092 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.084+0000 7f670cff9640 1 -- 172.21.15.169:0/923545114 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f8011cd0 con 0x7f67100ad7d0 2023-10-26T01:36:22.093 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.084+0000 7f6715078640 1 -- 172.21.15.169:0/923545114 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67100ae010 con 0x7f67100ad7d0 2023-10-26T01:36:22.093 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.084+0000 7f6715078640 1 -- 172.21.15.169:0/923545114 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f671014a770 con 0x7f67100ad7d0 2023-10-26T01:36:22.093 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.085+0000 7f6715078640 1 -- 172.21.15.169:0/923545114 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6710066af0 con 0x7f67100ad7d0 2023-10-26T01:36:22.093 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.086+0000 7f670cff9640 1 -- 172.21.15.169:0/923545114 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f66f801a400 con 0x7f67100ad7d0 2023-10-26T01:36:22.094 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.087+0000 7f670cff9640 1 --2- 172.21.15.169:0/923545114 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f66f006ae70 0x7f66f006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:22.094 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.087+0000 7f670cff9640 1 -- 172.21.15.169:0/923545114 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f66f808a690 con 0x7f67100ad7d0 2023-10-26T01:36:22.094 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.087+0000 7f670cff9640 1 -- 172.21.15.169:0/923545114 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=89}) v3 -- 0x7f66f00729a0 con 0x7f67100ad7d0 2023-10-26T01:36:22.095 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.087+0000 7f670ffff640 1 --2- 172.21.15.169:0/923545114 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f66f006ae70 0x7f66f006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:22.095 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.088+0000 7f670ffff640 1 --2- 172.21.15.169:0/923545114 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f66f006ae70 0x7f66f006d330 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f67000058c0 tx=0x7f6700005850 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:22.095 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.091+0000 7f670cff9640 1 -- 172.21.15.169:0/923545114 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f66f801f060 con 0x7f67100ad7d0 2023-10-26T01:36:22.284 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.281+0000 7f6715078640 1 -- 172.21.15.169:0/923545114 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f67100ae250 con 0x7f67100ad7d0 2023-10-26T01:36:22.284 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.282+0000 7f670cff9640 1 -- 172.21.15.169:0/923545114 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7f66f801f070 con 0x7f67100ad7d0 2023-10-26T01:36:22.285 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 1 2023-10-26T01:36:22.287 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.284+0000 7f6715078640 1 -- 172.21.15.169:0/923545114 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f66f006ae70 msgr2=0x7f66f006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:22.287 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.284+0000 7f6715078640 1 --2- 172.21.15.169:0/923545114 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f66f006ae70 0x7f66f006d330 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f67000058c0 tx=0x7f6700005850 comp rx=0 tx=0).stop 2023-10-26T01:36:22.287 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.284+0000 7f6715078640 1 -- 172.21.15.169:0/923545114 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f67100ad7d0 msgr2=0x7f67100c43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:22.288 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.284+0000 7f6715078640 1 --2- 172.21.15.169:0/923545114 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f67100ad7d0 0x7f67100c43e0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f66f800ec20 tx=0x7f66f800c640 comp rx=0 tx=0).stop 2023-10-26T01:36:22.288 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.284+0000 7f6715078640 1 -- 172.21.15.169:0/923545114 shutdown_connections 2023-10-26T01:36:22.288 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.284+0000 7f6715078640 1 --2- 172.21.15.169:0/923545114 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f66f006ae70 0x7f66f006d330 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:22.288 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.284+0000 7f6715078640 1 --2- 172.21.15.169:0/923545114 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f67100ad7d0 0x7f67100c43e0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:22.288 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.284+0000 7f6715078640 1 --2- 172.21.15.169:0/923545114 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67100c4920 0x7f6710146af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:22.288 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.284+0000 7f6715078640 1 --2- 172.21.15.169:0/923545114 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f67100cdda0 0x7f67100ad290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:22.289 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.284+0000 7f6715078640 1 -- 172.21.15.169:0/923545114 >> 172.21.15.169:0/923545114 conn(0x7f67100b1050 msgr2=0x7f6710155f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:22.289 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.284+0000 7f6715078640 1 -- 172.21.15.169:0/923545114 shutdown_connections 2023-10-26T01:36:22.289 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.285+0000 7f6715078640 1 -- 172.21.15.169:0/923545114 wait complete. 2023-10-26T01:36:22.304 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-26T01:36:22.529 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.527+0000 7f5edc765640 1 -- 172.21.15.169:0/3063463380 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5ec00047c0 msgr2=0x7f5ec0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:22.530 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.527+0000 7f5edc765640 1 --2- 172.21.15.169:0/3063463380 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5ec00047c0 0x7f5ec0004c40 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f5ec400c260 tx=0x7f5ec00057f0 comp rx=0 tx=0).stop 2023-10-26T01:36:22.530 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.527+0000 7f5edad02640 1 -- 172.21.15.169:0/3063463380 reap_dead start 2023-10-26T01:36:22.530 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.527+0000 7f5edc765640 1 -- 172.21.15.169:0/3063463380 shutdown_connections 2023-10-26T01:36:22.531 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.527+0000 7f5edc765640 1 -- 172.21.15.169:0/3063463380 >> 172.21.15.169:0/3063463380 conn(0x7f5ed4069930 msgr2=0x7f5ed4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:22.531 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.527+0000 7f5edc765640 1 -- 172.21.15.169:0/3063463380 shutdown_connections 2023-10-26T01:36:22.531 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.527+0000 7f5edc765640 1 -- 172.21.15.169:0/3063463380 wait complete. 2023-10-26T01:36:22.531 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.528+0000 7f5edc765640 1 Processor -- start 2023-10-26T01:36:22.532 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.529+0000 7f5edc765640 1 -- start start 2023-10-26T01:36:22.533 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.529+0000 7f5edc765640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5ec00047c0 0x7f5ed40c8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:22.534 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.530+0000 7f5edc765640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5ed406cad0 0x7f5ed415a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:22.534 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.530+0000 7f5edc765640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ed40c6c50 0x7f5ed40c70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:22.534 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.530+0000 7f5edc765640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ed40c9110 con 0x7f5ec00047c0 2023-10-26T01:36:22.534 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.530+0000 7f5edc765640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ed41f5150 con 0x7f5ed40c6c50 2023-10-26T01:36:22.534 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.530+0000 7f5edc765640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ed41f53f0 con 0x7f5ed406cad0 2023-10-26T01:36:22.535 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.530+0000 7f5eda501640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5ec00047c0 0x7f5ed40c8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:22.535 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.530+0000 7f5eda501640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5ec00047c0 0x7f5ed40c8bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:50342/0 (socket says 172.21.15.169:50342) 2023-10-26T01:36:22.535 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.530+0000 7f5eda501640 1 -- 172.21.15.169:0/4221796016 learned_addr learned my addr 172.21.15.169:0/4221796016 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:36:22.535 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.530+0000 7f5ed9d00640 1 --2- 172.21.15.169:0/4221796016 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5ed406cad0 0x7f5ed415a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:22.536 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.530+0000 7f5eda501640 1 -- 172.21.15.169:0/4221796016 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5ed406cad0 msgr2=0x7f5ed415a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:22.536 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.530+0000 7f5eda501640 1 --2- 172.21.15.169:0/4221796016 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5ed406cad0 0x7f5ed415a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:22.536 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.530+0000 7f5eda501640 1 -- 172.21.15.169:0/4221796016 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ed40c6c50 msgr2=0x7f5ed40c70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:22.536 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.530+0000 7f5eda501640 1 --2- 172.21.15.169:0/4221796016 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ed40c6c50 0x7f5ed40c70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:22.536 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.530+0000 7f5eda501640 1 -- 172.21.15.169:0/4221796016 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ed41f56d0 con 0x7f5ec00047c0 2023-10-26T01:36:22.536 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.531+0000 7f5eda501640 1 --2- 172.21.15.169:0/4221796016 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5ec00047c0 0x7f5ed40c8bd0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f5ed000e990 tx=0x7f5ed000ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:22.537 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.531+0000 7f5ecaffd640 1 -- 172.21.15.169:0/4221796016 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ed000cbe0 con 0x7f5ec00047c0 2023-10-26T01:36:22.537 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.531+0000 7f5ecaffd640 1 -- 172.21.15.169:0/4221796016 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5ed000ce30 con 0x7f5ec00047c0 2023-10-26T01:36:22.537 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.531+0000 7f5edc765640 1 -- 172.21.15.169:0/4221796016 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ed41f59b0 con 0x7f5ec00047c0 2023-10-26T01:36:22.537 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.531+0000 7f5ecaffd640 1 -- 172.21.15.169:0/4221796016 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ed0005280 con 0x7f5ec00047c0 2023-10-26T01:36:22.537 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.531+0000 7f5edc765640 1 -- 172.21.15.169:0/4221796016 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ed41f5f70 con 0x7f5ec00047c0 2023-10-26T01:36:22.538 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.532+0000 7f5edc765640 1 -- 172.21.15.169:0/4221796016 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ed404db10 con 0x7f5ec00047c0 2023-10-26T01:36:22.539 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.533+0000 7f5ecaffd640 1 -- 172.21.15.169:0/4221796016 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f5ed00053e0 con 0x7f5ec00047c0 2023-10-26T01:36:22.539 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.535+0000 7f5ecaffd640 1 --2- 172.21.15.169:0/4221796016 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5eb806ae40 0x7f5eb806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:22.539 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.535+0000 7f5ecaffd640 1 -- 172.21.15.169:0/4221796016 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5ed0089f50 con 0x7f5ec00047c0 2023-10-26T01:36:22.539 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.535+0000 7f5ecaffd640 1 -- 172.21.15.169:0/4221796016 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=89}) v3 -- 0x7f5eb8072970 con 0x7f5ec00047c0 2023-10-26T01:36:22.539 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.536+0000 7f5ed9d00640 1 --2- 172.21.15.169:0/4221796016 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5eb806ae40 0x7f5eb806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:22.540 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.537+0000 7f5ed9d00640 1 --2- 172.21.15.169:0/4221796016 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5eb806ae40 0x7f5eb806d300 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f5ec400c260 tx=0x7f5ec4001a00 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:22.542 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.540+0000 7f5ecaffd640 1 -- 172.21.15.169:0/4221796016 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5ed001f070 con 0x7f5ec00047c0 2023-10-26T01:36:22.734 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.731+0000 7f5edc765640 1 -- 172.21.15.169:0/4221796016 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5ed40b2170 con 0x7f5ec00047c0 2023-10-26T01:36:22.735 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.732+0000 7f5ecaffd640 1 -- 172.21.15.169:0/4221796016 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5ed0058a80 con 0x7f5ec00047c0 2023-10-26T01:36:22.735 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 32 2023-10-26T01:36:22.738 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.734+0000 7f5edc765640 1 -- 172.21.15.169:0/4221796016 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5eb806ae40 msgr2=0x7f5eb806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:22.738 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.734+0000 7f5edc765640 1 --2- 172.21.15.169:0/4221796016 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5eb806ae40 0x7f5eb806d300 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f5ec400c260 tx=0x7f5ec4001a00 comp rx=0 tx=0).stop 2023-10-26T01:36:22.738 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.734+0000 7f5edc765640 1 -- 172.21.15.169:0/4221796016 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5ec00047c0 msgr2=0x7f5ed40c8bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:22.738 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.734+0000 7f5edc765640 1 --2- 172.21.15.169:0/4221796016 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5ec00047c0 0x7f5ed40c8bd0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f5ed000e990 tx=0x7f5ed000ee50 comp rx=0 tx=0).stop 2023-10-26T01:36:22.739 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.735+0000 7f5edc765640 1 -- 172.21.15.169:0/4221796016 shutdown_connections 2023-10-26T01:36:22.739 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.735+0000 7f5edc765640 1 --2- 172.21.15.169:0/4221796016 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5eb806ae40 0x7f5eb806d300 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:22.739 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.735+0000 7f5edc765640 1 --2- 172.21.15.169:0/4221796016 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ed40c6c50 0x7f5ed40c70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:22.739 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.735+0000 7f5edc765640 1 --2- 172.21.15.169:0/4221796016 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5ed406cad0 0x7f5ed415a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:22.739 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.735+0000 7f5edc765640 1 --2- 172.21.15.169:0/4221796016 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5ec00047c0 0x7f5ed40c8bd0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:22.739 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.735+0000 7f5edc765640 1 -- 172.21.15.169:0/4221796016 >> 172.21.15.169:0/4221796016 conn(0x7f5ed4069930 msgr2=0x7f5ed40ab5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:22.740 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.735+0000 7f5edc765640 1 -- 172.21.15.169:0/4221796016 shutdown_connections 2023-10-26T01:36:22.740 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.735+0000 7f5edc765640 1 -- 172.21.15.169:0/4221796016 wait complete. 2023-10-26T01:36:22.757 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-26T01:36:22.983 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.979+0000 7fbdd72b4640 1 -- 172.21.15.169:0/2079226654 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbdd0159440 msgr2=0x7fbdd00cc990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:22.983 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.979+0000 7fbdd72b4640 1 --2- 172.21.15.169:0/2079226654 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbdd0159440 0x7fbdd00cc990 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fbdd006e270 tx=0x7fbdcc02bdb0 comp rx=0 tx=0).stop 2023-10-26T01:36:22.983 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.980+0000 7fbdd72b4640 1 -- 172.21.15.169:0/2079226654 shutdown_connections 2023-10-26T01:36:22.983 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.980+0000 7fbdd72b4640 1 --2- 172.21.15.169:0/2079226654 >> v2:172.21.15.170:3300/0 conn(0x7fbdd0159820 0x7fbdd00ac200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:22.984 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.980+0000 7fbdd72b4640 1 --2- 172.21.15.169:0/2079226654 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbdd0159440 0x7fbdd00cc990 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:22.984 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.980+0000 7fbdd72b4640 1 -- 172.21.15.169:0/2079226654 >> 172.21.15.169:0/2079226654 conn(0x7fbdd00aefe0 msgr2=0x7fbdd00af3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:22.984 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.980+0000 7fbdd72b4640 1 -- 172.21.15.169:0/2079226654 shutdown_connections 2023-10-26T01:36:22.984 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.980+0000 7fbdd72b4640 1 -- 172.21.15.169:0/2079226654 wait complete. 2023-10-26T01:36:22.986 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.981+0000 7fbdd72b4640 1 Processor -- start 2023-10-26T01:36:22.986 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.982+0000 7fbdd72b4640 1 -- start start 2023-10-26T01:36:22.986 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.982+0000 7fbdd72b4640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbdd00abdb0 0x7fbdd0159c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:22.986 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.982+0000 7fbdd72b4640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbdd0159440 0x7fbdd014d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:22.986 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.982+0000 7fbdd72b4640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbdd0158180 0x7fbdd0155900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:22.987 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.982+0000 7fbdd72b4640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd015b560 con 0x7fbdd0158180 2023-10-26T01:36:22.987 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.982+0000 7fbdd72b4640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd0151a30 con 0x7fbdd0159440 2023-10-26T01:36:22.987 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.982+0000 7fbdd72b4640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd0151d10 con 0x7fbdd00abdb0 2023-10-26T01:36:22.987 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.983+0000 7fbdd5851640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbdd0158180 0x7fbdd0155900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:22.987 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.983+0000 7fbdd5851640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbdd0158180 0x7fbdd0155900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:50362/0 (socket says 172.21.15.169:50362) 2023-10-26T01:36:22.988 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.983+0000 7fbdd5851640 1 -- 172.21.15.169:0/1026692691 learned_addr learned my addr 172.21.15.169:0/1026692691 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:36:22.988 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.983+0000 7fbdd5050640 1 --2- 172.21.15.169:0/1026692691 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbdd00abdb0 0x7fbdd0159c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:22.988 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.983+0000 7fbdd5851640 1 -- 172.21.15.169:0/1026692691 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbdd00abdb0 msgr2=0x7fbdd0159c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:22.988 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.983+0000 7fbdd5851640 1 --2- 172.21.15.169:0/1026692691 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbdd00abdb0 0x7fbdd0159c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:22.988 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.983+0000 7fbdd5851640 1 -- 172.21.15.169:0/1026692691 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbdd0159440 msgr2=0x7fbdd014d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:22.988 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.983+0000 7fbdd484f640 1 --2- 172.21.15.169:0/1026692691 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbdd0159440 0x7fbdd014d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:22.989 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.983+0000 7fbdd5851640 1 --2- 172.21.15.169:0/1026692691 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbdd0159440 0x7fbdd014d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:22.989 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.983+0000 7fbdd5851640 1 -- 172.21.15.169:0/1026692691 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdcc009560 con 0x7fbdd0158180 2023-10-26T01:36:22.990 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.984+0000 7fbdd5851640 1 --2- 172.21.15.169:0/1026692691 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbdd0158180 0x7fbdd0155900 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fbdc800c730 tx=0x7fbdc800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:22.990 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.984+0000 7fbdc5ffb640 1 -- 172.21.15.169:0/1026692691 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdc8003b70 con 0x7fbdd0158180 2023-10-26T01:36:22.990 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.984+0000 7fbdc5ffb640 1 -- 172.21.15.169:0/1026692691 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fbdc8003cd0 con 0x7fbdd0158180 2023-10-26T01:36:22.991 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.984+0000 7fbdc5ffb640 1 -- 172.21.15.169:0/1026692691 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdc800aab0 con 0x7fbdd0158180 2023-10-26T01:36:22.991 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.984+0000 7fbdd72b4640 1 -- 172.21.15.169:0/1026692691 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdd0151ff0 con 0x7fbdd0158180 2023-10-26T01:36:22.991 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.984+0000 7fbdd72b4640 1 -- 172.21.15.169:0/1026692691 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdd01525b0 con 0x7fbdd0158180 2023-10-26T01:36:22.991 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.985+0000 7fbdd72b4640 1 -- 172.21.15.169:0/1026692691 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdd0066af0 con 0x7fbdd0158180 2023-10-26T01:36:22.991 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.986+0000 7fbdc5ffb640 1 -- 172.21.15.169:0/1026692691 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fbdc800ac10 con 0x7fbdd0158180 2023-10-26T01:36:22.992 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.987+0000 7fbdc5ffb640 1 --2- 172.21.15.169:0/1026692691 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbdb406ae40 0x7fbdb406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:22.992 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.987+0000 7fbdc5ffb640 1 -- 172.21.15.169:0/1026692691 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbdc8089660 con 0x7fbdd0158180 2023-10-26T01:36:22.992 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.987+0000 7fbdc5ffb640 1 -- 172.21.15.169:0/1026692691 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=89}) v3 -- 0x7fbdb4072970 con 0x7fbdd0158180 2023-10-26T01:36:22.992 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.988+0000 7fbdd5050640 1 --2- 172.21.15.169:0/1026692691 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbdb406ae40 0x7fbdb406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:22.992 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.989+0000 7fbdd5050640 1 --2- 172.21.15.169:0/1026692691 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbdb406ae40 0x7fbdb406d300 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc035000 tx=0x7fbdcc004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:22.994 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:22.991+0000 7fbdc5ffb640 1 -- 172.21.15.169:0/1026692691 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbdc8054670 con 0x7fbdd0158180 2023-10-26T01:36:23.185 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:23.183+0000 7fbdd72b4640 1 -- 172.21.15.169:0/1026692691 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbdd004da70 con 0x7fbdd0158180 2023-10-26T01:36:23.186 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:23.184+0000 7fbdc5ffb640 1 -- 172.21.15.169:0/1026692691 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7fbdc8057c90 con 0x7fbdd0158180 2023-10-26T01:36:23.187 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 64 2023-10-26T01:36:23.189 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:23.186+0000 7fbdd72b4640 1 -- 172.21.15.169:0/1026692691 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbdb406ae40 msgr2=0x7fbdb406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:23.189 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:23.186+0000 7fbdd72b4640 1 --2- 172.21.15.169:0/1026692691 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbdb406ae40 0x7fbdb406d300 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc035000 tx=0x7fbdcc004730 comp rx=0 tx=0).stop 2023-10-26T01:36:23.189 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:23.186+0000 7fbdd72b4640 1 -- 172.21.15.169:0/1026692691 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbdd0158180 msgr2=0x7fbdd0155900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:23.189 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:23.186+0000 7fbdd72b4640 1 --2- 172.21.15.169:0/1026692691 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbdd0158180 0x7fbdd0155900 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fbdc800c730 tx=0x7fbdc800cbf0 comp rx=0 tx=0).stop 2023-10-26T01:36:23.190 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:23.186+0000 7fbdd72b4640 1 -- 172.21.15.169:0/1026692691 shutdown_connections 2023-10-26T01:36:23.190 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:23.186+0000 7fbdd72b4640 1 --2- 172.21.15.169:0/1026692691 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbdb406ae40 0x7fbdb406d300 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:23.190 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:23.186+0000 7fbdd72b4640 1 --2- 172.21.15.169:0/1026692691 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbdd0158180 0x7fbdd0155900 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:23.190 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:23.186+0000 7fbdd72b4640 1 --2- 172.21.15.169:0/1026692691 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbdd0159440 0x7fbdd014d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:23.190 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:23.186+0000 7fbdd72b4640 1 --2- 172.21.15.169:0/1026692691 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbdd00abdb0 0x7fbdd0159c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:23.191 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:23.186+0000 7fbdd72b4640 1 -- 172.21.15.169:0/1026692691 >> 172.21.15.169:0/1026692691 conn(0x7fbdd00aefe0 msgr2=0x7fbdd00acaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:23.191 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:23.186+0000 7fbdd72b4640 1 -- 172.21.15.169:0/1026692691 shutdown_connections 2023-10-26T01:36:23.191 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:23.186+0000 7fbdd72b4640 1 -- 172.21.15.169:0/1026692691 wait complete. 2023-10-26T01:36:23.208 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-26T01:36:23.423 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:23.420+0000 7f864910f640 1 -- 172.21.15.169:0/1489982808 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8630004ab0 msgr2=0x7f8630004ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:23.424 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:23.420+0000 7f864910f640 1 --2- 172.21.15.169:0/1489982808 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8630004ab0 0x7f8630004ed0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f8634006fd0 tx=0x7f8630006900 comp rx=0 tx=0).stop 2023-10-26T01:36:23.424 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:23.420+0000 7f864359e640 1 -- 172.21.15.169:0/1489982808 reap_dead start 2023-10-26T01:36:23.424 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:23.420+0000 7f864910f640 1 -- 172.21.15.169:0/1489982808 shutdown_connections 2023-10-26T01:36:23.424 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:23.420+0000 7f864910f640 1 -- 172.21.15.169:0/1489982808 >> 172.21.15.169:0/1489982808 conn(0x7f8644069930 msgr2=0x7f8644069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:23.424 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:23.421+0000 7f864910f640 1 -- 172.21.15.169:0/1489982808 shutdown_connections 2023-10-26T01:36:23.425 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:23.421+0000 7f864910f640 1 -- 172.21.15.169:0/1489982808 wait complete. 2023-10-26T01:36:23.425 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:23.422+0000 7f864910f640 1 Processor -- start 2023-10-26T01:36:23.425 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:23.422+0000 7f864910f640 1 -- start start 2023-10-26T01:36:23.427 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:23.423+0000 7f864910f640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8630004ab0 0x7f86440cc300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:23.427 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:23.423+0000 7f864910f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f86440cadd0 0x7f86440af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:23.427 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:23.423+0000 7f864910f640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f864414e330 0x7f86440b0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:23.427 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:23.423+0000 7f864910f640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86440cc840 con 0x7f8630004ab0 2023-10-26T01:36:23.428 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:23.423+0000 7f864910f640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86440afe90 con 0x7f86440cadd0 2023-10-26T01:36:23.428 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:23.423+0000 7f864910f640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86440cb260 con 0x7f864414e330 2023-10-26T01:36:23.428 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:23.424+0000 7f8642d9d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8630004ab0 0x7f86440cc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:23.428 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:23.424+0000 7f8642d9d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8630004ab0 0x7f86440cc300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:50374/0 (socket says 172.21.15.169:50374) 2023-10-26T01:36:23.428 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:23.424+0000 7f8642d9d640 1 -- 172.21.15.169:0/2416833560 learned_addr learned my addr 172.21.15.169:0/2416833560 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:36:23.429 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:23.424+0000 7f863bfff640 1 --2- 172.21.15.169:0/2416833560 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f86440cadd0 0x7f86440af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:23.429 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:23.424+0000 7f864359e640 1 --2- 172.21.15.169:0/2416833560 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f864414e330 0x7f86440b0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:23.429 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:23.424+0000 7f8642d9d640 1 -- 172.21.15.169:0/2416833560 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f864414e330 msgr2=0x7f86440b0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:23.429 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:23.424+0000 7f8642d9d640 1 --2- 172.21.15.169:0/2416833560 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f864414e330 0x7f86440b0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:23.429 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:23.424+0000 7f8642d9d640 1 -- 172.21.15.169:0/2416833560 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f86440cadd0 msgr2=0x7f86440af950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:23.430 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:23.424+0000 7f8642d9d640 1 --2- 172.21.15.169:0/2416833560 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f86440cadd0 0x7f86440af950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:23.430 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:23.424+0000 7f8642d9d640 1 -- 172.21.15.169:0/2416833560 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86440cb540 con 0x7f8630004ab0 2023-10-26T01:36:23.430 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:23.424+0000 7f864359e640 1 --2- 172.21.15.169:0/2416833560 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f864414e330 0x7f86440b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:36:23.430 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:23.424+0000 7f8642d9d640 1 --2- 172.21.15.169:0/2416833560 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8630004ab0 0x7f86440cc300 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f8634001930 tx=0x7f8634000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:23.432 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:23.425+0000 7f863b7fe640 1 -- 172.21.15.169:0/2416833560 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8634006050 con 0x7f8630004ab0 2023-10-26T01:36:23.432 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:23.425+0000 7f863b7fe640 1 -- 172.21.15.169:0/2416833560 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f86340061b0 con 0x7f8630004ab0 2023-10-26T01:36:23.432 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:23.425+0000 7f863b7fe640 1 -- 172.21.15.169:0/2416833560 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86340065a0 con 0x7f8630004ab0 2023-10-26T01:36:23.432 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:23.425+0000 7f864910f640 1 -- 172.21.15.169:0/2416833560 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86440cb820 con 0x7f8630004ab0 2023-10-26T01:36:23.432 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:23.425+0000 7f864910f640 1 -- 172.21.15.169:0/2416833560 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86440cbde0 con 0x7f8630004ab0 2023-10-26T01:36:23.433 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:23.426+0000 7f864910f640 1 -- 172.21.15.169:0/2416833560 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86440cbf70 con 0x7f8630004ab0 2023-10-26T01:36:23.433 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:23.427+0000 7f863b7fe640 1 -- 172.21.15.169:0/2416833560 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f8634033030 con 0x7f8630004ab0 2023-10-26T01:36:23.433 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:23.428+0000 7f863b7fe640 1 --2- 172.21.15.169:0/2416833560 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f862406ae70 0x7f862406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:23.433 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:23.428+0000 7f863b7fe640 1 -- 172.21.15.169:0/2416833560 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f86340aabb0 con 0x7f8630004ab0 2023-10-26T01:36:23.433 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:23.428+0000 7f863b7fe640 1 -- 172.21.15.169:0/2416833560 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=89}) v3 -- 0x7f86240729a0 con 0x7f8630004ab0 2023-10-26T01:36:23.434 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:23.429+0000 7f863bfff640 1 --2- 172.21.15.169:0/2416833560 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f862406ae70 0x7f862406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:23.434 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:23.430+0000 7f863bfff640 1 --2- 172.21.15.169:0/2416833560 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f862406ae70 0x7f862406d330 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f86440aa920 tx=0x7f862c009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:23.435 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:23.433+0000 7f863b7fe640 1 -- 172.21.15.169:0/2416833560 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8634075cf0 con 0x7f8630004ab0 2023-10-26T01:36:23.638 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:23.635+0000 7f864910f640 1 -- 172.21.15.169:0/2416833560 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f864404db10 con 0x7f8630004ab0 2023-10-26T01:36:23.639 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:23.636+0000 7f863b7fe640 1 -- 172.21.15.169:0/2416833560 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v88) v1 ==== 80+0+0 (secure 0 0 0) 0x7f8634079310 con 0x7f8630004ab0 2023-10-26T01:36:23.639 INFO:teuthology.orchestra.run.smithi169.stderr:listed 0 entries 2023-10-26T01:36:23.641 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:23.639+0000 7f864910f640 1 -- 172.21.15.169:0/2416833560 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f862406ae70 msgr2=0x7f862406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:23.642 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:23.639+0000 7f864910f640 1 --2- 172.21.15.169:0/2416833560 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f862406ae70 0x7f862406d330 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f86440aa920 tx=0x7f862c009040 comp rx=0 tx=0).stop 2023-10-26T01:36:23.642 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:23.639+0000 7f864910f640 1 -- 172.21.15.169:0/2416833560 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8630004ab0 msgr2=0x7f86440cc300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:23.642 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:23.639+0000 7f864910f640 1 --2- 172.21.15.169:0/2416833560 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8630004ab0 0x7f86440cc300 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f8634001930 tx=0x7f8634000fe0 comp rx=0 tx=0).stop 2023-10-26T01:36:23.642 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:23.639+0000 7f864910f640 1 -- 172.21.15.169:0/2416833560 shutdown_connections 2023-10-26T01:36:23.643 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:23.639+0000 7f864910f640 1 --2- 172.21.15.169:0/2416833560 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f862406ae70 0x7f862406d330 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:23.643 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:23.639+0000 7f864910f640 1 --2- 172.21.15.169:0/2416833560 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f864414e330 0x7f86440b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:23.643 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:23.639+0000 7f864910f640 1 --2- 172.21.15.169:0/2416833560 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f86440cadd0 0x7f86440af950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:23.643 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:23.639+0000 7f864910f640 1 --2- 172.21.15.169:0/2416833560 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8630004ab0 0x7f86440cc300 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:23.643 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:23.639+0000 7f864910f640 1 -- 172.21.15.169:0/2416833560 >> 172.21.15.169:0/2416833560 conn(0x7f8644069930 msgr2=0x7f86441595d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:23.644 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:23.639+0000 7f864910f640 1 -- 172.21.15.169:0/2416833560 shutdown_connections 2023-10-26T01:36:23.644 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:23.639+0000 7f864910f640 1 -- 172.21.15.169:0/2416833560 wait complete. 2023-10-26T01:36:23.661 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-26T01:36:23.661 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:36:23.661 DEBUG:teuthology.orchestra.run.smithi170:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-26T01:36:23.777 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi170.front.sepia.ceph.com 2023-10-26T01:36:23.778 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:36:23.778 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-26T01:36:23.835 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:36:23.835 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:36:23.895 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054301,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-26T01:36:23.895 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-26T01:36:23.895 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:36:23.955 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054303,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-26T01:36:23.956 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T01:36:24.078 INFO:tasks.ceph.mon.a.smithi169.stderr:2023-10-26T01:36:24.075+0000 7f158b279640 -1 log_channel(cluster) log [ERR] : Health check update: 2 full osd(s) (OSD_FULL) 2023-10-26T01:36:24.183 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.180+0000 7f36c8797640 1 -- 172.21.15.169:0/3201264511 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f36c00ac930 msgr2=0x7f36c00cd6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:24.183 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.180+0000 7f36c8797640 1 --2- 172.21.15.169:0/3201264511 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f36c00ac930 0x7f36c00cd6f0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f36b0009830 tx=0x7f36b002bcd0 comp rx=0 tx=0).stop 2023-10-26T01:36:24.183 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.180+0000 7f36c8797640 1 -- 172.21.15.169:0/3201264511 shutdown_connections 2023-10-26T01:36:24.184 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.180+0000 7f36c8797640 1 --2- 172.21.15.169:0/3201264511 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f36c00ac930 0x7f36c00cd6f0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:24.184 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.180+0000 7f36c8797640 1 -- 172.21.15.169:0/3201264511 >> 172.21.15.169:0/3201264511 conn(0x7f36c0069930 msgr2=0x7f36c0069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:24.184 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.180+0000 7f36c8797640 1 -- 172.21.15.169:0/3201264511 shutdown_connections 2023-10-26T01:36:24.184 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.180+0000 7f36c8797640 1 -- 172.21.15.169:0/3201264511 wait complete. 2023-10-26T01:36:24.185 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.182+0000 7f36c8797640 1 Processor -- start 2023-10-26T01:36:24.185 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.182+0000 7f36c8797640 1 -- start start 2023-10-26T01:36:24.186 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.183+0000 7f36c8797640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f36c00ac930 0x7f36c0149960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:24.186 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.183+0000 7f36c8797640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f36c00cdc30 0x7f36c00c8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:24.186 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.183+0000 7f36c8797640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f36c01592d0 0x7f36c00c9060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:24.186 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.183+0000 7f36c8797640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36c00c95a0 con 0x7f36c00ac930 2023-10-26T01:36:24.187 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.183+0000 7f36c8797640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36c006fe50 con 0x7f36c00cdc30 2023-10-26T01:36:24.187 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.183+0000 7f36c8797640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36c00b0b80 con 0x7f36c01592d0 2023-10-26T01:36:24.187 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.183+0000 7f36c6533640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f36c00ac930 0x7f36c0149960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:24.187 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.183+0000 7f36c6533640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f36c00ac930 0x7f36c0149960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:51916/0 (socket says 172.21.15.169:51916) 2023-10-26T01:36:24.187 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.183+0000 7f36c6533640 1 -- 172.21.15.169:0/2129739613 learned_addr learned my addr 172.21.15.169:0/2129739613 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:36:24.189 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.184+0000 7f36c6533640 1 -- 172.21.15.169:0/2129739613 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f36c01592d0 msgr2=0x7f36c00c9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:24.189 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.184+0000 7f36c5d32640 1 --2- 172.21.15.169:0/2129739613 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f36c00cdc30 0x7f36c00c8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:24.189 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.184+0000 7f36c6d34640 1 --2- 172.21.15.169:0/2129739613 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f36c01592d0 0x7f36c00c9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:24.189 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.184+0000 7f36c6533640 1 --2- 172.21.15.169:0/2129739613 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f36c01592d0 0x7f36c00c9060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:24.190 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.184+0000 7f36c6533640 1 -- 172.21.15.169:0/2129739613 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f36c00cdc30 msgr2=0x7f36c00c8b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:24.190 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.184+0000 7f36c6533640 1 --2- 172.21.15.169:0/2129739613 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f36c00cdc30 0x7f36c00c8b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:24.190 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.184+0000 7f36c6533640 1 -- 172.21.15.169:0/2129739613 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36b0009490 con 0x7f36c00ac930 2023-10-26T01:36:24.190 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.184+0000 7f36c5d32640 1 --2- 172.21.15.169:0/2129739613 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f36c00cdc30 0x7f36c00c8b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:36:24.190 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.184+0000 7f36c6533640 1 --2- 172.21.15.169:0/2129739613 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f36c00ac930 0x7f36c0149960 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f36bc00c3e0 tx=0x7f36bc00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:24.191 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.184+0000 7f36b6ffd640 1 -- 172.21.15.169:0/2129739613 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36bc011790 con 0x7f36c00ac930 2023-10-26T01:36:24.191 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.184+0000 7f36b6ffd640 1 -- 172.21.15.169:0/2129739613 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f36bc0119e0 con 0x7f36c00ac930 2023-10-26T01:36:24.191 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.184+0000 7f36b6ffd640 1 -- 172.21.15.169:0/2129739613 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36bc011e30 con 0x7f36c00ac930 2023-10-26T01:36:24.191 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.184+0000 7f36c8797640 1 -- 172.21.15.169:0/2129739613 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36c00b08a0 con 0x7f36c00ac930 2023-10-26T01:36:24.192 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.184+0000 7f36c8797640 1 -- 172.21.15.169:0/2129739613 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36c0070360 con 0x7f36c00ac930 2023-10-26T01:36:24.192 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.186+0000 7f36c8797640 1 -- 172.21.15.169:0/2129739613 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36c006cfd0 con 0x7f36c00ac930 2023-10-26T01:36:24.192 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.186+0000 7f36b6ffd640 1 -- 172.21.15.169:0/2129739613 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f36bc01c460 con 0x7f36c00ac930 2023-10-26T01:36:24.193 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.188+0000 7f36b6ffd640 1 --2- 172.21.15.169:0/2129739613 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f36a406ae70 0x7f36a406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:24.194 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.188+0000 7f36b6ffd640 1 -- 172.21.15.169:0/2129739613 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f36bc089690 con 0x7f36c00ac930 2023-10-26T01:36:24.194 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.188+0000 7f36b6ffd640 1 -- 172.21.15.169:0/2129739613 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=90}) v3 -- 0x7f36a40729a0 con 0x7f36c00ac930 2023-10-26T01:36:24.194 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.189+0000 7f36c5d32640 1 --2- 172.21.15.169:0/2129739613 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f36a406ae70 0x7f36a406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:24.194 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.190+0000 7f36c5d32640 1 --2- 172.21.15.169:0/2129739613 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f36a406ae70 0x7f36a406d330 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f36b0036000 tx=0x7f36b0004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:24.195 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.192+0000 7f36b6ffd640 1 -- 172.21.15.169:0/2129739613 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f36bc0546a0 con 0x7f36c00ac930 2023-10-26T01:36:24.425 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.422+0000 7f36c8797640 1 -- 172.21.15.169:0/2129739613 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f36c00ce080 con 0x7f36c00ac930 2023-10-26T01:36:24.426 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.424+0000 7f36b6ffd640 1 -- 172.21.15.169:0/2129739613 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+4927 (secure 0 0 0) 0x7f36c00ce080 con 0x7f36c00ac930 2023-10-26T01:36:24.427 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:36:24.427 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":42,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5347,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/2629782429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2629782429},{"type":"v1","addr":"172.21.15.170:6836","nonce":2629782429}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":6474,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2561200417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2561200417},{"type":"v1","addr":"172.21.15.169:6833","nonce":2561200417}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":6480,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/2834878721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":2834878721},{"type":"v1","addr":"172.21.15.170:6837","nonce":2834878721}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T01:35:27.834145+0000","modified":"2023-10-26T01:35:30.917014+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6486},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6486":{"gid":6486,"name":"a","rank":0,"incarnation":38,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6835/4004306080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":4004306080},{"type":"v1","addr":"172.21.15.169:6835","nonce":4004306080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-10-26T01:36:24.427 INFO:teuthology.orchestra.run.smithi169.stderr:dumped fsmap epoch 42 2023-10-26T01:36:24.429 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.426+0000 7f36c8797640 1 -- 172.21.15.169:0/2129739613 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f36a406ae70 msgr2=0x7f36a406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:24.430 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.426+0000 7f36c8797640 1 --2- 172.21.15.169:0/2129739613 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f36a406ae70 0x7f36a406d330 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f36b0036000 tx=0x7f36b0004730 comp rx=0 tx=0).stop 2023-10-26T01:36:24.430 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.426+0000 7f36c8797640 1 -- 172.21.15.169:0/2129739613 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f36c00ac930 msgr2=0x7f36c0149960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:24.430 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.426+0000 7f36c8797640 1 --2- 172.21.15.169:0/2129739613 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f36c00ac930 0x7f36c0149960 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f36bc00c3e0 tx=0x7f36bc00c8a0 comp rx=0 tx=0).stop 2023-10-26T01:36:24.430 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.426+0000 7f36c8797640 1 -- 172.21.15.169:0/2129739613 shutdown_connections 2023-10-26T01:36:24.430 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.426+0000 7f36c8797640 1 --2- 172.21.15.169:0/2129739613 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f36a406ae70 0x7f36a406d330 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:24.431 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.426+0000 7f36c8797640 1 --2- 172.21.15.169:0/2129739613 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f36c01592d0 0x7f36c00c9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:24.431 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.426+0000 7f36c8797640 1 --2- 172.21.15.169:0/2129739613 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f36c00cdc30 0x7f36c00c8b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:24.431 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.426+0000 7f36c8797640 1 --2- 172.21.15.169:0/2129739613 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f36c00ac930 0x7f36c0149960 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:24.431 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.426+0000 7f36c8797640 1 -- 172.21.15.169:0/2129739613 >> 172.21.15.169:0/2129739613 conn(0x7f36c0069930 msgr2=0x7f36c00ae130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:24.431 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.427+0000 7f36c8797640 1 -- 172.21.15.169:0/2129739613 shutdown_connections 2023-10-26T01:36:24.431 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.427+0000 7f36c8797640 1 -- 172.21.15.169:0/2129739613 wait complete. 2023-10-26T01:36:24.448 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:36:24.670 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.667+0000 7f032359e640 1 -- 172.21.15.169:0/1986189902 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03240ac6d0 msgr2=0x7f032415d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:24.670 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.667+0000 7f032359e640 1 --2- 172.21.15.169:0/1986189902 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03240ac6d0 0x7f032415d840 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f03240cbd50 tx=0x7f031402bdb0 comp rx=0 tx=0).stop 2023-10-26T01:36:24.671 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.667+0000 7f032359e640 1 -- 172.21.15.169:0/1986189902 shutdown_connections 2023-10-26T01:36:24.671 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.667+0000 7f032359e640 1 --2- 172.21.15.169:0/1986189902 >> v2:172.21.15.170:3300/0 conn(0x7f03240c9970 0x7f0324098a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:24.671 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.667+0000 7f032359e640 1 --2- 172.21.15.169:0/1986189902 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03240ac6d0 0x7f032415d840 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:24.671 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.667+0000 7f032359e640 1 -- 172.21.15.169:0/1986189902 >> 172.21.15.169:0/1986189902 conn(0x7f03240aedd0 msgr2=0x7f03240af1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:24.671 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.667+0000 7f032359e640 1 -- 172.21.15.169:0/1986189902 shutdown_connections 2023-10-26T01:36:24.671 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.667+0000 7f032359e640 1 -- 172.21.15.169:0/1986189902 wait complete. 2023-10-26T01:36:24.672 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.669+0000 7f032359e640 1 Processor -- start 2023-10-26T01:36:24.672 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.669+0000 7f032359e640 1 -- start start 2023-10-26T01:36:24.674 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.670+0000 7f032359e640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f032409a5c0 0x7f032413f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:24.674 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.670+0000 7f032359e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03240ac6d0 0x7f03240afce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:24.675 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.670+0000 7f032359e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f03240c9970 0x7f0324071070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:24.675 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.670+0000 7f032359e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f032413e770 con 0x7f03240ac6d0 2023-10-26T01:36:24.675 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.670+0000 7f032359e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03241496b0 con 0x7f03240c9970 2023-10-26T01:36:24.675 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.670+0000 7f032359e640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f032413ea10 con 0x7f032409a5c0 2023-10-26T01:36:24.675 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.670+0000 7f031bfff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03240ac6d0 0x7f03240afce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:24.676 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.670+0000 7f031bfff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03240ac6d0 0x7f03240afce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:51924/0 (socket says 172.21.15.169:51924) 2023-10-26T01:36:24.676 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.670+0000 7f031bfff640 1 -- 172.21.15.169:0/3332882201 learned_addr learned my addr 172.21.15.169:0/3332882201 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:36:24.676 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.671+0000 7f0322d9d640 1 --2- 172.21.15.169:0/3332882201 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f03240c9970 0x7f0324071070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:24.676 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.671+0000 7f032259c640 1 --2- 172.21.15.169:0/3332882201 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f032409a5c0 0x7f032413f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:24.676 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.671+0000 7f031bfff640 1 -- 172.21.15.169:0/3332882201 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f032409a5c0 msgr2=0x7f032413f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:24.677 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.671+0000 7f031bfff640 1 --2- 172.21.15.169:0/3332882201 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f032409a5c0 0x7f032413f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:24.677 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.671+0000 7f031bfff640 1 -- 172.21.15.169:0/3332882201 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f03240c9970 msgr2=0x7f0324071070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:24.677 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.671+0000 7f031bfff640 1 --2- 172.21.15.169:0/3332882201 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f03240c9970 0x7f0324071070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:24.677 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.671+0000 7f031bfff640 1 -- 172.21.15.169:0/3332882201 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0314009560 con 0x7f03240ac6d0 2023-10-26T01:36:24.677 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.671+0000 7f0322d9d640 1 --2- 172.21.15.169:0/3332882201 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f03240c9970 0x7f0324071070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:36:24.678 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.671+0000 7f031bfff640 1 --2- 172.21.15.169:0/3332882201 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03240ac6d0 0x7f03240afce0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f030c00c970 tx=0x7f030c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:24.678 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.671+0000 7f031affd640 1 -- 172.21.15.169:0/3332882201 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f030c007bb0 con 0x7f03240ac6d0 2023-10-26T01:36:24.678 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.671+0000 7f031affd640 1 -- 172.21.15.169:0/3332882201 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f030c007e20 con 0x7f03240ac6d0 2023-10-26T01:36:24.678 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.671+0000 7f031affd640 1 -- 172.21.15.169:0/3332882201 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f030c005280 con 0x7f03240ac6d0 2023-10-26T01:36:24.678 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.671+0000 7f032359e640 1 -- 172.21.15.169:0/3332882201 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f032413ecf0 con 0x7f03240ac6d0 2023-10-26T01:36:24.679 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.672+0000 7f032359e640 1 -- 172.21.15.169:0/3332882201 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03240715b0 con 0x7f03240ac6d0 2023-10-26T01:36:24.679 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.673+0000 7f032359e640 1 -- 172.21.15.169:0/3332882201 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0324073c10 con 0x7f03240ac6d0 2023-10-26T01:36:24.680 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.673+0000 7f031affd640 1 -- 172.21.15.169:0/3332882201 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f030c0053e0 con 0x7f03240ac6d0 2023-10-26T01:36:24.680 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.674+0000 7f031affd640 1 --2- 172.21.15.169:0/3332882201 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f030406ae40 0x7f030406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:24.680 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.674+0000 7f031affd640 1 -- 172.21.15.169:0/3332882201 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f030c089c90 con 0x7f03240ac6d0 2023-10-26T01:36:24.680 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.675+0000 7f031affd640 1 -- 172.21.15.169:0/3332882201 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=90}) v3 -- 0x7f0304072970 con 0x7f03240ac6d0 2023-10-26T01:36:24.680 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.675+0000 7f032259c640 1 --2- 172.21.15.169:0/3332882201 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f030406ae40 0x7f030406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:24.681 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.676+0000 7f032259c640 1 --2- 172.21.15.169:0/3332882201 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f030406ae40 0x7f030406d300 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f0314035000 tx=0x7f0314004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:24.681 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.679+0000 7f031affd640 1 -- 172.21.15.169:0/3332882201 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f030c01d030 con 0x7f03240ac6d0 2023-10-26T01:36:24.873 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.870+0000 7f032359e640 1 -- 172.21.15.169:0/3332882201 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f03240cc6e0 con 0x7f03240ac6d0 2023-10-26T01:36:24.875 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.872+0000 7f031affd640 1 -- 172.21.15.169:0/3332882201 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+16645 (secure 0 0 0) 0x7f030c01d030 con 0x7f03240ac6d0 2023-10-26T01:36:24.875 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:36:24.876 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":89,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:36:24.076851+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-10-26T01:35:26.320950+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-10-26T01:35:26.855916+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","full","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","full","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:36:24.878 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.874+0000 7f032359e640 1 -- 172.21.15.169:0/3332882201 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f030406ae40 msgr2=0x7f030406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:24.878 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.874+0000 7f032359e640 1 --2- 172.21.15.169:0/3332882201 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f030406ae40 0x7f030406d300 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f0314035000 tx=0x7f0314004730 comp rx=0 tx=0).stop 2023-10-26T01:36:24.878 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.874+0000 7f032359e640 1 -- 172.21.15.169:0/3332882201 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03240ac6d0 msgr2=0x7f03240afce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:24.878 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.874+0000 7f032359e640 1 --2- 172.21.15.169:0/3332882201 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03240ac6d0 0x7f03240afce0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f030c00c970 tx=0x7f030c00ce30 comp rx=0 tx=0).stop 2023-10-26T01:36:24.879 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.875+0000 7f032359e640 1 -- 172.21.15.169:0/3332882201 shutdown_connections 2023-10-26T01:36:24.879 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.875+0000 7f032359e640 1 --2- 172.21.15.169:0/3332882201 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f030406ae40 0x7f030406d300 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:24.879 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.875+0000 7f032359e640 1 --2- 172.21.15.169:0/3332882201 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03240ac6d0 0x7f03240afce0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:24.879 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.875+0000 7f032359e640 1 --2- 172.21.15.169:0/3332882201 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f03240c9970 0x7f0324071070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:24.879 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.875+0000 7f032359e640 1 --2- 172.21.15.169:0/3332882201 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f032409a5c0 0x7f032413f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:24.880 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.875+0000 7f032359e640 1 -- 172.21.15.169:0/3332882201 >> 172.21.15.169:0/3332882201 conn(0x7f03240aedd0 msgr2=0x7f03240ab670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:24.880 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.875+0000 7f032359e640 1 -- 172.21.15.169:0/3332882201 shutdown_connections 2023-10-26T01:36:24.880 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:24.875+0000 7f032359e640 1 -- 172.21.15.169:0/3332882201 wait complete. 2023-10-26T01:36:24.897 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T01:30:32.200808+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-26T01:35:26.320950+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2023-10-26T01:35:26.855916+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T01:36:24.898 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T01:36:25.125 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:25.123+0000 7f359cef2640 1 -- 172.21.15.169:0/4016117959 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3584003970 msgr2=0x7f3584003d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:25.126 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:25.123+0000 7f359cef2640 1 --2- 172.21.15.169:0/4016117959 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3584003970 0x7f3584003d70 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f358800c690 tx=0x7f35840057f0 comp rx=0 tx=0).stop 2023-10-26T01:36:25.126 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:25.123+0000 7f359cef2640 1 -- 172.21.15.169:0/4016117959 shutdown_connections 2023-10-26T01:36:25.126 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:25.123+0000 7f359cef2640 1 --2- 172.21.15.169:0/4016117959 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3584003970 0x7f3584003d70 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:25.126 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:25.123+0000 7f359cef2640 1 -- 172.21.15.169:0/4016117959 >> 172.21.15.169:0/4016117959 conn(0x7f359806b560 msgr2=0x7f359806b970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:25.127 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:25.123+0000 7f359cef2640 1 -- 172.21.15.169:0/4016117959 shutdown_connections 2023-10-26T01:36:25.127 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:25.123+0000 7f359cef2640 1 -- 172.21.15.169:0/4016117959 wait complete. 2023-10-26T01:36:25.128 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:25.124+0000 7f359cef2640 1 Processor -- start 2023-10-26T01:36:25.128 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:25.125+0000 7f359cef2640 1 -- start start 2023-10-26T01:36:25.130 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:25.125+0000 7f359cef2640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3584003970 0x7f359814ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:25.130 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:25.125+0000 7f359cef2640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3598156be0 0x7f3598157060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:25.130 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:25.126+0000 7f359cef2640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3598154b80 0x7f3598155000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:25.131 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:25.126+0000 7f359cef2640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35981575a0 con 0x7f3598154b80 2023-10-26T01:36:25.131 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:25.126+0000 7f359cef2640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f359814ef80 con 0x7f3598156be0 2023-10-26T01:36:25.131 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:25.126+0000 7f359cef2640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3598159ff0 con 0x7f3584003970 2023-10-26T01:36:25.131 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:25.126+0000 7f3596d9d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3598154b80 0x7f3598155000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:25.131 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:25.126+0000 7f3596d9d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3598154b80 0x7f3598155000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:51938/0 (socket says 172.21.15.169:51938) 2023-10-26T01:36:25.132 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:25.126+0000 7f3596d9d640 1 -- 172.21.15.169:0/2253514398 learned_addr learned my addr 172.21.15.169:0/2253514398 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:36:25.132 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:25.126+0000 7f359659c640 1 --2- 172.21.15.169:0/2253514398 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3584003970 0x7f359814ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:25.132 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:25.126+0000 7f3596d9d640 1 -- 172.21.15.169:0/2253514398 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3584003970 msgr2=0x7f359814ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:25.132 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:25.126+0000 7f3596d9d640 1 --2- 172.21.15.169:0/2253514398 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3584003970 0x7f359814ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:25.132 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:25.126+0000 7f358dd9b640 1 --2- 172.21.15.169:0/2253514398 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3598156be0 0x7f3598157060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:25.133 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:25.126+0000 7f3596d9d640 1 -- 172.21.15.169:0/2253514398 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3598156be0 msgr2=0x7f3598157060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:25.133 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:25.126+0000 7f3596d9d640 1 --2- 172.21.15.169:0/2253514398 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3598156be0 0x7f3598157060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:25.133 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:25.126+0000 7f3596d9d640 1 -- 172.21.15.169:0/2253514398 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35980cbdf0 con 0x7f3598154b80 2023-10-26T01:36:25.133 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:25.127+0000 7f3596d9d640 1 --2- 172.21.15.169:0/2253514398 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3598154b80 0x7f3598155000 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f35980caaf0 tx=0x7f359000c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:25.133 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:25.127+0000 7f358f7fe640 1 -- 172.21.15.169:0/2253514398 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3590005230 con 0x7f3598154b80 2023-10-26T01:36:25.134 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:25.127+0000 7f358f7fe640 1 -- 172.21.15.169:0/2253514398 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f35900054a0 con 0x7f3598154b80 2023-10-26T01:36:25.134 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:25.127+0000 7f358f7fe640 1 -- 172.21.15.169:0/2253514398 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35900117f0 con 0x7f3598154b80 2023-10-26T01:36:25.134 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:25.127+0000 7f359cef2640 1 -- 172.21.15.169:0/2253514398 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35981593e0 con 0x7f3598154b80 2023-10-26T01:36:25.134 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:25.127+0000 7f359cef2640 1 -- 172.21.15.169:0/2253514398 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35981599a0 con 0x7f3598154b80 2023-10-26T01:36:25.135 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:25.128+0000 7f359cef2640 1 -- 172.21.15.169:0/2253514398 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3598066ac0 con 0x7f3598154b80 2023-10-26T01:36:25.135 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:25.129+0000 7f358f7fe640 1 -- 172.21.15.169:0/2253514398 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f3590011950 con 0x7f3598154b80 2023-10-26T01:36:25.135 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:25.130+0000 7f358f7fe640 1 --2- 172.21.15.169:0/2253514398 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f357806ae40 0x7f357806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:25.136 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:25.131+0000 7f358f7fe640 1 -- 172.21.15.169:0/2253514398 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3590088960 con 0x7f3598154b80 2023-10-26T01:36:25.136 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:25.131+0000 7f359659c640 1 --2- 172.21.15.169:0/2253514398 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f357806ae40 0x7f357806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:25.136 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:25.132+0000 7f359659c640 1 --2- 172.21.15.169:0/2253514398 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f357806ae40 0x7f357806d300 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f3588034000 tx=0x7f358802cf90 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:25.138 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:25.135+0000 7f358f7fe640 1 -- 172.21.15.169:0/2253514398 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3590053aa0 con 0x7f3598154b80 2023-10-26T01:36:25.327 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:25.324+0000 7f359cef2640 1 -- 172.21.15.169:0/2253514398 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f359815a180 con 0x7f3598154b80 2023-10-26T01:36:25.328 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:25.325+0000 7f358f7fe640 1 -- 172.21.15.169:0/2253514398 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3590017070 con 0x7f3598154b80 2023-10-26T01:36:25.328 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 1 2023-10-26T01:36:25.331 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:25.328+0000 7f359cef2640 1 -- 172.21.15.169:0/2253514398 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f357806ae40 msgr2=0x7f357806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:25.331 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:25.328+0000 7f359cef2640 1 --2- 172.21.15.169:0/2253514398 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f357806ae40 0x7f357806d300 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f3588034000 tx=0x7f358802cf90 comp rx=0 tx=0).stop 2023-10-26T01:36:25.331 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:25.328+0000 7f359cef2640 1 -- 172.21.15.169:0/2253514398 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3598154b80 msgr2=0x7f3598155000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:25.332 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:25.328+0000 7f359cef2640 1 --2- 172.21.15.169:0/2253514398 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3598154b80 0x7f3598155000 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f35980caaf0 tx=0x7f359000c820 comp rx=0 tx=0).stop 2023-10-26T01:36:25.332 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:25.328+0000 7f359cef2640 1 -- 172.21.15.169:0/2253514398 shutdown_connections 2023-10-26T01:36:25.332 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:25.328+0000 7f359cef2640 1 --2- 172.21.15.169:0/2253514398 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f357806ae40 0x7f357806d300 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:25.332 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:25.328+0000 7f359cef2640 1 --2- 172.21.15.169:0/2253514398 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3598154b80 0x7f3598155000 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:25.332 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:25.328+0000 7f359cef2640 1 --2- 172.21.15.169:0/2253514398 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3598156be0 0x7f3598157060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:25.332 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:25.328+0000 7f359cef2640 1 --2- 172.21.15.169:0/2253514398 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3584003970 0x7f359814ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:25.333 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:25.328+0000 7f359cef2640 1 -- 172.21.15.169:0/2253514398 >> 172.21.15.169:0/2253514398 conn(0x7f359806b560 msgr2=0x7f3598065490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:25.333 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:25.328+0000 7f359cef2640 1 -- 172.21.15.169:0/2253514398 shutdown_connections 2023-10-26T01:36:25.333 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:25.329+0000 7f359cef2640 1 -- 172.21.15.169:0/2253514398 wait complete. 2023-10-26T01:36:25.350 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-26T01:36:25.571 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:25.567+0000 7f1bcda94640 1 -- 172.21.15.169:0/2505403472 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1bc81593e0 msgr2=0x7f1bc80ac200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:25.571 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:25.567+0000 7f1bcda94640 1 --2- 172.21.15.169:0/2505403472 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1bc81593e0 0x7f1bc80ac200 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f1bb8005fd0 tx=0x7f1bb802bd90 comp rx=0 tx=0).stop 2023-10-26T01:36:25.571 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:25.567+0000 7f1bcda94640 1 -- 172.21.15.169:0/2505403472 shutdown_connections 2023-10-26T01:36:25.571 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:25.567+0000 7f1bcda94640 1 --2- 172.21.15.169:0/2505403472 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1bc80b0c80 0x7f1bc80cd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:25.572 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:25.567+0000 7f1bcda94640 1 --2- 172.21.15.169:0/2505403472 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1bc81593e0 0x7f1bc80ac200 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:25.572 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:25.567+0000 7f1bcda94640 1 -- 172.21.15.169:0/2505403472 >> 172.21.15.169:0/2505403472 conn(0x7f1bc8069930 msgr2=0x7f1bc8069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:25.572 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:25.568+0000 7f1bcda94640 1 -- 172.21.15.169:0/2505403472 shutdown_connections 2023-10-26T01:36:25.572 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:25.568+0000 7f1bcda94640 1 -- 172.21.15.169:0/2505403472 wait complete. 2023-10-26T01:36:25.575 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:25.569+0000 7f1bcda94640 1 Processor -- start 2023-10-26T01:36:25.575 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:25.569+0000 7f1bcda94640 1 -- start start 2023-10-26T01:36:25.575 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:25.570+0000 7f1bcda94640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1bc80b0c80 0x7f1bc81f1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:25.575 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:25.570+0000 7f1bcda94640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1bc81593e0 0x7f1bc81558a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:25.575 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:25.570+0000 7f1bcda94640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1bc81597c0 0x7f1bc815acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:25.576 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:25.570+0000 7f1bcda94640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc814e5f0 con 0x7f1bc80b0c80 2023-10-26T01:36:25.576 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:25.570+0000 7f1bcda94640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc8158730 con 0x7f1bc81593e0 2023-10-26T01:36:25.576 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:25.570+0000 7f1bcda94640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc8158450 con 0x7f1bc81597c0 2023-10-26T01:36:25.576 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:25.571+0000 7f1bc77fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1bc80b0c80 0x7f1bc81f1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:25.576 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:25.571+0000 7f1bc77fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1bc80b0c80 0x7f1bc81f1790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:51954/0 (socket says 172.21.15.169:51954) 2023-10-26T01:36:25.577 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:25.571+0000 7f1bc77fe640 1 -- 172.21.15.169:0/2909915668 learned_addr learned my addr 172.21.15.169:0/2909915668 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:36:25.577 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:25.571+0000 7f1bc7fff640 1 --2- 172.21.15.169:0/2909915668 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1bc81597c0 0x7f1bc815acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:25.577 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:25.571+0000 7f1bbffff640 1 --2- 172.21.15.169:0/2909915668 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1bc81593e0 0x7f1bc81558a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:25.577 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:25.571+0000 7f1bc77fe640 1 -- 172.21.15.169:0/2909915668 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1bc81597c0 msgr2=0x7f1bc815acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:25.577 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:25.571+0000 7f1bc77fe640 1 --2- 172.21.15.169:0/2909915668 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1bc81597c0 0x7f1bc815acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:25.578 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:25.571+0000 7f1bc77fe640 1 -- 172.21.15.169:0/2909915668 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1bc81593e0 msgr2=0x7f1bc81558a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:25.578 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:25.571+0000 7f1bc77fe640 1 --2- 172.21.15.169:0/2909915668 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1bc81593e0 0x7f1bc81558a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:25.578 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:25.571+0000 7f1bc77fe640 1 -- 172.21.15.169:0/2909915668 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bb8009560 con 0x7f1bc80b0c80 2023-10-26T01:36:25.578 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:25.571+0000 7f1bc7fff640 1 --2- 172.21.15.169:0/2909915668 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1bc81597c0 0x7f1bc815acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:36:25.578 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:25.571+0000 7f1bbffff640 1 --2- 172.21.15.169:0/2909915668 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1bc81593e0 0x7f1bc81558a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:36:25.579 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:25.571+0000 7f1bc77fe640 1 --2- 172.21.15.169:0/2909915668 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1bc80b0c80 0x7f1bc81f1790 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f1bb8006010 tx=0x7f1bb802db30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:25.579 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:25.572+0000 7f1bc4ff9640 1 -- 172.21.15.169:0/2909915668 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bb80035f0 con 0x7f1bc80b0c80 2023-10-26T01:36:25.579 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:25.572+0000 7f1bc4ff9640 1 -- 172.21.15.169:0/2909915668 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1bb8003840 con 0x7f1bc80b0c80 2023-10-26T01:36:25.579 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:25.572+0000 7f1bc4ff9640 1 -- 172.21.15.169:0/2909915668 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bb8003c50 con 0x7f1bc80b0c80 2023-10-26T01:36:25.579 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:25.572+0000 7f1bcda94640 1 -- 172.21.15.169:0/2909915668 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bc806eff0 con 0x7f1bc80b0c80 2023-10-26T01:36:25.579 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:25.572+0000 7f1bcda94640 1 -- 172.21.15.169:0/2909915668 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bc806f5b0 con 0x7f1bc80b0c80 2023-10-26T01:36:25.580 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:25.573+0000 7f1bcda94640 1 -- 172.21.15.169:0/2909915668 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bc806ee50 con 0x7f1bc80b0c80 2023-10-26T01:36:25.580 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:25.574+0000 7f1bc4ff9640 1 -- 172.21.15.169:0/2909915668 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f1bb803e450 con 0x7f1bc80b0c80 2023-10-26T01:36:25.581 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:25.575+0000 7f1bc4ff9640 1 --2- 172.21.15.169:0/2909915668 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f1ba806ae40 0x7f1ba806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:25.581 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:25.575+0000 7f1bc4ff9640 1 -- 172.21.15.169:0/2909915668 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1bb80ac1b0 con 0x7f1bc80b0c80 2023-10-26T01:36:25.581 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:25.576+0000 7f1bbffff640 1 --2- 172.21.15.169:0/2909915668 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f1ba806ae40 0x7f1ba806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:25.581 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:25.577+0000 7f1bbffff640 1 --2- 172.21.15.169:0/2909915668 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f1ba806ae40 0x7f1ba806d300 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f1bc815cb30 tx=0x7f1bb0008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:25.582 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:25.580+0000 7f1bc4ff9640 1 -- 172.21.15.169:0/2909915668 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1bc806ee50 con 0x7f1bc80b0c80 2023-10-26T01:36:25.770 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:25.767+0000 7f1bcda94640 1 -- 172.21.15.169:0/2909915668 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1bc806ee50 con 0x7f1bc80b0c80 2023-10-26T01:36:25.770 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:25.768+0000 7f1bc4ff9640 1 -- 172.21.15.169:0/2909915668 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1bc806ee50 con 0x7f1bc80b0c80 2023-10-26T01:36:25.771 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 32 2023-10-26T01:36:25.773 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:25.770+0000 7f1bcda94640 1 -- 172.21.15.169:0/2909915668 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f1ba806ae40 msgr2=0x7f1ba806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:25.773 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:25.770+0000 7f1bcda94640 1 --2- 172.21.15.169:0/2909915668 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f1ba806ae40 0x7f1ba806d300 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f1bc815cb30 tx=0x7f1bb0008040 comp rx=0 tx=0).stop 2023-10-26T01:36:25.774 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:25.770+0000 7f1bcda94640 1 -- 172.21.15.169:0/2909915668 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1bc80b0c80 msgr2=0x7f1bc81f1790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:25.774 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:25.770+0000 7f1bcda94640 1 --2- 172.21.15.169:0/2909915668 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1bc80b0c80 0x7f1bc81f1790 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f1bb8006010 tx=0x7f1bb802db30 comp rx=0 tx=0).stop 2023-10-26T01:36:25.774 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:25.770+0000 7f1bcda94640 1 -- 172.21.15.169:0/2909915668 shutdown_connections 2023-10-26T01:36:25.774 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:25.770+0000 7f1bcda94640 1 --2- 172.21.15.169:0/2909915668 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f1ba806ae40 0x7f1ba806d300 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:25.774 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:25.770+0000 7f1bcda94640 1 --2- 172.21.15.169:0/2909915668 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1bc81597c0 0x7f1bc815acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:25.775 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:25.770+0000 7f1bcda94640 1 --2- 172.21.15.169:0/2909915668 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1bc81593e0 0x7f1bc81558a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:25.775 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:25.770+0000 7f1bcda94640 1 --2- 172.21.15.169:0/2909915668 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1bc80b0c80 0x7f1bc81f1790 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:25.775 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:25.770+0000 7f1bcda94640 1 -- 172.21.15.169:0/2909915668 >> 172.21.15.169:0/2909915668 conn(0x7f1bc8069930 msgr2=0x7f1bc80ad000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:25.775 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:25.770+0000 7f1bcda94640 1 -- 172.21.15.169:0/2909915668 shutdown_connections 2023-10-26T01:36:25.775 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:25.771+0000 7f1bcda94640 1 -- 172.21.15.169:0/2909915668 wait complete. 2023-10-26T01:36:25.792 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-26T01:36:26.014 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.010+0000 7f1398d85640 1 -- 172.21.15.169:0/1786299627 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f13940b0840 msgr2=0x7f1394069a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:26.014 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.010+0000 7f1398d85640 1 --2- 172.21.15.169:0/1786299627 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f13940b0840 0x7f1394069a10 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f138c005fd0 tx=0x7f138c02c1b0 comp rx=0 tx=0).stop 2023-10-26T01:36:26.014 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.011+0000 7f1398d85640 1 -- 172.21.15.169:0/1786299627 shutdown_connections 2023-10-26T01:36:26.014 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.011+0000 7f1398d85640 1 --2- 172.21.15.169:0/1786299627 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f13940b0840 0x7f1394069a10 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:26.015 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.011+0000 7f1398d85640 1 --2- 172.21.15.169:0/1786299627 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f139415b140 0x7f13940b0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:26.015 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.011+0000 7f1398d85640 1 -- 172.21.15.169:0/1786299627 >> 172.21.15.169:0/1786299627 conn(0x7f13940af110 msgr2=0x7f13940af520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:26.015 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.011+0000 7f1398d85640 1 -- 172.21.15.169:0/1786299627 shutdown_connections 2023-10-26T01:36:26.015 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.011+0000 7f1398d85640 1 -- 172.21.15.169:0/1786299627 wait complete. 2023-10-26T01:36:26.016 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.013+0000 7f1398d85640 1 Processor -- start 2023-10-26T01:36:26.016 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.013+0000 7f1398d85640 1 -- start start 2023-10-26T01:36:26.018 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.014+0000 7f1398d85640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f139406cba0 0x7f13941e4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:26.018 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.014+0000 7f1398d85640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f13940b0840 0x7f1394150390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:26.018 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.014+0000 7f1398d85640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f139415b140 0x7f13940ae110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:26.018 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.014+0000 7f1398d85640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13940cbca0 con 0x7f139415b140 2023-10-26T01:36:26.018 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.014+0000 7f1398d85640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13940cbf10 con 0x7f139406cba0 2023-10-26T01:36:26.019 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.014+0000 7f1398d85640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13940cc1f0 con 0x7f13940b0840 2023-10-26T01:36:26.019 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.015+0000 7f1393fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f139415b140 0x7f13940ae110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:26.019 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.015+0000 7f1393fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f139415b140 0x7f13940ae110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:51976/0 (socket says 172.21.15.169:51976) 2023-10-26T01:36:26.019 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.015+0000 7f1393fff640 1 -- 172.21.15.169:0/4095177064 learned_addr learned my addr 172.21.15.169:0/4095177064 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:36:26.019 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.015+0000 7f1392ffd640 1 --2- 172.21.15.169:0/4095177064 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f13940b0840 0x7f1394150390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:26.020 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.015+0000 7f13937fe640 1 --2- 172.21.15.169:0/4095177064 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f139406cba0 0x7f13941e4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:26.020 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.015+0000 7f1393fff640 1 -- 172.21.15.169:0/4095177064 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f13940b0840 msgr2=0x7f1394150390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:26.020 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.015+0000 7f1393fff640 1 --2- 172.21.15.169:0/4095177064 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f13940b0840 0x7f1394150390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:26.020 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.015+0000 7f1393fff640 1 -- 172.21.15.169:0/4095177064 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f139406cba0 msgr2=0x7f13941e4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:26.020 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.015+0000 7f1393fff640 1 --2- 172.21.15.169:0/4095177064 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f139406cba0 0x7f13941e4e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:26.021 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.015+0000 7f1393fff640 1 -- 172.21.15.169:0/4095177064 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f138c009900 con 0x7f139415b140 2023-10-26T01:36:26.022 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.015+0000 7f1393fff640 1 --2- 172.21.15.169:0/4095177064 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f139415b140 0x7f13940ae110 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f138c037040 tx=0x7f138c035db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:26.022 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.016+0000 7f137bfff640 1 -- 172.21.15.169:0/4095177064 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f138c034440 con 0x7f139415b140 2023-10-26T01:36:26.022 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.016+0000 7f137bfff640 1 -- 172.21.15.169:0/4095177064 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f138c004540 con 0x7f139415b140 2023-10-26T01:36:26.023 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.016+0000 7f1398d85640 1 -- 172.21.15.169:0/4095177064 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13940aaf80 con 0x7f139415b140 2023-10-26T01:36:26.023 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.016+0000 7f137bfff640 1 -- 172.21.15.169:0/4095177064 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f138c03f440 con 0x7f139415b140 2023-10-26T01:36:26.023 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.016+0000 7f1398d85640 1 -- 172.21.15.169:0/4095177064 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f139409b0a0 con 0x7f139415b140 2023-10-26T01:36:26.023 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.017+0000 7f1398d85640 1 -- 172.21.15.169:0/4095177064 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13940cc710 con 0x7f139415b140 2023-10-26T01:36:26.023 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.018+0000 7f137bfff640 1 -- 172.21.15.169:0/4095177064 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f138c03b040 con 0x7f139415b140 2023-10-26T01:36:26.024 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.019+0000 7f137bfff640 1 --2- 172.21.15.169:0/4095177064 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f138006ae40 0x7f138006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:26.024 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.019+0000 7f137bfff640 1 -- 172.21.15.169:0/4095177064 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f138c0ac4e0 con 0x7f139415b140 2023-10-26T01:36:26.024 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.020+0000 7f13937fe640 1 --2- 172.21.15.169:0/4095177064 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f138006ae40 0x7f138006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:26.024 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.021+0000 7f13937fe640 1 --2- 172.21.15.169:0/4095177064 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f138006ae40 0x7f138006d300 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f1384006fd0 tx=0x7f1384008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:26.026 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.023+0000 7f137bfff640 1 -- 172.21.15.169:0/4095177064 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f13940cc710 con 0x7f139415b140 2023-10-26T01:36:26.216 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.213+0000 7f1398d85640 1 -- 172.21.15.169:0/4095177064 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f13940cc710 con 0x7f139415b140 2023-10-26T01:36:26.216 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.213+0000 7f137bfff640 1 -- 172.21.15.169:0/4095177064 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) v1 ==== 100+0+11 (secure 0 0 0) 0x7f13940cc710 con 0x7f139415b140 2023-10-26T01:36:26.217 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 64 2023-10-26T01:36:26.219 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.216+0000 7f1398d85640 1 -- 172.21.15.169:0/4095177064 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f138006ae40 msgr2=0x7f138006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:26.220 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.216+0000 7f1398d85640 1 --2- 172.21.15.169:0/4095177064 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f138006ae40 0x7f138006d300 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f1384006fd0 tx=0x7f1384008040 comp rx=0 tx=0).stop 2023-10-26T01:36:26.220 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.216+0000 7f1398d85640 1 -- 172.21.15.169:0/4095177064 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f139415b140 msgr2=0x7f13940ae110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:26.220 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.216+0000 7f1398d85640 1 --2- 172.21.15.169:0/4095177064 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f139415b140 0x7f13940ae110 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f138c037040 tx=0x7f138c035db0 comp rx=0 tx=0).stop 2023-10-26T01:36:26.220 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.217+0000 7f1398d85640 1 -- 172.21.15.169:0/4095177064 shutdown_connections 2023-10-26T01:36:26.220 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.217+0000 7f1398d85640 1 --2- 172.21.15.169:0/4095177064 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f138006ae40 0x7f138006d300 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:26.221 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.217+0000 7f1398d85640 1 --2- 172.21.15.169:0/4095177064 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f139415b140 0x7f13940ae110 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:26.221 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.217+0000 7f1398d85640 1 --2- 172.21.15.169:0/4095177064 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f13940b0840 0x7f1394150390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:26.221 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.217+0000 7f1398d85640 1 --2- 172.21.15.169:0/4095177064 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f139406cba0 0x7f13941e4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:26.221 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.217+0000 7f1398d85640 1 -- 172.21.15.169:0/4095177064 >> 172.21.15.169:0/4095177064 conn(0x7f13940af110 msgr2=0x7f139415a510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:26.222 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.217+0000 7f1398d85640 1 -- 172.21.15.169:0/4095177064 shutdown_connections 2023-10-26T01:36:26.222 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.217+0000 7f1398d85640 1 -- 172.21.15.169:0/4095177064 wait complete. 2023-10-26T01:36:26.239 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T01:36:26.452 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.449+0000 7fda3dc6f640 1 -- 172.21.15.169:0/3470010361 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda380aaf20 msgr2=0x7fda380b1ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:26.453 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.449+0000 7fda3dc6f640 1 --2- 172.21.15.169:0/3470010361 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda380aaf20 0x7fda380b1ab0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fda20005fd0 tx=0x7fda2002bd90 comp rx=0 tx=0).stop 2023-10-26T01:36:26.453 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.449+0000 7fda3dc6f640 1 -- 172.21.15.169:0/3470010361 shutdown_connections 2023-10-26T01:36:26.453 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.450+0000 7fda3dc6f640 1 --2- 172.21.15.169:0/3470010361 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda380aaf20 0x7fda380b1ab0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:26.453 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.450+0000 7fda3dc6f640 1 -- 172.21.15.169:0/3470010361 >> 172.21.15.169:0/3470010361 conn(0x7fda380aeee0 msgr2=0x7fda380af2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:26.454 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.450+0000 7fda3dc6f640 1 -- 172.21.15.169:0/3470010361 shutdown_connections 2023-10-26T01:36:26.454 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.450+0000 7fda3dc6f640 1 -- 172.21.15.169:0/3470010361 wait complete. 2023-10-26T01:36:26.454 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.452+0000 7fda3dc6f640 1 Processor -- start 2023-10-26T01:36:26.455 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.452+0000 7fda3dc6f640 1 -- start start 2023-10-26T01:36:26.457 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.453+0000 7fda3dc6f640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fda3806cd20 0x7fda38155290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:26.457 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.453+0000 7fda3dc6f640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda380aaf20 0x7fda380cacf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:26.457 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.453+0000 7fda3dc6f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fda380b1ff0 0x7fda38153d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:26.457 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.453+0000 7fda3dc6f640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda38156c30 con 0x7fda380aaf20 2023-10-26T01:36:26.458 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.453+0000 7fda3dc6f640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda38155b90 con 0x7fda380b1ff0 2023-10-26T01:36:26.458 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.453+0000 7fda3dc6f640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda38155e70 con 0x7fda3806cd20 2023-10-26T01:36:26.458 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.454+0000 7fda2ffff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda380aaf20 0x7fda380cacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:26.458 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.454+0000 7fda2ffff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda380aaf20 0x7fda380cacf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:51982/0 (socket says 172.21.15.169:51982) 2023-10-26T01:36:26.458 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.454+0000 7fda2ffff640 1 -- 172.21.15.169:0/2716639947 learned_addr learned my addr 172.21.15.169:0/2716639947 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:36:26.459 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.454+0000 7fda377fe640 1 --2- 172.21.15.169:0/2716639947 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fda3806cd20 0x7fda38155290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:26.459 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.454+0000 7fda37fff640 1 --2- 172.21.15.169:0/2716639947 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fda380b1ff0 0x7fda38153d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:26.459 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.454+0000 7fda2ffff640 1 -- 172.21.15.169:0/2716639947 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fda3806cd20 msgr2=0x7fda38155290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:26.459 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.454+0000 7fda2ffff640 1 --2- 172.21.15.169:0/2716639947 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fda3806cd20 0x7fda38155290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:26.460 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.454+0000 7fda2ffff640 1 -- 172.21.15.169:0/2716639947 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fda380b1ff0 msgr2=0x7fda38153d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:26.460 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.454+0000 7fda2ffff640 1 --2- 172.21.15.169:0/2716639947 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fda380b1ff0 0x7fda38153d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:26.460 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.454+0000 7fda2ffff640 1 -- 172.21.15.169:0/2716639947 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda20009560 con 0x7fda380aaf20 2023-10-26T01:36:26.460 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.454+0000 7fda37fff640 1 --2- 172.21.15.169:0/2716639947 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fda380b1ff0 0x7fda38153d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:36:26.460 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.454+0000 7fda2ffff640 1 --2- 172.21.15.169:0/2716639947 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda380aaf20 0x7fda380cacf0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fda20006010 tx=0x7fda2002dd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:26.461 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.455+0000 7fda34ff9640 1 -- 172.21.15.169:0/2716639947 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda20005650 con 0x7fda380aaf20 2023-10-26T01:36:26.462 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.455+0000 7fda34ff9640 1 -- 172.21.15.169:0/2716639947 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fda200038f0 con 0x7fda380aaf20 2023-10-26T01:36:26.462 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.455+0000 7fda34ff9640 1 -- 172.21.15.169:0/2716639947 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda20003d00 con 0x7fda380aaf20 2023-10-26T01:36:26.462 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.455+0000 7fda3dc6f640 1 -- 172.21.15.169:0/2716639947 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda38156150 con 0x7fda380aaf20 2023-10-26T01:36:26.462 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.455+0000 7fda3dc6f640 1 -- 172.21.15.169:0/2716639947 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda38156710 con 0x7fda380aaf20 2023-10-26T01:36:26.463 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.456+0000 7fda3dc6f640 1 -- 172.21.15.169:0/2716639947 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda3804da70 con 0x7fda380aaf20 2023-10-26T01:36:26.463 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.457+0000 7fda34ff9640 1 -- 172.21.15.169:0/2716639947 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fda2003e450 con 0x7fda380aaf20 2023-10-26T01:36:26.463 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.458+0000 7fda34ff9640 1 --2- 172.21.15.169:0/2716639947 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fda1c06ae40 0x7fda1c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:26.463 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.458+0000 7fda34ff9640 1 -- 172.21.15.169:0/2716639947 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fda200ac2d0 con 0x7fda380aaf20 2023-10-26T01:36:26.463 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.459+0000 7fda377fe640 1 --2- 172.21.15.169:0/2716639947 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fda1c06ae40 0x7fda1c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:26.464 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.460+0000 7fda377fe640 1 --2- 172.21.15.169:0/2716639947 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fda1c06ae40 0x7fda1c06d300 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fda28004ea0 tx=0x7fda28009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:26.465 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.462+0000 7fda34ff9640 1 -- 172.21.15.169:0/2716639947 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fda20077570 con 0x7fda380aaf20 2023-10-26T01:36:26.693 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.690+0000 7fda3dc6f640 1 -- 172.21.15.169:0/2716639947 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fda380af610 con 0x7fda380aaf20 2023-10-26T01:36:26.693 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.691+0000 7fda34ff9640 1 -- 172.21.15.169:0/2716639947 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+4927 (secure 0 0 0) 0x7fda20003e60 con 0x7fda380aaf20 2023-10-26T01:36:26.694 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:36:26.694 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":42,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5347,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/2629782429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2629782429},{"type":"v1","addr":"172.21.15.170:6836","nonce":2629782429}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":6474,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2561200417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2561200417},{"type":"v1","addr":"172.21.15.169:6833","nonce":2561200417}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":6480,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/2834878721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":2834878721},{"type":"v1","addr":"172.21.15.170:6837","nonce":2834878721}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T01:35:27.834145+0000","modified":"2023-10-26T01:35:30.917014+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6486},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6486":{"gid":6486,"name":"a","rank":0,"incarnation":38,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6835/4004306080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":4004306080},{"type":"v1","addr":"172.21.15.169:6835","nonce":4004306080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-10-26T01:36:26.694 INFO:teuthology.orchestra.run.smithi169.stderr:dumped fsmap epoch 42 2023-10-26T01:36:26.696 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.693+0000 7fda3dc6f640 1 -- 172.21.15.169:0/2716639947 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fda1c06ae40 msgr2=0x7fda1c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:26.696 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.693+0000 7fda3dc6f640 1 --2- 172.21.15.169:0/2716639947 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fda1c06ae40 0x7fda1c06d300 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fda28004ea0 tx=0x7fda28009040 comp rx=0 tx=0).stop 2023-10-26T01:36:26.696 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.693+0000 7fda3dc6f640 1 -- 172.21.15.169:0/2716639947 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda380aaf20 msgr2=0x7fda380cacf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:26.697 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.693+0000 7fda3dc6f640 1 --2- 172.21.15.169:0/2716639947 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda380aaf20 0x7fda380cacf0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fda20006010 tx=0x7fda2002dd60 comp rx=0 tx=0).stop 2023-10-26T01:36:26.697 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.693+0000 7fda3dc6f640 1 -- 172.21.15.169:0/2716639947 shutdown_connections 2023-10-26T01:36:26.697 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.693+0000 7fda3dc6f640 1 --2- 172.21.15.169:0/2716639947 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fda1c06ae40 0x7fda1c06d300 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:26.697 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.693+0000 7fda3dc6f640 1 --2- 172.21.15.169:0/2716639947 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda380aaf20 0x7fda380cacf0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:26.697 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.693+0000 7fda3dc6f640 1 --2- 172.21.15.169:0/2716639947 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fda380b1ff0 0x7fda38153d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:26.698 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.693+0000 7fda3dc6f640 1 --2- 172.21.15.169:0/2716639947 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fda3806cd20 0x7fda38155290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:26.698 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.693+0000 7fda3dc6f640 1 -- 172.21.15.169:0/2716639947 >> 172.21.15.169:0/2716639947 conn(0x7fda380aeee0 msgr2=0x7fda3806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:26.698 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.693+0000 7fda3dc6f640 1 -- 172.21.15.169:0/2716639947 shutdown_connections 2023-10-26T01:36:26.698 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.693+0000 7fda3dc6f640 1 -- 172.21.15.169:0/2716639947 wait complete. 2023-10-26T01:36:26.714 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:36:26.939 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.936+0000 7f0f485e2640 1 -- 172.21.15.169:0/786726587 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0f400cde30 msgr2=0x7f0f400cc440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:26.939 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.936+0000 7f0f485e2640 1 --2- 172.21.15.169:0/786726587 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0f400cde30 0x7f0f400cc440 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f0f38009ca0 tx=0x7f0f3802c360 comp rx=0 tx=0).stop 2023-10-26T01:36:26.939 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.936+0000 7f0f485e2640 1 -- 172.21.15.169:0/786726587 shutdown_connections 2023-10-26T01:36:26.940 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.936+0000 7f0f485e2640 1 --2- 172.21.15.169:0/786726587 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0f400cde30 0x7f0f400cc440 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:26.940 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.936+0000 7f0f485e2640 1 -- 172.21.15.169:0/786726587 >> 172.21.15.169:0/786726587 conn(0x7f0f40069930 msgr2=0x7f0f40069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:26.940 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.936+0000 7f0f485e2640 1 -- 172.21.15.169:0/786726587 shutdown_connections 2023-10-26T01:36:26.940 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.936+0000 7f0f485e2640 1 -- 172.21.15.169:0/786726587 wait complete. 2023-10-26T01:36:26.941 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.938+0000 7f0f485e2640 1 Processor -- start 2023-10-26T01:36:26.941 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.938+0000 7f0f485e2640 1 -- start start 2023-10-26T01:36:26.943 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.939+0000 7f0f485e2640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0f400cde30 0x7f0f40071630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:26.943 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.939+0000 7f0f485e2640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0f4014f2c0 0x7f0f4015a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:26.944 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.939+0000 7f0f485e2640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0f4014f6a0 0x7f0f40144ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:26.944 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.939+0000 7f0f485e2640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f40159fc0 con 0x7f0f4014f6a0 2023-10-26T01:36:26.944 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.939+0000 7f0f485e2640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f4006b890 con 0x7f0f400cde30 2023-10-26T01:36:26.944 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.939+0000 7f0f485e2640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f40145410 con 0x7f0f4014f2c0 2023-10-26T01:36:26.944 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.939+0000 7f0f46b7f640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0f4014f6a0 0x7f0f40144ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:26.945 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.939+0000 7f0f46b7f640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0f4014f6a0 0x7f0f40144ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:51996/0 (socket says 172.21.15.169:51996) 2023-10-26T01:36:26.945 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.939+0000 7f0f46b7f640 1 -- 172.21.15.169:0/3636721939 learned_addr learned my addr 172.21.15.169:0/3636721939 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:36:26.945 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.940+0000 7f0f46b7f640 1 -- 172.21.15.169:0/3636721939 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0f4014f2c0 msgr2=0x7f0f4015a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:26.945 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.940+0000 7f0f46b7f640 1 --2- 172.21.15.169:0/3636721939 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0f4014f2c0 0x7f0f4015a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:26.946 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.940+0000 7f0f46b7f640 1 -- 172.21.15.169:0/3636721939 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0f400cde30 msgr2=0x7f0f40071630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:26.946 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.940+0000 7f0f46b7f640 1 --2- 172.21.15.169:0/3636721939 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0f400cde30 0x7f0f40071630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:26.946 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.940+0000 7f0f46b7f640 1 -- 172.21.15.169:0/3636721939 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f38009900 con 0x7f0f4014f6a0 2023-10-26T01:36:26.946 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.940+0000 7f0f46b7f640 1 --2- 172.21.15.169:0/3636721939 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0f4014f6a0 0x7f0f40144ed0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f0f38006010 tx=0x7f0f38035b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:26.946 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.940+0000 7f0f36ffd640 1 -- 172.21.15.169:0/3636721939 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f38004240 con 0x7f0f4014f6a0 2023-10-26T01:36:26.947 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.940+0000 7f0f36ffd640 1 -- 172.21.15.169:0/3636721939 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0f38004490 con 0x7f0f4014f6a0 2023-10-26T01:36:26.947 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.940+0000 7f0f36ffd640 1 -- 172.21.15.169:0/3636721939 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f38004820 con 0x7f0f4014f6a0 2023-10-26T01:36:26.947 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.940+0000 7f0f485e2640 1 -- 172.21.15.169:0/3636721939 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f4006b2d0 con 0x7f0f4014f6a0 2023-10-26T01:36:26.947 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.940+0000 7f0f485e2640 1 -- 172.21.15.169:0/3636721939 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f40159c50 con 0x7f0f4014f6a0 2023-10-26T01:36:26.948 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.941+0000 7f0f485e2640 1 -- 172.21.15.169:0/3636721939 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f40159de0 con 0x7f0f4014f6a0 2023-10-26T01:36:26.949 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.942+0000 7f0f36ffd640 1 -- 172.21.15.169:0/3636721939 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f0f3803b040 con 0x7f0f4014f6a0 2023-10-26T01:36:26.949 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.943+0000 7f0f36ffd640 1 --2- 172.21.15.169:0/3636721939 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f0f2406ae40 0x7f0f2406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:26.949 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.943+0000 7f0f36ffd640 1 -- 172.21.15.169:0/3636721939 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0f380ac350 con 0x7f0f4014f6a0 2023-10-26T01:36:26.949 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.944+0000 7f0f4637e640 1 --2- 172.21.15.169:0/3636721939 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f0f2406ae40 0x7f0f2406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:26.950 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.945+0000 7f0f4637e640 1 --2- 172.21.15.169:0/3636721939 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f0f2406ae40 0x7f0f2406d300 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c000f60 tx=0x7f0f3c0079b0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:26.950 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:26.948+0000 7f0f36ffd640 1 -- 172.21.15.169:0/3636721939 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0f380775b0 con 0x7f0f4014f6a0 2023-10-26T01:36:27.143 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:27.140+0000 7f0f485e2640 1 -- 172.21.15.169:0/3636721939 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0f40152dc0 con 0x7f0f4014f6a0 2023-10-26T01:36:27.145 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:27.142+0000 7f0f36ffd640 1 -- 172.21.15.169:0/3636721939 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+16616 (secure 0 0 0) 0x7f0f3807abd0 con 0x7f0f4014f6a0 2023-10-26T01:36:27.146 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:36:27.146 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":90,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:36:25.092474+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-10-26T01:35:26.320950+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-10-26T01:35:26.855916+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:36:27.148 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:27.145+0000 7f0f485e2640 1 -- 172.21.15.169:0/3636721939 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f0f2406ae40 msgr2=0x7f0f2406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:27.148 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:27.145+0000 7f0f485e2640 1 --2- 172.21.15.169:0/3636721939 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f0f2406ae40 0x7f0f2406d300 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c000f60 tx=0x7f0f3c0079b0 comp rx=0 tx=0).stop 2023-10-26T01:36:27.148 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:27.145+0000 7f0f485e2640 1 -- 172.21.15.169:0/3636721939 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0f4014f6a0 msgr2=0x7f0f40144ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:27.148 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:27.145+0000 7f0f485e2640 1 --2- 172.21.15.169:0/3636721939 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0f4014f6a0 0x7f0f40144ed0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f0f38006010 tx=0x7f0f38035b50 comp rx=0 tx=0).stop 2023-10-26T01:36:27.149 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:27.145+0000 7f0f485e2640 1 -- 172.21.15.169:0/3636721939 shutdown_connections 2023-10-26T01:36:27.149 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:27.145+0000 7f0f485e2640 1 --2- 172.21.15.169:0/3636721939 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f0f2406ae40 0x7f0f2406d300 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:27.149 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:27.145+0000 7f0f485e2640 1 --2- 172.21.15.169:0/3636721939 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0f4014f6a0 0x7f0f40144ed0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:27.149 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:27.145+0000 7f0f485e2640 1 --2- 172.21.15.169:0/3636721939 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0f4014f2c0 0x7f0f4015a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:27.149 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:27.145+0000 7f0f485e2640 1 --2- 172.21.15.169:0/3636721939 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0f400cde30 0x7f0f40071630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:27.150 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:27.145+0000 7f0f485e2640 1 -- 172.21.15.169:0/3636721939 >> 172.21.15.169:0/3636721939 conn(0x7f0f40069930 msgr2=0x7f0f4014e940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:27.150 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:27.145+0000 7f0f485e2640 1 -- 172.21.15.169:0/3636721939 shutdown_connections 2023-10-26T01:36:27.150 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:27.145+0000 7f0f485e2640 1 -- 172.21.15.169:0/3636721939 wait complete. 2023-10-26T01:36:27.165 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-26T01:36:27.166 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T01:36:27.370 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:27.367+0000 7febb68d7640 1 -- 172.21.15.169:0/3561063672 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7febb006cae0 msgr2=0x7febb00c7ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:27.371 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:27.367+0000 7febb68d7640 1 --2- 172.21.15.169:0/3561063672 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7febb006cae0 0x7febb00c7ea0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7febac009830 tx=0x7febac02bcc0 comp rx=0 tx=0).stop 2023-10-26T01:36:27.371 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:27.367+0000 7febb68d7640 1 -- 172.21.15.169:0/3561063672 shutdown_connections 2023-10-26T01:36:27.371 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:27.367+0000 7febb68d7640 1 --2- 172.21.15.169:0/3561063672 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7febb006a990 0x7febb00c96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:27.371 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:27.367+0000 7febb68d7640 1 --2- 172.21.15.169:0/3561063672 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7febb006cae0 0x7febb00c7ea0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:27.372 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:27.367+0000 7febb68d7640 1 -- 172.21.15.169:0/3561063672 >> 172.21.15.169:0/3561063672 conn(0x7febb00aedd0 msgr2=0x7febb00af1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:27.372 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:27.368+0000 7febb68d7640 1 -- 172.21.15.169:0/3561063672 shutdown_connections 2023-10-26T01:36:27.372 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:27.368+0000 7febb68d7640 1 -- 172.21.15.169:0/3561063672 wait complete. 2023-10-26T01:36:27.373 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:27.369+0000 7febb68d7640 1 Processor -- start 2023-10-26T01:36:27.373 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:27.369+0000 7febb68d7640 1 -- start start 2023-10-26T01:36:27.373 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:27.370+0000 7febb68d7640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7febb006a990 0x7febb01f0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:27.374 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:27.370+0000 7febb68d7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7febb006cae0 0x7febb01f0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:27.374 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:27.370+0000 7febb68d7640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7febb00c9c30 0x7febb0071240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:27.374 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:27.370+0000 7febb68d7640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febb00aaec0 con 0x7febb00c9c30 2023-10-26T01:36:27.374 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:27.370+0000 7febb68d7640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febb006e8f0 con 0x7febb006cae0 2023-10-26T01:36:27.374 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:27.370+0000 7febb68d7640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febb00a9ff0 con 0x7febb006a990 2023-10-26T01:36:27.375 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:27.371+0000 7febb60d6640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7febb00c9c30 0x7febb0071240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:27.375 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:27.371+0000 7febb60d6640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7febb00c9c30 0x7febb0071240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:52004/0 (socket says 172.21.15.169:52004) 2023-10-26T01:36:27.375 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:27.371+0000 7febb60d6640 1 -- 172.21.15.169:0/1253425392 learned_addr learned my addr 172.21.15.169:0/1253425392 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:36:27.376 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:27.371+0000 7febb58d5640 1 --2- 172.21.15.169:0/1253425392 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7febb006a990 0x7febb01f0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:27.376 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:27.371+0000 7febb60d6640 1 -- 172.21.15.169:0/1253425392 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7febb006a990 msgr2=0x7febb01f0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:27.377 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:27.371+0000 7febb60d6640 1 --2- 172.21.15.169:0/1253425392 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7febb006a990 0x7febb01f0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:27.377 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:27.371+0000 7febb60d6640 1 -- 172.21.15.169:0/1253425392 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7febb006cae0 msgr2=0x7febb01f0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:27.377 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:27.371+0000 7febb50d4640 1 --2- 172.21.15.169:0/1253425392 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7febb006cae0 0x7febb01f0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:27.377 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:27.371+0000 7febb60d6640 1 --2- 172.21.15.169:0/1253425392 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7febb006cae0 0x7febb01f0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:27.377 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:27.371+0000 7febb60d6640 1 -- 172.21.15.169:0/1253425392 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febac009490 con 0x7febb00c9c30 2023-10-26T01:36:27.378 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:27.371+0000 7febb60d6640 1 --2- 172.21.15.169:0/1253425392 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7febb00c9c30 0x7febb0071240 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7feba800caa0 tx=0x7feba800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:27.378 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:27.372+0000 7feba67fc640 1 -- 172.21.15.169:0/1253425392 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba800ac00 con 0x7febb00c9c30 2023-10-26T01:36:27.378 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:27.372+0000 7feba67fc640 1 -- 172.21.15.169:0/1253425392 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7feba800ae70 con 0x7febb00c9c30 2023-10-26T01:36:27.378 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:27.372+0000 7feba67fc640 1 -- 172.21.15.169:0/1253425392 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba8012660 con 0x7febb00c9c30 2023-10-26T01:36:27.379 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:27.372+0000 7febb68d7640 1 -- 172.21.15.169:0/1253425392 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febb01f0b50 con 0x7febb00c9c30 2023-10-26T01:36:27.379 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:27.372+0000 7febb68d7640 1 -- 172.21.15.169:0/1253425392 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febb00aa520 con 0x7febb00c9c30 2023-10-26T01:36:27.380 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:27.373+0000 7febb68d7640 1 -- 172.21.15.169:0/1253425392 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febb00ab000 con 0x7febb00c9c30 2023-10-26T01:36:27.380 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:27.374+0000 7feba67fc640 1 -- 172.21.15.169:0/1253425392 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7feba801a3b0 con 0x7febb00c9c30 2023-10-26T01:36:27.381 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:27.375+0000 7feba67fc640 1 --2- 172.21.15.169:0/1253425392 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7feb9406ae40 0x7feb9406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:27.381 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:27.375+0000 7feba67fc640 1 -- 172.21.15.169:0/1253425392 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5709+0+0 (secure 0 0 0) 0x7feba8089b20 con 0x7febb00c9c30 2023-10-26T01:36:27.381 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:27.376+0000 7febb58d5640 1 --2- 172.21.15.169:0/1253425392 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7feb9406ae40 0x7feb9406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:27.381 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:27.377+0000 7febb58d5640 1 --2- 172.21.15.169:0/1253425392 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7feb9406ae40 0x7feb9406d300 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7febac036000 tx=0x7febac004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:27.382 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:27.380+0000 7feba67fc640 1 -- 172.21.15.169:0/1253425392 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7febb00ab000 con 0x7febb00c9c30 2023-10-26T01:36:27.614 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:27.611+0000 7febb68d7640 1 -- 172.21.15.169:0/1253425392 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7febb006ae10 con 0x7febb00c9c30 2023-10-26T01:36:27.614 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:27.612+0000 7feba67fc640 1 -- 172.21.15.169:0/1253425392 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+4927 (secure 0 0 0) 0x7febb006ae10 con 0x7febb00c9c30 2023-10-26T01:36:27.615 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:36:27.615 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":42,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5347,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/2629782429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2629782429},{"type":"v1","addr":"172.21.15.170:6836","nonce":2629782429}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":6474,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2561200417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2561200417},{"type":"v1","addr":"172.21.15.169:6833","nonce":2561200417}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":6480,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/2834878721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":2834878721},{"type":"v1","addr":"172.21.15.170:6837","nonce":2834878721}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T01:35:27.834145+0000","modified":"2023-10-26T01:35:30.917014+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6486},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6486":{"gid":6486,"name":"a","rank":0,"incarnation":38,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6835/4004306080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":4004306080},{"type":"v1","addr":"172.21.15.169:6835","nonce":4004306080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-10-26T01:36:27.615 INFO:teuthology.orchestra.run.smithi169.stderr:dumped fsmap epoch 42 2023-10-26T01:36:27.617 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:27.614+0000 7febb68d7640 1 -- 172.21.15.169:0/1253425392 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7feb9406ae40 msgr2=0x7feb9406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:27.618 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:27.614+0000 7febb68d7640 1 --2- 172.21.15.169:0/1253425392 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7feb9406ae40 0x7feb9406d300 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7febac036000 tx=0x7febac004730 comp rx=0 tx=0).stop 2023-10-26T01:36:27.618 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:27.614+0000 7febb68d7640 1 -- 172.21.15.169:0/1253425392 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7febb00c9c30 msgr2=0x7febb0071240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:27.618 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:27.614+0000 7febb68d7640 1 --2- 172.21.15.169:0/1253425392 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7febb00c9c30 0x7febb0071240 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7feba800caa0 tx=0x7feba800cf60 comp rx=0 tx=0).stop 2023-10-26T01:36:27.618 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:27.615+0000 7febb68d7640 1 -- 172.21.15.169:0/1253425392 shutdown_connections 2023-10-26T01:36:27.619 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:27.615+0000 7febb68d7640 1 --2- 172.21.15.169:0/1253425392 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7feb9406ae40 0x7feb9406d300 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:27.619 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:27.615+0000 7febb68d7640 1 --2- 172.21.15.169:0/1253425392 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7febb00c9c30 0x7febb0071240 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:27.619 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:27.615+0000 7febb68d7640 1 --2- 172.21.15.169:0/1253425392 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7febb006cae0 0x7febb01f0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:27.619 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:27.615+0000 7febb68d7640 1 --2- 172.21.15.169:0/1253425392 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7febb006a990 0x7febb01f0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:27.619 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:27.615+0000 7febb68d7640 1 -- 172.21.15.169:0/1253425392 >> 172.21.15.169:0/1253425392 conn(0x7febb00aedd0 msgr2=0x7febb0154dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:27.619 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:27.615+0000 7febb68d7640 1 -- 172.21.15.169:0/1253425392 shutdown_connections 2023-10-26T01:36:27.620 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:27.615+0000 7febb68d7640 1 -- 172.21.15.169:0/1253425392 wait complete. 2023-10-26T01:36:27.635 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T01:36:27.856 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:27.853+0000 7fe697aa9640 1 -- 172.21.15.169:0/3056215146 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe67c0047c0 msgr2=0x7fe67c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:27.856 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:27.853+0000 7fe697aa9640 1 --2- 172.21.15.169:0/3056215146 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe67c0047c0 0x7fe67c004c40 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fe680001680 tx=0x7fe67c0057f0 comp rx=0 tx=0).stop 2023-10-26T01:36:27.856 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:27.853+0000 7fe697aa9640 1 -- 172.21.15.169:0/3056215146 shutdown_connections 2023-10-26T01:36:27.856 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:27.853+0000 7fe697aa9640 1 --2- 172.21.15.169:0/3056215146 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe67c0047c0 0x7fe67c004c40 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:27.857 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:27.853+0000 7fe697aa9640 1 -- 172.21.15.169:0/3056215146 >> 172.21.15.169:0/3056215146 conn(0x7fe6900d1600 msgr2=0x7fe6900d19e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:27.857 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:27.853+0000 7fe697aa9640 1 -- 172.21.15.169:0/3056215146 shutdown_connections 2023-10-26T01:36:27.857 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:27.853+0000 7fe697aa9640 1 -- 172.21.15.169:0/3056215146 wait complete. 2023-10-26T01:36:27.858 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:27.855+0000 7fe697aa9640 1 Processor -- start 2023-10-26T01:36:27.858 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:27.855+0000 7fe697aa9640 1 -- start start 2023-10-26T01:36:27.860 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:27.856+0000 7fe697aa9640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe67c0047c0 0x7fe6901e49e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:27.860 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:27.856+0000 7fe697aa9640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe6901f4970 0x7fe6900b1060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:27.860 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:27.856+0000 7fe697aa9640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe6901f4df0 0x7fe6900b3210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:27.860 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:27.856+0000 7fe697aa9640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6900c6330 con 0x7fe6901f4970 2023-10-26T01:36:27.860 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:27.856+0000 7fe697aa9640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6900b1de0 con 0x7fe67c0047c0 2023-10-26T01:36:27.861 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:27.856+0000 7fe697aa9640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6900b1b00 con 0x7fe6901f4df0 2023-10-26T01:36:27.861 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:27.856+0000 7fe695044640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe6901f4970 0x7fe6900b1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:27.861 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:27.856+0000 7fe695044640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe6901f4970 0x7fe6900b1060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:52018/0 (socket says 172.21.15.169:52018) 2023-10-26T01:36:27.861 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:27.856+0000 7fe695044640 1 -- 172.21.15.169:0/1196231255 learned_addr learned my addr 172.21.15.169:0/1196231255 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:36:27.861 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:27.856+0000 7fe695044640 1 -- 172.21.15.169:0/1196231255 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe6901f4df0 msgr2=0x7fe6900b3210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:27.862 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:27.856+0000 7fe695845640 1 --2- 172.21.15.169:0/1196231255 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe67c0047c0 0x7fe6901e49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:27.862 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:27.856+0000 7fe695044640 1 --2- 172.21.15.169:0/1196231255 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe6901f4df0 0x7fe6900b3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:27.862 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:27.856+0000 7fe695044640 1 -- 172.21.15.169:0/1196231255 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe67c0047c0 msgr2=0x7fe6901e49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:27.862 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:27.856+0000 7fe695044640 1 --2- 172.21.15.169:0/1196231255 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe67c0047c0 0x7fe6901e49e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:27.862 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:27.856+0000 7fe695044640 1 -- 172.21.15.169:0/1196231255 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6900c6510 con 0x7fe6901f4970 2023-10-26T01:36:27.863 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:27.857+0000 7fe695044640 1 --2- 172.21.15.169:0/1196231255 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe6901f4970 0x7fe6900b1060 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fe6800017a0 tx=0x7fe680006310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:27.863 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:27.857+0000 7fe6867fc640 1 -- 172.21.15.169:0/1196231255 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe680003b80 con 0x7fe6901f4970 2023-10-26T01:36:27.863 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:27.857+0000 7fe6867fc640 1 -- 172.21.15.169:0/1196231255 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe680003df0 con 0x7fe6901f4970 2023-10-26T01:36:27.863 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:27.857+0000 7fe6867fc640 1 -- 172.21.15.169:0/1196231255 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe68002c7b0 con 0x7fe6901f4970 2023-10-26T01:36:27.863 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:27.857+0000 7fe697aa9640 1 -- 172.21.15.169:0/1196231255 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6900c67f0 con 0x7fe6901f4970 2023-10-26T01:36:27.864 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:27.857+0000 7fe697aa9640 1 -- 172.21.15.169:0/1196231255 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6901d88f0 con 0x7fe6901f4970 2023-10-26T01:36:27.864 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:27.858+0000 7fe697aa9640 1 -- 172.21.15.169:0/1196231255 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6900c6330 con 0x7fe6901f4970 2023-10-26T01:36:27.865 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:27.859+0000 7fe6867fc640 1 -- 172.21.15.169:0/1196231255 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fe68002c910 con 0x7fe6901f4970 2023-10-26T01:36:27.865 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:27.861+0000 7fe6867fc640 1 --2- 172.21.15.169:0/1196231255 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe67406ae40 0x7fe67406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:27.865 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:27.861+0000 7fe6867fc640 1 -- 172.21.15.169:0/1196231255 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe6800aa860 con 0x7fe6901f4970 2023-10-26T01:36:27.865 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:27.861+0000 7fe695845640 1 --2- 172.21.15.169:0/1196231255 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe67406ae40 0x7fe67406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:27.866 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:27.862+0000 7fe695845640 1 --2- 172.21.15.169:0/1196231255 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe67406ae40 0x7fe67406d300 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fe690067350 tx=0x7fe68c005f90 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:27.868 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:27.865+0000 7fe6867fc640 1 -- 172.21.15.169:0/1196231255 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe6800759b0 con 0x7fe6901f4970 2023-10-26T01:36:28.095 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:28.092+0000 7fe697aa9640 1 -- 172.21.15.169:0/1196231255 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe69006adf0 con 0x7fe6901f4970 2023-10-26T01:36:28.096 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:28.094+0000 7fe6867fc640 1 -- 172.21.15.169:0/1196231255 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+4927 (secure 0 0 0) 0x7fe680044020 con 0x7fe6901f4970 2023-10-26T01:36:28.097 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:36:28.097 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":42,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5347,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/2629782429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2629782429},{"type":"v1","addr":"172.21.15.170:6836","nonce":2629782429}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":6474,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2561200417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2561200417},{"type":"v1","addr":"172.21.15.169:6833","nonce":2561200417}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":6480,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/2834878721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":2834878721},{"type":"v1","addr":"172.21.15.170:6837","nonce":2834878721}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T01:35:27.834145+0000","modified":"2023-10-26T01:35:30.917014+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6486},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6486":{"gid":6486,"name":"a","rank":0,"incarnation":38,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6835/4004306080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":4004306080},{"type":"v1","addr":"172.21.15.169:6835","nonce":4004306080}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-10-26T01:36:28.097 INFO:teuthology.orchestra.run.smithi169.stderr:dumped fsmap epoch 42 2023-10-26T01:36:28.100 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:28.096+0000 7fe697aa9640 1 -- 172.21.15.169:0/1196231255 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe67406ae40 msgr2=0x7fe67406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:28.100 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:28.096+0000 7fe697aa9640 1 --2- 172.21.15.169:0/1196231255 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe67406ae40 0x7fe67406d300 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fe690067350 tx=0x7fe68c005f90 comp rx=0 tx=0).stop 2023-10-26T01:36:28.100 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:28.096+0000 7fe697aa9640 1 -- 172.21.15.169:0/1196231255 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe6901f4970 msgr2=0x7fe6900b1060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:28.100 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:28.096+0000 7fe697aa9640 1 --2- 172.21.15.169:0/1196231255 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe6901f4970 0x7fe6900b1060 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fe6800017a0 tx=0x7fe680006310 comp rx=0 tx=0).stop 2023-10-26T01:36:28.100 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:28.097+0000 7fe697aa9640 1 -- 172.21.15.169:0/1196231255 shutdown_connections 2023-10-26T01:36:28.101 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:28.097+0000 7fe697aa9640 1 --2- 172.21.15.169:0/1196231255 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe67406ae40 0x7fe67406d300 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:28.101 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:28.097+0000 7fe697aa9640 1 --2- 172.21.15.169:0/1196231255 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe6901f4970 0x7fe6900b1060 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:28.101 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:28.097+0000 7fe697aa9640 1 --2- 172.21.15.169:0/1196231255 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe6901f4df0 0x7fe6900b3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:28.101 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:28.097+0000 7fe697aa9640 1 --2- 172.21.15.169:0/1196231255 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe67c0047c0 0x7fe6901e49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:28.101 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:28.097+0000 7fe697aa9640 1 -- 172.21.15.169:0/1196231255 >> 172.21.15.169:0/1196231255 conn(0x7fe6900d1600 msgr2=0x7fe6901e3670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:28.102 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:28.097+0000 7fe697aa9640 1 -- 172.21.15.169:0/1196231255 shutdown_connections 2023-10-26T01:36:28.102 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:28.097+0000 7fe697aa9640 1 -- 172.21.15.169:0/1196231255 wait complete. 2023-10-26T01:36:28.119 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:36:28.329 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:28.326+0000 7f4903a10640 1 -- 172.21.15.169:0/1908418105 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f48e80047c0 msgr2=0x7f48e8004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:28.330 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:28.326+0000 7f4903a10640 1 --2- 172.21.15.169:0/1908418105 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f48e80047c0 0x7f48e8004c40 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f48ec006fd0 tx=0x7f48e80057f0 comp rx=0 tx=0).stop 2023-10-26T01:36:28.330 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:28.326+0000 7f4903a10640 1 -- 172.21.15.169:0/1908418105 shutdown_connections 2023-10-26T01:36:28.330 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:28.326+0000 7f4903a10640 1 --2- 172.21.15.169:0/1908418105 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f48e80047c0 0x7f48e8004c40 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:28.330 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:28.326+0000 7f4903a10640 1 -- 172.21.15.169:0/1908418105 >> 172.21.15.169:0/1908418105 conn(0x7f48fc098a60 msgr2=0x7f48fc0b4970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:28.330 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:28.326+0000 7f4903a10640 1 -- 172.21.15.169:0/1908418105 shutdown_connections 2023-10-26T01:36:28.331 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:28.327+0000 7f4903a10640 1 -- 172.21.15.169:0/1908418105 wait complete. 2023-10-26T01:36:28.331 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:28.328+0000 7f4903a10640 1 Processor -- start 2023-10-26T01:36:28.331 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:28.328+0000 7f4903a10640 1 -- start start 2023-10-26T01:36:28.333 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:28.329+0000 7f4903a10640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f48e80047c0 0x7f48fc146040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:28.333 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:28.329+0000 7f4903a10640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f48fc15df40 0x7f48fc15e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:28.334 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:28.329+0000 7f4903a10640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48fc0c87d0 0x7f48fc0c8c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:28.334 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:28.329+0000 7f4903a10640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48fc15e900 con 0x7f48e80047c0 2023-10-26T01:36:28.334 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:28.329+0000 7f4903a10640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48fc0c9190 con 0x7f48fc0c87d0 2023-10-26T01:36:28.334 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:28.329+0000 7f4903a10640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48fc0cb5c0 con 0x7f48fc15df40 2023-10-26T01:36:28.334 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:28.329+0000 7f49017ac640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f48e80047c0 0x7f48fc146040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:28.335 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:28.330+0000 7f49017ac640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f48e80047c0 0x7f48fc146040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:52024/0 (socket says 172.21.15.169:52024) 2023-10-26T01:36:28.335 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:28.330+0000 7f49017ac640 1 -- 172.21.15.169:0/3961127623 learned_addr learned my addr 172.21.15.169:0/3961127623 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:36:28.335 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:28.330+0000 7f4900fab640 1 --2- 172.21.15.169:0/3961127623 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f48fc15df40 0x7f48fc15e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:28.335 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:28.330+0000 7f49017ac640 1 -- 172.21.15.169:0/3961127623 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f48fc15df40 msgr2=0x7f48fc15e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:28.335 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:28.330+0000 7f49017ac640 1 --2- 172.21.15.169:0/3961127623 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f48fc15df40 0x7f48fc15e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:28.336 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:28.330+0000 7f49017ac640 1 -- 172.21.15.169:0/3961127623 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48fc0c87d0 msgr2=0x7f48fc0c8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:28.336 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:28.330+0000 7f4901fad640 1 --2- 172.21.15.169:0/3961127623 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48fc0c87d0 0x7f48fc0c8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:28.336 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:28.330+0000 7f49017ac640 1 --2- 172.21.15.169:0/3961127623 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48fc0c87d0 0x7f48fc0c8c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:28.336 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:28.330+0000 7f49017ac640 1 -- 172.21.15.169:0/3961127623 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48fc0c93b0 con 0x7f48e80047c0 2023-10-26T01:36:28.336 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:28.330+0000 7f4901fad640 1 --2- 172.21.15.169:0/3961127623 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48fc0c87d0 0x7f48fc0c8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:36:28.337 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:28.330+0000 7f49017ac640 1 --2- 172.21.15.169:0/3961127623 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f48e80047c0 0x7f48fc146040 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f48f800e9c0 tx=0x7f48f800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:28.337 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:28.331+0000 7f48f1ffb640 1 -- 172.21.15.169:0/3961127623 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f800cc60 con 0x7f48e80047c0 2023-10-26T01:36:28.337 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:28.331+0000 7f48f1ffb640 1 -- 172.21.15.169:0/3961127623 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f48f800ceb0 con 0x7f48e80047c0 2023-10-26T01:36:28.337 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:28.331+0000 7f4903a10640 1 -- 172.21.15.169:0/3961127623 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48fc0c9690 con 0x7f48e80047c0 2023-10-26T01:36:28.337 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:28.331+0000 7f48f1ffb640 1 -- 172.21.15.169:0/3961127623 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f8005260 con 0x7f48e80047c0 2023-10-26T01:36:28.337 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:28.331+0000 7f4903a10640 1 -- 172.21.15.169:0/3961127623 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48fc0c9c50 con 0x7f48e80047c0 2023-10-26T01:36:28.338 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:28.332+0000 7f4903a10640 1 -- 172.21.15.169:0/3961127623 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48fc04db10 con 0x7f48e80047c0 2023-10-26T01:36:28.339 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:28.333+0000 7f48f1ffb640 1 -- 172.21.15.169:0/3961127623 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f48f80053c0 con 0x7f48e80047c0 2023-10-26T01:36:28.339 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:28.334+0000 7f48f1ffb640 1 --2- 172.21.15.169:0/3961127623 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f48e006ae40 0x7f48e006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:28.339 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:28.334+0000 7f48f1ffb640 1 -- 172.21.15.169:0/3961127623 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f48f8089eb0 con 0x7f48e80047c0 2023-10-26T01:36:28.339 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:28.335+0000 7f4900fab640 1 --2- 172.21.15.169:0/3961127623 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f48e006ae40 0x7f48e006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:28.339 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:28.336+0000 7f4900fab640 1 --2- 172.21.15.169:0/3961127623 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f48e006ae40 0x7f48e006d300 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f48ec034000 tx=0x7f48ec02e000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:28.341 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:28.339+0000 7f48f1ffb640 1 -- 172.21.15.169:0/3961127623 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f48f8054ff0 con 0x7f48e80047c0 2023-10-26T01:36:28.538 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:28.535+0000 7f4903a10640 1 -- 172.21.15.169:0/3961127623 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f48fc0cba50 con 0x7f48e80047c0 2023-10-26T01:36:28.539 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:28.537+0000 7f48f1ffb640 1 -- 172.21.15.169:0/3961127623 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+16616 (secure 0 0 0) 0x7f48f801f020 con 0x7f48e80047c0 2023-10-26T01:36:28.540 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:36:28.540 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":90,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:36:25.092474+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-10-26T01:35:26.320950+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-10-26T01:35:26.855916+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:36:28.543 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:28.540+0000 7f4903a10640 1 -- 172.21.15.169:0/3961127623 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f48e006ae40 msgr2=0x7f48e006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:28.543 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:28.540+0000 7f4903a10640 1 --2- 172.21.15.169:0/3961127623 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f48e006ae40 0x7f48e006d300 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f48ec034000 tx=0x7f48ec02e000 comp rx=0 tx=0).stop 2023-10-26T01:36:28.543 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:28.540+0000 7f4903a10640 1 -- 172.21.15.169:0/3961127623 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f48e80047c0 msgr2=0x7f48fc146040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:28.544 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:28.540+0000 7f4903a10640 1 --2- 172.21.15.169:0/3961127623 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f48e80047c0 0x7f48fc146040 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f48f800e9c0 tx=0x7f48f800ee80 comp rx=0 tx=0).stop 2023-10-26T01:36:28.544 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:28.540+0000 7f4903a10640 1 -- 172.21.15.169:0/3961127623 shutdown_connections 2023-10-26T01:36:28.544 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:28.540+0000 7f4903a10640 1 --2- 172.21.15.169:0/3961127623 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f48e006ae40 0x7f48e006d300 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:28.544 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:28.540+0000 7f4903a10640 1 --2- 172.21.15.169:0/3961127623 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48fc0c87d0 0x7f48fc0c8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:28.544 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:28.540+0000 7f4903a10640 1 --2- 172.21.15.169:0/3961127623 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f48fc15df40 0x7f48fc15e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:28.545 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:28.540+0000 7f4903a10640 1 --2- 172.21.15.169:0/3961127623 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f48e80047c0 0x7f48fc146040 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:28.545 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:28.540+0000 7f4903a10640 1 -- 172.21.15.169:0/3961127623 >> 172.21.15.169:0/3961127623 conn(0x7f48fc098a60 msgr2=0x7f48fc06afe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:28.545 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:28.540+0000 7f4903a10640 1 -- 172.21.15.169:0/3961127623 shutdown_connections 2023-10-26T01:36:28.545 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:28.540+0000 7f4903a10640 1 -- 172.21.15.169:0/3961127623 wait complete. 2023-10-26T01:36:28.563 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-26T01:36:28.786 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:28.783+0000 7f1c26bce640 1 -- 172.21.15.169:0/3232274673 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c201593e0 msgr2=0x7f1c200ac200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:28.786 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:28.783+0000 7f1c26bce640 1 --2- 172.21.15.169:0/3232274673 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c201593e0 0x7f1c200ac200 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f1c200c9aa0 tx=0x7f1c1c02bdb0 comp rx=0 tx=0).stop 2023-10-26T01:36:28.786 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:28.783+0000 7f1c26bce640 1 -- 172.21.15.169:0/3232274673 shutdown_connections 2023-10-26T01:36:28.787 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:28.783+0000 7f1c26bce640 1 --2- 172.21.15.169:0/3232274673 >> v2:172.21.15.170:3300/0 conn(0x7f1c200b0c80 0x7f1c200cd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:28.787 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:28.783+0000 7f1c26bce640 1 --2- 172.21.15.169:0/3232274673 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c201593e0 0x7f1c200ac200 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:28.787 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:28.783+0000 7f1c26bce640 1 -- 172.21.15.169:0/3232274673 >> 172.21.15.169:0/3232274673 conn(0x7f1c20069930 msgr2=0x7f1c20069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:28.787 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:28.783+0000 7f1c26bce640 1 -- 172.21.15.169:0/3232274673 shutdown_connections 2023-10-26T01:36:28.787 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:28.783+0000 7f1c26bce640 1 -- 172.21.15.169:0/3232274673 wait complete. 2023-10-26T01:36:28.789 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:28.785+0000 7f1c26bce640 1 Processor -- start 2023-10-26T01:36:28.789 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:28.785+0000 7f1c26bce640 1 -- start start 2023-10-26T01:36:28.789 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:28.786+0000 7f1c26bce640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1c200b0c80 0x7f1c201490b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:28.790 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:28.786+0000 7f1c26bce640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c201593e0 0x7f1c200c7d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:28.790 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:28.786+0000 7f1c26bce640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1c201597c0 0x7f1c200c8290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:28.790 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:28.786+0000 7f1c26bce640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c200c7ae0 con 0x7f1c201593e0 2023-10-26T01:36:28.790 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:28.786+0000 7f1c26bce640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c20149b90 con 0x7f1c201597c0 2023-10-26T01:36:28.790 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:28.786+0000 7f1c26bce640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c200c7240 con 0x7f1c200b0c80 2023-10-26T01:36:28.791 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:28.786+0000 7f1c17fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c201593e0 0x7f1c200c7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:28.791 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:28.786+0000 7f1c17fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c201593e0 0x7f1c200c7d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:52044/0 (socket says 172.21.15.169:52044) 2023-10-26T01:36:28.791 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:28.786+0000 7f1c17fff640 1 -- 172.21.15.169:0/3490359965 learned_addr learned my addr 172.21.15.169:0/3490359965 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:36:28.791 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:28.786+0000 7f1c2496a640 1 --2- 172.21.15.169:0/3490359965 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1c200b0c80 0x7f1c201490b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:28.791 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:28.786+0000 7f1c2516b640 1 --2- 172.21.15.169:0/3490359965 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1c201597c0 0x7f1c200c8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:28.792 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:28.786+0000 7f1c17fff640 1 -- 172.21.15.169:0/3490359965 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1c200b0c80 msgr2=0x7f1c201490b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:28.792 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:28.786+0000 7f1c17fff640 1 --2- 172.21.15.169:0/3490359965 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1c200b0c80 0x7f1c201490b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:28.792 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:28.786+0000 7f1c17fff640 1 -- 172.21.15.169:0/3490359965 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1c201597c0 msgr2=0x7f1c200c8290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:28.792 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:28.786+0000 7f1c17fff640 1 --2- 172.21.15.169:0/3490359965 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1c201597c0 0x7f1c200c8290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:28.792 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:28.787+0000 7f1c17fff640 1 -- 172.21.15.169:0/3490359965 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c1c009560 con 0x7f1c201593e0 2023-10-26T01:36:28.793 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:28.787+0000 7f1c2516b640 1 --2- 172.21.15.169:0/3490359965 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1c201597c0 0x7f1c200c8290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:36:28.794 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:28.787+0000 7f1c17fff640 1 --2- 172.21.15.169:0/3490359965 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c201593e0 0x7f1c200c7d50 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f1c1000b3e0 tx=0x7f1c1000b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:28.794 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:28.787+0000 7f1c157fa640 1 -- 172.21.15.169:0/3490359965 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c10011830 con 0x7f1c201593e0 2023-10-26T01:36:28.794 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:28.787+0000 7f1c157fa640 1 -- 172.21.15.169:0/3490359965 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1c10011aa0 con 0x7f1c201593e0 2023-10-26T01:36:28.795 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:28.787+0000 7f1c26bce640 1 -- 172.21.15.169:0/3490359965 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c200c7520 con 0x7f1c201593e0 2023-10-26T01:36:28.795 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:28.787+0000 7f1c157fa640 1 -- 172.21.15.169:0/3490359965 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c10010380 con 0x7f1c201593e0 2023-10-26T01:36:28.795 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:28.787+0000 7f1c26bce640 1 -- 172.21.15.169:0/3490359965 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c2006af30 con 0x7f1c201593e0 2023-10-26T01:36:28.795 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:28.788+0000 7f1c26bce640 1 -- 172.21.15.169:0/3490359965 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c2006b150 con 0x7f1c201593e0 2023-10-26T01:36:28.795 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:28.789+0000 7f1c157fa640 1 -- 172.21.15.169:0/3490359965 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f1c100104e0 con 0x7f1c201593e0 2023-10-26T01:36:28.796 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:28.790+0000 7f1c157fa640 1 --2- 172.21.15.169:0/3490359965 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f1c0c06ae40 0x7f1c0c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:28.796 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:28.790+0000 7f1c157fa640 1 -- 172.21.15.169:0/3490359965 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1c10089a00 con 0x7f1c201593e0 2023-10-26T01:36:28.796 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:28.791+0000 7f1c2496a640 1 --2- 172.21.15.169:0/3490359965 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f1c0c06ae40 0x7f1c0c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:28.797 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:28.792+0000 7f1c2496a640 1 --2- 172.21.15.169:0/3490359965 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f1c0c06ae40 0x7f1c0c06d300 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c035000 tx=0x7f1c1c004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:28.797 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:28.795+0000 7f1c157fa640 1 -- 172.21.15.169:0/3490359965 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1c10054aa0 con 0x7f1c201593e0 2023-10-26T01:36:29.025 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:29.022+0000 7f1c26bce640 1 -- 172.21.15.169:0/3490359965 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f1c2004db10 con 0x7f1c201593e0 2023-10-26T01:36:29.026 INFO:tasks.ceph.mon.a.smithi169.stderr:2023-10-26T01:36:29.023+0000 7f1588a74640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-26T01:36:29.030 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:29.027+0000 7f1c157fa640 1 -- 172.21.15.169:0/3490359965 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v43) v1 ==== 161+0+0 (secure 0 0 0) 0x7f1c10015090 con 0x7f1c201593e0 2023-10-26T01:36:29.030 INFO:teuthology.orchestra.run.smithi169.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-26T01:36:29.037 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:29.033+0000 7f1c26bce640 1 -- 172.21.15.169:0/3490359965 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f1c0c06ae40 msgr2=0x7f1c0c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:29.037 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:29.033+0000 7f1c26bce640 1 --2- 172.21.15.169:0/3490359965 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f1c0c06ae40 0x7f1c0c06d300 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c035000 tx=0x7f1c1c004730 comp rx=0 tx=0).stop 2023-10-26T01:36:29.037 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:29.033+0000 7f1c26bce640 1 -- 172.21.15.169:0/3490359965 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c201593e0 msgr2=0x7f1c200c7d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:29.037 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:29.033+0000 7f1c26bce640 1 --2- 172.21.15.169:0/3490359965 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c201593e0 0x7f1c200c7d50 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f1c1000b3e0 tx=0x7f1c1000b8a0 comp rx=0 tx=0).stop 2023-10-26T01:36:29.038 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:29.034+0000 7f1c26bce640 1 -- 172.21.15.169:0/3490359965 shutdown_connections 2023-10-26T01:36:29.038 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:29.034+0000 7f1c26bce640 1 --2- 172.21.15.169:0/3490359965 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f1c0c06ae40 0x7f1c0c06d300 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:29.038 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:29.034+0000 7f1c26bce640 1 --2- 172.21.15.169:0/3490359965 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c201593e0 0x7f1c200c7d50 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:29.038 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:29.034+0000 7f1c26bce640 1 --2- 172.21.15.169:0/3490359965 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1c201597c0 0x7f1c200c8290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:29.039 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:29.034+0000 7f1c26bce640 1 --2- 172.21.15.169:0/3490359965 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1c200b0c80 0x7f1c201490b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:29.039 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:29.034+0000 7f1c26bce640 1 -- 172.21.15.169:0/3490359965 >> 172.21.15.169:0/3490359965 conn(0x7f1c20069930 msgr2=0x7f1c20154f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:29.039 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:29.034+0000 7f1c26bce640 1 -- 172.21.15.169:0/3490359965 shutdown_connections 2023-10-26T01:36:29.039 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:29.034+0000 7f1c26bce640 1 -- 172.21.15.169:0/3490359965 wait complete. 2023-10-26T01:36:29.055 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-26T01:36:29.145 INFO:tasks.ceph.mds.a.smithi169.stdout:starting mds.a at 2023-10-26T01:36:29.268 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:29.265+0000 7f88f61d0640 1 -- 172.21.15.169:0/1863310198 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f88f0157f80 msgr2=0x7f88f015b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:29.269 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:29.265+0000 7f88f61d0640 1 --2- 172.21.15.169:0/1863310198 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f88f0157f80 0x7f88f015b140 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f88ec005fd0 tx=0x7f88ec02bd90 comp rx=0 tx=0).stop 2023-10-26T01:36:29.269 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:29.265+0000 7f88f61d0640 1 -- 172.21.15.169:0/1863310198 shutdown_connections 2023-10-26T01:36:29.269 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:29.265+0000 7f88f61d0640 1 --2- 172.21.15.169:0/1863310198 >> v2:172.21.15.170:3300/0 conn(0x7f88f01587e0 0x7f88f00b0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:29.269 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:29.265+0000 7f88f61d0640 1 --2- 172.21.15.169:0/1863310198 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f88f0157f80 0x7f88f015b140 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:29.270 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:29.265+0000 7f88f61d0640 1 -- 172.21.15.169:0/1863310198 >> 172.21.15.169:0/1863310198 conn(0x7f88f00af110 msgr2=0x7f88f00af520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:29.270 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:29.266+0000 7f88f61d0640 1 -- 172.21.15.169:0/1863310198 shutdown_connections 2023-10-26T01:36:29.270 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:29.266+0000 7f88f61d0640 1 -- 172.21.15.169:0/1863310198 wait complete. 2023-10-26T01:36:29.271 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:29.268+0000 7f88f61d0640 1 Processor -- start 2023-10-26T01:36:29.271 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:29.268+0000 7f88f61d0640 1 -- start start 2023-10-26T01:36:29.273 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:29.269+0000 7f88f61d0640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f88f0157f80 0x7f88f01450d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:29.273 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:29.269+0000 7f88f61d0640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f88f0158360 0x7f88f0147880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:29.273 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:29.269+0000 7f88f61d0640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88f01587e0 0x7f88f0147dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:29.274 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:29.269+0000 7f88f61d0640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88f00cbdf0 con 0x7f88f01587e0 2023-10-26T01:36:29.274 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:29.269+0000 7f88f61d0640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88f0146f80 con 0x7f88f0157f80 2023-10-26T01:36:29.274 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:29.269+0000 7f88f61d0640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88f00cc040 con 0x7f88f0158360 2023-10-26T01:36:29.274 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:29.269+0000 7f88f59cf640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88f01587e0 0x7f88f0147dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:29.274 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:29.269+0000 7f88f59cf640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88f01587e0 0x7f88f0147dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:52052/0 (socket says 172.21.15.169:52052) 2023-10-26T01:36:29.275 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:29.269+0000 7f88f59cf640 1 -- 172.21.15.169:0/931532489 learned_addr learned my addr 172.21.15.169:0/931532489 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:36:29.275 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:29.269+0000 7f88f49cd640 1 --2- 172.21.15.169:0/931532489 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f88f0158360 0x7f88f0147880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:29.275 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:29.270+0000 7f88f51ce640 1 --2- 172.21.15.169:0/931532489 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f88f0157f80 0x7f88f01450d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:29.275 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:29.270+0000 7f88f59cf640 1 -- 172.21.15.169:0/931532489 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f88f0158360 msgr2=0x7f88f0147880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:29.275 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:29.270+0000 7f88f59cf640 1 --2- 172.21.15.169:0/931532489 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f88f0158360 0x7f88f0147880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:29.276 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:29.270+0000 7f88f59cf640 1 -- 172.21.15.169:0/931532489 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f88f0157f80 msgr2=0x7f88f01450d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:29.276 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:29.270+0000 7f88f59cf640 1 --2- 172.21.15.169:0/931532489 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f88f0157f80 0x7f88f01450d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:29.276 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:29.270+0000 7f88f59cf640 1 -- 172.21.15.169:0/931532489 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88ec009560 con 0x7f88f01587e0 2023-10-26T01:36:29.276 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:29.270+0000 7f88f49cd640 1 --2- 172.21.15.169:0/931532489 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f88f0158360 0x7f88f0147880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T01:36:29.276 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:29.270+0000 7f88f59cf640 1 --2- 172.21.15.169:0/931532489 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88f01587e0 0x7f88f0147dc0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f88e800eb10 tx=0x7f88e800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:29.277 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:29.270+0000 7f88e5ffb640 1 -- 172.21.15.169:0/931532489 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88e8004b20 con 0x7f88f01587e0 2023-10-26T01:36:29.277 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:29.271+0000 7f88e5ffb640 1 -- 172.21.15.169:0/931532489 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f88e8004ca0 con 0x7f88f01587e0 2023-10-26T01:36:29.277 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:29.271+0000 7f88f61d0640 1 -- 172.21.15.169:0/931532489 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88f00cc320 con 0x7f88f01587e0 2023-10-26T01:36:29.277 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:29.271+0000 7f88e5ffb640 1 -- 172.21.15.169:0/931532489 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88e8005280 con 0x7f88f01587e0 2023-10-26T01:36:29.277 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:29.271+0000 7f88f61d0640 1 -- 172.21.15.169:0/931532489 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88f00713a0 con 0x7f88f01587e0 2023-10-26T01:36:29.278 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:29.272+0000 7f88f61d0640 1 -- 172.21.15.169:0/931532489 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88f0073c10 con 0x7f88f01587e0 2023-10-26T01:36:29.278 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:29.273+0000 7f88e5ffb640 1 -- 172.21.15.169:0/931532489 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f88e80053e0 con 0x7f88f01587e0 2023-10-26T01:36:29.279 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:29.274+0000 7f88e5ffb640 1 --2- 172.21.15.169:0/931532489 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f88d406ae40 0x7f88d406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:29.279 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:29.274+0000 7f88e5ffb640 1 -- 172.21.15.169:0/931532489 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f88e8014070 con 0x7f88f01587e0 2023-10-26T01:36:29.279 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:29.274+0000 7f88f51ce640 1 --2- 172.21.15.169:0/931532489 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f88d406ae40 0x7f88d406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:29.279 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:29.275+0000 7f88f51ce640 1 --2- 172.21.15.169:0/931532489 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f88d406ae40 0x7f88d406d300 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f88ec035000 tx=0x7f88ec004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:29.280 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:29.278+0000 7f88e5ffb640 1 -- 172.21.15.169:0/931532489 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f88e80553d0 con 0x7f88f01587e0 2023-10-26T01:36:29.502 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:29.499+0000 7f88f61d0640 1 -- 172.21.15.169:0/931532489 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f88f00ac910 con 0x7f88f01587e0 2023-10-26T01:36:30.125 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:30.122+0000 7f88e5ffb640 1 -- 172.21.15.169:0/931532489 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v44) v1 ==== 104+0+0 (secure 0 0 0) 0x7f88e80589f0 con 0x7f88f01587e0 2023-10-26T01:36:30.129 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:30.126+0000 7f88f61d0640 1 -- 172.21.15.169:0/931532489 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f88d406ae40 msgr2=0x7f88d406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:30.129 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:30.126+0000 7f88f61d0640 1 --2- 172.21.15.169:0/931532489 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f88d406ae40 0x7f88d406d300 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f88ec035000 tx=0x7f88ec004730 comp rx=0 tx=0).stop 2023-10-26T01:36:30.129 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:30.126+0000 7f88f61d0640 1 -- 172.21.15.169:0/931532489 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88f01587e0 msgr2=0x7f88f0147dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:30.129 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:30.126+0000 7f88f61d0640 1 --2- 172.21.15.169:0/931532489 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88f01587e0 0x7f88f0147dc0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f88e800eb10 tx=0x7f88e800c2d0 comp rx=0 tx=0).stop 2023-10-26T01:36:30.130 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:30.126+0000 7f88f61d0640 1 -- 172.21.15.169:0/931532489 shutdown_connections 2023-10-26T01:36:30.130 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:30.126+0000 7f88f61d0640 1 --2- 172.21.15.169:0/931532489 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f88d406ae40 0x7f88d406d300 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:30.130 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:30.126+0000 7f88f61d0640 1 --2- 172.21.15.169:0/931532489 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88f01587e0 0x7f88f0147dc0 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:30.130 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:30.126+0000 7f88f61d0640 1 --2- 172.21.15.169:0/931532489 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f88f0158360 0x7f88f0147880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:30.130 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:30.126+0000 7f88f61d0640 1 --2- 172.21.15.169:0/931532489 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f88f0157f80 0x7f88f01450d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:30.131 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:30.126+0000 7f88f61d0640 1 -- 172.21.15.169:0/931532489 >> 172.21.15.169:0/931532489 conn(0x7f88f00af110 msgr2=0x7f88f0069b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:30.131 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:30.126+0000 7f88f61d0640 1 -- 172.21.15.169:0/931532489 shutdown_connections 2023-10-26T01:36:30.131 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:30.126+0000 7f88f61d0640 1 -- 172.21.15.169:0/931532489 wait complete. 2023-10-26T01:36:30.151 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-26T01:36:30.366 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:30.363+0000 7f9ecb856640 1 -- 172.21.15.169:0/561923402 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9ec406cdb0 msgr2=0x7f9ec415b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:30.367 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:30.363+0000 7f9ecb856640 1 --2- 172.21.15.169:0/561923402 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9ec406cdb0 0x7f9ec415b2c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f9eb4009830 tx=0x7f9eb402bcd0 comp rx=0 tx=0).stop 2023-10-26T01:36:30.367 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:30.363+0000 7f9ecb856640 1 -- 172.21.15.169:0/561923402 shutdown_connections 2023-10-26T01:36:30.367 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:30.363+0000 7f9ecb856640 1 --2- 172.21.15.169:0/561923402 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9ec406cdb0 0x7f9ec415b2c0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:30.367 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:30.363+0000 7f9ecb856640 1 --2- 172.21.15.169:0/561923402 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9ec40aab40 0x7f9ec40ac6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:30.368 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:30.363+0000 7f9ecb856640 1 -- 172.21.15.169:0/561923402 >> 172.21.15.169:0/561923402 conn(0x7f9ec40aeee0 msgr2=0x7f9ec40af2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:30.368 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:30.363+0000 7f9ecb856640 1 -- 172.21.15.169:0/561923402 shutdown_connections 2023-10-26T01:36:30.368 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:30.363+0000 7f9ecb856640 1 -- 172.21.15.169:0/561923402 wait complete. 2023-10-26T01:36:30.369 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:30.365+0000 7f9ecb856640 1 Processor -- start 2023-10-26T01:36:30.369 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:30.366+0000 7f9ecb856640 1 -- start start 2023-10-26T01:36:30.370 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:30.367+0000 7f9ecb856640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9ec406cdb0 0x7f9ec4145030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:30.371 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:30.367+0000 7f9ecb856640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ec415b800 0x7f9ec4144550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:30.371 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:30.367+0000 7f9ecb856640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9ec40ccb50 0x7f9ec406bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:30.371 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:30.367+0000 7f9ecb856640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ec4158f20 con 0x7f9ec415b800 2023-10-26T01:36:30.371 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:30.367+0000 7f9ecb856640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ec41582e0 con 0x7f9ec406cdb0 2023-10-26T01:36:30.372 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:30.367+0000 7f9ecb856640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ec4145570 con 0x7f9ec40ccb50 2023-10-26T01:36:30.372 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:30.367+0000 7f9ec8df1640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ec415b800 0x7f9ec4144550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:30.372 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:30.367+0000 7f9ec8df1640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ec415b800 0x7f9ec4144550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:52062/0 (socket says 172.21.15.169:52062) 2023-10-26T01:36:30.372 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:30.367+0000 7f9ec8df1640 1 -- 172.21.15.169:0/846688371 learned_addr learned my addr 172.21.15.169:0/846688371 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:36:30.372 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:30.368+0000 7f9ec95f2640 1 --2- 172.21.15.169:0/846688371 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9ec406cdb0 0x7f9ec4145030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:30.373 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:30.368+0000 7f9ec9df3640 1 --2- 172.21.15.169:0/846688371 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9ec40ccb50 0x7f9ec406bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:30.373 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:30.368+0000 7f9ec8df1640 1 -- 172.21.15.169:0/846688371 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9ec40ccb50 msgr2=0x7f9ec406bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:30.373 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:30.368+0000 7f9ec8df1640 1 --2- 172.21.15.169:0/846688371 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9ec40ccb50 0x7f9ec406bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:30.373 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:30.368+0000 7f9ec8df1640 1 -- 172.21.15.169:0/846688371 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9ec406cdb0 msgr2=0x7f9ec4145030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:30.373 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:30.368+0000 7f9ec8df1640 1 --2- 172.21.15.169:0/846688371 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9ec406cdb0 0x7f9ec4145030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:30.374 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:30.368+0000 7f9ec8df1640 1 -- 172.21.15.169:0/846688371 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ec0009490 con 0x7f9ec415b800 2023-10-26T01:36:30.374 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:30.368+0000 7f9ec8df1640 1 --2- 172.21.15.169:0/846688371 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ec415b800 0x7f9ec4144550 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f9eb4000c00 tx=0x7f9eb4003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:30.375 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:30.368+0000 7f9eb9ffb640 1 -- 172.21.15.169:0/846688371 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eb4003e20 con 0x7f9ec415b800 2023-10-26T01:36:30.375 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:30.368+0000 7f9eb9ffb640 1 -- 172.21.15.169:0/846688371 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9eb4034440 con 0x7f9ec415b800 2023-10-26T01:36:30.375 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:30.369+0000 7f9ecb856640 1 -- 172.21.15.169:0/846688371 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9eb4009490 con 0x7f9ec415b800 2023-10-26T01:36:30.376 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:30.369+0000 7f9eb9ffb640 1 -- 172.21.15.169:0/846688371 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eb4034850 con 0x7f9ec415b800 2023-10-26T01:36:30.376 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:30.369+0000 7f9ecb856640 1 -- 172.21.15.169:0/846688371 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ec4158860 con 0x7f9ec415b800 2023-10-26T01:36:30.376 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:30.370+0000 7f9ecb856640 1 -- 172.21.15.169:0/846688371 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ec4066af0 con 0x7f9ec415b800 2023-10-26T01:36:30.376 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:30.371+0000 7f9eb9ffb640 1 -- 172.21.15.169:0/846688371 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f9eb40349b0 con 0x7f9ec415b800 2023-10-26T01:36:30.377 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:30.371+0000 7f9eb9ffb640 1 --2- 172.21.15.169:0/846688371 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9ea806ae40 0x7f9ea806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:30.377 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:30.372+0000 7f9eb9ffb640 1 -- 172.21.15.169:0/846688371 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f9eb40abe10 con 0x7f9ec415b800 2023-10-26T01:36:30.377 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:30.372+0000 7f9ec95f2640 1 --2- 172.21.15.169:0/846688371 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9ea806ae40 0x7f9ea806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:30.377 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:30.373+0000 7f9ec95f2640 1 --2- 172.21.15.169:0/846688371 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9ea806ae40 0x7f9ea806d300 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f9ec0009cf0 tx=0x7f9ec0005f90 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:30.378 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:30.376+0000 7f9eb9ffb640 1 -- 172.21.15.169:0/846688371 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9eb40af030 con 0x7f9ec415b800 2023-10-26T01:36:30.571 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:30.568+0000 7f9ecb856640 1 -- 172.21.15.169:0/846688371 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f9ec4159060 con 0x7f9ec415b800 2023-10-26T01:36:31.143 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:31.140+0000 7f9eb9ffb640 1 -- 172.21.15.169:0/846688371 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v93) v1 ==== 181+0+0 (secure 0 0 0) 0x7f9ec4159060 con 0x7f9ec415b800 2023-10-26T01:36:31.144 INFO:teuthology.orchestra.run.smithi169.stderr:pool 'cephfs_metadata' removed 2023-10-26T01:36:31.150 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:31.146+0000 7f9ecb856640 1 -- 172.21.15.169:0/846688371 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9ea806ae40 msgr2=0x7f9ea806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:31.150 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:31.146+0000 7f9ecb856640 1 --2- 172.21.15.169:0/846688371 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9ea806ae40 0x7f9ea806d300 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f9ec0009cf0 tx=0x7f9ec0005f90 comp rx=0 tx=0).stop 2023-10-26T01:36:31.150 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:31.146+0000 7f9ecb856640 1 -- 172.21.15.169:0/846688371 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ec415b800 msgr2=0x7f9ec4144550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:31.151 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:31.146+0000 7f9ecb856640 1 --2- 172.21.15.169:0/846688371 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ec415b800 0x7f9ec4144550 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f9eb4000c00 tx=0x7f9eb4003820 comp rx=0 tx=0).stop 2023-10-26T01:36:31.152 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:31.149+0000 7f9ecb856640 1 -- 172.21.15.169:0/846688371 shutdown_connections 2023-10-26T01:36:31.152 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:31.149+0000 7f9ecb856640 1 --2- 172.21.15.169:0/846688371 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9ea806ae40 0x7f9ea806d300 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:31.152 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:31.149+0000 7f9ecb856640 1 --2- 172.21.15.169:0/846688371 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ec415b800 0x7f9ec4144550 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:31.152 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:31.149+0000 7f9ecb856640 1 --2- 172.21.15.169:0/846688371 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9ec40ccb50 0x7f9ec406bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:31.152 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:31.149+0000 7f9ecb856640 1 --2- 172.21.15.169:0/846688371 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9ec406cdb0 0x7f9ec4145030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:31.153 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:31.149+0000 7f9ecb856640 1 -- 172.21.15.169:0/846688371 >> 172.21.15.169:0/846688371 conn(0x7f9ec40aeee0 msgr2=0x7f9ec40ae3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:31.153 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:31.149+0000 7f9ecb856640 1 -- 172.21.15.169:0/846688371 shutdown_connections 2023-10-26T01:36:31.153 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:31.149+0000 7f9ecb856640 1 -- 172.21.15.169:0/846688371 wait complete. 2023-10-26T01:36:31.173 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-26T01:36:31.396 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:31.393+0000 7f33fbb4c640 1 -- 172.21.15.169:0/938975312 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33e00042d0 msgr2=0x7f33e00047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:31.396 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:31.393+0000 7f33fbb4c640 1 --2- 172.21.15.169:0/938975312 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33e00042d0 0x7f33e00047c0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f33e400ac30 tx=0x7f33e00057f0 comp rx=0 tx=0).stop 2023-10-26T01:36:31.396 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:31.393+0000 7f33fa0e9640 1 -- 172.21.15.169:0/938975312 reap_dead start 2023-10-26T01:36:31.397 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:31.393+0000 7f33fbb4c640 1 -- 172.21.15.169:0/938975312 shutdown_connections 2023-10-26T01:36:31.397 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:31.393+0000 7f33fbb4c640 1 -- 172.21.15.169:0/938975312 >> 172.21.15.169:0/938975312 conn(0x7f33f40aeee0 msgr2=0x7f33f40af2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:31.397 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:31.393+0000 7f33fbb4c640 1 -- 172.21.15.169:0/938975312 shutdown_connections 2023-10-26T01:36:31.397 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:31.393+0000 7f33fbb4c640 1 -- 172.21.15.169:0/938975312 wait complete. 2023-10-26T01:36:31.398 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:31.395+0000 7f33fbb4c640 1 Processor -- start 2023-10-26T01:36:31.398 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:31.395+0000 7f33fbb4c640 1 -- start start 2023-10-26T01:36:31.399 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:31.396+0000 7f33fbb4c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33e00042d0 0x7f33f414b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:31.400 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:31.396+0000 7f33fbb4c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33f41e4f20 0x7f33f41e53a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:31.400 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:31.396+0000 7f33fbb4c640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f33f40c76f0 0x7f33f40c7b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:31.400 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:31.396+0000 7f33fbb4c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33f40c80b0 con 0x7f33e00042d0 2023-10-26T01:36:31.400 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:31.396+0000 7f33fbb4c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33f4158cb0 con 0x7f33f41e4f20 2023-10-26T01:36:31.400 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:31.396+0000 7f33fbb4c640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33f414fb00 con 0x7f33f40c76f0 2023-10-26T01:36:31.401 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:31.396+0000 7f33f98e8640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33e00042d0 0x7f33f414b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:31.401 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:31.396+0000 7f33f98e8640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33e00042d0 0x7f33f414b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:52094/0 (socket says 172.21.15.169:52094) 2023-10-26T01:36:31.401 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:31.396+0000 7f33f98e8640 1 -- 172.21.15.169:0/3009347291 learned_addr learned my addr 172.21.15.169:0/3009347291 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:36:31.401 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:31.397+0000 7f33f90e7640 1 --2- 172.21.15.169:0/3009347291 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33f41e4f20 0x7f33f41e53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:31.401 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:31.397+0000 7f33fa0e9640 1 --2- 172.21.15.169:0/3009347291 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f33f40c76f0 0x7f33f40c7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:31.402 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:31.397+0000 7f33f98e8640 1 -- 172.21.15.169:0/3009347291 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f33f40c76f0 msgr2=0x7f33f40c7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:31.402 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:31.397+0000 7f33f98e8640 1 --2- 172.21.15.169:0/3009347291 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f33f40c76f0 0x7f33f40c7b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:31.402 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:31.397+0000 7f33f98e8640 1 -- 172.21.15.169:0/3009347291 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33f41e4f20 msgr2=0x7f33f41e53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:31.402 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:31.397+0000 7f33f98e8640 1 --2- 172.21.15.169:0/3009347291 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33f41e4f20 0x7f33f41e53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:31.402 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:31.397+0000 7f33f98e8640 1 -- 172.21.15.169:0/3009347291 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33f4158f10 con 0x7f33e00042d0 2023-10-26T01:36:31.403 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:31.397+0000 7f33f90e7640 1 --2- 172.21.15.169:0/3009347291 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33f41e4f20 0x7f33f41e53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:36:31.403 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:31.397+0000 7f33fa0e9640 1 --2- 172.21.15.169:0/3009347291 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f33f40c76f0 0x7f33f40c7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:36:31.403 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:31.397+0000 7f33f98e8640 1 --2- 172.21.15.169:0/3009347291 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33e00042d0 0x7f33f414b9e0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f33f0010a10 tx=0x7f33f0010ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:31.404 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:31.397+0000 7f33ea7fc640 1 -- 172.21.15.169:0/3009347291 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f000eda0 con 0x7f33e00042d0 2023-10-26T01:36:31.405 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:31.397+0000 7f33ea7fc640 1 -- 172.21.15.169:0/3009347291 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f33f0004030 con 0x7f33e00042d0 2023-10-26T01:36:31.405 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:31.397+0000 7f33fbb4c640 1 -- 172.21.15.169:0/3009347291 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33f41591f0 con 0x7f33e00042d0 2023-10-26T01:36:31.405 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:31.397+0000 7f33ea7fc640 1 -- 172.21.15.169:0/3009347291 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f0004400 con 0x7f33e00042d0 2023-10-26T01:36:31.405 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:31.398+0000 7f33fbb4c640 1 -- 172.21.15.169:0/3009347291 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33f41597b0 con 0x7f33e00042d0 2023-10-26T01:36:31.405 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:31.398+0000 7f33fbb4c640 1 -- 172.21.15.169:0/3009347291 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33f4066af0 con 0x7f33e00042d0 2023-10-26T01:36:31.406 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:31.399+0000 7f33ea7fc640 1 -- 172.21.15.169:0/3009347291 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f33f00035f0 con 0x7f33e00042d0 2023-10-26T01:36:31.406 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:31.400+0000 7f33ea7fc640 1 --2- 172.21.15.169:0/3009347291 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f33d806ae40 0x7f33d806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:31.406 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:31.400+0000 7f33ea7fc640 1 -- 172.21.15.169:0/3009347291 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5415+0+0 (secure 0 0 0) 0x7f33f0016070 con 0x7f33e00042d0 2023-10-26T01:36:31.406 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:31.401+0000 7f33f90e7640 1 --2- 172.21.15.169:0/3009347291 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f33d806ae40 0x7f33d806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:31.406 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:31.402+0000 7f33f90e7640 1 --2- 172.21.15.169:0/3009347291 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f33d806ae40 0x7f33d806d300 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f33e4038000 tx=0x7f33e40046c0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:31.407 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:31.404+0000 7f33ea7fc640 1 -- 172.21.15.169:0/3009347291 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f33f0055520 con 0x7f33e00042d0 2023-10-26T01:36:31.595 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:31.593+0000 7f33fbb4c640 1 -- 172.21.15.169:0/3009347291 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f33f404da70 con 0x7f33e00042d0 2023-10-26T01:36:32.147 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:32.144+0000 7f33ea7fc640 1 -- 172.21.15.169:0/3009347291 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v94) v1 ==== 169+0+0 (secure 0 0 0) 0x7f33f0058b40 con 0x7f33e00042d0 2023-10-26T01:36:32.147 INFO:teuthology.orchestra.run.smithi169.stderr:pool 'cephfs_data' removed 2023-10-26T01:36:32.150 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:32.147+0000 7f33fbb4c640 1 -- 172.21.15.169:0/3009347291 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f33d806ae40 msgr2=0x7f33d806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:32.151 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:32.147+0000 7f33fbb4c640 1 --2- 172.21.15.169:0/3009347291 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f33d806ae40 0x7f33d806d300 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f33e4038000 tx=0x7f33e40046c0 comp rx=0 tx=0).stop 2023-10-26T01:36:32.151 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:32.147+0000 7f33fbb4c640 1 -- 172.21.15.169:0/3009347291 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33e00042d0 msgr2=0x7f33f414b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:32.151 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:32.147+0000 7f33fbb4c640 1 --2- 172.21.15.169:0/3009347291 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33e00042d0 0x7f33f414b9e0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f33f0010a10 tx=0x7f33f0010ed0 comp rx=0 tx=0).stop 2023-10-26T01:36:32.151 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:32.148+0000 7f33fbb4c640 1 -- 172.21.15.169:0/3009347291 shutdown_connections 2023-10-26T01:36:32.151 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:32.148+0000 7f33fbb4c640 1 --2- 172.21.15.169:0/3009347291 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f33d806ae40 0x7f33d806d300 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:32.152 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:32.148+0000 7f33fbb4c640 1 --2- 172.21.15.169:0/3009347291 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f33f40c76f0 0x7f33f40c7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:32.152 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:32.148+0000 7f33fbb4c640 1 --2- 172.21.15.169:0/3009347291 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33f41e4f20 0x7f33f41e53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:32.152 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:32.148+0000 7f33fbb4c640 1 --2- 172.21.15.169:0/3009347291 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33e00042d0 0x7f33f414b9e0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:32.152 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:32.148+0000 7f33fbb4c640 1 -- 172.21.15.169:0/3009347291 >> 172.21.15.169:0/3009347291 conn(0x7f33f40aeee0 msgr2=0x7f33f406ace0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:32.152 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:32.148+0000 7f33fbb4c640 1 -- 172.21.15.169:0/3009347291 shutdown_connections 2023-10-26T01:36:32.153 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:32.148+0000 7f33fbb4c640 1 -- 172.21.15.169:0/3009347291 wait complete. 2023-10-26T01:36:32.170 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 18 2023-10-26T01:36:32.392 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:32.389+0000 7fbfd3fff640 1 -- 172.21.15.169:0/1814241224 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbfd415d7e0 msgr2=0x7fbfd4069e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:32.392 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:32.389+0000 7fbfd3fff640 1 --2- 172.21.15.169:0/1814241224 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbfd415d7e0 0x7fbfd4069e40 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc005fd0 tx=0x7fbfcc02c2d0 comp rx=0 tx=0).stop 2023-10-26T01:36:32.393 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:32.389+0000 7fbfd3fff640 1 -- 172.21.15.169:0/1814241224 shutdown_connections 2023-10-26T01:36:32.393 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:32.389+0000 7fbfd3fff640 1 --2- 172.21.15.169:0/1814241224 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbfd415d7e0 0x7fbfd4069e40 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:32.393 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:32.389+0000 7fbfd3fff640 1 --2- 172.21.15.169:0/1814241224 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbfd4153b90 0x7fbfd4069900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:32.393 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:32.389+0000 7fbfd3fff640 1 -- 172.21.15.169:0/1814241224 >> 172.21.15.169:0/1814241224 conn(0x7fbfd40af110 msgr2=0x7fbfd40af520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:32.393 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:32.389+0000 7fbfd3fff640 1 -- 172.21.15.169:0/1814241224 shutdown_connections 2023-10-26T01:36:32.394 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:32.389+0000 7fbfd3fff640 1 -- 172.21.15.169:0/1814241224 wait complete. 2023-10-26T01:36:32.395 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:32.391+0000 7fbfd3fff640 1 Processor -- start 2023-10-26T01:36:32.395 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:32.391+0000 7fbfd3fff640 1 -- start start 2023-10-26T01:36:32.396 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:32.392+0000 7fbfd3fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbfd40b0e50 0x7fbfd4152ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:32.396 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:32.392+0000 7fbfd3fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbfd4153b90 0x7fbfd4154360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:32.396 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:32.392+0000 7fbfd3fff640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbfd415d7e0 0x7fbfd406bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:32.396 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:32.392+0000 7fbfd3fff640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd4098650 con 0x7fbfd40b0e50 2023-10-26T01:36:32.397 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:32.392+0000 7fbfd3fff640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd4155570 con 0x7fbfd4153b90 2023-10-26T01:36:32.397 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:32.392+0000 7fbfd3fff640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd41533e0 con 0x7fbfd415d7e0 2023-10-26T01:36:32.397 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:32.392+0000 7fbfd2ffd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbfd40b0e50 0x7fbfd4152ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:32.397 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:32.392+0000 7fbfd2ffd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbfd40b0e50 0x7fbfd4152ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:52118/0 (socket says 172.21.15.169:52118) 2023-10-26T01:36:32.397 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:32.392+0000 7fbfd2ffd640 1 -- 172.21.15.169:0/2290075532 learned_addr learned my addr 172.21.15.169:0/2290075532 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:36:32.398 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:32.393+0000 7fbfd2ffd640 1 -- 172.21.15.169:0/2290075532 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbfd415d7e0 msgr2=0x7fbfd406bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:32.398 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:32.393+0000 7fbfd37fe640 1 --2- 172.21.15.169:0/2290075532 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbfd415d7e0 0x7fbfd406bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:32.398 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:32.393+0000 7fbfca7fc640 1 --2- 172.21.15.169:0/2290075532 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbfd4153b90 0x7fbfd4154360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:32.398 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:32.393+0000 7fbfd2ffd640 1 --2- 172.21.15.169:0/2290075532 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbfd415d7e0 0x7fbfd406bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:32.399 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:32.393+0000 7fbfd2ffd640 1 -- 172.21.15.169:0/2290075532 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbfd4153b90 msgr2=0x7fbfd4154360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:32.399 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:32.393+0000 7fbfd2ffd640 1 --2- 172.21.15.169:0/2290075532 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbfd4153b90 0x7fbfd4154360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:32.399 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:32.393+0000 7fbfd2ffd640 1 -- 172.21.15.169:0/2290075532 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfcc009900 con 0x7fbfd40b0e50 2023-10-26T01:36:32.399 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:32.393+0000 7fbfd2ffd640 1 --2- 172.21.15.169:0/2290075532 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbfd40b0e50 0x7fbfd4152ea0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fbfc400c950 tx=0x7fbfc400ce10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:32.400 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:32.393+0000 7fbfd0ff9640 1 -- 172.21.15.169:0/2290075532 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc4007c70 con 0x7fbfd40b0e50 2023-10-26T01:36:32.401 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:32.393+0000 7fbfd0ff9640 1 -- 172.21.15.169:0/2290075532 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fbfc400f450 con 0x7fbfd40b0e50 2023-10-26T01:36:32.401 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:32.393+0000 7fbfd0ff9640 1 -- 172.21.15.169:0/2290075532 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc400f860 con 0x7fbfd40b0e50 2023-10-26T01:36:32.401 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:32.393+0000 7fbfd3fff640 1 -- 172.21.15.169:0/2290075532 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfd4154cd0 con 0x7fbfd40b0e50 2023-10-26T01:36:32.401 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:32.393+0000 7fbfd3fff640 1 -- 172.21.15.169:0/2290075532 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfd4155290 con 0x7fbfd40b0e50 2023-10-26T01:36:32.401 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:32.395+0000 7fbfd3fff640 1 -- 172.21.15.169:0/2290075532 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfd4073c10 con 0x7fbfd40b0e50 2023-10-26T01:36:32.401 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:32.395+0000 7fbfd0ff9640 1 -- 172.21.15.169:0/2290075532 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fbfc400f9c0 con 0x7fbfd40b0e50 2023-10-26T01:36:32.402 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:32.397+0000 7fbfd0ff9640 1 --2- 172.21.15.169:0/2290075532 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbfb406ae40 0x7fbfb406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:32.402 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:32.397+0000 7fbfd0ff9640 1 -- 172.21.15.169:0/2290075532 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fbfc4089b50 con 0x7fbfd40b0e50 2023-10-26T01:36:32.402 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:32.398+0000 7fbfca7fc640 1 --2- 172.21.15.169:0/2290075532 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbfb406ae40 0x7fbfb406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:32.403 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:32.399+0000 7fbfca7fc640 1 --2- 172.21.15.169:0/2290075532 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbfb406ae40 0x7fbfb406d300 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fbfd406cb90 tx=0x7fbfbc009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:32.404 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:32.402+0000 7fbfd0ff9640 1 -- 172.21.15.169:0/2290075532 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbfc4054f00 con 0x7fbfd40b0e50 2023-10-26T01:36:32.607 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:32.604+0000 7fbfd3fff640 1 -- 172.21.15.169:0/2290075532 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "config reset", "num": 18} v 0) v1 -- 0x7fbfd40b1c80 con 0x7fbfd40b0e50 2023-10-26T01:36:32.614 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:32.611+0000 7fbfd0ff9640 1 -- 172.21.15.169:0/2290075532 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 18}]=0 v26) v1 ==== 71+0+0 (secure 0 0 0) 0x7fbfc4058520 con 0x7fbfd40b0e50 2023-10-26T01:36:32.617 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:32.614+0000 7fbfd3fff640 1 -- 172.21.15.169:0/2290075532 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbfb406ae40 msgr2=0x7fbfb406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:32.617 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:32.614+0000 7fbfd3fff640 1 --2- 172.21.15.169:0/2290075532 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbfb406ae40 0x7fbfb406d300 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fbfd406cb90 tx=0x7fbfbc009040 comp rx=0 tx=0).stop 2023-10-26T01:36:32.617 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:32.614+0000 7fbfd3fff640 1 -- 172.21.15.169:0/2290075532 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbfd40b0e50 msgr2=0x7fbfd4152ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:32.617 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:32.614+0000 7fbfd3fff640 1 --2- 172.21.15.169:0/2290075532 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbfd40b0e50 0x7fbfd4152ea0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fbfc400c950 tx=0x7fbfc400ce10 comp rx=0 tx=0).stop 2023-10-26T01:36:32.618 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:32.614+0000 7fbfd3fff640 1 -- 172.21.15.169:0/2290075532 shutdown_connections 2023-10-26T01:36:32.618 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:32.614+0000 7fbfd3fff640 1 --2- 172.21.15.169:0/2290075532 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbfb406ae40 0x7fbfb406d300 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:32.618 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:32.614+0000 7fbfd3fff640 1 --2- 172.21.15.169:0/2290075532 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbfd415d7e0 0x7fbfd406bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:32.618 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:32.614+0000 7fbfd3fff640 1 --2- 172.21.15.169:0/2290075532 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbfd4153b90 0x7fbfd4154360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:32.618 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:32.614+0000 7fbfd3fff640 1 --2- 172.21.15.169:0/2290075532 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbfd40b0e50 0x7fbfd4152ea0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:32.619 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:32.614+0000 7fbfd3fff640 1 -- 172.21.15.169:0/2290075532 >> 172.21.15.169:0/2290075532 conn(0x7fbfd40af110 msgr2=0x7fbfd40ae4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:32.619 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:32.614+0000 7fbfd3fff640 1 -- 172.21.15.169:0/2290075532 shutdown_connections 2023-10-26T01:36:32.619 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:32.614+0000 7fbfd3fff640 1 -- 172.21.15.169:0/2290075532 wait complete. 2023-10-26T01:36:32.636 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestClusterFull.test_full_fsync' 2023-10-26T01:36:32.846 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:32.843+0000 7f9f01ac9640 1 -- 172.21.15.169:0/178811810 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9efc0cdc30 msgr2=0x7f9efc0cbac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:32.847 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:32.843+0000 7f9f01ac9640 1 --2- 172.21.15.169:0/178811810 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9efc0cdc30 0x7f9efc0cbac0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f9ef4005fd0 tx=0x7f9ef402c240 comp rx=0 tx=0).stop 2023-10-26T01:36:32.847 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:32.843+0000 7f9f01ac9640 1 -- 172.21.15.169:0/178811810 shutdown_connections 2023-10-26T01:36:32.847 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:32.843+0000 7f9f01ac9640 1 --2- 172.21.15.169:0/178811810 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9efc0cdc30 0x7f9efc0cbac0 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:32.847 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:32.843+0000 7f9f01ac9640 1 --2- 172.21.15.169:0/178811810 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9efc0ac930 0x7f9efc0cd6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:32.848 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:32.843+0000 7f9f01ac9640 1 -- 172.21.15.169:0/178811810 >> 172.21.15.169:0/178811810 conn(0x7f9efc069930 msgr2=0x7f9efc069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:32.848 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:32.844+0000 7f9f01ac9640 1 -- 172.21.15.169:0/178811810 shutdown_connections 2023-10-26T01:36:32.848 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:32.844+0000 7f9f01ac9640 1 -- 172.21.15.169:0/178811810 wait complete. 2023-10-26T01:36:32.850 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:32.845+0000 7f9f01ac9640 1 Processor -- start 2023-10-26T01:36:32.850 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:32.845+0000 7f9f01ac9640 1 -- start start 2023-10-26T01:36:32.850 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:32.846+0000 7f9f01ac9640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9efc0ac930 0x7f9efc150d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:32.850 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:32.846+0000 7f9f01ac9640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9efc0cdc30 0x7f9efc14a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:32.850 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:32.846+0000 7f9f01ac9640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9efc1592d0 0x7f9efc0b1db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:32.851 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:32.846+0000 7f9f01ac9640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9efc0b0f90 con 0x7f9efc0ac930 2023-10-26T01:36:32.851 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:32.846+0000 7f9f01ac9640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9efc14ad50 con 0x7f9efc0cdc30 2023-10-26T01:36:32.851 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:32.846+0000 7f9f01ac9640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9efc0b0380 con 0x7f9efc1592d0 2023-10-26T01:36:32.851 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:32.846+0000 7f9efb7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9efc0ac930 0x7f9efc150d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:32.851 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:32.847+0000 7f9efb7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9efc0ac930 0x7f9efc150d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:52138/0 (socket says 172.21.15.169:52138) 2023-10-26T01:36:32.852 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:32.847+0000 7f9efb7fe640 1 -- 172.21.15.169:0/3284147723 learned_addr learned my addr 172.21.15.169:0/3284147723 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:36:32.852 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:32.847+0000 7f9ef3fff640 1 --2- 172.21.15.169:0/3284147723 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9efc0cdc30 0x7f9efc14a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:32.852 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:32.847+0000 7f9efbfff640 1 --2- 172.21.15.169:0/3284147723 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9efc1592d0 0x7f9efc0b1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:32.852 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:32.847+0000 7f9efb7fe640 1 -- 172.21.15.169:0/3284147723 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9efc1592d0 msgr2=0x7f9efc0b1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:32.852 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:32.847+0000 7f9efb7fe640 1 --2- 172.21.15.169:0/3284147723 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9efc1592d0 0x7f9efc0b1db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:32.853 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:32.847+0000 7f9efb7fe640 1 -- 172.21.15.169:0/3284147723 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9efc0cdc30 msgr2=0x7f9efc14a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:32.853 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:32.847+0000 7f9efb7fe640 1 --2- 172.21.15.169:0/3284147723 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9efc0cdc30 0x7f9efc14a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:32.853 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:32.847+0000 7f9efb7fe640 1 -- 172.21.15.169:0/3284147723 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ef4009900 con 0x7f9efc0ac930 2023-10-26T01:36:32.853 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:32.847+0000 7f9ef3fff640 1 --2- 172.21.15.169:0/3284147723 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9efc0cdc30 0x7f9efc14a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:36:32.853 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:32.847+0000 7f9efb7fe640 1 --2- 172.21.15.169:0/3284147723 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9efc0ac930 0x7f9efc150d20 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f9eec00c460 tx=0x7f9eec00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:32.855 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:32.848+0000 7f9ef8ff9640 1 -- 172.21.15.169:0/3284147723 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eec0052d0 con 0x7f9efc0ac930 2023-10-26T01:36:32.855 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:32.848+0000 7f9ef8ff9640 1 -- 172.21.15.169:0/3284147723 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9eec005430 con 0x7f9efc0ac930 2023-10-26T01:36:32.855 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:32.848+0000 7f9f01ac9640 1 -- 172.21.15.169:0/3284147723 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9efc0b22f0 con 0x7f9efc0ac930 2023-10-26T01:36:32.855 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:32.848+0000 7f9ef8ff9640 1 -- 172.21.15.169:0/3284147723 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eec012890 con 0x7f9efc0ac930 2023-10-26T01:36:32.855 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:32.848+0000 7f9f01ac9640 1 -- 172.21.15.169:0/3284147723 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9efc0b08d0 con 0x7f9efc0ac930 2023-10-26T01:36:32.856 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:32.849+0000 7f9f01ac9640 1 -- 172.21.15.169:0/3284147723 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9efc04db10 con 0x7f9efc0ac930 2023-10-26T01:36:32.856 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:32.850+0000 7f9ef8ff9640 1 -- 172.21.15.169:0/3284147723 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f9eec0035f0 con 0x7f9efc0ac930 2023-10-26T01:36:32.856 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:32.851+0000 7f9ef8ff9640 1 --2- 172.21.15.169:0/3284147723 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9edc06ae40 0x7f9edc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:32.856 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:32.851+0000 7f9ef8ff9640 1 -- 172.21.15.169:0/3284147723 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f9eec088d10 con 0x7f9efc0ac930 2023-10-26T01:36:32.856 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:32.851+0000 7f9ef3fff640 1 --2- 172.21.15.169:0/3284147723 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9edc06ae40 0x7f9edc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:32.857 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:32.853+0000 7f9ef3fff640 1 --2- 172.21.15.169:0/3284147723 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9edc06ae40 0x7f9edc06d300 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f9ee4006fd0 tx=0x7f9ee4008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:32.858 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:32.855+0000 7f9ef8ff9640 1 -- 172.21.15.169:0/3284147723 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9eec08e020 con 0x7f9efc0ac930 2023-10-26T01:36:33.117 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:33.114+0000 7f9f01ac9640 1 -- 172.21.15.169:0/3284147723 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_fsync"]} v 0) v1 -- 0x7f9efc0ce060 con 0x7f9efc0ac930 2023-10-26T01:36:33.173 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:33.171+0000 7f9ef8ff9640 1 -- 172.21.15.169:0/3284147723 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_fsync"]}]=0 v300) v1 ==== 133+0+0 (secure 0 0 0) 0x7f9eec0540b0 con 0x7f9efc0ac930 2023-10-26T01:36:33.175 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:33.174+0000 7f9f01ac9640 1 -- 172.21.15.169:0/3284147723 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9edc06ae40 msgr2=0x7f9edc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:33.176 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:33.174+0000 7f9f01ac9640 1 --2- 172.21.15.169:0/3284147723 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9edc06ae40 0x7f9edc06d300 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f9ee4006fd0 tx=0x7f9ee4008040 comp rx=0 tx=0).stop 2023-10-26T01:36:33.176 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:33.174+0000 7f9f01ac9640 1 -- 172.21.15.169:0/3284147723 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9efc0ac930 msgr2=0x7f9efc150d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:33.176 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:33.174+0000 7f9f01ac9640 1 --2- 172.21.15.169:0/3284147723 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9efc0ac930 0x7f9efc150d20 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f9eec00c460 tx=0x7f9eec00c920 comp rx=0 tx=0).stop 2023-10-26T01:36:33.176 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:33.174+0000 7f9f01ac9640 1 -- 172.21.15.169:0/3284147723 shutdown_connections 2023-10-26T01:36:33.177 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:33.174+0000 7f9f01ac9640 1 --2- 172.21.15.169:0/3284147723 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9edc06ae40 0x7f9edc06d300 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:33.177 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:33.174+0000 7f9f01ac9640 1 --2- 172.21.15.169:0/3284147723 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9efc1592d0 0x7f9efc0b1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:33.177 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:33.174+0000 7f9f01ac9640 1 --2- 172.21.15.169:0/3284147723 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9efc0cdc30 0x7f9efc14a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:33.177 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:33.174+0000 7f9f01ac9640 1 --2- 172.21.15.169:0/3284147723 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9efc0ac930 0x7f9efc150d20 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:33.178 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:33.174+0000 7f9f01ac9640 1 -- 172.21.15.169:0/3284147723 >> 172.21.15.169:0/3284147723 conn(0x7f9efc069930 msgr2=0x7f9efc0ccae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:33.178 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:33.174+0000 7f9f01ac9640 1 -- 172.21.15.169:0/3284147723 shutdown_connections 2023-10-26T01:36:33.178 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:33.174+0000 7f9f01ac9640 1 -- 172.21.15.169:0/3284147723 wait complete. 2023-10-26T01:36:33.197 INFO:tasks.cephfs_test_runner:That when the full flag is encountered during asynchronous ... ok 2023-10-26T01:36:33.198 INFO:tasks.cephfs_test_runner:Starting test: test_full_same_file (tasks.cephfs.test_full.TestClusterFull) 2023-10-26T01:36:33.198 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:36:33.423 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:33.420+0000 7ff72a355640 1 -- 172.21.15.169:0/2703276711 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff710003970 msgr2=0x7ff710003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:33.423 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:33.420+0000 7ff72a355640 1 --2- 172.21.15.169:0/2703276711 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff710003970 0x7ff710003d90 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7ff714009b50 tx=0x7ff7100057f0 comp rx=0 tx=0).stop 2023-10-26T01:36:33.423 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:33.421+0000 7ff7288f2640 1 -- 172.21.15.169:0/2703276711 reap_dead start 2023-10-26T01:36:33.423 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:33.421+0000 7ff72a355640 1 -- 172.21.15.169:0/2703276711 shutdown_connections 2023-10-26T01:36:33.424 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:33.421+0000 7ff72a355640 1 -- 172.21.15.169:0/2703276711 >> 172.21.15.169:0/2703276711 conn(0x7ff724069930 msgr2=0x7ff724069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:33.424 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:33.421+0000 7ff72a355640 1 -- 172.21.15.169:0/2703276711 shutdown_connections 2023-10-26T01:36:33.424 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:33.421+0000 7ff72a355640 1 -- 172.21.15.169:0/2703276711 wait complete. 2023-10-26T01:36:33.425 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:33.422+0000 7ff72a355640 1 Processor -- start 2023-10-26T01:36:33.425 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:33.423+0000 7ff72a355640 1 -- start start 2023-10-26T01:36:33.426 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:33.423+0000 7ff72a355640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff710003970 0x7ff7240c7c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:33.426 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:33.423+0000 7ff72a355640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff72414b050 0x7ff724070ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:33.426 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:33.423+0000 7ff72a355640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff724071420 0x7ff7240c8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:33.426 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:33.423+0000 7ff72a355640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff724071bc0 con 0x7ff724071420 2023-10-26T01:36:33.426 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:33.423+0000 7ff72a355640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7240c77c0 con 0x7ff710003970 2023-10-26T01:36:33.427 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:33.423+0000 7ff72a355640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff72414b4d0 con 0x7ff72414b050 2023-10-26T01:36:33.427 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:33.424+0000 7ff7288f2640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff724071420 0x7ff7240c8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:33.427 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:33.424+0000 7ff7288f2640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff724071420 0x7ff7240c8460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:52166/0 (socket says 172.21.15.169:52166) 2023-10-26T01:36:33.427 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:33.424+0000 7ff7288f2640 1 -- 172.21.15.169:0/3573096673 learned_addr learned my addr 172.21.15.169:0/3573096673 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:36:33.428 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:33.424+0000 7ff7237fe640 1 --2- 172.21.15.169:0/3573096673 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff72414b050 0x7ff724070ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:33.429 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:33.424+0000 7ff7288f2640 1 -- 172.21.15.169:0/3573096673 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff72414b050 msgr2=0x7ff724070ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:33.429 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:33.424+0000 7ff7288f2640 1 --2- 172.21.15.169:0/3573096673 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff72414b050 0x7ff724070ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:33.429 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:33.424+0000 7ff7288f2640 1 -- 172.21.15.169:0/3573096673 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff710003970 msgr2=0x7ff7240c7c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:33.429 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:33.424+0000 7ff7288f2640 1 --2- 172.21.15.169:0/3573096673 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff710003970 0x7ff7240c7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:33.429 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:33.424+0000 7ff7288f2640 1 -- 172.21.15.169:0/3573096673 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7240718e0 con 0x7ff724071420 2023-10-26T01:36:33.430 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:33.424+0000 7ff7288f2640 1 --2- 172.21.15.169:0/3573096673 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff724071420 0x7ff7240c8460 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7ff7240ab9b0 tx=0x7ff71c00edd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:33.430 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:33.425+0000 7ff720ff9640 1 -- 172.21.15.169:0/3573096673 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff71c004d20 con 0x7ff724071420 2023-10-26T01:36:33.430 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:33.425+0000 7ff720ff9640 1 -- 172.21.15.169:0/3573096673 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff71c00cad0 con 0x7ff724071420 2023-10-26T01:36:33.430 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:33.425+0000 7ff72a355640 1 -- 172.21.15.169:0/3573096673 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7240c7170 con 0x7ff724071420 2023-10-26T01:36:33.430 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:33.425+0000 7ff720ff9640 1 -- 172.21.15.169:0/3573096673 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff71c00cec0 con 0x7ff724071420 2023-10-26T01:36:33.431 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:33.425+0000 7ff72a355640 1 -- 172.21.15.169:0/3573096673 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7240c69c0 con 0x7ff724071420 2023-10-26T01:36:33.431 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:33.426+0000 7ff72a355640 1 -- 172.21.15.169:0/3573096673 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7240c6fd0 con 0x7ff724071420 2023-10-26T01:36:33.431 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:33.427+0000 7ff720ff9640 1 -- 172.21.15.169:0/3573096673 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ff71c003970 con 0x7ff724071420 2023-10-26T01:36:33.432 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:33.428+0000 7ff720ff9640 1 --2- 172.21.15.169:0/3573096673 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff70406ae40 0x7ff70406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:33.432 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:33.428+0000 7ff720ff9640 1 -- 172.21.15.169:0/3573096673 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 4943+0+0 (secure 0 0 0) 0x7ff71c014070 con 0x7ff724071420 2023-10-26T01:36:33.432 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:33.428+0000 7ff723fff640 1 --2- 172.21.15.169:0/3573096673 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff70406ae40 0x7ff70406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:33.432 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:33.429+0000 7ff723fff640 1 --2- 172.21.15.169:0/3573096673 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff70406ae40 0x7ff70406d300 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7ff714036000 tx=0x7ff714035040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:33.434 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:33.432+0000 7ff720ff9640 1 -- 172.21.15.169:0/3573096673 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff7240c6fd0 con 0x7ff724071420 2023-10-26T01:36:33.624 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:33.622+0000 7ff72a355640 1 -- 172.21.15.169:0/3573096673 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff7240c6fd0 con 0x7ff724071420 2023-10-26T01:36:33.626 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:33.624+0000 7ff720ff9640 1 -- 172.21.15.169:0/3573096673 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+13037 (secure 0 0 0) 0x7ff7240c6fd0 con 0x7ff724071420 2023-10-26T01:36:33.626 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:36:33.627 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":94,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:36:32.141877+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.169:6835/4004306080":"2023-10-27T01:36:29.024881+0000","172.21.15.169:6834/4004306080":"2023-10-27T01:36:29.024881+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:36:33.629 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:33.626+0000 7ff72a355640 1 -- 172.21.15.169:0/3573096673 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff70406ae40 msgr2=0x7ff70406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:33.629 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:33.626+0000 7ff72a355640 1 --2- 172.21.15.169:0/3573096673 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff70406ae40 0x7ff70406d300 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7ff714036000 tx=0x7ff714035040 comp rx=0 tx=0).stop 2023-10-26T01:36:33.629 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:33.627+0000 7ff72a355640 1 -- 172.21.15.169:0/3573096673 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff724071420 msgr2=0x7ff7240c8460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:33.629 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:33.627+0000 7ff72a355640 1 --2- 172.21.15.169:0/3573096673 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff724071420 0x7ff7240c8460 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7ff7240ab9b0 tx=0x7ff71c00edd0 comp rx=0 tx=0).stop 2023-10-26T01:36:33.629 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:33.627+0000 7ff72a355640 1 -- 172.21.15.169:0/3573096673 shutdown_connections 2023-10-26T01:36:33.630 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:33.627+0000 7ff72a355640 1 --2- 172.21.15.169:0/3573096673 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff70406ae40 0x7ff70406d300 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:33.630 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:33.627+0000 7ff72a355640 1 --2- 172.21.15.169:0/3573096673 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff724071420 0x7ff7240c8460 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:33.630 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:33.627+0000 7ff72a355640 1 --2- 172.21.15.169:0/3573096673 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff72414b050 0x7ff724070ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:33.630 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:33.627+0000 7ff72a355640 1 --2- 172.21.15.169:0/3573096673 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff710003970 0x7ff7240c7c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:33.631 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:33.627+0000 7ff72a355640 1 -- 172.21.15.169:0/3573096673 >> 172.21.15.169:0/3573096673 conn(0x7ff724069930 msgr2=0x7ff72414cd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:33.631 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:33.627+0000 7ff72a355640 1 -- 172.21.15.169:0/3573096673 shutdown_connections 2023-10-26T01:36:33.631 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:33.627+0000 7ff72a355640 1 -- 172.21.15.169:0/3573096673 wait complete. 2023-10-26T01:36:33.648 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T01:30:32.200808+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T01:36:33.649 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T01:36:33.874 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:33.872+0000 7fa5b2ac3640 1 -- 172.21.15.169:0/1457908336 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa5ac15b790 msgr2=0x7fa5ac0cc5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:33.874 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:33.872+0000 7fa5b2ac3640 1 --2- 172.21.15.169:0/1457908336 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa5ac15b790 0x7fa5ac0cc5f0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fa5a8009830 tx=0x7fa5a802bdf0 comp rx=0 tx=0).stop 2023-10-26T01:36:33.874 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:33.872+0000 7fa5b2ac3640 1 -- 172.21.15.169:0/1457908336 shutdown_connections 2023-10-26T01:36:33.875 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:33.872+0000 7fa5b2ac3640 1 --2- 172.21.15.169:0/1457908336 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa5ac15b790 0x7fa5ac0cc5f0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:33.875 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:33.872+0000 7fa5b2ac3640 1 -- 172.21.15.169:0/1457908336 >> 172.21.15.169:0/1457908336 conn(0x7fa5ac0b2f10 msgr2=0x7fa5ac0b3320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:33.875 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:33.872+0000 7fa5b2ac3640 1 -- 172.21.15.169:0/1457908336 shutdown_connections 2023-10-26T01:36:33.875 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:33.872+0000 7fa5b2ac3640 1 -- 172.21.15.169:0/1457908336 wait complete. 2023-10-26T01:36:33.876 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:33.874+0000 7fa5b2ac3640 1 Processor -- start 2023-10-26T01:36:33.876 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:33.874+0000 7fa5b2ac3640 1 -- start start 2023-10-26T01:36:33.878 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:33.875+0000 7fa5b2ac3640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa5ac0ce450 0x7fa5ac14c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:33.878 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:33.875+0000 7fa5b2ac3640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa5ac0d0a30 0x7fa5ac1e4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:33.878 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:33.875+0000 7fa5b2ac3640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa5ac15b790 0x7fa5ac0b5e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:33.878 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:33.875+0000 7fa5b2ac3640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5ac0c8450 con 0x7fa5ac15b790 2023-10-26T01:36:33.879 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:33.875+0000 7fa5b2ac3640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5ac14cbf0 con 0x7fa5ac0ce450 2023-10-26T01:36:33.879 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:33.875+0000 7fa5b2ac3640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5ac1e5c70 con 0x7fa5ac0d0a30 2023-10-26T01:36:33.879 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:33.875+0000 7fa5b22c2640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa5ac15b790 0x7fa5ac0b5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:33.879 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:33.875+0000 7fa5b22c2640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa5ac15b790 0x7fa5ac0b5e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:52188/0 (socket says 172.21.15.169:52188) 2023-10-26T01:36:33.879 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:33.875+0000 7fa5b22c2640 1 -- 172.21.15.169:0/806659823 learned_addr learned my addr 172.21.15.169:0/806659823 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:36:33.880 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:33.876+0000 7fa5b22c2640 1 -- 172.21.15.169:0/806659823 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa5ac0d0a30 msgr2=0x7fa5ac1e4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:33.880 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:33.876+0000 7fa5b22c2640 1 --2- 172.21.15.169:0/806659823 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa5ac0d0a30 0x7fa5ac1e4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:33.880 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:33.876+0000 7fa5b22c2640 1 -- 172.21.15.169:0/806659823 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa5ac0ce450 msgr2=0x7fa5ac14c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:33.880 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:33.876+0000 7fa5b22c2640 1 --2- 172.21.15.169:0/806659823 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa5ac0ce450 0x7fa5ac14c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:33.880 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:33.876+0000 7fa5b22c2640 1 -- 172.21.15.169:0/806659823 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5a8009490 con 0x7fa5ac15b790 2023-10-26T01:36:33.881 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:33.876+0000 7fa5b22c2640 1 --2- 172.21.15.169:0/806659823 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa5ac15b790 0x7fa5ac0b5e30 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fa5a400cbc0 tx=0x7fa5a400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:33.881 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:33.876+0000 7fa5a27fc640 1 -- 172.21.15.169:0/806659823 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a400aca0 con 0x7fa5ac15b790 2023-10-26T01:36:33.881 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:33.876+0000 7fa5a27fc640 1 -- 172.21.15.169:0/806659823 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa5a4012420 con 0x7fa5ac15b790 2023-10-26T01:36:33.881 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:33.876+0000 7fa5b2ac3640 1 -- 172.21.15.169:0/806659823 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5ac1e56b0 con 0x7fa5ac15b790 2023-10-26T01:36:33.881 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:33.876+0000 7fa5a27fc640 1 -- 172.21.15.169:0/806659823 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a4012830 con 0x7fa5ac15b790 2023-10-26T01:36:33.882 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:33.876+0000 7fa5b2ac3640 1 -- 172.21.15.169:0/806659823 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5ac0c6630 con 0x7fa5ac15b790 2023-10-26T01:36:33.883 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:33.878+0000 7fa5b2ac3640 1 -- 172.21.15.169:0/806659823 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5ac0c6490 con 0x7fa5ac15b790 2023-10-26T01:36:33.883 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:33.878+0000 7fa5a27fc640 1 -- 172.21.15.169:0/806659823 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fa5a401b400 con 0x7fa5ac15b790 2023-10-26T01:36:33.883 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:33.879+0000 7fa5a27fc640 1 --2- 172.21.15.169:0/806659823 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa59006ae40 0x7fa59006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:33.883 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:33.879+0000 7fa5a27fc640 1 -- 172.21.15.169:0/806659823 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fa5a4089df0 con 0x7fa5ac15b790 2023-10-26T01:36:33.883 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:33.880+0000 7fa5b1ac1640 1 --2- 172.21.15.169:0/806659823 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa59006ae40 0x7fa59006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:33.884 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:33.881+0000 7fa5b1ac1640 1 --2- 172.21.15.169:0/806659823 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa59006ae40 0x7fa59006d300 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fa5a8036000 tx=0x7fa5a80046c0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:33.885 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:33.884+0000 7fa5a27fc640 1 -- 172.21.15.169:0/806659823 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa5ac0c6490 con 0x7fa5ac15b790 2023-10-26T01:36:34.078 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:34.077+0000 7fa5b2ac3640 1 -- 172.21.15.169:0/806659823 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa5ac0c6490 con 0x7fa5ac15b790 2023-10-26T01:36:34.079 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:34.077+0000 7fa5a27fc640 1 -- 172.21.15.169:0/806659823 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa5ac0c6490 con 0x7fa5ac15b790 2023-10-26T01:36:34.079 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 1 2023-10-26T01:36:34.082 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:34.080+0000 7fa5b2ac3640 1 -- 172.21.15.169:0/806659823 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa59006ae40 msgr2=0x7fa59006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:34.082 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:34.080+0000 7fa5b2ac3640 1 --2- 172.21.15.169:0/806659823 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa59006ae40 0x7fa59006d300 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fa5a8036000 tx=0x7fa5a80046c0 comp rx=0 tx=0).stop 2023-10-26T01:36:34.082 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:34.080+0000 7fa5b2ac3640 1 -- 172.21.15.169:0/806659823 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa5ac15b790 msgr2=0x7fa5ac0b5e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:34.083 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:34.080+0000 7fa5b2ac3640 1 --2- 172.21.15.169:0/806659823 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa5ac15b790 0x7fa5ac0b5e30 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fa5a400cbc0 tx=0x7fa5a400a060 comp rx=0 tx=0).stop 2023-10-26T01:36:34.083 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:34.080+0000 7fa5b2ac3640 1 -- 172.21.15.169:0/806659823 shutdown_connections 2023-10-26T01:36:34.083 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:34.080+0000 7fa5b2ac3640 1 --2- 172.21.15.169:0/806659823 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa59006ae40 0x7fa59006d300 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:34.083 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:34.080+0000 7fa5b2ac3640 1 --2- 172.21.15.169:0/806659823 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa5ac15b790 0x7fa5ac0b5e30 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:34.083 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:34.080+0000 7fa5b2ac3640 1 --2- 172.21.15.169:0/806659823 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa5ac0d0a30 0x7fa5ac1e4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:34.084 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:34.080+0000 7fa5b2ac3640 1 --2- 172.21.15.169:0/806659823 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa5ac0ce450 0x7fa5ac14c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:34.084 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:34.080+0000 7fa5b2ac3640 1 -- 172.21.15.169:0/806659823 >> 172.21.15.169:0/806659823 conn(0x7fa5ac0b2f10 msgr2=0x7fa5ac0b0f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:34.084 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:34.081+0000 7fa5b2ac3640 1 -- 172.21.15.169:0/806659823 shutdown_connections 2023-10-26T01:36:34.084 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:34.081+0000 7fa5b2ac3640 1 -- 172.21.15.169:0/806659823 wait complete. 2023-10-26T01:36:34.102 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestClusterFull.test_full_same_file' 2023-10-26T01:36:34.326 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:34.324+0000 7f923359e640 1 -- 172.21.15.169:0/31955381 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9220005180 msgr2=0x7f922000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:34.326 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:34.324+0000 7f923359e640 1 --2- 172.21.15.169:0/31955381 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9220005180 0x7f922000b980 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f92340b4e90 tx=0x7f92200057f0 comp rx=0 tx=0).stop 2023-10-26T01:36:34.327 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:34.324+0000 7f9232d9d640 1 -- 172.21.15.169:0/31955381 reap_dead start 2023-10-26T01:36:34.327 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:34.324+0000 7f923359e640 1 -- 172.21.15.169:0/31955381 shutdown_connections 2023-10-26T01:36:34.327 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:34.324+0000 7f923359e640 1 -- 172.21.15.169:0/31955381 >> 172.21.15.169:0/31955381 conn(0x7f92340d1360 msgr2=0x7f92340d1770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:34.327 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:34.324+0000 7f923359e640 1 -- 172.21.15.169:0/31955381 shutdown_connections 2023-10-26T01:36:34.327 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:34.324+0000 7f923359e640 1 -- 172.21.15.169:0/31955381 wait complete. 2023-10-26T01:36:34.328 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:34.326+0000 7f923359e640 1 Processor -- start 2023-10-26T01:36:34.328 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:34.326+0000 7f923359e640 1 -- start start 2023-10-26T01:36:34.330 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:34.327+0000 7f923359e640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9220005180 0x7f92340a7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:34.330 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:34.327+0000 7f923359e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f923415d350 0x7f92340a9f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:34.330 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:34.327+0000 7f923359e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f92340aa470 0x7f923415e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:34.331 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:34.327+0000 7f923359e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92341f1de0 con 0x7f923415d350 2023-10-26T01:36:34.331 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:34.327+0000 7f923359e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92340b3aa0 con 0x7f92340aa470 2023-10-26T01:36:34.331 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:34.327+0000 7f923359e640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92340b3200 con 0x7f9220005180 2023-10-26T01:36:34.331 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:34.327+0000 7f9231d9b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f923415d350 0x7f92340a9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:34.331 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:34.327+0000 7f9231d9b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f923415d350 0x7f92340a9f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:58630/0 (socket says 172.21.15.169:58630) 2023-10-26T01:36:34.332 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:34.327+0000 7f9231d9b640 1 -- 172.21.15.169:0/1563692266 learned_addr learned my addr 172.21.15.169:0/1563692266 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:36:34.332 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:34.328+0000 7f923259c640 1 --2- 172.21.15.169:0/1563692266 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9220005180 0x7f92340a7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:34.332 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:34.328+0000 7f9231d9b640 1 -- 172.21.15.169:0/1563692266 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9220005180 msgr2=0x7f92340a7ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:34.332 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:34.328+0000 7f9231d9b640 1 --2- 172.21.15.169:0/1563692266 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9220005180 0x7f92340a7ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:34.332 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:34.328+0000 7f9231d9b640 1 -- 172.21.15.169:0/1563692266 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f92340aa470 msgr2=0x7f923415e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:34.332 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:34.328+0000 7f9232d9d640 1 --2- 172.21.15.169:0/1563692266 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f92340aa470 0x7f923415e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:34.333 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:34.328+0000 7f9231d9b640 1 --2- 172.21.15.169:0/1563692266 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f92340aa470 0x7f923415e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:34.333 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:34.328+0000 7f9231d9b640 1 -- 172.21.15.169:0/1563692266 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92340b34e0 con 0x7f923415d350 2023-10-26T01:36:34.333 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:34.328+0000 7f9231d9b640 1 --2- 172.21.15.169:0/1563692266 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f923415d350 0x7f92340a9f30 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f921c005fd0 tx=0x7f921c00db60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:34.333 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:34.328+0000 7f922affd640 1 -- 172.21.15.169:0/1563692266 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f921c004220 con 0x7f923415d350 2023-10-26T01:36:34.333 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:34.328+0000 7f922affd640 1 -- 172.21.15.169:0/1563692266 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f921c0044b0 con 0x7f923415d350 2023-10-26T01:36:34.334 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:34.328+0000 7f922affd640 1 -- 172.21.15.169:0/1563692266 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f921c009a00 con 0x7f923415d350 2023-10-26T01:36:34.334 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:34.328+0000 7f923359e640 1 -- 172.21.15.169:0/1563692266 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92340b37c0 con 0x7f923415d350 2023-10-26T01:36:34.334 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:34.329+0000 7f923359e640 1 -- 172.21.15.169:0/1563692266 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92340b2e20 con 0x7f923415d350 2023-10-26T01:36:34.335 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:34.330+0000 7f923359e640 1 -- 172.21.15.169:0/1563692266 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92340b2fb0 con 0x7f923415d350 2023-10-26T01:36:34.335 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:34.331+0000 7f922affd640 1 -- 172.21.15.169:0/1563692266 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f921c009b60 con 0x7f923415d350 2023-10-26T01:36:34.335 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:34.331+0000 7f922affd640 1 --2- 172.21.15.169:0/1563692266 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f921406ae40 0x7f921406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:34.336 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:34.332+0000 7f922affd640 1 -- 172.21.15.169:0/1563692266 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f921c0892f0 con 0x7f923415d350 2023-10-26T01:36:34.336 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:34.332+0000 7f923259c640 1 --2- 172.21.15.169:0/1563692266 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f921406ae40 0x7f921406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:34.336 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:34.333+0000 7f923259c640 1 --2- 172.21.15.169:0/1563692266 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f921406ae40 0x7f921406d300 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f9224009270 tx=0x7f92240093c0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:34.337 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:34.336+0000 7f922affd640 1 -- 172.21.15.169:0/1563692266 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f921c054730 con 0x7f923415d350 2023-10-26T01:36:34.596 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:34.594+0000 7f923359e640 1 -- 172.21.15.169:0/1563692266 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_same_file"]} v 0) v1 -- 0x7f9234073c10 con 0x7f923415d350 2023-10-26T01:36:35.185 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:35.183+0000 7f922affd640 1 -- 172.21.15.169:0/1563692266 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_same_file"]}]=0 v302) v1 ==== 140+0+0 (secure 0 0 0) 0x7f921c057d50 con 0x7f923415d350 2023-10-26T01:36:35.188 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:35.186+0000 7f923359e640 1 -- 172.21.15.169:0/1563692266 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f921406ae40 msgr2=0x7f921406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:35.189 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:35.186+0000 7f923359e640 1 --2- 172.21.15.169:0/1563692266 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f921406ae40 0x7f921406d300 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f9224009270 tx=0x7f92240093c0 comp rx=0 tx=0).stop 2023-10-26T01:36:35.189 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:35.186+0000 7f923359e640 1 -- 172.21.15.169:0/1563692266 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f923415d350 msgr2=0x7f92340a9f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:35.189 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:35.186+0000 7f923359e640 1 --2- 172.21.15.169:0/1563692266 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f923415d350 0x7f92340a9f30 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f921c005fd0 tx=0x7f921c00db60 comp rx=0 tx=0).stop 2023-10-26T01:36:35.189 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:35.187+0000 7f923359e640 1 -- 172.21.15.169:0/1563692266 shutdown_connections 2023-10-26T01:36:35.189 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:35.187+0000 7f923359e640 1 --2- 172.21.15.169:0/1563692266 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f921406ae40 0x7f921406d300 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:35.190 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:35.187+0000 7f923359e640 1 --2- 172.21.15.169:0/1563692266 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f923415d350 0x7f92340a9f30 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:35.190 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:35.187+0000 7f923359e640 1 --2- 172.21.15.169:0/1563692266 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f92340aa470 0x7f923415e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:35.190 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:35.187+0000 7f923359e640 1 --2- 172.21.15.169:0/1563692266 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9220005180 0x7f92340a7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:35.190 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:35.187+0000 7f923359e640 1 -- 172.21.15.169:0/1563692266 >> 172.21.15.169:0/1563692266 conn(0x7f92340d1360 msgr2=0x7f923415d850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:35.190 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:35.187+0000 7f923359e640 1 -- 172.21.15.169:0/1563692266 shutdown_connections 2023-10-26T01:36:35.191 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:35.187+0000 7f923359e640 1 -- 172.21.15.169:0/1563692266 wait complete. 2023-10-26T01:36:35.210 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_objectstore 2023-10-26T01:36:35.431 INFO:teuthology.orchestra.run.smithi169.stdout:{"osd_objectstore":"memstore"} 2023-10-26T01:36:35.432 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_objectstore": "memstore" } 2023-10-26T01:36:35.432 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2023-10-26T01:36:35.647 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:35.645+0000 7fd68529b640 1 -- 172.21.15.169:0/2856295929 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd66c004d00 msgr2=0x7fd66c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:35.647 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:35.645+0000 7fd68529b640 1 --2- 172.21.15.169:0/2856295929 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd66c004d00 0x7fd66c005180 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fd67800c610 tx=0x7fd66c0057f0 comp rx=0 tx=0).stop 2023-10-26T01:36:35.648 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:35.645+0000 7fd67f7fe640 1 -- 172.21.15.169:0/2856295929 reap_dead start 2023-10-26T01:36:35.648 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:35.645+0000 7fd68529b640 1 -- 172.21.15.169:0/2856295929 shutdown_connections 2023-10-26T01:36:35.648 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:35.645+0000 7fd68529b640 1 -- 172.21.15.169:0/2856295929 >> 172.21.15.169:0/2856295929 conn(0x7fd680069930 msgr2=0x7fd680069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:35.648 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:35.645+0000 7fd68529b640 1 -- 172.21.15.169:0/2856295929 shutdown_connections 2023-10-26T01:36:35.648 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:35.645+0000 7fd68529b640 1 -- 172.21.15.169:0/2856295929 wait complete. 2023-10-26T01:36:35.649 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:35.647+0000 7fd68529b640 1 Processor -- start 2023-10-26T01:36:35.649 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:35.647+0000 7fd68529b640 1 -- start start 2023-10-26T01:36:35.650 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:35.648+0000 7fd68529b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd66c004d00 0x7fd6800af280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:35.650 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:35.648+0000 7fd68529b640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd6801efac0 0x7fd68009a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:35.651 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:35.648+0000 7fd68529b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6800ae8e0 0x7fd680099240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:35.651 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:35.648+0000 7fd68529b640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6800984c0 con 0x7fd6800ae8e0 2023-10-26T01:36:35.651 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:35.648+0000 7fd68529b640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd680098760 con 0x7fd66c004d00 2023-10-26T01:36:35.651 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:35.648+0000 7fd68529b640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd680098a40 con 0x7fd6801efac0 2023-10-26T01:36:35.651 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:35.648+0000 7fd67f7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6800ae8e0 0x7fd680099240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:35.652 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:35.648+0000 7fd67f7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6800ae8e0 0x7fd680099240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:58650/0 (socket says 172.21.15.169:58650) 2023-10-26T01:36:35.652 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:35.648+0000 7fd67f7fe640 1 -- 172.21.15.169:0/2443690879 learned_addr learned my addr 172.21.15.169:0/2443690879 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:36:35.652 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:35.649+0000 7fd67f7fe640 1 -- 172.21.15.169:0/2443690879 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd6801efac0 msgr2=0x7fd68009a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:35.652 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:35.649+0000 7fd67f7fe640 1 --2- 172.21.15.169:0/2443690879 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd6801efac0 0x7fd68009a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:35.652 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:35.649+0000 7fd67f7fe640 1 -- 172.21.15.169:0/2443690879 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd66c004d00 msgr2=0x7fd6800af280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:35.652 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:35.649+0000 7fd67f7fe640 1 --2- 172.21.15.169:0/2443690879 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd66c004d00 0x7fd6800af280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:35.653 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:35.649+0000 7fd67f7fe640 1 -- 172.21.15.169:0/2443690879 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd680098d20 con 0x7fd6800ae8e0 2023-10-26T01:36:35.654 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:35.649+0000 7fd67f7fe640 1 --2- 172.21.15.169:0/2443690879 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6800ae8e0 0x7fd680099240 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fd67802ccc0 tx=0x7fd678001320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:35.654 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:35.649+0000 7fd6777fe640 1 -- 172.21.15.169:0/2443690879 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd678001890 con 0x7fd6800ae8e0 2023-10-26T01:36:35.654 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:35.649+0000 7fd6777fe640 1 -- 172.21.15.169:0/2443690879 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd6780019f0 con 0x7fd6800ae8e0 2023-10-26T01:36:35.654 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:35.649+0000 7fd6777fe640 1 -- 172.21.15.169:0/2443690879 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd678003bf0 con 0x7fd6800ae8e0 2023-10-26T01:36:35.654 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:35.649+0000 7fd68529b640 1 -- 172.21.15.169:0/2443690879 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd68009b240 con 0x7fd6800ae8e0 2023-10-26T01:36:35.655 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:35.650+0000 7fd68529b640 1 -- 172.21.15.169:0/2443690879 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd68009ae80 con 0x7fd6800ae8e0 2023-10-26T01:36:35.655 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:35.651+0000 7fd68529b640 1 -- 172.21.15.169:0/2443690879 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd680098f10 con 0x7fd6800ae8e0 2023-10-26T01:36:35.655 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:35.652+0000 7fd6777fe640 1 -- 172.21.15.169:0/2443690879 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fd678031030 con 0x7fd6800ae8e0 2023-10-26T01:36:35.656 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:35.653+0000 7fd6777fe640 1 --2- 172.21.15.169:0/2443690879 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd66006ae40 0x7fd66006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:35.656 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:35.653+0000 7fd6777fe640 1 -- 172.21.15.169:0/2443690879 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fd6780aa0a0 con 0x7fd6800ae8e0 2023-10-26T01:36:35.656 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:35.653+0000 7fd67effd640 1 --2- 172.21.15.169:0/2443690879 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd66006ae40 0x7fd66006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:35.656 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:35.654+0000 7fd67effd640 1 --2- 172.21.15.169:0/2443690879 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd66006ae40 0x7fd66006d300 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fd670005d80 tx=0x7fd670005c90 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:35.658 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:35.657+0000 7fd6777fe640 1 -- 172.21.15.169:0/2443690879 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd678078ee0 con 0x7fd6800ae8e0 2023-10-26T01:36:35.850 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T01:36:35.851 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:35.848+0000 7fd68529b640 1 -- 172.21.15.169:0/2443690879 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "config dump"} v 0) v1 -- 0x7fd6801586f0 con 0x7fd6800ae8e0 2023-10-26T01:36:35.851 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:35.849+0000 7fd6777fe640 1 -- 172.21.15.169:0/2443690879 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v26) v1 ==== 59+0+1116 (secure 0 0 0) 0x7fd678043470 con 0x7fd6800ae8e0 2023-10-26T01:36:35.852 INFO:teuthology.orchestra.run.smithi169.stdout:WHO MASK LEVEL OPTION VALUE RO 2023-10-26T01:36:35.852 INFO:teuthology.orchestra.run.smithi169.stdout:mon advanced mon_op_complaint_time 120 2023-10-26T01:36:35.852 INFO:teuthology.orchestra.run.smithi169.stdout:osd advanced osd_op_complaint_time 180.000000 2023-10-26T01:36:35.852 INFO:teuthology.orchestra.run.smithi169.stdout:mds advanced debug_mds 20/20 2023-10-26T01:36:35.853 INFO:teuthology.orchestra.run.smithi169.stdout:mds advanced debug_mds_balancer 20/20 2023-10-26T01:36:35.853 INFO:teuthology.orchestra.run.smithi169.stdout:mds advanced debug_ms 1/1 2023-10-26T01:36:35.853 INFO:teuthology.orchestra.run.smithi169.stdout:mds dev mds_debug_frag true 2023-10-26T01:36:35.853 INFO:teuthology.orchestra.run.smithi169.stdout:mds dev mds_debug_scatterstat true 2023-10-26T01:36:35.853 INFO:teuthology.orchestra.run.smithi169.stdout:mds advanced mds_op_complaint_time 180.000000 2023-10-26T01:36:35.854 INFO:teuthology.orchestra.run.smithi169.stdout:mds dev mds_verify_scatter true 2023-10-26T01:36:35.854 INFO:teuthology.orchestra.run.smithi169.stdout:mds advanced osd_op_complaint_time 180.000000 2023-10-26T01:36:35.854 INFO:teuthology.orchestra.run.smithi169.stdout:mds advanced rados_mon_op_timeout 900 2023-10-26T01:36:35.854 INFO:teuthology.orchestra.run.smithi169.stdout:mds advanced rados_osd_op_timeout 900 2023-10-26T01:36:35.854 INFO:teuthology.orchestra.run.smithi169.stdout:client advanced client_mount_timeout 600 2023-10-26T01:36:35.855 INFO:teuthology.orchestra.run.smithi169.stdout:client advanced debug_client 20/20 2023-10-26T01:36:35.855 INFO:teuthology.orchestra.run.smithi169.stdout:client advanced debug_ms 1/1 2023-10-26T01:36:35.855 INFO:teuthology.orchestra.run.smithi169.stdout:client advanced rados_mon_op_timeout 900 2023-10-26T01:36:35.855 INFO:teuthology.orchestra.run.smithi169.stdout:client advanced rados_osd_op_timeout 900 2023-10-26T01:36:35.856 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:35.852+0000 7fd68529b640 1 -- 172.21.15.169:0/2443690879 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd66006ae40 msgr2=0x7fd66006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:35.856 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:35.852+0000 7fd68529b640 1 --2- 172.21.15.169:0/2443690879 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd66006ae40 0x7fd66006d300 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fd670005d80 tx=0x7fd670005c90 comp rx=0 tx=0).stop 2023-10-26T01:36:35.857 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:35.852+0000 7fd68529b640 1 -- 172.21.15.169:0/2443690879 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6800ae8e0 msgr2=0x7fd680099240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:35.857 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:35.852+0000 7fd68529b640 1 --2- 172.21.15.169:0/2443690879 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6800ae8e0 0x7fd680099240 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fd67802ccc0 tx=0x7fd678001320 comp rx=0 tx=0).stop 2023-10-26T01:36:35.857 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:35.852+0000 7fd68529b640 1 -- 172.21.15.169:0/2443690879 shutdown_connections 2023-10-26T01:36:35.857 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:35.852+0000 7fd68529b640 1 --2- 172.21.15.169:0/2443690879 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd66006ae40 0x7fd66006d300 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:35.857 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:35.852+0000 7fd68529b640 1 --2- 172.21.15.169:0/2443690879 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6800ae8e0 0x7fd680099240 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:35.858 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:35.852+0000 7fd68529b640 1 --2- 172.21.15.169:0/2443690879 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd6801efac0 0x7fd68009a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:35.858 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:35.852+0000 7fd68529b640 1 --2- 172.21.15.169:0/2443690879 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd66c004d00 0x7fd6800af280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:35.858 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:35.852+0000 7fd68529b640 1 -- 172.21.15.169:0/2443690879 >> 172.21.15.169:0/2443690879 conn(0x7fd680069930 msgr2=0x7fd6800cba40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:35.858 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:35.852+0000 7fd68529b640 1 -- 172.21.15.169:0/2443690879 shutdown_connections 2023-10-26T01:36:35.858 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:35.852+0000 7fd68529b640 1 -- 172.21.15.169:0/2443690879 wait complete. 2023-10-26T01:36:35.859 DEBUG:teuthology.orchestra.run.smithi170:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T01:36:35.871 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-26T01:36:36.121 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:36.119+0000 7fe21974c640 1 -- 172.21.15.169:0/149178237 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2140cde20 msgr2=0x7fe214166630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:36.122 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:36.119+0000 7fe21974c640 1 --2- 172.21.15.169:0/149178237 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2140cde20 0x7fe214166630 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fe20c009830 tx=0x7fe20c02bcd0 comp rx=0 tx=0).stop 2023-10-26T01:36:36.122 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:36.119+0000 7fe21974c640 1 -- 172.21.15.169:0/149178237 shutdown_connections 2023-10-26T01:36:36.122 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:36.119+0000 7fe21974c640 1 --2- 172.21.15.169:0/149178237 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2140cde20 0x7fe214166630 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:36.122 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:36.119+0000 7fe21974c640 1 -- 172.21.15.169:0/149178237 >> 172.21.15.169:0/149178237 conn(0x7fe214069930 msgr2=0x7fe214069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:36.122 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:36.119+0000 7fe21974c640 1 -- 172.21.15.169:0/149178237 shutdown_connections 2023-10-26T01:36:36.123 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:36.119+0000 7fe21974c640 1 -- 172.21.15.169:0/149178237 wait complete. 2023-10-26T01:36:36.123 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:36.121+0000 7fe21974c640 1 Processor -- start 2023-10-26T01:36:36.123 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:36.121+0000 7fe21974c640 1 -- start start 2023-10-26T01:36:36.125 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:36.122+0000 7fe21974c640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe2140c9d00 0x7fe21414a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:36.125 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:36.122+0000 7fe21974c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe2140cde20 0x7fe21414aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:36.126 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:36.122+0000 7fe21974c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe21415b620 0x7fe2140adfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:36.126 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:36.122+0000 7fe21974c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe21406eeb0 con 0x7fe21415b620 2023-10-26T01:36:36.126 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:36.122+0000 7fe21974c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe21415e100 con 0x7fe2140cde20 2023-10-26T01:36:36.126 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:36.122+0000 7fe21974c640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe21415e3e0 con 0x7fe2140c9d00 2023-10-26T01:36:36.126 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:36.123+0000 7fe2137fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe21415b620 0x7fe2140adfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:36.127 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:36.123+0000 7fe2137fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe21415b620 0x7fe2140adfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:58658/0 (socket says 172.21.15.169:58658) 2023-10-26T01:36:36.127 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:36.123+0000 7fe2137fe640 1 -- 172.21.15.169:0/546314521 learned_addr learned my addr 172.21.15.169:0/546314521 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:36:36.127 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:36.123+0000 7fe212ffd640 1 --2- 172.21.15.169:0/546314521 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe2140c9d00 0x7fe21414a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:36.127 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:36.123+0000 7fe2137fe640 1 -- 172.21.15.169:0/546314521 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe2140c9d00 msgr2=0x7fe21414a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:36.127 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:36.123+0000 7fe2137fe640 1 --2- 172.21.15.169:0/546314521 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe2140c9d00 0x7fe21414a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:36.128 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:36.123+0000 7fe2137fe640 1 -- 172.21.15.169:0/546314521 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe2140cde20 msgr2=0x7fe21414aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:36.128 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:36.123+0000 7fe2127fc640 1 --2- 172.21.15.169:0/546314521 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe2140cde20 0x7fe21414aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:36.128 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:36.123+0000 7fe2137fe640 1 --2- 172.21.15.169:0/546314521 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe2140cde20 0x7fe21414aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:36.128 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:36.123+0000 7fe2137fe640 1 -- 172.21.15.169:0/546314521 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe20c009490 con 0x7fe21415b620 2023-10-26T01:36:36.129 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:36.123+0000 7fe2137fe640 1 --2- 172.21.15.169:0/546314521 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe21415b620 0x7fe2140adfa0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fe20c037040 tx=0x7fe20c02cdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:36.129 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:36.124+0000 7fe20b7fe640 1 -- 172.21.15.169:0/546314521 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe20c005220 con 0x7fe21415b620 2023-10-26T01:36:36.129 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:36.124+0000 7fe20b7fe640 1 -- 172.21.15.169:0/546314521 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe20c005470 con 0x7fe21415b620 2023-10-26T01:36:36.129 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:36.124+0000 7fe20b7fe640 1 -- 172.21.15.169:0/546314521 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe20c003a60 con 0x7fe21415b620 2023-10-26T01:36:36.129 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:36.124+0000 7fe21974c640 1 -- 172.21.15.169:0/546314521 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe21406f190 con 0x7fe21415b620 2023-10-26T01:36:36.129 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:36.124+0000 7fe21974c640 1 -- 172.21.15.169:0/546314521 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe21415c050 con 0x7fe21415b620 2023-10-26T01:36:36.130 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:36.125+0000 7fe21974c640 1 -- 172.21.15.169:0/546314521 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe21415e600 con 0x7fe21415b620 2023-10-26T01:36:36.131 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:36.126+0000 7fe20b7fe640 1 -- 172.21.15.169:0/546314521 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fe20c03b040 con 0x7fe21415b620 2023-10-26T01:36:36.131 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:36.127+0000 7fe20b7fe640 1 --2- 172.21.15.169:0/546314521 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe1f406ae40 0x7fe1f406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:36.131 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:36.127+0000 7fe20b7fe640 1 -- 172.21.15.169:0/546314521 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fe20c0ab070 con 0x7fe21415b620 2023-10-26T01:36:36.131 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:36.128+0000 7fe212ffd640 1 --2- 172.21.15.169:0/546314521 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe1f406ae40 0x7fe1f406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:36.131 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:36.129+0000 7fe212ffd640 1 --2- 172.21.15.169:0/546314521 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe1f406ae40 0x7fe1f406d300 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fe21414fe40 tx=0x7fe1fc009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:36.133 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:36.132+0000 7fe20b7fe640 1 -- 172.21.15.169:0/546314521 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe20c0764b0 con 0x7fe21415b620 2023-10-26T01:36:36.325 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:36.323+0000 7fe21974c640 1 -- 172.21.15.169:0/546314521 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fe21404db10 con 0x7fe21415b620 2023-10-26T01:36:36.330 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:36.328+0000 7fe20b7fe640 1 -- 172.21.15.169:0/546314521 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v27)=0 v27) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe20c005c90 con 0x7fe21415b620 2023-10-26T01:36:36.333 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:36.331+0000 7fe21974c640 1 -- 172.21.15.169:0/546314521 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe1f406ae40 msgr2=0x7fe1f406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:36.334 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:36.331+0000 7fe21974c640 1 --2- 172.21.15.169:0/546314521 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe1f406ae40 0x7fe1f406d300 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fe21414fe40 tx=0x7fe1fc009040 comp rx=0 tx=0).stop 2023-10-26T01:36:36.334 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:36.331+0000 7fe21974c640 1 -- 172.21.15.169:0/546314521 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe21415b620 msgr2=0x7fe2140adfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:36.334 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:36.331+0000 7fe21974c640 1 --2- 172.21.15.169:0/546314521 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe21415b620 0x7fe2140adfa0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fe20c037040 tx=0x7fe20c02cdf0 comp rx=0 tx=0).stop 2023-10-26T01:36:36.334 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:36.331+0000 7fe21974c640 1 -- 172.21.15.169:0/546314521 shutdown_connections 2023-10-26T01:36:36.334 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:36.331+0000 7fe21974c640 1 --2- 172.21.15.169:0/546314521 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe1f406ae40 0x7fe1f406d300 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:36.335 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:36.332+0000 7fe21974c640 1 --2- 172.21.15.169:0/546314521 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe21415b620 0x7fe2140adfa0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:36.335 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:36.332+0000 7fe21974c640 1 --2- 172.21.15.169:0/546314521 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe2140cde20 0x7fe21414aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:36.335 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:36.332+0000 7fe21974c640 1 --2- 172.21.15.169:0/546314521 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe2140c9d00 0x7fe21414a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:36.335 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:36.332+0000 7fe21974c640 1 -- 172.21.15.169:0/546314521 >> 172.21.15.169:0/546314521 conn(0x7fe214069930 msgr2=0x7fe214158c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:36.335 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:36.332+0000 7fe21974c640 1 -- 172.21.15.169:0/546314521 shutdown_connections 2023-10-26T01:36:36.336 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:36.332+0000 7fe21974c640 1 -- 172.21.15.169:0/546314521 wait complete. 2023-10-26T01:36:36.352 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-26T01:36:36.353 DEBUG:teuthology.orchestra.run.smithi169:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-26T01:36:36.469 DEBUG:teuthology.orchestra.run.smithi170:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-26T01:36:36.590 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-26T01:36:36.591 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:36:36.591 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:36:36.651 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054301,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-26T01:36:36.651 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-26T01:36:36.651 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:36:36.711 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054303,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-26T01:36:36.711 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:36:36.711 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:36:36.772 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054301,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-26T01:36:36.772 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-26T01:36:36.773 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:36:36.833 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054303,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-26T01:36:36.833 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-26T01:36:36.834 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T01:36:37.066 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:37.064+0000 7f75f93d0640 1 -- 172.21.15.169:0/4108159052 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f75f40cde00 msgr2=0x7f75f40cbac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:37.066 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:37.064+0000 7f75f93d0640 1 --2- 172.21.15.169:0/4108159052 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f75f40cde00 0x7f75f40cbac0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f75ec009830 tx=0x7f75ec02bcd0 comp rx=0 tx=0).stop 2023-10-26T01:36:37.067 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:37.064+0000 7f75f93d0640 1 -- 172.21.15.169:0/4108159052 shutdown_connections 2023-10-26T01:36:37.067 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:37.064+0000 7f75f93d0640 1 --2- 172.21.15.169:0/4108159052 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f75f40cde00 0x7f75f40cbac0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:37.067 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:37.064+0000 7f75f93d0640 1 --2- 172.21.15.169:0/4108159052 >> v2:172.21.15.170:3300/0 conn(0x7f75f41592d0 0x7f75f40aa3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:37.067 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:37.064+0000 7f75f93d0640 1 -- 172.21.15.169:0/4108159052 >> 172.21.15.169:0/4108159052 conn(0x7f75f4069930 msgr2=0x7f75f4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:37.067 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:37.064+0000 7f75f93d0640 1 -- 172.21.15.169:0/4108159052 shutdown_connections 2023-10-26T01:36:37.067 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:37.064+0000 7f75f93d0640 1 -- 172.21.15.169:0/4108159052 wait complete. 2023-10-26T01:36:37.068 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:37.066+0000 7f75f93d0640 1 Processor -- start 2023-10-26T01:36:37.068 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:37.066+0000 7f75f93d0640 1 -- start start 2023-10-26T01:36:37.070 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:37.067+0000 7f75f93d0640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75f40ac930 0x7f75f41f1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:37.070 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:37.067+0000 7f75f93d0640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f75f40cde00 0x7f75f4152080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:37.070 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:37.067+0000 7f75f93d0640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f75f41592d0 0x7f75f4144460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:37.070 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:37.067+0000 7f75f93d0640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75f41f3470 con 0x7f75f40ac930 2023-10-26T01:36:37.071 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:37.067+0000 7f75f93d0640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75f41f23d0 con 0x7f75f40cde00 2023-10-26T01:36:37.071 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:37.067+0000 7f75f93d0640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75f41f26b0 con 0x7f75f41592d0 2023-10-26T01:36:37.071 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:37.067+0000 7f75f2ffd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75f40ac930 0x7f75f41f1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:37.071 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:37.067+0000 7f75f2ffd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75f40ac930 0x7f75f41f1b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:58664/0 (socket says 172.21.15.169:58664) 2023-10-26T01:36:37.071 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:37.067+0000 7f75f2ffd640 1 -- 172.21.15.169:0/2249786325 learned_addr learned my addr 172.21.15.169:0/2249786325 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:36:37.072 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:37.068+0000 7f75f2ffd640 1 -- 172.21.15.169:0/2249786325 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f75f41592d0 msgr2=0x7f75f4144460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:37.072 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:37.068+0000 7f75f2ffd640 1 --2- 172.21.15.169:0/2249786325 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f75f41592d0 0x7f75f4144460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:37.072 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:37.068+0000 7f75f2ffd640 1 -- 172.21.15.169:0/2249786325 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f75f40cde00 msgr2=0x7f75f4152080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:37.072 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:37.068+0000 7f75f2ffd640 1 --2- 172.21.15.169:0/2249786325 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f75f40cde00 0x7f75f4152080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:37.073 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:37.068+0000 7f75f2ffd640 1 -- 172.21.15.169:0/2249786325 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75ec009490 con 0x7f75f40ac930 2023-10-26T01:36:37.073 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:37.068+0000 7f75f2ffd640 1 --2- 172.21.15.169:0/2249786325 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75f40ac930 0x7f75f41f1b80 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f75e400e790 tx=0x7f75e400ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:37.073 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:37.068+0000 7f75ebfff640 1 -- 172.21.15.169:0/2249786325 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e4009580 con 0x7f75f40ac930 2023-10-26T01:36:37.073 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:37.068+0000 7f75ebfff640 1 -- 172.21.15.169:0/2249786325 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f75e400cdb0 con 0x7f75f40ac930 2023-10-26T01:36:37.073 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:37.068+0000 7f75ebfff640 1 -- 172.21.15.169:0/2249786325 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e4010690 con 0x7f75f40ac930 2023-10-26T01:36:37.074 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:37.068+0000 7f75f93d0640 1 -- 172.21.15.169:0/2249786325 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75f41f2990 con 0x7f75f40ac930 2023-10-26T01:36:37.074 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:37.068+0000 7f75f93d0640 1 -- 172.21.15.169:0/2249786325 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75f41f2f50 con 0x7f75f40ac930 2023-10-26T01:36:37.077 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:37.070+0000 7f75f93d0640 1 -- 172.21.15.169:0/2249786325 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75f41f3110 con 0x7f75f40ac930 2023-10-26T01:36:37.077 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:37.071+0000 7f75ebfff640 1 -- 172.21.15.169:0/2249786325 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f75e40107f0 con 0x7f75f40ac930 2023-10-26T01:36:37.077 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:37.072+0000 7f75ebfff640 1 --2- 172.21.15.169:0/2249786325 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f75d406ae40 0x7f75d406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:37.077 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:37.072+0000 7f75ebfff640 1 -- 172.21.15.169:0/2249786325 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f75e4014070 con 0x7f75f40ac930 2023-10-26T01:36:37.078 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:37.073+0000 7f75ea7fc640 1 --2- 172.21.15.169:0/2249786325 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f75d406ae40 0x7f75d406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:37.078 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:37.074+0000 7f75ea7fc640 1 --2- 172.21.15.169:0/2249786325 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f75d406ae40 0x7f75d406d300 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f75dc004a30 tx=0x7f75dc001670 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:37.078 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:37.076+0000 7f75ebfff640 1 -- 172.21.15.169:0/2249786325 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f75e4051ca0 con 0x7f75f40ac930 2023-10-26T01:36:37.308 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:37.306+0000 7f75f93d0640 1 -- 172.21.15.169:0/2249786325 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f75f40ad270 con 0x7f75f40ac930 2023-10-26T01:36:37.309 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:37.307+0000 7f75ebfff640 1 -- 172.21.15.169:0/2249786325 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+3711 (secure 0 0 0) 0x7f75e4055160 con 0x7f75f40ac930 2023-10-26T01:36:37.309 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:36:37.310 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":44,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5347,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/2629782429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2629782429},{"type":"v1","addr":"172.21.15.170:6836","nonce":2629782429}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":6474,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2561200417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2561200417},{"type":"v1","addr":"172.21.15.169:6833","nonce":2561200417}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":6480,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/2834878721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":2834878721},{"type":"v1","addr":"172.21.15.170:6837","nonce":2834878721}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":6801,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/3849844624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":3849844624},{"type":"v1","addr":"172.21.15.169:6835","nonce":3849844624}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44}],"filesystems":[]} 2023-10-26T01:36:37.310 INFO:teuthology.orchestra.run.smithi169.stderr:dumped fsmap epoch 44 2023-10-26T01:36:37.312 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:37.310+0000 7f75f93d0640 1 -- 172.21.15.169:0/2249786325 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f75d406ae40 msgr2=0x7f75d406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:37.312 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:37.310+0000 7f75f93d0640 1 --2- 172.21.15.169:0/2249786325 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f75d406ae40 0x7f75d406d300 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f75dc004a30 tx=0x7f75dc001670 comp rx=0 tx=0).stop 2023-10-26T01:36:37.312 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:37.310+0000 7f75f93d0640 1 -- 172.21.15.169:0/2249786325 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75f40ac930 msgr2=0x7f75f41f1b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:37.313 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:37.310+0000 7f75f93d0640 1 --2- 172.21.15.169:0/2249786325 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75f40ac930 0x7f75f41f1b80 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f75e400e790 tx=0x7f75e400ec50 comp rx=0 tx=0).stop 2023-10-26T01:36:37.313 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:37.310+0000 7f75f93d0640 1 -- 172.21.15.169:0/2249786325 shutdown_connections 2023-10-26T01:36:37.313 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:37.310+0000 7f75f93d0640 1 --2- 172.21.15.169:0/2249786325 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f75d406ae40 0x7f75d406d300 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:37.313 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:37.310+0000 7f75f93d0640 1 --2- 172.21.15.169:0/2249786325 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f75f41592d0 0x7f75f4144460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:37.313 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:37.310+0000 7f75f93d0640 1 --2- 172.21.15.169:0/2249786325 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f75f40cde00 0x7f75f4152080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:37.314 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:37.310+0000 7f75f93d0640 1 --2- 172.21.15.169:0/2249786325 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75f40ac930 0x7f75f41f1b80 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:37.314 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:37.310+0000 7f75f93d0640 1 -- 172.21.15.169:0/2249786325 >> 172.21.15.169:0/2249786325 conn(0x7f75f4069930 msgr2=0x7f75f40c95f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:37.314 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:37.311+0000 7f75f93d0640 1 -- 172.21.15.169:0/2249786325 shutdown_connections 2023-10-26T01:36:37.314 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:37.311+0000 7f75f93d0640 1 -- 172.21.15.169:0/2249786325 wait complete. 2023-10-26T01:36:37.333 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-26T01:36:37.333 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-26T01:36:37.333 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-26T01:36:37.333 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T01:36:37.334 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-26T01:36:37.334 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-26T01:36:37.334 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-26T01:36:37.335 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T01:36:37.335 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-26T01:36:37.335 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-26T01:36:37.335 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-26T01:36:37.336 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T01:36:37.336 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-26T01:36:37.336 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-26T01:36:37.336 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-26T01:36:37.337 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T01:36:37.350 INFO:tasks.ceph.mds.a:Stopped 2023-10-26T01:36:37.351 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-26T01:36:37.351 INFO:tasks.ceph.mds.c:Stopped 2023-10-26T01:36:37.352 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-26T01:36:37.355 INFO:tasks.ceph.mds.b:Stopped 2023-10-26T01:36:37.355 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-26T01:36:37.366 INFO:tasks.ceph.mds.d:Stopped 2023-10-26T01:36:37.366 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-26T01:36:37.399 INFO:tasks.ceph.mds.a:Started 2023-10-26T01:36:37.399 DEBUG:teuthology.parallel:result is None 2023-10-26T01:36:37.406 INFO:tasks.ceph.mds.c:Started 2023-10-26T01:36:37.406 DEBUG:teuthology.parallel:result is None 2023-10-26T01:36:37.414 INFO:tasks.ceph.mds.b:Started 2023-10-26T01:36:37.415 DEBUG:teuthology.parallel:result is None 2023-10-26T01:36:37.422 INFO:tasks.ceph.mds.d:Started 2023-10-26T01:36:37.422 DEBUG:teuthology.parallel:result is None 2023-10-26T01:36:37.423 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-26T01:36:37.559 INFO:tasks.ceph.mds.a.smithi169.stdout:starting mds.a at 2023-10-26T01:36:37.559 INFO:tasks.ceph.mds.c.smithi169.stdout:starting mds.c at 2023-10-26T01:36:37.575 INFO:tasks.ceph.mds.b.smithi170.stdout:starting mds.b at 2023-10-26T01:36:37.578 INFO:tasks.ceph.mds.d.smithi170.stdout:starting mds.d at 2023-10-26T01:36:37.614 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:37.612+0000 7fb3fbda5640 1 -- 172.21.15.169:0/3600276448 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3f41593e0 msgr2=0x7fb3f40ac200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:37.614 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:37.612+0000 7fb3fbda5640 1 --2- 172.21.15.169:0/3600276448 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3f41593e0 0x7fb3f40ac200 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb3f0009830 tx=0x7fb3f002bd60 comp rx=0 tx=0).stop 2023-10-26T01:36:37.614 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:37.612+0000 7fb3fbda5640 1 -- 172.21.15.169:0/3600276448 shutdown_connections 2023-10-26T01:36:37.615 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:37.612+0000 7fb3fbda5640 1 --2- 172.21.15.169:0/3600276448 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb3f41597c0 0x7fb3f415dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:37.615 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:37.612+0000 7fb3fbda5640 1 --2- 172.21.15.169:0/3600276448 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3f41593e0 0x7fb3f40ac200 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:37.615 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:37.612+0000 7fb3fbda5640 1 -- 172.21.15.169:0/3600276448 >> 172.21.15.169:0/3600276448 conn(0x7fb3f4069930 msgr2=0x7fb3f4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:37.615 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:37.612+0000 7fb3fbda5640 1 -- 172.21.15.169:0/3600276448 shutdown_connections 2023-10-26T01:36:37.615 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:37.612+0000 7fb3fbda5640 1 -- 172.21.15.169:0/3600276448 wait complete. 2023-10-26T01:36:37.616 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:37.614+0000 7fb3fbda5640 1 Processor -- start 2023-10-26T01:36:37.616 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:37.614+0000 7fb3fbda5640 1 -- start start 2023-10-26T01:36:37.617 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:37.615+0000 7fb3fbda5640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3f40b0c80 0x7fb3f41560b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:37.617 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:37.615+0000 7fb3fbda5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3f41593e0 0x7fb3f409a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:37.618 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:37.615+0000 7fb3fbda5640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb3f41597c0 0x7fb3f41558a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:37.618 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:37.615+0000 7fb3fbda5640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f4155de0 con 0x7fb3f40b0c80 2023-10-26T01:36:37.618 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:37.615+0000 7fb3fbda5640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f414e300 con 0x7fb3f41593e0 2023-10-26T01:36:37.618 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:37.615+0000 7fb3fbda5640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f4152d60 con 0x7fb3f41597c0 2023-10-26T01:36:37.618 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:37.615+0000 7fb3f9b41640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3f40b0c80 0x7fb3f41560b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:37.619 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:37.616+0000 7fb3f9b41640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3f40b0c80 0x7fb3f41560b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:58708/0 (socket says 172.21.15.169:58708) 2023-10-26T01:36:37.619 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:37.616+0000 7fb3f9b41640 1 -- 172.21.15.169:0/2610081314 learned_addr learned my addr 172.21.15.169:0/2610081314 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:36:37.619 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:37.616+0000 7fb3fa342640 1 --2- 172.21.15.169:0/2610081314 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb3f41597c0 0x7fb3f41558a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:37.619 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:37.616+0000 7fb3f9340640 1 --2- 172.21.15.169:0/2610081314 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3f41593e0 0x7fb3f409a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:37.620 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:37.616+0000 7fb3f9b41640 1 -- 172.21.15.169:0/2610081314 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb3f41597c0 msgr2=0x7fb3f41558a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:37.620 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:37.616+0000 7fb3f9b41640 1 --2- 172.21.15.169:0/2610081314 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb3f41597c0 0x7fb3f41558a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:37.621 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:37.616+0000 7fb3f9b41640 1 -- 172.21.15.169:0/2610081314 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3f41593e0 msgr2=0x7fb3f409a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:37.621 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:37.616+0000 7fb3f9b41640 1 --2- 172.21.15.169:0/2610081314 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3f41593e0 0x7fb3f409a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:37.621 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:37.616+0000 7fb3f9b41640 1 -- 172.21.15.169:0/2610081314 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3e4009490 con 0x7fb3f40b0c80 2023-10-26T01:36:37.621 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:37.616+0000 7fb3f9b41640 1 --2- 172.21.15.169:0/2610081314 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3f40b0c80 0x7fb3f41560b0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fb3f0009780 tx=0x7fb3f00038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:37.621 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:37.617+0000 7fb3ea7fc640 1 -- 172.21.15.169:0/2610081314 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f0003ea0 con 0x7fb3f40b0c80 2023-10-26T01:36:37.622 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:37.617+0000 7fb3ea7fc640 1 -- 172.21.15.169:0/2610081314 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb3f0034440 con 0x7fb3f40b0c80 2023-10-26T01:36:37.622 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:37.617+0000 7fb3fbda5640 1 -- 172.21.15.169:0/2610081314 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3f0009490 con 0x7fb3f40b0c80 2023-10-26T01:36:37.622 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:37.617+0000 7fb3ea7fc640 1 -- 172.21.15.169:0/2610081314 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f0034850 con 0x7fb3f40b0c80 2023-10-26T01:36:37.622 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:37.617+0000 7fb3fbda5640 1 -- 172.21.15.169:0/2610081314 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3f41527a0 con 0x7fb3f40b0c80 2023-10-26T01:36:37.622 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:37.618+0000 7fb3fbda5640 1 -- 172.21.15.169:0/2610081314 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3f404db10 con 0x7fb3f40b0c80 2023-10-26T01:36:37.623 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:37.619+0000 7fb3ea7fc640 1 -- 172.21.15.169:0/2610081314 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fb3f003b040 con 0x7fb3f40b0c80 2023-10-26T01:36:37.623 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:37.620+0000 7fb3ea7fc640 1 --2- 172.21.15.169:0/2610081314 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb3d806ae40 0x7fb3d806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:37.624 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:37.620+0000 7fb3ea7fc640 1 -- 172.21.15.169:0/2610081314 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fb3f00ab6c0 con 0x7fb3f40b0c80 2023-10-26T01:36:37.624 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:37.620+0000 7fb3f9340640 1 --2- 172.21.15.169:0/2610081314 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb3d806ae40 0x7fb3d806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:37.624 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:37.621+0000 7fb3f9340640 1 --2- 172.21.15.169:0/2610081314 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb3d806ae40 0x7fb3d806d300 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fb3f41e4880 tx=0x7fb3e4005f90 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:37.625 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:37.624+0000 7fb3ea7fc640 1 -- 172.21.15.169:0/2610081314 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb3f0076a80 con 0x7fb3f40b0c80 2023-10-26T01:36:37.824 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:37.822+0000 7fb3fbda5640 1 -- 172.21.15.169:0/2610081314 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fb3f40973d0 con 0x7fb3f40b0c80 2023-10-26T01:36:39.044 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:39.042+0000 7fb3ea7fc640 1 -- 172.21.15.169:0/2610081314 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v99) v1 ==== 97+0+0 (secure 0 0 0) 0x7fb3f0039080 con 0x7fb3f40b0c80 2023-10-26T01:36:39.044 INFO:teuthology.orchestra.run.smithi169.stderr: removed all blocklist entries 2023-10-26T01:36:39.047 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:39.045+0000 7fb3fbda5640 1 -- 172.21.15.169:0/2610081314 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb3d806ae40 msgr2=0x7fb3d806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:39.048 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:39.045+0000 7fb3fbda5640 1 --2- 172.21.15.169:0/2610081314 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb3d806ae40 0x7fb3d806d300 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fb3f41e4880 tx=0x7fb3e4005f90 comp rx=0 tx=0).stop 2023-10-26T01:36:39.048 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:39.045+0000 7fb3fbda5640 1 -- 172.21.15.169:0/2610081314 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3f40b0c80 msgr2=0x7fb3f41560b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:39.048 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:39.045+0000 7fb3fbda5640 1 --2- 172.21.15.169:0/2610081314 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3f40b0c80 0x7fb3f41560b0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fb3f0009780 tx=0x7fb3f00038a0 comp rx=0 tx=0).stop 2023-10-26T01:36:39.048 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:39.045+0000 7fb3fbda5640 1 -- 172.21.15.169:0/2610081314 shutdown_connections 2023-10-26T01:36:39.048 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:39.045+0000 7fb3fbda5640 1 --2- 172.21.15.169:0/2610081314 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb3d806ae40 0x7fb3d806d300 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:39.049 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:39.045+0000 7fb3fbda5640 1 --2- 172.21.15.169:0/2610081314 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb3f41597c0 0x7fb3f41558a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:39.049 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:39.045+0000 7fb3fbda5640 1 --2- 172.21.15.169:0/2610081314 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3f41593e0 0x7fb3f409a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:39.049 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:39.045+0000 7fb3fbda5640 1 --2- 172.21.15.169:0/2610081314 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3f40b0c80 0x7fb3f41560b0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:39.049 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:39.046+0000 7fb3fbda5640 1 -- 172.21.15.169:0/2610081314 >> 172.21.15.169:0/2610081314 conn(0x7fb3f4069930 msgr2=0x7fb3f40cbaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:39.049 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:39.046+0000 7fb3fbda5640 1 -- 172.21.15.169:0/2610081314 shutdown_connections 2023-10-26T01:36:39.050 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:39.046+0000 7fb3fbda5640 1 -- 172.21.15.169:0/2610081314 wait complete. 2023-10-26T01:36:39.067 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-26T01:36:39.284 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:39.282+0000 7f9f212f7640 1 -- 172.21.15.169:0/1474943375 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f1c06ed30 msgr2=0x7f9f1c0cd670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:39.285 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:39.282+0000 7f9f212f7640 1 --2- 172.21.15.169:0/1474943375 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f1c06ed30 0x7f9f1c0cd670 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f9f10009830 tx=0x7f9f1002bd70 comp rx=0 tx=0).stop 2023-10-26T01:36:39.285 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:39.282+0000 7f9f212f7640 1 -- 172.21.15.169:0/1474943375 shutdown_connections 2023-10-26T01:36:39.285 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:39.282+0000 7f9f212f7640 1 --2- 172.21.15.169:0/1474943375 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f1c06ed30 0x7f9f1c0cd670 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:39.285 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:39.282+0000 7f9f212f7640 1 -- 172.21.15.169:0/1474943375 >> 172.21.15.169:0/1474943375 conn(0x7f9f1c069930 msgr2=0x7f9f1c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:39.286 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:39.283+0000 7f9f212f7640 1 -- 172.21.15.169:0/1474943375 shutdown_connections 2023-10-26T01:36:39.286 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:39.283+0000 7f9f212f7640 1 -- 172.21.15.169:0/1474943375 wait complete. 2023-10-26T01:36:39.286 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:39.284+0000 7f9f212f7640 1 Processor -- start 2023-10-26T01:36:39.287 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:39.284+0000 7f9f212f7640 1 -- start start 2023-10-26T01:36:39.288 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:39.285+0000 7f9f212f7640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9f1c06ed30 0x7f9f1c141410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:39.288 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:39.285+0000 7f9f212f7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9f1c0cdbb0 0x7f9f1c070910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:39.288 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:39.285+0000 7f9f212f7640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f1c15b610 0x7f9f1c154120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:39.289 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:39.285+0000 7f9f212f7640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f1c0704f0 con 0x7f9f1c15b610 2023-10-26T01:36:39.289 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:39.285+0000 7f9f212f7640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f1c06f620 con 0x7f9f1c0cdbb0 2023-10-26T01:36:39.289 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:39.285+0000 7f9f212f7640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f1c1f23b0 con 0x7f9f1c06ed30 2023-10-26T01:36:39.289 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:39.285+0000 7f9f1b7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f1c15b610 0x7f9f1c154120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:39.289 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:39.286+0000 7f9f1b7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f1c15b610 0x7f9f1c154120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:58736/0 (socket says 172.21.15.169:58736) 2023-10-26T01:36:39.290 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:39.286+0000 7f9f1b7fe640 1 -- 172.21.15.169:0/1711689353 learned_addr learned my addr 172.21.15.169:0/1711689353 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:36:39.290 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:39.286+0000 7f9f1affd640 1 --2- 172.21.15.169:0/1711689353 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9f1c06ed30 0x7f9f1c141410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:39.290 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:39.286+0000 7f9f1b7fe640 1 -- 172.21.15.169:0/1711689353 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9f1c06ed30 msgr2=0x7f9f1c141410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:39.290 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:39.286+0000 7f9f1b7fe640 1 --2- 172.21.15.169:0/1711689353 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9f1c06ed30 0x7f9f1c141410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:39.290 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:39.286+0000 7f9f1b7fe640 1 -- 172.21.15.169:0/1711689353 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9f1c0cdbb0 msgr2=0x7f9f1c070910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:39.290 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:39.286+0000 7f9f1b7fe640 1 --2- 172.21.15.169:0/1711689353 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9f1c0cdbb0 0x7f9f1c070910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:39.291 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:39.286+0000 7f9f1b7fe640 1 -- 172.21.15.169:0/1711689353 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f10009490 con 0x7f9f1c15b610 2023-10-26T01:36:39.291 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:39.286+0000 7f9f1affd640 1 --2- 172.21.15.169:0/1711689353 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9f1c06ed30 0x7f9f1c141410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:36:39.291 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:39.286+0000 7f9f1b7fe640 1 --2- 172.21.15.169:0/1711689353 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f1c15b610 0x7f9f1c154120 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f9f1400cb70 tx=0x7f9f1400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:39.291 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:39.287+0000 7f9f037fe640 1 -- 172.21.15.169:0/1711689353 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f1400ad40 con 0x7f9f1c15b610 2023-10-26T01:36:39.291 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:39.287+0000 7f9f037fe640 1 -- 172.21.15.169:0/1711689353 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9f14012420 con 0x7f9f1c15b610 2023-10-26T01:36:39.291 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:39.287+0000 7f9f212f7640 1 -- 172.21.15.169:0/1711689353 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f1c06f870 con 0x7f9f1c15b610 2023-10-26T01:36:39.292 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:39.287+0000 7f9f037fe640 1 -- 172.21.15.169:0/1711689353 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f14012810 con 0x7f9f1c15b610 2023-10-26T01:36:39.292 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:39.287+0000 7f9f212f7640 1 -- 172.21.15.169:0/1711689353 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f1c06fe30 con 0x7f9f1c15b610 2023-10-26T01:36:39.293 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:39.288+0000 7f9f212f7640 1 -- 172.21.15.169:0/1711689353 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f1c070630 con 0x7f9f1c15b610 2023-10-26T01:36:39.293 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:39.289+0000 7f9f037fe640 1 -- 172.21.15.169:0/1711689353 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f9f1401a3b0 con 0x7f9f1c15b610 2023-10-26T01:36:39.293 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:39.290+0000 7f9f037fe640 1 --2- 172.21.15.169:0/1711689353 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9f0806ae40 0x7f9f0806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:39.293 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:39.290+0000 7f9f037fe640 1 -- 172.21.15.169:0/1711689353 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f9f140899f0 con 0x7f9f1c15b610 2023-10-26T01:36:39.293 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:39.291+0000 7f9f1affd640 1 --2- 172.21.15.169:0/1711689353 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9f0806ae40 0x7f9f0806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:39.294 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:39.292+0000 7f9f1affd640 1 --2- 172.21.15.169:0/1711689353 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9f0806ae40 0x7f9f0806d300 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c00a830 tx=0x7f9f0c008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:39.296 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:39.295+0000 7f9f037fe640 1 -- 172.21.15.169:0/1711689353 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9f1c070630 con 0x7f9f1c15b610 2023-10-26T01:36:39.560 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:39.558+0000 7f9f212f7640 1 -- 172.21.15.169:0/1711689353 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f9f1c070630 con 0x7f9f1c15b610 2023-10-26T01:36:39.561 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:39.559+0000 7f9f037fe640 1 -- 172.21.15.169:0/1711689353 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f9f1c070630 con 0x7f9f1c15b610 2023-10-26T01:36:39.561 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:36:39.562 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2023-10-26T01:36:39.562 INFO:teuthology.orchestra.run.smithi169.stdout: "auth_dump": [ 2023-10-26T01:36:39.562 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:36:39.562 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "mds.a", 2023-10-26T01:36:39.562 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAgwTllWtUfAhAA28+12VwWEF/BUEu+ErSVyA==", 2023-10-26T01:36:39.563 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:36:39.563 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow", 2023-10-26T01:36:39.563 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow *", 2023-10-26T01:36:39.563 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow *", 2023-10-26T01:36:39.563 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:36:39.563 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:36:39.564 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:36:39.564 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:36:39.564 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "mds.b", 2023-10-26T01:36:39.564 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAgwTlleA8lHxAA+jMcMo+QnYAGllpKJ6eYgQ==", 2023-10-26T01:36:39.564 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:36:39.564 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow", 2023-10-26T01:36:39.565 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow *", 2023-10-26T01:36:39.565 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow *", 2023-10-26T01:36:39.565 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:36:39.565 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:36:39.565 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:36:39.565 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:36:39.566 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "mds.c", 2023-10-26T01:36:39.566 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAgwTllaCCqEBAAokeHvwOyp/Jv2x0yHDwysw==", 2023-10-26T01:36:39.566 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:36:39.566 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow", 2023-10-26T01:36:39.566 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow *", 2023-10-26T01:36:39.567 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow *", 2023-10-26T01:36:39.567 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:36:39.567 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:36:39.567 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:36:39.567 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:36:39.567 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "mds.d", 2023-10-26T01:36:39.568 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAgwTll51JQLhAA4Wzso4zQj+Htv6cXqvwRmg==", 2023-10-26T01:36:39.568 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:36:39.568 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow", 2023-10-26T01:36:39.568 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow *", 2023-10-26T01:36:39.568 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow *", 2023-10-26T01:36:39.569 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:36:39.569 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:36:39.569 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:36:39.569 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:36:39.569 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "osd.0", 2023-10-26T01:36:39.570 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAlwTll8RZ+HRAAt/sBK+Mi91YmTyQYbhKHLg==", 2023-10-26T01:36:39.570 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:36:39.570 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow profile osd", 2023-10-26T01:36:39.570 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow profile osd", 2023-10-26T01:36:39.570 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:36:39.571 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:36:39.571 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:36:39.571 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:36:39.571 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "osd.1", 2023-10-26T01:36:39.571 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAlwTllLXwWLRAAuHB2lJ+oDJou/IJBtsYY4Q==", 2023-10-26T01:36:39.571 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:36:39.572 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow profile osd", 2023-10-26T01:36:39.572 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow profile osd", 2023-10-26T01:36:39.572 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:36:39.572 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:36:39.572 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:36:39.572 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:36:39.573 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "osd.2", 2023-10-26T01:36:39.573 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAmwTllj1nSARAA5Jl34Q0X7a71ySd4wz17lw==", 2023-10-26T01:36:39.573 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:36:39.573 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow profile osd", 2023-10-26T01:36:39.573 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow profile osd", 2023-10-26T01:36:39.573 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:36:39.574 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:36:39.574 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:36:39.574 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:36:39.574 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "osd.3", 2023-10-26T01:36:39.574 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAmwTllC8MeEhAAfGxRVSwD+YbU/U5aeenO3w==", 2023-10-26T01:36:39.575 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:36:39.575 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow profile osd", 2023-10-26T01:36:39.575 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow profile osd", 2023-10-26T01:36:39.575 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:36:39.575 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:36:39.576 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:36:39.576 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:36:39.576 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "osd.4", 2023-10-26T01:36:39.576 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAowTll4Md6DhAAuXfadUNAdouMP28MiX/D3Q==", 2023-10-26T01:36:39.576 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:36:39.577 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow profile osd", 2023-10-26T01:36:39.577 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow profile osd", 2023-10-26T01:36:39.577 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:36:39.577 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:36:39.577 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:36:39.577 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:36:39.577 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "osd.5", 2023-10-26T01:36:39.578 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAowTllDdViHRAAec9QyhZCsjJ2rgGhhcjwpQ==", 2023-10-26T01:36:39.578 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:36:39.578 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow profile osd", 2023-10-26T01:36:39.578 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow profile osd", 2023-10-26T01:36:39.578 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:36:39.579 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:36:39.579 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:36:39.579 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:36:39.579 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "osd.6", 2023-10-26T01:36:39.579 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAowTll23X5KxAAsmQvG0yq7ilkVG5fNiaCUg==", 2023-10-26T01:36:39.580 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:36:39.580 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow profile osd", 2023-10-26T01:36:39.580 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow profile osd", 2023-10-26T01:36:39.580 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:36:39.580 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:36:39.580 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:36:39.581 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:36:39.581 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "osd.7", 2023-10-26T01:36:39.581 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAowTllhcy/OhAAY5Yjheh76ibOVREqN6WBpQ==", 2023-10-26T01:36:39.581 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:36:39.581 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow profile osd", 2023-10-26T01:36:39.582 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow profile osd", 2023-10-26T01:36:39.582 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:36:39.582 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:36:39.582 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:36:39.582 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:36:39.583 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "client.0", 2023-10-26T01:36:39.583 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAgwTllKC7gOhAAPsvxGcuUA8/p8xqbUgGteA==", 2023-10-26T01:36:39.583 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:36:39.583 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow", 2023-10-26T01:36:39.583 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow r", 2023-10-26T01:36:39.583 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-26T01:36:39.584 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:36:39.584 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:36:39.584 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:36:39.584 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "client.1", 2023-10-26T01:36:39.584 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAhwTllhxb6ExAAeYTsurWpnjohWlrNIlLpsw==", 2023-10-26T01:36:39.585 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:36:39.585 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow", 2023-10-26T01:36:39.585 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow r", 2023-10-26T01:36:39.585 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow rw", 2023-10-26T01:36:39.585 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow rwx" 2023-10-26T01:36:39.586 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:36:39.586 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:36:39.586 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:36:39.586 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "client.2", 2023-10-26T01:36:39.586 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAhwTllLWMoChAAsUHzBc29GDqmEQAFnl7dHg==", 2023-10-26T01:36:39.587 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:36:39.587 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow", 2023-10-26T01:36:39.587 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow r", 2023-10-26T01:36:39.587 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow rw", 2023-10-26T01:36:39.587 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow rwx" 2023-10-26T01:36:39.588 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:36:39.588 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:36:39.588 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:36:39.588 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "client.3", 2023-10-26T01:36:39.588 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAhwTll875HHRAAGbwikoR/hDU8P/bj0OPWAw==", 2023-10-26T01:36:39.589 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:36:39.589 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow", 2023-10-26T01:36:39.589 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow r", 2023-10-26T01:36:39.589 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow rw", 2023-10-26T01:36:39.589 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow rwx" 2023-10-26T01:36:39.589 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:36:39.590 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:36:39.590 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:36:39.590 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "client.admin", 2023-10-26T01:36:39.590 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAewTll2yddMhAAO8bnXNevDGe/KEeNNa36pQ==", 2023-10-26T01:36:39.590 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:36:39.591 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow *", 2023-10-26T01:36:39.591 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow *", 2023-10-26T01:36:39.591 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow *", 2023-10-26T01:36:39.591 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:36:39.591 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:36:39.591 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:36:39.592 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:36:39.592 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "mgr.x", 2023-10-26T01:36:39.592 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAfwTllPu3FMxAAV8d5oRFXAeeZH7e1xzPEDg==", 2023-10-26T01:36:39.592 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:36:39.592 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow *", 2023-10-26T01:36:39.593 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow profile mgr", 2023-10-26T01:36:39.593 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:36:39.593 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:36:39.593 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:36:39.593 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:36:39.593 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "mgr.y", 2023-10-26T01:36:39.594 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAfwTllZc+DJxAAsZ2TWn8H7Kw/lNTc2qyKzA==", 2023-10-26T01:36:39.594 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:36:39.594 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow *", 2023-10-26T01:36:39.594 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow profile mgr", 2023-10-26T01:36:39.594 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:36:39.595 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:36:39.595 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:36:39.595 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:36:39.595 INFO:teuthology.orchestra.run.smithi169.stdout:} 2023-10-26T01:36:39.597 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:39.562+0000 7f9f212f7640 1 -- 172.21.15.169:0/1711689353 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9f0806ae40 msgr2=0x7f9f0806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:39.597 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:39.562+0000 7f9f212f7640 1 --2- 172.21.15.169:0/1711689353 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9f0806ae40 0x7f9f0806d300 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c00a830 tx=0x7f9f0c008040 comp rx=0 tx=0).stop 2023-10-26T01:36:39.597 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:39.562+0000 7f9f212f7640 1 -- 172.21.15.169:0/1711689353 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f1c15b610 msgr2=0x7f9f1c154120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:39.597 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:39.562+0000 7f9f212f7640 1 --2- 172.21.15.169:0/1711689353 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f1c15b610 0x7f9f1c154120 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f9f1400cb70 tx=0x7f9f1400a430 comp rx=0 tx=0).stop 2023-10-26T01:36:39.598 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:39.562+0000 7f9f212f7640 1 -- 172.21.15.169:0/1711689353 shutdown_connections 2023-10-26T01:36:39.598 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:39.562+0000 7f9f212f7640 1 --2- 172.21.15.169:0/1711689353 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9f0806ae40 0x7f9f0806d300 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:39.598 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:39.562+0000 7f9f212f7640 1 --2- 172.21.15.169:0/1711689353 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f1c15b610 0x7f9f1c154120 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:39.598 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:39.562+0000 7f9f212f7640 1 --2- 172.21.15.169:0/1711689353 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9f1c0cdbb0 0x7f9f1c070910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:39.598 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:39.562+0000 7f9f212f7640 1 --2- 172.21.15.169:0/1711689353 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9f1c06ed30 0x7f9f1c141410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:39.599 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:39.562+0000 7f9f212f7640 1 -- 172.21.15.169:0/1711689353 >> 172.21.15.169:0/1711689353 conn(0x7f9f1c069930 msgr2=0x7f9f1c0ae630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:39.599 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:39.562+0000 7f9f212f7640 1 -- 172.21.15.169:0/1711689353 shutdown_connections 2023-10-26T01:36:39.599 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:39.562+0000 7f9f212f7640 1 -- 172.21.15.169:0/1711689353 wait complete. 2023-10-26T01:36:39.599 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:36:39.824 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:39.822+0000 7f744096c640 1 -- 172.21.15.169:0/1189177761 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f74280058d0 msgr2=0x7f7428005d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:39.824 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:39.822+0000 7f744096c640 1 --2- 172.21.15.169:0/1189177761 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f74280058d0 0x7f7428005d50 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f7424006fd0 tx=0x7f7428006900 comp rx=0 tx=0).stop 2023-10-26T01:36:39.824 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:39.822+0000 7f743bfff640 1 -- 172.21.15.169:0/1189177761 reap_dead start 2023-10-26T01:36:39.825 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:39.822+0000 7f744096c640 1 -- 172.21.15.169:0/1189177761 shutdown_connections 2023-10-26T01:36:39.825 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:39.822+0000 7f744096c640 1 -- 172.21.15.169:0/1189177761 >> 172.21.15.169:0/1189177761 conn(0x7f743c0aedd0 msgr2=0x7f743c0af1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:39.825 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:39.822+0000 7f744096c640 1 -- 172.21.15.169:0/1189177761 shutdown_connections 2023-10-26T01:36:39.825 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:39.822+0000 7f744096c640 1 -- 172.21.15.169:0/1189177761 wait complete. 2023-10-26T01:36:39.826 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:39.824+0000 7f744096c640 1 Processor -- start 2023-10-26T01:36:39.826 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:39.824+0000 7f744096c640 1 -- start start 2023-10-26T01:36:39.828 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:39.825+0000 7f744096c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f74280058d0 0x7f743c0c64e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:39.828 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:39.825+0000 7f744096c640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f743c14b980 0x7f743c14bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:39.828 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:39.825+0000 7f744096c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f743c0aaf50 0x7f743c0ab3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:39.829 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:39.825+0000 7f744096c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f743c14c320 con 0x7f743c0aaf50 2023-10-26T01:36:39.829 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:39.825+0000 7f744096c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f743c0abc80 con 0x7f74280058d0 2023-10-26T01:36:39.829 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:39.825+0000 7f744096c640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f743c0b0ab0 con 0x7f743c14b980 2023-10-26T01:36:39.829 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:39.825+0000 7f743bfff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f743c0aaf50 0x7f743c0ab3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:39.829 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:39.825+0000 7f743bfff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f743c0aaf50 0x7f743c0ab3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:58756/0 (socket says 172.21.15.169:58756) 2023-10-26T01:36:39.830 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:39.825+0000 7f743bfff640 1 -- 172.21.15.169:0/4157345121 learned_addr learned my addr 172.21.15.169:0/4157345121 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:36:39.830 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:39.825+0000 7f743b7fe640 1 --2- 172.21.15.169:0/4157345121 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f74280058d0 0x7f743c0c64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:39.830 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:39.826+0000 7f7432ffd640 1 --2- 172.21.15.169:0/4157345121 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f743c14b980 0x7f743c14bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:39.830 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:39.826+0000 7f743bfff640 1 -- 172.21.15.169:0/4157345121 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f743c14b980 msgr2=0x7f743c14bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:39.830 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:39.826+0000 7f743bfff640 1 --2- 172.21.15.169:0/4157345121 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f743c14b980 0x7f743c14bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:39.831 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:39.826+0000 7f743bfff640 1 -- 172.21.15.169:0/4157345121 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f74280058d0 msgr2=0x7f743c0c64e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:39.831 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:39.826+0000 7f743bfff640 1 --2- 172.21.15.169:0/4157345121 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f74280058d0 0x7f743c0c64e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:39.831 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:39.826+0000 7f743bfff640 1 -- 172.21.15.169:0/4157345121 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f743c0b0d90 con 0x7f743c0aaf50 2023-10-26T01:36:39.831 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:39.826+0000 7f743bfff640 1 --2- 172.21.15.169:0/4157345121 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f743c0aaf50 0x7f743c0ab3d0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f743c071eb0 tx=0x7f7434001a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:39.831 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:39.826+0000 7f7438ff9640 1 -- 172.21.15.169:0/4157345121 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7434011870 con 0x7f743c0aaf50 2023-10-26T01:36:39.832 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:39.826+0000 7f7438ff9640 1 -- 172.21.15.169:0/4157345121 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7434011ac0 con 0x7f743c0aaf50 2023-10-26T01:36:39.832 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:39.826+0000 7f7438ff9640 1 -- 172.21.15.169:0/4157345121 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74340103c0 con 0x7f743c0aaf50 2023-10-26T01:36:39.832 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:39.826+0000 7f744096c640 1 -- 172.21.15.169:0/4157345121 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f743c0b1070 con 0x7f743c0aaf50 2023-10-26T01:36:39.832 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:39.826+0000 7f744096c640 1 -- 172.21.15.169:0/4157345121 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f743c06e8f0 con 0x7f743c0aaf50 2023-10-26T01:36:39.833 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:39.828+0000 7f744096c640 1 -- 172.21.15.169:0/4157345121 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f743c0b1260 con 0x7f743c0aaf50 2023-10-26T01:36:39.833 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:39.829+0000 7f7438ff9640 1 -- 172.21.15.169:0/4157345121 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f7434010520 con 0x7f743c0aaf50 2023-10-26T01:36:39.833 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:39.829+0000 7f7438ff9640 1 --2- 172.21.15.169:0/4157345121 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f741c06ae40 0x7f741c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:39.833 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:39.830+0000 7f7438ff9640 1 -- 172.21.15.169:0/4157345121 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f7434089e00 con 0x7f743c0aaf50 2023-10-26T01:36:39.834 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:39.830+0000 7f743b7fe640 1 --2- 172.21.15.169:0/4157345121 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f741c06ae40 0x7f741c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:39.834 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:39.831+0000 7f743b7fe640 1 --2- 172.21.15.169:0/4157345121 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f741c06ae40 0x7f741c06d300 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f742c005fd0 tx=0x7f742c00c4e0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:39.836 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:39.835+0000 7f7438ff9640 1 -- 172.21.15.169:0/4157345121 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7434055200 con 0x7f743c0aaf50 2023-10-26T01:36:40.027 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.025+0000 7f744096c640 1 -- 172.21.15.169:0/4157345121 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f743c157cf0 con 0x7f743c0aaf50 2023-10-26T01:36:40.028 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.026+0000 7f7438ff9640 1 -- 172.21.15.169:0/4157345121 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) v1 ==== 74+0+12906 (secure 0 0 0) 0x7f7434058820 con 0x7f743c0aaf50 2023-10-26T01:36:40.028 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:36:40.029 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":99,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:36:39.033165+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:36:40.032 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.030+0000 7f744096c640 1 -- 172.21.15.169:0/4157345121 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f741c06ae40 msgr2=0x7f741c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:40.032 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.030+0000 7f744096c640 1 --2- 172.21.15.169:0/4157345121 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f741c06ae40 0x7f741c06d300 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f742c005fd0 tx=0x7f742c00c4e0 comp rx=0 tx=0).stop 2023-10-26T01:36:40.033 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.030+0000 7f744096c640 1 -- 172.21.15.169:0/4157345121 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f743c0aaf50 msgr2=0x7f743c0ab3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:40.033 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.030+0000 7f744096c640 1 --2- 172.21.15.169:0/4157345121 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f743c0aaf50 0x7f743c0ab3d0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f743c071eb0 tx=0x7f7434001a70 comp rx=0 tx=0).stop 2023-10-26T01:36:40.033 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.030+0000 7f744096c640 1 -- 172.21.15.169:0/4157345121 shutdown_connections 2023-10-26T01:36:40.033 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.030+0000 7f744096c640 1 --2- 172.21.15.169:0/4157345121 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f741c06ae40 0x7f741c06d300 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:40.034 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.030+0000 7f744096c640 1 --2- 172.21.15.169:0/4157345121 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f743c0aaf50 0x7f743c0ab3d0 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:40.034 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.030+0000 7f744096c640 1 --2- 172.21.15.169:0/4157345121 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f743c14b980 0x7f743c14bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:40.034 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.030+0000 7f744096c640 1 --2- 172.21.15.169:0/4157345121 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f74280058d0 0x7f743c0c64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:40.034 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.030+0000 7f744096c640 1 -- 172.21.15.169:0/4157345121 >> 172.21.15.169:0/4157345121 conn(0x7f743c0aedd0 msgr2=0x7f743c1595e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:40.034 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.031+0000 7f744096c640 1 -- 172.21.15.169:0/4157345121 shutdown_connections 2023-10-26T01:36:40.035 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.031+0000 7f744096c640 1 -- 172.21.15.169:0/4157345121 wait complete. 2023-10-26T01:36:40.052 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T01:30:32.200808+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T01:36:40.053 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T01:36:40.278 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.276+0000 7f99bb1a5640 1 -- 172.21.15.169:0/2765325110 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99a0003970 msgr2=0x7f99a0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:40.278 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.276+0000 7f99bb1a5640 1 --2- 172.21.15.169:0/2765325110 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99a0003970 0x7f99a0003d90 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f99b0006f40 tx=0x7f99a00057f0 comp rx=0 tx=0).stop 2023-10-26T01:36:40.278 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.276+0000 7f99ba9a4640 1 -- 172.21.15.169:0/2765325110 reap_dead start 2023-10-26T01:36:40.278 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.276+0000 7f99bb1a5640 1 -- 172.21.15.169:0/2765325110 shutdown_connections 2023-10-26T01:36:40.279 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.276+0000 7f99bb1a5640 1 -- 172.21.15.169:0/2765325110 >> 172.21.15.169:0/2765325110 conn(0x7f99b4069510 msgr2=0x7f99b4069920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:40.279 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.276+0000 7f99bb1a5640 1 -- 172.21.15.169:0/2765325110 shutdown_connections 2023-10-26T01:36:40.279 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.276+0000 7f99bb1a5640 1 -- 172.21.15.169:0/2765325110 wait complete. 2023-10-26T01:36:40.279 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.278+0000 7f99bb1a5640 1 Processor -- start 2023-10-26T01:36:40.280 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.278+0000 7f99bb1a5640 1 -- start start 2023-10-26T01:36:40.281 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.279+0000 7f99bb1a5640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f99a0003970 0x7f99b414f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:40.281 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.279+0000 7f99bb1a5640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99b40af590 0x7f99b40ae720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:40.281 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.279+0000 7f99bb1a5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f99b415cb40 0x7f99b40ab090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:40.281 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.279+0000 7f99bb1a5640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99b414f930 con 0x7f99b40af590 2023-10-26T01:36:40.281 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.279+0000 7f99bb1a5640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99b414fdc0 con 0x7f99b415cb40 2023-10-26T01:36:40.282 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.279+0000 7f99bb1a5640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99b40aec60 con 0x7f99a0003970 2023-10-26T01:36:40.282 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.279+0000 7f99b99a2640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99b40af590 0x7f99b40ae720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:40.282 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.279+0000 7f99b99a2640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99b40af590 0x7f99b40ae720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:58770/0 (socket says 172.21.15.169:58770) 2023-10-26T01:36:40.282 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.279+0000 7f99b99a2640 1 -- 172.21.15.169:0/867384019 learned_addr learned my addr 172.21.15.169:0/867384019 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:36:40.283 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.280+0000 7f99ba9a4640 1 --2- 172.21.15.169:0/867384019 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f99b415cb40 0x7f99b40ab090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:40.284 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.280+0000 7f99ba1a3640 1 --2- 172.21.15.169:0/867384019 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f99a0003970 0x7f99b414f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:40.284 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.280+0000 7f99b99a2640 1 -- 172.21.15.169:0/867384019 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f99a0003970 msgr2=0x7f99b414f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:40.284 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.280+0000 7f99b99a2640 1 --2- 172.21.15.169:0/867384019 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f99a0003970 0x7f99b414f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:40.284 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.280+0000 7f99b99a2640 1 -- 172.21.15.169:0/867384019 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f99b415cb40 msgr2=0x7f99b40ab090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:40.284 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.280+0000 7f99b99a2640 1 --2- 172.21.15.169:0/867384019 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f99b415cb40 0x7f99b40ab090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:40.285 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.280+0000 7f99b99a2640 1 -- 172.21.15.169:0/867384019 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99b415bdc0 con 0x7f99b40af590 2023-10-26T01:36:40.285 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.280+0000 7f99ba9a4640 1 --2- 172.21.15.169:0/867384019 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f99b415cb40 0x7f99b40ab090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:36:40.285 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.280+0000 7f99b99a2640 1 --2- 172.21.15.169:0/867384019 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99b40af590 0x7f99b40ae720 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f99a400f9d0 tx=0x7f99a400fdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:40.285 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.280+0000 7f99aaffd640 1 -- 172.21.15.169:0/867384019 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a400bd70 con 0x7f99b40af590 2023-10-26T01:36:40.285 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.280+0000 7f99aaffd640 1 -- 172.21.15.169:0/867384019 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f99a4011440 con 0x7f99b40af590 2023-10-26T01:36:40.286 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.280+0000 7f99bb1a5640 1 -- 172.21.15.169:0/867384019 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99b415c060 con 0x7f99b40af590 2023-10-26T01:36:40.286 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.281+0000 7f99aaffd640 1 -- 172.21.15.169:0/867384019 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a4011830 con 0x7f99b40af590 2023-10-26T01:36:40.286 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.281+0000 7f99bb1a5640 1 -- 172.21.15.169:0/867384019 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99b415c620 con 0x7f99b40af590 2023-10-26T01:36:40.286 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.282+0000 7f99bb1a5640 1 -- 172.21.15.169:0/867384019 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99b415c840 con 0x7f99b40af590 2023-10-26T01:36:40.287 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.283+0000 7f99aaffd640 1 -- 172.21.15.169:0/867384019 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f99a4011990 con 0x7f99b40af590 2023-10-26T01:36:40.287 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.284+0000 7f99aaffd640 1 --2- 172.21.15.169:0/867384019 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f999806ae40 0x7f999806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:40.287 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.285+0000 7f99aaffd640 1 -- 172.21.15.169:0/867384019 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f99a408a390 con 0x7f99b40af590 2023-10-26T01:36:40.287 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.285+0000 7f99ba1a3640 1 --2- 172.21.15.169:0/867384019 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f999806ae40 0x7f999806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:40.288 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.286+0000 7f99ba1a3640 1 --2- 172.21.15.169:0/867384019 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f999806ae40 0x7f999806d300 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f99b0034000 tx=0x7f99b0039040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:40.290 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.289+0000 7f99aaffd640 1 -- 172.21.15.169:0/867384019 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f99a40557a0 con 0x7f99b40af590 2023-10-26T01:36:40.482 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.480+0000 7f99bb1a5640 1 -- 172.21.15.169:0/867384019 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f99b4073c10 con 0x7f99b40af590 2023-10-26T01:36:40.483 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.481+0000 7f99aaffd640 1 -- 172.21.15.169:0/867384019 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v99) v1 ==== 93+0+10 (secure 0 0 0) 0x7f99a4058dc0 con 0x7f99b40af590 2023-10-26T01:36:40.483 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 1 2023-10-26T01:36:40.485 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.483+0000 7f99bb1a5640 1 -- 172.21.15.169:0/867384019 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f999806ae40 msgr2=0x7f999806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:40.485 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.483+0000 7f99bb1a5640 1 --2- 172.21.15.169:0/867384019 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f999806ae40 0x7f999806d300 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f99b0034000 tx=0x7f99b0039040 comp rx=0 tx=0).stop 2023-10-26T01:36:40.486 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.483+0000 7f99bb1a5640 1 -- 172.21.15.169:0/867384019 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99b40af590 msgr2=0x7f99b40ae720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:40.486 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.483+0000 7f99bb1a5640 1 --2- 172.21.15.169:0/867384019 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99b40af590 0x7f99b40ae720 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f99a400f9d0 tx=0x7f99a400fdd0 comp rx=0 tx=0).stop 2023-10-26T01:36:40.486 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.484+0000 7f99bb1a5640 1 -- 172.21.15.169:0/867384019 shutdown_connections 2023-10-26T01:36:40.486 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.484+0000 7f99bb1a5640 1 --2- 172.21.15.169:0/867384019 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f999806ae40 0x7f999806d300 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:40.486 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.484+0000 7f99bb1a5640 1 --2- 172.21.15.169:0/867384019 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99b40af590 0x7f99b40ae720 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:40.487 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.484+0000 7f99bb1a5640 1 --2- 172.21.15.169:0/867384019 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f99b415cb40 0x7f99b40ab090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:40.487 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.484+0000 7f99bb1a5640 1 --2- 172.21.15.169:0/867384019 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f99a0003970 0x7f99b414f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:40.487 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.484+0000 7f99bb1a5640 1 -- 172.21.15.169:0/867384019 >> 172.21.15.169:0/867384019 conn(0x7f99b4069510 msgr2=0x7f99b4150670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:40.487 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.484+0000 7f99bb1a5640 1 -- 172.21.15.169:0/867384019 shutdown_connections 2023-10-26T01:36:40.487 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.484+0000 7f99bb1a5640 1 -- 172.21.15.169:0/867384019 wait complete. 2023-10-26T01:36:40.503 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-26T01:36:40.727 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.725+0000 7fd17914c640 1 -- 172.21.15.169:0/2103134027 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd15c004d00 msgr2=0x7fd15c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:40.728 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.725+0000 7fd17914c640 1 --2- 172.21.15.169:0/2103134027 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd15c004d00 0x7fd15c005180 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fd16c00ad20 tx=0x7fd15c0057f0 comp rx=0 tx=0).stop 2023-10-26T01:36:40.728 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.725+0000 7fd17359e640 1 -- 172.21.15.169:0/2103134027 reap_dead start 2023-10-26T01:36:40.728 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.726+0000 7fd17914c640 1 -- 172.21.15.169:0/2103134027 shutdown_connections 2023-10-26T01:36:40.728 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.726+0000 7fd17914c640 1 -- 172.21.15.169:0/2103134027 >> 172.21.15.169:0/2103134027 conn(0x7fd174069930 msgr2=0x7fd174069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:40.729 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.726+0000 7fd17914c640 1 -- 172.21.15.169:0/2103134027 shutdown_connections 2023-10-26T01:36:40.729 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.726+0000 7fd17914c640 1 -- 172.21.15.169:0/2103134027 wait complete. 2023-10-26T01:36:40.729 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.727+0000 7fd17914c640 1 Processor -- start 2023-10-26T01:36:40.729 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.728+0000 7fd17914c640 1 -- start start 2023-10-26T01:36:40.731 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.729+0000 7fd17914c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd15c004d00 0x7fd1740b0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:40.731 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.729+0000 7fd17914c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd1740b0c80 0x7fd17415d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:40.731 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.729+0000 7fd17914c640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd17415de20 0x7fd17415e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:40.731 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.729+0000 7fd17914c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd17415edd0 con 0x7fd15c004d00 2023-10-26T01:36:40.731 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.729+0000 7fd17914c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd174153ef0 con 0x7fd1740b0c80 2023-10-26T01:36:40.732 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.729+0000 7fd17914c640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1740b10e0 con 0x7fd17415de20 2023-10-26T01:36:40.732 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.729+0000 7fd172d9d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd15c004d00 0x7fd1740b0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:40.732 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.729+0000 7fd172d9d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd15c004d00 0x7fd1740b0740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:58798/0 (socket says 172.21.15.169:58798) 2023-10-26T01:36:40.732 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.729+0000 7fd172d9d640 1 -- 172.21.15.169:0/3343719622 learned_addr learned my addr 172.21.15.169:0/3343719622 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:36:40.732 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.729+0000 7fd172d9d640 1 -- 172.21.15.169:0/3343719622 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd17415de20 msgr2=0x7fd17415e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:40.733 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.729+0000 7fd172d9d640 1 --2- 172.21.15.169:0/3343719622 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd17415de20 0x7fd17415e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:40.733 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.729+0000 7fd172d9d640 1 -- 172.21.15.169:0/3343719622 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd1740b0c80 msgr2=0x7fd17415d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:40.733 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.729+0000 7fd172d9d640 1 --2- 172.21.15.169:0/3343719622 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd1740b0c80 0x7fd17415d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:40.733 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.729+0000 7fd172d9d640 1 -- 172.21.15.169:0/3343719622 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd174153550 con 0x7fd15c004d00 2023-10-26T01:36:40.734 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.730+0000 7fd172d9d640 1 --2- 172.21.15.169:0/3343719622 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd15c004d00 0x7fd1740b0740 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fd16400e9a0 tx=0x7fd16400ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:40.735 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.730+0000 7fd15b7fe640 1 -- 172.21.15.169:0/3343719622 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd16400cc40 con 0x7fd15c004d00 2023-10-26T01:36:40.735 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.730+0000 7fd15b7fe640 1 -- 172.21.15.169:0/3343719622 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd16400cda0 con 0x7fd15c004d00 2023-10-26T01:36:40.735 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.730+0000 7fd17914c640 1 -- 172.21.15.169:0/3343719622 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd174153830 con 0x7fd15c004d00 2023-10-26T01:36:40.735 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.730+0000 7fd15b7fe640 1 -- 172.21.15.169:0/3343719622 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd164005280 con 0x7fd15c004d00 2023-10-26T01:36:40.735 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.730+0000 7fd17914c640 1 -- 172.21.15.169:0/3343719622 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd17415e810 con 0x7fd15c004d00 2023-10-26T01:36:40.735 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.731+0000 7fd17914c640 1 -- 172.21.15.169:0/3343719622 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd17404db10 con 0x7fd15c004d00 2023-10-26T01:36:40.736 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.732+0000 7fd15b7fe640 1 -- 172.21.15.169:0/3343719622 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fd1640053e0 con 0x7fd15c004d00 2023-10-26T01:36:40.736 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.733+0000 7fd15b7fe640 1 --2- 172.21.15.169:0/3343719622 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd16006ae70 0x7fd16006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:40.736 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.733+0000 7fd15b7fe640 1 -- 172.21.15.169:0/3343719622 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd164014070 con 0x7fd15c004d00 2023-10-26T01:36:40.736 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.734+0000 7fd17259c640 1 --2- 172.21.15.169:0/3343719622 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd16006ae70 0x7fd16006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:40.737 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.735+0000 7fd17259c640 1 --2- 172.21.15.169:0/3343719622 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd16006ae70 0x7fd16006d330 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fd17415b1b0 tx=0x7fd1680015c0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:40.739 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.737+0000 7fd15b7fe640 1 -- 172.21.15.169:0/3343719622 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd164054fd0 con 0x7fd15c004d00 2023-10-26T01:36:40.930 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.928+0000 7fd17914c640 1 -- 172.21.15.169:0/3343719622 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fd17406c990 con 0x7fd15c004d00 2023-10-26T01:36:40.931 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.929+0000 7fd15b7fe640 1 -- 172.21.15.169:0/3343719622 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v99) v1 ==== 84+0+65 (secure 0 0 0) 0x7fd1640585f0 con 0x7fd15c004d00 2023-10-26T01:36:40.931 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:36:40.931 INFO:teuthology.orchestra.run.smithi169.stdout:[ 2023-10-26T01:36:40.932 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:36:40.932 INFO:teuthology.orchestra.run.smithi169.stdout: "poolnum": 1, 2023-10-26T01:36:40.932 INFO:teuthology.orchestra.run.smithi169.stdout: "poolname": ".mgr" 2023-10-26T01:36:40.932 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:36:40.932 INFO:teuthology.orchestra.run.smithi169.stdout:] 2023-10-26T01:36:40.933 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.932+0000 7fd17914c640 1 -- 172.21.15.169:0/3343719622 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd16006ae70 msgr2=0x7fd16006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:40.934 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.932+0000 7fd17914c640 1 --2- 172.21.15.169:0/3343719622 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd16006ae70 0x7fd16006d330 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fd17415b1b0 tx=0x7fd1680015c0 comp rx=0 tx=0).stop 2023-10-26T01:36:40.934 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.932+0000 7fd17914c640 1 -- 172.21.15.169:0/3343719622 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd15c004d00 msgr2=0x7fd1740b0740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:40.934 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.932+0000 7fd17914c640 1 --2- 172.21.15.169:0/3343719622 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd15c004d00 0x7fd1740b0740 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fd16400e9a0 tx=0x7fd16400ee60 comp rx=0 tx=0).stop 2023-10-26T01:36:40.934 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.932+0000 7fd17914c640 1 -- 172.21.15.169:0/3343719622 shutdown_connections 2023-10-26T01:36:40.935 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.932+0000 7fd17914c640 1 --2- 172.21.15.169:0/3343719622 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd16006ae70 0x7fd16006d330 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:40.935 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.932+0000 7fd17914c640 1 --2- 172.21.15.169:0/3343719622 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd17415de20 0x7fd17415e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:40.935 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.932+0000 7fd17914c640 1 --2- 172.21.15.169:0/3343719622 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd1740b0c80 0x7fd17415d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:40.935 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.932+0000 7fd17914c640 1 --2- 172.21.15.169:0/3343719622 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd15c004d00 0x7fd1740b0740 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:40.935 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.932+0000 7fd17914c640 1 -- 172.21.15.169:0/3343719622 >> 172.21.15.169:0/3343719622 conn(0x7fd174069930 msgr2=0x7fd174069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:40.935 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.932+0000 7fd17914c640 1 -- 172.21.15.169:0/3343719622 shutdown_connections 2023-10-26T01:36:40.936 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:40.932+0000 7fd17914c640 1 -- 172.21.15.169:0/3343719622 wait complete. 2023-10-26T01:36:40.953 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-26T01:36:40.953 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-26T01:36:41.180 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:41.178+0000 7fe22fa7c640 1 -- 172.21.15.169:0/2965729553 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe228152b70 msgr2=0x7fe22806bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:41.180 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:41.178+0000 7fe22fa7c640 1 --2- 172.21.15.169:0/2965729553 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe228152b70 0x7fe22806bd90 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fe22806ec50 tx=0x7fe22402bd70 comp rx=0 tx=0).stop 2023-10-26T01:36:41.181 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:41.178+0000 7fe22fa7c640 1 -- 172.21.15.169:0/2965729553 shutdown_connections 2023-10-26T01:36:41.181 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:41.178+0000 7fe22fa7c640 1 --2- 172.21.15.169:0/2965729553 >> v2:172.21.15.170:3300/0 conn(0x7fe2280abd20 0x7fe228158b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:41.181 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:41.178+0000 7fe22fa7c640 1 --2- 172.21.15.169:0/2965729553 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe228152b70 0x7fe22806bd90 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:41.181 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:41.178+0000 7fe22fa7c640 1 -- 172.21.15.169:0/2965729553 >> 172.21.15.169:0/2965729553 conn(0x7fe228069930 msgr2=0x7fe228069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:41.181 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:41.178+0000 7fe22fa7c640 1 -- 172.21.15.169:0/2965729553 shutdown_connections 2023-10-26T01:36:41.181 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:41.178+0000 7fe22fa7c640 1 -- 172.21.15.169:0/2965729553 wait complete. 2023-10-26T01:36:41.183 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:41.180+0000 7fe22fa7c640 1 Processor -- start 2023-10-26T01:36:41.183 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:41.180+0000 7fe22fa7c640 1 -- start start 2023-10-26T01:36:41.183 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:41.181+0000 7fe22fa7c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe2280ab8a0 0x7fe2280af910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:41.183 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:41.181+0000 7fe22fa7c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2280abd20 0x7fe22814e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:41.183 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:41.181+0000 7fe22fa7c640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe228152b70 0x7fe22814eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:41.184 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:41.181+0000 7fe22fa7c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2280aeec0 con 0x7fe2280abd20 2023-10-26T01:36:41.184 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:41.181+0000 7fe22fa7c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2280af480 con 0x7fe2280ab8a0 2023-10-26T01:36:41.184 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:41.181+0000 7fe22fa7c640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2280afe50 con 0x7fe228152b70 2023-10-26T01:36:41.184 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:41.181+0000 7fe22d017640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2280abd20 0x7fe22814e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:41.184 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:41.181+0000 7fe22d017640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2280abd20 0x7fe22814e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:58814/0 (socket says 172.21.15.169:58814) 2023-10-26T01:36:41.185 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:41.181+0000 7fe22d017640 1 -- 172.21.15.169:0/3579498926 learned_addr learned my addr 172.21.15.169:0/3579498926 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:36:41.186 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:41.182+0000 7fe22d017640 1 -- 172.21.15.169:0/3579498926 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe228152b70 msgr2=0x7fe22814eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:41.186 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:41.182+0000 7fe22d017640 1 --2- 172.21.15.169:0/3579498926 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe228152b70 0x7fe22814eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:41.186 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:41.182+0000 7fe22d818640 1 --2- 172.21.15.169:0/3579498926 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe2280ab8a0 0x7fe2280af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:41.187 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:41.182+0000 7fe22d017640 1 -- 172.21.15.169:0/3579498926 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe2280ab8a0 msgr2=0x7fe2280af910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:41.187 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:41.182+0000 7fe22d017640 1 --2- 172.21.15.169:0/3579498926 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe2280ab8a0 0x7fe2280af910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:41.187 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:41.182+0000 7fe22d017640 1 -- 172.21.15.169:0/3579498926 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe224009490 con 0x7fe2280abd20 2023-10-26T01:36:41.187 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:41.182+0000 7fe22d818640 1 --2- 172.21.15.169:0/3579498926 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe2280ab8a0 0x7fe2280af910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:36:41.187 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:41.182+0000 7fe22d017640 1 --2- 172.21.15.169:0/3579498926 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2280abd20 0x7fe22814e5c0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fe21c00c460 tx=0x7fe21c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:41.187 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:41.182+0000 7fe21a7fc640 1 -- 172.21.15.169:0/3579498926 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe21c004230 con 0x7fe2280abd20 2023-10-26T01:36:41.188 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:41.182+0000 7fe21a7fc640 1 -- 172.21.15.169:0/3579498926 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe21c0044a0 con 0x7fe2280abd20 2023-10-26T01:36:41.188 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:41.182+0000 7fe22fa7c640 1 -- 172.21.15.169:0/3579498926 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2281f06a0 con 0x7fe2280abd20 2023-10-26T01:36:41.188 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:41.182+0000 7fe21a7fc640 1 -- 172.21.15.169:0/3579498926 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe21c012890 con 0x7fe2280abd20 2023-10-26T01:36:41.188 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:41.183+0000 7fe22fa7c640 1 -- 172.21.15.169:0/3579498926 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2280af110 con 0x7fe2280abd20 2023-10-26T01:36:41.188 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:41.184+0000 7fe22fa7c640 1 -- 172.21.15.169:0/3579498926 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2281f0bc0 con 0x7fe2280abd20 2023-10-26T01:36:41.189 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:41.185+0000 7fe21a7fc640 1 -- 172.21.15.169:0/3579498926 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fe21c0035f0 con 0x7fe2280abd20 2023-10-26T01:36:41.189 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:41.186+0000 7fe21a7fc640 1 --2- 172.21.15.169:0/3579498926 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe20c06ae70 0x7fe20c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:41.189 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:41.186+0000 7fe21a7fc640 1 -- 172.21.15.169:0/3579498926 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fe21c0896d0 con 0x7fe2280abd20 2023-10-26T01:36:41.190 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:41.186+0000 7fe22d818640 1 --2- 172.21.15.169:0/3579498926 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe20c06ae70 0x7fe20c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:41.190 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:41.188+0000 7fe22d818640 1 --2- 172.21.15.169:0/3579498926 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe20c06ae70 0x7fe20c06d330 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fe224036000 tx=0x7fe224004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:41.191 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:41.190+0000 7fe21a7fc640 1 -- 172.21.15.169:0/3579498926 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe21c054b80 con 0x7fe2280abd20 2023-10-26T01:36:41.382 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:41.379+0000 7fe22fa7c640 1 -- 172.21.15.169:0/3579498926 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fe22806b2e0 con 0x7fe2280abd20 2023-10-26T01:36:42.373 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:42.370+0000 7fe21a7fc640 1 -- 172.21.15.169:0/3579498926 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v100) v1 ==== 138+0+0 (secure 0 0 0) 0x7fe21c0581a0 con 0x7fe2280abd20 2023-10-26T01:36:42.373 INFO:teuthology.orchestra.run.smithi169.stderr:pool 'cephfs_metadata' created 2023-10-26T01:36:42.377 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:42.375+0000 7fe22fa7c640 1 -- 172.21.15.169:0/3579498926 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe20c06ae70 msgr2=0x7fe20c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:42.378 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:42.375+0000 7fe22fa7c640 1 --2- 172.21.15.169:0/3579498926 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe20c06ae70 0x7fe20c06d330 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fe224036000 tx=0x7fe224004730 comp rx=0 tx=0).stop 2023-10-26T01:36:42.378 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:42.375+0000 7fe22fa7c640 1 -- 172.21.15.169:0/3579498926 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2280abd20 msgr2=0x7fe22814e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:42.378 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:42.375+0000 7fe22fa7c640 1 --2- 172.21.15.169:0/3579498926 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2280abd20 0x7fe22814e5c0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fe21c00c460 tx=0x7fe21c00c920 comp rx=0 tx=0).stop 2023-10-26T01:36:42.378 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:42.375+0000 7fe22fa7c640 1 -- 172.21.15.169:0/3579498926 shutdown_connections 2023-10-26T01:36:42.379 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:42.375+0000 7fe22fa7c640 1 --2- 172.21.15.169:0/3579498926 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe20c06ae70 0x7fe20c06d330 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:42.379 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:42.375+0000 7fe22fa7c640 1 --2- 172.21.15.169:0/3579498926 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2280abd20 0x7fe22814e5c0 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:42.379 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:42.375+0000 7fe22fa7c640 1 --2- 172.21.15.169:0/3579498926 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe228152b70 0x7fe22814eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:42.379 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:42.375+0000 7fe22fa7c640 1 --2- 172.21.15.169:0/3579498926 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe2280ab8a0 0x7fe2280af910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:42.380 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:42.375+0000 7fe22fa7c640 1 -- 172.21.15.169:0/3579498926 >> 172.21.15.169:0/3579498926 conn(0x7fe228069930 msgr2=0x7fe2280aa1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:42.380 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:42.376+0000 7fe22fa7c640 1 -- 172.21.15.169:0/3579498926 shutdown_connections 2023-10-26T01:36:42.380 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:42.376+0000 7fe22fa7c640 1 -- 172.21.15.169:0/3579498926 wait complete. 2023-10-26T01:36:42.402 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-26T01:36:42.622 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:42.620+0000 7fcf2bffc640 1 -- 172.21.15.169:0/3320710839 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf24070c70 msgr2=0x7fcf240cb8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:42.623 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:42.620+0000 7fcf2bffc640 1 --2- 172.21.15.169:0/3320710839 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf24070c70 0x7fcf240cb8e0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fcf2406f2e0 tx=0x7fcf1c02bda0 comp rx=0 tx=0).stop 2023-10-26T01:36:42.623 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:42.620+0000 7fcf2bffc640 1 -- 172.21.15.169:0/3320710839 shutdown_connections 2023-10-26T01:36:42.623 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:42.620+0000 7fcf2bffc640 1 --2- 172.21.15.169:0/3320710839 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf24070c70 0x7fcf240cb8e0 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:42.623 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:42.620+0000 7fcf2bffc640 1 --2- 172.21.15.169:0/3320710839 >> v2:172.21.15.170:3300/0 conn(0x7fcf240713d0 0x7fcf24070730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:42.623 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:42.620+0000 7fcf2bffc640 1 --2- 172.21.15.169:0/3320710839 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcf2415ca70 0x7fcf2415d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:42.624 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:42.620+0000 7fcf2bffc640 1 -- 172.21.15.169:0/3320710839 >> 172.21.15.169:0/3320710839 conn(0x7fcf240af8a0 msgr2=0x7fcf240afc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:42.624 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:42.620+0000 7fcf2bffc640 1 -- 172.21.15.169:0/3320710839 shutdown_connections 2023-10-26T01:36:42.624 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:42.621+0000 7fcf2bffc640 1 -- 172.21.15.169:0/3320710839 wait complete. 2023-10-26T01:36:42.626 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:42.622+0000 7fcf2bffc640 1 Processor -- start 2023-10-26T01:36:42.626 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:42.622+0000 7fcf2bffc640 1 -- start start 2023-10-26T01:36:42.626 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:42.623+0000 7fcf2bffc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf24070c70 0x7fcf241f2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:42.627 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:42.623+0000 7fcf2bffc640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcf240713d0 0x7fcf2415ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:42.627 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:42.623+0000 7fcf2bffc640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf2415ca70 0x7fcf2414d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:42.627 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:42.623+0000 7fcf2bffc640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf2415a700 con 0x7fcf2415ca70 2023-10-26T01:36:42.627 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:42.623+0000 7fcf2bffc640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf2415b060 con 0x7fcf24070c70 2023-10-26T01:36:42.627 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:42.623+0000 7fcf2bffc640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf2415a060 con 0x7fcf240713d0 2023-10-26T01:36:42.628 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:42.624+0000 7fcf2a599640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf2415ca70 0x7fcf2414d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:42.628 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:42.624+0000 7fcf2a599640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf2415ca70 0x7fcf2414d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:58844/0 (socket says 172.21.15.169:58844) 2023-10-26T01:36:42.628 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:42.624+0000 7fcf2a599640 1 -- 172.21.15.169:0/327100723 learned_addr learned my addr 172.21.15.169:0/327100723 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:36:42.628 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:42.624+0000 7fcf29d98640 1 --2- 172.21.15.169:0/327100723 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf24070c70 0x7fcf241f2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:42.629 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:42.624+0000 7fcf29597640 1 --2- 172.21.15.169:0/327100723 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcf240713d0 0x7fcf2415ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:42.629 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:42.624+0000 7fcf2a599640 1 -- 172.21.15.169:0/327100723 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcf240713d0 msgr2=0x7fcf2415ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:42.629 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:42.624+0000 7fcf2a599640 1 --2- 172.21.15.169:0/327100723 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcf240713d0 0x7fcf2415ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:42.629 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:42.624+0000 7fcf2a599640 1 -- 172.21.15.169:0/327100723 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf24070c70 msgr2=0x7fcf241f2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:42.629 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:42.624+0000 7fcf2a599640 1 --2- 172.21.15.169:0/327100723 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf24070c70 0x7fcf241f2ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:42.630 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:42.624+0000 7fcf2a599640 1 -- 172.21.15.169:0/327100723 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf1c009560 con 0x7fcf2415ca70 2023-10-26T01:36:42.630 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:42.624+0000 7fcf2a599640 1 --2- 172.21.15.169:0/327100723 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf2415ca70 0x7fcf2414d300 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c036040 tx=0x7fcf1c02dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:42.631 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:42.625+0000 7fcf227fc640 1 -- 172.21.15.169:0/327100723 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf1c003800 con 0x7fcf2415ca70 2023-10-26T01:36:42.631 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:42.625+0000 7fcf227fc640 1 -- 172.21.15.169:0/327100723 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fcf1c003a50 con 0x7fcf2415ca70 2023-10-26T01:36:42.632 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:42.625+0000 7fcf227fc640 1 -- 172.21.15.169:0/327100723 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf1c003e60 con 0x7fcf2415ca70 2023-10-26T01:36:42.632 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:42.625+0000 7fcf2bffc640 1 -- 172.21.15.169:0/327100723 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf241f2240 con 0x7fcf2415ca70 2023-10-26T01:36:42.632 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:42.625+0000 7fcf2bffc640 1 -- 172.21.15.169:0/327100723 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf2414e150 con 0x7fcf2415ca70 2023-10-26T01:36:42.632 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:42.626+0000 7fcf2bffc640 1 -- 172.21.15.169:0/327100723 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf2414e340 con 0x7fcf2415ca70 2023-10-26T01:36:42.632 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:42.627+0000 7fcf227fc640 1 -- 172.21.15.169:0/327100723 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fcf1c03e450 con 0x7fcf2415ca70 2023-10-26T01:36:42.633 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:42.628+0000 7fcf227fc640 1 --2- 172.21.15.169:0/327100723 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcf1006ae40 0x7fcf1006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:42.633 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:42.628+0000 7fcf227fc640 1 -- 172.21.15.169:0/327100723 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fcf1c0ac3c0 con 0x7fcf2415ca70 2023-10-26T01:36:42.633 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:42.629+0000 7fcf29d98640 1 --2- 172.21.15.169:0/327100723 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcf1006ae40 0x7fcf1006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:42.633 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:42.630+0000 7fcf29d98640 1 --2- 172.21.15.169:0/327100723 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcf1006ae40 0x7fcf1006d300 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fcf140092f0 tx=0x7fcf14009280 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:42.634 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:42.632+0000 7fcf227fc640 1 -- 172.21.15.169:0/327100723 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcf1c0778e0 con 0x7fcf2415ca70 2023-10-26T01:36:42.830 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:42.828+0000 7fcf2bffc640 1 -- 172.21.15.169:0/327100723 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fcf24070260 con 0x7fcf2415ca70 2023-10-26T01:36:43.386 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:43.384+0000 7fcf227fc640 1 -- 172.21.15.169:0/327100723 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v101) v1 ==== 170+0+0 (secure 0 0 0) 0x7fcf1c0420d0 con 0x7fcf2415ca70 2023-10-26T01:36:43.387 INFO:teuthology.orchestra.run.smithi169.stderr:pool 'cephfs_data' created 2023-10-26T01:36:43.391 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:43.388+0000 7fcf2bffc640 1 -- 172.21.15.169:0/327100723 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcf1006ae40 msgr2=0x7fcf1006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:43.391 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:43.389+0000 7fcf2bffc640 1 --2- 172.21.15.169:0/327100723 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcf1006ae40 0x7fcf1006d300 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fcf140092f0 tx=0x7fcf14009280 comp rx=0 tx=0).stop 2023-10-26T01:36:43.392 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:43.389+0000 7fcf2bffc640 1 -- 172.21.15.169:0/327100723 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf2415ca70 msgr2=0x7fcf2414d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:43.392 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:43.389+0000 7fcf2bffc640 1 --2- 172.21.15.169:0/327100723 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf2415ca70 0x7fcf2414d300 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c036040 tx=0x7fcf1c02dac0 comp rx=0 tx=0).stop 2023-10-26T01:36:43.392 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:43.389+0000 7fcf2bffc640 1 -- 172.21.15.169:0/327100723 shutdown_connections 2023-10-26T01:36:43.393 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:43.389+0000 7fcf2bffc640 1 --2- 172.21.15.169:0/327100723 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcf1006ae40 0x7fcf1006d300 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:43.393 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:43.389+0000 7fcf2bffc640 1 --2- 172.21.15.169:0/327100723 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf2415ca70 0x7fcf2414d300 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:43.393 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:43.389+0000 7fcf2bffc640 1 --2- 172.21.15.169:0/327100723 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcf240713d0 0x7fcf2415ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:43.394 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:43.389+0000 7fcf2bffc640 1 --2- 172.21.15.169:0/327100723 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf24070c70 0x7fcf241f2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:43.394 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:43.389+0000 7fcf2bffc640 1 -- 172.21.15.169:0/327100723 >> 172.21.15.169:0/327100723 conn(0x7fcf240af8a0 msgr2=0x7fcf240aea20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:43.395 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:43.389+0000 7fcf2bffc640 1 -- 172.21.15.169:0/327100723 shutdown_connections 2023-10-26T01:36:43.395 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:43.389+0000 7fcf2bffc640 1 -- 172.21.15.169:0/327100723 wait complete. 2023-10-26T01:36:43.422 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-26T01:36:43.620 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:43.618+0000 7fc1c30a3640 1 -- 172.21.15.169:0/482510718 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1bc06a070 msgr2=0x7fc1bc15f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:43.621 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:43.618+0000 7fc1c30a3640 1 --2- 172.21.15.169:0/482510718 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1bc06a070 0x7fc1bc15f450 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc06e490 tx=0x7fc1b402c260 comp rx=0 tx=0).stop 2023-10-26T01:36:43.621 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:43.619+0000 7fc1c30a3640 1 -- 172.21.15.169:0/482510718 shutdown_connections 2023-10-26T01:36:43.621 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:43.619+0000 7fc1c30a3640 1 --2- 172.21.15.169:0/482510718 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1bc06a070 0x7fc1bc15f450 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:43.621 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:43.619+0000 7fc1c30a3640 1 --2- 172.21.15.169:0/482510718 >> v2:172.21.15.170:3300/0 conn(0x7fc1bc0ac510 0x7fc1bc069b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:43.622 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:43.619+0000 7fc1c30a3640 1 --2- 172.21.15.169:0/482510718 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc1bc0ac130 0x7fc1bc06aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:43.622 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:43.619+0000 7fc1c30a3640 1 -- 172.21.15.169:0/482510718 >> 172.21.15.169:0/482510718 conn(0x7fc1bc0af110 msgr2=0x7fc1bc0af520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:43.622 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:43.619+0000 7fc1c30a3640 1 -- 172.21.15.169:0/482510718 shutdown_connections 2023-10-26T01:36:43.622 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:43.619+0000 7fc1c30a3640 1 -- 172.21.15.169:0/482510718 wait complete. 2023-10-26T01:36:43.625 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:43.620+0000 7fc1c30a3640 1 Processor -- start 2023-10-26T01:36:43.625 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:43.621+0000 7fc1c30a3640 1 -- start start 2023-10-26T01:36:43.625 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:43.621+0000 7fc1c30a3640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1bc06a070 0x7fc1bc0709f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:43.625 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:43.621+0000 7fc1c30a3640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc1bc0ac130 0x7fc1bc140b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:43.625 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:43.621+0000 7fc1c30a3640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1bc0ac510 0x7fc1bc1589a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:43.626 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:43.621+0000 7fc1c30a3640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1bc152ea0 con 0x7fc1bc06a070 2023-10-26T01:36:43.626 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:43.621+0000 7fc1c30a3640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1bc1410d0 con 0x7fc1bc0ac510 2023-10-26T01:36:43.626 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:43.621+0000 7fc1c30a3640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1bc0710d0 con 0x7fc1bc0ac130 2023-10-26T01:36:43.627 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:43.622+0000 7fc1c20a1640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1bc06a070 0x7fc1bc0709f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:43.627 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:43.622+0000 7fc1c20a1640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1bc06a070 0x7fc1bc0709f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:58872/0 (socket says 172.21.15.169:58872) 2023-10-26T01:36:43.627 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:43.622+0000 7fc1c20a1640 1 -- 172.21.15.169:0/7118675 learned_addr learned my addr 172.21.15.169:0/7118675 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:36:43.628 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:43.622+0000 7fc1c18a0640 1 --2- 172.21.15.169:0/7118675 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc1bc0ac130 0x7fc1bc140b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:43.628 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:43.622+0000 7fc1c20a1640 1 -- 172.21.15.169:0/7118675 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc1bc0ac130 msgr2=0x7fc1bc140b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:43.628 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:43.622+0000 7fc1c20a1640 1 --2- 172.21.15.169:0/7118675 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc1bc0ac130 0x7fc1bc140b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:43.628 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:43.622+0000 7fc1c20a1640 1 -- 172.21.15.169:0/7118675 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1bc0ac510 msgr2=0x7fc1bc1589a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:43.629 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:43.622+0000 7fc1c20a1640 1 --2- 172.21.15.169:0/7118675 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1bc0ac510 0x7fc1bc1589a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:43.629 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:43.622+0000 7fc1c20a1640 1 -- 172.21.15.169:0/7118675 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1b4009900 con 0x7fc1bc06a070 2023-10-26T01:36:43.630 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:43.623+0000 7fc1c20a1640 1 --2- 172.21.15.169:0/7118675 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1bc06a070 0x7fc1bc0709f0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fc1b800d750 tx=0x7fc1b800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:43.630 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:43.623+0000 7fc1b2ffd640 1 -- 172.21.15.169:0/7118675 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b8009bd0 con 0x7fc1bc06a070 2023-10-26T01:36:43.630 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:43.623+0000 7fc1b2ffd640 1 -- 172.21.15.169:0/7118675 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc1b8009e20 con 0x7fc1bc06a070 2023-10-26T01:36:43.631 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:43.623+0000 7fc1b2ffd640 1 -- 172.21.15.169:0/7118675 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b8005280 con 0x7fc1bc06a070 2023-10-26T01:36:43.631 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:43.623+0000 7fc1c30a3640 1 -- 172.21.15.169:0/7118675 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1bc0713b0 con 0x7fc1bc06a070 2023-10-26T01:36:43.631 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:43.623+0000 7fc1c30a3640 1 -- 172.21.15.169:0/7118675 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1bc071970 con 0x7fc1bc06a070 2023-10-26T01:36:43.633 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:43.624+0000 7fc1c30a3640 1 -- 172.21.15.169:0/7118675 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1bc153010 con 0x7fc1bc06a070 2023-10-26T01:36:43.633 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:43.625+0000 7fc1b2ffd640 1 -- 172.21.15.169:0/7118675 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fc1b80053e0 con 0x7fc1bc06a070 2023-10-26T01:36:43.634 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:43.626+0000 7fc1b2ffd640 1 --2- 172.21.15.169:0/7118675 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc1a006ae40 0x7fc1a006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:43.634 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:43.626+0000 7fc1b2ffd640 1 -- 172.21.15.169:0/7118675 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fc1b8089860 con 0x7fc1bc06a070 2023-10-26T01:36:43.634 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:43.627+0000 7fc1c18a0640 1 --2- 172.21.15.169:0/7118675 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc1a006ae40 0x7fc1a006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:43.635 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:43.628+0000 7fc1c18a0640 1 --2- 172.21.15.169:0/7118675 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc1a006ae40 0x7fc1a006d300 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac005fd0 tx=0x7fc1ac005850 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:43.635 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:43.631+0000 7fc1b2ffd640 1 -- 172.21.15.169:0/7118675 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc1bc153010 con 0x7fc1bc06a070 2023-10-26T01:36:43.848 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:43.846+0000 7fc1c30a3640 1 -- 172.21.15.169:0/7118675 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fc1bc152fe0 con 0x7fc1bc06a070 2023-10-26T01:36:44.414 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:44.412+0000 7fc1b2ffd640 1 -- 172.21.15.169:0/7118675 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 12 and data pool 13 v46) v1 ==== 174+0+0 (secure 0 0 0) 0x7fc1b80170c0 con 0x7fc1bc06a070 2023-10-26T01:36:44.417 INFO:teuthology.orchestra.run.smithi169.stderr:new fs with metadata pool 12 and data pool 13 2023-10-26T01:36:44.422 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:44.421+0000 7fc1c30a3640 1 -- 172.21.15.169:0/7118675 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc1a006ae40 msgr2=0x7fc1a006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:44.423 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:44.421+0000 7fc1c30a3640 1 --2- 172.21.15.169:0/7118675 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc1a006ae40 0x7fc1a006d300 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac005fd0 tx=0x7fc1ac005850 comp rx=0 tx=0).stop 2023-10-26T01:36:44.423 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:44.421+0000 7fc1c30a3640 1 -- 172.21.15.169:0/7118675 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1bc06a070 msgr2=0x7fc1bc0709f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:44.423 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:44.421+0000 7fc1c30a3640 1 --2- 172.21.15.169:0/7118675 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1bc06a070 0x7fc1bc0709f0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fc1b800d750 tx=0x7fc1b800dc10 comp rx=0 tx=0).stop 2023-10-26T01:36:44.424 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:44.421+0000 7fc1c30a3640 1 -- 172.21.15.169:0/7118675 shutdown_connections 2023-10-26T01:36:44.424 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:44.421+0000 7fc1c30a3640 1 --2- 172.21.15.169:0/7118675 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc1a006ae40 0x7fc1a006d300 secure :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac005fd0 tx=0x7fc1ac005850 comp rx=0 tx=0).stop 2023-10-26T01:36:44.424 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:44.421+0000 7fc1c30a3640 1 --2- 172.21.15.169:0/7118675 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1bc0ac510 0x7fc1bc1589a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:44.425 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:44.422+0000 7fc1c30a3640 1 --2- 172.21.15.169:0/7118675 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc1bc0ac130 0x7fc1bc140b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:44.425 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:44.422+0000 7fc1c30a3640 1 --2- 172.21.15.169:0/7118675 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1bc06a070 0x7fc1bc0709f0 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:44.425 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:44.422+0000 7fc1c30a3640 1 -- 172.21.15.169:0/7118675 >> 172.21.15.169:0/7118675 conn(0x7fc1bc0af110 msgr2=0x7fc1bc159820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:44.425 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:44.422+0000 7fc1c30a3640 1 -- 172.21.15.169:0/7118675 shutdown_connections 2023-10-26T01:36:44.426 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:44.423+0000 7fc1c30a3640 1 -- 172.21.15.169:0/7118675 wait complete. 2023-10-26T01:36:44.451 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:36:44.643 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:44.640+0000 7f9ec5b93640 1 -- 172.21.15.169:0/1738690533 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9ec006b8b0 msgr2=0x7f9ec006bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:44.643 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:44.640+0000 7f9ec5b93640 1 --2- 172.21.15.169:0/1738690533 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9ec006b8b0 0x7f9ec006bd20 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f9eb8005fd0 tx=0x7f9eb802c0f0 comp rx=0 tx=0).stop 2023-10-26T01:36:44.643 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:44.641+0000 7f9ec5b93640 1 -- 172.21.15.169:0/1738690533 shutdown_connections 2023-10-26T01:36:44.644 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:44.641+0000 7f9ec5b93640 1 --2- 172.21.15.169:0/1738690533 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9ec006b8b0 0x7f9ec006bd20 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:44.644 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:44.641+0000 7f9ec5b93640 1 --2- 172.21.15.169:0/1738690533 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9ec00c8280 0x7f9ec00c86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:44.644 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:44.641+0000 7f9ec5b93640 1 -- 172.21.15.169:0/1738690533 >> 172.21.15.169:0/1738690533 conn(0x7f9ec00aedd0 msgr2=0x7f9ec00af1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:44.644 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:44.641+0000 7f9ec5b93640 1 -- 172.21.15.169:0/1738690533 shutdown_connections 2023-10-26T01:36:44.644 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:44.641+0000 7f9ec5b93640 1 -- 172.21.15.169:0/1738690533 wait complete. 2023-10-26T01:36:44.646 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:44.642+0000 7f9ec5b93640 1 Processor -- start 2023-10-26T01:36:44.647 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:44.643+0000 7f9ec5b93640 1 -- start start 2023-10-26T01:36:44.647 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:44.644+0000 7f9ec5b93640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9ec006b8b0 0x7f9ec00b2970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:44.647 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:44.644+0000 7f9ec5b93640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ec00c8280 0x7f9ec01f1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:44.647 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:44.644+0000 7f9ec5b93640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9ec0158070 0x7f9ec014cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:44.647 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:44.644+0000 7f9ec5b93640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ec01f2cd0 con 0x7f9ec00c8280 2023-10-26T01:36:44.648 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:44.644+0000 7f9ec5b93640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ec01f2f70 con 0x7f9ec006b8b0 2023-10-26T01:36:44.648 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:44.644+0000 7f9ec5b93640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ec01f2360 con 0x7f9ec0158070 2023-10-26T01:36:44.648 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:44.644+0000 7f9ebffff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ec00c8280 0x7f9ec01f1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:44.648 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:44.644+0000 7f9ebffff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ec00c8280 0x7f9ec01f1b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:59372/0 (socket says 172.21.15.169:59372) 2023-10-26T01:36:44.649 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:44.644+0000 7f9ebffff640 1 -- 172.21.15.169:0/1255142247 learned_addr learned my addr 172.21.15.169:0/1255142247 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:36:44.649 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:44.644+0000 7f9ec4b91640 1 --2- 172.21.15.169:0/1255142247 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9ec006b8b0 0x7f9ec00b2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:44.649 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:44.644+0000 7f9ec5392640 1 --2- 172.21.15.169:0/1255142247 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9ec0158070 0x7f9ec014cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:44.649 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:44.644+0000 7f9ebffff640 1 -- 172.21.15.169:0/1255142247 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9ec0158070 msgr2=0x7f9ec014cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:44.649 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:44.644+0000 7f9ebffff640 1 --2- 172.21.15.169:0/1255142247 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9ec0158070 0x7f9ec014cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:44.650 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:44.644+0000 7f9ebffff640 1 -- 172.21.15.169:0/1255142247 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9ec006b8b0 msgr2=0x7f9ec00b2970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:44.650 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:44.644+0000 7f9ebffff640 1 --2- 172.21.15.169:0/1255142247 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9ec006b8b0 0x7f9ec00b2970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:44.650 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:44.644+0000 7f9ebffff640 1 -- 172.21.15.169:0/1255142247 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ea8009490 con 0x7f9ec00c8280 2023-10-26T01:36:44.650 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:44.644+0000 7f9ec4b91640 1 --2- 172.21.15.169:0/1255142247 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9ec006b8b0 0x7f9ec00b2970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:36:44.650 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:44.645+0000 7f9ebffff640 1 --2- 172.21.15.169:0/1255142247 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ec00c8280 0x7f9ec01f1b20 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f9ea800ea60 tx=0x7f9ea800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:44.651 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:44.645+0000 7f9ebd7fa640 1 -- 172.21.15.169:0/1255142247 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea800cde0 con 0x7f9ec00c8280 2023-10-26T01:36:44.651 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:44.645+0000 7f9ec5b93640 1 -- 172.21.15.169:0/1255142247 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9eb8009840 con 0x7f9ec00c8280 2023-10-26T01:36:44.651 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:44.645+0000 7f9ebd7fa640 1 -- 172.21.15.169:0/1255142247 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9ea8004020 con 0x7f9ec00c8280 2023-10-26T01:36:44.651 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:44.645+0000 7f9ebd7fa640 1 -- 172.21.15.169:0/1255142247 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea8004410 con 0x7f9ec00c8280 2023-10-26T01:36:44.651 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:44.645+0000 7f9ec5b93640 1 -- 172.21.15.169:0/1255142247 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ec01f2920 con 0x7f9ec00c8280 2023-10-26T01:36:44.653 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:44.646+0000 7f9ec5b93640 1 -- 172.21.15.169:0/1255142247 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ec0073c10 con 0x7f9ec00c8280 2023-10-26T01:36:44.653 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:44.647+0000 7f9ebd7fa640 1 -- 172.21.15.169:0/1255142247 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f9ea80035f0 con 0x7f9ec00c8280 2023-10-26T01:36:44.653 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:44.648+0000 7f9ebd7fa640 1 --2- 172.21.15.169:0/1255142247 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9ea006ae40 0x7f9ea006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:44.653 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:44.648+0000 7f9ebd7fa640 1 -- 172.21.15.169:0/1255142247 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9ea8014070 con 0x7f9ec00c8280 2023-10-26T01:36:44.653 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:44.649+0000 7f9ec4b91640 1 --2- 172.21.15.169:0/1255142247 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9ea006ae40 0x7f9ea006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:44.654 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:44.650+0000 7f9ec4b91640 1 --2- 172.21.15.169:0/1255142247 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9ea006ae40 0x7f9ea006d300 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f9eb00046b0 tx=0x7f9eb0009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:44.654 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:44.652+0000 7f9ebd7fa640 1 -- 172.21.15.169:0/1255142247 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9ea80552f0 con 0x7f9ec00c8280 2023-10-26T01:36:44.845 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:44.843+0000 7f9ec5b93640 1 -- 172.21.15.169:0/1255142247 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9ec009ae30 con 0x7f9ec00c8280 2023-10-26T01:36:44.848 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:44.846+0000 7f9ebd7fa640 1 -- 172.21.15.169:0/1255142247 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) v1 ==== 74+0+16632 (secure 0 0 0) 0x7f9ea8058910 con 0x7f9ec00c8280 2023-10-26T01:36:44.848 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:36:44.848 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":102,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:36:43.848186+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-10-26T01:36:41.430455+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-10-26T01:36:42.883252+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:36:44.851 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:44.848+0000 7f9ec5b93640 1 -- 172.21.15.169:0/1255142247 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9ea006ae40 msgr2=0x7f9ea006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:44.851 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:44.848+0000 7f9ec5b93640 1 --2- 172.21.15.169:0/1255142247 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9ea006ae40 0x7f9ea006d300 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f9eb00046b0 tx=0x7f9eb0009040 comp rx=0 tx=0).stop 2023-10-26T01:36:44.851 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:44.848+0000 7f9ec5b93640 1 -- 172.21.15.169:0/1255142247 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ec00c8280 msgr2=0x7f9ec01f1b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:44.851 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:44.848+0000 7f9ec5b93640 1 --2- 172.21.15.169:0/1255142247 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ec00c8280 0x7f9ec01f1b20 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f9ea800ea60 tx=0x7f9ea800ef20 comp rx=0 tx=0).stop 2023-10-26T01:36:44.851 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:44.849+0000 7f9ec5b93640 1 -- 172.21.15.169:0/1255142247 shutdown_connections 2023-10-26T01:36:44.852 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:44.849+0000 7f9ec5b93640 1 --2- 172.21.15.169:0/1255142247 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9ea006ae40 0x7f9ea006d300 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:44.852 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:44.849+0000 7f9ec5b93640 1 --2- 172.21.15.169:0/1255142247 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ec00c8280 0x7f9ec01f1b20 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:44.852 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:44.849+0000 7f9ec5b93640 1 --2- 172.21.15.169:0/1255142247 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9ec0158070 0x7f9ec014cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:44.852 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:44.849+0000 7f9ec5b93640 1 --2- 172.21.15.169:0/1255142247 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9ec006b8b0 0x7f9ec00b2970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:44.852 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:44.849+0000 7f9ec5b93640 1 -- 172.21.15.169:0/1255142247 >> 172.21.15.169:0/1255142247 conn(0x7f9ec00aedd0 msgr2=0x7f9ec00acb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:44.853 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:44.849+0000 7f9ec5b93640 1 -- 172.21.15.169:0/1255142247 shutdown_connections 2023-10-26T01:36:44.853 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:44.849+0000 7f9ec5b93640 1 -- 172.21.15.169:0/1255142247 wait complete. 2023-10-26T01:36:44.869 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:36:45.079 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:45.077+0000 7fcdea725640 1 -- 172.21.15.169:0/200131345 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcdd0003970 msgr2=0x7fcdd0003d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:45.080 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:45.077+0000 7fcdea725640 1 --2- 172.21.15.169:0/200131345 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcdd0003970 0x7fcdd0003d70 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fcdd4006fc0 tx=0x7fcdd00057f0 comp rx=0 tx=0).stop 2023-10-26T01:36:45.080 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:45.078+0000 7fcdea725640 1 -- 172.21.15.169:0/200131345 shutdown_connections 2023-10-26T01:36:45.080 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:45.078+0000 7fcdea725640 1 --2- 172.21.15.169:0/200131345 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcdd0003970 0x7fcdd0003d70 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:45.081 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:45.078+0000 7fcdea725640 1 -- 172.21.15.169:0/200131345 >> 172.21.15.169:0/200131345 conn(0x7fcde4069930 msgr2=0x7fcde4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:45.081 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:45.078+0000 7fcdea725640 1 -- 172.21.15.169:0/200131345 shutdown_connections 2023-10-26T01:36:45.081 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:45.078+0000 7fcdea725640 1 -- 172.21.15.169:0/200131345 wait complete. 2023-10-26T01:36:45.082 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:45.079+0000 7fcdea725640 1 Processor -- start 2023-10-26T01:36:45.082 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:45.080+0000 7fcdea725640 1 -- start start 2023-10-26T01:36:45.085 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:45.080+0000 7fcdea725640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcdd0003970 0x7fcde41418c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:45.085 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:45.081+0000 7fcdea725640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcde4152310 0x7fcde4152790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:45.086 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:45.081+0000 7fcdea725640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcde406fd20 0x7fcde40701d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:45.086 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:45.081+0000 7fcdea725640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcde4152cd0 con 0x7fcde4152310 2023-10-26T01:36:45.086 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:45.081+0000 7fcdea725640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcde4154810 con 0x7fcdd0003970 2023-10-26T01:36:45.086 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:45.081+0000 7fcdea725640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcde4141e00 con 0x7fcde406fd20 2023-10-26T01:36:45.086 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:45.081+0000 7fcddbfff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcde4152310 0x7fcde4152790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:45.087 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:45.081+0000 7fcddbfff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcde4152310 0x7fcde4152790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:59394/0 (socket says 172.21.15.169:59394) 2023-10-26T01:36:45.087 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:45.081+0000 7fcddbfff640 1 -- 172.21.15.169:0/3574368494 learned_addr learned my addr 172.21.15.169:0/3574368494 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:36:45.087 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:45.081+0000 7fcddbfff640 1 -- 172.21.15.169:0/3574368494 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcde406fd20 msgr2=0x7fcde40701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:45.087 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:45.081+0000 7fcddbfff640 1 --2- 172.21.15.169:0/3574368494 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcde406fd20 0x7fcde40701d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:45.087 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:45.081+0000 7fcddbfff640 1 -- 172.21.15.169:0/3574368494 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcdd0003970 msgr2=0x7fcde41418c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:45.088 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:45.081+0000 7fcddbfff640 1 --2- 172.21.15.169:0/3574368494 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcdd0003970 0x7fcde41418c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:45.088 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:45.081+0000 7fcddbfff640 1 -- 172.21.15.169:0/3574368494 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcde4070710 con 0x7fcde4152310 2023-10-26T01:36:45.088 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:45.081+0000 7fcddbfff640 1 --2- 172.21.15.169:0/3574368494 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcde4152310 0x7fcde4152790 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc00d330 tx=0x7fcdcc00d7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:45.088 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:45.082+0000 7fcde17fa640 1 -- 172.21.15.169:0/3574368494 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc00f880 con 0x7fcde4152310 2023-10-26T01:36:45.088 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:45.082+0000 7fcde17fa640 1 -- 172.21.15.169:0/3574368494 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fcdcc00faf0 con 0x7fcde4152310 2023-10-26T01:36:45.089 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:45.082+0000 7fcdea725640 1 -- 172.21.15.169:0/3574368494 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcde4070970 con 0x7fcde4152310 2023-10-26T01:36:45.089 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:45.082+0000 7fcde17fa640 1 -- 172.21.15.169:0/3574368494 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc010450 con 0x7fcde4152310 2023-10-26T01:36:45.089 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:45.082+0000 7fcdea725640 1 -- 172.21.15.169:0/3574368494 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcde4070f30 con 0x7fcde4152310 2023-10-26T01:36:45.089 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:45.083+0000 7fcdea725640 1 -- 172.21.15.169:0/3574368494 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcde404db10 con 0x7fcde4152310 2023-10-26T01:36:45.089 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:45.084+0000 7fcde17fa640 1 -- 172.21.15.169:0/3574368494 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fcdcc019400 con 0x7fcde4152310 2023-10-26T01:36:45.090 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:45.085+0000 7fcde17fa640 1 --2- 172.21.15.169:0/3574368494 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcdc406ae70 0x7fcdc406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:45.090 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:45.085+0000 7fcde17fa640 1 -- 172.21.15.169:0/3574368494 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcdcc089e10 con 0x7fcde4152310 2023-10-26T01:36:45.091 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:45.085+0000 7fcde3fff640 1 --2- 172.21.15.169:0/3574368494 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcdc406ae70 0x7fcdc406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:45.091 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:45.087+0000 7fcde3fff640 1 --2- 172.21.15.169:0/3574368494 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcdc406ae70 0x7fcdc406d330 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fcdd4034000 tx=0x7fcdd4039040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:45.091 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:45.089+0000 7fcde17fa640 1 -- 172.21.15.169:0/3574368494 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcdcc08d030 con 0x7fcde4152310 2023-10-26T01:36:45.285 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:45.283+0000 7fcdea725640 1 -- 172.21.15.169:0/3574368494 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcde40b0070 con 0x7fcde4152310 2023-10-26T01:36:45.286 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:45.285+0000 7fcde17fa640 1 -- 172.21.15.169:0/3574368494 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) v1 ==== 74+0+16632 (secure 0 0 0) 0x7fcdcc014020 con 0x7fcde4152310 2023-10-26T01:36:45.287 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:36:45.287 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":102,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:36:43.848186+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-10-26T01:36:41.430455+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-10-26T01:36:42.883252+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:36:45.289 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:45.287+0000 7fcdea725640 1 -- 172.21.15.169:0/3574368494 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcdc406ae70 msgr2=0x7fcdc406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:45.289 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:45.287+0000 7fcdea725640 1 --2- 172.21.15.169:0/3574368494 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcdc406ae70 0x7fcdc406d330 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fcdd4034000 tx=0x7fcdd4039040 comp rx=0 tx=0).stop 2023-10-26T01:36:45.290 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:45.287+0000 7fcdea725640 1 -- 172.21.15.169:0/3574368494 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcde4152310 msgr2=0x7fcde4152790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:45.290 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:45.287+0000 7fcdea725640 1 --2- 172.21.15.169:0/3574368494 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcde4152310 0x7fcde4152790 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc00d330 tx=0x7fcdcc00d7f0 comp rx=0 tx=0).stop 2023-10-26T01:36:45.290 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:45.287+0000 7fcdea725640 1 -- 172.21.15.169:0/3574368494 shutdown_connections 2023-10-26T01:36:45.290 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:45.287+0000 7fcdea725640 1 --2- 172.21.15.169:0/3574368494 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcdc406ae70 0x7fcdc406d330 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:45.290 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:45.287+0000 7fcdea725640 1 --2- 172.21.15.169:0/3574368494 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcde4152310 0x7fcde4152790 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:45.291 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:45.287+0000 7fcdea725640 1 --2- 172.21.15.169:0/3574368494 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcde406fd20 0x7fcde40701d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:45.291 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:45.287+0000 7fcdea725640 1 --2- 172.21.15.169:0/3574368494 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcdd0003970 0x7fcde41418c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:45.291 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:45.287+0000 7fcdea725640 1 -- 172.21.15.169:0/3574368494 >> 172.21.15.169:0/3574368494 conn(0x7fcde4069930 msgr2=0x7fcde40cc7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:45.291 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:45.287+0000 7fcdea725640 1 -- 172.21.15.169:0/3574368494 shutdown_connections 2023-10-26T01:36:45.292 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:45.287+0000 7fcdea725640 1 -- 172.21.15.169:0/3574368494 wait complete. 2023-10-26T01:36:45.309 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-26T01:36:45.540 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:45.538+0000 7fd21d36c640 1 -- 172.21.15.169:0/10042744 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd218143080 msgr2=0x7fd2180cbe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:45.540 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:45.538+0000 7fd21d36c640 1 --2- 172.21.15.169:0/10042744 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd218143080 0x7fd2180cbe10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd208005fd0 tx=0x7fd20802bd90 comp rx=0 tx=0).stop 2023-10-26T01:36:45.540 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:45.538+0000 7fd21d36c640 1 -- 172.21.15.169:0/10042744 shutdown_connections 2023-10-26T01:36:45.540 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:45.538+0000 7fd21d36c640 1 --2- 172.21.15.169:0/10042744 >> v2:172.21.15.170:3300/0 conn(0x7fd218069c20 0x7fd218140670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:45.540 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:45.538+0000 7fd21d36c640 1 --2- 172.21.15.169:0/10042744 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd218143080 0x7fd2180cbe10 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:45.541 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:45.538+0000 7fd21d36c640 1 -- 172.21.15.169:0/10042744 >> 172.21.15.169:0/10042744 conn(0x7fd21806a620 msgr2=0x7fd21806aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:45.541 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:45.538+0000 7fd21d36c640 1 -- 172.21.15.169:0/10042744 shutdown_connections 2023-10-26T01:36:45.541 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:45.538+0000 7fd21d36c640 1 -- 172.21.15.169:0/10042744 wait complete. 2023-10-26T01:36:45.542 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:45.540+0000 7fd21d36c640 1 Processor -- start 2023-10-26T01:36:45.542 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:45.540+0000 7fd21d36c640 1 -- start start 2023-10-26T01:36:45.543 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:45.541+0000 7fd21d36c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd218069c20 0x7fd21815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:45.543 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:45.541+0000 7fd21d36c640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd21806c260 0x7fd2180af600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:45.544 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:45.541+0000 7fd21d36c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd218143080 0x7fd21815b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:45.544 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:45.541+0000 7fd21d36c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd218159ee0 con 0x7fd218069c20 2023-10-26T01:36:45.544 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:45.541+0000 7fd21d36c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2181f2d00 con 0x7fd218143080 2023-10-26T01:36:45.544 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:45.541+0000 7fd21d36c640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd21815a0f0 con 0x7fd21806c260 2023-10-26T01:36:45.544 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:45.541+0000 7fd216ffd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd218069c20 0x7fd21815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:45.545 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:45.541+0000 7fd216ffd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd218069c20 0x7fd21815abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:59410/0 (socket says 172.21.15.169:59410) 2023-10-26T01:36:45.545 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:45.541+0000 7fd216ffd640 1 -- 172.21.15.169:0/129357803 learned_addr learned my addr 172.21.15.169:0/129357803 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:36:45.545 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:45.541+0000 7fd216ffd640 1 -- 172.21.15.169:0/129357803 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd21806c260 msgr2=0x7fd2180af600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:45.545 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:45.541+0000 7fd216ffd640 1 --2- 172.21.15.169:0/129357803 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd21806c260 0x7fd2180af600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:45.546 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:45.541+0000 7fd216ffd640 1 -- 172.21.15.169:0/129357803 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd218143080 msgr2=0x7fd21815b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:45.546 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:45.541+0000 7fd216ffd640 1 --2- 172.21.15.169:0/129357803 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd218143080 0x7fd21815b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:45.546 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:45.541+0000 7fd216ffd640 1 -- 172.21.15.169:0/129357803 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd208009560 con 0x7fd218069c20 2023-10-26T01:36:45.546 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:45.542+0000 7fd216ffd640 1 --2- 172.21.15.169:0/129357803 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd218069c20 0x7fd21815abd0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fd208036040 tx=0x7fd20802da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:45.547 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:45.542+0000 7fd20ffff640 1 -- 172.21.15.169:0/129357803 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd20802de60 con 0x7fd218069c20 2023-10-26T01:36:45.547 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:45.542+0000 7fd20ffff640 1 -- 172.21.15.169:0/129357803 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd2080036e0 con 0x7fd218069c20 2023-10-26T01:36:45.548 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:45.542+0000 7fd20ffff640 1 -- 172.21.15.169:0/129357803 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd208003ab0 con 0x7fd218069c20 2023-10-26T01:36:45.548 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:45.542+0000 7fd21d36c640 1 -- 172.21.15.169:0/129357803 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd21815a3d0 con 0x7fd218069c20 2023-10-26T01:36:45.548 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:45.542+0000 7fd21d36c640 1 -- 172.21.15.169:0/129357803 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2180afb40 con 0x7fd218069c20 2023-10-26T01:36:45.548 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:45.543+0000 7fd21d36c640 1 -- 172.21.15.169:0/129357803 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd218066af0 con 0x7fd218069c20 2023-10-26T01:36:45.549 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:45.544+0000 7fd20ffff640 1 -- 172.21.15.169:0/129357803 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fd208003c10 con 0x7fd218069c20 2023-10-26T01:36:45.549 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:45.546+0000 7fd20ffff640 1 --2- 172.21.15.169:0/129357803 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd1f806ae40 0x7fd1f806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:45.549 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:45.546+0000 7fd20ffff640 1 -- 172.21.15.169:0/129357803 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd2080ac100 con 0x7fd218069c20 2023-10-26T01:36:45.550 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:45.546+0000 7fd20e7fc640 1 --2- 172.21.15.169:0/129357803 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd1f806ae40 0x7fd1f806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:45.550 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:45.547+0000 7fd20e7fc640 1 --2- 172.21.15.169:0/129357803 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd1f806ae40 0x7fd1f806d300 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fd200000f60 tx=0x7fd2000079b0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:45.552 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:45.550+0000 7fd20ffff640 1 -- 172.21.15.169:0/129357803 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd208077360 con 0x7fd218069c20 2023-10-26T01:36:45.779 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:45.778+0000 7fd21d36c640 1 -- 172.21.15.169:0/129357803 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fd21815a8f0 con 0x7fd218069c20 2023-10-26T01:36:46.433 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:46.431+0000 7fd20ffff640 1 -- 172.21.15.169:0/129357803 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v49) v1 ==== 118+0+0 (secure 0 0 0) 0x7fd20807a980 con 0x7fd218069c20 2023-10-26T01:36:46.436 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:46.434+0000 7fd21d36c640 1 -- 172.21.15.169:0/129357803 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd1f806ae40 msgr2=0x7fd1f806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:46.436 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:46.434+0000 7fd21d36c640 1 --2- 172.21.15.169:0/129357803 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd1f806ae40 0x7fd1f806d300 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fd200000f60 tx=0x7fd2000079b0 comp rx=0 tx=0).stop 2023-10-26T01:36:46.437 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:46.434+0000 7fd21d36c640 1 -- 172.21.15.169:0/129357803 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd218069c20 msgr2=0x7fd21815abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:46.437 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:46.434+0000 7fd21d36c640 1 --2- 172.21.15.169:0/129357803 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd218069c20 0x7fd21815abd0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fd208036040 tx=0x7fd20802da60 comp rx=0 tx=0).stop 2023-10-26T01:36:46.437 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:46.434+0000 7fd21d36c640 1 -- 172.21.15.169:0/129357803 shutdown_connections 2023-10-26T01:36:46.437 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:46.434+0000 7fd21d36c640 1 --2- 172.21.15.169:0/129357803 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd1f806ae40 0x7fd1f806d300 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:46.437 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:46.434+0000 7fd21d36c640 1 --2- 172.21.15.169:0/129357803 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd218143080 0x7fd21815b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:46.438 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:46.434+0000 7fd21d36c640 1 --2- 172.21.15.169:0/129357803 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd21806c260 0x7fd2180af600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:46.438 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:46.434+0000 7fd21d36c640 1 --2- 172.21.15.169:0/129357803 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd218069c20 0x7fd21815abd0 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:46.438 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:46.434+0000 7fd21d36c640 1 -- 172.21.15.169:0/129357803 >> 172.21.15.169:0/129357803 conn(0x7fd21806a620 msgr2=0x7fd21813f990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:46.438 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:46.434+0000 7fd21d36c640 1 -- 172.21.15.169:0/129357803 shutdown_connections 2023-10-26T01:36:46.438 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:46.434+0000 7fd21d36c640 1 -- 172.21.15.169:0/129357803 wait complete. 2023-10-26T01:36:46.457 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-26T01:36:46.457 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-26T01:36:46.686 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:46.683+0000 7f691c9bf640 1 -- 172.21.15.169:0/3958770209 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6904003fa0 msgr2=0x7f69040047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:46.686 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:46.683+0000 7f691c9bf640 1 --2- 172.21.15.169:0/3958770209 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6904003fa0 0x7f69040047c0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f690000c260 tx=0x7f69040057f0 comp rx=0 tx=0).stop 2023-10-26T01:36:46.687 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:46.683+0000 7f6917fff640 1 -- 172.21.15.169:0/3958770209 reap_dead start 2023-10-26T01:36:46.687 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:46.683+0000 7f691c9bf640 1 -- 172.21.15.169:0/3958770209 shutdown_connections 2023-10-26T01:36:46.687 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:46.683+0000 7f691c9bf640 1 -- 172.21.15.169:0/3958770209 >> 172.21.15.169:0/3958770209 conn(0x7f69180d1370 msgr2=0x7f69180d1780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:46.688 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:46.683+0000 7f691c9bf640 1 -- 172.21.15.169:0/3958770209 shutdown_connections 2023-10-26T01:36:46.688 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:46.683+0000 7f691c9bf640 1 -- 172.21.15.169:0/3958770209 wait complete. 2023-10-26T01:36:46.692 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:46.685+0000 7f691c9bf640 1 Processor -- start 2023-10-26T01:36:46.692 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:46.685+0000 7f691c9bf640 1 -- start start 2023-10-26T01:36:46.692 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:46.686+0000 7f691c9bf640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6904003fa0 0x7f6918156b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:46.692 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:46.686+0000 7f691c9bf640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f691815a9d0 0x7f691814b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:46.693 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:46.686+0000 7f691c9bf640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f691815ae50 0x7f69181f45a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:46.693 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:46.686+0000 7f691c9bf640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f691814c090 con 0x7f6904003fa0 2023-10-26T01:36:46.693 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:46.686+0000 7f691c9bf640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f691814c370 con 0x7f691815a9d0 2023-10-26T01:36:46.693 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:46.686+0000 7f691c9bf640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6918156310 con 0x7f691815ae50 2023-10-26T01:36:46.694 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:46.687+0000 7f69177fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6904003fa0 0x7f6918156b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:46.694 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:46.687+0000 7f69177fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6904003fa0 0x7f6918156b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:59420/0 (socket says 172.21.15.169:59420) 2023-10-26T01:36:46.694 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:46.687+0000 7f69177fe640 1 -- 172.21.15.169:0/2577461532 learned_addr learned my addr 172.21.15.169:0/2577461532 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:36:46.694 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:46.687+0000 7f6917fff640 1 --2- 172.21.15.169:0/2577461532 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f691815ae50 0x7f69181f45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:46.694 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:46.687+0000 7f6916ffd640 1 --2- 172.21.15.169:0/2577461532 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f691815a9d0 0x7f691814b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:46.695 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:46.687+0000 7f69177fe640 1 -- 172.21.15.169:0/2577461532 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f691815ae50 msgr2=0x7f69181f45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:46.695 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:46.687+0000 7f69177fe640 1 --2- 172.21.15.169:0/2577461532 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f691815ae50 0x7f69181f45a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:46.695 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:46.687+0000 7f69177fe640 1 -- 172.21.15.169:0/2577461532 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f691815a9d0 msgr2=0x7f691814b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:46.695 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:46.687+0000 7f69177fe640 1 --2- 172.21.15.169:0/2577461532 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f691815a9d0 0x7f691814b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:46.695 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:46.687+0000 7f69177fe640 1 -- 172.21.15.169:0/2577461532 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69181565f0 con 0x7f6904003fa0 2023-10-26T01:36:46.695 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:46.687+0000 7f6916ffd640 1 --2- 172.21.15.169:0/2577461532 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f691815a9d0 0x7f691814b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:36:46.696 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:46.687+0000 7f6917fff640 1 --2- 172.21.15.169:0/2577461532 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f691815ae50 0x7f69181f45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:36:46.696 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:46.687+0000 7f69177fe640 1 --2- 172.21.15.169:0/2577461532 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6904003fa0 0x7f6918156b10 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f6908009a30 tx=0x7f6908009ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:46.696 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:46.688+0000 7f690ffff640 1 -- 172.21.15.169:0/2577461532 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6908004d20 con 0x7f6904003fa0 2023-10-26T01:36:46.696 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:46.688+0000 7f690ffff640 1 -- 172.21.15.169:0/2577461532 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f690800fb50 con 0x7f6904003fa0 2023-10-26T01:36:46.696 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:46.688+0000 7f690ffff640 1 -- 172.21.15.169:0/2577461532 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6908005020 con 0x7f6904003fa0 2023-10-26T01:36:46.697 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:46.688+0000 7f691c9bf640 1 -- 172.21.15.169:0/2577461532 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6918157050 con 0x7f6904003fa0 2023-10-26T01:36:46.697 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:46.688+0000 7f691c9bf640 1 -- 172.21.15.169:0/2577461532 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f691814bb10 con 0x7f6904003fa0 2023-10-26T01:36:46.697 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:46.689+0000 7f691c9bf640 1 -- 172.21.15.169:0/2577461532 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6918073c10 con 0x7f6904003fa0 2023-10-26T01:36:46.697 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:46.690+0000 7f690ffff640 1 -- 172.21.15.169:0/2577461532 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f6908005180 con 0x7f6904003fa0 2023-10-26T01:36:46.698 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:46.691+0000 7f690ffff640 1 --2- 172.21.15.169:0/2577461532 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f68f806ae40 0x7f68f806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:46.698 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:46.691+0000 7f690ffff640 1 -- 172.21.15.169:0/2577461532 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6908015070 con 0x7f6904003fa0 2023-10-26T01:36:46.699 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:46.692+0000 7f6916ffd640 1 --2- 172.21.15.169:0/2577461532 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f68f806ae40 0x7f68f806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:46.699 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:46.693+0000 7f6916ffd640 1 --2- 172.21.15.169:0/2577461532 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f68f806ae40 0x7f68f806d300 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f6900034000 tx=0x7f6900001a00 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:46.699 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:46.696+0000 7f690ffff640 1 -- 172.21.15.169:0/2577461532 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f69080555b0 con 0x7f6904003fa0 2023-10-26T01:36:46.926 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:46.924+0000 7f691c9bf640 1 -- 172.21.15.169:0/2577461532 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f691806ae50 con 0x7f6904003fa0 2023-10-26T01:36:47.436 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:47.434+0000 7f690ffff640 1 -- 172.21.15.169:0/2577461532 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v50) v1 ==== 122+0+0 (secure 0 0 0) 0x7f690801e0c0 con 0x7f6904003fa0 2023-10-26T01:36:47.440 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:47.438+0000 7f691c9bf640 1 -- 172.21.15.169:0/2577461532 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f68f806ae40 msgr2=0x7f68f806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:47.441 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:47.438+0000 7f691c9bf640 1 --2- 172.21.15.169:0/2577461532 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f68f806ae40 0x7f68f806d300 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f6900034000 tx=0x7f6900001a00 comp rx=0 tx=0).stop 2023-10-26T01:36:47.441 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:47.438+0000 7f691c9bf640 1 -- 172.21.15.169:0/2577461532 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6904003fa0 msgr2=0x7f6918156b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:47.441 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:47.438+0000 7f691c9bf640 1 --2- 172.21.15.169:0/2577461532 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6904003fa0 0x7f6918156b10 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f6908009a30 tx=0x7f6908009ef0 comp rx=0 tx=0).stop 2023-10-26T01:36:47.441 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:47.438+0000 7f691c9bf640 1 -- 172.21.15.169:0/2577461532 shutdown_connections 2023-10-26T01:36:47.442 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:47.438+0000 7f691c9bf640 1 --2- 172.21.15.169:0/2577461532 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f68f806ae40 0x7f68f806d300 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:47.442 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:47.438+0000 7f691c9bf640 1 --2- 172.21.15.169:0/2577461532 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f691815ae50 0x7f69181f45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:47.442 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:47.438+0000 7f691c9bf640 1 --2- 172.21.15.169:0/2577461532 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f691815a9d0 0x7f691814b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:47.442 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:47.438+0000 7f691c9bf640 1 --2- 172.21.15.169:0/2577461532 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6904003fa0 0x7f6918156b10 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:47.443 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:47.438+0000 7f691c9bf640 1 -- 172.21.15.169:0/2577461532 >> 172.21.15.169:0/2577461532 conn(0x7f69180d1370 msgr2=0x7f69181559e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:47.443 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:47.438+0000 7f691c9bf640 1 -- 172.21.15.169:0/2577461532 shutdown_connections 2023-10-26T01:36:47.443 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:47.438+0000 7f691c9bf640 1 -- 172.21.15.169:0/2577461532 wait complete. 2023-10-26T01:36:47.460 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-26T01:36:47.460 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T01:36:47.686 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:47.684+0000 7f0bde3ee640 1 -- 172.21.15.169:0/2127929504 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0bc40039a0 msgr2=0x7f0bc4003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:47.686 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:47.684+0000 7f0bde3ee640 1 --2- 172.21.15.169:0/2127929504 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0bc40039a0 0x7f0bc4003dc0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f0bc8006fc0 tx=0x7f0bc40057f0 comp rx=0 tx=0).stop 2023-10-26T01:36:47.686 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:47.684+0000 7f0bdc98b640 1 -- 172.21.15.169:0/2127929504 reap_dead start 2023-10-26T01:36:47.687 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:47.684+0000 7f0bde3ee640 1 -- 172.21.15.169:0/2127929504 shutdown_connections 2023-10-26T01:36:47.687 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:47.684+0000 7f0bde3ee640 1 -- 172.21.15.169:0/2127929504 >> 172.21.15.169:0/2127929504 conn(0x7f0bd8069930 msgr2=0x7f0bd8069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:47.687 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:47.684+0000 7f0bde3ee640 1 -- 172.21.15.169:0/2127929504 shutdown_connections 2023-10-26T01:36:47.687 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:47.684+0000 7f0bde3ee640 1 -- 172.21.15.169:0/2127929504 wait complete. 2023-10-26T01:36:47.689 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:47.686+0000 7f0bde3ee640 1 Processor -- start 2023-10-26T01:36:47.689 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:47.686+0000 7f0bde3ee640 1 -- start start 2023-10-26T01:36:47.690 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:47.687+0000 7f0bde3ee640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0bc40039a0 0x7f0bd81f49e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:47.690 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:47.687+0000 7f0bde3ee640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0bd81f5b60 0x7f0bd8159a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:47.690 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:47.687+0000 7f0bde3ee640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0bd8070a30 0x7f0bd81f4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:47.690 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:47.687+0000 7f0bde3ee640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd806ffe0 con 0x7f0bd8070a30 2023-10-26T01:36:47.691 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:47.687+0000 7f0bde3ee640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd815c470 con 0x7f0bc40039a0 2023-10-26T01:36:47.691 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:47.687+0000 7f0bde3ee640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd8159f80 con 0x7f0bd81f5b60 2023-10-26T01:36:47.691 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:47.687+0000 7f0bdc98b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0bd8070a30 0x7f0bd81f4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:47.691 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:47.687+0000 7f0bdc98b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0bd8070a30 0x7f0bd81f4fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:59444/0 (socket says 172.21.15.169:59444) 2023-10-26T01:36:47.691 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:47.687+0000 7f0bdc98b640 1 -- 172.21.15.169:0/405874396 learned_addr learned my addr 172.21.15.169:0/405874396 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:36:47.692 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:47.687+0000 7f0bcffff640 1 --2- 172.21.15.169:0/405874396 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0bd81f5b60 0x7f0bd8159a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:47.692 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:47.687+0000 7f0bd7fff640 1 --2- 172.21.15.169:0/405874396 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0bc40039a0 0x7f0bd81f49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:47.692 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:47.687+0000 7f0bdc98b640 1 -- 172.21.15.169:0/405874396 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0bd81f5b60 msgr2=0x7f0bd8159a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:47.693 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:47.687+0000 7f0bdc98b640 1 --2- 172.21.15.169:0/405874396 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0bd81f5b60 0x7f0bd8159a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:47.693 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:47.687+0000 7f0bdc98b640 1 -- 172.21.15.169:0/405874396 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0bc40039a0 msgr2=0x7f0bd81f49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:47.693 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:47.687+0000 7f0bdc98b640 1 --2- 172.21.15.169:0/405874396 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0bc40039a0 0x7f0bd81f49e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:47.693 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:47.687+0000 7f0bdc98b640 1 -- 172.21.15.169:0/405874396 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bd815c710 con 0x7f0bd8070a30 2023-10-26T01:36:47.694 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:47.687+0000 7f0bcffff640 1 --2- 172.21.15.169:0/405874396 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0bd81f5b60 0x7f0bd8159a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:36:47.694 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:47.687+0000 7f0bd7fff640 1 --2- 172.21.15.169:0/405874396 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0bc40039a0 0x7f0bd81f49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:36:47.694 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:47.688+0000 7f0bdc98b640 1 --2- 172.21.15.169:0/405874396 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0bd8070a30 0x7f0bd81f4fc0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f0bd80ab9b0 tx=0x7f0bd000eda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:47.695 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:47.688+0000 7f0bd57fa640 1 -- 172.21.15.169:0/405874396 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bd000cc30 con 0x7f0bd8070a30 2023-10-26T01:36:47.696 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:47.688+0000 7f0bd57fa640 1 -- 172.21.15.169:0/405874396 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0bd000ce80 con 0x7f0bd8070a30 2023-10-26T01:36:47.696 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:47.688+0000 7f0bd57fa640 1 -- 172.21.15.169:0/405874396 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bd00116d0 con 0x7f0bd8070a30 2023-10-26T01:36:47.696 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:47.688+0000 7f0bde3ee640 1 -- 172.21.15.169:0/405874396 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bd8070230 con 0x7f0bd8070a30 2023-10-26T01:36:47.696 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:47.688+0000 7f0bde3ee640 1 -- 172.21.15.169:0/405874396 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bd8070eb0 con 0x7f0bd8070a30 2023-10-26T01:36:47.696 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:47.689+0000 7f0bde3ee640 1 -- 172.21.15.169:0/405874396 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bd804db10 con 0x7f0bd8070a30 2023-10-26T01:36:47.697 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:47.690+0000 7f0bd57fa640 1 -- 172.21.15.169:0/405874396 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f0bd0011830 con 0x7f0bd8070a30 2023-10-26T01:36:47.697 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:47.691+0000 7f0bd57fa640 1 --2- 172.21.15.169:0/405874396 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f0bb806ae40 0x7f0bb806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:47.697 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:47.692+0000 7f0bd57fa640 1 -- 172.21.15.169:0/405874396 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0bd0089ea0 con 0x7f0bd8070a30 2023-10-26T01:36:47.697 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:47.692+0000 7f0bd7fff640 1 --2- 172.21.15.169:0/405874396 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f0bb806ae40 0x7f0bb806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:47.697 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:47.693+0000 7f0bd7fff640 1 --2- 172.21.15.169:0/405874396 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f0bb806ae40 0x7f0bb806d300 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f0bc8034000 tx=0x7f0bc8036040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:47.698 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:47.696+0000 7f0bd57fa640 1 -- 172.21.15.169:0/405874396 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0bd0054f40 con 0x7f0bd8070a30 2023-10-26T01:36:47.927 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:47.925+0000 7f0bde3ee640 1 -- 172.21.15.169:0/405874396 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0bd80710d0 con 0x7f0bd8070a30 2023-10-26T01:36:47.928 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:47.926+0000 7f0bd57fa640 1 -- 172.21.15.169:0/405874396 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+4924 (secure 0 0 0) 0x7f0bd0016090 con 0x7f0bd8070a30 2023-10-26T01:36:47.928 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:36:47.929 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":50,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5240,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/3870332417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":3870332417},{"type":"v1","addr":"172.21.15.170:6837","nonce":3870332417}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45},{"gid":5497,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/2533771698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2533771698},{"type":"v1","addr":"172.21.15.170:6835","nonce":2533771698}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45},{"gid":6885,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/2117946713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2117946713},{"type":"v1","addr":"172.21.15.169:6834","nonce":2117946713}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T01:36:43.849046+0000","modified":"2023-10-26T01:36:46.926339+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6888},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6888":{"gid":6888,"name":"c","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6835/657142078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":657142078},{"type":"v1","addr":"172.21.15.169:6835","nonce":657142078}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-10-26T01:36:47.929 INFO:teuthology.orchestra.run.smithi169.stderr:dumped fsmap epoch 50 2023-10-26T01:36:47.931 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:47.929+0000 7f0bde3ee640 1 -- 172.21.15.169:0/405874396 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f0bb806ae40 msgr2=0x7f0bb806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:47.931 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:47.929+0000 7f0bde3ee640 1 --2- 172.21.15.169:0/405874396 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f0bb806ae40 0x7f0bb806d300 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f0bc8034000 tx=0x7f0bc8036040 comp rx=0 tx=0).stop 2023-10-26T01:36:47.931 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:47.929+0000 7f0bde3ee640 1 -- 172.21.15.169:0/405874396 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0bd8070a30 msgr2=0x7f0bd81f4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:47.931 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:47.929+0000 7f0bde3ee640 1 --2- 172.21.15.169:0/405874396 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0bd8070a30 0x7f0bd81f4fc0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f0bd80ab9b0 tx=0x7f0bd000eda0 comp rx=0 tx=0).stop 2023-10-26T01:36:47.932 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:47.929+0000 7f0bde3ee640 1 -- 172.21.15.169:0/405874396 shutdown_connections 2023-10-26T01:36:47.932 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:47.929+0000 7f0bde3ee640 1 --2- 172.21.15.169:0/405874396 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f0bb806ae40 0x7f0bb806d300 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:47.932 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:47.929+0000 7f0bde3ee640 1 --2- 172.21.15.169:0/405874396 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0bd8070a30 0x7f0bd81f4fc0 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:47.932 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:47.929+0000 7f0bde3ee640 1 --2- 172.21.15.169:0/405874396 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0bd81f5b60 0x7f0bd8159a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:47.932 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:47.929+0000 7f0bde3ee640 1 --2- 172.21.15.169:0/405874396 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0bc40039a0 0x7f0bd81f49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:47.932 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:47.929+0000 7f0bde3ee640 1 -- 172.21.15.169:0/405874396 >> 172.21.15.169:0/405874396 conn(0x7f0bd8069930 msgr2=0x7f0bd8152860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:47.933 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:47.929+0000 7f0bde3ee640 1 -- 172.21.15.169:0/405874396 shutdown_connections 2023-10-26T01:36:47.933 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:47.930+0000 7f0bde3ee640 1 -- 172.21.15.169:0/405874396 wait complete. 2023-10-26T01:36:47.951 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:36:48.165 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:48.163+0000 7fc8368e5640 1 -- 172.21.15.169:0/1797368370 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc83006ed30 msgr2=0x7fc8300cd670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:48.165 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:48.163+0000 7fc8368e5640 1 --2- 172.21.15.169:0/1797368370 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc83006ed30 0x7fc8300cd670 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fc8300cd170 tx=0x7fc81802bd70 comp rx=0 tx=0).stop 2023-10-26T01:36:48.165 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:48.163+0000 7fc8368e5640 1 -- 172.21.15.169:0/1797368370 shutdown_connections 2023-10-26T01:36:48.166 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:48.163+0000 7fc8368e5640 1 --2- 172.21.15.169:0/1797368370 >> v2:172.21.15.170:3300/0 conn(0x7fc8300cdbb0 0x7fc8300b07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:48.166 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:48.163+0000 7fc8368e5640 1 --2- 172.21.15.169:0/1797368370 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc83006ed30 0x7fc8300cd670 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:48.166 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:48.163+0000 7fc8368e5640 1 -- 172.21.15.169:0/1797368370 >> 172.21.15.169:0/1797368370 conn(0x7fc830069930 msgr2=0x7fc830069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:48.166 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:48.163+0000 7fc8368e5640 1 -- 172.21.15.169:0/1797368370 shutdown_connections 2023-10-26T01:36:48.166 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:48.163+0000 7fc8368e5640 1 -- 172.21.15.169:0/1797368370 wait complete. 2023-10-26T01:36:48.167 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:48.165+0000 7fc8368e5640 1 Processor -- start 2023-10-26T01:36:48.168 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:48.165+0000 7fc8368e5640 1 -- start start 2023-10-26T01:36:48.168 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:48.166+0000 7fc8368e5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc83006ed30 0x7fc8300adb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:48.168 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:48.166+0000 7fc8368e5640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc8300cdbb0 0x7fc8301526f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:48.168 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:48.166+0000 7fc8368e5640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc83015b610 0x7fc830152c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:48.168 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:48.166+0000 7fc8368e5640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc83015d580 con 0x7fc83015b610 2023-10-26T01:36:48.169 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:48.166+0000 7fc8368e5640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc83015c900 con 0x7fc83006ed30 2023-10-26T01:36:48.169 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:48.166+0000 7fc8368e5640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc83015cbe0 con 0x7fc8300cdbb0 2023-10-26T01:36:48.169 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:48.166+0000 7fc834e82640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc83015b610 0x7fc830152c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:48.169 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:48.166+0000 7fc834e82640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc83015b610 0x7fc830152c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:59464/0 (socket says 172.21.15.169:59464) 2023-10-26T01:36:48.169 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:48.166+0000 7fc834e82640 1 -- 172.21.15.169:0/2110830107 learned_addr learned my addr 172.21.15.169:0/2110830107 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:36:48.171 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:48.166+0000 7fc827fff640 1 --2- 172.21.15.169:0/2110830107 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc8300cdbb0 0x7fc8301526f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:48.171 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:48.166+0000 7fc82ffff640 1 --2- 172.21.15.169:0/2110830107 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc83006ed30 0x7fc8300adb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:48.171 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:48.166+0000 7fc834e82640 1 -- 172.21.15.169:0/2110830107 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc8300cdbb0 msgr2=0x7fc8301526f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:48.171 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:48.167+0000 7fc834e82640 1 --2- 172.21.15.169:0/2110830107 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc8300cdbb0 0x7fc8301526f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:48.172 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:48.167+0000 7fc834e82640 1 -- 172.21.15.169:0/2110830107 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc83006ed30 msgr2=0x7fc8300adb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:48.172 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:48.167+0000 7fc834e82640 1 --2- 172.21.15.169:0/2110830107 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc83006ed30 0x7fc8300adb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:48.172 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:48.167+0000 7fc834e82640 1 -- 172.21.15.169:0/2110830107 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc818009490 con 0x7fc83015b610 2023-10-26T01:36:48.172 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:48.167+0000 7fc82ffff640 1 --2- 172.21.15.169:0/2110830107 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc83006ed30 0x7fc8300adb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:36:48.172 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:48.167+0000 7fc834e82640 1 --2- 172.21.15.169:0/2110830107 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc83015b610 0x7fc830152c30 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fc82800eaa0 tx=0x7fc82800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:48.173 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:48.167+0000 7fc82d7fa640 1 -- 172.21.15.169:0/2110830107 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc82800cdb0 con 0x7fc83015b610 2023-10-26T01:36:48.173 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:48.167+0000 7fc82d7fa640 1 -- 172.21.15.169:0/2110830107 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc828004b20 con 0x7fc83015b610 2023-10-26T01:36:48.173 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:48.167+0000 7fc82d7fa640 1 -- 172.21.15.169:0/2110830107 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc828005020 con 0x7fc83015b610 2023-10-26T01:36:48.173 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:48.167+0000 7fc8368e5640 1 -- 172.21.15.169:0/2110830107 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc83015cec0 con 0x7fc83015b610 2023-10-26T01:36:48.173 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:48.167+0000 7fc8368e5640 1 -- 172.21.15.169:0/2110830107 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8300ae050 con 0x7fc83015b610 2023-10-26T01:36:48.174 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:48.168+0000 7fc8368e5640 1 -- 172.21.15.169:0/2110830107 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc83004db10 con 0x7fc83015b610 2023-10-26T01:36:48.174 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:48.169+0000 7fc82d7fa640 1 -- 172.21.15.169:0/2110830107 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fc828005180 con 0x7fc83015b610 2023-10-26T01:36:48.175 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:48.171+0000 7fc82d7fa640 1 --2- 172.21.15.169:0/2110830107 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc81006ae40 0x7fc81006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:48.175 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:48.171+0000 7fc82d7fa640 1 -- 172.21.15.169:0/2110830107 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc828089fe0 con 0x7fc83015b610 2023-10-26T01:36:48.175 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:48.171+0000 7fc82ffff640 1 --2- 172.21.15.169:0/2110830107 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc81006ae40 0x7fc81006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:48.175 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:48.172+0000 7fc82ffff640 1 --2- 172.21.15.169:0/2110830107 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc81006ae40 0x7fc81006d300 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fc820000f60 tx=0x7fc820009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:48.177 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:48.176+0000 7fc82d7fa640 1 -- 172.21.15.169:0/2110830107 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc828055080 con 0x7fc83015b610 2023-10-26T01:36:48.359 INFO:tasks.ceph.mds.c.smithi169.stderr:2023-10-26T01:36:48.358+0000 7f697889c640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-26T01:36:48.366 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:48.365+0000 7fc8368e5640 1 -- 172.21.15.169:0/2110830107 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc83006c990 con 0x7fc83015b610 2023-10-26T01:36:48.369 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:48.367+0000 7fc82d7fa640 1 -- 172.21.15.169:0/2110830107 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) v1 ==== 74+0+16619 (secure 0 0 0) 0x7fc82801f020 con 0x7fc83015b610 2023-10-26T01:36:48.370 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:36:48.370 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":103,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:36:45.388928+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-10-26T01:36:41.430455+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-10-26T01:36:42.883252+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:36:48.372 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:48.370+0000 7fc8368e5640 1 -- 172.21.15.169:0/2110830107 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc81006ae40 msgr2=0x7fc81006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:48.372 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:48.370+0000 7fc8368e5640 1 --2- 172.21.15.169:0/2110830107 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc81006ae40 0x7fc81006d300 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fc820000f60 tx=0x7fc820009040 comp rx=0 tx=0).stop 2023-10-26T01:36:48.372 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:48.370+0000 7fc8368e5640 1 -- 172.21.15.169:0/2110830107 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc83015b610 msgr2=0x7fc830152c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:48.373 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:48.370+0000 7fc8368e5640 1 --2- 172.21.15.169:0/2110830107 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc83015b610 0x7fc830152c30 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fc82800eaa0 tx=0x7fc82800ef60 comp rx=0 tx=0).stop 2023-10-26T01:36:48.373 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:48.370+0000 7fc8368e5640 1 -- 172.21.15.169:0/2110830107 shutdown_connections 2023-10-26T01:36:48.373 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:48.370+0000 7fc8368e5640 1 --2- 172.21.15.169:0/2110830107 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc81006ae40 0x7fc81006d300 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:48.373 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:48.370+0000 7fc8368e5640 1 --2- 172.21.15.169:0/2110830107 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc83015b610 0x7fc830152c30 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:48.373 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:48.370+0000 7fc8368e5640 1 --2- 172.21.15.169:0/2110830107 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc8300cdbb0 0x7fc8301526f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:48.373 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:48.370+0000 7fc8368e5640 1 --2- 172.21.15.169:0/2110830107 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc83006ed30 0x7fc8300adb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:48.374 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:48.370+0000 7fc8368e5640 1 -- 172.21.15.169:0/2110830107 >> 172.21.15.169:0/2110830107 conn(0x7fc830069930 msgr2=0x7fc8300aeec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:48.374 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:48.370+0000 7fc8368e5640 1 -- 172.21.15.169:0/2110830107 shutdown_connections 2023-10-26T01:36:48.374 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:48.370+0000 7fc8368e5640 1 -- 172.21.15.169:0/2110830107 wait complete. 2023-10-26T01:36:48.391 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-26T01:36:48.391 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-26T01:36:48.611 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:48.609+0000 7f3aaf2b4640 1 -- 172.21.15.169:0/3538361142 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3aa814f2c0 msgr2=0x7f3aa8159240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:48.611 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:48.609+0000 7f3aaf2b4640 1 --2- 172.21.15.169:0/3538361142 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3aa814f2c0 0x7f3aa8159240 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f3aa4009860 tx=0x7f3aa402bd50 comp rx=0 tx=0).stop 2023-10-26T01:36:48.611 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:48.609+0000 7f3aaf2b4640 1 -- 172.21.15.169:0/3538361142 shutdown_connections 2023-10-26T01:36:48.611 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:48.609+0000 7f3aaf2b4640 1 --2- 172.21.15.169:0/3538361142 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3aa814f2c0 0x7f3aa8159240 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:48.612 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:48.609+0000 7f3aaf2b4640 1 -- 172.21.15.169:0/3538361142 >> 172.21.15.169:0/3538361142 conn(0x7f3aa8069930 msgr2=0x7f3aa8069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:48.612 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:48.609+0000 7f3aaf2b4640 1 -- 172.21.15.169:0/3538361142 shutdown_connections 2023-10-26T01:36:48.612 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:48.609+0000 7f3aaf2b4640 1 -- 172.21.15.169:0/3538361142 wait complete. 2023-10-26T01:36:48.613 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:48.611+0000 7f3aaf2b4640 1 Processor -- start 2023-10-26T01:36:48.613 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:48.611+0000 7f3aaf2b4640 1 -- start start 2023-10-26T01:36:48.615 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:48.612+0000 7f3aaf2b4640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3aa80cde30 0x7f3aa814a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:48.615 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:48.612+0000 7f3aaf2b4640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3aa814f2c0 0x7f3aa8151430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:48.615 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:48.612+0000 7f3aaf2b4640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3aa814f6a0 0x7f3aa8070d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:48.616 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:48.612+0000 7f3aaf2b4640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa81f5c50 con 0x7f3aa814f6a0 2023-10-26T01:36:48.616 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:48.612+0000 7f3aaf2b4640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa81f5f30 con 0x7f3aa814f2c0 2023-10-26T01:36:48.616 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:48.612+0000 7f3aaf2b4640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa81f6210 con 0x7f3aa80cde30 2023-10-26T01:36:48.616 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:48.612+0000 7f3aad851640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3aa814f6a0 0x7f3aa8070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:48.616 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:48.612+0000 7f3aad851640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3aa814f6a0 0x7f3aa8070d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:59484/0 (socket says 172.21.15.169:59484) 2023-10-26T01:36:48.616 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:48.612+0000 7f3aad851640 1 -- 172.21.15.169:0/3995473971 learned_addr learned my addr 172.21.15.169:0/3995473971 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:36:48.617 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:48.613+0000 7f3aac84f640 1 --2- 172.21.15.169:0/3995473971 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3aa814f2c0 0x7f3aa8151430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:48.617 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:48.613+0000 7f3aad050640 1 --2- 172.21.15.169:0/3995473971 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3aa80cde30 0x7f3aa814a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:48.617 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:48.613+0000 7f3aad851640 1 -- 172.21.15.169:0/3995473971 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3aa80cde30 msgr2=0x7f3aa814a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:48.617 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:48.613+0000 7f3aad851640 1 --2- 172.21.15.169:0/3995473971 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3aa80cde30 0x7f3aa814a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:48.617 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:48.613+0000 7f3aad851640 1 -- 172.21.15.169:0/3995473971 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3aa814f2c0 msgr2=0x7f3aa8151430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:48.617 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:48.613+0000 7f3aad851640 1 --2- 172.21.15.169:0/3995473971 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3aa814f2c0 0x7f3aa8151430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:48.618 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:48.613+0000 7f3aad851640 1 -- 172.21.15.169:0/3995473971 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aa4009490 con 0x7f3aa814f6a0 2023-10-26T01:36:48.618 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:48.613+0000 7f3aad050640 1 --2- 172.21.15.169:0/3995473971 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3aa80cde30 0x7f3aa814a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:36:48.618 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:48.613+0000 7f3aad851640 1 --2- 172.21.15.169:0/3995473971 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3aa814f6a0 0x7f3aa8070d40 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f3aa000cb10 tx=0x7f3aa000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:48.618 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:48.613+0000 7f3a9dffb640 1 -- 172.21.15.169:0/3995473971 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa000ac70 con 0x7f3aa814f6a0 2023-10-26T01:36:48.619 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:48.613+0000 7f3a9dffb640 1 -- 172.21.15.169:0/3995473971 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3aa0012420 con 0x7f3aa814f6a0 2023-10-26T01:36:48.619 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:48.613+0000 7f3a9dffb640 1 -- 172.21.15.169:0/3995473971 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa0012850 con 0x7f3aa814f6a0 2023-10-26T01:36:48.619 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:48.613+0000 7f3aaf2b4640 1 -- 172.21.15.169:0/3995473971 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3aa81f64f0 con 0x7f3aa814f6a0 2023-10-26T01:36:48.619 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:48.614+0000 7f3aaf2b4640 1 -- 172.21.15.169:0/3995473971 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3aa8151970 con 0x7f3aa814f6a0 2023-10-26T01:36:48.620 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:48.615+0000 7f3aaf2b4640 1 -- 172.21.15.169:0/3995473971 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3aa804db10 con 0x7f3aa814f6a0 2023-10-26T01:36:48.620 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:48.616+0000 7f3a9dffb640 1 -- 172.21.15.169:0/3995473971 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f3aa001b400 con 0x7f3aa814f6a0 2023-10-26T01:36:48.620 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:48.617+0000 7f3a9dffb640 1 --2- 172.21.15.169:0/3995473971 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3a8c06ae40 0x7f3a8c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:48.621 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:48.617+0000 7f3a9dffb640 1 -- 172.21.15.169:0/3995473971 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3aa008a0a0 con 0x7f3aa814f6a0 2023-10-26T01:36:48.621 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:48.617+0000 7f3aad050640 1 --2- 172.21.15.169:0/3995473971 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3a8c06ae40 0x7f3a8c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:48.621 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:48.618+0000 7f3aad050640 1 --2- 172.21.15.169:0/3995473971 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3a8c06ae40 0x7f3a8c06d300 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f3aa4036000 tx=0x7f3aa4004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:48.622 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:48.621+0000 7f3a9dffb640 1 -- 172.21.15.169:0/3995473971 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3aa0055150 con 0x7f3aa814f6a0 2023-10-26T01:36:48.812 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:48.810+0000 7f3aaf2b4640 1 -- 172.21.15.169:0/3995473971 --> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3aa80abe20 con 0x7f3a8c06ae40 2023-10-26T01:36:48.836 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:48.834+0000 7f3a9dffb640 1 -- 172.21.15.169:0/3995473971 <== mgr.4112 v2:172.21.15.170:6832/1187156918 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+283113 (secure 0 0 0) 0x7f3aa80abe20 con 0x7f3a8c06ae40 2023-10-26T01:36:48.842 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:36:48.843 INFO:teuthology.orchestra.run.smithi169.stderr:dumped all 2023-10-26T01:36:48.845 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:48.840+0000 7f3aaf2b4640 1 -- 172.21.15.169:0/3995473971 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3a8c06ae40 msgr2=0x7f3a8c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:48.845 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:48.840+0000 7f3aaf2b4640 1 --2- 172.21.15.169:0/3995473971 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3a8c06ae40 0x7f3a8c06d300 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f3aa4036000 tx=0x7f3aa4004730 comp rx=0 tx=0).stop 2023-10-26T01:36:48.845 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:48.840+0000 7f3aaf2b4640 1 -- 172.21.15.169:0/3995473971 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3aa814f6a0 msgr2=0x7f3aa8070d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:48.845 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:48.841+0000 7f3aaf2b4640 1 --2- 172.21.15.169:0/3995473971 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3aa814f6a0 0x7f3aa8070d40 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f3aa000cb10 tx=0x7f3aa000a060 comp rx=0 tx=0).stop 2023-10-26T01:36:48.846 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:48.841+0000 7f3aaf2b4640 1 -- 172.21.15.169:0/3995473971 shutdown_connections 2023-10-26T01:36:48.846 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:48.841+0000 7f3aaf2b4640 1 --2- 172.21.15.169:0/3995473971 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3a8c06ae40 0x7f3a8c06d300 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:48.846 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:48.841+0000 7f3aaf2b4640 1 --2- 172.21.15.169:0/3995473971 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3aa814f6a0 0x7f3aa8070d40 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:48.846 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:48.841+0000 7f3aaf2b4640 1 --2- 172.21.15.169:0/3995473971 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3aa814f2c0 0x7f3aa8151430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:48.847 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:48.841+0000 7f3aaf2b4640 1 --2- 172.21.15.169:0/3995473971 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3aa80cde30 0x7f3aa814a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:48.847 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:48.841+0000 7f3aaf2b4640 1 -- 172.21.15.169:0/3995473971 >> 172.21.15.169:0/3995473971 conn(0x7f3aa8069930 msgr2=0x7f3aa80cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:48.847 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:48.841+0000 7f3aaf2b4640 1 -- 172.21.15.169:0/3995473971 shutdown_connections 2023-10-26T01:36:48.848 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:48.841+0000 7f3aaf2b4640 1 -- 172.21.15.169:0/3995473971 wait complete. 2023-10-26T01:36:48.863 INFO:teuthology.orchestra.run.smithi169.stdout:{"pg_ready":true,"pg_map":{"version":285,"stamp":"2023-10-26T01:36:48.261662+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":5486,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1557013,"statfs":{"total":1600000000,"available":1594381726,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.887873"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.414212+0000","last_change":"2023-10-26T01:36:44.414114+0000","last_active":"2023-10-26T01:36:44.414212+0000","last_peered":"2023-10-26T01:36:44.414212+0000","last_clean":"2023-10-26T01:36:44.414212+0000","last_became_active":"2023-10-26T01:36:44.413833+0000","last_became_peered":"2023-10-26T01:36:44.413833+0000","last_unstale":"2023-10-26T01:36:44.414212+0000","last_undegraded":"2023-10-26T01:36:44.414212+0000","last_fullsized":"2023-10-26T01:36:44.414212+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:07:55.539688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.424610+0000","last_change":"2023-10-26T01:36:44.423269+0000","last_active":"2023-10-26T01:36:44.424610+0000","last_peered":"2023-10-26T01:36:44.424610+0000","last_clean":"2023-10-26T01:36:44.424610+0000","last_became_active":"2023-10-26T01:36:44.421972+0000","last_became_peered":"2023-10-26T01:36:44.421972+0000","last_unstale":"2023-10-26T01:36:44.424610+0000","last_undegraded":"2023-10-26T01:36:44.424610+0000","last_fullsized":"2023-10-26T01:36:44.424610+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:46:28.059120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T01:36:45.397183+0000","last_change":"2023-10-26T01:36:44.417152+0000","last_active":"2023-10-26T01:36:45.397183+0000","last_peered":"2023-10-26T01:36:45.397183+0000","last_clean":"2023-10-26T01:36:45.397183+0000","last_became_active":"2023-10-26T01:36:44.415589+0000","last_became_peered":"2023-10-26T01:36:44.415589+0000","last_unstale":"2023-10-26T01:36:45.397183+0000","last_undegraded":"2023-10-26T01:36:45.397183+0000","last_fullsized":"2023-10-26T01:36:45.397183+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:23:09.299929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.431809+0000","last_change":"2023-10-26T01:36:44.431733+0000","last_active":"2023-10-26T01:36:44.431809+0000","last_peered":"2023-10-26T01:36:44.431809+0000","last_clean":"2023-10-26T01:36:44.431809+0000","last_became_active":"2023-10-26T01:36:44.429537+0000","last_became_peered":"2023-10-26T01:36:44.429537+0000","last_unstale":"2023-10-26T01:36:44.431809+0000","last_undegraded":"2023-10-26T01:36:44.431809+0000","last_fullsized":"2023-10-26T01:36:44.431809+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:07:08.053814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T01:36:45.398114+0000","last_change":"2023-10-26T01:36:44.412599+0000","last_active":"2023-10-26T01:36:45.398114+0000","last_peered":"2023-10-26T01:36:45.398114+0000","last_clean":"2023-10-26T01:36:45.398114+0000","last_became_active":"2023-10-26T01:36:44.412261+0000","last_became_peered":"2023-10-26T01:36:44.412261+0000","last_unstale":"2023-10-26T01:36:45.398114+0000","last_undegraded":"2023-10-26T01:36:45.398114+0000","last_fullsized":"2023-10-26T01:36:45.398114+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:40:43.052815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T01:36:45.395272+0000","last_change":"2023-10-26T01:36:44.426899+0000","last_active":"2023-10-26T01:36:45.395272+0000","last_peered":"2023-10-26T01:36:45.395272+0000","last_clean":"2023-10-26T01:36:45.395272+0000","last_became_active":"2023-10-26T01:36:44.426565+0000","last_became_peered":"2023-10-26T01:36:44.426565+0000","last_unstale":"2023-10-26T01:36:45.395272+0000","last_undegraded":"2023-10-26T01:36:45.395272+0000","last_fullsized":"2023-10-26T01:36:45.395272+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:39:53.140736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.418498+0000","last_change":"2023-10-26T01:36:44.417812+0000","last_active":"2023-10-26T01:36:44.418498+0000","last_peered":"2023-10-26T01:36:44.418498+0000","last_clean":"2023-10-26T01:36:44.418498+0000","last_became_active":"2023-10-26T01:36:44.417384+0000","last_became_peered":"2023-10-26T01:36:44.417384+0000","last_unstale":"2023-10-26T01:36:44.418498+0000","last_undegraded":"2023-10-26T01:36:44.418498+0000","last_fullsized":"2023-10-26T01:36:44.418498+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:00:56.306188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T01:36:45.394850+0000","last_change":"2023-10-26T01:36:44.415752+0000","last_active":"2023-10-26T01:36:45.394850+0000","last_peered":"2023-10-26T01:36:45.394850+0000","last_clean":"2023-10-26T01:36:45.394850+0000","last_became_active":"2023-10-26T01:36:44.415073+0000","last_became_peered":"2023-10-26T01:36:44.415073+0000","last_unstale":"2023-10-26T01:36:45.394850+0000","last_undegraded":"2023-10-26T01:36:45.394850+0000","last_fullsized":"2023-10-26T01:36:45.394850+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:17:01.708641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T01:36:45.394671+0000","last_change":"2023-10-26T01:36:44.414150+0000","last_active":"2023-10-26T01:36:45.394671+0000","last_peered":"2023-10-26T01:36:45.394671+0000","last_clean":"2023-10-26T01:36:45.394671+0000","last_became_active":"2023-10-26T01:36:44.413892+0000","last_became_peered":"2023-10-26T01:36:44.413892+0000","last_unstale":"2023-10-26T01:36:45.394671+0000","last_undegraded":"2023-10-26T01:36:45.394671+0000","last_fullsized":"2023-10-26T01:36:45.394671+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:47:35.769327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.421024+0000","last_change":"2023-10-26T01:36:44.420947+0000","last_active":"2023-10-26T01:36:44.421024+0000","last_peered":"2023-10-26T01:36:44.421024+0000","last_clean":"2023-10-26T01:36:44.421024+0000","last_became_active":"2023-10-26T01:36:44.420693+0000","last_became_peered":"2023-10-26T01:36:44.420693+0000","last_unstale":"2023-10-26T01:36:44.421024+0000","last_undegraded":"2023-10-26T01:36:44.421024+0000","last_fullsized":"2023-10-26T01:36:44.421024+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:06:52.992741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.411244+0000","last_change":"2023-10-26T01:36:44.411029+0000","last_active":"2023-10-26T01:36:44.411244+0000","last_peered":"2023-10-26T01:36:44.411244+0000","last_clean":"2023-10-26T01:36:44.411244+0000","last_became_active":"2023-10-26T01:36:44.410541+0000","last_became_peered":"2023-10-26T01:36:44.410541+0000","last_unstale":"2023-10-26T01:36:44.411244+0000","last_undegraded":"2023-10-26T01:36:44.411244+0000","last_fullsized":"2023-10-26T01:36:44.411244+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:00:59.472904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.420853+0000","last_change":"2023-10-26T01:36:44.420237+0000","last_active":"2023-10-26T01:36:44.420853+0000","last_peered":"2023-10-26T01:36:44.420853+0000","last_clean":"2023-10-26T01:36:44.420853+0000","last_became_active":"2023-10-26T01:36:44.419936+0000","last_became_peered":"2023-10-26T01:36:44.419936+0000","last_unstale":"2023-10-26T01:36:44.420853+0000","last_undegraded":"2023-10-26T01:36:44.420853+0000","last_fullsized":"2023-10-26T01:36:44.420853+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:47:00.376441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T01:36:45.397538+0000","last_change":"2023-10-26T01:36:44.410566+0000","last_active":"2023-10-26T01:36:45.397538+0000","last_peered":"2023-10-26T01:36:45.397538+0000","last_clean":"2023-10-26T01:36:45.397538+0000","last_became_active":"2023-10-26T01:36:44.410125+0000","last_became_peered":"2023-10-26T01:36:44.410125+0000","last_unstale":"2023-10-26T01:36:45.397538+0000","last_undegraded":"2023-10-26T01:36:45.397538+0000","last_fullsized":"2023-10-26T01:36:45.397538+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:49:19.484142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.431570+0000","last_change":"2023-10-26T01:36:44.431294+0000","last_active":"2023-10-26T01:36:44.431570+0000","last_peered":"2023-10-26T01:36:44.431570+0000","last_clean":"2023-10-26T01:36:44.431570+0000","last_became_active":"2023-10-26T01:36:44.428358+0000","last_became_peered":"2023-10-26T01:36:44.428358+0000","last_unstale":"2023-10-26T01:36:44.431570+0000","last_undegraded":"2023-10-26T01:36:44.431570+0000","last_fullsized":"2023-10-26T01:36:44.431570+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:19:39.814025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T01:36:46.740583+0000","last_change":"2023-10-26T01:36:44.411865+0000","last_active":"2023-10-26T01:36:46.740583+0000","last_peered":"2023-10-26T01:36:46.740583+0000","last_clean":"2023-10-26T01:36:46.740583+0000","last_became_active":"2023-10-26T01:36:44.411247+0000","last_became_peered":"2023-10-26T01:36:44.411247+0000","last_unstale":"2023-10-26T01:36:46.740583+0000","last_undegraded":"2023-10-26T01:36:46.740583+0000","last_fullsized":"2023-10-26T01:36:46.740583+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:11:27.820420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T01:36:45.399102+0000","last_change":"2023-10-26T01:36:44.416818+0000","last_active":"2023-10-26T01:36:45.399102+0000","last_peered":"2023-10-26T01:36:45.399102+0000","last_clean":"2023-10-26T01:36:45.399102+0000","last_became_active":"2023-10-26T01:36:44.413677+0000","last_became_peered":"2023-10-26T01:36:44.413677+0000","last_unstale":"2023-10-26T01:36:45.399102+0000","last_undegraded":"2023-10-26T01:36:45.399102+0000","last_fullsized":"2023-10-26T01:36:45.399102+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:15:45.948645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T01:36:45.400187+0000","last_change":"2023-10-26T01:36:44.418530+0000","last_active":"2023-10-26T01:36:45.400187+0000","last_peered":"2023-10-26T01:36:45.400187+0000","last_clean":"2023-10-26T01:36:45.400187+0000","last_became_active":"2023-10-26T01:36:44.418276+0000","last_became_peered":"2023-10-26T01:36:44.418276+0000","last_unstale":"2023-10-26T01:36:45.400187+0000","last_undegraded":"2023-10-26T01:36:45.400187+0000","last_fullsized":"2023-10-26T01:36:45.400187+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:53:42.040042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.425986+0000","last_change":"2023-10-26T01:36:44.425317+0000","last_active":"2023-10-26T01:36:44.425986+0000","last_peered":"2023-10-26T01:36:44.425986+0000","last_clean":"2023-10-26T01:36:44.425986+0000","last_became_active":"2023-10-26T01:36:44.425001+0000","last_became_peered":"2023-10-26T01:36:44.425001+0000","last_unstale":"2023-10-26T01:36:44.425986+0000","last_undegraded":"2023-10-26T01:36:44.425986+0000","last_fullsized":"2023-10-26T01:36:44.425986+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:54:41.116566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.421409+0000","last_change":"2023-10-26T01:36:44.421329+0000","last_active":"2023-10-26T01:36:44.421409+0000","last_peered":"2023-10-26T01:36:44.421409+0000","last_clean":"2023-10-26T01:36:44.421409+0000","last_became_active":"2023-10-26T01:36:44.420858+0000","last_became_peered":"2023-10-26T01:36:44.420858+0000","last_unstale":"2023-10-26T01:36:44.421409+0000","last_undegraded":"2023-10-26T01:36:44.421409+0000","last_fullsized":"2023-10-26T01:36:44.421409+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:44:01.585234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.428069+0000","last_change":"2023-10-26T01:36:44.427089+0000","last_active":"2023-10-26T01:36:44.428069+0000","last_peered":"2023-10-26T01:36:44.428069+0000","last_clean":"2023-10-26T01:36:44.428069+0000","last_became_active":"2023-10-26T01:36:44.426855+0000","last_became_peered":"2023-10-26T01:36:44.426855+0000","last_unstale":"2023-10-26T01:36:44.428069+0000","last_undegraded":"2023-10-26T01:36:44.428069+0000","last_fullsized":"2023-10-26T01:36:44.428069+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:05:27.816078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.428269+0000","last_change":"2023-10-26T01:36:44.428187+0000","last_active":"2023-10-26T01:36:44.428269+0000","last_peered":"2023-10-26T01:36:44.428269+0000","last_clean":"2023-10-26T01:36:44.428269+0000","last_became_active":"2023-10-26T01:36:44.426548+0000","last_became_peered":"2023-10-26T01:36:44.426548+0000","last_unstale":"2023-10-26T01:36:44.428269+0000","last_undegraded":"2023-10-26T01:36:44.428269+0000","last_fullsized":"2023-10-26T01:36:44.428269+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:36:34.521544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.424666+0000","last_change":"2023-10-26T01:36:44.423551+0000","last_active":"2023-10-26T01:36:44.424666+0000","last_peered":"2023-10-26T01:36:44.424666+0000","last_clean":"2023-10-26T01:36:44.424666+0000","last_became_active":"2023-10-26T01:36:44.422694+0000","last_became_peered":"2023-10-26T01:36:44.422694+0000","last_unstale":"2023-10-26T01:36:44.424666+0000","last_undegraded":"2023-10-26T01:36:44.424666+0000","last_fullsized":"2023-10-26T01:36:44.424666+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:36:34.521544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.420675+0000","last_change":"2023-10-26T01:36:44.420192+0000","last_active":"2023-10-26T01:36:44.420675+0000","last_peered":"2023-10-26T01:36:44.420675+0000","last_clean":"2023-10-26T01:36:44.420675+0000","last_became_active":"2023-10-26T01:36:44.419366+0000","last_became_peered":"2023-10-26T01:36:44.419366+0000","last_unstale":"2023-10-26T01:36:44.420675+0000","last_undegraded":"2023-10-26T01:36:44.420675+0000","last_fullsized":"2023-10-26T01:36:44.420675+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:22:24.461105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.430953+0000","last_change":"2023-10-26T01:36:44.430668+0000","last_active":"2023-10-26T01:36:44.430953+0000","last_peered":"2023-10-26T01:36:44.430953+0000","last_clean":"2023-10-26T01:36:44.430953+0000","last_became_active":"2023-10-26T01:36:44.426478+0000","last_became_peered":"2023-10-26T01:36:44.426478+0000","last_unstale":"2023-10-26T01:36:44.430953+0000","last_undegraded":"2023-10-26T01:36:44.430953+0000","last_fullsized":"2023-10-26T01:36:44.430953+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:59:03.849415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.420957+0000","last_change":"2023-10-26T01:36:44.420836+0000","last_active":"2023-10-26T01:36:44.420957+0000","last_peered":"2023-10-26T01:36:44.420957+0000","last_clean":"2023-10-26T01:36:44.420957+0000","last_became_active":"2023-10-26T01:36:44.420120+0000","last_became_peered":"2023-10-26T01:36:44.420120+0000","last_unstale":"2023-10-26T01:36:44.420957+0000","last_undegraded":"2023-10-26T01:36:44.420957+0000","last_fullsized":"2023-10-26T01:36:44.420957+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:19:39.814025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T01:36:45.398189+0000","last_change":"2023-10-26T01:36:44.408697+0000","last_active":"2023-10-26T01:36:45.398189+0000","last_peered":"2023-10-26T01:36:45.398189+0000","last_clean":"2023-10-26T01:36:45.398189+0000","last_became_active":"2023-10-26T01:36:44.408247+0000","last_became_peered":"2023-10-26T01:36:44.408247+0000","last_unstale":"2023-10-26T01:36:45.398189+0000","last_undegraded":"2023-10-26T01:36:45.398189+0000","last_fullsized":"2023-10-26T01:36:45.398189+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:03:27.636888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T01:36:45.401077+0000","last_change":"2023-10-26T01:36:44.416347+0000","last_active":"2023-10-26T01:36:45.401077+0000","last_peered":"2023-10-26T01:36:45.401077+0000","last_clean":"2023-10-26T01:36:45.401077+0000","last_became_active":"2023-10-26T01:36:44.416111+0000","last_became_peered":"2023-10-26T01:36:44.416111+0000","last_unstale":"2023-10-26T01:36:45.401077+0000","last_undegraded":"2023-10-26T01:36:45.401077+0000","last_fullsized":"2023-10-26T01:36:45.401077+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:23:26.163872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.419742+0000","last_change":"2023-10-26T01:36:44.419604+0000","last_active":"2023-10-26T01:36:44.419742+0000","last_peered":"2023-10-26T01:36:44.419742+0000","last_clean":"2023-10-26T01:36:44.419742+0000","last_became_active":"2023-10-26T01:36:44.417497+0000","last_became_peered":"2023-10-26T01:36:44.417497+0000","last_unstale":"2023-10-26T01:36:44.419742+0000","last_undegraded":"2023-10-26T01:36:44.419742+0000","last_fullsized":"2023-10-26T01:36:44.419742+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:59:34.360763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.425549+0000","last_change":"2023-10-26T01:36:44.425447+0000","last_active":"2023-10-26T01:36:44.425549+0000","last_peered":"2023-10-26T01:36:44.425549+0000","last_clean":"2023-10-26T01:36:44.425549+0000","last_became_active":"2023-10-26T01:36:44.425003+0000","last_became_peered":"2023-10-26T01:36:44.425003+0000","last_unstale":"2023-10-26T01:36:44.425549+0000","last_undegraded":"2023-10-26T01:36:44.425549+0000","last_fullsized":"2023-10-26T01:36:44.425549+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:00:56.306188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.421353+0000","last_change":"2023-10-26T01:36:44.421053+0000","last_active":"2023-10-26T01:36:44.421353+0000","last_peered":"2023-10-26T01:36:44.421353+0000","last_clean":"2023-10-26T01:36:44.421353+0000","last_became_active":"2023-10-26T01:36:44.420529+0000","last_became_peered":"2023-10-26T01:36:44.420529+0000","last_unstale":"2023-10-26T01:36:44.421353+0000","last_undegraded":"2023-10-26T01:36:44.421353+0000","last_fullsized":"2023-10-26T01:36:44.421353+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:54:41.116566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T01:36:45.399624+0000","last_change":"2023-10-26T01:36:44.417085+0000","last_active":"2023-10-26T01:36:45.399624+0000","last_peered":"2023-10-26T01:36:45.399624+0000","last_clean":"2023-10-26T01:36:45.399624+0000","last_became_active":"2023-10-26T01:36:44.416250+0000","last_became_peered":"2023-10-26T01:36:44.416250+0000","last_unstale":"2023-10-26T01:36:45.399624+0000","last_undegraded":"2023-10-26T01:36:45.399624+0000","last_fullsized":"2023-10-26T01:36:45.399624+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:39:05.780690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.418854+0000","last_change":"2023-10-26T01:36:44.418641+0000","last_active":"2023-10-26T01:36:44.418854+0000","last_peered":"2023-10-26T01:36:44.418854+0000","last_clean":"2023-10-26T01:36:44.418854+0000","last_became_active":"2023-10-26T01:36:44.418334+0000","last_became_peered":"2023-10-26T01:36:44.418334+0000","last_unstale":"2023-10-26T01:36:44.418854+0000","last_undegraded":"2023-10-26T01:36:44.418854+0000","last_fullsized":"2023-10-26T01:36:44.418854+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:54:11.340492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T01:36:45.398874+0000","last_change":"2023-10-26T01:36:44.415370+0000","last_active":"2023-10-26T01:36:45.398874+0000","last_peered":"2023-10-26T01:36:45.398874+0000","last_clean":"2023-10-26T01:36:45.398874+0000","last_became_active":"2023-10-26T01:36:44.415100+0000","last_became_peered":"2023-10-26T01:36:44.415100+0000","last_unstale":"2023-10-26T01:36:45.398874+0000","last_undegraded":"2023-10-26T01:36:45.398874+0000","last_fullsized":"2023-10-26T01:36:45.398874+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:24:10.566721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"102'4","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.435633+0000","last_change":"2023-10-26T01:36:43.413293+0000","last_active":"2023-10-26T01:36:44.435633+0000","last_peered":"2023-10-26T01:36:44.435633+0000","last_clean":"2023-10-26T01:36:44.435633+0000","last_became_active":"2023-10-26T01:36:43.411577+0000","last_became_peered":"2023-10-26T01:36:43.411577+0000","last_unstale":"2023-10-26T01:36:44.435633+0000","last_undegraded":"2023-10-26T01:36:44.435633+0000","last_fullsized":"2023-10-26T01:36:44.435633+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_clean_scrub_stamp":"2023-10-26T01:36:42.357920+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:51:19.298136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.420054+0000","last_change":"2023-10-26T01:36:44.419973+0000","last_active":"2023-10-26T01:36:44.420054+0000","last_peered":"2023-10-26T01:36:44.420054+0000","last_clean":"2023-10-26T01:36:44.420054+0000","last_became_active":"2023-10-26T01:36:44.419067+0000","last_became_peered":"2023-10-26T01:36:44.419067+0000","last_unstale":"2023-10-26T01:36:44.420054+0000","last_undegraded":"2023-10-26T01:36:44.420054+0000","last_fullsized":"2023-10-26T01:36:44.420054+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:30:25.463376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"102'1","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T01:36:45.399433+0000","last_change":"2023-10-26T01:36:43.405474+0000","last_active":"2023-10-26T01:36:45.399433+0000","last_peered":"2023-10-26T01:36:45.399433+0000","last_clean":"2023-10-26T01:36:45.399433+0000","last_became_active":"2023-10-26T01:36:43.404222+0000","last_became_peered":"2023-10-26T01:36:43.404222+0000","last_unstale":"2023-10-26T01:36:45.399433+0000","last_undegraded":"2023-10-26T01:36:45.399433+0000","last_fullsized":"2023-10-26T01:36:45.399433+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_clean_scrub_stamp":"2023-10-26T01:36:42.357920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:49:10.957265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.418911+0000","last_change":"2023-10-26T01:36:44.418623+0000","last_active":"2023-10-26T01:36:44.418911+0000","last_peered":"2023-10-26T01:36:44.418911+0000","last_clean":"2023-10-26T01:36:44.418911+0000","last_became_active":"2023-10-26T01:36:44.418274+0000","last_became_peered":"2023-10-26T01:36:44.418274+0000","last_unstale":"2023-10-26T01:36:44.418911+0000","last_undegraded":"2023-10-26T01:36:44.418911+0000","last_fullsized":"2023-10-26T01:36:44.418911+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:26:22.614645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"102'2","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.436018+0000","last_change":"2023-10-26T01:36:43.411787+0000","last_active":"2023-10-26T01:36:44.436018+0000","last_peered":"2023-10-26T01:36:44.436018+0000","last_clean":"2023-10-26T01:36:44.436018+0000","last_became_active":"2023-10-26T01:36:43.408087+0000","last_became_peered":"2023-10-26T01:36:43.408087+0000","last_unstale":"2023-10-26T01:36:44.436018+0000","last_undegraded":"2023-10-26T01:36:44.436018+0000","last_fullsized":"2023-10-26T01:36:44.436018+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_clean_scrub_stamp":"2023-10-26T01:36:42.357920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:23:25.150110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T01:36:45.399549+0000","last_change":"2023-10-26T01:36:44.418114+0000","last_active":"2023-10-26T01:36:45.399549+0000","last_peered":"2023-10-26T01:36:45.399549+0000","last_clean":"2023-10-26T01:36:45.399549+0000","last_became_active":"2023-10-26T01:36:44.417888+0000","last_became_peered":"2023-10-26T01:36:44.417888+0000","last_unstale":"2023-10-26T01:36:45.399549+0000","last_undegraded":"2023-10-26T01:36:45.399549+0000","last_fullsized":"2023-10-26T01:36:45.399549+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:29:25.931265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"102'2","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.435273+0000","last_change":"2023-10-26T01:36:43.406074+0000","last_active":"2023-10-26T01:36:44.435273+0000","last_peered":"2023-10-26T01:36:44.435273+0000","last_clean":"2023-10-26T01:36:44.435273+0000","last_became_active":"2023-10-26T01:36:43.405845+0000","last_became_peered":"2023-10-26T01:36:43.405845+0000","last_unstale":"2023-10-26T01:36:44.435273+0000","last_undegraded":"2023-10-26T01:36:44.435273+0000","last_fullsized":"2023-10-26T01:36:44.435273+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_clean_scrub_stamp":"2023-10-26T01:36:42.357920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:27:51.549146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.429845+0000","last_change":"2023-10-26T01:36:44.428810+0000","last_active":"2023-10-26T01:36:44.429845+0000","last_peered":"2023-10-26T01:36:44.429845+0000","last_clean":"2023-10-26T01:36:44.429845+0000","last_became_active":"2023-10-26T01:36:44.428529+0000","last_became_peered":"2023-10-26T01:36:44.428529+0000","last_unstale":"2023-10-26T01:36:44.429845+0000","last_undegraded":"2023-10-26T01:36:44.429845+0000","last_fullsized":"2023-10-26T01:36:44.429845+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:34:32.496565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"102'1","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T01:36:45.400759+0000","last_change":"2023-10-26T01:36:43.400400+0000","last_active":"2023-10-26T01:36:45.400759+0000","last_peered":"2023-10-26T01:36:45.400759+0000","last_clean":"2023-10-26T01:36:45.400759+0000","last_became_active":"2023-10-26T01:36:43.399296+0000","last_became_peered":"2023-10-26T01:36:43.399296+0000","last_unstale":"2023-10-26T01:36:45.400759+0000","last_undegraded":"2023-10-26T01:36:45.400759+0000","last_fullsized":"2023-10-26T01:36:45.400759+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_clean_scrub_stamp":"2023-10-26T01:36:42.357920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:17:00.694879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T01:36:45.400528+0000","last_change":"2023-10-26T01:36:44.419023+0000","last_active":"2023-10-26T01:36:45.400528+0000","last_peered":"2023-10-26T01:36:45.400528+0000","last_clean":"2023-10-26T01:36:45.400528+0000","last_became_active":"2023-10-26T01:36:44.418808+0000","last_became_peered":"2023-10-26T01:36:44.418808+0000","last_unstale":"2023-10-26T01:36:45.400528+0000","last_undegraded":"2023-10-26T01:36:45.400528+0000","last_fullsized":"2023-10-26T01:36:45.400528+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:28:30.194209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"102'1","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T01:36:45.398349+0000","last_change":"2023-10-26T01:36:43.402719+0000","last_active":"2023-10-26T01:36:45.398349+0000","last_peered":"2023-10-26T01:36:45.398349+0000","last_clean":"2023-10-26T01:36:45.398349+0000","last_became_active":"2023-10-26T01:36:43.402064+0000","last_became_peered":"2023-10-26T01:36:43.402064+0000","last_unstale":"2023-10-26T01:36:45.398349+0000","last_undegraded":"2023-10-26T01:36:45.398349+0000","last_fullsized":"2023-10-26T01:36:45.398349+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_clean_scrub_stamp":"2023-10-26T01:36:42.357920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:47:52.274415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T01:36:45.398904+0000","last_change":"2023-10-26T01:36:44.416913+0000","last_active":"2023-10-26T01:36:45.398904+0000","last_peered":"2023-10-26T01:36:45.398904+0000","last_clean":"2023-10-26T01:36:45.398904+0000","last_became_active":"2023-10-26T01:36:44.414717+0000","last_became_peered":"2023-10-26T01:36:44.414717+0000","last_unstale":"2023-10-26T01:36:45.398904+0000","last_undegraded":"2023-10-26T01:36:45.398904+0000","last_fullsized":"2023-10-26T01:36:45.398904+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:13:56.403796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.404317+0000","last_change":"2023-10-26T01:36:43.410769+0000","last_active":"2023-10-26T01:36:44.404317+0000","last_peered":"2023-10-26T01:36:44.404317+0000","last_clean":"2023-10-26T01:36:44.404317+0000","last_became_active":"2023-10-26T01:36:43.410528+0000","last_became_peered":"2023-10-26T01:36:43.410528+0000","last_unstale":"2023-10-26T01:36:44.404317+0000","last_undegraded":"2023-10-26T01:36:44.404317+0000","last_fullsized":"2023-10-26T01:36:44.404317+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_clean_scrub_stamp":"2023-10-26T01:36:42.357920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:44:18.601112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T01:36:46.740706+0000","last_change":"2023-10-26T01:36:44.409818+0000","last_active":"2023-10-26T01:36:46.740706+0000","last_peered":"2023-10-26T01:36:46.740706+0000","last_clean":"2023-10-26T01:36:46.740706+0000","last_became_active":"2023-10-26T01:36:44.408687+0000","last_became_peered":"2023-10-26T01:36:44.408687+0000","last_unstale":"2023-10-26T01:36:46.740706+0000","last_undegraded":"2023-10-26T01:36:46.740706+0000","last_fullsized":"2023-10-26T01:36:46.740706+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:08:16.050379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.402810+0000","last_change":"2023-10-26T01:36:43.403865+0000","last_active":"2023-10-26T01:36:44.402810+0000","last_peered":"2023-10-26T01:36:44.402810+0000","last_clean":"2023-10-26T01:36:44.402810+0000","last_became_active":"2023-10-26T01:36:43.403529+0000","last_became_peered":"2023-10-26T01:36:43.403529+0000","last_unstale":"2023-10-26T01:36:44.402810+0000","last_undegraded":"2023-10-26T01:36:44.402810+0000","last_fullsized":"2023-10-26T01:36:44.402810+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_clean_scrub_stamp":"2023-10-26T01:36:42.357920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:58:45.070927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T01:36:46.740316+0000","last_change":"2023-10-26T01:36:44.418337+0000","last_active":"2023-10-26T01:36:46.740316+0000","last_peered":"2023-10-26T01:36:46.740316+0000","last_clean":"2023-10-26T01:36:46.740316+0000","last_became_active":"2023-10-26T01:36:44.417971+0000","last_became_peered":"2023-10-26T01:36:44.417971+0000","last_unstale":"2023-10-26T01:36:46.740316+0000","last_undegraded":"2023-10-26T01:36:46.740316+0000","last_fullsized":"2023-10-26T01:36:46.740316+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:23:48.053407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.410759+0000","last_change":"2023-10-26T01:36:43.413125+0000","last_active":"2023-10-26T01:36:44.410759+0000","last_peered":"2023-10-26T01:36:44.410759+0000","last_clean":"2023-10-26T01:36:44.410759+0000","last_became_active":"2023-10-26T01:36:43.412871+0000","last_became_peered":"2023-10-26T01:36:43.412871+0000","last_unstale":"2023-10-26T01:36:44.410759+0000","last_undegraded":"2023-10-26T01:36:44.410759+0000","last_fullsized":"2023-10-26T01:36:44.410759+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_clean_scrub_stamp":"2023-10-26T01:36:42.357920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:54:00.451217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T01:36:45.396194+0000","last_change":"2023-10-26T01:36:44.428648+0000","last_active":"2023-10-26T01:36:45.396194+0000","last_peered":"2023-10-26T01:36:45.396194+0000","last_clean":"2023-10-26T01:36:45.396194+0000","last_became_active":"2023-10-26T01:36:44.428465+0000","last_became_peered":"2023-10-26T01:36:44.428465+0000","last_unstale":"2023-10-26T01:36:45.396194+0000","last_undegraded":"2023-10-26T01:36:45.396194+0000","last_fullsized":"2023-10-26T01:36:45.396194+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:17:14.458413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.413953+0000","last_change":"2023-10-26T01:36:43.411329+0000","last_active":"2023-10-26T01:36:44.413953+0000","last_peered":"2023-10-26T01:36:44.413953+0000","last_clean":"2023-10-26T01:36:44.413953+0000","last_became_active":"2023-10-26T01:36:43.409687+0000","last_became_peered":"2023-10-26T01:36:43.409687+0000","last_unstale":"2023-10-26T01:36:44.413953+0000","last_undegraded":"2023-10-26T01:36:44.413953+0000","last_fullsized":"2023-10-26T01:36:44.413953+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_clean_scrub_stamp":"2023-10-26T01:36:42.357920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:47:34.755565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.420627+0000","last_change":"2023-10-26T01:36:44.420555+0000","last_active":"2023-10-26T01:36:44.420627+0000","last_peered":"2023-10-26T01:36:44.420627+0000","last_clean":"2023-10-26T01:36:44.420627+0000","last_became_active":"2023-10-26T01:36:44.414973+0000","last_became_peered":"2023-10-26T01:36:44.414973+0000","last_unstale":"2023-10-26T01:36:44.420627+0000","last_undegraded":"2023-10-26T01:36:44.420627+0000","last_fullsized":"2023-10-26T01:36:44.420627+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:17:08.255587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"102'3","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.434666+0000","last_change":"2023-10-26T01:36:43.414186+0000","last_active":"2023-10-26T01:36:44.434666+0000","last_peered":"2023-10-26T01:36:44.434666+0000","last_clean":"2023-10-26T01:36:44.434666+0000","last_became_active":"2023-10-26T01:36:43.413573+0000","last_became_peered":"2023-10-26T01:36:43.413573+0000","last_unstale":"2023-10-26T01:36:44.434666+0000","last_undegraded":"2023-10-26T01:36:44.434666+0000","last_fullsized":"2023-10-26T01:36:44.434666+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_clean_scrub_stamp":"2023-10-26T01:36:42.357920+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:49:10.957265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T01:36:45.397779+0000","last_change":"2023-10-26T01:36:44.409595+0000","last_active":"2023-10-26T01:36:45.397779+0000","last_peered":"2023-10-26T01:36:45.397779+0000","last_clean":"2023-10-26T01:36:45.397779+0000","last_became_active":"2023-10-26T01:36:44.409186+0000","last_became_peered":"2023-10-26T01:36:44.409186+0000","last_unstale":"2023-10-26T01:36:45.397779+0000","last_undegraded":"2023-10-26T01:36:45.397779+0000","last_fullsized":"2023-10-26T01:36:45.397779+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:31:18.065778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"102'6","reported_seq":18,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T01:36:45.395060+0000","last_change":"2023-10-26T01:36:43.407360+0000","last_active":"2023-10-26T01:36:45.395060+0000","last_peered":"2023-10-26T01:36:45.395060+0000","last_clean":"2023-10-26T01:36:45.395060+0000","last_became_active":"2023-10-26T01:36:43.407068+0000","last_became_peered":"2023-10-26T01:36:43.407068+0000","last_unstale":"2023-10-26T01:36:45.395060+0000","last_undegraded":"2023-10-26T01:36:45.395060+0000","last_fullsized":"2023-10-26T01:36:45.395060+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_clean_scrub_stamp":"2023-10-26T01:36:42.357920+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:22:23.447343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.414739+0000","last_change":"2023-10-26T01:36:44.413671+0000","last_active":"2023-10-26T01:36:44.414739+0000","last_peered":"2023-10-26T01:36:44.414739+0000","last_clean":"2023-10-26T01:36:44.414739+0000","last_became_active":"2023-10-26T01:36:44.411171+0000","last_became_peered":"2023-10-26T01:36:44.411171+0000","last_unstale":"2023-10-26T01:36:44.414739+0000","last_undegraded":"2023-10-26T01:36:44.414739+0000","last_fullsized":"2023-10-26T01:36:44.414739+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:29:42.674658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"102'2","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.433183+0000","last_change":"2023-10-26T01:36:43.406935+0000","last_active":"2023-10-26T01:36:44.433183+0000","last_peered":"2023-10-26T01:36:44.433183+0000","last_clean":"2023-10-26T01:36:44.433183+0000","last_became_active":"2023-10-26T01:36:43.405455+0000","last_became_peered":"2023-10-26T01:36:43.405455+0000","last_unstale":"2023-10-26T01:36:44.433183+0000","last_undegraded":"2023-10-26T01:36:44.433183+0000","last_fullsized":"2023-10-26T01:36:44.433183+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_clean_scrub_stamp":"2023-10-26T01:36:42.357920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:46:15.128764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T01:36:45.399222+0000","last_change":"2023-10-26T01:36:44.417076+0000","last_active":"2023-10-26T01:36:45.399222+0000","last_peered":"2023-10-26T01:36:45.399222+0000","last_clean":"2023-10-26T01:36:45.399222+0000","last_became_active":"2023-10-26T01:36:44.416479+0000","last_became_peered":"2023-10-26T01:36:44.416479+0000","last_unstale":"2023-10-26T01:36:45.399222+0000","last_undegraded":"2023-10-26T01:36:45.399222+0000","last_fullsized":"2023-10-26T01:36:45.399222+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:39:17.285562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T01:36:46.740226+0000","last_change":"2023-10-26T01:36:43.399012+0000","last_active":"2023-10-26T01:36:46.740226+0000","last_peered":"2023-10-26T01:36:46.740226+0000","last_clean":"2023-10-26T01:36:46.740226+0000","last_became_active":"2023-10-26T01:36:43.397583+0000","last_became_peered":"2023-10-26T01:36:43.397583+0000","last_unstale":"2023-10-26T01:36:46.740226+0000","last_undegraded":"2023-10-26T01:36:46.740226+0000","last_fullsized":"2023-10-26T01:36:46.740226+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_clean_scrub_stamp":"2023-10-26T01:36:42.357920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:17:05.159088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.425525+0000","last_change":"2023-10-26T01:36:44.424737+0000","last_active":"2023-10-26T01:36:44.425525+0000","last_peered":"2023-10-26T01:36:44.425525+0000","last_clean":"2023-10-26T01:36:44.425525+0000","last_became_active":"2023-10-26T01:36:44.424496+0000","last_became_peered":"2023-10-26T01:36:44.424496+0000","last_unstale":"2023-10-26T01:36:44.425525+0000","last_undegraded":"2023-10-26T01:36:44.425525+0000","last_fullsized":"2023-10-26T01:36:44.425525+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:05:27.816078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T01:36:45.395258+0000","last_change":"2023-10-26T01:36:43.412003+0000","last_active":"2023-10-26T01:36:45.395258+0000","last_peered":"2023-10-26T01:36:45.395258+0000","last_clean":"2023-10-26T01:36:45.395258+0000","last_became_active":"2023-10-26T01:36:43.410776+0000","last_became_peered":"2023-10-26T01:36:43.410776+0000","last_unstale":"2023-10-26T01:36:45.395258+0000","last_undegraded":"2023-10-26T01:36:45.395258+0000","last_fullsized":"2023-10-26T01:36:45.395258+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_clean_scrub_stamp":"2023-10-26T01:36:42.357920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:51:19.298136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.424632+0000","last_change":"2023-10-26T01:36:44.423416+0000","last_active":"2023-10-26T01:36:44.424632+0000","last_peered":"2023-10-26T01:36:44.424632+0000","last_clean":"2023-10-26T01:36:44.424632+0000","last_became_active":"2023-10-26T01:36:44.422328+0000","last_became_peered":"2023-10-26T01:36:44.422328+0000","last_unstale":"2023-10-26T01:36:44.424632+0000","last_undegraded":"2023-10-26T01:36:44.424632+0000","last_fullsized":"2023-10-26T01:36:44.424632+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:03:02.870381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"102'8","reported_seq":19,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.436423+0000","last_change":"2023-10-26T01:36:43.402704+0000","last_active":"2023-10-26T01:36:44.436423+0000","last_peered":"2023-10-26T01:36:44.436423+0000","last_clean":"2023-10-26T01:36:44.436423+0000","last_became_active":"2023-10-26T01:36:43.402404+0000","last_became_peered":"2023-10-26T01:36:43.402404+0000","last_unstale":"2023-10-26T01:36:44.436423+0000","last_undegraded":"2023-10-26T01:36:44.436423+0000","last_fullsized":"2023-10-26T01:36:44.436423+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_clean_scrub_stamp":"2023-10-26T01:36:42.357920+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:59:45.462209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T01:36:45.399858+0000","last_change":"2023-10-26T01:36:44.408926+0000","last_active":"2023-10-26T01:36:45.399858+0000","last_peered":"2023-10-26T01:36:45.399858+0000","last_clean":"2023-10-26T01:36:45.399858+0000","last_became_active":"2023-10-26T01:36:44.408516+0000","last_became_peered":"2023-10-26T01:36:44.408516+0000","last_unstale":"2023-10-26T01:36:45.399858+0000","last_undegraded":"2023-10-26T01:36:45.399858+0000","last_fullsized":"2023-10-26T01:36:45.399858+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:17:06.172850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"102'1","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.431854+0000","last_change":"2023-10-26T01:36:43.413010+0000","last_active":"2023-10-26T01:36:44.431854+0000","last_peered":"2023-10-26T01:36:44.431854+0000","last_clean":"2023-10-26T01:36:44.431854+0000","last_became_active":"2023-10-26T01:36:43.412642+0000","last_became_peered":"2023-10-26T01:36:43.412642+0000","last_unstale":"2023-10-26T01:36:44.431854+0000","last_undegraded":"2023-10-26T01:36:44.431854+0000","last_fullsized":"2023-10-26T01:36:44.431854+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_clean_scrub_stamp":"2023-10-26T01:36:42.357920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:30:33.922915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.414193+0000","last_change":"2023-10-26T01:36:44.414021+0000","last_active":"2023-10-26T01:36:44.414193+0000","last_peered":"2023-10-26T01:36:44.414193+0000","last_clean":"2023-10-26T01:36:44.414193+0000","last_became_active":"2023-10-26T01:36:44.413731+0000","last_became_peered":"2023-10-26T01:36:44.413731+0000","last_unstale":"2023-10-26T01:36:44.414193+0000","last_undegraded":"2023-10-26T01:36:44.414193+0000","last_fullsized":"2023-10-26T01:36:44.414193+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:26:07.785221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"102'2","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.436744+0000","last_change":"2023-10-26T01:36:43.412151+0000","last_active":"2023-10-26T01:36:44.436744+0000","last_peered":"2023-10-26T01:36:44.436744+0000","last_clean":"2023-10-26T01:36:44.436744+0000","last_became_active":"2023-10-26T01:36:43.408391+0000","last_became_peered":"2023-10-26T01:36:43.408391+0000","last_unstale":"2023-10-26T01:36:44.436744+0000","last_undegraded":"2023-10-26T01:36:44.436744+0000","last_fullsized":"2023-10-26T01:36:44.436744+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_clean_scrub_stamp":"2023-10-26T01:36:42.357920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:47:32.591256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T01:36:45.399926+0000","last_change":"2023-10-26T01:36:44.413370+0000","last_active":"2023-10-26T01:36:45.399926+0000","last_peered":"2023-10-26T01:36:45.399926+0000","last_clean":"2023-10-26T01:36:45.399926+0000","last_became_active":"2023-10-26T01:36:44.412999+0000","last_became_peered":"2023-10-26T01:36:44.412999+0000","last_unstale":"2023-10-26T01:36:45.399926+0000","last_undegraded":"2023-10-26T01:36:45.399926+0000","last_fullsized":"2023-10-26T01:36:45.399926+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:52:13.995886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.402066+0000","last_change":"2023-10-26T01:36:43.409895+0000","last_active":"2023-10-26T01:36:44.402066+0000","last_peered":"2023-10-26T01:36:44.402066+0000","last_clean":"2023-10-26T01:36:44.402066+0000","last_became_active":"2023-10-26T01:36:43.409665+0000","last_became_peered":"2023-10-26T01:36:43.409665+0000","last_unstale":"2023-10-26T01:36:44.402066+0000","last_undegraded":"2023-10-26T01:36:44.402066+0000","last_fullsized":"2023-10-26T01:36:44.402066+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_clean_scrub_stamp":"2023-10-26T01:36:42.357920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:05:00.560066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.425933+0000","last_change":"2023-10-26T01:36:44.421624+0000","last_active":"2023-10-26T01:36:44.425933+0000","last_peered":"2023-10-26T01:36:44.425933+0000","last_clean":"2023-10-26T01:36:44.425933+0000","last_became_active":"2023-10-26T01:36:44.421379+0000","last_became_peered":"2023-10-26T01:36:44.421379+0000","last_unstale":"2023-10-26T01:36:44.425933+0000","last_undegraded":"2023-10-26T01:36:44.425933+0000","last_fullsized":"2023-10-26T01:36:44.425933+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:51:56.198313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"102'2","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T01:36:46.739957+0000","last_change":"2023-10-26T01:36:43.411409+0000","last_active":"2023-10-26T01:36:46.739957+0000","last_peered":"2023-10-26T01:36:46.739957+0000","last_clean":"2023-10-26T01:36:46.739957+0000","last_became_active":"2023-10-26T01:36:43.410628+0000","last_became_peered":"2023-10-26T01:36:43.410628+0000","last_unstale":"2023-10-26T01:36:46.739957+0000","last_undegraded":"2023-10-26T01:36:46.739957+0000","last_fullsized":"2023-10-26T01:36:46.739957+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_clean_scrub_stamp":"2023-10-26T01:36:42.357920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:13:55.390034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T01:36:45.395219+0000","last_change":"2023-10-26T01:36:44.415900+0000","last_active":"2023-10-26T01:36:45.395219+0000","last_peered":"2023-10-26T01:36:45.395219+0000","last_clean":"2023-10-26T01:36:45.395219+0000","last_became_active":"2023-10-26T01:36:44.415487+0000","last_became_peered":"2023-10-26T01:36:44.415487+0000","last_unstale":"2023-10-26T01:36:45.395219+0000","last_undegraded":"2023-10-26T01:36:45.395219+0000","last_fullsized":"2023-10-26T01:36:45.395219+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:02:41.666058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.416426+0000","last_change":"2023-10-26T01:36:43.406198+0000","last_active":"2023-10-26T01:36:44.416426+0000","last_peered":"2023-10-26T01:36:44.416426+0000","last_clean":"2023-10-26T01:36:44.416426+0000","last_became_active":"2023-10-26T01:36:43.405954+0000","last_became_peered":"2023-10-26T01:36:43.405954+0000","last_unstale":"2023-10-26T01:36:44.416426+0000","last_undegraded":"2023-10-26T01:36:44.416426+0000","last_fullsized":"2023-10-26T01:36:44.416426+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_clean_scrub_stamp":"2023-10-26T01:36:42.357920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:46:27.045358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.425970+0000","last_change":"2023-10-26T01:36:44.422932+0000","last_active":"2023-10-26T01:36:44.425970+0000","last_peered":"2023-10-26T01:36:44.425970+0000","last_clean":"2023-10-26T01:36:44.425970+0000","last_became_active":"2023-10-26T01:36:44.422694+0000","last_became_peered":"2023-10-26T01:36:44.422694+0000","last_unstale":"2023-10-26T01:36:44.425970+0000","last_undegraded":"2023-10-26T01:36:44.425970+0000","last_fullsized":"2023-10-26T01:36:44.425970+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:00:59.472904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.415321+0000","last_change":"2023-10-26T01:36:43.407596+0000","last_active":"2023-10-26T01:36:44.415321+0000","last_peered":"2023-10-26T01:36:44.415321+0000","last_clean":"2023-10-26T01:36:44.415321+0000","last_became_active":"2023-10-26T01:36:43.407270+0000","last_became_peered":"2023-10-26T01:36:43.407270+0000","last_unstale":"2023-10-26T01:36:44.415321+0000","last_undegraded":"2023-10-26T01:36:44.415321+0000","last_fullsized":"2023-10-26T01:36:44.415321+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_clean_scrub_stamp":"2023-10-26T01:36:42.357920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:34:31.482803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T01:36:45.395121+0000","last_change":"2023-10-26T01:36:44.405991+0000","last_active":"2023-10-26T01:36:45.395121+0000","last_peered":"2023-10-26T01:36:45.395121+0000","last_clean":"2023-10-26T01:36:45.395121+0000","last_became_active":"2023-10-26T01:36:44.405313+0000","last_became_peered":"2023-10-26T01:36:44.405313+0000","last_unstale":"2023-10-26T01:36:45.395121+0000","last_undegraded":"2023-10-26T01:36:45.395121+0000","last_fullsized":"2023-10-26T01:36:45.395121+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:03:30.551428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T01:36:46.740320+0000","last_change":"2023-10-26T01:36:43.405113+0000","last_active":"2023-10-26T01:36:46.740320+0000","last_peered":"2023-10-26T01:36:46.740320+0000","last_clean":"2023-10-26T01:36:46.740320+0000","last_became_active":"2023-10-26T01:36:43.404832+0000","last_became_peered":"2023-10-26T01:36:43.404832+0000","last_unstale":"2023-10-26T01:36:46.740320+0000","last_undegraded":"2023-10-26T01:36:46.740320+0000","last_fullsized":"2023-10-26T01:36:46.740320+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_clean_scrub_stamp":"2023-10-26T01:36:42.357920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:19:47.358532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T01:36:45.398110+0000","last_change":"2023-10-26T01:36:44.406938+0000","last_active":"2023-10-26T01:36:45.398110+0000","last_peered":"2023-10-26T01:36:45.398110+0000","last_clean":"2023-10-26T01:36:45.398110+0000","last_became_active":"2023-10-26T01:36:44.406262+0000","last_became_peered":"2023-10-26T01:36:44.406262+0000","last_unstale":"2023-10-26T01:36:45.398110+0000","last_undegraded":"2023-10-26T01:36:45.398110+0000","last_fullsized":"2023-10-26T01:36:45.398110+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:48:50.672248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"102'2","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.430733+0000","last_change":"2023-10-26T01:36:43.406790+0000","last_active":"2023-10-26T01:36:44.430733+0000","last_peered":"2023-10-26T01:36:44.430733+0000","last_clean":"2023-10-26T01:36:44.430733+0000","last_became_active":"2023-10-26T01:36:43.405322+0000","last_became_peered":"2023-10-26T01:36:43.405322+0000","last_unstale":"2023-10-26T01:36:44.430733+0000","last_undegraded":"2023-10-26T01:36:44.430733+0000","last_fullsized":"2023-10-26T01:36:44.430733+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_clean_scrub_stamp":"2023-10-26T01:36:42.357920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:31:26.478685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T01:36:45.397589+0000","last_change":"2023-10-26T01:36:44.417160+0000","last_active":"2023-10-26T01:36:45.397589+0000","last_peered":"2023-10-26T01:36:45.397589+0000","last_clean":"2023-10-26T01:36:45.397589+0000","last_became_active":"2023-10-26T01:36:44.415241+0000","last_became_peered":"2023-10-26T01:36:44.415241+0000","last_unstale":"2023-10-26T01:36:45.397589+0000","last_undegraded":"2023-10-26T01:36:45.397589+0000","last_fullsized":"2023-10-26T01:36:45.397589+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:12:30.105816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.401222+0000","last_change":"2023-10-26T01:36:43.403107+0000","last_active":"2023-10-26T01:36:44.401222+0000","last_peered":"2023-10-26T01:36:44.401222+0000","last_clean":"2023-10-26T01:36:44.401222+0000","last_became_active":"2023-10-26T01:36:43.402793+0000","last_became_peered":"2023-10-26T01:36:43.402793+0000","last_unstale":"2023-10-26T01:36:44.401222+0000","last_undegraded":"2023-10-26T01:36:44.401222+0000","last_fullsized":"2023-10-26T01:36:44.401222+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_clean_scrub_stamp":"2023-10-26T01:36:42.357920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:07:07.040053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.420693+0000","last_change":"2023-10-26T01:36:44.420208+0000","last_active":"2023-10-26T01:36:44.420693+0000","last_peered":"2023-10-26T01:36:44.420693+0000","last_clean":"2023-10-26T01:36:44.420693+0000","last_became_active":"2023-10-26T01:36:44.419705+0000","last_became_peered":"2023-10-26T01:36:44.419705+0000","last_unstale":"2023-10-26T01:36:44.420693+0000","last_undegraded":"2023-10-26T01:36:44.420693+0000","last_fullsized":"2023-10-26T01:36:44.420693+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:13:11.418923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.406508+0000","last_change":"2023-10-26T01:36:43.414122+0000","last_active":"2023-10-26T01:36:44.406508+0000","last_peered":"2023-10-26T01:36:44.406508+0000","last_clean":"2023-10-26T01:36:44.406508+0000","last_became_active":"2023-10-26T01:36:43.413589+0000","last_became_peered":"2023-10-26T01:36:43.413589+0000","last_unstale":"2023-10-26T01:36:44.406508+0000","last_undegraded":"2023-10-26T01:36:44.406508+0000","last_fullsized":"2023-10-26T01:36:44.406508+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_clean_scrub_stamp":"2023-10-26T01:36:42.357920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T01:52:13.748946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T01:36:45.399215+0000","last_change":"2023-10-26T01:36:44.409854+0000","last_active":"2023-10-26T01:36:45.399215+0000","last_peered":"2023-10-26T01:36:45.399215+0000","last_clean":"2023-10-26T01:36:45.399215+0000","last_became_active":"2023-10-26T01:36:44.409588+0000","last_became_peered":"2023-10-26T01:36:44.409588+0000","last_unstale":"2023-10-26T01:36:45.399215+0000","last_undegraded":"2023-10-26T01:36:45.399215+0000","last_fullsized":"2023-10-26T01:36:45.399215+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:04:02.797079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.411601+0000","last_change":"2023-10-26T01:36:43.413170+0000","last_active":"2023-10-26T01:36:44.411601+0000","last_peered":"2023-10-26T01:36:44.411601+0000","last_clean":"2023-10-26T01:36:44.411601+0000","last_became_active":"2023-10-26T01:36:43.412927+0000","last_became_peered":"2023-10-26T01:36:43.412927+0000","last_unstale":"2023-10-26T01:36:44.411601+0000","last_undegraded":"2023-10-26T01:36:44.411601+0000","last_fullsized":"2023-10-26T01:36:44.411601+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_clean_scrub_stamp":"2023-10-26T01:36:42.357920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:00:05.181878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T01:36:45.394877+0000","last_change":"2023-10-26T01:36:44.409405+0000","last_active":"2023-10-26T01:36:45.394877+0000","last_peered":"2023-10-26T01:36:45.394877+0000","last_clean":"2023-10-26T01:36:45.394877+0000","last_became_active":"2023-10-26T01:36:44.409093+0000","last_became_peered":"2023-10-26T01:36:44.409093+0000","last_unstale":"2023-10-26T01:36:45.394877+0000","last_undegraded":"2023-10-26T01:36:45.394877+0000","last_fullsized":"2023-10-26T01:36:45.394877+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:01:20.291335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.408321+0000","last_change":"2023-10-26T01:36:43.411506+0000","last_active":"2023-10-26T01:36:44.408321+0000","last_peered":"2023-10-26T01:36:44.408321+0000","last_clean":"2023-10-26T01:36:44.408321+0000","last_became_active":"2023-10-26T01:36:43.411002+0000","last_became_peered":"2023-10-26T01:36:43.411002+0000","last_unstale":"2023-10-26T01:36:44.408321+0000","last_undegraded":"2023-10-26T01:36:44.408321+0000","last_fullsized":"2023-10-26T01:36:44.408321+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_clean_scrub_stamp":"2023-10-26T01:36:42.357920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:39:01.465641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.414758+0000","last_change":"2023-10-26T01:36:44.414491+0000","last_active":"2023-10-26T01:36:44.414758+0000","last_peered":"2023-10-26T01:36:44.414758+0000","last_clean":"2023-10-26T01:36:44.414758+0000","last_became_active":"2023-10-26T01:36:44.414215+0000","last_became_peered":"2023-10-26T01:36:44.414215+0000","last_unstale":"2023-10-26T01:36:44.414758+0000","last_undegraded":"2023-10-26T01:36:44.414758+0000","last_fullsized":"2023-10-26T01:36:44.414758+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:47:17.876626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T01:36:45.401274+0000","last_change":"2023-10-26T01:36:43.399987+0000","last_active":"2023-10-26T01:36:45.401274+0000","last_peered":"2023-10-26T01:36:45.401274+0000","last_clean":"2023-10-26T01:36:45.401274+0000","last_became_active":"2023-10-26T01:36:43.397978+0000","last_became_peered":"2023-10-26T01:36:43.397978+0000","last_unstale":"2023-10-26T01:36:45.401274+0000","last_undegraded":"2023-10-26T01:36:45.401274+0000","last_fullsized":"2023-10-26T01:36:45.401274+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_clean_scrub_stamp":"2023-10-26T01:36:42.357920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:17:13.444651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T01:36:45.397179+0000","last_change":"2023-10-26T01:36:44.406012+0000","last_active":"2023-10-26T01:36:45.397179+0000","last_peered":"2023-10-26T01:36:45.397179+0000","last_clean":"2023-10-26T01:36:45.397179+0000","last_became_active":"2023-10-26T01:36:44.405330+0000","last_became_peered":"2023-10-26T01:36:44.405330+0000","last_unstale":"2023-10-26T01:36:45.397179+0000","last_undegraded":"2023-10-26T01:36:45.397179+0000","last_fullsized":"2023-10-26T01:36:45.397179+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:01:37.708171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.409122+0000","last_change":"2023-10-26T01:36:43.414328+0000","last_active":"2023-10-26T01:36:44.409122+0000","last_peered":"2023-10-26T01:36:44.409122+0000","last_clean":"2023-10-26T01:36:44.409122+0000","last_became_active":"2023-10-26T01:36:43.413959+0000","last_became_peered":"2023-10-26T01:36:43.413959+0000","last_unstale":"2023-10-26T01:36:44.409122+0000","last_undegraded":"2023-10-26T01:36:44.409122+0000","last_fullsized":"2023-10-26T01:36:44.409122+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_clean_scrub_stamp":"2023-10-26T01:36:42.357920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:47:52.274415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.424161+0000","last_change":"2023-10-26T01:36:44.424112+0000","last_active":"2023-10-26T01:36:44.424161+0000","last_peered":"2023-10-26T01:36:44.424161+0000","last_clean":"2023-10-26T01:36:44.424161+0000","last_became_active":"2023-10-26T01:36:44.423934+0000","last_became_peered":"2023-10-26T01:36:44.423934+0000","last_unstale":"2023-10-26T01:36:44.424161+0000","last_undegraded":"2023-10-26T01:36:44.424161+0000","last_fullsized":"2023-10-26T01:36:44.424161+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:51:38.585631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T01:36:45.399661+0000","last_change":"2023-10-26T01:36:43.411715+0000","last_active":"2023-10-26T01:36:45.399661+0000","last_peered":"2023-10-26T01:36:45.399661+0000","last_clean":"2023-10-26T01:36:45.399661+0000","last_became_active":"2023-10-26T01:36:43.411407+0000","last_became_peered":"2023-10-26T01:36:43.411407+0000","last_unstale":"2023-10-26T01:36:45.399661+0000","last_undegraded":"2023-10-26T01:36:45.399661+0000","last_fullsized":"2023-10-26T01:36:45.399661+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_clean_scrub_stamp":"2023-10-26T01:36:42.357920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:54:00.451217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.414839+0000","last_change":"2023-10-26T01:36:44.414419+0000","last_active":"2023-10-26T01:36:44.414839+0000","last_peered":"2023-10-26T01:36:44.414839+0000","last_clean":"2023-10-26T01:36:44.414839+0000","last_became_active":"2023-10-26T01:36:44.414195+0000","last_became_peered":"2023-10-26T01:36:44.414195+0000","last_unstale":"2023-10-26T01:36:44.414839+0000","last_undegraded":"2023-10-26T01:36:44.414839+0000","last_fullsized":"2023-10-26T01:36:44.414839+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:11:12.571272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"102'1","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T01:36:45.398401+0000","last_change":"2023-10-26T01:36:43.405515+0000","last_active":"2023-10-26T01:36:45.398401+0000","last_peered":"2023-10-26T01:36:45.398401+0000","last_clean":"2023-10-26T01:36:45.398401+0000","last_became_active":"2023-10-26T01:36:43.399976+0000","last_became_peered":"2023-10-26T01:36:43.399976+0000","last_unstale":"2023-10-26T01:36:45.398401+0000","last_undegraded":"2023-10-26T01:36:45.398401+0000","last_fullsized":"2023-10-26T01:36:45.398401+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_clean_scrub_stamp":"2023-10-26T01:36:42.357920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:44:00.571472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":250,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.396338+0000","last_change":"2023-10-26T01:30:34.883484+0000","last_active":"2023-10-26T01:36:44.396338+0000","last_peered":"2023-10-26T01:36:44.396338+0000","last_clean":"2023-10-26T01:36:44.396338+0000","last_became_active":"2023-10-26T01:30:34.883071+0000","last_became_peered":"2023-10-26T01:30:34.883071+0000","last_unstale":"2023-10-26T01:36:44.396338+0000","last_undegraded":"2023-10-26T01:36:44.396338+0000","last_fullsized":"2023-10-26T01:36:44.396338+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:32.890096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:32.890096+0000","last_clean_scrub_stamp":"2023-10-26T01:30:32.890096+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:55:09.809749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":14,"seq":60129542222,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":546,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194765,"statfs":{"total":200000000,"available":199440284,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:35:47 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.80000000000000004,"15min":0.78800000000000003},"min":{"1min":0.35599999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.119,"5min":2.2549999999999999,"15min":2.2549999999999999},"last":0.95299999999999996},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.86299999999999999,"15min":0.89300000000000002},"min":{"1min":0.54600000000000004,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.556,"5min":1.8200000000000001,"15min":1.8200000000000001},"last":0.53200000000000003}]},{"osd":1,"last update":"Thu Oct 26 01:35:47 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.88200000000000001,"15min":0.95899999999999996},"min":{"1min":0.48999999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.3959999999999999,"5min":4.4340000000000002,"15min":4.4340000000000002},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.97699999999999998,"5min":0.97299999999999998,"15min":1.0029999999999999},"min":{"1min":0.54600000000000004,"5min":0.374,"15min":0.374},"max":{"1min":1.377,"5min":2.3330000000000002,"15min":2.3330000000000002},"last":0.56799999999999995}]},{"osd":2,"last update":"Thu Oct 26 01:35:47 2023","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":1.03,"15min":1.087},"min":{"1min":0.52500000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.476,"5min":4.2800000000000002,"15min":4.2800000000000002},"last":1.927},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.1200000000000001,"15min":1.091},"min":{"1min":0.54600000000000004,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.5640000000000001,"5min":2.302,"15min":2.302},"last":0.86799999999999999}]},{"osd":4,"last update":"Thu Oct 26 01:35:47 2023","interfaces":[{"interface":"back","average":{"1min":1.177,"5min":1.1759999999999999,"15min":1.1579999999999999},"min":{"1min":0.84899999999999998,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.4339999999999999,"5min":2.1070000000000002,"15min":2.1070000000000002},"last":1.7150000000000001},{"interface":"front","average":{"1min":1.3240000000000001,"5min":1.3440000000000001,"15min":1.4039999999999999},"min":{"1min":0.97999999999999998,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":1.5620000000000001,"5min":4.8250000000000002,"15min":4.8250000000000002},"last":2.4239999999999999}]},{"osd":5,"last update":"Thu Oct 26 01:35:47 2023","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.2849999999999999,"15min":1.349},"min":{"1min":0.86599999999999999,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.6040000000000001,"5min":4.9080000000000004,"15min":4.9080000000000004},"last":2.3380000000000001},{"interface":"front","average":{"1min":1.24,"5min":1.2549999999999999,"15min":1.272},"min":{"1min":0.79900000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.8899999999999999,"5min":2.8980000000000001,"15min":2.8980000000000001},"last":1.512}]},{"osd":6,"last update":"Thu Oct 26 01:35:47 2023","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.3300000000000001,"15min":1.3240000000000001},"min":{"1min":0.89600000000000002,"5min":0.85099999999999998,"15min":0.85099999999999998},"max":{"1min":1.8240000000000001,"5min":2.8530000000000002,"15min":2.8530000000000002},"last":1.583},{"interface":"front","average":{"1min":1.294,"5min":1.2470000000000001,"15min":1.268},"min":{"1min":0.84699999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.6160000000000001,"5min":2.181,"15min":2.181},"last":1.466}]},{"osd":7,"last update":"Thu Oct 26 01:35:47 2023","interfaces":[{"interface":"back","average":{"1min":1.345,"5min":1.296,"15min":1.2889999999999999},"min":{"1min":0.92000000000000004,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.6879999999999999,"5min":2.0680000000000001,"15min":2.0680000000000001},"last":1.365},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.379,"15min":1.4259999999999999},"min":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":1.875,"5min":4.8730000000000002,"15min":4.8730000000000002},"last":2.2370000000000001}]}]},{"osd":7,"up_from":13,"seq":55834574925,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1121,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194190,"statfs":{"total":200000000,"available":198851458,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:36:45 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.145,"15min":1.091},"min":{"1min":0.69699999999999995,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.6020000000000001,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":1.4630000000000001},{"interface":"front","average":{"1min":1.036,"5min":1.157,"15min":1.085},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.28,"5min":2.6779999999999999,"15min":2.6779999999999999},"last":1.214}]},{"osd":1,"last update":"Thu Oct 26 01:36:45 2023","interfaces":[{"interface":"back","average":{"1min":1.1699999999999999,"5min":1.2050000000000001,"15min":1.091},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.65200000000000002},"max":{"1min":1.6040000000000001,"5min":2.8260000000000001,"15min":2.8260000000000001},"last":1.2849999999999999},{"interface":"front","average":{"1min":1.097,"5min":1.216,"15min":1.1879999999999999},"min":{"1min":0.68799999999999994,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.542,"5min":3.2730000000000001,"15min":3.2730000000000001},"last":0.76300000000000001}]},{"osd":2,"last update":"Thu Oct 26 01:36:45 2023","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.175,"15min":1.1140000000000001},"min":{"1min":0.69099999999999995,"5min":0.61199999999999999,"15min":0.58499999999999996},"max":{"1min":1.8680000000000001,"5min":2.218,"15min":2.218},"last":1.0600000000000001},{"interface":"front","average":{"1min":1.2909999999999999,"5min":1.296,"15min":1.236},"min":{"1min":0.79400000000000004,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.8999999999999999,"5min":3.0510000000000002,"15min":3.0510000000000002},"last":1.3979999999999999}]},{"osd":3,"last update":"Thu Oct 26 01:36:45 2023","interfaces":[{"interface":"back","average":{"1min":1.264,"5min":1.272,"15min":1.208},"min":{"1min":0.80600000000000005,"5min":0.79000000000000004,"15min":0.72099999999999997},"max":{"1min":1.796,"5min":1.9830000000000001,"15min":1.9830000000000001},"last":0.95899999999999996},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.419,"15min":1.377},"min":{"1min":0.82899999999999996,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.8680000000000001,"5min":5.5039999999999996,"15min":5.5039999999999996},"last":1.0129999999999999}]},{"osd":4,"last update":"Thu Oct 26 01:36:45 2023","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.3500000000000001,"15min":1.3460000000000001},"min":{"1min":0.77300000000000002,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.5640000000000001,"5min":2.77,"15min":2.77},"last":1.161},{"interface":"front","average":{"1min":1.298,"5min":1.425,"15min":1.3939999999999999},"min":{"1min":0.96899999999999997,"5min":0.496,"15min":0.496},"max":{"1min":1.6830000000000001,"5min":3.1419999999999999,"15min":3.1419999999999999},"last":1.125}]},{"osd":5,"last update":"Thu Oct 26 01:36:45 2023","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.4710000000000001,"15min":1.419},"min":{"1min":0.65600000000000003,"5min":0.441,"15min":0.441},"max":{"1min":1.746,"5min":5.734,"15min":5.734},"last":0.92100000000000004},{"interface":"front","average":{"1min":1.4339999999999999,"5min":1.5109999999999999,"15min":1.476},"min":{"1min":0.96099999999999997,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.829,"5min":2.8980000000000001,"15min":2.8980000000000001},"last":1.25}]},{"osd":6,"last update":"Thu Oct 26 01:36:45 2023","interfaces":[{"interface":"back","average":{"1min":1.4670000000000001,"5min":1.603,"15min":1.877},"min":{"1min":0.753,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.7430000000000001,"5min":5.6340000000000003,"15min":11.577999999999999},"last":1.3180000000000001},{"interface":"front","average":{"1min":1.552,"5min":1.661,"15min":1.8879999999999999},"min":{"1min":0.91800000000000004,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.9339999999999999,"5min":5.0919999999999996,"15min":10.704000000000001},"last":1.361}]}]},{"osd":6,"up_from":13,"seq":55834574925,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":538,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194773,"statfs":{"total":200000000,"available":199448116,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:36:42 2023","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.097,"15min":1.1599999999999999},"min":{"1min":0.61699999999999999,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.647,"5min":2.395,"15min":2.395},"last":0.877},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.1299999999999999,"15min":1.1200000000000001},"min":{"1min":0.72899999999999998,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.7390000000000001,"5min":1.7390000000000001,"15min":1.843},"last":1.7390000000000001}]},{"osd":1,"last update":"Thu Oct 26 01:36:42 2023","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.1639999999999999,"15min":1.1719999999999999},"min":{"1min":0.74399999999999999,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.784,"5min":2.0529999999999999,"15min":2.0529999999999999},"last":1.1180000000000001},{"interface":"front","average":{"1min":1.139,"5min":1.1519999999999999,"15min":1.2010000000000001},"min":{"1min":0.79000000000000004,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.5229999999999999,"5min":1.581,"15min":2.085},"last":1.4379999999999999}]},{"osd":2,"last update":"Thu Oct 26 01:36:42 2023","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.1779999999999999,"15min":1.0940000000000001},"min":{"1min":0.83099999999999996,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.7949999999999999,"5min":3.7879999999999998,"15min":3.7879999999999998},"last":1.3580000000000001},{"interface":"front","average":{"1min":1.319,"5min":1.2829999999999999,"15min":1.2589999999999999},"min":{"1min":0.85799999999999998,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.9910000000000001,"5min":2.2890000000000001,"15min":2.2890000000000001},"last":1.224}]},{"osd":3,"last update":"Thu Oct 26 01:36:42 2023","interfaces":[{"interface":"back","average":{"1min":1.391,"5min":1.3080000000000001,"15min":1.2070000000000001},"min":{"1min":0.871,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.8600000000000001,"5min":4.4560000000000004,"15min":4.4560000000000004},"last":1.595},{"interface":"front","average":{"1min":1.4119999999999999,"5min":1.341,"15min":1.327},"min":{"1min":1.0369999999999999,"5min":0.82299999999999995,"15min":0.69499999999999995},"max":{"1min":1.877,"5min":2.5019999999999998,"15min":2.5019999999999998},"last":1.3089999999999999}]},{"osd":4,"last update":"Thu Oct 26 01:36:42 2023","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.252,"15min":1.22},"min":{"1min":0.93200000000000005,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.7,"5min":2.9489999999999998,"15min":2.9489999999999998},"last":0.99399999999999999},{"interface":"front","average":{"1min":1.4339999999999999,"5min":1.4330000000000001,"15min":1.3140000000000001},"min":{"1min":0.93400000000000005,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.9039999999999999,"5min":6.609,"15min":6.609},"last":1.5109999999999999}]},{"osd":5,"last update":"Thu Oct 26 01:36:42 2023","interfaces":[{"interface":"back","average":{"1min":1.6339999999999999,"5min":1.548,"15min":1.4430000000000001},"min":{"1min":1.2030000000000001,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":2.1240000000000001,"5min":5.4370000000000003,"15min":5.4370000000000003},"last":1.6950000000000001},{"interface":"front","average":{"1min":1.5309999999999999,"5min":1.448,"15min":1.409},"min":{"1min":1.1319999999999999,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.8169999999999999,"5min":2.621,"15min":2.621},"last":1.403}]},{"osd":7,"last update":"Thu Oct 26 01:36:42 2023","interfaces":[{"interface":"back","average":{"1min":1.4650000000000001,"5min":1.5229999999999999,"15min":1.411},"min":{"1min":1.2589999999999999,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.9770000000000001,"5min":11.804,"15min":11.804},"last":1.6439999999999999},{"interface":"front","average":{"1min":1.6299999999999999,"5min":1.657,"15min":1.5049999999999999},"min":{"1min":1.296,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.992,"5min":11.541,"15min":11.541},"last":1.79}]}]},{"osd":5,"up_from":13,"seq":55834574925,"num_pgs":16,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":540,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194772,"statfs":{"total":200000000,"available":199446562,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:36:44 2023","interfaces":[{"interface":"back","average":{"1min":1.143,"5min":1.1739999999999999,"15min":1.1240000000000001},"min":{"1min":0.746,"5min":0.66200000000000003,"15min":0.63200000000000001},"max":{"1min":1.546,"5min":1.7390000000000001,"15min":1.7390000000000001},"last":1.042},{"interface":"front","average":{"1min":1.262,"5min":1.2709999999999999,"15min":1.177},"min":{"1min":0.81699999999999995,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.601,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":1.0980000000000001}]},{"osd":1,"last update":"Thu Oct 26 01:36:44 2023","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.1439999999999999,"15min":1.091},"min":{"1min":0.80700000000000005,"5min":0.67600000000000005,"15min":0.66400000000000003},"max":{"1min":1.7829999999999999,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":0.90600000000000003},{"interface":"front","average":{"1min":1.1040000000000001,"5min":1.1599999999999999,"15min":1.095},"min":{"1min":0.85799999999999998,"5min":0.68200000000000005,"15min":0.60599999999999998},"max":{"1min":1.4570000000000001,"5min":1.6000000000000001,"15min":1.7669999999999999},"last":1.341}]},{"osd":2,"last update":"Thu Oct 26 01:36:44 2023","interfaces":[{"interface":"back","average":{"1min":1.2290000000000001,"5min":1.204,"15min":1.1619999999999999},"min":{"1min":0.79600000000000004,"5min":0.74399999999999999,"15min":0.69299999999999995},"max":{"1min":1.6879999999999999,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":0.79600000000000004},{"interface":"front","average":{"1min":1.3320000000000001,"5min":1.2350000000000001,"15min":1.1850000000000001},"min":{"1min":0.93600000000000005,"5min":0.85499999999999998,"15min":0.63900000000000001},"max":{"1min":1.6899999999999999,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":1.1930000000000001}]},{"osd":3,"last update":"Thu Oct 26 01:36:44 2023","interfaces":[{"interface":"back","average":{"1min":1.2809999999999999,"5min":1.3080000000000001,"15min":1.246},"min":{"1min":0.92200000000000004,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.613,"5min":1.7649999999999999,"15min":1.835},"last":1.462},{"interface":"front","average":{"1min":1.373,"5min":1.321,"15min":1.2949999999999999},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.788,"5min":1.788,"15min":2.9249999999999998},"last":1.298}]},{"osd":4,"last update":"Thu Oct 26 01:36:44 2023","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.286,"15min":1.206},"min":{"1min":0.97699999999999998,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.845,"5min":1.845,"15min":1.845},"last":1.6559999999999999},{"interface":"front","average":{"1min":1.4810000000000001,"5min":1.4470000000000001,"15min":1.353},"min":{"1min":0.90600000000000003,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.7849999999999999,"5min":1.867,"15min":1.867},"last":1.5840000000000001}]},{"osd":6,"last update":"Thu Oct 26 01:36:44 2023","interfaces":[{"interface":"back","average":{"1min":1.5269999999999999,"5min":1.444,"15min":1.3220000000000001},"min":{"1min":1.0660000000000001,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.883,"5min":2.8250000000000002,"15min":2.8250000000000002},"last":1.6080000000000001},{"interface":"front","average":{"1min":1.5429999999999999,"5min":1.4490000000000001,"15min":1.3260000000000001},"min":{"1min":1.1879999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.849,"5min":1.849,"15min":1.8560000000000001},"last":1.5469999999999999}]},{"osd":7,"last update":"Thu Oct 26 01:36:44 2023","interfaces":[{"interface":"back","average":{"1min":1.599,"5min":1.5449999999999999,"15min":1.417},"min":{"1min":1.119,"5min":0.64000000000000001,"15min":0.61699999999999999},"max":{"1min":2.1040000000000001,"5min":2.1040000000000001,"15min":2.1040000000000001},"last":1.415},{"interface":"front","average":{"1min":1.6619999999999999,"5min":1.5660000000000001,"15min":1.4299999999999999},"min":{"1min":1.389,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":2.0960000000000001,"5min":2.0960000000000001,"15min":2.0960000000000001},"last":1.7090000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574925,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":545,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194767,"statfs":{"total":200000000,"available":199441896,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:36:37 2023","interfaces":[{"interface":"back","average":{"1min":1.081,"5min":1.04,"15min":0.97699999999999998},"min":{"1min":0.69199999999999995,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.5309999999999999,"5min":2.0289999999999999,"15min":2.0289999999999999},"last":1.3089999999999999},{"interface":"front","average":{"1min":1.071,"5min":1.0649999999999999,"15min":1.0229999999999999},"min":{"1min":0.66400000000000003,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.6599999999999999,"5min":2.6829999999999998,"15min":2.6829999999999998},"last":1.5029999999999999}]},{"osd":1,"last update":"Thu Oct 26 01:36:37 2023","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.1799999999999999,"15min":1.1220000000000001},"min":{"1min":0.98399999999999999,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.599,"5min":4.0519999999999996,"15min":4.0519999999999996},"last":1.1180000000000001},{"interface":"front","average":{"1min":1.119,"5min":1.1439999999999999,"15min":1.105},"min":{"1min":0.77900000000000003,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.6240000000000001,"5min":2.0760000000000001,"15min":2.0760000000000001},"last":1.4470000000000001}]},{"osd":2,"last update":"Thu Oct 26 01:36:37 2023","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":1.3280000000000001,"15min":1.222},"min":{"1min":0.79100000000000004,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.5660000000000001,"5min":5.6749999999999998,"15min":5.6749999999999998},"last":1.6819999999999999},{"interface":"front","average":{"1min":1.1970000000000001,"5min":1.1879999999999999,"15min":1.1220000000000001},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.746,"5min":1.849,"15min":1.849},"last":1.0700000000000001}]},{"osd":3,"last update":"Thu Oct 26 01:36:37 2023","interfaces":[{"interface":"back","average":{"1min":1.286,"5min":1.2,"15min":1.1839999999999999},"min":{"1min":0.84399999999999997,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.847,"5min":1.891,"15min":1.891},"last":1.5449999999999999},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.2030000000000001,"15min":1.155},"min":{"1min":0.94499999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.5669999999999999,"5min":1.7210000000000001,"15min":1.7210000000000001},"last":1.391}]},{"osd":5,"last update":"Thu Oct 26 01:36:37 2023","interfaces":[{"interface":"back","average":{"1min":1.2749999999999999,"5min":1.2509999999999999,"15min":1.286},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.46300000000000002},"max":{"1min":2.1429999999999998,"5min":2.1429999999999998,"15min":2.1429999999999998},"last":1.196},{"interface":"front","average":{"1min":1.28,"5min":1.331,"15min":1.327},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.716,"5min":5.8579999999999997,"15min":5.8579999999999997},"last":1.7529999999999999}]},{"osd":6,"last update":"Thu Oct 26 01:36:37 2023","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":1.4199999999999999,"15min":1.3340000000000001},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.639,"5min":12.343999999999999,"15min":12.343999999999999},"last":1.2629999999999999},{"interface":"front","average":{"1min":1.54,"5min":1.6220000000000001,"15min":1.5129999999999999},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.55000000000000004},"max":{"1min":2.0369999999999999,"5min":12.605,"15min":12.605},"last":1.6379999999999999}]},{"osd":7,"last update":"Thu Oct 26 01:36:37 2023","interfaces":[{"interface":"back","average":{"1min":1.4650000000000001,"5min":1.518,"15min":1.4910000000000001},"min":{"1min":0.748,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":2.762,"5min":5.774,"15min":5.774},"last":1.363},{"interface":"front","average":{"1min":1.5249999999999999,"5min":1.518,"15min":1.4530000000000001},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":2.8330000000000002,"5min":4.6520000000000001,"15min":4.6520000000000001},"last":1.5840000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574925,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":539,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194773,"statfs":{"total":200000000,"available":199447614,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":0.875,"15min":0.86099999999999999},"min":{"1min":0.63,"5min":0.308,"15min":0.308},"max":{"1min":1.5249999999999999,"5min":6.3200000000000003,"15min":6.3200000000000003},"last":0.90700000000000003},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.0229999999999999,"15min":0.93600000000000005},"min":{"1min":0.52000000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.7130000000000001,"5min":4.2619999999999996,"15min":4.2619999999999996},"last":0.77600000000000002}]},{"osd":1,"last update":"Thu Oct 26 01:36:41 2023","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":0.93200000000000005,"15min":0.90500000000000003},"min":{"1min":0.66700000000000004,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.629,"5min":5.1260000000000003,"15min":5.1260000000000003},"last":1.0309999999999999},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.0649999999999999,"15min":1.012},"min":{"1min":0.59599999999999997,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.873,"5min":6.1520000000000001,"15min":6.1520000000000001},"last":1.109}]},{"osd":3,"last update":"Thu Oct 26 01:35:43 2023","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.98699999999999999,"15min":0.92500000000000004},"min":{"1min":0.55200000000000005,"5min":0.443,"15min":0.443},"max":{"1min":1.3600000000000001,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":1.149},{"interface":"front","average":{"1min":1.1080000000000001,"5min":1.115,"15min":1.0009999999999999},"min":{"1min":0.624,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.577,"5min":8.9019999999999992,"15min":8.9019999999999992},"last":1.27}]},{"osd":4,"last update":"Thu Oct 26 01:36:41 2023","interfaces":[{"interface":"back","average":{"1min":1.454,"5min":1.365,"15min":1.268},"min":{"1min":0.80500000000000005,"5min":0.751,"15min":0.73899999999999999},"max":{"1min":2.6190000000000002,"5min":6.2670000000000003,"15min":6.2670000000000003},"last":1.3620000000000001},{"interface":"front","average":{"1min":1.5089999999999999,"5min":1.367,"15min":1.2629999999999999},"min":{"1min":0.93700000000000006,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":2.4359999999999999,"5min":7.343,"15min":7.343},"last":1.399}]},{"osd":5,"last update":"Thu Oct 26 01:36:41 2023","interfaces":[{"interface":"back","average":{"1min":1.5649999999999999,"5min":1.369,"15min":1.2909999999999999},"min":{"1min":1.018,"5min":0.80800000000000005,"15min":0.71899999999999997},"max":{"1min":2.5790000000000002,"5min":3.7170000000000001,"15min":3.7170000000000001},"last":1.206},{"interface":"front","average":{"1min":1.4650000000000001,"5min":1.3540000000000001,"15min":1.3},"min":{"1min":0.88800000000000001,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":2.298,"5min":6.3899999999999997,"15min":6.3899999999999997},"last":1.2969999999999999}]},{"osd":6,"last update":"Thu Oct 26 01:36:41 2023","interfaces":[{"interface":"back","average":{"1min":1.4690000000000001,"5min":1.4259999999999999,"15min":1.335},"min":{"1min":0.871,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":2.73,"5min":6.0490000000000004,"15min":6.0490000000000004},"last":1.4690000000000001},{"interface":"front","average":{"1min":1.599,"5min":1.4399999999999999,"15min":1.375},"min":{"1min":1.0660000000000001,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":2.4089999999999998,"5min":5.0650000000000004,"15min":5.0650000000000004},"last":1.327}]},{"osd":7,"last update":"Thu Oct 26 01:36:41 2023","interfaces":[{"interface":"back","average":{"1min":1.532,"5min":1.4390000000000001,"15min":1.405},"min":{"1min":1.0449999999999999,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":2.3370000000000002,"5min":6.4480000000000004,"15min":6.4480000000000004},"last":1.226},{"interface":"front","average":{"1min":1.6279999999999999,"5min":1.5169999999999999,"15min":1.4570000000000001},"min":{"1min":1.117,"5min":0.91800000000000004,"15min":0.91800000000000004},"max":{"1min":2.8290000000000002,"5min":2.8439999999999999,"15min":2.8439999999999999},"last":1.5620000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574925,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":539,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194773,"statfs":{"total":200000000,"available":199448002,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:35:40 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.76100000000000001,"15min":0.71399999999999997},"min":{"1min":0.38,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.147,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.872,"5min":0.82099999999999995,"15min":0.78300000000000003},"min":{"1min":0.40899999999999997,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.6399999999999999,"5min":1.6399999999999999,"15min":1.6399999999999999},"last":0.36299999999999999}]},{"osd":2,"last update":"Thu Oct 26 01:35:40 2023","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.95699999999999996,"15min":0.997},"min":{"1min":0.42899999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.4970000000000001,"5min":2.4830000000000001,"15min":2.4830000000000001},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.86099999999999999,"15min":0.84899999999999998},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.296,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.64300000000000002}]},{"osd":3,"last update":"Thu Oct 26 01:35:46 2023","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.96299999999999997,"15min":0.93999999999999995},"min":{"1min":0.44700000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.194,"5min":1.75,"15min":1.75},"last":0.42499999999999999},{"interface":"front","average":{"1min":1.0189999999999999,"5min":1.004,"15min":0.98099999999999998},"min":{"1min":0.48799999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.361,"5min":1.802,"15min":1.802},"last":0.625}]},{"osd":4,"last update":"Thu Oct 26 01:35:40 2023","interfaces":[{"interface":"back","average":{"1min":1.177,"5min":1.2130000000000001,"15min":1.1919999999999999},"min":{"1min":0.80800000000000005,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.5069999999999999,"5min":2.0659999999999998,"15min":2.0659999999999998},"last":0.69099999999999995},{"interface":"front","average":{"1min":1.099,"5min":1.151,"15min":1.1220000000000001},"min":{"1min":0.76600000000000001,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.5920000000000001,"5min":1.637,"15min":1.637},"last":0.88200000000000001}]},{"osd":5,"last update":"Thu Oct 26 01:35:40 2023","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.238,"15min":1.224},"min":{"1min":0.94799999999999995,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":1.746,"5min":1.746,"15min":1.746},"last":1.1359999999999999},{"interface":"front","average":{"1min":1.3740000000000001,"5min":1.3400000000000001,"15min":1.3320000000000001},"min":{"1min":0.78800000000000003,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.772,"5min":2.2200000000000002,"15min":2.2200000000000002},"last":0.80300000000000005}]},{"osd":6,"last update":"Thu Oct 26 01:35:40 2023","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":1.2969999999999999,"15min":1.2709999999999999},"min":{"1min":0.90000000000000002,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.8340000000000001,"5min":2.3780000000000001,"15min":2.3780000000000001},"last":0.70199999999999996},{"interface":"front","average":{"1min":1.246,"5min":1.256,"15min":1.236},"min":{"1min":0.82599999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.6060000000000001,"5min":1.984,"15min":1.984},"last":0.89600000000000002}]},{"osd":7,"last update":"Thu Oct 26 01:35:40 2023","interfaces":[{"interface":"back","average":{"1min":1.2410000000000001,"5min":1.244,"15min":1.262},"min":{"1min":0.84199999999999997,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.798,"5min":2.6930000000000001,"15min":2.6930000000000001},"last":1.1779999999999999},{"interface":"front","average":{"1min":1.357,"5min":1.351,"15min":1.319},"min":{"1min":0.81399999999999995,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.7310000000000001,"5min":1.869,"15min":1.869},"last":0.95299999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574925,"num_pgs":8,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1115,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194197,"statfs":{"total":200000000,"available":198857794,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Oct 26 01:36:43 2023","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.998,"15min":0.92300000000000004},"min":{"1min":0.64000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.49,"5min":19.425000000000001,"15min":19.425000000000001},"last":1.1319999999999999},{"interface":"front","average":{"1min":0.98199999999999998,"5min":1.075,"15min":1.0549999999999999},"min":{"1min":0.67400000000000004,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.339,"5min":15.162000000000001,"15min":15.162000000000001},"last":0.86599999999999999}]},{"osd":2,"last update":"Thu Oct 26 01:36:43 2023","interfaces":[{"interface":"back","average":{"1min":1,"5min":1.2969999999999999,"15min":1.1930000000000001},"min":{"1min":0.59899999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.4970000000000001,"5min":28.439,"15min":28.439},"last":1.399},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.90800000000000003,"15min":0.91400000000000003},"min":{"1min":0.69399999999999995,"5min":0.45200000000000001,"15min":0.38300000000000001},"max":{"1min":1.268,"5min":1.502,"15min":4.9459999999999997},"last":0.98199999999999998}]},{"osd":3,"last update":"Thu Oct 26 01:36:43 2023","interfaces":[{"interface":"back","average":{"1min":1.101,"5min":1.0469999999999999,"15min":1.093},"min":{"1min":0.64100000000000001,"5min":0.46000000000000002,"15min":0.44600000000000001},"max":{"1min":1.47,"5min":1.867,"15min":4.9800000000000004},"last":1.095},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.1359999999999999,"15min":1.1619999999999999},"min":{"1min":0.752,"5min":0.56699999999999995,"15min":0.54000000000000004},"max":{"1min":1.4450000000000001,"5min":1.992,"15min":5.016},"last":1.1859999999999999}]},{"osd":4,"last update":"Thu Oct 26 01:36:43 2023","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.387,"15min":1.3009999999999999},"min":{"1min":0.82799999999999996,"5min":0.66800000000000004,"15min":0.61499999999999999},"max":{"1min":1.681,"5min":19.329000000000001,"15min":19.329000000000001},"last":1.359},{"interface":"front","average":{"1min":1.319,"5min":1.27,"15min":1.306},"min":{"1min":0.82499999999999996,"5min":0.79100000000000004,"15min":0.67600000000000005},"max":{"1min":1.718,"5min":2.036,"15min":5.5229999999999997},"last":1.3049999999999999}]},{"osd":5,"last update":"Thu Oct 26 01:36:43 2023","interfaces":[{"interface":"back","average":{"1min":1.5129999999999999,"5min":1.4039999999999999,"15min":1.4350000000000001},"min":{"1min":1.1359999999999999,"5min":0.80000000000000004,"15min":0.72499999999999998},"max":{"1min":1.857,"5min":2.2290000000000001,"15min":5.6360000000000001},"last":1.595},{"interface":"front","average":{"1min":1.399,"5min":1.492,"15min":1.4119999999999999},"min":{"1min":1.0880000000000001,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.8200000000000001,"5min":19.384,"15min":19.384},"last":1.643}]},{"osd":6,"last update":"Thu Oct 26 01:36:43 2023","interfaces":[{"interface":"back","average":{"1min":1.3580000000000001,"5min":1.375,"15min":1.3720000000000001},"min":{"1min":1.034,"5min":0.77200000000000002,"15min":0.73799999999999999},"max":{"1min":1.6970000000000001,"5min":8.2609999999999992,"15min":8.2609999999999992},"last":1.5089999999999999},{"interface":"front","average":{"1min":1.377,"5min":1.4370000000000001,"15min":1.4099999999999999},"min":{"1min":0.85899999999999999,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.9450000000000001,"5min":11.849,"15min":11.849},"last":1.4730000000000001}]},{"osd":7,"last update":"Thu Oct 26 01:36:43 2023","interfaces":[{"interface":"back","average":{"1min":1.4970000000000001,"5min":1.4450000000000001,"15min":1.488},"min":{"1min":0.999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.7809999999999999,"5min":2.4060000000000001,"15min":5.6689999999999996},"last":1.4430000000000001},{"interface":"front","average":{"1min":1.4930000000000001,"5min":1.6080000000000001,"15min":1.5089999999999999},"min":{"1min":1.1579999999999999,"5min":0.81299999999999994,"15min":0.69399999999999995},"max":{"1min":1.9390000000000001,"5min":19.245999999999999,"15min":19.245999999999999},"last":1.7110000000000001}]}]}],"pool_statfs":[]}} 2023-10-26T01:36:48.870 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-26T01:36:49.135 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:49.133+0000 7f8588859640 1 -- 172.21.15.169:0/2457822368 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f85840c9e00 msgr2=0x7f85840c7ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:49.136 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:49.133+0000 7f8588859640 1 --2- 172.21.15.169:0/2457822368 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f85840c9e00 0x7f85840c7ea0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f856c009830 tx=0x7f856c02bcd0 comp rx=0 tx=0).stop 2023-10-26T01:36:49.136 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:49.133+0000 7f8588859640 1 -- 172.21.15.169:0/2457822368 shutdown_connections 2023-10-26T01:36:49.136 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:49.133+0000 7f8588859640 1 --2- 172.21.15.169:0/2457822368 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f85840c9e00 0x7f85840c7ea0 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:49.136 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:49.134+0000 7f8588859640 1 -- 172.21.15.169:0/2457822368 >> 172.21.15.169:0/2457822368 conn(0x7f85840aedd0 msgr2=0x7f85840af1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:49.136 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:49.134+0000 7f8588859640 1 -- 172.21.15.169:0/2457822368 shutdown_connections 2023-10-26T01:36:49.137 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:49.134+0000 7f8588859640 1 -- 172.21.15.169:0/2457822368 wait complete. 2023-10-26T01:36:49.137 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:49.135+0000 7f8588859640 1 Processor -- start 2023-10-26T01:36:49.137 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:49.135+0000 7f8588859640 1 -- start start 2023-10-26T01:36:49.139 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:49.136+0000 7f8588859640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f858406a990 0x7f8584143e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:49.140 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:49.136+0000 7f8588859640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f858406cae0 0x7f85840afce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:49.140 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:49.136+0000 7f8588859640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f85840c9e00 0x7f85840b27c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:49.140 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:49.136+0000 7f8588859640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85840b0220 con 0x7f858406a990 2023-10-26T01:36:49.140 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:49.136+0000 7f8588859640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85840b1a00 con 0x7f858406cae0 2023-10-26T01:36:49.141 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:49.136+0000 7f8588859640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85840b1ce0 con 0x7f85840c9e00 2023-10-26T01:36:49.141 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:49.137+0000 7f85837fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f858406a990 0x7f8584143e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:49.141 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:49.137+0000 7f85837fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f858406a990 0x7f8584143e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:59512/0 (socket says 172.21.15.169:59512) 2023-10-26T01:36:49.141 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:49.137+0000 7f85837fe640 1 -- 172.21.15.169:0/3274741087 learned_addr learned my addr 172.21.15.169:0/3274741087 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:36:49.141 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:49.137+0000 7f857bfff640 1 --2- 172.21.15.169:0/3274741087 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f858406cae0 0x7f85840afce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:49.142 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:49.137+0000 7f8583fff640 1 --2- 172.21.15.169:0/3274741087 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f85840c9e00 0x7f85840b27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:49.142 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:49.137+0000 7f85837fe640 1 -- 172.21.15.169:0/3274741087 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f85840c9e00 msgr2=0x7f85840b27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:49.142 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:49.137+0000 7f85837fe640 1 --2- 172.21.15.169:0/3274741087 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f85840c9e00 0x7f85840b27c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:49.142 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:49.137+0000 7f85837fe640 1 -- 172.21.15.169:0/3274741087 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f858406cae0 msgr2=0x7f85840afce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:49.142 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:49.137+0000 7f85837fe640 1 --2- 172.21.15.169:0/3274741087 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f858406cae0 0x7f85840afce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:49.143 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:49.137+0000 7f85837fe640 1 -- 172.21.15.169:0/3274741087 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f856c009490 con 0x7f858406a990 2023-10-26T01:36:49.143 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:49.137+0000 7f857bfff640 1 --2- 172.21.15.169:0/3274741087 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f858406cae0 0x7f85840afce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:36:49.143 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:49.137+0000 7f85837fe640 1 --2- 172.21.15.169:0/3274741087 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f858406a990 0x7f8584143e50 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f857400c490 tx=0x7f857400c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:49.143 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:49.138+0000 7f8580ff9640 1 -- 172.21.15.169:0/3274741087 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8574012830 con 0x7f858406a990 2023-10-26T01:36:49.143 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:49.138+0000 7f8580ff9640 1 -- 172.21.15.169:0/3274741087 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8574012a80 con 0x7f858406a990 2023-10-26T01:36:49.144 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:49.138+0000 7f8580ff9640 1 -- 172.21.15.169:0/3274741087 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8574011380 con 0x7f858406a990 2023-10-26T01:36:49.144 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:49.138+0000 7f8588859640 1 -- 172.21.15.169:0/3274741087 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85840b1fc0 con 0x7f858406a990 2023-10-26T01:36:49.144 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:49.138+0000 7f8588859640 1 -- 172.21.15.169:0/3274741087 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85840b2d00 con 0x7f858406a990 2023-10-26T01:36:49.145 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:49.139+0000 7f8588859640 1 -- 172.21.15.169:0/3274741087 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85840b24e0 con 0x7f858406a990 2023-10-26T01:36:49.145 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:49.140+0000 7f8580ff9640 1 -- 172.21.15.169:0/3274741087 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f85740114e0 con 0x7f858406a990 2023-10-26T01:36:49.145 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:49.141+0000 7f8580ff9640 1 --2- 172.21.15.169:0/3274741087 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f856406ae40 0x7f856406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:49.146 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:49.141+0000 7f8580ff9640 1 -- 172.21.15.169:0/3274741087 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f85740897f0 con 0x7f858406a990 2023-10-26T01:36:49.146 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:49.142+0000 7f857bfff640 1 --2- 172.21.15.169:0/3274741087 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f856406ae40 0x7f856406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:49.146 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:49.143+0000 7f857bfff640 1 --2- 172.21.15.169:0/3274741087 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f856406ae40 0x7f856406d300 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f856c036000 tx=0x7f856c004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:49.147 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:49.145+0000 7f8580ff9640 1 -- 172.21.15.169:0/3274741087 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f85840b24e0 con 0x7f858406a990 2023-10-26T01:36:49.333 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:49.332+0000 7f8588859640 1 -- 172.21.15.169:0/3274741087 --> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f85840b24e0 con 0x7f856406ae40 2023-10-26T01:36:49.356 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:49.354+0000 7f8580ff9640 1 -- 172.21.15.169:0/3274741087 <== mgr.4112 v2:172.21.15.170:6832/1187156918 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+283113 (secure 0 0 0) 0x7f85840b24e0 con 0x7f856406ae40 2023-10-26T01:36:49.358 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:36:49.361 INFO:teuthology.orchestra.run.smithi169.stderr:dumped all 2023-10-26T01:36:49.362 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:49.361+0000 7f8588859640 1 -- 172.21.15.169:0/3274741087 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f856406ae40 msgr2=0x7f856406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:49.363 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:49.361+0000 7f8588859640 1 --2- 172.21.15.169:0/3274741087 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f856406ae40 0x7f856406d300 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f856c036000 tx=0x7f856c004730 comp rx=0 tx=0).stop 2023-10-26T01:36:49.363 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:49.361+0000 7f8588859640 1 -- 172.21.15.169:0/3274741087 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f858406a990 msgr2=0x7f8584143e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:49.363 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:49.361+0000 7f8588859640 1 --2- 172.21.15.169:0/3274741087 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f858406a990 0x7f8584143e50 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f857400c490 tx=0x7f857400c950 comp rx=0 tx=0).stop 2023-10-26T01:36:49.363 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:49.361+0000 7f8588859640 1 -- 172.21.15.169:0/3274741087 shutdown_connections 2023-10-26T01:36:49.364 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:49.361+0000 7f8588859640 1 --2- 172.21.15.169:0/3274741087 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f856406ae40 0x7f856406d300 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:49.364 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:49.361+0000 7f8588859640 1 --2- 172.21.15.169:0/3274741087 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f85840c9e00 0x7f85840b27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:49.364 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:49.361+0000 7f8588859640 1 --2- 172.21.15.169:0/3274741087 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f858406cae0 0x7f85840afce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:49.364 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:49.361+0000 7f8588859640 1 --2- 172.21.15.169:0/3274741087 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f858406a990 0x7f8584143e50 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:49.364 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:49.361+0000 7f8588859640 1 -- 172.21.15.169:0/3274741087 >> 172.21.15.169:0/3274741087 conn(0x7f85840aedd0 msgr2=0x7f85840cb390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:49.365 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:49.361+0000 7f8588859640 1 -- 172.21.15.169:0/3274741087 shutdown_connections 2023-10-26T01:36:49.365 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:49.361+0000 7f8588859640 1 -- 172.21.15.169:0/3274741087 wait complete. 2023-10-26T01:36:49.381 INFO:teuthology.orchestra.run.smithi169.stdout:{"pg_ready":true,"pg_map":{"version":285,"stamp":"2023-10-26T01:36:48.261662+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":5486,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1557013,"statfs":{"total":1600000000,"available":1594381726,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.887873"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.414212+0000","last_change":"2023-10-26T01:36:44.414114+0000","last_active":"2023-10-26T01:36:44.414212+0000","last_peered":"2023-10-26T01:36:44.414212+0000","last_clean":"2023-10-26T01:36:44.414212+0000","last_became_active":"2023-10-26T01:36:44.413833+0000","last_became_peered":"2023-10-26T01:36:44.413833+0000","last_unstale":"2023-10-26T01:36:44.414212+0000","last_undegraded":"2023-10-26T01:36:44.414212+0000","last_fullsized":"2023-10-26T01:36:44.414212+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:07:55.539688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.424610+0000","last_change":"2023-10-26T01:36:44.423269+0000","last_active":"2023-10-26T01:36:44.424610+0000","last_peered":"2023-10-26T01:36:44.424610+0000","last_clean":"2023-10-26T01:36:44.424610+0000","last_became_active":"2023-10-26T01:36:44.421972+0000","last_became_peered":"2023-10-26T01:36:44.421972+0000","last_unstale":"2023-10-26T01:36:44.424610+0000","last_undegraded":"2023-10-26T01:36:44.424610+0000","last_fullsized":"2023-10-26T01:36:44.424610+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:46:28.059120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T01:36:45.397183+0000","last_change":"2023-10-26T01:36:44.417152+0000","last_active":"2023-10-26T01:36:45.397183+0000","last_peered":"2023-10-26T01:36:45.397183+0000","last_clean":"2023-10-26T01:36:45.397183+0000","last_became_active":"2023-10-26T01:36:44.415589+0000","last_became_peered":"2023-10-26T01:36:44.415589+0000","last_unstale":"2023-10-26T01:36:45.397183+0000","last_undegraded":"2023-10-26T01:36:45.397183+0000","last_fullsized":"2023-10-26T01:36:45.397183+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:23:09.299929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.431809+0000","last_change":"2023-10-26T01:36:44.431733+0000","last_active":"2023-10-26T01:36:44.431809+0000","last_peered":"2023-10-26T01:36:44.431809+0000","last_clean":"2023-10-26T01:36:44.431809+0000","last_became_active":"2023-10-26T01:36:44.429537+0000","last_became_peered":"2023-10-26T01:36:44.429537+0000","last_unstale":"2023-10-26T01:36:44.431809+0000","last_undegraded":"2023-10-26T01:36:44.431809+0000","last_fullsized":"2023-10-26T01:36:44.431809+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:07:08.053814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T01:36:45.398114+0000","last_change":"2023-10-26T01:36:44.412599+0000","last_active":"2023-10-26T01:36:45.398114+0000","last_peered":"2023-10-26T01:36:45.398114+0000","last_clean":"2023-10-26T01:36:45.398114+0000","last_became_active":"2023-10-26T01:36:44.412261+0000","last_became_peered":"2023-10-26T01:36:44.412261+0000","last_unstale":"2023-10-26T01:36:45.398114+0000","last_undegraded":"2023-10-26T01:36:45.398114+0000","last_fullsized":"2023-10-26T01:36:45.398114+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:40:43.052815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T01:36:45.395272+0000","last_change":"2023-10-26T01:36:44.426899+0000","last_active":"2023-10-26T01:36:45.395272+0000","last_peered":"2023-10-26T01:36:45.395272+0000","last_clean":"2023-10-26T01:36:45.395272+0000","last_became_active":"2023-10-26T01:36:44.426565+0000","last_became_peered":"2023-10-26T01:36:44.426565+0000","last_unstale":"2023-10-26T01:36:45.395272+0000","last_undegraded":"2023-10-26T01:36:45.395272+0000","last_fullsized":"2023-10-26T01:36:45.395272+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:39:53.140736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.418498+0000","last_change":"2023-10-26T01:36:44.417812+0000","last_active":"2023-10-26T01:36:44.418498+0000","last_peered":"2023-10-26T01:36:44.418498+0000","last_clean":"2023-10-26T01:36:44.418498+0000","last_became_active":"2023-10-26T01:36:44.417384+0000","last_became_peered":"2023-10-26T01:36:44.417384+0000","last_unstale":"2023-10-26T01:36:44.418498+0000","last_undegraded":"2023-10-26T01:36:44.418498+0000","last_fullsized":"2023-10-26T01:36:44.418498+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:00:56.306188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T01:36:45.394850+0000","last_change":"2023-10-26T01:36:44.415752+0000","last_active":"2023-10-26T01:36:45.394850+0000","last_peered":"2023-10-26T01:36:45.394850+0000","last_clean":"2023-10-26T01:36:45.394850+0000","last_became_active":"2023-10-26T01:36:44.415073+0000","last_became_peered":"2023-10-26T01:36:44.415073+0000","last_unstale":"2023-10-26T01:36:45.394850+0000","last_undegraded":"2023-10-26T01:36:45.394850+0000","last_fullsized":"2023-10-26T01:36:45.394850+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:17:01.708641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T01:36:45.394671+0000","last_change":"2023-10-26T01:36:44.414150+0000","last_active":"2023-10-26T01:36:45.394671+0000","last_peered":"2023-10-26T01:36:45.394671+0000","last_clean":"2023-10-26T01:36:45.394671+0000","last_became_active":"2023-10-26T01:36:44.413892+0000","last_became_peered":"2023-10-26T01:36:44.413892+0000","last_unstale":"2023-10-26T01:36:45.394671+0000","last_undegraded":"2023-10-26T01:36:45.394671+0000","last_fullsized":"2023-10-26T01:36:45.394671+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:47:35.769327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.421024+0000","last_change":"2023-10-26T01:36:44.420947+0000","last_active":"2023-10-26T01:36:44.421024+0000","last_peered":"2023-10-26T01:36:44.421024+0000","last_clean":"2023-10-26T01:36:44.421024+0000","last_became_active":"2023-10-26T01:36:44.420693+0000","last_became_peered":"2023-10-26T01:36:44.420693+0000","last_unstale":"2023-10-26T01:36:44.421024+0000","last_undegraded":"2023-10-26T01:36:44.421024+0000","last_fullsized":"2023-10-26T01:36:44.421024+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:06:52.992741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.411244+0000","last_change":"2023-10-26T01:36:44.411029+0000","last_active":"2023-10-26T01:36:44.411244+0000","last_peered":"2023-10-26T01:36:44.411244+0000","last_clean":"2023-10-26T01:36:44.411244+0000","last_became_active":"2023-10-26T01:36:44.410541+0000","last_became_peered":"2023-10-26T01:36:44.410541+0000","last_unstale":"2023-10-26T01:36:44.411244+0000","last_undegraded":"2023-10-26T01:36:44.411244+0000","last_fullsized":"2023-10-26T01:36:44.411244+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:00:59.472904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.420853+0000","last_change":"2023-10-26T01:36:44.420237+0000","last_active":"2023-10-26T01:36:44.420853+0000","last_peered":"2023-10-26T01:36:44.420853+0000","last_clean":"2023-10-26T01:36:44.420853+0000","last_became_active":"2023-10-26T01:36:44.419936+0000","last_became_peered":"2023-10-26T01:36:44.419936+0000","last_unstale":"2023-10-26T01:36:44.420853+0000","last_undegraded":"2023-10-26T01:36:44.420853+0000","last_fullsized":"2023-10-26T01:36:44.420853+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:47:00.376441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T01:36:45.397538+0000","last_change":"2023-10-26T01:36:44.410566+0000","last_active":"2023-10-26T01:36:45.397538+0000","last_peered":"2023-10-26T01:36:45.397538+0000","last_clean":"2023-10-26T01:36:45.397538+0000","last_became_active":"2023-10-26T01:36:44.410125+0000","last_became_peered":"2023-10-26T01:36:44.410125+0000","last_unstale":"2023-10-26T01:36:45.397538+0000","last_undegraded":"2023-10-26T01:36:45.397538+0000","last_fullsized":"2023-10-26T01:36:45.397538+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:49:19.484142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.431570+0000","last_change":"2023-10-26T01:36:44.431294+0000","last_active":"2023-10-26T01:36:44.431570+0000","last_peered":"2023-10-26T01:36:44.431570+0000","last_clean":"2023-10-26T01:36:44.431570+0000","last_became_active":"2023-10-26T01:36:44.428358+0000","last_became_peered":"2023-10-26T01:36:44.428358+0000","last_unstale":"2023-10-26T01:36:44.431570+0000","last_undegraded":"2023-10-26T01:36:44.431570+0000","last_fullsized":"2023-10-26T01:36:44.431570+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:19:39.814025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T01:36:46.740583+0000","last_change":"2023-10-26T01:36:44.411865+0000","last_active":"2023-10-26T01:36:46.740583+0000","last_peered":"2023-10-26T01:36:46.740583+0000","last_clean":"2023-10-26T01:36:46.740583+0000","last_became_active":"2023-10-26T01:36:44.411247+0000","last_became_peered":"2023-10-26T01:36:44.411247+0000","last_unstale":"2023-10-26T01:36:46.740583+0000","last_undegraded":"2023-10-26T01:36:46.740583+0000","last_fullsized":"2023-10-26T01:36:46.740583+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:11:27.820420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T01:36:45.399102+0000","last_change":"2023-10-26T01:36:44.416818+0000","last_active":"2023-10-26T01:36:45.399102+0000","last_peered":"2023-10-26T01:36:45.399102+0000","last_clean":"2023-10-26T01:36:45.399102+0000","last_became_active":"2023-10-26T01:36:44.413677+0000","last_became_peered":"2023-10-26T01:36:44.413677+0000","last_unstale":"2023-10-26T01:36:45.399102+0000","last_undegraded":"2023-10-26T01:36:45.399102+0000","last_fullsized":"2023-10-26T01:36:45.399102+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:15:45.948645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T01:36:45.400187+0000","last_change":"2023-10-26T01:36:44.418530+0000","last_active":"2023-10-26T01:36:45.400187+0000","last_peered":"2023-10-26T01:36:45.400187+0000","last_clean":"2023-10-26T01:36:45.400187+0000","last_became_active":"2023-10-26T01:36:44.418276+0000","last_became_peered":"2023-10-26T01:36:44.418276+0000","last_unstale":"2023-10-26T01:36:45.400187+0000","last_undegraded":"2023-10-26T01:36:45.400187+0000","last_fullsized":"2023-10-26T01:36:45.400187+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:53:42.040042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.425986+0000","last_change":"2023-10-26T01:36:44.425317+0000","last_active":"2023-10-26T01:36:44.425986+0000","last_peered":"2023-10-26T01:36:44.425986+0000","last_clean":"2023-10-26T01:36:44.425986+0000","last_became_active":"2023-10-26T01:36:44.425001+0000","last_became_peered":"2023-10-26T01:36:44.425001+0000","last_unstale":"2023-10-26T01:36:44.425986+0000","last_undegraded":"2023-10-26T01:36:44.425986+0000","last_fullsized":"2023-10-26T01:36:44.425986+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:54:41.116566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.421409+0000","last_change":"2023-10-26T01:36:44.421329+0000","last_active":"2023-10-26T01:36:44.421409+0000","last_peered":"2023-10-26T01:36:44.421409+0000","last_clean":"2023-10-26T01:36:44.421409+0000","last_became_active":"2023-10-26T01:36:44.420858+0000","last_became_peered":"2023-10-26T01:36:44.420858+0000","last_unstale":"2023-10-26T01:36:44.421409+0000","last_undegraded":"2023-10-26T01:36:44.421409+0000","last_fullsized":"2023-10-26T01:36:44.421409+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:44:01.585234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.428069+0000","last_change":"2023-10-26T01:36:44.427089+0000","last_active":"2023-10-26T01:36:44.428069+0000","last_peered":"2023-10-26T01:36:44.428069+0000","last_clean":"2023-10-26T01:36:44.428069+0000","last_became_active":"2023-10-26T01:36:44.426855+0000","last_became_peered":"2023-10-26T01:36:44.426855+0000","last_unstale":"2023-10-26T01:36:44.428069+0000","last_undegraded":"2023-10-26T01:36:44.428069+0000","last_fullsized":"2023-10-26T01:36:44.428069+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:05:27.816078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.428269+0000","last_change":"2023-10-26T01:36:44.428187+0000","last_active":"2023-10-26T01:36:44.428269+0000","last_peered":"2023-10-26T01:36:44.428269+0000","last_clean":"2023-10-26T01:36:44.428269+0000","last_became_active":"2023-10-26T01:36:44.426548+0000","last_became_peered":"2023-10-26T01:36:44.426548+0000","last_unstale":"2023-10-26T01:36:44.428269+0000","last_undegraded":"2023-10-26T01:36:44.428269+0000","last_fullsized":"2023-10-26T01:36:44.428269+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:36:34.521544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.424666+0000","last_change":"2023-10-26T01:36:44.423551+0000","last_active":"2023-10-26T01:36:44.424666+0000","last_peered":"2023-10-26T01:36:44.424666+0000","last_clean":"2023-10-26T01:36:44.424666+0000","last_became_active":"2023-10-26T01:36:44.422694+0000","last_became_peered":"2023-10-26T01:36:44.422694+0000","last_unstale":"2023-10-26T01:36:44.424666+0000","last_undegraded":"2023-10-26T01:36:44.424666+0000","last_fullsized":"2023-10-26T01:36:44.424666+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:36:34.521544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.420675+0000","last_change":"2023-10-26T01:36:44.420192+0000","last_active":"2023-10-26T01:36:44.420675+0000","last_peered":"2023-10-26T01:36:44.420675+0000","last_clean":"2023-10-26T01:36:44.420675+0000","last_became_active":"2023-10-26T01:36:44.419366+0000","last_became_peered":"2023-10-26T01:36:44.419366+0000","last_unstale":"2023-10-26T01:36:44.420675+0000","last_undegraded":"2023-10-26T01:36:44.420675+0000","last_fullsized":"2023-10-26T01:36:44.420675+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:22:24.461105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.430953+0000","last_change":"2023-10-26T01:36:44.430668+0000","last_active":"2023-10-26T01:36:44.430953+0000","last_peered":"2023-10-26T01:36:44.430953+0000","last_clean":"2023-10-26T01:36:44.430953+0000","last_became_active":"2023-10-26T01:36:44.426478+0000","last_became_peered":"2023-10-26T01:36:44.426478+0000","last_unstale":"2023-10-26T01:36:44.430953+0000","last_undegraded":"2023-10-26T01:36:44.430953+0000","last_fullsized":"2023-10-26T01:36:44.430953+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:59:03.849415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.420957+0000","last_change":"2023-10-26T01:36:44.420836+0000","last_active":"2023-10-26T01:36:44.420957+0000","last_peered":"2023-10-26T01:36:44.420957+0000","last_clean":"2023-10-26T01:36:44.420957+0000","last_became_active":"2023-10-26T01:36:44.420120+0000","last_became_peered":"2023-10-26T01:36:44.420120+0000","last_unstale":"2023-10-26T01:36:44.420957+0000","last_undegraded":"2023-10-26T01:36:44.420957+0000","last_fullsized":"2023-10-26T01:36:44.420957+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:19:39.814025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T01:36:45.398189+0000","last_change":"2023-10-26T01:36:44.408697+0000","last_active":"2023-10-26T01:36:45.398189+0000","last_peered":"2023-10-26T01:36:45.398189+0000","last_clean":"2023-10-26T01:36:45.398189+0000","last_became_active":"2023-10-26T01:36:44.408247+0000","last_became_peered":"2023-10-26T01:36:44.408247+0000","last_unstale":"2023-10-26T01:36:45.398189+0000","last_undegraded":"2023-10-26T01:36:45.398189+0000","last_fullsized":"2023-10-26T01:36:45.398189+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:03:27.636888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T01:36:45.401077+0000","last_change":"2023-10-26T01:36:44.416347+0000","last_active":"2023-10-26T01:36:45.401077+0000","last_peered":"2023-10-26T01:36:45.401077+0000","last_clean":"2023-10-26T01:36:45.401077+0000","last_became_active":"2023-10-26T01:36:44.416111+0000","last_became_peered":"2023-10-26T01:36:44.416111+0000","last_unstale":"2023-10-26T01:36:45.401077+0000","last_undegraded":"2023-10-26T01:36:45.401077+0000","last_fullsized":"2023-10-26T01:36:45.401077+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:23:26.163872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.419742+0000","last_change":"2023-10-26T01:36:44.419604+0000","last_active":"2023-10-26T01:36:44.419742+0000","last_peered":"2023-10-26T01:36:44.419742+0000","last_clean":"2023-10-26T01:36:44.419742+0000","last_became_active":"2023-10-26T01:36:44.417497+0000","last_became_peered":"2023-10-26T01:36:44.417497+0000","last_unstale":"2023-10-26T01:36:44.419742+0000","last_undegraded":"2023-10-26T01:36:44.419742+0000","last_fullsized":"2023-10-26T01:36:44.419742+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:59:34.360763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.425549+0000","last_change":"2023-10-26T01:36:44.425447+0000","last_active":"2023-10-26T01:36:44.425549+0000","last_peered":"2023-10-26T01:36:44.425549+0000","last_clean":"2023-10-26T01:36:44.425549+0000","last_became_active":"2023-10-26T01:36:44.425003+0000","last_became_peered":"2023-10-26T01:36:44.425003+0000","last_unstale":"2023-10-26T01:36:44.425549+0000","last_undegraded":"2023-10-26T01:36:44.425549+0000","last_fullsized":"2023-10-26T01:36:44.425549+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:00:56.306188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.421353+0000","last_change":"2023-10-26T01:36:44.421053+0000","last_active":"2023-10-26T01:36:44.421353+0000","last_peered":"2023-10-26T01:36:44.421353+0000","last_clean":"2023-10-26T01:36:44.421353+0000","last_became_active":"2023-10-26T01:36:44.420529+0000","last_became_peered":"2023-10-26T01:36:44.420529+0000","last_unstale":"2023-10-26T01:36:44.421353+0000","last_undegraded":"2023-10-26T01:36:44.421353+0000","last_fullsized":"2023-10-26T01:36:44.421353+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:54:41.116566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T01:36:45.399624+0000","last_change":"2023-10-26T01:36:44.417085+0000","last_active":"2023-10-26T01:36:45.399624+0000","last_peered":"2023-10-26T01:36:45.399624+0000","last_clean":"2023-10-26T01:36:45.399624+0000","last_became_active":"2023-10-26T01:36:44.416250+0000","last_became_peered":"2023-10-26T01:36:44.416250+0000","last_unstale":"2023-10-26T01:36:45.399624+0000","last_undegraded":"2023-10-26T01:36:45.399624+0000","last_fullsized":"2023-10-26T01:36:45.399624+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:39:05.780690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.418854+0000","last_change":"2023-10-26T01:36:44.418641+0000","last_active":"2023-10-26T01:36:44.418854+0000","last_peered":"2023-10-26T01:36:44.418854+0000","last_clean":"2023-10-26T01:36:44.418854+0000","last_became_active":"2023-10-26T01:36:44.418334+0000","last_became_peered":"2023-10-26T01:36:44.418334+0000","last_unstale":"2023-10-26T01:36:44.418854+0000","last_undegraded":"2023-10-26T01:36:44.418854+0000","last_fullsized":"2023-10-26T01:36:44.418854+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:54:11.340492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T01:36:45.398874+0000","last_change":"2023-10-26T01:36:44.415370+0000","last_active":"2023-10-26T01:36:45.398874+0000","last_peered":"2023-10-26T01:36:45.398874+0000","last_clean":"2023-10-26T01:36:45.398874+0000","last_became_active":"2023-10-26T01:36:44.415100+0000","last_became_peered":"2023-10-26T01:36:44.415100+0000","last_unstale":"2023-10-26T01:36:45.398874+0000","last_undegraded":"2023-10-26T01:36:45.398874+0000","last_fullsized":"2023-10-26T01:36:45.398874+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:24:10.566721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"102'4","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.435633+0000","last_change":"2023-10-26T01:36:43.413293+0000","last_active":"2023-10-26T01:36:44.435633+0000","last_peered":"2023-10-26T01:36:44.435633+0000","last_clean":"2023-10-26T01:36:44.435633+0000","last_became_active":"2023-10-26T01:36:43.411577+0000","last_became_peered":"2023-10-26T01:36:43.411577+0000","last_unstale":"2023-10-26T01:36:44.435633+0000","last_undegraded":"2023-10-26T01:36:44.435633+0000","last_fullsized":"2023-10-26T01:36:44.435633+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_clean_scrub_stamp":"2023-10-26T01:36:42.357920+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:51:19.298136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.420054+0000","last_change":"2023-10-26T01:36:44.419973+0000","last_active":"2023-10-26T01:36:44.420054+0000","last_peered":"2023-10-26T01:36:44.420054+0000","last_clean":"2023-10-26T01:36:44.420054+0000","last_became_active":"2023-10-26T01:36:44.419067+0000","last_became_peered":"2023-10-26T01:36:44.419067+0000","last_unstale":"2023-10-26T01:36:44.420054+0000","last_undegraded":"2023-10-26T01:36:44.420054+0000","last_fullsized":"2023-10-26T01:36:44.420054+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:30:25.463376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"102'1","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T01:36:45.399433+0000","last_change":"2023-10-26T01:36:43.405474+0000","last_active":"2023-10-26T01:36:45.399433+0000","last_peered":"2023-10-26T01:36:45.399433+0000","last_clean":"2023-10-26T01:36:45.399433+0000","last_became_active":"2023-10-26T01:36:43.404222+0000","last_became_peered":"2023-10-26T01:36:43.404222+0000","last_unstale":"2023-10-26T01:36:45.399433+0000","last_undegraded":"2023-10-26T01:36:45.399433+0000","last_fullsized":"2023-10-26T01:36:45.399433+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_clean_scrub_stamp":"2023-10-26T01:36:42.357920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:49:10.957265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.418911+0000","last_change":"2023-10-26T01:36:44.418623+0000","last_active":"2023-10-26T01:36:44.418911+0000","last_peered":"2023-10-26T01:36:44.418911+0000","last_clean":"2023-10-26T01:36:44.418911+0000","last_became_active":"2023-10-26T01:36:44.418274+0000","last_became_peered":"2023-10-26T01:36:44.418274+0000","last_unstale":"2023-10-26T01:36:44.418911+0000","last_undegraded":"2023-10-26T01:36:44.418911+0000","last_fullsized":"2023-10-26T01:36:44.418911+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:26:22.614645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"102'2","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.436018+0000","last_change":"2023-10-26T01:36:43.411787+0000","last_active":"2023-10-26T01:36:44.436018+0000","last_peered":"2023-10-26T01:36:44.436018+0000","last_clean":"2023-10-26T01:36:44.436018+0000","last_became_active":"2023-10-26T01:36:43.408087+0000","last_became_peered":"2023-10-26T01:36:43.408087+0000","last_unstale":"2023-10-26T01:36:44.436018+0000","last_undegraded":"2023-10-26T01:36:44.436018+0000","last_fullsized":"2023-10-26T01:36:44.436018+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_clean_scrub_stamp":"2023-10-26T01:36:42.357920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:23:25.150110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T01:36:45.399549+0000","last_change":"2023-10-26T01:36:44.418114+0000","last_active":"2023-10-26T01:36:45.399549+0000","last_peered":"2023-10-26T01:36:45.399549+0000","last_clean":"2023-10-26T01:36:45.399549+0000","last_became_active":"2023-10-26T01:36:44.417888+0000","last_became_peered":"2023-10-26T01:36:44.417888+0000","last_unstale":"2023-10-26T01:36:45.399549+0000","last_undegraded":"2023-10-26T01:36:45.399549+0000","last_fullsized":"2023-10-26T01:36:45.399549+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:29:25.931265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"102'2","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.435273+0000","last_change":"2023-10-26T01:36:43.406074+0000","last_active":"2023-10-26T01:36:44.435273+0000","last_peered":"2023-10-26T01:36:44.435273+0000","last_clean":"2023-10-26T01:36:44.435273+0000","last_became_active":"2023-10-26T01:36:43.405845+0000","last_became_peered":"2023-10-26T01:36:43.405845+0000","last_unstale":"2023-10-26T01:36:44.435273+0000","last_undegraded":"2023-10-26T01:36:44.435273+0000","last_fullsized":"2023-10-26T01:36:44.435273+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_clean_scrub_stamp":"2023-10-26T01:36:42.357920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:27:51.549146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.429845+0000","last_change":"2023-10-26T01:36:44.428810+0000","last_active":"2023-10-26T01:36:44.429845+0000","last_peered":"2023-10-26T01:36:44.429845+0000","last_clean":"2023-10-26T01:36:44.429845+0000","last_became_active":"2023-10-26T01:36:44.428529+0000","last_became_peered":"2023-10-26T01:36:44.428529+0000","last_unstale":"2023-10-26T01:36:44.429845+0000","last_undegraded":"2023-10-26T01:36:44.429845+0000","last_fullsized":"2023-10-26T01:36:44.429845+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:34:32.496565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"102'1","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T01:36:45.400759+0000","last_change":"2023-10-26T01:36:43.400400+0000","last_active":"2023-10-26T01:36:45.400759+0000","last_peered":"2023-10-26T01:36:45.400759+0000","last_clean":"2023-10-26T01:36:45.400759+0000","last_became_active":"2023-10-26T01:36:43.399296+0000","last_became_peered":"2023-10-26T01:36:43.399296+0000","last_unstale":"2023-10-26T01:36:45.400759+0000","last_undegraded":"2023-10-26T01:36:45.400759+0000","last_fullsized":"2023-10-26T01:36:45.400759+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_clean_scrub_stamp":"2023-10-26T01:36:42.357920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:17:00.694879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T01:36:45.400528+0000","last_change":"2023-10-26T01:36:44.419023+0000","last_active":"2023-10-26T01:36:45.400528+0000","last_peered":"2023-10-26T01:36:45.400528+0000","last_clean":"2023-10-26T01:36:45.400528+0000","last_became_active":"2023-10-26T01:36:44.418808+0000","last_became_peered":"2023-10-26T01:36:44.418808+0000","last_unstale":"2023-10-26T01:36:45.400528+0000","last_undegraded":"2023-10-26T01:36:45.400528+0000","last_fullsized":"2023-10-26T01:36:45.400528+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:28:30.194209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"102'1","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T01:36:45.398349+0000","last_change":"2023-10-26T01:36:43.402719+0000","last_active":"2023-10-26T01:36:45.398349+0000","last_peered":"2023-10-26T01:36:45.398349+0000","last_clean":"2023-10-26T01:36:45.398349+0000","last_became_active":"2023-10-26T01:36:43.402064+0000","last_became_peered":"2023-10-26T01:36:43.402064+0000","last_unstale":"2023-10-26T01:36:45.398349+0000","last_undegraded":"2023-10-26T01:36:45.398349+0000","last_fullsized":"2023-10-26T01:36:45.398349+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_clean_scrub_stamp":"2023-10-26T01:36:42.357920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:47:52.274415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T01:36:45.398904+0000","last_change":"2023-10-26T01:36:44.416913+0000","last_active":"2023-10-26T01:36:45.398904+0000","last_peered":"2023-10-26T01:36:45.398904+0000","last_clean":"2023-10-26T01:36:45.398904+0000","last_became_active":"2023-10-26T01:36:44.414717+0000","last_became_peered":"2023-10-26T01:36:44.414717+0000","last_unstale":"2023-10-26T01:36:45.398904+0000","last_undegraded":"2023-10-26T01:36:45.398904+0000","last_fullsized":"2023-10-26T01:36:45.398904+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:13:56.403796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.404317+0000","last_change":"2023-10-26T01:36:43.410769+0000","last_active":"2023-10-26T01:36:44.404317+0000","last_peered":"2023-10-26T01:36:44.404317+0000","last_clean":"2023-10-26T01:36:44.404317+0000","last_became_active":"2023-10-26T01:36:43.410528+0000","last_became_peered":"2023-10-26T01:36:43.410528+0000","last_unstale":"2023-10-26T01:36:44.404317+0000","last_undegraded":"2023-10-26T01:36:44.404317+0000","last_fullsized":"2023-10-26T01:36:44.404317+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_clean_scrub_stamp":"2023-10-26T01:36:42.357920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:44:18.601112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T01:36:46.740706+0000","last_change":"2023-10-26T01:36:44.409818+0000","last_active":"2023-10-26T01:36:46.740706+0000","last_peered":"2023-10-26T01:36:46.740706+0000","last_clean":"2023-10-26T01:36:46.740706+0000","last_became_active":"2023-10-26T01:36:44.408687+0000","last_became_peered":"2023-10-26T01:36:44.408687+0000","last_unstale":"2023-10-26T01:36:46.740706+0000","last_undegraded":"2023-10-26T01:36:46.740706+0000","last_fullsized":"2023-10-26T01:36:46.740706+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:08:16.050379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.402810+0000","last_change":"2023-10-26T01:36:43.403865+0000","last_active":"2023-10-26T01:36:44.402810+0000","last_peered":"2023-10-26T01:36:44.402810+0000","last_clean":"2023-10-26T01:36:44.402810+0000","last_became_active":"2023-10-26T01:36:43.403529+0000","last_became_peered":"2023-10-26T01:36:43.403529+0000","last_unstale":"2023-10-26T01:36:44.402810+0000","last_undegraded":"2023-10-26T01:36:44.402810+0000","last_fullsized":"2023-10-26T01:36:44.402810+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_clean_scrub_stamp":"2023-10-26T01:36:42.357920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:58:45.070927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T01:36:46.740316+0000","last_change":"2023-10-26T01:36:44.418337+0000","last_active":"2023-10-26T01:36:46.740316+0000","last_peered":"2023-10-26T01:36:46.740316+0000","last_clean":"2023-10-26T01:36:46.740316+0000","last_became_active":"2023-10-26T01:36:44.417971+0000","last_became_peered":"2023-10-26T01:36:44.417971+0000","last_unstale":"2023-10-26T01:36:46.740316+0000","last_undegraded":"2023-10-26T01:36:46.740316+0000","last_fullsized":"2023-10-26T01:36:46.740316+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:23:48.053407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.410759+0000","last_change":"2023-10-26T01:36:43.413125+0000","last_active":"2023-10-26T01:36:44.410759+0000","last_peered":"2023-10-26T01:36:44.410759+0000","last_clean":"2023-10-26T01:36:44.410759+0000","last_became_active":"2023-10-26T01:36:43.412871+0000","last_became_peered":"2023-10-26T01:36:43.412871+0000","last_unstale":"2023-10-26T01:36:44.410759+0000","last_undegraded":"2023-10-26T01:36:44.410759+0000","last_fullsized":"2023-10-26T01:36:44.410759+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_clean_scrub_stamp":"2023-10-26T01:36:42.357920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:54:00.451217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T01:36:45.396194+0000","last_change":"2023-10-26T01:36:44.428648+0000","last_active":"2023-10-26T01:36:45.396194+0000","last_peered":"2023-10-26T01:36:45.396194+0000","last_clean":"2023-10-26T01:36:45.396194+0000","last_became_active":"2023-10-26T01:36:44.428465+0000","last_became_peered":"2023-10-26T01:36:44.428465+0000","last_unstale":"2023-10-26T01:36:45.396194+0000","last_undegraded":"2023-10-26T01:36:45.396194+0000","last_fullsized":"2023-10-26T01:36:45.396194+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:17:14.458413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.413953+0000","last_change":"2023-10-26T01:36:43.411329+0000","last_active":"2023-10-26T01:36:44.413953+0000","last_peered":"2023-10-26T01:36:44.413953+0000","last_clean":"2023-10-26T01:36:44.413953+0000","last_became_active":"2023-10-26T01:36:43.409687+0000","last_became_peered":"2023-10-26T01:36:43.409687+0000","last_unstale":"2023-10-26T01:36:44.413953+0000","last_undegraded":"2023-10-26T01:36:44.413953+0000","last_fullsized":"2023-10-26T01:36:44.413953+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_clean_scrub_stamp":"2023-10-26T01:36:42.357920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:47:34.755565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.420627+0000","last_change":"2023-10-26T01:36:44.420555+0000","last_active":"2023-10-26T01:36:44.420627+0000","last_peered":"2023-10-26T01:36:44.420627+0000","last_clean":"2023-10-26T01:36:44.420627+0000","last_became_active":"2023-10-26T01:36:44.414973+0000","last_became_peered":"2023-10-26T01:36:44.414973+0000","last_unstale":"2023-10-26T01:36:44.420627+0000","last_undegraded":"2023-10-26T01:36:44.420627+0000","last_fullsized":"2023-10-26T01:36:44.420627+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:17:08.255587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"102'3","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.434666+0000","last_change":"2023-10-26T01:36:43.414186+0000","last_active":"2023-10-26T01:36:44.434666+0000","last_peered":"2023-10-26T01:36:44.434666+0000","last_clean":"2023-10-26T01:36:44.434666+0000","last_became_active":"2023-10-26T01:36:43.413573+0000","last_became_peered":"2023-10-26T01:36:43.413573+0000","last_unstale":"2023-10-26T01:36:44.434666+0000","last_undegraded":"2023-10-26T01:36:44.434666+0000","last_fullsized":"2023-10-26T01:36:44.434666+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_clean_scrub_stamp":"2023-10-26T01:36:42.357920+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:49:10.957265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T01:36:45.397779+0000","last_change":"2023-10-26T01:36:44.409595+0000","last_active":"2023-10-26T01:36:45.397779+0000","last_peered":"2023-10-26T01:36:45.397779+0000","last_clean":"2023-10-26T01:36:45.397779+0000","last_became_active":"2023-10-26T01:36:44.409186+0000","last_became_peered":"2023-10-26T01:36:44.409186+0000","last_unstale":"2023-10-26T01:36:45.397779+0000","last_undegraded":"2023-10-26T01:36:45.397779+0000","last_fullsized":"2023-10-26T01:36:45.397779+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:31:18.065778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"102'6","reported_seq":18,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T01:36:45.395060+0000","last_change":"2023-10-26T01:36:43.407360+0000","last_active":"2023-10-26T01:36:45.395060+0000","last_peered":"2023-10-26T01:36:45.395060+0000","last_clean":"2023-10-26T01:36:45.395060+0000","last_became_active":"2023-10-26T01:36:43.407068+0000","last_became_peered":"2023-10-26T01:36:43.407068+0000","last_unstale":"2023-10-26T01:36:45.395060+0000","last_undegraded":"2023-10-26T01:36:45.395060+0000","last_fullsized":"2023-10-26T01:36:45.395060+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_clean_scrub_stamp":"2023-10-26T01:36:42.357920+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:22:23.447343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.414739+0000","last_change":"2023-10-26T01:36:44.413671+0000","last_active":"2023-10-26T01:36:44.414739+0000","last_peered":"2023-10-26T01:36:44.414739+0000","last_clean":"2023-10-26T01:36:44.414739+0000","last_became_active":"2023-10-26T01:36:44.411171+0000","last_became_peered":"2023-10-26T01:36:44.411171+0000","last_unstale":"2023-10-26T01:36:44.414739+0000","last_undegraded":"2023-10-26T01:36:44.414739+0000","last_fullsized":"2023-10-26T01:36:44.414739+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:29:42.674658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"102'2","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.433183+0000","last_change":"2023-10-26T01:36:43.406935+0000","last_active":"2023-10-26T01:36:44.433183+0000","last_peered":"2023-10-26T01:36:44.433183+0000","last_clean":"2023-10-26T01:36:44.433183+0000","last_became_active":"2023-10-26T01:36:43.405455+0000","last_became_peered":"2023-10-26T01:36:43.405455+0000","last_unstale":"2023-10-26T01:36:44.433183+0000","last_undegraded":"2023-10-26T01:36:44.433183+0000","last_fullsized":"2023-10-26T01:36:44.433183+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_clean_scrub_stamp":"2023-10-26T01:36:42.357920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:46:15.128764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T01:36:45.399222+0000","last_change":"2023-10-26T01:36:44.417076+0000","last_active":"2023-10-26T01:36:45.399222+0000","last_peered":"2023-10-26T01:36:45.399222+0000","last_clean":"2023-10-26T01:36:45.399222+0000","last_became_active":"2023-10-26T01:36:44.416479+0000","last_became_peered":"2023-10-26T01:36:44.416479+0000","last_unstale":"2023-10-26T01:36:45.399222+0000","last_undegraded":"2023-10-26T01:36:45.399222+0000","last_fullsized":"2023-10-26T01:36:45.399222+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:39:17.285562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T01:36:46.740226+0000","last_change":"2023-10-26T01:36:43.399012+0000","last_active":"2023-10-26T01:36:46.740226+0000","last_peered":"2023-10-26T01:36:46.740226+0000","last_clean":"2023-10-26T01:36:46.740226+0000","last_became_active":"2023-10-26T01:36:43.397583+0000","last_became_peered":"2023-10-26T01:36:43.397583+0000","last_unstale":"2023-10-26T01:36:46.740226+0000","last_undegraded":"2023-10-26T01:36:46.740226+0000","last_fullsized":"2023-10-26T01:36:46.740226+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_clean_scrub_stamp":"2023-10-26T01:36:42.357920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:17:05.159088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.425525+0000","last_change":"2023-10-26T01:36:44.424737+0000","last_active":"2023-10-26T01:36:44.425525+0000","last_peered":"2023-10-26T01:36:44.425525+0000","last_clean":"2023-10-26T01:36:44.425525+0000","last_became_active":"2023-10-26T01:36:44.424496+0000","last_became_peered":"2023-10-26T01:36:44.424496+0000","last_unstale":"2023-10-26T01:36:44.425525+0000","last_undegraded":"2023-10-26T01:36:44.425525+0000","last_fullsized":"2023-10-26T01:36:44.425525+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:05:27.816078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T01:36:45.395258+0000","last_change":"2023-10-26T01:36:43.412003+0000","last_active":"2023-10-26T01:36:45.395258+0000","last_peered":"2023-10-26T01:36:45.395258+0000","last_clean":"2023-10-26T01:36:45.395258+0000","last_became_active":"2023-10-26T01:36:43.410776+0000","last_became_peered":"2023-10-26T01:36:43.410776+0000","last_unstale":"2023-10-26T01:36:45.395258+0000","last_undegraded":"2023-10-26T01:36:45.395258+0000","last_fullsized":"2023-10-26T01:36:45.395258+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_clean_scrub_stamp":"2023-10-26T01:36:42.357920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:51:19.298136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.424632+0000","last_change":"2023-10-26T01:36:44.423416+0000","last_active":"2023-10-26T01:36:44.424632+0000","last_peered":"2023-10-26T01:36:44.424632+0000","last_clean":"2023-10-26T01:36:44.424632+0000","last_became_active":"2023-10-26T01:36:44.422328+0000","last_became_peered":"2023-10-26T01:36:44.422328+0000","last_unstale":"2023-10-26T01:36:44.424632+0000","last_undegraded":"2023-10-26T01:36:44.424632+0000","last_fullsized":"2023-10-26T01:36:44.424632+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:03:02.870381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"102'8","reported_seq":19,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.436423+0000","last_change":"2023-10-26T01:36:43.402704+0000","last_active":"2023-10-26T01:36:44.436423+0000","last_peered":"2023-10-26T01:36:44.436423+0000","last_clean":"2023-10-26T01:36:44.436423+0000","last_became_active":"2023-10-26T01:36:43.402404+0000","last_became_peered":"2023-10-26T01:36:43.402404+0000","last_unstale":"2023-10-26T01:36:44.436423+0000","last_undegraded":"2023-10-26T01:36:44.436423+0000","last_fullsized":"2023-10-26T01:36:44.436423+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_clean_scrub_stamp":"2023-10-26T01:36:42.357920+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:59:45.462209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T01:36:45.399858+0000","last_change":"2023-10-26T01:36:44.408926+0000","last_active":"2023-10-26T01:36:45.399858+0000","last_peered":"2023-10-26T01:36:45.399858+0000","last_clean":"2023-10-26T01:36:45.399858+0000","last_became_active":"2023-10-26T01:36:44.408516+0000","last_became_peered":"2023-10-26T01:36:44.408516+0000","last_unstale":"2023-10-26T01:36:45.399858+0000","last_undegraded":"2023-10-26T01:36:45.399858+0000","last_fullsized":"2023-10-26T01:36:45.399858+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:17:06.172850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"102'1","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.431854+0000","last_change":"2023-10-26T01:36:43.413010+0000","last_active":"2023-10-26T01:36:44.431854+0000","last_peered":"2023-10-26T01:36:44.431854+0000","last_clean":"2023-10-26T01:36:44.431854+0000","last_became_active":"2023-10-26T01:36:43.412642+0000","last_became_peered":"2023-10-26T01:36:43.412642+0000","last_unstale":"2023-10-26T01:36:44.431854+0000","last_undegraded":"2023-10-26T01:36:44.431854+0000","last_fullsized":"2023-10-26T01:36:44.431854+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_clean_scrub_stamp":"2023-10-26T01:36:42.357920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:30:33.922915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.414193+0000","last_change":"2023-10-26T01:36:44.414021+0000","last_active":"2023-10-26T01:36:44.414193+0000","last_peered":"2023-10-26T01:36:44.414193+0000","last_clean":"2023-10-26T01:36:44.414193+0000","last_became_active":"2023-10-26T01:36:44.413731+0000","last_became_peered":"2023-10-26T01:36:44.413731+0000","last_unstale":"2023-10-26T01:36:44.414193+0000","last_undegraded":"2023-10-26T01:36:44.414193+0000","last_fullsized":"2023-10-26T01:36:44.414193+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:26:07.785221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"102'2","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.436744+0000","last_change":"2023-10-26T01:36:43.412151+0000","last_active":"2023-10-26T01:36:44.436744+0000","last_peered":"2023-10-26T01:36:44.436744+0000","last_clean":"2023-10-26T01:36:44.436744+0000","last_became_active":"2023-10-26T01:36:43.408391+0000","last_became_peered":"2023-10-26T01:36:43.408391+0000","last_unstale":"2023-10-26T01:36:44.436744+0000","last_undegraded":"2023-10-26T01:36:44.436744+0000","last_fullsized":"2023-10-26T01:36:44.436744+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_clean_scrub_stamp":"2023-10-26T01:36:42.357920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:47:32.591256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T01:36:45.399926+0000","last_change":"2023-10-26T01:36:44.413370+0000","last_active":"2023-10-26T01:36:45.399926+0000","last_peered":"2023-10-26T01:36:45.399926+0000","last_clean":"2023-10-26T01:36:45.399926+0000","last_became_active":"2023-10-26T01:36:44.412999+0000","last_became_peered":"2023-10-26T01:36:44.412999+0000","last_unstale":"2023-10-26T01:36:45.399926+0000","last_undegraded":"2023-10-26T01:36:45.399926+0000","last_fullsized":"2023-10-26T01:36:45.399926+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:52:13.995886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.402066+0000","last_change":"2023-10-26T01:36:43.409895+0000","last_active":"2023-10-26T01:36:44.402066+0000","last_peered":"2023-10-26T01:36:44.402066+0000","last_clean":"2023-10-26T01:36:44.402066+0000","last_became_active":"2023-10-26T01:36:43.409665+0000","last_became_peered":"2023-10-26T01:36:43.409665+0000","last_unstale":"2023-10-26T01:36:44.402066+0000","last_undegraded":"2023-10-26T01:36:44.402066+0000","last_fullsized":"2023-10-26T01:36:44.402066+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_clean_scrub_stamp":"2023-10-26T01:36:42.357920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:05:00.560066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.425933+0000","last_change":"2023-10-26T01:36:44.421624+0000","last_active":"2023-10-26T01:36:44.425933+0000","last_peered":"2023-10-26T01:36:44.425933+0000","last_clean":"2023-10-26T01:36:44.425933+0000","last_became_active":"2023-10-26T01:36:44.421379+0000","last_became_peered":"2023-10-26T01:36:44.421379+0000","last_unstale":"2023-10-26T01:36:44.425933+0000","last_undegraded":"2023-10-26T01:36:44.425933+0000","last_fullsized":"2023-10-26T01:36:44.425933+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:51:56.198313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"102'2","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T01:36:46.739957+0000","last_change":"2023-10-26T01:36:43.411409+0000","last_active":"2023-10-26T01:36:46.739957+0000","last_peered":"2023-10-26T01:36:46.739957+0000","last_clean":"2023-10-26T01:36:46.739957+0000","last_became_active":"2023-10-26T01:36:43.410628+0000","last_became_peered":"2023-10-26T01:36:43.410628+0000","last_unstale":"2023-10-26T01:36:46.739957+0000","last_undegraded":"2023-10-26T01:36:46.739957+0000","last_fullsized":"2023-10-26T01:36:46.739957+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_clean_scrub_stamp":"2023-10-26T01:36:42.357920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:13:55.390034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T01:36:45.395219+0000","last_change":"2023-10-26T01:36:44.415900+0000","last_active":"2023-10-26T01:36:45.395219+0000","last_peered":"2023-10-26T01:36:45.395219+0000","last_clean":"2023-10-26T01:36:45.395219+0000","last_became_active":"2023-10-26T01:36:44.415487+0000","last_became_peered":"2023-10-26T01:36:44.415487+0000","last_unstale":"2023-10-26T01:36:45.395219+0000","last_undegraded":"2023-10-26T01:36:45.395219+0000","last_fullsized":"2023-10-26T01:36:45.395219+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:02:41.666058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.416426+0000","last_change":"2023-10-26T01:36:43.406198+0000","last_active":"2023-10-26T01:36:44.416426+0000","last_peered":"2023-10-26T01:36:44.416426+0000","last_clean":"2023-10-26T01:36:44.416426+0000","last_became_active":"2023-10-26T01:36:43.405954+0000","last_became_peered":"2023-10-26T01:36:43.405954+0000","last_unstale":"2023-10-26T01:36:44.416426+0000","last_undegraded":"2023-10-26T01:36:44.416426+0000","last_fullsized":"2023-10-26T01:36:44.416426+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_clean_scrub_stamp":"2023-10-26T01:36:42.357920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:46:27.045358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.425970+0000","last_change":"2023-10-26T01:36:44.422932+0000","last_active":"2023-10-26T01:36:44.425970+0000","last_peered":"2023-10-26T01:36:44.425970+0000","last_clean":"2023-10-26T01:36:44.425970+0000","last_became_active":"2023-10-26T01:36:44.422694+0000","last_became_peered":"2023-10-26T01:36:44.422694+0000","last_unstale":"2023-10-26T01:36:44.425970+0000","last_undegraded":"2023-10-26T01:36:44.425970+0000","last_fullsized":"2023-10-26T01:36:44.425970+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:00:59.472904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.415321+0000","last_change":"2023-10-26T01:36:43.407596+0000","last_active":"2023-10-26T01:36:44.415321+0000","last_peered":"2023-10-26T01:36:44.415321+0000","last_clean":"2023-10-26T01:36:44.415321+0000","last_became_active":"2023-10-26T01:36:43.407270+0000","last_became_peered":"2023-10-26T01:36:43.407270+0000","last_unstale":"2023-10-26T01:36:44.415321+0000","last_undegraded":"2023-10-26T01:36:44.415321+0000","last_fullsized":"2023-10-26T01:36:44.415321+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_clean_scrub_stamp":"2023-10-26T01:36:42.357920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:34:31.482803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T01:36:45.395121+0000","last_change":"2023-10-26T01:36:44.405991+0000","last_active":"2023-10-26T01:36:45.395121+0000","last_peered":"2023-10-26T01:36:45.395121+0000","last_clean":"2023-10-26T01:36:45.395121+0000","last_became_active":"2023-10-26T01:36:44.405313+0000","last_became_peered":"2023-10-26T01:36:44.405313+0000","last_unstale":"2023-10-26T01:36:45.395121+0000","last_undegraded":"2023-10-26T01:36:45.395121+0000","last_fullsized":"2023-10-26T01:36:45.395121+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:03:30.551428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T01:36:46.740320+0000","last_change":"2023-10-26T01:36:43.405113+0000","last_active":"2023-10-26T01:36:46.740320+0000","last_peered":"2023-10-26T01:36:46.740320+0000","last_clean":"2023-10-26T01:36:46.740320+0000","last_became_active":"2023-10-26T01:36:43.404832+0000","last_became_peered":"2023-10-26T01:36:43.404832+0000","last_unstale":"2023-10-26T01:36:46.740320+0000","last_undegraded":"2023-10-26T01:36:46.740320+0000","last_fullsized":"2023-10-26T01:36:46.740320+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_clean_scrub_stamp":"2023-10-26T01:36:42.357920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:19:47.358532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T01:36:45.398110+0000","last_change":"2023-10-26T01:36:44.406938+0000","last_active":"2023-10-26T01:36:45.398110+0000","last_peered":"2023-10-26T01:36:45.398110+0000","last_clean":"2023-10-26T01:36:45.398110+0000","last_became_active":"2023-10-26T01:36:44.406262+0000","last_became_peered":"2023-10-26T01:36:44.406262+0000","last_unstale":"2023-10-26T01:36:45.398110+0000","last_undegraded":"2023-10-26T01:36:45.398110+0000","last_fullsized":"2023-10-26T01:36:45.398110+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:48:50.672248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"102'2","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.430733+0000","last_change":"2023-10-26T01:36:43.406790+0000","last_active":"2023-10-26T01:36:44.430733+0000","last_peered":"2023-10-26T01:36:44.430733+0000","last_clean":"2023-10-26T01:36:44.430733+0000","last_became_active":"2023-10-26T01:36:43.405322+0000","last_became_peered":"2023-10-26T01:36:43.405322+0000","last_unstale":"2023-10-26T01:36:44.430733+0000","last_undegraded":"2023-10-26T01:36:44.430733+0000","last_fullsized":"2023-10-26T01:36:44.430733+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_clean_scrub_stamp":"2023-10-26T01:36:42.357920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:31:26.478685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T01:36:45.397589+0000","last_change":"2023-10-26T01:36:44.417160+0000","last_active":"2023-10-26T01:36:45.397589+0000","last_peered":"2023-10-26T01:36:45.397589+0000","last_clean":"2023-10-26T01:36:45.397589+0000","last_became_active":"2023-10-26T01:36:44.415241+0000","last_became_peered":"2023-10-26T01:36:44.415241+0000","last_unstale":"2023-10-26T01:36:45.397589+0000","last_undegraded":"2023-10-26T01:36:45.397589+0000","last_fullsized":"2023-10-26T01:36:45.397589+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:12:30.105816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.401222+0000","last_change":"2023-10-26T01:36:43.403107+0000","last_active":"2023-10-26T01:36:44.401222+0000","last_peered":"2023-10-26T01:36:44.401222+0000","last_clean":"2023-10-26T01:36:44.401222+0000","last_became_active":"2023-10-26T01:36:43.402793+0000","last_became_peered":"2023-10-26T01:36:43.402793+0000","last_unstale":"2023-10-26T01:36:44.401222+0000","last_undegraded":"2023-10-26T01:36:44.401222+0000","last_fullsized":"2023-10-26T01:36:44.401222+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_clean_scrub_stamp":"2023-10-26T01:36:42.357920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:07:07.040053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.420693+0000","last_change":"2023-10-26T01:36:44.420208+0000","last_active":"2023-10-26T01:36:44.420693+0000","last_peered":"2023-10-26T01:36:44.420693+0000","last_clean":"2023-10-26T01:36:44.420693+0000","last_became_active":"2023-10-26T01:36:44.419705+0000","last_became_peered":"2023-10-26T01:36:44.419705+0000","last_unstale":"2023-10-26T01:36:44.420693+0000","last_undegraded":"2023-10-26T01:36:44.420693+0000","last_fullsized":"2023-10-26T01:36:44.420693+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:13:11.418923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.406508+0000","last_change":"2023-10-26T01:36:43.414122+0000","last_active":"2023-10-26T01:36:44.406508+0000","last_peered":"2023-10-26T01:36:44.406508+0000","last_clean":"2023-10-26T01:36:44.406508+0000","last_became_active":"2023-10-26T01:36:43.413589+0000","last_became_peered":"2023-10-26T01:36:43.413589+0000","last_unstale":"2023-10-26T01:36:44.406508+0000","last_undegraded":"2023-10-26T01:36:44.406508+0000","last_fullsized":"2023-10-26T01:36:44.406508+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_clean_scrub_stamp":"2023-10-26T01:36:42.357920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T01:52:13.748946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T01:36:45.399215+0000","last_change":"2023-10-26T01:36:44.409854+0000","last_active":"2023-10-26T01:36:45.399215+0000","last_peered":"2023-10-26T01:36:45.399215+0000","last_clean":"2023-10-26T01:36:45.399215+0000","last_became_active":"2023-10-26T01:36:44.409588+0000","last_became_peered":"2023-10-26T01:36:44.409588+0000","last_unstale":"2023-10-26T01:36:45.399215+0000","last_undegraded":"2023-10-26T01:36:45.399215+0000","last_fullsized":"2023-10-26T01:36:45.399215+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:04:02.797079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.411601+0000","last_change":"2023-10-26T01:36:43.413170+0000","last_active":"2023-10-26T01:36:44.411601+0000","last_peered":"2023-10-26T01:36:44.411601+0000","last_clean":"2023-10-26T01:36:44.411601+0000","last_became_active":"2023-10-26T01:36:43.412927+0000","last_became_peered":"2023-10-26T01:36:43.412927+0000","last_unstale":"2023-10-26T01:36:44.411601+0000","last_undegraded":"2023-10-26T01:36:44.411601+0000","last_fullsized":"2023-10-26T01:36:44.411601+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_clean_scrub_stamp":"2023-10-26T01:36:42.357920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:00:05.181878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T01:36:45.394877+0000","last_change":"2023-10-26T01:36:44.409405+0000","last_active":"2023-10-26T01:36:45.394877+0000","last_peered":"2023-10-26T01:36:45.394877+0000","last_clean":"2023-10-26T01:36:45.394877+0000","last_became_active":"2023-10-26T01:36:44.409093+0000","last_became_peered":"2023-10-26T01:36:44.409093+0000","last_unstale":"2023-10-26T01:36:45.394877+0000","last_undegraded":"2023-10-26T01:36:45.394877+0000","last_fullsized":"2023-10-26T01:36:45.394877+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:01:20.291335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.408321+0000","last_change":"2023-10-26T01:36:43.411506+0000","last_active":"2023-10-26T01:36:44.408321+0000","last_peered":"2023-10-26T01:36:44.408321+0000","last_clean":"2023-10-26T01:36:44.408321+0000","last_became_active":"2023-10-26T01:36:43.411002+0000","last_became_peered":"2023-10-26T01:36:43.411002+0000","last_unstale":"2023-10-26T01:36:44.408321+0000","last_undegraded":"2023-10-26T01:36:44.408321+0000","last_fullsized":"2023-10-26T01:36:44.408321+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_clean_scrub_stamp":"2023-10-26T01:36:42.357920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:39:01.465641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.414758+0000","last_change":"2023-10-26T01:36:44.414491+0000","last_active":"2023-10-26T01:36:44.414758+0000","last_peered":"2023-10-26T01:36:44.414758+0000","last_clean":"2023-10-26T01:36:44.414758+0000","last_became_active":"2023-10-26T01:36:44.414215+0000","last_became_peered":"2023-10-26T01:36:44.414215+0000","last_unstale":"2023-10-26T01:36:44.414758+0000","last_undegraded":"2023-10-26T01:36:44.414758+0000","last_fullsized":"2023-10-26T01:36:44.414758+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:47:17.876626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T01:36:45.401274+0000","last_change":"2023-10-26T01:36:43.399987+0000","last_active":"2023-10-26T01:36:45.401274+0000","last_peered":"2023-10-26T01:36:45.401274+0000","last_clean":"2023-10-26T01:36:45.401274+0000","last_became_active":"2023-10-26T01:36:43.397978+0000","last_became_peered":"2023-10-26T01:36:43.397978+0000","last_unstale":"2023-10-26T01:36:45.401274+0000","last_undegraded":"2023-10-26T01:36:45.401274+0000","last_fullsized":"2023-10-26T01:36:45.401274+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_clean_scrub_stamp":"2023-10-26T01:36:42.357920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:17:13.444651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T01:36:45.397179+0000","last_change":"2023-10-26T01:36:44.406012+0000","last_active":"2023-10-26T01:36:45.397179+0000","last_peered":"2023-10-26T01:36:45.397179+0000","last_clean":"2023-10-26T01:36:45.397179+0000","last_became_active":"2023-10-26T01:36:44.405330+0000","last_became_peered":"2023-10-26T01:36:44.405330+0000","last_unstale":"2023-10-26T01:36:45.397179+0000","last_undegraded":"2023-10-26T01:36:45.397179+0000","last_fullsized":"2023-10-26T01:36:45.397179+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:01:37.708171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.409122+0000","last_change":"2023-10-26T01:36:43.414328+0000","last_active":"2023-10-26T01:36:44.409122+0000","last_peered":"2023-10-26T01:36:44.409122+0000","last_clean":"2023-10-26T01:36:44.409122+0000","last_became_active":"2023-10-26T01:36:43.413959+0000","last_became_peered":"2023-10-26T01:36:43.413959+0000","last_unstale":"2023-10-26T01:36:44.409122+0000","last_undegraded":"2023-10-26T01:36:44.409122+0000","last_fullsized":"2023-10-26T01:36:44.409122+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_clean_scrub_stamp":"2023-10-26T01:36:42.357920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:47:52.274415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.424161+0000","last_change":"2023-10-26T01:36:44.424112+0000","last_active":"2023-10-26T01:36:44.424161+0000","last_peered":"2023-10-26T01:36:44.424161+0000","last_clean":"2023-10-26T01:36:44.424161+0000","last_became_active":"2023-10-26T01:36:44.423934+0000","last_became_peered":"2023-10-26T01:36:44.423934+0000","last_unstale":"2023-10-26T01:36:44.424161+0000","last_undegraded":"2023-10-26T01:36:44.424161+0000","last_fullsized":"2023-10-26T01:36:44.424161+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:51:38.585631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T01:36:45.399661+0000","last_change":"2023-10-26T01:36:43.411715+0000","last_active":"2023-10-26T01:36:45.399661+0000","last_peered":"2023-10-26T01:36:45.399661+0000","last_clean":"2023-10-26T01:36:45.399661+0000","last_became_active":"2023-10-26T01:36:43.411407+0000","last_became_peered":"2023-10-26T01:36:43.411407+0000","last_unstale":"2023-10-26T01:36:45.399661+0000","last_undegraded":"2023-10-26T01:36:45.399661+0000","last_fullsized":"2023-10-26T01:36:45.399661+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_clean_scrub_stamp":"2023-10-26T01:36:42.357920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:54:00.451217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.414839+0000","last_change":"2023-10-26T01:36:44.414419+0000","last_active":"2023-10-26T01:36:44.414839+0000","last_peered":"2023-10-26T01:36:44.414839+0000","last_clean":"2023-10-26T01:36:44.414839+0000","last_became_active":"2023-10-26T01:36:44.414195+0000","last_became_peered":"2023-10-26T01:36:44.414195+0000","last_unstale":"2023-10-26T01:36:44.414839+0000","last_undegraded":"2023-10-26T01:36:44.414839+0000","last_fullsized":"2023-10-26T01:36:44.414839+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:43.371682+0000","last_clean_scrub_stamp":"2023-10-26T01:36:43.371682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:11:12.571272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"102'1","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T01:36:45.398401+0000","last_change":"2023-10-26T01:36:43.405515+0000","last_active":"2023-10-26T01:36:45.398401+0000","last_peered":"2023-10-26T01:36:45.398401+0000","last_clean":"2023-10-26T01:36:45.398401+0000","last_became_active":"2023-10-26T01:36:43.399976+0000","last_became_peered":"2023-10-26T01:36:43.399976+0000","last_unstale":"2023-10-26T01:36:45.398401+0000","last_undegraded":"2023-10-26T01:36:45.398401+0000","last_fullsized":"2023-10-26T01:36:45.398401+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:36:42.357920+0000","last_clean_scrub_stamp":"2023-10-26T01:36:42.357920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:44:00.571472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":250,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T01:36:44.396338+0000","last_change":"2023-10-26T01:30:34.883484+0000","last_active":"2023-10-26T01:36:44.396338+0000","last_peered":"2023-10-26T01:36:44.396338+0000","last_clean":"2023-10-26T01:36:44.396338+0000","last_became_active":"2023-10-26T01:30:34.883071+0000","last_became_peered":"2023-10-26T01:30:34.883071+0000","last_unstale":"2023-10-26T01:36:44.396338+0000","last_undegraded":"2023-10-26T01:36:44.396338+0000","last_fullsized":"2023-10-26T01:36:44.396338+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:32.890096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:32.890096+0000","last_clean_scrub_stamp":"2023-10-26T01:30:32.890096+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:55:09.809749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":14,"seq":60129542222,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":546,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194765,"statfs":{"total":200000000,"available":199440284,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:35:47 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.80000000000000004,"15min":0.78800000000000003},"min":{"1min":0.35599999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.119,"5min":2.2549999999999999,"15min":2.2549999999999999},"last":0.95299999999999996},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.86299999999999999,"15min":0.89300000000000002},"min":{"1min":0.54600000000000004,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.556,"5min":1.8200000000000001,"15min":1.8200000000000001},"last":0.53200000000000003}]},{"osd":1,"last update":"Thu Oct 26 01:35:47 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.88200000000000001,"15min":0.95899999999999996},"min":{"1min":0.48999999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.3959999999999999,"5min":4.4340000000000002,"15min":4.4340000000000002},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.97699999999999998,"5min":0.97299999999999998,"15min":1.0029999999999999},"min":{"1min":0.54600000000000004,"5min":0.374,"15min":0.374},"max":{"1min":1.377,"5min":2.3330000000000002,"15min":2.3330000000000002},"last":0.56799999999999995}]},{"osd":2,"last update":"Thu Oct 26 01:35:47 2023","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":1.03,"15min":1.087},"min":{"1min":0.52500000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.476,"5min":4.2800000000000002,"15min":4.2800000000000002},"last":1.927},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.1200000000000001,"15min":1.091},"min":{"1min":0.54600000000000004,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.5640000000000001,"5min":2.302,"15min":2.302},"last":0.86799999999999999}]},{"osd":4,"last update":"Thu Oct 26 01:35:47 2023","interfaces":[{"interface":"back","average":{"1min":1.177,"5min":1.1759999999999999,"15min":1.1579999999999999},"min":{"1min":0.84899999999999998,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.4339999999999999,"5min":2.1070000000000002,"15min":2.1070000000000002},"last":1.7150000000000001},{"interface":"front","average":{"1min":1.3240000000000001,"5min":1.3440000000000001,"15min":1.4039999999999999},"min":{"1min":0.97999999999999998,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":1.5620000000000001,"5min":4.8250000000000002,"15min":4.8250000000000002},"last":2.4239999999999999}]},{"osd":5,"last update":"Thu Oct 26 01:35:47 2023","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.2849999999999999,"15min":1.349},"min":{"1min":0.86599999999999999,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.6040000000000001,"5min":4.9080000000000004,"15min":4.9080000000000004},"last":2.3380000000000001},{"interface":"front","average":{"1min":1.24,"5min":1.2549999999999999,"15min":1.272},"min":{"1min":0.79900000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.8899999999999999,"5min":2.8980000000000001,"15min":2.8980000000000001},"last":1.512}]},{"osd":6,"last update":"Thu Oct 26 01:35:47 2023","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.3300000000000001,"15min":1.3240000000000001},"min":{"1min":0.89600000000000002,"5min":0.85099999999999998,"15min":0.85099999999999998},"max":{"1min":1.8240000000000001,"5min":2.8530000000000002,"15min":2.8530000000000002},"last":1.583},{"interface":"front","average":{"1min":1.294,"5min":1.2470000000000001,"15min":1.268},"min":{"1min":0.84699999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.6160000000000001,"5min":2.181,"15min":2.181},"last":1.466}]},{"osd":7,"last update":"Thu Oct 26 01:35:47 2023","interfaces":[{"interface":"back","average":{"1min":1.345,"5min":1.296,"15min":1.2889999999999999},"min":{"1min":0.92000000000000004,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.6879999999999999,"5min":2.0680000000000001,"15min":2.0680000000000001},"last":1.365},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.379,"15min":1.4259999999999999},"min":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":1.875,"5min":4.8730000000000002,"15min":4.8730000000000002},"last":2.2370000000000001}]}]},{"osd":7,"up_from":13,"seq":55834574925,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1121,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194190,"statfs":{"total":200000000,"available":198851458,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:36:45 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.145,"15min":1.091},"min":{"1min":0.69699999999999995,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.6020000000000001,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":1.4630000000000001},{"interface":"front","average":{"1min":1.036,"5min":1.157,"15min":1.085},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.28,"5min":2.6779999999999999,"15min":2.6779999999999999},"last":1.214}]},{"osd":1,"last update":"Thu Oct 26 01:36:45 2023","interfaces":[{"interface":"back","average":{"1min":1.1699999999999999,"5min":1.2050000000000001,"15min":1.091},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.65200000000000002},"max":{"1min":1.6040000000000001,"5min":2.8260000000000001,"15min":2.8260000000000001},"last":1.2849999999999999},{"interface":"front","average":{"1min":1.097,"5min":1.216,"15min":1.1879999999999999},"min":{"1min":0.68799999999999994,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.542,"5min":3.2730000000000001,"15min":3.2730000000000001},"last":0.76300000000000001}]},{"osd":2,"last update":"Thu Oct 26 01:36:45 2023","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.175,"15min":1.1140000000000001},"min":{"1min":0.69099999999999995,"5min":0.61199999999999999,"15min":0.58499999999999996},"max":{"1min":1.8680000000000001,"5min":2.218,"15min":2.218},"last":1.0600000000000001},{"interface":"front","average":{"1min":1.2909999999999999,"5min":1.296,"15min":1.236},"min":{"1min":0.79400000000000004,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.8999999999999999,"5min":3.0510000000000002,"15min":3.0510000000000002},"last":1.3979999999999999}]},{"osd":3,"last update":"Thu Oct 26 01:36:45 2023","interfaces":[{"interface":"back","average":{"1min":1.264,"5min":1.272,"15min":1.208},"min":{"1min":0.80600000000000005,"5min":0.79000000000000004,"15min":0.72099999999999997},"max":{"1min":1.796,"5min":1.9830000000000001,"15min":1.9830000000000001},"last":0.95899999999999996},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.419,"15min":1.377},"min":{"1min":0.82899999999999996,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.8680000000000001,"5min":5.5039999999999996,"15min":5.5039999999999996},"last":1.0129999999999999}]},{"osd":4,"last update":"Thu Oct 26 01:36:45 2023","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.3500000000000001,"15min":1.3460000000000001},"min":{"1min":0.77300000000000002,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.5640000000000001,"5min":2.77,"15min":2.77},"last":1.161},{"interface":"front","average":{"1min":1.298,"5min":1.425,"15min":1.3939999999999999},"min":{"1min":0.96899999999999997,"5min":0.496,"15min":0.496},"max":{"1min":1.6830000000000001,"5min":3.1419999999999999,"15min":3.1419999999999999},"last":1.125}]},{"osd":5,"last update":"Thu Oct 26 01:36:45 2023","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.4710000000000001,"15min":1.419},"min":{"1min":0.65600000000000003,"5min":0.441,"15min":0.441},"max":{"1min":1.746,"5min":5.734,"15min":5.734},"last":0.92100000000000004},{"interface":"front","average":{"1min":1.4339999999999999,"5min":1.5109999999999999,"15min":1.476},"min":{"1min":0.96099999999999997,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.829,"5min":2.8980000000000001,"15min":2.8980000000000001},"last":1.25}]},{"osd":6,"last update":"Thu Oct 26 01:36:45 2023","interfaces":[{"interface":"back","average":{"1min":1.4670000000000001,"5min":1.603,"15min":1.877},"min":{"1min":0.753,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.7430000000000001,"5min":5.6340000000000003,"15min":11.577999999999999},"last":1.3180000000000001},{"interface":"front","average":{"1min":1.552,"5min":1.661,"15min":1.8879999999999999},"min":{"1min":0.91800000000000004,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.9339999999999999,"5min":5.0919999999999996,"15min":10.704000000000001},"last":1.361}]}]},{"osd":6,"up_from":13,"seq":55834574925,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":538,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194773,"statfs":{"total":200000000,"available":199448116,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:36:42 2023","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.097,"15min":1.1599999999999999},"min":{"1min":0.61699999999999999,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.647,"5min":2.395,"15min":2.395},"last":0.877},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.1299999999999999,"15min":1.1200000000000001},"min":{"1min":0.72899999999999998,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.7390000000000001,"5min":1.7390000000000001,"15min":1.843},"last":1.7390000000000001}]},{"osd":1,"last update":"Thu Oct 26 01:36:42 2023","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.1639999999999999,"15min":1.1719999999999999},"min":{"1min":0.74399999999999999,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.784,"5min":2.0529999999999999,"15min":2.0529999999999999},"last":1.1180000000000001},{"interface":"front","average":{"1min":1.139,"5min":1.1519999999999999,"15min":1.2010000000000001},"min":{"1min":0.79000000000000004,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.5229999999999999,"5min":1.581,"15min":2.085},"last":1.4379999999999999}]},{"osd":2,"last update":"Thu Oct 26 01:36:42 2023","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.1779999999999999,"15min":1.0940000000000001},"min":{"1min":0.83099999999999996,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.7949999999999999,"5min":3.7879999999999998,"15min":3.7879999999999998},"last":1.3580000000000001},{"interface":"front","average":{"1min":1.319,"5min":1.2829999999999999,"15min":1.2589999999999999},"min":{"1min":0.85799999999999998,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.9910000000000001,"5min":2.2890000000000001,"15min":2.2890000000000001},"last":1.224}]},{"osd":3,"last update":"Thu Oct 26 01:36:42 2023","interfaces":[{"interface":"back","average":{"1min":1.391,"5min":1.3080000000000001,"15min":1.2070000000000001},"min":{"1min":0.871,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.8600000000000001,"5min":4.4560000000000004,"15min":4.4560000000000004},"last":1.595},{"interface":"front","average":{"1min":1.4119999999999999,"5min":1.341,"15min":1.327},"min":{"1min":1.0369999999999999,"5min":0.82299999999999995,"15min":0.69499999999999995},"max":{"1min":1.877,"5min":2.5019999999999998,"15min":2.5019999999999998},"last":1.3089999999999999}]},{"osd":4,"last update":"Thu Oct 26 01:36:42 2023","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.252,"15min":1.22},"min":{"1min":0.93200000000000005,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.7,"5min":2.9489999999999998,"15min":2.9489999999999998},"last":0.99399999999999999},{"interface":"front","average":{"1min":1.4339999999999999,"5min":1.4330000000000001,"15min":1.3140000000000001},"min":{"1min":0.93400000000000005,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.9039999999999999,"5min":6.609,"15min":6.609},"last":1.5109999999999999}]},{"osd":5,"last update":"Thu Oct 26 01:36:42 2023","interfaces":[{"interface":"back","average":{"1min":1.6339999999999999,"5min":1.548,"15min":1.4430000000000001},"min":{"1min":1.2030000000000001,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":2.1240000000000001,"5min":5.4370000000000003,"15min":5.4370000000000003},"last":1.6950000000000001},{"interface":"front","average":{"1min":1.5309999999999999,"5min":1.448,"15min":1.409},"min":{"1min":1.1319999999999999,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.8169999999999999,"5min":2.621,"15min":2.621},"last":1.403}]},{"osd":7,"last update":"Thu Oct 26 01:36:42 2023","interfaces":[{"interface":"back","average":{"1min":1.4650000000000001,"5min":1.5229999999999999,"15min":1.411},"min":{"1min":1.2589999999999999,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.9770000000000001,"5min":11.804,"15min":11.804},"last":1.6439999999999999},{"interface":"front","average":{"1min":1.6299999999999999,"5min":1.657,"15min":1.5049999999999999},"min":{"1min":1.296,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.992,"5min":11.541,"15min":11.541},"last":1.79}]}]},{"osd":5,"up_from":13,"seq":55834574925,"num_pgs":16,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":540,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194772,"statfs":{"total":200000000,"available":199446562,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:36:44 2023","interfaces":[{"interface":"back","average":{"1min":1.143,"5min":1.1739999999999999,"15min":1.1240000000000001},"min":{"1min":0.746,"5min":0.66200000000000003,"15min":0.63200000000000001},"max":{"1min":1.546,"5min":1.7390000000000001,"15min":1.7390000000000001},"last":1.042},{"interface":"front","average":{"1min":1.262,"5min":1.2709999999999999,"15min":1.177},"min":{"1min":0.81699999999999995,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.601,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":1.0980000000000001}]},{"osd":1,"last update":"Thu Oct 26 01:36:44 2023","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.1439999999999999,"15min":1.091},"min":{"1min":0.80700000000000005,"5min":0.67600000000000005,"15min":0.66400000000000003},"max":{"1min":1.7829999999999999,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":0.90600000000000003},{"interface":"front","average":{"1min":1.1040000000000001,"5min":1.1599999999999999,"15min":1.095},"min":{"1min":0.85799999999999998,"5min":0.68200000000000005,"15min":0.60599999999999998},"max":{"1min":1.4570000000000001,"5min":1.6000000000000001,"15min":1.7669999999999999},"last":1.341}]},{"osd":2,"last update":"Thu Oct 26 01:36:44 2023","interfaces":[{"interface":"back","average":{"1min":1.2290000000000001,"5min":1.204,"15min":1.1619999999999999},"min":{"1min":0.79600000000000004,"5min":0.74399999999999999,"15min":0.69299999999999995},"max":{"1min":1.6879999999999999,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":0.79600000000000004},{"interface":"front","average":{"1min":1.3320000000000001,"5min":1.2350000000000001,"15min":1.1850000000000001},"min":{"1min":0.93600000000000005,"5min":0.85499999999999998,"15min":0.63900000000000001},"max":{"1min":1.6899999999999999,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":1.1930000000000001}]},{"osd":3,"last update":"Thu Oct 26 01:36:44 2023","interfaces":[{"interface":"back","average":{"1min":1.2809999999999999,"5min":1.3080000000000001,"15min":1.246},"min":{"1min":0.92200000000000004,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.613,"5min":1.7649999999999999,"15min":1.835},"last":1.462},{"interface":"front","average":{"1min":1.373,"5min":1.321,"15min":1.2949999999999999},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.788,"5min":1.788,"15min":2.9249999999999998},"last":1.298}]},{"osd":4,"last update":"Thu Oct 26 01:36:44 2023","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.286,"15min":1.206},"min":{"1min":0.97699999999999998,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.845,"5min":1.845,"15min":1.845},"last":1.6559999999999999},{"interface":"front","average":{"1min":1.4810000000000001,"5min":1.4470000000000001,"15min":1.353},"min":{"1min":0.90600000000000003,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.7849999999999999,"5min":1.867,"15min":1.867},"last":1.5840000000000001}]},{"osd":6,"last update":"Thu Oct 26 01:36:44 2023","interfaces":[{"interface":"back","average":{"1min":1.5269999999999999,"5min":1.444,"15min":1.3220000000000001},"min":{"1min":1.0660000000000001,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.883,"5min":2.8250000000000002,"15min":2.8250000000000002},"last":1.6080000000000001},{"interface":"front","average":{"1min":1.5429999999999999,"5min":1.4490000000000001,"15min":1.3260000000000001},"min":{"1min":1.1879999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.849,"5min":1.849,"15min":1.8560000000000001},"last":1.5469999999999999}]},{"osd":7,"last update":"Thu Oct 26 01:36:44 2023","interfaces":[{"interface":"back","average":{"1min":1.599,"5min":1.5449999999999999,"15min":1.417},"min":{"1min":1.119,"5min":0.64000000000000001,"15min":0.61699999999999999},"max":{"1min":2.1040000000000001,"5min":2.1040000000000001,"15min":2.1040000000000001},"last":1.415},{"interface":"front","average":{"1min":1.6619999999999999,"5min":1.5660000000000001,"15min":1.4299999999999999},"min":{"1min":1.389,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":2.0960000000000001,"5min":2.0960000000000001,"15min":2.0960000000000001},"last":1.7090000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574925,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":545,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194767,"statfs":{"total":200000000,"available":199441896,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:36:37 2023","interfaces":[{"interface":"back","average":{"1min":1.081,"5min":1.04,"15min":0.97699999999999998},"min":{"1min":0.69199999999999995,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.5309999999999999,"5min":2.0289999999999999,"15min":2.0289999999999999},"last":1.3089999999999999},{"interface":"front","average":{"1min":1.071,"5min":1.0649999999999999,"15min":1.0229999999999999},"min":{"1min":0.66400000000000003,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.6599999999999999,"5min":2.6829999999999998,"15min":2.6829999999999998},"last":1.5029999999999999}]},{"osd":1,"last update":"Thu Oct 26 01:36:37 2023","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.1799999999999999,"15min":1.1220000000000001},"min":{"1min":0.98399999999999999,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.599,"5min":4.0519999999999996,"15min":4.0519999999999996},"last":1.1180000000000001},{"interface":"front","average":{"1min":1.119,"5min":1.1439999999999999,"15min":1.105},"min":{"1min":0.77900000000000003,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.6240000000000001,"5min":2.0760000000000001,"15min":2.0760000000000001},"last":1.4470000000000001}]},{"osd":2,"last update":"Thu Oct 26 01:36:37 2023","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":1.3280000000000001,"15min":1.222},"min":{"1min":0.79100000000000004,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.5660000000000001,"5min":5.6749999999999998,"15min":5.6749999999999998},"last":1.6819999999999999},{"interface":"front","average":{"1min":1.1970000000000001,"5min":1.1879999999999999,"15min":1.1220000000000001},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.746,"5min":1.849,"15min":1.849},"last":1.0700000000000001}]},{"osd":3,"last update":"Thu Oct 26 01:36:37 2023","interfaces":[{"interface":"back","average":{"1min":1.286,"5min":1.2,"15min":1.1839999999999999},"min":{"1min":0.84399999999999997,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.847,"5min":1.891,"15min":1.891},"last":1.5449999999999999},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.2030000000000001,"15min":1.155},"min":{"1min":0.94499999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.5669999999999999,"5min":1.7210000000000001,"15min":1.7210000000000001},"last":1.391}]},{"osd":5,"last update":"Thu Oct 26 01:36:37 2023","interfaces":[{"interface":"back","average":{"1min":1.2749999999999999,"5min":1.2509999999999999,"15min":1.286},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.46300000000000002},"max":{"1min":2.1429999999999998,"5min":2.1429999999999998,"15min":2.1429999999999998},"last":1.196},{"interface":"front","average":{"1min":1.28,"5min":1.331,"15min":1.327},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.716,"5min":5.8579999999999997,"15min":5.8579999999999997},"last":1.7529999999999999}]},{"osd":6,"last update":"Thu Oct 26 01:36:37 2023","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":1.4199999999999999,"15min":1.3340000000000001},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.639,"5min":12.343999999999999,"15min":12.343999999999999},"last":1.2629999999999999},{"interface":"front","average":{"1min":1.54,"5min":1.6220000000000001,"15min":1.5129999999999999},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.55000000000000004},"max":{"1min":2.0369999999999999,"5min":12.605,"15min":12.605},"last":1.6379999999999999}]},{"osd":7,"last update":"Thu Oct 26 01:36:37 2023","interfaces":[{"interface":"back","average":{"1min":1.4650000000000001,"5min":1.518,"15min":1.4910000000000001},"min":{"1min":0.748,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":2.762,"5min":5.774,"15min":5.774},"last":1.363},{"interface":"front","average":{"1min":1.5249999999999999,"5min":1.518,"15min":1.4530000000000001},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":2.8330000000000002,"5min":4.6520000000000001,"15min":4.6520000000000001},"last":1.5840000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574925,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":539,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194773,"statfs":{"total":200000000,"available":199447614,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":0.875,"15min":0.86099999999999999},"min":{"1min":0.63,"5min":0.308,"15min":0.308},"max":{"1min":1.5249999999999999,"5min":6.3200000000000003,"15min":6.3200000000000003},"last":0.90700000000000003},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.0229999999999999,"15min":0.93600000000000005},"min":{"1min":0.52000000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.7130000000000001,"5min":4.2619999999999996,"15min":4.2619999999999996},"last":0.77600000000000002}]},{"osd":1,"last update":"Thu Oct 26 01:36:41 2023","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":0.93200000000000005,"15min":0.90500000000000003},"min":{"1min":0.66700000000000004,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.629,"5min":5.1260000000000003,"15min":5.1260000000000003},"last":1.0309999999999999},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.0649999999999999,"15min":1.012},"min":{"1min":0.59599999999999997,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.873,"5min":6.1520000000000001,"15min":6.1520000000000001},"last":1.109}]},{"osd":3,"last update":"Thu Oct 26 01:35:43 2023","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.98699999999999999,"15min":0.92500000000000004},"min":{"1min":0.55200000000000005,"5min":0.443,"15min":0.443},"max":{"1min":1.3600000000000001,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":1.149},{"interface":"front","average":{"1min":1.1080000000000001,"5min":1.115,"15min":1.0009999999999999},"min":{"1min":0.624,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.577,"5min":8.9019999999999992,"15min":8.9019999999999992},"last":1.27}]},{"osd":4,"last update":"Thu Oct 26 01:36:41 2023","interfaces":[{"interface":"back","average":{"1min":1.454,"5min":1.365,"15min":1.268},"min":{"1min":0.80500000000000005,"5min":0.751,"15min":0.73899999999999999},"max":{"1min":2.6190000000000002,"5min":6.2670000000000003,"15min":6.2670000000000003},"last":1.3620000000000001},{"interface":"front","average":{"1min":1.5089999999999999,"5min":1.367,"15min":1.2629999999999999},"min":{"1min":0.93700000000000006,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":2.4359999999999999,"5min":7.343,"15min":7.343},"last":1.399}]},{"osd":5,"last update":"Thu Oct 26 01:36:41 2023","interfaces":[{"interface":"back","average":{"1min":1.5649999999999999,"5min":1.369,"15min":1.2909999999999999},"min":{"1min":1.018,"5min":0.80800000000000005,"15min":0.71899999999999997},"max":{"1min":2.5790000000000002,"5min":3.7170000000000001,"15min":3.7170000000000001},"last":1.206},{"interface":"front","average":{"1min":1.4650000000000001,"5min":1.3540000000000001,"15min":1.3},"min":{"1min":0.88800000000000001,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":2.298,"5min":6.3899999999999997,"15min":6.3899999999999997},"last":1.2969999999999999}]},{"osd":6,"last update":"Thu Oct 26 01:36:41 2023","interfaces":[{"interface":"back","average":{"1min":1.4690000000000001,"5min":1.4259999999999999,"15min":1.335},"min":{"1min":0.871,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":2.73,"5min":6.0490000000000004,"15min":6.0490000000000004},"last":1.4690000000000001},{"interface":"front","average":{"1min":1.599,"5min":1.4399999999999999,"15min":1.375},"min":{"1min":1.0660000000000001,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":2.4089999999999998,"5min":5.0650000000000004,"15min":5.0650000000000004},"last":1.327}]},{"osd":7,"last update":"Thu Oct 26 01:36:41 2023","interfaces":[{"interface":"back","average":{"1min":1.532,"5min":1.4390000000000001,"15min":1.405},"min":{"1min":1.0449999999999999,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":2.3370000000000002,"5min":6.4480000000000004,"15min":6.4480000000000004},"last":1.226},{"interface":"front","average":{"1min":1.6279999999999999,"5min":1.5169999999999999,"15min":1.4570000000000001},"min":{"1min":1.117,"5min":0.91800000000000004,"15min":0.91800000000000004},"max":{"1min":2.8290000000000002,"5min":2.8439999999999999,"15min":2.8439999999999999},"last":1.5620000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574925,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":539,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194773,"statfs":{"total":200000000,"available":199448002,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:35:40 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.76100000000000001,"15min":0.71399999999999997},"min":{"1min":0.38,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.147,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.872,"5min":0.82099999999999995,"15min":0.78300000000000003},"min":{"1min":0.40899999999999997,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.6399999999999999,"5min":1.6399999999999999,"15min":1.6399999999999999},"last":0.36299999999999999}]},{"osd":2,"last update":"Thu Oct 26 01:35:40 2023","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.95699999999999996,"15min":0.997},"min":{"1min":0.42899999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.4970000000000001,"5min":2.4830000000000001,"15min":2.4830000000000001},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.86099999999999999,"15min":0.84899999999999998},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.296,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.64300000000000002}]},{"osd":3,"last update":"Thu Oct 26 01:35:46 2023","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.96299999999999997,"15min":0.93999999999999995},"min":{"1min":0.44700000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.194,"5min":1.75,"15min":1.75},"last":0.42499999999999999},{"interface":"front","average":{"1min":1.0189999999999999,"5min":1.004,"15min":0.98099999999999998},"min":{"1min":0.48799999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.361,"5min":1.802,"15min":1.802},"last":0.625}]},{"osd":4,"last update":"Thu Oct 26 01:35:40 2023","interfaces":[{"interface":"back","average":{"1min":1.177,"5min":1.2130000000000001,"15min":1.1919999999999999},"min":{"1min":0.80800000000000005,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.5069999999999999,"5min":2.0659999999999998,"15min":2.0659999999999998},"last":0.69099999999999995},{"interface":"front","average":{"1min":1.099,"5min":1.151,"15min":1.1220000000000001},"min":{"1min":0.76600000000000001,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.5920000000000001,"5min":1.637,"15min":1.637},"last":0.88200000000000001}]},{"osd":5,"last update":"Thu Oct 26 01:35:40 2023","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.238,"15min":1.224},"min":{"1min":0.94799999999999995,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":1.746,"5min":1.746,"15min":1.746},"last":1.1359999999999999},{"interface":"front","average":{"1min":1.3740000000000001,"5min":1.3400000000000001,"15min":1.3320000000000001},"min":{"1min":0.78800000000000003,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.772,"5min":2.2200000000000002,"15min":2.2200000000000002},"last":0.80300000000000005}]},{"osd":6,"last update":"Thu Oct 26 01:35:40 2023","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":1.2969999999999999,"15min":1.2709999999999999},"min":{"1min":0.90000000000000002,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.8340000000000001,"5min":2.3780000000000001,"15min":2.3780000000000001},"last":0.70199999999999996},{"interface":"front","average":{"1min":1.246,"5min":1.256,"15min":1.236},"min":{"1min":0.82599999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.6060000000000001,"5min":1.984,"15min":1.984},"last":0.89600000000000002}]},{"osd":7,"last update":"Thu Oct 26 01:35:40 2023","interfaces":[{"interface":"back","average":{"1min":1.2410000000000001,"5min":1.244,"15min":1.262},"min":{"1min":0.84199999999999997,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.798,"5min":2.6930000000000001,"15min":2.6930000000000001},"last":1.1779999999999999},{"interface":"front","average":{"1min":1.357,"5min":1.351,"15min":1.319},"min":{"1min":0.81399999999999995,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.7310000000000001,"5min":1.869,"15min":1.869},"last":0.95299999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574925,"num_pgs":8,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1115,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194197,"statfs":{"total":200000000,"available":198857794,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Oct 26 01:36:43 2023","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.998,"15min":0.92300000000000004},"min":{"1min":0.64000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.49,"5min":19.425000000000001,"15min":19.425000000000001},"last":1.1319999999999999},{"interface":"front","average":{"1min":0.98199999999999998,"5min":1.075,"15min":1.0549999999999999},"min":{"1min":0.67400000000000004,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.339,"5min":15.162000000000001,"15min":15.162000000000001},"last":0.86599999999999999}]},{"osd":2,"last update":"Thu Oct 26 01:36:43 2023","interfaces":[{"interface":"back","average":{"1min":1,"5min":1.2969999999999999,"15min":1.1930000000000001},"min":{"1min":0.59899999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.4970000000000001,"5min":28.439,"15min":28.439},"last":1.399},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.90800000000000003,"15min":0.91400000000000003},"min":{"1min":0.69399999999999995,"5min":0.45200000000000001,"15min":0.38300000000000001},"max":{"1min":1.268,"5min":1.502,"15min":4.9459999999999997},"last":0.98199999999999998}]},{"osd":3,"last update":"Thu Oct 26 01:36:43 2023","interfaces":[{"interface":"back","average":{"1min":1.101,"5min":1.0469999999999999,"15min":1.093},"min":{"1min":0.64100000000000001,"5min":0.46000000000000002,"15min":0.44600000000000001},"max":{"1min":1.47,"5min":1.867,"15min":4.9800000000000004},"last":1.095},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.1359999999999999,"15min":1.1619999999999999},"min":{"1min":0.752,"5min":0.56699999999999995,"15min":0.54000000000000004},"max":{"1min":1.4450000000000001,"5min":1.992,"15min":5.016},"last":1.1859999999999999}]},{"osd":4,"last update":"Thu Oct 26 01:36:43 2023","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.387,"15min":1.3009999999999999},"min":{"1min":0.82799999999999996,"5min":0.66800000000000004,"15min":0.61499999999999999},"max":{"1min":1.681,"5min":19.329000000000001,"15min":19.329000000000001},"last":1.359},{"interface":"front","average":{"1min":1.319,"5min":1.27,"15min":1.306},"min":{"1min":0.82499999999999996,"5min":0.79100000000000004,"15min":0.67600000000000005},"max":{"1min":1.718,"5min":2.036,"15min":5.5229999999999997},"last":1.3049999999999999}]},{"osd":5,"last update":"Thu Oct 26 01:36:43 2023","interfaces":[{"interface":"back","average":{"1min":1.5129999999999999,"5min":1.4039999999999999,"15min":1.4350000000000001},"min":{"1min":1.1359999999999999,"5min":0.80000000000000004,"15min":0.72499999999999998},"max":{"1min":1.857,"5min":2.2290000000000001,"15min":5.6360000000000001},"last":1.595},{"interface":"front","average":{"1min":1.399,"5min":1.492,"15min":1.4119999999999999},"min":{"1min":1.0880000000000001,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.8200000000000001,"5min":19.384,"15min":19.384},"last":1.643}]},{"osd":6,"last update":"Thu Oct 26 01:36:43 2023","interfaces":[{"interface":"back","average":{"1min":1.3580000000000001,"5min":1.375,"15min":1.3720000000000001},"min":{"1min":1.034,"5min":0.77200000000000002,"15min":0.73799999999999999},"max":{"1min":1.6970000000000001,"5min":8.2609999999999992,"15min":8.2609999999999992},"last":1.5089999999999999},{"interface":"front","average":{"1min":1.377,"5min":1.4370000000000001,"15min":1.4099999999999999},"min":{"1min":0.85899999999999999,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.9450000000000001,"5min":11.849,"15min":11.849},"last":1.4730000000000001}]},{"osd":7,"last update":"Thu Oct 26 01:36:43 2023","interfaces":[{"interface":"back","average":{"1min":1.4970000000000001,"5min":1.4450000000000001,"15min":1.488},"min":{"1min":0.999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.7809999999999999,"5min":2.4060000000000001,"15min":5.6689999999999996},"last":1.4430000000000001},{"interface":"front","average":{"1min":1.4930000000000001,"5min":1.6080000000000001,"15min":1.5089999999999999},"min":{"1min":1.1579999999999999,"5min":0.81299999999999994,"15min":0.69399999999999995},"max":{"1min":1.9390000000000001,"5min":19.245999999999999,"15min":19.245999999999999},"last":1.7110000000000001}]}]}],"pool_statfs":[]}} 2023-10-26T01:36:49.387 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-26T01:36:49.388 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-26T01:36:49.658 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:49.656+0000 7f49aad47640 1 -- 172.21.15.169:0/1197286741 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f49a4099170 msgr2=0x7f49a40afef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:49.658 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:49.656+0000 7f49aad47640 1 --2- 172.21.15.169:0/1197286741 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f49a4099170 0x7f49a40afef0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f4994005fd0 tx=0x7f499402bd80 comp rx=0 tx=0).stop 2023-10-26T01:36:49.658 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:49.656+0000 7f49aad47640 1 -- 172.21.15.169:0/1197286741 shutdown_connections 2023-10-26T01:36:49.659 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:49.656+0000 7f49aad47640 1 --2- 172.21.15.169:0/1197286741 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f49a40b0430 0x7f49a406ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:49.659 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:49.656+0000 7f49aad47640 1 --2- 172.21.15.169:0/1197286741 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f49a4099170 0x7f49a40afef0 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:49.659 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:49.656+0000 7f49aad47640 1 -- 172.21.15.169:0/1197286741 >> 172.21.15.169:0/1197286741 conn(0x7f49a4069b70 msgr2=0x7f49a40aebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:49.659 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:49.656+0000 7f49aad47640 1 -- 172.21.15.169:0/1197286741 shutdown_connections 2023-10-26T01:36:49.659 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:49.656+0000 7f49aad47640 1 -- 172.21.15.169:0/1197286741 wait complete. 2023-10-26T01:36:49.661 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:49.658+0000 7f49aad47640 1 Processor -- start 2023-10-26T01:36:49.661 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:49.658+0000 7f49aad47640 1 -- start start 2023-10-26T01:36:49.662 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:49.659+0000 7f49aad47640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f49a406cad0 0x7f49a40cbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:49.662 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:49.659+0000 7f49aad47640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f49a4099170 0x7f49a40c9990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:49.662 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:49.659+0000 7f49aad47640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f49a40b0430 0x7f49a40aa2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:49.662 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:49.659+0000 7f49aad47640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49a40c7760 con 0x7f49a40b0430 2023-10-26T01:36:49.662 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:49.659+0000 7f49aad47640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49a40caa10 con 0x7f49a4099170 2023-10-26T01:36:49.663 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:49.659+0000 7f49aad47640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49a415b590 con 0x7f49a406cad0 2023-10-26T01:36:49.663 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:49.659+0000 7f49aa546640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f49a40b0430 0x7f49a40aa2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:49.663 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:49.659+0000 7f49aa546640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f49a40b0430 0x7f49a40aa2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:59530/0 (socket says 172.21.15.169:59530) 2023-10-26T01:36:49.663 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:49.659+0000 7f49aa546640 1 -- 172.21.15.169:0/2287321319 learned_addr learned my addr 172.21.15.169:0/2287321319 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:36:49.663 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:49.659+0000 7f49a9544640 1 --2- 172.21.15.169:0/2287321319 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f49a4099170 0x7f49a40c9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:49.664 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:49.659+0000 7f49aa546640 1 -- 172.21.15.169:0/2287321319 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f49a406cad0 msgr2=0x7f49a40cbaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:49.664 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:49.659+0000 7f49aa546640 1 --2- 172.21.15.169:0/2287321319 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f49a406cad0 0x7f49a40cbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:49.664 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:49.659+0000 7f49aa546640 1 -- 172.21.15.169:0/2287321319 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f49a4099170 msgr2=0x7f49a40c9990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:49.664 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:49.660+0000 7f49aa546640 1 --2- 172.21.15.169:0/2287321319 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f49a4099170 0x7f49a40c9990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:49.664 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:49.660+0000 7f49aa546640 1 -- 172.21.15.169:0/2287321319 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4994009560 con 0x7f49a40b0430 2023-10-26T01:36:49.664 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:49.660+0000 7f49a9544640 1 --2- 172.21.15.169:0/2287321319 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f49a4099170 0x7f49a40c9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:36:49.666 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:49.660+0000 7f49aa546640 1 --2- 172.21.15.169:0/2287321319 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f49a40b0430 0x7f49a40aa2c0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f499c00cae0 tx=0x7f499c007220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:49.666 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:49.660+0000 7f499a7fc640 1 -- 172.21.15.169:0/2287321319 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f499c007da0 con 0x7f49a40b0430 2023-10-26T01:36:49.666 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:49.660+0000 7f499a7fc640 1 -- 172.21.15.169:0/2287321319 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f499c004b20 con 0x7f49a40b0430 2023-10-26T01:36:49.666 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:49.660+0000 7f49aad47640 1 -- 172.21.15.169:0/2287321319 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49a40ca450 con 0x7f49a40b0430 2023-10-26T01:36:49.667 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:49.660+0000 7f499a7fc640 1 -- 172.21.15.169:0/2287321319 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f499c005020 con 0x7f49a40b0430 2023-10-26T01:36:49.667 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:49.660+0000 7f49aad47640 1 -- 172.21.15.169:0/2287321319 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49a40c73f0 con 0x7f49a40b0430 2023-10-26T01:36:49.667 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:49.661+0000 7f49aad47640 1 -- 172.21.15.169:0/2287321319 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49a4066ef0 con 0x7f49a40b0430 2023-10-26T01:36:49.667 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:49.662+0000 7f499a7fc640 1 -- 172.21.15.169:0/2287321319 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f499c005180 con 0x7f49a40b0430 2023-10-26T01:36:49.667 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:49.663+0000 7f499a7fc640 1 --2- 172.21.15.169:0/2287321319 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f498806ae40 0x7f498806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:49.668 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:49.663+0000 7f499a7fc640 1 -- 172.21.15.169:0/2287321319 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f499c08a0b0 con 0x7f49a40b0430 2023-10-26T01:36:49.668 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:49.664+0000 7f49a9d45640 1 --2- 172.21.15.169:0/2287321319 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f498806ae40 0x7f498806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:49.668 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:49.665+0000 7f49a9d45640 1 --2- 172.21.15.169:0/2287321319 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f498806ae40 0x7f498806d300 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f49a0003cc0 tx=0x7f49a00079b0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:49.669 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:49.667+0000 7f499a7fc640 1 -- 172.21.15.169:0/2287321319 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f499c0551f0 con 0x7f49a40b0430 2023-10-26T01:36:49.921 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:49.919+0000 7f49aad47640 1 -- 172.21.15.169:0/2287321319 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f49a4073c10 con 0x7f49a40b0430 2023-10-26T01:36:49.925 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:49.923+0000 7f499a7fc640 1 -- 172.21.15.169:0/2287321319 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) v1 ==== 190+0+147 (secure 0 0 0) 0x7f499c01d0e0 con 0x7f49a40b0430 2023-10-26T01:36:49.926 INFO:teuthology.orchestra.run.smithi169.stdout:[client.0] 2023-10-26T01:36:49.926 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQAgwTllKC7gOhAAPsvxGcuUA8/p8xqbUgGteA== 2023-10-26T01:36:49.926 INFO:teuthology.orchestra.run.smithi169.stdout: caps mds = "allow" 2023-10-26T01:36:49.926 INFO:teuthology.orchestra.run.smithi169.stdout: caps mon = "allow r" 2023-10-26T01:36:49.927 INFO:teuthology.orchestra.run.smithi169.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-10-26T01:36:49.927 INFO:teuthology.orchestra.run.smithi169.stderr:updated caps for client.0 2023-10-26T01:36:49.928 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:49.926+0000 7f49aad47640 1 -- 172.21.15.169:0/2287321319 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f498806ae40 msgr2=0x7f498806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:49.928 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:49.926+0000 7f49aad47640 1 --2- 172.21.15.169:0/2287321319 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f498806ae40 0x7f498806d300 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f49a0003cc0 tx=0x7f49a00079b0 comp rx=0 tx=0).stop 2023-10-26T01:36:49.928 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:49.926+0000 7f49aad47640 1 -- 172.21.15.169:0/2287321319 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f49a40b0430 msgr2=0x7f49a40aa2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:49.929 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:49.926+0000 7f49aad47640 1 --2- 172.21.15.169:0/2287321319 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f49a40b0430 0x7f49a40aa2c0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f499c00cae0 tx=0x7f499c007220 comp rx=0 tx=0).stop 2023-10-26T01:36:49.929 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:49.926+0000 7f49aad47640 1 -- 172.21.15.169:0/2287321319 shutdown_connections 2023-10-26T01:36:49.929 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:49.926+0000 7f49aad47640 1 --2- 172.21.15.169:0/2287321319 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f498806ae40 0x7f498806d300 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:49.929 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:49.926+0000 7f49aad47640 1 --2- 172.21.15.169:0/2287321319 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f49a40b0430 0x7f49a40aa2c0 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:49.930 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:49.926+0000 7f49aad47640 1 --2- 172.21.15.169:0/2287321319 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f49a4099170 0x7f49a40c9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:49.930 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:49.926+0000 7f49aad47640 1 --2- 172.21.15.169:0/2287321319 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f49a406cad0 0x7f49a40cbaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:49.930 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:49.926+0000 7f49aad47640 1 -- 172.21.15.169:0/2287321319 >> 172.21.15.169:0/2287321319 conn(0x7f49a4069b70 msgr2=0x7f49a4143dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:49.930 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:49.926+0000 7f49aad47640 1 -- 172.21.15.169:0/2287321319 shutdown_connections 2023-10-26T01:36:49.930 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:49.926+0000 7f49aad47640 1 -- 172.21.15.169:0/2287321319 wait complete. 2023-10-26T01:36:49.947 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T01:36:50.163 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:50.161+0000 7fc20f3ac640 1 -- 172.21.15.169:0/338845663 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1f4003970 msgr2=0x7fc1f4003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:50.164 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:50.161+0000 7fc20f3ac640 1 --2- 172.21.15.169:0/338845663 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1f4003970 0x7fc1f4003d90 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fc20400baf0 tx=0x7fc1f40057f0 comp rx=0 tx=0).stop 2023-10-26T01:36:50.164 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:50.161+0000 7fc20ebab640 1 -- 172.21.15.169:0/338845663 reap_dead start 2023-10-26T01:36:50.164 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:50.161+0000 7fc20f3ac640 1 -- 172.21.15.169:0/338845663 shutdown_connections 2023-10-26T01:36:50.164 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:50.161+0000 7fc20f3ac640 1 -- 172.21.15.169:0/338845663 >> 172.21.15.169:0/338845663 conn(0x7fc2080aedd0 msgr2=0x7fc2080af1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:50.164 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:50.161+0000 7fc20f3ac640 1 -- 172.21.15.169:0/338845663 shutdown_connections 2023-10-26T01:36:50.165 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:50.161+0000 7fc20f3ac640 1 -- 172.21.15.169:0/338845663 wait complete. 2023-10-26T01:36:50.166 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:50.163+0000 7fc20f3ac640 1 Processor -- start 2023-10-26T01:36:50.166 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:50.163+0000 7fc20f3ac640 1 -- start start 2023-10-26T01:36:50.166 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:50.164+0000 7fc20f3ac640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1f4003970 0x7fc2080711a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:50.166 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:50.164+0000 7fc20f3ac640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc20806e8f0 0x7fc208158020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:50.167 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:50.164+0000 7fc20f3ac640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc20813ebd0 0x7fc2080c8520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:50.167 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:50.164+0000 7fc20f3ac640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc208158560 con 0x7fc1f4003970 2023-10-26T01:36:50.167 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:50.164+0000 7fc20f3ac640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc20813f3b0 con 0x7fc20813ebd0 2023-10-26T01:36:50.167 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:50.164+0000 7fc20f3ac640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc20813f690 con 0x7fc20806e8f0 2023-10-26T01:36:50.167 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:50.164+0000 7fc20e3aa640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1f4003970 0x7fc2080711a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:50.167 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:50.164+0000 7fc20e3aa640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1f4003970 0x7fc2080711a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:59552/0 (socket says 172.21.15.169:59552) 2023-10-26T01:36:50.168 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:50.164+0000 7fc20e3aa640 1 -- 172.21.15.169:0/3360933788 learned_addr learned my addr 172.21.15.169:0/3360933788 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:36:50.169 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:50.165+0000 7fc20e3aa640 1 -- 172.21.15.169:0/3360933788 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc20806e8f0 msgr2=0x7fc208158020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:50.169 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:50.165+0000 7fc20e3aa640 1 --2- 172.21.15.169:0/3360933788 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc20806e8f0 0x7fc208158020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:50.169 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:50.165+0000 7fc20e3aa640 1 -- 172.21.15.169:0/3360933788 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc20813ebd0 msgr2=0x7fc2080c8520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:50.169 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:50.165+0000 7fc20e3aa640 1 --2- 172.21.15.169:0/3360933788 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc20813ebd0 0x7fc2080c8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:50.170 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:50.165+0000 7fc20e3aa640 1 -- 172.21.15.169:0/3360933788 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc20813f970 con 0x7fc1f4003970 2023-10-26T01:36:50.170 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:50.165+0000 7fc20e3aa640 1 --2- 172.21.15.169:0/3360933788 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1f4003970 0x7fc2080711a0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fc20400ba40 tx=0x7fc204036720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:50.170 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:50.165+0000 7fc1feffd640 1 -- 172.21.15.169:0/3360933788 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc204036d20 con 0x7fc1f4003970 2023-10-26T01:36:50.170 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:50.165+0000 7fc1feffd640 1 -- 172.21.15.169:0/3360933788 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc204041470 con 0x7fc1f4003970 2023-10-26T01:36:50.170 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:50.165+0000 7fc20f3ac640 1 -- 172.21.15.169:0/3360933788 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc20813fc50 con 0x7fc1f4003970 2023-10-26T01:36:50.171 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:50.165+0000 7fc1feffd640 1 -- 172.21.15.169:0/3360933788 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc204041840 con 0x7fc1f4003970 2023-10-26T01:36:50.171 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:50.165+0000 7fc20f3ac640 1 -- 172.21.15.169:0/3360933788 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc20813f050 con 0x7fc1f4003970 2023-10-26T01:36:50.171 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:50.167+0000 7fc20f3ac640 1 -- 172.21.15.169:0/3360933788 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc208073c10 con 0x7fc1f4003970 2023-10-26T01:36:50.171 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:50.167+0000 7fc1feffd640 1 -- 172.21.15.169:0/3360933788 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fc2040419a0 con 0x7fc1f4003970 2023-10-26T01:36:50.172 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:50.168+0000 7fc1feffd640 1 --2- 172.21.15.169:0/3360933788 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc1ec06ae70 0x7fc1ec06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:50.172 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:50.168+0000 7fc1feffd640 1 -- 172.21.15.169:0/3360933788 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc2040ac0c0 con 0x7fc1f4003970 2023-10-26T01:36:50.173 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:50.169+0000 7fc20dba9640 1 --2- 172.21.15.169:0/3360933788 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc1ec06ae70 0x7fc1ec06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:50.173 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:50.170+0000 7fc20dba9640 1 --2- 172.21.15.169:0/3360933788 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc1ec06ae70 0x7fc1ec06d330 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fc1f80012b0 tx=0x7fc1f8001240 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:50.174 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:50.173+0000 7fc1feffd640 1 -- 172.21.15.169:0/3360933788 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc204077170 con 0x7fc1f4003970 2023-10-26T01:36:50.402 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:50.400+0000 7fc20f3ac640 1 -- 172.21.15.169:0/3360933788 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc2080cc6e0 con 0x7fc1f4003970 2023-10-26T01:36:50.402 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:50.401+0000 7fc1feffd640 1 -- 172.21.15.169:0/3360933788 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+4924 (secure 0 0 0) 0x7fc20407a790 con 0x7fc1f4003970 2023-10-26T01:36:50.403 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:36:50.403 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":50,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5240,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/3870332417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":3870332417},{"type":"v1","addr":"172.21.15.170:6837","nonce":3870332417}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45},{"gid":5497,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/2533771698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2533771698},{"type":"v1","addr":"172.21.15.170:6835","nonce":2533771698}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45},{"gid":6885,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/2117946713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2117946713},{"type":"v1","addr":"172.21.15.169:6834","nonce":2117946713}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T01:36:43.849046+0000","modified":"2023-10-26T01:36:46.926339+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6888},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6888":{"gid":6888,"name":"c","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6835/657142078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":657142078},{"type":"v1","addr":"172.21.15.169:6835","nonce":657142078}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-10-26T01:36:50.404 INFO:teuthology.orchestra.run.smithi169.stderr:dumped fsmap epoch 50 2023-10-26T01:36:50.405 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:50.403+0000 7fc20f3ac640 1 -- 172.21.15.169:0/3360933788 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc1ec06ae70 msgr2=0x7fc1ec06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:50.405 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:50.403+0000 7fc20f3ac640 1 --2- 172.21.15.169:0/3360933788 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc1ec06ae70 0x7fc1ec06d330 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fc1f80012b0 tx=0x7fc1f8001240 comp rx=0 tx=0).stop 2023-10-26T01:36:50.406 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:50.403+0000 7fc20f3ac640 1 -- 172.21.15.169:0/3360933788 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1f4003970 msgr2=0x7fc2080711a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:50.406 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:50.403+0000 7fc20f3ac640 1 --2- 172.21.15.169:0/3360933788 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1f4003970 0x7fc2080711a0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fc20400ba40 tx=0x7fc204036720 comp rx=0 tx=0).stop 2023-10-26T01:36:50.406 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:50.403+0000 7fc20f3ac640 1 -- 172.21.15.169:0/3360933788 shutdown_connections 2023-10-26T01:36:50.406 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:50.403+0000 7fc20f3ac640 1 --2- 172.21.15.169:0/3360933788 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc1ec06ae70 0x7fc1ec06d330 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:50.407 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:50.403+0000 7fc20f3ac640 1 --2- 172.21.15.169:0/3360933788 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc20813ebd0 0x7fc2080c8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:50.407 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:50.403+0000 7fc20f3ac640 1 --2- 172.21.15.169:0/3360933788 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc20806e8f0 0x7fc208158020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:50.407 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:50.403+0000 7fc20f3ac640 1 --2- 172.21.15.169:0/3360933788 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1f4003970 0x7fc2080711a0 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:50.407 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:50.403+0000 7fc20f3ac640 1 -- 172.21.15.169:0/3360933788 >> 172.21.15.169:0/3360933788 conn(0x7fc2080aedd0 msgr2=0x7fc2080af1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:50.407 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:50.404+0000 7fc20f3ac640 1 -- 172.21.15.169:0/3360933788 shutdown_connections 2023-10-26T01:36:50.407 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:50.404+0000 7fc20f3ac640 1 -- 172.21.15.169:0/3360933788 wait complete. 2023-10-26T01:36:50.825 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 50, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-26T01:36:43.849046+0000', 'modified': '2023-10-26T01:36:46.926339+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6888}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6888': {'gid': 6888, 'name': 'c', 'rank': 0, 'incarnation': 46, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.169:6835/657142078', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.169:6833', 'nonce': 657142078}, {'type': 'v1', 'addr': '172.21.15.169:6835', 'nonce': 657142078}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-26T01:36:50.825 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-26T01:36:50.826 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-26T01:36:51.058 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.056+0000 7f08ebe75640 1 -- 172.21.15.169:0/2621267512 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08d0006290 msgr2=0x7f08d000ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:51.058 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.056+0000 7f08ebe75640 1 --2- 172.21.15.169:0/2621267512 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08d0006290 0x7f08d000ca90 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f08dc00c640 tx=0x7f08d0006900 comp rx=0 tx=0).stop 2023-10-26T01:36:51.059 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.056+0000 7f08ebe75640 1 -- 172.21.15.169:0/2621267512 shutdown_connections 2023-10-26T01:36:51.059 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.056+0000 7f08ebe75640 1 --2- 172.21.15.169:0/2621267512 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08d0006290 0x7f08d000ca90 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:51.059 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.056+0000 7f08ebe75640 1 -- 172.21.15.169:0/2621267512 >> 172.21.15.169:0/2621267512 conn(0x7f08e4069930 msgr2=0x7f08e4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:51.059 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.056+0000 7f08ebe75640 1 -- 172.21.15.169:0/2621267512 shutdown_connections 2023-10-26T01:36:51.060 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.056+0000 7f08ebe75640 1 -- 172.21.15.169:0/2621267512 wait complete. 2023-10-26T01:36:51.060 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.058+0000 7f08ebe75640 1 Processor -- start 2023-10-26T01:36:51.060 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.058+0000 7f08ebe75640 1 -- start start 2023-10-26T01:36:51.062 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.059+0000 7f08ebe75640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f08d0006290 0x7f08e41f22b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:51.063 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.059+0000 7f08ebe75640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f08e41e5de0 0x7f08e41536e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:51.063 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.059+0000 7f08ebe75640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08e41e3cf0 0x7f08e41e4170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:51.063 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.059+0000 7f08ebe75640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08e41e46b0 con 0x7f08e41e3cf0 2023-10-26T01:36:51.063 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.059+0000 7f08ebe75640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08e41f27f0 con 0x7f08d0006290 2023-10-26T01:36:51.064 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.059+0000 7f08ebe75640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08e413dbd0 con 0x7f08e41e5de0 2023-10-26T01:36:51.064 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.059+0000 7f08ea412640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08e41e3cf0 0x7f08e41e4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:51.064 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.059+0000 7f08ea412640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08e41e3cf0 0x7f08e41e4170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:59580/0 (socket says 172.21.15.169:59580) 2023-10-26T01:36:51.064 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.059+0000 7f08ea412640 1 -- 172.21.15.169:0/3596811507 learned_addr learned my addr 172.21.15.169:0/3596811507 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:36:51.064 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.060+0000 7f08e9410640 1 --2- 172.21.15.169:0/3596811507 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f08e41e5de0 0x7f08e41536e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:51.065 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.060+0000 7f08e9c11640 1 --2- 172.21.15.169:0/3596811507 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f08d0006290 0x7f08e41f22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:51.065 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.060+0000 7f08ea412640 1 -- 172.21.15.169:0/3596811507 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f08e41e5de0 msgr2=0x7f08e41536e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:51.065 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.060+0000 7f08ea412640 1 --2- 172.21.15.169:0/3596811507 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f08e41e5de0 0x7f08e41536e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:51.065 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.060+0000 7f08ea412640 1 -- 172.21.15.169:0/3596811507 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f08d0006290 msgr2=0x7f08e41f22b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:51.066 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.060+0000 7f08ea412640 1 --2- 172.21.15.169:0/3596811507 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f08d0006290 0x7f08e41f22b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:51.066 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.060+0000 7f08ea412640 1 -- 172.21.15.169:0/3596811507 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08e4158cd0 con 0x7f08e41e3cf0 2023-10-26T01:36:51.066 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.060+0000 7f08ea412640 1 --2- 172.21.15.169:0/3596811507 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08e41e3cf0 0x7f08e41e4170 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f08dc02cd40 tx=0x7f08dc004a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:51.066 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.060+0000 7f08da7fc640 1 -- 172.21.15.169:0/3596811507 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08dc0017b0 con 0x7f08e41e3cf0 2023-10-26T01:36:51.067 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.060+0000 7f08da7fc640 1 -- 172.21.15.169:0/3596811507 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f08dc001930 con 0x7f08e41e3cf0 2023-10-26T01:36:51.067 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.060+0000 7f08da7fc640 1 -- 172.21.15.169:0/3596811507 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08dc006030 con 0x7f08e41e3cf0 2023-10-26T01:36:51.067 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.061+0000 7f08ebe75640 1 -- 172.21.15.169:0/3596811507 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08e413d610 con 0x7f08e41e3cf0 2023-10-26T01:36:51.067 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.061+0000 7f08ebe75640 1 -- 172.21.15.169:0/3596811507 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08e40b0a90 con 0x7f08e41e3cf0 2023-10-26T01:36:51.070 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.063+0000 7f08da7fc640 1 -- 172.21.15.169:0/3596811507 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f08dc031030 con 0x7f08e41e3cf0 2023-10-26T01:36:51.070 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.064+0000 7f08da7fc640 1 --2- 172.21.15.169:0/3596811507 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f08c806ae70 0x7f08c806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:51.070 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.064+0000 7f08da7fc640 1 -- 172.21.15.169:0/3596811507 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f08dc0aa690 con 0x7f08e41e3cf0 2023-10-26T01:36:51.071 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.065+0000 7f08e9c11640 1 --2- 172.21.15.169:0/3596811507 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f08c806ae70 0x7f08c806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:51.071 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.065+0000 7f08eea01740 1 Processor -- start 2023-10-26T01:36:51.071 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.065+0000 7f08eea01740 1 -- start start 2023-10-26T01:36:51.071 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.066+0000 7f08e9c11640 1 --2- 172.21.15.169:0/3596811507 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f08c806ae70 0x7f08c806d330 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f08e40719f0 tx=0x7f08e00068e0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:51.071 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.066+0000 7f08eea01740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558279dc77d0 0x558279d9aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:51.072 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.066+0000 7f08eea01740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558279dc7bb0 0x558279dca560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:51.072 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.066+0000 7f08eea01740 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x558279d9af40 0x558279dcaaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:51.072 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.066+0000 7f08eea01740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x558279d0d740 con 0x558279dc7bb0 2023-10-26T01:36:51.072 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.066+0000 7f08eea01740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x558279d0ae30 con 0x558279dc77d0 2023-10-26T01:36:51.072 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.066+0000 7f08eea01740 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x558279d06720 con 0x558279d9af40 2023-10-26T01:36:51.073 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.066+0000 7f08e9410640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558279dc7bb0 0x558279dca560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:51.073 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.066+0000 7f08e9410640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558279dc7bb0 0x558279dca560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:59590/0 (socket says 172.21.15.169:59590) 2023-10-26T01:36:51.073 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.066+0000 7f08e9410640 1 -- 172.21.15.169:0/952402844 learned_addr learned my addr 172.21.15.169:0/952402844 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:36:51.073 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.067+0000 7f08e9c11640 1 --2- 172.21.15.169:0/952402844 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558279dc77d0 0x558279d9aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:51.074 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.067+0000 7f08ea412640 1 --2- 172.21.15.169:0/952402844 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x558279d9af40 0x558279dcaaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:51.074 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.067+0000 7f08e9410640 1 -- 172.21.15.169:0/952402844 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x558279d9af40 msgr2=0x558279dcaaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T01:36:51.074 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.067+0000 7f08e9410640 1 --2- 172.21.15.169:0/952402844 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x558279d9af40 0x558279dcaaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:51.074 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.067+0000 7f08e9410640 1 -- 172.21.15.169:0/952402844 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558279dc77d0 msgr2=0x558279d9aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T01:36:51.074 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.067+0000 7f08e9410640 1 --2- 172.21.15.169:0/952402844 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558279dc77d0 0x558279d9aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:51.075 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.067+0000 7f08e9410640 1 -- 172.21.15.169:0/952402844 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558279cca910 con 0x558279dc7bb0 2023-10-26T01:36:51.075 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.067+0000 7f08e9c11640 1 --2- 172.21.15.169:0/952402844 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558279dc77d0 0x558279d9aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:36:51.075 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.067+0000 7f08e9410640 1 --2- 172.21.15.169:0/952402844 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558279dc7bb0 0x558279dca560 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f08d400fbe0 tx=0x7f08d40117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37512f090889efa4 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:51.075 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.067+0000 7f08c77fe640 1 -- 172.21.15.169:0/952402844 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08d400fed0 con 0x558279dc7bb0 2023-10-26T01:36:51.075 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.067+0000 7f08c77fe640 1 -- 172.21.15.169:0/952402844 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f08d4016070 con 0x558279dc7bb0 2023-10-26T01:36:51.076 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.067+0000 7f08c77fe640 1 -- 172.21.15.169:0/952402844 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08d401d7f0 con 0x558279dc7bb0 2023-10-26T01:36:51.076 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.068+0000 7f08eea01740 1 -- 172.21.15.169:0/952402844 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558279dc7bb0 msgr2=0x558279dca560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:51.076 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.068+0000 7f08eea01740 1 --2- 172.21.15.169:0/952402844 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558279dc7bb0 0x558279dca560 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f08d400fbe0 tx=0x7f08d40117b0 comp rx=0 tx=0).stop 2023-10-26T01:36:51.076 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.068+0000 7f08eea01740 1 -- 172.21.15.169:0/952402844 shutdown_connections 2023-10-26T01:36:51.077 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.068+0000 7f08eea01740 1 --2- 172.21.15.169:0/952402844 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558279dc7bb0 0x558279dca560 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:51.077 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.068+0000 7f08eea01740 1 --2- 172.21.15.169:0/952402844 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x558279d9af40 0x558279dcaaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:51.077 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.068+0000 7f08eea01740 1 --2- 172.21.15.169:0/952402844 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558279dc77d0 0x558279d9aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:51.077 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.068+0000 7f08eea01740 1 -- 172.21.15.169:0/952402844 >> 172.21.15.169:0/952402844 conn(0x558279d9cc70 msgr2=0x558279de5560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:51.077 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.068+0000 7f08eea01740 1 -- 172.21.15.169:0/952402844 shutdown_connections 2023-10-26T01:36:51.078 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.068+0000 7f08eea01740 1 -- 172.21.15.169:0/952402844 wait complete. 2023-10-26T01:36:51.081 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.069+0000 7f08eea01740 1 -- start start 2023-10-26T01:36:51.082 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.069+0000 7f08eea01740 1 Processor -- start 2023-10-26T01:36:51.082 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.070+0000 7f08eea01740 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x558279dc77d0 0x558279dcd220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:51.082 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.070+0000 7f08eea01740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558279dc7bb0 0x558279ddfc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:51.082 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.070+0000 7f08eea01740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558279de0140 0x558279de3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:51.083 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.070+0000 7f08eea01740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x558279ca4970 con 0x558279de0140 2023-10-26T01:36:51.083 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.070+0000 7f08eea01740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x558279ca4610 con 0x558279dc7bb0 2023-10-26T01:36:51.083 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.070+0000 7f08eea01740 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x558279a6af20 con 0x558279dc77d0 2023-10-26T01:36:51.083 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.070+0000 7f08eea01740 10 client.0 _ll_register_callbacks cb 0x558279cc80c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T01:36:51.084 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.071+0000 7f08ea412640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558279de0140 0x558279de3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:51.084 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.071+0000 7f08ea412640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558279de0140 0x558279de3e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:59598/0 (socket says 172.21.15.169:59598) 2023-10-26T01:36:51.084 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.071+0000 7f08ea412640 1 -- 172.21.15.169:0/1097039267 learned_addr learned my addr 172.21.15.169:0/1097039267 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:36:51.084 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.071+0000 7f08e9c11640 1 --2- 172.21.15.169:0/1097039267 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x558279dc77d0 0x558279dcd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:51.085 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.071+0000 7f08e9410640 1 --2- 172.21.15.169:0/1097039267 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558279dc7bb0 0x558279ddfc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:51.085 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.071+0000 7f08ea412640 1 -- 172.21.15.169:0/1097039267 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x558279dc77d0 msgr2=0x558279dcd220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T01:36:51.085 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.071+0000 7f08ea412640 1 --2- 172.21.15.169:0/1097039267 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x558279dc77d0 0x558279dcd220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:51.085 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.071+0000 7f08ea412640 1 -- 172.21.15.169:0/1097039267 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558279dc7bb0 msgr2=0x558279ddfc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T01:36:51.085 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.071+0000 7f08ea412640 1 --2- 172.21.15.169:0/1097039267 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558279dc7bb0 0x558279ddfc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:51.085 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.071+0000 7f08ea412640 1 -- 172.21.15.169:0/1097039267 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08d400f7e0 con 0x558279de0140 2023-10-26T01:36:51.086 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.071+0000 7f08ea412640 1 --2- 172.21.15.169:0/1097039267 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558279de0140 0x558279de3e10 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f08dc0a8ac0 tx=0x7f08dc079b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1fd6142fc0619b8 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:51.086 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.071+0000 7f08c4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.169:3300/0 2023-10-26T01:36:51.086 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.072+0000 7f08c4ff9640 1 -- 172.21.15.169:0/1097039267 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08dc081660 con 0x558279de0140 2023-10-26T01:36:51.086 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.072+0000 7f08c4ff9640 1 -- 172.21.15.169:0/1097039267 <== mon.0 v2:172.21.15.169:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f08dc07d120 con 0x558279de0140 2023-10-26T01:36:51.087 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.072+0000 7f08eea01740 1 -- 172.21.15.169:0/1097039267 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558279cff700 con 0x558279de0140 2023-10-26T01:36:51.087 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.072+0000 7f08c4ff9640 1 -- 172.21.15.169:0/1097039267 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f08dc07d2e0 con 0x558279de0140 2023-10-26T01:36:51.087 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.072+0000 7f08eea01740 10 client.7026 fetch_fsmap learned FSMap version 50 2023-10-26T01:36:51.087 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.072+0000 7f08eea01740 1 -- 172.21.15.169:0/1097039267 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) v3 -- 0x558279d04da0 con 0x558279de0140 2023-10-26T01:36:51.087 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.072+0000 7f08c4ff9640 1 -- 172.21.15.169:0/1097039267 <== mon.0 v2:172.21.15.169:3300/0 4 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x558279cff700 con 0x558279de0140 2023-10-26T01:36:51.088 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.072+0000 7f08c4ff9640 1 -- 172.21.15.169:0/1097039267 <== mon.0 v2:172.21.15.169:3300/0 5 ==== fsmap(e 50) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f08dc078850 con 0x558279de0140 2023-10-26T01:36:51.088 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.072+0000 7f08eea01740 10 client.7026 fetch_fsmap finished waiting for FSMap version 50 2023-10-26T01:36:51.088 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.072+0000 7f08eea01740 10 client.7026 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-26T01:36:51.088 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.072+0000 7f08eea01740 20 client.7026 populate_metadata read hostname 'smithi169' 2023-10-26T01:36:51.088 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.072+0000 7f08eea01740 1 --2- 172.21.15.169:0/1097039267 >> [v2:172.21.15.169:6833/657142078,v1:172.21.15.169:6835/657142078] conn(0x558279df9240 0x558279df9620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:51.089 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.072+0000 7f08eea01740 4 client.7026 mds_command: new command op to 6888 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-26T01:36:51.089 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.072+0000 7f08eea01740 1 -- 172.21.15.169:0/1097039267 --> [v2:172.21.15.169:6833/657142078,v1:172.21.15.169:6835/657142078] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558279cff420 con 0x558279df9240 2023-10-26T01:36:51.089 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.073+0000 7f08e9c11640 1 --2- 172.21.15.169:0/1097039267 >> [v2:172.21.15.169:6833/657142078,v1:172.21.15.169:6835/657142078] conn(0x558279df9240 0x558279df9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:51.089 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.073+0000 7f08e9c11640 1 --2- 172.21.15.169:0/1097039267 >> [v2:172.21.15.169:6833/657142078,v1:172.21.15.169:6835/657142078] conn(0x558279df9240 0x558279df9620 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b513c80ef71a4844 server_cookie=d5828ea006bc85bf in_seq=0 out_seq=0 2023-10-26T01:36:51.089 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.073+0000 7f08c4ff9640 10 client.7026 ms_handle_connect on v2:172.21.15.169:6833/657142078 2023-10-26T01:36:51.090 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.074+0000 7f08c4ff9640 1 -- 172.21.15.169:0/1097039267 <== mds.0 v2:172.21.15.169:6833/657142078 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x558279cff420 con 0x558279df9240 2023-10-26T01:36:51.090 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.074+0000 7f08c4ff9640 10 client.7026 handle_command_reply: tid=0 2023-10-26T01:36:51.090 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.074+0000 7f08eea01740 1 client.7026 shutdown 2023-10-26T01:36:51.090 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.074+0000 7f08eea01740 1 -- 172.21.15.169:0/1097039267 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558279de0140 msgr2=0x558279de3e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:51.090 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.074+0000 7f08eea01740 1 --2- 172.21.15.169:0/1097039267 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558279de0140 0x558279de3e10 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f08dc0a8ac0 tx=0x7f08dc079b20 comp rx=0 tx=0).stop 2023-10-26T01:36:51.091 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.074+0000 7f08eea01740 1 -- 172.21.15.169:0/1097039267 shutdown_connections 2023-10-26T01:36:51.091 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.074+0000 7f08eea01740 1 --2- 172.21.15.169:0/1097039267 >> [v2:172.21.15.169:6833/657142078,v1:172.21.15.169:6835/657142078] conn(0x558279df9240 0x558279df9620 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:51.091 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.074+0000 7f08eea01740 1 --2- 172.21.15.169:0/1097039267 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558279de0140 0x558279de3e10 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:51.091 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.074+0000 7f08eea01740 1 --2- 172.21.15.169:0/1097039267 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558279dc7bb0 0x558279ddfc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:51.091 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.074+0000 7f08eea01740 1 --2- 172.21.15.169:0/1097039267 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x558279dc77d0 0x558279dcd220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:51.092 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.074+0000 7f08c4ff9640 0 client.7026 ms_handle_reset on v2:172.21.15.169:6833/657142078 2023-10-26T01:36:51.092 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.074+0000 7f08eea01740 1 -- 172.21.15.169:0/1097039267 >> 172.21.15.169:0/1097039267 conn(0x558279d9af40 msgr2=0x558279d9e650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:51.092 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.074+0000 7f08eea01740 1 -- 172.21.15.169:0/1097039267 shutdown_connections 2023-10-26T01:36:51.092 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.074+0000 7f08eea01740 1 -- 172.21.15.169:0/1097039267 wait complete. 2023-10-26T01:36:51.092 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.075+0000 7f08eea01740 20 client.7026 trim_cache size 0 max 16384 2023-10-26T01:36:51.094 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.091+0000 7f08eea01740 1 Processor -- start 2023-10-26T01:36:51.094 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.091+0000 7f08eea01740 1 -- start start 2023-10-26T01:36:51.094 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.092+0000 7f08eea01740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558279dc77d0 0x558279df5000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:51.095 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.092+0000 7f08eea01740 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x558279dc7bb0 0x558279dca550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:51.095 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.092+0000 7f08eea01740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558279de0140 0x558279dcaa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:51.095 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.092+0000 7f08eea01740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x558279ddf220 con 0x558279dc77d0 2023-10-26T01:36:51.095 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.092+0000 7f08eea01740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x558279ca39f0 con 0x558279de0140 2023-10-26T01:36:51.095 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.092+0000 7f08eea01740 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x558279de58e0 con 0x558279dc7bb0 2023-10-26T01:36:51.095 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.092+0000 7f08e9c11640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558279dc77d0 0x558279df5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:51.096 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.092+0000 7f08e9c11640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558279dc77d0 0x558279df5000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:59610/0 (socket says 172.21.15.169:59610) 2023-10-26T01:36:51.096 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.092+0000 7f08e9c11640 1 -- 172.21.15.169:0/2675946876 learned_addr learned my addr 172.21.15.169:0/2675946876 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:36:51.100 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.093+0000 7f08e9c11640 1 -- 172.21.15.169:0/2675946876 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x558279dc7bb0 msgr2=0x558279dca550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T01:36:51.100 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.093+0000 7f08e9c11640 1 --2- 172.21.15.169:0/2675946876 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x558279dc7bb0 0x558279dca550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:51.100 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.093+0000 7f08e9c11640 1 -- 172.21.15.169:0/2675946876 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558279de0140 msgr2=0x558279dcaa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T01:36:51.100 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.093+0000 7f08e9c11640 1 --2- 172.21.15.169:0/2675946876 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558279de0140 0x558279dcaa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:51.100 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.093+0000 7f08e9c11640 1 -- 172.21.15.169:0/2675946876 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558279ca3b80 con 0x558279dc77d0 2023-10-26T01:36:51.101 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.093+0000 7f08e9c11640 1 --2- 172.21.15.169:0/2675946876 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558279dc77d0 0x558279df5000 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f08e0006ad0 tx=0x7f08e0011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afb7d5f75da4417a server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:51.101 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.093+0000 7f08c7fff640 1 -- 172.21.15.169:0/2675946876 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e00250b0 con 0x558279dc77d0 2023-10-26T01:36:51.101 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.093+0000 7f08c7fff640 1 -- 172.21.15.169:0/2675946876 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f08e000c050 con 0x558279dc77d0 2023-10-26T01:36:51.101 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.093+0000 7f08c7fff640 1 -- 172.21.15.169:0/2675946876 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e0025210 con 0x558279dc77d0 2023-10-26T01:36:51.101 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.094+0000 7f08eea01740 1 -- 172.21.15.169:0/2675946876 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558279dc77d0 msgr2=0x558279df5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:51.102 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.094+0000 7f08eea01740 1 --2- 172.21.15.169:0/2675946876 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558279dc77d0 0x558279df5000 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f08e0006ad0 tx=0x7f08e0011000 comp rx=0 tx=0).stop 2023-10-26T01:36:51.102 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.094+0000 7f08eea01740 1 -- 172.21.15.169:0/2675946876 shutdown_connections 2023-10-26T01:36:51.102 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.094+0000 7f08eea01740 1 --2- 172.21.15.169:0/2675946876 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558279de0140 0x558279dcaa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:51.102 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.094+0000 7f08eea01740 1 --2- 172.21.15.169:0/2675946876 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x558279dc7bb0 0x558279dca550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:51.103 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.094+0000 7f08eea01740 1 --2- 172.21.15.169:0/2675946876 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558279dc77d0 0x558279df5000 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:51.103 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.094+0000 7f08eea01740 1 -- 172.21.15.169:0/2675946876 >> 172.21.15.169:0/2675946876 conn(0x558279d9af40 msgr2=0x558279de8230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:51.103 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.094+0000 7f08eea01740 1 -- 172.21.15.169:0/2675946876 shutdown_connections 2023-10-26T01:36:51.103 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.094+0000 7f08eea01740 1 -- 172.21.15.169:0/2675946876 wait complete. 2023-10-26T01:36:51.103 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.095+0000 7f08eea01740 1 -- start start 2023-10-26T01:36:51.104 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.095+0000 7f08eea01740 1 Processor -- start 2023-10-26T01:36:51.104 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.096+0000 7f08eea01740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558279dc77d0 0x558279d9e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:51.104 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.096+0000 7f08eea01740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558279dc7bb0 0x558279d9f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:51.104 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.096+0000 7f08eea01740 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x558279de0140 0x558279d9fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:51.105 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.096+0000 7f08eea01740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x558279da0160 con 0x558279dc77d0 2023-10-26T01:36:51.105 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.096+0000 7f08eea01740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x558279e4ec20 con 0x558279dc7bb0 2023-10-26T01:36:51.105 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.096+0000 7f08eea01740 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x558279e4e600 con 0x558279de0140 2023-10-26T01:36:51.105 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.096+0000 7f08eea01740 10 client.0 _ll_register_callbacks cb 0x7f08d400f360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T01:36:51.105 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.096+0000 7f08e9c11640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558279dc77d0 0x558279d9e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:51.106 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.096+0000 7f08e9c11640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558279dc77d0 0x558279d9e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:59612/0 (socket says 172.21.15.169:59612) 2023-10-26T01:36:51.106 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.096+0000 7f08e9c11640 1 -- 172.21.15.169:0/177581432 learned_addr learned my addr 172.21.15.169:0/177581432 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:36:51.106 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.096+0000 7f08e9c11640 1 -- 172.21.15.169:0/177581432 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x558279de0140 msgr2=0x558279d9fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T01:36:51.106 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.096+0000 7f08e9c11640 1 --2- 172.21.15.169:0/177581432 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x558279de0140 0x558279d9fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:51.106 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.096+0000 7f08e9c11640 1 -- 172.21.15.169:0/177581432 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558279dc7bb0 msgr2=0x558279d9f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T01:36:51.107 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.097+0000 7f08e9c11640 1 --2- 172.21.15.169:0/177581432 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558279dc7bb0 0x558279d9f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:51.107 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.097+0000 7f08e9c11640 1 -- 172.21.15.169:0/177581432 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08e0018040 con 0x558279dc77d0 2023-10-26T01:36:51.107 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.097+0000 7f08e9c11640 1 --2- 172.21.15.169:0/177581432 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558279dc77d0 0x558279d9e620 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f08e0001e20 tx=0x7f08e00254e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35f740ad1fbc03 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:51.108 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.097+0000 7f08c4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.169:3300/0 2023-10-26T01:36:51.108 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.097+0000 7f08c4ff9640 1 -- 172.21.15.169:0/177581432 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e0022720 con 0x558279dc77d0 2023-10-26T01:36:51.108 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.097+0000 7f08c4ff9640 1 -- 172.21.15.169:0/177581432 <== mon.0 v2:172.21.15.169:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f08e0022880 con 0x558279dc77d0 2023-10-26T01:36:51.108 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.097+0000 7f08eea01740 1 -- 172.21.15.169:0/177581432 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f08d4016070 con 0x558279dc77d0 2023-10-26T01:36:51.108 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.097+0000 7f08c4ff9640 1 -- 172.21.15.169:0/177581432 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f08e0022a40 con 0x558279dc77d0 2023-10-26T01:36:51.109 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.097+0000 7f08eea01740 10 client.7032 fetch_fsmap learned FSMap version 50 2023-10-26T01:36:51.109 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.097+0000 7f08eea01740 1 -- 172.21.15.169:0/177581432 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) v3 -- 0x558279cc9170 con 0x558279dc77d0 2023-10-26T01:36:51.109 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.098+0000 7f08c4ff9640 1 -- 172.21.15.169:0/177581432 <== mon.0 v2:172.21.15.169:3300/0 4 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f08d4016070 con 0x558279dc77d0 2023-10-26T01:36:51.109 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.098+0000 7f08c4ff9640 1 -- 172.21.15.169:0/177581432 <== mon.0 v2:172.21.15.169:3300/0 5 ==== fsmap(e 50) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f08e0011070 con 0x558279dc77d0 2023-10-26T01:36:51.109 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.098+0000 7f08eea01740 10 client.7032 fetch_fsmap finished waiting for FSMap version 50 2023-10-26T01:36:51.110 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.098+0000 7f08eea01740 10 client.7032 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-26T01:36:51.110 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.098+0000 7f08eea01740 20 client.7032 populate_metadata read hostname 'smithi169' 2023-10-26T01:36:51.110 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.098+0000 7f08eea01740 1 --2- 172.21.15.169:0/177581432 >> [v2:172.21.15.169:6833/657142078,v1:172.21.15.169:6835/657142078] conn(0x558279df4500 0x558279df48e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:51.110 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.098+0000 7f08eea01740 4 client.7032 mds_command: new command op to 6888 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-26T01:36:51.110 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.098+0000 7f08eea01740 1 -- 172.21.15.169:0/177581432 --> [v2:172.21.15.169:6833/657142078,v1:172.21.15.169:6835/657142078] -- command(tid 0: {"prefix": "status"}) v1 -- 0x558279a086c0 con 0x558279df4500 2023-10-26T01:36:51.111 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.098+0000 7f08e9410640 1 --2- 172.21.15.169:0/177581432 >> [v2:172.21.15.169:6833/657142078,v1:172.21.15.169:6835/657142078] conn(0x558279df4500 0x558279df48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:51.112 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.099+0000 7f08e9410640 1 --2- 172.21.15.169:0/177581432 >> [v2:172.21.15.169:6833/657142078,v1:172.21.15.169:6835/657142078] conn(0x558279df4500 0x558279df48e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f95ddb7e1c76fffe server_cookie=1399aca5cdc52fbe in_seq=0 out_seq=0 2023-10-26T01:36:51.112 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.099+0000 7f08c4ff9640 10 client.7032 ms_handle_connect on v2:172.21.15.169:6833/657142078 2023-10-26T01:36:51.112 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.099+0000 7f08c4ff9640 1 -- 172.21.15.169:0/177581432 <== mds.0 v2:172.21.15.169:6833/657142078 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x558279a086c0 con 0x558279df4500 2023-10-26T01:36:51.113 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.099+0000 7f08c4ff9640 10 client.7032 handle_command_reply: tid=0 2023-10-26T01:36:51.113 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.099+0000 7f08eea01740 1 client.7032 shutdown 2023-10-26T01:36:51.113 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.099+0000 7f08eea01740 1 -- 172.21.15.169:0/177581432 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558279dc77d0 msgr2=0x558279d9e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:51.113 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.099+0000 7f08eea01740 1 --2- 172.21.15.169:0/177581432 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558279dc77d0 0x558279d9e620 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f08e0001e20 tx=0x7f08e00254e0 comp rx=0 tx=0).stop 2023-10-26T01:36:51.114 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.099+0000 7f08eea01740 1 -- 172.21.15.169:0/177581432 shutdown_connections 2023-10-26T01:36:51.114 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.099+0000 7f08eea01740 1 --2- 172.21.15.169:0/177581432 >> [v2:172.21.15.169:6833/657142078,v1:172.21.15.169:6835/657142078] conn(0x558279df4500 0x558279df48e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:51.114 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.099+0000 7f08eea01740 1 --2- 172.21.15.169:0/177581432 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x558279de0140 0x558279d9fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:51.114 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.099+0000 7f08eea01740 1 --2- 172.21.15.169:0/177581432 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558279dc7bb0 0x558279d9f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:51.115 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.099+0000 7f08c4ff9640 0 client.7032 ms_handle_reset on v2:172.21.15.169:6833/657142078 2023-10-26T01:36:51.115 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.099+0000 7f08eea01740 1 --2- 172.21.15.169:0/177581432 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558279dc77d0 0x558279d9e620 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:51.115 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.099+0000 7f08eea01740 1 -- 172.21.15.169:0/177581432 >> 172.21.15.169:0/177581432 conn(0x558279d9af40 msgr2=0x558279de8230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:51.115 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.099+0000 7f08eea01740 1 -- 172.21.15.169:0/177581432 shutdown_connections 2023-10-26T01:36:51.115 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.099+0000 7f08eea01740 1 -- 172.21.15.169:0/177581432 wait complete. 2023-10-26T01:36:51.115 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.100+0000 7f08eea01740 20 client.7032 trim_cache size 0 max 16384 2023-10-26T01:36:51.116 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2023-10-26T01:36:51.116 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_fsid": "30203b28-ed8a-44e8-a9a6-a9bf25f980f1", 2023-10-26T01:36:51.116 INFO:teuthology.orchestra.run.smithi169.stdout: "whoami": 0, 2023-10-26T01:36:51.116 INFO:teuthology.orchestra.run.smithi169.stdout: "id": 6888, 2023-10-26T01:36:51.117 INFO:teuthology.orchestra.run.smithi169.stdout: "want_state": "up:active", 2023-10-26T01:36:51.117 INFO:teuthology.orchestra.run.smithi169.stdout: "state": "up:active", 2023-10-26T01:36:51.117 INFO:teuthology.orchestra.run.smithi169.stdout: "fs_name": "cephfs", 2023-10-26T01:36:51.117 INFO:teuthology.orchestra.run.smithi169.stdout: "rank_uptime": 6.6850843439999998, 2023-10-26T01:36:51.118 INFO:teuthology.orchestra.run.smithi169.stdout: "mdsmap_epoch": 50, 2023-10-26T01:36:51.118 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_epoch": 103, 2023-10-26T01:36:51.118 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_epoch_barrier": 103, 2023-10-26T01:36:51.118 INFO:teuthology.orchestra.run.smithi169.stdout: "uptime": 13.536220265000001 2023-10-26T01:36:51.119 INFO:teuthology.orchestra.run.smithi169.stdout:} 2023-10-26T01:36:51.120 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.100+0000 7f08ebe75640 1 -- 172.21.15.169:0/3596811507 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f08c806ae70 msgr2=0x7f08c806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:51.120 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.100+0000 7f08ebe75640 1 --2- 172.21.15.169:0/3596811507 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f08c806ae70 0x7f08c806d330 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f08e40719f0 tx=0x7f08e00068e0 comp rx=0 tx=0).stop 2023-10-26T01:36:51.120 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.100+0000 7f08ebe75640 1 -- 172.21.15.169:0/3596811507 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08e41e3cf0 msgr2=0x7f08e41e4170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:51.120 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.100+0000 7f08ebe75640 1 --2- 172.21.15.169:0/3596811507 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08e41e3cf0 0x7f08e41e4170 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f08dc02cd40 tx=0x7f08dc004a00 comp rx=0 tx=0).stop 2023-10-26T01:36:51.120 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.100+0000 7f08ebe75640 1 -- 172.21.15.169:0/3596811507 shutdown_connections 2023-10-26T01:36:51.121 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.100+0000 7f08ebe75640 1 --2- 172.21.15.169:0/3596811507 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f08c806ae70 0x7f08c806d330 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:51.121 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.100+0000 7f08ebe75640 1 --2- 172.21.15.169:0/3596811507 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08e41e3cf0 0x7f08e41e4170 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:51.121 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.100+0000 7f08ebe75640 1 --2- 172.21.15.169:0/3596811507 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f08e41e5de0 0x7f08e41536e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:51.121 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.100+0000 7f08ebe75640 1 --2- 172.21.15.169:0/3596811507 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f08d0006290 0x7f08e41f22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:51.121 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.100+0000 7f08ebe75640 1 -- 172.21.15.169:0/3596811507 >> 172.21.15.169:0/3596811507 conn(0x7f08e4069930 msgr2=0x7f08e4156f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:51.122 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.100+0000 7f08ebe75640 1 -- 172.21.15.169:0/3596811507 shutdown_connections 2023-10-26T01:36:51.122 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.100+0000 7f08ebe75640 1 -- 172.21.15.169:0/3596811507 wait complete. 2023-10-26T01:36:51.123 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-26T01:36:51.340 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.338+0000 7f8560ed7640 1 -- 172.21.15.169:0/1563277315 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f853c003f40 msgr2=0x7f853c0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:51.340 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.338+0000 7f8560ed7640 1 --2- 172.21.15.169:0/1563277315 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f853c003f40 0x7f853c0047c0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f8544009b80 tx=0x7f853c0057f0 comp rx=0 tx=0).stop 2023-10-26T01:36:51.340 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.338+0000 7f855ad9d640 1 -- 172.21.15.169:0/1563277315 reap_dead start 2023-10-26T01:36:51.340 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.338+0000 7f8560ed7640 1 -- 172.21.15.169:0/1563277315 shutdown_connections 2023-10-26T01:36:51.341 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.338+0000 7f8560ed7640 1 -- 172.21.15.169:0/1563277315 >> 172.21.15.169:0/1563277315 conn(0x7f855c069930 msgr2=0x7f855c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:51.341 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.338+0000 7f8560ed7640 1 -- 172.21.15.169:0/1563277315 shutdown_connections 2023-10-26T01:36:51.341 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.338+0000 7f8560ed7640 1 -- 172.21.15.169:0/1563277315 wait complete. 2023-10-26T01:36:51.342 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.340+0000 7f8560ed7640 1 Processor -- start 2023-10-26T01:36:51.342 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.340+0000 7f8560ed7640 1 -- start start 2023-10-26T01:36:51.344 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.341+0000 7f8560ed7640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f853c003f40 0x7f855c151a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:51.344 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.341+0000 7f8560ed7640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f855c0acf00 0x7f855c0c91e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:51.344 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.341+0000 7f8560ed7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f855c1e4050 0x7f855c1e4500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:51.345 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.341+0000 7f8560ed7640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f855c1e4a40 con 0x7f855c0acf00 2023-10-26T01:36:51.345 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.341+0000 7f8560ed7640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f855c0ad390 con 0x7f855c1e4050 2023-10-26T01:36:51.345 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.341+0000 7f8560ed7640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f855c0ad670 con 0x7f853c003f40 2023-10-26T01:36:51.345 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.341+0000 7f8551d9b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f855c0acf00 0x7f855c0c91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:51.345 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.341+0000 7f8551d9b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f855c0acf00 0x7f855c0c91e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:59620/0 (socket says 172.21.15.169:59620) 2023-10-26T01:36:51.346 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.341+0000 7f8551d9b640 1 -- 172.21.15.169:0/1079614855 learned_addr learned my addr 172.21.15.169:0/1079614855 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:36:51.346 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.341+0000 7f855a59c640 1 --2- 172.21.15.169:0/1079614855 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f853c003f40 0x7f855c151a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:51.346 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.341+0000 7f8551d9b640 1 -- 172.21.15.169:0/1079614855 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f853c003f40 msgr2=0x7f855c151a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:51.346 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.341+0000 7f8551d9b640 1 --2- 172.21.15.169:0/1079614855 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f853c003f40 0x7f855c151a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:51.346 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.341+0000 7f8551d9b640 1 -- 172.21.15.169:0/1079614855 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f855c1e4050 msgr2=0x7f855c1e4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:51.347 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.341+0000 7f8551d9b640 1 --2- 172.21.15.169:0/1079614855 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f855c1e4050 0x7f855c1e4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:51.347 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.342+0000 7f8551d9b640 1 -- 172.21.15.169:0/1079614855 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f855c0ad950 con 0x7f855c0acf00 2023-10-26T01:36:51.347 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.342+0000 7f8551d9b640 1 --2- 172.21.15.169:0/1079614855 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f855c0acf00 0x7f855c0c91e0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f854402cb60 tx=0x7f8544033c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:51.347 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.342+0000 7f85537fe640 1 -- 172.21.15.169:0/1079614855 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f854403d820 con 0x7f855c0acf00 2023-10-26T01:36:51.347 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.342+0000 7f85537fe640 1 -- 172.21.15.169:0/1079614855 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f854403d980 con 0x7f855c0acf00 2023-10-26T01:36:51.348 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.342+0000 7f85537fe640 1 -- 172.21.15.169:0/1079614855 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f854403dd50 con 0x7f855c0acf00 2023-10-26T01:36:51.348 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.342+0000 7f8560ed7640 1 -- 172.21.15.169:0/1079614855 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f855c0adc30 con 0x7f855c0acf00 2023-10-26T01:36:51.348 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.342+0000 7f8560ed7640 1 -- 172.21.15.169:0/1079614855 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f855c151fa0 con 0x7f855c0acf00 2023-10-26T01:36:51.349 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.343+0000 7f8560ed7640 1 -- 172.21.15.169:0/1079614855 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f855c04db10 con 0x7f855c0acf00 2023-10-26T01:36:51.349 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.344+0000 7f85537fe640 1 -- 172.21.15.169:0/1079614855 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f8544045400 con 0x7f855c0acf00 2023-10-26T01:36:51.350 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.346+0000 7f85537fe640 1 --2- 172.21.15.169:0/1079614855 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f854006ae40 0x7f854006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:51.350 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.346+0000 7f85537fe640 1 -- 172.21.15.169:0/1079614855 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f85440ac2d0 con 0x7f855c0acf00 2023-10-26T01:36:51.350 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.346+0000 7f855a59c640 1 --2- 172.21.15.169:0/1079614855 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f854006ae40 0x7f854006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:51.350 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.347+0000 7f855a59c640 1 --2- 172.21.15.169:0/1079614855 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f854006ae40 0x7f854006d300 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f854c009ad0 tx=0x7f854c005f90 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:51.352 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.350+0000 7f85537fe640 1 -- 172.21.15.169:0/1079614855 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8544077370 con 0x7f855c0acf00 2023-10-26T01:36:51.584 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.583+0000 7f8560ed7640 1 -- 172.21.15.169:0/1079614855 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f855c0b2170 con 0x7f855c0acf00 2023-10-26T01:36:51.585 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.584+0000 7f85537fe640 1 -- 172.21.15.169:0/1079614855 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v50) v1 ==== 53+0+73 (secure 0 0 0) 0x7f8544033e60 con 0x7f855c0acf00 2023-10-26T01:36:51.585 INFO:teuthology.orchestra.run.smithi169.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-26T01:36:51.588 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.586+0000 7f8560ed7640 1 -- 172.21.15.169:0/1079614855 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f854006ae40 msgr2=0x7f854006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:51.588 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.586+0000 7f8560ed7640 1 --2- 172.21.15.169:0/1079614855 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f854006ae40 0x7f854006d300 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f854c009ad0 tx=0x7f854c005f90 comp rx=0 tx=0).stop 2023-10-26T01:36:51.589 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.586+0000 7f8560ed7640 1 -- 172.21.15.169:0/1079614855 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f855c0acf00 msgr2=0x7f855c0c91e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:51.589 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.586+0000 7f8560ed7640 1 --2- 172.21.15.169:0/1079614855 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f855c0acf00 0x7f855c0c91e0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f854402cb60 tx=0x7f8544033c70 comp rx=0 tx=0).stop 2023-10-26T01:36:51.589 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.586+0000 7f8560ed7640 1 -- 172.21.15.169:0/1079614855 shutdown_connections 2023-10-26T01:36:51.589 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.586+0000 7f8560ed7640 1 --2- 172.21.15.169:0/1079614855 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f854006ae40 0x7f854006d300 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:51.589 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.586+0000 7f8560ed7640 1 --2- 172.21.15.169:0/1079614855 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f855c0acf00 0x7f855c0c91e0 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:51.589 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.586+0000 7f8560ed7640 1 --2- 172.21.15.169:0/1079614855 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f855c1e4050 0x7f855c1e4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:51.590 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.586+0000 7f8560ed7640 1 --2- 172.21.15.169:0/1079614855 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f853c003f40 0x7f855c151a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:51.590 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.586+0000 7f8560ed7640 1 -- 172.21.15.169:0/1079614855 >> 172.21.15.169:0/1079614855 conn(0x7f855c069930 msgr2=0x7f855c15b390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:51.590 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.586+0000 7f8560ed7640 1 -- 172.21.15.169:0/1079614855 shutdown_connections 2023-10-26T01:36:51.590 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:51.586+0000 7f8560ed7640 1 -- 172.21.15.169:0/1079614855 wait complete. 2023-10-26T01:36:51.607 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-26T01:36:51.607 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-26T01:36:51.607 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi169.front.sepia.ceph.com 2023-10-26T01:36:51.608 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-26T01:36:51.608 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-26T01:36:51.608 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-26T01:36:51.608 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-26T01:36:51.609 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-26T01:36:51.609 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-26T01:36:51.609 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:36:51.610 DEBUG:teuthology.orchestra.run.smithi169:> ip addr 2023-10-26T01:36:51.661 INFO:teuthology.orchestra.run.smithi169.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-26T01:36:51.661 INFO:teuthology.orchestra.run.smithi169.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-26T01:36:51.661 INFO:teuthology.orchestra.run.smithi169.stdout: inet 127.0.0.1/8 scope host lo 2023-10-26T01:36:51.661 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:36:51.661 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 ::1/128 scope host 2023-10-26T01:36:51.662 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:36:51.662 INFO:teuthology.orchestra.run.smithi169.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-26T01:36:51.662 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:d9:3b:b4 brd ff:ff:ff:ff:ff:ff 2023-10-26T01:36:51.662 INFO:teuthology.orchestra.run.smithi169.stdout: altname enp7s0f0 2023-10-26T01:36:51.663 INFO:teuthology.orchestra.run.smithi169.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-26T01:36:51.663 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:d9:3b:b5 brd ff:ff:ff:ff:ff:ff 2023-10-26T01:36:51.663 INFO:teuthology.orchestra.run.smithi169.stdout: altname enp7s0f1 2023-10-26T01:36:51.663 INFO:teuthology.orchestra.run.smithi169.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-26T01:36:51.663 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:88:7f:4c brd ff:ff:ff:ff:ff:ff 2023-10-26T01:36:51.663 INFO:teuthology.orchestra.run.smithi169.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-26T01:36:51.664 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:88:7f:4d brd ff:ff:ff:ff:ff:ff 2023-10-26T01:36:51.664 INFO:teuthology.orchestra.run.smithi169.stdout: inet 172.21.15.169/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-26T01:36:51.664 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft 42084sec preferred_lft 42084sec 2023-10-26T01:36:51.664 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 fe80::ec4:7aff:fe88:7f4d/64 scope link 2023-10-26T01:36:51.664 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:36:51.664 INFO:teuthology.orchestra.run.smithi169.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-26T01:36:51.665 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 22:a0:6b:43:7b:04 brd ff:ff:ff:ff:ff:ff 2023-10-26T01:36:51.665 INFO:teuthology.orchestra.run.smithi169.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-26T01:36:51.665 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:36:51.665 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 fe80::d815:31ff:fe87:d148/64 scope link 2023-10-26T01:36:51.665 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:36:51.665 INFO:teuthology.orchestra.run.smithi169.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-26T01:36:51.666 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 2a:80:f8:45:5f:9f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-26T01:36:51.666 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 fe80::2880:f8ff:fe45:5f9f/64 scope link 2023-10-26T01:36:51.666 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:36:51.666 INFO:teuthology.orchestra.run.smithi169.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-26T01:36:51.666 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 22:a0:6b:43:7b:04 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-26T01:36:51.667 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 fe80::20a0:6bff:fe43:7b04/64 scope link 2023-10-26T01:36:51.667 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:36:51.668 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:36:51.668 DEBUG:teuthology.orchestra.run.smithi169:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-26T01:36:51.755 INFO:teuthology.orchestra.run.smithi169.stdout:1 2023-10-26T01:36:51.759 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:36:51.760 DEBUG:teuthology.orchestra.run.smithi169:> route 2023-10-26T01:36:51.811 INFO:teuthology.orchestra.run.smithi169.stdout:Kernel IP routing table 2023-10-26T01:36:51.811 INFO:teuthology.orchestra.run.smithi169.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-26T01:36:51.812 INFO:teuthology.orchestra.run.smithi169.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-26T01:36:51.812 INFO:teuthology.orchestra.run.smithi169.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-26T01:36:51.812 INFO:teuthology.orchestra.run.smithi169.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-26T01:36:51.814 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:36:51.815 DEBUG:teuthology.orchestra.run.smithi169:> (cd / && exec bash -c ' 2023-10-26T01:36:51.816 DEBUG:teuthology.orchestra.run.smithi169:> set -e 2023-10-26T01:36:51.816 DEBUG:teuthology.orchestra.run.smithi169:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-26T01:36:51.817 DEBUG:teuthology.orchestra.run.smithi169:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-26T01:36:51.818 DEBUG:teuthology.orchestra.run.smithi169:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-26T01:36:51.818 DEBUG:teuthology.orchestra.run.smithi169:> ') 2023-10-26T01:36:52.009 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:36:52.009 DEBUG:teuthology.orchestra.run.smithi169:> ip netns list 2023-10-26T01:36:52.060 INFO:teuthology.orchestra.run.smithi169.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-26T01:36:52.060 INFO:teuthology.orchestra.run.smithi169.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-26T01:36:52.061 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-26T01:36:52.062 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:36:52.062 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link set brx.0 up 2023-10-26T01:36:52.155 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-26T01:36:52.156 DEBUG:teuthology.orchestra.run.smithi169:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-26T01:36:52.210 INFO:teuthology.orchestra.run.smithi169.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-26T01:36:52.211 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-26T01:36:52.211 DEBUG:teuthology.orchestra.run.smithi169:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-26T01:36:52.266 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-26T01:36:52.266 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:36:52.267 DEBUG:teuthology.orchestra.run.smithi169:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-26T01:36:52.403 INFO:teuthology.orchestra.run.smithi169.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-26T01:36:52.403 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-26T01:36:52.404 INFO:teuthology.orchestra.run.smithi169.stdout:invalid new device string format 2023-10-26T01:36:52.429 INFO:teuthology.orchestra.run.smithi169.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-26T01:36:52.429 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-26T01:36:52.430 INFO:teuthology.orchestra.run.smithi169.stdout:invalid new device string format 2023-10-26T01:36:52.430 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: resolved to: "172.21.15.169:3300,172.21.15.170:3300,172.21.15.170:3301" 2023-10-26T01:36:52.430 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-26T01:36:52.455 INFO:teuthology.orchestra.run.smithi169.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-26T01:36:52.455 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-26T01:36:52.455 INFO:teuthology.orchestra.run.smithi169.stdout:invalid new device string format 2023-10-26T01:36:52.455 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: resolved to: "172.21.15.169:3300,172.21.15.170:3300,172.21.15.170:3301" 2023-10-26T01:36:52.456 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-26T01:36:52.456 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: resolved to: "172.21.15.169:6789,172.21.15.170:6789,172.21.15.170:6790" 2023-10-26T01:36:52.456 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.169:6789,172.21.15.170:6789,172.21.15.170:6790:/ 2023-10-26T01:36:52.456 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=30203b28-ed8a-44e8-a9a6-a9bf25f980f1" will pass to kernel 2023-10-26T01:36:52.458 INFO:teuthology.orchestra.run.smithi169.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2023-10-26T01:36:52.458 INFO:teuthology.orchestra.run.smithi169.stdout: You just mounted a file system that supports labels which does not 2023-10-26T01:36:52.458 INFO:teuthology.orchestra.run.smithi169.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-10-26T01:36:52.459 INFO:teuthology.orchestra.run.smithi169.stdout: applications will generate AVC messages and not be allowed access to 2023-10-26T01:36:52.459 INFO:teuthology.orchestra.run.smithi169.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-10-26T01:36:52.462 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-26T01:36:52.463 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:36:52.463 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-26T01:36:52.545 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:36:52.546 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-26T01:36:52.602 INFO:teuthology.orchestra.run.smithi169.stdout:client5569 2023-10-26T01:36:52.602 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:36:52.603 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:36:52.652 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-26T01:36:52.919 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:52.916+0000 7fa0a4fb6640 1 -- 172.21.15.169:0/2528396072 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa0a00cdd20 msgr2=0x7fa0a015b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:52.919 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:52.916+0000 7fa0a4fb6640 1 --2- 172.21.15.169:0/2528396072 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa0a00cdd20 0x7fa0a015b4a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa088005fd0 tx=0x7fa08802caa0 comp rx=0 tx=0).stop 2023-10-26T01:36:52.919 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:52.916+0000 7fa0a4fb6640 1 -- 172.21.15.169:0/2528396072 shutdown_connections 2023-10-26T01:36:52.919 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:52.916+0000 7fa0a4fb6640 1 --2- 172.21.15.169:0/2528396072 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa0a00cdd20 0x7fa0a015b4a0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:52.920 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:52.916+0000 7fa0a4fb6640 1 --2- 172.21.15.169:0/2528396072 >> v2:172.21.15.170:3300/0 conn(0x7fa0a0155360 0x7fa0a00cd7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:52.920 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:52.916+0000 7fa0a4fb6640 1 -- 172.21.15.169:0/2528396072 >> 172.21.15.169:0/2528396072 conn(0x7fa0a0069930 msgr2=0x7fa0a0069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:52.921 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:52.916+0000 7fa0a4fb6640 1 -- 172.21.15.169:0/2528396072 shutdown_connections 2023-10-26T01:36:52.921 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:52.916+0000 7fa0a4fb6640 1 -- 172.21.15.169:0/2528396072 wait complete. 2023-10-26T01:36:52.925 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:52.918+0000 7fa0a4fb6640 1 Processor -- start 2023-10-26T01:36:52.925 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:52.918+0000 7fa0a4fb6640 1 -- start start 2023-10-26T01:36:52.925 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:52.919+0000 7fa0a4fb6640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa0a00cdd20 0x7fa0a01f1220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:52.925 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:52.919+0000 7fa0a4fb6640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa0a0155360 0x7fa0a015a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:52.926 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:52.919+0000 7fa0a4fb6640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa0a015b9e0 0x7fa0a015a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:52.926 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:52.919+0000 7fa0a4fb6640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a00c87b0 con 0x7fa0a00cdd20 2023-10-26T01:36:52.926 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:52.919+0000 7fa0a4fb6640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a00c8a90 con 0x7fa0a0155360 2023-10-26T01:36:52.926 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:52.919+0000 7fa0a4fb6640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a00c8d70 con 0x7fa0a015b9e0 2023-10-26T01:36:52.927 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:52.919+0000 7fa09e59c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa0a00cdd20 0x7fa0a01f1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:52.927 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:52.920+0000 7fa09e59c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa0a00cdd20 0x7fa0a01f1220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:59630/0 (socket says 172.21.15.169:59630) 2023-10-26T01:36:52.927 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:52.920+0000 7fa09e59c640 1 -- 172.21.15.169:0/2381571799 learned_addr learned my addr 172.21.15.169:0/2381571799 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:36:52.927 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:52.920+0000 7fa097fff640 1 --2- 172.21.15.169:0/2381571799 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa0a0155360 0x7fa0a015a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:52.928 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:52.920+0000 7fa09ed9d640 1 --2- 172.21.15.169:0/2381571799 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa0a015b9e0 0x7fa0a015a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:52.928 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:52.920+0000 7fa09e59c640 1 -- 172.21.15.169:0/2381571799 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa0a015b9e0 msgr2=0x7fa0a015a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:52.928 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:52.920+0000 7fa09e59c640 1 --2- 172.21.15.169:0/2381571799 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa0a015b9e0 0x7fa0a015a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:52.928 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:52.920+0000 7fa09e59c640 1 -- 172.21.15.169:0/2381571799 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa0a0155360 msgr2=0x7fa0a015a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:52.929 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:52.920+0000 7fa09e59c640 1 --2- 172.21.15.169:0/2381571799 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa0a0155360 0x7fa0a015a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:52.929 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:52.920+0000 7fa09e59c640 1 -- 172.21.15.169:0/2381571799 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0880071c0 con 0x7fa0a00cdd20 2023-10-26T01:36:52.929 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:52.920+0000 7fa09ed9d640 1 --2- 172.21.15.169:0/2381571799 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa0a015b9e0 0x7fa0a015a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:36:52.929 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:52.920+0000 7fa09e59c640 1 --2- 172.21.15.169:0/2381571799 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa0a00cdd20 0x7fa0a01f1220 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fa09000e9f0 tx=0x7fa09000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:52.930 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:52.921+0000 7fa096ffd640 1 -- 172.21.15.169:0/2381571799 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa090009700 con 0x7fa0a00cdd20 2023-10-26T01:36:52.930 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:52.921+0000 7fa096ffd640 1 -- 172.21.15.169:0/2381571799 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa09000ce70 con 0x7fa0a00cdd20 2023-10-26T01:36:52.930 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:52.921+0000 7fa0a4fb6640 1 -- 172.21.15.169:0/2381571799 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0a00c9050 con 0x7fa0a00cdd20 2023-10-26T01:36:52.930 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:52.921+0000 7fa096ffd640 1 -- 172.21.15.169:0/2381571799 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa090010670 con 0x7fa0a00cdd20 2023-10-26T01:36:52.931 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:52.921+0000 7fa0a4fb6640 1 -- 172.21.15.169:0/2381571799 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0a01f1760 con 0x7fa0a00cdd20 2023-10-26T01:36:52.931 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:52.922+0000 7fa0a4fb6640 1 -- 172.21.15.169:0/2381571799 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0a006ce00 con 0x7fa0a00cdd20 2023-10-26T01:36:52.931 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:52.923+0000 7fa096ffd640 1 -- 172.21.15.169:0/2381571799 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fa0900107d0 con 0x7fa0a00cdd20 2023-10-26T01:36:52.932 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:52.924+0000 7fa096ffd640 1 --2- 172.21.15.169:0/2381571799 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa08006ae40 0x7fa08006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:52.932 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:52.924+0000 7fa096ffd640 1 -- 172.21.15.169:0/2381571799 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa090014070 con 0x7fa0a00cdd20 2023-10-26T01:36:52.932 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:52.924+0000 7fa097fff640 1 --2- 172.21.15.169:0/2381571799 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa08006ae40 0x7fa08006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:52.933 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:52.926+0000 7fa097fff640 1 --2- 172.21.15.169:0/2381571799 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa08006ae40 0x7fa08006d300 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fa088035000 tx=0x7fa08803a040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:52.933 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:52.928+0000 7fa096ffd640 1 -- 172.21.15.169:0/2381571799 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa090051ca0 con 0x7fa0a00cdd20 2023-10-26T01:36:53.160 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:53.158+0000 7fa0a4fb6640 1 -- 172.21.15.169:0/2381571799 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fa0a00c88f0 con 0x7fa0a00cdd20 2023-10-26T01:36:53.160 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:53.159+0000 7fa096ffd640 1 -- 172.21.15.169:0/2381571799 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v50) v1 ==== 53+0+73 (secure 0 0 0) 0x7fa0a00c88f0 con 0x7fa0a00cdd20 2023-10-26T01:36:53.161 INFO:teuthology.orchestra.run.smithi169.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-26T01:36:53.163 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:53.161+0000 7fa0a4fb6640 1 -- 172.21.15.169:0/2381571799 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa08006ae40 msgr2=0x7fa08006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:53.164 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:53.161+0000 7fa0a4fb6640 1 --2- 172.21.15.169:0/2381571799 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa08006ae40 0x7fa08006d300 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fa088035000 tx=0x7fa08803a040 comp rx=0 tx=0).stop 2023-10-26T01:36:53.164 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:53.161+0000 7fa0a4fb6640 1 -- 172.21.15.169:0/2381571799 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa0a00cdd20 msgr2=0x7fa0a01f1220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:53.164 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:53.161+0000 7fa0a4fb6640 1 --2- 172.21.15.169:0/2381571799 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa0a00cdd20 0x7fa0a01f1220 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fa09000e9f0 tx=0x7fa09000eeb0 comp rx=0 tx=0).stop 2023-10-26T01:36:53.164 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:53.162+0000 7fa0a4fb6640 1 -- 172.21.15.169:0/2381571799 shutdown_connections 2023-10-26T01:36:53.164 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:53.162+0000 7fa0a4fb6640 1 --2- 172.21.15.169:0/2381571799 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa08006ae40 0x7fa08006d300 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:53.165 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:53.162+0000 7fa0a4fb6640 1 --2- 172.21.15.169:0/2381571799 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa0a015b9e0 0x7fa0a015a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:53.165 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:53.162+0000 7fa0a4fb6640 1 --2- 172.21.15.169:0/2381571799 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa0a0155360 0x7fa0a015a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:53.165 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:53.162+0000 7fa0a4fb6640 1 --2- 172.21.15.169:0/2381571799 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa0a00cdd20 0x7fa0a01f1220 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:53.165 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:53.162+0000 7fa0a4fb6640 1 -- 172.21.15.169:0/2381571799 >> 172.21.15.169:0/2381571799 conn(0x7fa0a0069930 msgr2=0x7fa0a00aed60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:53.165 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:53.162+0000 7fa0a4fb6640 1 -- 172.21.15.169:0/2381571799 shutdown_connections 2023-10-26T01:36:53.166 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:53.162+0000 7fa0a4fb6640 1 -- 172.21.15.169:0/2381571799 wait complete. 2023-10-26T01:36:53.183 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-26T01:36:53.183 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-26T01:36:53.183 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi170.front.sepia.ceph.com 2023-10-26T01:36:53.184 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-26T01:36:53.184 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-26T01:36:53.184 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-26T01:36:53.185 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-26T01:36:53.185 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-26T01:36:53.185 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-26T01:36:53.186 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:36:53.186 DEBUG:teuthology.orchestra.run.smithi170:> ip addr 2023-10-26T01:36:53.248 INFO:teuthology.orchestra.run.smithi170.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-26T01:36:53.249 INFO:teuthology.orchestra.run.smithi170.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-26T01:36:53.249 INFO:teuthology.orchestra.run.smithi170.stdout: inet 127.0.0.1/8 scope host lo 2023-10-26T01:36:53.249 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:36:53.249 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 ::1/128 scope host 2023-10-26T01:36:53.249 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:36:53.250 INFO:teuthology.orchestra.run.smithi170.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-26T01:36:53.250 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:d9:3b:54 brd ff:ff:ff:ff:ff:ff 2023-10-26T01:36:53.250 INFO:teuthology.orchestra.run.smithi170.stdout: altname enp7s0f0 2023-10-26T01:36:53.250 INFO:teuthology.orchestra.run.smithi170.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-26T01:36:53.250 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:d9:3b:55 brd ff:ff:ff:ff:ff:ff 2023-10-26T01:36:53.251 INFO:teuthology.orchestra.run.smithi170.stdout: altname enp7s0f1 2023-10-26T01:36:53.251 INFO:teuthology.orchestra.run.smithi170.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-26T01:36:53.251 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:88:72:54 brd ff:ff:ff:ff:ff:ff 2023-10-26T01:36:53.251 INFO:teuthology.orchestra.run.smithi170.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-26T01:36:53.251 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:88:72:55 brd ff:ff:ff:ff:ff:ff 2023-10-26T01:36:53.251 INFO:teuthology.orchestra.run.smithi170.stdout: inet 172.21.15.170/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-26T01:36:53.252 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft 42067sec preferred_lft 42067sec 2023-10-26T01:36:53.252 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::ec4:7aff:fe88:7255/64 scope link 2023-10-26T01:36:53.252 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:36:53.252 INFO:teuthology.orchestra.run.smithi170.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-26T01:36:53.252 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 32:b5:d1:ee:10:36 brd ff:ff:ff:ff:ff:ff 2023-10-26T01:36:53.253 INFO:teuthology.orchestra.run.smithi170.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-26T01:36:53.253 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:36:53.253 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::28ed:d1ff:fe37:70d/64 scope link 2023-10-26T01:36:53.253 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:36:53.253 INFO:teuthology.orchestra.run.smithi170.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-26T01:36:53.254 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 32:b5:d1:ee:10:36 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-26T01:36:53.254 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::30b5:d1ff:feee:1036/64 scope link 2023-10-26T01:36:53.254 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:36:53.254 INFO:teuthology.orchestra.run.smithi170.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-26T01:36:53.254 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether b2:c1:53:19:69:e4 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-26T01:36:53.255 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::b0c1:53ff:fe19:69e4/64 scope link 2023-10-26T01:36:53.255 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:36:53.256 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:36:53.256 DEBUG:teuthology.orchestra.run.smithi170:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-26T01:36:53.352 INFO:teuthology.orchestra.run.smithi170.stdout:1 2023-10-26T01:36:53.356 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:36:53.357 DEBUG:teuthology.orchestra.run.smithi170:> route 2023-10-26T01:36:53.413 INFO:teuthology.orchestra.run.smithi170.stdout:Kernel IP routing table 2023-10-26T01:36:53.414 INFO:teuthology.orchestra.run.smithi170.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-26T01:36:53.414 INFO:teuthology.orchestra.run.smithi170.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-26T01:36:53.414 INFO:teuthology.orchestra.run.smithi170.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-26T01:36:53.414 INFO:teuthology.orchestra.run.smithi170.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-26T01:36:53.415 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:36:53.416 DEBUG:teuthology.orchestra.run.smithi170:> (cd / && exec bash -c ' 2023-10-26T01:36:53.416 DEBUG:teuthology.orchestra.run.smithi170:> set -e 2023-10-26T01:36:53.417 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-26T01:36:53.417 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-26T01:36:53.417 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-26T01:36:53.418 DEBUG:teuthology.orchestra.run.smithi170:> ') 2023-10-26T01:36:53.605 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:36:53.606 DEBUG:teuthology.orchestra.run.smithi170:> ip netns list 2023-10-26T01:36:53.659 INFO:teuthology.orchestra.run.smithi170.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-26T01:36:53.660 INFO:teuthology.orchestra.run.smithi170.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-26T01:36:53.660 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-26T01:36:53.661 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:36:53.662 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip link set brx.0 up 2023-10-26T01:36:53.755 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-26T01:36:53.756 DEBUG:teuthology.orchestra.run.smithi170:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-26T01:36:53.811 INFO:teuthology.orchestra.run.smithi170.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-26T01:36:53.812 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-26T01:36:53.813 DEBUG:teuthology.orchestra.run.smithi170:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-26T01:36:53.866 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-26T01:36:53.866 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:36:53.867 DEBUG:teuthology.orchestra.run.smithi170:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-26T01:36:54.009 INFO:teuthology.orchestra.run.smithi170.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-26T01:36:54.010 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-26T01:36:54.010 INFO:teuthology.orchestra.run.smithi170.stdout:invalid new device string format 2023-10-26T01:36:54.031 INFO:teuthology.orchestra.run.smithi170.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-26T01:36:54.031 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-26T01:36:54.031 INFO:teuthology.orchestra.run.smithi170.stdout:invalid new device string format 2023-10-26T01:36:54.032 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: resolved to: "172.21.15.169:3300,172.21.15.170:3300,172.21.15.170:3301" 2023-10-26T01:36:54.032 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-26T01:36:54.060 INFO:teuthology.orchestra.run.smithi170.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-26T01:36:54.060 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-26T01:36:54.061 INFO:teuthology.orchestra.run.smithi170.stdout:invalid new device string format 2023-10-26T01:36:54.061 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: resolved to: "172.21.15.169:3300,172.21.15.170:3300,172.21.15.170:3301" 2023-10-26T01:36:54.061 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-26T01:36:54.061 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: resolved to: "172.21.15.169:6789,172.21.15.170:6789,172.21.15.170:6790" 2023-10-26T01:36:54.061 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.169:6789,172.21.15.170:6789,172.21.15.170:6790:/ 2023-10-26T01:36:54.062 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=30203b28-ed8a-44e8-a9a6-a9bf25f980f1" will pass to kernel 2023-10-26T01:36:54.062 INFO:teuthology.orchestra.run.smithi170.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2023-10-26T01:36:54.063 INFO:teuthology.orchestra.run.smithi170.stdout: You just mounted a file system that supports labels which does not 2023-10-26T01:36:54.063 INFO:teuthology.orchestra.run.smithi170.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-10-26T01:36:54.063 INFO:teuthology.orchestra.run.smithi170.stdout: applications will generate AVC messages and not be allowed access to 2023-10-26T01:36:54.063 INFO:teuthology.orchestra.run.smithi170.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-10-26T01:36:54.067 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-26T01:36:54.068 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:36:54.068 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-26T01:36:54.155 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:36:54.156 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-26T01:36:54.211 INFO:teuthology.orchestra.run.smithi170.stdout:client5581 2023-10-26T01:36:54.212 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-26T01:36:54.212 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:36:54.265 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T01:36:54.493 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:54.491+0000 7f44dc75e640 1 -- 172.21.15.169:0/2580640403 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44d41592d0 msgr2=0x7f44d40aa3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:54.493 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:54.491+0000 7f44dc75e640 1 --2- 172.21.15.169:0/2580640403 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44d41592d0 0x7f44d40aa3d0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f44d40b2650 tx=0x7f44d002bcf0 comp rx=0 tx=0).stop 2023-10-26T01:36:54.493 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:54.491+0000 7f44dc75e640 1 -- 172.21.15.169:0/2580640403 shutdown_connections 2023-10-26T01:36:54.494 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:54.491+0000 7f44dc75e640 1 --2- 172.21.15.169:0/2580640403 >> v2:172.21.15.170:3300/0 conn(0x7f44d40ac930 0x7f44d40cd6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:54.494 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:54.491+0000 7f44dc75e640 1 --2- 172.21.15.169:0/2580640403 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44d41592d0 0x7f44d40aa3d0 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:54.494 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:54.491+0000 7f44dc75e640 1 -- 172.21.15.169:0/2580640403 >> 172.21.15.169:0/2580640403 conn(0x7f44d4069930 msgr2=0x7f44d4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:54.494 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:54.491+0000 7f44dc75e640 1 -- 172.21.15.169:0/2580640403 shutdown_connections 2023-10-26T01:36:54.494 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:54.491+0000 7f44dc75e640 1 -- 172.21.15.169:0/2580640403 wait complete. 2023-10-26T01:36:54.496 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:54.493+0000 7f44dc75e640 1 Processor -- start 2023-10-26T01:36:54.496 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:54.493+0000 7f44dc75e640 1 -- start start 2023-10-26T01:36:54.496 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:54.494+0000 7f44dc75e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44d40ac930 0x7f44d41f5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:54.496 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:54.494+0000 7f44dc75e640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f44d40cdc30 0x7f44d41e3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:54.497 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:54.494+0000 7f44dc75e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f44d41592d0 0x7f44d41e40c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:54.497 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:54.494+0000 7f44dc75e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44d41e4a20 con 0x7f44d40ac930 2023-10-26T01:36:54.497 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:54.494+0000 7f44dc75e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44d406f590 con 0x7f44d41592d0 2023-10-26T01:36:54.497 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:54.494+0000 7f44dc75e640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44d41e4740 con 0x7f44d40cdc30 2023-10-26T01:36:54.497 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:54.494+0000 7f44da4fa640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44d40ac930 0x7f44d41f5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:54.498 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:54.494+0000 7f44da4fa640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44d40ac930 0x7f44d41f5210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:38360/0 (socket says 172.21.15.169:38360) 2023-10-26T01:36:54.498 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:54.494+0000 7f44da4fa640 1 -- 172.21.15.169:0/3869326877 learned_addr learned my addr 172.21.15.169:0/3869326877 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:36:54.498 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:54.494+0000 7f44da4fa640 1 -- 172.21.15.169:0/3869326877 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f44d40cdc30 msgr2=0x7f44d41e3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:54.498 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:54.494+0000 7f44da4fa640 1 --2- 172.21.15.169:0/3869326877 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f44d40cdc30 0x7f44d41e3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:54.498 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:54.494+0000 7f44da4fa640 1 -- 172.21.15.169:0/3869326877 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f44d41592d0 msgr2=0x7f44d41e40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:54.499 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:54.494+0000 7f44da4fa640 1 --2- 172.21.15.169:0/3869326877 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f44d41592d0 0x7f44d41e40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:54.499 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:54.494+0000 7f44da4fa640 1 -- 172.21.15.169:0/3869326877 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44d0009490 con 0x7f44d40ac930 2023-10-26T01:36:54.500 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:54.495+0000 7f44da4fa640 1 --2- 172.21.15.169:0/3869326877 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44d40ac930 0x7f44d41f5210 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f44d0000c00 tx=0x7f44d0004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:54.500 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:54.495+0000 7f44caffd640 1 -- 172.21.15.169:0/3869326877 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d0003a00 con 0x7f44d40ac930 2023-10-26T01:36:54.500 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:54.495+0000 7f44caffd640 1 -- 172.21.15.169:0/3869326877 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f44d0003c70 con 0x7f44d40ac930 2023-10-26T01:36:54.501 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:54.495+0000 7f44dc75e640 1 -- 172.21.15.169:0/3869326877 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44d406eed0 con 0x7f44d40ac930 2023-10-26T01:36:54.501 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:54.495+0000 7f44caffd640 1 -- 172.21.15.169:0/3869326877 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d0034440 con 0x7f44d40ac930 2023-10-26T01:36:54.501 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:54.495+0000 7f44dc75e640 1 -- 172.21.15.169:0/3869326877 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44d4071c30 con 0x7f44d40ac930 2023-10-26T01:36:54.501 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:54.496+0000 7f44dc75e640 1 -- 172.21.15.169:0/3869326877 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44d406ed30 con 0x7f44d40ac930 2023-10-26T01:36:54.501 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:54.497+0000 7f44caffd640 1 -- 172.21.15.169:0/3869326877 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f44d003b040 con 0x7f44d40ac930 2023-10-26T01:36:54.502 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:54.498+0000 7f44caffd640 1 --2- 172.21.15.169:0/3869326877 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f44b806ae40 0x7f44b806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:54.502 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:54.498+0000 7f44caffd640 1 -- 172.21.15.169:0/3869326877 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f44d00ab790 con 0x7f44d40ac930 2023-10-26T01:36:54.502 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:54.499+0000 7f44d9cf9640 1 --2- 172.21.15.169:0/3869326877 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f44b806ae40 0x7f44b806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:54.503 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:54.500+0000 7f44d9cf9640 1 --2- 172.21.15.169:0/3869326877 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f44b806ae40 0x7f44b806d300 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f44c4005fd0 tx=0x7f44c4005850 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:54.504 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:54.503+0000 7f44caffd640 1 -- 172.21.15.169:0/3869326877 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f44d406ed30 con 0x7f44d40ac930 2023-10-26T01:36:54.730 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:54.728+0000 7f44dc75e640 1 -- 172.21.15.169:0/3869326877 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f44d406ed30 con 0x7f44d40ac930 2023-10-26T01:36:54.731 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:54.729+0000 7f44caffd640 1 -- 172.21.15.169:0/3869326877 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+4924 (secure 0 0 0) 0x7f44d406ed30 con 0x7f44d40ac930 2023-10-26T01:36:54.731 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:36:54.731 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":50,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5240,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/3870332417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":3870332417},{"type":"v1","addr":"172.21.15.170:6837","nonce":3870332417}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45},{"gid":5497,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/2533771698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2533771698},{"type":"v1","addr":"172.21.15.170:6835","nonce":2533771698}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45},{"gid":6885,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/2117946713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2117946713},{"type":"v1","addr":"172.21.15.169:6834","nonce":2117946713}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T01:36:43.849046+0000","modified":"2023-10-26T01:36:46.926339+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6888},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6888":{"gid":6888,"name":"c","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6835/657142078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":657142078},{"type":"v1","addr":"172.21.15.169:6835","nonce":657142078}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-10-26T01:36:54.732 INFO:teuthology.orchestra.run.smithi169.stderr:dumped fsmap epoch 50 2023-10-26T01:36:54.734 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:54.732+0000 7f44dc75e640 1 -- 172.21.15.169:0/3869326877 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f44b806ae40 msgr2=0x7f44b806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:54.734 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:54.732+0000 7f44dc75e640 1 --2- 172.21.15.169:0/3869326877 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f44b806ae40 0x7f44b806d300 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f44c4005fd0 tx=0x7f44c4005850 comp rx=0 tx=0).stop 2023-10-26T01:36:54.734 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:54.732+0000 7f44dc75e640 1 -- 172.21.15.169:0/3869326877 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44d40ac930 msgr2=0x7f44d41f5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:54.734 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:54.732+0000 7f44dc75e640 1 --2- 172.21.15.169:0/3869326877 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44d40ac930 0x7f44d41f5210 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f44d0000c00 tx=0x7f44d0004410 comp rx=0 tx=0).stop 2023-10-26T01:36:54.735 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:54.732+0000 7f44dc75e640 1 -- 172.21.15.169:0/3869326877 shutdown_connections 2023-10-26T01:36:54.735 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:54.732+0000 7f44dc75e640 1 --2- 172.21.15.169:0/3869326877 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f44b806ae40 0x7f44b806d300 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:54.735 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:54.732+0000 7f44dc75e640 1 --2- 172.21.15.169:0/3869326877 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f44d41592d0 0x7f44d41e40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:54.735 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:54.732+0000 7f44dc75e640 1 --2- 172.21.15.169:0/3869326877 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f44d40cdc30 0x7f44d41e3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:54.735 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:54.732+0000 7f44dc75e640 1 --2- 172.21.15.169:0/3869326877 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44d40ac930 0x7f44d41f5210 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:54.736 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:54.732+0000 7f44dc75e640 1 -- 172.21.15.169:0/3869326877 >> 172.21.15.169:0/3869326877 conn(0x7f44d4069930 msgr2=0x7f44d40af9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:54.736 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:54.732+0000 7f44dc75e640 1 -- 172.21.15.169:0/3869326877 shutdown_connections 2023-10-26T01:36:54.736 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:54.732+0000 7f44dc75e640 1 -- 172.21.15.169:0/3869326877 wait complete. 2023-10-26T01:36:54.753 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2023-10-26T01:36:54.991 INFO:teuthology.orchestra.run.smithi169.stdout:{"cluster_fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","whoami":0,"id":6888,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":10.558564045000001,"mdsmap_epoch":50,"osdmap_epoch":103,"osdmap_epoch_barrier":103,"uptime":17.409713406000002} 2023-10-26T01:36:54.992 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "30203b28-ed8a-44e8-a9a6-a9bf25f980f1", "fs_name": "cephfs", "id": 6888, "mdsmap_epoch": 50, "osdmap_epoch": 103, "osdmap_epoch_barrier": 103, "rank_uptime": 10.558564045, "state": "up:active", "uptime": 17.409713406, "want_state": "up:active", "whoami": 0 } 2023-10-26T01:36:54.992 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2023-10-26T01:36:55.227 INFO:teuthology.orchestra.run.smithi169.stdout:{"osd_mon_report_interval":"5"} 2023-10-26T01:36:55.227 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2023-10-26T01:36:55.228 INFO:tasks.cephfs.test_full:Writing 531MB should fill this cluster 2023-10-26T01:36:55.228 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:36:55.228 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:36:55.282 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:36:55.283 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_a bs=1M conv=fdatasync count=265 seek=0) 2023-10-26T01:36:57.352 INFO:teuthology.orchestra.run.smithi169.stderr:265+0 records in 2023-10-26T01:36:57.352 INFO:teuthology.orchestra.run.smithi169.stderr:265+0 records out 2023-10-26T01:36:57.352 INFO:teuthology.orchestra.run.smithi169.stderr:277872640 bytes (278 MB, 265 MiB) copied, 2.01236 s, 138 MB/s 2023-10-26T01:36:57.353 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:36:57.354 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:36:57.393 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:36:57.393 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=292 seek=0) 2023-10-26T01:36:59.498 INFO:teuthology.orchestra.run.smithi169.stderr:292+0 records in 2023-10-26T01:36:59.498 INFO:teuthology.orchestra.run.smithi169.stderr:292+0 records out 2023-10-26T01:36:59.498 INFO:teuthology.orchestra.run.smithi169.stderr:306184192 bytes (306 MB, 292 MiB) copied, 2.00902 s, 152 MB/s 2023-10-26T01:36:59.499 INFO:tasks.cephfs.test_full:Writing file B succeeded (full status will happen soon) 2023-10-26T01:36:59.500 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-26T01:36:59.689 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:59.687+0000 7fa3db6a9640 1 -- 172.21.15.169:0/1079846111 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa3d40cde00 msgr2=0x7fa3d40cbac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:59.689 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:59.687+0000 7fa3db6a9640 1 --2- 172.21.15.169:0/1079846111 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa3d40cde00 0x7fa3d40cbac0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fa3d406be60 tx=0x7fa3cc02c260 comp rx=0 tx=0).stop 2023-10-26T01:36:59.690 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:59.687+0000 7fa3db6a9640 1 -- 172.21.15.169:0/1079846111 shutdown_connections 2023-10-26T01:36:59.690 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:59.687+0000 7fa3db6a9640 1 --2- 172.21.15.169:0/1079846111 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa3d40cde00 0x7fa3d40cbac0 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:59.690 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:59.687+0000 7fa3db6a9640 1 --2- 172.21.15.169:0/1079846111 >> v2:172.21.15.170:3300/0 conn(0x7fa3d41592d0 0x7fa3d40aa3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:59.690 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:59.687+0000 7fa3db6a9640 1 -- 172.21.15.169:0/1079846111 >> 172.21.15.169:0/1079846111 conn(0x7fa3d4069930 msgr2=0x7fa3d4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:36:59.690 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:59.687+0000 7fa3db6a9640 1 -- 172.21.15.169:0/1079846111 shutdown_connections 2023-10-26T01:36:59.691 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:59.687+0000 7fa3db6a9640 1 -- 172.21.15.169:0/1079846111 wait complete. 2023-10-26T01:36:59.692 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:59.689+0000 7fa3db6a9640 1 Processor -- start 2023-10-26T01:36:59.693 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:59.689+0000 7fa3db6a9640 1 -- start start 2023-10-26T01:36:59.693 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:59.690+0000 7fa3db6a9640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa3d40ac930 0x7fa3d41f5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:59.693 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:59.690+0000 7fa3db6a9640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa3d40cde00 0x7fa3d40c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:59.693 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:59.690+0000 7fa3db6a9640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa3d41592d0 0x7fa3d41f0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:59.694 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:59.690+0000 7fa3db6a9640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d40c90d0 con 0x7fa3d40cde00 2023-10-26T01:36:59.694 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:59.690+0000 7fa3db6a9640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d40c8240 con 0x7fa3d41592d0 2023-10-26T01:36:59.694 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:59.690+0000 7fa3db6a9640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d40c8450 con 0x7fa3d40ac930 2023-10-26T01:36:59.694 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:59.690+0000 7fa3d8c44640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa3d40cde00 0x7fa3d40c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:59.694 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:59.690+0000 7fa3d8c44640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa3d40cde00 0x7fa3d40c66f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:38380/0 (socket says 172.21.15.169:38380) 2023-10-26T01:36:59.695 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:59.690+0000 7fa3d8c44640 1 -- 172.21.15.169:0/4142503073 learned_addr learned my addr 172.21.15.169:0/4142503073 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:36:59.695 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:59.690+0000 7fa3d8c44640 1 -- 172.21.15.169:0/4142503073 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa3d40ac930 msgr2=0x7fa3d41f5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:59.695 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:59.690+0000 7fa3d8c44640 1 --2- 172.21.15.169:0/4142503073 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa3d40ac930 0x7fa3d41f5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:59.695 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:59.690+0000 7fa3d8c44640 1 -- 172.21.15.169:0/4142503073 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa3d41592d0 msgr2=0x7fa3d41f0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:36:59.695 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:59.690+0000 7fa3d8c44640 1 --2- 172.21.15.169:0/4142503073 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa3d41592d0 0x7fa3d41f0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:36:59.696 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:59.690+0000 7fa3d8c44640 1 -- 172.21.15.169:0/4142503073 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3cc009900 con 0x7fa3d40cde00 2023-10-26T01:36:59.696 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:59.691+0000 7fa3d8c44640 1 --2- 172.21.15.169:0/4142503073 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa3d40cde00 0x7fa3d40c66f0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fa3c800c450 tx=0x7fa3c800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:59.696 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:59.691+0000 7fa3c5ffb640 1 -- 172.21.15.169:0/4142503073 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c80052d0 con 0x7fa3d40cde00 2023-10-26T01:36:59.696 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:59.691+0000 7fa3c5ffb640 1 -- 172.21.15.169:0/4142503073 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa3c8005430 con 0x7fa3d40cde00 2023-10-26T01:36:59.696 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:59.691+0000 7fa3db6a9640 1 -- 172.21.15.169:0/4142503073 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3d40c8a10 con 0x7fa3d40cde00 2023-10-26T01:36:59.697 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:59.691+0000 7fa3c5ffb640 1 -- 172.21.15.169:0/4142503073 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c8012890 con 0x7fa3d40cde00 2023-10-26T01:36:59.697 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:59.691+0000 7fa3db6a9640 1 -- 172.21.15.169:0/4142503073 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3d40c94d0 con 0x7fa3d40cde00 2023-10-26T01:36:59.698 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:59.692+0000 7fa3db6a9640 1 -- 172.21.15.169:0/4142503073 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3d40c9340 con 0x7fa3d40cde00 2023-10-26T01:36:59.698 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:59.693+0000 7fa3c5ffb640 1 -- 172.21.15.169:0/4142503073 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fa3c80035f0 con 0x7fa3d40cde00 2023-10-26T01:36:59.698 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:59.694+0000 7fa3c5ffb640 1 --2- 172.21.15.169:0/4142503073 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa3b806ae40 0x7fa3b806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:36:59.698 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:59.694+0000 7fa3c5ffb640 1 -- 172.21.15.169:0/4142503073 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa3c80892d0 con 0x7fa3d40cde00 2023-10-26T01:36:59.699 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:59.695+0000 7fa3d9445640 1 --2- 172.21.15.169:0/4142503073 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa3b806ae40 0x7fa3b806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:36:59.699 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:59.696+0000 7fa3d9445640 1 --2- 172.21.15.169:0/4142503073 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa3b806ae40 0x7fa3b806d300 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fa3d000a860 tx=0x7fa3d0005b50 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:36:59.699 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:59.698+0000 7fa3c5ffb640 1 -- 172.21.15.169:0/4142503073 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa3c8054370 con 0x7fa3d40cde00 2023-10-26T01:36:59.875 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:59.873+0000 7fa3db6a9640 1 -- 172.21.15.169:0/4142503073 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fa3d404db10 con 0x7fa3d40cde00 2023-10-26T01:36:59.878 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:59.877+0000 7fa3c5ffb640 1 -- 172.21.15.169:0/4142503073 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v103) v1 ==== 81+0+34054 (secure 0 0 0) 0x7fa3c8057990 con 0x7fa3d40cde00 2023-10-26T01:36:59.879 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:36:59.879 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2023-10-26T01:36:59.880 INFO:teuthology.orchestra.run.smithi169.stdout: "epoch": 103, 2023-10-26T01:36:59.880 INFO:teuthology.orchestra.run.smithi169.stdout: "fsid": "30203b28-ed8a-44e8-a9a6-a9bf25f980f1", 2023-10-26T01:36:59.880 INFO:teuthology.orchestra.run.smithi169.stdout: "created": "2023-10-26T01:30:22.897155+0000", 2023-10-26T01:36:59.880 INFO:teuthology.orchestra.run.smithi169.stdout: "modified": "2023-10-26T01:36:45.388928+0000", 2023-10-26T01:36:59.881 INFO:teuthology.orchestra.run.smithi169.stdout: "last_up_change": "2023-10-26T01:30:31.875281+0000", 2023-10-26T01:36:59.881 INFO:teuthology.orchestra.run.smithi169.stdout: "last_in_change": "2023-10-26T01:30:27.637941+0000", 2023-10-26T01:36:59.881 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-26T01:36:59.881 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_num": 5799936, 2023-10-26T01:36:59.882 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_set": [ 2023-10-26T01:36:59.882 INFO:teuthology.orchestra.run.smithi169.stdout: "pglog_hardlimit", 2023-10-26T01:36:59.882 INFO:teuthology.orchestra.run.smithi169.stdout: "purged_snapdirs", 2023-10-26T01:36:59.882 INFO:teuthology.orchestra.run.smithi169.stdout: "recovery_deletes", 2023-10-26T01:36:59.882 INFO:teuthology.orchestra.run.smithi169.stdout: "sortbitwise" 2023-10-26T01:36:59.883 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:36:59.883 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_version": 4, 2023-10-26T01:36:59.883 INFO:teuthology.orchestra.run.smithi169.stdout: "full_ratio": 0.69999998807907104, 2023-10-26T01:36:59.883 INFO:teuthology.orchestra.run.smithi169.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-26T01:36:59.884 INFO:teuthology.orchestra.run.smithi169.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-26T01:36:59.884 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_snapshot": "", 2023-10-26T01:36:59.884 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_max": 13, 2023-10-26T01:36:59.884 INFO:teuthology.orchestra.run.smithi169.stdout: "max_osd": 8, 2023-10-26T01:36:59.885 INFO:teuthology.orchestra.run.smithi169.stdout: "require_min_compat_client": "luminous", 2023-10-26T01:36:59.885 INFO:teuthology.orchestra.run.smithi169.stdout: "min_compat_client": "jewel", 2023-10-26T01:36:59.885 INFO:teuthology.orchestra.run.smithi169.stdout: "require_osd_release": "reef", 2023-10-26T01:36:59.885 INFO:teuthology.orchestra.run.smithi169.stdout: "allow_crimson": false, 2023-10-26T01:36:59.886 INFO:teuthology.orchestra.run.smithi169.stdout: "pools": [ 2023-10-26T01:36:59.886 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:36:59.886 INFO:teuthology.orchestra.run.smithi169.stdout: "pool": 1, 2023-10-26T01:36:59.886 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_name": ".mgr", 2023-10-26T01:36:59.887 INFO:teuthology.orchestra.run.smithi169.stdout: "create_time": "2023-10-26T01:30:32.200808+0000", 2023-10-26T01:36:59.887 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": 1, 2023-10-26T01:36:59.887 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_names": "hashpspool", 2023-10-26T01:36:59.888 INFO:teuthology.orchestra.run.smithi169.stdout: "type": 1, 2023-10-26T01:36:59.888 INFO:teuthology.orchestra.run.smithi169.stdout: "size": 2, 2023-10-26T01:36:59.888 INFO:teuthology.orchestra.run.smithi169.stdout: "min_size": 1, 2023-10-26T01:36:59.888 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_rule": 0, 2023-10-26T01:36:59.888 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_count": 0, 2023-10-26T01:36:59.889 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_target": 0, 2023-10-26T01:36:59.889 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T01:36:59.889 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T01:36:59.890 INFO:teuthology.orchestra.run.smithi169.stdout: "object_hash": 2, 2023-10-26T01:36:59.890 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_mode": "off", 2023-10-26T01:36:59.890 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num": 1, 2023-10-26T01:36:59.891 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num": 1, 2023-10-26T01:36:59.891 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num_target": 1, 2023-10-26T01:36:59.891 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_target": 1, 2023-10-26T01:36:59.891 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_pending": 1, 2023-10-26T01:36:59.892 INFO:teuthology.orchestra.run.smithi169.stdout: "last_pg_merge_meta": { 2023-10-26T01:36:59.892 INFO:teuthology.orchestra.run.smithi169.stdout: "source_pgid": "0.0", 2023-10-26T01:36:59.892 INFO:teuthology.orchestra.run.smithi169.stdout: "ready_epoch": 0, 2023-10-26T01:36:59.892 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_started": 0, 2023-10-26T01:36:59.893 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_clean": 0, 2023-10-26T01:36:59.893 INFO:teuthology.orchestra.run.smithi169.stdout: "source_version": "0'0", 2023-10-26T01:36:59.893 INFO:teuthology.orchestra.run.smithi169.stdout: "target_version": "0'0" 2023-10-26T01:36:59.893 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:36:59.894 INFO:teuthology.orchestra.run.smithi169.stdout: "last_change": "90", 2023-10-26T01:36:59.894 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend": "0", 2023-10-26T01:36:59.894 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T01:36:59.894 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T01:36:59.894 INFO:teuthology.orchestra.run.smithi169.stdout: "auid": 0, 2023-10-26T01:36:59.895 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_mode": "selfmanaged", 2023-10-26T01:36:59.895 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_seq": 0, 2023-10-26T01:36:59.895 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_epoch": 0, 2023-10-26T01:36:59.895 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_snaps": [], 2023-10-26T01:36:59.895 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps": "[]", 2023-10-26T01:36:59.896 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_bytes": 0, 2023-10-26T01:36:59.896 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_objects": 0, 2023-10-26T01:36:59.896 INFO:teuthology.orchestra.run.smithi169.stdout: "tiers": [], 2023-10-26T01:36:59.896 INFO:teuthology.orchestra.run.smithi169.stdout: "tier_of": -1, 2023-10-26T01:36:59.896 INFO:teuthology.orchestra.run.smithi169.stdout: "read_tier": -1, 2023-10-26T01:36:59.897 INFO:teuthology.orchestra.run.smithi169.stdout: "write_tier": -1, 2023-10-26T01:36:59.897 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_mode": "none", 2023-10-26T01:36:59.897 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_bytes": 0, 2023-10-26T01:36:59.898 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_objects": 0, 2023-10-26T01:36:59.898 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T01:36:59.898 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T01:36:59.898 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T01:36:59.898 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_flush_age": 0, 2023-10-26T01:36:59.899 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_evict_age": 0, 2023-10-26T01:36:59.899 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profile": "", 2023-10-26T01:36:59.899 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_params": { 2023-10-26T01:36:59.899 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "none" 2023-10-26T01:36:59.900 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:36:59.900 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_period": 0, 2023-10-26T01:36:59.900 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_count": 0, 2023-10-26T01:36:59.900 INFO:teuthology.orchestra.run.smithi169.stdout: "use_gmt_hitset": true, 2023-10-26T01:36:59.900 INFO:teuthology.orchestra.run.smithi169.stdout: "min_read_recency_for_promote": 0, 2023-10-26T01:36:59.901 INFO:teuthology.orchestra.run.smithi169.stdout: "min_write_recency_for_promote": 0, 2023-10-26T01:36:59.901 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T01:36:59.901 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_search_last_n": 0, 2023-10-26T01:36:59.901 INFO:teuthology.orchestra.run.smithi169.stdout: "grade_table": [], 2023-10-26T01:36:59.902 INFO:teuthology.orchestra.run.smithi169.stdout: "stripe_width": 0, 2023-10-26T01:36:59.902 INFO:teuthology.orchestra.run.smithi169.stdout: "expected_num_objects": 0, 2023-10-26T01:36:59.902 INFO:teuthology.orchestra.run.smithi169.stdout: "fast_read": false, 2023-10-26T01:36:59.902 INFO:teuthology.orchestra.run.smithi169.stdout: "options": { 2023-10-26T01:36:59.903 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_max": 32, 2023-10-26T01:36:59.903 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_min": 1 2023-10-26T01:36:59.903 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:36:59.903 INFO:teuthology.orchestra.run.smithi169.stdout: "application_metadata": { 2023-10-26T01:36:59.903 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": {} 2023-10-26T01:36:59.904 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:36:59.904 INFO:teuthology.orchestra.run.smithi169.stdout: "read_balance": { 2023-10-26T01:36:59.904 INFO:teuthology.orchestra.run.smithi169.stdout: "score_acting": 8, 2023-10-26T01:36:59.904 INFO:teuthology.orchestra.run.smithi169.stdout: "score_stable": 8, 2023-10-26T01:36:59.904 INFO:teuthology.orchestra.run.smithi169.stdout: "optimal_score": 0.25, 2023-10-26T01:36:59.905 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_acting": 2, 2023-10-26T01:36:59.905 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_stable": 2, 2023-10-26T01:36:59.905 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity_weighted": 1, 2023-10-26T01:36:59.905 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity": 1, 2023-10-26T01:36:59.905 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity_weighted": 1 2023-10-26T01:36:59.906 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:36:59.906 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:36:59.906 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:36:59.906 INFO:teuthology.orchestra.run.smithi169.stdout: "pool": 12, 2023-10-26T01:36:59.907 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_name": "cephfs_metadata", 2023-10-26T01:36:59.907 INFO:teuthology.orchestra.run.smithi169.stdout: "create_time": "2023-10-26T01:36:41.430455+0000", 2023-10-26T01:36:59.907 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": 1, 2023-10-26T01:36:59.907 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_names": "hashpspool", 2023-10-26T01:36:59.907 INFO:teuthology.orchestra.run.smithi169.stdout: "type": 1, 2023-10-26T01:36:59.908 INFO:teuthology.orchestra.run.smithi169.stdout: "size": 2, 2023-10-26T01:36:59.908 INFO:teuthology.orchestra.run.smithi169.stdout: "min_size": 1, 2023-10-26T01:36:59.908 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_rule": 0, 2023-10-26T01:36:59.908 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_count": 0, 2023-10-26T01:36:59.908 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_target": 0, 2023-10-26T01:36:59.909 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T01:36:59.909 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T01:36:59.909 INFO:teuthology.orchestra.run.smithi169.stdout: "object_hash": 2, 2023-10-26T01:36:59.909 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_mode": "off", 2023-10-26T01:36:59.909 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num": 32, 2023-10-26T01:36:59.910 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num": 32, 2023-10-26T01:36:59.910 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num_target": 32, 2023-10-26T01:36:59.910 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_target": 32, 2023-10-26T01:36:59.910 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_pending": 32, 2023-10-26T01:36:59.911 INFO:teuthology.orchestra.run.smithi169.stdout: "last_pg_merge_meta": { 2023-10-26T01:36:59.911 INFO:teuthology.orchestra.run.smithi169.stdout: "source_pgid": "0.0", 2023-10-26T01:36:59.911 INFO:teuthology.orchestra.run.smithi169.stdout: "ready_epoch": 0, 2023-10-26T01:36:59.911 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_started": 0, 2023-10-26T01:36:59.912 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_clean": 0, 2023-10-26T01:36:59.912 INFO:teuthology.orchestra.run.smithi169.stdout: "source_version": "0'0", 2023-10-26T01:36:59.913 INFO:teuthology.orchestra.run.smithi169.stdout: "target_version": "0'0" 2023-10-26T01:36:59.913 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:36:59.913 INFO:teuthology.orchestra.run.smithi169.stdout: "last_change": "102", 2023-10-26T01:36:59.914 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend": "0", 2023-10-26T01:36:59.914 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T01:36:59.915 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T01:36:59.915 INFO:teuthology.orchestra.run.smithi169.stdout: "auid": 0, 2023-10-26T01:36:59.915 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_mode": "selfmanaged", 2023-10-26T01:36:59.916 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_seq": 0, 2023-10-26T01:36:59.916 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_epoch": 0, 2023-10-26T01:36:59.917 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_snaps": [], 2023-10-26T01:36:59.917 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps": "[]", 2023-10-26T01:36:59.917 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_bytes": 0, 2023-10-26T01:36:59.918 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_objects": 0, 2023-10-26T01:36:59.918 INFO:teuthology.orchestra.run.smithi169.stdout: "tiers": [], 2023-10-26T01:36:59.918 INFO:teuthology.orchestra.run.smithi169.stdout: "tier_of": -1, 2023-10-26T01:36:59.919 INFO:teuthology.orchestra.run.smithi169.stdout: "read_tier": -1, 2023-10-26T01:36:59.919 INFO:teuthology.orchestra.run.smithi169.stdout: "write_tier": -1, 2023-10-26T01:36:59.919 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_mode": "none", 2023-10-26T01:36:59.920 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_bytes": 0, 2023-10-26T01:36:59.920 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_objects": 0, 2023-10-26T01:36:59.921 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T01:36:59.921 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T01:36:59.921 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T01:36:59.922 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_flush_age": 0, 2023-10-26T01:36:59.922 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_evict_age": 0, 2023-10-26T01:36:59.922 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profile": "", 2023-10-26T01:36:59.922 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_params": { 2023-10-26T01:36:59.922 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "none" 2023-10-26T01:36:59.923 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:36:59.923 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_period": 0, 2023-10-26T01:36:59.923 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_count": 0, 2023-10-26T01:36:59.923 INFO:teuthology.orchestra.run.smithi169.stdout: "use_gmt_hitset": true, 2023-10-26T01:36:59.924 INFO:teuthology.orchestra.run.smithi169.stdout: "min_read_recency_for_promote": 0, 2023-10-26T01:36:59.924 INFO:teuthology.orchestra.run.smithi169.stdout: "min_write_recency_for_promote": 0, 2023-10-26T01:36:59.924 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T01:36:59.924 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_search_last_n": 0, 2023-10-26T01:36:59.924 INFO:teuthology.orchestra.run.smithi169.stdout: "grade_table": [], 2023-10-26T01:36:59.925 INFO:teuthology.orchestra.run.smithi169.stdout: "stripe_width": 0, 2023-10-26T01:36:59.925 INFO:teuthology.orchestra.run.smithi169.stdout: "expected_num_objects": 0, 2023-10-26T01:36:59.925 INFO:teuthology.orchestra.run.smithi169.stdout: "fast_read": false, 2023-10-26T01:36:59.925 INFO:teuthology.orchestra.run.smithi169.stdout: "options": { 2023-10-26T01:36:59.925 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_bias": 4, 2023-10-26T01:36:59.926 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_min": 16, 2023-10-26T01:36:59.926 INFO:teuthology.orchestra.run.smithi169.stdout: "recovery_priority": 5 2023-10-26T01:36:59.926 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:36:59.926 INFO:teuthology.orchestra.run.smithi169.stdout: "application_metadata": { 2023-10-26T01:36:59.927 INFO:teuthology.orchestra.run.smithi169.stdout: "cephfs": { 2023-10-26T01:36:59.927 INFO:teuthology.orchestra.run.smithi169.stdout: "metadata": "cephfs" 2023-10-26T01:36:59.927 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:36:59.927 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:36:59.927 INFO:teuthology.orchestra.run.smithi169.stdout: "read_balance": { 2023-10-26T01:36:59.928 INFO:teuthology.orchestra.run.smithi169.stdout: "score_acting": 2, 2023-10-26T01:36:59.928 INFO:teuthology.orchestra.run.smithi169.stdout: "score_stable": 2, 2023-10-26T01:36:59.928 INFO:teuthology.orchestra.run.smithi169.stdout: "optimal_score": 1, 2023-10-26T01:36:59.928 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_acting": 2, 2023-10-26T01:36:59.928 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_stable": 2, 2023-10-26T01:36:59.929 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity_weighted": 1, 2023-10-26T01:36:59.929 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity": 1, 2023-10-26T01:36:59.929 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity_weighted": 1 2023-10-26T01:36:59.929 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:36:59.930 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:36:59.930 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:36:59.930 INFO:teuthology.orchestra.run.smithi169.stdout: "pool": 13, 2023-10-26T01:36:59.930 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_name": "cephfs_data", 2023-10-26T01:36:59.930 INFO:teuthology.orchestra.run.smithi169.stdout: "create_time": "2023-10-26T01:36:42.883252+0000", 2023-10-26T01:36:59.931 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": 1, 2023-10-26T01:36:59.931 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_names": "hashpspool", 2023-10-26T01:36:59.931 INFO:teuthology.orchestra.run.smithi169.stdout: "type": 1, 2023-10-26T01:36:59.932 INFO:teuthology.orchestra.run.smithi169.stdout: "size": 2, 2023-10-26T01:36:59.932 INFO:teuthology.orchestra.run.smithi169.stdout: "min_size": 1, 2023-10-26T01:36:59.932 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_rule": 0, 2023-10-26T01:36:59.933 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_count": 0, 2023-10-26T01:36:59.933 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_target": 0, 2023-10-26T01:36:59.933 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T01:36:59.933 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T01:36:59.934 INFO:teuthology.orchestra.run.smithi169.stdout: "object_hash": 2, 2023-10-26T01:36:59.934 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_mode": "off", 2023-10-26T01:36:59.934 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num": 64, 2023-10-26T01:36:59.934 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num": 64, 2023-10-26T01:36:59.935 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num_target": 64, 2023-10-26T01:36:59.935 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_target": 64, 2023-10-26T01:36:59.935 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_pending": 64, 2023-10-26T01:36:59.935 INFO:teuthology.orchestra.run.smithi169.stdout: "last_pg_merge_meta": { 2023-10-26T01:36:59.935 INFO:teuthology.orchestra.run.smithi169.stdout: "source_pgid": "0.0", 2023-10-26T01:36:59.936 INFO:teuthology.orchestra.run.smithi169.stdout: "ready_epoch": 0, 2023-10-26T01:36:59.936 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_started": 0, 2023-10-26T01:36:59.936 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_clean": 0, 2023-10-26T01:36:59.936 INFO:teuthology.orchestra.run.smithi169.stdout: "source_version": "0'0", 2023-10-26T01:36:59.936 INFO:teuthology.orchestra.run.smithi169.stdout: "target_version": "0'0" 2023-10-26T01:36:59.937 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:36:59.937 INFO:teuthology.orchestra.run.smithi169.stdout: "last_change": "103", 2023-10-26T01:36:59.937 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend": "0", 2023-10-26T01:36:59.937 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T01:36:59.937 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T01:36:59.938 INFO:teuthology.orchestra.run.smithi169.stdout: "auid": 0, 2023-10-26T01:36:59.938 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_mode": "selfmanaged", 2023-10-26T01:36:59.938 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_seq": 0, 2023-10-26T01:36:59.938 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_epoch": 0, 2023-10-26T01:36:59.939 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_snaps": [], 2023-10-26T01:36:59.939 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps": "[]", 2023-10-26T01:36:59.939 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_bytes": 0, 2023-10-26T01:36:59.939 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_objects": 0, 2023-10-26T01:36:59.940 INFO:teuthology.orchestra.run.smithi169.stdout: "tiers": [], 2023-10-26T01:36:59.940 INFO:teuthology.orchestra.run.smithi169.stdout: "tier_of": -1, 2023-10-26T01:36:59.940 INFO:teuthology.orchestra.run.smithi169.stdout: "read_tier": -1, 2023-10-26T01:36:59.941 INFO:teuthology.orchestra.run.smithi169.stdout: "write_tier": -1, 2023-10-26T01:36:59.941 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_mode": "none", 2023-10-26T01:36:59.941 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_bytes": 0, 2023-10-26T01:36:59.941 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_objects": 0, 2023-10-26T01:36:59.941 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T01:36:59.942 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T01:36:59.942 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T01:36:59.942 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_flush_age": 0, 2023-10-26T01:36:59.942 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_evict_age": 0, 2023-10-26T01:36:59.943 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profile": "", 2023-10-26T01:36:59.943 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_params": { 2023-10-26T01:36:59.943 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "none" 2023-10-26T01:36:59.943 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:36:59.944 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_period": 0, 2023-10-26T01:36:59.944 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_count": 0, 2023-10-26T01:36:59.944 INFO:teuthology.orchestra.run.smithi169.stdout: "use_gmt_hitset": true, 2023-10-26T01:36:59.944 INFO:teuthology.orchestra.run.smithi169.stdout: "min_read_recency_for_promote": 0, 2023-10-26T01:36:59.944 INFO:teuthology.orchestra.run.smithi169.stdout: "min_write_recency_for_promote": 0, 2023-10-26T01:36:59.945 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T01:36:59.945 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_search_last_n": 0, 2023-10-26T01:36:59.945 INFO:teuthology.orchestra.run.smithi169.stdout: "grade_table": [], 2023-10-26T01:36:59.945 INFO:teuthology.orchestra.run.smithi169.stdout: "stripe_width": 0, 2023-10-26T01:36:59.945 INFO:teuthology.orchestra.run.smithi169.stdout: "expected_num_objects": 0, 2023-10-26T01:36:59.946 INFO:teuthology.orchestra.run.smithi169.stdout: "fast_read": false, 2023-10-26T01:36:59.946 INFO:teuthology.orchestra.run.smithi169.stdout: "options": { 2023-10-26T01:36:59.946 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_min": 64, 2023-10-26T01:36:59.946 INFO:teuthology.orchestra.run.smithi169.stdout: "target_size_ratio": 0.89999997615814209 2023-10-26T01:36:59.946 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:36:59.947 INFO:teuthology.orchestra.run.smithi169.stdout: "application_metadata": { 2023-10-26T01:36:59.947 INFO:teuthology.orchestra.run.smithi169.stdout: "cephfs": { 2023-10-26T01:36:59.947 INFO:teuthology.orchestra.run.smithi169.stdout: "data": "cephfs" 2023-10-26T01:36:59.947 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:36:59.947 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:36:59.948 INFO:teuthology.orchestra.run.smithi169.stdout: "read_balance": { 2023-10-26T01:36:59.948 INFO:teuthology.orchestra.run.smithi169.stdout: "score_acting": 1.3799999952316284, 2023-10-26T01:36:59.948 INFO:teuthology.orchestra.run.smithi169.stdout: "score_stable": 1.3799999952316284, 2023-10-26T01:36:59.948 INFO:teuthology.orchestra.run.smithi169.stdout: "optimal_score": 1, 2023-10-26T01:36:59.949 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_acting": 1.3799999952316284, 2023-10-26T01:36:59.949 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_stable": 1.3799999952316284, 2023-10-26T01:36:59.949 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity_weighted": 1, 2023-10-26T01:36:59.949 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity": 1, 2023-10-26T01:36:59.949 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity_weighted": 1 2023-10-26T01:36:59.950 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:36:59.950 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:36:59.950 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:36:59.950 INFO:teuthology.orchestra.run.smithi169.stdout: "osds": [ 2023-10-26T01:36:59.950 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:36:59.951 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 0, 2023-10-26T01:36:59.951 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:36:59.951 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:36:59.951 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:36:59.952 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:36:59.952 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:36:59.952 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:36:59.952 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:36:59.952 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:36:59.953 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 101, 2023-10-26T01:36:59.953 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:36:59.953 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:36:59.953 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:36:59.954 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:36:59.954 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:36:59.954 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:36:59.954 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6816", 2023-10-26T01:36:59.955 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:36:59.955 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:36:59.955 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:36:59.955 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:36:59.956 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6817", 2023-10-26T01:36:59.956 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:36:59.956 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:36:59.956 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:36:59.956 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:36:59.957 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:36:59.957 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:36:59.957 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:36:59.957 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:36:59.957 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6818", 2023-10-26T01:36:59.958 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:36:59.958 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:36:59.958 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:36:59.958 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:36:59.958 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6819", 2023-10-26T01:36:59.959 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:36:59.959 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:36:59.959 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:36:59.959 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:36:59.959 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:36:59.960 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:36:59.960 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:36:59.960 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:36:59.960 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6822", 2023-10-26T01:36:59.961 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:36:59.961 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:36:59.961 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:36:59.962 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:36:59.962 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6823", 2023-10-26T01:36:59.962 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:36:59.963 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:36:59.963 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:36:59.964 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:36:59.964 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:36:59.964 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:36:59.965 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:36:59.965 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:36:59.965 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6820", 2023-10-26T01:36:59.966 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:36:59.966 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:36:59.966 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:36:59.966 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:36:59.967 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6821", 2023-10-26T01:36:59.967 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:36:59.967 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:36:59.968 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:36:59.968 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:36:59.968 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6817/4204255864", 2023-10-26T01:36:59.968 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6819/4204255864", 2023-10-26T01:36:59.969 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6823/4204255864", 2023-10-26T01:36:59.969 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6821/4204255864", 2023-10-26T01:36:59.969 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:36:59.969 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:36:59.969 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:36:59.970 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:36:59.970 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:36:59.970 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:36:59.970 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 1, 2023-10-26T01:36:59.971 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:36:59.971 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:36:59.971 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:36:59.971 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:36:59.971 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:36:59.972 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:36:59.972 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:36:59.972 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:36:59.972 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 101, 2023-10-26T01:36:59.972 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:36:59.972 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:36:59.973 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:36:59.973 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:36:59.973 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:36:59.973 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:36:59.974 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6808", 2023-10-26T01:36:59.974 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:36:59.974 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:36:59.974 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:36:59.974 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:36:59.975 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6809", 2023-10-26T01:36:59.975 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:36:59.975 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:36:59.975 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:36:59.976 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:36:59.976 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:36:59.976 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:36:59.976 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:36:59.976 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:36:59.977 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6810", 2023-10-26T01:36:59.977 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:36:59.977 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:36:59.977 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:36:59.978 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:36:59.978 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6811", 2023-10-26T01:36:59.978 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:36:59.978 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:36:59.978 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:36:59.979 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:36:59.979 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:36:59.979 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:36:59.979 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:36:59.980 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:36:59.980 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6814", 2023-10-26T01:36:59.980 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:36:59.980 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:36:59.980 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:36:59.981 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:36:59.981 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6815", 2023-10-26T01:36:59.981 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:36:59.981 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:36:59.981 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:36:59.982 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:36:59.982 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:36:59.982 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:36:59.982 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:36:59.983 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:36:59.983 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6812", 2023-10-26T01:36:59.983 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:36:59.983 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:36:59.983 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:36:59.984 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:36:59.984 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6813", 2023-10-26T01:36:59.984 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:36:59.984 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:36:59.984 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:36:59.985 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:36:59.985 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6809/238206046", 2023-10-26T01:36:59.985 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6811/238206046", 2023-10-26T01:36:59.985 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6815/238206046", 2023-10-26T01:36:59.985 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6813/238206046", 2023-10-26T01:36:59.986 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:36:59.986 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:36:59.986 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:36:59.986 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:36:59.986 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:36:59.987 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:36:59.987 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 2, 2023-10-26T01:36:59.987 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:36:59.987 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:36:59.988 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:36:59.988 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:36:59.988 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:36:59.988 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:36:59.988 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:36:59.989 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:36:59.989 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 101, 2023-10-26T01:36:59.989 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:36:59.989 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:36:59.989 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:36:59.990 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:36:59.990 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:36:59.990 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:36:59.990 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6800", 2023-10-26T01:36:59.991 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:36:59.991 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:36:59.991 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:36:59.991 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:36:59.991 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6801", 2023-10-26T01:36:59.992 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:36:59.992 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:36:59.992 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:36:59.992 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:36:59.992 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:36:59.993 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:36:59.993 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:36:59.993 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:36:59.993 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6802", 2023-10-26T01:36:59.994 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:36:59.994 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:36:59.994 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:36:59.995 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:36:59.995 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6803", 2023-10-26T01:36:59.995 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:36:59.995 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:36:59.995 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:36:59.996 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:36:59.996 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:36:59.996 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:36:59.996 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:36:59.996 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:36:59.997 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6806", 2023-10-26T01:36:59.997 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:36:59.997 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:36:59.997 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:36:59.997 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:36:59.998 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6807", 2023-10-26T01:36:59.998 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:36:59.998 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:36:59.998 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:36:59.999 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:36:59.999 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:36:59.999 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:00.000 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:00.000 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:00.000 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6804", 2023-10-26T01:37:00.001 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:37:00.001 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:00.001 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:00.002 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:00.002 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6805", 2023-10-26T01:37:00.002 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:37:00.003 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:00.003 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:00.003 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:00.004 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6801/3215791846", 2023-10-26T01:37:00.004 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6803/3215791846", 2023-10-26T01:37:00.005 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6807/3215791846", 2023-10-26T01:37:00.005 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6805/3215791846", 2023-10-26T01:37:00.005 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:37:00.006 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:37:00.006 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:37:00.006 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:00.007 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:00.007 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:00.007 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 3, 2023-10-26T01:37:00.007 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:37:00.007 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:37:00.008 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:37:00.008 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:37:00.008 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:37:00.008 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:37:00.008 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:37:00.009 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 14, 2023-10-26T01:37:00.009 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 101, 2023-10-26T01:37:00.009 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:37:00.009 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:37:00.009 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:37:00.010 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:00.010 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:00.010 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:00.010 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6824", 2023-10-26T01:37:00.011 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:37:00.011 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:00.011 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:00.011 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:00.011 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6825", 2023-10-26T01:37:00.012 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:37:00.012 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:00.012 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:00.012 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:00.012 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:37:00.013 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:00.013 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:00.013 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:00.013 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6826", 2023-10-26T01:37:00.013 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:37:00.014 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:00.014 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:00.014 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:00.014 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6827", 2023-10-26T01:37:00.015 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:37:00.015 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:00.015 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:00.015 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:00.015 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:37:00.016 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:00.016 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:00.016 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:00.016 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6830", 2023-10-26T01:37:00.017 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:37:00.017 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:00.017 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:00.017 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:00.017 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6831", 2023-10-26T01:37:00.018 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:37:00.018 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:00.018 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:00.018 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:00.019 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:37:00.019 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:00.019 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:00.019 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:00.019 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6828", 2023-10-26T01:37:00.020 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:37:00.020 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:00.020 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:00.020 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:00.021 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6829", 2023-10-26T01:37:00.021 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:37:00.021 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:00.021 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:00.022 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:00.022 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6825/2256701226", 2023-10-26T01:37:00.022 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6827/2256701226", 2023-10-26T01:37:00.022 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6831/2256701226", 2023-10-26T01:37:00.023 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6829/2256701226", 2023-10-26T01:37:00.023 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:37:00.023 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:37:00.023 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:37:00.024 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:00.024 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:00.024 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:00.024 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 4, 2023-10-26T01:37:00.024 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:37:00.025 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:37:00.025 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:37:00.025 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:37:00.025 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:37:00.025 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:37:00.026 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:37:00.026 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:37:00.026 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 101, 2023-10-26T01:37:00.026 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:37:00.026 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:37:00.027 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:37:00.027 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:00.027 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:00.027 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:00.028 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6816", 2023-10-26T01:37:00.029 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:37:00.029 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:00.029 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:00.030 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:00.030 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6817", 2023-10-26T01:37:00.030 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:37:00.031 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:00.031 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:00.031 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:00.031 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:37:00.032 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:00.032 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:00.032 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:00.033 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6818", 2023-10-26T01:37:00.033 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:37:00.033 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:00.033 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:00.034 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:00.034 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6819", 2023-10-26T01:37:00.034 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:37:00.034 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:00.035 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:00.035 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:00.035 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:37:00.035 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:00.036 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:00.036 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:00.036 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6825", 2023-10-26T01:37:00.036 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:37:00.037 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:00.037 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:00.037 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:00.037 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6828", 2023-10-26T01:37:00.037 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:37:00.038 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:00.038 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:00.038 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:00.039 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:37:00.039 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:00.039 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:00.039 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:00.040 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6820", 2023-10-26T01:37:00.040 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:37:00.040 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:00.040 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:00.041 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:00.041 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6823", 2023-10-26T01:37:00.041 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:37:00.041 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:00.042 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:00.042 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:00.042 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6817/1082279874", 2023-10-26T01:37:00.043 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6819/1082279874", 2023-10-26T01:37:00.043 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6828/1082279874", 2023-10-26T01:37:00.043 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6823/1082279874", 2023-10-26T01:37:00.043 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:37:00.044 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:37:00.044 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:37:00.044 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:00.045 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:00.045 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:00.046 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 5, 2023-10-26T01:37:00.046 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:37:00.046 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:37:00.047 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:37:00.047 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:37:00.047 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:37:00.048 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:37:00.048 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:37:00.048 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:37:00.048 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 101, 2023-10-26T01:37:00.048 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:37:00.049 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:37:00.049 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:37:00.049 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:00.049 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:00.049 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:00.050 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6808", 2023-10-26T01:37:00.050 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:37:00.050 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:00.050 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:00.051 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:00.051 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6809", 2023-10-26T01:37:00.051 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:37:00.051 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:00.051 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:00.052 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:00.052 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:37:00.052 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:00.052 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:00.053 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:00.053 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6810", 2023-10-26T01:37:00.053 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:37:00.053 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:00.053 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:00.054 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:00.054 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6811", 2023-10-26T01:37:00.054 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:37:00.054 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:00.055 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:00.055 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:00.055 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:37:00.055 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:00.055 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:00.056 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:00.056 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6814", 2023-10-26T01:37:00.056 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:37:00.056 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:00.056 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:00.057 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:00.057 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6815", 2023-10-26T01:37:00.057 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:37:00.057 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:00.057 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:00.058 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:00.058 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:37:00.058 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:00.058 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:00.059 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:00.059 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6812", 2023-10-26T01:37:00.059 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:37:00.059 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:00.059 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:00.060 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:00.060 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6813", 2023-10-26T01:37:00.060 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:37:00.060 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:00.060 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:00.061 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:00.061 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6809/3335654351", 2023-10-26T01:37:00.061 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6811/3335654351", 2023-10-26T01:37:00.061 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6815/3335654351", 2023-10-26T01:37:00.062 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6813/3335654351", 2023-10-26T01:37:00.062 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:37:00.062 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:37:00.062 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:37:00.062 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:00.063 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:00.063 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:00.063 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 6, 2023-10-26T01:37:00.063 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:37:00.064 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:37:00.064 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:37:00.064 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:37:00.064 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:37:00.064 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:37:00.065 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:37:00.065 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:37:00.065 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 101, 2023-10-26T01:37:00.065 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:37:00.065 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:37:00.066 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:37:00.066 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:00.066 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:00.067 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:00.067 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6800", 2023-10-26T01:37:00.067 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:37:00.067 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:00.068 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:00.068 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:00.068 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6801", 2023-10-26T01:37:00.068 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:37:00.069 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:00.069 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:00.069 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:00.069 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:37:00.070 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:00.070 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:00.070 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:00.070 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6802", 2023-10-26T01:37:00.070 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:37:00.071 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:00.071 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:00.071 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:00.071 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6803", 2023-10-26T01:37:00.071 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:37:00.071 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:00.072 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:00.072 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:00.072 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:37:00.072 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:00.072 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:00.072 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:00.073 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6806", 2023-10-26T01:37:00.073 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:37:00.073 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:00.073 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:00.073 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:00.073 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6807", 2023-10-26T01:37:00.074 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:37:00.074 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:00.074 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:00.074 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:00.074 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:37:00.075 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:00.075 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:00.075 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:00.075 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6804", 2023-10-26T01:37:00.075 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:37:00.075 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:00.076 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:00.076 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:00.076 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6805", 2023-10-26T01:37:00.076 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:37:00.076 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:00.077 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:00.077 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:00.077 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6801/2520460697", 2023-10-26T01:37:00.077 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6803/2520460697", 2023-10-26T01:37:00.077 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6807/2520460697", 2023-10-26T01:37:00.077 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6805/2520460697", 2023-10-26T01:37:00.078 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:37:00.078 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:37:00.078 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:37:00.078 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:00.078 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:00.079 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:00.079 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 7, 2023-10-26T01:37:00.079 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:37:00.079 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:37:00.079 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:37:00.080 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:37:00.080 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:37:00.080 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:37:00.080 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:37:00.080 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:37:00.080 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 101, 2023-10-26T01:37:00.081 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:37:00.081 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:37:00.081 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:37:00.081 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:00.081 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:00.081 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:00.082 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6821", 2023-10-26T01:37:00.082 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:37:00.082 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:00.082 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:00.082 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:00.083 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6822", 2023-10-26T01:37:00.083 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:37:00.083 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:00.083 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:00.083 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:00.083 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:37:00.084 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:00.084 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:00.084 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:00.084 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6824", 2023-10-26T01:37:00.084 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:37:00.084 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:00.085 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:00.085 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:00.085 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6826", 2023-10-26T01:37:00.085 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:37:00.085 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:00.085 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:00.086 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:00.086 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:37:00.086 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:00.086 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:00.086 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:00.087 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6830", 2023-10-26T01:37:00.087 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:37:00.087 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:00.087 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:00.087 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:00.088 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6831", 2023-10-26T01:37:00.088 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:37:00.088 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:00.088 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:00.088 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:00.089 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:37:00.089 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:00.089 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:00.089 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:00.089 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6827", 2023-10-26T01:37:00.089 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:37:00.090 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:00.090 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:00.090 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:00.090 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6829", 2023-10-26T01:37:00.090 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:37:00.091 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:00.091 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:00.091 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:00.091 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6822/1115930619", 2023-10-26T01:37:00.091 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6826/1115930619", 2023-10-26T01:37:00.091 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6831/1115930619", 2023-10-26T01:37:00.092 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6829/1115930619", 2023-10-26T01:37:00.092 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:37:00.092 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:37:00.092 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:37:00.092 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:00.092 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:00.093 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:37:00.093 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_xinfo": [ 2023-10-26T01:37:00.093 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:00.093 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 0, 2023-10-26T01:37:00.093 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:37:00.093 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:37:00.094 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:37:00.094 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:37:00.094 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:37:00.094 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.889675+0000", 2023-10-26T01:37:00.094 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:37:00.095 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:00.095 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:00.095 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 1, 2023-10-26T01:37:00.095 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:37:00.095 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:37:00.095 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:37:00.096 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:37:00.096 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:37:00.096 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.864157+0000", 2023-10-26T01:37:00.096 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:37:00.096 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:00.096 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:00.097 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 2, 2023-10-26T01:37:00.097 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:37:00.097 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:37:00.097 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:37:00.097 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:37:00.098 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:37:00.098 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.876919+0000", 2023-10-26T01:37:00.098 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:37:00.098 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:00.098 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:00.099 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 3, 2023-10-26T01:37:00.099 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:37:00.099 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:37:00.099 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:37:00.099 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:37:00.099 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:37:00.100 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.974541+0000", 2023-10-26T01:37:00.100 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:37:00.100 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:00.100 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:00.100 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 4, 2023-10-26T01:37:00.100 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:37:00.101 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:37:00.101 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:37:00.101 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:37:00.101 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:37:00.101 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.969696+0000", 2023-10-26T01:37:00.101 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:37:00.101 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:00.102 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:00.102 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 5, 2023-10-26T01:37:00.102 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:37:00.102 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:37:00.102 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:37:00.103 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:37:00.103 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:37:00.103 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.859703+0000", 2023-10-26T01:37:00.103 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:37:00.103 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:00.103 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:00.104 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 6, 2023-10-26T01:37:00.104 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:37:00.104 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:37:00.104 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:37:00.104 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:37:00.105 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:37:00.105 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.867917+0000", 2023-10-26T01:37:00.105 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:37:00.105 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:00.105 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:00.105 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 7, 2023-10-26T01:37:00.106 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:37:00.106 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:37:00.106 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:37:00.106 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:37:00.106 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:37:00.106 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.898605+0000", 2023-10-26T01:37:00.107 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:37:00.107 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:00.107 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:37:00.107 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_upmap": [], 2023-10-26T01:37:00.107 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_upmap_items": [], 2023-10-26T01:37:00.108 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_upmap_primaries": [], 2023-10-26T01:37:00.108 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_temp": [], 2023-10-26T01:37:00.108 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_temp": [], 2023-10-26T01:37:00.108 INFO:teuthology.orchestra.run.smithi169.stdout: "blocklist": {}, 2023-10-26T01:37:00.108 INFO:teuthology.orchestra.run.smithi169.stdout: "range_blocklist": {}, 2023-10-26T01:37:00.109 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profiles": { 2023-10-26T01:37:00.109 INFO:teuthology.orchestra.run.smithi169.stdout: "default": { 2023-10-26T01:37:00.109 INFO:teuthology.orchestra.run.smithi169.stdout: "crush-failure-domain": "osd", 2023-10-26T01:37:00.109 INFO:teuthology.orchestra.run.smithi169.stdout: "k": "2", 2023-10-26T01:37:00.109 INFO:teuthology.orchestra.run.smithi169.stdout: "m": "1", 2023-10-26T01:37:00.109 INFO:teuthology.orchestra.run.smithi169.stdout: "plugin": "jerasure", 2023-10-26T01:37:00.110 INFO:teuthology.orchestra.run.smithi169.stdout: "technique": "reed_sol_van" 2023-10-26T01:37:00.110 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:00.110 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:00.110 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps_queue": [], 2023-10-26T01:37:00.110 INFO:teuthology.orchestra.run.smithi169.stdout: "new_removed_snaps": [], 2023-10-26T01:37:00.111 INFO:teuthology.orchestra.run.smithi169.stdout: "new_purged_snaps": [], 2023-10-26T01:37:00.111 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_node_flags": {}, 2023-10-26T01:37:00.111 INFO:teuthology.orchestra.run.smithi169.stdout: "device_class_flags": {}, 2023-10-26T01:37:00.111 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_mode": { 2023-10-26T01:37:00.111 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_mode_enabled": false, 2023-10-26T01:37:00.111 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_bucket_count": 0, 2023-10-26T01:37:00.112 INFO:teuthology.orchestra.run.smithi169.stdout: "degraded_stretch_mode": 0, 2023-10-26T01:37:00.112 INFO:teuthology.orchestra.run.smithi169.stdout: "recovering_stretch_mode": 0, 2023-10-26T01:37:00.112 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_mode_bucket": 0 2023-10-26T01:37:00.112 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:00.112 INFO:teuthology.orchestra.run.smithi169.stdout:} 2023-10-26T01:37:00.114 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:59.879+0000 7fa3db6a9640 1 -- 172.21.15.169:0/4142503073 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa3b806ae40 msgr2=0x7fa3b806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:00.114 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:59.879+0000 7fa3db6a9640 1 --2- 172.21.15.169:0/4142503073 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa3b806ae40 0x7fa3b806d300 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fa3d000a860 tx=0x7fa3d0005b50 comp rx=0 tx=0).stop 2023-10-26T01:37:00.114 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:59.879+0000 7fa3db6a9640 1 -- 172.21.15.169:0/4142503073 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa3d40cde00 msgr2=0x7fa3d40c66f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:00.114 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:59.879+0000 7fa3db6a9640 1 --2- 172.21.15.169:0/4142503073 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa3d40cde00 0x7fa3d40c66f0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fa3c800c450 tx=0x7fa3c800c910 comp rx=0 tx=0).stop 2023-10-26T01:37:00.115 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:59.879+0000 7fa3db6a9640 1 -- 172.21.15.169:0/4142503073 shutdown_connections 2023-10-26T01:37:00.115 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:59.879+0000 7fa3db6a9640 1 --2- 172.21.15.169:0/4142503073 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa3b806ae40 0x7fa3b806d300 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:00.115 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:59.879+0000 7fa3db6a9640 1 --2- 172.21.15.169:0/4142503073 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa3d40cde00 0x7fa3d40c66f0 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:00.115 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:59.879+0000 7fa3db6a9640 1 --2- 172.21.15.169:0/4142503073 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa3d41592d0 0x7fa3d41f0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:00.115 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:59.879+0000 7fa3db6a9640 1 --2- 172.21.15.169:0/4142503073 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa3d40ac930 0x7fa3d41f5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:00.116 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:59.879+0000 7fa3db6a9640 1 -- 172.21.15.169:0/4142503073 >> 172.21.15.169:0/4142503073 conn(0x7fa3d4069930 msgr2=0x7fa3d41567a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:00.116 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:59.879+0000 7fa3db6a9640 1 -- 172.21.15.169:0/4142503073 shutdown_connections 2023-10-26T01:37:00.116 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:36:59.880+0000 7fa3db6a9640 1 -- 172.21.15.169:0/4142503073 wait complete. 2023-10-26T01:37:00.117 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2023-10-26T01:37:02.114 INFO:tasks.ceph.mon.a.smithi169.stderr:2023-10-26T01:37:02.113+0000 7f158b279640 -1 log_channel(cluster) log [ERR] : Health check failed: 5 full osd(s) (OSD_FULL) 2023-10-26T01:37:05.118 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-26T01:37:05.339 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:05.336+0000 7f306fde2640 1 -- 172.21.15.169:0/1283129501 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30680cde30 msgr2=0x7f3068162270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:05.340 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:05.336+0000 7f306fde2640 1 --2- 172.21.15.169:0/1283129501 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30680cde30 0x7f3068162270 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f3060005fd0 tx=0x7f306002d410 comp rx=0 tx=0).stop 2023-10-26T01:37:05.340 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:05.337+0000 7f306fde2640 1 -- 172.21.15.169:0/1283129501 shutdown_connections 2023-10-26T01:37:05.340 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:05.337+0000 7f306fde2640 1 --2- 172.21.15.169:0/1283129501 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30680cde30 0x7f3068162270 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:05.340 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:05.337+0000 7f306fde2640 1 -- 172.21.15.169:0/1283129501 >> 172.21.15.169:0/1283129501 conn(0x7f3068069930 msgr2=0x7f3068069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:05.341 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:05.337+0000 7f306fde2640 1 -- 172.21.15.169:0/1283129501 shutdown_connections 2023-10-26T01:37:05.341 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:05.337+0000 7f306fde2640 1 -- 172.21.15.169:0/1283129501 wait complete. 2023-10-26T01:37:05.342 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:05.338+0000 7f306fde2640 1 Processor -- start 2023-10-26T01:37:05.342 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:05.338+0000 7f306fde2640 1 -- start start 2023-10-26T01:37:05.343 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:05.339+0000 7f306fde2640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30680cde30 0x7f3068142660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:05.343 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:05.339+0000 7f306fde2640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f306814e790 0x7f3068141b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:05.343 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:05.339+0000 7f306fde2640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3068158850 0x7f306814c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:05.343 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:05.339+0000 7f306fde2640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3068142050 con 0x7f30680cde30 2023-10-26T01:37:05.343 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:05.339+0000 7f306fde2640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3068142e80 con 0x7f3068158850 2023-10-26T01:37:05.344 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:05.339+0000 7f306fde2640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30681430d0 con 0x7f306814e790 2023-10-26T01:37:05.344 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:05.340+0000 7f306db7e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30680cde30 0x7f3068142660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:05.344 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:05.340+0000 7f306db7e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30680cde30 0x7f3068142660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:59908/0 (socket says 172.21.15.169:59908) 2023-10-26T01:37:05.344 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:05.340+0000 7f306db7e640 1 -- 172.21.15.169:0/521904249 learned_addr learned my addr 172.21.15.169:0/521904249 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:37:05.345 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:05.340+0000 7f306db7e640 1 -- 172.21.15.169:0/521904249 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f306814e790 msgr2=0x7f3068141b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:05.346 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:05.340+0000 7f306e37f640 1 --2- 172.21.15.169:0/521904249 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3068158850 0x7f306814c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:05.346 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:05.340+0000 7f306db7e640 1 --2- 172.21.15.169:0/521904249 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f306814e790 0x7f3068141b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:05.346 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:05.340+0000 7f306db7e640 1 -- 172.21.15.169:0/521904249 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3068158850 msgr2=0x7f306814c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:05.346 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:05.340+0000 7f306db7e640 1 --2- 172.21.15.169:0/521904249 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3068158850 0x7f306814c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:05.347 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:05.340+0000 7f306db7e640 1 -- 172.21.15.169:0/521904249 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f306000a950 con 0x7f30680cde30 2023-10-26T01:37:05.347 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:05.340+0000 7f306db7e640 1 --2- 172.21.15.169:0/521904249 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30680cde30 0x7f3068142660 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f306400c430 tx=0x7f306400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:05.347 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:05.341+0000 7f305e7fc640 1 -- 172.21.15.169:0/521904249 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3064011790 con 0x7f30680cde30 2023-10-26T01:37:05.347 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:05.341+0000 7f305e7fc640 1 -- 172.21.15.169:0/521904249 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f30640119e0 con 0x7f30680cde30 2023-10-26T01:37:05.347 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:05.341+0000 7f306fde2640 1 -- 172.21.15.169:0/521904249 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30681433b0 con 0x7f30680cde30 2023-10-26T01:37:05.347 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:05.341+0000 7f305e7fc640 1 -- 172.21.15.169:0/521904249 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3064011e30 con 0x7f30680cde30 2023-10-26T01:37:05.348 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:05.341+0000 7f306fde2640 1 -- 172.21.15.169:0/521904249 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3068143970 con 0x7f30680cde30 2023-10-26T01:37:05.348 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:05.342+0000 7f306fde2640 1 -- 172.21.15.169:0/521904249 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f306804db10 con 0x7f30680cde30 2023-10-26T01:37:05.348 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:05.343+0000 7f305e7fc640 1 -- 172.21.15.169:0/521904249 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f306401c460 con 0x7f30680cde30 2023-10-26T01:37:05.349 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:05.344+0000 7f305e7fc640 1 --2- 172.21.15.169:0/521904249 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f304c06ae40 0x7f304c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:05.349 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:05.344+0000 7f305e7fc640 1 -- 172.21.15.169:0/521904249 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3064089900 con 0x7f30680cde30 2023-10-26T01:37:05.349 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:05.344+0000 7f305e7fc640 1 -- 172.21.15.169:0/521904249 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=106}) v3 -- 0x7f304c072970 con 0x7f30680cde30 2023-10-26T01:37:05.350 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:05.345+0000 7f306d37d640 1 --2- 172.21.15.169:0/521904249 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f304c06ae40 0x7f304c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:05.350 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:05.346+0000 7f306d37d640 1 --2- 172.21.15.169:0/521904249 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f304c06ae40 0x7f304c06d300 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f3058007fd0 tx=0x7f30580078e0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:05.351 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:05.349+0000 7f305e7fc640 1 -- 172.21.15.169:0/521904249 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3064054910 con 0x7f30680cde30 2023-10-26T01:37:05.541 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:05.538+0000 7f306fde2640 1 -- 172.21.15.169:0/521904249 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f30680aca70 con 0x7f30680cde30 2023-10-26T01:37:05.543 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:05.540+0000 7f305e7fc640 1 -- 172.21.15.169:0/521904249 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v105) v1 ==== 81+0+34222 (secure 0 0 0) 0x7f3064057f30 con 0x7f30680cde30 2023-10-26T01:37:05.544 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:37:05.544 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2023-10-26T01:37:05.545 INFO:teuthology.orchestra.run.smithi169.stdout: "epoch": 105, 2023-10-26T01:37:05.545 INFO:teuthology.orchestra.run.smithi169.stdout: "fsid": "30203b28-ed8a-44e8-a9a6-a9bf25f980f1", 2023-10-26T01:37:05.546 INFO:teuthology.orchestra.run.smithi169.stdout: "created": "2023-10-26T01:30:22.897155+0000", 2023-10-26T01:37:05.546 INFO:teuthology.orchestra.run.smithi169.stdout: "modified": "2023-10-26T01:37:02.113248+0000", 2023-10-26T01:37:05.546 INFO:teuthology.orchestra.run.smithi169.stdout: "last_up_change": "2023-10-26T01:30:31.875281+0000", 2023-10-26T01:37:05.547 INFO:teuthology.orchestra.run.smithi169.stdout: "last_in_change": "2023-10-26T01:30:27.637941+0000", 2023-10-26T01:37:05.547 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-26T01:37:05.547 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_num": 5799936, 2023-10-26T01:37:05.548 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_set": [ 2023-10-26T01:37:05.548 INFO:teuthology.orchestra.run.smithi169.stdout: "pglog_hardlimit", 2023-10-26T01:37:05.549 INFO:teuthology.orchestra.run.smithi169.stdout: "purged_snapdirs", 2023-10-26T01:37:05.549 INFO:teuthology.orchestra.run.smithi169.stdout: "recovery_deletes", 2023-10-26T01:37:05.550 INFO:teuthology.orchestra.run.smithi169.stdout: "sortbitwise" 2023-10-26T01:37:05.550 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:37:05.550 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_version": 4, 2023-10-26T01:37:05.551 INFO:teuthology.orchestra.run.smithi169.stdout: "full_ratio": 0.69999998807907104, 2023-10-26T01:37:05.551 INFO:teuthology.orchestra.run.smithi169.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-26T01:37:05.552 INFO:teuthology.orchestra.run.smithi169.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-26T01:37:05.552 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_snapshot": "", 2023-10-26T01:37:05.552 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_max": 13, 2023-10-26T01:37:05.553 INFO:teuthology.orchestra.run.smithi169.stdout: "max_osd": 8, 2023-10-26T01:37:05.553 INFO:teuthology.orchestra.run.smithi169.stdout: "require_min_compat_client": "luminous", 2023-10-26T01:37:05.554 INFO:teuthology.orchestra.run.smithi169.stdout: "min_compat_client": "jewel", 2023-10-26T01:37:05.554 INFO:teuthology.orchestra.run.smithi169.stdout: "require_osd_release": "reef", 2023-10-26T01:37:05.554 INFO:teuthology.orchestra.run.smithi169.stdout: "allow_crimson": false, 2023-10-26T01:37:05.555 INFO:teuthology.orchestra.run.smithi169.stdout: "pools": [ 2023-10-26T01:37:05.555 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:05.556 INFO:teuthology.orchestra.run.smithi169.stdout: "pool": 1, 2023-10-26T01:37:05.556 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_name": ".mgr", 2023-10-26T01:37:05.556 INFO:teuthology.orchestra.run.smithi169.stdout: "create_time": "2023-10-26T01:30:32.200808+0000", 2023-10-26T01:37:05.557 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": 3, 2023-10-26T01:37:05.557 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_names": "hashpspool,full", 2023-10-26T01:37:05.557 INFO:teuthology.orchestra.run.smithi169.stdout: "type": 1, 2023-10-26T01:37:05.558 INFO:teuthology.orchestra.run.smithi169.stdout: "size": 2, 2023-10-26T01:37:05.558 INFO:teuthology.orchestra.run.smithi169.stdout: "min_size": 1, 2023-10-26T01:37:05.559 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_rule": 0, 2023-10-26T01:37:05.559 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_count": 0, 2023-10-26T01:37:05.559 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_target": 0, 2023-10-26T01:37:05.560 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T01:37:05.560 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T01:37:05.560 INFO:teuthology.orchestra.run.smithi169.stdout: "object_hash": 2, 2023-10-26T01:37:05.561 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_mode": "off", 2023-10-26T01:37:05.561 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num": 1, 2023-10-26T01:37:05.561 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num": 1, 2023-10-26T01:37:05.561 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num_target": 1, 2023-10-26T01:37:05.562 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_target": 1, 2023-10-26T01:37:05.562 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_pending": 1, 2023-10-26T01:37:05.563 INFO:teuthology.orchestra.run.smithi169.stdout: "last_pg_merge_meta": { 2023-10-26T01:37:05.563 INFO:teuthology.orchestra.run.smithi169.stdout: "source_pgid": "0.0", 2023-10-26T01:37:05.563 INFO:teuthology.orchestra.run.smithi169.stdout: "ready_epoch": 0, 2023-10-26T01:37:05.564 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_started": 0, 2023-10-26T01:37:05.564 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_clean": 0, 2023-10-26T01:37:05.564 INFO:teuthology.orchestra.run.smithi169.stdout: "source_version": "0'0", 2023-10-26T01:37:05.564 INFO:teuthology.orchestra.run.smithi169.stdout: "target_version": "0'0" 2023-10-26T01:37:05.565 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:05.565 INFO:teuthology.orchestra.run.smithi169.stdout: "last_change": "105", 2023-10-26T01:37:05.565 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend": "0", 2023-10-26T01:37:05.566 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T01:37:05.566 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T01:37:05.566 INFO:teuthology.orchestra.run.smithi169.stdout: "auid": 0, 2023-10-26T01:37:05.567 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_mode": "selfmanaged", 2023-10-26T01:37:05.567 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_seq": 0, 2023-10-26T01:37:05.567 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_epoch": 0, 2023-10-26T01:37:05.568 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_snaps": [], 2023-10-26T01:37:05.568 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps": "[]", 2023-10-26T01:37:05.568 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_bytes": 0, 2023-10-26T01:37:05.569 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_objects": 0, 2023-10-26T01:37:05.569 INFO:teuthology.orchestra.run.smithi169.stdout: "tiers": [], 2023-10-26T01:37:05.569 INFO:teuthology.orchestra.run.smithi169.stdout: "tier_of": -1, 2023-10-26T01:37:05.570 INFO:teuthology.orchestra.run.smithi169.stdout: "read_tier": -1, 2023-10-26T01:37:05.570 INFO:teuthology.orchestra.run.smithi169.stdout: "write_tier": -1, 2023-10-26T01:37:05.570 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_mode": "none", 2023-10-26T01:37:05.570 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_bytes": 0, 2023-10-26T01:37:05.571 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_objects": 0, 2023-10-26T01:37:05.571 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T01:37:05.571 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T01:37:05.572 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T01:37:05.572 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_flush_age": 0, 2023-10-26T01:37:05.572 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_evict_age": 0, 2023-10-26T01:37:05.572 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profile": "", 2023-10-26T01:37:05.573 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_params": { 2023-10-26T01:37:05.573 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "none" 2023-10-26T01:37:05.573 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:05.574 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_period": 0, 2023-10-26T01:37:05.574 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_count": 0, 2023-10-26T01:37:05.574 INFO:teuthology.orchestra.run.smithi169.stdout: "use_gmt_hitset": true, 2023-10-26T01:37:05.575 INFO:teuthology.orchestra.run.smithi169.stdout: "min_read_recency_for_promote": 0, 2023-10-26T01:37:05.575 INFO:teuthology.orchestra.run.smithi169.stdout: "min_write_recency_for_promote": 0, 2023-10-26T01:37:05.575 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T01:37:05.575 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_search_last_n": 0, 2023-10-26T01:37:05.575 INFO:teuthology.orchestra.run.smithi169.stdout: "grade_table": [], 2023-10-26T01:37:05.576 INFO:teuthology.orchestra.run.smithi169.stdout: "stripe_width": 0, 2023-10-26T01:37:05.576 INFO:teuthology.orchestra.run.smithi169.stdout: "expected_num_objects": 0, 2023-10-26T01:37:05.576 INFO:teuthology.orchestra.run.smithi169.stdout: "fast_read": false, 2023-10-26T01:37:05.576 INFO:teuthology.orchestra.run.smithi169.stdout: "options": { 2023-10-26T01:37:05.576 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_max": 32, 2023-10-26T01:37:05.577 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_min": 1 2023-10-26T01:37:05.577 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:05.577 INFO:teuthology.orchestra.run.smithi169.stdout: "application_metadata": { 2023-10-26T01:37:05.577 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": {} 2023-10-26T01:37:05.577 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:05.578 INFO:teuthology.orchestra.run.smithi169.stdout: "read_balance": { 2023-10-26T01:37:05.578 INFO:teuthology.orchestra.run.smithi169.stdout: "score_acting": 8, 2023-10-26T01:37:05.578 INFO:teuthology.orchestra.run.smithi169.stdout: "score_stable": 8, 2023-10-26T01:37:05.578 INFO:teuthology.orchestra.run.smithi169.stdout: "optimal_score": 0.25, 2023-10-26T01:37:05.578 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_acting": 2, 2023-10-26T01:37:05.579 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_stable": 2, 2023-10-26T01:37:05.579 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity_weighted": 1, 2023-10-26T01:37:05.579 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity": 1, 2023-10-26T01:37:05.579 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity_weighted": 1 2023-10-26T01:37:05.579 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:05.580 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:05.580 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:05.580 INFO:teuthology.orchestra.run.smithi169.stdout: "pool": 12, 2023-10-26T01:37:05.580 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_name": "cephfs_metadata", 2023-10-26T01:37:05.580 INFO:teuthology.orchestra.run.smithi169.stdout: "create_time": "2023-10-26T01:36:41.430455+0000", 2023-10-26T01:37:05.580 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": 3, 2023-10-26T01:37:05.581 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_names": "hashpspool,full", 2023-10-26T01:37:05.581 INFO:teuthology.orchestra.run.smithi169.stdout: "type": 1, 2023-10-26T01:37:05.581 INFO:teuthology.orchestra.run.smithi169.stdout: "size": 2, 2023-10-26T01:37:05.581 INFO:teuthology.orchestra.run.smithi169.stdout: "min_size": 1, 2023-10-26T01:37:05.581 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_rule": 0, 2023-10-26T01:37:05.582 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_count": 0, 2023-10-26T01:37:05.582 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_target": 0, 2023-10-26T01:37:05.582 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T01:37:05.582 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T01:37:05.582 INFO:teuthology.orchestra.run.smithi169.stdout: "object_hash": 2, 2023-10-26T01:37:05.583 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_mode": "off", 2023-10-26T01:37:05.583 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num": 32, 2023-10-26T01:37:05.583 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num": 32, 2023-10-26T01:37:05.583 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num_target": 32, 2023-10-26T01:37:05.583 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_target": 32, 2023-10-26T01:37:05.583 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_pending": 32, 2023-10-26T01:37:05.584 INFO:teuthology.orchestra.run.smithi169.stdout: "last_pg_merge_meta": { 2023-10-26T01:37:05.584 INFO:teuthology.orchestra.run.smithi169.stdout: "source_pgid": "0.0", 2023-10-26T01:37:05.584 INFO:teuthology.orchestra.run.smithi169.stdout: "ready_epoch": 0, 2023-10-26T01:37:05.584 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_started": 0, 2023-10-26T01:37:05.584 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_clean": 0, 2023-10-26T01:37:05.585 INFO:teuthology.orchestra.run.smithi169.stdout: "source_version": "0'0", 2023-10-26T01:37:05.585 INFO:teuthology.orchestra.run.smithi169.stdout: "target_version": "0'0" 2023-10-26T01:37:05.585 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:05.585 INFO:teuthology.orchestra.run.smithi169.stdout: "last_change": "105", 2023-10-26T01:37:05.585 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend": "0", 2023-10-26T01:37:05.586 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T01:37:05.586 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T01:37:05.586 INFO:teuthology.orchestra.run.smithi169.stdout: "auid": 0, 2023-10-26T01:37:05.586 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_mode": "selfmanaged", 2023-10-26T01:37:05.586 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_seq": 0, 2023-10-26T01:37:05.587 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_epoch": 0, 2023-10-26T01:37:05.587 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_snaps": [], 2023-10-26T01:37:05.587 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps": "[]", 2023-10-26T01:37:05.587 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_bytes": 0, 2023-10-26T01:37:05.587 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_objects": 0, 2023-10-26T01:37:05.587 INFO:teuthology.orchestra.run.smithi169.stdout: "tiers": [], 2023-10-26T01:37:05.588 INFO:teuthology.orchestra.run.smithi169.stdout: "tier_of": -1, 2023-10-26T01:37:05.588 INFO:teuthology.orchestra.run.smithi169.stdout: "read_tier": -1, 2023-10-26T01:37:05.588 INFO:teuthology.orchestra.run.smithi169.stdout: "write_tier": -1, 2023-10-26T01:37:05.588 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_mode": "none", 2023-10-26T01:37:05.588 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_bytes": 0, 2023-10-26T01:37:05.588 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_objects": 0, 2023-10-26T01:37:05.589 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T01:37:05.589 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T01:37:05.589 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T01:37:05.589 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_flush_age": 0, 2023-10-26T01:37:05.589 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_evict_age": 0, 2023-10-26T01:37:05.589 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profile": "", 2023-10-26T01:37:05.590 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_params": { 2023-10-26T01:37:05.590 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "none" 2023-10-26T01:37:05.590 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:05.590 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_period": 0, 2023-10-26T01:37:05.590 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_count": 0, 2023-10-26T01:37:05.591 INFO:teuthology.orchestra.run.smithi169.stdout: "use_gmt_hitset": true, 2023-10-26T01:37:05.591 INFO:teuthology.orchestra.run.smithi169.stdout: "min_read_recency_for_promote": 0, 2023-10-26T01:37:05.591 INFO:teuthology.orchestra.run.smithi169.stdout: "min_write_recency_for_promote": 0, 2023-10-26T01:37:05.591 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T01:37:05.591 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_search_last_n": 0, 2023-10-26T01:37:05.591 INFO:teuthology.orchestra.run.smithi169.stdout: "grade_table": [], 2023-10-26T01:37:05.591 INFO:teuthology.orchestra.run.smithi169.stdout: "stripe_width": 0, 2023-10-26T01:37:05.592 INFO:teuthology.orchestra.run.smithi169.stdout: "expected_num_objects": 0, 2023-10-26T01:37:05.592 INFO:teuthology.orchestra.run.smithi169.stdout: "fast_read": false, 2023-10-26T01:37:05.592 INFO:teuthology.orchestra.run.smithi169.stdout: "options": { 2023-10-26T01:37:05.592 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_bias": 4, 2023-10-26T01:37:05.592 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_min": 16, 2023-10-26T01:37:05.593 INFO:teuthology.orchestra.run.smithi169.stdout: "recovery_priority": 5 2023-10-26T01:37:05.593 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:05.593 INFO:teuthology.orchestra.run.smithi169.stdout: "application_metadata": { 2023-10-26T01:37:05.593 INFO:teuthology.orchestra.run.smithi169.stdout: "cephfs": { 2023-10-26T01:37:05.593 INFO:teuthology.orchestra.run.smithi169.stdout: "metadata": "cephfs" 2023-10-26T01:37:05.593 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:05.594 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:05.594 INFO:teuthology.orchestra.run.smithi169.stdout: "read_balance": { 2023-10-26T01:37:05.594 INFO:teuthology.orchestra.run.smithi169.stdout: "score_acting": 2, 2023-10-26T01:37:05.594 INFO:teuthology.orchestra.run.smithi169.stdout: "score_stable": 2, 2023-10-26T01:37:05.594 INFO:teuthology.orchestra.run.smithi169.stdout: "optimal_score": 1, 2023-10-26T01:37:05.595 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_acting": 2, 2023-10-26T01:37:05.595 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_stable": 2, 2023-10-26T01:37:05.595 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity_weighted": 1, 2023-10-26T01:37:05.595 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity": 1, 2023-10-26T01:37:05.595 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity_weighted": 1 2023-10-26T01:37:05.595 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:05.596 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:05.596 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:05.596 INFO:teuthology.orchestra.run.smithi169.stdout: "pool": 13, 2023-10-26T01:37:05.596 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_name": "cephfs_data", 2023-10-26T01:37:05.596 INFO:teuthology.orchestra.run.smithi169.stdout: "create_time": "2023-10-26T01:36:42.883252+0000", 2023-10-26T01:37:05.596 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": 3, 2023-10-26T01:37:05.597 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_names": "hashpspool,full", 2023-10-26T01:37:05.597 INFO:teuthology.orchestra.run.smithi169.stdout: "type": 1, 2023-10-26T01:37:05.597 INFO:teuthology.orchestra.run.smithi169.stdout: "size": 2, 2023-10-26T01:37:05.597 INFO:teuthology.orchestra.run.smithi169.stdout: "min_size": 1, 2023-10-26T01:37:05.597 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_rule": 0, 2023-10-26T01:37:05.597 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_count": 0, 2023-10-26T01:37:05.598 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_target": 0, 2023-10-26T01:37:05.598 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T01:37:05.598 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T01:37:05.598 INFO:teuthology.orchestra.run.smithi169.stdout: "object_hash": 2, 2023-10-26T01:37:05.598 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_mode": "off", 2023-10-26T01:37:05.599 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num": 64, 2023-10-26T01:37:05.599 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num": 64, 2023-10-26T01:37:05.599 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num_target": 64, 2023-10-26T01:37:05.599 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_target": 64, 2023-10-26T01:37:05.599 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_pending": 64, 2023-10-26T01:37:05.600 INFO:teuthology.orchestra.run.smithi169.stdout: "last_pg_merge_meta": { 2023-10-26T01:37:05.600 INFO:teuthology.orchestra.run.smithi169.stdout: "source_pgid": "0.0", 2023-10-26T01:37:05.600 INFO:teuthology.orchestra.run.smithi169.stdout: "ready_epoch": 0, 2023-10-26T01:37:05.600 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_started": 0, 2023-10-26T01:37:05.601 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_clean": 0, 2023-10-26T01:37:05.601 INFO:teuthology.orchestra.run.smithi169.stdout: "source_version": "0'0", 2023-10-26T01:37:05.601 INFO:teuthology.orchestra.run.smithi169.stdout: "target_version": "0'0" 2023-10-26T01:37:05.602 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:05.602 INFO:teuthology.orchestra.run.smithi169.stdout: "last_change": "105", 2023-10-26T01:37:05.602 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend": "0", 2023-10-26T01:37:05.603 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T01:37:05.603 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T01:37:05.603 INFO:teuthology.orchestra.run.smithi169.stdout: "auid": 0, 2023-10-26T01:37:05.604 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_mode": "selfmanaged", 2023-10-26T01:37:05.604 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_seq": 0, 2023-10-26T01:37:05.604 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_epoch": 0, 2023-10-26T01:37:05.604 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_snaps": [], 2023-10-26T01:37:05.605 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps": "[]", 2023-10-26T01:37:05.605 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_bytes": 0, 2023-10-26T01:37:05.605 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_objects": 0, 2023-10-26T01:37:05.606 INFO:teuthology.orchestra.run.smithi169.stdout: "tiers": [], 2023-10-26T01:37:05.606 INFO:teuthology.orchestra.run.smithi169.stdout: "tier_of": -1, 2023-10-26T01:37:05.606 INFO:teuthology.orchestra.run.smithi169.stdout: "read_tier": -1, 2023-10-26T01:37:05.607 INFO:teuthology.orchestra.run.smithi169.stdout: "write_tier": -1, 2023-10-26T01:37:05.607 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_mode": "none", 2023-10-26T01:37:05.607 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_bytes": 0, 2023-10-26T01:37:05.607 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_objects": 0, 2023-10-26T01:37:05.608 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T01:37:05.608 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T01:37:05.608 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T01:37:05.608 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_flush_age": 0, 2023-10-26T01:37:05.609 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_evict_age": 0, 2023-10-26T01:37:05.609 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profile": "", 2023-10-26T01:37:05.609 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_params": { 2023-10-26T01:37:05.609 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "none" 2023-10-26T01:37:05.609 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:05.609 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_period": 0, 2023-10-26T01:37:05.610 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_count": 0, 2023-10-26T01:37:05.610 INFO:teuthology.orchestra.run.smithi169.stdout: "use_gmt_hitset": true, 2023-10-26T01:37:05.610 INFO:teuthology.orchestra.run.smithi169.stdout: "min_read_recency_for_promote": 0, 2023-10-26T01:37:05.610 INFO:teuthology.orchestra.run.smithi169.stdout: "min_write_recency_for_promote": 0, 2023-10-26T01:37:05.611 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T01:37:05.611 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_search_last_n": 0, 2023-10-26T01:37:05.611 INFO:teuthology.orchestra.run.smithi169.stdout: "grade_table": [], 2023-10-26T01:37:05.611 INFO:teuthology.orchestra.run.smithi169.stdout: "stripe_width": 0, 2023-10-26T01:37:05.612 INFO:teuthology.orchestra.run.smithi169.stdout: "expected_num_objects": 0, 2023-10-26T01:37:05.612 INFO:teuthology.orchestra.run.smithi169.stdout: "fast_read": false, 2023-10-26T01:37:05.612 INFO:teuthology.orchestra.run.smithi169.stdout: "options": { 2023-10-26T01:37:05.612 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_min": 64, 2023-10-26T01:37:05.612 INFO:teuthology.orchestra.run.smithi169.stdout: "target_size_ratio": 0.89999997615814209 2023-10-26T01:37:05.613 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:05.613 INFO:teuthology.orchestra.run.smithi169.stdout: "application_metadata": { 2023-10-26T01:37:05.613 INFO:teuthology.orchestra.run.smithi169.stdout: "cephfs": { 2023-10-26T01:37:05.613 INFO:teuthology.orchestra.run.smithi169.stdout: "data": "cephfs" 2023-10-26T01:37:05.613 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:05.614 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:05.614 INFO:teuthology.orchestra.run.smithi169.stdout: "read_balance": { 2023-10-26T01:37:05.614 INFO:teuthology.orchestra.run.smithi169.stdout: "score_acting": 1.3799999952316284, 2023-10-26T01:37:05.614 INFO:teuthology.orchestra.run.smithi169.stdout: "score_stable": 1.3799999952316284, 2023-10-26T01:37:05.614 INFO:teuthology.orchestra.run.smithi169.stdout: "optimal_score": 1, 2023-10-26T01:37:05.615 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_acting": 1.3799999952316284, 2023-10-26T01:37:05.615 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_stable": 1.3799999952316284, 2023-10-26T01:37:05.615 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity_weighted": 1, 2023-10-26T01:37:05.615 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity": 1, 2023-10-26T01:37:05.615 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity_weighted": 1 2023-10-26T01:37:05.616 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:05.616 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:05.616 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:37:05.616 INFO:teuthology.orchestra.run.smithi169.stdout: "osds": [ 2023-10-26T01:37:05.616 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:05.616 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 0, 2023-10-26T01:37:05.617 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:37:05.617 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:37:05.617 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:37:05.617 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:37:05.617 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:37:05.618 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:37:05.618 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:37:05.618 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:37:05.618 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 101, 2023-10-26T01:37:05.618 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:37:05.619 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:37:05.619 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:37:05.619 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:05.619 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:05.619 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:05.619 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6816", 2023-10-26T01:37:05.620 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:37:05.620 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:05.620 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:05.620 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:05.620 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6817", 2023-10-26T01:37:05.621 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:37:05.621 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:05.621 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:05.621 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:05.621 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:37:05.621 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:05.622 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:05.622 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:05.622 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6818", 2023-10-26T01:37:05.622 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:37:05.623 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:05.623 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:05.623 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:05.623 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6819", 2023-10-26T01:37:05.623 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:37:05.624 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:05.624 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:05.624 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:05.624 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:37:05.624 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:05.624 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:05.625 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:05.625 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6822", 2023-10-26T01:37:05.625 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:37:05.625 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:05.625 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:05.625 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:05.626 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6823", 2023-10-26T01:37:05.626 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:37:05.626 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:05.626 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:05.626 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:05.627 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:37:05.627 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:05.627 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:05.627 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:05.627 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6820", 2023-10-26T01:37:05.627 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:37:05.628 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:05.628 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:05.628 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:05.628 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6821", 2023-10-26T01:37:05.628 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:37:05.628 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:05.629 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:05.629 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:05.629 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6817/4204255864", 2023-10-26T01:37:05.629 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6819/4204255864", 2023-10-26T01:37:05.629 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6823/4204255864", 2023-10-26T01:37:05.629 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6821/4204255864", 2023-10-26T01:37:05.629 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:37:05.630 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:37:05.630 INFO:teuthology.orchestra.run.smithi169.stdout: "full", 2023-10-26T01:37:05.630 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:37:05.630 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:05.630 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:05.631 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:05.631 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 1, 2023-10-26T01:37:05.631 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:37:05.631 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:37:05.631 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:37:05.631 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:37:05.632 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:37:05.632 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:37:05.632 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:37:05.632 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:37:05.632 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 101, 2023-10-26T01:37:05.632 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:37:05.633 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:37:05.633 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:37:05.633 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:05.633 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:05.633 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:05.634 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6808", 2023-10-26T01:37:05.634 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:37:05.634 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:05.634 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:05.634 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:05.634 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6809", 2023-10-26T01:37:05.635 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:37:05.635 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:05.635 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:05.635 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:05.635 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:37:05.635 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:05.636 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:05.636 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:05.636 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6810", 2023-10-26T01:37:05.636 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:37:05.636 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:05.636 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:05.637 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:05.637 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6811", 2023-10-26T01:37:05.637 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:37:05.637 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:05.637 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:05.637 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:05.638 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:37:05.638 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:05.638 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:05.638 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:05.638 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6814", 2023-10-26T01:37:05.638 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:37:05.639 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:05.639 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:05.639 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:05.639 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6815", 2023-10-26T01:37:05.639 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:37:05.639 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:05.640 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:05.640 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:05.640 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:37:05.640 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:05.640 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:05.641 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:05.641 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6812", 2023-10-26T01:37:05.641 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:37:05.641 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:05.641 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:05.642 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:05.642 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6813", 2023-10-26T01:37:05.642 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:37:05.642 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:05.642 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:05.643 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:05.643 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6809/238206046", 2023-10-26T01:37:05.643 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6811/238206046", 2023-10-26T01:37:05.643 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6815/238206046", 2023-10-26T01:37:05.643 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6813/238206046", 2023-10-26T01:37:05.643 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:37:05.644 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:37:05.644 INFO:teuthology.orchestra.run.smithi169.stdout: "full", 2023-10-26T01:37:05.644 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:37:05.644 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:05.644 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:05.645 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:05.645 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 2, 2023-10-26T01:37:05.645 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:37:05.645 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:37:05.645 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:37:05.645 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:37:05.646 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:37:05.646 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:37:05.646 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:37:05.646 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:37:05.646 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 101, 2023-10-26T01:37:05.647 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:37:05.647 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:37:05.647 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:37:05.647 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:05.647 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:05.647 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:05.648 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6800", 2023-10-26T01:37:05.648 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:37:05.648 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:05.648 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:05.648 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:05.649 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6801", 2023-10-26T01:37:05.649 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:37:05.649 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:05.649 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:05.649 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:05.650 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:37:05.650 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:05.650 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:05.650 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:05.651 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6802", 2023-10-26T01:37:05.651 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:37:05.651 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:05.651 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:05.651 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:05.651 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6803", 2023-10-26T01:37:05.652 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:37:05.652 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:05.652 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:05.652 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:05.652 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:37:05.653 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:05.653 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:05.653 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:05.653 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6806", 2023-10-26T01:37:05.653 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:37:05.653 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:05.654 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:05.654 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:05.654 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6807", 2023-10-26T01:37:05.654 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:37:05.654 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:05.655 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:05.655 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:05.655 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:37:05.655 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:05.655 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:05.655 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:05.656 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6804", 2023-10-26T01:37:05.656 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:37:05.656 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:05.656 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:05.656 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:05.656 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6805", 2023-10-26T01:37:05.657 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:37:05.657 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:05.657 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:05.657 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:05.657 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6801/3215791846", 2023-10-26T01:37:05.658 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6803/3215791846", 2023-10-26T01:37:05.658 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6807/3215791846", 2023-10-26T01:37:05.658 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6805/3215791846", 2023-10-26T01:37:05.658 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:37:05.658 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:37:05.659 INFO:teuthology.orchestra.run.smithi169.stdout: "full", 2023-10-26T01:37:05.659 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:37:05.659 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:05.659 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:05.659 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:05.659 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 3, 2023-10-26T01:37:05.660 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:37:05.660 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:37:05.660 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:37:05.660 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:37:05.660 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:37:05.661 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:37:05.661 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:37:05.661 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 14, 2023-10-26T01:37:05.661 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 101, 2023-10-26T01:37:05.661 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:37:05.661 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:37:05.662 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:37:05.662 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:05.662 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:05.662 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:05.662 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6824", 2023-10-26T01:37:05.663 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:37:05.663 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:05.663 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:05.663 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:05.663 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6825", 2023-10-26T01:37:05.663 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:37:05.664 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:05.664 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:05.664 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:05.664 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:37:05.664 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:05.665 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:05.665 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:05.665 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6826", 2023-10-26T01:37:05.665 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:37:05.665 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:05.665 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:05.666 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:05.666 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6827", 2023-10-26T01:37:05.666 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:37:05.666 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:05.666 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:05.667 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:05.667 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:37:05.667 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:05.667 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:05.667 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:05.668 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6830", 2023-10-26T01:37:05.668 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:37:05.668 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:05.668 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:05.668 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:05.668 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6831", 2023-10-26T01:37:05.669 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:37:05.669 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:05.669 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:05.669 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:05.669 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:37:05.670 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:05.670 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:05.670 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:05.670 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6828", 2023-10-26T01:37:05.670 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:37:05.670 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:05.671 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:05.671 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:05.671 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6829", 2023-10-26T01:37:05.671 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:37:05.671 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:05.671 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:05.672 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:05.672 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6825/2256701226", 2023-10-26T01:37:05.672 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6827/2256701226", 2023-10-26T01:37:05.672 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6831/2256701226", 2023-10-26T01:37:05.672 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6829/2256701226", 2023-10-26T01:37:05.672 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:37:05.673 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:37:05.673 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:37:05.673 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:05.673 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:05.673 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:05.674 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 4, 2023-10-26T01:37:05.674 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:37:05.674 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:37:05.674 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:37:05.674 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:37:05.675 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:37:05.675 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:37:05.675 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:37:05.675 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:37:05.675 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 101, 2023-10-26T01:37:05.675 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:37:05.676 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:37:05.676 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:37:05.676 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:05.676 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:05.676 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:05.676 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6816", 2023-10-26T01:37:05.677 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:37:05.677 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:05.677 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:05.677 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:05.677 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6817", 2023-10-26T01:37:05.677 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:37:05.678 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:05.678 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:05.678 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:05.678 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:37:05.678 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:05.679 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:05.679 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:05.679 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6818", 2023-10-26T01:37:05.679 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:37:05.679 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:05.679 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:05.680 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:05.680 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6819", 2023-10-26T01:37:05.680 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:37:05.680 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:05.680 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:05.680 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:05.681 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:37:05.681 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:05.681 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:05.681 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:05.681 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6825", 2023-10-26T01:37:05.682 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:37:05.682 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:05.682 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:05.682 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:05.682 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6828", 2023-10-26T01:37:05.682 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:37:05.683 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:05.683 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:05.683 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:05.683 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:37:05.683 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:05.684 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:05.684 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:05.684 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6820", 2023-10-26T01:37:05.684 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:37:05.684 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:05.684 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:05.685 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:05.685 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6823", 2023-10-26T01:37:05.685 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:37:05.685 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:05.685 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:05.686 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:05.686 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6817/1082279874", 2023-10-26T01:37:05.686 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6819/1082279874", 2023-10-26T01:37:05.686 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6828/1082279874", 2023-10-26T01:37:05.686 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6823/1082279874", 2023-10-26T01:37:05.686 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:37:05.687 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:37:05.687 INFO:teuthology.orchestra.run.smithi169.stdout: "full", 2023-10-26T01:37:05.687 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:37:05.687 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:05.687 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:05.688 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:05.688 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 5, 2023-10-26T01:37:05.688 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:37:05.688 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:37:05.688 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:37:05.688 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:37:05.689 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:37:05.689 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:37:05.689 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:37:05.689 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:37:05.689 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 101, 2023-10-26T01:37:05.689 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:37:05.690 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:37:05.690 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:37:05.690 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:05.690 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:05.690 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:05.691 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6808", 2023-10-26T01:37:05.691 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:37:05.691 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:05.691 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:05.691 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:05.691 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6809", 2023-10-26T01:37:05.692 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:37:05.692 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:05.692 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:05.692 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:05.692 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:37:05.692 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:05.693 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:05.693 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:05.693 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6810", 2023-10-26T01:37:05.693 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:37:05.693 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:05.694 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:05.694 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:05.694 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6811", 2023-10-26T01:37:05.694 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:37:05.694 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:05.695 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:05.695 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:05.695 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:37:05.695 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:05.695 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:05.695 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:05.696 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6814", 2023-10-26T01:37:05.696 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:37:05.696 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:05.696 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:05.696 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:05.696 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6815", 2023-10-26T01:37:05.697 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:37:05.697 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:05.697 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:05.697 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:05.697 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:37:05.697 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:05.698 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:05.698 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:05.698 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6812", 2023-10-26T01:37:05.698 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:37:05.698 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:05.699 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:05.699 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:05.699 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6813", 2023-10-26T01:37:05.699 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:37:05.699 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:05.699 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:05.700 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:05.700 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6809/3335654351", 2023-10-26T01:37:05.700 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6811/3335654351", 2023-10-26T01:37:05.700 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6815/3335654351", 2023-10-26T01:37:05.700 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6813/3335654351", 2023-10-26T01:37:05.701 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:37:05.701 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:37:05.701 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:37:05.701 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:05.701 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:05.701 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:05.702 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 6, 2023-10-26T01:37:05.702 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:37:05.702 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:37:05.702 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:37:05.702 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:37:05.703 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:37:05.703 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:37:05.703 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:37:05.703 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:37:05.703 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 101, 2023-10-26T01:37:05.704 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:37:05.704 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:37:05.704 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:37:05.704 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:05.704 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:05.704 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:05.704 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6800", 2023-10-26T01:37:05.705 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:37:05.705 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:05.705 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:05.705 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:05.705 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6801", 2023-10-26T01:37:05.705 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:37:05.706 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:05.706 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:05.706 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:05.706 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:37:05.706 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:05.707 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:05.707 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:05.707 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6802", 2023-10-26T01:37:05.707 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:37:05.707 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:05.707 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:05.708 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:05.708 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6803", 2023-10-26T01:37:05.708 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:37:05.708 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:05.708 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:05.708 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:05.709 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:37:05.709 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:05.709 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:05.709 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:05.709 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6806", 2023-10-26T01:37:05.710 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:37:05.710 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:05.710 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:05.710 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:05.710 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6807", 2023-10-26T01:37:05.710 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:37:05.711 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:05.711 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:05.711 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:05.711 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:37:05.711 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:05.711 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:05.712 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:05.712 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6804", 2023-10-26T01:37:05.712 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:37:05.712 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:05.712 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:05.712 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:05.713 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6805", 2023-10-26T01:37:05.713 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:37:05.713 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:05.713 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:05.713 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:05.713 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6801/2520460697", 2023-10-26T01:37:05.714 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6803/2520460697", 2023-10-26T01:37:05.714 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6807/2520460697", 2023-10-26T01:37:05.714 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6805/2520460697", 2023-10-26T01:37:05.714 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:37:05.714 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:37:05.715 INFO:teuthology.orchestra.run.smithi169.stdout: "full", 2023-10-26T01:37:05.715 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:37:05.715 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:05.715 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:05.715 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:05.715 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 7, 2023-10-26T01:37:05.716 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:37:05.716 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:37:05.716 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:37:05.716 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:37:05.716 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:37:05.716 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:37:05.717 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:37:05.717 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:37:05.717 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 101, 2023-10-26T01:37:05.717 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:37:05.717 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:37:05.717 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:37:05.718 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:05.718 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:05.718 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:05.718 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6821", 2023-10-26T01:37:05.718 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:37:05.718 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:05.719 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:05.719 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:05.719 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6822", 2023-10-26T01:37:05.719 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:37:05.719 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:05.719 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:05.720 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:05.720 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:37:05.720 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:05.720 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:05.720 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:05.720 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6824", 2023-10-26T01:37:05.721 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:37:05.721 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:05.721 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:05.721 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:05.721 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6826", 2023-10-26T01:37:05.721 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:37:05.722 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:05.722 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:05.722 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:05.722 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:37:05.722 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:05.723 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:05.723 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:05.723 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6830", 2023-10-26T01:37:05.723 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:37:05.723 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:05.724 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:05.724 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:05.724 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6831", 2023-10-26T01:37:05.724 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:37:05.724 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:05.724 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:05.725 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:05.725 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:37:05.725 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:05.725 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:05.725 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:05.725 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6827", 2023-10-26T01:37:05.726 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:37:05.726 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:05.726 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:05.726 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:05.727 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6829", 2023-10-26T01:37:05.727 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:37:05.728 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:05.728 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:05.728 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:05.729 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6822/1115930619", 2023-10-26T01:37:05.729 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6826/1115930619", 2023-10-26T01:37:05.729 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6831/1115930619", 2023-10-26T01:37:05.730 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6829/1115930619", 2023-10-26T01:37:05.730 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:37:05.731 INFO:teuthology.orchestra.run.smithi169.stdout: "backfillfull", 2023-10-26T01:37:05.731 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:37:05.731 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:37:05.732 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:05.732 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:05.733 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:37:05.733 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_xinfo": [ 2023-10-26T01:37:05.733 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:05.734 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 0, 2023-10-26T01:37:05.734 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:37:05.734 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:37:05.735 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:37:05.735 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:37:05.735 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:37:05.736 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.889675+0000", 2023-10-26T01:37:05.736 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:37:05.737 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:05.737 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:05.737 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 1, 2023-10-26T01:37:05.738 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:37:05.738 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:37:05.738 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:37:05.739 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:37:05.739 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:37:05.740 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.864157+0000", 2023-10-26T01:37:05.740 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:37:05.740 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:05.741 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:05.741 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 2, 2023-10-26T01:37:05.741 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:37:05.742 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:37:05.742 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:37:05.742 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:37:05.743 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:37:05.743 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.876919+0000", 2023-10-26T01:37:05.743 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:37:05.744 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:05.744 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:05.744 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 3, 2023-10-26T01:37:05.745 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:37:05.745 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:37:05.745 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:37:05.746 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:37:05.746 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:37:05.747 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.974541+0000", 2023-10-26T01:37:05.747 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:37:05.747 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:05.748 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:05.748 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 4, 2023-10-26T01:37:05.748 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:37:05.748 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:37:05.749 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:37:05.749 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:37:05.749 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:37:05.749 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.969696+0000", 2023-10-26T01:37:05.749 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:37:05.750 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:05.750 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:05.750 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 5, 2023-10-26T01:37:05.750 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:37:05.750 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:37:05.751 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:37:05.751 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:37:05.751 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:37:05.751 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.859703+0000", 2023-10-26T01:37:05.751 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:37:05.752 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:05.752 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:05.752 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 6, 2023-10-26T01:37:05.752 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:37:05.752 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:37:05.753 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:37:05.753 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:37:05.753 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:37:05.753 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.867917+0000", 2023-10-26T01:37:05.753 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:37:05.753 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:05.754 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:05.754 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 7, 2023-10-26T01:37:05.754 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:37:05.754 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:37:05.754 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:37:05.755 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:37:05.755 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:37:05.755 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.898605+0000", 2023-10-26T01:37:05.755 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:37:05.755 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:05.756 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:37:05.756 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_upmap": [], 2023-10-26T01:37:05.756 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_upmap_items": [], 2023-10-26T01:37:05.756 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_upmap_primaries": [], 2023-10-26T01:37:05.756 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_temp": [], 2023-10-26T01:37:05.756 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_temp": [], 2023-10-26T01:37:05.757 INFO:teuthology.orchestra.run.smithi169.stdout: "blocklist": {}, 2023-10-26T01:37:05.757 INFO:teuthology.orchestra.run.smithi169.stdout: "range_blocklist": {}, 2023-10-26T01:37:05.757 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profiles": { 2023-10-26T01:37:05.757 INFO:teuthology.orchestra.run.smithi169.stdout: "default": { 2023-10-26T01:37:05.757 INFO:teuthology.orchestra.run.smithi169.stdout: "crush-failure-domain": "osd", 2023-10-26T01:37:05.758 INFO:teuthology.orchestra.run.smithi169.stdout: "k": "2", 2023-10-26T01:37:05.758 INFO:teuthology.orchestra.run.smithi169.stdout: "m": "1", 2023-10-26T01:37:05.758 INFO:teuthology.orchestra.run.smithi169.stdout: "plugin": "jerasure", 2023-10-26T01:37:05.758 INFO:teuthology.orchestra.run.smithi169.stdout: "technique": "reed_sol_van" 2023-10-26T01:37:05.758 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:05.759 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:05.759 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps_queue": [], 2023-10-26T01:37:05.759 INFO:teuthology.orchestra.run.smithi169.stdout: "new_removed_snaps": [], 2023-10-26T01:37:05.759 INFO:teuthology.orchestra.run.smithi169.stdout: "new_purged_snaps": [], 2023-10-26T01:37:05.759 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_node_flags": {}, 2023-10-26T01:37:05.759 INFO:teuthology.orchestra.run.smithi169.stdout: "device_class_flags": {}, 2023-10-26T01:37:05.760 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_mode": { 2023-10-26T01:37:05.760 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_mode_enabled": false, 2023-10-26T01:37:05.760 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_bucket_count": 0, 2023-10-26T01:37:05.760 INFO:teuthology.orchestra.run.smithi169.stdout: "degraded_stretch_mode": 0, 2023-10-26T01:37:05.760 INFO:teuthology.orchestra.run.smithi169.stdout: "recovering_stretch_mode": 0, 2023-10-26T01:37:05.761 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_mode_bucket": 0 2023-10-26T01:37:05.761 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:05.761 INFO:teuthology.orchestra.run.smithi169.stdout:} 2023-10-26T01:37:05.762 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:05.543+0000 7f306fde2640 1 -- 172.21.15.169:0/521904249 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f304c06ae40 msgr2=0x7f304c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:05.763 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:05.543+0000 7f306fde2640 1 --2- 172.21.15.169:0/521904249 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f304c06ae40 0x7f304c06d300 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f3058007fd0 tx=0x7f30580078e0 comp rx=0 tx=0).stop 2023-10-26T01:37:05.763 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:05.543+0000 7f306fde2640 1 -- 172.21.15.169:0/521904249 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30680cde30 msgr2=0x7f3068142660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:05.763 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:05.543+0000 7f306fde2640 1 --2- 172.21.15.169:0/521904249 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30680cde30 0x7f3068142660 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f306400c430 tx=0x7f306400c8f0 comp rx=0 tx=0).stop 2023-10-26T01:37:05.763 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:05.543+0000 7f306fde2640 1 -- 172.21.15.169:0/521904249 shutdown_connections 2023-10-26T01:37:05.763 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:05.543+0000 7f306fde2640 1 --2- 172.21.15.169:0/521904249 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f304c06ae40 0x7f304c06d300 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:05.764 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:05.543+0000 7f306fde2640 1 --2- 172.21.15.169:0/521904249 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3068158850 0x7f306814c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:05.764 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:05.543+0000 7f306fde2640 1 --2- 172.21.15.169:0/521904249 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f306814e790 0x7f3068141b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:05.764 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:05.543+0000 7f306fde2640 1 --2- 172.21.15.169:0/521904249 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30680cde30 0x7f3068142660 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:05.764 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:05.543+0000 7f306fde2640 1 -- 172.21.15.169:0/521904249 >> 172.21.15.169:0/521904249 conn(0x7f3068069930 msgr2=0x7f3068151670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:05.764 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:05.543+0000 7f306fde2640 1 -- 172.21.15.169:0/521904249 shutdown_connections 2023-10-26T01:37:05.765 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:05.543+0000 7f306fde2640 1 -- 172.21.15.169:0/521904249 wait complete. 2023-10-26T01:37:05.765 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-10-26T01:37:05.766 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:37:05.766 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:37:05.828 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:37:05.829 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=50 seek=265) 2023-10-26T01:37:06.021 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T01:37:06.030 DEBUG:teuthology.orchestra.run.smithi170:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T01:37:06.253 INFO:teuthology.orchestra.run.smithi169.stderr:dd: fdatasync failed for 'large_file_b': No space left on device 2023-10-26T01:37:06.254 INFO:teuthology.orchestra.run.smithi169.stderr:50+0 records in 2023-10-26T01:37:06.254 INFO:teuthology.orchestra.run.smithi169.stderr:50+0 records out 2023-10-26T01:37:06.254 INFO:teuthology.orchestra.run.smithi169.stderr:52428800 bytes (52 MB, 50 MiB) copied, 0.35527 s, 148 MB/s 2023-10-26T01:37:06.255 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-26T01:37:06.255 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-26T01:37:06.462 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:06.459+0000 7f828521a640 1 -- 172.21.15.169:0/3273285250 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f828006cae0 msgr2=0x7f82800c7ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:06.463 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:06.459+0000 7f828521a640 1 --2- 172.21.15.169:0/3273285250 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f828006cae0 0x7f82800c7ea0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f8270009830 tx=0x7f827002bcd0 comp rx=0 tx=0).stop 2023-10-26T01:37:06.463 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:06.459+0000 7f828521a640 1 -- 172.21.15.169:0/3273285250 shutdown_connections 2023-10-26T01:37:06.463 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:06.459+0000 7f828521a640 1 --2- 172.21.15.169:0/3273285250 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f828006cae0 0x7f82800c7ea0 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:06.463 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:06.459+0000 7f828521a640 1 -- 172.21.15.169:0/3273285250 >> 172.21.15.169:0/3273285250 conn(0x7f82800aedd0 msgr2=0x7f82800af1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:06.463 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:06.459+0000 7f828521a640 1 -- 172.21.15.169:0/3273285250 shutdown_connections 2023-10-26T01:37:06.464 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:06.459+0000 7f828521a640 1 -- 172.21.15.169:0/3273285250 wait complete. 2023-10-26T01:37:06.464 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:06.461+0000 7f828521a640 1 Processor -- start 2023-10-26T01:37:06.464 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:06.461+0000 7f828521a640 1 -- start start 2023-10-26T01:37:06.467 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:06.462+0000 7f828521a640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f828006a990 0x7f82800c6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:06.467 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:06.462+0000 7f828521a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f828006cae0 0x7f82800c67b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:06.467 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:06.462+0000 7f828521a640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f82800c96f0 0x7f8280099dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:06.467 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:06.462+0000 7f828521a640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f828015b0b0 con 0x7f82800c96f0 2023-10-26T01:37:06.467 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:06.462+0000 7f828521a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f828015b390 con 0x7f828006cae0 2023-10-26T01:37:06.468 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:06.462+0000 7f828521a640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f828015b670 con 0x7f828006a990 2023-10-26T01:37:06.468 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:06.463+0000 7f8284a19640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f82800c96f0 0x7f8280099dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:06.468 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:06.463+0000 7f8284a19640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f82800c96f0 0x7f8280099dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:59932/0 (socket says 172.21.15.169:59932) 2023-10-26T01:37:06.468 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:06.463+0000 7f8284a19640 1 -- 172.21.15.169:0/2018803421 learned_addr learned my addr 172.21.15.169:0/2018803421 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:37:06.468 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:06.463+0000 7f827ffff640 1 --2- 172.21.15.169:0/2018803421 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f828006a990 0x7f82800c6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:06.469 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:06.463+0000 7f8277fff640 1 --2- 172.21.15.169:0/2018803421 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f828006cae0 0x7f82800c67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:06.469 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:06.463+0000 7f8284a19640 1 -- 172.21.15.169:0/2018803421 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f828006a990 msgr2=0x7f82800c6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:06.469 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:06.463+0000 7f8284a19640 1 --2- 172.21.15.169:0/2018803421 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f828006a990 0x7f82800c6270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:06.469 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:06.463+0000 7f8284a19640 1 -- 172.21.15.169:0/2018803421 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f828006cae0 msgr2=0x7f82800c67b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:06.469 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:06.463+0000 7f8284a19640 1 --2- 172.21.15.169:0/2018803421 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f828006cae0 0x7f82800c67b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:06.470 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:06.463+0000 7f8284a19640 1 -- 172.21.15.169:0/2018803421 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8270009490 con 0x7f82800c96f0 2023-10-26T01:37:06.470 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:06.463+0000 7f8284a19640 1 --2- 172.21.15.169:0/2018803421 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f82800c96f0 0x7f8280099dd0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f827800b990 tx=0x7f827800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:06.470 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:06.464+0000 7f827d7fa640 1 -- 172.21.15.169:0/2018803421 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f827800cba0 con 0x7f82800c96f0 2023-10-26T01:37:06.470 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:06.464+0000 7f827d7fa640 1 -- 172.21.15.169:0/2018803421 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f827800cdf0 con 0x7f82800c96f0 2023-10-26T01:37:06.470 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:06.464+0000 7f828521a640 1 -- 172.21.15.169:0/2018803421 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f828015b950 con 0x7f82800c96f0 2023-10-26T01:37:06.471 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:06.464+0000 7f827d7fa640 1 -- 172.21.15.169:0/2018803421 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82780146a0 con 0x7f82800c96f0 2023-10-26T01:37:06.471 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:06.464+0000 7f828521a640 1 -- 172.21.15.169:0/2018803421 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f828015c150 con 0x7f82800c96f0 2023-10-26T01:37:06.472 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:06.465+0000 7f828521a640 1 -- 172.21.15.169:0/2018803421 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f828015be70 con 0x7f82800c96f0 2023-10-26T01:37:06.472 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:06.466+0000 7f827d7fa640 1 -- 172.21.15.169:0/2018803421 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f82780148c0 con 0x7f82800c96f0 2023-10-26T01:37:06.472 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:06.467+0000 7f827d7fa640 1 --2- 172.21.15.169:0/2018803421 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f826006ae40 0x7f826006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:06.472 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:06.467+0000 7f827d7fa640 1 -- 172.21.15.169:0/2018803421 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f82780898a0 con 0x7f82800c96f0 2023-10-26T01:37:06.473 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:06.468+0000 7f827d7fa640 1 -- 172.21.15.169:0/2018803421 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=106}) v3 -- 0x7f8260072970 con 0x7f82800c96f0 2023-10-26T01:37:06.473 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:06.468+0000 7f827ffff640 1 --2- 172.21.15.169:0/2018803421 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f826006ae40 0x7f826006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:06.473 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:06.469+0000 7f827ffff640 1 --2- 172.21.15.169:0/2018803421 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f826006ae40 0x7f826006d300 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f8270036000 tx=0x7f8270004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:06.474 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:06.472+0000 7f827d7fa640 1 -- 172.21.15.169:0/2018803421 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8278019020 con 0x7f82800c96f0 2023-10-26T01:37:06.664 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:06.662+0000 7f828521a640 1 -- 172.21.15.169:0/2018803421 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f8280073c10 con 0x7f82800c96f0 2023-10-26T01:37:06.666 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:06.664+0000 7f827d7fa640 1 -- 172.21.15.169:0/2018803421 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v105) v1 ==== 81+0+34222 (secure 0 0 0) 0x7f828015be70 con 0x7f82800c96f0 2023-10-26T01:37:06.667 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:37:06.667 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2023-10-26T01:37:06.668 INFO:teuthology.orchestra.run.smithi169.stdout: "epoch": 105, 2023-10-26T01:37:06.668 INFO:teuthology.orchestra.run.smithi169.stdout: "fsid": "30203b28-ed8a-44e8-a9a6-a9bf25f980f1", 2023-10-26T01:37:06.668 INFO:teuthology.orchestra.run.smithi169.stdout: "created": "2023-10-26T01:30:22.897155+0000", 2023-10-26T01:37:06.668 INFO:teuthology.orchestra.run.smithi169.stdout: "modified": "2023-10-26T01:37:02.113248+0000", 2023-10-26T01:37:06.668 INFO:teuthology.orchestra.run.smithi169.stdout: "last_up_change": "2023-10-26T01:30:31.875281+0000", 2023-10-26T01:37:06.669 INFO:teuthology.orchestra.run.smithi169.stdout: "last_in_change": "2023-10-26T01:30:27.637941+0000", 2023-10-26T01:37:06.669 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-26T01:37:06.669 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_num": 5799936, 2023-10-26T01:37:06.669 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_set": [ 2023-10-26T01:37:06.669 INFO:teuthology.orchestra.run.smithi169.stdout: "pglog_hardlimit", 2023-10-26T01:37:06.670 INFO:teuthology.orchestra.run.smithi169.stdout: "purged_snapdirs", 2023-10-26T01:37:06.670 INFO:teuthology.orchestra.run.smithi169.stdout: "recovery_deletes", 2023-10-26T01:37:06.670 INFO:teuthology.orchestra.run.smithi169.stdout: "sortbitwise" 2023-10-26T01:37:06.670 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:37:06.670 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_version": 4, 2023-10-26T01:37:06.671 INFO:teuthology.orchestra.run.smithi169.stdout: "full_ratio": 0.69999998807907104, 2023-10-26T01:37:06.671 INFO:teuthology.orchestra.run.smithi169.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-26T01:37:06.671 INFO:teuthology.orchestra.run.smithi169.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-26T01:37:06.671 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_snapshot": "", 2023-10-26T01:37:06.671 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_max": 13, 2023-10-26T01:37:06.671 INFO:teuthology.orchestra.run.smithi169.stdout: "max_osd": 8, 2023-10-26T01:37:06.672 INFO:teuthology.orchestra.run.smithi169.stdout: "require_min_compat_client": "luminous", 2023-10-26T01:37:06.672 INFO:teuthology.orchestra.run.smithi169.stdout: "min_compat_client": "jewel", 2023-10-26T01:37:06.672 INFO:teuthology.orchestra.run.smithi169.stdout: "require_osd_release": "reef", 2023-10-26T01:37:06.672 INFO:teuthology.orchestra.run.smithi169.stdout: "allow_crimson": false, 2023-10-26T01:37:06.672 INFO:teuthology.orchestra.run.smithi169.stdout: "pools": [ 2023-10-26T01:37:06.673 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:06.673 INFO:teuthology.orchestra.run.smithi169.stdout: "pool": 1, 2023-10-26T01:37:06.673 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_name": ".mgr", 2023-10-26T01:37:06.673 INFO:teuthology.orchestra.run.smithi169.stdout: "create_time": "2023-10-26T01:30:32.200808+0000", 2023-10-26T01:37:06.673 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": 3, 2023-10-26T01:37:06.674 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_names": "hashpspool,full", 2023-10-26T01:37:06.674 INFO:teuthology.orchestra.run.smithi169.stdout: "type": 1, 2023-10-26T01:37:06.674 INFO:teuthology.orchestra.run.smithi169.stdout: "size": 2, 2023-10-26T01:37:06.674 INFO:teuthology.orchestra.run.smithi169.stdout: "min_size": 1, 2023-10-26T01:37:06.674 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_rule": 0, 2023-10-26T01:37:06.675 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_count": 0, 2023-10-26T01:37:06.675 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_target": 0, 2023-10-26T01:37:06.675 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T01:37:06.675 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T01:37:06.675 INFO:teuthology.orchestra.run.smithi169.stdout: "object_hash": 2, 2023-10-26T01:37:06.675 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_mode": "off", 2023-10-26T01:37:06.676 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num": 1, 2023-10-26T01:37:06.676 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num": 1, 2023-10-26T01:37:06.676 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num_target": 1, 2023-10-26T01:37:06.676 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_target": 1, 2023-10-26T01:37:06.676 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_pending": 1, 2023-10-26T01:37:06.677 INFO:teuthology.orchestra.run.smithi169.stdout: "last_pg_merge_meta": { 2023-10-26T01:37:06.677 INFO:teuthology.orchestra.run.smithi169.stdout: "source_pgid": "0.0", 2023-10-26T01:37:06.677 INFO:teuthology.orchestra.run.smithi169.stdout: "ready_epoch": 0, 2023-10-26T01:37:06.677 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_started": 0, 2023-10-26T01:37:06.677 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_clean": 0, 2023-10-26T01:37:06.678 INFO:teuthology.orchestra.run.smithi169.stdout: "source_version": "0'0", 2023-10-26T01:37:06.678 INFO:teuthology.orchestra.run.smithi169.stdout: "target_version": "0'0" 2023-10-26T01:37:06.678 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:06.678 INFO:teuthology.orchestra.run.smithi169.stdout: "last_change": "105", 2023-10-26T01:37:06.678 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend": "0", 2023-10-26T01:37:06.679 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T01:37:06.679 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T01:37:06.679 INFO:teuthology.orchestra.run.smithi169.stdout: "auid": 0, 2023-10-26T01:37:06.679 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_mode": "selfmanaged", 2023-10-26T01:37:06.679 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_seq": 0, 2023-10-26T01:37:06.679 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_epoch": 0, 2023-10-26T01:37:06.680 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_snaps": [], 2023-10-26T01:37:06.680 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps": "[]", 2023-10-26T01:37:06.680 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_bytes": 0, 2023-10-26T01:37:06.680 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_objects": 0, 2023-10-26T01:37:06.680 INFO:teuthology.orchestra.run.smithi169.stdout: "tiers": [], 2023-10-26T01:37:06.681 INFO:teuthology.orchestra.run.smithi169.stdout: "tier_of": -1, 2023-10-26T01:37:06.681 INFO:teuthology.orchestra.run.smithi169.stdout: "read_tier": -1, 2023-10-26T01:37:06.681 INFO:teuthology.orchestra.run.smithi169.stdout: "write_tier": -1, 2023-10-26T01:37:06.681 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_mode": "none", 2023-10-26T01:37:06.681 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_bytes": 0, 2023-10-26T01:37:06.681 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_objects": 0, 2023-10-26T01:37:06.682 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T01:37:06.682 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T01:37:06.682 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T01:37:06.682 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_flush_age": 0, 2023-10-26T01:37:06.682 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_evict_age": 0, 2023-10-26T01:37:06.682 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profile": "", 2023-10-26T01:37:06.683 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_params": { 2023-10-26T01:37:06.683 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "none" 2023-10-26T01:37:06.683 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:06.683 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_period": 0, 2023-10-26T01:37:06.684 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_count": 0, 2023-10-26T01:37:06.684 INFO:teuthology.orchestra.run.smithi169.stdout: "use_gmt_hitset": true, 2023-10-26T01:37:06.684 INFO:teuthology.orchestra.run.smithi169.stdout: "min_read_recency_for_promote": 0, 2023-10-26T01:37:06.684 INFO:teuthology.orchestra.run.smithi169.stdout: "min_write_recency_for_promote": 0, 2023-10-26T01:37:06.684 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T01:37:06.685 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_search_last_n": 0, 2023-10-26T01:37:06.685 INFO:teuthology.orchestra.run.smithi169.stdout: "grade_table": [], 2023-10-26T01:37:06.685 INFO:teuthology.orchestra.run.smithi169.stdout: "stripe_width": 0, 2023-10-26T01:37:06.685 INFO:teuthology.orchestra.run.smithi169.stdout: "expected_num_objects": 0, 2023-10-26T01:37:06.685 INFO:teuthology.orchestra.run.smithi169.stdout: "fast_read": false, 2023-10-26T01:37:06.685 INFO:teuthology.orchestra.run.smithi169.stdout: "options": { 2023-10-26T01:37:06.686 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_max": 32, 2023-10-26T01:37:06.686 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_min": 1 2023-10-26T01:37:06.686 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:06.686 INFO:teuthology.orchestra.run.smithi169.stdout: "application_metadata": { 2023-10-26T01:37:06.686 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": {} 2023-10-26T01:37:06.687 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:06.687 INFO:teuthology.orchestra.run.smithi169.stdout: "read_balance": { 2023-10-26T01:37:06.688 INFO:teuthology.orchestra.run.smithi169.stdout: "score_acting": 8, 2023-10-26T01:37:06.688 INFO:teuthology.orchestra.run.smithi169.stdout: "score_stable": 8, 2023-10-26T01:37:06.688 INFO:teuthology.orchestra.run.smithi169.stdout: "optimal_score": 0.25, 2023-10-26T01:37:06.688 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_acting": 2, 2023-10-26T01:37:06.689 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_stable": 2, 2023-10-26T01:37:06.689 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity_weighted": 1, 2023-10-26T01:37:06.689 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity": 1, 2023-10-26T01:37:06.689 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity_weighted": 1 2023-10-26T01:37:06.689 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:06.690 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:06.690 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:06.690 INFO:teuthology.orchestra.run.smithi169.stdout: "pool": 12, 2023-10-26T01:37:06.690 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_name": "cephfs_metadata", 2023-10-26T01:37:06.690 INFO:teuthology.orchestra.run.smithi169.stdout: "create_time": "2023-10-26T01:36:41.430455+0000", 2023-10-26T01:37:06.691 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": 3, 2023-10-26T01:37:06.691 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_names": "hashpspool,full", 2023-10-26T01:37:06.691 INFO:teuthology.orchestra.run.smithi169.stdout: "type": 1, 2023-10-26T01:37:06.691 INFO:teuthology.orchestra.run.smithi169.stdout: "size": 2, 2023-10-26T01:37:06.691 INFO:teuthology.orchestra.run.smithi169.stdout: "min_size": 1, 2023-10-26T01:37:06.692 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_rule": 0, 2023-10-26T01:37:06.692 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_count": 0, 2023-10-26T01:37:06.692 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_target": 0, 2023-10-26T01:37:06.692 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T01:37:06.692 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T01:37:06.693 INFO:teuthology.orchestra.run.smithi169.stdout: "object_hash": 2, 2023-10-26T01:37:06.693 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_mode": "off", 2023-10-26T01:37:06.693 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num": 32, 2023-10-26T01:37:06.693 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num": 32, 2023-10-26T01:37:06.694 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num_target": 32, 2023-10-26T01:37:06.694 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_target": 32, 2023-10-26T01:37:06.694 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_pending": 32, 2023-10-26T01:37:06.694 INFO:teuthology.orchestra.run.smithi169.stdout: "last_pg_merge_meta": { 2023-10-26T01:37:06.695 INFO:teuthology.orchestra.run.smithi169.stdout: "source_pgid": "0.0", 2023-10-26T01:37:06.695 INFO:teuthology.orchestra.run.smithi169.stdout: "ready_epoch": 0, 2023-10-26T01:37:06.696 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_started": 0, 2023-10-26T01:37:06.696 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_clean": 0, 2023-10-26T01:37:06.696 INFO:teuthology.orchestra.run.smithi169.stdout: "source_version": "0'0", 2023-10-26T01:37:06.696 INFO:teuthology.orchestra.run.smithi169.stdout: "target_version": "0'0" 2023-10-26T01:37:06.697 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:06.697 INFO:teuthology.orchestra.run.smithi169.stdout: "last_change": "105", 2023-10-26T01:37:06.697 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend": "0", 2023-10-26T01:37:06.697 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T01:37:06.697 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T01:37:06.698 INFO:teuthology.orchestra.run.smithi169.stdout: "auid": 0, 2023-10-26T01:37:06.698 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_mode": "selfmanaged", 2023-10-26T01:37:06.698 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_seq": 0, 2023-10-26T01:37:06.698 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_epoch": 0, 2023-10-26T01:37:06.699 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_snaps": [], 2023-10-26T01:37:06.699 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps": "[]", 2023-10-26T01:37:06.699 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_bytes": 0, 2023-10-26T01:37:06.699 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_objects": 0, 2023-10-26T01:37:06.699 INFO:teuthology.orchestra.run.smithi169.stdout: "tiers": [], 2023-10-26T01:37:06.700 INFO:teuthology.orchestra.run.smithi169.stdout: "tier_of": -1, 2023-10-26T01:37:06.700 INFO:teuthology.orchestra.run.smithi169.stdout: "read_tier": -1, 2023-10-26T01:37:06.700 INFO:teuthology.orchestra.run.smithi169.stdout: "write_tier": -1, 2023-10-26T01:37:06.700 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_mode": "none", 2023-10-26T01:37:06.700 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_bytes": 0, 2023-10-26T01:37:06.701 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_objects": 0, 2023-10-26T01:37:06.701 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T01:37:06.701 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T01:37:06.701 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T01:37:06.701 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_flush_age": 0, 2023-10-26T01:37:06.702 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_evict_age": 0, 2023-10-26T01:37:06.702 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profile": "", 2023-10-26T01:37:06.702 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_params": { 2023-10-26T01:37:06.702 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "none" 2023-10-26T01:37:06.702 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:06.703 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_period": 0, 2023-10-26T01:37:06.703 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_count": 0, 2023-10-26T01:37:06.703 INFO:teuthology.orchestra.run.smithi169.stdout: "use_gmt_hitset": true, 2023-10-26T01:37:06.703 INFO:teuthology.orchestra.run.smithi169.stdout: "min_read_recency_for_promote": 0, 2023-10-26T01:37:06.704 INFO:teuthology.orchestra.run.smithi169.stdout: "min_write_recency_for_promote": 0, 2023-10-26T01:37:06.704 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T01:37:06.704 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_search_last_n": 0, 2023-10-26T01:37:06.704 INFO:teuthology.orchestra.run.smithi169.stdout: "grade_table": [], 2023-10-26T01:37:06.704 INFO:teuthology.orchestra.run.smithi169.stdout: "stripe_width": 0, 2023-10-26T01:37:06.704 INFO:teuthology.orchestra.run.smithi169.stdout: "expected_num_objects": 0, 2023-10-26T01:37:06.705 INFO:teuthology.orchestra.run.smithi169.stdout: "fast_read": false, 2023-10-26T01:37:06.705 INFO:teuthology.orchestra.run.smithi169.stdout: "options": { 2023-10-26T01:37:06.705 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_bias": 4, 2023-10-26T01:37:06.705 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_min": 16, 2023-10-26T01:37:06.705 INFO:teuthology.orchestra.run.smithi169.stdout: "recovery_priority": 5 2023-10-26T01:37:06.706 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:06.706 INFO:teuthology.orchestra.run.smithi169.stdout: "application_metadata": { 2023-10-26T01:37:06.706 INFO:teuthology.orchestra.run.smithi169.stdout: "cephfs": { 2023-10-26T01:37:06.706 INFO:teuthology.orchestra.run.smithi169.stdout: "metadata": "cephfs" 2023-10-26T01:37:06.706 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:06.707 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:06.707 INFO:teuthology.orchestra.run.smithi169.stdout: "read_balance": { 2023-10-26T01:37:06.707 INFO:teuthology.orchestra.run.smithi169.stdout: "score_acting": 2, 2023-10-26T01:37:06.707 INFO:teuthology.orchestra.run.smithi169.stdout: "score_stable": 2, 2023-10-26T01:37:06.707 INFO:teuthology.orchestra.run.smithi169.stdout: "optimal_score": 1, 2023-10-26T01:37:06.708 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_acting": 2, 2023-10-26T01:37:06.708 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_stable": 2, 2023-10-26T01:37:06.708 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity_weighted": 1, 2023-10-26T01:37:06.708 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity": 1, 2023-10-26T01:37:06.708 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity_weighted": 1 2023-10-26T01:37:06.708 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:06.709 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:06.709 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:06.709 INFO:teuthology.orchestra.run.smithi169.stdout: "pool": 13, 2023-10-26T01:37:06.709 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_name": "cephfs_data", 2023-10-26T01:37:06.709 INFO:teuthology.orchestra.run.smithi169.stdout: "create_time": "2023-10-26T01:36:42.883252+0000", 2023-10-26T01:37:06.710 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": 3, 2023-10-26T01:37:06.710 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_names": "hashpspool,full", 2023-10-26T01:37:06.710 INFO:teuthology.orchestra.run.smithi169.stdout: "type": 1, 2023-10-26T01:37:06.710 INFO:teuthology.orchestra.run.smithi169.stdout: "size": 2, 2023-10-26T01:37:06.711 INFO:teuthology.orchestra.run.smithi169.stdout: "min_size": 1, 2023-10-26T01:37:06.711 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_rule": 0, 2023-10-26T01:37:06.711 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_count": 0, 2023-10-26T01:37:06.711 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_target": 0, 2023-10-26T01:37:06.711 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T01:37:06.712 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T01:37:06.712 INFO:teuthology.orchestra.run.smithi169.stdout: "object_hash": 2, 2023-10-26T01:37:06.712 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_mode": "off", 2023-10-26T01:37:06.712 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num": 64, 2023-10-26T01:37:06.712 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num": 64, 2023-10-26T01:37:06.713 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num_target": 64, 2023-10-26T01:37:06.713 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_target": 64, 2023-10-26T01:37:06.713 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_pending": 64, 2023-10-26T01:37:06.713 INFO:teuthology.orchestra.run.smithi169.stdout: "last_pg_merge_meta": { 2023-10-26T01:37:06.713 INFO:teuthology.orchestra.run.smithi169.stdout: "source_pgid": "0.0", 2023-10-26T01:37:06.714 INFO:teuthology.orchestra.run.smithi169.stdout: "ready_epoch": 0, 2023-10-26T01:37:06.714 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_started": 0, 2023-10-26T01:37:06.714 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_clean": 0, 2023-10-26T01:37:06.715 INFO:teuthology.orchestra.run.smithi169.stdout: "source_version": "0'0", 2023-10-26T01:37:06.715 INFO:teuthology.orchestra.run.smithi169.stdout: "target_version": "0'0" 2023-10-26T01:37:06.715 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:06.715 INFO:teuthology.orchestra.run.smithi169.stdout: "last_change": "105", 2023-10-26T01:37:06.716 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend": "0", 2023-10-26T01:37:06.716 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T01:37:06.716 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T01:37:06.716 INFO:teuthology.orchestra.run.smithi169.stdout: "auid": 0, 2023-10-26T01:37:06.717 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_mode": "selfmanaged", 2023-10-26T01:37:06.717 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_seq": 0, 2023-10-26T01:37:06.717 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_epoch": 0, 2023-10-26T01:37:06.717 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_snaps": [], 2023-10-26T01:37:06.718 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps": "[]", 2023-10-26T01:37:06.718 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_bytes": 0, 2023-10-26T01:37:06.718 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_objects": 0, 2023-10-26T01:37:06.718 INFO:teuthology.orchestra.run.smithi169.stdout: "tiers": [], 2023-10-26T01:37:06.718 INFO:teuthology.orchestra.run.smithi169.stdout: "tier_of": -1, 2023-10-26T01:37:06.719 INFO:teuthology.orchestra.run.smithi169.stdout: "read_tier": -1, 2023-10-26T01:37:06.719 INFO:teuthology.orchestra.run.smithi169.stdout: "write_tier": -1, 2023-10-26T01:37:06.719 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_mode": "none", 2023-10-26T01:37:06.719 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_bytes": 0, 2023-10-26T01:37:06.719 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_objects": 0, 2023-10-26T01:37:06.720 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T01:37:06.720 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T01:37:06.720 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T01:37:06.720 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_flush_age": 0, 2023-10-26T01:37:06.720 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_evict_age": 0, 2023-10-26T01:37:06.721 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profile": "", 2023-10-26T01:37:06.721 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_params": { 2023-10-26T01:37:06.721 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "none" 2023-10-26T01:37:06.721 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:06.722 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_period": 0, 2023-10-26T01:37:06.722 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_count": 0, 2023-10-26T01:37:06.723 INFO:teuthology.orchestra.run.smithi169.stdout: "use_gmt_hitset": true, 2023-10-26T01:37:06.723 INFO:teuthology.orchestra.run.smithi169.stdout: "min_read_recency_for_promote": 0, 2023-10-26T01:37:06.723 INFO:teuthology.orchestra.run.smithi169.stdout: "min_write_recency_for_promote": 0, 2023-10-26T01:37:06.723 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T01:37:06.723 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_search_last_n": 0, 2023-10-26T01:37:06.724 INFO:teuthology.orchestra.run.smithi169.stdout: "grade_table": [], 2023-10-26T01:37:06.724 INFO:teuthology.orchestra.run.smithi169.stdout: "stripe_width": 0, 2023-10-26T01:37:06.724 INFO:teuthology.orchestra.run.smithi169.stdout: "expected_num_objects": 0, 2023-10-26T01:37:06.724 INFO:teuthology.orchestra.run.smithi169.stdout: "fast_read": false, 2023-10-26T01:37:06.725 INFO:teuthology.orchestra.run.smithi169.stdout: "options": { 2023-10-26T01:37:06.725 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_min": 64, 2023-10-26T01:37:06.725 INFO:teuthology.orchestra.run.smithi169.stdout: "target_size_ratio": 0.89999997615814209 2023-10-26T01:37:06.725 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:06.725 INFO:teuthology.orchestra.run.smithi169.stdout: "application_metadata": { 2023-10-26T01:37:06.726 INFO:teuthology.orchestra.run.smithi169.stdout: "cephfs": { 2023-10-26T01:37:06.726 INFO:teuthology.orchestra.run.smithi169.stdout: "data": "cephfs" 2023-10-26T01:37:06.726 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:06.726 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:06.727 INFO:teuthology.orchestra.run.smithi169.stdout: "read_balance": { 2023-10-26T01:37:06.727 INFO:teuthology.orchestra.run.smithi169.stdout: "score_acting": 1.3799999952316284, 2023-10-26T01:37:06.727 INFO:teuthology.orchestra.run.smithi169.stdout: "score_stable": 1.3799999952316284, 2023-10-26T01:37:06.727 INFO:teuthology.orchestra.run.smithi169.stdout: "optimal_score": 1, 2023-10-26T01:37:06.727 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_acting": 1.3799999952316284, 2023-10-26T01:37:06.728 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_stable": 1.3799999952316284, 2023-10-26T01:37:06.728 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity_weighted": 1, 2023-10-26T01:37:06.728 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity": 1, 2023-10-26T01:37:06.728 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity_weighted": 1 2023-10-26T01:37:06.728 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:06.728 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:06.729 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:37:06.729 INFO:teuthology.orchestra.run.smithi169.stdout: "osds": [ 2023-10-26T01:37:06.729 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:06.729 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 0, 2023-10-26T01:37:06.730 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:37:06.730 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:37:06.730 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:37:06.730 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:37:06.730 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:37:06.731 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:37:06.731 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:37:06.731 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:37:06.731 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 101, 2023-10-26T01:37:06.731 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:37:06.731 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:37:06.732 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:37:06.732 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:06.732 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:06.732 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:06.732 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6816", 2023-10-26T01:37:06.733 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:37:06.733 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:06.733 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:06.733 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:06.733 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6817", 2023-10-26T01:37:06.734 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:37:06.734 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:06.734 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:06.734 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:06.734 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:37:06.735 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:06.735 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:06.735 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:06.735 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6818", 2023-10-26T01:37:06.736 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:37:06.736 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:06.736 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:06.736 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:06.736 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6819", 2023-10-26T01:37:06.736 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:37:06.737 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:06.737 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:06.737 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:06.737 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:37:06.737 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:06.738 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:06.738 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:06.738 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6822", 2023-10-26T01:37:06.738 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:37:06.738 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:06.739 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:06.739 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:06.739 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6823", 2023-10-26T01:37:06.739 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:37:06.739 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:06.740 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:06.740 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:06.740 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:37:06.740 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:06.740 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:06.741 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:06.741 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6820", 2023-10-26T01:37:06.741 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:37:06.741 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:06.742 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:06.742 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:06.742 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6821", 2023-10-26T01:37:06.742 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:37:06.742 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:06.743 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:06.743 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:06.743 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6817/4204255864", 2023-10-26T01:37:06.743 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6819/4204255864", 2023-10-26T01:37:06.743 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6823/4204255864", 2023-10-26T01:37:06.743 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6821/4204255864", 2023-10-26T01:37:06.744 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:37:06.744 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:37:06.744 INFO:teuthology.orchestra.run.smithi169.stdout: "full", 2023-10-26T01:37:06.744 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:37:06.744 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:06.745 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:06.745 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:06.745 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 1, 2023-10-26T01:37:06.745 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:37:06.746 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:37:06.746 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:37:06.746 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:37:06.746 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:37:06.746 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:37:06.747 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:37:06.747 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:37:06.747 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 101, 2023-10-26T01:37:06.747 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:37:06.747 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:37:06.747 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:37:06.748 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:06.748 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:06.748 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:06.748 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6808", 2023-10-26T01:37:06.749 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:37:06.749 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:06.749 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:06.749 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:06.749 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6809", 2023-10-26T01:37:06.750 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:37:06.750 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:06.750 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:06.750 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:06.750 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:37:06.750 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:06.751 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:06.751 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:06.751 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6810", 2023-10-26T01:37:06.751 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:37:06.751 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:06.752 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:06.752 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:06.752 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6811", 2023-10-26T01:37:06.752 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:37:06.752 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:06.753 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:06.753 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:06.753 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:37:06.753 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:06.753 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:06.754 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:06.754 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6814", 2023-10-26T01:37:06.754 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:37:06.754 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:06.754 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:06.754 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:06.755 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6815", 2023-10-26T01:37:06.755 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:37:06.755 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:06.755 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:06.756 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:06.756 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:37:06.756 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:06.756 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:06.756 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:06.757 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6812", 2023-10-26T01:37:06.757 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:37:06.757 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:06.757 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:06.757 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:06.758 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6813", 2023-10-26T01:37:06.758 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:37:06.758 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:06.758 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:06.759 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:06.759 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6809/238206046", 2023-10-26T01:37:06.759 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6811/238206046", 2023-10-26T01:37:06.759 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6815/238206046", 2023-10-26T01:37:06.759 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6813/238206046", 2023-10-26T01:37:06.760 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:37:06.760 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:37:06.760 INFO:teuthology.orchestra.run.smithi169.stdout: "full", 2023-10-26T01:37:06.760 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:37:06.760 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:06.761 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:06.761 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:06.761 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 2, 2023-10-26T01:37:06.761 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:37:06.761 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:37:06.761 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:37:06.762 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:37:06.762 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:37:06.762 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:37:06.762 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:37:06.763 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:37:06.763 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 101, 2023-10-26T01:37:06.763 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:37:06.763 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:37:06.763 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:37:06.764 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:06.764 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:06.764 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:06.764 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6800", 2023-10-26T01:37:06.764 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:37:06.765 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:06.765 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:06.765 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:06.765 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6801", 2023-10-26T01:37:06.765 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:37:06.765 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:06.766 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:06.766 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:06.766 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:37:06.767 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:06.767 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:06.767 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:06.767 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6802", 2023-10-26T01:37:06.767 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:37:06.768 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:06.768 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:06.768 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:06.768 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6803", 2023-10-26T01:37:06.768 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:37:06.768 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:06.769 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:06.769 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:06.769 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:37:06.769 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:06.769 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:06.770 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:06.770 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6806", 2023-10-26T01:37:06.770 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:37:06.770 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:06.770 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:06.771 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:06.771 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6807", 2023-10-26T01:37:06.771 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:37:06.771 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:06.771 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:06.772 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:06.772 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:37:06.772 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:06.772 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:06.772 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:06.772 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6804", 2023-10-26T01:37:06.773 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:37:06.773 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:06.773 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:06.773 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:06.773 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6805", 2023-10-26T01:37:06.774 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:37:06.774 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:06.774 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:06.774 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:06.774 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6801/3215791846", 2023-10-26T01:37:06.775 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6803/3215791846", 2023-10-26T01:37:06.775 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6807/3215791846", 2023-10-26T01:37:06.775 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6805/3215791846", 2023-10-26T01:37:06.775 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:37:06.775 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:37:06.776 INFO:teuthology.orchestra.run.smithi169.stdout: "full", 2023-10-26T01:37:06.776 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:37:06.776 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:06.776 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:06.777 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:06.777 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 3, 2023-10-26T01:37:06.777 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:37:06.777 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:37:06.777 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:37:06.778 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:37:06.778 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:37:06.778 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:37:06.778 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:37:06.778 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 14, 2023-10-26T01:37:06.779 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 101, 2023-10-26T01:37:06.779 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:37:06.779 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:37:06.779 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:37:06.779 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:06.780 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:06.780 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:06.780 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6824", 2023-10-26T01:37:06.780 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:37:06.780 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:06.780 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:06.781 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:06.781 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6825", 2023-10-26T01:37:06.781 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:37:06.781 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:06.781 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:06.781 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:06.782 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:37:06.782 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:06.782 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:06.782 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:06.782 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6826", 2023-10-26T01:37:06.783 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:37:06.783 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:06.783 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:06.783 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:06.783 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6827", 2023-10-26T01:37:06.784 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:37:06.784 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:06.784 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:06.784 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:06.784 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:37:06.785 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:06.785 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:06.785 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:06.785 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6830", 2023-10-26T01:37:06.785 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:37:06.786 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:06.786 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:06.786 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:06.786 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6831", 2023-10-26T01:37:06.786 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:37:06.787 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:06.787 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:06.787 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:06.787 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:37:06.788 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:06.788 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:06.788 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:06.788 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6828", 2023-10-26T01:37:06.788 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:37:06.789 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:06.789 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:06.789 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:06.789 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6829", 2023-10-26T01:37:06.790 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:37:06.790 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:06.790 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:06.790 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:06.790 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6825/2256701226", 2023-10-26T01:37:06.791 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6827/2256701226", 2023-10-26T01:37:06.791 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6831/2256701226", 2023-10-26T01:37:06.791 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6829/2256701226", 2023-10-26T01:37:06.791 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:37:06.791 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:37:06.792 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:37:06.792 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:06.792 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:06.792 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:06.792 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 4, 2023-10-26T01:37:06.793 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:37:06.793 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:37:06.793 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:37:06.793 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:37:06.793 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:37:06.794 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:37:06.794 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:37:06.794 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:37:06.794 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 101, 2023-10-26T01:37:06.794 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:37:06.795 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:37:06.795 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:37:06.795 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:06.795 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:06.795 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:06.796 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6816", 2023-10-26T01:37:06.796 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:37:06.796 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:06.796 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:06.796 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:06.797 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6817", 2023-10-26T01:37:06.797 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:37:06.797 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:06.797 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:06.798 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:06.798 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:37:06.798 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:06.798 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:06.798 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:06.799 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6818", 2023-10-26T01:37:06.799 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:37:06.799 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:06.799 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:06.799 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:06.799 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6819", 2023-10-26T01:37:06.800 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:37:06.800 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:06.800 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:06.800 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:06.800 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:37:06.801 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:06.801 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:06.801 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:06.801 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6825", 2023-10-26T01:37:06.801 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:37:06.802 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:06.802 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:06.802 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:06.802 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6828", 2023-10-26T01:37:06.802 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:37:06.803 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:06.803 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:06.803 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:06.803 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:37:06.803 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:06.804 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:06.804 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:06.804 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6820", 2023-10-26T01:37:06.804 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:37:06.804 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:06.805 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:06.805 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:06.805 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6823", 2023-10-26T01:37:06.805 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:37:06.805 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:06.806 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:06.806 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:06.806 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6817/1082279874", 2023-10-26T01:37:06.806 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6819/1082279874", 2023-10-26T01:37:06.806 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6828/1082279874", 2023-10-26T01:37:06.807 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6823/1082279874", 2023-10-26T01:37:06.807 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:37:06.807 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:37:06.807 INFO:teuthology.orchestra.run.smithi169.stdout: "full", 2023-10-26T01:37:06.808 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:37:06.808 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:06.808 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:06.808 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:06.808 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 5, 2023-10-26T01:37:06.809 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:37:06.809 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:37:06.809 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:37:06.809 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:37:06.809 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:37:06.809 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:37:06.810 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:37:06.810 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:37:06.810 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 101, 2023-10-26T01:37:06.810 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:37:06.810 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:37:06.811 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:37:06.811 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:06.811 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:06.811 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:06.811 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6808", 2023-10-26T01:37:06.812 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:37:06.812 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:06.812 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:06.812 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:06.812 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6809", 2023-10-26T01:37:06.812 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:37:06.813 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:06.813 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:06.813 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:06.813 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:37:06.813 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:06.814 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:06.814 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:06.814 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6810", 2023-10-26T01:37:06.814 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:37:06.814 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:06.815 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:06.815 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:06.815 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6811", 2023-10-26T01:37:06.815 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:37:06.815 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:06.816 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:06.816 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:06.816 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:37:06.816 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:06.816 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:06.816 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:06.817 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6814", 2023-10-26T01:37:06.817 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:37:06.817 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:06.817 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:06.817 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:06.818 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6815", 2023-10-26T01:37:06.818 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:37:06.818 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:06.818 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:06.819 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:06.819 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:37:06.819 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:06.819 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:06.819 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:06.820 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6812", 2023-10-26T01:37:06.820 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:37:06.820 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:06.820 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:06.820 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:06.820 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6813", 2023-10-26T01:37:06.821 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:37:06.821 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:06.821 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:06.821 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:06.821 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6809/3335654351", 2023-10-26T01:37:06.822 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6811/3335654351", 2023-10-26T01:37:06.822 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6815/3335654351", 2023-10-26T01:37:06.822 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6813/3335654351", 2023-10-26T01:37:06.822 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:37:06.822 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:37:06.823 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:37:06.823 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:06.823 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:06.823 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:06.823 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 6, 2023-10-26T01:37:06.824 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:37:06.824 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:37:06.824 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:37:06.824 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:37:06.824 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:37:06.824 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:37:06.825 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:37:06.825 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:37:06.825 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 101, 2023-10-26T01:37:06.825 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:37:06.825 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:37:06.825 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:37:06.826 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:06.826 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:06.826 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:06.826 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6800", 2023-10-26T01:37:06.826 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:37:06.827 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:06.827 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:06.827 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:06.827 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6801", 2023-10-26T01:37:06.827 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:37:06.828 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:06.828 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:06.828 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:06.828 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:37:06.829 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:06.829 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:06.829 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:06.829 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6802", 2023-10-26T01:37:06.829 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:37:06.830 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:06.830 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:06.830 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:06.830 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6803", 2023-10-26T01:37:06.830 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:37:06.831 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:06.831 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:06.831 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:06.831 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:37:06.831 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:06.831 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:06.832 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:06.832 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6806", 2023-10-26T01:37:06.832 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:37:06.832 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:06.832 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:06.833 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:06.833 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6807", 2023-10-26T01:37:06.833 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:37:06.833 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:06.833 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:06.834 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:06.834 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:37:06.834 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:06.834 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:06.835 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:06.835 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6804", 2023-10-26T01:37:06.835 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:37:06.835 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:06.835 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:06.835 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:06.836 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6805", 2023-10-26T01:37:06.836 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:37:06.836 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:06.836 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:06.836 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:06.837 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6801/2520460697", 2023-10-26T01:37:06.837 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6803/2520460697", 2023-10-26T01:37:06.837 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6807/2520460697", 2023-10-26T01:37:06.837 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6805/2520460697", 2023-10-26T01:37:06.837 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:37:06.838 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:37:06.838 INFO:teuthology.orchestra.run.smithi169.stdout: "full", 2023-10-26T01:37:06.838 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:37:06.838 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:06.838 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:06.839 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:06.839 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 7, 2023-10-26T01:37:06.839 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:37:06.839 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:37:06.840 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:37:06.840 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:37:06.840 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:37:06.840 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:37:06.841 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:37:06.841 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:37:06.841 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 101, 2023-10-26T01:37:06.841 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:37:06.841 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:37:06.841 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:37:06.842 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:06.842 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:06.842 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:06.842 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6821", 2023-10-26T01:37:06.842 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:37:06.843 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:06.843 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:06.843 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:06.843 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6822", 2023-10-26T01:37:06.843 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:37:06.844 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:06.844 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:06.844 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:06.844 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:37:06.844 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:06.844 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:06.845 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:06.845 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6824", 2023-10-26T01:37:06.845 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:37:06.845 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:06.845 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:06.846 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:06.846 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6826", 2023-10-26T01:37:06.846 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:37:06.846 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:06.846 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:06.847 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:06.847 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:37:06.847 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:06.847 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:06.847 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:06.848 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6830", 2023-10-26T01:37:06.848 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:37:06.848 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:06.848 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:06.848 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:06.849 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6831", 2023-10-26T01:37:06.849 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:37:06.849 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:06.849 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:06.850 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:06.850 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:37:06.850 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:06.850 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:06.850 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:06.851 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6827", 2023-10-26T01:37:06.851 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:37:06.851 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:06.851 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:06.851 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:06.851 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6829", 2023-10-26T01:37:06.852 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:37:06.852 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:06.852 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:06.852 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:06.852 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6822/1115930619", 2023-10-26T01:37:06.853 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6826/1115930619", 2023-10-26T01:37:06.853 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6831/1115930619", 2023-10-26T01:37:06.853 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6829/1115930619", 2023-10-26T01:37:06.853 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:37:06.853 INFO:teuthology.orchestra.run.smithi169.stdout: "backfillfull", 2023-10-26T01:37:06.854 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:37:06.854 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:37:06.854 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:06.854 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:06.854 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:37:06.855 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_xinfo": [ 2023-10-26T01:37:06.855 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:06.855 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 0, 2023-10-26T01:37:06.855 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:37:06.855 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:37:06.856 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:37:06.856 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:37:06.856 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:37:06.856 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.889675+0000", 2023-10-26T01:37:06.856 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:37:06.857 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:06.857 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:06.857 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 1, 2023-10-26T01:37:06.857 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:37:06.857 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:37:06.858 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:37:06.858 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:37:06.858 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:37:06.858 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.864157+0000", 2023-10-26T01:37:06.859 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:37:06.859 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:06.859 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:06.859 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 2, 2023-10-26T01:37:06.860 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:37:06.860 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:37:06.860 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:37:06.860 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:37:06.860 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:37:06.860 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.876919+0000", 2023-10-26T01:37:06.861 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:37:06.861 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:06.861 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:06.861 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 3, 2023-10-26T01:37:06.861 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:37:06.862 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:37:06.862 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:37:06.862 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:37:06.862 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:37:06.862 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.974541+0000", 2023-10-26T01:37:06.863 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:37:06.863 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:06.863 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:06.863 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 4, 2023-10-26T01:37:06.863 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:37:06.863 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:37:06.864 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:37:06.864 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:37:06.864 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:37:06.864 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.969696+0000", 2023-10-26T01:37:06.864 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:37:06.865 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:06.865 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:06.865 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 5, 2023-10-26T01:37:06.865 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:37:06.865 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:37:06.866 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:37:06.866 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:37:06.866 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:37:06.866 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.859703+0000", 2023-10-26T01:37:06.866 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:37:06.866 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:06.867 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:06.867 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 6, 2023-10-26T01:37:06.867 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:37:06.867 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:37:06.867 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:37:06.867 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:37:06.868 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:37:06.868 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.867917+0000", 2023-10-26T01:37:06.868 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:37:06.868 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:06.869 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:06.869 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 7, 2023-10-26T01:37:06.869 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:37:06.869 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:37:06.869 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:37:06.869 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:37:06.870 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:37:06.870 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.898605+0000", 2023-10-26T01:37:06.870 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:37:06.870 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:06.871 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:37:06.871 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_upmap": [], 2023-10-26T01:37:06.871 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_upmap_items": [], 2023-10-26T01:37:06.871 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_upmap_primaries": [], 2023-10-26T01:37:06.871 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_temp": [], 2023-10-26T01:37:06.872 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_temp": [], 2023-10-26T01:37:06.872 INFO:teuthology.orchestra.run.smithi169.stdout: "blocklist": {}, 2023-10-26T01:37:06.872 INFO:teuthology.orchestra.run.smithi169.stdout: "range_blocklist": {}, 2023-10-26T01:37:06.872 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profiles": { 2023-10-26T01:37:06.872 INFO:teuthology.orchestra.run.smithi169.stdout: "default": { 2023-10-26T01:37:06.872 INFO:teuthology.orchestra.run.smithi169.stdout: "crush-failure-domain": "osd", 2023-10-26T01:37:06.873 INFO:teuthology.orchestra.run.smithi169.stdout: "k": "2", 2023-10-26T01:37:06.873 INFO:teuthology.orchestra.run.smithi169.stdout: "m": "1", 2023-10-26T01:37:06.873 INFO:teuthology.orchestra.run.smithi169.stdout: "plugin": "jerasure", 2023-10-26T01:37:06.873 INFO:teuthology.orchestra.run.smithi169.stdout: "technique": "reed_sol_van" 2023-10-26T01:37:06.873 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:06.873 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:06.874 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps_queue": [], 2023-10-26T01:37:06.874 INFO:teuthology.orchestra.run.smithi169.stdout: "new_removed_snaps": [], 2023-10-26T01:37:06.874 INFO:teuthology.orchestra.run.smithi169.stdout: "new_purged_snaps": [], 2023-10-26T01:37:06.874 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_node_flags": {}, 2023-10-26T01:37:06.874 INFO:teuthology.orchestra.run.smithi169.stdout: "device_class_flags": {}, 2023-10-26T01:37:06.875 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_mode": { 2023-10-26T01:37:06.875 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_mode_enabled": false, 2023-10-26T01:37:06.875 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_bucket_count": 0, 2023-10-26T01:37:06.875 INFO:teuthology.orchestra.run.smithi169.stdout: "degraded_stretch_mode": 0, 2023-10-26T01:37:06.875 INFO:teuthology.orchestra.run.smithi169.stdout: "recovering_stretch_mode": 0, 2023-10-26T01:37:06.875 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_mode_bucket": 0 2023-10-26T01:37:06.876 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:06.876 INFO:teuthology.orchestra.run.smithi169.stdout:} 2023-10-26T01:37:06.877 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:06.666+0000 7f828521a640 1 -- 172.21.15.169:0/2018803421 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f826006ae40 msgr2=0x7f826006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:06.878 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:06.666+0000 7f828521a640 1 --2- 172.21.15.169:0/2018803421 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f826006ae40 0x7f826006d300 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f8270036000 tx=0x7f8270004730 comp rx=0 tx=0).stop 2023-10-26T01:37:06.878 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:06.666+0000 7f828521a640 1 -- 172.21.15.169:0/2018803421 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f82800c96f0 msgr2=0x7f8280099dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:06.878 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:06.666+0000 7f828521a640 1 --2- 172.21.15.169:0/2018803421 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f82800c96f0 0x7f8280099dd0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f827800b990 tx=0x7f827800be50 comp rx=0 tx=0).stop 2023-10-26T01:37:06.878 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:06.666+0000 7f828521a640 1 -- 172.21.15.169:0/2018803421 shutdown_connections 2023-10-26T01:37:06.879 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:06.667+0000 7f828521a640 1 --2- 172.21.15.169:0/2018803421 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f826006ae40 0x7f826006d300 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:06.879 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:06.667+0000 7f828521a640 1 --2- 172.21.15.169:0/2018803421 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f82800c96f0 0x7f8280099dd0 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:06.879 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:06.667+0000 7f828521a640 1 --2- 172.21.15.169:0/2018803421 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f828006cae0 0x7f82800c67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:06.879 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:06.667+0000 7f828521a640 1 --2- 172.21.15.169:0/2018803421 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f828006a990 0x7f82800c6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:06.879 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:06.667+0000 7f828521a640 1 -- 172.21.15.169:0/2018803421 >> 172.21.15.169:0/2018803421 conn(0x7f82800aedd0 msgr2=0x7f82800ccda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:06.879 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:06.667+0000 7f828521a640 1 -- 172.21.15.169:0/2018803421 shutdown_connections 2023-10-26T01:37:06.880 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:06.667+0000 7f828521a640 1 -- 172.21.15.169:0/2018803421 wait complete. 2023-10-26T01:37:06.880 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T01:37:07.108 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:07.104+0000 7f4fc9c15640 1 -- 172.21.15.169:0/51542232 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4fb00047c0 msgr2=0x7f4fb0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:07.108 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:07.104+0000 7f4fc9c15640 1 --2- 172.21.15.169:0/51542232 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4fb00047c0 0x7f4fb0004c40 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f4fac00c2e0 tx=0x7f4fb00057f0 comp rx=0 tx=0).stop 2023-10-26T01:37:07.109 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:07.104+0000 7f4fc9c15640 1 -- 172.21.15.169:0/51542232 shutdown_connections 2023-10-26T01:37:07.109 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:07.104+0000 7f4fc9c15640 1 --2- 172.21.15.169:0/51542232 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4fb00047c0 0x7f4fb0004c40 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:07.110 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:07.104+0000 7f4fc9c15640 1 -- 172.21.15.169:0/51542232 >> 172.21.15.169:0/51542232 conn(0x7f4fc4069930 msgr2=0x7f4fc4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:07.110 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:07.104+0000 7f4fc9c15640 1 -- 172.21.15.169:0/51542232 shutdown_connections 2023-10-26T01:37:07.110 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:07.105+0000 7f4fc9c15640 1 -- 172.21.15.169:0/51542232 wait complete. 2023-10-26T01:37:07.115 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:07.106+0000 7f4fc9c15640 1 Processor -- start 2023-10-26T01:37:07.115 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:07.106+0000 7f4fc9c15640 1 -- start start 2023-10-26T01:37:07.116 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:07.107+0000 7f4fc9c15640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4fb00047c0 0x7f4fc4155290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:07.116 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:07.107+0000 7f4fc9c15640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4fc41522d0 0x7f4fc40703e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:07.116 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:07.107+0000 7f4fc9c15640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4fc4070920 0x7f4fc406f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:07.117 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:07.107+0000 7f4fc9c15640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc406f380 con 0x7f4fb00047c0 2023-10-26T01:37:07.117 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:07.107+0000 7f4fc9c15640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc41edac0 con 0x7f4fc4070920 2023-10-26T01:37:07.117 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:07.107+0000 7f4fc9c15640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc4152ec0 con 0x7f4fc41522d0 2023-10-26T01:37:07.118 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:07.108+0000 7f4fc37fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4fb00047c0 0x7f4fc4155290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:07.118 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:07.108+0000 7f4fc37fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4fb00047c0 0x7f4fc4155290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:59944/0 (socket says 172.21.15.169:59944) 2023-10-26T01:37:07.119 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:07.108+0000 7f4fc37fe640 1 -- 172.21.15.169:0/2134903620 learned_addr learned my addr 172.21.15.169:0/2134903620 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:37:07.119 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:07.108+0000 7f4fc37fe640 1 -- 172.21.15.169:0/2134903620 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4fc41522d0 msgr2=0x7f4fc40703e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:07.119 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:07.108+0000 7f4fc37fe640 1 --2- 172.21.15.169:0/2134903620 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4fc41522d0 0x7f4fc40703e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:07.120 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:07.108+0000 7f4fc3fff640 1 --2- 172.21.15.169:0/2134903620 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4fc4070920 0x7f4fc406f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:07.120 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:07.108+0000 7f4fc37fe640 1 -- 172.21.15.169:0/2134903620 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4fc4070920 msgr2=0x7f4fc406f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:07.121 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:07.108+0000 7f4fc37fe640 1 --2- 172.21.15.169:0/2134903620 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4fc4070920 0x7f4fc406f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:07.121 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:07.108+0000 7f4fc37fe640 1 -- 172.21.15.169:0/2134903620 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fc41531a0 con 0x7f4fb00047c0 2023-10-26T01:37:07.121 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:07.108+0000 7f4fc37fe640 1 --2- 172.21.15.169:0/2134903620 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4fb00047c0 0x7f4fc4155290 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f4fb400d5d0 tx=0x7f4fb400da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:07.122 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:07.109+0000 7f4fc0ff9640 1 -- 172.21.15.169:0/2134903620 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb400f880 con 0x7f4fb00047c0 2023-10-26T01:37:07.122 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:07.109+0000 7f4fc0ff9640 1 -- 172.21.15.169:0/2134903620 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4fb400faf0 con 0x7f4fb00047c0 2023-10-26T01:37:07.123 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:07.109+0000 7f4fc9c15640 1 -- 172.21.15.169:0/2134903620 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fc406ed30 con 0x7f4fb00047c0 2023-10-26T01:37:07.123 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:07.109+0000 7f4fc0ff9640 1 -- 172.21.15.169:0/2134903620 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb4010450 con 0x7f4fb00047c0 2023-10-26T01:37:07.123 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:07.109+0000 7f4fc9c15640 1 -- 172.21.15.169:0/2134903620 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fc4152b50 con 0x7f4fb00047c0 2023-10-26T01:37:07.124 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:07.110+0000 7f4fc9c15640 1 -- 172.21.15.169:0/2134903620 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fc404db10 con 0x7f4fb00047c0 2023-10-26T01:37:07.124 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:07.111+0000 7f4fc0ff9640 1 -- 172.21.15.169:0/2134903620 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f4fb40105b0 con 0x7f4fb00047c0 2023-10-26T01:37:07.124 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:07.112+0000 7f4fc0ff9640 1 --2- 172.21.15.169:0/2134903620 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f4fa406ae40 0x7f4fa406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:07.125 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:07.112+0000 7f4fc0ff9640 1 -- 172.21.15.169:0/2134903620 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4fb4089c50 con 0x7f4fb00047c0 2023-10-26T01:37:07.125 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:07.112+0000 7f4fc0ff9640 1 -- 172.21.15.169:0/2134903620 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=106}) v3 -- 0x7f4fa4072970 con 0x7f4fb00047c0 2023-10-26T01:37:07.125 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:07.112+0000 7f4fbbfff640 1 --2- 172.21.15.169:0/2134903620 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f4fa406ae40 0x7f4fa406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:07.126 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:07.113+0000 7f4fbbfff640 1 --2- 172.21.15.169:0/2134903620 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f4fa406ae40 0x7f4fa406d300 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f4fac034000 tx=0x7f4fac039040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:07.127 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:07.116+0000 7f4fc0ff9640 1 -- 172.21.15.169:0/2134903620 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4fb401e020 con 0x7f4fb00047c0 2023-10-26T01:37:07.343 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:07.340+0000 7f4fc9c15640 1 -- 172.21.15.169:0/2134903620 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4fc40adbf0 con 0x7f4fb00047c0 2023-10-26T01:37:07.343 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:07.341+0000 7f4fc0ff9640 1 -- 172.21.15.169:0/2134903620 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+4924 (secure 0 0 0) 0x7f4fb4014070 con 0x7f4fb00047c0 2023-10-26T01:37:07.344 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:37:07.344 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":50,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5240,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/3870332417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":3870332417},{"type":"v1","addr":"172.21.15.170:6837","nonce":3870332417}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45},{"gid":5497,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/2533771698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2533771698},{"type":"v1","addr":"172.21.15.170:6835","nonce":2533771698}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45},{"gid":6885,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/2117946713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2117946713},{"type":"v1","addr":"172.21.15.169:6834","nonce":2117946713}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T01:36:43.849046+0000","modified":"2023-10-26T01:36:46.926339+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6888},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6888":{"gid":6888,"name":"c","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6835/657142078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":657142078},{"type":"v1","addr":"172.21.15.169:6835","nonce":657142078}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-10-26T01:37:07.345 INFO:teuthology.orchestra.run.smithi169.stderr:dumped fsmap epoch 50 2023-10-26T01:37:07.347 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:07.344+0000 7f4fc9c15640 1 -- 172.21.15.169:0/2134903620 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f4fa406ae40 msgr2=0x7f4fa406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:07.347 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:07.344+0000 7f4fc9c15640 1 --2- 172.21.15.169:0/2134903620 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f4fa406ae40 0x7f4fa406d300 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f4fac034000 tx=0x7f4fac039040 comp rx=0 tx=0).stop 2023-10-26T01:37:07.347 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:07.344+0000 7f4fc9c15640 1 -- 172.21.15.169:0/2134903620 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4fb00047c0 msgr2=0x7f4fc4155290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:07.348 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:07.344+0000 7f4fc9c15640 1 --2- 172.21.15.169:0/2134903620 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4fb00047c0 0x7f4fc4155290 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f4fb400d5d0 tx=0x7f4fb400da90 comp rx=0 tx=0).stop 2023-10-26T01:37:07.348 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:07.344+0000 7f4fc9c15640 1 -- 172.21.15.169:0/2134903620 shutdown_connections 2023-10-26T01:37:07.348 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:07.344+0000 7f4fc9c15640 1 --2- 172.21.15.169:0/2134903620 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f4fa406ae40 0x7f4fa406d300 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:07.348 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:07.344+0000 7f4fc9c15640 1 --2- 172.21.15.169:0/2134903620 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4fc4070920 0x7f4fc406f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:07.348 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:07.344+0000 7f4fc9c15640 1 --2- 172.21.15.169:0/2134903620 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4fc41522d0 0x7f4fc40703e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:07.349 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:07.344+0000 7f4fc9c15640 1 --2- 172.21.15.169:0/2134903620 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4fb00047c0 0x7f4fc4155290 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:07.349 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:07.344+0000 7f4fc9c15640 1 -- 172.21.15.169:0/2134903620 >> 172.21.15.169:0/2134903620 conn(0x7f4fc4069930 msgr2=0x7f4fc40cdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:07.349 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:07.344+0000 7f4fc9c15640 1 -- 172.21.15.169:0/2134903620 shutdown_connections 2023-10-26T01:37:07.349 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:07.344+0000 7f4fc9c15640 1 -- 172.21.15.169:0/2134903620 wait complete. 2023-10-26T01:37:07.364 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2023-10-26T01:37:07.591 INFO:teuthology.orchestra.run.smithi169.stdout:{"cluster_fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","whoami":0,"id":6888,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":23.160665924,"mdsmap_epoch":50,"osdmap_epoch":105,"osdmap_epoch_barrier":105,"uptime":30.011812948999999} 2023-10-26T01:37:07.592 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "30203b28-ed8a-44e8-a9a6-a9bf25f980f1", "fs_name": "cephfs", "id": 6888, "mdsmap_epoch": 50, "osdmap_epoch": 105, "osdmap_epoch_barrier": 105, "rank_uptime": 23.160665924, "state": "up:active", "uptime": 30.011812949, "want_state": "up:active", "whoami": 0 } 2023-10-26T01:37:07.592 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-26T01:37:07.593 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:37:07.593 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:37:07.648 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:37:07.649 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=small_file_1 bs=1M conv=fdatasync count=0 seek=0) 2023-10-26T01:37:07.701 INFO:teuthology.orchestra.run.smithi169.stderr:dd: failed to open 'small_file_1': Permission denied 2023-10-26T01:37:07.702 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-26T01:37:07.703 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:37:07.703 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_b) 2023-10-26T01:37:07.756 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:37:07.757 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_a) 2023-10-26T01:37:07.892 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-26T01:37:08.106 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:08.103+0000 7f594a1a5640 1 -- 172.21.15.169:0/3756393263 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f59441592d0 msgr2=0x7f59440aa3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:08.107 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:08.103+0000 7f594a1a5640 1 --2- 172.21.15.169:0/3756393263 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f59441592d0 0x7f59440aa3d0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f5934009830 tx=0x7f593402bcd0 comp rx=0 tx=0).stop 2023-10-26T01:37:08.107 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:08.103+0000 7f594a1a5640 1 -- 172.21.15.169:0/3756393263 shutdown_connections 2023-10-26T01:37:08.107 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:08.103+0000 7f594a1a5640 1 --2- 172.21.15.169:0/3756393263 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f59440cde00 0x7f59440cbac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:08.107 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:08.103+0000 7f594a1a5640 1 --2- 172.21.15.169:0/3756393263 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f59441592d0 0x7f59440aa3d0 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:08.108 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:08.103+0000 7f594a1a5640 1 -- 172.21.15.169:0/3756393263 >> 172.21.15.169:0/3756393263 conn(0x7f5944069930 msgr2=0x7f5944069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:08.108 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:08.103+0000 7f594a1a5640 1 -- 172.21.15.169:0/3756393263 shutdown_connections 2023-10-26T01:37:08.108 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:08.103+0000 7f594a1a5640 1 -- 172.21.15.169:0/3756393263 wait complete. 2023-10-26T01:37:08.109 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:08.105+0000 7f594a1a5640 1 Processor -- start 2023-10-26T01:37:08.110 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:08.105+0000 7f594a1a5640 1 -- start start 2023-10-26T01:37:08.110 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:08.106+0000 7f594a1a5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f59440ac930 0x7f59441e4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:08.110 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:08.106+0000 7f594a1a5640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f59440cde00 0x7f5944151830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:08.110 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:08.106+0000 7f594a1a5640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f59441592d0 0x7f594415bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:08.111 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:08.106+0000 7f594a1a5640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f594415b770 con 0x7f59440cde00 2023-10-26T01:37:08.111 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:08.106+0000 7f594a1a5640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59441e48d0 con 0x7f59440ac930 2023-10-26T01:37:08.111 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:08.106+0000 7f594a1a5640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5944151d70 con 0x7f59441592d0 2023-10-26T01:37:08.111 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:08.106+0000 7f593affd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f59440cde00 0x7f5944151830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:08.111 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:08.107+0000 7f593affd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f59440cde00 0x7f5944151830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:59954/0 (socket says 172.21.15.169:59954) 2023-10-26T01:37:08.112 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:08.107+0000 7f593affd640 1 -- 172.21.15.169:0/1483228370 learned_addr learned my addr 172.21.15.169:0/1483228370 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:37:08.112 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:08.107+0000 7f5943fff640 1 --2- 172.21.15.169:0/1483228370 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f59441592d0 0x7f594415bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:08.112 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:08.107+0000 7f59437fe640 1 --2- 172.21.15.169:0/1483228370 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f59440ac930 0x7f59441e4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:08.112 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:08.107+0000 7f593affd640 1 -- 172.21.15.169:0/1483228370 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f59441592d0 msgr2=0x7f594415bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:08.112 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:08.107+0000 7f593affd640 1 --2- 172.21.15.169:0/1483228370 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f59441592d0 0x7f594415bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:08.113 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:08.107+0000 7f593affd640 1 -- 172.21.15.169:0/1483228370 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f59440ac930 msgr2=0x7f59441e4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:08.113 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:08.107+0000 7f593affd640 1 --2- 172.21.15.169:0/1483228370 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f59440ac930 0x7f59441e4390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:08.113 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:08.107+0000 7f593affd640 1 -- 172.21.15.169:0/1483228370 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5934009490 con 0x7f59440cde00 2023-10-26T01:37:08.113 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:08.107+0000 7f593affd640 1 --2- 172.21.15.169:0/1483228370 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f59440cde00 0x7f5944151830 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f592c00c490 tx=0x7f592c00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:08.115 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:08.107+0000 7f5940ff9640 1 -- 172.21.15.169:0/1483228370 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f592c012830 con 0x7f59440cde00 2023-10-26T01:37:08.115 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:08.108+0000 7f5940ff9640 1 -- 172.21.15.169:0/1483228370 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f592c012a80 con 0x7f59440cde00 2023-10-26T01:37:08.115 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:08.108+0000 7f594a1a5640 1 -- 172.21.15.169:0/1483228370 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f594415ab60 con 0x7f59440cde00 2023-10-26T01:37:08.116 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:08.108+0000 7f5940ff9640 1 -- 172.21.15.169:0/1483228370 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f592c011380 con 0x7f59440cde00 2023-10-26T01:37:08.116 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:08.108+0000 7f594a1a5640 1 -- 172.21.15.169:0/1483228370 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f594415b120 con 0x7f59440cde00 2023-10-26T01:37:08.116 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:08.109+0000 7f594a1a5640 1 -- 172.21.15.169:0/1483228370 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f594415b920 con 0x7f59440cde00 2023-10-26T01:37:08.116 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:08.110+0000 7f5940ff9640 1 -- 172.21.15.169:0/1483228370 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f592c0114e0 con 0x7f59440cde00 2023-10-26T01:37:08.116 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:08.111+0000 7f5940ff9640 1 --2- 172.21.15.169:0/1483228370 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f592406ae70 0x7f592406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:08.116 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:08.111+0000 7f5940ff9640 1 -- 172.21.15.169:0/1483228370 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f592c089a50 con 0x7f59440cde00 2023-10-26T01:37:08.117 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:08.111+0000 7f5940ff9640 1 -- 172.21.15.169:0/1483228370 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=106}) v3 -- 0x7f59240729a0 con 0x7f59440cde00 2023-10-26T01:37:08.117 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:08.111+0000 7f59437fe640 1 --2- 172.21.15.169:0/1483228370 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f592406ae70 0x7f592406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:08.117 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:08.113+0000 7f59437fe640 1 --2- 172.21.15.169:0/1483228370 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f592406ae70 0x7f592406d330 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f5934036000 tx=0x7f5934004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:08.118 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:08.115+0000 7f5940ff9640 1 -- 172.21.15.169:0/1483228370 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f592c016060 con 0x7f59440cde00 2023-10-26T01:37:08.158 INFO:tasks.ceph.mon.a.smithi169.stderr:2023-10-26T01:37:08.156+0000 7f158b279640 -1 log_channel(cluster) log [ERR] : Health check update: 4 full osd(s) (OSD_FULL) 2023-10-26T01:37:08.163 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:08.160+0000 7f5940ff9640 1 -- 172.21.15.169:0/1483228370 <== mon.0 v2:172.21.15.169:3300/0 7 ==== osd_map(106..106 src has 1..106) v4 ==== 296+0+0 (secure 0 0 0) 0x7f592c058f50 con 0x7f59440cde00 2023-10-26T01:37:08.163 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:08.160+0000 7f5940ff9640 1 -- 172.21.15.169:0/1483228370 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=107}) v3 -- 0x7f5924073250 con 0x7f59440cde00 2023-10-26T01:37:08.319 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:08.316+0000 7f594a1a5640 1 -- 172.21.15.169:0/1483228370 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f59441598d0 con 0x7f59440cde00 2023-10-26T01:37:08.322 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:08.320+0000 7f5940ff9640 1 -- 172.21.15.169:0/1483228370 <== mon.0 v2:172.21.15.169:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v106) v1 ==== 81+0+34230 (secure 0 0 0) 0x7f594415b920 con 0x7f59440cde00 2023-10-26T01:37:08.323 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:37:08.324 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2023-10-26T01:37:08.324 INFO:teuthology.orchestra.run.smithi169.stdout: "epoch": 106, 2023-10-26T01:37:08.324 INFO:teuthology.orchestra.run.smithi169.stdout: "fsid": "30203b28-ed8a-44e8-a9a6-a9bf25f980f1", 2023-10-26T01:37:08.324 INFO:teuthology.orchestra.run.smithi169.stdout: "created": "2023-10-26T01:30:22.897155+0000", 2023-10-26T01:37:08.324 INFO:teuthology.orchestra.run.smithi169.stdout: "modified": "2023-10-26T01:37:08.157670+0000", 2023-10-26T01:37:08.325 INFO:teuthology.orchestra.run.smithi169.stdout: "last_up_change": "2023-10-26T01:30:31.875281+0000", 2023-10-26T01:37:08.325 INFO:teuthology.orchestra.run.smithi169.stdout: "last_in_change": "2023-10-26T01:30:27.637941+0000", 2023-10-26T01:37:08.325 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-26T01:37:08.325 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_num": 5799936, 2023-10-26T01:37:08.325 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_set": [ 2023-10-26T01:37:08.326 INFO:teuthology.orchestra.run.smithi169.stdout: "pglog_hardlimit", 2023-10-26T01:37:08.326 INFO:teuthology.orchestra.run.smithi169.stdout: "purged_snapdirs", 2023-10-26T01:37:08.326 INFO:teuthology.orchestra.run.smithi169.stdout: "recovery_deletes", 2023-10-26T01:37:08.326 INFO:teuthology.orchestra.run.smithi169.stdout: "sortbitwise" 2023-10-26T01:37:08.327 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:37:08.327 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_version": 4, 2023-10-26T01:37:08.327 INFO:teuthology.orchestra.run.smithi169.stdout: "full_ratio": 0.69999998807907104, 2023-10-26T01:37:08.327 INFO:teuthology.orchestra.run.smithi169.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-26T01:37:08.327 INFO:teuthology.orchestra.run.smithi169.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-26T01:37:08.328 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_snapshot": "", 2023-10-26T01:37:08.328 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_max": 13, 2023-10-26T01:37:08.328 INFO:teuthology.orchestra.run.smithi169.stdout: "max_osd": 8, 2023-10-26T01:37:08.328 INFO:teuthology.orchestra.run.smithi169.stdout: "require_min_compat_client": "luminous", 2023-10-26T01:37:08.328 INFO:teuthology.orchestra.run.smithi169.stdout: "min_compat_client": "jewel", 2023-10-26T01:37:08.329 INFO:teuthology.orchestra.run.smithi169.stdout: "require_osd_release": "reef", 2023-10-26T01:37:08.329 INFO:teuthology.orchestra.run.smithi169.stdout: "allow_crimson": false, 2023-10-26T01:37:08.329 INFO:teuthology.orchestra.run.smithi169.stdout: "pools": [ 2023-10-26T01:37:08.329 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:08.329 INFO:teuthology.orchestra.run.smithi169.stdout: "pool": 1, 2023-10-26T01:37:08.330 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_name": ".mgr", 2023-10-26T01:37:08.330 INFO:teuthology.orchestra.run.smithi169.stdout: "create_time": "2023-10-26T01:30:32.200808+0000", 2023-10-26T01:37:08.330 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": 3, 2023-10-26T01:37:08.330 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_names": "hashpspool,full", 2023-10-26T01:37:08.331 INFO:teuthology.orchestra.run.smithi169.stdout: "type": 1, 2023-10-26T01:37:08.331 INFO:teuthology.orchestra.run.smithi169.stdout: "size": 2, 2023-10-26T01:37:08.331 INFO:teuthology.orchestra.run.smithi169.stdout: "min_size": 1, 2023-10-26T01:37:08.331 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_rule": 0, 2023-10-26T01:37:08.331 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_count": 0, 2023-10-26T01:37:08.332 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_target": 0, 2023-10-26T01:37:08.332 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T01:37:08.332 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T01:37:08.332 INFO:teuthology.orchestra.run.smithi169.stdout: "object_hash": 2, 2023-10-26T01:37:08.332 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_mode": "off", 2023-10-26T01:37:08.333 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num": 1, 2023-10-26T01:37:08.333 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num": 1, 2023-10-26T01:37:08.333 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num_target": 1, 2023-10-26T01:37:08.333 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_target": 1, 2023-10-26T01:37:08.333 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_pending": 1, 2023-10-26T01:37:08.334 INFO:teuthology.orchestra.run.smithi169.stdout: "last_pg_merge_meta": { 2023-10-26T01:37:08.334 INFO:teuthology.orchestra.run.smithi169.stdout: "source_pgid": "0.0", 2023-10-26T01:37:08.334 INFO:teuthology.orchestra.run.smithi169.stdout: "ready_epoch": 0, 2023-10-26T01:37:08.334 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_started": 0, 2023-10-26T01:37:08.334 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_clean": 0, 2023-10-26T01:37:08.335 INFO:teuthology.orchestra.run.smithi169.stdout: "source_version": "0'0", 2023-10-26T01:37:08.335 INFO:teuthology.orchestra.run.smithi169.stdout: "target_version": "0'0" 2023-10-26T01:37:08.335 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:08.335 INFO:teuthology.orchestra.run.smithi169.stdout: "last_change": "105", 2023-10-26T01:37:08.335 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend": "0", 2023-10-26T01:37:08.336 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T01:37:08.336 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T01:37:08.336 INFO:teuthology.orchestra.run.smithi169.stdout: "auid": 0, 2023-10-26T01:37:08.336 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_mode": "selfmanaged", 2023-10-26T01:37:08.336 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_seq": 0, 2023-10-26T01:37:08.337 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_epoch": 0, 2023-10-26T01:37:08.337 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_snaps": [], 2023-10-26T01:37:08.337 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps": "[]", 2023-10-26T01:37:08.337 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_bytes": 0, 2023-10-26T01:37:08.337 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_objects": 0, 2023-10-26T01:37:08.338 INFO:teuthology.orchestra.run.smithi169.stdout: "tiers": [], 2023-10-26T01:37:08.338 INFO:teuthology.orchestra.run.smithi169.stdout: "tier_of": -1, 2023-10-26T01:37:08.338 INFO:teuthology.orchestra.run.smithi169.stdout: "read_tier": -1, 2023-10-26T01:37:08.338 INFO:teuthology.orchestra.run.smithi169.stdout: "write_tier": -1, 2023-10-26T01:37:08.338 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_mode": "none", 2023-10-26T01:37:08.339 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_bytes": 0, 2023-10-26T01:37:08.339 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_objects": 0, 2023-10-26T01:37:08.339 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T01:37:08.340 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T01:37:08.340 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T01:37:08.340 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_flush_age": 0, 2023-10-26T01:37:08.340 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_evict_age": 0, 2023-10-26T01:37:08.341 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profile": "", 2023-10-26T01:37:08.341 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_params": { 2023-10-26T01:37:08.342 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "none" 2023-10-26T01:37:08.342 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:08.342 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_period": 0, 2023-10-26T01:37:08.343 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_count": 0, 2023-10-26T01:37:08.343 INFO:teuthology.orchestra.run.smithi169.stdout: "use_gmt_hitset": true, 2023-10-26T01:37:08.344 INFO:teuthology.orchestra.run.smithi169.stdout: "min_read_recency_for_promote": 0, 2023-10-26T01:37:08.344 INFO:teuthology.orchestra.run.smithi169.stdout: "min_write_recency_for_promote": 0, 2023-10-26T01:37:08.344 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T01:37:08.345 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_search_last_n": 0, 2023-10-26T01:37:08.345 INFO:teuthology.orchestra.run.smithi169.stdout: "grade_table": [], 2023-10-26T01:37:08.345 INFO:teuthology.orchestra.run.smithi169.stdout: "stripe_width": 0, 2023-10-26T01:37:08.346 INFO:teuthology.orchestra.run.smithi169.stdout: "expected_num_objects": 0, 2023-10-26T01:37:08.346 INFO:teuthology.orchestra.run.smithi169.stdout: "fast_read": false, 2023-10-26T01:37:08.346 INFO:teuthology.orchestra.run.smithi169.stdout: "options": { 2023-10-26T01:37:08.347 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_max": 32, 2023-10-26T01:37:08.347 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_min": 1 2023-10-26T01:37:08.347 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:08.347 INFO:teuthology.orchestra.run.smithi169.stdout: "application_metadata": { 2023-10-26T01:37:08.348 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": {} 2023-10-26T01:37:08.348 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:08.348 INFO:teuthology.orchestra.run.smithi169.stdout: "read_balance": { 2023-10-26T01:37:08.349 INFO:teuthology.orchestra.run.smithi169.stdout: "score_acting": 8, 2023-10-26T01:37:08.349 INFO:teuthology.orchestra.run.smithi169.stdout: "score_stable": 8, 2023-10-26T01:37:08.349 INFO:teuthology.orchestra.run.smithi169.stdout: "optimal_score": 0.25, 2023-10-26T01:37:08.349 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_acting": 2, 2023-10-26T01:37:08.349 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_stable": 2, 2023-10-26T01:37:08.350 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity_weighted": 1, 2023-10-26T01:37:08.350 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity": 1, 2023-10-26T01:37:08.350 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity_weighted": 1 2023-10-26T01:37:08.350 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:08.350 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:08.351 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:08.351 INFO:teuthology.orchestra.run.smithi169.stdout: "pool": 12, 2023-10-26T01:37:08.351 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_name": "cephfs_metadata", 2023-10-26T01:37:08.351 INFO:teuthology.orchestra.run.smithi169.stdout: "create_time": "2023-10-26T01:36:41.430455+0000", 2023-10-26T01:37:08.351 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": 3, 2023-10-26T01:37:08.352 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_names": "hashpspool,full", 2023-10-26T01:37:08.352 INFO:teuthology.orchestra.run.smithi169.stdout: "type": 1, 2023-10-26T01:37:08.352 INFO:teuthology.orchestra.run.smithi169.stdout: "size": 2, 2023-10-26T01:37:08.352 INFO:teuthology.orchestra.run.smithi169.stdout: "min_size": 1, 2023-10-26T01:37:08.353 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_rule": 0, 2023-10-26T01:37:08.353 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_count": 0, 2023-10-26T01:37:08.353 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_target": 0, 2023-10-26T01:37:08.354 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T01:37:08.354 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T01:37:08.354 INFO:teuthology.orchestra.run.smithi169.stdout: "object_hash": 2, 2023-10-26T01:37:08.355 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_mode": "off", 2023-10-26T01:37:08.355 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num": 32, 2023-10-26T01:37:08.355 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num": 32, 2023-10-26T01:37:08.356 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num_target": 32, 2023-10-26T01:37:08.356 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_target": 32, 2023-10-26T01:37:08.356 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_pending": 32, 2023-10-26T01:37:08.357 INFO:teuthology.orchestra.run.smithi169.stdout: "last_pg_merge_meta": { 2023-10-26T01:37:08.357 INFO:teuthology.orchestra.run.smithi169.stdout: "source_pgid": "0.0", 2023-10-26T01:37:08.357 INFO:teuthology.orchestra.run.smithi169.stdout: "ready_epoch": 0, 2023-10-26T01:37:08.358 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_started": 0, 2023-10-26T01:37:08.358 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_clean": 0, 2023-10-26T01:37:08.358 INFO:teuthology.orchestra.run.smithi169.stdout: "source_version": "0'0", 2023-10-26T01:37:08.359 INFO:teuthology.orchestra.run.smithi169.stdout: "target_version": "0'0" 2023-10-26T01:37:08.359 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:08.359 INFO:teuthology.orchestra.run.smithi169.stdout: "last_change": "105", 2023-10-26T01:37:08.360 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend": "0", 2023-10-26T01:37:08.360 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T01:37:08.360 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T01:37:08.360 INFO:teuthology.orchestra.run.smithi169.stdout: "auid": 0, 2023-10-26T01:37:08.360 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_mode": "selfmanaged", 2023-10-26T01:37:08.361 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_seq": 0, 2023-10-26T01:37:08.361 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_epoch": 0, 2023-10-26T01:37:08.361 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_snaps": [], 2023-10-26T01:37:08.361 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps": "[]", 2023-10-26T01:37:08.361 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_bytes": 0, 2023-10-26T01:37:08.361 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_objects": 0, 2023-10-26T01:37:08.362 INFO:teuthology.orchestra.run.smithi169.stdout: "tiers": [], 2023-10-26T01:37:08.362 INFO:teuthology.orchestra.run.smithi169.stdout: "tier_of": -1, 2023-10-26T01:37:08.362 INFO:teuthology.orchestra.run.smithi169.stdout: "read_tier": -1, 2023-10-26T01:37:08.362 INFO:teuthology.orchestra.run.smithi169.stdout: "write_tier": -1, 2023-10-26T01:37:08.363 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_mode": "none", 2023-10-26T01:37:08.363 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_bytes": 0, 2023-10-26T01:37:08.363 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_objects": 0, 2023-10-26T01:37:08.363 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T01:37:08.363 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T01:37:08.364 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T01:37:08.364 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_flush_age": 0, 2023-10-26T01:37:08.364 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_evict_age": 0, 2023-10-26T01:37:08.364 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profile": "", 2023-10-26T01:37:08.364 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_params": { 2023-10-26T01:37:08.365 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "none" 2023-10-26T01:37:08.365 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:08.365 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_period": 0, 2023-10-26T01:37:08.365 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_count": 0, 2023-10-26T01:37:08.365 INFO:teuthology.orchestra.run.smithi169.stdout: "use_gmt_hitset": true, 2023-10-26T01:37:08.366 INFO:teuthology.orchestra.run.smithi169.stdout: "min_read_recency_for_promote": 0, 2023-10-26T01:37:08.366 INFO:teuthology.orchestra.run.smithi169.stdout: "min_write_recency_for_promote": 0, 2023-10-26T01:37:08.366 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T01:37:08.366 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_search_last_n": 0, 2023-10-26T01:37:08.366 INFO:teuthology.orchestra.run.smithi169.stdout: "grade_table": [], 2023-10-26T01:37:08.367 INFO:teuthology.orchestra.run.smithi169.stdout: "stripe_width": 0, 2023-10-26T01:37:08.367 INFO:teuthology.orchestra.run.smithi169.stdout: "expected_num_objects": 0, 2023-10-26T01:37:08.367 INFO:teuthology.orchestra.run.smithi169.stdout: "fast_read": false, 2023-10-26T01:37:08.367 INFO:teuthology.orchestra.run.smithi169.stdout: "options": { 2023-10-26T01:37:08.367 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_bias": 4, 2023-10-26T01:37:08.368 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_min": 16, 2023-10-26T01:37:08.368 INFO:teuthology.orchestra.run.smithi169.stdout: "recovery_priority": 5 2023-10-26T01:37:08.368 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:08.368 INFO:teuthology.orchestra.run.smithi169.stdout: "application_metadata": { 2023-10-26T01:37:08.368 INFO:teuthology.orchestra.run.smithi169.stdout: "cephfs": { 2023-10-26T01:37:08.368 INFO:teuthology.orchestra.run.smithi169.stdout: "metadata": "cephfs" 2023-10-26T01:37:08.369 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:08.369 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:08.369 INFO:teuthology.orchestra.run.smithi169.stdout: "read_balance": { 2023-10-26T01:37:08.369 INFO:teuthology.orchestra.run.smithi169.stdout: "score_acting": 2, 2023-10-26T01:37:08.369 INFO:teuthology.orchestra.run.smithi169.stdout: "score_stable": 2, 2023-10-26T01:37:08.370 INFO:teuthology.orchestra.run.smithi169.stdout: "optimal_score": 1, 2023-10-26T01:37:08.370 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_acting": 2, 2023-10-26T01:37:08.370 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_stable": 2, 2023-10-26T01:37:08.370 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity_weighted": 1, 2023-10-26T01:37:08.370 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity": 1, 2023-10-26T01:37:08.371 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity_weighted": 1 2023-10-26T01:37:08.371 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:08.371 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:08.371 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:08.371 INFO:teuthology.orchestra.run.smithi169.stdout: "pool": 13, 2023-10-26T01:37:08.371 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_name": "cephfs_data", 2023-10-26T01:37:08.372 INFO:teuthology.orchestra.run.smithi169.stdout: "create_time": "2023-10-26T01:36:42.883252+0000", 2023-10-26T01:37:08.372 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": 3, 2023-10-26T01:37:08.372 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_names": "hashpspool,full", 2023-10-26T01:37:08.372 INFO:teuthology.orchestra.run.smithi169.stdout: "type": 1, 2023-10-26T01:37:08.372 INFO:teuthology.orchestra.run.smithi169.stdout: "size": 2, 2023-10-26T01:37:08.373 INFO:teuthology.orchestra.run.smithi169.stdout: "min_size": 1, 2023-10-26T01:37:08.373 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_rule": 0, 2023-10-26T01:37:08.373 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_count": 0, 2023-10-26T01:37:08.373 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_target": 0, 2023-10-26T01:37:08.373 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T01:37:08.374 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T01:37:08.374 INFO:teuthology.orchestra.run.smithi169.stdout: "object_hash": 2, 2023-10-26T01:37:08.374 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_mode": "off", 2023-10-26T01:37:08.374 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num": 64, 2023-10-26T01:37:08.374 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num": 64, 2023-10-26T01:37:08.375 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num_target": 64, 2023-10-26T01:37:08.375 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_target": 64, 2023-10-26T01:37:08.375 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_pending": 64, 2023-10-26T01:37:08.375 INFO:teuthology.orchestra.run.smithi169.stdout: "last_pg_merge_meta": { 2023-10-26T01:37:08.375 INFO:teuthology.orchestra.run.smithi169.stdout: "source_pgid": "0.0", 2023-10-26T01:37:08.375 INFO:teuthology.orchestra.run.smithi169.stdout: "ready_epoch": 0, 2023-10-26T01:37:08.376 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_started": 0, 2023-10-26T01:37:08.376 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_clean": 0, 2023-10-26T01:37:08.376 INFO:teuthology.orchestra.run.smithi169.stdout: "source_version": "0'0", 2023-10-26T01:37:08.376 INFO:teuthology.orchestra.run.smithi169.stdout: "target_version": "0'0" 2023-10-26T01:37:08.376 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:08.377 INFO:teuthology.orchestra.run.smithi169.stdout: "last_change": "105", 2023-10-26T01:37:08.377 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend": "0", 2023-10-26T01:37:08.377 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T01:37:08.377 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T01:37:08.377 INFO:teuthology.orchestra.run.smithi169.stdout: "auid": 0, 2023-10-26T01:37:08.378 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_mode": "selfmanaged", 2023-10-26T01:37:08.378 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_seq": 0, 2023-10-26T01:37:08.378 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_epoch": 0, 2023-10-26T01:37:08.378 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_snaps": [], 2023-10-26T01:37:08.378 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps": "[]", 2023-10-26T01:37:08.379 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_bytes": 0, 2023-10-26T01:37:08.379 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_objects": 0, 2023-10-26T01:37:08.379 INFO:teuthology.orchestra.run.smithi169.stdout: "tiers": [], 2023-10-26T01:37:08.379 INFO:teuthology.orchestra.run.smithi169.stdout: "tier_of": -1, 2023-10-26T01:37:08.379 INFO:teuthology.orchestra.run.smithi169.stdout: "read_tier": -1, 2023-10-26T01:37:08.380 INFO:teuthology.orchestra.run.smithi169.stdout: "write_tier": -1, 2023-10-26T01:37:08.380 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_mode": "none", 2023-10-26T01:37:08.380 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_bytes": 0, 2023-10-26T01:37:08.380 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_objects": 0, 2023-10-26T01:37:08.380 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T01:37:08.381 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T01:37:08.381 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T01:37:08.381 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_flush_age": 0, 2023-10-26T01:37:08.381 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_evict_age": 0, 2023-10-26T01:37:08.381 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profile": "", 2023-10-26T01:37:08.381 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_params": { 2023-10-26T01:37:08.382 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "none" 2023-10-26T01:37:08.382 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:08.382 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_period": 0, 2023-10-26T01:37:08.382 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_count": 0, 2023-10-26T01:37:08.383 INFO:teuthology.orchestra.run.smithi169.stdout: "use_gmt_hitset": true, 2023-10-26T01:37:08.383 INFO:teuthology.orchestra.run.smithi169.stdout: "min_read_recency_for_promote": 0, 2023-10-26T01:37:08.383 INFO:teuthology.orchestra.run.smithi169.stdout: "min_write_recency_for_promote": 0, 2023-10-26T01:37:08.383 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T01:37:08.383 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_search_last_n": 0, 2023-10-26T01:37:08.384 INFO:teuthology.orchestra.run.smithi169.stdout: "grade_table": [], 2023-10-26T01:37:08.384 INFO:teuthology.orchestra.run.smithi169.stdout: "stripe_width": 0, 2023-10-26T01:37:08.384 INFO:teuthology.orchestra.run.smithi169.stdout: "expected_num_objects": 0, 2023-10-26T01:37:08.384 INFO:teuthology.orchestra.run.smithi169.stdout: "fast_read": false, 2023-10-26T01:37:08.384 INFO:teuthology.orchestra.run.smithi169.stdout: "options": { 2023-10-26T01:37:08.385 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_min": 64, 2023-10-26T01:37:08.385 INFO:teuthology.orchestra.run.smithi169.stdout: "target_size_ratio": 0.89999997615814209 2023-10-26T01:37:08.385 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:08.385 INFO:teuthology.orchestra.run.smithi169.stdout: "application_metadata": { 2023-10-26T01:37:08.385 INFO:teuthology.orchestra.run.smithi169.stdout: "cephfs": { 2023-10-26T01:37:08.386 INFO:teuthology.orchestra.run.smithi169.stdout: "data": "cephfs" 2023-10-26T01:37:08.386 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:08.386 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:08.386 INFO:teuthology.orchestra.run.smithi169.stdout: "read_balance": { 2023-10-26T01:37:08.387 INFO:teuthology.orchestra.run.smithi169.stdout: "score_acting": 1.3799999952316284, 2023-10-26T01:37:08.387 INFO:teuthology.orchestra.run.smithi169.stdout: "score_stable": 1.3799999952316284, 2023-10-26T01:37:08.387 INFO:teuthology.orchestra.run.smithi169.stdout: "optimal_score": 1, 2023-10-26T01:37:08.387 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_acting": 1.3799999952316284, 2023-10-26T01:37:08.388 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_stable": 1.3799999952316284, 2023-10-26T01:37:08.388 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity_weighted": 1, 2023-10-26T01:37:08.388 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity": 1, 2023-10-26T01:37:08.388 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity_weighted": 1 2023-10-26T01:37:08.388 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:08.388 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:08.389 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:37:08.389 INFO:teuthology.orchestra.run.smithi169.stdout: "osds": [ 2023-10-26T01:37:08.389 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:08.389 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 0, 2023-10-26T01:37:08.389 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:37:08.390 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:37:08.390 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:37:08.390 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:37:08.390 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:37:08.390 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:37:08.391 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:37:08.391 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:37:08.391 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 101, 2023-10-26T01:37:08.391 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:37:08.391 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:37:08.392 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:37:08.392 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:08.392 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:08.392 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:08.392 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6816", 2023-10-26T01:37:08.393 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:37:08.393 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:08.393 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:08.393 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:08.393 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6817", 2023-10-26T01:37:08.393 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:37:08.394 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:08.394 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:08.394 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:08.394 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:37:08.394 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:08.395 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:08.395 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:08.395 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6818", 2023-10-26T01:37:08.395 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:37:08.395 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:08.396 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:08.396 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:08.396 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6819", 2023-10-26T01:37:08.396 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:37:08.396 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:08.397 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:08.397 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:08.397 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:37:08.397 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:08.397 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:08.397 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:08.398 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6822", 2023-10-26T01:37:08.398 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:37:08.398 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:08.398 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:08.398 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:08.399 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6823", 2023-10-26T01:37:08.399 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:37:08.399 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:08.399 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:08.399 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:08.399 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:37:08.400 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:08.400 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:08.400 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:08.400 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6820", 2023-10-26T01:37:08.400 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:37:08.400 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:08.401 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:08.401 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:08.401 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6821", 2023-10-26T01:37:08.401 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:37:08.401 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:08.401 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:08.402 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:08.402 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6817/4204255864", 2023-10-26T01:37:08.402 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6819/4204255864", 2023-10-26T01:37:08.402 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6823/4204255864", 2023-10-26T01:37:08.402 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6821/4204255864", 2023-10-26T01:37:08.403 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:37:08.403 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:37:08.403 INFO:teuthology.orchestra.run.smithi169.stdout: "full", 2023-10-26T01:37:08.403 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:37:08.403 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:08.404 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:08.404 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:08.404 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 1, 2023-10-26T01:37:08.404 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:37:08.404 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:37:08.404 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:37:08.405 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:37:08.405 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:37:08.405 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:37:08.405 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:37:08.405 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:37:08.406 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 101, 2023-10-26T01:37:08.406 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:37:08.406 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:37:08.406 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:37:08.406 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:08.407 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:08.407 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:08.407 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6808", 2023-10-26T01:37:08.407 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:37:08.407 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:08.408 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:08.408 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:08.408 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6809", 2023-10-26T01:37:08.408 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:37:08.408 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:08.408 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:08.409 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:08.409 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:37:08.409 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:08.409 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:08.409 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:08.409 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6810", 2023-10-26T01:37:08.410 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:37:08.410 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:08.410 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:08.410 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:08.410 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6811", 2023-10-26T01:37:08.411 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:37:08.411 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:08.411 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:08.411 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:08.411 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:37:08.411 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:08.412 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:08.412 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:08.412 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6814", 2023-10-26T01:37:08.412 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:37:08.412 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:08.412 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:08.413 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:08.413 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6815", 2023-10-26T01:37:08.413 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:37:08.413 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:08.413 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:08.414 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:08.414 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:37:08.414 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:08.414 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:08.414 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:08.415 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6812", 2023-10-26T01:37:08.415 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:37:08.415 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:08.415 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:08.415 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:08.415 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6813", 2023-10-26T01:37:08.416 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:37:08.416 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:08.416 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:08.416 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:08.416 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6809/238206046", 2023-10-26T01:37:08.417 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6811/238206046", 2023-10-26T01:37:08.417 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6815/238206046", 2023-10-26T01:37:08.417 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6813/238206046", 2023-10-26T01:37:08.417 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:37:08.418 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:37:08.418 INFO:teuthology.orchestra.run.smithi169.stdout: "full", 2023-10-26T01:37:08.418 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:37:08.419 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:08.419 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:08.419 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:08.420 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 2, 2023-10-26T01:37:08.420 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:37:08.421 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:37:08.421 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:37:08.421 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:37:08.422 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:37:08.422 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:37:08.422 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:37:08.423 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:37:08.423 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 101, 2023-10-26T01:37:08.423 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:37:08.424 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:37:08.424 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:37:08.425 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:08.425 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:08.425 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:08.426 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6800", 2023-10-26T01:37:08.426 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:37:08.426 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:08.427 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:08.427 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:08.427 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6801", 2023-10-26T01:37:08.427 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:37:08.428 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:08.428 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:08.428 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:08.428 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:37:08.428 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:08.429 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:08.429 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:08.429 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6802", 2023-10-26T01:37:08.429 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:37:08.429 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:08.430 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:08.430 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:08.430 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6803", 2023-10-26T01:37:08.430 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:37:08.430 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:08.431 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:08.431 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:08.431 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:37:08.431 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:08.431 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:08.431 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:08.432 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6806", 2023-10-26T01:37:08.432 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:37:08.432 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:08.432 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:08.432 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:08.432 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6807", 2023-10-26T01:37:08.433 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:37:08.433 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:08.433 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:08.433 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:08.433 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:37:08.434 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:08.434 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:08.434 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:08.434 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6804", 2023-10-26T01:37:08.435 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:37:08.435 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:08.435 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:08.435 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:08.435 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6805", 2023-10-26T01:37:08.435 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:37:08.436 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:08.436 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:08.436 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:08.436 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6801/3215791846", 2023-10-26T01:37:08.436 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6803/3215791846", 2023-10-26T01:37:08.436 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6807/3215791846", 2023-10-26T01:37:08.437 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6805/3215791846", 2023-10-26T01:37:08.437 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:37:08.437 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:37:08.437 INFO:teuthology.orchestra.run.smithi169.stdout: "full", 2023-10-26T01:37:08.437 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:37:08.437 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:08.438 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:08.438 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:08.438 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 3, 2023-10-26T01:37:08.438 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:37:08.438 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:37:08.439 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:37:08.439 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:37:08.439 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:37:08.439 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:37:08.439 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:37:08.439 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 14, 2023-10-26T01:37:08.440 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 101, 2023-10-26T01:37:08.440 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:37:08.440 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:37:08.440 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:37:08.440 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:08.440 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:08.441 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:08.441 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6824", 2023-10-26T01:37:08.441 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:37:08.441 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:08.441 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:08.441 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:08.442 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6825", 2023-10-26T01:37:08.442 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:37:08.442 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:08.442 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:08.442 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:08.443 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:37:08.443 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:08.443 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:08.443 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:08.443 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6826", 2023-10-26T01:37:08.444 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:37:08.444 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:08.444 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:08.444 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:08.444 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6827", 2023-10-26T01:37:08.445 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:37:08.445 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:08.445 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:08.445 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:08.445 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:37:08.445 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:08.446 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:08.446 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:08.446 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6830", 2023-10-26T01:37:08.446 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:37:08.446 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:08.447 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:08.447 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:08.447 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6831", 2023-10-26T01:37:08.447 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:37:08.447 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:08.447 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:08.448 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:08.448 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:37:08.448 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:08.448 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:08.448 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:08.448 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6828", 2023-10-26T01:37:08.449 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:37:08.449 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:08.449 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:08.449 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:08.449 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6829", 2023-10-26T01:37:08.450 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:37:08.450 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:08.450 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:08.450 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:08.450 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6825/2256701226", 2023-10-26T01:37:08.451 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6827/2256701226", 2023-10-26T01:37:08.451 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6831/2256701226", 2023-10-26T01:37:08.451 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6829/2256701226", 2023-10-26T01:37:08.451 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:37:08.451 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:37:08.451 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:37:08.452 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:08.452 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:08.452 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:08.452 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 4, 2023-10-26T01:37:08.452 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:37:08.452 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:37:08.453 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:37:08.453 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:37:08.453 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:37:08.453 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:37:08.453 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:37:08.454 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:37:08.454 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 101, 2023-10-26T01:37:08.454 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:37:08.454 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:37:08.454 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:37:08.455 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:08.455 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:08.455 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:08.455 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6816", 2023-10-26T01:37:08.455 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:37:08.456 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:08.456 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:08.456 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:08.456 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6817", 2023-10-26T01:37:08.456 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:37:08.456 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:08.457 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:08.457 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:08.457 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:37:08.457 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:08.457 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:08.458 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:08.458 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6818", 2023-10-26T01:37:08.458 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:37:08.458 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:08.458 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:08.459 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:08.459 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6819", 2023-10-26T01:37:08.459 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:37:08.459 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:08.459 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:08.459 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:08.460 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:37:08.460 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:08.460 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:08.460 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:08.460 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6825", 2023-10-26T01:37:08.461 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:37:08.461 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:08.461 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:08.461 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:08.461 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6828", 2023-10-26T01:37:08.461 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:37:08.462 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:08.462 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:08.462 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:08.462 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:37:08.462 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:08.462 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:08.463 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:08.463 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6820", 2023-10-26T01:37:08.463 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:37:08.463 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:08.463 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:08.463 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:08.464 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6823", 2023-10-26T01:37:08.464 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:37:08.464 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:08.464 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:08.464 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:08.465 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6817/1082279874", 2023-10-26T01:37:08.465 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6819/1082279874", 2023-10-26T01:37:08.465 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6828/1082279874", 2023-10-26T01:37:08.465 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6823/1082279874", 2023-10-26T01:37:08.465 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:37:08.466 INFO:teuthology.orchestra.run.smithi169.stdout: "backfillfull", 2023-10-26T01:37:08.466 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:37:08.466 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:37:08.466 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:08.466 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:08.466 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:08.467 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 5, 2023-10-26T01:37:08.467 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:37:08.467 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:37:08.467 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:37:08.467 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:37:08.467 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:37:08.468 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:37:08.468 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:37:08.468 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:37:08.468 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 101, 2023-10-26T01:37:08.468 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:37:08.469 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:37:08.469 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:37:08.469 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:08.469 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:08.469 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:08.469 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6808", 2023-10-26T01:37:08.470 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:37:08.470 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:08.470 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:08.470 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:08.470 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6809", 2023-10-26T01:37:08.470 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:37:08.471 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:08.471 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:08.471 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:08.471 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:37:08.471 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:08.472 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:08.472 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:08.472 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6810", 2023-10-26T01:37:08.472 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:37:08.472 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:08.472 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:08.473 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:08.473 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6811", 2023-10-26T01:37:08.473 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:37:08.473 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:08.473 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:08.473 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:08.474 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:37:08.474 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:08.474 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:08.474 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:08.474 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6814", 2023-10-26T01:37:08.474 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:37:08.475 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:08.475 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:08.475 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:08.475 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6815", 2023-10-26T01:37:08.475 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:37:08.475 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:08.476 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:08.476 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:08.476 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:37:08.476 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:08.476 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:08.476 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:08.477 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6812", 2023-10-26T01:37:08.477 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:37:08.477 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:08.477 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:08.477 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:08.477 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6813", 2023-10-26T01:37:08.478 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:37:08.478 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:08.478 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:08.478 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:08.478 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6809/3335654351", 2023-10-26T01:37:08.479 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6811/3335654351", 2023-10-26T01:37:08.479 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6815/3335654351", 2023-10-26T01:37:08.479 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6813/3335654351", 2023-10-26T01:37:08.479 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:37:08.479 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:37:08.479 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:37:08.480 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:08.480 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:08.480 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:08.480 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 6, 2023-10-26T01:37:08.480 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:37:08.480 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:37:08.481 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:37:08.481 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:37:08.481 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:37:08.481 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:37:08.481 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:37:08.481 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:37:08.482 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 101, 2023-10-26T01:37:08.482 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:37:08.482 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:37:08.482 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:37:08.482 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:08.483 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:08.483 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:08.483 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6800", 2023-10-26T01:37:08.483 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:37:08.483 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:08.483 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:08.484 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:08.484 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6801", 2023-10-26T01:37:08.484 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:37:08.484 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:08.484 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:08.485 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:08.485 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:37:08.485 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:08.485 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:08.485 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:08.486 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6802", 2023-10-26T01:37:08.486 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:37:08.486 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:08.486 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:08.487 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:08.487 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6803", 2023-10-26T01:37:08.487 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:37:08.487 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:08.487 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:08.488 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:08.488 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:37:08.488 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:08.488 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:08.488 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:08.489 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6806", 2023-10-26T01:37:08.489 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:37:08.489 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:08.489 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:08.489 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:08.490 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6807", 2023-10-26T01:37:08.490 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:37:08.490 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:08.490 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:08.490 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:08.491 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:37:08.491 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:08.491 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:08.491 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:08.491 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6804", 2023-10-26T01:37:08.492 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:37:08.492 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:08.492 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:08.493 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:08.493 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6805", 2023-10-26T01:37:08.493 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:37:08.493 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:08.493 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:08.494 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:08.494 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6801/2520460697", 2023-10-26T01:37:08.494 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6803/2520460697", 2023-10-26T01:37:08.494 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6807/2520460697", 2023-10-26T01:37:08.494 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6805/2520460697", 2023-10-26T01:37:08.495 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:37:08.495 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:37:08.495 INFO:teuthology.orchestra.run.smithi169.stdout: "full", 2023-10-26T01:37:08.495 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:37:08.495 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:08.496 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:08.496 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:08.496 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 7, 2023-10-26T01:37:08.496 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:37:08.496 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:37:08.497 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:37:08.497 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:37:08.497 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:37:08.497 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:37:08.497 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:37:08.497 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:37:08.498 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 101, 2023-10-26T01:37:08.498 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:37:08.498 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:37:08.498 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:37:08.499 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:08.499 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:08.499 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:08.499 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6821", 2023-10-26T01:37:08.499 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:37:08.499 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:08.500 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:08.500 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:08.500 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6822", 2023-10-26T01:37:08.500 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:37:08.500 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:08.501 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:08.501 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:08.501 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:37:08.501 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:08.501 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:08.501 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:08.502 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6824", 2023-10-26T01:37:08.502 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:37:08.502 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:08.502 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:08.502 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:08.503 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6826", 2023-10-26T01:37:08.503 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:37:08.503 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:08.503 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:08.503 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:08.504 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:37:08.504 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:08.504 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:08.504 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:08.504 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6830", 2023-10-26T01:37:08.504 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:37:08.505 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:08.505 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:08.505 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:08.505 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6831", 2023-10-26T01:37:08.505 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:37:08.505 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:08.506 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:08.506 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:08.506 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:37:08.506 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:08.506 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:08.507 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:08.507 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6827", 2023-10-26T01:37:08.507 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:37:08.508 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:08.508 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:08.508 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:08.508 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6829", 2023-10-26T01:37:08.508 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:37:08.508 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:08.509 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:08.509 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:08.509 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6822/1115930619", 2023-10-26T01:37:08.509 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6826/1115930619", 2023-10-26T01:37:08.509 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6831/1115930619", 2023-10-26T01:37:08.509 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6829/1115930619", 2023-10-26T01:37:08.510 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:37:08.510 INFO:teuthology.orchestra.run.smithi169.stdout: "backfillfull", 2023-10-26T01:37:08.510 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:37:08.510 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:37:08.510 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:08.511 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:08.511 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:37:08.511 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_xinfo": [ 2023-10-26T01:37:08.511 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:08.511 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 0, 2023-10-26T01:37:08.511 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:37:08.512 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:37:08.512 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:37:08.512 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:37:08.512 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:37:08.512 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.889675+0000", 2023-10-26T01:37:08.513 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:37:08.513 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:08.513 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:08.513 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 1, 2023-10-26T01:37:08.513 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:37:08.513 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:37:08.514 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:37:08.514 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:37:08.514 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:37:08.514 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.864157+0000", 2023-10-26T01:37:08.515 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:37:08.515 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:08.515 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:08.515 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 2, 2023-10-26T01:37:08.515 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:37:08.516 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:37:08.516 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:37:08.516 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:37:08.516 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:37:08.517 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.876919+0000", 2023-10-26T01:37:08.517 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:37:08.517 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:08.517 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:08.517 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 3, 2023-10-26T01:37:08.518 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:37:08.518 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:37:08.518 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:37:08.518 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:37:08.519 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:37:08.519 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.974541+0000", 2023-10-26T01:37:08.519 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:37:08.519 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:08.519 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:08.520 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 4, 2023-10-26T01:37:08.520 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:37:08.520 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:37:08.520 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:37:08.520 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:37:08.520 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:37:08.521 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.969696+0000", 2023-10-26T01:37:08.521 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:37:08.521 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:08.521 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:08.521 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 5, 2023-10-26T01:37:08.522 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:37:08.522 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:37:08.522 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:37:08.522 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:37:08.522 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:37:08.523 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.859703+0000", 2023-10-26T01:37:08.523 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:37:08.523 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:08.523 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:08.523 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 6, 2023-10-26T01:37:08.524 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:37:08.524 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:37:08.524 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:37:08.524 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:37:08.524 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:37:08.524 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.867917+0000", 2023-10-26T01:37:08.525 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:37:08.525 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:08.525 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:08.525 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 7, 2023-10-26T01:37:08.525 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:37:08.526 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:37:08.526 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:37:08.526 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:37:08.526 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:37:08.527 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.898605+0000", 2023-10-26T01:37:08.527 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:37:08.527 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:08.527 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:37:08.527 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_upmap": [], 2023-10-26T01:37:08.528 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_upmap_items": [], 2023-10-26T01:37:08.528 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_upmap_primaries": [], 2023-10-26T01:37:08.528 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_temp": [], 2023-10-26T01:37:08.529 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_temp": [], 2023-10-26T01:37:08.529 INFO:teuthology.orchestra.run.smithi169.stdout: "blocklist": {}, 2023-10-26T01:37:08.529 INFO:teuthology.orchestra.run.smithi169.stdout: "range_blocklist": {}, 2023-10-26T01:37:08.529 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profiles": { 2023-10-26T01:37:08.529 INFO:teuthology.orchestra.run.smithi169.stdout: "default": { 2023-10-26T01:37:08.530 INFO:teuthology.orchestra.run.smithi169.stdout: "crush-failure-domain": "osd", 2023-10-26T01:37:08.530 INFO:teuthology.orchestra.run.smithi169.stdout: "k": "2", 2023-10-26T01:37:08.530 INFO:teuthology.orchestra.run.smithi169.stdout: "m": "1", 2023-10-26T01:37:08.530 INFO:teuthology.orchestra.run.smithi169.stdout: "plugin": "jerasure", 2023-10-26T01:37:08.530 INFO:teuthology.orchestra.run.smithi169.stdout: "technique": "reed_sol_van" 2023-10-26T01:37:08.531 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:08.531 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:08.531 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps_queue": [], 2023-10-26T01:37:08.531 INFO:teuthology.orchestra.run.smithi169.stdout: "new_removed_snaps": [], 2023-10-26T01:37:08.531 INFO:teuthology.orchestra.run.smithi169.stdout: "new_purged_snaps": [], 2023-10-26T01:37:08.532 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_node_flags": {}, 2023-10-26T01:37:08.532 INFO:teuthology.orchestra.run.smithi169.stdout: "device_class_flags": {}, 2023-10-26T01:37:08.532 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_mode": { 2023-10-26T01:37:08.532 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_mode_enabled": false, 2023-10-26T01:37:08.532 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_bucket_count": 0, 2023-10-26T01:37:08.532 INFO:teuthology.orchestra.run.smithi169.stdout: "degraded_stretch_mode": 0, 2023-10-26T01:37:08.533 INFO:teuthology.orchestra.run.smithi169.stdout: "recovering_stretch_mode": 0, 2023-10-26T01:37:08.533 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_mode_bucket": 0 2023-10-26T01:37:08.533 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:08.533 INFO:teuthology.orchestra.run.smithi169.stdout:} 2023-10-26T01:37:08.535 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:08.323+0000 7f594a1a5640 1 -- 172.21.15.169:0/1483228370 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f592406ae70 msgr2=0x7f592406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:08.535 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:08.323+0000 7f594a1a5640 1 --2- 172.21.15.169:0/1483228370 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f592406ae70 0x7f592406d330 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f5934036000 tx=0x7f5934004730 comp rx=0 tx=0).stop 2023-10-26T01:37:08.536 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:08.323+0000 7f594a1a5640 1 -- 172.21.15.169:0/1483228370 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f59440cde00 msgr2=0x7f5944151830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:08.536 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:08.323+0000 7f594a1a5640 1 --2- 172.21.15.169:0/1483228370 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f59440cde00 0x7f5944151830 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f592c00c490 tx=0x7f592c00c950 comp rx=0 tx=0).stop 2023-10-26T01:37:08.536 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:08.323+0000 7f594a1a5640 1 -- 172.21.15.169:0/1483228370 shutdown_connections 2023-10-26T01:37:08.536 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:08.323+0000 7f594a1a5640 1 --2- 172.21.15.169:0/1483228370 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f592406ae70 0x7f592406d330 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:08.536 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:08.323+0000 7f594a1a5640 1 --2- 172.21.15.169:0/1483228370 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f59440cde00 0x7f5944151830 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:08.537 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:08.323+0000 7f594a1a5640 1 --2- 172.21.15.169:0/1483228370 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f59441592d0 0x7f594415bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:08.537 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:08.323+0000 7f594a1a5640 1 --2- 172.21.15.169:0/1483228370 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f59440ac930 0x7f59441e4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:08.537 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:08.323+0000 7f594a1a5640 1 -- 172.21.15.169:0/1483228370 >> 172.21.15.169:0/1483228370 conn(0x7f5944069930 msgr2=0x7f59441567a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:08.537 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:08.323+0000 7f594a1a5640 1 -- 172.21.15.169:0/1483228370 shutdown_connections 2023-10-26T01:37:08.538 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:08.323+0000 7f594a1a5640 1 -- 172.21.15.169:0/1483228370 wait complete. 2023-10-26T01:37:08.538 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2023-10-26T01:37:13.539 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-26T01:37:13.773 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:13.770+0000 7f4eb4b8e640 1 -- 172.21.15.169:0/731200058 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4eb006a990 msgr2=0x7f4eb00cb390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:13.774 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:13.770+0000 7f4eb4b8e640 1 --2- 172.21.15.169:0/731200058 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4eb006a990 0x7f4eb00cb390 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f4ea8005fd0 tx=0x7f4ea802d0a0 comp rx=0 tx=0).stop 2023-10-26T01:37:13.774 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:13.770+0000 7f4eb4b8e640 1 -- 172.21.15.169:0/731200058 shutdown_connections 2023-10-26T01:37:13.774 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:13.770+0000 7f4eb4b8e640 1 --2- 172.21.15.169:0/731200058 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4eb006a990 0x7f4eb00cb390 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:13.775 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:13.770+0000 7f4eb4b8e640 1 --2- 172.21.15.169:0/731200058 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4eb00c9e00 0x7f4eb00c7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:13.775 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:13.770+0000 7f4eb4b8e640 1 -- 172.21.15.169:0/731200058 >> 172.21.15.169:0/731200058 conn(0x7f4eb00aedd0 msgr2=0x7f4eb00af1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:13.775 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:13.771+0000 7f4eb4b8e640 1 -- 172.21.15.169:0/731200058 shutdown_connections 2023-10-26T01:37:13.775 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:13.771+0000 7f4eb4b8e640 1 -- 172.21.15.169:0/731200058 wait complete. 2023-10-26T01:37:13.777 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:13.772+0000 7f4eb4b8e640 1 Processor -- start 2023-10-26T01:37:13.778 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:13.772+0000 7f4eb4b8e640 1 -- start start 2023-10-26T01:37:13.778 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:13.773+0000 7f4eb4b8e640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4eb006a990 0x7f4eb00b25a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:13.778 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:13.773+0000 7f4eb4b8e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4eb006cae0 0x7f4eb00aff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:13.779 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:13.773+0000 7f4eb4b8e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4eb00c9e00 0x7f4eb00b0440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:13.779 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:13.773+0000 7f4eb4b8e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb014a310 con 0x7f4eb006cae0 2023-10-26T01:37:13.779 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:13.773+0000 7f4eb4b8e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb00b2ae0 con 0x7f4eb00c9e00 2023-10-26T01:37:13.779 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:13.773+0000 7f4eb4b8e640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb0149700 con 0x7f4eb006a990 2023-10-26T01:37:13.779 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:13.774+0000 7f4ea7fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4eb006cae0 0x7f4eb00aff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:13.779 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:13.774+0000 7f4ea7fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4eb006cae0 0x7f4eb00aff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:59986/0 (socket says 172.21.15.169:59986) 2023-10-26T01:37:13.780 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:13.774+0000 7f4ea7fff640 1 -- 172.21.15.169:0/3417178620 learned_addr learned my addr 172.21.15.169:0/3417178620 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:37:13.780 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:13.774+0000 7f4eaf7fe640 1 --2- 172.21.15.169:0/3417178620 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4eb006a990 0x7f4eb00b25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:13.780 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:13.774+0000 7f4eaffff640 1 --2- 172.21.15.169:0/3417178620 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4eb00c9e00 0x7f4eb00b0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:13.780 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:13.774+0000 7f4ea7fff640 1 -- 172.21.15.169:0/3417178620 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4eb006a990 msgr2=0x7f4eb00b25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:13.780 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:13.774+0000 7f4ea7fff640 1 --2- 172.21.15.169:0/3417178620 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4eb006a990 0x7f4eb00b25a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:13.781 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:13.774+0000 7f4ea7fff640 1 -- 172.21.15.169:0/3417178620 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4eb00c9e00 msgr2=0x7f4eb00b0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:13.781 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:13.774+0000 7f4ea7fff640 1 --2- 172.21.15.169:0/3417178620 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4eb00c9e00 0x7f4eb00b0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:13.781 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:13.774+0000 7f4ea7fff640 1 -- 172.21.15.169:0/3417178620 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ea800aa10 con 0x7f4eb006cae0 2023-10-26T01:37:13.781 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:13.774+0000 7f4ea7fff640 1 --2- 172.21.15.169:0/3417178620 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4eb006cae0 0x7f4eb00aff00 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f4e9800c970 tx=0x7f4e9800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:13.781 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:13.775+0000 7f4eacff9640 1 -- 172.21.15.169:0/3417178620 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e98007bb0 con 0x7f4eb006cae0 2023-10-26T01:37:13.781 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:13.775+0000 7f4eacff9640 1 -- 172.21.15.169:0/3417178620 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4e98007e20 con 0x7f4eb006cae0 2023-10-26T01:37:13.782 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:13.775+0000 7f4eacff9640 1 -- 172.21.15.169:0/3417178620 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e98005280 con 0x7f4eb006cae0 2023-10-26T01:37:13.782 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:13.775+0000 7f4eb4b8e640 1 -- 172.21.15.169:0/3417178620 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4eb01499e0 con 0x7f4eb006cae0 2023-10-26T01:37:13.782 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:13.775+0000 7f4eb4b8e640 1 -- 172.21.15.169:0/3417178620 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4eb0149fa0 con 0x7f4eb006cae0 2023-10-26T01:37:13.783 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:13.776+0000 7f4eb4b8e640 1 -- 172.21.15.169:0/3417178620 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4eb0147360 con 0x7f4eb006cae0 2023-10-26T01:37:13.783 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:13.777+0000 7f4eacff9640 1 -- 172.21.15.169:0/3417178620 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f4e980053e0 con 0x7f4eb006cae0 2023-10-26T01:37:13.783 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:13.778+0000 7f4eacff9640 1 --2- 172.21.15.169:0/3417178620 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f4e9006ae40 0x7f4e9006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:13.784 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:13.778+0000 7f4eacff9640 1 -- 172.21.15.169:0/3417178620 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4e98089ae0 con 0x7f4eb006cae0 2023-10-26T01:37:13.784 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:13.779+0000 7f4eaf7fe640 1 --2- 172.21.15.169:0/3417178620 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f4e9006ae40 0x7f4e9006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:13.784 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:13.780+0000 7f4eaf7fe640 1 --2- 172.21.15.169:0/3417178620 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f4e9006ae40 0x7f4e9006d300 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f4ea0007fd0 tx=0x7f4ea0009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:13.785 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:13.783+0000 7f4eacff9640 1 -- 172.21.15.169:0/3417178620 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4eb0147360 con 0x7f4eb006cae0 2023-10-26T01:37:13.974 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:13.971+0000 7f4eb4b8e640 1 -- 172.21.15.169:0/3417178620 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f4eb0147360 con 0x7f4eb006cae0 2023-10-26T01:37:13.976 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:13.974+0000 7f4eacff9640 1 -- 172.21.15.169:0/3417178620 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v109) v1 ==== 81+0+34055 (secure 0 0 0) 0x7f4eb0147360 con 0x7f4eb006cae0 2023-10-26T01:37:13.977 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:37:13.977 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2023-10-26T01:37:13.977 INFO:teuthology.orchestra.run.smithi169.stdout: "epoch": 109, 2023-10-26T01:37:13.978 INFO:teuthology.orchestra.run.smithi169.stdout: "fsid": "30203b28-ed8a-44e8-a9a6-a9bf25f980f1", 2023-10-26T01:37:13.978 INFO:teuthology.orchestra.run.smithi169.stdout: "created": "2023-10-26T01:30:22.897155+0000", 2023-10-26T01:37:13.978 INFO:teuthology.orchestra.run.smithi169.stdout: "modified": "2023-10-26T01:37:13.181064+0000", 2023-10-26T01:37:13.978 INFO:teuthology.orchestra.run.smithi169.stdout: "last_up_change": "2023-10-26T01:30:31.875281+0000", 2023-10-26T01:37:13.979 INFO:teuthology.orchestra.run.smithi169.stdout: "last_in_change": "2023-10-26T01:30:27.637941+0000", 2023-10-26T01:37:13.979 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-26T01:37:13.979 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_num": 5799936, 2023-10-26T01:37:13.979 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_set": [ 2023-10-26T01:37:13.980 INFO:teuthology.orchestra.run.smithi169.stdout: "pglog_hardlimit", 2023-10-26T01:37:13.980 INFO:teuthology.orchestra.run.smithi169.stdout: "purged_snapdirs", 2023-10-26T01:37:13.980 INFO:teuthology.orchestra.run.smithi169.stdout: "recovery_deletes", 2023-10-26T01:37:13.980 INFO:teuthology.orchestra.run.smithi169.stdout: "sortbitwise" 2023-10-26T01:37:13.980 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:37:13.981 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_version": 4, 2023-10-26T01:37:13.981 INFO:teuthology.orchestra.run.smithi169.stdout: "full_ratio": 0.69999998807907104, 2023-10-26T01:37:13.981 INFO:teuthology.orchestra.run.smithi169.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-26T01:37:13.981 INFO:teuthology.orchestra.run.smithi169.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-26T01:37:13.981 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_snapshot": "", 2023-10-26T01:37:13.982 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_max": 13, 2023-10-26T01:37:13.982 INFO:teuthology.orchestra.run.smithi169.stdout: "max_osd": 8, 2023-10-26T01:37:13.982 INFO:teuthology.orchestra.run.smithi169.stdout: "require_min_compat_client": "luminous", 2023-10-26T01:37:13.982 INFO:teuthology.orchestra.run.smithi169.stdout: "min_compat_client": "jewel", 2023-10-26T01:37:13.983 INFO:teuthology.orchestra.run.smithi169.stdout: "require_osd_release": "reef", 2023-10-26T01:37:13.983 INFO:teuthology.orchestra.run.smithi169.stdout: "allow_crimson": false, 2023-10-26T01:37:13.983 INFO:teuthology.orchestra.run.smithi169.stdout: "pools": [ 2023-10-26T01:37:13.983 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:13.983 INFO:teuthology.orchestra.run.smithi169.stdout: "pool": 1, 2023-10-26T01:37:13.984 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_name": ".mgr", 2023-10-26T01:37:13.984 INFO:teuthology.orchestra.run.smithi169.stdout: "create_time": "2023-10-26T01:30:32.200808+0000", 2023-10-26T01:37:13.984 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": 1, 2023-10-26T01:37:13.984 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_names": "hashpspool", 2023-10-26T01:37:13.985 INFO:teuthology.orchestra.run.smithi169.stdout: "type": 1, 2023-10-26T01:37:13.985 INFO:teuthology.orchestra.run.smithi169.stdout: "size": 2, 2023-10-26T01:37:13.985 INFO:teuthology.orchestra.run.smithi169.stdout: "min_size": 1, 2023-10-26T01:37:13.985 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_rule": 0, 2023-10-26T01:37:13.985 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_count": 0, 2023-10-26T01:37:13.986 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_target": 0, 2023-10-26T01:37:13.986 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T01:37:13.986 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T01:37:13.986 INFO:teuthology.orchestra.run.smithi169.stdout: "object_hash": 2, 2023-10-26T01:37:13.987 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_mode": "off", 2023-10-26T01:37:13.987 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num": 1, 2023-10-26T01:37:13.987 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num": 1, 2023-10-26T01:37:13.987 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num_target": 1, 2023-10-26T01:37:13.987 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_target": 1, 2023-10-26T01:37:13.988 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_pending": 1, 2023-10-26T01:37:13.988 INFO:teuthology.orchestra.run.smithi169.stdout: "last_pg_merge_meta": { 2023-10-26T01:37:13.988 INFO:teuthology.orchestra.run.smithi169.stdout: "source_pgid": "0.0", 2023-10-26T01:37:13.988 INFO:teuthology.orchestra.run.smithi169.stdout: "ready_epoch": 0, 2023-10-26T01:37:13.988 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_started": 0, 2023-10-26T01:37:13.989 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_clean": 0, 2023-10-26T01:37:13.989 INFO:teuthology.orchestra.run.smithi169.stdout: "source_version": "0'0", 2023-10-26T01:37:13.989 INFO:teuthology.orchestra.run.smithi169.stdout: "target_version": "0'0" 2023-10-26T01:37:13.989 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:13.989 INFO:teuthology.orchestra.run.smithi169.stdout: "last_change": "109", 2023-10-26T01:37:13.990 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend": "0", 2023-10-26T01:37:13.990 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T01:37:13.990 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T01:37:13.990 INFO:teuthology.orchestra.run.smithi169.stdout: "auid": 0, 2023-10-26T01:37:13.990 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_mode": "selfmanaged", 2023-10-26T01:37:13.991 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_seq": 0, 2023-10-26T01:37:13.991 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_epoch": 0, 2023-10-26T01:37:13.991 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_snaps": [], 2023-10-26T01:37:13.991 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps": "[]", 2023-10-26T01:37:13.991 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_bytes": 0, 2023-10-26T01:37:13.992 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_objects": 0, 2023-10-26T01:37:13.992 INFO:teuthology.orchestra.run.smithi169.stdout: "tiers": [], 2023-10-26T01:37:13.992 INFO:teuthology.orchestra.run.smithi169.stdout: "tier_of": -1, 2023-10-26T01:37:13.992 INFO:teuthology.orchestra.run.smithi169.stdout: "read_tier": -1, 2023-10-26T01:37:13.992 INFO:teuthology.orchestra.run.smithi169.stdout: "write_tier": -1, 2023-10-26T01:37:13.993 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_mode": "none", 2023-10-26T01:37:13.993 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_bytes": 0, 2023-10-26T01:37:13.993 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_objects": 0, 2023-10-26T01:37:13.993 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T01:37:13.993 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T01:37:13.994 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T01:37:13.994 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_flush_age": 0, 2023-10-26T01:37:13.994 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_evict_age": 0, 2023-10-26T01:37:13.994 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profile": "", 2023-10-26T01:37:13.994 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_params": { 2023-10-26T01:37:13.995 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "none" 2023-10-26T01:37:13.995 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:13.995 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_period": 0, 2023-10-26T01:37:13.995 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_count": 0, 2023-10-26T01:37:13.995 INFO:teuthology.orchestra.run.smithi169.stdout: "use_gmt_hitset": true, 2023-10-26T01:37:13.996 INFO:teuthology.orchestra.run.smithi169.stdout: "min_read_recency_for_promote": 0, 2023-10-26T01:37:13.996 INFO:teuthology.orchestra.run.smithi169.stdout: "min_write_recency_for_promote": 0, 2023-10-26T01:37:13.996 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T01:37:13.996 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_search_last_n": 0, 2023-10-26T01:37:13.996 INFO:teuthology.orchestra.run.smithi169.stdout: "grade_table": [], 2023-10-26T01:37:13.997 INFO:teuthology.orchestra.run.smithi169.stdout: "stripe_width": 0, 2023-10-26T01:37:13.997 INFO:teuthology.orchestra.run.smithi169.stdout: "expected_num_objects": 0, 2023-10-26T01:37:13.997 INFO:teuthology.orchestra.run.smithi169.stdout: "fast_read": false, 2023-10-26T01:37:13.997 INFO:teuthology.orchestra.run.smithi169.stdout: "options": { 2023-10-26T01:37:13.997 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_max": 32, 2023-10-26T01:37:13.998 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_min": 1 2023-10-26T01:37:13.998 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:13.998 INFO:teuthology.orchestra.run.smithi169.stdout: "application_metadata": { 2023-10-26T01:37:13.998 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": {} 2023-10-26T01:37:13.998 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:13.999 INFO:teuthology.orchestra.run.smithi169.stdout: "read_balance": { 2023-10-26T01:37:13.999 INFO:teuthology.orchestra.run.smithi169.stdout: "score_acting": 8, 2023-10-26T01:37:13.999 INFO:teuthology.orchestra.run.smithi169.stdout: "score_stable": 8, 2023-10-26T01:37:13.999 INFO:teuthology.orchestra.run.smithi169.stdout: "optimal_score": 0.25, 2023-10-26T01:37:13.999 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_acting": 2, 2023-10-26T01:37:14.000 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_stable": 2, 2023-10-26T01:37:14.000 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity_weighted": 1, 2023-10-26T01:37:14.000 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity": 1, 2023-10-26T01:37:14.000 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity_weighted": 1 2023-10-26T01:37:14.000 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:14.001 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.001 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.001 INFO:teuthology.orchestra.run.smithi169.stdout: "pool": 12, 2023-10-26T01:37:14.001 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_name": "cephfs_metadata", 2023-10-26T01:37:14.001 INFO:teuthology.orchestra.run.smithi169.stdout: "create_time": "2023-10-26T01:36:41.430455+0000", 2023-10-26T01:37:14.001 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": 1, 2023-10-26T01:37:14.002 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_names": "hashpspool", 2023-10-26T01:37:14.002 INFO:teuthology.orchestra.run.smithi169.stdout: "type": 1, 2023-10-26T01:37:14.002 INFO:teuthology.orchestra.run.smithi169.stdout: "size": 2, 2023-10-26T01:37:14.002 INFO:teuthology.orchestra.run.smithi169.stdout: "min_size": 1, 2023-10-26T01:37:14.002 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_rule": 0, 2023-10-26T01:37:14.003 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_count": 0, 2023-10-26T01:37:14.003 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_target": 0, 2023-10-26T01:37:14.003 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T01:37:14.003 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T01:37:14.003 INFO:teuthology.orchestra.run.smithi169.stdout: "object_hash": 2, 2023-10-26T01:37:14.003 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_mode": "off", 2023-10-26T01:37:14.004 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num": 32, 2023-10-26T01:37:14.004 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num": 32, 2023-10-26T01:37:14.004 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num_target": 32, 2023-10-26T01:37:14.004 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_target": 32, 2023-10-26T01:37:14.004 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_pending": 32, 2023-10-26T01:37:14.005 INFO:teuthology.orchestra.run.smithi169.stdout: "last_pg_merge_meta": { 2023-10-26T01:37:14.005 INFO:teuthology.orchestra.run.smithi169.stdout: "source_pgid": "0.0", 2023-10-26T01:37:14.005 INFO:teuthology.orchestra.run.smithi169.stdout: "ready_epoch": 0, 2023-10-26T01:37:14.005 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_started": 0, 2023-10-26T01:37:14.005 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_clean": 0, 2023-10-26T01:37:14.005 INFO:teuthology.orchestra.run.smithi169.stdout: "source_version": "0'0", 2023-10-26T01:37:14.006 INFO:teuthology.orchestra.run.smithi169.stdout: "target_version": "0'0" 2023-10-26T01:37:14.006 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.006 INFO:teuthology.orchestra.run.smithi169.stdout: "last_change": "109", 2023-10-26T01:37:14.006 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend": "0", 2023-10-26T01:37:14.006 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T01:37:14.007 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T01:37:14.007 INFO:teuthology.orchestra.run.smithi169.stdout: "auid": 0, 2023-10-26T01:37:14.007 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_mode": "selfmanaged", 2023-10-26T01:37:14.007 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_seq": 0, 2023-10-26T01:37:14.007 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_epoch": 0, 2023-10-26T01:37:14.008 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_snaps": [], 2023-10-26T01:37:14.008 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps": "[]", 2023-10-26T01:37:14.008 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_bytes": 0, 2023-10-26T01:37:14.008 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_objects": 0, 2023-10-26T01:37:14.008 INFO:teuthology.orchestra.run.smithi169.stdout: "tiers": [], 2023-10-26T01:37:14.009 INFO:teuthology.orchestra.run.smithi169.stdout: "tier_of": -1, 2023-10-26T01:37:14.009 INFO:teuthology.orchestra.run.smithi169.stdout: "read_tier": -1, 2023-10-26T01:37:14.009 INFO:teuthology.orchestra.run.smithi169.stdout: "write_tier": -1, 2023-10-26T01:37:14.009 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_mode": "none", 2023-10-26T01:37:14.009 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_bytes": 0, 2023-10-26T01:37:14.009 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_objects": 0, 2023-10-26T01:37:14.010 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T01:37:14.010 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T01:37:14.010 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T01:37:14.010 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_flush_age": 0, 2023-10-26T01:37:14.010 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_evict_age": 0, 2023-10-26T01:37:14.011 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profile": "", 2023-10-26T01:37:14.011 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_params": { 2023-10-26T01:37:14.011 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "none" 2023-10-26T01:37:14.011 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.011 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_period": 0, 2023-10-26T01:37:14.011 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_count": 0, 2023-10-26T01:37:14.012 INFO:teuthology.orchestra.run.smithi169.stdout: "use_gmt_hitset": true, 2023-10-26T01:37:14.012 INFO:teuthology.orchestra.run.smithi169.stdout: "min_read_recency_for_promote": 0, 2023-10-26T01:37:14.012 INFO:teuthology.orchestra.run.smithi169.stdout: "min_write_recency_for_promote": 0, 2023-10-26T01:37:14.012 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T01:37:14.012 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_search_last_n": 0, 2023-10-26T01:37:14.013 INFO:teuthology.orchestra.run.smithi169.stdout: "grade_table": [], 2023-10-26T01:37:14.013 INFO:teuthology.orchestra.run.smithi169.stdout: "stripe_width": 0, 2023-10-26T01:37:14.013 INFO:teuthology.orchestra.run.smithi169.stdout: "expected_num_objects": 0, 2023-10-26T01:37:14.013 INFO:teuthology.orchestra.run.smithi169.stdout: "fast_read": false, 2023-10-26T01:37:14.013 INFO:teuthology.orchestra.run.smithi169.stdout: "options": { 2023-10-26T01:37:14.014 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_bias": 4, 2023-10-26T01:37:14.014 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_min": 16, 2023-10-26T01:37:14.014 INFO:teuthology.orchestra.run.smithi169.stdout: "recovery_priority": 5 2023-10-26T01:37:14.014 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.014 INFO:teuthology.orchestra.run.smithi169.stdout: "application_metadata": { 2023-10-26T01:37:14.014 INFO:teuthology.orchestra.run.smithi169.stdout: "cephfs": { 2023-10-26T01:37:14.015 INFO:teuthology.orchestra.run.smithi169.stdout: "metadata": "cephfs" 2023-10-26T01:37:14.015 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:14.015 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.015 INFO:teuthology.orchestra.run.smithi169.stdout: "read_balance": { 2023-10-26T01:37:14.015 INFO:teuthology.orchestra.run.smithi169.stdout: "score_acting": 2, 2023-10-26T01:37:14.015 INFO:teuthology.orchestra.run.smithi169.stdout: "score_stable": 2, 2023-10-26T01:37:14.016 INFO:teuthology.orchestra.run.smithi169.stdout: "optimal_score": 1, 2023-10-26T01:37:14.016 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_acting": 2, 2023-10-26T01:37:14.016 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_stable": 2, 2023-10-26T01:37:14.016 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity_weighted": 1, 2023-10-26T01:37:14.016 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity": 1, 2023-10-26T01:37:14.017 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity_weighted": 1 2023-10-26T01:37:14.017 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:14.017 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.017 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.017 INFO:teuthology.orchestra.run.smithi169.stdout: "pool": 13, 2023-10-26T01:37:14.018 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_name": "cephfs_data", 2023-10-26T01:37:14.018 INFO:teuthology.orchestra.run.smithi169.stdout: "create_time": "2023-10-26T01:36:42.883252+0000", 2023-10-26T01:37:14.018 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": 1, 2023-10-26T01:37:14.018 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_names": "hashpspool", 2023-10-26T01:37:14.018 INFO:teuthology.orchestra.run.smithi169.stdout: "type": 1, 2023-10-26T01:37:14.019 INFO:teuthology.orchestra.run.smithi169.stdout: "size": 2, 2023-10-26T01:37:14.019 INFO:teuthology.orchestra.run.smithi169.stdout: "min_size": 1, 2023-10-26T01:37:14.019 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_rule": 0, 2023-10-26T01:37:14.019 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_count": 0, 2023-10-26T01:37:14.019 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_target": 0, 2023-10-26T01:37:14.020 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T01:37:14.020 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T01:37:14.020 INFO:teuthology.orchestra.run.smithi169.stdout: "object_hash": 2, 2023-10-26T01:37:14.020 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_mode": "off", 2023-10-26T01:37:14.020 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num": 64, 2023-10-26T01:37:14.020 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num": 64, 2023-10-26T01:37:14.021 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num_target": 64, 2023-10-26T01:37:14.021 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_target": 64, 2023-10-26T01:37:14.021 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_pending": 64, 2023-10-26T01:37:14.021 INFO:teuthology.orchestra.run.smithi169.stdout: "last_pg_merge_meta": { 2023-10-26T01:37:14.021 INFO:teuthology.orchestra.run.smithi169.stdout: "source_pgid": "0.0", 2023-10-26T01:37:14.022 INFO:teuthology.orchestra.run.smithi169.stdout: "ready_epoch": 0, 2023-10-26T01:37:14.022 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_started": 0, 2023-10-26T01:37:14.022 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_clean": 0, 2023-10-26T01:37:14.022 INFO:teuthology.orchestra.run.smithi169.stdout: "source_version": "0'0", 2023-10-26T01:37:14.022 INFO:teuthology.orchestra.run.smithi169.stdout: "target_version": "0'0" 2023-10-26T01:37:14.023 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.023 INFO:teuthology.orchestra.run.smithi169.stdout: "last_change": "109", 2023-10-26T01:37:14.023 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend": "0", 2023-10-26T01:37:14.023 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T01:37:14.023 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T01:37:14.024 INFO:teuthology.orchestra.run.smithi169.stdout: "auid": 0, 2023-10-26T01:37:14.024 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_mode": "selfmanaged", 2023-10-26T01:37:14.024 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_seq": 0, 2023-10-26T01:37:14.024 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_epoch": 0, 2023-10-26T01:37:14.024 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_snaps": [], 2023-10-26T01:37:14.024 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps": "[]", 2023-10-26T01:37:14.025 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_bytes": 0, 2023-10-26T01:37:14.025 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_objects": 0, 2023-10-26T01:37:14.025 INFO:teuthology.orchestra.run.smithi169.stdout: "tiers": [], 2023-10-26T01:37:14.025 INFO:teuthology.orchestra.run.smithi169.stdout: "tier_of": -1, 2023-10-26T01:37:14.026 INFO:teuthology.orchestra.run.smithi169.stdout: "read_tier": -1, 2023-10-26T01:37:14.026 INFO:teuthology.orchestra.run.smithi169.stdout: "write_tier": -1, 2023-10-26T01:37:14.026 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_mode": "none", 2023-10-26T01:37:14.026 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_bytes": 0, 2023-10-26T01:37:14.026 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_objects": 0, 2023-10-26T01:37:14.027 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T01:37:14.027 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T01:37:14.027 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T01:37:14.027 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_flush_age": 0, 2023-10-26T01:37:14.027 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_evict_age": 0, 2023-10-26T01:37:14.028 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profile": "", 2023-10-26T01:37:14.028 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_params": { 2023-10-26T01:37:14.028 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "none" 2023-10-26T01:37:14.028 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.028 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_period": 0, 2023-10-26T01:37:14.029 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_count": 0, 2023-10-26T01:37:14.029 INFO:teuthology.orchestra.run.smithi169.stdout: "use_gmt_hitset": true, 2023-10-26T01:37:14.029 INFO:teuthology.orchestra.run.smithi169.stdout: "min_read_recency_for_promote": 0, 2023-10-26T01:37:14.029 INFO:teuthology.orchestra.run.smithi169.stdout: "min_write_recency_for_promote": 0, 2023-10-26T01:37:14.029 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T01:37:14.029 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_search_last_n": 0, 2023-10-26T01:37:14.030 INFO:teuthology.orchestra.run.smithi169.stdout: "grade_table": [], 2023-10-26T01:37:14.030 INFO:teuthology.orchestra.run.smithi169.stdout: "stripe_width": 0, 2023-10-26T01:37:14.030 INFO:teuthology.orchestra.run.smithi169.stdout: "expected_num_objects": 0, 2023-10-26T01:37:14.030 INFO:teuthology.orchestra.run.smithi169.stdout: "fast_read": false, 2023-10-26T01:37:14.030 INFO:teuthology.orchestra.run.smithi169.stdout: "options": { 2023-10-26T01:37:14.030 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_min": 64, 2023-10-26T01:37:14.031 INFO:teuthology.orchestra.run.smithi169.stdout: "target_size_ratio": 0.89999997615814209 2023-10-26T01:37:14.031 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.031 INFO:teuthology.orchestra.run.smithi169.stdout: "application_metadata": { 2023-10-26T01:37:14.031 INFO:teuthology.orchestra.run.smithi169.stdout: "cephfs": { 2023-10-26T01:37:14.031 INFO:teuthology.orchestra.run.smithi169.stdout: "data": "cephfs" 2023-10-26T01:37:14.031 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:14.032 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.032 INFO:teuthology.orchestra.run.smithi169.stdout: "read_balance": { 2023-10-26T01:37:14.032 INFO:teuthology.orchestra.run.smithi169.stdout: "score_acting": 1.3799999952316284, 2023-10-26T01:37:14.032 INFO:teuthology.orchestra.run.smithi169.stdout: "score_stable": 1.3799999952316284, 2023-10-26T01:37:14.032 INFO:teuthology.orchestra.run.smithi169.stdout: "optimal_score": 1, 2023-10-26T01:37:14.033 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_acting": 1.3799999952316284, 2023-10-26T01:37:14.033 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_stable": 1.3799999952316284, 2023-10-26T01:37:14.033 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity_weighted": 1, 2023-10-26T01:37:14.033 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity": 1, 2023-10-26T01:37:14.033 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity_weighted": 1 2023-10-26T01:37:14.033 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:14.034 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:14.034 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:37:14.034 INFO:teuthology.orchestra.run.smithi169.stdout: "osds": [ 2023-10-26T01:37:14.034 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.034 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 0, 2023-10-26T01:37:14.034 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:37:14.035 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:37:14.035 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:37:14.035 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:37:14.035 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:37:14.035 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:37:14.035 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:37:14.036 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:37:14.036 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 101, 2023-10-26T01:37:14.036 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:37:14.036 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:37:14.036 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:37:14.036 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:14.037 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.037 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:14.037 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6816", 2023-10-26T01:37:14.037 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:37:14.037 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.038 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.038 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:14.038 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6817", 2023-10-26T01:37:14.038 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:37:14.038 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:14.038 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:14.039 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.039 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:37:14.039 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:14.039 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.039 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:14.039 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6818", 2023-10-26T01:37:14.040 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:37:14.040 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.040 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.040 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:14.040 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6819", 2023-10-26T01:37:14.040 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:37:14.041 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:14.041 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:14.041 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.041 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:37:14.041 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:14.042 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.042 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:14.042 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6822", 2023-10-26T01:37:14.042 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:37:14.042 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.043 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.043 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:14.043 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6823", 2023-10-26T01:37:14.043 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:37:14.043 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:14.043 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:14.044 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.044 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:37:14.044 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:14.044 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.044 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:14.044 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6820", 2023-10-26T01:37:14.045 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:37:14.045 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.045 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.045 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:14.045 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6821", 2023-10-26T01:37:14.045 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:37:14.046 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:14.046 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:14.046 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.046 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6817/4204255864", 2023-10-26T01:37:14.046 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6819/4204255864", 2023-10-26T01:37:14.046 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6823/4204255864", 2023-10-26T01:37:14.047 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6821/4204255864", 2023-10-26T01:37:14.047 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:37:14.047 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:37:14.047 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:37:14.047 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:14.047 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.048 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.048 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 1, 2023-10-26T01:37:14.048 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:37:14.048 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:37:14.048 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:37:14.048 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:37:14.049 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:37:14.049 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:37:14.049 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:37:14.049 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:37:14.049 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 101, 2023-10-26T01:37:14.049 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:37:14.050 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:37:14.050 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:37:14.050 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:14.050 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.050 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:14.051 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6808", 2023-10-26T01:37:14.051 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:37:14.051 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.051 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.051 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:14.051 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6809", 2023-10-26T01:37:14.052 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:37:14.052 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:14.052 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:14.052 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.052 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:37:14.052 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:14.053 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.053 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:14.053 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6810", 2023-10-26T01:37:14.053 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:37:14.053 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.053 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.054 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:14.054 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6811", 2023-10-26T01:37:14.054 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:37:14.054 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:14.054 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:14.054 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.055 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:37:14.055 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:14.055 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.055 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:14.055 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6814", 2023-10-26T01:37:14.055 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:37:14.056 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.056 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.056 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:14.056 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6815", 2023-10-26T01:37:14.056 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:37:14.057 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:14.057 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:14.057 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.057 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:37:14.057 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:14.057 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.058 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:14.058 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6812", 2023-10-26T01:37:14.058 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:37:14.058 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.058 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.059 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:14.059 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6813", 2023-10-26T01:37:14.059 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:37:14.059 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:14.059 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:14.060 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.060 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6809/238206046", 2023-10-26T01:37:14.060 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6811/238206046", 2023-10-26T01:37:14.060 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6815/238206046", 2023-10-26T01:37:14.060 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6813/238206046", 2023-10-26T01:37:14.060 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:37:14.061 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:37:14.061 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:37:14.061 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:14.061 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.061 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.061 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 2, 2023-10-26T01:37:14.062 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:37:14.062 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:37:14.062 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:37:14.062 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:37:14.062 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:37:14.062 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:37:14.063 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:37:14.063 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:37:14.063 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 101, 2023-10-26T01:37:14.063 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:37:14.063 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:37:14.064 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:37:14.064 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:14.064 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.064 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:14.064 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6800", 2023-10-26T01:37:14.064 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:37:14.065 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.065 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.065 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:14.065 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6801", 2023-10-26T01:37:14.065 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:37:14.066 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:14.066 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:14.066 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.066 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:37:14.066 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:14.067 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.067 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:14.067 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6802", 2023-10-26T01:37:14.067 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:37:14.067 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.067 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.068 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:14.068 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6803", 2023-10-26T01:37:14.068 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:37:14.068 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:14.068 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:14.069 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.069 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:37:14.069 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:14.069 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.069 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:14.069 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6806", 2023-10-26T01:37:14.069 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:37:14.070 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.070 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.070 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:14.070 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6807", 2023-10-26T01:37:14.070 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:37:14.071 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:14.071 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:14.071 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.071 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:37:14.071 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:14.071 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.072 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:14.072 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6804", 2023-10-26T01:37:14.072 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:37:14.072 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.072 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.072 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:14.073 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6805", 2023-10-26T01:37:14.073 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:37:14.073 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:14.073 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:14.073 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.073 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6801/3215791846", 2023-10-26T01:37:14.074 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6803/3215791846", 2023-10-26T01:37:14.074 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6807/3215791846", 2023-10-26T01:37:14.074 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6805/3215791846", 2023-10-26T01:37:14.074 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:37:14.074 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:37:14.075 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:37:14.075 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:14.075 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.075 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.075 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 3, 2023-10-26T01:37:14.076 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:37:14.076 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:37:14.076 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:37:14.076 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:37:14.076 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:37:14.077 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:37:14.077 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:37:14.077 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 14, 2023-10-26T01:37:14.077 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 101, 2023-10-26T01:37:14.077 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:37:14.077 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:37:14.078 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:37:14.078 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:14.078 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.078 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:14.078 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6824", 2023-10-26T01:37:14.079 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:37:14.079 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.079 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.079 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:14.079 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6825", 2023-10-26T01:37:14.079 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:37:14.080 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:14.080 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:14.080 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.080 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:37:14.080 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:14.081 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.081 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:14.081 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6826", 2023-10-26T01:37:14.081 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:37:14.081 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.081 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.082 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:14.082 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6827", 2023-10-26T01:37:14.082 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:37:14.082 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:14.082 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:14.083 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.083 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:37:14.083 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:14.083 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.083 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:14.083 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6830", 2023-10-26T01:37:14.084 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:37:14.084 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.084 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.084 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:14.084 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6831", 2023-10-26T01:37:14.084 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:37:14.085 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:14.085 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:14.085 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.085 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:37:14.085 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:14.086 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.086 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:14.086 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6828", 2023-10-26T01:37:14.086 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:37:14.086 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.087 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.087 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:14.087 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6829", 2023-10-26T01:37:14.087 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:37:14.087 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:14.088 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:14.088 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.088 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6825/2256701226", 2023-10-26T01:37:14.088 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6827/2256701226", 2023-10-26T01:37:14.088 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6831/2256701226", 2023-10-26T01:37:14.089 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6829/2256701226", 2023-10-26T01:37:14.089 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:37:14.089 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:37:14.089 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:37:14.089 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:14.090 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.090 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.090 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 4, 2023-10-26T01:37:14.090 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:37:14.090 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:37:14.090 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:37:14.091 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:37:14.091 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:37:14.091 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:37:14.091 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:37:14.091 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:37:14.092 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 101, 2023-10-26T01:37:14.092 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:37:14.092 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:37:14.092 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:37:14.092 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:14.092 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.093 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:14.093 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6816", 2023-10-26T01:37:14.093 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:37:14.093 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.093 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.094 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:14.094 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6817", 2023-10-26T01:37:14.094 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:37:14.094 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:14.094 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:14.094 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.095 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:37:14.095 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:14.095 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.095 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:14.095 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6818", 2023-10-26T01:37:14.096 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:37:14.096 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.096 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.096 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:14.096 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6819", 2023-10-26T01:37:14.096 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:37:14.097 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:14.097 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:14.097 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.097 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:37:14.097 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:14.098 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.098 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:14.098 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6825", 2023-10-26T01:37:14.098 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:37:14.099 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.099 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.099 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:14.099 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6828", 2023-10-26T01:37:14.099 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:37:14.099 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:14.100 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:14.100 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.100 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:37:14.100 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:14.100 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.100 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:14.101 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6820", 2023-10-26T01:37:14.101 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:37:14.101 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.101 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.101 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:14.101 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6823", 2023-10-26T01:37:14.102 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:37:14.102 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:14.102 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:14.102 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.102 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6817/1082279874", 2023-10-26T01:37:14.103 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6819/1082279874", 2023-10-26T01:37:14.103 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6828/1082279874", 2023-10-26T01:37:14.103 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6823/1082279874", 2023-10-26T01:37:14.103 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:37:14.103 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:37:14.104 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:37:14.104 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:14.104 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.104 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.104 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 5, 2023-10-26T01:37:14.104 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:37:14.105 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:37:14.105 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:37:14.105 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:37:14.105 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:37:14.105 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:37:14.106 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:37:14.106 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:37:14.106 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 101, 2023-10-26T01:37:14.106 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:37:14.106 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:37:14.106 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:37:14.107 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:14.107 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.107 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:14.107 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6808", 2023-10-26T01:37:14.107 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:37:14.108 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.108 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.108 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:14.108 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6809", 2023-10-26T01:37:14.108 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:37:14.109 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:14.109 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:14.109 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.109 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:37:14.109 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:14.109 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.110 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:14.110 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6810", 2023-10-26T01:37:14.110 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:37:14.110 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.110 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.110 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:14.111 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6811", 2023-10-26T01:37:14.111 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:37:14.111 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:14.111 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:14.111 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.112 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:37:14.112 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:14.112 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.112 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:14.112 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6814", 2023-10-26T01:37:14.113 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:37:14.113 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.113 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.113 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:14.113 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6815", 2023-10-26T01:37:14.113 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:37:14.114 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:14.114 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:14.114 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.114 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:37:14.114 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:14.115 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.115 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:14.115 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6812", 2023-10-26T01:37:14.115 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:37:14.115 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.115 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.116 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:14.116 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6813", 2023-10-26T01:37:14.116 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:37:14.116 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:14.116 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:14.116 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.117 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6809/3335654351", 2023-10-26T01:37:14.117 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6811/3335654351", 2023-10-26T01:37:14.117 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6815/3335654351", 2023-10-26T01:37:14.117 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6813/3335654351", 2023-10-26T01:37:14.117 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:37:14.118 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:37:14.118 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:37:14.118 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:14.118 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.118 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.118 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 6, 2023-10-26T01:37:14.119 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:37:14.119 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:37:14.119 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:37:14.119 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:37:14.119 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:37:14.119 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:37:14.120 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:37:14.120 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:37:14.120 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 101, 2023-10-26T01:37:14.120 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:37:14.120 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:37:14.121 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:37:14.121 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:14.121 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.121 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:14.121 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6800", 2023-10-26T01:37:14.121 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:37:14.122 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.122 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.122 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:14.122 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6801", 2023-10-26T01:37:14.122 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:37:14.122 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:14.123 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:14.123 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.123 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:37:14.123 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:14.123 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.124 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:14.124 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6802", 2023-10-26T01:37:14.124 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:37:14.124 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.124 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.125 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:14.125 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6803", 2023-10-26T01:37:14.125 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:37:14.125 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:14.125 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:14.126 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.126 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:37:14.126 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:14.126 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.126 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:14.127 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6806", 2023-10-26T01:37:14.127 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:37:14.127 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.127 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.127 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:14.127 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6807", 2023-10-26T01:37:14.128 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:37:14.128 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:14.128 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:14.128 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.128 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:37:14.129 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:14.129 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.129 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:14.129 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6804", 2023-10-26T01:37:14.129 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:37:14.130 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.130 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.130 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:14.130 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6805", 2023-10-26T01:37:14.130 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:37:14.131 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:14.131 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:14.131 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.131 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6801/2520460697", 2023-10-26T01:37:14.131 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6803/2520460697", 2023-10-26T01:37:14.132 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6807/2520460697", 2023-10-26T01:37:14.132 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6805/2520460697", 2023-10-26T01:37:14.132 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:37:14.132 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:37:14.132 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:37:14.133 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:14.133 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.133 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.133 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 7, 2023-10-26T01:37:14.133 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:37:14.133 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:37:14.134 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:37:14.134 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:37:14.134 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:37:14.134 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:37:14.134 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:37:14.134 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:37:14.135 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 101, 2023-10-26T01:37:14.135 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:37:14.135 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:37:14.135 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:37:14.135 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:14.135 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.136 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:14.136 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6821", 2023-10-26T01:37:14.136 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:37:14.136 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.136 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.137 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:14.137 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6822", 2023-10-26T01:37:14.137 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:37:14.137 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:14.137 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:14.137 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.138 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:37:14.138 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:14.138 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.138 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:14.138 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6824", 2023-10-26T01:37:14.139 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:37:14.139 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.139 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.139 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:14.139 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6826", 2023-10-26T01:37:14.140 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:37:14.140 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:14.140 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:14.140 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.140 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:37:14.140 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:14.141 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.141 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:14.141 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6830", 2023-10-26T01:37:14.141 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:37:14.141 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.141 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.142 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:14.142 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6831", 2023-10-26T01:37:14.142 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:37:14.142 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:14.142 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:14.143 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.143 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:37:14.143 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:14.143 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.143 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:14.143 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6827", 2023-10-26T01:37:14.144 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:37:14.144 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.144 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.144 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:14.144 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6829", 2023-10-26T01:37:14.144 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:37:14.145 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:14.145 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:14.145 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.145 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6822/1115930619", 2023-10-26T01:37:14.145 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6826/1115930619", 2023-10-26T01:37:14.146 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6831/1115930619", 2023-10-26T01:37:14.146 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6829/1115930619", 2023-10-26T01:37:14.146 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:37:14.146 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:37:14.146 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:37:14.147 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:14.147 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:14.147 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:37:14.147 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_xinfo": [ 2023-10-26T01:37:14.147 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.147 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 0, 2023-10-26T01:37:14.148 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:37:14.148 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:37:14.148 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:37:14.148 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:37:14.149 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:37:14.149 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.889675+0000", 2023-10-26T01:37:14.149 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:37:14.149 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.149 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.149 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 1, 2023-10-26T01:37:14.150 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:37:14.150 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:37:14.150 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:37:14.150 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:37:14.150 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:37:14.151 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.864157+0000", 2023-10-26T01:37:14.151 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:37:14.151 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.151 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.151 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 2, 2023-10-26T01:37:14.152 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:37:14.152 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:37:14.152 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:37:14.152 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:37:14.152 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:37:14.152 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.876919+0000", 2023-10-26T01:37:14.153 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:37:14.153 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.153 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.153 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 3, 2023-10-26T01:37:14.153 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:37:14.153 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:37:14.154 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:37:14.154 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:37:14.154 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:37:14.154 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.974541+0000", 2023-10-26T01:37:14.154 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:37:14.155 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.155 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.155 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 4, 2023-10-26T01:37:14.155 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:37:14.155 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:37:14.155 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:37:14.156 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:37:14.156 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:37:14.156 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.969696+0000", 2023-10-26T01:37:14.156 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:37:14.156 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.156 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.157 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 5, 2023-10-26T01:37:14.157 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:37:14.157 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:37:14.157 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:37:14.157 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:37:14.157 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:37:14.158 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.859703+0000", 2023-10-26T01:37:14.158 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:37:14.158 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.158 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.158 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 6, 2023-10-26T01:37:14.159 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:37:14.159 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:37:14.159 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:37:14.159 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:37:14.159 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:37:14.159 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.867917+0000", 2023-10-26T01:37:14.160 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:37:14.160 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.160 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.160 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 7, 2023-10-26T01:37:14.160 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:37:14.161 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:37:14.161 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:37:14.161 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:37:14.161 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:37:14.161 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.898605+0000", 2023-10-26T01:37:14.161 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:37:14.162 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:14.162 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:37:14.162 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_upmap": [], 2023-10-26T01:37:14.162 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_upmap_items": [], 2023-10-26T01:37:14.162 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_upmap_primaries": [], 2023-10-26T01:37:14.163 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_temp": [], 2023-10-26T01:37:14.163 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_temp": [], 2023-10-26T01:37:14.163 INFO:teuthology.orchestra.run.smithi169.stdout: "blocklist": {}, 2023-10-26T01:37:14.163 INFO:teuthology.orchestra.run.smithi169.stdout: "range_blocklist": {}, 2023-10-26T01:37:14.163 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profiles": { 2023-10-26T01:37:14.164 INFO:teuthology.orchestra.run.smithi169.stdout: "default": { 2023-10-26T01:37:14.164 INFO:teuthology.orchestra.run.smithi169.stdout: "crush-failure-domain": "osd", 2023-10-26T01:37:14.164 INFO:teuthology.orchestra.run.smithi169.stdout: "k": "2", 2023-10-26T01:37:14.164 INFO:teuthology.orchestra.run.smithi169.stdout: "m": "1", 2023-10-26T01:37:14.164 INFO:teuthology.orchestra.run.smithi169.stdout: "plugin": "jerasure", 2023-10-26T01:37:14.164 INFO:teuthology.orchestra.run.smithi169.stdout: "technique": "reed_sol_van" 2023-10-26T01:37:14.165 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:14.165 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.165 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps_queue": [], 2023-10-26T01:37:14.165 INFO:teuthology.orchestra.run.smithi169.stdout: "new_removed_snaps": [], 2023-10-26T01:37:14.165 INFO:teuthology.orchestra.run.smithi169.stdout: "new_purged_snaps": [], 2023-10-26T01:37:14.165 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_node_flags": {}, 2023-10-26T01:37:14.166 INFO:teuthology.orchestra.run.smithi169.stdout: "device_class_flags": {}, 2023-10-26T01:37:14.166 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_mode": { 2023-10-26T01:37:14.166 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_mode_enabled": false, 2023-10-26T01:37:14.166 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_bucket_count": 0, 2023-10-26T01:37:14.166 INFO:teuthology.orchestra.run.smithi169.stdout: "degraded_stretch_mode": 0, 2023-10-26T01:37:14.167 INFO:teuthology.orchestra.run.smithi169.stdout: "recovering_stretch_mode": 0, 2023-10-26T01:37:14.167 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_mode_bucket": 0 2023-10-26T01:37:14.167 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:14.167 INFO:teuthology.orchestra.run.smithi169.stdout:} 2023-10-26T01:37:14.168 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:13.977+0000 7f4eb4b8e640 1 -- 172.21.15.169:0/3417178620 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f4e9006ae40 msgr2=0x7f4e9006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:14.169 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:13.977+0000 7f4eb4b8e640 1 --2- 172.21.15.169:0/3417178620 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f4e9006ae40 0x7f4e9006d300 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f4ea0007fd0 tx=0x7f4ea0009040 comp rx=0 tx=0).stop 2023-10-26T01:37:14.169 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:13.977+0000 7f4eb4b8e640 1 -- 172.21.15.169:0/3417178620 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4eb006cae0 msgr2=0x7f4eb00aff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:14.169 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:13.977+0000 7f4eb4b8e640 1 --2- 172.21.15.169:0/3417178620 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4eb006cae0 0x7f4eb00aff00 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f4e9800c970 tx=0x7f4e9800ce30 comp rx=0 tx=0).stop 2023-10-26T01:37:14.169 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:13.977+0000 7f4eb4b8e640 1 -- 172.21.15.169:0/3417178620 shutdown_connections 2023-10-26T01:37:14.169 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:13.977+0000 7f4eb4b8e640 1 --2- 172.21.15.169:0/3417178620 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f4e9006ae40 0x7f4e9006d300 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:14.170 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:13.977+0000 7f4eb4b8e640 1 --2- 172.21.15.169:0/3417178620 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4eb006cae0 0x7f4eb00aff00 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:14.170 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:13.977+0000 7f4eb4b8e640 1 --2- 172.21.15.169:0/3417178620 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4eb00c9e00 0x7f4eb00b0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:14.170 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:13.977+0000 7f4eb4b8e640 1 --2- 172.21.15.169:0/3417178620 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4eb006a990 0x7f4eb00b25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:14.170 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:13.977+0000 7f4eb4b8e640 1 -- 172.21.15.169:0/3417178620 >> 172.21.15.169:0/3417178620 conn(0x7f4eb00aedd0 msgr2=0x7f4eb0159a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:14.170 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:13.977+0000 7f4eb4b8e640 1 -- 172.21.15.169:0/3417178620 shutdown_connections 2023-10-26T01:37:14.170 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:13.977+0000 7f4eb4b8e640 1 -- 172.21.15.169:0/3417178620 wait complete. 2023-10-26T01:37:14.171 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-10-26T01:37:14.171 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-26T01:37:14.401 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:14.398+0000 7fc431bb0640 1 -- 172.21.15.169:0/2740501413 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc42c070ce0 msgr2=0x7fc42c15da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:14.402 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:14.398+0000 7fc431bb0640 1 --2- 172.21.15.169:0/2740501413 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc42c070ce0 0x7fc42c15da30 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc41c009830 tx=0x7fc41c02bd60 comp rx=0 tx=0).stop 2023-10-26T01:37:14.402 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:14.398+0000 7fc431bb0640 1 -- 172.21.15.169:0/2740501413 shutdown_connections 2023-10-26T01:37:14.402 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:14.398+0000 7fc431bb0640 1 --2- 172.21.15.169:0/2740501413 >> v2:172.21.15.170:3300/0 conn(0x7fc42c0710c0 0x7fc42c13edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:14.402 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:14.398+0000 7fc431bb0640 1 --2- 172.21.15.169:0/2740501413 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc42c070ce0 0x7fc42c15da30 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:14.403 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:14.398+0000 7fc431bb0640 1 -- 172.21.15.169:0/2740501413 >> 172.21.15.169:0/2740501413 conn(0x7fc42c069510 msgr2=0x7fc42c0698f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:14.403 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:14.398+0000 7fc431bb0640 1 -- 172.21.15.169:0/2740501413 shutdown_connections 2023-10-26T01:37:14.403 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:14.398+0000 7fc431bb0640 1 -- 172.21.15.169:0/2740501413 wait complete. 2023-10-26T01:37:14.405 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:14.400+0000 7fc431bb0640 1 Processor -- start 2023-10-26T01:37:14.405 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:14.400+0000 7fc431bb0640 1 -- start start 2023-10-26T01:37:14.406 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:14.401+0000 7fc431bb0640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc42c070ce0 0x7fc42c136110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:14.406 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:14.401+0000 7fc431bb0640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc42c0710c0 0x7fc42c0b2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:14.406 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:14.401+0000 7fc431bb0640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc42c13f310 0x7fc42c1f16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:14.406 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:14.401+0000 7fc431bb0640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc42c0afc00 con 0x7fc42c13f310 2023-10-26T01:37:14.407 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:14.401+0000 7fc431bb0640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc42c1366f0 con 0x7fc42c0710c0 2023-10-26T01:37:14.407 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:14.401+0000 7fc431bb0640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc42c0b2a60 con 0x7fc42c070ce0 2023-10-26T01:37:14.407 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:14.401+0000 7fc42bfff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc42c13f310 0x7fc42c1f16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:14.407 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:14.402+0000 7fc42bfff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc42c13f310 0x7fc42c1f16d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:60972/0 (socket says 172.21.15.169:60972) 2023-10-26T01:37:14.407 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:14.402+0000 7fc42bfff640 1 -- 172.21.15.169:0/1839473439 learned_addr learned my addr 172.21.15.169:0/1839473439 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:37:14.408 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:14.402+0000 7fc42b7fe640 1 --2- 172.21.15.169:0/1839473439 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc42c070ce0 0x7fc42c136110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:14.408 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:14.402+0000 7fc422ffd640 1 --2- 172.21.15.169:0/1839473439 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc42c0710c0 0x7fc42c0b2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:14.408 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:14.402+0000 7fc42bfff640 1 -- 172.21.15.169:0/1839473439 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc42c070ce0 msgr2=0x7fc42c136110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:14.408 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:14.402+0000 7fc42bfff640 1 --2- 172.21.15.169:0/1839473439 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc42c070ce0 0x7fc42c136110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:14.408 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:14.402+0000 7fc42bfff640 1 -- 172.21.15.169:0/1839473439 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc42c0710c0 msgr2=0x7fc42c0b2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:14.409 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:14.402+0000 7fc42bfff640 1 --2- 172.21.15.169:0/1839473439 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc42c0710c0 0x7fc42c0b2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:14.409 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:14.402+0000 7fc42bfff640 1 -- 172.21.15.169:0/1839473439 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc41c009490 con 0x7fc42c13f310 2023-10-26T01:37:14.409 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:14.402+0000 7fc42bfff640 1 --2- 172.21.15.169:0/1839473439 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc42c13f310 0x7fc42c1f16d0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fc42400c7a0 tx=0x7fc42400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:14.409 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:14.402+0000 7fc428ff9640 1 -- 172.21.15.169:0/1839473439 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc424003b70 con 0x7fc42c13f310 2023-10-26T01:37:14.409 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:14.403+0000 7fc428ff9640 1 -- 172.21.15.169:0/1839473439 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc424003cf0 con 0x7fc42c13f310 2023-10-26T01:37:14.410 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:14.403+0000 7fc431bb0640 1 -- 172.21.15.169:0/1839473439 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc42c159c10 con 0x7fc42c13f310 2023-10-26T01:37:14.410 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:14.403+0000 7fc428ff9640 1 -- 172.21.15.169:0/1839473439 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc42400aab0 con 0x7fc42c13f310 2023-10-26T01:37:14.410 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:14.403+0000 7fc431bb0640 1 -- 172.21.15.169:0/1839473439 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc42c15a1d0 con 0x7fc42c13f310 2023-10-26T01:37:14.411 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:14.404+0000 7fc431bb0640 1 -- 172.21.15.169:0/1839473439 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc42c04db10 con 0x7fc42c13f310 2023-10-26T01:37:14.411 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:14.405+0000 7fc428ff9640 1 -- 172.21.15.169:0/1839473439 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fc42400ac10 con 0x7fc42c13f310 2023-10-26T01:37:14.411 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:14.406+0000 7fc428ff9640 1 --2- 172.21.15.169:0/1839473439 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc40c06ae40 0x7fc40c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:14.412 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:14.406+0000 7fc428ff9640 1 -- 172.21.15.169:0/1839473439 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc424089690 con 0x7fc42c13f310 2023-10-26T01:37:14.412 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:14.406+0000 7fc42b7fe640 1 --2- 172.21.15.169:0/1839473439 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc40c06ae40 0x7fc40c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:14.412 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:14.407+0000 7fc42b7fe640 1 --2- 172.21.15.169:0/1839473439 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc40c06ae40 0x7fc40c06d300 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fc41c036000 tx=0x7fc41c004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:14.412 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:14.410+0000 7fc428ff9640 1 -- 172.21.15.169:0/1839473439 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc4240546b0 con 0x7fc42c13f310 2023-10-26T01:37:14.602 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:14.599+0000 7fc431bb0640 1 -- 172.21.15.169:0/1839473439 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fc42c13e240 con 0x7fc42c13f310 2023-10-26T01:37:14.604 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:14.601+0000 7fc428ff9640 1 -- 172.21.15.169:0/1839473439 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v109) v1 ==== 81+0+34055 (secure 0 0 0) 0x7fc424057cd0 con 0x7fc42c13f310 2023-10-26T01:37:14.605 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:37:14.605 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2023-10-26T01:37:14.605 INFO:teuthology.orchestra.run.smithi169.stdout: "epoch": 109, 2023-10-26T01:37:14.605 INFO:teuthology.orchestra.run.smithi169.stdout: "fsid": "30203b28-ed8a-44e8-a9a6-a9bf25f980f1", 2023-10-26T01:37:14.606 INFO:teuthology.orchestra.run.smithi169.stdout: "created": "2023-10-26T01:30:22.897155+0000", 2023-10-26T01:37:14.606 INFO:teuthology.orchestra.run.smithi169.stdout: "modified": "2023-10-26T01:37:13.181064+0000", 2023-10-26T01:37:14.606 INFO:teuthology.orchestra.run.smithi169.stdout: "last_up_change": "2023-10-26T01:30:31.875281+0000", 2023-10-26T01:37:14.606 INFO:teuthology.orchestra.run.smithi169.stdout: "last_in_change": "2023-10-26T01:30:27.637941+0000", 2023-10-26T01:37:14.606 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-26T01:37:14.607 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_num": 5799936, 2023-10-26T01:37:14.607 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_set": [ 2023-10-26T01:37:14.607 INFO:teuthology.orchestra.run.smithi169.stdout: "pglog_hardlimit", 2023-10-26T01:37:14.607 INFO:teuthology.orchestra.run.smithi169.stdout: "purged_snapdirs", 2023-10-26T01:37:14.607 INFO:teuthology.orchestra.run.smithi169.stdout: "recovery_deletes", 2023-10-26T01:37:14.607 INFO:teuthology.orchestra.run.smithi169.stdout: "sortbitwise" 2023-10-26T01:37:14.608 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:37:14.608 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_version": 4, 2023-10-26T01:37:14.608 INFO:teuthology.orchestra.run.smithi169.stdout: "full_ratio": 0.69999998807907104, 2023-10-26T01:37:14.608 INFO:teuthology.orchestra.run.smithi169.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-26T01:37:14.608 INFO:teuthology.orchestra.run.smithi169.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-26T01:37:14.608 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_snapshot": "", 2023-10-26T01:37:14.609 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_max": 13, 2023-10-26T01:37:14.609 INFO:teuthology.orchestra.run.smithi169.stdout: "max_osd": 8, 2023-10-26T01:37:14.609 INFO:teuthology.orchestra.run.smithi169.stdout: "require_min_compat_client": "luminous", 2023-10-26T01:37:14.609 INFO:teuthology.orchestra.run.smithi169.stdout: "min_compat_client": "jewel", 2023-10-26T01:37:14.609 INFO:teuthology.orchestra.run.smithi169.stdout: "require_osd_release": "reef", 2023-10-26T01:37:14.609 INFO:teuthology.orchestra.run.smithi169.stdout: "allow_crimson": false, 2023-10-26T01:37:14.610 INFO:teuthology.orchestra.run.smithi169.stdout: "pools": [ 2023-10-26T01:37:14.610 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.610 INFO:teuthology.orchestra.run.smithi169.stdout: "pool": 1, 2023-10-26T01:37:14.610 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_name": ".mgr", 2023-10-26T01:37:14.610 INFO:teuthology.orchestra.run.smithi169.stdout: "create_time": "2023-10-26T01:30:32.200808+0000", 2023-10-26T01:37:14.611 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": 1, 2023-10-26T01:37:14.611 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_names": "hashpspool", 2023-10-26T01:37:14.611 INFO:teuthology.orchestra.run.smithi169.stdout: "type": 1, 2023-10-26T01:37:14.611 INFO:teuthology.orchestra.run.smithi169.stdout: "size": 2, 2023-10-26T01:37:14.611 INFO:teuthology.orchestra.run.smithi169.stdout: "min_size": 1, 2023-10-26T01:37:14.611 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_rule": 0, 2023-10-26T01:37:14.612 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_count": 0, 2023-10-26T01:37:14.612 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_target": 0, 2023-10-26T01:37:14.612 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T01:37:14.612 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T01:37:14.612 INFO:teuthology.orchestra.run.smithi169.stdout: "object_hash": 2, 2023-10-26T01:37:14.612 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_mode": "off", 2023-10-26T01:37:14.613 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num": 1, 2023-10-26T01:37:14.613 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num": 1, 2023-10-26T01:37:14.613 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num_target": 1, 2023-10-26T01:37:14.613 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_target": 1, 2023-10-26T01:37:14.613 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_pending": 1, 2023-10-26T01:37:14.613 INFO:teuthology.orchestra.run.smithi169.stdout: "last_pg_merge_meta": { 2023-10-26T01:37:14.614 INFO:teuthology.orchestra.run.smithi169.stdout: "source_pgid": "0.0", 2023-10-26T01:37:14.614 INFO:teuthology.orchestra.run.smithi169.stdout: "ready_epoch": 0, 2023-10-26T01:37:14.614 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_started": 0, 2023-10-26T01:37:14.614 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_clean": 0, 2023-10-26T01:37:14.614 INFO:teuthology.orchestra.run.smithi169.stdout: "source_version": "0'0", 2023-10-26T01:37:14.615 INFO:teuthology.orchestra.run.smithi169.stdout: "target_version": "0'0" 2023-10-26T01:37:14.615 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.615 INFO:teuthology.orchestra.run.smithi169.stdout: "last_change": "109", 2023-10-26T01:37:14.615 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend": "0", 2023-10-26T01:37:14.615 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T01:37:14.615 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T01:37:14.616 INFO:teuthology.orchestra.run.smithi169.stdout: "auid": 0, 2023-10-26T01:37:14.616 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_mode": "selfmanaged", 2023-10-26T01:37:14.616 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_seq": 0, 2023-10-26T01:37:14.616 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_epoch": 0, 2023-10-26T01:37:14.616 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_snaps": [], 2023-10-26T01:37:14.616 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps": "[]", 2023-10-26T01:37:14.617 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_bytes": 0, 2023-10-26T01:37:14.617 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_objects": 0, 2023-10-26T01:37:14.617 INFO:teuthology.orchestra.run.smithi169.stdout: "tiers": [], 2023-10-26T01:37:14.617 INFO:teuthology.orchestra.run.smithi169.stdout: "tier_of": -1, 2023-10-26T01:37:14.617 INFO:teuthology.orchestra.run.smithi169.stdout: "read_tier": -1, 2023-10-26T01:37:14.617 INFO:teuthology.orchestra.run.smithi169.stdout: "write_tier": -1, 2023-10-26T01:37:14.617 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_mode": "none", 2023-10-26T01:37:14.618 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_bytes": 0, 2023-10-26T01:37:14.618 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_objects": 0, 2023-10-26T01:37:14.618 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T01:37:14.618 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T01:37:14.618 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T01:37:14.619 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_flush_age": 0, 2023-10-26T01:37:14.619 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_evict_age": 0, 2023-10-26T01:37:14.619 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profile": "", 2023-10-26T01:37:14.619 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_params": { 2023-10-26T01:37:14.619 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "none" 2023-10-26T01:37:14.619 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.620 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_period": 0, 2023-10-26T01:37:14.620 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_count": 0, 2023-10-26T01:37:14.620 INFO:teuthology.orchestra.run.smithi169.stdout: "use_gmt_hitset": true, 2023-10-26T01:37:14.620 INFO:teuthology.orchestra.run.smithi169.stdout: "min_read_recency_for_promote": 0, 2023-10-26T01:37:14.620 INFO:teuthology.orchestra.run.smithi169.stdout: "min_write_recency_for_promote": 0, 2023-10-26T01:37:14.620 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T01:37:14.621 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_search_last_n": 0, 2023-10-26T01:37:14.621 INFO:teuthology.orchestra.run.smithi169.stdout: "grade_table": [], 2023-10-26T01:37:14.621 INFO:teuthology.orchestra.run.smithi169.stdout: "stripe_width": 0, 2023-10-26T01:37:14.621 INFO:teuthology.orchestra.run.smithi169.stdout: "expected_num_objects": 0, 2023-10-26T01:37:14.621 INFO:teuthology.orchestra.run.smithi169.stdout: "fast_read": false, 2023-10-26T01:37:14.621 INFO:teuthology.orchestra.run.smithi169.stdout: "options": { 2023-10-26T01:37:14.622 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_max": 32, 2023-10-26T01:37:14.622 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_min": 1 2023-10-26T01:37:14.622 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.622 INFO:teuthology.orchestra.run.smithi169.stdout: "application_metadata": { 2023-10-26T01:37:14.622 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": {} 2023-10-26T01:37:14.622 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.623 INFO:teuthology.orchestra.run.smithi169.stdout: "read_balance": { 2023-10-26T01:37:14.623 INFO:teuthology.orchestra.run.smithi169.stdout: "score_acting": 8, 2023-10-26T01:37:14.623 INFO:teuthology.orchestra.run.smithi169.stdout: "score_stable": 8, 2023-10-26T01:37:14.623 INFO:teuthology.orchestra.run.smithi169.stdout: "optimal_score": 0.25, 2023-10-26T01:37:14.623 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_acting": 2, 2023-10-26T01:37:14.624 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_stable": 2, 2023-10-26T01:37:14.624 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity_weighted": 1, 2023-10-26T01:37:14.624 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity": 1, 2023-10-26T01:37:14.624 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity_weighted": 1 2023-10-26T01:37:14.624 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:14.624 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.625 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.625 INFO:teuthology.orchestra.run.smithi169.stdout: "pool": 12, 2023-10-26T01:37:14.625 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_name": "cephfs_metadata", 2023-10-26T01:37:14.625 INFO:teuthology.orchestra.run.smithi169.stdout: "create_time": "2023-10-26T01:36:41.430455+0000", 2023-10-26T01:37:14.625 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": 1, 2023-10-26T01:37:14.625 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_names": "hashpspool", 2023-10-26T01:37:14.626 INFO:teuthology.orchestra.run.smithi169.stdout: "type": 1, 2023-10-26T01:37:14.626 INFO:teuthology.orchestra.run.smithi169.stdout: "size": 2, 2023-10-26T01:37:14.626 INFO:teuthology.orchestra.run.smithi169.stdout: "min_size": 1, 2023-10-26T01:37:14.626 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_rule": 0, 2023-10-26T01:37:14.626 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_count": 0, 2023-10-26T01:37:14.627 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_target": 0, 2023-10-26T01:37:14.627 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T01:37:14.627 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T01:37:14.627 INFO:teuthology.orchestra.run.smithi169.stdout: "object_hash": 2, 2023-10-26T01:37:14.627 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_mode": "off", 2023-10-26T01:37:14.627 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num": 32, 2023-10-26T01:37:14.628 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num": 32, 2023-10-26T01:37:14.628 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num_target": 32, 2023-10-26T01:37:14.628 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_target": 32, 2023-10-26T01:37:14.628 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_pending": 32, 2023-10-26T01:37:14.628 INFO:teuthology.orchestra.run.smithi169.stdout: "last_pg_merge_meta": { 2023-10-26T01:37:14.628 INFO:teuthology.orchestra.run.smithi169.stdout: "source_pgid": "0.0", 2023-10-26T01:37:14.629 INFO:teuthology.orchestra.run.smithi169.stdout: "ready_epoch": 0, 2023-10-26T01:37:14.629 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_started": 0, 2023-10-26T01:37:14.629 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_clean": 0, 2023-10-26T01:37:14.629 INFO:teuthology.orchestra.run.smithi169.stdout: "source_version": "0'0", 2023-10-26T01:37:14.629 INFO:teuthology.orchestra.run.smithi169.stdout: "target_version": "0'0" 2023-10-26T01:37:14.629 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.630 INFO:teuthology.orchestra.run.smithi169.stdout: "last_change": "109", 2023-10-26T01:37:14.630 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend": "0", 2023-10-26T01:37:14.630 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T01:37:14.630 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T01:37:14.630 INFO:teuthology.orchestra.run.smithi169.stdout: "auid": 0, 2023-10-26T01:37:14.631 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_mode": "selfmanaged", 2023-10-26T01:37:14.631 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_seq": 0, 2023-10-26T01:37:14.631 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_epoch": 0, 2023-10-26T01:37:14.631 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_snaps": [], 2023-10-26T01:37:14.631 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps": "[]", 2023-10-26T01:37:14.632 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_bytes": 0, 2023-10-26T01:37:14.632 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_objects": 0, 2023-10-26T01:37:14.632 INFO:teuthology.orchestra.run.smithi169.stdout: "tiers": [], 2023-10-26T01:37:14.632 INFO:teuthology.orchestra.run.smithi169.stdout: "tier_of": -1, 2023-10-26T01:37:14.632 INFO:teuthology.orchestra.run.smithi169.stdout: "read_tier": -1, 2023-10-26T01:37:14.632 INFO:teuthology.orchestra.run.smithi169.stdout: "write_tier": -1, 2023-10-26T01:37:14.633 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_mode": "none", 2023-10-26T01:37:14.633 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_bytes": 0, 2023-10-26T01:37:14.633 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_objects": 0, 2023-10-26T01:37:14.633 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T01:37:14.633 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T01:37:14.633 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T01:37:14.634 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_flush_age": 0, 2023-10-26T01:37:14.634 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_evict_age": 0, 2023-10-26T01:37:14.634 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profile": "", 2023-10-26T01:37:14.634 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_params": { 2023-10-26T01:37:14.634 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "none" 2023-10-26T01:37:14.635 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.635 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_period": 0, 2023-10-26T01:37:14.635 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_count": 0, 2023-10-26T01:37:14.635 INFO:teuthology.orchestra.run.smithi169.stdout: "use_gmt_hitset": true, 2023-10-26T01:37:14.635 INFO:teuthology.orchestra.run.smithi169.stdout: "min_read_recency_for_promote": 0, 2023-10-26T01:37:14.636 INFO:teuthology.orchestra.run.smithi169.stdout: "min_write_recency_for_promote": 0, 2023-10-26T01:37:14.636 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T01:37:14.636 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_search_last_n": 0, 2023-10-26T01:37:14.636 INFO:teuthology.orchestra.run.smithi169.stdout: "grade_table": [], 2023-10-26T01:37:14.636 INFO:teuthology.orchestra.run.smithi169.stdout: "stripe_width": 0, 2023-10-26T01:37:14.636 INFO:teuthology.orchestra.run.smithi169.stdout: "expected_num_objects": 0, 2023-10-26T01:37:14.637 INFO:teuthology.orchestra.run.smithi169.stdout: "fast_read": false, 2023-10-26T01:37:14.637 INFO:teuthology.orchestra.run.smithi169.stdout: "options": { 2023-10-26T01:37:14.637 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_bias": 4, 2023-10-26T01:37:14.637 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_min": 16, 2023-10-26T01:37:14.637 INFO:teuthology.orchestra.run.smithi169.stdout: "recovery_priority": 5 2023-10-26T01:37:14.637 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.638 INFO:teuthology.orchestra.run.smithi169.stdout: "application_metadata": { 2023-10-26T01:37:14.638 INFO:teuthology.orchestra.run.smithi169.stdout: "cephfs": { 2023-10-26T01:37:14.638 INFO:teuthology.orchestra.run.smithi169.stdout: "metadata": "cephfs" 2023-10-26T01:37:14.638 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:14.638 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.639 INFO:teuthology.orchestra.run.smithi169.stdout: "read_balance": { 2023-10-26T01:37:14.639 INFO:teuthology.orchestra.run.smithi169.stdout: "score_acting": 2, 2023-10-26T01:37:14.639 INFO:teuthology.orchestra.run.smithi169.stdout: "score_stable": 2, 2023-10-26T01:37:14.639 INFO:teuthology.orchestra.run.smithi169.stdout: "optimal_score": 1, 2023-10-26T01:37:14.639 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_acting": 2, 2023-10-26T01:37:14.639 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_stable": 2, 2023-10-26T01:37:14.640 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity_weighted": 1, 2023-10-26T01:37:14.640 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity": 1, 2023-10-26T01:37:14.640 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity_weighted": 1 2023-10-26T01:37:14.640 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:14.640 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.641 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.641 INFO:teuthology.orchestra.run.smithi169.stdout: "pool": 13, 2023-10-26T01:37:14.641 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_name": "cephfs_data", 2023-10-26T01:37:14.641 INFO:teuthology.orchestra.run.smithi169.stdout: "create_time": "2023-10-26T01:36:42.883252+0000", 2023-10-26T01:37:14.641 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": 1, 2023-10-26T01:37:14.641 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_names": "hashpspool", 2023-10-26T01:37:14.642 INFO:teuthology.orchestra.run.smithi169.stdout: "type": 1, 2023-10-26T01:37:14.642 INFO:teuthology.orchestra.run.smithi169.stdout: "size": 2, 2023-10-26T01:37:14.642 INFO:teuthology.orchestra.run.smithi169.stdout: "min_size": 1, 2023-10-26T01:37:14.642 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_rule": 0, 2023-10-26T01:37:14.642 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_count": 0, 2023-10-26T01:37:14.643 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_target": 0, 2023-10-26T01:37:14.643 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T01:37:14.643 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T01:37:14.643 INFO:teuthology.orchestra.run.smithi169.stdout: "object_hash": 2, 2023-10-26T01:37:14.643 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_mode": "off", 2023-10-26T01:37:14.644 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num": 64, 2023-10-26T01:37:14.644 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num": 64, 2023-10-26T01:37:14.644 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num_target": 64, 2023-10-26T01:37:14.644 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_target": 64, 2023-10-26T01:37:14.644 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_pending": 64, 2023-10-26T01:37:14.644 INFO:teuthology.orchestra.run.smithi169.stdout: "last_pg_merge_meta": { 2023-10-26T01:37:14.645 INFO:teuthology.orchestra.run.smithi169.stdout: "source_pgid": "0.0", 2023-10-26T01:37:14.645 INFO:teuthology.orchestra.run.smithi169.stdout: "ready_epoch": 0, 2023-10-26T01:37:14.645 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_started": 0, 2023-10-26T01:37:14.645 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_clean": 0, 2023-10-26T01:37:14.645 INFO:teuthology.orchestra.run.smithi169.stdout: "source_version": "0'0", 2023-10-26T01:37:14.645 INFO:teuthology.orchestra.run.smithi169.stdout: "target_version": "0'0" 2023-10-26T01:37:14.646 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.646 INFO:teuthology.orchestra.run.smithi169.stdout: "last_change": "109", 2023-10-26T01:37:14.646 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend": "0", 2023-10-26T01:37:14.646 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T01:37:14.646 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T01:37:14.647 INFO:teuthology.orchestra.run.smithi169.stdout: "auid": 0, 2023-10-26T01:37:14.647 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_mode": "selfmanaged", 2023-10-26T01:37:14.647 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_seq": 0, 2023-10-26T01:37:14.647 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_epoch": 0, 2023-10-26T01:37:14.647 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_snaps": [], 2023-10-26T01:37:14.647 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps": "[]", 2023-10-26T01:37:14.648 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_bytes": 0, 2023-10-26T01:37:14.648 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_objects": 0, 2023-10-26T01:37:14.648 INFO:teuthology.orchestra.run.smithi169.stdout: "tiers": [], 2023-10-26T01:37:14.648 INFO:teuthology.orchestra.run.smithi169.stdout: "tier_of": -1, 2023-10-26T01:37:14.648 INFO:teuthology.orchestra.run.smithi169.stdout: "read_tier": -1, 2023-10-26T01:37:14.649 INFO:teuthology.orchestra.run.smithi169.stdout: "write_tier": -1, 2023-10-26T01:37:14.649 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_mode": "none", 2023-10-26T01:37:14.649 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_bytes": 0, 2023-10-26T01:37:14.649 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_objects": 0, 2023-10-26T01:37:14.649 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T01:37:14.650 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T01:37:14.650 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T01:37:14.650 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_flush_age": 0, 2023-10-26T01:37:14.650 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_evict_age": 0, 2023-10-26T01:37:14.650 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profile": "", 2023-10-26T01:37:14.651 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_params": { 2023-10-26T01:37:14.651 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "none" 2023-10-26T01:37:14.651 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.651 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_period": 0, 2023-10-26T01:37:14.651 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_count": 0, 2023-10-26T01:37:14.651 INFO:teuthology.orchestra.run.smithi169.stdout: "use_gmt_hitset": true, 2023-10-26T01:37:14.652 INFO:teuthology.orchestra.run.smithi169.stdout: "min_read_recency_for_promote": 0, 2023-10-26T01:37:14.652 INFO:teuthology.orchestra.run.smithi169.stdout: "min_write_recency_for_promote": 0, 2023-10-26T01:37:14.652 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T01:37:14.652 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_search_last_n": 0, 2023-10-26T01:37:14.652 INFO:teuthology.orchestra.run.smithi169.stdout: "grade_table": [], 2023-10-26T01:37:14.653 INFO:teuthology.orchestra.run.smithi169.stdout: "stripe_width": 0, 2023-10-26T01:37:14.653 INFO:teuthology.orchestra.run.smithi169.stdout: "expected_num_objects": 0, 2023-10-26T01:37:14.653 INFO:teuthology.orchestra.run.smithi169.stdout: "fast_read": false, 2023-10-26T01:37:14.653 INFO:teuthology.orchestra.run.smithi169.stdout: "options": { 2023-10-26T01:37:14.653 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_min": 64, 2023-10-26T01:37:14.654 INFO:teuthology.orchestra.run.smithi169.stdout: "target_size_ratio": 0.89999997615814209 2023-10-26T01:37:14.654 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.654 INFO:teuthology.orchestra.run.smithi169.stdout: "application_metadata": { 2023-10-26T01:37:14.654 INFO:teuthology.orchestra.run.smithi169.stdout: "cephfs": { 2023-10-26T01:37:14.654 INFO:teuthology.orchestra.run.smithi169.stdout: "data": "cephfs" 2023-10-26T01:37:14.655 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:14.655 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.655 INFO:teuthology.orchestra.run.smithi169.stdout: "read_balance": { 2023-10-26T01:37:14.655 INFO:teuthology.orchestra.run.smithi169.stdout: "score_acting": 1.3799999952316284, 2023-10-26T01:37:14.655 INFO:teuthology.orchestra.run.smithi169.stdout: "score_stable": 1.3799999952316284, 2023-10-26T01:37:14.655 INFO:teuthology.orchestra.run.smithi169.stdout: "optimal_score": 1, 2023-10-26T01:37:14.656 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_acting": 1.3799999952316284, 2023-10-26T01:37:14.656 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_stable": 1.3799999952316284, 2023-10-26T01:37:14.656 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity_weighted": 1, 2023-10-26T01:37:14.656 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity": 1, 2023-10-26T01:37:14.656 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity_weighted": 1 2023-10-26T01:37:14.657 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:14.657 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:14.657 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:37:14.657 INFO:teuthology.orchestra.run.smithi169.stdout: "osds": [ 2023-10-26T01:37:14.657 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.658 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 0, 2023-10-26T01:37:14.658 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:37:14.658 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:37:14.658 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:37:14.658 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:37:14.659 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:37:14.659 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:37:14.659 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:37:14.659 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:37:14.659 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 101, 2023-10-26T01:37:14.659 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:37:14.660 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:37:14.660 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:37:14.660 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:14.660 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.660 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:14.660 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6816", 2023-10-26T01:37:14.661 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:37:14.661 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.661 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.661 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:14.661 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6817", 2023-10-26T01:37:14.662 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:37:14.662 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:14.662 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:14.662 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.662 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:37:14.663 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:14.663 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.663 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:14.663 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6818", 2023-10-26T01:37:14.663 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:37:14.663 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.664 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.664 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:14.664 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6819", 2023-10-26T01:37:14.664 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:37:14.664 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:14.665 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:14.665 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.665 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:37:14.665 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:14.665 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.665 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:14.666 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6822", 2023-10-26T01:37:14.666 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:37:14.666 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.666 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.666 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:14.667 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6823", 2023-10-26T01:37:14.667 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:37:14.667 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:14.667 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:14.667 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.668 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:37:14.668 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:14.668 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.668 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:14.668 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6820", 2023-10-26T01:37:14.668 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:37:14.669 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.669 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.669 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:14.669 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6821", 2023-10-26T01:37:14.669 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:37:14.669 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:14.670 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:14.670 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.670 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6817/4204255864", 2023-10-26T01:37:14.670 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6819/4204255864", 2023-10-26T01:37:14.670 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6823/4204255864", 2023-10-26T01:37:14.671 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6821/4204255864", 2023-10-26T01:37:14.671 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:37:14.671 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:37:14.671 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:37:14.671 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:14.672 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.672 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.672 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 1, 2023-10-26T01:37:14.672 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:37:14.672 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:37:14.672 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:37:14.673 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:37:14.673 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:37:14.673 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:37:14.673 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:37:14.674 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:37:14.674 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 101, 2023-10-26T01:37:14.674 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:37:14.674 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:37:14.674 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:37:14.674 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:14.675 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.675 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:14.675 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6808", 2023-10-26T01:37:14.675 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:37:14.675 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.676 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.676 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:14.676 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6809", 2023-10-26T01:37:14.676 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:37:14.677 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:14.677 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:14.677 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.677 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:37:14.677 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:14.678 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.678 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:14.678 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6810", 2023-10-26T01:37:14.678 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:37:14.678 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.679 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.679 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:14.679 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6811", 2023-10-26T01:37:14.679 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:37:14.679 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:14.679 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:14.680 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.680 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:37:14.680 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:14.680 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.680 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:14.680 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6814", 2023-10-26T01:37:14.681 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:37:14.681 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.681 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.681 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:14.681 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6815", 2023-10-26T01:37:14.681 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:37:14.682 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:14.682 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:14.682 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.682 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:37:14.682 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:14.683 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.683 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:14.683 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6812", 2023-10-26T01:37:14.683 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:37:14.683 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.684 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.684 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:14.684 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6813", 2023-10-26T01:37:14.684 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:37:14.684 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:14.684 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:14.685 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.685 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6809/238206046", 2023-10-26T01:37:14.685 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6811/238206046", 2023-10-26T01:37:14.685 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6815/238206046", 2023-10-26T01:37:14.685 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6813/238206046", 2023-10-26T01:37:14.685 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:37:14.686 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:37:14.686 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:37:14.686 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:14.686 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.686 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.686 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 2, 2023-10-26T01:37:14.687 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:37:14.687 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:37:14.687 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:37:14.687 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:37:14.687 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:37:14.687 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:37:14.688 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:37:14.688 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:37:14.688 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 101, 2023-10-26T01:37:14.688 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:37:14.688 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:37:14.689 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:37:14.689 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:14.689 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.689 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:14.689 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6800", 2023-10-26T01:37:14.689 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:37:14.690 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.690 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.690 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:14.690 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6801", 2023-10-26T01:37:14.690 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:37:14.691 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:14.691 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:14.691 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.691 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:37:14.691 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:14.692 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.692 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:14.692 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6802", 2023-10-26T01:37:14.692 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:37:14.692 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.693 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.693 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:14.693 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6803", 2023-10-26T01:37:14.693 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:37:14.693 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:14.693 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:14.694 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.694 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:37:14.694 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:14.694 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.694 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:14.695 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6806", 2023-10-26T01:37:14.695 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:37:14.695 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.695 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.695 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:14.696 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6807", 2023-10-26T01:37:14.696 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:37:14.696 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:14.696 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:14.696 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.697 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:37:14.697 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:14.697 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.697 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:14.697 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6804", 2023-10-26T01:37:14.698 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:37:14.698 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.698 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.698 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:14.698 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6805", 2023-10-26T01:37:14.698 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:37:14.699 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:14.699 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:14.699 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.699 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6801/3215791846", 2023-10-26T01:37:14.699 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6803/3215791846", 2023-10-26T01:37:14.700 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6807/3215791846", 2023-10-26T01:37:14.700 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6805/3215791846", 2023-10-26T01:37:14.700 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:37:14.700 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:37:14.700 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:37:14.701 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:14.701 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.701 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.701 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 3, 2023-10-26T01:37:14.701 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:37:14.701 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:37:14.702 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:37:14.702 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:37:14.702 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:37:14.702 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:37:14.702 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:37:14.703 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 14, 2023-10-26T01:37:14.703 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 101, 2023-10-26T01:37:14.703 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:37:14.703 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:37:14.703 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:37:14.703 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:14.704 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.704 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:14.704 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6824", 2023-10-26T01:37:14.704 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:37:14.704 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.704 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.704 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:14.705 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6825", 2023-10-26T01:37:14.705 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:37:14.705 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:14.705 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:14.705 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.705 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:37:14.706 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:14.706 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.706 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:14.706 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6826", 2023-10-26T01:37:14.706 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:37:14.707 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.707 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.707 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:14.707 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6827", 2023-10-26T01:37:14.707 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:37:14.707 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:14.708 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:14.708 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.708 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:37:14.708 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:14.708 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.709 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:14.709 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6830", 2023-10-26T01:37:14.709 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:37:14.709 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.709 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.709 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:14.710 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6831", 2023-10-26T01:37:14.710 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:37:14.710 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:14.710 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:14.710 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.710 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:37:14.711 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:14.711 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.711 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:14.711 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6828", 2023-10-26T01:37:14.711 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:37:14.711 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.712 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.712 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:14.712 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6829", 2023-10-26T01:37:14.712 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:37:14.712 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:14.712 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:14.713 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.713 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6825/2256701226", 2023-10-26T01:37:14.713 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6827/2256701226", 2023-10-26T01:37:14.713 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6831/2256701226", 2023-10-26T01:37:14.713 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6829/2256701226", 2023-10-26T01:37:14.713 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:37:14.714 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:37:14.714 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:37:14.714 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:14.714 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.714 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.715 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 4, 2023-10-26T01:37:14.715 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:37:14.715 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:37:14.715 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:37:14.715 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:37:14.715 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:37:14.716 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:37:14.716 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:37:14.716 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:37:14.716 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 101, 2023-10-26T01:37:14.716 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:37:14.716 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:37:14.717 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:37:14.717 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:14.717 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.717 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:14.717 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6816", 2023-10-26T01:37:14.718 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:37:14.718 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.718 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.718 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:14.718 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6817", 2023-10-26T01:37:14.718 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:37:14.719 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:14.719 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:14.719 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.719 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:37:14.719 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:14.719 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.720 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:14.720 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6818", 2023-10-26T01:37:14.720 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:37:14.720 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.720 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.720 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:14.721 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6819", 2023-10-26T01:37:14.721 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:37:14.721 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:14.721 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:14.721 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.721 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:37:14.722 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:14.722 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.722 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:14.722 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6825", 2023-10-26T01:37:14.722 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:37:14.723 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.723 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.723 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:14.723 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6828", 2023-10-26T01:37:14.723 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:37:14.724 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:14.724 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:14.724 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.724 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:37:14.724 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:14.724 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.725 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:14.725 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6820", 2023-10-26T01:37:14.725 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:37:14.725 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.725 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.725 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:14.726 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6823", 2023-10-26T01:37:14.726 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:37:14.726 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:14.726 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:14.726 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.727 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6817/1082279874", 2023-10-26T01:37:14.727 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6819/1082279874", 2023-10-26T01:37:14.727 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6828/1082279874", 2023-10-26T01:37:14.727 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6823/1082279874", 2023-10-26T01:37:14.727 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:37:14.728 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:37:14.728 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:37:14.728 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:14.728 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.728 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.729 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 5, 2023-10-26T01:37:14.729 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:37:14.729 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:37:14.729 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:37:14.729 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:37:14.729 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:37:14.730 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:37:14.730 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:37:14.730 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:37:14.730 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 101, 2023-10-26T01:37:14.730 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:37:14.730 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:37:14.731 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:37:14.731 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:14.731 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.731 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:14.731 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6808", 2023-10-26T01:37:14.731 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:37:14.732 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.732 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.732 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:14.732 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6809", 2023-10-26T01:37:14.732 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:37:14.733 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:14.733 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:14.733 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.733 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:37:14.733 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:14.733 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.734 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:14.734 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6810", 2023-10-26T01:37:14.734 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:37:14.734 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.734 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.734 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:14.735 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6811", 2023-10-26T01:37:14.735 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:37:14.735 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:14.735 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:14.735 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.735 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:37:14.736 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:14.736 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.736 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:14.736 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6814", 2023-10-26T01:37:14.736 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:37:14.736 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.737 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.737 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:14.737 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6815", 2023-10-26T01:37:14.737 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:37:14.737 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:14.738 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:14.738 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.738 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:37:14.738 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:14.738 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.738 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:14.739 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6812", 2023-10-26T01:37:14.739 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:37:14.739 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.739 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.739 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:14.739 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6813", 2023-10-26T01:37:14.740 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:37:14.740 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:14.740 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:14.740 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.740 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6809/3335654351", 2023-10-26T01:37:14.740 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6811/3335654351", 2023-10-26T01:37:14.741 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6815/3335654351", 2023-10-26T01:37:14.741 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6813/3335654351", 2023-10-26T01:37:14.741 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:37:14.741 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:37:14.741 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:37:14.742 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:14.742 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.742 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.742 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 6, 2023-10-26T01:37:14.742 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:37:14.742 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:37:14.743 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:37:14.743 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:37:14.743 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:37:14.743 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:37:14.743 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:37:14.744 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:37:14.744 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 101, 2023-10-26T01:37:14.744 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:37:14.744 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:37:14.744 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:37:14.744 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:14.745 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.745 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:14.745 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6800", 2023-10-26T01:37:14.745 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:37:14.745 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.745 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.746 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:14.746 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6801", 2023-10-26T01:37:14.746 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:37:14.746 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:14.746 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:14.746 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.747 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:37:14.747 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:14.747 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.747 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:14.747 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6802", 2023-10-26T01:37:14.748 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:37:14.748 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.748 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.748 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:14.748 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6803", 2023-10-26T01:37:14.749 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:37:14.749 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:14.749 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:14.749 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.749 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:37:14.750 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:14.750 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.750 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:14.750 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6806", 2023-10-26T01:37:14.750 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:37:14.750 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.751 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.751 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:14.751 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6807", 2023-10-26T01:37:14.751 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:37:14.751 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:14.751 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:14.752 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.752 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:37:14.752 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:14.752 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.752 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:14.752 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6804", 2023-10-26T01:37:14.753 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:37:14.753 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.753 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.753 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:14.753 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6805", 2023-10-26T01:37:14.754 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:37:14.754 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:14.754 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:14.754 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.754 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6801/2520460697", 2023-10-26T01:37:14.754 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6803/2520460697", 2023-10-26T01:37:14.755 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6807/2520460697", 2023-10-26T01:37:14.755 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6805/2520460697", 2023-10-26T01:37:14.755 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:37:14.755 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:37:14.755 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:37:14.755 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:14.756 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.756 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.756 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 7, 2023-10-26T01:37:14.756 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:37:14.756 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:37:14.756 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:37:14.757 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:37:14.757 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:37:14.757 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:37:14.757 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:37:14.757 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:37:14.757 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 101, 2023-10-26T01:37:14.758 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:37:14.758 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:37:14.758 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:37:14.758 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:14.758 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.759 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:14.759 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6821", 2023-10-26T01:37:14.759 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:37:14.759 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.759 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.760 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:14.760 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6822", 2023-10-26T01:37:14.760 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:37:14.760 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:14.760 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:14.760 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.761 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:37:14.761 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:14.761 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.761 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:14.761 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6824", 2023-10-26T01:37:14.762 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:37:14.762 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.762 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.762 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:14.762 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6826", 2023-10-26T01:37:14.763 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:37:14.763 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:14.763 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:14.763 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.763 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:37:14.764 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:14.764 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.764 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:14.764 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6830", 2023-10-26T01:37:14.764 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:37:14.764 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.765 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.765 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:14.765 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6831", 2023-10-26T01:37:14.765 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:37:14.765 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:14.766 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:14.766 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.766 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:37:14.766 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:37:14.766 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.767 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:37:14.767 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6827", 2023-10-26T01:37:14.767 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:37:14.767 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.767 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.767 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:37:14.768 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6829", 2023-10-26T01:37:14.768 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:37:14.768 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:14.768 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:14.768 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.768 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6822/1115930619", 2023-10-26T01:37:14.769 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6826/1115930619", 2023-10-26T01:37:14.769 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6831/1115930619", 2023-10-26T01:37:14.769 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6829/1115930619", 2023-10-26T01:37:14.769 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:37:14.769 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:37:14.770 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:37:14.770 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:14.770 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:14.770 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:37:14.770 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_xinfo": [ 2023-10-26T01:37:14.770 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.771 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 0, 2023-10-26T01:37:14.771 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:37:14.771 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:37:14.771 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:37:14.771 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:37:14.772 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:37:14.772 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.889675+0000", 2023-10-26T01:37:14.772 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:37:14.772 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.772 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.772 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 1, 2023-10-26T01:37:14.773 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:37:14.773 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:37:14.773 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:37:14.773 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:37:14.773 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:37:14.773 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.864157+0000", 2023-10-26T01:37:14.774 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:37:14.774 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.774 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.774 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 2, 2023-10-26T01:37:14.774 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:37:14.774 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:37:14.775 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:37:14.775 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:37:14.775 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:37:14.775 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.876919+0000", 2023-10-26T01:37:14.775 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:37:14.776 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.776 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.776 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 3, 2023-10-26T01:37:14.776 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:37:14.776 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:37:14.776 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:37:14.777 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:37:14.777 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:37:14.777 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.974541+0000", 2023-10-26T01:37:14.777 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:37:14.777 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.778 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.778 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 4, 2023-10-26T01:37:14.778 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:37:14.778 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:37:14.778 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:37:14.778 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:37:14.779 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:37:14.779 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.969696+0000", 2023-10-26T01:37:14.779 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:37:14.779 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.779 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.780 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 5, 2023-10-26T01:37:14.780 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:37:14.780 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:37:14.780 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:37:14.780 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:37:14.780 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:37:14.781 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.859703+0000", 2023-10-26T01:37:14.781 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:37:14.781 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.781 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.781 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 6, 2023-10-26T01:37:14.781 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:37:14.782 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:37:14.782 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:37:14.782 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:37:14.782 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:37:14.782 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.867917+0000", 2023-10-26T01:37:14.783 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:37:14.783 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.783 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:14.783 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 7, 2023-10-26T01:37:14.783 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:37:14.784 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:37:14.784 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:37:14.784 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:37:14.784 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:37:14.784 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.898605+0000", 2023-10-26T01:37:14.784 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:37:14.785 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:14.785 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:37:14.785 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_upmap": [], 2023-10-26T01:37:14.785 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_upmap_items": [], 2023-10-26T01:37:14.785 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_upmap_primaries": [], 2023-10-26T01:37:14.786 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_temp": [], 2023-10-26T01:37:14.786 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_temp": [], 2023-10-26T01:37:14.786 INFO:teuthology.orchestra.run.smithi169.stdout: "blocklist": {}, 2023-10-26T01:37:14.786 INFO:teuthology.orchestra.run.smithi169.stdout: "range_blocklist": {}, 2023-10-26T01:37:14.786 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profiles": { 2023-10-26T01:37:14.786 INFO:teuthology.orchestra.run.smithi169.stdout: "default": { 2023-10-26T01:37:14.787 INFO:teuthology.orchestra.run.smithi169.stdout: "crush-failure-domain": "osd", 2023-10-26T01:37:14.787 INFO:teuthology.orchestra.run.smithi169.stdout: "k": "2", 2023-10-26T01:37:14.787 INFO:teuthology.orchestra.run.smithi169.stdout: "m": "1", 2023-10-26T01:37:14.787 INFO:teuthology.orchestra.run.smithi169.stdout: "plugin": "jerasure", 2023-10-26T01:37:14.787 INFO:teuthology.orchestra.run.smithi169.stdout: "technique": "reed_sol_van" 2023-10-26T01:37:14.788 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:14.788 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:14.788 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps_queue": [], 2023-10-26T01:37:14.788 INFO:teuthology.orchestra.run.smithi169.stdout: "new_removed_snaps": [], 2023-10-26T01:37:14.788 INFO:teuthology.orchestra.run.smithi169.stdout: "new_purged_snaps": [], 2023-10-26T01:37:14.789 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_node_flags": {}, 2023-10-26T01:37:14.789 INFO:teuthology.orchestra.run.smithi169.stdout: "device_class_flags": {}, 2023-10-26T01:37:14.789 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_mode": { 2023-10-26T01:37:14.789 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_mode_enabled": false, 2023-10-26T01:37:14.789 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_bucket_count": 0, 2023-10-26T01:37:14.789 INFO:teuthology.orchestra.run.smithi169.stdout: "degraded_stretch_mode": 0, 2023-10-26T01:37:14.790 INFO:teuthology.orchestra.run.smithi169.stdout: "recovering_stretch_mode": 0, 2023-10-26T01:37:14.790 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_mode_bucket": 0 2023-10-26T01:37:14.790 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:14.790 INFO:teuthology.orchestra.run.smithi169.stdout:} 2023-10-26T01:37:14.792 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:14.604+0000 7fc431bb0640 1 -- 172.21.15.169:0/1839473439 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc40c06ae40 msgr2=0x7fc40c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:14.792 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:14.604+0000 7fc431bb0640 1 --2- 172.21.15.169:0/1839473439 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc40c06ae40 0x7fc40c06d300 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fc41c036000 tx=0x7fc41c004730 comp rx=0 tx=0).stop 2023-10-26T01:37:14.793 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:14.604+0000 7fc431bb0640 1 -- 172.21.15.169:0/1839473439 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc42c13f310 msgr2=0x7fc42c1f16d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:14.793 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:14.604+0000 7fc431bb0640 1 --2- 172.21.15.169:0/1839473439 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc42c13f310 0x7fc42c1f16d0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fc42400c7a0 tx=0x7fc42400cc60 comp rx=0 tx=0).stop 2023-10-26T01:37:14.793 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:14.604+0000 7fc431bb0640 1 -- 172.21.15.169:0/1839473439 shutdown_connections 2023-10-26T01:37:14.793 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:14.604+0000 7fc431bb0640 1 --2- 172.21.15.169:0/1839473439 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc40c06ae40 0x7fc40c06d300 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:14.793 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:14.604+0000 7fc431bb0640 1 --2- 172.21.15.169:0/1839473439 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc42c13f310 0x7fc42c1f16d0 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:14.794 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:14.604+0000 7fc431bb0640 1 --2- 172.21.15.169:0/1839473439 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc42c0710c0 0x7fc42c0b2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:14.794 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:14.604+0000 7fc431bb0640 1 --2- 172.21.15.169:0/1839473439 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc42c070ce0 0x7fc42c136110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:14.794 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:14.604+0000 7fc431bb0640 1 -- 172.21.15.169:0/1839473439 >> 172.21.15.169:0/1839473439 conn(0x7fc42c069510 msgr2=0x7fc42c070480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:14.794 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:14.604+0000 7fc431bb0640 1 -- 172.21.15.169:0/1839473439 shutdown_connections 2023-10-26T01:37:14.794 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:14.605+0000 7fc431bb0640 1 -- 172.21.15.169:0/1839473439 wait complete. 2023-10-26T01:37:14.795 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T01:37:15.026 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:15.023+0000 7f6703ada640 1 -- 172.21.15.169:0/2763977868 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66e8004d00 msgr2=0x7f66e8005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:15.026 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:15.023+0000 7f6703ada640 1 --2- 172.21.15.169:0/2763977868 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66e8004d00 0x7f66e8005180 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f66f400c4e0 tx=0x7f66e80057f0 comp rx=0 tx=0).stop 2023-10-26T01:37:15.026 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:15.023+0000 7f6702077640 1 -- 172.21.15.169:0/2763977868 reap_dead start 2023-10-26T01:37:15.026 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:15.023+0000 7f6703ada640 1 -- 172.21.15.169:0/2763977868 shutdown_connections 2023-10-26T01:37:15.027 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:15.023+0000 7f6703ada640 1 -- 172.21.15.169:0/2763977868 >> 172.21.15.169:0/2763977868 conn(0x7f66fc069930 msgr2=0x7f66fc069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:15.027 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:15.023+0000 7f6703ada640 1 -- 172.21.15.169:0/2763977868 shutdown_connections 2023-10-26T01:37:15.027 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:15.023+0000 7f6703ada640 1 -- 172.21.15.169:0/2763977868 wait complete. 2023-10-26T01:37:15.028 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:15.025+0000 7f6703ada640 1 Processor -- start 2023-10-26T01:37:15.028 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:15.025+0000 7f6703ada640 1 -- start start 2023-10-26T01:37:15.029 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:15.026+0000 7f6703ada640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66e8004d00 0x7f66fc09aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:15.029 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:15.026+0000 7f6703ada640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66fc1f4610 0x7f66fc151730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:15.029 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:15.026+0000 7f6703ada640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f66fc151c70 0x7f66fc1f2960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:15.030 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:15.026+0000 7f6703ada640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66fc1f41f0 con 0x7f66e8004d00 2023-10-26T01:37:15.030 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:15.026+0000 7f6703ada640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66fc152120 con 0x7f66fc1f4610 2023-10-26T01:37:15.030 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:15.026+0000 7f6703ada640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66fc09af90 con 0x7f66fc151c70 2023-10-26T01:37:15.030 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:15.026+0000 7f6701876640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66e8004d00 0x7f66fc09aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:15.030 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:15.026+0000 7f6701876640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66e8004d00 0x7f66fc09aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:60990/0 (socket says 172.21.15.169:60990) 2023-10-26T01:37:15.031 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:15.026+0000 7f6701876640 1 -- 172.21.15.169:0/431308709 learned_addr learned my addr 172.21.15.169:0/431308709 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:37:15.031 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:15.026+0000 7f6701876640 1 -- 172.21.15.169:0/431308709 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f66fc151c70 msgr2=0x7f66fc1f2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:15.031 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:15.026+0000 7f6701876640 1 --2- 172.21.15.169:0/431308709 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f66fc151c70 0x7f66fc1f2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:15.031 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:15.026+0000 7f6701876640 1 -- 172.21.15.169:0/431308709 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66fc1f4610 msgr2=0x7f66fc151730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:15.031 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:15.026+0000 7f6701876640 1 --2- 172.21.15.169:0/431308709 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66fc1f4610 0x7f66fc151730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:15.032 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:15.026+0000 7f6701876640 1 -- 172.21.15.169:0/431308709 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66fc071b00 con 0x7f66e8004d00 2023-10-26T01:37:15.032 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:15.027+0000 7f6701876640 1 --2- 172.21.15.169:0/431308709 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66e8004d00 0x7f66fc09aa50 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f66f8004df0 tx=0x7f66f8001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:15.033 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:15.027+0000 7f66f27fc640 1 -- 172.21.15.169:0/431308709 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f8011660 con 0x7f66e8004d00 2023-10-26T01:37:15.033 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:15.027+0000 7f66f27fc640 1 -- 172.21.15.169:0/431308709 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f66f80118b0 con 0x7f66e8004d00 2023-10-26T01:37:15.033 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:15.027+0000 7f66f27fc640 1 -- 172.21.15.169:0/431308709 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f8011cc0 con 0x7f66e8004d00 2023-10-26T01:37:15.034 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:15.027+0000 7f6703ada640 1 -- 172.21.15.169:0/431308709 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66fc1f3570 con 0x7f66e8004d00 2023-10-26T01:37:15.034 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:15.027+0000 7f6703ada640 1 -- 172.21.15.169:0/431308709 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66fc1f3b30 con 0x7f66e8004d00 2023-10-26T01:37:15.034 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:15.028+0000 7f6703ada640 1 -- 172.21.15.169:0/431308709 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66fc04db10 con 0x7f66e8004d00 2023-10-26T01:37:15.034 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:15.029+0000 7f66f27fc640 1 -- 172.21.15.169:0/431308709 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f66f801b460 con 0x7f66e8004d00 2023-10-26T01:37:15.034 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:15.030+0000 7f66f27fc640 1 --2- 172.21.15.169:0/431308709 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f66e006ae70 0x7f66e006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:15.035 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:15.030+0000 7f66f27fc640 1 -- 172.21.15.169:0/431308709 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f66f8089f40 con 0x7f66e8004d00 2023-10-26T01:37:15.035 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:15.031+0000 7f6701075640 1 --2- 172.21.15.169:0/431308709 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f66e006ae70 0x7f66e006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:15.037 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:15.032+0000 7f6701075640 1 --2- 172.21.15.169:0/431308709 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f66e006ae70 0x7f66e006d330 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f66fc06d800 tx=0x7f66ec009550 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:15.038 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:15.034+0000 7f66f27fc640 1 -- 172.21.15.169:0/431308709 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f66f8055080 con 0x7f66e8004d00 2023-10-26T01:37:15.262 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:15.259+0000 7f6703ada640 1 -- 172.21.15.169:0/431308709 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f66fc1598d0 con 0x7f66e8004d00 2023-10-26T01:37:15.263 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:15.260+0000 7f66f27fc640 1 -- 172.21.15.169:0/431308709 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+4924 (secure 0 0 0) 0x7f66f80586a0 con 0x7f66e8004d00 2023-10-26T01:37:15.263 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:37:15.263 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":50,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5240,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/3870332417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":3870332417},{"type":"v1","addr":"172.21.15.170:6837","nonce":3870332417}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45},{"gid":5497,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/2533771698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2533771698},{"type":"v1","addr":"172.21.15.170:6835","nonce":2533771698}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45},{"gid":6885,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/2117946713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2117946713},{"type":"v1","addr":"172.21.15.169:6834","nonce":2117946713}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T01:36:43.849046+0000","modified":"2023-10-26T01:36:46.926339+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6888},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6888":{"gid":6888,"name":"c","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6835/657142078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":657142078},{"type":"v1","addr":"172.21.15.169:6835","nonce":657142078}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-10-26T01:37:15.263 INFO:teuthology.orchestra.run.smithi169.stderr:dumped fsmap epoch 50 2023-10-26T01:37:15.265 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:15.262+0000 7f6703ada640 1 -- 172.21.15.169:0/431308709 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f66e006ae70 msgr2=0x7f66e006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:15.266 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:15.262+0000 7f6703ada640 1 --2- 172.21.15.169:0/431308709 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f66e006ae70 0x7f66e006d330 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f66fc06d800 tx=0x7f66ec009550 comp rx=0 tx=0).stop 2023-10-26T01:37:15.266 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:15.262+0000 7f6703ada640 1 -- 172.21.15.169:0/431308709 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66e8004d00 msgr2=0x7f66fc09aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:15.266 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:15.262+0000 7f6703ada640 1 --2- 172.21.15.169:0/431308709 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66e8004d00 0x7f66fc09aa50 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f66f8004df0 tx=0x7f66f8001630 comp rx=0 tx=0).stop 2023-10-26T01:37:15.266 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:15.263+0000 7f6703ada640 1 -- 172.21.15.169:0/431308709 shutdown_connections 2023-10-26T01:37:15.267 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:15.263+0000 7f6703ada640 1 --2- 172.21.15.169:0/431308709 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f66e006ae70 0x7f66e006d330 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:15.267 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:15.263+0000 7f6703ada640 1 --2- 172.21.15.169:0/431308709 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f66fc151c70 0x7f66fc1f2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:15.267 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:15.263+0000 7f6703ada640 1 --2- 172.21.15.169:0/431308709 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66fc1f4610 0x7f66fc151730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:15.267 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:15.263+0000 7f6703ada640 1 --2- 172.21.15.169:0/431308709 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66e8004d00 0x7f66fc09aa50 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:15.267 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:15.263+0000 7f6703ada640 1 -- 172.21.15.169:0/431308709 >> 172.21.15.169:0/431308709 conn(0x7f66fc069930 msgr2=0x7f66fc06f100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:15.267 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:15.263+0000 7f6703ada640 1 -- 172.21.15.169:0/431308709 shutdown_connections 2023-10-26T01:37:15.268 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:15.263+0000 7f6703ada640 1 -- 172.21.15.169:0/431308709 wait complete. 2023-10-26T01:37:15.284 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2023-10-26T01:37:15.515 INFO:teuthology.orchestra.run.smithi169.stdout:{"cluster_fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","whoami":0,"id":6888,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":31.084947535000001,"mdsmap_epoch":50,"osdmap_epoch":109,"osdmap_epoch_barrier":105,"uptime":37.936094736999998} 2023-10-26T01:37:15.516 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "30203b28-ed8a-44e8-a9a6-a9bf25f980f1", "fs_name": "cephfs", "id": 6888, "mdsmap_epoch": 50, "osdmap_epoch": 109, "osdmap_epoch_barrier": 105, "rank_uptime": 31.084947535, "state": "up:active", "uptime": 37.936094737, "want_state": "up:active", "whoami": 0 } 2023-10-26T01:37:15.516 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-26T01:37:15.517 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:37:15.517 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:37:15.567 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:37:15.568 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file bs=1M conv=fdatasync count=50 seek=0) 2023-10-26T01:37:16.066 INFO:teuthology.orchestra.run.smithi169.stderr:50+0 records in 2023-10-26T01:37:16.067 INFO:teuthology.orchestra.run.smithi169.stderr:50+0 records out 2023-10-26T01:37:16.067 INFO:teuthology.orchestra.run.smithi169.stderr:52428800 bytes (52 MB, 50 MiB) copied, 0.408557 s, 128 MB/s 2023-10-26T01:37:16.068 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-26T01:37:16.068 DEBUG:teuthology.orchestra.run.smithi169:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-26T01:37:16.165 DEBUG:teuthology.orchestra.run.smithi170:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-26T01:37:16.288 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-26T01:37:16.289 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:37:16.289 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:37:16.352 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:37:16.352 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:37:16.409 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:37:16.627 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:16.624+0000 7f7ea0475640 1 -- 172.21.15.169:0/901065250 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e980cde00 msgr2=0x7f7e980cbac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:16.627 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:16.624+0000 7f7ea0475640 1 --2- 172.21.15.169:0/901065250 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e980cde00 0x7f7e980cbac0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f7e90005fd0 tx=0x7f7e9002c060 comp rx=0 tx=0).stop 2023-10-26T01:37:16.628 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:16.624+0000 7f7ea0475640 1 -- 172.21.15.169:0/901065250 shutdown_connections 2023-10-26T01:37:16.628 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:16.624+0000 7f7ea0475640 1 --2- 172.21.15.169:0/901065250 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e980cde00 0x7f7e980cbac0 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:16.628 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:16.624+0000 7f7ea0475640 1 --2- 172.21.15.169:0/901065250 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7e981592d0 0x7f7e980aa3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:16.628 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:16.624+0000 7f7ea0475640 1 -- 172.21.15.169:0/901065250 >> 172.21.15.169:0/901065250 conn(0x7f7e98069930 msgr2=0x7f7e98069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:16.628 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:16.624+0000 7f7ea0475640 1 -- 172.21.15.169:0/901065250 shutdown_connections 2023-10-26T01:37:16.629 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:16.624+0000 7f7ea0475640 1 -- 172.21.15.169:0/901065250 wait complete. 2023-10-26T01:37:16.629 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:16.626+0000 7f7ea0475640 1 Processor -- start 2023-10-26T01:37:16.629 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:16.626+0000 7f7ea0475640 1 -- start start 2023-10-26T01:37:16.631 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:16.627+0000 7f7ea0475640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7e980ac930 0x7f7e981465a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:16.631 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:16.627+0000 7f7ea0475640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7e980cde00 0x7f7e98146ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:16.631 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:16.627+0000 7f7ea0475640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e981592d0 0x7f7e98147020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:16.631 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:16.627+0000 7f7ea0475640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9806f530 con 0x7f7e981592d0 2023-10-26T01:37:16.631 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:16.627+0000 7f7ea0475640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e98159d90 con 0x7f7e980ac930 2023-10-26T01:37:16.632 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:16.627+0000 7f7ea0475640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9815a070 con 0x7f7e980cde00 2023-10-26T01:37:16.632 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:16.628+0000 7f7e9ea12640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e981592d0 0x7f7e98147020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:16.632 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:16.628+0000 7f7e9ea12640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e981592d0 0x7f7e98147020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:32778/0 (socket says 172.21.15.169:32778) 2023-10-26T01:37:16.632 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:16.628+0000 7f7e9ea12640 1 -- 172.21.15.169:0/24408808 learned_addr learned my addr 172.21.15.169:0/24408808 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:37:16.632 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:16.628+0000 7f7e9ea12640 1 -- 172.21.15.169:0/24408808 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7e980cde00 msgr2=0x7f7e98146ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:16.633 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:16.628+0000 7f7e9e211640 1 --2- 172.21.15.169:0/24408808 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7e980ac930 0x7f7e981465a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:16.633 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:16.628+0000 7f7e9da10640 1 --2- 172.21.15.169:0/24408808 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7e980cde00 0x7f7e98146ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:16.633 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:16.628+0000 7f7e9ea12640 1 --2- 172.21.15.169:0/24408808 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7e980cde00 0x7f7e98146ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:16.633 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:16.628+0000 7f7e9ea12640 1 -- 172.21.15.169:0/24408808 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7e980ac930 msgr2=0x7f7e981465a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:16.633 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:16.628+0000 7f7e9ea12640 1 --2- 172.21.15.169:0/24408808 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7e980ac930 0x7f7e981465a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:16.633 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:16.628+0000 7f7e9ea12640 1 -- 172.21.15.169:0/24408808 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e90009840 con 0x7f7e981592d0 2023-10-26T01:37:16.634 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:16.628+0000 7f7e9ea12640 1 --2- 172.21.15.169:0/24408808 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e981592d0 0x7f7e98147020 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f7e90036040 tx=0x7f7e90004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:16.635 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:16.629+0000 7f7e8effd640 1 -- 172.21.15.169:0/24408808 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e90004870 con 0x7f7e981592d0 2023-10-26T01:37:16.635 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:16.629+0000 7f7e8effd640 1 -- 172.21.15.169:0/24408808 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7e9003d540 con 0x7f7e981592d0 2023-10-26T01:37:16.635 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:16.629+0000 7f7e8effd640 1 -- 172.21.15.169:0/24408808 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e9003d950 con 0x7f7e981592d0 2023-10-26T01:37:16.635 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:16.629+0000 7f7ea0475640 1 -- 172.21.15.169:0/24408808 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e9806ed30 con 0x7f7e981592d0 2023-10-26T01:37:16.636 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:16.629+0000 7f7ea0475640 1 -- 172.21.15.169:0/24408808 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e98144210 con 0x7f7e981592d0 2023-10-26T01:37:16.636 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:16.630+0000 7f7ea0475640 1 -- 172.21.15.169:0/24408808 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e9806f250 con 0x7f7e981592d0 2023-10-26T01:37:16.636 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:16.631+0000 7f7e8effd640 1 -- 172.21.15.169:0/24408808 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f7e9003dab0 con 0x7f7e981592d0 2023-10-26T01:37:16.636 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:16.632+0000 7f7e8effd640 1 --2- 172.21.15.169:0/24408808 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7e7c06ae40 0x7f7e7c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:16.636 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:16.632+0000 7f7e8effd640 1 -- 172.21.15.169:0/24408808 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7e900ac8a0 con 0x7f7e981592d0 2023-10-26T01:37:16.637 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:16.633+0000 7f7e9e211640 1 --2- 172.21.15.169:0/24408808 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7e7c06ae40 0x7f7e7c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:16.637 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:16.634+0000 7f7e9e211640 1 --2- 172.21.15.169:0/24408808 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7e7c06ae40 0x7f7e7c06d300 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f7e940058c0 tx=0x7f7e94005850 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:16.638 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:16.636+0000 7f7e8effd640 1 -- 172.21.15.169:0/24408808 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7e9806f250 con 0x7f7e981592d0 2023-10-26T01:37:16.830 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:16.828+0000 7f7ea0475640 1 -- 172.21.15.169:0/24408808 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7e9806f250 con 0x7f7e981592d0 2023-10-26T01:37:16.832 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:16.830+0000 7f7e8effd640 1 -- 172.21.15.169:0/24408808 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v109) v1 ==== 74+0+16620 (secure 0 0 0) 0x7f7e9806f250 con 0x7f7e981592d0 2023-10-26T01:37:16.833 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:37:16.833 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":109,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:37:13.181064+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-10-26T01:36:41.430455+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-10-26T01:36:42.883252+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:37:16.835 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:16.832+0000 7f7ea0475640 1 -- 172.21.15.169:0/24408808 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7e7c06ae40 msgr2=0x7f7e7c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:16.836 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:16.832+0000 7f7ea0475640 1 --2- 172.21.15.169:0/24408808 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7e7c06ae40 0x7f7e7c06d300 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f7e940058c0 tx=0x7f7e94005850 comp rx=0 tx=0).stop 2023-10-26T01:37:16.836 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:16.832+0000 7f7ea0475640 1 -- 172.21.15.169:0/24408808 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e981592d0 msgr2=0x7f7e98147020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:16.836 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:16.832+0000 7f7ea0475640 1 --2- 172.21.15.169:0/24408808 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e981592d0 0x7f7e98147020 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f7e90036040 tx=0x7f7e90004270 comp rx=0 tx=0).stop 2023-10-26T01:37:16.836 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:16.832+0000 7f7ea0475640 1 -- 172.21.15.169:0/24408808 shutdown_connections 2023-10-26T01:37:16.836 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:16.832+0000 7f7ea0475640 1 --2- 172.21.15.169:0/24408808 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7e7c06ae40 0x7f7e7c06d300 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:16.837 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:16.832+0000 7f7ea0475640 1 --2- 172.21.15.169:0/24408808 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e981592d0 0x7f7e98147020 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:16.837 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:16.832+0000 7f7ea0475640 1 --2- 172.21.15.169:0/24408808 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7e980cde00 0x7f7e98146ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:16.837 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:16.832+0000 7f7ea0475640 1 --2- 172.21.15.169:0/24408808 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7e980ac930 0x7f7e981465a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:16.837 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:16.832+0000 7f7ea0475640 1 -- 172.21.15.169:0/24408808 >> 172.21.15.169:0/24408808 conn(0x7f7e98069930 msgr2=0x7f7e98153a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:16.837 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:16.833+0000 7f7ea0475640 1 -- 172.21.15.169:0/24408808 shutdown_connections 2023-10-26T01:37:16.838 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:16.833+0000 7f7ea0475640 1 -- 172.21.15.169:0/24408808 wait complete. 2023-10-26T01:37:16.853 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T01:30:32.200808+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '109', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-26T01:36:41.430455+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '109', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2023-10-26T01:36:42.883252+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '109', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T01:37:16.854 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T01:37:17.067 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.064+0000 7ff943fff640 1 -- 172.21.15.169:0/3915353245 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9440ce450 msgr2=0x7ff9440cc5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:17.068 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.064+0000 7ff943fff640 1 --2- 172.21.15.169:0/3915353245 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9440ce450 0x7ff9440cc5f0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7ff938009830 tx=0x7ff93802bd50 comp rx=0 tx=0).stop 2023-10-26T01:37:17.068 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.064+0000 7ff943fff640 1 -- 172.21.15.169:0/3915353245 shutdown_connections 2023-10-26T01:37:17.068 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.064+0000 7ff943fff640 1 --2- 172.21.15.169:0/3915353245 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff9440d0a30 0x7ff9440d0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:17.068 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.064+0000 7ff943fff640 1 --2- 172.21.15.169:0/3915353245 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9440ce450 0x7ff9440cc5f0 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:17.068 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.064+0000 7ff943fff640 1 -- 172.21.15.169:0/3915353245 >> 172.21.15.169:0/3915353245 conn(0x7ff9440b2f10 msgr2=0x7ff9440b3320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:17.069 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.064+0000 7ff943fff640 1 -- 172.21.15.169:0/3915353245 shutdown_connections 2023-10-26T01:37:17.069 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.064+0000 7ff943fff640 1 -- 172.21.15.169:0/3915353245 wait complete. 2023-10-26T01:37:17.069 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.066+0000 7ff943fff640 1 Processor -- start 2023-10-26T01:37:17.070 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.066+0000 7ff943fff640 1 -- start start 2023-10-26T01:37:17.071 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.068+0000 7ff943fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9440ce450 0x7ff9440b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:17.071 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.068+0000 7ff943fff640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff9440d0a30 0x7ff94414c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:17.071 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.068+0000 7ff943fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff94415b790 0x7ff94414c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:17.071 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.068+0000 7ff943fff640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff94415db30 con 0x7ff9440ce450 2023-10-26T01:37:17.072 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.068+0000 7ff943fff640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9441f5820 con 0x7ff94415b790 2023-10-26T01:37:17.072 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.068+0000 7ff943fff640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9441f4be0 con 0x7ff9440d0a30 2023-10-26T01:37:17.072 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.068+0000 7ff942ffd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9440ce450 0x7ff9440b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:17.072 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.068+0000 7ff942ffd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9440ce450 0x7ff9440b21f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:32792/0 (socket says 172.21.15.169:32792) 2023-10-26T01:37:17.073 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.068+0000 7ff942ffd640 1 -- 172.21.15.169:0/4093869660 learned_addr learned my addr 172.21.15.169:0/4093869660 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:37:17.073 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.068+0000 7ff942ffd640 1 -- 172.21.15.169:0/4093869660 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff9440d0a30 msgr2=0x7ff94414c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:17.073 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.068+0000 7ff942ffd640 1 --2- 172.21.15.169:0/4093869660 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff9440d0a30 0x7ff94414c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:17.073 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.068+0000 7ff942ffd640 1 -- 172.21.15.169:0/4093869660 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff94415b790 msgr2=0x7ff94414c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:17.073 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.068+0000 7ff942ffd640 1 --2- 172.21.15.169:0/4093869660 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff94415b790 0x7ff94414c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:17.073 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.068+0000 7ff942ffd640 1 -- 172.21.15.169:0/4093869660 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff938009490 con 0x7ff9440ce450 2023-10-26T01:37:17.075 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.069+0000 7ff942ffd640 1 --2- 172.21.15.169:0/4093869660 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9440ce450 0x7ff9440b21f0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7ff93400c430 tx=0x7ff93400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:17.075 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.069+0000 7ff92b7fe640 1 -- 172.21.15.169:0/4093869660 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff934011790 con 0x7ff9440ce450 2023-10-26T01:37:17.075 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.069+0000 7ff92b7fe640 1 -- 172.21.15.169:0/4093869660 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff934011a00 con 0x7ff9440ce450 2023-10-26T01:37:17.075 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.069+0000 7ff92b7fe640 1 -- 172.21.15.169:0/4093869660 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff934011e10 con 0x7ff9440ce450 2023-10-26T01:37:17.076 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.069+0000 7ff943fff640 1 -- 172.21.15.169:0/4093869660 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9441f5160 con 0x7ff9440ce450 2023-10-26T01:37:17.076 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.069+0000 7ff943fff640 1 -- 172.21.15.169:0/4093869660 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9440c59c0 con 0x7ff9440ce450 2023-10-26T01:37:17.076 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.070+0000 7ff943fff640 1 -- 172.21.15.169:0/4093869660 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff944073c10 con 0x7ff9440ce450 2023-10-26T01:37:17.076 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.071+0000 7ff92b7fe640 1 -- 172.21.15.169:0/4093869660 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ff93401c460 con 0x7ff9440ce450 2023-10-26T01:37:17.076 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.072+0000 7ff92b7fe640 1 --2- 172.21.15.169:0/4093869660 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff93006ae40 0x7ff93006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:17.077 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.072+0000 7ff92b7fe640 1 -- 172.21.15.169:0/4093869660 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff934089740 con 0x7ff9440ce450 2023-10-26T01:37:17.077 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.073+0000 7ff9427fc640 1 --2- 172.21.15.169:0/4093869660 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff93006ae40 0x7ff93006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:17.077 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.074+0000 7ff9427fc640 1 --2- 172.21.15.169:0/4093869660 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff93006ae40 0x7ff93006d300 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7ff938036000 tx=0x7ff9380046c0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:17.079 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.077+0000 7ff92b7fe640 1 -- 172.21.15.169:0/4093869660 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff934054880 con 0x7ff9440ce450 2023-10-26T01:37:17.270 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.267+0000 7ff943fff640 1 -- 172.21.15.169:0/4093869660 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff9440b6c80 con 0x7ff9440ce450 2023-10-26T01:37:17.271 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.268+0000 7ff92b7fe640 1 -- 172.21.15.169:0/4093869660 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v109) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff934057ea0 con 0x7ff9440ce450 2023-10-26T01:37:17.271 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 1 2023-10-26T01:37:17.274 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.271+0000 7ff943fff640 1 -- 172.21.15.169:0/4093869660 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff93006ae40 msgr2=0x7ff93006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:17.274 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.271+0000 7ff943fff640 1 --2- 172.21.15.169:0/4093869660 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff93006ae40 0x7ff93006d300 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7ff938036000 tx=0x7ff9380046c0 comp rx=0 tx=0).stop 2023-10-26T01:37:17.274 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.271+0000 7ff943fff640 1 -- 172.21.15.169:0/4093869660 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9440ce450 msgr2=0x7ff9440b21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:17.275 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.271+0000 7ff943fff640 1 --2- 172.21.15.169:0/4093869660 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9440ce450 0x7ff9440b21f0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7ff93400c430 tx=0x7ff93400c8f0 comp rx=0 tx=0).stop 2023-10-26T01:37:17.275 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.271+0000 7ff943fff640 1 -- 172.21.15.169:0/4093869660 shutdown_connections 2023-10-26T01:37:17.275 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.271+0000 7ff943fff640 1 --2- 172.21.15.169:0/4093869660 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff93006ae40 0x7ff93006d300 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:17.275 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.271+0000 7ff943fff640 1 --2- 172.21.15.169:0/4093869660 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff94415b790 0x7ff94414c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:17.275 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.271+0000 7ff943fff640 1 --2- 172.21.15.169:0/4093869660 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff9440d0a30 0x7ff94414c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:17.276 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.271+0000 7ff943fff640 1 --2- 172.21.15.169:0/4093869660 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9440ce450 0x7ff9440b21f0 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:17.276 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.271+0000 7ff943fff640 1 -- 172.21.15.169:0/4093869660 >> 172.21.15.169:0/4093869660 conn(0x7ff9440b2f10 msgr2=0x7ff9440cc100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:17.276 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.271+0000 7ff943fff640 1 -- 172.21.15.169:0/4093869660 shutdown_connections 2023-10-26T01:37:17.276 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.271+0000 7ff943fff640 1 -- 172.21.15.169:0/4093869660 wait complete. 2023-10-26T01:37:17.291 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-26T01:37:17.510 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.507+0000 7fdff0af2640 1 -- 172.21.15.169:0/3696794978 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdfec157be0 msgr2=0x7fdfec06b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:17.511 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.507+0000 7fdff0af2640 1 --2- 172.21.15.169:0/3696794978 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdfec157be0 0x7fdfec06b740 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc005fd0 tx=0x7fdfdc02c050 comp rx=0 tx=0).stop 2023-10-26T01:37:17.511 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.507+0000 7fdff0af2640 1 -- 172.21.15.169:0/3696794978 shutdown_connections 2023-10-26T01:37:17.511 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.507+0000 7fdff0af2640 1 --2- 172.21.15.169:0/3696794978 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdfec157be0 0x7fdfec06b740 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:17.511 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.507+0000 7fdff0af2640 1 -- 172.21.15.169:0/3696794978 >> 172.21.15.169:0/3696794978 conn(0x7fdfec069930 msgr2=0x7fdfec069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:17.512 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.507+0000 7fdff0af2640 1 -- 172.21.15.169:0/3696794978 shutdown_connections 2023-10-26T01:37:17.512 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.507+0000 7fdff0af2640 1 -- 172.21.15.169:0/3696794978 wait complete. 2023-10-26T01:37:17.513 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.509+0000 7fdff0af2640 1 Processor -- start 2023-10-26T01:37:17.513 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.509+0000 7fdff0af2640 1 -- start start 2023-10-26T01:37:17.513 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.510+0000 7fdff0af2640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdfec06ce50 0x7fdfec1f2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:17.514 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.510+0000 7fdff0af2640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdfec0ab300 0x7fdfec147390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:17.514 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.510+0000 7fdff0af2640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdfec157be0 0x7fdfec1e0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:17.514 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.510+0000 7fdff0af2640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfec147de0 con 0x7fdfec157be0 2023-10-26T01:37:17.514 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.510+0000 7fdff0af2640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfec1480c0 con 0x7fdfec0ab300 2023-10-26T01:37:17.515 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.510+0000 7fdff0af2640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfec1483a0 con 0x7fdfec06ce50 2023-10-26T01:37:17.515 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.511+0000 7fdfead9d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdfec157be0 0x7fdfec1e0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:17.515 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.511+0000 7fdfead9d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdfec157be0 0x7fdfec1e0fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:32806/0 (socket says 172.21.15.169:32806) 2023-10-26T01:37:17.515 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.511+0000 7fdfead9d640 1 -- 172.21.15.169:0/3686654620 learned_addr learned my addr 172.21.15.169:0/3686654620 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:37:17.517 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.511+0000 7fdfe9d9b640 1 --2- 172.21.15.169:0/3686654620 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdfec0ab300 0x7fdfec147390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:17.517 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.511+0000 7fdfea59c640 1 --2- 172.21.15.169:0/3686654620 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdfec06ce50 0x7fdfec1f2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:17.517 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.511+0000 7fdfead9d640 1 -- 172.21.15.169:0/3686654620 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdfec06ce50 msgr2=0x7fdfec1f2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:17.517 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.511+0000 7fdfead9d640 1 --2- 172.21.15.169:0/3686654620 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdfec06ce50 0x7fdfec1f2a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:17.517 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.511+0000 7fdfead9d640 1 -- 172.21.15.169:0/3686654620 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdfec0ab300 msgr2=0x7fdfec147390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:17.518 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.511+0000 7fdfead9d640 1 --2- 172.21.15.169:0/3686654620 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdfec0ab300 0x7fdfec147390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:17.518 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.511+0000 7fdfead9d640 1 -- 172.21.15.169:0/3686654620 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfdc009840 con 0x7fdfec157be0 2023-10-26T01:37:17.518 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.511+0000 7fdfead9d640 1 --2- 172.21.15.169:0/3686654620 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdfec157be0 0x7fdfec1e0fe0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fdfe400c410 tx=0x7fdfe400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:17.518 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.512+0000 7fdfe2ffd640 1 -- 172.21.15.169:0/3686654620 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe4011790 con 0x7fdfec157be0 2023-10-26T01:37:17.519 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.512+0000 7fdfe2ffd640 1 -- 172.21.15.169:0/3686654620 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fdfe40119e0 con 0x7fdfec157be0 2023-10-26T01:37:17.519 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.512+0000 7fdff0af2640 1 -- 172.21.15.169:0/3686654620 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfec148680 con 0x7fdfec157be0 2023-10-26T01:37:17.519 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.512+0000 7fdfe2ffd640 1 -- 172.21.15.169:0/3686654620 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe4011e30 con 0x7fdfec157be0 2023-10-26T01:37:17.519 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.512+0000 7fdff0af2640 1 -- 172.21.15.169:0/3686654620 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfec1f2fb0 con 0x7fdfec157be0 2023-10-26T01:37:17.519 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.513+0000 7fdff0af2640 1 -- 172.21.15.169:0/3686654620 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfec148ba0 con 0x7fdfec157be0 2023-10-26T01:37:17.520 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.514+0000 7fdfe2ffd640 1 -- 172.21.15.169:0/3686654620 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fdfe401c460 con 0x7fdfec157be0 2023-10-26T01:37:17.520 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.515+0000 7fdfe2ffd640 1 --2- 172.21.15.169:0/3686654620 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fdfcc06ae40 0x7fdfcc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:17.521 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.515+0000 7fdfe2ffd640 1 -- 172.21.15.169:0/3686654620 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdfe40895f0 con 0x7fdfec157be0 2023-10-26T01:37:17.521 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.516+0000 7fdfea59c640 1 --2- 172.21.15.169:0/3686654620 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fdfcc06ae40 0x7fdfcc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:17.521 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.517+0000 7fdfea59c640 1 --2- 172.21.15.169:0/3686654620 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fdfcc06ae40 0x7fdfcc06d300 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc035000 tx=0x7fdfdc004ab0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:17.522 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.520+0000 7fdfe2ffd640 1 -- 172.21.15.169:0/3686654620 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdfec148ba0 con 0x7fdfec157be0 2023-10-26T01:37:17.714 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.711+0000 7fdff0af2640 1 -- 172.21.15.169:0/3686654620 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdfec148ba0 con 0x7fdfec157be0 2023-10-26T01:37:17.715 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.712+0000 7fdfe2ffd640 1 -- 172.21.15.169:0/3686654620 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v109) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdfec148ba0 con 0x7fdfec157be0 2023-10-26T01:37:17.715 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 32 2023-10-26T01:37:17.718 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.715+0000 7fdff0af2640 1 -- 172.21.15.169:0/3686654620 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fdfcc06ae40 msgr2=0x7fdfcc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:17.718 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.715+0000 7fdff0af2640 1 --2- 172.21.15.169:0/3686654620 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fdfcc06ae40 0x7fdfcc06d300 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc035000 tx=0x7fdfdc004ab0 comp rx=0 tx=0).stop 2023-10-26T01:37:17.718 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.715+0000 7fdff0af2640 1 -- 172.21.15.169:0/3686654620 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdfec157be0 msgr2=0x7fdfec1e0fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:17.719 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.715+0000 7fdff0af2640 1 --2- 172.21.15.169:0/3686654620 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdfec157be0 0x7fdfec1e0fe0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fdfe400c410 tx=0x7fdfe400c8d0 comp rx=0 tx=0).stop 2023-10-26T01:37:17.719 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.715+0000 7fdff0af2640 1 -- 172.21.15.169:0/3686654620 shutdown_connections 2023-10-26T01:37:17.719 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.715+0000 7fdff0af2640 1 --2- 172.21.15.169:0/3686654620 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fdfcc06ae40 0x7fdfcc06d300 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:17.719 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.715+0000 7fdff0af2640 1 --2- 172.21.15.169:0/3686654620 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdfec157be0 0x7fdfec1e0fe0 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:17.719 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.715+0000 7fdff0af2640 1 --2- 172.21.15.169:0/3686654620 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdfec0ab300 0x7fdfec147390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:17.719 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.715+0000 7fdff0af2640 1 --2- 172.21.15.169:0/3686654620 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdfec06ce50 0x7fdfec1f2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:17.720 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.715+0000 7fdff0af2640 1 -- 172.21.15.169:0/3686654620 >> 172.21.15.169:0/3686654620 conn(0x7fdfec069930 msgr2=0x7fdfec098040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:17.720 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.715+0000 7fdff0af2640 1 -- 172.21.15.169:0/3686654620 shutdown_connections 2023-10-26T01:37:17.720 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.715+0000 7fdff0af2640 1 -- 172.21.15.169:0/3686654620 wait complete. 2023-10-26T01:37:17.735 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-26T01:37:17.958 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.955+0000 7fd06a9f5640 1 -- 172.21.15.169:0/2678122687 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd0640713d0 msgr2=0x7fd06415d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:17.959 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.955+0000 7fd06a9f5640 1 --2- 172.21.15.169:0/2678122687 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd0640713d0 0x7fd06415d990 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd04c009830 tx=0x7fd04c02bd50 comp rx=0 tx=0).stop 2023-10-26T01:37:17.959 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.955+0000 7fd06a9f5640 1 -- 172.21.15.169:0/2678122687 shutdown_connections 2023-10-26T01:37:17.959 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.955+0000 7fd06a9f5640 1 --2- 172.21.15.169:0/2678122687 >> v2:172.21.15.170:3300/0 conn(0x7fd0640cb8e0 0x7fd0640cbd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:17.959 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.955+0000 7fd06a9f5640 1 --2- 172.21.15.169:0/2678122687 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd0640713d0 0x7fd06415d990 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:17.959 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.955+0000 7fd06a9f5640 1 -- 172.21.15.169:0/2678122687 >> 172.21.15.169:0/2678122687 conn(0x7fd0640af8a0 msgr2=0x7fd0640afc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:17.960 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.955+0000 7fd06a9f5640 1 -- 172.21.15.169:0/2678122687 shutdown_connections 2023-10-26T01:37:17.960 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.955+0000 7fd06a9f5640 1 -- 172.21.15.169:0/2678122687 wait complete. 2023-10-26T01:37:17.960 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.957+0000 7fd06a9f5640 1 Processor -- start 2023-10-26T01:37:17.960 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.957+0000 7fd06a9f5640 1 -- start start 2023-10-26T01:37:17.963 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.959+0000 7fd06a9f5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd0640713d0 0x7fd0641508f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:17.963 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.959+0000 7fd06a9f5640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd0640cb8e0 0x7fd06414d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:17.964 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.959+0000 7fd06a9f5640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd06415ca70 0x7fd064157b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:17.964 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.959+0000 7fd06a9f5640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd064154f10 con 0x7fd0640cb8e0 2023-10-26T01:37:17.964 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.959+0000 7fd06a9f5640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd064150ed0 con 0x7fd0640713d0 2023-10-26T01:37:17.964 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.959+0000 7fd06a9f5640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd064154850 con 0x7fd06415ca70 2023-10-26T01:37:17.964 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.959+0000 7fd05b7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd0640cb8e0 0x7fd06414d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:17.965 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.959+0000 7fd05b7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd0640cb8e0 0x7fd06414d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:32814/0 (socket says 172.21.15.169:32814) 2023-10-26T01:37:17.965 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.959+0000 7fd05b7fe640 1 -- 172.21.15.169:0/1023930040 learned_addr learned my addr 172.21.15.169:0/1023930040 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:37:17.965 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.959+0000 7fd063fff640 1 --2- 172.21.15.169:0/1023930040 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd0640713d0 0x7fd0641508f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:17.965 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.959+0000 7fd068f92640 1 --2- 172.21.15.169:0/1023930040 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd06415ca70 0x7fd064157b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:17.966 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.960+0000 7fd05b7fe640 1 -- 172.21.15.169:0/1023930040 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd06415ca70 msgr2=0x7fd064157b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:17.966 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.960+0000 7fd05b7fe640 1 --2- 172.21.15.169:0/1023930040 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd06415ca70 0x7fd064157b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:17.966 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.960+0000 7fd05b7fe640 1 -- 172.21.15.169:0/1023930040 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd0640713d0 msgr2=0x7fd0641508f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:17.966 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.960+0000 7fd05b7fe640 1 --2- 172.21.15.169:0/1023930040 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd0640713d0 0x7fd0641508f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:17.966 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.960+0000 7fd05b7fe640 1 -- 172.21.15.169:0/1023930040 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd05c009560 con 0x7fd0640cb8e0 2023-10-26T01:37:17.967 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.960+0000 7fd068f92640 1 --2- 172.21.15.169:0/1023930040 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd06415ca70 0x7fd064157b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:37:17.967 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.960+0000 7fd063fff640 1 --2- 172.21.15.169:0/1023930040 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd0640713d0 0x7fd0641508f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T01:37:17.967 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.960+0000 7fd05b7fe640 1 --2- 172.21.15.169:0/1023930040 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd0640cb8e0 0x7fd06414d7a0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fd04c000c00 tx=0x7fd04c0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:17.967 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.960+0000 7fd0617fa640 1 -- 172.21.15.169:0/1023930040 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd04c003ea0 con 0x7fd0640cb8e0 2023-10-26T01:37:17.967 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.960+0000 7fd0617fa640 1 -- 172.21.15.169:0/1023930040 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd04c034440 con 0x7fd0640cb8e0 2023-10-26T01:37:17.968 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.960+0000 7fd0617fa640 1 -- 172.21.15.169:0/1023930040 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd04c034850 con 0x7fd0640cb8e0 2023-10-26T01:37:17.968 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.960+0000 7fd06a9f5640 1 -- 172.21.15.169:0/1023930040 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd04c009490 con 0x7fd0640cb8e0 2023-10-26T01:37:17.968 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.961+0000 7fd06a9f5640 1 -- 172.21.15.169:0/1023930040 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd064153260 con 0x7fd0640cb8e0 2023-10-26T01:37:17.969 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.962+0000 7fd06a9f5640 1 -- 172.21.15.169:0/1023930040 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd06404db10 con 0x7fd0640cb8e0 2023-10-26T01:37:17.969 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.963+0000 7fd0617fa640 1 -- 172.21.15.169:0/1023930040 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fd04c0349b0 con 0x7fd0640cb8e0 2023-10-26T01:37:17.969 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.964+0000 7fd0617fa640 1 --2- 172.21.15.169:0/1023930040 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd04406ae40 0x7fd04406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:17.969 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.964+0000 7fd0617fa640 1 -- 172.21.15.169:0/1023930040 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd04c0abe00 con 0x7fd0640cb8e0 2023-10-26T01:37:17.970 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.964+0000 7fd063fff640 1 --2- 172.21.15.169:0/1023930040 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd04406ae40 0x7fd04406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:17.970 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.965+0000 7fd063fff640 1 --2- 172.21.15.169:0/1023930040 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd04406ae40 0x7fd04406d300 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fd054001d60 tx=0x7fd054009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:17.971 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:17.968+0000 7fd0617fa640 1 -- 172.21.15.169:0/1023930040 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd04c076f40 con 0x7fd0640cb8e0 2023-10-26T01:37:18.164 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:18.162+0000 7fd06a9f5640 1 -- 172.21.15.169:0/1023930040 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd0641568a0 con 0x7fd0640cb8e0 2023-10-26T01:37:18.165 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:18.163+0000 7fd0617fa640 1 -- 172.21.15.169:0/1023930040 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v109) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd04c0420a0 con 0x7fd0640cb8e0 2023-10-26T01:37:18.166 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 64 2023-10-26T01:37:18.168 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:18.165+0000 7fd06a9f5640 1 -- 172.21.15.169:0/1023930040 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd04406ae40 msgr2=0x7fd04406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:18.168 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:18.165+0000 7fd06a9f5640 1 --2- 172.21.15.169:0/1023930040 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd04406ae40 0x7fd04406d300 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fd054001d60 tx=0x7fd054009040 comp rx=0 tx=0).stop 2023-10-26T01:37:18.169 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:18.165+0000 7fd06a9f5640 1 -- 172.21.15.169:0/1023930040 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd0640cb8e0 msgr2=0x7fd06414d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:18.169 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:18.165+0000 7fd06a9f5640 1 --2- 172.21.15.169:0/1023930040 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd0640cb8e0 0x7fd06414d7a0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fd04c000c00 tx=0x7fd04c0038a0 comp rx=0 tx=0).stop 2023-10-26T01:37:18.169 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:18.165+0000 7fd06a9f5640 1 -- 172.21.15.169:0/1023930040 shutdown_connections 2023-10-26T01:37:18.169 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:18.165+0000 7fd06a9f5640 1 --2- 172.21.15.169:0/1023930040 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd04406ae40 0x7fd04406d300 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:18.169 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:18.165+0000 7fd06a9f5640 1 --2- 172.21.15.169:0/1023930040 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd0640cb8e0 0x7fd06414d7a0 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:18.169 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:18.165+0000 7fd06a9f5640 1 --2- 172.21.15.169:0/1023930040 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd06415ca70 0x7fd064157b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:18.170 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:18.165+0000 7fd06a9f5640 1 --2- 172.21.15.169:0/1023930040 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd0640713d0 0x7fd0641508f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:18.170 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:18.165+0000 7fd06a9f5640 1 -- 172.21.15.169:0/1023930040 >> 172.21.15.169:0/1023930040 conn(0x7fd0640af8a0 msgr2=0x7fd06414e760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:18.170 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:18.166+0000 7fd06a9f5640 1 -- 172.21.15.169:0/1023930040 shutdown_connections 2023-10-26T01:37:18.170 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:18.166+0000 7fd06a9f5640 1 -- 172.21.15.169:0/1023930040 wait complete. 2023-10-26T01:37:18.187 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-26T01:37:18.414 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:18.411+0000 7f4483894640 1 -- 172.21.15.169:0/2728768846 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4468004d00 msgr2=0x7f4468005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:18.415 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:18.411+0000 7f4483894640 1 --2- 172.21.15.169:0/2728768846 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4468004d00 0x7f4468005180 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f447c0655f0 tx=0x7f44680057f0 comp rx=0 tx=0).stop 2023-10-26T01:37:18.415 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:18.411+0000 7f4481e31640 1 -- 172.21.15.169:0/2728768846 reap_dead start 2023-10-26T01:37:18.415 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:18.411+0000 7f4483894640 1 -- 172.21.15.169:0/2728768846 shutdown_connections 2023-10-26T01:37:18.415 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:18.411+0000 7f4483894640 1 -- 172.21.15.169:0/2728768846 >> 172.21.15.169:0/2728768846 conn(0x7f447c0af8a0 msgr2=0x7f447c0afc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:18.416 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:18.412+0000 7f4483894640 1 -- 172.21.15.169:0/2728768846 shutdown_connections 2023-10-26T01:37:18.416 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:18.412+0000 7f4483894640 1 -- 172.21.15.169:0/2728768846 wait complete. 2023-10-26T01:37:18.416 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:18.413+0000 7f4483894640 1 Processor -- start 2023-10-26T01:37:18.417 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:18.414+0000 7f4483894640 1 -- start start 2023-10-26T01:37:18.418 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:18.414+0000 7f4483894640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4468004d00 0x7f447c147480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:18.419 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:18.414+0000 7f4483894640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f447c15bc70 0x7f447c1558d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:18.419 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:18.415+0000 7f4483894640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f447c15c0d0 0x7f447c155e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:18.419 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:18.415+0000 7f4483894640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f447c155440 con 0x7f4468004d00 2023-10-26T01:37:18.419 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:18.415+0000 7f4483894640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f447c1479c0 con 0x7f447c15c0d0 2023-10-26T01:37:18.419 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:18.415+0000 7f4483894640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f447c154830 con 0x7f447c15bc70 2023-10-26T01:37:18.420 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:18.415+0000 7f4481630640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4468004d00 0x7f447c147480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:18.420 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:18.415+0000 7f4481630640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4468004d00 0x7f447c147480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:32836/0 (socket says 172.21.15.169:32836) 2023-10-26T01:37:18.420 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:18.415+0000 7f4481630640 1 -- 172.21.15.169:0/4059694080 learned_addr learned my addr 172.21.15.169:0/4059694080 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:37:18.420 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:18.415+0000 7f4481630640 1 -- 172.21.15.169:0/4059694080 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f447c15bc70 msgr2=0x7f447c1558d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:18.420 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:18.415+0000 7f4481630640 1 --2- 172.21.15.169:0/4059694080 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f447c15bc70 0x7f447c1558d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:18.421 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:18.415+0000 7f4481630640 1 -- 172.21.15.169:0/4059694080 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f447c15c0d0 msgr2=0x7f447c155e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:18.421 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:18.415+0000 7f4481630640 1 --2- 172.21.15.169:0/4059694080 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f447c15c0d0 0x7f447c155e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:18.421 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:18.415+0000 7f4481630640 1 -- 172.21.15.169:0/4059694080 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f447c154b10 con 0x7f4468004d00 2023-10-26T01:37:18.421 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:18.415+0000 7f4481630640 1 --2- 172.21.15.169:0/4059694080 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4468004d00 0x7f447c147480 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f447800e6d0 tx=0x7f447800eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:18.421 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:18.416+0000 7f446dffb640 1 -- 172.21.15.169:0/4059694080 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4478004250 con 0x7f4468004d00 2023-10-26T01:37:18.422 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:18.416+0000 7f446dffb640 1 -- 172.21.15.169:0/4059694080 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f44780044a0 con 0x7f4468004d00 2023-10-26T01:37:18.422 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:18.416+0000 7f446dffb640 1 -- 172.21.15.169:0/4059694080 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f447800ca20 con 0x7f4468004d00 2023-10-26T01:37:18.422 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:18.416+0000 7f4483894640 1 -- 172.21.15.169:0/4059694080 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f447c154df0 con 0x7f4468004d00 2023-10-26T01:37:18.422 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:18.416+0000 7f4483894640 1 -- 172.21.15.169:0/4059694080 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f447c0ca770 con 0x7f4468004d00 2023-10-26T01:37:18.423 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:18.417+0000 7f4483894640 1 -- 172.21.15.169:0/4059694080 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f447c04db10 con 0x7f4468004d00 2023-10-26T01:37:18.423 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:18.418+0000 7f446dffb640 1 -- 172.21.15.169:0/4059694080 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f44780035f0 con 0x7f4468004d00 2023-10-26T01:37:18.423 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:18.419+0000 7f446dffb640 1 --2- 172.21.15.169:0/4059694080 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f446006ae40 0x7f446006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:18.424 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:18.419+0000 7f446dffb640 1 -- 172.21.15.169:0/4059694080 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4478089fe0 con 0x7f4468004d00 2023-10-26T01:37:18.424 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:18.420+0000 7f4480e2f640 1 --2- 172.21.15.169:0/4059694080 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f446006ae40 0x7f446006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:18.424 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:18.421+0000 7f4480e2f640 1 --2- 172.21.15.169:0/4059694080 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f446006ae40 0x7f446006d300 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f447c1f5470 tx=0x7f4470006920 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:18.426 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:18.424+0000 7f446dffb640 1 -- 172.21.15.169:0/4059694080 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4478054ff0 con 0x7f4468004d00 2023-10-26T01:37:18.629 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:18.626+0000 7f4483894640 1 -- 172.21.15.169:0/4059694080 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f447c0adc40 con 0x7f4468004d00 2023-10-26T01:37:18.630 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:18.627+0000 7f446dffb640 1 -- 172.21.15.169:0/4059694080 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v109) v1 ==== 80+0+0 (secure 0 0 0) 0x7f4478015080 con 0x7f4468004d00 2023-10-26T01:37:18.630 INFO:teuthology.orchestra.run.smithi169.stderr:listed 0 entries 2023-10-26T01:37:18.633 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:18.630+0000 7f4483894640 1 -- 172.21.15.169:0/4059694080 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f446006ae40 msgr2=0x7f446006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:18.633 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:18.630+0000 7f4483894640 1 --2- 172.21.15.169:0/4059694080 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f446006ae40 0x7f446006d300 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f447c1f5470 tx=0x7f4470006920 comp rx=0 tx=0).stop 2023-10-26T01:37:18.633 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:18.630+0000 7f4483894640 1 -- 172.21.15.169:0/4059694080 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4468004d00 msgr2=0x7f447c147480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:18.633 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:18.630+0000 7f4483894640 1 --2- 172.21.15.169:0/4059694080 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4468004d00 0x7f447c147480 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f447800e6d0 tx=0x7f447800eb90 comp rx=0 tx=0).stop 2023-10-26T01:37:18.633 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:18.630+0000 7f4483894640 1 -- 172.21.15.169:0/4059694080 shutdown_connections 2023-10-26T01:37:18.634 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:18.630+0000 7f4483894640 1 --2- 172.21.15.169:0/4059694080 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f446006ae40 0x7f446006d300 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:18.634 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:18.630+0000 7f4483894640 1 --2- 172.21.15.169:0/4059694080 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f447c15c0d0 0x7f447c155e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:18.634 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:18.630+0000 7f4483894640 1 --2- 172.21.15.169:0/4059694080 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f447c15bc70 0x7f447c1558d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:18.634 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:18.630+0000 7f4483894640 1 --2- 172.21.15.169:0/4059694080 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4468004d00 0x7f447c147480 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:18.634 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:18.630+0000 7f4483894640 1 -- 172.21.15.169:0/4059694080 >> 172.21.15.169:0/4059694080 conn(0x7f447c0af8a0 msgr2=0x7f447c0b0db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:18.635 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:18.630+0000 7f4483894640 1 -- 172.21.15.169:0/4059694080 shutdown_connections 2023-10-26T01:37:18.635 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:18.630+0000 7f4483894640 1 -- 172.21.15.169:0/4059694080 wait complete. 2023-10-26T01:37:18.652 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-26T01:37:18.653 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:37:18.653 DEBUG:teuthology.orchestra.run.smithi169:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-26T01:37:18.804 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi169.front.sepia.ceph.com 2023-10-26T01:37:18.804 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:37:18.805 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-26T01:37:18.853 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-26T01:37:18.854 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:37:18.899 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-26T01:37:18.900 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:37:18.993 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:37:19.225 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:19.222+0000 7f5393dd0640 1 -- 172.21.15.169:0/2598542871 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53780047c0 msgr2=0x7f5378004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:19.226 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:19.222+0000 7f5393dd0640 1 --2- 172.21.15.169:0/2598542871 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53780047c0 0x7f5378004c40 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f537c00bb20 tx=0x7f53780057f0 comp rx=0 tx=0).stop 2023-10-26T01:37:19.226 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:19.222+0000 7f539236d640 1 -- 172.21.15.169:0/2598542871 reap_dead start 2023-10-26T01:37:19.226 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:19.222+0000 7f5393dd0640 1 -- 172.21.15.169:0/2598542871 shutdown_connections 2023-10-26T01:37:19.227 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:19.222+0000 7f5393dd0640 1 -- 172.21.15.169:0/2598542871 >> 172.21.15.169:0/2598542871 conn(0x7f538c069930 msgr2=0x7f538c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:19.227 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:19.223+0000 7f5393dd0640 1 -- 172.21.15.169:0/2598542871 shutdown_connections 2023-10-26T01:37:19.227 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:19.223+0000 7f5393dd0640 1 -- 172.21.15.169:0/2598542871 wait complete. 2023-10-26T01:37:19.228 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:19.224+0000 7f5393dd0640 1 Processor -- start 2023-10-26T01:37:19.229 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:19.224+0000 7f5393dd0640 1 -- start start 2023-10-26T01:37:19.229 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:19.225+0000 7f5393dd0640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53780047c0 0x7f538c0cc600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:19.229 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:19.225+0000 7f5393dd0640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f538c09a510 0x7f538c1561d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:19.229 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:19.225+0000 7f5393dd0640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f538c156710 0x7f538c1544d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:19.229 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:19.225+0000 7f5393dd0640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f538c156bc0 con 0x7f53780047c0 2023-10-26T01:37:19.230 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:19.225+0000 7f5393dd0640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f538c0ccb40 con 0x7f538c09a510 2023-10-26T01:37:19.230 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:19.225+0000 7f5393dd0640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f538c1f3740 con 0x7f538c156710 2023-10-26T01:37:19.230 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:19.226+0000 7f5391b6c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53780047c0 0x7f538c0cc600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:19.230 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:19.226+0000 7f5391b6c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53780047c0 0x7f538c0cc600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:32864/0 (socket says 172.21.15.169:32864) 2023-10-26T01:37:19.231 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:19.226+0000 7f5391b6c640 1 -- 172.21.15.169:0/3701206198 learned_addr learned my addr 172.21.15.169:0/3701206198 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:37:19.231 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:19.226+0000 7f539136b640 1 --2- 172.21.15.169:0/3701206198 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f538c09a510 0x7f538c1561d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:19.231 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:19.226+0000 7f539236d640 1 --2- 172.21.15.169:0/3701206198 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f538c156710 0x7f538c1544d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:19.231 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:19.226+0000 7f5391b6c640 1 -- 172.21.15.169:0/3701206198 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f538c156710 msgr2=0x7f538c1544d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:19.231 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:19.226+0000 7f5391b6c640 1 --2- 172.21.15.169:0/3701206198 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f538c156710 0x7f538c1544d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:19.232 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:19.226+0000 7f5391b6c640 1 -- 172.21.15.169:0/3701206198 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f538c09a510 msgr2=0x7f538c1561d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:19.232 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:19.226+0000 7f5391b6c640 1 --2- 172.21.15.169:0/3701206198 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f538c09a510 0x7f538c1561d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:19.232 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:19.226+0000 7f5391b6c640 1 -- 172.21.15.169:0/3701206198 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f538c1f2bc0 con 0x7f53780047c0 2023-10-26T01:37:19.233 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:19.226+0000 7f5391b6c640 1 --2- 172.21.15.169:0/3701206198 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53780047c0 0x7f538c0cc600 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f5388001eb0 tx=0x7f538800da80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:19.233 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:19.227+0000 7f53827fc640 1 -- 172.21.15.169:0/3701206198 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53880062e0 con 0x7f53780047c0 2023-10-26T01:37:19.234 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:19.227+0000 7f53827fc640 1 -- 172.21.15.169:0/3701206198 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5388006530 con 0x7f53780047c0 2023-10-26T01:37:19.234 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:19.227+0000 7f53827fc640 1 -- 172.21.15.169:0/3701206198 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5388003850 con 0x7f53780047c0 2023-10-26T01:37:19.234 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:19.227+0000 7f5393dd0640 1 -- 172.21.15.169:0/3701206198 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f538c1f2ea0 con 0x7f53780047c0 2023-10-26T01:37:19.234 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:19.227+0000 7f5393dd0640 1 -- 172.21.15.169:0/3701206198 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f538c1f3460 con 0x7f53780047c0 2023-10-26T01:37:19.235 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:19.228+0000 7f5393dd0640 1 -- 172.21.15.169:0/3701206198 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f538c04db10 con 0x7f53780047c0 2023-10-26T01:37:19.235 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:19.229+0000 7f53827fc640 1 -- 172.21.15.169:0/3701206198 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f53880039b0 con 0x7f53780047c0 2023-10-26T01:37:19.235 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:19.230+0000 7f53827fc640 1 --2- 172.21.15.169:0/3701206198 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f537006ae40 0x7f537006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:19.235 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:19.230+0000 7f53827fc640 1 -- 172.21.15.169:0/3701206198 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5388089ad0 con 0x7f53780047c0 2023-10-26T01:37:19.235 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:19.231+0000 7f539136b640 1 --2- 172.21.15.169:0/3701206198 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f537006ae40 0x7f537006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:19.236 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:19.232+0000 7f539136b640 1 --2- 172.21.15.169:0/3701206198 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f537006ae40 0x7f537006d300 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f537c037000 tx=0x7f537c02e040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:19.238 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:19.235+0000 7f53827fc640 1 -- 172.21.15.169:0/3701206198 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5388054c10 con 0x7f53780047c0 2023-10-26T01:37:19.426 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:19.423+0000 7f5393dd0640 1 -- 172.21.15.169:0/3701206198 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f538c0aa570 con 0x7f53780047c0 2023-10-26T01:37:19.429 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:19.426+0000 7f53827fc640 1 -- 172.21.15.169:0/3701206198 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v109) v1 ==== 74+0+16620 (secure 0 0 0) 0x7f5388014070 con 0x7f53780047c0 2023-10-26T01:37:19.430 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:37:19.430 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":109,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:37:13.181064+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-10-26T01:36:41.430455+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-10-26T01:36:42.883252+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:37:19.432 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:19.429+0000 7f5393dd0640 1 -- 172.21.15.169:0/3701206198 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f537006ae40 msgr2=0x7f537006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:19.432 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:19.429+0000 7f5393dd0640 1 --2- 172.21.15.169:0/3701206198 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f537006ae40 0x7f537006d300 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f537c037000 tx=0x7f537c02e040 comp rx=0 tx=0).stop 2023-10-26T01:37:19.433 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:19.429+0000 7f5393dd0640 1 -- 172.21.15.169:0/3701206198 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53780047c0 msgr2=0x7f538c0cc600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:19.433 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:19.429+0000 7f5393dd0640 1 --2- 172.21.15.169:0/3701206198 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53780047c0 0x7f538c0cc600 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f5388001eb0 tx=0x7f538800da80 comp rx=0 tx=0).stop 2023-10-26T01:37:19.433 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:19.429+0000 7f5393dd0640 1 -- 172.21.15.169:0/3701206198 shutdown_connections 2023-10-26T01:37:19.433 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:19.429+0000 7f5393dd0640 1 --2- 172.21.15.169:0/3701206198 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f537006ae40 0x7f537006d300 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:19.433 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:19.429+0000 7f5393dd0640 1 --2- 172.21.15.169:0/3701206198 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f538c156710 0x7f538c1544d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:19.434 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:19.429+0000 7f5393dd0640 1 --2- 172.21.15.169:0/3701206198 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f538c09a510 0x7f538c1561d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:19.434 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:19.429+0000 7f5393dd0640 1 --2- 172.21.15.169:0/3701206198 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53780047c0 0x7f538c0cc600 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:19.434 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:19.429+0000 7f5393dd0640 1 -- 172.21.15.169:0/3701206198 >> 172.21.15.169:0/3701206198 conn(0x7f538c069930 msgr2=0x7f538c150a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:19.434 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:19.429+0000 7f5393dd0640 1 -- 172.21.15.169:0/3701206198 shutdown_connections 2023-10-26T01:37:19.434 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:19.429+0000 7f5393dd0640 1 -- 172.21.15.169:0/3701206198 wait complete. 2023-10-26T01:37:19.450 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T01:30:32.200808+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '109', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-26T01:36:41.430455+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '109', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2023-10-26T01:36:42.883252+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '109', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T01:37:19.452 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T01:37:19.680 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:19.677+0000 7f0d86ac2640 1 -- 172.21.15.169:0/4043956272 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0d800d00d0 msgr2=0x7f0d800b2e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:19.681 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:19.677+0000 7f0d86ac2640 1 --2- 172.21.15.169:0/4043956272 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0d800d00d0 0x7f0d800b2e80 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f0d800706d0 tx=0x7f0d6802c1d0 comp rx=0 tx=0).stop 2023-10-26T01:37:19.681 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:19.677+0000 7f0d86ac2640 1 -- 172.21.15.169:0/4043956272 shutdown_connections 2023-10-26T01:37:19.682 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:19.677+0000 7f0d86ac2640 1 --2- 172.21.15.169:0/4043956272 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0d800b2810 0x7f0d800b33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:19.682 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:19.677+0000 7f0d86ac2640 1 --2- 172.21.15.169:0/4043956272 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0d800d00d0 0x7f0d800b2e80 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:19.682 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:19.677+0000 7f0d86ac2640 1 --2- 172.21.15.169:0/4043956272 >> v2:172.21.15.170:3300/0 conn(0x7f0d80159570 0x7f0d8015a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:19.683 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:19.677+0000 7f0d86ac2640 1 -- 172.21.15.169:0/4043956272 >> 172.21.15.169:0/4043956272 conn(0x7f0d80098a60 msgr2=0x7f0d800b4890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:19.683 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:19.677+0000 7f0d86ac2640 1 -- 172.21.15.169:0/4043956272 shutdown_connections 2023-10-26T01:37:19.683 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:19.677+0000 7f0d86ac2640 1 -- 172.21.15.169:0/4043956272 wait complete. 2023-10-26T01:37:19.688 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:19.679+0000 7f0d86ac2640 1 Processor -- start 2023-10-26T01:37:19.688 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:19.679+0000 7f0d86ac2640 1 -- start start 2023-10-26T01:37:19.689 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:19.680+0000 7f0d86ac2640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0d800b2810 0x7f0d80156b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:19.689 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:19.680+0000 7f0d86ac2640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0d800d00d0 0x7f0d80158860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:19.689 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:19.680+0000 7f0d86ac2640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0d80159570 0x7f0d80143df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:19.690 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:19.680+0000 7f0d86ac2640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d80157e70 con 0x7f0d800b2810 2023-10-26T01:37:19.690 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:19.680+0000 7f0d86ac2640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d80158da0 con 0x7f0d800d00d0 2023-10-26T01:37:19.691 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:19.680+0000 7f0d86ac2640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d801571f0 con 0x7f0d80159570 2023-10-26T01:37:19.691 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:19.680+0000 7f0d8485e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0d800b2810 0x7f0d80156b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:19.691 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:19.680+0000 7f0d8485e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0d800b2810 0x7f0d80156b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:32882/0 (socket says 172.21.15.169:32882) 2023-10-26T01:37:19.692 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:19.680+0000 7f0d8485e640 1 -- 172.21.15.169:0/834544700 learned_addr learned my addr 172.21.15.169:0/834544700 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:37:19.692 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:19.680+0000 7f0d8485e640 1 -- 172.21.15.169:0/834544700 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0d80159570 msgr2=0x7f0d80143df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:19.692 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:19.680+0000 7f0d8485e640 1 --2- 172.21.15.169:0/834544700 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0d80159570 0x7f0d80143df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:19.693 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:19.680+0000 7f0d8485e640 1 -- 172.21.15.169:0/834544700 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0d800d00d0 msgr2=0x7f0d80158860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:19.693 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:19.680+0000 7f0d8485e640 1 --2- 172.21.15.169:0/834544700 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0d800d00d0 0x7f0d80158860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:19.693 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:19.680+0000 7f0d8485e640 1 -- 172.21.15.169:0/834544700 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d68009900 con 0x7f0d800b2810 2023-10-26T01:37:19.694 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:19.681+0000 7f0d8485e640 1 --2- 172.21.15.169:0/834544700 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0d800b2810 0x7f0d80156b60 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f0d7000d740 tx=0x7f0d7000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:19.694 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:19.681+0000 7f0d7d7fa640 1 -- 172.21.15.169:0/834544700 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d70009c30 con 0x7f0d800b2810 2023-10-26T01:37:19.695 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:19.681+0000 7f0d7d7fa640 1 -- 172.21.15.169:0/834544700 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0d70009e80 con 0x7f0d800b2810 2023-10-26T01:37:19.695 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:19.681+0000 7f0d7d7fa640 1 -- 172.21.15.169:0/834544700 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d700106c0 con 0x7f0d800b2810 2023-10-26T01:37:19.695 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:19.681+0000 7f0d86ac2640 1 -- 172.21.15.169:0/834544700 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d801574d0 con 0x7f0d800b2810 2023-10-26T01:37:19.696 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:19.681+0000 7f0d86ac2640 1 -- 172.21.15.169:0/834544700 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d80157a90 con 0x7f0d800b2810 2023-10-26T01:37:19.696 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:19.682+0000 7f0d86ac2640 1 -- 172.21.15.169:0/834544700 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d8004da70 con 0x7f0d800b2810 2023-10-26T01:37:19.696 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:19.683+0000 7f0d7d7fa640 1 -- 172.21.15.169:0/834544700 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f0d70010820 con 0x7f0d800b2810 2023-10-26T01:37:19.697 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:19.685+0000 7f0d7d7fa640 1 --2- 172.21.15.169:0/834544700 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f0d6006ae40 0x7f0d6006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:19.697 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:19.685+0000 7f0d7d7fa640 1 -- 172.21.15.169:0/834544700 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0d70089a80 con 0x7f0d800b2810 2023-10-26T01:37:19.697 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:19.685+0000 7f0d7ffff640 1 --2- 172.21.15.169:0/834544700 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f0d6006ae40 0x7f0d6006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:19.699 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:19.686+0000 7f0d7ffff640 1 --2- 172.21.15.169:0/834544700 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f0d6006ae40 0x7f0d6006d300 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f0d68036000 tx=0x7f0d68004a40 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:19.699 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:19.690+0000 7f0d7d7fa640 1 -- 172.21.15.169:0/834544700 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0d70054bc0 con 0x7f0d800b2810 2023-10-26T01:37:19.879 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:19.877+0000 7f0d86ac2640 1 -- 172.21.15.169:0/834544700 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0d800b3160 con 0x7f0d800b2810 2023-10-26T01:37:19.880 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:19.878+0000 7f0d7d7fa640 1 -- 172.21.15.169:0/834544700 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v109) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0d700210a0 con 0x7f0d800b2810 2023-10-26T01:37:19.881 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 1 2023-10-26T01:37:19.884 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:19.880+0000 7f0d86ac2640 1 -- 172.21.15.169:0/834544700 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f0d6006ae40 msgr2=0x7f0d6006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:19.884 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:19.880+0000 7f0d86ac2640 1 --2- 172.21.15.169:0/834544700 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f0d6006ae40 0x7f0d6006d300 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f0d68036000 tx=0x7f0d68004a40 comp rx=0 tx=0).stop 2023-10-26T01:37:19.884 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:19.880+0000 7f0d86ac2640 1 -- 172.21.15.169:0/834544700 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0d800b2810 msgr2=0x7f0d80156b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:19.886 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:19.880+0000 7f0d86ac2640 1 --2- 172.21.15.169:0/834544700 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0d800b2810 0x7f0d80156b60 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f0d7000d740 tx=0x7f0d7000dc00 comp rx=0 tx=0).stop 2023-10-26T01:37:19.886 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:19.880+0000 7f0d86ac2640 1 -- 172.21.15.169:0/834544700 shutdown_connections 2023-10-26T01:37:19.886 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:19.880+0000 7f0d86ac2640 1 --2- 172.21.15.169:0/834544700 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f0d6006ae40 0x7f0d6006d300 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:19.886 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:19.880+0000 7f0d86ac2640 1 --2- 172.21.15.169:0/834544700 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0d80159570 0x7f0d80143df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:19.887 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:19.880+0000 7f0d86ac2640 1 --2- 172.21.15.169:0/834544700 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0d800d00d0 0x7f0d80158860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:19.887 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:19.880+0000 7f0d86ac2640 1 --2- 172.21.15.169:0/834544700 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0d800b2810 0x7f0d80156b60 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:19.887 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:19.880+0000 7f0d86ac2640 1 -- 172.21.15.169:0/834544700 >> 172.21.15.169:0/834544700 conn(0x7f0d80098a60 msgr2=0x7f0d800b17e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:19.887 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:19.881+0000 7f0d86ac2640 1 -- 172.21.15.169:0/834544700 shutdown_connections 2023-10-26T01:37:19.887 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:19.881+0000 7f0d86ac2640 1 -- 172.21.15.169:0/834544700 wait complete. 2023-10-26T01:37:19.902 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-26T01:37:20.134 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.131+0000 7f814e4da640 1 -- 172.21.15.169:0/1116410243 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f814815f450 msgr2=0x7f814815f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:20.134 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.131+0000 7f814e4da640 1 --2- 172.21.15.169:0/1116410243 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f814815f450 0x7f814815f8e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f8140005fd0 tx=0x7f814002c0e0 comp rx=0 tx=0).stop 2023-10-26T01:37:20.135 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.131+0000 7f814e4da640 1 -- 172.21.15.169:0/1116410243 shutdown_connections 2023-10-26T01:37:20.135 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.131+0000 7f814e4da640 1 --2- 172.21.15.169:0/1116410243 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f814815f450 0x7f814815f8e0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:20.135 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.131+0000 7f814e4da640 1 --2- 172.21.15.169:0/1116410243 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f81480ac130 0x7f814806aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:20.135 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.131+0000 7f814e4da640 1 -- 172.21.15.169:0/1116410243 >> 172.21.15.169:0/1116410243 conn(0x7f81480af110 msgr2=0x7f81480af520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:20.135 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.131+0000 7f814e4da640 1 -- 172.21.15.169:0/1116410243 shutdown_connections 2023-10-26T01:37:20.136 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.131+0000 7f814e4da640 1 -- 172.21.15.169:0/1116410243 wait complete. 2023-10-26T01:37:20.136 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.133+0000 7f814e4da640 1 Processor -- start 2023-10-26T01:37:20.136 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.133+0000 7f814e4da640 1 -- start start 2023-10-26T01:37:20.137 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.134+0000 7f814e4da640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f81480ac130 0x7f8148146530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:20.138 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.135+0000 7f814e4da640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81480ac510 0x7f8148099f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:20.138 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.135+0000 7f814e4da640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f814815f450 0x7f814809a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:20.138 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.135+0000 7f814e4da640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81480971c0 con 0x7f81480ac510 2023-10-26T01:37:20.138 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.135+0000 7f814e4da640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8148097780 con 0x7f814815f450 2023-10-26T01:37:20.139 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.135+0000 7f814e4da640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8148154690 con 0x7f81480ac130 2023-10-26T01:37:20.139 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.135+0000 7f814ccd7640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81480ac510 0x7f8148099f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:20.139 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.135+0000 7f814ccd7640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81480ac510 0x7f8148099f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:32894/0 (socket says 172.21.15.169:32894) 2023-10-26T01:37:20.139 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.135+0000 7f814ccd7640 1 -- 172.21.15.169:0/2211415336 learned_addr learned my addr 172.21.15.169:0/2211415336 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:37:20.139 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.135+0000 7f814d4d8640 1 --2- 172.21.15.169:0/2211415336 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f81480ac130 0x7f8148146530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:20.141 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.135+0000 7f814dcd9640 1 --2- 172.21.15.169:0/2211415336 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f814815f450 0x7f814809a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:20.141 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.135+0000 7f814ccd7640 1 -- 172.21.15.169:0/2211415336 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f81480ac130 msgr2=0x7f8148146530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:20.141 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.135+0000 7f814ccd7640 1 --2- 172.21.15.169:0/2211415336 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f81480ac130 0x7f8148146530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:20.141 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.135+0000 7f814ccd7640 1 -- 172.21.15.169:0/2211415336 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f814815f450 msgr2=0x7f814809a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:20.142 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.135+0000 7f814ccd7640 1 --2- 172.21.15.169:0/2211415336 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f814815f450 0x7f814809a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:20.142 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.136+0000 7f814ccd7640 1 -- 172.21.15.169:0/2211415336 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8144009560 con 0x7f81480ac510 2023-10-26T01:37:20.142 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.136+0000 7f814dcd9640 1 --2- 172.21.15.169:0/2211415336 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f814815f450 0x7f814809a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:37:20.142 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.136+0000 7f814ccd7640 1 --2- 172.21.15.169:0/2211415336 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81480ac510 0x7f8148099f10 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f813800c4c0 tx=0x7f813800c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:20.142 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.136+0000 7f813dffb640 1 -- 172.21.15.169:0/2211415336 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81380052d0 con 0x7f81480ac510 2023-10-26T01:37:20.143 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.136+0000 7f813dffb640 1 -- 172.21.15.169:0/2211415336 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8138005430 con 0x7f81480ac510 2023-10-26T01:37:20.143 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.136+0000 7f813dffb640 1 -- 172.21.15.169:0/2211415336 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8138012890 con 0x7f81480ac510 2023-10-26T01:37:20.143 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.136+0000 7f814e4da640 1 -- 172.21.15.169:0/2211415336 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8140009840 con 0x7f81480ac510 2023-10-26T01:37:20.143 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.136+0000 7f814e4da640 1 -- 172.21.15.169:0/2211415336 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8148097410 con 0x7f81480ac510 2023-10-26T01:37:20.143 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.137+0000 7f814e4da640 1 -- 172.21.15.169:0/2211415336 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8148097910 con 0x7f81480ac510 2023-10-26T01:37:20.144 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.138+0000 7f813dffb640 1 -- 172.21.15.169:0/2211415336 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f81380035f0 con 0x7f81480ac510 2023-10-26T01:37:20.144 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.139+0000 7f813dffb640 1 --2- 172.21.15.169:0/2211415336 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f812c06ae40 0x7f812c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:20.145 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.139+0000 7f813dffb640 1 -- 172.21.15.169:0/2211415336 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8138089390 con 0x7f81480ac510 2023-10-26T01:37:20.145 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.140+0000 7f814d4d8640 1 --2- 172.21.15.169:0/2211415336 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f812c06ae40 0x7f812c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:20.145 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.141+0000 7f814d4d8640 1 --2- 172.21.15.169:0/2211415336 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f812c06ae40 0x7f812c06d300 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f8148098e80 tx=0x7f8144009260 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:20.146 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.144+0000 7f813dffb640 1 -- 172.21.15.169:0/2211415336 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8138054430 con 0x7f81480ac510 2023-10-26T01:37:20.330 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.327+0000 7f814e4da640 1 -- 172.21.15.169:0/2211415336 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8148073c10 con 0x7f81480ac510 2023-10-26T01:37:20.330 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.328+0000 7f813dffb640 1 -- 172.21.15.169:0/2211415336 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v109) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8138057a50 con 0x7f81480ac510 2023-10-26T01:37:20.331 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 32 2023-10-26T01:37:20.333 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.330+0000 7f814e4da640 1 -- 172.21.15.169:0/2211415336 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f812c06ae40 msgr2=0x7f812c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:20.333 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.330+0000 7f814e4da640 1 --2- 172.21.15.169:0/2211415336 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f812c06ae40 0x7f812c06d300 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f8148098e80 tx=0x7f8144009260 comp rx=0 tx=0).stop 2023-10-26T01:37:20.333 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.330+0000 7f814e4da640 1 -- 172.21.15.169:0/2211415336 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81480ac510 msgr2=0x7f8148099f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:20.334 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.330+0000 7f814e4da640 1 --2- 172.21.15.169:0/2211415336 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81480ac510 0x7f8148099f10 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f813800c4c0 tx=0x7f813800c980 comp rx=0 tx=0).stop 2023-10-26T01:37:20.334 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.330+0000 7f814e4da640 1 -- 172.21.15.169:0/2211415336 shutdown_connections 2023-10-26T01:37:20.334 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.330+0000 7f814e4da640 1 --2- 172.21.15.169:0/2211415336 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f812c06ae40 0x7f812c06d300 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:20.334 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.330+0000 7f814e4da640 1 --2- 172.21.15.169:0/2211415336 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81480ac510 0x7f8148099f10 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:20.334 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.330+0000 7f814e4da640 1 --2- 172.21.15.169:0/2211415336 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f814815f450 0x7f814809a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:20.335 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.330+0000 7f814e4da640 1 --2- 172.21.15.169:0/2211415336 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f81480ac130 0x7f8148146530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:20.335 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.330+0000 7f814e4da640 1 -- 172.21.15.169:0/2211415336 >> 172.21.15.169:0/2211415336 conn(0x7f81480af110 msgr2=0x7f814815d8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:20.335 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.330+0000 7f814e4da640 1 -- 172.21.15.169:0/2211415336 shutdown_connections 2023-10-26T01:37:20.335 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.330+0000 7f814e4da640 1 -- 172.21.15.169:0/2211415336 wait complete. 2023-10-26T01:37:20.351 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-26T01:37:20.564 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.561+0000 7f2c8759e640 1 -- 172.21.15.169:0/2293808217 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c88069ff0 msgr2=0x7f2c88158f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:20.565 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.561+0000 7f2c8759e640 1 --2- 172.21.15.169:0/2293808217 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c88069ff0 0x7f2c88158f30 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f2c8806ce30 tx=0x7f2c7002bda0 comp rx=0 tx=0).stop 2023-10-26T01:37:20.565 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.562+0000 7f2c8759e640 1 -- 172.21.15.169:0/2293808217 shutdown_connections 2023-10-26T01:37:20.565 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.562+0000 7f2c8759e640 1 --2- 172.21.15.169:0/2293808217 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c88069ff0 0x7f2c88158f30 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:20.566 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.562+0000 7f2c8759e640 1 --2- 172.21.15.169:0/2293808217 >> v2:172.21.15.170:3300/0 conn(0x7f2c88069c10 0x7f2c880b0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:20.566 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.562+0000 7f2c8759e640 1 -- 172.21.15.169:0/2293808217 >> 172.21.15.169:0/2293808217 conn(0x7f2c880af110 msgr2=0x7f2c880af520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:20.566 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.562+0000 7f2c8759e640 1 -- 172.21.15.169:0/2293808217 shutdown_connections 2023-10-26T01:37:20.566 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.562+0000 7f2c8759e640 1 -- 172.21.15.169:0/2293808217 wait complete. 2023-10-26T01:37:20.567 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.564+0000 7f2c8759e640 1 Processor -- start 2023-10-26T01:37:20.567 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.564+0000 7f2c8759e640 1 -- start start 2023-10-26T01:37:20.569 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.565+0000 7f2c8759e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c88069c10 0x7f2c880c6830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:20.569 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.565+0000 7f2c8759e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2c88069ff0 0x7f2c8814d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:20.569 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.565+0000 7f2c8759e640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2c88159470 0x7f2c880ca280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:20.570 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.565+0000 7f2c8759e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c880ca7c0 con 0x7f2c88069c10 2023-10-26T01:37:20.570 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.565+0000 7f2c8759e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c880c7ad0 con 0x7f2c88069ff0 2023-10-26T01:37:20.570 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.565+0000 7f2c8759e640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c880c6ec0 con 0x7f2c88159470 2023-10-26T01:37:20.570 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.565+0000 7f2c8659c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c88069c10 0x7f2c880c6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:20.571 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.565+0000 7f2c8659c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c88069c10 0x7f2c880c6830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:32914/0 (socket says 172.21.15.169:32914) 2023-10-26T01:37:20.571 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.565+0000 7f2c8659c640 1 -- 172.21.15.169:0/1177338908 learned_addr learned my addr 172.21.15.169:0/1177338908 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:37:20.571 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.566+0000 7f2c8659c640 1 -- 172.21.15.169:0/1177338908 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2c88159470 msgr2=0x7f2c880ca280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:20.571 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.566+0000 7f2c8659c640 1 --2- 172.21.15.169:0/1177338908 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2c88159470 0x7f2c880ca280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:20.571 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.566+0000 7f2c8659c640 1 -- 172.21.15.169:0/1177338908 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2c88069ff0 msgr2=0x7f2c8814d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:20.571 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.566+0000 7f2c8659c640 1 --2- 172.21.15.169:0/1177338908 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2c88069ff0 0x7f2c8814d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:20.572 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.566+0000 7f2c8659c640 1 -- 172.21.15.169:0/1177338908 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c70009560 con 0x7f2c88069c10 2023-10-26T01:37:20.572 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.566+0000 7f2c8659c640 1 --2- 172.21.15.169:0/1177338908 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c88069c10 0x7f2c880c6830 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f2c7800b4a0 tx=0x7f2c7800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:20.572 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.566+0000 7f2c7effd640 1 -- 172.21.15.169:0/1177338908 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c78010610 con 0x7f2c88069c10 2023-10-26T01:37:20.572 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.566+0000 7f2c7effd640 1 -- 172.21.15.169:0/1177338908 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2c78010860 con 0x7f2c88069c10 2023-10-26T01:37:20.572 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.566+0000 7f2c7effd640 1 -- 172.21.15.169:0/1177338908 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c78010c70 con 0x7f2c88069c10 2023-10-26T01:37:20.573 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.566+0000 7f2c8759e640 1 -- 172.21.15.169:0/1177338908 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c880c71a0 con 0x7f2c88069c10 2023-10-26T01:37:20.573 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.567+0000 7f2c8759e640 1 -- 172.21.15.169:0/1177338908 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c880c7760 con 0x7f2c88069c10 2023-10-26T01:37:20.574 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.568+0000 7f2c8759e640 1 -- 172.21.15.169:0/1177338908 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c88073c10 con 0x7f2c88069c10 2023-10-26T01:37:20.574 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.569+0000 7f2c7effd640 1 -- 172.21.15.169:0/1177338908 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f2c78017360 con 0x7f2c88069c10 2023-10-26T01:37:20.574 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.569+0000 7f2c7effd640 1 --2- 172.21.15.169:0/1177338908 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2c6806ae40 0x7f2c6806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:20.574 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.570+0000 7f2c7effd640 1 -- 172.21.15.169:0/1177338908 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2c780895c0 con 0x7f2c88069c10 2023-10-26T01:37:20.575 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.570+0000 7f2c7ffff640 1 --2- 172.21.15.169:0/1177338908 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2c6806ae40 0x7f2c6806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:20.575 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.571+0000 7f2c7ffff640 1 --2- 172.21.15.169:0/1177338908 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2c6806ae40 0x7f2c6806d300 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f2c70035000 tx=0x7f2c700046c0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:20.576 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.574+0000 7f2c7effd640 1 -- 172.21.15.169:0/1177338908 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2c78058080 con 0x7f2c88069c10 2023-10-26T01:37:20.765 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.762+0000 7f2c8759e640 1 -- 172.21.15.169:0/1177338908 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2c880add10 con 0x7f2c88069c10 2023-10-26T01:37:20.766 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.763+0000 7f2c7effd640 1 -- 172.21.15.169:0/1177338908 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v109) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2c78057c10 con 0x7f2c88069c10 2023-10-26T01:37:20.766 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 64 2023-10-26T01:37:20.769 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.766+0000 7f2c8759e640 1 -- 172.21.15.169:0/1177338908 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2c6806ae40 msgr2=0x7f2c6806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:20.769 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.766+0000 7f2c8759e640 1 --2- 172.21.15.169:0/1177338908 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2c6806ae40 0x7f2c6806d300 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f2c70035000 tx=0x7f2c700046c0 comp rx=0 tx=0).stop 2023-10-26T01:37:20.769 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.766+0000 7f2c8759e640 1 -- 172.21.15.169:0/1177338908 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c88069c10 msgr2=0x7f2c880c6830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:20.770 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.766+0000 7f2c8759e640 1 --2- 172.21.15.169:0/1177338908 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c88069c10 0x7f2c880c6830 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f2c7800b4a0 tx=0x7f2c7800b960 comp rx=0 tx=0).stop 2023-10-26T01:37:20.770 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.766+0000 7f2c8759e640 1 -- 172.21.15.169:0/1177338908 shutdown_connections 2023-10-26T01:37:20.770 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.766+0000 7f2c8759e640 1 --2- 172.21.15.169:0/1177338908 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2c6806ae40 0x7f2c6806d300 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:20.770 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.766+0000 7f2c8759e640 1 --2- 172.21.15.169:0/1177338908 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2c88159470 0x7f2c880ca280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:20.770 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.766+0000 7f2c8759e640 1 --2- 172.21.15.169:0/1177338908 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2c88069ff0 0x7f2c8814d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:20.771 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.766+0000 7f2c8759e640 1 --2- 172.21.15.169:0/1177338908 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c88069c10 0x7f2c880c6830 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:20.771 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.766+0000 7f2c8759e640 1 -- 172.21.15.169:0/1177338908 >> 172.21.15.169:0/1177338908 conn(0x7f2c880af110 msgr2=0x7f2c880ae510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:20.771 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.766+0000 7f2c8759e640 1 -- 172.21.15.169:0/1177338908 shutdown_connections 2023-10-26T01:37:20.771 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.766+0000 7f2c8759e640 1 -- 172.21.15.169:0/1177338908 wait complete. 2023-10-26T01:37:20.786 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-26T01:37:20.977 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.974+0000 7f568d78a640 1 -- 172.21.15.169:0/3651347880 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f56680039a0 msgr2=0x7f5668003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:20.977 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.974+0000 7f568d78a640 1 --2- 172.21.15.169:0/3651347880 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f56680039a0 0x7f5668003da0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f567800c710 tx=0x7f56680057f0 comp rx=0 tx=0).stop 2023-10-26T01:37:20.977 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.974+0000 7f56877fe640 1 -- 172.21.15.169:0/3651347880 reap_dead start 2023-10-26T01:37:20.978 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.974+0000 7f568d78a640 1 -- 172.21.15.169:0/3651347880 shutdown_connections 2023-10-26T01:37:20.978 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.974+0000 7f568d78a640 1 -- 172.21.15.169:0/3651347880 >> 172.21.15.169:0/3651347880 conn(0x7f56880af8a0 msgr2=0x7f56880afc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:20.978 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.974+0000 7f568d78a640 1 -- 172.21.15.169:0/3651347880 shutdown_connections 2023-10-26T01:37:20.978 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.974+0000 7f568d78a640 1 -- 172.21.15.169:0/3651347880 wait complete. 2023-10-26T01:37:20.980 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.976+0000 7f568d78a640 1 Processor -- start 2023-10-26T01:37:20.981 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.976+0000 7f568d78a640 1 -- start start 2023-10-26T01:37:20.981 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.976+0000 7f568d78a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f56680039a0 0x7f56881ef780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:20.981 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.977+0000 7f568d78a640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f56880ace50 0x7f56881591a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:20.981 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.977+0000 7f568d78a640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f56881596e0 0x7f56881549e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:20.982 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.977+0000 7f568d78a640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56881f2b90 con 0x7f56881596e0 2023-10-26T01:37:20.982 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.977+0000 7f568d78a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56881efcc0 con 0x7f56680039a0 2023-10-26T01:37:20.982 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.977+0000 7f568d78a640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f568815b1f0 con 0x7f56880ace50 2023-10-26T01:37:20.982 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.977+0000 7f56877fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f56881596e0 0x7f56881549e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:20.982 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.977+0000 7f56877fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f56881596e0 0x7f56881549e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:32940/0 (socket says 172.21.15.169:32940) 2023-10-26T01:37:20.983 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.977+0000 7f56877fe640 1 -- 172.21.15.169:0/2103399446 learned_addr learned my addr 172.21.15.169:0/2103399446 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:37:20.983 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.977+0000 7f56877fe640 1 -- 172.21.15.169:0/2103399446 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f56880ace50 msgr2=0x7f56881591a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:20.983 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.977+0000 7f56877fe640 1 --2- 172.21.15.169:0/2103399446 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f56880ace50 0x7f56881591a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:20.983 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.977+0000 7f56877fe640 1 -- 172.21.15.169:0/2103399446 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f56680039a0 msgr2=0x7f56881ef780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:20.983 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.977+0000 7f56877fe640 1 --2- 172.21.15.169:0/2103399446 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f56680039a0 0x7f56881ef780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:20.984 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.977+0000 7f56877fe640 1 -- 172.21.15.169:0/2103399446 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5688153940 con 0x7f56881596e0 2023-10-26T01:37:20.984 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.977+0000 7f56877fe640 1 --2- 172.21.15.169:0/2103399446 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f56881596e0 0x7f56881549e0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f56880b0c50 tx=0x7f568000da20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:20.984 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.978+0000 7f567f7fe640 1 -- 172.21.15.169:0/2103399446 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5680018070 con 0x7f56881596e0 2023-10-26T01:37:20.984 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.978+0000 7f567f7fe640 1 -- 172.21.15.169:0/2103399446 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5680005110 con 0x7f56881596e0 2023-10-26T01:37:20.984 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.978+0000 7f567f7fe640 1 -- 172.21.15.169:0/2103399446 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5680005520 con 0x7f56881596e0 2023-10-26T01:37:20.985 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.978+0000 7f568d78a640 1 -- 172.21.15.169:0/2103399446 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5688153c20 con 0x7f56881596e0 2023-10-26T01:37:20.985 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.978+0000 7f568d78a640 1 -- 172.21.15.169:0/2103399446 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56881541e0 con 0x7f56881596e0 2023-10-26T01:37:20.986 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.979+0000 7f568d78a640 1 -- 172.21.15.169:0/2103399446 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f568804db10 con 0x7f56881596e0 2023-10-26T01:37:20.986 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.980+0000 7f567f7fe640 1 -- 172.21.15.169:0/2103399446 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f56800047a0 con 0x7f56881596e0 2023-10-26T01:37:20.986 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.981+0000 7f567f7fe640 1 --2- 172.21.15.169:0/2103399446 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f566c06ae70 0x7f566c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:20.986 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.981+0000 7f567f7fe640 1 -- 172.21.15.169:0/2103399446 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5680088fd0 con 0x7f56881596e0 2023-10-26T01:37:20.987 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.982+0000 7f5686ffd640 1 --2- 172.21.15.169:0/2103399446 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f566c06ae70 0x7f566c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:20.987 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.983+0000 7f5686ffd640 1 --2- 172.21.15.169:0/2103399446 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f566c06ae70 0x7f566c06d330 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f5678034000 tx=0x7f56780036f0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:20.988 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:20.986+0000 7f567f7fe640 1 -- 172.21.15.169:0/2103399446 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5680054080 con 0x7f56881596e0 2023-10-26T01:37:21.178 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:21.175+0000 7f568d78a640 1 -- 172.21.15.169:0/2103399446 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f56880add40 con 0x7f56881596e0 2023-10-26T01:37:21.179 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:21.177+0000 7f567f7fe640 1 -- 172.21.15.169:0/2103399446 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v109) v1 ==== 80+0+0 (secure 0 0 0) 0x7f56800576a0 con 0x7f56881596e0 2023-10-26T01:37:21.179 INFO:teuthology.orchestra.run.smithi169.stderr:listed 0 entries 2023-10-26T01:37:21.181 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:21.178+0000 7f568d78a640 1 -- 172.21.15.169:0/2103399446 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f566c06ae70 msgr2=0x7f566c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:21.182 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:21.178+0000 7f568d78a640 1 --2- 172.21.15.169:0/2103399446 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f566c06ae70 0x7f566c06d330 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f5678034000 tx=0x7f56780036f0 comp rx=0 tx=0).stop 2023-10-26T01:37:21.182 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:21.178+0000 7f568d78a640 1 -- 172.21.15.169:0/2103399446 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f56881596e0 msgr2=0x7f56881549e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:21.182 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:21.178+0000 7f568d78a640 1 --2- 172.21.15.169:0/2103399446 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f56881596e0 0x7f56881549e0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f56880b0c50 tx=0x7f568000da20 comp rx=0 tx=0).stop 2023-10-26T01:37:21.182 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:21.180+0000 7f568d78a640 1 -- 172.21.15.169:0/2103399446 shutdown_connections 2023-10-26T01:37:21.182 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:21.180+0000 7f568d78a640 1 --2- 172.21.15.169:0/2103399446 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f566c06ae70 0x7f566c06d330 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:21.183 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:21.180+0000 7f568d78a640 1 --2- 172.21.15.169:0/2103399446 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f56881596e0 0x7f56881549e0 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:21.183 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:21.180+0000 7f568d78a640 1 --2- 172.21.15.169:0/2103399446 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f56880ace50 0x7f56881591a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:21.183 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:21.180+0000 7f568d78a640 1 --2- 172.21.15.169:0/2103399446 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f56680039a0 0x7f56881ef780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:21.183 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:21.180+0000 7f568d78a640 1 -- 172.21.15.169:0/2103399446 >> 172.21.15.169:0/2103399446 conn(0x7f56880af8a0 msgr2=0x7f568806c540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:21.183 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:21.180+0000 7f568d78a640 1 -- 172.21.15.169:0/2103399446 shutdown_connections 2023-10-26T01:37:21.184 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:21.180+0000 7f568d78a640 1 -- 172.21.15.169:0/2103399446 wait complete. 2023-10-26T01:37:21.200 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-26T01:37:21.200 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:37:21.201 DEBUG:teuthology.orchestra.run.smithi170:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-26T01:37:21.314 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi170.front.sepia.ceph.com 2023-10-26T01:37:21.315 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:37:21.315 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-26T01:37:21.366 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:37:21.367 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:37:21.427 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054301,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-26T01:37:21.427 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-26T01:37:21.428 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:37:21.485 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054303,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-26T01:37:21.485 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T01:37:21.711 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:21.709+0000 7fc5febf5640 1 -- 172.21.15.169:0/3781790322 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc5e80039c0 msgr2=0x7fc5e8003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:21.711 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:21.709+0000 7fc5febf5640 1 --2- 172.21.15.169:0/3781790322 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc5e80039c0 0x7fc5e8003de0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec00bb20 tx=0x7fc5e80057f0 comp rx=0 tx=0).stop 2023-10-26T01:37:21.712 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:21.709+0000 7fc5febf5640 1 -- 172.21.15.169:0/3781790322 shutdown_connections 2023-10-26T01:37:21.712 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:21.709+0000 7fc5febf5640 1 --2- 172.21.15.169:0/3781790322 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc5e80039c0 0x7fc5e8003de0 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:21.712 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:21.709+0000 7fc5febf5640 1 -- 172.21.15.169:0/3781790322 >> 172.21.15.169:0/3781790322 conn(0x7fc5f8069930 msgr2=0x7fc5f8069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:21.713 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:21.709+0000 7fc5febf5640 1 -- 172.21.15.169:0/3781790322 shutdown_connections 2023-10-26T01:37:21.713 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:21.709+0000 7fc5febf5640 1 -- 172.21.15.169:0/3781790322 wait complete. 2023-10-26T01:37:21.715 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:21.711+0000 7fc5febf5640 1 Processor -- start 2023-10-26T01:37:21.715 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:21.711+0000 7fc5febf5640 1 -- start start 2023-10-26T01:37:21.716 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:21.712+0000 7fc5febf5640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc5e80039c0 0x7fc5f814f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:21.716 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:21.712+0000 7fc5febf5640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc5f80714d0 0x7fc5f8155380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:21.716 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:21.712+0000 7fc5febf5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc5f8071930 0x7fc5f81558c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:21.717 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:21.712+0000 7fc5febf5640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f814fd50 con 0x7fc5e80039c0 2023-10-26T01:37:21.717 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:21.712+0000 7fc5febf5640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f8159c20 con 0x7fc5f8071930 2023-10-26T01:37:21.718 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:21.712+0000 7fc5febf5640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f8159f00 con 0x7fc5f80714d0 2023-10-26T01:37:21.718 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:21.712+0000 7fc5fc991640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc5e80039c0 0x7fc5f814f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:21.719 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:21.712+0000 7fc5fc991640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc5e80039c0 0x7fc5f814f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:32958/0 (socket says 172.21.15.169:32958) 2023-10-26T01:37:21.719 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:21.712+0000 7fc5fc991640 1 -- 172.21.15.169:0/371117898 learned_addr learned my addr 172.21.15.169:0/371117898 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:37:21.719 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:21.712+0000 7fc5fc991640 1 -- 172.21.15.169:0/371117898 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc5f80714d0 msgr2=0x7fc5f8155380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:21.720 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:21.712+0000 7fc5fc991640 1 --2- 172.21.15.169:0/371117898 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc5f80714d0 0x7fc5f8155380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:21.720 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:21.712+0000 7fc5fc991640 1 -- 172.21.15.169:0/371117898 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc5f8071930 msgr2=0x7fc5f81558c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:21.720 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:21.712+0000 7fc5fc991640 1 --2- 172.21.15.169:0/371117898 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc5f8071930 0x7fc5f81558c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:21.721 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:21.712+0000 7fc5fc991640 1 -- 172.21.15.169:0/371117898 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5f815a1e0 con 0x7fc5e80039c0 2023-10-26T01:37:21.721 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:21.713+0000 7fc5fc991640 1 --2- 172.21.15.169:0/371117898 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc5e80039c0 0x7fc5f814f810 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec00ba40 tx=0x7fc5ec0366b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:21.721 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:21.713+0000 7fc5f57fa640 1 -- 172.21.15.169:0/371117898 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5ec036d70 con 0x7fc5e80039c0 2023-10-26T01:37:21.722 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:21.713+0000 7fc5f57fa640 1 -- 172.21.15.169:0/371117898 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc5ec004020 con 0x7fc5e80039c0 2023-10-26T01:37:21.722 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:21.713+0000 7fc5f57fa640 1 -- 172.21.15.169:0/371117898 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5ec004430 con 0x7fc5e80039c0 2023-10-26T01:37:21.723 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:21.713+0000 7fc5febf5640 1 -- 172.21.15.169:0/371117898 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5f815a4c0 con 0x7fc5e80039c0 2023-10-26T01:37:21.723 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:21.713+0000 7fc5febf5640 1 -- 172.21.15.169:0/371117898 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5f806ee80 con 0x7fc5e80039c0 2023-10-26T01:37:21.725 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:21.714+0000 7fc5febf5640 1 -- 172.21.15.169:0/371117898 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5f80c9f90 con 0x7fc5e80039c0 2023-10-26T01:37:21.725 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:21.715+0000 7fc5f57fa640 1 -- 172.21.15.169:0/371117898 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fc5ec0035f0 con 0x7fc5e80039c0 2023-10-26T01:37:21.725 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:21.716+0000 7fc5f57fa640 1 --2- 172.21.15.169:0/371117898 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc5dc06ae70 0x7fc5dc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:21.726 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:21.716+0000 7fc5f57fa640 1 -- 172.21.15.169:0/371117898 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc5ec0ab510 con 0x7fc5e80039c0 2023-10-26T01:37:21.726 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:21.717+0000 7fc5f7fff640 1 --2- 172.21.15.169:0/371117898 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc5dc06ae70 0x7fc5dc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:21.726 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:21.718+0000 7fc5f7fff640 1 --2- 172.21.15.169:0/371117898 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc5dc06ae70 0x7fc5dc06d330 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fc5f81f34e0 tx=0x7fc5e40079d0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:21.726 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:21.721+0000 7fc5f57fa640 1 -- 172.21.15.169:0/371117898 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc5ec076520 con 0x7fc5e80039c0 2023-10-26T01:37:21.951 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:21.949+0000 7fc5febf5640 1 -- 172.21.15.169:0/371117898 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc5f8158480 con 0x7fc5e80039c0 2023-10-26T01:37:21.952 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:21.950+0000 7fc5f57fa640 1 -- 172.21.15.169:0/371117898 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+4924 (secure 0 0 0) 0x7fc5ec079b40 con 0x7fc5e80039c0 2023-10-26T01:37:21.952 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:37:21.953 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":50,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5240,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/3870332417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":3870332417},{"type":"v1","addr":"172.21.15.170:6837","nonce":3870332417}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45},{"gid":5497,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/2533771698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2533771698},{"type":"v1","addr":"172.21.15.170:6835","nonce":2533771698}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45},{"gid":6885,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/2117946713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2117946713},{"type":"v1","addr":"172.21.15.169:6834","nonce":2117946713}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T01:36:43.849046+0000","modified":"2023-10-26T01:36:46.926339+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6888},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6888":{"gid":6888,"name":"c","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6835/657142078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":657142078},{"type":"v1","addr":"172.21.15.169:6835","nonce":657142078}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-10-26T01:37:21.953 INFO:teuthology.orchestra.run.smithi169.stderr:dumped fsmap epoch 50 2023-10-26T01:37:21.954 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:21.952+0000 7fc5febf5640 1 -- 172.21.15.169:0/371117898 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc5dc06ae70 msgr2=0x7fc5dc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:21.955 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:21.952+0000 7fc5febf5640 1 --2- 172.21.15.169:0/371117898 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc5dc06ae70 0x7fc5dc06d330 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fc5f81f34e0 tx=0x7fc5e40079d0 comp rx=0 tx=0).stop 2023-10-26T01:37:21.955 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:21.952+0000 7fc5febf5640 1 -- 172.21.15.169:0/371117898 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc5e80039c0 msgr2=0x7fc5f814f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:21.955 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:21.952+0000 7fc5febf5640 1 --2- 172.21.15.169:0/371117898 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc5e80039c0 0x7fc5f814f810 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec00ba40 tx=0x7fc5ec0366b0 comp rx=0 tx=0).stop 2023-10-26T01:37:21.955 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:21.953+0000 7fc5febf5640 1 -- 172.21.15.169:0/371117898 shutdown_connections 2023-10-26T01:37:21.956 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:21.953+0000 7fc5febf5640 1 --2- 172.21.15.169:0/371117898 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc5dc06ae70 0x7fc5dc06d330 secure :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fc5f81f34e0 tx=0x7fc5e40079d0 comp rx=0 tx=0).stop 2023-10-26T01:37:21.956 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:21.953+0000 7fc5febf5640 1 --2- 172.21.15.169:0/371117898 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc5f8071930 0x7fc5f81558c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:21.956 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:21.953+0000 7fc5febf5640 1 --2- 172.21.15.169:0/371117898 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc5f80714d0 0x7fc5f8155380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:21.956 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:21.953+0000 7fc5febf5640 1 --2- 172.21.15.169:0/371117898 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc5e80039c0 0x7fc5f814f810 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:21.956 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:21.953+0000 7fc5febf5640 1 -- 172.21.15.169:0/371117898 >> 172.21.15.169:0/371117898 conn(0x7fc5f8069930 msgr2=0x7fc5f8153a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:21.957 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:21.953+0000 7fc5febf5640 1 -- 172.21.15.169:0/371117898 shutdown_connections 2023-10-26T01:37:21.957 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:21.953+0000 7fc5febf5640 1 -- 172.21.15.169:0/371117898 wait complete. 2023-10-26T01:37:21.975 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:37:22.195 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:22.191+0000 7f6a19e00640 1 -- 172.21.15.169:0/1481641303 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6a14158850 msgr2=0x7f6a140cd720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:22.195 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:22.191+0000 7f6a19e00640 1 --2- 172.21.15.169:0/1481641303 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6a14158850 0x7f6a140cd720 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f69fc005fd0 tx=0x7f69fc02d0d0 comp rx=0 tx=0).stop 2023-10-26T01:37:22.196 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:22.191+0000 7f6a19e00640 1 -- 172.21.15.169:0/1481641303 shutdown_connections 2023-10-26T01:37:22.196 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:22.191+0000 7f6a19e00640 1 --2- 172.21.15.169:0/1481641303 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6a14158850 0x7f6a140cd720 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:22.196 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:22.191+0000 7f6a19e00640 1 --2- 172.21.15.169:0/1481641303 >> v2:172.21.15.170:3300/0 conn(0x7f6a1414e790 0x7f6a140aab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:22.196 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:22.191+0000 7f6a19e00640 1 -- 172.21.15.169:0/1481641303 >> 172.21.15.169:0/1481641303 conn(0x7f6a14069930 msgr2=0x7f6a14069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:22.197 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:22.192+0000 7f6a19e00640 1 -- 172.21.15.169:0/1481641303 shutdown_connections 2023-10-26T01:37:22.197 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:22.192+0000 7f6a19e00640 1 -- 172.21.15.169:0/1481641303 wait complete. 2023-10-26T01:37:22.197 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:22.193+0000 7f6a19e00640 1 Processor -- start 2023-10-26T01:37:22.197 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:22.193+0000 7f6a19e00640 1 -- start start 2023-10-26T01:37:22.199 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:22.194+0000 7f6a19e00640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a140cdc60 0x7f6a140b1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:22.199 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:22.194+0000 7f6a19e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6a1414e790 0x7f6a141f4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:22.200 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:22.194+0000 7f6a19e00640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6a14158850 0x7f6a14159d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:22.200 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:22.194+0000 7f6a19e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a14153ca0 con 0x7f6a1414e790 2023-10-26T01:37:22.200 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:22.194+0000 7f6a19e00640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a140b1d50 con 0x7f6a140cdc60 2023-10-26T01:37:22.200 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:22.194+0000 7f6a19e00640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a14154860 con 0x7f6a14158850 2023-10-26T01:37:22.201 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:22.195+0000 7f6a0affd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6a1414e790 0x7f6a141f4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:22.201 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:22.195+0000 7f6a0affd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6a1414e790 0x7f6a141f4840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:32972/0 (socket says 172.21.15.169:32972) 2023-10-26T01:37:22.201 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:22.195+0000 7f6a0affd640 1 -- 172.21.15.169:0/3333558930 learned_addr learned my addr 172.21.15.169:0/3333558930 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:37:22.201 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:22.195+0000 7f6a137fe640 1 --2- 172.21.15.169:0/3333558930 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a140cdc60 0x7f6a140b1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:22.201 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:22.195+0000 7f6a13fff640 1 --2- 172.21.15.169:0/3333558930 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6a14158850 0x7f6a14159d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:22.201 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:22.195+0000 7f6a0affd640 1 -- 172.21.15.169:0/3333558930 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6a14158850 msgr2=0x7f6a14159d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:22.202 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:22.195+0000 7f6a0affd640 1 --2- 172.21.15.169:0/3333558930 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6a14158850 0x7f6a14159d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:22.202 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:22.195+0000 7f6a0affd640 1 -- 172.21.15.169:0/3333558930 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a140cdc60 msgr2=0x7f6a140b1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:22.202 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:22.195+0000 7f6a0affd640 1 --2- 172.21.15.169:0/3333558930 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a140cdc60 0x7f6a140b1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:22.202 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:22.195+0000 7f6a0affd640 1 -- 172.21.15.169:0/3333558930 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69fc00a670 con 0x7f6a1414e790 2023-10-26T01:37:22.203 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:22.195+0000 7f6a13fff640 1 --2- 172.21.15.169:0/3333558930 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6a14158850 0x7f6a14159d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:37:22.203 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:22.196+0000 7f6a0affd640 1 --2- 172.21.15.169:0/3333558930 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6a1414e790 0x7f6a141f4840 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f69fc037040 tx=0x7f69fc0047a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:22.203 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:22.196+0000 7f6a10ff9640 1 -- 172.21.15.169:0/3333558930 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69fc004da0 con 0x7f6a1414e790 2023-10-26T01:37:22.203 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:22.196+0000 7f6a10ff9640 1 -- 172.21.15.169:0/3333558930 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f69fc0035f0 con 0x7f6a1414e790 2023-10-26T01:37:22.204 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:22.196+0000 7f6a19e00640 1 -- 172.21.15.169:0/3333558930 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a14154b40 con 0x7f6a1414e790 2023-10-26T01:37:22.204 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:22.196+0000 7f6a10ff9640 1 -- 172.21.15.169:0/3333558930 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69fc003a00 con 0x7f6a1414e790 2023-10-26T01:37:22.204 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:22.196+0000 7f6a19e00640 1 -- 172.21.15.169:0/3333558930 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a14155100 con 0x7f6a1414e790 2023-10-26T01:37:22.204 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:22.197+0000 7f6a19e00640 1 -- 172.21.15.169:0/3333558930 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a1404db10 con 0x7f6a1414e790 2023-10-26T01:37:22.204 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:22.198+0000 7f6a10ff9640 1 -- 172.21.15.169:0/3333558930 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f69fc003b60 con 0x7f6a1414e790 2023-10-26T01:37:22.205 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:22.199+0000 7f6a10ff9640 1 --2- 172.21.15.169:0/3333558930 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f69f406ae40 0x7f69f406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:22.205 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:22.199+0000 7f6a10ff9640 1 -- 172.21.15.169:0/3333558930 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f69fc0ac820 con 0x7f6a1414e790 2023-10-26T01:37:22.205 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:22.200+0000 7f6a137fe640 1 --2- 172.21.15.169:0/3333558930 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f69f406ae40 0x7f69f406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:22.206 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:22.201+0000 7f6a137fe640 1 --2- 172.21.15.169:0/3333558930 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f69f406ae40 0x7f69f406d300 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f6a04009b60 tx=0x7f6a04005f90 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:22.206 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:22.203+0000 7f6a10ff9640 1 -- 172.21.15.169:0/3333558930 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f69fc0778c0 con 0x7f6a1414e790 2023-10-26T01:37:22.397 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:22.395+0000 7f6a19e00640 1 -- 172.21.15.169:0/3333558930 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6a140aca70 con 0x7f6a1414e790 2023-10-26T01:37:22.398 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:22.397+0000 7f6a10ff9640 1 -- 172.21.15.169:0/3333558930 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v109) v1 ==== 74+0+16620 (secure 0 0 0) 0x7f69fc07aee0 con 0x7f6a1414e790 2023-10-26T01:37:22.399 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:37:22.399 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":109,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:37:13.181064+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-10-26T01:36:41.430455+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-10-26T01:36:42.883252+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:37:22.402 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:22.400+0000 7f6a19e00640 1 -- 172.21.15.169:0/3333558930 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f69f406ae40 msgr2=0x7f69f406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:22.402 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:22.400+0000 7f6a19e00640 1 --2- 172.21.15.169:0/3333558930 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f69f406ae40 0x7f69f406d300 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f6a04009b60 tx=0x7f6a04005f90 comp rx=0 tx=0).stop 2023-10-26T01:37:22.403 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:22.400+0000 7f6a19e00640 1 -- 172.21.15.169:0/3333558930 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6a1414e790 msgr2=0x7f6a141f4840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:22.403 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:22.400+0000 7f6a19e00640 1 --2- 172.21.15.169:0/3333558930 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6a1414e790 0x7f6a141f4840 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f69fc037040 tx=0x7f69fc0047a0 comp rx=0 tx=0).stop 2023-10-26T01:37:22.403 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:22.400+0000 7f6a19e00640 1 -- 172.21.15.169:0/3333558930 shutdown_connections 2023-10-26T01:37:22.403 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:22.400+0000 7f6a19e00640 1 --2- 172.21.15.169:0/3333558930 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f69f406ae40 0x7f69f406d300 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:22.403 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:22.400+0000 7f6a19e00640 1 --2- 172.21.15.169:0/3333558930 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6a1414e790 0x7f6a141f4840 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:22.404 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:22.400+0000 7f6a19e00640 1 --2- 172.21.15.169:0/3333558930 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6a14158850 0x7f6a14159d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:22.404 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:22.400+0000 7f6a19e00640 1 --2- 172.21.15.169:0/3333558930 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a140cdc60 0x7f6a140b1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:22.404 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:22.400+0000 7f6a19e00640 1 -- 172.21.15.169:0/3333558930 >> 172.21.15.169:0/3333558930 conn(0x7f6a14069930 msgr2=0x7f6a14151680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:22.404 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:22.400+0000 7f6a19e00640 1 -- 172.21.15.169:0/3333558930 shutdown_connections 2023-10-26T01:37:22.405 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:22.400+0000 7f6a19e00640 1 -- 172.21.15.169:0/3333558930 wait complete. 2023-10-26T01:37:22.420 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T01:30:32.200808+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '109', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-26T01:36:41.430455+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '109', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2023-10-26T01:36:42.883252+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '109', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T01:37:22.421 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T01:37:22.647 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:22.645+0000 7fd4da810640 1 -- 172.21.15.169:0/3199924461 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4d40cdda0 msgr2=0x7fd4d40ab3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:22.647 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:22.645+0000 7fd4da810640 1 --2- 172.21.15.169:0/3199924461 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4d40cdda0 0x7fd4d40ab3c0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc005fd0 tx=0x7fd4cc02c2f0 comp rx=0 tx=0).stop 2023-10-26T01:37:22.647 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:22.645+0000 7fd4da810640 1 -- 172.21.15.169:0/3199924461 shutdown_connections 2023-10-26T01:37:22.647 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:22.645+0000 7fd4da810640 1 --2- 172.21.15.169:0/3199924461 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4d40cdda0 0x7fd4d40ab3c0 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:22.648 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:22.645+0000 7fd4da810640 1 -- 172.21.15.169:0/3199924461 >> 172.21.15.169:0/3199924461 conn(0x7fd4d4069930 msgr2=0x7fd4d4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:22.648 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:22.645+0000 7fd4da810640 1 -- 172.21.15.169:0/3199924461 shutdown_connections 2023-10-26T01:37:22.648 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:22.645+0000 7fd4da810640 1 -- 172.21.15.169:0/3199924461 wait complete. 2023-10-26T01:37:22.650 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:22.646+0000 7fd4da810640 1 Processor -- start 2023-10-26T01:37:22.650 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:22.647+0000 7fd4da810640 1 -- start start 2023-10-26T01:37:22.651 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:22.648+0000 7fd4da810640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4d40aaf80 0x7fd4d40c6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:22.651 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:22.648+0000 7fd4da810640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd4d40cdda0 0x7fd4d40c7520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:22.651 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:22.648+0000 7fd4da810640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd4d4159480 0x7fd4d406ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:22.651 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:22.648+0000 7fd4da810640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d406c670 con 0x7fd4d40aaf80 2023-10-26T01:37:22.651 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:22.648+0000 7fd4da810640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d406ff60 con 0x7fd4d40cdda0 2023-10-26T01:37:22.652 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:22.648+0000 7fd4da810640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d406bdd0 con 0x7fd4d4159480 2023-10-26T01:37:22.652 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:22.648+0000 7fd4d3fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4d40aaf80 0x7fd4d40c6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:22.652 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:22.648+0000 7fd4d3fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4d40aaf80 0x7fd4d40c6fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:32996/0 (socket says 172.21.15.169:32996) 2023-10-26T01:37:22.652 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:22.648+0000 7fd4d3fff640 1 -- 172.21.15.169:0/3481903755 learned_addr learned my addr 172.21.15.169:0/3481903755 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:37:22.652 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:22.648+0000 7fd4d8dad640 1 --2- 172.21.15.169:0/3481903755 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd4d4159480 0x7fd4d406ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:22.653 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:22.648+0000 7fd4d3fff640 1 -- 172.21.15.169:0/3481903755 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd4d4159480 msgr2=0x7fd4d406ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:22.653 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:22.648+0000 7fd4d3fff640 1 --2- 172.21.15.169:0/3481903755 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd4d4159480 0x7fd4d406ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:22.653 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:22.648+0000 7fd4d3fff640 1 -- 172.21.15.169:0/3481903755 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd4d40cdda0 msgr2=0x7fd4d40c7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:22.653 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:22.648+0000 7fd4cbfff640 1 --2- 172.21.15.169:0/3481903755 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd4d40cdda0 0x7fd4d40c7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:22.653 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:22.648+0000 7fd4d3fff640 1 --2- 172.21.15.169:0/3481903755 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd4d40cdda0 0x7fd4d40c7520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:22.654 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:22.648+0000 7fd4d3fff640 1 -- 172.21.15.169:0/3481903755 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4cc009900 con 0x7fd4d40aaf80 2023-10-26T01:37:22.654 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:22.648+0000 7fd4d8dad640 1 --2- 172.21.15.169:0/3481903755 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd4d4159480 0x7fd4d406ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:37:22.654 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:22.648+0000 7fd4cbfff640 1 --2- 172.21.15.169:0/3481903755 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd4d40cdda0 0x7fd4d40c7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:37:22.654 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:22.649+0000 7fd4d3fff640 1 --2- 172.21.15.169:0/3481903755 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4d40aaf80 0x7fd4d40c6fe0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fd4c400b3f0 tx=0x7fd4c400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:22.655 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:22.649+0000 7fd4d17fa640 1 -- 172.21.15.169:0/3481903755 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c4011830 con 0x7fd4d40aaf80 2023-10-26T01:37:22.656 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:22.649+0000 7fd4d17fa640 1 -- 172.21.15.169:0/3481903755 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd4c4011a80 con 0x7fd4d40aaf80 2023-10-26T01:37:22.656 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:22.649+0000 7fd4d17fa640 1 -- 172.21.15.169:0/3481903755 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c4011e90 con 0x7fd4d40aaf80 2023-10-26T01:37:22.656 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:22.649+0000 7fd4da810640 1 -- 172.21.15.169:0/3481903755 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4d406c0b0 con 0x7fd4d40aaf80 2023-10-26T01:37:22.656 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:22.649+0000 7fd4da810640 1 -- 172.21.15.169:0/3481903755 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4d40c68d0 con 0x7fd4d40aaf80 2023-10-26T01:37:22.656 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:22.651+0000 7fd4da810640 1 -- 172.21.15.169:0/3481903755 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4d4066af0 con 0x7fd4d40aaf80 2023-10-26T01:37:22.657 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:22.651+0000 7fd4d17fa640 1 -- 172.21.15.169:0/3481903755 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fd4c401a400 con 0x7fd4d40aaf80 2023-10-26T01:37:22.657 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:22.652+0000 7fd4d17fa640 1 --2- 172.21.15.169:0/3481903755 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd4b406ae40 0x7fd4b406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:22.657 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:22.652+0000 7fd4d17fa640 1 -- 172.21.15.169:0/3481903755 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd4c4089690 con 0x7fd4d40aaf80 2023-10-26T01:37:22.657 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:22.653+0000 7fd4cbfff640 1 --2- 172.21.15.169:0/3481903755 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd4b406ae40 0x7fd4b406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:22.657 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:22.654+0000 7fd4cbfff640 1 --2- 172.21.15.169:0/3481903755 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd4b406ae40 0x7fd4b406d300 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fd4d414b720 tx=0x7fd4bc009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:22.658 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:22.657+0000 7fd4d17fa640 1 -- 172.21.15.169:0/3481903755 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd4c40547d0 con 0x7fd4d40aaf80 2023-10-26T01:37:22.831 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:22.829+0000 7fd4da810640 1 -- 172.21.15.169:0/3481903755 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd4d404da70 con 0x7fd4d40aaf80 2023-10-26T01:37:22.832 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:22.830+0000 7fd4d17fa640 1 -- 172.21.15.169:0/3481903755 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v109) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd4c4057df0 con 0x7fd4d40aaf80 2023-10-26T01:37:22.832 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 1 2023-10-26T01:37:22.835 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:22.832+0000 7fd4da810640 1 -- 172.21.15.169:0/3481903755 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd4b406ae40 msgr2=0x7fd4b406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:22.835 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:22.832+0000 7fd4da810640 1 --2- 172.21.15.169:0/3481903755 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd4b406ae40 0x7fd4b406d300 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fd4d414b720 tx=0x7fd4bc009040 comp rx=0 tx=0).stop 2023-10-26T01:37:22.835 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:22.832+0000 7fd4da810640 1 -- 172.21.15.169:0/3481903755 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4d40aaf80 msgr2=0x7fd4d40c6fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:22.835 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:22.832+0000 7fd4da810640 1 --2- 172.21.15.169:0/3481903755 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4d40aaf80 0x7fd4d40c6fe0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fd4c400b3f0 tx=0x7fd4c400b8b0 comp rx=0 tx=0).stop 2023-10-26T01:37:22.835 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:22.833+0000 7fd4da810640 1 -- 172.21.15.169:0/3481903755 shutdown_connections 2023-10-26T01:37:22.836 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:22.833+0000 7fd4da810640 1 --2- 172.21.15.169:0/3481903755 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd4b406ae40 0x7fd4b406d300 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:22.836 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:22.833+0000 7fd4da810640 1 --2- 172.21.15.169:0/3481903755 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd4d4159480 0x7fd4d406ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:22.836 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:22.833+0000 7fd4da810640 1 --2- 172.21.15.169:0/3481903755 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd4d40cdda0 0x7fd4d40c7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:22.836 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:22.833+0000 7fd4da810640 1 --2- 172.21.15.169:0/3481903755 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4d40aaf80 0x7fd4d40c6fe0 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:22.836 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:22.833+0000 7fd4da810640 1 -- 172.21.15.169:0/3481903755 >> 172.21.15.169:0/3481903755 conn(0x7fd4d4069930 msgr2=0x7fd4d40cd660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:22.837 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:22.833+0000 7fd4da810640 1 -- 172.21.15.169:0/3481903755 shutdown_connections 2023-10-26T01:37:22.837 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:22.833+0000 7fd4da810640 1 -- 172.21.15.169:0/3481903755 wait complete. 2023-10-26T01:37:22.852 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-26T01:37:23.072 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.070+0000 7f0c0c36c640 1 -- 172.21.15.169:0/1246975388 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0bf0003970 msgr2=0x7f0bf0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:23.072 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.070+0000 7f0c0c36c640 1 --2- 172.21.15.169:0/1246975388 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0bf0003970 0x7f0bf0003d90 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f0c00001eb0 tx=0x7f0bf00057f0 comp rx=0 tx=0).stop 2023-10-26T01:37:23.072 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.070+0000 7f0c0a909640 1 -- 172.21.15.169:0/1246975388 reap_dead start 2023-10-26T01:37:23.073 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.070+0000 7f0c0c36c640 1 -- 172.21.15.169:0/1246975388 shutdown_connections 2023-10-26T01:37:23.073 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.070+0000 7f0c0c36c640 1 -- 172.21.15.169:0/1246975388 >> 172.21.15.169:0/1246975388 conn(0x7f0c04069930 msgr2=0x7f0c04069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:23.073 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.070+0000 7f0c0c36c640 1 -- 172.21.15.169:0/1246975388 shutdown_connections 2023-10-26T01:37:23.073 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.070+0000 7f0c0c36c640 1 -- 172.21.15.169:0/1246975388 wait complete. 2023-10-26T01:37:23.074 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.072+0000 7f0c0c36c640 1 Processor -- start 2023-10-26T01:37:23.074 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.072+0000 7f0c0c36c640 1 -- start start 2023-10-26T01:37:23.076 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.073+0000 7f0c0c36c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0bf0003970 0x7f0c041f2760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:23.076 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.073+0000 7f0c0c36c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c0413e060 0x7f0c040afcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:23.076 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.073+0000 7f0c0c36c640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0c0406ad90 0x7f0c0406b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:23.077 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.073+0000 7f0c0c36c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c040af500 con 0x7f0c0413e060 2023-10-26T01:37:23.077 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.073+0000 7f0c0c36c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c0406b750 con 0x7f0bf0003970 2023-10-26T01:37:23.077 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.073+0000 7f0c0c36c640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c040cac90 con 0x7f0c0406ad90 2023-10-26T01:37:23.077 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.073+0000 7f0c09907640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c0413e060 0x7f0c040afcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:23.077 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.073+0000 7f0c09907640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c0413e060 0x7f0c040afcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:33022/0 (socket says 172.21.15.169:33022) 2023-10-26T01:37:23.078 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.073+0000 7f0c09907640 1 -- 172.21.15.169:0/2712193144 learned_addr learned my addr 172.21.15.169:0/2712193144 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:37:23.078 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.073+0000 7f0c0a909640 1 --2- 172.21.15.169:0/2712193144 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0c0406ad90 0x7f0c0406b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:23.078 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.074+0000 7f0c09907640 1 -- 172.21.15.169:0/2712193144 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0c0406ad90 msgr2=0x7f0c0406b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:23.078 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.074+0000 7f0c09907640 1 --2- 172.21.15.169:0/2712193144 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0c0406ad90 0x7f0c0406b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:23.078 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.074+0000 7f0c09907640 1 -- 172.21.15.169:0/2712193144 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0bf0003970 msgr2=0x7f0c041f2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:23.079 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.074+0000 7f0c0a108640 1 --2- 172.21.15.169:0/2712193144 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0bf0003970 0x7f0c041f2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:23.079 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.074+0000 7f0c09907640 1 --2- 172.21.15.169:0/2712193144 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0bf0003970 0x7f0c041f2760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:23.079 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.074+0000 7f0c09907640 1 -- 172.21.15.169:0/2712193144 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c0413e4c0 con 0x7f0c0413e060 2023-10-26T01:37:23.079 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.074+0000 7f0c0a909640 1 --2- 172.21.15.169:0/2712193144 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0c0406ad90 0x7f0c0406b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:37:23.080 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.074+0000 7f0c09907640 1 --2- 172.21.15.169:0/2712193144 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c0413e060 0x7f0c040afcd0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f0bf400d770 tx=0x7f0bf400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:23.080 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.074+0000 7f0bfaffd640 1 -- 172.21.15.169:0/2712193144 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf4009c30 con 0x7f0c0413e060 2023-10-26T01:37:23.080 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.074+0000 7f0bfaffd640 1 -- 172.21.15.169:0/2712193144 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0bf400de80 con 0x7f0c0413e060 2023-10-26T01:37:23.080 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.074+0000 7f0bfaffd640 1 -- 172.21.15.169:0/2712193144 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf40106c0 con 0x7f0c0413e060 2023-10-26T01:37:23.080 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.074+0000 7f0c0c36c640 1 -- 172.21.15.169:0/2712193144 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c040adfe0 con 0x7f0c0413e060 2023-10-26T01:37:23.081 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.074+0000 7f0c0c36c640 1 -- 172.21.15.169:0/2712193144 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c040ae5a0 con 0x7f0c0413e060 2023-10-26T01:37:23.082 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.075+0000 7f0c0c36c640 1 -- 172.21.15.169:0/2712193144 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c040caa80 con 0x7f0c0413e060 2023-10-26T01:37:23.082 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.076+0000 7f0bfaffd640 1 -- 172.21.15.169:0/2712193144 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f0bf4010820 con 0x7f0c0413e060 2023-10-26T01:37:23.082 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.077+0000 7f0bfaffd640 1 --2- 172.21.15.169:0/2712193144 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f0be806ae70 0x7f0be806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:23.082 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.077+0000 7f0bfaffd640 1 -- 172.21.15.169:0/2712193144 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0bf4089bd0 con 0x7f0c0413e060 2023-10-26T01:37:23.082 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.078+0000 7f0c0a108640 1 --2- 172.21.15.169:0/2712193144 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f0be806ae70 0x7f0be806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:23.083 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.079+0000 7f0c0a108640 1 --2- 172.21.15.169:0/2712193144 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f0be806ae70 0x7f0be806d330 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f0c00001eb0 tx=0x7f0c00039040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:23.083 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.082+0000 7f0bfaffd640 1 -- 172.21.15.169:0/2712193144 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0bf4054b50 con 0x7f0c0413e060 2023-10-26T01:37:23.258 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.257+0000 7f0c0c36c640 1 -- 172.21.15.169:0/2712193144 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0c0404db10 con 0x7f0c0413e060 2023-10-26T01:37:23.259 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.257+0000 7f0bfaffd640 1 -- 172.21.15.169:0/2712193144 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v109) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0bf4021020 con 0x7f0c0413e060 2023-10-26T01:37:23.260 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 32 2023-10-26T01:37:23.262 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.260+0000 7f0c0c36c640 1 -- 172.21.15.169:0/2712193144 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f0be806ae70 msgr2=0x7f0be806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:23.262 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.260+0000 7f0c0c36c640 1 --2- 172.21.15.169:0/2712193144 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f0be806ae70 0x7f0be806d330 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f0c00001eb0 tx=0x7f0c00039040 comp rx=0 tx=0).stop 2023-10-26T01:37:23.262 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.260+0000 7f0c0c36c640 1 -- 172.21.15.169:0/2712193144 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c0413e060 msgr2=0x7f0c040afcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:23.262 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.260+0000 7f0c0c36c640 1 --2- 172.21.15.169:0/2712193144 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c0413e060 0x7f0c040afcd0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f0bf400d770 tx=0x7f0bf400dc30 comp rx=0 tx=0).stop 2023-10-26T01:37:23.263 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.260+0000 7f0c0c36c640 1 -- 172.21.15.169:0/2712193144 shutdown_connections 2023-10-26T01:37:23.263 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.260+0000 7f0c0c36c640 1 --2- 172.21.15.169:0/2712193144 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f0be806ae70 0x7f0be806d330 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:23.263 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.260+0000 7f0c0c36c640 1 --2- 172.21.15.169:0/2712193144 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c0413e060 0x7f0c040afcd0 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:23.263 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.260+0000 7f0c0c36c640 1 --2- 172.21.15.169:0/2712193144 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0c0406ad90 0x7f0c0406b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:23.263 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.260+0000 7f0c0c36c640 1 --2- 172.21.15.169:0/2712193144 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0bf0003970 0x7f0c041f2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:23.264 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.260+0000 7f0c0c36c640 1 -- 172.21.15.169:0/2712193144 >> 172.21.15.169:0/2712193144 conn(0x7f0c04069930 msgr2=0x7f0c040b0e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:23.264 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.260+0000 7f0c0c36c640 1 -- 172.21.15.169:0/2712193144 shutdown_connections 2023-10-26T01:37:23.264 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.260+0000 7f0c0c36c640 1 -- 172.21.15.169:0/2712193144 wait complete. 2023-10-26T01:37:23.279 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-26T01:37:23.499 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.497+0000 7fe0a2ffd640 1 -- 172.21.15.169:0/1117793472 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe094036660 con 0x7fe08c0039c0 2023-10-26T01:37:23.500 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.498+0000 7fe0aa6e5640 1 -- 172.21.15.169:0/1117793472 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe08c0039c0 msgr2=0x7fe08c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:23.500 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.498+0000 7fe0aa6e5640 1 --2- 172.21.15.169:0/1117793472 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe08c0039c0 0x7fe08c003de0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fe09400bb20 tx=0x7fe08c0057f0 comp rx=0 tx=0).stop 2023-10-26T01:37:23.500 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.498+0000 7fe0aa6e5640 1 -- 172.21.15.169:0/1117793472 shutdown_connections 2023-10-26T01:37:23.500 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.498+0000 7fe0aa6e5640 1 --2- 172.21.15.169:0/1117793472 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe08c0039c0 0x7fe08c003de0 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:23.500 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.498+0000 7fe0aa6e5640 1 -- 172.21.15.169:0/1117793472 >> 172.21.15.169:0/1117793472 conn(0x7fe0a4069930 msgr2=0x7fe0a4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:23.501 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.498+0000 7fe0aa6e5640 1 -- 172.21.15.169:0/1117793472 shutdown_connections 2023-10-26T01:37:23.501 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.498+0000 7fe0aa6e5640 1 -- 172.21.15.169:0/1117793472 wait complete. 2023-10-26T01:37:23.501 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.499+0000 7fe0aa6e5640 1 Processor -- start 2023-10-26T01:37:23.502 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.500+0000 7fe0aa6e5640 1 -- start start 2023-10-26T01:37:23.504 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.500+0000 7fe0aa6e5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe08c0039c0 0x7fe0a4071ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:23.504 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.500+0000 7fe0aa6e5640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0a41533a0 0x7fe0a40acf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:23.504 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.500+0000 7fe0aa6e5640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe0a415a690 0x7fe0a415ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:23.504 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.500+0000 7fe0aa6e5640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a415b050 con 0x7fe0a41533a0 2023-10-26T01:37:23.505 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.500+0000 7fe0aa6e5640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a4153b50 con 0x7fe08c0039c0 2023-10-26T01:37:23.505 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.500+0000 7fe0aa6e5640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a40ad600 con 0x7fe0a415a690 2023-10-26T01:37:23.505 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.501+0000 7fe0a37fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0a41533a0 0x7fe0a40acf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:23.505 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.501+0000 7fe0a37fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0a41533a0 0x7fe0a40acf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:33028/0 (socket says 172.21.15.169:33028) 2023-10-26T01:37:23.506 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.501+0000 7fe0a37fe640 1 -- 172.21.15.169:0/1675119381 learned_addr learned my addr 172.21.15.169:0/1675119381 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:37:23.506 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.501+0000 7fe0a8c82640 1 --2- 172.21.15.169:0/1675119381 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe0a415a690 0x7fe0a415ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:23.506 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.501+0000 7fe0a3fff640 1 --2- 172.21.15.169:0/1675119381 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe08c0039c0 0x7fe0a4071ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:23.506 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.501+0000 7fe0a37fe640 1 -- 172.21.15.169:0/1675119381 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe0a415a690 msgr2=0x7fe0a415ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:23.507 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.501+0000 7fe0a37fe640 1 --2- 172.21.15.169:0/1675119381 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe0a415a690 0x7fe0a415ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:23.507 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.501+0000 7fe0a37fe640 1 -- 172.21.15.169:0/1675119381 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe08c0039c0 msgr2=0x7fe0a4071ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:23.507 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.501+0000 7fe0a37fe640 1 --2- 172.21.15.169:0/1675119381 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe08c0039c0 0x7fe0a4071ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:23.507 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.501+0000 7fe0a37fe640 1 -- 172.21.15.169:0/1675119381 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0a40ad8e0 con 0x7fe0a41533a0 2023-10-26T01:37:23.507 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.502+0000 7fe0a37fe640 1 --2- 172.21.15.169:0/1675119381 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0a41533a0 0x7fe0a40acf00 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fe09800c3c0 tx=0x7fe09800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:23.507 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.502+0000 7fe0a0ff9640 1 -- 172.21.15.169:0/1675119381 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0980052d0 con 0x7fe0a41533a0 2023-10-26T01:37:23.508 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.502+0000 7fe0a0ff9640 1 -- 172.21.15.169:0/1675119381 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe0980035f0 con 0x7fe0a41533a0 2023-10-26T01:37:23.508 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.502+0000 7fe0a0ff9640 1 -- 172.21.15.169:0/1675119381 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe098003a20 con 0x7fe0a41533a0 2023-10-26T01:37:23.508 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.502+0000 7fe0aa6e5640 1 -- 172.21.15.169:0/1675119381 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0a40adbc0 con 0x7fe0a41533a0 2023-10-26T01:37:23.508 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.502+0000 7fe0aa6e5640 1 -- 172.21.15.169:0/1675119381 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0a41e3020 con 0x7fe0a41533a0 2023-10-26T01:37:23.509 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.503+0000 7fe0aa6e5640 1 -- 172.21.15.169:0/1675119381 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0a404db10 con 0x7fe0a41533a0 2023-10-26T01:37:23.509 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.504+0000 7fe0a0ff9640 1 -- 172.21.15.169:0/1675119381 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fe098003b80 con 0x7fe0a41533a0 2023-10-26T01:37:23.510 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.505+0000 7fe0a0ff9640 1 --2- 172.21.15.169:0/1675119381 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe09006ae70 0x7fe09006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:23.510 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.505+0000 7fe0a0ff9640 1 -- 172.21.15.169:0/1675119381 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe098088d70 con 0x7fe0a41533a0 2023-10-26T01:37:23.510 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.505+0000 7fe0a3fff640 1 --2- 172.21.15.169:0/1675119381 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe09006ae70 0x7fe09006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:23.510 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.506+0000 7fe0a3fff640 1 --2- 172.21.15.169:0/1675119381 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe09006ae70 0x7fe09006d330 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fe094037000 tx=0x7fe09402e040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:23.511 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.509+0000 7fe0a0ff9640 1 -- 172.21.15.169:0/1675119381 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe098053d80 con 0x7fe0a41533a0 2023-10-26T01:37:23.697 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.695+0000 7fe0aa6e5640 1 -- 172.21.15.169:0/1675119381 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe0a4153d60 con 0x7fe0a41533a0 2023-10-26T01:37:23.698 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.696+0000 7fe0a0ff9640 1 -- 172.21.15.169:0/1675119381 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v109) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe0980170a0 con 0x7fe0a41533a0 2023-10-26T01:37:23.698 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 64 2023-10-26T01:37:23.700 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.699+0000 7fe0aa6e5640 1 -- 172.21.15.169:0/1675119381 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe09006ae70 msgr2=0x7fe09006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:23.701 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.699+0000 7fe0aa6e5640 1 --2- 172.21.15.169:0/1675119381 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe09006ae70 0x7fe09006d330 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fe094037000 tx=0x7fe09402e040 comp rx=0 tx=0).stop 2023-10-26T01:37:23.701 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.699+0000 7fe0aa6e5640 1 -- 172.21.15.169:0/1675119381 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0a41533a0 msgr2=0x7fe0a40acf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:23.701 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.699+0000 7fe0aa6e5640 1 --2- 172.21.15.169:0/1675119381 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0a41533a0 0x7fe0a40acf00 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fe09800c3c0 tx=0x7fe09800c880 comp rx=0 tx=0).stop 2023-10-26T01:37:23.701 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.699+0000 7fe0aa6e5640 1 -- 172.21.15.169:0/1675119381 shutdown_connections 2023-10-26T01:37:23.702 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.699+0000 7fe0aa6e5640 1 --2- 172.21.15.169:0/1675119381 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe09006ae70 0x7fe09006d330 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:23.702 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.699+0000 7fe0aa6e5640 1 --2- 172.21.15.169:0/1675119381 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0a41533a0 0x7fe0a40acf00 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:23.702 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.699+0000 7fe0aa6e5640 1 --2- 172.21.15.169:0/1675119381 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe0a415a690 0x7fe0a415ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:23.702 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.699+0000 7fe0aa6e5640 1 --2- 172.21.15.169:0/1675119381 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe08c0039c0 0x7fe0a4071ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:23.702 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.699+0000 7fe0aa6e5640 1 -- 172.21.15.169:0/1675119381 >> 172.21.15.169:0/1675119381 conn(0x7fe0a4069930 msgr2=0x7fe0a415b8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:23.703 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.699+0000 7fe0aa6e5640 1 -- 172.21.15.169:0/1675119381 shutdown_connections 2023-10-26T01:37:23.703 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.699+0000 7fe0aa6e5640 1 -- 172.21.15.169:0/1675119381 wait complete. 2023-10-26T01:37:23.720 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T01:37:23.942 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.940+0000 7f2a907a9640 1 -- 172.21.15.169:0/4231096366 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a740047c0 msgr2=0x7f2a74004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:23.942 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.940+0000 7f2a907a9640 1 --2- 172.21.15.169:0/4231096366 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a740047c0 0x7f2a74004c40 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c006fc0 tx=0x7f2a740057f0 comp rx=0 tx=0).stop 2023-10-26T01:37:23.943 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.940+0000 7f2a8ed46640 1 -- 172.21.15.169:0/4231096366 reap_dead start 2023-10-26T01:37:23.943 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.940+0000 7f2a907a9640 1 -- 172.21.15.169:0/4231096366 shutdown_connections 2023-10-26T01:37:23.943 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.940+0000 7f2a907a9640 1 -- 172.21.15.169:0/4231096366 >> 172.21.15.169:0/4231096366 conn(0x7f2a88069930 msgr2=0x7f2a88069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:23.943 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.940+0000 7f2a907a9640 1 -- 172.21.15.169:0/4231096366 shutdown_connections 2023-10-26T01:37:23.944 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.940+0000 7f2a907a9640 1 -- 172.21.15.169:0/4231096366 wait complete. 2023-10-26T01:37:23.946 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.942+0000 7f2a907a9640 1 Processor -- start 2023-10-26T01:37:23.946 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.942+0000 7f2a907a9640 1 -- start start 2023-10-26T01:37:23.946 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.943+0000 7f2a907a9640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2a740047c0 0x7f2a8815a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:23.947 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.943+0000 7f2a907a9640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a88159a40 0x7f2a88149840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:23.947 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.943+0000 7f2a907a9640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a8806f660 0x7f2a8806fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:23.947 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.943+0000 7f2a907a9640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a8814b7f0 con 0x7f2a88159a40 2023-10-26T01:37:23.947 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.943+0000 7f2a907a9640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a8815afd0 con 0x7f2a8806f660 2023-10-26T01:37:23.948 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.943+0000 7f2a907a9640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a8814ba20 con 0x7f2a740047c0 2023-10-26T01:37:23.948 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.943+0000 7f2a8dd44640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a88159a40 0x7f2a88149840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:23.948 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.943+0000 7f2a8dd44640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a88159a40 0x7f2a88149840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:33054/0 (socket says 172.21.15.169:33054) 2023-10-26T01:37:23.948 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.943+0000 7f2a8dd44640 1 -- 172.21.15.169:0/3442090451 learned_addr learned my addr 172.21.15.169:0/3442090451 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:37:23.948 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.944+0000 7f2a8dd44640 1 -- 172.21.15.169:0/3442090451 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2a740047c0 msgr2=0x7f2a8815a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:23.949 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.944+0000 7f2a8e545640 1 --2- 172.21.15.169:0/3442090451 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2a740047c0 0x7f2a8815a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:23.949 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.944+0000 7f2a8ed46640 1 --2- 172.21.15.169:0/3442090451 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a8806f660 0x7f2a8806fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:23.949 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.944+0000 7f2a8dd44640 1 --2- 172.21.15.169:0/3442090451 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2a740047c0 0x7f2a8815a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:23.949 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.944+0000 7f2a8dd44640 1 -- 172.21.15.169:0/3442090451 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a8806f660 msgr2=0x7f2a8806fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:23.950 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.944+0000 7f2a8dd44640 1 --2- 172.21.15.169:0/3442090451 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a8806f660 0x7f2a8806fae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:23.950 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.944+0000 7f2a8dd44640 1 -- 172.21.15.169:0/3442090451 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a8814bcc0 con 0x7f2a88159a40 2023-10-26T01:37:23.950 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.944+0000 7f2a8e545640 1 --2- 172.21.15.169:0/3442090451 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2a740047c0 0x7f2a8815a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:37:23.950 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.944+0000 7f2a8dd44640 1 --2- 172.21.15.169:0/3442090451 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a88159a40 0x7f2a88149840 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c02c8e0 tx=0x7f2a7c000f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:23.950 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.944+0000 7f2a7affd640 1 -- 172.21.15.169:0/3442090451 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a7c001690 con 0x7f2a88159a40 2023-10-26T01:37:23.951 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.944+0000 7f2a7affd640 1 -- 172.21.15.169:0/3442090451 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2a7c006050 con 0x7f2a88159a40 2023-10-26T01:37:23.951 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.944+0000 7f2a7affd640 1 -- 172.21.15.169:0/3442090451 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a7c0063e0 con 0x7f2a88159a40 2023-10-26T01:37:23.951 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.944+0000 7f2a907a9640 1 -- 172.21.15.169:0/3442090451 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a8814bfa0 con 0x7f2a88159a40 2023-10-26T01:37:23.951 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.945+0000 7f2a907a9640 1 -- 172.21.15.169:0/3442090451 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a8815bcb0 con 0x7f2a88159a40 2023-10-26T01:37:23.952 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.946+0000 7f2a907a9640 1 -- 172.21.15.169:0/3442090451 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a8804db10 con 0x7f2a88159a40 2023-10-26T01:37:23.953 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.947+0000 7f2a7affd640 1 -- 172.21.15.169:0/3442090451 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f2a7c031030 con 0x7f2a88159a40 2023-10-26T01:37:23.953 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.947+0000 7f2a7affd640 1 --2- 172.21.15.169:0/3442090451 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2a6c06ae70 0x7f2a6c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:23.953 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.948+0000 7f2a7affd640 1 -- 172.21.15.169:0/3442090451 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2a7c0aaca0 con 0x7f2a88159a40 2023-10-26T01:37:23.953 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.948+0000 7f2a8e545640 1 --2- 172.21.15.169:0/3442090451 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2a6c06ae70 0x7f2a6c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:23.953 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.949+0000 7f2a8e545640 1 --2- 172.21.15.169:0/3442090451 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2a6c06ae70 0x7f2a6c06d330 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f2a84001740 tx=0x7f2a84006a70 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:23.954 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:23.952+0000 7f2a7affd640 1 -- 172.21.15.169:0/3442090451 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2a7c079740 con 0x7f2a88159a40 2023-10-26T01:37:24.182 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:24.180+0000 7f2a907a9640 1 -- 172.21.15.169:0/3442090451 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2a8804db10 con 0x7f2a88159a40 2023-10-26T01:37:24.183 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:24.182+0000 7f2a7affd640 1 -- 172.21.15.169:0/3442090451 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+4924 (secure 0 0 0) 0x7f2a7c0792d0 con 0x7f2a88159a40 2023-10-26T01:37:24.184 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:37:24.184 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":50,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5240,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/3870332417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":3870332417},{"type":"v1","addr":"172.21.15.170:6837","nonce":3870332417}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45},{"gid":5497,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/2533771698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2533771698},{"type":"v1","addr":"172.21.15.170:6835","nonce":2533771698}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45},{"gid":6885,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/2117946713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2117946713},{"type":"v1","addr":"172.21.15.169:6834","nonce":2117946713}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T01:36:43.849046+0000","modified":"2023-10-26T01:36:46.926339+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6888},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6888":{"gid":6888,"name":"c","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6835/657142078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":657142078},{"type":"v1","addr":"172.21.15.169:6835","nonce":657142078}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-10-26T01:37:24.184 INFO:teuthology.orchestra.run.smithi169.stderr:dumped fsmap epoch 50 2023-10-26T01:37:24.186 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:24.184+0000 7f2a907a9640 1 -- 172.21.15.169:0/3442090451 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2a6c06ae70 msgr2=0x7f2a6c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:24.186 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:24.184+0000 7f2a907a9640 1 --2- 172.21.15.169:0/3442090451 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2a6c06ae70 0x7f2a6c06d330 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f2a84001740 tx=0x7f2a84006a70 comp rx=0 tx=0).stop 2023-10-26T01:37:24.187 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:24.184+0000 7f2a907a9640 1 -- 172.21.15.169:0/3442090451 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a88159a40 msgr2=0x7f2a88149840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:24.187 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:24.184+0000 7f2a907a9640 1 --2- 172.21.15.169:0/3442090451 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a88159a40 0x7f2a88149840 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c02c8e0 tx=0x7f2a7c000f50 comp rx=0 tx=0).stop 2023-10-26T01:37:24.187 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:24.184+0000 7f2a907a9640 1 -- 172.21.15.169:0/3442090451 shutdown_connections 2023-10-26T01:37:24.187 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:24.184+0000 7f2a907a9640 1 --2- 172.21.15.169:0/3442090451 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2a6c06ae70 0x7f2a6c06d330 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:24.187 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:24.184+0000 7f2a907a9640 1 --2- 172.21.15.169:0/3442090451 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a88159a40 0x7f2a88149840 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:24.188 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:24.184+0000 7f2a907a9640 1 --2- 172.21.15.169:0/3442090451 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a8806f660 0x7f2a8806fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:24.188 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:24.184+0000 7f2a907a9640 1 --2- 172.21.15.169:0/3442090451 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2a740047c0 0x7f2a8815a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:24.188 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:24.184+0000 7f2a907a9640 1 -- 172.21.15.169:0/3442090451 >> 172.21.15.169:0/3442090451 conn(0x7f2a88069930 msgr2=0x7f2a88069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:24.188 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:24.184+0000 7f2a907a9640 1 -- 172.21.15.169:0/3442090451 shutdown_connections 2023-10-26T01:37:24.188 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:24.184+0000 7f2a907a9640 1 -- 172.21.15.169:0/3442090451 wait complete. 2023-10-26T01:37:24.206 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:37:24.428 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:24.426+0000 7f49009d0640 1 -- 172.21.15.169:0/4130308869 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f48e4003970 msgr2=0x7f48e4003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:24.428 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:24.426+0000 7f49009d0640 1 --2- 172.21.15.169:0/4130308869 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f48e4003970 0x7f48e4003d90 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f48ec001650 tx=0x7f48e40057f0 comp rx=0 tx=0).stop 2023-10-26T01:37:24.429 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:24.426+0000 7f49009d0640 1 -- 172.21.15.169:0/4130308869 shutdown_connections 2023-10-26T01:37:24.429 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:24.426+0000 7f49009d0640 1 --2- 172.21.15.169:0/4130308869 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f48e4003970 0x7f48e4003d90 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:24.429 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:24.426+0000 7f49009d0640 1 -- 172.21.15.169:0/4130308869 >> 172.21.15.169:0/4130308869 conn(0x7f48fc069930 msgr2=0x7f48fc069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:24.429 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:24.426+0000 7f49009d0640 1 -- 172.21.15.169:0/4130308869 shutdown_connections 2023-10-26T01:37:24.429 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:24.426+0000 7f49009d0640 1 -- 172.21.15.169:0/4130308869 wait complete. 2023-10-26T01:37:24.431 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:24.428+0000 7f49009d0640 1 Processor -- start 2023-10-26T01:37:24.431 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:24.428+0000 7f49009d0640 1 -- start start 2023-10-26T01:37:24.433 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:24.429+0000 7f49009d0640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48e4003970 0x7f48fc143d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:24.433 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:24.429+0000 7f49009d0640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f48fc1492e0 0x7f48fc0c7f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:24.433 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:24.429+0000 7f49009d0640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f48fc0cba30 0x7f48fc0cbeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:24.433 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:24.429+0000 7f49009d0640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48fc0cc3f0 con 0x7f48fc1492e0 2023-10-26T01:37:24.434 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:24.429+0000 7f49009d0640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48fc145190 con 0x7f48e4003970 2023-10-26T01:37:24.434 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:24.429+0000 7f49009d0640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48fc144580 con 0x7f48fc0cba30 2023-10-26T01:37:24.434 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:24.430+0000 7f48f9d9b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f48fc1492e0 0x7f48fc0c7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:24.435 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:24.430+0000 7f48f9d9b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f48fc1492e0 0x7f48fc0c7f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:53628/0 (socket says 172.21.15.169:53628) 2023-10-26T01:37:24.435 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:24.430+0000 7f48f9d9b640 1 -- 172.21.15.169:0/1462233875 learned_addr learned my addr 172.21.15.169:0/1462233875 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:37:24.435 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:24.430+0000 7f48fa59c640 1 --2- 172.21.15.169:0/1462233875 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48e4003970 0x7f48fc143d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:24.435 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:24.430+0000 7f48fad9d640 1 --2- 172.21.15.169:0/1462233875 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f48fc0cba30 0x7f48fc0cbeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:24.435 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:24.430+0000 7f48f9d9b640 1 -- 172.21.15.169:0/1462233875 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f48fc0cba30 msgr2=0x7f48fc0cbeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:24.436 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:24.430+0000 7f48f9d9b640 1 --2- 172.21.15.169:0/1462233875 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f48fc0cba30 0x7f48fc0cbeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:24.436 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:24.430+0000 7f48f9d9b640 1 -- 172.21.15.169:0/1462233875 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48e4003970 msgr2=0x7f48fc143d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:24.436 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:24.430+0000 7f48f9d9b640 1 --2- 172.21.15.169:0/1462233875 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48e4003970 0x7f48fc143d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:24.436 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:24.430+0000 7f48f9d9b640 1 -- 172.21.15.169:0/1462233875 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48fc144860 con 0x7f48fc1492e0 2023-10-26T01:37:24.436 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:24.430+0000 7f48fad9d640 1 --2- 172.21.15.169:0/1462233875 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f48fc0cba30 0x7f48fc0cbeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:37:24.437 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:24.430+0000 7f48f9d9b640 1 --2- 172.21.15.169:0/1462233875 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f48fc1492e0 0x7f48fc0c7f40 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f48f000e690 tx=0x7f48f000eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:24.437 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:24.431+0000 7f48eaffd640 1 -- 172.21.15.169:0/1462233875 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f000ca50 con 0x7f48fc1492e0 2023-10-26T01:37:24.437 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:24.431+0000 7f48eaffd640 1 -- 172.21.15.169:0/1462233875 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f48f000cca0 con 0x7f48fc1492e0 2023-10-26T01:37:24.437 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:24.431+0000 7f48eaffd640 1 -- 172.21.15.169:0/1462233875 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f00116d0 con 0x7f48fc1492e0 2023-10-26T01:37:24.437 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:24.431+0000 7f49009d0640 1 -- 172.21.15.169:0/1462233875 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48fc144b40 con 0x7f48fc1492e0 2023-10-26T01:37:24.438 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:24.431+0000 7f49009d0640 1 -- 172.21.15.169:0/1462233875 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48fc144240 con 0x7f48fc1492e0 2023-10-26T01:37:24.439 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:24.432+0000 7f49009d0640 1 -- 172.21.15.169:0/1462233875 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48fc04db10 con 0x7f48fc1492e0 2023-10-26T01:37:24.439 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:24.433+0000 7f48eaffd640 1 -- 172.21.15.169:0/1462233875 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f48f0011830 con 0x7f48fc1492e0 2023-10-26T01:37:24.439 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:24.434+0000 7f48eaffd640 1 --2- 172.21.15.169:0/1462233875 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f48dc06ae40 0x7f48dc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:24.439 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:24.434+0000 7f48eaffd640 1 -- 172.21.15.169:0/1462233875 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f48f0089ff0 con 0x7f48fc1492e0 2023-10-26T01:37:24.440 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:24.435+0000 7f48fa59c640 1 --2- 172.21.15.169:0/1462233875 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f48dc06ae40 0x7f48dc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:24.440 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:24.436+0000 7f48fa59c640 1 --2- 172.21.15.169:0/1462233875 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f48dc06ae40 0x7f48dc06d300 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f48ec034000 tx=0x7f48ec02e040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:24.440 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:24.438+0000 7f48eaffd640 1 -- 172.21.15.169:0/1462233875 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f48f00550a0 con 0x7f48fc1492e0 2023-10-26T01:37:24.631 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:24.630+0000 7f49009d0640 1 -- 172.21.15.169:0/1462233875 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f48fc0c6560 con 0x7f48fc1492e0 2023-10-26T01:37:24.634 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:24.632+0000 7f48eaffd640 1 -- 172.21.15.169:0/1462233875 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v109) v1 ==== 74+0+16620 (secure 0 0 0) 0x7f48f00586c0 con 0x7f48fc1492e0 2023-10-26T01:37:24.635 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:37:24.635 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":109,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:37:13.181064+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-10-26T01:36:41.430455+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-10-26T01:36:42.883252+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:37:24.637 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:24.635+0000 7f49009d0640 1 -- 172.21.15.169:0/1462233875 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f48dc06ae40 msgr2=0x7f48dc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:24.638 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:24.635+0000 7f49009d0640 1 --2- 172.21.15.169:0/1462233875 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f48dc06ae40 0x7f48dc06d300 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f48ec034000 tx=0x7f48ec02e040 comp rx=0 tx=0).stop 2023-10-26T01:37:24.638 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:24.635+0000 7f49009d0640 1 -- 172.21.15.169:0/1462233875 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f48fc1492e0 msgr2=0x7f48fc0c7f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:24.638 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:24.635+0000 7f49009d0640 1 --2- 172.21.15.169:0/1462233875 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f48fc1492e0 0x7f48fc0c7f40 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f48f000e690 tx=0x7f48f000eb50 comp rx=0 tx=0).stop 2023-10-26T01:37:24.639 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:24.635+0000 7f49009d0640 1 -- 172.21.15.169:0/1462233875 shutdown_connections 2023-10-26T01:37:24.639 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:24.636+0000 7f49009d0640 1 --2- 172.21.15.169:0/1462233875 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f48dc06ae40 0x7f48dc06d300 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:24.639 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:24.636+0000 7f49009d0640 1 --2- 172.21.15.169:0/1462233875 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f48fc1492e0 0x7f48fc0c7f40 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:24.639 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:24.636+0000 7f49009d0640 1 --2- 172.21.15.169:0/1462233875 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f48fc0cba30 0x7f48fc0cbeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:24.640 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:24.636+0000 7f49009d0640 1 --2- 172.21.15.169:0/1462233875 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48e4003970 0x7f48fc143d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:24.640 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:24.636+0000 7f49009d0640 1 -- 172.21.15.169:0/1462233875 >> 172.21.15.169:0/1462233875 conn(0x7f48fc069930 msgr2=0x7f48fc1550a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:24.640 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:24.636+0000 7f49009d0640 1 -- 172.21.15.169:0/1462233875 shutdown_connections 2023-10-26T01:37:24.640 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:24.636+0000 7f49009d0640 1 -- 172.21.15.169:0/1462233875 wait complete. 2023-10-26T01:37:24.655 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-26T01:37:24.655 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T01:37:24.875 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:24.873+0000 7f398ee87640 1 -- 172.21.15.169:0/2208084862 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3978004ab0 msgr2=0x7f3978004ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:24.875 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:24.873+0000 7f398ee87640 1 --2- 172.21.15.169:0/2208084862 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3978004ab0 0x7f3978004ed0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f397c006f00 tx=0x7f3978006900 comp rx=0 tx=0).stop 2023-10-26T01:37:24.876 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:24.873+0000 7f398d424640 1 -- 172.21.15.169:0/2208084862 reap_dead start 2023-10-26T01:37:24.876 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:24.873+0000 7f398ee87640 1 -- 172.21.15.169:0/2208084862 shutdown_connections 2023-10-26T01:37:24.876 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:24.873+0000 7f398ee87640 1 -- 172.21.15.169:0/2208084862 >> 172.21.15.169:0/2208084862 conn(0x7f3988069930 msgr2=0x7f3988069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:24.876 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:24.873+0000 7f398ee87640 1 -- 172.21.15.169:0/2208084862 shutdown_connections 2023-10-26T01:37:24.877 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:24.873+0000 7f398ee87640 1 -- 172.21.15.169:0/2208084862 wait complete. 2023-10-26T01:37:24.878 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:24.875+0000 7f398ee87640 1 Processor -- start 2023-10-26T01:37:24.878 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:24.875+0000 7f398ee87640 1 -- start start 2023-10-26T01:37:24.878 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:24.876+0000 7f398ee87640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3978004ab0 0x7f39881559a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:24.879 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:24.876+0000 7f398ee87640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3988155ee0 0x7f3988154980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:24.879 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:24.876+0000 7f398ee87640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f39880718e0 0x7f3988154ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:24.879 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:24.876+0000 7f398ee87640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3988157030 con 0x7f39880718e0 2023-10-26T01:37:24.879 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:24.876+0000 7f398ee87640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3988155400 con 0x7f3988155ee0 2023-10-26T01:37:24.880 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:24.876+0000 7f398ee87640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3988156420 con 0x7f3978004ab0 2023-10-26T01:37:24.880 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:24.876+0000 7f398d424640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f39880718e0 0x7f3988154ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:24.880 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:24.876+0000 7f398d424640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f39880718e0 0x7f3988154ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:53642/0 (socket says 172.21.15.169:53642) 2023-10-26T01:37:24.880 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:24.876+0000 7f398d424640 1 -- 172.21.15.169:0/2210688951 learned_addr learned my addr 172.21.15.169:0/2210688951 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:37:24.881 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:24.876+0000 7f398d424640 1 -- 172.21.15.169:0/2210688951 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3978004ab0 msgr2=0x7f39881559a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:24.881 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:24.876+0000 7f398cc23640 1 --2- 172.21.15.169:0/2210688951 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3978004ab0 0x7f39881559a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:24.881 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:24.876+0000 7f3987fff640 1 --2- 172.21.15.169:0/2210688951 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3988155ee0 0x7f3988154980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:24.881 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:24.876+0000 7f398d424640 1 --2- 172.21.15.169:0/2210688951 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3978004ab0 0x7f39881559a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:24.881 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:24.876+0000 7f398d424640 1 -- 172.21.15.169:0/2210688951 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3988155ee0 msgr2=0x7f3988154980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:24.882 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:24.876+0000 7f398d424640 1 --2- 172.21.15.169:0/2210688951 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3988155ee0 0x7f3988154980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:24.882 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:24.876+0000 7f398d424640 1 -- 172.21.15.169:0/2210688951 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3988156700 con 0x7f39880718e0 2023-10-26T01:37:24.884 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:24.877+0000 7f398d424640 1 --2- 172.21.15.169:0/2210688951 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f39880718e0 0x7f3988154ec0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f3980005fd0 tx=0x7f3980009b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:24.885 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:24.877+0000 7f39857fa640 1 -- 172.21.15.169:0/2210688951 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f398000e900 con 0x7f39880718e0 2023-10-26T01:37:24.885 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:24.877+0000 7f39857fa640 1 -- 172.21.15.169:0/2210688951 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f398000eb50 con 0x7f39880718e0 2023-10-26T01:37:24.885 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:24.877+0000 7f39857fa640 1 -- 172.21.15.169:0/2210688951 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3980011470 con 0x7f39880718e0 2023-10-26T01:37:24.886 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:24.877+0000 7f398ee87640 1 -- 172.21.15.169:0/2210688951 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39881569e0 con 0x7f39880718e0 2023-10-26T01:37:24.886 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:24.877+0000 7f398ee87640 1 -- 172.21.15.169:0/2210688951 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3988148430 con 0x7f39880718e0 2023-10-26T01:37:24.886 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:24.878+0000 7f398ee87640 1 -- 172.21.15.169:0/2210688951 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39881571a0 con 0x7f39880718e0 2023-10-26T01:37:24.887 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:24.879+0000 7f39857fa640 1 -- 172.21.15.169:0/2210688951 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f39800115d0 con 0x7f39880718e0 2023-10-26T01:37:24.887 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:24.881+0000 7f39857fa640 1 --2- 172.21.15.169:0/2210688951 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f396c06ae70 0x7f396c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:24.887 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:24.881+0000 7f39857fa640 1 -- 172.21.15.169:0/2210688951 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3980089db0 con 0x7f39880718e0 2023-10-26T01:37:24.887 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:24.881+0000 7f398cc23640 1 --2- 172.21.15.169:0/2210688951 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f396c06ae70 0x7f396c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:24.888 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:24.883+0000 7f398cc23640 1 --2- 172.21.15.169:0/2210688951 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f396c06ae70 0x7f396c06d330 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f397c034000 tx=0x7f397c02e070 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:24.888 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:24.885+0000 7f39857fa640 1 -- 172.21.15.169:0/2210688951 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3980054ef0 con 0x7f39880718e0 2023-10-26T01:37:25.117 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.115+0000 7f398ee87640 1 -- 172.21.15.169:0/2210688951 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f398804db10 con 0x7f39880718e0 2023-10-26T01:37:25.117 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.116+0000 7f39857fa640 1 -- 172.21.15.169:0/2210688951 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+4924 (secure 0 0 0) 0x7f3980058510 con 0x7f39880718e0 2023-10-26T01:37:25.118 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:37:25.118 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":50,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5240,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/3870332417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":3870332417},{"type":"v1","addr":"172.21.15.170:6837","nonce":3870332417}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45},{"gid":5497,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/2533771698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2533771698},{"type":"v1","addr":"172.21.15.170:6835","nonce":2533771698}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45},{"gid":6885,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/2117946713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2117946713},{"type":"v1","addr":"172.21.15.169:6834","nonce":2117946713}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T01:36:43.849046+0000","modified":"2023-10-26T01:36:46.926339+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6888},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6888":{"gid":6888,"name":"c","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6835/657142078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":657142078},{"type":"v1","addr":"172.21.15.169:6835","nonce":657142078}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-10-26T01:37:25.119 INFO:teuthology.orchestra.run.smithi169.stderr:dumped fsmap epoch 50 2023-10-26T01:37:25.121 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.118+0000 7f398ee87640 1 -- 172.21.15.169:0/2210688951 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f396c06ae70 msgr2=0x7f396c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:25.121 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.118+0000 7f398ee87640 1 --2- 172.21.15.169:0/2210688951 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f396c06ae70 0x7f396c06d330 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f397c034000 tx=0x7f397c02e070 comp rx=0 tx=0).stop 2023-10-26T01:37:25.121 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.118+0000 7f398ee87640 1 -- 172.21.15.169:0/2210688951 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f39880718e0 msgr2=0x7f3988154ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:25.121 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.118+0000 7f398ee87640 1 --2- 172.21.15.169:0/2210688951 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f39880718e0 0x7f3988154ec0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f3980005fd0 tx=0x7f3980009b00 comp rx=0 tx=0).stop 2023-10-26T01:37:25.121 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.119+0000 7f398ee87640 1 -- 172.21.15.169:0/2210688951 shutdown_connections 2023-10-26T01:37:25.122 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.119+0000 7f398ee87640 1 --2- 172.21.15.169:0/2210688951 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f396c06ae70 0x7f396c06d330 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:25.122 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.119+0000 7f398ee87640 1 --2- 172.21.15.169:0/2210688951 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f39880718e0 0x7f3988154ec0 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:25.122 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.119+0000 7f398ee87640 1 --2- 172.21.15.169:0/2210688951 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3988155ee0 0x7f3988154980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:25.122 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.119+0000 7f398ee87640 1 --2- 172.21.15.169:0/2210688951 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3978004ab0 0x7f39881559a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:25.122 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.119+0000 7f398ee87640 1 -- 172.21.15.169:0/2210688951 >> 172.21.15.169:0/2210688951 conn(0x7f3988069930 msgr2=0x7f3988150230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:25.123 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.119+0000 7f398ee87640 1 -- 172.21.15.169:0/2210688951 shutdown_connections 2023-10-26T01:37:25.123 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.119+0000 7f398ee87640 1 -- 172.21.15.169:0/2210688951 wait complete. 2023-10-26T01:37:25.139 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T01:37:25.346 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.344+0000 7fbfef59e640 1 -- 172.21.15.169:0/105914142 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbff00ac6d0 msgr2=0x7fbff015d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:25.347 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.344+0000 7fbfef59e640 1 --2- 172.21.15.169:0/105914142 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbff00ac6d0 0x7fbff015d840 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fbfe0005fd0 tx=0x7fbfe002be20 comp rx=0 tx=0).stop 2023-10-26T01:37:25.347 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.344+0000 7fbfef59e640 1 -- 172.21.15.169:0/105914142 shutdown_connections 2023-10-26T01:37:25.347 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.344+0000 7fbfef59e640 1 --2- 172.21.15.169:0/105914142 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbff00ac6d0 0x7fbff015d840 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:25.347 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.344+0000 7fbfef59e640 1 -- 172.21.15.169:0/105914142 >> 172.21.15.169:0/105914142 conn(0x7fbff00aedd0 msgr2=0x7fbff00af1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:25.348 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.344+0000 7fbfef59e640 1 -- 172.21.15.169:0/105914142 shutdown_connections 2023-10-26T01:37:25.348 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.344+0000 7fbfef59e640 1 -- 172.21.15.169:0/105914142 wait complete. 2023-10-26T01:37:25.350 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.346+0000 7fbfef59e640 1 Processor -- start 2023-10-26T01:37:25.351 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.346+0000 7fbfef59e640 1 -- start start 2023-10-26T01:37:25.351 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.347+0000 7fbfef59e640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbff009a5c0 0x7fbff015c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:25.351 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.347+0000 7fbfef59e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbff00ac6d0 0x7fbff00c89f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:25.351 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.347+0000 7fbfef59e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbff00c9970 0x7fbff00b0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:25.351 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.347+0000 7fbfef59e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbff015c950 con 0x7fbff00c9970 2023-10-26T01:37:25.352 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.347+0000 7fbfef59e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbff015bf80 con 0x7fbff00ac6d0 2023-10-26T01:37:25.352 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.347+0000 7fbfef59e640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbff00c8f30 con 0x7fbff009a5c0 2023-10-26T01:37:25.352 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.347+0000 7fbfeed9d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbff00c9970 0x7fbff00b0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:25.352 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.347+0000 7fbfeed9d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbff00c9970 0x7fbff00b0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:53648/0 (socket says 172.21.15.169:53648) 2023-10-26T01:37:25.352 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.347+0000 7fbfeed9d640 1 -- 172.21.15.169:0/1219880911 learned_addr learned my addr 172.21.15.169:0/1219880911 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:37:25.353 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.348+0000 7fbfee59c640 1 --2- 172.21.15.169:0/1219880911 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbff009a5c0 0x7fbff015c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:25.353 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.348+0000 7fbfeed9d640 1 -- 172.21.15.169:0/1219880911 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbff009a5c0 msgr2=0x7fbff015c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:25.353 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.348+0000 7fbfeed9d640 1 --2- 172.21.15.169:0/1219880911 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbff009a5c0 0x7fbff015c410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:25.353 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.348+0000 7fbfeed9d640 1 -- 172.21.15.169:0/1219880911 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbff00ac6d0 msgr2=0x7fbff00c89f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:25.354 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.348+0000 7fbfe7fff640 1 --2- 172.21.15.169:0/1219880911 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbff00ac6d0 0x7fbff00c89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:25.354 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.348+0000 7fbfeed9d640 1 --2- 172.21.15.169:0/1219880911 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbff00ac6d0 0x7fbff00c89f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:25.354 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.348+0000 7fbfeed9d640 1 -- 172.21.15.169:0/1219880911 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfe0009560 con 0x7fbff00c9970 2023-10-26T01:37:25.354 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.348+0000 7fbfe7fff640 1 --2- 172.21.15.169:0/1219880911 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbff00ac6d0 0x7fbff00c89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:37:25.354 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.348+0000 7fbfeed9d640 1 --2- 172.21.15.169:0/1219880911 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbff00c9970 0x7fbff00b0c10 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fbfe800c7a0 tx=0x7fbfe800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:25.355 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.348+0000 7fbfe6ffd640 1 -- 172.21.15.169:0/1219880911 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe8003b70 con 0x7fbff00c9970 2023-10-26T01:37:25.355 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.349+0000 7fbfe6ffd640 1 -- 172.21.15.169:0/1219880911 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fbfe8003cf0 con 0x7fbff00c9970 2023-10-26T01:37:25.355 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.349+0000 7fbfe6ffd640 1 -- 172.21.15.169:0/1219880911 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe8003ea0 con 0x7fbff00c9970 2023-10-26T01:37:25.355 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.349+0000 7fbfef59e640 1 -- 172.21.15.169:0/1219880911 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbff015b3e0 con 0x7fbff00c9970 2023-10-26T01:37:25.355 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.349+0000 7fbfef59e640 1 -- 172.21.15.169:0/1219880911 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbff015b930 con 0x7fbff00c9970 2023-10-26T01:37:25.356 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.350+0000 7fbfef59e640 1 -- 172.21.15.169:0/1219880911 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbff015c0f0 con 0x7fbff00c9970 2023-10-26T01:37:25.356 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.351+0000 7fbfe6ffd640 1 -- 172.21.15.169:0/1219880911 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fbfe800aab0 con 0x7fbff00c9970 2023-10-26T01:37:25.357 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.352+0000 7fbfe6ffd640 1 --2- 172.21.15.169:0/1219880911 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbfd006ae40 0x7fbfd006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:25.357 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.352+0000 7fbfe6ffd640 1 -- 172.21.15.169:0/1219880911 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbfe8089480 con 0x7fbff00c9970 2023-10-26T01:37:25.357 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.353+0000 7fbfee59c640 1 --2- 172.21.15.169:0/1219880911 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbfd006ae40 0x7fbfd006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:25.357 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.354+0000 7fbfee59c640 1 --2- 172.21.15.169:0/1219880911 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbfd006ae40 0x7fbfd006d300 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fbfe0035000 tx=0x7fbfe0004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:25.358 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.356+0000 7fbfe6ffd640 1 -- 172.21.15.169:0/1219880911 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbfe8054520 con 0x7fbff00c9970 2023-10-26T01:37:25.568 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.566+0000 7fbfef59e640 1 -- 172.21.15.169:0/1219880911 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbff00ca410 con 0x7fbff00c9970 2023-10-26T01:37:25.569 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.567+0000 7fbfe6ffd640 1 -- 172.21.15.169:0/1219880911 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+4924 (secure 0 0 0) 0x7fbff00ca410 con 0x7fbff00c9970 2023-10-26T01:37:25.569 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:37:25.569 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":50,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5240,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/3870332417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":3870332417},{"type":"v1","addr":"172.21.15.170:6837","nonce":3870332417}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45},{"gid":5497,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/2533771698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2533771698},{"type":"v1","addr":"172.21.15.170:6835","nonce":2533771698}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45},{"gid":6885,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/2117946713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2117946713},{"type":"v1","addr":"172.21.15.169:6834","nonce":2117946713}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T01:36:43.849046+0000","modified":"2023-10-26T01:36:46.926339+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6888},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6888":{"gid":6888,"name":"c","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6835/657142078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":657142078},{"type":"v1","addr":"172.21.15.169:6835","nonce":657142078}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-10-26T01:37:25.570 INFO:teuthology.orchestra.run.smithi169.stderr:dumped fsmap epoch 50 2023-10-26T01:37:25.572 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.570+0000 7fbfef59e640 1 -- 172.21.15.169:0/1219880911 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbfd006ae40 msgr2=0x7fbfd006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:25.572 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.570+0000 7fbfef59e640 1 --2- 172.21.15.169:0/1219880911 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbfd006ae40 0x7fbfd006d300 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fbfe0035000 tx=0x7fbfe0004730 comp rx=0 tx=0).stop 2023-10-26T01:37:25.572 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.570+0000 7fbfef59e640 1 -- 172.21.15.169:0/1219880911 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbff00c9970 msgr2=0x7fbff00b0c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:25.572 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.570+0000 7fbfef59e640 1 --2- 172.21.15.169:0/1219880911 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbff00c9970 0x7fbff00b0c10 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fbfe800c7a0 tx=0x7fbfe800cc60 comp rx=0 tx=0).stop 2023-10-26T01:37:25.573 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.570+0000 7fbfef59e640 1 -- 172.21.15.169:0/1219880911 shutdown_connections 2023-10-26T01:37:25.573 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.570+0000 7fbfef59e640 1 --2- 172.21.15.169:0/1219880911 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbfd006ae40 0x7fbfd006d300 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:25.573 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.570+0000 7fbfef59e640 1 --2- 172.21.15.169:0/1219880911 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbff00c9970 0x7fbff00b0c10 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:25.573 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.570+0000 7fbfef59e640 1 --2- 172.21.15.169:0/1219880911 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbff00ac6d0 0x7fbff00c89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:25.573 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.570+0000 7fbfef59e640 1 --2- 172.21.15.169:0/1219880911 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbff009a5c0 0x7fbff015c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:25.574 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.570+0000 7fbfef59e640 1 -- 172.21.15.169:0/1219880911 >> 172.21.15.169:0/1219880911 conn(0x7fbff00aedd0 msgr2=0x7fbff0098a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:25.574 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.570+0000 7fbfef59e640 1 -- 172.21.15.169:0/1219880911 shutdown_connections 2023-10-26T01:37:25.574 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.570+0000 7fbfef59e640 1 -- 172.21.15.169:0/1219880911 wait complete. 2023-10-26T01:37:25.589 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:37:25.803 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.800+0000 7f67852f2640 1 -- 172.21.15.169:0/1226966286 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6780158850 msgr2=0x7f67800aab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:25.803 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.800+0000 7f67852f2640 1 --2- 172.21.15.169:0/1226966286 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6780158850 0x7f67800aab40 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f67800accf0 tx=0x7f676802bd30 comp rx=0 tx=0).stop 2023-10-26T01:37:25.803 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.801+0000 7f67852f2640 1 -- 172.21.15.169:0/1226966286 shutdown_connections 2023-10-26T01:37:25.803 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.801+0000 7f67852f2640 1 --2- 172.21.15.169:0/1226966286 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6780158850 0x7f67800aab40 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:25.804 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.801+0000 7f67852f2640 1 --2- 172.21.15.169:0/1226966286 >> v2:172.21.15.170:3300/0 conn(0x7f678006ce20 0x7f678014e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:25.804 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.801+0000 7f67852f2640 1 -- 172.21.15.169:0/1226966286 >> 172.21.15.169:0/1226966286 conn(0x7f6780069930 msgr2=0x7f6780069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:25.804 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.801+0000 7f67852f2640 1 -- 172.21.15.169:0/1226966286 shutdown_connections 2023-10-26T01:37:25.804 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.801+0000 7f67852f2640 1 -- 172.21.15.169:0/1226966286 wait complete. 2023-10-26T01:37:25.806 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.802+0000 7f67852f2640 1 Processor -- start 2023-10-26T01:37:25.806 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.803+0000 7f67852f2640 1 -- start start 2023-10-26T01:37:25.806 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.803+0000 7f67852f2640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f678006ce20 0x7f6780151570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:25.807 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.803+0000 7f67852f2640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f67800cd720 0x7f67800aeec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:25.807 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.803+0000 7f67852f2640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6780158850 0x7f67800af400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:25.807 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.804+0000 7f67852f2640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67800b02e0 con 0x7f67800cd720 2023-10-26T01:37:25.807 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.804+0000 7f67852f2640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6780142b90 con 0x7f6780158850 2023-10-26T01:37:25.807 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.804+0000 7f67852f2640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f678014fa20 con 0x7f678006ce20 2023-10-26T01:37:25.807 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.804+0000 7f67767fc640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f67800cd720 0x7f67800aeec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:25.808 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.804+0000 7f67767fc640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f67800cd720 0x7f67800aeec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:53670/0 (socket says 172.21.15.169:53670) 2023-10-26T01:37:25.808 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.804+0000 7f67767fc640 1 -- 172.21.15.169:0/642999694 learned_addr learned my addr 172.21.15.169:0/642999694 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:37:25.808 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.804+0000 7f67767fc640 1 -- 172.21.15.169:0/642999694 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f678006ce20 msgr2=0x7f6780151570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:25.808 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.804+0000 7f67767fc640 1 --2- 172.21.15.169:0/642999694 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f678006ce20 0x7f6780151570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:25.808 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.804+0000 7f67767fc640 1 -- 172.21.15.169:0/642999694 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6780158850 msgr2=0x7f67800af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:25.809 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.804+0000 7f67767fc640 1 --2- 172.21.15.169:0/642999694 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6780158850 0x7f67800af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:25.809 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.804+0000 7f67767fc640 1 -- 172.21.15.169:0/642999694 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6768009560 con 0x7f67800cd720 2023-10-26T01:37:25.809 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.805+0000 7f67767fc640 1 --2- 172.21.15.169:0/642999694 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f67800cd720 0x7f67800aeec0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f6768036040 tx=0x7f676802dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:25.810 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.805+0000 7f6777fff640 1 -- 172.21.15.169:0/642999694 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6768003800 con 0x7f67800cd720 2023-10-26T01:37:25.811 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.805+0000 7f6777fff640 1 -- 172.21.15.169:0/642999694 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6768003a50 con 0x7f67800cd720 2023-10-26T01:37:25.811 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.805+0000 7f67852f2640 1 -- 172.21.15.169:0/642999694 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f678014fd00 con 0x7f67800cd720 2023-10-26T01:37:25.811 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.805+0000 7f6777fff640 1 -- 172.21.15.169:0/642999694 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6768003de0 con 0x7f67800cd720 2023-10-26T01:37:25.811 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.805+0000 7f67852f2640 1 -- 172.21.15.169:0/642999694 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67800aff70 con 0x7f67800cd720 2023-10-26T01:37:25.811 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.806+0000 7f67852f2640 1 -- 172.21.15.169:0/642999694 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6780066e60 con 0x7f67800cd720 2023-10-26T01:37:25.812 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.807+0000 7f6777fff640 1 -- 172.21.15.169:0/642999694 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f676803e450 con 0x7f67800cd720 2023-10-26T01:37:25.812 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.808+0000 7f6777fff640 1 --2- 172.21.15.169:0/642999694 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f676006ae70 0x7f676006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:25.812 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.808+0000 7f6777fff640 1 -- 172.21.15.169:0/642999694 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f67680ac170 con 0x7f67800cd720 2023-10-26T01:37:25.812 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.809+0000 7f677effd640 1 --2- 172.21.15.169:0/642999694 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f676006ae70 0x7f676006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:25.813 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.810+0000 7f677effd640 1 --2- 172.21.15.169:0/642999694 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f676006ae70 0x7f676006d330 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f6770005fd0 tx=0x7f6770005850 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:25.814 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.812+0000 7f6777fff640 1 -- 172.21.15.169:0/642999694 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6768077340 con 0x7f67800cd720 2023-10-26T01:37:25.991 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.989+0000 7f67852f2640 1 -- 172.21.15.169:0/642999694 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f678004db10 con 0x7f67800cd720 2023-10-26T01:37:25.993 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.991+0000 7f6777fff640 1 -- 172.21.15.169:0/642999694 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v109) v1 ==== 74+0+16620 (secure 0 0 0) 0x7f676807a960 con 0x7f67800cd720 2023-10-26T01:37:25.993 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:37:25.994 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":109,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:37:13.181064+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-10-26T01:36:41.430455+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-10-26T01:36:42.883252+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:37:25.996 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.994+0000 7f67852f2640 1 -- 172.21.15.169:0/642999694 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f676006ae70 msgr2=0x7f676006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:25.996 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.994+0000 7f67852f2640 1 --2- 172.21.15.169:0/642999694 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f676006ae70 0x7f676006d330 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f6770005fd0 tx=0x7f6770005850 comp rx=0 tx=0).stop 2023-10-26T01:37:25.996 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.994+0000 7f67852f2640 1 -- 172.21.15.169:0/642999694 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f67800cd720 msgr2=0x7f67800aeec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:25.996 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.994+0000 7f67852f2640 1 --2- 172.21.15.169:0/642999694 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f67800cd720 0x7f67800aeec0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f6768036040 tx=0x7f676802dab0 comp rx=0 tx=0).stop 2023-10-26T01:37:25.996 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.994+0000 7f67852f2640 1 -- 172.21.15.169:0/642999694 shutdown_connections 2023-10-26T01:37:25.997 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.994+0000 7f67852f2640 1 --2- 172.21.15.169:0/642999694 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f676006ae70 0x7f676006d330 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:25.997 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.994+0000 7f67852f2640 1 --2- 172.21.15.169:0/642999694 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f67800cd720 0x7f67800aeec0 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:25.997 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.994+0000 7f67852f2640 1 --2- 172.21.15.169:0/642999694 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6780158850 0x7f67800af400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:25.997 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.994+0000 7f67852f2640 1 --2- 172.21.15.169:0/642999694 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f678006ce20 0x7f6780151570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:25.997 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.994+0000 7f67852f2640 1 -- 172.21.15.169:0/642999694 >> 172.21.15.169:0/642999694 conn(0x7f6780069930 msgr2=0x7f67800acd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:25.998 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.994+0000 7f67852f2640 1 -- 172.21.15.169:0/642999694 shutdown_connections 2023-10-26T01:37:25.998 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:25.994+0000 7f67852f2640 1 -- 172.21.15.169:0/642999694 wait complete. 2023-10-26T01:37:26.012 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-26T01:37:26.226 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:26.224+0000 7f2a0db97640 1 -- 172.21.15.169:0/1157051567 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a080b0c80 msgr2=0x7f2a080cd710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:26.226 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:26.224+0000 7f2a0db97640 1 --2- 172.21.15.169:0/1157051567 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a080b0c80 0x7f2a080cd710 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f2a00005fd0 tx=0x7f2a0002c0e0 comp rx=0 tx=0).stop 2023-10-26T01:37:26.226 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:26.224+0000 7f2a0db97640 1 -- 172.21.15.169:0/1157051567 shutdown_connections 2023-10-26T01:37:26.227 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:26.224+0000 7f2a0db97640 1 --2- 172.21.15.169:0/1157051567 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a080b0c80 0x7f2a080cd710 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:26.227 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:26.224+0000 7f2a0db97640 1 --2- 172.21.15.169:0/1157051567 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2a081597c0 0x7f2a080ac200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:26.227 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:26.224+0000 7f2a0db97640 1 -- 172.21.15.169:0/1157051567 >> 172.21.15.169:0/1157051567 conn(0x7f2a08069930 msgr2=0x7f2a08069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:26.227 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:26.224+0000 7f2a0db97640 1 -- 172.21.15.169:0/1157051567 shutdown_connections 2023-10-26T01:37:26.227 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:26.224+0000 7f2a0db97640 1 -- 172.21.15.169:0/1157051567 wait complete. 2023-10-26T01:37:26.228 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:26.226+0000 7f2a0db97640 1 Processor -- start 2023-10-26T01:37:26.228 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:26.226+0000 7f2a0db97640 1 -- start start 2023-10-26T01:37:26.230 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:26.227+0000 7f2a0db97640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a080b0c80 0x7f2a080af870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:26.230 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:26.227+0000 7f2a0db97640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a081593e0 0x7f2a08152f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:26.230 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:26.227+0000 7f2a0db97640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2a081597c0 0x7f2a0806c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:26.230 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:26.227+0000 7f2a0db97640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a080afdb0 con 0x7f2a081593e0 2023-10-26T01:37:26.230 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:26.227+0000 7f2a0db97640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a080c6a40 con 0x7f2a080b0c80 2023-10-26T01:37:26.231 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:26.227+0000 7f2a0db97640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a080c6d20 con 0x7f2a081597c0 2023-10-26T01:37:26.231 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:26.228+0000 7f29fffff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a081593e0 0x7f2a08152f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:26.231 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:26.228+0000 7f29fffff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a081593e0 0x7f2a08152f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:53678/0 (socket says 172.21.15.169:53678) 2023-10-26T01:37:26.231 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:26.228+0000 7f29fffff640 1 -- 172.21.15.169:0/1123670513 learned_addr learned my addr 172.21.15.169:0/1123670513 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:37:26.232 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:26.228+0000 7f2a077fe640 1 --2- 172.21.15.169:0/1123670513 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a080b0c80 0x7f2a080af870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:26.232 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:26.228+0000 7f2a07fff640 1 --2- 172.21.15.169:0/1123670513 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2a081597c0 0x7f2a0806c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:26.232 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:26.228+0000 7f29fffff640 1 -- 172.21.15.169:0/1123670513 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2a081597c0 msgr2=0x7f2a0806c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:26.232 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:26.228+0000 7f29fffff640 1 --2- 172.21.15.169:0/1123670513 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2a081597c0 0x7f2a0806c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:26.233 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:26.228+0000 7f29fffff640 1 -- 172.21.15.169:0/1123670513 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a080b0c80 msgr2=0x7f2a080af870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:26.233 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:26.228+0000 7f29fffff640 1 --2- 172.21.15.169:0/1123670513 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a080b0c80 0x7f2a080af870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:26.233 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:26.228+0000 7f29fffff640 1 -- 172.21.15.169:0/1123670513 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a00009840 con 0x7f2a081593e0 2023-10-26T01:37:26.234 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:26.229+0000 7f29fffff640 1 --2- 172.21.15.169:0/1123670513 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a081593e0 0x7f2a08152f10 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f29f000c970 tx=0x7f29f000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:26.235 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:26.229+0000 7f2a04ff9640 1 -- 172.21.15.169:0/1123670513 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29f0007bb0 con 0x7f2a081593e0 2023-10-26T01:37:26.235 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:26.229+0000 7f2a04ff9640 1 -- 172.21.15.169:0/1123670513 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f29f0007e00 con 0x7f2a081593e0 2023-10-26T01:37:26.235 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:26.229+0000 7f2a04ff9640 1 -- 172.21.15.169:0/1123670513 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29f0005280 con 0x7f2a081593e0 2023-10-26T01:37:26.235 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:26.229+0000 7f2a0db97640 1 -- 172.21.15.169:0/1123670513 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a080c7000 con 0x7f2a081593e0 2023-10-26T01:37:26.235 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:26.229+0000 7f2a0db97640 1 -- 172.21.15.169:0/1123670513 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a080c75c0 con 0x7f2a081593e0 2023-10-26T01:37:26.236 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:26.230+0000 7f2a0db97640 1 -- 172.21.15.169:0/1123670513 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a080c77b0 con 0x7f2a081593e0 2023-10-26T01:37:26.236 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:26.231+0000 7f2a04ff9640 1 -- 172.21.15.169:0/1123670513 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f29f00053e0 con 0x7f2a081593e0 2023-10-26T01:37:26.236 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:26.232+0000 7f2a04ff9640 1 --2- 172.21.15.169:0/1123670513 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f29e806ae40 0x7f29e806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:26.236 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:26.232+0000 7f2a04ff9640 1 -- 172.21.15.169:0/1123670513 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f29f0089ad0 con 0x7f2a081593e0 2023-10-26T01:37:26.236 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:26.233+0000 7f2a077fe640 1 --2- 172.21.15.169:0/1123670513 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f29e806ae40 0x7f29e806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:26.237 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:26.234+0000 7f2a077fe640 1 --2- 172.21.15.169:0/1123670513 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f29e806ae40 0x7f29e806d300 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f29f80011e0 tx=0x7f29f8009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:26.238 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:26.236+0000 7f2a04ff9640 1 -- 172.21.15.169:0/1123670513 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f29f0054c40 con 0x7f2a081593e0 2023-10-26T01:37:26.467 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:26.466+0000 7f2a0db97640 1 -- 172.21.15.169:0/1123670513 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f2a0804db10 con 0x7f2a081593e0 2023-10-26T01:37:26.468 INFO:tasks.ceph.mon.a.smithi169.stderr:2023-10-26T01:37:26.467+0000 7f1588a74640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-26T01:37:26.472 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:26.471+0000 7f2a04ff9640 1 -- 172.21.15.169:0/1123670513 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v51) v1 ==== 161+0+0 (secure 0 0 0) 0x7f29f001d070 con 0x7f2a081593e0 2023-10-26T01:37:26.473 INFO:teuthology.orchestra.run.smithi169.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-26T01:37:26.478 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:26.476+0000 7f2a0db97640 1 -- 172.21.15.169:0/1123670513 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f29e806ae40 msgr2=0x7f29e806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:26.478 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:26.476+0000 7f2a0db97640 1 --2- 172.21.15.169:0/1123670513 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f29e806ae40 0x7f29e806d300 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f29f80011e0 tx=0x7f29f8009040 comp rx=0 tx=0).stop 2023-10-26T01:37:26.478 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:26.476+0000 7f2a0db97640 1 -- 172.21.15.169:0/1123670513 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a081593e0 msgr2=0x7f2a08152f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:26.478 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:26.476+0000 7f2a0db97640 1 --2- 172.21.15.169:0/1123670513 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a081593e0 0x7f2a08152f10 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f29f000c970 tx=0x7f29f000ce30 comp rx=0 tx=0).stop 2023-10-26T01:37:26.479 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:26.477+0000 7f2a0db97640 1 -- 172.21.15.169:0/1123670513 shutdown_connections 2023-10-26T01:37:26.479 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:26.477+0000 7f2a0db97640 1 --2- 172.21.15.169:0/1123670513 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f29e806ae40 0x7f29e806d300 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:26.479 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:26.477+0000 7f2a0db97640 1 --2- 172.21.15.169:0/1123670513 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a081593e0 0x7f2a08152f10 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:26.479 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:26.477+0000 7f2a0db97640 1 --2- 172.21.15.169:0/1123670513 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2a081597c0 0x7f2a0806c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:26.479 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:26.477+0000 7f2a0db97640 1 --2- 172.21.15.169:0/1123670513 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a080b0c80 0x7f2a080af870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:26.480 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:26.477+0000 7f2a0db97640 1 -- 172.21.15.169:0/1123670513 >> 172.21.15.169:0/1123670513 conn(0x7f2a08069930 msgr2=0x7f2a0815a250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:26.480 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:26.478+0000 7f2a0db97640 1 -- 172.21.15.169:0/1123670513 shutdown_connections 2023-10-26T01:37:26.480 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:26.478+0000 7f2a0db97640 1 -- 172.21.15.169:0/1123670513 wait complete. 2023-10-26T01:37:26.491 INFO:tasks.ceph.mds.c.smithi169.stderr: -9470> 2023-10-26T01:36:48.358+0000 7f697889c640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-26T01:37:26.501 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-26T01:37:26.560 INFO:tasks.ceph.mds.c.smithi169.stdout:starting mds.c at 2023-10-26T01:37:26.699 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:26.697+0000 7f55b00b4640 1 -- 172.21.15.169:0/4197057918 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5594003fd0 msgr2=0x7f55940047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:26.700 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:26.697+0000 7f55b00b4640 1 --2- 172.21.15.169:0/4197057918 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5594003fd0 0x7f55940047c0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f5598006fd0 tx=0x7f55940057f0 comp rx=0 tx=0).stop 2023-10-26T01:37:26.700 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:26.697+0000 7f55ae651640 1 -- 172.21.15.169:0/4197057918 reap_dead start 2023-10-26T01:37:26.700 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:26.697+0000 7f55b00b4640 1 -- 172.21.15.169:0/4197057918 shutdown_connections 2023-10-26T01:37:26.700 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:26.697+0000 7f55b00b4640 1 -- 172.21.15.169:0/4197057918 >> 172.21.15.169:0/4197057918 conn(0x7f55a80af8a0 msgr2=0x7f55a80afcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:26.700 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:26.697+0000 7f55b00b4640 1 -- 172.21.15.169:0/4197057918 shutdown_connections 2023-10-26T01:37:26.701 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:26.697+0000 7f55b00b4640 1 -- 172.21.15.169:0/4197057918 wait complete. 2023-10-26T01:37:26.702 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:26.699+0000 7f55b00b4640 1 Processor -- start 2023-10-26T01:37:26.703 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:26.699+0000 7f55b00b4640 1 -- start start 2023-10-26T01:37:26.703 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:26.700+0000 7f55b00b4640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5594003fd0 0x7f55a8149570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:26.703 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:26.700+0000 7f55b00b4640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f55a814bfe0 0x7f55a815b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:26.703 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:26.700+0000 7f55b00b4640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f55a8155f50 0x7f55a815baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:26.703 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:26.700+0000 7f55b00b4640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55a814cd70 con 0x7f5594003fd0 2023-10-26T01:37:26.704 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:26.700+0000 7f55b00b4640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55a814d030 con 0x7f55a8155f50 2023-10-26T01:37:26.704 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:26.700+0000 7f55b00b4640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55a814d310 con 0x7f55a814bfe0 2023-10-26T01:37:26.704 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:26.700+0000 7f55ade50640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5594003fd0 0x7f55a8149570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:26.704 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:26.700+0000 7f55ade50640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5594003fd0 0x7f55a8149570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:53724/0 (socket says 172.21.15.169:53724) 2023-10-26T01:37:26.704 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:26.700+0000 7f55ade50640 1 -- 172.21.15.169:0/2436941648 learned_addr learned my addr 172.21.15.169:0/2436941648 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:37:26.705 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:26.701+0000 7f55ad64f640 1 --2- 172.21.15.169:0/2436941648 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f55a814bfe0 0x7f55a815b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:26.705 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:26.701+0000 7f55ae651640 1 --2- 172.21.15.169:0/2436941648 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f55a8155f50 0x7f55a815baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:26.705 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:26.701+0000 7f55ade50640 1 -- 172.21.15.169:0/2436941648 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f55a814bfe0 msgr2=0x7f55a815b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:26.705 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:26.701+0000 7f55ade50640 1 --2- 172.21.15.169:0/2436941648 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f55a814bfe0 0x7f55a815b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:26.705 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:26.701+0000 7f55ade50640 1 -- 172.21.15.169:0/2436941648 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f55a8155f50 msgr2=0x7f55a815baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:26.706 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:26.701+0000 7f55ade50640 1 --2- 172.21.15.169:0/2436941648 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f55a8155f50 0x7f55a815baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:26.706 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:26.701+0000 7f55ade50640 1 -- 172.21.15.169:0/2436941648 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55a814d5f0 con 0x7f5594003fd0 2023-10-26T01:37:26.706 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:26.701+0000 7f55ae651640 1 --2- 172.21.15.169:0/2436941648 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f55a8155f50 0x7f55a815baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:37:26.706 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:26.701+0000 7f55ade50640 1 --2- 172.21.15.169:0/2436941648 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5594003fd0 0x7f55a8149570 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f55a400e990 tx=0x7f55a400ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:26.707 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:26.701+0000 7f559e7fc640 1 -- 172.21.15.169:0/2436941648 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55a400cbe0 con 0x7f5594003fd0 2023-10-26T01:37:26.708 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:26.701+0000 7f559e7fc640 1 -- 172.21.15.169:0/2436941648 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f55a400ce30 con 0x7f5594003fd0 2023-10-26T01:37:26.708 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:26.701+0000 7f55b00b4640 1 -- 172.21.15.169:0/2436941648 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55a814d8d0 con 0x7f5594003fd0 2023-10-26T01:37:26.708 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:26.701+0000 7f559e7fc640 1 -- 172.21.15.169:0/2436941648 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55a4005280 con 0x7f5594003fd0 2023-10-26T01:37:26.708 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:26.701+0000 7f55b00b4640 1 -- 172.21.15.169:0/2436941648 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55a81585b0 con 0x7f5594003fd0 2023-10-26T01:37:26.708 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:26.702+0000 7f55b00b4640 1 -- 172.21.15.169:0/2436941648 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55a804db10 con 0x7f5594003fd0 2023-10-26T01:37:26.709 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:26.704+0000 7f559e7fc640 1 -- 172.21.15.169:0/2436941648 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f55a40053e0 con 0x7f5594003fd0 2023-10-26T01:37:26.709 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:26.704+0000 7f559e7fc640 1 --2- 172.21.15.169:0/2436941648 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f558c06ae70 0x7f558c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:26.709 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:26.705+0000 7f559e7fc640 1 -- 172.21.15.169:0/2436941648 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f55a408a090 con 0x7f5594003fd0 2023-10-26T01:37:26.709 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:26.705+0000 7f55ad64f640 1 --2- 172.21.15.169:0/2436941648 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f558c06ae70 0x7f558c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:26.710 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:26.706+0000 7f55ad64f640 1 --2- 172.21.15.169:0/2436941648 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f558c06ae70 0x7f558c06d330 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f5598034000 tx=0x7f559802e000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:26.710 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:26.709+0000 7f559e7fc640 1 -- 172.21.15.169:0/2436941648 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f55a4054fc0 con 0x7f5594003fd0 2023-10-26T01:37:26.935 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:26.934+0000 7f55b00b4640 1 -- 172.21.15.169:0/2436941648 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f55a8066a60 con 0x7f5594003fd0 2023-10-26T01:37:27.476 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:27.474+0000 7f559e7fc640 1 -- 172.21.15.169:0/2436941648 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v52) v1 ==== 104+0+0 (secure 0 0 0) 0x7f55a401f0a0 con 0x7f5594003fd0 2023-10-26T01:37:27.481 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:27.478+0000 7f55b00b4640 1 -- 172.21.15.169:0/2436941648 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f558c06ae70 msgr2=0x7f558c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:27.481 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:27.478+0000 7f55b00b4640 1 --2- 172.21.15.169:0/2436941648 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f558c06ae70 0x7f558c06d330 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f5598034000 tx=0x7f559802e000 comp rx=0 tx=0).stop 2023-10-26T01:37:27.481 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:27.478+0000 7f55b00b4640 1 -- 172.21.15.169:0/2436941648 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5594003fd0 msgr2=0x7f55a8149570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:27.481 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:27.478+0000 7f55b00b4640 1 --2- 172.21.15.169:0/2436941648 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5594003fd0 0x7f55a8149570 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f55a400e990 tx=0x7f55a400ee50 comp rx=0 tx=0).stop 2023-10-26T01:37:27.481 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:27.479+0000 7f55b00b4640 1 -- 172.21.15.169:0/2436941648 shutdown_connections 2023-10-26T01:37:27.482 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:27.479+0000 7f55b00b4640 1 --2- 172.21.15.169:0/2436941648 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f558c06ae70 0x7f558c06d330 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:27.482 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:27.479+0000 7f55b00b4640 1 --2- 172.21.15.169:0/2436941648 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f55a8155f50 0x7f55a815baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:27.482 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:27.479+0000 7f55b00b4640 1 --2- 172.21.15.169:0/2436941648 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f55a814bfe0 0x7f55a815b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:27.482 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:27.479+0000 7f55b00b4640 1 --2- 172.21.15.169:0/2436941648 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5594003fd0 0x7f55a8149570 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:27.482 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:27.479+0000 7f55b00b4640 1 -- 172.21.15.169:0/2436941648 >> 172.21.15.169:0/2436941648 conn(0x7f55a80af8a0 msgr2=0x7f55a81599b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:27.483 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:27.479+0000 7f55b00b4640 1 -- 172.21.15.169:0/2436941648 shutdown_connections 2023-10-26T01:37:27.483 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:27.479+0000 7f55b00b4640 1 -- 172.21.15.169:0/2436941648 wait complete. 2023-10-26T01:37:27.503 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-26T01:37:27.710 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:27.708+0000 7f1f70dd0640 1 -- 172.21.15.169:0/258897146 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1f58004ab0 msgr2=0x7f1f58004ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:27.710 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:27.708+0000 7f1f70dd0640 1 --2- 172.21.15.169:0/258897146 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1f58004ab0 0x7f1f58004ed0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c00c640 tx=0x7f1f58006900 comp rx=0 tx=0).stop 2023-10-26T01:37:27.710 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:27.708+0000 7f1f6ad9d640 1 -- 172.21.15.169:0/258897146 reap_dead start 2023-10-26T01:37:27.711 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:27.708+0000 7f1f70dd0640 1 -- 172.21.15.169:0/258897146 shutdown_connections 2023-10-26T01:37:27.711 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:27.708+0000 7f1f70dd0640 1 -- 172.21.15.169:0/258897146 >> 172.21.15.169:0/258897146 conn(0x7f1f6c0aeee0 msgr2=0x7f1f6c0af2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:27.711 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:27.708+0000 7f1f70dd0640 1 -- 172.21.15.169:0/258897146 shutdown_connections 2023-10-26T01:37:27.711 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:27.708+0000 7f1f70dd0640 1 -- 172.21.15.169:0/258897146 wait complete. 2023-10-26T01:37:27.712 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:27.710+0000 7f1f70dd0640 1 Processor -- start 2023-10-26T01:37:27.712 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:27.710+0000 7f1f70dd0640 1 -- start start 2023-10-26T01:37:27.714 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:27.711+0000 7f1f70dd0640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1f58004ab0 0x7f1f6c1464b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:27.714 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:27.711+0000 7f1f70dd0640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f6c153820 0x7f1f6c1e15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:27.715 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:27.711+0000 7f1f70dd0640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1f6c1e2bc0 0x7f1f6c1e3040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:27.715 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:27.711+0000 7f1f70dd0640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6c097650 con 0x7f1f58004ab0 2023-10-26T01:37:27.715 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:27.711+0000 7f1f70dd0640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6c1e3580 con 0x7f1f6c153820 2023-10-26T01:37:27.715 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:27.711+0000 7f1f70dd0640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6c1e1f10 con 0x7f1f6c1e2bc0 2023-10-26T01:37:27.715 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:27.711+0000 7f1f6a59c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1f58004ab0 0x7f1f6c1464b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:27.716 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:27.711+0000 7f1f6a59c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1f58004ab0 0x7f1f6c1464b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:53734/0 (socket says 172.21.15.169:53734) 2023-10-26T01:37:27.716 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:27.711+0000 7f1f6a59c640 1 -- 172.21.15.169:0/3951863941 learned_addr learned my addr 172.21.15.169:0/3951863941 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:37:27.716 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:27.711+0000 7f1f69d9b640 1 --2- 172.21.15.169:0/3951863941 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f6c153820 0x7f1f6c1e15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:27.716 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:27.712+0000 7f1f6ad9d640 1 --2- 172.21.15.169:0/3951863941 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1f6c1e2bc0 0x7f1f6c1e3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:27.717 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:27.712+0000 7f1f6a59c640 1 -- 172.21.15.169:0/3951863941 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1f6c1e2bc0 msgr2=0x7f1f6c1e3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:27.717 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:27.712+0000 7f1f6a59c640 1 --2- 172.21.15.169:0/3951863941 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1f6c1e2bc0 0x7f1f6c1e3040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:27.717 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:27.712+0000 7f1f6a59c640 1 -- 172.21.15.169:0/3951863941 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f6c153820 msgr2=0x7f1f6c1e15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:27.717 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:27.712+0000 7f1f6a59c640 1 --2- 172.21.15.169:0/3951863941 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f6c153820 0x7f1f6c1e15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:27.717 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:27.712+0000 7f1f6a59c640 1 -- 172.21.15.169:0/3951863941 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f6c1e21f0 con 0x7f1f58004ab0 2023-10-26T01:37:27.718 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:27.712+0000 7f1f6a59c640 1 --2- 172.21.15.169:0/3951863941 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1f58004ab0 0x7f1f6c1464b0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c02cbc0 tx=0x7f1f5c001a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:27.718 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:27.712+0000 7f1f62ffd640 1 -- 172.21.15.169:0/3951863941 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c003b70 con 0x7f1f58004ab0 2023-10-26T01:37:27.718 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:27.712+0000 7f1f62ffd640 1 -- 172.21.15.169:0/3951863941 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1f5c003cd0 con 0x7f1f58004ab0 2023-10-26T01:37:27.718 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:27.712+0000 7f1f62ffd640 1 -- 172.21.15.169:0/3951863941 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c006320 con 0x7f1f58004ab0 2023-10-26T01:37:27.718 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:27.712+0000 7f1f70dd0640 1 -- 172.21.15.169:0/3951863941 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f6c1e24d0 con 0x7f1f58004ab0 2023-10-26T01:37:27.719 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:27.713+0000 7f1f70dd0640 1 -- 172.21.15.169:0/3951863941 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f6c1469f0 con 0x7f1f58004ab0 2023-10-26T01:37:27.720 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:27.714+0000 7f1f70dd0640 1 -- 172.21.15.169:0/3951863941 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f6c066af0 con 0x7f1f58004ab0 2023-10-26T01:37:27.720 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:27.715+0000 7f1f62ffd640 1 -- 172.21.15.169:0/3951863941 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f1f5c031030 con 0x7f1f58004ab0 2023-10-26T01:37:27.720 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:27.715+0000 7f1f62ffd640 1 --2- 172.21.15.169:0/3951863941 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f1f4c06ae70 0x7f1f4c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:27.720 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:27.716+0000 7f1f62ffd640 1 -- 172.21.15.169:0/3951863941 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f1f5c0aa650 con 0x7f1f58004ab0 2023-10-26T01:37:27.720 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:27.716+0000 7f1f69d9b640 1 --2- 172.21.15.169:0/3951863941 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f1f4c06ae70 0x7f1f4c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:27.721 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:27.717+0000 7f1f69d9b640 1 --2- 172.21.15.169:0/3951863941 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f1f4c06ae70 0x7f1f4c06d330 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c09a080 tx=0x7f1f54006ae0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:27.721 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:27.720+0000 7f1f62ffd640 1 -- 172.21.15.169:0/3951863941 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1f5c075600 con 0x7f1f58004ab0 2023-10-26T01:37:27.914 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:27.912+0000 7f1f70dd0640 1 -- 172.21.15.169:0/3951863941 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f1f6c0afe40 con 0x7f1f58004ab0 2023-10-26T01:37:28.479 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:28.477+0000 7f1f62ffd640 1 -- 172.21.15.169:0/3951863941 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v111) v1 ==== 181+0+0 (secure 0 0 0) 0x7f1f5c078c20 con 0x7f1f58004ab0 2023-10-26T01:37:28.480 INFO:teuthology.orchestra.run.smithi169.stderr:pool 'cephfs_metadata' removed 2023-10-26T01:37:28.484 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:28.482+0000 7f1f70dd0640 1 -- 172.21.15.169:0/3951863941 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f1f4c06ae70 msgr2=0x7f1f4c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:28.484 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:28.482+0000 7f1f70dd0640 1 --2- 172.21.15.169:0/3951863941 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f1f4c06ae70 0x7f1f4c06d330 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c09a080 tx=0x7f1f54006ae0 comp rx=0 tx=0).stop 2023-10-26T01:37:28.485 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:28.482+0000 7f1f70dd0640 1 -- 172.21.15.169:0/3951863941 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1f58004ab0 msgr2=0x7f1f6c1464b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:28.485 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:28.482+0000 7f1f70dd0640 1 --2- 172.21.15.169:0/3951863941 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1f58004ab0 0x7f1f6c1464b0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c02cbc0 tx=0x7f1f5c001a20 comp rx=0 tx=0).stop 2023-10-26T01:37:28.485 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:28.483+0000 7f1f70dd0640 1 -- 172.21.15.169:0/3951863941 shutdown_connections 2023-10-26T01:37:28.486 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:28.483+0000 7f1f70dd0640 1 --2- 172.21.15.169:0/3951863941 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f1f4c06ae70 0x7f1f4c06d330 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:28.486 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:28.483+0000 7f1f70dd0640 1 --2- 172.21.15.169:0/3951863941 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1f6c1e2bc0 0x7f1f6c1e3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:28.486 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:28.483+0000 7f1f70dd0640 1 --2- 172.21.15.169:0/3951863941 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f6c153820 0x7f1f6c1e15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:28.486 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:28.483+0000 7f1f70dd0640 1 --2- 172.21.15.169:0/3951863941 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1f58004ab0 0x7f1f6c1464b0 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:28.486 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:28.483+0000 7f1f70dd0640 1 -- 172.21.15.169:0/3951863941 >> 172.21.15.169:0/3951863941 conn(0x7f1f6c0aeee0 msgr2=0x7f1f6c06eda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:28.487 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:28.483+0000 7f1f70dd0640 1 -- 172.21.15.169:0/3951863941 shutdown_connections 2023-10-26T01:37:28.487 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:28.483+0000 7f1f70dd0640 1 -- 172.21.15.169:0/3951863941 wait complete. 2023-10-26T01:37:28.506 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-26T01:37:28.724 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:28.722+0000 7f399bab4640 1 -- 172.21.15.169:0/3544955701 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f39940cf860 msgr2=0x7f39940b2fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:28.724 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:28.722+0000 7f399bab4640 1 --2- 172.21.15.169:0/3544955701 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f39940cf860 0x7f39940b2fd0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f3994071c50 tx=0x7f398c02c1d0 comp rx=0 tx=0).stop 2023-10-26T01:37:28.724 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:28.722+0000 7f399bab4640 1 -- 172.21.15.169:0/3544955701 shutdown_connections 2023-10-26T01:37:28.725 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:28.722+0000 7f399bab4640 1 --2- 172.21.15.169:0/3544955701 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f39940cf860 0x7f39940b2fd0 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:28.725 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:28.722+0000 7f399bab4640 1 --2- 172.21.15.169:0/3544955701 >> v2:172.21.15.170:3300/0 conn(0x7f3994152960 0x7f399406a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:28.725 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:28.722+0000 7f399bab4640 1 -- 172.21.15.169:0/3544955701 >> 172.21.15.169:0/3544955701 conn(0x7f39940b4b20 msgr2=0x7f3994098a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:28.725 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:28.722+0000 7f399bab4640 1 -- 172.21.15.169:0/3544955701 shutdown_connections 2023-10-26T01:37:28.725 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:28.722+0000 7f399bab4640 1 -- 172.21.15.169:0/3544955701 wait complete. 2023-10-26T01:37:28.727 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:28.723+0000 7f399bab4640 1 Processor -- start 2023-10-26T01:37:28.727 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:28.724+0000 7f399bab4640 1 -- start start 2023-10-26T01:37:28.728 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:28.725+0000 7f399bab4640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f39940cf860 0x7f39941510f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:28.728 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:28.725+0000 7f399bab4640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3994152960 0x7f3994147be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:28.728 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:28.725+0000 7f399bab4640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f399415c910 0x7f399415f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:28.728 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:28.725+0000 7f399bab4640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3994148120 con 0x7f39940cf860 2023-10-26T01:37:28.728 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:28.725+0000 7f399bab4640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39941503f0 con 0x7f3994152960 2023-10-26T01:37:28.728 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:28.725+0000 7f399bab4640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39941506d0 con 0x7f399415c910 2023-10-26T01:37:28.729 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:28.725+0000 7f3999850640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f39940cf860 0x7f39941510f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:28.729 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:28.725+0000 7f3999850640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f39940cf860 0x7f39941510f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:53748/0 (socket says 172.21.15.169:53748) 2023-10-26T01:37:28.729 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:28.725+0000 7f3999850640 1 -- 172.21.15.169:0/616639283 learned_addr learned my addr 172.21.15.169:0/616639283 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:37:28.729 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:28.725+0000 7f3999850640 1 -- 172.21.15.169:0/616639283 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f399415c910 msgr2=0x7f399415f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:28.729 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:28.725+0000 7f3999850640 1 --2- 172.21.15.169:0/616639283 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f399415c910 0x7f399415f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:28.730 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:28.725+0000 7f3999850640 1 -- 172.21.15.169:0/616639283 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3994152960 msgr2=0x7f3994147be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:28.730 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:28.725+0000 7f3999850640 1 --2- 172.21.15.169:0/616639283 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3994152960 0x7f3994147be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:28.730 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:28.725+0000 7f3999850640 1 -- 172.21.15.169:0/616639283 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f398c009900 con 0x7f39940cf860 2023-10-26T01:37:28.730 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:28.726+0000 7f3999850640 1 --2- 172.21.15.169:0/616639283 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f39940cf860 0x7f39941510f0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f399000b4a0 tx=0x7f399000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:28.732 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:28.726+0000 7f398a7fc640 1 -- 172.21.15.169:0/616639283 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3990010610 con 0x7f39940cf860 2023-10-26T01:37:28.732 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:28.726+0000 7f398a7fc640 1 -- 172.21.15.169:0/616639283 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3990010860 con 0x7f39940cf860 2023-10-26T01:37:28.732 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:28.726+0000 7f398a7fc640 1 -- 172.21.15.169:0/616639283 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3990010c70 con 0x7f39940cf860 2023-10-26T01:37:28.732 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:28.726+0000 7f399bab4640 1 -- 172.21.15.169:0/616639283 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39941509b0 con 0x7f39940cf860 2023-10-26T01:37:28.732 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:28.726+0000 7f399bab4640 1 -- 172.21.15.169:0/616639283 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f399415ef10 con 0x7f39940cf860 2023-10-26T01:37:28.733 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:28.727+0000 7f399bab4640 1 -- 172.21.15.169:0/616639283 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f399415ea90 con 0x7f39940cf860 2023-10-26T01:37:28.733 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:28.728+0000 7f398a7fc640 1 -- 172.21.15.169:0/616639283 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f3990017360 con 0x7f39940cf860 2023-10-26T01:37:28.733 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:28.729+0000 7f398a7fc640 1 --2- 172.21.15.169:0/616639283 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f397806ae40 0x7f397806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:28.733 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:28.729+0000 7f398a7fc640 1 -- 172.21.15.169:0/616639283 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5357+0+0 (secure 0 0 0) 0x7f3990089450 con 0x7f39940cf860 2023-10-26T01:37:28.733 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:28.730+0000 7f399904f640 1 --2- 172.21.15.169:0/616639283 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f397806ae40 0x7f397806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:28.734 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:28.731+0000 7f399904f640 1 --2- 172.21.15.169:0/616639283 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f397806ae40 0x7f397806d300 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f39940ced90 tx=0x7f3984006ba0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:28.735 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:28.734+0000 7f398a7fc640 1 -- 172.21.15.169:0/616639283 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f399415ea90 con 0x7f39940cf860 2023-10-26T01:37:28.934 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:28.932+0000 7f399bab4640 1 -- 172.21.15.169:0/616639283 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f399415ea90 con 0x7f39940cf860 2023-10-26T01:37:29.492 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:29.490+0000 7f398a7fc640 1 -- 172.21.15.169:0/616639283 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v112) v1 ==== 169+0+0 (secure 0 0 0) 0x7f399415ea90 con 0x7f39940cf860 2023-10-26T01:37:29.492 INFO:teuthology.orchestra.run.smithi169.stderr:pool 'cephfs_data' removed 2023-10-26T01:37:29.496 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:29.494+0000 7f399bab4640 1 -- 172.21.15.169:0/616639283 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f397806ae40 msgr2=0x7f397806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:29.496 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:29.494+0000 7f399bab4640 1 --2- 172.21.15.169:0/616639283 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f397806ae40 0x7f397806d300 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f39940ced90 tx=0x7f3984006ba0 comp rx=0 tx=0).stop 2023-10-26T01:37:29.496 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:29.494+0000 7f399bab4640 1 -- 172.21.15.169:0/616639283 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f39940cf860 msgr2=0x7f39941510f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:29.497 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:29.494+0000 7f399bab4640 1 --2- 172.21.15.169:0/616639283 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f39940cf860 0x7f39941510f0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f399000b4a0 tx=0x7f399000b960 comp rx=0 tx=0).stop 2023-10-26T01:37:29.497 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:29.494+0000 7f399bab4640 1 -- 172.21.15.169:0/616639283 shutdown_connections 2023-10-26T01:37:29.497 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:29.494+0000 7f399bab4640 1 --2- 172.21.15.169:0/616639283 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f397806ae40 0x7f397806d300 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:29.497 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:29.494+0000 7f399bab4640 1 --2- 172.21.15.169:0/616639283 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f399415c910 0x7f399415f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:29.497 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:29.494+0000 7f399bab4640 1 --2- 172.21.15.169:0/616639283 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3994152960 0x7f3994147be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:29.498 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:29.494+0000 7f399bab4640 1 --2- 172.21.15.169:0/616639283 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f39940cf860 0x7f39941510f0 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:29.498 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:29.494+0000 7f399bab4640 1 -- 172.21.15.169:0/616639283 >> 172.21.15.169:0/616639283 conn(0x7f39940b4b20 msgr2=0x7f39940ac3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:29.498 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:29.494+0000 7f399bab4640 1 -- 172.21.15.169:0/616639283 shutdown_connections 2023-10-26T01:37:29.498 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:29.494+0000 7f399bab4640 1 -- 172.21.15.169:0/616639283 wait complete. 2023-10-26T01:37:29.517 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 18 2023-10-26T01:37:29.740 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:29.738+0000 7f4ec1602640 1 -- 172.21.15.169:0/4244360466 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ea80047c0 msgr2=0x7f4ea8004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:29.740 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:29.738+0000 7f4ec1602640 1 --2- 172.21.15.169:0/4244360466 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ea80047c0 0x7f4ea8004c40 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f4ea4001680 tx=0x7f4ea80057f0 comp rx=0 tx=0).stop 2023-10-26T01:37:29.741 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:29.738+0000 7f4ec1602640 1 -- 172.21.15.169:0/4244360466 shutdown_connections 2023-10-26T01:37:29.741 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:29.738+0000 7f4ec1602640 1 --2- 172.21.15.169:0/4244360466 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ea80047c0 0x7f4ea8004c40 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:29.741 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:29.738+0000 7f4ec1602640 1 -- 172.21.15.169:0/4244360466 >> 172.21.15.169:0/4244360466 conn(0x7f4ebc0d1370 msgr2=0x7f4ebc0d1780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:29.741 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:29.738+0000 7f4ec1602640 1 -- 172.21.15.169:0/4244360466 shutdown_connections 2023-10-26T01:37:29.741 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:29.738+0000 7f4ec1602640 1 -- 172.21.15.169:0/4244360466 wait complete. 2023-10-26T01:37:29.743 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:29.740+0000 7f4ec1602640 1 Processor -- start 2023-10-26T01:37:29.743 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:29.740+0000 7f4ec1602640 1 -- start start 2023-10-26T01:37:29.743 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:29.741+0000 7f4ec1602640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ea80047c0 0x7f4ebc1477e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:29.744 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:29.741+0000 7f4ec1602640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4ebc1e67d0 0x7f4ebc15b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:29.744 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:29.741+0000 7f4ec1602640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ebc1e6c30 0x7f4ebc15b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:29.744 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:29.741+0000 7f4ec1602640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ebc0c9160 con 0x7f4ea80047c0 2023-10-26T01:37:29.744 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:29.741+0000 7f4ec1602640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ebc147d20 con 0x7f4ebc1e6c30 2023-10-26T01:37:29.745 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:29.741+0000 7f4ec1602640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ebc0c85e0 con 0x7f4ebc1e67d0 2023-10-26T01:37:29.745 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:29.741+0000 7f4ebbfff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ea80047c0 0x7f4ebc1477e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:29.745 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:29.741+0000 7f4ebbfff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ea80047c0 0x7f4ebc1477e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:53776/0 (socket says 172.21.15.169:53776) 2023-10-26T01:37:29.745 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:29.741+0000 7f4ebbfff640 1 -- 172.21.15.169:0/2961269995 learned_addr learned my addr 172.21.15.169:0/2961269995 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:37:29.746 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:29.741+0000 7f4ebbfff640 1 -- 172.21.15.169:0/2961269995 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4ebc1e67d0 msgr2=0x7f4ebc15b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:29.746 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:29.741+0000 7f4ebbfff640 1 --2- 172.21.15.169:0/2961269995 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4ebc1e67d0 0x7f4ebc15b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:29.746 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:29.741+0000 7f4ebbfff640 1 -- 172.21.15.169:0/2961269995 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ebc1e6c30 msgr2=0x7f4ebc15b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:29.746 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:29.741+0000 7f4ebbfff640 1 --2- 172.21.15.169:0/2961269995 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ebc1e6c30 0x7f4ebc15b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:29.746 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:29.741+0000 7f4ebbfff640 1 -- 172.21.15.169:0/2961269995 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ebc0c88c0 con 0x7f4ea80047c0 2023-10-26T01:37:29.748 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:29.742+0000 7f4ebbfff640 1 --2- 172.21.15.169:0/2961269995 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ea80047c0 0x7f4ebc1477e0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f4eac00da70 tx=0x7f4eac00df30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:29.748 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:29.742+0000 7f4eb8ff9640 1 -- 172.21.15.169:0/2961269995 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eac00bd70 con 0x7f4ea80047c0 2023-10-26T01:37:29.748 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:29.742+0000 7f4eb8ff9640 1 -- 172.21.15.169:0/2961269995 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4eac003970 con 0x7f4ea80047c0 2023-10-26T01:37:29.748 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:29.742+0000 7f4eb8ff9640 1 -- 172.21.15.169:0/2961269995 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eac003d60 con 0x7f4ea80047c0 2023-10-26T01:37:29.749 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:29.742+0000 7f4ec1602640 1 -- 172.21.15.169:0/2961269995 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ebc0c8ba0 con 0x7f4ea80047c0 2023-10-26T01:37:29.749 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:29.742+0000 7f4ec1602640 1 -- 172.21.15.169:0/2961269995 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ebc15df40 con 0x7f4ea80047c0 2023-10-26T01:37:29.749 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:29.743+0000 7f4ec1602640 1 -- 172.21.15.169:0/2961269995 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ebc15e0d0 con 0x7f4ea80047c0 2023-10-26T01:37:29.749 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:29.744+0000 7f4eb8ff9640 1 -- 172.21.15.169:0/2961269995 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f4eac003ec0 con 0x7f4ea80047c0 2023-10-26T01:37:29.749 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:29.745+0000 7f4eb8ff9640 1 --2- 172.21.15.169:0/2961269995 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f4e9c06ae40 0x7f4e9c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:29.750 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:29.745+0000 7f4eb8ff9640 1 -- 172.21.15.169:0/2961269995 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f4eac089d40 con 0x7f4ea80047c0 2023-10-26T01:37:29.750 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:29.746+0000 7f4ebb7fe640 1 --2- 172.21.15.169:0/2961269995 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f4e9c06ae40 0x7f4e9c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:29.750 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:29.747+0000 7f4ebb7fe640 1 --2- 172.21.15.169:0/2961269995 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f4e9c06ae40 0x7f4e9c06d300 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f4ea4034000 tx=0x7f4ea4001120 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:29.751 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:29.750+0000 7f4eb8ff9640 1 -- 172.21.15.169:0/2961269995 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4eac055050 con 0x7f4ea80047c0 2023-10-26T01:37:29.946 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:29.945+0000 7f4ec1602640 1 -- 172.21.15.169:0/2961269995 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "config reset", "num": 18} v 0) v1 -- 0x7f4ebc073c10 con 0x7f4ea80047c0 2023-10-26T01:37:29.954 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:29.953+0000 7f4eb8ff9640 1 -- 172.21.15.169:0/2961269995 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 18}]=0 v28) v1 ==== 71+0+0 (secure 0 0 0) 0x7f4eac058670 con 0x7f4ea80047c0 2023-10-26T01:37:29.957 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:29.955+0000 7f4ec1602640 1 -- 172.21.15.169:0/2961269995 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f4e9c06ae40 msgr2=0x7f4e9c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:29.958 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:29.955+0000 7f4ec1602640 1 --2- 172.21.15.169:0/2961269995 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f4e9c06ae40 0x7f4e9c06d300 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f4ea4034000 tx=0x7f4ea4001120 comp rx=0 tx=0).stop 2023-10-26T01:37:29.958 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:29.955+0000 7f4ec1602640 1 -- 172.21.15.169:0/2961269995 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ea80047c0 msgr2=0x7f4ebc1477e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:29.958 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:29.955+0000 7f4ec1602640 1 --2- 172.21.15.169:0/2961269995 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ea80047c0 0x7f4ebc1477e0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f4eac00da70 tx=0x7f4eac00df30 comp rx=0 tx=0).stop 2023-10-26T01:37:29.958 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:29.956+0000 7f4ec1602640 1 -- 172.21.15.169:0/2961269995 shutdown_connections 2023-10-26T01:37:29.959 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:29.956+0000 7f4ec1602640 1 --2- 172.21.15.169:0/2961269995 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f4e9c06ae40 0x7f4e9c06d300 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:29.959 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:29.956+0000 7f4ec1602640 1 --2- 172.21.15.169:0/2961269995 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ebc1e6c30 0x7f4ebc15b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:29.959 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:29.956+0000 7f4ec1602640 1 --2- 172.21.15.169:0/2961269995 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4ebc1e67d0 0x7f4ebc15b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:29.959 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:29.956+0000 7f4ec1602640 1 --2- 172.21.15.169:0/2961269995 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ea80047c0 0x7f4ebc1477e0 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:29.959 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:29.956+0000 7f4ec1602640 1 -- 172.21.15.169:0/2961269995 >> 172.21.15.169:0/2961269995 conn(0x7f4ebc0d1370 msgr2=0x7f4ebc0b1560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:29.960 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:29.956+0000 7f4ec1602640 1 -- 172.21.15.169:0/2961269995 shutdown_connections 2023-10-26T01:37:29.960 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:29.956+0000 7f4ec1602640 1 -- 172.21.15.169:0/2961269995 wait complete. 2023-10-26T01:37:29.977 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestClusterFull.test_full_same_file' 2023-10-26T01:37:30.201 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:30.198+0000 7f9422b33640 1 -- 172.21.15.169:0/982125547 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f941c0ac930 msgr2=0x7f941c0cd6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:30.201 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:30.198+0000 7f9422b33640 1 --2- 172.21.15.169:0/982125547 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f941c0ac930 0x7f941c0cd6f0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f941c06be60 tx=0x7f941002bd10 comp rx=0 tx=0).stop 2023-10-26T01:37:30.201 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:30.198+0000 7f9422b33640 1 -- 172.21.15.169:0/982125547 shutdown_connections 2023-10-26T01:37:30.201 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:30.198+0000 7f9422b33640 1 --2- 172.21.15.169:0/982125547 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f941c0cdc30 0x7f941c06ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:30.202 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:30.198+0000 7f9422b33640 1 --2- 172.21.15.169:0/982125547 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f941c0ac930 0x7f941c0cd6f0 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:30.202 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:30.198+0000 7f9422b33640 1 --2- 172.21.15.169:0/982125547 >> v2:172.21.15.170:3300/0 conn(0x7f941c1592d0 0x7f941c0aa3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:30.202 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:30.198+0000 7f9422b33640 1 -- 172.21.15.169:0/982125547 >> 172.21.15.169:0/982125547 conn(0x7f941c069930 msgr2=0x7f941c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:30.202 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:30.199+0000 7f9422b33640 1 -- 172.21.15.169:0/982125547 shutdown_connections 2023-10-26T01:37:30.203 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:30.199+0000 7f9422b33640 1 -- 172.21.15.169:0/982125547 wait complete. 2023-10-26T01:37:30.205 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:30.200+0000 7f9422b33640 1 Processor -- start 2023-10-26T01:37:30.205 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:30.200+0000 7f9422b33640 1 -- start start 2023-10-26T01:37:30.205 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:30.201+0000 7f9422b33640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f941c0ac930 0x7f941c151f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:30.206 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:30.201+0000 7f9422b33640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f941c0cdc30 0x7f941c09aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:30.206 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:30.201+0000 7f9422b33640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f941c1592d0 0x7f941c1548b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:30.206 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:30.201+0000 7f9422b33640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f941c09b3f0 con 0x7f941c0ac930 2023-10-26T01:37:30.206 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:30.201+0000 7f9422b33640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f941c1432e0 con 0x7f941c1592d0 2023-10-26T01:37:30.207 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:30.201+0000 7f9422b33640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f941c143550 con 0x7f941c0cdc30 2023-10-26T01:37:30.207 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:30.202+0000 7f94208cf640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f941c0ac930 0x7f941c151f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:30.207 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:30.202+0000 7f94208cf640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f941c0ac930 0x7f941c151f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:53802/0 (socket says 172.21.15.169:53802) 2023-10-26T01:37:30.207 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:30.202+0000 7f94208cf640 1 -- 172.21.15.169:0/3910256409 learned_addr learned my addr 172.21.15.169:0/3910256409 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:37:30.208 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:30.202+0000 7f94210d0640 1 --2- 172.21.15.169:0/3910256409 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f941c1592d0 0x7f941c1548b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:30.208 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:30.202+0000 7f941bfff640 1 --2- 172.21.15.169:0/3910256409 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f941c0cdc30 0x7f941c09aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:30.208 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:30.202+0000 7f94208cf640 1 -- 172.21.15.169:0/3910256409 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f941c0cdc30 msgr2=0x7f941c09aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:30.208 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:30.202+0000 7f94208cf640 1 --2- 172.21.15.169:0/3910256409 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f941c0cdc30 0x7f941c09aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:30.209 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:30.202+0000 7f94208cf640 1 -- 172.21.15.169:0/3910256409 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f941c1592d0 msgr2=0x7f941c1548b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:30.209 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:30.202+0000 7f94208cf640 1 --2- 172.21.15.169:0/3910256409 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f941c1592d0 0x7f941c1548b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:30.209 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:30.202+0000 7f94208cf640 1 -- 172.21.15.169:0/3910256409 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9410009560 con 0x7f941c0ac930 2023-10-26T01:37:30.209 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:30.202+0000 7f941bfff640 1 --2- 172.21.15.169:0/3910256409 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f941c0cdc30 0x7f941c09aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:37:30.209 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:30.202+0000 7f94210d0640 1 --2- 172.21.15.169:0/3910256409 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f941c1592d0 0x7f941c1548b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:37:30.209 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:30.202+0000 7f94208cf640 1 --2- 172.21.15.169:0/3910256409 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f941c0ac930 0x7f941c151f70 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f940c00d770 tx=0x7f940c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:30.210 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:30.203+0000 7f94197fa640 1 -- 172.21.15.169:0/3910256409 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f940c009c30 con 0x7f941c0ac930 2023-10-26T01:37:30.210 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:30.203+0000 7f94197fa640 1 -- 172.21.15.169:0/3910256409 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f940c009e80 con 0x7f941c0ac930 2023-10-26T01:37:30.210 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:30.203+0000 7f94197fa640 1 -- 172.21.15.169:0/3910256409 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f940c0106c0 con 0x7f941c0ac930 2023-10-26T01:37:30.210 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:30.203+0000 7f9422b33640 1 -- 172.21.15.169:0/3910256409 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f941c143830 con 0x7f941c0ac930 2023-10-26T01:37:30.210 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:30.203+0000 7f9422b33640 1 -- 172.21.15.169:0/3910256409 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f941c143df0 con 0x7f941c0ac930 2023-10-26T01:37:30.211 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:30.204+0000 7f9422b33640 1 -- 172.21.15.169:0/3910256409 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f941c06c190 con 0x7f941c0ac930 2023-10-26T01:37:30.212 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:30.205+0000 7f94197fa640 1 -- 172.21.15.169:0/3910256409 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f940c010820 con 0x7f941c0ac930 2023-10-26T01:37:30.212 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:30.206+0000 7f94197fa640 1 --2- 172.21.15.169:0/3910256409 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f940806ae40 0x7f940806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:30.212 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:30.206+0000 7f94197fa640 1 -- 172.21.15.169:0/3910256409 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f940c089790 con 0x7f941c0ac930 2023-10-26T01:37:30.212 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:30.207+0000 7f941bfff640 1 --2- 172.21.15.169:0/3910256409 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f940806ae40 0x7f940806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:30.212 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:30.208+0000 7f941bfff640 1 --2- 172.21.15.169:0/3910256409 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f940806ae40 0x7f940806d300 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f9410035000 tx=0x7f9410004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:30.213 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:30.210+0000 7f94197fa640 1 -- 172.21.15.169:0/3910256409 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f940c054b30 con 0x7f941c0ac930 2023-10-26T01:37:30.460 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:30.459+0000 7f9422b33640 1 -- 172.21.15.169:0/3910256409 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_same_file"]} v 0) v1 -- 0x7f941c04db10 con 0x7f941c0ac930 2023-10-26T01:37:30.495 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:30.493+0000 7f94197fa640 1 -- 172.21.15.169:0/3910256409 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_same_file"]}]=0 v351) v1 ==== 137+0+0 (secure 0 0 0) 0x7f940c0170d0 con 0x7f941c0ac930 2023-10-26T01:37:30.498 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:30.496+0000 7f9422b33640 1 -- 172.21.15.169:0/3910256409 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f940806ae40 msgr2=0x7f940806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:30.498 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:30.496+0000 7f9422b33640 1 --2- 172.21.15.169:0/3910256409 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f940806ae40 0x7f940806d300 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f9410035000 tx=0x7f9410004730 comp rx=0 tx=0).stop 2023-10-26T01:37:30.498 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:30.496+0000 7f9422b33640 1 -- 172.21.15.169:0/3910256409 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f941c0ac930 msgr2=0x7f941c151f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:30.498 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:30.496+0000 7f9422b33640 1 --2- 172.21.15.169:0/3910256409 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f941c0ac930 0x7f941c151f70 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f940c00d770 tx=0x7f940c00dc30 comp rx=0 tx=0).stop 2023-10-26T01:37:30.498 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:30.496+0000 7f9422b33640 1 -- 172.21.15.169:0/3910256409 shutdown_connections 2023-10-26T01:37:30.499 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:30.496+0000 7f9422b33640 1 --2- 172.21.15.169:0/3910256409 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f940806ae40 0x7f940806d300 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:30.499 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:30.496+0000 7f9422b33640 1 --2- 172.21.15.169:0/3910256409 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f941c1592d0 0x7f941c1548b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:30.499 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:30.496+0000 7f9422b33640 1 --2- 172.21.15.169:0/3910256409 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f941c0cdc30 0x7f941c09aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:30.499 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:30.496+0000 7f9422b33640 1 --2- 172.21.15.169:0/3910256409 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f941c0ac930 0x7f941c151f70 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:30.499 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:30.496+0000 7f9422b33640 1 -- 172.21.15.169:0/3910256409 >> 172.21.15.169:0/3910256409 conn(0x7f941c069930 msgr2=0x7f941c155f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:30.500 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:30.496+0000 7f9422b33640 1 -- 172.21.15.169:0/3910256409 shutdown_connections 2023-10-26T01:37:30.500 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:30.496+0000 7f9422b33640 1 -- 172.21.15.169:0/3910256409 wait complete. 2023-10-26T01:37:30.516 INFO:tasks.cephfs_test_runner:test_full_same_file (tasks.cephfs.test_full.TestClusterFull) ... ok 2023-10-26T01:37:30.517 INFO:tasks.cephfs_test_runner:Starting test: test_barrier (tasks.cephfs.test_full.TestQuotaFull) That when an OSD epoch barrier is set on an MDS, subsequently 2023-10-26T01:37:30.517 INFO:tasks.cephfs_test_runner:test_barrier (tasks.cephfs.test_full.TestQuotaFull) 2023-10-26T01:37:30.518 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:37:30.744 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:30.742+0000 7fe614000640 1 -- 172.21.15.169:0/3754006335 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5f80047c0 msgr2=0x7fe5f8004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:30.745 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:30.742+0000 7fe614000640 1 --2- 172.21.15.169:0/3754006335 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5f80047c0 0x7fe5f8004c40 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc009b80 tx=0x7fe5f80057f0 comp rx=0 tx=0).stop 2023-10-26T01:37:30.745 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:30.742+0000 7fe61259d640 1 -- 172.21.15.169:0/3754006335 reap_dead start 2023-10-26T01:37:30.745 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:30.742+0000 7fe614000640 1 -- 172.21.15.169:0/3754006335 shutdown_connections 2023-10-26T01:37:30.745 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:30.742+0000 7fe614000640 1 -- 172.21.15.169:0/3754006335 >> 172.21.15.169:0/3754006335 conn(0x7fe60c069930 msgr2=0x7fe60c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:30.745 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:30.742+0000 7fe614000640 1 -- 172.21.15.169:0/3754006335 shutdown_connections 2023-10-26T01:37:30.746 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:30.742+0000 7fe614000640 1 -- 172.21.15.169:0/3754006335 wait complete. 2023-10-26T01:37:30.747 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:30.744+0000 7fe614000640 1 Processor -- start 2023-10-26T01:37:30.748 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:30.744+0000 7fe614000640 1 -- start start 2023-10-26T01:37:30.748 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:30.745+0000 7fe614000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5f80047c0 0x7fe60c1425d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:30.748 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:30.745+0000 7fe614000640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe60c0c99e0 0x7fe60c15aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:30.748 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:30.745+0000 7fe614000640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe60c14bff0 0x7fe60c14c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:30.748 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:30.745+0000 7fe614000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe60c06f140 con 0x7fe5f80047c0 2023-10-26T01:37:30.749 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:30.745+0000 7fe614000640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe60c142b10 con 0x7fe60c0c99e0 2023-10-26T01:37:30.749 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:30.745+0000 7fe614000640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe60c15b8d0 con 0x7fe60c14bff0 2023-10-26T01:37:30.749 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:30.745+0000 7fe611d9c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5f80047c0 0x7fe60c1425d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:30.749 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:30.745+0000 7fe611d9c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5f80047c0 0x7fe60c1425d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:53826/0 (socket says 172.21.15.169:53826) 2023-10-26T01:37:30.749 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:30.746+0000 7fe611d9c640 1 -- 172.21.15.169:0/2652203013 learned_addr learned my addr 172.21.15.169:0/2652203013 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:37:30.750 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:30.746+0000 7fe611d9c640 1 -- 172.21.15.169:0/2652203013 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe60c14bff0 msgr2=0x7fe60c14c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:30.750 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:30.746+0000 7fe611d9c640 1 --2- 172.21.15.169:0/2652203013 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe60c14bff0 0x7fe60c14c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:30.750 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:30.746+0000 7fe611d9c640 1 -- 172.21.15.169:0/2652203013 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe60c0c99e0 msgr2=0x7fe60c15aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:30.750 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:30.746+0000 7fe611d9c640 1 --2- 172.21.15.169:0/2652203013 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe60c0c99e0 0x7fe60c15aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:30.750 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:30.746+0000 7fe611d9c640 1 -- 172.21.15.169:0/2652203013 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe60c15b310 con 0x7fe5f80047c0 2023-10-26T01:37:30.751 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:30.746+0000 7fe611d9c640 1 --2- 172.21.15.169:0/2652203013 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5f80047c0 0x7fe60c1425d0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fe60800e970 tx=0x7fe60800ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:30.752 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:30.746+0000 7fe6027fc640 1 -- 172.21.15.169:0/2652203013 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe60800ccc0 con 0x7fe5f80047c0 2023-10-26T01:37:30.752 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:30.746+0000 7fe6027fc640 1 -- 172.21.15.169:0/2652203013 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe60800ce20 con 0x7fe5f80047c0 2023-10-26T01:37:30.752 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:30.746+0000 7fe6027fc640 1 -- 172.21.15.169:0/2652203013 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe608005280 con 0x7fe5f80047c0 2023-10-26T01:37:30.752 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:30.746+0000 7fe614000640 1 -- 172.21.15.169:0/2652203013 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe60c15b5f0 con 0x7fe5f80047c0 2023-10-26T01:37:30.753 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:30.747+0000 7fe614000640 1 -- 172.21.15.169:0/2652203013 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe60c15af70 con 0x7fe5f80047c0 2023-10-26T01:37:30.753 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:30.748+0000 7fe614000640 1 -- 172.21.15.169:0/2652203013 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe60c04db10 con 0x7fe5f80047c0 2023-10-26T01:37:30.753 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:30.749+0000 7fe6027fc640 1 -- 172.21.15.169:0/2652203013 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fe6080053e0 con 0x7fe5f80047c0 2023-10-26T01:37:30.753 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:30.749+0000 7fe6027fc640 1 --2- 172.21.15.169:0/2652203013 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe5f006ae70 0x7fe5f006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:30.753 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:30.750+0000 7fe6027fc640 1 -- 172.21.15.169:0/2652203013 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fe608014070 con 0x7fe5f80047c0 2023-10-26T01:37:30.754 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:30.750+0000 7fe61159b640 1 --2- 172.21.15.169:0/2652203013 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe5f006ae70 0x7fe5f006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:30.754 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:30.751+0000 7fe61159b640 1 --2- 172.21.15.169:0/2652203013 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe5f006ae70 0x7fe5f006d330 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc036000 tx=0x7fe5fc004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:30.755 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:30.754+0000 7fe6027fc640 1 -- 172.21.15.169:0/2652203013 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe6080550e0 con 0x7fe5f80047c0 2023-10-26T01:37:30.951 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:30.949+0000 7fe614000640 1 -- 172.21.15.169:0/2652203013 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe60c157e00 con 0x7fe5f80047c0 2023-10-26T01:37:30.953 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:30.952+0000 7fe6027fc640 1 -- 172.21.15.169:0/2652203013 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v112) v1 ==== 74+0+13045 (secure 0 0 0) 0x7fe608089e60 con 0x7fe5f80047c0 2023-10-26T01:37:30.954 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:37:30.954 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":112,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:37:29.478486+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.169:6835/657142078":"2023-10-27T01:37:26.467469+0000","172.21.15.169:6833/657142078":"2023-10-27T01:37:26.467469+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:37:30.959 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:30.955+0000 7fe614000640 1 -- 172.21.15.169:0/2652203013 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe5f006ae70 msgr2=0x7fe5f006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:30.960 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:30.955+0000 7fe614000640 1 --2- 172.21.15.169:0/2652203013 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe5f006ae70 0x7fe5f006d330 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc036000 tx=0x7fe5fc004730 comp rx=0 tx=0).stop 2023-10-26T01:37:30.960 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:30.955+0000 7fe614000640 1 -- 172.21.15.169:0/2652203013 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5f80047c0 msgr2=0x7fe60c1425d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:30.960 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:30.955+0000 7fe614000640 1 --2- 172.21.15.169:0/2652203013 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5f80047c0 0x7fe60c1425d0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fe60800e970 tx=0x7fe60800ee30 comp rx=0 tx=0).stop 2023-10-26T01:37:30.960 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:30.955+0000 7fe614000640 1 -- 172.21.15.169:0/2652203013 shutdown_connections 2023-10-26T01:37:30.961 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:30.955+0000 7fe614000640 1 --2- 172.21.15.169:0/2652203013 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe5f006ae70 0x7fe5f006d330 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:30.961 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:30.955+0000 7fe614000640 1 --2- 172.21.15.169:0/2652203013 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe60c14bff0 0x7fe60c14c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:30.961 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:30.955+0000 7fe614000640 1 --2- 172.21.15.169:0/2652203013 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe60c0c99e0 0x7fe60c15aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:30.961 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:30.955+0000 7fe614000640 1 --2- 172.21.15.169:0/2652203013 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5f80047c0 0x7fe60c1425d0 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:30.961 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:30.955+0000 7fe614000640 1 -- 172.21.15.169:0/2652203013 >> 172.21.15.169:0/2652203013 conn(0x7fe60c069930 msgr2=0x7fe60c0afb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:30.962 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:30.955+0000 7fe614000640 1 -- 172.21.15.169:0/2652203013 shutdown_connections 2023-10-26T01:37:30.962 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:30.955+0000 7fe614000640 1 -- 172.21.15.169:0/2652203013 wait complete. 2023-10-26T01:37:30.975 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T01:30:32.200808+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '109', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T01:37:30.976 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T01:37:31.188 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:31.186+0000 7fe8b359e640 1 -- 172.21.15.169:0/170888810 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8b40cb980 msgr2=0x7fe8b406c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:31.189 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:31.186+0000 7fe8b359e640 1 --2- 172.21.15.169:0/170888810 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8b40cb980 0x7fe8b406c600 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fe8a4005fd0 tx=0x7fe8a402bd10 comp rx=0 tx=0).stop 2023-10-26T01:37:31.189 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:31.186+0000 7fe8b359e640 1 -- 172.21.15.169:0/170888810 shutdown_connections 2023-10-26T01:37:31.189 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:31.186+0000 7fe8b359e640 1 --2- 172.21.15.169:0/170888810 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe8b415d070 0x7fe8b40c9a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:31.189 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:31.186+0000 7fe8b359e640 1 --2- 172.21.15.169:0/170888810 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8b40cb980 0x7fe8b406c600 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:31.189 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:31.186+0000 7fe8b359e640 1 -- 172.21.15.169:0/170888810 >> 172.21.15.169:0/170888810 conn(0x7fe8b4069510 msgr2=0x7fe8b4069920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:31.190 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:31.186+0000 7fe8b359e640 1 -- 172.21.15.169:0/170888810 shutdown_connections 2023-10-26T01:37:31.190 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:31.186+0000 7fe8b359e640 1 -- 172.21.15.169:0/170888810 wait complete. 2023-10-26T01:37:31.190 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:31.189+0000 7fe8b359e640 1 Processor -- start 2023-10-26T01:37:31.191 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:31.189+0000 7fe8b359e640 1 -- start start 2023-10-26T01:37:31.192 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:31.190+0000 7fe8b359e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe8b40cb980 0x7fe8b40afbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:31.192 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:31.190+0000 7fe8b359e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8b40cbd60 0x7fe8b41f26b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:31.192 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:31.190+0000 7fe8b359e640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe8b415d070 0x7fe8b415bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:31.192 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:31.190+0000 7fe8b359e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b40b0130 con 0x7fe8b40cbd60 2023-10-26T01:37:31.193 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:31.190+0000 7fe8b359e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b4154090 con 0x7fe8b40cb980 2023-10-26T01:37:31.193 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:31.190+0000 7fe8b359e640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b4146c90 con 0x7fe8b415d070 2023-10-26T01:37:31.193 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:31.190+0000 7fe8abfff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8b40cbd60 0x7fe8b41f26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:31.193 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:31.190+0000 7fe8abfff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8b40cbd60 0x7fe8b41f26b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:53838/0 (socket says 172.21.15.169:53838) 2023-10-26T01:37:31.193 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:31.190+0000 7fe8abfff640 1 -- 172.21.15.169:0/1157768390 learned_addr learned my addr 172.21.15.169:0/1157768390 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:37:31.195 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:31.191+0000 7fe8b2d9d640 1 --2- 172.21.15.169:0/1157768390 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe8b415d070 0x7fe8b415bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:31.195 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:31.191+0000 7fe8b259c640 1 --2- 172.21.15.169:0/1157768390 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe8b40cb980 0x7fe8b40afbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:31.196 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:31.191+0000 7fe8abfff640 1 -- 172.21.15.169:0/1157768390 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe8b415d070 msgr2=0x7fe8b415bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:31.196 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:31.191+0000 7fe8abfff640 1 --2- 172.21.15.169:0/1157768390 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe8b415d070 0x7fe8b415bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:31.196 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:31.191+0000 7fe8abfff640 1 -- 172.21.15.169:0/1157768390 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe8b40cb980 msgr2=0x7fe8b40afbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:31.196 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:31.191+0000 7fe8abfff640 1 --2- 172.21.15.169:0/1157768390 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe8b40cb980 0x7fe8b40afbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:31.196 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:31.191+0000 7fe8abfff640 1 -- 172.21.15.169:0/1157768390 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8a4009560 con 0x7fe8b40cbd60 2023-10-26T01:37:31.197 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:31.191+0000 7fe8b2d9d640 1 --2- 172.21.15.169:0/1157768390 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe8b415d070 0x7fe8b415bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:37:31.197 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:31.191+0000 7fe8b259c640 1 --2- 172.21.15.169:0/1157768390 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe8b40cb980 0x7fe8b40afbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:37:31.197 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:31.191+0000 7fe8abfff640 1 --2- 172.21.15.169:0/1157768390 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8b40cbd60 0x7fe8b41f26b0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fe89c00c430 tx=0x7fe89c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:31.197 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:31.191+0000 7fe8aaffd640 1 -- 172.21.15.169:0/1157768390 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe89c011790 con 0x7fe8b40cbd60 2023-10-26T01:37:31.197 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:31.191+0000 7fe8aaffd640 1 -- 172.21.15.169:0/1157768390 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe89c0119e0 con 0x7fe8b40cbd60 2023-10-26T01:37:31.198 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:31.191+0000 7fe8aaffd640 1 -- 172.21.15.169:0/1157768390 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe89c011e30 con 0x7fe8b40cbd60 2023-10-26T01:37:31.198 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:31.191+0000 7fe8b359e640 1 -- 172.21.15.169:0/1157768390 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8b4158930 con 0x7fe8b40cbd60 2023-10-26T01:37:31.198 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:31.192+0000 7fe8b359e640 1 -- 172.21.15.169:0/1157768390 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8b4158ef0 con 0x7fe8b40cbd60 2023-10-26T01:37:31.198 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:31.193+0000 7fe8b359e640 1 -- 172.21.15.169:0/1157768390 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8b4154220 con 0x7fe8b40cbd60 2023-10-26T01:37:31.199 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:31.193+0000 7fe8aaffd640 1 -- 172.21.15.169:0/1157768390 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fe89c01c460 con 0x7fe8b40cbd60 2023-10-26T01:37:31.199 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:31.194+0000 7fe8aaffd640 1 --2- 172.21.15.169:0/1157768390 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe89406ae40 0x7fe89406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:31.200 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:31.194+0000 7fe8aaffd640 1 -- 172.21.15.169:0/1157768390 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fe89c0894f0 con 0x7fe8b40cbd60 2023-10-26T01:37:31.200 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:31.195+0000 7fe8b259c640 1 --2- 172.21.15.169:0/1157768390 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe89406ae40 0x7fe89406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:31.200 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:31.196+0000 7fe8b259c640 1 --2- 172.21.15.169:0/1157768390 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe89406ae40 0x7fe89406d300 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fe8a4035000 tx=0x7fe8a4004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:31.200 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:31.199+0000 7fe8aaffd640 1 -- 172.21.15.169:0/1157768390 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe89c054910 con 0x7fe8b40cbd60 2023-10-26T01:37:31.388 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:31.386+0000 7fe8b359e640 1 -- 172.21.15.169:0/1157768390 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe8b4073c10 con 0x7fe8b40cbd60 2023-10-26T01:37:31.389 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:31.387+0000 7fe8aaffd640 1 -- 172.21.15.169:0/1157768390 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v112) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe89c057f30 con 0x7fe8b40cbd60 2023-10-26T01:37:31.389 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 1 2023-10-26T01:37:31.391 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:31.390+0000 7fe8b359e640 1 -- 172.21.15.169:0/1157768390 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe89406ae40 msgr2=0x7fe89406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:31.392 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:31.390+0000 7fe8b359e640 1 --2- 172.21.15.169:0/1157768390 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe89406ae40 0x7fe89406d300 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fe8a4035000 tx=0x7fe8a4004730 comp rx=0 tx=0).stop 2023-10-26T01:37:31.392 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:31.390+0000 7fe8b359e640 1 -- 172.21.15.169:0/1157768390 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8b40cbd60 msgr2=0x7fe8b41f26b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:31.392 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:31.390+0000 7fe8b359e640 1 --2- 172.21.15.169:0/1157768390 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8b40cbd60 0x7fe8b41f26b0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fe89c00c430 tx=0x7fe89c00c8f0 comp rx=0 tx=0).stop 2023-10-26T01:37:31.392 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:31.390+0000 7fe8b359e640 1 -- 172.21.15.169:0/1157768390 shutdown_connections 2023-10-26T01:37:31.393 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:31.390+0000 7fe8b359e640 1 --2- 172.21.15.169:0/1157768390 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe89406ae40 0x7fe89406d300 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:31.393 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:31.390+0000 7fe8b359e640 1 --2- 172.21.15.169:0/1157768390 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8b40cbd60 0x7fe8b41f26b0 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:31.393 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:31.390+0000 7fe8b359e640 1 --2- 172.21.15.169:0/1157768390 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe8b415d070 0x7fe8b415bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:31.393 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:31.390+0000 7fe8b359e640 1 --2- 172.21.15.169:0/1157768390 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe8b40cb980 0x7fe8b40afbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:31.394 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:31.390+0000 7fe8b359e640 1 -- 172.21.15.169:0/1157768390 >> 172.21.15.169:0/1157768390 conn(0x7fe8b4069510 msgr2=0x7fe8b40698f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:31.394 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:31.390+0000 7fe8b359e640 1 -- 172.21.15.169:0/1157768390 shutdown_connections 2023-10-26T01:37:31.394 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:31.390+0000 7fe8b359e640 1 -- 172.21.15.169:0/1157768390 wait complete. 2023-10-26T01:37:31.410 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestQuotaFull.test_barrier' 2023-10-26T01:37:31.625 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:31.622+0000 7f29771d7640 1 -- 172.21.15.169:0/1051952918 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29701592d0 msgr2=0x7f29700aa3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:31.625 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:31.622+0000 7f29771d7640 1 --2- 172.21.15.169:0/1051952918 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29701592d0 0x7f29700aa3d0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f2964005fd0 tx=0x7f296402bd90 comp rx=0 tx=0).stop 2023-10-26T01:37:31.625 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:31.623+0000 7f29771d7640 1 -- 172.21.15.169:0/1051952918 shutdown_connections 2023-10-26T01:37:31.625 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:31.623+0000 7f29771d7640 1 --2- 172.21.15.169:0/1051952918 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f29700cde00 0x7f29700cbac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:31.626 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:31.623+0000 7f29771d7640 1 --2- 172.21.15.169:0/1051952918 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29701592d0 0x7f29700aa3d0 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:31.626 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:31.623+0000 7f29771d7640 1 -- 172.21.15.169:0/1051952918 >> 172.21.15.169:0/1051952918 conn(0x7f2970069930 msgr2=0x7f2970069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:31.626 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:31.623+0000 7f29771d7640 1 -- 172.21.15.169:0/1051952918 shutdown_connections 2023-10-26T01:37:31.626 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:31.623+0000 7f29771d7640 1 -- 172.21.15.169:0/1051952918 wait complete. 2023-10-26T01:37:31.628 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:31.624+0000 7f29771d7640 1 Processor -- start 2023-10-26T01:37:31.628 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:31.625+0000 7f29771d7640 1 -- start start 2023-10-26T01:37:31.628 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:31.625+0000 7f29771d7640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29700ac930 0x7f29701f1490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:31.628 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:31.625+0000 7f29771d7640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f29700cde00 0x7f29701e3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:31.629 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:31.626+0000 7f29771d7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f29701592d0 0x7f29701e3d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:31.629 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:31.626+0000 7f29771d7640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29701e42c0 con 0x7f29700ac930 2023-10-26T01:37:31.629 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:31.626+0000 7f29771d7640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29700c6ec0 con 0x7f29701592d0 2023-10-26T01:37:31.629 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:31.626+0000 7f29771d7640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29700c71a0 con 0x7f29700cde00 2023-10-26T01:37:31.629 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:31.626+0000 7f2974f73640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29700ac930 0x7f29701f1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:31.630 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:31.626+0000 7f2974f73640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29700ac930 0x7f29701f1490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:53858/0 (socket says 172.21.15.169:53858) 2023-10-26T01:37:31.630 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:31.626+0000 7f2974f73640 1 -- 172.21.15.169:0/885253904 learned_addr learned my addr 172.21.15.169:0/885253904 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:37:31.630 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:31.626+0000 7f2974f73640 1 -- 172.21.15.169:0/885253904 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f29700cde00 msgr2=0x7f29701e3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:31.630 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:31.626+0000 7f2974f73640 1 --2- 172.21.15.169:0/885253904 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f29700cde00 0x7f29701e3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:31.630 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:31.626+0000 7f2974f73640 1 -- 172.21.15.169:0/885253904 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f29701592d0 msgr2=0x7f29701e3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:31.630 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:31.626+0000 7f2974f73640 1 --2- 172.21.15.169:0/885253904 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f29701592d0 0x7f29701e3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:31.631 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:31.626+0000 7f2974f73640 1 -- 172.21.15.169:0/885253904 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2964009560 con 0x7f29700ac930 2023-10-26T01:37:31.631 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:31.626+0000 7f2974f73640 1 --2- 172.21.15.169:0/885253904 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29700ac930 0x7f29701f1490 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f2964009850 tx=0x7f296402db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:31.632 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:31.627+0000 7f296d7fa640 1 -- 172.21.15.169:0/885253904 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29640035f0 con 0x7f29700ac930 2023-10-26T01:37:31.632 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:31.627+0000 7f296d7fa640 1 -- 172.21.15.169:0/885253904 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2964003750 con 0x7f29700ac930 2023-10-26T01:37:31.632 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:31.627+0000 7f296d7fa640 1 -- 172.21.15.169:0/885253904 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2964003b00 con 0x7f29700ac930 2023-10-26T01:37:31.633 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:31.627+0000 7f29771d7640 1 -- 172.21.15.169:0/885253904 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29700c7480 con 0x7f29700ac930 2023-10-26T01:37:31.633 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:31.627+0000 7f29771d7640 1 -- 172.21.15.169:0/885253904 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29700c67d0 con 0x7f29700ac930 2023-10-26T01:37:31.633 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:31.628+0000 7f29771d7640 1 -- 172.21.15.169:0/885253904 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29700c79a0 con 0x7f29700ac930 2023-10-26T01:37:31.633 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:31.629+0000 7f296d7fa640 1 -- 172.21.15.169:0/885253904 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f2964003c60 con 0x7f29700ac930 2023-10-26T01:37:31.633 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:31.630+0000 7f296d7fa640 1 --2- 172.21.15.169:0/885253904 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f295406ae40 0x7f295406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:31.633 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:31.630+0000 7f296d7fa640 1 -- 172.21.15.169:0/885253904 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f29640abf60 con 0x7f29700ac930 2023-10-26T01:37:31.634 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:31.630+0000 7f296ffff640 1 --2- 172.21.15.169:0/885253904 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f295406ae40 0x7f295406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:31.634 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:31.632+0000 7f296ffff640 1 --2- 172.21.15.169:0/885253904 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f295406ae40 0x7f295406d300 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f297009b510 tx=0x7f295c007910 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:31.635 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:31.634+0000 7f296d7fa640 1 -- 172.21.15.169:0/885253904 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f29700c79a0 con 0x7f29700ac930 2023-10-26T01:37:31.874 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:31.872+0000 7f29771d7640 1 -- 172.21.15.169:0/885253904 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_barrier"]} v 0) v1 -- 0x7f29700c79a0 con 0x7f29700ac930 2023-10-26T01:37:32.507 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:32.505+0000 7f296d7fa640 1 -- 172.21.15.169:0/885253904 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_barrier"]}]=0 v353) v1 ==== 131+0+0 (secure 0 0 0) 0x7f29700c79a0 con 0x7f29700ac930 2023-10-26T01:37:32.511 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:32.509+0000 7f29771d7640 1 -- 172.21.15.169:0/885253904 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f295406ae40 msgr2=0x7f295406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:32.511 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:32.509+0000 7f29771d7640 1 --2- 172.21.15.169:0/885253904 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f295406ae40 0x7f295406d300 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f297009b510 tx=0x7f295c007910 comp rx=0 tx=0).stop 2023-10-26T01:37:32.512 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:32.509+0000 7f29771d7640 1 -- 172.21.15.169:0/885253904 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29700ac930 msgr2=0x7f29701f1490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:32.512 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:32.509+0000 7f29771d7640 1 --2- 172.21.15.169:0/885253904 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29700ac930 0x7f29701f1490 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f2964009850 tx=0x7f296402db10 comp rx=0 tx=0).stop 2023-10-26T01:37:32.512 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:32.509+0000 7f29771d7640 1 -- 172.21.15.169:0/885253904 shutdown_connections 2023-10-26T01:37:32.512 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:32.509+0000 7f29771d7640 1 --2- 172.21.15.169:0/885253904 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f295406ae40 0x7f295406d300 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:32.512 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:32.509+0000 7f29771d7640 1 --2- 172.21.15.169:0/885253904 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f29701592d0 0x7f29701e3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:32.512 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:32.509+0000 7f29771d7640 1 --2- 172.21.15.169:0/885253904 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f29700cde00 0x7f29701e3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:32.513 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:32.509+0000 7f29771d7640 1 --2- 172.21.15.169:0/885253904 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29700ac930 0x7f29701f1490 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:32.513 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:32.509+0000 7f29771d7640 1 -- 172.21.15.169:0/885253904 >> 172.21.15.169:0/885253904 conn(0x7f2970069930 msgr2=0x7f297014cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:32.513 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:32.509+0000 7f29771d7640 1 -- 172.21.15.169:0/885253904 shutdown_connections 2023-10-26T01:37:32.513 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:32.509+0000 7f29771d7640 1 -- 172.21.15.169:0/885253904 wait complete. 2023-10-26T01:37:32.531 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2023-10-26T01:37:32.746 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:32.743+0000 7fb1e955e640 1 -- 172.21.15.169:0/3368332353 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1e406ce00 msgr2=0x7fb1e4155360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:32.746 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:32.743+0000 7fb1e955e640 1 --2- 172.21.15.169:0/3368332353 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1e406ce00 0x7fb1e4155360 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7fb1e406c2a0 tx=0x7fb1d402bd10 comp rx=0 tx=0).stop 2023-10-26T01:37:32.746 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:32.744+0000 7fb1e955e640 1 -- 172.21.15.169:0/3368332353 shutdown_connections 2023-10-26T01:37:32.747 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:32.744+0000 7fb1e955e640 1 --2- 172.21.15.169:0/3368332353 >> v2:172.21.15.170:3300/0 conn(0x7fb1e415b4a0 0x7fb1e415b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:32.747 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:32.744+0000 7fb1e955e640 1 --2- 172.21.15.169:0/3368332353 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb1e40cd7e0 0x7fb1e40cdc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:32.747 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:32.744+0000 7fb1e955e640 1 --2- 172.21.15.169:0/3368332353 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1e406ce00 0x7fb1e4155360 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:32.747 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:32.744+0000 7fb1e955e640 1 -- 172.21.15.169:0/3368332353 >> 172.21.15.169:0/3368332353 conn(0x7fb1e4069930 msgr2=0x7fb1e4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:32.748 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:32.744+0000 7fb1e955e640 1 -- 172.21.15.169:0/3368332353 shutdown_connections 2023-10-26T01:37:32.748 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:32.744+0000 7fb1e955e640 1 -- 172.21.15.169:0/3368332353 wait complete. 2023-10-26T01:37:32.752 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:32.745+0000 7fb1e955e640 1 Processor -- start 2023-10-26T01:37:32.752 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:32.745+0000 7fb1e955e640 1 -- start start 2023-10-26T01:37:32.753 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:32.746+0000 7fb1e955e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1e406ce00 0x7fb1e41e1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:32.753 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:32.746+0000 7fb1e955e640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb1e40cd7e0 0x7fb1e4099670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:32.753 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:32.746+0000 7fb1e955e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1e415b4a0 0x7fb1e4096550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:32.754 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:32.746+0000 7fb1e955e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e409acd0 con 0x7fb1e415b4a0 2023-10-26T01:37:32.754 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:32.746+0000 7fb1e955e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e409a0c0 con 0x7fb1e406ce00 2023-10-26T01:37:32.754 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:32.746+0000 7fb1e955e640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e409a3a0 con 0x7fb1e40cd7e0 2023-10-26T01:37:32.755 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:32.747+0000 7fb1e37fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1e415b4a0 0x7fb1e4096550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:32.755 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:32.747+0000 7fb1e37fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1e415b4a0 0x7fb1e4096550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:53868/0 (socket says 172.21.15.169:53868) 2023-10-26T01:37:32.756 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:32.747+0000 7fb1e37fe640 1 -- 172.21.15.169:0/1104750557 learned_addr learned my addr 172.21.15.169:0/1104750557 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:37:32.756 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:32.747+0000 7fb1e27fc640 1 --2- 172.21.15.169:0/1104750557 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb1e40cd7e0 0x7fb1e4099670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:32.756 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:32.747+0000 7fb1e2ffd640 1 --2- 172.21.15.169:0/1104750557 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1e406ce00 0x7fb1e41e1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:32.757 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:32.747+0000 7fb1e37fe640 1 -- 172.21.15.169:0/1104750557 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb1e40cd7e0 msgr2=0x7fb1e4099670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:32.757 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:32.747+0000 7fb1e37fe640 1 --2- 172.21.15.169:0/1104750557 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb1e40cd7e0 0x7fb1e4099670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:32.757 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:32.747+0000 7fb1e37fe640 1 -- 172.21.15.169:0/1104750557 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1e406ce00 msgr2=0x7fb1e41e1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:32.758 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:32.747+0000 7fb1e37fe640 1 --2- 172.21.15.169:0/1104750557 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1e406ce00 0x7fb1e41e1b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:32.758 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:32.747+0000 7fb1e37fe640 1 -- 172.21.15.169:0/1104750557 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1d4009560 con 0x7fb1e415b4a0 2023-10-26T01:37:32.759 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:32.747+0000 7fb1e2ffd640 1 --2- 172.21.15.169:0/1104750557 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1e406ce00 0x7fb1e41e1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:37:32.759 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:32.747+0000 7fb1e37fe640 1 --2- 172.21.15.169:0/1104750557 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1e415b4a0 0x7fb1e4096550 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc00cab0 tx=0x7fb1dc00cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:32.759 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:32.748+0000 7fb1cb7fe640 1 -- 172.21.15.169:0/1104750557 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1dc007d90 con 0x7fb1e415b4a0 2023-10-26T01:37:32.760 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:32.748+0000 7fb1cb7fe640 1 -- 172.21.15.169:0/1104750557 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb1dc00f450 con 0x7fb1e415b4a0 2023-10-26T01:37:32.760 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:32.748+0000 7fb1cb7fe640 1 -- 172.21.15.169:0/1104750557 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1dc00f840 con 0x7fb1e415b4a0 2023-10-26T01:37:32.760 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:32.748+0000 7fb1e955e640 1 -- 172.21.15.169:0/1104750557 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1e409a680 con 0x7fb1e415b4a0 2023-10-26T01:37:32.761 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:32.748+0000 7fb1e955e640 1 -- 172.21.15.169:0/1104750557 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1e4154010 con 0x7fb1e415b4a0 2023-10-26T01:37:32.761 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:32.749+0000 7fb1e955e640 1 -- 172.21.15.169:0/1104750557 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1e409ae80 con 0x7fb1e415b4a0 2023-10-26T01:37:32.761 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:32.750+0000 7fb1cb7fe640 1 -- 172.21.15.169:0/1104750557 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fb1dc00f9a0 con 0x7fb1e415b4a0 2023-10-26T01:37:32.763 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:32.751+0000 7fb1cb7fe640 1 --2- 172.21.15.169:0/1104750557 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb1d006ae70 0x7fb1d006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:32.763 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:32.751+0000 7fb1cb7fe640 1 -- 172.21.15.169:0/1104750557 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fb1dc089d80 con 0x7fb1e415b4a0 2023-10-26T01:37:32.763 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:32.752+0000 7fb1e2ffd640 1 --2- 172.21.15.169:0/1104750557 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb1d006ae70 0x7fb1d006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:32.764 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:32.753+0000 7fb1e2ffd640 1 --2- 172.21.15.169:0/1104750557 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb1d006ae70 0x7fb1d006d330 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fb1d4035000 tx=0x7fb1d4004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:32.764 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:32.755+0000 7fb1cb7fe640 1 -- 172.21.15.169:0/1104750557 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb1e409ae80 con 0x7fb1e415b4a0 2023-10-26T01:37:32.946 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:32.944+0000 7fb1e955e640 1 -- 172.21.15.169:0/1104750557 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "config dump"} v 0) v1 -- 0x7fb1e404db10 con 0x7fb1e415b4a0 2023-10-26T01:37:32.947 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:32.945+0000 7fb1cb7fe640 1 -- 172.21.15.169:0/1104750557 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v28) v1 ==== 59+0+1116 (secure 0 0 0) 0x7fb1dc01d080 con 0x7fb1e415b4a0 2023-10-26T01:37:32.947 INFO:teuthology.orchestra.run.smithi169.stdout:WHO MASK LEVEL OPTION VALUE RO 2023-10-26T01:37:32.948 INFO:teuthology.orchestra.run.smithi169.stdout:mon advanced mon_op_complaint_time 120 2023-10-26T01:37:32.948 INFO:teuthology.orchestra.run.smithi169.stdout:osd advanced osd_op_complaint_time 180.000000 2023-10-26T01:37:32.948 INFO:teuthology.orchestra.run.smithi169.stdout:mds advanced debug_mds 20/20 2023-10-26T01:37:32.948 INFO:teuthology.orchestra.run.smithi169.stdout:mds advanced debug_mds_balancer 20/20 2023-10-26T01:37:32.949 INFO:teuthology.orchestra.run.smithi169.stdout:mds advanced debug_ms 1/1 2023-10-26T01:37:32.949 INFO:teuthology.orchestra.run.smithi169.stdout:mds dev mds_debug_frag true 2023-10-26T01:37:32.949 INFO:teuthology.orchestra.run.smithi169.stdout:mds dev mds_debug_scatterstat true 2023-10-26T01:37:32.949 INFO:teuthology.orchestra.run.smithi169.stdout:mds advanced mds_op_complaint_time 180.000000 2023-10-26T01:37:32.949 INFO:teuthology.orchestra.run.smithi169.stdout:mds dev mds_verify_scatter true 2023-10-26T01:37:32.950 INFO:teuthology.orchestra.run.smithi169.stdout:mds advanced osd_op_complaint_time 180.000000 2023-10-26T01:37:32.950 INFO:teuthology.orchestra.run.smithi169.stdout:mds advanced rados_mon_op_timeout 900 2023-10-26T01:37:32.950 INFO:teuthology.orchestra.run.smithi169.stdout:mds advanced rados_osd_op_timeout 900 2023-10-26T01:37:32.950 INFO:teuthology.orchestra.run.smithi169.stdout:client advanced client_mount_timeout 600 2023-10-26T01:37:32.950 INFO:teuthology.orchestra.run.smithi169.stdout:client advanced debug_client 20/20 2023-10-26T01:37:32.951 INFO:teuthology.orchestra.run.smithi169.stdout:client advanced debug_ms 1/1 2023-10-26T01:37:32.951 INFO:teuthology.orchestra.run.smithi169.stdout:client advanced rados_mon_op_timeout 900 2023-10-26T01:37:32.951 INFO:teuthology.orchestra.run.smithi169.stdout:client advanced rados_osd_op_timeout 900 2023-10-26T01:37:32.952 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:32.948+0000 7fb1e955e640 1 -- 172.21.15.169:0/1104750557 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb1d006ae70 msgr2=0x7fb1d006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:32.952 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:32.948+0000 7fb1e955e640 1 --2- 172.21.15.169:0/1104750557 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb1d006ae70 0x7fb1d006d330 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fb1d4035000 tx=0x7fb1d4004730 comp rx=0 tx=0).stop 2023-10-26T01:37:32.953 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:32.948+0000 7fb1e955e640 1 -- 172.21.15.169:0/1104750557 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1e415b4a0 msgr2=0x7fb1e4096550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:32.953 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:32.948+0000 7fb1e955e640 1 --2- 172.21.15.169:0/1104750557 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1e415b4a0 0x7fb1e4096550 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc00cab0 tx=0x7fb1dc00cf70 comp rx=0 tx=0).stop 2023-10-26T01:37:32.953 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:32.948+0000 7fb1e955e640 1 -- 172.21.15.169:0/1104750557 shutdown_connections 2023-10-26T01:37:32.953 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:32.948+0000 7fb1e955e640 1 --2- 172.21.15.169:0/1104750557 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb1d006ae70 0x7fb1d006d330 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:32.954 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:32.948+0000 7fb1e955e640 1 --2- 172.21.15.169:0/1104750557 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1e415b4a0 0x7fb1e4096550 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:32.954 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:32.948+0000 7fb1e955e640 1 --2- 172.21.15.169:0/1104750557 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb1e40cd7e0 0x7fb1e4099670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:32.954 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:32.948+0000 7fb1e955e640 1 --2- 172.21.15.169:0/1104750557 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1e406ce00 0x7fb1e41e1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:32.954 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:32.948+0000 7fb1e955e640 1 -- 172.21.15.169:0/1104750557 >> 172.21.15.169:0/1104750557 conn(0x7fb1e4069930 msgr2=0x7fb1e41597b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:32.955 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:32.948+0000 7fb1e955e640 1 -- 172.21.15.169:0/1104750557 shutdown_connections 2023-10-26T01:37:32.955 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:32.949+0000 7fb1e955e640 1 -- 172.21.15.169:0/1104750557 wait complete. 2023-10-26T01:37:32.969 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-26T01:37:33.181 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:33.180+0000 7f5f0138a640 1 -- 172.21.15.169:0/2340679506 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5efc0b0c80 msgr2=0x7f5efc0cd710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:33.182 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:33.180+0000 7f5f0138a640 1 --2- 172.21.15.169:0/2340679506 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5efc0b0c80 0x7f5efc0cd710 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f5efc0b2670 tx=0x7f5ef402bc50 comp rx=0 tx=0).stop 2023-10-26T01:37:33.182 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:33.180+0000 7f5f0138a640 1 -- 172.21.15.169:0/2340679506 shutdown_connections 2023-10-26T01:37:33.182 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:33.180+0000 7f5f0138a640 1 --2- 172.21.15.169:0/2340679506 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5efc0b0c80 0x7f5efc0cd710 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:33.182 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:33.180+0000 7f5f0138a640 1 --2- 172.21.15.169:0/2340679506 >> v2:172.21.15.170:3300/0 conn(0x7f5efc1597c0 0x7f5efc0ac200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:33.183 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:33.180+0000 7f5f0138a640 1 -- 172.21.15.169:0/2340679506 >> 172.21.15.169:0/2340679506 conn(0x7f5efc069930 msgr2=0x7f5efc069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:33.183 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:33.180+0000 7f5f0138a640 1 -- 172.21.15.169:0/2340679506 shutdown_connections 2023-10-26T01:37:33.183 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:33.180+0000 7f5f0138a640 1 -- 172.21.15.169:0/2340679506 wait complete. 2023-10-26T01:37:33.184 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:33.181+0000 7f5f0138a640 1 Processor -- start 2023-10-26T01:37:33.184 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:33.181+0000 7f5f0138a640 1 -- start start 2023-10-26T01:37:33.184 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:33.182+0000 7f5f0138a640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5efc0b0c80 0x7f5efc0c8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:33.185 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:33.182+0000 7f5f0138a640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5efc1593e0 0x7f5efc153c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:33.185 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:33.182+0000 7f5f0138a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5efc1597c0 0x7f5efc1543a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:33.185 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:33.182+0000 7f5f0138a640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5efc0c87b0 con 0x7f5efc1593e0 2023-10-26T01:37:33.185 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:33.182+0000 7f5f0138a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5efc154150 con 0x7f5efc1597c0 2023-10-26T01:37:33.185 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:33.182+0000 7f5f0138a640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5efc06f180 con 0x7f5efc0b0c80 2023-10-26T01:37:33.185 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:33.183+0000 7f5efa7fc640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5efc1593e0 0x7f5efc153c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:33.186 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:33.183+0000 7f5efa7fc640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5efc1593e0 0x7f5efc153c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:53884/0 (socket says 172.21.15.169:53884) 2023-10-26T01:37:33.186 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:33.183+0000 7f5efa7fc640 1 -- 172.21.15.169:0/1528086365 learned_addr learned my addr 172.21.15.169:0/1528086365 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:37:33.187 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:33.183+0000 7f5efa7fc640 1 -- 172.21.15.169:0/1528086365 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5efc0b0c80 msgr2=0x7f5efc0c8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:33.187 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:33.183+0000 7f5efa7fc640 1 --2- 172.21.15.169:0/1528086365 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5efc0b0c80 0x7f5efc0c8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:33.187 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:33.183+0000 7f5efa7fc640 1 -- 172.21.15.169:0/1528086365 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5efc1597c0 msgr2=0x7f5efc1543a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:33.188 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:33.183+0000 7f5efa7fc640 1 --2- 172.21.15.169:0/1528086365 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5efc1597c0 0x7f5efc1543a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:33.188 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:33.183+0000 7f5efa7fc640 1 -- 172.21.15.169:0/1528086365 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ef4009490 con 0x7f5efc1593e0 2023-10-26T01:37:33.188 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:33.183+0000 7f5efa7fc640 1 --2- 172.21.15.169:0/1528086365 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5efc1593e0 0x7f5efc153c10 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f5ee400d740 tx=0x7f5ee400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:33.188 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:33.183+0000 7f5ef37fe640 1 -- 172.21.15.169:0/1528086365 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee4009c30 con 0x7f5efc1593e0 2023-10-26T01:37:33.188 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:33.184+0000 7f5ef37fe640 1 -- 172.21.15.169:0/1528086365 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5ee4009e80 con 0x7f5efc1593e0 2023-10-26T01:37:33.189 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:33.184+0000 7f5ef37fe640 1 -- 172.21.15.169:0/1528086365 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee40106c0 con 0x7f5efc1593e0 2023-10-26T01:37:33.189 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:33.184+0000 7f5f0138a640 1 -- 172.21.15.169:0/1528086365 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5efc06f460 con 0x7f5efc1593e0 2023-10-26T01:37:33.189 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:33.184+0000 7f5f0138a640 1 -- 172.21.15.169:0/1528086365 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5efc0c8440 con 0x7f5efc1593e0 2023-10-26T01:37:33.189 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:33.185+0000 7f5f0138a640 1 -- 172.21.15.169:0/1528086365 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5efc04db10 con 0x7f5efc1593e0 2023-10-26T01:37:33.189 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:33.186+0000 7f5ef37fe640 1 -- 172.21.15.169:0/1528086365 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f5ee4010820 con 0x7f5efc1593e0 2023-10-26T01:37:33.190 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:33.186+0000 7f5ef37fe640 1 --2- 172.21.15.169:0/1528086365 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5edc06ae40 0x7f5edc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:33.190 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:33.187+0000 7f5ef37fe640 1 -- 172.21.15.169:0/1528086365 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f5ee40898b0 con 0x7f5efc1593e0 2023-10-26T01:37:33.190 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:33.187+0000 7f5efaffd640 1 --2- 172.21.15.169:0/1528086365 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5edc06ae40 0x7f5edc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:33.190 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:33.188+0000 7f5efaffd640 1 --2- 172.21.15.169:0/1528086365 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5edc06ae40 0x7f5edc06d300 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f5eec001700 tx=0x7f5eec001470 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:33.192 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:33.191+0000 7f5ef37fe640 1 -- 172.21.15.169:0/1528086365 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5ee4054bc0 con 0x7f5efc1593e0 2023-10-26T01:37:33.366 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:33.364+0000 7f5f0138a640 1 -- 172.21.15.169:0/1528086365 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f5efc0b2170 con 0x7f5efc1593e0 2023-10-26T01:37:33.373 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:33.372+0000 7f5ef37fe640 1 -- 172.21.15.169:0/1528086365 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v29)=0 v29) v1 ==== 122+0+0 (secure 0 0 0) 0x7f5ee40210c0 con 0x7f5efc1593e0 2023-10-26T01:37:33.377 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:33.375+0000 7f5f0138a640 1 -- 172.21.15.169:0/1528086365 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5edc06ae40 msgr2=0x7f5edc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:33.377 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:33.375+0000 7f5f0138a640 1 --2- 172.21.15.169:0/1528086365 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5edc06ae40 0x7f5edc06d300 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f5eec001700 tx=0x7f5eec001470 comp rx=0 tx=0).stop 2023-10-26T01:37:33.377 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:33.375+0000 7f5f0138a640 1 -- 172.21.15.169:0/1528086365 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5efc1593e0 msgr2=0x7f5efc153c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:33.377 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:33.375+0000 7f5f0138a640 1 --2- 172.21.15.169:0/1528086365 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5efc1593e0 0x7f5efc153c10 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f5ee400d740 tx=0x7f5ee400dc00 comp rx=0 tx=0).stop 2023-10-26T01:37:33.378 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:33.375+0000 7f5f0138a640 1 -- 172.21.15.169:0/1528086365 shutdown_connections 2023-10-26T01:37:33.378 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:33.375+0000 7f5f0138a640 1 --2- 172.21.15.169:0/1528086365 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5edc06ae40 0x7f5edc06d300 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:33.378 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:33.375+0000 7f5f0138a640 1 --2- 172.21.15.169:0/1528086365 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5efc1593e0 0x7f5efc153c10 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:33.378 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:33.375+0000 7f5f0138a640 1 --2- 172.21.15.169:0/1528086365 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5efc1597c0 0x7f5efc1543a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:33.378 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:33.375+0000 7f5f0138a640 1 --2- 172.21.15.169:0/1528086365 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5efc0b0c80 0x7f5efc0c8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:33.379 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:33.375+0000 7f5f0138a640 1 -- 172.21.15.169:0/1528086365 >> 172.21.15.169:0/1528086365 conn(0x7f5efc069930 msgr2=0x7f5efc15bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:33.379 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:33.375+0000 7f5f0138a640 1 -- 172.21.15.169:0/1528086365 shutdown_connections 2023-10-26T01:37:33.379 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:33.375+0000 7f5f0138a640 1 -- 172.21.15.169:0/1528086365 wait complete. 2023-10-26T01:37:33.394 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-26T01:37:33.394 DEBUG:teuthology.orchestra.run.smithi169:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-26T01:37:33.502 DEBUG:teuthology.orchestra.run.smithi170:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-26T01:37:33.625 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-26T01:37:33.625 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:37:33.626 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:37:33.687 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054301,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-26T01:37:33.687 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-26T01:37:33.687 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:37:33.748 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054303,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-26T01:37:33.748 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:37:33.748 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:37:33.809 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054301,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-26T01:37:33.809 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-26T01:37:33.809 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:37:33.868 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054303,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-26T01:37:33.869 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-26T01:37:33.869 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T01:37:34.101 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:34.099+0000 7ff30a2f2640 1 -- 172.21.15.169:0/3456758661 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2f0003970 msgr2=0x7ff2f0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:34.101 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:34.099+0000 7ff30a2f2640 1 --2- 172.21.15.169:0/3456758661 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2f0003970 0x7ff2f0003d90 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7ff2f4009b20 tx=0x7ff2f00057f0 comp rx=0 tx=0).stop 2023-10-26T01:37:34.101 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:34.099+0000 7ff30888f640 1 -- 172.21.15.169:0/3456758661 reap_dead start 2023-10-26T01:37:34.101 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:34.099+0000 7ff30a2f2640 1 -- 172.21.15.169:0/3456758661 shutdown_connections 2023-10-26T01:37:34.102 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:34.099+0000 7ff30a2f2640 1 -- 172.21.15.169:0/3456758661 >> 172.21.15.169:0/3456758661 conn(0x7ff3040658b0 msgr2=0x7ff304065cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:34.102 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:34.099+0000 7ff30a2f2640 1 -- 172.21.15.169:0/3456758661 shutdown_connections 2023-10-26T01:37:34.102 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:34.099+0000 7ff30a2f2640 1 -- 172.21.15.169:0/3456758661 wait complete. 2023-10-26T01:37:34.103 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:34.101+0000 7ff30a2f2640 1 Processor -- start 2023-10-26T01:37:34.104 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:34.101+0000 7ff30a2f2640 1 -- start start 2023-10-26T01:37:34.104 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:34.102+0000 7ff30a2f2640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff2f0003970 0x7ff3040c89d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:34.104 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:34.102+0000 7ff30a2f2640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff30415ea70 0x7ff304148670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:34.104 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:34.102+0000 7ff30a2f2640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff304148bb0 0x7ff3041f53a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:34.105 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:34.102+0000 7ff30a2f2640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff304149030 con 0x7ff30415ea70 2023-10-26T01:37:34.105 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:34.102+0000 7ff30a2f2640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff30415dd40 con 0x7ff2f0003970 2023-10-26T01:37:34.105 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:34.102+0000 7ff30a2f2640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3041f6110 con 0x7ff304148bb0 2023-10-26T01:37:34.105 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:34.102+0000 7ff2fbfff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff30415ea70 0x7ff304148670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:34.105 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:34.102+0000 7ff2fbfff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff30415ea70 0x7ff304148670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:46976/0 (socket says 172.21.15.169:46976) 2023-10-26T01:37:34.105 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:34.102+0000 7ff2fbfff640 1 -- 172.21.15.169:0/2922982022 learned_addr learned my addr 172.21.15.169:0/2922982022 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:37:34.107 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:34.102+0000 7ff30888f640 1 --2- 172.21.15.169:0/2922982022 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff304148bb0 0x7ff3041f53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:34.107 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:34.102+0000 7ff303fff640 1 --2- 172.21.15.169:0/2922982022 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff2f0003970 0x7ff3040c89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:34.107 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:34.102+0000 7ff2fbfff640 1 -- 172.21.15.169:0/2922982022 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff304148bb0 msgr2=0x7ff3041f53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:34.108 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:34.102+0000 7ff2fbfff640 1 --2- 172.21.15.169:0/2922982022 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff304148bb0 0x7ff3041f53a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:34.108 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:34.102+0000 7ff2fbfff640 1 -- 172.21.15.169:0/2922982022 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff2f0003970 msgr2=0x7ff3040c89d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:34.108 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:34.102+0000 7ff2fbfff640 1 --2- 172.21.15.169:0/2922982022 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff2f0003970 0x7ff3040c89d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:34.108 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:34.102+0000 7ff2fbfff640 1 -- 172.21.15.169:0/2922982022 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff30415df90 con 0x7ff30415ea70 2023-10-26T01:37:34.108 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:34.102+0000 7ff303fff640 1 --2- 172.21.15.169:0/2922982022 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff2f0003970 0x7ff3040c89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:37:34.109 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:34.103+0000 7ff2fbfff640 1 --2- 172.21.15.169:0/2922982022 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff30415ea70 0x7ff304148670 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec00c680 tx=0x7ff2ec00cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:34.109 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:34.103+0000 7ff3017fa640 1 -- 172.21.15.169:0/2922982022 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2ec005270 con 0x7ff30415ea70 2023-10-26T01:37:34.109 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:34.103+0000 7ff3017fa640 1 -- 172.21.15.169:0/2922982022 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff2ec00a640 con 0x7ff30415ea70 2023-10-26T01:37:34.109 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:34.103+0000 7ff3017fa640 1 -- 172.21.15.169:0/2922982022 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2ec00aa50 con 0x7ff30415ea70 2023-10-26T01:37:34.109 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:34.103+0000 7ff30a2f2640 1 -- 172.21.15.169:0/2922982022 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff30415e270 con 0x7ff30415ea70 2023-10-26T01:37:34.110 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:34.103+0000 7ff30a2f2640 1 -- 172.21.15.169:0/2922982022 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3040c8f10 con 0x7ff30415ea70 2023-10-26T01:37:34.110 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:34.104+0000 7ff30a2f2640 1 -- 172.21.15.169:0/2922982022 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff30415e790 con 0x7ff30415ea70 2023-10-26T01:37:34.110 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:34.105+0000 7ff3017fa640 1 -- 172.21.15.169:0/2922982022 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ff2ec00abb0 con 0x7ff30415ea70 2023-10-26T01:37:34.111 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:34.106+0000 7ff3017fa640 1 --2- 172.21.15.169:0/2922982022 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff2e406ae70 0x7ff2e406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:34.111 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:34.106+0000 7ff3017fa640 1 -- 172.21.15.169:0/2922982022 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 4943+0+0 (secure 0 0 0) 0x7ff2ec088d90 con 0x7ff30415ea70 2023-10-26T01:37:34.111 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:34.107+0000 7ff303fff640 1 --2- 172.21.15.169:0/2922982022 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff2e406ae70 0x7ff2e406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:34.111 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:34.108+0000 7ff303fff640 1 --2- 172.21.15.169:0/2922982022 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff2e406ae70 0x7ff2e406d330 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7ff2f4036000 tx=0x7ff2f4035040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:34.113 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:34.111+0000 7ff3017fa640 1 -- 172.21.15.169:0/2922982022 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff30415e790 con 0x7ff30415ea70 2023-10-26T01:37:34.342 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:34.340+0000 7ff30a2f2640 1 -- 172.21.15.169:0/2922982022 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff30415e790 con 0x7ff30415ea70 2023-10-26T01:37:34.343 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:34.342+0000 7ff3017fa640 1 -- 172.21.15.169:0/2922982022 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) v1 ==== 94+0+3711 (secure 0 0 0) 0x7ff30415e790 con 0x7ff30415ea70 2023-10-26T01:37:34.344 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:37:34.344 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":52,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5240,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/3870332417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":3870332417},{"type":"v1","addr":"172.21.15.170:6837","nonce":3870332417}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45},{"gid":5497,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/2533771698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2533771698},{"type":"v1","addr":"172.21.15.170:6835","nonce":2533771698}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45},{"gid":6885,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/2117946713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2117946713},{"type":"v1","addr":"172.21.15.169:6834","nonce":2117946713}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45},{"gid":7251,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1420338353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":1420338353},{"type":"v1","addr":"172.21.15.169:6835","nonce":1420338353}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52}],"filesystems":[]} 2023-10-26T01:37:34.344 INFO:teuthology.orchestra.run.smithi169.stderr:dumped fsmap epoch 52 2023-10-26T01:37:34.346 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:34.344+0000 7ff30a2f2640 1 -- 172.21.15.169:0/2922982022 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff2e406ae70 msgr2=0x7ff2e406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:34.346 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:34.344+0000 7ff30a2f2640 1 --2- 172.21.15.169:0/2922982022 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff2e406ae70 0x7ff2e406d330 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7ff2f4036000 tx=0x7ff2f4035040 comp rx=0 tx=0).stop 2023-10-26T01:37:34.346 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:34.344+0000 7ff30a2f2640 1 -- 172.21.15.169:0/2922982022 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff30415ea70 msgr2=0x7ff304148670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:34.347 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:34.344+0000 7ff30a2f2640 1 --2- 172.21.15.169:0/2922982022 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff30415ea70 0x7ff304148670 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec00c680 tx=0x7ff2ec00cb40 comp rx=0 tx=0).stop 2023-10-26T01:37:34.347 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:34.344+0000 7ff30a2f2640 1 -- 172.21.15.169:0/2922982022 shutdown_connections 2023-10-26T01:37:34.347 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:34.344+0000 7ff30a2f2640 1 --2- 172.21.15.169:0/2922982022 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff2e406ae70 0x7ff2e406d330 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:34.347 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:34.344+0000 7ff30a2f2640 1 --2- 172.21.15.169:0/2922982022 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff30415ea70 0x7ff304148670 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:34.347 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:34.344+0000 7ff30a2f2640 1 --2- 172.21.15.169:0/2922982022 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff304148bb0 0x7ff3041f53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:34.347 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:34.344+0000 7ff30a2f2640 1 --2- 172.21.15.169:0/2922982022 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff2f0003970 0x7ff3040c89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:34.348 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:34.344+0000 7ff30a2f2640 1 -- 172.21.15.169:0/2922982022 >> 172.21.15.169:0/2922982022 conn(0x7ff3040658b0 msgr2=0x7ff304159cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:34.348 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:34.345+0000 7ff30a2f2640 1 -- 172.21.15.169:0/2922982022 shutdown_connections 2023-10-26T01:37:34.348 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:34.345+0000 7ff30a2f2640 1 -- 172.21.15.169:0/2922982022 wait complete. 2023-10-26T01:37:34.367 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-26T01:37:34.367 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-26T01:37:34.368 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-26T01:37:34.368 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T01:37:34.368 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-26T01:37:34.368 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-26T01:37:34.369 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-26T01:37:34.369 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T01:37:34.369 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-26T01:37:34.370 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-26T01:37:34.370 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-26T01:37:34.370 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T01:37:34.371 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-26T01:37:34.371 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-26T01:37:34.371 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-26T01:37:34.371 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T01:37:34.386 INFO:tasks.ceph.mds.a:Stopped 2023-10-26T01:37:34.386 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-26T01:37:34.387 INFO:tasks.ceph.mds.c:Stopped 2023-10-26T01:37:34.387 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-26T01:37:34.389 INFO:tasks.ceph.mds.d:Stopped 2023-10-26T01:37:34.390 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-26T01:37:34.399 INFO:tasks.ceph.mds.b:Stopped 2023-10-26T01:37:34.399 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-26T01:37:34.435 INFO:tasks.ceph.mds.a:Started 2023-10-26T01:37:34.436 DEBUG:teuthology.parallel:result is None 2023-10-26T01:37:34.442 INFO:tasks.ceph.mds.c:Started 2023-10-26T01:37:34.443 DEBUG:teuthology.parallel:result is None 2023-10-26T01:37:34.447 INFO:tasks.ceph.mds.d:Started 2023-10-26T01:37:34.448 DEBUG:teuthology.parallel:result is None 2023-10-26T01:37:34.455 INFO:tasks.ceph.mds.b:Started 2023-10-26T01:37:34.455 DEBUG:teuthology.parallel:result is None 2023-10-26T01:37:34.455 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-26T01:37:34.593 INFO:tasks.ceph.mds.a.smithi169.stdout:starting mds.a at 2023-10-26T01:37:34.599 INFO:tasks.ceph.mds.c.smithi169.stdout:starting mds.c at 2023-10-26T01:37:34.613 INFO:tasks.ceph.mds.d.smithi170.stdout:starting mds.d at 2023-10-26T01:37:34.615 INFO:tasks.ceph.mds.b.smithi170.stdout:starting mds.b at 2023-10-26T01:37:34.653 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:34.651+0000 7f929e5b6640 1 -- 172.21.15.169:0/1563926716 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f92981593e0 msgr2=0x7f92980ac200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:34.653 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:34.651+0000 7f929e5b6640 1 --2- 172.21.15.169:0/1563926716 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f92981593e0 0x7f92980ac200 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f9288009830 tx=0x7f928802bdf0 comp rx=0 tx=0).stop 2023-10-26T01:37:34.653 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:34.651+0000 7f929e5b6640 1 -- 172.21.15.169:0/1563926716 shutdown_connections 2023-10-26T01:37:34.653 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:34.651+0000 7f929e5b6640 1 --2- 172.21.15.169:0/1563926716 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f92981593e0 0x7f92980ac200 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:34.654 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:34.651+0000 7f929e5b6640 1 -- 172.21.15.169:0/1563926716 >> 172.21.15.169:0/1563926716 conn(0x7f9298069930 msgr2=0x7f9298069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:34.654 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:34.651+0000 7f929e5b6640 1 -- 172.21.15.169:0/1563926716 shutdown_connections 2023-10-26T01:37:34.654 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:34.651+0000 7f929e5b6640 1 -- 172.21.15.169:0/1563926716 wait complete. 2023-10-26T01:37:34.655 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:34.652+0000 7f929e5b6640 1 Processor -- start 2023-10-26T01:37:34.655 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:34.653+0000 7f929e5b6640 1 -- start start 2023-10-26T01:37:34.656 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:34.653+0000 7f929e5b6640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f92980b0c80 0x7f92981f25d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:34.657 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:34.654+0000 7f929e5b6640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f92981593e0 0x7f92981e5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:34.657 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:34.654+0000 7f929e5b6640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f92981597c0 0x7f929815a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:34.657 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:34.654+0000 7f929e5b6640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92981f2140 con 0x7f92981597c0 2023-10-26T01:37:34.657 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:34.654+0000 7f929e5b6640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92980c69a0 con 0x7f92981593e0 2023-10-26T01:37:34.658 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:34.654+0000 7f929e5b6640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92980c6c80 con 0x7f92980b0c80 2023-10-26T01:37:34.658 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:34.654+0000 7f929cb53640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f92981597c0 0x7f929815a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:34.658 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:34.654+0000 7f929cb53640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f92981597c0 0x7f929815a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:47002/0 (socket says 172.21.15.169:47002) 2023-10-26T01:37:34.658 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:34.654+0000 7f929cb53640 1 -- 172.21.15.169:0/1189975927 learned_addr learned my addr 172.21.15.169:0/1189975927 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:37:34.659 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:34.654+0000 7f9297fff640 1 --2- 172.21.15.169:0/1189975927 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f92980b0c80 0x7f92981f25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:34.659 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:34.654+0000 7f928ffff640 1 --2- 172.21.15.169:0/1189975927 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f92981593e0 0x7f92981e5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:34.659 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:34.654+0000 7f929cb53640 1 -- 172.21.15.169:0/1189975927 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f92980b0c80 msgr2=0x7f92981f25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:34.659 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:34.654+0000 7f929cb53640 1 --2- 172.21.15.169:0/1189975927 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f92980b0c80 0x7f92981f25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:34.659 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:34.654+0000 7f929cb53640 1 -- 172.21.15.169:0/1189975927 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f92981593e0 msgr2=0x7f92981e5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:34.660 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:34.654+0000 7f929cb53640 1 --2- 172.21.15.169:0/1189975927 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f92981593e0 0x7f92981e5180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:34.660 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:34.654+0000 7f929cb53640 1 -- 172.21.15.169:0/1189975927 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9288009490 con 0x7f92981597c0 2023-10-26T01:37:34.660 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:34.655+0000 7f929cb53640 1 --2- 172.21.15.169:0/1189975927 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f92981597c0 0x7f929815a310 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f929000c7a0 tx=0x7f929000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:34.660 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:34.655+0000 7f92957fa640 1 -- 172.21.15.169:0/1189975927 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9290003b70 con 0x7f92981597c0 2023-10-26T01:37:34.660 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:34.655+0000 7f92957fa640 1 -- 172.21.15.169:0/1189975927 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9290003cf0 con 0x7f92981597c0 2023-10-26T01:37:34.661 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:34.655+0000 7f92957fa640 1 -- 172.21.15.169:0/1189975927 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f929000aab0 con 0x7f92981597c0 2023-10-26T01:37:34.661 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:34.655+0000 7f929e5b6640 1 -- 172.21.15.169:0/1189975927 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92980c6f60 con 0x7f92981597c0 2023-10-26T01:37:34.661 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:34.655+0000 7f929e5b6640 1 -- 172.21.15.169:0/1189975927 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92981f2b10 con 0x7f92981597c0 2023-10-26T01:37:34.662 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:34.656+0000 7f929e5b6640 1 -- 172.21.15.169:0/1189975927 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92981f22f0 con 0x7f92981597c0 2023-10-26T01:37:34.662 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:34.657+0000 7f92957fa640 1 -- 172.21.15.169:0/1189975927 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f929000ac10 con 0x7f92981597c0 2023-10-26T01:37:34.662 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:34.658+0000 7f92957fa640 1 --2- 172.21.15.169:0/1189975927 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f927806ae40 0x7f927806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:34.663 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:34.658+0000 7f92957fa640 1 -- 172.21.15.169:0/1189975927 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f92900891b0 con 0x7f92981597c0 2023-10-26T01:37:34.663 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:34.658+0000 7f9297fff640 1 --2- 172.21.15.169:0/1189975927 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f927806ae40 0x7f927806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:34.663 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:34.659+0000 7f9297fff640 1 --2- 172.21.15.169:0/1189975927 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f927806ae40 0x7f927806d300 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f9288036000 tx=0x7f9288004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:34.664 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:34.662+0000 7f92957fa640 1 -- 172.21.15.169:0/1189975927 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f92981f22f0 con 0x7f92981597c0 2023-10-26T01:37:34.864 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:34.862+0000 7f929e5b6640 1 -- 172.21.15.169:0/1189975927 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f929804db10 con 0x7f92981597c0 2023-10-26T01:37:35.542 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:35.540+0000 7f92957fa640 1 -- 172.21.15.169:0/1189975927 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v114) v1 ==== 97+0+0 (secure 0 0 0) 0x7f9290054600 con 0x7f92981597c0 2023-10-26T01:37:35.542 INFO:teuthology.orchestra.run.smithi169.stderr: removed all blocklist entries 2023-10-26T01:37:35.545 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:35.543+0000 7f929e5b6640 1 -- 172.21.15.169:0/1189975927 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f927806ae40 msgr2=0x7f927806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:35.545 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:35.543+0000 7f929e5b6640 1 --2- 172.21.15.169:0/1189975927 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f927806ae40 0x7f927806d300 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f9288036000 tx=0x7f9288004730 comp rx=0 tx=0).stop 2023-10-26T01:37:35.545 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:35.543+0000 7f929e5b6640 1 -- 172.21.15.169:0/1189975927 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f92981597c0 msgr2=0x7f929815a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:35.546 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:35.543+0000 7f929e5b6640 1 --2- 172.21.15.169:0/1189975927 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f92981597c0 0x7f929815a310 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f929000c7a0 tx=0x7f929000cc60 comp rx=0 tx=0).stop 2023-10-26T01:37:35.547 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:35.544+0000 7f929e5b6640 1 -- 172.21.15.169:0/1189975927 shutdown_connections 2023-10-26T01:37:35.547 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:35.544+0000 7f929e5b6640 1 --2- 172.21.15.169:0/1189975927 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f927806ae40 0x7f927806d300 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:35.547 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:35.544+0000 7f929e5b6640 1 --2- 172.21.15.169:0/1189975927 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f92981597c0 0x7f929815a310 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:35.547 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:35.544+0000 7f929e5b6640 1 --2- 172.21.15.169:0/1189975927 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f92981593e0 0x7f92981e5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:35.547 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:35.544+0000 7f929e5b6640 1 --2- 172.21.15.169:0/1189975927 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f92980b0c80 0x7f92981f25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:35.548 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:35.544+0000 7f929e5b6640 1 -- 172.21.15.169:0/1189975927 >> 172.21.15.169:0/1189975927 conn(0x7f9298069930 msgr2=0x7f929814e1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:35.548 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:35.544+0000 7f929e5b6640 1 -- 172.21.15.169:0/1189975927 shutdown_connections 2023-10-26T01:37:35.548 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:35.544+0000 7f929e5b6640 1 -- 172.21.15.169:0/1189975927 wait complete. 2023-10-26T01:37:35.567 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-26T01:37:35.785 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:35.783+0000 7f62ec4b4640 1 -- 172.21.15.169:0/4140925907 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62e40ac930 msgr2=0x7f62e40cd6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:35.786 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:35.783+0000 7f62ec4b4640 1 --2- 172.21.15.169:0/4140925907 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62e40ac930 0x7f62e40cd6f0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f62e40aab10 tx=0x7f62d402bd30 comp rx=0 tx=0).stop 2023-10-26T01:37:35.786 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:35.784+0000 7f62ec4b4640 1 -- 172.21.15.169:0/4140925907 shutdown_connections 2023-10-26T01:37:35.786 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:35.784+0000 7f62ec4b4640 1 --2- 172.21.15.169:0/4140925907 >> v2:172.21.15.170:3300/0 conn(0x7f62e40cdc30 0x7f62e40cbac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:35.786 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:35.784+0000 7f62ec4b4640 1 --2- 172.21.15.169:0/4140925907 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62e40ac930 0x7f62e40cd6f0 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:35.787 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:35.784+0000 7f62ec4b4640 1 -- 172.21.15.169:0/4140925907 >> 172.21.15.169:0/4140925907 conn(0x7f62e4069930 msgr2=0x7f62e4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:35.787 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:35.784+0000 7f62ec4b4640 1 -- 172.21.15.169:0/4140925907 shutdown_connections 2023-10-26T01:37:35.787 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:35.784+0000 7f62ec4b4640 1 -- 172.21.15.169:0/4140925907 wait complete. 2023-10-26T01:37:35.788 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:35.786+0000 7f62ec4b4640 1 Processor -- start 2023-10-26T01:37:35.788 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:35.786+0000 7f62ec4b4640 1 -- start start 2023-10-26T01:37:35.790 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:35.787+0000 7f62ec4b4640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f62e40ac930 0x7f62e40b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:35.790 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:35.787+0000 7f62ec4b4640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f62e40cdc30 0x7f62e41505d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:35.790 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:35.787+0000 7f62ec4b4640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62e41592d0 0x7f62e4151f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:35.791 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:35.787+0000 7f62ec4b4640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62e41519f0 con 0x7f62e41592d0 2023-10-26T01:37:35.791 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:35.787+0000 7f62ec4b4640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62e40b0a00 con 0x7f62e40cdc30 2023-10-26T01:37:35.791 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:35.787+0000 7f62ec4b4640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62e40b0c30 con 0x7f62e40ac930 2023-10-26T01:37:35.791 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:35.787+0000 7f62eaa51640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62e41592d0 0x7f62e4151f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:35.791 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:35.787+0000 7f62eaa51640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62e41592d0 0x7f62e4151f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:47020/0 (socket says 172.21.15.169:47020) 2023-10-26T01:37:35.791 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:35.787+0000 7f62eaa51640 1 -- 172.21.15.169:0/1738092965 learned_addr learned my addr 172.21.15.169:0/1738092965 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:37:35.792 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:35.787+0000 7f62ea250640 1 --2- 172.21.15.169:0/1738092965 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f62e40ac930 0x7f62e40b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:35.792 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:35.787+0000 7f62e9a4f640 1 --2- 172.21.15.169:0/1738092965 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f62e40cdc30 0x7f62e41505d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:35.792 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:35.787+0000 7f62eaa51640 1 -- 172.21.15.169:0/1738092965 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f62e40ac930 msgr2=0x7f62e40b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:35.792 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:35.787+0000 7f62eaa51640 1 --2- 172.21.15.169:0/1738092965 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f62e40ac930 0x7f62e40b04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:35.793 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:35.787+0000 7f62eaa51640 1 -- 172.21.15.169:0/1738092965 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f62e40cdc30 msgr2=0x7f62e41505d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:35.793 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:35.787+0000 7f62eaa51640 1 --2- 172.21.15.169:0/1738092965 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f62e40cdc30 0x7f62e41505d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:35.793 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:35.787+0000 7f62eaa51640 1 -- 172.21.15.169:0/1738092965 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62d4009560 con 0x7f62e41592d0 2023-10-26T01:37:35.793 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:35.787+0000 7f62e9a4f640 1 --2- 172.21.15.169:0/1738092965 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f62e40cdc30 0x7f62e41505d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:37:35.793 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:35.788+0000 7f62eaa51640 1 --2- 172.21.15.169:0/1738092965 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62e41592d0 0x7f62e4151f10 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f62dc00cad0 tx=0x7f62dc00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:35.793 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:35.788+0000 7f62daffd640 1 -- 172.21.15.169:0/1738092965 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62dc00ae20 con 0x7f62e41592d0 2023-10-26T01:37:35.794 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:35.788+0000 7f62daffd640 1 -- 172.21.15.169:0/1738092965 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f62dc004b20 con 0x7f62e41592d0 2023-10-26T01:37:35.794 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:35.788+0000 7f62ec4b4640 1 -- 172.21.15.169:0/1738092965 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62e40b0f10 con 0x7f62e41592d0 2023-10-26T01:37:35.794 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:35.788+0000 7f62daffd640 1 -- 172.21.15.169:0/1738092965 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62dc005020 con 0x7f62e41592d0 2023-10-26T01:37:35.794 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:35.788+0000 7f62ec4b4640 1 -- 172.21.15.169:0/1738092965 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62e40b14d0 con 0x7f62e41592d0 2023-10-26T01:37:35.795 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:35.789+0000 7f62ec4b4640 1 -- 172.21.15.169:0/1738092965 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62e404db10 con 0x7f62e41592d0 2023-10-26T01:37:35.795 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:35.791+0000 7f62daffd640 1 -- 172.21.15.169:0/1738092965 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f62dc005180 con 0x7f62e41592d0 2023-10-26T01:37:35.796 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:35.792+0000 7f62daffd640 1 --2- 172.21.15.169:0/1738092965 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f62c806ae40 0x7f62c806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:35.796 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:35.792+0000 7f62daffd640 1 -- 172.21.15.169:0/1738092965 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f62dc089c60 con 0x7f62e41592d0 2023-10-26T01:37:35.796 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:35.792+0000 7f62ea250640 1 --2- 172.21.15.169:0/1738092965 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f62c806ae40 0x7f62c806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:35.796 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:35.793+0000 7f62ea250640 1 --2- 172.21.15.169:0/1738092965 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f62c806ae40 0x7f62c806d300 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f62e0006fd0 tx=0x7f62e0008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:35.797 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:35.796+0000 7f62daffd640 1 -- 172.21.15.169:0/1738092965 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f62dc054fc0 con 0x7f62e41592d0 2023-10-26T01:37:36.059 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:36.057+0000 7f62ec4b4640 1 -- 172.21.15.169:0/1738092965 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f62e4154000 con 0x7f62e41592d0 2023-10-26T01:37:36.060 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:36.059+0000 7f62daffd640 1 -- 172.21.15.169:0/1738092965 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f62dc01d070 con 0x7f62e41592d0 2023-10-26T01:37:36.060 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:37:36.061 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2023-10-26T01:37:36.061 INFO:teuthology.orchestra.run.smithi169.stdout: "auth_dump": [ 2023-10-26T01:37:36.061 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:36.061 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "mds.a", 2023-10-26T01:37:36.061 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAgwTllWtUfAhAA28+12VwWEF/BUEu+ErSVyA==", 2023-10-26T01:37:36.062 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:37:36.062 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow", 2023-10-26T01:37:36.062 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow *", 2023-10-26T01:37:36.062 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow *", 2023-10-26T01:37:36.062 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:37:36.063 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:36.063 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:36.063 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:36.063 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "mds.b", 2023-10-26T01:37:36.063 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAgwTlleA8lHxAA+jMcMo+QnYAGllpKJ6eYgQ==", 2023-10-26T01:37:36.064 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:37:36.064 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow", 2023-10-26T01:37:36.064 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow *", 2023-10-26T01:37:36.064 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow *", 2023-10-26T01:37:36.064 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:37:36.065 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:36.065 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:36.065 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:36.065 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "mds.c", 2023-10-26T01:37:36.065 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAgwTllaCCqEBAAokeHvwOyp/Jv2x0yHDwysw==", 2023-10-26T01:37:36.065 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:37:36.066 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow", 2023-10-26T01:37:36.066 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow *", 2023-10-26T01:37:36.066 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow *", 2023-10-26T01:37:36.066 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:37:36.066 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:36.067 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:36.067 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:36.067 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "mds.d", 2023-10-26T01:37:36.067 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAgwTll51JQLhAA4Wzso4zQj+Htv6cXqvwRmg==", 2023-10-26T01:37:36.067 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:37:36.067 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow", 2023-10-26T01:37:36.068 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow *", 2023-10-26T01:37:36.068 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow *", 2023-10-26T01:37:36.068 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:37:36.068 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:36.068 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:36.069 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:36.069 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "osd.0", 2023-10-26T01:37:36.069 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAlwTll8RZ+HRAAt/sBK+Mi91YmTyQYbhKHLg==", 2023-10-26T01:37:36.069 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:37:36.069 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow profile osd", 2023-10-26T01:37:36.070 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow profile osd", 2023-10-26T01:37:36.070 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:37:36.070 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:36.070 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:36.070 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:36.070 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "osd.1", 2023-10-26T01:37:36.071 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAlwTllLXwWLRAAuHB2lJ+oDJou/IJBtsYY4Q==", 2023-10-26T01:37:36.071 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:37:36.071 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow profile osd", 2023-10-26T01:37:36.071 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow profile osd", 2023-10-26T01:37:36.071 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:37:36.072 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:36.072 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:36.072 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:36.072 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "osd.2", 2023-10-26T01:37:36.072 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAmwTllj1nSARAA5Jl34Q0X7a71ySd4wz17lw==", 2023-10-26T01:37:36.072 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:37:36.073 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow profile osd", 2023-10-26T01:37:36.073 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow profile osd", 2023-10-26T01:37:36.073 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:37:36.073 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:36.073 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:36.073 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:36.074 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "osd.3", 2023-10-26T01:37:36.074 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAmwTllC8MeEhAAfGxRVSwD+YbU/U5aeenO3w==", 2023-10-26T01:37:36.074 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:37:36.074 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow profile osd", 2023-10-26T01:37:36.075 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow profile osd", 2023-10-26T01:37:36.075 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:37:36.075 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:36.075 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:36.075 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:36.075 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "osd.4", 2023-10-26T01:37:36.075 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAowTll4Md6DhAAuXfadUNAdouMP28MiX/D3Q==", 2023-10-26T01:37:36.076 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:37:36.076 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow profile osd", 2023-10-26T01:37:36.076 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow profile osd", 2023-10-26T01:37:36.076 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:37:36.076 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:36.077 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:36.077 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:36.077 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "osd.5", 2023-10-26T01:37:36.077 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAowTllDdViHRAAec9QyhZCsjJ2rgGhhcjwpQ==", 2023-10-26T01:37:36.077 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:37:36.077 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow profile osd", 2023-10-26T01:37:36.078 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow profile osd", 2023-10-26T01:37:36.078 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:37:36.078 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:36.078 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:36.078 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:36.079 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "osd.6", 2023-10-26T01:37:36.079 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAowTll23X5KxAAsmQvG0yq7ilkVG5fNiaCUg==", 2023-10-26T01:37:36.079 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:37:36.079 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow profile osd", 2023-10-26T01:37:36.079 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow profile osd", 2023-10-26T01:37:36.079 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:37:36.080 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:36.080 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:36.080 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:36.080 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "osd.7", 2023-10-26T01:37:36.080 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAowTllhcy/OhAAY5Yjheh76ibOVREqN6WBpQ==", 2023-10-26T01:37:36.080 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:37:36.081 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow profile osd", 2023-10-26T01:37:36.081 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow profile osd", 2023-10-26T01:37:36.081 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:37:36.081 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:36.081 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:36.082 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:36.082 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "client.0", 2023-10-26T01:37:36.082 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAgwTllKC7gOhAAPsvxGcuUA8/p8xqbUgGteA==", 2023-10-26T01:37:36.082 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:37:36.082 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow", 2023-10-26T01:37:36.083 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow r", 2023-10-26T01:37:36.083 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-26T01:37:36.083 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:36.083 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:36.083 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:36.084 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "client.1", 2023-10-26T01:37:36.084 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAhwTllhxb6ExAAeYTsurWpnjohWlrNIlLpsw==", 2023-10-26T01:37:36.084 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:37:36.084 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow", 2023-10-26T01:37:36.084 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow r", 2023-10-26T01:37:36.084 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow rw", 2023-10-26T01:37:36.085 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow rwx" 2023-10-26T01:37:36.085 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:36.085 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:36.085 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:36.085 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "client.2", 2023-10-26T01:37:36.086 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAhwTllLWMoChAAsUHzBc29GDqmEQAFnl7dHg==", 2023-10-26T01:37:36.086 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:37:36.086 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow", 2023-10-26T01:37:36.086 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow r", 2023-10-26T01:37:36.086 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow rw", 2023-10-26T01:37:36.087 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow rwx" 2023-10-26T01:37:36.087 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:36.087 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:36.087 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:36.087 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "client.3", 2023-10-26T01:37:36.087 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAhwTll875HHRAAGbwikoR/hDU8P/bj0OPWAw==", 2023-10-26T01:37:36.088 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:37:36.088 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow", 2023-10-26T01:37:36.088 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow r", 2023-10-26T01:37:36.088 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow rw", 2023-10-26T01:37:36.088 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow rwx" 2023-10-26T01:37:36.088 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:36.089 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:36.089 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:36.089 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "client.admin", 2023-10-26T01:37:36.089 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAewTll2yddMhAAO8bnXNevDGe/KEeNNa36pQ==", 2023-10-26T01:37:36.089 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:37:36.090 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow *", 2023-10-26T01:37:36.090 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow *", 2023-10-26T01:37:36.090 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow *", 2023-10-26T01:37:36.090 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:37:36.090 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:36.091 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:36.091 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:36.091 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "mgr.x", 2023-10-26T01:37:36.091 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAfwTllPu3FMxAAV8d5oRFXAeeZH7e1xzPEDg==", 2023-10-26T01:37:36.091 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:37:36.091 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow *", 2023-10-26T01:37:36.092 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow profile mgr", 2023-10-26T01:37:36.092 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:37:36.092 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:36.092 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:37:36.092 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:36.093 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "mgr.y", 2023-10-26T01:37:36.093 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAfwTllZc+DJxAAsZ2TWn8H7Kw/lNTc2qyKzA==", 2023-10-26T01:37:36.093 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:37:36.093 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow *", 2023-10-26T01:37:36.093 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow profile mgr", 2023-10-26T01:37:36.094 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:37:36.094 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:36.094 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:36.094 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:37:36.094 INFO:teuthology.orchestra.run.smithi169.stdout:} 2023-10-26T01:37:36.096 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:36.061+0000 7f62ec4b4640 1 -- 172.21.15.169:0/1738092965 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f62c806ae40 msgr2=0x7f62c806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:36.096 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:36.061+0000 7f62ec4b4640 1 --2- 172.21.15.169:0/1738092965 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f62c806ae40 0x7f62c806d300 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f62e0006fd0 tx=0x7f62e0008040 comp rx=0 tx=0).stop 2023-10-26T01:37:36.096 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:36.061+0000 7f62ec4b4640 1 -- 172.21.15.169:0/1738092965 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62e41592d0 msgr2=0x7f62e4151f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:36.096 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:36.061+0000 7f62ec4b4640 1 --2- 172.21.15.169:0/1738092965 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62e41592d0 0x7f62e4151f10 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f62dc00cad0 tx=0x7f62dc00cf90 comp rx=0 tx=0).stop 2023-10-26T01:37:36.096 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:36.062+0000 7f62ec4b4640 1 -- 172.21.15.169:0/1738092965 shutdown_connections 2023-10-26T01:37:36.097 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:36.062+0000 7f62ec4b4640 1 --2- 172.21.15.169:0/1738092965 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f62c806ae40 0x7f62c806d300 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:36.097 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:36.062+0000 7f62ec4b4640 1 --2- 172.21.15.169:0/1738092965 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62e41592d0 0x7f62e4151f10 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:36.097 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:36.062+0000 7f62ec4b4640 1 --2- 172.21.15.169:0/1738092965 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f62e40cdc30 0x7f62e41505d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:36.097 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:36.062+0000 7f62ec4b4640 1 --2- 172.21.15.169:0/1738092965 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f62e40ac930 0x7f62e40b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:36.097 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:36.062+0000 7f62ec4b4640 1 -- 172.21.15.169:0/1738092965 >> 172.21.15.169:0/1738092965 conn(0x7f62e4069930 msgr2=0x7f62e4069d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:36.097 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:36.062+0000 7f62ec4b4640 1 -- 172.21.15.169:0/1738092965 shutdown_connections 2023-10-26T01:37:36.098 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:36.062+0000 7f62ec4b4640 1 -- 172.21.15.169:0/1738092965 wait complete. 2023-10-26T01:37:36.098 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:37:36.169 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T01:37:36.175 DEBUG:teuthology.orchestra.run.smithi170:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T01:37:36.309 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:36.307+0000 7fe2effff640 1 -- 172.21.15.169:0/2504768546 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2f00cb240 msgr2=0x7fe2f00c91f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:36.309 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:36.307+0000 7fe2effff640 1 --2- 172.21.15.169:0/2504768546 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2f00cb240 0x7fe2f00c91f0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fe2e8005fd0 tx=0x7fe2e802bd90 comp rx=0 tx=0).stop 2023-10-26T01:37:36.310 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:36.307+0000 7fe2effff640 1 -- 172.21.15.169:0/2504768546 shutdown_connections 2023-10-26T01:37:36.310 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:36.307+0000 7fe2effff640 1 --2- 172.21.15.169:0/2504768546 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2f00cb240 0x7fe2f00c91f0 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:36.310 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:36.307+0000 7fe2effff640 1 --2- 172.21.15.169:0/2504768546 >> v2:172.21.15.170:3300/0 conn(0x7fe2f0152150 0x7fe2f00ac420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:36.310 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:36.307+0000 7fe2effff640 1 -- 172.21.15.169:0/2504768546 >> 172.21.15.169:0/2504768546 conn(0x7fe2f00aedd0 msgr2=0x7fe2f00af1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:36.311 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:36.307+0000 7fe2effff640 1 -- 172.21.15.169:0/2504768546 shutdown_connections 2023-10-26T01:37:36.311 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:36.307+0000 7fe2effff640 1 -- 172.21.15.169:0/2504768546 wait complete. 2023-10-26T01:37:36.313 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:36.309+0000 7fe2effff640 1 Processor -- start 2023-10-26T01:37:36.313 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:36.309+0000 7fe2effff640 1 -- start start 2023-10-26T01:37:36.314 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:36.310+0000 7fe2effff640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe2f00cb240 0x7fe2f0150f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:36.314 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:36.310+0000 7fe2effff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2f0151d70 0x7fe2f006f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:36.314 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:36.310+0000 7fe2effff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe2f0152150 0x7fe2f00c6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:36.314 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:36.310+0000 7fe2effff640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f0099d30 con 0x7fe2f0151d70 2023-10-26T01:37:36.314 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:36.310+0000 7fe2effff640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f006ec20 con 0x7fe2f0152150 2023-10-26T01:37:36.315 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:36.310+0000 7fe2effff640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f0099f80 con 0x7fe2f00cb240 2023-10-26T01:37:36.315 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:36.310+0000 7fe2e67fc640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2f0151d70 0x7fe2f006f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:36.315 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:36.310+0000 7fe2e67fc640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2f0151d70 0x7fe2f006f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:47034/0 (socket says 172.21.15.169:47034) 2023-10-26T01:37:36.315 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:36.310+0000 7fe2e67fc640 1 -- 172.21.15.169:0/4155547619 learned_addr learned my addr 172.21.15.169:0/4155547619 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:37:36.315 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:36.311+0000 7fe2eeffd640 1 --2- 172.21.15.169:0/4155547619 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe2f00cb240 0x7fe2f0150f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:36.316 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:36.311+0000 7fe2e67fc640 1 -- 172.21.15.169:0/4155547619 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe2f00cb240 msgr2=0x7fe2f0150f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:36.316 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:36.311+0000 7fe2e67fc640 1 --2- 172.21.15.169:0/4155547619 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe2f00cb240 0x7fe2f0150f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:36.316 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:36.311+0000 7fe2e67fc640 1 -- 172.21.15.169:0/4155547619 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe2f0152150 msgr2=0x7fe2f00c6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:36.316 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:36.311+0000 7fe2ef7fe640 1 --2- 172.21.15.169:0/4155547619 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe2f0152150 0x7fe2f00c6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:36.316 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:36.311+0000 7fe2e67fc640 1 --2- 172.21.15.169:0/4155547619 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe2f0152150 0x7fe2f00c6270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:36.317 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:36.311+0000 7fe2e67fc640 1 -- 172.21.15.169:0/4155547619 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2e8009560 con 0x7fe2f0151d70 2023-10-26T01:37:36.317 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:36.311+0000 7fe2eeffd640 1 --2- 172.21.15.169:0/4155547619 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe2f00cb240 0x7fe2f0150f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:37:36.317 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:36.311+0000 7fe2ef7fe640 1 --2- 172.21.15.169:0/4155547619 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe2f0152150 0x7fe2f00c6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:37:36.317 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:36.311+0000 7fe2e67fc640 1 --2- 172.21.15.169:0/4155547619 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2f0151d70 0x7fe2f006f290 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fe2d800e880 tx=0x7fe2d800ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:36.317 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:36.311+0000 7fe2e7fff640 1 -- 172.21.15.169:0/4155547619 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2d800cbc0 con 0x7fe2f0151d70 2023-10-26T01:37:36.318 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:36.311+0000 7fe2e7fff640 1 -- 172.21.15.169:0/4155547619 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe2d800ce30 con 0x7fe2f0151d70 2023-10-26T01:37:36.318 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:36.311+0000 7fe2e7fff640 1 -- 172.21.15.169:0/4155547619 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2d8004270 con 0x7fe2f0151d70 2023-10-26T01:37:36.318 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:36.311+0000 7fe2effff640 1 -- 172.21.15.169:0/4155547619 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2f009a260 con 0x7fe2f0151d70 2023-10-26T01:37:36.318 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:36.312+0000 7fe2effff640 1 -- 172.21.15.169:0/4155547619 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2f009a820 con 0x7fe2f0151d70 2023-10-26T01:37:36.319 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:36.313+0000 7fe2effff640 1 -- 172.21.15.169:0/4155547619 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2f0073c10 con 0x7fe2f0151d70 2023-10-26T01:37:36.319 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:36.314+0000 7fe2e7fff640 1 -- 172.21.15.169:0/4155547619 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fe2d80035f0 con 0x7fe2f0151d70 2023-10-26T01:37:36.320 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:36.314+0000 7fe2e7fff640 1 --2- 172.21.15.169:0/4155547619 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe2d006ae40 0x7fe2d006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:36.320 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:36.315+0000 7fe2e7fff640 1 -- 172.21.15.169:0/4155547619 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fe2d8089a40 con 0x7fe2f0151d70 2023-10-26T01:37:36.320 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:36.315+0000 7fe2eeffd640 1 --2- 172.21.15.169:0/4155547619 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe2d006ae40 0x7fe2d006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:36.320 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:36.316+0000 7fe2eeffd640 1 --2- 172.21.15.169:0/4155547619 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe2d006ae40 0x7fe2d006d300 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fe2e0004520 tx=0x7fe2e0009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:36.321 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:36.319+0000 7fe2e7fff640 1 -- 172.21.15.169:0/4155547619 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe2d8058830 con 0x7fe2f0151d70 2023-10-26T01:37:36.512 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:36.510+0000 7fe2effff640 1 -- 172.21.15.169:0/4155547619 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe2f014f730 con 0x7fe2f0151d70 2023-10-26T01:37:36.513 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:36.512+0000 7fe2e7fff640 1 -- 172.21.15.169:0/4155547619 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v116) v1 ==== 74+0+12916 (secure 0 0 0) 0x7fe2d801c020 con 0x7fe2f0151d70 2023-10-26T01:37:36.514 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:37:36.514 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":116,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:37:35.550735+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:37:36.516 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:36.514+0000 7fe2effff640 1 -- 172.21.15.169:0/4155547619 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe2d006ae40 msgr2=0x7fe2d006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:36.517 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:36.514+0000 7fe2effff640 1 --2- 172.21.15.169:0/4155547619 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe2d006ae40 0x7fe2d006d300 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fe2e0004520 tx=0x7fe2e0009040 comp rx=0 tx=0).stop 2023-10-26T01:37:36.517 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:36.514+0000 7fe2effff640 1 -- 172.21.15.169:0/4155547619 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2f0151d70 msgr2=0x7fe2f006f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:36.517 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:36.514+0000 7fe2effff640 1 --2- 172.21.15.169:0/4155547619 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2f0151d70 0x7fe2f006f290 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fe2d800e880 tx=0x7fe2d800ed40 comp rx=0 tx=0).stop 2023-10-26T01:37:36.517 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:36.515+0000 7fe2effff640 1 -- 172.21.15.169:0/4155547619 shutdown_connections 2023-10-26T01:37:36.518 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:36.515+0000 7fe2effff640 1 --2- 172.21.15.169:0/4155547619 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe2d006ae40 0x7fe2d006d300 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:36.518 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:36.515+0000 7fe2effff640 1 --2- 172.21.15.169:0/4155547619 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2f0151d70 0x7fe2f006f290 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:36.518 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:36.515+0000 7fe2effff640 1 --2- 172.21.15.169:0/4155547619 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe2f0152150 0x7fe2f00c6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:36.518 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:36.515+0000 7fe2effff640 1 --2- 172.21.15.169:0/4155547619 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe2f00cb240 0x7fe2f0150f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:36.518 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:36.515+0000 7fe2effff640 1 -- 172.21.15.169:0/4155547619 >> 172.21.15.169:0/4155547619 conn(0x7fe2f00aedd0 msgr2=0x7fe2f00af1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:36.519 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:36.515+0000 7fe2effff640 1 -- 172.21.15.169:0/4155547619 shutdown_connections 2023-10-26T01:37:36.519 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:36.515+0000 7fe2effff640 1 -- 172.21.15.169:0/4155547619 wait complete. 2023-10-26T01:37:36.537 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T01:30:32.200808+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '109', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T01:37:36.538 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T01:37:36.804 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:36.802+0000 7f95fabe9640 1 -- 172.21.15.169:0/1988343965 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95f40b37a0 msgr2=0x7f95f40cc260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:36.804 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:36.802+0000 7f95fabe9640 1 --2- 172.21.15.169:0/1988343965 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95f40b37a0 0x7f95f40cc260 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f95f4068920 tx=0x7f95ec02c080 comp rx=0 tx=0).stop 2023-10-26T01:37:36.804 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:36.802+0000 7f95fabe9640 1 -- 172.21.15.169:0/1988343965 shutdown_connections 2023-10-26T01:37:36.805 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:36.802+0000 7f95fabe9640 1 --2- 172.21.15.169:0/1988343965 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95f40b37a0 0x7f95f40cc260 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:36.805 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:36.802+0000 7f95fabe9640 1 --2- 172.21.15.169:0/1988343965 >> v2:172.21.15.170:3300/0 conn(0x7f95f40b0d00 0x7f95f40b3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:36.805 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:36.802+0000 7f95fabe9640 1 -- 172.21.15.169:0/1988343965 >> 172.21.15.169:0/1988343965 conn(0x7f95f406b560 msgr2=0x7f95f406b970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:36.805 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:36.802+0000 7f95fabe9640 1 -- 172.21.15.169:0/1988343965 shutdown_connections 2023-10-26T01:37:36.805 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:36.802+0000 7f95fabe9640 1 -- 172.21.15.169:0/1988343965 wait complete. 2023-10-26T01:37:36.807 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:36.804+0000 7f95fabe9640 1 Processor -- start 2023-10-26T01:37:36.807 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:36.804+0000 7f95fabe9640 1 -- start start 2023-10-26T01:37:36.807 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:36.805+0000 7f95fabe9640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95f40b0d00 0x7f95f41f1b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:36.807 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:36.805+0000 7f95fabe9640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f95f40b37a0 0x7f95f4153320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:36.808 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:36.805+0000 7f95fabe9640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f95f4158420 0x7f95f4153860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:36.808 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:36.805+0000 7f95fabe9640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95f4156ad0 con 0x7f95f40b0d00 2023-10-26T01:37:36.808 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:36.805+0000 7f95fabe9640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95f4159050 con 0x7f95f40b37a0 2023-10-26T01:37:36.808 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:36.805+0000 7f95fabe9640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95f4158a40 con 0x7f95f4158420 2023-10-26T01:37:36.808 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:36.805+0000 7f95f8985640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95f40b0d00 0x7f95f41f1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:36.809 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:36.805+0000 7f95f8985640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95f40b0d00 0x7f95f41f1b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:47042/0 (socket says 172.21.15.169:47042) 2023-10-26T01:37:36.809 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:36.805+0000 7f95f8985640 1 -- 172.21.15.169:0/1663747794 learned_addr learned my addr 172.21.15.169:0/1663747794 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:37:36.810 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:36.806+0000 7f95f8985640 1 -- 172.21.15.169:0/1663747794 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f95f4158420 msgr2=0x7f95f4153860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:36.810 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:36.806+0000 7f95f8985640 1 --2- 172.21.15.169:0/1663747794 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f95f4158420 0x7f95f4153860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:36.811 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:36.806+0000 7f95f8985640 1 -- 172.21.15.169:0/1663747794 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f95f40b37a0 msgr2=0x7f95f4153320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:36.811 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:36.806+0000 7f95f8985640 1 --2- 172.21.15.169:0/1663747794 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f95f40b37a0 0x7f95f4153320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:36.811 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:36.806+0000 7f95f8985640 1 -- 172.21.15.169:0/1663747794 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95ec009840 con 0x7f95f40b0d00 2023-10-26T01:37:36.811 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:36.806+0000 7f95f8985640 1 --2- 172.21.15.169:0/1663747794 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95f40b0d00 0x7f95f41f1b90 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f95e800c3c0 tx=0x7f95e800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:36.812 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:36.806+0000 7f95f17fa640 1 -- 172.21.15.169:0/1663747794 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95e8004290 con 0x7f95f40b0d00 2023-10-26T01:37:36.812 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:36.806+0000 7f95f17fa640 1 -- 172.21.15.169:0/1663747794 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f95e80035f0 con 0x7f95f40b0d00 2023-10-26T01:37:36.812 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:36.806+0000 7f95fabe9640 1 -- 172.21.15.169:0/1663747794 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95f41567f0 con 0x7f95f40b0d00 2023-10-26T01:37:36.812 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:36.806+0000 7f95f17fa640 1 -- 172.21.15.169:0/1663747794 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95e8004500 con 0x7f95f40b0d00 2023-10-26T01:37:36.812 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:36.806+0000 7f95fabe9640 1 -- 172.21.15.169:0/1663747794 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95f41438b0 con 0x7f95f40b0d00 2023-10-26T01:37:36.813 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:36.807+0000 7f95fabe9640 1 -- 172.21.15.169:0/1663747794 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95f41588a0 con 0x7f95f40b0d00 2023-10-26T01:37:36.813 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:36.808+0000 7f95f17fa640 1 -- 172.21.15.169:0/1663747794 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f95e8003a00 con 0x7f95f40b0d00 2023-10-26T01:37:36.814 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:36.809+0000 7f95f17fa640 1 --2- 172.21.15.169:0/1663747794 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f95d806ae40 0x7f95d806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:36.814 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:36.809+0000 7f95f17fa640 1 -- 172.21.15.169:0/1663747794 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f95e8088a60 con 0x7f95f40b0d00 2023-10-26T01:37:36.814 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:36.810+0000 7f95f3fff640 1 --2- 172.21.15.169:0/1663747794 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f95d806ae40 0x7f95d806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:36.814 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:36.811+0000 7f95f3fff640 1 --2- 172.21.15.169:0/1663747794 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f95d806ae40 0x7f95d806d300 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f95e0005fd0 tx=0x7f95e0005850 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:36.815 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:36.814+0000 7f95f17fa640 1 -- 172.21.15.169:0/1663747794 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f95f41588a0 con 0x7f95f40b0d00 2023-10-26T01:37:37.009 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:37.007+0000 7f95fabe9640 1 -- 172.21.15.169:0/1663747794 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f95f41588a0 con 0x7f95f40b0d00 2023-10-26T01:37:37.010 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:37.008+0000 7f95f17fa640 1 -- 172.21.15.169:0/1663747794 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v116) v1 ==== 93+0+10 (secure 0 0 0) 0x7f95f41588a0 con 0x7f95f40b0d00 2023-10-26T01:37:37.010 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 1 2023-10-26T01:37:37.013 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:37.011+0000 7f95fabe9640 1 -- 172.21.15.169:0/1663747794 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f95d806ae40 msgr2=0x7f95d806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:37.013 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:37.011+0000 7f95fabe9640 1 --2- 172.21.15.169:0/1663747794 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f95d806ae40 0x7f95d806d300 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f95e0005fd0 tx=0x7f95e0005850 comp rx=0 tx=0).stop 2023-10-26T01:37:37.013 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:37.011+0000 7f95fabe9640 1 -- 172.21.15.169:0/1663747794 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95f40b0d00 msgr2=0x7f95f41f1b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:37.013 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:37.011+0000 7f95fabe9640 1 --2- 172.21.15.169:0/1663747794 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95f40b0d00 0x7f95f41f1b90 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f95e800c3c0 tx=0x7f95e800c880 comp rx=0 tx=0).stop 2023-10-26T01:37:37.014 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:37.011+0000 7f95fabe9640 1 -- 172.21.15.169:0/1663747794 shutdown_connections 2023-10-26T01:37:37.014 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:37.011+0000 7f95fabe9640 1 --2- 172.21.15.169:0/1663747794 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f95d806ae40 0x7f95d806d300 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:37.014 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:37.011+0000 7f95fabe9640 1 --2- 172.21.15.169:0/1663747794 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f95f4158420 0x7f95f4153860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:37.014 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:37.011+0000 7f95fabe9640 1 --2- 172.21.15.169:0/1663747794 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f95f40b37a0 0x7f95f4153320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:37.015 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:37.011+0000 7f95fabe9640 1 --2- 172.21.15.169:0/1663747794 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95f40b0d00 0x7f95f41f1b90 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:37.015 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:37.011+0000 7f95fabe9640 1 -- 172.21.15.169:0/1663747794 >> 172.21.15.169:0/1663747794 conn(0x7f95f406b560 msgr2=0x7f95f414e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:37.015 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:37.011+0000 7f95fabe9640 1 -- 172.21.15.169:0/1663747794 shutdown_connections 2023-10-26T01:37:37.015 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:37.011+0000 7f95fabe9640 1 -- 172.21.15.169:0/1663747794 wait complete. 2023-10-26T01:37:37.033 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-26T01:37:37.257 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:37.255+0000 7f1b4d6bb640 1 -- 172.21.15.169:0/4294520102 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b4806a990 msgr2=0x7f1b480cb390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:37.257 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:37.255+0000 7f1b4d6bb640 1 --2- 172.21.15.169:0/4294520102 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b4806a990 0x7f1b480cb390 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f1b40005fd0 tx=0x7f1b4002c1b0 comp rx=0 tx=0).stop 2023-10-26T01:37:37.257 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:37.255+0000 7f1b4d6bb640 1 -- 172.21.15.169:0/4294520102 shutdown_connections 2023-10-26T01:37:37.258 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:37.255+0000 7f1b4d6bb640 1 --2- 172.21.15.169:0/4294520102 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b4806a990 0x7f1b480cb390 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:37.258 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:37.255+0000 7f1b4d6bb640 1 --2- 172.21.15.169:0/4294520102 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1b480c9e00 0x7f1b480c7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:37.258 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:37.255+0000 7f1b4d6bb640 1 -- 172.21.15.169:0/4294520102 >> 172.21.15.169:0/4294520102 conn(0x7f1b480aedd0 msgr2=0x7f1b480af1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:37.258 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:37.255+0000 7f1b4d6bb640 1 -- 172.21.15.169:0/4294520102 shutdown_connections 2023-10-26T01:37:37.258 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:37.255+0000 7f1b4d6bb640 1 -- 172.21.15.169:0/4294520102 wait complete. 2023-10-26T01:37:37.260 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:37.257+0000 7f1b4d6bb640 1 Processor -- start 2023-10-26T01:37:37.260 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:37.257+0000 7f1b4d6bb640 1 -- start start 2023-10-26T01:37:37.260 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:37.258+0000 7f1b4d6bb640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b4806a990 0x7f1b480afce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:37.260 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:37.258+0000 7f1b4d6bb640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1b4806cae0 0x7f1b480ab050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:37.261 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:37.258+0000 7f1b4d6bb640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1b480c9e00 0x7f1b480ab590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:37.261 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:37.258+0000 7f1b4d6bb640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b480b1140 con 0x7f1b4806a990 2023-10-26T01:37:37.261 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:37.258+0000 7f1b4d6bb640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b480b0530 con 0x7f1b4806cae0 2023-10-26T01:37:37.261 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:37.258+0000 7f1b4d6bb640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b480b0810 con 0x7f1b480c9e00 2023-10-26T01:37:37.261 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:37.258+0000 7f1b47fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b4806a990 0x7f1b480afce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:37.262 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:37.258+0000 7f1b47fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b4806a990 0x7f1b480afce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:47064/0 (socket says 172.21.15.169:47064) 2023-10-26T01:37:37.262 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:37.258+0000 7f1b47fff640 1 -- 172.21.15.169:0/2218561958 learned_addr learned my addr 172.21.15.169:0/2218561958 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:37:37.263 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:37.259+0000 7f1b4ceba640 1 --2- 172.21.15.169:0/2218561958 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1b480c9e00 0x7f1b480ab590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:37.264 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:37.259+0000 7f1b47fff640 1 -- 172.21.15.169:0/2218561958 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1b480c9e00 msgr2=0x7f1b480ab590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:37.264 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:37.259+0000 7f1b47fff640 1 --2- 172.21.15.169:0/2218561958 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1b480c9e00 0x7f1b480ab590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:37.264 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:37.259+0000 7f1b477fe640 1 --2- 172.21.15.169:0/2218561958 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1b4806cae0 0x7f1b480ab050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:37.264 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:37.259+0000 7f1b47fff640 1 -- 172.21.15.169:0/2218561958 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1b4806cae0 msgr2=0x7f1b480ab050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:37.265 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:37.259+0000 7f1b47fff640 1 --2- 172.21.15.169:0/2218561958 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1b4806cae0 0x7f1b480ab050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:37.265 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:37.259+0000 7f1b47fff640 1 -- 172.21.15.169:0/2218561958 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b40009900 con 0x7f1b4806a990 2023-10-26T01:37:37.265 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:37.259+0000 7f1b4ceba640 1 --2- 172.21.15.169:0/2218561958 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1b480c9e00 0x7f1b480ab590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:37:37.265 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:37.259+0000 7f1b47fff640 1 --2- 172.21.15.169:0/2218561958 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b4806a990 0x7f1b480afce0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f1b3800c430 tx=0x7f1b3800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:37.265 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:37.259+0000 7f1b44ff9640 1 -- 172.21.15.169:0/2218561958 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b38011790 con 0x7f1b4806a990 2023-10-26T01:37:37.266 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:37.259+0000 7f1b44ff9640 1 -- 172.21.15.169:0/2218561958 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1b380119e0 con 0x7f1b4806a990 2023-10-26T01:37:37.266 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:37.259+0000 7f1b4d6bb640 1 -- 172.21.15.169:0/2218561958 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b480b0af0 con 0x7f1b4806a990 2023-10-26T01:37:37.266 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:37.259+0000 7f1b44ff9640 1 -- 172.21.15.169:0/2218561958 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b38011e30 con 0x7f1b4806a990 2023-10-26T01:37:37.266 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:37.260+0000 7f1b4d6bb640 1 -- 172.21.15.169:0/2218561958 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b480b0220 con 0x7f1b4806a990 2023-10-26T01:37:37.266 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:37.261+0000 7f1b4d6bb640 1 -- 172.21.15.169:0/2218561958 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b48073c10 con 0x7f1b4806a990 2023-10-26T01:37:37.267 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:37.262+0000 7f1b44ff9640 1 -- 172.21.15.169:0/2218561958 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f1b3801c460 con 0x7f1b4806a990 2023-10-26T01:37:37.267 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:37.263+0000 7f1b44ff9640 1 --2- 172.21.15.169:0/2218561958 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f1b3406ae70 0x7f1b3406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:37.268 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:37.263+0000 7f1b44ff9640 1 -- 172.21.15.169:0/2218561958 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f1b38089540 con 0x7f1b4806a990 2023-10-26T01:37:37.268 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:37.263+0000 7f1b477fe640 1 --2- 172.21.15.169:0/2218561958 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f1b3406ae70 0x7f1b3406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:37.268 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:37.265+0000 7f1b477fe640 1 --2- 172.21.15.169:0/2218561958 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f1b3406ae70 0x7f1b3406d330 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c0058c0 tx=0x7f1b3c005850 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:37.268 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:37.267+0000 7f1b44ff9640 1 -- 172.21.15.169:0/2218561958 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1b380548a0 con 0x7f1b4806a990 2023-10-26T01:37:37.461 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:37.459+0000 7f1b4d6bb640 1 -- 172.21.15.169:0/2218561958 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f1b48071c90 con 0x7f1b4806a990 2023-10-26T01:37:37.461 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:37.460+0000 7f1b44ff9640 1 -- 172.21.15.169:0/2218561958 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v116) v1 ==== 84+0+65 (secure 0 0 0) 0x7f1b38057ec0 con 0x7f1b4806a990 2023-10-26T01:37:37.462 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:37:37.462 INFO:teuthology.orchestra.run.smithi169.stdout:[ 2023-10-26T01:37:37.462 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:37:37.463 INFO:teuthology.orchestra.run.smithi169.stdout: "poolnum": 1, 2023-10-26T01:37:37.463 INFO:teuthology.orchestra.run.smithi169.stdout: "poolname": ".mgr" 2023-10-26T01:37:37.463 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:37:37.463 INFO:teuthology.orchestra.run.smithi169.stdout:] 2023-10-26T01:37:37.465 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:37.462+0000 7f1b4d6bb640 1 -- 172.21.15.169:0/2218561958 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f1b3406ae70 msgr2=0x7f1b3406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:37.465 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:37.462+0000 7f1b4d6bb640 1 --2- 172.21.15.169:0/2218561958 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f1b3406ae70 0x7f1b3406d330 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c0058c0 tx=0x7f1b3c005850 comp rx=0 tx=0).stop 2023-10-26T01:37:37.465 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:37.462+0000 7f1b4d6bb640 1 -- 172.21.15.169:0/2218561958 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b4806a990 msgr2=0x7f1b480afce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:37.465 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:37.462+0000 7f1b4d6bb640 1 --2- 172.21.15.169:0/2218561958 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b4806a990 0x7f1b480afce0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f1b3800c430 tx=0x7f1b3800c8f0 comp rx=0 tx=0).stop 2023-10-26T01:37:37.465 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:37.463+0000 7f1b4d6bb640 1 -- 172.21.15.169:0/2218561958 shutdown_connections 2023-10-26T01:37:37.466 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:37.463+0000 7f1b4d6bb640 1 --2- 172.21.15.169:0/2218561958 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f1b3406ae70 0x7f1b3406d330 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:37.466 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:37.463+0000 7f1b4d6bb640 1 --2- 172.21.15.169:0/2218561958 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1b480c9e00 0x7f1b480ab590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:37.466 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:37.463+0000 7f1b4d6bb640 1 --2- 172.21.15.169:0/2218561958 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1b4806cae0 0x7f1b480ab050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:37.466 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:37.463+0000 7f1b4d6bb640 1 --2- 172.21.15.169:0/2218561958 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b4806a990 0x7f1b480afce0 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:37.467 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:37.463+0000 7f1b4d6bb640 1 -- 172.21.15.169:0/2218561958 >> 172.21.15.169:0/2218561958 conn(0x7f1b480aedd0 msgr2=0x7f1b480af1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:37.467 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:37.463+0000 7f1b4d6bb640 1 -- 172.21.15.169:0/2218561958 shutdown_connections 2023-10-26T01:37:37.467 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:37.463+0000 7f1b4d6bb640 1 -- 172.21.15.169:0/2218561958 wait complete. 2023-10-26T01:37:37.483 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-26T01:37:37.483 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-26T01:37:37.704 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:37.702+0000 7fbb55100640 1 -- 172.21.15.169:0/3430814978 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb38003970 msgr2=0x7fbb38003d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:37.704 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:37.702+0000 7fbb55100640 1 --2- 172.21.15.169:0/3430814978 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb38003970 0x7fbb38003d70 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7fbb40006fb0 tx=0x7fbb380057f0 comp rx=0 tx=0).stop 2023-10-26T01:37:37.705 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:37.702+0000 7fbb55100640 1 -- 172.21.15.169:0/3430814978 shutdown_connections 2023-10-26T01:37:37.705 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:37.702+0000 7fbb55100640 1 --2- 172.21.15.169:0/3430814978 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb38003970 0x7fbb38003d70 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:37.705 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:37.702+0000 7fbb55100640 1 -- 172.21.15.169:0/3430814978 >> 172.21.15.169:0/3430814978 conn(0x7fbb50069930 msgr2=0x7fbb50069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:37.705 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:37.702+0000 7fbb55100640 1 -- 172.21.15.169:0/3430814978 shutdown_connections 2023-10-26T01:37:37.705 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:37.702+0000 7fbb55100640 1 -- 172.21.15.169:0/3430814978 wait complete. 2023-10-26T01:37:37.706 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:37.704+0000 7fbb55100640 1 Processor -- start 2023-10-26T01:37:37.706 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:37.704+0000 7fbb55100640 1 -- start start 2023-10-26T01:37:37.708 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:37.705+0000 7fbb55100640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb38003970 0x7fbb5014d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:37.708 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:37.705+0000 7fbb55100640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbb50071680 0x7fbb500cb440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:37.708 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:37.705+0000 7fbb55100640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb50071ac0 0x7fbb500cb980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:37.708 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:37.705+0000 7fbb55100640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb501500d0 con 0x7fbb38003970 2023-10-26T01:37:37.708 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:37.705+0000 7fbb55100640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb50150b40 con 0x7fbb50071ac0 2023-10-26T01:37:37.709 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:37.705+0000 7fbb55100640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb50150e20 con 0x7fbb50071680 2023-10-26T01:37:37.709 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:37.706+0000 7fbb4ed9d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb38003970 0x7fbb5014d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:37.709 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:37.706+0000 7fbb4ed9d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb38003970 0x7fbb5014d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:47090/0 (socket says 172.21.15.169:47090) 2023-10-26T01:37:37.709 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:37.706+0000 7fbb4ed9d640 1 -- 172.21.15.169:0/3801654166 learned_addr learned my addr 172.21.15.169:0/3801654166 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:37:37.709 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:37.706+0000 7fbb4f59e640 1 --2- 172.21.15.169:0/3801654166 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb50071ac0 0x7fbb500cb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:37.710 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:37.706+0000 7fbb4e59c640 1 --2- 172.21.15.169:0/3801654166 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbb50071680 0x7fbb500cb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:37.710 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:37.706+0000 7fbb4ed9d640 1 -- 172.21.15.169:0/3801654166 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbb50071680 msgr2=0x7fbb500cb440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:37.710 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:37.706+0000 7fbb4ed9d640 1 --2- 172.21.15.169:0/3801654166 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbb50071680 0x7fbb500cb440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:37.710 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:37.706+0000 7fbb4ed9d640 1 -- 172.21.15.169:0/3801654166 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb50071ac0 msgr2=0x7fbb500cb980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:37.710 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:37.706+0000 7fbb4ed9d640 1 --2- 172.21.15.169:0/3801654166 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb50071ac0 0x7fbb500cb980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:37.711 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:37.706+0000 7fbb4ed9d640 1 -- 172.21.15.169:0/3801654166 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb50151100 con 0x7fbb38003970 2023-10-26T01:37:37.711 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:37.706+0000 7fbb4e59c640 1 --2- 172.21.15.169:0/3801654166 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbb50071680 0x7fbb500cb440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:37:37.711 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:37.706+0000 7fbb4ed9d640 1 --2- 172.21.15.169:0/3801654166 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb38003970 0x7fbb5014d6c0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7fbb4002c8e0 tx=0x7fbb400017a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:37.712 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:37.707+0000 7fbb377fe640 1 -- 172.21.15.169:0/3801654166 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb40006230 con 0x7fbb38003970 2023-10-26T01:37:37.712 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:37.707+0000 7fbb377fe640 1 -- 172.21.15.169:0/3801654166 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fbb40006390 con 0x7fbb38003970 2023-10-26T01:37:37.712 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:37.707+0000 7fbb377fe640 1 -- 172.21.15.169:0/3801654166 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb400035f0 con 0x7fbb38003970 2023-10-26T01:37:37.712 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:37.707+0000 7fbb55100640 1 -- 172.21.15.169:0/3801654166 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb501513e0 con 0x7fbb38003970 2023-10-26T01:37:37.713 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:37.707+0000 7fbb55100640 1 -- 172.21.15.169:0/3801654166 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb5014dc00 con 0x7fbb38003970 2023-10-26T01:37:37.713 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:37.708+0000 7fbb55100640 1 -- 172.21.15.169:0/3801654166 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb50066af0 con 0x7fbb38003970 2023-10-26T01:37:37.713 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:37.709+0000 7fbb377fe640 1 -- 172.21.15.169:0/3801654166 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fbb40031030 con 0x7fbb38003970 2023-10-26T01:37:37.713 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:37.710+0000 7fbb377fe640 1 --2- 172.21.15.169:0/3801654166 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbb3c06ae40 0x7fbb3c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:37.713 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:37.711+0000 7fbb377fe640 1 -- 172.21.15.169:0/3801654166 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fbb400a9e50 con 0x7fbb38003970 2023-10-26T01:37:37.714 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:37.711+0000 7fbb4e59c640 1 --2- 172.21.15.169:0/3801654166 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbb3c06ae40 0x7fbb3c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:37.714 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:37.712+0000 7fbb4e59c640 1 --2- 172.21.15.169:0/3801654166 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbb3c06ae40 0x7fbb3c06d300 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fbb44005d00 tx=0x7fbb44005ab0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:37.716 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:37.715+0000 7fbb377fe640 1 -- 172.21.15.169:0/3801654166 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbb400751b0 con 0x7fbb38003970 2023-10-26T01:37:37.905 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:37.904+0000 7fbb55100640 1 -- 172.21.15.169:0/3801654166 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fbb500ab2f0 con 0x7fbb38003970 2023-10-26T01:37:38.549 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:38.547+0000 7fbb377fe640 1 -- 172.21.15.169:0/3801654166 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v117) v1 ==== 138+0+0 (secure 0 0 0) 0x7fbb500ab2f0 con 0x7fbb38003970 2023-10-26T01:37:38.549 INFO:teuthology.orchestra.run.smithi169.stderr:pool 'cephfs_metadata' created 2023-10-26T01:37:38.553 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:38.552+0000 7fbb55100640 1 -- 172.21.15.169:0/3801654166 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbb3c06ae40 msgr2=0x7fbb3c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:38.554 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:38.552+0000 7fbb55100640 1 --2- 172.21.15.169:0/3801654166 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbb3c06ae40 0x7fbb3c06d300 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fbb44005d00 tx=0x7fbb44005ab0 comp rx=0 tx=0).stop 2023-10-26T01:37:38.554 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:38.552+0000 7fbb55100640 1 -- 172.21.15.169:0/3801654166 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb38003970 msgr2=0x7fbb5014d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:38.554 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:38.552+0000 7fbb55100640 1 --2- 172.21.15.169:0/3801654166 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb38003970 0x7fbb5014d6c0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7fbb4002c8e0 tx=0x7fbb400017a0 comp rx=0 tx=0).stop 2023-10-26T01:37:38.555 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:38.553+0000 7fbb55100640 1 -- 172.21.15.169:0/3801654166 shutdown_connections 2023-10-26T01:37:38.555 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:38.553+0000 7fbb55100640 1 --2- 172.21.15.169:0/3801654166 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbb3c06ae40 0x7fbb3c06d300 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:38.555 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:38.553+0000 7fbb55100640 1 --2- 172.21.15.169:0/3801654166 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb50071ac0 0x7fbb500cb980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:38.555 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:38.553+0000 7fbb55100640 1 --2- 172.21.15.169:0/3801654166 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbb50071680 0x7fbb500cb440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:38.556 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:38.553+0000 7fbb55100640 1 --2- 172.21.15.169:0/3801654166 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb38003970 0x7fbb5014d6c0 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:38.556 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:38.553+0000 7fbb55100640 1 -- 172.21.15.169:0/3801654166 >> 172.21.15.169:0/3801654166 conn(0x7fbb50069930 msgr2=0x7fbb50159af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:38.556 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:38.553+0000 7fbb55100640 1 -- 172.21.15.169:0/3801654166 shutdown_connections 2023-10-26T01:37:38.557 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:38.554+0000 7fbb55100640 1 -- 172.21.15.169:0/3801654166 wait complete. 2023-10-26T01:37:38.578 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-26T01:37:38.801 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:38.799+0000 7fd4f5cfc640 1 -- 172.21.15.169:0/1975432195 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4f00aca30 msgr2=0x7fd4f00cdc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:38.801 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:38.799+0000 7fd4f5cfc640 1 --2- 172.21.15.169:0/1975432195 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4f00aca30 0x7fd4f00cdc50 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7fd4e8005fd0 tx=0x7fd4e802c2d0 comp rx=0 tx=0).stop 2023-10-26T01:37:38.801 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:38.799+0000 7fd4f5cfc640 1 -- 172.21.15.169:0/1975432195 shutdown_connections 2023-10-26T01:37:38.802 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:38.799+0000 7fd4f5cfc640 1 --2- 172.21.15.169:0/1975432195 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4f00aca30 0x7fd4f00cdc50 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:38.802 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:38.799+0000 7fd4f5cfc640 1 --2- 172.21.15.169:0/1975432195 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd4f015b620 0x7fd4f00cd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:38.802 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:38.799+0000 7fd4f5cfc640 1 -- 172.21.15.169:0/1975432195 >> 172.21.15.169:0/1975432195 conn(0x7fd4f0069930 msgr2=0x7fd4f0069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:38.802 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:38.799+0000 7fd4f5cfc640 1 -- 172.21.15.169:0/1975432195 shutdown_connections 2023-10-26T01:37:38.803 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:38.799+0000 7fd4f5cfc640 1 -- 172.21.15.169:0/1975432195 wait complete. 2023-10-26T01:37:38.805 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:38.801+0000 7fd4f5cfc640 1 Processor -- start 2023-10-26T01:37:38.805 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:38.801+0000 7fd4f5cfc640 1 -- start start 2023-10-26T01:37:38.805 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:38.802+0000 7fd4f5cfc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd4f00aca30 0x7fd4f00afa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:38.805 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:38.802+0000 7fd4f5cfc640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd4f00c9d00 0x7fd4f006ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:38.806 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:38.802+0000 7fd4f5cfc640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4f015b620 0x7fd4f0070ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:38.806 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:38.802+0000 7fd4f5cfc640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f006f510 con 0x7fd4f015b620 2023-10-26T01:37:38.806 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:38.802+0000 7fd4f5cfc640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f0071cc0 con 0x7fd4f00aca30 2023-10-26T01:37:38.806 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:38.802+0000 7fd4f5cfc640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f00ad470 con 0x7fd4f00c9d00 2023-10-26T01:37:38.806 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:38.802+0000 7fd4effff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4f015b620 0x7fd4f0070ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:38.807 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:38.802+0000 7fd4effff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4f015b620 0x7fd4f0070ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:47100/0 (socket says 172.21.15.169:47100) 2023-10-26T01:37:38.807 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:38.802+0000 7fd4effff640 1 -- 172.21.15.169:0/1907350857 learned_addr learned my addr 172.21.15.169:0/1907350857 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:37:38.807 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:38.802+0000 7fd4ef7fe640 1 --2- 172.21.15.169:0/1907350857 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd4f00aca30 0x7fd4f00afa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:38.807 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:38.802+0000 7fd4effff640 1 -- 172.21.15.169:0/1907350857 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd4f00c9d00 msgr2=0x7fd4f006ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:38.808 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:38.802+0000 7fd4effff640 1 --2- 172.21.15.169:0/1907350857 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd4f00c9d00 0x7fd4f006ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:38.808 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:38.802+0000 7fd4effff640 1 -- 172.21.15.169:0/1907350857 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd4f00aca30 msgr2=0x7fd4f00afa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:38.808 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:38.802+0000 7fd4effff640 1 --2- 172.21.15.169:0/1907350857 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd4f00aca30 0x7fd4f00afa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:38.808 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:38.802+0000 7fd4effff640 1 -- 172.21.15.169:0/1907350857 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4e8009900 con 0x7fd4f015b620 2023-10-26T01:37:38.808 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:38.803+0000 7fd4ef7fe640 1 --2- 172.21.15.169:0/1907350857 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd4f00aca30 0x7fd4f00afa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:37:38.808 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:38.803+0000 7fd4effff640 1 --2- 172.21.15.169:0/1907350857 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4f015b620 0x7fd4f0070ca0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7fd4e8037040 tx=0x7fd4e8003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:38.809 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:38.803+0000 7fd4ecff9640 1 -- 172.21.15.169:0/1907350857 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e8004240 con 0x7fd4f015b620 2023-10-26T01:37:38.809 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:38.803+0000 7fd4ecff9640 1 -- 172.21.15.169:0/1907350857 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd4e8004490 con 0x7fd4f015b620 2023-10-26T01:37:38.809 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:38.803+0000 7fd4ecff9640 1 -- 172.21.15.169:0/1907350857 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e8004840 con 0x7fd4f015b620 2023-10-26T01:37:38.809 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:38.803+0000 7fd4f5cfc640 1 -- 172.21.15.169:0/1907350857 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4f006f270 con 0x7fd4f015b620 2023-10-26T01:37:38.809 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:38.803+0000 7fd4f5cfc640 1 -- 172.21.15.169:0/1907350857 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4f00719e0 con 0x7fd4f015b620 2023-10-26T01:37:38.810 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:38.804+0000 7fd4f5cfc640 1 -- 172.21.15.169:0/1907350857 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4f00ad240 con 0x7fd4f015b620 2023-10-26T01:37:38.811 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:38.805+0000 7fd4ecff9640 1 -- 172.21.15.169:0/1907350857 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fd4e803b040 con 0x7fd4f015b620 2023-10-26T01:37:38.811 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:38.806+0000 7fd4ecff9640 1 --2- 172.21.15.169:0/1907350857 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd4d006ae40 0x7fd4d006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:38.811 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:38.806+0000 7fd4ecff9640 1 -- 172.21.15.169:0/1907350857 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fd4e80ac3b0 con 0x7fd4f015b620 2023-10-26T01:37:38.811 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:38.807+0000 7fd4ef7fe640 1 --2- 172.21.15.169:0/1907350857 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd4d006ae40 0x7fd4d006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:38.811 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:38.808+0000 7fd4ef7fe640 1 --2- 172.21.15.169:0/1907350857 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd4d006ae40 0x7fd4d006d300 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fd4e00058c0 tx=0x7fd4e0005c90 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:38.813 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:38.812+0000 7fd4ecff9640 1 -- 172.21.15.169:0/1907350857 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd4f00ad240 con 0x7fd4f015b620 2023-10-26T01:37:39.004 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:39.002+0000 7fd4f5cfc640 1 -- 172.21.15.169:0/1907350857 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fd4f00ad240 con 0x7fd4f015b620 2023-10-26T01:37:39.555 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:39.553+0000 7fd4ecff9640 1 -- 172.21.15.169:0/1907350857 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v118) v1 ==== 170+0+0 (secure 0 0 0) 0x7fd4f00ad240 con 0x7fd4f015b620 2023-10-26T01:37:39.555 INFO:teuthology.orchestra.run.smithi169.stderr:pool 'cephfs_data' created 2023-10-26T01:37:39.559 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:39.557+0000 7fd4f5cfc640 1 -- 172.21.15.169:0/1907350857 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd4d006ae40 msgr2=0x7fd4d006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:39.559 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:39.557+0000 7fd4f5cfc640 1 --2- 172.21.15.169:0/1907350857 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd4d006ae40 0x7fd4d006d300 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fd4e00058c0 tx=0x7fd4e0005c90 comp rx=0 tx=0).stop 2023-10-26T01:37:39.560 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:39.557+0000 7fd4f5cfc640 1 -- 172.21.15.169:0/1907350857 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4f015b620 msgr2=0x7fd4f0070ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:39.560 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:39.557+0000 7fd4f5cfc640 1 --2- 172.21.15.169:0/1907350857 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4f015b620 0x7fd4f0070ca0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7fd4e8037040 tx=0x7fd4e8003ba0 comp rx=0 tx=0).stop 2023-10-26T01:37:39.560 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:39.558+0000 7fd4f5cfc640 1 -- 172.21.15.169:0/1907350857 shutdown_connections 2023-10-26T01:37:39.561 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:39.558+0000 7fd4f5cfc640 1 --2- 172.21.15.169:0/1907350857 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd4d006ae40 0x7fd4d006d300 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:39.561 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:39.558+0000 7fd4f5cfc640 1 --2- 172.21.15.169:0/1907350857 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4f015b620 0x7fd4f0070ca0 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:39.561 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:39.558+0000 7fd4f5cfc640 1 --2- 172.21.15.169:0/1907350857 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd4f00c9d00 0x7fd4f006ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:39.561 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:39.558+0000 7fd4f5cfc640 1 --2- 172.21.15.169:0/1907350857 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd4f00aca30 0x7fd4f00afa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:39.561 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:39.558+0000 7fd4f5cfc640 1 -- 172.21.15.169:0/1907350857 >> 172.21.15.169:0/1907350857 conn(0x7fd4f0069930 msgr2=0x7fd4f0151910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:39.561 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:39.558+0000 7fd4f5cfc640 1 -- 172.21.15.169:0/1907350857 shutdown_connections 2023-10-26T01:37:39.562 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:39.559+0000 7fd4f5cfc640 1 -- 172.21.15.169:0/1907350857 wait complete. 2023-10-26T01:37:39.591 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-26T01:37:39.789 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:39.787+0000 7f5416210640 1 -- 172.21.15.169:0/3851917934 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f54100aca30 msgr2=0x7f54100cdc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:39.790 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:39.787+0000 7f5416210640 1 --2- 172.21.15.169:0/3851917934 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f54100aca30 0x7f54100cdc50 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f541009b5a0 tx=0x7f540802c1d0 comp rx=0 tx=0).stop 2023-10-26T01:37:39.790 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:39.787+0000 7f5416210640 1 -- 172.21.15.169:0/3851917934 shutdown_connections 2023-10-26T01:37:39.790 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:39.787+0000 7f5416210640 1 --2- 172.21.15.169:0/3851917934 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f54100aca30 0x7f54100cdc50 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:39.790 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:39.787+0000 7f5416210640 1 --2- 172.21.15.169:0/3851917934 >> v2:172.21.15.170:3300/0 conn(0x7f541015b620 0x7f54100cd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:39.790 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:39.787+0000 7f5416210640 1 -- 172.21.15.169:0/3851917934 >> 172.21.15.169:0/3851917934 conn(0x7f5410069930 msgr2=0x7f5410069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:39.791 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:39.787+0000 7f5416210640 1 -- 172.21.15.169:0/3851917934 shutdown_connections 2023-10-26T01:37:39.791 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:39.787+0000 7f5416210640 1 -- 172.21.15.169:0/3851917934 wait complete. 2023-10-26T01:37:39.792 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:39.789+0000 7f5416210640 1 Processor -- start 2023-10-26T01:37:39.792 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:39.789+0000 7f5416210640 1 -- start start 2023-10-26T01:37:39.792 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:39.790+0000 7f5416210640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f54100aca30 0x7f541015bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:39.793 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:39.790+0000 7f5416210640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f54100c9d00 0x7f5410153810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:39.793 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:39.790+0000 7f5416210640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f541015b620 0x7f5410150080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:39.793 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:39.790+0000 7f5416210640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f541015c1b0 con 0x7f54100c9d00 2023-10-26T01:37:39.793 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:39.790+0000 7f5416210640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f541006f650 con 0x7f541015b620 2023-10-26T01:37:39.793 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:39.790+0000 7f5416210640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f541006fc10 con 0x7f54100aca30 2023-10-26T01:37:39.794 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:39.790+0000 7f5406ffd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f54100c9d00 0x7f5410153810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:39.794 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:39.790+0000 7f5406ffd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f54100c9d00 0x7f5410153810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:47104/0 (socket says 172.21.15.169:47104) 2023-10-26T01:37:39.794 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:39.790+0000 7f5406ffd640 1 -- 172.21.15.169:0/643513843 learned_addr learned my addr 172.21.15.169:0/643513843 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:37:39.795 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:39.791+0000 7f540ffff640 1 --2- 172.21.15.169:0/643513843 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f541015b620 0x7f5410150080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:39.796 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:39.791+0000 7f540f7fe640 1 --2- 172.21.15.169:0/643513843 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f54100aca30 0x7f541015bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:39.796 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:39.791+0000 7f5406ffd640 1 -- 172.21.15.169:0/643513843 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f54100aca30 msgr2=0x7f541015bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:39.796 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:39.791+0000 7f5406ffd640 1 --2- 172.21.15.169:0/643513843 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f54100aca30 0x7f541015bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:39.796 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:39.791+0000 7f5406ffd640 1 -- 172.21.15.169:0/643513843 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f541015b620 msgr2=0x7f5410150080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:39.797 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:39.791+0000 7f5406ffd640 1 --2- 172.21.15.169:0/643513843 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f541015b620 0x7f5410150080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:39.797 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:39.791+0000 7f5406ffd640 1 -- 172.21.15.169:0/643513843 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5408009900 con 0x7f54100c9d00 2023-10-26T01:37:39.797 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:39.791+0000 7f5406ffd640 1 --2- 172.21.15.169:0/643513843 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f54100c9d00 0x7f5410153810 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f53f800c460 tx=0x7f53f800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:39.797 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:39.791+0000 7f540cff9640 1 -- 172.21.15.169:0/643513843 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53f8012830 con 0x7f54100c9d00 2023-10-26T01:37:39.797 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:39.792+0000 7f540cff9640 1 -- 172.21.15.169:0/643513843 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f53f8012ac0 con 0x7f54100c9d00 2023-10-26T01:37:39.798 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:39.792+0000 7f540cff9640 1 -- 172.21.15.169:0/643513843 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53f8011380 con 0x7f54100c9d00 2023-10-26T01:37:39.798 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:39.792+0000 7f5416210640 1 -- 172.21.15.169:0/643513843 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5410153d50 con 0x7f54100c9d00 2023-10-26T01:37:39.798 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:39.792+0000 7f5416210640 1 -- 172.21.15.169:0/643513843 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5410150d40 con 0x7f54100c9d00 2023-10-26T01:37:39.798 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:39.793+0000 7f5416210640 1 -- 172.21.15.169:0/643513843 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f541004db10 con 0x7f54100c9d00 2023-10-26T01:37:39.798 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:39.794+0000 7f540cff9640 1 -- 172.21.15.169:0/643513843 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f53f80114e0 con 0x7f54100c9d00 2023-10-26T01:37:39.799 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:39.795+0000 7f540cff9640 1 --2- 172.21.15.169:0/643513843 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f53f006ae70 0x7f53f006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:39.799 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:39.795+0000 7f540cff9640 1 -- 172.21.15.169:0/643513843 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f53f8089d00 con 0x7f54100c9d00 2023-10-26T01:37:39.799 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:39.796+0000 7f540f7fe640 1 --2- 172.21.15.169:0/643513843 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f53f006ae70 0x7f53f006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:39.800 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:39.797+0000 7f540f7fe640 1 --2- 172.21.15.169:0/643513843 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f53f006ae70 0x7f53f006d330 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f54000058c0 tx=0x7f5400005850 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:39.800 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:39.799+0000 7f540cff9640 1 -- 172.21.15.169:0/643513843 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f53f8054df0 con 0x7f54100c9d00 2023-10-26T01:37:40.027 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:40.025+0000 7f5416210640 1 -- 172.21.15.169:0/643513843 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f5410069e60 con 0x7f54100c9d00 2023-10-26T01:37:40.583 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:40.581+0000 7f540cff9640 1 -- 172.21.15.169:0/643513843 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 14 and data pool 15 v55) v1 ==== 174+0+0 (secure 0 0 0) 0x7f53f8058410 con 0x7f54100c9d00 2023-10-26T01:37:40.584 INFO:teuthology.orchestra.run.smithi169.stderr:new fs with metadata pool 14 and data pool 15 2023-10-26T01:37:40.588 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:40.586+0000 7f5416210640 1 -- 172.21.15.169:0/643513843 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f53f006ae70 msgr2=0x7f53f006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:40.588 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:40.586+0000 7f5416210640 1 --2- 172.21.15.169:0/643513843 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f53f006ae70 0x7f53f006d330 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f54000058c0 tx=0x7f5400005850 comp rx=0 tx=0).stop 2023-10-26T01:37:40.588 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:40.586+0000 7f5416210640 1 -- 172.21.15.169:0/643513843 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f54100c9d00 msgr2=0x7f5410153810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:40.588 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:40.586+0000 7f5416210640 1 --2- 172.21.15.169:0/643513843 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f54100c9d00 0x7f5410153810 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f53f800c460 tx=0x7f53f800c920 comp rx=0 tx=0).stop 2023-10-26T01:37:40.589 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:40.586+0000 7f5416210640 1 -- 172.21.15.169:0/643513843 shutdown_connections 2023-10-26T01:37:40.589 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:40.586+0000 7f5416210640 1 --2- 172.21.15.169:0/643513843 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f53f006ae70 0x7f53f006d330 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:40.589 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:40.586+0000 7f5416210640 1 --2- 172.21.15.169:0/643513843 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f54100c9d00 0x7f5410153810 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:40.589 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:40.586+0000 7f5416210640 1 --2- 172.21.15.169:0/643513843 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f541015b620 0x7f5410150080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:40.589 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:40.586+0000 7f5416210640 1 --2- 172.21.15.169:0/643513843 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f54100aca30 0x7f541015bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:40.590 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:40.586+0000 7f5416210640 1 -- 172.21.15.169:0/643513843 >> 172.21.15.169:0/643513843 conn(0x7f5410069930 msgr2=0x7f54100cd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:40.590 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:40.588+0000 7f5416210640 1 -- 172.21.15.169:0/643513843 shutdown_connections 2023-10-26T01:37:40.590 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:40.588+0000 7f5416210640 1 -- 172.21.15.169:0/643513843 wait complete. 2023-10-26T01:37:40.625 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:37:40.806 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:40.804+0000 7f9189ce1640 1 -- 172.21.15.169:0/1262572784 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f91840ac930 msgr2=0x7f91840cd6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:40.806 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:40.804+0000 7f9189ce1640 1 --2- 172.21.15.169:0/1262572784 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f91840ac930 0x7f91840cd6f0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f918406be60 tx=0x7f916c02bcf0 comp rx=0 tx=0).stop 2023-10-26T01:37:40.806 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:40.804+0000 7f9189ce1640 1 -- 172.21.15.169:0/1262572784 shutdown_connections 2023-10-26T01:37:40.807 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:40.804+0000 7f9189ce1640 1 --2- 172.21.15.169:0/1262572784 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f91840ac930 0x7f91840cd6f0 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:40.807 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:40.804+0000 7f9189ce1640 1 --2- 172.21.15.169:0/1262572784 >> v2:172.21.15.170:3300/0 conn(0x7f91841592d0 0x7f91840aa3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:40.807 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:40.804+0000 7f9189ce1640 1 -- 172.21.15.169:0/1262572784 >> 172.21.15.169:0/1262572784 conn(0x7f9184069930 msgr2=0x7f9184069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:40.807 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:40.804+0000 7f9189ce1640 1 -- 172.21.15.169:0/1262572784 shutdown_connections 2023-10-26T01:37:40.807 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:40.804+0000 7f9189ce1640 1 -- 172.21.15.169:0/1262572784 wait complete. 2023-10-26T01:37:40.809 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:40.805+0000 7f9189ce1640 1 Processor -- start 2023-10-26T01:37:40.810 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:40.806+0000 7f9189ce1640 1 -- start start 2023-10-26T01:37:40.810 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:40.806+0000 7f9189ce1640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f91840ac930 0x7f91841f0950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:40.810 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:40.806+0000 7f9189ce1640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f91840cdc30 0x7f91841e40a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:40.810 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:40.806+0000 7f9189ce1640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91841592d0 0x7f91841e45e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:40.811 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:40.807+0000 7f9189ce1640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f918414fca0 con 0x7f91840cdc30 2023-10-26T01:37:40.811 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:40.807+0000 7f9189ce1640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91841f0670 con 0x7f91841592d0 2023-10-26T01:37:40.811 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:40.807+0000 7f9189ce1640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f918414fef0 con 0x7f91840ac930 2023-10-26T01:37:40.811 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:40.807+0000 7f917bfff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f91840cdc30 0x7f91841e40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:40.811 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:40.807+0000 7f917bfff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f91840cdc30 0x7f91841e40a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:47118/0 (socket says 172.21.15.169:47118) 2023-10-26T01:37:40.812 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:40.807+0000 7f917bfff640 1 -- 172.21.15.169:0/4223816927 learned_addr learned my addr 172.21.15.169:0/4223816927 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:37:40.812 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:40.807+0000 7f91837fe640 1 --2- 172.21.15.169:0/4223816927 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f91840ac930 0x7f91841f0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:40.812 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:40.807+0000 7f917bfff640 1 -- 172.21.15.169:0/4223816927 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f91840ac930 msgr2=0x7f91841f0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:40.812 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:40.807+0000 7f917bfff640 1 --2- 172.21.15.169:0/4223816927 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f91840ac930 0x7f91841f0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:40.812 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:40.807+0000 7f917bfff640 1 -- 172.21.15.169:0/4223816927 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91841592d0 msgr2=0x7f91841e45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:40.813 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:40.807+0000 7f917bfff640 1 --2- 172.21.15.169:0/4223816927 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91841592d0 0x7f91841e45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:40.813 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:40.807+0000 7f917bfff640 1 -- 172.21.15.169:0/4223816927 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f916c009490 con 0x7f91840cdc30 2023-10-26T01:37:40.813 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:40.808+0000 7f917bfff640 1 --2- 172.21.15.169:0/4223816927 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f91840cdc30 0x7f91841e40a0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f916c000c00 tx=0x7f916c02c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:40.813 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:40.808+0000 7f9180ff9640 1 -- 172.21.15.169:0/4223816927 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f916c003b80 con 0x7f91840cdc30 2023-10-26T01:37:40.814 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:40.808+0000 7f9180ff9640 1 -- 172.21.15.169:0/4223816927 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f916c003df0 con 0x7f91840cdc30 2023-10-26T01:37:40.814 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:40.808+0000 7f9180ff9640 1 -- 172.21.15.169:0/4223816927 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f916c0346a0 con 0x7f91840cdc30 2023-10-26T01:37:40.814 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:40.808+0000 7f9189ce1640 1 -- 172.21.15.169:0/4223816927 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91841501d0 con 0x7f91840cdc30 2023-10-26T01:37:40.814 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:40.808+0000 7f9189ce1640 1 -- 172.21.15.169:0/4223816927 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9184150790 con 0x7f91840cdc30 2023-10-26T01:37:40.815 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:40.809+0000 7f9189ce1640 1 -- 172.21.15.169:0/4223816927 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f918404db10 con 0x7f91840cdc30 2023-10-26T01:37:40.815 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:40.810+0000 7f9180ff9640 1 -- 172.21.15.169:0/4223816927 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f916c03b040 con 0x7f91840cdc30 2023-10-26T01:37:40.816 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:40.811+0000 7f9180ff9640 1 --2- 172.21.15.169:0/4223816927 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f916406ae40 0x7f916406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:40.816 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:40.811+0000 7f9180ff9640 1 -- 172.21.15.169:0/4223816927 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f916c0ab950 con 0x7f91840cdc30 2023-10-26T01:37:40.816 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:40.812+0000 7f91837fe640 1 --2- 172.21.15.169:0/4223816927 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f916406ae40 0x7f916406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:40.816 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:40.813+0000 7f91837fe640 1 --2- 172.21.15.169:0/4223816927 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f916406ae40 0x7f916406d300 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f9174005d70 tx=0x7f9174005d00 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:40.817 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:40.816+0000 7f9180ff9640 1 -- 172.21.15.169:0/4223816927 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f916c076960 con 0x7f91840cdc30 2023-10-26T01:37:41.001 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:41.000+0000 7f9189ce1640 1 -- 172.21.15.169:0/4223816927 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f918406b8a0 con 0x7f91840cdc30 2023-10-26T01:37:41.003 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:41.002+0000 7f9180ff9640 1 -- 172.21.15.169:0/4223816927 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v119) v1 ==== 74+0+16581 (secure 0 0 0) 0x7f916c039090 con 0x7f91840cdc30 2023-10-26T01:37:41.004 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:37:41.004 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":119,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:37:40.027291+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-10-26T01:37:37.959102+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-10-26T01:37:39.053240+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:37:41.007 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:41.005+0000 7f9189ce1640 1 -- 172.21.15.169:0/4223816927 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f916406ae40 msgr2=0x7f916406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:41.007 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:41.005+0000 7f9189ce1640 1 --2- 172.21.15.169:0/4223816927 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f916406ae40 0x7f916406d300 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f9174005d70 tx=0x7f9174005d00 comp rx=0 tx=0).stop 2023-10-26T01:37:41.007 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:41.005+0000 7f9189ce1640 1 -- 172.21.15.169:0/4223816927 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f91840cdc30 msgr2=0x7f91841e40a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:41.008 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:41.005+0000 7f9189ce1640 1 --2- 172.21.15.169:0/4223816927 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f91840cdc30 0x7f91841e40a0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f916c000c00 tx=0x7f916c02c920 comp rx=0 tx=0).stop 2023-10-26T01:37:41.008 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:41.005+0000 7f9189ce1640 1 -- 172.21.15.169:0/4223816927 shutdown_connections 2023-10-26T01:37:41.008 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:41.005+0000 7f9189ce1640 1 --2- 172.21.15.169:0/4223816927 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f916406ae40 0x7f916406d300 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:41.008 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:41.005+0000 7f9189ce1640 1 --2- 172.21.15.169:0/4223816927 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f91840cdc30 0x7f91841e40a0 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:41.008 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:41.005+0000 7f9189ce1640 1 --2- 172.21.15.169:0/4223816927 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91841592d0 0x7f91841e45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:41.009 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:41.005+0000 7f9189ce1640 1 --2- 172.21.15.169:0/4223816927 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f91840ac930 0x7f91841f0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:41.009 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:41.005+0000 7f9189ce1640 1 -- 172.21.15.169:0/4223816927 >> 172.21.15.169:0/4223816927 conn(0x7f9184069930 msgr2=0x7f918406ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:41.009 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:41.005+0000 7f9189ce1640 1 -- 172.21.15.169:0/4223816927 shutdown_connections 2023-10-26T01:37:41.009 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:41.005+0000 7f9189ce1640 1 -- 172.21.15.169:0/4223816927 wait complete. 2023-10-26T01:37:41.025 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:37:41.240 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:41.238+0000 7f43c6742640 1 -- 172.21.15.169:0/2698497592 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f43ac003ef0 msgr2=0x7f43ac0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:41.240 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:41.238+0000 7f43c6742640 1 --2- 172.21.15.169:0/2698497592 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f43ac003ef0 0x7f43ac0047c0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f43b0009b50 tx=0x7f43ac0057f0 comp rx=0 tx=0).stop 2023-10-26T01:37:41.241 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:41.238+0000 7f43c5f41640 1 -- 172.21.15.169:0/2698497592 reap_dead start 2023-10-26T01:37:41.241 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:41.238+0000 7f43c6742640 1 -- 172.21.15.169:0/2698497592 shutdown_connections 2023-10-26T01:37:41.241 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:41.238+0000 7f43c6742640 1 -- 172.21.15.169:0/2698497592 >> 172.21.15.169:0/2698497592 conn(0x7f43c00aedd0 msgr2=0x7f43c00af1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:41.241 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:41.238+0000 7f43c6742640 1 -- 172.21.15.169:0/2698497592 shutdown_connections 2023-10-26T01:37:41.242 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:41.238+0000 7f43c6742640 1 -- 172.21.15.169:0/2698497592 wait complete. 2023-10-26T01:37:41.248 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:41.240+0000 7f43c6742640 1 Processor -- start 2023-10-26T01:37:41.248 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:41.240+0000 7f43c6742640 1 -- start start 2023-10-26T01:37:41.249 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:41.241+0000 7f43c6742640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f43ac003ef0 0x7f43c015c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:41.249 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:41.241+0000 7f43c6742640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f43c0070120 0x7f43c006e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:41.249 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:41.241+0000 7f43c6742640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f43c0071030 0x7f43c01e2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:41.249 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:41.241+0000 7f43c6742640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43c006fd20 con 0x7f43ac003ef0 2023-10-26T01:37:41.249 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:41.241+0000 7f43c6742640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43c006ee70 con 0x7f43c0070120 2023-10-26T01:37:41.250 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:41.241+0000 7f43c6742640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43c006f0a0 con 0x7f43c0071030 2023-10-26T01:37:41.250 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:41.241+0000 7f43c5740640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f43ac003ef0 0x7f43c015c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:41.250 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:41.242+0000 7f43c5740640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f43ac003ef0 0x7f43c015c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:47124/0 (socket says 172.21.15.169:47124) 2023-10-26T01:37:41.250 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:41.242+0000 7f43c5740640 1 -- 172.21.15.169:0/287811721 learned_addr learned my addr 172.21.15.169:0/287811721 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:37:41.250 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:41.242+0000 7f43c4f3f640 1 --2- 172.21.15.169:0/287811721 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f43c0070120 0x7f43c006e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:41.251 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:41.242+0000 7f43c5740640 1 -- 172.21.15.169:0/287811721 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f43c0071030 msgr2=0x7f43c01e2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:41.251 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:41.242+0000 7f43c5740640 1 --2- 172.21.15.169:0/287811721 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f43c0071030 0x7f43c01e2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:41.251 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:41.242+0000 7f43c5740640 1 -- 172.21.15.169:0/287811721 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f43c0070120 msgr2=0x7f43c006e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:41.251 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:41.242+0000 7f43c5740640 1 --2- 172.21.15.169:0/287811721 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f43c0070120 0x7f43c006e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:41.251 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:41.242+0000 7f43c5740640 1 -- 172.21.15.169:0/287811721 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43c006f380 con 0x7f43ac003ef0 2023-10-26T01:37:41.252 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:41.242+0000 7f43c4f3f640 1 --2- 172.21.15.169:0/287811721 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f43c0070120 0x7f43c006e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:37:41.252 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:41.242+0000 7f43c5740640 1 --2- 172.21.15.169:0/287811721 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f43ac003ef0 0x7f43c015c4a0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f43bc00e970 tx=0x7f43bc00ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:41.252 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:41.242+0000 7f43b5ffb640 1 -- 172.21.15.169:0/287811721 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43bc00ccc0 con 0x7f43ac003ef0 2023-10-26T01:37:41.252 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:41.243+0000 7f43b5ffb640 1 -- 172.21.15.169:0/287811721 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f43bc00ce20 con 0x7f43ac003ef0 2023-10-26T01:37:41.252 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:41.243+0000 7f43b5ffb640 1 -- 172.21.15.169:0/287811721 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43bc005280 con 0x7f43ac003ef0 2023-10-26T01:37:41.253 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:41.243+0000 7f43c6742640 1 -- 172.21.15.169:0/287811721 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43c006f660 con 0x7f43ac003ef0 2023-10-26T01:37:41.253 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:41.243+0000 7f43c6742640 1 -- 172.21.15.169:0/287811721 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43c00714a0 con 0x7f43ac003ef0 2023-10-26T01:37:41.253 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:41.244+0000 7f43c6742640 1 -- 172.21.15.169:0/287811721 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43c0073c10 con 0x7f43ac003ef0 2023-10-26T01:37:41.253 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:41.245+0000 7f43b5ffb640 1 -- 172.21.15.169:0/287811721 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f43bc0053e0 con 0x7f43ac003ef0 2023-10-26T01:37:41.253 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:41.245+0000 7f43b5ffb640 1 --2- 172.21.15.169:0/287811721 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f43a406ae40 0x7f43a406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:41.254 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:41.246+0000 7f43b5ffb640 1 -- 172.21.15.169:0/287811721 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f43bc014070 con 0x7f43ac003ef0 2023-10-26T01:37:41.254 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:41.246+0000 7f43c4f3f640 1 --2- 172.21.15.169:0/287811721 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f43a406ae40 0x7f43a406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:41.254 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:41.247+0000 7f43c4f3f640 1 --2- 172.21.15.169:0/287811721 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f43a406ae40 0x7f43a406d300 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f43b0036000 tx=0x7f43b0035040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:41.255 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:41.250+0000 7f43b5ffb640 1 -- 172.21.15.169:0/287811721 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f43bc054fa0 con 0x7f43ac003ef0 2023-10-26T01:37:41.442 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:41.440+0000 7f43c6742640 1 -- 172.21.15.169:0/287811721 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f43c0071920 con 0x7f43ac003ef0 2023-10-26T01:37:41.444 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:41.442+0000 7f43b5ffb640 1 -- 172.21.15.169:0/287811721 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v119) v1 ==== 74+0+16581 (secure 0 0 0) 0x7f43bc0585c0 con 0x7f43ac003ef0 2023-10-26T01:37:41.444 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:37:41.445 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":119,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:37:40.027291+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-10-26T01:37:37.959102+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-10-26T01:37:39.053240+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:37:41.447 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:41.445+0000 7f43c6742640 1 -- 172.21.15.169:0/287811721 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f43a406ae40 msgr2=0x7f43a406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:41.447 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:41.445+0000 7f43c6742640 1 --2- 172.21.15.169:0/287811721 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f43a406ae40 0x7f43a406d300 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f43b0036000 tx=0x7f43b0035040 comp rx=0 tx=0).stop 2023-10-26T01:37:41.447 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:41.445+0000 7f43c6742640 1 -- 172.21.15.169:0/287811721 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f43ac003ef0 msgr2=0x7f43c015c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:41.447 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:41.445+0000 7f43c6742640 1 --2- 172.21.15.169:0/287811721 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f43ac003ef0 0x7f43c015c4a0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f43bc00e970 tx=0x7f43bc00ee30 comp rx=0 tx=0).stop 2023-10-26T01:37:41.447 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:41.445+0000 7f43c6742640 1 -- 172.21.15.169:0/287811721 shutdown_connections 2023-10-26T01:37:41.448 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:41.445+0000 7f43c6742640 1 --2- 172.21.15.169:0/287811721 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f43a406ae40 0x7f43a406d300 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:41.448 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:41.445+0000 7f43c6742640 1 --2- 172.21.15.169:0/287811721 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f43c0071030 0x7f43c01e2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:41.448 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:41.445+0000 7f43c6742640 1 --2- 172.21.15.169:0/287811721 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f43c0070120 0x7f43c006e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:41.448 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:41.445+0000 7f43c6742640 1 --2- 172.21.15.169:0/287811721 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f43ac003ef0 0x7f43c015c4a0 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:41.448 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:41.445+0000 7f43c6742640 1 -- 172.21.15.169:0/287811721 >> 172.21.15.169:0/287811721 conn(0x7f43c00aedd0 msgr2=0x7f43c00ab090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:41.449 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:41.445+0000 7f43c6742640 1 -- 172.21.15.169:0/287811721 shutdown_connections 2023-10-26T01:37:41.449 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:41.445+0000 7f43c6742640 1 -- 172.21.15.169:0/287811721 wait complete. 2023-10-26T01:37:41.466 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-26T01:37:41.693 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:41.691+0000 7f315dc39640 1 -- 172.21.15.169:0/3250354089 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f31581597c0 msgr2=0x7f31580ac200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:41.694 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:41.691+0000 7f315dc39640 1 --2- 172.21.15.169:0/3250354089 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f31581597c0 0x7f31580ac200 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f3140005fd0 tx=0x7f314002be20 comp rx=0 tx=0).stop 2023-10-26T01:37:41.694 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:41.691+0000 7f315dc39640 1 -- 172.21.15.169:0/3250354089 shutdown_connections 2023-10-26T01:37:41.694 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:41.691+0000 7f315dc39640 1 --2- 172.21.15.169:0/3250354089 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f31581597c0 0x7f31580ac200 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:41.695 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:41.691+0000 7f315dc39640 1 -- 172.21.15.169:0/3250354089 >> 172.21.15.169:0/3250354089 conn(0x7f3158069930 msgr2=0x7f3158069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:41.695 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:41.691+0000 7f315dc39640 1 -- 172.21.15.169:0/3250354089 shutdown_connections 2023-10-26T01:37:41.695 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:41.692+0000 7f315dc39640 1 -- 172.21.15.169:0/3250354089 wait complete. 2023-10-26T01:37:41.696 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:41.694+0000 7f315dc39640 1 Processor -- start 2023-10-26T01:37:41.696 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:41.694+0000 7f315dc39640 1 -- start start 2023-10-26T01:37:41.697 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:41.695+0000 7f315dc39640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f31580b0c80 0x7f31580ad910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:41.698 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:41.695+0000 7f315dc39640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f31581593e0 0x7f3158149b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:41.698 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:41.695+0000 7f315dc39640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f31581597c0 0x7f315806f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:41.698 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:41.695+0000 7f315dc39640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f315814a040 con 0x7f31581593e0 2023-10-26T01:37:41.698 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:41.695+0000 7f315dc39640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3158156ac0 con 0x7f31580b0c80 2023-10-26T01:37:41.699 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:41.695+0000 7f315dc39640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f315806fa10 con 0x7f31581597c0 2023-10-26T01:37:41.699 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:41.695+0000 7f3156ffd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f31581593e0 0x7f3158149b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:41.699 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:41.695+0000 7f3156ffd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f31581593e0 0x7f3158149b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:47152/0 (socket says 172.21.15.169:47152) 2023-10-26T01:37:41.699 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:41.695+0000 7f3156ffd640 1 -- 172.21.15.169:0/858934917 learned_addr learned my addr 172.21.15.169:0/858934917 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:37:41.699 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:41.696+0000 7f31577fe640 1 --2- 172.21.15.169:0/858934917 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f31580b0c80 0x7f31580ad910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:41.700 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:41.696+0000 7f3156ffd640 1 -- 172.21.15.169:0/858934917 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f31581597c0 msgr2=0x7f315806f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:41.700 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:41.696+0000 7f3156ffd640 1 --2- 172.21.15.169:0/858934917 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f31581597c0 0x7f315806f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:41.700 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:41.696+0000 7f3156ffd640 1 -- 172.21.15.169:0/858934917 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f31580b0c80 msgr2=0x7f31580ad910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:41.700 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:41.696+0000 7f3156ffd640 1 --2- 172.21.15.169:0/858934917 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f31580b0c80 0x7f31580ad910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:41.701 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:41.696+0000 7f3156ffd640 1 -- 172.21.15.169:0/858934917 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3140009560 con 0x7f31581593e0 2023-10-26T01:37:41.701 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:41.696+0000 7f31577fe640 1 --2- 172.21.15.169:0/858934917 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f31580b0c80 0x7f31580ad910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:37:41.701 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:41.696+0000 7f3156ffd640 1 --2- 172.21.15.169:0/858934917 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f31581593e0 0x7f3158149b00 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f3140006010 tx=0x7f314002ddf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:41.702 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:41.696+0000 7f314ffff640 1 -- 172.21.15.169:0/858934917 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3140005650 con 0x7f31581593e0 2023-10-26T01:37:41.703 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:41.696+0000 7f314ffff640 1 -- 172.21.15.169:0/858934917 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f31400038f0 con 0x7f31581593e0 2023-10-26T01:37:41.703 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:41.696+0000 7f314ffff640 1 -- 172.21.15.169:0/858934917 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3140003ca0 con 0x7f31581593e0 2023-10-26T01:37:41.703 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:41.696+0000 7f315dc39640 1 -- 172.21.15.169:0/858934917 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31581f5630 con 0x7f31581593e0 2023-10-26T01:37:41.703 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:41.697+0000 7f315dc39640 1 -- 172.21.15.169:0/858934917 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31581f5bf0 con 0x7f31581593e0 2023-10-26T01:37:41.704 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:41.698+0000 7f315dc39640 1 -- 172.21.15.169:0/858934917 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3158156c80 con 0x7f31581593e0 2023-10-26T01:37:41.704 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:41.699+0000 7f314ffff640 1 -- 172.21.15.169:0/858934917 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f314003e450 con 0x7f31581593e0 2023-10-26T01:37:41.704 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:41.699+0000 7f314ffff640 1 --2- 172.21.15.169:0/858934917 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f313806ae40 0x7f313806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:41.704 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:41.700+0000 7f314ffff640 1 -- 172.21.15.169:0/858934917 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f31400ac320 con 0x7f31581593e0 2023-10-26T01:37:41.705 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:41.700+0000 7f31577fe640 1 --2- 172.21.15.169:0/858934917 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f313806ae40 0x7f313806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:41.705 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:41.701+0000 7f31577fe640 1 --2- 172.21.15.169:0/858934917 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f313806ae40 0x7f313806d300 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f3148004520 tx=0x7f3148009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:41.705 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:41.704+0000 7f314ffff640 1 -- 172.21.15.169:0/858934917 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f31400775a0 con 0x7f31581593e0 2023-10-26T01:37:41.932 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:41.930+0000 7f315dc39640 1 -- 172.21.15.169:0/858934917 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f315804db10 con 0x7f31581593e0 2023-10-26T01:37:42.637 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:42.635+0000 7f314ffff640 1 -- 172.21.15.169:0/858934917 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v58) v1 ==== 118+0+0 (secure 0 0 0) 0x7f314007abc0 con 0x7f31581593e0 2023-10-26T01:37:42.640 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:42.638+0000 7f315dc39640 1 -- 172.21.15.169:0/858934917 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f313806ae40 msgr2=0x7f313806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:42.640 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:42.638+0000 7f315dc39640 1 --2- 172.21.15.169:0/858934917 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f313806ae40 0x7f313806d300 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f3148004520 tx=0x7f3148009040 comp rx=0 tx=0).stop 2023-10-26T01:37:42.641 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:42.638+0000 7f315dc39640 1 -- 172.21.15.169:0/858934917 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f31581593e0 msgr2=0x7f3158149b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:42.641 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:42.638+0000 7f315dc39640 1 --2- 172.21.15.169:0/858934917 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f31581593e0 0x7f3158149b00 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f3140006010 tx=0x7f314002ddf0 comp rx=0 tx=0).stop 2023-10-26T01:37:42.641 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:42.638+0000 7f315dc39640 1 -- 172.21.15.169:0/858934917 shutdown_connections 2023-10-26T01:37:42.641 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:42.638+0000 7f315dc39640 1 --2- 172.21.15.169:0/858934917 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f313806ae40 0x7f313806d300 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:42.641 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:42.638+0000 7f315dc39640 1 --2- 172.21.15.169:0/858934917 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f31581593e0 0x7f3158149b00 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:42.642 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:42.638+0000 7f315dc39640 1 --2- 172.21.15.169:0/858934917 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f31581597c0 0x7f315806f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:42.642 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:42.638+0000 7f315dc39640 1 --2- 172.21.15.169:0/858934917 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f31580b0c80 0x7f31580ad910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:42.642 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:42.638+0000 7f315dc39640 1 -- 172.21.15.169:0/858934917 >> 172.21.15.169:0/858934917 conn(0x7f3158069930 msgr2=0x7f3158154900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:42.642 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:42.639+0000 7f315dc39640 1 -- 172.21.15.169:0/858934917 shutdown_connections 2023-10-26T01:37:42.643 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:42.639+0000 7f315dc39640 1 -- 172.21.15.169:0/858934917 wait complete. 2023-10-26T01:37:42.659 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-26T01:37:42.660 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-26T01:37:42.882 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:42.880+0000 7f7a524f8640 1 -- 172.21.15.169:0/1797555045 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7a4c158200 msgr2=0x7f7a4c06f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:42.883 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:42.880+0000 7f7a524f8640 1 --2- 172.21.15.169:0/1797555045 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7a4c158200 0x7f7a4c06f090 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c0b00a0 tx=0x7f7a4802bc80 comp rx=0 tx=0).stop 2023-10-26T01:37:42.883 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:42.880+0000 7f7a524f8640 1 -- 172.21.15.169:0/1797555045 shutdown_connections 2023-10-26T01:37:42.883 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:42.880+0000 7f7a524f8640 1 --2- 172.21.15.169:0/1797555045 >> v2:172.21.15.170:3300/0 conn(0x7f7a4c069a10 0x7f7a4c069e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:42.884 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:42.880+0000 7f7a524f8640 1 --2- 172.21.15.169:0/1797555045 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7a4c158200 0x7f7a4c06f090 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:42.884 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:42.880+0000 7f7a524f8640 1 -- 172.21.15.169:0/1797555045 >> 172.21.15.169:0/1797555045 conn(0x7f7a4c0af110 msgr2=0x7f7a4c0af520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:42.884 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:42.880+0000 7f7a524f8640 1 -- 172.21.15.169:0/1797555045 shutdown_connections 2023-10-26T01:37:42.884 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:42.880+0000 7f7a524f8640 1 -- 172.21.15.169:0/1797555045 wait complete. 2023-10-26T01:37:42.886 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:42.882+0000 7f7a524f8640 1 Processor -- start 2023-10-26T01:37:42.886 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:42.882+0000 7f7a524f8640 1 -- start start 2023-10-26T01:37:42.886 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:42.883+0000 7f7a524f8640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7a4c069a10 0x7f7a4c1f21b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:42.887 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:42.883+0000 7f7a524f8640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7a4c06bbd0 0x7f7a4c142550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:42.887 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:42.883+0000 7f7a524f8640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7a4c158200 0x7f7a4c153840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:42.887 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:42.883+0000 7f7a524f8640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a4c14e010 con 0x7f7a4c06bbd0 2023-10-26T01:37:42.887 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:42.883+0000 7f7a524f8640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a4c1f26f0 con 0x7f7a4c069a10 2023-10-26T01:37:42.887 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:42.883+0000 7f7a524f8640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a4c142a90 con 0x7f7a4c158200 2023-10-26T01:37:42.888 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:42.883+0000 7f7a50cf5640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7a4c06bbd0 0x7f7a4c142550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:42.888 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:42.884+0000 7f7a50cf5640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7a4c06bbd0 0x7f7a4c142550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:47172/0 (socket says 172.21.15.169:47172) 2023-10-26T01:37:42.888 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:42.884+0000 7f7a50cf5640 1 -- 172.21.15.169:0/2924997156 learned_addr learned my addr 172.21.15.169:0/2924997156 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:37:42.888 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:42.884+0000 7f7a514f6640 1 --2- 172.21.15.169:0/2924997156 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7a4c069a10 0x7f7a4c1f21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:42.888 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:42.884+0000 7f7a50cf5640 1 -- 172.21.15.169:0/2924997156 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7a4c158200 msgr2=0x7f7a4c153840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:42.889 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:42.884+0000 7f7a50cf5640 1 --2- 172.21.15.169:0/2924997156 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7a4c158200 0x7f7a4c153840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:42.889 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:42.884+0000 7f7a50cf5640 1 -- 172.21.15.169:0/2924997156 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7a4c069a10 msgr2=0x7f7a4c1f21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:42.889 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:42.884+0000 7f7a50cf5640 1 --2- 172.21.15.169:0/2924997156 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7a4c069a10 0x7f7a4c1f21b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:42.889 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:42.884+0000 7f7a50cf5640 1 -- 172.21.15.169:0/2924997156 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a48009560 con 0x7f7a4c06bbd0 2023-10-26T01:37:42.889 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:42.884+0000 7f7a50cf5640 1 --2- 172.21.15.169:0/2924997156 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7a4c06bbd0 0x7f7a4c142550 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c00c400 tx=0x7f7a3c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:42.890 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:42.884+0000 7f7a41ffb640 1 -- 172.21.15.169:0/2924997156 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a3c0035f0 con 0x7f7a4c06bbd0 2023-10-26T01:37:42.890 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:42.884+0000 7f7a41ffb640 1 -- 172.21.15.169:0/2924997156 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7a3c003750 con 0x7f7a4c06bbd0 2023-10-26T01:37:42.890 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:42.884+0000 7f7a41ffb640 1 -- 172.21.15.169:0/2924997156 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a3c003b60 con 0x7f7a4c06bbd0 2023-10-26T01:37:42.890 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:42.884+0000 7f7a524f8640 1 -- 172.21.15.169:0/2924997156 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a4c1f28e0 con 0x7f7a4c06bbd0 2023-10-26T01:37:42.891 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:42.885+0000 7f7a524f8640 1 -- 172.21.15.169:0/2924997156 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a4c1f2ea0 con 0x7f7a4c06bbd0 2023-10-26T01:37:42.891 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:42.886+0000 7f7a524f8640 1 -- 172.21.15.169:0/2924997156 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a4c073c10 con 0x7f7a4c06bbd0 2023-10-26T01:37:42.892 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:42.886+0000 7f7a41ffb640 1 -- 172.21.15.169:0/2924997156 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f7a3c003cc0 con 0x7f7a4c06bbd0 2023-10-26T01:37:42.892 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:42.887+0000 7f7a41ffb640 1 --2- 172.21.15.169:0/2924997156 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7a3006ae40 0x7f7a3006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:42.892 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:42.887+0000 7f7a41ffb640 1 -- 172.21.15.169:0/2924997156 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7a3c088860 con 0x7f7a4c06bbd0 2023-10-26T01:37:42.892 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:42.888+0000 7f7a514f6640 1 --2- 172.21.15.169:0/2924997156 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7a3006ae40 0x7f7a3006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:42.892 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:42.889+0000 7f7a514f6640 1 --2- 172.21.15.169:0/2924997156 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7a3006ae40 0x7f7a3006d300 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f7a48035000 tx=0x7f7a48004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:42.893 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:42.892+0000 7f7a41ffb640 1 -- 172.21.15.169:0/2924997156 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7a3c053a50 con 0x7f7a4c06bbd0 2023-10-26T01:37:43.103 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:43.102+0000 7f7a524f8640 1 -- 172.21.15.169:0/2924997156 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f7a4c06b940 con 0x7f7a4c06bbd0 2023-10-26T01:37:44.040 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.038+0000 7f7a41ffb640 1 -- 172.21.15.169:0/2924997156 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v59) v1 ==== 122+0+0 (secure 0 0 0) 0x7f7a3c017020 con 0x7f7a4c06bbd0 2023-10-26T01:37:44.045 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.043+0000 7f7a524f8640 1 -- 172.21.15.169:0/2924997156 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7a3006ae40 msgr2=0x7f7a3006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:44.046 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.043+0000 7f7a524f8640 1 --2- 172.21.15.169:0/2924997156 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7a3006ae40 0x7f7a3006d300 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f7a48035000 tx=0x7f7a48004730 comp rx=0 tx=0).stop 2023-10-26T01:37:44.046 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.043+0000 7f7a524f8640 1 -- 172.21.15.169:0/2924997156 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7a4c06bbd0 msgr2=0x7f7a4c142550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:44.046 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.043+0000 7f7a524f8640 1 --2- 172.21.15.169:0/2924997156 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7a4c06bbd0 0x7f7a4c142550 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c00c400 tx=0x7f7a3c00c8c0 comp rx=0 tx=0).stop 2023-10-26T01:37:44.046 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.044+0000 7f7a524f8640 1 -- 172.21.15.169:0/2924997156 shutdown_connections 2023-10-26T01:37:44.046 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.044+0000 7f7a524f8640 1 --2- 172.21.15.169:0/2924997156 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7a3006ae40 0x7f7a3006d300 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:44.047 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.044+0000 7f7a524f8640 1 --2- 172.21.15.169:0/2924997156 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7a4c06bbd0 0x7f7a4c142550 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:44.047 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.044+0000 7f7a524f8640 1 --2- 172.21.15.169:0/2924997156 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7a4c158200 0x7f7a4c153840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:44.047 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.044+0000 7f7a524f8640 1 --2- 172.21.15.169:0/2924997156 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7a4c069a10 0x7f7a4c1f21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:44.047 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.044+0000 7f7a524f8640 1 -- 172.21.15.169:0/2924997156 >> 172.21.15.169:0/2924997156 conn(0x7f7a4c0af110 msgr2=0x7f7a4c157400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:44.047 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.044+0000 7f7a524f8640 1 -- 172.21.15.169:0/2924997156 shutdown_connections 2023-10-26T01:37:44.048 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.044+0000 7f7a524f8640 1 -- 172.21.15.169:0/2924997156 wait complete. 2023-10-26T01:37:44.068 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-26T01:37:44.069 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T01:37:44.289 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.287+0000 7f8324875640 1 -- 172.21.15.169:0/745151936 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8320158850 msgr2=0x7f83200cd720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:44.289 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.287+0000 7f8324875640 1 --2- 172.21.15.169:0/745151936 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8320158850 0x7f83200cd720 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f8308005fd0 tx=0x7f830802d160 comp rx=0 tx=0).stop 2023-10-26T01:37:44.290 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.287+0000 7f8324875640 1 -- 172.21.15.169:0/745151936 shutdown_connections 2023-10-26T01:37:44.290 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.287+0000 7f8324875640 1 --2- 172.21.15.169:0/745151936 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8320158850 0x7f83200cd720 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:44.290 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.287+0000 7f8324875640 1 -- 172.21.15.169:0/745151936 >> 172.21.15.169:0/745151936 conn(0x7f8320069930 msgr2=0x7f8320069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:44.290 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.287+0000 7f8324875640 1 -- 172.21.15.169:0/745151936 shutdown_connections 2023-10-26T01:37:44.290 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.287+0000 7f8324875640 1 -- 172.21.15.169:0/745151936 wait complete. 2023-10-26T01:37:44.291 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.289+0000 7f8324875640 1 Processor -- start 2023-10-26T01:37:44.291 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.289+0000 7f8324875640 1 -- start start 2023-10-26T01:37:44.293 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.290+0000 7f8324875640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f83200cdc60 0x7f832006fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:44.293 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.290+0000 7f8324875640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f832014e790 0x7f83200b1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:44.293 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.290+0000 7f8324875640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8320158850 0x7f83200b1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:44.294 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.290+0000 7f8324875640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83200b0490 con 0x7f832014e790 2023-10-26T01:37:44.294 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.290+0000 7f8324875640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83201524a0 con 0x7f8320158850 2023-10-26T01:37:44.294 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.290+0000 7f8324875640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8320152780 con 0x7f83200cdc60 2023-10-26T01:37:44.294 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.290+0000 7f831dd9b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f832014e790 0x7f83200b1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:44.294 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.290+0000 7f831dd9b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f832014e790 0x7f83200b1880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:35616/0 (socket says 172.21.15.169:35616) 2023-10-26T01:37:44.295 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.290+0000 7f831dd9b640 1 -- 172.21.15.169:0/2855111842 learned_addr learned my addr 172.21.15.169:0/2855111842 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:37:44.295 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.291+0000 7f831dd9b640 1 -- 172.21.15.169:0/2855111842 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f83200cdc60 msgr2=0x7f832006fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:44.295 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.291+0000 7f831dd9b640 1 --2- 172.21.15.169:0/2855111842 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f83200cdc60 0x7f832006fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:44.295 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.291+0000 7f831ed9d640 1 --2- 172.21.15.169:0/2855111842 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8320158850 0x7f83200b1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:44.295 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.291+0000 7f831dd9b640 1 -- 172.21.15.169:0/2855111842 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8320158850 msgr2=0x7f83200b1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:44.296 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.291+0000 7f831dd9b640 1 --2- 172.21.15.169:0/2855111842 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8320158850 0x7f83200b1dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:44.296 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.291+0000 7f831dd9b640 1 -- 172.21.15.169:0/2855111842 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f830800a670 con 0x7f832014e790 2023-10-26T01:37:44.296 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.291+0000 7f831dd9b640 1 --2- 172.21.15.169:0/2855111842 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f832014e790 0x7f83200b1880 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f8308037040 tx=0x7f8308004820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:44.296 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.291+0000 7f8316ffd640 1 -- 172.21.15.169:0/2855111842 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8308004e20 con 0x7f832014e790 2023-10-26T01:37:44.296 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.291+0000 7f8316ffd640 1 -- 172.21.15.169:0/2855111842 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f83080035f0 con 0x7f832014e790 2023-10-26T01:37:44.297 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.291+0000 7f8316ffd640 1 -- 172.21.15.169:0/2855111842 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83080039a0 con 0x7f832014e790 2023-10-26T01:37:44.297 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.291+0000 7f8324875640 1 -- 172.21.15.169:0/2855111842 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8320152a60 con 0x7f832014e790 2023-10-26T01:37:44.297 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.292+0000 7f8324875640 1 -- 172.21.15.169:0/2855111842 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f832014d700 con 0x7f832014e790 2023-10-26T01:37:44.298 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.293+0000 7f8324875640 1 -- 172.21.15.169:0/2855111842 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f832004db10 con 0x7f832014e790 2023-10-26T01:37:44.298 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.294+0000 7f8316ffd640 1 -- 172.21.15.169:0/2855111842 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f8308003b00 con 0x7f832014e790 2023-10-26T01:37:44.298 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.294+0000 7f8316ffd640 1 --2- 172.21.15.169:0/2855111842 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f830006ae40 0x7f830006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:44.299 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.295+0000 7f8316ffd640 1 -- 172.21.15.169:0/2855111842 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f83080ac750 con 0x7f832014e790 2023-10-26T01:37:44.299 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.295+0000 7f831e59c640 1 --2- 172.21.15.169:0/2855111842 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f830006ae40 0x7f830006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:44.299 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.296+0000 7f831e59c640 1 --2- 172.21.15.169:0/2855111842 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f830006ae40 0x7f830006d300 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f83100012e0 tx=0x7f8310007d30 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:44.301 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.299+0000 7f8316ffd640 1 -- 172.21.15.169:0/2855111842 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f83080778a0 con 0x7f832014e790 2023-10-26T01:37:44.510 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.508+0000 7f8324875640 1 -- 172.21.15.169:0/2855111842 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f83200af6b0 con 0x7f832014e790 2023-10-26T01:37:44.511 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.509+0000 7f8316ffd640 1 -- 172.21.15.169:0/2855111842 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) v1 ==== 94+0+4924 (secure 0 0 0) 0x7f8308003db0 con 0x7f832014e790 2023-10-26T01:37:44.511 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:37:44.512 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":59,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5387,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/1186563921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1186563921},{"type":"v1","addr":"172.21.15.170:6835","nonce":1186563921}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":54},{"gid":5390,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/3702036693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":3702036693},{"type":"v1","addr":"172.21.15.170:6837","nonce":3702036693}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":54},{"gid":7332,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/3193888686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":3193888686},{"type":"v1","addr":"172.21.15.169:6833","nonce":3193888686}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T01:37:40.027858+0000","modified":"2023-10-26T01:37:43.103474+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7335},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7335":{"gid":7335,"name":"c","rank":0,"incarnation":55,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6835/668588916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":668588916},{"type":"v1","addr":"172.21.15.169:6835","nonce":668588916}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-10-26T01:37:44.512 INFO:teuthology.orchestra.run.smithi169.stderr:dumped fsmap epoch 59 2023-10-26T01:37:44.513 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.511+0000 7f8324875640 1 -- 172.21.15.169:0/2855111842 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f830006ae40 msgr2=0x7f830006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:44.513 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.511+0000 7f8324875640 1 --2- 172.21.15.169:0/2855111842 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f830006ae40 0x7f830006d300 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f83100012e0 tx=0x7f8310007d30 comp rx=0 tx=0).stop 2023-10-26T01:37:44.514 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.511+0000 7f8324875640 1 -- 172.21.15.169:0/2855111842 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f832014e790 msgr2=0x7f83200b1880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:44.514 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.511+0000 7f8324875640 1 --2- 172.21.15.169:0/2855111842 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f832014e790 0x7f83200b1880 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f8308037040 tx=0x7f8308004820 comp rx=0 tx=0).stop 2023-10-26T01:37:44.514 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.511+0000 7f8324875640 1 -- 172.21.15.169:0/2855111842 shutdown_connections 2023-10-26T01:37:44.514 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.511+0000 7f8324875640 1 --2- 172.21.15.169:0/2855111842 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f830006ae40 0x7f830006d300 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:44.514 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.511+0000 7f8324875640 1 --2- 172.21.15.169:0/2855111842 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f832014e790 0x7f83200b1880 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:44.515 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.511+0000 7f8324875640 1 --2- 172.21.15.169:0/2855111842 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8320158850 0x7f83200b1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:44.515 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.511+0000 7f8324875640 1 --2- 172.21.15.169:0/2855111842 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f83200cdc60 0x7f832006fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:44.515 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.511+0000 7f8324875640 1 -- 172.21.15.169:0/2855111842 >> 172.21.15.169:0/2855111842 conn(0x7f8320069930 msgr2=0x7f832014ed60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:44.515 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.512+0000 7f8324875640 1 -- 172.21.15.169:0/2855111842 shutdown_connections 2023-10-26T01:37:44.515 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.512+0000 7f8324875640 1 -- 172.21.15.169:0/2855111842 wait complete. 2023-10-26T01:37:44.532 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:37:44.747 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.744+0000 7fbc735f0640 1 -- 172.21.15.169:0/308815986 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbc580047c0 msgr2=0x7fbc58004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:44.747 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.744+0000 7fbc735f0640 1 --2- 172.21.15.169:0/308815986 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbc580047c0 0x7fbc58004c40 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c00bb20 tx=0x7fbc580057f0 comp rx=0 tx=0).stop 2023-10-26T01:37:44.747 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.744+0000 7fbc71b8d640 1 -- 172.21.15.169:0/308815986 reap_dead start 2023-10-26T01:37:44.747 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.745+0000 7fbc735f0640 1 -- 172.21.15.169:0/308815986 shutdown_connections 2023-10-26T01:37:44.748 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.745+0000 7fbc735f0640 1 -- 172.21.15.169:0/308815986 >> 172.21.15.169:0/308815986 conn(0x7fbc6c069930 msgr2=0x7fbc6c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:44.748 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.745+0000 7fbc735f0640 1 -- 172.21.15.169:0/308815986 shutdown_connections 2023-10-26T01:37:44.748 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.745+0000 7fbc735f0640 1 -- 172.21.15.169:0/308815986 wait complete. 2023-10-26T01:37:44.749 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.746+0000 7fbc735f0640 1 Processor -- start 2023-10-26T01:37:44.749 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.747+0000 7fbc735f0640 1 -- start start 2023-10-26T01:37:44.750 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.747+0000 7fbc735f0640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbc580047c0 0x7fbc6c0c98f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:44.751 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.747+0000 7fbc735f0640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbc6c1f2210 0x7fbc6c0c8e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:44.751 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.748+0000 7fbc735f0640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbc6c15afe0 0x7fbc6c15b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:44.751 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.748+0000 7fbc735f0640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc6c1f3400 con 0x7fbc580047c0 2023-10-26T01:37:44.751 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.748+0000 7fbc735f0640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc6c15b9a0 con 0x7fbc6c15afe0 2023-10-26T01:37:44.751 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.748+0000 7fbc735f0640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc6c097600 con 0x7fbc6c1f2210 2023-10-26T01:37:44.752 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.748+0000 7fbc7138c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbc580047c0 0x7fbc6c0c98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:44.752 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.748+0000 7fbc7138c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbc580047c0 0x7fbc6c0c98f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:35626/0 (socket says 172.21.15.169:35626) 2023-10-26T01:37:44.752 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.748+0000 7fbc7138c640 1 -- 172.21.15.169:0/3865597219 learned_addr learned my addr 172.21.15.169:0/3865597219 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:37:44.752 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.748+0000 7fbc7138c640 1 -- 172.21.15.169:0/3865597219 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbc6c1f2210 msgr2=0x7fbc6c0c8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:44.752 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.748+0000 7fbc7138c640 1 --2- 172.21.15.169:0/3865597219 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbc6c1f2210 0x7fbc6c0c8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:44.753 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.748+0000 7fbc7138c640 1 -- 172.21.15.169:0/3865597219 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbc6c15afe0 msgr2=0x7fbc6c15b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:44.753 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.748+0000 7fbc7138c640 1 --2- 172.21.15.169:0/3865597219 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbc6c15afe0 0x7fbc6c15b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:44.753 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.748+0000 7fbc7138c640 1 -- 172.21.15.169:0/3865597219 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc6c0c9e30 con 0x7fbc580047c0 2023-10-26T01:37:44.753 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.748+0000 7fbc7138c640 1 --2- 172.21.15.169:0/3865597219 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbc580047c0 0x7fbc6c0c98f0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7fbc6800d940 tx=0x7fbc6800de00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:44.754 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.749+0000 7fbc61ffb640 1 -- 172.21.15.169:0/3865597219 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc6800b930 con 0x7fbc580047c0 2023-10-26T01:37:44.754 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.749+0000 7fbc61ffb640 1 -- 172.21.15.169:0/3865597219 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fbc6800ba90 con 0x7fbc580047c0 2023-10-26T01:37:44.754 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.749+0000 7fbc735f0640 1 -- 172.21.15.169:0/3865597219 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc6c0c9340 con 0x7fbc580047c0 2023-10-26T01:37:44.754 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.749+0000 7fbc61ffb640 1 -- 172.21.15.169:0/3865597219 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc6800be80 con 0x7fbc580047c0 2023-10-26T01:37:44.754 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.749+0000 7fbc735f0640 1 -- 172.21.15.169:0/3865597219 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc6c1f2760 con 0x7fbc580047c0 2023-10-26T01:37:44.755 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.750+0000 7fbc735f0640 1 -- 172.21.15.169:0/3865597219 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc6c0cd960 con 0x7fbc580047c0 2023-10-26T01:37:44.756 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.751+0000 7fbc61ffb640 1 -- 172.21.15.169:0/3865597219 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fbc6801b400 con 0x7fbc580047c0 2023-10-26T01:37:44.756 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.752+0000 7fbc61ffb640 1 --2- 172.21.15.169:0/3865597219 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbc5006ae40 0x7fbc5006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:44.756 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.752+0000 7fbc61ffb640 1 -- 172.21.15.169:0/3865597219 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbc68089cd0 con 0x7fbc580047c0 2023-10-26T01:37:44.756 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.752+0000 7fbc70b8b640 1 --2- 172.21.15.169:0/3865597219 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbc5006ae40 0x7fbc5006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:44.756 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.753+0000 7fbc70b8b640 1 --2- 172.21.15.169:0/3865597219 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbc5006ae40 0x7fbc5006d300 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c037000 tx=0x7fbc5c02e040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:44.758 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.756+0000 7fbc61ffb640 1 -- 172.21.15.169:0/3865597219 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbc6c0cd960 con 0x7fbc580047c0 2023-10-26T01:37:44.950 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.948+0000 7fbc735f0640 1 -- 172.21.15.169:0/3865597219 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbc6c0cd960 con 0x7fbc580047c0 2023-10-26T01:37:44.953 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.951+0000 7fbc61ffb640 1 -- 172.21.15.169:0/3865597219 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v120) v1 ==== 74+0+16568 (secure 0 0 0) 0x7fbc6c0cd960 con 0x7fbc580047c0 2023-10-26T01:37:44.953 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:37:44.953 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":120,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:37:41.581744+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-10-26T01:37:37.959102+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-10-26T01:37:39.053240+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:37:44.956 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.953+0000 7fbc735f0640 1 -- 172.21.15.169:0/3865597219 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbc5006ae40 msgr2=0x7fbc5006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:44.956 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.953+0000 7fbc735f0640 1 --2- 172.21.15.169:0/3865597219 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbc5006ae40 0x7fbc5006d300 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c037000 tx=0x7fbc5c02e040 comp rx=0 tx=0).stop 2023-10-26T01:37:44.956 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.953+0000 7fbc735f0640 1 -- 172.21.15.169:0/3865597219 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbc580047c0 msgr2=0x7fbc6c0c98f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:44.956 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.953+0000 7fbc735f0640 1 --2- 172.21.15.169:0/3865597219 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbc580047c0 0x7fbc6c0c98f0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7fbc6800d940 tx=0x7fbc6800de00 comp rx=0 tx=0).stop 2023-10-26T01:37:44.956 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.954+0000 7fbc735f0640 1 -- 172.21.15.169:0/3865597219 shutdown_connections 2023-10-26T01:37:44.957 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.954+0000 7fbc735f0640 1 --2- 172.21.15.169:0/3865597219 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbc5006ae40 0x7fbc5006d300 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:44.957 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.954+0000 7fbc735f0640 1 --2- 172.21.15.169:0/3865597219 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbc6c15afe0 0x7fbc6c15b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:44.957 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.954+0000 7fbc735f0640 1 --2- 172.21.15.169:0/3865597219 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbc6c1f2210 0x7fbc6c0c8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:44.957 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.954+0000 7fbc735f0640 1 --2- 172.21.15.169:0/3865597219 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbc580047c0 0x7fbc6c0c98f0 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:44.957 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.954+0000 7fbc735f0640 1 -- 172.21.15.169:0/3865597219 >> 172.21.15.169:0/3865597219 conn(0x7fbc6c069930 msgr2=0x7fbc6c1538c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:44.958 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.954+0000 7fbc735f0640 1 -- 172.21.15.169:0/3865597219 shutdown_connections 2023-10-26T01:37:44.958 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:44.954+0000 7fbc735f0640 1 -- 172.21.15.169:0/3865597219 wait complete. 2023-10-26T01:37:44.975 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-26T01:37:44.975 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-26T01:37:45.189 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:45.187+0000 7fb03053c640 1 -- 172.21.15.169:0/1812797595 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb02814ece0 msgr2=0x7fb0280aa1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:45.189 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:45.187+0000 7fb03053c640 1 --2- 172.21.15.169:0/1812797595 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb02814ece0 0x7fb0280aa1c0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fb024009830 tx=0x7fb02402bd60 comp rx=0 tx=0).stop 2023-10-26T01:37:45.190 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:45.187+0000 7fb03053c640 1 -- 172.21.15.169:0/1812797595 shutdown_connections 2023-10-26T01:37:45.190 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:45.187+0000 7fb03053c640 1 --2- 172.21.15.169:0/1812797595 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb02814ece0 0x7fb0280aa1c0 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:45.190 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:45.187+0000 7fb03053c640 1 -- 172.21.15.169:0/1812797595 >> 172.21.15.169:0/1812797595 conn(0x7fb028069930 msgr2=0x7fb028069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:45.190 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:45.187+0000 7fb03053c640 1 -- 172.21.15.169:0/1812797595 shutdown_connections 2023-10-26T01:37:45.191 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:45.187+0000 7fb03053c640 1 -- 172.21.15.169:0/1812797595 wait complete. 2023-10-26T01:37:45.191 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:45.189+0000 7fb03053c640 1 Processor -- start 2023-10-26T01:37:45.191 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:45.189+0000 7fb03053c640 1 -- start start 2023-10-26T01:37:45.193 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:45.190+0000 7fb03053c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb02806bf70 0x7fb02814a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:45.193 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:45.190+0000 7fb03053c640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb02806c420 0x7fb028151330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:45.193 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:45.190+0000 7fb03053c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb02814ece0 0x7fb028156d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:45.193 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:45.190+0000 7fb03053c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb02814a690 con 0x7fb02814ece0 2023-10-26T01:37:45.193 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:45.190+0000 7fb03053c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb02814b220 con 0x7fb02806bf70 2023-10-26T01:37:45.194 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:45.190+0000 7fb03053c640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb02814b500 con 0x7fb02806c420 2023-10-26T01:37:45.194 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:45.191+0000 7fb02ead9640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb02814ece0 0x7fb028156d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:45.194 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:45.191+0000 7fb02ead9640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb02814ece0 0x7fb028156d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:35640/0 (socket says 172.21.15.169:35640) 2023-10-26T01:37:45.194 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:45.191+0000 7fb02ead9640 1 -- 172.21.15.169:0/667275624 learned_addr learned my addr 172.21.15.169:0/667275624 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:37:45.194 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:45.191+0000 7fb02e2d8640 1 --2- 172.21.15.169:0/667275624 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb02806bf70 0x7fb02814a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:45.195 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:45.191+0000 7fb02ead9640 1 -- 172.21.15.169:0/667275624 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb02806c420 msgr2=0x7fb028151330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:45.195 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:45.191+0000 7fb02ead9640 1 --2- 172.21.15.169:0/667275624 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb02806c420 0x7fb028151330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:45.195 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:45.191+0000 7fb02ead9640 1 -- 172.21.15.169:0/667275624 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb02806bf70 msgr2=0x7fb02814a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:45.195 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:45.191+0000 7fb02ead9640 1 --2- 172.21.15.169:0/667275624 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb02806bf70 0x7fb02814a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:45.195 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:45.191+0000 7fb02ead9640 1 -- 172.21.15.169:0/667275624 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb024009490 con 0x7fb02814ece0 2023-10-26T01:37:45.196 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:45.191+0000 7fb02ead9640 1 --2- 172.21.15.169:0/667275624 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb02814ece0 0x7fb028156d70 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fb02000c780 tx=0x7fb02000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:45.197 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:45.192+0000 7fb01effd640 1 -- 172.21.15.169:0/667275624 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb020004b60 con 0x7fb02814ece0 2023-10-26T01:37:45.197 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:45.192+0000 7fb01effd640 1 -- 172.21.15.169:0/667275624 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb02000ce90 con 0x7fb02814ece0 2023-10-26T01:37:45.197 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:45.192+0000 7fb01effd640 1 -- 172.21.15.169:0/667275624 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb02000aab0 con 0x7fb02814ece0 2023-10-26T01:37:45.198 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:45.192+0000 7fb03053c640 1 -- 172.21.15.169:0/667275624 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb02814b7e0 con 0x7fb02814ece0 2023-10-26T01:37:45.198 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:45.192+0000 7fb03053c640 1 -- 172.21.15.169:0/667275624 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb028151870 con 0x7fb02814ece0 2023-10-26T01:37:45.198 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:45.193+0000 7fb03053c640 1 -- 172.21.15.169:0/667275624 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb02814bfe0 con 0x7fb02814ece0 2023-10-26T01:37:45.198 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:45.194+0000 7fb01effd640 1 -- 172.21.15.169:0/667275624 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fb02000ac10 con 0x7fb02814ece0 2023-10-26T01:37:45.198 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:45.195+0000 7fb01effd640 1 --2- 172.21.15.169:0/667275624 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb00c06ae40 0x7fb00c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:45.199 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:45.195+0000 7fb01effd640 1 -- 172.21.15.169:0/667275624 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb020089a30 con 0x7fb02814ece0 2023-10-26T01:37:45.199 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:45.195+0000 7fb02e2d8640 1 --2- 172.21.15.169:0/667275624 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb00c06ae40 0x7fb00c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:45.199 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:45.197+0000 7fb02e2d8640 1 --2- 172.21.15.169:0/667275624 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb00c06ae40 0x7fb00c06d300 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fb024036000 tx=0x7fb024004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:45.200 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:45.199+0000 7fb01effd640 1 -- 172.21.15.169:0/667275624 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb02814bfe0 con 0x7fb02814ece0 2023-10-26T01:37:45.373 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:45.372+0000 7fb03053c640 1 -- 172.21.15.169:0/667275624 --> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb0280ab220 con 0x7fb00c06ae40 2023-10-26T01:37:45.396 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:45.394+0000 7fb01effd640 1 -- 172.21.15.169:0/667275624 <== mgr.4112 v2:172.21.15.170:6832/1187156918 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+282995 (secure 0 0 0) 0x7fb02402c700 con 0x7fb00c06ae40 2023-10-26T01:37:45.400 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:37:45.402 INFO:teuthology.orchestra.run.smithi169.stderr:dumped all 2023-10-26T01:37:45.403 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:45.400+0000 7fb03053c640 1 -- 172.21.15.169:0/667275624 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb00c06ae40 msgr2=0x7fb00c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:45.404 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:45.400+0000 7fb03053c640 1 --2- 172.21.15.169:0/667275624 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb00c06ae40 0x7fb00c06d300 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fb024036000 tx=0x7fb024004730 comp rx=0 tx=0).stop 2023-10-26T01:37:45.404 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:45.400+0000 7fb03053c640 1 -- 172.21.15.169:0/667275624 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb02814ece0 msgr2=0x7fb028156d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:45.404 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:45.400+0000 7fb03053c640 1 --2- 172.21.15.169:0/667275624 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb02814ece0 0x7fb028156d70 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fb02000c780 tx=0x7fb02000cc40 comp rx=0 tx=0).stop 2023-10-26T01:37:45.404 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:45.400+0000 7fb03053c640 1 -- 172.21.15.169:0/667275624 shutdown_connections 2023-10-26T01:37:45.404 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:45.400+0000 7fb03053c640 1 --2- 172.21.15.169:0/667275624 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb00c06ae40 0x7fb00c06d300 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:45.404 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:45.400+0000 7fb03053c640 1 --2- 172.21.15.169:0/667275624 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb02814ece0 0x7fb028156d70 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:45.405 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:45.400+0000 7fb03053c640 1 --2- 172.21.15.169:0/667275624 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb02806c420 0x7fb028151330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:45.405 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:45.400+0000 7fb03053c640 1 --2- 172.21.15.169:0/667275624 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb02806bf70 0x7fb02814a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:45.405 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:45.400+0000 7fb03053c640 1 -- 172.21.15.169:0/667275624 >> 172.21.15.169:0/667275624 conn(0x7fb028069930 msgr2=0x7fb028072630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:45.405 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:45.401+0000 7fb03053c640 1 -- 172.21.15.169:0/667275624 shutdown_connections 2023-10-26T01:37:45.405 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:45.401+0000 7fb03053c640 1 -- 172.21.15.169:0/667275624 wait complete. 2023-10-26T01:37:45.420 INFO:teuthology.orchestra.run.smithi169.stdout:{"pg_ready":true,"pg_map":{"version":330,"stamp":"2023-10-26T01:37:44.280678+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":47,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":6237,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1556262,"statfs":{"total":1600000000,"available":1593612613,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.731490"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.611425+0000","last_change":"2023-10-26T01:37:40.610968+0000","last_active":"2023-10-26T01:37:40.611425+0000","last_peered":"2023-10-26T01:37:40.611425+0000","last_clean":"2023-10-26T01:37:40.611425+0000","last_became_active":"2023-10-26T01:37:40.609289+0000","last_became_peered":"2023-10-26T01:37:40.609289+0000","last_unstale":"2023-10-26T01:37:40.611425+0000","last_undegraded":"2023-10-26T01:37:40.611425+0000","last_fullsized":"2023-10-26T01:37:40.611425+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:04:23.812581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.612864+0000","last_change":"2023-10-26T01:37:40.612608+0000","last_active":"2023-10-26T01:37:40.612864+0000","last_peered":"2023-10-26T01:37:40.612864+0000","last_clean":"2023-10-26T01:37:40.612864+0000","last_became_active":"2023-10-26T01:37:40.610953+0000","last_became_peered":"2023-10-26T01:37:40.610953+0000","last_unstale":"2023-10-26T01:37:40.612864+0000","last_undegraded":"2023-10-26T01:37:40.612864+0000","last_fullsized":"2023-10-26T01:37:40.612864+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:23:53.795891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.614358+0000","last_change":"2023-10-26T01:37:40.612245+0000","last_active":"2023-10-26T01:37:40.614358+0000","last_peered":"2023-10-26T01:37:40.614358+0000","last_clean":"2023-10-26T01:37:40.614358+0000","last_became_active":"2023-10-26T01:37:40.612000+0000","last_became_peered":"2023-10-26T01:37:40.612000+0000","last_unstale":"2023-10-26T01:37:40.614358+0000","last_undegraded":"2023-10-26T01:37:40.614358+0000","last_fullsized":"2023-10-26T01:37:40.614358+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:13:26.281509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":11,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-10-26T01:37:41.587285+0000","last_change":"2023-10-26T01:37:40.602746+0000","last_active":"2023-10-26T01:37:41.587285+0000","last_peered":"2023-10-26T01:37:41.587285+0000","last_clean":"2023-10-26T01:37:41.587285+0000","last_became_active":"2023-10-26T01:37:40.602448+0000","last_became_peered":"2023-10-26T01:37:40.602448+0000","last_unstale":"2023-10-26T01:37:41.587285+0000","last_undegraded":"2023-10-26T01:37:41.587285+0000","last_fullsized":"2023-10-26T01:37:41.587285+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:41:33.659471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.608803+0000","last_change":"2023-10-26T01:37:40.605540+0000","last_active":"2023-10-26T01:37:40.608803+0000","last_peered":"2023-10-26T01:37:40.608803+0000","last_clean":"2023-10-26T01:37:40.608803+0000","last_became_active":"2023-10-26T01:37:40.604069+0000","last_became_peered":"2023-10-26T01:37:40.604069+0000","last_unstale":"2023-10-26T01:37:40.608803+0000","last_undegraded":"2023-10-26T01:37:40.608803+0000","last_fullsized":"2023-10-26T01:37:40.608803+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:00:46.370884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.617100+0000","last_change":"2023-10-26T01:37:40.616264+0000","last_active":"2023-10-26T01:37:40.617100+0000","last_peered":"2023-10-26T01:37:40.617100+0000","last_clean":"2023-10-26T01:37:40.617100+0000","last_became_active":"2023-10-26T01:37:40.615542+0000","last_became_peered":"2023-10-26T01:37:40.615542+0000","last_unstale":"2023-10-26T01:37:40.617100+0000","last_undegraded":"2023-10-26T01:37:40.617100+0000","last_fullsized":"2023-10-26T01:37:40.617100+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:23:53.795891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.616030+0000","last_change":"2023-10-26T01:37:40.611830+0000","last_active":"2023-10-26T01:37:40.616030+0000","last_peered":"2023-10-26T01:37:40.616030+0000","last_clean":"2023-10-26T01:37:40.616030+0000","last_became_active":"2023-10-26T01:37:40.611084+0000","last_became_peered":"2023-10-26T01:37:40.611084+0000","last_unstale":"2023-10-26T01:37:40.616030+0000","last_undegraded":"2023-10-26T01:37:40.616030+0000","last_fullsized":"2023-10-26T01:37:40.616030+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:58:41.530615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.603979+0000","last_change":"2023-10-26T01:37:40.603565+0000","last_active":"2023-10-26T01:37:40.603979+0000","last_peered":"2023-10-26T01:37:40.603979+0000","last_clean":"2023-10-26T01:37:40.603979+0000","last_became_active":"2023-10-26T01:37:40.603264+0000","last_became_peered":"2023-10-26T01:37:40.603264+0000","last_unstale":"2023-10-26T01:37:40.603979+0000","last_undegraded":"2023-10-26T01:37:40.603979+0000","last_fullsized":"2023-10-26T01:37:40.603979+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:58:41.530615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.609396+0000","last_change":"2023-10-26T01:37:40.605549+0000","last_active":"2023-10-26T01:37:40.609396+0000","last_peered":"2023-10-26T01:37:40.609396+0000","last_clean":"2023-10-26T01:37:40.609396+0000","last_became_active":"2023-10-26T01:37:40.604888+0000","last_became_peered":"2023-10-26T01:37:40.604888+0000","last_unstale":"2023-10-26T01:37:40.609396+0000","last_undegraded":"2023-10-26T01:37:40.609396+0000","last_fullsized":"2023-10-26T01:37:40.609396+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:52:22.871839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.608820+0000","last_change":"2023-10-26T01:37:40.605714+0000","last_active":"2023-10-26T01:37:40.608820+0000","last_peered":"2023-10-26T01:37:40.608820+0000","last_clean":"2023-10-26T01:37:40.608820+0000","last_became_active":"2023-10-26T01:37:40.604567+0000","last_became_peered":"2023-10-26T01:37:40.604567+0000","last_unstale":"2023-10-26T01:37:40.608820+0000","last_undegraded":"2023-10-26T01:37:40.608820+0000","last_fullsized":"2023-10-26T01:37:40.608820+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:52:22.871839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.617035+0000","last_change":"2023-10-26T01:37:40.615942+0000","last_active":"2023-10-26T01:37:40.617035+0000","last_peered":"2023-10-26T01:37:40.617035+0000","last_clean":"2023-10-26T01:37:40.617035+0000","last_became_active":"2023-10-26T01:37:40.614830+0000","last_became_peered":"2023-10-26T01:37:40.614830+0000","last_unstale":"2023-10-26T01:37:40.617035+0000","last_undegraded":"2023-10-26T01:37:40.617035+0000","last_fullsized":"2023-10-26T01:37:40.617035+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:33:21.466763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.610276+0000","last_change":"2023-10-26T01:37:40.606507+0000","last_active":"2023-10-26T01:37:40.610276+0000","last_peered":"2023-10-26T01:37:40.610276+0000","last_clean":"2023-10-26T01:37:40.610276+0000","last_became_active":"2023-10-26T01:37:40.606174+0000","last_became_peered":"2023-10-26T01:37:40.606174+0000","last_unstale":"2023-10-26T01:37:40.610276+0000","last_undegraded":"2023-10-26T01:37:40.610276+0000","last_fullsized":"2023-10-26T01:37:40.610276+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:57:42.748953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":11,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-10-26T01:37:41.587505+0000","last_change":"2023-10-26T01:37:40.608213+0000","last_active":"2023-10-26T01:37:41.587505+0000","last_peered":"2023-10-26T01:37:41.587505+0000","last_clean":"2023-10-26T01:37:41.587505+0000","last_became_active":"2023-10-26T01:37:40.607952+0000","last_became_peered":"2023-10-26T01:37:40.607952+0000","last_unstale":"2023-10-26T01:37:41.587505+0000","last_undegraded":"2023-10-26T01:37:41.587505+0000","last_fullsized":"2023-10-26T01:37:41.587505+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:32:02.668533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.613766+0000","last_change":"2023-10-26T01:37:40.613691+0000","last_active":"2023-10-26T01:37:40.613766+0000","last_peered":"2023-10-26T01:37:40.613766+0000","last_clean":"2023-10-26T01:37:40.613766+0000","last_became_active":"2023-10-26T01:37:40.613462+0000","last_became_peered":"2023-10-26T01:37:40.613462+0000","last_unstale":"2023-10-26T01:37:40.613766+0000","last_undegraded":"2023-10-26T01:37:40.613766+0000","last_fullsized":"2023-10-26T01:37:40.613766+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:30:38.850351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":11,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-10-26T01:37:41.588298+0000","last_change":"2023-10-26T01:37:40.600401+0000","last_active":"2023-10-26T01:37:41.588298+0000","last_peered":"2023-10-26T01:37:41.588298+0000","last_clean":"2023-10-26T01:37:41.588298+0000","last_became_active":"2023-10-26T01:37:40.599717+0000","last_became_peered":"2023-10-26T01:37:40.599717+0000","last_unstale":"2023-10-26T01:37:41.588298+0000","last_undegraded":"2023-10-26T01:37:41.588298+0000","last_fullsized":"2023-10-26T01:37:41.588298+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:20:43.946448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.616982+0000","last_change":"2023-10-26T01:37:40.615795+0000","last_active":"2023-10-26T01:37:40.616982+0000","last_peered":"2023-10-26T01:37:40.616982+0000","last_clean":"2023-10-26T01:37:40.616982+0000","last_became_active":"2023-10-26T01:37:40.609556+0000","last_became_peered":"2023-10-26T01:37:40.609556+0000","last_unstale":"2023-10-26T01:37:40.616982+0000","last_undegraded":"2023-10-26T01:37:40.616982+0000","last_fullsized":"2023-10-26T01:37:40.616982+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:43:19.958268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.593892+0000","last_change":"2023-10-26T01:37:40.593208+0000","last_active":"2023-10-26T01:37:40.593892+0000","last_peered":"2023-10-26T01:37:40.593892+0000","last_clean":"2023-10-26T01:37:40.593892+0000","last_became_active":"2023-10-26T01:37:40.592943+0000","last_became_peered":"2023-10-26T01:37:40.592943+0000","last_unstale":"2023-10-26T01:37:40.593892+0000","last_undegraded":"2023-10-26T01:37:40.593892+0000","last_fullsized":"2023-10-26T01:37:40.593892+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:34:05.902643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":11,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-10-26T01:37:41.587678+0000","last_change":"2023-10-26T01:37:40.601965+0000","last_active":"2023-10-26T01:37:41.587678+0000","last_peered":"2023-10-26T01:37:41.587678+0000","last_clean":"2023-10-26T01:37:41.587678+0000","last_became_active":"2023-10-26T01:37:40.601736+0000","last_became_peered":"2023-10-26T01:37:40.601736+0000","last_unstale":"2023-10-26T01:37:41.587678+0000","last_undegraded":"2023-10-26T01:37:41.587678+0000","last_fullsized":"2023-10-26T01:37:41.587678+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:57:42.748953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.605734+0000","last_change":"2023-10-26T01:37:40.605655+0000","last_active":"2023-10-26T01:37:40.605734+0000","last_peered":"2023-10-26T01:37:40.605734+0000","last_clean":"2023-10-26T01:37:40.605734+0000","last_became_active":"2023-10-26T01:37:40.605413+0000","last_became_peered":"2023-10-26T01:37:40.605413+0000","last_unstale":"2023-10-26T01:37:40.605734+0000","last_undegraded":"2023-10-26T01:37:40.605734+0000","last_fullsized":"2023-10-26T01:37:40.605734+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:05:57.749521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.610250+0000","last_change":"2023-10-26T01:37:40.605731+0000","last_active":"2023-10-26T01:37:40.610250+0000","last_peered":"2023-10-26T01:37:40.610250+0000","last_clean":"2023-10-26T01:37:40.610250+0000","last_became_active":"2023-10-26T01:37:40.605123+0000","last_became_peered":"2023-10-26T01:37:40.605123+0000","last_unstale":"2023-10-26T01:37:40.610250+0000","last_undegraded":"2023-10-26T01:37:40.610250+0000","last_fullsized":"2023-10-26T01:37:40.610250+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:58:41.530615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.609492+0000","last_change":"2023-10-26T01:37:40.609424+0000","last_active":"2023-10-26T01:37:40.609492+0000","last_peered":"2023-10-26T01:37:40.609492+0000","last_clean":"2023-10-26T01:37:40.609492+0000","last_became_active":"2023-10-26T01:37:40.609185+0000","last_became_peered":"2023-10-26T01:37:40.609185+0000","last_unstale":"2023-10-26T01:37:40.609492+0000","last_undegraded":"2023-10-26T01:37:40.609492+0000","last_fullsized":"2023-10-26T01:37:40.609492+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:28:48.738602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.612904+0000","last_change":"2023-10-26T01:37:40.612801+0000","last_active":"2023-10-26T01:37:40.612904+0000","last_peered":"2023-10-26T01:37:40.612904+0000","last_clean":"2023-10-26T01:37:40.612904+0000","last_became_active":"2023-10-26T01:37:40.612320+0000","last_became_peered":"2023-10-26T01:37:40.612320+0000","last_unstale":"2023-10-26T01:37:40.612904+0000","last_undegraded":"2023-10-26T01:37:40.612904+0000","last_fullsized":"2023-10-26T01:37:40.612904+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:41:54.411711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.607141+0000","last_change":"2023-10-26T01:37:40.607032+0000","last_active":"2023-10-26T01:37:40.607141+0000","last_peered":"2023-10-26T01:37:40.607141+0000","last_clean":"2023-10-26T01:37:40.607141+0000","last_became_active":"2023-10-26T01:37:40.606807+0000","last_became_peered":"2023-10-26T01:37:40.606807+0000","last_unstale":"2023-10-26T01:37:40.607141+0000","last_undegraded":"2023-10-26T01:37:40.607141+0000","last_fullsized":"2023-10-26T01:37:40.607141+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:00:46.370884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.607126+0000","last_change":"2023-10-26T01:37:40.605029+0000","last_active":"2023-10-26T01:37:40.607126+0000","last_peered":"2023-10-26T01:37:40.607126+0000","last_clean":"2023-10-26T01:37:40.607126+0000","last_became_active":"2023-10-26T01:37:40.604635+0000","last_became_peered":"2023-10-26T01:37:40.604635+0000","last_unstale":"2023-10-26T01:37:40.607126+0000","last_undegraded":"2023-10-26T01:37:40.607126+0000","last_fullsized":"2023-10-26T01:37:40.607126+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:57:42.748953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.608783+0000","last_change":"2023-10-26T01:37:40.605294+0000","last_active":"2023-10-26T01:37:40.608783+0000","last_peered":"2023-10-26T01:37:40.608783+0000","last_clean":"2023-10-26T01:37:40.608783+0000","last_became_active":"2023-10-26T01:37:40.603593+0000","last_became_peered":"2023-10-26T01:37:40.603593+0000","last_unstale":"2023-10-26T01:37:40.608783+0000","last_undegraded":"2023-10-26T01:37:40.608783+0000","last_fullsized":"2023-10-26T01:37:40.608783+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:23:53.795891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.601126+0000","last_change":"2023-10-26T01:37:40.601052+0000","last_active":"2023-10-26T01:37:40.601126+0000","last_peered":"2023-10-26T01:37:40.601126+0000","last_clean":"2023-10-26T01:37:40.601126+0000","last_became_active":"2023-10-26T01:37:40.600790+0000","last_became_peered":"2023-10-26T01:37:40.600790+0000","last_unstale":"2023-10-26T01:37:40.601126+0000","last_undegraded":"2023-10-26T01:37:40.601126+0000","last_fullsized":"2023-10-26T01:37:40.601126+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:58:42.116497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.604364+0000","last_change":"2023-10-26T01:37:40.604186+0000","last_active":"2023-10-26T01:37:40.604364+0000","last_peered":"2023-10-26T01:37:40.604364+0000","last_clean":"2023-10-26T01:37:40.604364+0000","last_became_active":"2023-10-26T01:37:40.603880+0000","last_became_peered":"2023-10-26T01:37:40.603880+0000","last_unstale":"2023-10-26T01:37:40.604364+0000","last_undegraded":"2023-10-26T01:37:40.604364+0000","last_fullsized":"2023-10-26T01:37:40.604364+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:00:46.370884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.603086+0000","last_change":"2023-10-26T01:37:40.599308+0000","last_active":"2023-10-26T01:37:40.603086+0000","last_peered":"2023-10-26T01:37:40.603086+0000","last_clean":"2023-10-26T01:37:40.603086+0000","last_became_active":"2023-10-26T01:37:40.598237+0000","last_became_peered":"2023-10-26T01:37:40.598237+0000","last_unstale":"2023-10-26T01:37:40.603086+0000","last_undegraded":"2023-10-26T01:37:40.603086+0000","last_fullsized":"2023-10-26T01:37:40.603086+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:42:26.945329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.598565+0000","last_change":"2023-10-26T01:37:40.598489+0000","last_active":"2023-10-26T01:37:40.598565+0000","last_peered":"2023-10-26T01:37:40.598565+0000","last_clean":"2023-10-26T01:37:40.598565+0000","last_became_active":"2023-10-26T01:37:40.598268+0000","last_became_peered":"2023-10-26T01:37:40.598268+0000","last_unstale":"2023-10-26T01:37:40.598565+0000","last_undegraded":"2023-10-26T01:37:40.598565+0000","last_fullsized":"2023-10-26T01:37:40.598565+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:42:26.945329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.608136+0000","last_change":"2023-10-26T01:37:40.608080+0000","last_active":"2023-10-26T01:37:40.608136+0000","last_peered":"2023-10-26T01:37:40.608136+0000","last_clean":"2023-10-26T01:37:40.608136+0000","last_became_active":"2023-10-26T01:37:40.607354+0000","last_became_peered":"2023-10-26T01:37:40.607354+0000","last_unstale":"2023-10-26T01:37:40.608136+0000","last_undegraded":"2023-10-26T01:37:40.608136+0000","last_fullsized":"2023-10-26T01:37:40.608136+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:48:29.780711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":11,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-10-26T01:37:41.587364+0000","last_change":"2023-10-26T01:37:40.609980+0000","last_active":"2023-10-26T01:37:41.587364+0000","last_peered":"2023-10-26T01:37:41.587364+0000","last_clean":"2023-10-26T01:37:41.587364+0000","last_became_active":"2023-10-26T01:37:40.609759+0000","last_became_peered":"2023-10-26T01:37:40.609759+0000","last_unstale":"2023-10-26T01:37:41.587364+0000","last_undegraded":"2023-10-26T01:37:41.587364+0000","last_fullsized":"2023-10-26T01:37:41.587364+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:23:28.232518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.606019+0000","last_change":"2023-10-26T01:37:40.605894+0000","last_active":"2023-10-26T01:37:40.606019+0000","last_peered":"2023-10-26T01:37:40.606019+0000","last_clean":"2023-10-26T01:37:40.606019+0000","last_became_active":"2023-10-26T01:37:40.603038+0000","last_became_peered":"2023-10-26T01:37:40.603038+0000","last_unstale":"2023-10-26T01:37:40.606019+0000","last_undegraded":"2023-10-26T01:37:40.606019+0000","last_fullsized":"2023-10-26T01:37:40.606019+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:24:56.801054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.611087+0000","last_change":"2023-10-26T01:37:40.610095+0000","last_active":"2023-10-26T01:37:40.611087+0000","last_peered":"2023-10-26T01:37:40.611087+0000","last_clean":"2023-10-26T01:37:40.611087+0000","last_became_active":"2023-10-26T01:37:40.609457+0000","last_became_peered":"2023-10-26T01:37:40.609457+0000","last_unstale":"2023-10-26T01:37:40.611087+0000","last_undegraded":"2023-10-26T01:37:40.611087+0000","last_fullsized":"2023-10-26T01:37:40.611087+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:44:57.760928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"119'4","reported_seq":15,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.614071+0000","last_change":"2023-10-26T01:37:39.577351+0000","last_active":"2023-10-26T01:37:40.614071+0000","last_peered":"2023-10-26T01:37:40.614071+0000","last_clean":"2023-10-26T01:37:40.614071+0000","last_became_active":"2023-10-26T01:37:39.577122+0000","last_became_peered":"2023-10-26T01:37:39.577122+0000","last_unstale":"2023-10-26T01:37:40.614071+0000","last_undegraded":"2023-10-26T01:37:40.614071+0000","last_fullsized":"2023-10-26T01:37:40.614071+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_clean_scrub_stamp":"2023-10-26T01:37:38.536854+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:43:18.947747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":11,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-10-26T01:37:41.587820+0000","last_change":"2023-10-26T01:37:40.613479+0000","last_active":"2023-10-26T01:37:41.587820+0000","last_peered":"2023-10-26T01:37:41.587820+0000","last_clean":"2023-10-26T01:37:41.587820+0000","last_became_active":"2023-10-26T01:37:40.613301+0000","last_became_peered":"2023-10-26T01:37:40.613301+0000","last_unstale":"2023-10-26T01:37:41.587820+0000","last_undegraded":"2023-10-26T01:37:41.587820+0000","last_fullsized":"2023-10-26T01:37:41.587820+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:48:38.290864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"119'1","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.619307+0000","last_change":"2023-10-26T01:37:39.580337+0000","last_active":"2023-10-26T01:37:40.619307+0000","last_peered":"2023-10-26T01:37:40.619307+0000","last_clean":"2023-10-26T01:37:40.619307+0000","last_became_active":"2023-10-26T01:37:39.579565+0000","last_became_peered":"2023-10-26T01:37:39.579565+0000","last_unstale":"2023-10-26T01:37:40.619307+0000","last_undegraded":"2023-10-26T01:37:40.619307+0000","last_fullsized":"2023-10-26T01:37:40.619307+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_clean_scrub_stamp":"2023-10-26T01:37:38.536854+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:48:37.280343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.605029+0000","last_change":"2023-10-26T01:37:40.604955+0000","last_active":"2023-10-26T01:37:40.605029+0000","last_peered":"2023-10-26T01:37:40.605029+0000","last_clean":"2023-10-26T01:37:40.605029+0000","last_became_active":"2023-10-26T01:37:40.603088+0000","last_became_peered":"2023-10-26T01:37:40.603088+0000","last_unstale":"2023-10-26T01:37:40.605029+0000","last_undegraded":"2023-10-26T01:37:40.605029+0000","last_fullsized":"2023-10-26T01:37:40.605029+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:50:15.659835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"119'2","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.612069+0000","last_change":"2023-10-26T01:37:39.566835+0000","last_active":"2023-10-26T01:37:40.612069+0000","last_peered":"2023-10-26T01:37:40.612069+0000","last_clean":"2023-10-26T01:37:40.612069+0000","last_became_active":"2023-10-26T01:37:39.566594+0000","last_became_peered":"2023-10-26T01:37:39.566594+0000","last_unstale":"2023-10-26T01:37:40.612069+0000","last_undegraded":"2023-10-26T01:37:40.612069+0000","last_fullsized":"2023-10-26T01:37:40.612069+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_clean_scrub_stamp":"2023-10-26T01:37:38.536854+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:45:54.356734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.611721+0000","last_change":"2023-10-26T01:37:40.611322+0000","last_active":"2023-10-26T01:37:40.611721+0000","last_peered":"2023-10-26T01:37:40.611721+0000","last_clean":"2023-10-26T01:37:40.611721+0000","last_became_active":"2023-10-26T01:37:40.610605+0000","last_became_peered":"2023-10-26T01:37:40.610605+0000","last_unstale":"2023-10-26T01:37:40.611721+0000","last_undegraded":"2023-10-26T01:37:40.611721+0000","last_fullsized":"2023-10-26T01:37:40.611721+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:42:26.945329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"119'2","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.612476+0000","last_change":"2023-10-26T01:37:39.572304+0000","last_active":"2023-10-26T01:37:40.612476+0000","last_peered":"2023-10-26T01:37:40.612476+0000","last_clean":"2023-10-26T01:37:40.612476+0000","last_became_active":"2023-10-26T01:37:39.571814+0000","last_became_peered":"2023-10-26T01:37:39.571814+0000","last_unstale":"2023-10-26T01:37:40.612476+0000","last_undegraded":"2023-10-26T01:37:40.612476+0000","last_fullsized":"2023-10-26T01:37:40.612476+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_clean_scrub_stamp":"2023-10-26T01:37:38.536854+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:54:40.095332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.611145+0000","last_change":"2023-10-26T01:37:40.611058+0000","last_active":"2023-10-26T01:37:40.611145+0000","last_peered":"2023-10-26T01:37:40.611145+0000","last_clean":"2023-10-26T01:37:40.611145+0000","last_became_active":"2023-10-26T01:37:40.608611+0000","last_became_peered":"2023-10-26T01:37:40.608611+0000","last_unstale":"2023-10-26T01:37:40.611145+0000","last_undegraded":"2023-10-26T01:37:40.611145+0000","last_fullsized":"2023-10-26T01:37:40.611145+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:51:50.526372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"119'1","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.613593+0000","last_change":"2023-10-26T01:37:39.580640+0000","last_active":"2023-10-26T01:37:40.613593+0000","last_peered":"2023-10-26T01:37:40.613593+0000","last_clean":"2023-10-26T01:37:40.613593+0000","last_became_active":"2023-10-26T01:37:39.580479+0000","last_became_peered":"2023-10-26T01:37:39.580479+0000","last_unstale":"2023-10-26T01:37:40.613593+0000","last_undegraded":"2023-10-26T01:37:40.613593+0000","last_fullsized":"2023-10-26T01:37:40.613593+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_clean_scrub_stamp":"2023-10-26T01:37:38.536854+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:30:08.638588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.616890+0000","last_change":"2023-10-26T01:37:40.614007+0000","last_active":"2023-10-26T01:37:40.616890+0000","last_peered":"2023-10-26T01:37:40.616890+0000","last_clean":"2023-10-26T01:37:40.616890+0000","last_became_active":"2023-10-26T01:37:40.613733+0000","last_became_peered":"2023-10-26T01:37:40.613733+0000","last_unstale":"2023-10-26T01:37:40.616890+0000","last_undegraded":"2023-10-26T01:37:40.616890+0000","last_fullsized":"2023-10-26T01:37:40.616890+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:52:22.871839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"119'1","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.611892+0000","last_change":"2023-10-26T01:37:39.573277+0000","last_active":"2023-10-26T01:37:40.611892+0000","last_peered":"2023-10-26T01:37:40.611892+0000","last_clean":"2023-10-26T01:37:40.611892+0000","last_became_active":"2023-10-26T01:37:39.572898+0000","last_became_peered":"2023-10-26T01:37:39.572898+0000","last_unstale":"2023-10-26T01:37:40.611892+0000","last_undegraded":"2023-10-26T01:37:40.611892+0000","last_fullsized":"2023-10-26T01:37:40.611892+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_clean_scrub_stamp":"2023-10-26T01:37:38.536854+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:49:03.499843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":11,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-10-26T01:37:41.588733+0000","last_change":"2023-10-26T01:37:40.600540+0000","last_active":"2023-10-26T01:37:41.588733+0000","last_peered":"2023-10-26T01:37:41.588733+0000","last_clean":"2023-10-26T01:37:41.588733+0000","last_became_active":"2023-10-26T01:37:40.600026+0000","last_became_peered":"2023-10-26T01:37:40.600026+0000","last_unstale":"2023-10-26T01:37:41.588733+0000","last_undegraded":"2023-10-26T01:37:41.588733+0000","last_fullsized":"2023-10-26T01:37:41.588733+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:58:41.530615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":11,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.597428+0000","last_change":"2023-10-26T01:37:39.580881+0000","last_active":"2023-10-26T01:37:40.597428+0000","last_peered":"2023-10-26T01:37:40.597428+0000","last_clean":"2023-10-26T01:37:40.597428+0000","last_became_active":"2023-10-26T01:37:39.579140+0000","last_became_peered":"2023-10-26T01:37:39.579140+0000","last_unstale":"2023-10-26T01:37:40.597428+0000","last_undegraded":"2023-10-26T01:37:40.597428+0000","last_fullsized":"2023-10-26T01:37:40.597428+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_clean_scrub_stamp":"2023-10-26T01:37:38.536854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:41:33.183791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.610297+0000","last_change":"2023-10-26T01:37:40.609475+0000","last_active":"2023-10-26T01:37:40.610297+0000","last_peered":"2023-10-26T01:37:40.610297+0000","last_clean":"2023-10-26T01:37:40.610297+0000","last_became_active":"2023-10-26T01:37:40.609181+0000","last_became_peered":"2023-10-26T01:37:40.609181+0000","last_unstale":"2023-10-26T01:37:40.610297+0000","last_undegraded":"2023-10-26T01:37:40.610297+0000","last_fullsized":"2023-10-26T01:37:40.610297+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:16:38.400941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":11,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.590805+0000","last_change":"2023-10-26T01:37:39.573240+0000","last_active":"2023-10-26T01:37:40.590805+0000","last_peered":"2023-10-26T01:37:40.590805+0000","last_clean":"2023-10-26T01:37:40.590805+0000","last_became_active":"2023-10-26T01:37:39.571928+0000","last_became_peered":"2023-10-26T01:37:39.571928+0000","last_unstale":"2023-10-26T01:37:40.590805+0000","last_undegraded":"2023-10-26T01:37:40.590805+0000","last_fullsized":"2023-10-26T01:37:40.590805+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_clean_scrub_stamp":"2023-10-26T01:37:38.536854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:20:43.537466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.605504+0000","last_change":"2023-10-26T01:37:40.605386+0000","last_active":"2023-10-26T01:37:40.605504+0000","last_peered":"2023-10-26T01:37:40.605504+0000","last_clean":"2023-10-26T01:37:40.605504+0000","last_became_active":"2023-10-26T01:37:40.605027+0000","last_became_peered":"2023-10-26T01:37:40.605027+0000","last_unstale":"2023-10-26T01:37:40.605504+0000","last_undegraded":"2023-10-26T01:37:40.605504+0000","last_fullsized":"2023-10-26T01:37:40.605504+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:52:22.871839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":11,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.590266+0000","last_change":"2023-10-26T01:37:39.574600+0000","last_active":"2023-10-26T01:37:40.590266+0000","last_peered":"2023-10-26T01:37:40.590266+0000","last_clean":"2023-10-26T01:37:40.590266+0000","last_became_active":"2023-10-26T01:37:39.574159+0000","last_became_peered":"2023-10-26T01:37:39.574159+0000","last_unstale":"2023-10-26T01:37:40.590266+0000","last_undegraded":"2023-10-26T01:37:40.590266+0000","last_fullsized":"2023-10-26T01:37:40.590266+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_clean_scrub_stamp":"2023-10-26T01:37:38.536854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:18:01.338022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.612259+0000","last_change":"2023-10-26T01:37:40.611919+0000","last_active":"2023-10-26T01:37:40.612259+0000","last_peered":"2023-10-26T01:37:40.612259+0000","last_clean":"2023-10-26T01:37:40.612259+0000","last_became_active":"2023-10-26T01:37:40.611276+0000","last_became_peered":"2023-10-26T01:37:40.611276+0000","last_unstale":"2023-10-26T01:37:40.612259+0000","last_undegraded":"2023-10-26T01:37:40.612259+0000","last_fullsized":"2023-10-26T01:37:40.612259+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:43:15.435913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-10-26T01:37:41.587114+0000","last_change":"2023-10-26T01:37:39.579641+0000","last_active":"2023-10-26T01:37:41.587114+0000","last_peered":"2023-10-26T01:37:41.587114+0000","last_clean":"2023-10-26T01:37:41.587114+0000","last_became_active":"2023-10-26T01:37:39.579235+0000","last_became_peered":"2023-10-26T01:37:39.579235+0000","last_unstale":"2023-10-26T01:37:41.587114+0000","last_undegraded":"2023-10-26T01:37:41.587114+0000","last_fullsized":"2023-10-26T01:37:41.587114+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_clean_scrub_stamp":"2023-10-26T01:37:38.536854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:22:56.075780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.612141+0000","last_change":"2023-10-26T01:37:40.610343+0000","last_active":"2023-10-26T01:37:40.612141+0000","last_peered":"2023-10-26T01:37:40.612141+0000","last_clean":"2023-10-26T01:37:40.612141+0000","last_became_active":"2023-10-26T01:37:40.606987+0000","last_became_peered":"2023-10-26T01:37:40.606987+0000","last_unstale":"2023-10-26T01:37:40.612141+0000","last_undegraded":"2023-10-26T01:37:40.612141+0000","last_fullsized":"2023-10-26T01:37:40.612141+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:58:41.530615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"119'3","reported_seq":14,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.618824+0000","last_change":"2023-10-26T01:37:39.583614+0000","last_active":"2023-10-26T01:37:40.618824+0000","last_peered":"2023-10-26T01:37:40.618824+0000","last_clean":"2023-10-26T01:37:40.618824+0000","last_became_active":"2023-10-26T01:37:39.583194+0000","last_became_peered":"2023-10-26T01:37:39.583194+0000","last_unstale":"2023-10-26T01:37:40.618824+0000","last_undegraded":"2023-10-26T01:37:40.618824+0000","last_fullsized":"2023-10-26T01:37:40.618824+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_clean_scrub_stamp":"2023-10-26T01:37:38.536854+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:42:25.934808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.613462+0000","last_change":"2023-10-26T01:37:40.610086+0000","last_active":"2023-10-26T01:37:40.613462+0000","last_peered":"2023-10-26T01:37:40.613462+0000","last_clean":"2023-10-26T01:37:40.613462+0000","last_became_active":"2023-10-26T01:37:40.608497+0000","last_became_peered":"2023-10-26T01:37:40.608497+0000","last_unstale":"2023-10-26T01:37:40.613462+0000","last_undegraded":"2023-10-26T01:37:40.613462+0000","last_fullsized":"2023-10-26T01:37:40.613462+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:20:43.946448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"119'6","reported_seq":17,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.620626+0000","last_change":"2023-10-26T01:37:39.574828+0000","last_active":"2023-10-26T01:37:40.620626+0000","last_peered":"2023-10-26T01:37:40.620626+0000","last_clean":"2023-10-26T01:37:40.620626+0000","last_became_active":"2023-10-26T01:37:39.573696+0000","last_became_peered":"2023-10-26T01:37:39.573696+0000","last_unstale":"2023-10-26T01:37:40.620626+0000","last_undegraded":"2023-10-26T01:37:40.620626+0000","last_fullsized":"2023-10-26T01:37:40.620626+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_clean_scrub_stamp":"2023-10-26T01:37:38.536854+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:23:52.785370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":11,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-10-26T01:37:41.587562+0000","last_change":"2023-10-26T01:37:40.606092+0000","last_active":"2023-10-26T01:37:41.587562+0000","last_peered":"2023-10-26T01:37:41.587562+0000","last_clean":"2023-10-26T01:37:41.587562+0000","last_became_active":"2023-10-26T01:37:40.605832+0000","last_became_peered":"2023-10-26T01:37:40.605832+0000","last_unstale":"2023-10-26T01:37:41.587562+0000","last_undegraded":"2023-10-26T01:37:41.587562+0000","last_fullsized":"2023-10-26T01:37:41.587562+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:43:15.435913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"119'2","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.615045+0000","last_change":"2023-10-26T01:37:39.575193+0000","last_active":"2023-10-26T01:37:40.615045+0000","last_peered":"2023-10-26T01:37:40.615045+0000","last_clean":"2023-10-26T01:37:40.615045+0000","last_became_active":"2023-10-26T01:37:39.574952+0000","last_became_peered":"2023-10-26T01:37:39.574952+0000","last_unstale":"2023-10-26T01:37:40.615045+0000","last_undegraded":"2023-10-26T01:37:40.615045+0000","last_fullsized":"2023-10-26T01:37:40.615045+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_clean_scrub_stamp":"2023-10-26T01:37:38.536854+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:00:41.641143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.605008+0000","last_change":"2023-10-26T01:37:40.604736+0000","last_active":"2023-10-26T01:37:40.605008+0000","last_peered":"2023-10-26T01:37:40.605008+0000","last_clean":"2023-10-26T01:37:40.605008+0000","last_became_active":"2023-10-26T01:37:40.604505+0000","last_became_peered":"2023-10-26T01:37:40.604505+0000","last_unstale":"2023-10-26T01:37:40.605008+0000","last_undegraded":"2023-10-26T01:37:40.605008+0000","last_fullsized":"2023-10-26T01:37:40.605008+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:30:22.106958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":11,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.599412+0000","last_change":"2023-10-26T01:37:39.577118+0000","last_active":"2023-10-26T01:37:40.599412+0000","last_peered":"2023-10-26T01:37:40.599412+0000","last_clean":"2023-10-26T01:37:40.599412+0000","last_became_active":"2023-10-26T01:37:39.576391+0000","last_became_peered":"2023-10-26T01:37:39.576391+0000","last_unstale":"2023-10-26T01:37:40.599412+0000","last_undegraded":"2023-10-26T01:37:40.599412+0000","last_fullsized":"2023-10-26T01:37:40.599412+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_clean_scrub_stamp":"2023-10-26T01:37:38.536854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:45:14.780046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.610507+0000","last_change":"2023-10-26T01:37:40.607996+0000","last_active":"2023-10-26T01:37:40.610507+0000","last_peered":"2023-10-26T01:37:40.610507+0000","last_clean":"2023-10-26T01:37:40.610507+0000","last_became_active":"2023-10-26T01:37:40.607101+0000","last_became_peered":"2023-10-26T01:37:40.607101+0000","last_unstale":"2023-10-26T01:37:40.610507+0000","last_undegraded":"2023-10-26T01:37:40.610507+0000","last_fullsized":"2023-10-26T01:37:40.610507+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:32:02.668533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":11,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.589349+0000","last_change":"2023-10-26T01:37:39.579284+0000","last_active":"2023-10-26T01:37:40.589349+0000","last_peered":"2023-10-26T01:37:40.589349+0000","last_clean":"2023-10-26T01:37:40.589349+0000","last_became_active":"2023-10-26T01:37:39.578954+0000","last_became_peered":"2023-10-26T01:37:39.578954+0000","last_unstale":"2023-10-26T01:37:40.589349+0000","last_undegraded":"2023-10-26T01:37:40.589349+0000","last_fullsized":"2023-10-26T01:37:40.589349+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_clean_scrub_stamp":"2023-10-26T01:37:38.536854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:20:42.935927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.608820+0000","last_change":"2023-10-26T01:37:40.608399+0000","last_active":"2023-10-26T01:37:40.608820+0000","last_peered":"2023-10-26T01:37:40.608820+0000","last_clean":"2023-10-26T01:37:40.608820+0000","last_became_active":"2023-10-26T01:37:40.608120+0000","last_became_peered":"2023-10-26T01:37:40.608120+0000","last_unstale":"2023-10-26T01:37:40.608820+0000","last_undegraded":"2023-10-26T01:37:40.608820+0000","last_fullsized":"2023-10-26T01:37:40.608820+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:24:22.339565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"119'8","reported_seq":19,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.618782+0000","last_change":"2023-10-26T01:37:39.579895+0000","last_active":"2023-10-26T01:37:40.618782+0000","last_peered":"2023-10-26T01:37:40.618782+0000","last_clean":"2023-10-26T01:37:40.618782+0000","last_became_active":"2023-10-26T01:37:39.579641+0000","last_became_peered":"2023-10-26T01:37:39.579641+0000","last_unstale":"2023-10-26T01:37:40.618782+0000","last_undegraded":"2023-10-26T01:37:40.618782+0000","last_fullsized":"2023-10-26T01:37:40.618782+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_clean_scrub_stamp":"2023-10-26T01:37:38.536854+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:40:12.450735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.612185+0000","last_change":"2023-10-26T01:37:40.610736+0000","last_active":"2023-10-26T01:37:40.612185+0000","last_peered":"2023-10-26T01:37:40.612185+0000","last_clean":"2023-10-26T01:37:40.612185+0000","last_became_active":"2023-10-26T01:37:40.610095+0000","last_became_peered":"2023-10-26T01:37:40.610095+0000","last_unstale":"2023-10-26T01:37:40.612185+0000","last_undegraded":"2023-10-26T01:37:40.612185+0000","last_fullsized":"2023-10-26T01:37:40.612185+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:34:05.902643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"119'2","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.617254+0000","last_change":"2023-10-26T01:37:39.579610+0000","last_active":"2023-10-26T01:37:40.617254+0000","last_peered":"2023-10-26T01:37:40.617254+0000","last_clean":"2023-10-26T01:37:40.617254+0000","last_became_active":"2023-10-26T01:37:39.578843+0000","last_became_peered":"2023-10-26T01:37:39.578843+0000","last_unstale":"2023-10-26T01:37:40.617254+0000","last_undegraded":"2023-10-26T01:37:40.617254+0000","last_fullsized":"2023-10-26T01:37:40.617254+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_clean_scrub_stamp":"2023-10-26T01:37:38.536854+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:00:45.360362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.617114+0000","last_change":"2023-10-26T01:37:40.617026+0000","last_active":"2023-10-26T01:37:40.617114+0000","last_peered":"2023-10-26T01:37:40.617114+0000","last_clean":"2023-10-26T01:37:40.617114+0000","last_became_active":"2023-10-26T01:37:40.616771+0000","last_became_peered":"2023-10-26T01:37:40.616771+0000","last_unstale":"2023-10-26T01:37:40.617114+0000","last_undegraded":"2023-10-26T01:37:40.617114+0000","last_fullsized":"2023-10-26T01:37:40.617114+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:48:10.758339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"119'1","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.619167+0000","last_change":"2023-10-26T01:37:39.577452+0000","last_active":"2023-10-26T01:37:40.619167+0000","last_peered":"2023-10-26T01:37:40.619167+0000","last_clean":"2023-10-26T01:37:40.619167+0000","last_became_active":"2023-10-26T01:37:39.576922+0000","last_became_peered":"2023-10-26T01:37:39.576922+0000","last_unstale":"2023-10-26T01:37:40.619167+0000","last_undegraded":"2023-10-26T01:37:40.619167+0000","last_fullsized":"2023-10-26T01:37:40.619167+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_clean_scrub_stamp":"2023-10-26T01:37:38.536854+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:48:37.280343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.617083+0000","last_change":"2023-10-26T01:37:40.616124+0000","last_active":"2023-10-26T01:37:40.617083+0000","last_peered":"2023-10-26T01:37:40.617083+0000","last_clean":"2023-10-26T01:37:40.617083+0000","last_became_active":"2023-10-26T01:37:40.615180+0000","last_became_peered":"2023-10-26T01:37:40.615180+0000","last_unstale":"2023-10-26T01:37:40.617083+0000","last_undegraded":"2023-10-26T01:37:40.617083+0000","last_fullsized":"2023-10-26T01:37:40.617083+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:34:05.902643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":11,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.590989+0000","last_change":"2023-10-26T01:37:39.575894+0000","last_active":"2023-10-26T01:37:40.590989+0000","last_peered":"2023-10-26T01:37:40.590989+0000","last_clean":"2023-10-26T01:37:40.590989+0000","last_became_active":"2023-10-26T01:37:39.563576+0000","last_became_peered":"2023-10-26T01:37:39.563576+0000","last_unstale":"2023-10-26T01:37:40.590989+0000","last_undegraded":"2023-10-26T01:37:40.590989+0000","last_fullsized":"2023-10-26T01:37:40.590989+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_clean_scrub_stamp":"2023-10-26T01:37:38.536854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:26:50.722737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.611101+0000","last_change":"2023-10-26T01:37:40.610244+0000","last_active":"2023-10-26T01:37:40.611101+0000","last_peered":"2023-10-26T01:37:40.611101+0000","last_clean":"2023-10-26T01:37:40.611101+0000","last_became_active":"2023-10-26T01:37:40.609746+0000","last_became_peered":"2023-10-26T01:37:40.609746+0000","last_unstale":"2023-10-26T01:37:40.611101+0000","last_undegraded":"2023-10-26T01:37:40.611101+0000","last_fullsized":"2023-10-26T01:37:40.611101+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:29:26.369902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"119'2","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.612444+0000","last_change":"2023-10-26T01:37:39.561950+0000","last_active":"2023-10-26T01:37:40.612444+0000","last_peered":"2023-10-26T01:37:40.612444+0000","last_clean":"2023-10-26T01:37:40.612444+0000","last_became_active":"2023-10-26T01:37:39.561616+0000","last_became_peered":"2023-10-26T01:37:39.561616+0000","last_unstale":"2023-10-26T01:37:40.612444+0000","last_undegraded":"2023-10-26T01:37:40.612444+0000","last_fullsized":"2023-10-26T01:37:40.612444+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_clean_scrub_stamp":"2023-10-26T01:37:38.536854+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:41:33.183791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.600284+0000","last_change":"2023-10-26T01:37:40.600164+0000","last_active":"2023-10-26T01:37:40.600284+0000","last_peered":"2023-10-26T01:37:40.600284+0000","last_clean":"2023-10-26T01:37:40.600284+0000","last_became_active":"2023-10-26T01:37:40.599887+0000","last_became_peered":"2023-10-26T01:37:40.599887+0000","last_unstale":"2023-10-26T01:37:40.600284+0000","last_undegraded":"2023-10-26T01:37:40.600284+0000","last_fullsized":"2023-10-26T01:37:40.600284+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:40:01.956383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":11,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.604128+0000","last_change":"2023-10-26T01:37:39.571301+0000","last_active":"2023-10-26T01:37:40.604128+0000","last_peered":"2023-10-26T01:37:40.604128+0000","last_clean":"2023-10-26T01:37:40.604128+0000","last_became_active":"2023-10-26T01:37:39.570667+0000","last_became_peered":"2023-10-26T01:37:39.570667+0000","last_unstale":"2023-10-26T01:37:40.604128+0000","last_undegraded":"2023-10-26T01:37:40.604128+0000","last_fullsized":"2023-10-26T01:37:40.604128+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_clean_scrub_stamp":"2023-10-26T01:37:38.536854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:02:24.020794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.607668+0000","last_change":"2023-10-26T01:37:40.606485+0000","last_active":"2023-10-26T01:37:40.607668+0000","last_peered":"2023-10-26T01:37:40.607668+0000","last_clean":"2023-10-26T01:37:40.607668+0000","last_became_active":"2023-10-26T01:37:40.606130+0000","last_became_peered":"2023-10-26T01:37:40.606130+0000","last_unstale":"2023-10-26T01:37:40.607668+0000","last_undegraded":"2023-10-26T01:37:40.607668+0000","last_fullsized":"2023-10-26T01:37:40.607668+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:54:38.215736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":11,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.603770+0000","last_change":"2023-10-26T01:37:39.575020+0000","last_active":"2023-10-26T01:37:40.603770+0000","last_peered":"2023-10-26T01:37:40.603770+0000","last_clean":"2023-10-26T01:37:40.603770+0000","last_became_active":"2023-10-26T01:37:39.574702+0000","last_became_peered":"2023-10-26T01:37:39.574702+0000","last_unstale":"2023-10-26T01:37:40.603770+0000","last_undegraded":"2023-10-26T01:37:40.603770+0000","last_fullsized":"2023-10-26T01:37:40.603770+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_clean_scrub_stamp":"2023-10-26T01:37:38.536854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:06:49.733634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.608839+0000","last_change":"2023-10-26T01:37:40.605941+0000","last_active":"2023-10-26T01:37:40.608839+0000","last_peered":"2023-10-26T01:37:40.608839+0000","last_clean":"2023-10-26T01:37:40.608839+0000","last_became_active":"2023-10-26T01:37:40.604999+0000","last_became_peered":"2023-10-26T01:37:40.604999+0000","last_unstale":"2023-10-26T01:37:40.608839+0000","last_undegraded":"2023-10-26T01:37:40.608839+0000","last_fullsized":"2023-10-26T01:37:40.608839+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:33:21.466763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":11,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.591518+0000","last_change":"2023-10-26T01:37:39.564955+0000","last_active":"2023-10-26T01:37:40.591518+0000","last_peered":"2023-10-26T01:37:40.591518+0000","last_clean":"2023-10-26T01:37:40.591518+0000","last_became_active":"2023-10-26T01:37:39.564799+0000","last_became_peered":"2023-10-26T01:37:39.564799+0000","last_unstale":"2023-10-26T01:37:40.591518+0000","last_undegraded":"2023-10-26T01:37:40.591518+0000","last_fullsized":"2023-10-26T01:37:40.591518+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_clean_scrub_stamp":"2023-10-26T01:37:38.536854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:03:36.982769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.612888+0000","last_change":"2023-10-26T01:37:40.612635+0000","last_active":"2023-10-26T01:37:40.612888+0000","last_peered":"2023-10-26T01:37:40.612888+0000","last_clean":"2023-10-26T01:37:40.612888+0000","last_became_active":"2023-10-26T01:37:40.612141+0000","last_became_peered":"2023-10-26T01:37:40.612141+0000","last_unstale":"2023-10-26T01:37:40.612888+0000","last_undegraded":"2023-10-26T01:37:40.612888+0000","last_fullsized":"2023-10-26T01:37:40.612888+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:33:21.466763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"119'2","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.618922+0000","last_change":"2023-10-26T01:37:39.580633+0000","last_active":"2023-10-26T01:37:40.618922+0000","last_peered":"2023-10-26T01:37:40.618922+0000","last_clean":"2023-10-26T01:37:40.618922+0000","last_became_active":"2023-10-26T01:37:39.574770+0000","last_became_peered":"2023-10-26T01:37:39.574770+0000","last_unstale":"2023-10-26T01:37:40.618922+0000","last_undegraded":"2023-10-26T01:37:40.618922+0000","last_fullsized":"2023-10-26T01:37:40.618922+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_clean_scrub_stamp":"2023-10-26T01:37:38.536854+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:39:18.939024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.606003+0000","last_change":"2023-10-26T01:37:40.605736+0000","last_active":"2023-10-26T01:37:40.606003+0000","last_peered":"2023-10-26T01:37:40.606003+0000","last_clean":"2023-10-26T01:37:40.606003+0000","last_became_active":"2023-10-26T01:37:40.602584+0000","last_became_peered":"2023-10-26T01:37:40.602584+0000","last_unstale":"2023-10-26T01:37:40.606003+0000","last_undegraded":"2023-10-26T01:37:40.606003+0000","last_fullsized":"2023-10-26T01:37:40.606003+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:14:52.579489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":11,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.591733+0000","last_change":"2023-10-26T01:37:39.568014+0000","last_active":"2023-10-26T01:37:40.591733+0000","last_peered":"2023-10-26T01:37:40.591733+0000","last_clean":"2023-10-26T01:37:40.591733+0000","last_became_active":"2023-10-26T01:37:39.567684+0000","last_became_peered":"2023-10-26T01:37:39.567684+0000","last_unstale":"2023-10-26T01:37:40.591733+0000","last_undegraded":"2023-10-26T01:37:40.591733+0000","last_fullsized":"2023-10-26T01:37:40.591733+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_clean_scrub_stamp":"2023-10-26T01:37:38.536854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:01:32.458132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.615799+0000","last_change":"2023-10-26T01:37:40.615324+0000","last_active":"2023-10-26T01:37:40.615799+0000","last_peered":"2023-10-26T01:37:40.615799+0000","last_clean":"2023-10-26T01:37:40.615799+0000","last_became_active":"2023-10-26T01:37:40.615086+0000","last_became_peered":"2023-10-26T01:37:40.615086+0000","last_unstale":"2023-10-26T01:37:40.615799+0000","last_undegraded":"2023-10-26T01:37:40.615799+0000","last_fullsized":"2023-10-26T01:37:40.615799+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:24:05.475622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":11,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.589227+0000","last_change":"2023-10-26T01:37:39.566391+0000","last_active":"2023-10-26T01:37:40.589227+0000","last_peered":"2023-10-26T01:37:40.589227+0000","last_clean":"2023-10-26T01:37:40.589227+0000","last_became_active":"2023-10-26T01:37:39.565258+0000","last_became_peered":"2023-10-26T01:37:39.565258+0000","last_unstale":"2023-10-26T01:37:40.589227+0000","last_undegraded":"2023-10-26T01:37:40.589227+0000","last_fullsized":"2023-10-26T01:37:40.589227+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_clean_scrub_stamp":"2023-10-26T01:37:38.536854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:16:37.390420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.612167+0000","last_change":"2023-10-26T01:37:40.610590+0000","last_active":"2023-10-26T01:37:40.612167+0000","last_peered":"2023-10-26T01:37:40.612167+0000","last_clean":"2023-10-26T01:37:40.612167+0000","last_became_active":"2023-10-26T01:37:40.609644+0000","last_became_peered":"2023-10-26T01:37:40.609644+0000","last_unstale":"2023-10-26T01:37:40.612167+0000","last_undegraded":"2023-10-26T01:37:40.612167+0000","last_fullsized":"2023-10-26T01:37:40.612167+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:49:04.510364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":11,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.593016+0000","last_change":"2023-10-26T01:37:39.579228+0000","last_active":"2023-10-26T01:37:40.593016+0000","last_peered":"2023-10-26T01:37:40.593016+0000","last_clean":"2023-10-26T01:37:40.593016+0000","last_became_active":"2023-10-26T01:37:39.578701+0000","last_became_peered":"2023-10-26T01:37:39.578701+0000","last_unstale":"2023-10-26T01:37:40.593016+0000","last_undegraded":"2023-10-26T01:37:40.593016+0000","last_fullsized":"2023-10-26T01:37:40.593016+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_clean_scrub_stamp":"2023-10-26T01:37:38.536854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:43:18.947747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.603446+0000","last_change":"2023-10-26T01:37:40.602214+0000","last_active":"2023-10-26T01:37:40.603446+0000","last_peered":"2023-10-26T01:37:40.603446+0000","last_clean":"2023-10-26T01:37:40.603446+0000","last_became_active":"2023-10-26T01:37:40.602005+0000","last_became_peered":"2023-10-26T01:37:40.602005+0000","last_unstale":"2023-10-26T01:37:40.603446+0000","last_undegraded":"2023-10-26T01:37:40.603446+0000","last_fullsized":"2023-10-26T01:37:40.603446+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:53:10.171579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":11,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.591659+0000","last_change":"2023-10-26T01:37:39.572879+0000","last_active":"2023-10-26T01:37:40.591659+0000","last_peered":"2023-10-26T01:37:40.591659+0000","last_clean":"2023-10-26T01:37:40.591659+0000","last_became_active":"2023-10-26T01:37:39.572576+0000","last_became_peered":"2023-10-26T01:37:39.572576+0000","last_unstale":"2023-10-26T01:37:40.591659+0000","last_undegraded":"2023-10-26T01:37:40.591659+0000","last_fullsized":"2023-10-26T01:37:40.591659+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_clean_scrub_stamp":"2023-10-26T01:37:38.536854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:32:52.093485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.612278+0000","last_change":"2023-10-26T01:37:40.612077+0000","last_active":"2023-10-26T01:37:40.612278+0000","last_peered":"2023-10-26T01:37:40.612278+0000","last_clean":"2023-10-26T01:37:40.612278+0000","last_became_active":"2023-10-26T01:37:40.611672+0000","last_became_peered":"2023-10-26T01:37:40.611672+0000","last_unstale":"2023-10-26T01:37:40.612278+0000","last_undegraded":"2023-10-26T01:37:40.612278+0000","last_fullsized":"2023-10-26T01:37:40.612278+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:48:38.290864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":11,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.589942+0000","last_change":"2023-10-26T01:37:39.581597+0000","last_active":"2023-10-26T01:37:40.589942+0000","last_peered":"2023-10-26T01:37:40.589942+0000","last_clean":"2023-10-26T01:37:40.589942+0000","last_became_active":"2023-10-26T01:37:39.581071+0000","last_became_peered":"2023-10-26T01:37:39.581071+0000","last_unstale":"2023-10-26T01:37:40.589942+0000","last_undegraded":"2023-10-26T01:37:40.589942+0000","last_fullsized":"2023-10-26T01:37:40.589942+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_clean_scrub_stamp":"2023-10-26T01:37:38.536854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:16:37.390420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.611069+0000","last_change":"2023-10-26T01:37:40.609943+0000","last_active":"2023-10-26T01:37:40.611069+0000","last_peered":"2023-10-26T01:37:40.611069+0000","last_clean":"2023-10-26T01:37:40.611069+0000","last_became_active":"2023-10-26T01:37:40.609152+0000","last_became_peered":"2023-10-26T01:37:40.609152+0000","last_unstale":"2023-10-26T01:37:40.611069+0000","last_undegraded":"2023-10-26T01:37:40.611069+0000","last_fullsized":"2023-10-26T01:37:40.611069+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:04:58.972773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-10-26T01:37:41.587913+0000","last_change":"2023-10-26T01:37:39.571943+0000","last_active":"2023-10-26T01:37:41.587913+0000","last_peered":"2023-10-26T01:37:41.587913+0000","last_clean":"2023-10-26T01:37:41.587913+0000","last_became_active":"2023-10-26T01:37:39.571713+0000","last_became_peered":"2023-10-26T01:37:39.571713+0000","last_unstale":"2023-10-26T01:37:41.587913+0000","last_undegraded":"2023-10-26T01:37:41.587913+0000","last_fullsized":"2023-10-26T01:37:41.587913+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_clean_scrub_stamp":"2023-10-26T01:37:38.536854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:26:21.745971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.603047+0000","last_change":"2023-10-26T01:37:40.598934+0000","last_active":"2023-10-26T01:37:40.603047+0000","last_peered":"2023-10-26T01:37:40.603047+0000","last_clean":"2023-10-26T01:37:40.603047+0000","last_became_active":"2023-10-26T01:37:40.597261+0000","last_became_peered":"2023-10-26T01:37:40.597261+0000","last_unstale":"2023-10-26T01:37:40.603047+0000","last_undegraded":"2023-10-26T01:37:40.603047+0000","last_fullsized":"2023-10-26T01:37:40.603047+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:20:43.946448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":11,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.598663+0000","last_change":"2023-10-26T01:37:39.570969+0000","last_active":"2023-10-26T01:37:40.598663+0000","last_peered":"2023-10-26T01:37:40.598663+0000","last_clean":"2023-10-26T01:37:40.598663+0000","last_became_active":"2023-10-26T01:37:39.562270+0000","last_became_peered":"2023-10-26T01:37:39.562270+0000","last_unstale":"2023-10-26T01:37:40.598663+0000","last_undegraded":"2023-10-26T01:37:40.598663+0000","last_fullsized":"2023-10-26T01:37:40.598663+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_clean_scrub_stamp":"2023-10-26T01:37:38.536854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:48:13.041798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.602184+0000","last_change":"2023-10-26T01:37:40.601685+0000","last_active":"2023-10-26T01:37:40.602184+0000","last_peered":"2023-10-26T01:37:40.602184+0000","last_clean":"2023-10-26T01:37:40.602184+0000","last_became_active":"2023-10-26T01:37:40.601449+0000","last_became_peered":"2023-10-26T01:37:40.601449+0000","last_unstale":"2023-10-26T01:37:40.602184+0000","last_undegraded":"2023-10-26T01:37:40.602184+0000","last_fullsized":"2023-10-26T01:37:40.602184+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:18:04.431280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"119'1","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.616136+0000","last_change":"2023-10-26T01:37:39.580512+0000","last_active":"2023-10-26T01:37:40.616136+0000","last_peered":"2023-10-26T01:37:40.616136+0000","last_clean":"2023-10-26T01:37:40.616136+0000","last_became_active":"2023-10-26T01:37:39.579888+0000","last_became_peered":"2023-10-26T01:37:39.579888+0000","last_unstale":"2023-10-26T01:37:40.616136+0000","last_undegraded":"2023-10-26T01:37:40.616136+0000","last_fullsized":"2023-10-26T01:37:40.616136+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_clean_scrub_stamp":"2023-10-26T01:37:38.536854+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:16:37.390420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":266,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-26T01:37:39.577609+0000","last_change":"2023-10-26T01:30:34.883484+0000","last_active":"2023-10-26T01:37:39.577609+0000","last_peered":"2023-10-26T01:37:39.577609+0000","last_clean":"2023-10-26T01:37:39.577609+0000","last_became_active":"2023-10-26T01:30:34.883071+0000","last_became_peered":"2023-10-26T01:30:34.883071+0000","last_unstale":"2023-10-26T01:37:39.577609+0000","last_undegraded":"2023-10-26T01:37:39.577609+0000","last_fullsized":"2023-10-26T01:37:39.577609+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:32.890096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:32.890096+0000","last_clean_scrub_stamp":"2023-10-26T01:30:32.890096+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:26:31.879452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":14,"seq":60129542233,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":646,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194666,"statfs":{"total":200000000,"available":199337988,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.79800000000000004,"15min":0.78700000000000003},"min":{"1min":0.433,"5min":0.35599999999999998,"15min":0.33000000000000002},"max":{"1min":1.3029999999999999,"5min":1.454,"15min":2.2549999999999999},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.85899999999999999,"15min":0.89200000000000002},"min":{"1min":0.52400000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.3049999999999999,"5min":1.5800000000000001,"15min":1.8200000000000001},"last":0.58599999999999997}]},{"osd":1,"last update":"Thu Oct 26 01:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.84699999999999998,"15min":0.94699999999999995},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.23,"5min":1.444,"15min":4.4340000000000002},"last":0.76000000000000001},{"interface":"front","average":{"1min":1.006,"5min":0.97099999999999997,"15min":1.0029999999999999},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.374},"max":{"1min":1.419,"5min":1.538,"15min":2.3330000000000002},"last":0.70799999999999996}]},{"osd":2,"last update":"Thu Oct 26 01:36:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":1.0189999999999999,"15min":1.083},"min":{"1min":0.40999999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.927,"5min":1.927,"15min":4.2800000000000002},"last":0.60699999999999998},{"interface":"front","average":{"1min":1.1220000000000001,"5min":1.129,"15min":1.0940000000000001},"min":{"1min":0.495,"5min":0.495,"15min":0.49399999999999999},"max":{"1min":1.8680000000000001,"5min":1.8680000000000001,"15min":2.302},"last":1.0109999999999999}]},{"osd":4,"last update":"Thu Oct 26 01:36:47 2023","interfaces":[{"interface":"back","average":{"1min":1.2569999999999999,"5min":1.1970000000000001,"15min":1.165},"min":{"1min":0.89300000000000002,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.7150000000000001,"5min":1.7150000000000001,"15min":2.1070000000000002},"last":1.141},{"interface":"front","average":{"1min":1.395,"5min":1.3360000000000001,"15min":1.401},"min":{"1min":0.89400000000000002,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":2.4239999999999999,"5min":2.4239999999999999,"15min":4.8250000000000002},"last":1.1639999999999999}]},{"osd":5,"last update":"Thu Oct 26 01:36:47 2023","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.2649999999999999,"15min":1.343},"min":{"1min":0.76400000000000001,"5min":0.76200000000000001,"15min":0.72499999999999998},"max":{"1min":2.3380000000000001,"5min":2.3380000000000001,"15min":4.9080000000000004},"last":1.0629999999999999},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.274,"15min":1.278},"min":{"1min":0.86699999999999999,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":2.0310000000000001,"5min":2.0310000000000001,"15min":2.8980000000000001},"last":1.111}]},{"osd":6,"last update":"Thu Oct 26 01:36:47 2023","interfaces":[{"interface":"back","average":{"1min":1.4710000000000001,"5min":1.3600000000000001,"15min":1.3340000000000001},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":2.1499999999999999,"5min":2.1499999999999999,"15min":2.8530000000000002},"last":1.266},{"interface":"front","average":{"1min":1.347,"5min":1.2609999999999999,"15min":1.2729999999999999},"min":{"1min":0.81899999999999995,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":2.048,"5min":2.048,"15min":2.181},"last":0.97899999999999998}]},{"osd":7,"last update":"Thu Oct 26 01:36:47 2023","interfaces":[{"interface":"back","average":{"1min":1.417,"5min":1.3220000000000001,"15min":1.298},"min":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.71999999999999997},"max":{"1min":1.958,"5min":1.958,"15min":2.0680000000000001},"last":0.85299999999999998},{"interface":"front","average":{"1min":1.448,"5min":1.379,"15min":1.4259999999999999},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":2.2370000000000001,"5min":2.2370000000000001,"15min":4.8730000000000002},"last":0.95499999999999996}]}]},{"osd":7,"up_from":13,"seq":55834574936,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1208,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194103,"statfs":{"total":200000000,"available":198762329,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:36:45 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.145,"15min":1.091},"min":{"1min":0.69699999999999995,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.6020000000000001,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":1.1890000000000001},{"interface":"front","average":{"1min":1.036,"5min":1.157,"15min":1.085},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.28,"5min":2.6779999999999999,"15min":2.6779999999999999},"last":0.70599999999999996}]},{"osd":1,"last update":"Thu Oct 26 01:36:45 2023","interfaces":[{"interface":"back","average":{"1min":1.1699999999999999,"5min":1.2050000000000001,"15min":1.091},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.65200000000000002},"max":{"1min":1.6040000000000001,"5min":2.8260000000000001,"15min":2.8260000000000001},"last":0.92200000000000004},{"interface":"front","average":{"1min":1.097,"5min":1.216,"15min":1.1879999999999999},"min":{"1min":0.68799999999999994,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.542,"5min":3.2730000000000001,"15min":3.2730000000000001},"last":0.88600000000000001}]},{"osd":2,"last update":"Thu Oct 26 01:36:45 2023","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.175,"15min":1.1140000000000001},"min":{"1min":0.69099999999999995,"5min":0.61199999999999999,"15min":0.58499999999999996},"max":{"1min":1.8680000000000001,"5min":2.218,"15min":2.218},"last":1.1180000000000001},{"interface":"front","average":{"1min":1.2909999999999999,"5min":1.296,"15min":1.236},"min":{"1min":0.79400000000000004,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.8999999999999999,"5min":3.0510000000000002,"15min":3.0510000000000002},"last":1.036}]},{"osd":3,"last update":"Thu Oct 26 01:36:45 2023","interfaces":[{"interface":"back","average":{"1min":1.264,"5min":1.272,"15min":1.208},"min":{"1min":0.80600000000000005,"5min":0.79000000000000004,"15min":0.72099999999999997},"max":{"1min":1.796,"5min":1.9830000000000001,"15min":1.9830000000000001},"last":1.25},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.419,"15min":1.377},"min":{"1min":0.82899999999999996,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.8680000000000001,"5min":5.5039999999999996,"15min":5.5039999999999996},"last":0.98099999999999998}]},{"osd":4,"last update":"Thu Oct 26 01:36:45 2023","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.3500000000000001,"15min":1.3460000000000001},"min":{"1min":0.77300000000000002,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.5640000000000001,"5min":2.77,"15min":2.77},"last":1.4770000000000001},{"interface":"front","average":{"1min":1.298,"5min":1.425,"15min":1.3939999999999999},"min":{"1min":0.96899999999999997,"5min":0.496,"15min":0.496},"max":{"1min":1.6830000000000001,"5min":3.1419999999999999,"15min":3.1419999999999999},"last":1.29}]},{"osd":5,"last update":"Thu Oct 26 01:36:45 2023","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.4710000000000001,"15min":1.419},"min":{"1min":0.65600000000000003,"5min":0.441,"15min":0.441},"max":{"1min":1.746,"5min":5.734,"15min":5.734},"last":1.081},{"interface":"front","average":{"1min":1.4339999999999999,"5min":1.5109999999999999,"15min":1.476},"min":{"1min":0.96099999999999997,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.829,"5min":2.8980000000000001,"15min":2.8980000000000001},"last":1.4159999999999999}]},{"osd":6,"last update":"Thu Oct 26 01:36:45 2023","interfaces":[{"interface":"back","average":{"1min":1.4670000000000001,"5min":1.603,"15min":1.877},"min":{"1min":0.753,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.7430000000000001,"5min":5.6340000000000003,"15min":11.577999999999999},"last":1.3540000000000001},{"interface":"front","average":{"1min":1.552,"5min":1.661,"15min":1.8879999999999999},"min":{"1min":0.91800000000000004,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.9339999999999999,"5min":5.0919999999999996,"15min":10.704000000000001},"last":1.319}]}]},{"osd":6,"up_from":13,"seq":55834574936,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":631,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194680,"statfs":{"total":200000000,"available":199353311,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:36:42 2023","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.097,"15min":1.1599999999999999},"min":{"1min":0.61699999999999999,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.647,"5min":2.395,"15min":2.395},"last":1.258},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.1299999999999999,"15min":1.1200000000000001},"min":{"1min":0.72899999999999998,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.7390000000000001,"5min":1.7390000000000001,"15min":1.843},"last":1.0109999999999999}]},{"osd":1,"last update":"Thu Oct 26 01:36:42 2023","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.1639999999999999,"15min":1.1719999999999999},"min":{"1min":0.74399999999999999,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.784,"5min":2.0529999999999999,"15min":2.0529999999999999},"last":1.5920000000000001},{"interface":"front","average":{"1min":1.139,"5min":1.1519999999999999,"15min":1.2010000000000001},"min":{"1min":0.79000000000000004,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.5229999999999999,"5min":1.581,"15min":2.085},"last":1.208}]},{"osd":2,"last update":"Thu Oct 26 01:36:42 2023","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.1779999999999999,"15min":1.0940000000000001},"min":{"1min":0.83099999999999996,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.7949999999999999,"5min":3.7879999999999998,"15min":3.7879999999999998},"last":1.3480000000000001},{"interface":"front","average":{"1min":1.319,"5min":1.2829999999999999,"15min":1.2589999999999999},"min":{"1min":0.85799999999999998,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.9910000000000001,"5min":2.2890000000000001,"15min":2.2890000000000001},"last":1.5029999999999999}]},{"osd":3,"last update":"Thu Oct 26 01:36:42 2023","interfaces":[{"interface":"back","average":{"1min":1.391,"5min":1.3080000000000001,"15min":1.2070000000000001},"min":{"1min":0.871,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.8600000000000001,"5min":4.4560000000000004,"15min":4.4560000000000004},"last":1.446},{"interface":"front","average":{"1min":1.4119999999999999,"5min":1.341,"15min":1.327},"min":{"1min":1.0369999999999999,"5min":0.82299999999999995,"15min":0.69499999999999995},"max":{"1min":1.877,"5min":2.5019999999999998,"15min":2.5019999999999998},"last":1.4039999999999999}]},{"osd":4,"last update":"Thu Oct 26 01:36:42 2023","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.252,"15min":1.22},"min":{"1min":0.93200000000000005,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.7,"5min":2.9489999999999998,"15min":2.9489999999999998},"last":1.3069999999999999},{"interface":"front","average":{"1min":1.4339999999999999,"5min":1.4330000000000001,"15min":1.3140000000000001},"min":{"1min":0.93400000000000005,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.9039999999999999,"5min":6.609,"15min":6.609},"last":1.1100000000000001}]},{"osd":5,"last update":"Thu Oct 26 01:36:42 2023","interfaces":[{"interface":"back","average":{"1min":1.6339999999999999,"5min":1.548,"15min":1.4430000000000001},"min":{"1min":1.2030000000000001,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":2.1240000000000001,"5min":5.4370000000000003,"15min":5.4370000000000003},"last":1.6200000000000001},{"interface":"front","average":{"1min":1.5309999999999999,"5min":1.448,"15min":1.409},"min":{"1min":1.1319999999999999,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.8169999999999999,"5min":2.621,"15min":2.621},"last":1.165}]},{"osd":7,"last update":"Thu Oct 26 01:36:42 2023","interfaces":[{"interface":"back","average":{"1min":1.4650000000000001,"5min":1.5229999999999999,"15min":1.411},"min":{"1min":1.2589999999999999,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.9770000000000001,"5min":11.804,"15min":11.804},"last":1.4850000000000001},{"interface":"front","average":{"1min":1.6299999999999999,"5min":1.657,"15min":1.5049999999999999},"min":{"1min":1.296,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.992,"5min":11.541,"15min":11.541},"last":1.53}]}]},{"osd":5,"up_from":13,"seq":55834574936,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":632,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194680,"statfs":{"total":200000000,"available":199352697,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:36:44 2023","interfaces":[{"interface":"back","average":{"1min":1.143,"5min":1.1739999999999999,"15min":1.1240000000000001},"min":{"1min":0.746,"5min":0.66200000000000003,"15min":0.63200000000000001},"max":{"1min":1.546,"5min":1.7390000000000001,"15min":1.7390000000000001},"last":1.464},{"interface":"front","average":{"1min":1.262,"5min":1.2709999999999999,"15min":1.177},"min":{"1min":0.81699999999999995,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.601,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":1.1100000000000001}]},{"osd":1,"last update":"Thu Oct 26 01:36:44 2023","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.1439999999999999,"15min":1.091},"min":{"1min":0.80700000000000005,"5min":0.67600000000000005,"15min":0.66400000000000003},"max":{"1min":1.7829999999999999,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":1.0449999999999999},{"interface":"front","average":{"1min":1.1040000000000001,"5min":1.1599999999999999,"15min":1.095},"min":{"1min":0.85799999999999998,"5min":0.68200000000000005,"15min":0.60599999999999998},"max":{"1min":1.4570000000000001,"5min":1.6000000000000001,"15min":1.7669999999999999},"last":1.726}]},{"osd":2,"last update":"Thu Oct 26 01:36:44 2023","interfaces":[{"interface":"back","average":{"1min":1.2290000000000001,"5min":1.204,"15min":1.1619999999999999},"min":{"1min":0.79600000000000004,"5min":0.74399999999999999,"15min":0.69299999999999995},"max":{"1min":1.6879999999999999,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":1.681},{"interface":"front","average":{"1min":1.3320000000000001,"5min":1.2350000000000001,"15min":1.1850000000000001},"min":{"1min":0.93600000000000005,"5min":0.85499999999999998,"15min":0.63900000000000001},"max":{"1min":1.6899999999999999,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":1.359}]},{"osd":3,"last update":"Thu Oct 26 01:36:44 2023","interfaces":[{"interface":"back","average":{"1min":1.2809999999999999,"5min":1.3080000000000001,"15min":1.246},"min":{"1min":0.92200000000000004,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.613,"5min":1.7649999999999999,"15min":1.835},"last":1.6180000000000001},{"interface":"front","average":{"1min":1.373,"5min":1.321,"15min":1.2949999999999999},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.788,"5min":1.788,"15min":2.9249999999999998},"last":1.4139999999999999}]},{"osd":4,"last update":"Thu Oct 26 01:36:44 2023","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.286,"15min":1.206},"min":{"1min":0.97699999999999998,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.845,"5min":1.845,"15min":1.845},"last":1.2929999999999999},{"interface":"front","average":{"1min":1.4810000000000001,"5min":1.4470000000000001,"15min":1.353},"min":{"1min":0.90600000000000003,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.7849999999999999,"5min":1.867,"15min":1.867},"last":1.5289999999999999}]},{"osd":6,"last update":"Thu Oct 26 01:36:44 2023","interfaces":[{"interface":"back","average":{"1min":1.5269999999999999,"5min":1.444,"15min":1.3220000000000001},"min":{"1min":1.0660000000000001,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.883,"5min":2.8250000000000002,"15min":2.8250000000000002},"last":1.7490000000000001},{"interface":"front","average":{"1min":1.5429999999999999,"5min":1.4490000000000001,"15min":1.3260000000000001},"min":{"1min":1.1879999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.849,"5min":1.849,"15min":1.8560000000000001},"last":1.5880000000000001}]},{"osd":7,"last update":"Thu Oct 26 01:36:44 2023","interfaces":[{"interface":"back","average":{"1min":1.599,"5min":1.5449999999999999,"15min":1.417},"min":{"1min":1.119,"5min":0.64000000000000001,"15min":0.61699999999999999},"max":{"1min":2.1040000000000001,"5min":2.1040000000000001,"15min":2.1040000000000001},"last":1.8220000000000001},{"interface":"front","average":{"1min":1.6619999999999999,"5min":1.5660000000000001,"15min":1.4299999999999999},"min":{"1min":1.389,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":2.0960000000000001,"5min":2.0960000000000001,"15min":2.0960000000000001},"last":1.853}]}]},{"osd":4,"up_from":13,"seq":55834574936,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":632,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194680,"statfs":{"total":200000000,"available":199352697,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:36:37 2023","interfaces":[{"interface":"back","average":{"1min":1.081,"5min":1.04,"15min":0.97699999999999998},"min":{"1min":0.69199999999999995,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.5309999999999999,"5min":2.0289999999999999,"15min":2.0289999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":1.071,"5min":1.0649999999999999,"15min":1.0229999999999999},"min":{"1min":0.66400000000000003,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.6599999999999999,"5min":2.6829999999999998,"15min":2.6829999999999998},"last":0.76500000000000001}]},{"osd":1,"last update":"Thu Oct 26 01:36:37 2023","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.1799999999999999,"15min":1.1220000000000001},"min":{"1min":0.98399999999999999,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.599,"5min":4.0519999999999996,"15min":4.0519999999999996},"last":0.94199999999999995},{"interface":"front","average":{"1min":1.119,"5min":1.1439999999999999,"15min":1.105},"min":{"1min":0.77900000000000003,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.6240000000000001,"5min":2.0760000000000001,"15min":2.0760000000000001},"last":1.2669999999999999}]},{"osd":2,"last update":"Thu Oct 26 01:36:37 2023","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":1.3280000000000001,"15min":1.222},"min":{"1min":0.79100000000000004,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.5660000000000001,"5min":5.6749999999999998,"15min":5.6749999999999998},"last":1.054},{"interface":"front","average":{"1min":1.1970000000000001,"5min":1.1879999999999999,"15min":1.1220000000000001},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.746,"5min":1.849,"15min":1.849},"last":1.345}]},{"osd":3,"last update":"Thu Oct 26 01:36:37 2023","interfaces":[{"interface":"back","average":{"1min":1.286,"5min":1.2,"15min":1.1839999999999999},"min":{"1min":0.84399999999999997,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.847,"5min":1.891,"15min":1.891},"last":1.101},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.2030000000000001,"15min":1.155},"min":{"1min":0.94499999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.5669999999999999,"5min":1.7210000000000001,"15min":1.7210000000000001},"last":1.173}]},{"osd":5,"last update":"Thu Oct 26 01:36:37 2023","interfaces":[{"interface":"back","average":{"1min":1.2749999999999999,"5min":1.2509999999999999,"15min":1.286},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.46300000000000002},"max":{"1min":2.1429999999999998,"5min":2.1429999999999998,"15min":2.1429999999999998},"last":1.399},{"interface":"front","average":{"1min":1.28,"5min":1.331,"15min":1.327},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.716,"5min":5.8579999999999997,"15min":5.8579999999999997},"last":1.226}]},{"osd":6,"last update":"Thu Oct 26 01:36:37 2023","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":1.4199999999999999,"15min":1.3340000000000001},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.639,"5min":12.343999999999999,"15min":12.343999999999999},"last":1.4350000000000001},{"interface":"front","average":{"1min":1.54,"5min":1.6220000000000001,"15min":1.5129999999999999},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.55000000000000004},"max":{"1min":2.0369999999999999,"5min":12.605,"15min":12.605},"last":1.4650000000000001}]},{"osd":7,"last update":"Thu Oct 26 01:36:37 2023","interfaces":[{"interface":"back","average":{"1min":1.4650000000000001,"5min":1.518,"15min":1.4910000000000001},"min":{"1min":0.748,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":2.762,"5min":5.774,"15min":5.774},"last":1.306},{"interface":"front","average":{"1min":1.5249999999999999,"5min":1.518,"15min":1.4530000000000001},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":2.8330000000000002,"5min":4.6520000000000001,"15min":4.6520000000000001},"last":1.4830000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574936,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":632,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194680,"statfs":{"total":200000000,"available":199352697,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":0.875,"15min":0.86099999999999999},"min":{"1min":0.63,"5min":0.308,"15min":0.308},"max":{"1min":1.5249999999999999,"5min":6.3200000000000003,"15min":6.3200000000000003},"last":1.1899999999999999},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.0229999999999999,"15min":0.93600000000000005},"min":{"1min":0.52000000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.7130000000000001,"5min":4.2619999999999996,"15min":4.2619999999999996},"last":1.123}]},{"osd":1,"last update":"Thu Oct 26 01:36:41 2023","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":0.93200000000000005,"15min":0.90500000000000003},"min":{"1min":0.66700000000000004,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.629,"5min":5.1260000000000003,"15min":5.1260000000000003},"last":0.99399999999999999},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.0649999999999999,"15min":1.012},"min":{"1min":0.59599999999999997,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.873,"5min":6.1520000000000001,"15min":6.1520000000000001},"last":0.93200000000000005}]},{"osd":3,"last update":"Thu Oct 26 01:36:47 2023","interfaces":[{"interface":"back","average":{"1min":1.131,"5min":1.034,"15min":0.93999999999999995},"min":{"1min":0.54900000000000004,"5min":0.443,"15min":0.443},"max":{"1min":2.0640000000000001,"5min":2.0640000000000001,"15min":2.0640000000000001},"last":1.2989999999999999},{"interface":"front","average":{"1min":1.22,"5min":1.1699999999999999,"15min":1.0189999999999999},"min":{"1min":0.67300000000000004,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.9279999999999999,"5min":8.9019999999999992,"15min":8.9019999999999992},"last":1.325}]},{"osd":4,"last update":"Thu Oct 26 01:36:41 2023","interfaces":[{"interface":"back","average":{"1min":1.454,"5min":1.365,"15min":1.268},"min":{"1min":0.80500000000000005,"5min":0.751,"15min":0.73899999999999999},"max":{"1min":2.6190000000000002,"5min":6.2670000000000003,"15min":6.2670000000000003},"last":1.5469999999999999},{"interface":"front","average":{"1min":1.5089999999999999,"5min":1.367,"15min":1.2629999999999999},"min":{"1min":0.93700000000000006,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":2.4359999999999999,"5min":7.343,"15min":7.343},"last":1.411}]},{"osd":5,"last update":"Thu Oct 26 01:36:41 2023","interfaces":[{"interface":"back","average":{"1min":1.5649999999999999,"5min":1.369,"15min":1.2909999999999999},"min":{"1min":1.018,"5min":0.80800000000000005,"15min":0.71899999999999997},"max":{"1min":2.5790000000000002,"5min":3.7170000000000001,"15min":3.7170000000000001},"last":1.222},{"interface":"front","average":{"1min":1.4650000000000001,"5min":1.3540000000000001,"15min":1.3},"min":{"1min":0.88800000000000001,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":2.298,"5min":6.3899999999999997,"15min":6.3899999999999997},"last":1.2649999999999999}]},{"osd":6,"last update":"Thu Oct 26 01:36:41 2023","interfaces":[{"interface":"back","average":{"1min":1.4690000000000001,"5min":1.4259999999999999,"15min":1.335},"min":{"1min":0.871,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":2.73,"5min":6.0490000000000004,"15min":6.0490000000000004},"last":1.9379999999999999},{"interface":"front","average":{"1min":1.599,"5min":1.4399999999999999,"15min":1.375},"min":{"1min":1.0660000000000001,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":2.4089999999999998,"5min":5.0650000000000004,"15min":5.0650000000000004},"last":1.575}]},{"osd":7,"last update":"Thu Oct 26 01:36:41 2023","interfaces":[{"interface":"back","average":{"1min":1.532,"5min":1.4390000000000001,"15min":1.405},"min":{"1min":1.0449999999999999,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":2.3370000000000002,"5min":6.4480000000000004,"15min":6.4480000000000004},"last":1.488},{"interface":"front","average":{"1min":1.6279999999999999,"5min":1.5169999999999999,"15min":1.4570000000000001},"min":{"1min":1.117,"5min":0.91800000000000004,"15min":0.91800000000000004},"max":{"1min":2.8290000000000002,"5min":2.8439999999999999,"15min":2.8439999999999999},"last":1.877}]}]},{"osd":1,"up_from":13,"seq":55834574936,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":638,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194673,"statfs":{"total":200000000,"available":199345922,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.78100000000000003,"15min":0.72099999999999997},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.29499999999999998},"max":{"1min":1.0820000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.85599999999999998,"15min":0.79400000000000004},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36199999999999999},"max":{"1min":1.589,"5min":1.6399999999999999,"15min":1.6399999999999999},"last":1.331}]},{"osd":2,"last update":"Thu Oct 26 01:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.94599999999999995,"15min":0.99399999999999999},"min":{"1min":0.55300000000000005,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.5049999999999999,"5min":1.5389999999999999,"15min":2.4830000000000001},"last":1.099},{"interface":"front","average":{"1min":1.0209999999999999,"5min":0.89700000000000002,"15min":0.86099999999999999},"min":{"1min":0.57099999999999995,"5min":0.432,"15min":0.432},"max":{"1min":1.8080000000000001,"5min":1.8080000000000001,"15min":1.8080000000000001},"last":0.71899999999999997}]},{"osd":3,"last update":"Thu Oct 26 01:36:50 2023","interfaces":[{"interface":"back","average":{"1min":1.0389999999999999,"5min":0.98499999999999999,"15min":0.94699999999999995},"min":{"1min":0.42499999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.5289999999999999,"5min":1.75,"15min":1.75},"last":1.052},{"interface":"front","average":{"1min":1.095,"5min":1.0289999999999999,"15min":0.98899999999999999},"min":{"1min":0.54200000000000004,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.673,"5min":1.802,"15min":1.802},"last":1.2849999999999999}]},{"osd":4,"last update":"Thu Oct 26 01:36:46 2023","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.2250000000000001,"15min":1.196},"min":{"1min":0.69099999999999995,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.706,"5min":1.8129999999999999,"15min":2.0659999999999998},"last":1.407},{"interface":"front","average":{"1min":1.325,"5min":1.1950000000000001,"15min":1.137},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":2.0270000000000001,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":1.143}]},{"osd":5,"last update":"Thu Oct 26 01:36:46 2023","interfaces":[{"interface":"back","average":{"1min":1.3360000000000001,"5min":1.262,"15min":1.232},"min":{"1min":0.91800000000000004,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":1.8779999999999999,"5min":1.8779999999999999,"15min":1.8779999999999999},"last":1.573},{"interface":"front","average":{"1min":1.3620000000000001,"5min":1.347,"15min":1.3340000000000001},"min":{"1min":0.66000000000000003,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.744,"5min":1.782,"15min":2.2200000000000002},"last":1.534}]},{"osd":6,"last update":"Thu Oct 26 01:36:46 2023","interfaces":[{"interface":"back","average":{"1min":1.367,"5min":1.3180000000000001,"15min":1.278},"min":{"1min":0.70199999999999996,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.9179999999999999,"5min":2.0910000000000002,"15min":2.3780000000000001},"last":1.1910000000000001},{"interface":"front","average":{"1min":1.3260000000000001,"5min":1.276,"15min":1.242},"min":{"1min":0.89600000000000002,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.956,"5min":1.956,"15min":1.984},"last":1.226}]},{"osd":7,"last update":"Thu Oct 26 01:36:46 2023","interfaces":[{"interface":"back","average":{"1min":1.3620000000000001,"5min":1.262,"15min":1.268},"min":{"1min":0.95699999999999996,"5min":0.78000000000000003,"15min":0.69699999999999995},"max":{"1min":1.8380000000000001,"5min":1.8380000000000001,"15min":2.6930000000000001},"last":1.482},{"interface":"front","average":{"1min":1.506,"5min":1.3919999999999999,"15min":1.3320000000000001},"min":{"1min":0.75600000000000001,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":2.0190000000000001,"5min":2.0190000000000001,"15min":2.0190000000000001},"last":1.371}]}]},{"osd":0,"up_from":13,"seq":55834574936,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1215,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194096,"statfs":{"total":200000000,"available":198754972,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Oct 26 01:36:43 2023","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.998,"15min":0.92300000000000004},"min":{"1min":0.64000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.49,"5min":19.425000000000001,"15min":19.425000000000001},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.98199999999999998,"5min":1.075,"15min":1.0549999999999999},"min":{"1min":0.67400000000000004,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.339,"5min":15.162000000000001,"15min":15.162000000000001},"last":0.56100000000000005}]},{"osd":2,"last update":"Thu Oct 26 01:36:43 2023","interfaces":[{"interface":"back","average":{"1min":1,"5min":1.2969999999999999,"15min":1.1930000000000001},"min":{"1min":0.59899999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.4970000000000001,"5min":28.439,"15min":28.439},"last":0.94999999999999996},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.90800000000000003,"15min":0.91400000000000003},"min":{"1min":0.69399999999999995,"5min":0.45200000000000001,"15min":0.38300000000000001},"max":{"1min":1.268,"5min":1.502,"15min":4.9459999999999997},"last":0.90500000000000003}]},{"osd":3,"last update":"Thu Oct 26 01:36:43 2023","interfaces":[{"interface":"back","average":{"1min":1.101,"5min":1.0469999999999999,"15min":1.093},"min":{"1min":0.64100000000000001,"5min":0.46000000000000002,"15min":0.44600000000000001},"max":{"1min":1.47,"5min":1.867,"15min":4.9800000000000004},"last":1.163},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.1359999999999999,"15min":1.1619999999999999},"min":{"1min":0.752,"5min":0.56699999999999995,"15min":0.54000000000000004},"max":{"1min":1.4450000000000001,"5min":1.992,"15min":5.016},"last":1.319}]},{"osd":4,"last update":"Thu Oct 26 01:36:43 2023","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.387,"15min":1.3009999999999999},"min":{"1min":0.82799999999999996,"5min":0.66800000000000004,"15min":0.61499999999999999},"max":{"1min":1.681,"5min":19.329000000000001,"15min":19.329000000000001},"last":1.024},{"interface":"front","average":{"1min":1.319,"5min":1.27,"15min":1.306},"min":{"1min":0.82499999999999996,"5min":0.79100000000000004,"15min":0.67600000000000005},"max":{"1min":1.718,"5min":2.036,"15min":5.5229999999999997},"last":1.091}]},{"osd":5,"last update":"Thu Oct 26 01:36:43 2023","interfaces":[{"interface":"back","average":{"1min":1.5129999999999999,"5min":1.4039999999999999,"15min":1.4350000000000001},"min":{"1min":1.1359999999999999,"5min":0.80000000000000004,"15min":0.72499999999999998},"max":{"1min":1.857,"5min":2.2290000000000001,"15min":5.6360000000000001},"last":1.381},{"interface":"front","average":{"1min":1.399,"5min":1.492,"15min":1.4119999999999999},"min":{"1min":1.0880000000000001,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.8200000000000001,"5min":19.384,"15min":19.384},"last":1.129}]},{"osd":6,"last update":"Thu Oct 26 01:36:43 2023","interfaces":[{"interface":"back","average":{"1min":1.3580000000000001,"5min":1.375,"15min":1.3720000000000001},"min":{"1min":1.034,"5min":0.77200000000000002,"15min":0.73799999999999999},"max":{"1min":1.6970000000000001,"5min":8.2609999999999992,"15min":8.2609999999999992},"last":1.353},{"interface":"front","average":{"1min":1.377,"5min":1.4370000000000001,"15min":1.4099999999999999},"min":{"1min":0.85899999999999999,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.9450000000000001,"5min":11.849,"15min":11.849},"last":1.2829999999999999}]},{"osd":7,"last update":"Thu Oct 26 01:36:43 2023","interfaces":[{"interface":"back","average":{"1min":1.4970000000000001,"5min":1.4450000000000001,"15min":1.488},"min":{"1min":0.999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.7809999999999999,"5min":2.4060000000000001,"15min":5.6689999999999996},"last":1.4490000000000001},{"interface":"front","average":{"1min":1.4930000000000001,"5min":1.6080000000000001,"15min":1.5089999999999999},"min":{"1min":1.1579999999999999,"5min":0.81299999999999994,"15min":0.69399999999999995},"max":{"1min":1.9390000000000001,"5min":19.245999999999999,"15min":19.245999999999999},"last":1.417}]}]}],"pool_statfs":[]}} 2023-10-26T01:37:45.426 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-26T01:37:45.685 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:45.683+0000 7f62dbda9640 1 -- 172.21.15.169:0/2949248503 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62d406c420 msgr2=0x7f62d40ab050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:45.686 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:45.683+0000 7f62dbda9640 1 --2- 172.21.15.169:0/2949248503 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62d406c420 0x7f62d40ab050 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f62d406cd20 tx=0x7f62cc02c250 comp rx=0 tx=0).stop 2023-10-26T01:37:45.686 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:45.683+0000 7f62dbda9640 1 -- 172.21.15.169:0/2949248503 shutdown_connections 2023-10-26T01:37:45.686 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:45.683+0000 7f62dbda9640 1 --2- 172.21.15.169:0/2949248503 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62d406c420 0x7f62d40ab050 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:45.686 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:45.683+0000 7f62dbda9640 1 --2- 172.21.15.169:0/2949248503 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f62d406bf70 0x7f62d40aab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:45.687 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:45.683+0000 7f62dbda9640 1 --2- 172.21.15.169:0/2949248503 >> v2:172.21.15.170:3300/0 conn(0x7f62d414ece0 0x7f62d40aa1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:45.687 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:45.683+0000 7f62dbda9640 1 -- 172.21.15.169:0/2949248503 >> 172.21.15.169:0/2949248503 conn(0x7f62d4069930 msgr2=0x7f62d4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:45.687 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:45.683+0000 7f62dbda9640 1 -- 172.21.15.169:0/2949248503 shutdown_connections 2023-10-26T01:37:45.687 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:45.684+0000 7f62dbda9640 1 -- 172.21.15.169:0/2949248503 wait complete. 2023-10-26T01:37:45.689 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:45.685+0000 7f62dbda9640 1 Processor -- start 2023-10-26T01:37:45.689 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:45.685+0000 7f62dbda9640 1 -- start start 2023-10-26T01:37:45.690 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:45.686+0000 7f62dbda9640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f62d406bf70 0x7f62d4144d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:45.690 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:45.686+0000 7f62dbda9640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62d406c420 0x7f62d4096100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:45.690 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:45.686+0000 7f62dbda9640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f62d414ece0 0x7f62d4096640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:45.690 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:45.686+0000 7f62dbda9640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d4144940 con 0x7f62d406c420 2023-10-26T01:37:45.690 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:45.686+0000 7f62dbda9640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d40b2470 con 0x7f62d414ece0 2023-10-26T01:37:45.691 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:45.686+0000 7f62dbda9640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62d4143fa0 con 0x7f62d406bf70 2023-10-26T01:37:45.691 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:45.687+0000 7f62d9344640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62d406c420 0x7f62d4096100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:45.691 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:45.687+0000 7f62d9344640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62d406c420 0x7f62d4096100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:35658/0 (socket says 172.21.15.169:35658) 2023-10-26T01:37:45.691 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:45.687+0000 7f62d9344640 1 -- 172.21.15.169:0/3836439161 learned_addr learned my addr 172.21.15.169:0/3836439161 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:37:45.691 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:45.687+0000 7f62d9b45640 1 --2- 172.21.15.169:0/3836439161 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f62d406bf70 0x7f62d4144d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:45.692 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:45.687+0000 7f62da346640 1 --2- 172.21.15.169:0/3836439161 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f62d414ece0 0x7f62d4096640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:45.692 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:45.687+0000 7f62d9344640 1 -- 172.21.15.169:0/3836439161 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f62d406bf70 msgr2=0x7f62d4144d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:45.692 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:45.687+0000 7f62d9344640 1 --2- 172.21.15.169:0/3836439161 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f62d406bf70 0x7f62d4144d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:45.692 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:45.687+0000 7f62d9344640 1 -- 172.21.15.169:0/3836439161 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f62d414ece0 msgr2=0x7f62d4096640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:45.692 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:45.687+0000 7f62d9344640 1 --2- 172.21.15.169:0/3836439161 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f62d414ece0 0x7f62d4096640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:45.693 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:45.687+0000 7f62d9344640 1 -- 172.21.15.169:0/3836439161 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62cc009900 con 0x7f62d406c420 2023-10-26T01:37:45.693 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:45.687+0000 7f62da346640 1 --2- 172.21.15.169:0/3836439161 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f62d414ece0 0x7f62d4096640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:37:45.693 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:45.687+0000 7f62d9344640 1 --2- 172.21.15.169:0/3836439161 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62d406c420 0x7f62d4096100 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f62c400d750 tx=0x7f62c400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:45.693 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:45.687+0000 7f62ca7fc640 1 -- 172.21.15.169:0/3836439161 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c4009bd0 con 0x7f62d406c420 2023-10-26T01:37:45.693 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:45.688+0000 7f62ca7fc640 1 -- 172.21.15.169:0/3836439161 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f62c4009e20 con 0x7f62d406c420 2023-10-26T01:37:45.693 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:45.688+0000 7f62ca7fc640 1 -- 172.21.15.169:0/3836439161 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c4005280 con 0x7f62d406c420 2023-10-26T01:37:45.694 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:45.688+0000 7f62dbda9640 1 -- 172.21.15.169:0/3836439161 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62d4144280 con 0x7f62d406c420 2023-10-26T01:37:45.694 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:45.688+0000 7f62dbda9640 1 -- 172.21.15.169:0/3836439161 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62d414e400 con 0x7f62d406c420 2023-10-26T01:37:45.695 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:45.689+0000 7f62dbda9640 1 -- 172.21.15.169:0/3836439161 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62d40cd670 con 0x7f62d406c420 2023-10-26T01:37:45.695 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:45.690+0000 7f62ca7fc640 1 -- 172.21.15.169:0/3836439161 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f62c40053e0 con 0x7f62d406c420 2023-10-26T01:37:45.695 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:45.690+0000 7f62ca7fc640 1 --2- 172.21.15.169:0/3836439161 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f62b806ae70 0x7f62b806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:45.695 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:45.691+0000 7f62ca7fc640 1 -- 172.21.15.169:0/3836439161 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f62c4089720 con 0x7f62d406c420 2023-10-26T01:37:45.696 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:45.691+0000 7f62d9b45640 1 --2- 172.21.15.169:0/3836439161 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f62b806ae70 0x7f62b806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:45.696 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:45.692+0000 7f62d9b45640 1 --2- 172.21.15.169:0/3836439161 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f62b806ae70 0x7f62b806d330 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f62d0005fd0 tx=0x7f62d0005850 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:45.696 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:45.695+0000 7f62ca7fc640 1 -- 172.21.15.169:0/3836439161 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f62c4054860 con 0x7f62d406c420 2023-10-26T01:37:45.881 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:45.879+0000 7f62dbda9640 1 -- 172.21.15.169:0/3836439161 --> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f62d404db10 con 0x7f62b806ae70 2023-10-26T01:37:45.903 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:45.902+0000 7f62ca7fc640 1 -- 172.21.15.169:0/3836439161 <== mgr.4112 v2:172.21.15.170:6832/1187156918 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+282995 (secure 0 0 0) 0x7f62d404db10 con 0x7f62b806ae70 2023-10-26T01:37:45.907 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:37:45.909 INFO:teuthology.orchestra.run.smithi169.stderr:dumped all 2023-10-26T01:37:45.911 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:45.906+0000 7f62dbda9640 1 -- 172.21.15.169:0/3836439161 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f62b806ae70 msgr2=0x7f62b806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:45.911 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:45.906+0000 7f62dbda9640 1 --2- 172.21.15.169:0/3836439161 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f62b806ae70 0x7f62b806d330 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f62d0005fd0 tx=0x7f62d0005850 comp rx=0 tx=0).stop 2023-10-26T01:37:45.911 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:45.907+0000 7f62dbda9640 1 -- 172.21.15.169:0/3836439161 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62d406c420 msgr2=0x7f62d4096100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:45.912 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:45.907+0000 7f62dbda9640 1 --2- 172.21.15.169:0/3836439161 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62d406c420 0x7f62d4096100 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f62c400d750 tx=0x7f62c400dc10 comp rx=0 tx=0).stop 2023-10-26T01:37:45.912 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:45.907+0000 7f62dbda9640 1 -- 172.21.15.169:0/3836439161 shutdown_connections 2023-10-26T01:37:45.912 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:45.907+0000 7f62dbda9640 1 --2- 172.21.15.169:0/3836439161 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f62b806ae70 0x7f62b806d330 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:45.912 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:45.907+0000 7f62dbda9640 1 --2- 172.21.15.169:0/3836439161 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62d406c420 0x7f62d4096100 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:45.912 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:45.907+0000 7f62dbda9640 1 --2- 172.21.15.169:0/3836439161 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f62d414ece0 0x7f62d4096640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:45.913 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:45.907+0000 7f62dbda9640 1 --2- 172.21.15.169:0/3836439161 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f62d406bf70 0x7f62d4144d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:45.913 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:45.907+0000 7f62dbda9640 1 -- 172.21.15.169:0/3836439161 >> 172.21.15.169:0/3836439161 conn(0x7f62d4069930 msgr2=0x7f62d4156700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:45.913 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:45.907+0000 7f62dbda9640 1 -- 172.21.15.169:0/3836439161 shutdown_connections 2023-10-26T01:37:45.913 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:45.907+0000 7f62dbda9640 1 -- 172.21.15.169:0/3836439161 wait complete. 2023-10-26T01:37:45.928 INFO:teuthology.orchestra.run.smithi169.stdout:{"pg_ready":true,"pg_map":{"version":330,"stamp":"2023-10-26T01:37:44.280678+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":47,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":6237,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1556262,"statfs":{"total":1600000000,"available":1593612613,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.731490"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.611425+0000","last_change":"2023-10-26T01:37:40.610968+0000","last_active":"2023-10-26T01:37:40.611425+0000","last_peered":"2023-10-26T01:37:40.611425+0000","last_clean":"2023-10-26T01:37:40.611425+0000","last_became_active":"2023-10-26T01:37:40.609289+0000","last_became_peered":"2023-10-26T01:37:40.609289+0000","last_unstale":"2023-10-26T01:37:40.611425+0000","last_undegraded":"2023-10-26T01:37:40.611425+0000","last_fullsized":"2023-10-26T01:37:40.611425+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:04:23.812581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.612864+0000","last_change":"2023-10-26T01:37:40.612608+0000","last_active":"2023-10-26T01:37:40.612864+0000","last_peered":"2023-10-26T01:37:40.612864+0000","last_clean":"2023-10-26T01:37:40.612864+0000","last_became_active":"2023-10-26T01:37:40.610953+0000","last_became_peered":"2023-10-26T01:37:40.610953+0000","last_unstale":"2023-10-26T01:37:40.612864+0000","last_undegraded":"2023-10-26T01:37:40.612864+0000","last_fullsized":"2023-10-26T01:37:40.612864+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:23:53.795891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.614358+0000","last_change":"2023-10-26T01:37:40.612245+0000","last_active":"2023-10-26T01:37:40.614358+0000","last_peered":"2023-10-26T01:37:40.614358+0000","last_clean":"2023-10-26T01:37:40.614358+0000","last_became_active":"2023-10-26T01:37:40.612000+0000","last_became_peered":"2023-10-26T01:37:40.612000+0000","last_unstale":"2023-10-26T01:37:40.614358+0000","last_undegraded":"2023-10-26T01:37:40.614358+0000","last_fullsized":"2023-10-26T01:37:40.614358+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:13:26.281509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":11,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-10-26T01:37:41.587285+0000","last_change":"2023-10-26T01:37:40.602746+0000","last_active":"2023-10-26T01:37:41.587285+0000","last_peered":"2023-10-26T01:37:41.587285+0000","last_clean":"2023-10-26T01:37:41.587285+0000","last_became_active":"2023-10-26T01:37:40.602448+0000","last_became_peered":"2023-10-26T01:37:40.602448+0000","last_unstale":"2023-10-26T01:37:41.587285+0000","last_undegraded":"2023-10-26T01:37:41.587285+0000","last_fullsized":"2023-10-26T01:37:41.587285+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:41:33.659471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.608803+0000","last_change":"2023-10-26T01:37:40.605540+0000","last_active":"2023-10-26T01:37:40.608803+0000","last_peered":"2023-10-26T01:37:40.608803+0000","last_clean":"2023-10-26T01:37:40.608803+0000","last_became_active":"2023-10-26T01:37:40.604069+0000","last_became_peered":"2023-10-26T01:37:40.604069+0000","last_unstale":"2023-10-26T01:37:40.608803+0000","last_undegraded":"2023-10-26T01:37:40.608803+0000","last_fullsized":"2023-10-26T01:37:40.608803+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:00:46.370884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.617100+0000","last_change":"2023-10-26T01:37:40.616264+0000","last_active":"2023-10-26T01:37:40.617100+0000","last_peered":"2023-10-26T01:37:40.617100+0000","last_clean":"2023-10-26T01:37:40.617100+0000","last_became_active":"2023-10-26T01:37:40.615542+0000","last_became_peered":"2023-10-26T01:37:40.615542+0000","last_unstale":"2023-10-26T01:37:40.617100+0000","last_undegraded":"2023-10-26T01:37:40.617100+0000","last_fullsized":"2023-10-26T01:37:40.617100+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:23:53.795891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.616030+0000","last_change":"2023-10-26T01:37:40.611830+0000","last_active":"2023-10-26T01:37:40.616030+0000","last_peered":"2023-10-26T01:37:40.616030+0000","last_clean":"2023-10-26T01:37:40.616030+0000","last_became_active":"2023-10-26T01:37:40.611084+0000","last_became_peered":"2023-10-26T01:37:40.611084+0000","last_unstale":"2023-10-26T01:37:40.616030+0000","last_undegraded":"2023-10-26T01:37:40.616030+0000","last_fullsized":"2023-10-26T01:37:40.616030+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:58:41.530615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.603979+0000","last_change":"2023-10-26T01:37:40.603565+0000","last_active":"2023-10-26T01:37:40.603979+0000","last_peered":"2023-10-26T01:37:40.603979+0000","last_clean":"2023-10-26T01:37:40.603979+0000","last_became_active":"2023-10-26T01:37:40.603264+0000","last_became_peered":"2023-10-26T01:37:40.603264+0000","last_unstale":"2023-10-26T01:37:40.603979+0000","last_undegraded":"2023-10-26T01:37:40.603979+0000","last_fullsized":"2023-10-26T01:37:40.603979+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:58:41.530615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.609396+0000","last_change":"2023-10-26T01:37:40.605549+0000","last_active":"2023-10-26T01:37:40.609396+0000","last_peered":"2023-10-26T01:37:40.609396+0000","last_clean":"2023-10-26T01:37:40.609396+0000","last_became_active":"2023-10-26T01:37:40.604888+0000","last_became_peered":"2023-10-26T01:37:40.604888+0000","last_unstale":"2023-10-26T01:37:40.609396+0000","last_undegraded":"2023-10-26T01:37:40.609396+0000","last_fullsized":"2023-10-26T01:37:40.609396+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:52:22.871839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.608820+0000","last_change":"2023-10-26T01:37:40.605714+0000","last_active":"2023-10-26T01:37:40.608820+0000","last_peered":"2023-10-26T01:37:40.608820+0000","last_clean":"2023-10-26T01:37:40.608820+0000","last_became_active":"2023-10-26T01:37:40.604567+0000","last_became_peered":"2023-10-26T01:37:40.604567+0000","last_unstale":"2023-10-26T01:37:40.608820+0000","last_undegraded":"2023-10-26T01:37:40.608820+0000","last_fullsized":"2023-10-26T01:37:40.608820+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:52:22.871839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.617035+0000","last_change":"2023-10-26T01:37:40.615942+0000","last_active":"2023-10-26T01:37:40.617035+0000","last_peered":"2023-10-26T01:37:40.617035+0000","last_clean":"2023-10-26T01:37:40.617035+0000","last_became_active":"2023-10-26T01:37:40.614830+0000","last_became_peered":"2023-10-26T01:37:40.614830+0000","last_unstale":"2023-10-26T01:37:40.617035+0000","last_undegraded":"2023-10-26T01:37:40.617035+0000","last_fullsized":"2023-10-26T01:37:40.617035+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:33:21.466763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.610276+0000","last_change":"2023-10-26T01:37:40.606507+0000","last_active":"2023-10-26T01:37:40.610276+0000","last_peered":"2023-10-26T01:37:40.610276+0000","last_clean":"2023-10-26T01:37:40.610276+0000","last_became_active":"2023-10-26T01:37:40.606174+0000","last_became_peered":"2023-10-26T01:37:40.606174+0000","last_unstale":"2023-10-26T01:37:40.610276+0000","last_undegraded":"2023-10-26T01:37:40.610276+0000","last_fullsized":"2023-10-26T01:37:40.610276+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:57:42.748953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":11,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-10-26T01:37:41.587505+0000","last_change":"2023-10-26T01:37:40.608213+0000","last_active":"2023-10-26T01:37:41.587505+0000","last_peered":"2023-10-26T01:37:41.587505+0000","last_clean":"2023-10-26T01:37:41.587505+0000","last_became_active":"2023-10-26T01:37:40.607952+0000","last_became_peered":"2023-10-26T01:37:40.607952+0000","last_unstale":"2023-10-26T01:37:41.587505+0000","last_undegraded":"2023-10-26T01:37:41.587505+0000","last_fullsized":"2023-10-26T01:37:41.587505+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:32:02.668533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.613766+0000","last_change":"2023-10-26T01:37:40.613691+0000","last_active":"2023-10-26T01:37:40.613766+0000","last_peered":"2023-10-26T01:37:40.613766+0000","last_clean":"2023-10-26T01:37:40.613766+0000","last_became_active":"2023-10-26T01:37:40.613462+0000","last_became_peered":"2023-10-26T01:37:40.613462+0000","last_unstale":"2023-10-26T01:37:40.613766+0000","last_undegraded":"2023-10-26T01:37:40.613766+0000","last_fullsized":"2023-10-26T01:37:40.613766+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:30:38.850351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":11,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-10-26T01:37:41.588298+0000","last_change":"2023-10-26T01:37:40.600401+0000","last_active":"2023-10-26T01:37:41.588298+0000","last_peered":"2023-10-26T01:37:41.588298+0000","last_clean":"2023-10-26T01:37:41.588298+0000","last_became_active":"2023-10-26T01:37:40.599717+0000","last_became_peered":"2023-10-26T01:37:40.599717+0000","last_unstale":"2023-10-26T01:37:41.588298+0000","last_undegraded":"2023-10-26T01:37:41.588298+0000","last_fullsized":"2023-10-26T01:37:41.588298+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:20:43.946448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.616982+0000","last_change":"2023-10-26T01:37:40.615795+0000","last_active":"2023-10-26T01:37:40.616982+0000","last_peered":"2023-10-26T01:37:40.616982+0000","last_clean":"2023-10-26T01:37:40.616982+0000","last_became_active":"2023-10-26T01:37:40.609556+0000","last_became_peered":"2023-10-26T01:37:40.609556+0000","last_unstale":"2023-10-26T01:37:40.616982+0000","last_undegraded":"2023-10-26T01:37:40.616982+0000","last_fullsized":"2023-10-26T01:37:40.616982+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:43:19.958268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.593892+0000","last_change":"2023-10-26T01:37:40.593208+0000","last_active":"2023-10-26T01:37:40.593892+0000","last_peered":"2023-10-26T01:37:40.593892+0000","last_clean":"2023-10-26T01:37:40.593892+0000","last_became_active":"2023-10-26T01:37:40.592943+0000","last_became_peered":"2023-10-26T01:37:40.592943+0000","last_unstale":"2023-10-26T01:37:40.593892+0000","last_undegraded":"2023-10-26T01:37:40.593892+0000","last_fullsized":"2023-10-26T01:37:40.593892+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:34:05.902643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":11,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-10-26T01:37:41.587678+0000","last_change":"2023-10-26T01:37:40.601965+0000","last_active":"2023-10-26T01:37:41.587678+0000","last_peered":"2023-10-26T01:37:41.587678+0000","last_clean":"2023-10-26T01:37:41.587678+0000","last_became_active":"2023-10-26T01:37:40.601736+0000","last_became_peered":"2023-10-26T01:37:40.601736+0000","last_unstale":"2023-10-26T01:37:41.587678+0000","last_undegraded":"2023-10-26T01:37:41.587678+0000","last_fullsized":"2023-10-26T01:37:41.587678+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:57:42.748953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.605734+0000","last_change":"2023-10-26T01:37:40.605655+0000","last_active":"2023-10-26T01:37:40.605734+0000","last_peered":"2023-10-26T01:37:40.605734+0000","last_clean":"2023-10-26T01:37:40.605734+0000","last_became_active":"2023-10-26T01:37:40.605413+0000","last_became_peered":"2023-10-26T01:37:40.605413+0000","last_unstale":"2023-10-26T01:37:40.605734+0000","last_undegraded":"2023-10-26T01:37:40.605734+0000","last_fullsized":"2023-10-26T01:37:40.605734+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:05:57.749521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.610250+0000","last_change":"2023-10-26T01:37:40.605731+0000","last_active":"2023-10-26T01:37:40.610250+0000","last_peered":"2023-10-26T01:37:40.610250+0000","last_clean":"2023-10-26T01:37:40.610250+0000","last_became_active":"2023-10-26T01:37:40.605123+0000","last_became_peered":"2023-10-26T01:37:40.605123+0000","last_unstale":"2023-10-26T01:37:40.610250+0000","last_undegraded":"2023-10-26T01:37:40.610250+0000","last_fullsized":"2023-10-26T01:37:40.610250+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:58:41.530615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.609492+0000","last_change":"2023-10-26T01:37:40.609424+0000","last_active":"2023-10-26T01:37:40.609492+0000","last_peered":"2023-10-26T01:37:40.609492+0000","last_clean":"2023-10-26T01:37:40.609492+0000","last_became_active":"2023-10-26T01:37:40.609185+0000","last_became_peered":"2023-10-26T01:37:40.609185+0000","last_unstale":"2023-10-26T01:37:40.609492+0000","last_undegraded":"2023-10-26T01:37:40.609492+0000","last_fullsized":"2023-10-26T01:37:40.609492+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:28:48.738602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.612904+0000","last_change":"2023-10-26T01:37:40.612801+0000","last_active":"2023-10-26T01:37:40.612904+0000","last_peered":"2023-10-26T01:37:40.612904+0000","last_clean":"2023-10-26T01:37:40.612904+0000","last_became_active":"2023-10-26T01:37:40.612320+0000","last_became_peered":"2023-10-26T01:37:40.612320+0000","last_unstale":"2023-10-26T01:37:40.612904+0000","last_undegraded":"2023-10-26T01:37:40.612904+0000","last_fullsized":"2023-10-26T01:37:40.612904+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:41:54.411711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.607141+0000","last_change":"2023-10-26T01:37:40.607032+0000","last_active":"2023-10-26T01:37:40.607141+0000","last_peered":"2023-10-26T01:37:40.607141+0000","last_clean":"2023-10-26T01:37:40.607141+0000","last_became_active":"2023-10-26T01:37:40.606807+0000","last_became_peered":"2023-10-26T01:37:40.606807+0000","last_unstale":"2023-10-26T01:37:40.607141+0000","last_undegraded":"2023-10-26T01:37:40.607141+0000","last_fullsized":"2023-10-26T01:37:40.607141+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:00:46.370884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.607126+0000","last_change":"2023-10-26T01:37:40.605029+0000","last_active":"2023-10-26T01:37:40.607126+0000","last_peered":"2023-10-26T01:37:40.607126+0000","last_clean":"2023-10-26T01:37:40.607126+0000","last_became_active":"2023-10-26T01:37:40.604635+0000","last_became_peered":"2023-10-26T01:37:40.604635+0000","last_unstale":"2023-10-26T01:37:40.607126+0000","last_undegraded":"2023-10-26T01:37:40.607126+0000","last_fullsized":"2023-10-26T01:37:40.607126+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:57:42.748953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.608783+0000","last_change":"2023-10-26T01:37:40.605294+0000","last_active":"2023-10-26T01:37:40.608783+0000","last_peered":"2023-10-26T01:37:40.608783+0000","last_clean":"2023-10-26T01:37:40.608783+0000","last_became_active":"2023-10-26T01:37:40.603593+0000","last_became_peered":"2023-10-26T01:37:40.603593+0000","last_unstale":"2023-10-26T01:37:40.608783+0000","last_undegraded":"2023-10-26T01:37:40.608783+0000","last_fullsized":"2023-10-26T01:37:40.608783+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:23:53.795891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.601126+0000","last_change":"2023-10-26T01:37:40.601052+0000","last_active":"2023-10-26T01:37:40.601126+0000","last_peered":"2023-10-26T01:37:40.601126+0000","last_clean":"2023-10-26T01:37:40.601126+0000","last_became_active":"2023-10-26T01:37:40.600790+0000","last_became_peered":"2023-10-26T01:37:40.600790+0000","last_unstale":"2023-10-26T01:37:40.601126+0000","last_undegraded":"2023-10-26T01:37:40.601126+0000","last_fullsized":"2023-10-26T01:37:40.601126+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:58:42.116497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.604364+0000","last_change":"2023-10-26T01:37:40.604186+0000","last_active":"2023-10-26T01:37:40.604364+0000","last_peered":"2023-10-26T01:37:40.604364+0000","last_clean":"2023-10-26T01:37:40.604364+0000","last_became_active":"2023-10-26T01:37:40.603880+0000","last_became_peered":"2023-10-26T01:37:40.603880+0000","last_unstale":"2023-10-26T01:37:40.604364+0000","last_undegraded":"2023-10-26T01:37:40.604364+0000","last_fullsized":"2023-10-26T01:37:40.604364+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:00:46.370884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.603086+0000","last_change":"2023-10-26T01:37:40.599308+0000","last_active":"2023-10-26T01:37:40.603086+0000","last_peered":"2023-10-26T01:37:40.603086+0000","last_clean":"2023-10-26T01:37:40.603086+0000","last_became_active":"2023-10-26T01:37:40.598237+0000","last_became_peered":"2023-10-26T01:37:40.598237+0000","last_unstale":"2023-10-26T01:37:40.603086+0000","last_undegraded":"2023-10-26T01:37:40.603086+0000","last_fullsized":"2023-10-26T01:37:40.603086+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:42:26.945329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.598565+0000","last_change":"2023-10-26T01:37:40.598489+0000","last_active":"2023-10-26T01:37:40.598565+0000","last_peered":"2023-10-26T01:37:40.598565+0000","last_clean":"2023-10-26T01:37:40.598565+0000","last_became_active":"2023-10-26T01:37:40.598268+0000","last_became_peered":"2023-10-26T01:37:40.598268+0000","last_unstale":"2023-10-26T01:37:40.598565+0000","last_undegraded":"2023-10-26T01:37:40.598565+0000","last_fullsized":"2023-10-26T01:37:40.598565+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:42:26.945329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.608136+0000","last_change":"2023-10-26T01:37:40.608080+0000","last_active":"2023-10-26T01:37:40.608136+0000","last_peered":"2023-10-26T01:37:40.608136+0000","last_clean":"2023-10-26T01:37:40.608136+0000","last_became_active":"2023-10-26T01:37:40.607354+0000","last_became_peered":"2023-10-26T01:37:40.607354+0000","last_unstale":"2023-10-26T01:37:40.608136+0000","last_undegraded":"2023-10-26T01:37:40.608136+0000","last_fullsized":"2023-10-26T01:37:40.608136+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:48:29.780711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":11,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-10-26T01:37:41.587364+0000","last_change":"2023-10-26T01:37:40.609980+0000","last_active":"2023-10-26T01:37:41.587364+0000","last_peered":"2023-10-26T01:37:41.587364+0000","last_clean":"2023-10-26T01:37:41.587364+0000","last_became_active":"2023-10-26T01:37:40.609759+0000","last_became_peered":"2023-10-26T01:37:40.609759+0000","last_unstale":"2023-10-26T01:37:41.587364+0000","last_undegraded":"2023-10-26T01:37:41.587364+0000","last_fullsized":"2023-10-26T01:37:41.587364+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:23:28.232518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.606019+0000","last_change":"2023-10-26T01:37:40.605894+0000","last_active":"2023-10-26T01:37:40.606019+0000","last_peered":"2023-10-26T01:37:40.606019+0000","last_clean":"2023-10-26T01:37:40.606019+0000","last_became_active":"2023-10-26T01:37:40.603038+0000","last_became_peered":"2023-10-26T01:37:40.603038+0000","last_unstale":"2023-10-26T01:37:40.606019+0000","last_undegraded":"2023-10-26T01:37:40.606019+0000","last_fullsized":"2023-10-26T01:37:40.606019+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:24:56.801054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.611087+0000","last_change":"2023-10-26T01:37:40.610095+0000","last_active":"2023-10-26T01:37:40.611087+0000","last_peered":"2023-10-26T01:37:40.611087+0000","last_clean":"2023-10-26T01:37:40.611087+0000","last_became_active":"2023-10-26T01:37:40.609457+0000","last_became_peered":"2023-10-26T01:37:40.609457+0000","last_unstale":"2023-10-26T01:37:40.611087+0000","last_undegraded":"2023-10-26T01:37:40.611087+0000","last_fullsized":"2023-10-26T01:37:40.611087+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:44:57.760928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"119'4","reported_seq":15,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.614071+0000","last_change":"2023-10-26T01:37:39.577351+0000","last_active":"2023-10-26T01:37:40.614071+0000","last_peered":"2023-10-26T01:37:40.614071+0000","last_clean":"2023-10-26T01:37:40.614071+0000","last_became_active":"2023-10-26T01:37:39.577122+0000","last_became_peered":"2023-10-26T01:37:39.577122+0000","last_unstale":"2023-10-26T01:37:40.614071+0000","last_undegraded":"2023-10-26T01:37:40.614071+0000","last_fullsized":"2023-10-26T01:37:40.614071+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_clean_scrub_stamp":"2023-10-26T01:37:38.536854+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:43:18.947747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":11,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-10-26T01:37:41.587820+0000","last_change":"2023-10-26T01:37:40.613479+0000","last_active":"2023-10-26T01:37:41.587820+0000","last_peered":"2023-10-26T01:37:41.587820+0000","last_clean":"2023-10-26T01:37:41.587820+0000","last_became_active":"2023-10-26T01:37:40.613301+0000","last_became_peered":"2023-10-26T01:37:40.613301+0000","last_unstale":"2023-10-26T01:37:41.587820+0000","last_undegraded":"2023-10-26T01:37:41.587820+0000","last_fullsized":"2023-10-26T01:37:41.587820+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:48:38.290864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"119'1","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.619307+0000","last_change":"2023-10-26T01:37:39.580337+0000","last_active":"2023-10-26T01:37:40.619307+0000","last_peered":"2023-10-26T01:37:40.619307+0000","last_clean":"2023-10-26T01:37:40.619307+0000","last_became_active":"2023-10-26T01:37:39.579565+0000","last_became_peered":"2023-10-26T01:37:39.579565+0000","last_unstale":"2023-10-26T01:37:40.619307+0000","last_undegraded":"2023-10-26T01:37:40.619307+0000","last_fullsized":"2023-10-26T01:37:40.619307+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_clean_scrub_stamp":"2023-10-26T01:37:38.536854+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:48:37.280343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.605029+0000","last_change":"2023-10-26T01:37:40.604955+0000","last_active":"2023-10-26T01:37:40.605029+0000","last_peered":"2023-10-26T01:37:40.605029+0000","last_clean":"2023-10-26T01:37:40.605029+0000","last_became_active":"2023-10-26T01:37:40.603088+0000","last_became_peered":"2023-10-26T01:37:40.603088+0000","last_unstale":"2023-10-26T01:37:40.605029+0000","last_undegraded":"2023-10-26T01:37:40.605029+0000","last_fullsized":"2023-10-26T01:37:40.605029+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:50:15.659835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"119'2","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.612069+0000","last_change":"2023-10-26T01:37:39.566835+0000","last_active":"2023-10-26T01:37:40.612069+0000","last_peered":"2023-10-26T01:37:40.612069+0000","last_clean":"2023-10-26T01:37:40.612069+0000","last_became_active":"2023-10-26T01:37:39.566594+0000","last_became_peered":"2023-10-26T01:37:39.566594+0000","last_unstale":"2023-10-26T01:37:40.612069+0000","last_undegraded":"2023-10-26T01:37:40.612069+0000","last_fullsized":"2023-10-26T01:37:40.612069+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_clean_scrub_stamp":"2023-10-26T01:37:38.536854+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:45:54.356734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.611721+0000","last_change":"2023-10-26T01:37:40.611322+0000","last_active":"2023-10-26T01:37:40.611721+0000","last_peered":"2023-10-26T01:37:40.611721+0000","last_clean":"2023-10-26T01:37:40.611721+0000","last_became_active":"2023-10-26T01:37:40.610605+0000","last_became_peered":"2023-10-26T01:37:40.610605+0000","last_unstale":"2023-10-26T01:37:40.611721+0000","last_undegraded":"2023-10-26T01:37:40.611721+0000","last_fullsized":"2023-10-26T01:37:40.611721+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:42:26.945329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"119'2","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.612476+0000","last_change":"2023-10-26T01:37:39.572304+0000","last_active":"2023-10-26T01:37:40.612476+0000","last_peered":"2023-10-26T01:37:40.612476+0000","last_clean":"2023-10-26T01:37:40.612476+0000","last_became_active":"2023-10-26T01:37:39.571814+0000","last_became_peered":"2023-10-26T01:37:39.571814+0000","last_unstale":"2023-10-26T01:37:40.612476+0000","last_undegraded":"2023-10-26T01:37:40.612476+0000","last_fullsized":"2023-10-26T01:37:40.612476+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_clean_scrub_stamp":"2023-10-26T01:37:38.536854+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:54:40.095332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.611145+0000","last_change":"2023-10-26T01:37:40.611058+0000","last_active":"2023-10-26T01:37:40.611145+0000","last_peered":"2023-10-26T01:37:40.611145+0000","last_clean":"2023-10-26T01:37:40.611145+0000","last_became_active":"2023-10-26T01:37:40.608611+0000","last_became_peered":"2023-10-26T01:37:40.608611+0000","last_unstale":"2023-10-26T01:37:40.611145+0000","last_undegraded":"2023-10-26T01:37:40.611145+0000","last_fullsized":"2023-10-26T01:37:40.611145+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:51:50.526372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"119'1","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.613593+0000","last_change":"2023-10-26T01:37:39.580640+0000","last_active":"2023-10-26T01:37:40.613593+0000","last_peered":"2023-10-26T01:37:40.613593+0000","last_clean":"2023-10-26T01:37:40.613593+0000","last_became_active":"2023-10-26T01:37:39.580479+0000","last_became_peered":"2023-10-26T01:37:39.580479+0000","last_unstale":"2023-10-26T01:37:40.613593+0000","last_undegraded":"2023-10-26T01:37:40.613593+0000","last_fullsized":"2023-10-26T01:37:40.613593+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_clean_scrub_stamp":"2023-10-26T01:37:38.536854+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:30:08.638588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.616890+0000","last_change":"2023-10-26T01:37:40.614007+0000","last_active":"2023-10-26T01:37:40.616890+0000","last_peered":"2023-10-26T01:37:40.616890+0000","last_clean":"2023-10-26T01:37:40.616890+0000","last_became_active":"2023-10-26T01:37:40.613733+0000","last_became_peered":"2023-10-26T01:37:40.613733+0000","last_unstale":"2023-10-26T01:37:40.616890+0000","last_undegraded":"2023-10-26T01:37:40.616890+0000","last_fullsized":"2023-10-26T01:37:40.616890+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:52:22.871839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"119'1","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.611892+0000","last_change":"2023-10-26T01:37:39.573277+0000","last_active":"2023-10-26T01:37:40.611892+0000","last_peered":"2023-10-26T01:37:40.611892+0000","last_clean":"2023-10-26T01:37:40.611892+0000","last_became_active":"2023-10-26T01:37:39.572898+0000","last_became_peered":"2023-10-26T01:37:39.572898+0000","last_unstale":"2023-10-26T01:37:40.611892+0000","last_undegraded":"2023-10-26T01:37:40.611892+0000","last_fullsized":"2023-10-26T01:37:40.611892+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_clean_scrub_stamp":"2023-10-26T01:37:38.536854+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:49:03.499843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":11,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-10-26T01:37:41.588733+0000","last_change":"2023-10-26T01:37:40.600540+0000","last_active":"2023-10-26T01:37:41.588733+0000","last_peered":"2023-10-26T01:37:41.588733+0000","last_clean":"2023-10-26T01:37:41.588733+0000","last_became_active":"2023-10-26T01:37:40.600026+0000","last_became_peered":"2023-10-26T01:37:40.600026+0000","last_unstale":"2023-10-26T01:37:41.588733+0000","last_undegraded":"2023-10-26T01:37:41.588733+0000","last_fullsized":"2023-10-26T01:37:41.588733+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:58:41.530615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":11,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.597428+0000","last_change":"2023-10-26T01:37:39.580881+0000","last_active":"2023-10-26T01:37:40.597428+0000","last_peered":"2023-10-26T01:37:40.597428+0000","last_clean":"2023-10-26T01:37:40.597428+0000","last_became_active":"2023-10-26T01:37:39.579140+0000","last_became_peered":"2023-10-26T01:37:39.579140+0000","last_unstale":"2023-10-26T01:37:40.597428+0000","last_undegraded":"2023-10-26T01:37:40.597428+0000","last_fullsized":"2023-10-26T01:37:40.597428+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_clean_scrub_stamp":"2023-10-26T01:37:38.536854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:41:33.183791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.610297+0000","last_change":"2023-10-26T01:37:40.609475+0000","last_active":"2023-10-26T01:37:40.610297+0000","last_peered":"2023-10-26T01:37:40.610297+0000","last_clean":"2023-10-26T01:37:40.610297+0000","last_became_active":"2023-10-26T01:37:40.609181+0000","last_became_peered":"2023-10-26T01:37:40.609181+0000","last_unstale":"2023-10-26T01:37:40.610297+0000","last_undegraded":"2023-10-26T01:37:40.610297+0000","last_fullsized":"2023-10-26T01:37:40.610297+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:16:38.400941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":11,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.590805+0000","last_change":"2023-10-26T01:37:39.573240+0000","last_active":"2023-10-26T01:37:40.590805+0000","last_peered":"2023-10-26T01:37:40.590805+0000","last_clean":"2023-10-26T01:37:40.590805+0000","last_became_active":"2023-10-26T01:37:39.571928+0000","last_became_peered":"2023-10-26T01:37:39.571928+0000","last_unstale":"2023-10-26T01:37:40.590805+0000","last_undegraded":"2023-10-26T01:37:40.590805+0000","last_fullsized":"2023-10-26T01:37:40.590805+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_clean_scrub_stamp":"2023-10-26T01:37:38.536854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:20:43.537466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.605504+0000","last_change":"2023-10-26T01:37:40.605386+0000","last_active":"2023-10-26T01:37:40.605504+0000","last_peered":"2023-10-26T01:37:40.605504+0000","last_clean":"2023-10-26T01:37:40.605504+0000","last_became_active":"2023-10-26T01:37:40.605027+0000","last_became_peered":"2023-10-26T01:37:40.605027+0000","last_unstale":"2023-10-26T01:37:40.605504+0000","last_undegraded":"2023-10-26T01:37:40.605504+0000","last_fullsized":"2023-10-26T01:37:40.605504+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:52:22.871839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":11,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.590266+0000","last_change":"2023-10-26T01:37:39.574600+0000","last_active":"2023-10-26T01:37:40.590266+0000","last_peered":"2023-10-26T01:37:40.590266+0000","last_clean":"2023-10-26T01:37:40.590266+0000","last_became_active":"2023-10-26T01:37:39.574159+0000","last_became_peered":"2023-10-26T01:37:39.574159+0000","last_unstale":"2023-10-26T01:37:40.590266+0000","last_undegraded":"2023-10-26T01:37:40.590266+0000","last_fullsized":"2023-10-26T01:37:40.590266+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_clean_scrub_stamp":"2023-10-26T01:37:38.536854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:18:01.338022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.612259+0000","last_change":"2023-10-26T01:37:40.611919+0000","last_active":"2023-10-26T01:37:40.612259+0000","last_peered":"2023-10-26T01:37:40.612259+0000","last_clean":"2023-10-26T01:37:40.612259+0000","last_became_active":"2023-10-26T01:37:40.611276+0000","last_became_peered":"2023-10-26T01:37:40.611276+0000","last_unstale":"2023-10-26T01:37:40.612259+0000","last_undegraded":"2023-10-26T01:37:40.612259+0000","last_fullsized":"2023-10-26T01:37:40.612259+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:43:15.435913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-10-26T01:37:41.587114+0000","last_change":"2023-10-26T01:37:39.579641+0000","last_active":"2023-10-26T01:37:41.587114+0000","last_peered":"2023-10-26T01:37:41.587114+0000","last_clean":"2023-10-26T01:37:41.587114+0000","last_became_active":"2023-10-26T01:37:39.579235+0000","last_became_peered":"2023-10-26T01:37:39.579235+0000","last_unstale":"2023-10-26T01:37:41.587114+0000","last_undegraded":"2023-10-26T01:37:41.587114+0000","last_fullsized":"2023-10-26T01:37:41.587114+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_clean_scrub_stamp":"2023-10-26T01:37:38.536854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:22:56.075780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.612141+0000","last_change":"2023-10-26T01:37:40.610343+0000","last_active":"2023-10-26T01:37:40.612141+0000","last_peered":"2023-10-26T01:37:40.612141+0000","last_clean":"2023-10-26T01:37:40.612141+0000","last_became_active":"2023-10-26T01:37:40.606987+0000","last_became_peered":"2023-10-26T01:37:40.606987+0000","last_unstale":"2023-10-26T01:37:40.612141+0000","last_undegraded":"2023-10-26T01:37:40.612141+0000","last_fullsized":"2023-10-26T01:37:40.612141+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:58:41.530615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"119'3","reported_seq":14,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.618824+0000","last_change":"2023-10-26T01:37:39.583614+0000","last_active":"2023-10-26T01:37:40.618824+0000","last_peered":"2023-10-26T01:37:40.618824+0000","last_clean":"2023-10-26T01:37:40.618824+0000","last_became_active":"2023-10-26T01:37:39.583194+0000","last_became_peered":"2023-10-26T01:37:39.583194+0000","last_unstale":"2023-10-26T01:37:40.618824+0000","last_undegraded":"2023-10-26T01:37:40.618824+0000","last_fullsized":"2023-10-26T01:37:40.618824+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_clean_scrub_stamp":"2023-10-26T01:37:38.536854+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:42:25.934808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.613462+0000","last_change":"2023-10-26T01:37:40.610086+0000","last_active":"2023-10-26T01:37:40.613462+0000","last_peered":"2023-10-26T01:37:40.613462+0000","last_clean":"2023-10-26T01:37:40.613462+0000","last_became_active":"2023-10-26T01:37:40.608497+0000","last_became_peered":"2023-10-26T01:37:40.608497+0000","last_unstale":"2023-10-26T01:37:40.613462+0000","last_undegraded":"2023-10-26T01:37:40.613462+0000","last_fullsized":"2023-10-26T01:37:40.613462+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:20:43.946448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"119'6","reported_seq":17,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.620626+0000","last_change":"2023-10-26T01:37:39.574828+0000","last_active":"2023-10-26T01:37:40.620626+0000","last_peered":"2023-10-26T01:37:40.620626+0000","last_clean":"2023-10-26T01:37:40.620626+0000","last_became_active":"2023-10-26T01:37:39.573696+0000","last_became_peered":"2023-10-26T01:37:39.573696+0000","last_unstale":"2023-10-26T01:37:40.620626+0000","last_undegraded":"2023-10-26T01:37:40.620626+0000","last_fullsized":"2023-10-26T01:37:40.620626+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_clean_scrub_stamp":"2023-10-26T01:37:38.536854+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:23:52.785370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":11,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-10-26T01:37:41.587562+0000","last_change":"2023-10-26T01:37:40.606092+0000","last_active":"2023-10-26T01:37:41.587562+0000","last_peered":"2023-10-26T01:37:41.587562+0000","last_clean":"2023-10-26T01:37:41.587562+0000","last_became_active":"2023-10-26T01:37:40.605832+0000","last_became_peered":"2023-10-26T01:37:40.605832+0000","last_unstale":"2023-10-26T01:37:41.587562+0000","last_undegraded":"2023-10-26T01:37:41.587562+0000","last_fullsized":"2023-10-26T01:37:41.587562+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:43:15.435913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"119'2","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.615045+0000","last_change":"2023-10-26T01:37:39.575193+0000","last_active":"2023-10-26T01:37:40.615045+0000","last_peered":"2023-10-26T01:37:40.615045+0000","last_clean":"2023-10-26T01:37:40.615045+0000","last_became_active":"2023-10-26T01:37:39.574952+0000","last_became_peered":"2023-10-26T01:37:39.574952+0000","last_unstale":"2023-10-26T01:37:40.615045+0000","last_undegraded":"2023-10-26T01:37:40.615045+0000","last_fullsized":"2023-10-26T01:37:40.615045+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_clean_scrub_stamp":"2023-10-26T01:37:38.536854+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:00:41.641143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.605008+0000","last_change":"2023-10-26T01:37:40.604736+0000","last_active":"2023-10-26T01:37:40.605008+0000","last_peered":"2023-10-26T01:37:40.605008+0000","last_clean":"2023-10-26T01:37:40.605008+0000","last_became_active":"2023-10-26T01:37:40.604505+0000","last_became_peered":"2023-10-26T01:37:40.604505+0000","last_unstale":"2023-10-26T01:37:40.605008+0000","last_undegraded":"2023-10-26T01:37:40.605008+0000","last_fullsized":"2023-10-26T01:37:40.605008+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:30:22.106958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":11,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.599412+0000","last_change":"2023-10-26T01:37:39.577118+0000","last_active":"2023-10-26T01:37:40.599412+0000","last_peered":"2023-10-26T01:37:40.599412+0000","last_clean":"2023-10-26T01:37:40.599412+0000","last_became_active":"2023-10-26T01:37:39.576391+0000","last_became_peered":"2023-10-26T01:37:39.576391+0000","last_unstale":"2023-10-26T01:37:40.599412+0000","last_undegraded":"2023-10-26T01:37:40.599412+0000","last_fullsized":"2023-10-26T01:37:40.599412+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_clean_scrub_stamp":"2023-10-26T01:37:38.536854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:45:14.780046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.610507+0000","last_change":"2023-10-26T01:37:40.607996+0000","last_active":"2023-10-26T01:37:40.610507+0000","last_peered":"2023-10-26T01:37:40.610507+0000","last_clean":"2023-10-26T01:37:40.610507+0000","last_became_active":"2023-10-26T01:37:40.607101+0000","last_became_peered":"2023-10-26T01:37:40.607101+0000","last_unstale":"2023-10-26T01:37:40.610507+0000","last_undegraded":"2023-10-26T01:37:40.610507+0000","last_fullsized":"2023-10-26T01:37:40.610507+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:32:02.668533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":11,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.589349+0000","last_change":"2023-10-26T01:37:39.579284+0000","last_active":"2023-10-26T01:37:40.589349+0000","last_peered":"2023-10-26T01:37:40.589349+0000","last_clean":"2023-10-26T01:37:40.589349+0000","last_became_active":"2023-10-26T01:37:39.578954+0000","last_became_peered":"2023-10-26T01:37:39.578954+0000","last_unstale":"2023-10-26T01:37:40.589349+0000","last_undegraded":"2023-10-26T01:37:40.589349+0000","last_fullsized":"2023-10-26T01:37:40.589349+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_clean_scrub_stamp":"2023-10-26T01:37:38.536854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:20:42.935927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.608820+0000","last_change":"2023-10-26T01:37:40.608399+0000","last_active":"2023-10-26T01:37:40.608820+0000","last_peered":"2023-10-26T01:37:40.608820+0000","last_clean":"2023-10-26T01:37:40.608820+0000","last_became_active":"2023-10-26T01:37:40.608120+0000","last_became_peered":"2023-10-26T01:37:40.608120+0000","last_unstale":"2023-10-26T01:37:40.608820+0000","last_undegraded":"2023-10-26T01:37:40.608820+0000","last_fullsized":"2023-10-26T01:37:40.608820+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:24:22.339565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"119'8","reported_seq":19,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.618782+0000","last_change":"2023-10-26T01:37:39.579895+0000","last_active":"2023-10-26T01:37:40.618782+0000","last_peered":"2023-10-26T01:37:40.618782+0000","last_clean":"2023-10-26T01:37:40.618782+0000","last_became_active":"2023-10-26T01:37:39.579641+0000","last_became_peered":"2023-10-26T01:37:39.579641+0000","last_unstale":"2023-10-26T01:37:40.618782+0000","last_undegraded":"2023-10-26T01:37:40.618782+0000","last_fullsized":"2023-10-26T01:37:40.618782+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_clean_scrub_stamp":"2023-10-26T01:37:38.536854+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:40:12.450735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.612185+0000","last_change":"2023-10-26T01:37:40.610736+0000","last_active":"2023-10-26T01:37:40.612185+0000","last_peered":"2023-10-26T01:37:40.612185+0000","last_clean":"2023-10-26T01:37:40.612185+0000","last_became_active":"2023-10-26T01:37:40.610095+0000","last_became_peered":"2023-10-26T01:37:40.610095+0000","last_unstale":"2023-10-26T01:37:40.612185+0000","last_undegraded":"2023-10-26T01:37:40.612185+0000","last_fullsized":"2023-10-26T01:37:40.612185+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:34:05.902643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"119'2","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.617254+0000","last_change":"2023-10-26T01:37:39.579610+0000","last_active":"2023-10-26T01:37:40.617254+0000","last_peered":"2023-10-26T01:37:40.617254+0000","last_clean":"2023-10-26T01:37:40.617254+0000","last_became_active":"2023-10-26T01:37:39.578843+0000","last_became_peered":"2023-10-26T01:37:39.578843+0000","last_unstale":"2023-10-26T01:37:40.617254+0000","last_undegraded":"2023-10-26T01:37:40.617254+0000","last_fullsized":"2023-10-26T01:37:40.617254+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_clean_scrub_stamp":"2023-10-26T01:37:38.536854+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:00:45.360362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.617114+0000","last_change":"2023-10-26T01:37:40.617026+0000","last_active":"2023-10-26T01:37:40.617114+0000","last_peered":"2023-10-26T01:37:40.617114+0000","last_clean":"2023-10-26T01:37:40.617114+0000","last_became_active":"2023-10-26T01:37:40.616771+0000","last_became_peered":"2023-10-26T01:37:40.616771+0000","last_unstale":"2023-10-26T01:37:40.617114+0000","last_undegraded":"2023-10-26T01:37:40.617114+0000","last_fullsized":"2023-10-26T01:37:40.617114+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:48:10.758339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"119'1","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.619167+0000","last_change":"2023-10-26T01:37:39.577452+0000","last_active":"2023-10-26T01:37:40.619167+0000","last_peered":"2023-10-26T01:37:40.619167+0000","last_clean":"2023-10-26T01:37:40.619167+0000","last_became_active":"2023-10-26T01:37:39.576922+0000","last_became_peered":"2023-10-26T01:37:39.576922+0000","last_unstale":"2023-10-26T01:37:40.619167+0000","last_undegraded":"2023-10-26T01:37:40.619167+0000","last_fullsized":"2023-10-26T01:37:40.619167+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_clean_scrub_stamp":"2023-10-26T01:37:38.536854+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:48:37.280343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.617083+0000","last_change":"2023-10-26T01:37:40.616124+0000","last_active":"2023-10-26T01:37:40.617083+0000","last_peered":"2023-10-26T01:37:40.617083+0000","last_clean":"2023-10-26T01:37:40.617083+0000","last_became_active":"2023-10-26T01:37:40.615180+0000","last_became_peered":"2023-10-26T01:37:40.615180+0000","last_unstale":"2023-10-26T01:37:40.617083+0000","last_undegraded":"2023-10-26T01:37:40.617083+0000","last_fullsized":"2023-10-26T01:37:40.617083+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:34:05.902643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":11,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.590989+0000","last_change":"2023-10-26T01:37:39.575894+0000","last_active":"2023-10-26T01:37:40.590989+0000","last_peered":"2023-10-26T01:37:40.590989+0000","last_clean":"2023-10-26T01:37:40.590989+0000","last_became_active":"2023-10-26T01:37:39.563576+0000","last_became_peered":"2023-10-26T01:37:39.563576+0000","last_unstale":"2023-10-26T01:37:40.590989+0000","last_undegraded":"2023-10-26T01:37:40.590989+0000","last_fullsized":"2023-10-26T01:37:40.590989+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_clean_scrub_stamp":"2023-10-26T01:37:38.536854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:26:50.722737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.611101+0000","last_change":"2023-10-26T01:37:40.610244+0000","last_active":"2023-10-26T01:37:40.611101+0000","last_peered":"2023-10-26T01:37:40.611101+0000","last_clean":"2023-10-26T01:37:40.611101+0000","last_became_active":"2023-10-26T01:37:40.609746+0000","last_became_peered":"2023-10-26T01:37:40.609746+0000","last_unstale":"2023-10-26T01:37:40.611101+0000","last_undegraded":"2023-10-26T01:37:40.611101+0000","last_fullsized":"2023-10-26T01:37:40.611101+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:29:26.369902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"119'2","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.612444+0000","last_change":"2023-10-26T01:37:39.561950+0000","last_active":"2023-10-26T01:37:40.612444+0000","last_peered":"2023-10-26T01:37:40.612444+0000","last_clean":"2023-10-26T01:37:40.612444+0000","last_became_active":"2023-10-26T01:37:39.561616+0000","last_became_peered":"2023-10-26T01:37:39.561616+0000","last_unstale":"2023-10-26T01:37:40.612444+0000","last_undegraded":"2023-10-26T01:37:40.612444+0000","last_fullsized":"2023-10-26T01:37:40.612444+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_clean_scrub_stamp":"2023-10-26T01:37:38.536854+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:41:33.183791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.600284+0000","last_change":"2023-10-26T01:37:40.600164+0000","last_active":"2023-10-26T01:37:40.600284+0000","last_peered":"2023-10-26T01:37:40.600284+0000","last_clean":"2023-10-26T01:37:40.600284+0000","last_became_active":"2023-10-26T01:37:40.599887+0000","last_became_peered":"2023-10-26T01:37:40.599887+0000","last_unstale":"2023-10-26T01:37:40.600284+0000","last_undegraded":"2023-10-26T01:37:40.600284+0000","last_fullsized":"2023-10-26T01:37:40.600284+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:40:01.956383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":11,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.604128+0000","last_change":"2023-10-26T01:37:39.571301+0000","last_active":"2023-10-26T01:37:40.604128+0000","last_peered":"2023-10-26T01:37:40.604128+0000","last_clean":"2023-10-26T01:37:40.604128+0000","last_became_active":"2023-10-26T01:37:39.570667+0000","last_became_peered":"2023-10-26T01:37:39.570667+0000","last_unstale":"2023-10-26T01:37:40.604128+0000","last_undegraded":"2023-10-26T01:37:40.604128+0000","last_fullsized":"2023-10-26T01:37:40.604128+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_clean_scrub_stamp":"2023-10-26T01:37:38.536854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:02:24.020794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.607668+0000","last_change":"2023-10-26T01:37:40.606485+0000","last_active":"2023-10-26T01:37:40.607668+0000","last_peered":"2023-10-26T01:37:40.607668+0000","last_clean":"2023-10-26T01:37:40.607668+0000","last_became_active":"2023-10-26T01:37:40.606130+0000","last_became_peered":"2023-10-26T01:37:40.606130+0000","last_unstale":"2023-10-26T01:37:40.607668+0000","last_undegraded":"2023-10-26T01:37:40.607668+0000","last_fullsized":"2023-10-26T01:37:40.607668+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:54:38.215736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":11,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.603770+0000","last_change":"2023-10-26T01:37:39.575020+0000","last_active":"2023-10-26T01:37:40.603770+0000","last_peered":"2023-10-26T01:37:40.603770+0000","last_clean":"2023-10-26T01:37:40.603770+0000","last_became_active":"2023-10-26T01:37:39.574702+0000","last_became_peered":"2023-10-26T01:37:39.574702+0000","last_unstale":"2023-10-26T01:37:40.603770+0000","last_undegraded":"2023-10-26T01:37:40.603770+0000","last_fullsized":"2023-10-26T01:37:40.603770+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_clean_scrub_stamp":"2023-10-26T01:37:38.536854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:06:49.733634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.608839+0000","last_change":"2023-10-26T01:37:40.605941+0000","last_active":"2023-10-26T01:37:40.608839+0000","last_peered":"2023-10-26T01:37:40.608839+0000","last_clean":"2023-10-26T01:37:40.608839+0000","last_became_active":"2023-10-26T01:37:40.604999+0000","last_became_peered":"2023-10-26T01:37:40.604999+0000","last_unstale":"2023-10-26T01:37:40.608839+0000","last_undegraded":"2023-10-26T01:37:40.608839+0000","last_fullsized":"2023-10-26T01:37:40.608839+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:33:21.466763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":11,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.591518+0000","last_change":"2023-10-26T01:37:39.564955+0000","last_active":"2023-10-26T01:37:40.591518+0000","last_peered":"2023-10-26T01:37:40.591518+0000","last_clean":"2023-10-26T01:37:40.591518+0000","last_became_active":"2023-10-26T01:37:39.564799+0000","last_became_peered":"2023-10-26T01:37:39.564799+0000","last_unstale":"2023-10-26T01:37:40.591518+0000","last_undegraded":"2023-10-26T01:37:40.591518+0000","last_fullsized":"2023-10-26T01:37:40.591518+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_clean_scrub_stamp":"2023-10-26T01:37:38.536854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:03:36.982769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.612888+0000","last_change":"2023-10-26T01:37:40.612635+0000","last_active":"2023-10-26T01:37:40.612888+0000","last_peered":"2023-10-26T01:37:40.612888+0000","last_clean":"2023-10-26T01:37:40.612888+0000","last_became_active":"2023-10-26T01:37:40.612141+0000","last_became_peered":"2023-10-26T01:37:40.612141+0000","last_unstale":"2023-10-26T01:37:40.612888+0000","last_undegraded":"2023-10-26T01:37:40.612888+0000","last_fullsized":"2023-10-26T01:37:40.612888+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:33:21.466763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"119'2","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.618922+0000","last_change":"2023-10-26T01:37:39.580633+0000","last_active":"2023-10-26T01:37:40.618922+0000","last_peered":"2023-10-26T01:37:40.618922+0000","last_clean":"2023-10-26T01:37:40.618922+0000","last_became_active":"2023-10-26T01:37:39.574770+0000","last_became_peered":"2023-10-26T01:37:39.574770+0000","last_unstale":"2023-10-26T01:37:40.618922+0000","last_undegraded":"2023-10-26T01:37:40.618922+0000","last_fullsized":"2023-10-26T01:37:40.618922+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_clean_scrub_stamp":"2023-10-26T01:37:38.536854+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:39:18.939024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.606003+0000","last_change":"2023-10-26T01:37:40.605736+0000","last_active":"2023-10-26T01:37:40.606003+0000","last_peered":"2023-10-26T01:37:40.606003+0000","last_clean":"2023-10-26T01:37:40.606003+0000","last_became_active":"2023-10-26T01:37:40.602584+0000","last_became_peered":"2023-10-26T01:37:40.602584+0000","last_unstale":"2023-10-26T01:37:40.606003+0000","last_undegraded":"2023-10-26T01:37:40.606003+0000","last_fullsized":"2023-10-26T01:37:40.606003+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:14:52.579489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":11,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.591733+0000","last_change":"2023-10-26T01:37:39.568014+0000","last_active":"2023-10-26T01:37:40.591733+0000","last_peered":"2023-10-26T01:37:40.591733+0000","last_clean":"2023-10-26T01:37:40.591733+0000","last_became_active":"2023-10-26T01:37:39.567684+0000","last_became_peered":"2023-10-26T01:37:39.567684+0000","last_unstale":"2023-10-26T01:37:40.591733+0000","last_undegraded":"2023-10-26T01:37:40.591733+0000","last_fullsized":"2023-10-26T01:37:40.591733+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_clean_scrub_stamp":"2023-10-26T01:37:38.536854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:01:32.458132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.615799+0000","last_change":"2023-10-26T01:37:40.615324+0000","last_active":"2023-10-26T01:37:40.615799+0000","last_peered":"2023-10-26T01:37:40.615799+0000","last_clean":"2023-10-26T01:37:40.615799+0000","last_became_active":"2023-10-26T01:37:40.615086+0000","last_became_peered":"2023-10-26T01:37:40.615086+0000","last_unstale":"2023-10-26T01:37:40.615799+0000","last_undegraded":"2023-10-26T01:37:40.615799+0000","last_fullsized":"2023-10-26T01:37:40.615799+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:24:05.475622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":11,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.589227+0000","last_change":"2023-10-26T01:37:39.566391+0000","last_active":"2023-10-26T01:37:40.589227+0000","last_peered":"2023-10-26T01:37:40.589227+0000","last_clean":"2023-10-26T01:37:40.589227+0000","last_became_active":"2023-10-26T01:37:39.565258+0000","last_became_peered":"2023-10-26T01:37:39.565258+0000","last_unstale":"2023-10-26T01:37:40.589227+0000","last_undegraded":"2023-10-26T01:37:40.589227+0000","last_fullsized":"2023-10-26T01:37:40.589227+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_clean_scrub_stamp":"2023-10-26T01:37:38.536854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:16:37.390420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.612167+0000","last_change":"2023-10-26T01:37:40.610590+0000","last_active":"2023-10-26T01:37:40.612167+0000","last_peered":"2023-10-26T01:37:40.612167+0000","last_clean":"2023-10-26T01:37:40.612167+0000","last_became_active":"2023-10-26T01:37:40.609644+0000","last_became_peered":"2023-10-26T01:37:40.609644+0000","last_unstale":"2023-10-26T01:37:40.612167+0000","last_undegraded":"2023-10-26T01:37:40.612167+0000","last_fullsized":"2023-10-26T01:37:40.612167+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:49:04.510364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":11,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.593016+0000","last_change":"2023-10-26T01:37:39.579228+0000","last_active":"2023-10-26T01:37:40.593016+0000","last_peered":"2023-10-26T01:37:40.593016+0000","last_clean":"2023-10-26T01:37:40.593016+0000","last_became_active":"2023-10-26T01:37:39.578701+0000","last_became_peered":"2023-10-26T01:37:39.578701+0000","last_unstale":"2023-10-26T01:37:40.593016+0000","last_undegraded":"2023-10-26T01:37:40.593016+0000","last_fullsized":"2023-10-26T01:37:40.593016+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_clean_scrub_stamp":"2023-10-26T01:37:38.536854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:43:18.947747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.603446+0000","last_change":"2023-10-26T01:37:40.602214+0000","last_active":"2023-10-26T01:37:40.603446+0000","last_peered":"2023-10-26T01:37:40.603446+0000","last_clean":"2023-10-26T01:37:40.603446+0000","last_became_active":"2023-10-26T01:37:40.602005+0000","last_became_peered":"2023-10-26T01:37:40.602005+0000","last_unstale":"2023-10-26T01:37:40.603446+0000","last_undegraded":"2023-10-26T01:37:40.603446+0000","last_fullsized":"2023-10-26T01:37:40.603446+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:53:10.171579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":11,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.591659+0000","last_change":"2023-10-26T01:37:39.572879+0000","last_active":"2023-10-26T01:37:40.591659+0000","last_peered":"2023-10-26T01:37:40.591659+0000","last_clean":"2023-10-26T01:37:40.591659+0000","last_became_active":"2023-10-26T01:37:39.572576+0000","last_became_peered":"2023-10-26T01:37:39.572576+0000","last_unstale":"2023-10-26T01:37:40.591659+0000","last_undegraded":"2023-10-26T01:37:40.591659+0000","last_fullsized":"2023-10-26T01:37:40.591659+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_clean_scrub_stamp":"2023-10-26T01:37:38.536854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:32:52.093485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.612278+0000","last_change":"2023-10-26T01:37:40.612077+0000","last_active":"2023-10-26T01:37:40.612278+0000","last_peered":"2023-10-26T01:37:40.612278+0000","last_clean":"2023-10-26T01:37:40.612278+0000","last_became_active":"2023-10-26T01:37:40.611672+0000","last_became_peered":"2023-10-26T01:37:40.611672+0000","last_unstale":"2023-10-26T01:37:40.612278+0000","last_undegraded":"2023-10-26T01:37:40.612278+0000","last_fullsized":"2023-10-26T01:37:40.612278+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:48:38.290864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":11,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.589942+0000","last_change":"2023-10-26T01:37:39.581597+0000","last_active":"2023-10-26T01:37:40.589942+0000","last_peered":"2023-10-26T01:37:40.589942+0000","last_clean":"2023-10-26T01:37:40.589942+0000","last_became_active":"2023-10-26T01:37:39.581071+0000","last_became_peered":"2023-10-26T01:37:39.581071+0000","last_unstale":"2023-10-26T01:37:40.589942+0000","last_undegraded":"2023-10-26T01:37:40.589942+0000","last_fullsized":"2023-10-26T01:37:40.589942+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_clean_scrub_stamp":"2023-10-26T01:37:38.536854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:16:37.390420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.611069+0000","last_change":"2023-10-26T01:37:40.609943+0000","last_active":"2023-10-26T01:37:40.611069+0000","last_peered":"2023-10-26T01:37:40.611069+0000","last_clean":"2023-10-26T01:37:40.611069+0000","last_became_active":"2023-10-26T01:37:40.609152+0000","last_became_peered":"2023-10-26T01:37:40.609152+0000","last_unstale":"2023-10-26T01:37:40.611069+0000","last_undegraded":"2023-10-26T01:37:40.611069+0000","last_fullsized":"2023-10-26T01:37:40.611069+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:04:58.972773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-10-26T01:37:41.587913+0000","last_change":"2023-10-26T01:37:39.571943+0000","last_active":"2023-10-26T01:37:41.587913+0000","last_peered":"2023-10-26T01:37:41.587913+0000","last_clean":"2023-10-26T01:37:41.587913+0000","last_became_active":"2023-10-26T01:37:39.571713+0000","last_became_peered":"2023-10-26T01:37:39.571713+0000","last_unstale":"2023-10-26T01:37:41.587913+0000","last_undegraded":"2023-10-26T01:37:41.587913+0000","last_fullsized":"2023-10-26T01:37:41.587913+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_clean_scrub_stamp":"2023-10-26T01:37:38.536854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:26:21.745971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.603047+0000","last_change":"2023-10-26T01:37:40.598934+0000","last_active":"2023-10-26T01:37:40.603047+0000","last_peered":"2023-10-26T01:37:40.603047+0000","last_clean":"2023-10-26T01:37:40.603047+0000","last_became_active":"2023-10-26T01:37:40.597261+0000","last_became_peered":"2023-10-26T01:37:40.597261+0000","last_unstale":"2023-10-26T01:37:40.603047+0000","last_undegraded":"2023-10-26T01:37:40.603047+0000","last_fullsized":"2023-10-26T01:37:40.603047+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:20:43.946448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":11,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.598663+0000","last_change":"2023-10-26T01:37:39.570969+0000","last_active":"2023-10-26T01:37:40.598663+0000","last_peered":"2023-10-26T01:37:40.598663+0000","last_clean":"2023-10-26T01:37:40.598663+0000","last_became_active":"2023-10-26T01:37:39.562270+0000","last_became_peered":"2023-10-26T01:37:39.562270+0000","last_unstale":"2023-10-26T01:37:40.598663+0000","last_undegraded":"2023-10-26T01:37:40.598663+0000","last_fullsized":"2023-10-26T01:37:40.598663+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_clean_scrub_stamp":"2023-10-26T01:37:38.536854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:48:13.041798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.602184+0000","last_change":"2023-10-26T01:37:40.601685+0000","last_active":"2023-10-26T01:37:40.602184+0000","last_peered":"2023-10-26T01:37:40.602184+0000","last_clean":"2023-10-26T01:37:40.602184+0000","last_became_active":"2023-10-26T01:37:40.601449+0000","last_became_peered":"2023-10-26T01:37:40.601449+0000","last_unstale":"2023-10-26T01:37:40.602184+0000","last_undegraded":"2023-10-26T01:37:40.602184+0000","last_fullsized":"2023-10-26T01:37:40.602184+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:39.547375+0000","last_clean_scrub_stamp":"2023-10-26T01:37:39.547375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:18:04.431280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"119'1","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-26T01:37:40.616136+0000","last_change":"2023-10-26T01:37:39.580512+0000","last_active":"2023-10-26T01:37:40.616136+0000","last_peered":"2023-10-26T01:37:40.616136+0000","last_clean":"2023-10-26T01:37:40.616136+0000","last_became_active":"2023-10-26T01:37:39.579888+0000","last_became_peered":"2023-10-26T01:37:39.579888+0000","last_unstale":"2023-10-26T01:37:40.616136+0000","last_undegraded":"2023-10-26T01:37:40.616136+0000","last_fullsized":"2023-10-26T01:37:40.616136+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:37:38.536854+0000","last_clean_scrub_stamp":"2023-10-26T01:37:38.536854+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:16:37.390420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":266,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-26T01:37:39.577609+0000","last_change":"2023-10-26T01:30:34.883484+0000","last_active":"2023-10-26T01:37:39.577609+0000","last_peered":"2023-10-26T01:37:39.577609+0000","last_clean":"2023-10-26T01:37:39.577609+0000","last_became_active":"2023-10-26T01:30:34.883071+0000","last_became_peered":"2023-10-26T01:30:34.883071+0000","last_unstale":"2023-10-26T01:37:39.577609+0000","last_undegraded":"2023-10-26T01:37:39.577609+0000","last_fullsized":"2023-10-26T01:37:39.577609+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:32.890096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:32.890096+0000","last_clean_scrub_stamp":"2023-10-26T01:30:32.890096+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:26:31.879452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":14,"seq":60129542233,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":646,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194666,"statfs":{"total":200000000,"available":199337988,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.79800000000000004,"15min":0.78700000000000003},"min":{"1min":0.433,"5min":0.35599999999999998,"15min":0.33000000000000002},"max":{"1min":1.3029999999999999,"5min":1.454,"15min":2.2549999999999999},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.85899999999999999,"15min":0.89200000000000002},"min":{"1min":0.52400000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.3049999999999999,"5min":1.5800000000000001,"15min":1.8200000000000001},"last":0.58599999999999997}]},{"osd":1,"last update":"Thu Oct 26 01:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.84699999999999998,"15min":0.94699999999999995},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.23,"5min":1.444,"15min":4.4340000000000002},"last":0.76000000000000001},{"interface":"front","average":{"1min":1.006,"5min":0.97099999999999997,"15min":1.0029999999999999},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.374},"max":{"1min":1.419,"5min":1.538,"15min":2.3330000000000002},"last":0.70799999999999996}]},{"osd":2,"last update":"Thu Oct 26 01:36:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":1.0189999999999999,"15min":1.083},"min":{"1min":0.40999999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.927,"5min":1.927,"15min":4.2800000000000002},"last":0.60699999999999998},{"interface":"front","average":{"1min":1.1220000000000001,"5min":1.129,"15min":1.0940000000000001},"min":{"1min":0.495,"5min":0.495,"15min":0.49399999999999999},"max":{"1min":1.8680000000000001,"5min":1.8680000000000001,"15min":2.302},"last":1.0109999999999999}]},{"osd":4,"last update":"Thu Oct 26 01:36:47 2023","interfaces":[{"interface":"back","average":{"1min":1.2569999999999999,"5min":1.1970000000000001,"15min":1.165},"min":{"1min":0.89300000000000002,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.7150000000000001,"5min":1.7150000000000001,"15min":2.1070000000000002},"last":1.141},{"interface":"front","average":{"1min":1.395,"5min":1.3360000000000001,"15min":1.401},"min":{"1min":0.89400000000000002,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":2.4239999999999999,"5min":2.4239999999999999,"15min":4.8250000000000002},"last":1.1639999999999999}]},{"osd":5,"last update":"Thu Oct 26 01:36:47 2023","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.2649999999999999,"15min":1.343},"min":{"1min":0.76400000000000001,"5min":0.76200000000000001,"15min":0.72499999999999998},"max":{"1min":2.3380000000000001,"5min":2.3380000000000001,"15min":4.9080000000000004},"last":1.0629999999999999},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.274,"15min":1.278},"min":{"1min":0.86699999999999999,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":2.0310000000000001,"5min":2.0310000000000001,"15min":2.8980000000000001},"last":1.111}]},{"osd":6,"last update":"Thu Oct 26 01:36:47 2023","interfaces":[{"interface":"back","average":{"1min":1.4710000000000001,"5min":1.3600000000000001,"15min":1.3340000000000001},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":2.1499999999999999,"5min":2.1499999999999999,"15min":2.8530000000000002},"last":1.266},{"interface":"front","average":{"1min":1.347,"5min":1.2609999999999999,"15min":1.2729999999999999},"min":{"1min":0.81899999999999995,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":2.048,"5min":2.048,"15min":2.181},"last":0.97899999999999998}]},{"osd":7,"last update":"Thu Oct 26 01:36:47 2023","interfaces":[{"interface":"back","average":{"1min":1.417,"5min":1.3220000000000001,"15min":1.298},"min":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.71999999999999997},"max":{"1min":1.958,"5min":1.958,"15min":2.0680000000000001},"last":0.85299999999999998},{"interface":"front","average":{"1min":1.448,"5min":1.379,"15min":1.4259999999999999},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":2.2370000000000001,"5min":2.2370000000000001,"15min":4.8730000000000002},"last":0.95499999999999996}]}]},{"osd":7,"up_from":13,"seq":55834574936,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1208,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194103,"statfs":{"total":200000000,"available":198762329,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:36:45 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.145,"15min":1.091},"min":{"1min":0.69699999999999995,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.6020000000000001,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":1.1890000000000001},{"interface":"front","average":{"1min":1.036,"5min":1.157,"15min":1.085},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.28,"5min":2.6779999999999999,"15min":2.6779999999999999},"last":0.70599999999999996}]},{"osd":1,"last update":"Thu Oct 26 01:36:45 2023","interfaces":[{"interface":"back","average":{"1min":1.1699999999999999,"5min":1.2050000000000001,"15min":1.091},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.65200000000000002},"max":{"1min":1.6040000000000001,"5min":2.8260000000000001,"15min":2.8260000000000001},"last":0.92200000000000004},{"interface":"front","average":{"1min":1.097,"5min":1.216,"15min":1.1879999999999999},"min":{"1min":0.68799999999999994,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.542,"5min":3.2730000000000001,"15min":3.2730000000000001},"last":0.88600000000000001}]},{"osd":2,"last update":"Thu Oct 26 01:36:45 2023","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.175,"15min":1.1140000000000001},"min":{"1min":0.69099999999999995,"5min":0.61199999999999999,"15min":0.58499999999999996},"max":{"1min":1.8680000000000001,"5min":2.218,"15min":2.218},"last":1.1180000000000001},{"interface":"front","average":{"1min":1.2909999999999999,"5min":1.296,"15min":1.236},"min":{"1min":0.79400000000000004,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.8999999999999999,"5min":3.0510000000000002,"15min":3.0510000000000002},"last":1.036}]},{"osd":3,"last update":"Thu Oct 26 01:36:45 2023","interfaces":[{"interface":"back","average":{"1min":1.264,"5min":1.272,"15min":1.208},"min":{"1min":0.80600000000000005,"5min":0.79000000000000004,"15min":0.72099999999999997},"max":{"1min":1.796,"5min":1.9830000000000001,"15min":1.9830000000000001},"last":1.25},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.419,"15min":1.377},"min":{"1min":0.82899999999999996,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.8680000000000001,"5min":5.5039999999999996,"15min":5.5039999999999996},"last":0.98099999999999998}]},{"osd":4,"last update":"Thu Oct 26 01:36:45 2023","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.3500000000000001,"15min":1.3460000000000001},"min":{"1min":0.77300000000000002,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.5640000000000001,"5min":2.77,"15min":2.77},"last":1.4770000000000001},{"interface":"front","average":{"1min":1.298,"5min":1.425,"15min":1.3939999999999999},"min":{"1min":0.96899999999999997,"5min":0.496,"15min":0.496},"max":{"1min":1.6830000000000001,"5min":3.1419999999999999,"15min":3.1419999999999999},"last":1.29}]},{"osd":5,"last update":"Thu Oct 26 01:36:45 2023","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.4710000000000001,"15min":1.419},"min":{"1min":0.65600000000000003,"5min":0.441,"15min":0.441},"max":{"1min":1.746,"5min":5.734,"15min":5.734},"last":1.081},{"interface":"front","average":{"1min":1.4339999999999999,"5min":1.5109999999999999,"15min":1.476},"min":{"1min":0.96099999999999997,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.829,"5min":2.8980000000000001,"15min":2.8980000000000001},"last":1.4159999999999999}]},{"osd":6,"last update":"Thu Oct 26 01:36:45 2023","interfaces":[{"interface":"back","average":{"1min":1.4670000000000001,"5min":1.603,"15min":1.877},"min":{"1min":0.753,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.7430000000000001,"5min":5.6340000000000003,"15min":11.577999999999999},"last":1.3540000000000001},{"interface":"front","average":{"1min":1.552,"5min":1.661,"15min":1.8879999999999999},"min":{"1min":0.91800000000000004,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.9339999999999999,"5min":5.0919999999999996,"15min":10.704000000000001},"last":1.319}]}]},{"osd":6,"up_from":13,"seq":55834574936,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":631,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194680,"statfs":{"total":200000000,"available":199353311,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:36:42 2023","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.097,"15min":1.1599999999999999},"min":{"1min":0.61699999999999999,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.647,"5min":2.395,"15min":2.395},"last":1.258},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.1299999999999999,"15min":1.1200000000000001},"min":{"1min":0.72899999999999998,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.7390000000000001,"5min":1.7390000000000001,"15min":1.843},"last":1.0109999999999999}]},{"osd":1,"last update":"Thu Oct 26 01:36:42 2023","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.1639999999999999,"15min":1.1719999999999999},"min":{"1min":0.74399999999999999,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.784,"5min":2.0529999999999999,"15min":2.0529999999999999},"last":1.5920000000000001},{"interface":"front","average":{"1min":1.139,"5min":1.1519999999999999,"15min":1.2010000000000001},"min":{"1min":0.79000000000000004,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.5229999999999999,"5min":1.581,"15min":2.085},"last":1.208}]},{"osd":2,"last update":"Thu Oct 26 01:36:42 2023","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.1779999999999999,"15min":1.0940000000000001},"min":{"1min":0.83099999999999996,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.7949999999999999,"5min":3.7879999999999998,"15min":3.7879999999999998},"last":1.3480000000000001},{"interface":"front","average":{"1min":1.319,"5min":1.2829999999999999,"15min":1.2589999999999999},"min":{"1min":0.85799999999999998,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.9910000000000001,"5min":2.2890000000000001,"15min":2.2890000000000001},"last":1.5029999999999999}]},{"osd":3,"last update":"Thu Oct 26 01:36:42 2023","interfaces":[{"interface":"back","average":{"1min":1.391,"5min":1.3080000000000001,"15min":1.2070000000000001},"min":{"1min":0.871,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.8600000000000001,"5min":4.4560000000000004,"15min":4.4560000000000004},"last":1.446},{"interface":"front","average":{"1min":1.4119999999999999,"5min":1.341,"15min":1.327},"min":{"1min":1.0369999999999999,"5min":0.82299999999999995,"15min":0.69499999999999995},"max":{"1min":1.877,"5min":2.5019999999999998,"15min":2.5019999999999998},"last":1.4039999999999999}]},{"osd":4,"last update":"Thu Oct 26 01:36:42 2023","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.252,"15min":1.22},"min":{"1min":0.93200000000000005,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.7,"5min":2.9489999999999998,"15min":2.9489999999999998},"last":1.3069999999999999},{"interface":"front","average":{"1min":1.4339999999999999,"5min":1.4330000000000001,"15min":1.3140000000000001},"min":{"1min":0.93400000000000005,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.9039999999999999,"5min":6.609,"15min":6.609},"last":1.1100000000000001}]},{"osd":5,"last update":"Thu Oct 26 01:36:42 2023","interfaces":[{"interface":"back","average":{"1min":1.6339999999999999,"5min":1.548,"15min":1.4430000000000001},"min":{"1min":1.2030000000000001,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":2.1240000000000001,"5min":5.4370000000000003,"15min":5.4370000000000003},"last":1.6200000000000001},{"interface":"front","average":{"1min":1.5309999999999999,"5min":1.448,"15min":1.409},"min":{"1min":1.1319999999999999,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.8169999999999999,"5min":2.621,"15min":2.621},"last":1.165}]},{"osd":7,"last update":"Thu Oct 26 01:36:42 2023","interfaces":[{"interface":"back","average":{"1min":1.4650000000000001,"5min":1.5229999999999999,"15min":1.411},"min":{"1min":1.2589999999999999,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.9770000000000001,"5min":11.804,"15min":11.804},"last":1.4850000000000001},{"interface":"front","average":{"1min":1.6299999999999999,"5min":1.657,"15min":1.5049999999999999},"min":{"1min":1.296,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.992,"5min":11.541,"15min":11.541},"last":1.53}]}]},{"osd":5,"up_from":13,"seq":55834574936,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":632,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194680,"statfs":{"total":200000000,"available":199352697,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:36:44 2023","interfaces":[{"interface":"back","average":{"1min":1.143,"5min":1.1739999999999999,"15min":1.1240000000000001},"min":{"1min":0.746,"5min":0.66200000000000003,"15min":0.63200000000000001},"max":{"1min":1.546,"5min":1.7390000000000001,"15min":1.7390000000000001},"last":1.464},{"interface":"front","average":{"1min":1.262,"5min":1.2709999999999999,"15min":1.177},"min":{"1min":0.81699999999999995,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.601,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":1.1100000000000001}]},{"osd":1,"last update":"Thu Oct 26 01:36:44 2023","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.1439999999999999,"15min":1.091},"min":{"1min":0.80700000000000005,"5min":0.67600000000000005,"15min":0.66400000000000003},"max":{"1min":1.7829999999999999,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":1.0449999999999999},{"interface":"front","average":{"1min":1.1040000000000001,"5min":1.1599999999999999,"15min":1.095},"min":{"1min":0.85799999999999998,"5min":0.68200000000000005,"15min":0.60599999999999998},"max":{"1min":1.4570000000000001,"5min":1.6000000000000001,"15min":1.7669999999999999},"last":1.726}]},{"osd":2,"last update":"Thu Oct 26 01:36:44 2023","interfaces":[{"interface":"back","average":{"1min":1.2290000000000001,"5min":1.204,"15min":1.1619999999999999},"min":{"1min":0.79600000000000004,"5min":0.74399999999999999,"15min":0.69299999999999995},"max":{"1min":1.6879999999999999,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":1.681},{"interface":"front","average":{"1min":1.3320000000000001,"5min":1.2350000000000001,"15min":1.1850000000000001},"min":{"1min":0.93600000000000005,"5min":0.85499999999999998,"15min":0.63900000000000001},"max":{"1min":1.6899999999999999,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":1.359}]},{"osd":3,"last update":"Thu Oct 26 01:36:44 2023","interfaces":[{"interface":"back","average":{"1min":1.2809999999999999,"5min":1.3080000000000001,"15min":1.246},"min":{"1min":0.92200000000000004,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.613,"5min":1.7649999999999999,"15min":1.835},"last":1.6180000000000001},{"interface":"front","average":{"1min":1.373,"5min":1.321,"15min":1.2949999999999999},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.788,"5min":1.788,"15min":2.9249999999999998},"last":1.4139999999999999}]},{"osd":4,"last update":"Thu Oct 26 01:36:44 2023","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.286,"15min":1.206},"min":{"1min":0.97699999999999998,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.845,"5min":1.845,"15min":1.845},"last":1.2929999999999999},{"interface":"front","average":{"1min":1.4810000000000001,"5min":1.4470000000000001,"15min":1.353},"min":{"1min":0.90600000000000003,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.7849999999999999,"5min":1.867,"15min":1.867},"last":1.5289999999999999}]},{"osd":6,"last update":"Thu Oct 26 01:36:44 2023","interfaces":[{"interface":"back","average":{"1min":1.5269999999999999,"5min":1.444,"15min":1.3220000000000001},"min":{"1min":1.0660000000000001,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.883,"5min":2.8250000000000002,"15min":2.8250000000000002},"last":1.7490000000000001},{"interface":"front","average":{"1min":1.5429999999999999,"5min":1.4490000000000001,"15min":1.3260000000000001},"min":{"1min":1.1879999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.849,"5min":1.849,"15min":1.8560000000000001},"last":1.5880000000000001}]},{"osd":7,"last update":"Thu Oct 26 01:36:44 2023","interfaces":[{"interface":"back","average":{"1min":1.599,"5min":1.5449999999999999,"15min":1.417},"min":{"1min":1.119,"5min":0.64000000000000001,"15min":0.61699999999999999},"max":{"1min":2.1040000000000001,"5min":2.1040000000000001,"15min":2.1040000000000001},"last":1.8220000000000001},{"interface":"front","average":{"1min":1.6619999999999999,"5min":1.5660000000000001,"15min":1.4299999999999999},"min":{"1min":1.389,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":2.0960000000000001,"5min":2.0960000000000001,"15min":2.0960000000000001},"last":1.853}]}]},{"osd":4,"up_from":13,"seq":55834574936,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":632,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194680,"statfs":{"total":200000000,"available":199352697,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:36:37 2023","interfaces":[{"interface":"back","average":{"1min":1.081,"5min":1.04,"15min":0.97699999999999998},"min":{"1min":0.69199999999999995,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.5309999999999999,"5min":2.0289999999999999,"15min":2.0289999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":1.071,"5min":1.0649999999999999,"15min":1.0229999999999999},"min":{"1min":0.66400000000000003,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.6599999999999999,"5min":2.6829999999999998,"15min":2.6829999999999998},"last":0.76500000000000001}]},{"osd":1,"last update":"Thu Oct 26 01:36:37 2023","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.1799999999999999,"15min":1.1220000000000001},"min":{"1min":0.98399999999999999,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.599,"5min":4.0519999999999996,"15min":4.0519999999999996},"last":0.94199999999999995},{"interface":"front","average":{"1min":1.119,"5min":1.1439999999999999,"15min":1.105},"min":{"1min":0.77900000000000003,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.6240000000000001,"5min":2.0760000000000001,"15min":2.0760000000000001},"last":1.2669999999999999}]},{"osd":2,"last update":"Thu Oct 26 01:36:37 2023","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":1.3280000000000001,"15min":1.222},"min":{"1min":0.79100000000000004,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.5660000000000001,"5min":5.6749999999999998,"15min":5.6749999999999998},"last":1.054},{"interface":"front","average":{"1min":1.1970000000000001,"5min":1.1879999999999999,"15min":1.1220000000000001},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.746,"5min":1.849,"15min":1.849},"last":1.345}]},{"osd":3,"last update":"Thu Oct 26 01:36:37 2023","interfaces":[{"interface":"back","average":{"1min":1.286,"5min":1.2,"15min":1.1839999999999999},"min":{"1min":0.84399999999999997,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.847,"5min":1.891,"15min":1.891},"last":1.101},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.2030000000000001,"15min":1.155},"min":{"1min":0.94499999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.5669999999999999,"5min":1.7210000000000001,"15min":1.7210000000000001},"last":1.173}]},{"osd":5,"last update":"Thu Oct 26 01:36:37 2023","interfaces":[{"interface":"back","average":{"1min":1.2749999999999999,"5min":1.2509999999999999,"15min":1.286},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.46300000000000002},"max":{"1min":2.1429999999999998,"5min":2.1429999999999998,"15min":2.1429999999999998},"last":1.399},{"interface":"front","average":{"1min":1.28,"5min":1.331,"15min":1.327},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.716,"5min":5.8579999999999997,"15min":5.8579999999999997},"last":1.226}]},{"osd":6,"last update":"Thu Oct 26 01:36:37 2023","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":1.4199999999999999,"15min":1.3340000000000001},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.639,"5min":12.343999999999999,"15min":12.343999999999999},"last":1.4350000000000001},{"interface":"front","average":{"1min":1.54,"5min":1.6220000000000001,"15min":1.5129999999999999},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.55000000000000004},"max":{"1min":2.0369999999999999,"5min":12.605,"15min":12.605},"last":1.4650000000000001}]},{"osd":7,"last update":"Thu Oct 26 01:36:37 2023","interfaces":[{"interface":"back","average":{"1min":1.4650000000000001,"5min":1.518,"15min":1.4910000000000001},"min":{"1min":0.748,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":2.762,"5min":5.774,"15min":5.774},"last":1.306},{"interface":"front","average":{"1min":1.5249999999999999,"5min":1.518,"15min":1.4530000000000001},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":2.8330000000000002,"5min":4.6520000000000001,"15min":4.6520000000000001},"last":1.4830000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574936,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":632,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194680,"statfs":{"total":200000000,"available":199352697,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":0.875,"15min":0.86099999999999999},"min":{"1min":0.63,"5min":0.308,"15min":0.308},"max":{"1min":1.5249999999999999,"5min":6.3200000000000003,"15min":6.3200000000000003},"last":1.1899999999999999},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.0229999999999999,"15min":0.93600000000000005},"min":{"1min":0.52000000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.7130000000000001,"5min":4.2619999999999996,"15min":4.2619999999999996},"last":1.123}]},{"osd":1,"last update":"Thu Oct 26 01:36:41 2023","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":0.93200000000000005,"15min":0.90500000000000003},"min":{"1min":0.66700000000000004,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.629,"5min":5.1260000000000003,"15min":5.1260000000000003},"last":0.99399999999999999},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.0649999999999999,"15min":1.012},"min":{"1min":0.59599999999999997,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.873,"5min":6.1520000000000001,"15min":6.1520000000000001},"last":0.93200000000000005}]},{"osd":3,"last update":"Thu Oct 26 01:36:47 2023","interfaces":[{"interface":"back","average":{"1min":1.131,"5min":1.034,"15min":0.93999999999999995},"min":{"1min":0.54900000000000004,"5min":0.443,"15min":0.443},"max":{"1min":2.0640000000000001,"5min":2.0640000000000001,"15min":2.0640000000000001},"last":1.2989999999999999},{"interface":"front","average":{"1min":1.22,"5min":1.1699999999999999,"15min":1.0189999999999999},"min":{"1min":0.67300000000000004,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.9279999999999999,"5min":8.9019999999999992,"15min":8.9019999999999992},"last":1.325}]},{"osd":4,"last update":"Thu Oct 26 01:36:41 2023","interfaces":[{"interface":"back","average":{"1min":1.454,"5min":1.365,"15min":1.268},"min":{"1min":0.80500000000000005,"5min":0.751,"15min":0.73899999999999999},"max":{"1min":2.6190000000000002,"5min":6.2670000000000003,"15min":6.2670000000000003},"last":1.5469999999999999},{"interface":"front","average":{"1min":1.5089999999999999,"5min":1.367,"15min":1.2629999999999999},"min":{"1min":0.93700000000000006,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":2.4359999999999999,"5min":7.343,"15min":7.343},"last":1.411}]},{"osd":5,"last update":"Thu Oct 26 01:36:41 2023","interfaces":[{"interface":"back","average":{"1min":1.5649999999999999,"5min":1.369,"15min":1.2909999999999999},"min":{"1min":1.018,"5min":0.80800000000000005,"15min":0.71899999999999997},"max":{"1min":2.5790000000000002,"5min":3.7170000000000001,"15min":3.7170000000000001},"last":1.222},{"interface":"front","average":{"1min":1.4650000000000001,"5min":1.3540000000000001,"15min":1.3},"min":{"1min":0.88800000000000001,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":2.298,"5min":6.3899999999999997,"15min":6.3899999999999997},"last":1.2649999999999999}]},{"osd":6,"last update":"Thu Oct 26 01:36:41 2023","interfaces":[{"interface":"back","average":{"1min":1.4690000000000001,"5min":1.4259999999999999,"15min":1.335},"min":{"1min":0.871,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":2.73,"5min":6.0490000000000004,"15min":6.0490000000000004},"last":1.9379999999999999},{"interface":"front","average":{"1min":1.599,"5min":1.4399999999999999,"15min":1.375},"min":{"1min":1.0660000000000001,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":2.4089999999999998,"5min":5.0650000000000004,"15min":5.0650000000000004},"last":1.575}]},{"osd":7,"last update":"Thu Oct 26 01:36:41 2023","interfaces":[{"interface":"back","average":{"1min":1.532,"5min":1.4390000000000001,"15min":1.405},"min":{"1min":1.0449999999999999,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":2.3370000000000002,"5min":6.4480000000000004,"15min":6.4480000000000004},"last":1.488},{"interface":"front","average":{"1min":1.6279999999999999,"5min":1.5169999999999999,"15min":1.4570000000000001},"min":{"1min":1.117,"5min":0.91800000000000004,"15min":0.91800000000000004},"max":{"1min":2.8290000000000002,"5min":2.8439999999999999,"15min":2.8439999999999999},"last":1.877}]}]},{"osd":1,"up_from":13,"seq":55834574936,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":638,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194673,"statfs":{"total":200000000,"available":199345922,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.78100000000000003,"15min":0.72099999999999997},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.29499999999999998},"max":{"1min":1.0820000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.85599999999999998,"15min":0.79400000000000004},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36199999999999999},"max":{"1min":1.589,"5min":1.6399999999999999,"15min":1.6399999999999999},"last":1.331}]},{"osd":2,"last update":"Thu Oct 26 01:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.94599999999999995,"15min":0.99399999999999999},"min":{"1min":0.55300000000000005,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.5049999999999999,"5min":1.5389999999999999,"15min":2.4830000000000001},"last":1.099},{"interface":"front","average":{"1min":1.0209999999999999,"5min":0.89700000000000002,"15min":0.86099999999999999},"min":{"1min":0.57099999999999995,"5min":0.432,"15min":0.432},"max":{"1min":1.8080000000000001,"5min":1.8080000000000001,"15min":1.8080000000000001},"last":0.71899999999999997}]},{"osd":3,"last update":"Thu Oct 26 01:36:50 2023","interfaces":[{"interface":"back","average":{"1min":1.0389999999999999,"5min":0.98499999999999999,"15min":0.94699999999999995},"min":{"1min":0.42499999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.5289999999999999,"5min":1.75,"15min":1.75},"last":1.052},{"interface":"front","average":{"1min":1.095,"5min":1.0289999999999999,"15min":0.98899999999999999},"min":{"1min":0.54200000000000004,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.673,"5min":1.802,"15min":1.802},"last":1.2849999999999999}]},{"osd":4,"last update":"Thu Oct 26 01:36:46 2023","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.2250000000000001,"15min":1.196},"min":{"1min":0.69099999999999995,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.706,"5min":1.8129999999999999,"15min":2.0659999999999998},"last":1.407},{"interface":"front","average":{"1min":1.325,"5min":1.1950000000000001,"15min":1.137},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":2.0270000000000001,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":1.143}]},{"osd":5,"last update":"Thu Oct 26 01:36:46 2023","interfaces":[{"interface":"back","average":{"1min":1.3360000000000001,"5min":1.262,"15min":1.232},"min":{"1min":0.91800000000000004,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":1.8779999999999999,"5min":1.8779999999999999,"15min":1.8779999999999999},"last":1.573},{"interface":"front","average":{"1min":1.3620000000000001,"5min":1.347,"15min":1.3340000000000001},"min":{"1min":0.66000000000000003,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.744,"5min":1.782,"15min":2.2200000000000002},"last":1.534}]},{"osd":6,"last update":"Thu Oct 26 01:36:46 2023","interfaces":[{"interface":"back","average":{"1min":1.367,"5min":1.3180000000000001,"15min":1.278},"min":{"1min":0.70199999999999996,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.9179999999999999,"5min":2.0910000000000002,"15min":2.3780000000000001},"last":1.1910000000000001},{"interface":"front","average":{"1min":1.3260000000000001,"5min":1.276,"15min":1.242},"min":{"1min":0.89600000000000002,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.956,"5min":1.956,"15min":1.984},"last":1.226}]},{"osd":7,"last update":"Thu Oct 26 01:36:46 2023","interfaces":[{"interface":"back","average":{"1min":1.3620000000000001,"5min":1.262,"15min":1.268},"min":{"1min":0.95699999999999996,"5min":0.78000000000000003,"15min":0.69699999999999995},"max":{"1min":1.8380000000000001,"5min":1.8380000000000001,"15min":2.6930000000000001},"last":1.482},{"interface":"front","average":{"1min":1.506,"5min":1.3919999999999999,"15min":1.3320000000000001},"min":{"1min":0.75600000000000001,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":2.0190000000000001,"5min":2.0190000000000001,"15min":2.0190000000000001},"last":1.371}]}]},{"osd":0,"up_from":13,"seq":55834574936,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1215,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194096,"statfs":{"total":200000000,"available":198754972,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Oct 26 01:36:43 2023","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.998,"15min":0.92300000000000004},"min":{"1min":0.64000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.49,"5min":19.425000000000001,"15min":19.425000000000001},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.98199999999999998,"5min":1.075,"15min":1.0549999999999999},"min":{"1min":0.67400000000000004,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.339,"5min":15.162000000000001,"15min":15.162000000000001},"last":0.56100000000000005}]},{"osd":2,"last update":"Thu Oct 26 01:36:43 2023","interfaces":[{"interface":"back","average":{"1min":1,"5min":1.2969999999999999,"15min":1.1930000000000001},"min":{"1min":0.59899999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.4970000000000001,"5min":28.439,"15min":28.439},"last":0.94999999999999996},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.90800000000000003,"15min":0.91400000000000003},"min":{"1min":0.69399999999999995,"5min":0.45200000000000001,"15min":0.38300000000000001},"max":{"1min":1.268,"5min":1.502,"15min":4.9459999999999997},"last":0.90500000000000003}]},{"osd":3,"last update":"Thu Oct 26 01:36:43 2023","interfaces":[{"interface":"back","average":{"1min":1.101,"5min":1.0469999999999999,"15min":1.093},"min":{"1min":0.64100000000000001,"5min":0.46000000000000002,"15min":0.44600000000000001},"max":{"1min":1.47,"5min":1.867,"15min":4.9800000000000004},"last":1.163},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.1359999999999999,"15min":1.1619999999999999},"min":{"1min":0.752,"5min":0.56699999999999995,"15min":0.54000000000000004},"max":{"1min":1.4450000000000001,"5min":1.992,"15min":5.016},"last":1.319}]},{"osd":4,"last update":"Thu Oct 26 01:36:43 2023","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.387,"15min":1.3009999999999999},"min":{"1min":0.82799999999999996,"5min":0.66800000000000004,"15min":0.61499999999999999},"max":{"1min":1.681,"5min":19.329000000000001,"15min":19.329000000000001},"last":1.024},{"interface":"front","average":{"1min":1.319,"5min":1.27,"15min":1.306},"min":{"1min":0.82499999999999996,"5min":0.79100000000000004,"15min":0.67600000000000005},"max":{"1min":1.718,"5min":2.036,"15min":5.5229999999999997},"last":1.091}]},{"osd":5,"last update":"Thu Oct 26 01:36:43 2023","interfaces":[{"interface":"back","average":{"1min":1.5129999999999999,"5min":1.4039999999999999,"15min":1.4350000000000001},"min":{"1min":1.1359999999999999,"5min":0.80000000000000004,"15min":0.72499999999999998},"max":{"1min":1.857,"5min":2.2290000000000001,"15min":5.6360000000000001},"last":1.381},{"interface":"front","average":{"1min":1.399,"5min":1.492,"15min":1.4119999999999999},"min":{"1min":1.0880000000000001,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.8200000000000001,"5min":19.384,"15min":19.384},"last":1.129}]},{"osd":6,"last update":"Thu Oct 26 01:36:43 2023","interfaces":[{"interface":"back","average":{"1min":1.3580000000000001,"5min":1.375,"15min":1.3720000000000001},"min":{"1min":1.034,"5min":0.77200000000000002,"15min":0.73799999999999999},"max":{"1min":1.6970000000000001,"5min":8.2609999999999992,"15min":8.2609999999999992},"last":1.353},{"interface":"front","average":{"1min":1.377,"5min":1.4370000000000001,"15min":1.4099999999999999},"min":{"1min":0.85899999999999999,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.9450000000000001,"5min":11.849,"15min":11.849},"last":1.2829999999999999}]},{"osd":7,"last update":"Thu Oct 26 01:36:43 2023","interfaces":[{"interface":"back","average":{"1min":1.4970000000000001,"5min":1.4450000000000001,"15min":1.488},"min":{"1min":0.999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.7809999999999999,"5min":2.4060000000000001,"15min":5.6689999999999996},"last":1.4490000000000001},{"interface":"front","average":{"1min":1.4930000000000001,"5min":1.6080000000000001,"15min":1.5089999999999999},"min":{"1min":1.1579999999999999,"5min":0.81299999999999994,"15min":0.69399999999999995},"max":{"1min":1.9390000000000001,"5min":19.245999999999999,"15min":19.245999999999999},"last":1.417}]}]}],"pool_statfs":[]}} 2023-10-26T01:37:45.934 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-26T01:37:45.935 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-26T01:37:46.206 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.205+0000 7f6f8dd9b640 1 -- 172.21.15.169:0/125265972 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f8802fd90 con 0x7f6f78004d00 2023-10-26T01:37:46.207 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.205+0000 7f6f951b6640 1 -- 172.21.15.169:0/125265972 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f78004d00 msgr2=0x7f6f78005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:46.208 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.205+0000 7f6f951b6640 1 --2- 172.21.15.169:0/125265972 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f78004d00 0x7f6f78005180 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f6f88009a90 tx=0x7f6f780057f0 comp rx=0 tx=0).stop 2023-10-26T01:37:46.208 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.205+0000 7f6f8f59e640 1 -- 172.21.15.169:0/125265972 reap_dead start 2023-10-26T01:37:46.208 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.205+0000 7f6f951b6640 1 -- 172.21.15.169:0/125265972 shutdown_connections 2023-10-26T01:37:46.208 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.205+0000 7f6f951b6640 1 -- 172.21.15.169:0/125265972 >> 172.21.15.169:0/125265972 conn(0x7f6f90069930 msgr2=0x7f6f90069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:46.208 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.205+0000 7f6f951b6640 1 -- 172.21.15.169:0/125265972 shutdown_connections 2023-10-26T01:37:46.209 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.205+0000 7f6f951b6640 1 -- 172.21.15.169:0/125265972 wait complete. 2023-10-26T01:37:46.209 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.207+0000 7f6f951b6640 1 Processor -- start 2023-10-26T01:37:46.210 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.207+0000 7f6f951b6640 1 -- start start 2023-10-26T01:37:46.212 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.208+0000 7f6f951b6640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f78004d00 0x7f6f90096550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:46.212 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.208+0000 7f6f951b6640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6f9006ed30 0x7f6f901e2710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:46.212 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.208+0000 7f6f951b6640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f901e2c50 0x7f6f901e3100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:46.212 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.208+0000 7f6f951b6640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f901eeb20 con 0x7f6f901e2c50 2023-10-26T01:37:46.212 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.208+0000 7f6f951b6640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f901eee00 con 0x7f6f78004d00 2023-10-26T01:37:46.213 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.208+0000 7f6f951b6640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f90096a90 con 0x7f6f9006ed30 2023-10-26T01:37:46.213 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.208+0000 7f6f8f59e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f901e2c50 0x7f6f901e3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:46.213 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.209+0000 7f6f8f59e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f901e2c50 0x7f6f901e3100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:35680/0 (socket says 172.21.15.169:35680) 2023-10-26T01:37:46.213 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.209+0000 7f6f8f59e640 1 -- 172.21.15.169:0/2813772290 learned_addr learned my addr 172.21.15.169:0/2813772290 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:37:46.213 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.209+0000 7f6f8e59c640 1 --2- 172.21.15.169:0/2813772290 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6f9006ed30 0x7f6f901e2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:46.214 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.209+0000 7f6f8f59e640 1 -- 172.21.15.169:0/2813772290 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6f9006ed30 msgr2=0x7f6f901e2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:46.214 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.209+0000 7f6f8ed9d640 1 --2- 172.21.15.169:0/2813772290 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f78004d00 0x7f6f90096550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:46.214 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.209+0000 7f6f8f59e640 1 --2- 172.21.15.169:0/2813772290 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6f9006ed30 0x7f6f901e2710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:46.214 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.209+0000 7f6f8f59e640 1 -- 172.21.15.169:0/2813772290 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f78004d00 msgr2=0x7f6f90096550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:46.214 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.209+0000 7f6f8f59e640 1 --2- 172.21.15.169:0/2813772290 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f78004d00 0x7f6f90096550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:46.215 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.209+0000 7f6f8f59e640 1 -- 172.21.15.169:0/2813772290 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f900713a0 con 0x7f6f901e2c50 2023-10-26T01:37:46.215 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.209+0000 7f6f8e59c640 1 --2- 172.21.15.169:0/2813772290 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6f9006ed30 0x7f6f901e2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:37:46.215 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.209+0000 7f6f8ed9d640 1 --2- 172.21.15.169:0/2813772290 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f78004d00 0x7f6f90096550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:37:46.215 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.209+0000 7f6f8f59e640 1 --2- 172.21.15.169:0/2813772290 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f901e2c50 0x7f6f901e3100 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f6f8802f210 tx=0x7f6f8802f870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:46.215 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.210+0000 7f6f777fe640 1 -- 172.21.15.169:0/2813772290 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f88003b70 con 0x7f6f901e2c50 2023-10-26T01:37:46.216 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.210+0000 7f6f777fe640 1 -- 172.21.15.169:0/2813772290 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6f88003dc0 con 0x7f6f901e2c50 2023-10-26T01:37:46.216 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.210+0000 7f6f777fe640 1 -- 172.21.15.169:0/2813772290 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f88037970 con 0x7f6f901e2c50 2023-10-26T01:37:46.216 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.210+0000 7f6f951b6640 1 -- 172.21.15.169:0/2813772290 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f90071680 con 0x7f6f901e2c50 2023-10-26T01:37:46.216 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.210+0000 7f6f951b6640 1 -- 172.21.15.169:0/2813772290 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f90070fe0 con 0x7f6f901e2c50 2023-10-26T01:37:46.217 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.211+0000 7f6f951b6640 1 -- 172.21.15.169:0/2813772290 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f9004db10 con 0x7f6f901e2c50 2023-10-26T01:37:46.217 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.212+0000 7f6f777fe640 1 -- 172.21.15.169:0/2813772290 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f6f8803e040 con 0x7f6f901e2c50 2023-10-26T01:37:46.217 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.213+0000 7f6f777fe640 1 --2- 172.21.15.169:0/2813772290 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f6f7c06ae70 0x7f6f7c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:46.218 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.213+0000 7f6f777fe640 1 -- 172.21.15.169:0/2813772290 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6f880ad3c0 con 0x7f6f901e2c50 2023-10-26T01:37:46.218 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.214+0000 7f6f8ed9d640 1 --2- 172.21.15.169:0/2813772290 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f6f7c06ae70 0x7f6f7c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:46.218 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.215+0000 7f6f8ed9d640 1 --2- 172.21.15.169:0/2813772290 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f6f7c06ae70 0x7f6f7c06d330 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f6f80000fd0 tx=0x7f6f80009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:46.219 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.217+0000 7f6f777fe640 1 -- 172.21.15.169:0/2813772290 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6f88078510 con 0x7f6f901e2c50 2023-10-26T01:37:46.475 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.473+0000 7f6f951b6640 1 -- 172.21.15.169:0/2813772290 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f6f9006a000 con 0x7f6f901e2c50 2023-10-26T01:37:46.480 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.479+0000 7f6f777fe640 1 -- 172.21.15.169:0/2813772290 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v16) v1 ==== 190+0+147 (secure 0 0 0) 0x7f6f880460b0 con 0x7f6f901e2c50 2023-10-26T01:37:46.481 INFO:teuthology.orchestra.run.smithi169.stdout:[client.0] 2023-10-26T01:37:46.481 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQAgwTllKC7gOhAAPsvxGcuUA8/p8xqbUgGteA== 2023-10-26T01:37:46.482 INFO:teuthology.orchestra.run.smithi169.stdout: caps mds = "allow" 2023-10-26T01:37:46.482 INFO:teuthology.orchestra.run.smithi169.stdout: caps mon = "allow r" 2023-10-26T01:37:46.482 INFO:teuthology.orchestra.run.smithi169.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-10-26T01:37:46.482 INFO:teuthology.orchestra.run.smithi169.stderr:updated caps for client.0 2023-10-26T01:37:46.484 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.482+0000 7f6f951b6640 1 -- 172.21.15.169:0/2813772290 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f6f7c06ae70 msgr2=0x7f6f7c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:46.484 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.482+0000 7f6f951b6640 1 --2- 172.21.15.169:0/2813772290 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f6f7c06ae70 0x7f6f7c06d330 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f6f80000fd0 tx=0x7f6f80009040 comp rx=0 tx=0).stop 2023-10-26T01:37:46.484 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.482+0000 7f6f951b6640 1 -- 172.21.15.169:0/2813772290 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f901e2c50 msgr2=0x7f6f901e3100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:46.485 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.482+0000 7f6f951b6640 1 --2- 172.21.15.169:0/2813772290 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f901e2c50 0x7f6f901e3100 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f6f8802f210 tx=0x7f6f8802f870 comp rx=0 tx=0).stop 2023-10-26T01:37:46.485 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.482+0000 7f6f951b6640 1 -- 172.21.15.169:0/2813772290 shutdown_connections 2023-10-26T01:37:46.485 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.482+0000 7f6f951b6640 1 --2- 172.21.15.169:0/2813772290 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f6f7c06ae70 0x7f6f7c06d330 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:46.485 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.482+0000 7f6f951b6640 1 --2- 172.21.15.169:0/2813772290 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f901e2c50 0x7f6f901e3100 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:46.485 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.482+0000 7f6f951b6640 1 --2- 172.21.15.169:0/2813772290 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6f9006ed30 0x7f6f901e2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:46.486 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.482+0000 7f6f951b6640 1 --2- 172.21.15.169:0/2813772290 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f78004d00 0x7f6f90096550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:46.486 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.482+0000 7f6f951b6640 1 -- 172.21.15.169:0/2813772290 >> 172.21.15.169:0/2813772290 conn(0x7f6f90069930 msgr2=0x7f6f90158d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:46.486 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.482+0000 7f6f951b6640 1 -- 172.21.15.169:0/2813772290 shutdown_connections 2023-10-26T01:37:46.486 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.482+0000 7f6f951b6640 1 -- 172.21.15.169:0/2813772290 wait complete. 2023-10-26T01:37:46.502 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T01:37:46.544 INFO:tasks.ceph.mds.c.smithi169.stderr:2023-10-26T01:37:46.542+0000 7f33bf673640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-26T01:37:46.718 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.716+0000 7f2224490640 1 -- 172.21.15.169:0/2994022510 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f221c0ac930 msgr2=0x7f221c0cd6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:46.719 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.716+0000 7f2224490640 1 --2- 172.21.15.169:0/2994022510 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f221c0ac930 0x7f221c0cd6f0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f220c005fd0 tx=0x7f220c02bd10 comp rx=0 tx=0).stop 2023-10-26T01:37:46.719 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.716+0000 7f2224490640 1 -- 172.21.15.169:0/2994022510 shutdown_connections 2023-10-26T01:37:46.719 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.716+0000 7f2224490640 1 --2- 172.21.15.169:0/2994022510 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f221c0ac930 0x7f221c0cd6f0 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:46.719 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.716+0000 7f2224490640 1 --2- 172.21.15.169:0/2994022510 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f221c1592d0 0x7f221c0aa3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:46.720 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.716+0000 7f2224490640 1 -- 172.21.15.169:0/2994022510 >> 172.21.15.169:0/2994022510 conn(0x7f221c069930 msgr2=0x7f221c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:46.720 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.717+0000 7f2224490640 1 -- 172.21.15.169:0/2994022510 shutdown_connections 2023-10-26T01:37:46.720 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.717+0000 7f2224490640 1 -- 172.21.15.169:0/2994022510 wait complete. 2023-10-26T01:37:46.722 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.718+0000 7f2224490640 1 Processor -- start 2023-10-26T01:37:46.723 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.718+0000 7f2224490640 1 -- start start 2023-10-26T01:37:46.723 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.719+0000 7f2224490640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f221c0ac930 0x7f221c06f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:46.723 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.719+0000 7f2224490640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f221c0cdc30 0x7f221c0988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:46.723 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.719+0000 7f2224490640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f221c1592d0 0x7f221c098de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:46.723 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.719+0000 7f2224490640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f221c099640 con 0x7f221c0ac930 2023-10-26T01:37:46.724 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.719+0000 7f2224490640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f221c099920 con 0x7f221c0cdc30 2023-10-26T01:37:46.724 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.719+0000 7f2224490640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f221c099c00 con 0x7f221c1592d0 2023-10-26T01:37:46.724 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.720+0000 7f222222c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f221c0ac930 0x7f221c06f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:46.724 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.720+0000 7f222222c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f221c0ac930 0x7f221c06f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:35696/0 (socket says 172.21.15.169:35696) 2023-10-26T01:37:46.724 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.720+0000 7f222222c640 1 -- 172.21.15.169:0/34517688 learned_addr learned my addr 172.21.15.169:0/34517688 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:37:46.725 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.720+0000 7f2221a2b640 1 --2- 172.21.15.169:0/34517688 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f221c0cdc30 0x7f221c0988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:46.725 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.720+0000 7f222222c640 1 -- 172.21.15.169:0/34517688 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f221c1592d0 msgr2=0x7f221c098de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:46.725 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.720+0000 7f2222a2d640 1 --2- 172.21.15.169:0/34517688 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f221c1592d0 0x7f221c098de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:46.725 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.720+0000 7f222222c640 1 --2- 172.21.15.169:0/34517688 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f221c1592d0 0x7f221c098de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:46.726 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.720+0000 7f222222c640 1 -- 172.21.15.169:0/34517688 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f221c0cdc30 msgr2=0x7f221c0988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:46.726 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.720+0000 7f222222c640 1 --2- 172.21.15.169:0/34517688 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f221c0cdc30 0x7f221c0988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:46.726 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.720+0000 7f222222c640 1 -- 172.21.15.169:0/34517688 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f220c009560 con 0x7f221c0ac930 2023-10-26T01:37:46.726 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.720+0000 7f2221a2b640 1 --2- 172.21.15.169:0/34517688 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f221c0cdc30 0x7f221c0988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:37:46.726 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.720+0000 7f2222a2d640 1 --2- 172.21.15.169:0/34517688 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f221c1592d0 0x7f221c098de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:37:46.727 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.720+0000 7f222222c640 1 --2- 172.21.15.169:0/34517688 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f221c0ac930 0x7f221c06f260 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f221800b410 tx=0x7f221800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:46.727 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.721+0000 7f2212ffd640 1 -- 172.21.15.169:0/34517688 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2218011830 con 0x7f221c0ac930 2023-10-26T01:37:46.727 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.721+0000 7f2212ffd640 1 -- 172.21.15.169:0/34517688 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2218011aa0 con 0x7f221c0ac930 2023-10-26T01:37:46.727 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.721+0000 7f2212ffd640 1 -- 172.21.15.169:0/34517688 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2218010380 con 0x7f221c0ac930 2023-10-26T01:37:46.727 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.721+0000 7f2224490640 1 -- 172.21.15.169:0/34517688 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f221c099ee0 con 0x7f221c0ac930 2023-10-26T01:37:46.728 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.721+0000 7f2224490640 1 -- 172.21.15.169:0/34517688 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f221c099320 con 0x7f221c0ac930 2023-10-26T01:37:46.729 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.722+0000 7f2224490640 1 -- 172.21.15.169:0/34517688 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f221c04db10 con 0x7f221c0ac930 2023-10-26T01:37:46.729 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.723+0000 7f2212ffd640 1 -- 172.21.15.169:0/34517688 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f22180104e0 con 0x7f221c0ac930 2023-10-26T01:37:46.729 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.724+0000 7f2212ffd640 1 --2- 172.21.15.169:0/34517688 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f220006ae40 0x7f220006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:46.729 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.724+0000 7f2212ffd640 1 -- 172.21.15.169:0/34517688 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2218089930 con 0x7f221c0ac930 2023-10-26T01:37:46.729 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.724+0000 7f2221a2b640 1 --2- 172.21.15.169:0/34517688 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f220006ae40 0x7f220006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:46.730 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.726+0000 7f2221a2b640 1 --2- 172.21.15.169:0/34517688 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f220006ae40 0x7f220006d300 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f220c035000 tx=0x7f220c004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:46.730 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.728+0000 7f2212ffd640 1 -- 172.21.15.169:0/34517688 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2218054a70 con 0x7f221c0ac930 2023-10-26T01:37:46.961 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.960+0000 7f2224490640 1 -- 172.21.15.169:0/34517688 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f221c06b0e0 con 0x7f221c0ac930 2023-10-26T01:37:46.962 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.961+0000 7f2212ffd640 1 -- 172.21.15.169:0/34517688 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) v1 ==== 94+0+4924 (secure 0 0 0) 0x7f2218015090 con 0x7f221c0ac930 2023-10-26T01:37:46.963 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:37:46.963 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":59,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5387,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/1186563921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1186563921},{"type":"v1","addr":"172.21.15.170:6835","nonce":1186563921}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":54},{"gid":5390,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/3702036693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":3702036693},{"type":"v1","addr":"172.21.15.170:6837","nonce":3702036693}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":54},{"gid":7332,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/3193888686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":3193888686},{"type":"v1","addr":"172.21.15.169:6833","nonce":3193888686}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T01:37:40.027858+0000","modified":"2023-10-26T01:37:43.103474+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7335},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7335":{"gid":7335,"name":"c","rank":0,"incarnation":55,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6835/668588916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":668588916},{"type":"v1","addr":"172.21.15.169:6835","nonce":668588916}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-10-26T01:37:46.963 INFO:teuthology.orchestra.run.smithi169.stderr:dumped fsmap epoch 59 2023-10-26T01:37:46.965 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.963+0000 7f2224490640 1 -- 172.21.15.169:0/34517688 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f220006ae40 msgr2=0x7f220006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:46.966 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.963+0000 7f2224490640 1 --2- 172.21.15.169:0/34517688 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f220006ae40 0x7f220006d300 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f220c035000 tx=0x7f220c004730 comp rx=0 tx=0).stop 2023-10-26T01:37:46.966 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.963+0000 7f2224490640 1 -- 172.21.15.169:0/34517688 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f221c0ac930 msgr2=0x7f221c06f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:46.966 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.963+0000 7f2224490640 1 --2- 172.21.15.169:0/34517688 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f221c0ac930 0x7f221c06f260 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f221800b410 tx=0x7f221800b8d0 comp rx=0 tx=0).stop 2023-10-26T01:37:46.966 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.963+0000 7f2224490640 1 -- 172.21.15.169:0/34517688 shutdown_connections 2023-10-26T01:37:46.967 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.963+0000 7f2224490640 1 --2- 172.21.15.169:0/34517688 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f220006ae40 0x7f220006d300 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:46.967 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.963+0000 7f2224490640 1 --2- 172.21.15.169:0/34517688 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f221c1592d0 0x7f221c098de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:46.967 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.964+0000 7f2224490640 1 --2- 172.21.15.169:0/34517688 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f221c0cdc30 0x7f221c0988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:46.967 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.964+0000 7f2224490640 1 --2- 172.21.15.169:0/34517688 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f221c0ac930 0x7f221c06f260 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:46.967 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.964+0000 7f2224490640 1 -- 172.21.15.169:0/34517688 >> 172.21.15.169:0/34517688 conn(0x7f221c069930 msgr2=0x7f221c153bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:46.968 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.964+0000 7f2224490640 1 -- 172.21.15.169:0/34517688 shutdown_connections 2023-10-26T01:37:46.968 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:46.964+0000 7f2224490640 1 -- 172.21.15.169:0/34517688 wait complete. 2023-10-26T01:37:47.387 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 59, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-26T01:37:40.027858+0000', 'modified': '2023-10-26T01:37:43.103474+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7335}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7335': {'gid': 7335, 'name': 'c', 'rank': 0, 'incarnation': 55, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.169:6835/668588916', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.169:6834', 'nonce': 668588916}, {'type': 'v1', 'addr': '172.21.15.169:6835', 'nonce': 668588916}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-26T01:37:47.426 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-26T01:37:47.426 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-26T01:37:47.657 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.655+0000 7f0cb03bb640 1 -- 172.21.15.169:0/4100518783 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0ca815b620 msgr2=0x7f0ca80cd710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:47.657 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.655+0000 7f0cb03bb640 1 --2- 172.21.15.169:0/4100518783 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0ca815b620 0x7f0ca80cd710 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f0ca4005fd0 tx=0x7f0ca402be20 comp rx=0 tx=0).stop 2023-10-26T01:37:47.658 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.655+0000 7f0cb03bb640 1 -- 172.21.15.169:0/4100518783 shutdown_connections 2023-10-26T01:37:47.658 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.655+0000 7f0cb03bb640 1 --2- 172.21.15.169:0/4100518783 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0ca80aca30 0x7f0ca80cdc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:47.658 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.655+0000 7f0cb03bb640 1 --2- 172.21.15.169:0/4100518783 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0ca815b620 0x7f0ca80cd710 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:47.658 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.655+0000 7f0cb03bb640 1 -- 172.21.15.169:0/4100518783 >> 172.21.15.169:0/4100518783 conn(0x7f0ca8069930 msgr2=0x7f0ca8069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:47.659 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.655+0000 7f0cb03bb640 1 -- 172.21.15.169:0/4100518783 shutdown_connections 2023-10-26T01:37:47.659 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.655+0000 7f0cb03bb640 1 -- 172.21.15.169:0/4100518783 wait complete. 2023-10-26T01:37:47.660 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.657+0000 7f0cb03bb640 1 Processor -- start 2023-10-26T01:37:47.660 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.657+0000 7f0cb03bb640 1 -- start start 2023-10-26T01:37:47.660 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.658+0000 7f0cb03bb640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0ca80aca30 0x7f0ca80cb210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:47.661 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.658+0000 7f0cb03bb640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0ca80c9d00 0x7f0ca81f1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:47.661 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.658+0000 7f0cb03bb640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0ca815b620 0x7f0ca8070980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:47.661 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.658+0000 7f0cb03bb640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca80cb750 con 0x7f0ca815b620 2023-10-26T01:37:47.661 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.658+0000 7f0cb03bb640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca815eb40 con 0x7f0ca80aca30 2023-10-26T01:37:47.661 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.658+0000 7f0cb03bb640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca8143880 con 0x7f0ca80c9d00 2023-10-26T01:37:47.662 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.659+0000 7f0cae958640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0ca815b620 0x7f0ca8070980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:47.662 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.659+0000 7f0cae958640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0ca815b620 0x7f0ca8070980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:35712/0 (socket says 172.21.15.169:35712) 2023-10-26T01:37:47.662 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.659+0000 7f0cae958640 1 -- 172.21.15.169:0/969254735 learned_addr learned my addr 172.21.15.169:0/969254735 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:37:47.664 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.659+0000 7f0cad956640 1 --2- 172.21.15.169:0/969254735 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0ca80c9d00 0x7f0ca81f1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:47.664 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.659+0000 7f0cae157640 1 --2- 172.21.15.169:0/969254735 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0ca80aca30 0x7f0ca80cb210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:47.664 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.659+0000 7f0cae958640 1 -- 172.21.15.169:0/969254735 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0ca80c9d00 msgr2=0x7f0ca81f1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:47.664 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.659+0000 7f0cae958640 1 --2- 172.21.15.169:0/969254735 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0ca80c9d00 0x7f0ca81f1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:47.664 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.659+0000 7f0cae958640 1 -- 172.21.15.169:0/969254735 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0ca80aca30 msgr2=0x7f0ca80cb210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:47.665 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.659+0000 7f0cae958640 1 --2- 172.21.15.169:0/969254735 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0ca80aca30 0x7f0ca80cb210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:47.665 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.659+0000 7f0cae958640 1 -- 172.21.15.169:0/969254735 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ca4009560 con 0x7f0ca815b620 2023-10-26T01:37:47.665 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.659+0000 7f0cad956640 1 --2- 172.21.15.169:0/969254735 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0ca80c9d00 0x7f0ca81f1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:37:47.665 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.659+0000 7f0cae157640 1 --2- 172.21.15.169:0/969254735 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0ca80aca30 0x7f0ca80cb210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:37:47.665 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.659+0000 7f0cae958640 1 --2- 172.21.15.169:0/969254735 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0ca815b620 0x7f0ca8070980 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f0ca000eaa0 tx=0x7f0ca000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:47.665 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.660+0000 7f0c9effd640 1 -- 172.21.15.169:0/969254735 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ca000cdb0 con 0x7f0ca815b620 2023-10-26T01:37:47.666 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.660+0000 7f0c9effd640 1 -- 172.21.15.169:0/969254735 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0ca0004b20 con 0x7f0ca815b620 2023-10-26T01:37:47.666 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.660+0000 7f0cb03bb640 1 -- 172.21.15.169:0/969254735 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ca8143b60 con 0x7f0ca815b620 2023-10-26T01:37:47.666 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.660+0000 7f0c9effd640 1 -- 172.21.15.169:0/969254735 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ca0005020 con 0x7f0ca815b620 2023-10-26T01:37:47.666 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.660+0000 7f0cb03bb640 1 -- 172.21.15.169:0/969254735 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ca8144120 con 0x7f0ca815b620 2023-10-26T01:37:47.666 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.662+0000 7f0c9effd640 1 -- 172.21.15.169:0/969254735 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f0ca0005180 con 0x7f0ca815b620 2023-10-26T01:37:47.669 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.664+0000 7f0c9effd640 1 --2- 172.21.15.169:0/969254735 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f0c8c06ae40 0x7f0c8c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:47.669 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.664+0000 7f0c9effd640 1 -- 172.21.15.169:0/969254735 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0ca0089ff0 con 0x7f0ca815b620 2023-10-26T01:37:47.669 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.664+0000 7f0cae157640 1 --2- 172.21.15.169:0/969254735 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f0c8c06ae40 0x7f0c8c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:47.670 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.664+0000 7f0cb302a740 1 Processor -- start 2023-10-26T01:37:47.670 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.665+0000 7f0cb302a740 1 -- start start 2023-10-26T01:37:47.670 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.665+0000 7f0cae157640 1 --2- 172.21.15.169:0/969254735 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f0c8c06ae40 0x7f0c8c06d300 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f0ca4035000 tx=0x7f0ca4004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:47.670 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.665+0000 7f0cb302a740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557e482a67d0 0x557e48279a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:47.670 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.665+0000 7f0cb302a740 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x557e482a6bb0 0x557e482a9560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:47.671 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.665+0000 7f0cb302a740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557e48279f40 0x557e482a9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:47.671 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.666+0000 7f0cb302a740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e481ec740 con 0x557e48279f40 2023-10-26T01:37:47.671 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.666+0000 7f0cb302a740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e481e9e30 con 0x557e482a67d0 2023-10-26T01:37:47.671 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.666+0000 7f0cb302a740 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x557e481e5720 con 0x557e482a6bb0 2023-10-26T01:37:47.671 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.666+0000 7f0cae958640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557e48279f40 0x557e482a9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:47.672 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.666+0000 7f0cae958640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557e48279f40 0x557e482a9aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:35724/0 (socket says 172.21.15.169:35724) 2023-10-26T01:37:47.672 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.666+0000 7f0cae958640 1 -- 172.21.15.169:0/2584158400 learned_addr learned my addr 172.21.15.169:0/2584158400 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:37:47.672 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.666+0000 7f0cae157640 1 --2- 172.21.15.169:0/2584158400 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557e482a67d0 0x557e48279a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:47.672 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.666+0000 7f0cad956640 1 --2- 172.21.15.169:0/2584158400 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x557e482a6bb0 0x557e482a9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:47.672 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.666+0000 7f0cae958640 1 -- 172.21.15.169:0/2584158400 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x557e482a6bb0 msgr2=0x557e482a9560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T01:37:47.672 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.666+0000 7f0cae958640 1 --2- 172.21.15.169:0/2584158400 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x557e482a6bb0 0x557e482a9560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:47.673 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.666+0000 7f0cae958640 1 -- 172.21.15.169:0/2584158400 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557e482a67d0 msgr2=0x557e48279a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T01:37:47.673 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.666+0000 7f0cae958640 1 --2- 172.21.15.169:0/2584158400 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557e482a67d0 0x557e48279a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:47.673 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.666+0000 7f0cae958640 1 -- 172.21.15.169:0/2584158400 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557e481a9910 con 0x557e48279f40 2023-10-26T01:37:47.673 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.667+0000 7f0cae958640 1 --2- 172.21.15.169:0/2584158400 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557e48279f40 0x557e482a9aa0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f0ca0055260 tx=0x7f0ca00587c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=755c513eaa443831 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:47.673 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.667+0000 7f0c8bfff640 1 -- 172.21.15.169:0/2584158400 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ca005c050 con 0x557e48279f40 2023-10-26T01:37:47.673 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.667+0000 7f0c8bfff640 1 -- 172.21.15.169:0/2584158400 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0ca0088210 con 0x557e48279f40 2023-10-26T01:37:47.674 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.667+0000 7f0c8bfff640 1 -- 172.21.15.169:0/2584158400 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ca0061060 con 0x557e48279f40 2023-10-26T01:37:47.674 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.667+0000 7f0cb302a740 1 -- 172.21.15.169:0/2584158400 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557e48279f40 msgr2=0x557e482a9aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:47.674 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.667+0000 7f0cb302a740 1 --2- 172.21.15.169:0/2584158400 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557e48279f40 0x557e482a9aa0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f0ca0055260 tx=0x7f0ca00587c0 comp rx=0 tx=0).stop 2023-10-26T01:37:47.674 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.667+0000 7f0cb302a740 1 -- 172.21.15.169:0/2584158400 shutdown_connections 2023-10-26T01:37:47.674 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.667+0000 7f0cb302a740 1 --2- 172.21.15.169:0/2584158400 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557e48279f40 0x557e482a9aa0 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:47.675 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.667+0000 7f0cb302a740 1 --2- 172.21.15.169:0/2584158400 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x557e482a6bb0 0x557e482a9560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:47.675 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.667+0000 7f0cb302a740 1 --2- 172.21.15.169:0/2584158400 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557e482a67d0 0x557e48279a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:47.675 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.667+0000 7f0cb302a740 1 -- 172.21.15.169:0/2584158400 >> 172.21.15.169:0/2584158400 conn(0x557e4827bc70 msgr2=0x557e482c4560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:47.675 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.668+0000 7f0cb302a740 1 -- 172.21.15.169:0/2584158400 shutdown_connections 2023-10-26T01:37:47.675 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.668+0000 7f0cb302a740 1 -- 172.21.15.169:0/2584158400 wait complete. 2023-10-26T01:37:47.679 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.668+0000 7f0cb302a740 1 -- start start 2023-10-26T01:37:47.679 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.669+0000 7f0cb302a740 1 Processor -- start 2023-10-26T01:37:47.679 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.669+0000 7f0cb302a740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557e482a67d0 0x557e482a3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:47.679 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.670+0000 7f0cb302a740 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x557e482a6bb0 0x557e482ade90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:47.680 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.670+0000 7f0cb302a740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557e48296db0 0x557e482ae3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:47.680 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.670+0000 7f0cb302a740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e48183970 con 0x557e482a67d0 2023-10-26T01:37:47.680 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.670+0000 7f0cb302a740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e48183610 con 0x557e48296db0 2023-10-26T01:37:47.680 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.670+0000 7f0cb302a740 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x557e47f49f20 con 0x557e482a6bb0 2023-10-26T01:37:47.680 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.670+0000 7f0cb302a740 10 client.0 _ll_register_callbacks cb 0x557e481a70c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T01:37:47.681 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.670+0000 7f0cae157640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557e482a67d0 0x557e482a3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:47.681 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.670+0000 7f0cae157640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557e482a67d0 0x557e482a3430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:35738/0 (socket says 172.21.15.169:35738) 2023-10-26T01:37:47.681 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.670+0000 7f0cae157640 1 -- 172.21.15.169:0/4035010817 learned_addr learned my addr 172.21.15.169:0/4035010817 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:37:47.681 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.670+0000 7f0cae157640 1 -- 172.21.15.169:0/4035010817 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x557e482a6bb0 msgr2=0x557e482ade90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T01:37:47.681 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.670+0000 7f0cae157640 1 --2- 172.21.15.169:0/4035010817 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x557e482a6bb0 0x557e482ade90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:47.681 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.670+0000 7f0cae958640 1 --2- 172.21.15.169:0/4035010817 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557e48296db0 0x557e482ae3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:47.682 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.670+0000 7f0cae157640 1 -- 172.21.15.169:0/4035010817 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557e48296db0 msgr2=0x557e482ae3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T01:37:47.682 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.670+0000 7f0cae157640 1 --2- 172.21.15.169:0/4035010817 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557e48296db0 0x557e482ae3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:47.682 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.670+0000 7f0cae157640 1 -- 172.21.15.169:0/4035010817 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ca8144120 con 0x557e482a67d0 2023-10-26T01:37:47.682 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.671+0000 7f0cae157640 1 --2- 172.21.15.169:0/4035010817 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557e482a67d0 0x557e482a3430 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f0ca4000c00 tx=0x7f0ca4009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97d9329e09b1e2f9 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:47.682 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.671+0000 7f0c897fa640 10 client.0 ms_handle_connect on v2:172.21.15.169:3300/0 2023-10-26T01:37:47.683 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.671+0000 7f0c897fa640 1 -- 172.21.15.169:0/4035010817 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ca4003ac0 con 0x557e482a67d0 2023-10-26T01:37:47.683 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.671+0000 7f0c897fa640 1 -- 172.21.15.169:0/4035010817 <== mon.0 v2:172.21.15.169:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0ca4003c20 con 0x557e482a67d0 2023-10-26T01:37:47.683 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.671+0000 7f0cb302a740 1 -- 172.21.15.169:0/4035010817 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557e481de700 con 0x557e482a67d0 2023-10-26T01:37:47.683 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.671+0000 7f0c897fa640 1 -- 172.21.15.169:0/4035010817 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0ca4003e20 con 0x557e482a67d0 2023-10-26T01:37:47.683 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.671+0000 7f0cb302a740 10 client.7494 fetch_fsmap learned FSMap version 59 2023-10-26T01:37:47.684 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.671+0000 7f0cb302a740 1 -- 172.21.15.169:0/4035010817 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=2+}) v3 -- 0x557e481e3da0 con 0x557e482a67d0 2023-10-26T01:37:47.684 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.671+0000 7f0c897fa640 1 -- 172.21.15.169:0/4035010817 <== mon.0 v2:172.21.15.169:3300/0 4 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x557e481de700 con 0x557e482a67d0 2023-10-26T01:37:47.684 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.671+0000 7f0c897fa640 1 -- 172.21.15.169:0/4035010817 <== mon.0 v2:172.21.15.169:3300/0 5 ==== fsmap(e 59) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f0ca403c040 con 0x557e482a67d0 2023-10-26T01:37:47.684 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.672+0000 7f0cb302a740 10 client.7494 fetch_fsmap finished waiting for FSMap version 59 2023-10-26T01:37:47.684 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.672+0000 7f0cb302a740 10 client.7494 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-26T01:37:47.685 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.672+0000 7f0cb302a740 20 client.7494 populate_metadata read hostname 'smithi169' 2023-10-26T01:37:47.685 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.672+0000 7f0cb302a740 1 --2- 172.21.15.169:0/4035010817 >> [v2:172.21.15.169:6834/668588916,v1:172.21.15.169:6835/668588916] conn(0x557e482d8940 0x557e482abc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:47.685 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.672+0000 7f0cb302a740 4 client.7494 mds_command: new command op to 7335 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-26T01:37:47.685 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.672+0000 7f0cb302a740 1 -- 172.21.15.169:0/4035010817 --> [v2:172.21.15.169:6834/668588916,v1:172.21.15.169:6835/668588916] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557e481de420 con 0x557e482d8940 2023-10-26T01:37:47.685 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.672+0000 7f0cad956640 1 --2- 172.21.15.169:0/4035010817 >> [v2:172.21.15.169:6834/668588916,v1:172.21.15.169:6835/668588916] conn(0x557e482d8940 0x557e482abc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:47.686 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.672+0000 7f0cad956640 1 --2- 172.21.15.169:0/4035010817 >> [v2:172.21.15.169:6834/668588916,v1:172.21.15.169:6835/668588916] conn(0x557e482d8940 0x557e482abc60 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2bb8d57438a8bfad server_cookie=df0f0ed5aef54127 in_seq=0 out_seq=0 2023-10-26T01:37:47.686 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.672+0000 7f0c897fa640 10 client.7494 ms_handle_connect on v2:172.21.15.169:6834/668588916 2023-10-26T01:37:47.686 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.673+0000 7f0c897fa640 1 -- 172.21.15.169:0/4035010817 <== mds.0 v2:172.21.15.169:6834/668588916 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x557e481de420 con 0x557e482d8940 2023-10-26T01:37:47.686 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.673+0000 7f0c897fa640 10 client.7494 handle_command_reply: tid=0 2023-10-26T01:37:47.686 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.673+0000 7f0cb302a740 1 client.7494 shutdown 2023-10-26T01:37:47.687 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.674+0000 7f0cb302a740 1 -- 172.21.15.169:0/4035010817 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557e482a67d0 msgr2=0x557e482a3430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:47.687 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.674+0000 7f0cb302a740 1 --2- 172.21.15.169:0/4035010817 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557e482a67d0 0x557e482a3430 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f0ca4000c00 tx=0x7f0ca4009260 comp rx=0 tx=0).stop 2023-10-26T01:37:47.687 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.674+0000 7f0cb302a740 1 -- 172.21.15.169:0/4035010817 shutdown_connections 2023-10-26T01:37:47.687 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.674+0000 7f0cb302a740 1 --2- 172.21.15.169:0/4035010817 >> [v2:172.21.15.169:6834/668588916,v1:172.21.15.169:6835/668588916] conn(0x557e482d8940 0x557e482abc60 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:47.687 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.674+0000 7f0cb302a740 1 --2- 172.21.15.169:0/4035010817 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557e48296db0 0x557e482ae3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:47.688 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.674+0000 7f0cb302a740 1 --2- 172.21.15.169:0/4035010817 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x557e482a6bb0 0x557e482ade90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:47.688 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.674+0000 7f0cb302a740 1 --2- 172.21.15.169:0/4035010817 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557e482a67d0 0x557e482a3430 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:47.688 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.674+0000 7f0c897fa640 0 client.7494 ms_handle_reset on v2:172.21.15.169:6834/668588916 2023-10-26T01:37:47.688 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.674+0000 7f0cb302a740 1 -- 172.21.15.169:0/4035010817 >> 172.21.15.169:0/4035010817 conn(0x557e48279f40 msgr2=0x557e4827d650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:47.688 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.674+0000 7f0cb302a740 1 -- 172.21.15.169:0/4035010817 shutdown_connections 2023-10-26T01:37:47.688 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.674+0000 7f0cb302a740 1 -- 172.21.15.169:0/4035010817 wait complete. 2023-10-26T01:37:47.689 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.674+0000 7f0cb302a740 20 client.7494 trim_cache size 0 max 16384 2023-10-26T01:37:47.692 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.690+0000 7f0cb302a740 1 Processor -- start 2023-10-26T01:37:47.692 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.691+0000 7f0cb302a740 1 -- start start 2023-10-26T01:37:47.693 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.691+0000 7f0cb302a740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557e48296db0 0x557e482af7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:47.693 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.691+0000 7f0cb302a740 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x557e482a67d0 0x557e482ae350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:47.693 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.691+0000 7f0cb302a740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557e482a6bb0 0x557e482a2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:47.693 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.691+0000 7f0cb302a740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e482c5570 con 0x557e482a6bb0 2023-10-26T01:37:47.694 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.691+0000 7f0cb302a740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e482d46d0 con 0x557e48296db0 2023-10-26T01:37:47.694 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.691+0000 7f0cb302a740 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x557e48280800 con 0x557e482a67d0 2023-10-26T01:37:47.695 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.692+0000 7f0cae958640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557e482a6bb0 0x557e482a2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:47.696 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.692+0000 7f0cae958640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557e482a6bb0 0x557e482a2730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:35742/0 (socket says 172.21.15.169:35742) 2023-10-26T01:37:47.696 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.692+0000 7f0cae958640 1 -- 172.21.15.169:0/3801563491 learned_addr learned my addr 172.21.15.169:0/3801563491 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:37:47.696 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.692+0000 7f0cae958640 1 -- 172.21.15.169:0/3801563491 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x557e482a67d0 msgr2=0x557e482ae350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T01:37:47.696 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.692+0000 7f0cae958640 1 --2- 172.21.15.169:0/3801563491 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x557e482a67d0 0x557e482ae350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:47.696 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.692+0000 7f0cae958640 1 -- 172.21.15.169:0/3801563491 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557e48296db0 msgr2=0x557e482af7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T01:37:47.697 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.692+0000 7f0cae958640 1 --2- 172.21.15.169:0/3801563491 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557e48296db0 0x557e482af7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:47.697 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.692+0000 7f0cae958640 1 -- 172.21.15.169:0/3801563491 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557e48182b80 con 0x557e482a6bb0 2023-10-26T01:37:47.697 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.692+0000 7f0cae958640 1 --2- 172.21.15.169:0/3801563491 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557e482a6bb0 0x557e482a2730 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f0ca0009270 tx=0x7f0ca001c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea894f738f5cd499 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:47.697 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.693+0000 7f0c9ce00640 1 -- 172.21.15.169:0/3801563491 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ca0061060 con 0x557e482a6bb0 2023-10-26T01:37:47.697 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.693+0000 7f0c9ce00640 1 -- 172.21.15.169:0/3801563491 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0ca0057100 con 0x557e482a6bb0 2023-10-26T01:37:47.698 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.693+0000 7f0c9ce00640 1 -- 172.21.15.169:0/3801563491 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ca0060bd0 con 0x557e482a6bb0 2023-10-26T01:37:47.698 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.693+0000 7f0cb302a740 1 -- 172.21.15.169:0/3801563491 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557e482a6bb0 msgr2=0x557e482a2730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:47.698 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.693+0000 7f0cb302a740 1 --2- 172.21.15.169:0/3801563491 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557e482a6bb0 0x557e482a2730 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f0ca0009270 tx=0x7f0ca001c040 comp rx=0 tx=0).stop 2023-10-26T01:37:47.698 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.693+0000 7f0cb302a740 1 -- 172.21.15.169:0/3801563491 shutdown_connections 2023-10-26T01:37:47.698 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.693+0000 7f0cb302a740 1 --2- 172.21.15.169:0/3801563491 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557e482a6bb0 0x557e482a2730 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:47.699 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.693+0000 7f0cb302a740 1 --2- 172.21.15.169:0/3801563491 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x557e482a67d0 0x557e482ae350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:47.699 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.693+0000 7f0cb302a740 1 --2- 172.21.15.169:0/3801563491 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557e48296db0 0x557e482af7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:47.699 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.693+0000 7f0cb302a740 1 -- 172.21.15.169:0/3801563491 >> 172.21.15.169:0/3801563491 conn(0x557e48279f40 msgr2=0x557e482ad130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:47.699 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.693+0000 7f0cb302a740 1 -- 172.21.15.169:0/3801563491 shutdown_connections 2023-10-26T01:37:47.699 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.693+0000 7f0cb302a740 1 -- 172.21.15.169:0/3801563491 wait complete. 2023-10-26T01:37:47.699 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.694+0000 7f0cb302a740 1 -- start start 2023-10-26T01:37:47.700 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.694+0000 7f0cb302a740 1 Processor -- start 2023-10-26T01:37:47.703 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.695+0000 7f0cb302a740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557e48296db0 0x557e482c7c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:47.703 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.695+0000 7f0cb302a740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557e482a67d0 0x557e48278a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:47.703 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.695+0000 7f0cb302a740 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x557e482a6bb0 0x557e48278f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:47.703 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.695+0000 7f0cb302a740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e482a5460 con 0x557e482a67d0 2023-10-26T01:37:47.703 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.695+0000 7f0cb302a740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e482a9e90 con 0x557e48296db0 2023-10-26T01:37:47.704 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.695+0000 7f0cb302a740 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x557e482d3df0 con 0x557e482a6bb0 2023-10-26T01:37:47.704 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.695+0000 7f0cb302a740 10 client.0 _ll_register_callbacks cb 0x7f0ca00550a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T01:37:47.704 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.695+0000 7f0cad956640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557e482a67d0 0x557e48278a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:47.704 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.696+0000 7f0cad956640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557e482a67d0 0x557e48278a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:35744/0 (socket says 172.21.15.169:35744) 2023-10-26T01:37:47.704 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.696+0000 7f0cad956640 1 -- 172.21.15.169:0/1596499129 learned_addr learned my addr 172.21.15.169:0/1596499129 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:37:47.705 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.696+0000 7f0cae157640 1 --2- 172.21.15.169:0/1596499129 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557e48296db0 0x557e482c7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:47.705 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.696+0000 7f0cad956640 1 -- 172.21.15.169:0/1596499129 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x557e482a6bb0 msgr2=0x557e48278f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T01:37:47.705 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.696+0000 7f0cad956640 1 --2- 172.21.15.169:0/1596499129 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x557e482a6bb0 0x557e48278f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:47.705 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.696+0000 7f0cad956640 1 -- 172.21.15.169:0/1596499129 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557e48296db0 msgr2=0x557e482c7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T01:37:47.705 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.696+0000 7f0cad956640 1 --2- 172.21.15.169:0/1596499129 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557e48296db0 0x557e482c7c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:47.705 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.696+0000 7f0cad956640 1 -- 172.21.15.169:0/1596499129 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557e481a9910 con 0x557e482a67d0 2023-10-26T01:37:47.706 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.696+0000 7f0cad956640 1 --2- 172.21.15.169:0/1596499129 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557e482a67d0 0x557e48278a00 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f0ca8147790 tx=0x7f0c98015dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a47d2b7999707389 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:47.706 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.696+0000 7f0c897fa640 10 client.0 ms_handle_connect on v2:172.21.15.169:3300/0 2023-10-26T01:37:47.706 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.696+0000 7f0c897fa640 1 -- 172.21.15.169:0/1596499129 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c9800c040 con 0x557e482a67d0 2023-10-26T01:37:47.706 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.696+0000 7f0c897fa640 1 -- 172.21.15.169:0/1596499129 <== mon.0 v2:172.21.15.169:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0c980035f0 con 0x557e482a67d0 2023-10-26T01:37:47.706 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.696+0000 7f0cb302a740 1 -- 172.21.15.169:0/1596499129 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0ca0088210 con 0x557e482a67d0 2023-10-26T01:37:47.707 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.697+0000 7f0c897fa640 1 -- 172.21.15.169:0/1596499129 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0c98001d60 con 0x557e482a67d0 2023-10-26T01:37:47.707 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.697+0000 7f0cb302a740 10 client.7500 fetch_fsmap learned FSMap version 59 2023-10-26T01:37:47.707 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.697+0000 7f0cb302a740 1 -- 172.21.15.169:0/1596499129 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=2+}) v3 -- 0x557e481a8170 con 0x557e482a67d0 2023-10-26T01:37:47.707 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.697+0000 7f0c897fa640 1 -- 172.21.15.169:0/1596499129 <== mon.0 v2:172.21.15.169:3300/0 4 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0ca0088210 con 0x557e482a67d0 2023-10-26T01:37:47.707 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.697+0000 7f0c897fa640 1 -- 172.21.15.169:0/1596499129 <== mon.0 v2:172.21.15.169:3300/0 5 ==== fsmap(e 59) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f0c98012040 con 0x557e482a67d0 2023-10-26T01:37:47.707 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.697+0000 7f0cb302a740 10 client.7500 fetch_fsmap finished waiting for FSMap version 59 2023-10-26T01:37:47.708 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.697+0000 7f0cb302a740 10 client.7500 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-26T01:37:47.708 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.697+0000 7f0cb302a740 20 client.7500 populate_metadata read hostname 'smithi169' 2023-10-26T01:37:47.708 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.697+0000 7f0cb302a740 1 --2- 172.21.15.169:0/1596499129 >> [v2:172.21.15.169:6834/668588916,v1:172.21.15.169:6835/668588916] conn(0x557e482d30b0 0x557e482d3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:47.708 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.697+0000 7f0cb302a740 4 client.7500 mds_command: new command op to 7335 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-26T01:37:47.708 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.697+0000 7f0cb302a740 1 -- 172.21.15.169:0/1596499129 --> [v2:172.21.15.169:6834/668588916,v1:172.21.15.169:6835/668588916] -- command(tid 0: {"prefix": "status"}) v1 -- 0x557e47ee76c0 con 0x557e482d30b0 2023-10-26T01:37:47.709 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.697+0000 7f0cae157640 1 --2- 172.21.15.169:0/1596499129 >> [v2:172.21.15.169:6834/668588916,v1:172.21.15.169:6835/668588916] conn(0x557e482d30b0 0x557e482d3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:47.709 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.698+0000 7f0cae157640 1 --2- 172.21.15.169:0/1596499129 >> [v2:172.21.15.169:6834/668588916,v1:172.21.15.169:6835/668588916] conn(0x557e482d30b0 0x557e482d3490 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d94fb86c1c6e4443 server_cookie=98d71b814143a041 in_seq=0 out_seq=0 2023-10-26T01:37:47.709 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.698+0000 7f0c897fa640 10 client.7500 ms_handle_connect on v2:172.21.15.169:6834/668588916 2023-10-26T01:37:47.709 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.698+0000 7f0c897fa640 1 -- 172.21.15.169:0/1596499129 <== mds.0 v2:172.21.15.169:6834/668588916 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x557e47ee76c0 con 0x557e482d30b0 2023-10-26T01:37:47.709 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.698+0000 7f0c897fa640 10 client.7500 handle_command_reply: tid=0 2023-10-26T01:37:47.710 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.698+0000 7f0cb302a740 1 client.7500 shutdown 2023-10-26T01:37:47.710 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.698+0000 7f0cb302a740 1 -- 172.21.15.169:0/1596499129 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557e482a67d0 msgr2=0x557e48278a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:47.710 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.698+0000 7f0cb302a740 1 --2- 172.21.15.169:0/1596499129 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557e482a67d0 0x557e48278a00 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f0ca8147790 tx=0x7f0c98015dd0 comp rx=0 tx=0).stop 2023-10-26T01:37:47.710 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.698+0000 7f0cb302a740 1 -- 172.21.15.169:0/1596499129 shutdown_connections 2023-10-26T01:37:47.710 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.698+0000 7f0cb302a740 1 --2- 172.21.15.169:0/1596499129 >> [v2:172.21.15.169:6834/668588916,v1:172.21.15.169:6835/668588916] conn(0x557e482d30b0 0x557e482d3490 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:47.711 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.698+0000 7f0cb302a740 1 --2- 172.21.15.169:0/1596499129 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557e482a67d0 0x557e48278a00 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:47.711 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.698+0000 7f0cb302a740 1 --2- 172.21.15.169:0/1596499129 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x557e482a6bb0 0x557e48278f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:47.711 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.698+0000 7f0cb302a740 1 --2- 172.21.15.169:0/1596499129 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557e48296db0 0x557e482c7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:47.711 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.698+0000 7f0c897fa640 0 client.7500 ms_handle_reset on v2:172.21.15.169:6834/668588916 2023-10-26T01:37:47.711 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.698+0000 7f0cb302a740 1 -- 172.21.15.169:0/1596499129 >> 172.21.15.169:0/1596499129 conn(0x557e48279f40 msgr2=0x557e4827f510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:47.711 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.699+0000 7f0cb302a740 1 -- 172.21.15.169:0/1596499129 shutdown_connections 2023-10-26T01:37:47.712 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.699+0000 7f0cb302a740 1 -- 172.21.15.169:0/1596499129 wait complete. 2023-10-26T01:37:47.712 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.699+0000 7f0cb302a740 20 client.7500 trim_cache size 0 max 16384 2023-10-26T01:37:47.712 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2023-10-26T01:37:47.712 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_fsid": "30203b28-ed8a-44e8-a9a6-a9bf25f980f1", 2023-10-26T01:37:47.713 INFO:teuthology.orchestra.run.smithi169.stdout: "whoami": 0, 2023-10-26T01:37:47.713 INFO:teuthology.orchestra.run.smithi169.stdout: "id": 7335, 2023-10-26T01:37:47.713 INFO:teuthology.orchestra.run.smithi169.stdout: "want_state": "up:active", 2023-10-26T01:37:47.713 INFO:teuthology.orchestra.run.smithi169.stdout: "state": "up:active", 2023-10-26T01:37:47.713 INFO:teuthology.orchestra.run.smithi169.stdout: "fs_name": "cephfs", 2023-10-26T01:37:47.714 INFO:teuthology.orchestra.run.smithi169.stdout: "rank_uptime": 7.1150629399999996, 2023-10-26T01:37:47.714 INFO:teuthology.orchestra.run.smithi169.stdout: "mdsmap_epoch": 59, 2023-10-26T01:37:47.714 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_epoch": 120, 2023-10-26T01:37:47.714 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_epoch_barrier": 120, 2023-10-26T01:37:47.714 INFO:teuthology.orchestra.run.smithi169.stdout: "uptime": 13.096039378 2023-10-26T01:37:47.715 INFO:teuthology.orchestra.run.smithi169.stdout:} 2023-10-26T01:37:47.715 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.699+0000 7f0cb03bb640 1 -- 172.21.15.169:0/969254735 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f0c8c06ae40 msgr2=0x7f0c8c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:47.716 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.699+0000 7f0cb03bb640 1 --2- 172.21.15.169:0/969254735 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f0c8c06ae40 0x7f0c8c06d300 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f0ca4035000 tx=0x7f0ca4004730 comp rx=0 tx=0).stop 2023-10-26T01:37:47.716 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.699+0000 7f0cb03bb640 1 -- 172.21.15.169:0/969254735 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0ca815b620 msgr2=0x7f0ca8070980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:47.716 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.699+0000 7f0cb03bb640 1 --2- 172.21.15.169:0/969254735 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0ca815b620 0x7f0ca8070980 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f0ca000eaa0 tx=0x7f0ca000ef60 comp rx=0 tx=0).stop 2023-10-26T01:37:47.716 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.699+0000 7f0cb03bb640 1 -- 172.21.15.169:0/969254735 shutdown_connections 2023-10-26T01:37:47.717 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.699+0000 7f0cb03bb640 1 --2- 172.21.15.169:0/969254735 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f0c8c06ae40 0x7f0c8c06d300 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:47.717 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.699+0000 7f0cb03bb640 1 --2- 172.21.15.169:0/969254735 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0ca815b620 0x7f0ca8070980 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:47.717 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.699+0000 7f0cb03bb640 1 --2- 172.21.15.169:0/969254735 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0ca80c9d00 0x7f0ca81f1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:47.717 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.699+0000 7f0cb03bb640 1 --2- 172.21.15.169:0/969254735 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0ca80aca30 0x7f0ca80cb210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:47.717 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.699+0000 7f0cb03bb640 1 -- 172.21.15.169:0/969254735 >> 172.21.15.169:0/969254735 conn(0x7f0ca8069930 msgr2=0x7f0ca80aa430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:47.718 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.700+0000 7f0cb03bb640 1 -- 172.21.15.169:0/969254735 shutdown_connections 2023-10-26T01:37:47.718 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.700+0000 7f0cb03bb640 1 -- 172.21.15.169:0/969254735 wait complete. 2023-10-26T01:37:47.719 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-26T01:37:47.930 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.928+0000 7fdc239e2640 1 -- 172.21.15.169:0/1070670666 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdc080047c0 msgr2=0x7fdc08004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:47.930 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.928+0000 7fdc239e2640 1 --2- 172.21.15.169:0/1070670666 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdc080047c0 0x7fdc08004c40 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c1420d0 tx=0x7fdc080057f0 comp rx=0 tx=0).stop 2023-10-26T01:37:47.931 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.928+0000 7fdc21f7f640 1 -- 172.21.15.169:0/1070670666 reap_dead start 2023-10-26T01:37:47.931 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.928+0000 7fdc239e2640 1 -- 172.21.15.169:0/1070670666 shutdown_connections 2023-10-26T01:37:47.931 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.928+0000 7fdc239e2640 1 -- 172.21.15.169:0/1070670666 >> 172.21.15.169:0/1070670666 conn(0x7fdc1c069930 msgr2=0x7fdc1c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:47.931 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.928+0000 7fdc239e2640 1 -- 172.21.15.169:0/1070670666 shutdown_connections 2023-10-26T01:37:47.931 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.928+0000 7fdc239e2640 1 -- 172.21.15.169:0/1070670666 wait complete. 2023-10-26T01:37:47.932 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.930+0000 7fdc239e2640 1 Processor -- start 2023-10-26T01:37:47.932 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.930+0000 7fdc239e2640 1 -- start start 2023-10-26T01:37:47.933 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.931+0000 7fdc239e2640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdc080047c0 0x7fdc1c1550f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:47.934 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.931+0000 7fdc239e2640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdc1c1f07e0 0x7fdc1c070ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:47.934 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.931+0000 7fdc239e2640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdc1c0cb5b0 0x7fdc1c0cba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:47.934 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.931+0000 7fdc239e2640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1c0cbf70 con 0x7fdc080047c0 2023-10-26T01:37:47.934 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.931+0000 7fdc239e2640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1c1421d0 con 0x7fdc1c0cb5b0 2023-10-26T01:37:47.934 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.931+0000 7fdc239e2640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1c071360 con 0x7fdc1c1f07e0 2023-10-26T01:37:47.935 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.931+0000 7fdc2177e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdc080047c0 0x7fdc1c1550f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:47.935 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.932+0000 7fdc2177e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdc080047c0 0x7fdc1c1550f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:35756/0 (socket says 172.21.15.169:35756) 2023-10-26T01:37:47.935 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.932+0000 7fdc2177e640 1 -- 172.21.15.169:0/3752197049 learned_addr learned my addr 172.21.15.169:0/3752197049 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:37:47.935 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.932+0000 7fdc2177e640 1 -- 172.21.15.169:0/3752197049 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdc1c1f07e0 msgr2=0x7fdc1c070ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:47.935 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.932+0000 7fdc2177e640 1 --2- 172.21.15.169:0/3752197049 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdc1c1f07e0 0x7fdc1c070ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:47.936 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.932+0000 7fdc21f7f640 1 --2- 172.21.15.169:0/3752197049 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdc1c0cb5b0 0x7fdc1c0cba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:47.936 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.932+0000 7fdc2177e640 1 -- 172.21.15.169:0/3752197049 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdc1c0cb5b0 msgr2=0x7fdc1c0cba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:47.936 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.932+0000 7fdc2177e640 1 --2- 172.21.15.169:0/3752197049 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdc1c0cb5b0 0x7fdc1c0cba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:47.936 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.932+0000 7fdc2177e640 1 -- 172.21.15.169:0/3752197049 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc1c071640 con 0x7fdc080047c0 2023-10-26T01:37:47.937 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.932+0000 7fdc2177e640 1 --2- 172.21.15.169:0/3752197049 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdc080047c0 0x7fdc1c1550f0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7fdc18010a40 tx=0x7fdc18010f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:47.938 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.933+0000 7fdc11ffb640 1 -- 172.21.15.169:0/3752197049 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc180096a0 con 0x7fdc080047c0 2023-10-26T01:37:47.938 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.933+0000 7fdc11ffb640 1 -- 172.21.15.169:0/3752197049 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fdc1800ed20 con 0x7fdc080047c0 2023-10-26T01:37:47.938 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.933+0000 7fdc11ffb640 1 -- 172.21.15.169:0/3752197049 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc18005280 con 0x7fdc080047c0 2023-10-26T01:37:47.938 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.933+0000 7fdc239e2640 1 -- 172.21.15.169:0/3752197049 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc1c071920 con 0x7fdc080047c0 2023-10-26T01:37:47.938 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.933+0000 7fdc239e2640 1 -- 172.21.15.169:0/3752197049 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc1c071000 con 0x7fdc080047c0 2023-10-26T01:37:47.939 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.934+0000 7fdc239e2640 1 -- 172.21.15.169:0/3752197049 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc1c04db10 con 0x7fdc080047c0 2023-10-26T01:37:47.939 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.935+0000 7fdc11ffb640 1 -- 172.21.15.169:0/3752197049 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fdc180053e0 con 0x7fdc080047c0 2023-10-26T01:37:47.939 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.936+0000 7fdc11ffb640 1 --2- 172.21.15.169:0/3752197049 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fdc0006ae40 0x7fdc0006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:47.939 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.936+0000 7fdc11ffb640 1 -- 172.21.15.169:0/3752197049 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdc18016070 con 0x7fdc080047c0 2023-10-26T01:37:47.940 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.936+0000 7fdc20f7d640 1 --2- 172.21.15.169:0/3752197049 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fdc0006ae40 0x7fdc0006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:47.940 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.937+0000 7fdc20f7d640 1 --2- 172.21.15.169:0/3752197049 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fdc0006ae40 0x7fdc0006d300 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c034000 tx=0x7fdc0c001930 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:47.941 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:47.940+0000 7fdc11ffb640 1 -- 172.21.15.169:0/3752197049 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdc180553d0 con 0x7fdc080047c0 2023-10-26T01:37:48.169 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:48.167+0000 7fdc239e2640 1 -- 172.21.15.169:0/3752197049 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fdc1c06bad0 con 0x7fdc080047c0 2023-10-26T01:37:48.170 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:48.168+0000 7fdc11ffb640 1 -- 172.21.15.169:0/3752197049 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v59) v1 ==== 53+0+73 (secure 0 0 0) 0x7fdc180589f0 con 0x7fdc080047c0 2023-10-26T01:37:48.170 INFO:teuthology.orchestra.run.smithi169.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-26T01:37:48.174 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:48.171+0000 7fdc239e2640 1 -- 172.21.15.169:0/3752197049 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fdc0006ae40 msgr2=0x7fdc0006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:48.174 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:48.171+0000 7fdc239e2640 1 --2- 172.21.15.169:0/3752197049 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fdc0006ae40 0x7fdc0006d300 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c034000 tx=0x7fdc0c001930 comp rx=0 tx=0).stop 2023-10-26T01:37:48.174 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:48.171+0000 7fdc239e2640 1 -- 172.21.15.169:0/3752197049 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdc080047c0 msgr2=0x7fdc1c1550f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:48.175 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:48.171+0000 7fdc239e2640 1 --2- 172.21.15.169:0/3752197049 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdc080047c0 0x7fdc1c1550f0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7fdc18010a40 tx=0x7fdc18010f00 comp rx=0 tx=0).stop 2023-10-26T01:37:48.175 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:48.171+0000 7fdc239e2640 1 -- 172.21.15.169:0/3752197049 shutdown_connections 2023-10-26T01:37:48.175 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:48.171+0000 7fdc239e2640 1 --2- 172.21.15.169:0/3752197049 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fdc0006ae40 0x7fdc0006d300 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:48.175 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:48.171+0000 7fdc239e2640 1 --2- 172.21.15.169:0/3752197049 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdc1c0cb5b0 0x7fdc1c0cba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:48.175 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:48.171+0000 7fdc239e2640 1 --2- 172.21.15.169:0/3752197049 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdc1c1f07e0 0x7fdc1c070ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:48.176 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:48.171+0000 7fdc239e2640 1 --2- 172.21.15.169:0/3752197049 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdc080047c0 0x7fdc1c1550f0 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:48.176 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:48.171+0000 7fdc239e2640 1 -- 172.21.15.169:0/3752197049 >> 172.21.15.169:0/3752197049 conn(0x7fdc1c069930 msgr2=0x7fdc1c156310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:48.176 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:48.171+0000 7fdc239e2640 1 -- 172.21.15.169:0/3752197049 shutdown_connections 2023-10-26T01:37:48.176 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:48.171+0000 7fdc239e2640 1 -- 172.21.15.169:0/3752197049 wait complete. 2023-10-26T01:37:48.192 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-26T01:37:48.192 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-26T01:37:48.193 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi169.front.sepia.ceph.com 2023-10-26T01:37:48.193 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-26T01:37:48.193 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-26T01:37:48.194 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-26T01:37:48.194 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-26T01:37:48.195 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-26T01:37:48.195 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-26T01:37:48.196 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:37:48.196 DEBUG:teuthology.orchestra.run.smithi169:> ip addr 2023-10-26T01:37:48.251 INFO:teuthology.orchestra.run.smithi169.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-26T01:37:48.251 INFO:teuthology.orchestra.run.smithi169.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-26T01:37:48.251 INFO:teuthology.orchestra.run.smithi169.stdout: inet 127.0.0.1/8 scope host lo 2023-10-26T01:37:48.252 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:37:48.252 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 ::1/128 scope host 2023-10-26T01:37:48.252 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:37:48.252 INFO:teuthology.orchestra.run.smithi169.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-26T01:37:48.252 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:d9:3b:b4 brd ff:ff:ff:ff:ff:ff 2023-10-26T01:37:48.253 INFO:teuthology.orchestra.run.smithi169.stdout: altname enp7s0f0 2023-10-26T01:37:48.253 INFO:teuthology.orchestra.run.smithi169.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-26T01:37:48.253 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:d9:3b:b5 brd ff:ff:ff:ff:ff:ff 2023-10-26T01:37:48.253 INFO:teuthology.orchestra.run.smithi169.stdout: altname enp7s0f1 2023-10-26T01:37:48.253 INFO:teuthology.orchestra.run.smithi169.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-26T01:37:48.254 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:88:7f:4c brd ff:ff:ff:ff:ff:ff 2023-10-26T01:37:48.254 INFO:teuthology.orchestra.run.smithi169.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-26T01:37:48.254 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:88:7f:4d brd ff:ff:ff:ff:ff:ff 2023-10-26T01:37:48.254 INFO:teuthology.orchestra.run.smithi169.stdout: inet 172.21.15.169/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-26T01:37:48.254 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft 42027sec preferred_lft 42027sec 2023-10-26T01:37:48.255 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 fe80::ec4:7aff:fe88:7f4d/64 scope link 2023-10-26T01:37:48.255 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:37:48.255 INFO:teuthology.orchestra.run.smithi169.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-26T01:37:48.255 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 22:a0:6b:43:7b:04 brd ff:ff:ff:ff:ff:ff 2023-10-26T01:37:48.255 INFO:teuthology.orchestra.run.smithi169.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-26T01:37:48.256 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:37:48.256 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 fe80::d815:31ff:fe87:d148/64 scope link 2023-10-26T01:37:48.256 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:37:48.256 INFO:teuthology.orchestra.run.smithi169.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-26T01:37:48.257 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 2a:80:f8:45:5f:9f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-26T01:37:48.257 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 fe80::2880:f8ff:fe45:5f9f/64 scope link 2023-10-26T01:37:48.257 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:37:48.257 INFO:teuthology.orchestra.run.smithi169.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-26T01:37:48.258 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 22:a0:6b:43:7b:04 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-26T01:37:48.258 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 fe80::20a0:6bff:fe43:7b04/64 scope link 2023-10-26T01:37:48.258 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:37:48.259 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:37:48.260 DEBUG:teuthology.orchestra.run.smithi169:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-26T01:37:48.351 INFO:teuthology.orchestra.run.smithi169.stdout:1 2023-10-26T01:37:48.356 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:37:48.356 DEBUG:teuthology.orchestra.run.smithi169:> route 2023-10-26T01:37:48.409 INFO:teuthology.orchestra.run.smithi169.stdout:Kernel IP routing table 2023-10-26T01:37:48.409 INFO:teuthology.orchestra.run.smithi169.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-26T01:37:48.409 INFO:teuthology.orchestra.run.smithi169.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-26T01:37:48.410 INFO:teuthology.orchestra.run.smithi169.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-26T01:37:48.410 INFO:teuthology.orchestra.run.smithi169.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-26T01:37:48.411 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:37:48.411 DEBUG:teuthology.orchestra.run.smithi169:> (cd / && exec bash -c ' 2023-10-26T01:37:48.412 DEBUG:teuthology.orchestra.run.smithi169:> set -e 2023-10-26T01:37:48.412 DEBUG:teuthology.orchestra.run.smithi169:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-26T01:37:48.412 DEBUG:teuthology.orchestra.run.smithi169:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-26T01:37:48.413 DEBUG:teuthology.orchestra.run.smithi169:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-26T01:37:48.413 DEBUG:teuthology.orchestra.run.smithi169:> ') 2023-10-26T01:37:48.590 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:37:48.590 DEBUG:teuthology.orchestra.run.smithi169:> ip netns list 2023-10-26T01:37:48.644 INFO:teuthology.orchestra.run.smithi169.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-26T01:37:48.644 INFO:teuthology.orchestra.run.smithi169.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-26T01:37:48.644 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-26T01:37:48.645 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:37:48.646 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link set brx.0 up 2023-10-26T01:37:48.724 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-26T01:37:48.724 DEBUG:teuthology.orchestra.run.smithi169:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-26T01:37:48.774 INFO:teuthology.orchestra.run.smithi169.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-26T01:37:48.774 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-26T01:37:48.775 DEBUG:teuthology.orchestra.run.smithi169:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-26T01:37:48.824 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-26T01:37:48.825 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:37:48.825 DEBUG:teuthology.orchestra.run.smithi169:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-26T01:37:48.969 INFO:teuthology.orchestra.run.smithi169.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-26T01:37:48.970 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-26T01:37:48.970 INFO:teuthology.orchestra.run.smithi169.stdout:invalid new device string format 2023-10-26T01:37:48.991 INFO:teuthology.orchestra.run.smithi169.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-26T01:37:48.992 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-26T01:37:48.992 INFO:teuthology.orchestra.run.smithi169.stdout:invalid new device string format 2023-10-26T01:37:48.992 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: resolved to: "172.21.15.169:3300,172.21.15.170:3300,172.21.15.170:3301" 2023-10-26T01:37:48.992 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-26T01:37:49.017 INFO:teuthology.orchestra.run.smithi169.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-26T01:37:49.017 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-26T01:37:49.018 INFO:teuthology.orchestra.run.smithi169.stdout:invalid new device string format 2023-10-26T01:37:49.018 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: resolved to: "172.21.15.169:3300,172.21.15.170:3300,172.21.15.170:3301" 2023-10-26T01:37:49.018 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-26T01:37:49.018 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: resolved to: "172.21.15.169:6789,172.21.15.170:6789,172.21.15.170:6790" 2023-10-26T01:37:49.018 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.169:6789,172.21.15.170:6789,172.21.15.170:6790:/ 2023-10-26T01:37:49.019 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=30203b28-ed8a-44e8-a9a6-a9bf25f980f1" will pass to kernel 2023-10-26T01:37:49.021 INFO:teuthology.orchestra.run.smithi169.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2023-10-26T01:37:49.021 INFO:teuthology.orchestra.run.smithi169.stdout: You just mounted a file system that supports labels which does not 2023-10-26T01:37:49.021 INFO:teuthology.orchestra.run.smithi169.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-10-26T01:37:49.021 INFO:teuthology.orchestra.run.smithi169.stdout: applications will generate AVC messages and not be allowed access to 2023-10-26T01:37:49.022 INFO:teuthology.orchestra.run.smithi169.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-10-26T01:37:49.024 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-26T01:37:49.025 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:37:49.025 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-26T01:37:49.112 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:37:49.112 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-26T01:37:49.177 INFO:teuthology.orchestra.run.smithi169.stdout:client5764 2023-10-26T01:37:49.178 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:37:49.178 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:37:49.230 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-26T01:37:49.449 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:49.446+0000 7ff094525640 1 -- 172.21.15.169:0/2409020783 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff08c0cdef0 msgr2=0x7ff08c15b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:49.449 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:49.446+0000 7ff094525640 1 --2- 172.21.15.169:0/2409020783 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff08c0cdef0 0x7ff08c15b4a0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7ff08c13e070 tx=0x7ff07c02bdb0 comp rx=0 tx=0).stop 2023-10-26T01:37:49.450 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:49.446+0000 7ff094525640 1 -- 172.21.15.169:0/2409020783 shutdown_connections 2023-10-26T01:37:49.450 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:49.446+0000 7ff094525640 1 --2- 172.21.15.169:0/2409020783 >> v2:172.21.15.170:3300/0 conn(0x7ff08c15b9e0 0x7ff08c0b05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:49.450 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:49.446+0000 7ff094525640 1 --2- 172.21.15.169:0/2409020783 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff08c0cdef0 0x7ff08c15b4a0 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:49.451 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:49.446+0000 7ff094525640 1 -- 172.21.15.169:0/2409020783 >> 172.21.15.169:0/2409020783 conn(0x7ff08c069930 msgr2=0x7ff08c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:49.451 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:49.447+0000 7ff094525640 1 -- 172.21.15.169:0/2409020783 shutdown_connections 2023-10-26T01:37:49.451 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:49.447+0000 7ff094525640 1 -- 172.21.15.169:0/2409020783 wait complete. 2023-10-26T01:37:49.453 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:49.448+0000 7ff094525640 1 Processor -- start 2023-10-26T01:37:49.453 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:49.448+0000 7ff094525640 1 -- start start 2023-10-26T01:37:49.453 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:49.449+0000 7ff094525640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff08c0cdef0 0x7ff08c0c9d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:49.454 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:49.449+0000 7ff094525640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff08c155360 0x7ff08c096910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:49.454 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:49.449+0000 7ff094525640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff08c15b9e0 0x7ff08c14e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:49.454 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:49.449+0000 7ff094525640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff08c1566d0 con 0x7ff08c15b9e0 2023-10-26T01:37:49.454 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:49.449+0000 7ff094525640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff08c0af760 con 0x7ff08c0cdef0 2023-10-26T01:37:49.454 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:49.449+0000 7ff094525640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff08c0afa40 con 0x7ff08c155360 2023-10-26T01:37:49.455 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:49.450+0000 7ff092ac2640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff08c15b9e0 0x7ff08c14e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:49.455 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:49.450+0000 7ff092ac2640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff08c15b9e0 0x7ff08c14e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:35788/0 (socket says 172.21.15.169:35788) 2023-10-26T01:37:49.455 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:49.450+0000 7ff092ac2640 1 -- 172.21.15.169:0/2962699134 learned_addr learned my addr 172.21.15.169:0/2962699134 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:37:49.455 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:49.450+0000 7ff0922c1640 1 --2- 172.21.15.169:0/2962699134 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff08c0cdef0 0x7ff08c0c9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:49.455 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:49.450+0000 7ff091ac0640 1 --2- 172.21.15.169:0/2962699134 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff08c155360 0x7ff08c096910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:49.456 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:49.450+0000 7ff092ac2640 1 -- 172.21.15.169:0/2962699134 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff08c155360 msgr2=0x7ff08c096910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:49.456 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:49.450+0000 7ff092ac2640 1 --2- 172.21.15.169:0/2962699134 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff08c155360 0x7ff08c096910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:49.456 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:49.450+0000 7ff092ac2640 1 -- 172.21.15.169:0/2962699134 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff08c0cdef0 msgr2=0x7ff08c0c9d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:49.456 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:49.450+0000 7ff092ac2640 1 --2- 172.21.15.169:0/2962699134 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff08c0cdef0 0x7ff08c0c9d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:49.456 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:49.450+0000 7ff092ac2640 1 -- 172.21.15.169:0/2962699134 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff07c009560 con 0x7ff08c15b9e0 2023-10-26T01:37:49.456 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:49.450+0000 7ff092ac2640 1 --2- 172.21.15.169:0/2962699134 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff08c15b9e0 0x7ff08c14e840 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7ff08400b970 tx=0x7ff08400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:49.457 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:49.451+0000 7ff082ffd640 1 -- 172.21.15.169:0/2962699134 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff08400cb40 con 0x7ff08c15b9e0 2023-10-26T01:37:49.457 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:49.451+0000 7ff082ffd640 1 -- 172.21.15.169:0/2962699134 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff08400cdb0 con 0x7ff08c15b9e0 2023-10-26T01:37:49.457 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:49.451+0000 7ff082ffd640 1 -- 172.21.15.169:0/2962699134 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0840146a0 con 0x7ff08c15b9e0 2023-10-26T01:37:49.457 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:49.451+0000 7ff094525640 1 -- 172.21.15.169:0/2962699134 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff08c156940 con 0x7ff08c15b9e0 2023-10-26T01:37:49.457 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:49.451+0000 7ff094525640 1 -- 172.21.15.169:0/2962699134 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff08c096e50 con 0x7ff08c15b9e0 2023-10-26T01:37:49.458 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:49.452+0000 7ff094525640 1 -- 172.21.15.169:0/2962699134 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff08c04db10 con 0x7ff08c15b9e0 2023-10-26T01:37:49.459 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:49.453+0000 7ff082ffd640 1 -- 172.21.15.169:0/2962699134 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ff08401d3b0 con 0x7ff08c15b9e0 2023-10-26T01:37:49.459 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:49.454+0000 7ff082ffd640 1 --2- 172.21.15.169:0/2962699134 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff07006ae70 0x7ff07006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:49.459 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:49.454+0000 7ff082ffd640 1 -- 172.21.15.169:0/2962699134 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff0840897a0 con 0x7ff08c15b9e0 2023-10-26T01:37:49.459 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:49.455+0000 7ff0922c1640 1 --2- 172.21.15.169:0/2962699134 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff07006ae70 0x7ff07006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:49.459 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:49.456+0000 7ff0922c1640 1 --2- 172.21.15.169:0/2962699134 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff07006ae70 0x7ff07006d330 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7ff08c156e80 tx=0x7ff088009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:49.460 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:49.458+0000 7ff082ffd640 1 -- 172.21.15.169:0/2962699134 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff084054840 con 0x7ff08c15b9e0 2023-10-26T01:37:49.690 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:49.688+0000 7ff094525640 1 -- 172.21.15.169:0/2962699134 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff08c15b180 con 0x7ff08c15b9e0 2023-10-26T01:37:49.690 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:49.689+0000 7ff082ffd640 1 -- 172.21.15.169:0/2962699134 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v59) v1 ==== 53+0+73 (secure 0 0 0) 0x7ff084019070 con 0x7ff08c15b9e0 2023-10-26T01:37:49.691 INFO:teuthology.orchestra.run.smithi169.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-26T01:37:49.693 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:49.691+0000 7ff094525640 1 -- 172.21.15.169:0/2962699134 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff07006ae70 msgr2=0x7ff07006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:49.694 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:49.691+0000 7ff094525640 1 --2- 172.21.15.169:0/2962699134 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff07006ae70 0x7ff07006d330 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7ff08c156e80 tx=0x7ff088009040 comp rx=0 tx=0).stop 2023-10-26T01:37:49.694 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:49.691+0000 7ff094525640 1 -- 172.21.15.169:0/2962699134 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff08c15b9e0 msgr2=0x7ff08c14e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:49.694 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:49.691+0000 7ff094525640 1 --2- 172.21.15.169:0/2962699134 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff08c15b9e0 0x7ff08c14e840 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7ff08400b970 tx=0x7ff08400be30 comp rx=0 tx=0).stop 2023-10-26T01:37:49.694 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:49.691+0000 7ff094525640 1 -- 172.21.15.169:0/2962699134 shutdown_connections 2023-10-26T01:37:49.694 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:49.691+0000 7ff094525640 1 --2- 172.21.15.169:0/2962699134 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff07006ae70 0x7ff07006d330 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:49.695 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:49.691+0000 7ff094525640 1 --2- 172.21.15.169:0/2962699134 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff08c15b9e0 0x7ff08c14e840 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:49.695 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:49.691+0000 7ff094525640 1 --2- 172.21.15.169:0/2962699134 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff08c155360 0x7ff08c096910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:49.695 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:49.691+0000 7ff094525640 1 --2- 172.21.15.169:0/2962699134 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff08c0cdef0 0x7ff08c0c9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:49.695 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:49.691+0000 7ff094525640 1 -- 172.21.15.169:0/2962699134 >> 172.21.15.169:0/2962699134 conn(0x7ff08c069930 msgr2=0x7ff08c159940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:49.695 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:49.692+0000 7ff094525640 1 -- 172.21.15.169:0/2962699134 shutdown_connections 2023-10-26T01:37:49.696 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:49.692+0000 7ff094525640 1 -- 172.21.15.169:0/2962699134 wait complete. 2023-10-26T01:37:49.713 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-26T01:37:49.713 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-26T01:37:49.714 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi170.front.sepia.ceph.com 2023-10-26T01:37:49.714 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-26T01:37:49.714 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-26T01:37:49.715 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-26T01:37:49.715 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-26T01:37:49.716 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-26T01:37:49.716 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-26T01:37:49.716 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:37:49.717 DEBUG:teuthology.orchestra.run.smithi170:> ip addr 2023-10-26T01:37:49.778 INFO:teuthology.orchestra.run.smithi170.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-26T01:37:49.778 INFO:teuthology.orchestra.run.smithi170.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-26T01:37:49.778 INFO:teuthology.orchestra.run.smithi170.stdout: inet 127.0.0.1/8 scope host lo 2023-10-26T01:37:49.779 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:37:49.779 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 ::1/128 scope host 2023-10-26T01:37:49.779 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:37:49.779 INFO:teuthology.orchestra.run.smithi170.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-26T01:37:49.779 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:d9:3b:54 brd ff:ff:ff:ff:ff:ff 2023-10-26T01:37:49.780 INFO:teuthology.orchestra.run.smithi170.stdout: altname enp7s0f0 2023-10-26T01:37:49.780 INFO:teuthology.orchestra.run.smithi170.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-26T01:37:49.780 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:d9:3b:55 brd ff:ff:ff:ff:ff:ff 2023-10-26T01:37:49.780 INFO:teuthology.orchestra.run.smithi170.stdout: altname enp7s0f1 2023-10-26T01:37:49.780 INFO:teuthology.orchestra.run.smithi170.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-26T01:37:49.781 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:88:72:54 brd ff:ff:ff:ff:ff:ff 2023-10-26T01:37:49.781 INFO:teuthology.orchestra.run.smithi170.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-26T01:37:49.781 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:88:72:55 brd ff:ff:ff:ff:ff:ff 2023-10-26T01:37:49.781 INFO:teuthology.orchestra.run.smithi170.stdout: inet 172.21.15.170/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-26T01:37:49.781 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft 42010sec preferred_lft 42010sec 2023-10-26T01:37:49.782 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::ec4:7aff:fe88:7255/64 scope link 2023-10-26T01:37:49.782 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:37:49.782 INFO:teuthology.orchestra.run.smithi170.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-26T01:37:49.782 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 32:b5:d1:ee:10:36 brd ff:ff:ff:ff:ff:ff 2023-10-26T01:37:49.782 INFO:teuthology.orchestra.run.smithi170.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-26T01:37:49.783 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:37:49.783 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::28ed:d1ff:fe37:70d/64 scope link 2023-10-26T01:37:49.783 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:37:49.783 INFO:teuthology.orchestra.run.smithi170.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-26T01:37:49.784 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 32:b5:d1:ee:10:36 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-26T01:37:49.784 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::30b5:d1ff:feee:1036/64 scope link 2023-10-26T01:37:49.784 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:37:49.784 INFO:teuthology.orchestra.run.smithi170.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-26T01:37:49.784 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether b2:c1:53:19:69:e4 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-26T01:37:49.784 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::b0c1:53ff:fe19:69e4/64 scope link 2023-10-26T01:37:49.785 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:37:49.785 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:37:49.785 DEBUG:teuthology.orchestra.run.smithi170:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-26T01:37:49.882 INFO:teuthology.orchestra.run.smithi170.stdout:1 2023-10-26T01:37:49.886 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:37:49.886 DEBUG:teuthology.orchestra.run.smithi170:> route 2023-10-26T01:37:49.942 INFO:teuthology.orchestra.run.smithi170.stdout:Kernel IP routing table 2023-10-26T01:37:49.942 INFO:teuthology.orchestra.run.smithi170.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-26T01:37:49.943 INFO:teuthology.orchestra.run.smithi170.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-26T01:37:49.943 INFO:teuthology.orchestra.run.smithi170.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-26T01:37:49.943 INFO:teuthology.orchestra.run.smithi170.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-26T01:37:49.943 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:37:49.944 DEBUG:teuthology.orchestra.run.smithi170:> (cd / && exec bash -c ' 2023-10-26T01:37:49.944 DEBUG:teuthology.orchestra.run.smithi170:> set -e 2023-10-26T01:37:49.944 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-26T01:37:49.945 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-26T01:37:49.945 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-26T01:37:49.946 DEBUG:teuthology.orchestra.run.smithi170:> ') 2023-10-26T01:37:50.130 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:37:50.131 DEBUG:teuthology.orchestra.run.smithi170:> ip netns list 2023-10-26T01:37:50.185 INFO:teuthology.orchestra.run.smithi170.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-26T01:37:50.185 INFO:teuthology.orchestra.run.smithi170.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-26T01:37:50.186 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-26T01:37:50.186 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:37:50.187 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip link set brx.0 up 2023-10-26T01:37:50.281 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-26T01:37:50.281 DEBUG:teuthology.orchestra.run.smithi170:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-26T01:37:50.338 INFO:teuthology.orchestra.run.smithi170.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-26T01:37:50.338 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-26T01:37:50.339 DEBUG:teuthology.orchestra.run.smithi170:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-26T01:37:50.395 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-26T01:37:50.396 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:37:50.397 DEBUG:teuthology.orchestra.run.smithi170:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-26T01:37:50.540 INFO:teuthology.orchestra.run.smithi170.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-26T01:37:50.541 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-26T01:37:50.541 INFO:teuthology.orchestra.run.smithi170.stdout:invalid new device string format 2023-10-26T01:37:50.564 INFO:teuthology.orchestra.run.smithi170.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-26T01:37:50.565 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-26T01:37:50.565 INFO:teuthology.orchestra.run.smithi170.stdout:invalid new device string format 2023-10-26T01:37:50.565 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: resolved to: "172.21.15.169:3300,172.21.15.170:3300,172.21.15.170:3301" 2023-10-26T01:37:50.565 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-26T01:37:50.590 INFO:teuthology.orchestra.run.smithi170.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-26T01:37:50.590 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-26T01:37:50.591 INFO:teuthology.orchestra.run.smithi170.stdout:invalid new device string format 2023-10-26T01:37:50.591 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: resolved to: "172.21.15.169:3300,172.21.15.170:3300,172.21.15.170:3301" 2023-10-26T01:37:50.591 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-26T01:37:50.591 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: resolved to: "172.21.15.169:6789,172.21.15.170:6789,172.21.15.170:6790" 2023-10-26T01:37:50.591 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.169:6789,172.21.15.170:6789,172.21.15.170:6790:/ 2023-10-26T01:37:50.592 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=30203b28-ed8a-44e8-a9a6-a9bf25f980f1" will pass to kernel 2023-10-26T01:37:50.594 INFO:teuthology.orchestra.run.smithi170.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2023-10-26T01:37:50.594 INFO:teuthology.orchestra.run.smithi170.stdout: You just mounted a file system that supports labels which does not 2023-10-26T01:37:50.594 INFO:teuthology.orchestra.run.smithi170.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-10-26T01:37:50.595 INFO:teuthology.orchestra.run.smithi170.stdout: applications will generate AVC messages and not be allowed access to 2023-10-26T01:37:50.595 INFO:teuthology.orchestra.run.smithi170.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-10-26T01:37:50.599 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-26T01:37:50.599 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:37:50.600 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-26T01:37:50.688 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:37:50.689 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-26T01:37:50.744 INFO:teuthology.orchestra.run.smithi170.stdout:client5770 2023-10-26T01:37:50.745 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-26T01:37:50.745 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:37:50.792 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T01:37:51.015 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:51.013+0000 7f9298d55640 1 -- 172.21.15.169:0/3927557822 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f92941592d0 msgr2=0x7f92940aa3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:51.015 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:51.013+0000 7f9298d55640 1 --2- 172.21.15.169:0/3927557822 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f92941592d0 0x7f92940aa3d0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f92940b2650 tx=0x7f928402bd00 comp rx=0 tx=0).stop 2023-10-26T01:37:51.015 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:51.013+0000 7f9298d55640 1 -- 172.21.15.169:0/3927557822 shutdown_connections 2023-10-26T01:37:51.016 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:51.013+0000 7f9298d55640 1 --2- 172.21.15.169:0/3927557822 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f92940cdc30 0x7f929406ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:51.016 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:51.013+0000 7f9298d55640 1 --2- 172.21.15.169:0/3927557822 >> v2:172.21.15.170:3300/0 conn(0x7f92940ac930 0x7f92940cd6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:51.016 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:51.013+0000 7f9298d55640 1 --2- 172.21.15.169:0/3927557822 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f92941592d0 0x7f92940aa3d0 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:51.016 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:51.013+0000 7f9298d55640 1 -- 172.21.15.169:0/3927557822 >> 172.21.15.169:0/3927557822 conn(0x7f9294069930 msgr2=0x7f9294069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:51.016 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:51.013+0000 7f9298d55640 1 -- 172.21.15.169:0/3927557822 shutdown_connections 2023-10-26T01:37:51.017 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:51.013+0000 7f9298d55640 1 -- 172.21.15.169:0/3927557822 wait complete. 2023-10-26T01:37:51.019 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:51.015+0000 7f9298d55640 1 Processor -- start 2023-10-26T01:37:51.019 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:51.015+0000 7f9298d55640 1 -- start start 2023-10-26T01:37:51.019 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:51.016+0000 7f9298d55640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f92940ac930 0x7f9294158190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:51.019 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:51.016+0000 7f9298d55640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f92940cdc30 0x7f9294151550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:51.020 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:51.016+0000 7f9298d55640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f92941592d0 0x7f9294151a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:51.020 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:51.016+0000 7f9298d55640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92941e4bd0 con 0x7f92940ac930 2023-10-26T01:37:51.020 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:51.016+0000 7f9298d55640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92941e3fc0 con 0x7f92941592d0 2023-10-26T01:37:51.020 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:51.016+0000 7f9298d55640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92941e42a0 con 0x7f92940cdc30 2023-10-26T01:37:51.020 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:51.016+0000 7f929259c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f92940ac930 0x7f9294158190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:51.021 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:51.016+0000 7f929259c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f92940ac930 0x7f9294158190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:35808/0 (socket says 172.21.15.169:35808) 2023-10-26T01:37:51.021 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:51.016+0000 7f929259c640 1 -- 172.21.15.169:0/997869296 learned_addr learned my addr 172.21.15.169:0/997869296 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:37:51.021 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:51.016+0000 7f929259c640 1 -- 172.21.15.169:0/997869296 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f92940cdc30 msgr2=0x7f9294151550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:51.021 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:51.016+0000 7f929259c640 1 --2- 172.21.15.169:0/997869296 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f92940cdc30 0x7f9294151550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:51.021 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:51.016+0000 7f929259c640 1 -- 172.21.15.169:0/997869296 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f92941592d0 msgr2=0x7f9294151a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:51.021 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:51.016+0000 7f929259c640 1 --2- 172.21.15.169:0/997869296 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f92941592d0 0x7f9294151a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:51.022 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:51.016+0000 7f929259c640 1 -- 172.21.15.169:0/997869296 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9284009560 con 0x7f92940ac930 2023-10-26T01:37:51.022 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:51.017+0000 7f929259c640 1 --2- 172.21.15.169:0/997869296 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f92940ac930 0x7f9294158190 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f9284009850 tx=0x7f928402dcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:51.022 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:51.017+0000 7f928affd640 1 -- 172.21.15.169:0/997869296 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9284003800 con 0x7f92940ac930 2023-10-26T01:37:51.022 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:51.017+0000 7f928affd640 1 -- 172.21.15.169:0/997869296 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9284003a50 con 0x7f92940ac930 2023-10-26T01:37:51.022 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:51.017+0000 7f9298d55640 1 -- 172.21.15.169:0/997869296 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92941e4580 con 0x7f92940ac930 2023-10-26T01:37:51.023 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:51.017+0000 7f928affd640 1 -- 172.21.15.169:0/997869296 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9284003e40 con 0x7f92940ac930 2023-10-26T01:37:51.023 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:51.017+0000 7f9298d55640 1 -- 172.21.15.169:0/997869296 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92941586d0 con 0x7f92940ac930 2023-10-26T01:37:51.024 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:51.018+0000 7f9298d55640 1 -- 172.21.15.169:0/997869296 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9294158920 con 0x7f92940ac930 2023-10-26T01:37:51.024 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:51.019+0000 7f928affd640 1 -- 172.21.15.169:0/997869296 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f928403e450 con 0x7f92940ac930 2023-10-26T01:37:51.024 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:51.021+0000 7f928affd640 1 --2- 172.21.15.169:0/997869296 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f927406ae70 0x7f927406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:51.024 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:51.021+0000 7f928affd640 1 -- 172.21.15.169:0/997869296 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f92840ac090 con 0x7f92940ac930 2023-10-26T01:37:51.025 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:51.021+0000 7f928bfff640 1 --2- 172.21.15.169:0/997869296 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f927406ae70 0x7f927406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:51.025 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:51.023+0000 7f928bfff640 1 --2- 172.21.15.169:0/997869296 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f927406ae70 0x7f927406d330 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f9294158bb0 tx=0x7f927c006c90 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:51.027 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:51.025+0000 7f928affd640 1 -- 172.21.15.169:0/997869296 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f92840771c0 con 0x7f92940ac930 2023-10-26T01:37:51.253 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:51.252+0000 7f9298d55640 1 -- 172.21.15.169:0/997869296 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f929404db10 con 0x7f92940ac930 2023-10-26T01:37:51.254 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:51.253+0000 7f928affd640 1 -- 172.21.15.169:0/997869296 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) v1 ==== 94+0+4924 (secure 0 0 0) 0x7f928407a7e0 con 0x7f92940ac930 2023-10-26T01:37:51.255 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:37:51.255 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":59,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5387,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/1186563921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1186563921},{"type":"v1","addr":"172.21.15.170:6835","nonce":1186563921}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":54},{"gid":5390,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/3702036693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":3702036693},{"type":"v1","addr":"172.21.15.170:6837","nonce":3702036693}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":54},{"gid":7332,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/3193888686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":3193888686},{"type":"v1","addr":"172.21.15.169:6833","nonce":3193888686}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T01:37:40.027858+0000","modified":"2023-10-26T01:37:43.103474+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7335},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7335":{"gid":7335,"name":"c","rank":0,"incarnation":55,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6835/668588916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":668588916},{"type":"v1","addr":"172.21.15.169:6835","nonce":668588916}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-10-26T01:37:51.255 INFO:teuthology.orchestra.run.smithi169.stderr:dumped fsmap epoch 59 2023-10-26T01:37:51.257 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:51.255+0000 7f9298d55640 1 -- 172.21.15.169:0/997869296 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f927406ae70 msgr2=0x7f927406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:51.258 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:51.255+0000 7f9298d55640 1 --2- 172.21.15.169:0/997869296 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f927406ae70 0x7f927406d330 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f9294158bb0 tx=0x7f927c006c90 comp rx=0 tx=0).stop 2023-10-26T01:37:51.258 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:51.255+0000 7f9298d55640 1 -- 172.21.15.169:0/997869296 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f92940ac930 msgr2=0x7f9294158190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:51.258 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:51.255+0000 7f9298d55640 1 --2- 172.21.15.169:0/997869296 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f92940ac930 0x7f9294158190 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f9284009850 tx=0x7f928402dcd0 comp rx=0 tx=0).stop 2023-10-26T01:37:51.258 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:51.256+0000 7f9298d55640 1 -- 172.21.15.169:0/997869296 shutdown_connections 2023-10-26T01:37:51.258 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:51.256+0000 7f9298d55640 1 --2- 172.21.15.169:0/997869296 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f927406ae70 0x7f927406d330 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:51.259 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:51.256+0000 7f9298d55640 1 --2- 172.21.15.169:0/997869296 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f92941592d0 0x7f9294151a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:51.259 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:51.256+0000 7f9298d55640 1 --2- 172.21.15.169:0/997869296 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f92940cdc30 0x7f9294151550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:51.259 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:51.256+0000 7f9298d55640 1 --2- 172.21.15.169:0/997869296 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f92940ac930 0x7f9294158190 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:51.259 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:51.256+0000 7f9298d55640 1 -- 172.21.15.169:0/997869296 >> 172.21.15.169:0/997869296 conn(0x7f9294069930 msgr2=0x7f92940ae200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:51.260 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:51.256+0000 7f9298d55640 1 -- 172.21.15.169:0/997869296 shutdown_connections 2023-10-26T01:37:51.261 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:51.256+0000 7f9298d55640 1 -- 172.21.15.169:0/997869296 wait complete. 2023-10-26T01:37:51.276 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2023-10-26T01:37:51.512 INFO:teuthology.orchestra.run.smithi169.stdout:{"cluster_fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","whoami":0,"id":7335,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":10.911376756999999,"mdsmap_epoch":59,"osdmap_epoch":120,"osdmap_epoch_barrier":120,"uptime":16.892366739} 2023-10-26T01:37:51.512 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "30203b28-ed8a-44e8-a9a6-a9bf25f980f1", "fs_name": "cephfs", "id": 7335, "mdsmap_epoch": 59, "osdmap_epoch": 120, "osdmap_epoch_barrier": 120, "rank_uptime": 10.911376757, "state": "up:active", "uptime": 16.892366739, "want_state": "up:active", "whoami": 0 } 2023-10-26T01:37:51.513 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set-quota cephfs_data max_bytes 33554432 2023-10-26T01:37:51.742 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:51.740+0000 7f7e12600640 1 -- 172.21.15.169:0/2063277394 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7df80047c0 msgr2=0x7f7df8004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:51.742 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:51.740+0000 7f7e12600640 1 --2- 172.21.15.169:0/2063277394 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7df80047c0 0x7f7df8004c40 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f7df4009b20 tx=0x7f7df80057f0 comp rx=0 tx=0).stop 2023-10-26T01:37:51.743 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:51.740+0000 7f7e10b9d640 1 -- 172.21.15.169:0/2063277394 reap_dead start 2023-10-26T01:37:51.743 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:51.740+0000 7f7e12600640 1 -- 172.21.15.169:0/2063277394 shutdown_connections 2023-10-26T01:37:51.743 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:51.740+0000 7f7e12600640 1 -- 172.21.15.169:0/2063277394 >> 172.21.15.169:0/2063277394 conn(0x7f7e0c0aeee0 msgr2=0x7f7e0c0af2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:51.743 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:51.740+0000 7f7e12600640 1 -- 172.21.15.169:0/2063277394 shutdown_connections 2023-10-26T01:37:51.743 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:51.740+0000 7f7e12600640 1 -- 172.21.15.169:0/2063277394 wait complete. 2023-10-26T01:37:51.744 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:51.742+0000 7f7e12600640 1 Processor -- start 2023-10-26T01:37:51.744 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:51.742+0000 7f7e12600640 1 -- start start 2023-10-26T01:37:51.746 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:51.743+0000 7f7e12600640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7df80047c0 0x7f7e0c06f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:51.746 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:51.743+0000 7f7e12600640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7e0c1f1ae0 0x7f7e0c153680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:51.746 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:51.743+0000 7f7e12600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e0c157870 0x7f7e0c1f5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:51.747 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:51.743+0000 7f7e12600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e0c153bc0 con 0x7f7e0c157870 2023-10-26T01:37:51.747 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:51.743+0000 7f7e12600640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e0c070020 con 0x7f7e0c1f1ae0 2023-10-26T01:37:51.747 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:51.743+0000 7f7e12600640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e0c1f1f40 con 0x7f7df80047c0 2023-10-26T01:37:51.747 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:51.743+0000 7f7e10b9d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e0c157870 0x7f7e0c1f5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:51.747 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:51.743+0000 7f7e10b9d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e0c157870 0x7f7e0c1f5610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:35838/0 (socket says 172.21.15.169:35838) 2023-10-26T01:37:51.748 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:51.743+0000 7f7e10b9d640 1 -- 172.21.15.169:0/1727183634 learned_addr learned my addr 172.21.15.169:0/1727183634 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:37:51.748 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:51.743+0000 7f7e0b7fe640 1 --2- 172.21.15.169:0/1727183634 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7e0c1f1ae0 0x7f7e0c153680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:51.748 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:51.743+0000 7f7e0bfff640 1 --2- 172.21.15.169:0/1727183634 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7df80047c0 0x7f7e0c06f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:51.748 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:51.744+0000 7f7e10b9d640 1 -- 172.21.15.169:0/1727183634 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7df80047c0 msgr2=0x7f7e0c06f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:51.748 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:51.744+0000 7f7e10b9d640 1 --2- 172.21.15.169:0/1727183634 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7df80047c0 0x7f7e0c06f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:51.748 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:51.744+0000 7f7e10b9d640 1 -- 172.21.15.169:0/1727183634 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7e0c1f1ae0 msgr2=0x7f7e0c153680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:51.749 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:51.744+0000 7f7e10b9d640 1 --2- 172.21.15.169:0/1727183634 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7e0c1f1ae0 0x7f7e0c153680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:51.749 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:51.744+0000 7f7e10b9d640 1 -- 172.21.15.169:0/1727183634 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e0c06fd40 con 0x7f7e0c157870 2023-10-26T01:37:51.749 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:51.744+0000 7f7e0bfff640 1 --2- 172.21.15.169:0/1727183634 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7df80047c0 0x7f7e0c06f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:37:51.749 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:51.744+0000 7f7e10b9d640 1 --2- 172.21.15.169:0/1727183634 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e0c157870 0x7f7e0c1f5610 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c0afc20 tx=0x7f7e0400cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:51.749 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:51.744+0000 7f7e08ff9640 1 -- 172.21.15.169:0/1727183634 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e04005d40 con 0x7f7e0c157870 2023-10-26T01:37:51.750 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:51.744+0000 7f7e08ff9640 1 -- 172.21.15.169:0/1727183634 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7e0400ced0 con 0x7f7e0c157870 2023-10-26T01:37:51.750 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:51.744+0000 7f7e08ff9640 1 -- 172.21.15.169:0/1727183634 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e0400a870 con 0x7f7e0c157870 2023-10-26T01:37:51.750 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:51.744+0000 7f7e12600640 1 -- 172.21.15.169:0/1727183634 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e0c155c60 con 0x7f7e0c157870 2023-10-26T01:37:51.750 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:51.745+0000 7f7e12600640 1 -- 172.21.15.169:0/1727183634 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e0c156220 con 0x7f7e0c157870 2023-10-26T01:37:51.751 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:51.746+0000 7f7e12600640 1 -- 172.21.15.169:0/1727183634 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e0c066af0 con 0x7f7e0c157870 2023-10-26T01:37:51.751 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:51.747+0000 7f7e08ff9640 1 -- 172.21.15.169:0/1727183634 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f7e0400a9d0 con 0x7f7e0c157870 2023-10-26T01:37:51.752 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:51.747+0000 7f7e08ff9640 1 --2- 172.21.15.169:0/1727183634 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7dec06ae40 0x7f7dec06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:51.752 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:51.747+0000 7f7e08ff9640 1 -- 172.21.15.169:0/1727183634 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7e04088790 con 0x7f7e0c157870 2023-10-26T01:37:51.752 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:51.748+0000 7f7e0bfff640 1 --2- 172.21.15.169:0/1727183634 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7dec06ae40 0x7f7dec06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:51.752 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:51.749+0000 7f7e0bfff640 1 --2- 172.21.15.169:0/1727183634 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7dec06ae40 0x7f7dec06d300 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc001eb0 tx=0x7f7dfc001c60 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:51.753 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:51.752+0000 7f7e08ff9640 1 -- 172.21.15.169:0/1727183634 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7e040538f0 con 0x7f7e0c157870 2023-10-26T01:37:51.943 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:51.942+0000 7f7e12600640 1 -- 172.21.15.169:0/1727183634 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"} v 0) v1 -- 0x7f7e0c04da70 con 0x7f7e0c157870 2023-10-26T01:37:52.081 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:52.079+0000 7f7e08ff9640 1 -- 172.21.15.169:0/1727183634 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"}]=0 set-quota max_bytes = 33554432 for pool cephfs_data v121) v1 ==== 181+0+0 (secure 0 0 0) 0x7f7e04019020 con 0x7f7e0c157870 2023-10-26T01:37:52.082 INFO:teuthology.orchestra.run.smithi169.stderr:set-quota max_bytes = 33554432 for pool cephfs_data 2023-10-26T01:37:52.085 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:52.083+0000 7f7e12600640 1 -- 172.21.15.169:0/1727183634 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7dec06ae40 msgr2=0x7f7dec06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:52.085 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:52.083+0000 7f7e12600640 1 --2- 172.21.15.169:0/1727183634 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7dec06ae40 0x7f7dec06d300 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc001eb0 tx=0x7f7dfc001c60 comp rx=0 tx=0).stop 2023-10-26T01:37:52.085 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:52.083+0000 7f7e12600640 1 -- 172.21.15.169:0/1727183634 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e0c157870 msgr2=0x7f7e0c1f5610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:52.086 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:52.083+0000 7f7e12600640 1 --2- 172.21.15.169:0/1727183634 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e0c157870 0x7f7e0c1f5610 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c0afc20 tx=0x7f7e0400cb90 comp rx=0 tx=0).stop 2023-10-26T01:37:52.086 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:52.083+0000 7f7e12600640 1 -- 172.21.15.169:0/1727183634 shutdown_connections 2023-10-26T01:37:52.086 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:52.083+0000 7f7e12600640 1 --2- 172.21.15.169:0/1727183634 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7dec06ae40 0x7f7dec06d300 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:52.086 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:52.083+0000 7f7e12600640 1 --2- 172.21.15.169:0/1727183634 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e0c157870 0x7f7e0c1f5610 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:52.087 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:52.083+0000 7f7e12600640 1 --2- 172.21.15.169:0/1727183634 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7e0c1f1ae0 0x7f7e0c153680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:52.087 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:52.083+0000 7f7e12600640 1 --2- 172.21.15.169:0/1727183634 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7df80047c0 0x7f7e0c06f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:52.087 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:52.083+0000 7f7e12600640 1 -- 172.21.15.169:0/1727183634 >> 172.21.15.169:0/1727183634 conn(0x7f7e0c0aeee0 msgr2=0x7f7e0c15f1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:52.087 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:52.084+0000 7f7e12600640 1 -- 172.21.15.169:0/1727183634 shutdown_connections 2023-10-26T01:37:52.087 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:52.084+0000 7f7e12600640 1 -- 172.21.15.169:0/1727183634 wait complete. 2023-10-26T01:37:52.107 DEBUG:teuthology.orchestra.run.smithi169:> stdin-killer -- python3 -c ' 2023-10-26T01:37:52.108 DEBUG:teuthology.orchestra.run.smithi169:> import os 2023-10-26T01:37:52.108 DEBUG:teuthology.orchestra.run.smithi169:> fd = os.open("/home/ubuntu/cephtest/mnt.0/foo", os.O_CREAT | os.O_RDWR, 0O600) 2023-10-26T01:37:52.108 DEBUG:teuthology.orchestra.run.smithi169:> os.fchmod(fd, 0O666) 2023-10-26T01:37:52.109 DEBUG:teuthology.orchestra.run.smithi169:> os.fsync(fd) 2023-10-26T01:37:52.109 DEBUG:teuthology.orchestra.run.smithi169:> os.close(fd) 2023-10-26T01:37:52.109 DEBUG:teuthology.orchestra.run.smithi169:> ' 2023-10-26T01:37:52.252 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T01:37:52.262 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:37:52.262 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-26T01:37:52.363 INFO:teuthology.orchestra.run.smithi169.stdout:client5764 2023-10-26T01:37:52.363 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:37:52.364 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/30203b28-ed8a-44e8-a9a6-a9bf25f980f1.client5764/osdmap') 2023-10-26T01:37:52.457 INFO:teuthology.orchestra.run.smithi169.stdout:epoch 120 barrier 120 flags 0x588000 2023-10-26T01:37:52.457 INFO:teuthology.orchestra.run.smithi169.stdout:pool 1 '.mgr' type 1 size 2 min_size 1 pg_num 1 pg_num_mask 0 flags 0x1 lfor 0 read_tier -1 write_tier -1 2023-10-26T01:37:52.457 INFO:teuthology.orchestra.run.smithi169.stdout:pool 14 'cephfs_metadata' type 1 size 2 min_size 1 pg_num 32 pg_num_mask 31 flags 0x1 lfor 0 read_tier -1 write_tier -1 2023-10-26T01:37:52.458 INFO:teuthology.orchestra.run.smithi169.stdout:pool 15 'cephfs_data' type 1 size 2 min_size 1 pg_num 64 pg_num_mask 63 flags 0x1 lfor 0 read_tier -1 write_tier -1 2023-10-26T01:37:52.458 INFO:teuthology.orchestra.run.smithi169.stdout:osd0 (1)172.21.15.169:6817 100% (exists, up) 100% -1 2023-10-26T01:37:52.458 INFO:teuthology.orchestra.run.smithi169.stdout:osd1 (1)172.21.15.169:6809 100% (exists, up) 100% -1 2023-10-26T01:37:52.458 INFO:teuthology.orchestra.run.smithi169.stdout:osd2 (1)172.21.15.169:6801 100% (exists, up) 100% -1 2023-10-26T01:37:52.459 INFO:teuthology.orchestra.run.smithi169.stdout:osd3 (1)172.21.15.169:6825 100% (exists, up) 100% -1 2023-10-26T01:37:52.459 INFO:teuthology.orchestra.run.smithi169.stdout:osd4 (1)172.21.15.170:6817 100% (exists, up) 100% -1 2023-10-26T01:37:52.459 INFO:teuthology.orchestra.run.smithi169.stdout:osd5 (1)172.21.15.170:6809 100% (exists, up) 100% -1 2023-10-26T01:37:52.459 INFO:teuthology.orchestra.run.smithi169.stdout:osd6 (1)172.21.15.170:6801 100% (exists, up) 100% -1 2023-10-26T01:37:52.459 INFO:teuthology.orchestra.run.smithi169.stdout:osd7 (1)172.21.15.170:6822 100% (exists, up) 100% -1 2023-10-26T01:37:52.460 INFO:teuthology.orchestra.run.smithi169.stderr:1+1 records in 2023-10-26T01:37:52.460 INFO:teuthology.orchestra.run.smithi169.stderr:1+1 records out 2023-10-26T01:37:52.460 INFO:teuthology.orchestra.run.smithi169.stderr:803 bytes copied, 8.2971e-05 s, 9.7 MB/s 2023-10-26T01:37:52.461 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set pause 2023-10-26T01:37:52.683 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:52.680+0000 7f96d5780640 1 -- 172.21.15.169:0/116348144 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f96d00aa6d0 msgr2=0x7f96d00aab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:52.683 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:52.680+0000 7f96d5780640 1 --2- 172.21.15.169:0/116348144 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f96d00aa6d0 0x7f96d00aab80 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f96d00acb30 tx=0x7f96c402bd10 comp rx=0 tx=0).stop 2023-10-26T01:37:52.684 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:52.680+0000 7f96d5780640 1 -- 172.21.15.169:0/116348144 shutdown_connections 2023-10-26T01:37:52.684 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:52.680+0000 7f96d5780640 1 --2- 172.21.15.169:0/116348144 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f96d00aa6d0 0x7f96d00aab80 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:52.684 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:52.680+0000 7f96d5780640 1 --2- 172.21.15.169:0/116348144 >> v2:172.21.15.170:3300/0 conn(0x7f96d006cba0 0x7f96d015b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:52.684 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:52.680+0000 7f96d5780640 1 -- 172.21.15.169:0/116348144 >> 172.21.15.169:0/116348144 conn(0x7f96d00af110 msgr2=0x7f96d00af520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:52.684 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:52.681+0000 7f96d5780640 1 -- 172.21.15.169:0/116348144 shutdown_connections 2023-10-26T01:37:52.685 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:52.681+0000 7f96d5780640 1 -- 172.21.15.169:0/116348144 wait complete. 2023-10-26T01:37:52.685 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:52.682+0000 7f96d5780640 1 Processor -- start 2023-10-26T01:37:52.685 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:52.682+0000 7f96d5780640 1 -- start start 2023-10-26T01:37:52.687 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:52.683+0000 7f96d5780640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96d006cba0 0x7f96d0146300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:52.688 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:52.683+0000 7f96d5780640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f96d00aa6d0 0x7f96d0151040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:52.688 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:52.683+0000 7f96d5780640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f96d00b0300 0x7f96d0151580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:52.688 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:52.683+0000 7f96d5780640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96d01592a0 con 0x7f96d00b0300 2023-10-26T01:37:52.688 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:52.683+0000 7f96d5780640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96d0158620 con 0x7f96d006cba0 2023-10-26T01:37:52.688 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:52.683+0000 7f96d5780640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96d0158900 con 0x7f96d00aa6d0 2023-10-26T01:37:52.689 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:52.684+0000 7f96d4f7f640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f96d00b0300 0x7f96d0151580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:52.689 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:52.684+0000 7f96d4f7f640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f96d00b0300 0x7f96d0151580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:35862/0 (socket says 172.21.15.169:35862) 2023-10-26T01:37:52.689 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:52.684+0000 7f96d4f7f640 1 -- 172.21.15.169:0/992154882 learned_addr learned my addr 172.21.15.169:0/992154882 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:37:52.689 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:52.684+0000 7f96cffff640 1 --2- 172.21.15.169:0/992154882 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96d006cba0 0x7f96d0146300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:52.689 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:52.684+0000 7f96d4f7f640 1 -- 172.21.15.169:0/992154882 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f96d00aa6d0 msgr2=0x7f96d0151040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:52.690 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:52.684+0000 7f96cf7fe640 1 --2- 172.21.15.169:0/992154882 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f96d00aa6d0 0x7f96d0151040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:52.690 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:52.684+0000 7f96d4f7f640 1 --2- 172.21.15.169:0/992154882 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f96d00aa6d0 0x7f96d0151040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:52.690 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:52.684+0000 7f96d4f7f640 1 -- 172.21.15.169:0/992154882 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96d006cba0 msgr2=0x7f96d0146300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:52.690 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:52.684+0000 7f96d4f7f640 1 --2- 172.21.15.169:0/992154882 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96d006cba0 0x7f96d0146300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:52.690 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:52.684+0000 7f96d4f7f640 1 -- 172.21.15.169:0/992154882 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96c4009560 con 0x7f96d00b0300 2023-10-26T01:37:52.691 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:52.684+0000 7f96cffff640 1 --2- 172.21.15.169:0/992154882 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96d006cba0 0x7f96d0146300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:37:52.691 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:52.684+0000 7f96d4f7f640 1 --2- 172.21.15.169:0/992154882 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f96d00b0300 0x7f96d0151580 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f96c800c780 tx=0x7f96c800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:52.691 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:52.685+0000 7f96ccff9640 1 -- 172.21.15.169:0/992154882 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c8003d80 con 0x7f96d00b0300 2023-10-26T01:37:52.691 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:52.685+0000 7f96ccff9640 1 -- 172.21.15.169:0/992154882 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f96c800ce90 con 0x7f96d00b0300 2023-10-26T01:37:52.691 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:52.685+0000 7f96ccff9640 1 -- 172.21.15.169:0/992154882 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c80052d0 con 0x7f96d00b0300 2023-10-26T01:37:52.691 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:52.685+0000 7f96d5780640 1 -- 172.21.15.169:0/992154882 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96d0158be0 con 0x7f96d00b0300 2023-10-26T01:37:52.692 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:52.685+0000 7f96d5780640 1 -- 172.21.15.169:0/992154882 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96d0146840 con 0x7f96d00b0300 2023-10-26T01:37:52.693 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:52.686+0000 7f96d5780640 1 -- 172.21.15.169:0/992154882 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96d0073c10 con 0x7f96d00b0300 2023-10-26T01:37:52.693 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:52.687+0000 7f96ccff9640 1 -- 172.21.15.169:0/992154882 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f96c8005430 con 0x7f96d00b0300 2023-10-26T01:37:52.693 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:52.688+0000 7f96ccff9640 1 --2- 172.21.15.169:0/992154882 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f96bc06ae40 0x7f96bc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:52.693 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:52.688+0000 7f96ccff9640 1 -- 172.21.15.169:0/992154882 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f96c8088d50 con 0x7f96d00b0300 2023-10-26T01:37:52.693 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:52.689+0000 7f96cffff640 1 --2- 172.21.15.169:0/992154882 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f96bc06ae40 0x7f96bc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:52.694 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:52.690+0000 7f96cffff640 1 --2- 172.21.15.169:0/992154882 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f96bc06ae40 0x7f96bc06d300 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f96c00058c0 tx=0x7f96c0005cb0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:52.695 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:52.692+0000 7f96ccff9640 1 -- 172.21.15.169:0/992154882 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f96c808c030 con 0x7f96d00b0300 2023-10-26T01:37:52.894 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:52.891+0000 7f96d5780640 1 -- 172.21.15.169:0/992154882 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd set", "key": "pause"} v 0) v1 -- 0x7f96d00ac910 con 0x7f96d00b0300 2023-10-26T01:37:53.094 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:53.091+0000 7f96ccff9640 1 -- 172.21.15.169:0/992154882 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "pause"}]=0 pauserd,pausewr is set v122) v1 ==== 93+0+0 (secure 0 0 0) 0x7f96c8017080 con 0x7f96d00b0300 2023-10-26T01:37:53.095 INFO:teuthology.orchestra.run.smithi169.stderr:pauserd,pausewr is set 2023-10-26T01:37:53.097 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:53.094+0000 7f96d5780640 1 -- 172.21.15.169:0/992154882 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f96bc06ae40 msgr2=0x7f96bc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:53.098 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:53.094+0000 7f96d5780640 1 --2- 172.21.15.169:0/992154882 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f96bc06ae40 0x7f96bc06d300 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f96c00058c0 tx=0x7f96c0005cb0 comp rx=0 tx=0).stop 2023-10-26T01:37:53.098 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:53.094+0000 7f96d5780640 1 -- 172.21.15.169:0/992154882 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f96d00b0300 msgr2=0x7f96d0151580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:53.099 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:53.094+0000 7f96d5780640 1 --2- 172.21.15.169:0/992154882 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f96d00b0300 0x7f96d0151580 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f96c800c780 tx=0x7f96c800cc40 comp rx=0 tx=0).stop 2023-10-26T01:37:53.099 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:53.094+0000 7f96d5780640 1 -- 172.21.15.169:0/992154882 shutdown_connections 2023-10-26T01:37:53.099 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:53.094+0000 7f96d5780640 1 --2- 172.21.15.169:0/992154882 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f96bc06ae40 0x7f96bc06d300 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:53.100 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:53.094+0000 7f96d5780640 1 --2- 172.21.15.169:0/992154882 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f96d00b0300 0x7f96d0151580 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:53.100 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:53.094+0000 7f96d5780640 1 --2- 172.21.15.169:0/992154882 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f96d00aa6d0 0x7f96d0151040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:53.100 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:53.094+0000 7f96d5780640 1 --2- 172.21.15.169:0/992154882 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96d006cba0 0x7f96d0146300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:53.101 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:53.094+0000 7f96d5780640 1 -- 172.21.15.169:0/992154882 >> 172.21.15.169:0/992154882 conn(0x7f96d00af110 msgr2=0x7f96d00ace10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:53.101 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:53.095+0000 7f96d5780640 1 -- 172.21.15.169:0/992154882 shutdown_connections 2023-10-26T01:37:53.101 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:53.095+0000 7f96d5780640 1 -- 172.21.15.169:0/992154882 wait complete. 2023-10-26T01:37:53.119 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset pause 2023-10-26T01:37:53.333 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:53.330+0000 7fe2abe4b640 1 -- 172.21.15.169:0/585616131 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe290003ef0 msgr2=0x7fe2900047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:53.334 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:53.330+0000 7fe2abe4b640 1 --2- 172.21.15.169:0/585616131 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe290003ef0 0x7fe2900047c0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7fe298007540 tx=0x7fe2900057f0 comp rx=0 tx=0).stop 2023-10-26T01:37:53.334 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:53.330+0000 7fe2aa3e8640 1 -- 172.21.15.169:0/585616131 reap_dead start 2023-10-26T01:37:53.335 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:53.330+0000 7fe2abe4b640 1 -- 172.21.15.169:0/585616131 shutdown_connections 2023-10-26T01:37:53.335 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:53.330+0000 7fe2abe4b640 1 -- 172.21.15.169:0/585616131 >> 172.21.15.169:0/585616131 conn(0x7fe2a4069930 msgr2=0x7fe2a4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:53.335 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:53.330+0000 7fe2abe4b640 1 -- 172.21.15.169:0/585616131 shutdown_connections 2023-10-26T01:37:53.336 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:53.330+0000 7fe2abe4b640 1 -- 172.21.15.169:0/585616131 wait complete. 2023-10-26T01:37:53.339 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:53.332+0000 7fe2abe4b640 1 Processor -- start 2023-10-26T01:37:53.339 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:53.332+0000 7fe2abe4b640 1 -- start start 2023-10-26T01:37:53.340 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:53.333+0000 7fe2abe4b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe290003ef0 0x7fe2a41443f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:53.340 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:53.333+0000 7fe2abe4b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe2a40cc720 0x7fe2a40990a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:53.342 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:53.333+0000 7fe2abe4b640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe2a40ccba0 0x7fe2a40995e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:53.342 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:53.333+0000 7fe2abe4b640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2a4097470 con 0x7fe290003ef0 2023-10-26T01:37:53.343 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:53.333+0000 7fe2abe4b640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2a4097710 con 0x7fe2a40cc720 2023-10-26T01:37:53.344 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:53.333+0000 7fe2abe4b640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2a40979f0 con 0x7fe2a40ccba0 2023-10-26T01:37:53.344 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:53.333+0000 7fe2a9be7640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe290003ef0 0x7fe2a41443f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:53.344 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:53.333+0000 7fe2a9be7640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe290003ef0 0x7fe2a41443f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:35872/0 (socket says 172.21.15.169:35872) 2023-10-26T01:37:53.344 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:53.333+0000 7fe2a9be7640 1 -- 172.21.15.169:0/1799874060 learned_addr learned my addr 172.21.15.169:0/1799874060 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:37:53.344 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:53.334+0000 7fe2a93e6640 1 --2- 172.21.15.169:0/1799874060 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe2a40cc720 0x7fe2a40990a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:53.345 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:53.334+0000 7fe2a9be7640 1 -- 172.21.15.169:0/1799874060 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe2a40ccba0 msgr2=0x7fe2a40995e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:53.345 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:53.334+0000 7fe2a9be7640 1 --2- 172.21.15.169:0/1799874060 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe2a40ccba0 0x7fe2a40995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:53.345 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:53.334+0000 7fe2a9be7640 1 -- 172.21.15.169:0/1799874060 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe2a40cc720 msgr2=0x7fe2a40990a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:53.345 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:53.334+0000 7fe2a9be7640 1 --2- 172.21.15.169:0/1799874060 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe2a40cc720 0x7fe2a40990a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:53.345 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:53.334+0000 7fe2a9be7640 1 -- 172.21.15.169:0/1799874060 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2a4097cd0 con 0x7fe290003ef0 2023-10-26T01:37:53.346 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:53.334+0000 7fe2a9be7640 1 --2- 172.21.15.169:0/1799874060 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe290003ef0 0x7fe2a41443f0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7fe2a0000ff0 tx=0x7fe2a000cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:53.346 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:53.334+0000 7fe2967fc640 1 -- 172.21.15.169:0/1799874060 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a0005270 con 0x7fe290003ef0 2023-10-26T01:37:53.346 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:53.334+0000 7fe2967fc640 1 -- 172.21.15.169:0/1799874060 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe2a00035f0 con 0x7fe290003ef0 2023-10-26T01:37:53.346 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:53.334+0000 7fe2967fc640 1 -- 172.21.15.169:0/1799874060 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a0003a00 con 0x7fe290003ef0 2023-10-26T01:37:53.346 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:53.334+0000 7fe2abe4b640 1 -- 172.21.15.169:0/1799874060 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2a4147260 con 0x7fe290003ef0 2023-10-26T01:37:53.347 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:53.335+0000 7fe2abe4b640 1 -- 172.21.15.169:0/1799874060 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2a4099c10 con 0x7fe290003ef0 2023-10-26T01:37:53.347 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:53.336+0000 7fe2abe4b640 1 -- 172.21.15.169:0/1799874060 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2a4097f10 con 0x7fe290003ef0 2023-10-26T01:37:53.347 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:53.337+0000 7fe2967fc640 1 -- 172.21.15.169:0/1799874060 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fe2a000a5c0 con 0x7fe290003ef0 2023-10-26T01:37:53.348 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:53.337+0000 7fe2967fc640 1 --2- 172.21.15.169:0/1799874060 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe28806ae40 0x7fe28806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:53.348 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:53.338+0000 7fe2967fc640 1 -- 172.21.15.169:0/1799874060 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe2a0089310 con 0x7fe290003ef0 2023-10-26T01:37:53.348 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:53.338+0000 7fe2967fc640 1 -- 172.21.15.169:0/1799874060 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=123+}) v3 -- 0x7fe288072970 con 0x7fe290003ef0 2023-10-26T01:37:53.349 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:53.338+0000 7fe2a93e6640 1 --2- 172.21.15.169:0/1799874060 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe28806ae40 0x7fe28806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:53.349 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:53.339+0000 7fe2a93e6640 1 --2- 172.21.15.169:0/1799874060 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe28806ae40 0x7fe28806d300 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fe298036000 tx=0x7fe29800f040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:53.349 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:53.342+0000 7fe2967fc640 1 -- 172.21.15.169:0/1799874060 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe2a4097f10 con 0x7fe290003ef0 2023-10-26T01:37:53.546 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:53.543+0000 7fe2abe4b640 1 -- 172.21.15.169:0/1799874060 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd unset", "key": "pause"} v 0) v1 -- 0x7fe2a4097ec0 con 0x7fe290003ef0 2023-10-26T01:37:54.109 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:54.105+0000 7fe2967fc640 1 -- 172.21.15.169:0/1799874060 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "pause"}]=0 pauserd,pausewr is unset v123) v1 ==== 97+0+0 (secure 0 0 0) 0x7fe2a0017030 con 0x7fe290003ef0 2023-10-26T01:37:54.110 INFO:teuthology.orchestra.run.smithi169.stderr:pauserd,pausewr is unset 2023-10-26T01:37:54.110 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:54.105+0000 7fe2967fc640 1 -- 172.21.15.169:0/1799874060 <== mon.0 v2:172.21.15.169:3300/0 8 ==== osd_map(123..123 src has 1..123) v4 ==== 288+0+0 (secure 0 0 0) 0x7fe2a00586e0 con 0x7fe290003ef0 2023-10-26T01:37:54.110 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:54.106+0000 7fe2967fc640 1 -- 172.21.15.169:0/1799874060 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=124}) v3 -- 0x7fe288073370 con 0x7fe290003ef0 2023-10-26T01:37:54.112 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:54.110+0000 7fe2abe4b640 1 -- 172.21.15.169:0/1799874060 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe28806ae40 msgr2=0x7fe28806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:54.112 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:54.110+0000 7fe2abe4b640 1 --2- 172.21.15.169:0/1799874060 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe28806ae40 0x7fe28806d300 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fe298036000 tx=0x7fe29800f040 comp rx=0 tx=0).stop 2023-10-26T01:37:54.113 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:54.110+0000 7fe2abe4b640 1 -- 172.21.15.169:0/1799874060 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe290003ef0 msgr2=0x7fe2a41443f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:54.113 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:54.110+0000 7fe2abe4b640 1 --2- 172.21.15.169:0/1799874060 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe290003ef0 0x7fe2a41443f0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7fe2a0000ff0 tx=0x7fe2a000cb90 comp rx=0 tx=0).stop 2023-10-26T01:37:54.113 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:54.111+0000 7fe2abe4b640 1 -- 172.21.15.169:0/1799874060 shutdown_connections 2023-10-26T01:37:54.114 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:54.111+0000 7fe2abe4b640 1 --2- 172.21.15.169:0/1799874060 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe28806ae40 0x7fe28806d300 secure :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fe298036000 tx=0x7fe29800f040 comp rx=0 tx=0).stop 2023-10-26T01:37:54.114 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:54.111+0000 7fe2abe4b640 1 --2- 172.21.15.169:0/1799874060 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe2a40ccba0 0x7fe2a40995e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:54.114 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:54.111+0000 7fe2abe4b640 1 --2- 172.21.15.169:0/1799874060 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe2a40cc720 0x7fe2a40990a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:54.114 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:54.111+0000 7fe2abe4b640 1 --2- 172.21.15.169:0/1799874060 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe290003ef0 0x7fe2a41443f0 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:54.115 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:54.111+0000 7fe2abe4b640 1 -- 172.21.15.169:0/1799874060 >> 172.21.15.169:0/1799874060 conn(0x7fe2a4069930 msgr2=0x7fe2a40ad920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:54.115 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:54.111+0000 7fe2abe4b640 1 -- 172.21.15.169:0/1799874060 shutdown_connections 2023-10-26T01:37:54.115 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:54.112+0000 7fe2abe4b640 1 -- 172.21.15.169:0/1799874060 wait complete. 2023-10-26T01:37:54.134 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:37:54.343 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:54.340+0000 7f6c8e0c3640 1 -- 172.21.15.169:0/100187200 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c880c7ea0 msgr2=0x7f6c8806a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:54.343 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:54.340+0000 7f6c8e0c3640 1 --2- 172.21.15.169:0/100187200 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c880c7ea0 0x7f6c8806a990 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f6c880caf70 tx=0x7f6c8402bd90 comp rx=0 tx=0).stop 2023-10-26T01:37:54.344 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:54.340+0000 7f6c8e0c3640 1 -- 172.21.15.169:0/100187200 shutdown_connections 2023-10-26T01:37:54.344 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:54.340+0000 7f6c8e0c3640 1 --2- 172.21.15.169:0/100187200 >> v2:172.21.15.170:3300/0 conn(0x7f6c880c96f0 0x7f6c880c9ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:54.344 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:54.340+0000 7f6c8e0c3640 1 --2- 172.21.15.169:0/100187200 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6c880cc4f0 0x7f6c880cc970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:54.344 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:54.340+0000 7f6c8e0c3640 1 --2- 172.21.15.169:0/100187200 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c880c7ea0 0x7f6c8806a990 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:54.345 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:54.340+0000 7f6c8e0c3640 1 -- 172.21.15.169:0/100187200 >> 172.21.15.169:0/100187200 conn(0x7f6c880aedd0 msgr2=0x7f6c880af1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:54.345 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:54.340+0000 7f6c8e0c3640 1 -- 172.21.15.169:0/100187200 shutdown_connections 2023-10-26T01:37:54.345 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:54.340+0000 7f6c8e0c3640 1 -- 172.21.15.169:0/100187200 wait complete. 2023-10-26T01:37:54.347 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:54.342+0000 7f6c8e0c3640 1 Processor -- start 2023-10-26T01:37:54.348 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:54.342+0000 7f6c8e0c3640 1 -- start start 2023-10-26T01:37:54.348 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:54.343+0000 7f6c8e0c3640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6c880c7ea0 0x7f6c880ad690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:54.348 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:54.343+0000 7f6c8e0c3640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c880c96f0 0x7f6c8809a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:54.348 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:54.343+0000 7f6c8e0c3640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6c880cc4f0 0x7f6c881542a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:54.349 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:54.343+0000 7f6c8e0c3640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c8809adf0 con 0x7f6c880c96f0 2023-10-26T01:37:54.349 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:54.343+0000 7f6c8e0c3640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c88099bc0 con 0x7f6c880cc4f0 2023-10-26T01:37:54.349 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:54.343+0000 7f6c8e0c3640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c88154db0 con 0x7f6c880c7ea0 2023-10-26T01:37:54.349 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:54.343+0000 7f6c8c8c0640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c880c96f0 0x7f6c8809a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:54.349 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:54.343+0000 7f6c8c8c0640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c880c96f0 0x7f6c8809a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:48924/0 (socket says 172.21.15.169:48924) 2023-10-26T01:37:54.350 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:54.343+0000 7f6c8c8c0640 1 -- 172.21.15.169:0/3726679419 learned_addr learned my addr 172.21.15.169:0/3726679419 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:37:54.350 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:54.344+0000 7f6c8d0c1640 1 --2- 172.21.15.169:0/3726679419 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6c880c7ea0 0x7f6c880ad690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:54.350 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:54.344+0000 7f6c8d8c2640 1 --2- 172.21.15.169:0/3726679419 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6c880cc4f0 0x7f6c881542a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:54.350 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:54.344+0000 7f6c8c8c0640 1 -- 172.21.15.169:0/3726679419 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6c880c7ea0 msgr2=0x7f6c880ad690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:54.350 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:54.344+0000 7f6c8c8c0640 1 --2- 172.21.15.169:0/3726679419 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6c880c7ea0 0x7f6c880ad690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:54.351 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:54.344+0000 7f6c8c8c0640 1 -- 172.21.15.169:0/3726679419 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6c880cc4f0 msgr2=0x7f6c881542a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:54.351 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:54.344+0000 7f6c8c8c0640 1 --2- 172.21.15.169:0/3726679419 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6c880cc4f0 0x7f6c881542a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:54.351 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:54.344+0000 7f6c8c8c0640 1 -- 172.21.15.169:0/3726679419 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c84009560 con 0x7f6c880c96f0 2023-10-26T01:37:54.351 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:54.344+0000 7f6c8d0c1640 1 --2- 172.21.15.169:0/3726679419 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6c880c7ea0 0x7f6c880ad690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:37:54.351 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:54.344+0000 7f6c8c8c0640 1 --2- 172.21.15.169:0/3726679419 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c880c96f0 0x7f6c8809a8b0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f6c7800c970 tx=0x7f6c7800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:54.352 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:54.344+0000 7f6c7dffb640 1 -- 172.21.15.169:0/3726679419 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c78007bb0 con 0x7f6c880c96f0 2023-10-26T01:37:54.352 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:54.345+0000 7f6c7dffb640 1 -- 172.21.15.169:0/3726679419 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6c78007e00 con 0x7f6c880c96f0 2023-10-26T01:37:54.352 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:54.345+0000 7f6c7dffb640 1 -- 172.21.15.169:0/3726679419 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c78005280 con 0x7f6c880c96f0 2023-10-26T01:37:54.352 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:54.345+0000 7f6c8e0c3640 1 -- 172.21.15.169:0/3726679419 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c88154ad0 con 0x7f6c880c96f0 2023-10-26T01:37:54.352 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:54.345+0000 7f6c8e0c3640 1 -- 172.21.15.169:0/3726679419 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c8809a0b0 con 0x7f6c880c96f0 2023-10-26T01:37:54.353 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:54.346+0000 7f6c8e0c3640 1 -- 172.21.15.169:0/3726679419 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c88073c10 con 0x7f6c880c96f0 2023-10-26T01:37:54.354 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:54.347+0000 7f6c7dffb640 1 -- 172.21.15.169:0/3726679419 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f6c780053e0 con 0x7f6c880c96f0 2023-10-26T01:37:54.354 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:54.348+0000 7f6c7dffb640 1 --2- 172.21.15.169:0/3726679419 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f6c6c06ae40 0x7f6c6c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:54.354 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:54.348+0000 7f6c7dffb640 1 -- 172.21.15.169:0/3726679419 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6c78089b80 con 0x7f6c880c96f0 2023-10-26T01:37:54.354 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:54.348+0000 7f6c8d0c1640 1 --2- 172.21.15.169:0/3726679419 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f6c6c06ae40 0x7f6c6c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:54.354 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:54.349+0000 7f6c8d0c1640 1 --2- 172.21.15.169:0/3726679419 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f6c6c06ae40 0x7f6c6c06d300 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f6c84035000 tx=0x7f6c84004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:54.355 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:54.352+0000 7f6c7dffb640 1 -- 172.21.15.169:0/3726679419 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6c78054c50 con 0x7f6c880c96f0 2023-10-26T01:37:54.544 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:54.541+0000 7f6c8e0c3640 1 -- 172.21.15.169:0/3726679419 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6c8806c790 con 0x7f6c880c96f0 2023-10-26T01:37:54.547 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:54.544+0000 7f6c7dffb640 1 -- 172.21.15.169:0/3726679419 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v123) v1 ==== 74+0+16575 (secure 0 0 0) 0x7f6c78089e60 con 0x7f6c880c96f0 2023-10-26T01:37:54.547 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:37:54.548 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":123,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:37:54.102485+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-10-26T01:37:37.959102+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-10-26T01:37:39.053240+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:37:54.550 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:54.547+0000 7f6c8e0c3640 1 -- 172.21.15.169:0/3726679419 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f6c6c06ae40 msgr2=0x7f6c6c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:54.550 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:54.547+0000 7f6c8e0c3640 1 --2- 172.21.15.169:0/3726679419 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f6c6c06ae40 0x7f6c6c06d300 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f6c84035000 tx=0x7f6c84004730 comp rx=0 tx=0).stop 2023-10-26T01:37:54.550 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:54.547+0000 7f6c8e0c3640 1 -- 172.21.15.169:0/3726679419 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c880c96f0 msgr2=0x7f6c8809a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:54.551 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:54.547+0000 7f6c8e0c3640 1 --2- 172.21.15.169:0/3726679419 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c880c96f0 0x7f6c8809a8b0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f6c7800c970 tx=0x7f6c7800ce30 comp rx=0 tx=0).stop 2023-10-26T01:37:54.551 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:54.547+0000 7f6c8e0c3640 1 -- 172.21.15.169:0/3726679419 shutdown_connections 2023-10-26T01:37:54.551 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:54.547+0000 7f6c8e0c3640 1 --2- 172.21.15.169:0/3726679419 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f6c6c06ae40 0x7f6c6c06d300 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:54.551 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:54.547+0000 7f6c8e0c3640 1 --2- 172.21.15.169:0/3726679419 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c880c96f0 0x7f6c8809a8b0 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:54.551 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:54.547+0000 7f6c8e0c3640 1 --2- 172.21.15.169:0/3726679419 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6c880cc4f0 0x7f6c881542a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:54.551 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:54.547+0000 7f6c8e0c3640 1 --2- 172.21.15.169:0/3726679419 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6c880c7ea0 0x7f6c880ad690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:54.552 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:54.547+0000 7f6c8e0c3640 1 -- 172.21.15.169:0/3726679419 >> 172.21.15.169:0/3726679419 conn(0x7f6c880aedd0 msgr2=0x7f6c881557c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:54.552 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:54.547+0000 7f6c8e0c3640 1 -- 172.21.15.169:0/3726679419 shutdown_connections 2023-10-26T01:37:54.552 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:54.547+0000 7f6c8e0c3640 1 -- 172.21.15.169:0/3726679419 wait complete. 2023-10-26T01:37:54.569 DEBUG:teuthology.orchestra.run.smithi169:> stdin-killer -- python3 -c ' 2023-10-26T01:37:54.570 DEBUG:teuthology.orchestra.run.smithi169:> import os 2023-10-26T01:37:54.570 DEBUG:teuthology.orchestra.run.smithi169:> fd = os.open("/home/ubuntu/cephtest/mnt.0/foo", os.O_CREAT | os.O_RDWR, 0O600) 2023-10-26T01:37:54.570 DEBUG:teuthology.orchestra.run.smithi169:> os.fchmod(fd, 0O666) 2023-10-26T01:37:54.571 DEBUG:teuthology.orchestra.run.smithi169:> os.fsync(fd) 2023-10-26T01:37:54.571 DEBUG:teuthology.orchestra.run.smithi169:> os.close(fd) 2023-10-26T01:37:54.571 DEBUG:teuthology.orchestra.run.smithi169:> ' 2023-10-26T01:37:54.716 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T01:37:54.726 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:37:54.727 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-26T01:37:54.830 INFO:teuthology.orchestra.run.smithi169.stdout:client5764 2023-10-26T01:37:54.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:37:54.831 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/30203b28-ed8a-44e8-a9a6-a9bf25f980f1.client5764/osdmap') 2023-10-26T01:37:54.927 INFO:teuthology.orchestra.run.smithi169.stdout:epoch 120 barrier 120 flags 0x588000 2023-10-26T01:37:54.927 INFO:teuthology.orchestra.run.smithi169.stdout:pool 1 '.mgr' type 1 size 2 min_size 1 pg_num 1 pg_num_mask 0 flags 0x1 lfor 0 read_tier -1 write_tier -1 2023-10-26T01:37:54.929 INFO:teuthology.orchestra.run.smithi169.stdout:pool 14 'cephfs_metadata' type 1 size 2 min_size 1 pg_num 32 pg_num_mask 31 flags 0x1 lfor 0 read_tier -1 write_tier -1 2023-10-26T01:37:54.929 INFO:teuthology.orchestra.run.smithi169.stdout:pool 15 'cephfs_data' type 1 size 2 min_size 1 pg_num 64 pg_num_mask 63 flags 0x1 lfor 0 read_tier -1 write_tier -1 2023-10-26T01:37:54.929 INFO:teuthology.orchestra.run.smithi169.stdout:osd0 (1)172.21.15.169:6817 100% (exists, up) 100% -1 2023-10-26T01:37:54.930 INFO:teuthology.orchestra.run.smithi169.stdout:osd1 (1)172.21.15.169:6809 100% (exists, up) 100% -1 2023-10-26T01:37:54.930 INFO:teuthology.orchestra.run.smithi169.stdout:osd2 (1)172.21.15.169:6801 100% (exists, up) 100% -1 2023-10-26T01:37:54.930 INFO:teuthology.orchestra.run.smithi169.stdout:osd3 (1)172.21.15.169:6825 100% (exists, up) 100% -1 2023-10-26T01:37:54.930 INFO:teuthology.orchestra.run.smithi169.stdout:osd4 (1)172.21.15.170:6817 100% (exists, up) 100% -1 2023-10-26T01:37:54.931 INFO:teuthology.orchestra.run.smithi169.stdout:osd5 (1)172.21.15.170:6809 100% (exists, up) 100% -1 2023-10-26T01:37:54.931 INFO:teuthology.orchestra.run.smithi169.stdout:osd6 (1)172.21.15.170:6801 100% (exists, up) 100% -1 2023-10-26T01:37:54.931 INFO:teuthology.orchestra.run.smithi169.stdout:osd7 (1)172.21.15.170:6822 100% (exists, up) 100% -1 2023-10-26T01:37:54.932 INFO:teuthology.orchestra.run.smithi169.stderr:1+1 records in 2023-10-26T01:37:54.932 INFO:teuthology.orchestra.run.smithi169.stderr:1+1 records out 2023-10-26T01:37:54.932 INFO:teuthology.orchestra.run.smithi169.stderr:803 bytes copied, 8.4115e-05 s, 9.5 MB/s 2023-10-26T01:37:54.933 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T01:37:55.155 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:55.152+0000 7f6739f4c640 1 -- 172.21.15.169:0/3926780853 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6734158850 msgr2=0x7f673414e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:55.155 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:55.152+0000 7f6739f4c640 1 --2- 172.21.15.169:0/3926780853 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6734158850 0x7f673414e790 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f6728005fd0 tx=0x7f672802bd90 comp rx=0 tx=0).stop 2023-10-26T01:37:55.155 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:55.152+0000 7f6739f4c640 1 -- 172.21.15.169:0/3926780853 shutdown_connections 2023-10-26T01:37:55.155 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:55.152+0000 7f6739f4c640 1 --2- 172.21.15.169:0/3926780853 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f67340cd720 0x7f67340cdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:55.156 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:55.152+0000 7f6739f4c640 1 --2- 172.21.15.169:0/3926780853 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6734158850 0x7f673414e790 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:55.156 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:55.152+0000 7f6739f4c640 1 -- 172.21.15.169:0/3926780853 >> 172.21.15.169:0/3926780853 conn(0x7f6734069930 msgr2=0x7f6734069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:55.156 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:55.152+0000 7f6739f4c640 1 -- 172.21.15.169:0/3926780853 shutdown_connections 2023-10-26T01:37:55.156 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:55.152+0000 7f6739f4c640 1 -- 172.21.15.169:0/3926780853 wait complete. 2023-10-26T01:37:55.157 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:55.154+0000 7f6739f4c640 1 Processor -- start 2023-10-26T01:37:55.157 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:55.154+0000 7f6739f4c640 1 -- start start 2023-10-26T01:37:55.159 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:55.155+0000 7f6739f4c640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f673406ce20 0x7f67340b1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:55.159 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:55.155+0000 7f6739f4c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67340cd720 0x7f67341447c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:55.159 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:55.155+0000 7f6739f4c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6734158850 0x7f67340af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:55.159 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:55.155+0000 7f6739f4c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67340b05f0 con 0x7f6734158850 2023-10-26T01:37:55.160 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:55.156+0000 7f6739f4c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67340b1170 con 0x7f67340cd720 2023-10-26T01:37:55.160 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:55.156+0000 7f6739f4c640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67340b0840 con 0x7f673406ce20 2023-10-26T01:37:55.160 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:55.156+0000 7f6733fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6734158850 0x7f67340af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:55.160 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:55.156+0000 7f6733fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6734158850 0x7f67340af8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:48940/0 (socket says 172.21.15.169:48940) 2023-10-26T01:37:55.160 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:55.156+0000 7f6733fff640 1 -- 172.21.15.169:0/1139599249 learned_addr learned my addr 172.21.15.169:0/1139599249 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:37:55.161 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:55.156+0000 7f67337fe640 1 --2- 172.21.15.169:0/1139599249 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f673406ce20 0x7f67340b1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:55.161 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:55.156+0000 7f6732ffd640 1 --2- 172.21.15.169:0/1139599249 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67340cd720 0x7f67341447c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:55.161 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:55.156+0000 7f6733fff640 1 -- 172.21.15.169:0/1139599249 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f673406ce20 msgr2=0x7f67340b1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:55.161 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:55.156+0000 7f6733fff640 1 --2- 172.21.15.169:0/1139599249 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f673406ce20 0x7f67340b1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:55.161 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:55.156+0000 7f6733fff640 1 -- 172.21.15.169:0/1139599249 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67340cd720 msgr2=0x7f67341447c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:55.162 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:55.156+0000 7f6733fff640 1 --2- 172.21.15.169:0/1139599249 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67340cd720 0x7f67341447c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:55.162 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:55.156+0000 7f6733fff640 1 -- 172.21.15.169:0/1139599249 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6728009560 con 0x7f6734158850 2023-10-26T01:37:55.162 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:55.156+0000 7f6732ffd640 1 --2- 172.21.15.169:0/1139599249 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67340cd720 0x7f67341447c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:37:55.162 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:55.156+0000 7f67337fe640 1 --2- 172.21.15.169:0/1139599249 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f673406ce20 0x7f67340b1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T01:37:55.163 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:55.157+0000 7f6733fff640 1 --2- 172.21.15.169:0/1139599249 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6734158850 0x7f67340af8d0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f672c00eb40 tx=0x7f672c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:55.164 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:55.157+0000 7f671bfff640 1 -- 172.21.15.169:0/1139599249 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f672c004b20 con 0x7f6734158850 2023-10-26T01:37:55.164 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:55.157+0000 7f671bfff640 1 -- 172.21.15.169:0/1139599249 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f672c004ca0 con 0x7f6734158850 2023-10-26T01:37:55.164 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:55.157+0000 7f6739f4c640 1 -- 172.21.15.169:0/1139599249 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67340b0b20 con 0x7f6734158850 2023-10-26T01:37:55.164 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:55.157+0000 7f671bfff640 1 -- 172.21.15.169:0/1139599249 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f672c005280 con 0x7f6734158850 2023-10-26T01:37:55.165 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:55.157+0000 7f6739f4c640 1 -- 172.21.15.169:0/1139599249 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6734161150 con 0x7f6734158850 2023-10-26T01:37:55.165 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:55.158+0000 7f6739f4c640 1 -- 172.21.15.169:0/1139599249 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f673404db10 con 0x7f6734158850 2023-10-26T01:37:55.165 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:55.159+0000 7f671bfff640 1 -- 172.21.15.169:0/1139599249 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f672c0053e0 con 0x7f6734158850 2023-10-26T01:37:55.165 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:55.160+0000 7f671bfff640 1 --2- 172.21.15.169:0/1139599249 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f672006ae70 0x7f672006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:55.165 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:55.160+0000 7f671bfff640 1 -- 172.21.15.169:0/1139599249 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f672c014070 con 0x7f6734158850 2023-10-26T01:37:55.166 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:55.161+0000 7f67337fe640 1 --2- 172.21.15.169:0/1139599249 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f672006ae70 0x7f672006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:55.166 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:55.162+0000 7f67337fe640 1 --2- 172.21.15.169:0/1139599249 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f672006ae70 0x7f672006d330 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f67341617f0 tx=0x7f6724009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:55.167 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:55.164+0000 7f671bfff640 1 -- 172.21.15.169:0/1139599249 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f672c055490 con 0x7f6734158850 2023-10-26T01:37:55.393 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:55.390+0000 7f6739f4c640 1 -- 172.21.15.169:0/1139599249 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f67340b12b0 con 0x7f6734158850 2023-10-26T01:37:55.394 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:55.391+0000 7f671bfff640 1 -- 172.21.15.169:0/1139599249 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) v1 ==== 94+0+4924 (secure 0 0 0) 0x7f672c058ab0 con 0x7f6734158850 2023-10-26T01:37:55.394 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:37:55.395 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":59,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5387,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/1186563921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1186563921},{"type":"v1","addr":"172.21.15.170:6835","nonce":1186563921}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":54},{"gid":5390,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/3702036693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":3702036693},{"type":"v1","addr":"172.21.15.170:6837","nonce":3702036693}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":54},{"gid":7332,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/3193888686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":3193888686},{"type":"v1","addr":"172.21.15.169:6833","nonce":3193888686}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T01:37:40.027858+0000","modified":"2023-10-26T01:37:43.103474+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7335},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7335":{"gid":7335,"name":"c","rank":0,"incarnation":55,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6835/668588916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":668588916},{"type":"v1","addr":"172.21.15.169:6835","nonce":668588916}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-10-26T01:37:55.395 INFO:teuthology.orchestra.run.smithi169.stderr:dumped fsmap epoch 59 2023-10-26T01:37:55.396 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:55.394+0000 7f6739f4c640 1 -- 172.21.15.169:0/1139599249 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f672006ae70 msgr2=0x7f672006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:55.397 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:55.394+0000 7f6739f4c640 1 --2- 172.21.15.169:0/1139599249 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f672006ae70 0x7f672006d330 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f67341617f0 tx=0x7f6724009040 comp rx=0 tx=0).stop 2023-10-26T01:37:55.397 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:55.394+0000 7f6739f4c640 1 -- 172.21.15.169:0/1139599249 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6734158850 msgr2=0x7f67340af8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:55.397 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:55.394+0000 7f6739f4c640 1 --2- 172.21.15.169:0/1139599249 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6734158850 0x7f67340af8d0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f672c00eb40 tx=0x7f672c00c2d0 comp rx=0 tx=0).stop 2023-10-26T01:37:55.397 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:55.394+0000 7f6739f4c640 1 -- 172.21.15.169:0/1139599249 shutdown_connections 2023-10-26T01:37:55.397 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:55.394+0000 7f6739f4c640 1 --2- 172.21.15.169:0/1139599249 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f672006ae70 0x7f672006d330 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:55.398 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:55.394+0000 7f6739f4c640 1 --2- 172.21.15.169:0/1139599249 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6734158850 0x7f67340af8d0 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:55.398 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:55.394+0000 7f6739f4c640 1 --2- 172.21.15.169:0/1139599249 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67340cd720 0x7f67341447c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:55.398 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:55.394+0000 7f6739f4c640 1 --2- 172.21.15.169:0/1139599249 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f673406ce20 0x7f67340b1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:55.398 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:55.394+0000 7f6739f4c640 1 -- 172.21.15.169:0/1139599249 >> 172.21.15.169:0/1139599249 conn(0x7f6734069930 msgr2=0x7f6734157080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:55.398 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:55.394+0000 7f6739f4c640 1 -- 172.21.15.169:0/1139599249 shutdown_connections 2023-10-26T01:37:55.399 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:55.394+0000 7f6739f4c640 1 -- 172.21.15.169:0/1139599249 wait complete. 2023-10-26T01:37:55.416 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json osdmap barrier 123 2023-10-26T01:37:55.649 DEBUG:tasks.cephfs.filesystem:_json_asok output empty 2023-10-26T01:37:55.650 DEBUG:teuthology.orchestra.run.smithi169:> stdin-killer -- python3 -c ' 2023-10-26T01:37:55.650 DEBUG:teuthology.orchestra.run.smithi169:> import os 2023-10-26T01:37:55.650 DEBUG:teuthology.orchestra.run.smithi169:> fd = os.open("/home/ubuntu/cephtest/mnt.0/baz", os.O_CREAT | os.O_RDWR, 0O600) 2023-10-26T01:37:55.651 DEBUG:teuthology.orchestra.run.smithi169:> os.fchmod(fd, 0O666) 2023-10-26T01:37:55.651 DEBUG:teuthology.orchestra.run.smithi169:> os.fsync(fd) 2023-10-26T01:37:55.651 DEBUG:teuthology.orchestra.run.smithi169:> os.close(fd) 2023-10-26T01:37:55.652 DEBUG:teuthology.orchestra.run.smithi169:> ' 2023-10-26T01:37:55.794 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T01:37:55.804 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:37:55.804 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-26T01:37:55.909 INFO:teuthology.orchestra.run.smithi169.stdout:client5764 2023-10-26T01:37:55.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:37:55.910 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/30203b28-ed8a-44e8-a9a6-a9bf25f980f1.client5764/osdmap') 2023-10-26T01:37:56.009 INFO:teuthology.orchestra.run.smithi169.stdout:epoch 123 barrier 123 flags 0x588000 2023-10-26T01:37:56.009 INFO:teuthology.orchestra.run.smithi169.stdout:pool 1 '.mgr' type 1 size 2 min_size 1 pg_num 1 pg_num_mask 0 flags 0x1 lfor 0 read_tier -1 write_tier -1 2023-10-26T01:37:56.010 INFO:teuthology.orchestra.run.smithi169.stdout:pool 14 'cephfs_metadata' type 1 size 2 min_size 1 pg_num 32 pg_num_mask 31 flags 0x1 lfor 0 read_tier -1 write_tier -1 2023-10-26T01:37:56.010 INFO:teuthology.orchestra.run.smithi169.stdout:pool 15 'cephfs_data' type 1 size 2 min_size 1 pg_num 64 pg_num_mask 63 flags 0x1 lfor 0 read_tier -1 write_tier -1 2023-10-26T01:37:56.010 INFO:teuthology.orchestra.run.smithi169.stdout:osd0 (1)172.21.15.169:6817 100% (exists, up) 100% -1 2023-10-26T01:37:56.010 INFO:teuthology.orchestra.run.smithi169.stdout:osd1 (1)172.21.15.169:6809 100% (exists, up) 100% -1 2023-10-26T01:37:56.010 INFO:teuthology.orchestra.run.smithi169.stdout:osd2 (1)172.21.15.169:6801 100% (exists, up) 100% -1 2023-10-26T01:37:56.011 INFO:teuthology.orchestra.run.smithi169.stdout:osd3 (1)172.21.15.169:6825 100% (exists, up) 100% -1 2023-10-26T01:37:56.011 INFO:teuthology.orchestra.run.smithi169.stdout:osd4 (1)172.21.15.170:6817 100% (exists, up) 100% -1 2023-10-26T01:37:56.011 INFO:teuthology.orchestra.run.smithi169.stdout:osd5 (1)172.21.15.170:6809 100% (exists, up) 100% -1 2023-10-26T01:37:56.011 INFO:teuthology.orchestra.run.smithi169.stdout:osd6 (1)172.21.15.170:6801 100% (exists, up) 100% -1 2023-10-26T01:37:56.011 INFO:teuthology.orchestra.run.smithi169.stdout:osd7 (1)172.21.15.170:6822 100% (exists, up) 100% -1 2023-10-26T01:37:56.012 INFO:teuthology.orchestra.run.smithi169.stderr:1+1 records in 2023-10-26T01:37:56.012 INFO:teuthology.orchestra.run.smithi169.stderr:1+1 records out 2023-10-26T01:37:56.012 INFO:teuthology.orchestra.run.smithi169.stderr:803 bytes copied, 0.000114512 s, 7.0 MB/s 2023-10-26T01:37:56.013 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:37:56.014 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-26T01:37:56.070 INFO:teuthology.orchestra.run.smithi169.stdout:client5764 2023-10-26T01:37:56.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:37:56.072 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/30203b28-ed8a-44e8-a9a6-a9bf25f980f1.client5764/osdmap') 2023-10-26T01:37:56.169 INFO:teuthology.orchestra.run.smithi169.stdout:epoch 123 barrier 123 flags 0x588000 2023-10-26T01:37:56.170 INFO:teuthology.orchestra.run.smithi169.stdout:pool 1 '.mgr' type 1 size 2 min_size 1 pg_num 1 pg_num_mask 0 flags 0x1 lfor 0 read_tier -1 write_tier -1 2023-10-26T01:37:56.170 INFO:teuthology.orchestra.run.smithi169.stdout:pool 14 'cephfs_metadata' type 1 size 2 min_size 1 pg_num 32 pg_num_mask 31 flags 0x1 lfor 0 read_tier -1 write_tier -1 2023-10-26T01:37:56.170 INFO:teuthology.orchestra.run.smithi169.stdout:pool 15 'cephfs_data' type 1 size 2 min_size 1 pg_num 64 pg_num_mask 63 flags 0x1 lfor 0 read_tier -1 write_tier -1 2023-10-26T01:37:56.170 INFO:teuthology.orchestra.run.smithi169.stdout:osd0 (1)172.21.15.169:6817 100% (exists, up) 100% -1 2023-10-26T01:37:56.170 INFO:teuthology.orchestra.run.smithi169.stdout:osd1 (1)172.21.15.169:6809 100% (exists, up) 100% -1 2023-10-26T01:37:56.171 INFO:teuthology.orchestra.run.smithi169.stdout:osd2 (1)172.21.15.169:6801 100% (exists, up) 100% -1 2023-10-26T01:37:56.171 INFO:teuthology.orchestra.run.smithi169.stdout:osd3 (1)172.21.15.169:6825 100% (exists, up) 100% -1 2023-10-26T01:37:56.171 INFO:teuthology.orchestra.run.smithi169.stdout:osd4 (1)172.21.15.170:6817 100% (exists, up) 100% -1 2023-10-26T01:37:56.171 INFO:teuthology.orchestra.run.smithi169.stdout:osd5 (1)172.21.15.170:6809 100% (exists, up) 100% -1 2023-10-26T01:37:56.171 INFO:teuthology.orchestra.run.smithi169.stdout:osd6 (1)172.21.15.170:6801 100% (exists, up) 100% -1 2023-10-26T01:37:56.172 INFO:teuthology.orchestra.run.smithi169.stdout:osd7 (1)172.21.15.170:6822 100% (exists, up) 100% -1 2023-10-26T01:37:56.172 INFO:teuthology.orchestra.run.smithi169.stderr:1+1 records in 2023-10-26T01:37:56.172 INFO:teuthology.orchestra.run.smithi169.stderr:1+1 records out 2023-10-26T01:37:56.172 INFO:teuthology.orchestra.run.smithi169.stderr:803 bytes copied, 0.000101432 s, 7.9 MB/s 2023-10-26T01:37:56.173 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-26T01:37:56.174 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-26T01:37:56.174 DEBUG:teuthology.orchestra.run.smithi169:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-26T01:37:56.289 DEBUG:teuthology.orchestra.run.smithi170:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-26T01:37:56.408 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-26T01:37:56.409 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:37:56.409 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:37:56.471 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:37:56.471 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:37:56.525 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:37:56.754 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:56.750+0000 7f384ccb4640 1 -- 172.21.15.169:0/299984178 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f384806ce20 msgr2=0x7f384814e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:56.754 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:56.750+0000 7f384ccb4640 1 --2- 172.21.15.169:0/299984178 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f384806ce20 0x7f384814e790 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f3838009900 tx=0x7f383802bce0 comp rx=0 tx=0).stop 2023-10-26T01:37:56.754 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:56.751+0000 7f384ccb4640 1 -- 172.21.15.169:0/299984178 shutdown_connections 2023-10-26T01:37:56.754 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:56.751+0000 7f384ccb4640 1 --2- 172.21.15.169:0/299984178 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f38480cd720 0x7f38480cdbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:56.755 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:56.751+0000 7f384ccb4640 1 --2- 172.21.15.169:0/299984178 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f384806ce20 0x7f384814e790 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:56.755 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:56.751+0000 7f384ccb4640 1 -- 172.21.15.169:0/299984178 >> 172.21.15.169:0/299984178 conn(0x7f3848069930 msgr2=0x7f3848069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:56.755 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:56.751+0000 7f384ccb4640 1 -- 172.21.15.169:0/299984178 shutdown_connections 2023-10-26T01:37:56.755 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:56.751+0000 7f384ccb4640 1 -- 172.21.15.169:0/299984178 wait complete. 2023-10-26T01:37:56.757 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:56.753+0000 7f384ccb4640 1 Processor -- start 2023-10-26T01:37:56.757 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:56.753+0000 7f384ccb4640 1 -- start start 2023-10-26T01:37:56.757 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:56.754+0000 7f384ccb4640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f384806ce20 0x7f3848142740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:56.757 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:56.754+0000 7f384ccb4640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f38480cd720 0x7f3848149810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:56.758 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:56.754+0000 7f384ccb4640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3848158850 0x7f3848149d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:56.758 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:56.754+0000 7f384ccb4640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3848160250 con 0x7f384806ce20 2023-10-26T01:37:56.758 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:56.754+0000 7f384ccb4640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3848142c80 con 0x7f3848158850 2023-10-26T01:37:56.758 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:56.754+0000 7f384ccb4640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f384815fee0 con 0x7f38480cd720 2023-10-26T01:37:56.758 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:56.754+0000 7f384659c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f384806ce20 0x7f3848142740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:56.759 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:56.754+0000 7f384659c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f384806ce20 0x7f3848142740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:48944/0 (socket says 172.21.15.169:48944) 2023-10-26T01:37:56.759 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:56.754+0000 7f384659c640 1 -- 172.21.15.169:0/3587702789 learned_addr learned my addr 172.21.15.169:0/3587702789 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:37:56.759 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:56.754+0000 7f384659c640 1 -- 172.21.15.169:0/3587702789 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f38480cd720 msgr2=0x7f3848149810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:56.759 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:56.754+0000 7f384659c640 1 --2- 172.21.15.169:0/3587702789 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f38480cd720 0x7f3848149810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:56.759 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:56.754+0000 7f384659c640 1 -- 172.21.15.169:0/3587702789 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3848158850 msgr2=0x7f3848149d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:56.760 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:56.754+0000 7f384659c640 1 --2- 172.21.15.169:0/3587702789 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3848158850 0x7f3848149d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:56.760 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:56.754+0000 7f384659c640 1 -- 172.21.15.169:0/3587702789 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3838009560 con 0x7f384806ce20 2023-10-26T01:37:56.760 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:56.755+0000 7f384659c640 1 --2- 172.21.15.169:0/3587702789 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f384806ce20 0x7f3848142740 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f3838036040 tx=0x7f383802dcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:56.761 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:56.755+0000 7f3836ffd640 1 -- 172.21.15.169:0/3587702789 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f383802dea0 con 0x7f384806ce20 2023-10-26T01:37:56.761 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:56.755+0000 7f3836ffd640 1 -- 172.21.15.169:0/3587702789 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3838003800 con 0x7f384806ce20 2023-10-26T01:37:56.761 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:56.755+0000 7f3836ffd640 1 -- 172.21.15.169:0/3587702789 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3838003bf0 con 0x7f384806ce20 2023-10-26T01:37:56.762 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:56.755+0000 7f384ccb4640 1 -- 172.21.15.169:0/3587702789 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38480aeec0 con 0x7f384806ce20 2023-10-26T01:37:56.762 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:56.755+0000 7f384ccb4640 1 -- 172.21.15.169:0/3587702789 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38480af480 con 0x7f384806ce20 2023-10-26T01:37:56.762 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:56.756+0000 7f384ccb4640 1 -- 172.21.15.169:0/3587702789 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f384804db10 con 0x7f384806ce20 2023-10-26T01:37:56.762 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:56.757+0000 7f3836ffd640 1 -- 172.21.15.169:0/3587702789 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f3838003d50 con 0x7f384806ce20 2023-10-26T01:37:56.762 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:56.758+0000 7f3836ffd640 1 --2- 172.21.15.169:0/3587702789 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f382806ae40 0x7f382806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:56.763 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:56.758+0000 7f3836ffd640 1 -- 172.21.15.169:0/3587702789 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f38380abf90 con 0x7f384806ce20 2023-10-26T01:37:56.763 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:56.759+0000 7f3845d9b640 1 --2- 172.21.15.169:0/3587702789 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f382806ae40 0x7f382806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:56.763 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:56.760+0000 7f3845d9b640 1 --2- 172.21.15.169:0/3587702789 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f382806ae40 0x7f382806d300 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f38481621b0 tx=0x7f383c009250 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:56.765 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:56.762+0000 7f3836ffd640 1 -- 172.21.15.169:0/3587702789 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f38380deca0 con 0x7f384806ce20 2023-10-26T01:37:56.957 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:56.954+0000 7f384ccb4640 1 -- 172.21.15.169:0/3587702789 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f38480aca70 con 0x7f384806ce20 2023-10-26T01:37:56.958 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:56.956+0000 7f3836ffd640 1 -- 172.21.15.169:0/3587702789 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v123) v1 ==== 74+0+16575 (secure 0 0 0) 0x7f3838077160 con 0x7f384806ce20 2023-10-26T01:37:56.959 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:37:56.959 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":123,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:37:54.102485+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-10-26T01:37:37.959102+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-10-26T01:37:39.053240+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:37:56.962 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:56.958+0000 7f384ccb4640 1 -- 172.21.15.169:0/3587702789 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f382806ae40 msgr2=0x7f382806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:56.962 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:56.958+0000 7f384ccb4640 1 --2- 172.21.15.169:0/3587702789 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f382806ae40 0x7f382806d300 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f38481621b0 tx=0x7f383c009250 comp rx=0 tx=0).stop 2023-10-26T01:37:56.962 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:56.958+0000 7f384ccb4640 1 -- 172.21.15.169:0/3587702789 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f384806ce20 msgr2=0x7f3848142740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:56.962 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:56.958+0000 7f384ccb4640 1 --2- 172.21.15.169:0/3587702789 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f384806ce20 0x7f3848142740 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f3838036040 tx=0x7f383802dcb0 comp rx=0 tx=0).stop 2023-10-26T01:37:56.963 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:56.959+0000 7f384ccb4640 1 -- 172.21.15.169:0/3587702789 shutdown_connections 2023-10-26T01:37:56.963 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:56.959+0000 7f384ccb4640 1 --2- 172.21.15.169:0/3587702789 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f382806ae40 0x7f382806d300 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:56.963 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:56.959+0000 7f384ccb4640 1 --2- 172.21.15.169:0/3587702789 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3848158850 0x7f3848149d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:56.963 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:56.959+0000 7f384ccb4640 1 --2- 172.21.15.169:0/3587702789 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f38480cd720 0x7f3848149810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:56.963 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:56.959+0000 7f384ccb4640 1 --2- 172.21.15.169:0/3587702789 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f384806ce20 0x7f3848142740 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:56.964 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:56.959+0000 7f384ccb4640 1 -- 172.21.15.169:0/3587702789 >> 172.21.15.169:0/3587702789 conn(0x7f3848069930 msgr2=0x7f38480adef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:56.964 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:56.959+0000 7f384ccb4640 1 -- 172.21.15.169:0/3587702789 shutdown_connections 2023-10-26T01:37:56.964 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:56.959+0000 7f384ccb4640 1 -- 172.21.15.169:0/3587702789 wait complete. 2023-10-26T01:37:56.981 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T01:30:32.200808+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '109', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-26T01:37:37.959102+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '119', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2023-10-26T01:37:39.053240+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T01:37:56.982 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T01:37:57.208 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:57.204+0000 7f29e697d640 1 -- 172.21.15.169:0/670936514 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29cc003970 msgr2=0x7f29cc003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:57.208 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:57.204+0000 7f29e697d640 1 --2- 172.21.15.169:0/670936514 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29cc003970 0x7f29cc003d90 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f29d0006fd0 tx=0x7f29cc0057f0 comp rx=0 tx=0).stop 2023-10-26T01:37:57.208 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:57.205+0000 7f29e4f1a640 1 -- 172.21.15.169:0/670936514 reap_dead start 2023-10-26T01:37:57.208 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:57.205+0000 7f29e697d640 1 -- 172.21.15.169:0/670936514 shutdown_connections 2023-10-26T01:37:57.209 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:57.205+0000 7f29e697d640 1 -- 172.21.15.169:0/670936514 >> 172.21.15.169:0/670936514 conn(0x7f29e0069930 msgr2=0x7f29e0069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:57.209 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:57.205+0000 7f29e697d640 1 -- 172.21.15.169:0/670936514 shutdown_connections 2023-10-26T01:37:57.209 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:57.205+0000 7f29e697d640 1 -- 172.21.15.169:0/670936514 wait complete. 2023-10-26T01:37:57.209 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:57.207+0000 7f29e697d640 1 Processor -- start 2023-10-26T01:37:57.210 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:57.207+0000 7f29e697d640 1 -- start start 2023-10-26T01:37:57.211 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:57.208+0000 7f29e697d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29cc003970 0x7f29e0152310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:57.211 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:57.208+0000 7f29e697d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f29e00ae420 0x7f29e01e6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:57.212 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:57.208+0000 7f29e697d640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f29e01e6870 0x7f29e01e6d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:57.212 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:57.208+0000 7f29e697d640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29e00ae000 con 0x7f29cc003970 2023-10-26T01:37:57.212 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:57.208+0000 7f29e697d640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29e01e7ea0 con 0x7f29e00ae420 2023-10-26T01:37:57.212 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:57.208+0000 7f29e697d640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29e00ae8a0 con 0x7f29e01e6870 2023-10-26T01:37:57.212 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:57.208+0000 7f29dffff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29cc003970 0x7f29e0152310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:57.213 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:57.208+0000 7f29dffff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29cc003970 0x7f29e0152310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:48970/0 (socket says 172.21.15.169:48970) 2023-10-26T01:37:57.213 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:57.208+0000 7f29dffff640 1 -- 172.21.15.169:0/3316026500 learned_addr learned my addr 172.21.15.169:0/3316026500 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:37:57.213 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:57.208+0000 7f29dffff640 1 -- 172.21.15.169:0/3316026500 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f29e01e6870 msgr2=0x7f29e01e6d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:57.213 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:57.208+0000 7f29dffff640 1 --2- 172.21.15.169:0/3316026500 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f29e01e6870 0x7f29e01e6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:57.213 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:57.208+0000 7f29dffff640 1 -- 172.21.15.169:0/3316026500 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f29e00ae420 msgr2=0x7f29e01e6330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:57.214 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:57.208+0000 7f29dffff640 1 --2- 172.21.15.169:0/3316026500 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f29e00ae420 0x7f29e01e6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:57.214 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:57.208+0000 7f29dffff640 1 -- 172.21.15.169:0/3316026500 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29e00ad960 con 0x7f29cc003970 2023-10-26T01:37:57.214 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:57.209+0000 7f29dffff640 1 --2- 172.21.15.169:0/3316026500 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29cc003970 0x7f29e0152310 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f29d0000c00 tx=0x7f29d0001650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:57.215 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:57.209+0000 7f29dd7fa640 1 -- 172.21.15.169:0/3316026500 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d00062e0 con 0x7f29cc003970 2023-10-26T01:37:57.216 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:57.209+0000 7f29dd7fa640 1 -- 172.21.15.169:0/3316026500 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f29d0006440 con 0x7f29cc003970 2023-10-26T01:37:57.216 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:57.209+0000 7f29dd7fa640 1 -- 172.21.15.169:0/3316026500 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d00035f0 con 0x7f29cc003970 2023-10-26T01:37:57.216 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:57.209+0000 7f29e697d640 1 -- 172.21.15.169:0/3316026500 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29e01e49b0 con 0x7f29cc003970 2023-10-26T01:37:57.216 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:57.209+0000 7f29e697d640 1 -- 172.21.15.169:0/3316026500 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29e00adc20 con 0x7f29cc003970 2023-10-26T01:37:57.216 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:57.210+0000 7f29e697d640 1 -- 172.21.15.169:0/3316026500 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29e0066af0 con 0x7f29cc003970 2023-10-26T01:37:57.216 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:57.211+0000 7f29dd7fa640 1 -- 172.21.15.169:0/3316026500 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f29d0031030 con 0x7f29cc003970 2023-10-26T01:37:57.217 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:57.212+0000 7f29dd7fa640 1 --2- 172.21.15.169:0/3316026500 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f29c006ae70 0x7f29c006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:57.217 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:57.212+0000 7f29dd7fa640 1 -- 172.21.15.169:0/3316026500 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f29d00ab370 con 0x7f29cc003970 2023-10-26T01:37:57.217 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:57.213+0000 7f29d77fe640 1 --2- 172.21.15.169:0/3316026500 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f29c006ae70 0x7f29c006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:57.218 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:57.214+0000 7f29d77fe640 1 --2- 172.21.15.169:0/3316026500 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f29c006ae70 0x7f29c006d330 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f29c8005d80 tx=0x7f29c8005c90 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:57.219 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:57.216+0000 7f29dd7fa640 1 -- 172.21.15.169:0/3316026500 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f29d0079f40 con 0x7f29cc003970 2023-10-26T01:37:57.406 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:57.404+0000 7f29e697d640 1 -- 172.21.15.169:0/3316026500 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f29e0066af0 con 0x7f29cc003970 2023-10-26T01:37:57.407 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:57.404+0000 7f29dd7fa640 1 -- 172.21.15.169:0/3316026500 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v123) v1 ==== 93+0+10 (secure 0 0 0) 0x7f29d0079ad0 con 0x7f29cc003970 2023-10-26T01:37:57.408 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 1 2023-10-26T01:37:57.410 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:57.407+0000 7f29e697d640 1 -- 172.21.15.169:0/3316026500 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f29c006ae70 msgr2=0x7f29c006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:57.410 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:57.407+0000 7f29e697d640 1 --2- 172.21.15.169:0/3316026500 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f29c006ae70 0x7f29c006d330 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f29c8005d80 tx=0x7f29c8005c90 comp rx=0 tx=0).stop 2023-10-26T01:37:57.410 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:57.407+0000 7f29e697d640 1 -- 172.21.15.169:0/3316026500 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29cc003970 msgr2=0x7f29e0152310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:57.410 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:57.407+0000 7f29e697d640 1 --2- 172.21.15.169:0/3316026500 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29cc003970 0x7f29e0152310 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f29d0000c00 tx=0x7f29d0001650 comp rx=0 tx=0).stop 2023-10-26T01:37:57.411 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:57.407+0000 7f29e697d640 1 -- 172.21.15.169:0/3316026500 shutdown_connections 2023-10-26T01:37:57.411 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:57.407+0000 7f29e697d640 1 --2- 172.21.15.169:0/3316026500 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f29c006ae70 0x7f29c006d330 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:57.411 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:57.407+0000 7f29e697d640 1 --2- 172.21.15.169:0/3316026500 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f29e01e6870 0x7f29e01e6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:57.411 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:57.407+0000 7f29e697d640 1 --2- 172.21.15.169:0/3316026500 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f29e00ae420 0x7f29e01e6330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:57.411 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:57.407+0000 7f29e697d640 1 --2- 172.21.15.169:0/3316026500 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29cc003970 0x7f29e0152310 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:57.412 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:57.407+0000 7f29e697d640 1 -- 172.21.15.169:0/3316026500 >> 172.21.15.169:0/3316026500 conn(0x7f29e0069930 msgr2=0x7f29e00b08d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:57.412 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:57.407+0000 7f29e697d640 1 -- 172.21.15.169:0/3316026500 shutdown_connections 2023-10-26T01:37:57.412 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:57.407+0000 7f29e697d640 1 -- 172.21.15.169:0/3316026500 wait complete. 2023-10-26T01:37:57.428 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-26T01:37:57.657 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:57.654+0000 7fdbeaaca640 1 -- 172.21.15.169:0/78419397 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdbd0003970 msgr2=0x7fdbd0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:57.673 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:57.654+0000 7fdbeaaca640 1 --2- 172.21.15.169:0/78419397 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdbd0003970 0x7fdbd0003d90 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fdbe0001680 tx=0x7fdbd00057f0 comp rx=0 tx=0).stop 2023-10-26T01:37:57.674 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:57.654+0000 7fdbeaaca640 1 -- 172.21.15.169:0/78419397 shutdown_connections 2023-10-26T01:37:57.674 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:57.654+0000 7fdbeaaca640 1 --2- 172.21.15.169:0/78419397 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdbd0003970 0x7fdbd0003d90 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:57.674 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:57.654+0000 7fdbeaaca640 1 -- 172.21.15.169:0/78419397 >> 172.21.15.169:0/78419397 conn(0x7fdbe40af110 msgr2=0x7fdbe40af520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:57.674 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:57.654+0000 7fdbeaaca640 1 -- 172.21.15.169:0/78419397 shutdown_connections 2023-10-26T01:37:57.675 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:57.654+0000 7fdbeaaca640 1 -- 172.21.15.169:0/78419397 wait complete. 2023-10-26T01:37:57.677 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:57.656+0000 7fdbeaaca640 1 Processor -- start 2023-10-26T01:37:57.678 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:57.656+0000 7fdbeaaca640 1 -- start start 2023-10-26T01:37:57.678 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:57.657+0000 7fdbeaaca640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdbd0003970 0x7fdbe40cb120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:57.678 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:57.657+0000 7fdbeaaca640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdbe41504e0 0x7fdbe414b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:57.678 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:57.657+0000 7fdbeaaca640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdbe4159620 0x7fdbe4159aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:57.678 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:57.657+0000 7fdbeaaca640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe4159fe0 con 0x7fdbe4159620 2023-10-26T01:37:57.679 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:57.657+0000 7fdbeaaca640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe40cbf30 con 0x7fdbd0003970 2023-10-26T01:37:57.679 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:57.657+0000 7fdbeaaca640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe40cb970 con 0x7fdbe41504e0 2023-10-26T01:37:57.679 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:57.657+0000 7fdbea2c9640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdbe4159620 0x7fdbe4159aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:57.679 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:57.657+0000 7fdbea2c9640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdbe4159620 0x7fdbe4159aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:48990/0 (socket says 172.21.15.169:48990) 2023-10-26T01:37:57.680 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:57.657+0000 7fdbea2c9640 1 -- 172.21.15.169:0/1962061406 learned_addr learned my addr 172.21.15.169:0/1962061406 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:37:57.680 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:57.657+0000 7fdbe9ac8640 1 --2- 172.21.15.169:0/1962061406 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdbd0003970 0x7fdbe40cb120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:57.680 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:57.658+0000 7fdbe92c7640 1 --2- 172.21.15.169:0/1962061406 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdbe41504e0 0x7fdbe414b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:57.680 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:57.658+0000 7fdbea2c9640 1 -- 172.21.15.169:0/1962061406 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdbe41504e0 msgr2=0x7fdbe414b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:57.680 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:57.658+0000 7fdbea2c9640 1 --2- 172.21.15.169:0/1962061406 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdbe41504e0 0x7fdbe414b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:57.681 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:57.658+0000 7fdbea2c9640 1 -- 172.21.15.169:0/1962061406 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdbd0003970 msgr2=0x7fdbe40cb120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:57.681 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:57.658+0000 7fdbea2c9640 1 --2- 172.21.15.169:0/1962061406 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdbd0003970 0x7fdbe40cb120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:57.681 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:57.658+0000 7fdbea2c9640 1 -- 172.21.15.169:0/1962061406 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbe40cbc50 con 0x7fdbe4159620 2023-10-26T01:37:57.681 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:57.658+0000 7fdbea2c9640 1 --2- 172.21.15.169:0/1962061406 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdbe4159620 0x7fdbe4159aa0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc000f80 tx=0x7fdbdc00dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:57.681 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:57.658+0000 7fdbda7fc640 1 -- 172.21.15.169:0/1962061406 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbdc015030 con 0x7fdbe4159620 2023-10-26T01:37:57.682 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:57.658+0000 7fdbda7fc640 1 -- 172.21.15.169:0/1962061406 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fdbdc003e70 con 0x7fdbe4159620 2023-10-26T01:37:57.682 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:57.658+0000 7fdbda7fc640 1 -- 172.21.15.169:0/1962061406 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbdc00ba40 con 0x7fdbe4159620 2023-10-26T01:37:57.682 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:57.658+0000 7fdbeaaca640 1 -- 172.21.15.169:0/1962061406 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbe40972d0 con 0x7fdbe4159620 2023-10-26T01:37:57.682 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:57.658+0000 7fdbeaaca640 1 -- 172.21.15.169:0/1962061406 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbe40cb660 con 0x7fdbe4159620 2023-10-26T01:37:57.682 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:57.660+0000 7fdbeaaca640 1 -- 172.21.15.169:0/1962061406 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbe40977f0 con 0x7fdbe4159620 2023-10-26T01:37:57.683 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:57.661+0000 7fdbda7fc640 1 -- 172.21.15.169:0/1962061406 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fdbdc018070 con 0x7fdbe4159620 2023-10-26T01:37:57.683 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:57.661+0000 7fdbda7fc640 1 --2- 172.21.15.169:0/1962061406 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fdbc806ae70 0x7fdbc806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:57.683 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:57.662+0000 7fdbda7fc640 1 -- 172.21.15.169:0/1962061406 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdbdc016070 con 0x7fdbe4159620 2023-10-26T01:37:57.683 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:57.662+0000 7fdbe9ac8640 1 --2- 172.21.15.169:0/1962061406 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fdbc806ae70 0x7fdbc806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:57.683 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:57.663+0000 7fdbe9ac8640 1 --2- 172.21.15.169:0/1962061406 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fdbc806ae70 0x7fdbc806d330 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fdbe0034000 tx=0x7fdbe002e040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:57.684 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:57.666+0000 7fdbda7fc640 1 -- 172.21.15.169:0/1962061406 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdbe40977f0 con 0x7fdbe4159620 2023-10-26T01:37:57.859 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:57.856+0000 7fdbeaaca640 1 -- 172.21.15.169:0/1962061406 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdbe40977f0 con 0x7fdbe4159620 2023-10-26T01:37:57.859 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:57.857+0000 7fdbda7fc640 1 -- 172.21.15.169:0/1962061406 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v123) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdbe40977f0 con 0x7fdbe4159620 2023-10-26T01:37:57.860 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 32 2023-10-26T01:37:57.863 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:57.860+0000 7fdbeaaca640 1 -- 172.21.15.169:0/1962061406 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fdbc806ae70 msgr2=0x7fdbc806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:57.863 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:57.860+0000 7fdbeaaca640 1 --2- 172.21.15.169:0/1962061406 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fdbc806ae70 0x7fdbc806d330 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fdbe0034000 tx=0x7fdbe002e040 comp rx=0 tx=0).stop 2023-10-26T01:37:57.863 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:57.860+0000 7fdbeaaca640 1 -- 172.21.15.169:0/1962061406 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdbe4159620 msgr2=0x7fdbe4159aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:57.864 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:57.860+0000 7fdbeaaca640 1 --2- 172.21.15.169:0/1962061406 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdbe4159620 0x7fdbe4159aa0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc000f80 tx=0x7fdbdc00dd90 comp rx=0 tx=0).stop 2023-10-26T01:37:57.864 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:57.860+0000 7fdbeaaca640 1 -- 172.21.15.169:0/1962061406 shutdown_connections 2023-10-26T01:37:57.864 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:57.860+0000 7fdbeaaca640 1 --2- 172.21.15.169:0/1962061406 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fdbc806ae70 0x7fdbc806d330 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:57.864 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:57.860+0000 7fdbeaaca640 1 --2- 172.21.15.169:0/1962061406 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdbe4159620 0x7fdbe4159aa0 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:57.864 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:57.860+0000 7fdbeaaca640 1 --2- 172.21.15.169:0/1962061406 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdbe41504e0 0x7fdbe414b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:57.865 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:57.860+0000 7fdbeaaca640 1 --2- 172.21.15.169:0/1962061406 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdbd0003970 0x7fdbe40cb120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:57.865 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:57.860+0000 7fdbeaaca640 1 -- 172.21.15.169:0/1962061406 >> 172.21.15.169:0/1962061406 conn(0x7fdbe40af110 msgr2=0x7fdbe41586e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:57.865 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:57.860+0000 7fdbeaaca640 1 -- 172.21.15.169:0/1962061406 shutdown_connections 2023-10-26T01:37:57.865 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:57.860+0000 7fdbeaaca640 1 -- 172.21.15.169:0/1962061406 wait complete. 2023-10-26T01:37:57.882 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-26T01:37:58.108 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:58.105+0000 7f46848f2640 1 -- 172.21.15.169:0/2297707108 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f466c0047c0 msgr2=0x7f466c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:58.109 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:58.105+0000 7f46848f2640 1 --2- 172.21.15.169:0/2297707108 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f466c0047c0 0x7f466c004c40 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f466800baf0 tx=0x7f466c0057f0 comp rx=0 tx=0).stop 2023-10-26T01:37:58.109 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:58.105+0000 7f467ed9d640 1 -- 172.21.15.169:0/2297707108 reap_dead start 2023-10-26T01:37:58.109 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:58.106+0000 7f46848f2640 1 -- 172.21.15.169:0/2297707108 shutdown_connections 2023-10-26T01:37:58.110 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:58.106+0000 7f46848f2640 1 -- 172.21.15.169:0/2297707108 >> 172.21.15.169:0/2297707108 conn(0x7f468006a620 msgr2=0x7f468006aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:58.110 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:58.106+0000 7f46848f2640 1 -- 172.21.15.169:0/2297707108 shutdown_connections 2023-10-26T01:37:58.110 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:58.106+0000 7f46848f2640 1 -- 172.21.15.169:0/2297707108 wait complete. 2023-10-26T01:37:58.113 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:58.107+0000 7f46848f2640 1 Processor -- start 2023-10-26T01:37:58.113 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:58.108+0000 7f46848f2640 1 -- start start 2023-10-26T01:37:58.113 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:58.108+0000 7f46848f2640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f466c0047c0 0x7f4680130eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:58.113 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:58.108+0000 7f46848f2640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f46800a7e60 0x7f46800a82e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:58.114 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:58.108+0000 7f46848f2640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f46800c63e0 0x7f46800c6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:58.114 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:58.108+0000 7f46848f2640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f468012e550 con 0x7f466c0047c0 2023-10-26T01:37:58.114 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:58.108+0000 7f46848f2640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46800a8820 con 0x7f46800a7e60 2023-10-26T01:37:58.114 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:58.108+0000 7f46848f2640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46800a7810 con 0x7f46800c63e0 2023-10-26T01:37:58.115 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:58.109+0000 7f467e59c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f466c0047c0 0x7f4680130eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:58.115 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:58.109+0000 7f467e59c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f466c0047c0 0x7f4680130eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:49006/0 (socket says 172.21.15.169:49006) 2023-10-26T01:37:58.115 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:58.109+0000 7f467e59c640 1 -- 172.21.15.169:0/823220780 learned_addr learned my addr 172.21.15.169:0/823220780 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:37:58.115 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:58.109+0000 7f467dd9b640 1 --2- 172.21.15.169:0/823220780 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f46800a7e60 0x7f46800a82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:58.115 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:58.109+0000 7f467ed9d640 1 --2- 172.21.15.169:0/823220780 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f46800c63e0 0x7f46800c6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:58.116 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:58.109+0000 7f467e59c640 1 -- 172.21.15.169:0/823220780 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f46800c63e0 msgr2=0x7f46800c6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:58.116 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:58.109+0000 7f467e59c640 1 --2- 172.21.15.169:0/823220780 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f46800c63e0 0x7f46800c6860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:58.116 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:58.109+0000 7f467e59c640 1 -- 172.21.15.169:0/823220780 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f46800a7e60 msgr2=0x7f46800a82e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:58.116 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:58.109+0000 7f467e59c640 1 --2- 172.21.15.169:0/823220780 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f46800a7e60 0x7f46800a82e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:58.117 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:58.109+0000 7f467e59c640 1 -- 172.21.15.169:0/823220780 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46800c6da0 con 0x7f466c0047c0 2023-10-26T01:37:58.117 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:58.110+0000 7f467e59c640 1 --2- 172.21.15.169:0/823220780 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f466c0047c0 0x7f4680130eb0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f467000c730 tx=0x7f467000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:58.117 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:58.110+0000 7f4676ffd640 1 -- 172.21.15.169:0/823220780 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4670005270 con 0x7f466c0047c0 2023-10-26T01:37:58.117 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:58.110+0000 7f4676ffd640 1 -- 172.21.15.169:0/823220780 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f46700035f0 con 0x7f466c0047c0 2023-10-26T01:37:58.117 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:58.110+0000 7f4676ffd640 1 -- 172.21.15.169:0/823220780 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4670003a00 con 0x7f466c0047c0 2023-10-26T01:37:58.117 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:58.110+0000 7f46848f2640 1 -- 172.21.15.169:0/823220780 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46800c8770 con 0x7f466c0047c0 2023-10-26T01:37:58.118 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:58.110+0000 7f46848f2640 1 -- 172.21.15.169:0/823220780 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46800c6fd0 con 0x7f466c0047c0 2023-10-26T01:37:58.119 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:58.111+0000 7f46848f2640 1 -- 172.21.15.169:0/823220780 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4680066af0 con 0x7f466c0047c0 2023-10-26T01:37:58.119 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:58.112+0000 7f4676ffd640 1 -- 172.21.15.169:0/823220780 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f467000a5e0 con 0x7f466c0047c0 2023-10-26T01:37:58.119 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:58.113+0000 7f4676ffd640 1 --2- 172.21.15.169:0/823220780 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f466006ae40 0x7f466006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:58.119 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:58.113+0000 7f4676ffd640 1 -- 172.21.15.169:0/823220780 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4670089140 con 0x7f466c0047c0 2023-10-26T01:37:58.119 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:58.114+0000 7f467dd9b640 1 --2- 172.21.15.169:0/823220780 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f466006ae40 0x7f466006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:58.120 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:58.115+0000 7f467dd9b640 1 --2- 172.21.15.169:0/823220780 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f466006ae40 0x7f466006d300 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f4668037000 tx=0x7f466802e040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:58.120 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:58.118+0000 7f4676ffd640 1 -- 172.21.15.169:0/823220780 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f46700541f0 con 0x7f466c0047c0 2023-10-26T01:37:58.308 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:58.305+0000 7f46848f2640 1 -- 172.21.15.169:0/823220780 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4680069ea0 con 0x7f466c0047c0 2023-10-26T01:37:58.308 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:58.306+0000 7f4676ffd640 1 -- 172.21.15.169:0/823220780 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v123) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4670017020 con 0x7f466c0047c0 2023-10-26T01:37:58.309 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 64 2023-10-26T01:37:58.311 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:58.308+0000 7f46848f2640 1 -- 172.21.15.169:0/823220780 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f466006ae40 msgr2=0x7f466006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:58.311 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:58.308+0000 7f46848f2640 1 --2- 172.21.15.169:0/823220780 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f466006ae40 0x7f466006d300 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f4668037000 tx=0x7f466802e040 comp rx=0 tx=0).stop 2023-10-26T01:37:58.312 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:58.308+0000 7f46848f2640 1 -- 172.21.15.169:0/823220780 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f466c0047c0 msgr2=0x7f4680130eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:58.312 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:58.308+0000 7f46848f2640 1 --2- 172.21.15.169:0/823220780 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f466c0047c0 0x7f4680130eb0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f467000c730 tx=0x7f467000cbf0 comp rx=0 tx=0).stop 2023-10-26T01:37:58.312 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:58.308+0000 7f46848f2640 1 -- 172.21.15.169:0/823220780 shutdown_connections 2023-10-26T01:37:58.312 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:58.308+0000 7f46848f2640 1 --2- 172.21.15.169:0/823220780 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f466006ae40 0x7f466006d300 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:58.312 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:58.308+0000 7f46848f2640 1 --2- 172.21.15.169:0/823220780 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f46800c63e0 0x7f46800c6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:58.313 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:58.308+0000 7f46848f2640 1 --2- 172.21.15.169:0/823220780 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f46800a7e60 0x7f46800a82e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:58.313 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:58.308+0000 7f46848f2640 1 --2- 172.21.15.169:0/823220780 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f466c0047c0 0x7f4680130eb0 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:58.313 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:58.308+0000 7f46848f2640 1 -- 172.21.15.169:0/823220780 >> 172.21.15.169:0/823220780 conn(0x7f468006a620 msgr2=0x7f468013ee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:58.313 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:58.308+0000 7f46848f2640 1 -- 172.21.15.169:0/823220780 shutdown_connections 2023-10-26T01:37:58.314 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:58.309+0000 7f46848f2640 1 -- 172.21.15.169:0/823220780 wait complete. 2023-10-26T01:37:58.329 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-26T01:37:58.543 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:58.540+0000 7f1ac8e6f640 1 -- 172.21.15.169:0/1248680311 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ab0004d00 msgr2=0x7f1ab0005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:58.543 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:58.540+0000 7f1ac8e6f640 1 --2- 172.21.15.169:0/1248680311 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ab0004d00 0x7f1ab0005180 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f1abc006f10 tx=0x7f1ab00057f0 comp rx=0 tx=0).stop 2023-10-26T01:37:58.544 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:58.540+0000 7f1ac2d9d640 1 -- 172.21.15.169:0/1248680311 reap_dead start 2023-10-26T01:37:58.544 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:58.540+0000 7f1ac8e6f640 1 -- 172.21.15.169:0/1248680311 shutdown_connections 2023-10-26T01:37:58.544 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:58.540+0000 7f1ac8e6f640 1 -- 172.21.15.169:0/1248680311 >> 172.21.15.169:0/1248680311 conn(0x7f1ac4069930 msgr2=0x7f1ac4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:58.544 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:58.540+0000 7f1ac8e6f640 1 -- 172.21.15.169:0/1248680311 shutdown_connections 2023-10-26T01:37:58.544 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:58.540+0000 7f1ac8e6f640 1 -- 172.21.15.169:0/1248680311 wait complete. 2023-10-26T01:37:58.545 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:58.542+0000 7f1ac8e6f640 1 Processor -- start 2023-10-26T01:37:58.545 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:58.542+0000 7f1ac8e6f640 1 -- start start 2023-10-26T01:37:58.547 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:58.543+0000 7f1ac8e6f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ab0004d00 0x7f1ac4154ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:58.547 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:58.543+0000 7f1ac8e6f640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1ac41e43f0 0x7f1ac40702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:58.547 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:58.543+0000 7f1ac8e6f640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ac41e47f0 0x7f1ac4070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:58.548 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:58.543+0000 7f1ac8e6f640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac41550e0 con 0x7f1ac41e47f0 2023-10-26T01:37:58.548 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:58.543+0000 7f1ac8e6f640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac40af2e0 con 0x7f1ab0004d00 2023-10-26T01:37:58.548 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:58.543+0000 7f1ac8e6f640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac40af580 con 0x7f1ac41e43f0 2023-10-26T01:37:58.548 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:58.543+0000 7f1ac2d9d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ac41e47f0 0x7f1ac4070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:58.548 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:58.543+0000 7f1ac2d9d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ac41e47f0 0x7f1ac4070800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:49020/0 (socket says 172.21.15.169:49020) 2023-10-26T01:37:58.549 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:58.543+0000 7f1ac2d9d640 1 -- 172.21.15.169:0/1908855503 learned_addr learned my addr 172.21.15.169:0/1908855503 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:37:58.549 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:58.544+0000 7f1ac259c640 1 --2- 172.21.15.169:0/1908855503 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ab0004d00 0x7f1ac4154ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:58.549 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:58.544+0000 7f1ac2d9d640 1 -- 172.21.15.169:0/1908855503 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1ac41e43f0 msgr2=0x7f1ac40702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:58.549 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:58.544+0000 7f1ac2d9d640 1 --2- 172.21.15.169:0/1908855503 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1ac41e43f0 0x7f1ac40702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:58.549 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:58.544+0000 7f1ac2d9d640 1 -- 172.21.15.169:0/1908855503 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ab0004d00 msgr2=0x7f1ac4154ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:58.550 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:58.544+0000 7f1ac2d9d640 1 --2- 172.21.15.169:0/1908855503 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ab0004d00 0x7f1ac4154ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:58.550 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:58.544+0000 7f1ac2d9d640 1 -- 172.21.15.169:0/1908855503 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ac406f7e0 con 0x7f1ac41e47f0 2023-10-26T01:37:58.550 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:58.544+0000 7f1ac2d9d640 1 --2- 172.21.15.169:0/1908855503 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ac41e47f0 0x7f1ac4070800 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f1abc00c290 tx=0x7f1abc02ef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:58.550 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:58.544+0000 7f1abaffd640 1 -- 172.21.15.169:0/1908855503 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1abc001c00 con 0x7f1ac41e47f0 2023-10-26T01:37:58.550 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:58.544+0000 7f1abaffd640 1 -- 172.21.15.169:0/1908855503 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1abc0036e0 con 0x7f1ac41e47f0 2023-10-26T01:37:58.551 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:58.544+0000 7f1abaffd640 1 -- 172.21.15.169:0/1908855503 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1abc003af0 con 0x7f1ac41e47f0 2023-10-26T01:37:58.551 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:58.544+0000 7f1ac8e6f640 1 -- 172.21.15.169:0/1908855503 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ac406fac0 con 0x7f1ac41e47f0 2023-10-26T01:37:58.551 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:58.545+0000 7f1ac8e6f640 1 -- 172.21.15.169:0/1908855503 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ac4158b50 con 0x7f1ac41e47f0 2023-10-26T01:37:58.552 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:58.546+0000 7f1ac8e6f640 1 -- 172.21.15.169:0/1908855503 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ac406ffe0 con 0x7f1ac41e47f0 2023-10-26T01:37:58.552 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:58.547+0000 7f1abaffd640 1 -- 172.21.15.169:0/1908855503 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f1abc032070 con 0x7f1ac41e47f0 2023-10-26T01:37:58.552 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:58.548+0000 7f1abaffd640 1 --2- 172.21.15.169:0/1908855503 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f1aa406ae70 0x7f1aa406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:58.552 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:58.548+0000 7f1abaffd640 1 -- 172.21.15.169:0/1908855503 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1abc0aab80 con 0x7f1ac41e47f0 2023-10-26T01:37:58.553 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:58.548+0000 7f1ac259c640 1 --2- 172.21.15.169:0/1908855503 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f1aa406ae70 0x7f1aa406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:58.553 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:58.550+0000 7f1ac259c640 1 --2- 172.21.15.169:0/1908855503 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f1aa406ae70 0x7f1aa406d330 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f1ab4005c40 tx=0x7f1ab4005bd0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:58.555 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:58.553+0000 7f1abaffd640 1 -- 172.21.15.169:0/1908855503 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1ac406ffe0 con 0x7f1ac41e47f0 2023-10-26T01:37:58.760 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:58.758+0000 7f1ac8e6f640 1 -- 172.21.15.169:0/1908855503 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f1ac406ffe0 con 0x7f1ac41e47f0 2023-10-26T01:37:58.761 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:58.758+0000 7f1abaffd640 1 -- 172.21.15.169:0/1908855503 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v123) v1 ==== 80+0+0 (secure 0 0 0) 0x7f1ac406ffe0 con 0x7f1ac41e47f0 2023-10-26T01:37:58.761 INFO:teuthology.orchestra.run.smithi169.stderr:listed 0 entries 2023-10-26T01:37:58.763 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:58.761+0000 7f1ac8e6f640 1 -- 172.21.15.169:0/1908855503 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f1aa406ae70 msgr2=0x7f1aa406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:58.764 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:58.761+0000 7f1ac8e6f640 1 --2- 172.21.15.169:0/1908855503 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f1aa406ae70 0x7f1aa406d330 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f1ab4005c40 tx=0x7f1ab4005bd0 comp rx=0 tx=0).stop 2023-10-26T01:37:58.764 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:58.761+0000 7f1ac8e6f640 1 -- 172.21.15.169:0/1908855503 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ac41e47f0 msgr2=0x7f1ac4070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:58.764 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:58.761+0000 7f1ac8e6f640 1 --2- 172.21.15.169:0/1908855503 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ac41e47f0 0x7f1ac4070800 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f1abc00c290 tx=0x7f1abc02ef00 comp rx=0 tx=0).stop 2023-10-26T01:37:58.764 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:58.761+0000 7f1ac8e6f640 1 -- 172.21.15.169:0/1908855503 shutdown_connections 2023-10-26T01:37:58.765 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:58.761+0000 7f1ac8e6f640 1 --2- 172.21.15.169:0/1908855503 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f1aa406ae70 0x7f1aa406d330 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:58.765 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:58.761+0000 7f1ac8e6f640 1 --2- 172.21.15.169:0/1908855503 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ac41e47f0 0x7f1ac4070800 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:58.765 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:58.761+0000 7f1ac8e6f640 1 --2- 172.21.15.169:0/1908855503 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1ac41e43f0 0x7f1ac40702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:58.765 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:58.761+0000 7f1ac8e6f640 1 --2- 172.21.15.169:0/1908855503 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ab0004d00 0x7f1ac4154ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:58.765 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:58.761+0000 7f1ac8e6f640 1 -- 172.21.15.169:0/1908855503 >> 172.21.15.169:0/1908855503 conn(0x7f1ac4069930 msgr2=0x7f1ac40ab5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:58.766 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:58.761+0000 7f1ac8e6f640 1 -- 172.21.15.169:0/1908855503 shutdown_connections 2023-10-26T01:37:58.766 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:58.761+0000 7f1ac8e6f640 1 -- 172.21.15.169:0/1908855503 wait complete. 2023-10-26T01:37:58.782 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-26T01:37:58.782 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:37:58.783 DEBUG:teuthology.orchestra.run.smithi169:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-26T01:37:58.910 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi169.front.sepia.ceph.com 2023-10-26T01:37:58.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:37:58.911 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-26T01:37:58.965 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-26T01:37:58.966 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:37:59.011 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-26T01:37:59.012 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:37:59.108 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:37:59.337 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:59.334+0000 7f5dc8ef2640 1 -- 172.21.15.169:0/3899126382 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5dc415b610 msgr2=0x7f5dc406ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:59.338 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:59.334+0000 7f5dc8ef2640 1 --2- 172.21.15.169:0/3899126382 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5dc415b610 0x7f5dc406ed30 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f5db4009830 tx=0x7f5db402bc30 comp rx=0 tx=0).stop 2023-10-26T01:37:59.338 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:59.335+0000 7f5dc8ef2640 1 -- 172.21.15.169:0/3899126382 shutdown_connections 2023-10-26T01:37:59.338 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:59.335+0000 7f5dc8ef2640 1 --2- 172.21.15.169:0/3899126382 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5dc415b610 0x7f5dc406ed30 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:59.339 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:59.335+0000 7f5dc8ef2640 1 -- 172.21.15.169:0/3899126382 >> 172.21.15.169:0/3899126382 conn(0x7f5dc4069930 msgr2=0x7f5dc4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:59.339 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:59.335+0000 7f5dc8ef2640 1 -- 172.21.15.169:0/3899126382 shutdown_connections 2023-10-26T01:37:59.339 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:59.335+0000 7f5dc8ef2640 1 -- 172.21.15.169:0/3899126382 wait complete. 2023-10-26T01:37:59.340 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:59.337+0000 7f5dc8ef2640 1 Processor -- start 2023-10-26T01:37:59.340 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:59.337+0000 7f5dc8ef2640 1 -- start start 2023-10-26T01:37:59.342 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:59.338+0000 7f5dc8ef2640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5dc40ac4e0 0x7f5dc414de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:59.343 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:59.338+0000 7f5dc8ef2640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5dc40cdd80 0x7f5dc4156210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:59.343 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:59.338+0000 7f5dc8ef2640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5dc415b610 0x7f5dc40aa1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:59.343 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:59.338+0000 7f5dc8ef2640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc414e580 con 0x7f5dc40cdd80 2023-10-26T01:37:59.344 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:59.338+0000 7f5dc8ef2640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc414e350 con 0x7f5dc40ac4e0 2023-10-26T01:37:59.344 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:59.338+0000 7f5dc8ef2640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc41f51c0 con 0x7f5dc415b610 2023-10-26T01:37:59.344 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:59.338+0000 7f5dc1d9b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5dc40cdd80 0x7f5dc4156210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:59.344 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:59.338+0000 7f5dc1d9b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5dc40cdd80 0x7f5dc4156210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:49040/0 (socket says 172.21.15.169:49040) 2023-10-26T01:37:59.345 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:59.338+0000 7f5dc1d9b640 1 -- 172.21.15.169:0/2422024223 learned_addr learned my addr 172.21.15.169:0/2422024223 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:37:59.345 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:59.338+0000 7f5dc259c640 1 --2- 172.21.15.169:0/2422024223 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5dc40ac4e0 0x7f5dc414de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:59.345 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:59.338+0000 7f5dc1d9b640 1 -- 172.21.15.169:0/2422024223 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5dc415b610 msgr2=0x7f5dc40aa1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:59.345 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:59.338+0000 7f5dc1d9b640 1 --2- 172.21.15.169:0/2422024223 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5dc415b610 0x7f5dc40aa1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:59.345 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:59.338+0000 7f5dc1d9b640 1 -- 172.21.15.169:0/2422024223 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5dc40ac4e0 msgr2=0x7f5dc414de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:59.346 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:59.338+0000 7f5dc1d9b640 1 --2- 172.21.15.169:0/2422024223 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5dc40ac4e0 0x7f5dc414de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:59.346 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:59.338+0000 7f5dc1d9b640 1 -- 172.21.15.169:0/2422024223 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5db4009490 con 0x7f5dc40cdd80 2023-10-26T01:37:59.346 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:59.339+0000 7f5dc1d9b640 1 --2- 172.21.15.169:0/2422024223 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5dc40cdd80 0x7f5dc4156210 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f5db800c750 tx=0x7f5db800cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:59.346 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:59.339+0000 7f5db2ffd640 1 -- 172.21.15.169:0/2422024223 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db800a840 con 0x7f5dc40cdd80 2023-10-26T01:37:59.347 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:59.339+0000 7f5db2ffd640 1 -- 172.21.15.169:0/2422024223 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5db800aa90 con 0x7f5dc40cdd80 2023-10-26T01:37:59.347 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:59.339+0000 7f5dc8ef2640 1 -- 172.21.15.169:0/2422024223 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dc41f5480 con 0x7f5dc40cdd80 2023-10-26T01:37:59.347 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:59.339+0000 7f5db2ffd640 1 -- 172.21.15.169:0/2422024223 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db800ae80 con 0x7f5dc40cdd80 2023-10-26T01:37:59.347 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:59.339+0000 7f5dc8ef2640 1 -- 172.21.15.169:0/2422024223 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dc41f5a40 con 0x7f5dc40cdd80 2023-10-26T01:37:59.348 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:59.340+0000 7f5dc8ef2640 1 -- 172.21.15.169:0/2422024223 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dc406f070 con 0x7f5dc40cdd80 2023-10-26T01:37:59.348 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:59.341+0000 7f5db2ffd640 1 -- 172.21.15.169:0/2422024223 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f5db801c460 con 0x7f5dc40cdd80 2023-10-26T01:37:59.349 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:59.343+0000 7f5db2ffd640 1 --2- 172.21.15.169:0/2422024223 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5da406ae40 0x7f5da406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:59.349 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:59.343+0000 7f5db2ffd640 1 -- 172.21.15.169:0/2422024223 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5db8089880 con 0x7f5dc40cdd80 2023-10-26T01:37:59.349 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:59.343+0000 7f5dc259c640 1 --2- 172.21.15.169:0/2422024223 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5da406ae40 0x7f5da406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:59.349 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:59.345+0000 7f5dc259c640 1 --2- 172.21.15.169:0/2422024223 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5da406ae40 0x7f5da406d300 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f5db4036000 tx=0x7f5db4004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:59.350 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:59.347+0000 7f5db2ffd640 1 -- 172.21.15.169:0/2422024223 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5db8054920 con 0x7f5dc40cdd80 2023-10-26T01:37:59.541 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:59.538+0000 7f5dc8ef2640 1 -- 172.21.15.169:0/2422024223 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5dc404db10 con 0x7f5dc40cdd80 2023-10-26T01:37:59.544 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:59.541+0000 7f5db2ffd640 1 -- 172.21.15.169:0/2422024223 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v123) v1 ==== 74+0+16575 (secure 0 0 0) 0x7f5db8057f40 con 0x7f5dc40cdd80 2023-10-26T01:37:59.544 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:37:59.545 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":123,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:37:54.102485+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-10-26T01:37:37.959102+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-10-26T01:37:39.053240+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:37:59.547 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:59.543+0000 7f5dc8ef2640 1 -- 172.21.15.169:0/2422024223 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5da406ae40 msgr2=0x7f5da406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:59.547 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:59.543+0000 7f5dc8ef2640 1 --2- 172.21.15.169:0/2422024223 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5da406ae40 0x7f5da406d300 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f5db4036000 tx=0x7f5db4004730 comp rx=0 tx=0).stop 2023-10-26T01:37:59.547 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:59.544+0000 7f5dc8ef2640 1 -- 172.21.15.169:0/2422024223 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5dc40cdd80 msgr2=0x7f5dc4156210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:59.547 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:59.544+0000 7f5dc8ef2640 1 --2- 172.21.15.169:0/2422024223 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5dc40cdd80 0x7f5dc4156210 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f5db800c750 tx=0x7f5db800cc10 comp rx=0 tx=0).stop 2023-10-26T01:37:59.548 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:59.544+0000 7f5dc8ef2640 1 -- 172.21.15.169:0/2422024223 shutdown_connections 2023-10-26T01:37:59.548 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:59.544+0000 7f5dc8ef2640 1 --2- 172.21.15.169:0/2422024223 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5da406ae40 0x7f5da406d300 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:59.548 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:59.544+0000 7f5dc8ef2640 1 --2- 172.21.15.169:0/2422024223 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5dc40cdd80 0x7f5dc4156210 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:59.548 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:59.544+0000 7f5dc8ef2640 1 --2- 172.21.15.169:0/2422024223 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5dc415b610 0x7f5dc40aa1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:59.548 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:59.544+0000 7f5dc8ef2640 1 --2- 172.21.15.169:0/2422024223 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5dc40ac4e0 0x7f5dc414de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:59.549 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:59.544+0000 7f5dc8ef2640 1 -- 172.21.15.169:0/2422024223 >> 172.21.15.169:0/2422024223 conn(0x7f5dc4069930 msgr2=0x7f5dc40cc7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:59.549 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:59.544+0000 7f5dc8ef2640 1 -- 172.21.15.169:0/2422024223 shutdown_connections 2023-10-26T01:37:59.549 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:59.544+0000 7f5dc8ef2640 1 -- 172.21.15.169:0/2422024223 wait complete. 2023-10-26T01:37:59.566 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T01:30:32.200808+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '109', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-26T01:37:37.959102+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '119', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2023-10-26T01:37:39.053240+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T01:37:59.568 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T01:37:59.790 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:59.787+0000 7f68a5c4b640 1 -- 172.21.15.169:0/2684538842 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68a0159480 msgr2=0x7f68a00b02e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:59.790 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:59.787+0000 7f68a5c4b640 1 --2- 172.21.15.169:0/2684538842 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68a0159480 0x7f68a00b02e0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f6890005fd0 tx=0x7f689002c2f0 comp rx=0 tx=0).stop 2023-10-26T01:37:59.790 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:59.787+0000 7f68a5c4b640 1 -- 172.21.15.169:0/2684538842 shutdown_connections 2023-10-26T01:37:59.791 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:59.787+0000 7f68a5c4b640 1 --2- 172.21.15.169:0/2684538842 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68a0159480 0x7f68a00b02e0 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:59.791 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:59.787+0000 7f68a5c4b640 1 -- 172.21.15.169:0/2684538842 >> 172.21.15.169:0/2684538842 conn(0x7f68a0069930 msgr2=0x7f68a0069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:59.791 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:59.787+0000 7f68a5c4b640 1 -- 172.21.15.169:0/2684538842 shutdown_connections 2023-10-26T01:37:59.791 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:59.787+0000 7f68a5c4b640 1 -- 172.21.15.169:0/2684538842 wait complete. 2023-10-26T01:37:59.792 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:59.789+0000 7f68a5c4b640 1 Processor -- start 2023-10-26T01:37:59.793 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:59.789+0000 7f68a5c4b640 1 -- start start 2023-10-26T01:37:59.793 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:59.790+0000 7f68a5c4b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f68a00b0820 0x7f68a006ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:59.793 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:59.790+0000 7f68a5c4b640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f68a00cd690 0x7f68a00ae3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:59.793 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:59.790+0000 7f68a5c4b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68a0159480 0x7f68a0156680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:59.793 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:59.790+0000 7f68a5c4b640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68a01f21f0 con 0x7f68a0159480 2023-10-26T01:37:59.794 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:59.790+0000 7f68a5c4b640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68a0158d30 con 0x7f68a00b0820 2023-10-26T01:37:59.794 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:59.790+0000 7f68a5c4b640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68a01f5c50 con 0x7f68a00cd690 2023-10-26T01:37:59.794 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:59.790+0000 7f689ffff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68a0159480 0x7f68a0156680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:59.794 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:59.790+0000 7f689ffff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68a0159480 0x7f68a0156680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:49054/0 (socket says 172.21.15.169:49054) 2023-10-26T01:37:59.795 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:59.790+0000 7f689ffff640 1 -- 172.21.15.169:0/211394379 learned_addr learned my addr 172.21.15.169:0/211394379 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:37:59.796 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:59.790+0000 7f689f7fe640 1 --2- 172.21.15.169:0/211394379 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f68a00b0820 0x7f68a006ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:59.796 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:59.790+0000 7f689ffff640 1 -- 172.21.15.169:0/211394379 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f68a00cd690 msgr2=0x7f68a00ae3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:59.797 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:59.790+0000 7f6896ffd640 1 --2- 172.21.15.169:0/211394379 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f68a00cd690 0x7f68a00ae3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:59.797 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:59.790+0000 7f689ffff640 1 --2- 172.21.15.169:0/211394379 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f68a00cd690 0x7f68a00ae3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:59.797 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:59.790+0000 7f689ffff640 1 -- 172.21.15.169:0/211394379 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f68a00b0820 msgr2=0x7f68a006ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:59.797 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:59.790+0000 7f689ffff640 1 --2- 172.21.15.169:0/211394379 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f68a00b0820 0x7f68a006ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:59.797 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:59.790+0000 7f689ffff640 1 -- 172.21.15.169:0/211394379 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6890009900 con 0x7f68a0159480 2023-10-26T01:37:59.798 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:59.791+0000 7f689f7fe640 1 --2- 172.21.15.169:0/211394379 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f68a00b0820 0x7f68a006ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:37:59.798 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:59.791+0000 7f6896ffd640 1 --2- 172.21.15.169:0/211394379 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f68a00cd690 0x7f68a00ae3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:37:59.798 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:59.791+0000 7f689ffff640 1 --2- 172.21.15.169:0/211394379 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68a0159480 0x7f68a0156680 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f689800c410 tx=0x7f689800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:59.798 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:59.791+0000 7f689cff9640 1 -- 172.21.15.169:0/211394379 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6898011790 con 0x7f68a0159480 2023-10-26T01:37:59.798 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:59.791+0000 7f689cff9640 1 -- 172.21.15.169:0/211394379 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f68980119e0 con 0x7f68a0159480 2023-10-26T01:37:59.799 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:59.791+0000 7f68a5c4b640 1 -- 172.21.15.169:0/211394379 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68a01f5f30 con 0x7f68a0159480 2023-10-26T01:37:59.799 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:59.791+0000 7f689cff9640 1 -- 172.21.15.169:0/211394379 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6898011e30 con 0x7f68a0159480 2023-10-26T01:37:59.799 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:59.791+0000 7f68a5c4b640 1 -- 172.21.15.169:0/211394379 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68a01f62c0 con 0x7f68a0159480 2023-10-26T01:37:59.799 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:59.792+0000 7f68a5c4b640 1 -- 172.21.15.169:0/211394379 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68a0066af0 con 0x7f68a0159480 2023-10-26T01:37:59.799 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:59.793+0000 7f689cff9640 1 -- 172.21.15.169:0/211394379 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f689801c460 con 0x7f68a0159480 2023-10-26T01:37:59.800 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:59.794+0000 7f689cff9640 1 --2- 172.21.15.169:0/211394379 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f688006ae40 0x7f688006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:37:59.802 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:59.794+0000 7f689cff9640 1 -- 172.21.15.169:0/211394379 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6898089560 con 0x7f68a0159480 2023-10-26T01:37:59.802 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:59.795+0000 7f689f7fe640 1 --2- 172.21.15.169:0/211394379 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f688006ae40 0x7f688006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:37:59.803 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:59.796+0000 7f689f7fe640 1 --2- 172.21.15.169:0/211394379 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f688006ae40 0x7f688006d300 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f6890036000 tx=0x7f6890004ab0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:37:59.803 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:59.799+0000 7f689cff9640 1 -- 172.21.15.169:0/211394379 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f68980545f0 con 0x7f68a0159480 2023-10-26T01:37:59.989 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:59.987+0000 7f68a5c4b640 1 -- 172.21.15.169:0/211394379 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f68a01f2330 con 0x7f68a0159480 2023-10-26T01:37:59.990 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:59.988+0000 7f689cff9640 1 -- 172.21.15.169:0/211394379 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v123) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6898057c10 con 0x7f68a0159480 2023-10-26T01:37:59.991 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 1 2023-10-26T01:37:59.993 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:59.990+0000 7f68a5c4b640 1 -- 172.21.15.169:0/211394379 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f688006ae40 msgr2=0x7f688006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:59.993 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:59.990+0000 7f68a5c4b640 1 --2- 172.21.15.169:0/211394379 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f688006ae40 0x7f688006d300 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f6890036000 tx=0x7f6890004ab0 comp rx=0 tx=0).stop 2023-10-26T01:37:59.993 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:59.990+0000 7f68a5c4b640 1 -- 172.21.15.169:0/211394379 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68a0159480 msgr2=0x7f68a0156680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:37:59.994 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:59.990+0000 7f68a5c4b640 1 --2- 172.21.15.169:0/211394379 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68a0159480 0x7f68a0156680 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f689800c410 tx=0x7f689800c8d0 comp rx=0 tx=0).stop 2023-10-26T01:37:59.994 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:59.990+0000 7f68a5c4b640 1 -- 172.21.15.169:0/211394379 shutdown_connections 2023-10-26T01:37:59.994 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:59.990+0000 7f68a5c4b640 1 --2- 172.21.15.169:0/211394379 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f688006ae40 0x7f688006d300 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:59.994 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:59.990+0000 7f68a5c4b640 1 --2- 172.21.15.169:0/211394379 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68a0159480 0x7f68a0156680 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:59.994 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:59.990+0000 7f68a5c4b640 1 --2- 172.21.15.169:0/211394379 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f68a00cd690 0x7f68a00ae3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:59.995 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:59.990+0000 7f68a5c4b640 1 --2- 172.21.15.169:0/211394379 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f68a00b0820 0x7f68a006ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:37:59.995 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:59.990+0000 7f68a5c4b640 1 -- 172.21.15.169:0/211394379 >> 172.21.15.169:0/211394379 conn(0x7f68a0069930 msgr2=0x7f68a006c750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:37:59.995 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:59.990+0000 7f68a5c4b640 1 -- 172.21.15.169:0/211394379 shutdown_connections 2023-10-26T01:37:59.995 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:37:59.991+0000 7f68a5c4b640 1 -- 172.21.15.169:0/211394379 wait complete. 2023-10-26T01:38:00.013 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-26T01:38:00.240 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:00.237+0000 7fb72b17d640 1 -- 172.21.15.169:0/2493317500 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb7241597c0 msgr2=0x7fb72415dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:00.241 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:00.237+0000 7fb72b17d640 1 --2- 172.21.15.169:0/2493317500 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb7241597c0 0x7fb72415dbf0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb710009830 tx=0x7fb71002bd60 comp rx=0 tx=0).stop 2023-10-26T01:38:00.241 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:00.238+0000 7fb72b17d640 1 -- 172.21.15.169:0/2493317500 shutdown_connections 2023-10-26T01:38:00.241 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:00.238+0000 7fb72b17d640 1 --2- 172.21.15.169:0/2493317500 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb7241597c0 0x7fb72415dbf0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:00.241 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:00.238+0000 7fb72b17d640 1 --2- 172.21.15.169:0/2493317500 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb7241593e0 0x7fb7240ac200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:00.242 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:00.238+0000 7fb72b17d640 1 -- 172.21.15.169:0/2493317500 >> 172.21.15.169:0/2493317500 conn(0x7fb724069930 msgr2=0x7fb724069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:00.242 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:00.238+0000 7fb72b17d640 1 -- 172.21.15.169:0/2493317500 shutdown_connections 2023-10-26T01:38:00.242 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:00.238+0000 7fb72b17d640 1 -- 172.21.15.169:0/2493317500 wait complete. 2023-10-26T01:38:00.243 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:00.240+0000 7fb72b17d640 1 Processor -- start 2023-10-26T01:38:00.243 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:00.240+0000 7fb72b17d640 1 -- start start 2023-10-26T01:38:00.245 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:00.242+0000 7fb72b17d640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb7240b0c80 0x7fb72414cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:00.245 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:00.242+0000 7fb72b17d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb7241593e0 0x7fb72406c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:00.246 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:00.242+0000 7fb72b17d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb7241597c0 0x7fb7241e46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:00.246 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:00.242+0000 7fb72b17d640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb72414df40 con 0x7fb7241597c0 2023-10-26T01:38:00.246 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:00.242+0000 7fb72b17d640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb72406fe70 con 0x7fb7241593e0 2023-10-26T01:38:00.246 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:00.242+0000 7fb72b17d640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb72406b8d0 con 0x7fb7240b0c80 2023-10-26T01:38:00.247 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:00.242+0000 7fb72971a640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb7241597c0 0x7fb7241e46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:00.247 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:00.242+0000 7fb72971a640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb7241597c0 0x7fb7241e46d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:49070/0 (socket says 172.21.15.169:49070) 2023-10-26T01:38:00.247 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:00.242+0000 7fb72971a640 1 -- 172.21.15.169:0/3554512354 learned_addr learned my addr 172.21.15.169:0/3554512354 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:38:00.247 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:00.242+0000 7fb723fff640 1 --2- 172.21.15.169:0/3554512354 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb7241593e0 0x7fb72406c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:00.247 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:00.242+0000 7fb728f19640 1 --2- 172.21.15.169:0/3554512354 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb7240b0c80 0x7fb72414cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:00.247 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:00.243+0000 7fb72971a640 1 -- 172.21.15.169:0/3554512354 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb7240b0c80 msgr2=0x7fb72414cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:00.248 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:00.243+0000 7fb72971a640 1 --2- 172.21.15.169:0/3554512354 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb7240b0c80 0x7fb72414cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:00.248 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:00.243+0000 7fb72971a640 1 -- 172.21.15.169:0/3554512354 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb7241593e0 msgr2=0x7fb72406c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:00.248 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:00.243+0000 7fb72971a640 1 --2- 172.21.15.169:0/3554512354 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb7241593e0 0x7fb72406c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:00.248 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:00.243+0000 7fb72971a640 1 -- 172.21.15.169:0/3554512354 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb718009560 con 0x7fb7241597c0 2023-10-26T01:38:00.249 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:00.243+0000 7fb728f19640 1 --2- 172.21.15.169:0/3554512354 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb7240b0c80 0x7fb72414cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:38:00.249 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:00.243+0000 7fb72971a640 1 --2- 172.21.15.169:0/3554512354 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb7241597c0 0x7fb7241e46d0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7fb71c00c7a0 tx=0x7fb71c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:00.250 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:00.243+0000 7fb7217fa640 1 -- 172.21.15.169:0/3554512354 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb71c003b70 con 0x7fb7241597c0 2023-10-26T01:38:00.250 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:00.243+0000 7fb7217fa640 1 -- 172.21.15.169:0/3554512354 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb71c003cf0 con 0x7fb7241597c0 2023-10-26T01:38:00.250 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:00.243+0000 7fb7217fa640 1 -- 172.21.15.169:0/3554512354 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb71c00aab0 con 0x7fb7241597c0 2023-10-26T01:38:00.251 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:00.243+0000 7fb72b17d640 1 -- 172.21.15.169:0/3554512354 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb710009490 con 0x7fb7241597c0 2023-10-26T01:38:00.251 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:00.243+0000 7fb72b17d640 1 -- 172.21.15.169:0/3554512354 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb72406be90 con 0x7fb7241597c0 2023-10-26T01:38:00.251 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:00.244+0000 7fb72b17d640 1 -- 172.21.15.169:0/3554512354 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb72406fcd0 con 0x7fb7241597c0 2023-10-26T01:38:00.251 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:00.246+0000 7fb7217fa640 1 -- 172.21.15.169:0/3554512354 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fb71c00ac10 con 0x7fb7241597c0 2023-10-26T01:38:00.251 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:00.247+0000 7fb7217fa640 1 --2- 172.21.15.169:0/3554512354 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb70806ae40 0x7fb70806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:00.252 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:00.247+0000 7fb7217fa640 1 -- 172.21.15.169:0/3554512354 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb71c0895a0 con 0x7fb7241597c0 2023-10-26T01:38:00.252 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:00.248+0000 7fb728f19640 1 --2- 172.21.15.169:0/3554512354 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb70806ae40 0x7fb70806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:00.252 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:00.249+0000 7fb728f19640 1 --2- 172.21.15.169:0/3554512354 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb70806ae40 0x7fb70806d300 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fb718005fd0 tx=0x7fb718009260 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:00.253 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:00.251+0000 7fb7217fa640 1 -- 172.21.15.169:0/3554512354 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb72406fcd0 con 0x7fb7241597c0 2023-10-26T01:38:00.443 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:00.440+0000 7fb72b17d640 1 -- 172.21.15.169:0/3554512354 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb72406fcd0 con 0x7fb7241597c0 2023-10-26T01:38:00.444 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:00.441+0000 7fb7217fa640 1 -- 172.21.15.169:0/3554512354 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v123) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb72406fcd0 con 0x7fb7241597c0 2023-10-26T01:38:00.444 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 32 2023-10-26T01:38:00.450 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:00.444+0000 7fb72b17d640 1 -- 172.21.15.169:0/3554512354 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb70806ae40 msgr2=0x7fb70806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:00.450 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:00.444+0000 7fb72b17d640 1 --2- 172.21.15.169:0/3554512354 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb70806ae40 0x7fb70806d300 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fb718005fd0 tx=0x7fb718009260 comp rx=0 tx=0).stop 2023-10-26T01:38:00.450 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:00.444+0000 7fb72b17d640 1 -- 172.21.15.169:0/3554512354 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb7241597c0 msgr2=0x7fb7241e46d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:00.450 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:00.444+0000 7fb72b17d640 1 --2- 172.21.15.169:0/3554512354 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb7241597c0 0x7fb7241e46d0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7fb71c00c7a0 tx=0x7fb71c00cc60 comp rx=0 tx=0).stop 2023-10-26T01:38:00.451 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:00.444+0000 7fb72b17d640 1 -- 172.21.15.169:0/3554512354 shutdown_connections 2023-10-26T01:38:00.451 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:00.444+0000 7fb72b17d640 1 --2- 172.21.15.169:0/3554512354 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb70806ae40 0x7fb70806d300 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:00.451 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:00.444+0000 7fb72b17d640 1 --2- 172.21.15.169:0/3554512354 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb7241597c0 0x7fb7241e46d0 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:00.451 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:00.444+0000 7fb72b17d640 1 --2- 172.21.15.169:0/3554512354 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb7241593e0 0x7fb72406c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:00.451 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:00.444+0000 7fb72b17d640 1 --2- 172.21.15.169:0/3554512354 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb7240b0c80 0x7fb72414cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:00.452 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:00.445+0000 7fb72b17d640 1 -- 172.21.15.169:0/3554512354 >> 172.21.15.169:0/3554512354 conn(0x7fb724069930 msgr2=0x7fb7240cc380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:00.452 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:00.445+0000 7fb72b17d640 1 -- 172.21.15.169:0/3554512354 shutdown_connections 2023-10-26T01:38:00.452 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:00.445+0000 7fb72b17d640 1 -- 172.21.15.169:0/3554512354 wait complete. 2023-10-26T01:38:00.465 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-26T01:38:00.689 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:00.686+0000 7f1d7284f640 1 -- 172.21.15.169:0/2161703268 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d6c0cb6c0 msgr2=0x7f1d6c0cbb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:00.689 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:00.686+0000 7f1d7284f640 1 --2- 172.21.15.169:0/2161703268 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d6c0cb6c0 0x7f1d6c0cbb20 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f1d60009830 tx=0x7f1d6002bcd0 comp rx=0 tx=0).stop 2023-10-26T01:38:00.689 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:00.686+0000 7f1d7284f640 1 -- 172.21.15.169:0/2161703268 shutdown_connections 2023-10-26T01:38:00.690 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:00.686+0000 7f1d7284f640 1 --2- 172.21.15.169:0/2161703268 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1d6c0cc060 0x7f1d6c15f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:00.690 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:00.686+0000 7f1d7284f640 1 --2- 172.21.15.169:0/2161703268 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d6c0cb6c0 0x7f1d6c0cbb20 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:00.690 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:00.686+0000 7f1d7284f640 1 -- 172.21.15.169:0/2161703268 >> 172.21.15.169:0/2161703268 conn(0x7f1d6c0af570 msgr2=0x7f1d6c0af980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:00.690 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:00.686+0000 7f1d7284f640 1 -- 172.21.15.169:0/2161703268 shutdown_connections 2023-10-26T01:38:00.690 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:00.686+0000 7f1d7284f640 1 -- 172.21.15.169:0/2161703268 wait complete. 2023-10-26T01:38:00.692 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:00.688+0000 7f1d7284f640 1 Processor -- start 2023-10-26T01:38:00.692 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:00.688+0000 7f1d7284f640 1 -- start start 2023-10-26T01:38:00.692 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:00.689+0000 7f1d7284f640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d6c0cb6c0 0x7f1d6c149dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:00.692 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:00.689+0000 7f1d7284f640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1d6c0cc060 0x7f1d6c15ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:00.693 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:00.689+0000 7f1d7284f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d6c15b760 0x7f1d6c0ae690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:00.693 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:00.689+0000 7f1d7284f640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d6c15c980 con 0x7f1d6c0cb6c0 2023-10-26T01:38:00.693 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:00.689+0000 7f1d7284f640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d6c15bd70 con 0x7f1d6c15b760 2023-10-26T01:38:00.693 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:00.689+0000 7f1d7284f640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d6c15c050 con 0x7f1d6c0cc060 2023-10-26T01:38:00.693 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:00.689+0000 7f1d6bfff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d6c0cb6c0 0x7f1d6c149dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:00.694 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:00.689+0000 7f1d6bfff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d6c0cb6c0 0x7f1d6c149dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:49076/0 (socket says 172.21.15.169:49076) 2023-10-26T01:38:00.694 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:00.689+0000 7f1d6bfff640 1 -- 172.21.15.169:0/1661216756 learned_addr learned my addr 172.21.15.169:0/1661216756 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:38:00.695 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:00.690+0000 7f1d70dec640 1 --2- 172.21.15.169:0/1661216756 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d6c15b760 0x7f1d6c0ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:00.696 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:00.690+0000 7f1d6b7fe640 1 --2- 172.21.15.169:0/1661216756 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1d6c0cc060 0x7f1d6c15ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:00.696 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:00.690+0000 7f1d6bfff640 1 -- 172.21.15.169:0/1661216756 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1d6c0cc060 msgr2=0x7f1d6c15ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:00.696 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:00.690+0000 7f1d6bfff640 1 --2- 172.21.15.169:0/1661216756 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1d6c0cc060 0x7f1d6c15ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:00.696 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:00.690+0000 7f1d6bfff640 1 -- 172.21.15.169:0/1661216756 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d6c15b760 msgr2=0x7f1d6c0ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:00.697 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:00.690+0000 7f1d6bfff640 1 --2- 172.21.15.169:0/1661216756 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d6c15b760 0x7f1d6c0ae690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:00.697 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:00.690+0000 7f1d6bfff640 1 -- 172.21.15.169:0/1661216756 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d64009900 con 0x7f1d6c0cb6c0 2023-10-26T01:38:00.697 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:00.690+0000 7f1d70dec640 1 --2- 172.21.15.169:0/1661216756 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d6c15b760 0x7f1d6c0ae690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:38:00.697 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:00.690+0000 7f1d6bfff640 1 --2- 172.21.15.169:0/1661216756 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d6c0cb6c0 0x7f1d6c149dc0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c00c400 tx=0x7f1d5c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:00.697 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:00.690+0000 7f1d68ff9640 1 -- 172.21.15.169:0/1661216756 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d5c0052d0 con 0x7f1d6c0cb6c0 2023-10-26T01:38:00.698 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:00.690+0000 7f1d68ff9640 1 -- 172.21.15.169:0/1661216756 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1d5c0035f0 con 0x7f1d6c0cb6c0 2023-10-26T01:38:00.698 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:00.690+0000 7f1d68ff9640 1 -- 172.21.15.169:0/1661216756 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d5c003a00 con 0x7f1d6c0cb6c0 2023-10-26T01:38:00.698 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:00.690+0000 7f1d7284f640 1 -- 172.21.15.169:0/1661216756 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d60009490 con 0x7f1d6c0cb6c0 2023-10-26T01:38:00.698 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:00.691+0000 7f1d7284f640 1 -- 172.21.15.169:0/1661216756 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d6c15c610 con 0x7f1d6c0cb6c0 2023-10-26T01:38:00.698 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:00.692+0000 7f1d7284f640 1 -- 172.21.15.169:0/1661216756 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d6c04db10 con 0x7f1d6c0cb6c0 2023-10-26T01:38:00.699 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:00.693+0000 7f1d68ff9640 1 -- 172.21.15.169:0/1661216756 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f1d5c003b60 con 0x7f1d6c0cb6c0 2023-10-26T01:38:00.699 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:00.694+0000 7f1d68ff9640 1 --2- 172.21.15.169:0/1661216756 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f1d5806ae40 0x7f1d5806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:00.700 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:00.694+0000 7f1d68ff9640 1 -- 172.21.15.169:0/1661216756 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1d5c088ea0 con 0x7f1d6c0cb6c0 2023-10-26T01:38:00.700 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:00.695+0000 7f1d6b7fe640 1 --2- 172.21.15.169:0/1661216756 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f1d5806ae40 0x7f1d5806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:00.700 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:00.696+0000 7f1d6b7fe640 1 --2- 172.21.15.169:0/1661216756 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f1d5806ae40 0x7f1d5806d300 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f1d60036000 tx=0x7f1d60004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:00.701 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:00.699+0000 7f1d68ff9640 1 -- 172.21.15.169:0/1661216756 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1d5c053eb0 con 0x7f1d6c0cb6c0 2023-10-26T01:38:00.889 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:00.886+0000 7f1d7284f640 1 -- 172.21.15.169:0/1661216756 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1d6c066740 con 0x7f1d6c0cb6c0 2023-10-26T01:38:00.890 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:00.887+0000 7f1d68ff9640 1 -- 172.21.15.169:0/1661216756 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v123) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1d5c0170a0 con 0x7f1d6c0cb6c0 2023-10-26T01:38:00.890 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 64 2023-10-26T01:38:00.893 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:00.890+0000 7f1d7284f640 1 -- 172.21.15.169:0/1661216756 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f1d5806ae40 msgr2=0x7f1d5806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:00.893 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:00.890+0000 7f1d7284f640 1 --2- 172.21.15.169:0/1661216756 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f1d5806ae40 0x7f1d5806d300 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f1d60036000 tx=0x7f1d60004730 comp rx=0 tx=0).stop 2023-10-26T01:38:00.893 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:00.890+0000 7f1d7284f640 1 -- 172.21.15.169:0/1661216756 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d6c0cb6c0 msgr2=0x7f1d6c149dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:00.894 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:00.890+0000 7f1d7284f640 1 --2- 172.21.15.169:0/1661216756 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d6c0cb6c0 0x7f1d6c149dc0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c00c400 tx=0x7f1d5c00c8c0 comp rx=0 tx=0).stop 2023-10-26T01:38:00.894 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:00.890+0000 7f1d7284f640 1 -- 172.21.15.169:0/1661216756 shutdown_connections 2023-10-26T01:38:00.894 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:00.890+0000 7f1d7284f640 1 --2- 172.21.15.169:0/1661216756 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f1d5806ae40 0x7f1d5806d300 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:00.894 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:00.890+0000 7f1d7284f640 1 --2- 172.21.15.169:0/1661216756 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d6c15b760 0x7f1d6c0ae690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:00.894 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:00.890+0000 7f1d7284f640 1 --2- 172.21.15.169:0/1661216756 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1d6c0cc060 0x7f1d6c15ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:00.895 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:00.890+0000 7f1d7284f640 1 --2- 172.21.15.169:0/1661216756 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d6c0cb6c0 0x7f1d6c149dc0 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:00.895 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:00.890+0000 7f1d7284f640 1 -- 172.21.15.169:0/1661216756 >> 172.21.15.169:0/1661216756 conn(0x7f1d6c0af570 msgr2=0x7f1d6c157480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:00.895 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:00.890+0000 7f1d7284f640 1 -- 172.21.15.169:0/1661216756 shutdown_connections 2023-10-26T01:38:00.895 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:00.890+0000 7f1d7284f640 1 -- 172.21.15.169:0/1661216756 wait complete. 2023-10-26T01:38:00.911 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-26T01:38:01.127 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:01.124+0000 7f624f644640 1 -- 172.21.15.169:0/3326925802 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62481597c0 msgr2=0x7f62480ac200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:01.128 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:01.124+0000 7f624f644640 1 --2- 172.21.15.169:0/3326925802 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62481597c0 0x7f62480ac200 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f62480cd2c0 tx=0x7f623c02bda0 comp rx=0 tx=0).stop 2023-10-26T01:38:01.128 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:01.125+0000 7f624f644640 1 -- 172.21.15.169:0/3326925802 shutdown_connections 2023-10-26T01:38:01.128 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:01.125+0000 7f624f644640 1 --2- 172.21.15.169:0/3326925802 >> v2:172.21.15.170:3300/0 conn(0x7f62480b0c80 0x7f62480cd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:01.128 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:01.125+0000 7f624f644640 1 --2- 172.21.15.169:0/3326925802 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62481597c0 0x7f62480ac200 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:01.129 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:01.125+0000 7f624f644640 1 -- 172.21.15.169:0/3326925802 >> 172.21.15.169:0/3326925802 conn(0x7f6248069930 msgr2=0x7f6248069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:01.129 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:01.125+0000 7f624f644640 1 -- 172.21.15.169:0/3326925802 shutdown_connections 2023-10-26T01:38:01.129 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:01.125+0000 7f624f644640 1 -- 172.21.15.169:0/3326925802 wait complete. 2023-10-26T01:38:01.130 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:01.126+0000 7f624f644640 1 Processor -- start 2023-10-26T01:38:01.130 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:01.127+0000 7f624f644640 1 -- start start 2023-10-26T01:38:01.131 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:01.127+0000 7f624f644640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f62480b0c80 0x7f6248148840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:01.131 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:01.127+0000 7f624f644640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f62481593e0 0x7f62480af560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:01.131 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:01.127+0000 7f624f644640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62481597c0 0x7f62481492b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:01.131 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:01.127+0000 7f624f644640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62480af0d0 con 0x7f62481597c0 2023-10-26T01:38:01.131 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:01.127+0000 7f624f644640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6248148d80 con 0x7f62481593e0 2023-10-26T01:38:01.132 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:01.127+0000 7f624f644640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62480ae4c0 con 0x7f62480b0c80 2023-10-26T01:38:01.132 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:01.128+0000 7f624dbe1640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62481597c0 0x7f62481492b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:01.132 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:01.128+0000 7f624dbe1640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62481597c0 0x7f62481492b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:49096/0 (socket says 172.21.15.169:49096) 2023-10-26T01:38:01.132 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:01.128+0000 7f624dbe1640 1 -- 172.21.15.169:0/536746510 learned_addr learned my addr 172.21.15.169:0/536746510 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:38:01.134 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:01.128+0000 7f624cbdf640 1 --2- 172.21.15.169:0/536746510 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f62481593e0 0x7f62480af560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:01.134 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:01.128+0000 7f624dbe1640 1 -- 172.21.15.169:0/536746510 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f62480b0c80 msgr2=0x7f6248148840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:01.134 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:01.128+0000 7f624dbe1640 1 --2- 172.21.15.169:0/536746510 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f62480b0c80 0x7f6248148840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:01.134 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:01.128+0000 7f624dbe1640 1 -- 172.21.15.169:0/536746510 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f62481593e0 msgr2=0x7f62480af560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:01.135 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:01.128+0000 7f624dbe1640 1 --2- 172.21.15.169:0/536746510 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f62481593e0 0x7f62480af560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:01.135 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:01.128+0000 7f624dbe1640 1 -- 172.21.15.169:0/536746510 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f623c009560 con 0x7f62481597c0 2023-10-26T01:38:01.135 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:01.128+0000 7f624dbe1640 1 --2- 172.21.15.169:0/536746510 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62481597c0 0x7f62481492b0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f624000eaa0 tx=0x7f624000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:01.135 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:01.129+0000 7f6239ffb640 1 -- 172.21.15.169:0/536746510 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f624000cdb0 con 0x7f62481597c0 2023-10-26T01:38:01.135 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:01.129+0000 7f6239ffb640 1 -- 172.21.15.169:0/536746510 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6240004b20 con 0x7f62481597c0 2023-10-26T01:38:01.136 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:01.129+0000 7f624f644640 1 -- 172.21.15.169:0/536746510 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62480ae7a0 con 0x7f62481597c0 2023-10-26T01:38:01.136 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:01.129+0000 7f6239ffb640 1 -- 172.21.15.169:0/536746510 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6240005020 con 0x7f62481597c0 2023-10-26T01:38:01.136 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:01.129+0000 7f624f644640 1 -- 172.21.15.169:0/536746510 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62480aed60 con 0x7f62481597c0 2023-10-26T01:38:01.136 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:01.130+0000 7f624f644640 1 -- 172.21.15.169:0/536746510 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f624804db10 con 0x7f62481597c0 2023-10-26T01:38:01.136 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:01.131+0000 7f6239ffb640 1 -- 172.21.15.169:0/536746510 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f6240005180 con 0x7f62481597c0 2023-10-26T01:38:01.137 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:01.132+0000 7f6239ffb640 1 --2- 172.21.15.169:0/536746510 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f622c06ae40 0x7f622c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:01.137 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:01.132+0000 7f6239ffb640 1 -- 172.21.15.169:0/536746510 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f624008a0d0 con 0x7f62481597c0 2023-10-26T01:38:01.137 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:01.132+0000 7f624d3e0640 1 --2- 172.21.15.169:0/536746510 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f622c06ae40 0x7f622c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:01.138 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:01.133+0000 7f624d3e0640 1 --2- 172.21.15.169:0/536746510 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f622c06ae40 0x7f622c06d300 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f6244003cc0 tx=0x7f62440079b0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:01.139 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:01.136+0000 7f6239ffb640 1 -- 172.21.15.169:0/536746510 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f62400550e0 con 0x7f62481597c0 2023-10-26T01:38:01.342 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:01.339+0000 7f624f644640 1 -- 172.21.15.169:0/536746510 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f62480b2170 con 0x7f62481597c0 2023-10-26T01:38:01.343 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:01.340+0000 7f6239ffb640 1 -- 172.21.15.169:0/536746510 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v123) v1 ==== 80+0+0 (secure 0 0 0) 0x7f624001f080 con 0x7f62481597c0 2023-10-26T01:38:01.343 INFO:teuthology.orchestra.run.smithi169.stderr:listed 0 entries 2023-10-26T01:38:01.346 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:01.343+0000 7f624f644640 1 -- 172.21.15.169:0/536746510 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f622c06ae40 msgr2=0x7f622c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:01.346 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:01.343+0000 7f624f644640 1 --2- 172.21.15.169:0/536746510 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f622c06ae40 0x7f622c06d300 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f6244003cc0 tx=0x7f62440079b0 comp rx=0 tx=0).stop 2023-10-26T01:38:01.346 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:01.343+0000 7f624f644640 1 -- 172.21.15.169:0/536746510 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62481597c0 msgr2=0x7f62481492b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:01.346 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:01.343+0000 7f624f644640 1 --2- 172.21.15.169:0/536746510 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62481597c0 0x7f62481492b0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f624000eaa0 tx=0x7f624000ef60 comp rx=0 tx=0).stop 2023-10-26T01:38:01.347 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:01.343+0000 7f624f644640 1 -- 172.21.15.169:0/536746510 shutdown_connections 2023-10-26T01:38:01.347 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:01.343+0000 7f624f644640 1 --2- 172.21.15.169:0/536746510 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f622c06ae40 0x7f622c06d300 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:01.347 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:01.343+0000 7f624f644640 1 --2- 172.21.15.169:0/536746510 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62481597c0 0x7f62481492b0 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:01.347 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:01.343+0000 7f624f644640 1 --2- 172.21.15.169:0/536746510 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f62481593e0 0x7f62480af560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:01.347 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:01.343+0000 7f624f644640 1 --2- 172.21.15.169:0/536746510 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f62480b0c80 0x7f6248148840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:01.348 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:01.343+0000 7f624f644640 1 -- 172.21.15.169:0/536746510 >> 172.21.15.169:0/536746510 conn(0x7f6248069930 msgr2=0x7f624815a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:01.348 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:01.343+0000 7f624f644640 1 -- 172.21.15.169:0/536746510 shutdown_connections 2023-10-26T01:38:01.348 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:01.343+0000 7f624f644640 1 -- 172.21.15.169:0/536746510 wait complete. 2023-10-26T01:38:01.365 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-26T01:38:01.365 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:38:01.365 DEBUG:teuthology.orchestra.run.smithi170:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-26T01:38:01.481 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi170.front.sepia.ceph.com 2023-10-26T01:38:01.482 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:38:01.482 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-26T01:38:01.534 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:38:01.534 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:38:01.596 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054301,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-26T01:38:01.596 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-26T01:38:01.597 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:38:01.657 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054303,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-26T01:38:01.657 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T01:38:01.889 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:01.886+0000 7f10ad452640 1 -- 172.21.15.169:0/947113275 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f10a809a5c0 msgr2=0x7f10a809aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:01.889 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:01.886+0000 7f10ad452640 1 --2- 172.21.15.169:0/947113275 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f10a809a5c0 0x7f10a809aa70 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f10a0005fd0 tx=0x7f10a002be10 comp rx=0 tx=0).stop 2023-10-26T01:38:01.889 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:01.886+0000 7f10ad452640 1 -- 172.21.15.169:0/947113275 shutdown_connections 2023-10-26T01:38:01.889 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:01.886+0000 7f10ad452640 1 --2- 172.21.15.169:0/947113275 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f10a809a5c0 0x7f10a809aa70 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:01.890 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:01.886+0000 7f10ad452640 1 --2- 172.21.15.169:0/947113275 >> v2:172.21.15.170:3300/0 conn(0x7f10a80ac6d0 0x7f10a815d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:01.890 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:01.886+0000 7f10ad452640 1 -- 172.21.15.169:0/947113275 >> 172.21.15.169:0/947113275 conn(0x7f10a80aedd0 msgr2=0x7f10a80af1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:01.890 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:01.886+0000 7f10ad452640 1 -- 172.21.15.169:0/947113275 shutdown_connections 2023-10-26T01:38:01.890 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:01.886+0000 7f10ad452640 1 -- 172.21.15.169:0/947113275 wait complete. 2023-10-26T01:38:01.891 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:01.888+0000 7f10ad452640 1 Processor -- start 2023-10-26T01:38:01.891 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:01.888+0000 7f10ad452640 1 -- start start 2023-10-26T01:38:01.893 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:01.889+0000 7f10ad452640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f10a809a5c0 0x7f10a80c64e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:01.893 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:01.889+0000 7f10ad452640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f10a80ac6d0 0x7f10a80b1220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:01.893 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:01.889+0000 7f10ad452640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f10a80c9970 0x7f10a80b1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:01.893 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:01.889+0000 7f10ad452640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10a80b1d70 con 0x7f10a80ac6d0 2023-10-26T01:38:01.894 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:01.889+0000 7f10ad452640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10a80b2050 con 0x7f10a80c9970 2023-10-26T01:38:01.894 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:01.889+0000 7f10ad452640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10a80b2330 con 0x7f10a809a5c0 2023-10-26T01:38:01.894 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:01.889+0000 7f10a77fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f10a80ac6d0 0x7f10a80b1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:01.894 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:01.889+0000 7f10a77fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f10a80ac6d0 0x7f10a80b1220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:49098/0 (socket says 172.21.15.169:49098) 2023-10-26T01:38:01.894 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:01.889+0000 7f10a77fe640 1 -- 172.21.15.169:0/3245105949 learned_addr learned my addr 172.21.15.169:0/3245105949 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:38:01.895 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:01.889+0000 7f10a7fff640 1 --2- 172.21.15.169:0/3245105949 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f10a809a5c0 0x7f10a80c64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:01.895 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:01.889+0000 7f10acc51640 1 --2- 172.21.15.169:0/3245105949 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f10a80c9970 0x7f10a80b1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:01.895 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:01.890+0000 7f10a77fe640 1 -- 172.21.15.169:0/3245105949 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f10a809a5c0 msgr2=0x7f10a80c64e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:01.895 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:01.890+0000 7f10a77fe640 1 --2- 172.21.15.169:0/3245105949 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f10a809a5c0 0x7f10a80c64e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:01.895 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:01.890+0000 7f10a77fe640 1 -- 172.21.15.169:0/3245105949 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f10a80c9970 msgr2=0x7f10a80b1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:01.896 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:01.890+0000 7f10a77fe640 1 --2- 172.21.15.169:0/3245105949 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f10a80c9970 0x7f10a80b1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:01.896 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:01.890+0000 7f10a77fe640 1 -- 172.21.15.169:0/3245105949 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10a0009560 con 0x7f10a80ac6d0 2023-10-26T01:38:01.896 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:01.890+0000 7f10acc51640 1 --2- 172.21.15.169:0/3245105949 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f10a80c9970 0x7f10a80b1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:38:01.896 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:01.890+0000 7f10a77fe640 1 --2- 172.21.15.169:0/3245105949 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f10a80ac6d0 0x7f10a80b1220 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f109800c730 tx=0x7f109800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:01.896 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:01.890+0000 7f10a4ff9640 1 -- 172.21.15.169:0/3245105949 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1098006d50 con 0x7f10a80ac6d0 2023-10-26T01:38:01.897 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:01.890+0000 7f10a4ff9640 1 -- 172.21.15.169:0/3245105949 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f109800ce40 con 0x7f10a80ac6d0 2023-10-26T01:38:01.897 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:01.890+0000 7f10ad452640 1 -- 172.21.15.169:0/3245105949 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10a80b2610 con 0x7f10a80ac6d0 2023-10-26T01:38:01.897 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:01.890+0000 7f10a4ff9640 1 -- 172.21.15.169:0/3245105949 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1098003bd0 con 0x7f10a80ac6d0 2023-10-26T01:38:01.897 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:01.890+0000 7f10ad452640 1 -- 172.21.15.169:0/3245105949 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10a80b2e10 con 0x7f10a80ac6d0 2023-10-26T01:38:01.898 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:01.892+0000 7f10ad452640 1 -- 172.21.15.169:0/3245105949 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10a80b2b30 con 0x7f10a80ac6d0 2023-10-26T01:38:01.898 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:01.892+0000 7f10a4ff9640 1 -- 172.21.15.169:0/3245105949 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f1098003d30 con 0x7f10a80ac6d0 2023-10-26T01:38:01.899 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:01.894+0000 7f10a4ff9640 1 --2- 172.21.15.169:0/3245105949 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f108806ae40 0x7f108806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:01.899 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:01.894+0000 7f10a4ff9640 1 -- 172.21.15.169:0/3245105949 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1098089350 con 0x7f10a80ac6d0 2023-10-26T01:38:01.899 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:01.895+0000 7f10a7fff640 1 --2- 172.21.15.169:0/3245105949 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f108806ae40 0x7f108806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:01.899 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:01.896+0000 7f10a7fff640 1 --2- 172.21.15.169:0/3245105949 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f108806ae40 0x7f108806d300 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f1090009aa0 tx=0x7f1090005f90 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:01.901 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:01.898+0000 7f10a4ff9640 1 -- 172.21.15.169:0/3245105949 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f10a80b2b30 con 0x7f10a80ac6d0 2023-10-26T01:38:02.127 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:02.124+0000 7f10ad452640 1 -- 172.21.15.169:0/3245105949 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f10a80ca410 con 0x7f10a80ac6d0 2023-10-26T01:38:02.128 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:02.126+0000 7f10a4ff9640 1 -- 172.21.15.169:0/3245105949 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) v1 ==== 94+0+4924 (secure 0 0 0) 0x7f10a80ca410 con 0x7f10a80ac6d0 2023-10-26T01:38:02.129 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:38:02.129 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":59,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5387,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/1186563921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1186563921},{"type":"v1","addr":"172.21.15.170:6835","nonce":1186563921}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":54},{"gid":5390,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/3702036693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":3702036693},{"type":"v1","addr":"172.21.15.170:6837","nonce":3702036693}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":54},{"gid":7332,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/3193888686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":3193888686},{"type":"v1","addr":"172.21.15.169:6833","nonce":3193888686}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T01:37:40.027858+0000","modified":"2023-10-26T01:37:43.103474+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7335},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7335":{"gid":7335,"name":"c","rank":0,"incarnation":55,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6835/668588916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":668588916},{"type":"v1","addr":"172.21.15.169:6835","nonce":668588916}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-10-26T01:38:02.129 INFO:teuthology.orchestra.run.smithi169.stderr:dumped fsmap epoch 59 2023-10-26T01:38:02.131 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:02.128+0000 7f10ad452640 1 -- 172.21.15.169:0/3245105949 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f108806ae40 msgr2=0x7f108806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:02.131 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:02.128+0000 7f10ad452640 1 --2- 172.21.15.169:0/3245105949 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f108806ae40 0x7f108806d300 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f1090009aa0 tx=0x7f1090005f90 comp rx=0 tx=0).stop 2023-10-26T01:38:02.131 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:02.128+0000 7f10ad452640 1 -- 172.21.15.169:0/3245105949 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f10a80ac6d0 msgr2=0x7f10a80b1220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:02.131 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:02.128+0000 7f10ad452640 1 --2- 172.21.15.169:0/3245105949 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f10a80ac6d0 0x7f10a80b1220 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f109800c730 tx=0x7f109800cbf0 comp rx=0 tx=0).stop 2023-10-26T01:38:02.132 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:02.128+0000 7f10ad452640 1 -- 172.21.15.169:0/3245105949 shutdown_connections 2023-10-26T01:38:02.132 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:02.128+0000 7f10ad452640 1 --2- 172.21.15.169:0/3245105949 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f108806ae40 0x7f108806d300 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:02.132 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:02.128+0000 7f10ad452640 1 --2- 172.21.15.169:0/3245105949 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f10a80ac6d0 0x7f10a80b1220 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:02.133 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:02.128+0000 7f10ad452640 1 --2- 172.21.15.169:0/3245105949 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f10a80c9970 0x7f10a80b1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:02.133 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:02.128+0000 7f10ad452640 1 --2- 172.21.15.169:0/3245105949 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f10a809a5c0 0x7f10a80c64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:02.133 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:02.128+0000 7f10ad452640 1 -- 172.21.15.169:0/3245105949 >> 172.21.15.169:0/3245105949 conn(0x7f10a80aedd0 msgr2=0x7f10a80ab720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:02.133 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:02.129+0000 7f10ad452640 1 -- 172.21.15.169:0/3245105949 shutdown_connections 2023-10-26T01:38:02.133 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:02.129+0000 7f10ad452640 1 -- 172.21.15.169:0/3245105949 wait complete. 2023-10-26T01:38:02.150 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:38:02.367 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:02.364+0000 7f93b2ce5640 1 -- 172.21.15.169:0/3413061633 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93ac0cdbb0 msgr2=0x7f93ac0b07d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:02.367 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:02.364+0000 7f93b2ce5640 1 --2- 172.21.15.169:0/3413061633 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93ac0cdbb0 0x7f93ac0b07d0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f93ac06c960 tx=0x7f93a402c260 comp rx=0 tx=0).stop 2023-10-26T01:38:02.367 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:02.364+0000 7f93b2ce5640 1 -- 172.21.15.169:0/3413061633 shutdown_connections 2023-10-26T01:38:02.367 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:02.364+0000 7f93b2ce5640 1 --2- 172.21.15.169:0/3413061633 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93ac0cdbb0 0x7f93ac0b07d0 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:02.368 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:02.364+0000 7f93b2ce5640 1 --2- 172.21.15.169:0/3413061633 >> v2:172.21.15.170:3300/0 conn(0x7f93ac06ed30 0x7f93ac0cd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:02.368 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:02.364+0000 7f93b2ce5640 1 -- 172.21.15.169:0/3413061633 >> 172.21.15.169:0/3413061633 conn(0x7f93ac069930 msgr2=0x7f93ac069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:02.368 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:02.364+0000 7f93b2ce5640 1 -- 172.21.15.169:0/3413061633 shutdown_connections 2023-10-26T01:38:02.368 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:02.364+0000 7f93b2ce5640 1 -- 172.21.15.169:0/3413061633 wait complete. 2023-10-26T01:38:02.370 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:02.366+0000 7f93b2ce5640 1 Processor -- start 2023-10-26T01:38:02.370 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:02.366+0000 7f93b2ce5640 1 -- start start 2023-10-26T01:38:02.370 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:02.367+0000 7f93b2ce5640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93ac06ed30 0x7f93ac14f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:02.371 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:02.367+0000 7f93b2ce5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f93ac0cdbb0 0x7f93ac153d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:02.371 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:02.367+0000 7f93b2ce5640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f93ac15b610 0x7f93ac151c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:02.371 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:02.367+0000 7f93b2ce5640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93ac1542b0 con 0x7f93ac06ed30 2023-10-26T01:38:02.371 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:02.367+0000 7f93b2ce5640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93ac14fda0 con 0x7f93ac0cdbb0 2023-10-26T01:38:02.371 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:02.367+0000 7f93b2ce5640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93ac150080 con 0x7f93ac15b610 2023-10-26T01:38:02.371 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:02.367+0000 7f93b0a81640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93ac06ed30 0x7f93ac14f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:02.372 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:02.367+0000 7f93b0a81640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93ac06ed30 0x7f93ac14f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:49108/0 (socket says 172.21.15.169:49108) 2023-10-26T01:38:02.372 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:02.367+0000 7f93b0a81640 1 -- 172.21.15.169:0/3307965395 learned_addr learned my addr 172.21.15.169:0/3307965395 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:38:02.372 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:02.367+0000 7f93abfff640 1 --2- 172.21.15.169:0/3307965395 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f93ac0cdbb0 0x7f93ac153d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:02.372 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:02.367+0000 7f93b0a81640 1 -- 172.21.15.169:0/3307965395 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f93ac15b610 msgr2=0x7f93ac151c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:02.372 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:02.367+0000 7f93b0a81640 1 --2- 172.21.15.169:0/3307965395 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f93ac15b610 0x7f93ac151c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:02.373 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:02.367+0000 7f93b0a81640 1 -- 172.21.15.169:0/3307965395 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f93ac0cdbb0 msgr2=0x7f93ac153d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:02.373 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:02.367+0000 7f93b0a81640 1 --2- 172.21.15.169:0/3307965395 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f93ac0cdbb0 0x7f93ac153d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:02.373 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:02.367+0000 7f93b0a81640 1 -- 172.21.15.169:0/3307965395 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93a4009900 con 0x7f93ac06ed30 2023-10-26T01:38:02.374 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:02.368+0000 7f93b0a81640 1 --2- 172.21.15.169:0/3307965395 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93ac06ed30 0x7f93ac14f7c0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f93a000b3e0 tx=0x7f93a000b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:02.375 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:02.368+0000 7f93a97fa640 1 -- 172.21.15.169:0/3307965395 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a0011830 con 0x7f93ac06ed30 2023-10-26T01:38:02.375 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:02.368+0000 7f93a97fa640 1 -- 172.21.15.169:0/3307965395 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f93a0011aa0 con 0x7f93ac06ed30 2023-10-26T01:38:02.375 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:02.368+0000 7f93a97fa640 1 -- 172.21.15.169:0/3307965395 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a0010380 con 0x7f93ac06ed30 2023-10-26T01:38:02.375 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:02.368+0000 7f93b2ce5640 1 -- 172.21.15.169:0/3307965395 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93ac150360 con 0x7f93ac06ed30 2023-10-26T01:38:02.375 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:02.368+0000 7f93b2ce5640 1 -- 172.21.15.169:0/3307965395 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93ac150920 con 0x7f93ac06ed30 2023-10-26T01:38:02.376 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:02.369+0000 7f93b2ce5640 1 -- 172.21.15.169:0/3307965395 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93ac04db10 con 0x7f93ac06ed30 2023-10-26T01:38:02.376 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:02.370+0000 7f93a97fa640 1 -- 172.21.15.169:0/3307965395 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f93a00104e0 con 0x7f93ac06ed30 2023-10-26T01:38:02.376 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:02.371+0000 7f93a97fa640 1 --2- 172.21.15.169:0/3307965395 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f939006ae70 0x7f939006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:02.376 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:02.371+0000 7f93a97fa640 1 -- 172.21.15.169:0/3307965395 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f93a0089ac0 con 0x7f93ac06ed30 2023-10-26T01:38:02.376 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:02.372+0000 7f93abfff640 1 --2- 172.21.15.169:0/3307965395 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f939006ae70 0x7f939006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:02.377 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:02.373+0000 7f93abfff640 1 --2- 172.21.15.169:0/3307965395 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f939006ae70 0x7f939006d330 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f93ac1f5490 tx=0x7f9398005cc0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:02.378 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:02.376+0000 7f93a97fa640 1 -- 172.21.15.169:0/3307965395 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f93a0054ad0 con 0x7f93ac06ed30 2023-10-26T01:38:02.572 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:02.569+0000 7f93b2ce5640 1 -- 172.21.15.169:0/3307965395 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f93ac150d20 con 0x7f93ac06ed30 2023-10-26T01:38:02.573 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:02.570+0000 7f93a97fa640 1 -- 172.21.15.169:0/3307965395 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v123) v1 ==== 74+0+16575 (secure 0 0 0) 0x7f93a0015090 con 0x7f93ac06ed30 2023-10-26T01:38:02.574 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:38:02.574 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":123,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:37:54.102485+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-10-26T01:37:37.959102+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-10-26T01:37:39.053240+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:38:02.576 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:02.573+0000 7f93b2ce5640 1 -- 172.21.15.169:0/3307965395 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f939006ae70 msgr2=0x7f939006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:02.576 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:02.573+0000 7f93b2ce5640 1 --2- 172.21.15.169:0/3307965395 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f939006ae70 0x7f939006d330 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f93ac1f5490 tx=0x7f9398005cc0 comp rx=0 tx=0).stop 2023-10-26T01:38:02.576 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:02.573+0000 7f93b2ce5640 1 -- 172.21.15.169:0/3307965395 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93ac06ed30 msgr2=0x7f93ac14f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:02.577 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:02.573+0000 7f93b2ce5640 1 --2- 172.21.15.169:0/3307965395 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93ac06ed30 0x7f93ac14f7c0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f93a000b3e0 tx=0x7f93a000b8a0 comp rx=0 tx=0).stop 2023-10-26T01:38:02.577 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:02.573+0000 7f93b2ce5640 1 -- 172.21.15.169:0/3307965395 shutdown_connections 2023-10-26T01:38:02.577 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:02.573+0000 7f93b2ce5640 1 --2- 172.21.15.169:0/3307965395 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f939006ae70 0x7f939006d330 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:02.577 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:02.573+0000 7f93b2ce5640 1 --2- 172.21.15.169:0/3307965395 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f93ac15b610 0x7f93ac151c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:02.577 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:02.573+0000 7f93b2ce5640 1 --2- 172.21.15.169:0/3307965395 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f93ac0cdbb0 0x7f93ac153d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:02.578 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:02.573+0000 7f93b2ce5640 1 --2- 172.21.15.169:0/3307965395 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93ac06ed30 0x7f93ac14f7c0 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:02.578 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:02.573+0000 7f93b2ce5640 1 -- 172.21.15.169:0/3307965395 >> 172.21.15.169:0/3307965395 conn(0x7f93ac069930 msgr2=0x7f93ac0ae470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:02.578 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:02.573+0000 7f93b2ce5640 1 -- 172.21.15.169:0/3307965395 shutdown_connections 2023-10-26T01:38:02.578 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:02.573+0000 7f93b2ce5640 1 -- 172.21.15.169:0/3307965395 wait complete. 2023-10-26T01:38:02.596 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T01:30:32.200808+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '109', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-26T01:37:37.959102+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '119', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2023-10-26T01:37:39.053240+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T01:38:02.597 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T01:38:02.813 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:02.810+0000 7faa0ff3c640 1 -- 172.21.15.169:0/1515381286 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faa080b0a80 msgr2=0x7faa08158f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:02.814 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:02.810+0000 7faa0ff3c640 1 --2- 172.21.15.169:0/1515381286 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faa080b0a80 0x7faa08158f70 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7faa0806c3b0 tx=0x7fa9fc02bd70 comp rx=0 tx=0).stop 2023-10-26T01:38:02.814 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:02.810+0000 7faa0ff3c640 1 -- 172.21.15.169:0/1515381286 shutdown_connections 2023-10-26T01:38:02.814 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:02.810+0000 7faa0ff3c640 1 --2- 172.21.15.169:0/1515381286 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faa080b0a80 0x7faa08158f70 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:02.814 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:02.810+0000 7faa0ff3c640 1 --2- 172.21.15.169:0/1515381286 >> v2:172.21.15.170:3300/0 conn(0x7faa080ab070 0x7faa080ac870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:02.814 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:02.810+0000 7faa0ff3c640 1 -- 172.21.15.169:0/1515381286 >> 172.21.15.169:0/1515381286 conn(0x7faa08069930 msgr2=0x7faa08069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:02.815 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:02.810+0000 7faa0ff3c640 1 -- 172.21.15.169:0/1515381286 shutdown_connections 2023-10-26T01:38:02.815 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:02.810+0000 7faa0ff3c640 1 -- 172.21.15.169:0/1515381286 wait complete. 2023-10-26T01:38:02.816 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:02.812+0000 7faa0ff3c640 1 Processor -- start 2023-10-26T01:38:02.816 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:02.812+0000 7faa0ff3c640 1 -- start start 2023-10-26T01:38:02.816 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:02.813+0000 7faa0ff3c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faa080ab070 0x7faa0814e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:02.817 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:02.813+0000 7faa0ff3c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faa080cd760 0x7faa08147010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:02.817 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:02.813+0000 7faa0ff3c640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7faa0814f010 0x7faa080cbfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:02.817 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:02.813+0000 7faa0ff3c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa0814cb90 con 0x7faa080cd760 2023-10-26T01:38:02.817 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:02.813+0000 7faa0ff3c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa080cb1f0 con 0x7faa080ab070 2023-10-26T01:38:02.817 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:02.813+0000 7faa0ff3c640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa080cb4d0 con 0x7faa0814f010 2023-10-26T01:38:02.818 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:02.813+0000 7faa0d4d7640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faa080cd760 0x7faa08147010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:02.818 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:02.813+0000 7faa0d4d7640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faa080cd760 0x7faa08147010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:49130/0 (socket says 172.21.15.169:49130) 2023-10-26T01:38:02.818 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:02.813+0000 7faa0d4d7640 1 -- 172.21.15.169:0/3365357148 learned_addr learned my addr 172.21.15.169:0/3365357148 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:38:02.819 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:02.814+0000 7faa0d4d7640 1 -- 172.21.15.169:0/3365357148 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7faa0814f010 msgr2=0x7faa080cbfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:02.820 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:02.814+0000 7faa0d4d7640 1 --2- 172.21.15.169:0/3365357148 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7faa0814f010 0x7faa080cbfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:02.820 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:02.814+0000 7faa0d4d7640 1 -- 172.21.15.169:0/3365357148 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faa080ab070 msgr2=0x7faa0814e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:02.820 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:02.814+0000 7faa0d4d7640 1 --2- 172.21.15.169:0/3365357148 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faa080ab070 0x7faa0814e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:02.820 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:02.814+0000 7faa0d4d7640 1 -- 172.21.15.169:0/3365357148 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9fc009490 con 0x7faa080cd760 2023-10-26T01:38:02.821 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:02.814+0000 7faa0d4d7640 1 --2- 172.21.15.169:0/3365357148 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faa080cd760 0x7faa08147010 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc000c00 tx=0x7fa9fc0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:02.821 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:02.814+0000 7fa9fa7fc640 1 -- 172.21.15.169:0/3365357148 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9fc003ea0 con 0x7faa080cd760 2023-10-26T01:38:02.821 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:02.814+0000 7fa9fa7fc640 1 -- 172.21.15.169:0/3365357148 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa9fc034550 con 0x7faa080cd760 2023-10-26T01:38:02.821 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:02.814+0000 7faa0ff3c640 1 -- 172.21.15.169:0/3365357148 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa080cb7b0 con 0x7faa080cd760 2023-10-26T01:38:02.821 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:02.815+0000 7fa9fa7fc640 1 -- 172.21.15.169:0/3365357148 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9fc034960 con 0x7faa080cd760 2023-10-26T01:38:02.822 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:02.815+0000 7faa0ff3c640 1 -- 172.21.15.169:0/3365357148 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa08147550 con 0x7faa080cd760 2023-10-26T01:38:02.822 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:02.816+0000 7faa0ff3c640 1 -- 172.21.15.169:0/3365357148 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa08066af0 con 0x7faa080cd760 2023-10-26T01:38:02.822 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:02.817+0000 7fa9fa7fc640 1 -- 172.21.15.169:0/3365357148 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fa9fc03b040 con 0x7faa080cd760 2023-10-26T01:38:02.823 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:02.818+0000 7fa9fa7fc640 1 --2- 172.21.15.169:0/3365357148 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa9ec06ae40 0x7fa9ec06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:02.823 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:02.818+0000 7fa9fa7fc640 1 -- 172.21.15.169:0/3365357148 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa9fc0abbe0 con 0x7faa080cd760 2023-10-26T01:38:02.823 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:02.819+0000 7faa0dcd8640 1 --2- 172.21.15.169:0/3365357148 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa9ec06ae40 0x7fa9ec06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:02.823 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:02.820+0000 7faa0dcd8640 1 --2- 172.21.15.169:0/3365357148 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa9ec06ae40 0x7fa9ec06d300 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7faa04005fd0 tx=0x7faa04005850 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:02.826 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:02.824+0000 7fa9fa7fc640 1 -- 172.21.15.169:0/3365357148 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa9fc076c80 con 0x7faa080cd760 2023-10-26T01:38:03.015 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.012+0000 7faa0ff3c640 1 -- 172.21.15.169:0/3365357148 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7faa080cbcd0 con 0x7faa080cd760 2023-10-26T01:38:03.015 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.012+0000 7fa9fa7fc640 1 -- 172.21.15.169:0/3365357148 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v123) v1 ==== 93+0+10 (secure 0 0 0) 0x7faa080cbcd0 con 0x7faa080cd760 2023-10-26T01:38:03.016 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 1 2023-10-26T01:38:03.018 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.015+0000 7faa0ff3c640 1 -- 172.21.15.169:0/3365357148 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa9ec06ae40 msgr2=0x7fa9ec06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:03.018 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.015+0000 7faa0ff3c640 1 --2- 172.21.15.169:0/3365357148 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa9ec06ae40 0x7fa9ec06d300 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7faa04005fd0 tx=0x7faa04005850 comp rx=0 tx=0).stop 2023-10-26T01:38:03.018 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.015+0000 7faa0ff3c640 1 -- 172.21.15.169:0/3365357148 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faa080cd760 msgr2=0x7faa08147010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:03.019 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.015+0000 7faa0ff3c640 1 --2- 172.21.15.169:0/3365357148 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faa080cd760 0x7faa08147010 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc000c00 tx=0x7fa9fc0038a0 comp rx=0 tx=0).stop 2023-10-26T01:38:03.019 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.015+0000 7faa0ff3c640 1 -- 172.21.15.169:0/3365357148 shutdown_connections 2023-10-26T01:38:03.019 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.015+0000 7faa0ff3c640 1 --2- 172.21.15.169:0/3365357148 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa9ec06ae40 0x7fa9ec06d300 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:03.019 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.015+0000 7faa0ff3c640 1 --2- 172.21.15.169:0/3365357148 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faa080cd760 0x7faa08147010 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:03.019 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.015+0000 7faa0ff3c640 1 --2- 172.21.15.169:0/3365357148 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7faa0814f010 0x7faa080cbfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:03.020 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.015+0000 7faa0ff3c640 1 --2- 172.21.15.169:0/3365357148 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faa080ab070 0x7faa0814e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:03.020 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.015+0000 7faa0ff3c640 1 -- 172.21.15.169:0/3365357148 >> 172.21.15.169:0/3365357148 conn(0x7faa08069930 msgr2=0x7faa08154d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:03.020 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.015+0000 7faa0ff3c640 1 -- 172.21.15.169:0/3365357148 shutdown_connections 2023-10-26T01:38:03.020 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.015+0000 7faa0ff3c640 1 -- 172.21.15.169:0/3365357148 wait complete. 2023-10-26T01:38:03.036 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-26T01:38:03.252 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.249+0000 7f1bf19c9640 1 -- 172.21.15.169:0/1050284972 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1bec15bbb0 msgr2=0x7f1bec0cdc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:03.253 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.249+0000 7f1bf19c9640 1 --2- 172.21.15.169:0/1050284972 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1bec15bbb0 0x7f1bec0cdc90 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f1bec0b0570 tx=0x7f1be402c080 comp rx=0 tx=0).stop 2023-10-26T01:38:03.253 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.249+0000 7f1bf19c9640 1 -- 172.21.15.169:0/1050284972 shutdown_connections 2023-10-26T01:38:03.253 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.249+0000 7f1bf19c9640 1 --2- 172.21.15.169:0/1050284972 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1bec15bbb0 0x7f1bec0cdc90 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:03.253 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.250+0000 7f1bf19c9640 1 --2- 172.21.15.169:0/1050284972 >> v2:172.21.15.170:3300/0 conn(0x7f1bec06ce00 0x7f1bec155360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:03.254 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.250+0000 7f1bf19c9640 1 -- 172.21.15.169:0/1050284972 >> 172.21.15.169:0/1050284972 conn(0x7f1bec069930 msgr2=0x7f1bec069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:03.254 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.250+0000 7f1bf19c9640 1 -- 172.21.15.169:0/1050284972 shutdown_connections 2023-10-26T01:38:03.254 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.250+0000 7f1bf19c9640 1 -- 172.21.15.169:0/1050284972 wait complete. 2023-10-26T01:38:03.255 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.252+0000 7f1bf19c9640 1 Processor -- start 2023-10-26T01:38:03.255 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.252+0000 7f1bf19c9640 1 -- start start 2023-10-26T01:38:03.256 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.253+0000 7f1bf19c9640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1bec06ce00 0x7f1bec14d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:03.256 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.253+0000 7f1bf19c9640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1bec0cd7e0 0x7f1bec14bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:03.256 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.253+0000 7f1bf19c9640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1bec15bbb0 0x7f1bec14c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:03.257 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.253+0000 7f1bf19c9640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bec150670 con 0x7f1bec06ce00 2023-10-26T01:38:03.257 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.253+0000 7f1bf19c9640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bec150910 con 0x7f1bec0cd7e0 2023-10-26T01:38:03.257 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.253+0000 7f1bf19c9640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bec150bf0 con 0x7f1bec15bbb0 2023-10-26T01:38:03.257 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.253+0000 7f1beaffd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1bec06ce00 0x7f1bec14d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:03.257 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.253+0000 7f1beaffd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1bec06ce00 0x7f1bec14d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:49146/0 (socket says 172.21.15.169:49146) 2023-10-26T01:38:03.258 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.253+0000 7f1beaffd640 1 -- 172.21.15.169:0/281383758 learned_addr learned my addr 172.21.15.169:0/281383758 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:38:03.258 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.253+0000 7f1beaffd640 1 -- 172.21.15.169:0/281383758 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1bec15bbb0 msgr2=0x7f1bec14c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:03.258 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.253+0000 7f1beaffd640 1 --2- 172.21.15.169:0/281383758 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1bec15bbb0 0x7f1bec14c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:03.258 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.253+0000 7f1beaffd640 1 -- 172.21.15.169:0/281383758 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1bec0cd7e0 msgr2=0x7f1bec14bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:03.258 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.253+0000 7f1beaffd640 1 --2- 172.21.15.169:0/281383758 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1bec0cd7e0 0x7f1bec14bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:03.259 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.253+0000 7f1beaffd640 1 -- 172.21.15.169:0/281383758 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1be4009840 con 0x7f1bec06ce00 2023-10-26T01:38:03.259 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.254+0000 7f1beaffd640 1 --2- 172.21.15.169:0/281383758 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1bec06ce00 0x7f1bec14d520 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc00b4a0 tx=0x7f1bdc00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:03.260 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.254+0000 7f1bd3fff640 1 -- 172.21.15.169:0/281383758 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bdc010610 con 0x7f1bec06ce00 2023-10-26T01:38:03.260 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.254+0000 7f1bd3fff640 1 -- 172.21.15.169:0/281383758 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1bdc010860 con 0x7f1bec06ce00 2023-10-26T01:38:03.260 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.254+0000 7f1bd3fff640 1 -- 172.21.15.169:0/281383758 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bdc010c70 con 0x7f1bec06ce00 2023-10-26T01:38:03.260 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.254+0000 7f1bf19c9640 1 -- 172.21.15.169:0/281383758 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bec150ed0 con 0x7f1bec06ce00 2023-10-26T01:38:03.261 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.254+0000 7f1bf19c9640 1 -- 172.21.15.169:0/281383758 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bec14cca0 con 0x7f1bec06ce00 2023-10-26T01:38:03.261 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.255+0000 7f1bf19c9640 1 -- 172.21.15.169:0/281383758 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bec1513f0 con 0x7f1bec06ce00 2023-10-26T01:38:03.261 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.256+0000 7f1bd3fff640 1 -- 172.21.15.169:0/281383758 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f1bdc017360 con 0x7f1bec06ce00 2023-10-26T01:38:03.261 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.257+0000 7f1bd3fff640 1 --2- 172.21.15.169:0/281383758 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f1bd806ae70 0x7f1bd806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:03.261 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.257+0000 7f1bd3fff640 1 -- 172.21.15.169:0/281383758 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1bdc089490 con 0x7f1bec06ce00 2023-10-26T01:38:03.262 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.258+0000 7f1bea7fc640 1 --2- 172.21.15.169:0/281383758 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f1bd806ae70 0x7f1bd806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:03.262 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.259+0000 7f1bea7fc640 1 --2- 172.21.15.169:0/281383758 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f1bd806ae70 0x7f1bd806d330 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f1be0000f60 tx=0x7f1be0007a50 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:03.264 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.262+0000 7f1bd3fff640 1 -- 172.21.15.169:0/281383758 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1bec1513f0 con 0x7f1bec06ce00 2023-10-26T01:38:03.453 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.450+0000 7f1bf19c9640 1 -- 172.21.15.169:0/281383758 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1bec1513f0 con 0x7f1bec06ce00 2023-10-26T01:38:03.454 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.451+0000 7f1bd3fff640 1 -- 172.21.15.169:0/281383758 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v123) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1bec1513f0 con 0x7f1bec06ce00 2023-10-26T01:38:03.455 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 32 2023-10-26T01:38:03.458 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.454+0000 7f1bf19c9640 1 -- 172.21.15.169:0/281383758 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f1bd806ae70 msgr2=0x7f1bd806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:03.458 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.454+0000 7f1bf19c9640 1 --2- 172.21.15.169:0/281383758 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f1bd806ae70 0x7f1bd806d330 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f1be0000f60 tx=0x7f1be0007a50 comp rx=0 tx=0).stop 2023-10-26T01:38:03.458 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.454+0000 7f1bf19c9640 1 -- 172.21.15.169:0/281383758 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1bec06ce00 msgr2=0x7f1bec14d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:03.458 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.454+0000 7f1bf19c9640 1 --2- 172.21.15.169:0/281383758 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1bec06ce00 0x7f1bec14d520 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc00b4a0 tx=0x7f1bdc00b960 comp rx=0 tx=0).stop 2023-10-26T01:38:03.458 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.454+0000 7f1bf19c9640 1 -- 172.21.15.169:0/281383758 shutdown_connections 2023-10-26T01:38:03.459 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.454+0000 7f1bf19c9640 1 --2- 172.21.15.169:0/281383758 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f1bd806ae70 0x7f1bd806d330 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:03.459 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.454+0000 7f1bf19c9640 1 --2- 172.21.15.169:0/281383758 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1bec15bbb0 0x7f1bec14c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:03.459 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.454+0000 7f1bf19c9640 1 --2- 172.21.15.169:0/281383758 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1bec0cd7e0 0x7f1bec14bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:03.459 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.454+0000 7f1bf19c9640 1 --2- 172.21.15.169:0/281383758 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1bec06ce00 0x7f1bec14d520 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:03.459 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.454+0000 7f1bf19c9640 1 -- 172.21.15.169:0/281383758 >> 172.21.15.169:0/281383758 conn(0x7f1bec069930 msgr2=0x7f1bec159790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:03.460 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.455+0000 7f1bf19c9640 1 -- 172.21.15.169:0/281383758 shutdown_connections 2023-10-26T01:38:03.460 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.455+0000 7f1bf19c9640 1 -- 172.21.15.169:0/281383758 wait complete. 2023-10-26T01:38:03.475 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-26T01:38:03.696 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.693+0000 7f2f95c87640 1 -- 172.21.15.169:0/1186421730 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f78004ab0 msgr2=0x7f2f78004ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:03.697 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.693+0000 7f2f95c87640 1 --2- 172.21.15.169:0/1186421730 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f78004ab0 0x7f2f78004ed0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f2f80009b20 tx=0x7f2f78006900 comp rx=0 tx=0).stop 2023-10-26T01:38:03.697 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.693+0000 7f2f8ffff640 1 -- 172.21.15.169:0/1186421730 reap_dead start 2023-10-26T01:38:03.697 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.693+0000 7f2f95c87640 1 -- 172.21.15.169:0/1186421730 shutdown_connections 2023-10-26T01:38:03.698 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.693+0000 7f2f95c87640 1 -- 172.21.15.169:0/1186421730 >> 172.21.15.169:0/1186421730 conn(0x7f2f90069930 msgr2=0x7f2f90069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:03.698 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.694+0000 7f2f95c87640 1 -- 172.21.15.169:0/1186421730 shutdown_connections 2023-10-26T01:38:03.698 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.694+0000 7f2f95c87640 1 -- 172.21.15.169:0/1186421730 wait complete. 2023-10-26T01:38:03.700 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.695+0000 7f2f95c87640 1 Processor -- start 2023-10-26T01:38:03.701 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.695+0000 7f2f95c87640 1 -- start start 2023-10-26T01:38:03.701 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.696+0000 7f2f95c87640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f78004ab0 0x7f2f901543c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:03.701 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.696+0000 7f2f95c87640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2f90098290 0x7f2f901f1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:03.701 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.696+0000 7f2f95c87640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2f901f2410 0x7f2f900995a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:03.701 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.696+0000 7f2f95c87640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f90154900 con 0x7f2f78004ab0 2023-10-26T01:38:03.702 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.696+0000 7f2f95c87640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f9006feb0 con 0x7f2f90098290 2023-10-26T01:38:03.702 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.696+0000 7f2f95c87640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f90070190 con 0x7f2f901f2410 2023-10-26T01:38:03.702 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.697+0000 7f2f8f7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f78004ab0 0x7f2f901543c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:03.703 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.697+0000 7f2f8f7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f78004ab0 0x7f2f901543c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:49164/0 (socket says 172.21.15.169:49164) 2023-10-26T01:38:03.703 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.697+0000 7f2f8f7fe640 1 -- 172.21.15.169:0/2147023019 learned_addr learned my addr 172.21.15.169:0/2147023019 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:38:03.703 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.697+0000 7f2f8effd640 1 --2- 172.21.15.169:0/2147023019 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2f90098290 0x7f2f901f1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:03.704 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.697+0000 7f2f8ffff640 1 --2- 172.21.15.169:0/2147023019 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2f901f2410 0x7f2f900995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:03.704 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.697+0000 7f2f8f7fe640 1 -- 172.21.15.169:0/2147023019 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2f901f2410 msgr2=0x7f2f900995a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:03.704 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.697+0000 7f2f8f7fe640 1 --2- 172.21.15.169:0/2147023019 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2f901f2410 0x7f2f900995a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:03.704 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.697+0000 7f2f8f7fe640 1 -- 172.21.15.169:0/2147023019 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2f90098290 msgr2=0x7f2f901f1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:03.704 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.697+0000 7f2f8f7fe640 1 --2- 172.21.15.169:0/2147023019 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2f90098290 0x7f2f901f1ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:03.705 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.697+0000 7f2f8f7fe640 1 -- 172.21.15.169:0/2147023019 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f90070470 con 0x7f2f78004ab0 2023-10-26T01:38:03.705 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.697+0000 7f2f8effd640 1 --2- 172.21.15.169:0/2147023019 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2f90098290 0x7f2f901f1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:38:03.705 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.698+0000 7f2f8f7fe640 1 --2- 172.21.15.169:0/2147023019 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f78004ab0 0x7f2f901543c0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f2f80009a70 tx=0x7f2f80033c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:03.705 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.698+0000 7f2f77fff640 1 -- 172.21.15.169:0/2147023019 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f80004220 con 0x7f2f78004ab0 2023-10-26T01:38:03.705 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.698+0000 7f2f77fff640 1 -- 172.21.15.169:0/2147023019 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2f800043a0 con 0x7f2f78004ab0 2023-10-26T01:38:03.706 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.698+0000 7f2f95c87640 1 -- 172.21.15.169:0/2147023019 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f90070750 con 0x7f2f78004ab0 2023-10-26T01:38:03.706 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.698+0000 7f2f77fff640 1 -- 172.21.15.169:0/2147023019 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f8003d940 con 0x7f2f78004ab0 2023-10-26T01:38:03.706 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.698+0000 7f2f95c87640 1 -- 172.21.15.169:0/2147023019 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f90071de0 con 0x7f2f78004ab0 2023-10-26T01:38:03.707 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.699+0000 7f2f95c87640 1 -- 172.21.15.169:0/2147023019 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f90071f70 con 0x7f2f78004ab0 2023-10-26T01:38:03.707 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.700+0000 7f2f77fff640 1 -- 172.21.15.169:0/2147023019 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f2f8003daa0 con 0x7f2f78004ab0 2023-10-26T01:38:03.707 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.701+0000 7f2f77fff640 1 --2- 172.21.15.169:0/2147023019 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2f7c06ae40 0x7f2f7c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:03.708 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.701+0000 7f2f77fff640 1 -- 172.21.15.169:0/2147023019 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2f80033e40 con 0x7f2f78004ab0 2023-10-26T01:38:03.708 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.702+0000 7f2f8effd640 1 --2- 172.21.15.169:0/2147023019 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2f7c06ae40 0x7f2f7c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:03.708 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.703+0000 7f2f8effd640 1 --2- 172.21.15.169:0/2147023019 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2f7c06ae40 0x7f2f7c06d300 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f2f9013e070 tx=0x7f2f84009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:03.709 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.705+0000 7f2f77fff640 1 -- 172.21.15.169:0/2147023019 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2f800773b0 con 0x7f2f78004ab0 2023-10-26T01:38:03.897 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.894+0000 7f2f95c87640 1 -- 172.21.15.169:0/2147023019 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2f9004db10 con 0x7f2f78004ab0 2023-10-26T01:38:03.897 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.895+0000 7f2f77fff640 1 -- 172.21.15.169:0/2147023019 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v123) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2f8004a020 con 0x7f2f78004ab0 2023-10-26T01:38:03.898 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 64 2023-10-26T01:38:03.901 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.897+0000 7f2f95c87640 1 -- 172.21.15.169:0/2147023019 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2f7c06ae40 msgr2=0x7f2f7c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:03.901 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.897+0000 7f2f95c87640 1 --2- 172.21.15.169:0/2147023019 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2f7c06ae40 0x7f2f7c06d300 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f2f9013e070 tx=0x7f2f84009040 comp rx=0 tx=0).stop 2023-10-26T01:38:03.901 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.897+0000 7f2f95c87640 1 -- 172.21.15.169:0/2147023019 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f78004ab0 msgr2=0x7f2f901543c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:03.901 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.897+0000 7f2f95c87640 1 --2- 172.21.15.169:0/2147023019 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f78004ab0 0x7f2f901543c0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f2f80009a70 tx=0x7f2f80033c50 comp rx=0 tx=0).stop 2023-10-26T01:38:03.902 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.898+0000 7f2f95c87640 1 -- 172.21.15.169:0/2147023019 shutdown_connections 2023-10-26T01:38:03.902 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.898+0000 7f2f95c87640 1 --2- 172.21.15.169:0/2147023019 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2f7c06ae40 0x7f2f7c06d300 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:03.902 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.898+0000 7f2f95c87640 1 --2- 172.21.15.169:0/2147023019 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2f901f2410 0x7f2f900995a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:03.902 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.898+0000 7f2f95c87640 1 --2- 172.21.15.169:0/2147023019 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2f90098290 0x7f2f901f1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:03.903 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.898+0000 7f2f95c87640 1 --2- 172.21.15.169:0/2147023019 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f78004ab0 0x7f2f901543c0 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:03.903 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.898+0000 7f2f95c87640 1 -- 172.21.15.169:0/2147023019 >> 172.21.15.169:0/2147023019 conn(0x7f2f90069930 msgr2=0x7f2f900aed60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:03.903 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.898+0000 7f2f95c87640 1 -- 172.21.15.169:0/2147023019 shutdown_connections 2023-10-26T01:38:03.903 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:03.898+0000 7f2f95c87640 1 -- 172.21.15.169:0/2147023019 wait complete. 2023-10-26T01:38:03.918 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T01:38:04.133 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:04.130+0000 7f2c1fb8e640 1 -- 172.21.15.169:0/1824574632 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c1814e790 msgr2=0x7f2c180aab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:04.133 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:04.130+0000 7f2c1fb8e640 1 --2- 172.21.15.169:0/1824574632 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c1814e790 0x7f2c180aab40 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f2c18071f90 tx=0x7f2c1402be40 comp rx=0 tx=0).stop 2023-10-26T01:38:04.133 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:04.130+0000 7f2c1fb8e640 1 -- 172.21.15.169:0/1824574632 shutdown_connections 2023-10-26T01:38:04.134 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:04.130+0000 7f2c1fb8e640 1 --2- 172.21.15.169:0/1824574632 >> v2:172.21.15.170:3300/0 conn(0x7f2c180cde30 0x7f2c18159e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:04.134 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:04.130+0000 7f2c1fb8e640 1 --2- 172.21.15.169:0/1824574632 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c1814e790 0x7f2c180aab40 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:04.134 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:04.130+0000 7f2c1fb8e640 1 -- 172.21.15.169:0/1824574632 >> 172.21.15.169:0/1824574632 conn(0x7f2c18069930 msgr2=0x7f2c18069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:04.134 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:04.130+0000 7f2c1fb8e640 1 -- 172.21.15.169:0/1824574632 shutdown_connections 2023-10-26T01:38:04.135 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:04.130+0000 7f2c1fb8e640 1 -- 172.21.15.169:0/1824574632 wait complete. 2023-10-26T01:38:04.139 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:04.132+0000 7f2c1fb8e640 1 Processor -- start 2023-10-26T01:38:04.139 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:04.132+0000 7f2c1fb8e640 1 -- start start 2023-10-26T01:38:04.139 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:04.133+0000 7f2c1fb8e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2c180cde30 0x7f2c180715c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:04.139 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:04.133+0000 7f2c1fb8e640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2c1814e790 0x7f2c18096930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:04.139 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:04.133+0000 7f2c1fb8e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c18158850 0x7f2c1814b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:04.140 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:04.133+0000 7f2c1fb8e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c18146580 con 0x7f2c18158850 2023-10-26T01:38:04.140 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:04.133+0000 7f2c1fb8e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c180aeec0 con 0x7f2c180cde30 2023-10-26T01:38:04.140 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:04.133+0000 7f2c1fb8e640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c180af6d0 con 0x7f2c1814e790 2023-10-26T01:38:04.140 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:04.133+0000 7f2c1e12b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c18158850 0x7f2c1814b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:04.140 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:04.133+0000 7f2c1e12b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c18158850 0x7f2c1814b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:55778/0 (socket says 172.21.15.169:55778) 2023-10-26T01:38:04.141 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:04.133+0000 7f2c1e12b640 1 -- 172.21.15.169:0/1750828214 learned_addr learned my addr 172.21.15.169:0/1750828214 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:38:04.141 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:04.134+0000 7f2c1d92a640 1 --2- 172.21.15.169:0/1750828214 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2c180cde30 0x7f2c180715c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:04.141 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:04.134+0000 7f2c1d129640 1 --2- 172.21.15.169:0/1750828214 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2c1814e790 0x7f2c18096930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:04.141 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:04.134+0000 7f2c1e12b640 1 -- 172.21.15.169:0/1750828214 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2c1814e790 msgr2=0x7f2c18096930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:04.141 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:04.134+0000 7f2c1e12b640 1 --2- 172.21.15.169:0/1750828214 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2c1814e790 0x7f2c18096930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:04.142 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:04.134+0000 7f2c1e12b640 1 -- 172.21.15.169:0/1750828214 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2c180cde30 msgr2=0x7f2c180715c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:04.142 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:04.134+0000 7f2c1e12b640 1 --2- 172.21.15.169:0/1750828214 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2c180cde30 0x7f2c180715c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:04.142 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:04.134+0000 7f2c1e12b640 1 -- 172.21.15.169:0/1750828214 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c14009560 con 0x7f2c18158850 2023-10-26T01:38:04.142 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:04.134+0000 7f2c1d92a640 1 --2- 172.21.15.169:0/1750828214 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2c180cde30 0x7f2c180715c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T01:38:04.142 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:04.134+0000 7f2c1e12b640 1 --2- 172.21.15.169:0/1750828214 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c18158850 0x7f2c1814b650 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f2c1000eaa0 tx=0x7f2c1000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:04.143 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:04.134+0000 7f2c0e7fc640 1 -- 172.21.15.169:0/1750828214 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c1000cdb0 con 0x7f2c18158850 2023-10-26T01:38:04.143 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:04.134+0000 7f2c0e7fc640 1 -- 172.21.15.169:0/1750828214 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2c10004b20 con 0x7f2c18158850 2023-10-26T01:38:04.143 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:04.134+0000 7f2c1fb8e640 1 -- 172.21.15.169:0/1750828214 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c180b0b30 con 0x7f2c18158850 2023-10-26T01:38:04.143 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:04.135+0000 7f2c0e7fc640 1 -- 172.21.15.169:0/1750828214 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c10005020 con 0x7f2c18158850 2023-10-26T01:38:04.143 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:04.135+0000 7f2c1fb8e640 1 -- 172.21.15.169:0/1750828214 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c180af3f0 con 0x7f2c18158850 2023-10-26T01:38:04.144 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:04.136+0000 7f2c1fb8e640 1 -- 172.21.15.169:0/1750828214 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c1804db10 con 0x7f2c18158850 2023-10-26T01:38:04.144 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:04.137+0000 7f2c0e7fc640 1 -- 172.21.15.169:0/1750828214 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f2c10005180 con 0x7f2c18158850 2023-10-26T01:38:04.145 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:04.137+0000 7f2c0e7fc640 1 --2- 172.21.15.169:0/1750828214 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2bfc06ae40 0x7f2bfc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:04.145 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:04.137+0000 7f2c0e7fc640 1 -- 172.21.15.169:0/1750828214 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2c10089f90 con 0x7f2c18158850 2023-10-26T01:38:04.145 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:04.138+0000 7f2c1d92a640 1 --2- 172.21.15.169:0/1750828214 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2bfc06ae40 0x7f2bfc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:04.145 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:04.139+0000 7f2c1d92a640 1 --2- 172.21.15.169:0/1750828214 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2bfc06ae40 0x7f2bfc06d300 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f2c14035000 tx=0x7f2c14004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:04.146 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:04.142+0000 7f2c0e7fc640 1 -- 172.21.15.169:0/1750828214 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2c10055030 con 0x7f2c18158850 2023-10-26T01:38:04.371 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:04.368+0000 7f2c1fb8e640 1 -- 172.21.15.169:0/1750828214 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2c180aca70 con 0x7f2c18158850 2023-10-26T01:38:04.372 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:04.369+0000 7f2c0e7fc640 1 -- 172.21.15.169:0/1750828214 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) v1 ==== 94+0+4924 (secure 0 0 0) 0x7f2c1001f020 con 0x7f2c18158850 2023-10-26T01:38:04.372 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:38:04.372 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":59,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5387,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/1186563921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1186563921},{"type":"v1","addr":"172.21.15.170:6835","nonce":1186563921}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":54},{"gid":5390,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/3702036693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":3702036693},{"type":"v1","addr":"172.21.15.170:6837","nonce":3702036693}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":54},{"gid":7332,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/3193888686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":3193888686},{"type":"v1","addr":"172.21.15.169:6833","nonce":3193888686}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T01:37:40.027858+0000","modified":"2023-10-26T01:37:43.103474+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7335},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7335":{"gid":7335,"name":"c","rank":0,"incarnation":55,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6835/668588916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":668588916},{"type":"v1","addr":"172.21.15.169:6835","nonce":668588916}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-10-26T01:38:04.373 INFO:teuthology.orchestra.run.smithi169.stderr:dumped fsmap epoch 59 2023-10-26T01:38:04.374 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:04.371+0000 7f2c1fb8e640 1 -- 172.21.15.169:0/1750828214 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2bfc06ae40 msgr2=0x7f2bfc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:04.375 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:04.371+0000 7f2c1fb8e640 1 --2- 172.21.15.169:0/1750828214 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2bfc06ae40 0x7f2bfc06d300 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f2c14035000 tx=0x7f2c14004730 comp rx=0 tx=0).stop 2023-10-26T01:38:04.375 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:04.371+0000 7f2c1fb8e640 1 -- 172.21.15.169:0/1750828214 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c18158850 msgr2=0x7f2c1814b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:04.375 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:04.371+0000 7f2c1fb8e640 1 --2- 172.21.15.169:0/1750828214 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c18158850 0x7f2c1814b650 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f2c1000eaa0 tx=0x7f2c1000ef60 comp rx=0 tx=0).stop 2023-10-26T01:38:04.376 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:04.372+0000 7f2c1fb8e640 1 -- 172.21.15.169:0/1750828214 shutdown_connections 2023-10-26T01:38:04.376 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:04.372+0000 7f2c1fb8e640 1 --2- 172.21.15.169:0/1750828214 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2bfc06ae40 0x7f2bfc06d300 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:04.376 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:04.372+0000 7f2c1fb8e640 1 --2- 172.21.15.169:0/1750828214 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c18158850 0x7f2c1814b650 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:04.376 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:04.372+0000 7f2c1fb8e640 1 --2- 172.21.15.169:0/1750828214 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2c1814e790 0x7f2c18096930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:04.376 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:04.372+0000 7f2c1fb8e640 1 --2- 172.21.15.169:0/1750828214 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2c180cde30 0x7f2c180715c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:04.377 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:04.372+0000 7f2c1fb8e640 1 -- 172.21.15.169:0/1750828214 >> 172.21.15.169:0/1750828214 conn(0x7f2c18069930 msgr2=0x7f2c1814ed60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:04.377 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:04.372+0000 7f2c1fb8e640 1 -- 172.21.15.169:0/1750828214 shutdown_connections 2023-10-26T01:38:04.377 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:04.372+0000 7f2c1fb8e640 1 -- 172.21.15.169:0/1750828214 wait complete. 2023-10-26T01:38:04.394 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:38:04.608 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:04.605+0000 7f3c52173640 1 -- 172.21.15.169:0/804204164 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3c4c0b35c0 msgr2=0x7f3c4c164510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:04.609 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:04.605+0000 7f3c52173640 1 --2- 172.21.15.169:0/804204164 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3c4c0b35c0 0x7f3c4c164510 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f3c44005fd0 tx=0x7f3c4402c060 comp rx=0 tx=0).stop 2023-10-26T01:38:04.609 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:04.605+0000 7f3c52173640 1 -- 172.21.15.169:0/804204164 shutdown_connections 2023-10-26T01:38:04.609 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:04.605+0000 7f3c52173640 1 --2- 172.21.15.169:0/804204164 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3c4c0b35c0 0x7f3c4c164510 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:04.609 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:04.605+0000 7f3c52173640 1 -- 172.21.15.169:0/804204164 >> 172.21.15.169:0/804204164 conn(0x7f3c4c098a60 msgr2=0x7f3c4c0b4970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:04.609 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:04.605+0000 7f3c52173640 1 -- 172.21.15.169:0/804204164 shutdown_connections 2023-10-26T01:38:04.610 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:04.606+0000 7f3c52173640 1 -- 172.21.15.169:0/804204164 wait complete. 2023-10-26T01:38:04.610 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:04.607+0000 7f3c52173640 1 Processor -- start 2023-10-26T01:38:04.611 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:04.608+0000 7f3c52173640 1 -- start start 2023-10-26T01:38:04.611 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:04.609+0000 7f3c52173640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3c4c0b2a00 0x7f3c4c15a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:04.612 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:04.609+0000 7f3c52173640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c4c0b35c0 0x7f3c4c15e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:04.612 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:04.609+0000 7f3c52173640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3c4c142e60 0x7f3c4c0c8a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:04.612 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:04.609+0000 7f3c52173640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c4c06a620 con 0x7f3c4c0b2a00 2023-10-26T01:38:04.612 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:04.609+0000 7f3c52173640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c4c0af9f0 con 0x7f3c4c0b35c0 2023-10-26T01:38:04.612 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:04.609+0000 7f3c52173640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c4c0afcd0 con 0x7f3c4c142e60 2023-10-26T01:38:04.613 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:04.609+0000 7f3c4b7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3c4c0b2a00 0x7f3c4c15a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:04.613 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:04.609+0000 7f3c4b7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3c4c0b2a00 0x7f3c4c15a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:55792/0 (socket says 172.21.15.169:55792) 2023-10-26T01:38:04.613 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:04.609+0000 7f3c4b7fe640 1 -- 172.21.15.169:0/1490787490 learned_addr learned my addr 172.21.15.169:0/1490787490 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:38:04.614 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:04.609+0000 7f3c4b7fe640 1 -- 172.21.15.169:0/1490787490 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3c4c142e60 msgr2=0x7f3c4c0c8a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:04.615 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:04.609+0000 7f3c4b7fe640 1 --2- 172.21.15.169:0/1490787490 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3c4c142e60 0x7f3c4c0c8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:04.615 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:04.609+0000 7f3c4b7fe640 1 -- 172.21.15.169:0/1490787490 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c4c0b35c0 msgr2=0x7f3c4c15e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:04.615 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:04.609+0000 7f3c4b7fe640 1 --2- 172.21.15.169:0/1490787490 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c4c0b35c0 0x7f3c4c15e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:04.615 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:04.609+0000 7f3c4b7fe640 1 -- 172.21.15.169:0/1490787490 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c44009840 con 0x7f3c4c0b2a00 2023-10-26T01:38:04.616 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:04.610+0000 7f3c4b7fe640 1 --2- 172.21.15.169:0/1490787490 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3c4c0b2a00 0x7f3c4c15a080 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c00c410 tx=0x7f3c3c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:04.616 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:04.610+0000 7f3c33fff640 1 -- 172.21.15.169:0/1490787490 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c3c0035f0 con 0x7f3c4c0b2a00 2023-10-26T01:38:04.616 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:04.610+0000 7f3c33fff640 1 -- 172.21.15.169:0/1490787490 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3c3c003750 con 0x7f3c4c0b2a00 2023-10-26T01:38:04.616 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:04.610+0000 7f3c33fff640 1 -- 172.21.15.169:0/1490787490 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c3c003ba0 con 0x7f3c4c0b2a00 2023-10-26T01:38:04.616 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:04.610+0000 7f3c52173640 1 -- 172.21.15.169:0/1490787490 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c4c0affb0 con 0x7f3c4c0b2a00 2023-10-26T01:38:04.617 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:04.610+0000 7f3c52173640 1 -- 172.21.15.169:0/1490787490 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c4c15e720 con 0x7f3c4c0b2a00 2023-10-26T01:38:04.617 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:04.612+0000 7f3c52173640 1 -- 172.21.15.169:0/1490787490 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c4c0b04d0 con 0x7f3c4c0b2a00 2023-10-26T01:38:04.617 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:04.613+0000 7f3c33fff640 1 -- 172.21.15.169:0/1490787490 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f3c3c003d00 con 0x7f3c4c0b2a00 2023-10-26T01:38:04.618 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:04.613+0000 7f3c33fff640 1 --2- 172.21.15.169:0/1490787490 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3c3806ae40 0x7f3c3806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:04.618 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:04.614+0000 7f3c33fff640 1 -- 172.21.15.169:0/1490787490 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3c3c0889f0 con 0x7f3c4c0b2a00 2023-10-26T01:38:04.618 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:04.614+0000 7f3c4affd640 1 --2- 172.21.15.169:0/1490787490 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3c3806ae40 0x7f3c3806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:04.619 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:04.615+0000 7f3c4affd640 1 --2- 172.21.15.169:0/1490787490 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3c3806ae40 0x7f3c3806d300 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f3c40000f60 tx=0x7f3c40007a30 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:04.620 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:04.618+0000 7f3c33fff640 1 -- 172.21.15.169:0/1490787490 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3c3c053aa0 con 0x7f3c4c0b2a00 2023-10-26T01:38:04.814 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:04.811+0000 7f3c52173640 1 -- 172.21.15.169:0/1490787490 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3c4c143460 con 0x7f3c4c0b2a00 2023-10-26T01:38:04.816 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:04.813+0000 7f3c33fff640 1 -- 172.21.15.169:0/1490787490 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v123) v1 ==== 74+0+16575 (secure 0 0 0) 0x7f3c3c017090 con 0x7f3c4c0b2a00 2023-10-26T01:38:04.817 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:38:04.817 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":123,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:37:54.102485+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-10-26T01:37:37.959102+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-10-26T01:37:39.053240+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:38:04.819 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:04.816+0000 7f3c52173640 1 -- 172.21.15.169:0/1490787490 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3c3806ae40 msgr2=0x7f3c3806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:04.819 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:04.816+0000 7f3c52173640 1 --2- 172.21.15.169:0/1490787490 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3c3806ae40 0x7f3c3806d300 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f3c40000f60 tx=0x7f3c40007a30 comp rx=0 tx=0).stop 2023-10-26T01:38:04.819 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:04.816+0000 7f3c52173640 1 -- 172.21.15.169:0/1490787490 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3c4c0b2a00 msgr2=0x7f3c4c15a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:04.820 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:04.816+0000 7f3c52173640 1 --2- 172.21.15.169:0/1490787490 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3c4c0b2a00 0x7f3c4c15a080 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c00c410 tx=0x7f3c3c00c8d0 comp rx=0 tx=0).stop 2023-10-26T01:38:04.820 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:04.816+0000 7f3c52173640 1 -- 172.21.15.169:0/1490787490 shutdown_connections 2023-10-26T01:38:04.820 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:04.816+0000 7f3c52173640 1 --2- 172.21.15.169:0/1490787490 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3c3806ae40 0x7f3c3806d300 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:04.820 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:04.816+0000 7f3c52173640 1 --2- 172.21.15.169:0/1490787490 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3c4c142e60 0x7f3c4c0c8a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:04.820 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:04.816+0000 7f3c52173640 1 --2- 172.21.15.169:0/1490787490 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c4c0b35c0 0x7f3c4c15e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:04.821 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:04.816+0000 7f3c52173640 1 --2- 172.21.15.169:0/1490787490 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3c4c0b2a00 0x7f3c4c15a080 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:04.821 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:04.816+0000 7f3c52173640 1 -- 172.21.15.169:0/1490787490 >> 172.21.15.169:0/1490787490 conn(0x7f3c4c098a60 msgr2=0x7f3c4c141790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:04.821 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:04.817+0000 7f3c52173640 1 -- 172.21.15.169:0/1490787490 shutdown_connections 2023-10-26T01:38:04.821 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:04.817+0000 7f3c52173640 1 -- 172.21.15.169:0/1490787490 wait complete. 2023-10-26T01:38:04.839 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-26T01:38:04.839 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T01:38:05.051 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.048+0000 7f161c0e5640 1 -- 172.21.15.169:0/3780068490 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1614158850 msgr2=0x7f16140aab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:05.052 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.048+0000 7f161c0e5640 1 --2- 172.21.15.169:0/3780068490 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1614158850 0x7f16140aab40 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f16140accf0 tx=0x7f160402bd10 comp rx=0 tx=0).stop 2023-10-26T01:38:05.052 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.048+0000 7f161c0e5640 1 -- 172.21.15.169:0/3780068490 shutdown_connections 2023-10-26T01:38:05.052 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.048+0000 7f161c0e5640 1 --2- 172.21.15.169:0/3780068490 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f16140cd720 0x7f16140cdb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:05.053 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.048+0000 7f161c0e5640 1 --2- 172.21.15.169:0/3780068490 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1614158850 0x7f16140aab40 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:05.053 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.048+0000 7f161c0e5640 1 --2- 172.21.15.169:0/3780068490 >> v2:172.21.15.170:3300/0 conn(0x7f161406ce20 0x7f161414e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:05.053 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.048+0000 7f161c0e5640 1 -- 172.21.15.169:0/3780068490 >> 172.21.15.169:0/3780068490 conn(0x7f1614069930 msgr2=0x7f1614069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:05.053 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.048+0000 7f161c0e5640 1 -- 172.21.15.169:0/3780068490 shutdown_connections 2023-10-26T01:38:05.053 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.048+0000 7f161c0e5640 1 -- 172.21.15.169:0/3780068490 wait complete. 2023-10-26T01:38:05.055 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.050+0000 7f161c0e5640 1 Processor -- start 2023-10-26T01:38:05.056 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.050+0000 7f161c0e5640 1 -- start start 2023-10-26T01:38:05.056 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.051+0000 7f161c0e5640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f161406ce20 0x7f1614153350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:05.056 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.051+0000 7f161c0e5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f16140cd720 0x7f16141548a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:05.056 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.051+0000 7f161c0e5640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1614158850 0x7f1614159e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:05.057 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.051+0000 7f161c0e5640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16141f61e0 con 0x7f1614158850 2023-10-26T01:38:05.057 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.051+0000 7f161c0e5640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16141f5660 con 0x7f16140cd720 2023-10-26T01:38:05.057 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.051+0000 7f161c0e5640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16141f5940 con 0x7f161406ce20 2023-10-26T01:38:05.057 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.052+0000 7f161a682640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1614158850 0x7f1614159e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:05.057 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.052+0000 7f161a682640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1614158850 0x7f1614159e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:55810/0 (socket says 172.21.15.169:55810) 2023-10-26T01:38:05.057 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.052+0000 7f161a682640 1 -- 172.21.15.169:0/1561796624 learned_addr learned my addr 172.21.15.169:0/1561796624 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:38:05.058 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.052+0000 7f1619680640 1 --2- 172.21.15.169:0/1561796624 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f16140cd720 0x7f16141548a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:05.058 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.052+0000 7f1619e81640 1 --2- 172.21.15.169:0/1561796624 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f161406ce20 0x7f1614153350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:05.058 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.052+0000 7f161a682640 1 -- 172.21.15.169:0/1561796624 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f161406ce20 msgr2=0x7f1614153350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:05.058 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.052+0000 7f161a682640 1 --2- 172.21.15.169:0/1561796624 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f161406ce20 0x7f1614153350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:05.059 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.052+0000 7f161a682640 1 -- 172.21.15.169:0/1561796624 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f16140cd720 msgr2=0x7f16141548a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:05.059 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.052+0000 7f161a682640 1 --2- 172.21.15.169:0/1561796624 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f16140cd720 0x7f16141548a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:05.059 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.052+0000 7f161a682640 1 -- 172.21.15.169:0/1561796624 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1604009560 con 0x7f1614158850 2023-10-26T01:38:05.059 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.052+0000 7f1619e81640 1 --2- 172.21.15.169:0/1561796624 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f161406ce20 0x7f1614153350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:38:05.059 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.052+0000 7f161a682640 1 --2- 172.21.15.169:0/1561796624 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1614158850 0x7f1614159e10 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f160c00b970 tx=0x7f160c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:05.060 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.053+0000 7f160a7fc640 1 -- 172.21.15.169:0/1561796624 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f160c00cb40 con 0x7f1614158850 2023-10-26T01:38:05.060 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.053+0000 7f160a7fc640 1 -- 172.21.15.169:0/1561796624 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f160c00cdb0 con 0x7f1614158850 2023-10-26T01:38:05.060 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.053+0000 7f160a7fc640 1 -- 172.21.15.169:0/1561796624 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f160c0146a0 con 0x7f1614158850 2023-10-26T01:38:05.060 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.053+0000 7f161c0e5640 1 -- 172.21.15.169:0/1561796624 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16141f5c20 con 0x7f1614158850 2023-10-26T01:38:05.060 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.053+0000 7f161c0e5640 1 -- 172.21.15.169:0/1561796624 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16141f52f0 con 0x7f1614158850 2023-10-26T01:38:05.061 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.054+0000 7f161c0e5640 1 -- 172.21.15.169:0/1561796624 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f161404db10 con 0x7f1614158850 2023-10-26T01:38:05.062 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.055+0000 7f160a7fc640 1 -- 172.21.15.169:0/1561796624 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f160c01d3b0 con 0x7f1614158850 2023-10-26T01:38:05.062 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.056+0000 7f160a7fc640 1 --2- 172.21.15.169:0/1561796624 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f15f806ae40 0x7f15f806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:05.062 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.056+0000 7f160a7fc640 1 -- 172.21.15.169:0/1561796624 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f160c0897a0 con 0x7f1614158850 2023-10-26T01:38:05.062 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.056+0000 7f1619e81640 1 --2- 172.21.15.169:0/1561796624 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f15f806ae40 0x7f15f806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:05.062 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.057+0000 7f1619e81640 1 --2- 172.21.15.169:0/1561796624 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f15f806ae40 0x7f15f806d300 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f16100058c0 tx=0x7f1610005d30 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:05.063 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.060+0000 7f160a7fc640 1 -- 172.21.15.169:0/1561796624 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f160c054860 con 0x7f1614158850 2023-10-26T01:38:05.281 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.278+0000 7f161c0e5640 1 -- 172.21.15.169:0/1561796624 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f16140aca70 con 0x7f1614158850 2023-10-26T01:38:05.282 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.279+0000 7f160a7fc640 1 -- 172.21.15.169:0/1561796624 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) v1 ==== 94+0+4924 (secure 0 0 0) 0x7f160c057e80 con 0x7f1614158850 2023-10-26T01:38:05.283 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:38:05.283 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":59,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5387,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/1186563921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1186563921},{"type":"v1","addr":"172.21.15.170:6835","nonce":1186563921}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":54},{"gid":5390,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/3702036693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":3702036693},{"type":"v1","addr":"172.21.15.170:6837","nonce":3702036693}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":54},{"gid":7332,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/3193888686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":3193888686},{"type":"v1","addr":"172.21.15.169:6833","nonce":3193888686}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T01:37:40.027858+0000","modified":"2023-10-26T01:37:43.103474+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7335},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7335":{"gid":7335,"name":"c","rank":0,"incarnation":55,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6835/668588916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":668588916},{"type":"v1","addr":"172.21.15.169:6835","nonce":668588916}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-10-26T01:38:05.283 INFO:teuthology.orchestra.run.smithi169.stderr:dumped fsmap epoch 59 2023-10-26T01:38:05.285 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.282+0000 7f161c0e5640 1 -- 172.21.15.169:0/1561796624 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f15f806ae40 msgr2=0x7f15f806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:05.285 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.282+0000 7f161c0e5640 1 --2- 172.21.15.169:0/1561796624 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f15f806ae40 0x7f15f806d300 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f16100058c0 tx=0x7f1610005d30 comp rx=0 tx=0).stop 2023-10-26T01:38:05.285 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.282+0000 7f161c0e5640 1 -- 172.21.15.169:0/1561796624 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1614158850 msgr2=0x7f1614159e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:05.285 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.282+0000 7f161c0e5640 1 --2- 172.21.15.169:0/1561796624 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1614158850 0x7f1614159e10 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f160c00b970 tx=0x7f160c00be30 comp rx=0 tx=0).stop 2023-10-26T01:38:05.286 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.282+0000 7f161c0e5640 1 -- 172.21.15.169:0/1561796624 shutdown_connections 2023-10-26T01:38:05.286 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.282+0000 7f161c0e5640 1 --2- 172.21.15.169:0/1561796624 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f15f806ae40 0x7f15f806d300 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:05.286 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.282+0000 7f161c0e5640 1 --2- 172.21.15.169:0/1561796624 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1614158850 0x7f1614159e10 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:05.286 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.282+0000 7f161c0e5640 1 --2- 172.21.15.169:0/1561796624 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f16140cd720 0x7f16141548a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:05.286 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.282+0000 7f161c0e5640 1 --2- 172.21.15.169:0/1561796624 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f161406ce20 0x7f1614153350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:05.287 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.282+0000 7f161c0e5640 1 -- 172.21.15.169:0/1561796624 >> 172.21.15.169:0/1561796624 conn(0x7f1614069930 msgr2=0x7f161414fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:05.287 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.282+0000 7f161c0e5640 1 -- 172.21.15.169:0/1561796624 shutdown_connections 2023-10-26T01:38:05.287 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.282+0000 7f161c0e5640 1 -- 172.21.15.169:0/1561796624 wait complete. 2023-10-26T01:38:05.301 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T01:38:05.515 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.511+0000 7f5ac1fbd640 1 -- 172.21.15.169:0/1981710053 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5aa4003fa0 msgr2=0x7f5aa40047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:05.515 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.511+0000 7f5ac1fbd640 1 --2- 172.21.15.169:0/1981710053 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5aa4003fa0 0x7f5aa40047c0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f5aac00bbb0 tx=0x7f5aa40057f0 comp rx=0 tx=0).stop 2023-10-26T01:38:05.515 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.511+0000 7f5ac17bc640 1 -- 172.21.15.169:0/1981710053 reap_dead start 2023-10-26T01:38:05.515 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.512+0000 7f5ac1fbd640 1 -- 172.21.15.169:0/1981710053 shutdown_connections 2023-10-26T01:38:05.516 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.512+0000 7f5ac1fbd640 1 -- 172.21.15.169:0/1981710053 >> 172.21.15.169:0/1981710053 conn(0x7f5abc065f10 msgr2=0x7f5abc0cf1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:05.516 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.512+0000 7f5ac1fbd640 1 -- 172.21.15.169:0/1981710053 shutdown_connections 2023-10-26T01:38:05.516 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.512+0000 7f5ac1fbd640 1 -- 172.21.15.169:0/1981710053 wait complete. 2023-10-26T01:38:05.517 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.513+0000 7f5ac1fbd640 1 Processor -- start 2023-10-26T01:38:05.517 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.514+0000 7f5ac1fbd640 1 -- start start 2023-10-26T01:38:05.518 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.515+0000 7f5ac1fbd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5aa4003fa0 0x7f5abc1583b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:05.519 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.515+0000 7f5ac1fbd640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5abc0c8ef0 0x7f5abc0b37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:05.519 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.515+0000 7f5ac1fbd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5abc15de20 0x7f5abc15e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:05.519 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.515+0000 7f5ac1fbd640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5abc15e7e0 con 0x7f5abc15de20 2023-10-26T01:38:05.519 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.515+0000 7f5ac1fbd640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5abc1588f0 con 0x7f5aa4003fa0 2023-10-26T01:38:05.520 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.515+0000 7f5ac1fbd640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5abc0b3d10 con 0x7f5abc0c8ef0 2023-10-26T01:38:05.520 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.515+0000 7f5ac17bc640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5abc15de20 0x7f5abc15e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:05.520 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.515+0000 7f5ac17bc640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5abc15de20 0x7f5abc15e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:55834/0 (socket says 172.21.15.169:55834) 2023-10-26T01:38:05.520 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.515+0000 7f5ac17bc640 1 -- 172.21.15.169:0/2826463789 learned_addr learned my addr 172.21.15.169:0/2826463789 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:38:05.520 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.515+0000 7f5abbfff640 1 --2- 172.21.15.169:0/2826463789 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5abc0c8ef0 0x7f5abc0b37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:05.521 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.515+0000 7f5ac17bc640 1 -- 172.21.15.169:0/2826463789 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5abc0c8ef0 msgr2=0x7f5abc0b37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:05.521 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.515+0000 7f5ac0fbb640 1 --2- 172.21.15.169:0/2826463789 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5aa4003fa0 0x7f5abc1583b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:05.521 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.515+0000 7f5ac17bc640 1 --2- 172.21.15.169:0/2826463789 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5abc0c8ef0 0x7f5abc0b37d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:05.521 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.515+0000 7f5ac17bc640 1 -- 172.21.15.169:0/2826463789 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5aa4003fa0 msgr2=0x7f5abc1583b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:05.521 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.515+0000 7f5ac17bc640 1 --2- 172.21.15.169:0/2826463789 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5aa4003fa0 0x7f5abc1583b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:05.522 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.515+0000 7f5ac17bc640 1 -- 172.21.15.169:0/2826463789 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5abc0b3fb0 con 0x7f5abc15de20 2023-10-26T01:38:05.522 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.516+0000 7f5ac17bc640 1 --2- 172.21.15.169:0/2826463789 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5abc15de20 0x7f5abc15e2a0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f5ab4001330 tx=0x7f5ab400b620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:05.522 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.516+0000 7f5ab97fa640 1 -- 172.21.15.169:0/2826463789 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab4015030 con 0x7f5abc15de20 2023-10-26T01:38:05.522 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.516+0000 7f5ab97fa640 1 -- 172.21.15.169:0/2826463789 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5ab4003970 con 0x7f5abc15de20 2023-10-26T01:38:05.522 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.516+0000 7f5ab97fa640 1 -- 172.21.15.169:0/2826463789 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab4003d80 con 0x7f5abc15de20 2023-10-26T01:38:05.523 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.516+0000 7f5ac1fbd640 1 -- 172.21.15.169:0/2826463789 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5abc0b4290 con 0x7f5abc15de20 2023-10-26T01:38:05.523 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.516+0000 7f5ac1fbd640 1 -- 172.21.15.169:0/2826463789 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5abc0b4850 con 0x7f5abc15de20 2023-10-26T01:38:05.524 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.517+0000 7f5ac1fbd640 1 -- 172.21.15.169:0/2826463789 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5abc073c10 con 0x7f5abc15de20 2023-10-26T01:38:05.524 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.518+0000 7f5ab97fa640 1 -- 172.21.15.169:0/2826463789 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f5ab4018070 con 0x7f5abc15de20 2023-10-26T01:38:05.524 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.519+0000 7f5ab97fa640 1 --2- 172.21.15.169:0/2826463789 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5aa806ae70 0x7f5aa806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:05.524 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.519+0000 7f5ab97fa640 1 -- 172.21.15.169:0/2826463789 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5ab4016070 con 0x7f5abc15de20 2023-10-26T01:38:05.525 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.520+0000 7f5ac0fbb640 1 --2- 172.21.15.169:0/2826463789 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5aa806ae70 0x7f5aa806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:05.525 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.521+0000 7f5ac0fbb640 1 --2- 172.21.15.169:0/2826463789 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5aa806ae70 0x7f5aa806d330 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f5ab0005fd0 tx=0x7f5ab0005a90 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:05.526 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.523+0000 7f5ab97fa640 1 -- 172.21.15.169:0/2826463789 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5ab40549e0 con 0x7f5abc15de20 2023-10-26T01:38:05.754 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.751+0000 7f5ac1fbd640 1 -- 172.21.15.169:0/2826463789 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5abc067aa0 con 0x7f5abc15de20 2023-10-26T01:38:05.755 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.752+0000 7f5ab97fa640 1 -- 172.21.15.169:0/2826463789 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) v1 ==== 94+0+4924 (secure 0 0 0) 0x7f5ab4058000 con 0x7f5abc15de20 2023-10-26T01:38:05.755 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:38:05.755 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":59,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5387,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/1186563921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1186563921},{"type":"v1","addr":"172.21.15.170:6835","nonce":1186563921}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":54},{"gid":5390,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/3702036693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":3702036693},{"type":"v1","addr":"172.21.15.170:6837","nonce":3702036693}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":54},{"gid":7332,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/3193888686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":3193888686},{"type":"v1","addr":"172.21.15.169:6833","nonce":3193888686}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T01:37:40.027858+0000","modified":"2023-10-26T01:37:43.103474+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7335},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7335":{"gid":7335,"name":"c","rank":0,"incarnation":55,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6835/668588916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":668588916},{"type":"v1","addr":"172.21.15.169:6835","nonce":668588916}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-10-26T01:38:05.756 INFO:teuthology.orchestra.run.smithi169.stderr:dumped fsmap epoch 59 2023-10-26T01:38:05.757 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.754+0000 7f5ac1fbd640 1 -- 172.21.15.169:0/2826463789 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5aa806ae70 msgr2=0x7f5aa806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:05.758 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.754+0000 7f5ac1fbd640 1 --2- 172.21.15.169:0/2826463789 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5aa806ae70 0x7f5aa806d330 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f5ab0005fd0 tx=0x7f5ab0005a90 comp rx=0 tx=0).stop 2023-10-26T01:38:05.758 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.755+0000 7f5ac1fbd640 1 -- 172.21.15.169:0/2826463789 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5abc15de20 msgr2=0x7f5abc15e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:05.758 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.755+0000 7f5ac1fbd640 1 --2- 172.21.15.169:0/2826463789 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5abc15de20 0x7f5abc15e2a0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f5ab4001330 tx=0x7f5ab400b620 comp rx=0 tx=0).stop 2023-10-26T01:38:05.758 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.755+0000 7f5ac1fbd640 1 -- 172.21.15.169:0/2826463789 shutdown_connections 2023-10-26T01:38:05.758 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.755+0000 7f5ac1fbd640 1 --2- 172.21.15.169:0/2826463789 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5aa806ae70 0x7f5aa806d330 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:05.759 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.755+0000 7f5ac1fbd640 1 --2- 172.21.15.169:0/2826463789 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5abc15de20 0x7f5abc15e2a0 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:05.759 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.755+0000 7f5ac1fbd640 1 --2- 172.21.15.169:0/2826463789 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5abc0c8ef0 0x7f5abc0b37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:05.759 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.755+0000 7f5ac1fbd640 1 --2- 172.21.15.169:0/2826463789 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5aa4003fa0 0x7f5abc1583b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:05.759 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.755+0000 7f5ac1fbd640 1 -- 172.21.15.169:0/2826463789 >> 172.21.15.169:0/2826463789 conn(0x7f5abc065f10 msgr2=0x7f5abc15af20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:05.759 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.755+0000 7f5ac1fbd640 1 -- 172.21.15.169:0/2826463789 shutdown_connections 2023-10-26T01:38:05.760 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.755+0000 7f5ac1fbd640 1 -- 172.21.15.169:0/2826463789 wait complete. 2023-10-26T01:38:05.776 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:38:05.998 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.995+0000 7f320b6bb640 1 -- 172.21.15.169:0/1976027667 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f32040b1490 msgr2=0x7f3204069510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:05.998 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.995+0000 7f320b6bb640 1 --2- 172.21.15.169:0/1976027667 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f32040b1490 0x7f3204069510 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f31f0005fd0 tx=0x7f31f002bd90 comp rx=0 tx=0).stop 2023-10-26T01:38:05.999 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.995+0000 7f320b6bb640 1 -- 172.21.15.169:0/1976027667 shutdown_connections 2023-10-26T01:38:05.999 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.995+0000 7f320b6bb640 1 --2- 172.21.15.169:0/1976027667 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f32040b1490 0x7f3204069510 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:05.999 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.995+0000 7f320b6bb640 1 --2- 172.21.15.169:0/1976027667 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3204144f30 0x7f32040ca680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:05.999 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.995+0000 7f320b6bb640 1 -- 172.21.15.169:0/1976027667 >> 172.21.15.169:0/1976027667 conn(0x7f3204065670 msgr2=0x7f320406a620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:06.000 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.995+0000 7f320b6bb640 1 -- 172.21.15.169:0/1976027667 shutdown_connections 2023-10-26T01:38:06.000 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.995+0000 7f320b6bb640 1 -- 172.21.15.169:0/1976027667 wait complete. 2023-10-26T01:38:06.000 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.997+0000 7f320b6bb640 1 Processor -- start 2023-10-26T01:38:06.001 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.997+0000 7f320b6bb640 1 -- start start 2023-10-26T01:38:06.002 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.998+0000 7f320b6bb640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3204069a50 0x7f320413fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:06.002 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.998+0000 7f320b6bb640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32040b1490 0x7f320413b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:06.002 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.998+0000 7f320b6bb640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3204144f30 0x7f32041413f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:06.003 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.998+0000 7f320b6bb640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3204144810 con 0x7f32040b1490 2023-10-26T01:38:06.003 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.998+0000 7f320b6bb640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32041403d0 con 0x7f3204069a50 2023-10-26T01:38:06.003 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.998+0000 7f320b6bb640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32041406a0 con 0x7f3204144f30 2023-10-26T01:38:06.003 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.999+0000 7f3208c56640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32040b1490 0x7f320413b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:06.003 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.999+0000 7f3208c56640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32040b1490 0x7f320413b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:55850/0 (socket says 172.21.15.169:55850) 2023-10-26T01:38:06.003 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.999+0000 7f3208c56640 1 -- 172.21.15.169:0/4015647304 learned_addr learned my addr 172.21.15.169:0/4015647304 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:38:06.004 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.999+0000 7f3209457640 1 --2- 172.21.15.169:0/4015647304 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3204069a50 0x7f320413fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:06.004 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.999+0000 7f3209c58640 1 --2- 172.21.15.169:0/4015647304 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3204144f30 0x7f32041413f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:06.004 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.999+0000 7f3208c56640 1 -- 172.21.15.169:0/4015647304 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3204144f30 msgr2=0x7f32041413f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:06.004 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.999+0000 7f3208c56640 1 --2- 172.21.15.169:0/4015647304 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3204144f30 0x7f32041413f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:06.004 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.999+0000 7f3208c56640 1 -- 172.21.15.169:0/4015647304 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3204069a50 msgr2=0x7f320413fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:06.005 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.999+0000 7f3208c56640 1 --2- 172.21.15.169:0/4015647304 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3204069a50 0x7f320413fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:06.005 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:05.999+0000 7f3208c56640 1 -- 172.21.15.169:0/4015647304 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31f0009560 con 0x7f32040b1490 2023-10-26T01:38:06.006 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.000+0000 7f3208c56640 1 --2- 172.21.15.169:0/4015647304 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32040b1490 0x7f320413b270 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f31f0036040 tx=0x7f31f002da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:06.006 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.000+0000 7f3201ffb640 1 -- 172.21.15.169:0/4015647304 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f002de60 con 0x7f32040b1490 2023-10-26T01:38:06.007 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.000+0000 7f3201ffb640 1 -- 172.21.15.169:0/4015647304 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f31f00036e0 con 0x7f32040b1490 2023-10-26T01:38:06.007 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.000+0000 7f320b6bb640 1 -- 172.21.15.169:0/4015647304 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f320413c700 con 0x7f32040b1490 2023-10-26T01:38:06.007 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.000+0000 7f3201ffb640 1 -- 172.21.15.169:0/4015647304 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f0003ab0 con 0x7f32040b1490 2023-10-26T01:38:06.007 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.000+0000 7f320b6bb640 1 -- 172.21.15.169:0/4015647304 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f320413ccc0 con 0x7f32040b1490 2023-10-26T01:38:06.007 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.001+0000 7f320b6bb640 1 -- 172.21.15.169:0/4015647304 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3204066af0 con 0x7f32040b1490 2023-10-26T01:38:06.008 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.002+0000 7f3201ffb640 1 -- 172.21.15.169:0/4015647304 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f31f0003c10 con 0x7f32040b1490 2023-10-26T01:38:06.008 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.003+0000 7f3201ffb640 1 --2- 172.21.15.169:0/4015647304 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f31e806ae70 0x7f31e806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:06.008 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.003+0000 7f3201ffb640 1 -- 172.21.15.169:0/4015647304 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f31f00ac0e0 con 0x7f32040b1490 2023-10-26T01:38:06.008 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.004+0000 7f3209457640 1 --2- 172.21.15.169:0/4015647304 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f31e806ae70 0x7f31e806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:06.009 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.005+0000 7f3209457640 1 --2- 172.21.15.169:0/4015647304 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f31e806ae70 0x7f31e806d330 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f31f8005fd0 tx=0x7f31f8009500 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:06.010 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.008+0000 7f3201ffb640 1 -- 172.21.15.169:0/4015647304 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f31f0077210 con 0x7f32040b1490 2023-10-26T01:38:06.201 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.198+0000 7f320b6bb640 1 -- 172.21.15.169:0/4015647304 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f320404db10 con 0x7f32040b1490 2023-10-26T01:38:06.209 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.206+0000 7f3201ffb640 1 -- 172.21.15.169:0/4015647304 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v123) v1 ==== 74+0+16575 (secure 0 0 0) 0x7f31f007a830 con 0x7f32040b1490 2023-10-26T01:38:06.210 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:38:06.210 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":123,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:37:54.102485+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-10-26T01:37:37.959102+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-10-26T01:37:39.053240+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:38:06.212 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.209+0000 7f320b6bb640 1 -- 172.21.15.169:0/4015647304 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f31e806ae70 msgr2=0x7f31e806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:06.212 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.209+0000 7f320b6bb640 1 --2- 172.21.15.169:0/4015647304 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f31e806ae70 0x7f31e806d330 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f31f8005fd0 tx=0x7f31f8009500 comp rx=0 tx=0).stop 2023-10-26T01:38:06.212 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.209+0000 7f320b6bb640 1 -- 172.21.15.169:0/4015647304 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32040b1490 msgr2=0x7f320413b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:06.212 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.209+0000 7f320b6bb640 1 --2- 172.21.15.169:0/4015647304 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32040b1490 0x7f320413b270 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f31f0036040 tx=0x7f31f002da60 comp rx=0 tx=0).stop 2023-10-26T01:38:06.213 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.210+0000 7f320b6bb640 1 -- 172.21.15.169:0/4015647304 shutdown_connections 2023-10-26T01:38:06.213 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.210+0000 7f320b6bb640 1 --2- 172.21.15.169:0/4015647304 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f31e806ae70 0x7f31e806d330 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:06.214 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.210+0000 7f320b6bb640 1 --2- 172.21.15.169:0/4015647304 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32040b1490 0x7f320413b270 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:06.214 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.210+0000 7f320b6bb640 1 --2- 172.21.15.169:0/4015647304 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3204144f30 0x7f32041413f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:06.214 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.210+0000 7f320b6bb640 1 --2- 172.21.15.169:0/4015647304 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3204069a50 0x7f320413fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:06.214 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.210+0000 7f320b6bb640 1 -- 172.21.15.169:0/4015647304 >> 172.21.15.169:0/4015647304 conn(0x7f3204065670 msgr2=0x7f320415bb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:06.215 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.210+0000 7f320b6bb640 1 -- 172.21.15.169:0/4015647304 shutdown_connections 2023-10-26T01:38:06.215 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.210+0000 7f320b6bb640 1 -- 172.21.15.169:0/4015647304 wait complete. 2023-10-26T01:38:06.231 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-26T01:38:06.351 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T01:38:06.360 DEBUG:teuthology.orchestra.run.smithi170:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T01:38:06.441 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.438+0000 7f480dd6c640 1 -- 172.21.15.169:0/249841970 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f47f80047c0 msgr2=0x7f47f8004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:06.441 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.438+0000 7f480dd6c640 1 --2- 172.21.15.169:0/249841970 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f47f80047c0 0x7f47f8004c40 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f47f4001790 tx=0x7f47f80057f0 comp rx=0 tx=0).stop 2023-10-26T01:38:06.442 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.438+0000 7f480dd6c640 1 -- 172.21.15.169:0/249841970 shutdown_connections 2023-10-26T01:38:06.442 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.438+0000 7f480dd6c640 1 --2- 172.21.15.169:0/249841970 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f47f80047c0 0x7f47f8004c40 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:06.442 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.438+0000 7f480dd6c640 1 -- 172.21.15.169:0/249841970 >> 172.21.15.169:0/249841970 conn(0x7f48080d1370 msgr2=0x7f48080d1780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:06.442 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.438+0000 7f480dd6c640 1 -- 172.21.15.169:0/249841970 shutdown_connections 2023-10-26T01:38:06.442 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.438+0000 7f480dd6c640 1 -- 172.21.15.169:0/249841970 wait complete. 2023-10-26T01:38:06.443 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.440+0000 7f480dd6c640 1 Processor -- start 2023-10-26T01:38:06.443 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.440+0000 7f480dd6c640 1 -- start start 2023-10-26T01:38:06.445 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.441+0000 7f480dd6c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f47f80047c0 0x7f480814c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:06.445 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.441+0000 7f480dd6c640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f48080c8e50 0x7f48080c92d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:06.445 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.441+0000 7f480dd6c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48081f5070 0x7f48081f54d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:06.445 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.441+0000 7f480dd6c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48080c9b70 con 0x7f47f80047c0 2023-10-26T01:38:06.446 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.441+0000 7f480dd6c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48080ca9d0 con 0x7f48081f5070 2023-10-26T01:38:06.446 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.441+0000 7f480dd6c640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48080c9810 con 0x7f48080c8e50 2023-10-26T01:38:06.446 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.441+0000 7f480cd6a640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f47f80047c0 0x7f480814c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:06.446 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.441+0000 7f480cd6a640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f47f80047c0 0x7f480814c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:55860/0 (socket says 172.21.15.169:55860) 2023-10-26T01:38:06.446 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.441+0000 7f480cd6a640 1 -- 172.21.15.169:0/895119811 learned_addr learned my addr 172.21.15.169:0/895119811 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:38:06.447 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.442+0000 7f480d56b640 1 --2- 172.21.15.169:0/895119811 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48081f5070 0x7f48081f54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:06.447 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.442+0000 7f4807fff640 1 --2- 172.21.15.169:0/895119811 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f48080c8e50 0x7f48080c92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:06.447 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.442+0000 7f480cd6a640 1 -- 172.21.15.169:0/895119811 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f48080c8e50 msgr2=0x7f48080c92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:06.447 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.442+0000 7f480cd6a640 1 --2- 172.21.15.169:0/895119811 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f48080c8e50 0x7f48080c92d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:06.447 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.442+0000 7f480cd6a640 1 -- 172.21.15.169:0/895119811 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48081f5070 msgr2=0x7f48081f54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:06.448 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.442+0000 7f480cd6a640 1 --2- 172.21.15.169:0/895119811 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48081f5070 0x7f48081f54d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:06.448 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.442+0000 7f480cd6a640 1 -- 172.21.15.169:0/895119811 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48080c9d50 con 0x7f47f80047c0 2023-10-26T01:38:06.448 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.442+0000 7f480cd6a640 1 --2- 172.21.15.169:0/895119811 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f47f80047c0 0x7f480814c840 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f47fc00e5c0 tx=0x7f47fc00ea80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:06.449 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.443+0000 7f48057fa640 1 -- 172.21.15.169:0/895119811 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47fc010890 con 0x7f47f80047c0 2023-10-26T01:38:06.449 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.443+0000 7f48057fa640 1 -- 172.21.15.169:0/895119811 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f47fc010ae0 con 0x7f47f80047c0 2023-10-26T01:38:06.450 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.443+0000 7f48057fa640 1 -- 172.21.15.169:0/895119811 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47fc011460 con 0x7f47f80047c0 2023-10-26T01:38:06.450 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.443+0000 7f480dd6c640 1 -- 172.21.15.169:0/895119811 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48080ca030 con 0x7f47f80047c0 2023-10-26T01:38:06.450 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.443+0000 7f480dd6c640 1 -- 172.21.15.169:0/895119811 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f480814cd80 con 0x7f47f80047c0 2023-10-26T01:38:06.450 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.444+0000 7f480dd6c640 1 -- 172.21.15.169:0/895119811 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f480814cf70 con 0x7f47f80047c0 2023-10-26T01:38:06.451 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.445+0000 7f48057fa640 1 -- 172.21.15.169:0/895119811 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f47fc0115c0 con 0x7f47f80047c0 2023-10-26T01:38:06.451 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.446+0000 7f48057fa640 1 --2- 172.21.15.169:0/895119811 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f47ec06ae40 0x7f47ec06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:06.451 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.446+0000 7f48057fa640 1 -- 172.21.15.169:0/895119811 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f47fc089cd0 con 0x7f47f80047c0 2023-10-26T01:38:06.451 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.446+0000 7f4807fff640 1 --2- 172.21.15.169:0/895119811 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f47ec06ae40 0x7f47ec06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:06.451 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.447+0000 7f4807fff640 1 --2- 172.21.15.169:0/895119811 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f47ec06ae40 0x7f47ec06d300 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f47f4034000 tx=0x7f47f402e070 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:06.452 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.450+0000 7f48057fa640 1 -- 172.21.15.169:0/895119811 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f47fc054e20 con 0x7f47f80047c0 2023-10-26T01:38:06.684 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.681+0000 7f480dd6c640 1 -- 172.21.15.169:0/895119811 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f4808073c10 con 0x7f47f80047c0 2023-10-26T01:38:06.685 INFO:tasks.ceph.mon.a.smithi169.stderr:2023-10-26T01:38:06.683+0000 7f1588a74640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-26T01:38:06.689 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.686+0000 7f48057fa640 1 -- 172.21.15.169:0/895119811 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v60) v1 ==== 161+0+0 (secure 0 0 0) 0x7f47fc01f020 con 0x7f47f80047c0 2023-10-26T01:38:06.689 INFO:teuthology.orchestra.run.smithi169.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-26T01:38:06.694 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.691+0000 7f480dd6c640 1 -- 172.21.15.169:0/895119811 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f47ec06ae40 msgr2=0x7f47ec06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:06.694 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.691+0000 7f480dd6c640 1 --2- 172.21.15.169:0/895119811 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f47ec06ae40 0x7f47ec06d300 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f47f4034000 tx=0x7f47f402e070 comp rx=0 tx=0).stop 2023-10-26T01:38:06.694 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.691+0000 7f480dd6c640 1 -- 172.21.15.169:0/895119811 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f47f80047c0 msgr2=0x7f480814c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:06.695 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.691+0000 7f480dd6c640 1 --2- 172.21.15.169:0/895119811 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f47f80047c0 0x7f480814c840 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f47fc00e5c0 tx=0x7f47fc00ea80 comp rx=0 tx=0).stop 2023-10-26T01:38:06.695 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.692+0000 7f480dd6c640 1 -- 172.21.15.169:0/895119811 shutdown_connections 2023-10-26T01:38:06.695 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.692+0000 7f480dd6c640 1 --2- 172.21.15.169:0/895119811 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f47ec06ae40 0x7f47ec06d300 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:06.695 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.692+0000 7f480dd6c640 1 --2- 172.21.15.169:0/895119811 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48081f5070 0x7f48081f54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:06.696 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.692+0000 7f480dd6c640 1 --2- 172.21.15.169:0/895119811 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f48080c8e50 0x7f48080c92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:06.696 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.692+0000 7f480dd6c640 1 --2- 172.21.15.169:0/895119811 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f47f80047c0 0x7f480814c840 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:06.696 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.693+0000 7f480dd6c640 1 -- 172.21.15.169:0/895119811 >> 172.21.15.169:0/895119811 conn(0x7f48080d1370 msgr2=0x7f480815c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:06.697 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.693+0000 7f480dd6c640 1 -- 172.21.15.169:0/895119811 shutdown_connections 2023-10-26T01:38:06.697 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.693+0000 7f480dd6c640 1 -- 172.21.15.169:0/895119811 wait complete. 2023-10-26T01:38:06.703 INFO:tasks.ceph.mds.c.smithi169.stderr: -4810> 2023-10-26T01:37:46.542+0000 7f33bf673640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-26T01:38:06.716 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-26T01:38:06.764 INFO:tasks.ceph.mds.c.smithi169.stdout:starting mds.c at 2023-10-26T01:38:06.910 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.907+0000 7ff600c4f640 1 -- 172.21.15.169:0/2769414061 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff5fc142e10 msgr2=0x7ff5fc159c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:06.910 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.907+0000 7ff600c4f640 1 --2- 172.21.15.169:0/2769414061 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff5fc142e10 0x7ff5fc159c90 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc073630 tx=0x7ff5ec02bc50 comp rx=0 tx=0).stop 2023-10-26T01:38:06.911 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.907+0000 7ff600c4f640 1 -- 172.21.15.169:0/2769414061 shutdown_connections 2023-10-26T01:38:06.911 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.907+0000 7ff600c4f640 1 --2- 172.21.15.169:0/2769414061 >> v2:172.21.15.170:3300/0 conn(0x7ff5fc1431f0 0x7ff5fc0cd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:06.911 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.907+0000 7ff600c4f640 1 --2- 172.21.15.169:0/2769414061 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff5fc142e10 0x7ff5fc159c90 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:06.911 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.907+0000 7ff600c4f640 1 -- 172.21.15.169:0/2769414061 >> 172.21.15.169:0/2769414061 conn(0x7ff5fc06a620 msgr2=0x7ff5fc06aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:06.911 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.907+0000 7ff600c4f640 1 -- 172.21.15.169:0/2769414061 shutdown_connections 2023-10-26T01:38:06.912 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.907+0000 7ff600c4f640 1 -- 172.21.15.169:0/2769414061 wait complete. 2023-10-26T01:38:06.913 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.909+0000 7ff600c4f640 1 Processor -- start 2023-10-26T01:38:06.913 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.909+0000 7ff600c4f640 1 -- start start 2023-10-26T01:38:06.913 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.910+0000 7ff600c4f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff5fc142e10 0x7ff5fc157060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:06.913 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.910+0000 7ff600c4f640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff5fc1431f0 0x7ff5fc12f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:06.913 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.910+0000 7ff600c4f640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff5fc15a1d0 0x7ff5fc13f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:06.914 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.910+0000 7ff600c4f640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5fc13eb80 con 0x7ff5fc15a1d0 2023-10-26T01:38:06.914 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.910+0000 7ff600c4f640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5fc13df70 con 0x7ff5fc142e10 2023-10-26T01:38:06.914 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.910+0000 7ff600c4f640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5fc13e250 con 0x7ff5fc1431f0 2023-10-26T01:38:06.914 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.910+0000 7ff5fad9d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff5fc15a1d0 0x7ff5fc13f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:06.914 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.910+0000 7ff5fad9d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff5fc15a1d0 0x7ff5fc13f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:55906/0 (socket says 172.21.15.169:55906) 2023-10-26T01:38:06.915 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.910+0000 7ff5fad9d640 1 -- 172.21.15.169:0/3310779543 learned_addr learned my addr 172.21.15.169:0/3310779543 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:38:06.916 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.911+0000 7ff5fa59c640 1 --2- 172.21.15.169:0/3310779543 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff5fc142e10 0x7ff5fc157060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:06.916 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.911+0000 7ff5fad9d640 1 -- 172.21.15.169:0/3310779543 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff5fc1431f0 msgr2=0x7ff5fc12f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:06.916 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.911+0000 7ff5f3fff640 1 --2- 172.21.15.169:0/3310779543 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff5fc1431f0 0x7ff5fc12f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:06.917 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.911+0000 7ff5fad9d640 1 --2- 172.21.15.169:0/3310779543 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff5fc1431f0 0x7ff5fc12f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:06.917 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.911+0000 7ff5fad9d640 1 -- 172.21.15.169:0/3310779543 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff5fc142e10 msgr2=0x7ff5fc157060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:06.917 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.911+0000 7ff5fad9d640 1 --2- 172.21.15.169:0/3310779543 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff5fc142e10 0x7ff5fc157060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:06.917 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.911+0000 7ff5fad9d640 1 -- 172.21.15.169:0/3310779543 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5ec009490 con 0x7ff5fc15a1d0 2023-10-26T01:38:06.917 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.911+0000 7ff5fa59c640 1 --2- 172.21.15.169:0/3310779543 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff5fc142e10 0x7ff5fc157060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:38:06.918 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.911+0000 7ff5fad9d640 1 --2- 172.21.15.169:0/3310779543 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff5fc15a1d0 0x7ff5fc13f010 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7ff5f400caa0 tx=0x7ff5f400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:06.918 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.911+0000 7ff5f2ffd640 1 -- 172.21.15.169:0/3310779543 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f400ac00 con 0x7ff5fc15a1d0 2023-10-26T01:38:06.918 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.911+0000 7ff5f2ffd640 1 -- 172.21.15.169:0/3310779543 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff5f400ae70 con 0x7ff5fc15a1d0 2023-10-26T01:38:06.918 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.911+0000 7ff600c4f640 1 -- 172.21.15.169:0/3310779543 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5fc13e530 con 0x7ff5fc15a1d0 2023-10-26T01:38:06.918 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.911+0000 7ff5f2ffd640 1 -- 172.21.15.169:0/3310779543 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f4003bb0 con 0x7ff5fc15a1d0 2023-10-26T01:38:06.919 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.911+0000 7ff600c4f640 1 -- 172.21.15.169:0/3310779543 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5fc1575a0 con 0x7ff5fc15a1d0 2023-10-26T01:38:06.919 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.912+0000 7ff600c4f640 1 -- 172.21.15.169:0/3310779543 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5fc13ed30 con 0x7ff5fc15a1d0 2023-10-26T01:38:06.919 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.913+0000 7ff5f2ffd640 1 -- 172.21.15.169:0/3310779543 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ff5f4003d10 con 0x7ff5fc15a1d0 2023-10-26T01:38:06.920 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.914+0000 7ff5f2ffd640 1 --2- 172.21.15.169:0/3310779543 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff5dc06ae40 0x7ff5dc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:06.920 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.914+0000 7ff5f2ffd640 1 -- 172.21.15.169:0/3310779543 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5795+0+0 (secure 0 0 0) 0x7ff5f408a0c0 con 0x7ff5fc15a1d0 2023-10-26T01:38:06.920 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.915+0000 7ff5fa59c640 1 --2- 172.21.15.169:0/3310779543 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff5dc06ae40 0x7ff5dc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:06.920 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.916+0000 7ff5fa59c640 1 --2- 172.21.15.169:0/3310779543 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff5dc06ae40 0x7ff5dc06d300 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec036000 tx=0x7ff5ec004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:06.921 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:06.919+0000 7ff5f2ffd640 1 -- 172.21.15.169:0/3310779543 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff5fc13ed30 con 0x7ff5fc15a1d0 2023-10-26T01:38:07.148 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:07.146+0000 7ff600c4f640 1 -- 172.21.15.169:0/3310779543 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7ff5fc069e90 con 0x7ff5fc15a1d0 2023-10-26T01:38:07.693 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:07.690+0000 7ff5f2ffd640 1 -- 172.21.15.169:0/3310779543 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v61) v1 ==== 104+0+0 (secure 0 0 0) 0x7ff5f4055100 con 0x7ff5fc15a1d0 2023-10-26T01:38:07.696 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:07.693+0000 7ff600c4f640 1 -- 172.21.15.169:0/3310779543 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff5dc06ae40 msgr2=0x7ff5dc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:07.696 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:07.693+0000 7ff600c4f640 1 --2- 172.21.15.169:0/3310779543 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff5dc06ae40 0x7ff5dc06d300 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec036000 tx=0x7ff5ec004730 comp rx=0 tx=0).stop 2023-10-26T01:38:07.696 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:07.693+0000 7ff600c4f640 1 -- 172.21.15.169:0/3310779543 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff5fc15a1d0 msgr2=0x7ff5fc13f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:07.696 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:07.693+0000 7ff600c4f640 1 --2- 172.21.15.169:0/3310779543 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff5fc15a1d0 0x7ff5fc13f010 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7ff5f400caa0 tx=0x7ff5f400cf60 comp rx=0 tx=0).stop 2023-10-26T01:38:07.697 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:07.693+0000 7ff600c4f640 1 -- 172.21.15.169:0/3310779543 shutdown_connections 2023-10-26T01:38:07.697 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:07.693+0000 7ff600c4f640 1 --2- 172.21.15.169:0/3310779543 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff5dc06ae40 0x7ff5dc06d300 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:07.698 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:07.693+0000 7ff600c4f640 1 --2- 172.21.15.169:0/3310779543 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff5fc15a1d0 0x7ff5fc13f010 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:07.698 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:07.693+0000 7ff600c4f640 1 --2- 172.21.15.169:0/3310779543 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff5fc1431f0 0x7ff5fc12f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:07.698 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:07.693+0000 7ff600c4f640 1 --2- 172.21.15.169:0/3310779543 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff5fc142e10 0x7ff5fc157060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:07.698 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:07.693+0000 7ff600c4f640 1 -- 172.21.15.169:0/3310779543 >> 172.21.15.169:0/3310779543 conn(0x7ff5fc06a620 msgr2=0x7ff5fc158eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:07.698 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:07.694+0000 7ff600c4f640 1 -- 172.21.15.169:0/3310779543 shutdown_connections 2023-10-26T01:38:07.699 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:07.694+0000 7ff600c4f640 1 -- 172.21.15.169:0/3310779543 wait complete. 2023-10-26T01:38:07.718 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-26T01:38:07.930 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:07.927+0000 7fb24438a640 1 -- 172.21.15.169:0/771721132 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb23c06cac0 msgr2=0x7fb23c0b08f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:07.931 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:07.927+0000 7fb24438a640 1 --2- 172.21.15.169:0/771721132 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb23c06cac0 0x7fb23c0b08f0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7fb234005fd0 tx=0x7fb23402c1b0 comp rx=0 tx=0).stop 2023-10-26T01:38:07.931 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:07.927+0000 7fb24438a640 1 -- 172.21.15.169:0/771721132 shutdown_connections 2023-10-26T01:38:07.931 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:07.927+0000 7fb24438a640 1 --2- 172.21.15.169:0/771721132 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb23c06cac0 0x7fb23c0b08f0 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:07.931 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:07.927+0000 7fb24438a640 1 --2- 172.21.15.169:0/771721132 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb23c158060 0x7fb23c0aad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:07.932 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:07.927+0000 7fb24438a640 1 -- 172.21.15.169:0/771721132 >> 172.21.15.169:0/771721132 conn(0x7fb23c0aeee0 msgr2=0x7fb23c0af2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:07.932 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:07.927+0000 7fb24438a640 1 -- 172.21.15.169:0/771721132 shutdown_connections 2023-10-26T01:38:07.932 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:07.928+0000 7fb24438a640 1 -- 172.21.15.169:0/771721132 wait complete. 2023-10-26T01:38:07.933 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:07.929+0000 7fb24438a640 1 Processor -- start 2023-10-26T01:38:07.934 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:07.930+0000 7fb24438a640 1 -- start start 2023-10-26T01:38:07.934 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:07.930+0000 7fb24438a640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb23c06c680 0x7fb23c0c96c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:07.934 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:07.930+0000 7fb24438a640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb23c06cac0 0x7fb23c1544b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:07.934 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:07.931+0000 7fb24438a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb23c158060 0x7fb23c1f1de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:07.934 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:07.931+0000 7fb24438a640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb23c1549f0 con 0x7fb23c06c680 2023-10-26T01:38:07.935 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:07.931+0000 7fb24438a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb23c1f26a0 con 0x7fb23c158060 2023-10-26T01:38:07.935 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:07.931+0000 7fb24438a640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb23c1f2980 con 0x7fb23c06cac0 2023-10-26T01:38:07.935 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:07.931+0000 7fb242126640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb23c06c680 0x7fb23c0c96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:07.935 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:07.931+0000 7fb242126640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb23c06c680 0x7fb23c0c96c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:55924/0 (socket says 172.21.15.169:55924) 2023-10-26T01:38:07.936 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:07.931+0000 7fb242126640 1 -- 172.21.15.169:0/120575285 learned_addr learned my addr 172.21.15.169:0/120575285 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:38:07.937 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:07.931+0000 7fb241925640 1 --2- 172.21.15.169:0/120575285 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb23c06cac0 0x7fb23c1544b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:07.937 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:07.931+0000 7fb242126640 1 -- 172.21.15.169:0/120575285 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb23c06cac0 msgr2=0x7fb23c1544b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:07.938 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:07.931+0000 7fb242126640 1 --2- 172.21.15.169:0/120575285 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb23c06cac0 0x7fb23c1544b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:07.938 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:07.931+0000 7fb242126640 1 -- 172.21.15.169:0/120575285 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb23c158060 msgr2=0x7fb23c1f1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:07.938 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:07.931+0000 7fb242927640 1 --2- 172.21.15.169:0/120575285 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb23c158060 0x7fb23c1f1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:07.938 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:07.931+0000 7fb242126640 1 --2- 172.21.15.169:0/120575285 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb23c158060 0x7fb23c1f1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:07.938 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:07.931+0000 7fb242126640 1 -- 172.21.15.169:0/120575285 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb234009900 con 0x7fb23c06c680 2023-10-26T01:38:07.939 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:07.931+0000 7fb241925640 1 --2- 172.21.15.169:0/120575285 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb23c06cac0 0x7fb23c1544b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:38:07.939 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:07.932+0000 7fb242126640 1 --2- 172.21.15.169:0/120575285 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb23c06c680 0x7fb23c0c96c0 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7fb22c00c970 tx=0x7fb22c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:07.939 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:07.932+0000 7fb23affd640 1 -- 172.21.15.169:0/120575285 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb22c007bb0 con 0x7fb23c06c680 2023-10-26T01:38:07.939 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:07.932+0000 7fb23affd640 1 -- 172.21.15.169:0/120575285 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb22c007e20 con 0x7fb23c06c680 2023-10-26T01:38:07.939 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:07.932+0000 7fb23affd640 1 -- 172.21.15.169:0/120575285 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb22c005280 con 0x7fb23c06c680 2023-10-26T01:38:07.940 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:07.932+0000 7fb24438a640 1 -- 172.21.15.169:0/120575285 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb23c155150 con 0x7fb23c06c680 2023-10-26T01:38:07.940 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:07.932+0000 7fb24438a640 1 -- 172.21.15.169:0/120575285 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb23c1f2c60 con 0x7fb23c06c680 2023-10-26T01:38:07.940 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:07.933+0000 7fb24438a640 1 -- 172.21.15.169:0/120575285 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb23c066af0 con 0x7fb23c06c680 2023-10-26T01:38:07.940 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:07.934+0000 7fb23affd640 1 -- 172.21.15.169:0/120575285 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fb22c0053e0 con 0x7fb23c06c680 2023-10-26T01:38:07.941 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:07.935+0000 7fb23affd640 1 --2- 172.21.15.169:0/120575285 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb22806ae40 0x7fb22806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:07.941 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:07.935+0000 7fb23affd640 1 -- 172.21.15.169:0/120575285 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fb22c089cb0 con 0x7fb23c06c680 2023-10-26T01:38:07.941 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:07.936+0000 7fb241925640 1 --2- 172.21.15.169:0/120575285 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb22806ae40 0x7fb22806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:07.942 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:07.937+0000 7fb241925640 1 --2- 172.21.15.169:0/120575285 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb22806ae40 0x7fb22806d300 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fb23c066f90 tx=0x7fb230008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:07.942 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:07.940+0000 7fb23affd640 1 -- 172.21.15.169:0/120575285 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb22c054c60 con 0x7fb23c06c680 2023-10-26T01:38:08.119 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:08.116+0000 7fb24438a640 1 -- 172.21.15.169:0/120575285 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb23c0aed00 con 0x7fb23c06c680 2023-10-26T01:38:09.048 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:09.045+0000 7fb23affd640 1 -- 172.21.15.169:0/120575285 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v125) v1 ==== 181+0+0 (secure 0 0 0) 0x7fb22c01d070 con 0x7fb23c06c680 2023-10-26T01:38:09.049 INFO:teuthology.orchestra.run.smithi169.stderr:pool 'cephfs_metadata' removed 2023-10-26T01:38:09.055 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:09.050+0000 7fb24438a640 1 -- 172.21.15.169:0/120575285 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb22806ae40 msgr2=0x7fb22806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:09.055 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:09.050+0000 7fb24438a640 1 --2- 172.21.15.169:0/120575285 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb22806ae40 0x7fb22806d300 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fb23c066f90 tx=0x7fb230008040 comp rx=0 tx=0).stop 2023-10-26T01:38:09.055 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:09.050+0000 7fb24438a640 1 -- 172.21.15.169:0/120575285 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb23c06c680 msgr2=0x7fb23c0c96c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:09.056 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:09.050+0000 7fb24438a640 1 --2- 172.21.15.169:0/120575285 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb23c06c680 0x7fb23c0c96c0 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7fb22c00c970 tx=0x7fb22c00ce30 comp rx=0 tx=0).stop 2023-10-26T01:38:09.056 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:09.052+0000 7fb24438a640 1 -- 172.21.15.169:0/120575285 shutdown_connections 2023-10-26T01:38:09.056 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:09.052+0000 7fb24438a640 1 --2- 172.21.15.169:0/120575285 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb22806ae40 0x7fb22806d300 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:09.056 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:09.052+0000 7fb24438a640 1 --2- 172.21.15.169:0/120575285 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb23c158060 0x7fb23c1f1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:09.057 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:09.052+0000 7fb24438a640 1 --2- 172.21.15.169:0/120575285 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb23c06cac0 0x7fb23c1544b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:09.057 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:09.052+0000 7fb24438a640 1 --2- 172.21.15.169:0/120575285 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb23c06c680 0x7fb23c0c96c0 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:09.057 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:09.052+0000 7fb24438a640 1 -- 172.21.15.169:0/120575285 >> 172.21.15.169:0/120575285 conn(0x7fb23c0aeee0 msgr2=0x7fb23c15a430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:09.057 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:09.053+0000 7fb24438a640 1 -- 172.21.15.169:0/120575285 shutdown_connections 2023-10-26T01:38:09.058 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:09.053+0000 7fb24438a640 1 -- 172.21.15.169:0/120575285 wait complete. 2023-10-26T01:38:09.077 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-26T01:38:09.297 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:09.295+0000 7fcb64ea6640 1 -- 172.21.15.169:0/2953007532 <== mon.2 v2:172.21.15.170:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb58034d00 con 0x7fcb6006b4c0 2023-10-26T01:38:09.298 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:09.295+0000 7fcb6810c640 1 -- 172.21.15.169:0/2953007532 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcb6006b4c0 msgr2=0x7fcb600d03d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:09.299 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:09.295+0000 7fcb6810c640 1 --2- 172.21.15.169:0/2953007532 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcb6006b4c0 0x7fcb600d03d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fcb58005fd0 tx=0x7fcb5802c2d0 comp rx=0 tx=0).stop 2023-10-26T01:38:09.299 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:09.295+0000 7fcb6810c640 1 -- 172.21.15.169:0/2953007532 shutdown_connections 2023-10-26T01:38:09.299 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:09.295+0000 7fcb6810c640 1 --2- 172.21.15.169:0/2953007532 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcb6006b4c0 0x7fcb600d03d0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:09.299 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:09.295+0000 7fcb6810c640 1 --2- 172.21.15.169:0/2953007532 >> v2:172.21.15.170:3300/0 conn(0x7fcb600ccaf0 0x7fcb6006af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:09.300 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:09.295+0000 7fcb6810c640 1 -- 172.21.15.169:0/2953007532 >> 172.21.15.169:0/2953007532 conn(0x7fcb600b35f0 msgr2=0x7fcb600b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:09.300 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:09.295+0000 7fcb6810c640 1 -- 172.21.15.169:0/2953007532 shutdown_connections 2023-10-26T01:38:09.300 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:09.296+0000 7fcb6810c640 1 -- 172.21.15.169:0/2953007532 wait complete. 2023-10-26T01:38:09.300 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:09.297+0000 7fcb6810c640 1 Processor -- start 2023-10-26T01:38:09.301 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:09.297+0000 7fcb6810c640 1 -- start start 2023-10-26T01:38:09.303 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:09.298+0000 7fcb6810c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb6006b4c0 0x7fcb600ab350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:09.303 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:09.298+0000 7fcb6810c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb600ccaf0 0x7fcb600b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:09.303 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:09.298+0000 7fcb6810c640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcb6015ba40 0x7fcb600b5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:09.303 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:09.298+0000 7fcb6810c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb600abd30 con 0x7fcb600ccaf0 2023-10-26T01:38:09.304 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:09.298+0000 7fcb6810c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb600ac010 con 0x7fcb6006b4c0 2023-10-26T01:38:09.304 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:09.298+0000 7fcb6810c640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb600ac2f0 con 0x7fcb6015ba40 2023-10-26T01:38:09.304 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:09.299+0000 7fcb656a7640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb600ccaf0 0x7fcb600b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:09.304 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:09.299+0000 7fcb656a7640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb600ccaf0 0x7fcb600b4d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:55938/0 (socket says 172.21.15.169:55938) 2023-10-26T01:38:09.304 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:09.299+0000 7fcb656a7640 1 -- 172.21.15.169:0/2373327359 learned_addr learned my addr 172.21.15.169:0/2373327359 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:38:09.305 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:09.299+0000 7fcb65ea8640 1 --2- 172.21.15.169:0/2373327359 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb6006b4c0 0x7fcb600ab350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:09.305 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:09.299+0000 7fcb666a9640 1 --2- 172.21.15.169:0/2373327359 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcb6015ba40 0x7fcb600b5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:09.305 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:09.299+0000 7fcb656a7640 1 -- 172.21.15.169:0/2373327359 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcb6015ba40 msgr2=0x7fcb600b5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:09.305 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:09.299+0000 7fcb656a7640 1 --2- 172.21.15.169:0/2373327359 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcb6015ba40 0x7fcb600b5280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:09.305 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:09.299+0000 7fcb656a7640 1 -- 172.21.15.169:0/2373327359 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb6006b4c0 msgr2=0x7fcb600ab350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:09.306 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:09.299+0000 7fcb656a7640 1 --2- 172.21.15.169:0/2373327359 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb6006b4c0 0x7fcb600ab350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:09.306 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:09.299+0000 7fcb656a7640 1 -- 172.21.15.169:0/2373327359 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb58009900 con 0x7fcb600ccaf0 2023-10-26T01:38:09.306 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:09.300+0000 7fcb656a7640 1 --2- 172.21.15.169:0/2373327359 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb600ccaf0 0x7fcb600b4d40 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7fcb5000e9f0 tx=0x7fcb5000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:09.306 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:09.300+0000 7fcb567fc640 1 -- 172.21.15.169:0/2373327359 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb50009700 con 0x7fcb600ccaf0 2023-10-26T01:38:09.306 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:09.300+0000 7fcb567fc640 1 -- 172.21.15.169:0/2373327359 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fcb5000ce70 con 0x7fcb600ccaf0 2023-10-26T01:38:09.307 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:09.300+0000 7fcb6810c640 1 -- 172.21.15.169:0/2373327359 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb600ac5d0 con 0x7fcb600ccaf0 2023-10-26T01:38:09.307 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:09.300+0000 7fcb567fc640 1 -- 172.21.15.169:0/2373327359 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb50010670 con 0x7fcb600ccaf0 2023-10-26T01:38:09.307 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:09.300+0000 7fcb6810c640 1 -- 172.21.15.169:0/2373327359 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb600ab890 con 0x7fcb600ccaf0 2023-10-26T01:38:09.308 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:09.301+0000 7fcb6810c640 1 -- 172.21.15.169:0/2373327359 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb6004da70 con 0x7fcb600ccaf0 2023-10-26T01:38:09.308 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:09.302+0000 7fcb567fc640 1 -- 172.21.15.169:0/2373327359 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fcb500107d0 con 0x7fcb600ccaf0 2023-10-26T01:38:09.308 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:09.303+0000 7fcb567fc640 1 --2- 172.21.15.169:0/2373327359 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcb4406ae40 0x7fcb4406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:09.309 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:09.303+0000 7fcb567fc640 1 -- 172.21.15.169:0/2373327359 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5357+0+0 (secure 0 0 0) 0x7fcb50014070 con 0x7fcb600ccaf0 2023-10-26T01:38:09.309 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:09.304+0000 7fcb65ea8640 1 --2- 172.21.15.169:0/2373327359 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcb4406ae40 0x7fcb4406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:09.309 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:09.305+0000 7fcb65ea8640 1 --2- 172.21.15.169:0/2373327359 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcb4406ae40 0x7fcb4406d300 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c007fd0 tx=0x7fcb5c009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:09.311 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:09.308+0000 7fcb567fc640 1 -- 172.21.15.169:0/2373327359 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcb50051ca0 con 0x7fcb600ccaf0 2023-10-26T01:38:09.498 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:09.496+0000 7fcb6810c640 1 -- 172.21.15.169:0/2373327359 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fcb600cfc60 con 0x7fcb600ccaf0 2023-10-26T01:38:10.224 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:10.222+0000 7fcb567fc640 1 -- 172.21.15.169:0/2373327359 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v126) v1 ==== 169+0+0 (secure 0 0 0) 0x7fcb50058bb0 con 0x7fcb600ccaf0 2023-10-26T01:38:10.225 INFO:teuthology.orchestra.run.smithi169.stderr:pool 'cephfs_data' removed 2023-10-26T01:38:10.228 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:10.225+0000 7fcb6810c640 1 -- 172.21.15.169:0/2373327359 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcb4406ae40 msgr2=0x7fcb4406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:10.228 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:10.225+0000 7fcb6810c640 1 --2- 172.21.15.169:0/2373327359 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcb4406ae40 0x7fcb4406d300 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c007fd0 tx=0x7fcb5c009040 comp rx=0 tx=0).stop 2023-10-26T01:38:10.228 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:10.226+0000 7fcb6810c640 1 -- 172.21.15.169:0/2373327359 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb600ccaf0 msgr2=0x7fcb600b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:10.229 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:10.226+0000 7fcb6810c640 1 --2- 172.21.15.169:0/2373327359 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb600ccaf0 0x7fcb600b4d40 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7fcb5000e9f0 tx=0x7fcb5000eeb0 comp rx=0 tx=0).stop 2023-10-26T01:38:10.229 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:10.226+0000 7fcb6810c640 1 -- 172.21.15.169:0/2373327359 shutdown_connections 2023-10-26T01:38:10.229 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:10.226+0000 7fcb6810c640 1 --2- 172.21.15.169:0/2373327359 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcb4406ae40 0x7fcb4406d300 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:10.229 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:10.226+0000 7fcb6810c640 1 --2- 172.21.15.169:0/2373327359 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb600ccaf0 0x7fcb600b4d40 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:10.229 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:10.226+0000 7fcb6810c640 1 --2- 172.21.15.169:0/2373327359 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcb6015ba40 0x7fcb600b5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:10.230 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:10.226+0000 7fcb6810c640 1 --2- 172.21.15.169:0/2373327359 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb6006b4c0 0x7fcb600ab350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:10.230 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:10.226+0000 7fcb6810c640 1 -- 172.21.15.169:0/2373327359 >> 172.21.15.169:0/2373327359 conn(0x7fcb600b35f0 msgr2=0x7fcb600b26a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:10.230 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:10.226+0000 7fcb6810c640 1 -- 172.21.15.169:0/2373327359 shutdown_connections 2023-10-26T01:38:10.230 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:10.226+0000 7fcb6810c640 1 -- 172.21.15.169:0/2373327359 wait complete. 2023-10-26T01:38:10.248 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 18 2023-10-26T01:38:10.454 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:10.452+0000 7fbbbd094640 1 -- 172.21.15.169:0/2142669662 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbbb81593e0 msgr2=0x7fbbb80ac200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:10.454 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:10.452+0000 7fbbbd094640 1 --2- 172.21.15.169:0/2142669662 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbbb81593e0 0x7fbbb80ac200 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fbba8005fd0 tx=0x7fbba802c0e0 comp rx=0 tx=0).stop 2023-10-26T01:38:10.455 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:10.452+0000 7fbbbd094640 1 -- 172.21.15.169:0/2142669662 shutdown_connections 2023-10-26T01:38:10.455 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:10.452+0000 7fbbbd094640 1 --2- 172.21.15.169:0/2142669662 >> v2:172.21.15.170:3300/0 conn(0x7fbbb81597c0 0x7fbbb815dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:10.455 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:10.452+0000 7fbbbd094640 1 --2- 172.21.15.169:0/2142669662 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbbb81593e0 0x7fbbb80ac200 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:10.455 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:10.452+0000 7fbbbd094640 1 -- 172.21.15.169:0/2142669662 >> 172.21.15.169:0/2142669662 conn(0x7fbbb8069930 msgr2=0x7fbbb8069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:10.456 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:10.452+0000 7fbbbd094640 1 -- 172.21.15.169:0/2142669662 shutdown_connections 2023-10-26T01:38:10.456 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:10.452+0000 7fbbbd094640 1 -- 172.21.15.169:0/2142669662 wait complete. 2023-10-26T01:38:10.458 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:10.454+0000 7fbbbd094640 1 Processor -- start 2023-10-26T01:38:10.459 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:10.454+0000 7fbbbd094640 1 -- start start 2023-10-26T01:38:10.459 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:10.455+0000 7fbbbd094640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbbb80b0c80 0x7fbbb8151960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:10.459 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:10.455+0000 7fbbbd094640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbbb81593e0 0x7fbbb806b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:10.459 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:10.455+0000 7fbbbd094640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbbb81597c0 0x7fbbb815a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:10.460 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:10.455+0000 7fbbbd094640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb8151ea0 con 0x7fbbb81597c0 2023-10-26T01:38:10.460 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:10.455+0000 7fbbbd094640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb806c690 con 0x7fbbb80b0c80 2023-10-26T01:38:10.460 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:10.455+0000 7fbbbd094640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb806bce0 con 0x7fbbb81593e0 2023-10-26T01:38:10.460 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:10.455+0000 7fbbb759e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbbb81597c0 0x7fbbb815a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:10.461 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:10.455+0000 7fbbb759e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbbb81597c0 0x7fbbb815a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:55944/0 (socket says 172.21.15.169:55944) 2023-10-26T01:38:10.461 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:10.455+0000 7fbbb759e640 1 -- 172.21.15.169:0/1625601586 learned_addr learned my addr 172.21.15.169:0/1625601586 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:38:10.461 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:10.455+0000 7fbbb659c640 1 --2- 172.21.15.169:0/1625601586 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbbb81593e0 0x7fbbb806b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:10.461 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:10.456+0000 7fbbb6d9d640 1 --2- 172.21.15.169:0/1625601586 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbbb80b0c80 0x7fbbb8151960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:10.462 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:10.456+0000 7fbbb759e640 1 -- 172.21.15.169:0/1625601586 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbbb81593e0 msgr2=0x7fbbb806b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:10.462 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:10.456+0000 7fbbb759e640 1 --2- 172.21.15.169:0/1625601586 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbbb81593e0 0x7fbbb806b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:10.462 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:10.456+0000 7fbbb759e640 1 -- 172.21.15.169:0/1625601586 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbbb80b0c80 msgr2=0x7fbbb8151960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:10.462 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:10.456+0000 7fbbb759e640 1 --2- 172.21.15.169:0/1625601586 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbbb80b0c80 0x7fbbb8151960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:10.463 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:10.456+0000 7fbbb759e640 1 -- 172.21.15.169:0/1625601586 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbba8009840 con 0x7fbbb81597c0 2023-10-26T01:38:10.463 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:10.456+0000 7fbbb6d9d640 1 --2- 172.21.15.169:0/1625601586 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbbb80b0c80 0x7fbbb8151960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:38:10.463 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:10.456+0000 7fbbb759e640 1 --2- 172.21.15.169:0/1625601586 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbbb81597c0 0x7fbbb815a6c0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7fbbb000c430 tx=0x7fbbb000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:10.463 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:10.456+0000 7fbbaf7fe640 1 -- 172.21.15.169:0/1625601586 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb0004230 con 0x7fbbb81597c0 2023-10-26T01:38:10.463 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:10.456+0000 7fbbaf7fe640 1 -- 172.21.15.169:0/1625601586 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fbbb00044a0 con 0x7fbbb81597c0 2023-10-26T01:38:10.464 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:10.456+0000 7fbbaf7fe640 1 -- 172.21.15.169:0/1625601586 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb0003850 con 0x7fbbb81597c0 2023-10-26T01:38:10.464 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:10.456+0000 7fbbbd094640 1 -- 172.21.15.169:0/1625601586 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbb8157070 con 0x7fbbb81597c0 2023-10-26T01:38:10.464 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:10.457+0000 7fbbbd094640 1 -- 172.21.15.169:0/1625601586 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbb8157630 con 0x7fbbb81597c0 2023-10-26T01:38:10.465 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:10.458+0000 7fbbbd094640 1 -- 172.21.15.169:0/1625601586 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbb804db10 con 0x7fbbb81597c0 2023-10-26T01:38:10.465 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:10.459+0000 7fbbaf7fe640 1 -- 172.21.15.169:0/1625601586 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fbbb00039b0 con 0x7fbbb81597c0 2023-10-26T01:38:10.466 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:10.459+0000 7fbbaf7fe640 1 --2- 172.21.15.169:0/1625601586 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbb9806ae70 0x7fbb9806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:10.466 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:10.460+0000 7fbbaf7fe640 1 -- 172.21.15.169:0/1625601586 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fbbb0088ba0 con 0x7fbbb81597c0 2023-10-26T01:38:10.466 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:10.460+0000 7fbbb6d9d640 1 --2- 172.21.15.169:0/1625601586 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbb9806ae70 0x7fbb9806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:10.466 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:10.461+0000 7fbbb6d9d640 1 --2- 172.21.15.169:0/1625601586 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbb9806ae70 0x7fbb9806d330 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fbba8035000 tx=0x7fbba8004ab0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:10.467 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:10.464+0000 7fbbaf7fe640 1 -- 172.21.15.169:0/1625601586 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbbb0053eb0 con 0x7fbbb81597c0 2023-10-26T01:38:10.654 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:10.652+0000 7fbbbd094640 1 -- 172.21.15.169:0/1625601586 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "config reset", "num": 18} v 0) v1 -- 0x7fbbb815c600 con 0x7fbbb81597c0 2023-10-26T01:38:10.659 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:10.657+0000 7fbbaf7fe640 1 -- 172.21.15.169:0/1625601586 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 18}]=0 v30) v1 ==== 71+0+0 (secure 0 0 0) 0x7fbbb0017020 con 0x7fbbb81597c0 2023-10-26T01:38:10.662 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:10.660+0000 7fbbbd094640 1 -- 172.21.15.169:0/1625601586 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbb9806ae70 msgr2=0x7fbb9806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:10.662 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:10.660+0000 7fbbbd094640 1 --2- 172.21.15.169:0/1625601586 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbb9806ae70 0x7fbb9806d330 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fbba8035000 tx=0x7fbba8004ab0 comp rx=0 tx=0).stop 2023-10-26T01:38:10.662 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:10.660+0000 7fbbbd094640 1 -- 172.21.15.169:0/1625601586 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbbb81597c0 msgr2=0x7fbbb815a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:10.662 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:10.660+0000 7fbbbd094640 1 --2- 172.21.15.169:0/1625601586 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbbb81597c0 0x7fbbb815a6c0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7fbbb000c430 tx=0x7fbbb000c8f0 comp rx=0 tx=0).stop 2023-10-26T01:38:10.662 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:10.660+0000 7fbbbd094640 1 -- 172.21.15.169:0/1625601586 shutdown_connections 2023-10-26T01:38:10.663 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:10.660+0000 7fbbbd094640 1 --2- 172.21.15.169:0/1625601586 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbb9806ae70 0x7fbb9806d330 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:10.663 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:10.660+0000 7fbbbd094640 1 --2- 172.21.15.169:0/1625601586 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbbb81597c0 0x7fbbb815a6c0 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:10.663 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:10.660+0000 7fbbbd094640 1 --2- 172.21.15.169:0/1625601586 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbbb81593e0 0x7fbbb806b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:10.663 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:10.660+0000 7fbbbd094640 1 --2- 172.21.15.169:0/1625601586 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbbb80b0c80 0x7fbbb8151960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:10.663 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:10.660+0000 7fbbbd094640 1 -- 172.21.15.169:0/1625601586 >> 172.21.15.169:0/1625601586 conn(0x7fbbb8069930 msgr2=0x7fbbb80cbaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:10.664 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:10.660+0000 7fbbbd094640 1 -- 172.21.15.169:0/1625601586 shutdown_connections 2023-10-26T01:38:10.664 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:10.660+0000 7fbbbd094640 1 -- 172.21.15.169:0/1625601586 wait complete. 2023-10-26T01:38:10.680 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestQuotaFull.test_barrier' 2023-10-26T01:38:10.892 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:10.890+0000 7f1278075640 1 -- 172.21.15.169:0/1276410100 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1270158850 msgr2=0x7f12700aab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:10.893 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:10.890+0000 7f1278075640 1 --2- 172.21.15.169:0/1276410100 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1270158850 0x7f12700aab40 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f12700accf0 tx=0x7f126402bda0 comp rx=0 tx=0).stop 2023-10-26T01:38:10.893 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:10.890+0000 7f1278075640 1 -- 172.21.15.169:0/1276410100 shutdown_connections 2023-10-26T01:38:10.893 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:10.890+0000 7f1278075640 1 --2- 172.21.15.169:0/1276410100 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1270158850 0x7f12700aab40 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:10.893 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:10.890+0000 7f1278075640 1 --2- 172.21.15.169:0/1276410100 >> v2:172.21.15.170:3300/0 conn(0x7f127006ce20 0x7f127014e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:10.893 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:10.890+0000 7f1278075640 1 -- 172.21.15.169:0/1276410100 >> 172.21.15.169:0/1276410100 conn(0x7f1270069930 msgr2=0x7f1270069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:10.894 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:10.890+0000 7f1278075640 1 -- 172.21.15.169:0/1276410100 shutdown_connections 2023-10-26T01:38:10.894 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:10.890+0000 7f1278075640 1 -- 172.21.15.169:0/1276410100 wait complete. 2023-10-26T01:38:10.895 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:10.892+0000 7f1278075640 1 Processor -- start 2023-10-26T01:38:10.895 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:10.892+0000 7f1278075640 1 -- start start 2023-10-26T01:38:10.895 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:10.893+0000 7f1278075640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f127006ad30 0x7f12701ef530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:10.896 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:10.893+0000 7f1278075640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f127006ce20 0x7f1270153350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:10.896 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:10.893+0000 7f1278075640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1270158850 0x7f1270153890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:10.896 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:10.893+0000 7f1278075640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1270154ad0 con 0x7f127006ad30 2023-10-26T01:38:10.896 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:10.893+0000 7f1278075640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1270153ec0 con 0x7f127006ce20 2023-10-26T01:38:10.897 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:10.893+0000 7f1278075640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12701541a0 con 0x7f1270158850 2023-10-26T01:38:10.897 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:10.894+0000 7f1275e11640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f127006ad30 0x7f12701ef530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:10.897 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:10.894+0000 7f1275e11640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f127006ad30 0x7f12701ef530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:55954/0 (socket says 172.21.15.169:55954) 2023-10-26T01:38:10.897 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:10.894+0000 7f1275e11640 1 -- 172.21.15.169:0/2904007654 learned_addr learned my addr 172.21.15.169:0/2904007654 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:38:10.899 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:10.894+0000 7f1275610640 1 --2- 172.21.15.169:0/2904007654 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f127006ce20 0x7f1270153350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:10.899 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:10.894+0000 7f1275e11640 1 -- 172.21.15.169:0/2904007654 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1270158850 msgr2=0x7f1270153890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:10.899 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:10.894+0000 7f1275e11640 1 --2- 172.21.15.169:0/2904007654 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1270158850 0x7f1270153890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:10.899 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:10.894+0000 7f1275e11640 1 -- 172.21.15.169:0/2904007654 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f127006ce20 msgr2=0x7f1270153350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:10.900 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:10.894+0000 7f1275e11640 1 --2- 172.21.15.169:0/2904007654 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f127006ce20 0x7f1270153350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:10.900 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:10.894+0000 7f1275e11640 1 -- 172.21.15.169:0/2904007654 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1264009560 con 0x7f127006ad30 2023-10-26T01:38:10.900 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:10.894+0000 7f1275e11640 1 --2- 172.21.15.169:0/2904007654 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f127006ad30 0x7f12701ef530 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f126c00ddc0 tx=0x7f126c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:10.900 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:10.895+0000 7f12627fc640 1 -- 172.21.15.169:0/2904007654 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f126c00e650 con 0x7f127006ad30 2023-10-26T01:38:10.900 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:10.895+0000 7f12627fc640 1 -- 172.21.15.169:0/2904007654 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f126c00e8a0 con 0x7f127006ad30 2023-10-26T01:38:10.901 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:10.895+0000 7f12627fc640 1 -- 172.21.15.169:0/2904007654 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f126c00ecb0 con 0x7f127006ad30 2023-10-26T01:38:10.901 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:10.895+0000 7f1278075640 1 -- 172.21.15.169:0/2904007654 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1270154480 con 0x7f127006ad30 2023-10-26T01:38:10.901 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:10.895+0000 7f1278075640 1 -- 172.21.15.169:0/2904007654 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12700704a0 con 0x7f127006ad30 2023-10-26T01:38:10.901 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:10.896+0000 7f1278075640 1 -- 172.21.15.169:0/2904007654 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f127004db10 con 0x7f127006ad30 2023-10-26T01:38:10.901 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:10.897+0000 7f12627fc640 1 -- 172.21.15.169:0/2904007654 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f126c016440 con 0x7f127006ad30 2023-10-26T01:38:10.902 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:10.898+0000 7f12627fc640 1 --2- 172.21.15.169:0/2904007654 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f125406ae70 0x7f125406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:10.902 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:10.898+0000 7f12627fc640 1 -- 172.21.15.169:0/2904007654 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f126c089b20 con 0x7f127006ad30 2023-10-26T01:38:10.903 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:10.899+0000 7f1275610640 1 --2- 172.21.15.169:0/2904007654 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f125406ae70 0x7f125406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:10.903 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:10.900+0000 7f1275610640 1 --2- 172.21.15.169:0/2904007654 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f125406ae70 0x7f125406d330 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f1264035000 tx=0x7f1264004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:10.904 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:10.903+0000 7f12627fc640 1 -- 172.21.15.169:0/2904007654 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f126c054f80 con 0x7f127006ad30 2023-10-26T01:38:11.161 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:11.159+0000 7f1278075640 1 -- 172.21.15.169:0/2904007654 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_barrier"]} v 0) v1 -- 0x7f127014e800 con 0x7f127006ad30 2023-10-26T01:38:11.233 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:11.232+0000 7f12627fc640 1 -- 172.21.15.169:0/2904007654 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_barrier"]}]=0 v389) v1 ==== 128+0+0 (secure 0 0 0) 0x7f126c01b020 con 0x7f127006ad30 2023-10-26T01:38:11.238 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:11.236+0000 7f1278075640 1 -- 172.21.15.169:0/2904007654 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f125406ae70 msgr2=0x7f125406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:11.238 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:11.236+0000 7f1278075640 1 --2- 172.21.15.169:0/2904007654 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f125406ae70 0x7f125406d330 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f1264035000 tx=0x7f1264004730 comp rx=0 tx=0).stop 2023-10-26T01:38:11.238 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:11.236+0000 7f1278075640 1 -- 172.21.15.169:0/2904007654 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f127006ad30 msgr2=0x7f12701ef530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:11.238 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:11.236+0000 7f1278075640 1 --2- 172.21.15.169:0/2904007654 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f127006ad30 0x7f12701ef530 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f126c00ddc0 tx=0x7f126c00a620 comp rx=0 tx=0).stop 2023-10-26T01:38:11.239 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:11.236+0000 7f1278075640 1 -- 172.21.15.169:0/2904007654 shutdown_connections 2023-10-26T01:38:11.239 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:11.236+0000 7f1278075640 1 --2- 172.21.15.169:0/2904007654 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f125406ae70 0x7f125406d330 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:11.239 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:11.236+0000 7f1278075640 1 --2- 172.21.15.169:0/2904007654 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1270158850 0x7f1270153890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:11.239 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:11.236+0000 7f1278075640 1 --2- 172.21.15.169:0/2904007654 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f127006ce20 0x7f1270153350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:11.239 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:11.236+0000 7f1278075640 1 --2- 172.21.15.169:0/2904007654 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f127006ad30 0x7f12701ef530 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:11.240 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:11.236+0000 7f1278075640 1 -- 172.21.15.169:0/2904007654 >> 172.21.15.169:0/2904007654 conn(0x7f1270069930 msgr2=0x7f127014d3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:11.240 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:11.236+0000 7f1278075640 1 -- 172.21.15.169:0/2904007654 shutdown_connections 2023-10-26T01:38:11.240 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:11.236+0000 7f1278075640 1 -- 172.21.15.169:0/2904007654 wait complete. 2023-10-26T01:38:11.256 INFO:tasks.cephfs_test_runner:That when an OSD epoch barrier is set on an MDS, subsequently ... ok 2023-10-26T01:38:11.256 INFO:tasks.cephfs_test_runner:Starting test: test_full_different_file (tasks.cephfs.test_full.TestQuotaFull) 2023-10-26T01:38:11.257 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:38:11.473 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:11.471+0000 7eff1bad7640 1 -- 172.21.15.169:0/725001281 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7eff1414e790 msgr2=0x7eff140aab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:11.473 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:11.471+0000 7eff1bad7640 1 --2- 172.21.15.169:0/725001281 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7eff1414e790 0x7eff140aab40 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7eff10009830 tx=0x7eff1002bcd0 comp rx=0 tx=0).stop 2023-10-26T01:38:11.473 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:11.471+0000 7eff1bad7640 1 -- 172.21.15.169:0/725001281 shutdown_connections 2023-10-26T01:38:11.473 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:11.471+0000 7eff1bad7640 1 --2- 172.21.15.169:0/725001281 >> v2:172.21.15.170:3300/0 conn(0x7eff14158850 0x7eff140cd720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:11.474 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:11.471+0000 7eff1bad7640 1 --2- 172.21.15.169:0/725001281 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7eff1414e790 0x7eff140aab40 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:11.474 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:11.471+0000 7eff1bad7640 1 -- 172.21.15.169:0/725001281 >> 172.21.15.169:0/725001281 conn(0x7eff14069930 msgr2=0x7eff14069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:11.474 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:11.471+0000 7eff1bad7640 1 -- 172.21.15.169:0/725001281 shutdown_connections 2023-10-26T01:38:11.474 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:11.471+0000 7eff1bad7640 1 -- 172.21.15.169:0/725001281 wait complete. 2023-10-26T01:38:11.476 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:11.472+0000 7eff1bad7640 1 Processor -- start 2023-10-26T01:38:11.477 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:11.473+0000 7eff1bad7640 1 -- start start 2023-10-26T01:38:11.477 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:11.473+0000 7eff1bad7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff140cdc60 0x7eff14163c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:11.477 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:11.473+0000 7eff1bad7640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff1414e790 0x7eff141491c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:11.477 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:11.473+0000 7eff1bad7640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7eff14158850 0x7eff14149700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:11.477 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:11.473+0000 7eff1bad7640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff1414b940 con 0x7eff1414e790 2023-10-26T01:38:11.478 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:11.473+0000 7eff1bad7640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff14156b10 con 0x7eff140cdc60 2023-10-26T01:38:11.478 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:11.473+0000 7eff1bad7640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff14164170 con 0x7eff14158850 2023-10-26T01:38:11.478 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:11.474+0000 7eff19072640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff1414e790 0x7eff141491c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:11.478 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:11.474+0000 7eff19072640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff1414e790 0x7eff141491c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:55970/0 (socket says 172.21.15.169:55970) 2023-10-26T01:38:11.479 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:11.474+0000 7eff19072640 1 -- 172.21.15.169:0/2170260245 learned_addr learned my addr 172.21.15.169:0/2170260245 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:38:11.479 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:11.474+0000 7eff19873640 1 --2- 172.21.15.169:0/2170260245 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff140cdc60 0x7eff14163c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:11.479 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:11.474+0000 7eff1a074640 1 --2- 172.21.15.169:0/2170260245 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7eff14158850 0x7eff14149700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:11.479 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:11.474+0000 7eff19072640 1 -- 172.21.15.169:0/2170260245 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7eff14158850 msgr2=0x7eff14149700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:11.479 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:11.474+0000 7eff19072640 1 --2- 172.21.15.169:0/2170260245 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7eff14158850 0x7eff14149700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:11.480 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:11.474+0000 7eff19072640 1 -- 172.21.15.169:0/2170260245 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff140cdc60 msgr2=0x7eff14163c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:11.480 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:11.474+0000 7eff19072640 1 --2- 172.21.15.169:0/2170260245 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff140cdc60 0x7eff14163c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:11.480 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:11.474+0000 7eff19072640 1 -- 172.21.15.169:0/2170260245 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff10009490 con 0x7eff1414e790 2023-10-26T01:38:11.480 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:11.475+0000 7eff19072640 1 --2- 172.21.15.169:0/2170260245 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff1414e790 0x7eff141491c0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7eff0400e9c0 tx=0x7eff0400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:11.480 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:11.475+0000 7eff0a7fc640 1 -- 172.21.15.169:0/2170260245 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff0400ccc0 con 0x7eff1414e790 2023-10-26T01:38:11.481 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:11.475+0000 7eff0a7fc640 1 -- 172.21.15.169:0/2170260245 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7eff0400ce20 con 0x7eff1414e790 2023-10-26T01:38:11.482 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:11.475+0000 7eff1bad7640 1 -- 172.21.15.169:0/2170260245 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff14156db0 con 0x7eff1414e790 2023-10-26T01:38:11.485 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:11.475+0000 7eff0a7fc640 1 -- 172.21.15.169:0/2170260245 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff04005280 con 0x7eff1414e790 2023-10-26T01:38:11.485 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:11.475+0000 7eff1bad7640 1 -- 172.21.15.169:0/2170260245 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff14157370 con 0x7eff1414e790 2023-10-26T01:38:11.486 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:11.476+0000 7eff1bad7640 1 -- 172.21.15.169:0/2170260245 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff1404db10 con 0x7eff1414e790 2023-10-26T01:38:11.487 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:11.477+0000 7eff0a7fc640 1 -- 172.21.15.169:0/2170260245 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7eff040053e0 con 0x7eff1414e790 2023-10-26T01:38:11.487 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:11.478+0000 7eff0a7fc640 1 --2- 172.21.15.169:0/2170260245 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7efef806ae40 0x7efef806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:11.487 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:11.478+0000 7eff0a7fc640 1 -- 172.21.15.169:0/2170260245 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 4943+0+0 (secure 0 0 0) 0x7eff04014070 con 0x7eff1414e790 2023-10-26T01:38:11.487 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:11.479+0000 7eff19873640 1 --2- 172.21.15.169:0/2170260245 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7efef806ae40 0x7efef806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:11.487 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:11.480+0000 7eff19873640 1 --2- 172.21.15.169:0/2170260245 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7efef806ae40 0x7efef806d300 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7eff10036000 tx=0x7eff10004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:11.488 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:11.482+0000 7eff0a7fc640 1 -- 172.21.15.169:0/2170260245 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7eff04058c10 con 0x7eff1414e790 2023-10-26T01:38:11.676 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:11.674+0000 7eff1bad7640 1 -- 172.21.15.169:0/2170260245 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7eff140aca70 con 0x7eff1414e790 2023-10-26T01:38:11.678 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:11.676+0000 7eff0a7fc640 1 -- 172.21.15.169:0/2170260245 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v126) v1 ==== 74+0+13045 (secure 0 0 0) 0x7eff040587a0 con 0x7eff1414e790 2023-10-26T01:38:11.679 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:38:11.679 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":126,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:38:10.218936+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.169:6835/668588916":"2023-10-27T01:38:06.684183+0000","172.21.15.169:6834/668588916":"2023-10-27T01:38:06.684183+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:38:11.681 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:11.679+0000 7eff1bad7640 1 -- 172.21.15.169:0/2170260245 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7efef806ae40 msgr2=0x7efef806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:11.681 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:11.679+0000 7eff1bad7640 1 --2- 172.21.15.169:0/2170260245 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7efef806ae40 0x7efef806d300 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7eff10036000 tx=0x7eff10004730 comp rx=0 tx=0).stop 2023-10-26T01:38:11.682 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:11.679+0000 7eff1bad7640 1 -- 172.21.15.169:0/2170260245 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff1414e790 msgr2=0x7eff141491c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:11.682 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:11.679+0000 7eff1bad7640 1 --2- 172.21.15.169:0/2170260245 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff1414e790 0x7eff141491c0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7eff0400e9c0 tx=0x7eff0400ee80 comp rx=0 tx=0).stop 2023-10-26T01:38:11.682 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:11.679+0000 7eff1bad7640 1 -- 172.21.15.169:0/2170260245 shutdown_connections 2023-10-26T01:38:11.682 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:11.679+0000 7eff1bad7640 1 --2- 172.21.15.169:0/2170260245 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7efef806ae40 0x7efef806d300 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:11.682 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:11.679+0000 7eff1bad7640 1 --2- 172.21.15.169:0/2170260245 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff1414e790 0x7eff141491c0 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:11.683 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:11.679+0000 7eff1bad7640 1 --2- 172.21.15.169:0/2170260245 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7eff14158850 0x7eff14149700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:11.683 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:11.679+0000 7eff1bad7640 1 --2- 172.21.15.169:0/2170260245 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff140cdc60 0x7eff14163c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:11.683 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:11.679+0000 7eff1bad7640 1 -- 172.21.15.169:0/2170260245 >> 172.21.15.169:0/2170260245 conn(0x7eff14069930 msgr2=0x7eff14150ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:11.683 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:11.680+0000 7eff1bad7640 1 -- 172.21.15.169:0/2170260245 shutdown_connections 2023-10-26T01:38:11.683 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:11.680+0000 7eff1bad7640 1 -- 172.21.15.169:0/2170260245 wait complete. 2023-10-26T01:38:11.700 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T01:30:32.200808+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '109', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T01:38:11.701 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T01:38:11.921 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:11.919+0000 7f966fe87640 1 -- 172.21.15.169:0/2919406566 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f96680cd760 msgr2=0x7f96680cdbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:11.921 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:11.919+0000 7f966fe87640 1 --2- 172.21.15.169:0/2919406566 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f96680cd760 0x7f96680cdbd0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f966806c3b0 tx=0x7f966002bdb0 comp rx=0 tx=0).stop 2023-10-26T01:38:11.921 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:11.919+0000 7f966fe87640 1 -- 172.21.15.169:0/2919406566 shutdown_connections 2023-10-26T01:38:11.922 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:11.919+0000 7f966fe87640 1 --2- 172.21.15.169:0/2919406566 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f96680cd760 0x7f96680cdbd0 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:11.922 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:11.919+0000 7f966fe87640 1 --2- 172.21.15.169:0/2919406566 >> v2:172.21.15.170:3300/0 conn(0x7f96680ab070 0x7f96680ac870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:11.922 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:11.919+0000 7f966fe87640 1 -- 172.21.15.169:0/2919406566 >> 172.21.15.169:0/2919406566 conn(0x7f9668069930 msgr2=0x7f9668069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:11.922 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:11.919+0000 7f966fe87640 1 -- 172.21.15.169:0/2919406566 shutdown_connections 2023-10-26T01:38:11.922 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:11.919+0000 7f966fe87640 1 -- 172.21.15.169:0/2919406566 wait complete. 2023-10-26T01:38:11.923 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:11.921+0000 7f966fe87640 1 Processor -- start 2023-10-26T01:38:11.923 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:11.922+0000 7f966fe87640 1 -- start start 2023-10-26T01:38:11.925 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:11.923+0000 7f966fe87640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96680ab070 0x7f9668147210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:11.925 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:11.923+0000 7f966fe87640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f96680cd760 0x7f9668149ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:11.925 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:11.923+0000 7f966fe87640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9668157020 0x7f9668070090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:11.925 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:11.923+0000 7f966fe87640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9668155340 con 0x7f96680cd760 2023-10-26T01:38:11.926 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:11.923+0000 7f966fe87640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96681534c0 con 0x7f96680ab070 2023-10-26T01:38:11.926 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:11.923+0000 7f966fe87640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9668153d60 con 0x7f9668157020 2023-10-26T01:38:11.926 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:11.923+0000 7f966d422640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f96680cd760 0x7f9668149ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:11.926 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:11.923+0000 7f966d422640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f96680cd760 0x7f9668149ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:55982/0 (socket says 172.21.15.169:55982) 2023-10-26T01:38:11.926 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:11.923+0000 7f966d422640 1 -- 172.21.15.169:0/3551743758 learned_addr learned my addr 172.21.15.169:0/3551743758 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:38:11.928 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:11.924+0000 7f966e424640 1 --2- 172.21.15.169:0/3551743758 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9668157020 0x7f9668070090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:11.928 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:11.924+0000 7f966dc23640 1 --2- 172.21.15.169:0/3551743758 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96680ab070 0x7f9668147210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:11.928 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:11.924+0000 7f966d422640 1 -- 172.21.15.169:0/3551743758 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9668157020 msgr2=0x7f9668070090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:11.929 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:11.924+0000 7f966d422640 1 --2- 172.21.15.169:0/3551743758 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9668157020 0x7f9668070090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:11.929 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:11.924+0000 7f966d422640 1 -- 172.21.15.169:0/3551743758 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96680ab070 msgr2=0x7f9668147210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:11.929 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:11.924+0000 7f966d422640 1 --2- 172.21.15.169:0/3551743758 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96680ab070 0x7f9668147210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:11.929 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:11.924+0000 7f966d422640 1 -- 172.21.15.169:0/3551743758 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9660009560 con 0x7f96680cd760 2023-10-26T01:38:11.929 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:11.924+0000 7f966dc23640 1 --2- 172.21.15.169:0/3551743758 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96680ab070 0x7f9668147210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:38:11.930 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:11.924+0000 7f966d422640 1 --2- 172.21.15.169:0/3551743758 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f96680cd760 0x7f9668149ef0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f965c00c410 tx=0x7f965c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:11.930 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:11.924+0000 7f96667fc640 1 -- 172.21.15.169:0/3551743758 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f965c0035f0 con 0x7f96680cd760 2023-10-26T01:38:11.930 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:11.924+0000 7f96667fc640 1 -- 172.21.15.169:0/3551743758 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f965c003750 con 0x7f96680cd760 2023-10-26T01:38:11.930 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:11.924+0000 7f966fe87640 1 -- 172.21.15.169:0/3551743758 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9668157500 con 0x7f96680cd760 2023-10-26T01:38:11.930 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:11.924+0000 7f96667fc640 1 -- 172.21.15.169:0/3551743758 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f965c003b60 con 0x7f96680cd760 2023-10-26T01:38:11.931 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:11.925+0000 7f966fe87640 1 -- 172.21.15.169:0/3551743758 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96681539f0 con 0x7f96680cd760 2023-10-26T01:38:11.931 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:11.926+0000 7f966fe87640 1 -- 172.21.15.169:0/3551743758 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9668066af0 con 0x7f96680cd760 2023-10-26T01:38:11.933 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:11.927+0000 7f96667fc640 1 -- 172.21.15.169:0/3551743758 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f965c003cc0 con 0x7f96680cd760 2023-10-26T01:38:11.934 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:11.928+0000 7f96667fc640 1 --2- 172.21.15.169:0/3551743758 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f965406ae40 0x7f965406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:11.934 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:11.928+0000 7f96667fc640 1 -- 172.21.15.169:0/3551743758 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f965c088b20 con 0x7f96680cd760 2023-10-26T01:38:11.934 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:11.928+0000 7f966dc23640 1 --2- 172.21.15.169:0/3551743758 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f965406ae40 0x7f965406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:11.934 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:11.929+0000 7f966dc23640 1 --2- 172.21.15.169:0/3551743758 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f965406ae40 0x7f965406d300 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f9658005c40 tx=0x7f9658005f90 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:11.935 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:11.932+0000 7f96667fc640 1 -- 172.21.15.169:0/3551743758 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f965c053ec0 con 0x7f96680cd760 2023-10-26T01:38:12.118 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:12.116+0000 7f966fe87640 1 -- 172.21.15.169:0/3551743758 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9668153ed0 con 0x7f96680cd760 2023-10-26T01:38:12.119 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:12.117+0000 7f96667fc640 1 -- 172.21.15.169:0/3551743758 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v126) v1 ==== 93+0+10 (secure 0 0 0) 0x7f965c017070 con 0x7f96680cd760 2023-10-26T01:38:12.119 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 1 2023-10-26T01:38:12.122 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:12.120+0000 7f966fe87640 1 -- 172.21.15.169:0/3551743758 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f965406ae40 msgr2=0x7f965406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:12.122 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:12.120+0000 7f966fe87640 1 --2- 172.21.15.169:0/3551743758 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f965406ae40 0x7f965406d300 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f9658005c40 tx=0x7f9658005f90 comp rx=0 tx=0).stop 2023-10-26T01:38:12.123 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:12.120+0000 7f966fe87640 1 -- 172.21.15.169:0/3551743758 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f96680cd760 msgr2=0x7f9668149ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:12.123 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:12.120+0000 7f966fe87640 1 --2- 172.21.15.169:0/3551743758 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f96680cd760 0x7f9668149ef0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f965c00c410 tx=0x7f965c00c8d0 comp rx=0 tx=0).stop 2023-10-26T01:38:12.123 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:12.120+0000 7f966fe87640 1 -- 172.21.15.169:0/3551743758 shutdown_connections 2023-10-26T01:38:12.123 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:12.120+0000 7f966fe87640 1 --2- 172.21.15.169:0/3551743758 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f965406ae40 0x7f965406d300 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:12.123 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:12.120+0000 7f966fe87640 1 --2- 172.21.15.169:0/3551743758 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f96680cd760 0x7f9668149ef0 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:12.124 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:12.120+0000 7f966fe87640 1 --2- 172.21.15.169:0/3551743758 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9668157020 0x7f9668070090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:12.124 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:12.120+0000 7f966fe87640 1 --2- 172.21.15.169:0/3551743758 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96680ab070 0x7f9668147210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:12.124 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:12.120+0000 7f966fe87640 1 -- 172.21.15.169:0/3551743758 >> 172.21.15.169:0/3551743758 conn(0x7f9668069930 msgr2=0x7f9668151570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:12.124 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:12.120+0000 7f966fe87640 1 -- 172.21.15.169:0/3551743758 shutdown_connections 2023-10-26T01:38:12.124 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:12.120+0000 7f966fe87640 1 -- 172.21.15.169:0/3551743758 wait complete. 2023-10-26T01:38:12.138 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_different_file' 2023-10-26T01:38:12.343 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:12.341+0000 7f9bcb59e640 1 -- 172.21.15.169:0/2201916188 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9bcc155860 msgr2=0x7f9bcc06b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:12.343 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:12.341+0000 7f9bcb59e640 1 --2- 172.21.15.169:0/2201916188 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9bcc155860 0x7f9bcc06b4d0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc005fd0 tx=0x7f9bbc02be20 comp rx=0 tx=0).stop 2023-10-26T01:38:12.344 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:12.341+0000 7f9bcb59e640 1 -- 172.21.15.169:0/2201916188 shutdown_connections 2023-10-26T01:38:12.344 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:12.341+0000 7f9bcb59e640 1 --2- 172.21.15.169:0/2201916188 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9bcc0c6980 0x7f9bcc06ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:12.344 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:12.341+0000 7f9bcb59e640 1 --2- 172.21.15.169:0/2201916188 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9bcc155860 0x7f9bcc06b4d0 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:12.344 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:12.341+0000 7f9bcb59e640 1 -- 172.21.15.169:0/2201916188 >> 172.21.15.169:0/2201916188 conn(0x7f9bcc0aedd0 msgr2=0x7f9bcc0af1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:12.344 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:12.341+0000 7f9bcb59e640 1 -- 172.21.15.169:0/2201916188 shutdown_connections 2023-10-26T01:38:12.345 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:12.341+0000 7f9bcb59e640 1 -- 172.21.15.169:0/2201916188 wait complete. 2023-10-26T01:38:12.345 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:12.343+0000 7f9bcb59e640 1 Processor -- start 2023-10-26T01:38:12.345 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:12.343+0000 7f9bcb59e640 1 -- start start 2023-10-26T01:38:12.347 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:12.344+0000 7f9bcb59e640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9bcc0c6980 0x7f9bcc0ab9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:12.347 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:12.344+0000 7f9bcb59e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9bcc155860 0x7f9bcc0c8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:12.348 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:12.344+0000 7f9bcb59e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9bcc158260 0x7f9bcc158710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:12.348 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:12.344+0000 7f9bcb59e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bcc152f90 con 0x7f9bcc158260 2023-10-26T01:38:12.348 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:12.344+0000 7f9bcb59e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bcc13f2d0 con 0x7f9bcc155860 2023-10-26T01:38:12.348 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:12.344+0000 7f9bcb59e640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bcc071390 con 0x7f9bcc0c6980 2023-10-26T01:38:12.348 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:12.344+0000 7f9bcad9d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9bcc158260 0x7f9bcc158710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:12.349 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:12.345+0000 7f9bcad9d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9bcc158260 0x7f9bcc158710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:56008/0 (socket says 172.21.15.169:56008) 2023-10-26T01:38:12.349 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:12.345+0000 7f9bcad9d640 1 -- 172.21.15.169:0/218452443 learned_addr learned my addr 172.21.15.169:0/218452443 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:38:12.349 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:12.345+0000 7f9bca59c640 1 --2- 172.21.15.169:0/218452443 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9bcc0c6980 0x7f9bcc0ab9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:12.349 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:12.345+0000 7f9bc3fff640 1 --2- 172.21.15.169:0/218452443 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9bcc155860 0x7f9bcc0c8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:12.349 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:12.345+0000 7f9bcad9d640 1 -- 172.21.15.169:0/218452443 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9bcc0c6980 msgr2=0x7f9bcc0ab9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:12.350 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:12.345+0000 7f9bcad9d640 1 --2- 172.21.15.169:0/218452443 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9bcc0c6980 0x7f9bcc0ab9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:12.350 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:12.345+0000 7f9bcad9d640 1 -- 172.21.15.169:0/218452443 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9bcc155860 msgr2=0x7f9bcc0c8fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:12.350 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:12.345+0000 7f9bcad9d640 1 --2- 172.21.15.169:0/218452443 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9bcc155860 0x7f9bcc0c8fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:12.350 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:12.345+0000 7f9bcad9d640 1 -- 172.21.15.169:0/218452443 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bbc009560 con 0x7f9bcc158260 2023-10-26T01:38:12.350 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:12.345+0000 7f9bcad9d640 1 --2- 172.21.15.169:0/218452443 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9bcc158260 0x7f9bcc158710 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f9bc400cad0 tx=0x7f9bc400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:12.351 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:12.346+0000 7f9bc2ffd640 1 -- 172.21.15.169:0/218452443 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc400ae20 con 0x7f9bcc158260 2023-10-26T01:38:12.351 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:12.346+0000 7f9bc2ffd640 1 -- 172.21.15.169:0/218452443 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9bc4004b20 con 0x7f9bcc158260 2023-10-26T01:38:12.351 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:12.346+0000 7f9bcb59e640 1 -- 172.21.15.169:0/218452443 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bcc071670 con 0x7f9bcc158260 2023-10-26T01:38:12.351 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:12.346+0000 7f9bc2ffd640 1 -- 172.21.15.169:0/218452443 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc4005020 con 0x7f9bcc158260 2023-10-26T01:38:12.351 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:12.346+0000 7f9bcb59e640 1 -- 172.21.15.169:0/218452443 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bcc157d40 con 0x7f9bcc158260 2023-10-26T01:38:12.352 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:12.347+0000 7f9bcb59e640 1 -- 172.21.15.169:0/218452443 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bcc0c70f0 con 0x7f9bcc158260 2023-10-26T01:38:12.352 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:12.348+0000 7f9bc2ffd640 1 -- 172.21.15.169:0/218452443 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f9bc4005180 con 0x7f9bcc158260 2023-10-26T01:38:12.353 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:12.349+0000 7f9bc2ffd640 1 --2- 172.21.15.169:0/218452443 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9bac06ae70 0x7f9bac06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:12.353 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:12.349+0000 7f9bc2ffd640 1 -- 172.21.15.169:0/218452443 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f9bc4089bc0 con 0x7f9bcc158260 2023-10-26T01:38:12.353 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:12.349+0000 7f9bca59c640 1 --2- 172.21.15.169:0/218452443 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9bac06ae70 0x7f9bac06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:12.353 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:12.350+0000 7f9bca59c640 1 --2- 172.21.15.169:0/218452443 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9bac06ae70 0x7f9bac06d330 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc035000 tx=0x7f9bbc004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:12.354 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:12.353+0000 7f9bc2ffd640 1 -- 172.21.15.169:0/218452443 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9bc4054f60 con 0x7f9bcc158260 2023-10-26T01:38:12.613 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:12.611+0000 7f9bcb59e640 1 -- 172.21.15.169:0/218452443 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_different_file"]} v 0) v1 -- 0x7f9bcc157f60 con 0x7f9bcc158260 2023-10-26T01:38:13.239 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:13.237+0000 7f9bc2ffd640 1 -- 172.21.15.169:0/218452443 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_different_file"]}]=0 v391) v1 ==== 143+0+0 (secure 0 0 0) 0x7f9bc401d070 con 0x7f9bcc158260 2023-10-26T01:38:13.243 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:13.242+0000 7f9bcb59e640 1 -- 172.21.15.169:0/218452443 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9bac06ae70 msgr2=0x7f9bac06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:13.244 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:13.242+0000 7f9bcb59e640 1 --2- 172.21.15.169:0/218452443 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9bac06ae70 0x7f9bac06d330 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc035000 tx=0x7f9bbc004730 comp rx=0 tx=0).stop 2023-10-26T01:38:13.244 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:13.242+0000 7f9bcb59e640 1 -- 172.21.15.169:0/218452443 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9bcc158260 msgr2=0x7f9bcc158710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:13.244 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:13.242+0000 7f9bcb59e640 1 --2- 172.21.15.169:0/218452443 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9bcc158260 0x7f9bcc158710 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f9bc400cad0 tx=0x7f9bc400cf90 comp rx=0 tx=0).stop 2023-10-26T01:38:13.245 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:13.242+0000 7f9bcb59e640 1 -- 172.21.15.169:0/218452443 shutdown_connections 2023-10-26T01:38:13.245 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:13.242+0000 7f9bcb59e640 1 --2- 172.21.15.169:0/218452443 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9bac06ae70 0x7f9bac06d330 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:13.245 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:13.242+0000 7f9bcb59e640 1 --2- 172.21.15.169:0/218452443 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9bcc158260 0x7f9bcc158710 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:13.245 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:13.242+0000 7f9bcb59e640 1 --2- 172.21.15.169:0/218452443 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9bcc155860 0x7f9bcc0c8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:13.246 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:13.242+0000 7f9bcb59e640 1 --2- 172.21.15.169:0/218452443 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9bcc0c6980 0x7f9bcc0ab9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:13.246 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:13.242+0000 7f9bcb59e640 1 -- 172.21.15.169:0/218452443 >> 172.21.15.169:0/218452443 conn(0x7f9bcc0aedd0 msgr2=0x7f9bcc0af1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:13.246 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:13.243+0000 7f9bcb59e640 1 -- 172.21.15.169:0/218452443 shutdown_connections 2023-10-26T01:38:13.246 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:13.243+0000 7f9bcb59e640 1 -- 172.21.15.169:0/218452443 wait complete. 2023-10-26T01:38:13.267 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2023-10-26T01:38:13.489 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:13.487+0000 7f0cc209b640 1 -- 172.21.15.169:0/2918561221 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0cbc0cd7e0 msgr2=0x7f0cbc0cdc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:13.490 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:13.487+0000 7f0cc209b640 1 --2- 172.21.15.169:0/2918561221 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0cbc0cd7e0 0x7f0cbc0cdc90 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc0b0570 tx=0x7f0ca402bd10 comp rx=0 tx=0).stop 2023-10-26T01:38:13.490 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:13.487+0000 7f0cc209b640 1 -- 172.21.15.169:0/2918561221 shutdown_connections 2023-10-26T01:38:13.490 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:13.487+0000 7f0cc209b640 1 --2- 172.21.15.169:0/2918561221 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0cbc15b4a0 0x7f0cbc15b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:13.490 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:13.487+0000 7f0cc209b640 1 --2- 172.21.15.169:0/2918561221 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0cbc0cd7e0 0x7f0cbc0cdc90 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:13.490 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:13.487+0000 7f0cc209b640 1 --2- 172.21.15.169:0/2918561221 >> v2:172.21.15.170:3300/0 conn(0x7f0cbc06ce00 0x7f0cbc155360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:13.491 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:13.487+0000 7f0cc209b640 1 -- 172.21.15.169:0/2918561221 >> 172.21.15.169:0/2918561221 conn(0x7f0cbc069930 msgr2=0x7f0cbc069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:13.491 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:13.487+0000 7f0cc209b640 1 -- 172.21.15.169:0/2918561221 shutdown_connections 2023-10-26T01:38:13.491 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:13.487+0000 7f0cc209b640 1 -- 172.21.15.169:0/2918561221 wait complete. 2023-10-26T01:38:13.493 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:13.489+0000 7f0cc209b640 1 Processor -- start 2023-10-26T01:38:13.493 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:13.489+0000 7f0cc209b640 1 -- start start 2023-10-26T01:38:13.493 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:13.490+0000 7f0cc209b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0cbc06ce00 0x7f0cbc14fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:13.493 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:13.490+0000 7f0cc209b640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0cbc0cd7e0 0x7f0cbc0c7550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:13.494 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:13.490+0000 7f0cc209b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0cbc15b4a0 0x7f0cbc0adac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:13.494 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:13.490+0000 7f0cc209b640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cbc0c99e0 con 0x7f0cbc15b4a0 2023-10-26T01:38:13.494 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:13.490+0000 7f0cc209b640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cbc070b60 con 0x7f0cbc06ce00 2023-10-26T01:38:13.494 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:13.490+0000 7f0cc209b640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cbc06ff50 con 0x7f0cbc0cd7e0 2023-10-26T01:38:13.494 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:13.490+0000 7f0cbbfff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0cbc15b4a0 0x7f0cbc0adac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:13.495 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:13.490+0000 7f0cbbfff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0cbc15b4a0 0x7f0cbc0adac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:56032/0 (socket says 172.21.15.169:56032) 2023-10-26T01:38:13.495 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:13.490+0000 7f0cbbfff640 1 -- 172.21.15.169:0/431830668 learned_addr learned my addr 172.21.15.169:0/431830668 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:38:13.495 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:13.491+0000 7f0cbb7fe640 1 --2- 172.21.15.169:0/431830668 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0cbc06ce00 0x7f0cbc14fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:13.495 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:13.491+0000 7f0cb3fff640 1 --2- 172.21.15.169:0/431830668 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0cbc0cd7e0 0x7f0cbc0c7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:13.496 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:13.491+0000 7f0cbbfff640 1 -- 172.21.15.169:0/431830668 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0cbc0cd7e0 msgr2=0x7f0cbc0c7550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:13.496 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:13.491+0000 7f0cbbfff640 1 --2- 172.21.15.169:0/431830668 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0cbc0cd7e0 0x7f0cbc0c7550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:13.496 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:13.491+0000 7f0cbbfff640 1 -- 172.21.15.169:0/431830668 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0cbc06ce00 msgr2=0x7f0cbc14fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:13.496 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:13.491+0000 7f0cbbfff640 1 --2- 172.21.15.169:0/431830668 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0cbc06ce00 0x7f0cbc14fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:13.496 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:13.491+0000 7f0cbbfff640 1 -- 172.21.15.169:0/431830668 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ca4009560 con 0x7f0cbc15b4a0 2023-10-26T01:38:13.497 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:13.491+0000 7f0cbbfff640 1 --2- 172.21.15.169:0/431830668 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0cbc15b4a0 0x7f0cbc0adac0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f0cb400e960 tx=0x7f0cb400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:13.497 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:13.491+0000 7f0cb8ff9640 1 -- 172.21.15.169:0/431830668 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cb4004d20 con 0x7f0cbc15b4a0 2023-10-26T01:38:13.497 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:13.491+0000 7f0cb8ff9640 1 -- 172.21.15.169:0/431830668 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0cb400cb20 con 0x7f0cbc15b4a0 2023-10-26T01:38:13.497 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:13.491+0000 7f0cb8ff9640 1 -- 172.21.15.169:0/431830668 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cb4005020 con 0x7f0cbc15b4a0 2023-10-26T01:38:13.497 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:13.491+0000 7f0cc209b640 1 -- 172.21.15.169:0/431830668 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cbc070230 con 0x7f0cbc15b4a0 2023-10-26T01:38:13.497 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:13.492+0000 7f0cc209b640 1 -- 172.21.15.169:0/431830668 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cbc0707f0 con 0x7f0cbc15b4a0 2023-10-26T01:38:13.499 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:13.493+0000 7f0cc209b640 1 -- 172.21.15.169:0/431830668 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cbc070cd0 con 0x7f0cbc15b4a0 2023-10-26T01:38:13.499 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:13.493+0000 7f0cb8ff9640 1 -- 172.21.15.169:0/431830668 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f0cb4005180 con 0x7f0cbc15b4a0 2023-10-26T01:38:13.499 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:13.494+0000 7f0cb8ff9640 1 --2- 172.21.15.169:0/431830668 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f0c9c06ae40 0x7f0c9c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:13.499 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:13.494+0000 7f0cb8ff9640 1 -- 172.21.15.169:0/431830668 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f0cb4089d00 con 0x7f0cbc15b4a0 2023-10-26T01:38:13.500 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:13.495+0000 7f0cbb7fe640 1 --2- 172.21.15.169:0/431830668 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f0c9c06ae40 0x7f0c9c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:13.500 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:13.496+0000 7f0cbb7fe640 1 --2- 172.21.15.169:0/431830668 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f0c9c06ae40 0x7f0c9c06d300 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f0cac005d80 tx=0x7f0cac005c90 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:13.502 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:13.499+0000 7f0cb8ff9640 1 -- 172.21.15.169:0/431830668 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0cb4055010 con 0x7f0cbc15b4a0 2023-10-26T01:38:13.693 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:13.691+0000 7f0cc209b640 1 -- 172.21.15.169:0/431830668 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "config dump"} v 0) v1 -- 0x7f0cbc06ad10 con 0x7f0cbc15b4a0 2023-10-26T01:38:13.694 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:13.692+0000 7f0cb8ff9640 1 -- 172.21.15.169:0/431830668 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v30) v1 ==== 59+0+1116 (secure 0 0 0) 0x7f0cb401f020 con 0x7f0cbc15b4a0 2023-10-26T01:38:13.694 INFO:teuthology.orchestra.run.smithi169.stdout:WHO MASK LEVEL OPTION VALUE RO 2023-10-26T01:38:13.695 INFO:teuthology.orchestra.run.smithi169.stdout:mon advanced mon_op_complaint_time 120 2023-10-26T01:38:13.695 INFO:teuthology.orchestra.run.smithi169.stdout:osd advanced osd_op_complaint_time 180.000000 2023-10-26T01:38:13.695 INFO:teuthology.orchestra.run.smithi169.stdout:mds advanced debug_mds 20/20 2023-10-26T01:38:13.696 INFO:teuthology.orchestra.run.smithi169.stdout:mds advanced debug_mds_balancer 20/20 2023-10-26T01:38:13.696 INFO:teuthology.orchestra.run.smithi169.stdout:mds advanced debug_ms 1/1 2023-10-26T01:38:13.696 INFO:teuthology.orchestra.run.smithi169.stdout:mds dev mds_debug_frag true 2023-10-26T01:38:13.696 INFO:teuthology.orchestra.run.smithi169.stdout:mds dev mds_debug_scatterstat true 2023-10-26T01:38:13.697 INFO:teuthology.orchestra.run.smithi169.stdout:mds advanced mds_op_complaint_time 180.000000 2023-10-26T01:38:13.697 INFO:teuthology.orchestra.run.smithi169.stdout:mds dev mds_verify_scatter true 2023-10-26T01:38:13.697 INFO:teuthology.orchestra.run.smithi169.stdout:mds advanced osd_op_complaint_time 180.000000 2023-10-26T01:38:13.697 INFO:teuthology.orchestra.run.smithi169.stdout:mds advanced rados_mon_op_timeout 900 2023-10-26T01:38:13.697 INFO:teuthology.orchestra.run.smithi169.stdout:mds advanced rados_osd_op_timeout 900 2023-10-26T01:38:13.698 INFO:teuthology.orchestra.run.smithi169.stdout:client advanced client_mount_timeout 600 2023-10-26T01:38:13.698 INFO:teuthology.orchestra.run.smithi169.stdout:client advanced debug_client 20/20 2023-10-26T01:38:13.698 INFO:teuthology.orchestra.run.smithi169.stdout:client advanced debug_ms 1/1 2023-10-26T01:38:13.698 INFO:teuthology.orchestra.run.smithi169.stdout:client advanced rados_mon_op_timeout 900 2023-10-26T01:38:13.698 INFO:teuthology.orchestra.run.smithi169.stdout:client advanced rados_osd_op_timeout 900 2023-10-26T01:38:13.700 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:13.694+0000 7f0cc209b640 1 -- 172.21.15.169:0/431830668 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f0c9c06ae40 msgr2=0x7f0c9c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:13.700 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:13.694+0000 7f0cc209b640 1 --2- 172.21.15.169:0/431830668 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f0c9c06ae40 0x7f0c9c06d300 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f0cac005d80 tx=0x7f0cac005c90 comp rx=0 tx=0).stop 2023-10-26T01:38:13.700 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:13.695+0000 7f0cc209b640 1 -- 172.21.15.169:0/431830668 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0cbc15b4a0 msgr2=0x7f0cbc0adac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:13.700 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:13.695+0000 7f0cc209b640 1 --2- 172.21.15.169:0/431830668 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0cbc15b4a0 0x7f0cbc0adac0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f0cb400e960 tx=0x7f0cb400ee20 comp rx=0 tx=0).stop 2023-10-26T01:38:13.701 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:13.695+0000 7f0cc209b640 1 -- 172.21.15.169:0/431830668 shutdown_connections 2023-10-26T01:38:13.701 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:13.695+0000 7f0cc209b640 1 --2- 172.21.15.169:0/431830668 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f0c9c06ae40 0x7f0c9c06d300 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:13.701 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:13.695+0000 7f0cc209b640 1 --2- 172.21.15.169:0/431830668 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0cbc15b4a0 0x7f0cbc0adac0 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:13.701 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:13.695+0000 7f0cc209b640 1 --2- 172.21.15.169:0/431830668 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0cbc0cd7e0 0x7f0cbc0c7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:13.701 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:13.695+0000 7f0cc209b640 1 --2- 172.21.15.169:0/431830668 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0cbc06ce00 0x7f0cbc14fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:13.702 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:13.695+0000 7f0cc209b640 1 -- 172.21.15.169:0/431830668 >> 172.21.15.169:0/431830668 conn(0x7f0cbc069930 msgr2=0x7f0cbc15a3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:13.702 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:13.695+0000 7f0cc209b640 1 -- 172.21.15.169:0/431830668 shutdown_connections 2023-10-26T01:38:13.702 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:13.695+0000 7f0cc209b640 1 -- 172.21.15.169:0/431830668 wait complete. 2023-10-26T01:38:13.714 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-26T01:38:13.934 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:13.932+0000 7fbef660e640 1 -- 172.21.15.169:0/1215083516 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbef00b3020 msgr2=0x7fbef00cf420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:13.935 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:13.932+0000 7fbef660e640 1 --2- 172.21.15.169:0/1215083516 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbef00b3020 0x7fbef00cf420 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fbed8005fd0 tx=0x7fbed802bd90 comp rx=0 tx=0).stop 2023-10-26T01:38:13.935 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:13.932+0000 7fbef660e640 1 -- 172.21.15.169:0/1215083516 shutdown_connections 2023-10-26T01:38:13.935 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:13.932+0000 7fbef660e640 1 --2- 172.21.15.169:0/1215083516 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbef00b3020 0x7fbef00cf420 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:13.936 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:13.932+0000 7fbef660e640 1 --2- 172.21.15.169:0/1215083516 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbef01594c0 0x7fbef006ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:13.936 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:13.932+0000 7fbef660e640 1 -- 172.21.15.169:0/1215083516 >> 172.21.15.169:0/1215083516 conn(0x7fbef0098a60 msgr2=0x7fbef00b4a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:13.936 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:13.933+0000 7fbef660e640 1 -- 172.21.15.169:0/1215083516 shutdown_connections 2023-10-26T01:38:13.936 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:13.933+0000 7fbef660e640 1 -- 172.21.15.169:0/1215083516 wait complete. 2023-10-26T01:38:13.937 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:13.934+0000 7fbef660e640 1 Processor -- start 2023-10-26T01:38:13.938 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:13.935+0000 7fbef660e640 1 -- start start 2023-10-26T01:38:13.938 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:13.935+0000 7fbef660e640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbef00b3020 0x7fbef0144e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:13.938 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:13.936+0000 7fbef660e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbef00b34a0 0x7fbef014f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:13.938 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:13.936+0000 7fbef660e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbef01594c0 0x7fbef00b2320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:13.938 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:13.936+0000 7fbef660e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef014fd30 con 0x7fbef00b34a0 2023-10-26T01:38:13.939 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:13.936+0000 7fbef660e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef0150010 con 0x7fbef01594c0 2023-10-26T01:38:13.939 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:13.936+0000 7fbef660e640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbef01502f0 con 0x7fbef00b3020 2023-10-26T01:38:13.939 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:13.936+0000 7fbeef7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbef00b34a0 0x7fbef014f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:13.939 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:13.936+0000 7fbeef7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbef00b34a0 0x7fbef014f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:56042/0 (socket says 172.21.15.169:56042) 2023-10-26T01:38:13.939 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:13.936+0000 7fbeef7fe640 1 -- 172.21.15.169:0/2292271038 learned_addr learned my addr 172.21.15.169:0/2292271038 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:38:13.941 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:13.936+0000 7fbeeffff640 1 --2- 172.21.15.169:0/2292271038 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbef00b3020 0x7fbef0144e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:13.941 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:13.936+0000 7fbeef7fe640 1 -- 172.21.15.169:0/2292271038 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbef00b3020 msgr2=0x7fbef0144e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:13.941 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:13.936+0000 7fbeef7fe640 1 --2- 172.21.15.169:0/2292271038 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbef00b3020 0x7fbef0144e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:13.941 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:13.936+0000 7fbef4bab640 1 --2- 172.21.15.169:0/2292271038 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbef01594c0 0x7fbef00b2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:13.942 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:13.936+0000 7fbeef7fe640 1 -- 172.21.15.169:0/2292271038 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbef01594c0 msgr2=0x7fbef00b2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:13.942 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:13.936+0000 7fbeef7fe640 1 --2- 172.21.15.169:0/2292271038 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbef01594c0 0x7fbef00b2320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:13.942 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:13.936+0000 7fbeef7fe640 1 -- 172.21.15.169:0/2292271038 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbed8009560 con 0x7fbef00b34a0 2023-10-26T01:38:13.942 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:13.937+0000 7fbeef7fe640 1 --2- 172.21.15.169:0/2292271038 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbef00b34a0 0x7fbef014f270 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7fbed8009850 tx=0x7fbed802db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:13.942 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:13.937+0000 7fbeecff9640 1 -- 172.21.15.169:0/2292271038 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed80035f0 con 0x7fbef00b34a0 2023-10-26T01:38:13.943 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:13.937+0000 7fbeecff9640 1 -- 172.21.15.169:0/2292271038 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fbed8003750 con 0x7fbef00b34a0 2023-10-26T01:38:13.943 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:13.937+0000 7fbef660e640 1 -- 172.21.15.169:0/2292271038 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbef01505d0 con 0x7fbef00b34a0 2023-10-26T01:38:13.943 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:13.937+0000 7fbeecff9640 1 -- 172.21.15.169:0/2292271038 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed8003b00 con 0x7fbef00b34a0 2023-10-26T01:38:13.943 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:13.937+0000 7fbef660e640 1 -- 172.21.15.169:0/2292271038 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbef0145340 con 0x7fbef00b34a0 2023-10-26T01:38:13.943 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:13.938+0000 7fbef660e640 1 -- 172.21.15.169:0/2292271038 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbef004db10 con 0x7fbef00b34a0 2023-10-26T01:38:13.944 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:13.939+0000 7fbeecff9640 1 -- 172.21.15.169:0/2292271038 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fbed8003c60 con 0x7fbef00b34a0 2023-10-26T01:38:13.944 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:13.940+0000 7fbeecff9640 1 --2- 172.21.15.169:0/2292271038 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbed006ae40 0x7fbed006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:13.944 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:13.940+0000 7fbeecff9640 1 -- 172.21.15.169:0/2292271038 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fbed80abfd0 con 0x7fbef00b34a0 2023-10-26T01:38:13.945 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:13.941+0000 7fbeeffff640 1 --2- 172.21.15.169:0/2292271038 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbed006ae40 0x7fbed006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:13.945 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:13.942+0000 7fbeeffff640 1 --2- 172.21.15.169:0/2292271038 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbed006ae40 0x7fbed006d300 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fbee0005fd0 tx=0x7fbee0005f40 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:13.946 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:13.945+0000 7fbeecff9640 1 -- 172.21.15.169:0/2292271038 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbed8077380 con 0x7fbef00b34a0 2023-10-26T01:38:14.136 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:14.134+0000 7fbef660e640 1 -- 172.21.15.169:0/2292271038 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fbef0154d80 con 0x7fbef00b34a0 2023-10-26T01:38:14.141 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:14.140+0000 7fbeecff9640 1 -- 172.21.15.169:0/2292271038 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v31)=0 v31) v1 ==== 122+0+0 (secure 0 0 0) 0x7fbed807a9a0 con 0x7fbef00b34a0 2023-10-26T01:38:14.145 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:14.142+0000 7fbef660e640 1 -- 172.21.15.169:0/2292271038 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbed006ae40 msgr2=0x7fbed006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:14.145 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:14.143+0000 7fbef660e640 1 --2- 172.21.15.169:0/2292271038 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbed006ae40 0x7fbed006d300 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fbee0005fd0 tx=0x7fbee0005f40 comp rx=0 tx=0).stop 2023-10-26T01:38:14.145 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:14.143+0000 7fbef660e640 1 -- 172.21.15.169:0/2292271038 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbef00b34a0 msgr2=0x7fbef014f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:14.145 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:14.143+0000 7fbef660e640 1 --2- 172.21.15.169:0/2292271038 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbef00b34a0 0x7fbef014f270 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7fbed8009850 tx=0x7fbed802db10 comp rx=0 tx=0).stop 2023-10-26T01:38:14.146 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:14.143+0000 7fbef660e640 1 -- 172.21.15.169:0/2292271038 shutdown_connections 2023-10-26T01:38:14.146 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:14.143+0000 7fbef660e640 1 --2- 172.21.15.169:0/2292271038 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbed006ae40 0x7fbed006d300 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:14.146 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:14.143+0000 7fbef660e640 1 --2- 172.21.15.169:0/2292271038 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbef00b34a0 0x7fbef014f270 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:14.146 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:14.143+0000 7fbef660e640 1 --2- 172.21.15.169:0/2292271038 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbef01594c0 0x7fbef00b2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:14.146 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:14.143+0000 7fbef660e640 1 --2- 172.21.15.169:0/2292271038 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbef00b3020 0x7fbef0144e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:14.147 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:14.143+0000 7fbef660e640 1 -- 172.21.15.169:0/2292271038 >> 172.21.15.169:0/2292271038 conn(0x7fbef0098a60 msgr2=0x7fbef015c150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:14.147 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:14.143+0000 7fbef660e640 1 -- 172.21.15.169:0/2292271038 shutdown_connections 2023-10-26T01:38:14.147 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:14.143+0000 7fbef660e640 1 -- 172.21.15.169:0/2292271038 wait complete. 2023-10-26T01:38:14.163 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-26T01:38:14.163 DEBUG:teuthology.orchestra.run.smithi169:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-26T01:38:14.278 DEBUG:teuthology.orchestra.run.smithi170:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-26T01:38:14.404 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-26T01:38:14.405 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:38:14.405 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:38:14.466 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054301,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-26T01:38:14.467 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-26T01:38:14.467 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:38:14.528 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054303,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-26T01:38:14.528 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:38:14.529 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:38:14.590 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054301,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-26T01:38:14.591 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-26T01:38:14.591 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:38:14.651 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054303,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-26T01:38:14.652 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-26T01:38:14.652 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T01:38:14.883 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:14.881+0000 7f7bb336c640 1 -- 172.21.15.169:0/3330406525 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b94003970 msgr2=0x7f7b94003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:14.883 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:14.881+0000 7f7bb336c640 1 --2- 172.21.15.169:0/3330406525 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b94003970 0x7f7b94003d90 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c009ed0 tx=0x7f7b940057f0 comp rx=0 tx=0).stop 2023-10-26T01:38:14.884 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:14.881+0000 7f7bb1909640 1 -- 172.21.15.169:0/3330406525 reap_dead start 2023-10-26T01:38:14.884 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:14.881+0000 7f7bb336c640 1 -- 172.21.15.169:0/3330406525 shutdown_connections 2023-10-26T01:38:14.884 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:14.881+0000 7f7bb336c640 1 -- 172.21.15.169:0/3330406525 >> 172.21.15.169:0/3330406525 conn(0x7f7bac069930 msgr2=0x7f7bac069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:14.884 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:14.881+0000 7f7bb336c640 1 -- 172.21.15.169:0/3330406525 shutdown_connections 2023-10-26T01:38:14.884 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:14.881+0000 7f7bb336c640 1 -- 172.21.15.169:0/3330406525 wait complete. 2023-10-26T01:38:14.885 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:14.883+0000 7f7bb336c640 1 Processor -- start 2023-10-26T01:38:14.885 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:14.883+0000 7f7bb336c640 1 -- start start 2023-10-26T01:38:14.887 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:14.884+0000 7f7bb336c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b94003970 0x7f7bac148c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:14.887 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:14.884+0000 7f7bb336c640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7bac159a60 0x7f7bac1579e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:14.887 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:14.884+0000 7f7bb336c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7bac159ee0 0x7f7bac157f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:14.888 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:14.884+0000 7f7bb336c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bac1491c0 con 0x7f7b94003970 2023-10-26T01:38:14.888 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:14.884+0000 7f7bb336c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bac15a4b0 con 0x7f7bac159ee0 2023-10-26T01:38:14.888 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:14.884+0000 7f7bb336c640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bac15a790 con 0x7f7bac159a60 2023-10-26T01:38:14.888 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:14.884+0000 7f7bb1108640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b94003970 0x7f7bac148c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:14.888 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:14.884+0000 7f7bb1108640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b94003970 0x7f7bac148c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:41832/0 (socket says 172.21.15.169:41832) 2023-10-26T01:38:14.889 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:14.884+0000 7f7bb1108640 1 -- 172.21.15.169:0/2704619703 learned_addr learned my addr 172.21.15.169:0/2704619703 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:38:14.889 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:14.885+0000 7f7bb1108640 1 -- 172.21.15.169:0/2704619703 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7bac159a60 msgr2=0x7f7bac1579e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:14.889 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:14.885+0000 7f7bb1108640 1 --2- 172.21.15.169:0/2704619703 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7bac159a60 0x7f7bac1579e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:14.889 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:14.885+0000 7f7bb1108640 1 -- 172.21.15.169:0/2704619703 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7bac159ee0 msgr2=0x7f7bac157f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:14.889 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:14.885+0000 7f7bb1108640 1 --2- 172.21.15.169:0/2704619703 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7bac159ee0 0x7f7bac157f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:14.890 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:14.885+0000 7f7bb1108640 1 -- 172.21.15.169:0/2704619703 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bac15aa70 con 0x7f7b94003970 2023-10-26T01:38:14.890 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:14.885+0000 7f7bb1108640 1 --2- 172.21.15.169:0/2704619703 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b94003970 0x7f7bac148c80 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c009e20 tx=0x7f7b9c030c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:14.890 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:14.885+0000 7f7ba9ffb640 1 -- 172.21.15.169:0/2704619703 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b9c040600 con 0x7f7b94003970 2023-10-26T01:38:14.890 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:14.885+0000 7f7ba9ffb640 1 -- 172.21.15.169:0/2704619703 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7b9c040850 con 0x7f7b94003970 2023-10-26T01:38:14.890 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:14.885+0000 7f7bb336c640 1 -- 172.21.15.169:0/2704619703 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bac15ad50 con 0x7f7b94003970 2023-10-26T01:38:14.891 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:14.885+0000 7f7ba9ffb640 1 -- 172.21.15.169:0/2704619703 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b9c040c60 con 0x7f7b94003970 2023-10-26T01:38:14.891 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:14.885+0000 7f7bb336c640 1 -- 172.21.15.169:0/2704619703 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bac15b550 con 0x7f7b94003970 2023-10-26T01:38:14.892 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:14.886+0000 7f7bb336c640 1 -- 172.21.15.169:0/2704619703 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bac15b6e0 con 0x7f7b94003970 2023-10-26T01:38:14.892 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:14.888+0000 7f7ba9ffb640 1 -- 172.21.15.169:0/2704619703 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f7b9c04a440 con 0x7f7b94003970 2023-10-26T01:38:14.892 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:14.888+0000 7f7ba9ffb640 1 --2- 172.21.15.169:0/2704619703 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7b9806ae70 0x7f7b9806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:14.892 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:14.889+0000 7f7ba9ffb640 1 -- 172.21.15.169:0/2704619703 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f7b9c0abce0 con 0x7f7b94003970 2023-10-26T01:38:14.893 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:14.889+0000 7f7bb0907640 1 --2- 172.21.15.169:0/2704619703 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7b9806ae70 0x7f7b9806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:14.893 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:14.891+0000 7f7bb0907640 1 --2- 172.21.15.169:0/2704619703 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7b9806ae70 0x7f7b9806d330 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f7ba0000f60 tx=0x7f7ba0001560 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:14.894 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:14.893+0000 7f7ba9ffb640 1 -- 172.21.15.169:0/2704619703 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7bac15b6e0 con 0x7f7b94003970 2023-10-26T01:38:15.121 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:15.119+0000 7f7bb336c640 1 -- 172.21.15.169:0/2704619703 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7bac15b6e0 con 0x7f7b94003970 2023-10-26T01:38:15.122 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:15.120+0000 7f7ba9ffb640 1 -- 172.21.15.169:0/2704619703 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+3711 (secure 0 0 0) 0x7f7bac15b6e0 con 0x7f7b94003970 2023-10-26T01:38:15.123 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:38:15.123 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":61,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5387,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/1186563921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1186563921},{"type":"v1","addr":"172.21.15.170:6835","nonce":1186563921}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":54},{"gid":5390,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/3702036693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":3702036693},{"type":"v1","addr":"172.21.15.170:6837","nonce":3702036693}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":54},{"gid":7332,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/3193888686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":3193888686},{"type":"v1","addr":"172.21.15.169:6833","nonce":3193888686}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":7686,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/2590997737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":2590997737},{"type":"v1","addr":"172.21.15.169:6835","nonce":2590997737}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61}],"filesystems":[]} 2023-10-26T01:38:15.123 INFO:teuthology.orchestra.run.smithi169.stderr:dumped fsmap epoch 61 2023-10-26T01:38:15.125 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:15.123+0000 7f7bb336c640 1 -- 172.21.15.169:0/2704619703 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7b9806ae70 msgr2=0x7f7b9806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:15.126 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:15.123+0000 7f7bb336c640 1 --2- 172.21.15.169:0/2704619703 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7b9806ae70 0x7f7b9806d330 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f7ba0000f60 tx=0x7f7ba0001560 comp rx=0 tx=0).stop 2023-10-26T01:38:15.126 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:15.123+0000 7f7bb336c640 1 -- 172.21.15.169:0/2704619703 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b94003970 msgr2=0x7f7bac148c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:15.126 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:15.123+0000 7f7bb336c640 1 --2- 172.21.15.169:0/2704619703 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b94003970 0x7f7bac148c80 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c009e20 tx=0x7f7b9c030c40 comp rx=0 tx=0).stop 2023-10-26T01:38:15.126 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:15.123+0000 7f7bb336c640 1 -- 172.21.15.169:0/2704619703 shutdown_connections 2023-10-26T01:38:15.127 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:15.123+0000 7f7bb336c640 1 --2- 172.21.15.169:0/2704619703 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7b9806ae70 0x7f7b9806d330 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:15.127 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:15.123+0000 7f7bb336c640 1 --2- 172.21.15.169:0/2704619703 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7bac159ee0 0x7f7bac157f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:15.127 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:15.123+0000 7f7bb336c640 1 --2- 172.21.15.169:0/2704619703 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7bac159a60 0x7f7bac1579e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:15.127 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:15.123+0000 7f7bb336c640 1 --2- 172.21.15.169:0/2704619703 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b94003970 0x7f7bac148c80 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:15.127 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:15.123+0000 7f7bb336c640 1 -- 172.21.15.169:0/2704619703 >> 172.21.15.169:0/2704619703 conn(0x7f7bac069930 msgr2=0x7f7bac0aeec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:15.127 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:15.124+0000 7f7bb336c640 1 -- 172.21.15.169:0/2704619703 shutdown_connections 2023-10-26T01:38:15.128 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:15.124+0000 7f7bb336c640 1 -- 172.21.15.169:0/2704619703 wait complete. 2023-10-26T01:38:15.145 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-26T01:38:15.145 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-26T01:38:15.146 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-26T01:38:15.146 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T01:38:15.147 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-26T01:38:15.147 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-26T01:38:15.147 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-26T01:38:15.147 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T01:38:15.148 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-26T01:38:15.148 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-26T01:38:15.148 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-26T01:38:15.149 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T01:38:15.149 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-26T01:38:15.149 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-26T01:38:15.149 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-26T01:38:15.150 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T01:38:15.161 INFO:tasks.ceph.mds.a:Stopped 2023-10-26T01:38:15.162 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-26T01:38:15.166 INFO:tasks.ceph.mds.c:Stopped 2023-10-26T01:38:15.166 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-26T01:38:15.170 INFO:tasks.ceph.mds.d:Stopped 2023-10-26T01:38:15.170 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-26T01:38:15.173 INFO:tasks.ceph.mds.b:Stopped 2023-10-26T01:38:15.173 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-26T01:38:15.213 INFO:tasks.ceph.mds.a:Started 2023-10-26T01:38:15.214 DEBUG:teuthology.parallel:result is None 2023-10-26T01:38:15.221 INFO:tasks.ceph.mds.c:Started 2023-10-26T01:38:15.222 DEBUG:teuthology.parallel:result is None 2023-10-26T01:38:15.223 INFO:tasks.ceph.mds.d:Started 2023-10-26T01:38:15.223 DEBUG:teuthology.parallel:result is None 2023-10-26T01:38:15.229 INFO:tasks.ceph.mds.b:Started 2023-10-26T01:38:15.229 DEBUG:teuthology.parallel:result is None 2023-10-26T01:38:15.230 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-26T01:38:15.370 INFO:tasks.ceph.mds.a.smithi169.stdout:starting mds.a at 2023-10-26T01:38:15.374 INFO:tasks.ceph.mds.c.smithi169.stdout:starting mds.c at 2023-10-26T01:38:15.393 INFO:tasks.ceph.mds.d.smithi170.stdout:starting mds.d at 2023-10-26T01:38:15.394 INFO:tasks.ceph.mds.b.smithi170.stdout:starting mds.b at 2023-10-26T01:38:15.421 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:15.418+0000 7f3c25dd7640 1 -- 172.21.15.169:0/517857312 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3c2015b680 msgr2=0x7f3c200acbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:15.421 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:15.418+0000 7f3c25dd7640 1 --2- 172.21.15.169:0/517857312 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3c2015b680 0x7f3c200acbc0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f3c200ceb80 tx=0x7f3c1002be40 comp rx=0 tx=0).stop 2023-10-26T01:38:15.421 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:15.419+0000 7f3c25dd7640 1 -- 172.21.15.169:0/517857312 shutdown_connections 2023-10-26T01:38:15.421 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:15.419+0000 7f3c25dd7640 1 --2- 172.21.15.169:0/517857312 >> v2:172.21.15.170:3300/0 conn(0x7f3c200716b0 0x7f3c200b0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:15.422 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:15.419+0000 7f3c25dd7640 1 --2- 172.21.15.169:0/517857312 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3c2015b680 0x7f3c200acbc0 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:15.422 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:15.419+0000 7f3c25dd7640 1 -- 172.21.15.169:0/517857312 >> 172.21.15.169:0/517857312 conn(0x7f3c200af8a0 msgr2=0x7f3c200afc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:15.422 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:15.419+0000 7f3c25dd7640 1 -- 172.21.15.169:0/517857312 shutdown_connections 2023-10-26T01:38:15.422 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:15.419+0000 7f3c25dd7640 1 -- 172.21.15.169:0/517857312 wait complete. 2023-10-26T01:38:15.424 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:15.420+0000 7f3c25dd7640 1 Processor -- start 2023-10-26T01:38:15.424 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:15.421+0000 7f3c25dd7640 1 -- start start 2023-10-26T01:38:15.424 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:15.422+0000 7f3c25dd7640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3c2006fa00 0x7f3c2014f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:15.425 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:15.422+0000 7f3c25dd7640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3c200716b0 0x7f3c2014ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:15.425 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:15.422+0000 7f3c25dd7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c2015b680 0x7f3c200ae430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:15.425 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:15.422+0000 7f3c25dd7640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c2015bd30 con 0x7f3c2006fa00 2023-10-26T01:38:15.425 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:15.422+0000 7f3c25dd7640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c2015c010 con 0x7f3c2015b680 2023-10-26T01:38:15.425 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:15.422+0000 7f3c25dd7640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c2015c2f0 con 0x7f3c200716b0 2023-10-26T01:38:15.426 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:15.422+0000 7f3c1f7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3c2006fa00 0x7f3c2014f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:15.426 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:15.422+0000 7f3c1f7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3c2006fa00 0x7f3c2014f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:41882/0 (socket says 172.21.15.169:41882) 2023-10-26T01:38:15.426 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:15.422+0000 7f3c1f7fe640 1 -- 172.21.15.169:0/2710005528 learned_addr learned my addr 172.21.15.169:0/2710005528 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:38:15.426 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:15.422+0000 7f3c1ffff640 1 --2- 172.21.15.169:0/2710005528 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c2015b680 0x7f3c200ae430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:15.426 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:15.422+0000 7f3c17fff640 1 --2- 172.21.15.169:0/2710005528 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3c200716b0 0x7f3c2014ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:15.426 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:15.422+0000 7f3c1f7fe640 1 -- 172.21.15.169:0/2710005528 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3c200716b0 msgr2=0x7f3c2014ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:15.427 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:15.422+0000 7f3c1f7fe640 1 --2- 172.21.15.169:0/2710005528 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3c200716b0 0x7f3c2014ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:15.427 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:15.422+0000 7f3c1f7fe640 1 -- 172.21.15.169:0/2710005528 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c2015b680 msgr2=0x7f3c200ae430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:15.427 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:15.422+0000 7f3c1f7fe640 1 --2- 172.21.15.169:0/2710005528 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c2015b680 0x7f3c200ae430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:15.427 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:15.422+0000 7f3c1f7fe640 1 -- 172.21.15.169:0/2710005528 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c10009560 con 0x7f3c2006fa00 2023-10-26T01:38:15.429 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:15.423+0000 7f3c1f7fe640 1 --2- 172.21.15.169:0/2710005528 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3c2006fa00 0x7f3c2014f5b0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f3c10009850 tx=0x7f3c1002db40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:15.429 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:15.423+0000 7f3c1cff9640 1 -- 172.21.15.169:0/2710005528 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c100038c0 con 0x7f3c2006fa00 2023-10-26T01:38:15.429 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:15.423+0000 7f3c1cff9640 1 -- 172.21.15.169:0/2710005528 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3c10003b10 con 0x7f3c2006fa00 2023-10-26T01:38:15.429 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:15.423+0000 7f3c1cff9640 1 -- 172.21.15.169:0/2710005528 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c1003e450 con 0x7f3c2006fa00 2023-10-26T01:38:15.429 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:15.423+0000 7f3c25dd7640 1 -- 172.21.15.169:0/2710005528 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c200ad280 con 0x7f3c2006fa00 2023-10-26T01:38:15.430 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:15.423+0000 7f3c25dd7640 1 -- 172.21.15.169:0/2710005528 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c2015c8b0 con 0x7f3c2006fa00 2023-10-26T01:38:15.430 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:15.424+0000 7f3c25dd7640 1 -- 172.21.15.169:0/2710005528 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c2004db10 con 0x7f3c2006fa00 2023-10-26T01:38:15.430 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:15.425+0000 7f3c1cff9640 1 -- 172.21.15.169:0/2710005528 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f3c1003e5b0 con 0x7f3c2006fa00 2023-10-26T01:38:15.430 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:15.426+0000 7f3c1cff9640 1 --2- 172.21.15.169:0/2710005528 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3c0006ae40 0x7f3c0006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:15.430 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:15.426+0000 7f3c1cff9640 1 -- 172.21.15.169:0/2710005528 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f3c100abe80 con 0x7f3c2006fa00 2023-10-26T01:38:15.431 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:15.427+0000 7f3c17fff640 1 --2- 172.21.15.169:0/2710005528 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3c0006ae40 0x7f3c0006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:15.431 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:15.427+0000 7f3c17fff640 1 --2- 172.21.15.169:0/2710005528 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3c0006ae40 0x7f3c0006d300 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f3c08005fd0 tx=0x7f3c08005850 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:15.432 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:15.431+0000 7f3c1cff9640 1 -- 172.21.15.169:0/2710005528 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3c10077220 con 0x7f3c2006fa00 2023-10-26T01:38:15.636 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:15.635+0000 7f3c25dd7640 1 -- 172.21.15.169:0/2710005528 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f3c20070850 con 0x7f3c2006fa00 2023-10-26T01:38:16.270 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:16.268+0000 7f3c1cff9640 1 -- 172.21.15.169:0/2710005528 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v128) v1 ==== 97+0+0 (secure 0 0 0) 0x7f3c10042080 con 0x7f3c2006fa00 2023-10-26T01:38:16.271 INFO:teuthology.orchestra.run.smithi169.stderr: removed all blocklist entries 2023-10-26T01:38:16.274 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:16.272+0000 7f3c25dd7640 1 -- 172.21.15.169:0/2710005528 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3c0006ae40 msgr2=0x7f3c0006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:16.274 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:16.272+0000 7f3c25dd7640 1 --2- 172.21.15.169:0/2710005528 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3c0006ae40 0x7f3c0006d300 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f3c08005fd0 tx=0x7f3c08005850 comp rx=0 tx=0).stop 2023-10-26T01:38:16.274 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:16.272+0000 7f3c25dd7640 1 -- 172.21.15.169:0/2710005528 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3c2006fa00 msgr2=0x7f3c2014f5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:16.275 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:16.272+0000 7f3c25dd7640 1 --2- 172.21.15.169:0/2710005528 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3c2006fa00 0x7f3c2014f5b0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f3c10009850 tx=0x7f3c1002db40 comp rx=0 tx=0).stop 2023-10-26T01:38:16.275 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:16.272+0000 7f3c25dd7640 1 -- 172.21.15.169:0/2710005528 shutdown_connections 2023-10-26T01:38:16.275 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:16.272+0000 7f3c25dd7640 1 --2- 172.21.15.169:0/2710005528 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3c0006ae40 0x7f3c0006d300 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:16.275 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:16.272+0000 7f3c25dd7640 1 --2- 172.21.15.169:0/2710005528 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c2015b680 0x7f3c200ae430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:16.275 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:16.272+0000 7f3c25dd7640 1 --2- 172.21.15.169:0/2710005528 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3c200716b0 0x7f3c2014ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:16.276 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:16.272+0000 7f3c25dd7640 1 --2- 172.21.15.169:0/2710005528 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3c2006fa00 0x7f3c2014f5b0 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:16.276 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:16.272+0000 7f3c25dd7640 1 -- 172.21.15.169:0/2710005528 >> 172.21.15.169:0/2710005528 conn(0x7f3c200af8a0 msgr2=0x7f3c201538b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:16.276 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:16.272+0000 7f3c25dd7640 1 -- 172.21.15.169:0/2710005528 shutdown_connections 2023-10-26T01:38:16.276 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:16.272+0000 7f3c25dd7640 1 -- 172.21.15.169:0/2710005528 wait complete. 2023-10-26T01:38:16.296 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-26T01:38:16.503 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:16.501+0000 7f9f028d7640 1 -- 172.21.15.169:0/4153236792 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9efc0cd720 msgr2=0x7f9efc0cdb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:16.503 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:16.501+0000 7f9f028d7640 1 --2- 172.21.15.169:0/4153236792 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9efc0cd720 0x7f9efc0cdb90 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f9efc0acd20 tx=0x7f9ef402c1d0 comp rx=0 tx=0).stop 2023-10-26T01:38:16.504 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:16.501+0000 7f9f028d7640 1 -- 172.21.15.169:0/4153236792 shutdown_connections 2023-10-26T01:38:16.504 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:16.501+0000 7f9f028d7640 1 --2- 172.21.15.169:0/4153236792 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9efc0cd720 0x7f9efc0cdb90 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:16.504 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:16.501+0000 7f9f028d7640 1 --2- 172.21.15.169:0/4153236792 >> v2:172.21.15.170:3300/0 conn(0x7f9efc158850 0x7f9efc14e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:16.504 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:16.501+0000 7f9f028d7640 1 -- 172.21.15.169:0/4153236792 >> 172.21.15.169:0/4153236792 conn(0x7f9efc069930 msgr2=0x7f9efc069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:16.504 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:16.501+0000 7f9f028d7640 1 -- 172.21.15.169:0/4153236792 shutdown_connections 2023-10-26T01:38:16.505 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:16.501+0000 7f9f028d7640 1 -- 172.21.15.169:0/4153236792 wait complete. 2023-10-26T01:38:16.507 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:16.503+0000 7f9f028d7640 1 Processor -- start 2023-10-26T01:38:16.507 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:16.503+0000 7f9f028d7640 1 -- start start 2023-10-26T01:38:16.507 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:16.504+0000 7f9f028d7640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9efc06ce20 0x7f9efc0ae900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:16.508 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:16.504+0000 7f9f028d7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9efc0cd720 0x7f9efc1f5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:16.508 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:16.504+0000 7f9f028d7640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9efc158850 0x7f9efc1562d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:16.508 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:16.504+0000 7f9f028d7640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9efc0ae470 con 0x7f9efc06ce20 2023-10-26T01:38:16.508 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:16.504+0000 7f9f028d7640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9efc1f56c0 con 0x7f9efc0cd720 2023-10-26T01:38:16.508 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:16.504+0000 7f9f028d7640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9efc155d10 con 0x7f9efc158850 2023-10-26T01:38:16.509 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:16.504+0000 7f9efbfff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9efc06ce20 0x7f9efc0ae900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:16.509 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:16.504+0000 7f9efbfff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9efc06ce20 0x7f9efc0ae900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:41906/0 (socket says 172.21.15.169:41906) 2023-10-26T01:38:16.509 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:16.504+0000 7f9efbfff640 1 -- 172.21.15.169:0/2372132416 learned_addr learned my addr 172.21.15.169:0/2372132416 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:38:16.509 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:16.505+0000 7f9efb7fe640 1 --2- 172.21.15.169:0/2372132416 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9efc0cd720 0x7f9efc1f5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:16.510 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:16.505+0000 7f9efbfff640 1 -- 172.21.15.169:0/2372132416 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9efc158850 msgr2=0x7f9efc1562d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:16.510 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:16.505+0000 7f9efbfff640 1 --2- 172.21.15.169:0/2372132416 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9efc158850 0x7f9efc1562d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:16.510 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:16.505+0000 7f9efbfff640 1 -- 172.21.15.169:0/2372132416 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9efc0cd720 msgr2=0x7f9efc1f5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:16.510 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:16.505+0000 7f9efbfff640 1 --2- 172.21.15.169:0/2372132416 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9efc0cd720 0x7f9efc1f5180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:16.510 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:16.505+0000 7f9efbfff640 1 -- 172.21.15.169:0/2372132416 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ef4009900 con 0x7f9efc06ce20 2023-10-26T01:38:16.511 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:16.505+0000 7f9efbfff640 1 --2- 172.21.15.169:0/2372132416 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9efc06ce20 0x7f9efc0ae900 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f9eec00c430 tx=0x7f9eec00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:16.511 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:16.505+0000 7f9ef8ff9640 1 -- 172.21.15.169:0/2372132416 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eec011790 con 0x7f9efc06ce20 2023-10-26T01:38:16.511 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:16.505+0000 7f9ef8ff9640 1 -- 172.21.15.169:0/2372132416 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9eec0119e0 con 0x7f9efc06ce20 2023-10-26T01:38:16.511 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:16.505+0000 7f9ef8ff9640 1 -- 172.21.15.169:0/2372132416 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eec011e30 con 0x7f9efc06ce20 2023-10-26T01:38:16.512 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:16.505+0000 7f9f028d7640 1 -- 172.21.15.169:0/2372132416 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9efc155ff0 con 0x7f9efc06ce20 2023-10-26T01:38:16.512 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:16.505+0000 7f9f028d7640 1 -- 172.21.15.169:0/2372132416 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9efc161c30 con 0x7f9efc06ce20 2023-10-26T01:38:16.513 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:16.507+0000 7f9f028d7640 1 -- 172.21.15.169:0/2372132416 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9efc04db10 con 0x7f9efc06ce20 2023-10-26T01:38:16.513 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:16.507+0000 7f9ef8ff9640 1 -- 172.21.15.169:0/2372132416 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f9eec01c460 con 0x7f9efc06ce20 2023-10-26T01:38:16.513 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:16.508+0000 7f9ef8ff9640 1 --2- 172.21.15.169:0/2372132416 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9ee806ae40 0x7f9ee806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:16.514 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:16.508+0000 7f9ef8ff9640 1 -- 172.21.15.169:0/2372132416 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f9eec089570 con 0x7f9efc06ce20 2023-10-26T01:38:16.514 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:16.509+0000 7f9efb7fe640 1 --2- 172.21.15.169:0/2372132416 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9ee806ae40 0x7f9ee806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:16.514 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:16.510+0000 7f9efb7fe640 1 --2- 172.21.15.169:0/2372132416 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9ee806ae40 0x7f9ee806d300 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f9ef00058c0 tx=0x7f9ef0005850 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:16.515 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:16.513+0000 7f9ef8ff9640 1 -- 172.21.15.169:0/2372132416 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9eec054970 con 0x7f9efc06ce20 2023-10-26T01:38:16.765 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:16.763+0000 7f9f028d7640 1 -- 172.21.15.169:0/2372132416 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f9efc0aa4d0 con 0x7f9efc06ce20 2023-10-26T01:38:16.766 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:16.765+0000 7f9ef8ff9640 1 -- 172.21.15.169:0/2372132416 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f9eec057f90 con 0x7f9efc06ce20 2023-10-26T01:38:16.767 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:38:16.767 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2023-10-26T01:38:16.767 INFO:teuthology.orchestra.run.smithi169.stdout: "auth_dump": [ 2023-10-26T01:38:16.767 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:16.767 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "mds.a", 2023-10-26T01:38:16.768 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAgwTllWtUfAhAA28+12VwWEF/BUEu+ErSVyA==", 2023-10-26T01:38:16.768 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:38:16.768 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow", 2023-10-26T01:38:16.768 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow *", 2023-10-26T01:38:16.768 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow *", 2023-10-26T01:38:16.769 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:38:16.769 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:16.769 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:16.769 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:16.769 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "mds.b", 2023-10-26T01:38:16.770 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAgwTlleA8lHxAA+jMcMo+QnYAGllpKJ6eYgQ==", 2023-10-26T01:38:16.770 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:38:16.770 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow", 2023-10-26T01:38:16.770 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow *", 2023-10-26T01:38:16.770 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow *", 2023-10-26T01:38:16.771 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:38:16.771 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:16.771 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:16.771 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:16.771 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "mds.c", 2023-10-26T01:38:16.772 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAgwTllaCCqEBAAokeHvwOyp/Jv2x0yHDwysw==", 2023-10-26T01:38:16.772 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:38:16.772 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow", 2023-10-26T01:38:16.772 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow *", 2023-10-26T01:38:16.772 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow *", 2023-10-26T01:38:16.773 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:38:16.773 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:16.773 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:16.773 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:16.773 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "mds.d", 2023-10-26T01:38:16.774 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAgwTll51JQLhAA4Wzso4zQj+Htv6cXqvwRmg==", 2023-10-26T01:38:16.774 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:38:16.774 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow", 2023-10-26T01:38:16.774 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow *", 2023-10-26T01:38:16.774 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow *", 2023-10-26T01:38:16.774 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:38:16.775 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:16.775 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:16.775 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:16.775 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "osd.0", 2023-10-26T01:38:16.775 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAlwTll8RZ+HRAAt/sBK+Mi91YmTyQYbhKHLg==", 2023-10-26T01:38:16.776 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:38:16.776 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow profile osd", 2023-10-26T01:38:16.776 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow profile osd", 2023-10-26T01:38:16.776 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:38:16.776 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:16.777 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:16.777 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:16.777 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "osd.1", 2023-10-26T01:38:16.777 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAlwTllLXwWLRAAuHB2lJ+oDJou/IJBtsYY4Q==", 2023-10-26T01:38:16.777 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:38:16.778 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow profile osd", 2023-10-26T01:38:16.778 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow profile osd", 2023-10-26T01:38:16.778 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:38:16.778 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:16.778 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:16.778 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:16.779 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "osd.2", 2023-10-26T01:38:16.779 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAmwTllj1nSARAA5Jl34Q0X7a71ySd4wz17lw==", 2023-10-26T01:38:16.779 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:38:16.779 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow profile osd", 2023-10-26T01:38:16.779 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow profile osd", 2023-10-26T01:38:16.780 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:38:16.780 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:16.780 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:16.780 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:16.780 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "osd.3", 2023-10-26T01:38:16.781 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAmwTllC8MeEhAAfGxRVSwD+YbU/U5aeenO3w==", 2023-10-26T01:38:16.781 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:38:16.781 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow profile osd", 2023-10-26T01:38:16.781 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow profile osd", 2023-10-26T01:38:16.781 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:38:16.781 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:16.782 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:16.782 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:16.782 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "osd.4", 2023-10-26T01:38:16.782 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAowTll4Md6DhAAuXfadUNAdouMP28MiX/D3Q==", 2023-10-26T01:38:16.782 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:38:16.783 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow profile osd", 2023-10-26T01:38:16.783 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow profile osd", 2023-10-26T01:38:16.783 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:38:16.783 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:16.783 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:16.784 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:16.784 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "osd.5", 2023-10-26T01:38:16.784 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAowTllDdViHRAAec9QyhZCsjJ2rgGhhcjwpQ==", 2023-10-26T01:38:16.784 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:38:16.784 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow profile osd", 2023-10-26T01:38:16.785 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow profile osd", 2023-10-26T01:38:16.785 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:38:16.785 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:16.785 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:16.785 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:16.785 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "osd.6", 2023-10-26T01:38:16.786 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAowTll23X5KxAAsmQvG0yq7ilkVG5fNiaCUg==", 2023-10-26T01:38:16.786 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:38:16.786 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow profile osd", 2023-10-26T01:38:16.786 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow profile osd", 2023-10-26T01:38:16.786 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:38:16.787 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:16.787 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:16.787 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:16.787 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "osd.7", 2023-10-26T01:38:16.787 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAowTllhcy/OhAAY5Yjheh76ibOVREqN6WBpQ==", 2023-10-26T01:38:16.788 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:38:16.788 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow profile osd", 2023-10-26T01:38:16.788 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow profile osd", 2023-10-26T01:38:16.788 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:38:16.788 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:16.789 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:16.789 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:16.789 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "client.0", 2023-10-26T01:38:16.789 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAgwTllKC7gOhAAPsvxGcuUA8/p8xqbUgGteA==", 2023-10-26T01:38:16.789 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:38:16.789 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow", 2023-10-26T01:38:16.790 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow r", 2023-10-26T01:38:16.790 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-26T01:38:16.790 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:16.790 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:16.790 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:16.791 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "client.1", 2023-10-26T01:38:16.791 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAhwTllhxb6ExAAeYTsurWpnjohWlrNIlLpsw==", 2023-10-26T01:38:16.791 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:38:16.791 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow", 2023-10-26T01:38:16.791 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow r", 2023-10-26T01:38:16.792 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow rw", 2023-10-26T01:38:16.792 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow rwx" 2023-10-26T01:38:16.792 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:16.792 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:16.792 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:16.792 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "client.2", 2023-10-26T01:38:16.793 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAhwTllLWMoChAAsUHzBc29GDqmEQAFnl7dHg==", 2023-10-26T01:38:16.793 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:38:16.793 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow", 2023-10-26T01:38:16.793 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow r", 2023-10-26T01:38:16.793 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow rw", 2023-10-26T01:38:16.794 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow rwx" 2023-10-26T01:38:16.794 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:16.794 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:16.794 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:16.794 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "client.3", 2023-10-26T01:38:16.795 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAhwTll875HHRAAGbwikoR/hDU8P/bj0OPWAw==", 2023-10-26T01:38:16.795 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:38:16.795 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow", 2023-10-26T01:38:16.795 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow r", 2023-10-26T01:38:16.795 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow rw", 2023-10-26T01:38:16.795 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow rwx" 2023-10-26T01:38:16.796 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:16.796 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:16.796 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:16.796 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "client.admin", 2023-10-26T01:38:16.796 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAewTll2yddMhAAO8bnXNevDGe/KEeNNa36pQ==", 2023-10-26T01:38:16.797 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:38:16.797 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow *", 2023-10-26T01:38:16.797 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow *", 2023-10-26T01:38:16.797 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow *", 2023-10-26T01:38:16.797 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:38:16.797 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:16.798 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:16.798 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:16.798 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "mgr.x", 2023-10-26T01:38:16.798 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAfwTllPu3FMxAAV8d5oRFXAeeZH7e1xzPEDg==", 2023-10-26T01:38:16.798 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:38:16.799 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow *", 2023-10-26T01:38:16.799 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow profile mgr", 2023-10-26T01:38:16.799 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:38:16.799 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:16.799 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:16.800 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:16.800 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "mgr.y", 2023-10-26T01:38:16.800 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAfwTllZc+DJxAAsZ2TWn8H7Kw/lNTc2qyKzA==", 2023-10-26T01:38:16.800 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:38:16.800 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow *", 2023-10-26T01:38:16.801 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow profile mgr", 2023-10-26T01:38:16.801 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:38:16.801 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:16.801 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:16.801 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:16.802 INFO:teuthology.orchestra.run.smithi169.stdout:} 2023-10-26T01:38:16.803 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:16.767+0000 7f9f028d7640 1 -- 172.21.15.169:0/2372132416 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9ee806ae40 msgr2=0x7f9ee806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:16.803 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:16.767+0000 7f9f028d7640 1 --2- 172.21.15.169:0/2372132416 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9ee806ae40 0x7f9ee806d300 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f9ef00058c0 tx=0x7f9ef0005850 comp rx=0 tx=0).stop 2023-10-26T01:38:16.804 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:16.767+0000 7f9f028d7640 1 -- 172.21.15.169:0/2372132416 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9efc06ce20 msgr2=0x7f9efc0ae900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:16.804 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:16.767+0000 7f9f028d7640 1 --2- 172.21.15.169:0/2372132416 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9efc06ce20 0x7f9efc0ae900 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f9eec00c430 tx=0x7f9eec00c8f0 comp rx=0 tx=0).stop 2023-10-26T01:38:16.804 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:16.767+0000 7f9f028d7640 1 -- 172.21.15.169:0/2372132416 shutdown_connections 2023-10-26T01:38:16.804 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:16.767+0000 7f9f028d7640 1 --2- 172.21.15.169:0/2372132416 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9ee806ae40 0x7f9ee806d300 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:16.805 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:16.768+0000 7f9f028d7640 1 --2- 172.21.15.169:0/2372132416 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9efc158850 0x7f9efc1562d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:16.805 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:16.768+0000 7f9f028d7640 1 --2- 172.21.15.169:0/2372132416 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9efc0cd720 0x7f9efc1f5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:16.805 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:16.768+0000 7f9f028d7640 1 --2- 172.21.15.169:0/2372132416 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9efc06ce20 0x7f9efc0ae900 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:16.805 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:16.768+0000 7f9f028d7640 1 -- 172.21.15.169:0/2372132416 >> 172.21.15.169:0/2372132416 conn(0x7f9efc069930 msgr2=0x7f9efc142130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:16.805 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:16.768+0000 7f9f028d7640 1 -- 172.21.15.169:0/2372132416 shutdown_connections 2023-10-26T01:38:16.806 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:16.768+0000 7f9f028d7640 1 -- 172.21.15.169:0/2372132416 wait complete. 2023-10-26T01:38:16.806 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:38:17.025 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.023+0000 7fcc36aa9640 1 -- 172.21.15.169:0/4081919080 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcc30158850 msgr2=0x7fcc300cd720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:17.025 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.023+0000 7fcc36aa9640 1 --2- 172.21.15.169:0/4081919080 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcc30158850 0x7fcc300cd720 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c009830 tx=0x7fcc1c02bcd0 comp rx=0 tx=0).stop 2023-10-26T01:38:17.025 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.023+0000 7fcc36aa9640 1 -- 172.21.15.169:0/4081919080 shutdown_connections 2023-10-26T01:38:17.025 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.023+0000 7fcc36aa9640 1 --2- 172.21.15.169:0/4081919080 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcc30158850 0x7fcc300cd720 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:17.026 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.023+0000 7fcc36aa9640 1 --2- 172.21.15.169:0/4081919080 >> v2:172.21.15.170:3300/0 conn(0x7fcc3014e790 0x7fcc300aab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:17.026 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.023+0000 7fcc36aa9640 1 -- 172.21.15.169:0/4081919080 >> 172.21.15.169:0/4081919080 conn(0x7fcc30069930 msgr2=0x7fcc30069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:17.026 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.023+0000 7fcc36aa9640 1 -- 172.21.15.169:0/4081919080 shutdown_connections 2023-10-26T01:38:17.026 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.023+0000 7fcc36aa9640 1 -- 172.21.15.169:0/4081919080 wait complete. 2023-10-26T01:38:17.027 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.025+0000 7fcc36aa9640 1 Processor -- start 2023-10-26T01:38:17.028 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.025+0000 7fcc36aa9640 1 -- start start 2023-10-26T01:38:17.028 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.026+0000 7fcc36aa9640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc300cdc60 0x7fcc300ccba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:17.028 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.026+0000 7fcc36aa9640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc3014e790 0x7fcc30096910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:17.028 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.026+0000 7fcc36aa9640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcc30158850 0x7fcc30150f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:17.028 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.026+0000 7fcc36aa9640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc300676b0 con 0x7fcc3014e790 2023-10-26T01:38:17.029 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.026+0000 7fcc36aa9640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc30154c70 con 0x7fcc300cdc60 2023-10-26T01:38:17.029 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.026+0000 7fcc36aa9640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc30154ec0 con 0x7fcc30158850 2023-10-26T01:38:17.029 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.026+0000 7fcc2ffff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc3014e790 0x7fcc30096910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:17.029 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.026+0000 7fcc2ffff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc3014e790 0x7fcc30096910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:41916/0 (socket says 172.21.15.169:41916) 2023-10-26T01:38:17.029 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.026+0000 7fcc2ffff640 1 -- 172.21.15.169:0/3280857783 learned_addr learned my addr 172.21.15.169:0/3280857783 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:38:17.031 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.026+0000 7fcc34845640 1 --2- 172.21.15.169:0/3280857783 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc300cdc60 0x7fcc300ccba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:17.031 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.026+0000 7fcc35046640 1 --2- 172.21.15.169:0/3280857783 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcc30158850 0x7fcc30150f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:17.031 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.026+0000 7fcc2ffff640 1 -- 172.21.15.169:0/3280857783 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcc30158850 msgr2=0x7fcc30150f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:17.031 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.026+0000 7fcc2ffff640 1 --2- 172.21.15.169:0/3280857783 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcc30158850 0x7fcc30150f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:17.032 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.026+0000 7fcc2ffff640 1 -- 172.21.15.169:0/3280857783 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc300cdc60 msgr2=0x7fcc300ccba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:17.032 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.026+0000 7fcc2ffff640 1 --2- 172.21.15.169:0/3280857783 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc300cdc60 0x7fcc300ccba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:17.032 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.026+0000 7fcc2ffff640 1 -- 172.21.15.169:0/3280857783 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc1c009490 con 0x7fcc3014e790 2023-10-26T01:38:17.032 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.027+0000 7fcc35046640 1 --2- 172.21.15.169:0/3280857783 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcc30158850 0x7fcc30150f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:38:17.033 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.027+0000 7fcc2ffff640 1 --2- 172.21.15.169:0/3280857783 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc3014e790 0x7fcc30096910 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c009950 tx=0x7fcc1c0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:17.033 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.027+0000 7fcc2d7fa640 1 -- 172.21.15.169:0/3280857783 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c003ea0 con 0x7fcc3014e790 2023-10-26T01:38:17.033 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.027+0000 7fcc2d7fa640 1 -- 172.21.15.169:0/3280857783 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fcc1c034440 con 0x7fcc3014e790 2023-10-26T01:38:17.033 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.027+0000 7fcc36aa9640 1 -- 172.21.15.169:0/3280857783 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc301551a0 con 0x7fcc3014e790 2023-10-26T01:38:17.033 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.027+0000 7fcc2d7fa640 1 -- 172.21.15.169:0/3280857783 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c034850 con 0x7fcc3014e790 2023-10-26T01:38:17.033 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.027+0000 7fcc36aa9640 1 -- 172.21.15.169:0/3280857783 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc301527a0 con 0x7fcc3014e790 2023-10-26T01:38:17.034 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.028+0000 7fcc36aa9640 1 -- 172.21.15.169:0/3280857783 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc301553e0 con 0x7fcc3014e790 2023-10-26T01:38:17.034 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.029+0000 7fcc2d7fa640 1 -- 172.21.15.169:0/3280857783 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fcc1c03b040 con 0x7fcc3014e790 2023-10-26T01:38:17.035 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.031+0000 7fcc2d7fa640 1 --2- 172.21.15.169:0/3280857783 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcc1406ae70 0x7fcc1406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:17.035 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.031+0000 7fcc2d7fa640 1 -- 172.21.15.169:0/3280857783 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fcc1c0ab840 con 0x7fcc3014e790 2023-10-26T01:38:17.035 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.032+0000 7fcc34845640 1 --2- 172.21.15.169:0/3280857783 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcc1406ae70 0x7fcc1406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:17.035 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.033+0000 7fcc34845640 1 --2- 172.21.15.169:0/3280857783 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcc1406ae70 0x7fcc1406d330 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fcc24005fd0 tx=0x7fcc24009260 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:17.037 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.035+0000 7fcc2d7fa640 1 -- 172.21.15.169:0/3280857783 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcc1c076ba0 con 0x7fcc3014e790 2023-10-26T01:38:17.228 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.226+0000 7fcc36aa9640 1 -- 172.21.15.169:0/3280857783 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcc301522d0 con 0x7fcc3014e790 2023-10-26T01:38:17.230 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.228+0000 7fcc2d7fa640 1 -- 172.21.15.169:0/3280857783 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v130) v1 ==== 74+0+12916 (secure 0 0 0) 0x7fcc1c041090 con 0x7fcc3014e790 2023-10-26T01:38:17.230 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:38:17.231 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":130,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:38:16.282393+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:38:17.232 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.230+0000 7fcc36aa9640 1 -- 172.21.15.169:0/3280857783 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcc1406ae70 msgr2=0x7fcc1406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:17.233 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.230+0000 7fcc36aa9640 1 --2- 172.21.15.169:0/3280857783 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcc1406ae70 0x7fcc1406d330 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fcc24005fd0 tx=0x7fcc24009260 comp rx=0 tx=0).stop 2023-10-26T01:38:17.233 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.230+0000 7fcc36aa9640 1 -- 172.21.15.169:0/3280857783 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc3014e790 msgr2=0x7fcc30096910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:17.233 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.230+0000 7fcc36aa9640 1 --2- 172.21.15.169:0/3280857783 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc3014e790 0x7fcc30096910 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c009950 tx=0x7fcc1c0038a0 comp rx=0 tx=0).stop 2023-10-26T01:38:17.233 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.231+0000 7fcc36aa9640 1 -- 172.21.15.169:0/3280857783 shutdown_connections 2023-10-26T01:38:17.233 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.231+0000 7fcc36aa9640 1 --2- 172.21.15.169:0/3280857783 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcc1406ae70 0x7fcc1406d330 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:17.234 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.231+0000 7fcc36aa9640 1 --2- 172.21.15.169:0/3280857783 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc3014e790 0x7fcc30096910 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:17.234 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.231+0000 7fcc36aa9640 1 --2- 172.21.15.169:0/3280857783 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcc30158850 0x7fcc30150f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:17.234 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.231+0000 7fcc36aa9640 1 --2- 172.21.15.169:0/3280857783 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc300cdc60 0x7fcc300ccba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:17.234 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.231+0000 7fcc36aa9640 1 -- 172.21.15.169:0/3280857783 >> 172.21.15.169:0/3280857783 conn(0x7fcc30069930 msgr2=0x7fcc300ae700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:17.234 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.231+0000 7fcc36aa9640 1 -- 172.21.15.169:0/3280857783 shutdown_connections 2023-10-26T01:38:17.235 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.231+0000 7fcc36aa9640 1 -- 172.21.15.169:0/3280857783 wait complete. 2023-10-26T01:38:17.252 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T01:30:32.200808+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '109', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T01:38:17.253 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T01:38:17.470 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.468+0000 7fc4b9f05640 1 -- 172.21.15.169:0/842431609 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc49c0047c0 msgr2=0x7fc49c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:17.471 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.468+0000 7fc4b9f05640 1 --2- 172.21.15.169:0/842431609 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc49c0047c0 0x7fc49c004c40 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7fc4a8001680 tx=0x7fc49c0057f0 comp rx=0 tx=0).stop 2023-10-26T01:38:17.471 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.468+0000 7fc4b3fff640 1 -- 172.21.15.169:0/842431609 reap_dead start 2023-10-26T01:38:17.471 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.468+0000 7fc4b9f05640 1 -- 172.21.15.169:0/842431609 shutdown_connections 2023-10-26T01:38:17.471 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.468+0000 7fc4b9f05640 1 -- 172.21.15.169:0/842431609 >> 172.21.15.169:0/842431609 conn(0x7fc4b4069930 msgr2=0x7fc4b4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:17.471 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.468+0000 7fc4b9f05640 1 -- 172.21.15.169:0/842431609 shutdown_connections 2023-10-26T01:38:17.472 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.468+0000 7fc4b9f05640 1 -- 172.21.15.169:0/842431609 wait complete. 2023-10-26T01:38:17.472 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.470+0000 7fc4b9f05640 1 Processor -- start 2023-10-26T01:38:17.472 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.470+0000 7fc4b9f05640 1 -- start start 2023-10-26T01:38:17.474 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.471+0000 7fc4b9f05640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc49c0047c0 0x7fc4b40ae1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:17.474 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.471+0000 7fc4b9f05640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc4b41de850 0x7fc4b4149510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:17.475 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.471+0000 7fc4b9f05640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc4b40afa30 0x7fc4b4070e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:17.475 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.471+0000 7fc4b9f05640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b4149a50 con 0x7fc4b40afa30 2023-10-26T01:38:17.475 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.471+0000 7fc4b9f05640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b4151330 con 0x7fc4b41de850 2023-10-26T01:38:17.475 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.471+0000 7fc4b9f05640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b4151610 con 0x7fc49c0047c0 2023-10-26T01:38:17.475 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.471+0000 7fc4b3fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc4b40afa30 0x7fc4b4070e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:17.475 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.471+0000 7fc4b3fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc4b40afa30 0x7fc4b4070e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:41930/0 (socket says 172.21.15.169:41930) 2023-10-26T01:38:17.476 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.471+0000 7fc4b3fff640 1 -- 172.21.15.169:0/73157715 learned_addr learned my addr 172.21.15.169:0/73157715 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:38:17.476 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.472+0000 7fc4b37fe640 1 --2- 172.21.15.169:0/73157715 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc49c0047c0 0x7fc4b40ae1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:17.476 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.472+0000 7fc4b2ffd640 1 --2- 172.21.15.169:0/73157715 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc4b41de850 0x7fc4b4149510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:17.476 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.472+0000 7fc4b3fff640 1 -- 172.21.15.169:0/73157715 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc49c0047c0 msgr2=0x7fc4b40ae1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:17.477 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.472+0000 7fc4b3fff640 1 --2- 172.21.15.169:0/73157715 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc49c0047c0 0x7fc4b40ae1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:17.477 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.472+0000 7fc4b3fff640 1 -- 172.21.15.169:0/73157715 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc4b41de850 msgr2=0x7fc4b4149510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:17.477 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.472+0000 7fc4b3fff640 1 --2- 172.21.15.169:0/73157715 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc4b41de850 0x7fc4b4149510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:17.477 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.472+0000 7fc4b3fff640 1 -- 172.21.15.169:0/73157715 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4b41518f0 con 0x7fc4b40afa30 2023-10-26T01:38:17.477 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.472+0000 7fc4b37fe640 1 --2- 172.21.15.169:0/73157715 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc49c0047c0 0x7fc4b40ae1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:38:17.477 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.472+0000 7fc4b2ffd640 1 --2- 172.21.15.169:0/73157715 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc4b41de850 0x7fc4b4149510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:38:17.478 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.472+0000 7fc4b3fff640 1 --2- 172.21.15.169:0/73157715 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc4b40afa30 0x7fc4b4070e90 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7fc4b40acf20 tx=0x7fc4ac00fe60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:17.478 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.472+0000 7fc49bfff640 1 -- 172.21.15.169:0/73157715 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4ac00dda0 con 0x7fc4b40afa30 2023-10-26T01:38:17.478 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.472+0000 7fc49bfff640 1 -- 172.21.15.169:0/73157715 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc4ac011440 con 0x7fc4b40afa30 2023-10-26T01:38:17.478 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.472+0000 7fc4b9f05640 1 -- 172.21.15.169:0/73157715 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4b4151bd0 con 0x7fc4b40afa30 2023-10-26T01:38:17.478 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.472+0000 7fc49bfff640 1 -- 172.21.15.169:0/73157715 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4ac011850 con 0x7fc4b40afa30 2023-10-26T01:38:17.479 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.473+0000 7fc4b9f05640 1 -- 172.21.15.169:0/73157715 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4b41decb0 con 0x7fc4b40afa30 2023-10-26T01:38:17.480 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.474+0000 7fc4b9f05640 1 -- 172.21.15.169:0/73157715 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4b4066af0 con 0x7fc4b40afa30 2023-10-26T01:38:17.480 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.475+0000 7fc49bfff640 1 -- 172.21.15.169:0/73157715 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fc4ac0119b0 con 0x7fc4b40afa30 2023-10-26T01:38:17.480 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.476+0000 7fc49bfff640 1 --2- 172.21.15.169:0/73157715 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc4a006ae40 0x7fc4a006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:17.480 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.476+0000 7fc49bfff640 1 -- 172.21.15.169:0/73157715 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fc4ac089fb0 con 0x7fc4b40afa30 2023-10-26T01:38:17.480 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.476+0000 7fc4b37fe640 1 --2- 172.21.15.169:0/73157715 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc4a006ae40 0x7fc4a006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:17.481 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.477+0000 7fc4b37fe640 1 --2- 172.21.15.169:0/73157715 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc4a006ae40 0x7fc4a006d300 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fc4a40069f0 tx=0x7fc4a4001500 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:17.481 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.480+0000 7fc49bfff640 1 -- 172.21.15.169:0/73157715 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc4ac0553a0 con 0x7fc4b40afa30 2023-10-26T01:38:17.670 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.668+0000 7fc4b9f05640 1 -- 172.21.15.169:0/73157715 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc4b41520f0 con 0x7fc4b40afa30 2023-10-26T01:38:17.671 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.669+0000 7fc49bfff640 1 -- 172.21.15.169:0/73157715 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v130) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc4b41520f0 con 0x7fc4b40afa30 2023-10-26T01:38:17.671 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 1 2023-10-26T01:38:17.674 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.672+0000 7fc4b9f05640 1 -- 172.21.15.169:0/73157715 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc4a006ae40 msgr2=0x7fc4a006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:17.674 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.672+0000 7fc4b9f05640 1 --2- 172.21.15.169:0/73157715 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc4a006ae40 0x7fc4a006d300 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fc4a40069f0 tx=0x7fc4a4001500 comp rx=0 tx=0).stop 2023-10-26T01:38:17.674 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.672+0000 7fc4b9f05640 1 -- 172.21.15.169:0/73157715 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc4b40afa30 msgr2=0x7fc4b4070e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:17.675 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.672+0000 7fc4b9f05640 1 --2- 172.21.15.169:0/73157715 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc4b40afa30 0x7fc4b4070e90 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7fc4b40acf20 tx=0x7fc4ac00fe60 comp rx=0 tx=0).stop 2023-10-26T01:38:17.675 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.672+0000 7fc4b9f05640 1 -- 172.21.15.169:0/73157715 shutdown_connections 2023-10-26T01:38:17.675 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.672+0000 7fc4b9f05640 1 --2- 172.21.15.169:0/73157715 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc4a006ae40 0x7fc4a006d300 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:17.675 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.672+0000 7fc4b9f05640 1 --2- 172.21.15.169:0/73157715 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc4b40afa30 0x7fc4b4070e90 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:17.676 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.672+0000 7fc4b9f05640 1 --2- 172.21.15.169:0/73157715 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc4b41de850 0x7fc4b4149510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:17.676 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.672+0000 7fc4b9f05640 1 --2- 172.21.15.169:0/73157715 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc49c0047c0 0x7fc4b40ae1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:17.676 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.672+0000 7fc4b9f05640 1 -- 172.21.15.169:0/73157715 >> 172.21.15.169:0/73157715 conn(0x7fc4b4069930 msgr2=0x7fc4b4154e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:17.676 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.672+0000 7fc4b9f05640 1 -- 172.21.15.169:0/73157715 shutdown_connections 2023-10-26T01:38:17.677 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.672+0000 7fc4b9f05640 1 -- 172.21.15.169:0/73157715 wait complete. 2023-10-26T01:38:17.692 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-26T01:38:17.919 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.917+0000 7f2a09e26640 1 -- 172.21.15.169:0/1211232820 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a040ac930 msgr2=0x7f2a040cd6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:17.919 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.917+0000 7f2a09e26640 1 --2- 172.21.15.169:0/1211232820 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a040ac930 0x7f2a040cd6f0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f29ec005fd0 tx=0x7f29ec02bcf0 comp rx=0 tx=0).stop 2023-10-26T01:38:17.920 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.917+0000 7f2a09e26640 1 -- 172.21.15.169:0/1211232820 shutdown_connections 2023-10-26T01:38:17.920 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.917+0000 7f2a09e26640 1 --2- 172.21.15.169:0/1211232820 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2a040cdc30 0x7f2a040cbac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:17.920 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.917+0000 7f2a09e26640 1 --2- 172.21.15.169:0/1211232820 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a040ac930 0x7f2a040cd6f0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:17.920 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.917+0000 7f2a09e26640 1 -- 172.21.15.169:0/1211232820 >> 172.21.15.169:0/1211232820 conn(0x7f2a04069930 msgr2=0x7f2a04069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:17.920 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.917+0000 7f2a09e26640 1 -- 172.21.15.169:0/1211232820 shutdown_connections 2023-10-26T01:38:17.921 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.917+0000 7f2a09e26640 1 -- 172.21.15.169:0/1211232820 wait complete. 2023-10-26T01:38:17.921 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.919+0000 7f2a09e26640 1 Processor -- start 2023-10-26T01:38:17.922 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.920+0000 7f2a09e26640 1 -- start start 2023-10-26T01:38:17.923 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.921+0000 7f2a09e26640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a040ac930 0x7f2a040af460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:17.923 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.921+0000 7f2a09e26640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2a040cdc30 0x7f2a040af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:17.924 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.921+0000 7f2a09e26640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a041592d0 0x7f2a0415a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:17.924 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.921+0000 7f2a09e26640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a041513c0 con 0x7f2a041592d0 2023-10-26T01:38:17.924 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.921+0000 7f2a09e26640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a0415bd70 con 0x7f2a040ac930 2023-10-26T01:38:17.924 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.921+0000 7f2a09e26640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a041f3320 con 0x7f2a040cdc30 2023-10-26T01:38:17.924 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.921+0000 7f2a03fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a041592d0 0x7f2a0415a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:17.925 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.921+0000 7f2a03fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a041592d0 0x7f2a0415a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:41936/0 (socket says 172.21.15.169:41936) 2023-10-26T01:38:17.925 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.921+0000 7f2a03fff640 1 -- 172.21.15.169:0/3550317602 learned_addr learned my addr 172.21.15.169:0/3550317602 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:38:17.925 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.922+0000 7f2a037fe640 1 --2- 172.21.15.169:0/3550317602 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a040ac930 0x7f2a040af460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:17.925 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.922+0000 7f29fbfff640 1 --2- 172.21.15.169:0/3550317602 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2a040cdc30 0x7f2a040af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:17.926 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.922+0000 7f2a03fff640 1 -- 172.21.15.169:0/3550317602 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2a040cdc30 msgr2=0x7f2a040af9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:17.926 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.922+0000 7f2a03fff640 1 --2- 172.21.15.169:0/3550317602 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2a040cdc30 0x7f2a040af9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:17.926 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.922+0000 7f2a03fff640 1 -- 172.21.15.169:0/3550317602 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a040ac930 msgr2=0x7f2a040af460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:17.926 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.922+0000 7f2a03fff640 1 --2- 172.21.15.169:0/3550317602 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a040ac930 0x7f2a040af460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:17.926 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.922+0000 7f2a03fff640 1 -- 172.21.15.169:0/3550317602 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29ec009560 con 0x7f2a041592d0 2023-10-26T01:38:17.928 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.922+0000 7f2a03fff640 1 --2- 172.21.15.169:0/3550317602 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a041592d0 0x7f2a0415a8f0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f29fc00eb70 tx=0x7f29fc00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:17.928 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.922+0000 7f2a00ff9640 1 -- 172.21.15.169:0/3550317602 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29fc004d20 con 0x7f2a041592d0 2023-10-26T01:38:17.928 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.923+0000 7f2a00ff9640 1 -- 172.21.15.169:0/3550317602 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f29fc00ced0 con 0x7f2a041592d0 2023-10-26T01:38:17.928 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.923+0000 7f2a09e26640 1 -- 172.21.15.169:0/3550317602 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a0415b4d0 con 0x7f2a041592d0 2023-10-26T01:38:17.928 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.923+0000 7f2a00ff9640 1 -- 172.21.15.169:0/3550317602 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29fc005280 con 0x7f2a041592d0 2023-10-26T01:38:17.929 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.923+0000 7f2a09e26640 1 -- 172.21.15.169:0/3550317602 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a0415ba90 con 0x7f2a041592d0 2023-10-26T01:38:17.929 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.924+0000 7f2a09e26640 1 -- 172.21.15.169:0/3550317602 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a0404db10 con 0x7f2a041592d0 2023-10-26T01:38:17.929 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.925+0000 7f2a00ff9640 1 -- 172.21.15.169:0/3550317602 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f29fc0053e0 con 0x7f2a041592d0 2023-10-26T01:38:17.929 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.926+0000 7f2a00ff9640 1 --2- 172.21.15.169:0/3550317602 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f29e406ae40 0x7f29e406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:17.929 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.926+0000 7f2a00ff9640 1 -- 172.21.15.169:0/3550317602 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f29fc014070 con 0x7f2a041592d0 2023-10-26T01:38:17.930 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.926+0000 7f2a037fe640 1 --2- 172.21.15.169:0/3550317602 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f29e406ae40 0x7f29e406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:17.930 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.928+0000 7f2a037fe640 1 --2- 172.21.15.169:0/3550317602 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f29e406ae40 0x7f29e406d300 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f29f4000f60 tx=0x7f29f4009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:17.931 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:17.930+0000 7f2a00ff9640 1 -- 172.21.15.169:0/3550317602 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f29fc055440 con 0x7f2a041592d0 2023-10-26T01:38:18.126 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:18.124+0000 7f2a09e26640 1 -- 172.21.15.169:0/3550317602 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f2a04151500 con 0x7f2a041592d0 2023-10-26T01:38:18.127 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:18.125+0000 7f2a00ff9640 1 -- 172.21.15.169:0/3550317602 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v130) v1 ==== 84+0+65 (secure 0 0 0) 0x7f29fc058a60 con 0x7f2a041592d0 2023-10-26T01:38:18.127 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:38:18.128 INFO:teuthology.orchestra.run.smithi169.stdout:[ 2023-10-26T01:38:18.128 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:18.128 INFO:teuthology.orchestra.run.smithi169.stdout: "poolnum": 1, 2023-10-26T01:38:18.128 INFO:teuthology.orchestra.run.smithi169.stdout: "poolname": ".mgr" 2023-10-26T01:38:18.128 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:18.129 INFO:teuthology.orchestra.run.smithi169.stdout:] 2023-10-26T01:38:18.130 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:18.128+0000 7f2a09e26640 1 -- 172.21.15.169:0/3550317602 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f29e406ae40 msgr2=0x7f29e406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:18.130 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:18.128+0000 7f2a09e26640 1 --2- 172.21.15.169:0/3550317602 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f29e406ae40 0x7f29e406d300 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f29f4000f60 tx=0x7f29f4009040 comp rx=0 tx=0).stop 2023-10-26T01:38:18.130 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:18.128+0000 7f2a09e26640 1 -- 172.21.15.169:0/3550317602 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a041592d0 msgr2=0x7f2a0415a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:18.131 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:18.128+0000 7f2a09e26640 1 --2- 172.21.15.169:0/3550317602 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a041592d0 0x7f2a0415a8f0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f29fc00eb70 tx=0x7f29fc00c6a0 comp rx=0 tx=0).stop 2023-10-26T01:38:18.131 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:18.128+0000 7f2a09e26640 1 -- 172.21.15.169:0/3550317602 shutdown_connections 2023-10-26T01:38:18.131 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:18.128+0000 7f2a09e26640 1 --2- 172.21.15.169:0/3550317602 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f29e406ae40 0x7f29e406d300 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:18.131 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:18.128+0000 7f2a09e26640 1 --2- 172.21.15.169:0/3550317602 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a041592d0 0x7f2a0415a8f0 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:18.131 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:18.128+0000 7f2a09e26640 1 --2- 172.21.15.169:0/3550317602 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2a040cdc30 0x7f2a040af9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:18.132 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:18.128+0000 7f2a09e26640 1 --2- 172.21.15.169:0/3550317602 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a040ac930 0x7f2a040af460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:18.132 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:18.128+0000 7f2a09e26640 1 -- 172.21.15.169:0/3550317602 >> 172.21.15.169:0/3550317602 conn(0x7f2a04069930 msgr2=0x7f2a041581b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:18.132 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:18.128+0000 7f2a09e26640 1 -- 172.21.15.169:0/3550317602 shutdown_connections 2023-10-26T01:38:18.132 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:18.128+0000 7f2a09e26640 1 -- 172.21.15.169:0/3550317602 wait complete. 2023-10-26T01:38:18.149 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-26T01:38:18.150 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-26T01:38:18.365 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:18.363+0000 7f93f71c5640 1 -- 172.21.15.169:0/1401383785 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93f006cd40 msgr2=0x7f93f00aaa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:18.366 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:18.363+0000 7f93f71c5640 1 --2- 172.21.15.169:0/1401383785 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93f006cd40 0x7f93f00aaa60 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f93f00aff60 tx=0x7f93e402bdb0 comp rx=0 tx=0).stop 2023-10-26T01:38:18.366 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:18.363+0000 7f93f71c5640 1 -- 172.21.15.169:0/1401383785 shutdown_connections 2023-10-26T01:38:18.366 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:18.363+0000 7f93f71c5640 1 --2- 172.21.15.169:0/1401383785 >> v2:172.21.15.170:3300/0 conn(0x7f93f014e780 0x7f93f00cd6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:18.366 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:18.363+0000 7f93f71c5640 1 --2- 172.21.15.169:0/1401383785 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93f006cd40 0x7f93f00aaa60 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:18.367 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:18.363+0000 7f93f71c5640 1 -- 172.21.15.169:0/1401383785 >> 172.21.15.169:0/1401383785 conn(0x7f93f0069930 msgr2=0x7f93f0069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:18.367 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:18.363+0000 7f93f71c5640 1 -- 172.21.15.169:0/1401383785 shutdown_connections 2023-10-26T01:38:18.367 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:18.363+0000 7f93f71c5640 1 -- 172.21.15.169:0/1401383785 wait complete. 2023-10-26T01:38:18.369 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:18.365+0000 7f93f71c5640 1 Processor -- start 2023-10-26T01:38:18.370 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:18.365+0000 7f93f71c5640 1 -- start start 2023-10-26T01:38:18.370 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:18.366+0000 7f93f71c5640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93f006cd40 0x7f93f00707c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:18.370 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:18.366+0000 7f93f71c5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f93f00aafa0 0x7f93f01e3c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:18.370 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:18.366+0000 7f93f71c5640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f93f014e780 0x7f93f01ef050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:18.370 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:18.366+0000 7f93f71c5640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93f01e4e20 con 0x7f93f006cd40 2023-10-26T01:38:18.371 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:18.366+0000 7f93f71c5640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93f01e4580 con 0x7f93f00aafa0 2023-10-26T01:38:18.371 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:18.366+0000 7f93f71c5640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93f01e4860 con 0x7f93f014e780 2023-10-26T01:38:18.371 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:18.366+0000 7f93f4f61640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93f006cd40 0x7f93f00707c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:18.371 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:18.367+0000 7f93f4f61640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93f006cd40 0x7f93f00707c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:41954/0 (socket says 172.21.15.169:41954) 2023-10-26T01:38:18.371 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:18.367+0000 7f93f4f61640 1 -- 172.21.15.169:0/3973248989 learned_addr learned my addr 172.21.15.169:0/3973248989 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:38:18.372 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:18.367+0000 7f93f4f61640 1 -- 172.21.15.169:0/3973248989 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f93f014e780 msgr2=0x7f93f01ef050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:18.372 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:18.367+0000 7f93f4f61640 1 --2- 172.21.15.169:0/3973248989 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f93f014e780 0x7f93f01ef050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:18.372 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:18.367+0000 7f93f4f61640 1 -- 172.21.15.169:0/3973248989 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f93f00aafa0 msgr2=0x7f93f01e3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:18.372 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:18.367+0000 7f93effff640 1 --2- 172.21.15.169:0/3973248989 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f93f00aafa0 0x7f93f01e3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:18.372 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:18.367+0000 7f93f4f61640 1 --2- 172.21.15.169:0/3973248989 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f93f00aafa0 0x7f93f01e3c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:18.373 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:18.367+0000 7f93f4f61640 1 -- 172.21.15.169:0/3973248989 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93e4009560 con 0x7f93f006cd40 2023-10-26T01:38:18.373 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:18.367+0000 7f93f4f61640 1 --2- 172.21.15.169:0/3973248989 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93f006cd40 0x7f93f00707c0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f93e4009850 tx=0x7f93e402dd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:18.373 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:18.367+0000 7f93ed7fa640 1 -- 172.21.15.169:0/3973248989 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e4005650 con 0x7f93f006cd40 2023-10-26T01:38:18.373 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:18.368+0000 7f93ed7fa640 1 -- 172.21.15.169:0/3973248989 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f93e40038f0 con 0x7f93f006cd40 2023-10-26T01:38:18.373 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:18.368+0000 7f93ed7fa640 1 -- 172.21.15.169:0/3973248989 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e4003ce0 con 0x7f93f006cd40 2023-10-26T01:38:18.374 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:18.368+0000 7f93f71c5640 1 -- 172.21.15.169:0/3973248989 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93f01e4b40 con 0x7f93f006cd40 2023-10-26T01:38:18.374 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:18.368+0000 7f93f71c5640 1 -- 172.21.15.169:0/3973248989 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93f0070d00 con 0x7f93f006cd40 2023-10-26T01:38:18.375 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:18.369+0000 7f93f71c5640 1 -- 172.21.15.169:0/3973248989 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93f0066af0 con 0x7f93f006cd40 2023-10-26T01:38:18.375 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:18.370+0000 7f93ed7fa640 1 -- 172.21.15.169:0/3973248989 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f93e403e450 con 0x7f93f006cd40 2023-10-26T01:38:18.375 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:18.371+0000 7f93ed7fa640 1 --2- 172.21.15.169:0/3973248989 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f93d406ae40 0x7f93d406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:18.375 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:18.371+0000 7f93ed7fa640 1 -- 172.21.15.169:0/3973248989 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f93e40abc70 con 0x7f93f006cd40 2023-10-26T01:38:18.376 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:18.371+0000 7f93effff640 1 --2- 172.21.15.169:0/3973248989 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f93d406ae40 0x7f93d406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:18.376 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:18.373+0000 7f93effff640 1 --2- 172.21.15.169:0/3973248989 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f93d406ae40 0x7f93d406d300 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f93dc005fd0 tx=0x7f93dc005bc0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:18.377 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:18.375+0000 7f93ed7fa640 1 -- 172.21.15.169:0/3973248989 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f93e4077180 con 0x7f93f006cd40 2023-10-26T01:38:18.568 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:18.566+0000 7f93f71c5640 1 -- 172.21.15.169:0/3973248989 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f93f014b350 con 0x7f93f006cd40 2023-10-26T01:38:19.281 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:19.279+0000 7f93ed7fa640 1 -- 172.21.15.169:0/3973248989 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v131) v1 ==== 138+0+0 (secure 0 0 0) 0x7f93f014b350 con 0x7f93f006cd40 2023-10-26T01:38:19.281 INFO:teuthology.orchestra.run.smithi169.stderr:pool 'cephfs_metadata' created 2023-10-26T01:38:19.285 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:19.282+0000 7f93f71c5640 1 -- 172.21.15.169:0/3973248989 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f93d406ae40 msgr2=0x7f93d406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:19.285 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:19.282+0000 7f93f71c5640 1 --2- 172.21.15.169:0/3973248989 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f93d406ae40 0x7f93d406d300 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f93dc005fd0 tx=0x7f93dc005bc0 comp rx=0 tx=0).stop 2023-10-26T01:38:19.286 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:19.283+0000 7f93f71c5640 1 -- 172.21.15.169:0/3973248989 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93f006cd40 msgr2=0x7f93f00707c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:19.286 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:19.283+0000 7f93f71c5640 1 --2- 172.21.15.169:0/3973248989 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93f006cd40 0x7f93f00707c0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f93e4009850 tx=0x7f93e402dd80 comp rx=0 tx=0).stop 2023-10-26T01:38:19.286 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:19.283+0000 7f93f71c5640 1 -- 172.21.15.169:0/3973248989 shutdown_connections 2023-10-26T01:38:19.287 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:19.283+0000 7f93f71c5640 1 --2- 172.21.15.169:0/3973248989 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f93d406ae40 0x7f93d406d300 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:19.287 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:19.283+0000 7f93f71c5640 1 --2- 172.21.15.169:0/3973248989 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f93f014e780 0x7f93f01ef050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:19.288 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:19.283+0000 7f93f71c5640 1 --2- 172.21.15.169:0/3973248989 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f93f00aafa0 0x7f93f01e3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:19.288 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:19.283+0000 7f93f71c5640 1 --2- 172.21.15.169:0/3973248989 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93f006cd40 0x7f93f00707c0 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:19.288 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:19.283+0000 7f93f71c5640 1 -- 172.21.15.169:0/3973248989 >> 172.21.15.169:0/3973248989 conn(0x7f93f0069930 msgr2=0x7f93f006b570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:19.289 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:19.283+0000 7f93f71c5640 1 -- 172.21.15.169:0/3973248989 shutdown_connections 2023-10-26T01:38:19.289 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:19.283+0000 7f93f71c5640 1 -- 172.21.15.169:0/3973248989 wait complete. 2023-10-26T01:38:19.310 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-26T01:38:19.535 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:19.533+0000 7fb5cb8bb640 1 -- 172.21.15.169:0/996153987 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb5c41593e0 msgr2=0x7fb5c40ac200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:19.535 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:19.533+0000 7fb5cb8bb640 1 --2- 172.21.15.169:0/996153987 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb5c41593e0 0x7fb5c40ac200 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7fb5c0009830 tx=0x7fb5c002bd60 comp rx=0 tx=0).stop 2023-10-26T01:38:19.535 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:19.533+0000 7fb5cb8bb640 1 -- 172.21.15.169:0/996153987 shutdown_connections 2023-10-26T01:38:19.536 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:19.533+0000 7fb5cb8bb640 1 --2- 172.21.15.169:0/996153987 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb5c41593e0 0x7fb5c40ac200 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:19.536 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:19.533+0000 7fb5cb8bb640 1 -- 172.21.15.169:0/996153987 >> 172.21.15.169:0/996153987 conn(0x7fb5c4069930 msgr2=0x7fb5c4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:19.536 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:19.533+0000 7fb5cb8bb640 1 -- 172.21.15.169:0/996153987 shutdown_connections 2023-10-26T01:38:19.536 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:19.533+0000 7fb5cb8bb640 1 -- 172.21.15.169:0/996153987 wait complete. 2023-10-26T01:38:19.537 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:19.535+0000 7fb5cb8bb640 1 Processor -- start 2023-10-26T01:38:19.537 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:19.535+0000 7fb5cb8bb640 1 -- start start 2023-10-26T01:38:19.539 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:19.536+0000 7fb5cb8bb640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb5c40b0c80 0x7fb5c4150ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:19.539 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:19.536+0000 7fb5cb8bb640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb5c41593e0 0x7fb5c4159c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:19.539 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:19.536+0000 7fb5cb8bb640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb5c41597c0 0x7fb5c415a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:19.540 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:19.536+0000 7fb5cb8bb640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c41f63d0 con 0x7fb5c41593e0 2023-10-26T01:38:19.540 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:19.536+0000 7fb5cb8bb640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c4151000 con 0x7fb5c41597c0 2023-10-26T01:38:19.540 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:19.536+0000 7fb5cb8bb640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c41f57c0 con 0x7fb5c40b0c80 2023-10-26T01:38:19.540 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:19.536+0000 7fb5c8e56640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb5c41593e0 0x7fb5c4159c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:19.540 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:19.536+0000 7fb5c8e56640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb5c41593e0 0x7fb5c4159c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:41974/0 (socket says 172.21.15.169:41974) 2023-10-26T01:38:19.541 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:19.536+0000 7fb5c8e56640 1 -- 172.21.15.169:0/3799693172 learned_addr learned my addr 172.21.15.169:0/3799693172 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:38:19.541 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:19.536+0000 7fb5c9657640 1 --2- 172.21.15.169:0/3799693172 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb5c40b0c80 0x7fb5c4150ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:19.541 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:19.536+0000 7fb5c8e56640 1 -- 172.21.15.169:0/3799693172 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb5c40b0c80 msgr2=0x7fb5c4150ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:19.541 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:19.536+0000 7fb5c8e56640 1 --2- 172.21.15.169:0/3799693172 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb5c40b0c80 0x7fb5c4150ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:19.541 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:19.536+0000 7fb5c9e58640 1 --2- 172.21.15.169:0/3799693172 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb5c41597c0 0x7fb5c415a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:19.542 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:19.536+0000 7fb5c8e56640 1 -- 172.21.15.169:0/3799693172 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb5c41597c0 msgr2=0x7fb5c415a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:19.542 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:19.537+0000 7fb5c8e56640 1 --2- 172.21.15.169:0/3799693172 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb5c41597c0 0x7fb5c415a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:19.542 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:19.537+0000 7fb5c8e56640 1 -- 172.21.15.169:0/3799693172 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5c0009490 con 0x7fb5c41593e0 2023-10-26T01:38:19.542 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:19.537+0000 7fb5c9e58640 1 --2- 172.21.15.169:0/3799693172 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb5c41597c0 0x7fb5c415a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:38:19.542 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:19.537+0000 7fb5c8e56640 1 --2- 172.21.15.169:0/3799693172 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb5c41593e0 0x7fb5c4159c60 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7fb5b400b3f0 tx=0x7fb5b400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:19.543 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:19.537+0000 7fb5b9ffb640 1 -- 172.21.15.169:0/3799693172 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5b4011830 con 0x7fb5c41593e0 2023-10-26T01:38:19.543 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:19.537+0000 7fb5b9ffb640 1 -- 172.21.15.169:0/3799693172 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb5b4011a80 con 0x7fb5c41593e0 2023-10-26T01:38:19.543 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:19.537+0000 7fb5b9ffb640 1 -- 172.21.15.169:0/3799693172 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5b4011e90 con 0x7fb5c41593e0 2023-10-26T01:38:19.543 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:19.537+0000 7fb5cb8bb640 1 -- 172.21.15.169:0/3799693172 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5c41f5aa0 con 0x7fb5c41593e0 2023-10-26T01:38:19.543 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:19.537+0000 7fb5cb8bb640 1 -- 172.21.15.169:0/3799693172 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5c41f6060 con 0x7fb5c41593e0 2023-10-26T01:38:19.544 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:19.538+0000 7fb5cb8bb640 1 -- 172.21.15.169:0/3799693172 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5c404db10 con 0x7fb5c41593e0 2023-10-26T01:38:19.545 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:19.540+0000 7fb5b9ffb640 1 -- 172.21.15.169:0/3799693172 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fb5b401a400 con 0x7fb5c41593e0 2023-10-26T01:38:19.545 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:19.541+0000 7fb5b9ffb640 1 --2- 172.21.15.169:0/3799693172 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb5a806ae40 0x7fb5a806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:19.545 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:19.541+0000 7fb5b9ffb640 1 -- 172.21.15.169:0/3799693172 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fb5b4089540 con 0x7fb5c41593e0 2023-10-26T01:38:19.545 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:19.541+0000 7fb5c9657640 1 --2- 172.21.15.169:0/3799693172 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb5a806ae40 0x7fb5a806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:19.546 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:19.543+0000 7fb5c9657640 1 --2- 172.21.15.169:0/3799693172 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb5a806ae40 0x7fb5a806d300 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fb5c0036000 tx=0x7fb5c0004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:19.547 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:19.546+0000 7fb5b9ffb640 1 -- 172.21.15.169:0/3799693172 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb5b40547d0 con 0x7fb5c41593e0 2023-10-26T01:38:19.741 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:19.740+0000 7fb5cb8bb640 1 -- 172.21.15.169:0/3799693172 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fb5c40b2170 con 0x7fb5c41593e0 2023-10-26T01:38:20.299 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:20.297+0000 7fb5b9ffb640 1 -- 172.21.15.169:0/3799693172 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v132) v1 ==== 170+0+0 (secure 0 0 0) 0x7fb5b4057df0 con 0x7fb5c41593e0 2023-10-26T01:38:20.300 INFO:teuthology.orchestra.run.smithi169.stderr:pool 'cephfs_data' created 2023-10-26T01:38:20.303 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:20.302+0000 7fb5cb8bb640 1 -- 172.21.15.169:0/3799693172 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb5a806ae40 msgr2=0x7fb5a806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:20.304 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:20.302+0000 7fb5cb8bb640 1 --2- 172.21.15.169:0/3799693172 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb5a806ae40 0x7fb5a806d300 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fb5c0036000 tx=0x7fb5c0004730 comp rx=0 tx=0).stop 2023-10-26T01:38:20.304 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:20.302+0000 7fb5cb8bb640 1 -- 172.21.15.169:0/3799693172 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb5c41593e0 msgr2=0x7fb5c4159c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:20.304 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:20.302+0000 7fb5cb8bb640 1 --2- 172.21.15.169:0/3799693172 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb5c41593e0 0x7fb5c4159c60 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7fb5b400b3f0 tx=0x7fb5b400b8b0 comp rx=0 tx=0).stop 2023-10-26T01:38:20.305 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:20.302+0000 7fb5cb8bb640 1 -- 172.21.15.169:0/3799693172 shutdown_connections 2023-10-26T01:38:20.305 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:20.302+0000 7fb5cb8bb640 1 --2- 172.21.15.169:0/3799693172 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb5a806ae40 0x7fb5a806d300 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:20.305 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:20.302+0000 7fb5cb8bb640 1 --2- 172.21.15.169:0/3799693172 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb5c41593e0 0x7fb5c4159c60 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:20.305 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:20.302+0000 7fb5cb8bb640 1 --2- 172.21.15.169:0/3799693172 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb5c41597c0 0x7fb5c415a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:20.306 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:20.302+0000 7fb5cb8bb640 1 --2- 172.21.15.169:0/3799693172 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb5c40b0c80 0x7fb5c4150ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:20.306 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:20.302+0000 7fb5cb8bb640 1 -- 172.21.15.169:0/3799693172 >> 172.21.15.169:0/3799693172 conn(0x7fb5c4069930 msgr2=0x7fb5c40cd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:20.306 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:20.303+0000 7fb5cb8bb640 1 -- 172.21.15.169:0/3799693172 shutdown_connections 2023-10-26T01:38:20.306 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:20.303+0000 7fb5cb8bb640 1 -- 172.21.15.169:0/3799693172 wait complete. 2023-10-26T01:38:20.333 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-26T01:38:20.532 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:20.529+0000 7f8030376640 1 -- 172.21.15.169:0/1630349904 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f802806ce00 msgr2=0x7f8028155360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:20.532 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:20.529+0000 7f8030376640 1 --2- 172.21.15.169:0/1630349904 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f802806ce00 0x7f8028155360 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f80280b0520 tx=0x7f802402bd30 comp rx=0 tx=0).stop 2023-10-26T01:38:20.532 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:20.530+0000 7f8030376640 1 -- 172.21.15.169:0/1630349904 shutdown_connections 2023-10-26T01:38:20.533 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:20.530+0000 7f8030376640 1 --2- 172.21.15.169:0/1630349904 >> v2:172.21.15.170:3300/0 conn(0x7f802815b4a0 0x7f802815b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:20.533 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:20.530+0000 7f8030376640 1 --2- 172.21.15.169:0/1630349904 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f802806ce00 0x7f8028155360 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:20.533 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:20.530+0000 7f8030376640 1 -- 172.21.15.169:0/1630349904 >> 172.21.15.169:0/1630349904 conn(0x7f8028069930 msgr2=0x7f8028069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:20.533 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:20.530+0000 7f8030376640 1 -- 172.21.15.169:0/1630349904 shutdown_connections 2023-10-26T01:38:20.534 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:20.530+0000 7f8030376640 1 -- 172.21.15.169:0/1630349904 wait complete. 2023-10-26T01:38:20.537 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:20.531+0000 7f8030376640 1 Processor -- start 2023-10-26T01:38:20.538 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:20.532+0000 7f8030376640 1 -- start start 2023-10-26T01:38:20.538 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:20.533+0000 7f8030376640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f802806ad90 0x7f8028155e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:20.538 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:20.533+0000 7f8030376640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f802806ce00 0x7f80281f1c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:20.539 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:20.533+0000 7f8030376640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f802815b4a0 0x7f802806f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:20.539 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:20.533+0000 7f8030376640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80280c7930 con 0x7f802806ce00 2023-10-26T01:38:20.539 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:20.533+0000 7f8030376640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80280adb60 con 0x7f802806ad90 2023-10-26T01:38:20.540 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:20.533+0000 7f8030376640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f802814c770 con 0x7f802815b4a0 2023-10-26T01:38:20.540 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:20.533+0000 7f802d911640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f802806ce00 0x7f80281f1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:20.540 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:20.533+0000 7f802d911640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f802806ce00 0x7f80281f1c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:41984/0 (socket says 172.21.15.169:41984) 2023-10-26T01:38:20.540 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:20.533+0000 7f802d911640 1 -- 172.21.15.169:0/1604076169 learned_addr learned my addr 172.21.15.169:0/1604076169 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:38:20.541 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:20.533+0000 7f802e112640 1 --2- 172.21.15.169:0/1604076169 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f802806ad90 0x7f8028155e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:20.541 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:20.533+0000 7f802d911640 1 -- 172.21.15.169:0/1604076169 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f802815b4a0 msgr2=0x7f802806f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:20.541 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:20.533+0000 7f802e913640 1 --2- 172.21.15.169:0/1604076169 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f802815b4a0 0x7f802806f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:20.541 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:20.533+0000 7f802d911640 1 --2- 172.21.15.169:0/1604076169 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f802815b4a0 0x7f802806f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:20.542 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:20.534+0000 7f802d911640 1 -- 172.21.15.169:0/1604076169 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f802806ad90 msgr2=0x7f8028155e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:20.542 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:20.534+0000 7f802d911640 1 --2- 172.21.15.169:0/1604076169 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f802806ad90 0x7f8028155e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:20.542 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:20.534+0000 7f802d911640 1 -- 172.21.15.169:0/1604076169 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8024009560 con 0x7f802806ce00 2023-10-26T01:38:20.542 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:20.534+0000 7f802e112640 1 --2- 172.21.15.169:0/1604076169 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f802806ad90 0x7f8028155e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:38:20.542 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:20.534+0000 7f802d911640 1 --2- 172.21.15.169:0/1604076169 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f802806ce00 0x7f80281f1c10 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f801800dae0 tx=0x7f801800a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:20.543 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:20.534+0000 7f801effd640 1 -- 172.21.15.169:0/1604076169 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f801800ae30 con 0x7f802806ce00 2023-10-26T01:38:20.544 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:20.534+0000 7f801effd640 1 -- 172.21.15.169:0/1604076169 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8018005110 con 0x7f802806ce00 2023-10-26T01:38:20.544 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:20.534+0000 7f801effd640 1 -- 172.21.15.169:0/1604076169 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8018005500 con 0x7f802806ce00 2023-10-26T01:38:20.544 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:20.534+0000 7f8030376640 1 -- 172.21.15.169:0/1604076169 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f802814c490 con 0x7f802806ce00 2023-10-26T01:38:20.545 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:20.534+0000 7f8030376640 1 -- 172.21.15.169:0/1604076169 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8028071940 con 0x7f802806ce00 2023-10-26T01:38:20.545 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:20.535+0000 7f8030376640 1 -- 172.21.15.169:0/1604076169 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80280add60 con 0x7f802806ce00 2023-10-26T01:38:20.547 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:20.536+0000 7f801effd640 1 -- 172.21.15.169:0/1604076169 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f80180047a0 con 0x7f802806ce00 2023-10-26T01:38:20.547 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:20.537+0000 7f801effd640 1 --2- 172.21.15.169:0/1604076169 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f800c06ae70 0x7f800c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:20.547 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:20.537+0000 7f801effd640 1 -- 172.21.15.169:0/1604076169 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f8018089d70 con 0x7f802806ce00 2023-10-26T01:38:20.548 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:20.538+0000 7f802e112640 1 --2- 172.21.15.169:0/1604076169 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f800c06ae70 0x7f800c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:20.548 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:20.539+0000 7f802e112640 1 --2- 172.21.15.169:0/1604076169 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f800c06ae70 0x7f800c06d330 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f8024035000 tx=0x7f8024004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:20.548 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:20.543+0000 7f801effd640 1 -- 172.21.15.169:0/1604076169 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8018054de0 con 0x7f802806ce00 2023-10-26T01:38:20.772 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:20.771+0000 7f8030376640 1 -- 172.21.15.169:0/1604076169 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f80280afe10 con 0x7f802806ce00 2023-10-26T01:38:21.322 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:21.320+0000 7f801effd640 1 -- 172.21.15.169:0/1604076169 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 16 and data pool 17 v64) v1 ==== 174+0+0 (secure 0 0 0) 0x7f8018058400 con 0x7f802806ce00 2023-10-26T01:38:21.323 INFO:teuthology.orchestra.run.smithi169.stderr:new fs with metadata pool 16 and data pool 17 2023-10-26T01:38:21.327 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:21.325+0000 7f8030376640 1 -- 172.21.15.169:0/1604076169 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f800c06ae70 msgr2=0x7f800c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:21.327 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:21.325+0000 7f8030376640 1 --2- 172.21.15.169:0/1604076169 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f800c06ae70 0x7f800c06d330 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f8024035000 tx=0x7f8024004730 comp rx=0 tx=0).stop 2023-10-26T01:38:21.327 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:21.325+0000 7f8030376640 1 -- 172.21.15.169:0/1604076169 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f802806ce00 msgr2=0x7f80281f1c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:21.328 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:21.325+0000 7f8030376640 1 --2- 172.21.15.169:0/1604076169 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f802806ce00 0x7f80281f1c10 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f801800dae0 tx=0x7f801800a250 comp rx=0 tx=0).stop 2023-10-26T01:38:21.328 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:21.325+0000 7f8030376640 1 -- 172.21.15.169:0/1604076169 shutdown_connections 2023-10-26T01:38:21.328 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:21.325+0000 7f8030376640 1 --2- 172.21.15.169:0/1604076169 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f800c06ae70 0x7f800c06d330 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:21.328 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:21.325+0000 7f8030376640 1 --2- 172.21.15.169:0/1604076169 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f802806ce00 0x7f80281f1c10 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:21.329 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:21.325+0000 7f8030376640 1 --2- 172.21.15.169:0/1604076169 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f802815b4a0 0x7f802806f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:21.329 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:21.325+0000 7f8030376640 1 --2- 172.21.15.169:0/1604076169 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f802806ad90 0x7f8028155e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:21.329 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:21.326+0000 7f8030376640 1 -- 172.21.15.169:0/1604076169 >> 172.21.15.169:0/1604076169 conn(0x7f8028069930 msgr2=0x7f802814e040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:21.329 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:21.326+0000 7f8030376640 1 -- 172.21.15.169:0/1604076169 shutdown_connections 2023-10-26T01:38:21.330 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:21.326+0000 7f8030376640 1 -- 172.21.15.169:0/1604076169 wait complete. 2023-10-26T01:38:21.354 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:38:21.545 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:21.543+0000 7f25b16d7640 1 -- 172.21.15.169:0/2763371665 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25ac159e10 msgr2=0x7f25ac15a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:21.546 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:21.543+0000 7f25b16d7640 1 --2- 172.21.15.169:0/2763371665 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25ac159e10 0x7f25ac15a2c0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f25a4005fd0 tx=0x7f25a402e3e0 comp rx=0 tx=0).stop 2023-10-26T01:38:21.546 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:21.543+0000 7f25b16d7640 1 -- 172.21.15.169:0/2763371665 shutdown_connections 2023-10-26T01:38:21.546 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:21.543+0000 7f25b16d7640 1 --2- 172.21.15.169:0/2763371665 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25ac159e10 0x7f25ac15a2c0 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:21.546 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:21.543+0000 7f25b16d7640 1 --2- 172.21.15.169:0/2763371665 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f25ac06ce20 0x7f25ac14e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:21.546 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:21.543+0000 7f25b16d7640 1 -- 172.21.15.169:0/2763371665 >> 172.21.15.169:0/2763371665 conn(0x7f25ac069930 msgr2=0x7f25ac069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:21.547 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:21.543+0000 7f25b16d7640 1 -- 172.21.15.169:0/2763371665 shutdown_connections 2023-10-26T01:38:21.547 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:21.544+0000 7f25b16d7640 1 -- 172.21.15.169:0/2763371665 wait complete. 2023-10-26T01:38:21.548 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:21.545+0000 7f25b16d7640 1 Processor -- start 2023-10-26T01:38:21.549 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:21.545+0000 7f25b16d7640 1 -- start start 2023-10-26T01:38:21.549 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:21.546+0000 7f25b16d7640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f25ac06ce20 0x7f25ac14c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:21.549 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:21.546+0000 7f25b16d7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25ac158850 0x7f25ac06f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:21.549 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:21.546+0000 7f25b16d7640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25ac159e10 0x7f25ac150b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:21.549 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:21.546+0000 7f25b16d7640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25ac14f7f0 con 0x7f25ac159e10 2023-10-26T01:38:21.550 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:21.546+0000 7f25b16d7640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25ac14fad0 con 0x7f25ac158850 2023-10-26T01:38:21.550 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:21.546+0000 7f25b16d7640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25ac14fdb0 con 0x7f25ac06ce20 2023-10-26T01:38:21.550 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:21.547+0000 7f25ab7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25ac159e10 0x7f25ac150b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:21.550 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:21.547+0000 7f25ab7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25ac159e10 0x7f25ac150b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:41994/0 (socket says 172.21.15.169:41994) 2023-10-26T01:38:21.550 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:21.547+0000 7f25ab7fe640 1 -- 172.21.15.169:0/2481724072 learned_addr learned my addr 172.21.15.169:0/2481724072 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:38:21.551 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:21.547+0000 7f25aaffd640 1 --2- 172.21.15.169:0/2481724072 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f25ac06ce20 0x7f25ac14c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:21.551 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:21.547+0000 7f25a3fff640 1 --2- 172.21.15.169:0/2481724072 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25ac158850 0x7f25ac06f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:21.552 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:21.547+0000 7f25ab7fe640 1 -- 172.21.15.169:0/2481724072 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f25ac06ce20 msgr2=0x7f25ac14c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:21.552 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:21.547+0000 7f25ab7fe640 1 --2- 172.21.15.169:0/2481724072 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f25ac06ce20 0x7f25ac14c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:21.552 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:21.547+0000 7f25ab7fe640 1 -- 172.21.15.169:0/2481724072 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25ac158850 msgr2=0x7f25ac06f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:21.553 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:21.547+0000 7f25ab7fe640 1 --2- 172.21.15.169:0/2481724072 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25ac158850 0x7f25ac06f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:21.553 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:21.547+0000 7f25ab7fe640 1 -- 172.21.15.169:0/2481724072 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25a4009900 con 0x7f25ac159e10 2023-10-26T01:38:21.553 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:21.547+0000 7f25ab7fe640 1 --2- 172.21.15.169:0/2481724072 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25ac159e10 0x7f25ac150b70 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f25a4006010 tx=0x7f25a40045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:21.553 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:21.548+0000 7f25a37fe640 1 -- 172.21.15.169:0/2481724072 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a4033710 con 0x7f25ac159e10 2023-10-26T01:38:21.553 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:21.548+0000 7f25a37fe640 1 -- 172.21.15.169:0/2481724072 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f25a4033960 con 0x7f25ac159e10 2023-10-26T01:38:21.554 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:21.548+0000 7f25a37fe640 1 -- 172.21.15.169:0/2481724072 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a4033d30 con 0x7f25ac159e10 2023-10-26T01:38:21.554 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:21.548+0000 7f25b16d7640 1 -- 172.21.15.169:0/2481724072 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25ac150090 con 0x7f25ac159e10 2023-10-26T01:38:21.554 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:21.548+0000 7f25b16d7640 1 -- 172.21.15.169:0/2481724072 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25ac150650 con 0x7f25ac159e10 2023-10-26T01:38:21.554 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:21.549+0000 7f25b16d7640 1 -- 172.21.15.169:0/2481724072 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25ac150840 con 0x7f25ac159e10 2023-10-26T01:38:21.555 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:21.550+0000 7f25a37fe640 1 -- 172.21.15.169:0/2481724072 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f25a400a040 con 0x7f25ac159e10 2023-10-26T01:38:21.555 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:21.551+0000 7f25a37fe640 1 --2- 172.21.15.169:0/2481724072 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f258c06ae40 0x7f258c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:21.555 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:21.551+0000 7f25a37fe640 1 -- 172.21.15.169:0/2481724072 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f25a40aca60 con 0x7f25ac159e10 2023-10-26T01:38:21.556 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:21.552+0000 7f25aaffd640 1 --2- 172.21.15.169:0/2481724072 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f258c06ae40 0x7f258c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:21.556 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:21.553+0000 7f25aaffd640 1 --2- 172.21.15.169:0/2481724072 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f258c06ae40 0x7f258c06d300 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f259c005fd0 tx=0x7f259c005850 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:21.557 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:21.555+0000 7f25a37fe640 1 -- 172.21.15.169:0/2481724072 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f25a4077a70 con 0x7f25ac159e10 2023-10-26T01:38:21.747 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:21.745+0000 7f25b16d7640 1 -- 172.21.15.169:0/2481724072 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f25ac04db10 con 0x7f25ac159e10 2023-10-26T01:38:21.750 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:21.748+0000 7f25a37fe640 1 -- 172.21.15.169:0/2481724072 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v133) v1 ==== 74+0+16589 (secure 0 0 0) 0x7f25a404c090 con 0x7f25ac159e10 2023-10-26T01:38:21.750 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:38:21.751 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":133,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:38:20.772751+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-10-26T01:38:18.617877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"133","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-10-26T01:38:19.793041+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"133","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:38:21.753 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:21.751+0000 7f25b16d7640 1 -- 172.21.15.169:0/2481724072 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f258c06ae40 msgr2=0x7f258c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:21.753 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:21.751+0000 7f25b16d7640 1 --2- 172.21.15.169:0/2481724072 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f258c06ae40 0x7f258c06d300 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f259c005fd0 tx=0x7f259c005850 comp rx=0 tx=0).stop 2023-10-26T01:38:21.753 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:21.751+0000 7f25b16d7640 1 -- 172.21.15.169:0/2481724072 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25ac159e10 msgr2=0x7f25ac150b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:21.754 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:21.751+0000 7f25b16d7640 1 --2- 172.21.15.169:0/2481724072 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25ac159e10 0x7f25ac150b70 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f25a4006010 tx=0x7f25a40045e0 comp rx=0 tx=0).stop 2023-10-26T01:38:21.754 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:21.751+0000 7f25b16d7640 1 -- 172.21.15.169:0/2481724072 shutdown_connections 2023-10-26T01:38:21.754 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:21.751+0000 7f25b16d7640 1 --2- 172.21.15.169:0/2481724072 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f258c06ae40 0x7f258c06d300 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:21.754 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:21.751+0000 7f25b16d7640 1 --2- 172.21.15.169:0/2481724072 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25ac159e10 0x7f25ac150b70 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:21.754 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:21.751+0000 7f25b16d7640 1 --2- 172.21.15.169:0/2481724072 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25ac158850 0x7f25ac06f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:21.755 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:21.751+0000 7f25b16d7640 1 --2- 172.21.15.169:0/2481724072 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f25ac06ce20 0x7f25ac14c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:21.755 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:21.751+0000 7f25b16d7640 1 -- 172.21.15.169:0/2481724072 >> 172.21.15.169:0/2481724072 conn(0x7f25ac069930 msgr2=0x7f25ac0cd900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:21.755 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:21.751+0000 7f25b16d7640 1 -- 172.21.15.169:0/2481724072 shutdown_connections 2023-10-26T01:38:21.755 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:21.751+0000 7f25b16d7640 1 -- 172.21.15.169:0/2481724072 wait complete. 2023-10-26T01:38:21.771 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:38:21.982 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:21.980+0000 7f5751690640 1 -- 172.21.15.169:0/4092970915 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5734004d00 msgr2=0x7f5734005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:21.983 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:21.980+0000 7f5751690640 1 --2- 172.21.15.169:0/4092970915 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5734004d00 0x7f5734005160 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f574400c640 tx=0x7f57340057f0 comp rx=0 tx=0).stop 2023-10-26T01:38:21.983 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:21.980+0000 7f574b7fe640 1 -- 172.21.15.169:0/4092970915 reap_dead start 2023-10-26T01:38:21.983 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:21.980+0000 7f5751690640 1 -- 172.21.15.169:0/4092970915 shutdown_connections 2023-10-26T01:38:21.983 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:21.980+0000 7f5751690640 1 -- 172.21.15.169:0/4092970915 >> 172.21.15.169:0/4092970915 conn(0x7f574c069930 msgr2=0x7f574c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:21.984 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:21.980+0000 7f5751690640 1 -- 172.21.15.169:0/4092970915 shutdown_connections 2023-10-26T01:38:21.984 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:21.981+0000 7f5751690640 1 -- 172.21.15.169:0/4092970915 wait complete. 2023-10-26T01:38:21.984 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:21.982+0000 7f5751690640 1 Processor -- start 2023-10-26T01:38:21.985 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:21.982+0000 7f5751690640 1 -- start start 2023-10-26T01:38:21.987 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:21.983+0000 7f5751690640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5734004d00 0x7f574c13be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:21.987 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:21.983+0000 7f5751690640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f574c1f3c40 0x7f574c13f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:21.987 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:21.983+0000 7f5751690640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f574c13fa50 0x7f574c150330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:21.987 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:21.983+0000 7f5751690640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f574c099940 con 0x7f5734004d00 2023-10-26T01:38:21.988 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:21.983+0000 7f5751690640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f574c099610 con 0x7f574c13fa50 2023-10-26T01:38:21.988 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:21.983+0000 7f5751690640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f574c13cde0 con 0x7f574c1f3c40 2023-10-26T01:38:21.988 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:21.984+0000 7f574affd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5734004d00 0x7f574c13be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:21.988 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:21.984+0000 7f574affd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5734004d00 0x7f574c13be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:42010/0 (socket says 172.21.15.169:42010) 2023-10-26T01:38:21.988 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:21.984+0000 7f574affd640 1 -- 172.21.15.169:0/3499099389 learned_addr learned my addr 172.21.15.169:0/3499099389 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:38:21.989 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:21.984+0000 7f574a7fc640 1 --2- 172.21.15.169:0/3499099389 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f574c1f3c40 0x7f574c13f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:21.989 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:21.984+0000 7f574b7fe640 1 --2- 172.21.15.169:0/3499099389 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f574c13fa50 0x7f574c150330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:21.989 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:21.984+0000 7f574affd640 1 -- 172.21.15.169:0/3499099389 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f574c1f3c40 msgr2=0x7f574c13f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:21.989 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:21.984+0000 7f574affd640 1 --2- 172.21.15.169:0/3499099389 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f574c1f3c40 0x7f574c13f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:21.989 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:21.984+0000 7f574affd640 1 -- 172.21.15.169:0/3499099389 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f574c13fa50 msgr2=0x7f574c150330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:21.990 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:21.984+0000 7f574affd640 1 --2- 172.21.15.169:0/3499099389 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f574c13fa50 0x7f574c150330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:21.990 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:21.984+0000 7f574affd640 1 -- 172.21.15.169:0/3499099389 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f574c13c540 con 0x7f5734004d00 2023-10-26T01:38:21.990 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:21.984+0000 7f574a7fc640 1 --2- 172.21.15.169:0/3499099389 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f574c1f3c40 0x7f574c13f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:38:21.990 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:21.984+0000 7f574b7fe640 1 --2- 172.21.15.169:0/3499099389 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f574c13fa50 0x7f574c150330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:38:21.990 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:21.984+0000 7f574affd640 1 --2- 172.21.15.169:0/3499099389 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5734004d00 0x7f574c13be60 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f573c00d580 tx=0x7f573c00da40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:21.991 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:21.985+0000 7f57337fe640 1 -- 172.21.15.169:0/3499099389 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f573c012630 con 0x7f5734004d00 2023-10-26T01:38:21.991 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:21.985+0000 7f57337fe640 1 -- 172.21.15.169:0/3499099389 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f573c0127b0 con 0x7f5734004d00 2023-10-26T01:38:21.991 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:21.985+0000 7f5751690640 1 -- 172.21.15.169:0/3499099389 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f574c13c820 con 0x7f5734004d00 2023-10-26T01:38:21.991 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:21.985+0000 7f57337fe640 1 -- 172.21.15.169:0/3499099389 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f573c012bc0 con 0x7f5734004d00 2023-10-26T01:38:21.991 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:21.985+0000 7f5751690640 1 -- 172.21.15.169:0/3499099389 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f574c1f40c0 con 0x7f5734004d00 2023-10-26T01:38:21.992 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:21.986+0000 7f5751690640 1 -- 172.21.15.169:0/3499099389 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f574c099bb0 con 0x7f5734004d00 2023-10-26T01:38:21.993 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:21.987+0000 7f57337fe640 1 -- 172.21.15.169:0/3499099389 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f573c012d20 con 0x7f5734004d00 2023-10-26T01:38:21.993 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:21.988+0000 7f57337fe640 1 --2- 172.21.15.169:0/3499099389 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f573806ae40 0x7f573806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:21.993 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:21.988+0000 7f57337fe640 1 -- 172.21.15.169:0/3499099389 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f573c089570 con 0x7f5734004d00 2023-10-26T01:38:21.993 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:21.988+0000 7f574a7fc640 1 --2- 172.21.15.169:0/3499099389 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f573806ae40 0x7f573806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:21.993 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:21.989+0000 7f574a7fc640 1 --2- 172.21.15.169:0/3499099389 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f573806ae40 0x7f573806d300 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f574c0674b0 tx=0x7f5740005f90 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:21.994 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:21.992+0000 7f57337fe640 1 -- 172.21.15.169:0/3499099389 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f573c058020 con 0x7f5734004d00 2023-10-26T01:38:22.184 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:22.182+0000 7f5751690640 1 -- 172.21.15.169:0/3499099389 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f574c04db10 con 0x7f5734004d00 2023-10-26T01:38:22.186 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:22.184+0000 7f57337fe640 1 -- 172.21.15.169:0/3499099389 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v133) v1 ==== 74+0+16589 (secure 0 0 0) 0x7f573c057bb0 con 0x7f5734004d00 2023-10-26T01:38:22.187 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:38:22.187 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":133,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:38:20.772751+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-10-26T01:38:18.617877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"133","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-10-26T01:38:19.793041+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"133","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:38:22.189 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:22.187+0000 7f5751690640 1 -- 172.21.15.169:0/3499099389 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f573806ae40 msgr2=0x7f573806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:22.189 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:22.187+0000 7f5751690640 1 --2- 172.21.15.169:0/3499099389 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f573806ae40 0x7f573806d300 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f574c0674b0 tx=0x7f5740005f90 comp rx=0 tx=0).stop 2023-10-26T01:38:22.190 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:22.187+0000 7f5751690640 1 -- 172.21.15.169:0/3499099389 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5734004d00 msgr2=0x7f574c13be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:22.190 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:22.187+0000 7f5751690640 1 --2- 172.21.15.169:0/3499099389 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5734004d00 0x7f574c13be60 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f573c00d580 tx=0x7f573c00da40 comp rx=0 tx=0).stop 2023-10-26T01:38:22.190 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:22.187+0000 7f5751690640 1 -- 172.21.15.169:0/3499099389 shutdown_connections 2023-10-26T01:38:22.190 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:22.187+0000 7f5751690640 1 --2- 172.21.15.169:0/3499099389 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f573806ae40 0x7f573806d300 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:22.191 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:22.187+0000 7f5751690640 1 --2- 172.21.15.169:0/3499099389 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f574c13fa50 0x7f574c150330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:22.191 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:22.187+0000 7f5751690640 1 --2- 172.21.15.169:0/3499099389 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f574c1f3c40 0x7f574c13f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:22.191 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:22.187+0000 7f5751690640 1 --2- 172.21.15.169:0/3499099389 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5734004d00 0x7f574c13be60 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:22.191 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:22.187+0000 7f5751690640 1 -- 172.21.15.169:0/3499099389 >> 172.21.15.169:0/3499099389 conn(0x7f574c069930 msgr2=0x7f574c14d830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:22.191 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:22.187+0000 7f5751690640 1 -- 172.21.15.169:0/3499099389 shutdown_connections 2023-10-26T01:38:22.192 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:22.187+0000 7f5751690640 1 -- 172.21.15.169:0/3499099389 wait complete. 2023-10-26T01:38:22.207 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-26T01:38:22.427 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:22.425+0000 7fe5b9f8e640 1 -- 172.21.15.169:0/3167541899 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5a4004d00 msgr2=0x7fe5a4005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:22.428 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:22.425+0000 7fe5b9f8e640 1 --2- 172.21.15.169:0/3167541899 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5a4004d00 0x7fe5a4005180 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac00c610 tx=0x7fe5a40057f0 comp rx=0 tx=0).stop 2023-10-26T01:38:22.428 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:22.425+0000 7fe5b978d640 1 -- 172.21.15.169:0/3167541899 reap_dead start 2023-10-26T01:38:22.428 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:22.425+0000 7fe5b9f8e640 1 -- 172.21.15.169:0/3167541899 shutdown_connections 2023-10-26T01:38:22.428 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:22.425+0000 7fe5b9f8e640 1 -- 172.21.15.169:0/3167541899 >> 172.21.15.169:0/3167541899 conn(0x7fe5b40af110 msgr2=0x7fe5b40af520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:22.429 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:22.425+0000 7fe5b9f8e640 1 -- 172.21.15.169:0/3167541899 shutdown_connections 2023-10-26T01:38:22.429 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:22.425+0000 7fe5b9f8e640 1 -- 172.21.15.169:0/3167541899 wait complete. 2023-10-26T01:38:22.431 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:22.427+0000 7fe5b9f8e640 1 Processor -- start 2023-10-26T01:38:22.431 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:22.427+0000 7fe5b9f8e640 1 -- start start 2023-10-26T01:38:22.431 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:22.428+0000 7fe5b9f8e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5a4004d00 0x7fe5b406b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:22.432 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:22.428+0000 7fe5b9f8e640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe5b406b890 0x7fe5b4156850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:22.432 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:22.428+0000 7fe5b9f8e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe5b4156d90 0x7fe5b4157210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:22.432 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:22.428+0000 7fe5b9f8e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5b41f1c50 con 0x7fe5a4004d00 2023-10-26T01:38:22.432 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:22.428+0000 7fe5b9f8e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5b406c0a0 con 0x7fe5b4156d90 2023-10-26T01:38:22.432 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:22.428+0000 7fe5b9f8e640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5b406c380 con 0x7fe5b406b890 2023-10-26T01:38:22.433 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:22.428+0000 7fe5b8f8c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5a4004d00 0x7fe5b406b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:22.433 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:22.428+0000 7fe5b8f8c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5a4004d00 0x7fe5b406b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:42014/0 (socket says 172.21.15.169:42014) 2023-10-26T01:38:22.433 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:22.428+0000 7fe5b8f8c640 1 -- 172.21.15.169:0/1249165225 learned_addr learned my addr 172.21.15.169:0/1249165225 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:38:22.433 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:22.429+0000 7fe5b8f8c640 1 -- 172.21.15.169:0/1249165225 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe5b406b890 msgr2=0x7fe5b4156850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:22.434 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:22.429+0000 7fe5b3fff640 1 --2- 172.21.15.169:0/1249165225 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe5b406b890 0x7fe5b4156850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:22.434 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:22.429+0000 7fe5b8f8c640 1 --2- 172.21.15.169:0/1249165225 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe5b406b890 0x7fe5b4156850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:22.434 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:22.429+0000 7fe5b8f8c640 1 -- 172.21.15.169:0/1249165225 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe5b4156d90 msgr2=0x7fe5b4157210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:22.435 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:22.429+0000 7fe5b8f8c640 1 --2- 172.21.15.169:0/1249165225 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe5b4156d90 0x7fe5b4157210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:22.435 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:22.429+0000 7fe5b8f8c640 1 -- 172.21.15.169:0/1249165225 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5b406c660 con 0x7fe5a4004d00 2023-10-26T01:38:22.435 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:22.429+0000 7fe5b8f8c640 1 --2- 172.21.15.169:0/1249165225 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5a4004d00 0x7fe5b406b350 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fe5a800e990 tx=0x7fe5a800ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:22.435 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:22.429+0000 7fe5b17fa640 1 -- 172.21.15.169:0/1249165225 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5a800cb60 con 0x7fe5a4004d00 2023-10-26T01:38:22.435 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:22.429+0000 7fe5b17fa640 1 -- 172.21.15.169:0/1249165225 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe5a800cdb0 con 0x7fe5a4004d00 2023-10-26T01:38:22.436 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:22.429+0000 7fe5b9f8e640 1 -- 172.21.15.169:0/1249165225 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5b406c940 con 0x7fe5a4004d00 2023-10-26T01:38:22.436 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:22.429+0000 7fe5b17fa640 1 -- 172.21.15.169:0/1249165225 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5a8005280 con 0x7fe5a4004d00 2023-10-26T01:38:22.436 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:22.429+0000 7fe5b9f8e640 1 -- 172.21.15.169:0/1249165225 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5b40cb9f0 con 0x7fe5a4004d00 2023-10-26T01:38:22.437 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:22.431+0000 7fe5b9f8e640 1 -- 172.21.15.169:0/1249165225 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5b406cb60 con 0x7fe5a4004d00 2023-10-26T01:38:22.437 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:22.431+0000 7fe5b17fa640 1 -- 172.21.15.169:0/1249165225 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fe5a80053e0 con 0x7fe5a4004d00 2023-10-26T01:38:22.437 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:22.433+0000 7fe5b17fa640 1 --2- 172.21.15.169:0/1249165225 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe59806ae40 0x7fe59806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:22.437 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:22.433+0000 7fe5b17fa640 1 -- 172.21.15.169:0/1249165225 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe5a8089fe0 con 0x7fe5a4004d00 2023-10-26T01:38:22.438 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:22.433+0000 7fe5b3fff640 1 --2- 172.21.15.169:0/1249165225 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe59806ae40 0x7fe59806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:22.438 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:22.434+0000 7fe5b3fff640 1 --2- 172.21.15.169:0/1249165225 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe59806ae40 0x7fe59806d300 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fe5a0005fd0 tx=0x7fe5a0005850 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:22.439 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:22.437+0000 7fe5b17fa640 1 -- 172.21.15.169:0/1249165225 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe5a8054fa0 con 0x7fe5a4004d00 2023-10-26T01:38:22.656 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:22.654+0000 7fe5b9f8e640 1 -- 172.21.15.169:0/1249165225 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fe5b4073c10 con 0x7fe5a4004d00 2023-10-26T01:38:23.355 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:23.353+0000 7fe5b17fa640 1 -- 172.21.15.169:0/1249165225 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v67) v1 ==== 118+0+0 (secure 0 0 0) 0x7fe5a801f030 con 0x7fe5a4004d00 2023-10-26T01:38:23.359 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:23.356+0000 7fe5b9f8e640 1 -- 172.21.15.169:0/1249165225 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe59806ae40 msgr2=0x7fe59806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:23.359 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:23.356+0000 7fe5b9f8e640 1 --2- 172.21.15.169:0/1249165225 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe59806ae40 0x7fe59806d300 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fe5a0005fd0 tx=0x7fe5a0005850 comp rx=0 tx=0).stop 2023-10-26T01:38:23.359 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:23.356+0000 7fe5b9f8e640 1 -- 172.21.15.169:0/1249165225 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5a4004d00 msgr2=0x7fe5b406b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:23.359 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:23.356+0000 7fe5b9f8e640 1 --2- 172.21.15.169:0/1249165225 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5a4004d00 0x7fe5b406b350 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fe5a800e990 tx=0x7fe5a800ee50 comp rx=0 tx=0).stop 2023-10-26T01:38:23.359 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:23.356+0000 7fe5b9f8e640 1 -- 172.21.15.169:0/1249165225 shutdown_connections 2023-10-26T01:38:23.360 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:23.356+0000 7fe5b9f8e640 1 --2- 172.21.15.169:0/1249165225 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe59806ae40 0x7fe59806d300 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:23.360 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:23.356+0000 7fe5b9f8e640 1 --2- 172.21.15.169:0/1249165225 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe5b4156d90 0x7fe5b4157210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:23.360 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:23.356+0000 7fe5b9f8e640 1 --2- 172.21.15.169:0/1249165225 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe5b406b890 0x7fe5b4156850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:23.360 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:23.356+0000 7fe5b9f8e640 1 --2- 172.21.15.169:0/1249165225 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5a4004d00 0x7fe5b406b350 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:23.361 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:23.357+0000 7fe5b9f8e640 1 -- 172.21.15.169:0/1249165225 >> 172.21.15.169:0/1249165225 conn(0x7fe5b40af110 msgr2=0x7fe5b40acc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:23.361 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:23.357+0000 7fe5b9f8e640 1 -- 172.21.15.169:0/1249165225 shutdown_connections 2023-10-26T01:38:23.361 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:23.357+0000 7fe5b9f8e640 1 -- 172.21.15.169:0/1249165225 wait complete. 2023-10-26T01:38:23.379 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-26T01:38:23.380 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-26T01:38:23.603 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:23.600+0000 7fcee5864640 1 -- 172.21.15.169:0/1950083535 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcee01597c0 msgr2=0x7fcee015dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:23.603 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:23.600+0000 7fcee5864640 1 --2- 172.21.15.169:0/1950083535 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcee01597c0 0x7fcee015dbf0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fcec8005fd0 tx=0x7fcec802c240 comp rx=0 tx=0).stop 2023-10-26T01:38:23.603 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:23.600+0000 7fcee5864640 1 -- 172.21.15.169:0/1950083535 shutdown_connections 2023-10-26T01:38:23.603 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:23.600+0000 7fcee5864640 1 --2- 172.21.15.169:0/1950083535 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcee01597c0 0x7fcee015dbf0 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:23.604 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:23.600+0000 7fcee5864640 1 --2- 172.21.15.169:0/1950083535 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcee01593e0 0x7fcee00ac200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:23.604 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:23.600+0000 7fcee5864640 1 -- 172.21.15.169:0/1950083535 >> 172.21.15.169:0/1950083535 conn(0x7fcee0069930 msgr2=0x7fcee0069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:23.604 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:23.601+0000 7fcee5864640 1 -- 172.21.15.169:0/1950083535 shutdown_connections 2023-10-26T01:38:23.604 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:23.601+0000 7fcee5864640 1 -- 172.21.15.169:0/1950083535 wait complete. 2023-10-26T01:38:23.606 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:23.602+0000 7fcee5864640 1 Processor -- start 2023-10-26T01:38:23.607 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:23.602+0000 7fcee5864640 1 -- start start 2023-10-26T01:38:23.607 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:23.603+0000 7fcee5864640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcee00b0c80 0x7fcee01516d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:23.607 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:23.603+0000 7fcee5864640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcee01593e0 0x7fcee014deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:23.607 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:23.603+0000 7fcee5864640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcee01597c0 0x7fcee00aeb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:23.607 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:23.603+0000 7fcee5864640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcee0152370 con 0x7fcee01597c0 2023-10-26T01:38:23.608 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:23.603+0000 7fcee5864640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcee014e3f0 con 0x7fcee01593e0 2023-10-26T01:38:23.608 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:23.603+0000 7fcee5864640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcee00af070 con 0x7fcee00b0c80 2023-10-26T01:38:23.608 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:23.604+0000 7fcedf7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcee01597c0 0x7fcee00aeb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:23.608 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:23.604+0000 7fcedf7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcee01597c0 0x7fcee00aeb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:42022/0 (socket says 172.21.15.169:42022) 2023-10-26T01:38:23.608 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:23.604+0000 7fcedf7fe640 1 -- 172.21.15.169:0/2149783644 learned_addr learned my addr 172.21.15.169:0/2149783644 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:38:23.609 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:23.604+0000 7fcedeffd640 1 --2- 172.21.15.169:0/2149783644 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcee00b0c80 0x7fcee01516d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:23.609 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:23.604+0000 7fced67fc640 1 --2- 172.21.15.169:0/2149783644 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcee01593e0 0x7fcee014deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:23.609 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:23.604+0000 7fcedf7fe640 1 -- 172.21.15.169:0/2149783644 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcee00b0c80 msgr2=0x7fcee01516d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:23.609 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:23.604+0000 7fcedf7fe640 1 --2- 172.21.15.169:0/2149783644 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcee00b0c80 0x7fcee01516d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:23.609 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:23.604+0000 7fcedf7fe640 1 -- 172.21.15.169:0/2149783644 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcee01593e0 msgr2=0x7fcee014deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:23.610 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:23.604+0000 7fcedf7fe640 1 --2- 172.21.15.169:0/2149783644 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcee01593e0 0x7fcee014deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:23.610 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:23.604+0000 7fcedf7fe640 1 -- 172.21.15.169:0/2149783644 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcec8009900 con 0x7fcee01597c0 2023-10-26T01:38:23.610 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:23.604+0000 7fcedf7fe640 1 --2- 172.21.15.169:0/2149783644 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcee01597c0 0x7fcee00aeb30 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fced800c390 tx=0x7fced800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:23.610 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:23.604+0000 7fcedcff9640 1 -- 172.21.15.169:0/2149783644 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced80035f0 con 0x7fcee01597c0 2023-10-26T01:38:23.610 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:23.605+0000 7fcedcff9640 1 -- 172.21.15.169:0/2149783644 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fced8003750 con 0x7fcee01597c0 2023-10-26T01:38:23.611 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:23.605+0000 7fcedcff9640 1 -- 172.21.15.169:0/2149783644 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced8003b60 con 0x7fcee01597c0 2023-10-26T01:38:23.611 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:23.605+0000 7fcee5864640 1 -- 172.21.15.169:0/2149783644 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcee00ad410 con 0x7fcee01597c0 2023-10-26T01:38:23.611 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:23.605+0000 7fcee5864640 1 -- 172.21.15.169:0/2149783644 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcee0152090 con 0x7fcee01597c0 2023-10-26T01:38:23.612 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:23.606+0000 7fcee5864640 1 -- 172.21.15.169:0/2149783644 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcee0066e60 con 0x7fcee01597c0 2023-10-26T01:38:23.612 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:23.607+0000 7fcedcff9640 1 -- 172.21.15.169:0/2149783644 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fced8003cc0 con 0x7fcee01597c0 2023-10-26T01:38:23.612 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:23.608+0000 7fcedcff9640 1 --2- 172.21.15.169:0/2149783644 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcec006ae40 0x7fcec006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:23.612 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:23.608+0000 7fcedcff9640 1 -- 172.21.15.169:0/2149783644 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fced8088ae0 con 0x7fcee01597c0 2023-10-26T01:38:23.613 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:23.608+0000 7fcedeffd640 1 --2- 172.21.15.169:0/2149783644 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcec006ae40 0x7fcec006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:23.613 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:23.609+0000 7fcedeffd640 1 --2- 172.21.15.169:0/2149783644 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcec006ae40 0x7fcec006d300 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fced000a810 tx=0x7fced0008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:23.614 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:23.613+0000 7fcedcff9640 1 -- 172.21.15.169:0/2149783644 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fced8053af0 con 0x7fcee01597c0 2023-10-26T01:38:23.840 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:23.838+0000 7fcee5864640 1 -- 172.21.15.169:0/2149783644 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fcee00b1ef0 con 0x7fcee01597c0 2023-10-26T01:38:24.365 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:24.363+0000 7fcedcff9640 1 -- 172.21.15.169:0/2149783644 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v68) v1 ==== 122+0+0 (secure 0 0 0) 0x7fced8017020 con 0x7fcee01597c0 2023-10-26T01:38:24.369 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:24.367+0000 7fcee5864640 1 -- 172.21.15.169:0/2149783644 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcec006ae40 msgr2=0x7fcec006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:24.369 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:24.367+0000 7fcee5864640 1 --2- 172.21.15.169:0/2149783644 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcec006ae40 0x7fcec006d300 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fced000a810 tx=0x7fced0008040 comp rx=0 tx=0).stop 2023-10-26T01:38:24.369 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:24.367+0000 7fcee5864640 1 -- 172.21.15.169:0/2149783644 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcee01597c0 msgr2=0x7fcee00aeb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:24.370 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:24.367+0000 7fcee5864640 1 --2- 172.21.15.169:0/2149783644 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcee01597c0 0x7fcee00aeb30 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fced800c390 tx=0x7fced800c850 comp rx=0 tx=0).stop 2023-10-26T01:38:24.370 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:24.367+0000 7fcee5864640 1 -- 172.21.15.169:0/2149783644 shutdown_connections 2023-10-26T01:38:24.370 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:24.367+0000 7fcee5864640 1 --2- 172.21.15.169:0/2149783644 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcec006ae40 0x7fcec006d300 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:24.370 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:24.367+0000 7fcee5864640 1 --2- 172.21.15.169:0/2149783644 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcee01597c0 0x7fcee00aeb30 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:24.370 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:24.367+0000 7fcee5864640 1 --2- 172.21.15.169:0/2149783644 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcee01593e0 0x7fcee014deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:24.371 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:24.367+0000 7fcee5864640 1 --2- 172.21.15.169:0/2149783644 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcee00b0c80 0x7fcee01516d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:24.371 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:24.367+0000 7fcee5864640 1 -- 172.21.15.169:0/2149783644 >> 172.21.15.169:0/2149783644 conn(0x7fcee0069930 msgr2=0x7fcee015bb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:24.371 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:24.367+0000 7fcee5864640 1 -- 172.21.15.169:0/2149783644 shutdown_connections 2023-10-26T01:38:24.371 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:24.367+0000 7fcee5864640 1 -- 172.21.15.169:0/2149783644 wait complete. 2023-10-26T01:38:24.391 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-26T01:38:24.392 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T01:38:24.614 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:24.612+0000 7fcb404da640 1 -- 172.21.15.169:0/176642980 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb38159660 msgr2=0x7fcb380acae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:24.614 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:24.612+0000 7fcb404da640 1 --2- 172.21.15.169:0/176642980 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb38159660 0x7fcb380acae0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fcb34009930 tx=0x7fcb3402bdf0 comp rx=0 tx=0).stop 2023-10-26T01:38:24.615 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:24.612+0000 7fcb404da640 1 -- 172.21.15.169:0/176642980 shutdown_connections 2023-10-26T01:38:24.615 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:24.612+0000 7fcb404da640 1 --2- 172.21.15.169:0/176642980 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcb380ad020 0x7fcb38070810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:24.615 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:24.612+0000 7fcb404da640 1 --2- 172.21.15.169:0/176642980 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb38159660 0x7fcb380acae0 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:24.615 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:24.612+0000 7fcb404da640 1 -- 172.21.15.169:0/176642980 >> 172.21.15.169:0/176642980 conn(0x7fcb38069510 msgr2=0x7fcb380698f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:24.615 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:24.612+0000 7fcb404da640 1 -- 172.21.15.169:0/176642980 shutdown_connections 2023-10-26T01:38:24.616 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:24.612+0000 7fcb404da640 1 -- 172.21.15.169:0/176642980 wait complete. 2023-10-26T01:38:24.617 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:24.614+0000 7fcb404da640 1 Processor -- start 2023-10-26T01:38:24.618 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:24.614+0000 7fcb404da640 1 -- start start 2023-10-26T01:38:24.618 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:24.615+0000 7fcb404da640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcb38070d50 0x7fcb38145460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:24.618 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:24.615+0000 7fcb404da640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb380ad020 0x7fcb3809b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:24.618 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:24.615+0000 7fcb404da640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb38159660 0x7fcb3815baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:24.619 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:24.615+0000 7fcb404da640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb380b0950 con 0x7fcb380ad020 2023-10-26T01:38:24.619 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:24.615+0000 7fcb404da640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb381459a0 con 0x7fcb38159660 2023-10-26T01:38:24.619 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:24.615+0000 7fcb404da640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb380afdd0 con 0x7fcb38070d50 2023-10-26T01:38:24.619 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:24.615+0000 7fcb3da75640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb380ad020 0x7fcb3809b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:24.619 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:24.615+0000 7fcb3da75640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb380ad020 0x7fcb3809b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:34224/0 (socket says 172.21.15.169:34224) 2023-10-26T01:38:24.620 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:24.615+0000 7fcb3da75640 1 -- 172.21.15.169:0/2133571438 learned_addr learned my addr 172.21.15.169:0/2133571438 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:38:24.620 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:24.616+0000 7fcb3e276640 1 --2- 172.21.15.169:0/2133571438 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcb38070d50 0x7fcb38145460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:24.620 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:24.616+0000 7fcb3da75640 1 -- 172.21.15.169:0/2133571438 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcb38070d50 msgr2=0x7fcb38145460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:24.620 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:24.616+0000 7fcb3da75640 1 --2- 172.21.15.169:0/2133571438 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcb38070d50 0x7fcb38145460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:24.620 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:24.616+0000 7fcb3da75640 1 -- 172.21.15.169:0/2133571438 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb38159660 msgr2=0x7fcb3815baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:24.621 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:24.616+0000 7fcb3ea77640 1 --2- 172.21.15.169:0/2133571438 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb38159660 0x7fcb3815baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:24.621 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:24.616+0000 7fcb3da75640 1 --2- 172.21.15.169:0/2133571438 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb38159660 0x7fcb3815baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:24.621 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:24.616+0000 7fcb3da75640 1 -- 172.21.15.169:0/2133571438 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb34009560 con 0x7fcb380ad020 2023-10-26T01:38:24.621 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:24.616+0000 7fcb3e276640 1 --2- 172.21.15.169:0/2133571438 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcb38070d50 0x7fcb38145460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:38:24.621 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:24.616+0000 7fcb3ea77640 1 --2- 172.21.15.169:0/2133571438 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb38159660 0x7fcb3815baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:38:24.622 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:24.616+0000 7fcb3da75640 1 --2- 172.21.15.169:0/2133571438 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb380ad020 0x7fcb3809b170 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fcb2800d770 tx=0x7fcb2800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:24.623 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:24.616+0000 7fcb2effd640 1 -- 172.21.15.169:0/2133571438 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb28009c30 con 0x7fcb380ad020 2023-10-26T01:38:24.623 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:24.616+0000 7fcb2effd640 1 -- 172.21.15.169:0/2133571438 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fcb2800de80 con 0x7fcb380ad020 2023-10-26T01:38:24.623 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:24.616+0000 7fcb2effd640 1 -- 172.21.15.169:0/2133571438 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb28004260 con 0x7fcb380ad020 2023-10-26T01:38:24.623 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:24.616+0000 7fcb404da640 1 -- 172.21.15.169:0/2133571438 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb380b00b0 con 0x7fcb380ad020 2023-10-26T01:38:24.624 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:24.616+0000 7fcb404da640 1 -- 172.21.15.169:0/2133571438 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb380b0670 con 0x7fcb380ad020 2023-10-26T01:38:24.624 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:24.618+0000 7fcb404da640 1 -- 172.21.15.169:0/2133571438 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb3804db10 con 0x7fcb380ad020 2023-10-26T01:38:24.624 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:24.619+0000 7fcb2effd640 1 -- 172.21.15.169:0/2133571438 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fcb280035f0 con 0x7fcb380ad020 2023-10-26T01:38:24.624 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:24.619+0000 7fcb2effd640 1 --2- 172.21.15.169:0/2133571438 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcb1c06ae40 0x7fcb1c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:24.624 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:24.620+0000 7fcb2effd640 1 -- 172.21.15.169:0/2133571438 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcb28089820 con 0x7fcb380ad020 2023-10-26T01:38:24.625 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:24.620+0000 7fcb3e276640 1 --2- 172.21.15.169:0/2133571438 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcb1c06ae40 0x7fcb1c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:24.625 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:24.621+0000 7fcb3e276640 1 --2- 172.21.15.169:0/2133571438 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcb1c06ae40 0x7fcb1c06d300 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fcb34035000 tx=0x7fcb34004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:24.625 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:24.624+0000 7fcb2effd640 1 -- 172.21.15.169:0/2133571438 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcb280548c0 con 0x7fcb380ad020 2023-10-26T01:38:24.854 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:24.852+0000 7fcb404da640 1 -- 172.21.15.169:0/2133571438 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcb3806f930 con 0x7fcb380ad020 2023-10-26T01:38:24.855 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:24.853+0000 7fcb2effd640 1 -- 172.21.15.169:0/2133571438 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+4927 (secure 0 0 0) 0x7fcb28020090 con 0x7fcb380ad020 2023-10-26T01:38:24.856 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:38:24.856 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":68,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5917,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/3297412140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":3297412140},{"type":"v1","addr":"172.21.15.169:6835","nonce":3297412140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":5929,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/2575809940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2575809940},{"type":"v1","addr":"172.21.15.170:6836","nonce":2575809940}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":7761,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2632217991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2632217991},{"type":"v1","addr":"172.21.15.169:6833","nonce":2632217991}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T01:38:20.773556+0000","modified":"2023-10-26T01:38:23.839758+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7773},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7773":{"gid":7773,"name":"b","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6837/3615088922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":3615088922},{"type":"v1","addr":"172.21.15.170:6837","nonce":3615088922}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-10-26T01:38:24.857 INFO:teuthology.orchestra.run.smithi169.stderr:dumped fsmap epoch 68 2023-10-26T01:38:24.858 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:24.856+0000 7fcb404da640 1 -- 172.21.15.169:0/2133571438 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcb1c06ae40 msgr2=0x7fcb1c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:24.858 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:24.856+0000 7fcb404da640 1 --2- 172.21.15.169:0/2133571438 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcb1c06ae40 0x7fcb1c06d300 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fcb34035000 tx=0x7fcb34004730 comp rx=0 tx=0).stop 2023-10-26T01:38:24.859 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:24.856+0000 7fcb404da640 1 -- 172.21.15.169:0/2133571438 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb380ad020 msgr2=0x7fcb3809b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:24.859 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:24.856+0000 7fcb404da640 1 --2- 172.21.15.169:0/2133571438 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb380ad020 0x7fcb3809b170 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fcb2800d770 tx=0x7fcb2800dc30 comp rx=0 tx=0).stop 2023-10-26T01:38:24.859 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:24.856+0000 7fcb404da640 1 -- 172.21.15.169:0/2133571438 shutdown_connections 2023-10-26T01:38:24.859 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:24.856+0000 7fcb404da640 1 --2- 172.21.15.169:0/2133571438 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcb1c06ae40 0x7fcb1c06d300 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:24.860 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:24.856+0000 7fcb404da640 1 --2- 172.21.15.169:0/2133571438 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb380ad020 0x7fcb3809b170 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:24.860 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:24.856+0000 7fcb404da640 1 --2- 172.21.15.169:0/2133571438 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb38159660 0x7fcb3815baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:24.860 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:24.856+0000 7fcb404da640 1 --2- 172.21.15.169:0/2133571438 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcb38070d50 0x7fcb38145460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:24.860 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:24.856+0000 7fcb404da640 1 -- 172.21.15.169:0/2133571438 >> 172.21.15.169:0/2133571438 conn(0x7fcb38069510 msgr2=0x7fcb38155570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:24.860 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:24.856+0000 7fcb404da640 1 -- 172.21.15.169:0/2133571438 shutdown_connections 2023-10-26T01:38:24.860 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:24.857+0000 7fcb404da640 1 -- 172.21.15.169:0/2133571438 wait complete. 2023-10-26T01:38:24.876 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:38:25.096 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:25.094+0000 7f0dd0a10640 1 -- 172.21.15.169:0/417894279 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0db80039a0 msgr2=0x7f0db8003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:25.096 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:25.094+0000 7f0dd0a10640 1 --2- 172.21.15.169:0/417894279 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0db80039a0 0x7f0db8003dc0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc009b20 tx=0x7f0db80057f0 comp rx=0 tx=0).stop 2023-10-26T01:38:25.097 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:25.094+0000 7f0dcad9d640 1 -- 172.21.15.169:0/417894279 reap_dead start 2023-10-26T01:38:25.097 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:25.094+0000 7f0dd0a10640 1 -- 172.21.15.169:0/417894279 shutdown_connections 2023-10-26T01:38:25.097 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:25.094+0000 7f0dd0a10640 1 -- 172.21.15.169:0/417894279 >> 172.21.15.169:0/417894279 conn(0x7f0dcc069930 msgr2=0x7f0dcc069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:25.097 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:25.094+0000 7f0dd0a10640 1 -- 172.21.15.169:0/417894279 shutdown_connections 2023-10-26T01:38:25.097 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:25.094+0000 7f0dd0a10640 1 -- 172.21.15.169:0/417894279 wait complete. 2023-10-26T01:38:25.098 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:25.096+0000 7f0dd0a10640 1 Processor -- start 2023-10-26T01:38:25.098 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:25.096+0000 7f0dd0a10640 1 -- start start 2023-10-26T01:38:25.099 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:25.097+0000 7f0dd0a10640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0db80039a0 0x7f0dcc1f5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:25.099 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:25.097+0000 7f0dd0a10640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0dcc0ad910 0x7f0dcc1f2750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:25.099 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:25.097+0000 7f0dd0a10640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0dcc06ae10 0x7f0dcc1471f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:25.100 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:25.097+0000 7f0dd0a10640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dcc0afd90 con 0x7f0dcc0ad910 2023-10-26T01:38:25.100 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:25.097+0000 7f0dd0a10640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dcc1f6380 con 0x7f0db80039a0 2023-10-26T01:38:25.100 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:25.097+0000 7f0dd0a10640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dcc0ae400 con 0x7f0dcc06ae10 2023-10-26T01:38:25.100 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:25.097+0000 7f0dc3fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0dcc0ad910 0x7f0dcc1f2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:25.100 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:25.097+0000 7f0dc3fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0dcc0ad910 0x7f0dcc1f2750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:34246/0 (socket says 172.21.15.169:34246) 2023-10-26T01:38:25.101 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:25.097+0000 7f0dc3fff640 1 -- 172.21.15.169:0/1227610715 learned_addr learned my addr 172.21.15.169:0/1227610715 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:38:25.102 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:25.098+0000 7f0dca59c640 1 --2- 172.21.15.169:0/1227610715 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0db80039a0 0x7f0dcc1f5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:25.103 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:25.098+0000 7f0dc3fff640 1 -- 172.21.15.169:0/1227610715 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0dcc06ae10 msgr2=0x7f0dcc1471f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:25.103 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:25.098+0000 7f0dcad9d640 1 --2- 172.21.15.169:0/1227610715 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0dcc06ae10 0x7f0dcc1471f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:25.103 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:25.098+0000 7f0dc3fff640 1 --2- 172.21.15.169:0/1227610715 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0dcc06ae10 0x7f0dcc1471f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:25.103 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:25.098+0000 7f0dc3fff640 1 -- 172.21.15.169:0/1227610715 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0db80039a0 msgr2=0x7f0dcc1f5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:25.103 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:25.098+0000 7f0dc3fff640 1 --2- 172.21.15.169:0/1227610715 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0db80039a0 0x7f0dcc1f5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:25.104 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:25.098+0000 7f0dc3fff640 1 -- 172.21.15.169:0/1227610715 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dcc0ade40 con 0x7f0dcc0ad910 2023-10-26T01:38:25.104 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:25.098+0000 7f0dca59c640 1 --2- 172.21.15.169:0/1227610715 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0db80039a0 0x7f0dcc1f5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:38:25.104 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:25.098+0000 7f0dc3fff640 1 --2- 172.21.15.169:0/1227610715 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0dcc0ad910 0x7f0dcc1f2750 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f0db400d7a0 tx=0x7f0db400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:25.104 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:25.099+0000 7f0dc2ffd640 1 -- 172.21.15.169:0/1227610715 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db40052e0 con 0x7f0dcc0ad910 2023-10-26T01:38:25.104 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:25.099+0000 7f0dc2ffd640 1 -- 172.21.15.169:0/1227610715 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0db4005440 con 0x7f0dcc0ad910 2023-10-26T01:38:25.105 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:25.099+0000 7f0dc2ffd640 1 -- 172.21.15.169:0/1227610715 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db400b8c0 con 0x7f0dcc0ad910 2023-10-26T01:38:25.105 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:25.099+0000 7f0dd0a10640 1 -- 172.21.15.169:0/1227610715 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dcc0ae120 con 0x7f0dcc0ad910 2023-10-26T01:38:25.105 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:25.099+0000 7f0dd0a10640 1 -- 172.21.15.169:0/1227610715 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dcc1f2230 con 0x7f0dcc0ad910 2023-10-26T01:38:25.105 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:25.100+0000 7f0dd0a10640 1 -- 172.21.15.169:0/1227610715 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dcc1f2450 con 0x7f0dcc0ad910 2023-10-26T01:38:25.105 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:25.101+0000 7f0dc2ffd640 1 -- 172.21.15.169:0/1227610715 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f0db400ba20 con 0x7f0dcc0ad910 2023-10-26T01:38:25.106 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:25.102+0000 7f0dc2ffd640 1 --2- 172.21.15.169:0/1227610715 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f0dac06ae40 0x7f0dac06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:25.106 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:25.102+0000 7f0dc2ffd640 1 -- 172.21.15.169:0/1227610715 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0db4088ec0 con 0x7f0dcc0ad910 2023-10-26T01:38:25.107 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:25.102+0000 7f0dca59c640 1 --2- 172.21.15.169:0/1227610715 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f0dac06ae40 0x7f0dac06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:25.107 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:25.103+0000 7f0dca59c640 1 --2- 172.21.15.169:0/1227610715 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f0dac06ae40 0x7f0dac06d300 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc036000 tx=0x7f0dbc035040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:25.108 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:25.106+0000 7f0dc2ffd640 1 -- 172.21.15.169:0/1227610715 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0db4054000 con 0x7f0dcc0ad910 2023-10-26T01:38:25.298 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:25.297+0000 7f0dd0a10640 1 -- 172.21.15.169:0/1227610715 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0dcc04db10 con 0x7f0dcc0ad910 2023-10-26T01:38:25.301 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:25.300+0000 7f0dc2ffd640 1 -- 172.21.15.169:0/1227610715 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v134) v1 ==== 74+0+16576 (secure 0 0 0) 0x7f0db4019090 con 0x7f0dcc0ad910 2023-10-26T01:38:25.302 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:38:25.302 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":134,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:38:22.288993+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-10-26T01:38:18.617877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"133","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-10-26T01:38:19.793041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"134","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:38:25.304 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:25.302+0000 7f0dd0a10640 1 -- 172.21.15.169:0/1227610715 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f0dac06ae40 msgr2=0x7f0dac06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:25.305 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:25.302+0000 7f0dd0a10640 1 --2- 172.21.15.169:0/1227610715 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f0dac06ae40 0x7f0dac06d300 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc036000 tx=0x7f0dbc035040 comp rx=0 tx=0).stop 2023-10-26T01:38:25.305 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:25.302+0000 7f0dd0a10640 1 -- 172.21.15.169:0/1227610715 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0dcc0ad910 msgr2=0x7f0dcc1f2750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:25.305 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:25.302+0000 7f0dd0a10640 1 --2- 172.21.15.169:0/1227610715 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0dcc0ad910 0x7f0dcc1f2750 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f0db400d7a0 tx=0x7f0db400dc60 comp rx=0 tx=0).stop 2023-10-26T01:38:25.305 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:25.303+0000 7f0dd0a10640 1 -- 172.21.15.169:0/1227610715 shutdown_connections 2023-10-26T01:38:25.305 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:25.303+0000 7f0dd0a10640 1 --2- 172.21.15.169:0/1227610715 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f0dac06ae40 0x7f0dac06d300 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:25.306 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:25.303+0000 7f0dd0a10640 1 --2- 172.21.15.169:0/1227610715 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0dcc0ad910 0x7f0dcc1f2750 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:25.306 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:25.303+0000 7f0dd0a10640 1 --2- 172.21.15.169:0/1227610715 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0dcc06ae10 0x7f0dcc1471f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:25.306 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:25.303+0000 7f0dd0a10640 1 --2- 172.21.15.169:0/1227610715 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0db80039a0 0x7f0dcc1f5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:25.306 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:25.303+0000 7f0dd0a10640 1 -- 172.21.15.169:0/1227610715 >> 172.21.15.169:0/1227610715 conn(0x7f0dcc069930 msgr2=0x7f0dcc15c810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:25.307 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:25.303+0000 7f0dd0a10640 1 -- 172.21.15.169:0/1227610715 shutdown_connections 2023-10-26T01:38:25.307 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:25.303+0000 7f0dd0a10640 1 -- 172.21.15.169:0/1227610715 wait complete. 2023-10-26T01:38:25.326 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-26T01:38:25.327 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-26T01:38:25.555 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:25.552+0000 7ff5a4f25640 1 -- 172.21.15.169:0/2902534870 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff5a00cde30 msgr2=0x7ff5a0162270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:25.555 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:25.552+0000 7ff5a4f25640 1 --2- 172.21.15.169:0/2902534870 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff5a00cde30 0x7ff5a0162270 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7ff598005fd0 tx=0x7ff59802d1e0 comp rx=0 tx=0).stop 2023-10-26T01:38:25.555 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:25.553+0000 7ff5a4f25640 1 -- 172.21.15.169:0/2902534870 shutdown_connections 2023-10-26T01:38:25.555 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:25.553+0000 7ff5a4f25640 1 --2- 172.21.15.169:0/2902534870 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff5a00cde30 0x7ff5a0162270 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:25.556 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:25.553+0000 7ff5a4f25640 1 -- 172.21.15.169:0/2902534870 >> 172.21.15.169:0/2902534870 conn(0x7ff5a0069930 msgr2=0x7ff5a0069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:25.556 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:25.553+0000 7ff5a4f25640 1 -- 172.21.15.169:0/2902534870 shutdown_connections 2023-10-26T01:38:25.556 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:25.553+0000 7ff5a4f25640 1 -- 172.21.15.169:0/2902534870 wait complete. 2023-10-26T01:38:25.557 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:25.554+0000 7ff5a4f25640 1 Processor -- start 2023-10-26T01:38:25.557 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:25.555+0000 7ff5a4f25640 1 -- start start 2023-10-26T01:38:25.558 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:25.555+0000 7ff5a4f25640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff5a00cde30 0x7ff5a014d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:25.558 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:25.555+0000 7ff5a4f25640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff5a014e790 0x7ff5a0154d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:25.558 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:25.555+0000 7ff5a4f25640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff5a0158850 0x7ff5a00b1140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:25.558 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:25.555+0000 7ff5a4f25640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a0155270 con 0x7ff5a00cde30 2023-10-26T01:38:25.558 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:25.555+0000 7ff5a4f25640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a00b1680 con 0x7ff5a0158850 2023-10-26T01:38:25.559 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:25.555+0000 7ff5a4f25640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a00b18b0 con 0x7ff5a014e790 2023-10-26T01:38:25.559 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:25.556+0000 7ff59e59c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff5a00cde30 0x7ff5a014d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:25.559 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:25.556+0000 7ff59e59c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff5a00cde30 0x7ff5a014d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:34266/0 (socket says 172.21.15.169:34266) 2023-10-26T01:38:25.559 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:25.556+0000 7ff59e59c640 1 -- 172.21.15.169:0/2398428388 learned_addr learned my addr 172.21.15.169:0/2398428388 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:38:25.561 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:25.556+0000 7ff59ed9d640 1 --2- 172.21.15.169:0/2398428388 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff5a0158850 0x7ff5a00b1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:25.561 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:25.556+0000 7ff59e59c640 1 -- 172.21.15.169:0/2398428388 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff5a014e790 msgr2=0x7ff5a0154d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:25.561 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:25.556+0000 7ff597fff640 1 --2- 172.21.15.169:0/2398428388 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff5a014e790 0x7ff5a0154d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:25.562 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:25.556+0000 7ff59e59c640 1 --2- 172.21.15.169:0/2398428388 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff5a014e790 0x7ff5a0154d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:25.562 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:25.556+0000 7ff59e59c640 1 -- 172.21.15.169:0/2398428388 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff5a0158850 msgr2=0x7ff5a00b1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:25.562 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:25.556+0000 7ff59e59c640 1 --2- 172.21.15.169:0/2398428388 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff5a0158850 0x7ff5a00b1140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:25.562 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:25.556+0000 7ff59e59c640 1 -- 172.21.15.169:0/2398428388 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff59800aa10 con 0x7ff5a00cde30 2023-10-26T01:38:25.562 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:25.556+0000 7ff59ed9d640 1 --2- 172.21.15.169:0/2398428388 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff5a0158850 0x7ff5a00b1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:38:25.563 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:25.557+0000 7ff59e59c640 1 --2- 172.21.15.169:0/2398428388 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff5a00cde30 0x7ff5a014d8f0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7ff59000c4e0 tx=0x7ff59000c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:25.563 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:25.557+0000 7ff596ffd640 1 -- 172.21.15.169:0/2398428388 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff590005480 con 0x7ff5a00cde30 2023-10-26T01:38:25.563 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:25.557+0000 7ff596ffd640 1 -- 172.21.15.169:0/2398428388 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff590003720 con 0x7ff5a00cde30 2023-10-26T01:38:25.563 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:25.557+0000 7ff596ffd640 1 -- 172.21.15.169:0/2398428388 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff590003b30 con 0x7ff5a00cde30 2023-10-26T01:38:25.563 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:25.557+0000 7ff5a4f25640 1 -- 172.21.15.169:0/2398428388 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5a006edd0 con 0x7ff5a00cde30 2023-10-26T01:38:25.564 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:25.557+0000 7ff5a4f25640 1 -- 172.21.15.169:0/2398428388 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5a006f390 con 0x7ff5a00cde30 2023-10-26T01:38:25.564 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:25.558+0000 7ff5a4f25640 1 -- 172.21.15.169:0/2398428388 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5a004db10 con 0x7ff5a00cde30 2023-10-26T01:38:25.564 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:25.559+0000 7ff596ffd640 1 -- 172.21.15.169:0/2398428388 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ff590003c90 con 0x7ff5a00cde30 2023-10-26T01:38:25.565 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:25.560+0000 7ff596ffd640 1 --2- 172.21.15.169:0/2398428388 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff58006ae40 0x7ff58006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:25.565 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:25.560+0000 7ff596ffd640 1 -- 172.21.15.169:0/2398428388 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff590088cb0 con 0x7ff5a00cde30 2023-10-26T01:38:25.565 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:25.560+0000 7ff597fff640 1 --2- 172.21.15.169:0/2398428388 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff58006ae40 0x7ff58006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:25.565 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:25.562+0000 7ff597fff640 1 --2- 172.21.15.169:0/2398428388 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff58006ae40 0x7ff58006d300 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7ff5a006bd60 tx=0x7ff588008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:25.566 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:25.564+0000 7ff596ffd640 1 -- 172.21.15.169:0/2398428388 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff590053ea0 con 0x7ff5a00cde30 2023-10-26T01:38:25.747 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:25.745+0000 7ff5a4f25640 1 -- 172.21.15.169:0/2398428388 --> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff5a01627b0 con 0x7ff58006ae40 2023-10-26T01:38:25.769 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:25.767+0000 7ff596ffd640 1 -- 172.21.15.169:0/2398428388 <== mgr.4112 v2:172.21.15.170:6832/1187156918 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+278465 (secure 0 0 0) 0x7ff58800a380 con 0x7ff58006ae40 2023-10-26T01:38:25.773 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:38:25.774 INFO:teuthology.orchestra.run.smithi169.stderr:dumped all 2023-10-26T01:38:25.776 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:25.774+0000 7ff5a4f25640 1 -- 172.21.15.169:0/2398428388 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff58006ae40 msgr2=0x7ff58006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:25.776 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:25.774+0000 7ff5a4f25640 1 --2- 172.21.15.169:0/2398428388 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff58006ae40 0x7ff58006d300 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7ff5a006bd60 tx=0x7ff588008040 comp rx=0 tx=0).stop 2023-10-26T01:38:25.777 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:25.774+0000 7ff5a4f25640 1 -- 172.21.15.169:0/2398428388 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff5a00cde30 msgr2=0x7ff5a014d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:25.777 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:25.774+0000 7ff5a4f25640 1 --2- 172.21.15.169:0/2398428388 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff5a00cde30 0x7ff5a014d8f0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7ff59000c4e0 tx=0x7ff59000c9a0 comp rx=0 tx=0).stop 2023-10-26T01:38:25.777 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:25.774+0000 7ff5a4f25640 1 -- 172.21.15.169:0/2398428388 shutdown_connections 2023-10-26T01:38:25.777 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:25.774+0000 7ff5a4f25640 1 --2- 172.21.15.169:0/2398428388 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff58006ae40 0x7ff58006d300 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:25.778 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:25.774+0000 7ff5a4f25640 1 --2- 172.21.15.169:0/2398428388 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff5a0158850 0x7ff5a00b1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:25.778 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:25.774+0000 7ff5a4f25640 1 --2- 172.21.15.169:0/2398428388 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff5a014e790 0x7ff5a0154d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:25.778 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:25.774+0000 7ff5a4f25640 1 --2- 172.21.15.169:0/2398428388 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff5a00cde30 0x7ff5a014d8f0 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:25.778 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:25.774+0000 7ff5a4f25640 1 -- 172.21.15.169:0/2398428388 >> 172.21.15.169:0/2398428388 conn(0x7ff5a0069930 msgr2=0x7ff5a0069d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:25.779 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:25.774+0000 7ff5a4f25640 1 -- 172.21.15.169:0/2398428388 shutdown_connections 2023-10-26T01:38:25.779 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:25.774+0000 7ff5a4f25640 1 -- 172.21.15.169:0/2398428388 wait complete. 2023-10-26T01:38:25.795 INFO:teuthology.orchestra.run.smithi169.stdout:{"pg_ready":true,"pg_map":{"version":364,"stamp":"2023-10-26T01:38:24.294725+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":162,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":31,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":6834,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1555665,"statfs":{"total":1600000000,"available":1593001278,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.013822"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.312993+0000","last_change":"2023-10-26T01:38:20.312993+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.312993+0000","last_undegraded":"2023-10-26T01:38:20.312993+0000","last_fullsized":"2023-10-26T01:38:20.312993+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.309555+0000","last_change":"2023-10-26T01:38:20.309555+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.309555+0000","last_undegraded":"2023-10-26T01:38:20.309555+0000","last_fullsized":"2023-10-26T01:38:20.309555+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.313087+0000","last_change":"2023-10-26T01:38:20.313087+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.313087+0000","last_undegraded":"2023-10-26T01:38:20.313087+0000","last_fullsized":"2023-10-26T01:38:20.313087+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.310689+0000","last_change":"2023-10-26T01:38:20.310689+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.310689+0000","last_undegraded":"2023-10-26T01:38:20.310689+0000","last_fullsized":"2023-10-26T01:38:20.310689+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.304432+0000","last_change":"2023-10-26T01:38:20.304432+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.304432+0000","last_undegraded":"2023-10-26T01:38:20.304432+0000","last_fullsized":"2023-10-26T01:38:20.304432+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.315050+0000","last_change":"2023-10-26T01:38:20.315050+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.315050+0000","last_undegraded":"2023-10-26T01:38:20.315050+0000","last_fullsized":"2023-10-26T01:38:20.315050+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.322890+0000","last_change":"2023-10-26T01:38:20.322890+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.322890+0000","last_undegraded":"2023-10-26T01:38:20.322890+0000","last_fullsized":"2023-10-26T01:38:20.322890+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.311227+0000","last_change":"2023-10-26T01:38:20.311227+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.311227+0000","last_undegraded":"2023-10-26T01:38:20.311227+0000","last_fullsized":"2023-10-26T01:38:20.311227+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.310300+0000","last_change":"2023-10-26T01:38:20.310300+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.310300+0000","last_undegraded":"2023-10-26T01:38:20.310300+0000","last_fullsized":"2023-10-26T01:38:20.310300+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.309940+0000","last_change":"2023-10-26T01:38:20.309940+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.309940+0000","last_undegraded":"2023-10-26T01:38:20.309940+0000","last_fullsized":"2023-10-26T01:38:20.309940+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.309024+0000","last_change":"2023-10-26T01:38:20.309024+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.309024+0000","last_undegraded":"2023-10-26T01:38:20.309024+0000","last_fullsized":"2023-10-26T01:38:20.309024+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.309147+0000","last_change":"2023-10-26T01:38:20.309147+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.309147+0000","last_undegraded":"2023-10-26T01:38:20.309147+0000","last_fullsized":"2023-10-26T01:38:20.309147+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.312278+0000","last_change":"2023-10-26T01:38:20.312278+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.312278+0000","last_undegraded":"2023-10-26T01:38:20.312278+0000","last_fullsized":"2023-10-26T01:38:20.312278+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.314720+0000","last_change":"2023-10-26T01:38:20.314720+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.314720+0000","last_undegraded":"2023-10-26T01:38:20.314720+0000","last_fullsized":"2023-10-26T01:38:20.314720+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.315698+0000","last_change":"2023-10-26T01:38:20.315698+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.315698+0000","last_undegraded":"2023-10-26T01:38:20.315698+0000","last_fullsized":"2023-10-26T01:38:20.315698+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.312579+0000","last_change":"2023-10-26T01:38:20.312579+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.312579+0000","last_undegraded":"2023-10-26T01:38:20.312579+0000","last_fullsized":"2023-10-26T01:38:20.312579+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.308707+0000","last_change":"2023-10-26T01:38:20.308707+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.308707+0000","last_undegraded":"2023-10-26T01:38:20.308707+0000","last_fullsized":"2023-10-26T01:38:20.308707+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.315277+0000","last_change":"2023-10-26T01:38:20.315277+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.315277+0000","last_undegraded":"2023-10-26T01:38:20.315277+0000","last_fullsized":"2023-10-26T01:38:20.315277+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.312233+0000","last_change":"2023-10-26T01:38:20.312233+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.312233+0000","last_undegraded":"2023-10-26T01:38:20.312233+0000","last_fullsized":"2023-10-26T01:38:20.312233+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.310177+0000","last_change":"2023-10-26T01:38:20.310177+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.310177+0000","last_undegraded":"2023-10-26T01:38:20.310177+0000","last_fullsized":"2023-10-26T01:38:20.310177+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.308112+0000","last_change":"2023-10-26T01:38:20.308112+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.308112+0000","last_undegraded":"2023-10-26T01:38:20.308112+0000","last_fullsized":"2023-10-26T01:38:20.308112+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.317702+0000","last_change":"2023-10-26T01:38:21.314965+0000","last_active":"2023-10-26T01:38:21.317702+0000","last_peered":"2023-10-26T01:38:21.317702+0000","last_clean":"2023-10-26T01:38:21.317702+0000","last_became_active":"2023-10-26T01:38:21.314673+0000","last_became_peered":"2023-10-26T01:38:21.314673+0000","last_unstale":"2023-10-26T01:38:21.317702+0000","last_undegraded":"2023-10-26T01:38:21.317702+0000","last_fullsized":"2023-10-26T01:38:21.317702+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:33:14.986087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.304560+0000","last_change":"2023-10-26T01:38:20.304560+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.304560+0000","last_undegraded":"2023-10-26T01:38:20.304560+0000","last_fullsized":"2023-10-26T01:38:20.304560+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.320186+0000","last_change":"2023-10-26T01:38:21.320068+0000","last_active":"2023-10-26T01:38:21.320186+0000","last_peered":"2023-10-26T01:38:21.320186+0000","last_clean":"2023-10-26T01:38:21.320186+0000","last_became_active":"2023-10-26T01:38:21.319773+0000","last_became_peered":"2023-10-26T01:38:21.319773+0000","last_unstale":"2023-10-26T01:38:21.320186+0000","last_undegraded":"2023-10-26T01:38:21.320186+0000","last_fullsized":"2023-10-26T01:38:21.320186+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:03:39.495499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.306576+0000","last_change":"2023-10-26T01:38:20.306576+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.306576+0000","last_undegraded":"2023-10-26T01:38:20.306576+0000","last_fullsized":"2023-10-26T01:38:20.306576+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.312230+0000","last_change":"2023-10-26T01:38:20.312230+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.312230+0000","last_undegraded":"2023-10-26T01:38:20.312230+0000","last_fullsized":"2023-10-26T01:38:20.312230+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.316627+0000","last_change":"2023-10-26T01:38:20.316627+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.316627+0000","last_undegraded":"2023-10-26T01:38:20.316627+0000","last_fullsized":"2023-10-26T01:38:20.316627+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.320158+0000","last_change":"2023-10-26T01:38:21.319408+0000","last_active":"2023-10-26T01:38:21.320158+0000","last_peered":"2023-10-26T01:38:21.320158+0000","last_clean":"2023-10-26T01:38:21.320158+0000","last_became_active":"2023-10-26T01:38:21.318421+0000","last_became_peered":"2023-10-26T01:38:21.318421+0000","last_unstale":"2023-10-26T01:38:21.320158+0000","last_undegraded":"2023-10-26T01:38:21.320158+0000","last_fullsized":"2023-10-26T01:38:21.320158+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:23:42.697097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.306705+0000","last_change":"2023-10-26T01:38:20.306705+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.306705+0000","last_undegraded":"2023-10-26T01:38:20.306705+0000","last_fullsized":"2023-10-26T01:38:20.306705+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.316779+0000","last_change":"2023-10-26T01:38:20.316779+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.316779+0000","last_undegraded":"2023-10-26T01:38:20.316779+0000","last_fullsized":"2023-10-26T01:38:20.316779+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.311858+0000","last_change":"2023-10-26T01:38:20.311858+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.311858+0000","last_undegraded":"2023-10-26T01:38:20.311858+0000","last_fullsized":"2023-10-26T01:38:20.311858+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.316322+0000","last_change":"2023-10-26T01:38:20.316322+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.316322+0000","last_undegraded":"2023-10-26T01:38:20.316322+0000","last_fullsized":"2023-10-26T01:38:20.316322+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.316115+0000","last_change":"2023-10-26T01:38:20.316115+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.316115+0000","last_undegraded":"2023-10-26T01:38:20.316115+0000","last_fullsized":"2023-10-26T01:38:20.316115+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-26T01:38:20.322096+0000","last_change":"2023-10-26T01:38:20.322005+0000","last_active":"2023-10-26T01:38:20.322096+0000","last_peered":"2023-10-26T01:38:20.322096+0000","last_clean":"2023-10-26T01:38:20.322096+0000","last_became_active":"2023-10-26T01:38:20.320582+0000","last_became_peered":"2023-10-26T01:38:20.320582+0000","last_unstale":"2023-10-26T01:38:20.322096+0000","last_undegraded":"2023-10-26T01:38:20.322096+0000","last_fullsized":"2023-10-26T01:38:20.322096+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:19:17.861486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.314532+0000","last_change":"2023-10-26T01:38:20.314532+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.314532+0000","last_undegraded":"2023-10-26T01:38:20.314532+0000","last_fullsized":"2023-10-26T01:38:20.314532+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-26T01:38:20.326259+0000","last_change":"2023-10-26T01:38:20.326148+0000","last_active":"2023-10-26T01:38:20.326259+0000","last_peered":"2023-10-26T01:38:20.326259+0000","last_clean":"2023-10-26T01:38:20.326259+0000","last_became_active":"2023-10-26T01:38:20.325883+0000","last_became_peered":"2023-10-26T01:38:20.325883+0000","last_unstale":"2023-10-26T01:38:20.326259+0000","last_undegraded":"2023-10-26T01:38:20.326259+0000","last_fullsized":"2023-10-26T01:38:20.326259+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:55:08.910181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.314689+0000","last_change":"2023-10-26T01:38:20.314689+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.314689+0000","last_undegraded":"2023-10-26T01:38:20.314689+0000","last_fullsized":"2023-10-26T01:38:20.314689+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-26T01:38:20.324190+0000","last_change":"2023-10-26T01:38:20.321817+0000","last_active":"2023-10-26T01:38:20.324190+0000","last_peered":"2023-10-26T01:38:20.324190+0000","last_clean":"2023-10-26T01:38:20.324190+0000","last_became_active":"2023-10-26T01:38:20.321520+0000","last_became_peered":"2023-10-26T01:38:20.321520+0000","last_unstale":"2023-10-26T01:38:20.324190+0000","last_undegraded":"2023-10-26T01:38:20.324190+0000","last_fullsized":"2023-10-26T01:38:20.324190+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:32:00.607826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.316223+0000","last_change":"2023-10-26T01:38:20.316223+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.316223+0000","last_undegraded":"2023-10-26T01:38:20.316223+0000","last_fullsized":"2023-10-26T01:38:20.316223+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-26T01:38:20.325212+0000","last_change":"2023-10-26T01:38:20.325154+0000","last_active":"2023-10-26T01:38:20.325212+0000","last_peered":"2023-10-26T01:38:20.325212+0000","last_clean":"2023-10-26T01:38:20.325212+0000","last_became_active":"2023-10-26T01:38:20.324072+0000","last_became_peered":"2023-10-26T01:38:20.324072+0000","last_unstale":"2023-10-26T01:38:20.325212+0000","last_undegraded":"2023-10-26T01:38:20.325212+0000","last_fullsized":"2023-10-26T01:38:20.325212+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:18:21.690998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.315707+0000","last_change":"2023-10-26T01:38:20.315707+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.315707+0000","last_undegraded":"2023-10-26T01:38:20.315707+0000","last_fullsized":"2023-10-26T01:38:20.315707+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-26T01:38:20.324077+0000","last_change":"2023-10-26T01:38:20.323989+0000","last_active":"2023-10-26T01:38:20.324077+0000","last_peered":"2023-10-26T01:38:20.324077+0000","last_clean":"2023-10-26T01:38:20.324077+0000","last_became_active":"2023-10-26T01:38:20.323646+0000","last_became_peered":"2023-10-26T01:38:20.323646+0000","last_unstale":"2023-10-26T01:38:20.324077+0000","last_undegraded":"2023-10-26T01:38:20.324077+0000","last_fullsized":"2023-10-26T01:38:20.324077+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:48:31.373334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.305499+0000","last_change":"2023-10-26T01:38:20.305499+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.305499+0000","last_undegraded":"2023-10-26T01:38:20.305499+0000","last_fullsized":"2023-10-26T01:38:20.305499+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"133'1","reported_seq":12,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.335571+0000","last_change":"2023-10-26T01:38:20.320626+0000","last_active":"2023-10-26T01:38:21.335571+0000","last_peered":"2023-10-26T01:38:21.335571+0000","last_clean":"2023-10-26T01:38:21.335571+0000","last_became_active":"2023-10-26T01:38:20.320400+0000","last_became_peered":"2023-10-26T01:38:20.320400+0000","last_unstale":"2023-10-26T01:38:21.335571+0000","last_undegraded":"2023-10-26T01:38:21.335571+0000","last_fullsized":"2023-10-26T01:38:21.335571+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:31:15.331694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.309593+0000","last_change":"2023-10-26T01:38:20.309593+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.309593+0000","last_undegraded":"2023-10-26T01:38:20.309593+0000","last_fullsized":"2023-10-26T01:38:20.309593+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-26T01:38:20.318013+0000","last_change":"2023-10-26T01:38:20.317888+0000","last_active":"2023-10-26T01:38:20.318013+0000","last_peered":"2023-10-26T01:38:20.318013+0000","last_clean":"2023-10-26T01:38:20.318013+0000","last_became_active":"2023-10-26T01:38:20.317130+0000","last_became_peered":"2023-10-26T01:38:20.317130+0000","last_unstale":"2023-10-26T01:38:20.318013+0000","last_undegraded":"2023-10-26T01:38:20.318013+0000","last_fullsized":"2023-10-26T01:38:20.318013+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:59:26.990814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.317096+0000","last_change":"2023-10-26T01:38:20.317096+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.317096+0000","last_undegraded":"2023-10-26T01:38:20.317096+0000","last_fullsized":"2023-10-26T01:38:20.317096+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-26T01:38:20.316112+0000","last_change":"2023-10-26T01:38:20.314210+0000","last_active":"2023-10-26T01:38:20.316112+0000","last_peered":"2023-10-26T01:38:20.316112+0000","last_clean":"2023-10-26T01:38:20.316112+0000","last_became_active":"2023-10-26T01:38:20.313976+0000","last_became_peered":"2023-10-26T01:38:20.313976+0000","last_unstale":"2023-10-26T01:38:20.316112+0000","last_undegraded":"2023-10-26T01:38:20.316112+0000","last_fullsized":"2023-10-26T01:38:20.316112+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:44:56.136940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.316088+0000","last_change":"2023-10-26T01:38:20.316088+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.316088+0000","last_undegraded":"2023-10-26T01:38:20.316088+0000","last_fullsized":"2023-10-26T01:38:20.316088+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-26T01:38:20.325081+0000","last_change":"2023-10-26T01:38:20.325020+0000","last_active":"2023-10-26T01:38:20.325081+0000","last_peered":"2023-10-26T01:38:20.325081+0000","last_clean":"2023-10-26T01:38:20.325081+0000","last_became_active":"2023-10-26T01:38:20.323713+0000","last_became_peered":"2023-10-26T01:38:20.323713+0000","last_unstale":"2023-10-26T01:38:20.325081+0000","last_undegraded":"2023-10-26T01:38:20.325081+0000","last_fullsized":"2023-10-26T01:38:20.325081+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:19:17.861486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.312710+0000","last_change":"2023-10-26T01:38:20.312710+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.312710+0000","last_undegraded":"2023-10-26T01:38:20.312710+0000","last_fullsized":"2023-10-26T01:38:20.312710+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.293962+0000","last_change":"2023-10-26T01:38:20.322090+0000","last_active":"2023-10-26T01:38:21.293962+0000","last_peered":"2023-10-26T01:38:21.293962+0000","last_clean":"2023-10-26T01:38:21.293962+0000","last_became_active":"2023-10-26T01:38:20.321451+0000","last_became_peered":"2023-10-26T01:38:20.321451+0000","last_unstale":"2023-10-26T01:38:21.293962+0000","last_undegraded":"2023-10-26T01:38:21.293962+0000","last_fullsized":"2023-10-26T01:38:21.293962+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:45:01.083199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.302056+0000","last_change":"2023-10-26T01:38:21.301523+0000","last_active":"2023-10-26T01:38:21.302056+0000","last_peered":"2023-10-26T01:38:21.302056+0000","last_clean":"2023-10-26T01:38:21.302056+0000","last_became_active":"2023-10-26T01:38:21.300901+0000","last_became_peered":"2023-10-26T01:38:21.300901+0000","last_unstale":"2023-10-26T01:38:21.302056+0000","last_undegraded":"2023-10-26T01:38:21.302056+0000","last_fullsized":"2023-10-26T01:38:21.302056+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:21:55.273677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-26T01:38:20.319415+0000","last_change":"2023-10-26T01:38:20.319263+0000","last_active":"2023-10-26T01:38:20.319415+0000","last_peered":"2023-10-26T01:38:20.319415+0000","last_clean":"2023-10-26T01:38:20.319415+0000","last_became_active":"2023-10-26T01:38:20.319022+0000","last_became_peered":"2023-10-26T01:38:20.319022+0000","last_unstale":"2023-10-26T01:38:20.319415+0000","last_undegraded":"2023-10-26T01:38:20.319415+0000","last_fullsized":"2023-10-26T01:38:20.319415+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:19:16.513935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.310881+0000","last_change":"2023-10-26T01:38:20.310881+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.310881+0000","last_undegraded":"2023-10-26T01:38:20.310881+0000","last_fullsized":"2023-10-26T01:38:20.310881+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-26T01:38:20.316131+0000","last_change":"2023-10-26T01:38:20.314532+0000","last_active":"2023-10-26T01:38:20.316131+0000","last_peered":"2023-10-26T01:38:20.316131+0000","last_clean":"2023-10-26T01:38:20.316131+0000","last_became_active":"2023-10-26T01:38:20.314073+0000","last_became_peered":"2023-10-26T01:38:20.314073+0000","last_unstale":"2023-10-26T01:38:20.316131+0000","last_undegraded":"2023-10-26T01:38:20.316131+0000","last_fullsized":"2023-10-26T01:38:20.316131+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:00:59.371192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.310604+0000","last_change":"2023-10-26T01:38:21.310272+0000","last_active":"2023-10-26T01:38:21.310604+0000","last_peered":"2023-10-26T01:38:21.310604+0000","last_clean":"2023-10-26T01:38:21.310604+0000","last_became_active":"2023-10-26T01:38:21.309297+0000","last_became_peered":"2023-10-26T01:38:21.309297+0000","last_unstale":"2023-10-26T01:38:21.310604+0000","last_undegraded":"2023-10-26T01:38:21.310604+0000","last_fullsized":"2023-10-26T01:38:21.310604+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:07:03.778722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-26T01:38:20.324089+0000","last_change":"2023-10-26T01:38:20.323897+0000","last_active":"2023-10-26T01:38:20.324089+0000","last_peered":"2023-10-26T01:38:20.324089+0000","last_clean":"2023-10-26T01:38:20.324089+0000","last_became_active":"2023-10-26T01:38:20.323527+0000","last_became_peered":"2023-10-26T01:38:20.323527+0000","last_unstale":"2023-10-26T01:38:20.324089+0000","last_undegraded":"2023-10-26T01:38:20.324089+0000","last_fullsized":"2023-10-26T01:38:20.324089+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:12:30.722215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.317765+0000","last_change":"2023-10-26T01:38:21.317158+0000","last_active":"2023-10-26T01:38:21.317765+0000","last_peered":"2023-10-26T01:38:21.317765+0000","last_clean":"2023-10-26T01:38:21.317765+0000","last_became_active":"2023-10-26T01:38:21.316782+0000","last_became_peered":"2023-10-26T01:38:21.316782+0000","last_unstale":"2023-10-26T01:38:21.317765+0000","last_undegraded":"2023-10-26T01:38:21.317765+0000","last_fullsized":"2023-10-26T01:38:21.317765+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:38:50.874604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-26T01:38:20.324414+0000","last_change":"2023-10-26T01:38:20.323991+0000","last_active":"2023-10-26T01:38:20.324414+0000","last_peered":"2023-10-26T01:38:20.324414+0000","last_clean":"2023-10-26T01:38:20.324414+0000","last_became_active":"2023-10-26T01:38:20.323242+0000","last_became_peered":"2023-10-26T01:38:20.323242+0000","last_unstale":"2023-10-26T01:38:20.324414+0000","last_undegraded":"2023-10-26T01:38:20.324414+0000","last_fullsized":"2023-10-26T01:38:20.324414+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:00:35.003415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.309119+0000","last_change":"2023-10-26T01:38:21.308916+0000","last_active":"2023-10-26T01:38:21.309119+0000","last_peered":"2023-10-26T01:38:21.309119+0000","last_clean":"2023-10-26T01:38:21.309119+0000","last_became_active":"2023-10-26T01:38:21.308364+0000","last_became_peered":"2023-10-26T01:38:21.308364+0000","last_unstale":"2023-10-26T01:38:21.309119+0000","last_undegraded":"2023-10-26T01:38:21.309119+0000","last_fullsized":"2023-10-26T01:38:21.309119+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:18:02.522210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-26T01:38:20.324208+0000","last_change":"2023-10-26T01:38:20.322830+0000","last_active":"2023-10-26T01:38:20.324208+0000","last_peered":"2023-10-26T01:38:20.324208+0000","last_clean":"2023-10-26T01:38:20.324208+0000","last_became_active":"2023-10-26T01:38:20.318642+0000","last_became_peered":"2023-10-26T01:38:20.318642+0000","last_unstale":"2023-10-26T01:38:20.324208+0000","last_undegraded":"2023-10-26T01:38:20.324208+0000","last_fullsized":"2023-10-26T01:38:20.324208+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:12:30.722215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.303953+0000","last_change":"2023-10-26T01:38:20.303953+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.303953+0000","last_undegraded":"2023-10-26T01:38:20.303953+0000","last_fullsized":"2023-10-26T01:38:20.303953+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-26T01:38:20.322630+0000","last_change":"2023-10-26T01:38:20.322530+0000","last_active":"2023-10-26T01:38:20.322630+0000","last_peered":"2023-10-26T01:38:20.322630+0000","last_clean":"2023-10-26T01:38:20.322630+0000","last_became_active":"2023-10-26T01:38:20.322312+0000","last_became_peered":"2023-10-26T01:38:20.322312+0000","last_unstale":"2023-10-26T01:38:20.322630+0000","last_undegraded":"2023-10-26T01:38:20.322630+0000","last_fullsized":"2023-10-26T01:38:20.322630+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:00:41.104678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.315008+0000","last_change":"2023-10-26T01:38:20.315008+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.315008+0000","last_undegraded":"2023-10-26T01:38:20.315008+0000","last_fullsized":"2023-10-26T01:38:20.315008+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-26T01:38:20.318810+0000","last_change":"2023-10-26T01:38:20.318723+0000","last_active":"2023-10-26T01:38:20.318810+0000","last_peered":"2023-10-26T01:38:20.318810+0000","last_clean":"2023-10-26T01:38:20.318810+0000","last_became_active":"2023-10-26T01:38:20.317546+0000","last_became_peered":"2023-10-26T01:38:20.317546+0000","last_unstale":"2023-10-26T01:38:20.318810+0000","last_undegraded":"2023-10-26T01:38:20.318810+0000","last_fullsized":"2023-10-26T01:38:20.318810+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:05:00.823929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.314011+0000","last_change":"2023-10-26T01:38:20.314011+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.314011+0000","last_undegraded":"2023-10-26T01:38:20.314011+0000","last_fullsized":"2023-10-26T01:38:20.314011+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-26T01:38:20.323891+0000","last_change":"2023-10-26T01:38:20.323687+0000","last_active":"2023-10-26T01:38:20.323891+0000","last_peered":"2023-10-26T01:38:20.323891+0000","last_clean":"2023-10-26T01:38:20.323891+0000","last_became_active":"2023-10-26T01:38:20.322524+0000","last_became_peered":"2023-10-26T01:38:20.322524+0000","last_unstale":"2023-10-26T01:38:20.323891+0000","last_undegraded":"2023-10-26T01:38:20.323891+0000","last_fullsized":"2023-10-26T01:38:20.323891+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:10:48.737505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.312099+0000","last_change":"2023-10-26T01:38:20.312099+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.312099+0000","last_undegraded":"2023-10-26T01:38:20.312099+0000","last_fullsized":"2023-10-26T01:38:20.312099+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-26T01:38:20.322573+0000","last_change":"2023-10-26T01:38:20.322266+0000","last_active":"2023-10-26T01:38:20.322573+0000","last_peered":"2023-10-26T01:38:20.322573+0000","last_clean":"2023-10-26T01:38:20.322573+0000","last_became_active":"2023-10-26T01:38:20.315231+0000","last_became_peered":"2023-10-26T01:38:20.315231+0000","last_unstale":"2023-10-26T01:38:20.322573+0000","last_undegraded":"2023-10-26T01:38:20.322573+0000","last_fullsized":"2023-10-26T01:38:20.322573+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:36:33.502627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.310535+0000","last_change":"2023-10-26T01:38:20.310535+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.310535+0000","last_undegraded":"2023-10-26T01:38:20.310535+0000","last_fullsized":"2023-10-26T01:38:20.310535+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-26T01:38:20.320042+0000","last_change":"2023-10-26T01:38:20.319975+0000","last_active":"2023-10-26T01:38:20.320042+0000","last_peered":"2023-10-26T01:38:20.320042+0000","last_clean":"2023-10-26T01:38:20.320042+0000","last_became_active":"2023-10-26T01:38:20.319754+0000","last_became_peered":"2023-10-26T01:38:20.319754+0000","last_unstale":"2023-10-26T01:38:20.320042+0000","last_undegraded":"2023-10-26T01:38:20.320042+0000","last_fullsized":"2023-10-26T01:38:20.320042+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:18:21.690998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.315394+0000","last_change":"2023-10-26T01:38:20.315394+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.315394+0000","last_undegraded":"2023-10-26T01:38:20.315394+0000","last_fullsized":"2023-10-26T01:38:20.315394+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-26T01:38:20.325008+0000","last_change":"2023-10-26T01:38:20.323205+0000","last_active":"2023-10-26T01:38:20.325008+0000","last_peered":"2023-10-26T01:38:20.325008+0000","last_clean":"2023-10-26T01:38:20.325008+0000","last_became_active":"2023-10-26T01:38:20.322992+0000","last_became_peered":"2023-10-26T01:38:20.322992+0000","last_unstale":"2023-10-26T01:38:20.325008+0000","last_undegraded":"2023-10-26T01:38:20.325008+0000","last_fullsized":"2023-10-26T01:38:20.325008+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:01:26.691609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.311293+0000","last_change":"2023-10-26T01:38:20.311293+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.311293+0000","last_undegraded":"2023-10-26T01:38:20.311293+0000","last_fullsized":"2023-10-26T01:38:20.311293+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-26T01:38:20.321263+0000","last_change":"2023-10-26T01:38:20.320536+0000","last_active":"2023-10-26T01:38:20.321263+0000","last_peered":"2023-10-26T01:38:20.321263+0000","last_clean":"2023-10-26T01:38:20.321263+0000","last_became_active":"2023-10-26T01:38:20.320277+0000","last_became_peered":"2023-10-26T01:38:20.320277+0000","last_unstale":"2023-10-26T01:38:20.321263+0000","last_undegraded":"2023-10-26T01:38:20.321263+0000","last_fullsized":"2023-10-26T01:38:20.321263+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:32:00.607826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.322832+0000","last_change":"2023-10-26T01:38:21.322582+0000","last_active":"2023-10-26T01:38:21.322832+0000","last_peered":"2023-10-26T01:38:21.322832+0000","last_clean":"2023-10-26T01:38:21.322832+0000","last_became_active":"2023-10-26T01:38:21.321971+0000","last_became_peered":"2023-10-26T01:38:21.321971+0000","last_unstale":"2023-10-26T01:38:21.322832+0000","last_undegraded":"2023-10-26T01:38:21.322832+0000","last_fullsized":"2023-10-26T01:38:21.322832+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:44:44.680316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-26T01:38:20.324540+0000","last_change":"2023-10-26T01:38:20.324450+0000","last_active":"2023-10-26T01:38:20.324540+0000","last_peered":"2023-10-26T01:38:20.324540+0000","last_clean":"2023-10-26T01:38:20.324540+0000","last_became_active":"2023-10-26T01:38:20.324194+0000","last_became_peered":"2023-10-26T01:38:20.324194+0000","last_unstale":"2023-10-26T01:38:20.324540+0000","last_undegraded":"2023-10-26T01:38:20.324540+0000","last_fullsized":"2023-10-26T01:38:20.324540+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:29:51.785956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.313704+0000","last_change":"2023-10-26T01:38:20.313704+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.313704+0000","last_undegraded":"2023-10-26T01:38:20.313704+0000","last_fullsized":"2023-10-26T01:38:20.313704+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"133'2","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.338835+0000","last_change":"2023-10-26T01:38:20.315954+0000","last_active":"2023-10-26T01:38:21.338835+0000","last_peered":"2023-10-26T01:38:21.338835+0000","last_clean":"2023-10-26T01:38:21.338835+0000","last_became_active":"2023-10-26T01:38:20.315727+0000","last_became_peered":"2023-10-26T01:38:20.315727+0000","last_unstale":"2023-10-26T01:38:21.338835+0000","last_undegraded":"2023-10-26T01:38:21.338835+0000","last_fullsized":"2023-10-26T01:38:21.338835+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:01:47.759723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.308802+0000","last_change":"2023-10-26T01:38:20.308802+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.308802+0000","last_undegraded":"2023-10-26T01:38:20.308802+0000","last_fullsized":"2023-10-26T01:38:20.308802+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-26T01:38:20.325709+0000","last_change":"2023-10-26T01:38:20.325614+0000","last_active":"2023-10-26T01:38:20.325709+0000","last_peered":"2023-10-26T01:38:20.325709+0000","last_clean":"2023-10-26T01:38:20.325709+0000","last_became_active":"2023-10-26T01:38:20.325288+0000","last_became_peered":"2023-10-26T01:38:20.325288+0000","last_unstale":"2023-10-26T01:38:20.325709+0000","last_undegraded":"2023-10-26T01:38:20.325709+0000","last_fullsized":"2023-10-26T01:38:20.325709+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:48:44.256103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.308821+0000","last_change":"2023-10-26T01:38:20.308821+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.308821+0000","last_undegraded":"2023-10-26T01:38:20.308821+0000","last_fullsized":"2023-10-26T01:38:20.308821+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-26T01:38:20.320876+0000","last_change":"2023-10-26T01:38:20.320712+0000","last_active":"2023-10-26T01:38:20.320876+0000","last_peered":"2023-10-26T01:38:20.320876+0000","last_clean":"2023-10-26T01:38:20.320876+0000","last_became_active":"2023-10-26T01:38:20.319884+0000","last_became_peered":"2023-10-26T01:38:20.319884+0000","last_unstale":"2023-10-26T01:38:20.320876+0000","last_undegraded":"2023-10-26T01:38:20.320876+0000","last_fullsized":"2023-10-26T01:38:20.320876+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T01:50:21.256149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.313034+0000","last_change":"2023-10-26T01:38:20.313034+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.313034+0000","last_undegraded":"2023-10-26T01:38:20.313034+0000","last_fullsized":"2023-10-26T01:38:20.313034+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.294223+0000","last_change":"2023-10-26T01:38:20.319596+0000","last_active":"2023-10-26T01:38:21.294223+0000","last_peered":"2023-10-26T01:38:21.294223+0000","last_clean":"2023-10-26T01:38:21.294223+0000","last_became_active":"2023-10-26T01:38:20.319376+0000","last_became_peered":"2023-10-26T01:38:20.319376+0000","last_unstale":"2023-10-26T01:38:21.294223+0000","last_undegraded":"2023-10-26T01:38:21.294223+0000","last_fullsized":"2023-10-26T01:38:21.294223+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:59:26.990814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.304704+0000","last_change":"2023-10-26T01:38:20.304704+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.304704+0000","last_undegraded":"2023-10-26T01:38:20.304704+0000","last_fullsized":"2023-10-26T01:38:20.304704+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-26T01:38:20.324168+0000","last_change":"2023-10-26T01:38:20.324083+0000","last_active":"2023-10-26T01:38:20.324168+0000","last_peered":"2023-10-26T01:38:20.324168+0000","last_clean":"2023-10-26T01:38:20.324168+0000","last_became_active":"2023-10-26T01:38:20.320737+0000","last_became_peered":"2023-10-26T01:38:20.320737+0000","last_unstale":"2023-10-26T01:38:20.324168+0000","last_undegraded":"2023-10-26T01:38:20.324168+0000","last_fullsized":"2023-10-26T01:38:20.324168+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:05:00.823929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.304194+0000","last_change":"2023-10-26T01:38:20.304194+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.304194+0000","last_undegraded":"2023-10-26T01:38:20.304194+0000","last_fullsized":"2023-10-26T01:38:20.304194+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-26T01:38:20.322096+0000","last_change":"2023-10-26T01:38:20.321433+0000","last_active":"2023-10-26T01:38:20.322096+0000","last_peered":"2023-10-26T01:38:20.322096+0000","last_clean":"2023-10-26T01:38:20.322096+0000","last_became_active":"2023-10-26T01:38:20.321136+0000","last_became_peered":"2023-10-26T01:38:20.321136+0000","last_unstale":"2023-10-26T01:38:20.322096+0000","last_undegraded":"2023-10-26T01:38:20.322096+0000","last_fullsized":"2023-10-26T01:38:20.322096+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:01:47.759723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.311489+0000","last_change":"2023-10-26T01:38:20.311489+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.311489+0000","last_undegraded":"2023-10-26T01:38:20.311489+0000","last_fullsized":"2023-10-26T01:38:20.311489+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-26T01:38:20.324966+0000","last_change":"2023-10-26T01:38:20.324901+0000","last_active":"2023-10-26T01:38:20.324966+0000","last_peered":"2023-10-26T01:38:20.324966+0000","last_clean":"2023-10-26T01:38:20.324966+0000","last_became_active":"2023-10-26T01:38:20.324441+0000","last_became_peered":"2023-10-26T01:38:20.324441+0000","last_unstale":"2023-10-26T01:38:20.324966+0000","last_undegraded":"2023-10-26T01:38:20.324966+0000","last_fullsized":"2023-10-26T01:38:20.324966+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:48:31.373334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.314840+0000","last_change":"2023-10-26T01:38:20.314840+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.314840+0000","last_undegraded":"2023-10-26T01:38:20.314840+0000","last_fullsized":"2023-10-26T01:38:20.314840+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-26T01:38:20.320980+0000","last_change":"2023-10-26T01:38:20.320871+0000","last_active":"2023-10-26T01:38:20.320980+0000","last_peered":"2023-10-26T01:38:20.320980+0000","last_clean":"2023-10-26T01:38:20.320980+0000","last_became_active":"2023-10-26T01:38:20.320224+0000","last_became_peered":"2023-10-26T01:38:20.320224+0000","last_unstale":"2023-10-26T01:38:20.320980+0000","last_undegraded":"2023-10-26T01:38:20.320980+0000","last_fullsized":"2023-10-26T01:38:20.320980+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:51:41.698100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.322813+0000","last_change":"2023-10-26T01:38:21.321771+0000","last_active":"2023-10-26T01:38:21.322813+0000","last_peered":"2023-10-26T01:38:21.322813+0000","last_clean":"2023-10-26T01:38:21.322813+0000","last_became_active":"2023-10-26T01:38:21.321349+0000","last_became_peered":"2023-10-26T01:38:21.321349+0000","last_unstale":"2023-10-26T01:38:21.322813+0000","last_undegraded":"2023-10-26T01:38:21.322813+0000","last_fullsized":"2023-10-26T01:38:21.322813+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:22:49.920143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-26T01:38:20.322070+0000","last_change":"2023-10-26T01:38:20.321951+0000","last_active":"2023-10-26T01:38:20.322070+0000","last_peered":"2023-10-26T01:38:20.322070+0000","last_clean":"2023-10-26T01:38:20.322070+0000","last_became_active":"2023-10-26T01:38:20.310341+0000","last_became_peered":"2023-10-26T01:38:20.310341+0000","last_unstale":"2023-10-26T01:38:20.322070+0000","last_undegraded":"2023-10-26T01:38:20.322070+0000","last_fullsized":"2023-10-26T01:38:20.322070+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:54:02.310203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":276,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-26T01:38:19.284336+0000","last_change":"2023-10-26T01:30:34.883484+0000","last_active":"2023-10-26T01:38:19.284336+0000","last_peered":"2023-10-26T01:38:19.284336+0000","last_clean":"2023-10-26T01:38:19.284336+0000","last_became_active":"2023-10-26T01:30:34.883071+0000","last_became_peered":"2023-10-26T01:30:34.883071+0000","last_unstale":"2023-10-26T01:38:19.284336+0000","last_undegraded":"2023-10-26T01:38:19.284336+0000","last_fullsized":"2023-10-26T01:38:19.284336+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:32.890096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:32.890096+0000","last_clean_scrub_stamp":"2023-10-26T01:30:32.890096+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:26:31.879452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":14,"seq":60129542241,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":719,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194592,"statfs":{"total":200000000,"available":199262922,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.79200000000000004,"15min":0.78600000000000003},"min":{"1min":0.44900000000000001,"5min":0.35599999999999998,"15min":0.33000000000000002},"max":{"1min":1.6319999999999999,"5min":1.6319999999999999,"15min":2.2549999999999999},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.876,"15min":0.88800000000000001},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.236,"5min":1.5800000000000001,"15min":1.8200000000000001},"last":0.94099999999999995}]},{"osd":1,"last update":"Thu Oct 26 01:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.875,"15min":0.93700000000000006},"min":{"1min":0.40699999999999997,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.3560000000000001,"5min":1.444,"15min":4.4340000000000002},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.97599999999999998,"15min":0.997},"min":{"1min":0.51500000000000001,"5min":0.46700000000000003,"15min":0.374},"max":{"1min":1.4390000000000001,"5min":1.538,"15min":2.3330000000000002},"last":0.755}]},{"osd":2,"last update":"Thu Oct 26 01:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":1.032,"15min":1.075},"min":{"1min":0.53800000000000003,"5min":0.40999999999999998,"15min":0.39900000000000002},"max":{"1min":1.643,"5min":1.927,"15min":4.2800000000000002},"last":1.2689999999999999},{"interface":"front","average":{"1min":1.032,"5min":1.1279999999999999,"15min":1.091},"min":{"1min":0.50900000000000001,"5min":0.495,"15min":0.49399999999999999},"max":{"1min":1.607,"5min":1.8680000000000001,"15min":2.302},"last":1.3320000000000001}]},{"osd":4,"last update":"Thu Oct 26 01:37:49 2023","interfaces":[{"interface":"back","average":{"1min":1.175,"5min":1.2070000000000001,"15min":1.167},"min":{"1min":0.755,"5min":0.755,"15min":0.66400000000000003},"max":{"1min":1.536,"5min":1.7150000000000001,"15min":2.1070000000000002},"last":1.008},{"interface":"front","average":{"1min":1.252,"5min":1.3400000000000001,"15min":1.389},"min":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.79900000000000004},"max":{"1min":1.5209999999999999,"5min":2.4239999999999999,"15min":4.8250000000000002},"last":1.5740000000000001}]},{"osd":5,"last update":"Thu Oct 26 01:37:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.282,"15min":1.333},"min":{"1min":0.79300000000000004,"5min":0.76400000000000001,"15min":0.72499999999999998},"max":{"1min":1.639,"5min":2.3380000000000001,"15min":4.9080000000000004},"last":1.448},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.28,"15min":1.276},"min":{"1min":0.90300000000000002,"5min":0.79900000000000004,"15min":0.79600000000000004},"max":{"1min":1.722,"5min":2.0310000000000001,"15min":2.8980000000000001},"last":1.1120000000000001}]},{"osd":6,"last update":"Thu Oct 26 01:37:49 2023","interfaces":[{"interface":"back","average":{"1min":1.278,"5min":1.355,"15min":1.331},"min":{"1min":0.74199999999999999,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.5580000000000001,"5min":2.1499999999999999,"15min":2.8530000000000002},"last":1.2270000000000001},{"interface":"front","average":{"1min":1.206,"5min":1.268,"15min":1.268},"min":{"1min":0.81699999999999995,"5min":0.81499999999999995,"15min":0.73099999999999998},"max":{"1min":1.7410000000000001,"5min":2.048,"15min":2.181},"last":1.0820000000000001}]},{"osd":7,"last update":"Thu Oct 26 01:37:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.3300000000000001,"15min":1.296},"min":{"1min":0.85299999999999998,"5min":0.84599999999999997,"15min":0.71999999999999997},"max":{"1min":1.8129999999999999,"5min":1.958,"15min":2.0680000000000001},"last":1.046},{"interface":"front","average":{"1min":1.2889999999999999,"5min":1.3919999999999999,"15min":1.415},"min":{"1min":0.84199999999999997,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.647,"5min":2.2370000000000001,"15min":4.8730000000000002},"last":1.6399999999999999}]}]},{"osd":7,"up_from":13,"seq":55834574944,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1288,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194024,"statfs":{"total":200000000,"available":198681011,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:37:49 2023","interfaces":[{"interface":"back","average":{"1min":1.0920000000000001,"5min":1.139,"15min":1.093},"min":{"1min":0.81799999999999995,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.7,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":1.28},{"interface":"front","average":{"1min":0.99099999999999999,"5min":1.115,"15min":1.081},"min":{"1min":0.70599999999999996,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.46,"5min":2.6779999999999999,"15min":2.6779999999999999},"last":1.6970000000000001}]},{"osd":1,"last update":"Thu Oct 26 01:37:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.1870000000000001,"15min":1.0960000000000001},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65200000000000002},"max":{"1min":1.5549999999999999,"5min":2.8260000000000001,"15min":2.8260000000000001},"last":1.137},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.21,"15min":1.194},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.61199999999999999},"max":{"1min":1.7869999999999999,"5min":3.2730000000000001,"15min":3.2730000000000001},"last":1.76}]},{"osd":2,"last update":"Thu Oct 26 01:37:49 2023","interfaces":[{"interface":"back","average":{"1min":1.143,"5min":1.179,"15min":1.1180000000000001},"min":{"1min":0.79100000000000004,"5min":0.65700000000000003,"15min":0.58499999999999996},"max":{"1min":1.667,"5min":2.218,"15min":2.218},"last":1.3120000000000001},{"interface":"front","average":{"1min":1.2709999999999999,"5min":1.2989999999999999,"15min":1.24},"min":{"1min":0.89200000000000002,"5min":0.79400000000000004,"15min":0.73099999999999998},"max":{"1min":1.786,"5min":3.0510000000000002,"15min":3.0510000000000002},"last":1.347}]},{"osd":3,"last update":"Thu Oct 26 01:37:49 2023","interfaces":[{"interface":"back","average":{"1min":1.264,"5min":1.284,"15min":1.214},"min":{"1min":0.93500000000000005,"5min":0.79000000000000004,"15min":0.72099999999999997},"max":{"1min":1.5389999999999999,"5min":1.9830000000000001,"15min":1.9830000000000001},"last":1.5620000000000001},{"interface":"front","average":{"1min":1.296,"5min":1.407,"15min":1.373},"min":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.65700000000000003},"max":{"1min":1.8939999999999999,"5min":5.5039999999999996,"15min":5.5039999999999996},"last":1.383}]},{"osd":4,"last update":"Thu Oct 26 01:37:49 2023","interfaces":[{"interface":"back","average":{"1min":1.4350000000000001,"5min":1.373,"15min":1.3520000000000001},"min":{"1min":1.0589999999999999,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.7450000000000001,"5min":2.77,"15min":2.77},"last":1.0669999999999999},{"interface":"front","average":{"1min":1.456,"5min":1.429,"15min":1.399},"min":{"1min":0.92700000000000005,"5min":0.90400000000000003,"15min":0.496},"max":{"1min":1.95,"5min":3.1419999999999999,"15min":3.1419999999999999},"last":1.46}]},{"osd":5,"last update":"Thu Oct 26 01:37:49 2023","interfaces":[{"interface":"back","average":{"1min":1.498,"5min":1.488,"15min":1.4259999999999999},"min":{"1min":0.91700000000000004,"5min":0.65600000000000003,"15min":0.441},"max":{"1min":1.9890000000000001,"5min":5.734,"15min":5.734},"last":1.609},{"interface":"front","average":{"1min":1.4970000000000001,"5min":1.5269999999999999,"15min":1.478},"min":{"1min":1.1739999999999999,"5min":0.96099999999999997,"15min":0.51900000000000002},"max":{"1min":1.8799999999999999,"5min":2.8980000000000001,"15min":2.8980000000000001},"last":1.1870000000000001}]},{"osd":6,"last update":"Thu Oct 26 01:37:49 2023","interfaces":[{"interface":"back","average":{"1min":1.5580000000000001,"5min":1.6080000000000001,"15min":1.8460000000000001},"min":{"1min":1.0169999999999999,"5min":0.753,"15min":0.47399999999999998},"max":{"1min":2.0739999999999998,"5min":5.6340000000000003,"15min":11.577999999999999},"last":1.8049999999999999},{"interface":"front","average":{"1min":1.615,"5min":1.6779999999999999,"15min":1.8620000000000001},"min":{"1min":1.2889999999999999,"5min":0.91800000000000004,"15min":0.55100000000000005},"max":{"1min":1.986,"5min":5.0919999999999996,"15min":10.704000000000001},"last":1.6519999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574944,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":704,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194607,"statfs":{"total":200000000,"available":199278268,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:37:43 2023","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.113,"15min":1.1539999999999999},"min":{"1min":0.69299999999999995,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.6100000000000001,"5min":2.395,"15min":2.395},"last":1.1830000000000001},{"interface":"front","average":{"1min":1.139,"5min":1.173,"15min":1.1220000000000001},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.59199999999999997},"max":{"1min":2.081,"5min":2.081,"15min":2.081},"last":1.5349999999999999}]},{"osd":1,"last update":"Thu Oct 26 01:37:43 2023","interfaces":[{"interface":"back","average":{"1min":1.2749999999999999,"5min":1.2110000000000001,"15min":1.1779999999999999},"min":{"1min":0.82499999999999996,"5min":0.71599999999999997,"15min":0.68300000000000005},"max":{"1min":1.992,"5min":2.0529999999999999,"15min":2.0529999999999999},"last":0.83399999999999996},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.179,"15min":1.204},"min":{"1min":0.67900000000000005,"5min":0.67700000000000005,"15min":0.64000000000000001},"max":{"1min":1.968,"5min":1.968,"15min":2.085},"last":1.359}]},{"osd":2,"last update":"Thu Oct 26 01:37:43 2023","interfaces":[{"interface":"back","average":{"1min":1.077,"5min":1.1879999999999999,"15min":1.095},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.5649999999999999,"5min":3.7879999999999998,"15min":3.7879999999999998},"last":1.5629999999999999},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.3120000000000001,"15min":1.268},"min":{"1min":0.90600000000000003,"5min":0.75800000000000001,"15min":0.68899999999999995},"max":{"1min":2.2200000000000002,"5min":2.2890000000000001,"15min":2.2890000000000001},"last":1.3129999999999999}]},{"osd":3,"last update":"Thu Oct 26 01:37:43 2023","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.319,"15min":1.214},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.756,"5min":4.4560000000000004,"15min":4.4560000000000004},"last":1.4059999999999999},{"interface":"front","average":{"1min":1.288,"5min":1.337,"15min":1.325},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.69499999999999995},"max":{"1min":2.1520000000000001,"5min":2.5019999999999998,"15min":2.5019999999999998},"last":0.96299999999999997}]},{"osd":4,"last update":"Thu Oct 26 01:37:43 2023","interfaces":[{"interface":"back","average":{"1min":1.2709999999999999,"5min":1.272,"15min":1.224},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.48399999999999999},"max":{"1min":1.754,"5min":2.9489999999999998,"15min":2.9489999999999998},"last":1.042},{"interface":"front","average":{"1min":1.272,"5min":1.4299999999999999,"15min":1.3149999999999999},"min":{"1min":0.96799999999999997,"5min":0.81100000000000005,"15min":0.59299999999999997},"max":{"1min":1.7629999999999999,"5min":6.609,"15min":6.609},"last":1.2509999999999999}]},{"osd":5,"last update":"Thu Oct 26 01:37:43 2023","interfaces":[{"interface":"back","average":{"1min":1.4570000000000001,"5min":1.5469999999999999,"15min":1.4470000000000001},"min":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.67000000000000004},"max":{"1min":1.8380000000000001,"5min":5.4370000000000003,"15min":5.4370000000000003},"last":1.6950000000000001},{"interface":"front","average":{"1min":1.4370000000000001,"5min":1.4410000000000001,"15min":1.4119999999999999},"min":{"1min":0.626,"5min":0.61399999999999999,"15min":0.58199999999999996},"max":{"1min":2.3279999999999998,"5min":2.621,"15min":2.621},"last":1.5089999999999999}]},{"osd":7,"last update":"Thu Oct 26 01:37:43 2023","interfaces":[{"interface":"back","average":{"1min":1.3400000000000001,"5min":1.52,"15min":1.4099999999999999},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.54100000000000004},"max":{"1min":1.8740000000000001,"5min":11.804,"15min":11.804},"last":1.45},{"interface":"front","average":{"1min":1.5049999999999999,"5min":1.6559999999999999,"15min":1.51},"min":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.77300000000000002},"max":{"1min":2.0710000000000002,"5min":11.541,"15min":11.541},"last":1.6399999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574944,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":705,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194607,"statfs":{"total":200000000,"available":199277654,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:37:46 2023","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.1870000000000001,"15min":1.129},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.5660000000000001,"5min":1.7390000000000001,"15min":1.7390000000000001},"last":1.3700000000000001},{"interface":"front","average":{"1min":1.204,"5min":1.28,"15min":1.1819999999999999},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.613,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":1.198}]},{"osd":1,"last update":"Thu Oct 26 01:37:46 2023","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":1.1319999999999999,"15min":1.085},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.3640000000000001,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":0.97799999999999998},{"interface":"front","average":{"1min":1.069,"5min":1.137,"15min":1.095},"min":{"1min":0.629,"5min":0.629,"15min":0.60599999999999998},"max":{"1min":1.726,"5min":1.726,"15min":1.7669999999999999},"last":1.2330000000000001}]},{"osd":2,"last update":"Thu Oct 26 01:37:46 2023","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":1.1950000000000001,"15min":1.163},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.69299999999999995},"max":{"1min":1.681,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":1.3959999999999999},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.2290000000000001,"15min":1.181},"min":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.63900000000000001},"max":{"1min":1.4339999999999999,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":1.1639999999999999}]},{"osd":3,"last update":"Thu Oct 26 01:37:46 2023","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.2949999999999999,"15min":1.238},"min":{"1min":0.78900000000000003,"5min":0.72699999999999998,"15min":0.72299999999999998},"max":{"1min":1.6180000000000001,"5min":1.7649999999999999,"15min":1.835},"last":1.49},{"interface":"front","average":{"1min":1.2529999999999999,"5min":1.3220000000000001,"15min":1.2929999999999999},"min":{"1min":0.85199999999999998,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.7310000000000001,"5min":1.788,"15min":2.9249999999999998},"last":1.1439999999999999}]},{"osd":4,"last update":"Thu Oct 26 01:37:46 2023","interfaces":[{"interface":"back","average":{"1min":1.1519999999999999,"5min":1.2689999999999999,"15min":1.2050000000000001},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.49099999999999999},"max":{"1min":1.6719999999999999,"5min":1.845,"15min":1.845},"last":1.097},{"interface":"front","average":{"1min":1.3340000000000001,"5min":1.4359999999999999,"15min":1.355},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.56200000000000006},"max":{"1min":1.78,"5min":1.867,"15min":1.867},"last":1.0740000000000001}]},{"osd":6,"last update":"Thu Oct 26 01:37:46 2023","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.4319999999999999,"15min":1.323},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.51900000000000002},"max":{"1min":1.7490000000000001,"5min":2.8250000000000002,"15min":2.8250000000000002},"last":1.3440000000000001},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.423,"15min":1.327},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.61799999999999999},"max":{"1min":1.661,"5min":1.849,"15min":1.8560000000000001},"last":1.3160000000000001}]},{"osd":7,"last update":"Thu Oct 26 01:37:46 2023","interfaces":[{"interface":"back","average":{"1min":1.4219999999999999,"5min":1.536,"15min":1.4219999999999999},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.61699999999999999},"max":{"1min":1.8220000000000001,"5min":2.1040000000000001,"15min":2.1040000000000001},"last":1.2909999999999999},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.5609999999999999,"15min":1.431},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.59499999999999997},"max":{"1min":1.853,"5min":2.0960000000000001,"15min":2.0960000000000001},"last":1.431}]}]},{"osd":4,"up_from":13,"seq":55834574944,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":705,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194607,"statfs":{"total":200000000,"available":199277654,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:37:42 2023","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.056,"15min":0.98499999999999999},"min":{"1min":0.63800000000000001,"5min":0.58099999999999996,"15min":0.53100000000000003},"max":{"1min":1.5369999999999999,"5min":1.885,"15min":2.0289999999999999},"last":1.522},{"interface":"front","average":{"1min":1.0309999999999999,"5min":1.081,"15min":1.024},"min":{"1min":0.74099999999999999,"5min":0.66400000000000003,"15min":0.60599999999999998},"max":{"1min":1.5029999999999999,"5min":2.6829999999999998,"15min":2.6829999999999998},"last":1.4790000000000001}]},{"osd":1,"last update":"Thu Oct 26 01:37:42 2023","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.2,"15min":1.127},"min":{"1min":0.66700000000000004,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.591,"5min":4.0519999999999996,"15min":4.0519999999999996},"last":1.375},{"interface":"front","average":{"1min":1.1599999999999999,"5min":1.1559999999999999,"15min":1.1100000000000001},"min":{"1min":0.70399999999999996,"5min":0.627,"15min":0.60499999999999998},"max":{"1min":1.897,"5min":1.897,"15min":2.0760000000000001},"last":1.3100000000000001}]},{"osd":2,"last update":"Thu Oct 26 01:37:42 2023","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.3480000000000001,"15min":1.23},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.64300000000000002},"max":{"1min":1.845,"5min":5.6749999999999998,"15min":5.6749999999999998},"last":1.6140000000000001},{"interface":"front","average":{"1min":1.3,"5min":1.218,"15min":1.1359999999999999},"min":{"1min":0.73899999999999999,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":3.673,"5min":3.673,"15min":3.673},"last":1.3400000000000001}]},{"osd":3,"last update":"Thu Oct 26 01:37:42 2023","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.204,"15min":1.1870000000000001},"min":{"1min":0.91500000000000004,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.7669999999999999,"5min":1.847,"15min":1.891},"last":1.425},{"interface":"front","average":{"1min":1.194,"5min":1.21,"15min":1.159},"min":{"1min":0.85399999999999998,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.6419999999999999,"5min":1.7210000000000001,"15min":1.7210000000000001},"last":0.89800000000000002}]},{"osd":5,"last update":"Thu Oct 26 01:37:42 2023","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":1.2549999999999999,"15min":1.286},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.46300000000000002},"max":{"1min":1.8520000000000001,"5min":2.1429999999999998,"15min":2.1429999999999998},"last":1.087},{"interface":"front","average":{"1min":1.3460000000000001,"5min":1.3500000000000001,"15min":1.3280000000000001},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.7649999999999999,"5min":5.8579999999999997,"15min":5.8579999999999997},"last":1.208}]},{"osd":6,"last update":"Thu Oct 26 01:37:42 2023","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":1.427,"15min":1.3380000000000001},"min":{"1min":0.69199999999999995,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":2.2149999999999999,"5min":12.343999999999999,"15min":12.343999999999999},"last":0.97299999999999998},{"interface":"front","average":{"1min":1.492,"5min":1.619,"15min":1.5149999999999999},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.55000000000000004},"max":{"1min":2.1000000000000001,"5min":12.605,"15min":12.605},"last":1.5760000000000001}]},{"osd":7,"last update":"Thu Oct 26 01:37:42 2023","interfaces":[{"interface":"back","average":{"1min":1.4710000000000001,"5min":1.52,"15min":1.49},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":2.0390000000000001,"5min":5.774,"15min":5.774},"last":1.706},{"interface":"front","average":{"1min":1.53,"5min":1.5309999999999999,"15min":1.46},"min":{"1min":0.79000000000000004,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":2.3450000000000002,"5min":4.6520000000000001,"15min":4.6520000000000001},"last":1.2769999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574944,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":711,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194600,"statfs":{"total":200000000,"available":199271379,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:37:46 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.84099999999999997,"15min":0.85499999999999998},"min":{"1min":0.41199999999999998,"5min":0.308,"15min":0.308},"max":{"1min":1.4850000000000001,"5min":1.5249999999999999,"15min":6.3200000000000003},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.997,"15min":0.93100000000000005},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.2569999999999999,"5min":2.1400000000000001,"15min":4.2619999999999996},"last":1.1930000000000001}]},{"osd":1,"last update":"Thu Oct 26 01:37:46 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.90200000000000002,"15min":0.89700000000000002},"min":{"1min":0.39300000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.244,"5min":1.629,"15min":5.1260000000000003},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.85399999999999998,"5min":1.018,"15min":1.0029999999999999},"min":{"1min":0.47699999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.351,"5min":1.873,"15min":6.1520000000000001},"last":1.339}]},{"osd":3,"last update":"Thu Oct 26 01:37:48 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":1.0189999999999999,"15min":0.93899999999999995},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.443},"max":{"1min":1.4390000000000001,"5min":2.0640000000000001,"15min":2.0640000000000001},"last":1.0609999999999999},{"interface":"front","average":{"1min":0.93600000000000005,"5min":1.111,"15min":1.018},"min":{"1min":0.51100000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.375,"5min":1.9279999999999999,"15min":8.9019999999999992},"last":0.98199999999999998}]},{"osd":4,"last update":"Thu Oct 26 01:37:46 2023","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.3420000000000001,"15min":1.2689999999999999},"min":{"1min":0.86499999999999999,"5min":0.76000000000000001,"15min":0.73899999999999999},"max":{"1min":1.704,"5min":2.6190000000000002,"15min":6.2670000000000003},"last":1.294},{"interface":"front","average":{"1min":1.1879999999999999,"5min":1.3129999999999999,"15min":1.262},"min":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.78700000000000003},"max":{"1min":1.5920000000000001,"5min":2.4359999999999999,"15min":7.343},"last":1.1200000000000001}]},{"osd":5,"last update":"Thu Oct 26 01:37:46 2023","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.3540000000000001,"15min":1.2869999999999999},"min":{"1min":0.873,"5min":0.81699999999999995,"15min":0.71899999999999997},"max":{"1min":1.5740000000000001,"5min":2.5790000000000002,"15min":3.7170000000000001},"last":1.2470000000000001},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.3089999999999999,"15min":1.2909999999999999},"min":{"1min":0.81899999999999995,"5min":0.70999999999999996,"15min":0.65300000000000002},"max":{"1min":1.621,"5min":2.298,"15min":6.3899999999999997},"last":1.5169999999999999}]},{"osd":6,"last update":"Thu Oct 26 01:37:46 2023","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":1.399,"15min":1.335},"min":{"1min":0.81200000000000006,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.9379999999999999,"5min":2.73,"15min":6.0490000000000004},"last":1.4790000000000001},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.4179999999999999,"15min":1.3720000000000001},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.76800000000000002},"max":{"1min":1.8799999999999999,"5min":2.4089999999999998,"15min":5.0650000000000004},"last":1.4399999999999999}]},{"osd":7,"last update":"Thu Oct 26 01:37:46 2023","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.403,"15min":1.395},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.69399999999999995},"max":{"1min":1.7809999999999999,"5min":2.3370000000000002,"15min":6.4480000000000004},"last":1.6140000000000001},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.52,"15min":1.4550000000000001},"min":{"1min":1.036,"5min":0.91800000000000004,"15min":0.91800000000000004},"max":{"1min":1.877,"5min":2.8290000000000002,"15min":2.8439999999999999},"last":1.5640000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574944,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":711,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194600,"statfs":{"total":200000000,"available":199271379,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:37:48 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.78300000000000003,"15min":0.72499999999999998},"min":{"1min":0.36099999999999999,"5min":0.34000000000000002,"15min":0.29499999999999998},"max":{"1min":1.4099999999999999,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.753},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.86799999999999999,"15min":0.79600000000000004},"min":{"1min":0.435,"5min":0.36299999999999999,"15min":0.36199999999999999},"max":{"1min":1.4019999999999999,"5min":1.6399999999999999,"15min":1.6399999999999999},"last":0.84699999999999998}]},{"osd":2,"last update":"Thu Oct 26 01:37:48 2023","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":0.97899999999999998,"15min":1.0009999999999999},"min":{"1min":0.52200000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.8200000000000001,"5min":1.8200000000000001,"15min":2.4830000000000001},"last":1.0489999999999999},{"interface":"front","average":{"1min":1.0569999999999999,"5min":0.93700000000000006,"15min":0.875},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.772,"5min":1.8080000000000001,"15min":1.8080000000000001},"last":1.256}]},{"osd":3,"last update":"Thu Oct 26 01:37:52 2023","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.97899999999999998,"15min":0.94899999999999995},"min":{"1min":0.52900000000000003,"5min":0.42499999999999999,"15min":0.39500000000000002},"max":{"1min":1.288,"5min":1.611,"15min":1.75},"last":1.1399999999999999},{"interface":"front","average":{"1min":0.97899999999999998,"5min":1.028,"15min":0.98999999999999999},"min":{"1min":0.55900000000000005,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.45,"5min":1.673,"15min":1.802},"last":0.94399999999999995}]},{"osd":4,"last update":"Thu Oct 26 01:37:48 2023","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":1.2270000000000001,"15min":1.1990000000000001},"min":{"1min":0.82899999999999996,"5min":0.69099999999999995,"15min":0.56200000000000006},"max":{"1min":1.756,"5min":1.756,"15min":2.0659999999999998},"last":1.5249999999999999},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.1930000000000001,"15min":1.1359999999999999},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.474,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":1.292}]},{"osd":5,"last update":"Thu Oct 26 01:37:48 2023","interfaces":[{"interface":"back","average":{"1min":1.1459999999999999,"5min":1.262,"15min":1.2270000000000001},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.573,"5min":1.8779999999999999,"15min":1.8779999999999999},"last":1.077},{"interface":"front","average":{"1min":1.3720000000000001,"5min":1.3720000000000001,"15min":1.337},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":2.04,"5min":2.04,"15min":2.2200000000000002},"last":1.3220000000000001}]},{"osd":6,"last update":"Thu Oct 26 01:37:48 2023","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":1.3320000000000001,"15min":1.28},"min":{"1min":0.75,"5min":0.70199999999999996,"15min":0.68899999999999995},"max":{"1min":1.7330000000000001,"5min":2.0910000000000002,"15min":2.3780000000000001},"last":1.2370000000000001},{"interface":"front","average":{"1min":1.26,"5min":1.2729999999999999,"15min":1.244},"min":{"1min":0.80700000000000005,"5min":0.71299999999999997,"15min":0.70799999999999996},"max":{"1min":1.669,"5min":1.956,"15min":1.984},"last":1.3859999999999999}]},{"osd":7,"last update":"Thu Oct 26 01:37:48 2023","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.2629999999999999,"15min":1.262},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.69699999999999995},"max":{"1min":1.7230000000000001,"5min":1.8380000000000001,"15min":2.6930000000000001},"last":1.3560000000000001},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.4019999999999999,"15min":1.337},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.9370000000000001,"5min":2.0190000000000001,"15min":2.0190000000000001},"last":1.5720000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574944,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1288,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194024,"statfs":{"total":200000000,"available":198681011,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Oct 26 01:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":1.0029999999999999,"15min":0.91600000000000004},"min":{"1min":0.41699999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.2170000000000001,"5min":19.425000000000001,"15min":19.425000000000001},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.84199999999999997,"5min":1.0720000000000001,"15min":1.042},"min":{"1min":0.56100000000000005,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.1839999999999999,"5min":15.162000000000001,"15min":15.162000000000001},"last":1.1499999999999999}]},{"osd":2,"last update":"Thu Oct 26 01:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":1.3080000000000001,"15min":1.1819999999999999},"min":{"1min":0.57699999999999996,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.4930000000000001,"5min":28.439,"15min":28.439},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.91700000000000004,"15min":0.91200000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.4810000000000001,"5min":1.502,"15min":4.9459999999999997},"last":0.64300000000000002}]},{"osd":3,"last update":"Thu Oct 26 01:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":1.0549999999999999,"15min":1.085},"min":{"1min":0.48899999999999999,"5min":0.46000000000000002,"15min":0.44600000000000001},"max":{"1min":1.4330000000000001,"5min":1.6479999999999999,"15min":4.9800000000000004},"last":1.111},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.1459999999999999,"15min":1.155},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.4770000000000001,"5min":1.659,"15min":5.016},"last":1.2330000000000001}]},{"osd":4,"last update":"Thu Oct 26 01:37:45 2023","interfaces":[{"interface":"back","average":{"1min":1.091,"5min":1.379,"15min":1.29},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.49,"5min":19.329000000000001,"15min":19.329000000000001},"last":0.97799999999999998},{"interface":"front","average":{"1min":1.22,"5min":1.268,"15min":1.2989999999999999},"min":{"1min":0.88,"5min":0.82099999999999995,"15min":0.67600000000000005},"max":{"1min":1.5449999999999999,"5min":1.718,"15min":5.5229999999999997},"last":1.3460000000000001}]},{"osd":5,"last update":"Thu Oct 26 01:37:45 2023","interfaces":[{"interface":"back","average":{"1min":1.373,"5min":1.4039999999999999,"15min":1.4299999999999999},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.661,"5min":1.915,"15min":5.6360000000000001},"last":1.395},{"interface":"front","average":{"1min":1.202,"5min":1.4790000000000001,"15min":1.3999999999999999},"min":{"1min":0.73599999999999999,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.71,"5min":19.384,"15min":19.384},"last":1.1839999999999999}]},{"osd":6,"last update":"Thu Oct 26 01:37:45 2023","interfaces":[{"interface":"back","average":{"1min":1.2609999999999999,"5min":1.3799999999999999,"15min":1.365},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.73799999999999999},"max":{"1min":1.615,"5min":8.2609999999999992,"15min":8.2609999999999992},"last":1.4650000000000001},{"interface":"front","average":{"1min":1.3440000000000001,"5min":1.4590000000000001,"15min":1.407},"min":{"1min":0.83099999999999996,"5min":0.77900000000000003,"15min":0.75900000000000001},"max":{"1min":1.804,"5min":11.849,"15min":11.849},"last":1.429}]},{"osd":7,"last update":"Thu Oct 26 01:37:45 2023","interfaces":[{"interface":"back","average":{"1min":1.3979999999999999,"5min":1.4450000000000001,"15min":1.4810000000000001},"min":{"1min":0.871,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.8380000000000001,"5min":2.4060000000000001,"15min":5.6689999999999996},"last":1.3149999999999999},{"interface":"front","average":{"1min":1.365,"5min":1.6120000000000001,"15min":1.5029999999999999},"min":{"1min":0.89600000000000002,"5min":0.879,"15min":0.69399999999999995},"max":{"1min":1.841,"5min":19.245999999999999,"15min":19.245999999999999},"last":1.2809999999999999}]}]}],"pool_statfs":[]}} 2023-10-26T01:38:25.802 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-26T01:38:26.062 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:26.060+0000 7f249e67c640 1 -- 172.21.15.169:0/1108940191 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24840047c0 msgr2=0x7f2484004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:26.063 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:26.060+0000 7f249e67c640 1 --2- 172.21.15.169:0/1108940191 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24840047c0 0x7f2484004c40 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f2480009b50 tx=0x7f24840057f0 comp rx=0 tx=0).stop 2023-10-26T01:38:26.063 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:26.060+0000 7f249cc19640 1 -- 172.21.15.169:0/1108940191 reap_dead start 2023-10-26T01:38:26.063 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:26.061+0000 7f249e67c640 1 -- 172.21.15.169:0/1108940191 shutdown_connections 2023-10-26T01:38:26.063 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:26.061+0000 7f249e67c640 1 -- 172.21.15.169:0/1108940191 >> 172.21.15.169:0/1108940191 conn(0x7f2498069930 msgr2=0x7f2498069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:26.064 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:26.061+0000 7f249e67c640 1 -- 172.21.15.169:0/1108940191 shutdown_connections 2023-10-26T01:38:26.064 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:26.061+0000 7f249e67c640 1 -- 172.21.15.169:0/1108940191 wait complete. 2023-10-26T01:38:26.064 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:26.062+0000 7f249e67c640 1 Processor -- start 2023-10-26T01:38:26.065 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:26.063+0000 7f249e67c640 1 -- start start 2023-10-26T01:38:26.066 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:26.063+0000 7f249e67c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f24840047c0 0x7f24980997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:26.066 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:26.064+0000 7f249e67c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f249806fc00 0x7f24980aeec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:26.066 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:26.064+0000 7f249e67c640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2498070080 0x7f24980af400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:26.067 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:26.064+0000 7f249e67c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24981f61f0 con 0x7f249806fc00 2023-10-26T01:38:26.067 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:26.064+0000 7f249e67c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24981f5320 con 0x7f24840047c0 2023-10-26T01:38:26.067 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:26.064+0000 7f249e67c640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f249814ce60 con 0x7f2498070080 2023-10-26T01:38:26.067 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:26.064+0000 7f248ffff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f249806fc00 0x7f24980aeec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:26.067 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:26.064+0000 7f248ffff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f249806fc00 0x7f24980aeec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:34298/0 (socket says 172.21.15.169:34298) 2023-10-26T01:38:26.068 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:26.064+0000 7f248ffff640 1 -- 172.21.15.169:0/1631064229 learned_addr learned my addr 172.21.15.169:0/1631064229 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:38:26.068 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:26.064+0000 7f249cc19640 1 --2- 172.21.15.169:0/1631064229 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2498070080 0x7f24980af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:26.068 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:26.064+0000 7f2497fff640 1 --2- 172.21.15.169:0/1631064229 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f24840047c0 0x7f24980997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:26.068 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:26.064+0000 7f248ffff640 1 -- 172.21.15.169:0/1631064229 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2498070080 msgr2=0x7f24980af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:26.069 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:26.064+0000 7f248ffff640 1 --2- 172.21.15.169:0/1631064229 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2498070080 0x7f24980af400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:26.069 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:26.064+0000 7f248ffff640 1 -- 172.21.15.169:0/1631064229 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f24840047c0 msgr2=0x7f24980997f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:26.069 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:26.064+0000 7f248ffff640 1 --2- 172.21.15.169:0/1631064229 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f24840047c0 0x7f24980997f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:26.069 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:26.064+0000 7f248ffff640 1 -- 172.21.15.169:0/1631064229 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24981f5570 con 0x7f249806fc00 2023-10-26T01:38:26.070 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:26.065+0000 7f248ffff640 1 --2- 172.21.15.169:0/1631064229 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f249806fc00 0x7f24980aeec0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f2480009a70 tx=0x7f2480033c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:26.071 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:26.065+0000 7f24957fa640 1 -- 172.21.15.169:0/1631064229 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2480004220 con 0x7f249806fc00 2023-10-26T01:38:26.071 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:26.065+0000 7f24957fa640 1 -- 172.21.15.169:0/1631064229 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2480004380 con 0x7f249806fc00 2023-10-26T01:38:26.071 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:26.065+0000 7f24957fa640 1 -- 172.21.15.169:0/1631064229 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f248003d940 con 0x7f249806fc00 2023-10-26T01:38:26.072 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:26.065+0000 7f249e67c640 1 -- 172.21.15.169:0/1631064229 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24981f5850 con 0x7f249806fc00 2023-10-26T01:38:26.072 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:26.065+0000 7f249e67c640 1 -- 172.21.15.169:0/1631064229 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24981f5e10 con 0x7f249806fc00 2023-10-26T01:38:26.072 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:26.066+0000 7f249e67c640 1 -- 172.21.15.169:0/1631064229 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24981f61f0 con 0x7f249806fc00 2023-10-26T01:38:26.072 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:26.067+0000 7f24957fa640 1 -- 172.21.15.169:0/1631064229 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f248003daa0 con 0x7f249806fc00 2023-10-26T01:38:26.072 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:26.069+0000 7f24957fa640 1 --2- 172.21.15.169:0/1631064229 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f247806ae40 0x7f247806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:26.072 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:26.069+0000 7f24957fa640 1 -- 172.21.15.169:0/1631064229 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2480033e40 con 0x7f249806fc00 2023-10-26T01:38:26.073 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:26.069+0000 7f2497fff640 1 --2- 172.21.15.169:0/1631064229 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f247806ae40 0x7f247806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:26.073 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:26.070+0000 7f2497fff640 1 --2- 172.21.15.169:0/1631064229 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f247806ae40 0x7f247806d300 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f2488005d00 tx=0x7f2488005c50 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:26.074 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:26.073+0000 7f24957fa640 1 -- 172.21.15.169:0/1631064229 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f24800773e0 con 0x7f249806fc00 2023-10-26T01:38:26.262 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:26.260+0000 7f249e67c640 1 -- 172.21.15.169:0/1631064229 --> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f249804db10 con 0x7f247806ae40 2023-10-26T01:38:26.284 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:26.282+0000 7f24957fa640 1 -- 172.21.15.169:0/1631064229 <== mgr.4112 v2:172.21.15.170:6832/1187156918 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+278465 (secure 0 0 0) 0x7f249804db10 con 0x7f247806ae40 2023-10-26T01:38:26.288 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:38:26.289 INFO:teuthology.orchestra.run.smithi169.stderr:dumped all 2023-10-26T01:38:26.291 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:26.288+0000 7f249e67c640 1 -- 172.21.15.169:0/1631064229 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f247806ae40 msgr2=0x7f247806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:26.291 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:26.288+0000 7f249e67c640 1 --2- 172.21.15.169:0/1631064229 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f247806ae40 0x7f247806d300 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f2488005d00 tx=0x7f2488005c50 comp rx=0 tx=0).stop 2023-10-26T01:38:26.291 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:26.288+0000 7f249e67c640 1 -- 172.21.15.169:0/1631064229 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f249806fc00 msgr2=0x7f24980aeec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:26.291 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:26.288+0000 7f249e67c640 1 --2- 172.21.15.169:0/1631064229 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f249806fc00 0x7f24980aeec0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f2480009a70 tx=0x7f2480033c50 comp rx=0 tx=0).stop 2023-10-26T01:38:26.291 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:26.289+0000 7f249e67c640 1 -- 172.21.15.169:0/1631064229 shutdown_connections 2023-10-26T01:38:26.292 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:26.289+0000 7f249e67c640 1 --2- 172.21.15.169:0/1631064229 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f247806ae40 0x7f247806d300 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:26.292 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:26.289+0000 7f249e67c640 1 --2- 172.21.15.169:0/1631064229 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f249806fc00 0x7f24980aeec0 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:26.292 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:26.289+0000 7f249e67c640 1 --2- 172.21.15.169:0/1631064229 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2498070080 0x7f24980af400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:26.292 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:26.289+0000 7f249e67c640 1 --2- 172.21.15.169:0/1631064229 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f24840047c0 0x7f24980997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:26.292 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:26.289+0000 7f249e67c640 1 -- 172.21.15.169:0/1631064229 >> 172.21.15.169:0/1631064229 conn(0x7f2498069930 msgr2=0x7f249815a950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:26.293 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:26.289+0000 7f249e67c640 1 -- 172.21.15.169:0/1631064229 shutdown_connections 2023-10-26T01:38:26.293 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:26.289+0000 7f249e67c640 1 -- 172.21.15.169:0/1631064229 wait complete. 2023-10-26T01:38:26.310 INFO:teuthology.orchestra.run.smithi169.stdout:{"pg_ready":true,"pg_map":{"version":364,"stamp":"2023-10-26T01:38:24.294725+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":162,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":31,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":6834,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1555665,"statfs":{"total":1600000000,"available":1593001278,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.013822"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.312993+0000","last_change":"2023-10-26T01:38:20.312993+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.312993+0000","last_undegraded":"2023-10-26T01:38:20.312993+0000","last_fullsized":"2023-10-26T01:38:20.312993+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.309555+0000","last_change":"2023-10-26T01:38:20.309555+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.309555+0000","last_undegraded":"2023-10-26T01:38:20.309555+0000","last_fullsized":"2023-10-26T01:38:20.309555+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.313087+0000","last_change":"2023-10-26T01:38:20.313087+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.313087+0000","last_undegraded":"2023-10-26T01:38:20.313087+0000","last_fullsized":"2023-10-26T01:38:20.313087+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.310689+0000","last_change":"2023-10-26T01:38:20.310689+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.310689+0000","last_undegraded":"2023-10-26T01:38:20.310689+0000","last_fullsized":"2023-10-26T01:38:20.310689+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.304432+0000","last_change":"2023-10-26T01:38:20.304432+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.304432+0000","last_undegraded":"2023-10-26T01:38:20.304432+0000","last_fullsized":"2023-10-26T01:38:20.304432+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.315050+0000","last_change":"2023-10-26T01:38:20.315050+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.315050+0000","last_undegraded":"2023-10-26T01:38:20.315050+0000","last_fullsized":"2023-10-26T01:38:20.315050+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.322890+0000","last_change":"2023-10-26T01:38:20.322890+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.322890+0000","last_undegraded":"2023-10-26T01:38:20.322890+0000","last_fullsized":"2023-10-26T01:38:20.322890+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.311227+0000","last_change":"2023-10-26T01:38:20.311227+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.311227+0000","last_undegraded":"2023-10-26T01:38:20.311227+0000","last_fullsized":"2023-10-26T01:38:20.311227+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.310300+0000","last_change":"2023-10-26T01:38:20.310300+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.310300+0000","last_undegraded":"2023-10-26T01:38:20.310300+0000","last_fullsized":"2023-10-26T01:38:20.310300+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.309940+0000","last_change":"2023-10-26T01:38:20.309940+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.309940+0000","last_undegraded":"2023-10-26T01:38:20.309940+0000","last_fullsized":"2023-10-26T01:38:20.309940+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.309024+0000","last_change":"2023-10-26T01:38:20.309024+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.309024+0000","last_undegraded":"2023-10-26T01:38:20.309024+0000","last_fullsized":"2023-10-26T01:38:20.309024+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.309147+0000","last_change":"2023-10-26T01:38:20.309147+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.309147+0000","last_undegraded":"2023-10-26T01:38:20.309147+0000","last_fullsized":"2023-10-26T01:38:20.309147+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.312278+0000","last_change":"2023-10-26T01:38:20.312278+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.312278+0000","last_undegraded":"2023-10-26T01:38:20.312278+0000","last_fullsized":"2023-10-26T01:38:20.312278+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.314720+0000","last_change":"2023-10-26T01:38:20.314720+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.314720+0000","last_undegraded":"2023-10-26T01:38:20.314720+0000","last_fullsized":"2023-10-26T01:38:20.314720+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.315698+0000","last_change":"2023-10-26T01:38:20.315698+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.315698+0000","last_undegraded":"2023-10-26T01:38:20.315698+0000","last_fullsized":"2023-10-26T01:38:20.315698+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.312579+0000","last_change":"2023-10-26T01:38:20.312579+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.312579+0000","last_undegraded":"2023-10-26T01:38:20.312579+0000","last_fullsized":"2023-10-26T01:38:20.312579+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.308707+0000","last_change":"2023-10-26T01:38:20.308707+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.308707+0000","last_undegraded":"2023-10-26T01:38:20.308707+0000","last_fullsized":"2023-10-26T01:38:20.308707+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.315277+0000","last_change":"2023-10-26T01:38:20.315277+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.315277+0000","last_undegraded":"2023-10-26T01:38:20.315277+0000","last_fullsized":"2023-10-26T01:38:20.315277+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.312233+0000","last_change":"2023-10-26T01:38:20.312233+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.312233+0000","last_undegraded":"2023-10-26T01:38:20.312233+0000","last_fullsized":"2023-10-26T01:38:20.312233+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.310177+0000","last_change":"2023-10-26T01:38:20.310177+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.310177+0000","last_undegraded":"2023-10-26T01:38:20.310177+0000","last_fullsized":"2023-10-26T01:38:20.310177+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.308112+0000","last_change":"2023-10-26T01:38:20.308112+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.308112+0000","last_undegraded":"2023-10-26T01:38:20.308112+0000","last_fullsized":"2023-10-26T01:38:20.308112+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.317702+0000","last_change":"2023-10-26T01:38:21.314965+0000","last_active":"2023-10-26T01:38:21.317702+0000","last_peered":"2023-10-26T01:38:21.317702+0000","last_clean":"2023-10-26T01:38:21.317702+0000","last_became_active":"2023-10-26T01:38:21.314673+0000","last_became_peered":"2023-10-26T01:38:21.314673+0000","last_unstale":"2023-10-26T01:38:21.317702+0000","last_undegraded":"2023-10-26T01:38:21.317702+0000","last_fullsized":"2023-10-26T01:38:21.317702+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:33:14.986087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.304560+0000","last_change":"2023-10-26T01:38:20.304560+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.304560+0000","last_undegraded":"2023-10-26T01:38:20.304560+0000","last_fullsized":"2023-10-26T01:38:20.304560+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.320186+0000","last_change":"2023-10-26T01:38:21.320068+0000","last_active":"2023-10-26T01:38:21.320186+0000","last_peered":"2023-10-26T01:38:21.320186+0000","last_clean":"2023-10-26T01:38:21.320186+0000","last_became_active":"2023-10-26T01:38:21.319773+0000","last_became_peered":"2023-10-26T01:38:21.319773+0000","last_unstale":"2023-10-26T01:38:21.320186+0000","last_undegraded":"2023-10-26T01:38:21.320186+0000","last_fullsized":"2023-10-26T01:38:21.320186+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:03:39.495499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.306576+0000","last_change":"2023-10-26T01:38:20.306576+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.306576+0000","last_undegraded":"2023-10-26T01:38:20.306576+0000","last_fullsized":"2023-10-26T01:38:20.306576+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.312230+0000","last_change":"2023-10-26T01:38:20.312230+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.312230+0000","last_undegraded":"2023-10-26T01:38:20.312230+0000","last_fullsized":"2023-10-26T01:38:20.312230+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.316627+0000","last_change":"2023-10-26T01:38:20.316627+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.316627+0000","last_undegraded":"2023-10-26T01:38:20.316627+0000","last_fullsized":"2023-10-26T01:38:20.316627+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.320158+0000","last_change":"2023-10-26T01:38:21.319408+0000","last_active":"2023-10-26T01:38:21.320158+0000","last_peered":"2023-10-26T01:38:21.320158+0000","last_clean":"2023-10-26T01:38:21.320158+0000","last_became_active":"2023-10-26T01:38:21.318421+0000","last_became_peered":"2023-10-26T01:38:21.318421+0000","last_unstale":"2023-10-26T01:38:21.320158+0000","last_undegraded":"2023-10-26T01:38:21.320158+0000","last_fullsized":"2023-10-26T01:38:21.320158+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:23:42.697097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.306705+0000","last_change":"2023-10-26T01:38:20.306705+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.306705+0000","last_undegraded":"2023-10-26T01:38:20.306705+0000","last_fullsized":"2023-10-26T01:38:20.306705+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.316779+0000","last_change":"2023-10-26T01:38:20.316779+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.316779+0000","last_undegraded":"2023-10-26T01:38:20.316779+0000","last_fullsized":"2023-10-26T01:38:20.316779+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.311858+0000","last_change":"2023-10-26T01:38:20.311858+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.311858+0000","last_undegraded":"2023-10-26T01:38:20.311858+0000","last_fullsized":"2023-10-26T01:38:20.311858+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.316322+0000","last_change":"2023-10-26T01:38:20.316322+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.316322+0000","last_undegraded":"2023-10-26T01:38:20.316322+0000","last_fullsized":"2023-10-26T01:38:20.316322+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.316115+0000","last_change":"2023-10-26T01:38:20.316115+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.316115+0000","last_undegraded":"2023-10-26T01:38:20.316115+0000","last_fullsized":"2023-10-26T01:38:20.316115+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-26T01:38:20.322096+0000","last_change":"2023-10-26T01:38:20.322005+0000","last_active":"2023-10-26T01:38:20.322096+0000","last_peered":"2023-10-26T01:38:20.322096+0000","last_clean":"2023-10-26T01:38:20.322096+0000","last_became_active":"2023-10-26T01:38:20.320582+0000","last_became_peered":"2023-10-26T01:38:20.320582+0000","last_unstale":"2023-10-26T01:38:20.322096+0000","last_undegraded":"2023-10-26T01:38:20.322096+0000","last_fullsized":"2023-10-26T01:38:20.322096+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:19:17.861486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.314532+0000","last_change":"2023-10-26T01:38:20.314532+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.314532+0000","last_undegraded":"2023-10-26T01:38:20.314532+0000","last_fullsized":"2023-10-26T01:38:20.314532+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-26T01:38:20.326259+0000","last_change":"2023-10-26T01:38:20.326148+0000","last_active":"2023-10-26T01:38:20.326259+0000","last_peered":"2023-10-26T01:38:20.326259+0000","last_clean":"2023-10-26T01:38:20.326259+0000","last_became_active":"2023-10-26T01:38:20.325883+0000","last_became_peered":"2023-10-26T01:38:20.325883+0000","last_unstale":"2023-10-26T01:38:20.326259+0000","last_undegraded":"2023-10-26T01:38:20.326259+0000","last_fullsized":"2023-10-26T01:38:20.326259+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:55:08.910181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.314689+0000","last_change":"2023-10-26T01:38:20.314689+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.314689+0000","last_undegraded":"2023-10-26T01:38:20.314689+0000","last_fullsized":"2023-10-26T01:38:20.314689+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-26T01:38:20.324190+0000","last_change":"2023-10-26T01:38:20.321817+0000","last_active":"2023-10-26T01:38:20.324190+0000","last_peered":"2023-10-26T01:38:20.324190+0000","last_clean":"2023-10-26T01:38:20.324190+0000","last_became_active":"2023-10-26T01:38:20.321520+0000","last_became_peered":"2023-10-26T01:38:20.321520+0000","last_unstale":"2023-10-26T01:38:20.324190+0000","last_undegraded":"2023-10-26T01:38:20.324190+0000","last_fullsized":"2023-10-26T01:38:20.324190+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:32:00.607826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.316223+0000","last_change":"2023-10-26T01:38:20.316223+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.316223+0000","last_undegraded":"2023-10-26T01:38:20.316223+0000","last_fullsized":"2023-10-26T01:38:20.316223+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-26T01:38:20.325212+0000","last_change":"2023-10-26T01:38:20.325154+0000","last_active":"2023-10-26T01:38:20.325212+0000","last_peered":"2023-10-26T01:38:20.325212+0000","last_clean":"2023-10-26T01:38:20.325212+0000","last_became_active":"2023-10-26T01:38:20.324072+0000","last_became_peered":"2023-10-26T01:38:20.324072+0000","last_unstale":"2023-10-26T01:38:20.325212+0000","last_undegraded":"2023-10-26T01:38:20.325212+0000","last_fullsized":"2023-10-26T01:38:20.325212+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:18:21.690998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.315707+0000","last_change":"2023-10-26T01:38:20.315707+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.315707+0000","last_undegraded":"2023-10-26T01:38:20.315707+0000","last_fullsized":"2023-10-26T01:38:20.315707+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-26T01:38:20.324077+0000","last_change":"2023-10-26T01:38:20.323989+0000","last_active":"2023-10-26T01:38:20.324077+0000","last_peered":"2023-10-26T01:38:20.324077+0000","last_clean":"2023-10-26T01:38:20.324077+0000","last_became_active":"2023-10-26T01:38:20.323646+0000","last_became_peered":"2023-10-26T01:38:20.323646+0000","last_unstale":"2023-10-26T01:38:20.324077+0000","last_undegraded":"2023-10-26T01:38:20.324077+0000","last_fullsized":"2023-10-26T01:38:20.324077+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:48:31.373334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.305499+0000","last_change":"2023-10-26T01:38:20.305499+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.305499+0000","last_undegraded":"2023-10-26T01:38:20.305499+0000","last_fullsized":"2023-10-26T01:38:20.305499+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"133'1","reported_seq":12,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.335571+0000","last_change":"2023-10-26T01:38:20.320626+0000","last_active":"2023-10-26T01:38:21.335571+0000","last_peered":"2023-10-26T01:38:21.335571+0000","last_clean":"2023-10-26T01:38:21.335571+0000","last_became_active":"2023-10-26T01:38:20.320400+0000","last_became_peered":"2023-10-26T01:38:20.320400+0000","last_unstale":"2023-10-26T01:38:21.335571+0000","last_undegraded":"2023-10-26T01:38:21.335571+0000","last_fullsized":"2023-10-26T01:38:21.335571+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:31:15.331694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.309593+0000","last_change":"2023-10-26T01:38:20.309593+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.309593+0000","last_undegraded":"2023-10-26T01:38:20.309593+0000","last_fullsized":"2023-10-26T01:38:20.309593+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-26T01:38:20.318013+0000","last_change":"2023-10-26T01:38:20.317888+0000","last_active":"2023-10-26T01:38:20.318013+0000","last_peered":"2023-10-26T01:38:20.318013+0000","last_clean":"2023-10-26T01:38:20.318013+0000","last_became_active":"2023-10-26T01:38:20.317130+0000","last_became_peered":"2023-10-26T01:38:20.317130+0000","last_unstale":"2023-10-26T01:38:20.318013+0000","last_undegraded":"2023-10-26T01:38:20.318013+0000","last_fullsized":"2023-10-26T01:38:20.318013+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:59:26.990814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.317096+0000","last_change":"2023-10-26T01:38:20.317096+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.317096+0000","last_undegraded":"2023-10-26T01:38:20.317096+0000","last_fullsized":"2023-10-26T01:38:20.317096+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-26T01:38:20.316112+0000","last_change":"2023-10-26T01:38:20.314210+0000","last_active":"2023-10-26T01:38:20.316112+0000","last_peered":"2023-10-26T01:38:20.316112+0000","last_clean":"2023-10-26T01:38:20.316112+0000","last_became_active":"2023-10-26T01:38:20.313976+0000","last_became_peered":"2023-10-26T01:38:20.313976+0000","last_unstale":"2023-10-26T01:38:20.316112+0000","last_undegraded":"2023-10-26T01:38:20.316112+0000","last_fullsized":"2023-10-26T01:38:20.316112+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:44:56.136940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.316088+0000","last_change":"2023-10-26T01:38:20.316088+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.316088+0000","last_undegraded":"2023-10-26T01:38:20.316088+0000","last_fullsized":"2023-10-26T01:38:20.316088+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-26T01:38:20.325081+0000","last_change":"2023-10-26T01:38:20.325020+0000","last_active":"2023-10-26T01:38:20.325081+0000","last_peered":"2023-10-26T01:38:20.325081+0000","last_clean":"2023-10-26T01:38:20.325081+0000","last_became_active":"2023-10-26T01:38:20.323713+0000","last_became_peered":"2023-10-26T01:38:20.323713+0000","last_unstale":"2023-10-26T01:38:20.325081+0000","last_undegraded":"2023-10-26T01:38:20.325081+0000","last_fullsized":"2023-10-26T01:38:20.325081+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:19:17.861486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.312710+0000","last_change":"2023-10-26T01:38:20.312710+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.312710+0000","last_undegraded":"2023-10-26T01:38:20.312710+0000","last_fullsized":"2023-10-26T01:38:20.312710+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.293962+0000","last_change":"2023-10-26T01:38:20.322090+0000","last_active":"2023-10-26T01:38:21.293962+0000","last_peered":"2023-10-26T01:38:21.293962+0000","last_clean":"2023-10-26T01:38:21.293962+0000","last_became_active":"2023-10-26T01:38:20.321451+0000","last_became_peered":"2023-10-26T01:38:20.321451+0000","last_unstale":"2023-10-26T01:38:21.293962+0000","last_undegraded":"2023-10-26T01:38:21.293962+0000","last_fullsized":"2023-10-26T01:38:21.293962+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:45:01.083199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.302056+0000","last_change":"2023-10-26T01:38:21.301523+0000","last_active":"2023-10-26T01:38:21.302056+0000","last_peered":"2023-10-26T01:38:21.302056+0000","last_clean":"2023-10-26T01:38:21.302056+0000","last_became_active":"2023-10-26T01:38:21.300901+0000","last_became_peered":"2023-10-26T01:38:21.300901+0000","last_unstale":"2023-10-26T01:38:21.302056+0000","last_undegraded":"2023-10-26T01:38:21.302056+0000","last_fullsized":"2023-10-26T01:38:21.302056+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:21:55.273677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-26T01:38:20.319415+0000","last_change":"2023-10-26T01:38:20.319263+0000","last_active":"2023-10-26T01:38:20.319415+0000","last_peered":"2023-10-26T01:38:20.319415+0000","last_clean":"2023-10-26T01:38:20.319415+0000","last_became_active":"2023-10-26T01:38:20.319022+0000","last_became_peered":"2023-10-26T01:38:20.319022+0000","last_unstale":"2023-10-26T01:38:20.319415+0000","last_undegraded":"2023-10-26T01:38:20.319415+0000","last_fullsized":"2023-10-26T01:38:20.319415+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:19:16.513935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.310881+0000","last_change":"2023-10-26T01:38:20.310881+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.310881+0000","last_undegraded":"2023-10-26T01:38:20.310881+0000","last_fullsized":"2023-10-26T01:38:20.310881+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-26T01:38:20.316131+0000","last_change":"2023-10-26T01:38:20.314532+0000","last_active":"2023-10-26T01:38:20.316131+0000","last_peered":"2023-10-26T01:38:20.316131+0000","last_clean":"2023-10-26T01:38:20.316131+0000","last_became_active":"2023-10-26T01:38:20.314073+0000","last_became_peered":"2023-10-26T01:38:20.314073+0000","last_unstale":"2023-10-26T01:38:20.316131+0000","last_undegraded":"2023-10-26T01:38:20.316131+0000","last_fullsized":"2023-10-26T01:38:20.316131+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:00:59.371192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.310604+0000","last_change":"2023-10-26T01:38:21.310272+0000","last_active":"2023-10-26T01:38:21.310604+0000","last_peered":"2023-10-26T01:38:21.310604+0000","last_clean":"2023-10-26T01:38:21.310604+0000","last_became_active":"2023-10-26T01:38:21.309297+0000","last_became_peered":"2023-10-26T01:38:21.309297+0000","last_unstale":"2023-10-26T01:38:21.310604+0000","last_undegraded":"2023-10-26T01:38:21.310604+0000","last_fullsized":"2023-10-26T01:38:21.310604+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:07:03.778722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-26T01:38:20.324089+0000","last_change":"2023-10-26T01:38:20.323897+0000","last_active":"2023-10-26T01:38:20.324089+0000","last_peered":"2023-10-26T01:38:20.324089+0000","last_clean":"2023-10-26T01:38:20.324089+0000","last_became_active":"2023-10-26T01:38:20.323527+0000","last_became_peered":"2023-10-26T01:38:20.323527+0000","last_unstale":"2023-10-26T01:38:20.324089+0000","last_undegraded":"2023-10-26T01:38:20.324089+0000","last_fullsized":"2023-10-26T01:38:20.324089+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:12:30.722215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.317765+0000","last_change":"2023-10-26T01:38:21.317158+0000","last_active":"2023-10-26T01:38:21.317765+0000","last_peered":"2023-10-26T01:38:21.317765+0000","last_clean":"2023-10-26T01:38:21.317765+0000","last_became_active":"2023-10-26T01:38:21.316782+0000","last_became_peered":"2023-10-26T01:38:21.316782+0000","last_unstale":"2023-10-26T01:38:21.317765+0000","last_undegraded":"2023-10-26T01:38:21.317765+0000","last_fullsized":"2023-10-26T01:38:21.317765+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:38:50.874604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-26T01:38:20.324414+0000","last_change":"2023-10-26T01:38:20.323991+0000","last_active":"2023-10-26T01:38:20.324414+0000","last_peered":"2023-10-26T01:38:20.324414+0000","last_clean":"2023-10-26T01:38:20.324414+0000","last_became_active":"2023-10-26T01:38:20.323242+0000","last_became_peered":"2023-10-26T01:38:20.323242+0000","last_unstale":"2023-10-26T01:38:20.324414+0000","last_undegraded":"2023-10-26T01:38:20.324414+0000","last_fullsized":"2023-10-26T01:38:20.324414+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:00:35.003415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.309119+0000","last_change":"2023-10-26T01:38:21.308916+0000","last_active":"2023-10-26T01:38:21.309119+0000","last_peered":"2023-10-26T01:38:21.309119+0000","last_clean":"2023-10-26T01:38:21.309119+0000","last_became_active":"2023-10-26T01:38:21.308364+0000","last_became_peered":"2023-10-26T01:38:21.308364+0000","last_unstale":"2023-10-26T01:38:21.309119+0000","last_undegraded":"2023-10-26T01:38:21.309119+0000","last_fullsized":"2023-10-26T01:38:21.309119+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:18:02.522210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-26T01:38:20.324208+0000","last_change":"2023-10-26T01:38:20.322830+0000","last_active":"2023-10-26T01:38:20.324208+0000","last_peered":"2023-10-26T01:38:20.324208+0000","last_clean":"2023-10-26T01:38:20.324208+0000","last_became_active":"2023-10-26T01:38:20.318642+0000","last_became_peered":"2023-10-26T01:38:20.318642+0000","last_unstale":"2023-10-26T01:38:20.324208+0000","last_undegraded":"2023-10-26T01:38:20.324208+0000","last_fullsized":"2023-10-26T01:38:20.324208+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:12:30.722215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.303953+0000","last_change":"2023-10-26T01:38:20.303953+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.303953+0000","last_undegraded":"2023-10-26T01:38:20.303953+0000","last_fullsized":"2023-10-26T01:38:20.303953+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-26T01:38:20.322630+0000","last_change":"2023-10-26T01:38:20.322530+0000","last_active":"2023-10-26T01:38:20.322630+0000","last_peered":"2023-10-26T01:38:20.322630+0000","last_clean":"2023-10-26T01:38:20.322630+0000","last_became_active":"2023-10-26T01:38:20.322312+0000","last_became_peered":"2023-10-26T01:38:20.322312+0000","last_unstale":"2023-10-26T01:38:20.322630+0000","last_undegraded":"2023-10-26T01:38:20.322630+0000","last_fullsized":"2023-10-26T01:38:20.322630+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:00:41.104678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.315008+0000","last_change":"2023-10-26T01:38:20.315008+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.315008+0000","last_undegraded":"2023-10-26T01:38:20.315008+0000","last_fullsized":"2023-10-26T01:38:20.315008+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-26T01:38:20.318810+0000","last_change":"2023-10-26T01:38:20.318723+0000","last_active":"2023-10-26T01:38:20.318810+0000","last_peered":"2023-10-26T01:38:20.318810+0000","last_clean":"2023-10-26T01:38:20.318810+0000","last_became_active":"2023-10-26T01:38:20.317546+0000","last_became_peered":"2023-10-26T01:38:20.317546+0000","last_unstale":"2023-10-26T01:38:20.318810+0000","last_undegraded":"2023-10-26T01:38:20.318810+0000","last_fullsized":"2023-10-26T01:38:20.318810+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:05:00.823929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.314011+0000","last_change":"2023-10-26T01:38:20.314011+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.314011+0000","last_undegraded":"2023-10-26T01:38:20.314011+0000","last_fullsized":"2023-10-26T01:38:20.314011+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-26T01:38:20.323891+0000","last_change":"2023-10-26T01:38:20.323687+0000","last_active":"2023-10-26T01:38:20.323891+0000","last_peered":"2023-10-26T01:38:20.323891+0000","last_clean":"2023-10-26T01:38:20.323891+0000","last_became_active":"2023-10-26T01:38:20.322524+0000","last_became_peered":"2023-10-26T01:38:20.322524+0000","last_unstale":"2023-10-26T01:38:20.323891+0000","last_undegraded":"2023-10-26T01:38:20.323891+0000","last_fullsized":"2023-10-26T01:38:20.323891+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:10:48.737505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.312099+0000","last_change":"2023-10-26T01:38:20.312099+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.312099+0000","last_undegraded":"2023-10-26T01:38:20.312099+0000","last_fullsized":"2023-10-26T01:38:20.312099+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-26T01:38:20.322573+0000","last_change":"2023-10-26T01:38:20.322266+0000","last_active":"2023-10-26T01:38:20.322573+0000","last_peered":"2023-10-26T01:38:20.322573+0000","last_clean":"2023-10-26T01:38:20.322573+0000","last_became_active":"2023-10-26T01:38:20.315231+0000","last_became_peered":"2023-10-26T01:38:20.315231+0000","last_unstale":"2023-10-26T01:38:20.322573+0000","last_undegraded":"2023-10-26T01:38:20.322573+0000","last_fullsized":"2023-10-26T01:38:20.322573+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:36:33.502627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.310535+0000","last_change":"2023-10-26T01:38:20.310535+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.310535+0000","last_undegraded":"2023-10-26T01:38:20.310535+0000","last_fullsized":"2023-10-26T01:38:20.310535+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-26T01:38:20.320042+0000","last_change":"2023-10-26T01:38:20.319975+0000","last_active":"2023-10-26T01:38:20.320042+0000","last_peered":"2023-10-26T01:38:20.320042+0000","last_clean":"2023-10-26T01:38:20.320042+0000","last_became_active":"2023-10-26T01:38:20.319754+0000","last_became_peered":"2023-10-26T01:38:20.319754+0000","last_unstale":"2023-10-26T01:38:20.320042+0000","last_undegraded":"2023-10-26T01:38:20.320042+0000","last_fullsized":"2023-10-26T01:38:20.320042+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:18:21.690998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.315394+0000","last_change":"2023-10-26T01:38:20.315394+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.315394+0000","last_undegraded":"2023-10-26T01:38:20.315394+0000","last_fullsized":"2023-10-26T01:38:20.315394+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-26T01:38:20.325008+0000","last_change":"2023-10-26T01:38:20.323205+0000","last_active":"2023-10-26T01:38:20.325008+0000","last_peered":"2023-10-26T01:38:20.325008+0000","last_clean":"2023-10-26T01:38:20.325008+0000","last_became_active":"2023-10-26T01:38:20.322992+0000","last_became_peered":"2023-10-26T01:38:20.322992+0000","last_unstale":"2023-10-26T01:38:20.325008+0000","last_undegraded":"2023-10-26T01:38:20.325008+0000","last_fullsized":"2023-10-26T01:38:20.325008+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:01:26.691609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.311293+0000","last_change":"2023-10-26T01:38:20.311293+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.311293+0000","last_undegraded":"2023-10-26T01:38:20.311293+0000","last_fullsized":"2023-10-26T01:38:20.311293+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-26T01:38:20.321263+0000","last_change":"2023-10-26T01:38:20.320536+0000","last_active":"2023-10-26T01:38:20.321263+0000","last_peered":"2023-10-26T01:38:20.321263+0000","last_clean":"2023-10-26T01:38:20.321263+0000","last_became_active":"2023-10-26T01:38:20.320277+0000","last_became_peered":"2023-10-26T01:38:20.320277+0000","last_unstale":"2023-10-26T01:38:20.321263+0000","last_undegraded":"2023-10-26T01:38:20.321263+0000","last_fullsized":"2023-10-26T01:38:20.321263+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:32:00.607826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.322832+0000","last_change":"2023-10-26T01:38:21.322582+0000","last_active":"2023-10-26T01:38:21.322832+0000","last_peered":"2023-10-26T01:38:21.322832+0000","last_clean":"2023-10-26T01:38:21.322832+0000","last_became_active":"2023-10-26T01:38:21.321971+0000","last_became_peered":"2023-10-26T01:38:21.321971+0000","last_unstale":"2023-10-26T01:38:21.322832+0000","last_undegraded":"2023-10-26T01:38:21.322832+0000","last_fullsized":"2023-10-26T01:38:21.322832+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:44:44.680316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-26T01:38:20.324540+0000","last_change":"2023-10-26T01:38:20.324450+0000","last_active":"2023-10-26T01:38:20.324540+0000","last_peered":"2023-10-26T01:38:20.324540+0000","last_clean":"2023-10-26T01:38:20.324540+0000","last_became_active":"2023-10-26T01:38:20.324194+0000","last_became_peered":"2023-10-26T01:38:20.324194+0000","last_unstale":"2023-10-26T01:38:20.324540+0000","last_undegraded":"2023-10-26T01:38:20.324540+0000","last_fullsized":"2023-10-26T01:38:20.324540+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:29:51.785956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.313704+0000","last_change":"2023-10-26T01:38:20.313704+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.313704+0000","last_undegraded":"2023-10-26T01:38:20.313704+0000","last_fullsized":"2023-10-26T01:38:20.313704+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"133'2","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.338835+0000","last_change":"2023-10-26T01:38:20.315954+0000","last_active":"2023-10-26T01:38:21.338835+0000","last_peered":"2023-10-26T01:38:21.338835+0000","last_clean":"2023-10-26T01:38:21.338835+0000","last_became_active":"2023-10-26T01:38:20.315727+0000","last_became_peered":"2023-10-26T01:38:20.315727+0000","last_unstale":"2023-10-26T01:38:21.338835+0000","last_undegraded":"2023-10-26T01:38:21.338835+0000","last_fullsized":"2023-10-26T01:38:21.338835+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:01:47.759723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.308802+0000","last_change":"2023-10-26T01:38:20.308802+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.308802+0000","last_undegraded":"2023-10-26T01:38:20.308802+0000","last_fullsized":"2023-10-26T01:38:20.308802+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-26T01:38:20.325709+0000","last_change":"2023-10-26T01:38:20.325614+0000","last_active":"2023-10-26T01:38:20.325709+0000","last_peered":"2023-10-26T01:38:20.325709+0000","last_clean":"2023-10-26T01:38:20.325709+0000","last_became_active":"2023-10-26T01:38:20.325288+0000","last_became_peered":"2023-10-26T01:38:20.325288+0000","last_unstale":"2023-10-26T01:38:20.325709+0000","last_undegraded":"2023-10-26T01:38:20.325709+0000","last_fullsized":"2023-10-26T01:38:20.325709+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:48:44.256103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.308821+0000","last_change":"2023-10-26T01:38:20.308821+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.308821+0000","last_undegraded":"2023-10-26T01:38:20.308821+0000","last_fullsized":"2023-10-26T01:38:20.308821+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-26T01:38:20.320876+0000","last_change":"2023-10-26T01:38:20.320712+0000","last_active":"2023-10-26T01:38:20.320876+0000","last_peered":"2023-10-26T01:38:20.320876+0000","last_clean":"2023-10-26T01:38:20.320876+0000","last_became_active":"2023-10-26T01:38:20.319884+0000","last_became_peered":"2023-10-26T01:38:20.319884+0000","last_unstale":"2023-10-26T01:38:20.320876+0000","last_undegraded":"2023-10-26T01:38:20.320876+0000","last_fullsized":"2023-10-26T01:38:20.320876+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T01:50:21.256149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.313034+0000","last_change":"2023-10-26T01:38:20.313034+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.313034+0000","last_undegraded":"2023-10-26T01:38:20.313034+0000","last_fullsized":"2023-10-26T01:38:20.313034+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.294223+0000","last_change":"2023-10-26T01:38:20.319596+0000","last_active":"2023-10-26T01:38:21.294223+0000","last_peered":"2023-10-26T01:38:21.294223+0000","last_clean":"2023-10-26T01:38:21.294223+0000","last_became_active":"2023-10-26T01:38:20.319376+0000","last_became_peered":"2023-10-26T01:38:20.319376+0000","last_unstale":"2023-10-26T01:38:21.294223+0000","last_undegraded":"2023-10-26T01:38:21.294223+0000","last_fullsized":"2023-10-26T01:38:21.294223+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:59:26.990814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.304704+0000","last_change":"2023-10-26T01:38:20.304704+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.304704+0000","last_undegraded":"2023-10-26T01:38:20.304704+0000","last_fullsized":"2023-10-26T01:38:20.304704+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-26T01:38:20.324168+0000","last_change":"2023-10-26T01:38:20.324083+0000","last_active":"2023-10-26T01:38:20.324168+0000","last_peered":"2023-10-26T01:38:20.324168+0000","last_clean":"2023-10-26T01:38:20.324168+0000","last_became_active":"2023-10-26T01:38:20.320737+0000","last_became_peered":"2023-10-26T01:38:20.320737+0000","last_unstale":"2023-10-26T01:38:20.324168+0000","last_undegraded":"2023-10-26T01:38:20.324168+0000","last_fullsized":"2023-10-26T01:38:20.324168+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:05:00.823929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.304194+0000","last_change":"2023-10-26T01:38:20.304194+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.304194+0000","last_undegraded":"2023-10-26T01:38:20.304194+0000","last_fullsized":"2023-10-26T01:38:20.304194+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-26T01:38:20.322096+0000","last_change":"2023-10-26T01:38:20.321433+0000","last_active":"2023-10-26T01:38:20.322096+0000","last_peered":"2023-10-26T01:38:20.322096+0000","last_clean":"2023-10-26T01:38:20.322096+0000","last_became_active":"2023-10-26T01:38:20.321136+0000","last_became_peered":"2023-10-26T01:38:20.321136+0000","last_unstale":"2023-10-26T01:38:20.322096+0000","last_undegraded":"2023-10-26T01:38:20.322096+0000","last_fullsized":"2023-10-26T01:38:20.322096+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:01:47.759723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.311489+0000","last_change":"2023-10-26T01:38:20.311489+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.311489+0000","last_undegraded":"2023-10-26T01:38:20.311489+0000","last_fullsized":"2023-10-26T01:38:20.311489+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-26T01:38:20.324966+0000","last_change":"2023-10-26T01:38:20.324901+0000","last_active":"2023-10-26T01:38:20.324966+0000","last_peered":"2023-10-26T01:38:20.324966+0000","last_clean":"2023-10-26T01:38:20.324966+0000","last_became_active":"2023-10-26T01:38:20.324441+0000","last_became_peered":"2023-10-26T01:38:20.324441+0000","last_unstale":"2023-10-26T01:38:20.324966+0000","last_undegraded":"2023-10-26T01:38:20.324966+0000","last_fullsized":"2023-10-26T01:38:20.324966+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:48:31.373334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":1,"reported_epoch":132,"state":"creating+peering","last_fresh":"2023-10-26T01:38:20.314840+0000","last_change":"2023-10-26T01:38:20.314840+0000","last_active":"2023-10-26T01:38:20.280569+0000","last_peered":"2023-10-26T01:38:20.280569+0000","last_clean":"2023-10-26T01:38:20.280569+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:38:20.314840+0000","last_undegraded":"2023-10-26T01:38:20.314840+0000","last_fullsized":"2023-10-26T01:38:20.314840+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-26T01:38:20.320980+0000","last_change":"2023-10-26T01:38:20.320871+0000","last_active":"2023-10-26T01:38:20.320980+0000","last_peered":"2023-10-26T01:38:20.320980+0000","last_clean":"2023-10-26T01:38:20.320980+0000","last_became_active":"2023-10-26T01:38:20.320224+0000","last_became_peered":"2023-10-26T01:38:20.320224+0000","last_unstale":"2023-10-26T01:38:20.320980+0000","last_undegraded":"2023-10-26T01:38:20.320980+0000","last_fullsized":"2023-10-26T01:38:20.320980+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:51:41.698100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.322813+0000","last_change":"2023-10-26T01:38:21.321771+0000","last_active":"2023-10-26T01:38:21.322813+0000","last_peered":"2023-10-26T01:38:21.322813+0000","last_clean":"2023-10-26T01:38:21.322813+0000","last_became_active":"2023-10-26T01:38:21.321349+0000","last_became_peered":"2023-10-26T01:38:21.321349+0000","last_unstale":"2023-10-26T01:38:21.322813+0000","last_undegraded":"2023-10-26T01:38:21.322813+0000","last_fullsized":"2023-10-26T01:38:21.322813+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:22:49.920143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-26T01:38:20.322070+0000","last_change":"2023-10-26T01:38:20.321951+0000","last_active":"2023-10-26T01:38:20.322070+0000","last_peered":"2023-10-26T01:38:20.322070+0000","last_clean":"2023-10-26T01:38:20.322070+0000","last_became_active":"2023-10-26T01:38:20.310341+0000","last_became_peered":"2023-10-26T01:38:20.310341+0000","last_unstale":"2023-10-26T01:38:20.322070+0000","last_undegraded":"2023-10-26T01:38:20.322070+0000","last_fullsized":"2023-10-26T01:38:20.322070+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:54:02.310203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":276,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-26T01:38:19.284336+0000","last_change":"2023-10-26T01:30:34.883484+0000","last_active":"2023-10-26T01:38:19.284336+0000","last_peered":"2023-10-26T01:38:19.284336+0000","last_clean":"2023-10-26T01:38:19.284336+0000","last_became_active":"2023-10-26T01:30:34.883071+0000","last_became_peered":"2023-10-26T01:30:34.883071+0000","last_unstale":"2023-10-26T01:38:19.284336+0000","last_undegraded":"2023-10-26T01:38:19.284336+0000","last_fullsized":"2023-10-26T01:38:19.284336+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:32.890096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:32.890096+0000","last_clean_scrub_stamp":"2023-10-26T01:30:32.890096+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:26:31.879452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":14,"seq":60129542241,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":719,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194592,"statfs":{"total":200000000,"available":199262922,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.79200000000000004,"15min":0.78600000000000003},"min":{"1min":0.44900000000000001,"5min":0.35599999999999998,"15min":0.33000000000000002},"max":{"1min":1.6319999999999999,"5min":1.6319999999999999,"15min":2.2549999999999999},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.876,"15min":0.88800000000000001},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.236,"5min":1.5800000000000001,"15min":1.8200000000000001},"last":0.94099999999999995}]},{"osd":1,"last update":"Thu Oct 26 01:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.875,"15min":0.93700000000000006},"min":{"1min":0.40699999999999997,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.3560000000000001,"5min":1.444,"15min":4.4340000000000002},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.97599999999999998,"15min":0.997},"min":{"1min":0.51500000000000001,"5min":0.46700000000000003,"15min":0.374},"max":{"1min":1.4390000000000001,"5min":1.538,"15min":2.3330000000000002},"last":0.755}]},{"osd":2,"last update":"Thu Oct 26 01:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":1.032,"15min":1.075},"min":{"1min":0.53800000000000003,"5min":0.40999999999999998,"15min":0.39900000000000002},"max":{"1min":1.643,"5min":1.927,"15min":4.2800000000000002},"last":1.2689999999999999},{"interface":"front","average":{"1min":1.032,"5min":1.1279999999999999,"15min":1.091},"min":{"1min":0.50900000000000001,"5min":0.495,"15min":0.49399999999999999},"max":{"1min":1.607,"5min":1.8680000000000001,"15min":2.302},"last":1.3320000000000001}]},{"osd":4,"last update":"Thu Oct 26 01:37:49 2023","interfaces":[{"interface":"back","average":{"1min":1.175,"5min":1.2070000000000001,"15min":1.167},"min":{"1min":0.755,"5min":0.755,"15min":0.66400000000000003},"max":{"1min":1.536,"5min":1.7150000000000001,"15min":2.1070000000000002},"last":1.008},{"interface":"front","average":{"1min":1.252,"5min":1.3400000000000001,"15min":1.389},"min":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.79900000000000004},"max":{"1min":1.5209999999999999,"5min":2.4239999999999999,"15min":4.8250000000000002},"last":1.5740000000000001}]},{"osd":5,"last update":"Thu Oct 26 01:37:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.282,"15min":1.333},"min":{"1min":0.79300000000000004,"5min":0.76400000000000001,"15min":0.72499999999999998},"max":{"1min":1.639,"5min":2.3380000000000001,"15min":4.9080000000000004},"last":1.448},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.28,"15min":1.276},"min":{"1min":0.90300000000000002,"5min":0.79900000000000004,"15min":0.79600000000000004},"max":{"1min":1.722,"5min":2.0310000000000001,"15min":2.8980000000000001},"last":1.1120000000000001}]},{"osd":6,"last update":"Thu Oct 26 01:37:49 2023","interfaces":[{"interface":"back","average":{"1min":1.278,"5min":1.355,"15min":1.331},"min":{"1min":0.74199999999999999,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.5580000000000001,"5min":2.1499999999999999,"15min":2.8530000000000002},"last":1.2270000000000001},{"interface":"front","average":{"1min":1.206,"5min":1.268,"15min":1.268},"min":{"1min":0.81699999999999995,"5min":0.81499999999999995,"15min":0.73099999999999998},"max":{"1min":1.7410000000000001,"5min":2.048,"15min":2.181},"last":1.0820000000000001}]},{"osd":7,"last update":"Thu Oct 26 01:37:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.3300000000000001,"15min":1.296},"min":{"1min":0.85299999999999998,"5min":0.84599999999999997,"15min":0.71999999999999997},"max":{"1min":1.8129999999999999,"5min":1.958,"15min":2.0680000000000001},"last":1.046},{"interface":"front","average":{"1min":1.2889999999999999,"5min":1.3919999999999999,"15min":1.415},"min":{"1min":0.84199999999999997,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.647,"5min":2.2370000000000001,"15min":4.8730000000000002},"last":1.6399999999999999}]}]},{"osd":7,"up_from":13,"seq":55834574944,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1288,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194024,"statfs":{"total":200000000,"available":198681011,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:37:49 2023","interfaces":[{"interface":"back","average":{"1min":1.0920000000000001,"5min":1.139,"15min":1.093},"min":{"1min":0.81799999999999995,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.7,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":1.28},{"interface":"front","average":{"1min":0.99099999999999999,"5min":1.115,"15min":1.081},"min":{"1min":0.70599999999999996,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.46,"5min":2.6779999999999999,"15min":2.6779999999999999},"last":1.6970000000000001}]},{"osd":1,"last update":"Thu Oct 26 01:37:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.1870000000000001,"15min":1.0960000000000001},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65200000000000002},"max":{"1min":1.5549999999999999,"5min":2.8260000000000001,"15min":2.8260000000000001},"last":1.137},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.21,"15min":1.194},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.61199999999999999},"max":{"1min":1.7869999999999999,"5min":3.2730000000000001,"15min":3.2730000000000001},"last":1.76}]},{"osd":2,"last update":"Thu Oct 26 01:37:49 2023","interfaces":[{"interface":"back","average":{"1min":1.143,"5min":1.179,"15min":1.1180000000000001},"min":{"1min":0.79100000000000004,"5min":0.65700000000000003,"15min":0.58499999999999996},"max":{"1min":1.667,"5min":2.218,"15min":2.218},"last":1.3120000000000001},{"interface":"front","average":{"1min":1.2709999999999999,"5min":1.2989999999999999,"15min":1.24},"min":{"1min":0.89200000000000002,"5min":0.79400000000000004,"15min":0.73099999999999998},"max":{"1min":1.786,"5min":3.0510000000000002,"15min":3.0510000000000002},"last":1.347}]},{"osd":3,"last update":"Thu Oct 26 01:37:49 2023","interfaces":[{"interface":"back","average":{"1min":1.264,"5min":1.284,"15min":1.214},"min":{"1min":0.93500000000000005,"5min":0.79000000000000004,"15min":0.72099999999999997},"max":{"1min":1.5389999999999999,"5min":1.9830000000000001,"15min":1.9830000000000001},"last":1.5620000000000001},{"interface":"front","average":{"1min":1.296,"5min":1.407,"15min":1.373},"min":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.65700000000000003},"max":{"1min":1.8939999999999999,"5min":5.5039999999999996,"15min":5.5039999999999996},"last":1.383}]},{"osd":4,"last update":"Thu Oct 26 01:37:49 2023","interfaces":[{"interface":"back","average":{"1min":1.4350000000000001,"5min":1.373,"15min":1.3520000000000001},"min":{"1min":1.0589999999999999,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.7450000000000001,"5min":2.77,"15min":2.77},"last":1.0669999999999999},{"interface":"front","average":{"1min":1.456,"5min":1.429,"15min":1.399},"min":{"1min":0.92700000000000005,"5min":0.90400000000000003,"15min":0.496},"max":{"1min":1.95,"5min":3.1419999999999999,"15min":3.1419999999999999},"last":1.46}]},{"osd":5,"last update":"Thu Oct 26 01:37:49 2023","interfaces":[{"interface":"back","average":{"1min":1.498,"5min":1.488,"15min":1.4259999999999999},"min":{"1min":0.91700000000000004,"5min":0.65600000000000003,"15min":0.441},"max":{"1min":1.9890000000000001,"5min":5.734,"15min":5.734},"last":1.609},{"interface":"front","average":{"1min":1.4970000000000001,"5min":1.5269999999999999,"15min":1.478},"min":{"1min":1.1739999999999999,"5min":0.96099999999999997,"15min":0.51900000000000002},"max":{"1min":1.8799999999999999,"5min":2.8980000000000001,"15min":2.8980000000000001},"last":1.1870000000000001}]},{"osd":6,"last update":"Thu Oct 26 01:37:49 2023","interfaces":[{"interface":"back","average":{"1min":1.5580000000000001,"5min":1.6080000000000001,"15min":1.8460000000000001},"min":{"1min":1.0169999999999999,"5min":0.753,"15min":0.47399999999999998},"max":{"1min":2.0739999999999998,"5min":5.6340000000000003,"15min":11.577999999999999},"last":1.8049999999999999},{"interface":"front","average":{"1min":1.615,"5min":1.6779999999999999,"15min":1.8620000000000001},"min":{"1min":1.2889999999999999,"5min":0.91800000000000004,"15min":0.55100000000000005},"max":{"1min":1.986,"5min":5.0919999999999996,"15min":10.704000000000001},"last":1.6519999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574944,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":704,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194607,"statfs":{"total":200000000,"available":199278268,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:37:43 2023","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.113,"15min":1.1539999999999999},"min":{"1min":0.69299999999999995,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.6100000000000001,"5min":2.395,"15min":2.395},"last":1.1830000000000001},{"interface":"front","average":{"1min":1.139,"5min":1.173,"15min":1.1220000000000001},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.59199999999999997},"max":{"1min":2.081,"5min":2.081,"15min":2.081},"last":1.5349999999999999}]},{"osd":1,"last update":"Thu Oct 26 01:37:43 2023","interfaces":[{"interface":"back","average":{"1min":1.2749999999999999,"5min":1.2110000000000001,"15min":1.1779999999999999},"min":{"1min":0.82499999999999996,"5min":0.71599999999999997,"15min":0.68300000000000005},"max":{"1min":1.992,"5min":2.0529999999999999,"15min":2.0529999999999999},"last":0.83399999999999996},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.179,"15min":1.204},"min":{"1min":0.67900000000000005,"5min":0.67700000000000005,"15min":0.64000000000000001},"max":{"1min":1.968,"5min":1.968,"15min":2.085},"last":1.359}]},{"osd":2,"last update":"Thu Oct 26 01:37:43 2023","interfaces":[{"interface":"back","average":{"1min":1.077,"5min":1.1879999999999999,"15min":1.095},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.5649999999999999,"5min":3.7879999999999998,"15min":3.7879999999999998},"last":1.5629999999999999},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.3120000000000001,"15min":1.268},"min":{"1min":0.90600000000000003,"5min":0.75800000000000001,"15min":0.68899999999999995},"max":{"1min":2.2200000000000002,"5min":2.2890000000000001,"15min":2.2890000000000001},"last":1.3129999999999999}]},{"osd":3,"last update":"Thu Oct 26 01:37:43 2023","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.319,"15min":1.214},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.756,"5min":4.4560000000000004,"15min":4.4560000000000004},"last":1.4059999999999999},{"interface":"front","average":{"1min":1.288,"5min":1.337,"15min":1.325},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.69499999999999995},"max":{"1min":2.1520000000000001,"5min":2.5019999999999998,"15min":2.5019999999999998},"last":0.96299999999999997}]},{"osd":4,"last update":"Thu Oct 26 01:37:43 2023","interfaces":[{"interface":"back","average":{"1min":1.2709999999999999,"5min":1.272,"15min":1.224},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.48399999999999999},"max":{"1min":1.754,"5min":2.9489999999999998,"15min":2.9489999999999998},"last":1.042},{"interface":"front","average":{"1min":1.272,"5min":1.4299999999999999,"15min":1.3149999999999999},"min":{"1min":0.96799999999999997,"5min":0.81100000000000005,"15min":0.59299999999999997},"max":{"1min":1.7629999999999999,"5min":6.609,"15min":6.609},"last":1.2509999999999999}]},{"osd":5,"last update":"Thu Oct 26 01:37:43 2023","interfaces":[{"interface":"back","average":{"1min":1.4570000000000001,"5min":1.5469999999999999,"15min":1.4470000000000001},"min":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.67000000000000004},"max":{"1min":1.8380000000000001,"5min":5.4370000000000003,"15min":5.4370000000000003},"last":1.6950000000000001},{"interface":"front","average":{"1min":1.4370000000000001,"5min":1.4410000000000001,"15min":1.4119999999999999},"min":{"1min":0.626,"5min":0.61399999999999999,"15min":0.58199999999999996},"max":{"1min":2.3279999999999998,"5min":2.621,"15min":2.621},"last":1.5089999999999999}]},{"osd":7,"last update":"Thu Oct 26 01:37:43 2023","interfaces":[{"interface":"back","average":{"1min":1.3400000000000001,"5min":1.52,"15min":1.4099999999999999},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.54100000000000004},"max":{"1min":1.8740000000000001,"5min":11.804,"15min":11.804},"last":1.45},{"interface":"front","average":{"1min":1.5049999999999999,"5min":1.6559999999999999,"15min":1.51},"min":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.77300000000000002},"max":{"1min":2.0710000000000002,"5min":11.541,"15min":11.541},"last":1.6399999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574944,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":705,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194607,"statfs":{"total":200000000,"available":199277654,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:37:46 2023","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.1870000000000001,"15min":1.129},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.5660000000000001,"5min":1.7390000000000001,"15min":1.7390000000000001},"last":1.3700000000000001},{"interface":"front","average":{"1min":1.204,"5min":1.28,"15min":1.1819999999999999},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.613,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":1.198}]},{"osd":1,"last update":"Thu Oct 26 01:37:46 2023","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":1.1319999999999999,"15min":1.085},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.3640000000000001,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":0.97799999999999998},{"interface":"front","average":{"1min":1.069,"5min":1.137,"15min":1.095},"min":{"1min":0.629,"5min":0.629,"15min":0.60599999999999998},"max":{"1min":1.726,"5min":1.726,"15min":1.7669999999999999},"last":1.2330000000000001}]},{"osd":2,"last update":"Thu Oct 26 01:37:46 2023","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":1.1950000000000001,"15min":1.163},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.69299999999999995},"max":{"1min":1.681,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":1.3959999999999999},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.2290000000000001,"15min":1.181},"min":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.63900000000000001},"max":{"1min":1.4339999999999999,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":1.1639999999999999}]},{"osd":3,"last update":"Thu Oct 26 01:37:46 2023","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.2949999999999999,"15min":1.238},"min":{"1min":0.78900000000000003,"5min":0.72699999999999998,"15min":0.72299999999999998},"max":{"1min":1.6180000000000001,"5min":1.7649999999999999,"15min":1.835},"last":1.49},{"interface":"front","average":{"1min":1.2529999999999999,"5min":1.3220000000000001,"15min":1.2929999999999999},"min":{"1min":0.85199999999999998,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.7310000000000001,"5min":1.788,"15min":2.9249999999999998},"last":1.1439999999999999}]},{"osd":4,"last update":"Thu Oct 26 01:37:46 2023","interfaces":[{"interface":"back","average":{"1min":1.1519999999999999,"5min":1.2689999999999999,"15min":1.2050000000000001},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.49099999999999999},"max":{"1min":1.6719999999999999,"5min":1.845,"15min":1.845},"last":1.097},{"interface":"front","average":{"1min":1.3340000000000001,"5min":1.4359999999999999,"15min":1.355},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.56200000000000006},"max":{"1min":1.78,"5min":1.867,"15min":1.867},"last":1.0740000000000001}]},{"osd":6,"last update":"Thu Oct 26 01:37:46 2023","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.4319999999999999,"15min":1.323},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.51900000000000002},"max":{"1min":1.7490000000000001,"5min":2.8250000000000002,"15min":2.8250000000000002},"last":1.3440000000000001},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.423,"15min":1.327},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.61799999999999999},"max":{"1min":1.661,"5min":1.849,"15min":1.8560000000000001},"last":1.3160000000000001}]},{"osd":7,"last update":"Thu Oct 26 01:37:46 2023","interfaces":[{"interface":"back","average":{"1min":1.4219999999999999,"5min":1.536,"15min":1.4219999999999999},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.61699999999999999},"max":{"1min":1.8220000000000001,"5min":2.1040000000000001,"15min":2.1040000000000001},"last":1.2909999999999999},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.5609999999999999,"15min":1.431},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.59499999999999997},"max":{"1min":1.853,"5min":2.0960000000000001,"15min":2.0960000000000001},"last":1.431}]}]},{"osd":4,"up_from":13,"seq":55834574944,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":705,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194607,"statfs":{"total":200000000,"available":199277654,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:37:42 2023","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.056,"15min":0.98499999999999999},"min":{"1min":0.63800000000000001,"5min":0.58099999999999996,"15min":0.53100000000000003},"max":{"1min":1.5369999999999999,"5min":1.885,"15min":2.0289999999999999},"last":1.522},{"interface":"front","average":{"1min":1.0309999999999999,"5min":1.081,"15min":1.024},"min":{"1min":0.74099999999999999,"5min":0.66400000000000003,"15min":0.60599999999999998},"max":{"1min":1.5029999999999999,"5min":2.6829999999999998,"15min":2.6829999999999998},"last":1.4790000000000001}]},{"osd":1,"last update":"Thu Oct 26 01:37:42 2023","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.2,"15min":1.127},"min":{"1min":0.66700000000000004,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.591,"5min":4.0519999999999996,"15min":4.0519999999999996},"last":1.375},{"interface":"front","average":{"1min":1.1599999999999999,"5min":1.1559999999999999,"15min":1.1100000000000001},"min":{"1min":0.70399999999999996,"5min":0.627,"15min":0.60499999999999998},"max":{"1min":1.897,"5min":1.897,"15min":2.0760000000000001},"last":1.3100000000000001}]},{"osd":2,"last update":"Thu Oct 26 01:37:42 2023","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.3480000000000001,"15min":1.23},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.64300000000000002},"max":{"1min":1.845,"5min":5.6749999999999998,"15min":5.6749999999999998},"last":1.6140000000000001},{"interface":"front","average":{"1min":1.3,"5min":1.218,"15min":1.1359999999999999},"min":{"1min":0.73899999999999999,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":3.673,"5min":3.673,"15min":3.673},"last":1.3400000000000001}]},{"osd":3,"last update":"Thu Oct 26 01:37:42 2023","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.204,"15min":1.1870000000000001},"min":{"1min":0.91500000000000004,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.7669999999999999,"5min":1.847,"15min":1.891},"last":1.425},{"interface":"front","average":{"1min":1.194,"5min":1.21,"15min":1.159},"min":{"1min":0.85399999999999998,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.6419999999999999,"5min":1.7210000000000001,"15min":1.7210000000000001},"last":0.89800000000000002}]},{"osd":5,"last update":"Thu Oct 26 01:37:42 2023","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":1.2549999999999999,"15min":1.286},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.46300000000000002},"max":{"1min":1.8520000000000001,"5min":2.1429999999999998,"15min":2.1429999999999998},"last":1.087},{"interface":"front","average":{"1min":1.3460000000000001,"5min":1.3500000000000001,"15min":1.3280000000000001},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.7649999999999999,"5min":5.8579999999999997,"15min":5.8579999999999997},"last":1.208}]},{"osd":6,"last update":"Thu Oct 26 01:37:42 2023","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":1.427,"15min":1.3380000000000001},"min":{"1min":0.69199999999999995,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":2.2149999999999999,"5min":12.343999999999999,"15min":12.343999999999999},"last":0.97299999999999998},{"interface":"front","average":{"1min":1.492,"5min":1.619,"15min":1.5149999999999999},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.55000000000000004},"max":{"1min":2.1000000000000001,"5min":12.605,"15min":12.605},"last":1.5760000000000001}]},{"osd":7,"last update":"Thu Oct 26 01:37:42 2023","interfaces":[{"interface":"back","average":{"1min":1.4710000000000001,"5min":1.52,"15min":1.49},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":2.0390000000000001,"5min":5.774,"15min":5.774},"last":1.706},{"interface":"front","average":{"1min":1.53,"5min":1.5309999999999999,"15min":1.46},"min":{"1min":0.79000000000000004,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":2.3450000000000002,"5min":4.6520000000000001,"15min":4.6520000000000001},"last":1.2769999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574944,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":711,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194600,"statfs":{"total":200000000,"available":199271379,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:37:46 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.84099999999999997,"15min":0.85499999999999998},"min":{"1min":0.41199999999999998,"5min":0.308,"15min":0.308},"max":{"1min":1.4850000000000001,"5min":1.5249999999999999,"15min":6.3200000000000003},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.997,"15min":0.93100000000000005},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.2569999999999999,"5min":2.1400000000000001,"15min":4.2619999999999996},"last":1.1930000000000001}]},{"osd":1,"last update":"Thu Oct 26 01:37:46 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.90200000000000002,"15min":0.89700000000000002},"min":{"1min":0.39300000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.244,"5min":1.629,"15min":5.1260000000000003},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.85399999999999998,"5min":1.018,"15min":1.0029999999999999},"min":{"1min":0.47699999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.351,"5min":1.873,"15min":6.1520000000000001},"last":1.339}]},{"osd":3,"last update":"Thu Oct 26 01:37:48 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":1.0189999999999999,"15min":0.93899999999999995},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.443},"max":{"1min":1.4390000000000001,"5min":2.0640000000000001,"15min":2.0640000000000001},"last":1.0609999999999999},{"interface":"front","average":{"1min":0.93600000000000005,"5min":1.111,"15min":1.018},"min":{"1min":0.51100000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.375,"5min":1.9279999999999999,"15min":8.9019999999999992},"last":0.98199999999999998}]},{"osd":4,"last update":"Thu Oct 26 01:37:46 2023","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.3420000000000001,"15min":1.2689999999999999},"min":{"1min":0.86499999999999999,"5min":0.76000000000000001,"15min":0.73899999999999999},"max":{"1min":1.704,"5min":2.6190000000000002,"15min":6.2670000000000003},"last":1.294},{"interface":"front","average":{"1min":1.1879999999999999,"5min":1.3129999999999999,"15min":1.262},"min":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.78700000000000003},"max":{"1min":1.5920000000000001,"5min":2.4359999999999999,"15min":7.343},"last":1.1200000000000001}]},{"osd":5,"last update":"Thu Oct 26 01:37:46 2023","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.3540000000000001,"15min":1.2869999999999999},"min":{"1min":0.873,"5min":0.81699999999999995,"15min":0.71899999999999997},"max":{"1min":1.5740000000000001,"5min":2.5790000000000002,"15min":3.7170000000000001},"last":1.2470000000000001},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.3089999999999999,"15min":1.2909999999999999},"min":{"1min":0.81899999999999995,"5min":0.70999999999999996,"15min":0.65300000000000002},"max":{"1min":1.621,"5min":2.298,"15min":6.3899999999999997},"last":1.5169999999999999}]},{"osd":6,"last update":"Thu Oct 26 01:37:46 2023","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":1.399,"15min":1.335},"min":{"1min":0.81200000000000006,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.9379999999999999,"5min":2.73,"15min":6.0490000000000004},"last":1.4790000000000001},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.4179999999999999,"15min":1.3720000000000001},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.76800000000000002},"max":{"1min":1.8799999999999999,"5min":2.4089999999999998,"15min":5.0650000000000004},"last":1.4399999999999999}]},{"osd":7,"last update":"Thu Oct 26 01:37:46 2023","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.403,"15min":1.395},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.69399999999999995},"max":{"1min":1.7809999999999999,"5min":2.3370000000000002,"15min":6.4480000000000004},"last":1.6140000000000001},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.52,"15min":1.4550000000000001},"min":{"1min":1.036,"5min":0.91800000000000004,"15min":0.91800000000000004},"max":{"1min":1.877,"5min":2.8290000000000002,"15min":2.8439999999999999},"last":1.5640000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574944,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":711,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194600,"statfs":{"total":200000000,"available":199271379,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:37:48 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.78300000000000003,"15min":0.72499999999999998},"min":{"1min":0.36099999999999999,"5min":0.34000000000000002,"15min":0.29499999999999998},"max":{"1min":1.4099999999999999,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.753},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.86799999999999999,"15min":0.79600000000000004},"min":{"1min":0.435,"5min":0.36299999999999999,"15min":0.36199999999999999},"max":{"1min":1.4019999999999999,"5min":1.6399999999999999,"15min":1.6399999999999999},"last":0.84699999999999998}]},{"osd":2,"last update":"Thu Oct 26 01:37:48 2023","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":0.97899999999999998,"15min":1.0009999999999999},"min":{"1min":0.52200000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.8200000000000001,"5min":1.8200000000000001,"15min":2.4830000000000001},"last":1.0489999999999999},{"interface":"front","average":{"1min":1.0569999999999999,"5min":0.93700000000000006,"15min":0.875},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.772,"5min":1.8080000000000001,"15min":1.8080000000000001},"last":1.256}]},{"osd":3,"last update":"Thu Oct 26 01:37:52 2023","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.97899999999999998,"15min":0.94899999999999995},"min":{"1min":0.52900000000000003,"5min":0.42499999999999999,"15min":0.39500000000000002},"max":{"1min":1.288,"5min":1.611,"15min":1.75},"last":1.1399999999999999},{"interface":"front","average":{"1min":0.97899999999999998,"5min":1.028,"15min":0.98999999999999999},"min":{"1min":0.55900000000000005,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.45,"5min":1.673,"15min":1.802},"last":0.94399999999999995}]},{"osd":4,"last update":"Thu Oct 26 01:37:48 2023","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":1.2270000000000001,"15min":1.1990000000000001},"min":{"1min":0.82899999999999996,"5min":0.69099999999999995,"15min":0.56200000000000006},"max":{"1min":1.756,"5min":1.756,"15min":2.0659999999999998},"last":1.5249999999999999},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.1930000000000001,"15min":1.1359999999999999},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.474,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":1.292}]},{"osd":5,"last update":"Thu Oct 26 01:37:48 2023","interfaces":[{"interface":"back","average":{"1min":1.1459999999999999,"5min":1.262,"15min":1.2270000000000001},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.573,"5min":1.8779999999999999,"15min":1.8779999999999999},"last":1.077},{"interface":"front","average":{"1min":1.3720000000000001,"5min":1.3720000000000001,"15min":1.337},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":2.04,"5min":2.04,"15min":2.2200000000000002},"last":1.3220000000000001}]},{"osd":6,"last update":"Thu Oct 26 01:37:48 2023","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":1.3320000000000001,"15min":1.28},"min":{"1min":0.75,"5min":0.70199999999999996,"15min":0.68899999999999995},"max":{"1min":1.7330000000000001,"5min":2.0910000000000002,"15min":2.3780000000000001},"last":1.2370000000000001},{"interface":"front","average":{"1min":1.26,"5min":1.2729999999999999,"15min":1.244},"min":{"1min":0.80700000000000005,"5min":0.71299999999999997,"15min":0.70799999999999996},"max":{"1min":1.669,"5min":1.956,"15min":1.984},"last":1.3859999999999999}]},{"osd":7,"last update":"Thu Oct 26 01:37:48 2023","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.2629999999999999,"15min":1.262},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.69699999999999995},"max":{"1min":1.7230000000000001,"5min":1.8380000000000001,"15min":2.6930000000000001},"last":1.3560000000000001},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.4019999999999999,"15min":1.337},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.9370000000000001,"5min":2.0190000000000001,"15min":2.0190000000000001},"last":1.5720000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574944,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1288,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194024,"statfs":{"total":200000000,"available":198681011,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Oct 26 01:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":1.0029999999999999,"15min":0.91600000000000004},"min":{"1min":0.41699999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.2170000000000001,"5min":19.425000000000001,"15min":19.425000000000001},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.84199999999999997,"5min":1.0720000000000001,"15min":1.042},"min":{"1min":0.56100000000000005,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.1839999999999999,"5min":15.162000000000001,"15min":15.162000000000001},"last":1.1499999999999999}]},{"osd":2,"last update":"Thu Oct 26 01:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":1.3080000000000001,"15min":1.1819999999999999},"min":{"1min":0.57699999999999996,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.4930000000000001,"5min":28.439,"15min":28.439},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.91700000000000004,"15min":0.91200000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.4810000000000001,"5min":1.502,"15min":4.9459999999999997},"last":0.64300000000000002}]},{"osd":3,"last update":"Thu Oct 26 01:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":1.0549999999999999,"15min":1.085},"min":{"1min":0.48899999999999999,"5min":0.46000000000000002,"15min":0.44600000000000001},"max":{"1min":1.4330000000000001,"5min":1.6479999999999999,"15min":4.9800000000000004},"last":1.111},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.1459999999999999,"15min":1.155},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.4770000000000001,"5min":1.659,"15min":5.016},"last":1.2330000000000001}]},{"osd":4,"last update":"Thu Oct 26 01:37:45 2023","interfaces":[{"interface":"back","average":{"1min":1.091,"5min":1.379,"15min":1.29},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.49,"5min":19.329000000000001,"15min":19.329000000000001},"last":0.97799999999999998},{"interface":"front","average":{"1min":1.22,"5min":1.268,"15min":1.2989999999999999},"min":{"1min":0.88,"5min":0.82099999999999995,"15min":0.67600000000000005},"max":{"1min":1.5449999999999999,"5min":1.718,"15min":5.5229999999999997},"last":1.3460000000000001}]},{"osd":5,"last update":"Thu Oct 26 01:37:45 2023","interfaces":[{"interface":"back","average":{"1min":1.373,"5min":1.4039999999999999,"15min":1.4299999999999999},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.661,"5min":1.915,"15min":5.6360000000000001},"last":1.395},{"interface":"front","average":{"1min":1.202,"5min":1.4790000000000001,"15min":1.3999999999999999},"min":{"1min":0.73599999999999999,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.71,"5min":19.384,"15min":19.384},"last":1.1839999999999999}]},{"osd":6,"last update":"Thu Oct 26 01:37:45 2023","interfaces":[{"interface":"back","average":{"1min":1.2609999999999999,"5min":1.3799999999999999,"15min":1.365},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.73799999999999999},"max":{"1min":1.615,"5min":8.2609999999999992,"15min":8.2609999999999992},"last":1.4650000000000001},{"interface":"front","average":{"1min":1.3440000000000001,"5min":1.4590000000000001,"15min":1.407},"min":{"1min":0.83099999999999996,"5min":0.77900000000000003,"15min":0.75900000000000001},"max":{"1min":1.804,"5min":11.849,"15min":11.849},"last":1.429}]},{"osd":7,"last update":"Thu Oct 26 01:37:45 2023","interfaces":[{"interface":"back","average":{"1min":1.3979999999999999,"5min":1.4450000000000001,"15min":1.4810000000000001},"min":{"1min":0.871,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.8380000000000001,"5min":2.4060000000000001,"15min":5.6689999999999996},"last":1.3149999999999999},{"interface":"front","average":{"1min":1.365,"5min":1.6120000000000001,"15min":1.5029999999999999},"min":{"1min":0.89600000000000002,"5min":0.879,"15min":0.69399999999999995},"max":{"1min":1.841,"5min":19.245999999999999,"15min":19.245999999999999},"last":1.2809999999999999}]}]}],"pool_statfs":[]}} 2023-10-26T01:38:26.316 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-26T01:38:26.571 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:26.568+0000 7f9b7974c640 1 -- 172.21.15.169:0/3842876534 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9b740cdc30 msgr2=0x7f9b7406ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:26.571 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:26.568+0000 7f9b7974c640 1 --2- 172.21.15.169:0/3842876534 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9b740cdc30 0x7f9b7406ad30 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f9b740b3440 tx=0x7f9b6c02c1d0 comp rx=0 tx=0).stop 2023-10-26T01:38:26.571 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:26.569+0000 7f9b7974c640 1 -- 172.21.15.169:0/3842876534 shutdown_connections 2023-10-26T01:38:26.571 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:26.569+0000 7f9b7974c640 1 --2- 172.21.15.169:0/3842876534 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9b740cdc30 0x7f9b7406ad30 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:26.572 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:26.569+0000 7f9b7974c640 1 --2- 172.21.15.169:0/3842876534 >> v2:172.21.15.170:3300/0 conn(0x7f9b740ac930 0x7f9b740cd6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:26.572 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:26.569+0000 7f9b7974c640 1 --2- 172.21.15.169:0/3842876534 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9b741592d0 0x7f9b740aa3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:26.572 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:26.569+0000 7f9b7974c640 1 -- 172.21.15.169:0/3842876534 >> 172.21.15.169:0/3842876534 conn(0x7f9b74069930 msgr2=0x7f9b74069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:26.572 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:26.569+0000 7f9b7974c640 1 -- 172.21.15.169:0/3842876534 shutdown_connections 2023-10-26T01:38:26.572 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:26.569+0000 7f9b7974c640 1 -- 172.21.15.169:0/3842876534 wait complete. 2023-10-26T01:38:26.575 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:26.570+0000 7f9b7974c640 1 Processor -- start 2023-10-26T01:38:26.575 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:26.571+0000 7f9b7974c640 1 -- start start 2023-10-26T01:38:26.575 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:26.571+0000 7f9b7974c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9b740ac930 0x7f9b741f1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:26.575 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:26.571+0000 7f9b7974c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9b740cdc30 0x7f9b740c7750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:26.575 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:26.572+0000 7f9b7974c640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9b741592d0 0x7f9b7415b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:26.576 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:26.572+0000 7f9b7974c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b740c7c90 con 0x7f9b740cdc30 2023-10-26T01:38:26.576 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:26.572+0000 7f9b7974c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b7415bb90 con 0x7f9b740ac930 2023-10-26T01:38:26.576 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:26.572+0000 7f9b7974c640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b7415bdf0 con 0x7f9b741592d0 2023-10-26T01:38:26.576 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:26.572+0000 7f9b727fc640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9b740cdc30 0x7f9b740c7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:26.576 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:26.572+0000 7f9b727fc640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9b740cdc30 0x7f9b740c7750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:34308/0 (socket says 172.21.15.169:34308) 2023-10-26T01:38:26.577 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:26.572+0000 7f9b727fc640 1 -- 172.21.15.169:0/2876694596 learned_addr learned my addr 172.21.15.169:0/2876694596 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:38:26.577 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:26.572+0000 7f9b72ffd640 1 --2- 172.21.15.169:0/2876694596 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9b740ac930 0x7f9b741f1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:26.577 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:26.572+0000 7f9b737fe640 1 --2- 172.21.15.169:0/2876694596 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9b741592d0 0x7f9b7415b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:26.577 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:26.572+0000 7f9b727fc640 1 -- 172.21.15.169:0/2876694596 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9b741592d0 msgr2=0x7f9b7415b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:26.577 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:26.572+0000 7f9b727fc640 1 --2- 172.21.15.169:0/2876694596 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9b741592d0 0x7f9b7415b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:26.578 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:26.572+0000 7f9b727fc640 1 -- 172.21.15.169:0/2876694596 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9b740ac930 msgr2=0x7f9b741f1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:26.578 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:26.572+0000 7f9b727fc640 1 --2- 172.21.15.169:0/2876694596 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9b740ac930 0x7f9b741f1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:26.578 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:26.572+0000 7f9b727fc640 1 -- 172.21.15.169:0/2876694596 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b6c009900 con 0x7f9b740cdc30 2023-10-26T01:38:26.578 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:26.572+0000 7f9b72ffd640 1 --2- 172.21.15.169:0/2876694596 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9b740ac930 0x7f9b741f1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:38:26.578 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:26.573+0000 7f9b727fc640 1 --2- 172.21.15.169:0/2876694596 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9b740cdc30 0x7f9b740c7750 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f9b6800c920 tx=0x7f9b6800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:26.579 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:26.573+0000 7f9b5b7fe640 1 -- 172.21.15.169:0/2876694596 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b68007c70 con 0x7f9b740cdc30 2023-10-26T01:38:26.579 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:26.573+0000 7f9b5b7fe640 1 -- 172.21.15.169:0/2876694596 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9b6800f450 con 0x7f9b740cdc30 2023-10-26T01:38:26.579 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:26.573+0000 7f9b5b7fe640 1 -- 172.21.15.169:0/2876694596 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b6800f860 con 0x7f9b740cdc30 2023-10-26T01:38:26.579 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:26.573+0000 7f9b7974c640 1 -- 172.21.15.169:0/2876694596 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b7415c0d0 con 0x7f9b740cdc30 2023-10-26T01:38:26.579 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:26.573+0000 7f9b7974c640 1 -- 172.21.15.169:0/2876694596 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b74159a70 con 0x7f9b740cdc30 2023-10-26T01:38:26.580 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:26.574+0000 7f9b7974c640 1 -- 172.21.15.169:0/2876694596 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b7404db10 con 0x7f9b740cdc30 2023-10-26T01:38:26.581 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:26.575+0000 7f9b5b7fe640 1 -- 172.21.15.169:0/2876694596 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f9b6800f9c0 con 0x7f9b740cdc30 2023-10-26T01:38:26.581 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:26.576+0000 7f9b5b7fe640 1 --2- 172.21.15.169:0/2876694596 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9b6006ae40 0x7f9b6006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:26.581 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:26.576+0000 7f9b5b7fe640 1 -- 172.21.15.169:0/2876694596 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9b68089f00 con 0x7f9b740cdc30 2023-10-26T01:38:26.581 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:26.577+0000 7f9b72ffd640 1 --2- 172.21.15.169:0/2876694596 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9b6006ae40 0x7f9b6006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:26.581 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:26.578+0000 7f9b72ffd640 1 --2- 172.21.15.169:0/2876694596 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9b6006ae40 0x7f9b6006d300 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f9b640058c0 tx=0x7f9b64005c10 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:26.582 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:26.580+0000 7f9b5b7fe640 1 -- 172.21.15.169:0/2876694596 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9b68054fa0 con 0x7f9b740cdc30 2023-10-26T01:38:26.769 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:26.768+0000 7f9b7974c640 1 -- 172.21.15.169:0/2876694596 --> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9b740c9840 con 0x7f9b6006ae40 2023-10-26T01:38:26.792 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:26.790+0000 7f9b5b7fe640 1 -- 172.21.15.169:0/2876694596 <== mgr.4112 v2:172.21.15.170:6832/1187156918 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+283029 (secure 0 0 0) 0x7f9b740c9840 con 0x7f9b6006ae40 2023-10-26T01:38:26.797 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:38:26.798 INFO:teuthology.orchestra.run.smithi169.stderr:dumped all 2023-10-26T01:38:26.800 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:26.796+0000 7f9b7974c640 1 -- 172.21.15.169:0/2876694596 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9b6006ae40 msgr2=0x7f9b6006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:26.800 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:26.796+0000 7f9b7974c640 1 --2- 172.21.15.169:0/2876694596 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9b6006ae40 0x7f9b6006d300 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f9b640058c0 tx=0x7f9b64005c10 comp rx=0 tx=0).stop 2023-10-26T01:38:26.800 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:26.796+0000 7f9b7974c640 1 -- 172.21.15.169:0/2876694596 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9b740cdc30 msgr2=0x7f9b740c7750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:26.801 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:26.796+0000 7f9b7974c640 1 --2- 172.21.15.169:0/2876694596 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9b740cdc30 0x7f9b740c7750 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f9b6800c920 tx=0x7f9b6800cde0 comp rx=0 tx=0).stop 2023-10-26T01:38:26.801 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:26.796+0000 7f9b7974c640 1 -- 172.21.15.169:0/2876694596 shutdown_connections 2023-10-26T01:38:26.801 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:26.796+0000 7f9b7974c640 1 --2- 172.21.15.169:0/2876694596 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9b6006ae40 0x7f9b6006d300 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:26.801 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:26.796+0000 7f9b7974c640 1 --2- 172.21.15.169:0/2876694596 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9b740cdc30 0x7f9b740c7750 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:26.801 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:26.796+0000 7f9b7974c640 1 --2- 172.21.15.169:0/2876694596 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9b741592d0 0x7f9b7415b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:26.802 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:26.796+0000 7f9b7974c640 1 --2- 172.21.15.169:0/2876694596 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9b740ac930 0x7f9b741f1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:26.802 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:26.796+0000 7f9b7974c640 1 -- 172.21.15.169:0/2876694596 >> 172.21.15.169:0/2876694596 conn(0x7f9b74069930 msgr2=0x7f9b74151f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:26.802 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:26.796+0000 7f9b7974c640 1 -- 172.21.15.169:0/2876694596 shutdown_connections 2023-10-26T01:38:26.802 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:26.797+0000 7f9b7974c640 1 -- 172.21.15.169:0/2876694596 wait complete. 2023-10-26T01:38:26.817 INFO:teuthology.orchestra.run.smithi169.stdout:{"pg_ready":true,"pg_map":{"version":365,"stamp":"2023-10-26T01:38:26.295294+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":153,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":6945,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1555554,"statfs":{"total":1600000000,"available":1592887790,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002031"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.359274+0000","last_change":"2023-10-26T01:38:21.325023+0000","last_active":"2023-10-26T01:38:22.359274+0000","last_peered":"2023-10-26T01:38:22.359274+0000","last_clean":"2023-10-26T01:38:22.359274+0000","last_became_active":"2023-10-26T01:38:21.323959+0000","last_became_peered":"2023-10-26T01:38:21.323959+0000","last_unstale":"2023-10-26T01:38:22.359274+0000","last_undegraded":"2023-10-26T01:38:22.359274+0000","last_fullsized":"2023-10-26T01:38:22.359274+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:51:07.392163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.297000+0000","last_change":"2023-10-26T01:38:21.315511+0000","last_active":"2023-10-26T01:38:22.297000+0000","last_peered":"2023-10-26T01:38:22.297000+0000","last_clean":"2023-10-26T01:38:22.297000+0000","last_became_active":"2023-10-26T01:38:21.314652+0000","last_became_peered":"2023-10-26T01:38:21.314652+0000","last_unstale":"2023-10-26T01:38:22.297000+0000","last_undegraded":"2023-10-26T01:38:22.297000+0000","last_fullsized":"2023-10-26T01:38:22.297000+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:44:57.150927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.298865+0000","last_change":"2023-10-26T01:38:21.327284+0000","last_active":"2023-10-26T01:38:22.298865+0000","last_peered":"2023-10-26T01:38:22.298865+0000","last_clean":"2023-10-26T01:38:22.298865+0000","last_became_active":"2023-10-26T01:38:21.326309+0000","last_became_peered":"2023-10-26T01:38:21.326309+0000","last_unstale":"2023-10-26T01:38:22.298865+0000","last_undegraded":"2023-10-26T01:38:22.298865+0000","last_fullsized":"2023-10-26T01:38:22.298865+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:46:37.417585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.358670+0000","last_change":"2023-10-26T01:38:21.317851+0000","last_active":"2023-10-26T01:38:22.358670+0000","last_peered":"2023-10-26T01:38:22.358670+0000","last_clean":"2023-10-26T01:38:22.358670+0000","last_became_active":"2023-10-26T01:38:21.317269+0000","last_became_peered":"2023-10-26T01:38:21.317269+0000","last_unstale":"2023-10-26T01:38:22.358670+0000","last_undegraded":"2023-10-26T01:38:22.358670+0000","last_fullsized":"2023-10-26T01:38:22.358670+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T01:59:52.462839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.357989+0000","last_change":"2023-10-26T01:38:21.304138+0000","last_active":"2023-10-26T01:38:22.357989+0000","last_peered":"2023-10-26T01:38:22.357989+0000","last_clean":"2023-10-26T01:38:22.357989+0000","last_became_active":"2023-10-26T01:38:21.303352+0000","last_became_peered":"2023-10-26T01:38:21.303352+0000","last_unstale":"2023-10-26T01:38:22.357989+0000","last_undegraded":"2023-10-26T01:38:22.357989+0000","last_fullsized":"2023-10-26T01:38:22.357989+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:36:15.755953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.357771+0000","last_change":"2023-10-26T01:38:21.319100+0000","last_active":"2023-10-26T01:38:22.357771+0000","last_peered":"2023-10-26T01:38:22.357771+0000","last_clean":"2023-10-26T01:38:22.357771+0000","last_became_active":"2023-10-26T01:38:21.318169+0000","last_became_peered":"2023-10-26T01:38:21.318169+0000","last_unstale":"2023-10-26T01:38:22.357771+0000","last_undegraded":"2023-10-26T01:38:22.357771+0000","last_fullsized":"2023-10-26T01:38:22.357771+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:05:26.952713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.356781+0000","last_change":"2023-10-26T01:38:21.318003+0000","last_active":"2023-10-26T01:38:22.356781+0000","last_peered":"2023-10-26T01:38:22.356781+0000","last_clean":"2023-10-26T01:38:22.356781+0000","last_became_active":"2023-10-26T01:38:21.317493+0000","last_became_peered":"2023-10-26T01:38:21.317493+0000","last_unstale":"2023-10-26T01:38:22.356781+0000","last_undegraded":"2023-10-26T01:38:22.356781+0000","last_fullsized":"2023-10-26T01:38:22.356781+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:32:43.401727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.299581+0000","last_change":"2023-10-26T01:38:21.324036+0000","last_active":"2023-10-26T01:38:22.299581+0000","last_peered":"2023-10-26T01:38:22.299581+0000","last_clean":"2023-10-26T01:38:22.299581+0000","last_became_active":"2023-10-26T01:38:21.323785+0000","last_became_peered":"2023-10-26T01:38:21.323785+0000","last_unstale":"2023-10-26T01:38:22.299581+0000","last_undegraded":"2023-10-26T01:38:22.299581+0000","last_fullsized":"2023-10-26T01:38:22.299581+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:27:43.267716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.358805+0000","last_change":"2023-10-26T01:38:21.323358+0000","last_active":"2023-10-26T01:38:22.358805+0000","last_peered":"2023-10-26T01:38:22.358805+0000","last_clean":"2023-10-26T01:38:22.358805+0000","last_became_active":"2023-10-26T01:38:21.322567+0000","last_became_peered":"2023-10-26T01:38:21.322567+0000","last_unstale":"2023-10-26T01:38:22.358805+0000","last_undegraded":"2023-10-26T01:38:22.358805+0000","last_fullsized":"2023-10-26T01:38:22.358805+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:07:49.066636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.357274+0000","last_change":"2023-10-26T01:38:21.320421+0000","last_active":"2023-10-26T01:38:22.357274+0000","last_peered":"2023-10-26T01:38:22.357274+0000","last_clean":"2023-10-26T01:38:22.357274+0000","last_became_active":"2023-10-26T01:38:21.319296+0000","last_became_peered":"2023-10-26T01:38:21.319296+0000","last_unstale":"2023-10-26T01:38:22.357274+0000","last_undegraded":"2023-10-26T01:38:22.357274+0000","last_fullsized":"2023-10-26T01:38:22.357274+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:24:08.193245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.296689+0000","last_change":"2023-10-26T01:38:21.313660+0000","last_active":"2023-10-26T01:38:22.296689+0000","last_peered":"2023-10-26T01:38:22.296689+0000","last_clean":"2023-10-26T01:38:22.296689+0000","last_became_active":"2023-10-26T01:38:21.312980+0000","last_became_peered":"2023-10-26T01:38:21.312980+0000","last_unstale":"2023-10-26T01:38:22.296689+0000","last_undegraded":"2023-10-26T01:38:22.296689+0000","last_fullsized":"2023-10-26T01:38:22.296689+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:00:42.118665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.358755+0000","last_change":"2023-10-26T01:38:21.315876+0000","last_active":"2023-10-26T01:38:22.358755+0000","last_peered":"2023-10-26T01:38:22.358755+0000","last_clean":"2023-10-26T01:38:22.358755+0000","last_became_active":"2023-10-26T01:38:21.314851+0000","last_became_peered":"2023-10-26T01:38:21.314851+0000","last_unstale":"2023-10-26T01:38:22.358755+0000","last_undegraded":"2023-10-26T01:38:22.358755+0000","last_fullsized":"2023-10-26T01:38:22.358755+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:34:15.657860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.299218+0000","last_change":"2023-10-26T01:38:21.327458+0000","last_active":"2023-10-26T01:38:22.299218+0000","last_peered":"2023-10-26T01:38:22.299218+0000","last_clean":"2023-10-26T01:38:22.299218+0000","last_became_active":"2023-10-26T01:38:21.326720+0000","last_became_peered":"2023-10-26T01:38:21.326720+0000","last_unstale":"2023-10-26T01:38:22.299218+0000","last_undegraded":"2023-10-26T01:38:22.299218+0000","last_fullsized":"2023-10-26T01:38:22.299218+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:48:01.026075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.296528+0000","last_change":"2023-10-26T01:38:21.317035+0000","last_active":"2023-10-26T01:38:22.296528+0000","last_peered":"2023-10-26T01:38:22.296528+0000","last_clean":"2023-10-26T01:38:22.296528+0000","last_became_active":"2023-10-26T01:38:21.316156+0000","last_became_peered":"2023-10-26T01:38:21.316156+0000","last_unstale":"2023-10-26T01:38:22.296528+0000","last_undegraded":"2023-10-26T01:38:22.296528+0000","last_fullsized":"2023-10-26T01:38:22.296528+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T01:50:22.270135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.356685+0000","last_change":"2023-10-26T01:38:21.312264+0000","last_active":"2023-10-26T01:38:22.356685+0000","last_peered":"2023-10-26T01:38:22.356685+0000","last_clean":"2023-10-26T01:38:22.356685+0000","last_became_active":"2023-10-26T01:38:21.311809+0000","last_became_peered":"2023-10-26T01:38:21.311809+0000","last_unstale":"2023-10-26T01:38:22.356685+0000","last_undegraded":"2023-10-26T01:38:22.356685+0000","last_fullsized":"2023-10-26T01:38:22.356685+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:36:15.755953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.359566+0000","last_change":"2023-10-26T01:38:21.325166+0000","last_active":"2023-10-26T01:38:22.359566+0000","last_peered":"2023-10-26T01:38:22.359566+0000","last_clean":"2023-10-26T01:38:22.359566+0000","last_became_active":"2023-10-26T01:38:21.324612+0000","last_became_peered":"2023-10-26T01:38:21.324612+0000","last_unstale":"2023-10-26T01:38:22.359566+0000","last_undegraded":"2023-10-26T01:38:22.359566+0000","last_fullsized":"2023-10-26T01:38:22.359566+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:17:29.812141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.297352+0000","last_change":"2023-10-26T01:38:21.315001+0000","last_active":"2023-10-26T01:38:22.297352+0000","last_peered":"2023-10-26T01:38:22.297352+0000","last_clean":"2023-10-26T01:38:22.297352+0000","last_became_active":"2023-10-26T01:38:21.312976+0000","last_became_peered":"2023-10-26T01:38:21.312976+0000","last_unstale":"2023-10-26T01:38:22.297352+0000","last_undegraded":"2023-10-26T01:38:22.297352+0000","last_fullsized":"2023-10-26T01:38:22.297352+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:29:52.799943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.356647+0000","last_change":"2023-10-26T01:38:21.319948+0000","last_active":"2023-10-26T01:38:22.356647+0000","last_peered":"2023-10-26T01:38:22.356647+0000","last_clean":"2023-10-26T01:38:22.356647+0000","last_became_active":"2023-10-26T01:38:21.319058+0000","last_became_peered":"2023-10-26T01:38:21.319058+0000","last_unstale":"2023-10-26T01:38:22.356647+0000","last_undegraded":"2023-10-26T01:38:22.356647+0000","last_fullsized":"2023-10-26T01:38:22.356647+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:29:52.799943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.296873+0000","last_change":"2023-10-26T01:38:21.324164+0000","last_active":"2023-10-26T01:38:22.296873+0000","last_peered":"2023-10-26T01:38:22.296873+0000","last_clean":"2023-10-26T01:38:22.296873+0000","last_became_active":"2023-10-26T01:38:21.323176+0000","last_became_peered":"2023-10-26T01:38:21.323176+0000","last_unstale":"2023-10-26T01:38:22.296873+0000","last_undegraded":"2023-10-26T01:38:22.296873+0000","last_fullsized":"2023-10-26T01:38:22.296873+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:44:57.150927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.299813+0000","last_change":"2023-10-26T01:38:21.313793+0000","last_active":"2023-10-26T01:38:22.299813+0000","last_peered":"2023-10-26T01:38:22.299813+0000","last_clean":"2023-10-26T01:38:22.299813+0000","last_became_active":"2023-10-26T01:38:21.312693+0000","last_became_peered":"2023-10-26T01:38:21.312693+0000","last_unstale":"2023-10-26T01:38:22.299813+0000","last_undegraded":"2023-10-26T01:38:22.299813+0000","last_fullsized":"2023-10-26T01:38:22.299813+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:47:08.011620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.297043+0000","last_change":"2023-10-26T01:38:21.305093+0000","last_active":"2023-10-26T01:38:22.297043+0000","last_peered":"2023-10-26T01:38:22.297043+0000","last_clean":"2023-10-26T01:38:22.297043+0000","last_became_active":"2023-10-26T01:38:21.304404+0000","last_became_peered":"2023-10-26T01:38:21.304404+0000","last_unstale":"2023-10-26T01:38:22.297043+0000","last_undegraded":"2023-10-26T01:38:22.297043+0000","last_fullsized":"2023-10-26T01:38:22.297043+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:51:42.712087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.317702+0000","last_change":"2023-10-26T01:38:21.314965+0000","last_active":"2023-10-26T01:38:21.317702+0000","last_peered":"2023-10-26T01:38:21.317702+0000","last_clean":"2023-10-26T01:38:21.317702+0000","last_became_active":"2023-10-26T01:38:21.314673+0000","last_became_peered":"2023-10-26T01:38:21.314673+0000","last_unstale":"2023-10-26T01:38:21.317702+0000","last_undegraded":"2023-10-26T01:38:21.317702+0000","last_fullsized":"2023-10-26T01:38:21.317702+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:33:14.986087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.298857+0000","last_change":"2023-10-26T01:38:21.323272+0000","last_active":"2023-10-26T01:38:22.298857+0000","last_peered":"2023-10-26T01:38:22.298857+0000","last_clean":"2023-10-26T01:38:22.298857+0000","last_became_active":"2023-10-26T01:38:21.322991+0000","last_became_peered":"2023-10-26T01:38:21.322991+0000","last_unstale":"2023-10-26T01:38:22.298857+0000","last_undegraded":"2023-10-26T01:38:22.298857+0000","last_fullsized":"2023-10-26T01:38:22.298857+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:48:01.026075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.320186+0000","last_change":"2023-10-26T01:38:21.320068+0000","last_active":"2023-10-26T01:38:21.320186+0000","last_peered":"2023-10-26T01:38:21.320186+0000","last_clean":"2023-10-26T01:38:21.320186+0000","last_became_active":"2023-10-26T01:38:21.319773+0000","last_became_peered":"2023-10-26T01:38:21.319773+0000","last_unstale":"2023-10-26T01:38:21.320186+0000","last_undegraded":"2023-10-26T01:38:21.320186+0000","last_fullsized":"2023-10-26T01:38:21.320186+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:03:39.495499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.300137+0000","last_change":"2023-10-26T01:38:21.314000+0000","last_active":"2023-10-26T01:38:22.300137+0000","last_peered":"2023-10-26T01:38:22.300137+0000","last_clean":"2023-10-26T01:38:22.300137+0000","last_became_active":"2023-10-26T01:38:21.313194+0000","last_became_peered":"2023-10-26T01:38:21.313194+0000","last_unstale":"2023-10-26T01:38:22.300137+0000","last_undegraded":"2023-10-26T01:38:22.300137+0000","last_fullsized":"2023-10-26T01:38:22.300137+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T01:40:34.431360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.359873+0000","last_change":"2023-10-26T01:38:21.324296+0000","last_active":"2023-10-26T01:38:22.359873+0000","last_peered":"2023-10-26T01:38:22.359873+0000","last_clean":"2023-10-26T01:38:22.359873+0000","last_became_active":"2023-10-26T01:38:21.320669+0000","last_became_peered":"2023-10-26T01:38:21.320669+0000","last_unstale":"2023-10-26T01:38:22.359873+0000","last_undegraded":"2023-10-26T01:38:22.359873+0000","last_fullsized":"2023-10-26T01:38:22.359873+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:16:57.330926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.357018+0000","last_change":"2023-10-26T01:38:21.320441+0000","last_active":"2023-10-26T01:38:22.357018+0000","last_peered":"2023-10-26T01:38:22.357018+0000","last_clean":"2023-10-26T01:38:22.357018+0000","last_became_active":"2023-10-26T01:38:21.320185+0000","last_became_peered":"2023-10-26T01:38:21.320185+0000","last_unstale":"2023-10-26T01:38:22.357018+0000","last_undegraded":"2023-10-26T01:38:22.357018+0000","last_fullsized":"2023-10-26T01:38:22.357018+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:11:34.893562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.320158+0000","last_change":"2023-10-26T01:38:21.319408+0000","last_active":"2023-10-26T01:38:21.320158+0000","last_peered":"2023-10-26T01:38:21.320158+0000","last_clean":"2023-10-26T01:38:21.320158+0000","last_became_active":"2023-10-26T01:38:21.318421+0000","last_became_peered":"2023-10-26T01:38:21.318421+0000","last_unstale":"2023-10-26T01:38:21.320158+0000","last_undegraded":"2023-10-26T01:38:21.320158+0000","last_fullsized":"2023-10-26T01:38:21.320158+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:23:42.697097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.296824+0000","last_change":"2023-10-26T01:38:21.322363+0000","last_active":"2023-10-26T01:38:22.296824+0000","last_peered":"2023-10-26T01:38:22.296824+0000","last_clean":"2023-10-26T01:38:22.296824+0000","last_became_active":"2023-10-26T01:38:21.322104+0000","last_became_peered":"2023-10-26T01:38:21.322104+0000","last_unstale":"2023-10-26T01:38:22.296824+0000","last_undegraded":"2023-10-26T01:38:22.296824+0000","last_fullsized":"2023-10-26T01:38:22.296824+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T01:50:22.270135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.356583+0000","last_change":"2023-10-26T01:38:21.311895+0000","last_active":"2023-10-26T01:38:22.356583+0000","last_peered":"2023-10-26T01:38:22.356583+0000","last_clean":"2023-10-26T01:38:22.356583+0000","last_became_active":"2023-10-26T01:38:21.311281+0000","last_became_peered":"2023-10-26T01:38:21.311281+0000","last_unstale":"2023-10-26T01:38:22.356583+0000","last_undegraded":"2023-10-26T01:38:22.356583+0000","last_fullsized":"2023-10-26T01:38:22.356583+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:24:08.965712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.360039+0000","last_change":"2023-10-26T01:38:21.324857+0000","last_active":"2023-10-26T01:38:22.360039+0000","last_peered":"2023-10-26T01:38:22.360039+0000","last_clean":"2023-10-26T01:38:22.360039+0000","last_became_active":"2023-10-26T01:38:21.323546+0000","last_became_peered":"2023-10-26T01:38:21.323546+0000","last_unstale":"2023-10-26T01:38:22.360039+0000","last_undegraded":"2023-10-26T01:38:22.360039+0000","last_fullsized":"2023-10-26T01:38:22.360039+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:53:21.542954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.357316+0000","last_change":"2023-10-26T01:38:21.315041+0000","last_active":"2023-10-26T01:38:22.357316+0000","last_peered":"2023-10-26T01:38:22.357316+0000","last_clean":"2023-10-26T01:38:22.357316+0000","last_became_active":"2023-10-26T01:38:21.314746+0000","last_became_peered":"2023-10-26T01:38:21.314746+0000","last_unstale":"2023-10-26T01:38:22.357316+0000","last_undegraded":"2023-10-26T01:38:22.357316+0000","last_fullsized":"2023-10-26T01:38:22.357316+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:27:03.489686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.355991+0000","last_change":"2023-10-26T01:38:21.304208+0000","last_active":"2023-10-26T01:38:22.355991+0000","last_peered":"2023-10-26T01:38:22.355991+0000","last_clean":"2023-10-26T01:38:22.355991+0000","last_became_active":"2023-10-26T01:38:21.303492+0000","last_became_peered":"2023-10-26T01:38:21.303492+0000","last_unstale":"2023-10-26T01:38:22.355991+0000","last_undegraded":"2023-10-26T01:38:22.355991+0000","last_fullsized":"2023-10-26T01:38:22.355991+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:57:23.914586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"133'4","reported_seq":16,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.334628+0000","last_change":"2023-10-26T01:38:20.322005+0000","last_active":"2023-10-26T01:38:21.334628+0000","last_peered":"2023-10-26T01:38:21.334628+0000","last_clean":"2023-10-26T01:38:21.334628+0000","last_became_active":"2023-10-26T01:38:20.320582+0000","last_became_peered":"2023-10-26T01:38:20.320582+0000","last_unstale":"2023-10-26T01:38:21.334628+0000","last_undegraded":"2023-10-26T01:38:21.334628+0000","last_fullsized":"2023-10-26T01:38:21.334628+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:00:02.848161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.357007+0000","last_change":"2023-10-26T01:38:21.323121+0000","last_active":"2023-10-26T01:38:22.357007+0000","last_peered":"2023-10-26T01:38:22.357007+0000","last_clean":"2023-10-26T01:38:22.357007+0000","last_became_active":"2023-10-26T01:38:21.322808+0000","last_became_peered":"2023-10-26T01:38:21.322808+0000","last_unstale":"2023-10-26T01:38:22.357007+0000","last_undegraded":"2023-10-26T01:38:22.357007+0000","last_fullsized":"2023-10-26T01:38:22.357007+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:46:37.417585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"133'1","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.334372+0000","last_change":"2023-10-26T01:38:20.326148+0000","last_active":"2023-10-26T01:38:21.334372+0000","last_peered":"2023-10-26T01:38:21.334372+0000","last_clean":"2023-10-26T01:38:21.334372+0000","last_became_active":"2023-10-26T01:38:20.325883+0000","last_became_peered":"2023-10-26T01:38:20.325883+0000","last_unstale":"2023-10-26T01:38:21.334372+0000","last_undegraded":"2023-10-26T01:38:21.334372+0000","last_fullsized":"2023-10-26T01:38:21.334372+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:36:47.183420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.358082+0000","last_change":"2023-10-26T01:38:21.321027+0000","last_active":"2023-10-26T01:38:22.358082+0000","last_peered":"2023-10-26T01:38:22.358082+0000","last_clean":"2023-10-26T01:38:22.358082+0000","last_became_active":"2023-10-26T01:38:21.318702+0000","last_became_peered":"2023-10-26T01:38:21.318702+0000","last_unstale":"2023-10-26T01:38:22.358082+0000","last_undegraded":"2023-10-26T01:38:22.358082+0000","last_fullsized":"2023-10-26T01:38:22.358082+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:36:20.377483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"133'2","reported_seq":14,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.338315+0000","last_change":"2023-10-26T01:38:20.321817+0000","last_active":"2023-10-26T01:38:21.338315+0000","last_peered":"2023-10-26T01:38:21.338315+0000","last_clean":"2023-10-26T01:38:21.338315+0000","last_became_active":"2023-10-26T01:38:20.321520+0000","last_became_peered":"2023-10-26T01:38:20.321520+0000","last_unstale":"2023-10-26T01:38:21.338315+0000","last_undegraded":"2023-10-26T01:38:21.338315+0000","last_fullsized":"2023-10-26T01:38:21.338315+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:57:22.900600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.299225+0000","last_change":"2023-10-26T01:38:21.321216+0000","last_active":"2023-10-26T01:38:22.299225+0000","last_peered":"2023-10-26T01:38:22.299225+0000","last_clean":"2023-10-26T01:38:22.299225+0000","last_became_active":"2023-10-26T01:38:21.320953+0000","last_became_peered":"2023-10-26T01:38:21.320953+0000","last_unstale":"2023-10-26T01:38:22.299225+0000","last_undegraded":"2023-10-26T01:38:22.299225+0000","last_fullsized":"2023-10-26T01:38:22.299225+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:47:08.011620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"133'2","reported_seq":14,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.337393+0000","last_change":"2023-10-26T01:38:20.325154+0000","last_active":"2023-10-26T01:38:21.337393+0000","last_peered":"2023-10-26T01:38:21.337393+0000","last_clean":"2023-10-26T01:38:21.337393+0000","last_became_active":"2023-10-26T01:38:20.324072+0000","last_became_peered":"2023-10-26T01:38:20.324072+0000","last_unstale":"2023-10-26T01:38:21.337393+0000","last_undegraded":"2023-10-26T01:38:21.337393+0000","last_fullsized":"2023-10-26T01:38:21.337393+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:31:09.167929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.357578+0000","last_change":"2023-10-26T01:38:21.323345+0000","last_active":"2023-10-26T01:38:22.357578+0000","last_peered":"2023-10-26T01:38:22.357578+0000","last_clean":"2023-10-26T01:38:22.357578+0000","last_became_active":"2023-10-26T01:38:21.323051+0000","last_became_peered":"2023-10-26T01:38:21.323051+0000","last_unstale":"2023-10-26T01:38:22.357578+0000","last_undegraded":"2023-10-26T01:38:22.357578+0000","last_fullsized":"2023-10-26T01:38:22.357578+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:46:30.031341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"133'1","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.333965+0000","last_change":"2023-10-26T01:38:20.323989+0000","last_active":"2023-10-26T01:38:21.333965+0000","last_peered":"2023-10-26T01:38:21.333965+0000","last_clean":"2023-10-26T01:38:21.333965+0000","last_became_active":"2023-10-26T01:38:20.323646+0000","last_became_peered":"2023-10-26T01:38:20.323646+0000","last_unstale":"2023-10-26T01:38:21.333965+0000","last_undegraded":"2023-10-26T01:38:21.333965+0000","last_fullsized":"2023-10-26T01:38:21.333965+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:57:22.900600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.297706+0000","last_change":"2023-10-26T01:38:21.315216+0000","last_active":"2023-10-26T01:38:22.297706+0000","last_peered":"2023-10-26T01:38:22.297706+0000","last_clean":"2023-10-26T01:38:22.297706+0000","last_became_active":"2023-10-26T01:38:21.314095+0000","last_became_peered":"2023-10-26T01:38:21.314095+0000","last_unstale":"2023-10-26T01:38:22.297706+0000","last_undegraded":"2023-10-26T01:38:22.297706+0000","last_fullsized":"2023-10-26T01:38:22.297706+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:46:37.417585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"133'1","reported_seq":12,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.335571+0000","last_change":"2023-10-26T01:38:20.320626+0000","last_active":"2023-10-26T01:38:21.335571+0000","last_peered":"2023-10-26T01:38:21.335571+0000","last_clean":"2023-10-26T01:38:21.335571+0000","last_became_active":"2023-10-26T01:38:20.320400+0000","last_became_peered":"2023-10-26T01:38:20.320400+0000","last_unstale":"2023-10-26T01:38:21.335571+0000","last_undegraded":"2023-10-26T01:38:21.335571+0000","last_fullsized":"2023-10-26T01:38:21.335571+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:31:15.331694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.357408+0000","last_change":"2023-10-26T01:38:21.320573+0000","last_active":"2023-10-26T01:38:22.357408+0000","last_peered":"2023-10-26T01:38:22.357408+0000","last_clean":"2023-10-26T01:38:22.357408+0000","last_became_active":"2023-10-26T01:38:21.319770+0000","last_became_peered":"2023-10-26T01:38:21.319770+0000","last_unstale":"2023-10-26T01:38:22.357408+0000","last_undegraded":"2023-10-26T01:38:22.357408+0000","last_fullsized":"2023-10-26T01:38:22.357408+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:30:41.148919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":12,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.294385+0000","last_change":"2023-10-26T01:38:20.317888+0000","last_active":"2023-10-26T01:38:21.294385+0000","last_peered":"2023-10-26T01:38:21.294385+0000","last_clean":"2023-10-26T01:38:21.294385+0000","last_became_active":"2023-10-26T01:38:20.317130+0000","last_became_peered":"2023-10-26T01:38:20.317130+0000","last_unstale":"2023-10-26T01:38:21.294385+0000","last_undegraded":"2023-10-26T01:38:21.294385+0000","last_fullsized":"2023-10-26T01:38:21.294385+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:27:41.686982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.356245+0000","last_change":"2023-10-26T01:38:21.317074+0000","last_active":"2023-10-26T01:38:22.356245+0000","last_peered":"2023-10-26T01:38:22.356245+0000","last_clean":"2023-10-26T01:38:22.356245+0000","last_became_active":"2023-10-26T01:38:21.316665+0000","last_became_peered":"2023-10-26T01:38:21.316665+0000","last_unstale":"2023-10-26T01:38:22.356245+0000","last_undegraded":"2023-10-26T01:38:22.356245+0000","last_fullsized":"2023-10-26T01:38:22.356245+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:42:14.392665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":12,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.307295+0000","last_change":"2023-10-26T01:38:20.314210+0000","last_active":"2023-10-26T01:38:21.307295+0000","last_peered":"2023-10-26T01:38:21.307295+0000","last_clean":"2023-10-26T01:38:21.307295+0000","last_became_active":"2023-10-26T01:38:20.313976+0000","last_became_peered":"2023-10-26T01:38:20.313976+0000","last_unstale":"2023-10-26T01:38:21.307295+0000","last_undegraded":"2023-10-26T01:38:21.307295+0000","last_fullsized":"2023-10-26T01:38:21.307295+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:24:55.184951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.356077+0000","last_change":"2023-10-26T01:38:21.313827+0000","last_active":"2023-10-26T01:38:22.356077+0000","last_peered":"2023-10-26T01:38:22.356077+0000","last_clean":"2023-10-26T01:38:22.356077+0000","last_became_active":"2023-10-26T01:38:21.310630+0000","last_became_peered":"2023-10-26T01:38:21.310630+0000","last_unstale":"2023-10-26T01:38:22.356077+0000","last_undegraded":"2023-10-26T01:38:22.356077+0000","last_fullsized":"2023-10-26T01:38:22.356077+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:28:43.891836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":12,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.295375+0000","last_change":"2023-10-26T01:38:20.325020+0000","last_active":"2023-10-26T01:38:21.295375+0000","last_peered":"2023-10-26T01:38:21.295375+0000","last_clean":"2023-10-26T01:38:21.295375+0000","last_became_active":"2023-10-26T01:38:20.323713+0000","last_became_peered":"2023-10-26T01:38:20.323713+0000","last_unstale":"2023-10-26T01:38:21.295375+0000","last_undegraded":"2023-10-26T01:38:21.295375+0000","last_fullsized":"2023-10-26T01:38:21.295375+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:23:36.805508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.357896+0000","last_change":"2023-10-26T01:38:21.327094+0000","last_active":"2023-10-26T01:38:22.357896+0000","last_peered":"2023-10-26T01:38:22.357896+0000","last_clean":"2023-10-26T01:38:22.357896+0000","last_became_active":"2023-10-26T01:38:21.326783+0000","last_became_peered":"2023-10-26T01:38:21.326783+0000","last_unstale":"2023-10-26T01:38:22.357896+0000","last_undegraded":"2023-10-26T01:38:22.357896+0000","last_fullsized":"2023-10-26T01:38:22.357896+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:48:01.026075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.293962+0000","last_change":"2023-10-26T01:38:20.322090+0000","last_active":"2023-10-26T01:38:21.293962+0000","last_peered":"2023-10-26T01:38:21.293962+0000","last_clean":"2023-10-26T01:38:21.293962+0000","last_became_active":"2023-10-26T01:38:20.321451+0000","last_became_peered":"2023-10-26T01:38:20.321451+0000","last_unstale":"2023-10-26T01:38:21.293962+0000","last_undegraded":"2023-10-26T01:38:21.293962+0000","last_fullsized":"2023-10-26T01:38:21.293962+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:45:01.083199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.302056+0000","last_change":"2023-10-26T01:38:21.301523+0000","last_active":"2023-10-26T01:38:21.302056+0000","last_peered":"2023-10-26T01:38:21.302056+0000","last_clean":"2023-10-26T01:38:21.302056+0000","last_became_active":"2023-10-26T01:38:21.300901+0000","last_became_peered":"2023-10-26T01:38:21.300901+0000","last_unstale":"2023-10-26T01:38:21.302056+0000","last_undegraded":"2023-10-26T01:38:21.302056+0000","last_fullsized":"2023-10-26T01:38:21.302056+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:21:55.273677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"133'3","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.337431+0000","last_change":"2023-10-26T01:38:20.319263+0000","last_active":"2023-10-26T01:38:21.337431+0000","last_peered":"2023-10-26T01:38:21.337431+0000","last_clean":"2023-10-26T01:38:21.337431+0000","last_became_active":"2023-10-26T01:38:20.319022+0000","last_became_peered":"2023-10-26T01:38:20.319022+0000","last_unstale":"2023-10-26T01:38:21.337431+0000","last_undegraded":"2023-10-26T01:38:21.337431+0000","last_fullsized":"2023-10-26T01:38:21.337431+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:57:33.830348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.356119+0000","last_change":"2023-10-26T01:38:21.321635+0000","last_active":"2023-10-26T01:38:22.356119+0000","last_peered":"2023-10-26T01:38:22.356119+0000","last_clean":"2023-10-26T01:38:22.356119+0000","last_became_active":"2023-10-26T01:38:21.321297+0000","last_became_peered":"2023-10-26T01:38:21.321297+0000","last_unstale":"2023-10-26T01:38:22.356119+0000","last_undegraded":"2023-10-26T01:38:22.356119+0000","last_fullsized":"2023-10-26T01:38:22.356119+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:36:20.377483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"133'6","reported_seq":18,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.340461+0000","last_change":"2023-10-26T01:38:20.314532+0000","last_active":"2023-10-26T01:38:21.340461+0000","last_peered":"2023-10-26T01:38:21.340461+0000","last_clean":"2023-10-26T01:38:21.340461+0000","last_became_active":"2023-10-26T01:38:20.314073+0000","last_became_peered":"2023-10-26T01:38:20.314073+0000","last_unstale":"2023-10-26T01:38:21.340461+0000","last_undegraded":"2023-10-26T01:38:21.340461+0000","last_fullsized":"2023-10-26T01:38:21.340461+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:47:06.997633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.310604+0000","last_change":"2023-10-26T01:38:21.310272+0000","last_active":"2023-10-26T01:38:21.310604+0000","last_peered":"2023-10-26T01:38:21.310604+0000","last_clean":"2023-10-26T01:38:21.310604+0000","last_became_active":"2023-10-26T01:38:21.309297+0000","last_became_peered":"2023-10-26T01:38:21.309297+0000","last_unstale":"2023-10-26T01:38:21.310604+0000","last_undegraded":"2023-10-26T01:38:21.310604+0000","last_fullsized":"2023-10-26T01:38:21.310604+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:07:03.778722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"133'2","reported_seq":14,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.336757+0000","last_change":"2023-10-26T01:38:20.323897+0000","last_active":"2023-10-26T01:38:21.336757+0000","last_peered":"2023-10-26T01:38:21.336757+0000","last_clean":"2023-10-26T01:38:21.336757+0000","last_became_active":"2023-10-26T01:38:20.323527+0000","last_became_peered":"2023-10-26T01:38:20.323527+0000","last_unstale":"2023-10-26T01:38:21.336757+0000","last_undegraded":"2023-10-26T01:38:21.336757+0000","last_fullsized":"2023-10-26T01:38:21.336757+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:54:02.310203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.317765+0000","last_change":"2023-10-26T01:38:21.317158+0000","last_active":"2023-10-26T01:38:21.317765+0000","last_peered":"2023-10-26T01:38:21.317765+0000","last_clean":"2023-10-26T01:38:21.317765+0000","last_became_active":"2023-10-26T01:38:21.316782+0000","last_became_peered":"2023-10-26T01:38:21.316782+0000","last_unstale":"2023-10-26T01:38:21.317765+0000","last_undegraded":"2023-10-26T01:38:21.317765+0000","last_fullsized":"2023-10-26T01:38:21.317765+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:38:50.874604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":12,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.305214+0000","last_change":"2023-10-26T01:38:20.323991+0000","last_active":"2023-10-26T01:38:21.305214+0000","last_peered":"2023-10-26T01:38:21.305214+0000","last_clean":"2023-10-26T01:38:21.305214+0000","last_became_active":"2023-10-26T01:38:20.323242+0000","last_became_peered":"2023-10-26T01:38:20.323242+0000","last_unstale":"2023-10-26T01:38:21.305214+0000","last_undegraded":"2023-10-26T01:38:21.305214+0000","last_fullsized":"2023-10-26T01:38:21.305214+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:08:50.823275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.309119+0000","last_change":"2023-10-26T01:38:21.308916+0000","last_active":"2023-10-26T01:38:21.309119+0000","last_peered":"2023-10-26T01:38:21.309119+0000","last_clean":"2023-10-26T01:38:21.309119+0000","last_became_active":"2023-10-26T01:38:21.308364+0000","last_became_peered":"2023-10-26T01:38:21.308364+0000","last_unstale":"2023-10-26T01:38:21.309119+0000","last_undegraded":"2023-10-26T01:38:21.309119+0000","last_fullsized":"2023-10-26T01:38:21.309119+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:18:02.522210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"133'8","reported_seq":20,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.340109+0000","last_change":"2023-10-26T01:38:20.322830+0000","last_active":"2023-10-26T01:38:21.340109+0000","last_peered":"2023-10-26T01:38:21.340109+0000","last_clean":"2023-10-26T01:38:21.340109+0000","last_became_active":"2023-10-26T01:38:20.318642+0000","last_became_peered":"2023-10-26T01:38:20.318642+0000","last_unstale":"2023-10-26T01:38:21.340109+0000","last_undegraded":"2023-10-26T01:38:21.340109+0000","last_fullsized":"2023-10-26T01:38:21.340109+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:36:33.502627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.358107+0000","last_change":"2023-10-26T01:38:21.322681+0000","last_active":"2023-10-26T01:38:22.358107+0000","last_peered":"2023-10-26T01:38:22.358107+0000","last_clean":"2023-10-26T01:38:22.358107+0000","last_became_active":"2023-10-26T01:38:21.322406+0000","last_became_peered":"2023-10-26T01:38:21.322406+0000","last_unstale":"2023-10-26T01:38:22.358107+0000","last_undegraded":"2023-10-26T01:38:22.358107+0000","last_fullsized":"2023-10-26T01:38:22.358107+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:11:34.893562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"133'1","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.335822+0000","last_change":"2023-10-26T01:38:20.322530+0000","last_active":"2023-10-26T01:38:21.335822+0000","last_peered":"2023-10-26T01:38:21.335822+0000","last_clean":"2023-10-26T01:38:21.335822+0000","last_became_active":"2023-10-26T01:38:20.322312+0000","last_became_peered":"2023-10-26T01:38:20.322312+0000","last_unstale":"2023-10-26T01:38:21.335822+0000","last_undegraded":"2023-10-26T01:38:21.335822+0000","last_fullsized":"2023-10-26T01:38:21.335822+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:53:09.791518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.356325+0000","last_change":"2023-10-26T01:38:21.319104+0000","last_active":"2023-10-26T01:38:22.356325+0000","last_peered":"2023-10-26T01:38:22.356325+0000","last_clean":"2023-10-26T01:38:22.356325+0000","last_became_active":"2023-10-26T01:38:21.318760+0000","last_became_peered":"2023-10-26T01:38:21.318760+0000","last_unstale":"2023-10-26T01:38:22.356325+0000","last_undegraded":"2023-10-26T01:38:22.356325+0000","last_fullsized":"2023-10-26T01:38:22.356325+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:16:32.278743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"133'2","reported_seq":14,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.337172+0000","last_change":"2023-10-26T01:38:20.318723+0000","last_active":"2023-10-26T01:38:21.337172+0000","last_peered":"2023-10-26T01:38:21.337172+0000","last_clean":"2023-10-26T01:38:21.337172+0000","last_became_active":"2023-10-26T01:38:20.317546+0000","last_became_peered":"2023-10-26T01:38:20.317546+0000","last_unstale":"2023-10-26T01:38:21.337172+0000","last_undegraded":"2023-10-26T01:38:21.337172+0000","last_fullsized":"2023-10-26T01:38:21.337172+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:16:31.264756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.299270+0000","last_change":"2023-10-26T01:38:21.320514+0000","last_active":"2023-10-26T01:38:22.299270+0000","last_peered":"2023-10-26T01:38:22.299270+0000","last_clean":"2023-10-26T01:38:22.299270+0000","last_became_active":"2023-10-26T01:38:21.320080+0000","last_became_peered":"2023-10-26T01:38:21.320080+0000","last_unstale":"2023-10-26T01:38:22.299270+0000","last_undegraded":"2023-10-26T01:38:22.299270+0000","last_fullsized":"2023-10-26T01:38:22.299270+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T01:40:34.431360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"133'1","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.335116+0000","last_change":"2023-10-26T01:38:20.323687+0000","last_active":"2023-10-26T01:38:21.335116+0000","last_peered":"2023-10-26T01:38:21.335116+0000","last_clean":"2023-10-26T01:38:21.335116+0000","last_became_active":"2023-10-26T01:38:20.322524+0000","last_became_peered":"2023-10-26T01:38:20.322524+0000","last_unstale":"2023-10-26T01:38:21.335116+0000","last_undegraded":"2023-10-26T01:38:21.335116+0000","last_fullsized":"2023-10-26T01:38:21.335116+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:00:59.371192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.298852+0000","last_change":"2023-10-26T01:38:21.317249+0000","last_active":"2023-10-26T01:38:22.298852+0000","last_peered":"2023-10-26T01:38:22.298852+0000","last_clean":"2023-10-26T01:38:22.298852+0000","last_became_active":"2023-10-26T01:38:21.315753+0000","last_became_peered":"2023-10-26T01:38:21.315753+0000","last_unstale":"2023-10-26T01:38:22.298852+0000","last_undegraded":"2023-10-26T01:38:22.298852+0000","last_fullsized":"2023-10-26T01:38:22.298852+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:19:17.527922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":12,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.304169+0000","last_change":"2023-10-26T01:38:20.322266+0000","last_active":"2023-10-26T01:38:21.304169+0000","last_peered":"2023-10-26T01:38:21.304169+0000","last_clean":"2023-10-26T01:38:21.304169+0000","last_became_active":"2023-10-26T01:38:20.315231+0000","last_became_peered":"2023-10-26T01:38:20.315231+0000","last_unstale":"2023-10-26T01:38:21.304169+0000","last_undegraded":"2023-10-26T01:38:21.304169+0000","last_fullsized":"2023-10-26T01:38:21.304169+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:27:02.475699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.297531+0000","last_change":"2023-10-26T01:38:21.323986+0000","last_active":"2023-10-26T01:38:22.297531+0000","last_peered":"2023-10-26T01:38:22.297531+0000","last_clean":"2023-10-26T01:38:22.297531+0000","last_became_active":"2023-10-26T01:38:21.322360+0000","last_became_peered":"2023-10-26T01:38:21.322360+0000","last_unstale":"2023-10-26T01:38:22.297531+0000","last_undegraded":"2023-10-26T01:38:22.297531+0000","last_fullsized":"2023-10-26T01:38:22.297531+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:29:52.799943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"133'2","reported_seq":14,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.338836+0000","last_change":"2023-10-26T01:38:20.319975+0000","last_active":"2023-10-26T01:38:21.338836+0000","last_peered":"2023-10-26T01:38:21.338836+0000","last_clean":"2023-10-26T01:38:21.338836+0000","last_became_active":"2023-10-26T01:38:20.319754+0000","last_became_peered":"2023-10-26T01:38:20.319754+0000","last_unstale":"2023-10-26T01:38:21.338836+0000","last_undegraded":"2023-10-26T01:38:21.338836+0000","last_fullsized":"2023-10-26T01:38:21.338836+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:31:09.167929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.358204+0000","last_change":"2023-10-26T01:38:21.324252+0000","last_active":"2023-10-26T01:38:22.358204+0000","last_peered":"2023-10-26T01:38:22.358204+0000","last_clean":"2023-10-26T01:38:22.358204+0000","last_became_active":"2023-10-26T01:38:21.323975+0000","last_became_peered":"2023-10-26T01:38:21.323975+0000","last_unstale":"2023-10-26T01:38:22.358204+0000","last_undegraded":"2023-10-26T01:38:22.358204+0000","last_fullsized":"2023-10-26T01:38:22.358204+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:26:51.819282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":12,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.293744+0000","last_change":"2023-10-26T01:38:20.323205+0000","last_active":"2023-10-26T01:38:21.293744+0000","last_peered":"2023-10-26T01:38:21.293744+0000","last_clean":"2023-10-26T01:38:21.293744+0000","last_became_active":"2023-10-26T01:38:20.322992+0000","last_became_peered":"2023-10-26T01:38:20.322992+0000","last_unstale":"2023-10-26T01:38:21.293744+0000","last_undegraded":"2023-10-26T01:38:21.293744+0000","last_fullsized":"2023-10-26T01:38:21.293744+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:54:02.310203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.299167+0000","last_change":"2023-10-26T01:38:21.316303+0000","last_active":"2023-10-26T01:38:22.299167+0000","last_peered":"2023-10-26T01:38:22.299167+0000","last_clean":"2023-10-26T01:38:22.299167+0000","last_became_active":"2023-10-26T01:38:21.315023+0000","last_became_peered":"2023-10-26T01:38:21.315023+0000","last_unstale":"2023-10-26T01:38:22.299167+0000","last_undegraded":"2023-10-26T01:38:22.299167+0000","last_fullsized":"2023-10-26T01:38:22.299167+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T01:55:12.014852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":12,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.303952+0000","last_change":"2023-10-26T01:38:20.320536+0000","last_active":"2023-10-26T01:38:21.303952+0000","last_peered":"2023-10-26T01:38:21.303952+0000","last_clean":"2023-10-26T01:38:21.303952+0000","last_became_active":"2023-10-26T01:38:20.320277+0000","last_became_peered":"2023-10-26T01:38:20.320277+0000","last_unstale":"2023-10-26T01:38:21.303952+0000","last_undegraded":"2023-10-26T01:38:21.303952+0000","last_fullsized":"2023-10-26T01:38:21.303952+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:36:33.502627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.322832+0000","last_change":"2023-10-26T01:38:21.322582+0000","last_active":"2023-10-26T01:38:21.322832+0000","last_peered":"2023-10-26T01:38:21.322832+0000","last_clean":"2023-10-26T01:38:21.322832+0000","last_became_active":"2023-10-26T01:38:21.321971+0000","last_became_peered":"2023-10-26T01:38:21.321971+0000","last_unstale":"2023-10-26T01:38:21.322832+0000","last_undegraded":"2023-10-26T01:38:21.322832+0000","last_fullsized":"2023-10-26T01:38:21.322832+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:44:44.680316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":12,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.307293+0000","last_change":"2023-10-26T01:38:20.324450+0000","last_active":"2023-10-26T01:38:21.307293+0000","last_peered":"2023-10-26T01:38:21.307293+0000","last_clean":"2023-10-26T01:38:21.307293+0000","last_became_active":"2023-10-26T01:38:20.324194+0000","last_became_peered":"2023-10-26T01:38:20.324194+0000","last_unstale":"2023-10-26T01:38:21.307293+0000","last_undegraded":"2023-10-26T01:38:21.307293+0000","last_fullsized":"2023-10-26T01:38:21.307293+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:30:19.872449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.357340+0000","last_change":"2023-10-26T01:38:21.320203+0000","last_active":"2023-10-26T01:38:22.357340+0000","last_peered":"2023-10-26T01:38:22.357340+0000","last_clean":"2023-10-26T01:38:22.357340+0000","last_became_active":"2023-10-26T01:38:21.319553+0000","last_became_peered":"2023-10-26T01:38:21.319553+0000","last_unstale":"2023-10-26T01:38:22.357340+0000","last_undegraded":"2023-10-26T01:38:22.357340+0000","last_fullsized":"2023-10-26T01:38:22.357340+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T01:55:12.014852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"133'2","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.338835+0000","last_change":"2023-10-26T01:38:20.315954+0000","last_active":"2023-10-26T01:38:21.338835+0000","last_peered":"2023-10-26T01:38:21.338835+0000","last_clean":"2023-10-26T01:38:21.338835+0000","last_became_active":"2023-10-26T01:38:20.315727+0000","last_became_peered":"2023-10-26T01:38:20.315727+0000","last_unstale":"2023-10-26T01:38:21.338835+0000","last_undegraded":"2023-10-26T01:38:21.338835+0000","last_fullsized":"2023-10-26T01:38:21.338835+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:01:47.759723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.358948+0000","last_change":"2023-10-26T01:38:21.319574+0000","last_active":"2023-10-26T01:38:22.358948+0000","last_peered":"2023-10-26T01:38:22.358948+0000","last_clean":"2023-10-26T01:38:22.358948+0000","last_became_active":"2023-10-26T01:38:21.317481+0000","last_became_peered":"2023-10-26T01:38:21.317481+0000","last_unstale":"2023-10-26T01:38:22.358948+0000","last_undegraded":"2023-10-26T01:38:22.358948+0000","last_fullsized":"2023-10-26T01:38:22.358948+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:08:09.599874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"134'1","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.357908+0000","last_change":"2023-10-26T01:38:20.325614+0000","last_active":"2023-10-26T01:38:22.357908+0000","last_peered":"2023-10-26T01:38:22.357908+0000","last_clean":"2023-10-26T01:38:22.357908+0000","last_became_active":"2023-10-26T01:38:20.325288+0000","last_became_peered":"2023-10-26T01:38:20.325288+0000","last_unstale":"2023-10-26T01:38:22.357908+0000","last_undegraded":"2023-10-26T01:38:22.357908+0000","last_fullsized":"2023-10-26T01:38:22.357908+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:51:41.698100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.357610+0000","last_change":"2023-10-26T01:38:21.320766+0000","last_active":"2023-10-26T01:38:22.357610+0000","last_peered":"2023-10-26T01:38:22.357610+0000","last_clean":"2023-10-26T01:38:22.357610+0000","last_became_active":"2023-10-26T01:38:21.320148+0000","last_became_peered":"2023-10-26T01:38:21.320148+0000","last_unstale":"2023-10-26T01:38:22.357610+0000","last_undegraded":"2023-10-26T01:38:22.357610+0000","last_fullsized":"2023-10-26T01:38:22.357610+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:20:14.948944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":12,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.312905+0000","last_change":"2023-10-26T01:38:20.320712+0000","last_active":"2023-10-26T01:38:21.312905+0000","last_peered":"2023-10-26T01:38:21.312905+0000","last_clean":"2023-10-26T01:38:21.312905+0000","last_became_active":"2023-10-26T01:38:20.319884+0000","last_became_peered":"2023-10-26T01:38:20.319884+0000","last_unstale":"2023-10-26T01:38:21.312905+0000","last_undegraded":"2023-10-26T01:38:21.312905+0000","last_fullsized":"2023-10-26T01:38:21.312905+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:37:52.841387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.356773+0000","last_change":"2023-10-26T01:38:21.320274+0000","last_active":"2023-10-26T01:38:22.356773+0000","last_peered":"2023-10-26T01:38:22.356773+0000","last_clean":"2023-10-26T01:38:22.356773+0000","last_became_active":"2023-10-26T01:38:21.319994+0000","last_became_peered":"2023-10-26T01:38:21.319994+0000","last_unstale":"2023-10-26T01:38:22.356773+0000","last_undegraded":"2023-10-26T01:38:22.356773+0000","last_fullsized":"2023-10-26T01:38:22.356773+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:19:17.527922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.294223+0000","last_change":"2023-10-26T01:38:20.319596+0000","last_active":"2023-10-26T01:38:21.294223+0000","last_peered":"2023-10-26T01:38:21.294223+0000","last_clean":"2023-10-26T01:38:21.294223+0000","last_became_active":"2023-10-26T01:38:20.319376+0000","last_became_peered":"2023-10-26T01:38:20.319376+0000","last_unstale":"2023-10-26T01:38:21.294223+0000","last_undegraded":"2023-10-26T01:38:21.294223+0000","last_fullsized":"2023-10-26T01:38:21.294223+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:59:26.990814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.298020+0000","last_change":"2023-10-26T01:38:21.307155+0000","last_active":"2023-10-26T01:38:22.298020+0000","last_peered":"2023-10-26T01:38:22.298020+0000","last_clean":"2023-10-26T01:38:22.298020+0000","last_became_active":"2023-10-26T01:38:21.306473+0000","last_became_peered":"2023-10-26T01:38:21.306473+0000","last_unstale":"2023-10-26T01:38:22.298020+0000","last_undegraded":"2023-10-26T01:38:22.298020+0000","last_fullsized":"2023-10-26T01:38:22.298020+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T01:55:12.014852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":12,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.301190+0000","last_change":"2023-10-26T01:38:20.324083+0000","last_active":"2023-10-26T01:38:21.301190+0000","last_peered":"2023-10-26T01:38:21.301190+0000","last_clean":"2023-10-26T01:38:21.301190+0000","last_became_active":"2023-10-26T01:38:20.320737+0000","last_became_peered":"2023-10-26T01:38:20.320737+0000","last_unstale":"2023-10-26T01:38:21.301190+0000","last_undegraded":"2023-10-26T01:38:21.301190+0000","last_fullsized":"2023-10-26T01:38:21.301190+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:31:09.167929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.297522+0000","last_change":"2023-10-26T01:38:21.299637+0000","last_active":"2023-10-26T01:38:22.297522+0000","last_peered":"2023-10-26T01:38:22.297522+0000","last_clean":"2023-10-26T01:38:22.297522+0000","last_became_active":"2023-10-26T01:38:21.298931+0000","last_became_peered":"2023-10-26T01:38:21.298931+0000","last_unstale":"2023-10-26T01:38:22.297522+0000","last_undegraded":"2023-10-26T01:38:22.297522+0000","last_fullsized":"2023-10-26T01:38:22.297522+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:19:17.527922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":12,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.299217+0000","last_change":"2023-10-26T01:38:20.321433+0000","last_active":"2023-10-26T01:38:21.299217+0000","last_peered":"2023-10-26T01:38:21.299217+0000","last_clean":"2023-10-26T01:38:21.299217+0000","last_became_active":"2023-10-26T01:38:20.321136+0000","last_became_peered":"2023-10-26T01:38:20.321136+0000","last_unstale":"2023-10-26T01:38:21.299217+0000","last_undegraded":"2023-10-26T01:38:21.299217+0000","last_fullsized":"2023-10-26T01:38:21.299217+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:00:59.371192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.297193+0000","last_change":"2023-10-26T01:38:21.324339+0000","last_active":"2023-10-26T01:38:22.297193+0000","last_peered":"2023-10-26T01:38:22.297193+0000","last_clean":"2023-10-26T01:38:22.297193+0000","last_became_active":"2023-10-26T01:38:21.323704+0000","last_became_peered":"2023-10-26T01:38:21.323704+0000","last_unstale":"2023-10-26T01:38:22.297193+0000","last_undegraded":"2023-10-26T01:38:22.297193+0000","last_fullsized":"2023-10-26T01:38:22.297193+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:51:42.712087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":12,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.294774+0000","last_change":"2023-10-26T01:38:20.324901+0000","last_active":"2023-10-26T01:38:21.294774+0000","last_peered":"2023-10-26T01:38:21.294774+0000","last_clean":"2023-10-26T01:38:21.294774+0000","last_became_active":"2023-10-26T01:38:20.324441+0000","last_became_peered":"2023-10-26T01:38:20.324441+0000","last_unstale":"2023-10-26T01:38:21.294774+0000","last_undegraded":"2023-10-26T01:38:21.294774+0000","last_fullsized":"2023-10-26T01:38:21.294774+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:02:55.480456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.357911+0000","last_change":"2023-10-26T01:38:21.318362+0000","last_active":"2023-10-26T01:38:22.357911+0000","last_peered":"2023-10-26T01:38:22.357911+0000","last_clean":"2023-10-26T01:38:22.357911+0000","last_became_active":"2023-10-26T01:38:21.317684+0000","last_became_peered":"2023-10-26T01:38:21.317684+0000","last_unstale":"2023-10-26T01:38:22.357911+0000","last_undegraded":"2023-10-26T01:38:22.357911+0000","last_fullsized":"2023-10-26T01:38:22.357911+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:42:43.138486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":12,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.313106+0000","last_change":"2023-10-26T01:38:20.320871+0000","last_active":"2023-10-26T01:38:21.313106+0000","last_peered":"2023-10-26T01:38:21.313106+0000","last_clean":"2023-10-26T01:38:21.313106+0000","last_became_active":"2023-10-26T01:38:20.320224+0000","last_became_peered":"2023-10-26T01:38:20.320224+0000","last_unstale":"2023-10-26T01:38:21.313106+0000","last_undegraded":"2023-10-26T01:38:21.313106+0000","last_fullsized":"2023-10-26T01:38:21.313106+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:30:07.282370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.322813+0000","last_change":"2023-10-26T01:38:21.321771+0000","last_active":"2023-10-26T01:38:21.322813+0000","last_peered":"2023-10-26T01:38:21.322813+0000","last_clean":"2023-10-26T01:38:21.322813+0000","last_became_active":"2023-10-26T01:38:21.321349+0000","last_became_peered":"2023-10-26T01:38:21.321349+0000","last_unstale":"2023-10-26T01:38:21.322813+0000","last_undegraded":"2023-10-26T01:38:21.322813+0000","last_fullsized":"2023-10-26T01:38:21.322813+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:22:49.920143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":12,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.303830+0000","last_change":"2023-10-26T01:38:20.321951+0000","last_active":"2023-10-26T01:38:21.303830+0000","last_peered":"2023-10-26T01:38:21.303830+0000","last_clean":"2023-10-26T01:38:21.303830+0000","last_became_active":"2023-10-26T01:38:20.310341+0000","last_became_peered":"2023-10-26T01:38:20.310341+0000","last_unstale":"2023-10-26T01:38:21.303830+0000","last_undegraded":"2023-10-26T01:38:21.303830+0000","last_fullsized":"2023-10-26T01:38:21.303830+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:11:33.879576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":278,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.315091+0000","last_change":"2023-10-26T01:30:34.883484+0000","last_active":"2023-10-26T01:38:21.315091+0000","last_peered":"2023-10-26T01:38:21.315091+0000","last_clean":"2023-10-26T01:38:21.315091+0000","last_became_active":"2023-10-26T01:30:34.883071+0000","last_became_peered":"2023-10-26T01:30:34.883071+0000","last_unstale":"2023-10-26T01:38:21.315091+0000","last_undegraded":"2023-10-26T01:38:21.315091+0000","last_fullsized":"2023-10-26T01:38:21.315091+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:32.890096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:32.890096+0000","last_clean_scrub_stamp":"2023-10-26T01:30:32.890096+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:26:31.879452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":14,"seq":60129542241,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":719,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194592,"statfs":{"total":200000000,"available":199262922,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.79200000000000004,"15min":0.78600000000000003},"min":{"1min":0.44900000000000001,"5min":0.35599999999999998,"15min":0.33000000000000002},"max":{"1min":1.6319999999999999,"5min":1.6319999999999999,"15min":2.2549999999999999},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.876,"15min":0.88800000000000001},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.236,"5min":1.5800000000000001,"15min":1.8200000000000001},"last":0.94099999999999995}]},{"osd":1,"last update":"Thu Oct 26 01:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.875,"15min":0.93700000000000006},"min":{"1min":0.40699999999999997,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.3560000000000001,"5min":1.444,"15min":4.4340000000000002},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.97599999999999998,"15min":0.997},"min":{"1min":0.51500000000000001,"5min":0.46700000000000003,"15min":0.374},"max":{"1min":1.4390000000000001,"5min":1.538,"15min":2.3330000000000002},"last":0.755}]},{"osd":2,"last update":"Thu Oct 26 01:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":1.032,"15min":1.075},"min":{"1min":0.53800000000000003,"5min":0.40999999999999998,"15min":0.39900000000000002},"max":{"1min":1.643,"5min":1.927,"15min":4.2800000000000002},"last":1.2689999999999999},{"interface":"front","average":{"1min":1.032,"5min":1.1279999999999999,"15min":1.091},"min":{"1min":0.50900000000000001,"5min":0.495,"15min":0.49399999999999999},"max":{"1min":1.607,"5min":1.8680000000000001,"15min":2.302},"last":1.3320000000000001}]},{"osd":4,"last update":"Thu Oct 26 01:37:49 2023","interfaces":[{"interface":"back","average":{"1min":1.175,"5min":1.2070000000000001,"15min":1.167},"min":{"1min":0.755,"5min":0.755,"15min":0.66400000000000003},"max":{"1min":1.536,"5min":1.7150000000000001,"15min":2.1070000000000002},"last":1.008},{"interface":"front","average":{"1min":1.252,"5min":1.3400000000000001,"15min":1.389},"min":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.79900000000000004},"max":{"1min":1.5209999999999999,"5min":2.4239999999999999,"15min":4.8250000000000002},"last":1.5740000000000001}]},{"osd":5,"last update":"Thu Oct 26 01:37:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.282,"15min":1.333},"min":{"1min":0.79300000000000004,"5min":0.76400000000000001,"15min":0.72499999999999998},"max":{"1min":1.639,"5min":2.3380000000000001,"15min":4.9080000000000004},"last":1.448},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.28,"15min":1.276},"min":{"1min":0.90300000000000002,"5min":0.79900000000000004,"15min":0.79600000000000004},"max":{"1min":1.722,"5min":2.0310000000000001,"15min":2.8980000000000001},"last":1.1120000000000001}]},{"osd":6,"last update":"Thu Oct 26 01:37:49 2023","interfaces":[{"interface":"back","average":{"1min":1.278,"5min":1.355,"15min":1.331},"min":{"1min":0.74199999999999999,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.5580000000000001,"5min":2.1499999999999999,"15min":2.8530000000000002},"last":1.2270000000000001},{"interface":"front","average":{"1min":1.206,"5min":1.268,"15min":1.268},"min":{"1min":0.81699999999999995,"5min":0.81499999999999995,"15min":0.73099999999999998},"max":{"1min":1.7410000000000001,"5min":2.048,"15min":2.181},"last":1.0820000000000001}]},{"osd":7,"last update":"Thu Oct 26 01:37:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.3300000000000001,"15min":1.296},"min":{"1min":0.85299999999999998,"5min":0.84599999999999997,"15min":0.71999999999999997},"max":{"1min":1.8129999999999999,"5min":1.958,"15min":2.0680000000000001},"last":1.046},{"interface":"front","average":{"1min":1.2889999999999999,"5min":1.3919999999999999,"15min":1.415},"min":{"1min":0.84199999999999997,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.647,"5min":2.2370000000000001,"15min":4.8730000000000002},"last":1.6399999999999999}]}]},{"osd":7,"up_from":13,"seq":55834574945,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1301,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194011,"statfs":{"total":200000000,"available":198667320,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:37:49 2023","interfaces":[{"interface":"back","average":{"1min":1.0920000000000001,"5min":1.139,"15min":1.093},"min":{"1min":0.81799999999999995,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.7,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.99099999999999999,"5min":1.115,"15min":1.081},"min":{"1min":0.70599999999999996,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.46,"5min":2.6779999999999999,"15min":2.6779999999999999},"last":1.1020000000000001}]},{"osd":1,"last update":"Thu Oct 26 01:37:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.1870000000000001,"15min":1.0960000000000001},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65200000000000002},"max":{"1min":1.5549999999999999,"5min":2.8260000000000001,"15min":2.8260000000000001},"last":0.82899999999999996},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.21,"15min":1.194},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.61199999999999999},"max":{"1min":1.7869999999999999,"5min":3.2730000000000001,"15min":3.2730000000000001},"last":0.93899999999999995}]},{"osd":2,"last update":"Thu Oct 26 01:37:49 2023","interfaces":[{"interface":"back","average":{"1min":1.143,"5min":1.179,"15min":1.1180000000000001},"min":{"1min":0.79100000000000004,"5min":0.65700000000000003,"15min":0.58499999999999996},"max":{"1min":1.667,"5min":2.218,"15min":2.218},"last":0.873},{"interface":"front","average":{"1min":1.2709999999999999,"5min":1.2989999999999999,"15min":1.24},"min":{"1min":0.89200000000000002,"5min":0.79400000000000004,"15min":0.73099999999999998},"max":{"1min":1.786,"5min":3.0510000000000002,"15min":3.0510000000000002},"last":1.1459999999999999}]},{"osd":3,"last update":"Thu Oct 26 01:37:49 2023","interfaces":[{"interface":"back","average":{"1min":1.264,"5min":1.284,"15min":1.214},"min":{"1min":0.93500000000000005,"5min":0.79000000000000004,"15min":0.72099999999999997},"max":{"1min":1.5389999999999999,"5min":1.9830000000000001,"15min":1.9830000000000001},"last":0.79500000000000004},{"interface":"front","average":{"1min":1.296,"5min":1.407,"15min":1.373},"min":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.65700000000000003},"max":{"1min":1.8939999999999999,"5min":5.5039999999999996,"15min":5.5039999999999996},"last":1.1839999999999999}]},{"osd":4,"last update":"Thu Oct 26 01:37:49 2023","interfaces":[{"interface":"back","average":{"1min":1.4350000000000001,"5min":1.373,"15min":1.3520000000000001},"min":{"1min":1.0589999999999999,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.7450000000000001,"5min":2.77,"15min":2.77},"last":1.077},{"interface":"front","average":{"1min":1.456,"5min":1.429,"15min":1.399},"min":{"1min":0.92700000000000005,"5min":0.90400000000000003,"15min":0.496},"max":{"1min":1.95,"5min":3.1419999999999999,"15min":3.1419999999999999},"last":0.98599999999999999}]},{"osd":5,"last update":"Thu Oct 26 01:37:49 2023","interfaces":[{"interface":"back","average":{"1min":1.498,"5min":1.488,"15min":1.4259999999999999},"min":{"1min":0.91700000000000004,"5min":0.65600000000000003,"15min":0.441},"max":{"1min":1.9890000000000001,"5min":5.734,"15min":5.734},"last":1.054},{"interface":"front","average":{"1min":1.4970000000000001,"5min":1.5269999999999999,"15min":1.478},"min":{"1min":1.1739999999999999,"5min":0.96099999999999997,"15min":0.51900000000000002},"max":{"1min":1.8799999999999999,"5min":2.8980000000000001,"15min":2.8980000000000001},"last":1.022}]},{"osd":6,"last update":"Thu Oct 26 01:37:49 2023","interfaces":[{"interface":"back","average":{"1min":1.5580000000000001,"5min":1.6080000000000001,"15min":1.8460000000000001},"min":{"1min":1.0169999999999999,"5min":0.753,"15min":0.47399999999999998},"max":{"1min":2.0739999999999998,"5min":5.6340000000000003,"15min":11.577999999999999},"last":1.3280000000000001},{"interface":"front","average":{"1min":1.615,"5min":1.6779999999999999,"15min":1.8620000000000001},"min":{"1min":1.2889999999999999,"5min":0.91800000000000004,"15min":0.55100000000000005},"max":{"1min":1.986,"5min":5.0919999999999996,"15min":10.704000000000001},"last":1.117}]}]},{"osd":6,"up_from":13,"seq":55834574945,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":723,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194588,"statfs":{"total":200000000,"available":199258848,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:37:43 2023","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.113,"15min":1.1539999999999999},"min":{"1min":0.69299999999999995,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.6100000000000001,"5min":2.395,"15min":2.395},"last":0.86899999999999999},{"interface":"front","average":{"1min":1.139,"5min":1.173,"15min":1.1220000000000001},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.59199999999999997},"max":{"1min":2.081,"5min":2.081,"15min":2.081},"last":0.94099999999999995}]},{"osd":1,"last update":"Thu Oct 26 01:37:43 2023","interfaces":[{"interface":"back","average":{"1min":1.2749999999999999,"5min":1.2110000000000001,"15min":1.1779999999999999},"min":{"1min":0.82499999999999996,"5min":0.71599999999999997,"15min":0.68300000000000005},"max":{"1min":1.992,"5min":2.0529999999999999,"15min":2.0529999999999999},"last":1.3220000000000001},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.179,"15min":1.204},"min":{"1min":0.67900000000000005,"5min":0.67700000000000005,"15min":0.64000000000000001},"max":{"1min":1.968,"5min":1.968,"15min":2.085},"last":0.82799999999999996}]},{"osd":2,"last update":"Thu Oct 26 01:37:43 2023","interfaces":[{"interface":"back","average":{"1min":1.077,"5min":1.1879999999999999,"15min":1.095},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.5649999999999999,"5min":3.7879999999999998,"15min":3.7879999999999998},"last":1.131},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.3120000000000001,"15min":1.268},"min":{"1min":0.90600000000000003,"5min":0.75800000000000001,"15min":0.68899999999999995},"max":{"1min":2.2200000000000002,"5min":2.2890000000000001,"15min":2.2890000000000001},"last":1.079}]},{"osd":3,"last update":"Thu Oct 26 01:37:43 2023","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.319,"15min":1.214},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.756,"5min":4.4560000000000004,"15min":4.4560000000000004},"last":0.999},{"interface":"front","average":{"1min":1.288,"5min":1.337,"15min":1.325},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.69499999999999995},"max":{"1min":2.1520000000000001,"5min":2.5019999999999998,"15min":2.5019999999999998},"last":1.2190000000000001}]},{"osd":4,"last update":"Thu Oct 26 01:37:43 2023","interfaces":[{"interface":"back","average":{"1min":1.2709999999999999,"5min":1.272,"15min":1.224},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.48399999999999999},"max":{"1min":1.754,"5min":2.9489999999999998,"15min":2.9489999999999998},"last":1.1559999999999999},{"interface":"front","average":{"1min":1.272,"5min":1.4299999999999999,"15min":1.3149999999999999},"min":{"1min":0.96799999999999997,"5min":0.81100000000000005,"15min":0.59299999999999997},"max":{"1min":1.7629999999999999,"5min":6.609,"15min":6.609},"last":1.3779999999999999}]},{"osd":5,"last update":"Thu Oct 26 01:37:43 2023","interfaces":[{"interface":"back","average":{"1min":1.4570000000000001,"5min":1.5469999999999999,"15min":1.4470000000000001},"min":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.67000000000000004},"max":{"1min":1.8380000000000001,"5min":5.4370000000000003,"15min":5.4370000000000003},"last":1.278},{"interface":"front","average":{"1min":1.4370000000000001,"5min":1.4410000000000001,"15min":1.4119999999999999},"min":{"1min":0.626,"5min":0.61399999999999999,"15min":0.58199999999999996},"max":{"1min":2.3279999999999998,"5min":2.621,"15min":2.621},"last":1.431}]},{"osd":7,"last update":"Thu Oct 26 01:37:43 2023","interfaces":[{"interface":"back","average":{"1min":1.3400000000000001,"5min":1.52,"15min":1.4099999999999999},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.54100000000000004},"max":{"1min":1.8740000000000001,"5min":11.804,"15min":11.804},"last":1.4950000000000001},{"interface":"front","average":{"1min":1.5049999999999999,"5min":1.6559999999999999,"15min":1.51},"min":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.77300000000000002},"max":{"1min":2.0710000000000002,"5min":11.541,"15min":11.541},"last":1.5429999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574945,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":724,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194587,"statfs":{"total":200000000,"available":199257700,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:37:46 2023","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.1870000000000001,"15min":1.129},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.5660000000000001,"5min":1.7390000000000001,"15min":1.7390000000000001},"last":1.2609999999999999},{"interface":"front","average":{"1min":1.204,"5min":1.28,"15min":1.1819999999999999},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.613,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":1.6279999999999999}]},{"osd":1,"last update":"Thu Oct 26 01:37:46 2023","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":1.1319999999999999,"15min":1.085},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.3640000000000001,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":0.89900000000000002},{"interface":"front","average":{"1min":1.069,"5min":1.137,"15min":1.095},"min":{"1min":0.629,"5min":0.629,"15min":0.60599999999999998},"max":{"1min":1.726,"5min":1.726,"15min":1.7669999999999999},"last":1.1399999999999999}]},{"osd":2,"last update":"Thu Oct 26 01:37:46 2023","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":1.1950000000000001,"15min":1.163},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.69299999999999995},"max":{"1min":1.681,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":1.3129999999999999},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.2290000000000001,"15min":1.181},"min":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.63900000000000001},"max":{"1min":1.4339999999999999,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":1.401}]},{"osd":3,"last update":"Thu Oct 26 01:37:46 2023","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.2949999999999999,"15min":1.238},"min":{"1min":0.78900000000000003,"5min":0.72699999999999998,"15min":0.72299999999999998},"max":{"1min":1.6180000000000001,"5min":1.7649999999999999,"15min":1.835},"last":1.5860000000000001},{"interface":"front","average":{"1min":1.2529999999999999,"5min":1.3220000000000001,"15min":1.2929999999999999},"min":{"1min":0.85199999999999998,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.7310000000000001,"5min":1.788,"15min":2.9249999999999998},"last":1.4570000000000001}]},{"osd":4,"last update":"Thu Oct 26 01:37:46 2023","interfaces":[{"interface":"back","average":{"1min":1.1519999999999999,"5min":1.2689999999999999,"15min":1.2050000000000001},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.49099999999999999},"max":{"1min":1.6719999999999999,"5min":1.845,"15min":1.845},"last":1.349},{"interface":"front","average":{"1min":1.3340000000000001,"5min":1.4359999999999999,"15min":1.355},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.56200000000000006},"max":{"1min":1.78,"5min":1.867,"15min":1.867},"last":1.776}]},{"osd":6,"last update":"Thu Oct 26 01:37:46 2023","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.4319999999999999,"15min":1.323},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.51900000000000002},"max":{"1min":1.7490000000000001,"5min":2.8250000000000002,"15min":2.8250000000000002},"last":1.5069999999999999},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.423,"15min":1.327},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.61799999999999999},"max":{"1min":1.661,"5min":1.849,"15min":1.8560000000000001},"last":1.55}]},{"osd":7,"last update":"Thu Oct 26 01:37:46 2023","interfaces":[{"interface":"back","average":{"1min":1.4219999999999999,"5min":1.536,"15min":1.4219999999999999},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.61699999999999999},"max":{"1min":1.8220000000000001,"5min":2.1040000000000001,"15min":2.1040000000000001},"last":1.716},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.5609999999999999,"15min":1.431},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.59499999999999997},"max":{"1min":1.853,"5min":2.0960000000000001,"15min":2.0960000000000001},"last":1.6830000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574945,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":724,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194588,"statfs":{"total":200000000,"available":199258304,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:37:42 2023","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.056,"15min":0.98499999999999999},"min":{"1min":0.63800000000000001,"5min":0.58099999999999996,"15min":0.53100000000000003},"max":{"1min":1.5369999999999999,"5min":1.885,"15min":2.0289999999999999},"last":1.0589999999999999},{"interface":"front","average":{"1min":1.0309999999999999,"5min":1.081,"15min":1.024},"min":{"1min":0.74099999999999999,"5min":0.66400000000000003,"15min":0.60599999999999998},"max":{"1min":1.5029999999999999,"5min":2.6829999999999998,"15min":2.6829999999999998},"last":0.97599999999999998}]},{"osd":1,"last update":"Thu Oct 26 01:37:42 2023","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.2,"15min":1.127},"min":{"1min":0.66700000000000004,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.591,"5min":4.0519999999999996,"15min":4.0519999999999996},"last":1.3},{"interface":"front","average":{"1min":1.1599999999999999,"5min":1.1559999999999999,"15min":1.1100000000000001},"min":{"1min":0.70399999999999996,"5min":0.627,"15min":0.60499999999999998},"max":{"1min":1.897,"5min":1.897,"15min":2.0760000000000001},"last":0.70799999999999996}]},{"osd":2,"last update":"Thu Oct 26 01:37:42 2023","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.3480000000000001,"15min":1.23},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.64300000000000002},"max":{"1min":1.845,"5min":5.6749999999999998,"15min":5.6749999999999998},"last":1.51},{"interface":"front","average":{"1min":1.3,"5min":1.218,"15min":1.1359999999999999},"min":{"1min":0.73899999999999999,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":3.673,"5min":3.673,"15min":3.673},"last":1.0269999999999999}]},{"osd":3,"last update":"Thu Oct 26 01:37:42 2023","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.204,"15min":1.1870000000000001},"min":{"1min":0.91500000000000004,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.7669999999999999,"5min":1.847,"15min":1.891},"last":0.82999999999999996},{"interface":"front","average":{"1min":1.194,"5min":1.21,"15min":1.159},"min":{"1min":0.85399999999999998,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.6419999999999999,"5min":1.7210000000000001,"15min":1.7210000000000001},"last":0.90400000000000003}]},{"osd":5,"last update":"Thu Oct 26 01:37:42 2023","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":1.2549999999999999,"15min":1.286},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.46300000000000002},"max":{"1min":1.8520000000000001,"5min":2.1429999999999998,"15min":2.1429999999999998},"last":1.2549999999999999},{"interface":"front","average":{"1min":1.3460000000000001,"5min":1.3500000000000001,"15min":1.3280000000000001},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.7649999999999999,"5min":5.8579999999999997,"15min":5.8579999999999997},"last":1.4039999999999999}]},{"osd":6,"last update":"Thu Oct 26 01:37:42 2023","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":1.427,"15min":1.3380000000000001},"min":{"1min":0.69199999999999995,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":2.2149999999999999,"5min":12.343999999999999,"15min":12.343999999999999},"last":1.589},{"interface":"front","average":{"1min":1.492,"5min":1.619,"15min":1.5149999999999999},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.55000000000000004},"max":{"1min":2.1000000000000001,"5min":12.605,"15min":12.605},"last":1.361}]},{"osd":7,"last update":"Thu Oct 26 01:37:42 2023","interfaces":[{"interface":"back","average":{"1min":1.4710000000000001,"5min":1.52,"15min":1.49},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":2.0390000000000001,"5min":5.774,"15min":5.774},"last":1.639},{"interface":"front","average":{"1min":1.53,"5min":1.5309999999999999,"15min":1.46},"min":{"1min":0.79000000000000004,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":2.3450000000000002,"5min":4.6520000000000001,"15min":4.6520000000000001},"last":1.452}]}]},{"osd":2,"up_from":13,"seq":55834574945,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":724,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194588,"statfs":{"total":200000000,"available":199258304,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:37:46 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.84099999999999997,"15min":0.85499999999999998},"min":{"1min":0.41199999999999998,"5min":0.308,"15min":0.308},"max":{"1min":1.4850000000000001,"5min":1.5249999999999999,"15min":6.3200000000000003},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.997,"15min":0.93100000000000005},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.2569999999999999,"5min":2.1400000000000001,"15min":4.2619999999999996},"last":0.88900000000000001}]},{"osd":1,"last update":"Thu Oct 26 01:37:46 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.90200000000000002,"15min":0.89700000000000002},"min":{"1min":0.39300000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.244,"5min":1.629,"15min":5.1260000000000003},"last":1.0109999999999999},{"interface":"front","average":{"1min":0.85399999999999998,"5min":1.018,"15min":1.0029999999999999},"min":{"1min":0.47699999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.351,"5min":1.873,"15min":6.1520000000000001},"last":0.94699999999999995}]},{"osd":3,"last update":"Thu Oct 26 01:37:48 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":1.0189999999999999,"15min":0.93899999999999995},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.443},"max":{"1min":1.4390000000000001,"5min":2.0640000000000001,"15min":2.0640000000000001},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.93600000000000005,"5min":1.111,"15min":1.018},"min":{"1min":0.51100000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.375,"5min":1.9279999999999999,"15min":8.9019999999999992},"last":1.351}]},{"osd":4,"last update":"Thu Oct 26 01:37:46 2023","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.3420000000000001,"15min":1.2689999999999999},"min":{"1min":0.86499999999999999,"5min":0.76000000000000001,"15min":0.73899999999999999},"max":{"1min":1.704,"5min":2.6190000000000002,"15min":6.2670000000000003},"last":1.0720000000000001},{"interface":"front","average":{"1min":1.1879999999999999,"5min":1.3129999999999999,"15min":1.262},"min":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.78700000000000003},"max":{"1min":1.5920000000000001,"5min":2.4359999999999999,"15min":7.343},"last":1.24}]},{"osd":5,"last update":"Thu Oct 26 01:37:46 2023","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.3540000000000001,"15min":1.2869999999999999},"min":{"1min":0.873,"5min":0.81699999999999995,"15min":0.71899999999999997},"max":{"1min":1.5740000000000001,"5min":2.5790000000000002,"15min":3.7170000000000001},"last":1.149},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.3089999999999999,"15min":1.2909999999999999},"min":{"1min":0.81899999999999995,"5min":0.70999999999999996,"15min":0.65300000000000002},"max":{"1min":1.621,"5min":2.298,"15min":6.3899999999999997},"last":1.3879999999999999}]},{"osd":6,"last update":"Thu Oct 26 01:37:46 2023","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":1.399,"15min":1.335},"min":{"1min":0.81200000000000006,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.9379999999999999,"5min":2.73,"15min":6.0490000000000004},"last":1.3340000000000001},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.4179999999999999,"15min":1.3720000000000001},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.76800000000000002},"max":{"1min":1.8799999999999999,"5min":2.4089999999999998,"15min":5.0650000000000004},"last":1.4590000000000001}]},{"osd":7,"last update":"Thu Oct 26 01:37:46 2023","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.403,"15min":1.395},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.69399999999999995},"max":{"1min":1.7809999999999999,"5min":2.3370000000000002,"15min":6.4480000000000004},"last":1.4339999999999999},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.52,"15min":1.4550000000000001},"min":{"1min":1.036,"5min":0.91800000000000004,"15min":0.91800000000000004},"max":{"1min":1.877,"5min":2.8290000000000002,"15min":2.8439999999999999},"last":1.5029999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574945,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":724,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194588,"statfs":{"total":200000000,"available":199258180,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:37:48 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.78300000000000003,"15min":0.72499999999999998},"min":{"1min":0.36099999999999999,"5min":0.34000000000000002,"15min":0.29499999999999998},"max":{"1min":1.4099999999999999,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.86799999999999999,"15min":0.79600000000000004},"min":{"1min":0.435,"5min":0.36299999999999999,"15min":0.36199999999999999},"max":{"1min":1.4019999999999999,"5min":1.6399999999999999,"15min":1.6399999999999999},"last":0.94899999999999995}]},{"osd":2,"last update":"Thu Oct 26 01:37:48 2023","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":0.97899999999999998,"15min":1.0009999999999999},"min":{"1min":0.52200000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.8200000000000001,"5min":1.8200000000000001,"15min":2.4830000000000001},"last":1.258},{"interface":"front","average":{"1min":1.0569999999999999,"5min":0.93700000000000006,"15min":0.875},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.772,"5min":1.8080000000000001,"15min":1.8080000000000001},"last":1.5369999999999999}]},{"osd":3,"last update":"Thu Oct 26 01:37:52 2023","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.97899999999999998,"15min":0.94899999999999995},"min":{"1min":0.52900000000000003,"5min":0.42499999999999999,"15min":0.39500000000000002},"max":{"1min":1.288,"5min":1.611,"15min":1.75},"last":1.097},{"interface":"front","average":{"1min":0.97899999999999998,"5min":1.028,"15min":0.98999999999999999},"min":{"1min":0.55900000000000005,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.45,"5min":1.673,"15min":1.802},"last":1.0069999999999999}]},{"osd":4,"last update":"Thu Oct 26 01:37:48 2023","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":1.2270000000000001,"15min":1.1990000000000001},"min":{"1min":0.82899999999999996,"5min":0.69099999999999995,"15min":0.56200000000000006},"max":{"1min":1.756,"5min":1.756,"15min":2.0659999999999998},"last":1.3080000000000001},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.1930000000000001,"15min":1.1359999999999999},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.474,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":1.409}]},{"osd":5,"last update":"Thu Oct 26 01:37:48 2023","interfaces":[{"interface":"back","average":{"1min":1.1459999999999999,"5min":1.262,"15min":1.2270000000000001},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.573,"5min":1.8779999999999999,"15min":1.8779999999999999},"last":1.2010000000000001},{"interface":"front","average":{"1min":1.3720000000000001,"5min":1.3720000000000001,"15min":1.337},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":2.04,"5min":2.04,"15min":2.2200000000000002},"last":1.486}]},{"osd":6,"last update":"Thu Oct 26 01:37:48 2023","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":1.3320000000000001,"15min":1.28},"min":{"1min":0.75,"5min":0.70199999999999996,"15min":0.68899999999999995},"max":{"1min":1.7330000000000001,"5min":2.0910000000000002,"15min":2.3780000000000001},"last":1.369},{"interface":"front","average":{"1min":1.26,"5min":1.2729999999999999,"15min":1.244},"min":{"1min":0.80700000000000005,"5min":0.71299999999999997,"15min":0.70799999999999996},"max":{"1min":1.669,"5min":1.956,"15min":1.984},"last":1.4399999999999999}]},{"osd":7,"last update":"Thu Oct 26 01:37:48 2023","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.2629999999999999,"15min":1.262},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.69699999999999995},"max":{"1min":1.7230000000000001,"5min":1.8380000000000001,"15min":2.6930000000000001},"last":1.0429999999999999},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.4019999999999999,"15min":1.337},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.9370000000000001,"5min":2.0190000000000001,"15min":2.0190000000000001},"last":1.617}]}]},{"osd":0,"up_from":13,"seq":55834574945,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1302,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194009,"statfs":{"total":200000000,"available":198666212,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Oct 26 01:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":1.0029999999999999,"15min":0.91600000000000004},"min":{"1min":0.41699999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.2170000000000001,"5min":19.425000000000001,"15min":19.425000000000001},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.84199999999999997,"5min":1.0720000000000001,"15min":1.042},"min":{"1min":0.56100000000000005,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.1839999999999999,"5min":15.162000000000001,"15min":15.162000000000001},"last":1.1499999999999999}]},{"osd":2,"last update":"Thu Oct 26 01:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":1.3080000000000001,"15min":1.1819999999999999},"min":{"1min":0.57699999999999996,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.4930000000000001,"5min":28.439,"15min":28.439},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.91700000000000004,"15min":0.91200000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.4810000000000001,"5min":1.502,"15min":4.9459999999999997},"last":0.64300000000000002}]},{"osd":3,"last update":"Thu Oct 26 01:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":1.0549999999999999,"15min":1.085},"min":{"1min":0.48899999999999999,"5min":0.46000000000000002,"15min":0.44600000000000001},"max":{"1min":1.4330000000000001,"5min":1.6479999999999999,"15min":4.9800000000000004},"last":1.111},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.1459999999999999,"15min":1.155},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.4770000000000001,"5min":1.659,"15min":5.016},"last":1.2330000000000001}]},{"osd":4,"last update":"Thu Oct 26 01:37:45 2023","interfaces":[{"interface":"back","average":{"1min":1.091,"5min":1.379,"15min":1.29},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.49,"5min":19.329000000000001,"15min":19.329000000000001},"last":0.97799999999999998},{"interface":"front","average":{"1min":1.22,"5min":1.268,"15min":1.2989999999999999},"min":{"1min":0.88,"5min":0.82099999999999995,"15min":0.67600000000000005},"max":{"1min":1.5449999999999999,"5min":1.718,"15min":5.5229999999999997},"last":1.3460000000000001}]},{"osd":5,"last update":"Thu Oct 26 01:37:45 2023","interfaces":[{"interface":"back","average":{"1min":1.373,"5min":1.4039999999999999,"15min":1.4299999999999999},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.661,"5min":1.915,"15min":5.6360000000000001},"last":1.395},{"interface":"front","average":{"1min":1.202,"5min":1.4790000000000001,"15min":1.3999999999999999},"min":{"1min":0.73599999999999999,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.71,"5min":19.384,"15min":19.384},"last":1.1839999999999999}]},{"osd":6,"last update":"Thu Oct 26 01:37:45 2023","interfaces":[{"interface":"back","average":{"1min":1.2609999999999999,"5min":1.3799999999999999,"15min":1.365},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.73799999999999999},"max":{"1min":1.615,"5min":8.2609999999999992,"15min":8.2609999999999992},"last":1.4650000000000001},{"interface":"front","average":{"1min":1.3440000000000001,"5min":1.4590000000000001,"15min":1.407},"min":{"1min":0.83099999999999996,"5min":0.77900000000000003,"15min":0.75900000000000001},"max":{"1min":1.804,"5min":11.849,"15min":11.849},"last":1.429}]},{"osd":7,"last update":"Thu Oct 26 01:37:45 2023","interfaces":[{"interface":"back","average":{"1min":1.3979999999999999,"5min":1.4450000000000001,"15min":1.4810000000000001},"min":{"1min":0.871,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.8380000000000001,"5min":2.4060000000000001,"15min":5.6689999999999996},"last":1.3149999999999999},{"interface":"front","average":{"1min":1.365,"5min":1.6120000000000001,"15min":1.5029999999999999},"min":{"1min":0.89600000000000002,"5min":0.879,"15min":0.69399999999999995},"max":{"1min":1.841,"5min":19.245999999999999,"15min":19.245999999999999},"last":1.2809999999999999}]}]}],"pool_statfs":[]}} 2023-10-26T01:38:26.824 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-26T01:38:27.082 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:27.080+0000 7f09cc41a640 1 -- 172.21.15.169:0/2411373861 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f09b00047c0 msgr2=0x7f09b0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:27.082 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:27.080+0000 7f09cc41a640 1 --2- 172.21.15.169:0/2411373861 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f09b00047c0 0x7f09b0004c40 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f09b4005fd0 tx=0x7f09b00057f0 comp rx=0 tx=0).stop 2023-10-26T01:38:27.083 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:27.080+0000 7f09cc41a640 1 -- 172.21.15.169:0/2411373861 shutdown_connections 2023-10-26T01:38:27.083 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:27.080+0000 7f09cc41a640 1 --2- 172.21.15.169:0/2411373861 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f09b00047c0 0x7f09b0004c40 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:27.083 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:27.080+0000 7f09cc41a640 1 -- 172.21.15.169:0/2411373861 >> 172.21.15.169:0/2411373861 conn(0x7f09c4069930 msgr2=0x7f09c4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:27.083 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:27.080+0000 7f09cc41a640 1 -- 172.21.15.169:0/2411373861 shutdown_connections 2023-10-26T01:38:27.083 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:27.080+0000 7f09cc41a640 1 -- 172.21.15.169:0/2411373861 wait complete. 2023-10-26T01:38:27.084 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:27.082+0000 7f09cc41a640 1 Processor -- start 2023-10-26T01:38:27.084 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:27.082+0000 7f09cc41a640 1 -- start start 2023-10-26T01:38:27.086 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:27.083+0000 7f09cc41a640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f09b00047c0 0x7f09c415a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:27.087 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:27.083+0000 7f09cc41a640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f09c40af410 0x7f09c40af890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:27.087 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:27.083+0000 7f09cc41a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f09c41e1120 0x7f09c41e15a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:27.087 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:27.083+0000 7f09cc41a640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09c4070cc0 con 0x7f09b00047c0 2023-10-26T01:38:27.087 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:27.083+0000 7f09cc41a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09c40700b0 con 0x7f09c41e1120 2023-10-26T01:38:27.087 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:27.083+0000 7f09cc41a640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09c4070390 con 0x7f09c40af410 2023-10-26T01:38:27.088 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:27.083+0000 7f09ca1b6640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f09b00047c0 0x7f09c415a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:27.088 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:27.083+0000 7f09ca1b6640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f09b00047c0 0x7f09c415a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:34334/0 (socket says 172.21.15.169:34334) 2023-10-26T01:38:27.088 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:27.084+0000 7f09ca1b6640 1 -- 172.21.15.169:0/1850975707 learned_addr learned my addr 172.21.15.169:0/1850975707 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:38:27.088 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:27.084+0000 7f09c99b5640 1 --2- 172.21.15.169:0/1850975707 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f09c40af410 0x7f09c40af890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:27.088 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:27.084+0000 7f09ca9b7640 1 --2- 172.21.15.169:0/1850975707 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f09c41e1120 0x7f09c41e15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:27.089 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:27.084+0000 7f09ca1b6640 1 -- 172.21.15.169:0/1850975707 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f09c40af410 msgr2=0x7f09c40af890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:27.089 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:27.084+0000 7f09ca1b6640 1 --2- 172.21.15.169:0/1850975707 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f09c40af410 0x7f09c40af890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:27.089 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:27.084+0000 7f09ca1b6640 1 -- 172.21.15.169:0/1850975707 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f09c41e1120 msgr2=0x7f09c41e15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:27.089 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:27.084+0000 7f09ca1b6640 1 --2- 172.21.15.169:0/1850975707 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f09c41e1120 0x7f09c41e15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:27.089 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:27.084+0000 7f09ca1b6640 1 -- 172.21.15.169:0/1850975707 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09c4070670 con 0x7f09b00047c0 2023-10-26T01:38:27.090 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:27.084+0000 7f09c99b5640 1 --2- 172.21.15.169:0/1850975707 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f09c40af410 0x7f09c40af890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:38:27.090 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:27.084+0000 7f09ca9b7640 1 --2- 172.21.15.169:0/1850975707 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f09c41e1120 0x7f09c41e15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:38:27.090 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:27.084+0000 7f09ca1b6640 1 --2- 172.21.15.169:0/1850975707 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f09b00047c0 0x7f09c415a320 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f09c000d8f0 tx=0x7f09c000ddb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:27.090 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:27.084+0000 7f09baffd640 1 -- 172.21.15.169:0/1850975707 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c00062e0 con 0x7f09b00047c0 2023-10-26T01:38:27.090 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:27.085+0000 7f09baffd640 1 -- 172.21.15.169:0/1850975707 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f09c000b830 con 0x7f09b00047c0 2023-10-26T01:38:27.091 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:27.085+0000 7f09baffd640 1 -- 172.21.15.169:0/1850975707 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c000bc40 con 0x7f09b00047c0 2023-10-26T01:38:27.091 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:27.085+0000 7f09cc41a640 1 -- 172.21.15.169:0/1850975707 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09c4070950 con 0x7f09b00047c0 2023-10-26T01:38:27.091 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:27.085+0000 7f09cc41a640 1 -- 172.21.15.169:0/1850975707 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09c415aa40 con 0x7f09b00047c0 2023-10-26T01:38:27.092 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:27.086+0000 7f09cc41a640 1 -- 172.21.15.169:0/1850975707 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09c415b660 con 0x7f09b00047c0 2023-10-26T01:38:27.092 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:27.087+0000 7f09baffd640 1 -- 172.21.15.169:0/1850975707 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f09c00035f0 con 0x7f09b00047c0 2023-10-26T01:38:27.092 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:27.088+0000 7f09baffd640 1 --2- 172.21.15.169:0/1850975707 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f09a806ae70 0x7f09a806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:27.093 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:27.088+0000 7f09baffd640 1 -- 172.21.15.169:0/1850975707 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f09c0089ee0 con 0x7f09b00047c0 2023-10-26T01:38:27.093 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:27.088+0000 7f09c99b5640 1 --2- 172.21.15.169:0/1850975707 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f09a806ae70 0x7f09a806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:27.093 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:27.089+0000 7f09c99b5640 1 --2- 172.21.15.169:0/1850975707 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f09a806ae70 0x7f09a806d330 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f09b4036000 tx=0x7f09b4035040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:27.094 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:27.092+0000 7f09baffd640 1 -- 172.21.15.169:0/1850975707 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f09c0054f90 con 0x7f09b00047c0 2023-10-26T01:38:27.277 INFO:tasks.ceph.mds.b.smithi170.stderr:2023-10-26T01:38:27.275+0000 7f56829a1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-26T01:38:27.352 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:27.351+0000 7f09cc41a640 1 -- 172.21.15.169:0/1850975707 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f09c415aa40 con 0x7f09b00047c0 2023-10-26T01:38:27.353 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:27.352+0000 7f09baffd640 1 -- 172.21.15.169:0/1850975707 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1168 (secure 0 0 0) 0x7f09c415aa40 con 0x7f09b00047c0 2023-10-26T01:38:27.353 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:38:27.354 INFO:teuthology.orchestra.run.smithi169.stdout:{"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":484,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":134,"num_osds":8,"num_up_osds":8,"osd_up_since":1698283831,"num_in_osds":8,"osd_in_since":1698283827,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":24,"data_bytes":592778,"bytes_used":7112210,"bytes_avail":1592887790,"bytes_total":1600000000,"write_bytes_sec":2388,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":68,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"b","status":"up:active","gid":7773}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":9,"modified":"2023-10-26T01:36:20.251763+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-10-26T01:38:27.356 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:27.354+0000 7f09cc41a640 1 -- 172.21.15.169:0/1850975707 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f09a806ae70 msgr2=0x7f09a806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:27.356 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:27.354+0000 7f09cc41a640 1 --2- 172.21.15.169:0/1850975707 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f09a806ae70 0x7f09a806d330 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f09b4036000 tx=0x7f09b4035040 comp rx=0 tx=0).stop 2023-10-26T01:38:27.357 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:27.354+0000 7f09cc41a640 1 -- 172.21.15.169:0/1850975707 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f09b00047c0 msgr2=0x7f09c415a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:27.357 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:27.354+0000 7f09cc41a640 1 --2- 172.21.15.169:0/1850975707 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f09b00047c0 0x7f09c415a320 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f09c000d8f0 tx=0x7f09c000ddb0 comp rx=0 tx=0).stop 2023-10-26T01:38:27.357 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:27.354+0000 7f09cc41a640 1 -- 172.21.15.169:0/1850975707 shutdown_connections 2023-10-26T01:38:27.357 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:27.354+0000 7f09cc41a640 1 --2- 172.21.15.169:0/1850975707 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f09a806ae70 0x7f09a806d330 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:27.358 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:27.354+0000 7f09cc41a640 1 --2- 172.21.15.169:0/1850975707 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f09c41e1120 0x7f09c41e15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:27.358 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:27.354+0000 7f09cc41a640 1 --2- 172.21.15.169:0/1850975707 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f09c40af410 0x7f09c40af890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:27.358 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:27.354+0000 7f09cc41a640 1 --2- 172.21.15.169:0/1850975707 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f09b00047c0 0x7f09c415a320 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:27.358 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:27.354+0000 7f09cc41a640 1 -- 172.21.15.169:0/1850975707 >> 172.21.15.169:0/1850975707 conn(0x7f09c4069930 msgr2=0x7f09c41558f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:27.358 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:27.354+0000 7f09cc41a640 1 -- 172.21.15.169:0/1850975707 shutdown_connections 2023-10-26T01:38:27.359 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:27.354+0000 7f09cc41a640 1 -- 172.21.15.169:0/1850975707 wait complete. 2023-10-26T01:38:27.375 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-10-26T01:38:27.376 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-26T01:38:27.589 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:27.587+0000 7f5865e02640 1 -- 172.21.15.169:0/3111306164 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f586015b610 msgr2=0x7f586006ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:27.589 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:27.587+0000 7f5865e02640 1 --2- 172.21.15.169:0/3111306164 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f586015b610 0x7f586006ed30 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f586006c960 tx=0x7f585002be30 comp rx=0 tx=0).stop 2023-10-26T01:38:27.589 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:27.587+0000 7f5865e02640 1 -- 172.21.15.169:0/3111306164 shutdown_connections 2023-10-26T01:38:27.589 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:27.587+0000 7f5865e02640 1 --2- 172.21.15.169:0/3111306164 >> v2:172.21.15.170:3300/0 conn(0x7f58600ac4e0 0x7f58600cd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:27.590 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:27.587+0000 7f5865e02640 1 --2- 172.21.15.169:0/3111306164 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f586015b610 0x7f586006ed30 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:27.590 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:27.587+0000 7f5865e02640 1 -- 172.21.15.169:0/3111306164 >> 172.21.15.169:0/3111306164 conn(0x7f5860069930 msgr2=0x7f5860069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:27.590 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:27.587+0000 7f5865e02640 1 -- 172.21.15.169:0/3111306164 shutdown_connections 2023-10-26T01:38:27.590 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:27.587+0000 7f5865e02640 1 -- 172.21.15.169:0/3111306164 wait complete. 2023-10-26T01:38:27.591 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:27.589+0000 7f5865e02640 1 Processor -- start 2023-10-26T01:38:27.591 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:27.589+0000 7f5865e02640 1 -- start start 2023-10-26T01:38:27.593 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:27.591+0000 7f5865e02640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58600ac4e0 0x7f586009ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:27.593 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:27.591+0000 7f5865e02640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f58600ac940 0x7f5860152ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:27.593 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:27.591+0000 7f5865e02640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f586015b610 0x7f5860099490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:27.594 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:27.591+0000 7f5865e02640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5860099d90 con 0x7f58600ac4e0 2023-10-26T01:38:27.594 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:27.591+0000 7f5865e02640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f586009a070 con 0x7f58600ac940 2023-10-26T01:38:27.594 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:27.591+0000 7f5865e02640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f586009a350 con 0x7f586015b610 2023-10-26T01:38:27.594 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:27.591+0000 7f585f7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58600ac4e0 0x7f586009ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:27.594 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:27.591+0000 7f585f7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58600ac4e0 0x7f586009ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:34346/0 (socket says 172.21.15.169:34346) 2023-10-26T01:38:27.595 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:27.591+0000 7f585f7fe640 1 -- 172.21.15.169:0/2116135222 learned_addr learned my addr 172.21.15.169:0/2116135222 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:38:27.595 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:27.591+0000 7f5857fff640 1 --2- 172.21.15.169:0/2116135222 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f58600ac940 0x7f5860152ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:27.595 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:27.591+0000 7f585f7fe640 1 -- 172.21.15.169:0/2116135222 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f586015b610 msgr2=0x7f5860099490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:27.595 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:27.591+0000 7f585f7fe640 1 --2- 172.21.15.169:0/2116135222 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f586015b610 0x7f5860099490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:27.595 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:27.591+0000 7f585f7fe640 1 -- 172.21.15.169:0/2116135222 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f58600ac940 msgr2=0x7f5860152ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:27.596 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:27.591+0000 7f585f7fe640 1 --2- 172.21.15.169:0/2116135222 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f58600ac940 0x7f5860152ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:27.596 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:27.591+0000 7f585f7fe640 1 -- 172.21.15.169:0/2116135222 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5850009560 con 0x7f58600ac4e0 2023-10-26T01:38:27.596 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:27.592+0000 7f585f7fe640 1 --2- 172.21.15.169:0/2116135222 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58600ac4e0 0x7f586009ae30 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f5850009850 tx=0x7f585002de00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:27.597 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:27.592+0000 7f585cff9640 1 -- 172.21.15.169:0/2116135222 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5850003800 con 0x7f58600ac4e0 2023-10-26T01:38:27.597 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:27.592+0000 7f585cff9640 1 -- 172.21.15.169:0/2116135222 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5850003a50 con 0x7f58600ac4e0 2023-10-26T01:38:27.597 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:27.592+0000 7f585cff9640 1 -- 172.21.15.169:0/2116135222 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5850003e40 con 0x7f58600ac4e0 2023-10-26T01:38:27.598 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:27.592+0000 7f5865e02640 1 -- 172.21.15.169:0/2116135222 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f586009a630 con 0x7f58600ac4e0 2023-10-26T01:38:27.598 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:27.592+0000 7f5865e02640 1 -- 172.21.15.169:0/2116135222 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5860153010 con 0x7f58600ac4e0 2023-10-26T01:38:27.598 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:27.593+0000 7f5865e02640 1 -- 172.21.15.169:0/2116135222 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f586009ab50 con 0x7f58600ac4e0 2023-10-26T01:38:27.598 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:27.594+0000 7f585cff9640 1 -- 172.21.15.169:0/2116135222 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f585003e450 con 0x7f58600ac4e0 2023-10-26T01:38:27.599 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:27.595+0000 7f585cff9640 1 --2- 172.21.15.169:0/2116135222 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f584006ae40 0x7f584006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:27.599 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:27.595+0000 7f585cff9640 1 -- 172.21.15.169:0/2116135222 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f58500ac1d0 con 0x7f58600ac4e0 2023-10-26T01:38:27.599 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:27.596+0000 7f5857fff640 1 --2- 172.21.15.169:0/2116135222 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f584006ae40 0x7f584006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:27.599 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:27.597+0000 7f5857fff640 1 --2- 172.21.15.169:0/2116135222 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f584006ae40 0x7f584006d300 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f5860070fe0 tx=0x7f5848005cc0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:27.601 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:27.600+0000 7f585cff9640 1 -- 172.21.15.169:0/2116135222 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f586009ab50 con 0x7f58600ac4e0 2023-10-26T01:38:27.788 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:27.786+0000 7f5865e02640 1 -- 172.21.15.169:0/2116135222 --> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f586009ab50 con 0x7f584006ae40 2023-10-26T01:38:27.810 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:27.808+0000 7f585cff9640 1 -- 172.21.15.169:0/2116135222 <== mgr.4112 v2:172.21.15.170:6832/1187156918 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+283029 (secure 0 0 0) 0x7f586009ab50 con 0x7f584006ae40 2023-10-26T01:38:27.814 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:38:27.815 INFO:teuthology.orchestra.run.smithi169.stderr:dumped all 2023-10-26T01:38:27.817 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:27.814+0000 7f5865e02640 1 -- 172.21.15.169:0/2116135222 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f584006ae40 msgr2=0x7f584006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:27.817 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:27.814+0000 7f5865e02640 1 --2- 172.21.15.169:0/2116135222 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f584006ae40 0x7f584006d300 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f5860070fe0 tx=0x7f5848005cc0 comp rx=0 tx=0).stop 2023-10-26T01:38:27.817 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:27.814+0000 7f5865e02640 1 -- 172.21.15.169:0/2116135222 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58600ac4e0 msgr2=0x7f586009ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:27.817 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:27.814+0000 7f5865e02640 1 --2- 172.21.15.169:0/2116135222 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58600ac4e0 0x7f586009ae30 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f5850009850 tx=0x7f585002de00 comp rx=0 tx=0).stop 2023-10-26T01:38:27.817 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:27.814+0000 7f5865e02640 1 -- 172.21.15.169:0/2116135222 shutdown_connections 2023-10-26T01:38:27.818 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:27.814+0000 7f5865e02640 1 --2- 172.21.15.169:0/2116135222 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f584006ae40 0x7f584006d300 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:27.818 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:27.814+0000 7f5865e02640 1 --2- 172.21.15.169:0/2116135222 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f586015b610 0x7f5860099490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:27.818 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:27.814+0000 7f5865e02640 1 --2- 172.21.15.169:0/2116135222 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f58600ac940 0x7f5860152ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:27.818 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:27.814+0000 7f5865e02640 1 --2- 172.21.15.169:0/2116135222 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58600ac4e0 0x7f586009ae30 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:27.819 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:27.814+0000 7f5865e02640 1 -- 172.21.15.169:0/2116135222 >> 172.21.15.169:0/2116135222 conn(0x7f5860069930 msgr2=0x7f58600ae3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:27.819 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:27.815+0000 7f5865e02640 1 -- 172.21.15.169:0/2116135222 shutdown_connections 2023-10-26T01:38:27.819 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:27.815+0000 7f5865e02640 1 -- 172.21.15.169:0/2116135222 wait complete. 2023-10-26T01:38:27.835 INFO:teuthology.orchestra.run.smithi169.stdout:{"pg_ready":true,"pg_map":{"version":365,"stamp":"2023-10-26T01:38:26.295294+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":153,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":6945,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1555554,"statfs":{"total":1600000000,"available":1592887790,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002031"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.359274+0000","last_change":"2023-10-26T01:38:21.325023+0000","last_active":"2023-10-26T01:38:22.359274+0000","last_peered":"2023-10-26T01:38:22.359274+0000","last_clean":"2023-10-26T01:38:22.359274+0000","last_became_active":"2023-10-26T01:38:21.323959+0000","last_became_peered":"2023-10-26T01:38:21.323959+0000","last_unstale":"2023-10-26T01:38:22.359274+0000","last_undegraded":"2023-10-26T01:38:22.359274+0000","last_fullsized":"2023-10-26T01:38:22.359274+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:51:07.392163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.297000+0000","last_change":"2023-10-26T01:38:21.315511+0000","last_active":"2023-10-26T01:38:22.297000+0000","last_peered":"2023-10-26T01:38:22.297000+0000","last_clean":"2023-10-26T01:38:22.297000+0000","last_became_active":"2023-10-26T01:38:21.314652+0000","last_became_peered":"2023-10-26T01:38:21.314652+0000","last_unstale":"2023-10-26T01:38:22.297000+0000","last_undegraded":"2023-10-26T01:38:22.297000+0000","last_fullsized":"2023-10-26T01:38:22.297000+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:44:57.150927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.298865+0000","last_change":"2023-10-26T01:38:21.327284+0000","last_active":"2023-10-26T01:38:22.298865+0000","last_peered":"2023-10-26T01:38:22.298865+0000","last_clean":"2023-10-26T01:38:22.298865+0000","last_became_active":"2023-10-26T01:38:21.326309+0000","last_became_peered":"2023-10-26T01:38:21.326309+0000","last_unstale":"2023-10-26T01:38:22.298865+0000","last_undegraded":"2023-10-26T01:38:22.298865+0000","last_fullsized":"2023-10-26T01:38:22.298865+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:46:37.417585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.358670+0000","last_change":"2023-10-26T01:38:21.317851+0000","last_active":"2023-10-26T01:38:22.358670+0000","last_peered":"2023-10-26T01:38:22.358670+0000","last_clean":"2023-10-26T01:38:22.358670+0000","last_became_active":"2023-10-26T01:38:21.317269+0000","last_became_peered":"2023-10-26T01:38:21.317269+0000","last_unstale":"2023-10-26T01:38:22.358670+0000","last_undegraded":"2023-10-26T01:38:22.358670+0000","last_fullsized":"2023-10-26T01:38:22.358670+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T01:59:52.462839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.357989+0000","last_change":"2023-10-26T01:38:21.304138+0000","last_active":"2023-10-26T01:38:22.357989+0000","last_peered":"2023-10-26T01:38:22.357989+0000","last_clean":"2023-10-26T01:38:22.357989+0000","last_became_active":"2023-10-26T01:38:21.303352+0000","last_became_peered":"2023-10-26T01:38:21.303352+0000","last_unstale":"2023-10-26T01:38:22.357989+0000","last_undegraded":"2023-10-26T01:38:22.357989+0000","last_fullsized":"2023-10-26T01:38:22.357989+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:36:15.755953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.357771+0000","last_change":"2023-10-26T01:38:21.319100+0000","last_active":"2023-10-26T01:38:22.357771+0000","last_peered":"2023-10-26T01:38:22.357771+0000","last_clean":"2023-10-26T01:38:22.357771+0000","last_became_active":"2023-10-26T01:38:21.318169+0000","last_became_peered":"2023-10-26T01:38:21.318169+0000","last_unstale":"2023-10-26T01:38:22.357771+0000","last_undegraded":"2023-10-26T01:38:22.357771+0000","last_fullsized":"2023-10-26T01:38:22.357771+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:05:26.952713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.356781+0000","last_change":"2023-10-26T01:38:21.318003+0000","last_active":"2023-10-26T01:38:22.356781+0000","last_peered":"2023-10-26T01:38:22.356781+0000","last_clean":"2023-10-26T01:38:22.356781+0000","last_became_active":"2023-10-26T01:38:21.317493+0000","last_became_peered":"2023-10-26T01:38:21.317493+0000","last_unstale":"2023-10-26T01:38:22.356781+0000","last_undegraded":"2023-10-26T01:38:22.356781+0000","last_fullsized":"2023-10-26T01:38:22.356781+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:32:43.401727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.299581+0000","last_change":"2023-10-26T01:38:21.324036+0000","last_active":"2023-10-26T01:38:22.299581+0000","last_peered":"2023-10-26T01:38:22.299581+0000","last_clean":"2023-10-26T01:38:22.299581+0000","last_became_active":"2023-10-26T01:38:21.323785+0000","last_became_peered":"2023-10-26T01:38:21.323785+0000","last_unstale":"2023-10-26T01:38:22.299581+0000","last_undegraded":"2023-10-26T01:38:22.299581+0000","last_fullsized":"2023-10-26T01:38:22.299581+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:27:43.267716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.358805+0000","last_change":"2023-10-26T01:38:21.323358+0000","last_active":"2023-10-26T01:38:22.358805+0000","last_peered":"2023-10-26T01:38:22.358805+0000","last_clean":"2023-10-26T01:38:22.358805+0000","last_became_active":"2023-10-26T01:38:21.322567+0000","last_became_peered":"2023-10-26T01:38:21.322567+0000","last_unstale":"2023-10-26T01:38:22.358805+0000","last_undegraded":"2023-10-26T01:38:22.358805+0000","last_fullsized":"2023-10-26T01:38:22.358805+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:07:49.066636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.357274+0000","last_change":"2023-10-26T01:38:21.320421+0000","last_active":"2023-10-26T01:38:22.357274+0000","last_peered":"2023-10-26T01:38:22.357274+0000","last_clean":"2023-10-26T01:38:22.357274+0000","last_became_active":"2023-10-26T01:38:21.319296+0000","last_became_peered":"2023-10-26T01:38:21.319296+0000","last_unstale":"2023-10-26T01:38:22.357274+0000","last_undegraded":"2023-10-26T01:38:22.357274+0000","last_fullsized":"2023-10-26T01:38:22.357274+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:24:08.193245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.296689+0000","last_change":"2023-10-26T01:38:21.313660+0000","last_active":"2023-10-26T01:38:22.296689+0000","last_peered":"2023-10-26T01:38:22.296689+0000","last_clean":"2023-10-26T01:38:22.296689+0000","last_became_active":"2023-10-26T01:38:21.312980+0000","last_became_peered":"2023-10-26T01:38:21.312980+0000","last_unstale":"2023-10-26T01:38:22.296689+0000","last_undegraded":"2023-10-26T01:38:22.296689+0000","last_fullsized":"2023-10-26T01:38:22.296689+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:00:42.118665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.358755+0000","last_change":"2023-10-26T01:38:21.315876+0000","last_active":"2023-10-26T01:38:22.358755+0000","last_peered":"2023-10-26T01:38:22.358755+0000","last_clean":"2023-10-26T01:38:22.358755+0000","last_became_active":"2023-10-26T01:38:21.314851+0000","last_became_peered":"2023-10-26T01:38:21.314851+0000","last_unstale":"2023-10-26T01:38:22.358755+0000","last_undegraded":"2023-10-26T01:38:22.358755+0000","last_fullsized":"2023-10-26T01:38:22.358755+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:34:15.657860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.299218+0000","last_change":"2023-10-26T01:38:21.327458+0000","last_active":"2023-10-26T01:38:22.299218+0000","last_peered":"2023-10-26T01:38:22.299218+0000","last_clean":"2023-10-26T01:38:22.299218+0000","last_became_active":"2023-10-26T01:38:21.326720+0000","last_became_peered":"2023-10-26T01:38:21.326720+0000","last_unstale":"2023-10-26T01:38:22.299218+0000","last_undegraded":"2023-10-26T01:38:22.299218+0000","last_fullsized":"2023-10-26T01:38:22.299218+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:48:01.026075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.296528+0000","last_change":"2023-10-26T01:38:21.317035+0000","last_active":"2023-10-26T01:38:22.296528+0000","last_peered":"2023-10-26T01:38:22.296528+0000","last_clean":"2023-10-26T01:38:22.296528+0000","last_became_active":"2023-10-26T01:38:21.316156+0000","last_became_peered":"2023-10-26T01:38:21.316156+0000","last_unstale":"2023-10-26T01:38:22.296528+0000","last_undegraded":"2023-10-26T01:38:22.296528+0000","last_fullsized":"2023-10-26T01:38:22.296528+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T01:50:22.270135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.356685+0000","last_change":"2023-10-26T01:38:21.312264+0000","last_active":"2023-10-26T01:38:22.356685+0000","last_peered":"2023-10-26T01:38:22.356685+0000","last_clean":"2023-10-26T01:38:22.356685+0000","last_became_active":"2023-10-26T01:38:21.311809+0000","last_became_peered":"2023-10-26T01:38:21.311809+0000","last_unstale":"2023-10-26T01:38:22.356685+0000","last_undegraded":"2023-10-26T01:38:22.356685+0000","last_fullsized":"2023-10-26T01:38:22.356685+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:36:15.755953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.359566+0000","last_change":"2023-10-26T01:38:21.325166+0000","last_active":"2023-10-26T01:38:22.359566+0000","last_peered":"2023-10-26T01:38:22.359566+0000","last_clean":"2023-10-26T01:38:22.359566+0000","last_became_active":"2023-10-26T01:38:21.324612+0000","last_became_peered":"2023-10-26T01:38:21.324612+0000","last_unstale":"2023-10-26T01:38:22.359566+0000","last_undegraded":"2023-10-26T01:38:22.359566+0000","last_fullsized":"2023-10-26T01:38:22.359566+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:17:29.812141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.297352+0000","last_change":"2023-10-26T01:38:21.315001+0000","last_active":"2023-10-26T01:38:22.297352+0000","last_peered":"2023-10-26T01:38:22.297352+0000","last_clean":"2023-10-26T01:38:22.297352+0000","last_became_active":"2023-10-26T01:38:21.312976+0000","last_became_peered":"2023-10-26T01:38:21.312976+0000","last_unstale":"2023-10-26T01:38:22.297352+0000","last_undegraded":"2023-10-26T01:38:22.297352+0000","last_fullsized":"2023-10-26T01:38:22.297352+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:29:52.799943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.356647+0000","last_change":"2023-10-26T01:38:21.319948+0000","last_active":"2023-10-26T01:38:22.356647+0000","last_peered":"2023-10-26T01:38:22.356647+0000","last_clean":"2023-10-26T01:38:22.356647+0000","last_became_active":"2023-10-26T01:38:21.319058+0000","last_became_peered":"2023-10-26T01:38:21.319058+0000","last_unstale":"2023-10-26T01:38:22.356647+0000","last_undegraded":"2023-10-26T01:38:22.356647+0000","last_fullsized":"2023-10-26T01:38:22.356647+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:29:52.799943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.296873+0000","last_change":"2023-10-26T01:38:21.324164+0000","last_active":"2023-10-26T01:38:22.296873+0000","last_peered":"2023-10-26T01:38:22.296873+0000","last_clean":"2023-10-26T01:38:22.296873+0000","last_became_active":"2023-10-26T01:38:21.323176+0000","last_became_peered":"2023-10-26T01:38:21.323176+0000","last_unstale":"2023-10-26T01:38:22.296873+0000","last_undegraded":"2023-10-26T01:38:22.296873+0000","last_fullsized":"2023-10-26T01:38:22.296873+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:44:57.150927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.299813+0000","last_change":"2023-10-26T01:38:21.313793+0000","last_active":"2023-10-26T01:38:22.299813+0000","last_peered":"2023-10-26T01:38:22.299813+0000","last_clean":"2023-10-26T01:38:22.299813+0000","last_became_active":"2023-10-26T01:38:21.312693+0000","last_became_peered":"2023-10-26T01:38:21.312693+0000","last_unstale":"2023-10-26T01:38:22.299813+0000","last_undegraded":"2023-10-26T01:38:22.299813+0000","last_fullsized":"2023-10-26T01:38:22.299813+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:47:08.011620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.297043+0000","last_change":"2023-10-26T01:38:21.305093+0000","last_active":"2023-10-26T01:38:22.297043+0000","last_peered":"2023-10-26T01:38:22.297043+0000","last_clean":"2023-10-26T01:38:22.297043+0000","last_became_active":"2023-10-26T01:38:21.304404+0000","last_became_peered":"2023-10-26T01:38:21.304404+0000","last_unstale":"2023-10-26T01:38:22.297043+0000","last_undegraded":"2023-10-26T01:38:22.297043+0000","last_fullsized":"2023-10-26T01:38:22.297043+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:51:42.712087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.317702+0000","last_change":"2023-10-26T01:38:21.314965+0000","last_active":"2023-10-26T01:38:21.317702+0000","last_peered":"2023-10-26T01:38:21.317702+0000","last_clean":"2023-10-26T01:38:21.317702+0000","last_became_active":"2023-10-26T01:38:21.314673+0000","last_became_peered":"2023-10-26T01:38:21.314673+0000","last_unstale":"2023-10-26T01:38:21.317702+0000","last_undegraded":"2023-10-26T01:38:21.317702+0000","last_fullsized":"2023-10-26T01:38:21.317702+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:33:14.986087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.298857+0000","last_change":"2023-10-26T01:38:21.323272+0000","last_active":"2023-10-26T01:38:22.298857+0000","last_peered":"2023-10-26T01:38:22.298857+0000","last_clean":"2023-10-26T01:38:22.298857+0000","last_became_active":"2023-10-26T01:38:21.322991+0000","last_became_peered":"2023-10-26T01:38:21.322991+0000","last_unstale":"2023-10-26T01:38:22.298857+0000","last_undegraded":"2023-10-26T01:38:22.298857+0000","last_fullsized":"2023-10-26T01:38:22.298857+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:48:01.026075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.320186+0000","last_change":"2023-10-26T01:38:21.320068+0000","last_active":"2023-10-26T01:38:21.320186+0000","last_peered":"2023-10-26T01:38:21.320186+0000","last_clean":"2023-10-26T01:38:21.320186+0000","last_became_active":"2023-10-26T01:38:21.319773+0000","last_became_peered":"2023-10-26T01:38:21.319773+0000","last_unstale":"2023-10-26T01:38:21.320186+0000","last_undegraded":"2023-10-26T01:38:21.320186+0000","last_fullsized":"2023-10-26T01:38:21.320186+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:03:39.495499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.300137+0000","last_change":"2023-10-26T01:38:21.314000+0000","last_active":"2023-10-26T01:38:22.300137+0000","last_peered":"2023-10-26T01:38:22.300137+0000","last_clean":"2023-10-26T01:38:22.300137+0000","last_became_active":"2023-10-26T01:38:21.313194+0000","last_became_peered":"2023-10-26T01:38:21.313194+0000","last_unstale":"2023-10-26T01:38:22.300137+0000","last_undegraded":"2023-10-26T01:38:22.300137+0000","last_fullsized":"2023-10-26T01:38:22.300137+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T01:40:34.431360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.359873+0000","last_change":"2023-10-26T01:38:21.324296+0000","last_active":"2023-10-26T01:38:22.359873+0000","last_peered":"2023-10-26T01:38:22.359873+0000","last_clean":"2023-10-26T01:38:22.359873+0000","last_became_active":"2023-10-26T01:38:21.320669+0000","last_became_peered":"2023-10-26T01:38:21.320669+0000","last_unstale":"2023-10-26T01:38:22.359873+0000","last_undegraded":"2023-10-26T01:38:22.359873+0000","last_fullsized":"2023-10-26T01:38:22.359873+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:16:57.330926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.357018+0000","last_change":"2023-10-26T01:38:21.320441+0000","last_active":"2023-10-26T01:38:22.357018+0000","last_peered":"2023-10-26T01:38:22.357018+0000","last_clean":"2023-10-26T01:38:22.357018+0000","last_became_active":"2023-10-26T01:38:21.320185+0000","last_became_peered":"2023-10-26T01:38:21.320185+0000","last_unstale":"2023-10-26T01:38:22.357018+0000","last_undegraded":"2023-10-26T01:38:22.357018+0000","last_fullsized":"2023-10-26T01:38:22.357018+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:11:34.893562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.320158+0000","last_change":"2023-10-26T01:38:21.319408+0000","last_active":"2023-10-26T01:38:21.320158+0000","last_peered":"2023-10-26T01:38:21.320158+0000","last_clean":"2023-10-26T01:38:21.320158+0000","last_became_active":"2023-10-26T01:38:21.318421+0000","last_became_peered":"2023-10-26T01:38:21.318421+0000","last_unstale":"2023-10-26T01:38:21.320158+0000","last_undegraded":"2023-10-26T01:38:21.320158+0000","last_fullsized":"2023-10-26T01:38:21.320158+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:23:42.697097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.296824+0000","last_change":"2023-10-26T01:38:21.322363+0000","last_active":"2023-10-26T01:38:22.296824+0000","last_peered":"2023-10-26T01:38:22.296824+0000","last_clean":"2023-10-26T01:38:22.296824+0000","last_became_active":"2023-10-26T01:38:21.322104+0000","last_became_peered":"2023-10-26T01:38:21.322104+0000","last_unstale":"2023-10-26T01:38:22.296824+0000","last_undegraded":"2023-10-26T01:38:22.296824+0000","last_fullsized":"2023-10-26T01:38:22.296824+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T01:50:22.270135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.356583+0000","last_change":"2023-10-26T01:38:21.311895+0000","last_active":"2023-10-26T01:38:22.356583+0000","last_peered":"2023-10-26T01:38:22.356583+0000","last_clean":"2023-10-26T01:38:22.356583+0000","last_became_active":"2023-10-26T01:38:21.311281+0000","last_became_peered":"2023-10-26T01:38:21.311281+0000","last_unstale":"2023-10-26T01:38:22.356583+0000","last_undegraded":"2023-10-26T01:38:22.356583+0000","last_fullsized":"2023-10-26T01:38:22.356583+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:24:08.965712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.360039+0000","last_change":"2023-10-26T01:38:21.324857+0000","last_active":"2023-10-26T01:38:22.360039+0000","last_peered":"2023-10-26T01:38:22.360039+0000","last_clean":"2023-10-26T01:38:22.360039+0000","last_became_active":"2023-10-26T01:38:21.323546+0000","last_became_peered":"2023-10-26T01:38:21.323546+0000","last_unstale":"2023-10-26T01:38:22.360039+0000","last_undegraded":"2023-10-26T01:38:22.360039+0000","last_fullsized":"2023-10-26T01:38:22.360039+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:53:21.542954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.357316+0000","last_change":"2023-10-26T01:38:21.315041+0000","last_active":"2023-10-26T01:38:22.357316+0000","last_peered":"2023-10-26T01:38:22.357316+0000","last_clean":"2023-10-26T01:38:22.357316+0000","last_became_active":"2023-10-26T01:38:21.314746+0000","last_became_peered":"2023-10-26T01:38:21.314746+0000","last_unstale":"2023-10-26T01:38:22.357316+0000","last_undegraded":"2023-10-26T01:38:22.357316+0000","last_fullsized":"2023-10-26T01:38:22.357316+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:27:03.489686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.355991+0000","last_change":"2023-10-26T01:38:21.304208+0000","last_active":"2023-10-26T01:38:22.355991+0000","last_peered":"2023-10-26T01:38:22.355991+0000","last_clean":"2023-10-26T01:38:22.355991+0000","last_became_active":"2023-10-26T01:38:21.303492+0000","last_became_peered":"2023-10-26T01:38:21.303492+0000","last_unstale":"2023-10-26T01:38:22.355991+0000","last_undegraded":"2023-10-26T01:38:22.355991+0000","last_fullsized":"2023-10-26T01:38:22.355991+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:57:23.914586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"133'4","reported_seq":16,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.334628+0000","last_change":"2023-10-26T01:38:20.322005+0000","last_active":"2023-10-26T01:38:21.334628+0000","last_peered":"2023-10-26T01:38:21.334628+0000","last_clean":"2023-10-26T01:38:21.334628+0000","last_became_active":"2023-10-26T01:38:20.320582+0000","last_became_peered":"2023-10-26T01:38:20.320582+0000","last_unstale":"2023-10-26T01:38:21.334628+0000","last_undegraded":"2023-10-26T01:38:21.334628+0000","last_fullsized":"2023-10-26T01:38:21.334628+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:00:02.848161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.357007+0000","last_change":"2023-10-26T01:38:21.323121+0000","last_active":"2023-10-26T01:38:22.357007+0000","last_peered":"2023-10-26T01:38:22.357007+0000","last_clean":"2023-10-26T01:38:22.357007+0000","last_became_active":"2023-10-26T01:38:21.322808+0000","last_became_peered":"2023-10-26T01:38:21.322808+0000","last_unstale":"2023-10-26T01:38:22.357007+0000","last_undegraded":"2023-10-26T01:38:22.357007+0000","last_fullsized":"2023-10-26T01:38:22.357007+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:46:37.417585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"133'1","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.334372+0000","last_change":"2023-10-26T01:38:20.326148+0000","last_active":"2023-10-26T01:38:21.334372+0000","last_peered":"2023-10-26T01:38:21.334372+0000","last_clean":"2023-10-26T01:38:21.334372+0000","last_became_active":"2023-10-26T01:38:20.325883+0000","last_became_peered":"2023-10-26T01:38:20.325883+0000","last_unstale":"2023-10-26T01:38:21.334372+0000","last_undegraded":"2023-10-26T01:38:21.334372+0000","last_fullsized":"2023-10-26T01:38:21.334372+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:36:47.183420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.358082+0000","last_change":"2023-10-26T01:38:21.321027+0000","last_active":"2023-10-26T01:38:22.358082+0000","last_peered":"2023-10-26T01:38:22.358082+0000","last_clean":"2023-10-26T01:38:22.358082+0000","last_became_active":"2023-10-26T01:38:21.318702+0000","last_became_peered":"2023-10-26T01:38:21.318702+0000","last_unstale":"2023-10-26T01:38:22.358082+0000","last_undegraded":"2023-10-26T01:38:22.358082+0000","last_fullsized":"2023-10-26T01:38:22.358082+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:36:20.377483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"133'2","reported_seq":14,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.338315+0000","last_change":"2023-10-26T01:38:20.321817+0000","last_active":"2023-10-26T01:38:21.338315+0000","last_peered":"2023-10-26T01:38:21.338315+0000","last_clean":"2023-10-26T01:38:21.338315+0000","last_became_active":"2023-10-26T01:38:20.321520+0000","last_became_peered":"2023-10-26T01:38:20.321520+0000","last_unstale":"2023-10-26T01:38:21.338315+0000","last_undegraded":"2023-10-26T01:38:21.338315+0000","last_fullsized":"2023-10-26T01:38:21.338315+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:57:22.900600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.299225+0000","last_change":"2023-10-26T01:38:21.321216+0000","last_active":"2023-10-26T01:38:22.299225+0000","last_peered":"2023-10-26T01:38:22.299225+0000","last_clean":"2023-10-26T01:38:22.299225+0000","last_became_active":"2023-10-26T01:38:21.320953+0000","last_became_peered":"2023-10-26T01:38:21.320953+0000","last_unstale":"2023-10-26T01:38:22.299225+0000","last_undegraded":"2023-10-26T01:38:22.299225+0000","last_fullsized":"2023-10-26T01:38:22.299225+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:47:08.011620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"133'2","reported_seq":14,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.337393+0000","last_change":"2023-10-26T01:38:20.325154+0000","last_active":"2023-10-26T01:38:21.337393+0000","last_peered":"2023-10-26T01:38:21.337393+0000","last_clean":"2023-10-26T01:38:21.337393+0000","last_became_active":"2023-10-26T01:38:20.324072+0000","last_became_peered":"2023-10-26T01:38:20.324072+0000","last_unstale":"2023-10-26T01:38:21.337393+0000","last_undegraded":"2023-10-26T01:38:21.337393+0000","last_fullsized":"2023-10-26T01:38:21.337393+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:31:09.167929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.357578+0000","last_change":"2023-10-26T01:38:21.323345+0000","last_active":"2023-10-26T01:38:22.357578+0000","last_peered":"2023-10-26T01:38:22.357578+0000","last_clean":"2023-10-26T01:38:22.357578+0000","last_became_active":"2023-10-26T01:38:21.323051+0000","last_became_peered":"2023-10-26T01:38:21.323051+0000","last_unstale":"2023-10-26T01:38:22.357578+0000","last_undegraded":"2023-10-26T01:38:22.357578+0000","last_fullsized":"2023-10-26T01:38:22.357578+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:46:30.031341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"133'1","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.333965+0000","last_change":"2023-10-26T01:38:20.323989+0000","last_active":"2023-10-26T01:38:21.333965+0000","last_peered":"2023-10-26T01:38:21.333965+0000","last_clean":"2023-10-26T01:38:21.333965+0000","last_became_active":"2023-10-26T01:38:20.323646+0000","last_became_peered":"2023-10-26T01:38:20.323646+0000","last_unstale":"2023-10-26T01:38:21.333965+0000","last_undegraded":"2023-10-26T01:38:21.333965+0000","last_fullsized":"2023-10-26T01:38:21.333965+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:57:22.900600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.297706+0000","last_change":"2023-10-26T01:38:21.315216+0000","last_active":"2023-10-26T01:38:22.297706+0000","last_peered":"2023-10-26T01:38:22.297706+0000","last_clean":"2023-10-26T01:38:22.297706+0000","last_became_active":"2023-10-26T01:38:21.314095+0000","last_became_peered":"2023-10-26T01:38:21.314095+0000","last_unstale":"2023-10-26T01:38:22.297706+0000","last_undegraded":"2023-10-26T01:38:22.297706+0000","last_fullsized":"2023-10-26T01:38:22.297706+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:46:37.417585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"133'1","reported_seq":12,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.335571+0000","last_change":"2023-10-26T01:38:20.320626+0000","last_active":"2023-10-26T01:38:21.335571+0000","last_peered":"2023-10-26T01:38:21.335571+0000","last_clean":"2023-10-26T01:38:21.335571+0000","last_became_active":"2023-10-26T01:38:20.320400+0000","last_became_peered":"2023-10-26T01:38:20.320400+0000","last_unstale":"2023-10-26T01:38:21.335571+0000","last_undegraded":"2023-10-26T01:38:21.335571+0000","last_fullsized":"2023-10-26T01:38:21.335571+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:31:15.331694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.357408+0000","last_change":"2023-10-26T01:38:21.320573+0000","last_active":"2023-10-26T01:38:22.357408+0000","last_peered":"2023-10-26T01:38:22.357408+0000","last_clean":"2023-10-26T01:38:22.357408+0000","last_became_active":"2023-10-26T01:38:21.319770+0000","last_became_peered":"2023-10-26T01:38:21.319770+0000","last_unstale":"2023-10-26T01:38:22.357408+0000","last_undegraded":"2023-10-26T01:38:22.357408+0000","last_fullsized":"2023-10-26T01:38:22.357408+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:30:41.148919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":12,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.294385+0000","last_change":"2023-10-26T01:38:20.317888+0000","last_active":"2023-10-26T01:38:21.294385+0000","last_peered":"2023-10-26T01:38:21.294385+0000","last_clean":"2023-10-26T01:38:21.294385+0000","last_became_active":"2023-10-26T01:38:20.317130+0000","last_became_peered":"2023-10-26T01:38:20.317130+0000","last_unstale":"2023-10-26T01:38:21.294385+0000","last_undegraded":"2023-10-26T01:38:21.294385+0000","last_fullsized":"2023-10-26T01:38:21.294385+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:27:41.686982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.356245+0000","last_change":"2023-10-26T01:38:21.317074+0000","last_active":"2023-10-26T01:38:22.356245+0000","last_peered":"2023-10-26T01:38:22.356245+0000","last_clean":"2023-10-26T01:38:22.356245+0000","last_became_active":"2023-10-26T01:38:21.316665+0000","last_became_peered":"2023-10-26T01:38:21.316665+0000","last_unstale":"2023-10-26T01:38:22.356245+0000","last_undegraded":"2023-10-26T01:38:22.356245+0000","last_fullsized":"2023-10-26T01:38:22.356245+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:42:14.392665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":12,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.307295+0000","last_change":"2023-10-26T01:38:20.314210+0000","last_active":"2023-10-26T01:38:21.307295+0000","last_peered":"2023-10-26T01:38:21.307295+0000","last_clean":"2023-10-26T01:38:21.307295+0000","last_became_active":"2023-10-26T01:38:20.313976+0000","last_became_peered":"2023-10-26T01:38:20.313976+0000","last_unstale":"2023-10-26T01:38:21.307295+0000","last_undegraded":"2023-10-26T01:38:21.307295+0000","last_fullsized":"2023-10-26T01:38:21.307295+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:24:55.184951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.356077+0000","last_change":"2023-10-26T01:38:21.313827+0000","last_active":"2023-10-26T01:38:22.356077+0000","last_peered":"2023-10-26T01:38:22.356077+0000","last_clean":"2023-10-26T01:38:22.356077+0000","last_became_active":"2023-10-26T01:38:21.310630+0000","last_became_peered":"2023-10-26T01:38:21.310630+0000","last_unstale":"2023-10-26T01:38:22.356077+0000","last_undegraded":"2023-10-26T01:38:22.356077+0000","last_fullsized":"2023-10-26T01:38:22.356077+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:28:43.891836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":12,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.295375+0000","last_change":"2023-10-26T01:38:20.325020+0000","last_active":"2023-10-26T01:38:21.295375+0000","last_peered":"2023-10-26T01:38:21.295375+0000","last_clean":"2023-10-26T01:38:21.295375+0000","last_became_active":"2023-10-26T01:38:20.323713+0000","last_became_peered":"2023-10-26T01:38:20.323713+0000","last_unstale":"2023-10-26T01:38:21.295375+0000","last_undegraded":"2023-10-26T01:38:21.295375+0000","last_fullsized":"2023-10-26T01:38:21.295375+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:23:36.805508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.357896+0000","last_change":"2023-10-26T01:38:21.327094+0000","last_active":"2023-10-26T01:38:22.357896+0000","last_peered":"2023-10-26T01:38:22.357896+0000","last_clean":"2023-10-26T01:38:22.357896+0000","last_became_active":"2023-10-26T01:38:21.326783+0000","last_became_peered":"2023-10-26T01:38:21.326783+0000","last_unstale":"2023-10-26T01:38:22.357896+0000","last_undegraded":"2023-10-26T01:38:22.357896+0000","last_fullsized":"2023-10-26T01:38:22.357896+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:48:01.026075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.293962+0000","last_change":"2023-10-26T01:38:20.322090+0000","last_active":"2023-10-26T01:38:21.293962+0000","last_peered":"2023-10-26T01:38:21.293962+0000","last_clean":"2023-10-26T01:38:21.293962+0000","last_became_active":"2023-10-26T01:38:20.321451+0000","last_became_peered":"2023-10-26T01:38:20.321451+0000","last_unstale":"2023-10-26T01:38:21.293962+0000","last_undegraded":"2023-10-26T01:38:21.293962+0000","last_fullsized":"2023-10-26T01:38:21.293962+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:45:01.083199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.302056+0000","last_change":"2023-10-26T01:38:21.301523+0000","last_active":"2023-10-26T01:38:21.302056+0000","last_peered":"2023-10-26T01:38:21.302056+0000","last_clean":"2023-10-26T01:38:21.302056+0000","last_became_active":"2023-10-26T01:38:21.300901+0000","last_became_peered":"2023-10-26T01:38:21.300901+0000","last_unstale":"2023-10-26T01:38:21.302056+0000","last_undegraded":"2023-10-26T01:38:21.302056+0000","last_fullsized":"2023-10-26T01:38:21.302056+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:21:55.273677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"133'3","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.337431+0000","last_change":"2023-10-26T01:38:20.319263+0000","last_active":"2023-10-26T01:38:21.337431+0000","last_peered":"2023-10-26T01:38:21.337431+0000","last_clean":"2023-10-26T01:38:21.337431+0000","last_became_active":"2023-10-26T01:38:20.319022+0000","last_became_peered":"2023-10-26T01:38:20.319022+0000","last_unstale":"2023-10-26T01:38:21.337431+0000","last_undegraded":"2023-10-26T01:38:21.337431+0000","last_fullsized":"2023-10-26T01:38:21.337431+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:57:33.830348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.356119+0000","last_change":"2023-10-26T01:38:21.321635+0000","last_active":"2023-10-26T01:38:22.356119+0000","last_peered":"2023-10-26T01:38:22.356119+0000","last_clean":"2023-10-26T01:38:22.356119+0000","last_became_active":"2023-10-26T01:38:21.321297+0000","last_became_peered":"2023-10-26T01:38:21.321297+0000","last_unstale":"2023-10-26T01:38:22.356119+0000","last_undegraded":"2023-10-26T01:38:22.356119+0000","last_fullsized":"2023-10-26T01:38:22.356119+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:36:20.377483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"133'6","reported_seq":18,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.340461+0000","last_change":"2023-10-26T01:38:20.314532+0000","last_active":"2023-10-26T01:38:21.340461+0000","last_peered":"2023-10-26T01:38:21.340461+0000","last_clean":"2023-10-26T01:38:21.340461+0000","last_became_active":"2023-10-26T01:38:20.314073+0000","last_became_peered":"2023-10-26T01:38:20.314073+0000","last_unstale":"2023-10-26T01:38:21.340461+0000","last_undegraded":"2023-10-26T01:38:21.340461+0000","last_fullsized":"2023-10-26T01:38:21.340461+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:47:06.997633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.310604+0000","last_change":"2023-10-26T01:38:21.310272+0000","last_active":"2023-10-26T01:38:21.310604+0000","last_peered":"2023-10-26T01:38:21.310604+0000","last_clean":"2023-10-26T01:38:21.310604+0000","last_became_active":"2023-10-26T01:38:21.309297+0000","last_became_peered":"2023-10-26T01:38:21.309297+0000","last_unstale":"2023-10-26T01:38:21.310604+0000","last_undegraded":"2023-10-26T01:38:21.310604+0000","last_fullsized":"2023-10-26T01:38:21.310604+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:07:03.778722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"133'2","reported_seq":14,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.336757+0000","last_change":"2023-10-26T01:38:20.323897+0000","last_active":"2023-10-26T01:38:21.336757+0000","last_peered":"2023-10-26T01:38:21.336757+0000","last_clean":"2023-10-26T01:38:21.336757+0000","last_became_active":"2023-10-26T01:38:20.323527+0000","last_became_peered":"2023-10-26T01:38:20.323527+0000","last_unstale":"2023-10-26T01:38:21.336757+0000","last_undegraded":"2023-10-26T01:38:21.336757+0000","last_fullsized":"2023-10-26T01:38:21.336757+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:54:02.310203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.317765+0000","last_change":"2023-10-26T01:38:21.317158+0000","last_active":"2023-10-26T01:38:21.317765+0000","last_peered":"2023-10-26T01:38:21.317765+0000","last_clean":"2023-10-26T01:38:21.317765+0000","last_became_active":"2023-10-26T01:38:21.316782+0000","last_became_peered":"2023-10-26T01:38:21.316782+0000","last_unstale":"2023-10-26T01:38:21.317765+0000","last_undegraded":"2023-10-26T01:38:21.317765+0000","last_fullsized":"2023-10-26T01:38:21.317765+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:38:50.874604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":12,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.305214+0000","last_change":"2023-10-26T01:38:20.323991+0000","last_active":"2023-10-26T01:38:21.305214+0000","last_peered":"2023-10-26T01:38:21.305214+0000","last_clean":"2023-10-26T01:38:21.305214+0000","last_became_active":"2023-10-26T01:38:20.323242+0000","last_became_peered":"2023-10-26T01:38:20.323242+0000","last_unstale":"2023-10-26T01:38:21.305214+0000","last_undegraded":"2023-10-26T01:38:21.305214+0000","last_fullsized":"2023-10-26T01:38:21.305214+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:08:50.823275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.309119+0000","last_change":"2023-10-26T01:38:21.308916+0000","last_active":"2023-10-26T01:38:21.309119+0000","last_peered":"2023-10-26T01:38:21.309119+0000","last_clean":"2023-10-26T01:38:21.309119+0000","last_became_active":"2023-10-26T01:38:21.308364+0000","last_became_peered":"2023-10-26T01:38:21.308364+0000","last_unstale":"2023-10-26T01:38:21.309119+0000","last_undegraded":"2023-10-26T01:38:21.309119+0000","last_fullsized":"2023-10-26T01:38:21.309119+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:18:02.522210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"133'8","reported_seq":20,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.340109+0000","last_change":"2023-10-26T01:38:20.322830+0000","last_active":"2023-10-26T01:38:21.340109+0000","last_peered":"2023-10-26T01:38:21.340109+0000","last_clean":"2023-10-26T01:38:21.340109+0000","last_became_active":"2023-10-26T01:38:20.318642+0000","last_became_peered":"2023-10-26T01:38:20.318642+0000","last_unstale":"2023-10-26T01:38:21.340109+0000","last_undegraded":"2023-10-26T01:38:21.340109+0000","last_fullsized":"2023-10-26T01:38:21.340109+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:36:33.502627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.358107+0000","last_change":"2023-10-26T01:38:21.322681+0000","last_active":"2023-10-26T01:38:22.358107+0000","last_peered":"2023-10-26T01:38:22.358107+0000","last_clean":"2023-10-26T01:38:22.358107+0000","last_became_active":"2023-10-26T01:38:21.322406+0000","last_became_peered":"2023-10-26T01:38:21.322406+0000","last_unstale":"2023-10-26T01:38:22.358107+0000","last_undegraded":"2023-10-26T01:38:22.358107+0000","last_fullsized":"2023-10-26T01:38:22.358107+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:11:34.893562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"133'1","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.335822+0000","last_change":"2023-10-26T01:38:20.322530+0000","last_active":"2023-10-26T01:38:21.335822+0000","last_peered":"2023-10-26T01:38:21.335822+0000","last_clean":"2023-10-26T01:38:21.335822+0000","last_became_active":"2023-10-26T01:38:20.322312+0000","last_became_peered":"2023-10-26T01:38:20.322312+0000","last_unstale":"2023-10-26T01:38:21.335822+0000","last_undegraded":"2023-10-26T01:38:21.335822+0000","last_fullsized":"2023-10-26T01:38:21.335822+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:53:09.791518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.356325+0000","last_change":"2023-10-26T01:38:21.319104+0000","last_active":"2023-10-26T01:38:22.356325+0000","last_peered":"2023-10-26T01:38:22.356325+0000","last_clean":"2023-10-26T01:38:22.356325+0000","last_became_active":"2023-10-26T01:38:21.318760+0000","last_became_peered":"2023-10-26T01:38:21.318760+0000","last_unstale":"2023-10-26T01:38:22.356325+0000","last_undegraded":"2023-10-26T01:38:22.356325+0000","last_fullsized":"2023-10-26T01:38:22.356325+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:16:32.278743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"133'2","reported_seq":14,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.337172+0000","last_change":"2023-10-26T01:38:20.318723+0000","last_active":"2023-10-26T01:38:21.337172+0000","last_peered":"2023-10-26T01:38:21.337172+0000","last_clean":"2023-10-26T01:38:21.337172+0000","last_became_active":"2023-10-26T01:38:20.317546+0000","last_became_peered":"2023-10-26T01:38:20.317546+0000","last_unstale":"2023-10-26T01:38:21.337172+0000","last_undegraded":"2023-10-26T01:38:21.337172+0000","last_fullsized":"2023-10-26T01:38:21.337172+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:16:31.264756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.299270+0000","last_change":"2023-10-26T01:38:21.320514+0000","last_active":"2023-10-26T01:38:22.299270+0000","last_peered":"2023-10-26T01:38:22.299270+0000","last_clean":"2023-10-26T01:38:22.299270+0000","last_became_active":"2023-10-26T01:38:21.320080+0000","last_became_peered":"2023-10-26T01:38:21.320080+0000","last_unstale":"2023-10-26T01:38:22.299270+0000","last_undegraded":"2023-10-26T01:38:22.299270+0000","last_fullsized":"2023-10-26T01:38:22.299270+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T01:40:34.431360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"133'1","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.335116+0000","last_change":"2023-10-26T01:38:20.323687+0000","last_active":"2023-10-26T01:38:21.335116+0000","last_peered":"2023-10-26T01:38:21.335116+0000","last_clean":"2023-10-26T01:38:21.335116+0000","last_became_active":"2023-10-26T01:38:20.322524+0000","last_became_peered":"2023-10-26T01:38:20.322524+0000","last_unstale":"2023-10-26T01:38:21.335116+0000","last_undegraded":"2023-10-26T01:38:21.335116+0000","last_fullsized":"2023-10-26T01:38:21.335116+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:00:59.371192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.298852+0000","last_change":"2023-10-26T01:38:21.317249+0000","last_active":"2023-10-26T01:38:22.298852+0000","last_peered":"2023-10-26T01:38:22.298852+0000","last_clean":"2023-10-26T01:38:22.298852+0000","last_became_active":"2023-10-26T01:38:21.315753+0000","last_became_peered":"2023-10-26T01:38:21.315753+0000","last_unstale":"2023-10-26T01:38:22.298852+0000","last_undegraded":"2023-10-26T01:38:22.298852+0000","last_fullsized":"2023-10-26T01:38:22.298852+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:19:17.527922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":12,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.304169+0000","last_change":"2023-10-26T01:38:20.322266+0000","last_active":"2023-10-26T01:38:21.304169+0000","last_peered":"2023-10-26T01:38:21.304169+0000","last_clean":"2023-10-26T01:38:21.304169+0000","last_became_active":"2023-10-26T01:38:20.315231+0000","last_became_peered":"2023-10-26T01:38:20.315231+0000","last_unstale":"2023-10-26T01:38:21.304169+0000","last_undegraded":"2023-10-26T01:38:21.304169+0000","last_fullsized":"2023-10-26T01:38:21.304169+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:27:02.475699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.297531+0000","last_change":"2023-10-26T01:38:21.323986+0000","last_active":"2023-10-26T01:38:22.297531+0000","last_peered":"2023-10-26T01:38:22.297531+0000","last_clean":"2023-10-26T01:38:22.297531+0000","last_became_active":"2023-10-26T01:38:21.322360+0000","last_became_peered":"2023-10-26T01:38:21.322360+0000","last_unstale":"2023-10-26T01:38:22.297531+0000","last_undegraded":"2023-10-26T01:38:22.297531+0000","last_fullsized":"2023-10-26T01:38:22.297531+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:29:52.799943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"133'2","reported_seq":14,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.338836+0000","last_change":"2023-10-26T01:38:20.319975+0000","last_active":"2023-10-26T01:38:21.338836+0000","last_peered":"2023-10-26T01:38:21.338836+0000","last_clean":"2023-10-26T01:38:21.338836+0000","last_became_active":"2023-10-26T01:38:20.319754+0000","last_became_peered":"2023-10-26T01:38:20.319754+0000","last_unstale":"2023-10-26T01:38:21.338836+0000","last_undegraded":"2023-10-26T01:38:21.338836+0000","last_fullsized":"2023-10-26T01:38:21.338836+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:31:09.167929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.358204+0000","last_change":"2023-10-26T01:38:21.324252+0000","last_active":"2023-10-26T01:38:22.358204+0000","last_peered":"2023-10-26T01:38:22.358204+0000","last_clean":"2023-10-26T01:38:22.358204+0000","last_became_active":"2023-10-26T01:38:21.323975+0000","last_became_peered":"2023-10-26T01:38:21.323975+0000","last_unstale":"2023-10-26T01:38:22.358204+0000","last_undegraded":"2023-10-26T01:38:22.358204+0000","last_fullsized":"2023-10-26T01:38:22.358204+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:26:51.819282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":12,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.293744+0000","last_change":"2023-10-26T01:38:20.323205+0000","last_active":"2023-10-26T01:38:21.293744+0000","last_peered":"2023-10-26T01:38:21.293744+0000","last_clean":"2023-10-26T01:38:21.293744+0000","last_became_active":"2023-10-26T01:38:20.322992+0000","last_became_peered":"2023-10-26T01:38:20.322992+0000","last_unstale":"2023-10-26T01:38:21.293744+0000","last_undegraded":"2023-10-26T01:38:21.293744+0000","last_fullsized":"2023-10-26T01:38:21.293744+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:54:02.310203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.299167+0000","last_change":"2023-10-26T01:38:21.316303+0000","last_active":"2023-10-26T01:38:22.299167+0000","last_peered":"2023-10-26T01:38:22.299167+0000","last_clean":"2023-10-26T01:38:22.299167+0000","last_became_active":"2023-10-26T01:38:21.315023+0000","last_became_peered":"2023-10-26T01:38:21.315023+0000","last_unstale":"2023-10-26T01:38:22.299167+0000","last_undegraded":"2023-10-26T01:38:22.299167+0000","last_fullsized":"2023-10-26T01:38:22.299167+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T01:55:12.014852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":12,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.303952+0000","last_change":"2023-10-26T01:38:20.320536+0000","last_active":"2023-10-26T01:38:21.303952+0000","last_peered":"2023-10-26T01:38:21.303952+0000","last_clean":"2023-10-26T01:38:21.303952+0000","last_became_active":"2023-10-26T01:38:20.320277+0000","last_became_peered":"2023-10-26T01:38:20.320277+0000","last_unstale":"2023-10-26T01:38:21.303952+0000","last_undegraded":"2023-10-26T01:38:21.303952+0000","last_fullsized":"2023-10-26T01:38:21.303952+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:36:33.502627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.322832+0000","last_change":"2023-10-26T01:38:21.322582+0000","last_active":"2023-10-26T01:38:21.322832+0000","last_peered":"2023-10-26T01:38:21.322832+0000","last_clean":"2023-10-26T01:38:21.322832+0000","last_became_active":"2023-10-26T01:38:21.321971+0000","last_became_peered":"2023-10-26T01:38:21.321971+0000","last_unstale":"2023-10-26T01:38:21.322832+0000","last_undegraded":"2023-10-26T01:38:21.322832+0000","last_fullsized":"2023-10-26T01:38:21.322832+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:44:44.680316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":12,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.307293+0000","last_change":"2023-10-26T01:38:20.324450+0000","last_active":"2023-10-26T01:38:21.307293+0000","last_peered":"2023-10-26T01:38:21.307293+0000","last_clean":"2023-10-26T01:38:21.307293+0000","last_became_active":"2023-10-26T01:38:20.324194+0000","last_became_peered":"2023-10-26T01:38:20.324194+0000","last_unstale":"2023-10-26T01:38:21.307293+0000","last_undegraded":"2023-10-26T01:38:21.307293+0000","last_fullsized":"2023-10-26T01:38:21.307293+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:30:19.872449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.357340+0000","last_change":"2023-10-26T01:38:21.320203+0000","last_active":"2023-10-26T01:38:22.357340+0000","last_peered":"2023-10-26T01:38:22.357340+0000","last_clean":"2023-10-26T01:38:22.357340+0000","last_became_active":"2023-10-26T01:38:21.319553+0000","last_became_peered":"2023-10-26T01:38:21.319553+0000","last_unstale":"2023-10-26T01:38:22.357340+0000","last_undegraded":"2023-10-26T01:38:22.357340+0000","last_fullsized":"2023-10-26T01:38:22.357340+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T01:55:12.014852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"133'2","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.338835+0000","last_change":"2023-10-26T01:38:20.315954+0000","last_active":"2023-10-26T01:38:21.338835+0000","last_peered":"2023-10-26T01:38:21.338835+0000","last_clean":"2023-10-26T01:38:21.338835+0000","last_became_active":"2023-10-26T01:38:20.315727+0000","last_became_peered":"2023-10-26T01:38:20.315727+0000","last_unstale":"2023-10-26T01:38:21.338835+0000","last_undegraded":"2023-10-26T01:38:21.338835+0000","last_fullsized":"2023-10-26T01:38:21.338835+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:01:47.759723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.358948+0000","last_change":"2023-10-26T01:38:21.319574+0000","last_active":"2023-10-26T01:38:22.358948+0000","last_peered":"2023-10-26T01:38:22.358948+0000","last_clean":"2023-10-26T01:38:22.358948+0000","last_became_active":"2023-10-26T01:38:21.317481+0000","last_became_peered":"2023-10-26T01:38:21.317481+0000","last_unstale":"2023-10-26T01:38:22.358948+0000","last_undegraded":"2023-10-26T01:38:22.358948+0000","last_fullsized":"2023-10-26T01:38:22.358948+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:08:09.599874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"134'1","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.357908+0000","last_change":"2023-10-26T01:38:20.325614+0000","last_active":"2023-10-26T01:38:22.357908+0000","last_peered":"2023-10-26T01:38:22.357908+0000","last_clean":"2023-10-26T01:38:22.357908+0000","last_became_active":"2023-10-26T01:38:20.325288+0000","last_became_peered":"2023-10-26T01:38:20.325288+0000","last_unstale":"2023-10-26T01:38:22.357908+0000","last_undegraded":"2023-10-26T01:38:22.357908+0000","last_fullsized":"2023-10-26T01:38:22.357908+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:51:41.698100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.357610+0000","last_change":"2023-10-26T01:38:21.320766+0000","last_active":"2023-10-26T01:38:22.357610+0000","last_peered":"2023-10-26T01:38:22.357610+0000","last_clean":"2023-10-26T01:38:22.357610+0000","last_became_active":"2023-10-26T01:38:21.320148+0000","last_became_peered":"2023-10-26T01:38:21.320148+0000","last_unstale":"2023-10-26T01:38:22.357610+0000","last_undegraded":"2023-10-26T01:38:22.357610+0000","last_fullsized":"2023-10-26T01:38:22.357610+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:20:14.948944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":12,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.312905+0000","last_change":"2023-10-26T01:38:20.320712+0000","last_active":"2023-10-26T01:38:21.312905+0000","last_peered":"2023-10-26T01:38:21.312905+0000","last_clean":"2023-10-26T01:38:21.312905+0000","last_became_active":"2023-10-26T01:38:20.319884+0000","last_became_peered":"2023-10-26T01:38:20.319884+0000","last_unstale":"2023-10-26T01:38:21.312905+0000","last_undegraded":"2023-10-26T01:38:21.312905+0000","last_fullsized":"2023-10-26T01:38:21.312905+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:37:52.841387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.356773+0000","last_change":"2023-10-26T01:38:21.320274+0000","last_active":"2023-10-26T01:38:22.356773+0000","last_peered":"2023-10-26T01:38:22.356773+0000","last_clean":"2023-10-26T01:38:22.356773+0000","last_became_active":"2023-10-26T01:38:21.319994+0000","last_became_peered":"2023-10-26T01:38:21.319994+0000","last_unstale":"2023-10-26T01:38:22.356773+0000","last_undegraded":"2023-10-26T01:38:22.356773+0000","last_fullsized":"2023-10-26T01:38:22.356773+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:19:17.527922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.294223+0000","last_change":"2023-10-26T01:38:20.319596+0000","last_active":"2023-10-26T01:38:21.294223+0000","last_peered":"2023-10-26T01:38:21.294223+0000","last_clean":"2023-10-26T01:38:21.294223+0000","last_became_active":"2023-10-26T01:38:20.319376+0000","last_became_peered":"2023-10-26T01:38:20.319376+0000","last_unstale":"2023-10-26T01:38:21.294223+0000","last_undegraded":"2023-10-26T01:38:21.294223+0000","last_fullsized":"2023-10-26T01:38:21.294223+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:59:26.990814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.298020+0000","last_change":"2023-10-26T01:38:21.307155+0000","last_active":"2023-10-26T01:38:22.298020+0000","last_peered":"2023-10-26T01:38:22.298020+0000","last_clean":"2023-10-26T01:38:22.298020+0000","last_became_active":"2023-10-26T01:38:21.306473+0000","last_became_peered":"2023-10-26T01:38:21.306473+0000","last_unstale":"2023-10-26T01:38:22.298020+0000","last_undegraded":"2023-10-26T01:38:22.298020+0000","last_fullsized":"2023-10-26T01:38:22.298020+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T01:55:12.014852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":12,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.301190+0000","last_change":"2023-10-26T01:38:20.324083+0000","last_active":"2023-10-26T01:38:21.301190+0000","last_peered":"2023-10-26T01:38:21.301190+0000","last_clean":"2023-10-26T01:38:21.301190+0000","last_became_active":"2023-10-26T01:38:20.320737+0000","last_became_peered":"2023-10-26T01:38:20.320737+0000","last_unstale":"2023-10-26T01:38:21.301190+0000","last_undegraded":"2023-10-26T01:38:21.301190+0000","last_fullsized":"2023-10-26T01:38:21.301190+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:31:09.167929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.297522+0000","last_change":"2023-10-26T01:38:21.299637+0000","last_active":"2023-10-26T01:38:22.297522+0000","last_peered":"2023-10-26T01:38:22.297522+0000","last_clean":"2023-10-26T01:38:22.297522+0000","last_became_active":"2023-10-26T01:38:21.298931+0000","last_became_peered":"2023-10-26T01:38:21.298931+0000","last_unstale":"2023-10-26T01:38:22.297522+0000","last_undegraded":"2023-10-26T01:38:22.297522+0000","last_fullsized":"2023-10-26T01:38:22.297522+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:19:17.527922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":12,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.299217+0000","last_change":"2023-10-26T01:38:20.321433+0000","last_active":"2023-10-26T01:38:21.299217+0000","last_peered":"2023-10-26T01:38:21.299217+0000","last_clean":"2023-10-26T01:38:21.299217+0000","last_became_active":"2023-10-26T01:38:20.321136+0000","last_became_peered":"2023-10-26T01:38:20.321136+0000","last_unstale":"2023-10-26T01:38:21.299217+0000","last_undegraded":"2023-10-26T01:38:21.299217+0000","last_fullsized":"2023-10-26T01:38:21.299217+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:00:59.371192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.297193+0000","last_change":"2023-10-26T01:38:21.324339+0000","last_active":"2023-10-26T01:38:22.297193+0000","last_peered":"2023-10-26T01:38:22.297193+0000","last_clean":"2023-10-26T01:38:22.297193+0000","last_became_active":"2023-10-26T01:38:21.323704+0000","last_became_peered":"2023-10-26T01:38:21.323704+0000","last_unstale":"2023-10-26T01:38:22.297193+0000","last_undegraded":"2023-10-26T01:38:22.297193+0000","last_fullsized":"2023-10-26T01:38:22.297193+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:51:42.712087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":12,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.294774+0000","last_change":"2023-10-26T01:38:20.324901+0000","last_active":"2023-10-26T01:38:21.294774+0000","last_peered":"2023-10-26T01:38:21.294774+0000","last_clean":"2023-10-26T01:38:21.294774+0000","last_became_active":"2023-10-26T01:38:20.324441+0000","last_became_peered":"2023-10-26T01:38:20.324441+0000","last_unstale":"2023-10-26T01:38:21.294774+0000","last_undegraded":"2023-10-26T01:38:21.294774+0000","last_fullsized":"2023-10-26T01:38:21.294774+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:02:55.480456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.357911+0000","last_change":"2023-10-26T01:38:21.318362+0000","last_active":"2023-10-26T01:38:22.357911+0000","last_peered":"2023-10-26T01:38:22.357911+0000","last_clean":"2023-10-26T01:38:22.357911+0000","last_became_active":"2023-10-26T01:38:21.317684+0000","last_became_peered":"2023-10-26T01:38:21.317684+0000","last_unstale":"2023-10-26T01:38:22.357911+0000","last_undegraded":"2023-10-26T01:38:22.357911+0000","last_fullsized":"2023-10-26T01:38:22.357911+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:42:43.138486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":12,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.313106+0000","last_change":"2023-10-26T01:38:20.320871+0000","last_active":"2023-10-26T01:38:21.313106+0000","last_peered":"2023-10-26T01:38:21.313106+0000","last_clean":"2023-10-26T01:38:21.313106+0000","last_became_active":"2023-10-26T01:38:20.320224+0000","last_became_peered":"2023-10-26T01:38:20.320224+0000","last_unstale":"2023-10-26T01:38:21.313106+0000","last_undegraded":"2023-10-26T01:38:21.313106+0000","last_fullsized":"2023-10-26T01:38:21.313106+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:30:07.282370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.322813+0000","last_change":"2023-10-26T01:38:21.321771+0000","last_active":"2023-10-26T01:38:21.322813+0000","last_peered":"2023-10-26T01:38:21.322813+0000","last_clean":"2023-10-26T01:38:21.322813+0000","last_became_active":"2023-10-26T01:38:21.321349+0000","last_became_peered":"2023-10-26T01:38:21.321349+0000","last_unstale":"2023-10-26T01:38:21.322813+0000","last_undegraded":"2023-10-26T01:38:21.322813+0000","last_fullsized":"2023-10-26T01:38:21.322813+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:22:49.920143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":12,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.303830+0000","last_change":"2023-10-26T01:38:20.321951+0000","last_active":"2023-10-26T01:38:21.303830+0000","last_peered":"2023-10-26T01:38:21.303830+0000","last_clean":"2023-10-26T01:38:21.303830+0000","last_became_active":"2023-10-26T01:38:20.310341+0000","last_became_peered":"2023-10-26T01:38:20.310341+0000","last_unstale":"2023-10-26T01:38:21.303830+0000","last_undegraded":"2023-10-26T01:38:21.303830+0000","last_fullsized":"2023-10-26T01:38:21.303830+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:11:33.879576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":278,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.315091+0000","last_change":"2023-10-26T01:30:34.883484+0000","last_active":"2023-10-26T01:38:21.315091+0000","last_peered":"2023-10-26T01:38:21.315091+0000","last_clean":"2023-10-26T01:38:21.315091+0000","last_became_active":"2023-10-26T01:30:34.883071+0000","last_became_peered":"2023-10-26T01:30:34.883071+0000","last_unstale":"2023-10-26T01:38:21.315091+0000","last_undegraded":"2023-10-26T01:38:21.315091+0000","last_fullsized":"2023-10-26T01:38:21.315091+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:32.890096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:32.890096+0000","last_clean_scrub_stamp":"2023-10-26T01:30:32.890096+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:26:31.879452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":14,"seq":60129542241,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":719,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194592,"statfs":{"total":200000000,"available":199262922,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.79200000000000004,"15min":0.78600000000000003},"min":{"1min":0.44900000000000001,"5min":0.35599999999999998,"15min":0.33000000000000002},"max":{"1min":1.6319999999999999,"5min":1.6319999999999999,"15min":2.2549999999999999},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.876,"15min":0.88800000000000001},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.236,"5min":1.5800000000000001,"15min":1.8200000000000001},"last":0.94099999999999995}]},{"osd":1,"last update":"Thu Oct 26 01:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.875,"15min":0.93700000000000006},"min":{"1min":0.40699999999999997,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.3560000000000001,"5min":1.444,"15min":4.4340000000000002},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.97599999999999998,"15min":0.997},"min":{"1min":0.51500000000000001,"5min":0.46700000000000003,"15min":0.374},"max":{"1min":1.4390000000000001,"5min":1.538,"15min":2.3330000000000002},"last":0.755}]},{"osd":2,"last update":"Thu Oct 26 01:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":1.032,"15min":1.075},"min":{"1min":0.53800000000000003,"5min":0.40999999999999998,"15min":0.39900000000000002},"max":{"1min":1.643,"5min":1.927,"15min":4.2800000000000002},"last":1.2689999999999999},{"interface":"front","average":{"1min":1.032,"5min":1.1279999999999999,"15min":1.091},"min":{"1min":0.50900000000000001,"5min":0.495,"15min":0.49399999999999999},"max":{"1min":1.607,"5min":1.8680000000000001,"15min":2.302},"last":1.3320000000000001}]},{"osd":4,"last update":"Thu Oct 26 01:37:49 2023","interfaces":[{"interface":"back","average":{"1min":1.175,"5min":1.2070000000000001,"15min":1.167},"min":{"1min":0.755,"5min":0.755,"15min":0.66400000000000003},"max":{"1min":1.536,"5min":1.7150000000000001,"15min":2.1070000000000002},"last":1.008},{"interface":"front","average":{"1min":1.252,"5min":1.3400000000000001,"15min":1.389},"min":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.79900000000000004},"max":{"1min":1.5209999999999999,"5min":2.4239999999999999,"15min":4.8250000000000002},"last":1.5740000000000001}]},{"osd":5,"last update":"Thu Oct 26 01:37:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.282,"15min":1.333},"min":{"1min":0.79300000000000004,"5min":0.76400000000000001,"15min":0.72499999999999998},"max":{"1min":1.639,"5min":2.3380000000000001,"15min":4.9080000000000004},"last":1.448},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.28,"15min":1.276},"min":{"1min":0.90300000000000002,"5min":0.79900000000000004,"15min":0.79600000000000004},"max":{"1min":1.722,"5min":2.0310000000000001,"15min":2.8980000000000001},"last":1.1120000000000001}]},{"osd":6,"last update":"Thu Oct 26 01:37:49 2023","interfaces":[{"interface":"back","average":{"1min":1.278,"5min":1.355,"15min":1.331},"min":{"1min":0.74199999999999999,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.5580000000000001,"5min":2.1499999999999999,"15min":2.8530000000000002},"last":1.2270000000000001},{"interface":"front","average":{"1min":1.206,"5min":1.268,"15min":1.268},"min":{"1min":0.81699999999999995,"5min":0.81499999999999995,"15min":0.73099999999999998},"max":{"1min":1.7410000000000001,"5min":2.048,"15min":2.181},"last":1.0820000000000001}]},{"osd":7,"last update":"Thu Oct 26 01:37:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.3300000000000001,"15min":1.296},"min":{"1min":0.85299999999999998,"5min":0.84599999999999997,"15min":0.71999999999999997},"max":{"1min":1.8129999999999999,"5min":1.958,"15min":2.0680000000000001},"last":1.046},{"interface":"front","average":{"1min":1.2889999999999999,"5min":1.3919999999999999,"15min":1.415},"min":{"1min":0.84199999999999997,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.647,"5min":2.2370000000000001,"15min":4.8730000000000002},"last":1.6399999999999999}]}]},{"osd":7,"up_from":13,"seq":55834574945,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1301,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194011,"statfs":{"total":200000000,"available":198667320,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:37:49 2023","interfaces":[{"interface":"back","average":{"1min":1.0920000000000001,"5min":1.139,"15min":1.093},"min":{"1min":0.81799999999999995,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.7,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.99099999999999999,"5min":1.115,"15min":1.081},"min":{"1min":0.70599999999999996,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.46,"5min":2.6779999999999999,"15min":2.6779999999999999},"last":1.1020000000000001}]},{"osd":1,"last update":"Thu Oct 26 01:37:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.1870000000000001,"15min":1.0960000000000001},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65200000000000002},"max":{"1min":1.5549999999999999,"5min":2.8260000000000001,"15min":2.8260000000000001},"last":0.82899999999999996},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.21,"15min":1.194},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.61199999999999999},"max":{"1min":1.7869999999999999,"5min":3.2730000000000001,"15min":3.2730000000000001},"last":0.93899999999999995}]},{"osd":2,"last update":"Thu Oct 26 01:37:49 2023","interfaces":[{"interface":"back","average":{"1min":1.143,"5min":1.179,"15min":1.1180000000000001},"min":{"1min":0.79100000000000004,"5min":0.65700000000000003,"15min":0.58499999999999996},"max":{"1min":1.667,"5min":2.218,"15min":2.218},"last":0.873},{"interface":"front","average":{"1min":1.2709999999999999,"5min":1.2989999999999999,"15min":1.24},"min":{"1min":0.89200000000000002,"5min":0.79400000000000004,"15min":0.73099999999999998},"max":{"1min":1.786,"5min":3.0510000000000002,"15min":3.0510000000000002},"last":1.1459999999999999}]},{"osd":3,"last update":"Thu Oct 26 01:37:49 2023","interfaces":[{"interface":"back","average":{"1min":1.264,"5min":1.284,"15min":1.214},"min":{"1min":0.93500000000000005,"5min":0.79000000000000004,"15min":0.72099999999999997},"max":{"1min":1.5389999999999999,"5min":1.9830000000000001,"15min":1.9830000000000001},"last":0.79500000000000004},{"interface":"front","average":{"1min":1.296,"5min":1.407,"15min":1.373},"min":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.65700000000000003},"max":{"1min":1.8939999999999999,"5min":5.5039999999999996,"15min":5.5039999999999996},"last":1.1839999999999999}]},{"osd":4,"last update":"Thu Oct 26 01:37:49 2023","interfaces":[{"interface":"back","average":{"1min":1.4350000000000001,"5min":1.373,"15min":1.3520000000000001},"min":{"1min":1.0589999999999999,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.7450000000000001,"5min":2.77,"15min":2.77},"last":1.077},{"interface":"front","average":{"1min":1.456,"5min":1.429,"15min":1.399},"min":{"1min":0.92700000000000005,"5min":0.90400000000000003,"15min":0.496},"max":{"1min":1.95,"5min":3.1419999999999999,"15min":3.1419999999999999},"last":0.98599999999999999}]},{"osd":5,"last update":"Thu Oct 26 01:37:49 2023","interfaces":[{"interface":"back","average":{"1min":1.498,"5min":1.488,"15min":1.4259999999999999},"min":{"1min":0.91700000000000004,"5min":0.65600000000000003,"15min":0.441},"max":{"1min":1.9890000000000001,"5min":5.734,"15min":5.734},"last":1.054},{"interface":"front","average":{"1min":1.4970000000000001,"5min":1.5269999999999999,"15min":1.478},"min":{"1min":1.1739999999999999,"5min":0.96099999999999997,"15min":0.51900000000000002},"max":{"1min":1.8799999999999999,"5min":2.8980000000000001,"15min":2.8980000000000001},"last":1.022}]},{"osd":6,"last update":"Thu Oct 26 01:37:49 2023","interfaces":[{"interface":"back","average":{"1min":1.5580000000000001,"5min":1.6080000000000001,"15min":1.8460000000000001},"min":{"1min":1.0169999999999999,"5min":0.753,"15min":0.47399999999999998},"max":{"1min":2.0739999999999998,"5min":5.6340000000000003,"15min":11.577999999999999},"last":1.3280000000000001},{"interface":"front","average":{"1min":1.615,"5min":1.6779999999999999,"15min":1.8620000000000001},"min":{"1min":1.2889999999999999,"5min":0.91800000000000004,"15min":0.55100000000000005},"max":{"1min":1.986,"5min":5.0919999999999996,"15min":10.704000000000001},"last":1.117}]}]},{"osd":6,"up_from":13,"seq":55834574945,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":723,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194588,"statfs":{"total":200000000,"available":199258848,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:37:43 2023","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.113,"15min":1.1539999999999999},"min":{"1min":0.69299999999999995,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.6100000000000001,"5min":2.395,"15min":2.395},"last":0.86899999999999999},{"interface":"front","average":{"1min":1.139,"5min":1.173,"15min":1.1220000000000001},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.59199999999999997},"max":{"1min":2.081,"5min":2.081,"15min":2.081},"last":0.94099999999999995}]},{"osd":1,"last update":"Thu Oct 26 01:37:43 2023","interfaces":[{"interface":"back","average":{"1min":1.2749999999999999,"5min":1.2110000000000001,"15min":1.1779999999999999},"min":{"1min":0.82499999999999996,"5min":0.71599999999999997,"15min":0.68300000000000005},"max":{"1min":1.992,"5min":2.0529999999999999,"15min":2.0529999999999999},"last":1.3220000000000001},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.179,"15min":1.204},"min":{"1min":0.67900000000000005,"5min":0.67700000000000005,"15min":0.64000000000000001},"max":{"1min":1.968,"5min":1.968,"15min":2.085},"last":0.82799999999999996}]},{"osd":2,"last update":"Thu Oct 26 01:37:43 2023","interfaces":[{"interface":"back","average":{"1min":1.077,"5min":1.1879999999999999,"15min":1.095},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.5649999999999999,"5min":3.7879999999999998,"15min":3.7879999999999998},"last":1.131},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.3120000000000001,"15min":1.268},"min":{"1min":0.90600000000000003,"5min":0.75800000000000001,"15min":0.68899999999999995},"max":{"1min":2.2200000000000002,"5min":2.2890000000000001,"15min":2.2890000000000001},"last":1.079}]},{"osd":3,"last update":"Thu Oct 26 01:37:43 2023","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.319,"15min":1.214},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.756,"5min":4.4560000000000004,"15min":4.4560000000000004},"last":0.999},{"interface":"front","average":{"1min":1.288,"5min":1.337,"15min":1.325},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.69499999999999995},"max":{"1min":2.1520000000000001,"5min":2.5019999999999998,"15min":2.5019999999999998},"last":1.2190000000000001}]},{"osd":4,"last update":"Thu Oct 26 01:37:43 2023","interfaces":[{"interface":"back","average":{"1min":1.2709999999999999,"5min":1.272,"15min":1.224},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.48399999999999999},"max":{"1min":1.754,"5min":2.9489999999999998,"15min":2.9489999999999998},"last":1.1559999999999999},{"interface":"front","average":{"1min":1.272,"5min":1.4299999999999999,"15min":1.3149999999999999},"min":{"1min":0.96799999999999997,"5min":0.81100000000000005,"15min":0.59299999999999997},"max":{"1min":1.7629999999999999,"5min":6.609,"15min":6.609},"last":1.3779999999999999}]},{"osd":5,"last update":"Thu Oct 26 01:37:43 2023","interfaces":[{"interface":"back","average":{"1min":1.4570000000000001,"5min":1.5469999999999999,"15min":1.4470000000000001},"min":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.67000000000000004},"max":{"1min":1.8380000000000001,"5min":5.4370000000000003,"15min":5.4370000000000003},"last":1.278},{"interface":"front","average":{"1min":1.4370000000000001,"5min":1.4410000000000001,"15min":1.4119999999999999},"min":{"1min":0.626,"5min":0.61399999999999999,"15min":0.58199999999999996},"max":{"1min":2.3279999999999998,"5min":2.621,"15min":2.621},"last":1.431}]},{"osd":7,"last update":"Thu Oct 26 01:37:43 2023","interfaces":[{"interface":"back","average":{"1min":1.3400000000000001,"5min":1.52,"15min":1.4099999999999999},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.54100000000000004},"max":{"1min":1.8740000000000001,"5min":11.804,"15min":11.804},"last":1.4950000000000001},{"interface":"front","average":{"1min":1.5049999999999999,"5min":1.6559999999999999,"15min":1.51},"min":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.77300000000000002},"max":{"1min":2.0710000000000002,"5min":11.541,"15min":11.541},"last":1.5429999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574945,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":724,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194587,"statfs":{"total":200000000,"available":199257700,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:37:46 2023","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.1870000000000001,"15min":1.129},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.5660000000000001,"5min":1.7390000000000001,"15min":1.7390000000000001},"last":1.2609999999999999},{"interface":"front","average":{"1min":1.204,"5min":1.28,"15min":1.1819999999999999},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.613,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":1.6279999999999999}]},{"osd":1,"last update":"Thu Oct 26 01:37:46 2023","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":1.1319999999999999,"15min":1.085},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.3640000000000001,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":0.89900000000000002},{"interface":"front","average":{"1min":1.069,"5min":1.137,"15min":1.095},"min":{"1min":0.629,"5min":0.629,"15min":0.60599999999999998},"max":{"1min":1.726,"5min":1.726,"15min":1.7669999999999999},"last":1.1399999999999999}]},{"osd":2,"last update":"Thu Oct 26 01:37:46 2023","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":1.1950000000000001,"15min":1.163},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.69299999999999995},"max":{"1min":1.681,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":1.3129999999999999},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.2290000000000001,"15min":1.181},"min":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.63900000000000001},"max":{"1min":1.4339999999999999,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":1.401}]},{"osd":3,"last update":"Thu Oct 26 01:37:46 2023","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.2949999999999999,"15min":1.238},"min":{"1min":0.78900000000000003,"5min":0.72699999999999998,"15min":0.72299999999999998},"max":{"1min":1.6180000000000001,"5min":1.7649999999999999,"15min":1.835},"last":1.5860000000000001},{"interface":"front","average":{"1min":1.2529999999999999,"5min":1.3220000000000001,"15min":1.2929999999999999},"min":{"1min":0.85199999999999998,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.7310000000000001,"5min":1.788,"15min":2.9249999999999998},"last":1.4570000000000001}]},{"osd":4,"last update":"Thu Oct 26 01:37:46 2023","interfaces":[{"interface":"back","average":{"1min":1.1519999999999999,"5min":1.2689999999999999,"15min":1.2050000000000001},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.49099999999999999},"max":{"1min":1.6719999999999999,"5min":1.845,"15min":1.845},"last":1.349},{"interface":"front","average":{"1min":1.3340000000000001,"5min":1.4359999999999999,"15min":1.355},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.56200000000000006},"max":{"1min":1.78,"5min":1.867,"15min":1.867},"last":1.776}]},{"osd":6,"last update":"Thu Oct 26 01:37:46 2023","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.4319999999999999,"15min":1.323},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.51900000000000002},"max":{"1min":1.7490000000000001,"5min":2.8250000000000002,"15min":2.8250000000000002},"last":1.5069999999999999},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.423,"15min":1.327},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.61799999999999999},"max":{"1min":1.661,"5min":1.849,"15min":1.8560000000000001},"last":1.55}]},{"osd":7,"last update":"Thu Oct 26 01:37:46 2023","interfaces":[{"interface":"back","average":{"1min":1.4219999999999999,"5min":1.536,"15min":1.4219999999999999},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.61699999999999999},"max":{"1min":1.8220000000000001,"5min":2.1040000000000001,"15min":2.1040000000000001},"last":1.716},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.5609999999999999,"15min":1.431},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.59499999999999997},"max":{"1min":1.853,"5min":2.0960000000000001,"15min":2.0960000000000001},"last":1.6830000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574945,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":724,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194588,"statfs":{"total":200000000,"available":199258304,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:37:42 2023","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.056,"15min":0.98499999999999999},"min":{"1min":0.63800000000000001,"5min":0.58099999999999996,"15min":0.53100000000000003},"max":{"1min":1.5369999999999999,"5min":1.885,"15min":2.0289999999999999},"last":1.0589999999999999},{"interface":"front","average":{"1min":1.0309999999999999,"5min":1.081,"15min":1.024},"min":{"1min":0.74099999999999999,"5min":0.66400000000000003,"15min":0.60599999999999998},"max":{"1min":1.5029999999999999,"5min":2.6829999999999998,"15min":2.6829999999999998},"last":0.97599999999999998}]},{"osd":1,"last update":"Thu Oct 26 01:37:42 2023","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.2,"15min":1.127},"min":{"1min":0.66700000000000004,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.591,"5min":4.0519999999999996,"15min":4.0519999999999996},"last":1.3},{"interface":"front","average":{"1min":1.1599999999999999,"5min":1.1559999999999999,"15min":1.1100000000000001},"min":{"1min":0.70399999999999996,"5min":0.627,"15min":0.60499999999999998},"max":{"1min":1.897,"5min":1.897,"15min":2.0760000000000001},"last":0.70799999999999996}]},{"osd":2,"last update":"Thu Oct 26 01:37:42 2023","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.3480000000000001,"15min":1.23},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.64300000000000002},"max":{"1min":1.845,"5min":5.6749999999999998,"15min":5.6749999999999998},"last":1.51},{"interface":"front","average":{"1min":1.3,"5min":1.218,"15min":1.1359999999999999},"min":{"1min":0.73899999999999999,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":3.673,"5min":3.673,"15min":3.673},"last":1.0269999999999999}]},{"osd":3,"last update":"Thu Oct 26 01:37:42 2023","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.204,"15min":1.1870000000000001},"min":{"1min":0.91500000000000004,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.7669999999999999,"5min":1.847,"15min":1.891},"last":0.82999999999999996},{"interface":"front","average":{"1min":1.194,"5min":1.21,"15min":1.159},"min":{"1min":0.85399999999999998,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.6419999999999999,"5min":1.7210000000000001,"15min":1.7210000000000001},"last":0.90400000000000003}]},{"osd":5,"last update":"Thu Oct 26 01:37:42 2023","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":1.2549999999999999,"15min":1.286},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.46300000000000002},"max":{"1min":1.8520000000000001,"5min":2.1429999999999998,"15min":2.1429999999999998},"last":1.2549999999999999},{"interface":"front","average":{"1min":1.3460000000000001,"5min":1.3500000000000001,"15min":1.3280000000000001},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.7649999999999999,"5min":5.8579999999999997,"15min":5.8579999999999997},"last":1.4039999999999999}]},{"osd":6,"last update":"Thu Oct 26 01:37:42 2023","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":1.427,"15min":1.3380000000000001},"min":{"1min":0.69199999999999995,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":2.2149999999999999,"5min":12.343999999999999,"15min":12.343999999999999},"last":1.589},{"interface":"front","average":{"1min":1.492,"5min":1.619,"15min":1.5149999999999999},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.55000000000000004},"max":{"1min":2.1000000000000001,"5min":12.605,"15min":12.605},"last":1.361}]},{"osd":7,"last update":"Thu Oct 26 01:37:42 2023","interfaces":[{"interface":"back","average":{"1min":1.4710000000000001,"5min":1.52,"15min":1.49},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":2.0390000000000001,"5min":5.774,"15min":5.774},"last":1.639},{"interface":"front","average":{"1min":1.53,"5min":1.5309999999999999,"15min":1.46},"min":{"1min":0.79000000000000004,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":2.3450000000000002,"5min":4.6520000000000001,"15min":4.6520000000000001},"last":1.452}]}]},{"osd":2,"up_from":13,"seq":55834574945,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":724,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194588,"statfs":{"total":200000000,"available":199258304,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:37:46 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.84099999999999997,"15min":0.85499999999999998},"min":{"1min":0.41199999999999998,"5min":0.308,"15min":0.308},"max":{"1min":1.4850000000000001,"5min":1.5249999999999999,"15min":6.3200000000000003},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.997,"15min":0.93100000000000005},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.2569999999999999,"5min":2.1400000000000001,"15min":4.2619999999999996},"last":0.88900000000000001}]},{"osd":1,"last update":"Thu Oct 26 01:37:46 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.90200000000000002,"15min":0.89700000000000002},"min":{"1min":0.39300000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.244,"5min":1.629,"15min":5.1260000000000003},"last":1.0109999999999999},{"interface":"front","average":{"1min":0.85399999999999998,"5min":1.018,"15min":1.0029999999999999},"min":{"1min":0.47699999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.351,"5min":1.873,"15min":6.1520000000000001},"last":0.94699999999999995}]},{"osd":3,"last update":"Thu Oct 26 01:37:48 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":1.0189999999999999,"15min":0.93899999999999995},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.443},"max":{"1min":1.4390000000000001,"5min":2.0640000000000001,"15min":2.0640000000000001},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.93600000000000005,"5min":1.111,"15min":1.018},"min":{"1min":0.51100000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.375,"5min":1.9279999999999999,"15min":8.9019999999999992},"last":1.351}]},{"osd":4,"last update":"Thu Oct 26 01:37:46 2023","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.3420000000000001,"15min":1.2689999999999999},"min":{"1min":0.86499999999999999,"5min":0.76000000000000001,"15min":0.73899999999999999},"max":{"1min":1.704,"5min":2.6190000000000002,"15min":6.2670000000000003},"last":1.0720000000000001},{"interface":"front","average":{"1min":1.1879999999999999,"5min":1.3129999999999999,"15min":1.262},"min":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.78700000000000003},"max":{"1min":1.5920000000000001,"5min":2.4359999999999999,"15min":7.343},"last":1.24}]},{"osd":5,"last update":"Thu Oct 26 01:37:46 2023","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.3540000000000001,"15min":1.2869999999999999},"min":{"1min":0.873,"5min":0.81699999999999995,"15min":0.71899999999999997},"max":{"1min":1.5740000000000001,"5min":2.5790000000000002,"15min":3.7170000000000001},"last":1.149},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.3089999999999999,"15min":1.2909999999999999},"min":{"1min":0.81899999999999995,"5min":0.70999999999999996,"15min":0.65300000000000002},"max":{"1min":1.621,"5min":2.298,"15min":6.3899999999999997},"last":1.3879999999999999}]},{"osd":6,"last update":"Thu Oct 26 01:37:46 2023","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":1.399,"15min":1.335},"min":{"1min":0.81200000000000006,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.9379999999999999,"5min":2.73,"15min":6.0490000000000004},"last":1.3340000000000001},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.4179999999999999,"15min":1.3720000000000001},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.76800000000000002},"max":{"1min":1.8799999999999999,"5min":2.4089999999999998,"15min":5.0650000000000004},"last":1.4590000000000001}]},{"osd":7,"last update":"Thu Oct 26 01:37:46 2023","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.403,"15min":1.395},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.69399999999999995},"max":{"1min":1.7809999999999999,"5min":2.3370000000000002,"15min":6.4480000000000004},"last":1.4339999999999999},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.52,"15min":1.4550000000000001},"min":{"1min":1.036,"5min":0.91800000000000004,"15min":0.91800000000000004},"max":{"1min":1.877,"5min":2.8290000000000002,"15min":2.8439999999999999},"last":1.5029999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574945,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":724,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194588,"statfs":{"total":200000000,"available":199258180,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:37:48 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.78300000000000003,"15min":0.72499999999999998},"min":{"1min":0.36099999999999999,"5min":0.34000000000000002,"15min":0.29499999999999998},"max":{"1min":1.4099999999999999,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.86799999999999999,"15min":0.79600000000000004},"min":{"1min":0.435,"5min":0.36299999999999999,"15min":0.36199999999999999},"max":{"1min":1.4019999999999999,"5min":1.6399999999999999,"15min":1.6399999999999999},"last":0.94899999999999995}]},{"osd":2,"last update":"Thu Oct 26 01:37:48 2023","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":0.97899999999999998,"15min":1.0009999999999999},"min":{"1min":0.52200000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.8200000000000001,"5min":1.8200000000000001,"15min":2.4830000000000001},"last":1.258},{"interface":"front","average":{"1min":1.0569999999999999,"5min":0.93700000000000006,"15min":0.875},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.772,"5min":1.8080000000000001,"15min":1.8080000000000001},"last":1.5369999999999999}]},{"osd":3,"last update":"Thu Oct 26 01:37:52 2023","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.97899999999999998,"15min":0.94899999999999995},"min":{"1min":0.52900000000000003,"5min":0.42499999999999999,"15min":0.39500000000000002},"max":{"1min":1.288,"5min":1.611,"15min":1.75},"last":1.097},{"interface":"front","average":{"1min":0.97899999999999998,"5min":1.028,"15min":0.98999999999999999},"min":{"1min":0.55900000000000005,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.45,"5min":1.673,"15min":1.802},"last":1.0069999999999999}]},{"osd":4,"last update":"Thu Oct 26 01:37:48 2023","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":1.2270000000000001,"15min":1.1990000000000001},"min":{"1min":0.82899999999999996,"5min":0.69099999999999995,"15min":0.56200000000000006},"max":{"1min":1.756,"5min":1.756,"15min":2.0659999999999998},"last":1.3080000000000001},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.1930000000000001,"15min":1.1359999999999999},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.474,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":1.409}]},{"osd":5,"last update":"Thu Oct 26 01:37:48 2023","interfaces":[{"interface":"back","average":{"1min":1.1459999999999999,"5min":1.262,"15min":1.2270000000000001},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.573,"5min":1.8779999999999999,"15min":1.8779999999999999},"last":1.2010000000000001},{"interface":"front","average":{"1min":1.3720000000000001,"5min":1.3720000000000001,"15min":1.337},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":2.04,"5min":2.04,"15min":2.2200000000000002},"last":1.486}]},{"osd":6,"last update":"Thu Oct 26 01:37:48 2023","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":1.3320000000000001,"15min":1.28},"min":{"1min":0.75,"5min":0.70199999999999996,"15min":0.68899999999999995},"max":{"1min":1.7330000000000001,"5min":2.0910000000000002,"15min":2.3780000000000001},"last":1.369},{"interface":"front","average":{"1min":1.26,"5min":1.2729999999999999,"15min":1.244},"min":{"1min":0.80700000000000005,"5min":0.71299999999999997,"15min":0.70799999999999996},"max":{"1min":1.669,"5min":1.956,"15min":1.984},"last":1.4399999999999999}]},{"osd":7,"last update":"Thu Oct 26 01:37:48 2023","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.2629999999999999,"15min":1.262},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.69699999999999995},"max":{"1min":1.7230000000000001,"5min":1.8380000000000001,"15min":2.6930000000000001},"last":1.0429999999999999},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.4019999999999999,"15min":1.337},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.9370000000000001,"5min":2.0190000000000001,"15min":2.0190000000000001},"last":1.617}]}]},{"osd":0,"up_from":13,"seq":55834574945,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1302,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194009,"statfs":{"total":200000000,"available":198666212,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Oct 26 01:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":1.0029999999999999,"15min":0.91600000000000004},"min":{"1min":0.41699999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.2170000000000001,"5min":19.425000000000001,"15min":19.425000000000001},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.84199999999999997,"5min":1.0720000000000001,"15min":1.042},"min":{"1min":0.56100000000000005,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.1839999999999999,"5min":15.162000000000001,"15min":15.162000000000001},"last":1.1499999999999999}]},{"osd":2,"last update":"Thu Oct 26 01:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":1.3080000000000001,"15min":1.1819999999999999},"min":{"1min":0.57699999999999996,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.4930000000000001,"5min":28.439,"15min":28.439},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.91700000000000004,"15min":0.91200000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.4810000000000001,"5min":1.502,"15min":4.9459999999999997},"last":0.64300000000000002}]},{"osd":3,"last update":"Thu Oct 26 01:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":1.0549999999999999,"15min":1.085},"min":{"1min":0.48899999999999999,"5min":0.46000000000000002,"15min":0.44600000000000001},"max":{"1min":1.4330000000000001,"5min":1.6479999999999999,"15min":4.9800000000000004},"last":1.111},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.1459999999999999,"15min":1.155},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.4770000000000001,"5min":1.659,"15min":5.016},"last":1.2330000000000001}]},{"osd":4,"last update":"Thu Oct 26 01:37:45 2023","interfaces":[{"interface":"back","average":{"1min":1.091,"5min":1.379,"15min":1.29},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.49,"5min":19.329000000000001,"15min":19.329000000000001},"last":0.97799999999999998},{"interface":"front","average":{"1min":1.22,"5min":1.268,"15min":1.2989999999999999},"min":{"1min":0.88,"5min":0.82099999999999995,"15min":0.67600000000000005},"max":{"1min":1.5449999999999999,"5min":1.718,"15min":5.5229999999999997},"last":1.3460000000000001}]},{"osd":5,"last update":"Thu Oct 26 01:37:45 2023","interfaces":[{"interface":"back","average":{"1min":1.373,"5min":1.4039999999999999,"15min":1.4299999999999999},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.661,"5min":1.915,"15min":5.6360000000000001},"last":1.395},{"interface":"front","average":{"1min":1.202,"5min":1.4790000000000001,"15min":1.3999999999999999},"min":{"1min":0.73599999999999999,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.71,"5min":19.384,"15min":19.384},"last":1.1839999999999999}]},{"osd":6,"last update":"Thu Oct 26 01:37:45 2023","interfaces":[{"interface":"back","average":{"1min":1.2609999999999999,"5min":1.3799999999999999,"15min":1.365},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.73799999999999999},"max":{"1min":1.615,"5min":8.2609999999999992,"15min":8.2609999999999992},"last":1.4650000000000001},{"interface":"front","average":{"1min":1.3440000000000001,"5min":1.4590000000000001,"15min":1.407},"min":{"1min":0.83099999999999996,"5min":0.77900000000000003,"15min":0.75900000000000001},"max":{"1min":1.804,"5min":11.849,"15min":11.849},"last":1.429}]},{"osd":7,"last update":"Thu Oct 26 01:37:45 2023","interfaces":[{"interface":"back","average":{"1min":1.3979999999999999,"5min":1.4450000000000001,"15min":1.4810000000000001},"min":{"1min":0.871,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.8380000000000001,"5min":2.4060000000000001,"15min":5.6689999999999996},"last":1.3149999999999999},{"interface":"front","average":{"1min":1.365,"5min":1.6120000000000001,"15min":1.5029999999999999},"min":{"1min":0.89600000000000002,"5min":0.879,"15min":0.69399999999999995},"max":{"1min":1.841,"5min":19.245999999999999,"15min":19.245999999999999},"last":1.2809999999999999}]}]}],"pool_statfs":[]}} 2023-10-26T01:38:30.842 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-26T01:38:31.074 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:31.072+0000 7f449e93c640 1 -- 172.21.15.169:0/3968522700 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4498155c40 msgr2=0x7f44980c6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:31.074 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:31.072+0000 7f449e93c640 1 --2- 172.21.15.169:0/3968522700 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4498155c40 0x7f44980c6270 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f4488009830 tx=0x7f448802bd60 comp rx=0 tx=0).stop 2023-10-26T01:38:31.075 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:31.072+0000 7f449e93c640 1 -- 172.21.15.169:0/3968522700 shutdown_connections 2023-10-26T01:38:31.075 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:31.072+0000 7f449e93c640 1 --2- 172.21.15.169:0/3968522700 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f44980c67b0 0x7f44980cb300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:31.075 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:31.072+0000 7f449e93c640 1 --2- 172.21.15.169:0/3968522700 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4498155c40 0x7f44980c6270 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:31.075 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:31.072+0000 7f449e93c640 1 -- 172.21.15.169:0/3968522700 >> 172.21.15.169:0/3968522700 conn(0x7f44980aedd0 msgr2=0x7f44980af1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:31.075 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:31.072+0000 7f449e93c640 1 -- 172.21.15.169:0/3968522700 shutdown_connections 2023-10-26T01:38:31.076 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:31.072+0000 7f449e93c640 1 -- 172.21.15.169:0/3968522700 wait complete. 2023-10-26T01:38:31.077 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:31.074+0000 7f449e93c640 1 Processor -- start 2023-10-26T01:38:31.077 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:31.074+0000 7f449e93c640 1 -- start start 2023-10-26T01:38:31.078 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:31.075+0000 7f449e93c640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f44980c67b0 0x7f4498158f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:31.078 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:31.075+0000 7f449e93c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4498155860 0x7f449813f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:31.078 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:31.075+0000 7f449e93c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4498153e70 0x7f4498154300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:31.079 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:31.075+0000 7f449e93c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4498154840 con 0x7f4498155860 2023-10-26T01:38:31.079 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:31.075+0000 7f449e93c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4498157f40 con 0x7f4498153e70 2023-10-26T01:38:31.079 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:31.075+0000 7f449e93c640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44980abf90 con 0x7f44980c67b0 2023-10-26T01:38:31.079 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:31.075+0000 7f449d139640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4498155860 0x7f449813f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:31.080 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:31.075+0000 7f449d139640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4498155860 0x7f449813f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:34368/0 (socket says 172.21.15.169:34368) 2023-10-26T01:38:31.080 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:31.075+0000 7f449d139640 1 -- 172.21.15.169:0/2849626224 learned_addr learned my addr 172.21.15.169:0/2849626224 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:38:31.080 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:31.075+0000 7f449d93a640 1 --2- 172.21.15.169:0/2849626224 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f44980c67b0 0x7f4498158f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:31.080 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:31.075+0000 7f449e13b640 1 --2- 172.21.15.169:0/2849626224 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4498153e70 0x7f4498154300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:31.081 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:31.076+0000 7f449d139640 1 -- 172.21.15.169:0/2849626224 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f44980c67b0 msgr2=0x7f4498158f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:31.081 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:31.076+0000 7f449d139640 1 --2- 172.21.15.169:0/2849626224 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f44980c67b0 0x7f4498158f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:31.082 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:31.076+0000 7f449d139640 1 -- 172.21.15.169:0/2849626224 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4498153e70 msgr2=0x7f4498154300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:31.082 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:31.076+0000 7f449d139640 1 --2- 172.21.15.169:0/2849626224 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4498153e70 0x7f4498154300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:31.082 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:31.076+0000 7f449d139640 1 -- 172.21.15.169:0/2849626224 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4488009490 con 0x7f4498155860 2023-10-26T01:38:31.084 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:31.076+0000 7f449d139640 1 --2- 172.21.15.169:0/2849626224 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4498155860 0x7f449813f320 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f4488009950 tx=0x7f44880038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:31.085 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:31.078+0000 7f448e7fc640 1 -- 172.21.15.169:0/2849626224 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4488003ea0 con 0x7f4498155860 2023-10-26T01:38:31.085 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:31.078+0000 7f448e7fc640 1 -- 172.21.15.169:0/2849626224 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4488034550 con 0x7f4498155860 2023-10-26T01:38:31.085 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:31.078+0000 7f449e93c640 1 -- 172.21.15.169:0/2849626224 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4498158aa0 con 0x7f4498155860 2023-10-26T01:38:31.086 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:31.078+0000 7f448e7fc640 1 -- 172.21.15.169:0/2849626224 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4488034960 con 0x7f4498155860 2023-10-26T01:38:31.086 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:31.078+0000 7f449e93c640 1 -- 172.21.15.169:0/2849626224 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4498158450 con 0x7f4498155860 2023-10-26T01:38:31.086 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:31.080+0000 7f449e93c640 1 -- 172.21.15.169:0/2849626224 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4498158c30 con 0x7f4498155860 2023-10-26T01:38:31.087 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:31.081+0000 7f448e7fc640 1 -- 172.21.15.169:0/2849626224 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f448803b040 con 0x7f4498155860 2023-10-26T01:38:31.087 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:31.083+0000 7f448e7fc640 1 --2- 172.21.15.169:0/2849626224 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f447c06ae40 0x7f447c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:31.087 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:31.083+0000 7f448e7fc640 1 -- 172.21.15.169:0/2849626224 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f44880abcf0 con 0x7f4498155860 2023-10-26T01:38:31.088 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:31.084+0000 7f449d93a640 1 --2- 172.21.15.169:0/2849626224 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f447c06ae40 0x7f447c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:31.089 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:31.085+0000 7f449d93a640 1 --2- 172.21.15.169:0/2849626224 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f447c06ae40 0x7f447c06d300 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f44940046b0 tx=0x7f4494009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:31.089 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:31.088+0000 7f448e7fc640 1 -- 172.21.15.169:0/2849626224 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f44880b0070 con 0x7f4498155860 2023-10-26T01:38:31.275 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:31.273+0000 7f449e93c640 1 -- 172.21.15.169:0/2849626224 --> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4498073c10 con 0x7f447c06ae40 2023-10-26T01:38:31.297 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:31.295+0000 7f448e7fc640 1 -- 172.21.15.169:0/2849626224 <== mgr.4112 v2:172.21.15.170:6832/1187156918 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+282924 (secure 0 0 0) 0x7f4498073c10 con 0x7f447c06ae40 2023-10-26T01:38:31.301 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:38:31.303 INFO:teuthology.orchestra.run.smithi169.stderr:dumped all 2023-10-26T01:38:31.304 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:31.301+0000 7f449e93c640 1 -- 172.21.15.169:0/2849626224 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f447c06ae40 msgr2=0x7f447c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:31.304 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:31.301+0000 7f449e93c640 1 --2- 172.21.15.169:0/2849626224 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f447c06ae40 0x7f447c06d300 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f44940046b0 tx=0x7f4494009040 comp rx=0 tx=0).stop 2023-10-26T01:38:31.305 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:31.301+0000 7f449e93c640 1 -- 172.21.15.169:0/2849626224 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4498155860 msgr2=0x7f449813f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:31.305 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:31.301+0000 7f449e93c640 1 --2- 172.21.15.169:0/2849626224 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4498155860 0x7f449813f320 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f4488009950 tx=0x7f44880038a0 comp rx=0 tx=0).stop 2023-10-26T01:38:31.305 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:31.302+0000 7f449e93c640 1 -- 172.21.15.169:0/2849626224 shutdown_connections 2023-10-26T01:38:31.305 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:31.302+0000 7f449e93c640 1 --2- 172.21.15.169:0/2849626224 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f447c06ae40 0x7f447c06d300 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:31.305 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:31.302+0000 7f449e93c640 1 --2- 172.21.15.169:0/2849626224 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4498155860 0x7f449813f320 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:31.306 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:31.302+0000 7f449e93c640 1 --2- 172.21.15.169:0/2849626224 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4498153e70 0x7f4498154300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:31.306 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:31.302+0000 7f449e93c640 1 --2- 172.21.15.169:0/2849626224 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f44980c67b0 0x7f4498158f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:31.306 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:31.302+0000 7f449e93c640 1 -- 172.21.15.169:0/2849626224 >> 172.21.15.169:0/2849626224 conn(0x7f44980aedd0 msgr2=0x7f44980ae1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:31.306 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:31.302+0000 7f449e93c640 1 -- 172.21.15.169:0/2849626224 shutdown_connections 2023-10-26T01:38:31.306 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:31.302+0000 7f449e93c640 1 -- 172.21.15.169:0/2849626224 wait complete. 2023-10-26T01:38:31.322 INFO:teuthology.orchestra.run.smithi169.stdout:{"pg_ready":true,"pg_map":{"version":368,"stamp":"2023-10-26T01:38:31.077591+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":6951,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1555548,"statfs":{"total":1600000000,"available":1592881490,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":42,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.783983"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.359274+0000","last_change":"2023-10-26T01:38:21.325023+0000","last_active":"2023-10-26T01:38:22.359274+0000","last_peered":"2023-10-26T01:38:22.359274+0000","last_clean":"2023-10-26T01:38:22.359274+0000","last_became_active":"2023-10-26T01:38:21.323959+0000","last_became_peered":"2023-10-26T01:38:21.323959+0000","last_unstale":"2023-10-26T01:38:22.359274+0000","last_undegraded":"2023-10-26T01:38:22.359274+0000","last_fullsized":"2023-10-26T01:38:22.359274+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:51:07.392163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.297000+0000","last_change":"2023-10-26T01:38:21.315511+0000","last_active":"2023-10-26T01:38:22.297000+0000","last_peered":"2023-10-26T01:38:22.297000+0000","last_clean":"2023-10-26T01:38:22.297000+0000","last_became_active":"2023-10-26T01:38:21.314652+0000","last_became_peered":"2023-10-26T01:38:21.314652+0000","last_unstale":"2023-10-26T01:38:22.297000+0000","last_undegraded":"2023-10-26T01:38:22.297000+0000","last_fullsized":"2023-10-26T01:38:22.297000+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:44:57.150927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.298865+0000","last_change":"2023-10-26T01:38:21.327284+0000","last_active":"2023-10-26T01:38:22.298865+0000","last_peered":"2023-10-26T01:38:22.298865+0000","last_clean":"2023-10-26T01:38:22.298865+0000","last_became_active":"2023-10-26T01:38:21.326309+0000","last_became_peered":"2023-10-26T01:38:21.326309+0000","last_unstale":"2023-10-26T01:38:22.298865+0000","last_undegraded":"2023-10-26T01:38:22.298865+0000","last_fullsized":"2023-10-26T01:38:22.298865+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:46:37.417585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.358670+0000","last_change":"2023-10-26T01:38:21.317851+0000","last_active":"2023-10-26T01:38:22.358670+0000","last_peered":"2023-10-26T01:38:22.358670+0000","last_clean":"2023-10-26T01:38:22.358670+0000","last_became_active":"2023-10-26T01:38:21.317269+0000","last_became_peered":"2023-10-26T01:38:21.317269+0000","last_unstale":"2023-10-26T01:38:22.358670+0000","last_undegraded":"2023-10-26T01:38:22.358670+0000","last_fullsized":"2023-10-26T01:38:22.358670+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T01:59:52.462839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.357989+0000","last_change":"2023-10-26T01:38:21.304138+0000","last_active":"2023-10-26T01:38:22.357989+0000","last_peered":"2023-10-26T01:38:22.357989+0000","last_clean":"2023-10-26T01:38:22.357989+0000","last_became_active":"2023-10-26T01:38:21.303352+0000","last_became_peered":"2023-10-26T01:38:21.303352+0000","last_unstale":"2023-10-26T01:38:22.357989+0000","last_undegraded":"2023-10-26T01:38:22.357989+0000","last_fullsized":"2023-10-26T01:38:22.357989+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:36:15.755953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.357771+0000","last_change":"2023-10-26T01:38:21.319100+0000","last_active":"2023-10-26T01:38:22.357771+0000","last_peered":"2023-10-26T01:38:22.357771+0000","last_clean":"2023-10-26T01:38:22.357771+0000","last_became_active":"2023-10-26T01:38:21.318169+0000","last_became_peered":"2023-10-26T01:38:21.318169+0000","last_unstale":"2023-10-26T01:38:22.357771+0000","last_undegraded":"2023-10-26T01:38:22.357771+0000","last_fullsized":"2023-10-26T01:38:22.357771+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:05:26.952713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.356781+0000","last_change":"2023-10-26T01:38:21.318003+0000","last_active":"2023-10-26T01:38:22.356781+0000","last_peered":"2023-10-26T01:38:22.356781+0000","last_clean":"2023-10-26T01:38:22.356781+0000","last_became_active":"2023-10-26T01:38:21.317493+0000","last_became_peered":"2023-10-26T01:38:21.317493+0000","last_unstale":"2023-10-26T01:38:22.356781+0000","last_undegraded":"2023-10-26T01:38:22.356781+0000","last_fullsized":"2023-10-26T01:38:22.356781+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:32:43.401727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.299581+0000","last_change":"2023-10-26T01:38:21.324036+0000","last_active":"2023-10-26T01:38:22.299581+0000","last_peered":"2023-10-26T01:38:22.299581+0000","last_clean":"2023-10-26T01:38:22.299581+0000","last_became_active":"2023-10-26T01:38:21.323785+0000","last_became_peered":"2023-10-26T01:38:21.323785+0000","last_unstale":"2023-10-26T01:38:22.299581+0000","last_undegraded":"2023-10-26T01:38:22.299581+0000","last_fullsized":"2023-10-26T01:38:22.299581+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:27:43.267716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.358805+0000","last_change":"2023-10-26T01:38:21.323358+0000","last_active":"2023-10-26T01:38:22.358805+0000","last_peered":"2023-10-26T01:38:22.358805+0000","last_clean":"2023-10-26T01:38:22.358805+0000","last_became_active":"2023-10-26T01:38:21.322567+0000","last_became_peered":"2023-10-26T01:38:21.322567+0000","last_unstale":"2023-10-26T01:38:22.358805+0000","last_undegraded":"2023-10-26T01:38:22.358805+0000","last_fullsized":"2023-10-26T01:38:22.358805+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:07:49.066636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.357274+0000","last_change":"2023-10-26T01:38:21.320421+0000","last_active":"2023-10-26T01:38:22.357274+0000","last_peered":"2023-10-26T01:38:22.357274+0000","last_clean":"2023-10-26T01:38:22.357274+0000","last_became_active":"2023-10-26T01:38:21.319296+0000","last_became_peered":"2023-10-26T01:38:21.319296+0000","last_unstale":"2023-10-26T01:38:22.357274+0000","last_undegraded":"2023-10-26T01:38:22.357274+0000","last_fullsized":"2023-10-26T01:38:22.357274+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:24:08.193245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.296689+0000","last_change":"2023-10-26T01:38:21.313660+0000","last_active":"2023-10-26T01:38:22.296689+0000","last_peered":"2023-10-26T01:38:22.296689+0000","last_clean":"2023-10-26T01:38:22.296689+0000","last_became_active":"2023-10-26T01:38:21.312980+0000","last_became_peered":"2023-10-26T01:38:21.312980+0000","last_unstale":"2023-10-26T01:38:22.296689+0000","last_undegraded":"2023-10-26T01:38:22.296689+0000","last_fullsized":"2023-10-26T01:38:22.296689+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:00:42.118665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.358755+0000","last_change":"2023-10-26T01:38:21.315876+0000","last_active":"2023-10-26T01:38:22.358755+0000","last_peered":"2023-10-26T01:38:22.358755+0000","last_clean":"2023-10-26T01:38:22.358755+0000","last_became_active":"2023-10-26T01:38:21.314851+0000","last_became_peered":"2023-10-26T01:38:21.314851+0000","last_unstale":"2023-10-26T01:38:22.358755+0000","last_undegraded":"2023-10-26T01:38:22.358755+0000","last_fullsized":"2023-10-26T01:38:22.358755+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:34:15.657860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.299218+0000","last_change":"2023-10-26T01:38:21.327458+0000","last_active":"2023-10-26T01:38:22.299218+0000","last_peered":"2023-10-26T01:38:22.299218+0000","last_clean":"2023-10-26T01:38:22.299218+0000","last_became_active":"2023-10-26T01:38:21.326720+0000","last_became_peered":"2023-10-26T01:38:21.326720+0000","last_unstale":"2023-10-26T01:38:22.299218+0000","last_undegraded":"2023-10-26T01:38:22.299218+0000","last_fullsized":"2023-10-26T01:38:22.299218+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:48:01.026075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.296528+0000","last_change":"2023-10-26T01:38:21.317035+0000","last_active":"2023-10-26T01:38:22.296528+0000","last_peered":"2023-10-26T01:38:22.296528+0000","last_clean":"2023-10-26T01:38:22.296528+0000","last_became_active":"2023-10-26T01:38:21.316156+0000","last_became_peered":"2023-10-26T01:38:21.316156+0000","last_unstale":"2023-10-26T01:38:22.296528+0000","last_undegraded":"2023-10-26T01:38:22.296528+0000","last_fullsized":"2023-10-26T01:38:22.296528+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T01:50:22.270135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.356685+0000","last_change":"2023-10-26T01:38:21.312264+0000","last_active":"2023-10-26T01:38:22.356685+0000","last_peered":"2023-10-26T01:38:22.356685+0000","last_clean":"2023-10-26T01:38:22.356685+0000","last_became_active":"2023-10-26T01:38:21.311809+0000","last_became_peered":"2023-10-26T01:38:21.311809+0000","last_unstale":"2023-10-26T01:38:22.356685+0000","last_undegraded":"2023-10-26T01:38:22.356685+0000","last_fullsized":"2023-10-26T01:38:22.356685+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:36:15.755953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.359566+0000","last_change":"2023-10-26T01:38:21.325166+0000","last_active":"2023-10-26T01:38:22.359566+0000","last_peered":"2023-10-26T01:38:22.359566+0000","last_clean":"2023-10-26T01:38:22.359566+0000","last_became_active":"2023-10-26T01:38:21.324612+0000","last_became_peered":"2023-10-26T01:38:21.324612+0000","last_unstale":"2023-10-26T01:38:22.359566+0000","last_undegraded":"2023-10-26T01:38:22.359566+0000","last_fullsized":"2023-10-26T01:38:22.359566+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:17:29.812141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.297352+0000","last_change":"2023-10-26T01:38:21.315001+0000","last_active":"2023-10-26T01:38:22.297352+0000","last_peered":"2023-10-26T01:38:22.297352+0000","last_clean":"2023-10-26T01:38:22.297352+0000","last_became_active":"2023-10-26T01:38:21.312976+0000","last_became_peered":"2023-10-26T01:38:21.312976+0000","last_unstale":"2023-10-26T01:38:22.297352+0000","last_undegraded":"2023-10-26T01:38:22.297352+0000","last_fullsized":"2023-10-26T01:38:22.297352+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:29:52.799943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.356647+0000","last_change":"2023-10-26T01:38:21.319948+0000","last_active":"2023-10-26T01:38:22.356647+0000","last_peered":"2023-10-26T01:38:22.356647+0000","last_clean":"2023-10-26T01:38:22.356647+0000","last_became_active":"2023-10-26T01:38:21.319058+0000","last_became_peered":"2023-10-26T01:38:21.319058+0000","last_unstale":"2023-10-26T01:38:22.356647+0000","last_undegraded":"2023-10-26T01:38:22.356647+0000","last_fullsized":"2023-10-26T01:38:22.356647+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:29:52.799943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.296873+0000","last_change":"2023-10-26T01:38:21.324164+0000","last_active":"2023-10-26T01:38:22.296873+0000","last_peered":"2023-10-26T01:38:22.296873+0000","last_clean":"2023-10-26T01:38:22.296873+0000","last_became_active":"2023-10-26T01:38:21.323176+0000","last_became_peered":"2023-10-26T01:38:21.323176+0000","last_unstale":"2023-10-26T01:38:22.296873+0000","last_undegraded":"2023-10-26T01:38:22.296873+0000","last_fullsized":"2023-10-26T01:38:22.296873+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:44:57.150927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.299813+0000","last_change":"2023-10-26T01:38:21.313793+0000","last_active":"2023-10-26T01:38:22.299813+0000","last_peered":"2023-10-26T01:38:22.299813+0000","last_clean":"2023-10-26T01:38:22.299813+0000","last_became_active":"2023-10-26T01:38:21.312693+0000","last_became_peered":"2023-10-26T01:38:21.312693+0000","last_unstale":"2023-10-26T01:38:22.299813+0000","last_undegraded":"2023-10-26T01:38:22.299813+0000","last_fullsized":"2023-10-26T01:38:22.299813+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:47:08.011620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.297043+0000","last_change":"2023-10-26T01:38:21.305093+0000","last_active":"2023-10-26T01:38:22.297043+0000","last_peered":"2023-10-26T01:38:22.297043+0000","last_clean":"2023-10-26T01:38:22.297043+0000","last_became_active":"2023-10-26T01:38:21.304404+0000","last_became_peered":"2023-10-26T01:38:21.304404+0000","last_unstale":"2023-10-26T01:38:22.297043+0000","last_undegraded":"2023-10-26T01:38:22.297043+0000","last_fullsized":"2023-10-26T01:38:22.297043+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:51:42.712087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.294392+0000","last_change":"2023-10-26T01:38:21.314965+0000","last_active":"2023-10-26T01:38:22.294392+0000","last_peered":"2023-10-26T01:38:22.294392+0000","last_clean":"2023-10-26T01:38:22.294392+0000","last_became_active":"2023-10-26T01:38:21.314673+0000","last_became_peered":"2023-10-26T01:38:21.314673+0000","last_unstale":"2023-10-26T01:38:22.294392+0000","last_undegraded":"2023-10-26T01:38:22.294392+0000","last_fullsized":"2023-10-26T01:38:22.294392+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:55:09.924168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.298857+0000","last_change":"2023-10-26T01:38:21.323272+0000","last_active":"2023-10-26T01:38:22.298857+0000","last_peered":"2023-10-26T01:38:22.298857+0000","last_clean":"2023-10-26T01:38:22.298857+0000","last_became_active":"2023-10-26T01:38:21.322991+0000","last_became_peered":"2023-10-26T01:38:21.322991+0000","last_unstale":"2023-10-26T01:38:22.298857+0000","last_undegraded":"2023-10-26T01:38:22.298857+0000","last_fullsized":"2023-10-26T01:38:22.298857+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:48:01.026075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.294461+0000","last_change":"2023-10-26T01:38:21.320068+0000","last_active":"2023-10-26T01:38:22.294461+0000","last_peered":"2023-10-26T01:38:22.294461+0000","last_clean":"2023-10-26T01:38:22.294461+0000","last_became_active":"2023-10-26T01:38:21.319773+0000","last_became_peered":"2023-10-26T01:38:21.319773+0000","last_unstale":"2023-10-26T01:38:22.294461+0000","last_undegraded":"2023-10-26T01:38:22.294461+0000","last_fullsized":"2023-10-26T01:38:22.294461+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:27:42.700969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.300137+0000","last_change":"2023-10-26T01:38:21.314000+0000","last_active":"2023-10-26T01:38:22.300137+0000","last_peered":"2023-10-26T01:38:22.300137+0000","last_clean":"2023-10-26T01:38:22.300137+0000","last_became_active":"2023-10-26T01:38:21.313194+0000","last_became_peered":"2023-10-26T01:38:21.313194+0000","last_unstale":"2023-10-26T01:38:22.300137+0000","last_undegraded":"2023-10-26T01:38:22.300137+0000","last_fullsized":"2023-10-26T01:38:22.300137+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T01:40:34.431360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.359873+0000","last_change":"2023-10-26T01:38:21.324296+0000","last_active":"2023-10-26T01:38:22.359873+0000","last_peered":"2023-10-26T01:38:22.359873+0000","last_clean":"2023-10-26T01:38:22.359873+0000","last_became_active":"2023-10-26T01:38:21.320669+0000","last_became_peered":"2023-10-26T01:38:21.320669+0000","last_unstale":"2023-10-26T01:38:22.359873+0000","last_undegraded":"2023-10-26T01:38:22.359873+0000","last_fullsized":"2023-10-26T01:38:22.359873+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:16:57.330926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.357018+0000","last_change":"2023-10-26T01:38:21.320441+0000","last_active":"2023-10-26T01:38:22.357018+0000","last_peered":"2023-10-26T01:38:22.357018+0000","last_clean":"2023-10-26T01:38:22.357018+0000","last_became_active":"2023-10-26T01:38:21.320185+0000","last_became_peered":"2023-10-26T01:38:21.320185+0000","last_unstale":"2023-10-26T01:38:22.357018+0000","last_undegraded":"2023-10-26T01:38:22.357018+0000","last_fullsized":"2023-10-26T01:38:22.357018+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:11:34.893562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.294463+0000","last_change":"2023-10-26T01:38:21.319408+0000","last_active":"2023-10-26T01:38:22.294463+0000","last_peered":"2023-10-26T01:38:22.294463+0000","last_clean":"2023-10-26T01:38:22.294463+0000","last_became_active":"2023-10-26T01:38:21.318421+0000","last_became_peered":"2023-10-26T01:38:21.318421+0000","last_unstale":"2023-10-26T01:38:22.294463+0000","last_undegraded":"2023-10-26T01:38:22.294463+0000","last_fullsized":"2023-10-26T01:38:22.294463+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:07:29.184582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.296824+0000","last_change":"2023-10-26T01:38:21.322363+0000","last_active":"2023-10-26T01:38:22.296824+0000","last_peered":"2023-10-26T01:38:22.296824+0000","last_clean":"2023-10-26T01:38:22.296824+0000","last_became_active":"2023-10-26T01:38:21.322104+0000","last_became_peered":"2023-10-26T01:38:21.322104+0000","last_unstale":"2023-10-26T01:38:22.296824+0000","last_undegraded":"2023-10-26T01:38:22.296824+0000","last_fullsized":"2023-10-26T01:38:22.296824+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T01:50:22.270135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.356583+0000","last_change":"2023-10-26T01:38:21.311895+0000","last_active":"2023-10-26T01:38:22.356583+0000","last_peered":"2023-10-26T01:38:22.356583+0000","last_clean":"2023-10-26T01:38:22.356583+0000","last_became_active":"2023-10-26T01:38:21.311281+0000","last_became_peered":"2023-10-26T01:38:21.311281+0000","last_unstale":"2023-10-26T01:38:22.356583+0000","last_undegraded":"2023-10-26T01:38:22.356583+0000","last_fullsized":"2023-10-26T01:38:22.356583+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:24:08.965712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.360039+0000","last_change":"2023-10-26T01:38:21.324857+0000","last_active":"2023-10-26T01:38:22.360039+0000","last_peered":"2023-10-26T01:38:22.360039+0000","last_clean":"2023-10-26T01:38:22.360039+0000","last_became_active":"2023-10-26T01:38:21.323546+0000","last_became_peered":"2023-10-26T01:38:21.323546+0000","last_unstale":"2023-10-26T01:38:22.360039+0000","last_undegraded":"2023-10-26T01:38:22.360039+0000","last_fullsized":"2023-10-26T01:38:22.360039+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:53:21.542954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.357316+0000","last_change":"2023-10-26T01:38:21.315041+0000","last_active":"2023-10-26T01:38:22.357316+0000","last_peered":"2023-10-26T01:38:22.357316+0000","last_clean":"2023-10-26T01:38:22.357316+0000","last_became_active":"2023-10-26T01:38:21.314746+0000","last_became_peered":"2023-10-26T01:38:21.314746+0000","last_unstale":"2023-10-26T01:38:22.357316+0000","last_undegraded":"2023-10-26T01:38:22.357316+0000","last_fullsized":"2023-10-26T01:38:22.357316+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:27:03.489686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.355991+0000","last_change":"2023-10-26T01:38:21.304208+0000","last_active":"2023-10-26T01:38:22.355991+0000","last_peered":"2023-10-26T01:38:22.355991+0000","last_clean":"2023-10-26T01:38:22.355991+0000","last_became_active":"2023-10-26T01:38:21.303492+0000","last_became_peered":"2023-10-26T01:38:21.303492+0000","last_unstale":"2023-10-26T01:38:22.355991+0000","last_undegraded":"2023-10-26T01:38:22.355991+0000","last_fullsized":"2023-10-26T01:38:22.355991+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:57:23.914586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"133'4","reported_seq":16,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.334628+0000","last_change":"2023-10-26T01:38:20.322005+0000","last_active":"2023-10-26T01:38:21.334628+0000","last_peered":"2023-10-26T01:38:21.334628+0000","last_clean":"2023-10-26T01:38:21.334628+0000","last_became_active":"2023-10-26T01:38:20.320582+0000","last_became_peered":"2023-10-26T01:38:20.320582+0000","last_unstale":"2023-10-26T01:38:21.334628+0000","last_undegraded":"2023-10-26T01:38:21.334628+0000","last_fullsized":"2023-10-26T01:38:21.334628+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:00:02.848161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.357007+0000","last_change":"2023-10-26T01:38:21.323121+0000","last_active":"2023-10-26T01:38:22.357007+0000","last_peered":"2023-10-26T01:38:22.357007+0000","last_clean":"2023-10-26T01:38:22.357007+0000","last_became_active":"2023-10-26T01:38:21.322808+0000","last_became_peered":"2023-10-26T01:38:21.322808+0000","last_unstale":"2023-10-26T01:38:22.357007+0000","last_undegraded":"2023-10-26T01:38:22.357007+0000","last_fullsized":"2023-10-26T01:38:22.357007+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:46:37.417585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"133'1","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.334372+0000","last_change":"2023-10-26T01:38:20.326148+0000","last_active":"2023-10-26T01:38:21.334372+0000","last_peered":"2023-10-26T01:38:21.334372+0000","last_clean":"2023-10-26T01:38:21.334372+0000","last_became_active":"2023-10-26T01:38:20.325883+0000","last_became_peered":"2023-10-26T01:38:20.325883+0000","last_unstale":"2023-10-26T01:38:21.334372+0000","last_undegraded":"2023-10-26T01:38:21.334372+0000","last_fullsized":"2023-10-26T01:38:21.334372+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:36:47.183420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.358082+0000","last_change":"2023-10-26T01:38:21.321027+0000","last_active":"2023-10-26T01:38:22.358082+0000","last_peered":"2023-10-26T01:38:22.358082+0000","last_clean":"2023-10-26T01:38:22.358082+0000","last_became_active":"2023-10-26T01:38:21.318702+0000","last_became_peered":"2023-10-26T01:38:21.318702+0000","last_unstale":"2023-10-26T01:38:22.358082+0000","last_undegraded":"2023-10-26T01:38:22.358082+0000","last_fullsized":"2023-10-26T01:38:22.358082+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:36:20.377483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"133'2","reported_seq":14,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.338315+0000","last_change":"2023-10-26T01:38:20.321817+0000","last_active":"2023-10-26T01:38:21.338315+0000","last_peered":"2023-10-26T01:38:21.338315+0000","last_clean":"2023-10-26T01:38:21.338315+0000","last_became_active":"2023-10-26T01:38:20.321520+0000","last_became_peered":"2023-10-26T01:38:20.321520+0000","last_unstale":"2023-10-26T01:38:21.338315+0000","last_undegraded":"2023-10-26T01:38:21.338315+0000","last_fullsized":"2023-10-26T01:38:21.338315+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:57:22.900600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.299225+0000","last_change":"2023-10-26T01:38:21.321216+0000","last_active":"2023-10-26T01:38:22.299225+0000","last_peered":"2023-10-26T01:38:22.299225+0000","last_clean":"2023-10-26T01:38:22.299225+0000","last_became_active":"2023-10-26T01:38:21.320953+0000","last_became_peered":"2023-10-26T01:38:21.320953+0000","last_unstale":"2023-10-26T01:38:22.299225+0000","last_undegraded":"2023-10-26T01:38:22.299225+0000","last_fullsized":"2023-10-26T01:38:22.299225+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:47:08.011620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"133'2","reported_seq":14,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.337393+0000","last_change":"2023-10-26T01:38:20.325154+0000","last_active":"2023-10-26T01:38:21.337393+0000","last_peered":"2023-10-26T01:38:21.337393+0000","last_clean":"2023-10-26T01:38:21.337393+0000","last_became_active":"2023-10-26T01:38:20.324072+0000","last_became_peered":"2023-10-26T01:38:20.324072+0000","last_unstale":"2023-10-26T01:38:21.337393+0000","last_undegraded":"2023-10-26T01:38:21.337393+0000","last_fullsized":"2023-10-26T01:38:21.337393+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:31:09.167929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.357578+0000","last_change":"2023-10-26T01:38:21.323345+0000","last_active":"2023-10-26T01:38:22.357578+0000","last_peered":"2023-10-26T01:38:22.357578+0000","last_clean":"2023-10-26T01:38:22.357578+0000","last_became_active":"2023-10-26T01:38:21.323051+0000","last_became_peered":"2023-10-26T01:38:21.323051+0000","last_unstale":"2023-10-26T01:38:22.357578+0000","last_undegraded":"2023-10-26T01:38:22.357578+0000","last_fullsized":"2023-10-26T01:38:22.357578+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:46:30.031341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"133'1","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.333965+0000","last_change":"2023-10-26T01:38:20.323989+0000","last_active":"2023-10-26T01:38:21.333965+0000","last_peered":"2023-10-26T01:38:21.333965+0000","last_clean":"2023-10-26T01:38:21.333965+0000","last_became_active":"2023-10-26T01:38:20.323646+0000","last_became_peered":"2023-10-26T01:38:20.323646+0000","last_unstale":"2023-10-26T01:38:21.333965+0000","last_undegraded":"2023-10-26T01:38:21.333965+0000","last_fullsized":"2023-10-26T01:38:21.333965+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:57:22.900600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.297706+0000","last_change":"2023-10-26T01:38:21.315216+0000","last_active":"2023-10-26T01:38:22.297706+0000","last_peered":"2023-10-26T01:38:22.297706+0000","last_clean":"2023-10-26T01:38:22.297706+0000","last_became_active":"2023-10-26T01:38:21.314095+0000","last_became_peered":"2023-10-26T01:38:21.314095+0000","last_unstale":"2023-10-26T01:38:22.297706+0000","last_undegraded":"2023-10-26T01:38:22.297706+0000","last_fullsized":"2023-10-26T01:38:22.297706+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:46:37.417585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"133'1","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.947947+0000","last_change":"2023-10-26T01:38:20.320626+0000","last_active":"2023-10-26T01:38:21.947947+0000","last_peered":"2023-10-26T01:38:21.947947+0000","last_clean":"2023-10-26T01:38:21.947947+0000","last_became_active":"2023-10-26T01:38:20.320400+0000","last_became_peered":"2023-10-26T01:38:20.320400+0000","last_unstale":"2023-10-26T01:38:21.947947+0000","last_undegraded":"2023-10-26T01:38:21.947947+0000","last_fullsized":"2023-10-26T01:38:21.947947+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:31:15.331694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.357408+0000","last_change":"2023-10-26T01:38:21.320573+0000","last_active":"2023-10-26T01:38:22.357408+0000","last_peered":"2023-10-26T01:38:22.357408+0000","last_clean":"2023-10-26T01:38:22.357408+0000","last_became_active":"2023-10-26T01:38:21.319770+0000","last_became_peered":"2023-10-26T01:38:21.319770+0000","last_unstale":"2023-10-26T01:38:22.357408+0000","last_undegraded":"2023-10-26T01:38:22.357408+0000","last_fullsized":"2023-10-26T01:38:22.357408+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:30:41.148919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":12,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.294385+0000","last_change":"2023-10-26T01:38:20.317888+0000","last_active":"2023-10-26T01:38:21.294385+0000","last_peered":"2023-10-26T01:38:21.294385+0000","last_clean":"2023-10-26T01:38:21.294385+0000","last_became_active":"2023-10-26T01:38:20.317130+0000","last_became_peered":"2023-10-26T01:38:20.317130+0000","last_unstale":"2023-10-26T01:38:21.294385+0000","last_undegraded":"2023-10-26T01:38:21.294385+0000","last_fullsized":"2023-10-26T01:38:21.294385+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:27:41.686982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.356245+0000","last_change":"2023-10-26T01:38:21.317074+0000","last_active":"2023-10-26T01:38:22.356245+0000","last_peered":"2023-10-26T01:38:22.356245+0000","last_clean":"2023-10-26T01:38:22.356245+0000","last_became_active":"2023-10-26T01:38:21.316665+0000","last_became_peered":"2023-10-26T01:38:21.316665+0000","last_unstale":"2023-10-26T01:38:22.356245+0000","last_undegraded":"2023-10-26T01:38:22.356245+0000","last_fullsized":"2023-10-26T01:38:22.356245+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:42:14.392665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":12,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.307295+0000","last_change":"2023-10-26T01:38:20.314210+0000","last_active":"2023-10-26T01:38:21.307295+0000","last_peered":"2023-10-26T01:38:21.307295+0000","last_clean":"2023-10-26T01:38:21.307295+0000","last_became_active":"2023-10-26T01:38:20.313976+0000","last_became_peered":"2023-10-26T01:38:20.313976+0000","last_unstale":"2023-10-26T01:38:21.307295+0000","last_undegraded":"2023-10-26T01:38:21.307295+0000","last_fullsized":"2023-10-26T01:38:21.307295+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:24:55.184951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.356077+0000","last_change":"2023-10-26T01:38:21.313827+0000","last_active":"2023-10-26T01:38:22.356077+0000","last_peered":"2023-10-26T01:38:22.356077+0000","last_clean":"2023-10-26T01:38:22.356077+0000","last_became_active":"2023-10-26T01:38:21.310630+0000","last_became_peered":"2023-10-26T01:38:21.310630+0000","last_unstale":"2023-10-26T01:38:22.356077+0000","last_undegraded":"2023-10-26T01:38:22.356077+0000","last_fullsized":"2023-10-26T01:38:22.356077+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:28:43.891836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":12,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.295375+0000","last_change":"2023-10-26T01:38:20.325020+0000","last_active":"2023-10-26T01:38:21.295375+0000","last_peered":"2023-10-26T01:38:21.295375+0000","last_clean":"2023-10-26T01:38:21.295375+0000","last_became_active":"2023-10-26T01:38:20.323713+0000","last_became_peered":"2023-10-26T01:38:20.323713+0000","last_unstale":"2023-10-26T01:38:21.295375+0000","last_undegraded":"2023-10-26T01:38:21.295375+0000","last_fullsized":"2023-10-26T01:38:21.295375+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:23:36.805508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.357896+0000","last_change":"2023-10-26T01:38:21.327094+0000","last_active":"2023-10-26T01:38:22.357896+0000","last_peered":"2023-10-26T01:38:22.357896+0000","last_clean":"2023-10-26T01:38:22.357896+0000","last_became_active":"2023-10-26T01:38:21.326783+0000","last_became_peered":"2023-10-26T01:38:21.326783+0000","last_unstale":"2023-10-26T01:38:22.357896+0000","last_undegraded":"2023-10-26T01:38:22.357896+0000","last_fullsized":"2023-10-26T01:38:22.357896+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:48:01.026075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":12,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.948015+0000","last_change":"2023-10-26T01:38:20.322090+0000","last_active":"2023-10-26T01:38:21.948015+0000","last_peered":"2023-10-26T01:38:21.948015+0000","last_clean":"2023-10-26T01:38:21.948015+0000","last_became_active":"2023-10-26T01:38:20.321451+0000","last_became_peered":"2023-10-26T01:38:20.321451+0000","last_unstale":"2023-10-26T01:38:21.948015+0000","last_undegraded":"2023-10-26T01:38:21.948015+0000","last_fullsized":"2023-10-26T01:38:21.948015+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:45:01.083199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.294768+0000","last_change":"2023-10-26T01:38:21.301523+0000","last_active":"2023-10-26T01:38:22.294768+0000","last_peered":"2023-10-26T01:38:22.294768+0000","last_clean":"2023-10-26T01:38:22.294768+0000","last_became_active":"2023-10-26T01:38:21.300901+0000","last_became_peered":"2023-10-26T01:38:21.300901+0000","last_unstale":"2023-10-26T01:38:22.294768+0000","last_undegraded":"2023-10-26T01:38:22.294768+0000","last_fullsized":"2023-10-26T01:38:22.294768+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:48:45.270090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"133'3","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.337431+0000","last_change":"2023-10-26T01:38:20.319263+0000","last_active":"2023-10-26T01:38:21.337431+0000","last_peered":"2023-10-26T01:38:21.337431+0000","last_clean":"2023-10-26T01:38:21.337431+0000","last_became_active":"2023-10-26T01:38:20.319022+0000","last_became_peered":"2023-10-26T01:38:20.319022+0000","last_unstale":"2023-10-26T01:38:21.337431+0000","last_undegraded":"2023-10-26T01:38:21.337431+0000","last_fullsized":"2023-10-26T01:38:21.337431+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:57:33.830348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.356119+0000","last_change":"2023-10-26T01:38:21.321635+0000","last_active":"2023-10-26T01:38:22.356119+0000","last_peered":"2023-10-26T01:38:22.356119+0000","last_clean":"2023-10-26T01:38:22.356119+0000","last_became_active":"2023-10-26T01:38:21.321297+0000","last_became_peered":"2023-10-26T01:38:21.321297+0000","last_unstale":"2023-10-26T01:38:22.356119+0000","last_undegraded":"2023-10-26T01:38:22.356119+0000","last_fullsized":"2023-10-26T01:38:22.356119+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:36:20.377483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"133'6","reported_seq":18,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.340461+0000","last_change":"2023-10-26T01:38:20.314532+0000","last_active":"2023-10-26T01:38:21.340461+0000","last_peered":"2023-10-26T01:38:21.340461+0000","last_clean":"2023-10-26T01:38:21.340461+0000","last_became_active":"2023-10-26T01:38:20.314073+0000","last_became_peered":"2023-10-26T01:38:20.314073+0000","last_unstale":"2023-10-26T01:38:21.340461+0000","last_undegraded":"2023-10-26T01:38:21.340461+0000","last_fullsized":"2023-10-26T01:38:21.340461+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:47:06.997633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.295366+0000","last_change":"2023-10-26T01:38:21.310272+0000","last_active":"2023-10-26T01:38:22.295366+0000","last_peered":"2023-10-26T01:38:22.295366+0000","last_clean":"2023-10-26T01:38:22.295366+0000","last_became_active":"2023-10-26T01:38:21.309297+0000","last_became_peered":"2023-10-26T01:38:21.309297+0000","last_unstale":"2023-10-26T01:38:22.295366+0000","last_undegraded":"2023-10-26T01:38:22.295366+0000","last_fullsized":"2023-10-26T01:38:22.295366+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:10:49.751492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"133'2","reported_seq":14,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.336757+0000","last_change":"2023-10-26T01:38:20.323897+0000","last_active":"2023-10-26T01:38:21.336757+0000","last_peered":"2023-10-26T01:38:21.336757+0000","last_clean":"2023-10-26T01:38:21.336757+0000","last_became_active":"2023-10-26T01:38:20.323527+0000","last_became_peered":"2023-10-26T01:38:20.323527+0000","last_unstale":"2023-10-26T01:38:21.336757+0000","last_undegraded":"2023-10-26T01:38:21.336757+0000","last_fullsized":"2023-10-26T01:38:21.336757+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:54:02.310203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.295367+0000","last_change":"2023-10-26T01:38:21.317158+0000","last_active":"2023-10-26T01:38:22.295367+0000","last_peered":"2023-10-26T01:38:22.295367+0000","last_clean":"2023-10-26T01:38:22.295367+0000","last_became_active":"2023-10-26T01:38:21.316782+0000","last_became_peered":"2023-10-26T01:38:21.316782+0000","last_unstale":"2023-10-26T01:38:22.295367+0000","last_undegraded":"2023-10-26T01:38:22.295367+0000","last_fullsized":"2023-10-26T01:38:22.295367+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:00:36.017402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":12,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.305214+0000","last_change":"2023-10-26T01:38:20.323991+0000","last_active":"2023-10-26T01:38:21.305214+0000","last_peered":"2023-10-26T01:38:21.305214+0000","last_clean":"2023-10-26T01:38:21.305214+0000","last_became_active":"2023-10-26T01:38:20.323242+0000","last_became_peered":"2023-10-26T01:38:20.323242+0000","last_unstale":"2023-10-26T01:38:21.305214+0000","last_undegraded":"2023-10-26T01:38:21.305214+0000","last_fullsized":"2023-10-26T01:38:21.305214+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:08:50.823275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.294564+0000","last_change":"2023-10-26T01:38:21.308916+0000","last_active":"2023-10-26T01:38:22.294564+0000","last_peered":"2023-10-26T01:38:22.294564+0000","last_clean":"2023-10-26T01:38:22.294564+0000","last_became_active":"2023-10-26T01:38:21.308364+0000","last_became_peered":"2023-10-26T01:38:21.308364+0000","last_unstale":"2023-10-26T01:38:22.294564+0000","last_undegraded":"2023-10-26T01:38:22.294564+0000","last_fullsized":"2023-10-26T01:38:22.294564+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:06:34.887156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"133'8","reported_seq":20,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.340109+0000","last_change":"2023-10-26T01:38:20.322830+0000","last_active":"2023-10-26T01:38:21.340109+0000","last_peered":"2023-10-26T01:38:21.340109+0000","last_clean":"2023-10-26T01:38:21.340109+0000","last_became_active":"2023-10-26T01:38:20.318642+0000","last_became_peered":"2023-10-26T01:38:20.318642+0000","last_unstale":"2023-10-26T01:38:21.340109+0000","last_undegraded":"2023-10-26T01:38:21.340109+0000","last_fullsized":"2023-10-26T01:38:21.340109+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:36:33.502627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.358107+0000","last_change":"2023-10-26T01:38:21.322681+0000","last_active":"2023-10-26T01:38:22.358107+0000","last_peered":"2023-10-26T01:38:22.358107+0000","last_clean":"2023-10-26T01:38:22.358107+0000","last_became_active":"2023-10-26T01:38:21.322406+0000","last_became_peered":"2023-10-26T01:38:21.322406+0000","last_unstale":"2023-10-26T01:38:22.358107+0000","last_undegraded":"2023-10-26T01:38:22.358107+0000","last_fullsized":"2023-10-26T01:38:22.358107+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:11:34.893562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"133'1","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.335822+0000","last_change":"2023-10-26T01:38:20.322530+0000","last_active":"2023-10-26T01:38:21.335822+0000","last_peered":"2023-10-26T01:38:21.335822+0000","last_clean":"2023-10-26T01:38:21.335822+0000","last_became_active":"2023-10-26T01:38:20.322312+0000","last_became_peered":"2023-10-26T01:38:20.322312+0000","last_unstale":"2023-10-26T01:38:21.335822+0000","last_undegraded":"2023-10-26T01:38:21.335822+0000","last_fullsized":"2023-10-26T01:38:21.335822+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:53:09.791518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.356325+0000","last_change":"2023-10-26T01:38:21.319104+0000","last_active":"2023-10-26T01:38:22.356325+0000","last_peered":"2023-10-26T01:38:22.356325+0000","last_clean":"2023-10-26T01:38:22.356325+0000","last_became_active":"2023-10-26T01:38:21.318760+0000","last_became_peered":"2023-10-26T01:38:21.318760+0000","last_unstale":"2023-10-26T01:38:22.356325+0000","last_undegraded":"2023-10-26T01:38:22.356325+0000","last_fullsized":"2023-10-26T01:38:22.356325+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:16:32.278743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"133'2","reported_seq":14,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.337172+0000","last_change":"2023-10-26T01:38:20.318723+0000","last_active":"2023-10-26T01:38:21.337172+0000","last_peered":"2023-10-26T01:38:21.337172+0000","last_clean":"2023-10-26T01:38:21.337172+0000","last_became_active":"2023-10-26T01:38:20.317546+0000","last_became_peered":"2023-10-26T01:38:20.317546+0000","last_unstale":"2023-10-26T01:38:21.337172+0000","last_undegraded":"2023-10-26T01:38:21.337172+0000","last_fullsized":"2023-10-26T01:38:21.337172+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:16:31.264756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.299270+0000","last_change":"2023-10-26T01:38:21.320514+0000","last_active":"2023-10-26T01:38:22.299270+0000","last_peered":"2023-10-26T01:38:22.299270+0000","last_clean":"2023-10-26T01:38:22.299270+0000","last_became_active":"2023-10-26T01:38:21.320080+0000","last_became_peered":"2023-10-26T01:38:21.320080+0000","last_unstale":"2023-10-26T01:38:22.299270+0000","last_undegraded":"2023-10-26T01:38:22.299270+0000","last_fullsized":"2023-10-26T01:38:22.299270+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T01:40:34.431360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"133'1","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.335116+0000","last_change":"2023-10-26T01:38:20.323687+0000","last_active":"2023-10-26T01:38:21.335116+0000","last_peered":"2023-10-26T01:38:21.335116+0000","last_clean":"2023-10-26T01:38:21.335116+0000","last_became_active":"2023-10-26T01:38:20.322524+0000","last_became_peered":"2023-10-26T01:38:20.322524+0000","last_unstale":"2023-10-26T01:38:21.335116+0000","last_undegraded":"2023-10-26T01:38:21.335116+0000","last_fullsized":"2023-10-26T01:38:21.335116+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:00:59.371192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.298852+0000","last_change":"2023-10-26T01:38:21.317249+0000","last_active":"2023-10-26T01:38:22.298852+0000","last_peered":"2023-10-26T01:38:22.298852+0000","last_clean":"2023-10-26T01:38:22.298852+0000","last_became_active":"2023-10-26T01:38:21.315753+0000","last_became_peered":"2023-10-26T01:38:21.315753+0000","last_unstale":"2023-10-26T01:38:22.298852+0000","last_undegraded":"2023-10-26T01:38:22.298852+0000","last_fullsized":"2023-10-26T01:38:22.298852+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:19:17.527922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":12,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.304169+0000","last_change":"2023-10-26T01:38:20.322266+0000","last_active":"2023-10-26T01:38:21.304169+0000","last_peered":"2023-10-26T01:38:21.304169+0000","last_clean":"2023-10-26T01:38:21.304169+0000","last_became_active":"2023-10-26T01:38:20.315231+0000","last_became_peered":"2023-10-26T01:38:20.315231+0000","last_unstale":"2023-10-26T01:38:21.304169+0000","last_undegraded":"2023-10-26T01:38:21.304169+0000","last_fullsized":"2023-10-26T01:38:21.304169+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:27:02.475699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.297531+0000","last_change":"2023-10-26T01:38:21.323986+0000","last_active":"2023-10-26T01:38:22.297531+0000","last_peered":"2023-10-26T01:38:22.297531+0000","last_clean":"2023-10-26T01:38:22.297531+0000","last_became_active":"2023-10-26T01:38:21.322360+0000","last_became_peered":"2023-10-26T01:38:21.322360+0000","last_unstale":"2023-10-26T01:38:22.297531+0000","last_undegraded":"2023-10-26T01:38:22.297531+0000","last_fullsized":"2023-10-26T01:38:22.297531+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:29:52.799943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"133'2","reported_seq":14,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.338836+0000","last_change":"2023-10-26T01:38:20.319975+0000","last_active":"2023-10-26T01:38:21.338836+0000","last_peered":"2023-10-26T01:38:21.338836+0000","last_clean":"2023-10-26T01:38:21.338836+0000","last_became_active":"2023-10-26T01:38:20.319754+0000","last_became_peered":"2023-10-26T01:38:20.319754+0000","last_unstale":"2023-10-26T01:38:21.338836+0000","last_undegraded":"2023-10-26T01:38:21.338836+0000","last_fullsized":"2023-10-26T01:38:21.338836+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:31:09.167929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.358204+0000","last_change":"2023-10-26T01:38:21.324252+0000","last_active":"2023-10-26T01:38:22.358204+0000","last_peered":"2023-10-26T01:38:22.358204+0000","last_clean":"2023-10-26T01:38:22.358204+0000","last_became_active":"2023-10-26T01:38:21.323975+0000","last_became_peered":"2023-10-26T01:38:21.323975+0000","last_unstale":"2023-10-26T01:38:22.358204+0000","last_undegraded":"2023-10-26T01:38:22.358204+0000","last_fullsized":"2023-10-26T01:38:22.358204+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:26:51.819282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":12,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.293744+0000","last_change":"2023-10-26T01:38:20.323205+0000","last_active":"2023-10-26T01:38:21.293744+0000","last_peered":"2023-10-26T01:38:21.293744+0000","last_clean":"2023-10-26T01:38:21.293744+0000","last_became_active":"2023-10-26T01:38:20.322992+0000","last_became_peered":"2023-10-26T01:38:20.322992+0000","last_unstale":"2023-10-26T01:38:21.293744+0000","last_undegraded":"2023-10-26T01:38:21.293744+0000","last_fullsized":"2023-10-26T01:38:21.293744+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:54:02.310203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.299167+0000","last_change":"2023-10-26T01:38:21.316303+0000","last_active":"2023-10-26T01:38:22.299167+0000","last_peered":"2023-10-26T01:38:22.299167+0000","last_clean":"2023-10-26T01:38:22.299167+0000","last_became_active":"2023-10-26T01:38:21.315023+0000","last_became_peered":"2023-10-26T01:38:21.315023+0000","last_unstale":"2023-10-26T01:38:22.299167+0000","last_undegraded":"2023-10-26T01:38:22.299167+0000","last_fullsized":"2023-10-26T01:38:22.299167+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T01:55:12.014852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":12,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.303952+0000","last_change":"2023-10-26T01:38:20.320536+0000","last_active":"2023-10-26T01:38:21.303952+0000","last_peered":"2023-10-26T01:38:21.303952+0000","last_clean":"2023-10-26T01:38:21.303952+0000","last_became_active":"2023-10-26T01:38:20.320277+0000","last_became_peered":"2023-10-26T01:38:20.320277+0000","last_unstale":"2023-10-26T01:38:21.303952+0000","last_undegraded":"2023-10-26T01:38:21.303952+0000","last_fullsized":"2023-10-26T01:38:21.303952+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:36:33.502627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.295720+0000","last_change":"2023-10-26T01:38:21.322582+0000","last_active":"2023-10-26T01:38:22.295720+0000","last_peered":"2023-10-26T01:38:22.295720+0000","last_clean":"2023-10-26T01:38:22.295720+0000","last_became_active":"2023-10-26T01:38:21.321971+0000","last_became_peered":"2023-10-26T01:38:21.321971+0000","last_unstale":"2023-10-26T01:38:22.295720+0000","last_undegraded":"2023-10-26T01:38:22.295720+0000","last_fullsized":"2023-10-26T01:38:22.295720+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:05:46.828567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":12,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.307293+0000","last_change":"2023-10-26T01:38:20.324450+0000","last_active":"2023-10-26T01:38:21.307293+0000","last_peered":"2023-10-26T01:38:21.307293+0000","last_clean":"2023-10-26T01:38:21.307293+0000","last_became_active":"2023-10-26T01:38:20.324194+0000","last_became_peered":"2023-10-26T01:38:20.324194+0000","last_unstale":"2023-10-26T01:38:21.307293+0000","last_undegraded":"2023-10-26T01:38:21.307293+0000","last_fullsized":"2023-10-26T01:38:21.307293+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:30:19.872449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.357340+0000","last_change":"2023-10-26T01:38:21.320203+0000","last_active":"2023-10-26T01:38:22.357340+0000","last_peered":"2023-10-26T01:38:22.357340+0000","last_clean":"2023-10-26T01:38:22.357340+0000","last_became_active":"2023-10-26T01:38:21.319553+0000","last_became_peered":"2023-10-26T01:38:21.319553+0000","last_unstale":"2023-10-26T01:38:22.357340+0000","last_undegraded":"2023-10-26T01:38:22.357340+0000","last_fullsized":"2023-10-26T01:38:22.357340+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T01:55:12.014852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"133'2","reported_seq":14,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.948037+0000","last_change":"2023-10-26T01:38:20.315954+0000","last_active":"2023-10-26T01:38:21.948037+0000","last_peered":"2023-10-26T01:38:21.948037+0000","last_clean":"2023-10-26T01:38:21.948037+0000","last_became_active":"2023-10-26T01:38:20.315727+0000","last_became_peered":"2023-10-26T01:38:20.315727+0000","last_unstale":"2023-10-26T01:38:21.948037+0000","last_undegraded":"2023-10-26T01:38:21.948037+0000","last_fullsized":"2023-10-26T01:38:21.948037+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:01:47.759723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.358948+0000","last_change":"2023-10-26T01:38:21.319574+0000","last_active":"2023-10-26T01:38:22.358948+0000","last_peered":"2023-10-26T01:38:22.358948+0000","last_clean":"2023-10-26T01:38:22.358948+0000","last_became_active":"2023-10-26T01:38:21.317481+0000","last_became_peered":"2023-10-26T01:38:21.317481+0000","last_unstale":"2023-10-26T01:38:22.358948+0000","last_undegraded":"2023-10-26T01:38:22.358948+0000","last_fullsized":"2023-10-26T01:38:22.358948+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:08:09.599874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"134'1","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.357908+0000","last_change":"2023-10-26T01:38:20.325614+0000","last_active":"2023-10-26T01:38:22.357908+0000","last_peered":"2023-10-26T01:38:22.357908+0000","last_clean":"2023-10-26T01:38:22.357908+0000","last_became_active":"2023-10-26T01:38:20.325288+0000","last_became_peered":"2023-10-26T01:38:20.325288+0000","last_unstale":"2023-10-26T01:38:22.357908+0000","last_undegraded":"2023-10-26T01:38:22.357908+0000","last_fullsized":"2023-10-26T01:38:22.357908+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:51:41.698100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.357610+0000","last_change":"2023-10-26T01:38:21.320766+0000","last_active":"2023-10-26T01:38:22.357610+0000","last_peered":"2023-10-26T01:38:22.357610+0000","last_clean":"2023-10-26T01:38:22.357610+0000","last_became_active":"2023-10-26T01:38:21.320148+0000","last_became_peered":"2023-10-26T01:38:21.320148+0000","last_unstale":"2023-10-26T01:38:22.357610+0000","last_undegraded":"2023-10-26T01:38:22.357610+0000","last_fullsized":"2023-10-26T01:38:22.357610+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:20:14.948944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":12,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.312905+0000","last_change":"2023-10-26T01:38:20.320712+0000","last_active":"2023-10-26T01:38:21.312905+0000","last_peered":"2023-10-26T01:38:21.312905+0000","last_clean":"2023-10-26T01:38:21.312905+0000","last_became_active":"2023-10-26T01:38:20.319884+0000","last_became_peered":"2023-10-26T01:38:20.319884+0000","last_unstale":"2023-10-26T01:38:21.312905+0000","last_undegraded":"2023-10-26T01:38:21.312905+0000","last_fullsized":"2023-10-26T01:38:21.312905+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:37:52.841387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.356773+0000","last_change":"2023-10-26T01:38:21.320274+0000","last_active":"2023-10-26T01:38:22.356773+0000","last_peered":"2023-10-26T01:38:22.356773+0000","last_clean":"2023-10-26T01:38:22.356773+0000","last_became_active":"2023-10-26T01:38:21.319994+0000","last_became_peered":"2023-10-26T01:38:21.319994+0000","last_unstale":"2023-10-26T01:38:22.356773+0000","last_undegraded":"2023-10-26T01:38:22.356773+0000","last_fullsized":"2023-10-26T01:38:22.356773+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:19:17.527922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":12,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.948126+0000","last_change":"2023-10-26T01:38:20.319596+0000","last_active":"2023-10-26T01:38:21.948126+0000","last_peered":"2023-10-26T01:38:21.948126+0000","last_clean":"2023-10-26T01:38:21.948126+0000","last_became_active":"2023-10-26T01:38:20.319376+0000","last_became_peered":"2023-10-26T01:38:20.319376+0000","last_unstale":"2023-10-26T01:38:21.948126+0000","last_undegraded":"2023-10-26T01:38:21.948126+0000","last_fullsized":"2023-10-26T01:38:21.948126+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:59:26.990814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.298020+0000","last_change":"2023-10-26T01:38:21.307155+0000","last_active":"2023-10-26T01:38:22.298020+0000","last_peered":"2023-10-26T01:38:22.298020+0000","last_clean":"2023-10-26T01:38:22.298020+0000","last_became_active":"2023-10-26T01:38:21.306473+0000","last_became_peered":"2023-10-26T01:38:21.306473+0000","last_unstale":"2023-10-26T01:38:22.298020+0000","last_undegraded":"2023-10-26T01:38:22.298020+0000","last_fullsized":"2023-10-26T01:38:22.298020+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T01:55:12.014852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":12,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.301190+0000","last_change":"2023-10-26T01:38:20.324083+0000","last_active":"2023-10-26T01:38:21.301190+0000","last_peered":"2023-10-26T01:38:21.301190+0000","last_clean":"2023-10-26T01:38:21.301190+0000","last_became_active":"2023-10-26T01:38:20.320737+0000","last_became_peered":"2023-10-26T01:38:20.320737+0000","last_unstale":"2023-10-26T01:38:21.301190+0000","last_undegraded":"2023-10-26T01:38:21.301190+0000","last_fullsized":"2023-10-26T01:38:21.301190+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:31:09.167929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.297522+0000","last_change":"2023-10-26T01:38:21.299637+0000","last_active":"2023-10-26T01:38:22.297522+0000","last_peered":"2023-10-26T01:38:22.297522+0000","last_clean":"2023-10-26T01:38:22.297522+0000","last_became_active":"2023-10-26T01:38:21.298931+0000","last_became_peered":"2023-10-26T01:38:21.298931+0000","last_unstale":"2023-10-26T01:38:22.297522+0000","last_undegraded":"2023-10-26T01:38:22.297522+0000","last_fullsized":"2023-10-26T01:38:22.297522+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:19:17.527922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":12,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.299217+0000","last_change":"2023-10-26T01:38:20.321433+0000","last_active":"2023-10-26T01:38:21.299217+0000","last_peered":"2023-10-26T01:38:21.299217+0000","last_clean":"2023-10-26T01:38:21.299217+0000","last_became_active":"2023-10-26T01:38:20.321136+0000","last_became_peered":"2023-10-26T01:38:20.321136+0000","last_unstale":"2023-10-26T01:38:21.299217+0000","last_undegraded":"2023-10-26T01:38:21.299217+0000","last_fullsized":"2023-10-26T01:38:21.299217+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:00:59.371192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.297193+0000","last_change":"2023-10-26T01:38:21.324339+0000","last_active":"2023-10-26T01:38:22.297193+0000","last_peered":"2023-10-26T01:38:22.297193+0000","last_clean":"2023-10-26T01:38:22.297193+0000","last_became_active":"2023-10-26T01:38:21.323704+0000","last_became_peered":"2023-10-26T01:38:21.323704+0000","last_unstale":"2023-10-26T01:38:22.297193+0000","last_undegraded":"2023-10-26T01:38:22.297193+0000","last_fullsized":"2023-10-26T01:38:22.297193+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:51:42.712087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":12,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.294774+0000","last_change":"2023-10-26T01:38:20.324901+0000","last_active":"2023-10-26T01:38:21.294774+0000","last_peered":"2023-10-26T01:38:21.294774+0000","last_clean":"2023-10-26T01:38:21.294774+0000","last_became_active":"2023-10-26T01:38:20.324441+0000","last_became_peered":"2023-10-26T01:38:20.324441+0000","last_unstale":"2023-10-26T01:38:21.294774+0000","last_undegraded":"2023-10-26T01:38:21.294774+0000","last_fullsized":"2023-10-26T01:38:21.294774+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:02:55.480456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.357911+0000","last_change":"2023-10-26T01:38:21.318362+0000","last_active":"2023-10-26T01:38:22.357911+0000","last_peered":"2023-10-26T01:38:22.357911+0000","last_clean":"2023-10-26T01:38:22.357911+0000","last_became_active":"2023-10-26T01:38:21.317684+0000","last_became_peered":"2023-10-26T01:38:21.317684+0000","last_unstale":"2023-10-26T01:38:22.357911+0000","last_undegraded":"2023-10-26T01:38:22.357911+0000","last_fullsized":"2023-10-26T01:38:22.357911+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:42:43.138486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":12,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.313106+0000","last_change":"2023-10-26T01:38:20.320871+0000","last_active":"2023-10-26T01:38:21.313106+0000","last_peered":"2023-10-26T01:38:21.313106+0000","last_clean":"2023-10-26T01:38:21.313106+0000","last_became_active":"2023-10-26T01:38:20.320224+0000","last_became_peered":"2023-10-26T01:38:20.320224+0000","last_unstale":"2023-10-26T01:38:21.313106+0000","last_undegraded":"2023-10-26T01:38:21.313106+0000","last_fullsized":"2023-10-26T01:38:21.313106+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:30:07.282370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-26T01:38:22.294720+0000","last_change":"2023-10-26T01:38:21.321771+0000","last_active":"2023-10-26T01:38:22.294720+0000","last_peered":"2023-10-26T01:38:22.294720+0000","last_clean":"2023-10-26T01:38:22.294720+0000","last_became_active":"2023-10-26T01:38:21.321349+0000","last_became_peered":"2023-10-26T01:38:21.321349+0000","last_unstale":"2023-10-26T01:38:22.294720+0000","last_undegraded":"2023-10-26T01:38:22.294720+0000","last_fullsized":"2023-10-26T01:38:22.294720+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:20.280569+0000","last_clean_scrub_stamp":"2023-10-26T01:38:20.280569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:11:23.831539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":12,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.303830+0000","last_change":"2023-10-26T01:38:20.321951+0000","last_active":"2023-10-26T01:38:21.303830+0000","last_peered":"2023-10-26T01:38:21.303830+0000","last_clean":"2023-10-26T01:38:21.303830+0000","last_became_active":"2023-10-26T01:38:20.310341+0000","last_became_peered":"2023-10-26T01:38:20.310341+0000","last_unstale":"2023-10-26T01:38:21.303830+0000","last_undegraded":"2023-10-26T01:38:21.303830+0000","last_fullsized":"2023-10-26T01:38:21.303830+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:38:19.266582+0000","last_clean_scrub_stamp":"2023-10-26T01:38:19.266582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:11:33.879576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":278,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-26T01:38:21.315091+0000","last_change":"2023-10-26T01:30:34.883484+0000","last_active":"2023-10-26T01:38:21.315091+0000","last_peered":"2023-10-26T01:38:21.315091+0000","last_clean":"2023-10-26T01:38:21.315091+0000","last_became_active":"2023-10-26T01:30:34.883071+0000","last_became_peered":"2023-10-26T01:30:34.883071+0000","last_unstale":"2023-10-26T01:38:21.315091+0000","last_undegraded":"2023-10-26T01:38:21.315091+0000","last_fullsized":"2023-10-26T01:38:21.315091+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:32.890096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:32.890096+0000","last_clean_scrub_stamp":"2023-10-26T01:30:32.890096+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:26:31.879452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":14,"seq":60129542242,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":725,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194586,"statfs":{"total":200000000,"available":199256622,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.79200000000000004,"15min":0.78600000000000003},"min":{"1min":0.44900000000000001,"5min":0.35599999999999998,"15min":0.33000000000000002},"max":{"1min":1.6319999999999999,"5min":1.6319999999999999,"15min":2.2549999999999999},"last":1.2549999999999999},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.876,"15min":0.88800000000000001},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.236,"5min":1.5800000000000001,"15min":1.8200000000000001},"last":0.92500000000000004}]},{"osd":1,"last update":"Thu Oct 26 01:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.875,"15min":0.93700000000000006},"min":{"1min":0.40699999999999997,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.3560000000000001,"5min":1.444,"15min":4.4340000000000002},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.97599999999999998,"15min":0.997},"min":{"1min":0.51500000000000001,"5min":0.46700000000000003,"15min":0.374},"max":{"1min":1.4390000000000001,"5min":1.538,"15min":2.3330000000000002},"last":1}]},{"osd":2,"last update":"Thu Oct 26 01:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":1.032,"15min":1.075},"min":{"1min":0.53800000000000003,"5min":0.40999999999999998,"15min":0.39900000000000002},"max":{"1min":1.643,"5min":1.927,"15min":4.2800000000000002},"last":1.111},{"interface":"front","average":{"1min":1.032,"5min":1.1279999999999999,"15min":1.091},"min":{"1min":0.50900000000000001,"5min":0.495,"15min":0.49399999999999999},"max":{"1min":1.607,"5min":1.8680000000000001,"15min":2.302},"last":1.395}]},{"osd":4,"last update":"Thu Oct 26 01:37:49 2023","interfaces":[{"interface":"back","average":{"1min":1.175,"5min":1.2070000000000001,"15min":1.167},"min":{"1min":0.755,"5min":0.755,"15min":0.66400000000000003},"max":{"1min":1.536,"5min":1.7150000000000001,"15min":2.1070000000000002},"last":1.46},{"interface":"front","average":{"1min":1.252,"5min":1.3400000000000001,"15min":1.389},"min":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.79900000000000004},"max":{"1min":1.5209999999999999,"5min":2.4239999999999999,"15min":4.8250000000000002},"last":1.593}]},{"osd":5,"last update":"Thu Oct 26 01:37:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.282,"15min":1.333},"min":{"1min":0.79300000000000004,"5min":0.76400000000000001,"15min":0.72499999999999998},"max":{"1min":1.639,"5min":2.3380000000000001,"15min":4.9080000000000004},"last":1.4099999999999999},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.28,"15min":1.276},"min":{"1min":0.90300000000000002,"5min":0.79900000000000004,"15min":0.79600000000000004},"max":{"1min":1.722,"5min":2.0310000000000001,"15min":2.8980000000000001},"last":1.5069999999999999}]},{"osd":6,"last update":"Thu Oct 26 01:37:49 2023","interfaces":[{"interface":"back","average":{"1min":1.278,"5min":1.355,"15min":1.331},"min":{"1min":0.74199999999999999,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.5580000000000001,"5min":2.1499999999999999,"15min":2.8530000000000002},"last":1.3420000000000001},{"interface":"front","average":{"1min":1.206,"5min":1.268,"15min":1.268},"min":{"1min":0.81699999999999995,"5min":0.81499999999999995,"15min":0.73099999999999998},"max":{"1min":1.7410000000000001,"5min":2.048,"15min":2.181},"last":1.556}]},{"osd":7,"last update":"Thu Oct 26 01:37:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.3300000000000001,"15min":1.296},"min":{"1min":0.85299999999999998,"5min":0.84599999999999997,"15min":0.71999999999999997},"max":{"1min":1.8129999999999999,"5min":1.958,"15min":2.0680000000000001},"last":1.655},{"interface":"front","average":{"1min":1.2889999999999999,"5min":1.3919999999999999,"15min":1.415},"min":{"1min":0.84199999999999997,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.647,"5min":2.2370000000000001,"15min":4.8730000000000002},"last":1.702}]}]},{"osd":7,"up_from":13,"seq":55834574946,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1301,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194011,"statfs":{"total":200000000,"available":198667320,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:37:49 2023","interfaces":[{"interface":"back","average":{"1min":1.0920000000000001,"5min":1.139,"15min":1.093},"min":{"1min":0.81799999999999995,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.7,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":1.4350000000000001},{"interface":"front","average":{"1min":0.99099999999999999,"5min":1.115,"15min":1.081},"min":{"1min":0.70599999999999996,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.46,"5min":2.6779999999999999,"15min":2.6779999999999999},"last":1.28}]},{"osd":1,"last update":"Thu Oct 26 01:37:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.1870000000000001,"15min":1.0960000000000001},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65200000000000002},"max":{"1min":1.5549999999999999,"5min":2.8260000000000001,"15min":2.8260000000000001},"last":1.206},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.21,"15min":1.194},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.61199999999999999},"max":{"1min":1.7869999999999999,"5min":3.2730000000000001,"15min":3.2730000000000001},"last":1.3779999999999999}]},{"osd":2,"last update":"Thu Oct 26 01:37:49 2023","interfaces":[{"interface":"back","average":{"1min":1.143,"5min":1.179,"15min":1.1180000000000001},"min":{"1min":0.79100000000000004,"5min":0.65700000000000003,"15min":0.58499999999999996},"max":{"1min":1.667,"5min":2.218,"15min":2.218},"last":1.4850000000000001},{"interface":"front","average":{"1min":1.2709999999999999,"5min":1.2989999999999999,"15min":1.24},"min":{"1min":0.89200000000000002,"5min":0.79400000000000004,"15min":0.73099999999999998},"max":{"1min":1.786,"5min":3.0510000000000002,"15min":3.0510000000000002},"last":1.077}]},{"osd":3,"last update":"Thu Oct 26 01:37:49 2023","interfaces":[{"interface":"back","average":{"1min":1.264,"5min":1.284,"15min":1.214},"min":{"1min":0.93500000000000005,"5min":0.79000000000000004,"15min":0.72099999999999997},"max":{"1min":1.5389999999999999,"5min":1.9830000000000001,"15min":1.9830000000000001},"last":1.1160000000000001},{"interface":"front","average":{"1min":1.296,"5min":1.407,"15min":1.373},"min":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.65700000000000003},"max":{"1min":1.8939999999999999,"5min":5.5039999999999996,"15min":5.5039999999999996},"last":1.028}]},{"osd":4,"last update":"Thu Oct 26 01:37:49 2023","interfaces":[{"interface":"back","average":{"1min":1.4350000000000001,"5min":1.373,"15min":1.3520000000000001},"min":{"1min":1.0589999999999999,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.7450000000000001,"5min":2.77,"15min":2.77},"last":1.615},{"interface":"front","average":{"1min":1.456,"5min":1.429,"15min":1.399},"min":{"1min":0.92700000000000005,"5min":0.90400000000000003,"15min":0.496},"max":{"1min":1.95,"5min":3.1419999999999999,"15min":3.1419999999999999},"last":0.99099999999999999}]},{"osd":5,"last update":"Thu Oct 26 01:37:49 2023","interfaces":[{"interface":"back","average":{"1min":1.498,"5min":1.488,"15min":1.4259999999999999},"min":{"1min":0.91700000000000004,"5min":0.65600000000000003,"15min":0.441},"max":{"1min":1.9890000000000001,"5min":5.734,"15min":5.734},"last":0.91900000000000004},{"interface":"front","average":{"1min":1.4970000000000001,"5min":1.5269999999999999,"15min":1.478},"min":{"1min":1.1739999999999999,"5min":0.96099999999999997,"15min":0.51900000000000002},"max":{"1min":1.8799999999999999,"5min":2.8980000000000001,"15min":2.8980000000000001},"last":1.3360000000000001}]},{"osd":6,"last update":"Thu Oct 26 01:37:49 2023","interfaces":[{"interface":"back","average":{"1min":1.5580000000000001,"5min":1.6080000000000001,"15min":1.8460000000000001},"min":{"1min":1.0169999999999999,"5min":0.753,"15min":0.47399999999999998},"max":{"1min":2.0739999999999998,"5min":5.6340000000000003,"15min":11.577999999999999},"last":1.526},{"interface":"front","average":{"1min":1.615,"5min":1.6779999999999999,"15min":1.8620000000000001},"min":{"1min":1.2889999999999999,"5min":0.91800000000000004,"15min":0.55100000000000005},"max":{"1min":1.986,"5min":5.0919999999999996,"15min":10.704000000000001},"last":1.5549999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574946,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":723,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194588,"statfs":{"total":200000000,"available":199258848,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:37:43 2023","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.113,"15min":1.1539999999999999},"min":{"1min":0.69299999999999995,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.6100000000000001,"5min":2.395,"15min":2.395},"last":1.135},{"interface":"front","average":{"1min":1.139,"5min":1.173,"15min":1.1220000000000001},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.59199999999999997},"max":{"1min":2.081,"5min":2.081,"15min":2.081},"last":0.72299999999999998}]},{"osd":1,"last update":"Thu Oct 26 01:37:43 2023","interfaces":[{"interface":"back","average":{"1min":1.2749999999999999,"5min":1.2110000000000001,"15min":1.1779999999999999},"min":{"1min":0.82499999999999996,"5min":0.71599999999999997,"15min":0.68300000000000005},"max":{"1min":1.992,"5min":2.0529999999999999,"15min":2.0529999999999999},"last":1.0169999999999999},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.179,"15min":1.204},"min":{"1min":0.67900000000000005,"5min":0.67700000000000005,"15min":0.64000000000000001},"max":{"1min":1.968,"5min":1.968,"15min":2.085},"last":0.82999999999999996}]},{"osd":2,"last update":"Thu Oct 26 01:37:43 2023","interfaces":[{"interface":"back","average":{"1min":1.077,"5min":1.1879999999999999,"15min":1.095},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.5649999999999999,"5min":3.7879999999999998,"15min":3.7879999999999998},"last":0.76300000000000001},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.3120000000000001,"15min":1.268},"min":{"1min":0.90600000000000003,"5min":0.75800000000000001,"15min":0.68899999999999995},"max":{"1min":2.2200000000000002,"5min":2.2890000000000001,"15min":2.2890000000000001},"last":1.379}]},{"osd":3,"last update":"Thu Oct 26 01:37:43 2023","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.319,"15min":1.214},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.756,"5min":4.4560000000000004,"15min":4.4560000000000004},"last":0.92200000000000004},{"interface":"front","average":{"1min":1.288,"5min":1.337,"15min":1.325},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.69499999999999995},"max":{"1min":2.1520000000000001,"5min":2.5019999999999998,"15min":2.5019999999999998},"last":1.484}]},{"osd":4,"last update":"Thu Oct 26 01:37:43 2023","interfaces":[{"interface":"back","average":{"1min":1.2709999999999999,"5min":1.272,"15min":1.224},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.48399999999999999},"max":{"1min":1.754,"5min":2.9489999999999998,"15min":2.9489999999999998},"last":0.97699999999999998},{"interface":"front","average":{"1min":1.272,"5min":1.4299999999999999,"15min":1.3149999999999999},"min":{"1min":0.96799999999999997,"5min":0.81100000000000005,"15min":0.59299999999999997},"max":{"1min":1.7629999999999999,"5min":6.609,"15min":6.609},"last":1.321}]},{"osd":5,"last update":"Thu Oct 26 01:37:43 2023","interfaces":[{"interface":"back","average":{"1min":1.4570000000000001,"5min":1.5469999999999999,"15min":1.4470000000000001},"min":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.67000000000000004},"max":{"1min":1.8380000000000001,"5min":5.4370000000000003,"15min":5.4370000000000003},"last":1.2230000000000001},{"interface":"front","average":{"1min":1.4370000000000001,"5min":1.4410000000000001,"15min":1.4119999999999999},"min":{"1min":0.626,"5min":0.61399999999999999,"15min":0.58199999999999996},"max":{"1min":2.3279999999999998,"5min":2.621,"15min":2.621},"last":1.554}]},{"osd":7,"last update":"Thu Oct 26 01:37:43 2023","interfaces":[{"interface":"back","average":{"1min":1.3400000000000001,"5min":1.52,"15min":1.4099999999999999},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.54100000000000004},"max":{"1min":1.8740000000000001,"5min":11.804,"15min":11.804},"last":1.085},{"interface":"front","average":{"1min":1.5049999999999999,"5min":1.6559999999999999,"15min":1.51},"min":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.77300000000000002},"max":{"1min":2.0710000000000002,"5min":11.541,"15min":11.541},"last":1.4279999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574946,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":724,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194587,"statfs":{"total":200000000,"available":199257700,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:37:46 2023","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.1870000000000001,"15min":1.129},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.5660000000000001,"5min":1.7390000000000001,"15min":1.7390000000000001},"last":1.2150000000000001},{"interface":"front","average":{"1min":1.204,"5min":1.28,"15min":1.1819999999999999},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.613,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":1.4470000000000001}]},{"osd":1,"last update":"Thu Oct 26 01:37:46 2023","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":1.1319999999999999,"15min":1.085},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.3640000000000001,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":1.4019999999999999},{"interface":"front","average":{"1min":1.069,"5min":1.137,"15min":1.095},"min":{"1min":0.629,"5min":0.629,"15min":0.60599999999999998},"max":{"1min":1.726,"5min":1.726,"15min":1.7669999999999999},"last":1.274}]},{"osd":2,"last update":"Thu Oct 26 01:37:46 2023","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":1.1950000000000001,"15min":1.163},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.69299999999999995},"max":{"1min":1.681,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":1.242},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.2290000000000001,"15min":1.181},"min":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.63900000000000001},"max":{"1min":1.4339999999999999,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":1.169}]},{"osd":3,"last update":"Thu Oct 26 01:37:46 2023","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.2949999999999999,"15min":1.238},"min":{"1min":0.78900000000000003,"5min":0.72699999999999998,"15min":0.72299999999999998},"max":{"1min":1.6180000000000001,"5min":1.7649999999999999,"15min":1.835},"last":1.6459999999999999},{"interface":"front","average":{"1min":1.2529999999999999,"5min":1.3220000000000001,"15min":1.2929999999999999},"min":{"1min":0.85199999999999998,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.7310000000000001,"5min":1.788,"15min":2.9249999999999998},"last":1.343}]},{"osd":4,"last update":"Thu Oct 26 01:37:46 2023","interfaces":[{"interface":"back","average":{"1min":1.1519999999999999,"5min":1.2689999999999999,"15min":1.2050000000000001},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.49099999999999999},"max":{"1min":1.6719999999999999,"5min":1.845,"15min":1.845},"last":1.3740000000000001},{"interface":"front","average":{"1min":1.3340000000000001,"5min":1.4359999999999999,"15min":1.355},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.56200000000000006},"max":{"1min":1.78,"5min":1.867,"15min":1.867},"last":1.0589999999999999}]},{"osd":6,"last update":"Thu Oct 26 01:37:46 2023","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.4319999999999999,"15min":1.323},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.51900000000000002},"max":{"1min":1.7490000000000001,"5min":2.8250000000000002,"15min":2.8250000000000002},"last":1.2929999999999999},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.423,"15min":1.327},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.61799999999999999},"max":{"1min":1.661,"5min":1.849,"15min":1.8560000000000001},"last":1.119}]},{"osd":7,"last update":"Thu Oct 26 01:37:46 2023","interfaces":[{"interface":"back","average":{"1min":1.4219999999999999,"5min":1.536,"15min":1.4219999999999999},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.61699999999999999},"max":{"1min":1.8220000000000001,"5min":2.1040000000000001,"15min":2.1040000000000001},"last":1.5049999999999999},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.5609999999999999,"15min":1.431},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.59499999999999997},"max":{"1min":1.853,"5min":2.0960000000000001,"15min":2.0960000000000001},"last":1.5509999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574946,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":724,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194588,"statfs":{"total":200000000,"available":199258304,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:37:42 2023","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.056,"15min":0.98499999999999999},"min":{"1min":0.63800000000000001,"5min":0.58099999999999996,"15min":0.53100000000000003},"max":{"1min":1.5369999999999999,"5min":1.885,"15min":2.0289999999999999},"last":0.84299999999999997},{"interface":"front","average":{"1min":1.0309999999999999,"5min":1.081,"15min":1.024},"min":{"1min":0.74099999999999999,"5min":0.66400000000000003,"15min":0.60599999999999998},"max":{"1min":1.5029999999999999,"5min":2.6829999999999998,"15min":2.6829999999999998},"last":0.68899999999999995}]},{"osd":1,"last update":"Thu Oct 26 01:37:42 2023","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.2,"15min":1.127},"min":{"1min":0.66700000000000004,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.591,"5min":4.0519999999999996,"15min":4.0519999999999996},"last":1.022},{"interface":"front","average":{"1min":1.1599999999999999,"5min":1.1559999999999999,"15min":1.1100000000000001},"min":{"1min":0.70399999999999996,"5min":0.627,"15min":0.60499999999999998},"max":{"1min":1.897,"5min":1.897,"15min":2.0760000000000001},"last":0.95099999999999996}]},{"osd":2,"last update":"Thu Oct 26 01:37:42 2023","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.3480000000000001,"15min":1.23},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.64300000000000002},"max":{"1min":1.845,"5min":5.6749999999999998,"15min":5.6749999999999998},"last":1.2010000000000001},{"interface":"front","average":{"1min":1.3,"5min":1.218,"15min":1.1359999999999999},"min":{"1min":0.73899999999999999,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":3.673,"5min":3.673,"15min":3.673},"last":1.4590000000000001}]},{"osd":3,"last update":"Thu Oct 26 01:37:42 2023","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.204,"15min":1.1870000000000001},"min":{"1min":0.91500000000000004,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.7669999999999999,"5min":1.847,"15min":1.891},"last":1.2430000000000001},{"interface":"front","average":{"1min":1.194,"5min":1.21,"15min":1.159},"min":{"1min":0.85399999999999998,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.6419999999999999,"5min":1.7210000000000001,"15min":1.7210000000000001},"last":1.29}]},{"osd":5,"last update":"Thu Oct 26 01:37:42 2023","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":1.2549999999999999,"15min":1.286},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.46300000000000002},"max":{"1min":1.8520000000000001,"5min":2.1429999999999998,"15min":2.1429999999999998},"last":1.415},{"interface":"front","average":{"1min":1.3460000000000001,"5min":1.3500000000000001,"15min":1.3280000000000001},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.7649999999999999,"5min":5.8579999999999997,"15min":5.8579999999999997},"last":0.90400000000000003}]},{"osd":6,"last update":"Thu Oct 26 01:37:42 2023","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":1.427,"15min":1.3380000000000001},"min":{"1min":0.69199999999999995,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":2.2149999999999999,"5min":12.343999999999999,"15min":12.343999999999999},"last":0.97499999999999998},{"interface":"front","average":{"1min":1.492,"5min":1.619,"15min":1.5149999999999999},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.55000000000000004},"max":{"1min":2.1000000000000001,"5min":12.605,"15min":12.605},"last":1.488}]},{"osd":7,"last update":"Thu Oct 26 01:37:42 2023","interfaces":[{"interface":"back","average":{"1min":1.4710000000000001,"5min":1.52,"15min":1.49},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":2.0390000000000001,"5min":5.774,"15min":5.774},"last":1.341},{"interface":"front","average":{"1min":1.53,"5min":1.5309999999999999,"15min":1.46},"min":{"1min":0.79000000000000004,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":2.3450000000000002,"5min":4.6520000000000001,"15min":4.6520000000000001},"last":1.1499999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574946,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":724,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194588,"statfs":{"total":200000000,"available":199258304,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:37:46 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.84099999999999997,"15min":0.85499999999999998},"min":{"1min":0.41199999999999998,"5min":0.308,"15min":0.308},"max":{"1min":1.4850000000000001,"5min":1.5249999999999999,"15min":6.3200000000000003},"last":1.4079999999999999},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.997,"15min":0.93100000000000005},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.2569999999999999,"5min":2.1400000000000001,"15min":4.2619999999999996},"last":0.74099999999999999}]},{"osd":1,"last update":"Thu Oct 26 01:37:46 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.90200000000000002,"15min":0.89700000000000002},"min":{"1min":0.39300000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.244,"5min":1.629,"15min":5.1260000000000003},"last":0.877},{"interface":"front","average":{"1min":0.85399999999999998,"5min":1.018,"15min":1.0029999999999999},"min":{"1min":0.47699999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.351,"5min":1.873,"15min":6.1520000000000001},"last":0.78200000000000003}]},{"osd":3,"last update":"Thu Oct 26 01:37:48 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":1.0189999999999999,"15min":0.93899999999999995},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.443},"max":{"1min":1.4390000000000001,"5min":2.0640000000000001,"15min":2.0640000000000001},"last":0.96599999999999997},{"interface":"front","average":{"1min":0.93600000000000005,"5min":1.111,"15min":1.018},"min":{"1min":0.51100000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.375,"5min":1.9279999999999999,"15min":8.9019999999999992},"last":1.089}]},{"osd":4,"last update":"Thu Oct 26 01:37:46 2023","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.3420000000000001,"15min":1.2689999999999999},"min":{"1min":0.86499999999999999,"5min":0.76000000000000001,"15min":0.73899999999999999},"max":{"1min":1.704,"5min":2.6190000000000002,"15min":6.2670000000000003},"last":1.1279999999999999},{"interface":"front","average":{"1min":1.1879999999999999,"5min":1.3129999999999999,"15min":1.262},"min":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.78700000000000003},"max":{"1min":1.5920000000000001,"5min":2.4359999999999999,"15min":7.343},"last":1.385}]},{"osd":5,"last update":"Thu Oct 26 01:37:46 2023","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.3540000000000001,"15min":1.2869999999999999},"min":{"1min":0.873,"5min":0.81699999999999995,"15min":0.71899999999999997},"max":{"1min":1.5740000000000001,"5min":2.5790000000000002,"15min":3.7170000000000001},"last":1.446},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.3089999999999999,"15min":1.2909999999999999},"min":{"1min":0.81899999999999995,"5min":0.70999999999999996,"15min":0.65300000000000002},"max":{"1min":1.621,"5min":2.298,"15min":6.3899999999999997},"last":1.28}]},{"osd":6,"last update":"Thu Oct 26 01:37:46 2023","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":1.399,"15min":1.335},"min":{"1min":0.81200000000000006,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.9379999999999999,"5min":2.73,"15min":6.0490000000000004},"last":1.3400000000000001},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.4179999999999999,"15min":1.3720000000000001},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.76800000000000002},"max":{"1min":1.8799999999999999,"5min":2.4089999999999998,"15min":5.0650000000000004},"last":1.2470000000000001}]},{"osd":7,"last update":"Thu Oct 26 01:37:46 2023","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.403,"15min":1.395},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.69399999999999995},"max":{"1min":1.7809999999999999,"5min":2.3370000000000002,"15min":6.4480000000000004},"last":1.528},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.52,"15min":1.4550000000000001},"min":{"1min":1.036,"5min":0.91800000000000004,"15min":0.91800000000000004},"max":{"1min":1.877,"5min":2.8290000000000002,"15min":2.8439999999999999},"last":1.472}]}]},{"osd":1,"up_from":13,"seq":55834574946,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":724,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194588,"statfs":{"total":200000000,"available":199258180,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:37:48 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.78300000000000003,"15min":0.72499999999999998},"min":{"1min":0.36099999999999999,"5min":0.34000000000000002,"15min":0.29499999999999998},"max":{"1min":1.4099999999999999,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.88},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.86799999999999999,"15min":0.79600000000000004},"min":{"1min":0.435,"5min":0.36299999999999999,"15min":0.36199999999999999},"max":{"1min":1.4019999999999999,"5min":1.6399999999999999,"15min":1.6399999999999999},"last":1.2769999999999999}]},{"osd":2,"last update":"Thu Oct 26 01:37:48 2023","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":0.97899999999999998,"15min":1.0009999999999999},"min":{"1min":0.52200000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.8200000000000001,"5min":1.8200000000000001,"15min":2.4830000000000001},"last":1.0489999999999999},{"interface":"front","average":{"1min":1.0569999999999999,"5min":0.93700000000000006,"15min":0.875},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.772,"5min":1.8080000000000001,"15min":1.8080000000000001},"last":1.073}]},{"osd":3,"last update":"Thu Oct 26 01:37:52 2023","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.97899999999999998,"15min":0.94899999999999995},"min":{"1min":0.52900000000000003,"5min":0.42499999999999999,"15min":0.39500000000000002},"max":{"1min":1.288,"5min":1.611,"15min":1.75},"last":1.2310000000000001},{"interface":"front","average":{"1min":0.97899999999999998,"5min":1.028,"15min":0.98999999999999999},"min":{"1min":0.55900000000000005,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.45,"5min":1.673,"15min":1.802},"last":0.97099999999999997}]},{"osd":4,"last update":"Thu Oct 26 01:37:48 2023","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":1.2270000000000001,"15min":1.1990000000000001},"min":{"1min":0.82899999999999996,"5min":0.69099999999999995,"15min":0.56200000000000006},"max":{"1min":1.756,"5min":1.756,"15min":2.0659999999999998},"last":1.3240000000000001},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.1930000000000001,"15min":1.1359999999999999},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.474,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":1.117}]},{"osd":5,"last update":"Thu Oct 26 01:37:48 2023","interfaces":[{"interface":"back","average":{"1min":1.1459999999999999,"5min":1.262,"15min":1.2270000000000001},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.573,"5min":1.8779999999999999,"15min":1.8779999999999999},"last":1.4530000000000001},{"interface":"front","average":{"1min":1.3720000000000001,"5min":1.3720000000000001,"15min":1.337},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":2.04,"5min":2.04,"15min":2.2200000000000002},"last":1.262}]},{"osd":6,"last update":"Thu Oct 26 01:37:48 2023","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":1.3320000000000001,"15min":1.28},"min":{"1min":0.75,"5min":0.70199999999999996,"15min":0.68899999999999995},"max":{"1min":1.7330000000000001,"5min":2.0910000000000002,"15min":2.3780000000000001},"last":1.194},{"interface":"front","average":{"1min":1.26,"5min":1.2729999999999999,"15min":1.244},"min":{"1min":0.80700000000000005,"5min":0.71299999999999997,"15min":0.70799999999999996},"max":{"1min":1.669,"5min":1.956,"15min":1.984},"last":1.429}]},{"osd":7,"last update":"Thu Oct 26 01:37:48 2023","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.2629999999999999,"15min":1.262},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.69699999999999995},"max":{"1min":1.7230000000000001,"5min":1.8380000000000001,"15min":2.6930000000000001},"last":1.3560000000000001},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.4019999999999999,"15min":1.337},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.9370000000000001,"5min":2.0190000000000001,"15min":2.0190000000000001},"last":1.3089999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574946,"num_pgs":36,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1302,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194009,"statfs":{"total":200000000,"available":198666212,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Oct 26 01:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":1.0029999999999999,"15min":0.91600000000000004},"min":{"1min":0.41699999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.2170000000000001,"5min":19.425000000000001,"15min":19.425000000000001},"last":1.026},{"interface":"front","average":{"1min":0.84199999999999997,"5min":1.0720000000000001,"15min":1.042},"min":{"1min":0.56100000000000005,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.1839999999999999,"5min":15.162000000000001,"15min":15.162000000000001},"last":0.99099999999999999}]},{"osd":2,"last update":"Thu Oct 26 01:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":1.3080000000000001,"15min":1.1819999999999999},"min":{"1min":0.57699999999999996,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.4930000000000001,"5min":28.439,"15min":28.439},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.91700000000000004,"15min":0.91200000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.4810000000000001,"5min":1.502,"15min":4.9459999999999997},"last":1.2969999999999999}]},{"osd":3,"last update":"Thu Oct 26 01:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":1.0549999999999999,"15min":1.085},"min":{"1min":0.48899999999999999,"5min":0.46000000000000002,"15min":0.44600000000000001},"max":{"1min":1.4330000000000001,"5min":1.6479999999999999,"15min":4.9800000000000004},"last":1.4390000000000001},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.1459999999999999,"15min":1.155},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.4770000000000001,"5min":1.659,"15min":5.016},"last":0.94399999999999995}]},{"osd":4,"last update":"Thu Oct 26 01:37:45 2023","interfaces":[{"interface":"back","average":{"1min":1.091,"5min":1.379,"15min":1.29},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.49,"5min":19.329000000000001,"15min":19.329000000000001},"last":1.2},{"interface":"front","average":{"1min":1.22,"5min":1.268,"15min":1.2989999999999999},"min":{"1min":0.88,"5min":0.82099999999999995,"15min":0.67600000000000005},"max":{"1min":1.5449999999999999,"5min":1.718,"15min":5.5229999999999997},"last":1.3520000000000001}]},{"osd":5,"last update":"Thu Oct 26 01:37:45 2023","interfaces":[{"interface":"back","average":{"1min":1.373,"5min":1.4039999999999999,"15min":1.4299999999999999},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.661,"5min":1.915,"15min":5.6360000000000001},"last":1.5649999999999999},{"interface":"front","average":{"1min":1.202,"5min":1.4790000000000001,"15min":1.3999999999999999},"min":{"1min":0.73599999999999999,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.71,"5min":19.384,"15min":19.384},"last":1.5880000000000001}]},{"osd":6,"last update":"Thu Oct 26 01:37:45 2023","interfaces":[{"interface":"back","average":{"1min":1.2609999999999999,"5min":1.3799999999999999,"15min":1.365},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.73799999999999999},"max":{"1min":1.615,"5min":8.2609999999999992,"15min":8.2609999999999992},"last":1.4079999999999999},{"interface":"front","average":{"1min":1.3440000000000001,"5min":1.4590000000000001,"15min":1.407},"min":{"1min":0.83099999999999996,"5min":0.77900000000000003,"15min":0.75900000000000001},"max":{"1min":1.804,"5min":11.849,"15min":11.849},"last":1.4730000000000001}]},{"osd":7,"last update":"Thu Oct 26 01:37:45 2023","interfaces":[{"interface":"back","average":{"1min":1.3979999999999999,"5min":1.4450000000000001,"15min":1.4810000000000001},"min":{"1min":0.871,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.8380000000000001,"5min":2.4060000000000001,"15min":5.6689999999999996},"last":1.3899999999999999},{"interface":"front","average":{"1min":1.365,"5min":1.6120000000000001,"15min":1.5029999999999999},"min":{"1min":0.89600000000000002,"5min":0.879,"15min":0.69399999999999995},"max":{"1min":1.841,"5min":19.245999999999999,"15min":19.245999999999999},"last":1.5169999999999999}]}]}],"pool_statfs":[]}} 2023-10-26T01:38:31.328 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-26T01:38:31.328 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-26T01:38:31.595 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:31.593+0000 7f1d094e5640 1 -- 172.21.15.169:0/3402884783 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d0415bbb0 msgr2=0x7f1d040b05a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:31.596 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:31.593+0000 7f1d094e5640 1 --2- 172.21.15.169:0/3402884783 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d0415bbb0 0x7f1d040b05a0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f1d040b0570 tx=0x7f1cfc02be30 comp rx=0 tx=0).stop 2023-10-26T01:38:31.596 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:31.593+0000 7f1d094e5640 1 -- 172.21.15.169:0/3402884783 shutdown_connections 2023-10-26T01:38:31.596 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:31.593+0000 7f1d094e5640 1 --2- 172.21.15.169:0/3402884783 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d0415bbb0 0x7f1d040b05a0 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:31.596 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:31.593+0000 7f1d094e5640 1 --2- 172.21.15.169:0/3402884783 >> v2:172.21.15.170:3300/0 conn(0x7f1d04155360 0x7f1d040cd7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:31.596 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:31.593+0000 7f1d094e5640 1 -- 172.21.15.169:0/3402884783 >> 172.21.15.169:0/3402884783 conn(0x7f1d04069930 msgr2=0x7f1d04069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:31.597 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:31.594+0000 7f1d094e5640 1 -- 172.21.15.169:0/3402884783 shutdown_connections 2023-10-26T01:38:31.597 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:31.594+0000 7f1d094e5640 1 -- 172.21.15.169:0/3402884783 wait complete. 2023-10-26T01:38:31.599 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:31.595+0000 7f1d094e5640 1 Processor -- start 2023-10-26T01:38:31.599 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:31.595+0000 7f1d094e5640 1 -- start start 2023-10-26T01:38:31.600 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:31.596+0000 7f1d094e5640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d040cdd20 0x7f1d04155a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:31.600 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:31.596+0000 7f1d094e5640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1d04155360 0x7f1d0406ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:31.600 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:31.596+0000 7f1d094e5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d0415bbb0 0x7f1d0414aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:31.600 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:31.596+0000 7f1d094e5640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d0414b0e0 con 0x7f1d040cdd20 2023-10-26T01:38:31.600 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:31.596+0000 7f1d094e5640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d040afcf0 con 0x7f1d0415bbb0 2023-10-26T01:38:31.601 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:31.596+0000 7f1d094e5640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d040af0e0 con 0x7f1d04155360 2023-10-26T01:38:31.601 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:31.596+0000 7f1d02ffd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d040cdd20 0x7f1d04155a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:31.601 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:31.597+0000 7f1d02ffd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d040cdd20 0x7f1d04155a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:34380/0 (socket says 172.21.15.169:34380) 2023-10-26T01:38:31.601 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:31.597+0000 7f1d02ffd640 1 -- 172.21.15.169:0/1736629324 learned_addr learned my addr 172.21.15.169:0/1736629324 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:38:31.601 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:31.597+0000 7f1d037fe640 1 --2- 172.21.15.169:0/1736629324 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d0415bbb0 0x7f1d0414aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:31.601 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:31.597+0000 7f1d02ffd640 1 -- 172.21.15.169:0/1736629324 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1d04155360 msgr2=0x7f1d0406ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:31.602 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:31.597+0000 7f1d027fc640 1 --2- 172.21.15.169:0/1736629324 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1d04155360 0x7f1d0406ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:31.602 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:31.597+0000 7f1d02ffd640 1 --2- 172.21.15.169:0/1736629324 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1d04155360 0x7f1d0406ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:31.602 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:31.597+0000 7f1d02ffd640 1 -- 172.21.15.169:0/1736629324 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d0415bbb0 msgr2=0x7f1d0414aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:31.602 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:31.597+0000 7f1d02ffd640 1 --2- 172.21.15.169:0/1736629324 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d0415bbb0 0x7f1d0414aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:31.602 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:31.597+0000 7f1d02ffd640 1 -- 172.21.15.169:0/1736629324 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cfc009560 con 0x7f1d040cdd20 2023-10-26T01:38:31.603 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:31.597+0000 7f1d037fe640 1 --2- 172.21.15.169:0/1736629324 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d0415bbb0 0x7f1d0414aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:38:31.603 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:31.597+0000 7f1d027fc640 1 --2- 172.21.15.169:0/1736629324 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1d04155360 0x7f1d0406ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:38:31.603 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:31.597+0000 7f1d02ffd640 1 --2- 172.21.15.169:0/1736629324 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d040cdd20 0x7f1d04155a00 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f1cf400ba50 tx=0x7f1cf400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:31.603 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:31.598+0000 7f1cfb7fe640 1 -- 172.21.15.169:0/1736629324 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf400cbe0 con 0x7f1d040cdd20 2023-10-26T01:38:31.603 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:31.598+0000 7f1cfb7fe640 1 -- 172.21.15.169:0/1736629324 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1cf400ce50 con 0x7f1d040cdd20 2023-10-26T01:38:31.604 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:31.598+0000 7f1cfb7fe640 1 -- 172.21.15.169:0/1736629324 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf4014680 con 0x7f1d040cdd20 2023-10-26T01:38:31.604 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:31.598+0000 7f1d094e5640 1 -- 172.21.15.169:0/1736629324 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d040af3c0 con 0x7f1d040cdd20 2023-10-26T01:38:31.604 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:31.598+0000 7f1d094e5640 1 -- 172.21.15.169:0/1736629324 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d040af980 con 0x7f1d040cdd20 2023-10-26T01:38:31.605 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:31.599+0000 7f1d094e5640 1 -- 172.21.15.169:0/1736629324 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d0404db10 con 0x7f1d040cdd20 2023-10-26T01:38:31.605 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:31.600+0000 7f1cfb7fe640 1 -- 172.21.15.169:0/1736629324 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f1cf401d3b0 con 0x7f1d040cdd20 2023-10-26T01:38:31.605 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:31.601+0000 7f1cfb7fe640 1 --2- 172.21.15.169:0/1736629324 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f1ce406ae40 0x7f1ce406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:31.606 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:31.601+0000 7f1cfb7fe640 1 -- 172.21.15.169:0/1736629324 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f1cf4089920 con 0x7f1d040cdd20 2023-10-26T01:38:31.606 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:31.602+0000 7f1d027fc640 1 --2- 172.21.15.169:0/1736629324 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f1ce406ae40 0x7f1ce406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:31.606 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:31.603+0000 7f1d027fc640 1 --2- 172.21.15.169:0/1736629324 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f1ce406ae40 0x7f1ce406d300 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f1d04159ae0 tx=0x7f1cec009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:31.606 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:31.605+0000 7f1cfb7fe640 1 -- 172.21.15.169:0/1736629324 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1cf40548f0 con 0x7f1d040cdd20 2023-10-26T01:38:31.865 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:31.863+0000 7f1d094e5640 1 -- 172.21.15.169:0/1736629324 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f1d04157540 con 0x7f1d040cdd20 2023-10-26T01:38:31.869 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:31.867+0000 7f1cfb7fe640 1 -- 172.21.15.169:0/1736629324 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v18) v1 ==== 190+0+147 (secure 0 0 0) 0x7f1cf4019020 con 0x7f1d040cdd20 2023-10-26T01:38:31.869 INFO:teuthology.orchestra.run.smithi169.stdout:[client.0] 2023-10-26T01:38:31.869 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQAgwTllKC7gOhAAPsvxGcuUA8/p8xqbUgGteA== 2023-10-26T01:38:31.869 INFO:teuthology.orchestra.run.smithi169.stdout: caps mds = "allow" 2023-10-26T01:38:31.870 INFO:teuthology.orchestra.run.smithi169.stdout: caps mon = "allow r" 2023-10-26T01:38:31.870 INFO:teuthology.orchestra.run.smithi169.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-10-26T01:38:31.870 INFO:teuthology.orchestra.run.smithi169.stderr:updated caps for client.0 2023-10-26T01:38:31.872 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:31.869+0000 7f1d094e5640 1 -- 172.21.15.169:0/1736629324 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f1ce406ae40 msgr2=0x7f1ce406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:31.872 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:31.870+0000 7f1d094e5640 1 --2- 172.21.15.169:0/1736629324 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f1ce406ae40 0x7f1ce406d300 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f1d04159ae0 tx=0x7f1cec009040 comp rx=0 tx=0).stop 2023-10-26T01:38:31.872 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:31.870+0000 7f1d094e5640 1 -- 172.21.15.169:0/1736629324 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d040cdd20 msgr2=0x7f1d04155a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:31.872 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:31.870+0000 7f1d094e5640 1 --2- 172.21.15.169:0/1736629324 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d040cdd20 0x7f1d04155a00 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f1cf400ba50 tx=0x7f1cf400bf10 comp rx=0 tx=0).stop 2023-10-26T01:38:31.873 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:31.870+0000 7f1d094e5640 1 -- 172.21.15.169:0/1736629324 shutdown_connections 2023-10-26T01:38:31.873 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:31.870+0000 7f1d094e5640 1 --2- 172.21.15.169:0/1736629324 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f1ce406ae40 0x7f1ce406d300 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:31.873 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:31.870+0000 7f1d094e5640 1 --2- 172.21.15.169:0/1736629324 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d0415bbb0 0x7f1d0414aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:31.873 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:31.870+0000 7f1d094e5640 1 --2- 172.21.15.169:0/1736629324 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1d04155360 0x7f1d0406ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:31.873 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:31.870+0000 7f1d094e5640 1 --2- 172.21.15.169:0/1736629324 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d040cdd20 0x7f1d04155a00 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:31.874 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:31.870+0000 7f1d094e5640 1 -- 172.21.15.169:0/1736629324 >> 172.21.15.169:0/1736629324 conn(0x7f1d04069930 msgr2=0x7f1d04151f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:31.874 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:31.870+0000 7f1d094e5640 1 -- 172.21.15.169:0/1736629324 shutdown_connections 2023-10-26T01:38:31.874 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:31.870+0000 7f1d094e5640 1 -- 172.21.15.169:0/1736629324 wait complete. 2023-10-26T01:38:31.891 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T01:38:32.105 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:32.103+0000 7fe62ed7d640 1 -- 172.21.15.169:0/700655092 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe6280d0fb0 msgr2=0x7fe6280ce5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:32.106 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:32.103+0000 7fe62ed7d640 1 --2- 172.21.15.169:0/700655092 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe6280d0fb0 0x7fe6280ce5f0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7fe6280b4280 tx=0x7fe61002bd90 comp rx=0 tx=0).stop 2023-10-26T01:38:32.106 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:32.103+0000 7fe62ed7d640 1 -- 172.21.15.169:0/700655092 shutdown_connections 2023-10-26T01:38:32.106 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:32.103+0000 7fe62ed7d640 1 --2- 172.21.15.169:0/700655092 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe6280ceb30 0x7fe6280b3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:32.106 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:32.103+0000 7fe62ed7d640 1 --2- 172.21.15.169:0/700655092 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe6280d0fb0 0x7fe6280ce5f0 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:32.107 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:32.103+0000 7fe62ed7d640 1 --2- 172.21.15.169:0/700655092 >> v2:172.21.15.170:3300/0 conn(0x7fe62815baa0 0x7fe6280b6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:32.107 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:32.103+0000 7fe62ed7d640 1 -- 172.21.15.169:0/700655092 >> 172.21.15.169:0/700655092 conn(0x7fe6280658b0 msgr2=0x7fe628065cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:32.107 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:32.103+0000 7fe62ed7d640 1 -- 172.21.15.169:0/700655092 shutdown_connections 2023-10-26T01:38:32.107 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:32.104+0000 7fe62ed7d640 1 -- 172.21.15.169:0/700655092 wait complete. 2023-10-26T01:38:32.109 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:32.105+0000 7fe62ed7d640 1 Processor -- start 2023-10-26T01:38:32.109 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:32.105+0000 7fe62ed7d640 1 -- start start 2023-10-26T01:38:32.109 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:32.106+0000 7fe62ed7d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe6280ceb30 0x7fe62814fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:32.110 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:32.106+0000 7fe62ed7d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe6280d0fb0 0x7fe62815e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:32.110 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:32.106+0000 7fe62ed7d640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe62815e600 0x7fe62815c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:32.110 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:32.106+0000 7fe62ed7d640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe62815ea90 con 0x7fe6280ceb30 2023-10-26T01:38:32.110 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:32.106+0000 7fe62ed7d640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe62815cf20 con 0x7fe6280d0fb0 2023-10-26T01:38:32.110 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:32.106+0000 7fe62ed7d640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe62815d200 con 0x7fe62815e600 2023-10-26T01:38:32.111 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:32.107+0000 7fe62cb19640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe6280ceb30 0x7fe62814fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:32.111 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:32.107+0000 7fe62cb19640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe6280ceb30 0x7fe62814fe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:34402/0 (socket says 172.21.15.169:34402) 2023-10-26T01:38:32.111 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:32.107+0000 7fe62cb19640 1 -- 172.21.15.169:0/4256736569 learned_addr learned my addr 172.21.15.169:0/4256736569 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:38:32.111 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:32.107+0000 7fe627fff640 1 --2- 172.21.15.169:0/4256736569 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe6280d0fb0 0x7fe62815e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:32.111 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:32.107+0000 7fe62cb19640 1 -- 172.21.15.169:0/4256736569 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe62815e600 msgr2=0x7fe62815c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:32.112 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:32.107+0000 7fe62d31a640 1 --2- 172.21.15.169:0/4256736569 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe62815e600 0x7fe62815c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:32.112 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:32.107+0000 7fe62cb19640 1 --2- 172.21.15.169:0/4256736569 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe62815e600 0x7fe62815c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:32.112 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:32.107+0000 7fe62cb19640 1 -- 172.21.15.169:0/4256736569 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe6280d0fb0 msgr2=0x7fe62815e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:32.112 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:32.107+0000 7fe62cb19640 1 --2- 172.21.15.169:0/4256736569 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe6280d0fb0 0x7fe62815e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:32.112 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:32.107+0000 7fe62cb19640 1 -- 172.21.15.169:0/4256736569 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe610009560 con 0x7fe6280ceb30 2023-10-26T01:38:32.113 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:32.107+0000 7fe62d31a640 1 --2- 172.21.15.169:0/4256736569 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe62815e600 0x7fe62815c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:38:32.113 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:32.107+0000 7fe627fff640 1 --2- 172.21.15.169:0/4256736569 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe6280d0fb0 0x7fe62815e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:38:32.113 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:32.107+0000 7fe62cb19640 1 --2- 172.21.15.169:0/4256736569 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe6280ceb30 0x7fe62814fe90 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7fe61800ea20 tx=0x7fe61800eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:32.114 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:32.108+0000 7fe6257fa640 1 -- 172.21.15.169:0/4256736569 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe61800cde0 con 0x7fe6280ceb30 2023-10-26T01:38:32.115 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:32.108+0000 7fe6257fa640 1 -- 172.21.15.169:0/4256736569 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe618011560 con 0x7fe6280ceb30 2023-10-26T01:38:32.115 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:32.108+0000 7fe6257fa640 1 -- 172.21.15.169:0/4256736569 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe618011950 con 0x7fe6280ceb30 2023-10-26T01:38:32.115 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:32.108+0000 7fe62ed7d640 1 -- 172.21.15.169:0/4256736569 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe62815d4e0 con 0x7fe6280ceb30 2023-10-26T01:38:32.115 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:32.108+0000 7fe62ed7d640 1 -- 172.21.15.169:0/4256736569 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe62815daa0 con 0x7fe6280ceb30 2023-10-26T01:38:32.115 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:32.109+0000 7fe62ed7d640 1 -- 172.21.15.169:0/4256736569 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe62804db10 con 0x7fe6280ceb30 2023-10-26T01:38:32.116 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:32.110+0000 7fe6257fa640 1 -- 172.21.15.169:0/4256736569 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fe618011ab0 con 0x7fe6280ceb30 2023-10-26T01:38:32.116 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:32.112+0000 7fe6257fa640 1 --2- 172.21.15.169:0/4256736569 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe60806ae40 0x7fe60806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:32.116 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:32.112+0000 7fe6257fa640 1 -- 172.21.15.169:0/4256736569 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fe61808a380 con 0x7fe6280ceb30 2023-10-26T01:38:32.116 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:32.112+0000 7fe627fff640 1 --2- 172.21.15.169:0/4256736569 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe60806ae40 0x7fe60806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:32.117 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:32.113+0000 7fe627fff640 1 --2- 172.21.15.169:0/4256736569 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe60806ae40 0x7fe60806d300 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fe610035000 tx=0x7fe610004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:32.117 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:32.116+0000 7fe6257fa640 1 -- 172.21.15.169:0/4256736569 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe618055350 con 0x7fe6280ceb30 2023-10-26T01:38:32.345 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:32.344+0000 7fe62ed7d640 1 -- 172.21.15.169:0/4256736569 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe6280b0ed0 con 0x7fe6280ceb30 2023-10-26T01:38:32.347 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:32.345+0000 7fe6257fa640 1 -- 172.21.15.169:0/4256736569 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+4927 (secure 0 0 0) 0x7fe618058970 con 0x7fe6280ceb30 2023-10-26T01:38:32.347 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:38:32.347 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":68,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5917,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/3297412140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":3297412140},{"type":"v1","addr":"172.21.15.169:6835","nonce":3297412140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":5929,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/2575809940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2575809940},{"type":"v1","addr":"172.21.15.170:6836","nonce":2575809940}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":7761,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2632217991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2632217991},{"type":"v1","addr":"172.21.15.169:6833","nonce":2632217991}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T01:38:20.773556+0000","modified":"2023-10-26T01:38:23.839758+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7773},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7773":{"gid":7773,"name":"b","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6837/3615088922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":3615088922},{"type":"v1","addr":"172.21.15.170:6837","nonce":3615088922}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-10-26T01:38:32.348 INFO:teuthology.orchestra.run.smithi169.stderr:dumped fsmap epoch 68 2023-10-26T01:38:32.349 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:32.347+0000 7fe62ed7d640 1 -- 172.21.15.169:0/4256736569 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe60806ae40 msgr2=0x7fe60806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:32.350 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:32.348+0000 7fe62ed7d640 1 --2- 172.21.15.169:0/4256736569 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe60806ae40 0x7fe60806d300 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fe610035000 tx=0x7fe610004730 comp rx=0 tx=0).stop 2023-10-26T01:38:32.350 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:32.348+0000 7fe62ed7d640 1 -- 172.21.15.169:0/4256736569 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe6280ceb30 msgr2=0x7fe62814fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:32.350 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:32.348+0000 7fe62ed7d640 1 --2- 172.21.15.169:0/4256736569 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe6280ceb30 0x7fe62814fe90 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7fe61800ea20 tx=0x7fe61800eee0 comp rx=0 tx=0).stop 2023-10-26T01:38:32.350 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:32.348+0000 7fe62ed7d640 1 -- 172.21.15.169:0/4256736569 shutdown_connections 2023-10-26T01:38:32.351 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:32.348+0000 7fe62ed7d640 1 --2- 172.21.15.169:0/4256736569 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe60806ae40 0x7fe60806d300 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:32.351 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:32.348+0000 7fe62ed7d640 1 --2- 172.21.15.169:0/4256736569 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe62815e600 0x7fe62815c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:32.351 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:32.348+0000 7fe62ed7d640 1 --2- 172.21.15.169:0/4256736569 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe6280d0fb0 0x7fe62815e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:32.351 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:32.348+0000 7fe62ed7d640 1 --2- 172.21.15.169:0/4256736569 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe6280ceb30 0x7fe62814fe90 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:32.352 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:32.348+0000 7fe62ed7d640 1 -- 172.21.15.169:0/4256736569 >> 172.21.15.169:0/4256736569 conn(0x7fe6280658b0 msgr2=0x7fe62815ace0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:32.352 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:32.348+0000 7fe62ed7d640 1 -- 172.21.15.169:0/4256736569 shutdown_connections 2023-10-26T01:38:32.352 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:32.348+0000 7fe62ed7d640 1 -- 172.21.15.169:0/4256736569 wait complete. 2023-10-26T01:38:32.771 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 68, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-26T01:38:20.773556+0000', 'modified': '2023-10-26T01:38:23.839758+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7773}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7773': {'gid': 7773, 'name': 'b', 'rank': 0, 'incarnation': 64, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.170:6837/3615088922', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6835', 'nonce': 3615088922}, {'type': 'v1', 'addr': '172.21.15.170:6837', 'nonce': 3615088922}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-26T01:38:32.772 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-26T01:38:32.772 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-10-26T01:38:33.002 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.000+0000 7f766bfff640 1 -- 172.21.15.169:0/2170723558 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f766c0b0e50 msgr2=0x7f766c069e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:33.003 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.000+0000 7f766bfff640 1 --2- 172.21.15.169:0/2170723558 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f766c0b0e50 0x7f766c069e40 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f7654005fd0 tx=0x7f765402bd90 comp rx=0 tx=0).stop 2023-10-26T01:38:33.003 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.000+0000 7f766bfff640 1 -- 172.21.15.169:0/2170723558 shutdown_connections 2023-10-26T01:38:33.003 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.000+0000 7f766bfff640 1 --2- 172.21.15.169:0/2170723558 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f766c0b0e50 0x7f766c069e40 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:33.003 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.000+0000 7f766bfff640 1 --2- 172.21.15.169:0/2170723558 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f766c153b90 0x7f766c069900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:33.004 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.000+0000 7f766bfff640 1 -- 172.21.15.169:0/2170723558 >> 172.21.15.169:0/2170723558 conn(0x7f766c0af110 msgr2=0x7f766c0af520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:33.004 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.000+0000 7f766bfff640 1 -- 172.21.15.169:0/2170723558 shutdown_connections 2023-10-26T01:38:33.004 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.001+0000 7f766bfff640 1 -- 172.21.15.169:0/2170723558 wait complete. 2023-10-26T01:38:33.006 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.002+0000 7f766bfff640 1 Processor -- start 2023-10-26T01:38:33.006 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.002+0000 7f766bfff640 1 -- start start 2023-10-26T01:38:33.007 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.003+0000 7f766bfff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f766c0b0e50 0x7f766c152d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:33.007 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.003+0000 7f766bfff640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f766c153b90 0x7f766c155900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:33.007 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.003+0000 7f766bfff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f766c15d0d0 0x7f766c15bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:33.007 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.003+0000 7f766bfff640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f766c156e60 con 0x7f766c15d0d0 2023-10-26T01:38:33.007 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.003+0000 7f766bfff640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f766c1561e0 con 0x7f766c0b0e50 2023-10-26T01:38:33.008 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.003+0000 7f766bfff640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f766c1564c0 con 0x7f766c153b90 2023-10-26T01:38:33.008 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.004+0000 7f766b7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f766c15d0d0 0x7f766c15bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:33.008 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.004+0000 7f766b7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f766c15d0d0 0x7f766c15bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:34416/0 (socket says 172.21.15.169:34416) 2023-10-26T01:38:33.008 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.004+0000 7f766b7fe640 1 -- 172.21.15.169:0/1251710184 learned_addr learned my addr 172.21.15.169:0/1251710184 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:38:33.008 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.004+0000 7f766affd640 1 --2- 172.21.15.169:0/1251710184 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f766c0b0e50 0x7f766c152d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:33.008 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.004+0000 7f7663fff640 1 --2- 172.21.15.169:0/1251710184 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f766c153b90 0x7f766c155900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:33.009 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.004+0000 7f766b7fe640 1 -- 172.21.15.169:0/1251710184 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f766c153b90 msgr2=0x7f766c155900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:33.009 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.004+0000 7f766b7fe640 1 --2- 172.21.15.169:0/1251710184 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f766c153b90 0x7f766c155900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:33.009 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.004+0000 7f766b7fe640 1 -- 172.21.15.169:0/1251710184 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f766c0b0e50 msgr2=0x7f766c152d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:33.009 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.004+0000 7f766b7fe640 1 --2- 172.21.15.169:0/1251710184 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f766c0b0e50 0x7f766c152d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:33.010 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.004+0000 7f766b7fe640 1 -- 172.21.15.169:0/1251710184 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7654009560 con 0x7f766c15d0d0 2023-10-26T01:38:33.010 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.004+0000 7f766b7fe640 1 --2- 172.21.15.169:0/1251710184 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f766c15d0d0 0x7f766c15bf40 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f766400b970 tx=0x7f766400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:33.010 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.004+0000 7f76637fe640 1 -- 172.21.15.169:0/1251710184 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f766400cb40 con 0x7f766c15d0d0 2023-10-26T01:38:33.010 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.005+0000 7f76637fe640 1 -- 172.21.15.169:0/1251710184 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f766400cd90 con 0x7f766c15d0d0 2023-10-26T01:38:33.010 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.005+0000 7f76637fe640 1 -- 172.21.15.169:0/1251710184 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76640146a0 con 0x7f766c15d0d0 2023-10-26T01:38:33.010 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.005+0000 7f766bfff640 1 -- 172.21.15.169:0/1251710184 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f766c1567a0 con 0x7f766c15d0d0 2023-10-26T01:38:33.011 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.005+0000 7f766bfff640 1 -- 172.21.15.169:0/1251710184 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f766c1532c0 con 0x7f766c15d0d0 2023-10-26T01:38:33.013 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.007+0000 7f76637fe640 1 -- 172.21.15.169:0/1251710184 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f766401e440 con 0x7f766c15d0d0 2023-10-26T01:38:33.013 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.008+0000 7f76637fe640 1 --2- 172.21.15.169:0/1251710184 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f764c06ae70 0x7f764c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:33.013 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.008+0000 7f76637fe640 1 -- 172.21.15.169:0/1251710184 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f7664089760 con 0x7f766c15d0d0 2023-10-26T01:38:33.014 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.008+0000 7f7674277740 1 Processor -- start 2023-10-26T01:38:33.014 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.009+0000 7f7674277740 1 -- start start 2023-10-26T01:38:33.014 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.009+0000 7f766affd640 1 --2- 172.21.15.169:0/1251710184 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f764c06ae70 0x7f764c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:33.014 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.009+0000 7f7674277740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559fa575e970 0x559fa5731ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:33.014 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.009+0000 7f7674277740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559fa575ed50 0x559fa5761700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:33.015 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.009+0000 7f7674277740 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x559fa57320e0 0x559fa5761c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:33.015 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.009+0000 7f7674277740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fa56a4740 con 0x559fa575ed50 2023-10-26T01:38:33.015 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.009+0000 7f7674277740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fa56a1e30 con 0x559fa575e970 2023-10-26T01:38:33.015 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.009+0000 7f7674277740 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x559fa569d720 con 0x559fa57320e0 2023-10-26T01:38:33.015 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.010+0000 7f7663fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559fa575ed50 0x559fa5761700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:33.016 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.010+0000 7f766affd640 1 --2- 172.21.15.169:0/1251710184 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f764c06ae70 0x7f764c06d330 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f765c005e10 tx=0x7f765c005da0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:33.016 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.010+0000 7f7663fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559fa575ed50 0x559fa5761700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:34432/0 (socket says 172.21.15.169:34432) 2023-10-26T01:38:33.016 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.010+0000 7f7663fff640 1 -- 172.21.15.169:0/3723123994 learned_addr learned my addr 172.21.15.169:0/3723123994 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:38:33.016 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.010+0000 7f766affd640 1 --2- 172.21.15.169:0/3723123994 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559fa575e970 0x559fa5731ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:33.016 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.010+0000 7f766b7fe640 1 --2- 172.21.15.169:0/3723123994 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x559fa57320e0 0x559fa5761c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:33.017 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.010+0000 7f7663fff640 1 -- 172.21.15.169:0/3723123994 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x559fa57320e0 msgr2=0x559fa5761c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T01:38:33.017 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.010+0000 7f7663fff640 1 --2- 172.21.15.169:0/3723123994 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x559fa57320e0 0x559fa5761c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:33.017 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.010+0000 7f7663fff640 1 -- 172.21.15.169:0/3723123994 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559fa575e970 msgr2=0x559fa5731ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T01:38:33.017 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.010+0000 7f7663fff640 1 --2- 172.21.15.169:0/3723123994 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559fa575e970 0x559fa5731ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:33.017 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.010+0000 7f7663fff640 1 -- 172.21.15.169:0/3723123994 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559fa5661910 con 0x559fa575ed50 2023-10-26T01:38:33.017 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.011+0000 7f7663fff640 1 --2- 172.21.15.169:0/3723123994 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559fa575ed50 0x559fa5761700 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f76540099f0 tx=0x7f76540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c24574b832d13cf3 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:33.018 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.011+0000 7f7660ff9640 1 -- 172.21.15.169:0/3723123994 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76540042c0 con 0x559fa575ed50 2023-10-26T01:38:33.018 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.011+0000 7f7660ff9640 1 -- 172.21.15.169:0/3723123994 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7654004510 con 0x559fa575ed50 2023-10-26T01:38:33.018 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.011+0000 7f7660ff9640 1 -- 172.21.15.169:0/3723123994 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f765402c970 con 0x559fa575ed50 2023-10-26T01:38:33.018 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.011+0000 7f7674277740 1 -- 172.21.15.169:0/3723123994 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559fa575ed50 msgr2=0x559fa5761700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:33.018 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.011+0000 7f7674277740 1 --2- 172.21.15.169:0/3723123994 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559fa575ed50 0x559fa5761700 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f76540099f0 tx=0x7f76540046c0 comp rx=0 tx=0).stop 2023-10-26T01:38:33.019 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.012+0000 7f7674277740 1 -- 172.21.15.169:0/3723123994 shutdown_connections 2023-10-26T01:38:33.019 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.012+0000 7f7674277740 1 --2- 172.21.15.169:0/3723123994 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559fa575ed50 0x559fa5761700 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:33.019 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.012+0000 7f7674277740 1 --2- 172.21.15.169:0/3723123994 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x559fa57320e0 0x559fa5761c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:33.019 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.012+0000 7f7674277740 1 --2- 172.21.15.169:0/3723123994 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559fa575e970 0x559fa5731ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:33.019 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.012+0000 7f7674277740 1 -- 172.21.15.169:0/3723123994 >> 172.21.15.169:0/3723123994 conn(0x559fa5733e10 msgr2=0x559fa577c700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:33.020 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.012+0000 7f7674277740 1 -- 172.21.15.169:0/3723123994 shutdown_connections 2023-10-26T01:38:33.020 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.012+0000 7f7674277740 1 -- 172.21.15.169:0/3723123994 wait complete. 2023-10-26T01:38:33.023 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.012+0000 7f7674277740 1 -- start start 2023-10-26T01:38:33.023 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.013+0000 7f7674277740 1 Processor -- start 2023-10-26T01:38:33.024 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.014+0000 7f7674277740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559fa575e970 0x559fa5774a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:33.024 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.014+0000 7f7674277740 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x559fa575ed50 0x559fa5776350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:33.024 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.014+0000 7f7674277740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559fa578cbe0 0x559fa5777cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:33.024 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.014+0000 7f7674277740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fa563b970 con 0x559fa575e970 2023-10-26T01:38:33.024 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.014+0000 7f7674277740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fa563b610 con 0x559fa578cbe0 2023-10-26T01:38:33.024 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.014+0000 7f7674277740 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x559fa5401f20 con 0x559fa575ed50 2023-10-26T01:38:33.025 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.014+0000 7f7674277740 10 client.0 _ll_register_callbacks cb 0x559fa565f0c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T01:38:33.025 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.014+0000 7f766affd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559fa575e970 0x559fa5774a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:33.025 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.014+0000 7f766affd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559fa575e970 0x559fa5774a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:34440/0 (socket says 172.21.15.169:34440) 2023-10-26T01:38:33.025 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.014+0000 7f766affd640 1 -- 172.21.15.169:0/3226327332 learned_addr learned my addr 172.21.15.169:0/3226327332 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:38:33.025 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.014+0000 7f766b7fe640 1 --2- 172.21.15.169:0/3226327332 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559fa578cbe0 0x559fa5777cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:33.026 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.014+0000 7f766affd640 1 -- 172.21.15.169:0/3226327332 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x559fa575ed50 msgr2=0x559fa5776350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T01:38:33.026 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.014+0000 7f7663fff640 1 --2- 172.21.15.169:0/3226327332 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x559fa575ed50 0x559fa5776350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:33.026 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.014+0000 7f766affd640 1 --2- 172.21.15.169:0/3226327332 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x559fa575ed50 0x559fa5776350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:33.026 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.014+0000 7f766affd640 1 -- 172.21.15.169:0/3226327332 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559fa578cbe0 msgr2=0x559fa5777cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T01:38:33.026 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.014+0000 7f766affd640 1 --2- 172.21.15.169:0/3226327332 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559fa578cbe0 0x559fa5777cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:33.027 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.014+0000 7f766affd640 1 -- 172.21.15.169:0/3226327332 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f766c0b0650 con 0x559fa575e970 2023-10-26T01:38:33.027 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.014+0000 7f766b7fe640 1 --2- 172.21.15.169:0/3226327332 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559fa578cbe0 0x559fa5777cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:38:33.027 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.015+0000 7f766affd640 1 --2- 172.21.15.169:0/3226327332 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559fa575e970 0x559fa5774a60 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f765c010a60 tx=0x7f765c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62f271c2aa4d1776 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:33.027 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.015+0000 7f7649ffb640 10 client.0 ms_handle_connect on v2:172.21.15.169:3300/0 2023-10-26T01:38:33.027 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.015+0000 7f7649ffb640 1 -- 172.21.15.169:0/3226327332 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f765c00fd50 con 0x559fa575e970 2023-10-26T01:38:33.028 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.015+0000 7f7649ffb640 1 -- 172.21.15.169:0/3226327332 <== mon.0 v2:172.21.15.169:3300/0 2 ==== osd_map(136..136 src has 1..136) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f765c01e430 con 0x559fa575e970 2023-10-26T01:38:33.028 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.015+0000 7f7674277740 1 -- 172.21.15.169:0/3226327332 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559fa5696700 con 0x559fa575e970 2023-10-26T01:38:33.028 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.015+0000 7f7649ffb640 1 -- 172.21.15.169:0/3226327332 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f765c00feb0 con 0x559fa575e970 2023-10-26T01:38:33.028 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.015+0000 7f7674277740 10 client.7932 fetch_fsmap learned FSMap version 68 2023-10-26T01:38:33.028 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.015+0000 7f7674277740 1 -- 172.21.15.169:0/3226327332 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) v3 -- 0x559fa569bda0 con 0x559fa575e970 2023-10-26T01:38:33.028 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.016+0000 7f7649ffb640 1 -- 172.21.15.169:0/3226327332 <== mon.0 v2:172.21.15.169:3300/0 4 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x559fa5696700 con 0x559fa575e970 2023-10-26T01:38:33.029 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.016+0000 7f7649ffb640 1 -- 172.21.15.169:0/3226327332 <== mon.0 v2:172.21.15.169:3300/0 5 ==== fsmap(e 68) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f765c024020 con 0x559fa575e970 2023-10-26T01:38:33.029 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.016+0000 7f7674277740 10 client.7932 fetch_fsmap finished waiting for FSMap version 68 2023-10-26T01:38:33.029 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.016+0000 7f7674277740 10 client.7932 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-26T01:38:33.029 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.016+0000 7f7674277740 20 client.7932 populate_metadata read hostname 'smithi169' 2023-10-26T01:38:33.029 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.016+0000 7f7674277740 1 --2- 172.21.15.169:0/3226327332 >> [v2:172.21.15.170:6835/3615088922,v1:172.21.15.170:6837/3615088922] conn(0x559fa5764280 0x559fa57737b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:33.030 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.016+0000 7f7674277740 4 client.7932 mds_command: new command op to 7773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-26T01:38:33.030 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.016+0000 7f7674277740 1 -- 172.21.15.169:0/3226327332 --> [v2:172.21.15.170:6835/3615088922,v1:172.21.15.170:6837/3615088922] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559fa5696420 con 0x559fa5764280 2023-10-26T01:38:33.030 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.017+0000 7f7663fff640 1 --2- 172.21.15.169:0/3226327332 >> [v2:172.21.15.170:6835/3615088922,v1:172.21.15.170:6837/3615088922] conn(0x559fa5764280 0x559fa57737b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:33.030 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.018+0000 7f7663fff640 1 --2- 172.21.15.169:0/3226327332 >> [v2:172.21.15.170:6835/3615088922,v1:172.21.15.170:6837/3615088922] conn(0x559fa5764280 0x559fa57737b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd34cd61b65f5399 server_cookie=6c9354e9c70bbf9c in_seq=0 out_seq=0 2023-10-26T01:38:33.030 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.018+0000 7f7649ffb640 10 client.7932 ms_handle_connect on v2:172.21.15.170:6835/3615088922 2023-10-26T01:38:33.031 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.020+0000 7f7649ffb640 1 -- 172.21.15.169:0/3226327332 <== mds.0 v2:172.21.15.170:6835/3615088922 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x559fa5696420 con 0x559fa5764280 2023-10-26T01:38:33.031 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.020+0000 7f7649ffb640 10 client.7932 handle_command_reply: tid=0 2023-10-26T01:38:33.031 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.020+0000 7f7674277740 1 client.7932 shutdown 2023-10-26T01:38:33.031 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.020+0000 7f7674277740 1 -- 172.21.15.169:0/3226327332 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559fa575e970 msgr2=0x559fa5774a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:33.031 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.020+0000 7f7674277740 1 --2- 172.21.15.169:0/3226327332 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559fa575e970 0x559fa5774a60 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f765c010a60 tx=0x7f765c0046c0 comp rx=0 tx=0).stop 2023-10-26T01:38:33.032 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.020+0000 7f7674277740 1 -- 172.21.15.169:0/3226327332 shutdown_connections 2023-10-26T01:38:33.032 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.020+0000 7f7674277740 1 --2- 172.21.15.169:0/3226327332 >> [v2:172.21.15.170:6835/3615088922,v1:172.21.15.170:6837/3615088922] conn(0x559fa5764280 0x559fa57737b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:33.032 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.020+0000 7f7674277740 1 --2- 172.21.15.169:0/3226327332 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559fa578cbe0 0x559fa5777cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:33.032 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.020+0000 7f7674277740 1 --2- 172.21.15.169:0/3226327332 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x559fa575ed50 0x559fa5776350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:33.032 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.020+0000 7f7674277740 1 --2- 172.21.15.169:0/3226327332 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559fa575e970 0x559fa5774a60 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:33.032 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.020+0000 7f7649ffb640 0 client.7932 ms_handle_reset on v2:172.21.15.170:6835/3615088922 2023-10-26T01:38:33.033 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.020+0000 7f7674277740 1 -- 172.21.15.169:0/3226327332 >> 172.21.15.169:0/3226327332 conn(0x559fa57320e0 msgr2=0x559fa5735860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:33.033 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.020+0000 7f7674277740 1 -- 172.21.15.169:0/3226327332 shutdown_connections 2023-10-26T01:38:33.033 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.020+0000 7f7674277740 1 -- 172.21.15.169:0/3226327332 wait complete. 2023-10-26T01:38:33.033 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.020+0000 7f7674277740 20 client.7932 trim_cache size 0 max 16384 2023-10-26T01:38:33.038 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.037+0000 7f7674277740 1 Processor -- start 2023-10-26T01:38:33.039 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.037+0000 7f7674277740 1 -- start start 2023-10-26T01:38:33.039 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.038+0000 7f7674277740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559fa575e970 0x559fa5730c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:33.040 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.038+0000 7f7674277740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559fa575ed50 0x559fa5737670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:33.040 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.038+0000 7f7674277740 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x559fa578cbe0 0x559fa578ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:33.040 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.038+0000 7f7674277740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fa578d1e0 con 0x559fa575ed50 2023-10-26T01:38:33.040 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.038+0000 7f7674277740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fa578cfc0 con 0x559fa575e970 2023-10-26T01:38:33.041 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.038+0000 7f7674277740 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x559fa5765720 con 0x559fa578cbe0 2023-10-26T01:38:33.042 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.038+0000 7f7663fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559fa575ed50 0x559fa5737670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:33.042 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.038+0000 7f7663fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559fa575ed50 0x559fa5737670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:34454/0 (socket says 172.21.15.169:34454) 2023-10-26T01:38:33.043 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.038+0000 7f7663fff640 1 -- 172.21.15.169:0/2437905192 learned_addr learned my addr 172.21.15.169:0/2437905192 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:38:33.043 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.039+0000 7f766b7fe640 1 --2- 172.21.15.169:0/2437905192 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x559fa578cbe0 0x559fa578ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:33.043 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.039+0000 7f766affd640 1 --2- 172.21.15.169:0/2437905192 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559fa575e970 0x559fa5730c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:33.043 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.039+0000 7f7663fff640 1 -- 172.21.15.169:0/2437905192 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x559fa578cbe0 msgr2=0x559fa578ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T01:38:33.043 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.039+0000 7f7663fff640 1 --2- 172.21.15.169:0/2437905192 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x559fa578cbe0 0x559fa578ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:33.044 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.039+0000 7f7663fff640 1 -- 172.21.15.169:0/2437905192 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559fa575e970 msgr2=0x559fa5730c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T01:38:33.044 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.039+0000 7f7663fff640 1 --2- 172.21.15.169:0/2437905192 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559fa575e970 0x559fa5730c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:33.044 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.039+0000 7f7663fff640 1 -- 172.21.15.169:0/2437905192 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559fa563ab80 con 0x559fa575ed50 2023-10-26T01:38:33.044 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.039+0000 7f766affd640 1 --2- 172.21.15.169:0/2437905192 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559fa575e970 0x559fa5730c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:38:33.044 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.039+0000 7f7663fff640 1 --2- 172.21.15.169:0/2437905192 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559fa575ed50 0x559fa5737670 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f7654009230 tx=0x7f765402bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9fb027894d51be1 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:33.045 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.039+0000 7f76617fa640 1 -- 172.21.15.169:0/2437905192 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f765404e570 con 0x559fa575ed50 2023-10-26T01:38:33.045 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.039+0000 7f76617fa640 1 -- 172.21.15.169:0/2437905192 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f765404e810 con 0x559fa575ed50 2023-10-26T01:38:33.045 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.039+0000 7f76617fa640 1 -- 172.21.15.169:0/2437905192 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f765404ec00 con 0x559fa575ed50 2023-10-26T01:38:33.045 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.040+0000 7f7674277740 1 -- 172.21.15.169:0/2437905192 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559fa575ed50 msgr2=0x559fa5737670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:33.045 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.040+0000 7f7674277740 1 --2- 172.21.15.169:0/2437905192 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559fa575ed50 0x559fa5737670 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f7654009230 tx=0x7f765402bd90 comp rx=0 tx=0).stop 2023-10-26T01:38:33.045 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.040+0000 7f7674277740 1 -- 172.21.15.169:0/2437905192 shutdown_connections 2023-10-26T01:38:33.046 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.040+0000 7f7674277740 1 --2- 172.21.15.169:0/2437905192 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559fa575ed50 0x559fa5737670 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:33.046 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.040+0000 7f7674277740 1 --2- 172.21.15.169:0/2437905192 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x559fa578cbe0 0x559fa578ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:33.046 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.040+0000 7f7674277740 1 --2- 172.21.15.169:0/2437905192 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559fa575e970 0x559fa5730c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:33.046 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.040+0000 7f7674277740 1 -- 172.21.15.169:0/2437905192 >> 172.21.15.169:0/2437905192 conn(0x559fa57320e0 msgr2=0x559fa5776a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:33.047 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.040+0000 7f7674277740 1 -- 172.21.15.169:0/2437905192 shutdown_connections 2023-10-26T01:38:33.047 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.040+0000 7f7674277740 1 -- 172.21.15.169:0/2437905192 wait complete. 2023-10-26T01:38:33.047 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.041+0000 7f7674277740 1 -- start start 2023-10-26T01:38:33.047 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.041+0000 7f7674277740 1 Processor -- start 2023-10-26T01:38:33.050 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.042+0000 7f7674277740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559fa575e970 0x559fa57caab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:33.051 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.042+0000 7f7674277740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559fa575ed50 0x559fa5736480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:33.051 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.042+0000 7f7674277740 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x559fa578cbe0 0x559fa5776a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:33.051 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.042+0000 7f7674277740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fa57e8e00 con 0x559fa575e970 2023-10-26T01:38:33.051 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.042+0000 7f7674277740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fa57c8750 con 0x559fa575ed50 2023-10-26T01:38:33.052 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.042+0000 7f7674277740 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x559fa57e9c60 con 0x559fa578cbe0 2023-10-26T01:38:33.052 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.042+0000 7f7674277740 10 client.0 _ll_register_callbacks cb 0x7f7654009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T01:38:33.052 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.042+0000 7f766affd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559fa575e970 0x559fa57caab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:33.052 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.042+0000 7f766affd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559fa575e970 0x559fa57caab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:34456/0 (socket says 172.21.15.169:34456) 2023-10-26T01:38:33.052 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.042+0000 7f766affd640 1 -- 172.21.15.169:0/3898302105 learned_addr learned my addr 172.21.15.169:0/3898302105 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:38:33.053 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.042+0000 7f766affd640 1 -- 172.21.15.169:0/3898302105 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x559fa578cbe0 msgr2=0x559fa5776a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T01:38:33.053 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.042+0000 7f766affd640 1 --2- 172.21.15.169:0/3898302105 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x559fa578cbe0 0x559fa5776a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:33.053 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.042+0000 7f766affd640 1 -- 172.21.15.169:0/3898302105 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559fa575ed50 msgr2=0x559fa5736480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T01:38:33.053 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.042+0000 7f766affd640 1 --2- 172.21.15.169:0/3898302105 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559fa575ed50 0x559fa5736480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:33.054 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.042+0000 7f766affd640 1 -- 172.21.15.169:0/3898302105 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559fa5661910 con 0x559fa575e970 2023-10-26T01:38:33.054 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.043+0000 7f766affd640 1 --2- 172.21.15.169:0/3898302105 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559fa575e970 0x559fa57caab0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f765c005780 tx=0x7f765c020be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e92f9436c0bf5cd server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:33.054 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.043+0000 7f7649ffb640 10 client.0 ms_handle_connect on v2:172.21.15.169:3300/0 2023-10-26T01:38:33.054 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.043+0000 7f7649ffb640 1 -- 172.21.15.169:0/3898302105 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f765c037070 con 0x559fa575e970 2023-10-26T01:38:33.054 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.043+0000 7f7649ffb640 1 -- 172.21.15.169:0/3898302105 <== mon.0 v2:172.21.15.169:3300/0 2 ==== osd_map(136..136 src has 1..136) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f765c01e430 con 0x559fa575e970 2023-10-26T01:38:33.055 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.043+0000 7f7674277740 1 -- 172.21.15.169:0/3898302105 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7654004510 con 0x559fa575e970 2023-10-26T01:38:33.055 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.043+0000 7f7649ffb640 1 -- 172.21.15.169:0/3898302105 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f765c01e7d0 con 0x559fa575e970 2023-10-26T01:38:33.055 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.043+0000 7f7674277740 10 client.7938 fetch_fsmap learned FSMap version 68 2023-10-26T01:38:33.055 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.043+0000 7f7674277740 1 -- 172.21.15.169:0/3898302105 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) v3 -- 0x559fa5660170 con 0x559fa575e970 2023-10-26T01:38:33.055 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.044+0000 7f7649ffb640 1 -- 172.21.15.169:0/3898302105 <== mon.0 v2:172.21.15.169:3300/0 4 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7654004510 con 0x559fa575e970 2023-10-26T01:38:33.056 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.044+0000 7f7649ffb640 1 -- 172.21.15.169:0/3898302105 <== mon.0 v2:172.21.15.169:3300/0 5 ==== fsmap(e 68) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f765c043070 con 0x559fa575e970 2023-10-26T01:38:33.056 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.044+0000 7f7674277740 10 client.7938 fetch_fsmap finished waiting for FSMap version 68 2023-10-26T01:38:33.056 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.044+0000 7f7674277740 10 client.7938 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-26T01:38:33.056 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.044+0000 7f7674277740 20 client.7938 populate_metadata read hostname 'smithi169' 2023-10-26T01:38:33.056 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.044+0000 7f7674277740 1 --2- 172.21.15.169:0/3898302105 >> [v2:172.21.15.170:6835/3615088922,v1:172.21.15.170:6837/3615088922] conn(0x559fa57e5d00 0x559fa57e60e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:33.058 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.044+0000 7f7674277740 4 client.7938 mds_command: new command op to 7773 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-26T01:38:33.058 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.044+0000 7f7674277740 1 -- 172.21.15.169:0/3898302105 --> [v2:172.21.15.170:6835/3615088922,v1:172.21.15.170:6837/3615088922] -- command(tid 0: {"prefix": "status"}) v1 -- 0x559fa539f6c0 con 0x559fa57e5d00 2023-10-26T01:38:33.058 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.044+0000 7f7663fff640 1 --2- 172.21.15.169:0/3898302105 >> [v2:172.21.15.170:6835/3615088922,v1:172.21.15.170:6837/3615088922] conn(0x559fa57e5d00 0x559fa57e60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:33.058 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.045+0000 7f7663fff640 1 --2- 172.21.15.169:0/3898302105 >> [v2:172.21.15.170:6835/3615088922,v1:172.21.15.170:6837/3615088922] conn(0x559fa57e5d00 0x559fa57e60e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72e06dca8d101a1d server_cookie=8cafd8faa6df43c3 in_seq=0 out_seq=0 2023-10-26T01:38:33.058 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.045+0000 7f7649ffb640 10 client.7938 ms_handle_connect on v2:172.21.15.170:6835/3615088922 2023-10-26T01:38:33.059 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.046+0000 7f7649ffb640 1 -- 172.21.15.169:0/3898302105 <== mds.0 v2:172.21.15.170:6835/3615088922 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x559fa539f6c0 con 0x559fa57e5d00 2023-10-26T01:38:33.059 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.046+0000 7f7649ffb640 10 client.7938 handle_command_reply: tid=0 2023-10-26T01:38:33.059 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.046+0000 7f7674277740 1 client.7938 shutdown 2023-10-26T01:38:33.059 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.046+0000 7f7674277740 1 -- 172.21.15.169:0/3898302105 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559fa575e970 msgr2=0x559fa57caab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:33.059 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.046+0000 7f7674277740 1 --2- 172.21.15.169:0/3898302105 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559fa575e970 0x559fa57caab0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f765c005780 tx=0x7f765c020be0 comp rx=0 tx=0).stop 2023-10-26T01:38:33.060 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.046+0000 7f7674277740 1 -- 172.21.15.169:0/3898302105 shutdown_connections 2023-10-26T01:38:33.060 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.046+0000 7f7674277740 1 --2- 172.21.15.169:0/3898302105 >> [v2:172.21.15.170:6835/3615088922,v1:172.21.15.170:6837/3615088922] conn(0x559fa57e5d00 0x559fa57e60e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:33.060 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.046+0000 7f7674277740 1 --2- 172.21.15.169:0/3898302105 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x559fa578cbe0 0x559fa5776a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:33.060 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.046+0000 7f7674277740 1 --2- 172.21.15.169:0/3898302105 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559fa575ed50 0x559fa5736480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:33.060 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.046+0000 7f7649ffb640 0 client.7938 ms_handle_reset on v2:172.21.15.170:6835/3615088922 2023-10-26T01:38:33.061 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.046+0000 7f7674277740 1 --2- 172.21.15.169:0/3898302105 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559fa575e970 0x559fa57caab0 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:33.061 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.046+0000 7f7674277740 1 -- 172.21.15.169:0/3898302105 >> 172.21.15.169:0/3898302105 conn(0x559fa57320e0 msgr2=0x559fa572d7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:33.061 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.046+0000 7f7674277740 1 -- 172.21.15.169:0/3898302105 shutdown_connections 2023-10-26T01:38:33.061 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.046+0000 7f7674277740 1 -- 172.21.15.169:0/3898302105 wait complete. 2023-10-26T01:38:33.061 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.047+0000 7f7674277740 20 client.7938 trim_cache size 0 max 16384 2023-10-26T01:38:33.062 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2023-10-26T01:38:33.062 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_fsid": "30203b28-ed8a-44e8-a9a6-a9bf25f980f1", 2023-10-26T01:38:33.062 INFO:teuthology.orchestra.run.smithi169.stdout: "whoami": 0, 2023-10-26T01:38:33.062 INFO:teuthology.orchestra.run.smithi169.stdout: "id": 7773, 2023-10-26T01:38:33.063 INFO:teuthology.orchestra.run.smithi169.stdout: "want_state": "up:active", 2023-10-26T01:38:33.063 INFO:teuthology.orchestra.run.smithi169.stdout: "state": "up:active", 2023-10-26T01:38:33.063 INFO:teuthology.orchestra.run.smithi169.stdout: "fs_name": "cephfs", 2023-10-26T01:38:33.063 INFO:teuthology.orchestra.run.smithi169.stdout: "rank_uptime": 11.723921591, 2023-10-26T01:38:33.063 INFO:teuthology.orchestra.run.smithi169.stdout: "mdsmap_epoch": 68, 2023-10-26T01:38:33.064 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_epoch": 136, 2023-10-26T01:38:33.064 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_epoch_barrier": 134, 2023-10-26T01:38:33.064 INFO:teuthology.orchestra.run.smithi169.stdout: "uptime": 17.649429304000002 2023-10-26T01:38:33.064 INFO:teuthology.orchestra.run.smithi169.stdout:} 2023-10-26T01:38:33.065 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.047+0000 7f766bfff640 1 -- 172.21.15.169:0/1251710184 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f764c06ae70 msgr2=0x7f764c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:33.066 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.047+0000 7f766bfff640 1 --2- 172.21.15.169:0/1251710184 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f764c06ae70 0x7f764c06d330 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f765c005e10 tx=0x7f765c005da0 comp rx=0 tx=0).stop 2023-10-26T01:38:33.066 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.047+0000 7f766bfff640 1 -- 172.21.15.169:0/1251710184 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f766c15d0d0 msgr2=0x7f766c15bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:33.066 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.047+0000 7f766bfff640 1 --2- 172.21.15.169:0/1251710184 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f766c15d0d0 0x7f766c15bf40 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f766400b970 tx=0x7f766400be30 comp rx=0 tx=0).stop 2023-10-26T01:38:33.066 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.047+0000 7f766bfff640 1 -- 172.21.15.169:0/1251710184 shutdown_connections 2023-10-26T01:38:33.066 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.047+0000 7f766bfff640 1 --2- 172.21.15.169:0/1251710184 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f764c06ae70 0x7f764c06d330 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:33.067 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.047+0000 7f766bfff640 1 --2- 172.21.15.169:0/1251710184 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f766c15d0d0 0x7f766c15bf40 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:33.067 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.047+0000 7f766bfff640 1 --2- 172.21.15.169:0/1251710184 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f766c153b90 0x7f766c155900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:33.067 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.047+0000 7f766bfff640 1 --2- 172.21.15.169:0/1251710184 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f766c0b0e50 0x7f766c152d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:33.067 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.047+0000 7f766bfff640 1 -- 172.21.15.169:0/1251710184 >> 172.21.15.169:0/1251710184 conn(0x7f766c0af110 msgr2=0x7f766c099da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:33.067 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.047+0000 7f766bfff640 1 -- 172.21.15.169:0/1251710184 shutdown_connections 2023-10-26T01:38:33.068 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.047+0000 7f766bfff640 1 -- 172.21.15.169:0/1251710184 wait complete. 2023-10-26T01:38:33.068 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-26T01:38:33.275 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.273+0000 7ff96ffff640 1 -- 172.21.15.169:0/912548430 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9700cab80 msgr2=0x7ff97006bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:33.275 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.273+0000 7ff96ffff640 1 --2- 172.21.15.169:0/912548430 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9700cab80 0x7ff97006bf10 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7ff9700acb60 tx=0x7ff95802bda0 comp rx=0 tx=0).stop 2023-10-26T01:38:33.276 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.273+0000 7ff96ffff640 1 -- 172.21.15.169:0/912548430 shutdown_connections 2023-10-26T01:38:33.276 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.273+0000 7ff96ffff640 1 --2- 172.21.15.169:0/912548430 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9700cab80 0x7ff97006bf10 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:33.276 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.273+0000 7ff96ffff640 1 --2- 172.21.15.169:0/912548430 >> v2:172.21.15.170:3300/0 conn(0x7ff9700ca7a0 0x7ff9700717a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:33.276 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.273+0000 7ff96ffff640 1 -- 172.21.15.169:0/912548430 >> 172.21.15.169:0/912548430 conn(0x7ff9700af0c0 msgr2=0x7ff9700af4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:33.276 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.273+0000 7ff96ffff640 1 -- 172.21.15.169:0/912548430 shutdown_connections 2023-10-26T01:38:33.277 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.273+0000 7ff96ffff640 1 -- 172.21.15.169:0/912548430 wait complete. 2023-10-26T01:38:33.278 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.275+0000 7ff96ffff640 1 Processor -- start 2023-10-26T01:38:33.278 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.275+0000 7ff96ffff640 1 -- start start 2023-10-26T01:38:33.278 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.276+0000 7ff96ffff640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff9700ca7a0 0x7ff970097c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:33.278 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.276+0000 7ff96ffff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff9700cab80 0x7ff97015b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:33.278 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.276+0000 7ff96ffff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff97015a2e0 0x7ff97013dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:33.279 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.276+0000 7ff96ffff640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff970099760 con 0x7ff97015a2e0 2023-10-26T01:38:33.279 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.276+0000 7ff96ffff640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff970099a40 con 0x7ff9700cab80 2023-10-26T01:38:33.279 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.276+0000 7ff96ffff640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff970099d20 con 0x7ff9700ca7a0 2023-10-26T01:38:33.279 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.276+0000 7ff96f7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff97015a2e0 0x7ff97013dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:33.279 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.276+0000 7ff96f7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff97015a2e0 0x7ff97013dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:34470/0 (socket says 172.21.15.169:34470) 2023-10-26T01:38:33.280 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.276+0000 7ff96f7fe640 1 -- 172.21.15.169:0/289141276 learned_addr learned my addr 172.21.15.169:0/289141276 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:38:33.281 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.277+0000 7ff967fff640 1 --2- 172.21.15.169:0/289141276 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff9700cab80 0x7ff97015b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:33.281 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.277+0000 7ff96f7fe640 1 -- 172.21.15.169:0/289141276 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff9700ca7a0 msgr2=0x7ff970097c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:33.282 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.277+0000 7ff96f7fe640 1 --2- 172.21.15.169:0/289141276 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff9700ca7a0 0x7ff970097c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:33.282 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.277+0000 7ff96f7fe640 1 -- 172.21.15.169:0/289141276 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff9700cab80 msgr2=0x7ff97015b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:33.282 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.277+0000 7ff96f7fe640 1 --2- 172.21.15.169:0/289141276 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff9700cab80 0x7ff97015b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:33.282 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.277+0000 7ff96f7fe640 1 -- 172.21.15.169:0/289141276 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff958009560 con 0x7ff97015a2e0 2023-10-26T01:38:33.282 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.277+0000 7ff96f7fe640 1 --2- 172.21.15.169:0/289141276 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff97015a2e0 0x7ff97013dfb0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7ff96800c6b0 tx=0x7ff96800cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:33.283 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.277+0000 7ff96cff9640 1 -- 172.21.15.169:0/289141276 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff968003b70 con 0x7ff97015a2e0 2023-10-26T01:38:33.283 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.277+0000 7ff96cff9640 1 -- 172.21.15.169:0/289141276 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff968003cf0 con 0x7ff97015a2e0 2023-10-26T01:38:33.283 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.277+0000 7ff96ffff640 1 -- 172.21.15.169:0/289141276 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9700ccd10 con 0x7ff97015a2e0 2023-10-26T01:38:33.283 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.277+0000 7ff96cff9640 1 -- 172.21.15.169:0/289141276 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff968005330 con 0x7ff97015a2e0 2023-10-26T01:38:33.283 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.277+0000 7ff96ffff640 1 -- 172.21.15.169:0/289141276 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff97009a2e0 con 0x7ff97015a2e0 2023-10-26T01:38:33.284 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.278+0000 7ff96ffff640 1 -- 172.21.15.169:0/289141276 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff97009b540 con 0x7ff97015a2e0 2023-10-26T01:38:33.284 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.279+0000 7ff96cff9640 1 -- 172.21.15.169:0/289141276 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ff968005490 con 0x7ff97015a2e0 2023-10-26T01:38:33.285 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.280+0000 7ff96cff9640 1 --2- 172.21.15.169:0/289141276 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff95006ae40 0x7ff95006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:33.285 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.280+0000 7ff96cff9640 1 -- 172.21.15.169:0/289141276 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5767+0+0 (secure 0 0 0) 0x7ff968089480 con 0x7ff97015a2e0 2023-10-26T01:38:33.285 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.281+0000 7ff96effd640 1 --2- 172.21.15.169:0/289141276 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff95006ae40 0x7ff95006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:33.286 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.282+0000 7ff96effd640 1 --2- 172.21.15.169:0/289141276 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff95006ae40 0x7ff95006d300 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7ff96000a8f0 tx=0x7ff960005bf0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:33.286 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.285+0000 7ff96cff9640 1 -- 172.21.15.169:0/289141276 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff968054450 con 0x7ff97015a2e0 2023-10-26T01:38:33.514 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.512+0000 7ff96ffff640 1 -- 172.21.15.169:0/289141276 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff97015a770 con 0x7ff97015a2e0 2023-10-26T01:38:33.515 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.513+0000 7ff96cff9640 1 -- 172.21.15.169:0/289141276 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v68) v1 ==== 53+0+73 (secure 0 0 0) 0x7ff97015a770 con 0x7ff97015a2e0 2023-10-26T01:38:33.515 INFO:teuthology.orchestra.run.smithi169.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-26T01:38:33.517 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.515+0000 7ff96ffff640 1 -- 172.21.15.169:0/289141276 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff95006ae40 msgr2=0x7ff95006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:33.517 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.515+0000 7ff96ffff640 1 --2- 172.21.15.169:0/289141276 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff95006ae40 0x7ff95006d300 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7ff96000a8f0 tx=0x7ff960005bf0 comp rx=0 tx=0).stop 2023-10-26T01:38:33.518 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.515+0000 7ff96ffff640 1 -- 172.21.15.169:0/289141276 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff97015a2e0 msgr2=0x7ff97013dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:33.518 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.515+0000 7ff96ffff640 1 --2- 172.21.15.169:0/289141276 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff97015a2e0 0x7ff97013dfb0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7ff96800c6b0 tx=0x7ff96800cb70 comp rx=0 tx=0).stop 2023-10-26T01:38:33.518 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.516+0000 7ff96ffff640 1 -- 172.21.15.169:0/289141276 shutdown_connections 2023-10-26T01:38:33.518 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.516+0000 7ff96ffff640 1 --2- 172.21.15.169:0/289141276 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff95006ae40 0x7ff95006d300 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:33.519 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.516+0000 7ff96ffff640 1 --2- 172.21.15.169:0/289141276 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff97015a2e0 0x7ff97013dfb0 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:33.519 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.516+0000 7ff96ffff640 1 --2- 172.21.15.169:0/289141276 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff9700cab80 0x7ff97015b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:33.519 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.516+0000 7ff96ffff640 1 --2- 172.21.15.169:0/289141276 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff9700ca7a0 0x7ff970097c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:33.519 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.516+0000 7ff96ffff640 1 -- 172.21.15.169:0/289141276 >> 172.21.15.169:0/289141276 conn(0x7ff9700af0c0 msgr2=0x7ff970070b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:33.520 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.516+0000 7ff96ffff640 1 -- 172.21.15.169:0/289141276 shutdown_connections 2023-10-26T01:38:33.520 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:33.516+0000 7ff96ffff640 1 -- 172.21.15.169:0/289141276 wait complete. 2023-10-26T01:38:33.536 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-26T01:38:33.536 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-26T01:38:33.537 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi169.front.sepia.ceph.com 2023-10-26T01:38:33.537 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-26T01:38:33.537 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-26T01:38:33.538 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-26T01:38:33.538 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-26T01:38:33.538 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-26T01:38:33.539 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-26T01:38:33.539 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:38:33.540 DEBUG:teuthology.orchestra.run.smithi169:> ip addr 2023-10-26T01:38:33.593 INFO:teuthology.orchestra.run.smithi169.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-26T01:38:33.593 INFO:teuthology.orchestra.run.smithi169.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-26T01:38:33.593 INFO:teuthology.orchestra.run.smithi169.stdout: inet 127.0.0.1/8 scope host lo 2023-10-26T01:38:33.594 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:38:33.594 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 ::1/128 scope host 2023-10-26T01:38:33.594 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:38:33.594 INFO:teuthology.orchestra.run.smithi169.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-26T01:38:33.594 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:d9:3b:b4 brd ff:ff:ff:ff:ff:ff 2023-10-26T01:38:33.595 INFO:teuthology.orchestra.run.smithi169.stdout: altname enp7s0f0 2023-10-26T01:38:33.595 INFO:teuthology.orchestra.run.smithi169.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-26T01:38:33.595 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:d9:3b:b5 brd ff:ff:ff:ff:ff:ff 2023-10-26T01:38:33.595 INFO:teuthology.orchestra.run.smithi169.stdout: altname enp7s0f1 2023-10-26T01:38:33.595 INFO:teuthology.orchestra.run.smithi169.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-26T01:38:33.596 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:88:7f:4c brd ff:ff:ff:ff:ff:ff 2023-10-26T01:38:33.596 INFO:teuthology.orchestra.run.smithi169.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-26T01:38:33.596 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:88:7f:4d brd ff:ff:ff:ff:ff:ff 2023-10-26T01:38:33.596 INFO:teuthology.orchestra.run.smithi169.stdout: inet 172.21.15.169/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-26T01:38:33.596 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft 41982sec preferred_lft 41982sec 2023-10-26T01:38:33.597 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 fe80::ec4:7aff:fe88:7f4d/64 scope link 2023-10-26T01:38:33.597 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:38:33.597 INFO:teuthology.orchestra.run.smithi169.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-26T01:38:33.597 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 22:a0:6b:43:7b:04 brd ff:ff:ff:ff:ff:ff 2023-10-26T01:38:33.597 INFO:teuthology.orchestra.run.smithi169.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-26T01:38:33.597 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:38:33.598 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 fe80::d815:31ff:fe87:d148/64 scope link 2023-10-26T01:38:33.598 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:38:33.598 INFO:teuthology.orchestra.run.smithi169.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-26T01:38:33.598 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 2a:80:f8:45:5f:9f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-26T01:38:33.598 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 fe80::2880:f8ff:fe45:5f9f/64 scope link 2023-10-26T01:38:33.599 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:38:33.599 INFO:teuthology.orchestra.run.smithi169.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-26T01:38:33.599 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 22:a0:6b:43:7b:04 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-26T01:38:33.599 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 fe80::20a0:6bff:fe43:7b04/64 scope link 2023-10-26T01:38:33.599 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:38:33.600 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:38:33.601 DEBUG:teuthology.orchestra.run.smithi169:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-26T01:38:33.693 INFO:teuthology.orchestra.run.smithi169.stdout:1 2023-10-26T01:38:33.696 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:38:33.697 DEBUG:teuthology.orchestra.run.smithi169:> route 2023-10-26T01:38:33.750 INFO:teuthology.orchestra.run.smithi169.stdout:Kernel IP routing table 2023-10-26T01:38:33.751 INFO:teuthology.orchestra.run.smithi169.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-26T01:38:33.751 INFO:teuthology.orchestra.run.smithi169.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-26T01:38:33.751 INFO:teuthology.orchestra.run.smithi169.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-26T01:38:33.751 INFO:teuthology.orchestra.run.smithi169.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-26T01:38:33.752 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:38:33.752 DEBUG:teuthology.orchestra.run.smithi169:> (cd / && exec bash -c ' 2023-10-26T01:38:33.753 DEBUG:teuthology.orchestra.run.smithi169:> set -e 2023-10-26T01:38:33.753 DEBUG:teuthology.orchestra.run.smithi169:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-26T01:38:33.753 DEBUG:teuthology.orchestra.run.smithi169:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-26T01:38:33.754 DEBUG:teuthology.orchestra.run.smithi169:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-26T01:38:33.754 DEBUG:teuthology.orchestra.run.smithi169:> ') 2023-10-26T01:38:33.938 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:38:33.939 DEBUG:teuthology.orchestra.run.smithi169:> ip netns list 2023-10-26T01:38:33.989 INFO:teuthology.orchestra.run.smithi169.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-26T01:38:33.990 INFO:teuthology.orchestra.run.smithi169.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-26T01:38:33.990 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-26T01:38:33.990 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:38:33.991 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link set brx.0 up 2023-10-26T01:38:34.082 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-26T01:38:34.083 DEBUG:teuthology.orchestra.run.smithi169:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-26T01:38:34.139 INFO:teuthology.orchestra.run.smithi169.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-26T01:38:34.139 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-26T01:38:34.139 DEBUG:teuthology.orchestra.run.smithi169:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-26T01:38:34.191 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-26T01:38:34.192 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:38:34.192 DEBUG:teuthology.orchestra.run.smithi169:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-26T01:38:34.336 INFO:teuthology.orchestra.run.smithi169.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-26T01:38:34.336 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-26T01:38:34.336 INFO:teuthology.orchestra.run.smithi169.stdout:invalid new device string format 2023-10-26T01:38:34.358 INFO:teuthology.orchestra.run.smithi169.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-26T01:38:34.358 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-26T01:38:34.359 INFO:teuthology.orchestra.run.smithi169.stdout:invalid new device string format 2023-10-26T01:38:34.359 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: resolved to: "172.21.15.169:3300,172.21.15.170:3300,172.21.15.170:3301" 2023-10-26T01:38:34.359 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-26T01:38:34.383 INFO:teuthology.orchestra.run.smithi169.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-26T01:38:34.383 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-26T01:38:34.383 INFO:teuthology.orchestra.run.smithi169.stdout:invalid new device string format 2023-10-26T01:38:34.384 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: resolved to: "172.21.15.169:3300,172.21.15.170:3300,172.21.15.170:3301" 2023-10-26T01:38:34.384 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-26T01:38:34.384 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: resolved to: "172.21.15.169:6789,172.21.15.170:6789,172.21.15.170:6790" 2023-10-26T01:38:34.384 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.169:6789,172.21.15.170:6789,172.21.15.170:6790:/ 2023-10-26T01:38:34.384 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=30203b28-ed8a-44e8-a9a6-a9bf25f980f1" will pass to kernel 2023-10-26T01:38:34.388 INFO:teuthology.orchestra.run.smithi169.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2023-10-26T01:38:34.388 INFO:teuthology.orchestra.run.smithi169.stdout: You just mounted a file system that supports labels which does not 2023-10-26T01:38:34.388 INFO:teuthology.orchestra.run.smithi169.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-10-26T01:38:34.388 INFO:teuthology.orchestra.run.smithi169.stdout: applications will generate AVC messages and not be allowed access to 2023-10-26T01:38:34.389 INFO:teuthology.orchestra.run.smithi169.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-10-26T01:38:34.391 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-26T01:38:34.392 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:38:34.392 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-26T01:38:34.469 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:38:34.469 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-26T01:38:34.546 INFO:teuthology.orchestra.run.smithi169.stdout:client7950 2023-10-26T01:38:34.547 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:38:34.547 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:38:34.607 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-26T01:38:34.830 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:34.827+0000 7fdd8f85d640 1 -- 172.21.15.169:0/977229839 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdd881597c0 msgr2=0x7fdd880ac200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:34.830 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:34.827+0000 7fdd8f85d640 1 --2- 172.21.15.169:0/977229839 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdd881597c0 0x7fdd880ac200 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fdd78005fd0 tx=0x7fdd7802be20 comp rx=0 tx=0).stop 2023-10-26T01:38:34.830 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:34.828+0000 7fdd8f85d640 1 -- 172.21.15.169:0/977229839 shutdown_connections 2023-10-26T01:38:34.831 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:34.828+0000 7fdd8f85d640 1 --2- 172.21.15.169:0/977229839 >> v2:172.21.15.170:3300/0 conn(0x7fdd880b0c80 0x7fdd880cd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:34.831 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:34.828+0000 7fdd8f85d640 1 --2- 172.21.15.169:0/977229839 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdd881597c0 0x7fdd880ac200 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:34.831 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:34.828+0000 7fdd8f85d640 1 -- 172.21.15.169:0/977229839 >> 172.21.15.169:0/977229839 conn(0x7fdd88069930 msgr2=0x7fdd88069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:34.831 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:34.828+0000 7fdd8f85d640 1 -- 172.21.15.169:0/977229839 shutdown_connections 2023-10-26T01:38:34.831 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:34.828+0000 7fdd8f85d640 1 -- 172.21.15.169:0/977229839 wait complete. 2023-10-26T01:38:34.832 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:34.830+0000 7fdd8f85d640 1 Processor -- start 2023-10-26T01:38:34.832 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:34.830+0000 7fdd8f85d640 1 -- start start 2023-10-26T01:38:34.834 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:34.831+0000 7fdd8f85d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd880b0c80 0x7fdd88154bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:34.834 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:34.831+0000 7fdd8f85d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd881593e0 0x7fdd8806ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:34.834 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:34.831+0000 7fdd8f85d640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdd881597c0 0x7fdd881e3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:34.834 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:34.831+0000 7fdd8f85d640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd881e21d0 con 0x7fdd880b0c80 2023-10-26T01:38:34.835 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:34.831+0000 7fdd8f85d640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd881e4670 con 0x7fdd881593e0 2023-10-26T01:38:34.835 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:34.831+0000 7fdd8f85d640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd881e4950 con 0x7fdd881597c0 2023-10-26T01:38:34.835 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:34.832+0000 7fdd8d5f9640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd880b0c80 0x7fdd88154bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:34.835 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:34.832+0000 7fdd8d5f9640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd880b0c80 0x7fdd88154bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:55252/0 (socket says 172.21.15.169:55252) 2023-10-26T01:38:34.835 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:34.832+0000 7fdd8d5f9640 1 -- 172.21.15.169:0/1718292358 learned_addr learned my addr 172.21.15.169:0/1718292358 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:38:34.836 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:34.832+0000 7fdd8ddfa640 1 --2- 172.21.15.169:0/1718292358 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdd881597c0 0x7fdd881e3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:34.836 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:34.832+0000 7fdd8cdf8640 1 --2- 172.21.15.169:0/1718292358 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd881593e0 0x7fdd8806ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:34.836 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:34.832+0000 7fdd8d5f9640 1 -- 172.21.15.169:0/1718292358 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdd881597c0 msgr2=0x7fdd881e3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:34.836 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:34.832+0000 7fdd8d5f9640 1 --2- 172.21.15.169:0/1718292358 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdd881597c0 0x7fdd881e3ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:34.836 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:34.832+0000 7fdd8d5f9640 1 -- 172.21.15.169:0/1718292358 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd881593e0 msgr2=0x7fdd8806ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:34.837 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:34.832+0000 7fdd8d5f9640 1 --2- 172.21.15.169:0/1718292358 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd881593e0 0x7fdd8806ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:34.837 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:34.832+0000 7fdd8d5f9640 1 -- 172.21.15.169:0/1718292358 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd78009560 con 0x7fdd880b0c80 2023-10-26T01:38:34.837 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:34.832+0000 7fdd8cdf8640 1 --2- 172.21.15.169:0/1718292358 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd881593e0 0x7fdd8806ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:38:34.837 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:34.832+0000 7fdd8d5f9640 1 --2- 172.21.15.169:0/1718292358 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd880b0c80 0x7fdd88154bc0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7fdd8400c400 tx=0x7fdd8400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:34.838 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:34.833+0000 7fdd7dffb640 1 -- 172.21.15.169:0/1718292358 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd840052d0 con 0x7fdd880b0c80 2023-10-26T01:38:34.839 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:34.833+0000 7fdd7dffb640 1 -- 172.21.15.169:0/1718292358 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fdd84007020 con 0x7fdd880b0c80 2023-10-26T01:38:34.839 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:34.833+0000 7fdd7dffb640 1 -- 172.21.15.169:0/1718292358 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd84007470 con 0x7fdd880b0c80 2023-10-26T01:38:34.839 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:34.833+0000 7fdd8f85d640 1 -- 172.21.15.169:0/1718292358 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd881e4c30 con 0x7fdd880b0c80 2023-10-26T01:38:34.839 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:34.833+0000 7fdd8f85d640 1 -- 172.21.15.169:0/1718292358 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd8814ff90 con 0x7fdd880b0c80 2023-10-26T01:38:34.839 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:34.834+0000 7fdd8f85d640 1 -- 172.21.15.169:0/1718292358 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd8804db10 con 0x7fdd880b0c80 2023-10-26T01:38:34.840 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:34.835+0000 7fdd7dffb640 1 -- 172.21.15.169:0/1718292358 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fdd840067d0 con 0x7fdd880b0c80 2023-10-26T01:38:34.840 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:34.836+0000 7fdd7dffb640 1 --2- 172.21.15.169:0/1718292358 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fdd6c06ae40 0x7fdd6c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:34.840 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:34.836+0000 7fdd7dffb640 1 -- 172.21.15.169:0/1718292358 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fdd84088cf0 con 0x7fdd880b0c80 2023-10-26T01:38:34.840 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:34.837+0000 7fdd8cdf8640 1 --2- 172.21.15.169:0/1718292358 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fdd6c06ae40 0x7fdd6c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:34.841 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:34.838+0000 7fdd8cdf8640 1 --2- 172.21.15.169:0/1718292358 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fdd6c06ae40 0x7fdd6c06d300 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fdd78035000 tx=0x7fdd78004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:34.842 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:34.840+0000 7fdd7dffb640 1 -- 172.21.15.169:0/1718292358 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdd84053d50 con 0x7fdd880b0c80 2023-10-26T01:38:35.069 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:35.068+0000 7fdd8f85d640 1 -- 172.21.15.169:0/1718292358 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fdd880b2170 con 0x7fdd880b0c80 2023-10-26T01:38:35.070 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:35.068+0000 7fdd7dffb640 1 -- 172.21.15.169:0/1718292358 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v68) v1 ==== 53+0+73 (secure 0 0 0) 0x7fdd84057370 con 0x7fdd880b0c80 2023-10-26T01:38:35.070 INFO:teuthology.orchestra.run.smithi169.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-26T01:38:35.073 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:35.071+0000 7fdd8f85d640 1 -- 172.21.15.169:0/1718292358 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fdd6c06ae40 msgr2=0x7fdd6c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:35.074 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:35.071+0000 7fdd8f85d640 1 --2- 172.21.15.169:0/1718292358 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fdd6c06ae40 0x7fdd6c06d300 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fdd78035000 tx=0x7fdd78004730 comp rx=0 tx=0).stop 2023-10-26T01:38:35.074 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:35.071+0000 7fdd8f85d640 1 -- 172.21.15.169:0/1718292358 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd880b0c80 msgr2=0x7fdd88154bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:35.074 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:35.071+0000 7fdd8f85d640 1 --2- 172.21.15.169:0/1718292358 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd880b0c80 0x7fdd88154bc0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7fdd8400c400 tx=0x7fdd8400c8c0 comp rx=0 tx=0).stop 2023-10-26T01:38:35.074 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:35.071+0000 7fdd8f85d640 1 -- 172.21.15.169:0/1718292358 shutdown_connections 2023-10-26T01:38:35.075 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:35.071+0000 7fdd8f85d640 1 --2- 172.21.15.169:0/1718292358 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fdd6c06ae40 0x7fdd6c06d300 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:35.075 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:35.071+0000 7fdd8f85d640 1 --2- 172.21.15.169:0/1718292358 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdd881597c0 0x7fdd881e3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:35.075 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:35.071+0000 7fdd8f85d640 1 --2- 172.21.15.169:0/1718292358 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd881593e0 0x7fdd8806ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:35.075 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:35.071+0000 7fdd8f85d640 1 --2- 172.21.15.169:0/1718292358 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd880b0c80 0x7fdd88154bc0 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:35.075 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:35.071+0000 7fdd8f85d640 1 -- 172.21.15.169:0/1718292358 >> 172.21.15.169:0/1718292358 conn(0x7fdd88069930 msgr2=0x7fdd880acf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:35.076 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:35.071+0000 7fdd8f85d640 1 -- 172.21.15.169:0/1718292358 shutdown_connections 2023-10-26T01:38:35.076 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:35.072+0000 7fdd8f85d640 1 -- 172.21.15.169:0/1718292358 wait complete. 2023-10-26T01:38:35.092 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-26T01:38:35.093 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-26T01:38:35.093 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi170.front.sepia.ceph.com 2023-10-26T01:38:35.094 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-26T01:38:35.094 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-26T01:38:35.094 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-26T01:38:35.095 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-26T01:38:35.095 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-26T01:38:35.095 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-26T01:38:35.096 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:38:35.096 DEBUG:teuthology.orchestra.run.smithi170:> ip addr 2023-10-26T01:38:35.160 INFO:teuthology.orchestra.run.smithi170.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-26T01:38:35.160 INFO:teuthology.orchestra.run.smithi170.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-26T01:38:35.161 INFO:teuthology.orchestra.run.smithi170.stdout: inet 127.0.0.1/8 scope host lo 2023-10-26T01:38:35.161 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:38:35.161 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 ::1/128 scope host 2023-10-26T01:38:35.161 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:38:35.161 INFO:teuthology.orchestra.run.smithi170.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-26T01:38:35.161 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:d9:3b:54 brd ff:ff:ff:ff:ff:ff 2023-10-26T01:38:35.162 INFO:teuthology.orchestra.run.smithi170.stdout: altname enp7s0f0 2023-10-26T01:38:35.162 INFO:teuthology.orchestra.run.smithi170.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-26T01:38:35.162 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:d9:3b:55 brd ff:ff:ff:ff:ff:ff 2023-10-26T01:38:35.162 INFO:teuthology.orchestra.run.smithi170.stdout: altname enp7s0f1 2023-10-26T01:38:35.162 INFO:teuthology.orchestra.run.smithi170.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-26T01:38:35.163 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:88:72:54 brd ff:ff:ff:ff:ff:ff 2023-10-26T01:38:35.163 INFO:teuthology.orchestra.run.smithi170.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-26T01:38:35.163 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:88:72:55 brd ff:ff:ff:ff:ff:ff 2023-10-26T01:38:35.163 INFO:teuthology.orchestra.run.smithi170.stdout: inet 172.21.15.170/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-26T01:38:35.163 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft 41965sec preferred_lft 41965sec 2023-10-26T01:38:35.164 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::ec4:7aff:fe88:7255/64 scope link 2023-10-26T01:38:35.164 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:38:35.164 INFO:teuthology.orchestra.run.smithi170.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-26T01:38:35.164 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 32:b5:d1:ee:10:36 brd ff:ff:ff:ff:ff:ff 2023-10-26T01:38:35.164 INFO:teuthology.orchestra.run.smithi170.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-26T01:38:35.165 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:38:35.165 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::28ed:d1ff:fe37:70d/64 scope link 2023-10-26T01:38:35.165 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:38:35.165 INFO:teuthology.orchestra.run.smithi170.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-26T01:38:35.165 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 32:b5:d1:ee:10:36 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-26T01:38:35.166 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::30b5:d1ff:feee:1036/64 scope link 2023-10-26T01:38:35.166 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:38:35.166 INFO:teuthology.orchestra.run.smithi170.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-26T01:38:35.166 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether b2:c1:53:19:69:e4 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-26T01:38:35.166 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::b0c1:53ff:fe19:69e4/64 scope link 2023-10-26T01:38:35.166 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:38:35.167 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:38:35.167 DEBUG:teuthology.orchestra.run.smithi170:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-26T01:38:35.256 INFO:teuthology.orchestra.run.smithi170.stdout:1 2023-10-26T01:38:35.260 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:38:35.260 DEBUG:teuthology.orchestra.run.smithi170:> route 2023-10-26T01:38:35.314 INFO:teuthology.orchestra.run.smithi170.stdout:Kernel IP routing table 2023-10-26T01:38:35.314 INFO:teuthology.orchestra.run.smithi170.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-26T01:38:35.315 INFO:teuthology.orchestra.run.smithi170.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-26T01:38:35.315 INFO:teuthology.orchestra.run.smithi170.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-26T01:38:35.315 INFO:teuthology.orchestra.run.smithi170.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-26T01:38:35.315 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:38:35.316 DEBUG:teuthology.orchestra.run.smithi170:> (cd / && exec bash -c ' 2023-10-26T01:38:35.316 DEBUG:teuthology.orchestra.run.smithi170:> set -e 2023-10-26T01:38:35.317 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-26T01:38:35.317 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-26T01:38:35.317 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-26T01:38:35.318 DEBUG:teuthology.orchestra.run.smithi170:> ') 2023-10-26T01:38:35.495 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:38:35.495 DEBUG:teuthology.orchestra.run.smithi170:> ip netns list 2023-10-26T01:38:35.545 INFO:teuthology.orchestra.run.smithi170.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-26T01:38:35.545 INFO:teuthology.orchestra.run.smithi170.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-26T01:38:35.546 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-26T01:38:35.546 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:38:35.547 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip link set brx.0 up 2023-10-26T01:38:35.639 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-26T01:38:35.640 DEBUG:teuthology.orchestra.run.smithi170:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-26T01:38:35.696 INFO:teuthology.orchestra.run.smithi170.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-26T01:38:35.696 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-26T01:38:35.697 DEBUG:teuthology.orchestra.run.smithi170:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-26T01:38:35.748 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-26T01:38:35.748 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:38:35.749 DEBUG:teuthology.orchestra.run.smithi170:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-26T01:38:35.882 INFO:teuthology.orchestra.run.smithi170.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-26T01:38:35.882 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-26T01:38:35.883 INFO:teuthology.orchestra.run.smithi170.stdout:invalid new device string format 2023-10-26T01:38:35.904 INFO:teuthology.orchestra.run.smithi170.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-26T01:38:35.905 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-26T01:38:35.905 INFO:teuthology.orchestra.run.smithi170.stdout:invalid new device string format 2023-10-26T01:38:35.905 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: resolved to: "172.21.15.169:3300,172.21.15.170:3300,172.21.15.170:3301" 2023-10-26T01:38:35.905 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-26T01:38:35.930 INFO:teuthology.orchestra.run.smithi170.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-26T01:38:35.930 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-26T01:38:35.931 INFO:teuthology.orchestra.run.smithi170.stdout:invalid new device string format 2023-10-26T01:38:35.931 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: resolved to: "172.21.15.169:3300,172.21.15.170:3300,172.21.15.170:3301" 2023-10-26T01:38:35.931 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-26T01:38:35.931 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: resolved to: "172.21.15.169:6789,172.21.15.170:6789,172.21.15.170:6790" 2023-10-26T01:38:35.931 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.169:6789,172.21.15.170:6789,172.21.15.170:6790:/ 2023-10-26T01:38:35.931 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=30203b28-ed8a-44e8-a9a6-a9bf25f980f1" will pass to kernel 2023-10-26T01:38:35.934 INFO:teuthology.orchestra.run.smithi170.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2023-10-26T01:38:35.935 INFO:teuthology.orchestra.run.smithi170.stdout: You just mounted a file system that supports labels which does not 2023-10-26T01:38:35.935 INFO:teuthology.orchestra.run.smithi170.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-10-26T01:38:35.935 INFO:teuthology.orchestra.run.smithi170.stdout: applications will generate AVC messages and not be allowed access to 2023-10-26T01:38:35.935 INFO:teuthology.orchestra.run.smithi170.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-10-26T01:38:35.938 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-26T01:38:35.939 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:38:35.939 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-26T01:38:36.023 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:38:36.024 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-26T01:38:36.076 INFO:teuthology.orchestra.run.smithi170.stdout:client5576 2023-10-26T01:38:36.076 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-26T01:38:36.077 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:38:36.128 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T01:38:36.363 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:36.361+0000 7f682aee5640 1 -- 172.21.15.169:0/1768707664 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68100047c0 msgr2=0x7f6810004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:36.364 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:36.361+0000 7f682aee5640 1 --2- 172.21.15.169:0/1768707664 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68100047c0 0x7f6810004c40 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f680c001680 tx=0x7f68100057f0 comp rx=0 tx=0).stop 2023-10-26T01:38:36.364 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:36.361+0000 7f6829482640 1 -- 172.21.15.169:0/1768707664 reap_dead start 2023-10-26T01:38:36.364 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:36.361+0000 7f682aee5640 1 -- 172.21.15.169:0/1768707664 shutdown_connections 2023-10-26T01:38:36.364 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:36.361+0000 7f682aee5640 1 -- 172.21.15.169:0/1768707664 >> 172.21.15.169:0/1768707664 conn(0x7f6824069930 msgr2=0x7f6824069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:36.364 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:36.361+0000 7f682aee5640 1 -- 172.21.15.169:0/1768707664 shutdown_connections 2023-10-26T01:38:36.365 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:36.361+0000 7f682aee5640 1 -- 172.21.15.169:0/1768707664 wait complete. 2023-10-26T01:38:36.365 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:36.363+0000 7f682aee5640 1 Processor -- start 2023-10-26T01:38:36.365 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:36.363+0000 7f682aee5640 1 -- start start 2023-10-26T01:38:36.367 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:36.364+0000 7f682aee5640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f68100047c0 0x7f6824146560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:36.367 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:36.364+0000 7f682aee5640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6824146ca0 0x7f6824153de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:36.367 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:36.364+0000 7f682aee5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f68241514e0 0x7f6824151960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:36.368 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:36.364+0000 7f682aee5640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68241577d0 con 0x7f6824146ca0 2023-10-26T01:38:36.368 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:36.364+0000 7f682aee5640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68241f1b20 con 0x7f68241514e0 2023-10-26T01:38:36.368 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:36.364+0000 7f682aee5640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6824156b50 con 0x7f68100047c0 2023-10-26T01:38:36.368 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:36.365+0000 7f6823fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6824146ca0 0x7f6824153de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:36.368 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:36.365+0000 7f6823fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6824146ca0 0x7f6824153de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:55276/0 (socket says 172.21.15.169:55276) 2023-10-26T01:38:36.369 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:36.365+0000 7f6823fff640 1 -- 172.21.15.169:0/3799953748 learned_addr learned my addr 172.21.15.169:0/3799953748 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:38:36.369 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:36.365+0000 7f6828c81640 1 --2- 172.21.15.169:0/3799953748 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f68100047c0 0x7f6824146560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:36.369 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:36.365+0000 7f6829482640 1 --2- 172.21.15.169:0/3799953748 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f68241514e0 0x7f6824151960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:36.369 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:36.365+0000 7f6823fff640 1 -- 172.21.15.169:0/3799953748 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f68100047c0 msgr2=0x7f6824146560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:36.369 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:36.365+0000 7f6823fff640 1 --2- 172.21.15.169:0/3799953748 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f68100047c0 0x7f6824146560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:36.370 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:36.365+0000 7f6823fff640 1 -- 172.21.15.169:0/3799953748 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f68241514e0 msgr2=0x7f6824151960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:36.370 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:36.365+0000 7f6823fff640 1 --2- 172.21.15.169:0/3799953748 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f68241514e0 0x7f6824151960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:36.370 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:36.365+0000 7f6823fff640 1 -- 172.21.15.169:0/3799953748 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6824156e30 con 0x7f6824146ca0 2023-10-26T01:38:36.370 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:36.365+0000 7f6829482640 1 --2- 172.21.15.169:0/3799953748 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f68241514e0 0x7f6824151960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:38:36.370 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:36.366+0000 7f6823fff640 1 --2- 172.21.15.169:0/3799953748 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6824146ca0 0x7f6824153de0 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f680c0017a0 tx=0x7f680c006360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:36.372 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:36.366+0000 7f68217fa640 1 -- 172.21.15.169:0/3799953748 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f680c003b80 con 0x7f6824146ca0 2023-10-26T01:38:36.372 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:36.366+0000 7f68217fa640 1 -- 172.21.15.169:0/3799953748 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f680c003dd0 con 0x7f6824146ca0 2023-10-26T01:38:36.372 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:36.366+0000 7f682aee5640 1 -- 172.21.15.169:0/3799953748 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6824157110 con 0x7f6824146ca0 2023-10-26T01:38:36.372 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:36.366+0000 7f68217fa640 1 -- 172.21.15.169:0/3799953748 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f680c02c7b0 con 0x7f6824146ca0 2023-10-26T01:38:36.372 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:36.366+0000 7f682aee5640 1 -- 172.21.15.169:0/3799953748 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6824154320 con 0x7f6824146ca0 2023-10-26T01:38:36.373 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:36.367+0000 7f682aee5640 1 -- 172.21.15.169:0/3799953748 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f682404db10 con 0x7f6824146ca0 2023-10-26T01:38:36.373 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:36.368+0000 7f68217fa640 1 -- 172.21.15.169:0/3799953748 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f680c02c910 con 0x7f6824146ca0 2023-10-26T01:38:36.373 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:36.369+0000 7f68217fa640 1 --2- 172.21.15.169:0/3799953748 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f680406ae40 0x7f680406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:36.373 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:36.369+0000 7f68217fa640 1 -- 172.21.15.169:0/3799953748 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f680c0aa950 con 0x7f6824146ca0 2023-10-26T01:38:36.373 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:36.370+0000 7f6828c81640 1 --2- 172.21.15.169:0/3799953748 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f680406ae40 0x7f680406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:36.374 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:36.371+0000 7f6828c81640 1 --2- 172.21.15.169:0/3799953748 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f680406ae40 0x7f680406d300 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f6814001eb0 tx=0x7f6814001640 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:36.375 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:36.374+0000 7f68217fa640 1 -- 172.21.15.169:0/3799953748 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f680c0759b0 con 0x7f6824146ca0 2023-10-26T01:38:36.500 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T01:38:36.509 DEBUG:teuthology.orchestra.run.smithi170:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T01:38:36.602 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:36.601+0000 7f682aee5640 1 -- 172.21.15.169:0/3799953748 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f682406c990 con 0x7f6824146ca0 2023-10-26T01:38:36.603 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:36.602+0000 7f68217fa640 1 -- 172.21.15.169:0/3799953748 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+4927 (secure 0 0 0) 0x7f680c078fd0 con 0x7f6824146ca0 2023-10-26T01:38:36.604 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:38:36.604 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":68,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5917,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/3297412140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":3297412140},{"type":"v1","addr":"172.21.15.169:6835","nonce":3297412140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":5929,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/2575809940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2575809940},{"type":"v1","addr":"172.21.15.170:6836","nonce":2575809940}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":7761,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2632217991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2632217991},{"type":"v1","addr":"172.21.15.169:6833","nonce":2632217991}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T01:38:20.773556+0000","modified":"2023-10-26T01:38:23.839758+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7773},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7773":{"gid":7773,"name":"b","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6837/3615088922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":3615088922},{"type":"v1","addr":"172.21.15.170:6837","nonce":3615088922}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-10-26T01:38:36.604 INFO:teuthology.orchestra.run.smithi169.stderr:dumped fsmap epoch 68 2023-10-26T01:38:36.606 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:36.604+0000 7f682aee5640 1 -- 172.21.15.169:0/3799953748 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f680406ae40 msgr2=0x7f680406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:36.607 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:36.604+0000 7f682aee5640 1 --2- 172.21.15.169:0/3799953748 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f680406ae40 0x7f680406d300 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f6814001eb0 tx=0x7f6814001640 comp rx=0 tx=0).stop 2023-10-26T01:38:36.607 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:36.604+0000 7f682aee5640 1 -- 172.21.15.169:0/3799953748 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6824146ca0 msgr2=0x7f6824153de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:36.607 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:36.604+0000 7f682aee5640 1 --2- 172.21.15.169:0/3799953748 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6824146ca0 0x7f6824153de0 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f680c0017a0 tx=0x7f680c006360 comp rx=0 tx=0).stop 2023-10-26T01:38:36.607 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:36.604+0000 7f682aee5640 1 -- 172.21.15.169:0/3799953748 shutdown_connections 2023-10-26T01:38:36.608 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:36.604+0000 7f682aee5640 1 --2- 172.21.15.169:0/3799953748 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f680406ae40 0x7f680406d300 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:36.608 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:36.604+0000 7f682aee5640 1 --2- 172.21.15.169:0/3799953748 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6824146ca0 0x7f6824153de0 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:36.608 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:36.604+0000 7f682aee5640 1 --2- 172.21.15.169:0/3799953748 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f68241514e0 0x7f6824151960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:36.608 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:36.604+0000 7f682aee5640 1 --2- 172.21.15.169:0/3799953748 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f68100047c0 0x7f6824146560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:36.608 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:36.604+0000 7f682aee5640 1 -- 172.21.15.169:0/3799953748 >> 172.21.15.169:0/3799953748 conn(0x7f6824069930 msgr2=0x7f682415c090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:36.609 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:36.605+0000 7f682aee5640 1 -- 172.21.15.169:0/3799953748 shutdown_connections 2023-10-26T01:38:36.609 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:36.605+0000 7f682aee5640 1 -- 172.21.15.169:0/3799953748 wait complete. 2023-10-26T01:38:36.625 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2023-10-26T01:38:36.852 INFO:teuthology.orchestra.run.smithi170.stdout:{"cluster_fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","whoami":0,"id":7773,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":15.513634438,"mdsmap_epoch":68,"osdmap_epoch":136,"osdmap_epoch_barrier":134,"uptime":21.439132240999999} 2023-10-26T01:38:36.853 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "30203b28-ed8a-44e8-a9a6-a9bf25f980f1", "fs_name": "cephfs", "id": 7773, "mdsmap_epoch": 68, "osdmap_epoch": 136, "osdmap_epoch_barrier": 134, "rank_uptime": 15.513634438, "state": "up:active", "uptime": 21.439132241, "want_state": "up:active", "whoami": 0 } 2023-10-26T01:38:36.854 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set-quota cephfs_data max_bytes 33554432 2023-10-26T01:38:37.087 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:37.085+0000 7f4e7759e640 1 -- 172.21.15.169:0/2451447064 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4e7806ca60 msgr2=0x7f4e78157e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:37.087 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:37.085+0000 7f4e7759e640 1 --2- 172.21.15.169:0/2451447064 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4e7806ca60 0x7f4e78157e30 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f4e68005fd0 tx=0x7f4e6802be10 comp rx=0 tx=0).stop 2023-10-26T01:38:37.088 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:37.085+0000 7f4e7759e640 1 -- 172.21.15.169:0/2451447064 shutdown_connections 2023-10-26T01:38:37.088 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:37.085+0000 7f4e7759e640 1 --2- 172.21.15.169:0/2451447064 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4e78159fa0 0x7f4e7815a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:37.088 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:37.085+0000 7f4e7759e640 1 --2- 172.21.15.169:0/2451447064 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4e7806ca60 0x7f4e78157e30 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:37.088 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:37.085+0000 7f4e7759e640 1 -- 172.21.15.169:0/2451447064 >> 172.21.15.169:0/2451447064 conn(0x7f4e78069b40 msgr2=0x7f4e780aeb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:37.088 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:37.085+0000 7f4e7759e640 1 -- 172.21.15.169:0/2451447064 shutdown_connections 2023-10-26T01:38:37.089 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:37.085+0000 7f4e7759e640 1 -- 172.21.15.169:0/2451447064 wait complete. 2023-10-26T01:38:37.089 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:37.087+0000 7f4e7759e640 1 Processor -- start 2023-10-26T01:38:37.089 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:37.087+0000 7f4e7759e640 1 -- start start 2023-10-26T01:38:37.091 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:37.088+0000 7f4e7759e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e7806ca60 0x7f4e78148b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:37.091 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:37.088+0000 7f4e7759e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4e78158370 0x7f4e781e41f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:37.091 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:37.088+0000 7f4e7759e640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4e78159fa0 0x7f4e781e4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:37.092 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:37.088+0000 7f4e7759e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e780ae7d0 con 0x7f4e7806ca60 2023-10-26T01:38:37.092 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:37.088+0000 7f4e7759e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e781f3890 con 0x7f4e78158370 2023-10-26T01:38:37.092 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:37.088+0000 7f4e7759e640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e78149080 con 0x7f4e78159fa0 2023-10-26T01:38:37.092 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:37.089+0000 7f4e7659c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e7806ca60 0x7f4e78148b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:37.092 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:37.089+0000 7f4e7659c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e7806ca60 0x7f4e78148b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:55278/0 (socket says 172.21.15.169:55278) 2023-10-26T01:38:37.093 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:37.089+0000 7f4e7659c640 1 -- 172.21.15.169:0/2323008269 learned_addr learned my addr 172.21.15.169:0/2323008269 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:38:37.093 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:37.089+0000 7f4e6ffff640 1 --2- 172.21.15.169:0/2323008269 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4e78158370 0x7f4e781e41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:37.093 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:37.089+0000 7f4e76d9d640 1 --2- 172.21.15.169:0/2323008269 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4e78159fa0 0x7f4e781e4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:37.093 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:37.089+0000 7f4e7659c640 1 -- 172.21.15.169:0/2323008269 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4e78159fa0 msgr2=0x7f4e781e4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:37.093 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:37.089+0000 7f4e7659c640 1 --2- 172.21.15.169:0/2323008269 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4e78159fa0 0x7f4e781e4730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:37.094 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:37.089+0000 7f4e7659c640 1 -- 172.21.15.169:0/2323008269 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4e78158370 msgr2=0x7f4e781e41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:37.094 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:37.089+0000 7f4e7659c640 1 --2- 172.21.15.169:0/2323008269 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4e78158370 0x7f4e781e41f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:37.094 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:37.089+0000 7f4e7659c640 1 -- 172.21.15.169:0/2323008269 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e68009560 con 0x7f4e7806ca60 2023-10-26T01:38:37.094 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:37.090+0000 7f4e7659c640 1 --2- 172.21.15.169:0/2323008269 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e7806ca60 0x7f4e78148b40 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f4e68006010 tx=0x7f4e6802da70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:37.095 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:37.090+0000 7f4e6effd640 1 -- 172.21.15.169:0/2323008269 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e6802de70 con 0x7f4e7806ca60 2023-10-26T01:38:37.096 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:37.090+0000 7f4e6effd640 1 -- 172.21.15.169:0/2323008269 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4e680036e0 con 0x7f4e7806ca60 2023-10-26T01:38:37.096 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:37.090+0000 7f4e6effd640 1 -- 172.21.15.169:0/2323008269 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e68003af0 con 0x7f4e7806ca60 2023-10-26T01:38:37.096 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:37.090+0000 7f4e7759e640 1 -- 172.21.15.169:0/2323008269 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e780ae4f0 con 0x7f4e7806ca60 2023-10-26T01:38:37.096 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:37.090+0000 7f4e7759e640 1 -- 172.21.15.169:0/2323008269 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e781f3520 con 0x7f4e7806ca60 2023-10-26T01:38:37.096 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:37.091+0000 7f4e7759e640 1 -- 172.21.15.169:0/2323008269 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e781f3080 con 0x7f4e7806ca60 2023-10-26T01:38:37.097 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:37.092+0000 7f4e6effd640 1 -- 172.21.15.169:0/2323008269 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f4e68003c50 con 0x7f4e7806ca60 2023-10-26T01:38:37.097 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:37.093+0000 7f4e6effd640 1 --2- 172.21.15.169:0/2323008269 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f4e5806ae40 0x7f4e5806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:37.097 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:37.093+0000 7f4e6effd640 1 -- 172.21.15.169:0/2323008269 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f4e680ac270 con 0x7f4e7806ca60 2023-10-26T01:38:37.097 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:37.094+0000 7f4e6ffff640 1 --2- 172.21.15.169:0/2323008269 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f4e5806ae40 0x7f4e5806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:37.098 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:37.095+0000 7f4e6ffff640 1 --2- 172.21.15.169:0/2323008269 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f4e5806ae40 0x7f4e5806d300 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f4e781582e0 tx=0x7f4e60009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:37.099 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:37.098+0000 7f4e6effd640 1 -- 172.21.15.169:0/2323008269 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4e68077360 con 0x7f4e7806ca60 2023-10-26T01:38:37.291 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:37.289+0000 7f4e7759e640 1 -- 172.21.15.169:0/2323008269 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"} v 0) v1 -- 0x7f4e78073c10 con 0x7f4e7806ca60 2023-10-26T01:38:38.130 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:38.128+0000 7f4e6effd640 1 -- 172.21.15.169:0/2323008269 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"}]=0 set-quota max_bytes = 33554432 for pool cephfs_data v137) v1 ==== 181+0+0 (secure 0 0 0) 0x7f4e6807a980 con 0x7f4e7806ca60 2023-10-26T01:38:38.131 INFO:teuthology.orchestra.run.smithi169.stderr:set-quota max_bytes = 33554432 for pool cephfs_data 2023-10-26T01:38:38.134 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:38.132+0000 7f4e7759e640 1 -- 172.21.15.169:0/2323008269 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f4e5806ae40 msgr2=0x7f4e5806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:38.135 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:38.132+0000 7f4e7759e640 1 --2- 172.21.15.169:0/2323008269 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f4e5806ae40 0x7f4e5806d300 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f4e781582e0 tx=0x7f4e60009040 comp rx=0 tx=0).stop 2023-10-26T01:38:38.135 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:38.132+0000 7f4e7759e640 1 -- 172.21.15.169:0/2323008269 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e7806ca60 msgr2=0x7f4e78148b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:38.135 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:38.132+0000 7f4e7759e640 1 --2- 172.21.15.169:0/2323008269 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e7806ca60 0x7f4e78148b40 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f4e68006010 tx=0x7f4e6802da70 comp rx=0 tx=0).stop 2023-10-26T01:38:38.135 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:38.132+0000 7f4e7759e640 1 -- 172.21.15.169:0/2323008269 shutdown_connections 2023-10-26T01:38:38.136 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:38.132+0000 7f4e7759e640 1 --2- 172.21.15.169:0/2323008269 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f4e5806ae40 0x7f4e5806d300 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:38.136 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:38.132+0000 7f4e7759e640 1 --2- 172.21.15.169:0/2323008269 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4e78159fa0 0x7f4e781e4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:38.136 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:38.132+0000 7f4e7759e640 1 --2- 172.21.15.169:0/2323008269 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4e78158370 0x7f4e781e41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:38.136 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:38.132+0000 7f4e7759e640 1 --2- 172.21.15.169:0/2323008269 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e7806ca60 0x7f4e78148b40 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:38.136 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:38.132+0000 7f4e7759e640 1 -- 172.21.15.169:0/2323008269 >> 172.21.15.169:0/2323008269 conn(0x7f4e78069b40 msgr2=0x7f4e7814a8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:38.136 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:38.133+0000 7f4e7759e640 1 -- 172.21.15.169:0/2323008269 shutdown_connections 2023-10-26T01:38:38.137 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:38.133+0000 7f4e7759e640 1 -- 172.21.15.169:0/2323008269 wait complete. 2023-10-26T01:38:38.155 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2023-10-26T01:38:38.379 INFO:teuthology.orchestra.run.smithi169.stdout:{"osd_mon_report_interval":"5"} 2023-10-26T01:38:38.379 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2023-10-26T01:38:38.380 INFO:tasks.cephfs.test_full:Writing 32MB should fill this cluster 2023-10-26T01:38:38.380 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:38:38.381 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:38:38.436 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:38:38.437 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_a bs=1M conv=fdatasync count=16 seek=0) 2023-10-26T01:38:38.693 INFO:teuthology.orchestra.run.smithi169.stderr:16+0 records in 2023-10-26T01:38:38.792 INFO:teuthology.orchestra.run.smithi169.stderr:16+0 records out 2023-10-26T01:38:38.793 INFO:teuthology.orchestra.run.smithi169.stderr:16777216 bytes (17 MB, 16 MiB) copied, 0.196053 s, 85.6 MB/s 2023-10-26T01:38:38.794 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:38:38.794 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:38:38.854 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:38:38.854 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=17 seek=0) 2023-10-26T01:38:39.118 INFO:teuthology.orchestra.run.smithi169.stderr:17+0 records in 2023-10-26T01:38:39.118 INFO:teuthology.orchestra.run.smithi169.stderr:17+0 records out 2023-10-26T01:38:39.118 INFO:teuthology.orchestra.run.smithi169.stderr:17825792 bytes (18 MB, 17 MiB) copied, 0.206134 s, 86.5 MB/s 2023-10-26T01:38:39.119 INFO:tasks.cephfs.test_full:Writing file B succeeded (full status will happen soon) 2023-10-26T01:38:39.120 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-26T01:38:39.328 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:39.325+0000 7fa6c3fff640 1 -- 172.21.15.169:0/1000581116 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa6b0003fa0 msgr2=0x7fa6b00047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:39.328 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:39.325+0000 7fa6c3fff640 1 --2- 172.21.15.169:0/1000581116 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa6b0003fa0 0x7fa6b00047c0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac00c260 tx=0x7fa6b00057f0 comp rx=0 tx=0).stop 2023-10-26T01:38:39.328 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:39.325+0000 7fa6c37fe640 1 -- 172.21.15.169:0/1000581116 reap_dead start 2023-10-26T01:38:39.328 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:39.326+0000 7fa6c3fff640 1 -- 172.21.15.169:0/1000581116 shutdown_connections 2023-10-26T01:38:39.329 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:39.326+0000 7fa6c3fff640 1 -- 172.21.15.169:0/1000581116 >> 172.21.15.169:0/1000581116 conn(0x7fa6c40aedd0 msgr2=0x7fa6c40af1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:39.329 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:39.326+0000 7fa6c3fff640 1 -- 172.21.15.169:0/1000581116 shutdown_connections 2023-10-26T01:38:39.329 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:39.326+0000 7fa6c3fff640 1 -- 172.21.15.169:0/1000581116 wait complete. 2023-10-26T01:38:39.330 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:39.327+0000 7fa6c3fff640 1 Processor -- start 2023-10-26T01:38:39.330 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:39.328+0000 7fa6c3fff640 1 -- start start 2023-10-26T01:38:39.331 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:39.329+0000 7fa6c3fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa6b0003fa0 0x7fa6c4153c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:39.332 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:39.329+0000 7fa6c3fff640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa6c40afce0 0x7fa6c4099b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:39.332 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:39.329+0000 7fa6c3fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa6c40cc890 0x7fa6c40ccd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:39.332 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:39.329+0000 7fa6c3fff640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c4154190 con 0x7fa6b0003fa0 2023-10-26T01:38:39.332 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:39.329+0000 7fa6c3fff640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c40cd250 con 0x7fa6c40cc890 2023-10-26T01:38:39.332 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:39.329+0000 7fa6c3fff640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c406ec70 con 0x7fa6c40afce0 2023-10-26T01:38:39.333 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:39.329+0000 7fa6c2ffd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa6b0003fa0 0x7fa6c4153c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:39.333 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:39.329+0000 7fa6c2ffd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa6b0003fa0 0x7fa6c4153c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:55292/0 (socket says 172.21.15.169:55292) 2023-10-26T01:38:39.333 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:39.329+0000 7fa6c2ffd640 1 -- 172.21.15.169:0/484874691 learned_addr learned my addr 172.21.15.169:0/484874691 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:38:39.333 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:39.329+0000 7fa6c2ffd640 1 -- 172.21.15.169:0/484874691 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa6c40afce0 msgr2=0x7fa6c4099b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:39.334 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:39.329+0000 7fa6c2ffd640 1 --2- 172.21.15.169:0/484874691 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa6c40afce0 0x7fa6c4099b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:39.334 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:39.329+0000 7fa6c2ffd640 1 -- 172.21.15.169:0/484874691 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa6c40cc890 msgr2=0x7fa6c40ccd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:39.334 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:39.329+0000 7fa6c2ffd640 1 --2- 172.21.15.169:0/484874691 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa6c40cc890 0x7fa6c40ccd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:39.334 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:39.329+0000 7fa6c2ffd640 1 -- 172.21.15.169:0/484874691 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6c40b02a0 con 0x7fa6b0003fa0 2023-10-26T01:38:39.335 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:39.330+0000 7fa6c2ffd640 1 --2- 172.21.15.169:0/484874691 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa6b0003fa0 0x7fa6c4153c50 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7fa6b400e770 tx=0x7fa6b400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:39.335 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:39.330+0000 7fa6bb7fe640 1 -- 172.21.15.169:0/484874691 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b400cbe0 con 0x7fa6b0003fa0 2023-10-26T01:38:39.335 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:39.330+0000 7fa6bb7fe640 1 -- 172.21.15.169:0/484874691 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa6b400ce30 con 0x7fa6b0003fa0 2023-10-26T01:38:39.335 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:39.330+0000 7fa6bb7fe640 1 -- 172.21.15.169:0/484874691 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b4005280 con 0x7fa6b0003fa0 2023-10-26T01:38:39.335 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:39.330+0000 7fa6c3fff640 1 -- 172.21.15.169:0/484874691 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6c40b0580 con 0x7fa6b0003fa0 2023-10-26T01:38:39.335 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:39.330+0000 7fa6c3fff640 1 -- 172.21.15.169:0/484874691 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6c409a080 con 0x7fa6b0003fa0 2023-10-26T01:38:39.336 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:39.331+0000 7fa6c3fff640 1 -- 172.21.15.169:0/484874691 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6c406edb0 con 0x7fa6b0003fa0 2023-10-26T01:38:39.337 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:39.332+0000 7fa6bb7fe640 1 -- 172.21.15.169:0/484874691 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fa6b40053e0 con 0x7fa6b0003fa0 2023-10-26T01:38:39.337 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:39.333+0000 7fa6bb7fe640 1 --2- 172.21.15.169:0/484874691 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa6a406ae40 0x7fa6a406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:39.337 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:39.333+0000 7fa6bb7fe640 1 -- 172.21.15.169:0/484874691 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fa6b4089db0 con 0x7fa6b0003fa0 2023-10-26T01:38:39.337 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:39.334+0000 7fa6bbfff640 1 --2- 172.21.15.169:0/484874691 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa6a406ae40 0x7fa6a406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:39.337 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:39.335+0000 7fa6bbfff640 1 --2- 172.21.15.169:0/484874691 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa6a406ae40 0x7fa6a406d300 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac034000 tx=0x7fa6ac039040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:39.339 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:39.338+0000 7fa6bb7fe640 1 -- 172.21.15.169:0/484874691 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa6c406edb0 con 0x7fa6b0003fa0 2023-10-26T01:38:39.529 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:39.528+0000 7fa6c3fff640 1 -- 172.21.15.169:0/484874691 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fa6c406edb0 con 0x7fa6b0003fa0 2023-10-26T01:38:39.533 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:39.532+0000 7fa6bb7fe640 1 -- 172.21.15.169:0/484874691 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v137) v1 ==== 81+0+34450 (secure 0 0 0) 0x7fa6c406edb0 con 0x7fa6b0003fa0 2023-10-26T01:38:39.534 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:38:39.534 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2023-10-26T01:38:39.535 INFO:teuthology.orchestra.run.smithi169.stdout: "epoch": 137, 2023-10-26T01:38:39.535 INFO:teuthology.orchestra.run.smithi169.stdout: "fsid": "30203b28-ed8a-44e8-a9a6-a9bf25f980f1", 2023-10-26T01:38:39.535 INFO:teuthology.orchestra.run.smithi169.stdout: "created": "2023-10-26T01:30:22.897155+0000", 2023-10-26T01:38:39.535 INFO:teuthology.orchestra.run.smithi169.stdout: "modified": "2023-10-26T01:38:38.125497+0000", 2023-10-26T01:38:39.535 INFO:teuthology.orchestra.run.smithi169.stdout: "last_up_change": "2023-10-26T01:30:31.875281+0000", 2023-10-26T01:38:39.536 INFO:teuthology.orchestra.run.smithi169.stdout: "last_in_change": "2023-10-26T01:30:27.637941+0000", 2023-10-26T01:38:39.536 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-26T01:38:39.536 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_num": 5799936, 2023-10-26T01:38:39.536 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_set": [ 2023-10-26T01:38:39.536 INFO:teuthology.orchestra.run.smithi169.stdout: "pglog_hardlimit", 2023-10-26T01:38:39.536 INFO:teuthology.orchestra.run.smithi169.stdout: "purged_snapdirs", 2023-10-26T01:38:39.537 INFO:teuthology.orchestra.run.smithi169.stdout: "recovery_deletes", 2023-10-26T01:38:39.537 INFO:teuthology.orchestra.run.smithi169.stdout: "sortbitwise" 2023-10-26T01:38:39.537 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:38:39.537 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_version": 4, 2023-10-26T01:38:39.537 INFO:teuthology.orchestra.run.smithi169.stdout: "full_ratio": 0.69999998807907104, 2023-10-26T01:38:39.538 INFO:teuthology.orchestra.run.smithi169.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-26T01:38:39.538 INFO:teuthology.orchestra.run.smithi169.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-26T01:38:39.538 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_snapshot": "", 2023-10-26T01:38:39.538 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_max": 17, 2023-10-26T01:38:39.539 INFO:teuthology.orchestra.run.smithi169.stdout: "max_osd": 8, 2023-10-26T01:38:39.539 INFO:teuthology.orchestra.run.smithi169.stdout: "require_min_compat_client": "luminous", 2023-10-26T01:38:39.539 INFO:teuthology.orchestra.run.smithi169.stdout: "min_compat_client": "luminous", 2023-10-26T01:38:39.539 INFO:teuthology.orchestra.run.smithi169.stdout: "require_osd_release": "reef", 2023-10-26T01:38:39.539 INFO:teuthology.orchestra.run.smithi169.stdout: "allow_crimson": false, 2023-10-26T01:38:39.540 INFO:teuthology.orchestra.run.smithi169.stdout: "pools": [ 2023-10-26T01:38:39.540 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:39.540 INFO:teuthology.orchestra.run.smithi169.stdout: "pool": 1, 2023-10-26T01:38:39.540 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_name": ".mgr", 2023-10-26T01:38:39.540 INFO:teuthology.orchestra.run.smithi169.stdout: "create_time": "2023-10-26T01:30:32.200808+0000", 2023-10-26T01:38:39.540 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": 1, 2023-10-26T01:38:39.541 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_names": "hashpspool", 2023-10-26T01:38:39.541 INFO:teuthology.orchestra.run.smithi169.stdout: "type": 1, 2023-10-26T01:38:39.541 INFO:teuthology.orchestra.run.smithi169.stdout: "size": 2, 2023-10-26T01:38:39.541 INFO:teuthology.orchestra.run.smithi169.stdout: "min_size": 1, 2023-10-26T01:38:39.541 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_rule": 0, 2023-10-26T01:38:39.542 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_count": 0, 2023-10-26T01:38:39.542 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_target": 0, 2023-10-26T01:38:39.542 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T01:38:39.542 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T01:38:39.542 INFO:teuthology.orchestra.run.smithi169.stdout: "object_hash": 2, 2023-10-26T01:38:39.543 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_mode": "off", 2023-10-26T01:38:39.543 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num": 1, 2023-10-26T01:38:39.543 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num": 1, 2023-10-26T01:38:39.543 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num_target": 1, 2023-10-26T01:38:39.543 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_target": 1, 2023-10-26T01:38:39.544 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_pending": 1, 2023-10-26T01:38:39.544 INFO:teuthology.orchestra.run.smithi169.stdout: "last_pg_merge_meta": { 2023-10-26T01:38:39.544 INFO:teuthology.orchestra.run.smithi169.stdout: "source_pgid": "0.0", 2023-10-26T01:38:39.544 INFO:teuthology.orchestra.run.smithi169.stdout: "ready_epoch": 0, 2023-10-26T01:38:39.544 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_started": 0, 2023-10-26T01:38:39.544 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_clean": 0, 2023-10-26T01:38:39.545 INFO:teuthology.orchestra.run.smithi169.stdout: "source_version": "0'0", 2023-10-26T01:38:39.545 INFO:teuthology.orchestra.run.smithi169.stdout: "target_version": "0'0" 2023-10-26T01:38:39.545 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:39.545 INFO:teuthology.orchestra.run.smithi169.stdout: "last_change": "109", 2023-10-26T01:38:39.545 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend": "0", 2023-10-26T01:38:39.546 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T01:38:39.546 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T01:38:39.546 INFO:teuthology.orchestra.run.smithi169.stdout: "auid": 0, 2023-10-26T01:38:39.546 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_mode": "selfmanaged", 2023-10-26T01:38:39.546 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_seq": 0, 2023-10-26T01:38:39.546 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_epoch": 0, 2023-10-26T01:38:39.547 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_snaps": [], 2023-10-26T01:38:39.547 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps": "[]", 2023-10-26T01:38:39.547 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_bytes": 0, 2023-10-26T01:38:39.547 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_objects": 0, 2023-10-26T01:38:39.547 INFO:teuthology.orchestra.run.smithi169.stdout: "tiers": [], 2023-10-26T01:38:39.548 INFO:teuthology.orchestra.run.smithi169.stdout: "tier_of": -1, 2023-10-26T01:38:39.548 INFO:teuthology.orchestra.run.smithi169.stdout: "read_tier": -1, 2023-10-26T01:38:39.548 INFO:teuthology.orchestra.run.smithi169.stdout: "write_tier": -1, 2023-10-26T01:38:39.548 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_mode": "none", 2023-10-26T01:38:39.548 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_bytes": 0, 2023-10-26T01:38:39.548 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_objects": 0, 2023-10-26T01:38:39.549 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T01:38:39.549 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T01:38:39.549 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T01:38:39.549 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_flush_age": 0, 2023-10-26T01:38:39.549 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_evict_age": 0, 2023-10-26T01:38:39.550 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profile": "", 2023-10-26T01:38:39.550 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_params": { 2023-10-26T01:38:39.550 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "none" 2023-10-26T01:38:39.550 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:39.550 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_period": 0, 2023-10-26T01:38:39.550 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_count": 0, 2023-10-26T01:38:39.551 INFO:teuthology.orchestra.run.smithi169.stdout: "use_gmt_hitset": true, 2023-10-26T01:38:39.551 INFO:teuthology.orchestra.run.smithi169.stdout: "min_read_recency_for_promote": 0, 2023-10-26T01:38:39.551 INFO:teuthology.orchestra.run.smithi169.stdout: "min_write_recency_for_promote": 0, 2023-10-26T01:38:39.551 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T01:38:39.551 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_search_last_n": 0, 2023-10-26T01:38:39.552 INFO:teuthology.orchestra.run.smithi169.stdout: "grade_table": [], 2023-10-26T01:38:39.552 INFO:teuthology.orchestra.run.smithi169.stdout: "stripe_width": 0, 2023-10-26T01:38:39.552 INFO:teuthology.orchestra.run.smithi169.stdout: "expected_num_objects": 0, 2023-10-26T01:38:39.552 INFO:teuthology.orchestra.run.smithi169.stdout: "fast_read": false, 2023-10-26T01:38:39.552 INFO:teuthology.orchestra.run.smithi169.stdout: "options": { 2023-10-26T01:38:39.552 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_max": 32, 2023-10-26T01:38:39.553 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_min": 1 2023-10-26T01:38:39.553 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:39.553 INFO:teuthology.orchestra.run.smithi169.stdout: "application_metadata": { 2023-10-26T01:38:39.553 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": {} 2023-10-26T01:38:39.553 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:39.553 INFO:teuthology.orchestra.run.smithi169.stdout: "read_balance": { 2023-10-26T01:38:39.554 INFO:teuthology.orchestra.run.smithi169.stdout: "score_acting": 8, 2023-10-26T01:38:39.554 INFO:teuthology.orchestra.run.smithi169.stdout: "score_stable": 8, 2023-10-26T01:38:39.554 INFO:teuthology.orchestra.run.smithi169.stdout: "optimal_score": 0.25, 2023-10-26T01:38:39.554 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_acting": 2, 2023-10-26T01:38:39.554 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_stable": 2, 2023-10-26T01:38:39.555 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity_weighted": 1, 2023-10-26T01:38:39.555 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity": 1, 2023-10-26T01:38:39.555 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity_weighted": 1 2023-10-26T01:38:39.555 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:39.555 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:39.555 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:39.556 INFO:teuthology.orchestra.run.smithi169.stdout: "pool": 16, 2023-10-26T01:38:39.556 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_name": "cephfs_metadata", 2023-10-26T01:38:39.556 INFO:teuthology.orchestra.run.smithi169.stdout: "create_time": "2023-10-26T01:38:18.617877+0000", 2023-10-26T01:38:39.556 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": 1, 2023-10-26T01:38:39.556 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_names": "hashpspool", 2023-10-26T01:38:39.557 INFO:teuthology.orchestra.run.smithi169.stdout: "type": 1, 2023-10-26T01:38:39.557 INFO:teuthology.orchestra.run.smithi169.stdout: "size": 2, 2023-10-26T01:38:39.557 INFO:teuthology.orchestra.run.smithi169.stdout: "min_size": 1, 2023-10-26T01:38:39.557 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_rule": 0, 2023-10-26T01:38:39.557 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_count": 0, 2023-10-26T01:38:39.557 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_target": 0, 2023-10-26T01:38:39.558 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T01:38:39.558 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T01:38:39.558 INFO:teuthology.orchestra.run.smithi169.stdout: "object_hash": 2, 2023-10-26T01:38:39.558 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_mode": "off", 2023-10-26T01:38:39.558 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num": 32, 2023-10-26T01:38:39.559 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num": 32, 2023-10-26T01:38:39.559 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num_target": 32, 2023-10-26T01:38:39.559 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_target": 32, 2023-10-26T01:38:39.559 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_pending": 32, 2023-10-26T01:38:39.559 INFO:teuthology.orchestra.run.smithi169.stdout: "last_pg_merge_meta": { 2023-10-26T01:38:39.559 INFO:teuthology.orchestra.run.smithi169.stdout: "source_pgid": "0.0", 2023-10-26T01:38:39.560 INFO:teuthology.orchestra.run.smithi169.stdout: "ready_epoch": 0, 2023-10-26T01:38:39.560 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_started": 0, 2023-10-26T01:38:39.560 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_clean": 0, 2023-10-26T01:38:39.560 INFO:teuthology.orchestra.run.smithi169.stdout: "source_version": "0'0", 2023-10-26T01:38:39.560 INFO:teuthology.orchestra.run.smithi169.stdout: "target_version": "0'0" 2023-10-26T01:38:39.561 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:39.561 INFO:teuthology.orchestra.run.smithi169.stdout: "last_change": "133", 2023-10-26T01:38:39.561 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend": "0", 2023-10-26T01:38:39.561 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T01:38:39.561 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T01:38:39.561 INFO:teuthology.orchestra.run.smithi169.stdout: "auid": 0, 2023-10-26T01:38:39.562 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_mode": "selfmanaged", 2023-10-26T01:38:39.562 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_seq": 0, 2023-10-26T01:38:39.562 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_epoch": 0, 2023-10-26T01:38:39.562 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_snaps": [], 2023-10-26T01:38:39.562 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps": "[]", 2023-10-26T01:38:39.563 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_bytes": 0, 2023-10-26T01:38:39.563 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_objects": 0, 2023-10-26T01:38:39.563 INFO:teuthology.orchestra.run.smithi169.stdout: "tiers": [], 2023-10-26T01:38:39.563 INFO:teuthology.orchestra.run.smithi169.stdout: "tier_of": -1, 2023-10-26T01:38:39.563 INFO:teuthology.orchestra.run.smithi169.stdout: "read_tier": -1, 2023-10-26T01:38:39.563 INFO:teuthology.orchestra.run.smithi169.stdout: "write_tier": -1, 2023-10-26T01:38:39.564 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_mode": "none", 2023-10-26T01:38:39.564 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_bytes": 0, 2023-10-26T01:38:39.564 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_objects": 0, 2023-10-26T01:38:39.564 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T01:38:39.564 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T01:38:39.565 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T01:38:39.565 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_flush_age": 0, 2023-10-26T01:38:39.565 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_evict_age": 0, 2023-10-26T01:38:39.565 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profile": "", 2023-10-26T01:38:39.565 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_params": { 2023-10-26T01:38:39.565 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "none" 2023-10-26T01:38:39.566 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:39.566 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_period": 0, 2023-10-26T01:38:39.566 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_count": 0, 2023-10-26T01:38:39.566 INFO:teuthology.orchestra.run.smithi169.stdout: "use_gmt_hitset": true, 2023-10-26T01:38:39.566 INFO:teuthology.orchestra.run.smithi169.stdout: "min_read_recency_for_promote": 0, 2023-10-26T01:38:39.567 INFO:teuthology.orchestra.run.smithi169.stdout: "min_write_recency_for_promote": 0, 2023-10-26T01:38:39.567 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T01:38:39.567 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_search_last_n": 0, 2023-10-26T01:38:39.567 INFO:teuthology.orchestra.run.smithi169.stdout: "grade_table": [], 2023-10-26T01:38:39.567 INFO:teuthology.orchestra.run.smithi169.stdout: "stripe_width": 0, 2023-10-26T01:38:39.568 INFO:teuthology.orchestra.run.smithi169.stdout: "expected_num_objects": 0, 2023-10-26T01:38:39.568 INFO:teuthology.orchestra.run.smithi169.stdout: "fast_read": false, 2023-10-26T01:38:39.568 INFO:teuthology.orchestra.run.smithi169.stdout: "options": { 2023-10-26T01:38:39.568 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_bias": 4, 2023-10-26T01:38:39.568 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_min": 16, 2023-10-26T01:38:39.569 INFO:teuthology.orchestra.run.smithi169.stdout: "recovery_priority": 5 2023-10-26T01:38:39.569 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:39.569 INFO:teuthology.orchestra.run.smithi169.stdout: "application_metadata": { 2023-10-26T01:38:39.569 INFO:teuthology.orchestra.run.smithi169.stdout: "cephfs": { 2023-10-26T01:38:39.569 INFO:teuthology.orchestra.run.smithi169.stdout: "metadata": "cephfs" 2023-10-26T01:38:39.570 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:39.570 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:39.570 INFO:teuthology.orchestra.run.smithi169.stdout: "read_balance": { 2023-10-26T01:38:39.570 INFO:teuthology.orchestra.run.smithi169.stdout: "score_acting": 1.75, 2023-10-26T01:38:39.570 INFO:teuthology.orchestra.run.smithi169.stdout: "score_stable": 1.75, 2023-10-26T01:38:39.570 INFO:teuthology.orchestra.run.smithi169.stdout: "optimal_score": 1, 2023-10-26T01:38:39.571 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_acting": 1.75, 2023-10-26T01:38:39.571 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_stable": 1.75, 2023-10-26T01:38:39.571 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity_weighted": 1, 2023-10-26T01:38:39.571 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity": 1, 2023-10-26T01:38:39.571 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity_weighted": 1 2023-10-26T01:38:39.572 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:39.572 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:39.572 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:39.572 INFO:teuthology.orchestra.run.smithi169.stdout: "pool": 17, 2023-10-26T01:38:39.572 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_name": "cephfs_data", 2023-10-26T01:38:39.572 INFO:teuthology.orchestra.run.smithi169.stdout: "create_time": "2023-10-26T01:38:19.793041+0000", 2023-10-26T01:38:39.573 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": 1, 2023-10-26T01:38:39.573 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_names": "hashpspool", 2023-10-26T01:38:39.573 INFO:teuthology.orchestra.run.smithi169.stdout: "type": 1, 2023-10-26T01:38:39.573 INFO:teuthology.orchestra.run.smithi169.stdout: "size": 2, 2023-10-26T01:38:39.573 INFO:teuthology.orchestra.run.smithi169.stdout: "min_size": 1, 2023-10-26T01:38:39.574 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_rule": 0, 2023-10-26T01:38:39.574 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_count": 0, 2023-10-26T01:38:39.574 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_target": 0, 2023-10-26T01:38:39.574 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T01:38:39.574 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T01:38:39.575 INFO:teuthology.orchestra.run.smithi169.stdout: "object_hash": 2, 2023-10-26T01:38:39.575 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_mode": "off", 2023-10-26T01:38:39.575 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num": 64, 2023-10-26T01:38:39.575 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num": 64, 2023-10-26T01:38:39.575 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num_target": 64, 2023-10-26T01:38:39.576 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_target": 64, 2023-10-26T01:38:39.576 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_pending": 64, 2023-10-26T01:38:39.576 INFO:teuthology.orchestra.run.smithi169.stdout: "last_pg_merge_meta": { 2023-10-26T01:38:39.576 INFO:teuthology.orchestra.run.smithi169.stdout: "source_pgid": "0.0", 2023-10-26T01:38:39.576 INFO:teuthology.orchestra.run.smithi169.stdout: "ready_epoch": 0, 2023-10-26T01:38:39.576 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_started": 0, 2023-10-26T01:38:39.577 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_clean": 0, 2023-10-26T01:38:39.577 INFO:teuthology.orchestra.run.smithi169.stdout: "source_version": "0'0", 2023-10-26T01:38:39.577 INFO:teuthology.orchestra.run.smithi169.stdout: "target_version": "0'0" 2023-10-26T01:38:39.577 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:39.577 INFO:teuthology.orchestra.run.smithi169.stdout: "last_change": "137", 2023-10-26T01:38:39.577 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend": "0", 2023-10-26T01:38:39.578 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T01:38:39.578 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T01:38:39.578 INFO:teuthology.orchestra.run.smithi169.stdout: "auid": 0, 2023-10-26T01:38:39.578 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_mode": "selfmanaged", 2023-10-26T01:38:39.578 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_seq": 0, 2023-10-26T01:38:39.579 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_epoch": 0, 2023-10-26T01:38:39.579 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_snaps": [], 2023-10-26T01:38:39.579 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps": "[]", 2023-10-26T01:38:39.579 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_bytes": 33554432, 2023-10-26T01:38:39.579 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_objects": 0, 2023-10-26T01:38:39.580 INFO:teuthology.orchestra.run.smithi169.stdout: "tiers": [], 2023-10-26T01:38:39.580 INFO:teuthology.orchestra.run.smithi169.stdout: "tier_of": -1, 2023-10-26T01:38:39.580 INFO:teuthology.orchestra.run.smithi169.stdout: "read_tier": -1, 2023-10-26T01:38:39.580 INFO:teuthology.orchestra.run.smithi169.stdout: "write_tier": -1, 2023-10-26T01:38:39.580 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_mode": "none", 2023-10-26T01:38:39.580 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_bytes": 0, 2023-10-26T01:38:39.581 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_objects": 0, 2023-10-26T01:38:39.581 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T01:38:39.581 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T01:38:39.581 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T01:38:39.581 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_flush_age": 0, 2023-10-26T01:38:39.581 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_evict_age": 0, 2023-10-26T01:38:39.582 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profile": "", 2023-10-26T01:38:39.582 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_params": { 2023-10-26T01:38:39.582 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "none" 2023-10-26T01:38:39.582 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:39.582 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_period": 0, 2023-10-26T01:38:39.582 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_count": 0, 2023-10-26T01:38:39.583 INFO:teuthology.orchestra.run.smithi169.stdout: "use_gmt_hitset": true, 2023-10-26T01:38:39.583 INFO:teuthology.orchestra.run.smithi169.stdout: "min_read_recency_for_promote": 0, 2023-10-26T01:38:39.583 INFO:teuthology.orchestra.run.smithi169.stdout: "min_write_recency_for_promote": 0, 2023-10-26T01:38:39.583 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T01:38:39.583 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_search_last_n": 0, 2023-10-26T01:38:39.584 INFO:teuthology.orchestra.run.smithi169.stdout: "grade_table": [], 2023-10-26T01:38:39.584 INFO:teuthology.orchestra.run.smithi169.stdout: "stripe_width": 0, 2023-10-26T01:38:39.584 INFO:teuthology.orchestra.run.smithi169.stdout: "expected_num_objects": 0, 2023-10-26T01:38:39.584 INFO:teuthology.orchestra.run.smithi169.stdout: "fast_read": false, 2023-10-26T01:38:39.584 INFO:teuthology.orchestra.run.smithi169.stdout: "options": { 2023-10-26T01:38:39.585 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_min": 64, 2023-10-26T01:38:39.585 INFO:teuthology.orchestra.run.smithi169.stdout: "target_size_ratio": 0.89999997615814209 2023-10-26T01:38:39.585 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:39.585 INFO:teuthology.orchestra.run.smithi169.stdout: "application_metadata": { 2023-10-26T01:38:39.585 INFO:teuthology.orchestra.run.smithi169.stdout: "cephfs": { 2023-10-26T01:38:39.585 INFO:teuthology.orchestra.run.smithi169.stdout: "data": "cephfs" 2023-10-26T01:38:39.586 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:39.586 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:39.586 INFO:teuthology.orchestra.run.smithi169.stdout: "read_balance": { 2023-10-26T01:38:39.586 INFO:teuthology.orchestra.run.smithi169.stdout: "score_acting": 1.6299999952316284, 2023-10-26T01:38:39.586 INFO:teuthology.orchestra.run.smithi169.stdout: "score_stable": 1.6299999952316284, 2023-10-26T01:38:39.587 INFO:teuthology.orchestra.run.smithi169.stdout: "optimal_score": 1, 2023-10-26T01:38:39.587 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_acting": 1.6299999952316284, 2023-10-26T01:38:39.587 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_stable": 1.6299999952316284, 2023-10-26T01:38:39.587 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity_weighted": 1, 2023-10-26T01:38:39.587 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity": 1, 2023-10-26T01:38:39.588 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity_weighted": 1 2023-10-26T01:38:39.588 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:39.588 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:39.588 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:38:39.588 INFO:teuthology.orchestra.run.smithi169.stdout: "osds": [ 2023-10-26T01:38:39.588 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:39.589 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 0, 2023-10-26T01:38:39.589 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:38:39.589 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:38:39.589 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:38:39.589 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:38:39.590 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:38:39.590 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:38:39.590 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:38:39.590 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:38:39.590 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 132, 2023-10-26T01:38:39.590 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:38:39.591 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:38:39.591 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:38:39.591 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:39.591 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:39.591 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:39.592 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6816", 2023-10-26T01:38:39.592 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:38:39.592 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:39.592 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:39.592 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:39.592 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6817", 2023-10-26T01:38:39.593 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:38:39.593 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:39.593 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:39.593 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:39.593 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:38:39.594 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:39.594 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:39.594 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:39.594 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6818", 2023-10-26T01:38:39.594 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:38:39.595 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:39.595 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:39.595 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:39.595 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6819", 2023-10-26T01:38:39.595 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:38:39.596 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:39.596 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:39.596 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:39.596 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:38:39.596 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:39.596 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:39.597 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:39.597 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6822", 2023-10-26T01:38:39.597 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:38:39.597 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:39.597 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:39.598 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:39.598 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6823", 2023-10-26T01:38:39.598 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:38:39.598 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:39.598 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:39.598 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:39.599 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:38:39.599 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:39.599 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:39.599 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:39.599 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6820", 2023-10-26T01:38:39.600 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:38:39.600 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:39.600 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:39.600 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:39.600 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6821", 2023-10-26T01:38:39.601 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:38:39.601 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:39.601 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:39.601 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:39.601 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6817/4204255864", 2023-10-26T01:38:39.602 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6819/4204255864", 2023-10-26T01:38:39.602 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6823/4204255864", 2023-10-26T01:38:39.602 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6821/4204255864", 2023-10-26T01:38:39.602 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:38:39.602 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:38:39.603 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:38:39.603 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:39.603 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:39.603 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:39.603 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 1, 2023-10-26T01:38:39.604 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:38:39.604 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:38:39.604 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:38:39.604 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:38:39.604 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:38:39.605 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:38:39.605 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:38:39.605 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:38:39.605 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 132, 2023-10-26T01:38:39.605 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:38:39.605 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:38:39.606 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:38:39.606 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:39.606 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:39.606 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:39.606 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6808", 2023-10-26T01:38:39.607 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:38:39.607 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:39.607 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:39.607 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:39.607 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6809", 2023-10-26T01:38:39.607 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:38:39.608 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:39.608 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:39.608 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:39.608 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:38:39.608 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:39.609 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:39.609 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:39.609 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6810", 2023-10-26T01:38:39.609 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:38:39.609 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:39.609 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:39.610 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:39.610 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6811", 2023-10-26T01:38:39.610 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:38:39.610 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:39.610 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:39.611 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:39.611 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:38:39.611 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:39.611 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:39.611 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:39.611 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6814", 2023-10-26T01:38:39.612 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:38:39.612 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:39.612 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:39.612 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:39.612 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6815", 2023-10-26T01:38:39.613 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:38:39.613 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:39.613 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:39.613 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:39.613 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:38:39.613 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:39.614 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:39.614 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:39.614 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6812", 2023-10-26T01:38:39.614 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:38:39.614 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:39.615 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:39.615 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:39.615 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6813", 2023-10-26T01:38:39.615 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:38:39.615 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:39.615 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:39.616 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:39.616 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6809/238206046", 2023-10-26T01:38:39.616 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6811/238206046", 2023-10-26T01:38:39.616 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6815/238206046", 2023-10-26T01:38:39.616 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6813/238206046", 2023-10-26T01:38:39.616 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:38:39.617 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:38:39.617 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:38:39.617 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:39.617 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:39.617 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:39.618 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 2, 2023-10-26T01:38:39.618 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:38:39.618 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:38:39.618 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:38:39.618 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:38:39.619 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:38:39.619 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:38:39.619 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:38:39.619 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:38:39.619 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 135, 2023-10-26T01:38:39.620 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:38:39.620 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:38:39.620 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:38:39.620 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:39.620 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:39.621 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:39.621 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6800", 2023-10-26T01:38:39.621 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:38:39.621 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:39.621 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:39.621 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:39.622 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6801", 2023-10-26T01:38:39.622 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:38:39.622 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:39.622 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:39.622 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:39.623 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:38:39.623 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:39.623 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:39.623 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:39.623 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6802", 2023-10-26T01:38:39.624 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:38:39.624 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:39.624 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:39.624 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:39.624 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6803", 2023-10-26T01:38:39.624 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:38:39.625 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:39.625 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:39.625 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:39.625 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:38:39.625 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:39.626 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:39.626 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:39.626 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6806", 2023-10-26T01:38:39.626 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:38:39.626 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:39.627 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:39.627 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:39.627 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6807", 2023-10-26T01:38:39.627 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:38:39.627 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:39.627 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:39.628 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:39.628 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:38:39.628 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:39.628 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:39.628 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:39.629 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6804", 2023-10-26T01:38:39.629 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:38:39.629 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:39.629 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:39.629 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:39.629 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6805", 2023-10-26T01:38:39.630 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:38:39.630 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:39.630 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:39.630 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:39.630 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6801/3215791846", 2023-10-26T01:38:39.631 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6803/3215791846", 2023-10-26T01:38:39.631 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6807/3215791846", 2023-10-26T01:38:39.631 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6805/3215791846", 2023-10-26T01:38:39.631 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:38:39.631 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:38:39.632 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:38:39.632 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:39.632 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:39.632 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:39.632 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 3, 2023-10-26T01:38:39.632 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:38:39.633 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:38:39.633 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:38:39.633 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:38:39.633 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:38:39.633 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:38:39.634 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:38:39.634 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 14, 2023-10-26T01:38:39.634 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 132, 2023-10-26T01:38:39.634 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:38:39.634 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:38:39.635 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:38:39.635 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:39.635 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:39.635 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:39.635 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6824", 2023-10-26T01:38:39.635 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:38:39.636 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:39.636 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:39.636 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:39.636 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6825", 2023-10-26T01:38:39.636 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:38:39.637 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:39.637 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:39.637 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:39.637 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:38:39.637 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:39.637 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:39.638 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:39.638 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6826", 2023-10-26T01:38:39.638 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:38:39.638 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:39.638 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:39.638 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:39.639 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6827", 2023-10-26T01:38:39.639 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:38:39.639 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:39.639 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:39.639 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:39.640 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:38:39.640 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:39.640 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:39.640 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:39.640 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6830", 2023-10-26T01:38:39.640 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:38:39.641 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:39.641 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:39.641 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:39.641 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6831", 2023-10-26T01:38:39.641 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:38:39.642 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:39.642 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:39.642 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:39.642 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:38:39.642 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:39.643 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:39.643 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:39.643 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6828", 2023-10-26T01:38:39.643 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:38:39.643 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:39.643 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:39.644 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:39.644 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6829", 2023-10-26T01:38:39.644 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:38:39.644 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:39.644 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:39.645 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:39.645 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6825/2256701226", 2023-10-26T01:38:39.645 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6827/2256701226", 2023-10-26T01:38:39.645 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6831/2256701226", 2023-10-26T01:38:39.645 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6829/2256701226", 2023-10-26T01:38:39.646 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:38:39.646 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:38:39.646 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:38:39.646 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:39.646 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:39.646 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:39.647 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 4, 2023-10-26T01:38:39.647 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:38:39.647 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:38:39.647 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:38:39.647 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:38:39.648 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:38:39.648 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:38:39.648 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:38:39.648 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:38:39.648 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 132, 2023-10-26T01:38:39.648 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:38:39.649 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:38:39.649 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:38:39.649 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:39.649 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:39.649 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:39.650 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6816", 2023-10-26T01:38:39.650 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:38:39.650 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:39.650 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:39.650 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:39.651 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6817", 2023-10-26T01:38:39.651 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:38:39.651 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:39.651 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:39.651 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:39.651 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:38:39.652 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:39.652 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:39.652 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:39.652 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6818", 2023-10-26T01:38:39.652 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:38:39.653 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:39.653 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:39.653 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:39.653 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6819", 2023-10-26T01:38:39.653 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:38:39.654 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:39.654 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:39.654 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:39.654 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:38:39.654 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:39.655 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:39.655 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:39.655 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6825", 2023-10-26T01:38:39.656 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:38:39.656 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:39.656 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:39.656 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:39.656 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6828", 2023-10-26T01:38:39.657 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:38:39.657 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:39.657 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:39.657 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:39.657 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:38:39.657 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:39.658 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:39.658 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:39.658 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6820", 2023-10-26T01:38:39.658 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:38:39.658 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:39.659 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:39.659 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:39.659 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6823", 2023-10-26T01:38:39.659 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:38:39.659 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:39.660 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:39.660 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:39.660 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6817/1082279874", 2023-10-26T01:38:39.660 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6819/1082279874", 2023-10-26T01:38:39.660 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6828/1082279874", 2023-10-26T01:38:39.660 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6823/1082279874", 2023-10-26T01:38:39.661 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:38:39.661 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:38:39.661 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:38:39.661 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:39.661 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:39.662 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:39.662 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 5, 2023-10-26T01:38:39.662 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:38:39.662 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:38:39.662 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:38:39.663 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:38:39.663 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:38:39.663 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:38:39.663 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:38:39.663 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:38:39.663 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 132, 2023-10-26T01:38:39.664 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:38:39.664 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:38:39.664 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:38:39.664 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:39.664 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:39.665 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:39.665 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6808", 2023-10-26T01:38:39.665 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:38:39.665 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:39.665 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:39.665 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:39.666 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6809", 2023-10-26T01:38:39.666 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:38:39.666 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:39.666 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:39.666 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:39.667 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:38:39.667 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:39.667 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:39.667 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:39.667 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6810", 2023-10-26T01:38:39.668 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:38:39.668 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:39.668 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:39.668 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:39.668 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6811", 2023-10-26T01:38:39.668 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:38:39.669 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:39.669 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:39.669 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:39.669 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:38:39.669 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:39.670 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:39.670 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:39.670 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6814", 2023-10-26T01:38:39.670 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:38:39.670 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:39.670 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:39.671 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:39.671 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6815", 2023-10-26T01:38:39.671 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:38:39.671 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:39.671 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:39.672 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:39.672 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:38:39.672 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:39.672 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:39.672 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:39.672 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6812", 2023-10-26T01:38:39.673 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:38:39.673 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:39.673 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:39.673 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:39.673 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6813", 2023-10-26T01:38:39.674 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:38:39.674 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:39.674 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:39.674 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:39.674 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6809/3335654351", 2023-10-26T01:38:39.675 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6811/3335654351", 2023-10-26T01:38:39.675 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6815/3335654351", 2023-10-26T01:38:39.675 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6813/3335654351", 2023-10-26T01:38:39.675 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:38:39.675 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:38:39.675 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:38:39.676 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:39.676 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:39.676 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:39.676 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 6, 2023-10-26T01:38:39.676 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:38:39.677 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:38:39.677 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:38:39.677 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:38:39.677 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:38:39.677 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:38:39.678 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:38:39.678 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:38:39.678 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 132, 2023-10-26T01:38:39.678 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:38:39.678 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:38:39.679 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:38:39.679 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:39.679 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:39.679 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:39.679 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6800", 2023-10-26T01:38:39.680 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:38:39.680 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:39.680 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:39.680 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:39.680 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6801", 2023-10-26T01:38:39.680 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:38:39.681 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:39.681 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:39.681 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:39.681 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:38:39.681 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:39.682 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:39.682 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:39.682 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6802", 2023-10-26T01:38:39.682 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:38:39.682 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:39.683 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:39.683 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:39.683 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6803", 2023-10-26T01:38:39.683 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:38:39.683 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:39.683 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:39.684 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:39.684 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:38:39.684 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:39.684 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:39.684 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:39.685 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6806", 2023-10-26T01:38:39.685 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:38:39.685 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:39.685 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:39.685 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:39.685 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6807", 2023-10-26T01:38:39.686 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:38:39.686 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:39.686 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:39.686 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:39.686 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:38:39.687 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:39.687 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:39.687 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:39.687 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6804", 2023-10-26T01:38:39.687 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:38:39.688 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:39.688 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:39.688 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:39.688 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6805", 2023-10-26T01:38:39.688 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:38:39.688 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:39.689 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:39.689 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:39.689 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6801/2520460697", 2023-10-26T01:38:39.689 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6803/2520460697", 2023-10-26T01:38:39.689 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6807/2520460697", 2023-10-26T01:38:39.690 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6805/2520460697", 2023-10-26T01:38:39.690 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:38:39.690 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:38:39.690 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:38:39.690 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:39.691 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:39.691 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:39.691 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 7, 2023-10-26T01:38:39.691 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:38:39.691 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:38:39.691 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:38:39.692 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:38:39.692 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:38:39.692 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:38:39.692 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:38:39.692 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:38:39.693 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 132, 2023-10-26T01:38:39.693 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:38:39.693 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:38:39.693 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:38:39.693 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:39.693 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:39.694 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:39.694 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6821", 2023-10-26T01:38:39.694 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:38:39.694 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:39.694 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:39.695 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:39.695 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6822", 2023-10-26T01:38:39.695 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:38:39.695 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:39.695 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:39.696 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:39.696 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:38:39.696 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:39.696 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:39.696 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:39.696 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6824", 2023-10-26T01:38:39.697 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:38:39.697 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:39.697 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:39.697 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:39.697 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6826", 2023-10-26T01:38:39.698 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:38:39.698 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:39.698 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:39.698 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:39.698 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:38:39.699 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:39.699 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:39.699 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:39.699 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6830", 2023-10-26T01:38:39.699 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:38:39.699 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:39.700 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:39.700 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:39.700 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6831", 2023-10-26T01:38:39.700 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:38:39.700 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:39.701 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:39.701 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:39.701 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:38:39.701 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:39.701 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:39.702 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:39.702 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6827", 2023-10-26T01:38:39.702 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:38:39.702 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:39.702 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:39.702 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:39.703 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6829", 2023-10-26T01:38:39.703 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:38:39.703 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:39.703 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:39.703 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:39.704 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6822/1115930619", 2023-10-26T01:38:39.704 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6826/1115930619", 2023-10-26T01:38:39.704 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6831/1115930619", 2023-10-26T01:38:39.704 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6829/1115930619", 2023-10-26T01:38:39.704 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:38:39.704 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:38:39.705 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:38:39.705 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:39.705 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:39.705 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:38:39.705 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_xinfo": [ 2023-10-26T01:38:39.706 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:39.706 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 0, 2023-10-26T01:38:39.706 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:38:39.706 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:38:39.706 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:38:39.706 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:38:39.707 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:38:39.707 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.889675+0000", 2023-10-26T01:38:39.707 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:38:39.709 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:39.709 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:39.709 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 1, 2023-10-26T01:38:39.709 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:38:39.710 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:38:39.710 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:38:39.710 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:38:39.710 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:38:39.710 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.864157+0000", 2023-10-26T01:38:39.711 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:38:39.711 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:39.711 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:39.711 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 2, 2023-10-26T01:38:39.711 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:38:39.712 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:38:39.712 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:38:39.712 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:38:39.712 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:38:39.712 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.876919+0000", 2023-10-26T01:38:39.712 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:38:39.713 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:39.713 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:39.713 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 3, 2023-10-26T01:38:39.713 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:38:39.713 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:38:39.714 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:38:39.714 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:38:39.714 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:38:39.714 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.974541+0000", 2023-10-26T01:38:39.714 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:38:39.715 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:39.715 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:39.715 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 4, 2023-10-26T01:38:39.715 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:38:39.715 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:38:39.716 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:38:39.716 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:38:39.716 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:38:39.716 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.969696+0000", 2023-10-26T01:38:39.716 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:38:39.716 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:39.717 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:39.717 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 5, 2023-10-26T01:38:39.717 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:38:39.717 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:38:39.717 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:38:39.718 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:38:39.718 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:38:39.718 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.859703+0000", 2023-10-26T01:38:39.718 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:38:39.718 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:39.719 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:39.719 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 6, 2023-10-26T01:38:39.719 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:38:39.719 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:38:39.719 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:38:39.719 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:38:39.720 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:38:39.720 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.867917+0000", 2023-10-26T01:38:39.720 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:38:39.720 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:39.720 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:39.721 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 7, 2023-10-26T01:38:39.721 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:38:39.721 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:38:39.721 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:38:39.721 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:38:39.721 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:38:39.722 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.898605+0000", 2023-10-26T01:38:39.722 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:38:39.722 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:39.722 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:38:39.722 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_upmap": [], 2023-10-26T01:38:39.723 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_upmap_items": [ 2023-10-26T01:38:39.723 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:39.723 INFO:teuthology.orchestra.run.smithi169.stdout: "pgid": "17.e", 2023-10-26T01:38:39.723 INFO:teuthology.orchestra.run.smithi169.stdout: "mappings": [ 2023-10-26T01:38:39.723 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:39.724 INFO:teuthology.orchestra.run.smithi169.stdout: "from": 0, 2023-10-26T01:38:39.724 INFO:teuthology.orchestra.run.smithi169.stdout: "to": 5 2023-10-26T01:38:39.724 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:39.724 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:39.724 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:39.724 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:39.725 INFO:teuthology.orchestra.run.smithi169.stdout: "pgid": "17.21", 2023-10-26T01:38:39.725 INFO:teuthology.orchestra.run.smithi169.stdout: "mappings": [ 2023-10-26T01:38:39.725 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:39.725 INFO:teuthology.orchestra.run.smithi169.stdout: "from": 0, 2023-10-26T01:38:39.725 INFO:teuthology.orchestra.run.smithi169.stdout: "to": 2 2023-10-26T01:38:39.726 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:39.726 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:39.726 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:39.726 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:38:39.726 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_upmap_primaries": [], 2023-10-26T01:38:39.726 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_temp": [], 2023-10-26T01:38:39.727 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_temp": [], 2023-10-26T01:38:39.727 INFO:teuthology.orchestra.run.smithi169.stdout: "blocklist": {}, 2023-10-26T01:38:39.727 INFO:teuthology.orchestra.run.smithi169.stdout: "range_blocklist": {}, 2023-10-26T01:38:39.727 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profiles": { 2023-10-26T01:38:39.727 INFO:teuthology.orchestra.run.smithi169.stdout: "default": { 2023-10-26T01:38:39.728 INFO:teuthology.orchestra.run.smithi169.stdout: "crush-failure-domain": "osd", 2023-10-26T01:38:39.728 INFO:teuthology.orchestra.run.smithi169.stdout: "k": "2", 2023-10-26T01:38:39.728 INFO:teuthology.orchestra.run.smithi169.stdout: "m": "1", 2023-10-26T01:38:39.728 INFO:teuthology.orchestra.run.smithi169.stdout: "plugin": "jerasure", 2023-10-26T01:38:39.728 INFO:teuthology.orchestra.run.smithi169.stdout: "technique": "reed_sol_van" 2023-10-26T01:38:39.728 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:39.729 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:39.729 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps_queue": [], 2023-10-26T01:38:39.729 INFO:teuthology.orchestra.run.smithi169.stdout: "new_removed_snaps": [], 2023-10-26T01:38:39.729 INFO:teuthology.orchestra.run.smithi169.stdout: "new_purged_snaps": [], 2023-10-26T01:38:39.729 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_node_flags": {}, 2023-10-26T01:38:39.730 INFO:teuthology.orchestra.run.smithi169.stdout: "device_class_flags": {}, 2023-10-26T01:38:39.730 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_mode": { 2023-10-26T01:38:39.730 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_mode_enabled": false, 2023-10-26T01:38:39.730 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_bucket_count": 0, 2023-10-26T01:38:39.730 INFO:teuthology.orchestra.run.smithi169.stdout: "degraded_stretch_mode": 0, 2023-10-26T01:38:39.731 INFO:teuthology.orchestra.run.smithi169.stdout: "recovering_stretch_mode": 0, 2023-10-26T01:38:39.731 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_mode_bucket": 0 2023-10-26T01:38:39.731 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:39.731 INFO:teuthology.orchestra.run.smithi169.stdout:} 2023-10-26T01:38:39.733 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:39.534+0000 7fa6c3fff640 1 -- 172.21.15.169:0/484874691 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa6a406ae40 msgr2=0x7fa6a406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:39.733 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:39.534+0000 7fa6c3fff640 1 --2- 172.21.15.169:0/484874691 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa6a406ae40 0x7fa6a406d300 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac034000 tx=0x7fa6ac039040 comp rx=0 tx=0).stop 2023-10-26T01:38:39.733 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:39.534+0000 7fa6c3fff640 1 -- 172.21.15.169:0/484874691 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa6b0003fa0 msgr2=0x7fa6c4153c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:39.733 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:39.534+0000 7fa6c3fff640 1 --2- 172.21.15.169:0/484874691 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa6b0003fa0 0x7fa6c4153c50 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7fa6b400e770 tx=0x7fa6b400ec30 comp rx=0 tx=0).stop 2023-10-26T01:38:39.733 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:39.535+0000 7fa6c3fff640 1 -- 172.21.15.169:0/484874691 shutdown_connections 2023-10-26T01:38:39.734 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:39.535+0000 7fa6c3fff640 1 --2- 172.21.15.169:0/484874691 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa6a406ae40 0x7fa6a406d300 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:39.734 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:39.535+0000 7fa6c3fff640 1 --2- 172.21.15.169:0/484874691 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa6c40cc890 0x7fa6c40ccd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:39.734 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:39.535+0000 7fa6c3fff640 1 --2- 172.21.15.169:0/484874691 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa6c40afce0 0x7fa6c4099b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:39.734 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:39.535+0000 7fa6c3fff640 1 --2- 172.21.15.169:0/484874691 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa6b0003fa0 0x7fa6c4153c50 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:39.734 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:39.535+0000 7fa6c3fff640 1 -- 172.21.15.169:0/484874691 >> 172.21.15.169:0/484874691 conn(0x7fa6c40aedd0 msgr2=0x7fa6c406b120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:39.735 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:39.535+0000 7fa6c3fff640 1 -- 172.21.15.169:0/484874691 shutdown_connections 2023-10-26T01:38:39.735 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:39.535+0000 7fa6c3fff640 1 -- 172.21.15.169:0/484874691 wait complete. 2023-10-26T01:38:39.736 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2023-10-26T01:38:44.736 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-26T01:38:44.963 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:44.960+0000 7f08c784b640 1 -- 172.21.15.169:0/1367975655 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08c015b9d0 msgr2=0x7f08c00cbf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:44.963 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:44.960+0000 7f08c784b640 1 --2- 172.21.15.169:0/1367975655 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08c015b9d0 0x7f08c00cbf20 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f08c0072fc0 tx=0x7f08bc02bdb0 comp rx=0 tx=0).stop 2023-10-26T01:38:44.964 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:44.960+0000 7f08c784b640 1 -- 172.21.15.169:0/1367975655 shutdown_connections 2023-10-26T01:38:44.964 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:44.960+0000 7f08c784b640 1 --2- 172.21.15.169:0/1367975655 >> v2:172.21.15.170:3300/0 conn(0x7f08c00ced40 0x7f08c00d0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:44.964 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:44.960+0000 7f08c784b640 1 --2- 172.21.15.169:0/1367975655 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08c015b9d0 0x7f08c00cbf20 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:44.964 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:44.960+0000 7f08c784b640 1 -- 172.21.15.169:0/1367975655 >> 172.21.15.169:0/1367975655 conn(0x7f08c0069e00 msgr2=0x7f08c006a210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:44.964 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:44.960+0000 7f08c784b640 1 -- 172.21.15.169:0/1367975655 shutdown_connections 2023-10-26T01:38:44.965 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:44.960+0000 7f08c784b640 1 -- 172.21.15.169:0/1367975655 wait complete. 2023-10-26T01:38:44.966 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:44.962+0000 7f08c784b640 1 Processor -- start 2023-10-26T01:38:44.966 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:44.962+0000 7f08c784b640 1 -- start start 2023-10-26T01:38:44.966 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:44.963+0000 7f08c784b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08c00ced40 0x7f08c01f23a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:44.967 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:44.963+0000 7f08c784b640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f08c015b9d0 0x7f08c00c8840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:44.967 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:44.963+0000 7f08c784b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f08c015f610 0x7f08c0157f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:44.967 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:44.963+0000 7f08c784b640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08c015c160 con 0x7f08c00ced40 2023-10-26T01:38:44.967 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:44.963+0000 7f08c784b640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08c00c8d80 con 0x7f08c015f610 2023-10-26T01:38:44.968 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:44.963+0000 7f08c784b640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08c01f2980 con 0x7f08c015b9d0 2023-10-26T01:38:44.968 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:44.963+0000 7f08c55e7640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08c00ced40 0x7f08c01f23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:44.968 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:44.963+0000 7f08c55e7640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08c00ced40 0x7f08c01f23a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:53476/0 (socket says 172.21.15.169:53476) 2023-10-26T01:38:44.968 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:44.963+0000 7f08c55e7640 1 -- 172.21.15.169:0/2710123005 learned_addr learned my addr 172.21.15.169:0/2710123005 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:38:44.970 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:44.964+0000 7f08c4de6640 1 --2- 172.21.15.169:0/2710123005 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f08c015b9d0 0x7f08c00c8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:44.970 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:44.964+0000 7f08c55e7640 1 -- 172.21.15.169:0/2710123005 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f08c015b9d0 msgr2=0x7f08c00c8840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:44.970 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:44.964+0000 7f08c5de8640 1 --2- 172.21.15.169:0/2710123005 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f08c015f610 0x7f08c0157f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:44.971 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:44.964+0000 7f08c55e7640 1 --2- 172.21.15.169:0/2710123005 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f08c015b9d0 0x7f08c00c8840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:44.971 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:44.964+0000 7f08c55e7640 1 -- 172.21.15.169:0/2710123005 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f08c015f610 msgr2=0x7f08c0157f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:44.971 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:44.964+0000 7f08c55e7640 1 --2- 172.21.15.169:0/2710123005 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f08c015f610 0x7f08c0157f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:44.971 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:44.964+0000 7f08c55e7640 1 -- 172.21.15.169:0/2710123005 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08bc009560 con 0x7f08c00ced40 2023-10-26T01:38:44.971 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:44.964+0000 7f08c55e7640 1 --2- 172.21.15.169:0/2710123005 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08c00ced40 0x7f08c01f23a0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f08bc009850 tx=0x7f08bc02dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:44.972 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:44.964+0000 7f08b5ffb640 1 -- 172.21.15.169:0/2710123005 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08bc003800 con 0x7f08c00ced40 2023-10-26T01:38:44.972 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:44.964+0000 7f08b5ffb640 1 -- 172.21.15.169:0/2710123005 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f08bc003a50 con 0x7f08c00ced40 2023-10-26T01:38:44.972 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:44.964+0000 7f08b5ffb640 1 -- 172.21.15.169:0/2710123005 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08bc003e20 con 0x7f08c00ced40 2023-10-26T01:38:44.972 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:44.964+0000 7f08c784b640 1 -- 172.21.15.169:0/2710123005 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08c01f2c60 con 0x7f08c00ced40 2023-10-26T01:38:44.972 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:44.965+0000 7f08c784b640 1 -- 172.21.15.169:0/2710123005 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08c0158470 con 0x7f08c00ced40 2023-10-26T01:38:44.973 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:44.966+0000 7f08c784b640 1 -- 172.21.15.169:0/2710123005 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08c00cf270 con 0x7f08c00ced40 2023-10-26T01:38:44.973 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:44.967+0000 7f08b5ffb640 1 -- 172.21.15.169:0/2710123005 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f08bc03e450 con 0x7f08c00ced40 2023-10-26T01:38:44.973 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:44.967+0000 7f08b5ffb640 1 --2- 172.21.15.169:0/2710123005 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f08a406ae40 0x7f08a406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:44.973 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:44.968+0000 7f08b5ffb640 1 -- 172.21.15.169:0/2710123005 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f08bc0ac0f0 con 0x7f08c00ced40 2023-10-26T01:38:44.974 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:44.968+0000 7f08c4de6640 1 --2- 172.21.15.169:0/2710123005 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f08a406ae40 0x7f08a406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:44.974 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:44.969+0000 7f08c4de6640 1 --2- 172.21.15.169:0/2710123005 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f08a406ae40 0x7f08a406d300 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f08c00cb940 tx=0x7f08b0008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:44.975 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:44.972+0000 7f08b5ffb640 1 -- 172.21.15.169:0/2710123005 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f08bc0771f0 con 0x7f08c00ced40 2023-10-26T01:38:45.163 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:45.161+0000 7f08c784b640 1 -- 172.21.15.169:0/2710123005 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f08c004db10 con 0x7f08c00ced40 2023-10-26T01:38:45.167 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:45.165+0000 7f08b5ffb640 1 -- 172.21.15.169:0/2710123005 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v137) v1 ==== 81+0+34450 (secure 0 0 0) 0x7f08bc07a810 con 0x7f08c00ced40 2023-10-26T01:38:45.168 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:38:45.168 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2023-10-26T01:38:45.169 INFO:teuthology.orchestra.run.smithi169.stdout: "epoch": 137, 2023-10-26T01:38:45.169 INFO:teuthology.orchestra.run.smithi169.stdout: "fsid": "30203b28-ed8a-44e8-a9a6-a9bf25f980f1", 2023-10-26T01:38:45.169 INFO:teuthology.orchestra.run.smithi169.stdout: "created": "2023-10-26T01:30:22.897155+0000", 2023-10-26T01:38:45.169 INFO:teuthology.orchestra.run.smithi169.stdout: "modified": "2023-10-26T01:38:38.125497+0000", 2023-10-26T01:38:45.169 INFO:teuthology.orchestra.run.smithi169.stdout: "last_up_change": "2023-10-26T01:30:31.875281+0000", 2023-10-26T01:38:45.170 INFO:teuthology.orchestra.run.smithi169.stdout: "last_in_change": "2023-10-26T01:30:27.637941+0000", 2023-10-26T01:38:45.170 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-26T01:38:45.170 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_num": 5799936, 2023-10-26T01:38:45.170 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_set": [ 2023-10-26T01:38:45.171 INFO:teuthology.orchestra.run.smithi169.stdout: "pglog_hardlimit", 2023-10-26T01:38:45.171 INFO:teuthology.orchestra.run.smithi169.stdout: "purged_snapdirs", 2023-10-26T01:38:45.171 INFO:teuthology.orchestra.run.smithi169.stdout: "recovery_deletes", 2023-10-26T01:38:45.171 INFO:teuthology.orchestra.run.smithi169.stdout: "sortbitwise" 2023-10-26T01:38:45.171 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:38:45.171 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_version": 4, 2023-10-26T01:38:45.172 INFO:teuthology.orchestra.run.smithi169.stdout: "full_ratio": 0.69999998807907104, 2023-10-26T01:38:45.172 INFO:teuthology.orchestra.run.smithi169.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-26T01:38:45.172 INFO:teuthology.orchestra.run.smithi169.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-26T01:38:45.172 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_snapshot": "", 2023-10-26T01:38:45.172 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_max": 17, 2023-10-26T01:38:45.173 INFO:teuthology.orchestra.run.smithi169.stdout: "max_osd": 8, 2023-10-26T01:38:45.173 INFO:teuthology.orchestra.run.smithi169.stdout: "require_min_compat_client": "luminous", 2023-10-26T01:38:45.173 INFO:teuthology.orchestra.run.smithi169.stdout: "min_compat_client": "luminous", 2023-10-26T01:38:45.173 INFO:teuthology.orchestra.run.smithi169.stdout: "require_osd_release": "reef", 2023-10-26T01:38:45.173 INFO:teuthology.orchestra.run.smithi169.stdout: "allow_crimson": false, 2023-10-26T01:38:45.174 INFO:teuthology.orchestra.run.smithi169.stdout: "pools": [ 2023-10-26T01:38:45.174 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:45.174 INFO:teuthology.orchestra.run.smithi169.stdout: "pool": 1, 2023-10-26T01:38:45.174 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_name": ".mgr", 2023-10-26T01:38:45.174 INFO:teuthology.orchestra.run.smithi169.stdout: "create_time": "2023-10-26T01:30:32.200808+0000", 2023-10-26T01:38:45.175 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": 1, 2023-10-26T01:38:45.175 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_names": "hashpspool", 2023-10-26T01:38:45.176 INFO:teuthology.orchestra.run.smithi169.stdout: "type": 1, 2023-10-26T01:38:45.176 INFO:teuthology.orchestra.run.smithi169.stdout: "size": 2, 2023-10-26T01:38:45.176 INFO:teuthology.orchestra.run.smithi169.stdout: "min_size": 1, 2023-10-26T01:38:45.177 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_rule": 0, 2023-10-26T01:38:45.177 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_count": 0, 2023-10-26T01:38:45.177 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_target": 0, 2023-10-26T01:38:45.178 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T01:38:45.178 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T01:38:45.178 INFO:teuthology.orchestra.run.smithi169.stdout: "object_hash": 2, 2023-10-26T01:38:45.179 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_mode": "off", 2023-10-26T01:38:45.179 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num": 1, 2023-10-26T01:38:45.179 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num": 1, 2023-10-26T01:38:45.179 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num_target": 1, 2023-10-26T01:38:45.179 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_target": 1, 2023-10-26T01:38:45.179 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_pending": 1, 2023-10-26T01:38:45.180 INFO:teuthology.orchestra.run.smithi169.stdout: "last_pg_merge_meta": { 2023-10-26T01:38:45.180 INFO:teuthology.orchestra.run.smithi169.stdout: "source_pgid": "0.0", 2023-10-26T01:38:45.180 INFO:teuthology.orchestra.run.smithi169.stdout: "ready_epoch": 0, 2023-10-26T01:38:45.180 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_started": 0, 2023-10-26T01:38:45.180 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_clean": 0, 2023-10-26T01:38:45.181 INFO:teuthology.orchestra.run.smithi169.stdout: "source_version": "0'0", 2023-10-26T01:38:45.181 INFO:teuthology.orchestra.run.smithi169.stdout: "target_version": "0'0" 2023-10-26T01:38:45.181 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:45.181 INFO:teuthology.orchestra.run.smithi169.stdout: "last_change": "109", 2023-10-26T01:38:45.181 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend": "0", 2023-10-26T01:38:45.182 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T01:38:45.182 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T01:38:45.182 INFO:teuthology.orchestra.run.smithi169.stdout: "auid": 0, 2023-10-26T01:38:45.182 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_mode": "selfmanaged", 2023-10-26T01:38:45.182 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_seq": 0, 2023-10-26T01:38:45.183 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_epoch": 0, 2023-10-26T01:38:45.183 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_snaps": [], 2023-10-26T01:38:45.183 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps": "[]", 2023-10-26T01:38:45.183 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_bytes": 0, 2023-10-26T01:38:45.184 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_objects": 0, 2023-10-26T01:38:45.184 INFO:teuthology.orchestra.run.smithi169.stdout: "tiers": [], 2023-10-26T01:38:45.184 INFO:teuthology.orchestra.run.smithi169.stdout: "tier_of": -1, 2023-10-26T01:38:45.184 INFO:teuthology.orchestra.run.smithi169.stdout: "read_tier": -1, 2023-10-26T01:38:45.184 INFO:teuthology.orchestra.run.smithi169.stdout: "write_tier": -1, 2023-10-26T01:38:45.185 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_mode": "none", 2023-10-26T01:38:45.185 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_bytes": 0, 2023-10-26T01:38:45.185 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_objects": 0, 2023-10-26T01:38:45.186 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T01:38:45.186 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T01:38:45.186 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T01:38:45.187 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_flush_age": 0, 2023-10-26T01:38:45.187 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_evict_age": 0, 2023-10-26T01:38:45.187 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profile": "", 2023-10-26T01:38:45.187 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_params": { 2023-10-26T01:38:45.187 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "none" 2023-10-26T01:38:45.188 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:45.188 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_period": 0, 2023-10-26T01:38:45.188 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_count": 0, 2023-10-26T01:38:45.188 INFO:teuthology.orchestra.run.smithi169.stdout: "use_gmt_hitset": true, 2023-10-26T01:38:45.188 INFO:teuthology.orchestra.run.smithi169.stdout: "min_read_recency_for_promote": 0, 2023-10-26T01:38:45.189 INFO:teuthology.orchestra.run.smithi169.stdout: "min_write_recency_for_promote": 0, 2023-10-26T01:38:45.189 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T01:38:45.189 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_search_last_n": 0, 2023-10-26T01:38:45.189 INFO:teuthology.orchestra.run.smithi169.stdout: "grade_table": [], 2023-10-26T01:38:45.189 INFO:teuthology.orchestra.run.smithi169.stdout: "stripe_width": 0, 2023-10-26T01:38:45.190 INFO:teuthology.orchestra.run.smithi169.stdout: "expected_num_objects": 0, 2023-10-26T01:38:45.190 INFO:teuthology.orchestra.run.smithi169.stdout: "fast_read": false, 2023-10-26T01:38:45.190 INFO:teuthology.orchestra.run.smithi169.stdout: "options": { 2023-10-26T01:38:45.190 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_max": 32, 2023-10-26T01:38:45.190 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_min": 1 2023-10-26T01:38:45.191 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:45.191 INFO:teuthology.orchestra.run.smithi169.stdout: "application_metadata": { 2023-10-26T01:38:45.191 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": {} 2023-10-26T01:38:45.191 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:45.191 INFO:teuthology.orchestra.run.smithi169.stdout: "read_balance": { 2023-10-26T01:38:45.192 INFO:teuthology.orchestra.run.smithi169.stdout: "score_acting": 8, 2023-10-26T01:38:45.192 INFO:teuthology.orchestra.run.smithi169.stdout: "score_stable": 8, 2023-10-26T01:38:45.192 INFO:teuthology.orchestra.run.smithi169.stdout: "optimal_score": 0.25, 2023-10-26T01:38:45.192 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_acting": 2, 2023-10-26T01:38:45.192 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_stable": 2, 2023-10-26T01:38:45.193 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity_weighted": 1, 2023-10-26T01:38:45.193 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity": 1, 2023-10-26T01:38:45.193 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity_weighted": 1 2023-10-26T01:38:45.193 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:45.193 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:45.193 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:45.194 INFO:teuthology.orchestra.run.smithi169.stdout: "pool": 16, 2023-10-26T01:38:45.194 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_name": "cephfs_metadata", 2023-10-26T01:38:45.194 INFO:teuthology.orchestra.run.smithi169.stdout: "create_time": "2023-10-26T01:38:18.617877+0000", 2023-10-26T01:38:45.194 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": 1, 2023-10-26T01:38:45.194 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_names": "hashpspool", 2023-10-26T01:38:45.195 INFO:teuthology.orchestra.run.smithi169.stdout: "type": 1, 2023-10-26T01:38:45.195 INFO:teuthology.orchestra.run.smithi169.stdout: "size": 2, 2023-10-26T01:38:45.195 INFO:teuthology.orchestra.run.smithi169.stdout: "min_size": 1, 2023-10-26T01:38:45.195 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_rule": 0, 2023-10-26T01:38:45.195 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_count": 0, 2023-10-26T01:38:45.196 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_target": 0, 2023-10-26T01:38:45.196 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T01:38:45.196 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T01:38:45.196 INFO:teuthology.orchestra.run.smithi169.stdout: "object_hash": 2, 2023-10-26T01:38:45.196 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_mode": "off", 2023-10-26T01:38:45.197 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num": 32, 2023-10-26T01:38:45.197 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num": 32, 2023-10-26T01:38:45.197 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num_target": 32, 2023-10-26T01:38:45.197 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_target": 32, 2023-10-26T01:38:45.197 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_pending": 32, 2023-10-26T01:38:45.198 INFO:teuthology.orchestra.run.smithi169.stdout: "last_pg_merge_meta": { 2023-10-26T01:38:45.198 INFO:teuthology.orchestra.run.smithi169.stdout: "source_pgid": "0.0", 2023-10-26T01:38:45.198 INFO:teuthology.orchestra.run.smithi169.stdout: "ready_epoch": 0, 2023-10-26T01:38:45.198 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_started": 0, 2023-10-26T01:38:45.198 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_clean": 0, 2023-10-26T01:38:45.199 INFO:teuthology.orchestra.run.smithi169.stdout: "source_version": "0'0", 2023-10-26T01:38:45.199 INFO:teuthology.orchestra.run.smithi169.stdout: "target_version": "0'0" 2023-10-26T01:38:45.199 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:45.199 INFO:teuthology.orchestra.run.smithi169.stdout: "last_change": "133", 2023-10-26T01:38:45.199 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend": "0", 2023-10-26T01:38:45.200 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T01:38:45.200 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T01:38:45.200 INFO:teuthology.orchestra.run.smithi169.stdout: "auid": 0, 2023-10-26T01:38:45.200 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_mode": "selfmanaged", 2023-10-26T01:38:45.200 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_seq": 0, 2023-10-26T01:38:45.201 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_epoch": 0, 2023-10-26T01:38:45.201 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_snaps": [], 2023-10-26T01:38:45.201 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps": "[]", 2023-10-26T01:38:45.201 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_bytes": 0, 2023-10-26T01:38:45.201 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_objects": 0, 2023-10-26T01:38:45.202 INFO:teuthology.orchestra.run.smithi169.stdout: "tiers": [], 2023-10-26T01:38:45.202 INFO:teuthology.orchestra.run.smithi169.stdout: "tier_of": -1, 2023-10-26T01:38:45.202 INFO:teuthology.orchestra.run.smithi169.stdout: "read_tier": -1, 2023-10-26T01:38:45.202 INFO:teuthology.orchestra.run.smithi169.stdout: "write_tier": -1, 2023-10-26T01:38:45.202 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_mode": "none", 2023-10-26T01:38:45.202 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_bytes": 0, 2023-10-26T01:38:45.203 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_objects": 0, 2023-10-26T01:38:45.203 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T01:38:45.203 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T01:38:45.203 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T01:38:45.203 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_flush_age": 0, 2023-10-26T01:38:45.204 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_evict_age": 0, 2023-10-26T01:38:45.204 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profile": "", 2023-10-26T01:38:45.204 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_params": { 2023-10-26T01:38:45.204 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "none" 2023-10-26T01:38:45.204 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:45.205 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_period": 0, 2023-10-26T01:38:45.205 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_count": 0, 2023-10-26T01:38:45.205 INFO:teuthology.orchestra.run.smithi169.stdout: "use_gmt_hitset": true, 2023-10-26T01:38:45.205 INFO:teuthology.orchestra.run.smithi169.stdout: "min_read_recency_for_promote": 0, 2023-10-26T01:38:45.205 INFO:teuthology.orchestra.run.smithi169.stdout: "min_write_recency_for_promote": 0, 2023-10-26T01:38:45.206 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T01:38:45.206 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_search_last_n": 0, 2023-10-26T01:38:45.206 INFO:teuthology.orchestra.run.smithi169.stdout: "grade_table": [], 2023-10-26T01:38:45.206 INFO:teuthology.orchestra.run.smithi169.stdout: "stripe_width": 0, 2023-10-26T01:38:45.206 INFO:teuthology.orchestra.run.smithi169.stdout: "expected_num_objects": 0, 2023-10-26T01:38:45.207 INFO:teuthology.orchestra.run.smithi169.stdout: "fast_read": false, 2023-10-26T01:38:45.207 INFO:teuthology.orchestra.run.smithi169.stdout: "options": { 2023-10-26T01:38:45.207 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_bias": 4, 2023-10-26T01:38:45.207 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_min": 16, 2023-10-26T01:38:45.207 INFO:teuthology.orchestra.run.smithi169.stdout: "recovery_priority": 5 2023-10-26T01:38:45.208 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:45.208 INFO:teuthology.orchestra.run.smithi169.stdout: "application_metadata": { 2023-10-26T01:38:45.208 INFO:teuthology.orchestra.run.smithi169.stdout: "cephfs": { 2023-10-26T01:38:45.208 INFO:teuthology.orchestra.run.smithi169.stdout: "metadata": "cephfs" 2023-10-26T01:38:45.208 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:45.209 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:45.209 INFO:teuthology.orchestra.run.smithi169.stdout: "read_balance": { 2023-10-26T01:38:45.209 INFO:teuthology.orchestra.run.smithi169.stdout: "score_acting": 1.75, 2023-10-26T01:38:45.209 INFO:teuthology.orchestra.run.smithi169.stdout: "score_stable": 1.75, 2023-10-26T01:38:45.210 INFO:teuthology.orchestra.run.smithi169.stdout: "optimal_score": 1, 2023-10-26T01:38:45.210 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_acting": 1.75, 2023-10-26T01:38:45.210 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_stable": 1.75, 2023-10-26T01:38:45.210 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity_weighted": 1, 2023-10-26T01:38:45.210 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity": 1, 2023-10-26T01:38:45.211 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity_weighted": 1 2023-10-26T01:38:45.211 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:45.211 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:45.211 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:45.211 INFO:teuthology.orchestra.run.smithi169.stdout: "pool": 17, 2023-10-26T01:38:45.212 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_name": "cephfs_data", 2023-10-26T01:38:45.212 INFO:teuthology.orchestra.run.smithi169.stdout: "create_time": "2023-10-26T01:38:19.793041+0000", 2023-10-26T01:38:45.212 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": 1, 2023-10-26T01:38:45.212 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_names": "hashpspool", 2023-10-26T01:38:45.212 INFO:teuthology.orchestra.run.smithi169.stdout: "type": 1, 2023-10-26T01:38:45.212 INFO:teuthology.orchestra.run.smithi169.stdout: "size": 2, 2023-10-26T01:38:45.213 INFO:teuthology.orchestra.run.smithi169.stdout: "min_size": 1, 2023-10-26T01:38:45.213 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_rule": 0, 2023-10-26T01:38:45.213 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_count": 0, 2023-10-26T01:38:45.213 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_target": 0, 2023-10-26T01:38:45.213 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T01:38:45.214 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T01:38:45.214 INFO:teuthology.orchestra.run.smithi169.stdout: "object_hash": 2, 2023-10-26T01:38:45.214 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_mode": "off", 2023-10-26T01:38:45.214 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num": 64, 2023-10-26T01:38:45.214 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num": 64, 2023-10-26T01:38:45.215 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num_target": 64, 2023-10-26T01:38:45.215 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_target": 64, 2023-10-26T01:38:45.215 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_pending": 64, 2023-10-26T01:38:45.215 INFO:teuthology.orchestra.run.smithi169.stdout: "last_pg_merge_meta": { 2023-10-26T01:38:45.215 INFO:teuthology.orchestra.run.smithi169.stdout: "source_pgid": "0.0", 2023-10-26T01:38:45.216 INFO:teuthology.orchestra.run.smithi169.stdout: "ready_epoch": 0, 2023-10-26T01:38:45.216 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_started": 0, 2023-10-26T01:38:45.216 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_clean": 0, 2023-10-26T01:38:45.216 INFO:teuthology.orchestra.run.smithi169.stdout: "source_version": "0'0", 2023-10-26T01:38:45.216 INFO:teuthology.orchestra.run.smithi169.stdout: "target_version": "0'0" 2023-10-26T01:38:45.217 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:45.217 INFO:teuthology.orchestra.run.smithi169.stdout: "last_change": "137", 2023-10-26T01:38:45.217 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend": "0", 2023-10-26T01:38:45.217 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T01:38:45.217 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T01:38:45.218 INFO:teuthology.orchestra.run.smithi169.stdout: "auid": 0, 2023-10-26T01:38:45.218 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_mode": "selfmanaged", 2023-10-26T01:38:45.218 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_seq": 0, 2023-10-26T01:38:45.218 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_epoch": 0, 2023-10-26T01:38:45.218 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_snaps": [], 2023-10-26T01:38:45.219 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps": "[]", 2023-10-26T01:38:45.219 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_bytes": 33554432, 2023-10-26T01:38:45.219 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_objects": 0, 2023-10-26T01:38:45.219 INFO:teuthology.orchestra.run.smithi169.stdout: "tiers": [], 2023-10-26T01:38:45.219 INFO:teuthology.orchestra.run.smithi169.stdout: "tier_of": -1, 2023-10-26T01:38:45.220 INFO:teuthology.orchestra.run.smithi169.stdout: "read_tier": -1, 2023-10-26T01:38:45.220 INFO:teuthology.orchestra.run.smithi169.stdout: "write_tier": -1, 2023-10-26T01:38:45.220 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_mode": "none", 2023-10-26T01:38:45.220 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_bytes": 0, 2023-10-26T01:38:45.220 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_objects": 0, 2023-10-26T01:38:45.221 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T01:38:45.221 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T01:38:45.221 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T01:38:45.221 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_flush_age": 0, 2023-10-26T01:38:45.221 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_evict_age": 0, 2023-10-26T01:38:45.222 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profile": "", 2023-10-26T01:38:45.222 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_params": { 2023-10-26T01:38:45.222 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "none" 2023-10-26T01:38:45.222 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:45.222 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_period": 0, 2023-10-26T01:38:45.223 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_count": 0, 2023-10-26T01:38:45.223 INFO:teuthology.orchestra.run.smithi169.stdout: "use_gmt_hitset": true, 2023-10-26T01:38:45.223 INFO:teuthology.orchestra.run.smithi169.stdout: "min_read_recency_for_promote": 0, 2023-10-26T01:38:45.223 INFO:teuthology.orchestra.run.smithi169.stdout: "min_write_recency_for_promote": 0, 2023-10-26T01:38:45.223 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T01:38:45.224 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_search_last_n": 0, 2023-10-26T01:38:45.224 INFO:teuthology.orchestra.run.smithi169.stdout: "grade_table": [], 2023-10-26T01:38:45.224 INFO:teuthology.orchestra.run.smithi169.stdout: "stripe_width": 0, 2023-10-26T01:38:45.224 INFO:teuthology.orchestra.run.smithi169.stdout: "expected_num_objects": 0, 2023-10-26T01:38:45.224 INFO:teuthology.orchestra.run.smithi169.stdout: "fast_read": false, 2023-10-26T01:38:45.225 INFO:teuthology.orchestra.run.smithi169.stdout: "options": { 2023-10-26T01:38:45.225 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_min": 64, 2023-10-26T01:38:45.225 INFO:teuthology.orchestra.run.smithi169.stdout: "target_size_ratio": 0.89999997615814209 2023-10-26T01:38:45.225 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:45.225 INFO:teuthology.orchestra.run.smithi169.stdout: "application_metadata": { 2023-10-26T01:38:45.225 INFO:teuthology.orchestra.run.smithi169.stdout: "cephfs": { 2023-10-26T01:38:45.226 INFO:teuthology.orchestra.run.smithi169.stdout: "data": "cephfs" 2023-10-26T01:38:45.226 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:45.226 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:45.226 INFO:teuthology.orchestra.run.smithi169.stdout: "read_balance": { 2023-10-26T01:38:45.226 INFO:teuthology.orchestra.run.smithi169.stdout: "score_acting": 1.6299999952316284, 2023-10-26T01:38:45.227 INFO:teuthology.orchestra.run.smithi169.stdout: "score_stable": 1.6299999952316284, 2023-10-26T01:38:45.227 INFO:teuthology.orchestra.run.smithi169.stdout: "optimal_score": 1, 2023-10-26T01:38:45.227 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_acting": 1.6299999952316284, 2023-10-26T01:38:45.227 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_stable": 1.6299999952316284, 2023-10-26T01:38:45.227 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity_weighted": 1, 2023-10-26T01:38:45.228 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity": 1, 2023-10-26T01:38:45.228 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity_weighted": 1 2023-10-26T01:38:45.228 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:45.228 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:45.228 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:38:45.229 INFO:teuthology.orchestra.run.smithi169.stdout: "osds": [ 2023-10-26T01:38:45.229 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:45.229 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 0, 2023-10-26T01:38:45.229 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:38:45.229 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:38:45.229 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:38:45.230 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:38:45.230 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:38:45.230 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:38:45.230 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:38:45.230 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:38:45.231 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 132, 2023-10-26T01:38:45.231 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:38:45.231 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:38:45.231 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:38:45.231 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:45.232 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:45.232 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:45.232 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6816", 2023-10-26T01:38:45.232 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:38:45.232 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:45.233 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:45.233 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:45.233 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6817", 2023-10-26T01:38:45.233 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:38:45.233 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:45.233 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:45.234 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:45.234 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:38:45.234 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:45.234 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:45.234 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:45.235 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6818", 2023-10-26T01:38:45.235 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:38:45.235 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:45.235 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:45.235 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:45.236 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6819", 2023-10-26T01:38:45.236 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:38:45.236 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:45.236 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:45.236 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:45.237 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:38:45.237 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:45.237 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:45.237 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:45.237 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6822", 2023-10-26T01:38:45.238 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:38:45.238 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:45.238 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:45.238 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:45.238 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6823", 2023-10-26T01:38:45.239 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:38:45.239 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:45.239 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:45.239 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:45.239 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:38:45.240 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:45.240 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:45.240 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:45.240 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6820", 2023-10-26T01:38:45.240 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:38:45.240 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:45.241 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:45.241 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:45.241 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6821", 2023-10-26T01:38:45.241 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:38:45.241 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:45.242 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:45.242 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:45.242 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6817/4204255864", 2023-10-26T01:38:45.242 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6819/4204255864", 2023-10-26T01:38:45.242 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6823/4204255864", 2023-10-26T01:38:45.243 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6821/4204255864", 2023-10-26T01:38:45.243 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:38:45.243 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:38:45.243 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:38:45.243 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:45.244 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:45.244 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:45.244 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 1, 2023-10-26T01:38:45.244 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:38:45.244 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:38:45.244 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:38:45.245 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:38:45.245 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:38:45.245 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:38:45.245 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:38:45.245 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:38:45.246 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 132, 2023-10-26T01:38:45.246 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:38:45.246 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:38:45.246 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:38:45.246 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:45.246 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:45.247 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:45.247 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6808", 2023-10-26T01:38:45.247 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:38:45.247 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:45.247 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:45.248 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:45.248 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6809", 2023-10-26T01:38:45.248 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:38:45.248 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:45.248 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:45.249 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:45.249 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:38:45.249 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:45.249 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:45.249 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:45.249 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6810", 2023-10-26T01:38:45.250 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:38:45.250 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:45.250 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:45.250 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:45.251 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6811", 2023-10-26T01:38:45.251 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:38:45.251 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:45.251 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:45.251 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:45.251 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:38:45.252 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:45.252 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:45.252 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:45.252 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6814", 2023-10-26T01:38:45.252 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:38:45.253 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:45.253 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:45.253 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:45.253 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6815", 2023-10-26T01:38:45.253 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:38:45.254 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:45.254 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:45.254 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:45.254 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:38:45.254 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:45.255 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:45.255 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:45.255 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6812", 2023-10-26T01:38:45.255 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:38:45.255 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:45.256 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:45.256 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:45.256 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6813", 2023-10-26T01:38:45.256 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:38:45.256 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:45.257 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:45.257 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:45.257 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6809/238206046", 2023-10-26T01:38:45.257 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6811/238206046", 2023-10-26T01:38:45.257 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6815/238206046", 2023-10-26T01:38:45.258 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6813/238206046", 2023-10-26T01:38:45.258 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:38:45.258 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:38:45.258 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:38:45.258 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:45.259 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:45.259 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:45.259 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 2, 2023-10-26T01:38:45.259 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:38:45.259 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:38:45.259 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:38:45.260 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:38:45.260 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:38:45.260 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:38:45.260 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:38:45.260 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:38:45.261 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 135, 2023-10-26T01:38:45.261 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:38:45.261 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:38:45.261 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:38:45.261 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:45.262 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:45.262 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:45.262 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6800", 2023-10-26T01:38:45.262 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:38:45.262 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:45.263 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:45.263 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:45.263 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6801", 2023-10-26T01:38:45.263 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:38:45.263 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:45.264 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:45.264 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:45.264 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:38:45.264 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:45.264 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:45.265 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:45.265 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6802", 2023-10-26T01:38:45.265 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:38:45.265 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:45.265 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:45.266 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:45.266 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6803", 2023-10-26T01:38:45.266 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:38:45.266 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:45.266 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:45.267 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:45.267 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:38:45.267 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:45.267 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:45.267 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:45.268 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6806", 2023-10-26T01:38:45.268 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:38:45.268 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:45.268 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:45.268 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:45.269 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6807", 2023-10-26T01:38:45.269 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:38:45.269 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:45.269 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:45.269 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:45.269 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:38:45.270 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:45.270 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:45.270 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:45.270 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6804", 2023-10-26T01:38:45.270 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:38:45.271 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:45.271 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:45.271 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:45.271 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6805", 2023-10-26T01:38:45.271 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:38:45.272 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:45.272 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:45.272 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:45.272 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6801/3215791846", 2023-10-26T01:38:45.272 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6803/3215791846", 2023-10-26T01:38:45.273 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6807/3215791846", 2023-10-26T01:38:45.273 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6805/3215791846", 2023-10-26T01:38:45.273 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:38:45.273 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:38:45.273 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:38:45.273 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:45.274 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:45.274 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:45.274 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 3, 2023-10-26T01:38:45.274 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:38:45.274 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:38:45.275 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:38:45.275 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:38:45.275 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:38:45.275 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:38:45.275 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:38:45.275 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 14, 2023-10-26T01:38:45.276 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 132, 2023-10-26T01:38:45.276 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:38:45.276 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:38:45.276 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:38:45.276 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:45.277 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:45.277 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:45.277 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6824", 2023-10-26T01:38:45.277 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:38:45.277 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:45.278 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:45.278 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:45.278 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6825", 2023-10-26T01:38:45.278 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:38:45.278 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:45.279 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:45.279 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:45.279 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:38:45.279 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:45.279 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:45.280 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:45.280 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6826", 2023-10-26T01:38:45.280 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:38:45.280 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:45.280 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:45.281 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:45.281 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6827", 2023-10-26T01:38:45.281 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:38:45.281 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:45.282 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:45.282 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:45.282 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:38:45.282 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:45.282 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:45.283 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:45.283 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6830", 2023-10-26T01:38:45.283 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:38:45.283 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:45.283 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:45.284 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:45.284 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6831", 2023-10-26T01:38:45.284 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:38:45.284 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:45.284 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:45.284 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:45.285 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:38:45.285 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:45.285 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:45.285 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:45.285 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6828", 2023-10-26T01:38:45.286 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:38:45.286 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:45.286 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:45.286 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:45.287 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6829", 2023-10-26T01:38:45.287 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:38:45.287 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:45.287 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:45.287 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:45.287 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6825/2256701226", 2023-10-26T01:38:45.288 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6827/2256701226", 2023-10-26T01:38:45.288 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6831/2256701226", 2023-10-26T01:38:45.288 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6829/2256701226", 2023-10-26T01:38:45.288 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:38:45.289 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:38:45.289 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:38:45.289 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:45.289 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:45.289 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:45.289 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 4, 2023-10-26T01:38:45.290 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:38:45.290 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:38:45.290 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:38:45.290 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:38:45.290 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:38:45.291 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:38:45.291 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:38:45.291 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:38:45.291 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 132, 2023-10-26T01:38:45.291 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:38:45.292 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:38:45.292 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:38:45.292 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:45.292 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:45.292 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:45.292 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6816", 2023-10-26T01:38:45.293 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:38:45.293 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:45.293 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:45.293 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:45.293 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6817", 2023-10-26T01:38:45.294 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:38:45.294 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:45.294 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:45.294 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:45.294 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:38:45.295 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:45.295 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:45.295 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:45.295 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6818", 2023-10-26T01:38:45.295 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:38:45.296 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:45.296 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:45.296 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:45.296 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6819", 2023-10-26T01:38:45.296 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:38:45.297 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:45.297 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:45.297 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:45.297 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:38:45.297 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:45.297 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:45.298 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:45.298 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6825", 2023-10-26T01:38:45.298 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:38:45.298 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:45.298 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:45.299 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:45.299 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6828", 2023-10-26T01:38:45.299 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:38:45.299 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:45.299 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:45.300 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:45.300 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:38:45.300 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:45.300 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:45.300 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:45.301 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6820", 2023-10-26T01:38:45.301 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:38:45.301 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:45.301 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:45.301 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:45.301 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6823", 2023-10-26T01:38:45.302 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:38:45.302 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:45.302 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:45.302 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:45.302 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6817/1082279874", 2023-10-26T01:38:45.303 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6819/1082279874", 2023-10-26T01:38:45.303 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6828/1082279874", 2023-10-26T01:38:45.303 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6823/1082279874", 2023-10-26T01:38:45.303 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:38:45.303 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:38:45.304 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:38:45.304 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:45.304 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:45.304 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:45.304 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 5, 2023-10-26T01:38:45.305 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:38:45.305 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:38:45.305 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:38:45.305 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:38:45.305 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:38:45.306 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:38:45.306 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:38:45.306 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:38:45.306 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 132, 2023-10-26T01:38:45.306 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:38:45.307 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:38:45.307 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:38:45.307 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:45.307 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:45.307 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:45.307 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6808", 2023-10-26T01:38:45.308 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:38:45.308 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:45.308 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:45.308 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:45.308 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6809", 2023-10-26T01:38:45.309 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:38:45.309 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:45.309 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:45.309 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:45.309 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:38:45.310 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:45.310 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:45.310 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:45.310 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6810", 2023-10-26T01:38:45.310 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:38:45.310 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:45.311 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:45.311 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:45.311 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6811", 2023-10-26T01:38:45.311 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:38:45.311 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:45.312 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:45.312 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:45.312 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:38:45.312 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:45.312 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:45.313 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:45.313 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6814", 2023-10-26T01:38:45.313 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:38:45.313 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:45.313 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:45.314 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:45.314 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6815", 2023-10-26T01:38:45.314 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:38:45.314 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:45.314 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:45.315 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:45.315 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:38:45.315 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:45.315 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:45.315 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:45.315 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6812", 2023-10-26T01:38:45.316 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:38:45.316 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:45.316 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:45.316 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:45.316 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6813", 2023-10-26T01:38:45.317 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:38:45.317 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:45.317 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:45.317 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:45.317 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6809/3335654351", 2023-10-26T01:38:45.318 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6811/3335654351", 2023-10-26T01:38:45.318 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6815/3335654351", 2023-10-26T01:38:45.318 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6813/3335654351", 2023-10-26T01:38:45.318 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:38:45.318 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:38:45.319 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:38:45.319 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:45.319 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:45.319 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:45.319 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 6, 2023-10-26T01:38:45.320 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:38:45.320 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:38:45.320 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:38:45.320 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:38:45.320 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:38:45.321 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:38:45.321 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:38:45.321 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:38:45.321 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 132, 2023-10-26T01:38:45.321 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:38:45.322 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:38:45.322 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:38:45.322 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:45.322 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:45.322 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:45.323 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6800", 2023-10-26T01:38:45.323 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:38:45.323 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:45.323 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:45.323 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:45.324 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6801", 2023-10-26T01:38:45.324 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:38:45.324 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:45.324 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:45.325 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:45.325 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:38:45.325 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:45.325 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:45.325 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:45.326 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6802", 2023-10-26T01:38:45.326 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:38:45.326 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:45.326 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:45.326 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:45.327 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6803", 2023-10-26T01:38:45.327 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:38:45.327 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:45.327 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:45.327 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:45.328 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:38:45.328 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:45.328 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:45.328 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:45.328 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6806", 2023-10-26T01:38:45.328 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:38:45.329 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:45.329 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:45.329 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:45.329 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6807", 2023-10-26T01:38:45.329 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:38:45.330 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:45.330 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:45.330 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:45.330 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:38:45.330 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:45.331 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:45.331 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:45.331 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6804", 2023-10-26T01:38:45.331 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:38:45.331 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:45.331 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:45.332 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:45.332 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6805", 2023-10-26T01:38:45.332 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:38:45.332 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:45.332 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:45.333 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:45.333 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6801/2520460697", 2023-10-26T01:38:45.333 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6803/2520460697", 2023-10-26T01:38:45.333 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6807/2520460697", 2023-10-26T01:38:45.333 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6805/2520460697", 2023-10-26T01:38:45.334 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:38:45.334 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:38:45.334 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:38:45.334 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:45.334 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:45.335 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:45.335 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 7, 2023-10-26T01:38:45.335 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:38:45.335 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:38:45.335 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:38:45.336 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:38:45.336 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:38:45.336 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:38:45.336 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:38:45.336 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:38:45.336 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 132, 2023-10-26T01:38:45.337 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:38:45.337 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:38:45.337 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:38:45.337 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:45.337 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:45.338 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:45.338 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6821", 2023-10-26T01:38:45.338 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:38:45.338 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:45.338 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:45.339 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:45.339 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6822", 2023-10-26T01:38:45.339 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:38:45.339 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:45.339 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:45.340 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:45.340 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:38:45.340 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:45.340 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:45.340 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:45.341 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6824", 2023-10-26T01:38:45.341 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:38:45.341 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:45.341 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:45.341 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:45.341 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6826", 2023-10-26T01:38:45.342 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:38:45.342 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:45.342 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:45.342 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:45.342 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:38:45.343 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:45.343 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:45.343 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:45.343 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6830", 2023-10-26T01:38:45.343 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:38:45.344 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:45.344 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:45.344 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:45.344 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6831", 2023-10-26T01:38:45.344 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:38:45.344 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:45.345 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:45.345 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:45.345 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:38:45.345 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:45.345 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:45.345 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:45.346 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6827", 2023-10-26T01:38:45.346 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:38:45.346 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:45.346 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:45.346 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:45.347 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6829", 2023-10-26T01:38:45.347 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:38:45.347 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:45.347 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:45.347 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:45.348 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6822/1115930619", 2023-10-26T01:38:45.348 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6826/1115930619", 2023-10-26T01:38:45.348 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6831/1115930619", 2023-10-26T01:38:45.348 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6829/1115930619", 2023-10-26T01:38:45.348 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:38:45.349 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:38:45.349 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:38:45.349 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:45.349 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:45.349 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:38:45.350 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_xinfo": [ 2023-10-26T01:38:45.350 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:45.350 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 0, 2023-10-26T01:38:45.350 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:38:45.350 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:38:45.351 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:38:45.351 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:38:45.351 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:38:45.351 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.889675+0000", 2023-10-26T01:38:45.351 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:38:45.352 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:45.352 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:45.352 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 1, 2023-10-26T01:38:45.352 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:38:45.352 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:38:45.352 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:38:45.353 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:38:45.353 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:38:45.353 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.864157+0000", 2023-10-26T01:38:45.353 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:38:45.353 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:45.354 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:45.354 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 2, 2023-10-26T01:38:45.354 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:38:45.354 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:38:45.354 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:38:45.355 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:38:45.355 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:38:45.355 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.876919+0000", 2023-10-26T01:38:45.355 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:38:45.355 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:45.357 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:45.357 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 3, 2023-10-26T01:38:45.357 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:38:45.357 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:38:45.358 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:38:45.358 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:38:45.358 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:38:45.358 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.974541+0000", 2023-10-26T01:38:45.358 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:38:45.359 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:45.359 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:45.359 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 4, 2023-10-26T01:38:45.359 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:38:45.359 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:38:45.360 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:38:45.360 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:38:45.360 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:38:45.360 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.969696+0000", 2023-10-26T01:38:45.360 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:38:45.361 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:45.361 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:45.361 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 5, 2023-10-26T01:38:45.361 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:38:45.361 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:38:45.362 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:38:45.362 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:38:45.362 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:38:45.362 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.859703+0000", 2023-10-26T01:38:45.362 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:38:45.363 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:45.363 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:45.363 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 6, 2023-10-26T01:38:45.363 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:38:45.363 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:38:45.364 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:38:45.364 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:38:45.364 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:38:45.364 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.867917+0000", 2023-10-26T01:38:45.364 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:38:45.364 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:45.365 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:45.365 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 7, 2023-10-26T01:38:45.365 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:38:45.365 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:38:45.365 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:38:45.366 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:38:45.366 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:38:45.366 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.898605+0000", 2023-10-26T01:38:45.366 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:38:45.366 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:45.367 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:38:45.367 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_upmap": [], 2023-10-26T01:38:45.367 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_upmap_items": [ 2023-10-26T01:38:45.367 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:45.367 INFO:teuthology.orchestra.run.smithi169.stdout: "pgid": "17.e", 2023-10-26T01:38:45.368 INFO:teuthology.orchestra.run.smithi169.stdout: "mappings": [ 2023-10-26T01:38:45.368 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:45.368 INFO:teuthology.orchestra.run.smithi169.stdout: "from": 0, 2023-10-26T01:38:45.368 INFO:teuthology.orchestra.run.smithi169.stdout: "to": 5 2023-10-26T01:38:45.368 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:45.368 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:45.369 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:45.369 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:45.369 INFO:teuthology.orchestra.run.smithi169.stdout: "pgid": "17.21", 2023-10-26T01:38:45.369 INFO:teuthology.orchestra.run.smithi169.stdout: "mappings": [ 2023-10-26T01:38:45.369 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:45.370 INFO:teuthology.orchestra.run.smithi169.stdout: "from": 0, 2023-10-26T01:38:45.370 INFO:teuthology.orchestra.run.smithi169.stdout: "to": 2 2023-10-26T01:38:45.370 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:45.370 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:45.370 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:45.371 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:38:45.371 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_upmap_primaries": [], 2023-10-26T01:38:45.371 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_temp": [], 2023-10-26T01:38:45.371 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_temp": [], 2023-10-26T01:38:45.371 INFO:teuthology.orchestra.run.smithi169.stdout: "blocklist": {}, 2023-10-26T01:38:45.371 INFO:teuthology.orchestra.run.smithi169.stdout: "range_blocklist": {}, 2023-10-26T01:38:45.372 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profiles": { 2023-10-26T01:38:45.372 INFO:teuthology.orchestra.run.smithi169.stdout: "default": { 2023-10-26T01:38:45.372 INFO:teuthology.orchestra.run.smithi169.stdout: "crush-failure-domain": "osd", 2023-10-26T01:38:45.372 INFO:teuthology.orchestra.run.smithi169.stdout: "k": "2", 2023-10-26T01:38:45.372 INFO:teuthology.orchestra.run.smithi169.stdout: "m": "1", 2023-10-26T01:38:45.373 INFO:teuthology.orchestra.run.smithi169.stdout: "plugin": "jerasure", 2023-10-26T01:38:45.373 INFO:teuthology.orchestra.run.smithi169.stdout: "technique": "reed_sol_van" 2023-10-26T01:38:45.373 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:45.373 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:45.373 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps_queue": [], 2023-10-26T01:38:45.374 INFO:teuthology.orchestra.run.smithi169.stdout: "new_removed_snaps": [], 2023-10-26T01:38:45.374 INFO:teuthology.orchestra.run.smithi169.stdout: "new_purged_snaps": [], 2023-10-26T01:38:45.374 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_node_flags": {}, 2023-10-26T01:38:45.374 INFO:teuthology.orchestra.run.smithi169.stdout: "device_class_flags": {}, 2023-10-26T01:38:45.374 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_mode": { 2023-10-26T01:38:45.375 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_mode_enabled": false, 2023-10-26T01:38:45.375 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_bucket_count": 0, 2023-10-26T01:38:45.375 INFO:teuthology.orchestra.run.smithi169.stdout: "degraded_stretch_mode": 0, 2023-10-26T01:38:45.375 INFO:teuthology.orchestra.run.smithi169.stdout: "recovering_stretch_mode": 0, 2023-10-26T01:38:45.375 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_mode_bucket": 0 2023-10-26T01:38:45.376 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:45.376 INFO:teuthology.orchestra.run.smithi169.stdout:} 2023-10-26T01:38:45.378 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:45.167+0000 7f08c784b640 1 -- 172.21.15.169:0/2710123005 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f08a406ae40 msgr2=0x7f08a406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:45.378 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:45.167+0000 7f08c784b640 1 --2- 172.21.15.169:0/2710123005 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f08a406ae40 0x7f08a406d300 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f08c00cb940 tx=0x7f08b0008040 comp rx=0 tx=0).stop 2023-10-26T01:38:45.378 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:45.167+0000 7f08c784b640 1 -- 172.21.15.169:0/2710123005 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08c00ced40 msgr2=0x7f08c01f23a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:45.378 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:45.167+0000 7f08c784b640 1 --2- 172.21.15.169:0/2710123005 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08c00ced40 0x7f08c01f23a0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f08bc009850 tx=0x7f08bc02dae0 comp rx=0 tx=0).stop 2023-10-26T01:38:45.378 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:45.167+0000 7f08c784b640 1 -- 172.21.15.169:0/2710123005 shutdown_connections 2023-10-26T01:38:45.379 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:45.167+0000 7f08c784b640 1 --2- 172.21.15.169:0/2710123005 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f08a406ae40 0x7f08a406d300 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:45.379 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:45.167+0000 7f08c784b640 1 --2- 172.21.15.169:0/2710123005 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f08c015f610 0x7f08c0157f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:45.379 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:45.167+0000 7f08c784b640 1 --2- 172.21.15.169:0/2710123005 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f08c015b9d0 0x7f08c00c8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:45.379 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:45.167+0000 7f08c784b640 1 --2- 172.21.15.169:0/2710123005 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08c00ced40 0x7f08c01f23a0 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:45.379 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:45.167+0000 7f08c784b640 1 -- 172.21.15.169:0/2710123005 >> 172.21.15.169:0/2710123005 conn(0x7f08c0069e00 msgr2=0x7f08c015aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:45.380 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:45.168+0000 7f08c784b640 1 -- 172.21.15.169:0/2710123005 shutdown_connections 2023-10-26T01:38:45.380 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:45.168+0000 7f08c784b640 1 -- 172.21.15.169:0/2710123005 wait complete. 2023-10-26T01:38:45.381 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2023-10-26T01:38:50.381 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-26T01:38:50.613 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:50.610+0000 7fb886a4f640 1 -- 172.21.15.169:0/1245261684 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb88014e790 msgr2=0x7fb8800aab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:50.613 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:50.610+0000 7fb886a4f640 1 --2- 172.21.15.169:0/1245261684 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb88014e790 0x7fb8800aab40 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fb870005fd0 tx=0x7fb87002c240 comp rx=0 tx=0).stop 2023-10-26T01:38:50.613 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:50.610+0000 7fb886a4f640 1 -- 172.21.15.169:0/1245261684 shutdown_connections 2023-10-26T01:38:50.613 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:50.610+0000 7fb886a4f640 1 --2- 172.21.15.169:0/1245261684 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb880158850 0x7fb8800cd720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:50.614 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:50.610+0000 7fb886a4f640 1 --2- 172.21.15.169:0/1245261684 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb88014e790 0x7fb8800aab40 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:50.614 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:50.610+0000 7fb886a4f640 1 -- 172.21.15.169:0/1245261684 >> 172.21.15.169:0/1245261684 conn(0x7fb880069930 msgr2=0x7fb880069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:50.614 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:50.610+0000 7fb886a4f640 1 -- 172.21.15.169:0/1245261684 shutdown_connections 2023-10-26T01:38:50.614 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:50.610+0000 7fb886a4f640 1 -- 172.21.15.169:0/1245261684 wait complete. 2023-10-26T01:38:50.615 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:50.612+0000 7fb886a4f640 1 Processor -- start 2023-10-26T01:38:50.615 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:50.612+0000 7fb886a4f640 1 -- start start 2023-10-26T01:38:50.617 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:50.613+0000 7fb886a4f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb8800cdc60 0x7fb8801546a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:50.617 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:50.613+0000 7fb886a4f640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb88014e790 0x7fb8800aff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:50.618 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:50.613+0000 7fb886a4f640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb880158850 0x7fb8800b0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:50.618 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:50.613+0000 7fb886a4f640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8801536b0 con 0x7fb88014e790 2023-10-26T01:38:50.618 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:50.613+0000 7fb886a4f640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb880154210 con 0x7fb8800cdc60 2023-10-26T01:38:50.618 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:50.613+0000 7fb886a4f640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb880070240 con 0x7fb880158850 2023-10-26T01:38:50.618 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:50.613+0000 7fb87f7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb88014e790 0x7fb8800aff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:50.619 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:50.613+0000 7fb87f7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb88014e790 0x7fb8800aff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:53492/0 (socket says 172.21.15.169:53492) 2023-10-26T01:38:50.619 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:50.613+0000 7fb87f7fe640 1 -- 172.21.15.169:0/3579237634 learned_addr learned my addr 172.21.15.169:0/3579237634 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:38:50.619 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:50.613+0000 7fb87ffff640 1 --2- 172.21.15.169:0/3579237634 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb8800cdc60 0x7fb8801546a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:50.619 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:50.614+0000 7fb884fec640 1 --2- 172.21.15.169:0/3579237634 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb880158850 0x7fb8800b0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:50.619 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:50.614+0000 7fb87f7fe640 1 -- 172.21.15.169:0/3579237634 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb880158850 msgr2=0x7fb8800b0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:50.620 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:50.614+0000 7fb87f7fe640 1 --2- 172.21.15.169:0/3579237634 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb880158850 0x7fb8800b0450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:50.620 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:50.614+0000 7fb87f7fe640 1 -- 172.21.15.169:0/3579237634 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb8800cdc60 msgr2=0x7fb8801546a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:50.620 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:50.614+0000 7fb87f7fe640 1 --2- 172.21.15.169:0/3579237634 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb8800cdc60 0x7fb8801546a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:50.620 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:50.614+0000 7fb87f7fe640 1 -- 172.21.15.169:0/3579237634 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb870009900 con 0x7fb88014e790 2023-10-26T01:38:50.620 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:50.614+0000 7fb87ffff640 1 --2- 172.21.15.169:0/3579237634 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb8800cdc60 0x7fb8801546a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:38:50.621 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:50.614+0000 7fb87f7fe640 1 --2- 172.21.15.169:0/3579237634 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb88014e790 0x7fb8800aff10 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7fb868009800 tx=0x7fb868009cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:50.621 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:50.614+0000 7fb87cff9640 1 -- 172.21.15.169:0/3579237634 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb86800a300 con 0x7fb88014e790 2023-10-26T01:38:50.621 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:50.614+0000 7fb87cff9640 1 -- 172.21.15.169:0/3579237634 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb868010040 con 0x7fb88014e790 2023-10-26T01:38:50.621 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:50.614+0000 7fb886a4f640 1 -- 172.21.15.169:0/3579237634 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8801538e0 con 0x7fb88014e790 2023-10-26T01:38:50.621 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:50.614+0000 7fb87cff9640 1 -- 172.21.15.169:0/3579237634 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb86800ee30 con 0x7fb88014e790 2023-10-26T01:38:50.622 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:50.614+0000 7fb886a4f640 1 -- 172.21.15.169:0/3579237634 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb880153ea0 con 0x7fb88014e790 2023-10-26T01:38:50.623 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:50.616+0000 7fb886a4f640 1 -- 172.21.15.169:0/3579237634 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8801543c0 con 0x7fb88014e790 2023-10-26T01:38:50.623 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:50.617+0000 7fb87cff9640 1 -- 172.21.15.169:0/3579237634 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fb868014410 con 0x7fb88014e790 2023-10-26T01:38:50.623 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:50.617+0000 7fb87cff9640 1 --2- 172.21.15.169:0/3579237634 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb86006ae40 0x7fb86006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:50.623 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:50.618+0000 7fb87cff9640 1 -- 172.21.15.169:0/3579237634 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fb868089db0 con 0x7fb88014e790 2023-10-26T01:38:50.623 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:50.618+0000 7fb87cff9640 1 -- 172.21.15.169:0/3579237634 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=139}) v3 -- 0x7fb860072a30 con 0x7fb88014e790 2023-10-26T01:38:50.624 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:50.618+0000 7fb87ffff640 1 --2- 172.21.15.169:0/3579237634 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb86006ae40 0x7fb86006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:50.624 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:50.619+0000 7fb87ffff640 1 --2- 172.21.15.169:0/3579237634 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb86006ae40 0x7fb86006d300 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fb870036000 tx=0x7fb870004ab0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:50.625 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:50.622+0000 7fb87cff9640 1 -- 172.21.15.169:0/3579237634 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb868054ed0 con 0x7fb88014e790 2023-10-26T01:38:50.816 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:50.813+0000 7fb886a4f640 1 -- 172.21.15.169:0/3579237634 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fb88004db10 con 0x7fb88014e790 2023-10-26T01:38:50.820 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:50.817+0000 7fb87cff9640 1 -- 172.21.15.169:0/3579237634 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v138) v1 ==== 81+0+34469 (secure 0 0 0) 0x7fb8680584f0 con 0x7fb88014e790 2023-10-26T01:38:50.820 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:38:50.821 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2023-10-26T01:38:50.821 INFO:teuthology.orchestra.run.smithi169.stdout: "epoch": 138, 2023-10-26T01:38:50.821 INFO:teuthology.orchestra.run.smithi169.stdout: "fsid": "30203b28-ed8a-44e8-a9a6-a9bf25f980f1", 2023-10-26T01:38:50.821 INFO:teuthology.orchestra.run.smithi169.stdout: "created": "2023-10-26T01:30:22.897155+0000", 2023-10-26T01:38:50.822 INFO:teuthology.orchestra.run.smithi169.stdout: "modified": "2023-10-26T01:38:48.057101+0000", 2023-10-26T01:38:50.822 INFO:teuthology.orchestra.run.smithi169.stdout: "last_up_change": "2023-10-26T01:30:31.875281+0000", 2023-10-26T01:38:50.822 INFO:teuthology.orchestra.run.smithi169.stdout: "last_in_change": "2023-10-26T01:30:27.637941+0000", 2023-10-26T01:38:50.822 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-26T01:38:50.822 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_num": 5799936, 2023-10-26T01:38:50.823 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_set": [ 2023-10-26T01:38:50.823 INFO:teuthology.orchestra.run.smithi169.stdout: "pglog_hardlimit", 2023-10-26T01:38:50.823 INFO:teuthology.orchestra.run.smithi169.stdout: "purged_snapdirs", 2023-10-26T01:38:50.823 INFO:teuthology.orchestra.run.smithi169.stdout: "recovery_deletes", 2023-10-26T01:38:50.823 INFO:teuthology.orchestra.run.smithi169.stdout: "sortbitwise" 2023-10-26T01:38:50.824 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:38:50.824 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_version": 4, 2023-10-26T01:38:50.824 INFO:teuthology.orchestra.run.smithi169.stdout: "full_ratio": 0.69999998807907104, 2023-10-26T01:38:50.824 INFO:teuthology.orchestra.run.smithi169.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-26T01:38:50.825 INFO:teuthology.orchestra.run.smithi169.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-26T01:38:50.825 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_snapshot": "", 2023-10-26T01:38:50.825 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_max": 17, 2023-10-26T01:38:50.825 INFO:teuthology.orchestra.run.smithi169.stdout: "max_osd": 8, 2023-10-26T01:38:50.826 INFO:teuthology.orchestra.run.smithi169.stdout: "require_min_compat_client": "luminous", 2023-10-26T01:38:50.826 INFO:teuthology.orchestra.run.smithi169.stdout: "min_compat_client": "luminous", 2023-10-26T01:38:50.826 INFO:teuthology.orchestra.run.smithi169.stdout: "require_osd_release": "reef", 2023-10-26T01:38:50.826 INFO:teuthology.orchestra.run.smithi169.stdout: "allow_crimson": false, 2023-10-26T01:38:50.826 INFO:teuthology.orchestra.run.smithi169.stdout: "pools": [ 2023-10-26T01:38:50.827 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:50.827 INFO:teuthology.orchestra.run.smithi169.stdout: "pool": 1, 2023-10-26T01:38:50.827 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_name": ".mgr", 2023-10-26T01:38:50.827 INFO:teuthology.orchestra.run.smithi169.stdout: "create_time": "2023-10-26T01:30:32.200808+0000", 2023-10-26T01:38:50.828 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": 1, 2023-10-26T01:38:50.828 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_names": "hashpspool", 2023-10-26T01:38:50.828 INFO:teuthology.orchestra.run.smithi169.stdout: "type": 1, 2023-10-26T01:38:50.829 INFO:teuthology.orchestra.run.smithi169.stdout: "size": 2, 2023-10-26T01:38:50.829 INFO:teuthology.orchestra.run.smithi169.stdout: "min_size": 1, 2023-10-26T01:38:50.829 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_rule": 0, 2023-10-26T01:38:50.830 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_count": 0, 2023-10-26T01:38:50.830 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_target": 0, 2023-10-26T01:38:50.830 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T01:38:50.831 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T01:38:50.831 INFO:teuthology.orchestra.run.smithi169.stdout: "object_hash": 2, 2023-10-26T01:38:50.831 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_mode": "off", 2023-10-26T01:38:50.832 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num": 1, 2023-10-26T01:38:50.832 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num": 1, 2023-10-26T01:38:50.832 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num_target": 1, 2023-10-26T01:38:50.833 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_target": 1, 2023-10-26T01:38:50.833 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_pending": 1, 2023-10-26T01:38:50.833 INFO:teuthology.orchestra.run.smithi169.stdout: "last_pg_merge_meta": { 2023-10-26T01:38:50.834 INFO:teuthology.orchestra.run.smithi169.stdout: "source_pgid": "0.0", 2023-10-26T01:38:50.834 INFO:teuthology.orchestra.run.smithi169.stdout: "ready_epoch": 0, 2023-10-26T01:38:50.835 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_started": 0, 2023-10-26T01:38:50.835 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_clean": 0, 2023-10-26T01:38:50.835 INFO:teuthology.orchestra.run.smithi169.stdout: "source_version": "0'0", 2023-10-26T01:38:50.836 INFO:teuthology.orchestra.run.smithi169.stdout: "target_version": "0'0" 2023-10-26T01:38:50.836 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:50.836 INFO:teuthology.orchestra.run.smithi169.stdout: "last_change": "109", 2023-10-26T01:38:50.837 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend": "0", 2023-10-26T01:38:50.837 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T01:38:50.837 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T01:38:50.837 INFO:teuthology.orchestra.run.smithi169.stdout: "auid": 0, 2023-10-26T01:38:50.837 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_mode": "selfmanaged", 2023-10-26T01:38:50.838 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_seq": 0, 2023-10-26T01:38:50.838 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_epoch": 0, 2023-10-26T01:38:50.838 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_snaps": [], 2023-10-26T01:38:50.838 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps": "[]", 2023-10-26T01:38:50.839 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_bytes": 0, 2023-10-26T01:38:50.839 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_objects": 0, 2023-10-26T01:38:50.839 INFO:teuthology.orchestra.run.smithi169.stdout: "tiers": [], 2023-10-26T01:38:50.839 INFO:teuthology.orchestra.run.smithi169.stdout: "tier_of": -1, 2023-10-26T01:38:50.839 INFO:teuthology.orchestra.run.smithi169.stdout: "read_tier": -1, 2023-10-26T01:38:50.840 INFO:teuthology.orchestra.run.smithi169.stdout: "write_tier": -1, 2023-10-26T01:38:50.840 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_mode": "none", 2023-10-26T01:38:50.840 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_bytes": 0, 2023-10-26T01:38:50.840 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_objects": 0, 2023-10-26T01:38:50.840 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T01:38:50.841 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T01:38:50.841 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T01:38:50.841 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_flush_age": 0, 2023-10-26T01:38:50.841 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_evict_age": 0, 2023-10-26T01:38:50.841 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profile": "", 2023-10-26T01:38:50.842 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_params": { 2023-10-26T01:38:50.842 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "none" 2023-10-26T01:38:50.842 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:50.842 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_period": 0, 2023-10-26T01:38:50.843 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_count": 0, 2023-10-26T01:38:50.843 INFO:teuthology.orchestra.run.smithi169.stdout: "use_gmt_hitset": true, 2023-10-26T01:38:50.843 INFO:teuthology.orchestra.run.smithi169.stdout: "min_read_recency_for_promote": 0, 2023-10-26T01:38:50.843 INFO:teuthology.orchestra.run.smithi169.stdout: "min_write_recency_for_promote": 0, 2023-10-26T01:38:50.843 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T01:38:50.844 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_search_last_n": 0, 2023-10-26T01:38:50.844 INFO:teuthology.orchestra.run.smithi169.stdout: "grade_table": [], 2023-10-26T01:38:50.844 INFO:teuthology.orchestra.run.smithi169.stdout: "stripe_width": 0, 2023-10-26T01:38:50.844 INFO:teuthology.orchestra.run.smithi169.stdout: "expected_num_objects": 0, 2023-10-26T01:38:50.844 INFO:teuthology.orchestra.run.smithi169.stdout: "fast_read": false, 2023-10-26T01:38:50.845 INFO:teuthology.orchestra.run.smithi169.stdout: "options": { 2023-10-26T01:38:50.845 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_max": 32, 2023-10-26T01:38:50.845 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_min": 1 2023-10-26T01:38:50.845 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:50.845 INFO:teuthology.orchestra.run.smithi169.stdout: "application_metadata": { 2023-10-26T01:38:50.845 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": {} 2023-10-26T01:38:50.846 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:50.846 INFO:teuthology.orchestra.run.smithi169.stdout: "read_balance": { 2023-10-26T01:38:50.846 INFO:teuthology.orchestra.run.smithi169.stdout: "score_acting": 8, 2023-10-26T01:38:50.846 INFO:teuthology.orchestra.run.smithi169.stdout: "score_stable": 8, 2023-10-26T01:38:50.846 INFO:teuthology.orchestra.run.smithi169.stdout: "optimal_score": 0.25, 2023-10-26T01:38:50.847 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_acting": 2, 2023-10-26T01:38:50.847 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_stable": 2, 2023-10-26T01:38:50.847 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity_weighted": 1, 2023-10-26T01:38:50.847 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity": 1, 2023-10-26T01:38:50.848 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity_weighted": 1 2023-10-26T01:38:50.848 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:50.848 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:50.848 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:50.848 INFO:teuthology.orchestra.run.smithi169.stdout: "pool": 16, 2023-10-26T01:38:50.849 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_name": "cephfs_metadata", 2023-10-26T01:38:50.849 INFO:teuthology.orchestra.run.smithi169.stdout: "create_time": "2023-10-26T01:38:18.617877+0000", 2023-10-26T01:38:50.849 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": 1, 2023-10-26T01:38:50.849 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_names": "hashpspool", 2023-10-26T01:38:50.849 INFO:teuthology.orchestra.run.smithi169.stdout: "type": 1, 2023-10-26T01:38:50.850 INFO:teuthology.orchestra.run.smithi169.stdout: "size": 2, 2023-10-26T01:38:50.850 INFO:teuthology.orchestra.run.smithi169.stdout: "min_size": 1, 2023-10-26T01:38:50.850 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_rule": 0, 2023-10-26T01:38:50.850 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_count": 0, 2023-10-26T01:38:50.850 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_target": 0, 2023-10-26T01:38:50.851 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T01:38:50.851 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T01:38:50.851 INFO:teuthology.orchestra.run.smithi169.stdout: "object_hash": 2, 2023-10-26T01:38:50.851 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_mode": "off", 2023-10-26T01:38:50.851 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num": 32, 2023-10-26T01:38:50.852 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num": 32, 2023-10-26T01:38:50.852 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num_target": 32, 2023-10-26T01:38:50.852 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_target": 32, 2023-10-26T01:38:50.852 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_pending": 32, 2023-10-26T01:38:50.853 INFO:teuthology.orchestra.run.smithi169.stdout: "last_pg_merge_meta": { 2023-10-26T01:38:50.853 INFO:teuthology.orchestra.run.smithi169.stdout: "source_pgid": "0.0", 2023-10-26T01:38:50.853 INFO:teuthology.orchestra.run.smithi169.stdout: "ready_epoch": 0, 2023-10-26T01:38:50.854 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_started": 0, 2023-10-26T01:38:50.854 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_clean": 0, 2023-10-26T01:38:50.854 INFO:teuthology.orchestra.run.smithi169.stdout: "source_version": "0'0", 2023-10-26T01:38:50.855 INFO:teuthology.orchestra.run.smithi169.stdout: "target_version": "0'0" 2023-10-26T01:38:50.855 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:50.855 INFO:teuthology.orchestra.run.smithi169.stdout: "last_change": "133", 2023-10-26T01:38:50.856 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend": "0", 2023-10-26T01:38:50.856 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T01:38:50.856 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T01:38:50.857 INFO:teuthology.orchestra.run.smithi169.stdout: "auid": 0, 2023-10-26T01:38:50.857 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_mode": "selfmanaged", 2023-10-26T01:38:50.857 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_seq": 0, 2023-10-26T01:38:50.858 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_epoch": 0, 2023-10-26T01:38:50.858 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_snaps": [], 2023-10-26T01:38:50.858 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps": "[]", 2023-10-26T01:38:50.859 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_bytes": 0, 2023-10-26T01:38:50.859 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_objects": 0, 2023-10-26T01:38:50.859 INFO:teuthology.orchestra.run.smithi169.stdout: "tiers": [], 2023-10-26T01:38:50.860 INFO:teuthology.orchestra.run.smithi169.stdout: "tier_of": -1, 2023-10-26T01:38:50.860 INFO:teuthology.orchestra.run.smithi169.stdout: "read_tier": -1, 2023-10-26T01:38:50.860 INFO:teuthology.orchestra.run.smithi169.stdout: "write_tier": -1, 2023-10-26T01:38:50.861 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_mode": "none", 2023-10-26T01:38:50.861 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_bytes": 0, 2023-10-26T01:38:50.862 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_objects": 0, 2023-10-26T01:38:50.862 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T01:38:50.862 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T01:38:50.863 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T01:38:50.863 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_flush_age": 0, 2023-10-26T01:38:50.863 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_evict_age": 0, 2023-10-26T01:38:50.864 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profile": "", 2023-10-26T01:38:50.864 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_params": { 2023-10-26T01:38:50.864 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "none" 2023-10-26T01:38:50.864 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:50.864 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_period": 0, 2023-10-26T01:38:50.865 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_count": 0, 2023-10-26T01:38:50.865 INFO:teuthology.orchestra.run.smithi169.stdout: "use_gmt_hitset": true, 2023-10-26T01:38:50.865 INFO:teuthology.orchestra.run.smithi169.stdout: "min_read_recency_for_promote": 0, 2023-10-26T01:38:50.865 INFO:teuthology.orchestra.run.smithi169.stdout: "min_write_recency_for_promote": 0, 2023-10-26T01:38:50.865 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T01:38:50.865 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_search_last_n": 0, 2023-10-26T01:38:50.866 INFO:teuthology.orchestra.run.smithi169.stdout: "grade_table": [], 2023-10-26T01:38:50.866 INFO:teuthology.orchestra.run.smithi169.stdout: "stripe_width": 0, 2023-10-26T01:38:50.866 INFO:teuthology.orchestra.run.smithi169.stdout: "expected_num_objects": 0, 2023-10-26T01:38:50.866 INFO:teuthology.orchestra.run.smithi169.stdout: "fast_read": false, 2023-10-26T01:38:50.867 INFO:teuthology.orchestra.run.smithi169.stdout: "options": { 2023-10-26T01:38:50.867 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_bias": 4, 2023-10-26T01:38:50.867 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_min": 16, 2023-10-26T01:38:50.867 INFO:teuthology.orchestra.run.smithi169.stdout: "recovery_priority": 5 2023-10-26T01:38:50.867 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:50.868 INFO:teuthology.orchestra.run.smithi169.stdout: "application_metadata": { 2023-10-26T01:38:50.868 INFO:teuthology.orchestra.run.smithi169.stdout: "cephfs": { 2023-10-26T01:38:50.868 INFO:teuthology.orchestra.run.smithi169.stdout: "metadata": "cephfs" 2023-10-26T01:38:50.868 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:50.868 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:50.869 INFO:teuthology.orchestra.run.smithi169.stdout: "read_balance": { 2023-10-26T01:38:50.869 INFO:teuthology.orchestra.run.smithi169.stdout: "score_acting": 1.75, 2023-10-26T01:38:50.869 INFO:teuthology.orchestra.run.smithi169.stdout: "score_stable": 1.75, 2023-10-26T01:38:50.869 INFO:teuthology.orchestra.run.smithi169.stdout: "optimal_score": 1, 2023-10-26T01:38:50.869 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_acting": 1.75, 2023-10-26T01:38:50.870 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_stable": 1.75, 2023-10-26T01:38:50.870 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity_weighted": 1, 2023-10-26T01:38:50.870 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity": 1, 2023-10-26T01:38:50.870 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity_weighted": 1 2023-10-26T01:38:50.870 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:50.871 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:50.871 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:50.871 INFO:teuthology.orchestra.run.smithi169.stdout: "pool": 17, 2023-10-26T01:38:50.871 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_name": "cephfs_data", 2023-10-26T01:38:50.871 INFO:teuthology.orchestra.run.smithi169.stdout: "create_time": "2023-10-26T01:38:19.793041+0000", 2023-10-26T01:38:50.871 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": 1027, 2023-10-26T01:38:50.872 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_names": "hashpspool,full,full_quota", 2023-10-26T01:38:50.872 INFO:teuthology.orchestra.run.smithi169.stdout: "type": 1, 2023-10-26T01:38:50.872 INFO:teuthology.orchestra.run.smithi169.stdout: "size": 2, 2023-10-26T01:38:50.872 INFO:teuthology.orchestra.run.smithi169.stdout: "min_size": 1, 2023-10-26T01:38:50.872 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_rule": 0, 2023-10-26T01:38:50.873 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_count": 0, 2023-10-26T01:38:50.873 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_target": 0, 2023-10-26T01:38:50.873 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T01:38:50.873 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T01:38:50.873 INFO:teuthology.orchestra.run.smithi169.stdout: "object_hash": 2, 2023-10-26T01:38:50.873 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_mode": "off", 2023-10-26T01:38:50.874 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num": 64, 2023-10-26T01:38:50.874 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num": 64, 2023-10-26T01:38:50.874 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num_target": 64, 2023-10-26T01:38:50.874 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_target": 64, 2023-10-26T01:38:50.875 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_pending": 64, 2023-10-26T01:38:50.875 INFO:teuthology.orchestra.run.smithi169.stdout: "last_pg_merge_meta": { 2023-10-26T01:38:50.875 INFO:teuthology.orchestra.run.smithi169.stdout: "source_pgid": "0.0", 2023-10-26T01:38:50.875 INFO:teuthology.orchestra.run.smithi169.stdout: "ready_epoch": 0, 2023-10-26T01:38:50.875 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_started": 0, 2023-10-26T01:38:50.876 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_clean": 0, 2023-10-26T01:38:50.876 INFO:teuthology.orchestra.run.smithi169.stdout: "source_version": "0'0", 2023-10-26T01:38:50.876 INFO:teuthology.orchestra.run.smithi169.stdout: "target_version": "0'0" 2023-10-26T01:38:50.876 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:50.876 INFO:teuthology.orchestra.run.smithi169.stdout: "last_change": "138", 2023-10-26T01:38:50.877 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend": "0", 2023-10-26T01:38:50.877 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T01:38:50.877 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T01:38:50.877 INFO:teuthology.orchestra.run.smithi169.stdout: "auid": 0, 2023-10-26T01:38:50.877 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_mode": "selfmanaged", 2023-10-26T01:38:50.878 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_seq": 0, 2023-10-26T01:38:50.878 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_epoch": 0, 2023-10-26T01:38:50.878 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_snaps": [], 2023-10-26T01:38:50.878 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps": "[]", 2023-10-26T01:38:50.878 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_bytes": 33554432, 2023-10-26T01:38:50.879 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_objects": 0, 2023-10-26T01:38:50.879 INFO:teuthology.orchestra.run.smithi169.stdout: "tiers": [], 2023-10-26T01:38:50.879 INFO:teuthology.orchestra.run.smithi169.stdout: "tier_of": -1, 2023-10-26T01:38:50.879 INFO:teuthology.orchestra.run.smithi169.stdout: "read_tier": -1, 2023-10-26T01:38:50.879 INFO:teuthology.orchestra.run.smithi169.stdout: "write_tier": -1, 2023-10-26T01:38:50.879 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_mode": "none", 2023-10-26T01:38:50.880 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_bytes": 0, 2023-10-26T01:38:50.880 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_objects": 0, 2023-10-26T01:38:50.880 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T01:38:50.880 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T01:38:50.880 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T01:38:50.881 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_flush_age": 0, 2023-10-26T01:38:50.881 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_evict_age": 0, 2023-10-26T01:38:50.881 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profile": "", 2023-10-26T01:38:50.881 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_params": { 2023-10-26T01:38:50.881 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "none" 2023-10-26T01:38:50.882 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:50.882 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_period": 0, 2023-10-26T01:38:50.882 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_count": 0, 2023-10-26T01:38:50.882 INFO:teuthology.orchestra.run.smithi169.stdout: "use_gmt_hitset": true, 2023-10-26T01:38:50.882 INFO:teuthology.orchestra.run.smithi169.stdout: "min_read_recency_for_promote": 0, 2023-10-26T01:38:50.883 INFO:teuthology.orchestra.run.smithi169.stdout: "min_write_recency_for_promote": 0, 2023-10-26T01:38:50.883 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T01:38:50.883 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_search_last_n": 0, 2023-10-26T01:38:50.883 INFO:teuthology.orchestra.run.smithi169.stdout: "grade_table": [], 2023-10-26T01:38:50.883 INFO:teuthology.orchestra.run.smithi169.stdout: "stripe_width": 0, 2023-10-26T01:38:50.884 INFO:teuthology.orchestra.run.smithi169.stdout: "expected_num_objects": 0, 2023-10-26T01:38:50.884 INFO:teuthology.orchestra.run.smithi169.stdout: "fast_read": false, 2023-10-26T01:38:50.884 INFO:teuthology.orchestra.run.smithi169.stdout: "options": { 2023-10-26T01:38:50.884 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_min": 64, 2023-10-26T01:38:50.884 INFO:teuthology.orchestra.run.smithi169.stdout: "target_size_ratio": 0.89999997615814209 2023-10-26T01:38:50.884 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:50.885 INFO:teuthology.orchestra.run.smithi169.stdout: "application_metadata": { 2023-10-26T01:38:50.885 INFO:teuthology.orchestra.run.smithi169.stdout: "cephfs": { 2023-10-26T01:38:50.885 INFO:teuthology.orchestra.run.smithi169.stdout: "data": "cephfs" 2023-10-26T01:38:50.885 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:50.886 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:50.886 INFO:teuthology.orchestra.run.smithi169.stdout: "read_balance": { 2023-10-26T01:38:50.886 INFO:teuthology.orchestra.run.smithi169.stdout: "score_acting": 1.6299999952316284, 2023-10-26T01:38:50.887 INFO:teuthology.orchestra.run.smithi169.stdout: "score_stable": 1.6299999952316284, 2023-10-26T01:38:50.887 INFO:teuthology.orchestra.run.smithi169.stdout: "optimal_score": 1, 2023-10-26T01:38:50.887 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_acting": 1.6299999952316284, 2023-10-26T01:38:50.887 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_stable": 1.6299999952316284, 2023-10-26T01:38:50.887 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity_weighted": 1, 2023-10-26T01:38:50.888 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity": 1, 2023-10-26T01:38:50.888 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity_weighted": 1 2023-10-26T01:38:50.888 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:50.888 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:50.888 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:38:50.889 INFO:teuthology.orchestra.run.smithi169.stdout: "osds": [ 2023-10-26T01:38:50.889 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:50.889 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 0, 2023-10-26T01:38:50.889 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:38:50.889 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:38:50.890 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:38:50.890 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:38:50.890 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:38:50.890 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:38:50.890 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:38:50.891 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:38:50.891 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 132, 2023-10-26T01:38:50.891 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:38:50.891 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:38:50.891 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:38:50.892 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:50.892 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:50.892 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:50.892 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6816", 2023-10-26T01:38:50.893 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:38:50.893 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:50.893 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:50.894 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:50.894 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6817", 2023-10-26T01:38:50.895 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:38:50.895 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:50.895 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:50.895 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:50.895 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:38:50.895 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:50.896 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:50.896 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:50.896 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6818", 2023-10-26T01:38:50.896 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:38:50.896 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:50.897 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:50.897 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:50.897 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6819", 2023-10-26T01:38:50.898 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:38:50.898 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:50.898 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:50.898 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:50.899 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:38:50.899 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:50.899 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:50.899 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:50.900 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6822", 2023-10-26T01:38:50.900 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:38:50.900 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:50.900 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:50.900 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:50.900 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6823", 2023-10-26T01:38:50.901 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:38:50.901 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:50.901 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:50.901 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:50.901 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:38:50.902 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:50.902 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:50.902 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:50.902 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6820", 2023-10-26T01:38:50.902 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:38:50.903 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:50.903 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:50.903 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:50.903 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6821", 2023-10-26T01:38:50.903 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:38:50.904 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:50.904 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:50.904 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:50.904 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6817/4204255864", 2023-10-26T01:38:50.904 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6819/4204255864", 2023-10-26T01:38:50.904 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6823/4204255864", 2023-10-26T01:38:50.905 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6821/4204255864", 2023-10-26T01:38:50.905 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:38:50.905 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:38:50.905 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:38:50.906 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:50.906 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:50.906 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:50.907 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 1, 2023-10-26T01:38:50.907 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:38:50.907 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:38:50.908 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:38:50.908 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:38:50.908 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:38:50.909 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:38:50.909 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:38:50.910 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:38:50.910 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 132, 2023-10-26T01:38:50.911 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:38:50.911 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:38:50.911 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:38:50.912 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:50.912 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:50.912 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:50.913 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6808", 2023-10-26T01:38:50.913 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:38:50.913 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:50.914 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:50.914 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:50.914 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6809", 2023-10-26T01:38:50.915 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:38:50.915 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:50.915 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:50.916 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:50.916 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:38:50.916 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:50.917 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:50.917 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:50.918 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6810", 2023-10-26T01:38:50.918 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:38:50.918 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:50.919 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:50.919 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:50.920 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6811", 2023-10-26T01:38:50.920 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:38:50.920 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:50.921 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:50.921 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:50.921 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:38:50.922 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:50.922 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:50.923 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:50.923 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6814", 2023-10-26T01:38:50.923 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:38:50.924 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:50.924 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:50.924 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:50.925 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6815", 2023-10-26T01:38:50.925 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:38:50.926 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:50.926 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:50.926 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:50.927 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:38:50.927 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:50.928 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:50.928 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:50.928 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6812", 2023-10-26T01:38:50.929 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:38:50.929 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:50.929 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:50.930 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:50.930 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6813", 2023-10-26T01:38:50.930 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:38:50.931 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:50.931 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:50.931 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:50.932 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6809/238206046", 2023-10-26T01:38:50.932 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6811/238206046", 2023-10-26T01:38:50.932 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6815/238206046", 2023-10-26T01:38:50.933 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6813/238206046", 2023-10-26T01:38:50.933 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:38:50.933 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:38:50.933 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:38:50.934 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:50.934 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:50.934 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:50.935 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 2, 2023-10-26T01:38:50.935 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:38:50.935 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:38:50.935 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:38:50.936 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:38:50.936 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:38:50.936 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:38:50.936 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:38:50.937 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:38:50.937 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 135, 2023-10-26T01:38:50.937 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:38:50.937 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:38:50.937 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:38:50.938 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:50.938 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:50.938 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:50.938 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6800", 2023-10-26T01:38:50.938 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:38:50.939 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:50.939 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:50.939 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:50.939 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6801", 2023-10-26T01:38:50.939 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:38:50.940 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:50.940 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:50.940 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:50.940 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:38:50.940 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:50.940 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:50.941 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:50.941 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6802", 2023-10-26T01:38:50.941 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:38:50.941 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:50.941 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:50.942 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:50.942 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6803", 2023-10-26T01:38:50.942 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:38:50.942 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:50.943 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:50.943 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:50.943 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:38:50.943 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:50.943 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:50.944 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:50.944 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6806", 2023-10-26T01:38:50.944 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:38:50.944 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:50.944 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:50.944 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:50.945 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6807", 2023-10-26T01:38:50.945 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:38:50.945 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:50.945 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:50.945 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:50.946 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:38:50.946 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:50.946 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:50.946 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:50.946 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6804", 2023-10-26T01:38:50.946 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:38:50.947 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:50.947 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:50.947 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:50.947 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6805", 2023-10-26T01:38:50.947 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:38:50.948 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:50.948 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:50.948 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:50.948 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6801/3215791846", 2023-10-26T01:38:50.948 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6803/3215791846", 2023-10-26T01:38:50.948 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6807/3215791846", 2023-10-26T01:38:50.949 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6805/3215791846", 2023-10-26T01:38:50.949 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:38:50.949 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:38:50.949 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:38:50.949 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:50.949 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:50.950 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:50.950 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 3, 2023-10-26T01:38:50.950 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:38:50.950 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:38:50.950 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:38:50.951 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:38:50.951 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:38:50.951 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:38:50.951 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:38:50.951 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 14, 2023-10-26T01:38:50.952 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 132, 2023-10-26T01:38:50.952 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:38:50.952 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:38:50.952 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:38:50.952 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:50.952 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:50.953 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:50.953 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6824", 2023-10-26T01:38:50.953 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:38:50.953 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:50.954 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:50.954 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:50.954 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6825", 2023-10-26T01:38:50.954 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:38:50.954 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:50.955 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:50.955 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:50.955 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:38:50.955 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:50.955 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:50.955 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:50.956 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6826", 2023-10-26T01:38:50.956 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:38:50.956 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:50.956 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:50.956 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:50.957 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6827", 2023-10-26T01:38:50.957 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:38:50.957 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:50.957 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:50.957 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:50.957 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:38:50.958 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:50.958 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:50.958 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:50.958 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6830", 2023-10-26T01:38:50.958 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:38:50.959 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:50.959 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:50.959 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:50.959 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6831", 2023-10-26T01:38:50.959 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:38:50.960 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:50.960 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:50.960 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:50.960 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:38:50.960 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:50.961 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:50.961 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:50.961 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6828", 2023-10-26T01:38:50.961 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:38:50.961 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:50.962 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:50.962 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:50.962 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6829", 2023-10-26T01:38:50.962 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:38:50.963 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:50.963 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:50.963 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:50.963 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6825/2256701226", 2023-10-26T01:38:50.964 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6827/2256701226", 2023-10-26T01:38:50.964 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6831/2256701226", 2023-10-26T01:38:50.964 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6829/2256701226", 2023-10-26T01:38:50.964 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:38:50.964 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:38:50.964 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:38:50.965 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:50.965 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:50.965 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:50.965 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 4, 2023-10-26T01:38:50.965 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:38:50.966 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:38:50.966 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:38:50.966 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:38:50.966 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:38:50.966 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:38:50.967 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:38:50.967 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:38:50.967 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 132, 2023-10-26T01:38:50.967 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:38:50.967 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:38:50.968 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:38:50.968 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:50.968 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:50.968 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:50.968 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6816", 2023-10-26T01:38:50.969 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:38:50.969 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:50.969 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:50.969 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:50.969 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6817", 2023-10-26T01:38:50.969 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:38:50.970 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:50.970 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:50.970 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:50.970 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:38:50.970 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:50.971 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:50.971 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:50.971 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6818", 2023-10-26T01:38:50.971 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:38:50.971 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:50.971 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:50.972 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:50.972 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6819", 2023-10-26T01:38:50.972 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:38:50.972 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:50.972 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:50.973 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:50.973 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:38:50.973 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:50.973 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:50.973 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:50.974 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6825", 2023-10-26T01:38:50.974 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:38:50.974 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:50.974 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:50.974 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:50.975 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6828", 2023-10-26T01:38:50.975 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:38:50.975 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:50.975 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:50.975 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:50.976 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:38:50.976 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:50.976 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:50.976 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:50.976 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6820", 2023-10-26T01:38:50.977 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:38:50.977 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:50.977 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:50.977 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:50.977 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6823", 2023-10-26T01:38:50.977 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:38:50.978 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:50.978 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:50.978 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:50.978 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6817/1082279874", 2023-10-26T01:38:50.978 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6819/1082279874", 2023-10-26T01:38:50.979 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6828/1082279874", 2023-10-26T01:38:50.979 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6823/1082279874", 2023-10-26T01:38:50.979 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:38:50.979 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:38:50.979 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:38:50.980 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:50.980 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:50.980 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:50.980 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 5, 2023-10-26T01:38:50.981 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:38:50.981 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:38:50.981 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:38:50.981 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:38:50.982 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:38:50.982 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:38:50.982 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:38:50.982 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:38:50.982 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 132, 2023-10-26T01:38:50.983 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:38:50.983 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:38:50.983 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:38:50.983 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:50.983 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:50.984 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:50.984 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6808", 2023-10-26T01:38:50.984 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:38:50.984 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:50.984 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:50.985 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:50.985 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6809", 2023-10-26T01:38:50.985 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:38:50.985 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:50.985 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:50.986 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:50.986 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:38:50.986 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:50.986 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:50.986 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:50.986 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6810", 2023-10-26T01:38:50.987 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:38:50.987 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:50.987 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:50.987 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:50.987 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6811", 2023-10-26T01:38:50.988 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:38:50.988 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:50.988 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:50.988 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:50.988 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:38:50.989 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:50.989 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:50.989 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:50.989 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6814", 2023-10-26T01:38:50.989 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:38:50.989 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:50.990 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:50.990 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:50.990 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6815", 2023-10-26T01:38:50.990 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:38:50.991 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:50.991 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:50.991 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:50.991 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:38:50.991 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:50.992 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:50.992 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:50.992 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6812", 2023-10-26T01:38:50.992 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:38:50.992 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:50.992 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:50.993 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:50.993 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6813", 2023-10-26T01:38:50.993 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:38:50.993 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:50.993 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:50.994 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:50.994 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6809/3335654351", 2023-10-26T01:38:50.994 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6811/3335654351", 2023-10-26T01:38:50.994 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6815/3335654351", 2023-10-26T01:38:50.994 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6813/3335654351", 2023-10-26T01:38:50.994 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:38:50.995 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:38:50.995 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:38:50.995 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:50.995 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:50.995 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:50.995 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 6, 2023-10-26T01:38:50.996 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:38:50.996 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:38:50.996 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:38:50.996 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:38:50.996 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:38:50.996 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:38:50.997 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:38:50.997 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:38:50.997 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 132, 2023-10-26T01:38:50.997 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:38:50.997 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:38:50.997 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:38:50.998 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:50.998 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:50.998 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:50.998 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6800", 2023-10-26T01:38:50.998 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:38:50.999 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:50.999 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:50.999 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:50.999 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6801", 2023-10-26T01:38:50.999 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:38:51.000 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:51.000 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:51.000 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:51.000 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:38:51.000 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:51.001 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:51.001 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:51.001 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6802", 2023-10-26T01:38:51.001 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:38:51.001 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:51.002 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:51.002 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:51.002 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6803", 2023-10-26T01:38:51.002 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:38:51.002 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:51.002 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:51.003 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:51.003 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:38:51.003 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:51.003 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:51.003 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:51.004 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6806", 2023-10-26T01:38:51.004 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:38:51.004 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:51.004 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:51.004 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:51.005 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6807", 2023-10-26T01:38:51.005 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:38:51.005 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:51.005 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:51.005 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:51.005 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:38:51.006 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:51.006 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:51.006 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:51.006 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6804", 2023-10-26T01:38:51.006 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:38:51.007 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:51.007 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:51.007 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:51.007 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6805", 2023-10-26T01:38:51.007 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:38:51.008 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:51.008 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:51.008 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:51.008 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6801/2520460697", 2023-10-26T01:38:51.008 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6803/2520460697", 2023-10-26T01:38:51.009 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6807/2520460697", 2023-10-26T01:38:51.009 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6805/2520460697", 2023-10-26T01:38:51.009 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:38:51.009 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:38:51.009 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:38:51.010 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:51.010 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:51.010 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:51.010 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 7, 2023-10-26T01:38:51.010 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:38:51.010 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:38:51.011 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:38:51.011 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:38:51.011 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:38:51.011 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:38:51.011 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:38:51.012 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:38:51.012 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 132, 2023-10-26T01:38:51.012 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:38:51.012 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:38:51.012 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:38:51.013 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:51.013 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:51.013 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:51.013 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6821", 2023-10-26T01:38:51.013 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:38:51.013 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:51.014 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:51.014 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:51.014 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6822", 2023-10-26T01:38:51.014 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:38:51.014 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:51.015 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:51.015 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:51.015 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:38:51.015 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:51.015 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:51.016 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:51.016 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6824", 2023-10-26T01:38:51.016 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:38:51.016 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:51.016 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:51.017 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:51.017 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6826", 2023-10-26T01:38:51.017 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:38:51.017 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:51.017 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:51.018 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:51.018 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:38:51.018 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:51.018 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:51.018 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:51.019 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6830", 2023-10-26T01:38:51.019 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:38:51.019 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:51.019 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:51.019 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:51.020 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6831", 2023-10-26T01:38:51.020 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:38:51.020 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:51.020 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:51.020 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:51.021 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:38:51.021 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:51.021 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:51.021 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:51.021 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6827", 2023-10-26T01:38:51.022 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:38:51.022 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:51.022 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:51.022 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:51.022 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6829", 2023-10-26T01:38:51.022 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:38:51.023 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:51.023 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:51.023 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:51.023 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6822/1115930619", 2023-10-26T01:38:51.023 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6826/1115930619", 2023-10-26T01:38:51.024 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6831/1115930619", 2023-10-26T01:38:51.024 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6829/1115930619", 2023-10-26T01:38:51.024 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:38:51.024 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:38:51.024 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:38:51.025 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:51.025 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:51.025 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:38:51.025 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_xinfo": [ 2023-10-26T01:38:51.025 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:51.026 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 0, 2023-10-26T01:38:51.026 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:38:51.026 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:38:51.026 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:38:51.026 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:38:51.026 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:38:51.027 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.889675+0000", 2023-10-26T01:38:51.027 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:38:51.027 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:51.027 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:51.027 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 1, 2023-10-26T01:38:51.028 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:38:51.028 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:38:51.028 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:38:51.028 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:38:51.028 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:38:51.029 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.864157+0000", 2023-10-26T01:38:51.029 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:38:51.029 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:51.029 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:51.029 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 2, 2023-10-26T01:38:51.030 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:38:51.030 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:38:51.030 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:38:51.030 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:38:51.030 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:38:51.031 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.876919+0000", 2023-10-26T01:38:51.031 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:38:51.031 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:51.031 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:51.031 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 3, 2023-10-26T01:38:51.032 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:38:51.032 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:38:51.032 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:38:51.032 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:38:51.032 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:38:51.032 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.974541+0000", 2023-10-26T01:38:51.033 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:38:51.033 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:51.033 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:51.033 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 4, 2023-10-26T01:38:51.033 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:38:51.034 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:38:51.034 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:38:51.034 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:38:51.034 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:38:51.034 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.969696+0000", 2023-10-26T01:38:51.035 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:38:51.035 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:51.035 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:51.035 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 5, 2023-10-26T01:38:51.035 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:38:51.035 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:38:51.036 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:38:51.036 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:38:51.036 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:38:51.036 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.859703+0000", 2023-10-26T01:38:51.036 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:38:51.037 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:51.037 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:51.037 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 6, 2023-10-26T01:38:51.037 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:38:51.037 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:38:51.038 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:38:51.038 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:38:51.038 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:38:51.038 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.867917+0000", 2023-10-26T01:38:51.038 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:38:51.038 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:51.039 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:51.039 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 7, 2023-10-26T01:38:51.039 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:38:51.039 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:38:51.039 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:38:51.040 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:38:51.040 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:38:51.040 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.898605+0000", 2023-10-26T01:38:51.040 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:38:51.040 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:51.040 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:38:51.041 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_upmap": [], 2023-10-26T01:38:51.041 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_upmap_items": [ 2023-10-26T01:38:51.041 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:51.041 INFO:teuthology.orchestra.run.smithi169.stdout: "pgid": "17.e", 2023-10-26T01:38:51.041 INFO:teuthology.orchestra.run.smithi169.stdout: "mappings": [ 2023-10-26T01:38:51.042 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:51.042 INFO:teuthology.orchestra.run.smithi169.stdout: "from": 0, 2023-10-26T01:38:51.042 INFO:teuthology.orchestra.run.smithi169.stdout: "to": 5 2023-10-26T01:38:51.042 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:51.042 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:51.043 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:51.043 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:51.043 INFO:teuthology.orchestra.run.smithi169.stdout: "pgid": "17.21", 2023-10-26T01:38:51.043 INFO:teuthology.orchestra.run.smithi169.stdout: "mappings": [ 2023-10-26T01:38:51.043 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:51.044 INFO:teuthology.orchestra.run.smithi169.stdout: "from": 0, 2023-10-26T01:38:51.044 INFO:teuthology.orchestra.run.smithi169.stdout: "to": 2 2023-10-26T01:38:51.044 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:51.044 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:51.044 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:51.044 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:38:51.045 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_upmap_primaries": [], 2023-10-26T01:38:51.045 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_temp": [], 2023-10-26T01:38:51.045 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_temp": [], 2023-10-26T01:38:51.045 INFO:teuthology.orchestra.run.smithi169.stdout: "blocklist": {}, 2023-10-26T01:38:51.045 INFO:teuthology.orchestra.run.smithi169.stdout: "range_blocklist": {}, 2023-10-26T01:38:51.046 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profiles": { 2023-10-26T01:38:51.046 INFO:teuthology.orchestra.run.smithi169.stdout: "default": { 2023-10-26T01:38:51.046 INFO:teuthology.orchestra.run.smithi169.stdout: "crush-failure-domain": "osd", 2023-10-26T01:38:51.046 INFO:teuthology.orchestra.run.smithi169.stdout: "k": "2", 2023-10-26T01:38:51.046 INFO:teuthology.orchestra.run.smithi169.stdout: "m": "1", 2023-10-26T01:38:51.046 INFO:teuthology.orchestra.run.smithi169.stdout: "plugin": "jerasure", 2023-10-26T01:38:51.047 INFO:teuthology.orchestra.run.smithi169.stdout: "technique": "reed_sol_van" 2023-10-26T01:38:51.047 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:51.047 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:51.047 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps_queue": [], 2023-10-26T01:38:51.047 INFO:teuthology.orchestra.run.smithi169.stdout: "new_removed_snaps": [], 2023-10-26T01:38:51.048 INFO:teuthology.orchestra.run.smithi169.stdout: "new_purged_snaps": [], 2023-10-26T01:38:51.048 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_node_flags": {}, 2023-10-26T01:38:51.048 INFO:teuthology.orchestra.run.smithi169.stdout: "device_class_flags": {}, 2023-10-26T01:38:51.048 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_mode": { 2023-10-26T01:38:51.048 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_mode_enabled": false, 2023-10-26T01:38:51.049 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_bucket_count": 0, 2023-10-26T01:38:51.049 INFO:teuthology.orchestra.run.smithi169.stdout: "degraded_stretch_mode": 0, 2023-10-26T01:38:51.049 INFO:teuthology.orchestra.run.smithi169.stdout: "recovering_stretch_mode": 0, 2023-10-26T01:38:51.049 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_mode_bucket": 0 2023-10-26T01:38:51.049 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:51.049 INFO:teuthology.orchestra.run.smithi169.stdout:} 2023-10-26T01:38:51.051 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:50.820+0000 7fb886a4f640 1 -- 172.21.15.169:0/3579237634 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb86006ae40 msgr2=0x7fb86006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:51.052 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:50.820+0000 7fb886a4f640 1 --2- 172.21.15.169:0/3579237634 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb86006ae40 0x7fb86006d300 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fb870036000 tx=0x7fb870004ab0 comp rx=0 tx=0).stop 2023-10-26T01:38:51.052 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:50.820+0000 7fb886a4f640 1 -- 172.21.15.169:0/3579237634 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb88014e790 msgr2=0x7fb8800aff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:51.052 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:50.820+0000 7fb886a4f640 1 --2- 172.21.15.169:0/3579237634 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb88014e790 0x7fb8800aff10 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7fb868009800 tx=0x7fb868009cc0 comp rx=0 tx=0).stop 2023-10-26T01:38:51.052 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:50.820+0000 7fb886a4f640 1 -- 172.21.15.169:0/3579237634 shutdown_connections 2023-10-26T01:38:51.052 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:50.820+0000 7fb886a4f640 1 --2- 172.21.15.169:0/3579237634 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb86006ae40 0x7fb86006d300 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:51.052 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:50.820+0000 7fb886a4f640 1 --2- 172.21.15.169:0/3579237634 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb88014e790 0x7fb8800aff10 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:51.053 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:50.820+0000 7fb886a4f640 1 --2- 172.21.15.169:0/3579237634 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb880158850 0x7fb8800b0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:51.053 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:50.820+0000 7fb886a4f640 1 --2- 172.21.15.169:0/3579237634 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb8800cdc60 0x7fb8801546a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:51.053 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:50.820+0000 7fb886a4f640 1 -- 172.21.15.169:0/3579237634 >> 172.21.15.169:0/3579237634 conn(0x7fb880069930 msgr2=0x7fb8801466c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:51.053 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:50.820+0000 7fb886a4f640 1 -- 172.21.15.169:0/3579237634 shutdown_connections 2023-10-26T01:38:51.053 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:50.820+0000 7fb886a4f640 1 -- 172.21.15.169:0/3579237634 wait complete. 2023-10-26T01:38:51.054 DEBUG:tasks.ceph_test_case:wait_until_true: success in 10s and 0 retries 2023-10-26T01:38:51.055 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:38:51.055 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:38:51.116 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:38:51.117 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=50 seek=16) 2023-10-26T01:38:51.529 INFO:teuthology.orchestra.run.smithi169.stderr:dd: fdatasync failed for 'large_file_b': No space left on device 2023-10-26T01:38:51.529 INFO:teuthology.orchestra.run.smithi169.stderr:50+0 records in 2023-10-26T01:38:51.529 INFO:teuthology.orchestra.run.smithi169.stderr:50+0 records out 2023-10-26T01:38:51.529 INFO:teuthology.orchestra.run.smithi169.stderr:52428800 bytes (52 MB, 50 MiB) copied, 0.350701 s, 149 MB/s 2023-10-26T01:38:51.530 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-26T01:38:51.531 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-26T01:38:51.742 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:51.739+0000 7f029d7e2640 1 -- 172.21.15.169:0/208793970 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02980cde30 msgr2=0x7f0298159e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:51.742 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:51.739+0000 7f029d7e2640 1 --2- 172.21.15.169:0/208793970 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02980cde30 0x7f0298159e10 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f029000afd0 tx=0x7f029002cd80 comp rx=0 tx=0).stop 2023-10-26T01:38:51.743 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:51.739+0000 7f029d7e2640 1 -- 172.21.15.169:0/208793970 shutdown_connections 2023-10-26T01:38:51.743 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:51.739+0000 7f029d7e2640 1 --2- 172.21.15.169:0/208793970 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02980cde30 0x7f0298159e10 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:51.743 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:51.739+0000 7f029d7e2640 1 --2- 172.21.15.169:0/208793970 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f029814e790 0x7f02980aab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:51.743 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:51.739+0000 7f029d7e2640 1 -- 172.21.15.169:0/208793970 >> 172.21.15.169:0/208793970 conn(0x7f0298069930 msgr2=0x7f0298069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:51.743 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:51.739+0000 7f029d7e2640 1 -- 172.21.15.169:0/208793970 shutdown_connections 2023-10-26T01:38:51.744 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:51.739+0000 7f029d7e2640 1 -- 172.21.15.169:0/208793970 wait complete. 2023-10-26T01:38:51.744 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:51.741+0000 7f029d7e2640 1 Processor -- start 2023-10-26T01:38:51.744 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:51.741+0000 7f029d7e2640 1 -- start start 2023-10-26T01:38:51.746 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:51.743+0000 7f029d7e2640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02980cde30 0x7f0298150bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:51.746 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:51.743+0000 7f029d7e2640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f029814e790 0x7f02981571e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:51.746 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:51.743+0000 7f029d7e2640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0298158850 0x7f029814ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:51.747 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:51.743+0000 7f029d7e2640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02981527c0 con 0x7f02980cde30 2023-10-26T01:38:51.747 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:51.743+0000 7f029d7e2640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0298152aa0 con 0x7f029814e790 2023-10-26T01:38:51.747 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:51.743+0000 7f029d7e2640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0298152d80 con 0x7f0298158850 2023-10-26T01:38:51.747 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:51.743+0000 7f0296ffd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02980cde30 0x7f0298150bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:51.747 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:51.743+0000 7f0296ffd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02980cde30 0x7f0298150bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:53522/0 (socket says 172.21.15.169:53522) 2023-10-26T01:38:51.748 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:51.743+0000 7f0296ffd640 1 -- 172.21.15.169:0/2520678095 learned_addr learned my addr 172.21.15.169:0/2520678095 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:38:51.748 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:51.743+0000 7f0296ffd640 1 -- 172.21.15.169:0/2520678095 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0298158850 msgr2=0x7f029814ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:51.748 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:51.743+0000 7f0296ffd640 1 --2- 172.21.15.169:0/2520678095 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0298158850 0x7f029814ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:51.748 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:51.743+0000 7f0296ffd640 1 -- 172.21.15.169:0/2520678095 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f029814e790 msgr2=0x7f02981571e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:51.749 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:51.743+0000 7f0296ffd640 1 --2- 172.21.15.169:0/2520678095 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f029814e790 0x7f02981571e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:51.749 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:51.743+0000 7f0296ffd640 1 -- 172.21.15.169:0/2520678095 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0290007490 con 0x7f02980cde30 2023-10-26T01:38:51.750 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:51.744+0000 7f0296ffd640 1 --2- 172.21.15.169:0/2520678095 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02980cde30 0x7f0298150bb0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f028800d770 tx=0x7f028800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:51.750 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:51.744+0000 7f027f7fe640 1 -- 172.21.15.169:0/2520678095 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0288009c30 con 0x7f02980cde30 2023-10-26T01:38:51.751 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:51.744+0000 7f027f7fe640 1 -- 172.21.15.169:0/2520678095 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f028800de80 con 0x7f02980cde30 2023-10-26T01:38:51.751 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:51.744+0000 7f027f7fe640 1 -- 172.21.15.169:0/2520678095 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0288005280 con 0x7f02980cde30 2023-10-26T01:38:51.751 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:51.744+0000 7f029d7e2640 1 -- 172.21.15.169:0/2520678095 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0298153060 con 0x7f02980cde30 2023-10-26T01:38:51.751 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:51.744+0000 7f029d7e2640 1 -- 172.21.15.169:0/2520678095 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02981510f0 con 0x7f02980cde30 2023-10-26T01:38:51.752 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:51.746+0000 7f029d7e2640 1 -- 172.21.15.169:0/2520678095 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02981512b0 con 0x7f02980cde30 2023-10-26T01:38:51.752 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:51.746+0000 7f027f7fe640 1 -- 172.21.15.169:0/2520678095 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f02880053e0 con 0x7f02980cde30 2023-10-26T01:38:51.752 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:51.747+0000 7f027f7fe640 1 --2- 172.21.15.169:0/2520678095 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f028406ae40 0x7f028406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:51.752 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:51.747+0000 7f027f7fe640 1 -- 172.21.15.169:0/2520678095 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f02880898c0 con 0x7f02980cde30 2023-10-26T01:38:51.752 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:51.748+0000 7f027f7fe640 1 -- 172.21.15.169:0/2520678095 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=139}) v3 -- 0x7f0284072a30 con 0x7f02980cde30 2023-10-26T01:38:51.753 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:51.748+0000 7f02967fc640 1 --2- 172.21.15.169:0/2520678095 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f028406ae40 0x7f028406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:51.753 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:51.749+0000 7f02967fc640 1 --2- 172.21.15.169:0/2520678095 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f028406ae40 0x7f028406d300 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f028c000f60 tx=0x7f028c0079b0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:51.754 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:51.752+0000 7f027f7fe640 1 -- 172.21.15.169:0/2520678095 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0288017020 con 0x7f02980cde30 2023-10-26T01:38:51.945 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:51.942+0000 7f029d7e2640 1 -- 172.21.15.169:0/2520678095 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f029804db10 con 0x7f02980cde30 2023-10-26T01:38:51.947 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:51.944+0000 7f027f7fe640 1 -- 172.21.15.169:0/2520678095 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v138) v1 ==== 81+0+34469 (secure 0 0 0) 0x7f0288020070 con 0x7f02980cde30 2023-10-26T01:38:51.947 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:38:51.948 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2023-10-26T01:38:51.948 INFO:teuthology.orchestra.run.smithi169.stdout: "epoch": 138, 2023-10-26T01:38:51.948 INFO:teuthology.orchestra.run.smithi169.stdout: "fsid": "30203b28-ed8a-44e8-a9a6-a9bf25f980f1", 2023-10-26T01:38:51.948 INFO:teuthology.orchestra.run.smithi169.stdout: "created": "2023-10-26T01:30:22.897155+0000", 2023-10-26T01:38:51.949 INFO:teuthology.orchestra.run.smithi169.stdout: "modified": "2023-10-26T01:38:48.057101+0000", 2023-10-26T01:38:51.949 INFO:teuthology.orchestra.run.smithi169.stdout: "last_up_change": "2023-10-26T01:30:31.875281+0000", 2023-10-26T01:38:51.949 INFO:teuthology.orchestra.run.smithi169.stdout: "last_in_change": "2023-10-26T01:30:27.637941+0000", 2023-10-26T01:38:51.949 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-26T01:38:51.949 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_num": 5799936, 2023-10-26T01:38:51.950 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_set": [ 2023-10-26T01:38:51.950 INFO:teuthology.orchestra.run.smithi169.stdout: "pglog_hardlimit", 2023-10-26T01:38:51.950 INFO:teuthology.orchestra.run.smithi169.stdout: "purged_snapdirs", 2023-10-26T01:38:51.950 INFO:teuthology.orchestra.run.smithi169.stdout: "recovery_deletes", 2023-10-26T01:38:51.951 INFO:teuthology.orchestra.run.smithi169.stdout: "sortbitwise" 2023-10-26T01:38:51.951 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:38:51.951 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_version": 4, 2023-10-26T01:38:51.951 INFO:teuthology.orchestra.run.smithi169.stdout: "full_ratio": 0.69999998807907104, 2023-10-26T01:38:51.952 INFO:teuthology.orchestra.run.smithi169.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-26T01:38:51.952 INFO:teuthology.orchestra.run.smithi169.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-26T01:38:51.952 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_snapshot": "", 2023-10-26T01:38:51.952 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_max": 17, 2023-10-26T01:38:51.952 INFO:teuthology.orchestra.run.smithi169.stdout: "max_osd": 8, 2023-10-26T01:38:51.953 INFO:teuthology.orchestra.run.smithi169.stdout: "require_min_compat_client": "luminous", 2023-10-26T01:38:51.953 INFO:teuthology.orchestra.run.smithi169.stdout: "min_compat_client": "luminous", 2023-10-26T01:38:51.953 INFO:teuthology.orchestra.run.smithi169.stdout: "require_osd_release": "reef", 2023-10-26T01:38:51.953 INFO:teuthology.orchestra.run.smithi169.stdout: "allow_crimson": false, 2023-10-26T01:38:51.953 INFO:teuthology.orchestra.run.smithi169.stdout: "pools": [ 2023-10-26T01:38:51.954 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:51.954 INFO:teuthology.orchestra.run.smithi169.stdout: "pool": 1, 2023-10-26T01:38:51.954 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_name": ".mgr", 2023-10-26T01:38:51.954 INFO:teuthology.orchestra.run.smithi169.stdout: "create_time": "2023-10-26T01:30:32.200808+0000", 2023-10-26T01:38:51.955 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": 1, 2023-10-26T01:38:51.955 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_names": "hashpspool", 2023-10-26T01:38:51.955 INFO:teuthology.orchestra.run.smithi169.stdout: "type": 1, 2023-10-26T01:38:51.955 INFO:teuthology.orchestra.run.smithi169.stdout: "size": 2, 2023-10-26T01:38:51.955 INFO:teuthology.orchestra.run.smithi169.stdout: "min_size": 1, 2023-10-26T01:38:51.956 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_rule": 0, 2023-10-26T01:38:51.956 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_count": 0, 2023-10-26T01:38:51.956 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_target": 0, 2023-10-26T01:38:51.956 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T01:38:51.956 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T01:38:51.957 INFO:teuthology.orchestra.run.smithi169.stdout: "object_hash": 2, 2023-10-26T01:38:51.957 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_mode": "off", 2023-10-26T01:38:51.957 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num": 1, 2023-10-26T01:38:51.957 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num": 1, 2023-10-26T01:38:51.958 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num_target": 1, 2023-10-26T01:38:51.958 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_target": 1, 2023-10-26T01:38:51.958 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_pending": 1, 2023-10-26T01:38:51.958 INFO:teuthology.orchestra.run.smithi169.stdout: "last_pg_merge_meta": { 2023-10-26T01:38:51.958 INFO:teuthology.orchestra.run.smithi169.stdout: "source_pgid": "0.0", 2023-10-26T01:38:51.959 INFO:teuthology.orchestra.run.smithi169.stdout: "ready_epoch": 0, 2023-10-26T01:38:51.959 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_started": 0, 2023-10-26T01:38:51.959 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_clean": 0, 2023-10-26T01:38:51.959 INFO:teuthology.orchestra.run.smithi169.stdout: "source_version": "0'0", 2023-10-26T01:38:51.960 INFO:teuthology.orchestra.run.smithi169.stdout: "target_version": "0'0" 2023-10-26T01:38:51.960 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:51.960 INFO:teuthology.orchestra.run.smithi169.stdout: "last_change": "109", 2023-10-26T01:38:51.960 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend": "0", 2023-10-26T01:38:51.960 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T01:38:51.961 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T01:38:51.961 INFO:teuthology.orchestra.run.smithi169.stdout: "auid": 0, 2023-10-26T01:38:51.961 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_mode": "selfmanaged", 2023-10-26T01:38:51.961 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_seq": 0, 2023-10-26T01:38:51.961 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_epoch": 0, 2023-10-26T01:38:51.962 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_snaps": [], 2023-10-26T01:38:51.962 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps": "[]", 2023-10-26T01:38:51.962 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_bytes": 0, 2023-10-26T01:38:51.962 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_objects": 0, 2023-10-26T01:38:51.962 INFO:teuthology.orchestra.run.smithi169.stdout: "tiers": [], 2023-10-26T01:38:51.963 INFO:teuthology.orchestra.run.smithi169.stdout: "tier_of": -1, 2023-10-26T01:38:51.963 INFO:teuthology.orchestra.run.smithi169.stdout: "read_tier": -1, 2023-10-26T01:38:51.963 INFO:teuthology.orchestra.run.smithi169.stdout: "write_tier": -1, 2023-10-26T01:38:51.963 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_mode": "none", 2023-10-26T01:38:51.963 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_bytes": 0, 2023-10-26T01:38:51.964 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_objects": 0, 2023-10-26T01:38:51.964 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T01:38:51.964 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T01:38:51.964 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T01:38:51.965 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_flush_age": 0, 2023-10-26T01:38:51.965 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_evict_age": 0, 2023-10-26T01:38:51.965 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profile": "", 2023-10-26T01:38:51.965 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_params": { 2023-10-26T01:38:51.965 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "none" 2023-10-26T01:38:51.966 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:51.966 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_period": 0, 2023-10-26T01:38:51.966 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_count": 0, 2023-10-26T01:38:51.966 INFO:teuthology.orchestra.run.smithi169.stdout: "use_gmt_hitset": true, 2023-10-26T01:38:51.966 INFO:teuthology.orchestra.run.smithi169.stdout: "min_read_recency_for_promote": 0, 2023-10-26T01:38:51.967 INFO:teuthology.orchestra.run.smithi169.stdout: "min_write_recency_for_promote": 0, 2023-10-26T01:38:51.967 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T01:38:51.967 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_search_last_n": 0, 2023-10-26T01:38:51.967 INFO:teuthology.orchestra.run.smithi169.stdout: "grade_table": [], 2023-10-26T01:38:51.968 INFO:teuthology.orchestra.run.smithi169.stdout: "stripe_width": 0, 2023-10-26T01:38:51.968 INFO:teuthology.orchestra.run.smithi169.stdout: "expected_num_objects": 0, 2023-10-26T01:38:51.968 INFO:teuthology.orchestra.run.smithi169.stdout: "fast_read": false, 2023-10-26T01:38:51.968 INFO:teuthology.orchestra.run.smithi169.stdout: "options": { 2023-10-26T01:38:51.968 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_max": 32, 2023-10-26T01:38:51.969 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_min": 1 2023-10-26T01:38:51.969 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:51.969 INFO:teuthology.orchestra.run.smithi169.stdout: "application_metadata": { 2023-10-26T01:38:51.969 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": {} 2023-10-26T01:38:51.970 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:51.970 INFO:teuthology.orchestra.run.smithi169.stdout: "read_balance": { 2023-10-26T01:38:51.970 INFO:teuthology.orchestra.run.smithi169.stdout: "score_acting": 8, 2023-10-26T01:38:51.970 INFO:teuthology.orchestra.run.smithi169.stdout: "score_stable": 8, 2023-10-26T01:38:51.970 INFO:teuthology.orchestra.run.smithi169.stdout: "optimal_score": 0.25, 2023-10-26T01:38:51.971 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_acting": 2, 2023-10-26T01:38:51.971 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_stable": 2, 2023-10-26T01:38:51.971 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity_weighted": 1, 2023-10-26T01:38:51.971 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity": 1, 2023-10-26T01:38:51.971 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity_weighted": 1 2023-10-26T01:38:51.972 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:51.972 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:51.972 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:51.972 INFO:teuthology.orchestra.run.smithi169.stdout: "pool": 16, 2023-10-26T01:38:51.972 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_name": "cephfs_metadata", 2023-10-26T01:38:51.973 INFO:teuthology.orchestra.run.smithi169.stdout: "create_time": "2023-10-26T01:38:18.617877+0000", 2023-10-26T01:38:51.973 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": 1, 2023-10-26T01:38:51.973 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_names": "hashpspool", 2023-10-26T01:38:51.973 INFO:teuthology.orchestra.run.smithi169.stdout: "type": 1, 2023-10-26T01:38:51.974 INFO:teuthology.orchestra.run.smithi169.stdout: "size": 2, 2023-10-26T01:38:51.974 INFO:teuthology.orchestra.run.smithi169.stdout: "min_size": 1, 2023-10-26T01:38:51.974 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_rule": 0, 2023-10-26T01:38:51.974 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_count": 0, 2023-10-26T01:38:51.974 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_target": 0, 2023-10-26T01:38:51.975 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T01:38:51.975 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T01:38:51.975 INFO:teuthology.orchestra.run.smithi169.stdout: "object_hash": 2, 2023-10-26T01:38:51.975 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_mode": "off", 2023-10-26T01:38:51.975 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num": 32, 2023-10-26T01:38:51.976 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num": 32, 2023-10-26T01:38:51.976 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num_target": 32, 2023-10-26T01:38:51.976 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_target": 32, 2023-10-26T01:38:51.976 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_pending": 32, 2023-10-26T01:38:51.977 INFO:teuthology.orchestra.run.smithi169.stdout: "last_pg_merge_meta": { 2023-10-26T01:38:51.977 INFO:teuthology.orchestra.run.smithi169.stdout: "source_pgid": "0.0", 2023-10-26T01:38:51.977 INFO:teuthology.orchestra.run.smithi169.stdout: "ready_epoch": 0, 2023-10-26T01:38:51.977 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_started": 0, 2023-10-26T01:38:51.977 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_clean": 0, 2023-10-26T01:38:51.978 INFO:teuthology.orchestra.run.smithi169.stdout: "source_version": "0'0", 2023-10-26T01:38:51.978 INFO:teuthology.orchestra.run.smithi169.stdout: "target_version": "0'0" 2023-10-26T01:38:51.978 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:51.978 INFO:teuthology.orchestra.run.smithi169.stdout: "last_change": "133", 2023-10-26T01:38:51.979 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend": "0", 2023-10-26T01:38:51.979 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T01:38:51.979 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T01:38:51.979 INFO:teuthology.orchestra.run.smithi169.stdout: "auid": 0, 2023-10-26T01:38:51.979 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_mode": "selfmanaged", 2023-10-26T01:38:51.980 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_seq": 0, 2023-10-26T01:38:51.980 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_epoch": 0, 2023-10-26T01:38:51.980 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_snaps": [], 2023-10-26T01:38:51.980 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps": "[]", 2023-10-26T01:38:51.980 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_bytes": 0, 2023-10-26T01:38:51.981 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_objects": 0, 2023-10-26T01:38:51.981 INFO:teuthology.orchestra.run.smithi169.stdout: "tiers": [], 2023-10-26T01:38:51.981 INFO:teuthology.orchestra.run.smithi169.stdout: "tier_of": -1, 2023-10-26T01:38:51.981 INFO:teuthology.orchestra.run.smithi169.stdout: "read_tier": -1, 2023-10-26T01:38:51.982 INFO:teuthology.orchestra.run.smithi169.stdout: "write_tier": -1, 2023-10-26T01:38:51.982 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_mode": "none", 2023-10-26T01:38:51.982 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_bytes": 0, 2023-10-26T01:38:51.982 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_objects": 0, 2023-10-26T01:38:51.982 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T01:38:51.983 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T01:38:51.983 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T01:38:51.983 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_flush_age": 0, 2023-10-26T01:38:51.983 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_evict_age": 0, 2023-10-26T01:38:51.984 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profile": "", 2023-10-26T01:38:51.984 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_params": { 2023-10-26T01:38:51.984 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "none" 2023-10-26T01:38:51.984 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:51.984 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_period": 0, 2023-10-26T01:38:51.985 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_count": 0, 2023-10-26T01:38:51.985 INFO:teuthology.orchestra.run.smithi169.stdout: "use_gmt_hitset": true, 2023-10-26T01:38:51.985 INFO:teuthology.orchestra.run.smithi169.stdout: "min_read_recency_for_promote": 0, 2023-10-26T01:38:51.985 INFO:teuthology.orchestra.run.smithi169.stdout: "min_write_recency_for_promote": 0, 2023-10-26T01:38:51.985 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T01:38:51.986 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_search_last_n": 0, 2023-10-26T01:38:51.986 INFO:teuthology.orchestra.run.smithi169.stdout: "grade_table": [], 2023-10-26T01:38:51.986 INFO:teuthology.orchestra.run.smithi169.stdout: "stripe_width": 0, 2023-10-26T01:38:51.986 INFO:teuthology.orchestra.run.smithi169.stdout: "expected_num_objects": 0, 2023-10-26T01:38:51.987 INFO:teuthology.orchestra.run.smithi169.stdout: "fast_read": false, 2023-10-26T01:38:51.987 INFO:teuthology.orchestra.run.smithi169.stdout: "options": { 2023-10-26T01:38:51.987 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_bias": 4, 2023-10-26T01:38:51.987 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_min": 16, 2023-10-26T01:38:51.987 INFO:teuthology.orchestra.run.smithi169.stdout: "recovery_priority": 5 2023-10-26T01:38:51.988 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:51.988 INFO:teuthology.orchestra.run.smithi169.stdout: "application_metadata": { 2023-10-26T01:38:51.988 INFO:teuthology.orchestra.run.smithi169.stdout: "cephfs": { 2023-10-26T01:38:51.988 INFO:teuthology.orchestra.run.smithi169.stdout: "metadata": "cephfs" 2023-10-26T01:38:51.988 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:51.989 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:51.989 INFO:teuthology.orchestra.run.smithi169.stdout: "read_balance": { 2023-10-26T01:38:51.989 INFO:teuthology.orchestra.run.smithi169.stdout: "score_acting": 1.75, 2023-10-26T01:38:51.989 INFO:teuthology.orchestra.run.smithi169.stdout: "score_stable": 1.75, 2023-10-26T01:38:51.989 INFO:teuthology.orchestra.run.smithi169.stdout: "optimal_score": 1, 2023-10-26T01:38:51.990 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_acting": 1.75, 2023-10-26T01:38:51.990 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_stable": 1.75, 2023-10-26T01:38:51.990 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity_weighted": 1, 2023-10-26T01:38:51.990 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity": 1, 2023-10-26T01:38:51.991 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity_weighted": 1 2023-10-26T01:38:51.991 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:51.991 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:51.991 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:51.991 INFO:teuthology.orchestra.run.smithi169.stdout: "pool": 17, 2023-10-26T01:38:51.992 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_name": "cephfs_data", 2023-10-26T01:38:51.992 INFO:teuthology.orchestra.run.smithi169.stdout: "create_time": "2023-10-26T01:38:19.793041+0000", 2023-10-26T01:38:51.992 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": 1027, 2023-10-26T01:38:51.992 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_names": "hashpspool,full,full_quota", 2023-10-26T01:38:51.993 INFO:teuthology.orchestra.run.smithi169.stdout: "type": 1, 2023-10-26T01:38:51.993 INFO:teuthology.orchestra.run.smithi169.stdout: "size": 2, 2023-10-26T01:38:51.993 INFO:teuthology.orchestra.run.smithi169.stdout: "min_size": 1, 2023-10-26T01:38:51.993 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_rule": 0, 2023-10-26T01:38:51.993 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_count": 0, 2023-10-26T01:38:51.994 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_target": 0, 2023-10-26T01:38:51.994 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T01:38:51.994 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T01:38:51.994 INFO:teuthology.orchestra.run.smithi169.stdout: "object_hash": 2, 2023-10-26T01:38:51.995 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_mode": "off", 2023-10-26T01:38:51.995 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num": 64, 2023-10-26T01:38:51.995 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num": 64, 2023-10-26T01:38:51.995 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num_target": 64, 2023-10-26T01:38:51.995 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_target": 64, 2023-10-26T01:38:51.996 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_pending": 64, 2023-10-26T01:38:51.996 INFO:teuthology.orchestra.run.smithi169.stdout: "last_pg_merge_meta": { 2023-10-26T01:38:51.996 INFO:teuthology.orchestra.run.smithi169.stdout: "source_pgid": "0.0", 2023-10-26T01:38:51.996 INFO:teuthology.orchestra.run.smithi169.stdout: "ready_epoch": 0, 2023-10-26T01:38:51.996 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_started": 0, 2023-10-26T01:38:51.997 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_clean": 0, 2023-10-26T01:38:51.997 INFO:teuthology.orchestra.run.smithi169.stdout: "source_version": "0'0", 2023-10-26T01:38:51.997 INFO:teuthology.orchestra.run.smithi169.stdout: "target_version": "0'0" 2023-10-26T01:38:51.997 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:51.997 INFO:teuthology.orchestra.run.smithi169.stdout: "last_change": "138", 2023-10-26T01:38:51.998 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend": "0", 2023-10-26T01:38:51.998 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T01:38:51.998 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T01:38:51.998 INFO:teuthology.orchestra.run.smithi169.stdout: "auid": 0, 2023-10-26T01:38:51.999 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_mode": "selfmanaged", 2023-10-26T01:38:51.999 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_seq": 0, 2023-10-26T01:38:51.999 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_epoch": 0, 2023-10-26T01:38:51.999 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_snaps": [], 2023-10-26T01:38:51.999 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps": "[]", 2023-10-26T01:38:51.999 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_bytes": 33554432, 2023-10-26T01:38:52.000 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_objects": 0, 2023-10-26T01:38:52.000 INFO:teuthology.orchestra.run.smithi169.stdout: "tiers": [], 2023-10-26T01:38:52.000 INFO:teuthology.orchestra.run.smithi169.stdout: "tier_of": -1, 2023-10-26T01:38:52.000 INFO:teuthology.orchestra.run.smithi169.stdout: "read_tier": -1, 2023-10-26T01:38:52.001 INFO:teuthology.orchestra.run.smithi169.stdout: "write_tier": -1, 2023-10-26T01:38:52.001 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_mode": "none", 2023-10-26T01:38:52.001 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_bytes": 0, 2023-10-26T01:38:52.001 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_objects": 0, 2023-10-26T01:38:52.001 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T01:38:52.002 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T01:38:52.002 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T01:38:52.002 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_flush_age": 0, 2023-10-26T01:38:52.002 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_evict_age": 0, 2023-10-26T01:38:52.002 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profile": "", 2023-10-26T01:38:52.003 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_params": { 2023-10-26T01:38:52.003 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "none" 2023-10-26T01:38:52.003 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:52.003 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_period": 0, 2023-10-26T01:38:52.003 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_count": 0, 2023-10-26T01:38:52.004 INFO:teuthology.orchestra.run.smithi169.stdout: "use_gmt_hitset": true, 2023-10-26T01:38:52.004 INFO:teuthology.orchestra.run.smithi169.stdout: "min_read_recency_for_promote": 0, 2023-10-26T01:38:52.004 INFO:teuthology.orchestra.run.smithi169.stdout: "min_write_recency_for_promote": 0, 2023-10-26T01:38:52.004 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T01:38:52.004 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_search_last_n": 0, 2023-10-26T01:38:52.005 INFO:teuthology.orchestra.run.smithi169.stdout: "grade_table": [], 2023-10-26T01:38:52.005 INFO:teuthology.orchestra.run.smithi169.stdout: "stripe_width": 0, 2023-10-26T01:38:52.005 INFO:teuthology.orchestra.run.smithi169.stdout: "expected_num_objects": 0, 2023-10-26T01:38:52.005 INFO:teuthology.orchestra.run.smithi169.stdout: "fast_read": false, 2023-10-26T01:38:52.005 INFO:teuthology.orchestra.run.smithi169.stdout: "options": { 2023-10-26T01:38:52.006 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_min": 64, 2023-10-26T01:38:52.006 INFO:teuthology.orchestra.run.smithi169.stdout: "target_size_ratio": 0.89999997615814209 2023-10-26T01:38:52.006 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:52.006 INFO:teuthology.orchestra.run.smithi169.stdout: "application_metadata": { 2023-10-26T01:38:52.007 INFO:teuthology.orchestra.run.smithi169.stdout: "cephfs": { 2023-10-26T01:38:52.007 INFO:teuthology.orchestra.run.smithi169.stdout: "data": "cephfs" 2023-10-26T01:38:52.007 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:52.007 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:52.008 INFO:teuthology.orchestra.run.smithi169.stdout: "read_balance": { 2023-10-26T01:38:52.008 INFO:teuthology.orchestra.run.smithi169.stdout: "score_acting": 1.6299999952316284, 2023-10-26T01:38:52.008 INFO:teuthology.orchestra.run.smithi169.stdout: "score_stable": 1.6299999952316284, 2023-10-26T01:38:52.008 INFO:teuthology.orchestra.run.smithi169.stdout: "optimal_score": 1, 2023-10-26T01:38:52.008 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_acting": 1.6299999952316284, 2023-10-26T01:38:52.009 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_stable": 1.6299999952316284, 2023-10-26T01:38:52.009 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity_weighted": 1, 2023-10-26T01:38:52.009 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity": 1, 2023-10-26T01:38:52.009 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity_weighted": 1 2023-10-26T01:38:52.009 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:52.010 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:52.010 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:38:52.010 INFO:teuthology.orchestra.run.smithi169.stdout: "osds": [ 2023-10-26T01:38:52.010 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:52.010 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 0, 2023-10-26T01:38:52.011 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:38:52.011 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:38:52.011 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:38:52.011 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:38:52.012 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:38:52.012 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:38:52.012 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:38:52.012 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:38:52.012 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 132, 2023-10-26T01:38:52.013 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:38:52.013 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:38:52.013 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:38:52.013 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:52.013 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:52.014 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:52.014 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6816", 2023-10-26T01:38:52.014 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:38:52.014 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:52.014 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:52.015 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:52.015 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6817", 2023-10-26T01:38:52.015 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:38:52.015 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:52.015 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:52.016 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:52.016 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:38:52.016 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:52.016 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:52.017 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:52.017 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6818", 2023-10-26T01:38:52.017 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:38:52.017 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:52.017 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:52.018 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:52.018 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6819", 2023-10-26T01:38:52.018 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:38:52.018 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:52.019 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:52.019 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:52.019 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:38:52.019 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:52.019 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:52.020 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:52.020 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6822", 2023-10-26T01:38:52.020 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:38:52.020 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:52.020 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:52.021 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:52.021 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6823", 2023-10-26T01:38:52.021 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:38:52.021 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:52.021 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:52.022 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:52.022 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:38:52.022 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:52.022 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:52.022 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:52.023 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6820", 2023-10-26T01:38:52.023 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:38:52.023 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:52.023 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:52.023 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:52.024 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6821", 2023-10-26T01:38:52.024 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:38:52.024 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:52.024 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:52.024 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:52.025 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6817/4204255864", 2023-10-26T01:38:52.025 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6819/4204255864", 2023-10-26T01:38:52.025 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6823/4204255864", 2023-10-26T01:38:52.025 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6821/4204255864", 2023-10-26T01:38:52.025 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:38:52.026 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:38:52.026 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:38:52.026 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:52.026 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:52.026 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:52.027 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 1, 2023-10-26T01:38:52.027 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:38:52.027 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:38:52.027 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:38:52.028 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:38:52.028 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:38:52.028 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:38:52.028 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:38:52.028 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:38:52.029 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 132, 2023-10-26T01:38:52.029 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:38:52.029 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:38:52.029 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:38:52.029 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:52.030 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:52.030 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:52.030 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6808", 2023-10-26T01:38:52.030 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:38:52.031 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:52.031 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:52.031 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:52.031 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6809", 2023-10-26T01:38:52.031 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:38:52.032 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:52.032 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:52.032 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:52.032 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:38:52.032 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:52.033 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:52.033 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:52.033 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6810", 2023-10-26T01:38:52.033 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:38:52.033 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:52.034 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:52.034 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:52.034 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6811", 2023-10-26T01:38:52.034 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:38:52.035 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:52.035 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:52.035 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:52.035 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:38:52.035 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:52.036 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:52.036 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:52.036 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6814", 2023-10-26T01:38:52.036 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:38:52.036 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:52.037 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:52.037 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:52.037 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6815", 2023-10-26T01:38:52.037 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:38:52.037 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:52.038 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:52.038 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:52.038 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:38:52.038 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:52.039 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:52.039 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:52.039 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6812", 2023-10-26T01:38:52.039 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:38:52.039 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:52.040 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:52.040 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:52.040 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6813", 2023-10-26T01:38:52.040 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:38:52.040 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:52.041 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:52.041 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:52.041 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6809/238206046", 2023-10-26T01:38:52.041 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6811/238206046", 2023-10-26T01:38:52.041 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6815/238206046", 2023-10-26T01:38:52.042 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6813/238206046", 2023-10-26T01:38:52.042 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:38:52.042 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:38:52.042 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:38:52.042 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:52.043 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:52.043 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:52.043 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 2, 2023-10-26T01:38:52.043 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:38:52.044 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:38:52.044 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:38:52.044 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:38:52.044 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:38:52.044 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:38:52.045 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:38:52.045 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:38:52.045 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 135, 2023-10-26T01:38:52.045 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:38:52.045 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:38:52.045 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:38:52.046 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:52.046 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:52.046 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:52.046 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6800", 2023-10-26T01:38:52.046 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:38:52.047 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:52.047 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:52.047 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:52.047 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6801", 2023-10-26T01:38:52.047 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:38:52.048 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:52.048 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:52.048 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:52.048 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:38:52.049 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:52.049 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:52.049 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:52.049 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6802", 2023-10-26T01:38:52.049 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:38:52.050 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:52.050 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:52.050 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:52.050 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6803", 2023-10-26T01:38:52.050 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:38:52.051 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:52.051 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:52.051 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:52.051 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:38:52.051 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:52.052 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:52.052 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:52.052 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6806", 2023-10-26T01:38:52.052 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:38:52.052 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:52.053 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:52.053 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:52.053 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6807", 2023-10-26T01:38:52.053 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:38:52.053 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:52.054 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:52.054 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:52.054 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:38:52.054 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:52.054 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:52.055 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:52.055 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6804", 2023-10-26T01:38:52.055 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:38:52.055 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:52.055 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:52.056 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:52.056 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6805", 2023-10-26T01:38:52.056 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:38:52.056 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:52.056 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:52.056 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:52.057 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6801/3215791846", 2023-10-26T01:38:52.057 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6803/3215791846", 2023-10-26T01:38:52.057 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6807/3215791846", 2023-10-26T01:38:52.057 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6805/3215791846", 2023-10-26T01:38:52.058 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:38:52.058 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:38:52.058 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:38:52.058 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:52.058 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:52.059 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:52.059 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 3, 2023-10-26T01:38:52.059 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:38:52.059 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:38:52.059 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:38:52.060 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:38:52.060 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:38:52.060 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:38:52.060 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:38:52.060 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 14, 2023-10-26T01:38:52.061 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 132, 2023-10-26T01:38:52.061 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:38:52.061 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:38:52.061 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:38:52.061 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:52.062 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:52.062 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:52.062 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6824", 2023-10-26T01:38:52.062 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:38:52.063 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:52.063 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:52.063 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:52.063 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6825", 2023-10-26T01:38:52.063 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:38:52.064 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:52.064 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:52.064 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:52.064 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:38:52.064 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:52.065 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:52.065 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:52.065 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6826", 2023-10-26T01:38:52.065 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:38:52.065 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:52.066 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:52.066 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:52.066 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6827", 2023-10-26T01:38:52.066 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:38:52.066 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:52.067 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:52.067 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:52.067 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:38:52.067 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:52.067 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:52.068 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:52.068 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6830", 2023-10-26T01:38:52.068 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:38:52.068 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:52.068 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:52.069 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:52.069 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6831", 2023-10-26T01:38:52.069 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:38:52.069 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:52.069 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:52.070 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:52.070 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:38:52.070 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:52.070 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:52.071 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:52.071 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6828", 2023-10-26T01:38:52.071 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:38:52.071 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:52.071 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:52.072 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:52.072 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6829", 2023-10-26T01:38:52.072 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:38:52.072 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:52.072 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:52.073 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:52.073 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6825/2256701226", 2023-10-26T01:38:52.073 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6827/2256701226", 2023-10-26T01:38:52.073 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6831/2256701226", 2023-10-26T01:38:52.074 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6829/2256701226", 2023-10-26T01:38:52.074 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:38:52.074 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:38:52.074 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:38:52.074 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:52.075 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:52.075 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:52.075 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 4, 2023-10-26T01:38:52.075 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:38:52.075 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:38:52.076 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:38:52.076 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:38:52.076 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:38:52.076 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:38:52.077 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:38:52.077 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:38:52.077 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 132, 2023-10-26T01:38:52.077 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:38:52.077 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:38:52.078 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:38:52.078 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:52.078 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:52.078 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:52.079 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6816", 2023-10-26T01:38:52.079 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:38:52.079 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:52.079 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:52.079 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:52.080 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6817", 2023-10-26T01:38:52.080 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:38:52.080 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:52.080 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:52.081 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:52.081 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:38:52.081 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:52.081 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:52.081 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:52.082 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6818", 2023-10-26T01:38:52.082 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:38:52.082 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:52.082 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:52.083 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:52.083 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6819", 2023-10-26T01:38:52.083 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:38:52.083 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:52.083 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:52.084 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:52.084 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:38:52.084 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:52.084 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:52.085 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:52.085 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6825", 2023-10-26T01:38:52.085 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:38:52.085 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:52.085 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:52.086 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:52.086 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6828", 2023-10-26T01:38:52.086 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:38:52.086 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:52.086 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:52.087 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:52.087 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:38:52.087 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:52.087 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:52.087 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:52.088 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6820", 2023-10-26T01:38:52.088 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:38:52.088 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:52.088 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:52.088 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:52.089 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6823", 2023-10-26T01:38:52.089 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:38:52.089 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:52.089 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:52.089 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:52.090 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6817/1082279874", 2023-10-26T01:38:52.090 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6819/1082279874", 2023-10-26T01:38:52.090 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6828/1082279874", 2023-10-26T01:38:52.090 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6823/1082279874", 2023-10-26T01:38:52.090 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:38:52.091 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:38:52.091 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:38:52.091 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:52.091 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:52.092 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:52.092 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 5, 2023-10-26T01:38:52.092 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:38:52.092 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:38:52.092 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:38:52.093 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:38:52.093 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:38:52.093 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:38:52.093 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:38:52.093 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:38:52.094 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 132, 2023-10-26T01:38:52.094 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:38:52.094 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:38:52.094 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:38:52.094 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:52.095 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:52.095 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:52.095 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6808", 2023-10-26T01:38:52.095 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:38:52.095 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:52.096 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:52.096 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:52.096 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6809", 2023-10-26T01:38:52.096 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:38:52.096 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:52.097 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:52.097 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:52.097 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:38:52.097 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:52.097 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:52.098 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:52.098 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6810", 2023-10-26T01:38:52.098 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:38:52.098 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:52.099 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:52.099 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:52.099 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6811", 2023-10-26T01:38:52.099 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:38:52.099 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:52.099 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:52.100 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:52.100 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:38:52.100 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:52.100 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:52.100 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:52.101 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6814", 2023-10-26T01:38:52.101 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:38:52.101 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:52.101 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:52.101 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:52.102 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6815", 2023-10-26T01:38:52.102 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:38:52.102 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:52.102 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:52.103 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:52.103 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:38:52.103 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:52.103 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:52.103 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:52.104 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6812", 2023-10-26T01:38:52.104 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:38:52.104 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:52.104 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:52.104 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:52.105 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6813", 2023-10-26T01:38:52.105 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:38:52.105 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:52.105 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:52.105 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:52.106 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6809/3335654351", 2023-10-26T01:38:52.106 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6811/3335654351", 2023-10-26T01:38:52.106 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6815/3335654351", 2023-10-26T01:38:52.106 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6813/3335654351", 2023-10-26T01:38:52.106 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:38:52.107 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:38:52.107 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:38:52.107 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:52.107 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:52.107 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:52.108 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 6, 2023-10-26T01:38:52.108 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:38:52.108 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:38:52.108 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:38:52.108 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:38:52.109 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:38:52.109 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:38:52.109 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:38:52.109 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:38:52.109 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 132, 2023-10-26T01:38:52.110 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:38:52.110 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:38:52.110 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:38:52.110 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:52.111 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:52.111 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:52.111 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6800", 2023-10-26T01:38:52.111 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:38:52.111 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:52.112 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:52.112 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:52.112 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6801", 2023-10-26T01:38:52.112 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:38:52.112 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:52.113 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:52.113 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:52.113 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:38:52.113 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:52.113 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:52.114 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:52.114 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6802", 2023-10-26T01:38:52.114 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:38:52.114 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:52.115 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:52.115 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:52.115 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6803", 2023-10-26T01:38:52.115 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:38:52.115 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:52.116 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:52.116 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:52.116 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:38:52.116 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:52.116 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:52.117 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:52.117 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6806", 2023-10-26T01:38:52.117 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:38:52.117 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:52.117 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:52.118 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:52.118 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6807", 2023-10-26T01:38:52.118 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:38:52.118 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:52.118 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:52.119 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:52.119 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:38:52.119 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:52.119 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:52.120 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:52.120 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6804", 2023-10-26T01:38:52.120 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:38:52.120 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:52.120 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:52.121 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:52.121 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6805", 2023-10-26T01:38:52.121 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:38:52.121 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:52.122 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:52.122 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:52.122 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6801/2520460697", 2023-10-26T01:38:52.122 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6803/2520460697", 2023-10-26T01:38:52.122 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6807/2520460697", 2023-10-26T01:38:52.123 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6805/2520460697", 2023-10-26T01:38:52.123 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:38:52.123 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:38:52.123 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:38:52.124 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:52.124 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:52.124 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:52.124 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 7, 2023-10-26T01:38:52.124 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:38:52.125 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:38:52.125 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:38:52.125 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:38:52.125 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:38:52.125 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:38:52.126 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:38:52.126 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:38:52.126 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 132, 2023-10-26T01:38:52.126 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:38:52.127 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:38:52.127 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:38:52.127 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:52.127 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:52.127 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:52.128 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6821", 2023-10-26T01:38:52.128 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:38:52.128 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:52.128 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:52.128 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:52.129 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6822", 2023-10-26T01:38:52.129 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:38:52.129 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:52.129 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:52.130 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:52.130 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:38:52.130 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:52.130 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:52.130 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:52.131 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6824", 2023-10-26T01:38:52.131 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:38:52.131 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:52.131 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:52.131 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:52.132 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6826", 2023-10-26T01:38:52.132 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:38:52.132 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:52.132 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:52.133 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:52.133 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:38:52.133 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:52.133 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:52.133 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:52.134 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6830", 2023-10-26T01:38:52.134 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:38:52.134 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:52.134 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:52.134 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:52.135 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6831", 2023-10-26T01:38:52.135 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:38:52.135 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:52.135 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:52.135 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:52.136 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:38:52.136 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:52.136 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:52.136 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:52.137 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6827", 2023-10-26T01:38:52.137 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:38:52.137 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:52.137 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:52.138 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:52.138 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6829", 2023-10-26T01:38:52.138 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:38:52.138 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:52.138 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:52.139 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:52.139 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6822/1115930619", 2023-10-26T01:38:52.139 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6826/1115930619", 2023-10-26T01:38:52.139 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6831/1115930619", 2023-10-26T01:38:52.139 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6829/1115930619", 2023-10-26T01:38:52.140 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:38:52.140 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:38:52.140 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:38:52.140 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:52.140 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:52.141 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:38:52.141 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_xinfo": [ 2023-10-26T01:38:52.141 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:52.141 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 0, 2023-10-26T01:38:52.141 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:38:52.142 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:38:52.142 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:38:52.142 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:38:52.142 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:38:52.143 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.889675+0000", 2023-10-26T01:38:52.143 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:38:52.143 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:52.143 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:52.143 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 1, 2023-10-26T01:38:52.145 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:38:52.145 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:38:52.145 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:38:52.146 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:38:52.146 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:38:52.146 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.864157+0000", 2023-10-26T01:38:52.146 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:38:52.146 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:52.147 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:52.147 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 2, 2023-10-26T01:38:52.147 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:38:52.147 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:38:52.148 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:38:52.148 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:38:52.148 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:38:52.148 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.876919+0000", 2023-10-26T01:38:52.148 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:38:52.148 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:52.149 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:52.149 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 3, 2023-10-26T01:38:52.149 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:38:52.149 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:38:52.149 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:38:52.150 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:38:52.150 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:38:52.150 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.974541+0000", 2023-10-26T01:38:52.150 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:38:52.151 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:52.151 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:52.151 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 4, 2023-10-26T01:38:52.151 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:38:52.151 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:38:52.152 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:38:52.152 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:38:52.152 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:38:52.152 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.969696+0000", 2023-10-26T01:38:52.152 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:38:52.153 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:52.153 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:52.153 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 5, 2023-10-26T01:38:52.153 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:38:52.153 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:38:52.154 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:38:52.154 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:38:52.154 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:38:52.154 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.859703+0000", 2023-10-26T01:38:52.155 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:38:52.155 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:52.155 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:52.155 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 6, 2023-10-26T01:38:52.155 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:38:52.156 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:38:52.156 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:38:52.156 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:38:52.156 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:38:52.156 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.867917+0000", 2023-10-26T01:38:52.157 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:38:52.157 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:52.157 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:52.157 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 7, 2023-10-26T01:38:52.158 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:38:52.158 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:38:52.158 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:38:52.158 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:38:52.158 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:38:52.159 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.898605+0000", 2023-10-26T01:38:52.159 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:38:52.159 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:52.159 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:38:52.159 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_upmap": [], 2023-10-26T01:38:52.160 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_upmap_items": [ 2023-10-26T01:38:52.160 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:52.160 INFO:teuthology.orchestra.run.smithi169.stdout: "pgid": "17.e", 2023-10-26T01:38:52.160 INFO:teuthology.orchestra.run.smithi169.stdout: "mappings": [ 2023-10-26T01:38:52.161 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:52.161 INFO:teuthology.orchestra.run.smithi169.stdout: "from": 0, 2023-10-26T01:38:52.161 INFO:teuthology.orchestra.run.smithi169.stdout: "to": 5 2023-10-26T01:38:52.161 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:52.161 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:52.162 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:52.162 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:52.162 INFO:teuthology.orchestra.run.smithi169.stdout: "pgid": "17.21", 2023-10-26T01:38:52.162 INFO:teuthology.orchestra.run.smithi169.stdout: "mappings": [ 2023-10-26T01:38:52.162 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:52.163 INFO:teuthology.orchestra.run.smithi169.stdout: "from": 0, 2023-10-26T01:38:52.163 INFO:teuthology.orchestra.run.smithi169.stdout: "to": 2 2023-10-26T01:38:52.163 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:52.163 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:52.163 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:52.164 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:38:52.164 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_upmap_primaries": [], 2023-10-26T01:38:52.164 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_temp": [], 2023-10-26T01:38:52.164 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_temp": [], 2023-10-26T01:38:52.164 INFO:teuthology.orchestra.run.smithi169.stdout: "blocklist": {}, 2023-10-26T01:38:52.165 INFO:teuthology.orchestra.run.smithi169.stdout: "range_blocklist": {}, 2023-10-26T01:38:52.165 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profiles": { 2023-10-26T01:38:52.165 INFO:teuthology.orchestra.run.smithi169.stdout: "default": { 2023-10-26T01:38:52.165 INFO:teuthology.orchestra.run.smithi169.stdout: "crush-failure-domain": "osd", 2023-10-26T01:38:52.166 INFO:teuthology.orchestra.run.smithi169.stdout: "k": "2", 2023-10-26T01:38:52.166 INFO:teuthology.orchestra.run.smithi169.stdout: "m": "1", 2023-10-26T01:38:52.166 INFO:teuthology.orchestra.run.smithi169.stdout: "plugin": "jerasure", 2023-10-26T01:38:52.166 INFO:teuthology.orchestra.run.smithi169.stdout: "technique": "reed_sol_van" 2023-10-26T01:38:52.166 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:52.167 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:52.167 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps_queue": [], 2023-10-26T01:38:52.167 INFO:teuthology.orchestra.run.smithi169.stdout: "new_removed_snaps": [], 2023-10-26T01:38:52.167 INFO:teuthology.orchestra.run.smithi169.stdout: "new_purged_snaps": [], 2023-10-26T01:38:52.167 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_node_flags": {}, 2023-10-26T01:38:52.168 INFO:teuthology.orchestra.run.smithi169.stdout: "device_class_flags": {}, 2023-10-26T01:38:52.168 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_mode": { 2023-10-26T01:38:52.168 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_mode_enabled": false, 2023-10-26T01:38:52.168 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_bucket_count": 0, 2023-10-26T01:38:52.168 INFO:teuthology.orchestra.run.smithi169.stdout: "degraded_stretch_mode": 0, 2023-10-26T01:38:52.169 INFO:teuthology.orchestra.run.smithi169.stdout: "recovering_stretch_mode": 0, 2023-10-26T01:38:52.169 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_mode_bucket": 0 2023-10-26T01:38:52.169 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:52.169 INFO:teuthology.orchestra.run.smithi169.stdout:} 2023-10-26T01:38:52.171 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:51.947+0000 7f029d7e2640 1 -- 172.21.15.169:0/2520678095 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f028406ae40 msgr2=0x7f028406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:52.171 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:51.947+0000 7f029d7e2640 1 --2- 172.21.15.169:0/2520678095 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f028406ae40 0x7f028406d300 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f028c000f60 tx=0x7f028c0079b0 comp rx=0 tx=0).stop 2023-10-26T01:38:52.172 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:51.947+0000 7f029d7e2640 1 -- 172.21.15.169:0/2520678095 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02980cde30 msgr2=0x7f0298150bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:52.172 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:51.947+0000 7f029d7e2640 1 --2- 172.21.15.169:0/2520678095 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02980cde30 0x7f0298150bb0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f028800d770 tx=0x7f028800dc30 comp rx=0 tx=0).stop 2023-10-26T01:38:52.172 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:51.947+0000 7f029d7e2640 1 -- 172.21.15.169:0/2520678095 shutdown_connections 2023-10-26T01:38:52.172 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:51.947+0000 7f029d7e2640 1 --2- 172.21.15.169:0/2520678095 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f028406ae40 0x7f028406d300 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:52.172 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:51.947+0000 7f029d7e2640 1 --2- 172.21.15.169:0/2520678095 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0298158850 0x7f029814ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:52.173 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:51.947+0000 7f029d7e2640 1 --2- 172.21.15.169:0/2520678095 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f029814e790 0x7f02981571e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:52.173 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:51.947+0000 7f029d7e2640 1 --2- 172.21.15.169:0/2520678095 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02980cde30 0x7f0298150bb0 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:52.173 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:51.947+0000 7f029d7e2640 1 -- 172.21.15.169:0/2520678095 >> 172.21.15.169:0/2520678095 conn(0x7f0298069930 msgr2=0x7f02980acd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:52.173 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:51.947+0000 7f029d7e2640 1 -- 172.21.15.169:0/2520678095 shutdown_connections 2023-10-26T01:38:52.173 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:51.947+0000 7f029d7e2640 1 -- 172.21.15.169:0/2520678095 wait complete. 2023-10-26T01:38:52.174 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T01:38:52.408 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:52.405+0000 7f59da705640 1 -- 172.21.15.169:0/1514725964 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f59d4158850 msgr2=0x7f59d40cd720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:52.409 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:52.405+0000 7f59da705640 1 --2- 172.21.15.169:0/1514725964 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f59d4158850 0x7f59d40cd720 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f59c8009830 tx=0x7f59c802bcd0 comp rx=0 tx=0).stop 2023-10-26T01:38:52.409 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:52.405+0000 7f59da705640 1 -- 172.21.15.169:0/1514725964 shutdown_connections 2023-10-26T01:38:52.409 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:52.405+0000 7f59da705640 1 --2- 172.21.15.169:0/1514725964 >> v2:172.21.15.170:3300/0 conn(0x7f59d40cdc60 0x7f59d4159e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:52.409 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:52.405+0000 7f59da705640 1 --2- 172.21.15.169:0/1514725964 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f59d4158850 0x7f59d40cd720 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:52.410 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:52.405+0000 7f59da705640 1 -- 172.21.15.169:0/1514725964 >> 172.21.15.169:0/1514725964 conn(0x7f59d4069930 msgr2=0x7f59d4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:52.410 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:52.406+0000 7f59da705640 1 -- 172.21.15.169:0/1514725964 shutdown_connections 2023-10-26T01:38:52.410 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:52.406+0000 7f59da705640 1 -- 172.21.15.169:0/1514725964 wait complete. 2023-10-26T01:38:52.411 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:52.407+0000 7f59da705640 1 Processor -- start 2023-10-26T01:38:52.411 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:52.407+0000 7f59da705640 1 -- start start 2023-10-26T01:38:52.412 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:52.409+0000 7f59da705640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f59d40cdc60 0x7f59d41622c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:52.412 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:52.409+0000 7f59da705640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f59d414e790 0x7f59d406ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:52.413 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:52.409+0000 7f59da705640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f59d4158850 0x7f59d4145500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:52.413 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:52.409+0000 7f59da705640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59d4071d00 con 0x7f59d414e790 2023-10-26T01:38:52.413 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:52.409+0000 7f59da705640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59d4162800 con 0x7f59d40cdc60 2023-10-26T01:38:52.413 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:52.409+0000 7f59da705640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59d406f270 con 0x7f59d4158850 2023-10-26T01:38:52.413 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:52.409+0000 7f59d37fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f59d414e790 0x7f59d406ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:52.414 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:52.409+0000 7f59d37fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f59d414e790 0x7f59d406ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:53536/0 (socket says 172.21.15.169:53536) 2023-10-26T01:38:52.414 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:52.409+0000 7f59d37fe640 1 -- 172.21.15.169:0/2764618252 learned_addr learned my addr 172.21.15.169:0/2764618252 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:38:52.414 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:52.409+0000 7f59d3fff640 1 --2- 172.21.15.169:0/2764618252 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f59d40cdc60 0x7f59d41622c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:52.414 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:52.409+0000 7f59d8ca2640 1 --2- 172.21.15.169:0/2764618252 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f59d4158850 0x7f59d4145500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:52.414 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:52.409+0000 7f59d37fe640 1 -- 172.21.15.169:0/2764618252 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f59d4158850 msgr2=0x7f59d4145500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:52.415 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:52.409+0000 7f59d37fe640 1 --2- 172.21.15.169:0/2764618252 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f59d4158850 0x7f59d4145500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:52.415 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:52.409+0000 7f59d37fe640 1 -- 172.21.15.169:0/2764618252 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f59d40cdc60 msgr2=0x7f59d41622c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:52.415 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:52.409+0000 7f59d37fe640 1 --2- 172.21.15.169:0/2764618252 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f59d40cdc60 0x7f59d41622c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:52.415 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:52.410+0000 7f59d37fe640 1 -- 172.21.15.169:0/2764618252 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59c8009490 con 0x7f59d414e790 2023-10-26T01:38:52.415 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:52.410+0000 7f59d37fe640 1 --2- 172.21.15.169:0/2764618252 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f59d414e790 0x7f59d406ed30 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f59c8009950 tx=0x7f59c80038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:52.417 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:52.410+0000 7f59d0ff9640 1 -- 172.21.15.169:0/2764618252 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c8003ea0 con 0x7f59d414e790 2023-10-26T01:38:52.417 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:52.410+0000 7f59d0ff9640 1 -- 172.21.15.169:0/2764618252 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f59c8034440 con 0x7f59d414e790 2023-10-26T01:38:52.417 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:52.410+0000 7f59da705640 1 -- 172.21.15.169:0/2764618252 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59d40b07b0 con 0x7f59d414e790 2023-10-26T01:38:52.417 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:52.410+0000 7f59d0ff9640 1 -- 172.21.15.169:0/2764618252 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c8034850 con 0x7f59d414e790 2023-10-26T01:38:52.417 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:52.410+0000 7f59da705640 1 -- 172.21.15.169:0/2764618252 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59d40b0a30 con 0x7f59d414e790 2023-10-26T01:38:52.418 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:52.411+0000 7f59da705640 1 -- 172.21.15.169:0/2764618252 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59d404db10 con 0x7f59d414e790 2023-10-26T01:38:52.418 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:52.412+0000 7f59d0ff9640 1 -- 172.21.15.169:0/2764618252 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f59c803b040 con 0x7f59d414e790 2023-10-26T01:38:52.418 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:52.413+0000 7f59d0ff9640 1 --2- 172.21.15.169:0/2764618252 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f59c006ae40 0x7f59c006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:52.418 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:52.413+0000 7f59d0ff9640 1 -- 172.21.15.169:0/2764618252 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f59c80abad0 con 0x7f59d414e790 2023-10-26T01:38:52.418 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:52.414+0000 7f59d0ff9640 1 -- 172.21.15.169:0/2764618252 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=139}) v3 -- 0x7f59c0072a30 con 0x7f59d414e790 2023-10-26T01:38:52.419 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:52.414+0000 7f59d3fff640 1 --2- 172.21.15.169:0/2764618252 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f59c006ae40 0x7f59c006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:52.419 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:52.415+0000 7f59d3fff640 1 --2- 172.21.15.169:0/2764618252 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f59c006ae40 0x7f59c006d300 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f59c4000f60 tx=0x7f59c4009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:52.420 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:52.418+0000 7f59d0ff9640 1 -- 172.21.15.169:0/2764618252 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f59c8039060 con 0x7f59d414e790 2023-10-26T01:38:52.648 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:52.645+0000 7f59da705640 1 -- 172.21.15.169:0/2764618252 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f59d40aca70 con 0x7f59d414e790 2023-10-26T01:38:52.649 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:52.647+0000 7f59d0ff9640 1 -- 172.21.15.169:0/2764618252 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+4927 (secure 0 0 0) 0x7f59c8039070 con 0x7f59d414e790 2023-10-26T01:38:52.650 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:38:52.650 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":68,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5917,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/3297412140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":3297412140},{"type":"v1","addr":"172.21.15.169:6835","nonce":3297412140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":5929,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/2575809940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2575809940},{"type":"v1","addr":"172.21.15.170:6836","nonce":2575809940}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":7761,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2632217991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2632217991},{"type":"v1","addr":"172.21.15.169:6833","nonce":2632217991}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T01:38:20.773556+0000","modified":"2023-10-26T01:38:23.839758+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7773},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7773":{"gid":7773,"name":"b","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6837/3615088922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":3615088922},{"type":"v1","addr":"172.21.15.170:6837","nonce":3615088922}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-10-26T01:38:52.650 INFO:teuthology.orchestra.run.smithi169.stderr:dumped fsmap epoch 68 2023-10-26T01:38:52.652 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:52.649+0000 7f59da705640 1 -- 172.21.15.169:0/2764618252 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f59c006ae40 msgr2=0x7f59c006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:52.652 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:52.649+0000 7f59da705640 1 --2- 172.21.15.169:0/2764618252 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f59c006ae40 0x7f59c006d300 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f59c4000f60 tx=0x7f59c4009040 comp rx=0 tx=0).stop 2023-10-26T01:38:52.652 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:52.649+0000 7f59da705640 1 -- 172.21.15.169:0/2764618252 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f59d414e790 msgr2=0x7f59d406ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:52.653 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:52.649+0000 7f59da705640 1 --2- 172.21.15.169:0/2764618252 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f59d414e790 0x7f59d406ed30 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f59c8009950 tx=0x7f59c80038a0 comp rx=0 tx=0).stop 2023-10-26T01:38:52.653 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:52.649+0000 7f59da705640 1 -- 172.21.15.169:0/2764618252 shutdown_connections 2023-10-26T01:38:52.653 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:52.649+0000 7f59da705640 1 --2- 172.21.15.169:0/2764618252 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f59c006ae40 0x7f59c006d300 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:52.653 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:52.649+0000 7f59da705640 1 --2- 172.21.15.169:0/2764618252 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f59d414e790 0x7f59d406ed30 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:52.653 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:52.649+0000 7f59da705640 1 --2- 172.21.15.169:0/2764618252 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f59d4158850 0x7f59d4145500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:52.654 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:52.649+0000 7f59da705640 1 --2- 172.21.15.169:0/2764618252 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f59d40cdc60 0x7f59d41622c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:52.654 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:52.649+0000 7f59da705640 1 -- 172.21.15.169:0/2764618252 >> 172.21.15.169:0/2764618252 conn(0x7f59d4069930 msgr2=0x7f59d40ae130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:52.654 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:52.649+0000 7f59da705640 1 -- 172.21.15.169:0/2764618252 shutdown_connections 2023-10-26T01:38:52.654 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:52.649+0000 7f59da705640 1 -- 172.21.15.169:0/2764618252 wait complete. 2023-10-26T01:38:52.670 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2023-10-26T01:38:52.914 INFO:teuthology.orchestra.run.smithi170.stdout:{"cluster_fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","whoami":0,"id":7773,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":31.574316655000001,"mdsmap_epoch":68,"osdmap_epoch":138,"osdmap_epoch_barrier":138,"uptime":37.499833256999999} 2023-10-26T01:38:52.914 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "30203b28-ed8a-44e8-a9a6-a9bf25f980f1", "fs_name": "cephfs", "id": 7773, "mdsmap_epoch": 68, "osdmap_epoch": 138, "osdmap_epoch_barrier": 138, "rank_uptime": 31.574316655, "state": "up:active", "uptime": 37.499833257, "want_state": "up:active", "whoami": 0 } 2023-10-26T01:38:52.915 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-26T01:38:52.915 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:38:52.916 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_a) 2023-10-26T01:38:52.976 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:38:52.977 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_b) 2023-10-26T01:38:53.034 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-26T01:38:53.259 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:53.256+0000 7f3f4656c640 1 -- 172.21.15.169:0/1093450754 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3f400ac6d0 msgr2=0x7f3f4015d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:53.260 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:53.256+0000 7f3f4656c640 1 --2- 172.21.15.169:0/1093450754 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3f400ac6d0 0x7f3f4015d840 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c005fd0 tx=0x7f3f3c02be10 comp rx=0 tx=0).stop 2023-10-26T01:38:53.260 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:53.257+0000 7f3f4656c640 1 -- 172.21.15.169:0/1093450754 shutdown_connections 2023-10-26T01:38:53.260 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:53.257+0000 7f3f4656c640 1 --2- 172.21.15.169:0/1093450754 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3f400c9970 0x7f3f40098a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:53.261 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:53.257+0000 7f3f4656c640 1 --2- 172.21.15.169:0/1093450754 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3f400ac6d0 0x7f3f4015d840 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:53.261 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:53.257+0000 7f3f4656c640 1 -- 172.21.15.169:0/1093450754 >> 172.21.15.169:0/1093450754 conn(0x7f3f400aedd0 msgr2=0x7f3f400af1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:53.261 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:53.257+0000 7f3f4656c640 1 -- 172.21.15.169:0/1093450754 shutdown_connections 2023-10-26T01:38:53.261 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:53.257+0000 7f3f4656c640 1 -- 172.21.15.169:0/1093450754 wait complete. 2023-10-26T01:38:53.264 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:53.258+0000 7f3f4656c640 1 Processor -- start 2023-10-26T01:38:53.264 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:53.259+0000 7f3f4656c640 1 -- start start 2023-10-26T01:38:53.264 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:53.260+0000 7f3f4656c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3f4009a5c0 0x7f3f40157e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:53.264 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:53.260+0000 7f3f4656c640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3f400ac6d0 0x7f3f40070ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:53.265 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:53.260+0000 7f3f4656c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3f400c9970 0x7f3f400c8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:53.265 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:53.260+0000 7f3f4656c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f4015abd0 con 0x7f3f4009a5c0 2023-10-26T01:38:53.265 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:53.260+0000 7f3f4656c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f40159f50 con 0x7f3f400c9970 2023-10-26T01:38:53.265 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:53.260+0000 7f3f4656c640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f4015a230 con 0x7f3f400ac6d0 2023-10-26T01:38:53.266 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:53.260+0000 7f3f4556a640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3f4009a5c0 0x7f3f40157e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:53.266 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:53.260+0000 7f3f4556a640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3f4009a5c0 0x7f3f40157e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:53540/0 (socket says 172.21.15.169:53540) 2023-10-26T01:38:53.266 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:53.260+0000 7f3f4556a640 1 -- 172.21.15.169:0/3986871535 learned_addr learned my addr 172.21.15.169:0/3986871535 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:38:53.266 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:53.260+0000 7f3f4556a640 1 -- 172.21.15.169:0/3986871535 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3f400ac6d0 msgr2=0x7f3f40070ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:53.267 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:53.260+0000 7f3f44d69640 1 --2- 172.21.15.169:0/3986871535 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3f400ac6d0 0x7f3f40070ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:53.267 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:53.260+0000 7f3f45d6b640 1 --2- 172.21.15.169:0/3986871535 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3f400c9970 0x7f3f400c8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:53.267 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:53.260+0000 7f3f4556a640 1 --2- 172.21.15.169:0/3986871535 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3f400ac6d0 0x7f3f40070ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:53.267 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:53.260+0000 7f3f4556a640 1 -- 172.21.15.169:0/3986871535 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3f400c9970 msgr2=0x7f3f400c8ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:53.267 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:53.260+0000 7f3f4556a640 1 --2- 172.21.15.169:0/3986871535 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3f400c9970 0x7f3f400c8ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:53.268 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:53.261+0000 7f3f4556a640 1 -- 172.21.15.169:0/3986871535 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f3c009560 con 0x7f3f4009a5c0 2023-10-26T01:38:53.268 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:53.261+0000 7f3f4556a640 1 --2- 172.21.15.169:0/3986871535 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3f4009a5c0 0x7f3f40157e90 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c006010 tx=0x7f3f3c02da70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:53.268 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:53.261+0000 7f3f35ffb640 1 -- 172.21.15.169:0/3986871535 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f3c02de70 con 0x7f3f4009a5c0 2023-10-26T01:38:53.268 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:53.261+0000 7f3f35ffb640 1 -- 172.21.15.169:0/3986871535 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3f3c0036e0 con 0x7f3f4009a5c0 2023-10-26T01:38:53.268 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:53.261+0000 7f3f35ffb640 1 -- 172.21.15.169:0/3986871535 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f3c003af0 con 0x7f3f4009a5c0 2023-10-26T01:38:53.269 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:53.261+0000 7f3f4656c640 1 -- 172.21.15.169:0/3986871535 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f4015a510 con 0x7f3f4009a5c0 2023-10-26T01:38:53.269 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:53.261+0000 7f3f4656c640 1 -- 172.21.15.169:0/3986871535 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f40159bb0 con 0x7f3f4009a5c0 2023-10-26T01:38:53.270 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:53.263+0000 7f3f4656c640 1 -- 172.21.15.169:0/3986871535 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f4015ad10 con 0x7f3f4009a5c0 2023-10-26T01:38:53.270 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:53.263+0000 7f3f35ffb640 1 -- 172.21.15.169:0/3986871535 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f3f3c003c50 con 0x7f3f4009a5c0 2023-10-26T01:38:53.270 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:53.264+0000 7f3f35ffb640 1 --2- 172.21.15.169:0/3986871535 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3f2406ae40 0x7f3f2406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:53.270 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:53.264+0000 7f3f35ffb640 1 -- 172.21.15.169:0/3986871535 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f3f3c0ac300 con 0x7f3f4009a5c0 2023-10-26T01:38:53.271 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:53.265+0000 7f3f35ffb640 1 -- 172.21.15.169:0/3986871535 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=139}) v3 -- 0x7f3f24072a30 con 0x7f3f4009a5c0 2023-10-26T01:38:53.271 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:53.265+0000 7f3f44d69640 1 --2- 172.21.15.169:0/3986871535 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3f2406ae40 0x7f3f2406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:53.271 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:53.266+0000 7f3f44d69640 1 --2- 172.21.15.169:0/3986871535 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3f2406ae40 0x7f3f2406d300 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f3f400c7500 tx=0x7f3f30005f90 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:53.272 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:53.269+0000 7f3f35ffb640 1 -- 172.21.15.169:0/3986871535 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3f4015ad10 con 0x7f3f4009a5c0 2023-10-26T01:38:53.471 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:53.468+0000 7f3f4656c640 1 -- 172.21.15.169:0/3986871535 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f3f4006b980 con 0x7f3f4009a5c0 2023-10-26T01:38:53.473 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:53.470+0000 7f3f35ffb640 1 -- 172.21.15.169:0/3986871535 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v138) v1 ==== 81+0+34469 (secure 0 0 0) 0x7f3f3c043030 con 0x7f3f4009a5c0 2023-10-26T01:38:53.474 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:38:53.474 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2023-10-26T01:38:53.474 INFO:teuthology.orchestra.run.smithi169.stdout: "epoch": 138, 2023-10-26T01:38:53.474 INFO:teuthology.orchestra.run.smithi169.stdout: "fsid": "30203b28-ed8a-44e8-a9a6-a9bf25f980f1", 2023-10-26T01:38:53.475 INFO:teuthology.orchestra.run.smithi169.stdout: "created": "2023-10-26T01:30:22.897155+0000", 2023-10-26T01:38:53.475 INFO:teuthology.orchestra.run.smithi169.stdout: "modified": "2023-10-26T01:38:48.057101+0000", 2023-10-26T01:38:53.475 INFO:teuthology.orchestra.run.smithi169.stdout: "last_up_change": "2023-10-26T01:30:31.875281+0000", 2023-10-26T01:38:53.475 INFO:teuthology.orchestra.run.smithi169.stdout: "last_in_change": "2023-10-26T01:30:27.637941+0000", 2023-10-26T01:38:53.475 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-26T01:38:53.476 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_num": 5799936, 2023-10-26T01:38:53.476 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_set": [ 2023-10-26T01:38:53.476 INFO:teuthology.orchestra.run.smithi169.stdout: "pglog_hardlimit", 2023-10-26T01:38:53.476 INFO:teuthology.orchestra.run.smithi169.stdout: "purged_snapdirs", 2023-10-26T01:38:53.476 INFO:teuthology.orchestra.run.smithi169.stdout: "recovery_deletes", 2023-10-26T01:38:53.476 INFO:teuthology.orchestra.run.smithi169.stdout: "sortbitwise" 2023-10-26T01:38:53.477 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:38:53.477 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_version": 4, 2023-10-26T01:38:53.477 INFO:teuthology.orchestra.run.smithi169.stdout: "full_ratio": 0.69999998807907104, 2023-10-26T01:38:53.477 INFO:teuthology.orchestra.run.smithi169.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-26T01:38:53.477 INFO:teuthology.orchestra.run.smithi169.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-26T01:38:53.478 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_snapshot": "", 2023-10-26T01:38:53.478 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_max": 17, 2023-10-26T01:38:53.478 INFO:teuthology.orchestra.run.smithi169.stdout: "max_osd": 8, 2023-10-26T01:38:53.478 INFO:teuthology.orchestra.run.smithi169.stdout: "require_min_compat_client": "luminous", 2023-10-26T01:38:53.479 INFO:teuthology.orchestra.run.smithi169.stdout: "min_compat_client": "luminous", 2023-10-26T01:38:53.479 INFO:teuthology.orchestra.run.smithi169.stdout: "require_osd_release": "reef", 2023-10-26T01:38:53.479 INFO:teuthology.orchestra.run.smithi169.stdout: "allow_crimson": false, 2023-10-26T01:38:53.479 INFO:teuthology.orchestra.run.smithi169.stdout: "pools": [ 2023-10-26T01:38:53.479 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:53.479 INFO:teuthology.orchestra.run.smithi169.stdout: "pool": 1, 2023-10-26T01:38:53.480 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_name": ".mgr", 2023-10-26T01:38:53.480 INFO:teuthology.orchestra.run.smithi169.stdout: "create_time": "2023-10-26T01:30:32.200808+0000", 2023-10-26T01:38:53.480 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": 1, 2023-10-26T01:38:53.480 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_names": "hashpspool", 2023-10-26T01:38:53.480 INFO:teuthology.orchestra.run.smithi169.stdout: "type": 1, 2023-10-26T01:38:53.480 INFO:teuthology.orchestra.run.smithi169.stdout: "size": 2, 2023-10-26T01:38:53.481 INFO:teuthology.orchestra.run.smithi169.stdout: "min_size": 1, 2023-10-26T01:38:53.481 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_rule": 0, 2023-10-26T01:38:53.481 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_count": 0, 2023-10-26T01:38:53.481 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_target": 0, 2023-10-26T01:38:53.481 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T01:38:53.482 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T01:38:53.482 INFO:teuthology.orchestra.run.smithi169.stdout: "object_hash": 2, 2023-10-26T01:38:53.482 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_mode": "off", 2023-10-26T01:38:53.482 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num": 1, 2023-10-26T01:38:53.482 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num": 1, 2023-10-26T01:38:53.483 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num_target": 1, 2023-10-26T01:38:53.483 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_target": 1, 2023-10-26T01:38:53.483 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_pending": 1, 2023-10-26T01:38:53.483 INFO:teuthology.orchestra.run.smithi169.stdout: "last_pg_merge_meta": { 2023-10-26T01:38:53.483 INFO:teuthology.orchestra.run.smithi169.stdout: "source_pgid": "0.0", 2023-10-26T01:38:53.484 INFO:teuthology.orchestra.run.smithi169.stdout: "ready_epoch": 0, 2023-10-26T01:38:53.484 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_started": 0, 2023-10-26T01:38:53.484 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_clean": 0, 2023-10-26T01:38:53.484 INFO:teuthology.orchestra.run.smithi169.stdout: "source_version": "0'0", 2023-10-26T01:38:53.484 INFO:teuthology.orchestra.run.smithi169.stdout: "target_version": "0'0" 2023-10-26T01:38:53.484 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:53.485 INFO:teuthology.orchestra.run.smithi169.stdout: "last_change": "109", 2023-10-26T01:38:53.485 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend": "0", 2023-10-26T01:38:53.485 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T01:38:53.485 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T01:38:53.485 INFO:teuthology.orchestra.run.smithi169.stdout: "auid": 0, 2023-10-26T01:38:53.486 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_mode": "selfmanaged", 2023-10-26T01:38:53.486 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_seq": 0, 2023-10-26T01:38:53.486 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_epoch": 0, 2023-10-26T01:38:53.486 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_snaps": [], 2023-10-26T01:38:53.486 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps": "[]", 2023-10-26T01:38:53.486 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_bytes": 0, 2023-10-26T01:38:53.487 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_objects": 0, 2023-10-26T01:38:53.487 INFO:teuthology.orchestra.run.smithi169.stdout: "tiers": [], 2023-10-26T01:38:53.487 INFO:teuthology.orchestra.run.smithi169.stdout: "tier_of": -1, 2023-10-26T01:38:53.487 INFO:teuthology.orchestra.run.smithi169.stdout: "read_tier": -1, 2023-10-26T01:38:53.487 INFO:teuthology.orchestra.run.smithi169.stdout: "write_tier": -1, 2023-10-26T01:38:53.488 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_mode": "none", 2023-10-26T01:38:53.488 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_bytes": 0, 2023-10-26T01:38:53.488 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_objects": 0, 2023-10-26T01:38:53.488 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T01:38:53.488 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T01:38:53.488 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T01:38:53.489 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_flush_age": 0, 2023-10-26T01:38:53.489 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_evict_age": 0, 2023-10-26T01:38:53.489 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profile": "", 2023-10-26T01:38:53.489 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_params": { 2023-10-26T01:38:53.489 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "none" 2023-10-26T01:38:53.490 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:53.490 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_period": 0, 2023-10-26T01:38:53.490 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_count": 0, 2023-10-26T01:38:53.490 INFO:teuthology.orchestra.run.smithi169.stdout: "use_gmt_hitset": true, 2023-10-26T01:38:53.490 INFO:teuthology.orchestra.run.smithi169.stdout: "min_read_recency_for_promote": 0, 2023-10-26T01:38:53.491 INFO:teuthology.orchestra.run.smithi169.stdout: "min_write_recency_for_promote": 0, 2023-10-26T01:38:53.491 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T01:38:53.491 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_search_last_n": 0, 2023-10-26T01:38:53.491 INFO:teuthology.orchestra.run.smithi169.stdout: "grade_table": [], 2023-10-26T01:38:53.491 INFO:teuthology.orchestra.run.smithi169.stdout: "stripe_width": 0, 2023-10-26T01:38:53.492 INFO:teuthology.orchestra.run.smithi169.stdout: "expected_num_objects": 0, 2023-10-26T01:38:53.492 INFO:teuthology.orchestra.run.smithi169.stdout: "fast_read": false, 2023-10-26T01:38:53.492 INFO:teuthology.orchestra.run.smithi169.stdout: "options": { 2023-10-26T01:38:53.492 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_max": 32, 2023-10-26T01:38:53.492 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_min": 1 2023-10-26T01:38:53.493 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:53.493 INFO:teuthology.orchestra.run.smithi169.stdout: "application_metadata": { 2023-10-26T01:38:53.493 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": {} 2023-10-26T01:38:53.493 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:53.493 INFO:teuthology.orchestra.run.smithi169.stdout: "read_balance": { 2023-10-26T01:38:53.493 INFO:teuthology.orchestra.run.smithi169.stdout: "score_acting": 8, 2023-10-26T01:38:53.494 INFO:teuthology.orchestra.run.smithi169.stdout: "score_stable": 8, 2023-10-26T01:38:53.494 INFO:teuthology.orchestra.run.smithi169.stdout: "optimal_score": 0.25, 2023-10-26T01:38:53.494 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_acting": 2, 2023-10-26T01:38:53.494 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_stable": 2, 2023-10-26T01:38:53.495 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity_weighted": 1, 2023-10-26T01:38:53.495 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity": 1, 2023-10-26T01:38:53.495 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity_weighted": 1 2023-10-26T01:38:53.495 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:53.495 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:53.495 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:53.496 INFO:teuthology.orchestra.run.smithi169.stdout: "pool": 16, 2023-10-26T01:38:53.496 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_name": "cephfs_metadata", 2023-10-26T01:38:53.496 INFO:teuthology.orchestra.run.smithi169.stdout: "create_time": "2023-10-26T01:38:18.617877+0000", 2023-10-26T01:38:53.496 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": 1, 2023-10-26T01:38:53.496 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_names": "hashpspool", 2023-10-26T01:38:53.497 INFO:teuthology.orchestra.run.smithi169.stdout: "type": 1, 2023-10-26T01:38:53.497 INFO:teuthology.orchestra.run.smithi169.stdout: "size": 2, 2023-10-26T01:38:53.497 INFO:teuthology.orchestra.run.smithi169.stdout: "min_size": 1, 2023-10-26T01:38:53.497 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_rule": 0, 2023-10-26T01:38:53.497 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_count": 0, 2023-10-26T01:38:53.497 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_target": 0, 2023-10-26T01:38:53.498 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T01:38:53.498 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T01:38:53.498 INFO:teuthology.orchestra.run.smithi169.stdout: "object_hash": 2, 2023-10-26T01:38:53.498 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_mode": "off", 2023-10-26T01:38:53.498 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num": 32, 2023-10-26T01:38:53.499 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num": 32, 2023-10-26T01:38:53.499 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num_target": 32, 2023-10-26T01:38:53.499 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_target": 32, 2023-10-26T01:38:53.499 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_pending": 32, 2023-10-26T01:38:53.499 INFO:teuthology.orchestra.run.smithi169.stdout: "last_pg_merge_meta": { 2023-10-26T01:38:53.499 INFO:teuthology.orchestra.run.smithi169.stdout: "source_pgid": "0.0", 2023-10-26T01:38:53.500 INFO:teuthology.orchestra.run.smithi169.stdout: "ready_epoch": 0, 2023-10-26T01:38:53.500 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_started": 0, 2023-10-26T01:38:53.500 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_clean": 0, 2023-10-26T01:38:53.500 INFO:teuthology.orchestra.run.smithi169.stdout: "source_version": "0'0", 2023-10-26T01:38:53.500 INFO:teuthology.orchestra.run.smithi169.stdout: "target_version": "0'0" 2023-10-26T01:38:53.501 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:53.501 INFO:teuthology.orchestra.run.smithi169.stdout: "last_change": "133", 2023-10-26T01:38:53.501 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend": "0", 2023-10-26T01:38:53.501 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T01:38:53.501 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T01:38:53.502 INFO:teuthology.orchestra.run.smithi169.stdout: "auid": 0, 2023-10-26T01:38:53.502 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_mode": "selfmanaged", 2023-10-26T01:38:53.502 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_seq": 0, 2023-10-26T01:38:53.502 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_epoch": 0, 2023-10-26T01:38:53.502 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_snaps": [], 2023-10-26T01:38:53.502 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps": "[]", 2023-10-26T01:38:53.503 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_bytes": 0, 2023-10-26T01:38:53.503 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_objects": 0, 2023-10-26T01:38:53.503 INFO:teuthology.orchestra.run.smithi169.stdout: "tiers": [], 2023-10-26T01:38:53.503 INFO:teuthology.orchestra.run.smithi169.stdout: "tier_of": -1, 2023-10-26T01:38:53.503 INFO:teuthology.orchestra.run.smithi169.stdout: "read_tier": -1, 2023-10-26T01:38:53.504 INFO:teuthology.orchestra.run.smithi169.stdout: "write_tier": -1, 2023-10-26T01:38:53.504 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_mode": "none", 2023-10-26T01:38:53.504 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_bytes": 0, 2023-10-26T01:38:53.504 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_objects": 0, 2023-10-26T01:38:53.504 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T01:38:53.505 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T01:38:53.505 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T01:38:53.505 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_flush_age": 0, 2023-10-26T01:38:53.505 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_evict_age": 0, 2023-10-26T01:38:53.505 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profile": "", 2023-10-26T01:38:53.505 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_params": { 2023-10-26T01:38:53.506 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "none" 2023-10-26T01:38:53.506 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:53.506 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_period": 0, 2023-10-26T01:38:53.506 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_count": 0, 2023-10-26T01:38:53.506 INFO:teuthology.orchestra.run.smithi169.stdout: "use_gmt_hitset": true, 2023-10-26T01:38:53.507 INFO:teuthology.orchestra.run.smithi169.stdout: "min_read_recency_for_promote": 0, 2023-10-26T01:38:53.507 INFO:teuthology.orchestra.run.smithi169.stdout: "min_write_recency_for_promote": 0, 2023-10-26T01:38:53.507 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T01:38:53.507 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_search_last_n": 0, 2023-10-26T01:38:53.507 INFO:teuthology.orchestra.run.smithi169.stdout: "grade_table": [], 2023-10-26T01:38:53.508 INFO:teuthology.orchestra.run.smithi169.stdout: "stripe_width": 0, 2023-10-26T01:38:53.508 INFO:teuthology.orchestra.run.smithi169.stdout: "expected_num_objects": 0, 2023-10-26T01:38:53.508 INFO:teuthology.orchestra.run.smithi169.stdout: "fast_read": false, 2023-10-26T01:38:53.508 INFO:teuthology.orchestra.run.smithi169.stdout: "options": { 2023-10-26T01:38:53.508 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_bias": 4, 2023-10-26T01:38:53.509 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_min": 16, 2023-10-26T01:38:53.509 INFO:teuthology.orchestra.run.smithi169.stdout: "recovery_priority": 5 2023-10-26T01:38:53.509 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:53.509 INFO:teuthology.orchestra.run.smithi169.stdout: "application_metadata": { 2023-10-26T01:38:53.509 INFO:teuthology.orchestra.run.smithi169.stdout: "cephfs": { 2023-10-26T01:38:53.509 INFO:teuthology.orchestra.run.smithi169.stdout: "metadata": "cephfs" 2023-10-26T01:38:53.510 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:53.510 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:53.510 INFO:teuthology.orchestra.run.smithi169.stdout: "read_balance": { 2023-10-26T01:38:53.510 INFO:teuthology.orchestra.run.smithi169.stdout: "score_acting": 1.75, 2023-10-26T01:38:53.510 INFO:teuthology.orchestra.run.smithi169.stdout: "score_stable": 1.75, 2023-10-26T01:38:53.511 INFO:teuthology.orchestra.run.smithi169.stdout: "optimal_score": 1, 2023-10-26T01:38:53.511 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_acting": 1.75, 2023-10-26T01:38:53.511 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_stable": 1.75, 2023-10-26T01:38:53.511 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity_weighted": 1, 2023-10-26T01:38:53.511 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity": 1, 2023-10-26T01:38:53.511 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity_weighted": 1 2023-10-26T01:38:53.512 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:53.512 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:53.512 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:53.512 INFO:teuthology.orchestra.run.smithi169.stdout: "pool": 17, 2023-10-26T01:38:53.512 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_name": "cephfs_data", 2023-10-26T01:38:53.512 INFO:teuthology.orchestra.run.smithi169.stdout: "create_time": "2023-10-26T01:38:19.793041+0000", 2023-10-26T01:38:53.513 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": 1027, 2023-10-26T01:38:53.513 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_names": "hashpspool,full,full_quota", 2023-10-26T01:38:53.513 INFO:teuthology.orchestra.run.smithi169.stdout: "type": 1, 2023-10-26T01:38:53.513 INFO:teuthology.orchestra.run.smithi169.stdout: "size": 2, 2023-10-26T01:38:53.513 INFO:teuthology.orchestra.run.smithi169.stdout: "min_size": 1, 2023-10-26T01:38:53.513 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_rule": 0, 2023-10-26T01:38:53.514 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_count": 0, 2023-10-26T01:38:53.514 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_target": 0, 2023-10-26T01:38:53.514 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T01:38:53.514 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T01:38:53.514 INFO:teuthology.orchestra.run.smithi169.stdout: "object_hash": 2, 2023-10-26T01:38:53.515 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_mode": "off", 2023-10-26T01:38:53.515 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num": 64, 2023-10-26T01:38:53.515 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num": 64, 2023-10-26T01:38:53.515 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num_target": 64, 2023-10-26T01:38:53.515 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_target": 64, 2023-10-26T01:38:53.515 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_pending": 64, 2023-10-26T01:38:53.516 INFO:teuthology.orchestra.run.smithi169.stdout: "last_pg_merge_meta": { 2023-10-26T01:38:53.516 INFO:teuthology.orchestra.run.smithi169.stdout: "source_pgid": "0.0", 2023-10-26T01:38:53.516 INFO:teuthology.orchestra.run.smithi169.stdout: "ready_epoch": 0, 2023-10-26T01:38:53.516 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_started": 0, 2023-10-26T01:38:53.516 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_clean": 0, 2023-10-26T01:38:53.516 INFO:teuthology.orchestra.run.smithi169.stdout: "source_version": "0'0", 2023-10-26T01:38:53.517 INFO:teuthology.orchestra.run.smithi169.stdout: "target_version": "0'0" 2023-10-26T01:38:53.517 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:53.517 INFO:teuthology.orchestra.run.smithi169.stdout: "last_change": "138", 2023-10-26T01:38:53.517 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend": "0", 2023-10-26T01:38:53.517 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T01:38:53.518 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T01:38:53.518 INFO:teuthology.orchestra.run.smithi169.stdout: "auid": 0, 2023-10-26T01:38:53.518 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_mode": "selfmanaged", 2023-10-26T01:38:53.518 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_seq": 0, 2023-10-26T01:38:53.518 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_epoch": 0, 2023-10-26T01:38:53.519 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_snaps": [], 2023-10-26T01:38:53.519 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps": "[]", 2023-10-26T01:38:53.519 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_bytes": 33554432, 2023-10-26T01:38:53.519 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_objects": 0, 2023-10-26T01:38:53.519 INFO:teuthology.orchestra.run.smithi169.stdout: "tiers": [], 2023-10-26T01:38:53.519 INFO:teuthology.orchestra.run.smithi169.stdout: "tier_of": -1, 2023-10-26T01:38:53.520 INFO:teuthology.orchestra.run.smithi169.stdout: "read_tier": -1, 2023-10-26T01:38:53.520 INFO:teuthology.orchestra.run.smithi169.stdout: "write_tier": -1, 2023-10-26T01:38:53.520 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_mode": "none", 2023-10-26T01:38:53.520 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_bytes": 0, 2023-10-26T01:38:53.520 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_objects": 0, 2023-10-26T01:38:53.521 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T01:38:53.521 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T01:38:53.521 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T01:38:53.521 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_flush_age": 0, 2023-10-26T01:38:53.521 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_evict_age": 0, 2023-10-26T01:38:53.522 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profile": "", 2023-10-26T01:38:53.522 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_params": { 2023-10-26T01:38:53.522 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "none" 2023-10-26T01:38:53.522 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:53.522 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_period": 0, 2023-10-26T01:38:53.523 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_count": 0, 2023-10-26T01:38:53.523 INFO:teuthology.orchestra.run.smithi169.stdout: "use_gmt_hitset": true, 2023-10-26T01:38:53.523 INFO:teuthology.orchestra.run.smithi169.stdout: "min_read_recency_for_promote": 0, 2023-10-26T01:38:53.523 INFO:teuthology.orchestra.run.smithi169.stdout: "min_write_recency_for_promote": 0, 2023-10-26T01:38:53.523 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T01:38:53.523 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_search_last_n": 0, 2023-10-26T01:38:53.524 INFO:teuthology.orchestra.run.smithi169.stdout: "grade_table": [], 2023-10-26T01:38:53.524 INFO:teuthology.orchestra.run.smithi169.stdout: "stripe_width": 0, 2023-10-26T01:38:53.524 INFO:teuthology.orchestra.run.smithi169.stdout: "expected_num_objects": 0, 2023-10-26T01:38:53.524 INFO:teuthology.orchestra.run.smithi169.stdout: "fast_read": false, 2023-10-26T01:38:53.524 INFO:teuthology.orchestra.run.smithi169.stdout: "options": { 2023-10-26T01:38:53.524 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_min": 64, 2023-10-26T01:38:53.525 INFO:teuthology.orchestra.run.smithi169.stdout: "target_size_ratio": 0.89999997615814209 2023-10-26T01:38:53.525 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:53.525 INFO:teuthology.orchestra.run.smithi169.stdout: "application_metadata": { 2023-10-26T01:38:53.525 INFO:teuthology.orchestra.run.smithi169.stdout: "cephfs": { 2023-10-26T01:38:53.525 INFO:teuthology.orchestra.run.smithi169.stdout: "data": "cephfs" 2023-10-26T01:38:53.526 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:53.526 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:53.526 INFO:teuthology.orchestra.run.smithi169.stdout: "read_balance": { 2023-10-26T01:38:53.526 INFO:teuthology.orchestra.run.smithi169.stdout: "score_acting": 1.6299999952316284, 2023-10-26T01:38:53.526 INFO:teuthology.orchestra.run.smithi169.stdout: "score_stable": 1.6299999952316284, 2023-10-26T01:38:53.526 INFO:teuthology.orchestra.run.smithi169.stdout: "optimal_score": 1, 2023-10-26T01:38:53.527 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_acting": 1.6299999952316284, 2023-10-26T01:38:53.527 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_stable": 1.6299999952316284, 2023-10-26T01:38:53.527 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity_weighted": 1, 2023-10-26T01:38:53.527 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity": 1, 2023-10-26T01:38:53.527 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity_weighted": 1 2023-10-26T01:38:53.528 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:53.528 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:53.528 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:38:53.528 INFO:teuthology.orchestra.run.smithi169.stdout: "osds": [ 2023-10-26T01:38:53.528 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:53.529 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 0, 2023-10-26T01:38:53.529 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:38:53.529 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:38:53.529 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:38:53.529 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:38:53.529 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:38:53.530 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:38:53.530 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:38:53.530 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:38:53.530 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 132, 2023-10-26T01:38:53.530 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:38:53.531 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:38:53.531 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:38:53.531 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:53.531 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:53.531 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:53.531 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6816", 2023-10-26T01:38:53.532 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:38:53.532 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:53.532 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:53.532 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:53.532 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6817", 2023-10-26T01:38:53.533 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:38:53.533 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:53.533 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:53.533 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:53.533 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:38:53.533 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:53.534 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:53.534 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:53.534 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6818", 2023-10-26T01:38:53.534 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:38:53.534 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:53.535 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:53.535 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:53.535 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6819", 2023-10-26T01:38:53.535 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:38:53.535 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:53.536 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:53.536 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:53.536 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:38:53.536 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:53.536 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:53.536 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:53.537 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6822", 2023-10-26T01:38:53.537 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:38:53.537 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:53.537 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:53.537 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:53.538 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6823", 2023-10-26T01:38:53.538 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:38:53.538 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:53.538 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:53.538 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:53.539 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:38:53.539 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:53.539 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:53.539 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:53.539 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6820", 2023-10-26T01:38:53.539 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:38:53.540 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:53.540 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:53.540 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:53.540 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6821", 2023-10-26T01:38:53.540 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:38:53.541 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:53.541 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:53.541 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:53.541 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6817/4204255864", 2023-10-26T01:38:53.541 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6819/4204255864", 2023-10-26T01:38:53.542 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6823/4204255864", 2023-10-26T01:38:53.542 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6821/4204255864", 2023-10-26T01:38:53.542 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:38:53.542 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:38:53.542 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:38:53.543 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:53.543 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:53.543 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:53.543 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 1, 2023-10-26T01:38:53.543 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:38:53.543 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:38:53.544 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:38:53.544 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:38:53.544 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:38:53.544 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:38:53.544 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:38:53.545 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:38:53.545 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 132, 2023-10-26T01:38:53.545 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:38:53.545 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:38:53.545 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:38:53.545 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:53.546 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:53.546 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:53.546 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6808", 2023-10-26T01:38:53.546 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:38:53.546 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:53.547 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:53.547 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:53.547 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6809", 2023-10-26T01:38:53.547 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:38:53.547 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:53.548 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:53.548 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:53.548 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:38:53.548 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:53.548 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:53.549 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:53.549 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6810", 2023-10-26T01:38:53.549 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:38:53.549 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:53.549 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:53.549 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:53.550 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6811", 2023-10-26T01:38:53.550 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:38:53.550 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:53.550 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:53.550 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:53.551 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:38:53.551 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:53.551 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:53.551 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:53.551 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6814", 2023-10-26T01:38:53.552 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:38:53.552 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:53.552 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:53.552 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:53.552 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6815", 2023-10-26T01:38:53.553 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:38:53.553 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:53.553 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:53.553 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:53.553 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:38:53.553 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:53.554 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:53.554 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:53.554 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6812", 2023-10-26T01:38:53.554 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:38:53.554 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:53.554 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:53.555 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:53.555 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6813", 2023-10-26T01:38:53.555 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:38:53.555 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:53.555 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:53.555 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:53.556 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6809/238206046", 2023-10-26T01:38:53.556 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6811/238206046", 2023-10-26T01:38:53.556 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6815/238206046", 2023-10-26T01:38:53.556 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6813/238206046", 2023-10-26T01:38:53.556 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:38:53.557 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:38:53.557 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:38:53.557 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:53.557 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:53.557 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:53.557 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 2, 2023-10-26T01:38:53.558 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:38:53.558 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:38:53.558 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:38:53.558 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:38:53.558 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:38:53.559 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:38:53.559 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:38:53.559 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:38:53.559 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 135, 2023-10-26T01:38:53.559 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:38:53.560 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:38:53.560 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:38:53.560 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:53.560 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:53.560 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:53.560 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6800", 2023-10-26T01:38:53.561 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:38:53.561 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:53.561 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:53.561 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:53.561 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6801", 2023-10-26T01:38:53.561 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:38:53.562 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:53.562 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:53.562 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:53.562 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:38:53.562 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:53.562 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:53.563 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:53.563 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6802", 2023-10-26T01:38:53.563 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:38:53.563 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:53.563 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:53.563 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:53.564 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6803", 2023-10-26T01:38:53.564 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:38:53.564 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:53.564 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:53.564 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:53.564 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:38:53.565 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:53.565 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:53.565 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:53.565 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6806", 2023-10-26T01:38:53.565 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:38:53.565 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:53.566 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:53.566 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:53.566 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6807", 2023-10-26T01:38:53.566 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:38:53.566 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:53.567 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:53.567 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:53.567 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:38:53.567 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:53.567 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:53.567 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:53.567 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6804", 2023-10-26T01:38:53.568 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:38:53.568 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:53.568 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:53.568 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:53.568 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6805", 2023-10-26T01:38:53.569 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:38:53.569 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:53.569 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:53.569 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:53.569 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6801/3215791846", 2023-10-26T01:38:53.570 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6803/3215791846", 2023-10-26T01:38:53.570 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6807/3215791846", 2023-10-26T01:38:53.570 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6805/3215791846", 2023-10-26T01:38:53.570 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:38:53.570 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:38:53.571 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:38:53.571 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:53.571 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:53.571 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:53.571 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 3, 2023-10-26T01:38:53.571 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:38:53.572 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:38:53.572 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:38:53.572 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:38:53.572 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:38:53.572 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:38:53.572 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:38:53.573 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 14, 2023-10-26T01:38:53.573 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 132, 2023-10-26T01:38:53.573 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:38:53.573 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:38:53.573 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:38:53.573 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:53.574 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:53.574 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:53.574 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6824", 2023-10-26T01:38:53.574 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:38:53.574 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:53.574 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:53.575 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:53.575 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6825", 2023-10-26T01:38:53.575 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:38:53.575 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:53.575 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:53.576 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:53.576 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:38:53.576 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:53.576 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:53.576 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:53.576 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6826", 2023-10-26T01:38:53.577 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:38:53.577 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:53.577 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:53.577 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:53.577 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6827", 2023-10-26T01:38:53.577 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:38:53.578 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:53.578 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:53.578 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:53.578 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:38:53.578 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:53.579 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:53.579 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:53.579 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6830", 2023-10-26T01:38:53.579 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:38:53.579 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:53.579 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:53.579 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:53.580 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6831", 2023-10-26T01:38:53.580 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:38:53.580 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:53.580 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:53.580 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:53.581 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:38:53.581 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:53.581 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:53.581 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:53.581 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6828", 2023-10-26T01:38:53.581 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:38:53.582 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:53.582 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:53.582 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:53.582 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6829", 2023-10-26T01:38:53.582 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:38:53.582 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:53.583 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:53.583 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:53.583 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6825/2256701226", 2023-10-26T01:38:53.583 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6827/2256701226", 2023-10-26T01:38:53.583 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6831/2256701226", 2023-10-26T01:38:53.584 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6829/2256701226", 2023-10-26T01:38:53.584 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:38:53.584 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:38:53.584 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:38:53.584 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:53.584 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:53.585 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:53.585 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 4, 2023-10-26T01:38:53.585 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:38:53.585 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:38:53.585 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:38:53.585 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:38:53.586 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:38:53.586 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:38:53.586 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:38:53.586 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:38:53.586 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 132, 2023-10-26T01:38:53.587 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:38:53.587 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:38:53.587 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:38:53.587 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:53.587 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:53.587 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:53.588 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6816", 2023-10-26T01:38:53.588 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:38:53.588 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:53.588 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:53.588 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:53.588 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6817", 2023-10-26T01:38:53.589 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:38:53.589 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:53.589 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:53.589 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:53.589 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:38:53.589 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:53.590 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:53.590 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:53.590 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6818", 2023-10-26T01:38:53.590 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:38:53.590 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:53.591 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:53.591 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:53.591 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6819", 2023-10-26T01:38:53.593 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:38:53.593 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:53.593 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:53.593 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:53.594 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:38:53.594 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:53.594 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:53.594 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:53.594 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6825", 2023-10-26T01:38:53.594 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:38:53.595 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:53.595 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:53.595 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:53.595 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6828", 2023-10-26T01:38:53.595 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:38:53.595 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:53.596 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:53.596 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:53.596 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:38:53.596 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:53.596 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:53.596 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:53.597 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6820", 2023-10-26T01:38:53.597 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:38:53.597 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:53.597 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:53.597 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:53.598 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6823", 2023-10-26T01:38:53.598 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:38:53.598 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:53.598 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:53.598 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:53.598 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6817/1082279874", 2023-10-26T01:38:53.599 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6819/1082279874", 2023-10-26T01:38:53.599 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6828/1082279874", 2023-10-26T01:38:53.599 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6823/1082279874", 2023-10-26T01:38:53.599 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:38:53.599 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:38:53.599 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:38:53.600 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:53.600 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:53.600 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:53.600 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 5, 2023-10-26T01:38:53.600 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:38:53.601 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:38:53.601 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:38:53.601 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:38:53.601 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:38:53.601 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:38:53.601 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:38:53.602 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:38:53.602 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 132, 2023-10-26T01:38:53.602 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:38:53.602 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:38:53.602 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:38:53.602 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:53.603 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:53.603 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:53.603 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6808", 2023-10-26T01:38:53.603 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:38:53.603 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:53.603 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:53.604 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:53.604 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6809", 2023-10-26T01:38:53.604 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:38:53.604 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:53.604 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:53.604 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:53.605 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:38:53.605 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:53.605 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:53.605 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:53.605 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6810", 2023-10-26T01:38:53.605 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:38:53.606 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:53.606 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:53.606 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:53.606 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6811", 2023-10-26T01:38:53.606 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:38:53.607 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:53.607 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:53.607 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:53.607 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:38:53.607 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:53.607 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:53.608 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:53.608 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6814", 2023-10-26T01:38:53.608 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:38:53.608 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:53.608 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:53.608 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:53.609 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6815", 2023-10-26T01:38:53.609 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:38:53.609 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:53.609 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:53.609 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:53.609 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:38:53.610 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:53.610 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:53.610 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:53.610 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6812", 2023-10-26T01:38:53.610 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:38:53.611 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:53.611 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:53.611 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:53.611 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6813", 2023-10-26T01:38:53.611 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:38:53.611 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:53.612 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:53.612 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:53.612 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6809/3335654351", 2023-10-26T01:38:53.612 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6811/3335654351", 2023-10-26T01:38:53.612 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6815/3335654351", 2023-10-26T01:38:53.612 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6813/3335654351", 2023-10-26T01:38:53.613 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:38:53.613 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:38:53.613 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:38:53.613 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:53.613 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:53.613 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:53.614 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 6, 2023-10-26T01:38:53.614 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:38:53.614 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:38:53.614 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:38:53.614 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:38:53.614 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:38:53.615 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:38:53.615 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:38:53.615 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:38:53.615 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 132, 2023-10-26T01:38:53.615 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:38:53.615 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:38:53.616 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:38:53.616 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:53.616 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:53.616 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:53.616 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6800", 2023-10-26T01:38:53.616 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:38:53.617 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:53.617 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:53.617 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:53.617 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6801", 2023-10-26T01:38:53.617 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:38:53.617 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:53.618 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:53.618 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:53.618 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:38:53.618 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:53.618 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:53.619 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:53.619 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6802", 2023-10-26T01:38:53.619 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:38:53.619 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:53.619 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:53.619 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:53.620 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6803", 2023-10-26T01:38:53.620 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:38:53.620 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:53.620 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:53.620 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:53.620 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:38:53.621 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:53.621 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:53.621 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:53.621 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6806", 2023-10-26T01:38:53.621 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:38:53.622 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:53.622 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:53.622 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:53.622 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6807", 2023-10-26T01:38:53.622 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:38:53.622 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:53.623 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:53.623 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:53.623 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:38:53.623 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:53.623 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:53.623 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:53.624 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6804", 2023-10-26T01:38:53.624 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:38:53.624 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:53.624 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:53.624 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:53.624 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6805", 2023-10-26T01:38:53.625 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:38:53.625 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:53.625 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:53.625 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:53.625 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6801/2520460697", 2023-10-26T01:38:53.625 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6803/2520460697", 2023-10-26T01:38:53.626 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6807/2520460697", 2023-10-26T01:38:53.626 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6805/2520460697", 2023-10-26T01:38:53.626 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:38:53.626 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:38:53.626 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:38:53.627 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:53.627 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:53.627 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:53.627 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 7, 2023-10-26T01:38:53.627 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:38:53.627 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:38:53.628 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:38:53.628 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:38:53.628 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:38:53.628 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:38:53.628 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:38:53.628 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:38:53.629 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 132, 2023-10-26T01:38:53.629 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:38:53.629 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:38:53.629 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:38:53.629 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:53.629 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:53.630 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:53.630 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6821", 2023-10-26T01:38:53.630 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:38:53.630 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:53.630 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:53.631 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:53.631 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6822", 2023-10-26T01:38:53.631 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:38:53.631 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:53.631 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:53.631 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:53.632 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:38:53.632 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:53.632 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:53.632 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:53.632 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6824", 2023-10-26T01:38:53.632 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:38:53.633 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:53.633 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:53.633 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:53.633 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6826", 2023-10-26T01:38:53.633 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:38:53.633 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:53.634 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:53.634 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:53.634 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:38:53.634 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:53.634 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:53.635 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:53.635 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6830", 2023-10-26T01:38:53.635 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:38:53.635 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:53.635 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:53.636 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:53.636 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6831", 2023-10-26T01:38:53.636 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:38:53.636 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:53.636 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:53.636 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:53.637 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:38:53.637 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:53.637 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:53.637 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:53.637 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6827", 2023-10-26T01:38:53.638 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:38:53.638 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:53.638 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:53.638 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:53.638 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6829", 2023-10-26T01:38:53.639 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:38:53.639 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:53.639 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:53.639 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:53.639 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6822/1115930619", 2023-10-26T01:38:53.640 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6826/1115930619", 2023-10-26T01:38:53.640 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6831/1115930619", 2023-10-26T01:38:53.640 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6829/1115930619", 2023-10-26T01:38:53.640 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:38:53.640 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:38:53.640 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:38:53.641 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:53.641 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:53.641 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:38:53.641 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_xinfo": [ 2023-10-26T01:38:53.641 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:53.641 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 0, 2023-10-26T01:38:53.642 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:38:53.642 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:38:53.642 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:38:53.642 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:38:53.642 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:38:53.643 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.889675+0000", 2023-10-26T01:38:53.643 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:38:53.643 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:53.643 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:53.643 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 1, 2023-10-26T01:38:53.644 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:38:53.644 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:38:53.644 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:38:53.644 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:38:53.644 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:38:53.644 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.864157+0000", 2023-10-26T01:38:53.645 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:38:53.645 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:53.645 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:53.645 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 2, 2023-10-26T01:38:53.645 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:38:53.645 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:38:53.646 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:38:53.646 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:38:53.646 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:38:53.646 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.876919+0000", 2023-10-26T01:38:53.646 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:38:53.647 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:53.647 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:53.647 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 3, 2023-10-26T01:38:53.647 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:38:53.647 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:38:53.647 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:38:53.648 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:38:53.648 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:38:53.648 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.974541+0000", 2023-10-26T01:38:53.648 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:38:53.648 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:53.649 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:53.649 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 4, 2023-10-26T01:38:53.649 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:38:53.649 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:38:53.649 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:38:53.649 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:38:53.650 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:38:53.650 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.969696+0000", 2023-10-26T01:38:53.650 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:38:53.650 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:53.650 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:53.650 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 5, 2023-10-26T01:38:53.651 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:38:53.651 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:38:53.651 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:38:53.651 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:38:53.651 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:38:53.651 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.859703+0000", 2023-10-26T01:38:53.652 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:38:53.652 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:53.652 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:53.652 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 6, 2023-10-26T01:38:53.652 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:38:53.653 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:38:53.653 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:38:53.653 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:38:53.653 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:38:53.653 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.867917+0000", 2023-10-26T01:38:53.653 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:38:53.654 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:53.654 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:53.654 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 7, 2023-10-26T01:38:53.654 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:38:53.654 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:38:53.655 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:38:53.655 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:38:53.655 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:38:53.655 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.898605+0000", 2023-10-26T01:38:53.655 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:38:53.655 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:53.656 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:38:53.656 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_upmap": [], 2023-10-26T01:38:53.656 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_upmap_items": [ 2023-10-26T01:38:53.656 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:53.656 INFO:teuthology.orchestra.run.smithi169.stdout: "pgid": "17.e", 2023-10-26T01:38:53.656 INFO:teuthology.orchestra.run.smithi169.stdout: "mappings": [ 2023-10-26T01:38:53.657 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:53.657 INFO:teuthology.orchestra.run.smithi169.stdout: "from": 0, 2023-10-26T01:38:53.657 INFO:teuthology.orchestra.run.smithi169.stdout: "to": 5 2023-10-26T01:38:53.657 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:53.657 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:53.657 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:53.658 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:53.658 INFO:teuthology.orchestra.run.smithi169.stdout: "pgid": "17.21", 2023-10-26T01:38:53.658 INFO:teuthology.orchestra.run.smithi169.stdout: "mappings": [ 2023-10-26T01:38:53.658 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:53.658 INFO:teuthology.orchestra.run.smithi169.stdout: "from": 0, 2023-10-26T01:38:53.659 INFO:teuthology.orchestra.run.smithi169.stdout: "to": 2 2023-10-26T01:38:53.659 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:53.659 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:53.659 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:53.659 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:38:53.659 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_upmap_primaries": [], 2023-10-26T01:38:53.660 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_temp": [], 2023-10-26T01:38:53.660 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_temp": [], 2023-10-26T01:38:53.660 INFO:teuthology.orchestra.run.smithi169.stdout: "blocklist": {}, 2023-10-26T01:38:53.660 INFO:teuthology.orchestra.run.smithi169.stdout: "range_blocklist": {}, 2023-10-26T01:38:53.660 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profiles": { 2023-10-26T01:38:53.660 INFO:teuthology.orchestra.run.smithi169.stdout: "default": { 2023-10-26T01:38:53.661 INFO:teuthology.orchestra.run.smithi169.stdout: "crush-failure-domain": "osd", 2023-10-26T01:38:53.661 INFO:teuthology.orchestra.run.smithi169.stdout: "k": "2", 2023-10-26T01:38:53.661 INFO:teuthology.orchestra.run.smithi169.stdout: "m": "1", 2023-10-26T01:38:53.661 INFO:teuthology.orchestra.run.smithi169.stdout: "plugin": "jerasure", 2023-10-26T01:38:53.661 INFO:teuthology.orchestra.run.smithi169.stdout: "technique": "reed_sol_van" 2023-10-26T01:38:53.661 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:53.662 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:53.662 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps_queue": [], 2023-10-26T01:38:53.662 INFO:teuthology.orchestra.run.smithi169.stdout: "new_removed_snaps": [], 2023-10-26T01:38:53.662 INFO:teuthology.orchestra.run.smithi169.stdout: "new_purged_snaps": [], 2023-10-26T01:38:53.662 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_node_flags": {}, 2023-10-26T01:38:53.663 INFO:teuthology.orchestra.run.smithi169.stdout: "device_class_flags": {}, 2023-10-26T01:38:53.663 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_mode": { 2023-10-26T01:38:53.663 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_mode_enabled": false, 2023-10-26T01:38:53.663 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_bucket_count": 0, 2023-10-26T01:38:53.663 INFO:teuthology.orchestra.run.smithi169.stdout: "degraded_stretch_mode": 0, 2023-10-26T01:38:53.664 INFO:teuthology.orchestra.run.smithi169.stdout: "recovering_stretch_mode": 0, 2023-10-26T01:38:53.664 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_mode_bucket": 0 2023-10-26T01:38:53.664 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:53.664 INFO:teuthology.orchestra.run.smithi169.stdout:} 2023-10-26T01:38:53.665 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:53.472+0000 7f3f4656c640 1 -- 172.21.15.169:0/3986871535 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3f2406ae40 msgr2=0x7f3f2406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:53.666 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:53.472+0000 7f3f4656c640 1 --2- 172.21.15.169:0/3986871535 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3f2406ae40 0x7f3f2406d300 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f3f400c7500 tx=0x7f3f30005f90 comp rx=0 tx=0).stop 2023-10-26T01:38:53.666 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:53.472+0000 7f3f4656c640 1 -- 172.21.15.169:0/3986871535 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3f4009a5c0 msgr2=0x7f3f40157e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:53.666 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:53.472+0000 7f3f4656c640 1 --2- 172.21.15.169:0/3986871535 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3f4009a5c0 0x7f3f40157e90 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c006010 tx=0x7f3f3c02da70 comp rx=0 tx=0).stop 2023-10-26T01:38:53.666 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:53.472+0000 7f3f4656c640 1 -- 172.21.15.169:0/3986871535 shutdown_connections 2023-10-26T01:38:53.666 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:53.472+0000 7f3f4656c640 1 --2- 172.21.15.169:0/3986871535 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3f2406ae40 0x7f3f2406d300 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:53.667 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:53.472+0000 7f3f4656c640 1 --2- 172.21.15.169:0/3986871535 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3f400c9970 0x7f3f400c8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:53.667 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:53.472+0000 7f3f4656c640 1 --2- 172.21.15.169:0/3986871535 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3f400ac6d0 0x7f3f40070ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:53.667 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:53.472+0000 7f3f4656c640 1 --2- 172.21.15.169:0/3986871535 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3f4009a5c0 0x7f3f40157e90 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:53.667 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:53.472+0000 7f3f4656c640 1 -- 172.21.15.169:0/3986871535 >> 172.21.15.169:0/3986871535 conn(0x7f3f400aedd0 msgr2=0x7f3f40142630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:53.668 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:53.473+0000 7f3f4656c640 1 -- 172.21.15.169:0/3986871535 shutdown_connections 2023-10-26T01:38:53.668 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:53.473+0000 7f3f4656c640 1 -- 172.21.15.169:0/3986871535 wait complete. 2023-10-26T01:38:53.668 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2023-10-26T01:38:58.669 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-26T01:38:58.894 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:58.892+0000 7f4d4493c640 1 -- 172.21.15.169:0/1936840403 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d2c003970 msgr2=0x7f4d2c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:58.895 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:58.892+0000 7f4d4493c640 1 --2- 172.21.15.169:0/1936840403 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d2c003970 0x7f4d2c003d90 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f4d3000bb20 tx=0x7f4d2c0057f0 comp rx=0 tx=0).stop 2023-10-26T01:38:58.895 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:58.892+0000 7f4d3ffff640 1 -- 172.21.15.169:0/1936840403 reap_dead start 2023-10-26T01:38:58.895 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:58.893+0000 7f4d4493c640 1 -- 172.21.15.169:0/1936840403 shutdown_connections 2023-10-26T01:38:58.895 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:58.893+0000 7f4d4493c640 1 -- 172.21.15.169:0/1936840403 >> 172.21.15.169:0/1936840403 conn(0x7f4d400aedd0 msgr2=0x7f4d400af1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:58.896 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:58.893+0000 7f4d4493c640 1 -- 172.21.15.169:0/1936840403 shutdown_connections 2023-10-26T01:38:58.896 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:58.893+0000 7f4d4493c640 1 -- 172.21.15.169:0/1936840403 wait complete. 2023-10-26T01:38:58.896 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:58.894+0000 7f4d4493c640 1 Processor -- start 2023-10-26T01:38:58.897 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:58.895+0000 7f4d4493c640 1 -- start start 2023-10-26T01:38:58.898 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:58.896+0000 7f4d4493c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4d2c003970 0x7f4d4015df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:58.899 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:58.896+0000 7f4d4493c640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4d4015c250 0x7f4d40141340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:58.899 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:58.896+0000 7f4d4493c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d40141c20 0x7f4d400cb630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:58.899 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:58.896+0000 7f4d4493c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d40070a00 con 0x7f4d40141c20 2023-10-26T01:38:58.899 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:58.896+0000 7f4d4493c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d40147b40 con 0x7f4d2c003970 2023-10-26T01:38:58.900 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:58.896+0000 7f4d4493c640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d40070680 con 0x7f4d4015c250 2023-10-26T01:38:58.900 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:58.896+0000 7f4d3ffff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d40141c20 0x7f4d400cb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:58.900 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:58.896+0000 7f4d3ffff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d40141c20 0x7f4d400cb630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:44778/0 (socket says 172.21.15.169:44778) 2023-10-26T01:38:58.900 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:58.896+0000 7f4d3ffff640 1 -- 172.21.15.169:0/3474583634 learned_addr learned my addr 172.21.15.169:0/3474583634 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:38:58.900 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:58.896+0000 7f4d3f7fe640 1 --2- 172.21.15.169:0/3474583634 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4d2c003970 0x7f4d4015df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:58.901 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:58.896+0000 7f4d3ffff640 1 -- 172.21.15.169:0/3474583634 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4d4015c250 msgr2=0x7f4d40141340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:58.901 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:58.896+0000 7f4d3ffff640 1 --2- 172.21.15.169:0/3474583634 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4d4015c250 0x7f4d40141340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:58.901 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:58.896+0000 7f4d3ffff640 1 -- 172.21.15.169:0/3474583634 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4d2c003970 msgr2=0x7f4d4015df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:58.901 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:58.896+0000 7f4d3ffff640 1 --2- 172.21.15.169:0/3474583634 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4d2c003970 0x7f4d4015df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:58.901 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:58.896+0000 7f4d3ffff640 1 -- 172.21.15.169:0/3474583634 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d40157d50 con 0x7f4d40141c20 2023-10-26T01:38:58.901 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:58.897+0000 7f4d3ffff640 1 --2- 172.21.15.169:0/3474583634 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d40141c20 0x7f4d400cb630 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f4d400cce10 tx=0x7f4d3800dd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:58.902 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:58.897+0000 7f4d3cff9640 1 -- 172.21.15.169:0/3474583634 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d38015030 con 0x7f4d40141c20 2023-10-26T01:38:58.902 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:58.897+0000 7f4d3cff9640 1 -- 172.21.15.169:0/3474583634 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4d38003e90 con 0x7f4d40141c20 2023-10-26T01:38:58.902 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:58.897+0000 7f4d3cff9640 1 -- 172.21.15.169:0/3474583634 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d3800ba20 con 0x7f4d40141c20 2023-10-26T01:38:58.902 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:58.897+0000 7f4d4493c640 1 -- 172.21.15.169:0/3474583634 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d400ca190 con 0x7f4d40141c20 2023-10-26T01:38:58.902 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:58.897+0000 7f4d4493c640 1 -- 172.21.15.169:0/3474583634 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d4015cd40 con 0x7f4d40141c20 2023-10-26T01:38:58.903 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:58.898+0000 7f4d4493c640 1 -- 172.21.15.169:0/3474583634 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d40070b40 con 0x7f4d40141c20 2023-10-26T01:38:58.904 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:58.899+0000 7f4d3cff9640 1 -- 172.21.15.169:0/3474583634 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f4d38018070 con 0x7f4d40141c20 2023-10-26T01:38:58.904 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:58.900+0000 7f4d3cff9640 1 --2- 172.21.15.169:0/3474583634 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f4d2006ae40 0x7f4d2006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:58.904 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:58.900+0000 7f4d3cff9640 1 -- 172.21.15.169:0/3474583634 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f4d38016070 con 0x7f4d40141c20 2023-10-26T01:38:58.904 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:58.901+0000 7f4d3f7fe640 1 --2- 172.21.15.169:0/3474583634 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f4d2006ae40 0x7f4d2006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:58.905 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:58.902+0000 7f4d3f7fe640 1 --2- 172.21.15.169:0/3474583634 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f4d2006ae40 0x7f4d2006d300 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f4d30037000 tx=0x7f4d3002e040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:58.906 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:58.905+0000 7f4d3cff9640 1 -- 172.21.15.169:0/3474583634 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4d40070b40 con 0x7f4d40141c20 2023-10-26T01:38:59.096 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:59.094+0000 7f4d4493c640 1 -- 172.21.15.169:0/3474583634 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f4d4006ae10 con 0x7f4d40141c20 2023-10-26T01:38:59.174 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:59.098+0000 7f4d3cff9640 1 -- 172.21.15.169:0/3474583634 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v139) v1 ==== 81+0+34450 (secure 0 0 0) 0x7f4d4006ae10 con 0x7f4d40141c20 2023-10-26T01:38:59.174 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:38:59.175 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2023-10-26T01:38:59.175 INFO:teuthology.orchestra.run.smithi169.stdout: "epoch": 139, 2023-10-26T01:38:59.175 INFO:teuthology.orchestra.run.smithi169.stdout: "fsid": "30203b28-ed8a-44e8-a9a6-a9bf25f980f1", 2023-10-26T01:38:59.175 INFO:teuthology.orchestra.run.smithi169.stdout: "created": "2023-10-26T01:30:22.897155+0000", 2023-10-26T01:38:59.175 INFO:teuthology.orchestra.run.smithi169.stdout: "modified": "2023-10-26T01:38:58.061086+0000", 2023-10-26T01:38:59.176 INFO:teuthology.orchestra.run.smithi169.stdout: "last_up_change": "2023-10-26T01:30:31.875281+0000", 2023-10-26T01:38:59.176 INFO:teuthology.orchestra.run.smithi169.stdout: "last_in_change": "2023-10-26T01:30:27.637941+0000", 2023-10-26T01:38:59.176 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-26T01:38:59.176 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_num": 5799936, 2023-10-26T01:38:59.176 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_set": [ 2023-10-26T01:38:59.177 INFO:teuthology.orchestra.run.smithi169.stdout: "pglog_hardlimit", 2023-10-26T01:38:59.177 INFO:teuthology.orchestra.run.smithi169.stdout: "purged_snapdirs", 2023-10-26T01:38:59.177 INFO:teuthology.orchestra.run.smithi169.stdout: "recovery_deletes", 2023-10-26T01:38:59.177 INFO:teuthology.orchestra.run.smithi169.stdout: "sortbitwise" 2023-10-26T01:38:59.177 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:38:59.178 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_version": 4, 2023-10-26T01:38:59.178 INFO:teuthology.orchestra.run.smithi169.stdout: "full_ratio": 0.69999998807907104, 2023-10-26T01:38:59.178 INFO:teuthology.orchestra.run.smithi169.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-26T01:38:59.178 INFO:teuthology.orchestra.run.smithi169.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-26T01:38:59.178 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_snapshot": "", 2023-10-26T01:38:59.179 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_max": 17, 2023-10-26T01:38:59.179 INFO:teuthology.orchestra.run.smithi169.stdout: "max_osd": 8, 2023-10-26T01:38:59.179 INFO:teuthology.orchestra.run.smithi169.stdout: "require_min_compat_client": "luminous", 2023-10-26T01:38:59.179 INFO:teuthology.orchestra.run.smithi169.stdout: "min_compat_client": "luminous", 2023-10-26T01:38:59.179 INFO:teuthology.orchestra.run.smithi169.stdout: "require_osd_release": "reef", 2023-10-26T01:38:59.179 INFO:teuthology.orchestra.run.smithi169.stdout: "allow_crimson": false, 2023-10-26T01:38:59.180 INFO:teuthology.orchestra.run.smithi169.stdout: "pools": [ 2023-10-26T01:38:59.180 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.180 INFO:teuthology.orchestra.run.smithi169.stdout: "pool": 1, 2023-10-26T01:38:59.180 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_name": ".mgr", 2023-10-26T01:38:59.180 INFO:teuthology.orchestra.run.smithi169.stdout: "create_time": "2023-10-26T01:30:32.200808+0000", 2023-10-26T01:38:59.181 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": 1, 2023-10-26T01:38:59.181 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_names": "hashpspool", 2023-10-26T01:38:59.181 INFO:teuthology.orchestra.run.smithi169.stdout: "type": 1, 2023-10-26T01:38:59.181 INFO:teuthology.orchestra.run.smithi169.stdout: "size": 2, 2023-10-26T01:38:59.181 INFO:teuthology.orchestra.run.smithi169.stdout: "min_size": 1, 2023-10-26T01:38:59.182 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_rule": 0, 2023-10-26T01:38:59.182 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_count": 0, 2023-10-26T01:38:59.182 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_target": 0, 2023-10-26T01:38:59.182 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T01:38:59.182 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T01:38:59.183 INFO:teuthology.orchestra.run.smithi169.stdout: "object_hash": 2, 2023-10-26T01:38:59.183 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_mode": "off", 2023-10-26T01:38:59.183 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num": 1, 2023-10-26T01:38:59.183 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num": 1, 2023-10-26T01:38:59.183 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num_target": 1, 2023-10-26T01:38:59.184 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_target": 1, 2023-10-26T01:38:59.184 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_pending": 1, 2023-10-26T01:38:59.184 INFO:teuthology.orchestra.run.smithi169.stdout: "last_pg_merge_meta": { 2023-10-26T01:38:59.184 INFO:teuthology.orchestra.run.smithi169.stdout: "source_pgid": "0.0", 2023-10-26T01:38:59.184 INFO:teuthology.orchestra.run.smithi169.stdout: "ready_epoch": 0, 2023-10-26T01:38:59.184 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_started": 0, 2023-10-26T01:38:59.185 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_clean": 0, 2023-10-26T01:38:59.185 INFO:teuthology.orchestra.run.smithi169.stdout: "source_version": "0'0", 2023-10-26T01:38:59.185 INFO:teuthology.orchestra.run.smithi169.stdout: "target_version": "0'0" 2023-10-26T01:38:59.185 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.186 INFO:teuthology.orchestra.run.smithi169.stdout: "last_change": "109", 2023-10-26T01:38:59.186 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend": "0", 2023-10-26T01:38:59.186 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T01:38:59.186 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T01:38:59.186 INFO:teuthology.orchestra.run.smithi169.stdout: "auid": 0, 2023-10-26T01:38:59.187 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_mode": "selfmanaged", 2023-10-26T01:38:59.187 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_seq": 0, 2023-10-26T01:38:59.187 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_epoch": 0, 2023-10-26T01:38:59.187 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_snaps": [], 2023-10-26T01:38:59.187 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps": "[]", 2023-10-26T01:38:59.187 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_bytes": 0, 2023-10-26T01:38:59.188 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_objects": 0, 2023-10-26T01:38:59.188 INFO:teuthology.orchestra.run.smithi169.stdout: "tiers": [], 2023-10-26T01:38:59.188 INFO:teuthology.orchestra.run.smithi169.stdout: "tier_of": -1, 2023-10-26T01:38:59.188 INFO:teuthology.orchestra.run.smithi169.stdout: "read_tier": -1, 2023-10-26T01:38:59.188 INFO:teuthology.orchestra.run.smithi169.stdout: "write_tier": -1, 2023-10-26T01:38:59.189 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_mode": "none", 2023-10-26T01:38:59.189 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_bytes": 0, 2023-10-26T01:38:59.189 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_objects": 0, 2023-10-26T01:38:59.189 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T01:38:59.189 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T01:38:59.190 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T01:38:59.190 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_flush_age": 0, 2023-10-26T01:38:59.190 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_evict_age": 0, 2023-10-26T01:38:59.190 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profile": "", 2023-10-26T01:38:59.190 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_params": { 2023-10-26T01:38:59.191 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "none" 2023-10-26T01:38:59.191 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.191 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_period": 0, 2023-10-26T01:38:59.191 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_count": 0, 2023-10-26T01:38:59.191 INFO:teuthology.orchestra.run.smithi169.stdout: "use_gmt_hitset": true, 2023-10-26T01:38:59.192 INFO:teuthology.orchestra.run.smithi169.stdout: "min_read_recency_for_promote": 0, 2023-10-26T01:38:59.192 INFO:teuthology.orchestra.run.smithi169.stdout: "min_write_recency_for_promote": 0, 2023-10-26T01:38:59.192 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T01:38:59.192 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_search_last_n": 0, 2023-10-26T01:38:59.192 INFO:teuthology.orchestra.run.smithi169.stdout: "grade_table": [], 2023-10-26T01:38:59.193 INFO:teuthology.orchestra.run.smithi169.stdout: "stripe_width": 0, 2023-10-26T01:38:59.193 INFO:teuthology.orchestra.run.smithi169.stdout: "expected_num_objects": 0, 2023-10-26T01:38:59.193 INFO:teuthology.orchestra.run.smithi169.stdout: "fast_read": false, 2023-10-26T01:38:59.193 INFO:teuthology.orchestra.run.smithi169.stdout: "options": { 2023-10-26T01:38:59.193 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_max": 32, 2023-10-26T01:38:59.193 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_min": 1 2023-10-26T01:38:59.194 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.194 INFO:teuthology.orchestra.run.smithi169.stdout: "application_metadata": { 2023-10-26T01:38:59.194 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": {} 2023-10-26T01:38:59.194 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.194 INFO:teuthology.orchestra.run.smithi169.stdout: "read_balance": { 2023-10-26T01:38:59.195 INFO:teuthology.orchestra.run.smithi169.stdout: "score_acting": 8, 2023-10-26T01:38:59.195 INFO:teuthology.orchestra.run.smithi169.stdout: "score_stable": 8, 2023-10-26T01:38:59.195 INFO:teuthology.orchestra.run.smithi169.stdout: "optimal_score": 0.25, 2023-10-26T01:38:59.195 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_acting": 2, 2023-10-26T01:38:59.195 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_stable": 2, 2023-10-26T01:38:59.196 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity_weighted": 1, 2023-10-26T01:38:59.196 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity": 1, 2023-10-26T01:38:59.196 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity_weighted": 1 2023-10-26T01:38:59.196 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:59.196 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.196 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.197 INFO:teuthology.orchestra.run.smithi169.stdout: "pool": 16, 2023-10-26T01:38:59.197 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_name": "cephfs_metadata", 2023-10-26T01:38:59.197 INFO:teuthology.orchestra.run.smithi169.stdout: "create_time": "2023-10-26T01:38:18.617877+0000", 2023-10-26T01:38:59.197 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": 1, 2023-10-26T01:38:59.197 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_names": "hashpspool", 2023-10-26T01:38:59.198 INFO:teuthology.orchestra.run.smithi169.stdout: "type": 1, 2023-10-26T01:38:59.198 INFO:teuthology.orchestra.run.smithi169.stdout: "size": 2, 2023-10-26T01:38:59.198 INFO:teuthology.orchestra.run.smithi169.stdout: "min_size": 1, 2023-10-26T01:38:59.198 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_rule": 0, 2023-10-26T01:38:59.198 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_count": 0, 2023-10-26T01:38:59.199 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_target": 0, 2023-10-26T01:38:59.199 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T01:38:59.199 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T01:38:59.199 INFO:teuthology.orchestra.run.smithi169.stdout: "object_hash": 2, 2023-10-26T01:38:59.199 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_mode": "off", 2023-10-26T01:38:59.199 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num": 32, 2023-10-26T01:38:59.200 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num": 32, 2023-10-26T01:38:59.200 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num_target": 32, 2023-10-26T01:38:59.200 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_target": 32, 2023-10-26T01:38:59.200 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_pending": 32, 2023-10-26T01:38:59.200 INFO:teuthology.orchestra.run.smithi169.stdout: "last_pg_merge_meta": { 2023-10-26T01:38:59.201 INFO:teuthology.orchestra.run.smithi169.stdout: "source_pgid": "0.0", 2023-10-26T01:38:59.201 INFO:teuthology.orchestra.run.smithi169.stdout: "ready_epoch": 0, 2023-10-26T01:38:59.201 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_started": 0, 2023-10-26T01:38:59.201 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_clean": 0, 2023-10-26T01:38:59.201 INFO:teuthology.orchestra.run.smithi169.stdout: "source_version": "0'0", 2023-10-26T01:38:59.202 INFO:teuthology.orchestra.run.smithi169.stdout: "target_version": "0'0" 2023-10-26T01:38:59.202 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.202 INFO:teuthology.orchestra.run.smithi169.stdout: "last_change": "133", 2023-10-26T01:38:59.202 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend": "0", 2023-10-26T01:38:59.202 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T01:38:59.203 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T01:38:59.203 INFO:teuthology.orchestra.run.smithi169.stdout: "auid": 0, 2023-10-26T01:38:59.203 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_mode": "selfmanaged", 2023-10-26T01:38:59.203 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_seq": 0, 2023-10-26T01:38:59.203 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_epoch": 0, 2023-10-26T01:38:59.203 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_snaps": [], 2023-10-26T01:38:59.204 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps": "[]", 2023-10-26T01:38:59.204 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_bytes": 0, 2023-10-26T01:38:59.204 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_objects": 0, 2023-10-26T01:38:59.204 INFO:teuthology.orchestra.run.smithi169.stdout: "tiers": [], 2023-10-26T01:38:59.204 INFO:teuthology.orchestra.run.smithi169.stdout: "tier_of": -1, 2023-10-26T01:38:59.205 INFO:teuthology.orchestra.run.smithi169.stdout: "read_tier": -1, 2023-10-26T01:38:59.205 INFO:teuthology.orchestra.run.smithi169.stdout: "write_tier": -1, 2023-10-26T01:38:59.205 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_mode": "none", 2023-10-26T01:38:59.205 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_bytes": 0, 2023-10-26T01:38:59.205 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_objects": 0, 2023-10-26T01:38:59.205 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T01:38:59.206 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T01:38:59.206 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T01:38:59.206 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_flush_age": 0, 2023-10-26T01:38:59.206 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_evict_age": 0, 2023-10-26T01:38:59.206 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profile": "", 2023-10-26T01:38:59.207 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_params": { 2023-10-26T01:38:59.207 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "none" 2023-10-26T01:38:59.207 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.207 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_period": 0, 2023-10-26T01:38:59.207 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_count": 0, 2023-10-26T01:38:59.208 INFO:teuthology.orchestra.run.smithi169.stdout: "use_gmt_hitset": true, 2023-10-26T01:38:59.208 INFO:teuthology.orchestra.run.smithi169.stdout: "min_read_recency_for_promote": 0, 2023-10-26T01:38:59.208 INFO:teuthology.orchestra.run.smithi169.stdout: "min_write_recency_for_promote": 0, 2023-10-26T01:38:59.208 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T01:38:59.208 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_search_last_n": 0, 2023-10-26T01:38:59.209 INFO:teuthology.orchestra.run.smithi169.stdout: "grade_table": [], 2023-10-26T01:38:59.209 INFO:teuthology.orchestra.run.smithi169.stdout: "stripe_width": 0, 2023-10-26T01:38:59.209 INFO:teuthology.orchestra.run.smithi169.stdout: "expected_num_objects": 0, 2023-10-26T01:38:59.209 INFO:teuthology.orchestra.run.smithi169.stdout: "fast_read": false, 2023-10-26T01:38:59.209 INFO:teuthology.orchestra.run.smithi169.stdout: "options": { 2023-10-26T01:38:59.210 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_bias": 4, 2023-10-26T01:38:59.210 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_min": 16, 2023-10-26T01:38:59.210 INFO:teuthology.orchestra.run.smithi169.stdout: "recovery_priority": 5 2023-10-26T01:38:59.210 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.210 INFO:teuthology.orchestra.run.smithi169.stdout: "application_metadata": { 2023-10-26T01:38:59.210 INFO:teuthology.orchestra.run.smithi169.stdout: "cephfs": { 2023-10-26T01:38:59.211 INFO:teuthology.orchestra.run.smithi169.stdout: "metadata": "cephfs" 2023-10-26T01:38:59.211 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:59.211 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.211 INFO:teuthology.orchestra.run.smithi169.stdout: "read_balance": { 2023-10-26T01:38:59.211 INFO:teuthology.orchestra.run.smithi169.stdout: "score_acting": 1.75, 2023-10-26T01:38:59.212 INFO:teuthology.orchestra.run.smithi169.stdout: "score_stable": 1.75, 2023-10-26T01:38:59.212 INFO:teuthology.orchestra.run.smithi169.stdout: "optimal_score": 1, 2023-10-26T01:38:59.212 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_acting": 1.75, 2023-10-26T01:38:59.212 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_stable": 1.75, 2023-10-26T01:38:59.212 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity_weighted": 1, 2023-10-26T01:38:59.212 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity": 1, 2023-10-26T01:38:59.213 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity_weighted": 1 2023-10-26T01:38:59.213 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:59.213 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.213 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.213 INFO:teuthology.orchestra.run.smithi169.stdout: "pool": 17, 2023-10-26T01:38:59.214 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_name": "cephfs_data", 2023-10-26T01:38:59.214 INFO:teuthology.orchestra.run.smithi169.stdout: "create_time": "2023-10-26T01:38:19.793041+0000", 2023-10-26T01:38:59.214 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": 1, 2023-10-26T01:38:59.214 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_names": "hashpspool", 2023-10-26T01:38:59.214 INFO:teuthology.orchestra.run.smithi169.stdout: "type": 1, 2023-10-26T01:38:59.215 INFO:teuthology.orchestra.run.smithi169.stdout: "size": 2, 2023-10-26T01:38:59.215 INFO:teuthology.orchestra.run.smithi169.stdout: "min_size": 1, 2023-10-26T01:38:59.215 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_rule": 0, 2023-10-26T01:38:59.215 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_count": 0, 2023-10-26T01:38:59.215 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_target": 0, 2023-10-26T01:38:59.215 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T01:38:59.216 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T01:38:59.216 INFO:teuthology.orchestra.run.smithi169.stdout: "object_hash": 2, 2023-10-26T01:38:59.216 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_mode": "off", 2023-10-26T01:38:59.216 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num": 64, 2023-10-26T01:38:59.216 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num": 64, 2023-10-26T01:38:59.217 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num_target": 64, 2023-10-26T01:38:59.217 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_target": 64, 2023-10-26T01:38:59.217 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_pending": 64, 2023-10-26T01:38:59.217 INFO:teuthology.orchestra.run.smithi169.stdout: "last_pg_merge_meta": { 2023-10-26T01:38:59.217 INFO:teuthology.orchestra.run.smithi169.stdout: "source_pgid": "0.0", 2023-10-26T01:38:59.217 INFO:teuthology.orchestra.run.smithi169.stdout: "ready_epoch": 0, 2023-10-26T01:38:59.218 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_started": 0, 2023-10-26T01:38:59.218 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_clean": 0, 2023-10-26T01:38:59.218 INFO:teuthology.orchestra.run.smithi169.stdout: "source_version": "0'0", 2023-10-26T01:38:59.218 INFO:teuthology.orchestra.run.smithi169.stdout: "target_version": "0'0" 2023-10-26T01:38:59.218 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.219 INFO:teuthology.orchestra.run.smithi169.stdout: "last_change": "139", 2023-10-26T01:38:59.219 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend": "0", 2023-10-26T01:38:59.219 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T01:38:59.219 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T01:38:59.219 INFO:teuthology.orchestra.run.smithi169.stdout: "auid": 0, 2023-10-26T01:38:59.220 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_mode": "selfmanaged", 2023-10-26T01:38:59.220 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_seq": 0, 2023-10-26T01:38:59.220 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_epoch": 0, 2023-10-26T01:38:59.220 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_snaps": [], 2023-10-26T01:38:59.220 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps": "[]", 2023-10-26T01:38:59.221 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_bytes": 33554432, 2023-10-26T01:38:59.221 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_objects": 0, 2023-10-26T01:38:59.221 INFO:teuthology.orchestra.run.smithi169.stdout: "tiers": [], 2023-10-26T01:38:59.221 INFO:teuthology.orchestra.run.smithi169.stdout: "tier_of": -1, 2023-10-26T01:38:59.221 INFO:teuthology.orchestra.run.smithi169.stdout: "read_tier": -1, 2023-10-26T01:38:59.221 INFO:teuthology.orchestra.run.smithi169.stdout: "write_tier": -1, 2023-10-26T01:38:59.222 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_mode": "none", 2023-10-26T01:38:59.222 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_bytes": 0, 2023-10-26T01:38:59.222 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_objects": 0, 2023-10-26T01:38:59.222 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T01:38:59.222 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T01:38:59.223 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T01:38:59.223 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_flush_age": 0, 2023-10-26T01:38:59.223 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_evict_age": 0, 2023-10-26T01:38:59.223 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profile": "", 2023-10-26T01:38:59.223 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_params": { 2023-10-26T01:38:59.223 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "none" 2023-10-26T01:38:59.224 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.224 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_period": 0, 2023-10-26T01:38:59.224 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_count": 0, 2023-10-26T01:38:59.224 INFO:teuthology.orchestra.run.smithi169.stdout: "use_gmt_hitset": true, 2023-10-26T01:38:59.224 INFO:teuthology.orchestra.run.smithi169.stdout: "min_read_recency_for_promote": 0, 2023-10-26T01:38:59.225 INFO:teuthology.orchestra.run.smithi169.stdout: "min_write_recency_for_promote": 0, 2023-10-26T01:38:59.225 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T01:38:59.225 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_search_last_n": 0, 2023-10-26T01:38:59.225 INFO:teuthology.orchestra.run.smithi169.stdout: "grade_table": [], 2023-10-26T01:38:59.225 INFO:teuthology.orchestra.run.smithi169.stdout: "stripe_width": 0, 2023-10-26T01:38:59.226 INFO:teuthology.orchestra.run.smithi169.stdout: "expected_num_objects": 0, 2023-10-26T01:38:59.226 INFO:teuthology.orchestra.run.smithi169.stdout: "fast_read": false, 2023-10-26T01:38:59.226 INFO:teuthology.orchestra.run.smithi169.stdout: "options": { 2023-10-26T01:38:59.226 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_min": 64, 2023-10-26T01:38:59.226 INFO:teuthology.orchestra.run.smithi169.stdout: "target_size_ratio": 0.89999997615814209 2023-10-26T01:38:59.226 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.227 INFO:teuthology.orchestra.run.smithi169.stdout: "application_metadata": { 2023-10-26T01:38:59.227 INFO:teuthology.orchestra.run.smithi169.stdout: "cephfs": { 2023-10-26T01:38:59.227 INFO:teuthology.orchestra.run.smithi169.stdout: "data": "cephfs" 2023-10-26T01:38:59.227 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:59.227 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.228 INFO:teuthology.orchestra.run.smithi169.stdout: "read_balance": { 2023-10-26T01:38:59.228 INFO:teuthology.orchestra.run.smithi169.stdout: "score_acting": 1.6299999952316284, 2023-10-26T01:38:59.228 INFO:teuthology.orchestra.run.smithi169.stdout: "score_stable": 1.6299999952316284, 2023-10-26T01:38:59.228 INFO:teuthology.orchestra.run.smithi169.stdout: "optimal_score": 1, 2023-10-26T01:38:59.228 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_acting": 1.6299999952316284, 2023-10-26T01:38:59.228 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_stable": 1.6299999952316284, 2023-10-26T01:38:59.229 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity_weighted": 1, 2023-10-26T01:38:59.229 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity": 1, 2023-10-26T01:38:59.229 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity_weighted": 1 2023-10-26T01:38:59.229 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:59.229 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:59.230 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:38:59.230 INFO:teuthology.orchestra.run.smithi169.stdout: "osds": [ 2023-10-26T01:38:59.230 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.230 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 0, 2023-10-26T01:38:59.230 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:38:59.231 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:38:59.231 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:38:59.231 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:38:59.231 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:38:59.231 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:38:59.231 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:38:59.232 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:38:59.232 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 132, 2023-10-26T01:38:59.232 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:38:59.232 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:38:59.232 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:38:59.233 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:59.233 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.233 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:59.233 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6816", 2023-10-26T01:38:59.233 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:38:59.233 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.234 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.234 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:59.234 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6817", 2023-10-26T01:38:59.234 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:38:59.234 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:59.235 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:59.235 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.235 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:38:59.235 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:59.235 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.236 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:59.236 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6818", 2023-10-26T01:38:59.236 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:38:59.236 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.236 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.236 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:59.237 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6819", 2023-10-26T01:38:59.237 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:38:59.237 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:59.237 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:59.237 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.238 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:38:59.238 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:59.238 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.238 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:59.238 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6822", 2023-10-26T01:38:59.239 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:38:59.239 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.239 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.239 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:59.239 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6823", 2023-10-26T01:38:59.240 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:38:59.240 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:59.240 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:59.240 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.240 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:38:59.240 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:59.241 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.241 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:59.241 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6820", 2023-10-26T01:38:59.241 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:38:59.241 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.242 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.242 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:59.242 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6821", 2023-10-26T01:38:59.242 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:38:59.242 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:59.243 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:59.243 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.243 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6817/4204255864", 2023-10-26T01:38:59.243 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6819/4204255864", 2023-10-26T01:38:59.243 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6823/4204255864", 2023-10-26T01:38:59.243 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6821/4204255864", 2023-10-26T01:38:59.244 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:38:59.244 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:38:59.244 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:38:59.244 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:59.244 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.245 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.245 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 1, 2023-10-26T01:38:59.245 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:38:59.245 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:38:59.245 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:38:59.246 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:38:59.246 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:38:59.246 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:38:59.246 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:38:59.246 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:38:59.247 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 132, 2023-10-26T01:38:59.247 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:38:59.247 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:38:59.247 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:38:59.247 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:59.247 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.248 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:59.248 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6808", 2023-10-26T01:38:59.248 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:38:59.248 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.248 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.249 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:59.249 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6809", 2023-10-26T01:38:59.249 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:38:59.249 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:59.249 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:59.250 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.250 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:38:59.250 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:59.250 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.250 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:59.250 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6810", 2023-10-26T01:38:59.251 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:38:59.251 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.251 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.251 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:59.251 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6811", 2023-10-26T01:38:59.252 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:38:59.252 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:59.252 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:59.252 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.252 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:38:59.252 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:59.253 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.253 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:59.253 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6814", 2023-10-26T01:38:59.253 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:38:59.253 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.254 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.254 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:59.254 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6815", 2023-10-26T01:38:59.254 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:38:59.254 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:59.255 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:59.255 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.255 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:38:59.255 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:59.255 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.255 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:59.256 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6812", 2023-10-26T01:38:59.256 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:38:59.256 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.256 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.256 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:59.257 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6813", 2023-10-26T01:38:59.257 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:38:59.257 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:59.257 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:59.257 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.257 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6809/238206046", 2023-10-26T01:38:59.258 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6811/238206046", 2023-10-26T01:38:59.258 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6815/238206046", 2023-10-26T01:38:59.258 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6813/238206046", 2023-10-26T01:38:59.258 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:38:59.258 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:38:59.259 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:38:59.259 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:59.259 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.259 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.259 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 2, 2023-10-26T01:38:59.260 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:38:59.260 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:38:59.260 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:38:59.260 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:38:59.260 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:38:59.261 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:38:59.261 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:38:59.261 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:38:59.261 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 135, 2023-10-26T01:38:59.261 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:38:59.261 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:38:59.262 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:38:59.262 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:59.262 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.262 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:59.262 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6800", 2023-10-26T01:38:59.263 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:38:59.263 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.263 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.263 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:59.263 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6801", 2023-10-26T01:38:59.264 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:38:59.264 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:59.264 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:59.264 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.264 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:38:59.265 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:59.265 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.265 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:59.265 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6802", 2023-10-26T01:38:59.265 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:38:59.266 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.266 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.266 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:59.266 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6803", 2023-10-26T01:38:59.266 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:38:59.266 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:59.267 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:59.267 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.267 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:38:59.267 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:59.267 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.268 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:59.268 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6806", 2023-10-26T01:38:59.268 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:38:59.268 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.268 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.269 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:59.269 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6807", 2023-10-26T01:38:59.270 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:38:59.270 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:59.270 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:59.270 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.270 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:38:59.271 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:59.271 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.271 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:59.271 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6804", 2023-10-26T01:38:59.271 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:38:59.272 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.272 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.272 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:59.272 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6805", 2023-10-26T01:38:59.272 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:38:59.273 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:59.273 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:59.273 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.273 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6801/3215791846", 2023-10-26T01:38:59.273 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6803/3215791846", 2023-10-26T01:38:59.273 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6807/3215791846", 2023-10-26T01:38:59.274 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6805/3215791846", 2023-10-26T01:38:59.274 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:38:59.274 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:38:59.274 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:38:59.274 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:59.275 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.275 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.275 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 3, 2023-10-26T01:38:59.275 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:38:59.275 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:38:59.276 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:38:59.276 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:38:59.276 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:38:59.276 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:38:59.276 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:38:59.277 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 14, 2023-10-26T01:38:59.277 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 132, 2023-10-26T01:38:59.277 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:38:59.277 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:38:59.277 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:38:59.278 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:59.278 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.278 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:59.278 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6824", 2023-10-26T01:38:59.278 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:38:59.278 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.279 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.279 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:59.279 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6825", 2023-10-26T01:38:59.279 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:38:59.279 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:59.280 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:59.280 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.280 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:38:59.280 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:59.280 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.281 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:59.281 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6826", 2023-10-26T01:38:59.281 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:38:59.281 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.281 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.281 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:59.282 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6827", 2023-10-26T01:38:59.282 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:38:59.282 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:59.282 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:59.282 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.283 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:38:59.283 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:59.283 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.283 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:59.283 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6830", 2023-10-26T01:38:59.283 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:38:59.284 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.284 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.284 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:59.284 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6831", 2023-10-26T01:38:59.284 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:38:59.285 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:59.285 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:59.285 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.285 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:38:59.285 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:59.286 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.286 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:59.286 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6828", 2023-10-26T01:38:59.286 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:38:59.286 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.286 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.287 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:59.287 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6829", 2023-10-26T01:38:59.287 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:38:59.287 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:59.287 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:59.288 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.288 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6825/2256701226", 2023-10-26T01:38:59.288 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6827/2256701226", 2023-10-26T01:38:59.288 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6831/2256701226", 2023-10-26T01:38:59.288 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6829/2256701226", 2023-10-26T01:38:59.289 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:38:59.289 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:38:59.289 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:38:59.289 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:59.289 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.289 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.290 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 4, 2023-10-26T01:38:59.290 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:38:59.290 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:38:59.290 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:38:59.290 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:38:59.291 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:38:59.291 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:38:59.291 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:38:59.291 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:38:59.291 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 132, 2023-10-26T01:38:59.292 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:38:59.292 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:38:59.292 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:38:59.292 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:59.292 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.292 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:59.293 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6816", 2023-10-26T01:38:59.293 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:38:59.293 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.293 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.293 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:59.294 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6817", 2023-10-26T01:38:59.294 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:38:59.294 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:59.294 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:59.294 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.295 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:38:59.295 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:59.295 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.295 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:59.295 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6818", 2023-10-26T01:38:59.295 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:38:59.296 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.296 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.296 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:59.296 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6819", 2023-10-26T01:38:59.296 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:38:59.297 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:59.297 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:59.297 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.297 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:38:59.297 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:59.298 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.298 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:59.298 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6825", 2023-10-26T01:38:59.298 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:38:59.298 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.299 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.299 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:59.299 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6828", 2023-10-26T01:38:59.299 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:38:59.299 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:59.300 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:59.300 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.300 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:38:59.300 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:59.300 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.301 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:59.301 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6820", 2023-10-26T01:38:59.301 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:38:59.301 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.301 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.301 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:59.302 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6823", 2023-10-26T01:38:59.302 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:38:59.302 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:59.302 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:59.302 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.303 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6817/1082279874", 2023-10-26T01:38:59.303 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6819/1082279874", 2023-10-26T01:38:59.303 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6828/1082279874", 2023-10-26T01:38:59.303 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6823/1082279874", 2023-10-26T01:38:59.303 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:38:59.304 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:38:59.304 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:38:59.304 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:59.304 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.304 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.304 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 5, 2023-10-26T01:38:59.305 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:38:59.305 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:38:59.305 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:38:59.305 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:38:59.305 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:38:59.306 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:38:59.306 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:38:59.306 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:38:59.306 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 132, 2023-10-26T01:38:59.306 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:38:59.307 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:38:59.307 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:38:59.307 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:59.307 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.307 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:59.308 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6808", 2023-10-26T01:38:59.308 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:38:59.308 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.308 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.308 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:59.309 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6809", 2023-10-26T01:38:59.309 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:38:59.309 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:59.309 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:59.309 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.310 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:38:59.310 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:59.310 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.310 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:59.310 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6810", 2023-10-26T01:38:59.310 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:38:59.311 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.311 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.311 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:59.311 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6811", 2023-10-26T01:38:59.311 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:38:59.312 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:59.312 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:59.312 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.312 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:38:59.312 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:59.313 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.313 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:59.313 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6814", 2023-10-26T01:38:59.313 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:38:59.313 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.314 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.314 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:59.314 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6815", 2023-10-26T01:38:59.314 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:38:59.314 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:59.315 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:59.315 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.315 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:38:59.315 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:59.315 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.317 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:59.317 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6812", 2023-10-26T01:38:59.318 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:38:59.318 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.318 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.318 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:59.319 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6813", 2023-10-26T01:38:59.319 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:38:59.319 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:59.319 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:59.319 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.319 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6809/3335654351", 2023-10-26T01:38:59.320 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6811/3335654351", 2023-10-26T01:38:59.320 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6815/3335654351", 2023-10-26T01:38:59.320 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6813/3335654351", 2023-10-26T01:38:59.320 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:38:59.320 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:38:59.320 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:38:59.321 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:59.321 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.321 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.321 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 6, 2023-10-26T01:38:59.321 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:38:59.322 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:38:59.322 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:38:59.322 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:38:59.322 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:38:59.322 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:38:59.322 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:38:59.323 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:38:59.323 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 132, 2023-10-26T01:38:59.323 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:38:59.323 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:38:59.323 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:38:59.324 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:59.324 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.324 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:59.324 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6800", 2023-10-26T01:38:59.324 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:38:59.324 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.325 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.325 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:59.325 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6801", 2023-10-26T01:38:59.325 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:38:59.325 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:59.325 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:59.326 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.326 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:38:59.326 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:59.326 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.326 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:59.327 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6802", 2023-10-26T01:38:59.327 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:38:59.327 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.327 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.327 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:59.327 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6803", 2023-10-26T01:38:59.328 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:38:59.328 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:59.328 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:59.328 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.328 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:38:59.329 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:59.329 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.329 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:59.329 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6806", 2023-10-26T01:38:59.329 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:38:59.329 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.330 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.330 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:59.330 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6807", 2023-10-26T01:38:59.330 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:38:59.330 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:59.331 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:59.331 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.331 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:38:59.331 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:59.331 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.332 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:59.332 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6804", 2023-10-26T01:38:59.332 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:38:59.332 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.332 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.332 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:59.333 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6805", 2023-10-26T01:38:59.333 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:38:59.333 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:59.333 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:59.333 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.333 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6801/2520460697", 2023-10-26T01:38:59.334 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6803/2520460697", 2023-10-26T01:38:59.334 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6807/2520460697", 2023-10-26T01:38:59.334 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6805/2520460697", 2023-10-26T01:38:59.334 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:38:59.334 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:38:59.334 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:38:59.335 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:59.335 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.335 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.335 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 7, 2023-10-26T01:38:59.335 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:38:59.335 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:38:59.336 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:38:59.336 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:38:59.336 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:38:59.336 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:38:59.336 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:38:59.337 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:38:59.337 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 132, 2023-10-26T01:38:59.337 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:38:59.337 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:38:59.337 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:38:59.338 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:59.338 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.338 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:59.338 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6821", 2023-10-26T01:38:59.338 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:38:59.339 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.339 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.339 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:59.339 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6822", 2023-10-26T01:38:59.339 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:38:59.339 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:59.340 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:59.340 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.340 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:38:59.340 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:59.340 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.341 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:59.341 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6824", 2023-10-26T01:38:59.341 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:38:59.341 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.341 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.341 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:59.342 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6826", 2023-10-26T01:38:59.342 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:38:59.342 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:59.342 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:59.342 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.343 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:38:59.343 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:59.343 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.343 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:59.343 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6830", 2023-10-26T01:38:59.343 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:38:59.344 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.344 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.344 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:59.344 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6831", 2023-10-26T01:38:59.344 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:38:59.345 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:59.345 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:59.345 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.345 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:38:59.345 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:59.345 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.346 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:59.346 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6827", 2023-10-26T01:38:59.346 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:38:59.346 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.346 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.347 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:59.347 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6829", 2023-10-26T01:38:59.347 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:38:59.347 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:59.347 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:59.348 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.348 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6822/1115930619", 2023-10-26T01:38:59.348 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6826/1115930619", 2023-10-26T01:38:59.348 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6831/1115930619", 2023-10-26T01:38:59.348 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6829/1115930619", 2023-10-26T01:38:59.348 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:38:59.349 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:38:59.349 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:38:59.349 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:59.349 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:59.349 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:38:59.350 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_xinfo": [ 2023-10-26T01:38:59.350 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.350 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 0, 2023-10-26T01:38:59.350 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:38:59.350 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:38:59.351 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:38:59.351 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:38:59.351 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:38:59.351 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.889675+0000", 2023-10-26T01:38:59.351 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:38:59.351 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.352 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.352 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 1, 2023-10-26T01:38:59.352 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:38:59.352 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:38:59.352 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:38:59.352 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:38:59.353 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:38:59.353 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.864157+0000", 2023-10-26T01:38:59.353 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:38:59.353 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.353 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.354 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 2, 2023-10-26T01:38:59.354 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:38:59.354 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:38:59.354 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:38:59.354 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:38:59.355 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:38:59.355 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.876919+0000", 2023-10-26T01:38:59.355 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:38:59.355 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.355 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.355 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 3, 2023-10-26T01:38:59.356 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:38:59.356 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:38:59.356 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:38:59.356 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:38:59.356 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:38:59.357 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.974541+0000", 2023-10-26T01:38:59.357 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:38:59.357 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.357 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.357 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 4, 2023-10-26T01:38:59.357 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:38:59.358 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:38:59.358 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:38:59.358 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:38:59.358 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:38:59.358 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.969696+0000", 2023-10-26T01:38:59.359 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:38:59.359 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.359 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.359 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 5, 2023-10-26T01:38:59.359 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:38:59.360 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:38:59.360 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:38:59.360 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:38:59.360 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:38:59.360 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.859703+0000", 2023-10-26T01:38:59.361 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:38:59.361 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.361 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.361 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 6, 2023-10-26T01:38:59.361 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:38:59.361 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:38:59.362 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:38:59.362 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:38:59.362 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:38:59.362 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.867917+0000", 2023-10-26T01:38:59.362 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:38:59.363 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.363 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.363 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 7, 2023-10-26T01:38:59.363 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:38:59.363 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:38:59.364 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:38:59.364 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:38:59.364 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:38:59.364 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.898605+0000", 2023-10-26T01:38:59.364 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:38:59.364 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:59.365 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:38:59.365 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_upmap": [], 2023-10-26T01:38:59.365 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_upmap_items": [ 2023-10-26T01:38:59.365 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.365 INFO:teuthology.orchestra.run.smithi169.stdout: "pgid": "17.e", 2023-10-26T01:38:59.366 INFO:teuthology.orchestra.run.smithi169.stdout: "mappings": [ 2023-10-26T01:38:59.366 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.366 INFO:teuthology.orchestra.run.smithi169.stdout: "from": 0, 2023-10-26T01:38:59.366 INFO:teuthology.orchestra.run.smithi169.stdout: "to": 5 2023-10-26T01:38:59.366 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:59.366 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:59.367 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.367 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.367 INFO:teuthology.orchestra.run.smithi169.stdout: "pgid": "17.21", 2023-10-26T01:38:59.367 INFO:teuthology.orchestra.run.smithi169.stdout: "mappings": [ 2023-10-26T01:38:59.367 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.368 INFO:teuthology.orchestra.run.smithi169.stdout: "from": 0, 2023-10-26T01:38:59.368 INFO:teuthology.orchestra.run.smithi169.stdout: "to": 2 2023-10-26T01:38:59.368 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:59.368 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:59.368 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:59.368 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:38:59.369 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_upmap_primaries": [], 2023-10-26T01:38:59.369 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_temp": [], 2023-10-26T01:38:59.369 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_temp": [], 2023-10-26T01:38:59.369 INFO:teuthology.orchestra.run.smithi169.stdout: "blocklist": {}, 2023-10-26T01:38:59.369 INFO:teuthology.orchestra.run.smithi169.stdout: "range_blocklist": {}, 2023-10-26T01:38:59.370 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profiles": { 2023-10-26T01:38:59.370 INFO:teuthology.orchestra.run.smithi169.stdout: "default": { 2023-10-26T01:38:59.370 INFO:teuthology.orchestra.run.smithi169.stdout: "crush-failure-domain": "osd", 2023-10-26T01:38:59.370 INFO:teuthology.orchestra.run.smithi169.stdout: "k": "2", 2023-10-26T01:38:59.370 INFO:teuthology.orchestra.run.smithi169.stdout: "m": "1", 2023-10-26T01:38:59.370 INFO:teuthology.orchestra.run.smithi169.stdout: "plugin": "jerasure", 2023-10-26T01:38:59.371 INFO:teuthology.orchestra.run.smithi169.stdout: "technique": "reed_sol_van" 2023-10-26T01:38:59.371 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:59.371 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.371 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps_queue": [], 2023-10-26T01:38:59.371 INFO:teuthology.orchestra.run.smithi169.stdout: "new_removed_snaps": [], 2023-10-26T01:38:59.371 INFO:teuthology.orchestra.run.smithi169.stdout: "new_purged_snaps": [], 2023-10-26T01:38:59.372 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_node_flags": {}, 2023-10-26T01:38:59.372 INFO:teuthology.orchestra.run.smithi169.stdout: "device_class_flags": {}, 2023-10-26T01:38:59.372 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_mode": { 2023-10-26T01:38:59.372 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_mode_enabled": false, 2023-10-26T01:38:59.372 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_bucket_count": 0, 2023-10-26T01:38:59.373 INFO:teuthology.orchestra.run.smithi169.stdout: "degraded_stretch_mode": 0, 2023-10-26T01:38:59.373 INFO:teuthology.orchestra.run.smithi169.stdout: "recovering_stretch_mode": 0, 2023-10-26T01:38:59.373 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_mode_bucket": 0 2023-10-26T01:38:59.373 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:59.373 INFO:teuthology.orchestra.run.smithi169.stdout:} 2023-10-26T01:38:59.375 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:59.100+0000 7f4d4493c640 1 -- 172.21.15.169:0/3474583634 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f4d2006ae40 msgr2=0x7f4d2006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:59.375 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:59.100+0000 7f4d4493c640 1 --2- 172.21.15.169:0/3474583634 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f4d2006ae40 0x7f4d2006d300 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f4d30037000 tx=0x7f4d3002e040 comp rx=0 tx=0).stop 2023-10-26T01:38:59.375 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:59.100+0000 7f4d4493c640 1 -- 172.21.15.169:0/3474583634 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d40141c20 msgr2=0x7f4d400cb630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:59.375 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:59.100+0000 7f4d4493c640 1 --2- 172.21.15.169:0/3474583634 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d40141c20 0x7f4d400cb630 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f4d400cce10 tx=0x7f4d3800dd80 comp rx=0 tx=0).stop 2023-10-26T01:38:59.375 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:59.101+0000 7f4d4493c640 1 -- 172.21.15.169:0/3474583634 shutdown_connections 2023-10-26T01:38:59.376 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:59.101+0000 7f4d4493c640 1 --2- 172.21.15.169:0/3474583634 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f4d2006ae40 0x7f4d2006d300 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:59.376 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:59.101+0000 7f4d4493c640 1 --2- 172.21.15.169:0/3474583634 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d40141c20 0x7f4d400cb630 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:59.376 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:59.101+0000 7f4d4493c640 1 --2- 172.21.15.169:0/3474583634 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4d4015c250 0x7f4d40141340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:59.376 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:59.101+0000 7f4d4493c640 1 --2- 172.21.15.169:0/3474583634 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4d2c003970 0x7f4d4015df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:59.376 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:59.101+0000 7f4d4493c640 1 -- 172.21.15.169:0/3474583634 >> 172.21.15.169:0/3474583634 conn(0x7f4d400aedd0 msgr2=0x7f4d400ae330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:59.377 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:59.101+0000 7f4d4493c640 1 -- 172.21.15.169:0/3474583634 shutdown_connections 2023-10-26T01:38:59.377 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:59.101+0000 7f4d4493c640 1 -- 172.21.15.169:0/3474583634 wait complete. 2023-10-26T01:38:59.377 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-10-26T01:38:59.378 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-26T01:38:59.608 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:59.606+0000 7f5558b8a640 1 -- 172.21.15.169:0/2010302026 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5554158850 msgr2=0x7f555414e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:59.609 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:59.606+0000 7f5558b8a640 1 --2- 172.21.15.169:0/2010302026 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5554158850 0x7f555414e790 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f5548009830 tx=0x7f554802bc50 comp rx=0 tx=0).stop 2023-10-26T01:38:59.609 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:59.606+0000 7f5558b8a640 1 -- 172.21.15.169:0/2010302026 shutdown_connections 2023-10-26T01:38:59.609 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:59.606+0000 7f5558b8a640 1 --2- 172.21.15.169:0/2010302026 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5554158850 0x7f555414e790 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:59.609 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:59.606+0000 7f5558b8a640 1 -- 172.21.15.169:0/2010302026 >> 172.21.15.169:0/2010302026 conn(0x7f5554069930 msgr2=0x7f5554069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:38:59.609 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:59.607+0000 7f5558b8a640 1 -- 172.21.15.169:0/2010302026 shutdown_connections 2023-10-26T01:38:59.610 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:59.607+0000 7f5558b8a640 1 -- 172.21.15.169:0/2010302026 wait complete. 2023-10-26T01:38:59.610 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:59.608+0000 7f5558b8a640 1 Processor -- start 2023-10-26T01:38:59.610 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:59.608+0000 7f5558b8a640 1 -- start start 2023-10-26T01:38:59.611 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:59.609+0000 7f5558b8a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f555406ce20 0x7f55541f3080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:59.611 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:59.609+0000 7f5558b8a640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55540cd720 0x7f55540aeec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:59.612 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:59.609+0000 7f5558b8a640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5554158850 0x7f55540af400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:59.612 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:59.609+0000 7f5558b8a640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55541470d0 con 0x7f55540cd720 2023-10-26T01:38:59.612 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:59.609+0000 7f5558b8a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5554146730 con 0x7f555406ce20 2023-10-26T01:38:59.612 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:59.609+0000 7f5558b8a640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5554146a10 con 0x7f5554158850 2023-10-26T01:38:59.612 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:59.610+0000 7f5551d9b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55540cd720 0x7f55540aeec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:59.613 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:59.610+0000 7f5551d9b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55540cd720 0x7f55540aeec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:44806/0 (socket says 172.21.15.169:44806) 2023-10-26T01:38:59.613 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:59.610+0000 7f5551d9b640 1 -- 172.21.15.169:0/1714211864 learned_addr learned my addr 172.21.15.169:0/1714211864 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:38:59.614 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:59.610+0000 7f5551d9b640 1 -- 172.21.15.169:0/1714211864 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5554158850 msgr2=0x7f55540af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:59.614 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:59.610+0000 7f5551d9b640 1 --2- 172.21.15.169:0/1714211864 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5554158850 0x7f55540af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:59.615 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:59.610+0000 7f5551d9b640 1 -- 172.21.15.169:0/1714211864 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f555406ce20 msgr2=0x7f55541f3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:38:59.615 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:59.610+0000 7f5551d9b640 1 --2- 172.21.15.169:0/1714211864 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f555406ce20 0x7f55541f3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:38:59.615 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:59.610+0000 7f5551d9b640 1 -- 172.21.15.169:0/1714211864 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5548009490 con 0x7f55540cd720 2023-10-26T01:38:59.615 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:59.610+0000 7f5551d9b640 1 --2- 172.21.15.169:0/1714211864 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55540cd720 0x7f55540aeec0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f5548009780 tx=0x7f554802cdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:59.615 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:59.611+0000 7f5542ffd640 1 -- 172.21.15.169:0/1714211864 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5548005220 con 0x7f55540cd720 2023-10-26T01:38:59.616 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:59.611+0000 7f5542ffd640 1 -- 172.21.15.169:0/1714211864 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5548005470 con 0x7f55540cd720 2023-10-26T01:38:59.616 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:59.611+0000 7f5542ffd640 1 -- 172.21.15.169:0/1714211864 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5548003a20 con 0x7f55540cd720 2023-10-26T01:38:59.616 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:59.611+0000 7f5558b8a640 1 -- 172.21.15.169:0/1714211864 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5554146cf0 con 0x7f55540cd720 2023-10-26T01:38:59.616 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:59.611+0000 7f5558b8a640 1 -- 172.21.15.169:0/1714211864 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55541f35c0 con 0x7f55540cd720 2023-10-26T01:38:59.616 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:59.612+0000 7f5558b8a640 1 -- 172.21.15.169:0/1714211864 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f555404db10 con 0x7f55540cd720 2023-10-26T01:38:59.616 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:59.613+0000 7f5542ffd640 1 -- 172.21.15.169:0/1714211864 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f554803b040 con 0x7f55540cd720 2023-10-26T01:38:59.617 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:59.614+0000 7f5542ffd640 1 --2- 172.21.15.169:0/1714211864 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f553406ae40 0x7f553406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:38:59.617 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:59.614+0000 7f5542ffd640 1 -- 172.21.15.169:0/1714211864 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f55480ab380 con 0x7f55540cd720 2023-10-26T01:38:59.618 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:59.615+0000 7f555259c640 1 --2- 172.21.15.169:0/1714211864 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f553406ae40 0x7f553406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:38:59.618 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:59.616+0000 7f555259c640 1 --2- 172.21.15.169:0/1714211864 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f553406ae40 0x7f553406d300 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f5544002230 tx=0x7f5544001da0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:38:59.620 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:59.618+0000 7f5542ffd640 1 -- 172.21.15.169:0/1714211864 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f55480763e0 con 0x7f55540cd720 2023-10-26T01:38:59.808 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:59.806+0000 7f5558b8a640 1 -- 172.21.15.169:0/1714211864 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f55540aca70 con 0x7f55540cd720 2023-10-26T01:38:59.811 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:59.809+0000 7f5542ffd640 1 -- 172.21.15.169:0/1714211864 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v139) v1 ==== 81+0+34450 (secure 0 0 0) 0x7f5548079a00 con 0x7f55540cd720 2023-10-26T01:38:59.812 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:38:59.813 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2023-10-26T01:38:59.813 INFO:teuthology.orchestra.run.smithi169.stdout: "epoch": 139, 2023-10-26T01:38:59.813 INFO:teuthology.orchestra.run.smithi169.stdout: "fsid": "30203b28-ed8a-44e8-a9a6-a9bf25f980f1", 2023-10-26T01:38:59.814 INFO:teuthology.orchestra.run.smithi169.stdout: "created": "2023-10-26T01:30:22.897155+0000", 2023-10-26T01:38:59.814 INFO:teuthology.orchestra.run.smithi169.stdout: "modified": "2023-10-26T01:38:58.061086+0000", 2023-10-26T01:38:59.815 INFO:teuthology.orchestra.run.smithi169.stdout: "last_up_change": "2023-10-26T01:30:31.875281+0000", 2023-10-26T01:38:59.815 INFO:teuthology.orchestra.run.smithi169.stdout: "last_in_change": "2023-10-26T01:30:27.637941+0000", 2023-10-26T01:38:59.815 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-26T01:38:59.815 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_num": 5799936, 2023-10-26T01:38:59.816 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_set": [ 2023-10-26T01:38:59.816 INFO:teuthology.orchestra.run.smithi169.stdout: "pglog_hardlimit", 2023-10-26T01:38:59.816 INFO:teuthology.orchestra.run.smithi169.stdout: "purged_snapdirs", 2023-10-26T01:38:59.816 INFO:teuthology.orchestra.run.smithi169.stdout: "recovery_deletes", 2023-10-26T01:38:59.816 INFO:teuthology.orchestra.run.smithi169.stdout: "sortbitwise" 2023-10-26T01:38:59.816 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:38:59.817 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_version": 4, 2023-10-26T01:38:59.817 INFO:teuthology.orchestra.run.smithi169.stdout: "full_ratio": 0.69999998807907104, 2023-10-26T01:38:59.817 INFO:teuthology.orchestra.run.smithi169.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-26T01:38:59.817 INFO:teuthology.orchestra.run.smithi169.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-26T01:38:59.817 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_snapshot": "", 2023-10-26T01:38:59.818 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_max": 17, 2023-10-26T01:38:59.818 INFO:teuthology.orchestra.run.smithi169.stdout: "max_osd": 8, 2023-10-26T01:38:59.818 INFO:teuthology.orchestra.run.smithi169.stdout: "require_min_compat_client": "luminous", 2023-10-26T01:38:59.818 INFO:teuthology.orchestra.run.smithi169.stdout: "min_compat_client": "luminous", 2023-10-26T01:38:59.818 INFO:teuthology.orchestra.run.smithi169.stdout: "require_osd_release": "reef", 2023-10-26T01:38:59.819 INFO:teuthology.orchestra.run.smithi169.stdout: "allow_crimson": false, 2023-10-26T01:38:59.819 INFO:teuthology.orchestra.run.smithi169.stdout: "pools": [ 2023-10-26T01:38:59.819 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.819 INFO:teuthology.orchestra.run.smithi169.stdout: "pool": 1, 2023-10-26T01:38:59.819 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_name": ".mgr", 2023-10-26T01:38:59.820 INFO:teuthology.orchestra.run.smithi169.stdout: "create_time": "2023-10-26T01:30:32.200808+0000", 2023-10-26T01:38:59.820 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": 1, 2023-10-26T01:38:59.820 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_names": "hashpspool", 2023-10-26T01:38:59.820 INFO:teuthology.orchestra.run.smithi169.stdout: "type": 1, 2023-10-26T01:38:59.820 INFO:teuthology.orchestra.run.smithi169.stdout: "size": 2, 2023-10-26T01:38:59.820 INFO:teuthology.orchestra.run.smithi169.stdout: "min_size": 1, 2023-10-26T01:38:59.821 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_rule": 0, 2023-10-26T01:38:59.821 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_count": 0, 2023-10-26T01:38:59.821 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_target": 0, 2023-10-26T01:38:59.821 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T01:38:59.821 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T01:38:59.822 INFO:teuthology.orchestra.run.smithi169.stdout: "object_hash": 2, 2023-10-26T01:38:59.822 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_mode": "off", 2023-10-26T01:38:59.822 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num": 1, 2023-10-26T01:38:59.822 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num": 1, 2023-10-26T01:38:59.822 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num_target": 1, 2023-10-26T01:38:59.823 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_target": 1, 2023-10-26T01:38:59.823 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_pending": 1, 2023-10-26T01:38:59.823 INFO:teuthology.orchestra.run.smithi169.stdout: "last_pg_merge_meta": { 2023-10-26T01:38:59.823 INFO:teuthology.orchestra.run.smithi169.stdout: "source_pgid": "0.0", 2023-10-26T01:38:59.824 INFO:teuthology.orchestra.run.smithi169.stdout: "ready_epoch": 0, 2023-10-26T01:38:59.824 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_started": 0, 2023-10-26T01:38:59.824 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_clean": 0, 2023-10-26T01:38:59.824 INFO:teuthology.orchestra.run.smithi169.stdout: "source_version": "0'0", 2023-10-26T01:38:59.825 INFO:teuthology.orchestra.run.smithi169.stdout: "target_version": "0'0" 2023-10-26T01:38:59.825 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.825 INFO:teuthology.orchestra.run.smithi169.stdout: "last_change": "109", 2023-10-26T01:38:59.826 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend": "0", 2023-10-26T01:38:59.826 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T01:38:59.826 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T01:38:59.827 INFO:teuthology.orchestra.run.smithi169.stdout: "auid": 0, 2023-10-26T01:38:59.827 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_mode": "selfmanaged", 2023-10-26T01:38:59.827 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_seq": 0, 2023-10-26T01:38:59.828 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_epoch": 0, 2023-10-26T01:38:59.828 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_snaps": [], 2023-10-26T01:38:59.828 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps": "[]", 2023-10-26T01:38:59.828 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_bytes": 0, 2023-10-26T01:38:59.828 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_objects": 0, 2023-10-26T01:38:59.829 INFO:teuthology.orchestra.run.smithi169.stdout: "tiers": [], 2023-10-26T01:38:59.829 INFO:teuthology.orchestra.run.smithi169.stdout: "tier_of": -1, 2023-10-26T01:38:59.829 INFO:teuthology.orchestra.run.smithi169.stdout: "read_tier": -1, 2023-10-26T01:38:59.830 INFO:teuthology.orchestra.run.smithi169.stdout: "write_tier": -1, 2023-10-26T01:38:59.830 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_mode": "none", 2023-10-26T01:38:59.831 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_bytes": 0, 2023-10-26T01:38:59.831 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_objects": 0, 2023-10-26T01:38:59.831 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T01:38:59.831 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T01:38:59.832 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T01:38:59.832 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_flush_age": 0, 2023-10-26T01:38:59.832 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_evict_age": 0, 2023-10-26T01:38:59.832 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profile": "", 2023-10-26T01:38:59.832 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_params": { 2023-10-26T01:38:59.833 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "none" 2023-10-26T01:38:59.833 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.833 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_period": 0, 2023-10-26T01:38:59.833 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_count": 0, 2023-10-26T01:38:59.833 INFO:teuthology.orchestra.run.smithi169.stdout: "use_gmt_hitset": true, 2023-10-26T01:38:59.833 INFO:teuthology.orchestra.run.smithi169.stdout: "min_read_recency_for_promote": 0, 2023-10-26T01:38:59.834 INFO:teuthology.orchestra.run.smithi169.stdout: "min_write_recency_for_promote": 0, 2023-10-26T01:38:59.834 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T01:38:59.834 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_search_last_n": 0, 2023-10-26T01:38:59.834 INFO:teuthology.orchestra.run.smithi169.stdout: "grade_table": [], 2023-10-26T01:38:59.834 INFO:teuthology.orchestra.run.smithi169.stdout: "stripe_width": 0, 2023-10-26T01:38:59.835 INFO:teuthology.orchestra.run.smithi169.stdout: "expected_num_objects": 0, 2023-10-26T01:38:59.835 INFO:teuthology.orchestra.run.smithi169.stdout: "fast_read": false, 2023-10-26T01:38:59.835 INFO:teuthology.orchestra.run.smithi169.stdout: "options": { 2023-10-26T01:38:59.835 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_max": 32, 2023-10-26T01:38:59.835 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_min": 1 2023-10-26T01:38:59.836 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.836 INFO:teuthology.orchestra.run.smithi169.stdout: "application_metadata": { 2023-10-26T01:38:59.836 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": {} 2023-10-26T01:38:59.836 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.836 INFO:teuthology.orchestra.run.smithi169.stdout: "read_balance": { 2023-10-26T01:38:59.836 INFO:teuthology.orchestra.run.smithi169.stdout: "score_acting": 8, 2023-10-26T01:38:59.837 INFO:teuthology.orchestra.run.smithi169.stdout: "score_stable": 8, 2023-10-26T01:38:59.837 INFO:teuthology.orchestra.run.smithi169.stdout: "optimal_score": 0.25, 2023-10-26T01:38:59.837 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_acting": 2, 2023-10-26T01:38:59.837 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_stable": 2, 2023-10-26T01:38:59.837 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity_weighted": 1, 2023-10-26T01:38:59.838 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity": 1, 2023-10-26T01:38:59.838 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity_weighted": 1 2023-10-26T01:38:59.838 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:59.838 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.838 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.839 INFO:teuthology.orchestra.run.smithi169.stdout: "pool": 16, 2023-10-26T01:38:59.839 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_name": "cephfs_metadata", 2023-10-26T01:38:59.839 INFO:teuthology.orchestra.run.smithi169.stdout: "create_time": "2023-10-26T01:38:18.617877+0000", 2023-10-26T01:38:59.839 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": 1, 2023-10-26T01:38:59.839 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_names": "hashpspool", 2023-10-26T01:38:59.839 INFO:teuthology.orchestra.run.smithi169.stdout: "type": 1, 2023-10-26T01:38:59.840 INFO:teuthology.orchestra.run.smithi169.stdout: "size": 2, 2023-10-26T01:38:59.840 INFO:teuthology.orchestra.run.smithi169.stdout: "min_size": 1, 2023-10-26T01:38:59.840 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_rule": 0, 2023-10-26T01:38:59.840 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_count": 0, 2023-10-26T01:38:59.840 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_target": 0, 2023-10-26T01:38:59.841 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T01:38:59.841 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T01:38:59.841 INFO:teuthology.orchestra.run.smithi169.stdout: "object_hash": 2, 2023-10-26T01:38:59.841 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_mode": "off", 2023-10-26T01:38:59.841 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num": 32, 2023-10-26T01:38:59.841 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num": 32, 2023-10-26T01:38:59.842 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num_target": 32, 2023-10-26T01:38:59.842 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_target": 32, 2023-10-26T01:38:59.842 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_pending": 32, 2023-10-26T01:38:59.842 INFO:teuthology.orchestra.run.smithi169.stdout: "last_pg_merge_meta": { 2023-10-26T01:38:59.842 INFO:teuthology.orchestra.run.smithi169.stdout: "source_pgid": "0.0", 2023-10-26T01:38:59.843 INFO:teuthology.orchestra.run.smithi169.stdout: "ready_epoch": 0, 2023-10-26T01:38:59.843 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_started": 0, 2023-10-26T01:38:59.843 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_clean": 0, 2023-10-26T01:38:59.843 INFO:teuthology.orchestra.run.smithi169.stdout: "source_version": "0'0", 2023-10-26T01:38:59.843 INFO:teuthology.orchestra.run.smithi169.stdout: "target_version": "0'0" 2023-10-26T01:38:59.844 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.844 INFO:teuthology.orchestra.run.smithi169.stdout: "last_change": "133", 2023-10-26T01:38:59.844 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend": "0", 2023-10-26T01:38:59.844 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T01:38:59.844 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T01:38:59.844 INFO:teuthology.orchestra.run.smithi169.stdout: "auid": 0, 2023-10-26T01:38:59.845 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_mode": "selfmanaged", 2023-10-26T01:38:59.845 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_seq": 0, 2023-10-26T01:38:59.845 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_epoch": 0, 2023-10-26T01:38:59.845 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_snaps": [], 2023-10-26T01:38:59.845 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps": "[]", 2023-10-26T01:38:59.846 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_bytes": 0, 2023-10-26T01:38:59.846 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_objects": 0, 2023-10-26T01:38:59.846 INFO:teuthology.orchestra.run.smithi169.stdout: "tiers": [], 2023-10-26T01:38:59.846 INFO:teuthology.orchestra.run.smithi169.stdout: "tier_of": -1, 2023-10-26T01:38:59.846 INFO:teuthology.orchestra.run.smithi169.stdout: "read_tier": -1, 2023-10-26T01:38:59.846 INFO:teuthology.orchestra.run.smithi169.stdout: "write_tier": -1, 2023-10-26T01:38:59.847 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_mode": "none", 2023-10-26T01:38:59.847 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_bytes": 0, 2023-10-26T01:38:59.847 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_objects": 0, 2023-10-26T01:38:59.847 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T01:38:59.847 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T01:38:59.847 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T01:38:59.848 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_flush_age": 0, 2023-10-26T01:38:59.848 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_evict_age": 0, 2023-10-26T01:38:59.848 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profile": "", 2023-10-26T01:38:59.848 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_params": { 2023-10-26T01:38:59.848 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "none" 2023-10-26T01:38:59.849 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.849 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_period": 0, 2023-10-26T01:38:59.849 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_count": 0, 2023-10-26T01:38:59.849 INFO:teuthology.orchestra.run.smithi169.stdout: "use_gmt_hitset": true, 2023-10-26T01:38:59.849 INFO:teuthology.orchestra.run.smithi169.stdout: "min_read_recency_for_promote": 0, 2023-10-26T01:38:59.849 INFO:teuthology.orchestra.run.smithi169.stdout: "min_write_recency_for_promote": 0, 2023-10-26T01:38:59.850 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T01:38:59.850 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_search_last_n": 0, 2023-10-26T01:38:59.850 INFO:teuthology.orchestra.run.smithi169.stdout: "grade_table": [], 2023-10-26T01:38:59.850 INFO:teuthology.orchestra.run.smithi169.stdout: "stripe_width": 0, 2023-10-26T01:38:59.850 INFO:teuthology.orchestra.run.smithi169.stdout: "expected_num_objects": 0, 2023-10-26T01:38:59.850 INFO:teuthology.orchestra.run.smithi169.stdout: "fast_read": false, 2023-10-26T01:38:59.851 INFO:teuthology.orchestra.run.smithi169.stdout: "options": { 2023-10-26T01:38:59.851 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_bias": 4, 2023-10-26T01:38:59.851 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_min": 16, 2023-10-26T01:38:59.851 INFO:teuthology.orchestra.run.smithi169.stdout: "recovery_priority": 5 2023-10-26T01:38:59.851 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.852 INFO:teuthology.orchestra.run.smithi169.stdout: "application_metadata": { 2023-10-26T01:38:59.852 INFO:teuthology.orchestra.run.smithi169.stdout: "cephfs": { 2023-10-26T01:38:59.852 INFO:teuthology.orchestra.run.smithi169.stdout: "metadata": "cephfs" 2023-10-26T01:38:59.852 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:59.852 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.853 INFO:teuthology.orchestra.run.smithi169.stdout: "read_balance": { 2023-10-26T01:38:59.853 INFO:teuthology.orchestra.run.smithi169.stdout: "score_acting": 1.75, 2023-10-26T01:38:59.853 INFO:teuthology.orchestra.run.smithi169.stdout: "score_stable": 1.75, 2023-10-26T01:38:59.853 INFO:teuthology.orchestra.run.smithi169.stdout: "optimal_score": 1, 2023-10-26T01:38:59.853 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_acting": 1.75, 2023-10-26T01:38:59.854 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_stable": 1.75, 2023-10-26T01:38:59.854 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity_weighted": 1, 2023-10-26T01:38:59.854 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity": 1, 2023-10-26T01:38:59.854 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity_weighted": 1 2023-10-26T01:38:59.854 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:59.855 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.855 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.855 INFO:teuthology.orchestra.run.smithi169.stdout: "pool": 17, 2023-10-26T01:38:59.855 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_name": "cephfs_data", 2023-10-26T01:38:59.855 INFO:teuthology.orchestra.run.smithi169.stdout: "create_time": "2023-10-26T01:38:19.793041+0000", 2023-10-26T01:38:59.855 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": 1, 2023-10-26T01:38:59.856 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_names": "hashpspool", 2023-10-26T01:38:59.856 INFO:teuthology.orchestra.run.smithi169.stdout: "type": 1, 2023-10-26T01:38:59.856 INFO:teuthology.orchestra.run.smithi169.stdout: "size": 2, 2023-10-26T01:38:59.856 INFO:teuthology.orchestra.run.smithi169.stdout: "min_size": 1, 2023-10-26T01:38:59.856 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_rule": 0, 2023-10-26T01:38:59.857 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_count": 0, 2023-10-26T01:38:59.857 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_target": 0, 2023-10-26T01:38:59.857 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T01:38:59.857 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T01:38:59.857 INFO:teuthology.orchestra.run.smithi169.stdout: "object_hash": 2, 2023-10-26T01:38:59.857 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_mode": "off", 2023-10-26T01:38:59.858 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num": 64, 2023-10-26T01:38:59.858 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num": 64, 2023-10-26T01:38:59.858 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num_target": 64, 2023-10-26T01:38:59.858 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_target": 64, 2023-10-26T01:38:59.859 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_pending": 64, 2023-10-26T01:38:59.859 INFO:teuthology.orchestra.run.smithi169.stdout: "last_pg_merge_meta": { 2023-10-26T01:38:59.859 INFO:teuthology.orchestra.run.smithi169.stdout: "source_pgid": "0.0", 2023-10-26T01:38:59.859 INFO:teuthology.orchestra.run.smithi169.stdout: "ready_epoch": 0, 2023-10-26T01:38:59.859 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_started": 0, 2023-10-26T01:38:59.859 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_clean": 0, 2023-10-26T01:38:59.860 INFO:teuthology.orchestra.run.smithi169.stdout: "source_version": "0'0", 2023-10-26T01:38:59.860 INFO:teuthology.orchestra.run.smithi169.stdout: "target_version": "0'0" 2023-10-26T01:38:59.860 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.860 INFO:teuthology.orchestra.run.smithi169.stdout: "last_change": "139", 2023-10-26T01:38:59.860 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend": "0", 2023-10-26T01:38:59.860 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T01:38:59.861 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T01:38:59.861 INFO:teuthology.orchestra.run.smithi169.stdout: "auid": 0, 2023-10-26T01:38:59.861 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_mode": "selfmanaged", 2023-10-26T01:38:59.861 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_seq": 0, 2023-10-26T01:38:59.861 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_epoch": 0, 2023-10-26T01:38:59.862 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_snaps": [], 2023-10-26T01:38:59.862 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps": "[]", 2023-10-26T01:38:59.862 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_bytes": 33554432, 2023-10-26T01:38:59.862 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_objects": 0, 2023-10-26T01:38:59.862 INFO:teuthology.orchestra.run.smithi169.stdout: "tiers": [], 2023-10-26T01:38:59.862 INFO:teuthology.orchestra.run.smithi169.stdout: "tier_of": -1, 2023-10-26T01:38:59.863 INFO:teuthology.orchestra.run.smithi169.stdout: "read_tier": -1, 2023-10-26T01:38:59.863 INFO:teuthology.orchestra.run.smithi169.stdout: "write_tier": -1, 2023-10-26T01:38:59.863 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_mode": "none", 2023-10-26T01:38:59.863 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_bytes": 0, 2023-10-26T01:38:59.863 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_objects": 0, 2023-10-26T01:38:59.864 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T01:38:59.864 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T01:38:59.864 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T01:38:59.864 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_flush_age": 0, 2023-10-26T01:38:59.864 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_evict_age": 0, 2023-10-26T01:38:59.864 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profile": "", 2023-10-26T01:38:59.865 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_params": { 2023-10-26T01:38:59.865 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "none" 2023-10-26T01:38:59.865 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.865 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_period": 0, 2023-10-26T01:38:59.865 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_count": 0, 2023-10-26T01:38:59.866 INFO:teuthology.orchestra.run.smithi169.stdout: "use_gmt_hitset": true, 2023-10-26T01:38:59.866 INFO:teuthology.orchestra.run.smithi169.stdout: "min_read_recency_for_promote": 0, 2023-10-26T01:38:59.866 INFO:teuthology.orchestra.run.smithi169.stdout: "min_write_recency_for_promote": 0, 2023-10-26T01:38:59.866 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T01:38:59.866 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_search_last_n": 0, 2023-10-26T01:38:59.866 INFO:teuthology.orchestra.run.smithi169.stdout: "grade_table": [], 2023-10-26T01:38:59.867 INFO:teuthology.orchestra.run.smithi169.stdout: "stripe_width": 0, 2023-10-26T01:38:59.867 INFO:teuthology.orchestra.run.smithi169.stdout: "expected_num_objects": 0, 2023-10-26T01:38:59.867 INFO:teuthology.orchestra.run.smithi169.stdout: "fast_read": false, 2023-10-26T01:38:59.867 INFO:teuthology.orchestra.run.smithi169.stdout: "options": { 2023-10-26T01:38:59.867 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_min": 64, 2023-10-26T01:38:59.868 INFO:teuthology.orchestra.run.smithi169.stdout: "target_size_ratio": 0.89999997615814209 2023-10-26T01:38:59.868 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.868 INFO:teuthology.orchestra.run.smithi169.stdout: "application_metadata": { 2023-10-26T01:38:59.868 INFO:teuthology.orchestra.run.smithi169.stdout: "cephfs": { 2023-10-26T01:38:59.868 INFO:teuthology.orchestra.run.smithi169.stdout: "data": "cephfs" 2023-10-26T01:38:59.869 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:59.869 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.869 INFO:teuthology.orchestra.run.smithi169.stdout: "read_balance": { 2023-10-26T01:38:59.869 INFO:teuthology.orchestra.run.smithi169.stdout: "score_acting": 1.6299999952316284, 2023-10-26T01:38:59.869 INFO:teuthology.orchestra.run.smithi169.stdout: "score_stable": 1.6299999952316284, 2023-10-26T01:38:59.869 INFO:teuthology.orchestra.run.smithi169.stdout: "optimal_score": 1, 2023-10-26T01:38:59.870 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_acting": 1.6299999952316284, 2023-10-26T01:38:59.870 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_stable": 1.6299999952316284, 2023-10-26T01:38:59.870 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity_weighted": 1, 2023-10-26T01:38:59.870 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity": 1, 2023-10-26T01:38:59.870 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity_weighted": 1 2023-10-26T01:38:59.871 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:59.871 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:59.871 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:38:59.871 INFO:teuthology.orchestra.run.smithi169.stdout: "osds": [ 2023-10-26T01:38:59.871 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.872 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 0, 2023-10-26T01:38:59.872 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:38:59.872 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:38:59.872 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:38:59.872 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:38:59.873 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:38:59.873 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:38:59.873 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:38:59.873 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:38:59.873 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 132, 2023-10-26T01:38:59.873 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:38:59.874 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:38:59.874 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:38:59.874 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:59.874 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.874 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:59.875 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6816", 2023-10-26T01:38:59.875 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:38:59.875 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.875 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.875 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:59.875 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6817", 2023-10-26T01:38:59.876 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:38:59.876 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:59.876 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:59.876 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.876 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:38:59.877 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:59.877 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.877 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:59.877 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6818", 2023-10-26T01:38:59.877 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:38:59.878 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.878 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.878 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:59.878 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6819", 2023-10-26T01:38:59.878 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:38:59.878 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:59.879 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:59.879 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.879 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:38:59.879 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:59.879 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.880 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:59.880 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6822", 2023-10-26T01:38:59.880 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:38:59.880 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.880 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.880 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:59.881 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6823", 2023-10-26T01:38:59.881 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:38:59.881 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:59.881 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:59.881 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.882 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:38:59.882 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:59.882 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.882 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:59.882 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6820", 2023-10-26T01:38:59.883 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:38:59.883 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.883 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.883 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:59.883 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6821", 2023-10-26T01:38:59.883 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:38:59.884 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:59.884 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:59.884 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.884 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6817/4204255864", 2023-10-26T01:38:59.884 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6819/4204255864", 2023-10-26T01:38:59.884 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6823/4204255864", 2023-10-26T01:38:59.885 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6821/4204255864", 2023-10-26T01:38:59.885 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:38:59.885 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:38:59.885 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:38:59.885 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:59.886 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.886 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.886 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 1, 2023-10-26T01:38:59.886 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:38:59.886 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:38:59.887 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:38:59.887 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:38:59.887 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:38:59.887 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:38:59.887 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:38:59.887 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:38:59.888 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 132, 2023-10-26T01:38:59.888 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:38:59.888 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:38:59.888 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:38:59.888 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:59.889 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.889 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:59.889 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6808", 2023-10-26T01:38:59.889 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:38:59.889 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.890 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.890 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:59.890 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6809", 2023-10-26T01:38:59.890 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:38:59.890 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:59.890 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:59.891 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.891 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:38:59.891 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:59.891 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.891 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:59.892 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6810", 2023-10-26T01:38:59.892 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:38:59.892 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.892 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.892 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:59.892 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6811", 2023-10-26T01:38:59.893 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:38:59.893 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:59.893 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:59.893 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.893 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:38:59.894 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:59.894 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.894 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:59.894 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6814", 2023-10-26T01:38:59.894 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:38:59.895 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.895 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.895 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:59.895 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6815", 2023-10-26T01:38:59.895 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:38:59.895 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:59.896 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:59.896 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.896 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:38:59.896 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:59.896 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.897 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:59.897 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6812", 2023-10-26T01:38:59.897 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:38:59.897 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.897 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.897 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:59.898 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6813", 2023-10-26T01:38:59.898 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:38:59.898 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:59.898 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:59.898 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.899 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6809/238206046", 2023-10-26T01:38:59.899 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6811/238206046", 2023-10-26T01:38:59.899 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6815/238206046", 2023-10-26T01:38:59.899 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6813/238206046", 2023-10-26T01:38:59.899 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:38:59.900 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:38:59.900 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:38:59.900 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:59.900 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.900 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.900 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 2, 2023-10-26T01:38:59.901 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:38:59.901 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:38:59.901 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:38:59.901 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:38:59.901 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:38:59.902 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:38:59.902 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:38:59.902 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:38:59.902 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 135, 2023-10-26T01:38:59.902 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:38:59.903 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:38:59.903 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:38:59.903 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:59.903 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.903 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:59.903 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6800", 2023-10-26T01:38:59.904 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:38:59.904 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.904 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.904 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:59.904 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6801", 2023-10-26T01:38:59.905 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:38:59.905 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:59.905 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:59.905 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.905 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:38:59.906 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:59.906 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.906 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:59.906 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6802", 2023-10-26T01:38:59.906 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:38:59.907 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.907 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.907 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:59.907 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6803", 2023-10-26T01:38:59.907 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:38:59.907 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:59.908 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:59.908 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.908 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:38:59.908 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:59.908 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.909 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:59.909 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6806", 2023-10-26T01:38:59.909 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:38:59.909 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.909 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.910 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:59.910 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6807", 2023-10-26T01:38:59.910 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:38:59.910 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:59.910 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:59.911 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.911 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:38:59.911 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:59.911 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.911 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:59.912 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6804", 2023-10-26T01:38:59.912 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:38:59.912 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.912 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.912 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:59.912 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6805", 2023-10-26T01:38:59.913 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:38:59.913 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:59.913 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:59.913 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.913 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6801/3215791846", 2023-10-26T01:38:59.914 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6803/3215791846", 2023-10-26T01:38:59.914 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6807/3215791846", 2023-10-26T01:38:59.914 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6805/3215791846", 2023-10-26T01:38:59.914 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:38:59.914 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:38:59.915 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:38:59.915 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:59.915 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.915 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.915 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 3, 2023-10-26T01:38:59.915 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:38:59.916 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:38:59.916 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:38:59.916 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:38:59.916 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:38:59.916 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:38:59.917 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:38:59.917 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 14, 2023-10-26T01:38:59.917 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 132, 2023-10-26T01:38:59.917 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:38:59.917 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:38:59.918 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:38:59.918 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:59.918 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.918 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:59.918 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6824", 2023-10-26T01:38:59.919 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:38:59.919 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.919 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.919 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:59.919 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6825", 2023-10-26T01:38:59.920 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:38:59.920 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:59.920 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:59.920 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.920 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:38:59.920 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:59.921 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.921 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:59.921 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6826", 2023-10-26T01:38:59.921 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:38:59.921 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.922 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.922 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:59.922 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6827", 2023-10-26T01:38:59.922 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:38:59.922 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:59.923 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:59.923 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.923 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:38:59.923 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:59.923 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.924 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:59.924 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6830", 2023-10-26T01:38:59.924 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:38:59.924 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.924 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.924 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:59.925 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6831", 2023-10-26T01:38:59.925 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:38:59.925 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:59.925 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:59.925 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.926 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:38:59.926 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:59.926 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.926 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:59.926 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6828", 2023-10-26T01:38:59.926 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:38:59.927 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.927 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.927 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:59.927 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6829", 2023-10-26T01:38:59.927 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:38:59.928 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:59.928 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:59.928 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.928 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6825/2256701226", 2023-10-26T01:38:59.928 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6827/2256701226", 2023-10-26T01:38:59.929 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6831/2256701226", 2023-10-26T01:38:59.929 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6829/2256701226", 2023-10-26T01:38:59.929 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:38:59.929 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:38:59.929 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:38:59.929 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:59.930 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.930 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.930 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 4, 2023-10-26T01:38:59.930 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:38:59.930 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:38:59.931 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:38:59.931 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:38:59.931 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:38:59.931 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:38:59.931 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:38:59.931 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:38:59.932 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 132, 2023-10-26T01:38:59.932 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:38:59.932 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:38:59.932 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:38:59.932 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:59.933 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.933 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:59.933 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6816", 2023-10-26T01:38:59.933 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:38:59.933 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.934 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.934 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:59.934 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6817", 2023-10-26T01:38:59.934 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:38:59.934 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:59.934 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:59.935 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.935 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:38:59.938 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:59.938 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.938 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:59.939 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6818", 2023-10-26T01:38:59.939 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:38:59.939 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.939 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.939 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:59.940 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6819", 2023-10-26T01:38:59.940 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:38:59.940 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:59.940 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:59.940 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.940 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:38:59.941 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:59.941 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.941 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:59.941 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6825", 2023-10-26T01:38:59.941 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:38:59.942 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.942 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.942 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:59.942 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6828", 2023-10-26T01:38:59.942 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:38:59.943 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:59.943 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:59.943 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.943 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:38:59.943 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:59.944 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.944 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:59.944 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6820", 2023-10-26T01:38:59.944 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:38:59.944 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.944 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.945 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:59.945 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6823", 2023-10-26T01:38:59.945 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:38:59.945 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:59.945 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:59.946 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.946 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6817/1082279874", 2023-10-26T01:38:59.946 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6819/1082279874", 2023-10-26T01:38:59.946 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6828/1082279874", 2023-10-26T01:38:59.946 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6823/1082279874", 2023-10-26T01:38:59.947 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:38:59.947 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:38:59.947 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:38:59.947 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:59.947 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.947 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.948 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 5, 2023-10-26T01:38:59.948 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:38:59.948 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:38:59.948 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:38:59.948 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:38:59.949 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:38:59.949 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:38:59.949 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:38:59.949 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:38:59.949 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 132, 2023-10-26T01:38:59.949 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:38:59.950 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:38:59.950 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:38:59.950 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:59.950 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.950 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:59.951 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6808", 2023-10-26T01:38:59.951 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:38:59.951 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.951 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.951 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:59.951 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6809", 2023-10-26T01:38:59.952 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:38:59.952 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:59.952 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:59.952 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.952 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:38:59.953 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:59.953 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.953 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:59.953 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6810", 2023-10-26T01:38:59.953 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:38:59.953 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.954 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.954 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:59.954 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6811", 2023-10-26T01:38:59.954 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:38:59.954 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:59.955 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:59.955 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.955 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:38:59.955 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:59.955 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.955 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:59.956 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6814", 2023-10-26T01:38:59.956 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:38:59.956 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.956 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.956 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:59.956 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6815", 2023-10-26T01:38:59.957 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:38:59.957 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:59.957 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:59.957 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.957 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:38:59.958 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:59.958 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.958 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:59.958 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6812", 2023-10-26T01:38:59.958 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:38:59.959 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.959 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.959 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:59.959 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6813", 2023-10-26T01:38:59.959 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:38:59.960 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:59.960 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:59.960 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.960 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6809/3335654351", 2023-10-26T01:38:59.960 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6811/3335654351", 2023-10-26T01:38:59.960 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6815/3335654351", 2023-10-26T01:38:59.961 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6813/3335654351", 2023-10-26T01:38:59.961 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:38:59.961 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:38:59.961 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:38:59.961 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:59.962 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.962 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.962 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 6, 2023-10-26T01:38:59.962 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:38:59.962 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:38:59.963 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:38:59.963 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:38:59.963 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:38:59.963 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:38:59.963 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:38:59.963 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:38:59.964 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 132, 2023-10-26T01:38:59.964 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:38:59.964 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:38:59.964 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:38:59.964 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:59.965 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.965 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:59.965 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6800", 2023-10-26T01:38:59.965 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:38:59.965 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.966 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.966 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:59.966 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6801", 2023-10-26T01:38:59.966 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:38:59.966 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:59.967 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:59.967 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.967 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:38:59.967 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:59.967 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.967 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:59.968 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6802", 2023-10-26T01:38:59.968 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:38:59.968 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.968 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.968 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:59.969 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6803", 2023-10-26T01:38:59.969 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:38:59.969 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:59.969 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:59.969 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.969 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:38:59.970 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:59.970 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.970 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:59.970 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6806", 2023-10-26T01:38:59.970 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:38:59.971 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.971 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.971 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:59.971 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6807", 2023-10-26T01:38:59.971 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:38:59.972 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:59.972 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:59.972 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.972 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:38:59.972 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:59.973 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.973 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:59.973 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6804", 2023-10-26T01:38:59.973 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:38:59.973 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.973 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.974 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:59.974 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6805", 2023-10-26T01:38:59.974 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:38:59.974 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:59.974 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:59.975 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.975 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6801/2520460697", 2023-10-26T01:38:59.975 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6803/2520460697", 2023-10-26T01:38:59.975 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6807/2520460697", 2023-10-26T01:38:59.975 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6805/2520460697", 2023-10-26T01:38:59.975 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:38:59.976 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:38:59.976 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:38:59.976 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:59.976 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.976 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.977 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 7, 2023-10-26T01:38:59.977 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:38:59.977 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:38:59.977 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:38:59.977 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:38:59.978 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:38:59.978 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:38:59.978 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:38:59.978 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:38:59.978 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 132, 2023-10-26T01:38:59.979 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:38:59.979 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:38:59.979 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:38:59.979 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:59.979 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.979 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:59.980 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6821", 2023-10-26T01:38:59.980 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:38:59.980 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.980 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.980 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:59.980 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6822", 2023-10-26T01:38:59.981 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:38:59.981 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:59.981 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:59.981 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.981 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:38:59.981 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:59.982 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.982 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:59.982 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6824", 2023-10-26T01:38:59.982 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:38:59.982 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.983 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.983 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:59.983 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6826", 2023-10-26T01:38:59.983 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:38:59.983 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:59.983 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:59.984 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.984 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:38:59.984 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:59.984 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.984 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:59.985 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6830", 2023-10-26T01:38:59.985 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:38:59.985 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.985 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.985 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:59.985 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6831", 2023-10-26T01:38:59.986 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:38:59.986 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:59.986 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:59.986 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.986 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:38:59.987 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:38:59.987 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.987 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:38:59.987 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6827", 2023-10-26T01:38:59.987 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:38:59.987 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.988 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.988 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:38:59.988 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6829", 2023-10-26T01:38:59.988 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:38:59.988 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:59.989 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:59.989 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.989 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6822/1115930619", 2023-10-26T01:38:59.989 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6826/1115930619", 2023-10-26T01:38:59.989 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6831/1115930619", 2023-10-26T01:38:59.989 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6829/1115930619", 2023-10-26T01:38:59.990 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:38:59.990 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:38:59.990 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:38:59.990 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:38:59.990 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:38:59.990 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:38:59.991 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_xinfo": [ 2023-10-26T01:38:59.991 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.991 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 0, 2023-10-26T01:38:59.991 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:38:59.991 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:38:59.992 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:38:59.992 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:38:59.992 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:38:59.992 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.889675+0000", 2023-10-26T01:38:59.992 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:38:59.993 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.993 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.993 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 1, 2023-10-26T01:38:59.993 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:38:59.993 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:38:59.993 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:38:59.994 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:38:59.994 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:38:59.994 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.864157+0000", 2023-10-26T01:38:59.994 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:38:59.994 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.995 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.995 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 2, 2023-10-26T01:38:59.995 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:38:59.995 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:38:59.995 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:38:59.996 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:38:59.996 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:38:59.996 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.876919+0000", 2023-10-26T01:38:59.996 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:38:59.996 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.996 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.997 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 3, 2023-10-26T01:38:59.997 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:38:59.997 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:38:59.997 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:38:59.997 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:38:59.998 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:38:59.998 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.974541+0000", 2023-10-26T01:38:59.998 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:38:59.998 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:38:59.998 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:38:59.999 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 4, 2023-10-26T01:38:59.999 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:38:59.999 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:38:59.999 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:38:59.999 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:38:59.999 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:39:00.000 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.969696+0000", 2023-10-26T01:39:00.000 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:39:00.000 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:39:00.000 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:39:00.000 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 5, 2023-10-26T01:39:00.001 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:39:00.001 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:39:00.001 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:39:00.001 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:39:00.001 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:39:00.001 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.859703+0000", 2023-10-26T01:39:00.002 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:39:00.002 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:39:00.002 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:39:00.002 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 6, 2023-10-26T01:39:00.002 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:39:00.003 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:39:00.003 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:39:00.003 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:39:00.003 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:39:00.003 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.867917+0000", 2023-10-26T01:39:00.003 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:39:00.004 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:39:00.004 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:39:00.004 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 7, 2023-10-26T01:39:00.004 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:39:00.004 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:39:00.004 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:39:00.005 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:39:00.005 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:39:00.005 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.898605+0000", 2023-10-26T01:39:00.005 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:39:00.005 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:39:00.006 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:39:00.006 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_upmap": [], 2023-10-26T01:39:00.006 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_upmap_items": [ 2023-10-26T01:39:00.006 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:39:00.006 INFO:teuthology.orchestra.run.smithi169.stdout: "pgid": "17.e", 2023-10-26T01:39:00.006 INFO:teuthology.orchestra.run.smithi169.stdout: "mappings": [ 2023-10-26T01:39:00.007 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:39:00.007 INFO:teuthology.orchestra.run.smithi169.stdout: "from": 0, 2023-10-26T01:39:00.007 INFO:teuthology.orchestra.run.smithi169.stdout: "to": 5 2023-10-26T01:39:00.007 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:39:00.007 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:39:00.008 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:39:00.008 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:39:00.008 INFO:teuthology.orchestra.run.smithi169.stdout: "pgid": "17.21", 2023-10-26T01:39:00.008 INFO:teuthology.orchestra.run.smithi169.stdout: "mappings": [ 2023-10-26T01:39:00.008 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:39:00.009 INFO:teuthology.orchestra.run.smithi169.stdout: "from": 0, 2023-10-26T01:39:00.009 INFO:teuthology.orchestra.run.smithi169.stdout: "to": 2 2023-10-26T01:39:00.009 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:39:00.009 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:39:00.009 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:39:00.009 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:39:00.010 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_upmap_primaries": [], 2023-10-26T01:39:00.010 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_temp": [], 2023-10-26T01:39:00.010 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_temp": [], 2023-10-26T01:39:00.010 INFO:teuthology.orchestra.run.smithi169.stdout: "blocklist": {}, 2023-10-26T01:39:00.010 INFO:teuthology.orchestra.run.smithi169.stdout: "range_blocklist": {}, 2023-10-26T01:39:00.010 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profiles": { 2023-10-26T01:39:00.011 INFO:teuthology.orchestra.run.smithi169.stdout: "default": { 2023-10-26T01:39:00.011 INFO:teuthology.orchestra.run.smithi169.stdout: "crush-failure-domain": "osd", 2023-10-26T01:39:00.011 INFO:teuthology.orchestra.run.smithi169.stdout: "k": "2", 2023-10-26T01:39:00.011 INFO:teuthology.orchestra.run.smithi169.stdout: "m": "1", 2023-10-26T01:39:00.011 INFO:teuthology.orchestra.run.smithi169.stdout: "plugin": "jerasure", 2023-10-26T01:39:00.011 INFO:teuthology.orchestra.run.smithi169.stdout: "technique": "reed_sol_van" 2023-10-26T01:39:00.012 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:39:00.012 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:39:00.012 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps_queue": [], 2023-10-26T01:39:00.012 INFO:teuthology.orchestra.run.smithi169.stdout: "new_removed_snaps": [], 2023-10-26T01:39:00.012 INFO:teuthology.orchestra.run.smithi169.stdout: "new_purged_snaps": [], 2023-10-26T01:39:00.012 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_node_flags": {}, 2023-10-26T01:39:00.013 INFO:teuthology.orchestra.run.smithi169.stdout: "device_class_flags": {}, 2023-10-26T01:39:00.013 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_mode": { 2023-10-26T01:39:00.013 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_mode_enabled": false, 2023-10-26T01:39:00.013 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_bucket_count": 0, 2023-10-26T01:39:00.013 INFO:teuthology.orchestra.run.smithi169.stdout: "degraded_stretch_mode": 0, 2023-10-26T01:39:00.013 INFO:teuthology.orchestra.run.smithi169.stdout: "recovering_stretch_mode": 0, 2023-10-26T01:39:00.014 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_mode_bucket": 0 2023-10-26T01:39:00.014 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:39:00.014 INFO:teuthology.orchestra.run.smithi169.stdout:} 2023-10-26T01:39:00.016 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:59.812+0000 7f5558b8a640 1 -- 172.21.15.169:0/1714211864 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f553406ae40 msgr2=0x7f553406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:00.016 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:59.812+0000 7f5558b8a640 1 --2- 172.21.15.169:0/1714211864 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f553406ae40 0x7f553406d300 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f5544002230 tx=0x7f5544001da0 comp rx=0 tx=0).stop 2023-10-26T01:39:00.016 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:59.812+0000 7f5558b8a640 1 -- 172.21.15.169:0/1714211864 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55540cd720 msgr2=0x7f55540aeec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:00.017 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:59.812+0000 7f5558b8a640 1 --2- 172.21.15.169:0/1714211864 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55540cd720 0x7f55540aeec0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f5548009780 tx=0x7f554802cdf0 comp rx=0 tx=0).stop 2023-10-26T01:39:00.017 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:59.812+0000 7f5558b8a640 1 -- 172.21.15.169:0/1714211864 shutdown_connections 2023-10-26T01:39:00.017 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:59.812+0000 7f5558b8a640 1 --2- 172.21.15.169:0/1714211864 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f553406ae40 0x7f553406d300 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:00.017 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:59.812+0000 7f5558b8a640 1 --2- 172.21.15.169:0/1714211864 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55540cd720 0x7f55540aeec0 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:00.017 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:59.812+0000 7f5558b8a640 1 --2- 172.21.15.169:0/1714211864 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5554158850 0x7f55540af400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:00.018 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:59.812+0000 7f5558b8a640 1 --2- 172.21.15.169:0/1714211864 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f555406ce20 0x7f55541f3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:00.018 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:59.812+0000 7f5558b8a640 1 -- 172.21.15.169:0/1714211864 >> 172.21.15.169:0/1714211864 conn(0x7f5554069930 msgr2=0x7f5554150380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:00.018 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:59.813+0000 7f5558b8a640 1 -- 172.21.15.169:0/1714211864 shutdown_connections 2023-10-26T01:39:00.018 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:38:59.813+0000 7f5558b8a640 1 -- 172.21.15.169:0/1714211864 wait complete. 2023-10-26T01:39:00.019 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T01:39:00.244 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:00.242+0000 7f508ec58640 1 -- 172.21.15.169:0/2275945334 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5088158850 msgr2=0x7f50880aab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:00.244 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:00.242+0000 7f508ec58640 1 --2- 172.21.15.169:0/2275945334 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5088158850 0x7f50880aab40 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f5088071fc0 tx=0x7f507802bc80 comp rx=0 tx=0).stop 2023-10-26T01:39:00.245 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:00.242+0000 7f508ec58640 1 -- 172.21.15.169:0/2275945334 shutdown_connections 2023-10-26T01:39:00.245 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:00.242+0000 7f508ec58640 1 --2- 172.21.15.169:0/2275945334 >> v2:172.21.15.170:3300/0 conn(0x7f508806ad30 0x7f508806b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:00.245 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:00.242+0000 7f508ec58640 1 --2- 172.21.15.169:0/2275945334 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5088158850 0x7f50880aab40 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:00.245 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:00.242+0000 7f508ec58640 1 --2- 172.21.15.169:0/2275945334 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f508806ce20 0x7f508814e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:00.245 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:00.242+0000 7f508ec58640 1 -- 172.21.15.169:0/2275945334 >> 172.21.15.169:0/2275945334 conn(0x7f5088069930 msgr2=0x7f5088069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:00.245 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:00.242+0000 7f508ec58640 1 -- 172.21.15.169:0/2275945334 shutdown_connections 2023-10-26T01:39:00.246 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:00.242+0000 7f508ec58640 1 -- 172.21.15.169:0/2275945334 wait complete. 2023-10-26T01:39:00.247 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:00.244+0000 7f508ec58640 1 Processor -- start 2023-10-26T01:39:00.247 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:00.244+0000 7f508ec58640 1 -- start start 2023-10-26T01:39:00.248 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:00.245+0000 7f508ec58640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f508806ad30 0x7f50880c79d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:00.248 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:00.245+0000 7f508ec58640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f508806ce20 0x7f50881f5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:00.248 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:00.245+0000 7f508ec58640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5088158850 0x7f50881f18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:00.248 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:00.245+0000 7f508ec58640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50880c7f10 con 0x7f5088158850 2023-10-26T01:39:00.248 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:00.245+0000 7f508ec58640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50881f21b0 con 0x7f508806ce20 2023-10-26T01:39:00.249 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:00.245+0000 7f508ec58640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50881f2490 con 0x7f508806ad30 2023-10-26T01:39:00.249 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:00.245+0000 7f508d1f5640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5088158850 0x7f50881f18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:00.249 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:00.245+0000 7f508d1f5640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5088158850 0x7f50881f18f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:44820/0 (socket says 172.21.15.169:44820) 2023-10-26T01:39:00.249 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:00.245+0000 7f508d1f5640 1 -- 172.21.15.169:0/2423513781 learned_addr learned my addr 172.21.15.169:0/2423513781 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:39:00.249 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:00.245+0000 7f508c9f4640 1 --2- 172.21.15.169:0/2423513781 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f508806ad30 0x7f50880c79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:00.249 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:00.246+0000 7f5087fff640 1 --2- 172.21.15.169:0/2423513781 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f508806ce20 0x7f50881f5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:00.250 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:00.246+0000 7f508d1f5640 1 -- 172.21.15.169:0/2423513781 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f508806ad30 msgr2=0x7f50880c79d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:00.250 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:00.246+0000 7f508d1f5640 1 --2- 172.21.15.169:0/2423513781 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f508806ad30 0x7f50880c79d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:00.250 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:00.246+0000 7f508d1f5640 1 -- 172.21.15.169:0/2423513781 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f508806ce20 msgr2=0x7f50881f5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:00.250 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:00.246+0000 7f508d1f5640 1 --2- 172.21.15.169:0/2423513781 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f508806ce20 0x7f50881f5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:00.250 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:00.246+0000 7f508d1f5640 1 -- 172.21.15.169:0/2423513781 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5078009560 con 0x7f5088158850 2023-10-26T01:39:00.252 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:00.246+0000 7f508d1f5640 1 --2- 172.21.15.169:0/2423513781 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5088158850 0x7f50881f18f0 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f508000cce0 tx=0x7f508000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:00.252 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:00.246+0000 7f50857fa640 1 -- 172.21.15.169:0/2423513781 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f508000f450 con 0x7f5088158850 2023-10-26T01:39:00.252 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:00.246+0000 7f50857fa640 1 -- 172.21.15.169:0/2423513781 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f508000f6a0 con 0x7f5088158850 2023-10-26T01:39:00.252 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:00.246+0000 7f508ec58640 1 -- 172.21.15.169:0/2423513781 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50881f2770 con 0x7f5088158850 2023-10-26T01:39:00.252 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:00.246+0000 7f50857fa640 1 -- 172.21.15.169:0/2423513781 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f508000fab0 con 0x7f5088158850 2023-10-26T01:39:00.253 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:00.246+0000 7f508ec58640 1 -- 172.21.15.169:0/2423513781 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50881f5240 con 0x7f5088158850 2023-10-26T01:39:00.253 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:00.248+0000 7f508ec58640 1 -- 172.21.15.169:0/2423513781 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50881f5430 con 0x7f5088158850 2023-10-26T01:39:00.253 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:00.249+0000 7f50857fa640 1 -- 172.21.15.169:0/2423513781 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f508000fc10 con 0x7f5088158850 2023-10-26T01:39:00.253 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:00.250+0000 7f50857fa640 1 --2- 172.21.15.169:0/2423513781 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f507406ae70 0x7f507406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:00.253 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:00.250+0000 7f50857fa640 1 -- 172.21.15.169:0/2423513781 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f508008a320 con 0x7f5088158850 2023-10-26T01:39:00.254 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:00.250+0000 7f508c9f4640 1 --2- 172.21.15.169:0/2423513781 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f507406ae70 0x7f507406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:00.254 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:00.251+0000 7f508c9f4640 1 --2- 172.21.15.169:0/2423513781 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f507406ae70 0x7f507406d330 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f507c005fd0 tx=0x7f507c005e20 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:00.256 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:00.254+0000 7f50857fa640 1 -- 172.21.15.169:0/2423513781 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5080055260 con 0x7f5088158850 2023-10-26T01:39:00.485 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:00.484+0000 7f508ec58640 1 -- 172.21.15.169:0/2423513781 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f508804db10 con 0x7f5088158850 2023-10-26T01:39:00.486 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:00.485+0000 7f50857fa640 1 -- 172.21.15.169:0/2423513781 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+4927 (secure 0 0 0) 0x7f508001d090 con 0x7f5088158850 2023-10-26T01:39:00.487 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:39:00.487 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":68,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5917,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/3297412140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":3297412140},{"type":"v1","addr":"172.21.15.169:6835","nonce":3297412140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":5929,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/2575809940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2575809940},{"type":"v1","addr":"172.21.15.170:6836","nonce":2575809940}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":7761,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2632217991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2632217991},{"type":"v1","addr":"172.21.15.169:6833","nonce":2632217991}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T01:38:20.773556+0000","modified":"2023-10-26T01:38:23.839758+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7773},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7773":{"gid":7773,"name":"b","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6837/3615088922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":3615088922},{"type":"v1","addr":"172.21.15.170:6837","nonce":3615088922}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-10-26T01:39:00.487 INFO:teuthology.orchestra.run.smithi169.stderr:dumped fsmap epoch 68 2023-10-26T01:39:00.489 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:00.487+0000 7f508ec58640 1 -- 172.21.15.169:0/2423513781 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f507406ae70 msgr2=0x7f507406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:00.490 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:00.487+0000 7f508ec58640 1 --2- 172.21.15.169:0/2423513781 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f507406ae70 0x7f507406d330 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f507c005fd0 tx=0x7f507c005e20 comp rx=0 tx=0).stop 2023-10-26T01:39:00.490 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:00.488+0000 7f508ec58640 1 -- 172.21.15.169:0/2423513781 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5088158850 msgr2=0x7f50881f18f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:00.490 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:00.488+0000 7f508ec58640 1 --2- 172.21.15.169:0/2423513781 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5088158850 0x7f50881f18f0 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f508000cce0 tx=0x7f508000a620 comp rx=0 tx=0).stop 2023-10-26T01:39:00.490 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:00.488+0000 7f508ec58640 1 -- 172.21.15.169:0/2423513781 shutdown_connections 2023-10-26T01:39:00.491 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:00.488+0000 7f508ec58640 1 --2- 172.21.15.169:0/2423513781 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f507406ae70 0x7f507406d330 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:00.491 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:00.488+0000 7f508ec58640 1 --2- 172.21.15.169:0/2423513781 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5088158850 0x7f50881f18f0 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:00.491 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:00.488+0000 7f508ec58640 1 --2- 172.21.15.169:0/2423513781 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f508806ce20 0x7f50881f5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:00.491 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:00.488+0000 7f508ec58640 1 --2- 172.21.15.169:0/2423513781 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f508806ad30 0x7f50880c79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:00.491 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:00.488+0000 7f508ec58640 1 -- 172.21.15.169:0/2423513781 >> 172.21.15.169:0/2423513781 conn(0x7f5088069930 msgr2=0x7f508806c030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:00.492 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:00.488+0000 7f508ec58640 1 -- 172.21.15.169:0/2423513781 shutdown_connections 2023-10-26T01:39:00.492 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:00.488+0000 7f508ec58640 1 -- 172.21.15.169:0/2423513781 wait complete. 2023-10-26T01:39:00.509 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2023-10-26T01:39:00.749 INFO:teuthology.orchestra.run.smithi170.stdout:{"cluster_fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","whoami":0,"id":7773,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":39.409805268,"mdsmap_epoch":68,"osdmap_epoch":139,"osdmap_epoch_barrier":138,"uptime":45.335303568999997} 2023-10-26T01:39:00.749 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "30203b28-ed8a-44e8-a9a6-a9bf25f980f1", "fs_name": "cephfs", "id": 7773, "mdsmap_epoch": 68, "osdmap_epoch": 139, "osdmap_epoch_barrier": 138, "rank_uptime": 39.409805268, "state": "up:active", "uptime": 45.335303569, "want_state": "up:active", "whoami": 0 } 2023-10-26T01:39:00.750 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-26T01:39:00.750 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:39:00.751 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:39:00.810 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:39:00.811 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file bs=1M conv=fdatasync count=50 seek=0) 2023-10-26T01:39:01.316 INFO:teuthology.orchestra.run.smithi169.stderr:50+0 records in 2023-10-26T01:39:01.317 INFO:teuthology.orchestra.run.smithi169.stderr:50+0 records out 2023-10-26T01:39:01.317 INFO:teuthology.orchestra.run.smithi169.stderr:52428800 bytes (52 MB, 50 MiB) copied, 0.449774 s, 117 MB/s 2023-10-26T01:39:01.318 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-26T01:39:01.318 DEBUG:teuthology.orchestra.run.smithi169:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-26T01:39:01.421 DEBUG:teuthology.orchestra.run.smithi170:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-26T01:39:01.555 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-26T01:39:01.555 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:39:01.556 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:39:01.614 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:39:01.614 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:39:01.666 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:39:01.890 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:01.886+0000 7faa4df8a640 1 -- 172.21.15.169:0/2921396099 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faa340047c0 msgr2=0x7faa34004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:01.891 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:01.886+0000 7faa4df8a640 1 --2- 172.21.15.169:0/2921396099 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faa340047c0 0x7faa34004c40 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7faa30001d80 tx=0x7faa340057f0 comp rx=0 tx=0).stop 2023-10-26T01:39:01.891 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:01.887+0000 7faa4df8a640 1 -- 172.21.15.169:0/2921396099 shutdown_connections 2023-10-26T01:39:01.891 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:01.887+0000 7faa4df8a640 1 --2- 172.21.15.169:0/2921396099 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faa340047c0 0x7faa34004c40 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:01.891 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:01.887+0000 7faa4df8a640 1 -- 172.21.15.169:0/2921396099 >> 172.21.15.169:0/2921396099 conn(0x7faa48069930 msgr2=0x7faa48069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:01.891 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:01.887+0000 7faa4df8a640 1 -- 172.21.15.169:0/2921396099 shutdown_connections 2023-10-26T01:39:01.892 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:01.887+0000 7faa4df8a640 1 -- 172.21.15.169:0/2921396099 wait complete. 2023-10-26T01:39:01.892 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:01.888+0000 7faa4df8a640 1 Processor -- start 2023-10-26T01:39:01.892 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:01.889+0000 7faa4df8a640 1 -- start start 2023-10-26T01:39:01.894 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:01.889+0000 7faa4df8a640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faa340047c0 0x7faa4813a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:01.894 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:01.890+0000 7faa4df8a640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7faa481ef3a0 0x7faa4806ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:01.894 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:01.890+0000 7faa4df8a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faa4806f270 0x7faa48150d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:01.894 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:01.890+0000 7faa4df8a640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa4806f6f0 con 0x7faa340047c0 2023-10-26T01:39:01.895 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:01.890+0000 7faa4df8a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa4813d3c0 con 0x7faa4806f270 2023-10-26T01:39:01.895 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:01.890+0000 7faa4df8a640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa4814ffd0 con 0x7faa481ef3a0 2023-10-26T01:39:01.895 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:01.890+0000 7faa477fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faa340047c0 0x7faa4813a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:01.895 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:01.890+0000 7faa477fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faa340047c0 0x7faa4813a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:44852/0 (socket says 172.21.15.169:44852) 2023-10-26T01:39:01.895 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:01.890+0000 7faa477fe640 1 -- 172.21.15.169:0/2813479219 learned_addr learned my addr 172.21.15.169:0/2813479219 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:39:01.896 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:01.890+0000 7faa47fff640 1 --2- 172.21.15.169:0/2813479219 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faa4806f270 0x7faa48150d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:01.896 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:01.890+0000 7faa46ffd640 1 --2- 172.21.15.169:0/2813479219 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7faa481ef3a0 0x7faa4806ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:01.896 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:01.890+0000 7faa477fe640 1 -- 172.21.15.169:0/2813479219 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7faa481ef3a0 msgr2=0x7faa4806ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:01.896 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:01.890+0000 7faa477fe640 1 --2- 172.21.15.169:0/2813479219 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7faa481ef3a0 0x7faa4806ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:01.896 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:01.890+0000 7faa477fe640 1 -- 172.21.15.169:0/2813479219 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faa4806f270 msgr2=0x7faa48150d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:01.896 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:01.890+0000 7faa477fe640 1 --2- 172.21.15.169:0/2813479219 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faa4806f270 0x7faa48150d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:01.897 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:01.890+0000 7faa477fe640 1 -- 172.21.15.169:0/2813479219 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa481502b0 con 0x7faa340047c0 2023-10-26T01:39:01.897 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:01.891+0000 7faa477fe640 1 --2- 172.21.15.169:0/2813479219 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faa340047c0 0x7faa4813a470 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7faa3800c700 tx=0x7faa3800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:01.897 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:01.891+0000 7faa3ffff640 1 -- 172.21.15.169:0/2813479219 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa38005270 con 0x7faa340047c0 2023-10-26T01:39:01.897 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:01.891+0000 7faa3ffff640 1 -- 172.21.15.169:0/2813479219 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7faa3800a640 con 0x7faa340047c0 2023-10-26T01:39:01.897 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:01.891+0000 7faa4df8a640 1 -- 172.21.15.169:0/2813479219 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa48150590 con 0x7faa340047c0 2023-10-26T01:39:01.898 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:01.891+0000 7faa3ffff640 1 -- 172.21.15.169:0/2813479219 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa3800aa50 con 0x7faa340047c0 2023-10-26T01:39:01.898 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:01.891+0000 7faa4df8a640 1 -- 172.21.15.169:0/2813479219 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa4813a9b0 con 0x7faa340047c0 2023-10-26T01:39:01.898 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:01.892+0000 7faa4df8a640 1 -- 172.21.15.169:0/2813479219 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa48150ab0 con 0x7faa340047c0 2023-10-26T01:39:01.899 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:01.893+0000 7faa3ffff640 1 -- 172.21.15.169:0/2813479219 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7faa3800abb0 con 0x7faa340047c0 2023-10-26T01:39:01.899 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:01.894+0000 7faa3ffff640 1 --2- 172.21.15.169:0/2813479219 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7faa2806ae70 0x7faa2806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:01.899 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:01.894+0000 7faa3ffff640 1 -- 172.21.15.169:0/2813479219 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5767+0+0 (secure 0 0 0) 0x7faa38089090 con 0x7faa340047c0 2023-10-26T01:39:01.899 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:01.895+0000 7faa46ffd640 1 --2- 172.21.15.169:0/2813479219 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7faa2806ae70 0x7faa2806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:01.900 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:01.896+0000 7faa46ffd640 1 --2- 172.21.15.169:0/2813479219 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7faa2806ae70 0x7faa2806d330 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7faa30034000 tx=0x7faa30039040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:01.900 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:01.899+0000 7faa3ffff640 1 -- 172.21.15.169:0/2813479219 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7faa48150ab0 con 0x7faa340047c0 2023-10-26T01:39:02.092 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.090+0000 7faa4df8a640 1 -- 172.21.15.169:0/2813479219 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faa48150ab0 con 0x7faa340047c0 2023-10-26T01:39:02.095 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.093+0000 7faa3ffff640 1 -- 172.21.15.169:0/2813479219 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v139) v1 ==== 74+0+16736 (secure 0 0 0) 0x7faa48150ab0 con 0x7faa340047c0 2023-10-26T01:39:02.096 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:39:02.096 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":139,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:38:58.061086+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-10-26T01:38:18.617877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"133","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-10-26T01:38:19.793041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.e","mappings":[{"from":0,"to":5}]},{"pgid":"17.21","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:39:02.098 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.096+0000 7faa4df8a640 1 -- 172.21.15.169:0/2813479219 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7faa2806ae70 msgr2=0x7faa2806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:02.098 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.096+0000 7faa4df8a640 1 --2- 172.21.15.169:0/2813479219 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7faa2806ae70 0x7faa2806d330 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7faa30034000 tx=0x7faa30039040 comp rx=0 tx=0).stop 2023-10-26T01:39:02.098 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.096+0000 7faa4df8a640 1 -- 172.21.15.169:0/2813479219 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faa340047c0 msgr2=0x7faa4813a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:02.098 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.096+0000 7faa4df8a640 1 --2- 172.21.15.169:0/2813479219 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faa340047c0 0x7faa4813a470 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7faa3800c700 tx=0x7faa3800cbc0 comp rx=0 tx=0).stop 2023-10-26T01:39:02.099 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.096+0000 7faa4df8a640 1 -- 172.21.15.169:0/2813479219 shutdown_connections 2023-10-26T01:39:02.099 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.096+0000 7faa4df8a640 1 --2- 172.21.15.169:0/2813479219 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7faa2806ae70 0x7faa2806d330 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:02.099 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.096+0000 7faa4df8a640 1 --2- 172.21.15.169:0/2813479219 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faa4806f270 0x7faa48150d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:02.100 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.096+0000 7faa4df8a640 1 --2- 172.21.15.169:0/2813479219 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7faa481ef3a0 0x7faa4806ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:02.100 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.096+0000 7faa4df8a640 1 --2- 172.21.15.169:0/2813479219 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faa340047c0 0x7faa4813a470 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:02.100 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.096+0000 7faa4df8a640 1 -- 172.21.15.169:0/2813479219 >> 172.21.15.169:0/2813479219 conn(0x7faa48069930 msgr2=0x7faa480adf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:02.100 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.097+0000 7faa4df8a640 1 -- 172.21.15.169:0/2813479219 shutdown_connections 2023-10-26T01:39:02.100 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.097+0000 7faa4df8a640 1 -- 172.21.15.169:0/2813479219 wait complete. 2023-10-26T01:39:02.118 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T01:30:32.200808+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '109', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-26T01:38:18.617877+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '133', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2023-10-26T01:38:19.793041+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '139', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T01:39:02.119 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T01:39:02.329 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.327+0000 7f5f5ed7d640 1 -- 172.21.15.169:0/2176623824 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f58159480 msgr2=0x7f5f580b02e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:02.329 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.327+0000 7f5f5ed7d640 1 --2- 172.21.15.169:0/2176623824 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f58159480 0x7f5f580b02e0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f5f580b20a0 tx=0x7f5f4c02bda0 comp rx=0 tx=0).stop 2023-10-26T01:39:02.329 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.327+0000 7f5f5ed7d640 1 -- 172.21.15.169:0/2176623824 shutdown_connections 2023-10-26T01:39:02.329 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.327+0000 7f5f5ed7d640 1 --2- 172.21.15.169:0/2176623824 >> v2:172.21.15.170:3300/0 conn(0x7f5f580ab4c0 0x7f5f580cd690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:02.330 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.327+0000 7f5f5ed7d640 1 --2- 172.21.15.169:0/2176623824 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5f580b0820 0x7f5f580aaf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:02.330 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.327+0000 7f5f5ed7d640 1 --2- 172.21.15.169:0/2176623824 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f58159480 0x7f5f580b02e0 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:02.330 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.327+0000 7f5f5ed7d640 1 -- 172.21.15.169:0/2176623824 >> 172.21.15.169:0/2176623824 conn(0x7f5f58069930 msgr2=0x7f5f58069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:02.330 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.327+0000 7f5f5ed7d640 1 -- 172.21.15.169:0/2176623824 shutdown_connections 2023-10-26T01:39:02.330 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.327+0000 7f5f5ed7d640 1 -- 172.21.15.169:0/2176623824 wait complete. 2023-10-26T01:39:02.332 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.329+0000 7f5f5ed7d640 1 Processor -- start 2023-10-26T01:39:02.332 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.329+0000 7f5f5ed7d640 1 -- start start 2023-10-26T01:39:02.333 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.330+0000 7f5f5ed7d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f580ab4c0 0x7f5f581f13d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:02.333 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.330+0000 7f5f5ed7d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5f580b0820 0x7f5f58155940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:02.333 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.330+0000 7f5f5ed7d640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5f58159480 0x7f5f58155e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:02.333 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.330+0000 7f5f5ed7d640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f581563c0 con 0x7f5f580ab4c0 2023-10-26T01:39:02.333 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.330+0000 7f5f5ed7d640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f58156600 con 0x7f5f580b0820 2023-10-26T01:39:02.334 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.330+0000 7f5f5ed7d640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f581568e0 con 0x7f5f58159480 2023-10-26T01:39:02.334 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.330+0000 7f5f5cb19640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f580ab4c0 0x7f5f581f13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:02.334 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.330+0000 7f5f5cb19640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f580ab4c0 0x7f5f581f13d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:44870/0 (socket says 172.21.15.169:44870) 2023-10-26T01:39:02.334 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.330+0000 7f5f5cb19640 1 -- 172.21.15.169:0/542090995 learned_addr learned my addr 172.21.15.169:0/542090995 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:39:02.334 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.330+0000 7f5f5cb19640 1 -- 172.21.15.169:0/542090995 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5f58159480 msgr2=0x7f5f58155e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:02.335 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.330+0000 7f5f5cb19640 1 --2- 172.21.15.169:0/542090995 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5f58159480 0x7f5f58155e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:02.335 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.330+0000 7f5f5cb19640 1 -- 172.21.15.169:0/542090995 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5f580b0820 msgr2=0x7f5f58155940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:02.335 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.330+0000 7f5f5cb19640 1 --2- 172.21.15.169:0/542090995 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5f580b0820 0x7f5f58155940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:02.335 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.330+0000 7f5f5cb19640 1 -- 172.21.15.169:0/542090995 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f4c009560 con 0x7f5f580ab4c0 2023-10-26T01:39:02.335 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.331+0000 7f5f5cb19640 1 --2- 172.21.15.169:0/542090995 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f580ab4c0 0x7f5f581f13d0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f5f4c009850 tx=0x7f5f4c02dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:02.336 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.331+0000 7f5f557fa640 1 -- 172.21.15.169:0/542090995 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f4c005650 con 0x7f5f580ab4c0 2023-10-26T01:39:02.337 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.331+0000 7f5f557fa640 1 -- 172.21.15.169:0/542090995 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5f4c0038f0 con 0x7f5f580ab4c0 2023-10-26T01:39:02.337 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.331+0000 7f5f557fa640 1 -- 172.21.15.169:0/542090995 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f4c003ce0 con 0x7f5f580ab4c0 2023-10-26T01:39:02.337 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.331+0000 7f5f5ed7d640 1 -- 172.21.15.169:0/542090995 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f58156bc0 con 0x7f5f580ab4c0 2023-10-26T01:39:02.337 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.331+0000 7f5f5ed7d640 1 -- 172.21.15.169:0/542090995 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f58157180 con 0x7f5f580ab4c0 2023-10-26T01:39:02.338 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.332+0000 7f5f5ed7d640 1 -- 172.21.15.169:0/542090995 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f58066af0 con 0x7f5f580ab4c0 2023-10-26T01:39:02.338 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.333+0000 7f5f557fa640 1 -- 172.21.15.169:0/542090995 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f5f4c03e450 con 0x7f5f580ab4c0 2023-10-26T01:39:02.338 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.334+0000 7f5f557fa640 1 --2- 172.21.15.169:0/542090995 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5f3c06ae40 0x7f5f3c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:02.338 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.334+0000 7f5f557fa640 1 -- 172.21.15.169:0/542090995 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f5f4c0abfe0 con 0x7f5f580ab4c0 2023-10-26T01:39:02.338 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.335+0000 7f5f57fff640 1 --2- 172.21.15.169:0/542090995 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5f3c06ae40 0x7f5f3c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:02.339 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.336+0000 7f5f57fff640 1 --2- 172.21.15.169:0/542090995 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5f3c06ae40 0x7f5f3c06d300 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f5f5814f080 tx=0x7f5f44006c90 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:02.339 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.338+0000 7f5f557fa640 1 -- 172.21.15.169:0/542090995 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5f4c077180 con 0x7f5f580ab4c0 2023-10-26T01:39:02.530 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.528+0000 7f5f5ed7d640 1 -- 172.21.15.169:0/542090995 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5f5804da70 con 0x7f5f580ab4c0 2023-10-26T01:39:02.531 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.529+0000 7f5f557fa640 1 -- 172.21.15.169:0/542090995 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v139) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5f4c07a7a0 con 0x7f5f580ab4c0 2023-10-26T01:39:02.531 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 1 2023-10-26T01:39:02.534 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.532+0000 7f5f5ed7d640 1 -- 172.21.15.169:0/542090995 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5f3c06ae40 msgr2=0x7f5f3c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:02.534 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.532+0000 7f5f5ed7d640 1 --2- 172.21.15.169:0/542090995 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5f3c06ae40 0x7f5f3c06d300 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f5f5814f080 tx=0x7f5f44006c90 comp rx=0 tx=0).stop 2023-10-26T01:39:02.534 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.532+0000 7f5f5ed7d640 1 -- 172.21.15.169:0/542090995 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f580ab4c0 msgr2=0x7f5f581f13d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:02.534 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.532+0000 7f5f5ed7d640 1 --2- 172.21.15.169:0/542090995 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f580ab4c0 0x7f5f581f13d0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f5f4c009850 tx=0x7f5f4c02dd70 comp rx=0 tx=0).stop 2023-10-26T01:39:02.535 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.532+0000 7f5f5ed7d640 1 -- 172.21.15.169:0/542090995 shutdown_connections 2023-10-26T01:39:02.535 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.532+0000 7f5f5ed7d640 1 --2- 172.21.15.169:0/542090995 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5f3c06ae40 0x7f5f3c06d300 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:02.535 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.532+0000 7f5f5ed7d640 1 --2- 172.21.15.169:0/542090995 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5f58159480 0x7f5f58155e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:02.535 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.532+0000 7f5f5ed7d640 1 --2- 172.21.15.169:0/542090995 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5f580b0820 0x7f5f58155940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:02.535 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.532+0000 7f5f5ed7d640 1 --2- 172.21.15.169:0/542090995 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f580ab4c0 0x7f5f581f13d0 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:02.536 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.532+0000 7f5f5ed7d640 1 -- 172.21.15.169:0/542090995 >> 172.21.15.169:0/542090995 conn(0x7f5f58069930 msgr2=0x7f5f580ac2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:02.536 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.532+0000 7f5f5ed7d640 1 -- 172.21.15.169:0/542090995 shutdown_connections 2023-10-26T01:39:02.536 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.532+0000 7f5f5ed7d640 1 -- 172.21.15.169:0/542090995 wait complete. 2023-10-26T01:39:02.553 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-26T01:39:02.777 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.775+0000 7f23c6a52640 1 -- 172.21.15.169:0/2014245402 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f23c00699a0 msgr2=0x7f23c0069e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:02.778 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.775+0000 7f23c6a52640 1 --2- 172.21.15.169:0/2014245402 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f23c00699a0 0x7f23c0069e30 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f23c00ce100 tx=0x7f23b802c1d0 comp rx=0 tx=0).stop 2023-10-26T01:39:02.778 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.775+0000 7f23c6a52640 1 -- 172.21.15.169:0/2014245402 shutdown_connections 2023-10-26T01:39:02.778 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.775+0000 7f23c6a52640 1 --2- 172.21.15.169:0/2014245402 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f23c00699a0 0x7f23c0069e30 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:02.778 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.775+0000 7f23c6a52640 1 --2- 172.21.15.169:0/2014245402 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f23c015c860 0x7f23c00ab650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:02.778 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.775+0000 7f23c6a52640 1 --2- 172.21.15.169:0/2014245402 >> v2:172.21.15.170:3300/0 conn(0x7f23c0158270 0x7f23c00a9fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:02.779 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.775+0000 7f23c6a52640 1 -- 172.21.15.169:0/2014245402 >> 172.21.15.169:0/2014245402 conn(0x7f23c00af110 msgr2=0x7f23c00af520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:02.779 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.776+0000 7f23c6a52640 1 -- 172.21.15.169:0/2014245402 shutdown_connections 2023-10-26T01:39:02.779 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.776+0000 7f23c6a52640 1 -- 172.21.15.169:0/2014245402 wait complete. 2023-10-26T01:39:02.780 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.777+0000 7f23c6a52640 1 Processor -- start 2023-10-26T01:39:02.780 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.778+0000 7f23c6a52640 1 -- start start 2023-10-26T01:39:02.781 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.778+0000 7f23c6a52640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f23c00699a0 0x7f23c0149440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:02.781 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.778+0000 7f23c6a52640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f23c0158270 0x7f23c0155ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:02.781 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.779+0000 7f23c6a52640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f23c015c860 0x7f23c015a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:02.781 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.779+0000 7f23c6a52640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23c0154860 con 0x7f23c00699a0 2023-10-26T01:39:02.781 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.779+0000 7f23c6a52640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23c0154b40 con 0x7f23c015c860 2023-10-26T01:39:02.782 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.779+0000 7f23c6a52640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23c0154e20 con 0x7f23c0158270 2023-10-26T01:39:02.782 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.779+0000 7f23c5a50640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f23c00699a0 0x7f23c0149440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:02.782 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.779+0000 7f23c5a50640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f23c00699a0 0x7f23c0149440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:44882/0 (socket says 172.21.15.169:44882) 2023-10-26T01:39:02.782 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.779+0000 7f23c5a50640 1 -- 172.21.15.169:0/1484465816 learned_addr learned my addr 172.21.15.169:0/1484465816 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:39:02.784 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.779+0000 7f23c5a50640 1 -- 172.21.15.169:0/1484465816 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f23c0158270 msgr2=0x7f23c0155ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:02.784 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.779+0000 7f23c5a50640 1 --2- 172.21.15.169:0/1484465816 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f23c0158270 0x7f23c0155ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:02.784 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.779+0000 7f23c5a50640 1 -- 172.21.15.169:0/1484465816 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f23c015c860 msgr2=0x7f23c015a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:02.784 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.779+0000 7f23c5a50640 1 --2- 172.21.15.169:0/1484465816 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f23c015c860 0x7f23c015a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:02.784 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.779+0000 7f23c5a50640 1 -- 172.21.15.169:0/1484465816 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23b8009900 con 0x7f23c00699a0 2023-10-26T01:39:02.785 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.779+0000 7f23c5a50640 1 --2- 172.21.15.169:0/1484465816 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f23c00699a0 0x7f23c0149440 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f23bc00d770 tx=0x7f23bc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:02.785 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.780+0000 7f23b67fc640 1 -- 172.21.15.169:0/1484465816 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23bc009c30 con 0x7f23c00699a0 2023-10-26T01:39:02.785 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.780+0000 7f23b67fc640 1 -- 172.21.15.169:0/1484465816 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f23bc009e80 con 0x7f23c00699a0 2023-10-26T01:39:02.785 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.780+0000 7f23c6a52640 1 -- 172.21.15.169:0/1484465816 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23c00abda0 con 0x7f23c00699a0 2023-10-26T01:39:02.785 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.780+0000 7f23b67fc640 1 -- 172.21.15.169:0/1484465816 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23bc0106a0 con 0x7f23c00699a0 2023-10-26T01:39:02.786 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.780+0000 7f23c6a52640 1 -- 172.21.15.169:0/1484465816 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23c01553e0 con 0x7f23c00699a0 2023-10-26T01:39:02.786 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.781+0000 7f23c6a52640 1 -- 172.21.15.169:0/1484465816 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23c0073c10 con 0x7f23c00699a0 2023-10-26T01:39:02.786 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.782+0000 7f23b67fc640 1 -- 172.21.15.169:0/1484465816 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f23bc010800 con 0x7f23c00699a0 2023-10-26T01:39:02.787 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.783+0000 7f23b67fc640 1 --2- 172.21.15.169:0/1484465816 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f23a406ae40 0x7f23a406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:02.787 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.783+0000 7f23b67fc640 1 -- 172.21.15.169:0/1484465816 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f23bc089b70 con 0x7f23c00699a0 2023-10-26T01:39:02.787 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.783+0000 7f23c524f640 1 --2- 172.21.15.169:0/1484465816 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f23a406ae40 0x7f23a406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:02.787 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.784+0000 7f23c524f640 1 --2- 172.21.15.169:0/1484465816 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f23a406ae40 0x7f23a406d300 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f23b00058c0 tx=0x7f23b0005850 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:02.788 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.787+0000 7f23b67fc640 1 -- 172.21.15.169:0/1484465816 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f23bc058660 con 0x7f23c00699a0 2023-10-26T01:39:02.979 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.977+0000 7f23c6a52640 1 -- 172.21.15.169:0/1484465816 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f23c015cd70 con 0x7f23c00699a0 2023-10-26T01:39:02.980 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.978+0000 7f23b67fc640 1 -- 172.21.15.169:0/1484465816 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v139) v1 ==== 104+0+11 (secure 0 0 0) 0x7f23bc021020 con 0x7f23c00699a0 2023-10-26T01:39:02.980 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 32 2023-10-26T01:39:02.983 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.981+0000 7f23c6a52640 1 -- 172.21.15.169:0/1484465816 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f23a406ae40 msgr2=0x7f23a406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:02.983 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.981+0000 7f23c6a52640 1 --2- 172.21.15.169:0/1484465816 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f23a406ae40 0x7f23a406d300 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f23b00058c0 tx=0x7f23b0005850 comp rx=0 tx=0).stop 2023-10-26T01:39:02.983 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.981+0000 7f23c6a52640 1 -- 172.21.15.169:0/1484465816 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f23c00699a0 msgr2=0x7f23c0149440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:02.983 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.981+0000 7f23c6a52640 1 --2- 172.21.15.169:0/1484465816 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f23c00699a0 0x7f23c0149440 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f23bc00d770 tx=0x7f23bc00dc30 comp rx=0 tx=0).stop 2023-10-26T01:39:02.983 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.981+0000 7f23c6a52640 1 -- 172.21.15.169:0/1484465816 shutdown_connections 2023-10-26T01:39:02.984 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.981+0000 7f23c6a52640 1 --2- 172.21.15.169:0/1484465816 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f23a406ae40 0x7f23a406d300 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:02.984 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.981+0000 7f23c6a52640 1 --2- 172.21.15.169:0/1484465816 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f23c015c860 0x7f23c015a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:02.984 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.981+0000 7f23c6a52640 1 --2- 172.21.15.169:0/1484465816 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f23c0158270 0x7f23c0155ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:02.984 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.981+0000 7f23c6a52640 1 --2- 172.21.15.169:0/1484465816 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f23c00699a0 0x7f23c0149440 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:02.984 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.981+0000 7f23c6a52640 1 -- 172.21.15.169:0/1484465816 >> 172.21.15.169:0/1484465816 conn(0x7f23c00af110 msgr2=0x7f23c00afe90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:02.985 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.981+0000 7f23c6a52640 1 -- 172.21.15.169:0/1484465816 shutdown_connections 2023-10-26T01:39:02.985 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:02.982+0000 7f23c6a52640 1 -- 172.21.15.169:0/1484465816 wait complete. 2023-10-26T01:39:03.002 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-26T01:39:03.243 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:03.240+0000 7fb3e359e640 1 -- 172.21.15.169:0/480475108 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb3e4158360 msgr2=0x7fb3e40aa6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:03.243 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:03.240+0000 7fb3e359e640 1 --2- 172.21.15.169:0/480475108 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb3e4158360 0x7fb3e40aa6d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb3cc009830 tx=0x7fb3cc02bcc0 comp rx=0 tx=0).stop 2023-10-26T01:39:03.243 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:03.241+0000 7fb3e359e640 1 -- 172.21.15.169:0/480475108 shutdown_connections 2023-10-26T01:39:03.244 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:03.241+0000 7fb3e359e640 1 --2- 172.21.15.169:0/480475108 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb3e4158360 0x7fb3e40aa6d0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:03.244 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:03.241+0000 7fb3e359e640 1 --2- 172.21.15.169:0/480475108 >> v2:172.21.15.170:3300/0 conn(0x7fb3e4157f80 0x7fb3e415b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:03.244 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:03.241+0000 7fb3e359e640 1 -- 172.21.15.169:0/480475108 >> 172.21.15.169:0/480475108 conn(0x7fb3e40af110 msgr2=0x7fb3e40af520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:03.244 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:03.241+0000 7fb3e359e640 1 -- 172.21.15.169:0/480475108 shutdown_connections 2023-10-26T01:39:03.244 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:03.241+0000 7fb3e359e640 1 -- 172.21.15.169:0/480475108 wait complete. 2023-10-26T01:39:03.246 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:03.242+0000 7fb3e359e640 1 Processor -- start 2023-10-26T01:39:03.246 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:03.243+0000 7fb3e359e640 1 -- start start 2023-10-26T01:39:03.246 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:03.243+0000 7fb3e359e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3e40aac10 0x7fb3e4097d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:03.247 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:03.243+0000 7fb3e359e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3e4157f80 0x7fb3e40ca8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:03.247 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:03.243+0000 7fb3e359e640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb3e4158360 0x7fb3e409a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:03.247 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:03.243+0000 7fb3e359e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e4164d00 con 0x7fb3e4157f80 2023-10-26T01:39:03.247 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:03.243+0000 7fb3e359e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e409aa70 con 0x7fb3e40aac10 2023-10-26T01:39:03.247 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:03.243+0000 7fb3e359e640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e409acd0 con 0x7fb3e4158360 2023-10-26T01:39:03.248 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:03.244+0000 7fb3d9d9b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3e4157f80 0x7fb3e40ca8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:03.248 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:03.244+0000 7fb3d9d9b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3e4157f80 0x7fb3e40ca8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:44892/0 (socket says 172.21.15.169:44892) 2023-10-26T01:39:03.248 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:03.244+0000 7fb3d9d9b640 1 -- 172.21.15.169:0/1866032677 learned_addr learned my addr 172.21.15.169:0/1866032677 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:39:03.248 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:03.244+0000 7fb3e259c640 1 --2- 172.21.15.169:0/1866032677 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3e40aac10 0x7fb3e4097d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:03.248 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:03.244+0000 7fb3e2d9d640 1 --2- 172.21.15.169:0/1866032677 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb3e4158360 0x7fb3e409a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:03.248 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:03.244+0000 7fb3d9d9b640 1 -- 172.21.15.169:0/1866032677 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb3e4158360 msgr2=0x7fb3e409a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:03.249 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:03.244+0000 7fb3d9d9b640 1 --2- 172.21.15.169:0/1866032677 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb3e4158360 0x7fb3e409a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:03.249 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:03.244+0000 7fb3d9d9b640 1 -- 172.21.15.169:0/1866032677 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3e40aac10 msgr2=0x7fb3e4097d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:03.249 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:03.244+0000 7fb3d9d9b640 1 --2- 172.21.15.169:0/1866032677 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3e40aac10 0x7fb3e4097d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:03.249 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:03.244+0000 7fb3d9d9b640 1 -- 172.21.15.169:0/1866032677 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3d4009560 con 0x7fb3e4157f80 2023-10-26T01:39:03.251 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:03.245+0000 7fb3d9d9b640 1 --2- 172.21.15.169:0/1866032677 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3e4157f80 0x7fb3e40ca8c0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7fb3cc009950 tx=0x7fb3cc0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:03.251 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:03.245+0000 7fb3db7fe640 1 -- 172.21.15.169:0/1866032677 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3cc003ea0 con 0x7fb3e4157f80 2023-10-26T01:39:03.251 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:03.245+0000 7fb3db7fe640 1 -- 172.21.15.169:0/1866032677 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb3cc005c90 con 0x7fb3e4157f80 2023-10-26T01:39:03.251 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:03.245+0000 7fb3db7fe640 1 -- 172.21.15.169:0/1866032677 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3cc005280 con 0x7fb3e4157f80 2023-10-26T01:39:03.251 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:03.245+0000 7fb3e359e640 1 -- 172.21.15.169:0/1866032677 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3cc009490 con 0x7fb3e4157f80 2023-10-26T01:39:03.252 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:03.245+0000 7fb3e359e640 1 -- 172.21.15.169:0/1866032677 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3e4164f50 con 0x7fb3e4157f80 2023-10-26T01:39:03.252 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:03.246+0000 7fb3e359e640 1 -- 172.21.15.169:0/1866032677 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3e4073c10 con 0x7fb3e4157f80 2023-10-26T01:39:03.252 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:03.247+0000 7fb3db7fe640 1 -- 172.21.15.169:0/1866032677 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fb3cc0053e0 con 0x7fb3e4157f80 2023-10-26T01:39:03.252 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:03.248+0000 7fb3db7fe640 1 --2- 172.21.15.169:0/1866032677 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb3c406ae40 0x7fb3c406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:03.252 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:03.248+0000 7fb3db7fe640 1 -- 172.21.15.169:0/1866032677 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fb3cc0ac1b0 con 0x7fb3e4157f80 2023-10-26T01:39:03.253 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:03.249+0000 7fb3e259c640 1 --2- 172.21.15.169:0/1866032677 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb3c406ae40 0x7fb3c406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:03.253 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:03.250+0000 7fb3e259c640 1 --2- 172.21.15.169:0/1866032677 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb3c406ae40 0x7fb3c406d300 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fb3e40aebb0 tx=0x7fb3d4009260 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:03.254 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:03.252+0000 7fb3db7fe640 1 -- 172.21.15.169:0/1866032677 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb3cc077290 con 0x7fb3e4157f80 2023-10-26T01:39:03.442 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:03.440+0000 7fb3e359e640 1 -- 172.21.15.169:0/1866032677 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb3e40ac910 con 0x7fb3e4157f80 2023-10-26T01:39:03.442 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:03.441+0000 7fb3db7fe640 1 -- 172.21.15.169:0/1866032677 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v139) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb3cc07a8b0 con 0x7fb3e4157f80 2023-10-26T01:39:03.443 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 64 2023-10-26T01:39:03.445 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:03.443+0000 7fb3e359e640 1 -- 172.21.15.169:0/1866032677 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb3c406ae40 msgr2=0x7fb3c406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:03.445 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:03.443+0000 7fb3e359e640 1 --2- 172.21.15.169:0/1866032677 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb3c406ae40 0x7fb3c406d300 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fb3e40aebb0 tx=0x7fb3d4009260 comp rx=0 tx=0).stop 2023-10-26T01:39:03.445 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:03.443+0000 7fb3e359e640 1 -- 172.21.15.169:0/1866032677 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3e4157f80 msgr2=0x7fb3e40ca8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:03.446 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:03.443+0000 7fb3e359e640 1 --2- 172.21.15.169:0/1866032677 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3e4157f80 0x7fb3e40ca8c0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7fb3cc009950 tx=0x7fb3cc0038a0 comp rx=0 tx=0).stop 2023-10-26T01:39:03.446 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:03.443+0000 7fb3e359e640 1 -- 172.21.15.169:0/1866032677 shutdown_connections 2023-10-26T01:39:03.446 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:03.443+0000 7fb3e359e640 1 --2- 172.21.15.169:0/1866032677 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb3c406ae40 0x7fb3c406d300 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:03.446 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:03.443+0000 7fb3e359e640 1 --2- 172.21.15.169:0/1866032677 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3e4157f80 0x7fb3e40ca8c0 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:03.446 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:03.443+0000 7fb3e359e640 1 --2- 172.21.15.169:0/1866032677 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb3e4158360 0x7fb3e409a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:03.447 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:03.443+0000 7fb3e359e640 1 --2- 172.21.15.169:0/1866032677 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3e40aac10 0x7fb3e4097d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:03.447 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:03.443+0000 7fb3e359e640 1 -- 172.21.15.169:0/1866032677 >> 172.21.15.169:0/1866032677 conn(0x7fb3e40af110 msgr2=0x7fb3e4150fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:03.447 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:03.443+0000 7fb3e359e640 1 -- 172.21.15.169:0/1866032677 shutdown_connections 2023-10-26T01:39:03.447 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:03.443+0000 7fb3e359e640 1 -- 172.21.15.169:0/1866032677 wait complete. 2023-10-26T01:39:03.464 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-26T01:39:03.673 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:03.671+0000 7f461e3fb640 1 -- 172.21.15.169:0/275396369 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f46180ac510 msgr2=0x7f4618069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:03.673 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:03.671+0000 7f461e3fb640 1 --2- 172.21.15.169:0/275396369 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f46180ac510 0x7f4618069b30 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f4608005fd0 tx=0x7f460802bd90 comp rx=0 tx=0).stop 2023-10-26T01:39:03.674 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:03.671+0000 7f461e3fb640 1 -- 172.21.15.169:0/275396369 shutdown_connections 2023-10-26T01:39:03.674 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:03.671+0000 7f461e3fb640 1 --2- 172.21.15.169:0/275396369 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f46180ac510 0x7f4618069b30 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:03.674 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:03.671+0000 7f461e3fb640 1 --2- 172.21.15.169:0/275396369 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f46180ac130 0x7f461806aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:03.674 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:03.671+0000 7f461e3fb640 1 -- 172.21.15.169:0/275396369 >> 172.21.15.169:0/275396369 conn(0x7f46180af110 msgr2=0x7f46180af520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:03.674 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:03.671+0000 7f461e3fb640 1 -- 172.21.15.169:0/275396369 shutdown_connections 2023-10-26T01:39:03.675 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:03.671+0000 7f461e3fb640 1 -- 172.21.15.169:0/275396369 wait complete. 2023-10-26T01:39:03.675 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:03.673+0000 7f461e3fb640 1 Processor -- start 2023-10-26T01:39:03.675 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:03.674+0000 7f461e3fb640 1 -- start start 2023-10-26T01:39:03.676 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:03.675+0000 7f461e3fb640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f461806a070 0x7f46180971c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:03.677 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:03.675+0000 7f461e3fb640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f46180ac130 0x7f461806eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:03.677 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:03.675+0000 7f461e3fb640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f46180ac510 0x7f461806eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:03.677 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:03.675+0000 7f461e3fb640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4618156640 con 0x7f46180ac510 2023-10-26T01:39:03.677 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:03.675+0000 7f461e3fb640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4618156920 con 0x7f461806a070 2023-10-26T01:39:03.677 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:03.675+0000 7f461e3fb640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4618156c00 con 0x7f46180ac130 2023-10-26T01:39:03.679 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:03.675+0000 7f461dbfa640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f46180ac510 0x7f461806eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:03.679 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:03.675+0000 7f461d3f9640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f461806a070 0x7f46180971c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:03.679 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:03.675+0000 7f461dbfa640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f46180ac510 0x7f461806eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:44910/0 (socket says 172.21.15.169:44910) 2023-10-26T01:39:03.679 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:03.675+0000 7f461dbfa640 1 -- 172.21.15.169:0/2854305044 learned_addr learned my addr 172.21.15.169:0/2854305044 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:39:03.680 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:03.676+0000 7f461cbf8640 1 --2- 172.21.15.169:0/2854305044 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f46180ac130 0x7f461806eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:03.680 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:03.676+0000 7f461dbfa640 1 -- 172.21.15.169:0/2854305044 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f46180ac130 msgr2=0x7f461806eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:03.680 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:03.676+0000 7f461dbfa640 1 --2- 172.21.15.169:0/2854305044 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f46180ac130 0x7f461806eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:03.680 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:03.676+0000 7f461dbfa640 1 -- 172.21.15.169:0/2854305044 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f461806a070 msgr2=0x7f46180971c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:03.680 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:03.676+0000 7f461dbfa640 1 --2- 172.21.15.169:0/2854305044 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f461806a070 0x7f46180971c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:03.681 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:03.676+0000 7f461dbfa640 1 -- 172.21.15.169:0/2854305044 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4608009560 con 0x7f46180ac510 2023-10-26T01:39:03.681 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:03.676+0000 7f461cbf8640 1 --2- 172.21.15.169:0/2854305044 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f46180ac130 0x7f461806eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T01:39:03.681 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:03.676+0000 7f461dbfa640 1 --2- 172.21.15.169:0/2854305044 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f46180ac510 0x7f461806eff0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f461000c700 tx=0x7f461000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:03.681 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:03.676+0000 7f460dffb640 1 -- 172.21.15.169:0/2854305044 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f461000aa50 con 0x7f46180ac510 2023-10-26T01:39:03.681 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:03.677+0000 7f460dffb640 1 -- 172.21.15.169:0/2854305044 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f461000aca0 con 0x7f46180ac510 2023-10-26T01:39:03.682 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:03.677+0000 7f461e3fb640 1 -- 172.21.15.169:0/2854305044 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4618156ee0 con 0x7f46180ac510 2023-10-26T01:39:03.682 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:03.677+0000 7f460dffb640 1 -- 172.21.15.169:0/2854305044 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4610012680 con 0x7f46180ac510 2023-10-26T01:39:03.682 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:03.677+0000 7f461e3fb640 1 -- 172.21.15.169:0/2854305044 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46181561d0 con 0x7f46180ac510 2023-10-26T01:39:03.683 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:03.678+0000 7f461e3fb640 1 -- 172.21.15.169:0/2854305044 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46180704d0 con 0x7f46180ac510 2023-10-26T01:39:03.683 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:03.679+0000 7f460dffb640 1 -- 172.21.15.169:0/2854305044 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f461001b400 con 0x7f46180ac510 2023-10-26T01:39:03.683 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:03.680+0000 7f460dffb640 1 --2- 172.21.15.169:0/2854305044 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f45fc06ae40 0x7f45fc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:03.683 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:03.680+0000 7f460dffb640 1 -- 172.21.15.169:0/2854305044 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f4610089ac0 con 0x7f46180ac510 2023-10-26T01:39:03.684 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:03.681+0000 7f461d3f9640 1 --2- 172.21.15.169:0/2854305044 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f45fc06ae40 0x7f45fc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:03.684 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:03.682+0000 7f461d3f9640 1 --2- 172.21.15.169:0/2854305044 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f45fc06ae40 0x7f45fc06d300 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f4614005fd0 tx=0x7f461400b390 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:03.685 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:03.684+0000 7f460dffb640 1 -- 172.21.15.169:0/2854305044 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4610054a90 con 0x7f46180ac510 2023-10-26T01:39:03.892 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:03.890+0000 7f461e3fb640 1 -- 172.21.15.169:0/2854305044 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f4618157400 con 0x7f46180ac510 2023-10-26T01:39:03.893 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:03.891+0000 7f460dffb640 1 -- 172.21.15.169:0/2854305044 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v139) v1 ==== 80+0+0 (secure 0 0 0) 0x7f4618157400 con 0x7f46180ac510 2023-10-26T01:39:03.893 INFO:teuthology.orchestra.run.smithi169.stderr:listed 0 entries 2023-10-26T01:39:03.896 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:03.894+0000 7f461e3fb640 1 -- 172.21.15.169:0/2854305044 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f45fc06ae40 msgr2=0x7f45fc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:03.896 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:03.894+0000 7f461e3fb640 1 --2- 172.21.15.169:0/2854305044 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f45fc06ae40 0x7f45fc06d300 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f4614005fd0 tx=0x7f461400b390 comp rx=0 tx=0).stop 2023-10-26T01:39:03.896 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:03.894+0000 7f461e3fb640 1 -- 172.21.15.169:0/2854305044 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f46180ac510 msgr2=0x7f461806eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:03.896 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:03.894+0000 7f461e3fb640 1 --2- 172.21.15.169:0/2854305044 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f46180ac510 0x7f461806eff0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f461000c700 tx=0x7f461000cbc0 comp rx=0 tx=0).stop 2023-10-26T01:39:03.896 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:03.894+0000 7f461e3fb640 1 -- 172.21.15.169:0/2854305044 shutdown_connections 2023-10-26T01:39:03.897 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:03.894+0000 7f461e3fb640 1 --2- 172.21.15.169:0/2854305044 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f45fc06ae40 0x7f45fc06d300 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:03.897 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:03.894+0000 7f461e3fb640 1 --2- 172.21.15.169:0/2854305044 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f46180ac510 0x7f461806eff0 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:03.897 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:03.894+0000 7f461e3fb640 1 --2- 172.21.15.169:0/2854305044 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f46180ac130 0x7f461806eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:03.897 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:03.894+0000 7f461e3fb640 1 --2- 172.21.15.169:0/2854305044 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f461806a070 0x7f46180971c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:03.897 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:03.894+0000 7f461e3fb640 1 -- 172.21.15.169:0/2854305044 >> 172.21.15.169:0/2854305044 conn(0x7f46180af110 msgr2=0x7f46180accf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:03.898 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:03.894+0000 7f461e3fb640 1 -- 172.21.15.169:0/2854305044 shutdown_connections 2023-10-26T01:39:03.898 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:03.894+0000 7f461e3fb640 1 -- 172.21.15.169:0/2854305044 wait complete. 2023-10-26T01:39:03.914 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-26T01:39:03.915 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:39:03.915 DEBUG:teuthology.orchestra.run.smithi169:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-26T01:39:04.074 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi169.front.sepia.ceph.com 2023-10-26T01:39:04.075 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:39:04.075 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-26T01:39:04.117 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-26T01:39:04.117 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:39:04.163 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-26T01:39:04.164 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:39:04.259 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:39:04.484 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:04.482+0000 7f8373193640 1 -- 172.21.15.169:0/1798098160 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f836c06cae0 msgr2=0x7f836c0c7ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:04.485 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:04.482+0000 7f8373193640 1 --2- 172.21.15.169:0/1798098160 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f836c06cae0 0x7f836c0c7ea0 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f836c0b3120 tx=0x7f835c02c080 comp rx=0 tx=0).stop 2023-10-26T01:39:04.485 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:04.482+0000 7f8373193640 1 -- 172.21.15.169:0/1798098160 shutdown_connections 2023-10-26T01:39:04.485 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:04.482+0000 7f8373193640 1 --2- 172.21.15.169:0/1798098160 >> v2:172.21.15.170:3300/0 conn(0x7f836c0c96f0 0x7f836c0c9b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:04.485 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:04.482+0000 7f8373193640 1 --2- 172.21.15.169:0/1798098160 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f836c06cae0 0x7f836c0c7ea0 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:04.486 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:04.482+0000 7f8373193640 1 -- 172.21.15.169:0/1798098160 >> 172.21.15.169:0/1798098160 conn(0x7f836c0aedd0 msgr2=0x7f836c0af1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:04.486 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:04.482+0000 7f8373193640 1 -- 172.21.15.169:0/1798098160 shutdown_connections 2023-10-26T01:39:04.486 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:04.483+0000 7f8373193640 1 -- 172.21.15.169:0/1798098160 wait complete. 2023-10-26T01:39:04.487 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:04.484+0000 7f8373193640 1 Processor -- start 2023-10-26T01:39:04.487 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:04.484+0000 7f8373193640 1 -- start start 2023-10-26T01:39:04.487 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:04.485+0000 7f8373193640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f836c06a990 0x7f836c099ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:04.488 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:04.485+0000 7f8373193640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f836c06cae0 0x7f836c154430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:04.488 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:04.485+0000 7f8373193640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f836c0c96f0 0x7f836c0c6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:04.488 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:04.485+0000 7f8373193640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f836c0aada0 con 0x7f836c06cae0 2023-10-26T01:39:04.488 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:04.485+0000 7f8373193640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f836c0a9ed0 con 0x7f836c06a990 2023-10-26T01:39:04.488 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:04.485+0000 7f8373193640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f836c0aa120 con 0x7f836c0c96f0 2023-10-26T01:39:04.489 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:04.486+0000 7f8371990640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f836c06cae0 0x7f836c154430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:04.489 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:04.486+0000 7f8371990640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f836c06cae0 0x7f836c154430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:39000/0 (socket says 172.21.15.169:39000) 2023-10-26T01:39:04.489 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:04.486+0000 7f8371990640 1 -- 172.21.15.169:0/2081004396 learned_addr learned my addr 172.21.15.169:0/2081004396 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:39:04.490 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:04.486+0000 7f8372191640 1 --2- 172.21.15.169:0/2081004396 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f836c06a990 0x7f836c099ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:04.491 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:04.486+0000 7f8372992640 1 --2- 172.21.15.169:0/2081004396 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f836c0c96f0 0x7f836c0c6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:04.491 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:04.486+0000 7f8371990640 1 -- 172.21.15.169:0/2081004396 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f836c0c96f0 msgr2=0x7f836c0c6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:04.491 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:04.486+0000 7f8371990640 1 --2- 172.21.15.169:0/2081004396 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f836c0c96f0 0x7f836c0c6270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:04.491 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:04.486+0000 7f8371990640 1 -- 172.21.15.169:0/2081004396 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f836c06a990 msgr2=0x7f836c099ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:04.492 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:04.486+0000 7f8371990640 1 --2- 172.21.15.169:0/2081004396 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f836c06a990 0x7f836c099ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:04.492 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:04.486+0000 7f8371990640 1 -- 172.21.15.169:0/2081004396 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f835c009840 con 0x7f836c06cae0 2023-10-26T01:39:04.492 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:04.486+0000 7f8372992640 1 --2- 172.21.15.169:0/2081004396 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f836c0c96f0 0x7f836c0c6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:39:04.492 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:04.486+0000 7f8372191640 1 --2- 172.21.15.169:0/2081004396 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f836c06a990 0x7f836c099ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:39:04.492 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:04.486+0000 7f8371990640 1 --2- 172.21.15.169:0/2081004396 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f836c06cae0 0x7f836c154430 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f836000c4b0 tx=0x7f836000c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:04.492 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:04.487+0000 7f836affd640 1 -- 172.21.15.169:0/2081004396 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8360005480 con 0x7f836c06cae0 2023-10-26T01:39:04.493 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:04.487+0000 7f836affd640 1 -- 172.21.15.169:0/2081004396 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f83600036e0 con 0x7f836c06cae0 2023-10-26T01:39:04.493 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:04.487+0000 7f836affd640 1 -- 172.21.15.169:0/2081004396 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8360003b30 con 0x7f836c06cae0 2023-10-26T01:39:04.493 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:04.487+0000 7f8373193640 1 -- 172.21.15.169:0/2081004396 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f836c0aa400 con 0x7f836c06cae0 2023-10-26T01:39:04.493 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:04.487+0000 7f8373193640 1 -- 172.21.15.169:0/2081004396 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f836c0aa9c0 con 0x7f836c06cae0 2023-10-26T01:39:04.493 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:04.488+0000 7f8373193640 1 -- 172.21.15.169:0/2081004396 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f836c0cb370 con 0x7f836c06cae0 2023-10-26T01:39:04.494 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:04.489+0000 7f836affd640 1 -- 172.21.15.169:0/2081004396 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f8360003c90 con 0x7f836c06cae0 2023-10-26T01:39:04.494 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:04.490+0000 7f836affd640 1 --2- 172.21.15.169:0/2081004396 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f835806ae70 0x7f835806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:04.495 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:04.490+0000 7f836affd640 1 -- 172.21.15.169:0/2081004396 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f8360088eb0 con 0x7f836c06cae0 2023-10-26T01:39:04.495 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:04.490+0000 7f8372191640 1 --2- 172.21.15.169:0/2081004396 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f835806ae70 0x7f835806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:04.495 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:04.492+0000 7f8372191640 1 --2- 172.21.15.169:0/2081004396 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f835806ae70 0x7f835806d330 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f835c035000 tx=0x7f835c004ab0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:04.495 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:04.494+0000 7f836affd640 1 -- 172.21.15.169:0/2081004396 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f836c0cb370 con 0x7f836c06cae0 2023-10-26T01:39:04.685 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:04.683+0000 7f8373193640 1 -- 172.21.15.169:0/2081004396 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f836c0aaee0 con 0x7f836c06cae0 2023-10-26T01:39:04.686 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:04.685+0000 7f836affd640 1 -- 172.21.15.169:0/2081004396 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v139) v1 ==== 74+0+16736 (secure 0 0 0) 0x7f836c0aaee0 con 0x7f836c06cae0 2023-10-26T01:39:04.687 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:39:04.687 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":139,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:38:58.061086+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-10-26T01:38:18.617877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"133","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-10-26T01:38:19.793041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.e","mappings":[{"from":0,"to":5}]},{"pgid":"17.21","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:39:04.689 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:04.687+0000 7f8373193640 1 -- 172.21.15.169:0/2081004396 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f835806ae70 msgr2=0x7f835806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:04.689 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:04.687+0000 7f8373193640 1 --2- 172.21.15.169:0/2081004396 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f835806ae70 0x7f835806d330 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f835c035000 tx=0x7f835c004ab0 comp rx=0 tx=0).stop 2023-10-26T01:39:04.689 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:04.687+0000 7f8373193640 1 -- 172.21.15.169:0/2081004396 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f836c06cae0 msgr2=0x7f836c154430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:04.690 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:04.687+0000 7f8373193640 1 --2- 172.21.15.169:0/2081004396 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f836c06cae0 0x7f836c154430 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f836000c4b0 tx=0x7f836000c970 comp rx=0 tx=0).stop 2023-10-26T01:39:04.690 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:04.687+0000 7f8373193640 1 -- 172.21.15.169:0/2081004396 shutdown_connections 2023-10-26T01:39:04.690 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:04.687+0000 7f8373193640 1 --2- 172.21.15.169:0/2081004396 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f835806ae70 0x7f835806d330 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:04.690 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:04.687+0000 7f8373193640 1 --2- 172.21.15.169:0/2081004396 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f836c06cae0 0x7f836c154430 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:04.690 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:04.687+0000 7f8373193640 1 --2- 172.21.15.169:0/2081004396 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f836c0c96f0 0x7f836c0c6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:04.691 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:04.687+0000 7f8373193640 1 --2- 172.21.15.169:0/2081004396 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f836c06a990 0x7f836c099ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:04.691 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:04.687+0000 7f8373193640 1 -- 172.21.15.169:0/2081004396 >> 172.21.15.169:0/2081004396 conn(0x7f836c0aedd0 msgr2=0x7f836c0c71d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:04.691 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:04.687+0000 7f8373193640 1 -- 172.21.15.169:0/2081004396 shutdown_connections 2023-10-26T01:39:04.691 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:04.687+0000 7f8373193640 1 -- 172.21.15.169:0/2081004396 wait complete. 2023-10-26T01:39:04.708 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T01:30:32.200808+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '109', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-26T01:38:18.617877+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '133', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2023-10-26T01:38:19.793041+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '139', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T01:39:04.709 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T01:39:04.931 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:04.928+0000 7f7761105640 1 -- 172.21.15.169:0/2431588509 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f775c159480 msgr2=0x7f775c0b02e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:04.931 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:04.928+0000 7f7761105640 1 --2- 172.21.15.169:0/2431588509 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f775c159480 0x7f775c0b02e0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f774c005fd0 tx=0x7f774c02bd90 comp rx=0 tx=0).stop 2023-10-26T01:39:04.931 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:04.929+0000 7f7761105640 1 -- 172.21.15.169:0/2431588509 shutdown_connections 2023-10-26T01:39:04.931 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:04.929+0000 7f7761105640 1 --2- 172.21.15.169:0/2431588509 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f775c0b0820 0x7f775c0aaf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:04.932 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:04.929+0000 7f7761105640 1 --2- 172.21.15.169:0/2431588509 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f775c159480 0x7f775c0b02e0 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:04.932 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:04.929+0000 7f7761105640 1 -- 172.21.15.169:0/2431588509 >> 172.21.15.169:0/2431588509 conn(0x7f775c069930 msgr2=0x7f775c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:04.932 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:04.929+0000 7f7761105640 1 -- 172.21.15.169:0/2431588509 shutdown_connections 2023-10-26T01:39:04.932 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:04.929+0000 7f7761105640 1 -- 172.21.15.169:0/2431588509 wait complete. 2023-10-26T01:39:04.935 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:04.930+0000 7f7761105640 1 Processor -- start 2023-10-26T01:39:04.935 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:04.931+0000 7f7761105640 1 -- start start 2023-10-26T01:39:04.935 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:04.932+0000 7f7761105640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f775c0ab4c0 0x7f775c1ef4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:04.935 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:04.932+0000 7f7761105640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f775c0b0820 0x7f775c097d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:04.936 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:04.932+0000 7f7761105640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f775c159480 0x7f775c070e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:04.936 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:04.932+0000 7f7761105640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f775c09b210 con 0x7f775c159480 2023-10-26T01:39:04.936 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:04.932+0000 7f7761105640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f775c09a600 con 0x7f775c0b0820 2023-10-26T01:39:04.936 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:04.932+0000 7f7761105640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f775c09a8e0 con 0x7f775c0ab4c0 2023-10-26T01:39:04.936 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:04.932+0000 7f775b59e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f775c159480 0x7f775c070e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:04.937 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:04.932+0000 7f775b59e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f775c159480 0x7f775c070e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:39026/0 (socket says 172.21.15.169:39026) 2023-10-26T01:39:04.937 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:04.932+0000 7f775b59e640 1 -- 172.21.15.169:0/595872613 learned_addr learned my addr 172.21.15.169:0/595872613 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:39:04.937 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:04.932+0000 7f775ad9d640 1 --2- 172.21.15.169:0/595872613 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f775c0ab4c0 0x7f775c1ef4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:04.937 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:04.932+0000 7f775a59c640 1 --2- 172.21.15.169:0/595872613 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f775c0b0820 0x7f775c097d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:04.937 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:04.932+0000 7f775b59e640 1 -- 172.21.15.169:0/595872613 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f775c0ab4c0 msgr2=0x7f775c1ef4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:04.938 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:04.932+0000 7f775b59e640 1 --2- 172.21.15.169:0/595872613 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f775c0ab4c0 0x7f775c1ef4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:04.938 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:04.932+0000 7f775b59e640 1 -- 172.21.15.169:0/595872613 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f775c0b0820 msgr2=0x7f775c097d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:04.938 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:04.932+0000 7f775b59e640 1 --2- 172.21.15.169:0/595872613 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f775c0b0820 0x7f775c097d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:04.938 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:04.932+0000 7f775b59e640 1 -- 172.21.15.169:0/595872613 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f774c009560 con 0x7f775c159480 2023-10-26T01:39:04.938 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:04.933+0000 7f775b59e640 1 --2- 172.21.15.169:0/595872613 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f775c159480 0x7f775c070e90 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f775400c780 tx=0x7f775400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:04.939 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:04.933+0000 7f77437fe640 1 -- 172.21.15.169:0/595872613 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7754003d80 con 0x7f775c159480 2023-10-26T01:39:04.939 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:04.933+0000 7f77437fe640 1 -- 172.21.15.169:0/595872613 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f775400ce90 con 0x7f775c159480 2023-10-26T01:39:04.939 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:04.933+0000 7f77437fe640 1 -- 172.21.15.169:0/595872613 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77540052d0 con 0x7f775c159480 2023-10-26T01:39:04.939 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:04.933+0000 7f7761105640 1 -- 172.21.15.169:0/595872613 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f775c09abc0 con 0x7f775c159480 2023-10-26T01:39:04.939 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:04.933+0000 7f7761105640 1 -- 172.21.15.169:0/595872613 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f775c14ab80 con 0x7f775c159480 2023-10-26T01:39:04.940 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:04.934+0000 7f7761105640 1 -- 172.21.15.169:0/595872613 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f775c066af0 con 0x7f775c159480 2023-10-26T01:39:04.941 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:04.935+0000 7f77437fe640 1 -- 172.21.15.169:0/595872613 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f7754005430 con 0x7f775c159480 2023-10-26T01:39:04.941 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:04.936+0000 7f77437fe640 1 --2- 172.21.15.169:0/595872613 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f774806ae40 0x7f774806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:04.941 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:04.936+0000 7f77437fe640 1 -- 172.21.15.169:0/595872613 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f7754088cf0 con 0x7f775c159480 2023-10-26T01:39:04.941 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:04.937+0000 7f775ad9d640 1 --2- 172.21.15.169:0/595872613 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f774806ae40 0x7f774806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:04.941 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:04.938+0000 7f775ad9d640 1 --2- 172.21.15.169:0/595872613 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f774806ae40 0x7f774806d300 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f774c035000 tx=0x7f774c004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:04.942 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:04.941+0000 7f77437fe640 1 -- 172.21.15.169:0/595872613 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f775408d070 con 0x7f775c159480 2023-10-26T01:39:05.135 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:05.133+0000 7f7761105640 1 -- 172.21.15.169:0/595872613 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f775c04da70 con 0x7f775c159480 2023-10-26T01:39:05.136 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:05.134+0000 7f77437fe640 1 -- 172.21.15.169:0/595872613 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v139) v1 ==== 93+0+10 (secure 0 0 0) 0x7f77540170a0 con 0x7f775c159480 2023-10-26T01:39:05.136 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 1 2023-10-26T01:39:05.139 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:05.137+0000 7f7761105640 1 -- 172.21.15.169:0/595872613 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f774806ae40 msgr2=0x7f774806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:05.139 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:05.137+0000 7f7761105640 1 --2- 172.21.15.169:0/595872613 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f774806ae40 0x7f774806d300 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f774c035000 tx=0x7f774c004730 comp rx=0 tx=0).stop 2023-10-26T01:39:05.139 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:05.137+0000 7f7761105640 1 -- 172.21.15.169:0/595872613 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f775c159480 msgr2=0x7f775c070e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:05.140 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:05.137+0000 7f7761105640 1 --2- 172.21.15.169:0/595872613 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f775c159480 0x7f775c070e90 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f775400c780 tx=0x7f775400cc40 comp rx=0 tx=0).stop 2023-10-26T01:39:05.140 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:05.137+0000 7f7761105640 1 -- 172.21.15.169:0/595872613 shutdown_connections 2023-10-26T01:39:05.140 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:05.137+0000 7f7761105640 1 --2- 172.21.15.169:0/595872613 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f774806ae40 0x7f774806d300 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:05.140 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:05.137+0000 7f7761105640 1 --2- 172.21.15.169:0/595872613 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f775c159480 0x7f775c070e90 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:05.140 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:05.137+0000 7f7761105640 1 --2- 172.21.15.169:0/595872613 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f775c0b0820 0x7f775c097d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:05.140 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:05.137+0000 7f7761105640 1 --2- 172.21.15.169:0/595872613 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f775c0ab4c0 0x7f775c1ef4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:05.141 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:05.137+0000 7f7761105640 1 -- 172.21.15.169:0/595872613 >> 172.21.15.169:0/595872613 conn(0x7f775c069930 msgr2=0x7f775c069d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:05.141 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:05.137+0000 7f7761105640 1 -- 172.21.15.169:0/595872613 shutdown_connections 2023-10-26T01:39:05.141 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:05.137+0000 7f7761105640 1 -- 172.21.15.169:0/595872613 wait complete. 2023-10-26T01:39:05.158 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-26T01:39:05.379 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:05.377+0000 7f5a2deda640 1 -- 172.21.15.169:0/2431763402 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a281597c0 msgr2=0x7f5a2815dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:05.380 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:05.377+0000 7f5a2deda640 1 --2- 172.21.15.169:0/2431763402 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a281597c0 0x7f5a2815dbf0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f5a280b1ea0 tx=0x7f5a1002bda0 comp rx=0 tx=0).stop 2023-10-26T01:39:05.380 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:05.377+0000 7f5a2deda640 1 -- 172.21.15.169:0/2431763402 shutdown_connections 2023-10-26T01:39:05.380 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:05.377+0000 7f5a2deda640 1 --2- 172.21.15.169:0/2431763402 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a281597c0 0x7f5a2815dbf0 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:05.380 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:05.377+0000 7f5a2deda640 1 --2- 172.21.15.169:0/2431763402 >> v2:172.21.15.170:3300/0 conn(0x7f5a281593e0 0x7f5a280ac200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:05.380 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:05.377+0000 7f5a2deda640 1 -- 172.21.15.169:0/2431763402 >> 172.21.15.169:0/2431763402 conn(0x7f5a28069930 msgr2=0x7f5a28069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:05.381 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:05.377+0000 7f5a2deda640 1 -- 172.21.15.169:0/2431763402 shutdown_connections 2023-10-26T01:39:05.381 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:05.377+0000 7f5a2deda640 1 -- 172.21.15.169:0/2431763402 wait complete. 2023-10-26T01:39:05.382 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:05.379+0000 7f5a2deda640 1 Processor -- start 2023-10-26T01:39:05.382 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:05.379+0000 7f5a2deda640 1 -- start start 2023-10-26T01:39:05.382 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:05.380+0000 7f5a2deda640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5a280b0c80 0x7f5a280c7540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:05.382 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:05.380+0000 7f5a2deda640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a281593e0 0x7f5a2814c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:05.383 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:05.380+0000 7f5a2deda640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5a281597c0 0x7f5a280ad3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:05.383 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:05.380+0000 7f5a2deda640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a2806fe40 con 0x7f5a281593e0 2023-10-26T01:39:05.383 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:05.380+0000 7f5a2deda640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a2806f230 con 0x7f5a280b0c80 2023-10-26T01:39:05.383 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:05.380+0000 7f5a2deda640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a2806f510 con 0x7f5a281597c0 2023-10-26T01:39:05.383 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:05.380+0000 7f5a1ffff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a281593e0 0x7f5a2814c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:05.383 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:05.380+0000 7f5a1ffff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a281593e0 0x7f5a2814c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:39044/0 (socket says 172.21.15.169:39044) 2023-10-26T01:39:05.384 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:05.380+0000 7f5a1ffff640 1 -- 172.21.15.169:0/626690705 learned_addr learned my addr 172.21.15.169:0/626690705 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:39:05.385 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:05.381+0000 7f5a277fe640 1 --2- 172.21.15.169:0/626690705 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5a280b0c80 0x7f5a280c7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:05.385 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:05.381+0000 7f5a27fff640 1 --2- 172.21.15.169:0/626690705 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5a281597c0 0x7f5a280ad3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:05.385 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:05.381+0000 7f5a1ffff640 1 -- 172.21.15.169:0/626690705 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5a281597c0 msgr2=0x7f5a280ad3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:05.385 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:05.381+0000 7f5a1ffff640 1 --2- 172.21.15.169:0/626690705 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5a281597c0 0x7f5a280ad3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:05.386 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:05.381+0000 7f5a1ffff640 1 -- 172.21.15.169:0/626690705 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5a280b0c80 msgr2=0x7f5a280c7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:05.386 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:05.381+0000 7f5a1ffff640 1 --2- 172.21.15.169:0/626690705 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5a280b0c80 0x7f5a280c7540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:05.386 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:05.381+0000 7f5a1ffff640 1 -- 172.21.15.169:0/626690705 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a10009560 con 0x7f5a281593e0 2023-10-26T01:39:05.386 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:05.381+0000 7f5a1ffff640 1 --2- 172.21.15.169:0/626690705 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a281593e0 0x7f5a2814c880 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f5a10009850 tx=0x7f5a1002dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:05.386 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:05.381+0000 7f5a24ff9640 1 -- 172.21.15.169:0/626690705 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a10005650 con 0x7f5a281593e0 2023-10-26T01:39:05.387 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:05.381+0000 7f5a24ff9640 1 -- 172.21.15.169:0/626690705 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5a100038f0 con 0x7f5a281593e0 2023-10-26T01:39:05.387 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:05.381+0000 7f5a24ff9640 1 -- 172.21.15.169:0/626690705 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a10003ce0 con 0x7f5a281593e0 2023-10-26T01:39:05.387 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:05.381+0000 7f5a2deda640 1 -- 172.21.15.169:0/626690705 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a2806f7f0 con 0x7f5a281593e0 2023-10-26T01:39:05.387 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:05.382+0000 7f5a2deda640 1 -- 172.21.15.169:0/626690705 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a280ad900 con 0x7f5a281593e0 2023-10-26T01:39:05.387 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:05.383+0000 7f5a2deda640 1 -- 172.21.15.169:0/626690705 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a2806fff0 con 0x7f5a281593e0 2023-10-26T01:39:05.388 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:05.384+0000 7f5a24ff9640 1 -- 172.21.15.169:0/626690705 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f5a1003e450 con 0x7f5a281593e0 2023-10-26T01:39:05.388 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:05.385+0000 7f5a24ff9640 1 --2- 172.21.15.169:0/626690705 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5a0806ae40 0x7f5a0806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:05.389 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:05.385+0000 7f5a24ff9640 1 -- 172.21.15.169:0/626690705 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f5a100abfb0 con 0x7f5a281593e0 2023-10-26T01:39:05.389 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:05.385+0000 7f5a277fe640 1 --2- 172.21.15.169:0/626690705 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5a0806ae40 0x7f5a0806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:05.389 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:05.386+0000 7f5a277fe640 1 --2- 172.21.15.169:0/626690705 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5a0806ae40 0x7f5a0806d300 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f5a1800ba40 tx=0x7f5a18005cc0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:05.390 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:05.389+0000 7f5a24ff9640 1 -- 172.21.15.169:0/626690705 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5a100deca0 con 0x7f5a281593e0 2023-10-26T01:39:05.580 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:05.578+0000 7f5a2deda640 1 -- 172.21.15.169:0/626690705 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5a280ce090 con 0x7f5a281593e0 2023-10-26T01:39:05.581 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:05.579+0000 7f5a24ff9640 1 -- 172.21.15.169:0/626690705 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v139) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5a2806fff0 con 0x7f5a281593e0 2023-10-26T01:39:05.581 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 32 2023-10-26T01:39:05.584 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:05.581+0000 7f5a2deda640 1 -- 172.21.15.169:0/626690705 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5a0806ae40 msgr2=0x7f5a0806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:05.584 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:05.581+0000 7f5a2deda640 1 --2- 172.21.15.169:0/626690705 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5a0806ae40 0x7f5a0806d300 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f5a1800ba40 tx=0x7f5a18005cc0 comp rx=0 tx=0).stop 2023-10-26T01:39:05.584 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:05.581+0000 7f5a2deda640 1 -- 172.21.15.169:0/626690705 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a281593e0 msgr2=0x7f5a2814c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:05.584 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:05.581+0000 7f5a2deda640 1 --2- 172.21.15.169:0/626690705 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a281593e0 0x7f5a2814c880 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f5a10009850 tx=0x7f5a1002dd70 comp rx=0 tx=0).stop 2023-10-26T01:39:05.584 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:05.582+0000 7f5a2deda640 1 -- 172.21.15.169:0/626690705 shutdown_connections 2023-10-26T01:39:05.585 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:05.582+0000 7f5a2deda640 1 --2- 172.21.15.169:0/626690705 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5a0806ae40 0x7f5a0806d300 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:05.585 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:05.582+0000 7f5a2deda640 1 --2- 172.21.15.169:0/626690705 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a281593e0 0x7f5a2814c880 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:05.585 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:05.582+0000 7f5a2deda640 1 --2- 172.21.15.169:0/626690705 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5a281597c0 0x7f5a280ad3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:05.585 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:05.582+0000 7f5a2deda640 1 --2- 172.21.15.169:0/626690705 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5a280b0c80 0x7f5a280c7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:05.585 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:05.582+0000 7f5a2deda640 1 -- 172.21.15.169:0/626690705 >> 172.21.15.169:0/626690705 conn(0x7f5a28069930 msgr2=0x7f5a28159bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:05.586 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:05.582+0000 7f5a2deda640 1 -- 172.21.15.169:0/626690705 shutdown_connections 2023-10-26T01:39:05.586 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:05.582+0000 7f5a2deda640 1 -- 172.21.15.169:0/626690705 wait complete. 2023-10-26T01:39:05.603 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-26T01:39:05.827 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:05.825+0000 7fd6b58d7640 1 -- 172.21.15.169:0/3226313431 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6b0141cb0 msgr2=0x7fd6b00b14f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:05.828 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:05.825+0000 7fd6b58d7640 1 --2- 172.21.15.169:0/3226313431 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6b0141cb0 0x7fd6b00b14f0 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7fd698005fd0 tx=0x7fd69802be20 comp rx=0 tx=0).stop 2023-10-26T01:39:05.828 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:05.825+0000 7fd6b58d7640 1 -- 172.21.15.169:0/3226313431 shutdown_connections 2023-10-26T01:39:05.828 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:05.825+0000 7fd6b58d7640 1 --2- 172.21.15.169:0/3226313431 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6b0141cb0 0x7fd6b00b14f0 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:05.828 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:05.825+0000 7fd6b58d7640 1 --2- 172.21.15.169:0/3226313431 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd6b00ccda0 0x7fd6b015ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:05.828 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:05.825+0000 7fd6b58d7640 1 -- 172.21.15.169:0/3226313431 >> 172.21.15.169:0/3226313431 conn(0x7fd6b0069930 msgr2=0x7fd6b0069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:05.829 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:05.825+0000 7fd6b58d7640 1 -- 172.21.15.169:0/3226313431 shutdown_connections 2023-10-26T01:39:05.829 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:05.825+0000 7fd6b58d7640 1 -- 172.21.15.169:0/3226313431 wait complete. 2023-10-26T01:39:05.830 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:05.827+0000 7fd6b58d7640 1 Processor -- start 2023-10-26T01:39:05.830 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:05.827+0000 7fd6b58d7640 1 -- start start 2023-10-26T01:39:05.830 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:05.828+0000 7fd6b58d7640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6b00ccda0 0x7fd6b015a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:05.831 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:05.828+0000 7fd6b58d7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd6b00cd7e0 0x7fd6b0155390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:05.831 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:05.828+0000 7fd6b58d7640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd6b0141cb0 0x7fd6b00acd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:05.831 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:05.828+0000 7fd6b58d7640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b01f5670 con 0x7fd6b00ccda0 2023-10-26T01:39:05.831 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:05.828+0000 7fd6b58d7640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b00c8da0 con 0x7fd6b00cd7e0 2023-10-26T01:39:05.831 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:05.828+0000 7fd6b58d7640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b01558d0 con 0x7fd6b0141cb0 2023-10-26T01:39:05.832 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:05.829+0000 7fd6aeffd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6b00ccda0 0x7fd6b015a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:05.832 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:05.829+0000 7fd6aeffd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6b00ccda0 0x7fd6b015a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:39050/0 (socket says 172.21.15.169:39050) 2023-10-26T01:39:05.832 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:05.829+0000 7fd6aeffd640 1 -- 172.21.15.169:0/2670784287 learned_addr learned my addr 172.21.15.169:0/2670784287 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:39:05.833 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:05.829+0000 7fd6aeffd640 1 -- 172.21.15.169:0/2670784287 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd6b0141cb0 msgr2=0x7fd6b00acd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:05.833 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:05.829+0000 7fd6aeffd640 1 --2- 172.21.15.169:0/2670784287 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd6b0141cb0 0x7fd6b00acd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:05.834 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:05.829+0000 7fd6aeffd640 1 -- 172.21.15.169:0/2670784287 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd6b00cd7e0 msgr2=0x7fd6b0155390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:05.834 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:05.829+0000 7fd6aeffd640 1 --2- 172.21.15.169:0/2670784287 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd6b00cd7e0 0x7fd6b0155390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:05.834 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:05.829+0000 7fd6aeffd640 1 -- 172.21.15.169:0/2670784287 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd698009560 con 0x7fd6b00ccda0 2023-10-26T01:39:05.834 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:05.829+0000 7fd6aeffd640 1 --2- 172.21.15.169:0/2670784287 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6b00ccda0 0x7fd6b015a3c0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7fd6a000c950 tx=0x7fd6a000ce10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:05.834 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:05.829+0000 7fd6acff9640 1 -- 172.21.15.169:0/2670784287 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6a0007c70 con 0x7fd6b00ccda0 2023-10-26T01:39:05.835 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:05.830+0000 7fd6acff9640 1 -- 172.21.15.169:0/2670784287 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd6a000f450 con 0x7fd6b00ccda0 2023-10-26T01:39:05.835 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:05.830+0000 7fd6b58d7640 1 -- 172.21.15.169:0/2670784287 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6b01f5880 con 0x7fd6b00ccda0 2023-10-26T01:39:05.835 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:05.830+0000 7fd6acff9640 1 -- 172.21.15.169:0/2670784287 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6a000f860 con 0x7fd6b00ccda0 2023-10-26T01:39:05.835 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:05.830+0000 7fd6b58d7640 1 -- 172.21.15.169:0/2670784287 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6b01f5e40 con 0x7fd6b00ccda0 2023-10-26T01:39:05.835 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:05.831+0000 7fd6b58d7640 1 -- 172.21.15.169:0/2670784287 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6b004db10 con 0x7fd6b00ccda0 2023-10-26T01:39:05.836 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:05.832+0000 7fd6acff9640 1 -- 172.21.15.169:0/2670784287 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fd6a000f9c0 con 0x7fd6b00ccda0 2023-10-26T01:39:05.836 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:05.833+0000 7fd6acff9640 1 --2- 172.21.15.169:0/2670784287 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd69006ae40 0x7fd69006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:05.837 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:05.833+0000 7fd6acff9640 1 -- 172.21.15.169:0/2670784287 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fd6a0089f10 con 0x7fd6b00ccda0 2023-10-26T01:39:05.837 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:05.833+0000 7fd6a7fff640 1 --2- 172.21.15.169:0/2670784287 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd69006ae40 0x7fd69006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:05.837 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:05.834+0000 7fd6a7fff640 1 --2- 172.21.15.169:0/2670784287 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd69006ae40 0x7fd69006d300 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fd698035000 tx=0x7fd6980046c0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:05.838 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:05.837+0000 7fd6acff9640 1 -- 172.21.15.169:0/2670784287 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd6a0055010 con 0x7fd6b00ccda0 2023-10-26T01:39:06.028 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:06.026+0000 7fd6b58d7640 1 -- 172.21.15.169:0/2670784287 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd6b0067400 con 0x7fd6b00ccda0 2023-10-26T01:39:06.029 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:06.027+0000 7fd6acff9640 1 -- 172.21.15.169:0/2670784287 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v139) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd6a0058630 con 0x7fd6b00ccda0 2023-10-26T01:39:06.029 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 64 2023-10-26T01:39:06.032 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:06.029+0000 7fd6b58d7640 1 -- 172.21.15.169:0/2670784287 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd69006ae40 msgr2=0x7fd69006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:06.032 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:06.030+0000 7fd6b58d7640 1 --2- 172.21.15.169:0/2670784287 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd69006ae40 0x7fd69006d300 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fd698035000 tx=0x7fd6980046c0 comp rx=0 tx=0).stop 2023-10-26T01:39:06.032 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:06.030+0000 7fd6b58d7640 1 -- 172.21.15.169:0/2670784287 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6b00ccda0 msgr2=0x7fd6b015a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:06.032 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:06.030+0000 7fd6b58d7640 1 --2- 172.21.15.169:0/2670784287 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6b00ccda0 0x7fd6b015a3c0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7fd6a000c950 tx=0x7fd6a000ce10 comp rx=0 tx=0).stop 2023-10-26T01:39:06.032 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:06.030+0000 7fd6b58d7640 1 -- 172.21.15.169:0/2670784287 shutdown_connections 2023-10-26T01:39:06.033 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:06.030+0000 7fd6b58d7640 1 --2- 172.21.15.169:0/2670784287 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd69006ae40 0x7fd69006d300 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:06.033 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:06.030+0000 7fd6b58d7640 1 --2- 172.21.15.169:0/2670784287 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd6b0141cb0 0x7fd6b00acd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:06.033 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:06.030+0000 7fd6b58d7640 1 --2- 172.21.15.169:0/2670784287 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd6b00cd7e0 0x7fd6b0155390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:06.033 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:06.030+0000 7fd6b58d7640 1 --2- 172.21.15.169:0/2670784287 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6b00ccda0 0x7fd6b015a3c0 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:06.033 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:06.030+0000 7fd6b58d7640 1 -- 172.21.15.169:0/2670784287 >> 172.21.15.169:0/2670784287 conn(0x7fd6b0069930 msgr2=0x7fd6b0157de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:06.034 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:06.030+0000 7fd6b58d7640 1 -- 172.21.15.169:0/2670784287 shutdown_connections 2023-10-26T01:39:06.034 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:06.030+0000 7fd6b58d7640 1 -- 172.21.15.169:0/2670784287 wait complete. 2023-10-26T01:39:06.051 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-26T01:39:06.271 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:06.269+0000 7f26dd34c640 1 -- 172.21.15.169:0/903714993 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26d806ec80 msgr2=0x7f26d80cf420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:06.272 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:06.269+0000 7f26dd34c640 1 --2- 172.21.15.169:0/903714993 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26d806ec80 0x7f26d80cf420 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f26c8005fd0 tx=0x7f26c802c060 comp rx=0 tx=0).stop 2023-10-26T01:39:06.272 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:06.269+0000 7f26dd34c640 1 -- 172.21.15.169:0/903714993 shutdown_connections 2023-10-26T01:39:06.272 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:06.269+0000 7f26dd34c640 1 --2- 172.21.15.169:0/903714993 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26d806ec80 0x7f26d80cf420 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:06.272 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:06.269+0000 7f26dd34c640 1 -- 172.21.15.169:0/903714993 >> 172.21.15.169:0/903714993 conn(0x7f26d8098a60 msgr2=0x7f26d80b4a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:06.273 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:06.269+0000 7f26dd34c640 1 -- 172.21.15.169:0/903714993 shutdown_connections 2023-10-26T01:39:06.273 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:06.269+0000 7f26dd34c640 1 -- 172.21.15.169:0/903714993 wait complete. 2023-10-26T01:39:06.275 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:06.271+0000 7f26dd34c640 1 Processor -- start 2023-10-26T01:39:06.275 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:06.271+0000 7f26dd34c640 1 -- start start 2023-10-26T01:39:06.275 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:06.272+0000 7f26dd34c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f26d806ec80 0x7f26d8163b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:06.276 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:06.272+0000 7f26dd34c640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f26d80cf960 0x7f26d815b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:06.276 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:06.272+0000 7f26dd34c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26d81594c0 0x7f26d8157a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:06.276 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:06.272+0000 7f26dd34c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26d815c120 con 0x7f26d81594c0 2023-10-26T01:39:06.276 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:06.272+0000 7f26dd34c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26d8162ae0 con 0x7f26d806ec80 2023-10-26T01:39:06.276 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:06.272+0000 7f26dd34c640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26d8162d80 con 0x7f26d80cf960 2023-10-26T01:39:06.277 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:06.272+0000 7f26d77fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26d81594c0 0x7f26d8157a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:06.277 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:06.273+0000 7f26d77fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26d81594c0 0x7f26d8157a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:39056/0 (socket says 172.21.15.169:39056) 2023-10-26T01:39:06.277 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:06.273+0000 7f26d77fe640 1 -- 172.21.15.169:0/4036781113 learned_addr learned my addr 172.21.15.169:0/4036781113 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:39:06.277 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:06.273+0000 7f26d6ffd640 1 --2- 172.21.15.169:0/4036781113 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f26d806ec80 0x7f26d8163b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:06.277 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:06.273+0000 7f26d67fc640 1 --2- 172.21.15.169:0/4036781113 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f26d80cf960 0x7f26d815b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:06.278 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:06.273+0000 7f26d77fe640 1 -- 172.21.15.169:0/4036781113 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f26d80cf960 msgr2=0x7f26d815b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:06.278 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:06.273+0000 7f26d77fe640 1 --2- 172.21.15.169:0/4036781113 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f26d80cf960 0x7f26d815b8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:06.278 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:06.273+0000 7f26d77fe640 1 -- 172.21.15.169:0/4036781113 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f26d806ec80 msgr2=0x7f26d8163b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:06.278 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:06.273+0000 7f26d77fe640 1 --2- 172.21.15.169:0/4036781113 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f26d806ec80 0x7f26d8163b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:06.278 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:06.273+0000 7f26d77fe640 1 -- 172.21.15.169:0/4036781113 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26c8009840 con 0x7f26d81594c0 2023-10-26T01:39:06.279 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:06.273+0000 7f26d77fe640 1 --2- 172.21.15.169:0/4036781113 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26d81594c0 0x7f26d8157a60 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f26d000c410 tx=0x7f26d000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:06.279 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:06.274+0000 7f26cf7fe640 1 -- 172.21.15.169:0/4036781113 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d0011790 con 0x7f26d81594c0 2023-10-26T01:39:06.279 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:06.274+0000 7f26cf7fe640 1 -- 172.21.15.169:0/4036781113 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f26d00119e0 con 0x7f26d81594c0 2023-10-26T01:39:06.279 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:06.274+0000 7f26cf7fe640 1 -- 172.21.15.169:0/4036781113 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d0011e30 con 0x7f26d81594c0 2023-10-26T01:39:06.279 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:06.274+0000 7f26dd34c640 1 -- 172.21.15.169:0/4036781113 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26d8163060 con 0x7f26d81594c0 2023-10-26T01:39:06.280 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:06.274+0000 7f26dd34c640 1 -- 172.21.15.169:0/4036781113 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26d8163620 con 0x7f26d81594c0 2023-10-26T01:39:06.281 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:06.275+0000 7f26dd34c640 1 -- 172.21.15.169:0/4036781113 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26d8163810 con 0x7f26d81594c0 2023-10-26T01:39:06.281 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:06.276+0000 7f26cf7fe640 1 -- 172.21.15.169:0/4036781113 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f26d0007020 con 0x7f26d81594c0 2023-10-26T01:39:06.281 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:06.277+0000 7f26cf7fe640 1 --2- 172.21.15.169:0/4036781113 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f26b806ae90 0x7f26b806d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:06.281 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:06.277+0000 7f26cf7fe640 1 -- 172.21.15.169:0/4036781113 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f26d0089540 con 0x7f26d81594c0 2023-10-26T01:39:06.282 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:06.278+0000 7f26d6ffd640 1 --2- 172.21.15.169:0/4036781113 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f26b806ae90 0x7f26b806d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:06.282 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:06.279+0000 7f26d6ffd640 1 --2- 172.21.15.169:0/4036781113 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f26b806ae90 0x7f26b806d350 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f26c00046b0 tx=0x7f26c0009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:06.285 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:06.282+0000 7f26cf7fe640 1 -- 172.21.15.169:0/4036781113 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f26d00546b0 con 0x7f26d81594c0 2023-10-26T01:39:06.489 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:06.488+0000 7f26dd34c640 1 -- 172.21.15.169:0/4036781113 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f26d804db10 con 0x7f26d81594c0 2023-10-26T01:39:06.490 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:06.488+0000 7f26cf7fe640 1 -- 172.21.15.169:0/4036781113 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v139) v1 ==== 80+0+0 (secure 0 0 0) 0x7f26d0057cd0 con 0x7f26d81594c0 2023-10-26T01:39:06.490 INFO:teuthology.orchestra.run.smithi169.stderr:listed 0 entries 2023-10-26T01:39:06.494 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:06.491+0000 7f26dd34c640 1 -- 172.21.15.169:0/4036781113 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f26b806ae90 msgr2=0x7f26b806d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:06.494 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:06.491+0000 7f26dd34c640 1 --2- 172.21.15.169:0/4036781113 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f26b806ae90 0x7f26b806d350 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f26c00046b0 tx=0x7f26c0009040 comp rx=0 tx=0).stop 2023-10-26T01:39:06.494 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:06.492+0000 7f26dd34c640 1 -- 172.21.15.169:0/4036781113 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26d81594c0 msgr2=0x7f26d8157a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:06.494 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:06.492+0000 7f26dd34c640 1 --2- 172.21.15.169:0/4036781113 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26d81594c0 0x7f26d8157a60 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f26d000c410 tx=0x7f26d000c8d0 comp rx=0 tx=0).stop 2023-10-26T01:39:06.494 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:06.492+0000 7f26dd34c640 1 -- 172.21.15.169:0/4036781113 shutdown_connections 2023-10-26T01:39:06.495 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:06.492+0000 7f26dd34c640 1 --2- 172.21.15.169:0/4036781113 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f26b806ae90 0x7f26b806d350 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:06.495 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:06.492+0000 7f26dd34c640 1 --2- 172.21.15.169:0/4036781113 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26d81594c0 0x7f26d8157a60 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:06.495 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:06.492+0000 7f26dd34c640 1 --2- 172.21.15.169:0/4036781113 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f26d80cf960 0x7f26d815b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:06.495 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:06.492+0000 7f26dd34c640 1 --2- 172.21.15.169:0/4036781113 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f26d806ec80 0x7f26d8163b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:06.495 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:06.492+0000 7f26dd34c640 1 -- 172.21.15.169:0/4036781113 >> 172.21.15.169:0/4036781113 conn(0x7f26d8098a60 msgr2=0x7f26d80ad9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:06.496 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:06.492+0000 7f26dd34c640 1 -- 172.21.15.169:0/4036781113 shutdown_connections 2023-10-26T01:39:06.496 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:06.492+0000 7f26dd34c640 1 -- 172.21.15.169:0/4036781113 wait complete. 2023-10-26T01:39:06.513 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-26T01:39:06.513 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:39:06.514 DEBUG:teuthology.orchestra.run.smithi170:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-26T01:39:06.632 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi170.front.sepia.ceph.com 2023-10-26T01:39:06.632 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:39:06.633 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-26T01:39:06.638 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T01:39:06.647 DEBUG:teuthology.orchestra.run.smithi170:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T01:39:06.687 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:39:06.688 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:39:06.739 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054301,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-26T01:39:06.739 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-26T01:39:06.740 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:39:06.789 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054303,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-26T01:39:06.790 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T01:39:07.038 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.036+0000 7f7ca4626640 1 -- 172.21.15.169:0/3564277875 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c88005e10 msgr2=0x7f7c88006290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:07.039 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.036+0000 7f7ca4626640 1 --2- 172.21.15.169:0/3564277875 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c88005e10 0x7f7c88006290 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f7c9400bb20 tx=0x7f7c88006900 comp rx=0 tx=0).stop 2023-10-26T01:39:07.039 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.036+0000 7f7ca2bc3640 1 -- 172.21.15.169:0/3564277875 reap_dead start 2023-10-26T01:39:07.039 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.036+0000 7f7ca4626640 1 -- 172.21.15.169:0/3564277875 shutdown_connections 2023-10-26T01:39:07.039 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.036+0000 7f7ca4626640 1 -- 172.21.15.169:0/3564277875 >> 172.21.15.169:0/3564277875 conn(0x7f7c9c069930 msgr2=0x7f7c9c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:07.040 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.036+0000 7f7ca4626640 1 -- 172.21.15.169:0/3564277875 shutdown_connections 2023-10-26T01:39:07.040 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.036+0000 7f7ca4626640 1 -- 172.21.15.169:0/3564277875 wait complete. 2023-10-26T01:39:07.040 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.038+0000 7f7ca4626640 1 Processor -- start 2023-10-26T01:39:07.040 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.038+0000 7f7ca4626640 1 -- start start 2023-10-26T01:39:07.042 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.039+0000 7f7ca4626640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c88005e10 0x7f7c9c148730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:07.042 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.039+0000 7f7ca4626640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7c9c149c30 0x7f7c9c1ef850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:07.043 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.039+0000 7f7ca4626640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c9c0972d0 0x7f7c9c097750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:07.043 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.039+0000 7f7ca4626640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c9c146e10 con 0x7f7c88005e10 2023-10-26T01:39:07.043 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.039+0000 7f7ca4626640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c9c097c90 con 0x7f7c9c0972d0 2023-10-26T01:39:07.043 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.039+0000 7f7ca4626640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c9c148c70 con 0x7f7c9c149c30 2023-10-26T01:39:07.043 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.039+0000 7f7ca23c2640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c88005e10 0x7f7c9c148730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:07.044 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.040+0000 7f7ca23c2640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c88005e10 0x7f7c9c148730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:39066/0 (socket says 172.21.15.169:39066) 2023-10-26T01:39:07.044 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.040+0000 7f7ca23c2640 1 -- 172.21.15.169:0/2286510672 learned_addr learned my addr 172.21.15.169:0/2286510672 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:39:07.044 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.040+0000 7f7ca23c2640 1 -- 172.21.15.169:0/2286510672 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7c9c149c30 msgr2=0x7f7c9c1ef850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:07.044 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.040+0000 7f7ca2bc3640 1 --2- 172.21.15.169:0/2286510672 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c9c0972d0 0x7f7c9c097750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:07.044 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.040+0000 7f7ca1bc1640 1 --2- 172.21.15.169:0/2286510672 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7c9c149c30 0x7f7c9c1ef850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:07.045 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.040+0000 7f7ca23c2640 1 --2- 172.21.15.169:0/2286510672 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7c9c149c30 0x7f7c9c1ef850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:07.045 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.040+0000 7f7ca23c2640 1 -- 172.21.15.169:0/2286510672 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c9c0972d0 msgr2=0x7f7c9c097750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:07.045 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.040+0000 7f7ca23c2640 1 --2- 172.21.15.169:0/2286510672 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c9c0972d0 0x7f7c9c097750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:07.045 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.040+0000 7f7ca23c2640 1 -- 172.21.15.169:0/2286510672 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c9c0ade40 con 0x7f7c88005e10 2023-10-26T01:39:07.045 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.040+0000 7f7ca2bc3640 1 --2- 172.21.15.169:0/2286510672 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c9c0972d0 0x7f7c9c097750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:39:07.046 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.040+0000 7f7ca23c2640 1 --2- 172.21.15.169:0/2286510672 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c88005e10 0x7f7c9c148730 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f7c9800d970 tx=0x7f7c9800de30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:07.046 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.041+0000 7f7c92ffd640 1 -- 172.21.15.169:0/2286510672 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c9800bbe0 con 0x7f7c88005e10 2023-10-26T01:39:07.046 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.041+0000 7f7c92ffd640 1 -- 172.21.15.169:0/2286510672 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7c9800be30 con 0x7f7c88005e10 2023-10-26T01:39:07.046 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.041+0000 7f7c92ffd640 1 -- 172.21.15.169:0/2286510672 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c98010690 con 0x7f7c88005e10 2023-10-26T01:39:07.046 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.041+0000 7f7ca4626640 1 -- 172.21.15.169:0/2286510672 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c9c148ea0 con 0x7f7c88005e10 2023-10-26T01:39:07.047 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.041+0000 7f7ca4626640 1 -- 172.21.15.169:0/2286510672 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c9c149460 con 0x7f7c88005e10 2023-10-26T01:39:07.047 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.042+0000 7f7ca4626640 1 -- 172.21.15.169:0/2286510672 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c9c14c010 con 0x7f7c88005e10 2023-10-26T01:39:07.048 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.043+0000 7f7c92ffd640 1 -- 172.21.15.169:0/2286510672 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f7c980107f0 con 0x7f7c88005e10 2023-10-26T01:39:07.049 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.044+0000 7f7c92ffd640 1 --2- 172.21.15.169:0/2286510672 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7c8006ae40 0x7f7c8006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:07.049 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.044+0000 7f7c92ffd640 1 -- 172.21.15.169:0/2286510672 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f7c98089cf0 con 0x7f7c88005e10 2023-10-26T01:39:07.049 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.044+0000 7f7ca1bc1640 1 --2- 172.21.15.169:0/2286510672 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7c8006ae40 0x7f7c8006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:07.049 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.045+0000 7f7ca1bc1640 1 --2- 172.21.15.169:0/2286510672 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7c8006ae40 0x7f7c8006d300 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f7c8c001eb0 tx=0x7f7c8c0014b0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:07.050 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.048+0000 7f7c92ffd640 1 -- 172.21.15.169:0/2286510672 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7c98054d50 con 0x7f7c88005e10 2023-10-26T01:39:07.280 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.278+0000 7f7ca4626640 1 -- 172.21.15.169:0/2286510672 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7c9c06baa0 con 0x7f7c88005e10 2023-10-26T01:39:07.281 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.279+0000 7f7c92ffd640 1 -- 172.21.15.169:0/2286510672 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+4927 (secure 0 0 0) 0x7f7c9801e020 con 0x7f7c88005e10 2023-10-26T01:39:07.282 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:39:07.282 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":68,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5917,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/3297412140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":3297412140},{"type":"v1","addr":"172.21.15.169:6835","nonce":3297412140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":5929,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/2575809940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2575809940},{"type":"v1","addr":"172.21.15.170:6836","nonce":2575809940}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":7761,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2632217991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2632217991},{"type":"v1","addr":"172.21.15.169:6833","nonce":2632217991}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T01:38:20.773556+0000","modified":"2023-10-26T01:38:23.839758+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7773},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7773":{"gid":7773,"name":"b","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6837/3615088922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":3615088922},{"type":"v1","addr":"172.21.15.170:6837","nonce":3615088922}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-10-26T01:39:07.282 INFO:teuthology.orchestra.run.smithi169.stderr:dumped fsmap epoch 68 2023-10-26T01:39:07.284 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.282+0000 7f7ca4626640 1 -- 172.21.15.169:0/2286510672 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7c8006ae40 msgr2=0x7f7c8006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:07.284 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.282+0000 7f7ca4626640 1 --2- 172.21.15.169:0/2286510672 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7c8006ae40 0x7f7c8006d300 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f7c8c001eb0 tx=0x7f7c8c0014b0 comp rx=0 tx=0).stop 2023-10-26T01:39:07.284 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.282+0000 7f7ca4626640 1 -- 172.21.15.169:0/2286510672 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c88005e10 msgr2=0x7f7c9c148730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:07.285 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.282+0000 7f7ca4626640 1 --2- 172.21.15.169:0/2286510672 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c88005e10 0x7f7c9c148730 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f7c9800d970 tx=0x7f7c9800de30 comp rx=0 tx=0).stop 2023-10-26T01:39:07.285 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.282+0000 7f7ca4626640 1 -- 172.21.15.169:0/2286510672 shutdown_connections 2023-10-26T01:39:07.285 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.282+0000 7f7ca4626640 1 --2- 172.21.15.169:0/2286510672 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7c8006ae40 0x7f7c8006d300 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:07.285 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.282+0000 7f7ca4626640 1 --2- 172.21.15.169:0/2286510672 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c9c0972d0 0x7f7c9c097750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:07.285 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.282+0000 7f7ca4626640 1 --2- 172.21.15.169:0/2286510672 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7c9c149c30 0x7f7c9c1ef850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:07.286 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.282+0000 7f7ca4626640 1 --2- 172.21.15.169:0/2286510672 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c88005e10 0x7f7c9c148730 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:07.286 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.282+0000 7f7ca4626640 1 -- 172.21.15.169:0/2286510672 >> 172.21.15.169:0/2286510672 conn(0x7f7c9c069930 msgr2=0x7f7c9c15b670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:07.286 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.282+0000 7f7ca4626640 1 -- 172.21.15.169:0/2286510672 shutdown_connections 2023-10-26T01:39:07.286 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.283+0000 7f7ca4626640 1 -- 172.21.15.169:0/2286510672 wait complete. 2023-10-26T01:39:07.304 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:39:07.513 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.511+0000 7f51c816c640 1 -- 172.21.15.169:0/2392627763 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f51c006ce20 msgr2=0x7f51c014e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:07.513 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.511+0000 7f51c816c640 1 --2- 172.21.15.169:0/2392627763 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f51c006ce20 0x7f51c014e790 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f51bc005fd0 tx=0x7f51bc02bc60 comp rx=0 tx=0).stop 2023-10-26T01:39:07.514 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.511+0000 7f51c816c640 1 -- 172.21.15.169:0/2392627763 shutdown_connections 2023-10-26T01:39:07.514 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.511+0000 7f51c816c640 1 --2- 172.21.15.169:0/2392627763 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f51c0158850 0x7f51c00aab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:07.514 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.511+0000 7f51c816c640 1 --2- 172.21.15.169:0/2392627763 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f51c006ce20 0x7f51c014e790 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:07.514 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.511+0000 7f51c816c640 1 -- 172.21.15.169:0/2392627763 >> 172.21.15.169:0/2392627763 conn(0x7f51c0069930 msgr2=0x7f51c0069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:07.515 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.511+0000 7f51c816c640 1 -- 172.21.15.169:0/2392627763 shutdown_connections 2023-10-26T01:39:07.515 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.511+0000 7f51c816c640 1 -- 172.21.15.169:0/2392627763 wait complete. 2023-10-26T01:39:07.517 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.513+0000 7f51c816c640 1 Processor -- start 2023-10-26T01:39:07.518 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.513+0000 7f51c816c640 1 -- start start 2023-10-26T01:39:07.518 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.514+0000 7f51c816c640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f51c006ad30 0x7f51c0145990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:07.518 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.514+0000 7f51c816c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f51c006ce20 0x7f51c0146690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:07.518 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.514+0000 7f51c816c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f51c0158850 0x7f51c0146bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:07.518 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.514+0000 7f51c816c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51c0145040 con 0x7f51c0158850 2023-10-26T01:39:07.519 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.514+0000 7f51c816c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51c00acf30 con 0x7f51c006ce20 2023-10-26T01:39:07.519 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.514+0000 7f51c816c640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51c0156580 con 0x7f51c006ad30 2023-10-26T01:39:07.519 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.514+0000 7f51c6709640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f51c0158850 0x7f51c0146bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:07.519 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.514+0000 7f51c6709640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f51c0158850 0x7f51c0146bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:39082/0 (socket says 172.21.15.169:39082) 2023-10-26T01:39:07.519 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.514+0000 7f51c6709640 1 -- 172.21.15.169:0/998922097 learned_addr learned my addr 172.21.15.169:0/998922097 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:39:07.520 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.515+0000 7f51c5f08640 1 --2- 172.21.15.169:0/998922097 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f51c006ad30 0x7f51c0145990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:07.520 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.515+0000 7f51c5707640 1 --2- 172.21.15.169:0/998922097 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f51c006ce20 0x7f51c0146690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:07.520 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.515+0000 7f51c6709640 1 -- 172.21.15.169:0/998922097 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f51c006ad30 msgr2=0x7f51c0145990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:07.520 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.515+0000 7f51c6709640 1 --2- 172.21.15.169:0/998922097 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f51c006ad30 0x7f51c0145990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:07.520 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.515+0000 7f51c6709640 1 -- 172.21.15.169:0/998922097 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f51c006ce20 msgr2=0x7f51c0146690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:07.521 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.515+0000 7f51c6709640 1 --2- 172.21.15.169:0/998922097 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f51c006ce20 0x7f51c0146690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:07.521 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.515+0000 7f51c6709640 1 -- 172.21.15.169:0/998922097 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51bc009560 con 0x7f51c0158850 2023-10-26T01:39:07.521 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.515+0000 7f51c6709640 1 --2- 172.21.15.169:0/998922097 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f51c0158850 0x7f51c0146bd0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f51b800c780 tx=0x7f51b800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:07.521 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.515+0000 7f51b67fc640 1 -- 172.21.15.169:0/998922097 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b8003d80 con 0x7f51c0158850 2023-10-26T01:39:07.521 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.515+0000 7f51b67fc640 1 -- 172.21.15.169:0/998922097 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f51b800ce90 con 0x7f51c0158850 2023-10-26T01:39:07.522 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.515+0000 7f51c816c640 1 -- 172.21.15.169:0/998922097 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51c0156860 con 0x7f51c0158850 2023-10-26T01:39:07.522 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.515+0000 7f51b67fc640 1 -- 172.21.15.169:0/998922097 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b80052d0 con 0x7f51c0158850 2023-10-26T01:39:07.522 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.516+0000 7f51c816c640 1 -- 172.21.15.169:0/998922097 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51c0156e20 con 0x7f51c0158850 2023-10-26T01:39:07.526 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.517+0000 7f51c816c640 1 -- 172.21.15.169:0/998922097 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51c00ad0e0 con 0x7f51c0158850 2023-10-26T01:39:07.527 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.518+0000 7f51b67fc640 1 -- 172.21.15.169:0/998922097 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f51b8005430 con 0x7f51c0158850 2023-10-26T01:39:07.527 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.519+0000 7f51b67fc640 1 --2- 172.21.15.169:0/998922097 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f51a406ae40 0x7f51a406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:07.527 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.519+0000 7f51b67fc640 1 -- 172.21.15.169:0/998922097 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f51b8089180 con 0x7f51c0158850 2023-10-26T01:39:07.527 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.519+0000 7f51c5f08640 1 --2- 172.21.15.169:0/998922097 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f51a406ae40 0x7f51a406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:07.527 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.521+0000 7f51c5f08640 1 --2- 172.21.15.169:0/998922097 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f51a406ae40 0x7f51a406d300 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f51bc035000 tx=0x7f51bc004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:07.528 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.523+0000 7f51b67fc640 1 -- 172.21.15.169:0/998922097 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f51c00ad0e0 con 0x7f51c0158850 2023-10-26T01:39:07.719 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.717+0000 7f51c816c640 1 -- 172.21.15.169:0/998922097 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f51c00ad0a0 con 0x7f51c0158850 2023-10-26T01:39:07.722 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.720+0000 7f51b67fc640 1 -- 172.21.15.169:0/998922097 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v139) v1 ==== 74+0+16736 (secure 0 0 0) 0x7f51b80541e0 con 0x7f51c0158850 2023-10-26T01:39:07.723 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:39:07.723 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":139,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:38:58.061086+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-10-26T01:38:18.617877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"133","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-10-26T01:38:19.793041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.e","mappings":[{"from":0,"to":5}]},{"pgid":"17.21","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:39:07.725 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.723+0000 7f51c816c640 1 -- 172.21.15.169:0/998922097 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f51a406ae40 msgr2=0x7f51a406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:07.725 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.723+0000 7f51c816c640 1 --2- 172.21.15.169:0/998922097 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f51a406ae40 0x7f51a406d300 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f51bc035000 tx=0x7f51bc004730 comp rx=0 tx=0).stop 2023-10-26T01:39:07.725 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.723+0000 7f51c816c640 1 -- 172.21.15.169:0/998922097 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f51c0158850 msgr2=0x7f51c0146bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:07.726 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.723+0000 7f51c816c640 1 --2- 172.21.15.169:0/998922097 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f51c0158850 0x7f51c0146bd0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f51b800c780 tx=0x7f51b800cc40 comp rx=0 tx=0).stop 2023-10-26T01:39:07.726 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.723+0000 7f51c816c640 1 -- 172.21.15.169:0/998922097 shutdown_connections 2023-10-26T01:39:07.726 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.723+0000 7f51c816c640 1 --2- 172.21.15.169:0/998922097 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f51a406ae40 0x7f51a406d300 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:07.726 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.723+0000 7f51c816c640 1 --2- 172.21.15.169:0/998922097 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f51c0158850 0x7f51c0146bd0 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:07.726 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.723+0000 7f51c816c640 1 --2- 172.21.15.169:0/998922097 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f51c006ce20 0x7f51c0146690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:07.727 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.723+0000 7f51c816c640 1 --2- 172.21.15.169:0/998922097 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f51c006ad30 0x7f51c0145990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:07.727 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.723+0000 7f51c816c640 1 -- 172.21.15.169:0/998922097 >> 172.21.15.169:0/998922097 conn(0x7f51c0069930 msgr2=0x7f51c014ed60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:07.727 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.723+0000 7f51c816c640 1 -- 172.21.15.169:0/998922097 shutdown_connections 2023-10-26T01:39:07.727 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.723+0000 7f51c816c640 1 -- 172.21.15.169:0/998922097 wait complete. 2023-10-26T01:39:07.743 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T01:30:32.200808+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '109', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-26T01:38:18.617877+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '133', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2023-10-26T01:38:19.793041+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '139', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T01:39:07.744 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T01:39:07.968 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.966+0000 7fac6ce02640 1 -- 172.21.15.169:0/3074503011 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fac6815ba30 msgr2=0x7fac6806a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:07.968 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.966+0000 7fac6ce02640 1 --2- 172.21.15.169:0/3074503011 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fac6815ba30 0x7fac6806a3f0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7fac680cbd90 tx=0x7fac5802bda0 comp rx=0 tx=0).stop 2023-10-26T01:39:07.969 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.966+0000 7fac6ce02640 1 -- 172.21.15.169:0/3074503011 shutdown_connections 2023-10-26T01:39:07.969 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.966+0000 7fac6ce02640 1 --2- 172.21.15.169:0/3074503011 >> v2:172.21.15.170:3300/0 conn(0x7fac680b3130 0x7fac680b35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:07.969 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.966+0000 7fac6ce02640 1 --2- 172.21.15.169:0/3074503011 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fac680b0710 0x7fac680b48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:07.969 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.966+0000 7fac6ce02640 1 --2- 172.21.15.169:0/3074503011 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fac6815ba30 0x7fac6806a3f0 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:07.969 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.966+0000 7fac6ce02640 1 -- 172.21.15.169:0/3074503011 >> 172.21.15.169:0/3074503011 conn(0x7fac680ce130 msgr2=0x7fac680ce510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:07.970 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.966+0000 7fac6ce02640 1 -- 172.21.15.169:0/3074503011 shutdown_connections 2023-10-26T01:39:07.970 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.966+0000 7fac6ce02640 1 -- 172.21.15.169:0/3074503011 wait complete. 2023-10-26T01:39:07.970 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.968+0000 7fac6ce02640 1 Processor -- start 2023-10-26T01:39:07.971 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.969+0000 7fac6ce02640 1 -- start start 2023-10-26T01:39:07.972 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.970+0000 7fac6ce02640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fac680b0710 0x7fac680cab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:07.972 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.970+0000 7fac6ce02640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fac680b3130 0x7fac681f2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:07.973 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.970+0000 7fac6ce02640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fac6815ba30 0x7fac681f25c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:07.973 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.970+0000 7fac6ce02640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac680c8bf0 con 0x7fac680b3130 2023-10-26T01:39:07.973 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.970+0000 7fac6ce02640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac680c8ed0 con 0x7fac6815ba30 2023-10-26T01:39:07.973 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.970+0000 7fac6ce02640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac680c91b0 con 0x7fac680b0710 2023-10-26T01:39:07.973 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.970+0000 7fac5ffff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fac680b3130 0x7fac681f2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:07.974 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.970+0000 7fac5ffff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fac680b3130 0x7fac681f2080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:39104/0 (socket says 172.21.15.169:39104) 2023-10-26T01:39:07.974 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.970+0000 7fac5ffff640 1 -- 172.21.15.169:0/4268262064 learned_addr learned my addr 172.21.15.169:0/4268262064 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:39:07.974 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.970+0000 7fac6659c640 1 --2- 172.21.15.169:0/4268262064 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fac680b0710 0x7fac680cab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:07.974 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.971+0000 7fac66d9d640 1 --2- 172.21.15.169:0/4268262064 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fac6815ba30 0x7fac681f25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:07.974 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.971+0000 7fac5ffff640 1 -- 172.21.15.169:0/4268262064 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fac680b0710 msgr2=0x7fac680cab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:07.975 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.971+0000 7fac5ffff640 1 --2- 172.21.15.169:0/4268262064 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fac680b0710 0x7fac680cab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:07.975 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.971+0000 7fac5ffff640 1 -- 172.21.15.169:0/4268262064 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fac6815ba30 msgr2=0x7fac681f25c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:07.975 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.971+0000 7fac5ffff640 1 --2- 172.21.15.169:0/4268262064 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fac6815ba30 0x7fac681f25c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:07.975 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.971+0000 7fac5ffff640 1 -- 172.21.15.169:0/4268262064 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac58009560 con 0x7fac680b3130 2023-10-26T01:39:07.976 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.971+0000 7fac5ffff640 1 --2- 172.21.15.169:0/4268262064 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fac680b3130 0x7fac681f2080 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7fac5000d750 tx=0x7fac5000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:07.976 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.971+0000 7fac5effd640 1 -- 172.21.15.169:0/4268262064 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac50009bd0 con 0x7fac680b3130 2023-10-26T01:39:07.977 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.971+0000 7fac5effd640 1 -- 172.21.15.169:0/4268262064 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fac50009e20 con 0x7fac680b3130 2023-10-26T01:39:07.977 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.971+0000 7fac5effd640 1 -- 172.21.15.169:0/4268262064 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac50005280 con 0x7fac680b3130 2023-10-26T01:39:07.977 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.971+0000 7fac6ce02640 1 -- 172.21.15.169:0/4268262064 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac680c9490 con 0x7fac680b3130 2023-10-26T01:39:07.977 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.971+0000 7fac6ce02640 1 -- 172.21.15.169:0/4268262064 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac680c8780 con 0x7fac680b3130 2023-10-26T01:39:07.977 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.973+0000 7fac6ce02640 1 -- 172.21.15.169:0/4268262064 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac680cea00 con 0x7fac680b3130 2023-10-26T01:39:07.978 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.974+0000 7fac5effd640 1 -- 172.21.15.169:0/4268262064 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fac500053e0 con 0x7fac680b3130 2023-10-26T01:39:07.978 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.975+0000 7fac5effd640 1 --2- 172.21.15.169:0/4268262064 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fac4806ae40 0x7fac4806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:07.979 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.975+0000 7fac5effd640 1 -- 172.21.15.169:0/4268262064 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fac500897e0 con 0x7fac680b3130 2023-10-26T01:39:07.979 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.976+0000 7fac6659c640 1 --2- 172.21.15.169:0/4268262064 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fac4806ae40 0x7fac4806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:07.979 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.977+0000 7fac6659c640 1 --2- 172.21.15.169:0/4268262064 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fac4806ae40 0x7fac4806d300 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fac58035000 tx=0x7fac58004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:07.981 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:07.980+0000 7fac5effd640 1 -- 172.21.15.169:0/4268262064 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fac500548c0 con 0x7fac680b3130 2023-10-26T01:39:08.178 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.177+0000 7fac6ce02640 1 -- 172.21.15.169:0/4268262064 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fac680c8940 con 0x7fac680b3130 2023-10-26T01:39:08.182 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.180+0000 7fac5effd640 1 -- 172.21.15.169:0/4268262064 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v140) v1 ==== 93+0+10 (secure 0 0 0) 0x7fac500170a0 con 0x7fac680b3130 2023-10-26T01:39:08.182 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 1 2023-10-26T01:39:08.185 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.183+0000 7fac6ce02640 1 -- 172.21.15.169:0/4268262064 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fac4806ae40 msgr2=0x7fac4806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:08.186 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.183+0000 7fac6ce02640 1 --2- 172.21.15.169:0/4268262064 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fac4806ae40 0x7fac4806d300 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fac58035000 tx=0x7fac58004730 comp rx=0 tx=0).stop 2023-10-26T01:39:08.186 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.183+0000 7fac6ce02640 1 -- 172.21.15.169:0/4268262064 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fac680b3130 msgr2=0x7fac681f2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:08.186 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.183+0000 7fac6ce02640 1 --2- 172.21.15.169:0/4268262064 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fac680b3130 0x7fac681f2080 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7fac5000d750 tx=0x7fac5000dc10 comp rx=0 tx=0).stop 2023-10-26T01:39:08.186 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.183+0000 7fac6ce02640 1 -- 172.21.15.169:0/4268262064 shutdown_connections 2023-10-26T01:39:08.187 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.183+0000 7fac6ce02640 1 --2- 172.21.15.169:0/4268262064 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fac4806ae40 0x7fac4806d300 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:08.187 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.183+0000 7fac6ce02640 1 --2- 172.21.15.169:0/4268262064 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fac680b3130 0x7fac681f2080 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:08.187 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.183+0000 7fac6ce02640 1 --2- 172.21.15.169:0/4268262064 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fac6815ba30 0x7fac681f25c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:08.187 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.183+0000 7fac6ce02640 1 --2- 172.21.15.169:0/4268262064 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fac680b0710 0x7fac680cab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:08.187 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.183+0000 7fac6ce02640 1 -- 172.21.15.169:0/4268262064 >> 172.21.15.169:0/4268262064 conn(0x7fac680ce130 msgr2=0x7fac68159b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:08.188 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.184+0000 7fac6ce02640 1 -- 172.21.15.169:0/4268262064 shutdown_connections 2023-10-26T01:39:08.188 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.184+0000 7fac6ce02640 1 -- 172.21.15.169:0/4268262064 wait complete. 2023-10-26T01:39:08.203 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-26T01:39:08.423 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.421+0000 7f97dec64640 1 -- 172.21.15.169:0/3604977842 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97d80d0fc0 msgr2=0x7f97d80d1b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:08.423 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.421+0000 7f97dec64640 1 --2- 172.21.15.169:0/3604977842 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97d80d0fc0 0x7f97d80d1b80 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f97d80d1ac0 tx=0x7f97cc02bce0 comp rx=0 tx=0).stop 2023-10-26T01:39:08.424 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.421+0000 7f97dec64640 1 -- 172.21.15.169:0/3604977842 shutdown_connections 2023-10-26T01:39:08.424 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.421+0000 7f97dec64640 1 --2- 172.21.15.169:0/3604977842 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97d80d0fc0 0x7f97d80d1b80 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:08.424 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.421+0000 7f97dec64640 1 --2- 172.21.15.169:0/3604977842 >> v2:172.21.15.170:3300/0 conn(0x7f97d815ba00 0x7f97d80b6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:08.424 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.421+0000 7f97dec64640 1 -- 172.21.15.169:0/3604977842 >> 172.21.15.169:0/3604977842 conn(0x7f97d806b5c0 msgr2=0x7f97d806b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:08.424 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.421+0000 7f97dec64640 1 -- 172.21.15.169:0/3604977842 shutdown_connections 2023-10-26T01:39:08.425 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.421+0000 7f97dec64640 1 -- 172.21.15.169:0/3604977842 wait complete. 2023-10-26T01:39:08.426 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.423+0000 7f97dec64640 1 Processor -- start 2023-10-26T01:39:08.427 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.423+0000 7f97dec64640 1 -- start start 2023-10-26T01:39:08.427 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.424+0000 7f97dec64640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f97d80d0fc0 0x7f97d8147da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:08.427 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.424+0000 7f97dec64640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f97d80d20c0 0x7f97d815be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:08.427 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.424+0000 7f97dec64640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97d815ba00 0x7f97d815d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:08.427 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.424+0000 7f97dec64640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97d815ed10 con 0x7f97d815ba00 2023-10-26T01:39:08.428 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.424+0000 7f97dec64640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97d80c63b0 con 0x7f97d80d20c0 2023-10-26T01:39:08.428 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.424+0000 7f97dec64640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97d815ea30 con 0x7f97d80d0fc0 2023-10-26T01:39:08.428 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.424+0000 7f97dd201640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97d815ba00 0x7f97d815d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:08.428 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.424+0000 7f97dd201640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97d815ba00 0x7f97d815d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:39118/0 (socket says 172.21.15.169:39118) 2023-10-26T01:39:08.428 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.424+0000 7f97dd201640 1 -- 172.21.15.169:0/2770571011 learned_addr learned my addr 172.21.15.169:0/2770571011 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:39:08.429 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.425+0000 7f97d7fff640 1 --2- 172.21.15.169:0/2770571011 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f97d80d20c0 0x7f97d815be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:08.429 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.425+0000 7f97dd201640 1 -- 172.21.15.169:0/2770571011 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f97d80d0fc0 msgr2=0x7f97d8147da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:08.429 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.425+0000 7f97dca00640 1 --2- 172.21.15.169:0/2770571011 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f97d80d0fc0 0x7f97d8147da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:08.430 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.425+0000 7f97dd201640 1 --2- 172.21.15.169:0/2770571011 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f97d80d0fc0 0x7f97d8147da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:08.430 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.425+0000 7f97dd201640 1 -- 172.21.15.169:0/2770571011 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f97d80d20c0 msgr2=0x7f97d815be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:08.430 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.425+0000 7f97dd201640 1 --2- 172.21.15.169:0/2770571011 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f97d80d20c0 0x7f97d815be30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:08.430 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.425+0000 7f97dd201640 1 -- 172.21.15.169:0/2770571011 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97cc009490 con 0x7f97d815ba00 2023-10-26T01:39:08.430 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.425+0000 7f97dd201640 1 --2- 172.21.15.169:0/2770571011 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97d815ba00 0x7f97d815d990 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f97d000c760 tx=0x7f97d000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:08.432 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.425+0000 7f97d57fa640 1 -- 172.21.15.169:0/2770571011 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97d0003da0 con 0x7f97d815ba00 2023-10-26T01:39:08.432 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.425+0000 7f97d57fa640 1 -- 172.21.15.169:0/2770571011 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f97d000ce70 con 0x7f97d815ba00 2023-10-26T01:39:08.432 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.425+0000 7f97dec64640 1 -- 172.21.15.169:0/2770571011 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97d80c5d60 con 0x7f97d815ba00 2023-10-26T01:39:08.432 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.425+0000 7f97d57fa640 1 -- 172.21.15.169:0/2770571011 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97d00052b0 con 0x7f97d815ba00 2023-10-26T01:39:08.433 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.425+0000 7f97dec64640 1 -- 172.21.15.169:0/2770571011 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97d815c370 con 0x7f97d815ba00 2023-10-26T01:39:08.433 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.427+0000 7f97dec64640 1 -- 172.21.15.169:0/2770571011 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97d80c5bc0 con 0x7f97d815ba00 2023-10-26T01:39:08.433 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.427+0000 7f97d57fa640 1 -- 172.21.15.169:0/2770571011 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f97d0005410 con 0x7f97d815ba00 2023-10-26T01:39:08.433 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.428+0000 7f97d57fa640 1 --2- 172.21.15.169:0/2770571011 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f97c406ae40 0x7f97c406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:08.433 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.429+0000 7f97d57fa640 1 -- 172.21.15.169:0/2770571011 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f97d0089650 con 0x7f97d815ba00 2023-10-26T01:39:08.434 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.429+0000 7f97d57fa640 1 -- 172.21.15.169:0/2770571011 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=141}) v3 -- 0x7f97c4072a30 con 0x7f97d815ba00 2023-10-26T01:39:08.434 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.429+0000 7f97dca00640 1 --2- 172.21.15.169:0/2770571011 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f97c406ae40 0x7f97c406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:08.434 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.430+0000 7f97dca00640 1 --2- 172.21.15.169:0/2770571011 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f97c406ae40 0x7f97c406d300 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f97c80058c0 tx=0x7f97c8005850 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:08.435 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.433+0000 7f97d57fa640 1 -- 172.21.15.169:0/2770571011 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f97d80c5bc0 con 0x7f97d815ba00 2023-10-26T01:39:08.623 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.621+0000 7f97dec64640 1 -- 172.21.15.169:0/2770571011 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f97d80c5bc0 con 0x7f97d815ba00 2023-10-26T01:39:08.623 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.622+0000 7f97d57fa640 1 -- 172.21.15.169:0/2770571011 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v140) v1 ==== 104+0+11 (secure 0 0 0) 0x7f97d80c5bc0 con 0x7f97d815ba00 2023-10-26T01:39:08.624 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 32 2023-10-26T01:39:08.626 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.624+0000 7f97dec64640 1 -- 172.21.15.169:0/2770571011 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f97c406ae40 msgr2=0x7f97c406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:08.627 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.624+0000 7f97dec64640 1 --2- 172.21.15.169:0/2770571011 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f97c406ae40 0x7f97c406d300 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f97c80058c0 tx=0x7f97c8005850 comp rx=0 tx=0).stop 2023-10-26T01:39:08.627 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.624+0000 7f97dec64640 1 -- 172.21.15.169:0/2770571011 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97d815ba00 msgr2=0x7f97d815d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:08.627 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.624+0000 7f97dec64640 1 --2- 172.21.15.169:0/2770571011 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97d815ba00 0x7f97d815d990 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f97d000c760 tx=0x7f97d000cc20 comp rx=0 tx=0).stop 2023-10-26T01:39:08.627 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.624+0000 7f97dec64640 1 -- 172.21.15.169:0/2770571011 shutdown_connections 2023-10-26T01:39:08.627 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.624+0000 7f97dec64640 1 --2- 172.21.15.169:0/2770571011 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f97c406ae40 0x7f97c406d300 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:08.628 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.624+0000 7f97dec64640 1 --2- 172.21.15.169:0/2770571011 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97d815ba00 0x7f97d815d990 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:08.628 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.624+0000 7f97dec64640 1 --2- 172.21.15.169:0/2770571011 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f97d80d20c0 0x7f97d815be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:08.628 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.624+0000 7f97dec64640 1 --2- 172.21.15.169:0/2770571011 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f97d80d0fc0 0x7f97d8147da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:08.628 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.624+0000 7f97dec64640 1 -- 172.21.15.169:0/2770571011 >> 172.21.15.169:0/2770571011 conn(0x7f97d806b5c0 msgr2=0x7f97d80b58c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:08.628 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.625+0000 7f97dec64640 1 -- 172.21.15.169:0/2770571011 shutdown_connections 2023-10-26T01:39:08.629 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.625+0000 7f97dec64640 1 -- 172.21.15.169:0/2770571011 wait complete. 2023-10-26T01:39:08.645 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-26T01:39:08.851 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.848+0000 7f6bba456640 1 -- 172.21.15.169:0/4172434906 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6bb415b9e0 msgr2=0x7f6bb406ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:08.851 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.848+0000 7f6bba456640 1 --2- 172.21.15.169:0/4172434906 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6bb415b9e0 0x7f6bb406ad90 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f6bb40b0570 tx=0x7f6bac02c620 comp rx=0 tx=0).stop 2023-10-26T01:39:08.851 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.849+0000 7f6bba456640 1 -- 172.21.15.169:0/4172434906 shutdown_connections 2023-10-26T01:39:08.851 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.849+0000 7f6bba456640 1 --2- 172.21.15.169:0/4172434906 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6bb415b9e0 0x7f6bb406ad90 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:08.852 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.849+0000 7f6bba456640 1 --2- 172.21.15.169:0/4172434906 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6bb40cdd20 0x7f6bb415b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:08.852 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.849+0000 7f6bba456640 1 --2- 172.21.15.169:0/4172434906 >> v2:172.21.15.170:3300/0 conn(0x7f6bb4155360 0x7f6bb40cd7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:08.852 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.849+0000 7f6bba456640 1 -- 172.21.15.169:0/4172434906 >> 172.21.15.169:0/4172434906 conn(0x7f6bb4069930 msgr2=0x7f6bb4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:08.852 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.849+0000 7f6bba456640 1 -- 172.21.15.169:0/4172434906 shutdown_connections 2023-10-26T01:39:08.852 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.849+0000 7f6bba456640 1 -- 172.21.15.169:0/4172434906 wait complete. 2023-10-26T01:39:08.854 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.850+0000 7f6bba456640 1 Processor -- start 2023-10-26T01:39:08.854 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.851+0000 7f6bba456640 1 -- start start 2023-10-26T01:39:08.854 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.851+0000 7f6bba456640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6bb40cdd20 0x7f6bb4150d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:08.855 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.851+0000 7f6bba456640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6bb4155360 0x7f6bb414d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:08.855 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.851+0000 7f6bba456640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6bb415b9e0 0x7f6bb40c9a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:08.855 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.851+0000 7f6bba456640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb4151290 con 0x7f6bb40cdd20 2023-10-26T01:39:08.855 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.851+0000 7f6bba456640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb414fcb0 con 0x7f6bb4155360 2023-10-26T01:39:08.855 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.851+0000 7f6bba456640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb414ff90 con 0x7f6bb415b9e0 2023-10-26T01:39:08.856 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.852+0000 7f6bb3fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6bb40cdd20 0x7f6bb4150d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:08.856 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.852+0000 7f6bb3fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6bb40cdd20 0x7f6bb4150d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:39136/0 (socket says 172.21.15.169:39136) 2023-10-26T01:39:08.856 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.852+0000 7f6bb3fff640 1 -- 172.21.15.169:0/272786799 learned_addr learned my addr 172.21.15.169:0/272786799 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:39:08.856 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.852+0000 7f6bb37fe640 1 --2- 172.21.15.169:0/272786799 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6bb4155360 0x7f6bb414d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:08.856 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.852+0000 7f6bb89f3640 1 --2- 172.21.15.169:0/272786799 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6bb415b9e0 0x7f6bb40c9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:08.857 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.852+0000 7f6bb3fff640 1 -- 172.21.15.169:0/272786799 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6bb415b9e0 msgr2=0x7f6bb40c9a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:08.857 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.852+0000 7f6bb3fff640 1 --2- 172.21.15.169:0/272786799 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6bb415b9e0 0x7f6bb40c9a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:08.857 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.852+0000 7f6bb3fff640 1 -- 172.21.15.169:0/272786799 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6bb4155360 msgr2=0x7f6bb414d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:08.857 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.852+0000 7f6bb3fff640 1 --2- 172.21.15.169:0/272786799 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6bb4155360 0x7f6bb414d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:08.857 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.852+0000 7f6bb3fff640 1 -- 172.21.15.169:0/272786799 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bac009df0 con 0x7f6bb40cdd20 2023-10-26T01:39:08.858 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.853+0000 7f6bb3fff640 1 --2- 172.21.15.169:0/272786799 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6bb40cdd20 0x7f6bb4150d50 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f6ba400c920 tx=0x7f6ba400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:08.859 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.853+0000 7f6bb0ff9640 1 -- 172.21.15.169:0/272786799 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba4007c70 con 0x7f6bb40cdd20 2023-10-26T01:39:08.859 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.853+0000 7f6bb0ff9640 1 -- 172.21.15.169:0/272786799 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6ba400f450 con 0x7f6bb40cdd20 2023-10-26T01:39:08.859 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.853+0000 7f6bb0ff9640 1 -- 172.21.15.169:0/272786799 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba400f860 con 0x7f6bb40cdd20 2023-10-26T01:39:08.859 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.853+0000 7f6bba456640 1 -- 172.21.15.169:0/272786799 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bb4150270 con 0x7f6bb40cdd20 2023-10-26T01:39:08.860 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.853+0000 7f6bba456640 1 -- 172.21.15.169:0/272786799 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bb4150830 con 0x7f6bb40cdd20 2023-10-26T01:39:08.860 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.855+0000 7f6bba456640 1 -- 172.21.15.169:0/272786799 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bb404db10 con 0x7f6bb40cdd20 2023-10-26T01:39:08.860 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.855+0000 7f6bb0ff9640 1 -- 172.21.15.169:0/272786799 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f6ba400f9c0 con 0x7f6bb40cdd20 2023-10-26T01:39:08.860 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.856+0000 7f6bb0ff9640 1 --2- 172.21.15.169:0/272786799 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f6ba006ae40 0x7f6ba006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:08.860 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.856+0000 7f6bb0ff9640 1 -- 172.21.15.169:0/272786799 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f6ba4089ef0 con 0x7f6bb40cdd20 2023-10-26T01:39:08.861 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.856+0000 7f6bb0ff9640 1 -- 172.21.15.169:0/272786799 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=141}) v3 -- 0x7f6ba0072a30 con 0x7f6bb40cdd20 2023-10-26T01:39:08.861 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.857+0000 7f6bb37fe640 1 --2- 172.21.15.169:0/272786799 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f6ba006ae40 0x7f6ba006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:08.861 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.858+0000 7f6bb37fe640 1 --2- 172.21.15.169:0/272786799 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f6ba006ae40 0x7f6ba006d300 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f6bb4156e10 tx=0x7f6ba800a390 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:08.862 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:08.861+0000 7f6bb0ff9640 1 -- 172.21.15.169:0/272786799 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6ba401d060 con 0x7f6bb40cdd20 2023-10-26T01:39:09.051 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.049+0000 7f6bba456640 1 -- 172.21.15.169:0/272786799 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6bb4152060 con 0x7f6bb40cdd20 2023-10-26T01:39:09.051 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.050+0000 7f6bb0ff9640 1 -- 172.21.15.169:0/272786799 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v140) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6ba4054f60 con 0x7f6bb40cdd20 2023-10-26T01:39:09.052 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 64 2023-10-26T01:39:09.055 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.053+0000 7f6bba456640 1 -- 172.21.15.169:0/272786799 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f6ba006ae40 msgr2=0x7f6ba006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:09.055 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.053+0000 7f6bba456640 1 --2- 172.21.15.169:0/272786799 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f6ba006ae40 0x7f6ba006d300 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f6bb4156e10 tx=0x7f6ba800a390 comp rx=0 tx=0).stop 2023-10-26T01:39:09.055 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.053+0000 7f6bba456640 1 -- 172.21.15.169:0/272786799 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6bb40cdd20 msgr2=0x7f6bb4150d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:09.056 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.053+0000 7f6bba456640 1 --2- 172.21.15.169:0/272786799 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6bb40cdd20 0x7f6bb4150d50 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f6ba400c920 tx=0x7f6ba400cde0 comp rx=0 tx=0).stop 2023-10-26T01:39:09.056 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.053+0000 7f6bba456640 1 -- 172.21.15.169:0/272786799 shutdown_connections 2023-10-26T01:39:09.057 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.053+0000 7f6bba456640 1 --2- 172.21.15.169:0/272786799 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f6ba006ae40 0x7f6ba006d300 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:09.057 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.053+0000 7f6bba456640 1 --2- 172.21.15.169:0/272786799 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6bb415b9e0 0x7f6bb40c9a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:09.057 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.053+0000 7f6bba456640 1 --2- 172.21.15.169:0/272786799 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6bb4155360 0x7f6bb414d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:09.058 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.053+0000 7f6bba456640 1 --2- 172.21.15.169:0/272786799 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6bb40cdd20 0x7f6bb4150d50 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:09.058 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.053+0000 7f6bba456640 1 -- 172.21.15.169:0/272786799 >> 172.21.15.169:0/272786799 conn(0x7f6bb4069930 msgr2=0x7f6bb4156040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:09.058 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.053+0000 7f6bba456640 1 -- 172.21.15.169:0/272786799 shutdown_connections 2023-10-26T01:39:09.058 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.053+0000 7f6bba456640 1 -- 172.21.15.169:0/272786799 wait complete. 2023-10-26T01:39:09.073 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T01:39:09.290 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.288+0000 7f7ec09a9640 1 -- 172.21.15.169:0/3360082497 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ebc0cde00 msgr2=0x7f7ebc164040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:09.290 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.288+0000 7f7ec09a9640 1 --2- 172.21.15.169:0/3360082497 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ebc0cde00 0x7f7ebc164040 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f7eb4005fd0 tx=0x7f7eb402c060 comp rx=0 tx=0).stop 2023-10-26T01:39:09.291 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.288+0000 7f7ec09a9640 1 -- 172.21.15.169:0/3360082497 shutdown_connections 2023-10-26T01:39:09.291 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.288+0000 7f7ec09a9640 1 --2- 172.21.15.169:0/3360082497 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ebc0cde00 0x7f7ebc164040 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:09.291 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.288+0000 7f7ec09a9640 1 -- 172.21.15.169:0/3360082497 >> 172.21.15.169:0/3360082497 conn(0x7f7ebc069930 msgr2=0x7f7ebc069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:09.291 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.288+0000 7f7ec09a9640 1 -- 172.21.15.169:0/3360082497 shutdown_connections 2023-10-26T01:39:09.291 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.288+0000 7f7ec09a9640 1 -- 172.21.15.169:0/3360082497 wait complete. 2023-10-26T01:39:09.292 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.290+0000 7f7ec09a9640 1 Processor -- start 2023-10-26T01:39:09.293 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.290+0000 7f7ec09a9640 1 -- start start 2023-10-26T01:39:09.293 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.291+0000 7f7ec09a9640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ebc0ac930 0x7f7ebc1478b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:09.293 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.291+0000 7f7ec09a9640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7ebc0cde00 0x7f7ebc0b11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:09.293 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.291+0000 7f7ec09a9640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ebc1592d0 0x7f7ebc0b16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:09.293 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.291+0000 7f7ec09a9640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ebc0c6fa0 con 0x7f7ebc0ac930 2023-10-26T01:39:09.294 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.291+0000 7f7ec09a9640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ebc14b780 con 0x7f7ebc1592d0 2023-10-26T01:39:09.294 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.291+0000 7f7ec09a9640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ebc0c6600 con 0x7f7ebc0cde00 2023-10-26T01:39:09.296 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.291+0000 7f7eba59c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ebc0ac930 0x7f7ebc1478b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:09.296 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.291+0000 7f7eba59c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ebc0ac930 0x7f7ebc1478b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:39156/0 (socket says 172.21.15.169:39156) 2023-10-26T01:39:09.296 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.291+0000 7f7eba59c640 1 -- 172.21.15.169:0/887977907 learned_addr learned my addr 172.21.15.169:0/887977907 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:39:09.297 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.292+0000 7f7ebad9d640 1 --2- 172.21.15.169:0/887977907 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ebc1592d0 0x7f7ebc0b16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:09.297 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.292+0000 7f7eb3fff640 1 --2- 172.21.15.169:0/887977907 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7ebc0cde00 0x7f7ebc0b11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:09.297 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.292+0000 7f7eba59c640 1 -- 172.21.15.169:0/887977907 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7ebc0cde00 msgr2=0x7f7ebc0b11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:09.298 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.292+0000 7f7eba59c640 1 --2- 172.21.15.169:0/887977907 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7ebc0cde00 0x7f7ebc0b11a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:09.298 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.292+0000 7f7eba59c640 1 -- 172.21.15.169:0/887977907 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ebc1592d0 msgr2=0x7f7ebc0b16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:09.298 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.292+0000 7f7eba59c640 1 --2- 172.21.15.169:0/887977907 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ebc1592d0 0x7f7ebc0b16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:09.298 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.292+0000 7f7eba59c640 1 -- 172.21.15.169:0/887977907 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7eb4009840 con 0x7f7ebc0ac930 2023-10-26T01:39:09.298 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.292+0000 7f7ebad9d640 1 --2- 172.21.15.169:0/887977907 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ebc1592d0 0x7f7ebc0b16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:39:09.299 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.292+0000 7f7eba59c640 1 --2- 172.21.15.169:0/887977907 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ebc0ac930 0x7f7ebc1478b0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f7eac00b3f0 tx=0x7f7eac00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:09.299 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.292+0000 7f7eb2ffd640 1 -- 172.21.15.169:0/887977907 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eac011830 con 0x7f7ebc0ac930 2023-10-26T01:39:09.299 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.292+0000 7f7eb2ffd640 1 -- 172.21.15.169:0/887977907 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7eac011a80 con 0x7f7ebc0ac930 2023-10-26T01:39:09.299 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.292+0000 7f7eb2ffd640 1 -- 172.21.15.169:0/887977907 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eac011e90 con 0x7f7ebc0ac930 2023-10-26T01:39:09.299 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.292+0000 7f7ec09a9640 1 -- 172.21.15.169:0/887977907 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ebc0c68e0 con 0x7f7ebc0ac930 2023-10-26T01:39:09.300 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.293+0000 7f7ec09a9640 1 -- 172.21.15.169:0/887977907 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ebc147df0 con 0x7f7ebc0ac930 2023-10-26T01:39:09.300 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.294+0000 7f7ec09a9640 1 -- 172.21.15.169:0/887977907 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ebc0ab750 con 0x7f7ebc0ac930 2023-10-26T01:39:09.300 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.295+0000 7f7eb2ffd640 1 -- 172.21.15.169:0/887977907 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f7eac01a400 con 0x7f7ebc0ac930 2023-10-26T01:39:09.301 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.295+0000 7f7eb2ffd640 1 --2- 172.21.15.169:0/887977907 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7e9c06ae70 0x7f7e9c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:09.301 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.296+0000 7f7eb2ffd640 1 -- 172.21.15.169:0/887977907 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f7eac089760 con 0x7f7ebc0ac930 2023-10-26T01:39:09.301 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.296+0000 7f7eb2ffd640 1 -- 172.21.15.169:0/887977907 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=141}) v3 -- 0x7f7e9c072a60 con 0x7f7ebc0ac930 2023-10-26T01:39:09.301 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.296+0000 7f7eb3fff640 1 --2- 172.21.15.169:0/887977907 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7e9c06ae70 0x7f7e9c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:09.302 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.297+0000 7f7eb3fff640 1 --2- 172.21.15.169:0/887977907 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7e9c06ae70 0x7f7e9c06d330 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc0c9e40 tx=0x7f7ea4009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:09.302 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.300+0000 7f7eb2ffd640 1 -- 172.21.15.169:0/887977907 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7eac054730 con 0x7f7ebc0ac930 2023-10-26T01:39:09.530 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.529+0000 7f7ec09a9640 1 -- 172.21.15.169:0/887977907 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7ebc04db10 con 0x7f7ebc0ac930 2023-10-26T01:39:09.532 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.530+0000 7f7eb2ffd640 1 -- 172.21.15.169:0/887977907 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+4927 (secure 0 0 0) 0x7f7eac015020 con 0x7f7ebc0ac930 2023-10-26T01:39:09.532 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:39:09.533 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":68,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5917,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/3297412140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":3297412140},{"type":"v1","addr":"172.21.15.169:6835","nonce":3297412140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":5929,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/2575809940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2575809940},{"type":"v1","addr":"172.21.15.170:6836","nonce":2575809940}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":7761,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2632217991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2632217991},{"type":"v1","addr":"172.21.15.169:6833","nonce":2632217991}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T01:38:20.773556+0000","modified":"2023-10-26T01:38:23.839758+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7773},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7773":{"gid":7773,"name":"b","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6837/3615088922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":3615088922},{"type":"v1","addr":"172.21.15.170:6837","nonce":3615088922}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-10-26T01:39:09.533 INFO:teuthology.orchestra.run.smithi169.stderr:dumped fsmap epoch 68 2023-10-26T01:39:09.535 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.533+0000 7f7ec09a9640 1 -- 172.21.15.169:0/887977907 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7e9c06ae70 msgr2=0x7f7e9c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:09.535 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.533+0000 7f7ec09a9640 1 --2- 172.21.15.169:0/887977907 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7e9c06ae70 0x7f7e9c06d330 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc0c9e40 tx=0x7f7ea4009040 comp rx=0 tx=0).stop 2023-10-26T01:39:09.535 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.533+0000 7f7ec09a9640 1 -- 172.21.15.169:0/887977907 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ebc0ac930 msgr2=0x7f7ebc1478b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:09.535 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.533+0000 7f7ec09a9640 1 --2- 172.21.15.169:0/887977907 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ebc0ac930 0x7f7ebc1478b0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f7eac00b3f0 tx=0x7f7eac00b8b0 comp rx=0 tx=0).stop 2023-10-26T01:39:09.536 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.533+0000 7f7ec09a9640 1 -- 172.21.15.169:0/887977907 shutdown_connections 2023-10-26T01:39:09.536 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.533+0000 7f7ec09a9640 1 --2- 172.21.15.169:0/887977907 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7e9c06ae70 0x7f7e9c06d330 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:09.536 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.533+0000 7f7ec09a9640 1 --2- 172.21.15.169:0/887977907 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ebc1592d0 0x7f7ebc0b16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:09.536 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.533+0000 7f7ec09a9640 1 --2- 172.21.15.169:0/887977907 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7ebc0cde00 0x7f7ebc0b11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:09.536 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.533+0000 7f7ec09a9640 1 --2- 172.21.15.169:0/887977907 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ebc0ac930 0x7f7ebc1478b0 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:09.537 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.533+0000 7f7ec09a9640 1 -- 172.21.15.169:0/887977907 >> 172.21.15.169:0/887977907 conn(0x7f7ebc069930 msgr2=0x7f7ebc0ad1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:09.537 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.533+0000 7f7ec09a9640 1 -- 172.21.15.169:0/887977907 shutdown_connections 2023-10-26T01:39:09.537 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.533+0000 7f7ec09a9640 1 -- 172.21.15.169:0/887977907 wait complete. 2023-10-26T01:39:09.554 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:39:09.779 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.777+0000 7ff346993640 1 -- 172.21.15.169:0/3703453113 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff3400ac6d0 msgr2=0x7ff34015d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:09.780 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.777+0000 7ff346993640 1 --2- 172.21.15.169:0/3703453113 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff3400ac6d0 0x7ff34015d840 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7ff33c005fd0 tx=0x7ff33c02bd90 comp rx=0 tx=0).stop 2023-10-26T01:39:09.780 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.777+0000 7ff346993640 1 -- 172.21.15.169:0/3703453113 shutdown_connections 2023-10-26T01:39:09.780 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.777+0000 7ff346993640 1 --2- 172.21.15.169:0/3703453113 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff34009a5c0 0x7ff34009aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:09.780 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.777+0000 7ff346993640 1 --2- 172.21.15.169:0/3703453113 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff3400ac6d0 0x7ff34015d840 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:09.780 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.777+0000 7ff346993640 1 -- 172.21.15.169:0/3703453113 >> 172.21.15.169:0/3703453113 conn(0x7ff3400aedd0 msgr2=0x7ff3400af1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:09.781 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.777+0000 7ff346993640 1 -- 172.21.15.169:0/3703453113 shutdown_connections 2023-10-26T01:39:09.781 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.777+0000 7ff346993640 1 -- 172.21.15.169:0/3703453113 wait complete. 2023-10-26T01:39:09.782 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.779+0000 7ff346993640 1 Processor -- start 2023-10-26T01:39:09.782 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.779+0000 7ff346993640 1 -- start start 2023-10-26T01:39:09.782 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.780+0000 7ff346993640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff34009a5c0 0x7ff34006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:09.782 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.780+0000 7ff346993640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff3400ac6d0 0x7ff3400c6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:09.783 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.780+0000 7ff346993640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff3400c9970 0x7ff3400c67b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:09.783 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.780+0000 7ff346993640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3400719a0 con 0x7ff3400ac6d0 2023-10-26T01:39:09.783 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.780+0000 7ff346993640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff340133b30 con 0x7ff3400c9970 2023-10-26T01:39:09.783 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.780+0000 7ff346993640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff340133fc0 con 0x7ff34009a5c0 2023-10-26T01:39:09.785 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.780+0000 7ff345190640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff3400ac6d0 0x7ff3400c6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:09.785 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.781+0000 7ff345190640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff3400ac6d0 0x7ff3400c6270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:39172/0 (socket says 172.21.15.169:39172) 2023-10-26T01:39:09.785 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.781+0000 7ff345190640 1 -- 172.21.15.169:0/2943996618 learned_addr learned my addr 172.21.15.169:0/2943996618 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:39:09.785 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.781+0000 7ff345991640 1 --2- 172.21.15.169:0/2943996618 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff34009a5c0 0x7ff34006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:09.786 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.781+0000 7ff346192640 1 --2- 172.21.15.169:0/2943996618 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff3400c9970 0x7ff3400c67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:09.786 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.781+0000 7ff345190640 1 -- 172.21.15.169:0/2943996618 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff34009a5c0 msgr2=0x7ff34006f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:09.786 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.781+0000 7ff345190640 1 --2- 172.21.15.169:0/2943996618 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff34009a5c0 0x7ff34006f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:09.786 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.781+0000 7ff345190640 1 -- 172.21.15.169:0/2943996618 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff3400c9970 msgr2=0x7ff3400c67b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:09.787 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.781+0000 7ff345190640 1 --2- 172.21.15.169:0/2943996618 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff3400c9970 0x7ff3400c67b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:09.787 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.781+0000 7ff345190640 1 -- 172.21.15.169:0/2943996618 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff33c009560 con 0x7ff3400ac6d0 2023-10-26T01:39:09.787 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.781+0000 7ff345190640 1 --2- 172.21.15.169:0/2943996618 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff3400ac6d0 0x7ff3400c6270 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7ff33000b3f0 tx=0x7ff33000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:09.787 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.782+0000 7ff3367fc640 1 -- 172.21.15.169:0/2943996618 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff330011830 con 0x7ff3400ac6d0 2023-10-26T01:39:09.787 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.782+0000 7ff3367fc640 1 -- 172.21.15.169:0/2943996618 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff330011a80 con 0x7ff3400ac6d0 2023-10-26T01:39:09.788 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.782+0000 7ff3367fc640 1 -- 172.21.15.169:0/2943996618 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff330011e90 con 0x7ff3400ac6d0 2023-10-26T01:39:09.788 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.782+0000 7ff346993640 1 -- 172.21.15.169:0/2943996618 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff340133200 con 0x7ff3400ac6d0 2023-10-26T01:39:09.788 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.782+0000 7ff346993640 1 -- 172.21.15.169:0/2943996618 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3401337c0 con 0x7ff3400ac6d0 2023-10-26T01:39:09.788 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.783+0000 7ff346993640 1 -- 172.21.15.169:0/2943996618 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff340073c10 con 0x7ff3400ac6d0 2023-10-26T01:39:09.789 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.784+0000 7ff3367fc640 1 -- 172.21.15.169:0/2943996618 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ff33001a400 con 0x7ff3400ac6d0 2023-10-26T01:39:09.789 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.785+0000 7ff3367fc640 1 --2- 172.21.15.169:0/2943996618 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff32406ae40 0x7ff32406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:09.790 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.785+0000 7ff3367fc640 1 -- 172.21.15.169:0/2943996618 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5767+0+0 (secure 0 0 0) 0x7ff330089720 con 0x7ff3400ac6d0 2023-10-26T01:39:09.790 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.785+0000 7ff3367fc640 1 -- 172.21.15.169:0/2943996618 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=141}) v3 -- 0x7ff324072a30 con 0x7ff3400ac6d0 2023-10-26T01:39:09.790 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.786+0000 7ff345991640 1 --2- 172.21.15.169:0/2943996618 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff32406ae40 0x7ff32406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:09.793 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.787+0000 7ff345991640 1 --2- 172.21.15.169:0/2943996618 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff32406ae40 0x7ff32406d300 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7ff33c035000 tx=0x7ff33c004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:09.793 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.789+0000 7ff3367fc640 1 -- 172.21.15.169:0/2943996618 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff3300546f0 con 0x7ff3400ac6d0 2023-10-26T01:39:09.979 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.977+0000 7ff346993640 1 -- 172.21.15.169:0/2943996618 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff340133c70 con 0x7ff3400ac6d0 2023-10-26T01:39:09.980 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.979+0000 7ff3367fc640 1 -- 172.21.15.169:0/2943996618 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v140) v1 ==== 74+0+16755 (secure 0 0 0) 0x7ff330015020 con 0x7ff3400ac6d0 2023-10-26T01:39:09.981 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:39:09.981 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":140,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:39:08.067383+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-10-26T01:38:18.617877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"133","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-10-26T01:38:19.793041+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.e","mappings":[{"from":0,"to":5}]},{"pgid":"17.21","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:39:09.983 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.981+0000 7ff346993640 1 -- 172.21.15.169:0/2943996618 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff32406ae40 msgr2=0x7ff32406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:09.983 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.981+0000 7ff346993640 1 --2- 172.21.15.169:0/2943996618 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff32406ae40 0x7ff32406d300 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7ff33c035000 tx=0x7ff33c004730 comp rx=0 tx=0).stop 2023-10-26T01:39:09.983 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.981+0000 7ff346993640 1 -- 172.21.15.169:0/2943996618 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff3400ac6d0 msgr2=0x7ff3400c6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:09.984 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.981+0000 7ff346993640 1 --2- 172.21.15.169:0/2943996618 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff3400ac6d0 0x7ff3400c6270 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7ff33000b3f0 tx=0x7ff33000b8b0 comp rx=0 tx=0).stop 2023-10-26T01:39:09.984 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.981+0000 7ff346993640 1 -- 172.21.15.169:0/2943996618 shutdown_connections 2023-10-26T01:39:09.984 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.981+0000 7ff346993640 1 --2- 172.21.15.169:0/2943996618 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff32406ae40 0x7ff32406d300 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:09.984 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.981+0000 7ff346993640 1 --2- 172.21.15.169:0/2943996618 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff3400ac6d0 0x7ff3400c6270 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:09.984 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.981+0000 7ff346993640 1 --2- 172.21.15.169:0/2943996618 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff3400c9970 0x7ff3400c67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:09.985 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.981+0000 7ff346993640 1 --2- 172.21.15.169:0/2943996618 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff34009a5c0 0x7ff34006f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:09.985 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.981+0000 7ff346993640 1 -- 172.21.15.169:0/2943996618 >> 172.21.15.169:0/2943996618 conn(0x7ff3400aedd0 msgr2=0x7ff3400acb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:09.985 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.981+0000 7ff346993640 1 -- 172.21.15.169:0/2943996618 shutdown_connections 2023-10-26T01:39:09.985 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:09.981+0000 7ff346993640 1 -- 172.21.15.169:0/2943996618 wait complete. 2023-10-26T01:39:10.001 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-26T01:39:10.003 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T01:39:10.219 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.217+0000 7f03eb59e640 1 -- 172.21.15.169:0/2336676194 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03ec0b4a30 msgr2=0x7f03ec0cc220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:10.220 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.217+0000 7f03eb59e640 1 --2- 172.21.15.169:0/2336676194 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03ec0b4a30 0x7f03ec0cc220 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f03ec08a750 tx=0x7f03e402c250 comp rx=0 tx=0).stop 2023-10-26T01:39:10.220 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.217+0000 7f03eb59e640 1 -- 172.21.15.169:0/2336676194 shutdown_connections 2023-10-26T01:39:10.220 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.217+0000 7f03eb59e640 1 --2- 172.21.15.169:0/2336676194 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03ec0b4a30 0x7f03ec0cc220 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:10.220 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.217+0000 7f03eb59e640 1 --2- 172.21.15.169:0/2336676194 >> v2:172.21.15.170:3300/0 conn(0x7f03ec0b45d0 0x7f03ec0cbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:10.221 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.217+0000 7f03eb59e640 1 --2- 172.21.15.169:0/2336676194 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f03ec0b68c0 0x7f03ec065ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:10.221 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.217+0000 7f03eb59e640 1 -- 172.21.15.169:0/2336676194 >> 172.21.15.169:0/2336676194 conn(0x7f03ec0d1360 msgr2=0x7f03ec0d1770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:10.221 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.218+0000 7f03eb59e640 1 -- 172.21.15.169:0/2336676194 shutdown_connections 2023-10-26T01:39:10.221 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.218+0000 7f03eb59e640 1 -- 172.21.15.169:0/2336676194 wait complete. 2023-10-26T01:39:10.223 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.219+0000 7f03eb59e640 1 Processor -- start 2023-10-26T01:39:10.223 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.219+0000 7f03eb59e640 1 -- start start 2023-10-26T01:39:10.224 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.220+0000 7f03eb59e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03ec0b45d0 0x7f03ec1499f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:10.224 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.220+0000 7f03eb59e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f03ec0b4a30 0x7f03ec15b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:10.224 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.220+0000 7f03eb59e640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f03ec0b68c0 0x7f03ec15bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:10.224 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.220+0000 7f03eb59e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03ec14db30 con 0x7f03ec0b45d0 2023-10-26T01:39:10.225 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.220+0000 7f03eb59e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03ec1476f0 con 0x7f03ec0b4a30 2023-10-26T01:39:10.225 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.220+0000 7f03eb59e640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03ec147cb0 con 0x7f03ec0b68c0 2023-10-26T01:39:10.225 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.221+0000 7f03ea59c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03ec0b45d0 0x7f03ec1499f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:10.225 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.221+0000 7f03ea59c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03ec0b45d0 0x7f03ec1499f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:39190/0 (socket says 172.21.15.169:39190) 2023-10-26T01:39:10.225 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.221+0000 7f03ea59c640 1 -- 172.21.15.169:0/2932539474 learned_addr learned my addr 172.21.15.169:0/2932539474 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:39:10.226 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.221+0000 7f03ead9d640 1 --2- 172.21.15.169:0/2932539474 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f03ec0b68c0 0x7f03ec15bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:10.226 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.221+0000 7f03e9d9b640 1 --2- 172.21.15.169:0/2932539474 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f03ec0b4a30 0x7f03ec15b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:10.226 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.221+0000 7f03ea59c640 1 -- 172.21.15.169:0/2932539474 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f03ec0b68c0 msgr2=0x7f03ec15bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:10.226 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.221+0000 7f03ea59c640 1 --2- 172.21.15.169:0/2932539474 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f03ec0b68c0 0x7f03ec15bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:10.226 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.221+0000 7f03ea59c640 1 -- 172.21.15.169:0/2932539474 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f03ec0b4a30 msgr2=0x7f03ec15b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:10.227 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.221+0000 7f03ea59c640 1 --2- 172.21.15.169:0/2932539474 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f03ec0b4a30 0x7f03ec15b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:10.227 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.221+0000 7f03ea59c640 1 -- 172.21.15.169:0/2932539474 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03e4009900 con 0x7f03ec0b45d0 2023-10-26T01:39:10.227 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.221+0000 7f03ead9d640 1 --2- 172.21.15.169:0/2932539474 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f03ec0b68c0 0x7f03ec15bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:39:10.227 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.221+0000 7f03ea59c640 1 --2- 172.21.15.169:0/2932539474 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03ec0b45d0 0x7f03ec1499f0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f03dc00d6d0 tx=0x7f03dc00db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:10.227 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.222+0000 7f03e2ffd640 1 -- 172.21.15.169:0/2932539474 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03dc005220 con 0x7f03ec0b45d0 2023-10-26T01:39:10.228 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.222+0000 7f03e2ffd640 1 -- 172.21.15.169:0/2932539474 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f03dc005490 con 0x7f03ec0b45d0 2023-10-26T01:39:10.228 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.222+0000 7f03e2ffd640 1 -- 172.21.15.169:0/2932539474 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03dc009a80 con 0x7f03ec0b45d0 2023-10-26T01:39:10.228 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.222+0000 7f03eb59e640 1 -- 172.21.15.169:0/2932539474 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03ec15c300 con 0x7f03ec0b45d0 2023-10-26T01:39:10.228 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.222+0000 7f03eb59e640 1 -- 172.21.15.169:0/2932539474 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03ec147940 con 0x7f03ec0b45d0 2023-10-26T01:39:10.229 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.223+0000 7f03eb59e640 1 -- 172.21.15.169:0/2932539474 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03ec073c10 con 0x7f03ec0b45d0 2023-10-26T01:39:10.229 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.224+0000 7f03e2ffd640 1 -- 172.21.15.169:0/2932539474 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f03dc0047a0 con 0x7f03ec0b45d0 2023-10-26T01:39:10.230 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.225+0000 7f03e2ffd640 1 --2- 172.21.15.169:0/2932539474 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f03cc06ae40 0x7f03cc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:10.230 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.225+0000 7f03e2ffd640 1 -- 172.21.15.169:0/2932539474 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f03dc0899f0 con 0x7f03ec0b45d0 2023-10-26T01:39:10.230 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.225+0000 7f03e2ffd640 1 -- 172.21.15.169:0/2932539474 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=141}) v3 -- 0x7f03cc072a30 con 0x7f03ec0b45d0 2023-10-26T01:39:10.230 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.225+0000 7f03e9d9b640 1 --2- 172.21.15.169:0/2932539474 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f03cc06ae40 0x7f03cc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:10.230 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.226+0000 7f03e9d9b640 1 --2- 172.21.15.169:0/2932539474 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f03cc06ae40 0x7f03cc06d300 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f03d4005d80 tx=0x7f03d4005c90 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:10.231 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.229+0000 7f03e2ffd640 1 -- 172.21.15.169:0/2932539474 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f03dc017020 con 0x7f03ec0b45d0 2023-10-26T01:39:10.440 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.438+0000 7f03eb59e640 1 -- 172.21.15.169:0/2932539474 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f03ec0d1ee0 con 0x7f03ec0b45d0 2023-10-26T01:39:10.441 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.440+0000 7f03e2ffd640 1 -- 172.21.15.169:0/2932539474 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+4927 (secure 0 0 0) 0x7f03dc020070 con 0x7f03ec0b45d0 2023-10-26T01:39:10.442 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:39:10.442 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":68,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5917,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/3297412140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":3297412140},{"type":"v1","addr":"172.21.15.169:6835","nonce":3297412140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":5929,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/2575809940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2575809940},{"type":"v1","addr":"172.21.15.170:6836","nonce":2575809940}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":7761,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2632217991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2632217991},{"type":"v1","addr":"172.21.15.169:6833","nonce":2632217991}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T01:38:20.773556+0000","modified":"2023-10-26T01:38:23.839758+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7773},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7773":{"gid":7773,"name":"b","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6837/3615088922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":3615088922},{"type":"v1","addr":"172.21.15.170:6837","nonce":3615088922}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-10-26T01:39:10.442 INFO:teuthology.orchestra.run.smithi169.stderr:dumped fsmap epoch 68 2023-10-26T01:39:10.444 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.442+0000 7f03eb59e640 1 -- 172.21.15.169:0/2932539474 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f03cc06ae40 msgr2=0x7f03cc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:10.444 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.442+0000 7f03eb59e640 1 --2- 172.21.15.169:0/2932539474 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f03cc06ae40 0x7f03cc06d300 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f03d4005d80 tx=0x7f03d4005c90 comp rx=0 tx=0).stop 2023-10-26T01:39:10.444 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.442+0000 7f03eb59e640 1 -- 172.21.15.169:0/2932539474 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03ec0b45d0 msgr2=0x7f03ec1499f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:10.445 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.442+0000 7f03eb59e640 1 --2- 172.21.15.169:0/2932539474 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03ec0b45d0 0x7f03ec1499f0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f03dc00d6d0 tx=0x7f03dc00db90 comp rx=0 tx=0).stop 2023-10-26T01:39:10.445 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.442+0000 7f03eb59e640 1 -- 172.21.15.169:0/2932539474 shutdown_connections 2023-10-26T01:39:10.445 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.442+0000 7f03eb59e640 1 --2- 172.21.15.169:0/2932539474 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f03cc06ae40 0x7f03cc06d300 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:10.445 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.442+0000 7f03eb59e640 1 --2- 172.21.15.169:0/2932539474 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f03ec0b68c0 0x7f03ec15bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:10.445 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.442+0000 7f03eb59e640 1 --2- 172.21.15.169:0/2932539474 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f03ec0b4a30 0x7f03ec15b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:10.446 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.442+0000 7f03eb59e640 1 --2- 172.21.15.169:0/2932539474 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03ec0b45d0 0x7f03ec1499f0 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:10.446 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.442+0000 7f03eb59e640 1 -- 172.21.15.169:0/2932539474 >> 172.21.15.169:0/2932539474 conn(0x7f03ec0d1360 msgr2=0x7f03ec0cbad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:10.446 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.442+0000 7f03eb59e640 1 -- 172.21.15.169:0/2932539474 shutdown_connections 2023-10-26T01:39:10.446 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.443+0000 7f03eb59e640 1 -- 172.21.15.169:0/2932539474 wait complete. 2023-10-26T01:39:10.462 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T01:39:10.673 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.670+0000 7f300a644640 1 -- 172.21.15.169:0/2098791416 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f300415f610 msgr2=0x7f30040ce630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:10.673 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.671+0000 7f300a644640 1 --2- 172.21.15.169:0/2098791416 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f300415f610 0x7f30040ce630 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f30040cd200 tx=0x7f2fec02bd90 comp rx=0 tx=0).stop 2023-10-26T01:39:10.673 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.671+0000 7f300a644640 1 -- 172.21.15.169:0/2098791416 shutdown_connections 2023-10-26T01:39:10.673 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.671+0000 7f300a644640 1 --2- 172.21.15.169:0/2098791416 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f300415f610 0x7f30040ce630 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:10.674 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.671+0000 7f300a644640 1 --2- 172.21.15.169:0/2098791416 >> v2:172.21.15.170:3300/0 conn(0x7f300415b9d0 0x7f30040cbf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:10.674 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.671+0000 7f300a644640 1 -- 172.21.15.169:0/2098791416 >> 172.21.15.169:0/2098791416 conn(0x7f3004069e00 msgr2=0x7f300406a210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:10.674 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.671+0000 7f300a644640 1 -- 172.21.15.169:0/2098791416 shutdown_connections 2023-10-26T01:39:10.674 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.671+0000 7f300a644640 1 -- 172.21.15.169:0/2098791416 wait complete. 2023-10-26T01:39:10.676 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.672+0000 7f300a644640 1 Processor -- start 2023-10-26T01:39:10.676 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.673+0000 7f300a644640 1 -- start start 2023-10-26T01:39:10.676 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.673+0000 7f300a644640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30040ceb70 0x7f300415bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:10.676 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.673+0000 7f300a644640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f300415b9d0 0x7f3004159790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:10.677 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.673+0000 7f300a644640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f300415f610 0x7f30040c7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:10.677 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.674+0000 7f300a644640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f300415d290 con 0x7f30040ceb70 2023-10-26T01:39:10.677 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.674+0000 7f300a644640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30040c8010 con 0x7f300415b9d0 2023-10-26T01:39:10.677 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.674+0000 7f300a644640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f300415c680 con 0x7f300415f610 2023-10-26T01:39:10.677 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.674+0000 7f3003fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30040ceb70 0x7f300415bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:10.678 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.674+0000 7f3003fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30040ceb70 0x7f300415bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:39202/0 (socket says 172.21.15.169:39202) 2023-10-26T01:39:10.678 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.674+0000 7f3003fff640 1 -- 172.21.15.169:0/1162132980 learned_addr learned my addr 172.21.15.169:0/1162132980 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:39:10.678 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.674+0000 7f3008be1640 1 --2- 172.21.15.169:0/1162132980 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f300415f610 0x7f30040c7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:10.678 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.674+0000 7f30037fe640 1 --2- 172.21.15.169:0/1162132980 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f300415b9d0 0x7f3004159790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:10.678 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.674+0000 7f3003fff640 1 -- 172.21.15.169:0/1162132980 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f300415f610 msgr2=0x7f30040c7ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:10.679 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.674+0000 7f3003fff640 1 --2- 172.21.15.169:0/1162132980 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f300415f610 0x7f30040c7ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:10.679 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.674+0000 7f3003fff640 1 -- 172.21.15.169:0/1162132980 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f300415b9d0 msgr2=0x7f3004159790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:10.679 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.674+0000 7f3003fff640 1 --2- 172.21.15.169:0/1162132980 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f300415b9d0 0x7f3004159790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:10.680 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.674+0000 7f3003fff640 1 -- 172.21.15.169:0/1162132980 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fec009560 con 0x7f30040ceb70 2023-10-26T01:39:10.680 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.675+0000 7f3003fff640 1 --2- 172.21.15.169:0/1162132980 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30040ceb70 0x7f300415bff0 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f2ff400cce0 tx=0x7f2ff400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:10.682 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.675+0000 7f3000ff9640 1 -- 172.21.15.169:0/1162132980 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ff4004b20 con 0x7f30040ceb70 2023-10-26T01:39:10.682 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.675+0000 7f3000ff9640 1 -- 172.21.15.169:0/1162132980 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2ff4004d70 con 0x7f30040ceb70 2023-10-26T01:39:10.683 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.675+0000 7f300a644640 1 -- 172.21.15.169:0/1162132980 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f300415c960 con 0x7f30040ceb70 2023-10-26T01:39:10.683 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.675+0000 7f3000ff9640 1 -- 172.21.15.169:0/1162132980 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ff4005280 con 0x7f30040ceb70 2023-10-26T01:39:10.683 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.675+0000 7f300a644640 1 -- 172.21.15.169:0/1162132980 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f300415cf20 con 0x7f30040ceb70 2023-10-26T01:39:10.684 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.676+0000 7f300a644640 1 -- 172.21.15.169:0/1162132980 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f300404db10 con 0x7f30040ceb70 2023-10-26T01:39:10.684 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.677+0000 7f3000ff9640 1 -- 172.21.15.169:0/1162132980 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f2ff40053e0 con 0x7f30040ceb70 2023-10-26T01:39:10.685 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.678+0000 7f3000ff9640 1 --2- 172.21.15.169:0/1162132980 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2fe406ae70 0x7f2fe406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:10.685 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.678+0000 7f3000ff9640 1 -- 172.21.15.169:0/1162132980 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f2ff4089ee0 con 0x7f30040ceb70 2023-10-26T01:39:10.685 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.678+0000 7f3000ff9640 1 -- 172.21.15.169:0/1162132980 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=141}) v3 -- 0x7f2fe4072a60 con 0x7f30040ceb70 2023-10-26T01:39:10.686 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.679+0000 7f30037fe640 1 --2- 172.21.15.169:0/1162132980 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2fe406ae70 0x7f2fe406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:10.686 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.680+0000 7f30037fe640 1 --2- 172.21.15.169:0/1162132980 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2fe406ae70 0x7f2fe406d330 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f2fec035000 tx=0x7f2fec004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:10.687 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.682+0000 7f3000ff9640 1 -- 172.21.15.169:0/1162132980 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2ff401d070 con 0x7f30040ceb70 2023-10-26T01:39:10.895 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.894+0000 7f300a644640 1 -- 172.21.15.169:0/1162132980 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f30040cf4b0 con 0x7f30040ceb70 2023-10-26T01:39:10.896 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.895+0000 7f3000ff9640 1 -- 172.21.15.169:0/1162132980 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+4927 (secure 0 0 0) 0x7f2ff4054f30 con 0x7f30040ceb70 2023-10-26T01:39:10.897 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:39:10.897 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":68,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5917,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/3297412140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":3297412140},{"type":"v1","addr":"172.21.15.169:6835","nonce":3297412140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":5929,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/2575809940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2575809940},{"type":"v1","addr":"172.21.15.170:6836","nonce":2575809940}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":7761,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2632217991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2632217991},{"type":"v1","addr":"172.21.15.169:6833","nonce":2632217991}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T01:38:20.773556+0000","modified":"2023-10-26T01:38:23.839758+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7773},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7773":{"gid":7773,"name":"b","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6837/3615088922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":3615088922},{"type":"v1","addr":"172.21.15.170:6837","nonce":3615088922}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-10-26T01:39:10.897 INFO:teuthology.orchestra.run.smithi169.stderr:dumped fsmap epoch 68 2023-10-26T01:39:10.899 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.897+0000 7f300a644640 1 -- 172.21.15.169:0/1162132980 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2fe406ae70 msgr2=0x7f2fe406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:10.899 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.897+0000 7f300a644640 1 --2- 172.21.15.169:0/1162132980 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2fe406ae70 0x7f2fe406d330 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f2fec035000 tx=0x7f2fec004730 comp rx=0 tx=0).stop 2023-10-26T01:39:10.899 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.897+0000 7f300a644640 1 -- 172.21.15.169:0/1162132980 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30040ceb70 msgr2=0x7f300415bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:10.899 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.897+0000 7f300a644640 1 --2- 172.21.15.169:0/1162132980 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30040ceb70 0x7f300415bff0 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f2ff400cce0 tx=0x7f2ff400a620 comp rx=0 tx=0).stop 2023-10-26T01:39:10.900 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.897+0000 7f300a644640 1 -- 172.21.15.169:0/1162132980 shutdown_connections 2023-10-26T01:39:10.900 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.897+0000 7f300a644640 1 --2- 172.21.15.169:0/1162132980 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2fe406ae70 0x7f2fe406d330 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:10.900 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.897+0000 7f300a644640 1 --2- 172.21.15.169:0/1162132980 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f300415f610 0x7f30040c7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:10.900 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.897+0000 7f300a644640 1 --2- 172.21.15.169:0/1162132980 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f300415b9d0 0x7f3004159790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:10.900 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.897+0000 7f300a644640 1 --2- 172.21.15.169:0/1162132980 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30040ceb70 0x7f300415bff0 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:10.901 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.897+0000 7f300a644640 1 -- 172.21.15.169:0/1162132980 >> 172.21.15.169:0/1162132980 conn(0x7f3004069e00 msgr2=0x7f30040d1650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:10.901 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.897+0000 7f300a644640 1 -- 172.21.15.169:0/1162132980 shutdown_connections 2023-10-26T01:39:10.901 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:10.898+0000 7f300a644640 1 -- 172.21.15.169:0/1162132980 wait complete. 2023-10-26T01:39:10.916 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:39:11.126 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.124+0000 7f5ef1a32640 1 -- 172.21.15.169:0/695263684 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5ed40039a0 msgr2=0x7f5ed4003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:11.126 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.124+0000 7f5ef1a32640 1 --2- 172.21.15.169:0/695263684 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5ed40039a0 0x7f5ed4003dc0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f5edc001d50 tx=0x7f5ed40057f0 comp rx=0 tx=0).stop 2023-10-26T01:39:11.127 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.124+0000 7f5ef1231640 1 -- 172.21.15.169:0/695263684 reap_dead start 2023-10-26T01:39:11.127 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.124+0000 7f5ef1a32640 1 -- 172.21.15.169:0/695263684 shutdown_connections 2023-10-26T01:39:11.127 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.124+0000 7f5ef1a32640 1 -- 172.21.15.169:0/695263684 >> 172.21.15.169:0/695263684 conn(0x7f5eec0d1360 msgr2=0x7f5eec0d1770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:11.127 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.124+0000 7f5ef1a32640 1 -- 172.21.15.169:0/695263684 shutdown_connections 2023-10-26T01:39:11.127 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.124+0000 7f5ef1a32640 1 -- 172.21.15.169:0/695263684 wait complete. 2023-10-26T01:39:11.131 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.126+0000 7f5ef1a32640 1 Processor -- start 2023-10-26T01:39:11.131 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.126+0000 7f5ef1a32640 1 -- start start 2023-10-26T01:39:11.132 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.127+0000 7f5ef1a32640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5ed40039a0 0x7f5eec148e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:11.132 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.127+0000 7f5ef1a32640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5eec1f1670 0x7f5eec1498a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:11.132 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.127+0000 7f5ef1a32640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5eec0ca8f0 0x7f5eec0cada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:11.133 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.127+0000 7f5ef1a32640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eec0cb2e0 con 0x7f5ed40039a0 2023-10-26T01:39:11.133 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.127+0000 7f5ef1a32640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eec149390 con 0x7f5eec1f1670 2023-10-26T01:39:11.133 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.127+0000 7f5ef1a32640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5eec1466c0 con 0x7f5eec0ca8f0 2023-10-26T01:39:11.134 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.127+0000 7f5ef0a30640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5ed40039a0 0x7f5eec148e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:11.134 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.127+0000 7f5ef0a30640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5ed40039a0 0x7f5eec148e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:39234/0 (socket says 172.21.15.169:39234) 2023-10-26T01:39:11.134 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.127+0000 7f5ef0a30640 1 -- 172.21.15.169:0/2590249332 learned_addr learned my addr 172.21.15.169:0/2590249332 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:39:11.134 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.127+0000 7f5ef0a30640 1 -- 172.21.15.169:0/2590249332 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5eec0ca8f0 msgr2=0x7f5eec0cada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:11.134 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.127+0000 7f5ef0a30640 1 --2- 172.21.15.169:0/2590249332 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5eec0ca8f0 0x7f5eec0cada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:11.135 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.127+0000 7f5ef0a30640 1 -- 172.21.15.169:0/2590249332 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5eec1f1670 msgr2=0x7f5eec1498a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:11.135 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.127+0000 7f5ef0a30640 1 --2- 172.21.15.169:0/2590249332 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5eec1f1670 0x7f5eec1498a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:11.135 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.127+0000 7f5ef0a30640 1 -- 172.21.15.169:0/2590249332 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eec0b4d30 con 0x7f5ed40039a0 2023-10-26T01:39:11.135 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.128+0000 7f5ef0a30640 1 --2- 172.21.15.169:0/2590249332 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5ed40039a0 0x7f5eec148e50 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f5edc02c510 tx=0x7f5edc02c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:11.135 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.128+0000 7f5ee97fa640 1 -- 172.21.15.169:0/2590249332 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5edc02ce20 con 0x7f5ed40039a0 2023-10-26T01:39:11.136 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.128+0000 7f5ee97fa640 1 -- 172.21.15.169:0/2590249332 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5edc006050 con 0x7f5ed40039a0 2023-10-26T01:39:11.136 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.128+0000 7f5ee97fa640 1 -- 172.21.15.169:0/2590249332 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5edc006440 con 0x7f5ed40039a0 2023-10-26T01:39:11.136 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.128+0000 7f5ef1a32640 1 -- 172.21.15.169:0/2590249332 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5eec1469a0 con 0x7f5ed40039a0 2023-10-26T01:39:11.136 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.128+0000 7f5ef1a32640 1 -- 172.21.15.169:0/2590249332 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5eec146f60 con 0x7f5ed40039a0 2023-10-26T01:39:11.137 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.129+0000 7f5ef1a32640 1 -- 172.21.15.169:0/2590249332 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5eec073c10 con 0x7f5ed40039a0 2023-10-26T01:39:11.138 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.130+0000 7f5ee97fa640 1 -- 172.21.15.169:0/2590249332 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f5edc0065a0 con 0x7f5ed40039a0 2023-10-26T01:39:11.138 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.131+0000 7f5ee97fa640 1 --2- 172.21.15.169:0/2590249332 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5ed806ae40 0x7f5ed806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:11.138 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.131+0000 7f5ee97fa640 1 -- 172.21.15.169:0/2590249332 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f5edc0aae90 con 0x7f5ed40039a0 2023-10-26T01:39:11.139 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.131+0000 7f5ee97fa640 1 -- 172.21.15.169:0/2590249332 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=141}) v3 -- 0x7f5ed8072a30 con 0x7f5ed40039a0 2023-10-26T01:39:11.139 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.132+0000 7f5eebfff640 1 --2- 172.21.15.169:0/2590249332 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5ed806ae40 0x7f5ed806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:11.139 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.133+0000 7f5eebfff640 1 --2- 172.21.15.169:0/2590249332 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5ed806ae40 0x7f5ed806d300 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f5ee00017a0 tx=0x7f5ee0001470 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:11.139 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.135+0000 7f5ee97fa640 1 -- 172.21.15.169:0/2590249332 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5edc0ac060 con 0x7f5ed40039a0 2023-10-26T01:39:11.328 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.326+0000 7f5ef1a32640 1 -- 172.21.15.169:0/2590249332 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5eec0b3470 con 0x7f5ed40039a0 2023-10-26T01:39:11.329 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.328+0000 7f5ee97fa640 1 -- 172.21.15.169:0/2590249332 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v140) v1 ==== 74+0+16755 (secure 0 0 0) 0x7f5edc045070 con 0x7f5ed40039a0 2023-10-26T01:39:11.330 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:39:11.331 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":140,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:39:08.067383+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-10-26T01:38:18.617877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"133","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-10-26T01:38:19.793041+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.e","mappings":[{"from":0,"to":5}]},{"pgid":"17.21","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:39:11.333 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.331+0000 7f5ef1a32640 1 -- 172.21.15.169:0/2590249332 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5ed806ae40 msgr2=0x7f5ed806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:11.333 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.331+0000 7f5ef1a32640 1 --2- 172.21.15.169:0/2590249332 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5ed806ae40 0x7f5ed806d300 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f5ee00017a0 tx=0x7f5ee0001470 comp rx=0 tx=0).stop 2023-10-26T01:39:11.333 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.331+0000 7f5ef1a32640 1 -- 172.21.15.169:0/2590249332 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5ed40039a0 msgr2=0x7f5eec148e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:11.333 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.331+0000 7f5ef1a32640 1 --2- 172.21.15.169:0/2590249332 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5ed40039a0 0x7f5eec148e50 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f5edc02c510 tx=0x7f5edc02c8a0 comp rx=0 tx=0).stop 2023-10-26T01:39:11.334 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.331+0000 7f5ef1a32640 1 -- 172.21.15.169:0/2590249332 shutdown_connections 2023-10-26T01:39:11.334 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.331+0000 7f5ef1a32640 1 --2- 172.21.15.169:0/2590249332 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5ed806ae40 0x7f5ed806d300 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:11.334 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.331+0000 7f5ef1a32640 1 --2- 172.21.15.169:0/2590249332 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5eec0ca8f0 0x7f5eec0cada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:11.334 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.331+0000 7f5ef1a32640 1 --2- 172.21.15.169:0/2590249332 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5eec1f1670 0x7f5eec1498a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:11.334 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.331+0000 7f5ef1a32640 1 --2- 172.21.15.169:0/2590249332 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5ed40039a0 0x7f5eec148e50 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:11.335 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.331+0000 7f5ef1a32640 1 -- 172.21.15.169:0/2590249332 >> 172.21.15.169:0/2590249332 conn(0x7f5eec0d1360 msgr2=0x7f5eec0b2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:11.335 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.331+0000 7f5ef1a32640 1 -- 172.21.15.169:0/2590249332 shutdown_connections 2023-10-26T01:39:11.335 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.331+0000 7f5ef1a32640 1 -- 172.21.15.169:0/2590249332 wait complete. 2023-10-26T01:39:11.352 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-26T01:39:11.571 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.569+0000 7f35a5621640 1 -- 172.21.15.169:0/4274834624 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35a015b9a0 msgr2=0x7f35a015f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:11.572 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.569+0000 7f35a5621640 1 --2- 172.21.15.169:0/4274834624 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35a015b9a0 0x7f35a015f470 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f3590005fd0 tx=0x7f359002c240 comp rx=0 tx=0).stop 2023-10-26T01:39:11.572 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.569+0000 7f35a5621640 1 -- 172.21.15.169:0/4274834624 shutdown_connections 2023-10-26T01:39:11.572 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.569+0000 7f35a5621640 1 --2- 172.21.15.169:0/4274834624 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f35a00b3650 0x7f35a00699e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:11.573 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.569+0000 7f35a5621640 1 --2- 172.21.15.169:0/4274834624 >> v2:172.21.15.170:3300/0 conn(0x7f35a00b7330 0x7f35a00b3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:11.573 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.569+0000 7f35a5621640 1 --2- 172.21.15.169:0/4274834624 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35a015b9a0 0x7f35a015f470 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:11.573 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.569+0000 7f35a5621640 1 -- 172.21.15.169:0/4274834624 >> 172.21.15.169:0/4274834624 conn(0x7f35a00658b0 msgr2=0x7f35a0065cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:11.573 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.570+0000 7f35a5621640 1 -- 172.21.15.169:0/4274834624 shutdown_connections 2023-10-26T01:39:11.573 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.570+0000 7f35a5621640 1 -- 172.21.15.169:0/4274834624 wait complete. 2023-10-26T01:39:11.575 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.571+0000 7f35a5621640 1 Processor -- start 2023-10-26T01:39:11.576 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.571+0000 7f35a5621640 1 -- start start 2023-10-26T01:39:11.576 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.572+0000 7f35a5621640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f35a00b3650 0x7f35a015ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:11.576 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.572+0000 7f35a5621640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35a00b7330 0x7f35a00c9170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:11.576 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.572+0000 7f35a5621640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f35a015b9a0 0x7f35a014ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:11.577 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.573+0000 7f35a5621640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35a015c5f0 con 0x7f35a00b7330 2023-10-26T01:39:11.577 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.573+0000 7f35a5621640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35a015bf70 con 0x7f35a015b9a0 2023-10-26T01:39:11.577 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.573+0000 7f35a5621640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35a00c99a0 con 0x7f35a00b3650 2023-10-26T01:39:11.577 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.573+0000 7f35967fc640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35a00b7330 0x7f35a00c9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:11.577 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.573+0000 7f35967fc640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35a00b7330 0x7f35a00c9170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:39254/0 (socket says 172.21.15.169:39254) 2023-10-26T01:39:11.578 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.573+0000 7f35967fc640 1 -- 172.21.15.169:0/2954040781 learned_addr learned my addr 172.21.15.169:0/2954040781 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:39:11.578 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.573+0000 7f359effd640 1 --2- 172.21.15.169:0/2954040781 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f35a00b3650 0x7f35a015ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:11.578 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.573+0000 7f359f7fe640 1 --2- 172.21.15.169:0/2954040781 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f35a015b9a0 0x7f35a014ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:11.578 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.573+0000 7f35967fc640 1 -- 172.21.15.169:0/2954040781 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f35a00b3650 msgr2=0x7f35a015ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:11.578 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.573+0000 7f35967fc640 1 --2- 172.21.15.169:0/2954040781 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f35a00b3650 0x7f35a015ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:11.579 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.573+0000 7f35967fc640 1 -- 172.21.15.169:0/2954040781 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f35a015b9a0 msgr2=0x7f35a014ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:11.579 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.573+0000 7f35967fc640 1 --2- 172.21.15.169:0/2954040781 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f35a015b9a0 0x7f35a014ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:11.579 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.573+0000 7f35967fc640 1 -- 172.21.15.169:0/2954040781 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3590009900 con 0x7f35a00b7330 2023-10-26T01:39:11.579 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.573+0000 7f359effd640 1 --2- 172.21.15.169:0/2954040781 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f35a00b3650 0x7f35a015ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:39:11.579 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.574+0000 7f35967fc640 1 --2- 172.21.15.169:0/2954040781 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35a00b7330 0x7f35a00c9170 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f358800c890 tx=0x7f358800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:11.579 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.574+0000 7f3597fff640 1 -- 172.21.15.169:0/2954040781 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3588004220 con 0x7f35a00b7330 2023-10-26T01:39:11.580 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.574+0000 7f3597fff640 1 -- 172.21.15.169:0/2954040781 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3588004470 con 0x7f35a00b7330 2023-10-26T01:39:11.580 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.574+0000 7f3597fff640 1 -- 172.21.15.169:0/2954040781 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3588007c10 con 0x7f35a00b7330 2023-10-26T01:39:11.580 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.574+0000 7f35a5621640 1 -- 172.21.15.169:0/2954040781 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35a00c9c80 con 0x7f35a00b7330 2023-10-26T01:39:11.580 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.574+0000 7f35a5621640 1 -- 172.21.15.169:0/2954040781 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35a00c96b0 con 0x7f35a00b7330 2023-10-26T01:39:11.581 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.575+0000 7f35a5621640 1 -- 172.21.15.169:0/2954040781 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35a004db10 con 0x7f35a00b7330 2023-10-26T01:39:11.582 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.576+0000 7f3597fff640 1 -- 172.21.15.169:0/2954040781 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f3588005020 con 0x7f35a00b7330 2023-10-26T01:39:11.582 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.577+0000 7f3597fff640 1 --2- 172.21.15.169:0/2954040781 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f358006ae70 0x7f358006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:11.582 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.577+0000 7f3597fff640 1 -- 172.21.15.169:0/2954040781 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f3588089c20 con 0x7f35a00b7330 2023-10-26T01:39:11.582 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.577+0000 7f3597fff640 1 -- 172.21.15.169:0/2954040781 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=141}) v3 -- 0x7f3580072a60 con 0x7f35a00b7330 2023-10-26T01:39:11.582 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.578+0000 7f359effd640 1 --2- 172.21.15.169:0/2954040781 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f358006ae70 0x7f358006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:11.583 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.579+0000 7f359effd640 1 --2- 172.21.15.169:0/2954040781 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f358006ae70 0x7f358006d330 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f3590036000 tx=0x7f3590004ab0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:11.583 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.581+0000 7f3597fff640 1 -- 172.21.15.169:0/2954040781 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f358801d070 con 0x7f35a00b7330 2023-10-26T01:39:11.814 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.812+0000 7f35a5621640 1 -- 172.21.15.169:0/2954040781 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f35a00cefa0 con 0x7f35a00b7330 2023-10-26T01:39:11.815 INFO:tasks.ceph.mon.a.smithi169.stderr:2023-10-26T01:39:11.813+0000 7f1588a74640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-26T01:39:11.819 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.817+0000 7f3597fff640 1 -- 172.21.15.169:0/2954040781 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v69) v1 ==== 161+0+0 (secure 0 0 0) 0x7f3588054bf0 con 0x7f35a00b7330 2023-10-26T01:39:11.819 INFO:teuthology.orchestra.run.smithi169.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-26T01:39:11.824 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.820+0000 7f3597fff640 1 -- 172.21.15.169:0/2954040781 <== mon.0 v2:172.21.15.169:3300/0 8 ==== osd_map(141..141 src has 1..141) v4 ==== 432+0+0 (secure 0 0 0) 0x7f3588058fb0 con 0x7f35a00b7330 2023-10-26T01:39:11.824 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.820+0000 7f3597fff640 1 -- 172.21.15.169:0/2954040781 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=142}) v3 -- 0x7f3580073400 con 0x7f35a00b7330 2023-10-26T01:39:11.826 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.824+0000 7f35a5621640 1 -- 172.21.15.169:0/2954040781 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f358006ae70 msgr2=0x7f358006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:11.827 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.824+0000 7f35a5621640 1 --2- 172.21.15.169:0/2954040781 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f358006ae70 0x7f358006d330 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f3590036000 tx=0x7f3590004ab0 comp rx=0 tx=0).stop 2023-10-26T01:39:11.827 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.824+0000 7f35a5621640 1 -- 172.21.15.169:0/2954040781 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35a00b7330 msgr2=0x7f35a00c9170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:11.828 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.824+0000 7f35a5621640 1 --2- 172.21.15.169:0/2954040781 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35a00b7330 0x7f35a00c9170 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f358800c890 tx=0x7f358800cd50 comp rx=0 tx=0).stop 2023-10-26T01:39:11.828 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.824+0000 7f35a5621640 1 -- 172.21.15.169:0/2954040781 shutdown_connections 2023-10-26T01:39:11.828 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.824+0000 7f35a5621640 1 --2- 172.21.15.169:0/2954040781 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f358006ae70 0x7f358006d330 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:11.829 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.824+0000 7f35a5621640 1 --2- 172.21.15.169:0/2954040781 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35a00b7330 0x7f35a00c9170 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:11.829 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.824+0000 7f35a5621640 1 --2- 172.21.15.169:0/2954040781 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f35a015b9a0 0x7f35a014ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:11.829 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.824+0000 7f35a5621640 1 --2- 172.21.15.169:0/2954040781 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f35a00b3650 0x7f35a015ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:11.830 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.824+0000 7f35a5621640 1 -- 172.21.15.169:0/2954040781 >> 172.21.15.169:0/2954040781 conn(0x7f35a00658b0 msgr2=0x7f35a015aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:11.830 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.824+0000 7f35a5621640 1 -- 172.21.15.169:0/2954040781 shutdown_connections 2023-10-26T01:39:11.830 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:11.824+0000 7f35a5621640 1 -- 172.21.15.169:0/2954040781 wait complete. 2023-10-26T01:39:11.846 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-26T01:39:11.861 INFO:tasks.ceph.mds.b.smithi170.stderr: -9248> 2023-10-26T01:38:27.275+0000 7f56829a1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-26T01:39:11.947 INFO:tasks.ceph.mds.b.smithi170.stdout:starting mds.b at 2023-10-26T01:39:12.062 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:12.060+0000 7f25d4464640 1 -- 172.21.15.169:0/263540677 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25b8003970 msgr2=0x7f25b8003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:12.062 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:12.060+0000 7f25d4464640 1 --2- 172.21.15.169:0/263540677 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25b8003970 0x7f25b8003d90 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f25c8006fc0 tx=0x7f25b80057f0 comp rx=0 tx=0).stop 2023-10-26T01:39:12.063 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:12.060+0000 7f25d2a01640 1 -- 172.21.15.169:0/263540677 reap_dead start 2023-10-26T01:39:12.063 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:12.060+0000 7f25d4464640 1 -- 172.21.15.169:0/263540677 shutdown_connections 2023-10-26T01:39:12.063 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:12.060+0000 7f25d4464640 1 -- 172.21.15.169:0/263540677 >> 172.21.15.169:0/263540677 conn(0x7f25cc069930 msgr2=0x7f25cc069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:12.063 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:12.060+0000 7f25d4464640 1 -- 172.21.15.169:0/263540677 shutdown_connections 2023-10-26T01:39:12.063 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:12.060+0000 7f25d4464640 1 -- 172.21.15.169:0/263540677 wait complete. 2023-10-26T01:39:12.064 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:12.062+0000 7f25d4464640 1 Processor -- start 2023-10-26T01:39:12.064 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:12.062+0000 7f25d4464640 1 -- start start 2023-10-26T01:39:12.065 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:12.063+0000 7f25d4464640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25b8003970 0x7f25cc06ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:12.065 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:12.063+0000 7f25d4464640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f25cc1483b0 0x7f25cc14eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:12.065 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:12.063+0000 7f25d4464640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25cc14f1e0 0x7f25cc14f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:12.066 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:12.063+0000 7f25d4464640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25cc14d4c0 con 0x7f25b8003970 2023-10-26T01:39:12.066 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:12.063+0000 7f25d4464640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25cc071320 con 0x7f25cc14f1e0 2023-10-26T01:39:12.066 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:12.063+0000 7f25d4464640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25cc071600 con 0x7f25cc1483b0 2023-10-26T01:39:12.066 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:12.063+0000 7f25d2200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25b8003970 0x7f25cc06ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:12.066 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:12.064+0000 7f25d2200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25b8003970 0x7f25cc06ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:39276/0 (socket says 172.21.15.169:39276) 2023-10-26T01:39:12.067 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:12.064+0000 7f25d2200640 1 -- 172.21.15.169:0/3484185391 learned_addr learned my addr 172.21.15.169:0/3484185391 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:39:12.068 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:12.064+0000 7f25d2200640 1 -- 172.21.15.169:0/3484185391 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f25cc1483b0 msgr2=0x7f25cc14eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:12.068 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:12.064+0000 7f25d2200640 1 --2- 172.21.15.169:0/3484185391 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f25cc1483b0 0x7f25cc14eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:12.069 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:12.064+0000 7f25d2200640 1 -- 172.21.15.169:0/3484185391 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25cc14f1e0 msgr2=0x7f25cc14f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:12.069 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:12.064+0000 7f25d2200640 1 --2- 172.21.15.169:0/3484185391 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25cc14f1e0 0x7f25cc14f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:12.069 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:12.064+0000 7f25d2200640 1 -- 172.21.15.169:0/3484185391 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25cc0718e0 con 0x7f25b8003970 2023-10-26T01:39:12.069 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:12.064+0000 7f25d2200640 1 --2- 172.21.15.169:0/3484185391 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25b8003970 0x7f25cc06ed30 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f25c802c8e0 tx=0x7f25c8001da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:12.069 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:12.064+0000 7f25c2ffd640 1 -- 172.21.15.169:0/3484185391 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25c8001670 con 0x7f25b8003970 2023-10-26T01:39:12.070 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:12.065+0000 7f25c2ffd640 1 -- 172.21.15.169:0/3484185391 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f25c8006250 con 0x7f25b8003970 2023-10-26T01:39:12.070 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:12.065+0000 7f25c2ffd640 1 -- 172.21.15.169:0/3484185391 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25c80035f0 con 0x7f25b8003970 2023-10-26T01:39:12.070 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:12.065+0000 7f25d4464640 1 -- 172.21.15.169:0/3484185391 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25cc071bc0 con 0x7f25b8003970 2023-10-26T01:39:12.070 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:12.065+0000 7f25d4464640 1 -- 172.21.15.169:0/3484185391 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25cc148830 con 0x7f25b8003970 2023-10-26T01:39:12.070 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:12.066+0000 7f25d4464640 1 -- 172.21.15.169:0/3484185391 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25cc1489f0 con 0x7f25b8003970 2023-10-26T01:39:12.071 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:12.067+0000 7f25c2ffd640 1 -- 172.21.15.169:0/3484185391 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f25c8031030 con 0x7f25b8003970 2023-10-26T01:39:12.071 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:12.068+0000 7f25c2ffd640 1 --2- 172.21.15.169:0/3484185391 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f25b006ae70 0x7f25b006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:12.072 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:12.068+0000 7f25c2ffd640 1 -- 172.21.15.169:0/3484185391 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f25c80aa810 con 0x7f25b8003970 2023-10-26T01:39:12.072 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:12.068+0000 7f25c2ffd640 1 -- 172.21.15.169:0/3484185391 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=142}) v3 -- 0x7f25b0072b70 con 0x7f25b8003970 2023-10-26T01:39:12.072 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:12.068+0000 7f25d19ff640 1 --2- 172.21.15.169:0/3484185391 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f25b006ae70 0x7f25b006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:12.072 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:12.069+0000 7f25d19ff640 1 --2- 172.21.15.169:0/3484185391 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f25b006ae70 0x7f25b006d330 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f25bc00da10 tx=0x7f25bc00d3f0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:12.074 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:12.072+0000 7f25c2ffd640 1 -- 172.21.15.169:0/3484185391 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f25c8045020 con 0x7f25b8003970 2023-10-26T01:39:12.303 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:12.301+0000 7f25d4464640 1 -- 172.21.15.169:0/3484185391 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f25cc04db10 con 0x7f25b8003970 2023-10-26T01:39:12.822 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:12.820+0000 7f25c2ffd640 1 -- 172.21.15.169:0/3484185391 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v70) v1 ==== 104+0+0 (secure 0 0 0) 0x7f25c8075820 con 0x7f25b8003970 2023-10-26T01:39:12.826 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:12.824+0000 7f25d4464640 1 -- 172.21.15.169:0/3484185391 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f25b006ae70 msgr2=0x7f25b006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:12.826 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:12.824+0000 7f25d4464640 1 --2- 172.21.15.169:0/3484185391 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f25b006ae70 0x7f25b006d330 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f25bc00da10 tx=0x7f25bc00d3f0 comp rx=0 tx=0).stop 2023-10-26T01:39:12.826 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:12.824+0000 7f25d4464640 1 -- 172.21.15.169:0/3484185391 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25b8003970 msgr2=0x7f25cc06ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:12.827 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:12.824+0000 7f25d4464640 1 --2- 172.21.15.169:0/3484185391 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25b8003970 0x7f25cc06ed30 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f25c802c8e0 tx=0x7f25c8001da0 comp rx=0 tx=0).stop 2023-10-26T01:39:12.827 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:12.824+0000 7f25d4464640 1 -- 172.21.15.169:0/3484185391 shutdown_connections 2023-10-26T01:39:12.827 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:12.824+0000 7f25d4464640 1 --2- 172.21.15.169:0/3484185391 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f25b006ae70 0x7f25b006d330 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:12.827 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:12.824+0000 7f25d4464640 1 --2- 172.21.15.169:0/3484185391 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25cc14f1e0 0x7f25cc14f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:12.827 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:12.824+0000 7f25d4464640 1 --2- 172.21.15.169:0/3484185391 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f25cc1483b0 0x7f25cc14eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:12.828 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:12.824+0000 7f25d4464640 1 --2- 172.21.15.169:0/3484185391 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25b8003970 0x7f25cc06ed30 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:12.828 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:12.824+0000 7f25d4464640 1 -- 172.21.15.169:0/3484185391 >> 172.21.15.169:0/3484185391 conn(0x7f25cc069930 msgr2=0x7f25cc0aceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:12.828 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:12.824+0000 7f25d4464640 1 -- 172.21.15.169:0/3484185391 shutdown_connections 2023-10-26T01:39:12.828 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:12.824+0000 7f25d4464640 1 -- 172.21.15.169:0/3484185391 wait complete. 2023-10-26T01:39:12.847 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-26T01:39:13.077 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:13.075+0000 7f248f52c640 1 -- 172.21.15.169:0/1162768312 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f248806cd90 msgr2=0x7f24880aad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:13.078 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:13.075+0000 7f248f52c640 1 --2- 172.21.15.169:0/1162768312 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f248806cd90 0x7f24880aad40 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f2478009830 tx=0x7f247802bcd0 comp rx=0 tx=0).stop 2023-10-26T01:39:13.078 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:13.075+0000 7f248f52c640 1 -- 172.21.15.169:0/1162768312 shutdown_connections 2023-10-26T01:39:13.078 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:13.075+0000 7f248f52c640 1 --2- 172.21.15.169:0/1162768312 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f24880b01e0 0x7f24880b0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:13.078 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:13.076+0000 7f248f52c640 1 --2- 172.21.15.169:0/1162768312 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f248806cd90 0x7f24880aad40 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:13.079 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:13.076+0000 7f248f52c640 1 -- 172.21.15.169:0/1162768312 >> 172.21.15.169:0/1162768312 conn(0x7f24880aeee0 msgr2=0x7f24880af2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:13.079 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:13.076+0000 7f248f52c640 1 -- 172.21.15.169:0/1162768312 shutdown_connections 2023-10-26T01:39:13.079 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:13.076+0000 7f248f52c640 1 -- 172.21.15.169:0/1162768312 wait complete. 2023-10-26T01:39:13.080 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:13.077+0000 7f248f52c640 1 Processor -- start 2023-10-26T01:39:13.080 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:13.078+0000 7f248f52c640 1 -- start start 2023-10-26T01:39:13.080 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:13.078+0000 7f248f52c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f248806cd90 0x7f248814c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:13.081 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:13.078+0000 7f248f52c640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f24880b01e0 0x7f2488150ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:13.081 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:13.079+0000 7f248f52c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2488158060 0x7f24881513e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:13.081 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:13.079+0000 7f248f52c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f248814d0d0 con 0x7f2488158060 2023-10-26T01:39:13.081 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:13.079+0000 7f248f52c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f248814d3b0 con 0x7f248806cd90 2023-10-26T01:39:13.081 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:13.079+0000 7f248f52c640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f248814d690 con 0x7f24880b01e0 2023-10-26T01:39:13.083 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:13.079+0000 7f248dac9640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2488158060 0x7f24881513e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:13.083 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:13.079+0000 7f248dac9640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2488158060 0x7f24881513e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:39300/0 (socket says 172.21.15.169:39300) 2023-10-26T01:39:13.083 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:13.079+0000 7f248dac9640 1 -- 172.21.15.169:0/14987029 learned_addr learned my addr 172.21.15.169:0/14987029 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:39:13.083 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:13.079+0000 7f248dac9640 1 -- 172.21.15.169:0/14987029 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f24880b01e0 msgr2=0x7f2488150ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:13.084 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:13.079+0000 7f248dac9640 1 --2- 172.21.15.169:0/14987029 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f24880b01e0 0x7f2488150ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:13.084 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:13.079+0000 7f248dac9640 1 -- 172.21.15.169:0/14987029 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f248806cd90 msgr2=0x7f248814c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:13.084 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:13.079+0000 7f248d2c8640 1 --2- 172.21.15.169:0/14987029 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f248806cd90 0x7f248814c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:13.084 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:13.079+0000 7f248dac9640 1 --2- 172.21.15.169:0/14987029 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f248806cd90 0x7f248814c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:13.084 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:13.079+0000 7f248dac9640 1 -- 172.21.15.169:0/14987029 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2478009490 con 0x7f2488158060 2023-10-26T01:39:13.085 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:13.079+0000 7f248d2c8640 1 --2- 172.21.15.169:0/14987029 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f248806cd90 0x7f248814c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:39:13.085 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:13.080+0000 7f248dac9640 1 --2- 172.21.15.169:0/14987029 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2488158060 0x7f24881513e0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f248000eaa0 tx=0x7f248000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:13.085 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:13.080+0000 7f247dffb640 1 -- 172.21.15.169:0/14987029 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f248000cdb0 con 0x7f2488158060 2023-10-26T01:39:13.085 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:13.080+0000 7f247dffb640 1 -- 172.21.15.169:0/14987029 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2480004b20 con 0x7f2488158060 2023-10-26T01:39:13.086 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:13.080+0000 7f247dffb640 1 -- 172.21.15.169:0/14987029 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2480005020 con 0x7f2488158060 2023-10-26T01:39:13.086 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:13.080+0000 7f248f52c640 1 -- 172.21.15.169:0/14987029 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f248814d970 con 0x7f2488158060 2023-10-26T01:39:13.086 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:13.080+0000 7f248f52c640 1 -- 172.21.15.169:0/14987029 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2488151a10 con 0x7f2488158060 2023-10-26T01:39:13.086 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:13.081+0000 7f248f52c640 1 -- 172.21.15.169:0/14987029 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2488066af0 con 0x7f2488158060 2023-10-26T01:39:13.087 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:13.082+0000 7f247dffb640 1 -- 172.21.15.169:0/14987029 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f2480005180 con 0x7f2488158060 2023-10-26T01:39:13.087 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:13.083+0000 7f247dffb640 1 --2- 172.21.15.169:0/14987029 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f246c06ae40 0x7f246c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:13.087 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:13.083+0000 7f247dffb640 1 -- 172.21.15.169:0/14987029 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f248008a100 con 0x7f2488158060 2023-10-26T01:39:13.088 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:13.083+0000 7f247dffb640 1 -- 172.21.15.169:0/14987029 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=142}) v3 -- 0x7f246c072b40 con 0x7f2488158060 2023-10-26T01:39:13.088 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:13.084+0000 7f248d2c8640 1 --2- 172.21.15.169:0/14987029 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f246c06ae40 0x7f246c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:13.088 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:13.085+0000 7f248d2c8640 1 --2- 172.21.15.169:0/14987029 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f246c06ae40 0x7f246c06d300 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f2484001680 tx=0x7f2484009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:13.089 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:13.088+0000 7f247dffb640 1 -- 172.21.15.169:0/14987029 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f248001f020 con 0x7f2488158060 2023-10-26T01:39:13.281 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:13.279+0000 7f248f52c640 1 -- 172.21.15.169:0/14987029 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f248814d210 con 0x7f2488158060 2023-10-26T01:39:14.226 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:14.224+0000 7f247dffb640 1 -- 172.21.15.169:0/14987029 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v142) v1 ==== 181+0+0 (secure 0 0 0) 0x7f248814d210 con 0x7f2488158060 2023-10-26T01:39:14.226 INFO:teuthology.orchestra.run.smithi169.stderr:pool 'cephfs_metadata' removed 2023-10-26T01:39:14.227 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:14.225+0000 7f247dffb640 1 -- 172.21.15.169:0/14987029 <== mon.0 v2:172.21.15.169:3300/0 8 ==== osd_map(142..142 src has 1..142) v4 ==== 354+0+0 (secure 0 0 0) 0x7f2480058dd0 con 0x7f2488158060 2023-10-26T01:39:14.227 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:14.225+0000 7f247dffb640 1 -- 172.21.15.169:0/14987029 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=143}) v3 -- 0x7f246c070f70 con 0x7f2488158060 2023-10-26T01:39:14.232 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:14.230+0000 7f248f52c640 1 -- 172.21.15.169:0/14987029 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f246c06ae40 msgr2=0x7f246c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:14.232 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:14.230+0000 7f248f52c640 1 --2- 172.21.15.169:0/14987029 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f246c06ae40 0x7f246c06d300 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f2484001680 tx=0x7f2484009040 comp rx=0 tx=0).stop 2023-10-26T01:39:14.233 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:14.230+0000 7f248f52c640 1 -- 172.21.15.169:0/14987029 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2488158060 msgr2=0x7f24881513e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:14.233 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:14.230+0000 7f248f52c640 1 --2- 172.21.15.169:0/14987029 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2488158060 0x7f24881513e0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f248000eaa0 tx=0x7f248000ef60 comp rx=0 tx=0).stop 2023-10-26T01:39:14.236 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:14.235+0000 7f248f52c640 1 -- 172.21.15.169:0/14987029 shutdown_connections 2023-10-26T01:39:14.237 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:14.235+0000 7f248f52c640 1 --2- 172.21.15.169:0/14987029 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f246c06ae40 0x7f246c06d300 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:14.237 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:14.235+0000 7f248f52c640 1 --2- 172.21.15.169:0/14987029 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2488158060 0x7f24881513e0 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:14.237 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:14.235+0000 7f248f52c640 1 --2- 172.21.15.169:0/14987029 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f24880b01e0 0x7f2488150ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:14.238 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:14.235+0000 7f248f52c640 1 --2- 172.21.15.169:0/14987029 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f248806cd90 0x7f248814c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:14.238 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:14.235+0000 7f248f52c640 1 -- 172.21.15.169:0/14987029 >> 172.21.15.169:0/14987029 conn(0x7f24880aeee0 msgr2=0x7f2488159b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:14.238 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:14.235+0000 7f248f52c640 1 -- 172.21.15.169:0/14987029 shutdown_connections 2023-10-26T01:39:14.240 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:14.238+0000 7f248f52c640 1 -- 172.21.15.169:0/14987029 wait complete. 2023-10-26T01:39:14.264 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-26T01:39:14.470 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:14.468+0000 7fc86b59e640 1 -- 172.21.15.169:0/3992741680 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc8580051d0 msgr2=0x7fc85800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:14.470 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:14.468+0000 7fc86b59e640 1 --2- 172.21.15.169:0/3992741680 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc8580051d0 0x7fc85800b980 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7fc86c06f7a0 tx=0x7fc8580057f0 comp rx=0 tx=0).stop 2023-10-26T01:39:14.470 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:14.468+0000 7fc86b59e640 1 -- 172.21.15.169:0/3992741680 shutdown_connections 2023-10-26T01:39:14.471 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:14.468+0000 7fc86b59e640 1 --2- 172.21.15.169:0/3992741680 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc8580051d0 0x7fc85800b980 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:14.471 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:14.468+0000 7fc86b59e640 1 -- 172.21.15.169:0/3992741680 >> 172.21.15.169:0/3992741680 conn(0x7fc86c0cf2a0 msgr2=0x7fc86c0cf680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:14.471 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:14.468+0000 7fc86b59e640 1 -- 172.21.15.169:0/3992741680 shutdown_connections 2023-10-26T01:39:14.471 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:14.468+0000 7fc86b59e640 1 -- 172.21.15.169:0/3992741680 wait complete. 2023-10-26T01:39:14.472 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:14.470+0000 7fc86b59e640 1 Processor -- start 2023-10-26T01:39:14.472 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:14.470+0000 7fc86b59e640 1 -- start start 2023-10-26T01:39:14.474 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:14.471+0000 7fc86b59e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc8580051d0 0x7fc86c06bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:14.474 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:14.471+0000 7fc86b59e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc86c0ab8d0 0x7fc86c1f53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:14.475 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:14.471+0000 7fc86b59e640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc86c0abd50 0x7fc86c0ad060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:14.475 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:14.471+0000 7fc86b59e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc86c148190 con 0x7fc8580051d0 2023-10-26T01:39:14.475 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:14.471+0000 7fc86b59e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc86c1f5e40 con 0x7fc86c0ab8d0 2023-10-26T01:39:14.475 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:14.471+0000 7fc86b59e640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc86c1f6120 con 0x7fc86c0abd50 2023-10-26T01:39:14.475 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:14.471+0000 7fc86a59c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc8580051d0 0x7fc86c06bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:14.476 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:14.471+0000 7fc86a59c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc8580051d0 0x7fc86c06bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:53004/0 (socket says 172.21.15.169:53004) 2023-10-26T01:39:14.476 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:14.471+0000 7fc86a59c640 1 -- 172.21.15.169:0/2724514666 learned_addr learned my addr 172.21.15.169:0/2724514666 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:39:14.476 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:14.472+0000 7fc86ad9d640 1 --2- 172.21.15.169:0/2724514666 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc86c0abd50 0x7fc86c0ad060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:14.476 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:14.472+0000 7fc869d9b640 1 --2- 172.21.15.169:0/2724514666 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc86c0ab8d0 0x7fc86c1f53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:14.476 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:14.472+0000 7fc86a59c640 1 -- 172.21.15.169:0/2724514666 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc86c0abd50 msgr2=0x7fc86c0ad060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:14.477 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:14.472+0000 7fc86a59c640 1 --2- 172.21.15.169:0/2724514666 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc86c0abd50 0x7fc86c0ad060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:14.477 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:14.472+0000 7fc86a59c640 1 -- 172.21.15.169:0/2724514666 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc86c0ab8d0 msgr2=0x7fc86c1f53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:14.477 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:14.472+0000 7fc86a59c640 1 --2- 172.21.15.169:0/2724514666 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc86c0ab8d0 0x7fc86c1f53f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:14.477 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:14.472+0000 7fc86a59c640 1 -- 172.21.15.169:0/2724514666 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc86c148450 con 0x7fc8580051d0 2023-10-26T01:39:14.477 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:14.472+0000 7fc869d9b640 1 --2- 172.21.15.169:0/2724514666 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc86c0ab8d0 0x7fc86c1f53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:39:14.478 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:14.472+0000 7fc86a59c640 1 --2- 172.21.15.169:0/2724514666 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc8580051d0 0x7fc86c06bba0 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7fc85c00e590 tx=0x7fc85c00ea50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:14.478 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:14.472+0000 7fc862ffd640 1 -- 172.21.15.169:0/2724514666 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc85c010890 con 0x7fc8580051d0 2023-10-26T01:39:14.478 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:14.472+0000 7fc862ffd640 1 -- 172.21.15.169:0/2724514666 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc85c010b00 con 0x7fc8580051d0 2023-10-26T01:39:14.478 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:14.472+0000 7fc86b59e640 1 -- 172.21.15.169:0/2724514666 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc86c06c0e0 con 0x7fc8580051d0 2023-10-26T01:39:14.478 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:14.472+0000 7fc862ffd640 1 -- 172.21.15.169:0/2724514666 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc85c011460 con 0x7fc8580051d0 2023-10-26T01:39:14.479 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:14.473+0000 7fc86b59e640 1 -- 172.21.15.169:0/2724514666 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc86c1f5930 con 0x7fc8580051d0 2023-10-26T01:39:14.480 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:14.473+0000 7fc86b59e640 1 -- 172.21.15.169:0/2724514666 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc86c066af0 con 0x7fc8580051d0 2023-10-26T01:39:14.480 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:14.474+0000 7fc862ffd640 1 -- 172.21.15.169:0/2724514666 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fc85c0115c0 con 0x7fc8580051d0 2023-10-26T01:39:14.480 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:14.475+0000 7fc862ffd640 1 --2- 172.21.15.169:0/2724514666 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc84c06ae40 0x7fc84c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:14.480 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:14.475+0000 7fc862ffd640 1 -- 172.21.15.169:0/2724514666 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 5415+0+0 (secure 0 0 0) 0x7fc85c089cd0 con 0x7fc8580051d0 2023-10-26T01:39:14.481 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:14.476+0000 7fc862ffd640 1 -- 172.21.15.169:0/2724514666 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=143}) v3 -- 0x7fc84c071cd0 con 0x7fc8580051d0 2023-10-26T01:39:14.481 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:14.476+0000 7fc869d9b640 1 --2- 172.21.15.169:0/2724514666 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc84c06ae40 0x7fc84c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:14.481 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:14.477+0000 7fc869d9b640 1 --2- 172.21.15.169:0/2724514666 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc84c06ae40 0x7fc84c06d300 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fc86c14a0b0 tx=0x7fc854006af0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:14.481 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:14.480+0000 7fc862ffd640 1 -- 172.21.15.169:0/2724514666 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc85c01f070 con 0x7fc8580051d0 2023-10-26T01:39:14.670 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:14.668+0000 7fc86b59e640 1 -- 172.21.15.169:0/2724514666 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fc86c0b2300 con 0x7fc8580051d0 2023-10-26T01:39:15.237 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:15.235+0000 7fc862ffd640 1 -- 172.21.15.169:0/2724514666 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v143) v1 ==== 169+0+0 (secure 0 0 0) 0x7fc85c015030 con 0x7fc8580051d0 2023-10-26T01:39:15.237 INFO:teuthology.orchestra.run.smithi169.stderr:pool 'cephfs_data' removed 2023-10-26T01:39:15.238 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:15.236+0000 7fc862ffd640 1 -- 172.21.15.169:0/2724514666 <== mon.0 v2:172.21.15.169:3300/0 8 ==== osd_map(143..143 src has 1..143) v4 ==== 330+0+0 (secure 0 0 0) 0x7fc85c059250 con 0x7fc8580051d0 2023-10-26T01:39:15.238 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:15.236+0000 7fc862ffd640 1 -- 172.21.15.169:0/2724514666 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=144}) v3 -- 0x7fc84c070a60 con 0x7fc8580051d0 2023-10-26T01:39:15.248 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:15.247+0000 7fc86b59e640 1 -- 172.21.15.169:0/2724514666 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc84c06ae40 msgr2=0x7fc84c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:15.249 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:15.247+0000 7fc86b59e640 1 --2- 172.21.15.169:0/2724514666 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc84c06ae40 0x7fc84c06d300 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fc86c14a0b0 tx=0x7fc854006af0 comp rx=0 tx=0).stop 2023-10-26T01:39:15.249 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:15.247+0000 7fc86b59e640 1 -- 172.21.15.169:0/2724514666 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc8580051d0 msgr2=0x7fc86c06bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:15.249 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:15.247+0000 7fc86b59e640 1 --2- 172.21.15.169:0/2724514666 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc8580051d0 0x7fc86c06bba0 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7fc85c00e590 tx=0x7fc85c00ea50 comp rx=0 tx=0).stop 2023-10-26T01:39:15.249 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:15.247+0000 7fc86b59e640 1 -- 172.21.15.169:0/2724514666 shutdown_connections 2023-10-26T01:39:15.250 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:15.247+0000 7fc86b59e640 1 --2- 172.21.15.169:0/2724514666 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc84c06ae40 0x7fc84c06d300 secure :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fc86c14a0b0 tx=0x7fc854006af0 comp rx=0 tx=0).stop 2023-10-26T01:39:15.250 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:15.247+0000 7fc86b59e640 1 --2- 172.21.15.169:0/2724514666 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc86c0abd50 0x7fc86c0ad060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:15.250 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:15.247+0000 7fc86b59e640 1 --2- 172.21.15.169:0/2724514666 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc86c0ab8d0 0x7fc86c1f53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:15.250 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:15.247+0000 7fc86b59e640 1 --2- 172.21.15.169:0/2724514666 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc8580051d0 0x7fc86c06bba0 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:15.250 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:15.247+0000 7fc86b59e640 1 -- 172.21.15.169:0/2724514666 >> 172.21.15.169:0/2724514666 conn(0x7fc86c0cf2a0 msgr2=0x7fc86c0b06a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:15.251 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:15.250+0000 7fc86b59e640 1 -- 172.21.15.169:0/2724514666 shutdown_connections 2023-10-26T01:39:15.252 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:15.250+0000 7fc86b59e640 1 -- 172.21.15.169:0/2724514666 wait complete. 2023-10-26T01:39:15.273 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 18 2023-10-26T01:39:15.473 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:15.471+0000 7fb1a70a9640 1 -- 172.21.15.169:0/3482074989 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1a006cd60 msgr2=0x7fb1a014ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:15.473 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:15.471+0000 7fb1a70a9640 1 --2- 172.21.15.169:0/3482074989 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1a006cd60 0x7fb1a014ddf0 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7fb198009830 tx=0x7fb19802bc20 comp rx=0 tx=0).stop 2023-10-26T01:39:15.473 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:15.471+0000 7fb1a70a9640 1 -- 172.21.15.169:0/3482074989 shutdown_connections 2023-10-26T01:39:15.473 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:15.471+0000 7fb1a70a9640 1 --2- 172.21.15.169:0/3482074989 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1a006cd60 0x7fb1a014ddf0 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:15.474 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:15.471+0000 7fb1a70a9640 1 --2- 172.21.15.169:0/3482074989 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb1a00aac60 0x7fb1a015b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:15.474 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:15.471+0000 7fb1a70a9640 1 -- 172.21.15.169:0/3482074989 >> 172.21.15.169:0/3482074989 conn(0x7fb1a00aefe0 msgr2=0x7fb1a00af3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:15.474 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:15.471+0000 7fb1a70a9640 1 -- 172.21.15.169:0/3482074989 shutdown_connections 2023-10-26T01:39:15.474 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:15.471+0000 7fb1a70a9640 1 -- 172.21.15.169:0/3482074989 wait complete. 2023-10-26T01:39:15.475 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:15.473+0000 7fb1a70a9640 1 Processor -- start 2023-10-26T01:39:15.475 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:15.473+0000 7fb1a70a9640 1 -- start start 2023-10-26T01:39:15.476 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:15.474+0000 7fb1a70a9640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1a006cd60 0x7fb1a0155300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:15.476 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:15.474+0000 7fb1a70a9640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1a00aac60 0x7fb1a0152970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:15.476 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:15.474+0000 7fb1a70a9640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb1a015b860 0x7fb1a014d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:15.477 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:15.474+0000 7fb1a70a9640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a0154ee0 con 0x7fb1a006cd60 2023-10-26T01:39:15.477 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:15.474+0000 7fb1a70a9640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a01406e0 con 0x7fb1a00aac60 2023-10-26T01:39:15.477 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:15.474+0000 7fb1a70a9640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a0152eb0 con 0x7fb1a015b860 2023-10-26T01:39:15.478 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:15.474+0000 7fb1a4e45640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1a006cd60 0x7fb1a0155300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:15.478 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:15.474+0000 7fb1a4e45640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1a006cd60 0x7fb1a0155300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:53024/0 (socket says 172.21.15.169:53024) 2023-10-26T01:39:15.478 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:15.474+0000 7fb1a4e45640 1 -- 172.21.15.169:0/2989592037 learned_addr learned my addr 172.21.15.169:0/2989592037 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:39:15.479 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:15.474+0000 7fb1a4e45640 1 -- 172.21.15.169:0/2989592037 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb1a015b860 msgr2=0x7fb1a014d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:15.479 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:15.474+0000 7fb1a4e45640 1 --2- 172.21.15.169:0/2989592037 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb1a015b860 0x7fb1a014d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:15.479 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:15.474+0000 7fb1a4e45640 1 -- 172.21.15.169:0/2989592037 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1a00aac60 msgr2=0x7fb1a0152970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:15.479 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:15.474+0000 7fb1a4e45640 1 --2- 172.21.15.169:0/2989592037 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1a00aac60 0x7fb1a0152970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:15.479 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:15.474+0000 7fb1a4e45640 1 -- 172.21.15.169:0/2989592037 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb198009490 con 0x7fb1a006cd60 2023-10-26T01:39:15.480 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:15.475+0000 7fb1a4e45640 1 --2- 172.21.15.169:0/2989592037 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1a006cd60 0x7fb1a0155300 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7fb19c00eab0 tx=0x7fb19c00ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:15.481 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:15.475+0000 7fb1957fa640 1 -- 172.21.15.169:0/2989592037 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb19c00ce40 con 0x7fb1a006cd60 2023-10-26T01:39:15.481 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:15.475+0000 7fb1957fa640 1 -- 172.21.15.169:0/2989592037 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb19c011560 con 0x7fb1a006cd60 2023-10-26T01:39:15.481 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:15.475+0000 7fb1957fa640 1 -- 172.21.15.169:0/2989592037 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb19c011910 con 0x7fb1a006cd60 2023-10-26T01:39:15.481 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:15.475+0000 7fb1a70a9640 1 -- 172.21.15.169:0/2989592037 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1a01ef670 con 0x7fb1a006cd60 2023-10-26T01:39:15.482 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:15.475+0000 7fb1a70a9640 1 -- 172.21.15.169:0/2989592037 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1a0154820 con 0x7fb1a006cd60 2023-10-26T01:39:15.482 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:15.476+0000 7fb1a70a9640 1 -- 172.21.15.169:0/2989592037 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1a0066af0 con 0x7fb1a006cd60 2023-10-26T01:39:15.482 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:15.477+0000 7fb1957fa640 1 -- 172.21.15.169:0/2989592037 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fb19c011a70 con 0x7fb1a006cd60 2023-10-26T01:39:15.482 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:15.478+0000 7fb1957fa640 1 --2- 172.21.15.169:0/2989592037 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb18c06ae40 0x7fb18c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:15.482 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:15.478+0000 7fb1957fa640 1 -- 172.21.15.169:0/2989592037 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fb19c08a090 con 0x7fb1a006cd60 2023-10-26T01:39:15.483 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:15.479+0000 7fb197fff640 1 --2- 172.21.15.169:0/2989592037 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb18c06ae40 0x7fb18c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:15.483 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:15.480+0000 7fb197fff640 1 --2- 172.21.15.169:0/2989592037 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb18c06ae40 0x7fb18c06d300 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fb190001380 tx=0x7fb1900010f0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:15.483 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:15.482+0000 7fb1957fa640 1 -- 172.21.15.169:0/2989592037 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb19c058ec0 con 0x7fb1a006cd60 2023-10-26T01:39:15.671 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:15.670+0000 7fb1a70a9640 1 -- 172.21.15.169:0/2989592037 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "config reset", "num": 18} v 0) v1 -- 0x7fb1a004da70 con 0x7fb1a006cd60 2023-10-26T01:39:15.677 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:15.676+0000 7fb1957fa640 1 -- 172.21.15.169:0/2989592037 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 18}]=0 v32) v1 ==== 71+0+0 (secure 0 0 0) 0x7fb19c058a50 con 0x7fb1a006cd60 2023-10-26T01:39:15.681 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:15.679+0000 7fb1a70a9640 1 -- 172.21.15.169:0/2989592037 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb18c06ae40 msgr2=0x7fb18c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:15.681 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:15.679+0000 7fb1a70a9640 1 --2- 172.21.15.169:0/2989592037 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb18c06ae40 0x7fb18c06d300 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fb190001380 tx=0x7fb1900010f0 comp rx=0 tx=0).stop 2023-10-26T01:39:15.681 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:15.679+0000 7fb1a70a9640 1 -- 172.21.15.169:0/2989592037 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1a006cd60 msgr2=0x7fb1a0155300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:15.682 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:15.679+0000 7fb1a70a9640 1 --2- 172.21.15.169:0/2989592037 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1a006cd60 0x7fb1a0155300 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7fb19c00eab0 tx=0x7fb19c00ef70 comp rx=0 tx=0).stop 2023-10-26T01:39:15.682 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:15.679+0000 7fb1a70a9640 1 -- 172.21.15.169:0/2989592037 shutdown_connections 2023-10-26T01:39:15.682 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:15.679+0000 7fb1a70a9640 1 --2- 172.21.15.169:0/2989592037 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb18c06ae40 0x7fb18c06d300 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:15.682 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:15.679+0000 7fb1a70a9640 1 --2- 172.21.15.169:0/2989592037 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb1a015b860 0x7fb1a014d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:15.683 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:15.679+0000 7fb1a70a9640 1 --2- 172.21.15.169:0/2989592037 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1a00aac60 0x7fb1a0152970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:15.683 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:15.679+0000 7fb1a70a9640 1 --2- 172.21.15.169:0/2989592037 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1a006cd60 0x7fb1a0155300 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:15.683 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:15.679+0000 7fb1a70a9640 1 -- 172.21.15.169:0/2989592037 >> 172.21.15.169:0/2989592037 conn(0x7fb1a00aefe0 msgr2=0x7fb1a006c650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:15.683 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:15.679+0000 7fb1a70a9640 1 -- 172.21.15.169:0/2989592037 shutdown_connections 2023-10-26T01:39:15.683 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:15.679+0000 7fb1a70a9640 1 -- 172.21.15.169:0/2989592037 wait complete. 2023-10-26T01:39:15.700 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_different_file' 2023-10-26T01:39:15.920 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:15.918+0000 7f29eccc9640 1 -- 172.21.15.169:0/2908609591 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29e80ac4e0 msgr2=0x7f29e80cd670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:15.920 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:15.918+0000 7f29eccc9640 1 --2- 172.21.15.169:0/2908609591 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29e80ac4e0 0x7f29e80cd670 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f29d0009830 tx=0x7f29d002bd50 comp rx=0 tx=0).stop 2023-10-26T01:39:15.920 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:15.918+0000 7f29eccc9640 1 -- 172.21.15.169:0/2908609591 shutdown_connections 2023-10-26T01:39:15.921 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:15.918+0000 7f29eccc9640 1 --2- 172.21.15.169:0/2908609591 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29e80ac4e0 0x7f29e80cd670 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:15.921 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:15.918+0000 7f29eccc9640 1 --2- 172.21.15.169:0/2908609591 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f29e815b610 0x7f29e806ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:15.921 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:15.918+0000 7f29eccc9640 1 -- 172.21.15.169:0/2908609591 >> 172.21.15.169:0/2908609591 conn(0x7f29e8069930 msgr2=0x7f29e8069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:15.921 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:15.918+0000 7f29eccc9640 1 -- 172.21.15.169:0/2908609591 shutdown_connections 2023-10-26T01:39:15.921 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:15.918+0000 7f29eccc9640 1 -- 172.21.15.169:0/2908609591 wait complete. 2023-10-26T01:39:15.923 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:15.920+0000 7f29eccc9640 1 Processor -- start 2023-10-26T01:39:15.923 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:15.920+0000 7f29eccc9640 1 -- start start 2023-10-26T01:39:15.923 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:15.921+0000 7f29eccc9640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f29e80ac4e0 0x7f29e8146550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:15.924 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:15.921+0000 7f29eccc9640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29e80ac940 0x7f29e80ae080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:15.924 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:15.921+0000 7f29eccc9640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f29e815b610 0x7f29e80ae5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:15.924 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:15.921+0000 7f29eccc9640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29e81f5e20 con 0x7f29e80ac940 2023-10-26T01:39:15.924 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:15.921+0000 7f29eccc9640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29e81f6240 con 0x7f29e815b610 2023-10-26T01:39:15.924 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:15.921+0000 7f29eccc9640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29e8070ce0 con 0x7f29e80ac4e0 2023-10-26T01:39:15.925 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:15.921+0000 7f29dffff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29e80ac940 0x7f29e80ae080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:15.925 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:15.921+0000 7f29dffff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29e80ac940 0x7f29e80ae080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:53040/0 (socket says 172.21.15.169:53040) 2023-10-26T01:39:15.925 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:15.921+0000 7f29dffff640 1 -- 172.21.15.169:0/1365602416 learned_addr learned my addr 172.21.15.169:0/1365602416 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:39:15.925 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:15.921+0000 7f29dffff640 1 -- 172.21.15.169:0/1365602416 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f29e80ac4e0 msgr2=0x7f29e8146550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:15.925 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:15.921+0000 7f29dffff640 1 --2- 172.21.15.169:0/1365602416 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f29e80ac4e0 0x7f29e8146550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:15.925 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:15.921+0000 7f29dffff640 1 -- 172.21.15.169:0/1365602416 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f29e815b610 msgr2=0x7f29e80ae5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:15.926 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:15.921+0000 7f29dffff640 1 --2- 172.21.15.169:0/1365602416 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f29e815b610 0x7f29e80ae5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:15.926 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:15.921+0000 7f29dffff640 1 -- 172.21.15.169:0/1365602416 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29d0009490 con 0x7f29e80ac940 2023-10-26T01:39:15.926 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:15.922+0000 7f29dffff640 1 --2- 172.21.15.169:0/1365602416 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29e80ac940 0x7f29e80ae080 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f29d0009780 tx=0x7f29d00038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:15.927 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:15.922+0000 7f29deffd640 1 -- 172.21.15.169:0/1365602416 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d0003ea0 con 0x7f29e80ac940 2023-10-26T01:39:15.927 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:15.922+0000 7f29deffd640 1 -- 172.21.15.169:0/1365602416 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f29d0034440 con 0x7f29e80ac940 2023-10-26T01:39:15.928 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:15.922+0000 7f29deffd640 1 -- 172.21.15.169:0/1365602416 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d0034850 con 0x7f29e80ac940 2023-10-26T01:39:15.928 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:15.922+0000 7f29eccc9640 1 -- 172.21.15.169:0/1365602416 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29e81f5480 con 0x7f29e80ac940 2023-10-26T01:39:15.928 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:15.922+0000 7f29eccc9640 1 -- 172.21.15.169:0/1365602416 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29e81f5a40 con 0x7f29e80ac940 2023-10-26T01:39:15.928 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:15.923+0000 7f29eccc9640 1 -- 172.21.15.169:0/1365602416 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29e80cd9a0 con 0x7f29e80ac940 2023-10-26T01:39:15.928 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:15.924+0000 7f29deffd640 1 -- 172.21.15.169:0/1365602416 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f29d003b040 con 0x7f29e80ac940 2023-10-26T01:39:15.929 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:15.925+0000 7f29deffd640 1 --2- 172.21.15.169:0/1365602416 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f29c806ae40 0x7f29c806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:15.929 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:15.926+0000 7f29deffd640 1 -- 172.21.15.169:0/1365602416 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f29d00ab770 con 0x7f29e80ac940 2023-10-26T01:39:15.929 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:15.926+0000 7f29e659c640 1 --2- 172.21.15.169:0/1365602416 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f29c806ae40 0x7f29c806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:15.930 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:15.927+0000 7f29e659c640 1 --2- 172.21.15.169:0/1365602416 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f29c806ae40 0x7f29c806d300 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f29d80058c0 tx=0x7f29d8005850 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:15.932 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:15.930+0000 7f29deffd640 1 -- 172.21.15.169:0/1365602416 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f29d0076bb0 con 0x7f29e80ac940 2023-10-26T01:39:16.191 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:16.189+0000 7f29eccc9640 1 -- 172.21.15.169:0/1365602416 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_different_file"]} v 0) v1 -- 0x7f29e81f5f60 con 0x7f29e80ac940 2023-10-26T01:39:16.239 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:16.238+0000 7f29deffd640 1 -- 172.21.15.169:0/1365602416 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_different_file"]}]=0 v447) v1 ==== 140+0+0 (secure 0 0 0) 0x7f29e81f5f60 con 0x7f29e80ac940 2023-10-26T01:39:16.243 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:16.240+0000 7f29eccc9640 1 -- 172.21.15.169:0/1365602416 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f29c806ae40 msgr2=0x7f29c806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:16.243 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:16.240+0000 7f29eccc9640 1 --2- 172.21.15.169:0/1365602416 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f29c806ae40 0x7f29c806d300 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f29d80058c0 tx=0x7f29d8005850 comp rx=0 tx=0).stop 2023-10-26T01:39:16.243 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:16.240+0000 7f29eccc9640 1 -- 172.21.15.169:0/1365602416 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29e80ac940 msgr2=0x7f29e80ae080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:16.243 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:16.240+0000 7f29eccc9640 1 --2- 172.21.15.169:0/1365602416 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29e80ac940 0x7f29e80ae080 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f29d0009780 tx=0x7f29d00038a0 comp rx=0 tx=0).stop 2023-10-26T01:39:16.244 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:16.241+0000 7f29eccc9640 1 -- 172.21.15.169:0/1365602416 shutdown_connections 2023-10-26T01:39:16.244 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:16.241+0000 7f29eccc9640 1 --2- 172.21.15.169:0/1365602416 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f29c806ae40 0x7f29c806d300 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:16.244 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:16.241+0000 7f29eccc9640 1 --2- 172.21.15.169:0/1365602416 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29e80ac940 0x7f29e80ae080 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:16.244 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:16.241+0000 7f29eccc9640 1 --2- 172.21.15.169:0/1365602416 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f29e815b610 0x7f29e80ae5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:16.244 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:16.241+0000 7f29eccc9640 1 --2- 172.21.15.169:0/1365602416 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f29e80ac4e0 0x7f29e8146550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:16.245 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:16.241+0000 7f29eccc9640 1 -- 172.21.15.169:0/1365602416 >> 172.21.15.169:0/1365602416 conn(0x7f29e8069930 msgr2=0x7f29e80cc7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:16.245 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:16.241+0000 7f29eccc9640 1 -- 172.21.15.169:0/1365602416 shutdown_connections 2023-10-26T01:39:16.245 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:16.241+0000 7f29eccc9640 1 -- 172.21.15.169:0/1365602416 wait complete. 2023-10-26T01:39:16.261 INFO:tasks.cephfs_test_runner:test_full_different_file (tasks.cephfs.test_full.TestQuotaFull) ... ok 2023-10-26T01:39:16.262 INFO:tasks.cephfs_test_runner:Starting test: test_full_fclose (tasks.cephfs.test_full.TestQuotaFull) 2023-10-26T01:39:16.262 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:39:16.482 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:16.480+0000 7fd468039640 1 -- 172.21.15.169:0/919273293 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4600cb200 msgr2=0x7fd46006e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:16.482 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:16.480+0000 7fd468039640 1 --2- 172.21.15.169:0/919273293 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4600cb200 0x7fd46006e510 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7fd46006fb50 tx=0x7fd45002c260 comp rx=0 tx=0).stop 2023-10-26T01:39:16.483 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:16.480+0000 7fd468039640 1 -- 172.21.15.169:0/919273293 shutdown_connections 2023-10-26T01:39:16.483 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:16.480+0000 7fd468039640 1 --2- 172.21.15.169:0/919273293 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd460159b50 0x7fd4600cb5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:16.483 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:16.480+0000 7fd468039640 1 --2- 172.21.15.169:0/919273293 >> v2:172.21.15.170:3300/0 conn(0x7fd46006c010 0x7fd4600cc3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:16.483 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:16.480+0000 7fd468039640 1 --2- 172.21.15.169:0/919273293 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4600cb200 0x7fd46006e510 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:16.483 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:16.480+0000 7fd468039640 1 -- 172.21.15.169:0/919273293 >> 172.21.15.169:0/919273293 conn(0x7fd4600b0690 msgr2=0x7fd4600b0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:16.484 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:16.480+0000 7fd468039640 1 -- 172.21.15.169:0/919273293 shutdown_connections 2023-10-26T01:39:16.484 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:16.480+0000 7fd468039640 1 -- 172.21.15.169:0/919273293 wait complete. 2023-10-26T01:39:16.484 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:16.482+0000 7fd468039640 1 Processor -- start 2023-10-26T01:39:16.485 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:16.483+0000 7fd468039640 1 -- start start 2023-10-26T01:39:16.487 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:16.484+0000 7fd468039640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd46006c010 0x7fd460147fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:16.487 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:16.484+0000 7fd468039640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd4600cb200 0x7fd4600b30d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:16.488 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:16.484+0000 7fd468039640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd460159b50 0x7fd460158620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:16.488 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:16.484+0000 7fd468039640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd460148510 con 0x7fd460159b50 2023-10-26T01:39:16.488 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:16.484+0000 7fd468039640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4600b2030 con 0x7fd46006c010 2023-10-26T01:39:16.488 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:16.484+0000 7fd468039640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4600b2310 con 0x7fd4600cb200 2023-10-26T01:39:16.489 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:16.484+0000 7fd4665d6640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd460159b50 0x7fd460158620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:16.489 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:16.484+0000 7fd4665d6640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd460159b50 0x7fd460158620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:53054/0 (socket says 172.21.15.169:53054) 2023-10-26T01:39:16.489 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:16.484+0000 7fd4665d6640 1 -- 172.21.15.169:0/2499842074 learned_addr learned my addr 172.21.15.169:0/2499842074 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:39:16.490 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:16.484+0000 7fd465dd5640 1 --2- 172.21.15.169:0/2499842074 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd46006c010 0x7fd460147fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:16.490 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:16.485+0000 7fd4655d4640 1 --2- 172.21.15.169:0/2499842074 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd4600cb200 0x7fd4600b30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:16.490 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:16.485+0000 7fd4665d6640 1 -- 172.21.15.169:0/2499842074 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd4600cb200 msgr2=0x7fd4600b30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:16.490 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:16.485+0000 7fd4665d6640 1 --2- 172.21.15.169:0/2499842074 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd4600cb200 0x7fd4600b30d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:16.491 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:16.485+0000 7fd4665d6640 1 -- 172.21.15.169:0/2499842074 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd46006c010 msgr2=0x7fd460147fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:16.491 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:16.485+0000 7fd4665d6640 1 --2- 172.21.15.169:0/2499842074 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd46006c010 0x7fd460147fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:16.491 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:16.485+0000 7fd4665d6640 1 -- 172.21.15.169:0/2499842074 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd450009900 con 0x7fd460159b50 2023-10-26T01:39:16.492 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:16.485+0000 7fd4655d4640 1 --2- 172.21.15.169:0/2499842074 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd4600cb200 0x7fd4600b30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:39:16.492 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:16.485+0000 7fd4665d6640 1 --2- 172.21.15.169:0/2499842074 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd460159b50 0x7fd460158620 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7fd45800c740 tx=0x7fd45800cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:16.492 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:16.485+0000 7fd45e7fc640 1 -- 172.21.15.169:0/2499842074 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd458007bb0 con 0x7fd460159b50 2023-10-26T01:39:16.493 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:16.485+0000 7fd45e7fc640 1 -- 172.21.15.169:0/2499842074 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd458007e00 con 0x7fd460159b50 2023-10-26T01:39:16.493 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:16.485+0000 7fd45e7fc640 1 -- 172.21.15.169:0/2499842074 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd458005280 con 0x7fd460159b50 2023-10-26T01:39:16.493 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:16.485+0000 7fd468039640 1 -- 172.21.15.169:0/2499842074 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4600b25f0 con 0x7fd460159b50 2023-10-26T01:39:16.494 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:16.486+0000 7fd468039640 1 -- 172.21.15.169:0/2499842074 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4600b2bb0 con 0x7fd460159b50 2023-10-26T01:39:16.496 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:16.487+0000 7fd468039640 1 -- 172.21.15.169:0/2499842074 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd460072ec0 con 0x7fd460159b50 2023-10-26T01:39:16.496 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:16.488+0000 7fd45e7fc640 1 -- 172.21.15.169:0/2499842074 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fd4580053e0 con 0x7fd460159b50 2023-10-26T01:39:16.496 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:16.489+0000 7fd45e7fc640 1 --2- 172.21.15.169:0/2499842074 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd44c06ae40 0x7fd44c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:16.497 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:16.489+0000 7fd45e7fc640 1 -- 172.21.15.169:0/2499842074 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fd458089780 con 0x7fd460159b50 2023-10-26T01:39:16.497 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:16.489+0000 7fd465dd5640 1 --2- 172.21.15.169:0/2499842074 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd44c06ae40 0x7fd44c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:16.497 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:16.491+0000 7fd465dd5640 1 --2- 172.21.15.169:0/2499842074 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd44c06ae40 0x7fd44c06d300 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fd450036000 tx=0x7fd450004ab0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:16.498 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:16.493+0000 7fd45e7fc640 1 -- 172.21.15.169:0/2499842074 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd458054a90 con 0x7fd460159b50 2023-10-26T01:39:16.685 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:16.683+0000 7fd468039640 1 -- 172.21.15.169:0/2499842074 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd4600b2dd0 con 0x7fd460159b50 2023-10-26T01:39:16.687 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:16.685+0000 7fd45e7fc640 1 -- 172.21.15.169:0/2499842074 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v143) v1 ==== 74+0+13047 (secure 0 0 0) 0x7fd45801d020 con 0x7fd460159b50 2023-10-26T01:39:16.687 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:39:16.688 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":143,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:39:15.225036+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.170:6837/3615088922":"2023-10-27T01:39:11.814196+0000","172.21.15.170:6835/3615088922":"2023-10-27T01:39:11.814196+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:39:16.689 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:16.687+0000 7fd468039640 1 -- 172.21.15.169:0/2499842074 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd44c06ae40 msgr2=0x7fd44c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:16.690 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:16.687+0000 7fd468039640 1 --2- 172.21.15.169:0/2499842074 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd44c06ae40 0x7fd44c06d300 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fd450036000 tx=0x7fd450004ab0 comp rx=0 tx=0).stop 2023-10-26T01:39:16.690 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:16.688+0000 7fd468039640 1 -- 172.21.15.169:0/2499842074 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd460159b50 msgr2=0x7fd460158620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:16.690 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:16.688+0000 7fd468039640 1 --2- 172.21.15.169:0/2499842074 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd460159b50 0x7fd460158620 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7fd45800c740 tx=0x7fd45800cc00 comp rx=0 tx=0).stop 2023-10-26T01:39:16.691 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:16.688+0000 7fd468039640 1 -- 172.21.15.169:0/2499842074 shutdown_connections 2023-10-26T01:39:16.691 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:16.688+0000 7fd468039640 1 --2- 172.21.15.169:0/2499842074 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd44c06ae40 0x7fd44c06d300 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:16.691 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:16.688+0000 7fd468039640 1 --2- 172.21.15.169:0/2499842074 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd460159b50 0x7fd460158620 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:16.691 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:16.688+0000 7fd468039640 1 --2- 172.21.15.169:0/2499842074 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd4600cb200 0x7fd4600b30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:16.691 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:16.688+0000 7fd468039640 1 --2- 172.21.15.169:0/2499842074 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd46006c010 0x7fd460147fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:16.692 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:16.688+0000 7fd468039640 1 -- 172.21.15.169:0/2499842074 >> 172.21.15.169:0/2499842074 conn(0x7fd4600b0690 msgr2=0x7fd4600ccb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:16.692 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:16.688+0000 7fd468039640 1 -- 172.21.15.169:0/2499842074 shutdown_connections 2023-10-26T01:39:16.692 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:16.688+0000 7fd468039640 1 -- 172.21.15.169:0/2499842074 wait complete. 2023-10-26T01:39:16.709 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T01:30:32.200808+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '109', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T01:39:16.710 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T01:39:16.933 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:16.931+0000 7f66a736e640 1 -- 172.21.15.169:0/1309555857 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66a0159480 msgr2=0x7f66a00b02e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:16.933 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:16.931+0000 7f66a736e640 1 --2- 172.21.15.169:0/1309555857 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66a0159480 0x7f66a00b02e0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f669c005fd0 tx=0x7f669c02be20 comp rx=0 tx=0).stop 2023-10-26T01:39:16.933 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:16.931+0000 7f66a736e640 1 -- 172.21.15.169:0/1309555857 shutdown_connections 2023-10-26T01:39:16.934 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:16.931+0000 7f66a736e640 1 --2- 172.21.15.169:0/1309555857 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f66a00b0820 0x7f66a00aaf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:16.934 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:16.931+0000 7f66a736e640 1 --2- 172.21.15.169:0/1309555857 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66a0159480 0x7f66a00b02e0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:16.934 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:16.931+0000 7f66a736e640 1 -- 172.21.15.169:0/1309555857 >> 172.21.15.169:0/1309555857 conn(0x7f66a0069930 msgr2=0x7f66a0069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:16.934 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:16.931+0000 7f66a736e640 1 -- 172.21.15.169:0/1309555857 shutdown_connections 2023-10-26T01:39:16.934 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:16.931+0000 7f66a736e640 1 -- 172.21.15.169:0/1309555857 wait complete. 2023-10-26T01:39:16.936 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:16.933+0000 7f66a736e640 1 Processor -- start 2023-10-26T01:39:16.936 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:16.933+0000 7f66a736e640 1 -- start start 2023-10-26T01:39:16.936 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:16.934+0000 7f66a736e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66a00ab4c0 0x7f66a0098160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:16.936 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:16.934+0000 7f66a736e640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f66a00b0820 0x7f66a014a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:16.936 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:16.934+0000 7f66a736e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66a0159480 0x7f66a009a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:16.936 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:16.934+0000 7f66a736e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66a01efaa0 con 0x7f66a0159480 2023-10-26T01:39:16.937 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:16.934+0000 7f66a736e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66a009b350 con 0x7f66a00ab4c0 2023-10-26T01:39:16.937 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:16.934+0000 7f66a736e640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66a009ad90 con 0x7f66a00b0820 2023-10-26T01:39:16.937 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:16.934+0000 7f66a590b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66a0159480 0x7f66a009a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:16.937 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:16.934+0000 7f66a590b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66a0159480 0x7f66a009a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:53070/0 (socket says 172.21.15.169:53070) 2023-10-26T01:39:16.937 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:16.934+0000 7f66a590b640 1 -- 172.21.15.169:0/1990283616 learned_addr learned my addr 172.21.15.169:0/1990283616 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:39:16.939 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:16.934+0000 7f66a510a640 1 --2- 172.21.15.169:0/1990283616 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66a00ab4c0 0x7f66a0098160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:16.939 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:16.935+0000 7f66a4909640 1 --2- 172.21.15.169:0/1990283616 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f66a00b0820 0x7f66a014a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:16.939 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:16.935+0000 7f66a590b640 1 -- 172.21.15.169:0/1990283616 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f66a00b0820 msgr2=0x7f66a014a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:16.939 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:16.935+0000 7f66a590b640 1 --2- 172.21.15.169:0/1990283616 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f66a00b0820 0x7f66a014a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:16.939 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:16.935+0000 7f66a590b640 1 -- 172.21.15.169:0/1990283616 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66a00ab4c0 msgr2=0x7f66a0098160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:16.940 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:16.935+0000 7f66a590b640 1 --2- 172.21.15.169:0/1990283616 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66a00ab4c0 0x7f66a0098160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:16.940 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:16.935+0000 7f66a590b640 1 -- 172.21.15.169:0/1990283616 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6690009490 con 0x7f66a0159480 2023-10-26T01:39:16.940 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:16.935+0000 7f66a590b640 1 --2- 172.21.15.169:0/1990283616 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66a0159480 0x7f66a009a510 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f669800c7a0 tx=0x7f669800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:16.940 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:16.935+0000 7f6695ffb640 1 -- 172.21.15.169:0/1990283616 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6698003b70 con 0x7f66a0159480 2023-10-26T01:39:16.940 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:16.935+0000 7f6695ffb640 1 -- 172.21.15.169:0/1990283616 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6698003cf0 con 0x7f66a0159480 2023-10-26T01:39:16.940 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:16.935+0000 7f6695ffb640 1 -- 172.21.15.169:0/1990283616 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f669800aab0 con 0x7f66a0159480 2023-10-26T01:39:16.941 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:16.935+0000 7f66a736e640 1 -- 172.21.15.169:0/1990283616 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f669c009560 con 0x7f66a0159480 2023-10-26T01:39:16.941 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:16.936+0000 7f66a736e640 1 -- 172.21.15.169:0/1990283616 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66a014c4e0 con 0x7f66a0159480 2023-10-26T01:39:16.941 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:16.937+0000 7f66a736e640 1 -- 172.21.15.169:0/1990283616 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66a0066af0 con 0x7f66a0159480 2023-10-26T01:39:16.942 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:16.938+0000 7f6695ffb640 1 -- 172.21.15.169:0/1990283616 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f669800ac10 con 0x7f66a0159480 2023-10-26T01:39:16.942 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:16.938+0000 7f6695ffb640 1 --2- 172.21.15.169:0/1990283616 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f668406ae40 0x7f668406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:16.942 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:16.939+0000 7f6695ffb640 1 -- 172.21.15.169:0/1990283616 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f66980892a0 con 0x7f66a0159480 2023-10-26T01:39:16.942 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:16.939+0000 7f66a510a640 1 --2- 172.21.15.169:0/1990283616 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f668406ae40 0x7f668406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:16.943 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:16.940+0000 7f66a510a640 1 --2- 172.21.15.169:0/1990283616 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f668406ae40 0x7f668406d300 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f669c035000 tx=0x7f669c004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:16.944 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:16.943+0000 7f6695ffb640 1 -- 172.21.15.169:0/1990283616 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6698054660 con 0x7f66a0159480 2023-10-26T01:39:17.135 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:17.133+0000 7f66a736e640 1 -- 172.21.15.169:0/1990283616 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f66a004da70 con 0x7f66a0159480 2023-10-26T01:39:17.136 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:17.134+0000 7f6695ffb640 1 -- 172.21.15.169:0/1990283616 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v143) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6698057c80 con 0x7f66a0159480 2023-10-26T01:39:17.136 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 1 2023-10-26T01:39:17.139 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:17.137+0000 7f66a736e640 1 -- 172.21.15.169:0/1990283616 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f668406ae40 msgr2=0x7f668406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:17.139 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:17.137+0000 7f66a736e640 1 --2- 172.21.15.169:0/1990283616 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f668406ae40 0x7f668406d300 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f669c035000 tx=0x7f669c004730 comp rx=0 tx=0).stop 2023-10-26T01:39:17.140 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:17.137+0000 7f66a736e640 1 -- 172.21.15.169:0/1990283616 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66a0159480 msgr2=0x7f66a009a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:17.140 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:17.137+0000 7f66a736e640 1 --2- 172.21.15.169:0/1990283616 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66a0159480 0x7f66a009a510 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f669800c7a0 tx=0x7f669800cc60 comp rx=0 tx=0).stop 2023-10-26T01:39:17.140 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:17.137+0000 7f66a736e640 1 -- 172.21.15.169:0/1990283616 shutdown_connections 2023-10-26T01:39:17.140 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:17.137+0000 7f66a736e640 1 --2- 172.21.15.169:0/1990283616 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f668406ae40 0x7f668406d300 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:17.140 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:17.137+0000 7f66a736e640 1 --2- 172.21.15.169:0/1990283616 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66a0159480 0x7f66a009a510 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:17.141 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:17.137+0000 7f66a736e640 1 --2- 172.21.15.169:0/1990283616 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f66a00b0820 0x7f66a014a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:17.141 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:17.137+0000 7f66a736e640 1 --2- 172.21.15.169:0/1990283616 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66a00ab4c0 0x7f66a0098160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:17.141 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:17.137+0000 7f66a736e640 1 -- 172.21.15.169:0/1990283616 >> 172.21.15.169:0/1990283616 conn(0x7f66a0069930 msgr2=0x7f66a00aef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:17.141 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:17.137+0000 7f66a736e640 1 -- 172.21.15.169:0/1990283616 shutdown_connections 2023-10-26T01:39:17.141 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:17.138+0000 7f66a736e640 1 -- 172.21.15.169:0/1990283616 wait complete. 2023-10-26T01:39:17.157 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_fclose' 2023-10-26T01:39:17.372 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:17.370+0000 7f7b55cb4640 1 -- 172.21.15.169:0/1041755901 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7b500cde00 msgr2=0x7f7b500cbac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:17.372 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:17.370+0000 7f7b55cb4640 1 --2- 172.21.15.169:0/1041755901 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7b500cde00 0x7f7b500cbac0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7b48005fd0 tx=0x7f7b4802c050 comp rx=0 tx=0).stop 2023-10-26T01:39:17.372 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:17.370+0000 7f7b55cb4640 1 -- 172.21.15.169:0/1041755901 shutdown_connections 2023-10-26T01:39:17.373 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:17.370+0000 7f7b55cb4640 1 --2- 172.21.15.169:0/1041755901 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7b500cde00 0x7f7b500cbac0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:17.373 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:17.370+0000 7f7b55cb4640 1 --2- 172.21.15.169:0/1041755901 >> v2:172.21.15.170:3300/0 conn(0x7f7b501592d0 0x7f7b500aa3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:17.373 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:17.370+0000 7f7b55cb4640 1 -- 172.21.15.169:0/1041755901 >> 172.21.15.169:0/1041755901 conn(0x7f7b50069930 msgr2=0x7f7b50069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:17.373 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:17.370+0000 7f7b55cb4640 1 -- 172.21.15.169:0/1041755901 shutdown_connections 2023-10-26T01:39:17.373 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:17.370+0000 7f7b55cb4640 1 -- 172.21.15.169:0/1041755901 wait complete. 2023-10-26T01:39:17.374 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:17.372+0000 7f7b55cb4640 1 Processor -- start 2023-10-26T01:39:17.374 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:17.372+0000 7f7b55cb4640 1 -- start start 2023-10-26T01:39:17.375 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:17.373+0000 7f7b55cb4640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b500ac930 0x7f7b50150480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:17.375 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:17.373+0000 7f7b55cb4640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b500cde00 0x7f7b5015bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:17.376 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:17.373+0000 7f7b55cb4640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7b501592d0 0x7f7b5014b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:17.376 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:17.373+0000 7f7b55cb4640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b5015a230 con 0x7f7b500cde00 2023-10-26T01:39:17.376 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:17.373+0000 7f7b55cb4640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b50159940 con 0x7f7b500ac930 2023-10-26T01:39:17.376 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:17.373+0000 7f7b55cb4640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b50159be0 con 0x7f7b501592d0 2023-10-26T01:39:17.376 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:17.374+0000 7f7b4effd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b500cde00 0x7f7b5015bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:17.377 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:17.374+0000 7f7b4effd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b500cde00 0x7f7b5015bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:53076/0 (socket says 172.21.15.169:53076) 2023-10-26T01:39:17.377 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:17.374+0000 7f7b4effd640 1 -- 172.21.15.169:0/1481871543 learned_addr learned my addr 172.21.15.169:0/1481871543 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:39:17.378 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:17.374+0000 7f7b4f7fe640 1 --2- 172.21.15.169:0/1481871543 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b500ac930 0x7f7b50150480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:17.378 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:17.374+0000 7f7b4ffff640 1 --2- 172.21.15.169:0/1481871543 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7b501592d0 0x7f7b5014b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:17.379 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:17.374+0000 7f7b4effd640 1 -- 172.21.15.169:0/1481871543 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7b501592d0 msgr2=0x7f7b5014b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:17.379 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:17.374+0000 7f7b4effd640 1 --2- 172.21.15.169:0/1481871543 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7b501592d0 0x7f7b5014b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:17.379 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:17.374+0000 7f7b4effd640 1 -- 172.21.15.169:0/1481871543 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b500ac930 msgr2=0x7f7b50150480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:17.379 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:17.374+0000 7f7b4effd640 1 --2- 172.21.15.169:0/1481871543 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b500ac930 0x7f7b50150480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:17.379 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:17.374+0000 7f7b4effd640 1 -- 172.21.15.169:0/1481871543 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b40009560 con 0x7f7b500cde00 2023-10-26T01:39:17.380 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:17.375+0000 7f7b4effd640 1 --2- 172.21.15.169:0/1481871543 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b500cde00 0x7f7b5015bf10 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f7b3800c500 tx=0x7f7b3800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:17.380 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:17.375+0000 7f7b47fff640 1 -- 172.21.15.169:0/1481871543 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b380052d0 con 0x7f7b500cde00 2023-10-26T01:39:17.380 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:17.375+0000 7f7b47fff640 1 -- 172.21.15.169:0/1481871543 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7b38005450 con 0x7f7b500cde00 2023-10-26T01:39:17.380 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:17.375+0000 7f7b55cb4640 1 -- 172.21.15.169:0/1481871543 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b48009840 con 0x7f7b500cde00 2023-10-26T01:39:17.380 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:17.375+0000 7f7b47fff640 1 -- 172.21.15.169:0/1481871543 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b38012890 con 0x7f7b500cde00 2023-10-26T01:39:17.380 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:17.375+0000 7f7b55cb4640 1 -- 172.21.15.169:0/1481871543 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b5015a6c0 con 0x7f7b500cde00 2023-10-26T01:39:17.381 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:17.376+0000 7f7b55cb4640 1 -- 172.21.15.169:0/1481871543 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b5015a3e0 con 0x7f7b500cde00 2023-10-26T01:39:17.382 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:17.377+0000 7f7b47fff640 1 -- 172.21.15.169:0/1481871543 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f7b380035f0 con 0x7f7b500cde00 2023-10-26T01:39:17.382 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:17.378+0000 7f7b47fff640 1 --2- 172.21.15.169:0/1481871543 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7b3006ae40 0x7f7b3006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:17.382 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:17.378+0000 7f7b47fff640 1 -- 172.21.15.169:0/1481871543 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f7b38089030 con 0x7f7b500cde00 2023-10-26T01:39:17.382 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:17.379+0000 7f7b4f7fe640 1 --2- 172.21.15.169:0/1481871543 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7b3006ae40 0x7f7b3006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:17.382 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:17.380+0000 7f7b4f7fe640 1 --2- 172.21.15.169:0/1481871543 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7b3006ae40 0x7f7b3006d300 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f7b40005fd0 tx=0x7f7b40009260 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:17.385 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:17.383+0000 7f7b47fff640 1 -- 172.21.15.169:0/1481871543 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7b5015a3e0 con 0x7f7b500cde00 2023-10-26T01:39:17.644 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:17.642+0000 7f7b55cb4640 1 -- 172.21.15.169:0/1481871543 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_fclose"]} v 0) v1 -- 0x7f7b5004db10 con 0x7f7b500cde00 2023-10-26T01:39:18.251 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:18.249+0000 7f7b47fff640 1 -- 172.21.15.169:0/1481871543 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_fclose"]}]=0 v449) v1 ==== 135+0+0 (secure 0 0 0) 0x7f7b38057e80 con 0x7f7b500cde00 2023-10-26T01:39:18.255 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:18.253+0000 7f7b55cb4640 1 -- 172.21.15.169:0/1481871543 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7b3006ae40 msgr2=0x7f7b3006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:18.255 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:18.253+0000 7f7b55cb4640 1 --2- 172.21.15.169:0/1481871543 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7b3006ae40 0x7f7b3006d300 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f7b40005fd0 tx=0x7f7b40009260 comp rx=0 tx=0).stop 2023-10-26T01:39:18.256 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:18.253+0000 7f7b55cb4640 1 -- 172.21.15.169:0/1481871543 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b500cde00 msgr2=0x7f7b5015bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:18.256 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:18.253+0000 7f7b55cb4640 1 --2- 172.21.15.169:0/1481871543 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b500cde00 0x7f7b5015bf10 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f7b3800c500 tx=0x7f7b3800c9c0 comp rx=0 tx=0).stop 2023-10-26T01:39:18.256 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:18.253+0000 7f7b55cb4640 1 -- 172.21.15.169:0/1481871543 shutdown_connections 2023-10-26T01:39:18.256 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:18.253+0000 7f7b55cb4640 1 --2- 172.21.15.169:0/1481871543 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7b3006ae40 0x7f7b3006d300 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:18.256 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:18.253+0000 7f7b55cb4640 1 --2- 172.21.15.169:0/1481871543 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b500cde00 0x7f7b5015bf10 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:18.257 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:18.253+0000 7f7b55cb4640 1 --2- 172.21.15.169:0/1481871543 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7b501592d0 0x7f7b5014b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:18.257 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:18.253+0000 7f7b55cb4640 1 --2- 172.21.15.169:0/1481871543 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b500ac930 0x7f7b50150480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:18.257 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:18.253+0000 7f7b55cb4640 1 -- 172.21.15.169:0/1481871543 >> 172.21.15.169:0/1481871543 conn(0x7f7b50069930 msgr2=0x7f7b500af8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:18.257 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:18.254+0000 7f7b55cb4640 1 -- 172.21.15.169:0/1481871543 shutdown_connections 2023-10-26T01:39:18.257 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:18.254+0000 7f7b55cb4640 1 -- 172.21.15.169:0/1481871543 wait complete. 2023-10-26T01:39:18.276 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2023-10-26T01:39:18.493 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:18.490+0000 7f9946075640 1 -- 172.21.15.169:0/4166125167 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99401597c0 msgr2=0x7f994015dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:18.493 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:18.490+0000 7f9946075640 1 --2- 172.21.15.169:0/4166125167 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99401597c0 0x7f994015dbf0 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f99400cd2c0 tx=0x7f992802bd90 comp rx=0 tx=0).stop 2023-10-26T01:39:18.493 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:18.491+0000 7f9946075640 1 -- 172.21.15.169:0/4166125167 shutdown_connections 2023-10-26T01:39:18.493 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:18.491+0000 7f9946075640 1 --2- 172.21.15.169:0/4166125167 >> v2:172.21.15.170:3300/0 conn(0x7f99400b0c80 0x7f99400cd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:18.494 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:18.491+0000 7f9946075640 1 --2- 172.21.15.169:0/4166125167 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99401597c0 0x7f994015dbf0 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:18.494 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:18.491+0000 7f9946075640 1 --2- 172.21.15.169:0/4166125167 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f99401593e0 0x7f99400ac200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:18.494 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:18.491+0000 7f9946075640 1 -- 172.21.15.169:0/4166125167 >> 172.21.15.169:0/4166125167 conn(0x7f9940069930 msgr2=0x7f9940069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:18.494 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:18.491+0000 7f9946075640 1 -- 172.21.15.169:0/4166125167 shutdown_connections 2023-10-26T01:39:18.494 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:18.491+0000 7f9946075640 1 -- 172.21.15.169:0/4166125167 wait complete. 2023-10-26T01:39:18.497 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:18.492+0000 7f9946075640 1 Processor -- start 2023-10-26T01:39:18.497 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:18.493+0000 7f9946075640 1 -- start start 2023-10-26T01:39:18.497 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:18.493+0000 7f9946075640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99400b0c80 0x7f994014c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:18.497 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:18.494+0000 7f9946075640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f99401593e0 0x7f99400ae7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:18.497 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:18.494+0000 7f9946075640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f99401597c0 0x7f99400aecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:18.498 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:18.494+0000 7f9946075640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f994006f140 con 0x7f99400b0c80 2023-10-26T01:39:18.498 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:18.494+0000 7f9946075640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99400addf0 con 0x7f99401597c0 2023-10-26T01:39:18.498 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:18.494+0000 7f9946075640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99400ae0d0 con 0x7f99401593e0 2023-10-26T01:39:18.498 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:18.494+0000 7f993f7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99400b0c80 0x7f994014c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:18.498 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:18.494+0000 7f993f7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99400b0c80 0x7f994014c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:53108/0 (socket says 172.21.15.169:53108) 2023-10-26T01:39:18.499 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:18.494+0000 7f993f7fe640 1 -- 172.21.15.169:0/200683047 learned_addr learned my addr 172.21.15.169:0/200683047 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:39:18.499 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:18.494+0000 7f9936ffd640 1 --2- 172.21.15.169:0/200683047 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f99401593e0 0x7f99400ae7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:18.499 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:18.494+0000 7f993f7fe640 1 -- 172.21.15.169:0/200683047 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f99401593e0 msgr2=0x7f99400ae7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:18.499 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:18.494+0000 7f993f7fe640 1 --2- 172.21.15.169:0/200683047 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f99401593e0 0x7f99400ae7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:18.499 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:18.494+0000 7f993ffff640 1 --2- 172.21.15.169:0/200683047 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f99401597c0 0x7f99400aecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:18.500 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:18.494+0000 7f993f7fe640 1 -- 172.21.15.169:0/200683047 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f99401597c0 msgr2=0x7f99400aecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:18.500 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:18.494+0000 7f993f7fe640 1 --2- 172.21.15.169:0/200683047 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f99401597c0 0x7f99400aecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:18.500 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:18.494+0000 7f993f7fe640 1 -- 172.21.15.169:0/200683047 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9928009560 con 0x7f99400b0c80 2023-10-26T01:39:18.500 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:18.495+0000 7f993f7fe640 1 --2- 172.21.15.169:0/200683047 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99400b0c80 0x7f994014c630 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f993000b4a0 tx=0x7f993000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:18.500 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:18.495+0000 7f993cff9640 1 -- 172.21.15.169:0/200683047 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9930010610 con 0x7f99400b0c80 2023-10-26T01:39:18.500 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:18.495+0000 7f993cff9640 1 -- 172.21.15.169:0/200683047 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9930010860 con 0x7f99400b0c80 2023-10-26T01:39:18.501 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:18.495+0000 7f993cff9640 1 -- 172.21.15.169:0/200683047 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9930010c70 con 0x7f99400b0c80 2023-10-26T01:39:18.501 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:18.495+0000 7f9946075640 1 -- 172.21.15.169:0/200683047 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99400ae3b0 con 0x7f99400b0c80 2023-10-26T01:39:18.501 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:18.495+0000 7f9946075640 1 -- 172.21.15.169:0/200683047 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9940155f10 con 0x7f99400b0c80 2023-10-26T01:39:18.502 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:18.496+0000 7f9946075640 1 -- 172.21.15.169:0/200683047 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f994004db10 con 0x7f99400b0c80 2023-10-26T01:39:18.502 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:18.497+0000 7f993cff9640 1 -- 172.21.15.169:0/200683047 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f9930017360 con 0x7f99400b0c80 2023-10-26T01:39:18.502 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:18.499+0000 7f993cff9640 1 --2- 172.21.15.169:0/200683047 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f992006ae40 0x7f992006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:18.503 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:18.499+0000 7f993cff9640 1 -- 172.21.15.169:0/200683047 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f9930089260 con 0x7f99400b0c80 2023-10-26T01:39:18.503 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:18.500+0000 7f9936ffd640 1 --2- 172.21.15.169:0/200683047 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f992006ae40 0x7f992006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:18.503 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:18.501+0000 7f9936ffd640 1 --2- 172.21.15.169:0/200683047 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f992006ae40 0x7f992006d300 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f9928035000 tx=0x7f9928004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:18.505 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:18.504+0000 7f993cff9640 1 -- 172.21.15.169:0/200683047 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9930058000 con 0x7f99400b0c80 2023-10-26T01:39:18.696 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:18.694+0000 7f9946075640 1 -- 172.21.15.169:0/200683047 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "config dump"} v 0) v1 -- 0x7f99400b2170 con 0x7f99400b0c80 2023-10-26T01:39:18.697 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:18.695+0000 7f993cff9640 1 -- 172.21.15.169:0/200683047 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v32) v1 ==== 59+0+1116 (secure 0 0 0) 0x7f9930057b90 con 0x7f99400b0c80 2023-10-26T01:39:18.697 INFO:teuthology.orchestra.run.smithi169.stdout:WHO MASK LEVEL OPTION VALUE RO 2023-10-26T01:39:18.697 INFO:teuthology.orchestra.run.smithi169.stdout:mon advanced mon_op_complaint_time 120 2023-10-26T01:39:18.698 INFO:teuthology.orchestra.run.smithi169.stdout:osd advanced osd_op_complaint_time 180.000000 2023-10-26T01:39:18.698 INFO:teuthology.orchestra.run.smithi169.stdout:mds advanced debug_mds 20/20 2023-10-26T01:39:18.698 INFO:teuthology.orchestra.run.smithi169.stdout:mds advanced debug_mds_balancer 20/20 2023-10-26T01:39:18.698 INFO:teuthology.orchestra.run.smithi169.stdout:mds advanced debug_ms 1/1 2023-10-26T01:39:18.698 INFO:teuthology.orchestra.run.smithi169.stdout:mds dev mds_debug_frag true 2023-10-26T01:39:18.699 INFO:teuthology.orchestra.run.smithi169.stdout:mds dev mds_debug_scatterstat true 2023-10-26T01:39:18.699 INFO:teuthology.orchestra.run.smithi169.stdout:mds advanced mds_op_complaint_time 180.000000 2023-10-26T01:39:18.699 INFO:teuthology.orchestra.run.smithi169.stdout:mds dev mds_verify_scatter true 2023-10-26T01:39:18.699 INFO:teuthology.orchestra.run.smithi169.stdout:mds advanced osd_op_complaint_time 180.000000 2023-10-26T01:39:18.699 INFO:teuthology.orchestra.run.smithi169.stdout:mds advanced rados_mon_op_timeout 900 2023-10-26T01:39:18.700 INFO:teuthology.orchestra.run.smithi169.stdout:mds advanced rados_osd_op_timeout 900 2023-10-26T01:39:18.700 INFO:teuthology.orchestra.run.smithi169.stdout:client advanced client_mount_timeout 600 2023-10-26T01:39:18.700 INFO:teuthology.orchestra.run.smithi169.stdout:client advanced debug_client 20/20 2023-10-26T01:39:18.700 INFO:teuthology.orchestra.run.smithi169.stdout:client advanced debug_ms 1/1 2023-10-26T01:39:18.700 INFO:teuthology.orchestra.run.smithi169.stdout:client advanced rados_mon_op_timeout 900 2023-10-26T01:39:18.700 INFO:teuthology.orchestra.run.smithi169.stdout:client advanced rados_osd_op_timeout 900 2023-10-26T01:39:18.701 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:18.698+0000 7f9946075640 1 -- 172.21.15.169:0/200683047 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f992006ae40 msgr2=0x7f992006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:18.702 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:18.698+0000 7f9946075640 1 --2- 172.21.15.169:0/200683047 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f992006ae40 0x7f992006d300 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f9928035000 tx=0x7f9928004730 comp rx=0 tx=0).stop 2023-10-26T01:39:18.702 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:18.698+0000 7f9946075640 1 -- 172.21.15.169:0/200683047 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99400b0c80 msgr2=0x7f994014c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:18.702 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:18.698+0000 7f9946075640 1 --2- 172.21.15.169:0/200683047 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99400b0c80 0x7f994014c630 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f993000b4a0 tx=0x7f993000b960 comp rx=0 tx=0).stop 2023-10-26T01:39:18.702 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:18.698+0000 7f9946075640 1 -- 172.21.15.169:0/200683047 shutdown_connections 2023-10-26T01:39:18.702 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:18.698+0000 7f9946075640 1 --2- 172.21.15.169:0/200683047 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f992006ae40 0x7f992006d300 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:18.703 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:18.698+0000 7f9946075640 1 --2- 172.21.15.169:0/200683047 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f99401597c0 0x7f99400aecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:18.703 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:18.698+0000 7f9946075640 1 --2- 172.21.15.169:0/200683047 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f99401593e0 0x7f99400ae7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:18.703 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:18.698+0000 7f9946075640 1 --2- 172.21.15.169:0/200683047 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99400b0c80 0x7f994014c630 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:18.703 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:18.698+0000 7f9946075640 1 -- 172.21.15.169:0/200683047 >> 172.21.15.169:0/200683047 conn(0x7f9940069930 msgr2=0x7f99400c8670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:18.703 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:18.698+0000 7f9946075640 1 -- 172.21.15.169:0/200683047 shutdown_connections 2023-10-26T01:39:18.704 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:18.698+0000 7f9946075640 1 -- 172.21.15.169:0/200683047 wait complete. 2023-10-26T01:39:18.719 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-26T01:39:18.944 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:18.942+0000 7f9556b8e640 1 -- 172.21.15.169:0/2196222101 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95380047c0 msgr2=0x7f9538004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:18.944 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:18.942+0000 7f9556b8e640 1 --2- 172.21.15.169:0/2196222101 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95380047c0 0x7f9538004c40 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f9544001650 tx=0x7f95380057f0 comp rx=0 tx=0).stop 2023-10-26T01:39:18.945 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:18.942+0000 7f955638d640 1 -- 172.21.15.169:0/2196222101 reap_dead start 2023-10-26T01:39:18.945 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:18.942+0000 7f9556b8e640 1 -- 172.21.15.169:0/2196222101 shutdown_connections 2023-10-26T01:39:18.945 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:18.942+0000 7f9556b8e640 1 -- 172.21.15.169:0/2196222101 >> 172.21.15.169:0/2196222101 conn(0x7f95500b3740 msgr2=0x7f95500afd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:18.945 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:18.942+0000 7f9556b8e640 1 -- 172.21.15.169:0/2196222101 shutdown_connections 2023-10-26T01:39:18.945 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:18.942+0000 7f9556b8e640 1 -- 172.21.15.169:0/2196222101 wait complete. 2023-10-26T01:39:18.947 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:18.944+0000 7f9556b8e640 1 Processor -- start 2023-10-26T01:39:18.947 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:18.944+0000 7f9556b8e640 1 -- start start 2023-10-26T01:39:18.948 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:18.945+0000 7f9556b8e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95380047c0 0x7f9550145ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:18.948 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:18.945+0000 7f9556b8e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f955014a000 0x7f95500ce7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:18.948 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:18.945+0000 7f9556b8e640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9550143f60 0x7f9550144410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:18.948 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:18.945+0000 7f9556b8e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95500ced10 con 0x7f95380047c0 2023-10-26T01:39:18.948 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:18.945+0000 7f9556b8e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9550146f20 con 0x7f955014a000 2023-10-26T01:39:18.949 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:18.945+0000 7f9556b8e640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9550146680 con 0x7f9550143f60 2023-10-26T01:39:18.949 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:18.945+0000 7f9555b8c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95380047c0 0x7f9550145ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:18.949 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:18.945+0000 7f9555b8c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95380047c0 0x7f9550145ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:53124/0 (socket says 172.21.15.169:53124) 2023-10-26T01:39:18.949 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:18.945+0000 7f9555b8c640 1 -- 172.21.15.169:0/4088546816 learned_addr learned my addr 172.21.15.169:0/4088546816 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:39:18.950 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:18.945+0000 7f955538b640 1 --2- 172.21.15.169:0/4088546816 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f955014a000 0x7f95500ce7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:18.950 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:18.945+0000 7f9555b8c640 1 -- 172.21.15.169:0/4088546816 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9550143f60 msgr2=0x7f9550144410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:18.950 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:18.945+0000 7f9555b8c640 1 --2- 172.21.15.169:0/4088546816 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9550143f60 0x7f9550144410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:18.950 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:18.945+0000 7f9555b8c640 1 -- 172.21.15.169:0/4088546816 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f955014a000 msgr2=0x7f95500ce7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:18.951 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:18.945+0000 7f9555b8c640 1 --2- 172.21.15.169:0/4088546816 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f955014a000 0x7f95500ce7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:18.951 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:18.946+0000 7f9555b8c640 1 -- 172.21.15.169:0/4088546816 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9550146960 con 0x7f95380047c0 2023-10-26T01:39:18.951 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:18.946+0000 7f9555b8c640 1 --2- 172.21.15.169:0/4088546816 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95380047c0 0x7f9550145ff0 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f9540010c10 tx=0x7f954000e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:18.952 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:18.946+0000 7f954e7fc640 1 -- 172.21.15.169:0/4088546816 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9540003b70 con 0x7f95380047c0 2023-10-26T01:39:18.952 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:18.946+0000 7f954e7fc640 1 -- 172.21.15.169:0/4088546816 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9540015070 con 0x7f95380047c0 2023-10-26T01:39:18.953 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:18.946+0000 7f954e7fc640 1 -- 172.21.15.169:0/4088546816 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9540004020 con 0x7f95380047c0 2023-10-26T01:39:18.953 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:18.946+0000 7f9556b8e640 1 -- 172.21.15.169:0/4088546816 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9550146c40 con 0x7f95380047c0 2023-10-26T01:39:18.953 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:18.946+0000 7f9556b8e640 1 -- 172.21.15.169:0/4088546816 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95500ad190 con 0x7f95380047c0 2023-10-26T01:39:18.953 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:18.947+0000 7f9556b8e640 1 -- 172.21.15.169:0/4088546816 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9550073c10 con 0x7f95380047c0 2023-10-26T01:39:18.953 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:18.948+0000 7f954e7fc640 1 -- 172.21.15.169:0/4088546816 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f9540004180 con 0x7f95380047c0 2023-10-26T01:39:18.954 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:18.949+0000 7f954e7fc640 1 --2- 172.21.15.169:0/4088546816 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f953c06ae70 0x7f953c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:18.954 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:18.949+0000 7f954e7fc640 1 -- 172.21.15.169:0/4088546816 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f9540089e10 con 0x7f95380047c0 2023-10-26T01:39:18.954 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:18.950+0000 7f955538b640 1 --2- 172.21.15.169:0/4088546816 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f953c06ae70 0x7f953c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:18.954 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:18.951+0000 7f955538b640 1 --2- 172.21.15.169:0/4088546816 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f953c06ae70 0x7f953c06d330 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f9544034000 tx=0x7f954402f040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:18.955 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:18.953+0000 7f954e7fc640 1 -- 172.21.15.169:0/4088546816 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f954008d030 con 0x7f95380047c0 2023-10-26T01:39:19.146 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:19.144+0000 7f9556b8e640 1 -- 172.21.15.169:0/4088546816 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f9550071b80 con 0x7f95380047c0 2023-10-26T01:39:19.151 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:19.150+0000 7f954e7fc640 1 -- 172.21.15.169:0/4088546816 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v33)=0 v33) v1 ==== 122+0+0 (secure 0 0 0) 0x7f9540055260 con 0x7f95380047c0 2023-10-26T01:39:19.155 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:19.153+0000 7f9556b8e640 1 -- 172.21.15.169:0/4088546816 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f953c06ae70 msgr2=0x7f953c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:19.155 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:19.153+0000 7f9556b8e640 1 --2- 172.21.15.169:0/4088546816 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f953c06ae70 0x7f953c06d330 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f9544034000 tx=0x7f954402f040 comp rx=0 tx=0).stop 2023-10-26T01:39:19.155 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:19.153+0000 7f9556b8e640 1 -- 172.21.15.169:0/4088546816 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95380047c0 msgr2=0x7f9550145ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:19.155 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:19.153+0000 7f9556b8e640 1 --2- 172.21.15.169:0/4088546816 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95380047c0 0x7f9550145ff0 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f9540010c10 tx=0x7f954000e720 comp rx=0 tx=0).stop 2023-10-26T01:39:19.156 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:19.153+0000 7f9556b8e640 1 -- 172.21.15.169:0/4088546816 shutdown_connections 2023-10-26T01:39:19.156 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:19.153+0000 7f9556b8e640 1 --2- 172.21.15.169:0/4088546816 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f953c06ae70 0x7f953c06d330 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:19.156 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:19.153+0000 7f9556b8e640 1 --2- 172.21.15.169:0/4088546816 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9550143f60 0x7f9550144410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:19.156 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:19.153+0000 7f9556b8e640 1 --2- 172.21.15.169:0/4088546816 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f955014a000 0x7f95500ce7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:19.156 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:19.153+0000 7f9556b8e640 1 --2- 172.21.15.169:0/4088546816 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95380047c0 0x7f9550145ff0 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:19.157 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:19.153+0000 7f9556b8e640 1 -- 172.21.15.169:0/4088546816 >> 172.21.15.169:0/4088546816 conn(0x7f95500b3740 msgr2=0x7f955015dbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:19.157 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:19.153+0000 7f9556b8e640 1 -- 172.21.15.169:0/4088546816 shutdown_connections 2023-10-26T01:39:19.157 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:19.153+0000 7f9556b8e640 1 -- 172.21.15.169:0/4088546816 wait complete. 2023-10-26T01:39:19.173 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-26T01:39:19.174 DEBUG:teuthology.orchestra.run.smithi169:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-26T01:39:19.288 DEBUG:teuthology.orchestra.run.smithi170:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-26T01:39:19.416 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-26T01:39:19.416 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:39:19.417 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:39:19.477 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054301,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-26T01:39:19.478 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-26T01:39:19.479 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:39:19.539 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054303,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-26T01:39:19.540 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:39:19.540 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:39:19.601 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054301,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-26T01:39:19.602 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-26T01:39:19.602 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:39:19.662 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054303,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-26T01:39:19.663 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-26T01:39:19.664 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T01:39:19.894 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:19.891+0000 7f3f96c02640 1 -- 172.21.15.169:0/2637809027 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3f901592d0 msgr2=0x7f3f900aa3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:19.894 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:19.891+0000 7f3f96c02640 1 --2- 172.21.15.169:0/2637809027 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3f901592d0 0x7f3f900aa3d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3f84005fd0 tx=0x7f3f8402bd90 comp rx=0 tx=0).stop 2023-10-26T01:39:19.894 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:19.892+0000 7f3f96c02640 1 -- 172.21.15.169:0/2637809027 shutdown_connections 2023-10-26T01:39:19.895 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:19.892+0000 7f3f96c02640 1 --2- 172.21.15.169:0/2637809027 >> v2:172.21.15.170:3300/0 conn(0x7f3f900cde00 0x7f3f900cbac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:19.895 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:19.892+0000 7f3f96c02640 1 --2- 172.21.15.169:0/2637809027 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3f901592d0 0x7f3f900aa3d0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:19.895 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:19.892+0000 7f3f96c02640 1 -- 172.21.15.169:0/2637809027 >> 172.21.15.169:0/2637809027 conn(0x7f3f90069930 msgr2=0x7f3f90069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:19.895 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:19.892+0000 7f3f96c02640 1 -- 172.21.15.169:0/2637809027 shutdown_connections 2023-10-26T01:39:19.896 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:19.892+0000 7f3f96c02640 1 -- 172.21.15.169:0/2637809027 wait complete. 2023-10-26T01:39:19.896 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:19.894+0000 7f3f96c02640 1 Processor -- start 2023-10-26T01:39:19.896 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:19.894+0000 7f3f96c02640 1 -- start start 2023-10-26T01:39:19.899 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:19.895+0000 7f3f96c02640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3f900ac930 0x7f3f90158670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:19.899 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:19.895+0000 7f3f96c02640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3f900cde00 0x7f3f901f13b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:19.899 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:19.895+0000 7f3f96c02640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3f901592d0 0x7f3f90157700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:19.899 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:19.895+0000 7f3f96c02640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f901f18f0 con 0x7f3f900ac930 2023-10-26T01:39:19.899 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:19.895+0000 7f3f96c02640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f9014fdf0 con 0x7f3f900cde00 2023-10-26T01:39:19.900 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:19.895+0000 7f3f96c02640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f900b1260 con 0x7f3f901592d0 2023-10-26T01:39:19.900 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:19.896+0000 7f3f9499e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3f900ac930 0x7f3f90158670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:19.900 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:19.896+0000 7f3f9499e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3f900ac930 0x7f3f90158670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:53128/0 (socket says 172.21.15.169:53128) 2023-10-26T01:39:19.900 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:19.896+0000 7f3f9499e640 1 -- 172.21.15.169:0/861618274 learned_addr learned my addr 172.21.15.169:0/861618274 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:39:19.901 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:19.896+0000 7f3f8ffff640 1 --2- 172.21.15.169:0/861618274 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3f900cde00 0x7f3f901f13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:19.901 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:19.896+0000 7f3f9519f640 1 --2- 172.21.15.169:0/861618274 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3f901592d0 0x7f3f90157700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:19.901 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:19.896+0000 7f3f9499e640 1 -- 172.21.15.169:0/861618274 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3f901592d0 msgr2=0x7f3f90157700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:19.901 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:19.896+0000 7f3f9499e640 1 --2- 172.21.15.169:0/861618274 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3f901592d0 0x7f3f90157700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:19.902 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:19.896+0000 7f3f9499e640 1 -- 172.21.15.169:0/861618274 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3f900cde00 msgr2=0x7f3f901f13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:19.902 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:19.896+0000 7f3f9499e640 1 --2- 172.21.15.169:0/861618274 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3f900cde00 0x7f3f901f13b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:19.902 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:19.896+0000 7f3f9499e640 1 -- 172.21.15.169:0/861618274 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f88009840 con 0x7f3f900ac930 2023-10-26T01:39:19.902 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:19.896+0000 7f3f9499e640 1 --2- 172.21.15.169:0/861618274 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3f900ac930 0x7f3f90158670 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f3f84006010 tx=0x7f3f8402d9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:19.902 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:19.897+0000 7f3f8d7fa640 1 -- 172.21.15.169:0/861618274 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f8402ddf0 con 0x7f3f900ac930 2023-10-26T01:39:19.903 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:19.897+0000 7f3f8d7fa640 1 -- 172.21.15.169:0/861618274 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3f840036e0 con 0x7f3f900ac930 2023-10-26T01:39:19.903 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:19.897+0000 7f3f96c02640 1 -- 172.21.15.169:0/861618274 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f84009560 con 0x7f3f900ac930 2023-10-26T01:39:19.903 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:19.897+0000 7f3f8d7fa640 1 -- 172.21.15.169:0/861618274 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f84003af0 con 0x7f3f900ac930 2023-10-26T01:39:19.903 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:19.897+0000 7f3f96c02640 1 -- 172.21.15.169:0/861618274 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f900b1820 con 0x7f3f900ac930 2023-10-26T01:39:19.904 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:19.898+0000 7f3f96c02640 1 -- 172.21.15.169:0/861618274 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f9004db10 con 0x7f3f900ac930 2023-10-26T01:39:19.904 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:19.899+0000 7f3f8d7fa640 1 -- 172.21.15.169:0/861618274 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f3f84003c50 con 0x7f3f900ac930 2023-10-26T01:39:19.905 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:19.900+0000 7f3f8d7fa640 1 --2- 172.21.15.169:0/861618274 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3f7406ae40 0x7f3f7406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:19.905 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:19.900+0000 7f3f8d7fa640 1 -- 172.21.15.169:0/861618274 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f3f840abe90 con 0x7f3f900ac930 2023-10-26T01:39:19.905 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:19.901+0000 7f3f8ffff640 1 --2- 172.21.15.169:0/861618274 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3f7406ae40 0x7f3f7406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:19.905 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:19.902+0000 7f3f8ffff640 1 --2- 172.21.15.169:0/861618274 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3f7406ae40 0x7f3f7406d300 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f3f900ad910 tx=0x7f3f7c009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:19.906 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:19.905+0000 7f3f8d7fa640 1 -- 172.21.15.169:0/861618274 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3f84077410 con 0x7f3f900ac930 2023-10-26T01:39:20.136 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:20.134+0000 7f3f96c02640 1 -- 172.21.15.169:0/861618274 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3f901598d0 con 0x7f3f900ac930 2023-10-26T01:39:20.137 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:20.136+0000 7f3f8d7fa640 1 -- 172.21.15.169:0/861618274 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) v1 ==== 94+0+3708 (secure 0 0 0) 0x7f3f84043090 con 0x7f3f900ac930 2023-10-26T01:39:20.138 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:39:20.138 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":70,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5660,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/979285001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":979285001},{"type":"v1","addr":"172.21.15.170:6837","nonce":979285001}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70},{"gid":5917,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/3297412140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":3297412140},{"type":"v1","addr":"172.21.15.169:6835","nonce":3297412140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":5929,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/2575809940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2575809940},{"type":"v1","addr":"172.21.15.170:6836","nonce":2575809940}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":7761,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2632217991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2632217991},{"type":"v1","addr":"172.21.15.169:6833","nonce":2632217991}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62}],"filesystems":[]} 2023-10-26T01:39:20.138 INFO:teuthology.orchestra.run.smithi169.stderr:dumped fsmap epoch 70 2023-10-26T01:39:20.140 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:20.138+0000 7f3f96c02640 1 -- 172.21.15.169:0/861618274 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3f7406ae40 msgr2=0x7f3f7406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:20.140 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:20.138+0000 7f3f96c02640 1 --2- 172.21.15.169:0/861618274 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3f7406ae40 0x7f3f7406d300 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f3f900ad910 tx=0x7f3f7c009040 comp rx=0 tx=0).stop 2023-10-26T01:39:20.141 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:20.138+0000 7f3f96c02640 1 -- 172.21.15.169:0/861618274 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3f900ac930 msgr2=0x7f3f90158670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:20.141 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:20.138+0000 7f3f96c02640 1 --2- 172.21.15.169:0/861618274 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3f900ac930 0x7f3f90158670 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f3f84006010 tx=0x7f3f8402d9f0 comp rx=0 tx=0).stop 2023-10-26T01:39:20.141 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:20.138+0000 7f3f96c02640 1 -- 172.21.15.169:0/861618274 shutdown_connections 2023-10-26T01:39:20.141 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:20.138+0000 7f3f96c02640 1 --2- 172.21.15.169:0/861618274 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3f7406ae40 0x7f3f7406d300 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:20.141 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:20.138+0000 7f3f96c02640 1 --2- 172.21.15.169:0/861618274 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3f901592d0 0x7f3f90157700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:20.142 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:20.138+0000 7f3f96c02640 1 --2- 172.21.15.169:0/861618274 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3f900cde00 0x7f3f901f13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:20.142 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:20.138+0000 7f3f96c02640 1 --2- 172.21.15.169:0/861618274 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3f900ac930 0x7f3f90158670 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:20.142 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:20.138+0000 7f3f96c02640 1 -- 172.21.15.169:0/861618274 >> 172.21.15.169:0/861618274 conn(0x7f3f90069930 msgr2=0x7f3f90156ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:20.142 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:20.138+0000 7f3f96c02640 1 -- 172.21.15.169:0/861618274 shutdown_connections 2023-10-26T01:39:20.142 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:20.138+0000 7f3f96c02640 1 -- 172.21.15.169:0/861618274 wait complete. 2023-10-26T01:39:20.159 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-26T01:39:20.159 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-26T01:39:20.160 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-26T01:39:20.160 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T01:39:20.160 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-26T01:39:20.160 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-26T01:39:20.161 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-26T01:39:20.161 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T01:39:20.161 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-26T01:39:20.162 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-26T01:39:20.162 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-26T01:39:20.162 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T01:39:20.162 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-26T01:39:20.163 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-26T01:39:20.163 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-26T01:39:20.163 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T01:39:20.177 INFO:tasks.ceph.mds.a:Stopped 2023-10-26T01:39:20.177 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-26T01:39:20.179 INFO:tasks.ceph.mds.c:Stopped 2023-10-26T01:39:20.179 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-26T01:39:20.184 INFO:tasks.ceph.mds.d:Stopped 2023-10-26T01:39:20.184 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-26T01:39:20.185 INFO:tasks.ceph.mds.b:Stopped 2023-10-26T01:39:20.186 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-26T01:39:20.228 INFO:tasks.ceph.mds.a:Started 2023-10-26T01:39:20.228 DEBUG:teuthology.parallel:result is None 2023-10-26T01:39:20.233 INFO:tasks.ceph.mds.d:Started 2023-10-26T01:39:20.234 DEBUG:teuthology.parallel:result is None 2023-10-26T01:39:20.236 INFO:tasks.ceph.mds.c:Started 2023-10-26T01:39:20.236 DEBUG:teuthology.parallel:result is None 2023-10-26T01:39:20.240 INFO:tasks.ceph.mds.b:Started 2023-10-26T01:39:20.241 DEBUG:teuthology.parallel:result is None 2023-10-26T01:39:20.241 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-26T01:39:20.386 INFO:tasks.ceph.mds.a.smithi169.stdout:starting mds.a at 2023-10-26T01:39:20.390 INFO:tasks.ceph.mds.c.smithi169.stdout:starting mds.c at 2023-10-26T01:39:20.403 INFO:tasks.ceph.mds.d.smithi170.stdout:starting mds.d at 2023-10-26T01:39:20.403 INFO:tasks.ceph.mds.b.smithi170.stdout:starting mds.b at 2023-10-26T01:39:20.432 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:20.430+0000 7fb233fff640 1 -- 172.21.15.169:0/2902759166 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb234153b90 msgr2=0x7fb234069900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:20.432 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:20.430+0000 7fb233fff640 1 --2- 172.21.15.169:0/2902759166 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb234153b90 0x7fb234069900 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7fb23413fc00 tx=0x7fb22402bd90 comp rx=0 tx=0).stop 2023-10-26T01:39:20.433 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:20.430+0000 7fb233fff640 1 -- 172.21.15.169:0/2902759166 shutdown_connections 2023-10-26T01:39:20.433 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:20.430+0000 7fb233fff640 1 --2- 172.21.15.169:0/2902759166 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb23415d0d0 0x7fb23415d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:20.433 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:20.430+0000 7fb233fff640 1 --2- 172.21.15.169:0/2902759166 >> v2:172.21.15.170:3300/0 conn(0x7fb2340b0e50 0x7fb234069e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:20.433 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:20.430+0000 7fb233fff640 1 --2- 172.21.15.169:0/2902759166 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb234153b90 0x7fb234069900 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:20.434 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:20.430+0000 7fb233fff640 1 -- 172.21.15.169:0/2902759166 >> 172.21.15.169:0/2902759166 conn(0x7fb2340af110 msgr2=0x7fb2340af520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:20.434 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:20.430+0000 7fb233fff640 1 -- 172.21.15.169:0/2902759166 shutdown_connections 2023-10-26T01:39:20.434 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:20.430+0000 7fb233fff640 1 -- 172.21.15.169:0/2902759166 wait complete. 2023-10-26T01:39:20.436 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:20.431+0000 7fb233fff640 1 Processor -- start 2023-10-26T01:39:20.436 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:20.432+0000 7fb233fff640 1 -- start start 2023-10-26T01:39:20.436 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:20.432+0000 7fb233fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb2340b0e50 0x7fb2340999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:20.437 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:20.432+0000 7fb233fff640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb234153b90 0x7fb23406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:20.437 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:20.432+0000 7fb233fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb23415d0d0 0x7fb2340ab020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:20.437 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:20.433+0000 7fb233fff640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb234154470 con 0x7fb23415d0d0 2023-10-26T01:39:20.437 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:20.433+0000 7fb233fff640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2341542c0 con 0x7fb2340b0e50 2023-10-26T01:39:20.437 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:20.433+0000 7fb233fff640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb234154f80 con 0x7fb234153b90 2023-10-26T01:39:20.438 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:20.433+0000 7fb2337fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb23415d0d0 0x7fb2340ab020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:20.438 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:20.433+0000 7fb2337fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb23415d0d0 0x7fb2340ab020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:53194/0 (socket says 172.21.15.169:53194) 2023-10-26T01:39:20.438 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:20.433+0000 7fb2337fe640 1 -- 172.21.15.169:0/1268687573 learned_addr learned my addr 172.21.15.169:0/1268687573 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:39:20.438 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:20.433+0000 7fb2327fc640 1 --2- 172.21.15.169:0/1268687573 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb234153b90 0x7fb23406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:20.438 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:20.433+0000 7fb232ffd640 1 --2- 172.21.15.169:0/1268687573 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb2340b0e50 0x7fb2340999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:20.439 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:20.433+0000 7fb2337fe640 1 -- 172.21.15.169:0/1268687573 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb234153b90 msgr2=0x7fb23406c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:20.439 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:20.433+0000 7fb2337fe640 1 --2- 172.21.15.169:0/1268687573 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb234153b90 0x7fb23406c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:20.439 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:20.433+0000 7fb2337fe640 1 -- 172.21.15.169:0/1268687573 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb2340b0e50 msgr2=0x7fb2340999d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:20.439 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:20.433+0000 7fb2337fe640 1 --2- 172.21.15.169:0/1268687573 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb2340b0e50 0x7fb2340999d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:20.439 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:20.433+0000 7fb2337fe640 1 -- 172.21.15.169:0/1268687573 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb224009560 con 0x7fb23415d0d0 2023-10-26T01:39:20.440 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:20.434+0000 7fb2337fe640 1 --2- 172.21.15.169:0/1268687573 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb23415d0d0 0x7fb2340ab020 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7fb22c00e990 tx=0x7fb22c00ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:20.440 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:20.434+0000 7fb21b7fe640 1 -- 172.21.15.169:0/1268687573 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb22c004d20 con 0x7fb23415d0d0 2023-10-26T01:39:20.440 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:20.434+0000 7fb21b7fe640 1 -- 172.21.15.169:0/1268687573 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb22c00cb00 con 0x7fb23415d0d0 2023-10-26T01:39:20.440 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:20.434+0000 7fb21b7fe640 1 -- 172.21.15.169:0/1268687573 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb22c005020 con 0x7fb23415d0d0 2023-10-26T01:39:20.440 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:20.434+0000 7fb233fff640 1 -- 172.21.15.169:0/1268687573 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb234154650 con 0x7fb23415d0d0 2023-10-26T01:39:20.440 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:20.434+0000 7fb233fff640 1 -- 172.21.15.169:0/1268687573 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb234154c10 con 0x7fb23415d0d0 2023-10-26T01:39:20.441 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:20.435+0000 7fb233fff640 1 -- 172.21.15.169:0/1268687573 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb234155130 con 0x7fb23415d0d0 2023-10-26T01:39:20.442 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:20.436+0000 7fb21b7fe640 1 -- 172.21.15.169:0/1268687573 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fb22c005180 con 0x7fb23415d0d0 2023-10-26T01:39:20.442 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:20.437+0000 7fb21b7fe640 1 --2- 172.21.15.169:0/1268687573 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb22006ae40 0x7fb22006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:20.442 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:20.437+0000 7fb21b7fe640 1 -- 172.21.15.169:0/1268687573 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fb22c089b30 con 0x7fb23415d0d0 2023-10-26T01:39:20.442 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:20.438+0000 7fb232ffd640 1 --2- 172.21.15.169:0/1268687573 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb22006ae40 0x7fb22006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:20.443 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:20.438+0000 7fb232ffd640 1 --2- 172.21.15.169:0/1268687573 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb22006ae40 0x7fb22006d300 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fb224035000 tx=0x7fb224004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:20.443 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:20.441+0000 7fb21b7fe640 1 -- 172.21.15.169:0/1268687573 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb234155130 con 0x7fb23415d0d0 2023-10-26T01:39:20.644 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:20.642+0000 7fb233fff640 1 -- 172.21.15.169:0/1268687573 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fb2341550f0 con 0x7fb23415d0d0 2023-10-26T01:39:21.286 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:21.284+0000 7fb21b7fe640 1 -- 172.21.15.169:0/1268687573 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v145) v1 ==== 97+0+0 (secure 0 0 0) 0x7fb22c01f070 con 0x7fb23415d0d0 2023-10-26T01:39:21.287 INFO:teuthology.orchestra.run.smithi169.stderr: removed all blocklist entries 2023-10-26T01:39:21.289 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:21.288+0000 7fb233fff640 1 -- 172.21.15.169:0/1268687573 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb22006ae40 msgr2=0x7fb22006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:21.290 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:21.288+0000 7fb233fff640 1 --2- 172.21.15.169:0/1268687573 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb22006ae40 0x7fb22006d300 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fb224035000 tx=0x7fb224004730 comp rx=0 tx=0).stop 2023-10-26T01:39:21.290 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:21.288+0000 7fb233fff640 1 -- 172.21.15.169:0/1268687573 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb23415d0d0 msgr2=0x7fb2340ab020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:21.290 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:21.288+0000 7fb233fff640 1 --2- 172.21.15.169:0/1268687573 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb23415d0d0 0x7fb2340ab020 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7fb22c00e990 tx=0x7fb22c00ee50 comp rx=0 tx=0).stop 2023-10-26T01:39:21.291 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:21.288+0000 7fb233fff640 1 -- 172.21.15.169:0/1268687573 shutdown_connections 2023-10-26T01:39:21.291 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:21.288+0000 7fb233fff640 1 --2- 172.21.15.169:0/1268687573 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb22006ae40 0x7fb22006d300 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:21.291 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:21.288+0000 7fb233fff640 1 --2- 172.21.15.169:0/1268687573 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb23415d0d0 0x7fb2340ab020 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:21.292 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:21.288+0000 7fb233fff640 1 --2- 172.21.15.169:0/1268687573 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb234153b90 0x7fb23406c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:21.292 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:21.288+0000 7fb233fff640 1 --2- 172.21.15.169:0/1268687573 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb2340b0e50 0x7fb2340999d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:21.292 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:21.288+0000 7fb233fff640 1 -- 172.21.15.169:0/1268687573 >> 172.21.15.169:0/1268687573 conn(0x7fb2340af110 msgr2=0x7fb2340af4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:21.292 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:21.289+0000 7fb233fff640 1 -- 172.21.15.169:0/1268687573 shutdown_connections 2023-10-26T01:39:21.292 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:21.289+0000 7fb233fff640 1 -- 172.21.15.169:0/1268687573 wait complete. 2023-10-26T01:39:21.313 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-26T01:39:21.527 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:21.525+0000 7f5f52456640 1 -- 172.21.15.169:0/2866548058 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f4c0cd720 msgr2=0x7f5f4c0cdbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:21.527 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:21.525+0000 7f5f52456640 1 --2- 172.21.15.169:0/2866548058 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f4c0cd720 0x7f5f4c0cdbd0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f5f44005fd0 tx=0x7f5f4402c1b0 comp rx=0 tx=0).stop 2023-10-26T01:39:21.528 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:21.525+0000 7f5f52456640 1 -- 172.21.15.169:0/2866548058 shutdown_connections 2023-10-26T01:39:21.528 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:21.525+0000 7f5f52456640 1 --2- 172.21.15.169:0/2866548058 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f4c0cd720 0x7f5f4c0cdbd0 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:21.528 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:21.525+0000 7f5f52456640 1 --2- 172.21.15.169:0/2866548058 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5f4c06ce20 0x7f5f4c14e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:21.528 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:21.525+0000 7f5f52456640 1 -- 172.21.15.169:0/2866548058 >> 172.21.15.169:0/2866548058 conn(0x7f5f4c069930 msgr2=0x7f5f4c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:21.528 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:21.525+0000 7f5f52456640 1 -- 172.21.15.169:0/2866548058 shutdown_connections 2023-10-26T01:39:21.529 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:21.525+0000 7f5f52456640 1 -- 172.21.15.169:0/2866548058 wait complete. 2023-10-26T01:39:21.530 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:21.527+0000 7f5f52456640 1 Processor -- start 2023-10-26T01:39:21.530 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:21.527+0000 7f5f52456640 1 -- start start 2023-10-26T01:39:21.531 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:21.528+0000 7f5f52456640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5f4c06ce20 0x7f5f4c14d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:21.531 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:21.528+0000 7f5f52456640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5f4c0cd720 0x7f5f4c1f1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:21.531 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:21.528+0000 7f5f52456640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f4c158850 0x7f5f4c1e5750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:21.531 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:21.528+0000 7f5f52456640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f4c06f270 con 0x7f5f4c158850 2023-10-26T01:39:21.531 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:21.528+0000 7f5f52456640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f4c06eed0 con 0x7f5f4c0cd720 2023-10-26T01:39:21.532 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:21.528+0000 7f5f52456640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f4c1e6410 con 0x7f5f4c06ce20 2023-10-26T01:39:21.532 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:21.528+0000 7f5f509f3640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f4c158850 0x7f5f4c1e5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:21.532 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:21.528+0000 7f5f509f3640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f4c158850 0x7f5f4c1e5750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:53210/0 (socket says 172.21.15.169:53210) 2023-10-26T01:39:21.532 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:21.528+0000 7f5f509f3640 1 -- 172.21.15.169:0/3095439097 learned_addr learned my addr 172.21.15.169:0/3095439097 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:39:21.533 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:21.529+0000 7f5f4bfff640 1 --2- 172.21.15.169:0/3095439097 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5f4c06ce20 0x7f5f4c14d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:21.533 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:21.529+0000 7f5f43fff640 1 --2- 172.21.15.169:0/3095439097 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5f4c0cd720 0x7f5f4c1f1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:21.533 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:21.529+0000 7f5f509f3640 1 -- 172.21.15.169:0/3095439097 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5f4c06ce20 msgr2=0x7f5f4c14d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:21.533 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:21.529+0000 7f5f509f3640 1 --2- 172.21.15.169:0/3095439097 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5f4c06ce20 0x7f5f4c14d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:21.533 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:21.529+0000 7f5f509f3640 1 -- 172.21.15.169:0/3095439097 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5f4c0cd720 msgr2=0x7f5f4c1f1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:21.534 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:21.529+0000 7f5f509f3640 1 --2- 172.21.15.169:0/3095439097 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5f4c0cd720 0x7f5f4c1f1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:21.534 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:21.529+0000 7f5f509f3640 1 -- 172.21.15.169:0/3095439097 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f44009900 con 0x7f5f4c158850 2023-10-26T01:39:21.535 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:21.529+0000 7f5f509f3640 1 --2- 172.21.15.169:0/3095439097 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f4c158850 0x7f5f4c1e5750 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f5f44037040 tx=0x7f5f44035db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:21.536 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:21.529+0000 7f5f497fa640 1 -- 172.21.15.169:0/3095439097 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f44034440 con 0x7f5f4c158850 2023-10-26T01:39:21.536 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:21.529+0000 7f5f497fa640 1 -- 172.21.15.169:0/3095439097 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5f44003e80 con 0x7f5f4c158850 2023-10-26T01:39:21.536 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:21.529+0000 7f5f497fa640 1 -- 172.21.15.169:0/3095439097 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f44004470 con 0x7f5f4c158850 2023-10-26T01:39:21.536 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:21.529+0000 7f5f52456640 1 -- 172.21.15.169:0/3095439097 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f4c1e5e50 con 0x7f5f4c158850 2023-10-26T01:39:21.537 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:21.530+0000 7f5f52456640 1 -- 172.21.15.169:0/3095439097 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f4c0717f0 con 0x7f5f4c158850 2023-10-26T01:39:21.537 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:21.531+0000 7f5f52456640 1 -- 172.21.15.169:0/3095439097 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f4c06ed30 con 0x7f5f4c158850 2023-10-26T01:39:21.537 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:21.532+0000 7f5f497fa640 1 -- 172.21.15.169:0/3095439097 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f5f4403b040 con 0x7f5f4c158850 2023-10-26T01:39:21.537 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:21.532+0000 7f5f497fa640 1 --2- 172.21.15.169:0/3095439097 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5f2c06ae40 0x7f5f2c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:21.537 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:21.533+0000 7f5f497fa640 1 -- 172.21.15.169:0/3095439097 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5f440abe90 con 0x7f5f4c158850 2023-10-26T01:39:21.538 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:21.533+0000 7f5f4bfff640 1 --2- 172.21.15.169:0/3095439097 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5f2c06ae40 0x7f5f2c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:21.538 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:21.534+0000 7f5f4bfff640 1 --2- 172.21.15.169:0/3095439097 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5f2c06ae40 0x7f5f2c06d300 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c005fd0 tx=0x7f5f3c005850 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:21.539 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:21.537+0000 7f5f497fa640 1 -- 172.21.15.169:0/3095439097 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5f4c06ed30 con 0x7f5f4c158850 2023-10-26T01:39:21.793 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:21.792+0000 7f5f52456640 1 -- 172.21.15.169:0/3095439097 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f5f4c06f3b0 con 0x7f5f4c158850 2023-10-26T01:39:21.795 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:21.793+0000 7f5f497fa640 1 -- 172.21.15.169:0/3095439097 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v18) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f5f4c06f3b0 con 0x7f5f4c158850 2023-10-26T01:39:21.795 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:39:21.795 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2023-10-26T01:39:21.795 INFO:teuthology.orchestra.run.smithi169.stdout: "auth_dump": [ 2023-10-26T01:39:21.796 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:39:21.796 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "mds.a", 2023-10-26T01:39:21.796 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAgwTllWtUfAhAA28+12VwWEF/BUEu+ErSVyA==", 2023-10-26T01:39:21.796 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:39:21.796 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow", 2023-10-26T01:39:21.797 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow *", 2023-10-26T01:39:21.797 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow *", 2023-10-26T01:39:21.797 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:39:21.797 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:39:21.797 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:39:21.798 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:39:21.798 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "mds.b", 2023-10-26T01:39:21.798 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAgwTlleA8lHxAA+jMcMo+QnYAGllpKJ6eYgQ==", 2023-10-26T01:39:21.798 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:39:21.799 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow", 2023-10-26T01:39:21.799 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow *", 2023-10-26T01:39:21.799 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow *", 2023-10-26T01:39:21.799 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:39:21.799 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:39:21.799 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:39:21.800 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:39:21.800 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "mds.c", 2023-10-26T01:39:21.800 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAgwTllaCCqEBAAokeHvwOyp/Jv2x0yHDwysw==", 2023-10-26T01:39:21.800 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:39:21.800 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow", 2023-10-26T01:39:21.801 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow *", 2023-10-26T01:39:21.801 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow *", 2023-10-26T01:39:21.801 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:39:21.801 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:39:21.801 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:39:21.801 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:39:21.802 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "mds.d", 2023-10-26T01:39:21.802 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAgwTll51JQLhAA4Wzso4zQj+Htv6cXqvwRmg==", 2023-10-26T01:39:21.802 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:39:21.802 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow", 2023-10-26T01:39:21.802 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow *", 2023-10-26T01:39:21.803 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow *", 2023-10-26T01:39:21.803 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:39:21.803 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:39:21.803 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:39:21.803 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:39:21.804 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "osd.0", 2023-10-26T01:39:21.804 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAlwTll8RZ+HRAAt/sBK+Mi91YmTyQYbhKHLg==", 2023-10-26T01:39:21.804 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:39:21.804 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow profile osd", 2023-10-26T01:39:21.804 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow profile osd", 2023-10-26T01:39:21.805 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:39:21.805 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:39:21.805 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:39:21.805 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:39:21.805 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "osd.1", 2023-10-26T01:39:21.805 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAlwTllLXwWLRAAuHB2lJ+oDJou/IJBtsYY4Q==", 2023-10-26T01:39:21.806 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:39:21.806 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow profile osd", 2023-10-26T01:39:21.806 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow profile osd", 2023-10-26T01:39:21.806 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:39:21.806 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:39:21.807 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:39:21.807 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:39:21.807 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "osd.2", 2023-10-26T01:39:21.807 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAmwTllj1nSARAA5Jl34Q0X7a71ySd4wz17lw==", 2023-10-26T01:39:21.807 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:39:21.808 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow profile osd", 2023-10-26T01:39:21.808 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow profile osd", 2023-10-26T01:39:21.808 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:39:21.808 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:39:21.808 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:39:21.808 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:39:21.809 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "osd.3", 2023-10-26T01:39:21.809 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAmwTllC8MeEhAAfGxRVSwD+YbU/U5aeenO3w==", 2023-10-26T01:39:21.809 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:39:21.809 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow profile osd", 2023-10-26T01:39:21.810 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow profile osd", 2023-10-26T01:39:21.810 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:39:21.810 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:39:21.810 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:39:21.810 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:39:21.811 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "osd.4", 2023-10-26T01:39:21.811 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAowTll4Md6DhAAuXfadUNAdouMP28MiX/D3Q==", 2023-10-26T01:39:21.811 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:39:21.811 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow profile osd", 2023-10-26T01:39:21.811 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow profile osd", 2023-10-26T01:39:21.812 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:39:21.812 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:39:21.812 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:39:21.812 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:39:21.812 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "osd.5", 2023-10-26T01:39:21.813 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAowTllDdViHRAAec9QyhZCsjJ2rgGhhcjwpQ==", 2023-10-26T01:39:21.813 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:39:21.813 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow profile osd", 2023-10-26T01:39:21.813 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow profile osd", 2023-10-26T01:39:21.813 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:39:21.814 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:39:21.814 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:39:21.814 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:39:21.814 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "osd.6", 2023-10-26T01:39:21.815 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAowTll23X5KxAAsmQvG0yq7ilkVG5fNiaCUg==", 2023-10-26T01:39:21.815 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:39:21.815 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow profile osd", 2023-10-26T01:39:21.815 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow profile osd", 2023-10-26T01:39:21.815 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:39:21.815 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:39:21.816 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:39:21.816 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:39:21.816 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "osd.7", 2023-10-26T01:39:21.816 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAowTllhcy/OhAAY5Yjheh76ibOVREqN6WBpQ==", 2023-10-26T01:39:21.816 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:39:21.817 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow profile osd", 2023-10-26T01:39:21.817 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow profile osd", 2023-10-26T01:39:21.817 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:39:21.817 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:39:21.817 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:39:21.818 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:39:21.818 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "client.0", 2023-10-26T01:39:21.818 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAgwTllKC7gOhAAPsvxGcuUA8/p8xqbUgGteA==", 2023-10-26T01:39:21.818 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:39:21.818 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow", 2023-10-26T01:39:21.819 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow r", 2023-10-26T01:39:21.819 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-26T01:39:21.819 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:39:21.819 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:39:21.819 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:39:21.820 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "client.1", 2023-10-26T01:39:21.820 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAhwTllhxb6ExAAeYTsurWpnjohWlrNIlLpsw==", 2023-10-26T01:39:21.820 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:39:21.820 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow", 2023-10-26T01:39:21.820 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow r", 2023-10-26T01:39:21.821 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow rw", 2023-10-26T01:39:21.821 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow rwx" 2023-10-26T01:39:21.821 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:39:21.821 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:39:21.821 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:39:21.822 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "client.2", 2023-10-26T01:39:21.822 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAhwTllLWMoChAAsUHzBc29GDqmEQAFnl7dHg==", 2023-10-26T01:39:21.822 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:39:21.822 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow", 2023-10-26T01:39:21.822 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow r", 2023-10-26T01:39:21.823 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow rw", 2023-10-26T01:39:21.823 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow rwx" 2023-10-26T01:39:21.823 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:39:21.823 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:39:21.823 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:39:21.824 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "client.3", 2023-10-26T01:39:21.824 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAhwTll875HHRAAGbwikoR/hDU8P/bj0OPWAw==", 2023-10-26T01:39:21.824 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:39:21.824 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow", 2023-10-26T01:39:21.824 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow r", 2023-10-26T01:39:21.824 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow rw", 2023-10-26T01:39:21.825 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow rwx" 2023-10-26T01:39:21.825 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:39:21.825 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:39:21.825 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:39:21.825 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "client.admin", 2023-10-26T01:39:21.826 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAewTll2yddMhAAO8bnXNevDGe/KEeNNa36pQ==", 2023-10-26T01:39:21.826 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:39:21.826 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow *", 2023-10-26T01:39:21.826 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow *", 2023-10-26T01:39:21.826 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow *", 2023-10-26T01:39:21.827 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:39:21.827 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:39:21.827 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:39:21.827 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:39:21.827 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "mgr.x", 2023-10-26T01:39:21.828 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAfwTllPu3FMxAAV8d5oRFXAeeZH7e1xzPEDg==", 2023-10-26T01:39:21.828 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:39:21.828 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow *", 2023-10-26T01:39:21.828 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow profile mgr", 2023-10-26T01:39:21.828 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:39:21.828 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:39:21.829 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:39:21.829 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:39:21.829 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "mgr.y", 2023-10-26T01:39:21.829 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAfwTllZc+DJxAAsZ2TWn8H7Kw/lNTc2qyKzA==", 2023-10-26T01:39:21.829 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:39:21.830 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow *", 2023-10-26T01:39:21.830 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow profile mgr", 2023-10-26T01:39:21.830 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:39:21.830 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:39:21.830 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:39:21.831 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:39:21.831 INFO:teuthology.orchestra.run.smithi169.stdout:} 2023-10-26T01:39:21.832 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:21.795+0000 7f5f52456640 1 -- 172.21.15.169:0/3095439097 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5f2c06ae40 msgr2=0x7f5f2c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:21.833 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:21.795+0000 7f5f52456640 1 --2- 172.21.15.169:0/3095439097 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5f2c06ae40 0x7f5f2c06d300 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c005fd0 tx=0x7f5f3c005850 comp rx=0 tx=0).stop 2023-10-26T01:39:21.833 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:21.795+0000 7f5f52456640 1 -- 172.21.15.169:0/3095439097 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f4c158850 msgr2=0x7f5f4c1e5750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:21.833 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:21.795+0000 7f5f52456640 1 --2- 172.21.15.169:0/3095439097 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f4c158850 0x7f5f4c1e5750 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f5f44037040 tx=0x7f5f44035db0 comp rx=0 tx=0).stop 2023-10-26T01:39:21.833 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:21.795+0000 7f5f52456640 1 -- 172.21.15.169:0/3095439097 shutdown_connections 2023-10-26T01:39:21.833 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:21.795+0000 7f5f52456640 1 --2- 172.21.15.169:0/3095439097 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5f2c06ae40 0x7f5f2c06d300 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:21.834 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:21.795+0000 7f5f52456640 1 --2- 172.21.15.169:0/3095439097 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f4c158850 0x7f5f4c1e5750 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:21.834 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:21.796+0000 7f5f52456640 1 --2- 172.21.15.169:0/3095439097 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5f4c0cd720 0x7f5f4c1f1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:21.834 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:21.796+0000 7f5f52456640 1 --2- 172.21.15.169:0/3095439097 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5f4c06ce20 0x7f5f4c14d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:21.834 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:21.796+0000 7f5f52456640 1 -- 172.21.15.169:0/3095439097 >> 172.21.15.169:0/3095439097 conn(0x7f5f4c069930 msgr2=0x7f5f4c0aeec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:21.834 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:21.796+0000 7f5f52456640 1 -- 172.21.15.169:0/3095439097 shutdown_connections 2023-10-26T01:39:21.835 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:21.796+0000 7f5f52456640 1 -- 172.21.15.169:0/3095439097 wait complete. 2023-10-26T01:39:21.835 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:39:22.051 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.048+0000 7f97f68c2640 1 -- 172.21.15.169:0/3398784195 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97f0158850 msgr2=0x7f97f00aab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:22.051 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.048+0000 7f97f68c2640 1 --2- 172.21.15.169:0/3398784195 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97f0158850 0x7f97f00aab40 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f97f00accf0 tx=0x7f97d802bdb0 comp rx=0 tx=0).stop 2023-10-26T01:39:22.051 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.049+0000 7f97f68c2640 1 -- 172.21.15.169:0/3398784195 shutdown_connections 2023-10-26T01:39:22.051 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.049+0000 7f97f68c2640 1 --2- 172.21.15.169:0/3398784195 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97f0158850 0x7f97f00aab40 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:22.052 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.049+0000 7f97f68c2640 1 --2- 172.21.15.169:0/3398784195 >> v2:172.21.15.170:3300/0 conn(0x7f97f006ce20 0x7f97f014e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:22.052 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.049+0000 7f97f68c2640 1 -- 172.21.15.169:0/3398784195 >> 172.21.15.169:0/3398784195 conn(0x7f97f0069930 msgr2=0x7f97f0069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:22.052 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.049+0000 7f97f68c2640 1 -- 172.21.15.169:0/3398784195 shutdown_connections 2023-10-26T01:39:22.052 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.049+0000 7f97f68c2640 1 -- 172.21.15.169:0/3398784195 wait complete. 2023-10-26T01:39:22.054 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.050+0000 7f97f68c2640 1 Processor -- start 2023-10-26T01:39:22.054 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.051+0000 7f97f68c2640 1 -- start start 2023-10-26T01:39:22.054 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.051+0000 7f97f68c2640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f97f006ad30 0x7f97f00aeec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:22.054 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.051+0000 7f97f68c2640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f97f006ce20 0x7f97f00700e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:22.054 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.051+0000 7f97f68c2640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97f0158850 0x7f97f00ad220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:22.055 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.051+0000 7f97f68c2640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97f01541e0 con 0x7f97f0158850 2023-10-26T01:39:22.055 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.051+0000 7f97f68c2640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97f014af90 con 0x7f97f006ce20 2023-10-26T01:39:22.055 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.051+0000 7f97f68c2640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97f0153880 con 0x7f97f006ad30 2023-10-26T01:39:22.055 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.052+0000 7f97f4e5f640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97f0158850 0x7f97f00ad220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:22.055 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.052+0000 7f97f4e5f640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97f0158850 0x7f97f00ad220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:53238/0 (socket says 172.21.15.169:53238) 2023-10-26T01:39:22.056 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.052+0000 7f97f4e5f640 1 -- 172.21.15.169:0/4293301932 learned_addr learned my addr 172.21.15.169:0/4293301932 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:39:22.056 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.052+0000 7f97effff640 1 --2- 172.21.15.169:0/4293301932 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f97f006ad30 0x7f97f00aeec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:22.056 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.052+0000 7f97ef7fe640 1 --2- 172.21.15.169:0/4293301932 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f97f006ce20 0x7f97f00700e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:22.058 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.052+0000 7f97f4e5f640 1 -- 172.21.15.169:0/4293301932 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f97f006ad30 msgr2=0x7f97f00aeec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:22.058 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.052+0000 7f97f4e5f640 1 --2- 172.21.15.169:0/4293301932 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f97f006ad30 0x7f97f00aeec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:22.058 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.052+0000 7f97f4e5f640 1 -- 172.21.15.169:0/4293301932 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f97f006ce20 msgr2=0x7f97f00700e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:22.058 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.052+0000 7f97f4e5f640 1 --2- 172.21.15.169:0/4293301932 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f97f006ce20 0x7f97f00700e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:22.058 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.053+0000 7f97f4e5f640 1 -- 172.21.15.169:0/4293301932 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97d8009560 con 0x7f97f0158850 2023-10-26T01:39:22.059 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.053+0000 7f97ef7fe640 1 --2- 172.21.15.169:0/4293301932 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f97f006ce20 0x7f97f00700e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T01:39:22.059 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.053+0000 7f97f4e5f640 1 --2- 172.21.15.169:0/4293301932 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97f0158850 0x7f97f00ad220 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f97e8001eb0 tx=0x7f97e800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:22.059 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.053+0000 7f97ecff9640 1 -- 172.21.15.169:0/4293301932 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e80052d0 con 0x7f97f0158850 2023-10-26T01:39:22.059 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.053+0000 7f97ecff9640 1 -- 172.21.15.169:0/4293301932 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f97e80035f0 con 0x7f97f0158850 2023-10-26T01:39:22.059 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.053+0000 7f97ecff9640 1 -- 172.21.15.169:0/4293301932 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e80039e0 con 0x7f97f0158850 2023-10-26T01:39:22.060 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.053+0000 7f97f68c2640 1 -- 172.21.15.169:0/4293301932 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97f0153b20 con 0x7f97f0158850 2023-10-26T01:39:22.060 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.053+0000 7f97f68c2640 1 -- 172.21.15.169:0/4293301932 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97f014abb0 con 0x7f97f0158850 2023-10-26T01:39:22.060 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.054+0000 7f97f68c2640 1 -- 172.21.15.169:0/4293301932 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97f0154320 con 0x7f97f0158850 2023-10-26T01:39:22.060 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.055+0000 7f97ecff9640 1 -- 172.21.15.169:0/4293301932 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f97e8003b40 con 0x7f97f0158850 2023-10-26T01:39:22.060 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.056+0000 7f97ecff9640 1 --2- 172.21.15.169:0/4293301932 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f97d006ae40 0x7f97d006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:22.061 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.056+0000 7f97ecff9640 1 -- 172.21.15.169:0/4293301932 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f97e8088a30 con 0x7f97f0158850 2023-10-26T01:39:22.061 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.057+0000 7f97effff640 1 --2- 172.21.15.169:0/4293301932 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f97d006ae40 0x7f97d006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:22.061 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.058+0000 7f97effff640 1 --2- 172.21.15.169:0/4293301932 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f97d006ae40 0x7f97d006d300 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f97e0005c40 tx=0x7f97e000a590 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:22.062 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.060+0000 7f97ecff9640 1 -- 172.21.15.169:0/4293301932 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f97f0154320 con 0x7f97f0158850 2023-10-26T01:39:22.252 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.250+0000 7f97f68c2640 1 -- 172.21.15.169:0/4293301932 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f97f0154320 con 0x7f97f0158850 2023-10-26T01:39:22.253 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.251+0000 7f97ecff9640 1 -- 172.21.15.169:0/4293301932 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v147) v1 ==== 74+0+12916 (secure 0 0 0) 0x7f97f0154320 con 0x7f97f0158850 2023-10-26T01:39:22.253 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:39:22.254 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":147,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:39:21.299004+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:39:22.256 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.254+0000 7f97f68c2640 1 -- 172.21.15.169:0/4293301932 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f97d006ae40 msgr2=0x7f97d006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:22.256 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.254+0000 7f97f68c2640 1 --2- 172.21.15.169:0/4293301932 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f97d006ae40 0x7f97d006d300 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f97e0005c40 tx=0x7f97e000a590 comp rx=0 tx=0).stop 2023-10-26T01:39:22.256 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.254+0000 7f97f68c2640 1 -- 172.21.15.169:0/4293301932 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97f0158850 msgr2=0x7f97f00ad220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:22.256 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.254+0000 7f97f68c2640 1 --2- 172.21.15.169:0/4293301932 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97f0158850 0x7f97f00ad220 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f97e8001eb0 tx=0x7f97e800c890 comp rx=0 tx=0).stop 2023-10-26T01:39:22.257 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.254+0000 7f97f68c2640 1 -- 172.21.15.169:0/4293301932 shutdown_connections 2023-10-26T01:39:22.257 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.254+0000 7f97f68c2640 1 --2- 172.21.15.169:0/4293301932 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f97d006ae40 0x7f97d006d300 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:22.257 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.254+0000 7f97f68c2640 1 --2- 172.21.15.169:0/4293301932 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97f0158850 0x7f97f00ad220 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:22.257 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.254+0000 7f97f68c2640 1 --2- 172.21.15.169:0/4293301932 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f97f006ce20 0x7f97f00700e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:22.257 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.254+0000 7f97f68c2640 1 --2- 172.21.15.169:0/4293301932 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f97f006ad30 0x7f97f00aeec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:22.257 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.254+0000 7f97f68c2640 1 -- 172.21.15.169:0/4293301932 >> 172.21.15.169:0/4293301932 conn(0x7f97f0069930 msgr2=0x7f97f014d810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:22.258 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.254+0000 7f97f68c2640 1 -- 172.21.15.169:0/4293301932 shutdown_connections 2023-10-26T01:39:22.258 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.254+0000 7f97f68c2640 1 -- 172.21.15.169:0/4293301932 wait complete. 2023-10-26T01:39:22.276 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T01:30:32.200808+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '109', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T01:39:22.277 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T01:39:22.492 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.490+0000 7fcdf3fff640 1 -- 172.21.15.169:0/1560824277 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcdf40b0610 msgr2=0x7fcdf4158970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:22.492 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.490+0000 7fcdf3fff640 1 --2- 172.21.15.169:0/1560824277 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcdf40b0610 0x7fcdf4158970 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fcddc005fd0 tx=0x7fcddc02be20 comp rx=0 tx=0).stop 2023-10-26T01:39:22.492 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.490+0000 7fcdf3fff640 1 -- 172.21.15.169:0/1560824277 shutdown_connections 2023-10-26T01:39:22.493 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.490+0000 7fcdf3fff640 1 --2- 172.21.15.169:0/1560824277 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcdf40b0610 0x7fcdf4158970 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:22.493 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.490+0000 7fcdf3fff640 1 --2- 172.21.15.169:0/1560824277 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcdf414f2f0 0x7fcdf40b00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:22.493 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.490+0000 7fcdf3fff640 1 -- 172.21.15.169:0/1560824277 >> 172.21.15.169:0/1560824277 conn(0x7fcdf4069510 msgr2=0x7fcdf4069920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:22.493 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.490+0000 7fcdf3fff640 1 -- 172.21.15.169:0/1560824277 shutdown_connections 2023-10-26T01:39:22.493 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.490+0000 7fcdf3fff640 1 -- 172.21.15.169:0/1560824277 wait complete. 2023-10-26T01:39:22.494 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.492+0000 7fcdf3fff640 1 Processor -- start 2023-10-26T01:39:22.494 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.492+0000 7fcdf3fff640 1 -- start start 2023-10-26T01:39:22.495 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.493+0000 7fcdf3fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcdf40b0610 0x7fcdf40cd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:22.496 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.493+0000 7fcdf3fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcdf414f2f0 0x7fcdf414d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:22.496 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.493+0000 7fcdf3fff640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcdf4158eb0 0x7fcdf406fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:22.496 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.493+0000 7fcdf3fff640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf4146bc0 con 0x7fcdf414f2f0 2023-10-26T01:39:22.496 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.493+0000 7fcdf3fff640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf40714e0 con 0x7fcdf40b0610 2023-10-26T01:39:22.496 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.493+0000 7fcdf3fff640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf40717c0 con 0x7fcdf4158eb0 2023-10-26T01:39:22.497 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.494+0000 7fcdebfff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcdf414f2f0 0x7fcdf414d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:22.497 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.494+0000 7fcdebfff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcdf414f2f0 0x7fcdf414d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:53250/0 (socket says 172.21.15.169:53250) 2023-10-26T01:39:22.497 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.494+0000 7fcdebfff640 1 -- 172.21.15.169:0/4154066989 learned_addr learned my addr 172.21.15.169:0/4154066989 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:39:22.497 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.494+0000 7fcdf2ffd640 1 --2- 172.21.15.169:0/4154066989 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcdf40b0610 0x7fcdf40cd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:22.497 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.494+0000 7fcdf37fe640 1 --2- 172.21.15.169:0/4154066989 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcdf4158eb0 0x7fcdf406fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:22.499 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.494+0000 7fcdebfff640 1 -- 172.21.15.169:0/4154066989 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcdf4158eb0 msgr2=0x7fcdf406fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:22.499 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.494+0000 7fcdebfff640 1 --2- 172.21.15.169:0/4154066989 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcdf4158eb0 0x7fcdf406fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:22.499 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.494+0000 7fcdebfff640 1 -- 172.21.15.169:0/4154066989 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcdf40b0610 msgr2=0x7fcdf40cd9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:22.499 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.494+0000 7fcdebfff640 1 --2- 172.21.15.169:0/4154066989 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcdf40b0610 0x7fcdf40cd9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:22.499 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.494+0000 7fcdebfff640 1 -- 172.21.15.169:0/4154066989 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcddc009560 con 0x7fcdf414f2f0 2023-10-26T01:39:22.500 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.494+0000 7fcdf2ffd640 1 --2- 172.21.15.169:0/4154066989 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcdf40b0610 0x7fcdf40cd9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T01:39:22.500 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.495+0000 7fcdebfff640 1 --2- 172.21.15.169:0/4154066989 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcdf414f2f0 0x7fcdf414d3f0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7fcddc006010 tx=0x7fcddc02da80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:22.500 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.495+0000 7fcdf0ff9640 1 -- 172.21.15.169:0/4154066989 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcddc02de80 con 0x7fcdf414f2f0 2023-10-26T01:39:22.500 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.495+0000 7fcdf3fff640 1 -- 172.21.15.169:0/4154066989 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdf40adc70 con 0x7fcdf414f2f0 2023-10-26T01:39:22.500 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.495+0000 7fcdf0ff9640 1 -- 172.21.15.169:0/4154066989 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fcddc0036e0 con 0x7fcdf414f2f0 2023-10-26T01:39:22.501 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.495+0000 7fcdf0ff9640 1 -- 172.21.15.169:0/4154066989 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcddc003ad0 con 0x7fcdf414f2f0 2023-10-26T01:39:22.501 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.495+0000 7fcdf3fff640 1 -- 172.21.15.169:0/4154066989 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdf40ad8f0 con 0x7fcdf414f2f0 2023-10-26T01:39:22.501 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.496+0000 7fcdf3fff640 1 -- 172.21.15.169:0/4154066989 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdf40ae190 con 0x7fcdf414f2f0 2023-10-26T01:39:22.501 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.497+0000 7fcdf0ff9640 1 -- 172.21.15.169:0/4154066989 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fcddc003c30 con 0x7fcdf414f2f0 2023-10-26T01:39:22.502 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.498+0000 7fcdf0ff9640 1 --2- 172.21.15.169:0/4154066989 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcdd406ae40 0x7fcdd406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:22.502 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.498+0000 7fcdf0ff9640 1 -- 172.21.15.169:0/4154066989 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fcddc0abf80 con 0x7fcdf414f2f0 2023-10-26T01:39:22.502 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.499+0000 7fcdf2ffd640 1 --2- 172.21.15.169:0/4154066989 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcdd406ae40 0x7fcdd406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:22.502 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.500+0000 7fcdf2ffd640 1 --2- 172.21.15.169:0/4154066989 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcdd406ae40 0x7fcdd406d300 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fcde4009c20 tx=0x7fcde4005f90 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:22.505 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.503+0000 7fcdf0ff9640 1 -- 172.21.15.169:0/4154066989 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcdf40ae190 con 0x7fcdf414f2f0 2023-10-26T01:39:22.694 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.692+0000 7fcdf3fff640 1 -- 172.21.15.169:0/4154066989 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcdf40ae190 con 0x7fcdf414f2f0 2023-10-26T01:39:22.694 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.693+0000 7fcdf0ff9640 1 -- 172.21.15.169:0/4154066989 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v147) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcdf40ae190 con 0x7fcdf414f2f0 2023-10-26T01:39:22.695 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 1 2023-10-26T01:39:22.697 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.695+0000 7fcdf3fff640 1 -- 172.21.15.169:0/4154066989 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcdd406ae40 msgr2=0x7fcdd406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:22.697 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.695+0000 7fcdf3fff640 1 --2- 172.21.15.169:0/4154066989 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcdd406ae40 0x7fcdd406d300 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fcde4009c20 tx=0x7fcde4005f90 comp rx=0 tx=0).stop 2023-10-26T01:39:22.698 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.695+0000 7fcdf3fff640 1 -- 172.21.15.169:0/4154066989 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcdf414f2f0 msgr2=0x7fcdf414d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:22.698 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.695+0000 7fcdf3fff640 1 --2- 172.21.15.169:0/4154066989 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcdf414f2f0 0x7fcdf414d3f0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7fcddc006010 tx=0x7fcddc02da80 comp rx=0 tx=0).stop 2023-10-26T01:39:22.698 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.695+0000 7fcdf3fff640 1 -- 172.21.15.169:0/4154066989 shutdown_connections 2023-10-26T01:39:22.698 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.695+0000 7fcdf3fff640 1 --2- 172.21.15.169:0/4154066989 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcdd406ae40 0x7fcdd406d300 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:22.699 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.695+0000 7fcdf3fff640 1 --2- 172.21.15.169:0/4154066989 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcdf414f2f0 0x7fcdf414d3f0 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:22.699 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.695+0000 7fcdf3fff640 1 --2- 172.21.15.169:0/4154066989 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcdf4158eb0 0x7fcdf406fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:22.699 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.695+0000 7fcdf3fff640 1 --2- 172.21.15.169:0/4154066989 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcdf40b0610 0x7fcdf40cd9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:22.699 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.695+0000 7fcdf3fff640 1 -- 172.21.15.169:0/4154066989 >> 172.21.15.169:0/4154066989 conn(0x7fcdf4069510 msgr2=0x7fcdf4157c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:22.699 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.696+0000 7fcdf3fff640 1 -- 172.21.15.169:0/4154066989 shutdown_connections 2023-10-26T01:39:22.699 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.696+0000 7fcdf3fff640 1 -- 172.21.15.169:0/4154066989 wait complete. 2023-10-26T01:39:22.716 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-26T01:39:22.934 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.932+0000 7f283bab4640 1 -- 172.21.15.169:0/57859949 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f28340cd720 msgr2=0x7f28340cdb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:22.935 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.932+0000 7f283bab4640 1 --2- 172.21.15.169:0/57859949 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f28340cd720 0x7f28340cdb90 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f28340acd20 tx=0x7f282c02c1d0 comp rx=0 tx=0).stop 2023-10-26T01:39:22.935 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.932+0000 7f283bab4640 1 -- 172.21.15.169:0/57859949 shutdown_connections 2023-10-26T01:39:22.935 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.932+0000 7f283bab4640 1 --2- 172.21.15.169:0/57859949 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f28340cd720 0x7f28340cdb90 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:22.935 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.932+0000 7f283bab4640 1 --2- 172.21.15.169:0/57859949 >> v2:172.21.15.170:3300/0 conn(0x7f2834158850 0x7f283414e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:22.936 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.932+0000 7f283bab4640 1 -- 172.21.15.169:0/57859949 >> 172.21.15.169:0/57859949 conn(0x7f2834069930 msgr2=0x7f2834069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:22.936 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.932+0000 7f283bab4640 1 -- 172.21.15.169:0/57859949 shutdown_connections 2023-10-26T01:39:22.936 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.933+0000 7f283bab4640 1 -- 172.21.15.169:0/57859949 wait complete. 2023-10-26T01:39:22.938 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.934+0000 7f283bab4640 1 Processor -- start 2023-10-26T01:39:22.938 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.934+0000 7f283bab4640 1 -- start start 2023-10-26T01:39:22.938 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.935+0000 7f283bab4640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f283406ce20 0x7f2834142740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:22.938 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.935+0000 7f283bab4640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f28340cd720 0x7f2834157350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:22.939 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.935+0000 7f283bab4640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2834158850 0x7f28341563b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:22.939 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.935+0000 7f283bab4640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2834142c80 con 0x7f28340cd720 2023-10-26T01:39:22.939 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.935+0000 7f283bab4640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2834148410 con 0x7f2834158850 2023-10-26T01:39:22.939 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.935+0000 7f283bab4640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28340ae690 con 0x7f283406ce20 2023-10-26T01:39:22.939 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.936+0000 7f283904f640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f28340cd720 0x7f2834157350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:22.940 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.936+0000 7f283904f640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f28340cd720 0x7f2834157350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:53264/0 (socket says 172.21.15.169:53264) 2023-10-26T01:39:22.940 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.936+0000 7f283904f640 1 -- 172.21.15.169:0/3708269785 learned_addr learned my addr 172.21.15.169:0/3708269785 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:39:22.940 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.936+0000 7f2839850640 1 --2- 172.21.15.169:0/3708269785 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f283406ce20 0x7f2834142740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:22.940 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.936+0000 7f283a051640 1 --2- 172.21.15.169:0/3708269785 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2834158850 0x7f28341563b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:22.940 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.936+0000 7f283904f640 1 -- 172.21.15.169:0/3708269785 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f283406ce20 msgr2=0x7f2834142740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:22.941 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.936+0000 7f283904f640 1 --2- 172.21.15.169:0/3708269785 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f283406ce20 0x7f2834142740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:22.941 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.936+0000 7f283904f640 1 -- 172.21.15.169:0/3708269785 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2834158850 msgr2=0x7f28341563b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:22.941 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.936+0000 7f283904f640 1 --2- 172.21.15.169:0/3708269785 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2834158850 0x7f28341563b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:22.941 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.936+0000 7f283904f640 1 -- 172.21.15.169:0/3708269785 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f282c009900 con 0x7f28340cd720 2023-10-26T01:39:22.941 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.936+0000 7f283a051640 1 --2- 172.21.15.169:0/3708269785 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2834158850 0x7f28341563b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:39:22.943 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.936+0000 7f283904f640 1 --2- 172.21.15.169:0/3708269785 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f28340cd720 0x7f2834157350 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f282400d700 tx=0x7f282400dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:22.943 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.937+0000 7f282a7fc640 1 -- 172.21.15.169:0/3708269785 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2824005220 con 0x7f28340cd720 2023-10-26T01:39:22.943 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.937+0000 7f282a7fc640 1 -- 172.21.15.169:0/3708269785 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2824005470 con 0x7f28340cd720 2023-10-26T01:39:22.943 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.937+0000 7f283bab4640 1 -- 172.21.15.169:0/3708269785 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28340ae3b0 con 0x7f28340cd720 2023-10-26T01:39:22.944 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.937+0000 7f282a7fc640 1 -- 172.21.15.169:0/3708269785 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2824009a20 con 0x7f28340cd720 2023-10-26T01:39:22.944 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.937+0000 7f283bab4640 1 -- 172.21.15.169:0/3708269785 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2834161eb0 con 0x7f28340cd720 2023-10-26T01:39:22.944 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.938+0000 7f283bab4640 1 -- 172.21.15.169:0/3708269785 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28340aec00 con 0x7f28340cd720 2023-10-26T01:39:22.944 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.939+0000 7f282a7fc640 1 -- 172.21.15.169:0/3708269785 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f2824009b80 con 0x7f28340cd720 2023-10-26T01:39:22.944 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.940+0000 7f282a7fc640 1 --2- 172.21.15.169:0/3708269785 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f281806ae40 0x7f281806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:22.945 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.940+0000 7f282a7fc640 1 -- 172.21.15.169:0/3708269785 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f2824089330 con 0x7f28340cd720 2023-10-26T01:39:22.945 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.941+0000 7f2839850640 1 --2- 172.21.15.169:0/3708269785 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f281806ae40 0x7f281806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:22.945 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.942+0000 7f2839850640 1 --2- 172.21.15.169:0/3708269785 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f281806ae40 0x7f281806d300 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f2830007fd0 tx=0x7f2830009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:22.946 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:22.945+0000 7f282a7fc640 1 -- 172.21.15.169:0/3708269785 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f28340aec00 con 0x7f28340cd720 2023-10-26T01:39:23.141 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:23.139+0000 7f283bab4640 1 -- 172.21.15.169:0/3708269785 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f28340aec00 con 0x7f28340cd720 2023-10-26T01:39:23.141 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:23.140+0000 7f282a7fc640 1 -- 172.21.15.169:0/3708269785 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v147) v1 ==== 84+0+65 (secure 0 0 0) 0x7f2824017060 con 0x7f28340cd720 2023-10-26T01:39:23.142 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:39:23.142 INFO:teuthology.orchestra.run.smithi169.stdout:[ 2023-10-26T01:39:23.142 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:39:23.142 INFO:teuthology.orchestra.run.smithi169.stdout: "poolnum": 1, 2023-10-26T01:39:23.143 INFO:teuthology.orchestra.run.smithi169.stdout: "poolname": ".mgr" 2023-10-26T01:39:23.143 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:39:23.143 INFO:teuthology.orchestra.run.smithi169.stdout:] 2023-10-26T01:39:23.145 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:23.142+0000 7f283bab4640 1 -- 172.21.15.169:0/3708269785 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f281806ae40 msgr2=0x7f281806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:23.145 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:23.142+0000 7f283bab4640 1 --2- 172.21.15.169:0/3708269785 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f281806ae40 0x7f281806d300 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f2830007fd0 tx=0x7f2830009040 comp rx=0 tx=0).stop 2023-10-26T01:39:23.145 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:23.143+0000 7f283bab4640 1 -- 172.21.15.169:0/3708269785 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f28340cd720 msgr2=0x7f2834157350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:23.145 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:23.143+0000 7f283bab4640 1 --2- 172.21.15.169:0/3708269785 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f28340cd720 0x7f2834157350 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f282400d700 tx=0x7f282400dbc0 comp rx=0 tx=0).stop 2023-10-26T01:39:23.145 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:23.143+0000 7f283bab4640 1 -- 172.21.15.169:0/3708269785 shutdown_connections 2023-10-26T01:39:23.146 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:23.143+0000 7f283bab4640 1 --2- 172.21.15.169:0/3708269785 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f281806ae40 0x7f281806d300 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:23.146 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:23.143+0000 7f283bab4640 1 --2- 172.21.15.169:0/3708269785 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f28340cd720 0x7f2834157350 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:23.146 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:23.143+0000 7f283bab4640 1 --2- 172.21.15.169:0/3708269785 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2834158850 0x7f28341563b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:23.146 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:23.143+0000 7f283bab4640 1 --2- 172.21.15.169:0/3708269785 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f283406ce20 0x7f2834142740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:23.147 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:23.143+0000 7f283bab4640 1 -- 172.21.15.169:0/3708269785 >> 172.21.15.169:0/3708269785 conn(0x7f2834069930 msgr2=0x7f2834150620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:23.147 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:23.143+0000 7f283bab4640 1 -- 172.21.15.169:0/3708269785 shutdown_connections 2023-10-26T01:39:23.147 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:23.143+0000 7f283bab4640 1 -- 172.21.15.169:0/3708269785 wait complete. 2023-10-26T01:39:23.165 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-26T01:39:23.165 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-26T01:39:23.392 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:23.390+0000 7fdd022de640 1 -- 172.21.15.169:0/807395154 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdce4005180 msgr2=0x7fdce400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:23.392 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:23.390+0000 7fdd022de640 1 --2- 172.21.15.169:0/807395154 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdce4005180 0x7fdce400b980 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7fdcf4006fa0 tx=0x7fdce40057f0 comp rx=0 tx=0).stop 2023-10-26T01:39:23.393 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:23.390+0000 7fdd022de640 1 -- 172.21.15.169:0/807395154 shutdown_connections 2023-10-26T01:39:23.393 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:23.390+0000 7fdd022de640 1 --2- 172.21.15.169:0/807395154 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdce4005180 0x7fdce400b980 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:23.393 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:23.390+0000 7fdd022de640 1 -- 172.21.15.169:0/807395154 >> 172.21.15.169:0/807395154 conn(0x7fdcfc069930 msgr2=0x7fdcfc069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:23.393 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:23.390+0000 7fdd022de640 1 -- 172.21.15.169:0/807395154 shutdown_connections 2023-10-26T01:39:23.394 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:23.390+0000 7fdd022de640 1 -- 172.21.15.169:0/807395154 wait complete. 2023-10-26T01:39:23.396 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:23.392+0000 7fdd022de640 1 Processor -- start 2023-10-26T01:39:23.396 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:23.392+0000 7fdd022de640 1 -- start start 2023-10-26T01:39:23.397 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:23.393+0000 7fdd022de640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdce4005180 0x7fdcfc15b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:23.397 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:23.393+0000 7fdd022de640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdcfc1f3200 0x7fdcfc1568c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:23.397 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:23.393+0000 7fdd022de640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdcfc156e00 0x7fdcfc098190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:23.397 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:23.393+0000 7fdd022de640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcfc158670 con 0x7fdcfc1f3200 2023-10-26T01:39:23.398 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:23.393+0000 7fdd022de640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcfc099370 con 0x7fdce4005180 2023-10-26T01:39:23.398 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:23.393+0000 7fdd022de640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcfc157280 con 0x7fdcfc156e00 2023-10-26T01:39:23.398 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:23.393+0000 7fdcfb7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdcfc1f3200 0x7fdcfc1568c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:23.398 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:23.393+0000 7fdcfb7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdcfc1f3200 0x7fdcfc1568c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:53290/0 (socket says 172.21.15.169:53290) 2023-10-26T01:39:23.398 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:23.393+0000 7fdcfb7fe640 1 -- 172.21.15.169:0/3600107243 learned_addr learned my addr 172.21.15.169:0/3600107243 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:39:23.399 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:23.394+0000 7fdcfbfff640 1 --2- 172.21.15.169:0/3600107243 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdce4005180 0x7fdcfc15b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:23.399 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:23.394+0000 7fdd0087b640 1 --2- 172.21.15.169:0/3600107243 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdcfc156e00 0x7fdcfc098190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:23.399 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:23.394+0000 7fdcfb7fe640 1 -- 172.21.15.169:0/3600107243 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdcfc156e00 msgr2=0x7fdcfc098190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:23.399 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:23.394+0000 7fdcfb7fe640 1 --2- 172.21.15.169:0/3600107243 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdcfc156e00 0x7fdcfc098190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:23.399 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:23.394+0000 7fdcfb7fe640 1 -- 172.21.15.169:0/3600107243 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdce4005180 msgr2=0x7fdcfc15b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:23.400 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:23.394+0000 7fdcfb7fe640 1 --2- 172.21.15.169:0/3600107243 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdce4005180 0x7fdcfc15b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:23.400 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:23.394+0000 7fdcfb7fe640 1 -- 172.21.15.169:0/3600107243 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcfc099580 con 0x7fdcfc1f3200 2023-10-26T01:39:23.400 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:23.394+0000 7fdd0087b640 1 --2- 172.21.15.169:0/3600107243 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdcfc156e00 0x7fdcfc098190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:39:23.400 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:23.394+0000 7fdcfbfff640 1 --2- 172.21.15.169:0/3600107243 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdce4005180 0x7fdcfc15b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:39:23.401 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:23.394+0000 7fdcfb7fe640 1 --2- 172.21.15.169:0/3600107243 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdcfc1f3200 0x7fdcfc1568c0 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7fdcf000d7f0 tx=0x7fdcf000dcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:23.401 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:23.394+0000 7fdcf8ff9640 1 -- 172.21.15.169:0/3600107243 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf000b8d0 con 0x7fdcfc1f3200 2023-10-26T01:39:23.401 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:23.394+0000 7fdcf8ff9640 1 -- 172.21.15.169:0/3600107243 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fdcf000ba30 con 0x7fdcfc1f3200 2023-10-26T01:39:23.401 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:23.394+0000 7fdd022de640 1 -- 172.21.15.169:0/3600107243 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcfc099860 con 0x7fdcfc1f3200 2023-10-26T01:39:23.401 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:23.394+0000 7fdcf8ff9640 1 -- 172.21.15.169:0/3600107243 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf000be60 con 0x7fdcfc1f3200 2023-10-26T01:39:23.401 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:23.394+0000 7fdd022de640 1 -- 172.21.15.169:0/3600107243 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcfc099e20 con 0x7fdcfc1f3200 2023-10-26T01:39:23.402 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:23.396+0000 7fdd022de640 1 -- 172.21.15.169:0/3600107243 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcfc066af0 con 0x7fdcfc1f3200 2023-10-26T01:39:23.403 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:23.397+0000 7fdcf8ff9640 1 -- 172.21.15.169:0/3600107243 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fdcf001b400 con 0x7fdcfc1f3200 2023-10-26T01:39:23.403 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:23.397+0000 7fdcf8ff9640 1 --2- 172.21.15.169:0/3600107243 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fdce806ae40 0x7fdce806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:23.403 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:23.398+0000 7fdcf8ff9640 1 -- 172.21.15.169:0/3600107243 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fdcf00897d0 con 0x7fdcfc1f3200 2023-10-26T01:39:23.403 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:23.398+0000 7fdcfbfff640 1 --2- 172.21.15.169:0/3600107243 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fdce806ae40 0x7fdce806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:23.403 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:23.399+0000 7fdcfbfff640 1 --2- 172.21.15.169:0/3600107243 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fdce806ae40 0x7fdce806d300 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fdcec009a10 tx=0x7fdcec005f90 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:23.404 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:23.402+0000 7fdcf8ff9640 1 -- 172.21.15.169:0/3600107243 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdcf0054bd0 con 0x7fdcfc1f3200 2023-10-26T01:39:23.593 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:23.591+0000 7fdd022de640 1 -- 172.21.15.169:0/3600107243 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fdcfc0aea90 con 0x7fdcfc1f3200 2023-10-26T01:39:24.309 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:24.307+0000 7fdcf8ff9640 1 -- 172.21.15.169:0/3600107243 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v148) v1 ==== 138+0+0 (secure 0 0 0) 0x7fdcf00581f0 con 0x7fdcfc1f3200 2023-10-26T01:39:24.310 INFO:teuthology.orchestra.run.smithi169.stderr:pool 'cephfs_metadata' created 2023-10-26T01:39:24.313 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:24.311+0000 7fdd022de640 1 -- 172.21.15.169:0/3600107243 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fdce806ae40 msgr2=0x7fdce806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:24.313 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:24.311+0000 7fdd022de640 1 --2- 172.21.15.169:0/3600107243 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fdce806ae40 0x7fdce806d300 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fdcec009a10 tx=0x7fdcec005f90 comp rx=0 tx=0).stop 2023-10-26T01:39:24.313 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:24.311+0000 7fdd022de640 1 -- 172.21.15.169:0/3600107243 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdcfc1f3200 msgr2=0x7fdcfc1568c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:24.314 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:24.311+0000 7fdd022de640 1 --2- 172.21.15.169:0/3600107243 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdcfc1f3200 0x7fdcfc1568c0 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7fdcf000d7f0 tx=0x7fdcf000dcb0 comp rx=0 tx=0).stop 2023-10-26T01:39:24.314 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:24.311+0000 7fdd022de640 1 -- 172.21.15.169:0/3600107243 shutdown_connections 2023-10-26T01:39:24.314 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:24.311+0000 7fdd022de640 1 --2- 172.21.15.169:0/3600107243 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fdce806ae40 0x7fdce806d300 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:24.314 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:24.311+0000 7fdd022de640 1 --2- 172.21.15.169:0/3600107243 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdcfc1f3200 0x7fdcfc1568c0 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:24.315 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:24.311+0000 7fdd022de640 1 --2- 172.21.15.169:0/3600107243 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdcfc156e00 0x7fdcfc098190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:24.315 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:24.311+0000 7fdd022de640 1 --2- 172.21.15.169:0/3600107243 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdce4005180 0x7fdcfc15b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:24.315 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:24.311+0000 7fdd022de640 1 -- 172.21.15.169:0/3600107243 >> 172.21.15.169:0/3600107243 conn(0x7fdcfc069930 msgr2=0x7fdcfc0ae260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:24.315 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:24.311+0000 7fdd022de640 1 -- 172.21.15.169:0/3600107243 shutdown_connections 2023-10-26T01:39:24.315 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:24.311+0000 7fdd022de640 1 -- 172.21.15.169:0/3600107243 wait complete. 2023-10-26T01:39:24.338 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-26T01:39:24.561 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:24.559+0000 7f79b6739640 1 -- 172.21.15.169:0/3863471393 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79b00b0c80 msgr2=0x7f79b00cd710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:24.562 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:24.559+0000 7f79b6739640 1 --2- 172.21.15.169:0/3863471393 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79b00b0c80 0x7f79b00cd710 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f79b00b2670 tx=0x7f79a802c110 comp rx=0 tx=0).stop 2023-10-26T01:39:24.562 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:24.559+0000 7f79b6739640 1 -- 172.21.15.169:0/3863471393 shutdown_connections 2023-10-26T01:39:24.562 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:24.559+0000 7f79b6739640 1 --2- 172.21.15.169:0/3863471393 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79b00b0c80 0x7f79b00cd710 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:24.563 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:24.559+0000 7f79b6739640 1 --2- 172.21.15.169:0/3863471393 >> v2:172.21.15.170:3300/0 conn(0x7f79b01597c0 0x7f79b00ac200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:24.563 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:24.559+0000 7f79b6739640 1 -- 172.21.15.169:0/3863471393 >> 172.21.15.169:0/3863471393 conn(0x7f79b0069930 msgr2=0x7f79b0069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:24.563 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:24.560+0000 7f79b6739640 1 -- 172.21.15.169:0/3863471393 shutdown_connections 2023-10-26T01:39:24.563 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:24.560+0000 7f79b6739640 1 -- 172.21.15.169:0/3863471393 wait complete. 2023-10-26T01:39:24.564 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:24.561+0000 7f79b6739640 1 Processor -- start 2023-10-26T01:39:24.565 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:24.561+0000 7f79b6739640 1 -- start start 2023-10-26T01:39:24.565 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:24.562+0000 7f79b6739640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79b00b0c80 0x7f79b00aec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:24.565 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:24.562+0000 7f79b6739640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f79b01593e0 0x7f79b01f21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:24.565 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:24.562+0000 7f79b6739640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f79b01597c0 0x7f79b01f2730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:24.565 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:24.562+0000 7f79b6739640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79b01f5c10 con 0x7f79b00b0c80 2023-10-26T01:39:24.566 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:24.562+0000 7f79b6739640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79b00af350 con 0x7f79b01593e0 2023-10-26T01:39:24.566 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:24.562+0000 7f79b6739640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79b01f6030 con 0x7f79b01597c0 2023-10-26T01:39:24.566 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:24.563+0000 7f79affff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79b00b0c80 0x7f79b00aec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:24.566 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:24.563+0000 7f79affff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79b00b0c80 0x7f79b00aec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:37278/0 (socket says 172.21.15.169:37278) 2023-10-26T01:39:24.566 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:24.563+0000 7f79affff640 1 -- 172.21.15.169:0/2406436536 learned_addr learned my addr 172.21.15.169:0/2406436536 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:39:24.568 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:24.563+0000 7f79b4cd6640 1 --2- 172.21.15.169:0/2406436536 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f79b01597c0 0x7f79b01f2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:24.568 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:24.563+0000 7f79a7fff640 1 --2- 172.21.15.169:0/2406436536 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f79b01593e0 0x7f79b01f21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:24.568 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:24.563+0000 7f79affff640 1 -- 172.21.15.169:0/2406436536 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f79b01597c0 msgr2=0x7f79b01f2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:24.568 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:24.563+0000 7f79affff640 1 --2- 172.21.15.169:0/2406436536 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f79b01597c0 0x7f79b01f2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:24.568 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:24.563+0000 7f79affff640 1 -- 172.21.15.169:0/2406436536 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f79b01593e0 msgr2=0x7f79b01f21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:24.569 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:24.563+0000 7f79affff640 1 --2- 172.21.15.169:0/2406436536 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f79b01593e0 0x7f79b01f21f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:24.569 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:24.563+0000 7f79affff640 1 -- 172.21.15.169:0/2406436536 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79a8009840 con 0x7f79b00b0c80 2023-10-26T01:39:24.569 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:24.564+0000 7f79affff640 1 --2- 172.21.15.169:0/2406436536 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79b00b0c80 0x7f79b00aec50 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f79a000c500 tx=0x7f79a000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:24.569 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:24.564+0000 7f79ad7fa640 1 -- 172.21.15.169:0/2406436536 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a0012830 con 0x7f79b00b0c80 2023-10-26T01:39:24.569 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:24.564+0000 7f79ad7fa640 1 -- 172.21.15.169:0/2406436536 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f79a0012a80 con 0x7f79b00b0c80 2023-10-26T01:39:24.570 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:24.564+0000 7f79ad7fa640 1 -- 172.21.15.169:0/2406436536 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a0012e50 con 0x7f79b00b0c80 2023-10-26T01:39:24.570 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:24.564+0000 7f79b6739640 1 -- 172.21.15.169:0/2406436536 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79b01f5270 con 0x7f79b00b0c80 2023-10-26T01:39:24.570 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:24.564+0000 7f79b6739640 1 -- 172.21.15.169:0/2406436536 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79b01f5830 con 0x7f79b00b0c80 2023-10-26T01:39:24.570 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:24.565+0000 7f79b6739640 1 -- 172.21.15.169:0/2406436536 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79b01f5e80 con 0x7f79b00b0c80 2023-10-26T01:39:24.570 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:24.566+0000 7f79ad7fa640 1 -- 172.21.15.169:0/2406436536 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f79a001b400 con 0x7f79b00b0c80 2023-10-26T01:39:24.571 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:24.567+0000 7f79ad7fa640 1 --2- 172.21.15.169:0/2406436536 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f799006ae40 0x7f799006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:24.571 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:24.567+0000 7f79ad7fa640 1 -- 172.21.15.169:0/2406436536 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f79a0089860 con 0x7f79b00b0c80 2023-10-26T01:39:24.572 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:24.568+0000 7f79a7fff640 1 --2- 172.21.15.169:0/2406436536 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f799006ae40 0x7f799006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:24.572 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:24.569+0000 7f79a7fff640 1 --2- 172.21.15.169:0/2406436536 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f799006ae40 0x7f799006d300 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f7998005fd0 tx=0x7f7998005bc0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:24.574 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:24.572+0000 7f79ad7fa640 1 -- 172.21.15.169:0/2406436536 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f79a0054b00 con 0x7f79b00b0c80 2023-10-26T01:39:24.764 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:24.762+0000 7f79b6739640 1 -- 172.21.15.169:0/2406436536 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f79b004db10 con 0x7f79b00b0c80 2023-10-26T01:39:25.328 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:25.326+0000 7f79ad7fa640 1 -- 172.21.15.169:0/2406436536 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v149) v1 ==== 170+0+0 (secure 0 0 0) 0x7f79a0058120 con 0x7f79b00b0c80 2023-10-26T01:39:25.328 INFO:teuthology.orchestra.run.smithi169.stderr:pool 'cephfs_data' created 2023-10-26T01:39:25.333 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:25.330+0000 7f79b6739640 1 -- 172.21.15.169:0/2406436536 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f799006ae40 msgr2=0x7f799006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:25.333 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:25.331+0000 7f79b6739640 1 --2- 172.21.15.169:0/2406436536 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f799006ae40 0x7f799006d300 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f7998005fd0 tx=0x7f7998005bc0 comp rx=0 tx=0).stop 2023-10-26T01:39:25.333 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:25.331+0000 7f79b6739640 1 -- 172.21.15.169:0/2406436536 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79b00b0c80 msgr2=0x7f79b00aec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:25.333 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:25.331+0000 7f79b6739640 1 --2- 172.21.15.169:0/2406436536 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79b00b0c80 0x7f79b00aec50 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f79a000c500 tx=0x7f79a000c9c0 comp rx=0 tx=0).stop 2023-10-26T01:39:25.334 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:25.331+0000 7f79b6739640 1 -- 172.21.15.169:0/2406436536 shutdown_connections 2023-10-26T01:39:25.334 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:25.331+0000 7f79b6739640 1 --2- 172.21.15.169:0/2406436536 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f799006ae40 0x7f799006d300 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:25.334 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:25.331+0000 7f79b6739640 1 --2- 172.21.15.169:0/2406436536 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f79b01597c0 0x7f79b01f2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:25.334 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:25.331+0000 7f79b6739640 1 --2- 172.21.15.169:0/2406436536 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f79b01593e0 0x7f79b01f21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:25.335 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:25.331+0000 7f79b6739640 1 --2- 172.21.15.169:0/2406436536 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79b00b0c80 0x7f79b00aec50 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:25.335 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:25.331+0000 7f79b6739640 1 -- 172.21.15.169:0/2406436536 >> 172.21.15.169:0/2406436536 conn(0x7f79b0069930 msgr2=0x7f79b015a350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:25.335 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:25.331+0000 7f79b6739640 1 -- 172.21.15.169:0/2406436536 shutdown_connections 2023-10-26T01:39:25.335 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:25.331+0000 7f79b6739640 1 -- 172.21.15.169:0/2406436536 wait complete. 2023-10-26T01:39:25.360 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-26T01:39:25.558 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:25.556+0000 7f2030c97640 1 -- 172.21.15.169:0/2401189346 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f202c15bbb0 msgr2=0x7f202c162460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:25.558 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:25.556+0000 7f2030c97640 1 --2- 172.21.15.169:0/2401189346 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f202c15bbb0 0x7f202c162460 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f2024005fd0 tx=0x7f202402d150 comp rx=0 tx=0).stop 2023-10-26T01:39:25.558 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:25.556+0000 7f2030c97640 1 -- 172.21.15.169:0/2401189346 shutdown_connections 2023-10-26T01:39:25.559 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:25.556+0000 7f2030c97640 1 --2- 172.21.15.169:0/2401189346 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f202c15bbb0 0x7f202c162460 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:25.559 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:25.556+0000 7f2030c97640 1 -- 172.21.15.169:0/2401189346 >> 172.21.15.169:0/2401189346 conn(0x7f202c069930 msgr2=0x7f202c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:25.559 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:25.556+0000 7f2030c97640 1 -- 172.21.15.169:0/2401189346 shutdown_connections 2023-10-26T01:39:25.559 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:25.556+0000 7f2030c97640 1 -- 172.21.15.169:0/2401189346 wait complete. 2023-10-26T01:39:25.560 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:25.558+0000 7f2030c97640 1 Processor -- start 2023-10-26T01:39:25.561 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:25.558+0000 7f2030c97640 1 -- start start 2023-10-26T01:39:25.561 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:25.559+0000 7f2030c97640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f202c0cdef0 0x7f202c14d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:25.561 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:25.559+0000 7f2030c97640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f202c155360 0x7f202c1517e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:25.561 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:25.559+0000 7f2030c97640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f202c15bbb0 0x7f202c151d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:25.561 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:25.559+0000 7f2030c97640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f202c14dc60 con 0x7f202c15bbb0 2023-10-26T01:39:25.562 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:25.559+0000 7f2030c97640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f202c14e500 con 0x7f202c155360 2023-10-26T01:39:25.562 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:25.559+0000 7f2030c97640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f202c071e30 con 0x7f202c0cdef0 2023-10-26T01:39:25.562 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:25.559+0000 7f202ad9d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f202c15bbb0 0x7f202c151d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:25.562 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:25.559+0000 7f202ad9d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f202c15bbb0 0x7f202c151d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:37294/0 (socket says 172.21.15.169:37294) 2023-10-26T01:39:25.563 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:25.559+0000 7f202ad9d640 1 -- 172.21.15.169:0/4084469203 learned_addr learned my addr 172.21.15.169:0/4084469203 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:39:25.564 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:25.559+0000 7f2029d9b640 1 --2- 172.21.15.169:0/4084469203 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f202c155360 0x7f202c1517e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:25.564 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:25.559+0000 7f202a59c640 1 --2- 172.21.15.169:0/4084469203 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f202c0cdef0 0x7f202c14d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:25.565 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:25.560+0000 7f202ad9d640 1 -- 172.21.15.169:0/4084469203 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f202c0cdef0 msgr2=0x7f202c14d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:25.565 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:25.560+0000 7f202ad9d640 1 --2- 172.21.15.169:0/4084469203 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f202c0cdef0 0x7f202c14d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:25.565 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:25.560+0000 7f202ad9d640 1 -- 172.21.15.169:0/4084469203 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f202c155360 msgr2=0x7f202c1517e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:25.565 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:25.560+0000 7f202ad9d640 1 --2- 172.21.15.169:0/4084469203 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f202c155360 0x7f202c1517e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:25.565 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:25.560+0000 7f202ad9d640 1 -- 172.21.15.169:0/4084469203 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f202400a670 con 0x7f202c15bbb0 2023-10-26T01:39:25.566 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:25.560+0000 7f202ad9d640 1 --2- 172.21.15.169:0/4084469203 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f202c15bbb0 0x7f202c151d20 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f202400a340 tx=0x7f20240049d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:25.566 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:25.560+0000 7f2022ffd640 1 -- 172.21.15.169:0/4084469203 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2024004fd0 con 0x7f202c15bbb0 2023-10-26T01:39:25.566 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:25.560+0000 7f2022ffd640 1 -- 172.21.15.169:0/4084469203 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f20240036e0 con 0x7f202c15bbb0 2023-10-26T01:39:25.566 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:25.560+0000 7f2030c97640 1 -- 172.21.15.169:0/4084469203 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f202c14deb0 con 0x7f202c15bbb0 2023-10-26T01:39:25.566 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:25.560+0000 7f2022ffd640 1 -- 172.21.15.169:0/4084469203 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2024003af0 con 0x7f202c15bbb0 2023-10-26T01:39:25.567 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:25.560+0000 7f2030c97640 1 -- 172.21.15.169:0/4084469203 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f202c14d7a0 con 0x7f202c15bbb0 2023-10-26T01:39:25.567 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:25.561+0000 7f2030c97640 1 -- 172.21.15.169:0/4084469203 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f202c14e690 con 0x7f202c15bbb0 2023-10-26T01:39:25.567 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:25.563+0000 7f2022ffd640 1 -- 172.21.15.169:0/4084469203 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f2024003c50 con 0x7f202c15bbb0 2023-10-26T01:39:25.568 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:25.564+0000 7f2022ffd640 1 --2- 172.21.15.169:0/4084469203 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f200c06ae40 0x7f200c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:25.568 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:25.565+0000 7f2022ffd640 1 -- 172.21.15.169:0/4084469203 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f20240ad460 con 0x7f202c15bbb0 2023-10-26T01:39:25.568 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:25.565+0000 7f202a59c640 1 --2- 172.21.15.169:0/4084469203 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f200c06ae40 0x7f200c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:25.568 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:25.566+0000 7f202a59c640 1 --2- 172.21.15.169:0/4084469203 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f200c06ae40 0x7f200c06d300 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f201c007fd0 tx=0x7f201c009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:25.570 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:25.569+0000 7f2022ffd640 1 -- 172.21.15.169:0/4084469203 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2024078800 con 0x7f202c15bbb0 2023-10-26T01:39:25.789 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:25.787+0000 7f2030c97640 1 -- 172.21.15.169:0/4084469203 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f202c1586f0 con 0x7f202c15bbb0 2023-10-26T01:39:26.345 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:26.342+0000 7f2022ffd640 1 -- 172.21.15.169:0/4084469203 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 18 and data pool 19 v73) v1 ==== 174+0+0 (secure 0 0 0) 0x7f202407be20 con 0x7f202c15bbb0 2023-10-26T01:39:26.348 INFO:teuthology.orchestra.run.smithi169.stderr:new fs with metadata pool 18 and data pool 19 2023-10-26T01:39:26.352 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:26.351+0000 7f2030c97640 1 -- 172.21.15.169:0/4084469203 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f200c06ae40 msgr2=0x7f200c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:26.352 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:26.351+0000 7f2030c97640 1 --2- 172.21.15.169:0/4084469203 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f200c06ae40 0x7f200c06d300 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f201c007fd0 tx=0x7f201c009040 comp rx=0 tx=0).stop 2023-10-26T01:39:26.353 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:26.351+0000 7f2030c97640 1 -- 172.21.15.169:0/4084469203 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f202c15bbb0 msgr2=0x7f202c151d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:26.353 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:26.351+0000 7f2030c97640 1 --2- 172.21.15.169:0/4084469203 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f202c15bbb0 0x7f202c151d20 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f202400a340 tx=0x7f20240049d0 comp rx=0 tx=0).stop 2023-10-26T01:39:26.353 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:26.351+0000 7f2030c97640 1 -- 172.21.15.169:0/4084469203 shutdown_connections 2023-10-26T01:39:26.353 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:26.351+0000 7f2030c97640 1 --2- 172.21.15.169:0/4084469203 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f200c06ae40 0x7f200c06d300 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:26.354 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:26.351+0000 7f2030c97640 1 --2- 172.21.15.169:0/4084469203 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f202c15bbb0 0x7f202c151d20 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:26.354 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:26.351+0000 7f2030c97640 1 --2- 172.21.15.169:0/4084469203 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f202c155360 0x7f202c1517e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:26.354 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:26.351+0000 7f2030c97640 1 --2- 172.21.15.169:0/4084469203 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f202c0cdef0 0x7f202c14d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:26.354 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:26.351+0000 7f2030c97640 1 -- 172.21.15.169:0/4084469203 >> 172.21.15.169:0/4084469203 conn(0x7f202c069930 msgr2=0x7f202c155910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:26.356 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:26.353+0000 7f2030c97640 1 -- 172.21.15.169:0/4084469203 shutdown_connections 2023-10-26T01:39:26.356 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:26.355+0000 7f2030c97640 1 -- 172.21.15.169:0/4084469203 wait complete. 2023-10-26T01:39:26.380 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:39:26.572 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:26.570+0000 7fd0ff7f5640 1 -- 172.21.15.169:0/2901082566 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd0f815b610 msgr2=0x7fd0f806ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:26.573 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:26.570+0000 7fd0ff7f5640 1 --2- 172.21.15.169:0/2901082566 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd0f815b610 0x7fd0f806ed30 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7fd0f806c960 tx=0x7fd0f402bda0 comp rx=0 tx=0).stop 2023-10-26T01:39:26.573 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:26.570+0000 7fd0ff7f5640 1 -- 172.21.15.169:0/2901082566 shutdown_connections 2023-10-26T01:39:26.573 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:26.570+0000 7fd0ff7f5640 1 --2- 172.21.15.169:0/2901082566 >> v2:172.21.15.170:3300/0 conn(0x7fd0f80ac4e0 0x7fd0f80cd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:26.573 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:26.570+0000 7fd0ff7f5640 1 --2- 172.21.15.169:0/2901082566 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd0f815b610 0x7fd0f806ed30 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:26.573 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:26.570+0000 7fd0ff7f5640 1 -- 172.21.15.169:0/2901082566 >> 172.21.15.169:0/2901082566 conn(0x7fd0f8069930 msgr2=0x7fd0f8069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:26.574 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:26.570+0000 7fd0ff7f5640 1 -- 172.21.15.169:0/2901082566 shutdown_connections 2023-10-26T01:39:26.574 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:26.570+0000 7fd0ff7f5640 1 -- 172.21.15.169:0/2901082566 wait complete. 2023-10-26T01:39:26.574 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:26.572+0000 7fd0ff7f5640 1 Processor -- start 2023-10-26T01:39:26.575 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:26.573+0000 7fd0ff7f5640 1 -- start start 2023-10-26T01:39:26.577 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:26.574+0000 7fd0ff7f5640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd0f80ac4e0 0x7fd0f8157570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:26.577 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:26.574+0000 7fd0ff7f5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd0f80ac940 0x7fd0f8153f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:26.577 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:26.574+0000 7fd0ff7f5640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd0f815b610 0x7fd0f81525b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:26.578 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:26.574+0000 7fd0ff7f5640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f8153ae0 con 0x7fd0f815b610 2023-10-26T01:39:26.578 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:26.574+0000 7fd0ff7f5640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f8152ed0 con 0x7fd0f80ac940 2023-10-26T01:39:26.578 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:26.574+0000 7fd0ff7f5640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f81531b0 con 0x7fd0f80ac4e0 2023-10-26T01:39:26.578 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:26.574+0000 7fd0fdd92640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd0f815b610 0x7fd0f81525b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:26.579 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:26.574+0000 7fd0fdd92640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd0f815b610 0x7fd0f81525b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:37320/0 (socket says 172.21.15.169:37320) 2023-10-26T01:39:26.579 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:26.574+0000 7fd0fdd92640 1 -- 172.21.15.169:0/3715163425 learned_addr learned my addr 172.21.15.169:0/3715163425 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:39:26.579 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:26.574+0000 7fd0fd591640 1 --2- 172.21.15.169:0/3715163425 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd0f80ac4e0 0x7fd0f8157570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:26.579 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:26.574+0000 7fd0fdd92640 1 -- 172.21.15.169:0/3715163425 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd0f80ac4e0 msgr2=0x7fd0f8157570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:26.579 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:26.574+0000 7fd0fcd90640 1 --2- 172.21.15.169:0/3715163425 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd0f80ac940 0x7fd0f8153f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:26.580 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:26.574+0000 7fd0fdd92640 1 --2- 172.21.15.169:0/3715163425 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd0f80ac4e0 0x7fd0f8157570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:26.580 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:26.574+0000 7fd0fdd92640 1 -- 172.21.15.169:0/3715163425 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd0f80ac940 msgr2=0x7fd0f8153f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:26.580 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:26.574+0000 7fd0fdd92640 1 --2- 172.21.15.169:0/3715163425 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd0f80ac940 0x7fd0f8153f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:26.580 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:26.575+0000 7fd0fdd92640 1 -- 172.21.15.169:0/3715163425 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0f4009560 con 0x7fd0f815b610 2023-10-26T01:39:26.580 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:26.575+0000 7fd0fd591640 1 --2- 172.21.15.169:0/3715163425 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd0f80ac4e0 0x7fd0f8157570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:39:26.581 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:26.575+0000 7fd0fcd90640 1 --2- 172.21.15.169:0/3715163425 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd0f80ac940 0x7fd0f8153f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:39:26.581 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:26.575+0000 7fd0fdd92640 1 --2- 172.21.15.169:0/3715163425 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd0f815b610 0x7fd0f81525b0 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7fd0f000c780 tx=0x7fd0f000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:26.581 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:26.575+0000 7fd0edffb640 1 -- 172.21.15.169:0/3715163425 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f0003d80 con 0x7fd0f815b610 2023-10-26T01:39:26.581 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:26.575+0000 7fd0edffb640 1 -- 172.21.15.169:0/3715163425 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd0f000ce90 con 0x7fd0f815b610 2023-10-26T01:39:26.581 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:26.575+0000 7fd0edffb640 1 -- 172.21.15.169:0/3715163425 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f00052d0 con 0x7fd0f815b610 2023-10-26T01:39:26.582 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:26.575+0000 7fd0ff7f5640 1 -- 172.21.15.169:0/3715163425 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0f8153490 con 0x7fd0f815b610 2023-10-26T01:39:26.582 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:26.575+0000 7fd0ff7f5640 1 -- 172.21.15.169:0/3715163425 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0f8157ab0 con 0x7fd0f815b610 2023-10-26T01:39:26.583 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:26.576+0000 7fd0ff7f5640 1 -- 172.21.15.169:0/3715163425 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0f8153c50 con 0x7fd0f815b610 2023-10-26T01:39:26.583 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:26.577+0000 7fd0edffb640 1 -- 172.21.15.169:0/3715163425 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fd0f0005430 con 0x7fd0f815b610 2023-10-26T01:39:26.583 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:26.578+0000 7fd0edffb640 1 --2- 172.21.15.169:0/3715163425 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd0dc06ae70 0x7fd0dc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:26.583 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:26.578+0000 7fd0edffb640 1 -- 172.21.15.169:0/3715163425 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd0f0088d50 con 0x7fd0f815b610 2023-10-26T01:39:26.584 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:26.579+0000 7fd0fd591640 1 --2- 172.21.15.169:0/3715163425 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd0dc06ae70 0x7fd0dc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:26.584 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:26.580+0000 7fd0fd591640 1 --2- 172.21.15.169:0/3715163425 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd0dc06ae70 0x7fd0dc06d330 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fd0f4035000 tx=0x7fd0f4004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:26.585 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:26.583+0000 7fd0edffb640 1 -- 172.21.15.169:0/3715163425 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd0f008c030 con 0x7fd0f815b610 2023-10-26T01:39:26.774 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:26.772+0000 7fd0ff7f5640 1 -- 172.21.15.169:0/3715163425 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd0f804db10 con 0x7fd0f815b610 2023-10-26T01:39:26.776 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:26.775+0000 7fd0edffb640 1 -- 172.21.15.169:0/3715163425 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v150) v1 ==== 74+0+16641 (secure 0 0 0) 0x7fd0f0017090 con 0x7fd0f815b610 2023-10-26T01:39:26.777 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:39:26.777 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":150,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:39:25.789193+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-10-26T01:39:23.646624+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"150","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-10-26T01:39:24.813678+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"150","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:39:26.779 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:26.777+0000 7fd0ff7f5640 1 -- 172.21.15.169:0/3715163425 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd0dc06ae70 msgr2=0x7fd0dc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:26.780 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:26.777+0000 7fd0ff7f5640 1 --2- 172.21.15.169:0/3715163425 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd0dc06ae70 0x7fd0dc06d330 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fd0f4035000 tx=0x7fd0f4004730 comp rx=0 tx=0).stop 2023-10-26T01:39:26.780 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:26.777+0000 7fd0ff7f5640 1 -- 172.21.15.169:0/3715163425 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd0f815b610 msgr2=0x7fd0f81525b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:26.780 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:26.777+0000 7fd0ff7f5640 1 --2- 172.21.15.169:0/3715163425 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd0f815b610 0x7fd0f81525b0 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7fd0f000c780 tx=0x7fd0f000cc40 comp rx=0 tx=0).stop 2023-10-26T01:39:26.780 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:26.778+0000 7fd0ff7f5640 1 -- 172.21.15.169:0/3715163425 shutdown_connections 2023-10-26T01:39:26.781 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:26.778+0000 7fd0ff7f5640 1 --2- 172.21.15.169:0/3715163425 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd0dc06ae70 0x7fd0dc06d330 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:26.781 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:26.778+0000 7fd0ff7f5640 1 --2- 172.21.15.169:0/3715163425 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd0f815b610 0x7fd0f81525b0 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:26.781 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:26.778+0000 7fd0ff7f5640 1 --2- 172.21.15.169:0/3715163425 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd0f80ac940 0x7fd0f8153f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:26.781 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:26.778+0000 7fd0ff7f5640 1 --2- 172.21.15.169:0/3715163425 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd0f80ac4e0 0x7fd0f8157570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:26.781 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:26.778+0000 7fd0ff7f5640 1 -- 172.21.15.169:0/3715163425 >> 172.21.15.169:0/3715163425 conn(0x7fd0f8069930 msgr2=0x7fd0f815a640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:26.782 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:26.778+0000 7fd0ff7f5640 1 -- 172.21.15.169:0/3715163425 shutdown_connections 2023-10-26T01:39:26.782 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:26.778+0000 7fd0ff7f5640 1 -- 172.21.15.169:0/3715163425 wait complete. 2023-10-26T01:39:26.798 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:39:27.014 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:27.011+0000 7fe97116c640 1 -- 172.21.15.169:0/3476000398 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe954003970 msgr2=0x7fe954003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:27.014 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:27.011+0000 7fe97116c640 1 --2- 172.21.15.169:0/3476000398 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe954003970 0x7fe954003d90 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7fe95c006fd0 tx=0x7fe9540057f0 comp rx=0 tx=0).stop 2023-10-26T01:39:27.014 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:27.011+0000 7fe96b59e640 1 -- 172.21.15.169:0/3476000398 reap_dead start 2023-10-26T01:39:27.014 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:27.012+0000 7fe97116c640 1 -- 172.21.15.169:0/3476000398 shutdown_connections 2023-10-26T01:39:27.015 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:27.012+0000 7fe97116c640 1 -- 172.21.15.169:0/3476000398 >> 172.21.15.169:0/3476000398 conn(0x7fe96c069930 msgr2=0x7fe96c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:27.015 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:27.012+0000 7fe97116c640 1 -- 172.21.15.169:0/3476000398 shutdown_connections 2023-10-26T01:39:27.015 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:27.012+0000 7fe97116c640 1 -- 172.21.15.169:0/3476000398 wait complete. 2023-10-26T01:39:27.016 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:27.014+0000 7fe97116c640 1 Processor -- start 2023-10-26T01:39:27.016 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:27.014+0000 7fe97116c640 1 -- start start 2023-10-26T01:39:27.017 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:27.015+0000 7fe97116c640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe954003970 0x7fe96c14d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:27.017 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:27.015+0000 7fe97116c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe96c15b6a0 0x7fe96c06afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:27.017 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:27.015+0000 7fe97116c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe96c14cb50 0x7fe96c14cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:27.018 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:27.015+0000 7fe97116c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe96c14e9f0 con 0x7fe96c14cb50 2023-10-26T01:39:27.018 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:27.015+0000 7fe97116c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe96c15bb20 con 0x7fe96c15b6a0 2023-10-26T01:39:27.018 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:27.015+0000 7fe97116c640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe96c14d4f0 con 0x7fe954003970 2023-10-26T01:39:27.018 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:27.015+0000 7fe96b59e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe96c14cb50 0x7fe96c14cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:27.018 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:27.015+0000 7fe96b59e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe96c14cb50 0x7fe96c14cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:37334/0 (socket says 172.21.15.169:37334) 2023-10-26T01:39:27.019 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:27.015+0000 7fe96b59e640 1 -- 172.21.15.169:0/840989987 learned_addr learned my addr 172.21.15.169:0/840989987 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:39:27.020 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:27.016+0000 7fe96a59c640 1 --2- 172.21.15.169:0/840989987 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe96c15b6a0 0x7fe96c06afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:27.020 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:27.016+0000 7fe96ad9d640 1 --2- 172.21.15.169:0/840989987 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe954003970 0x7fe96c14d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:27.020 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:27.016+0000 7fe96b59e640 1 -- 172.21.15.169:0/840989987 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe954003970 msgr2=0x7fe96c14d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:27.020 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:27.016+0000 7fe96b59e640 1 --2- 172.21.15.169:0/840989987 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe954003970 0x7fe96c14d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:27.021 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:27.016+0000 7fe96b59e640 1 -- 172.21.15.169:0/840989987 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe96c15b6a0 msgr2=0x7fe96c06afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:27.021 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:27.016+0000 7fe96b59e640 1 --2- 172.21.15.169:0/840989987 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe96c15b6a0 0x7fe96c06afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:27.021 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:27.016+0000 7fe96b59e640 1 -- 172.21.15.169:0/840989987 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe96c14dde0 con 0x7fe96c14cb50 2023-10-26T01:39:27.021 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:27.016+0000 7fe96b59e640 1 --2- 172.21.15.169:0/840989987 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe96c14cb50 0x7fe96c14cfb0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7fe96c0cafb0 tx=0x7fe964009a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:27.021 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:27.017+0000 7fe9537fe640 1 -- 172.21.15.169:0/840989987 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe964017070 con 0x7fe96c14cb50 2023-10-26T01:39:27.022 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:27.017+0000 7fe9537fe640 1 -- 172.21.15.169:0/840989987 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe964004c10 con 0x7fe96c14cb50 2023-10-26T01:39:27.022 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:27.017+0000 7fe97116c640 1 -- 172.21.15.169:0/840989987 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe96c14e0c0 con 0x7fe96c14cb50 2023-10-26T01:39:27.022 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:27.017+0000 7fe9537fe640 1 -- 172.21.15.169:0/840989987 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9640106a0 con 0x7fe96c14cb50 2023-10-26T01:39:27.022 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:27.017+0000 7fe97116c640 1 -- 172.21.15.169:0/840989987 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe96c14e680 con 0x7fe96c14cb50 2023-10-26T01:39:27.022 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:27.018+0000 7fe97116c640 1 -- 172.21.15.169:0/840989987 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe96c04db10 con 0x7fe96c14cb50 2023-10-26T01:39:27.023 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:27.019+0000 7fe9537fe640 1 -- 172.21.15.169:0/840989987 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fe964010800 con 0x7fe96c14cb50 2023-10-26T01:39:27.023 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:27.020+0000 7fe9537fe640 1 --2- 172.21.15.169:0/840989987 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe95806ae70 0x7fe95806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:27.024 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:27.021+0000 7fe9537fe640 1 -- 172.21.15.169:0/840989987 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe9640899e0 con 0x7fe96c14cb50 2023-10-26T01:39:27.024 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:27.021+0000 7fe96ad9d640 1 --2- 172.21.15.169:0/840989987 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe95806ae70 0x7fe95806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:27.024 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:27.022+0000 7fe96ad9d640 1 --2- 172.21.15.169:0/840989987 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe95806ae70 0x7fe95806d330 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fe9540057a0 tx=0x7fe95c001c00 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:27.027 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:27.025+0000 7fe9537fe640 1 -- 172.21.15.169:0/840989987 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe9640549f0 con 0x7fe96c14cb50 2023-10-26T01:39:27.216 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:27.214+0000 7fe97116c640 1 -- 172.21.15.169:0/840989987 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe96c159880 con 0x7fe96c14cb50 2023-10-26T01:39:27.217 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:27.215+0000 7fe9537fe640 1 -- 172.21.15.169:0/840989987 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v150) v1 ==== 74+0+16641 (secure 0 0 0) 0x7fe964058010 con 0x7fe96c14cb50 2023-10-26T01:39:27.218 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:39:27.218 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":150,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:39:25.789193+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-10-26T01:39:23.646624+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"150","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-10-26T01:39:24.813678+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"150","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:39:27.220 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:27.218+0000 7fe97116c640 1 -- 172.21.15.169:0/840989987 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe95806ae70 msgr2=0x7fe95806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:27.221 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:27.218+0000 7fe97116c640 1 --2- 172.21.15.169:0/840989987 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe95806ae70 0x7fe95806d330 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fe9540057a0 tx=0x7fe95c001c00 comp rx=0 tx=0).stop 2023-10-26T01:39:27.221 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:27.218+0000 7fe97116c640 1 -- 172.21.15.169:0/840989987 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe96c14cb50 msgr2=0x7fe96c14cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:27.221 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:27.218+0000 7fe97116c640 1 --2- 172.21.15.169:0/840989987 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe96c14cb50 0x7fe96c14cfb0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7fe96c0cafb0 tx=0x7fe964009a80 comp rx=0 tx=0).stop 2023-10-26T01:39:27.221 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:27.219+0000 7fe97116c640 1 -- 172.21.15.169:0/840989987 shutdown_connections 2023-10-26T01:39:27.222 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:27.219+0000 7fe97116c640 1 --2- 172.21.15.169:0/840989987 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe95806ae70 0x7fe95806d330 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:27.222 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:27.219+0000 7fe97116c640 1 --2- 172.21.15.169:0/840989987 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe96c14cb50 0x7fe96c14cfb0 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:27.222 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:27.219+0000 7fe97116c640 1 --2- 172.21.15.169:0/840989987 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe96c15b6a0 0x7fe96c06afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:27.222 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:27.219+0000 7fe97116c640 1 --2- 172.21.15.169:0/840989987 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe954003970 0x7fe96c14d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:27.222 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:27.219+0000 7fe97116c640 1 -- 172.21.15.169:0/840989987 >> 172.21.15.169:0/840989987 conn(0x7fe96c069930 msgr2=0x7fe96c0afbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:27.223 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:27.219+0000 7fe97116c640 1 -- 172.21.15.169:0/840989987 shutdown_connections 2023-10-26T01:39:27.223 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:27.219+0000 7fe97116c640 1 -- 172.21.15.169:0/840989987 wait complete. 2023-10-26T01:39:27.239 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-26T01:39:27.452 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:27.449+0000 7f315633c640 1 -- 172.21.15.169:0/2237471764 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f313c003f90 msgr2=0x7f313c0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:27.452 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:27.449+0000 7f315633c640 1 --2- 172.21.15.169:0/2237471764 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f313c003f90 0x7f313c0047c0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f3138006fd0 tx=0x7f313c0057f0 comp rx=0 tx=0).stop 2023-10-26T01:39:27.453 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:27.449+0000 7f31548d9640 1 -- 172.21.15.169:0/2237471764 reap_dead start 2023-10-26T01:39:27.453 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:27.449+0000 7f315633c640 1 -- 172.21.15.169:0/2237471764 shutdown_connections 2023-10-26T01:39:27.453 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:27.449+0000 7f315633c640 1 -- 172.21.15.169:0/2237471764 >> 172.21.15.169:0/2237471764 conn(0x7f3150069930 msgr2=0x7f3150069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:27.454 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:27.449+0000 7f315633c640 1 -- 172.21.15.169:0/2237471764 shutdown_connections 2023-10-26T01:39:27.454 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:27.449+0000 7f315633c640 1 -- 172.21.15.169:0/2237471764 wait complete. 2023-10-26T01:39:27.457 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:27.450+0000 7f315633c640 1 Processor -- start 2023-10-26T01:39:27.457 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:27.451+0000 7f315633c640 1 -- start start 2023-10-26T01:39:27.458 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:27.451+0000 7f315633c640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f313c003f90 0x7f31500ca0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:27.458 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:27.452+0000 7f315633c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f31500cce10 0x7f315014d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:27.458 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:27.452+0000 7f315633c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f315015db10 0x7f315015dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:27.459 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:27.452+0000 7f315633c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f315015e4e0 con 0x7f31500cce10 2023-10-26T01:39:27.459 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:27.452+0000 7f315633c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31500cb500 con 0x7f315015db10 2023-10-26T01:39:27.459 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:27.452+0000 7f315633c640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31501f23d0 con 0x7f313c003f90 2023-10-26T01:39:27.459 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:27.452+0000 7f3147fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f31500cce10 0x7f315014d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:27.460 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:27.452+0000 7f3147fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f31500cce10 0x7f315014d110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:37352/0 (socket says 172.21.15.169:37352) 2023-10-26T01:39:27.460 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:27.452+0000 7f3147fff640 1 -- 172.21.15.169:0/106996642 learned_addr learned my addr 172.21.15.169:0/106996642 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:39:27.460 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:27.452+0000 7f3147fff640 1 -- 172.21.15.169:0/106996642 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f313c003f90 msgr2=0x7f31500ca0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:27.460 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:27.452+0000 7f3147fff640 1 --2- 172.21.15.169:0/106996642 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f313c003f90 0x7f31500ca0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:27.461 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:27.452+0000 7f3147fff640 1 -- 172.21.15.169:0/106996642 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f315015db10 msgr2=0x7f315015dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:27.461 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:27.452+0000 7f3147fff640 1 --2- 172.21.15.169:0/106996642 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f315015db10 0x7f315015dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:27.462 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:27.452+0000 7f3147fff640 1 -- 172.21.15.169:0/106996642 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31501f26b0 con 0x7f31500cce10 2023-10-26T01:39:27.462 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:27.453+0000 7f3147fff640 1 --2- 172.21.15.169:0/106996642 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f31500cce10 0x7f315014d110 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f313802c850 tx=0x7f31380016e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:27.462 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:27.453+0000 7f314d7fa640 1 -- 172.21.15.169:0/106996642 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f313803c040 con 0x7f31500cce10 2023-10-26T01:39:27.462 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:27.453+0000 7f314d7fa640 1 -- 172.21.15.169:0/106996642 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f31380037f0 con 0x7f31500cce10 2023-10-26T01:39:27.463 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:27.453+0000 7f315633c640 1 -- 172.21.15.169:0/106996642 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31501f2990 con 0x7f31500cce10 2023-10-26T01:39:27.463 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:27.453+0000 7f314d7fa640 1 -- 172.21.15.169:0/106996642 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3138003bc0 con 0x7f31500cce10 2023-10-26T01:39:27.463 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:27.453+0000 7f315633c640 1 -- 172.21.15.169:0/106996642 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31501f20c0 con 0x7f31500cce10 2023-10-26T01:39:27.464 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:27.454+0000 7f315633c640 1 -- 172.21.15.169:0/106996642 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f315004db10 con 0x7f31500cce10 2023-10-26T01:39:27.464 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:27.455+0000 7f314d7fa640 1 -- 172.21.15.169:0/106996642 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f3138032070 con 0x7f31500cce10 2023-10-26T01:39:27.466 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:27.456+0000 7f314d7fa640 1 --2- 172.21.15.169:0/106996642 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f313006ae40 0x7f313006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:27.467 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:27.456+0000 7f314d7fa640 1 -- 172.21.15.169:0/106996642 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f31380aabd0 con 0x7f31500cce10 2023-10-26T01:39:27.467 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:27.457+0000 7f314ffff640 1 --2- 172.21.15.169:0/106996642 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f313006ae40 0x7f313006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:27.467 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:27.458+0000 7f314ffff640 1 --2- 172.21.15.169:0/106996642 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f313006ae40 0x7f313006d300 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f3140005c40 tx=0x7f3140009260 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:27.467 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:27.460+0000 7f314d7fa640 1 -- 172.21.15.169:0/106996642 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3138075c80 con 0x7f31500cce10 2023-10-26T01:39:27.690 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:27.688+0000 7f315633c640 1 -- 172.21.15.169:0/106996642 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f31500c96a0 con 0x7f31500cce10 2023-10-26T01:39:28.357 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:28.355+0000 7f314d7fa640 1 -- 172.21.15.169:0/106996642 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v76) v1 ==== 118+0+0 (secure 0 0 0) 0x7f31380792a0 con 0x7f31500cce10 2023-10-26T01:39:28.360 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:28.359+0000 7f315633c640 1 -- 172.21.15.169:0/106996642 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f313006ae40 msgr2=0x7f313006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:28.361 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:28.359+0000 7f315633c640 1 --2- 172.21.15.169:0/106996642 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f313006ae40 0x7f313006d300 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f3140005c40 tx=0x7f3140009260 comp rx=0 tx=0).stop 2023-10-26T01:39:28.361 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:28.359+0000 7f315633c640 1 -- 172.21.15.169:0/106996642 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f31500cce10 msgr2=0x7f315014d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:28.361 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:28.359+0000 7f315633c640 1 --2- 172.21.15.169:0/106996642 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f31500cce10 0x7f315014d110 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f313802c850 tx=0x7f31380016e0 comp rx=0 tx=0).stop 2023-10-26T01:39:28.362 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:28.359+0000 7f315633c640 1 -- 172.21.15.169:0/106996642 shutdown_connections 2023-10-26T01:39:28.362 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:28.359+0000 7f315633c640 1 --2- 172.21.15.169:0/106996642 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f313006ae40 0x7f313006d300 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:28.362 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:28.359+0000 7f315633c640 1 --2- 172.21.15.169:0/106996642 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f31500cce10 0x7f315014d110 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:28.362 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:28.359+0000 7f315633c640 1 --2- 172.21.15.169:0/106996642 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f315015db10 0x7f315015dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:28.363 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:28.359+0000 7f315633c640 1 --2- 172.21.15.169:0/106996642 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f313c003f90 0x7f31500ca0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:28.363 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:28.359+0000 7f315633c640 1 -- 172.21.15.169:0/106996642 >> 172.21.15.169:0/106996642 conn(0x7f3150069930 msgr2=0x7f3150157330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:28.363 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:28.359+0000 7f315633c640 1 -- 172.21.15.169:0/106996642 shutdown_connections 2023-10-26T01:39:28.363 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:28.359+0000 7f315633c640 1 -- 172.21.15.169:0/106996642 wait complete. 2023-10-26T01:39:28.381 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-26T01:39:28.381 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-26T01:39:28.599 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:28.596+0000 7f6d1fec9640 1 -- 172.21.15.169:0/2151067695 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d04003ef0 msgr2=0x7f6d040047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:28.599 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:28.596+0000 7f6d1fec9640 1 --2- 172.21.15.169:0/2151067695 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d04003ef0 0x7f6d040047c0 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c009b80 tx=0x7f6d040057f0 comp rx=0 tx=0).stop 2023-10-26T01:39:28.599 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:28.596+0000 7f6d1e466640 1 -- 172.21.15.169:0/2151067695 reap_dead start 2023-10-26T01:39:28.600 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:28.596+0000 7f6d1fec9640 1 -- 172.21.15.169:0/2151067695 shutdown_connections 2023-10-26T01:39:28.600 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:28.596+0000 7f6d1fec9640 1 -- 172.21.15.169:0/2151067695 >> 172.21.15.169:0/2151067695 conn(0x7f6d18069930 msgr2=0x7f6d18069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:28.600 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:28.596+0000 7f6d1fec9640 1 -- 172.21.15.169:0/2151067695 shutdown_connections 2023-10-26T01:39:28.600 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:28.596+0000 7f6d1fec9640 1 -- 172.21.15.169:0/2151067695 wait complete. 2023-10-26T01:39:28.601 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:28.598+0000 7f6d1fec9640 1 Processor -- start 2023-10-26T01:39:28.601 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:28.598+0000 7f6d1fec9640 1 -- start start 2023-10-26T01:39:28.603 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:28.599+0000 7f6d1fec9640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d04003ef0 0x7f6d180c9050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:28.603 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:28.599+0000 7f6d1fec9640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6d180c7070 0x7f6d1806b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:28.603 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:28.599+0000 7f6d1fec9640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6d180ca6d0 0x7f6d180cab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:28.604 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:28.599+0000 7f6d1fec9640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d180c9590 con 0x7f6d04003ef0 2023-10-26T01:39:28.604 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:28.599+0000 7f6d1fec9640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d180c8bc0 con 0x7f6d180c7070 2023-10-26T01:39:28.604 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:28.599+0000 7f6d1fec9640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d180c7cd0 con 0x7f6d180ca6d0 2023-10-26T01:39:28.604 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:28.599+0000 7f6d1dc65640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d04003ef0 0x7f6d180c9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:28.605 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:28.599+0000 7f6d1dc65640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d04003ef0 0x7f6d180c9050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:37374/0 (socket says 172.21.15.169:37374) 2023-10-26T01:39:28.605 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:28.599+0000 7f6d1dc65640 1 -- 172.21.15.169:0/364365832 learned_addr learned my addr 172.21.15.169:0/364365832 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:39:28.605 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:28.599+0000 7f6d1dc65640 1 -- 172.21.15.169:0/364365832 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6d180ca6d0 msgr2=0x7f6d180cab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:28.605 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:28.599+0000 7f6d1dc65640 1 --2- 172.21.15.169:0/364365832 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6d180ca6d0 0x7f6d180cab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:28.606 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:28.599+0000 7f6d1dc65640 1 -- 172.21.15.169:0/364365832 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6d180c7070 msgr2=0x7f6d1806b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:28.606 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:28.599+0000 7f6d1dc65640 1 --2- 172.21.15.169:0/364365832 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6d180c7070 0x7f6d1806b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:28.606 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:28.599+0000 7f6d1dc65640 1 -- 172.21.15.169:0/364365832 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d180c7fb0 con 0x7f6d04003ef0 2023-10-26T01:39:28.606 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:28.600+0000 7f6d1dc65640 1 --2- 172.21.15.169:0/364365832 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d04003ef0 0x7f6d180c9050 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f6d1400e970 tx=0x7f6d1400ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:28.607 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:28.600+0000 7f6d0a7fc640 1 -- 172.21.15.169:0/364365832 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d1400ccc0 con 0x7f6d04003ef0 2023-10-26T01:39:28.607 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:28.600+0000 7f6d0a7fc640 1 -- 172.21.15.169:0/364365832 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6d1400ce20 con 0x7f6d04003ef0 2023-10-26T01:39:28.607 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:28.600+0000 7f6d1fec9640 1 -- 172.21.15.169:0/364365832 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d180c8290 con 0x7f6d04003ef0 2023-10-26T01:39:28.607 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:28.600+0000 7f6d0a7fc640 1 -- 172.21.15.169:0/364365832 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d14005280 con 0x7f6d04003ef0 2023-10-26T01:39:28.607 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:28.600+0000 7f6d1fec9640 1 -- 172.21.15.169:0/364365832 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d180c8850 con 0x7f6d04003ef0 2023-10-26T01:39:28.608 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:28.601+0000 7f6d1fec9640 1 -- 172.21.15.169:0/364365832 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d1804db10 con 0x7f6d04003ef0 2023-10-26T01:39:28.609 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:28.602+0000 7f6d0a7fc640 1 -- 172.21.15.169:0/364365832 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f6d140053e0 con 0x7f6d04003ef0 2023-10-26T01:39:28.609 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:28.603+0000 7f6d0a7fc640 1 --2- 172.21.15.169:0/364365832 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f6cfc06ae40 0x7f6cfc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:28.609 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:28.603+0000 7f6d0a7fc640 1 -- 172.21.15.169:0/364365832 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6d14014070 con 0x7f6d04003ef0 2023-10-26T01:39:28.609 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:28.604+0000 7f6d1d464640 1 --2- 172.21.15.169:0/364365832 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f6cfc06ae40 0x7f6cfc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:28.610 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:28.605+0000 7f6d1d464640 1 --2- 172.21.15.169:0/364365832 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f6cfc06ae40 0x7f6cfc06d300 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c036000 tx=0x7f6d0c035040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:28.610 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:28.608+0000 7f6d0a7fc640 1 -- 172.21.15.169:0/364365832 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6d14055140 con 0x7f6d04003ef0 2023-10-26T01:39:28.842 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:28.839+0000 7f6d1fec9640 1 -- 172.21.15.169:0/364365832 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f6d1815c950 con 0x7f6d04003ef0 2023-10-26T01:39:29.360 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:29.357+0000 7f6d0a7fc640 1 -- 172.21.15.169:0/364365832 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v77) v1 ==== 122+0+0 (secure 0 0 0) 0x7f6d14058760 con 0x7f6d04003ef0 2023-10-26T01:39:29.365 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:29.362+0000 7f6d1fec9640 1 -- 172.21.15.169:0/364365832 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f6cfc06ae40 msgr2=0x7f6cfc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:29.365 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:29.362+0000 7f6d1fec9640 1 --2- 172.21.15.169:0/364365832 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f6cfc06ae40 0x7f6cfc06d300 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c036000 tx=0x7f6d0c035040 comp rx=0 tx=0).stop 2023-10-26T01:39:29.365 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:29.362+0000 7f6d1fec9640 1 -- 172.21.15.169:0/364365832 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d04003ef0 msgr2=0x7f6d180c9050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:29.365 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:29.362+0000 7f6d1fec9640 1 --2- 172.21.15.169:0/364365832 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d04003ef0 0x7f6d180c9050 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f6d1400e970 tx=0x7f6d1400ee30 comp rx=0 tx=0).stop 2023-10-26T01:39:29.366 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:29.362+0000 7f6d1fec9640 1 -- 172.21.15.169:0/364365832 shutdown_connections 2023-10-26T01:39:29.366 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:29.362+0000 7f6d1fec9640 1 --2- 172.21.15.169:0/364365832 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f6cfc06ae40 0x7f6cfc06d300 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:29.366 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:29.362+0000 7f6d1fec9640 1 --2- 172.21.15.169:0/364365832 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6d180ca6d0 0x7f6d180cab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:29.366 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:29.362+0000 7f6d1fec9640 1 --2- 172.21.15.169:0/364365832 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6d180c7070 0x7f6d1806b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:29.366 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:29.362+0000 7f6d1fec9640 1 --2- 172.21.15.169:0/364365832 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d04003ef0 0x7f6d180c9050 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:29.367 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:29.362+0000 7f6d1fec9640 1 -- 172.21.15.169:0/364365832 >> 172.21.15.169:0/364365832 conn(0x7f6d18069930 msgr2=0x7f6d1815dba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:29.367 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:29.362+0000 7f6d1fec9640 1 -- 172.21.15.169:0/364365832 shutdown_connections 2023-10-26T01:39:29.367 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:29.363+0000 7f6d1fec9640 1 -- 172.21.15.169:0/364365832 wait complete. 2023-10-26T01:39:29.388 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-26T01:39:29.389 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T01:39:29.605 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:29.602+0000 7f22197c0640 1 -- 172.21.15.169:0/2840494932 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22000039a0 msgr2=0x7f2200003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:29.606 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:29.602+0000 7f22197c0640 1 --2- 172.21.15.169:0/2840494932 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22000039a0 0x7f2200003dc0 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f220400ac60 tx=0x7f22000057f0 comp rx=0 tx=0).stop 2023-10-26T01:39:29.606 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:29.602+0000 7f22137fe640 1 -- 172.21.15.169:0/2840494932 reap_dead start 2023-10-26T01:39:29.606 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:29.602+0000 7f22197c0640 1 -- 172.21.15.169:0/2840494932 shutdown_connections 2023-10-26T01:39:29.606 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:29.602+0000 7f22197c0640 1 -- 172.21.15.169:0/2840494932 >> 172.21.15.169:0/2840494932 conn(0x7f2214069930 msgr2=0x7f2214069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:29.607 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:29.602+0000 7f22197c0640 1 -- 172.21.15.169:0/2840494932 shutdown_connections 2023-10-26T01:39:29.607 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:29.602+0000 7f22197c0640 1 -- 172.21.15.169:0/2840494932 wait complete. 2023-10-26T01:39:29.607 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:29.604+0000 7f22197c0640 1 Processor -- start 2023-10-26T01:39:29.608 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:29.604+0000 7f22197c0640 1 -- start start 2023-10-26T01:39:29.609 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:29.605+0000 7f22197c0640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f22000039a0 0x7f2214148630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:29.610 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:29.605+0000 7f22197c0640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22140cd6f0 0x7f22140cdb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:29.610 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:29.605+0000 7f22197c0640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f22140b1330 0x7f22140b17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:29.610 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:29.605+0000 7f22197c0640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22140ce0b0 con 0x7f22140cd6f0 2023-10-26T01:39:29.610 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:29.605+0000 7f22197c0640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22140b1cf0 con 0x7f22000039a0 2023-10-26T01:39:29.611 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:29.605+0000 7f22197c0640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2214150970 con 0x7f22140b1330 2023-10-26T01:39:29.611 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:29.606+0000 7f22127fc640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22140cd6f0 0x7f22140cdb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:29.611 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:29.606+0000 7f22127fc640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22140cd6f0 0x7f22140cdb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:37392/0 (socket says 172.21.15.169:37392) 2023-10-26T01:39:29.611 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:29.606+0000 7f22127fc640 1 -- 172.21.15.169:0/4242483775 learned_addr learned my addr 172.21.15.169:0/4242483775 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:39:29.612 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:29.606+0000 7f2212ffd640 1 --2- 172.21.15.169:0/4242483775 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f22000039a0 0x7f2214148630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:29.612 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:29.606+0000 7f22137fe640 1 --2- 172.21.15.169:0/4242483775 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f22140b1330 0x7f22140b17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:29.612 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:29.606+0000 7f22127fc640 1 -- 172.21.15.169:0/4242483775 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f22140b1330 msgr2=0x7f22140b17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:29.612 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:29.606+0000 7f22127fc640 1 --2- 172.21.15.169:0/4242483775 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f22140b1330 0x7f22140b17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:29.613 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:29.606+0000 7f22127fc640 1 -- 172.21.15.169:0/4242483775 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f22000039a0 msgr2=0x7f2214148630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:29.613 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:29.606+0000 7f22127fc640 1 --2- 172.21.15.169:0/4242483775 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f22000039a0 0x7f2214148630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:29.613 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:29.606+0000 7f22127fc640 1 -- 172.21.15.169:0/4242483775 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2214150c10 con 0x7f22140cd6f0 2023-10-26T01:39:29.613 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:29.606+0000 7f22127fc640 1 --2- 172.21.15.169:0/4242483775 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22140cd6f0 0x7f22140cdb70 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f21fc00c7a0 tx=0x7f21fc00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:29.614 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:29.607+0000 7f220b7fe640 1 -- 172.21.15.169:0/4242483775 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21fc003b70 con 0x7f22140cd6f0 2023-10-26T01:39:29.614 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:29.607+0000 7f220b7fe640 1 -- 172.21.15.169:0/4242483775 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f21fc003cf0 con 0x7f22140cd6f0 2023-10-26T01:39:29.614 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:29.607+0000 7f220b7fe640 1 -- 172.21.15.169:0/4242483775 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21fc003ea0 con 0x7f22140cd6f0 2023-10-26T01:39:29.615 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:29.607+0000 7f22197c0640 1 -- 172.21.15.169:0/4242483775 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2214150ef0 con 0x7f22140cd6f0 2023-10-26T01:39:29.615 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:29.607+0000 7f22197c0640 1 -- 172.21.15.169:0/4242483775 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22141514b0 con 0x7f22140cd6f0 2023-10-26T01:39:29.616 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:29.608+0000 7f22197c0640 1 -- 172.21.15.169:0/4242483775 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f221404db10 con 0x7f22140cd6f0 2023-10-26T01:39:29.616 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:29.609+0000 7f220b7fe640 1 -- 172.21.15.169:0/4242483775 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f21fc00aab0 con 0x7f22140cd6f0 2023-10-26T01:39:29.616 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:29.610+0000 7f220b7fe640 1 --2- 172.21.15.169:0/4242483775 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f21f406ae40 0x7f21f406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:29.617 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:29.610+0000 7f220b7fe640 1 -- 172.21.15.169:0/4242483775 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f21fc089650 con 0x7f22140cd6f0 2023-10-26T01:39:29.617 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:29.610+0000 7f2212ffd640 1 --2- 172.21.15.169:0/4242483775 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f21f406ae40 0x7f21f406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:29.617 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:29.611+0000 7f2212ffd640 1 --2- 172.21.15.169:0/4242483775 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f21f406ae40 0x7f21f406d300 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f2204038000 tx=0x7f22040046c0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:29.618 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:29.614+0000 7f220b7fe640 1 -- 172.21.15.169:0/4242483775 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f21fc0546f0 con 0x7f22140cd6f0 2023-10-26T01:39:29.842 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:29.839+0000 7f22197c0640 1 -- 172.21.15.169:0/4242483775 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f22140aca70 con 0x7f22140cd6f0 2023-10-26T01:39:29.843 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:29.840+0000 7f220b7fe640 1 -- 172.21.15.169:0/4242483775 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+4921 (secure 0 0 0) 0x7f21fc057d10 con 0x7f22140cd6f0 2023-10-26T01:39:29.843 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:39:29.844 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":77,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5711,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/3037106607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":3037106607},{"type":"v1","addr":"172.21.15.170:6836","nonce":3037106607}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72},{"gid":5714,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/1847866184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":1847866184},{"type":"v1","addr":"172.21.15.170:6837","nonce":1847866184}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72},{"gid":8229,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2707290348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2707290348},{"type":"v1","addr":"172.21.15.169:6833","nonce":2707290348}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T01:39:25.790062+0000","modified":"2023-10-26T01:39:28.841639+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8232},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8232":{"gid":8232,"name":"c","rank":0,"incarnation":73,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6835/66540264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":66540264},{"type":"v1","addr":"172.21.15.169:6835","nonce":66540264}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-10-26T01:39:29.844 INFO:teuthology.orchestra.run.smithi169.stderr:dumped fsmap epoch 77 2023-10-26T01:39:29.846 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:29.843+0000 7f22197c0640 1 -- 172.21.15.169:0/4242483775 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f21f406ae40 msgr2=0x7f21f406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:29.846 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:29.843+0000 7f22197c0640 1 --2- 172.21.15.169:0/4242483775 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f21f406ae40 0x7f21f406d300 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f2204038000 tx=0x7f22040046c0 comp rx=0 tx=0).stop 2023-10-26T01:39:29.846 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:29.843+0000 7f22197c0640 1 -- 172.21.15.169:0/4242483775 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22140cd6f0 msgr2=0x7f22140cdb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:29.846 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:29.843+0000 7f22197c0640 1 --2- 172.21.15.169:0/4242483775 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22140cd6f0 0x7f22140cdb70 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f21fc00c7a0 tx=0x7f21fc00cc60 comp rx=0 tx=0).stop 2023-10-26T01:39:29.847 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:29.843+0000 7f22197c0640 1 -- 172.21.15.169:0/4242483775 shutdown_connections 2023-10-26T01:39:29.847 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:29.843+0000 7f22197c0640 1 --2- 172.21.15.169:0/4242483775 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f21f406ae40 0x7f21f406d300 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:29.847 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:29.843+0000 7f22197c0640 1 --2- 172.21.15.169:0/4242483775 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22140cd6f0 0x7f22140cdb70 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:29.847 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:29.843+0000 7f22197c0640 1 --2- 172.21.15.169:0/4242483775 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f22140b1330 0x7f22140b17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:29.847 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:29.843+0000 7f22197c0640 1 --2- 172.21.15.169:0/4242483775 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f22000039a0 0x7f2214148630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:29.848 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:29.843+0000 7f22197c0640 1 -- 172.21.15.169:0/4242483775 >> 172.21.15.169:0/4242483775 conn(0x7f2214069930 msgr2=0x7f22140acd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:29.848 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:29.843+0000 7f22197c0640 1 -- 172.21.15.169:0/4242483775 shutdown_connections 2023-10-26T01:39:29.848 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:29.843+0000 7f22197c0640 1 -- 172.21.15.169:0/4242483775 wait complete. 2023-10-26T01:39:29.863 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:39:30.086 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:30.083+0000 7f8fd520e640 1 -- 172.21.15.169:0/333036131 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8fbc003f40 msgr2=0x7f8fbc0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:30.087 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:30.083+0000 7f8fd520e640 1 --2- 172.21.15.169:0/333036131 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8fbc003f40 0x7f8fbc0047c0 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f8fb8009b50 tx=0x7f8fbc0057f0 comp rx=0 tx=0).stop 2023-10-26T01:39:30.087 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:30.083+0000 7f8fcf59e640 1 -- 172.21.15.169:0/333036131 reap_dead start 2023-10-26T01:39:30.087 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:30.083+0000 7f8fd520e640 1 -- 172.21.15.169:0/333036131 shutdown_connections 2023-10-26T01:39:30.087 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:30.083+0000 7f8fd520e640 1 -- 172.21.15.169:0/333036131 >> 172.21.15.169:0/333036131 conn(0x7f8fd0069930 msgr2=0x7f8fd0069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:30.088 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:30.083+0000 7f8fd520e640 1 -- 172.21.15.169:0/333036131 shutdown_connections 2023-10-26T01:39:30.088 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:30.083+0000 7f8fd520e640 1 -- 172.21.15.169:0/333036131 wait complete. 2023-10-26T01:39:30.091 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:30.085+0000 7f8fd520e640 1 Processor -- start 2023-10-26T01:39:30.091 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:30.085+0000 7f8fd520e640 1 -- start start 2023-10-26T01:39:30.091 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:30.086+0000 7f8fd520e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8fbc003f40 0x7f8fd015e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:30.091 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:30.086+0000 7f8fd520e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8fd0152550 0x7f8fd006f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:30.091 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:30.086+0000 7f8fd520e640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f8fd0152e60 0x7f8fd01532f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:30.092 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:30.086+0000 7f8fd520e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd0153830 con 0x7f8fd0152550 2023-10-26T01:39:30.092 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:30.086+0000 7f8fd520e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd0070300 con 0x7f8fbc003f40 2023-10-26T01:39:30.092 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:30.086+0000 7f8fd520e640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd00705e0 con 0x7f8fd0152e60 2023-10-26T01:39:30.092 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:30.086+0000 7f8fc7fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8fd0152550 0x7f8fd006f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:30.093 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:30.086+0000 7f8fc7fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8fd0152550 0x7f8fd006f990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:37408/0 (socket says 172.21.15.169:37408) 2023-10-26T01:39:30.093 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:30.086+0000 7f8fc7fff640 1 -- 172.21.15.169:0/1145060538 learned_addr learned my addr 172.21.15.169:0/1145060538 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:39:30.093 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:30.087+0000 7f8fced9d640 1 --2- 172.21.15.169:0/1145060538 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8fbc003f40 0x7f8fd015e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:30.093 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:30.087+0000 7f8fcf59e640 1 --2- 172.21.15.169:0/1145060538 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f8fd0152e60 0x7f8fd01532f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:30.093 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:30.087+0000 7f8fc7fff640 1 -- 172.21.15.169:0/1145060538 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f8fd0152e60 msgr2=0x7f8fd01532f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:30.094 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:30.087+0000 7f8fc7fff640 1 --2- 172.21.15.169:0/1145060538 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f8fd0152e60 0x7f8fd01532f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:30.094 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:30.087+0000 7f8fc7fff640 1 -- 172.21.15.169:0/1145060538 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8fbc003f40 msgr2=0x7f8fd015e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:30.094 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:30.087+0000 7f8fc7fff640 1 --2- 172.21.15.169:0/1145060538 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8fbc003f40 0x7f8fd015e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:30.094 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:30.087+0000 7f8fc7fff640 1 -- 172.21.15.169:0/1145060538 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fd00708c0 con 0x7f8fd0152550 2023-10-26T01:39:30.095 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:30.087+0000 7f8fcf59e640 1 --2- 172.21.15.169:0/1145060538 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f8fd0152e60 0x7f8fd01532f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:39:30.095 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:30.087+0000 7f8fc7fff640 1 --2- 172.21.15.169:0/1145060538 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8fd0152550 0x7f8fd006f990 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f8fb8009a70 tx=0x7f8fb8033c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:30.095 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:30.087+0000 7f8fc77fe640 1 -- 172.21.15.169:0/1145060538 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb8004220 con 0x7f8fd0152550 2023-10-26T01:39:30.095 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:30.087+0000 7f8fc77fe640 1 -- 172.21.15.169:0/1145060538 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8fb8004380 con 0x7f8fd0152550 2023-10-26T01:39:30.095 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:30.087+0000 7f8fc77fe640 1 -- 172.21.15.169:0/1145060538 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb803d940 con 0x7f8fd0152550 2023-10-26T01:39:30.096 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:30.087+0000 7f8fd520e640 1 -- 172.21.15.169:0/1145060538 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fd0070ba0 con 0x7f8fd0152550 2023-10-26T01:39:30.096 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:30.088+0000 7f8fd520e640 1 -- 172.21.15.169:0/1145060538 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fd015eb70 con 0x7f8fd0152550 2023-10-26T01:39:30.097 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:30.089+0000 7f8fd520e640 1 -- 172.21.15.169:0/1145060538 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fd015ed00 con 0x7f8fd0152550 2023-10-26T01:39:30.097 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:30.090+0000 7f8fc77fe640 1 -- 172.21.15.169:0/1145060538 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f8fb803daa0 con 0x7f8fd0152550 2023-10-26T01:39:30.098 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:30.091+0000 7f8fc77fe640 1 --2- 172.21.15.169:0/1145060538 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f8fb006ae40 0x7f8fb006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:30.098 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:30.091+0000 7f8fc77fe640 1 -- 172.21.15.169:0/1145060538 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8fb8033e40 con 0x7f8fd0152550 2023-10-26T01:39:30.098 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:30.092+0000 7f8fced9d640 1 --2- 172.21.15.169:0/1145060538 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f8fb006ae40 0x7f8fb006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:30.098 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:30.093+0000 7f8fced9d640 1 --2- 172.21.15.169:0/1145060538 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f8fb006ae40 0x7f8fb006d300 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f8fc0005d00 tx=0x7f8fc0005ed0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:30.099 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:30.096+0000 7f8fc77fe640 1 -- 172.21.15.169:0/1145060538 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8fb8077410 con 0x7f8fd0152550 2023-10-26T01:39:30.277 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:30.274+0000 7f8fd520e640 1 -- 172.21.15.169:0/1145060538 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8fd00713a0 con 0x7f8fd0152550 2023-10-26T01:39:30.279 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:30.277+0000 7f8fc77fe640 1 -- 172.21.15.169:0/1145060538 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v151) v1 ==== 74+0+16628 (secure 0 0 0) 0x7f8fb807aa30 con 0x7f8fd0152550 2023-10-26T01:39:30.280 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:39:30.280 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":151,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:39:27.316963+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-10-26T01:39:23.646624+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"150","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-10-26T01:39:24.813678+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"151","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:39:30.282 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:30.279+0000 7f8fd520e640 1 -- 172.21.15.169:0/1145060538 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f8fb006ae40 msgr2=0x7f8fb006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:30.283 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:30.279+0000 7f8fd520e640 1 --2- 172.21.15.169:0/1145060538 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f8fb006ae40 0x7f8fb006d300 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f8fc0005d00 tx=0x7f8fc0005ed0 comp rx=0 tx=0).stop 2023-10-26T01:39:30.285 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:30.279+0000 7f8fd520e640 1 -- 172.21.15.169:0/1145060538 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8fd0152550 msgr2=0x7f8fd006f990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:30.285 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:30.279+0000 7f8fd520e640 1 --2- 172.21.15.169:0/1145060538 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8fd0152550 0x7f8fd006f990 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f8fb8009a70 tx=0x7f8fb8033c50 comp rx=0 tx=0).stop 2023-10-26T01:39:30.286 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:30.280+0000 7f8fd520e640 1 -- 172.21.15.169:0/1145060538 shutdown_connections 2023-10-26T01:39:30.286 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:30.280+0000 7f8fd520e640 1 --2- 172.21.15.169:0/1145060538 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f8fb006ae40 0x7f8fb006d300 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:30.286 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:30.280+0000 7f8fd520e640 1 --2- 172.21.15.169:0/1145060538 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8fd0152550 0x7f8fd006f990 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:30.286 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:30.280+0000 7f8fd520e640 1 --2- 172.21.15.169:0/1145060538 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f8fd0152e60 0x7f8fd01532f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:30.286 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:30.280+0000 7f8fd520e640 1 --2- 172.21.15.169:0/1145060538 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8fbc003f40 0x7f8fd015e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:30.287 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:30.280+0000 7f8fd520e640 1 -- 172.21.15.169:0/1145060538 >> 172.21.15.169:0/1145060538 conn(0x7f8fd0069930 msgr2=0x7f8fd0159510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:30.287 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:30.280+0000 7f8fd520e640 1 -- 172.21.15.169:0/1145060538 shutdown_connections 2023-10-26T01:39:30.287 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:30.280+0000 7f8fd520e640 1 -- 172.21.15.169:0/1145060538 wait complete. 2023-10-26T01:39:30.301 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-26T01:39:30.301 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-26T01:39:30.521 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:30.518+0000 7f1dfc14c640 1 -- 172.21.15.169:0/3823027520 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1df40cde30 msgr2=0x7f1df40cc440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:30.521 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:30.518+0000 7f1dfc14c640 1 --2- 172.21.15.169:0/3823027520 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1df40cde30 0x7f1df40cc440 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f1dec009ca0 tx=0x7f1dec02c2d0 comp rx=0 tx=0).stop 2023-10-26T01:39:30.522 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:30.518+0000 7f1dfc14c640 1 -- 172.21.15.169:0/3823027520 shutdown_connections 2023-10-26T01:39:30.522 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:30.518+0000 7f1dfc14c640 1 --2- 172.21.15.169:0/3823027520 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1df40cde30 0x7f1df40cc440 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:30.522 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:30.518+0000 7f1dfc14c640 1 -- 172.21.15.169:0/3823027520 >> 172.21.15.169:0/3823027520 conn(0x7f1df4069930 msgr2=0x7f1df4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:30.522 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:30.518+0000 7f1dfc14c640 1 -- 172.21.15.169:0/3823027520 shutdown_connections 2023-10-26T01:39:30.522 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:30.518+0000 7f1dfc14c640 1 -- 172.21.15.169:0/3823027520 wait complete. 2023-10-26T01:39:30.523 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:30.520+0000 7f1dfc14c640 1 Processor -- start 2023-10-26T01:39:30.523 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:30.520+0000 7f1dfc14c640 1 -- start start 2023-10-26T01:39:30.526 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:30.521+0000 7f1dfc14c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1df40cde30 0x7f1df4071240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:30.526 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:30.521+0000 7f1dfc14c640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1df414f2c0 0x7f1df414d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:30.526 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:30.521+0000 7f1dfc14c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1df414f6a0 0x7f1df4150700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:30.526 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:30.521+0000 7f1dfc14c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1df4071780 con 0x7f1df40cde30 2023-10-26T01:39:30.527 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:30.521+0000 7f1dfc14c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1df414d560 con 0x7f1df414f6a0 2023-10-26T01:39:30.527 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:30.521+0000 7f1dfc14c640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1df4150270 con 0x7f1df414f2c0 2023-10-26T01:39:30.527 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:30.521+0000 7f1df9ee8640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1df40cde30 0x7f1df4071240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:30.527 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:30.522+0000 7f1df9ee8640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1df40cde30 0x7f1df4071240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:37422/0 (socket says 172.21.15.169:37422) 2023-10-26T01:39:30.527 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:30.522+0000 7f1df9ee8640 1 -- 172.21.15.169:0/3218794934 learned_addr learned my addr 172.21.15.169:0/3218794934 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:39:30.528 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:30.522+0000 7f1dfa6e9640 1 --2- 172.21.15.169:0/3218794934 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1df414f6a0 0x7f1df4150700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:30.528 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:30.522+0000 7f1df96e7640 1 --2- 172.21.15.169:0/3218794934 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1df414f2c0 0x7f1df414d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:30.528 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:30.522+0000 7f1df9ee8640 1 -- 172.21.15.169:0/3218794934 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1df414f2c0 msgr2=0x7f1df414d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:30.528 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:30.522+0000 7f1df9ee8640 1 --2- 172.21.15.169:0/3218794934 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1df414f2c0 0x7f1df414d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:30.528 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:30.522+0000 7f1df9ee8640 1 -- 172.21.15.169:0/3218794934 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1df414f6a0 msgr2=0x7f1df4150700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:30.529 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:30.522+0000 7f1df9ee8640 1 --2- 172.21.15.169:0/3218794934 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1df414f6a0 0x7f1df4150700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:30.529 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:30.522+0000 7f1df9ee8640 1 -- 172.21.15.169:0/3218794934 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dec009900 con 0x7f1df40cde30 2023-10-26T01:39:30.529 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:30.522+0000 7f1dfa6e9640 1 --2- 172.21.15.169:0/3218794934 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1df414f6a0 0x7f1df4150700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:39:30.529 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:30.522+0000 7f1df96e7640 1 --2- 172.21.15.169:0/3218794934 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1df414f2c0 0x7f1df414d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:39:30.529 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:30.522+0000 7f1df9ee8640 1 --2- 172.21.15.169:0/3218794934 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1df40cde30 0x7f1df4071240 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f1df000c450 tx=0x7f1df000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:30.530 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:30.523+0000 7f1dea7fc640 1 -- 172.21.15.169:0/3218794934 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df00052d0 con 0x7f1df40cde30 2023-10-26T01:39:30.530 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:30.523+0000 7f1dea7fc640 1 -- 172.21.15.169:0/3218794934 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1df0005430 con 0x7f1df40cde30 2023-10-26T01:39:30.530 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:30.523+0000 7f1dea7fc640 1 -- 172.21.15.169:0/3218794934 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df0012890 con 0x7f1df40cde30 2023-10-26T01:39:30.530 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:30.523+0000 7f1dfc14c640 1 -- 172.21.15.169:0/3218794934 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1df41f0d60 con 0x7f1df40cde30 2023-10-26T01:39:30.531 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:30.523+0000 7f1dfc14c640 1 -- 172.21.15.169:0/3218794934 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1df414ff00 con 0x7f1df40cde30 2023-10-26T01:39:30.532 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:30.524+0000 7f1dfc14c640 1 -- 172.21.15.169:0/3218794934 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1df414fa80 con 0x7f1df40cde30 2023-10-26T01:39:30.532 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:30.525+0000 7f1dea7fc640 1 -- 172.21.15.169:0/3218794934 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f1df00035f0 con 0x7f1df40cde30 2023-10-26T01:39:30.532 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:30.526+0000 7f1dea7fc640 1 --2- 172.21.15.169:0/3218794934 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f1dd806ae40 0x7f1dd806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:30.532 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:30.526+0000 7f1dea7fc640 1 -- 172.21.15.169:0/3218794934 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1df0089350 con 0x7f1df40cde30 2023-10-26T01:39:30.532 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:30.526+0000 7f1df96e7640 1 --2- 172.21.15.169:0/3218794934 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f1dd806ae40 0x7f1dd806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:30.533 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:30.527+0000 7f1df96e7640 1 --2- 172.21.15.169:0/3218794934 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f1dd806ae40 0x7f1dd806d300 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f1df4152a80 tx=0x7f1de4008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:30.533 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:30.530+0000 7f1dea7fc640 1 -- 172.21.15.169:0/3218794934 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1df414fa80 con 0x7f1df40cde30 2023-10-26T01:39:30.703 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:30.700+0000 7f1dfc14c640 1 -- 172.21.15.169:0/3218794934 --> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1df40ab660 con 0x7f1dd806ae40 2023-10-26T01:39:30.727 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:30.724+0000 7f1dea7fc640 1 -- 172.21.15.169:0/3218794934 <== mgr.4112 v2:172.21.15.170:6832/1187156918 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+278073 (secure 0 0 0) 0x7f1de400ad60 con 0x7f1dd806ae40 2023-10-26T01:39:30.731 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:39:30.732 INFO:teuthology.orchestra.run.smithi169.stderr:dumped all 2023-10-26T01:39:30.734 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:30.730+0000 7f1dfc14c640 1 -- 172.21.15.169:0/3218794934 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f1dd806ae40 msgr2=0x7f1dd806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:30.734 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:30.730+0000 7f1dfc14c640 1 --2- 172.21.15.169:0/3218794934 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f1dd806ae40 0x7f1dd806d300 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f1df4152a80 tx=0x7f1de4008040 comp rx=0 tx=0).stop 2023-10-26T01:39:30.734 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:30.730+0000 7f1dfc14c640 1 -- 172.21.15.169:0/3218794934 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1df40cde30 msgr2=0x7f1df4071240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:30.734 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:30.730+0000 7f1dfc14c640 1 --2- 172.21.15.169:0/3218794934 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1df40cde30 0x7f1df4071240 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f1df000c450 tx=0x7f1df000c910 comp rx=0 tx=0).stop 2023-10-26T01:39:30.734 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:30.730+0000 7f1dfc14c640 1 -- 172.21.15.169:0/3218794934 shutdown_connections 2023-10-26T01:39:30.735 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:30.730+0000 7f1dfc14c640 1 --2- 172.21.15.169:0/3218794934 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f1dd806ae40 0x7f1dd806d300 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:30.735 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:30.730+0000 7f1dfc14c640 1 --2- 172.21.15.169:0/3218794934 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1df414f6a0 0x7f1df4150700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:30.735 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:30.730+0000 7f1dfc14c640 1 --2- 172.21.15.169:0/3218794934 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1df414f2c0 0x7f1df414d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:30.735 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:30.730+0000 7f1dfc14c640 1 --2- 172.21.15.169:0/3218794934 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1df40cde30 0x7f1df4071240 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:30.735 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:30.730+0000 7f1dfc14c640 1 -- 172.21.15.169:0/3218794934 >> 172.21.15.169:0/3218794934 conn(0x7f1df4069930 msgr2=0x7f1df414d720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:30.736 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:30.730+0000 7f1dfc14c640 1 -- 172.21.15.169:0/3218794934 shutdown_connections 2023-10-26T01:39:30.736 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:30.730+0000 7f1dfc14c640 1 -- 172.21.15.169:0/3218794934 wait complete. 2023-10-26T01:39:30.750 INFO:teuthology.orchestra.run.smithi169.stdout:{"pg_ready":true,"pg_map":{"version":414,"stamp":"2023-10-26T01:39:30.320024+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118,"ondisk_log_size":118,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":33,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":7623,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1554876,"statfs":{"total":1600000000,"available":1592193296,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.004175"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.339741+0000","last_change":"2023-10-26T01:39:25.339741+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.339741+0000","last_undegraded":"2023-10-26T01:39:25.339741+0000","last_fullsized":"2023-10-26T01:39:25.339741+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.342024+0000","last_change":"2023-10-26T01:39:25.342024+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.342024+0000","last_undegraded":"2023-10-26T01:39:25.342024+0000","last_fullsized":"2023-10-26T01:39:25.342024+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.349198+0000","last_change":"2023-10-26T01:39:25.349198+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.349198+0000","last_undegraded":"2023-10-26T01:39:25.349198+0000","last_fullsized":"2023-10-26T01:39:25.349198+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.347440+0000","last_change":"2023-10-26T01:39:25.347440+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.347440+0000","last_undegraded":"2023-10-26T01:39:25.347440+0000","last_fullsized":"2023-10-26T01:39:25.347440+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.337848+0000","last_change":"2023-10-26T01:39:25.337848+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.337848+0000","last_undegraded":"2023-10-26T01:39:25.337848+0000","last_fullsized":"2023-10-26T01:39:25.337848+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.343578+0000","last_change":"2023-10-26T01:39:25.343578+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.343578+0000","last_undegraded":"2023-10-26T01:39:25.343578+0000","last_fullsized":"2023-10-26T01:39:25.343578+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.345665+0000","last_change":"2023-10-26T01:39:25.345665+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.345665+0000","last_undegraded":"2023-10-26T01:39:25.345665+0000","last_fullsized":"2023-10-26T01:39:25.345665+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.348666+0000","last_change":"2023-10-26T01:39:25.348666+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.348666+0000","last_undegraded":"2023-10-26T01:39:25.348666+0000","last_fullsized":"2023-10-26T01:39:25.348666+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.349996+0000","last_change":"2023-10-26T01:39:25.349996+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.349996+0000","last_undegraded":"2023-10-26T01:39:25.349996+0000","last_fullsized":"2023-10-26T01:39:25.349996+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.342807+0000","last_change":"2023-10-26T01:39:25.342807+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.342807+0000","last_undegraded":"2023-10-26T01:39:25.342807+0000","last_fullsized":"2023-10-26T01:39:25.342807+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.340008+0000","last_change":"2023-10-26T01:39:25.340008+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.340008+0000","last_undegraded":"2023-10-26T01:39:25.340008+0000","last_fullsized":"2023-10-26T01:39:25.340008+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.348637+0000","last_change":"2023-10-26T01:39:25.348637+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.348637+0000","last_undegraded":"2023-10-26T01:39:25.348637+0000","last_fullsized":"2023-10-26T01:39:25.348637+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.343722+0000","last_change":"2023-10-26T01:39:25.343722+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.343722+0000","last_undegraded":"2023-10-26T01:39:25.343722+0000","last_fullsized":"2023-10-26T01:39:25.343722+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.343921+0000","last_change":"2023-10-26T01:39:25.343921+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.343921+0000","last_undegraded":"2023-10-26T01:39:25.343921+0000","last_fullsized":"2023-10-26T01:39:25.343921+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.345622+0000","last_change":"2023-10-26T01:39:25.345622+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.345622+0000","last_undegraded":"2023-10-26T01:39:25.345622+0000","last_fullsized":"2023-10-26T01:39:25.345622+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":9,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-26T01:39:26.356388+0000","last_change":"2023-10-26T01:39:26.355761+0000","last_active":"2023-10-26T01:39:26.356388+0000","last_peered":"2023-10-26T01:39:26.356388+0000","last_clean":"2023-10-26T01:39:26.356388+0000","last_became_active":"2023-10-26T01:39:26.355188+0000","last_became_peered":"2023-10-26T01:39:26.355188+0000","last_unstale":"2023-10-26T01:39:26.356388+0000","last_undegraded":"2023-10-26T01:39:26.356388+0000","last_fullsized":"2023-10-26T01:39:26.356388+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:19:25.353033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.341466+0000","last_change":"2023-10-26T01:39:25.341466+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.341466+0000","last_undegraded":"2023-10-26T01:39:25.341466+0000","last_fullsized":"2023-10-26T01:39:25.341466+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":9,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-26T01:39:26.348459+0000","last_change":"2023-10-26T01:39:26.348373+0000","last_active":"2023-10-26T01:39:26.348459+0000","last_peered":"2023-10-26T01:39:26.348459+0000","last_clean":"2023-10-26T01:39:26.348459+0000","last_became_active":"2023-10-26T01:39:26.346867+0000","last_became_peered":"2023-10-26T01:39:26.346867+0000","last_unstale":"2023-10-26T01:39:26.348459+0000","last_undegraded":"2023-10-26T01:39:26.348459+0000","last_fullsized":"2023-10-26T01:39:26.348459+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:21:37.836002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.341042+0000","last_change":"2023-10-26T01:39:25.341042+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.341042+0000","last_undegraded":"2023-10-26T01:39:25.341042+0000","last_fullsized":"2023-10-26T01:39:25.341042+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":9,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-26T01:39:26.351465+0000","last_change":"2023-10-26T01:39:26.351200+0000","last_active":"2023-10-26T01:39:26.351465+0000","last_peered":"2023-10-26T01:39:26.351465+0000","last_clean":"2023-10-26T01:39:26.351465+0000","last_became_active":"2023-10-26T01:39:26.350941+0000","last_became_peered":"2023-10-26T01:39:26.350941+0000","last_unstale":"2023-10-26T01:39:26.351465+0000","last_undegraded":"2023-10-26T01:39:26.351465+0000","last_fullsized":"2023-10-26T01:39:26.351465+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:27:04.484703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.338200+0000","last_change":"2023-10-26T01:39:25.338200+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.338200+0000","last_undegraded":"2023-10-26T01:39:25.338200+0000","last_fullsized":"2023-10-26T01:39:25.338200+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.339459+0000","last_change":"2023-10-26T01:39:25.339459+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.339459+0000","last_undegraded":"2023-10-26T01:39:25.339459+0000","last_fullsized":"2023-10-26T01:39:25.339459+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.347072+0000","last_change":"2023-10-26T01:39:25.347072+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.347072+0000","last_undegraded":"2023-10-26T01:39:25.347072+0000","last_fullsized":"2023-10-26T01:39:25.347072+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.340599+0000","last_change":"2023-10-26T01:39:25.340599+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.340599+0000","last_undegraded":"2023-10-26T01:39:25.340599+0000","last_fullsized":"2023-10-26T01:39:25.340599+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.342364+0000","last_change":"2023-10-26T01:39:25.342364+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.342364+0000","last_undegraded":"2023-10-26T01:39:25.342364+0000","last_fullsized":"2023-10-26T01:39:25.342364+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":9,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-26T01:39:26.356448+0000","last_change":"2023-10-26T01:39:26.355914+0000","last_active":"2023-10-26T01:39:26.356448+0000","last_peered":"2023-10-26T01:39:26.356448+0000","last_clean":"2023-10-26T01:39:26.356448+0000","last_became_active":"2023-10-26T01:39:26.355508+0000","last_became_peered":"2023-10-26T01:39:26.355508+0000","last_unstale":"2023-10-26T01:39:26.356448+0000","last_undegraded":"2023-10-26T01:39:26.356448+0000","last_fullsized":"2023-10-26T01:39:26.356448+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:15:35.499430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.341388+0000","last_change":"2023-10-26T01:39:25.341388+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.341388+0000","last_undegraded":"2023-10-26T01:39:25.341388+0000","last_fullsized":"2023-10-26T01:39:25.341388+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.340994+0000","last_change":"2023-10-26T01:39:25.340994+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.340994+0000","last_undegraded":"2023-10-26T01:39:25.340994+0000","last_fullsized":"2023-10-26T01:39:25.340994+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.346960+0000","last_change":"2023-10-26T01:39:25.346960+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.346960+0000","last_undegraded":"2023-10-26T01:39:25.346960+0000","last_fullsized":"2023-10-26T01:39:25.346960+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.345158+0000","last_change":"2023-10-26T01:39:25.345158+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.345158+0000","last_undegraded":"2023-10-26T01:39:25.345158+0000","last_fullsized":"2023-10-26T01:39:25.345158+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.337346+0000","last_change":"2023-10-26T01:39:25.337346+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.337346+0000","last_undegraded":"2023-10-26T01:39:25.337346+0000","last_fullsized":"2023-10-26T01:39:25.337346+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.340549+0000","last_change":"2023-10-26T01:39:25.340549+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.340549+0000","last_undegraded":"2023-10-26T01:39:25.340549+0000","last_fullsized":"2023-10-26T01:39:25.340549+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":9,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-26T01:39:26.344302+0000","last_change":"2023-10-26T01:39:26.344022+0000","last_active":"2023-10-26T01:39:26.344302+0000","last_peered":"2023-10-26T01:39:26.344302+0000","last_clean":"2023-10-26T01:39:26.344302+0000","last_became_active":"2023-10-26T01:39:26.341874+0000","last_became_peered":"2023-10-26T01:39:26.341874+0000","last_unstale":"2023-10-26T01:39:26.344302+0000","last_undegraded":"2023-10-26T01:39:26.344302+0000","last_fullsized":"2023-10-26T01:39:26.344302+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:34:17.840143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-26T01:39:25.347668+0000","last_change":"2023-10-26T01:39:25.347523+0000","last_active":"2023-10-26T01:39:25.347668+0000","last_peered":"2023-10-26T01:39:25.347668+0000","last_clean":"2023-10-26T01:39:25.347668+0000","last_became_active":"2023-10-26T01:39:25.346388+0000","last_became_peered":"2023-10-26T01:39:25.346388+0000","last_unstale":"2023-10-26T01:39:25.347668+0000","last_undegraded":"2023-10-26T01:39:25.347668+0000","last_fullsized":"2023-10-26T01:39:25.347668+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:01:31.862431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.346506+0000","last_change":"2023-10-26T01:39:25.346506+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.346506+0000","last_undegraded":"2023-10-26T01:39:25.346506+0000","last_fullsized":"2023-10-26T01:39:25.346506+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-26T01:39:25.348380+0000","last_change":"2023-10-26T01:39:25.347683+0000","last_active":"2023-10-26T01:39:25.348380+0000","last_peered":"2023-10-26T01:39:25.348380+0000","last_clean":"2023-10-26T01:39:25.348380+0000","last_became_active":"2023-10-26T01:39:25.346998+0000","last_became_peered":"2023-10-26T01:39:25.346998+0000","last_unstale":"2023-10-26T01:39:25.348380+0000","last_undegraded":"2023-10-26T01:39:25.348380+0000","last_fullsized":"2023-10-26T01:39:25.348380+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:08:33.196097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.335905+0000","last_change":"2023-10-26T01:39:25.335905+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.335905+0000","last_undegraded":"2023-10-26T01:39:25.335905+0000","last_fullsized":"2023-10-26T01:39:25.335905+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-26T01:39:25.352874+0000","last_change":"2023-10-26T01:39:25.352802+0000","last_active":"2023-10-26T01:39:25.352874+0000","last_peered":"2023-10-26T01:39:25.352874+0000","last_clean":"2023-10-26T01:39:25.352874+0000","last_became_active":"2023-10-26T01:39:25.352537+0000","last_became_peered":"2023-10-26T01:39:25.352537+0000","last_unstale":"2023-10-26T01:39:25.352874+0000","last_undegraded":"2023-10-26T01:39:25.352874+0000","last_fullsized":"2023-10-26T01:39:25.352874+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:43:42.148308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.336364+0000","last_change":"2023-10-26T01:39:25.336364+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.336364+0000","last_undegraded":"2023-10-26T01:39:25.336364+0000","last_fullsized":"2023-10-26T01:39:25.336364+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-26T01:39:25.348146+0000","last_change":"2023-10-26T01:39:25.347942+0000","last_active":"2023-10-26T01:39:25.348146+0000","last_peered":"2023-10-26T01:39:25.348146+0000","last_clean":"2023-10-26T01:39:25.348146+0000","last_became_active":"2023-10-26T01:39:25.347373+0000","last_became_peered":"2023-10-26T01:39:25.347373+0000","last_unstale":"2023-10-26T01:39:25.348146+0000","last_undegraded":"2023-10-26T01:39:25.348146+0000","last_fullsized":"2023-10-26T01:39:25.348146+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:56:13.935683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":9,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-26T01:39:26.348723+0000","last_change":"2023-10-26T01:39:26.348640+0000","last_active":"2023-10-26T01:39:26.348723+0000","last_peered":"2023-10-26T01:39:26.348723+0000","last_clean":"2023-10-26T01:39:26.348723+0000","last_became_active":"2023-10-26T01:39:26.348127+0000","last_became_peered":"2023-10-26T01:39:26.348127+0000","last_unstale":"2023-10-26T01:39:26.348723+0000","last_undegraded":"2023-10-26T01:39:26.348723+0000","last_fullsized":"2023-10-26T01:39:26.348723+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:35:50.504438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-26T01:39:25.351507+0000","last_change":"2023-10-26T01:39:25.351159+0000","last_active":"2023-10-26T01:39:25.351507+0000","last_peered":"2023-10-26T01:39:25.351507+0000","last_clean":"2023-10-26T01:39:25.351507+0000","last_became_active":"2023-10-26T01:39:25.349181+0000","last_became_peered":"2023-10-26T01:39:25.349181+0000","last_unstale":"2023-10-26T01:39:25.351507+0000","last_undegraded":"2023-10-26T01:39:25.351507+0000","last_fullsized":"2023-10-26T01:39:25.351507+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:32:20.357196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.339661+0000","last_change":"2023-10-26T01:39:25.339661+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.339661+0000","last_undegraded":"2023-10-26T01:39:25.339661+0000","last_fullsized":"2023-10-26T01:39:25.339661+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"150'1","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-26T01:39:26.362881+0000","last_change":"2023-10-26T01:39:25.352234+0000","last_active":"2023-10-26T01:39:26.362881+0000","last_peered":"2023-10-26T01:39:26.362881+0000","last_clean":"2023-10-26T01:39:26.362881+0000","last_became_active":"2023-10-26T01:39:25.351930+0000","last_became_peered":"2023-10-26T01:39:25.351930+0000","last_unstale":"2023-10-26T01:39:26.362881+0000","last_undegraded":"2023-10-26T01:39:26.362881+0000","last_fullsized":"2023-10-26T01:39:26.362881+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:23:47.193891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.345341+0000","last_change":"2023-10-26T01:39:25.345341+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.345341+0000","last_undegraded":"2023-10-26T01:39:25.345341+0000","last_fullsized":"2023-10-26T01:39:25.345341+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-26T01:39:25.345560+0000","last_change":"2023-10-26T01:39:25.345280+0000","last_active":"2023-10-26T01:39:25.345560+0000","last_peered":"2023-10-26T01:39:25.345560+0000","last_clean":"2023-10-26T01:39:25.345560+0000","last_became_active":"2023-10-26T01:39:25.344841+0000","last_became_peered":"2023-10-26T01:39:25.344841+0000","last_unstale":"2023-10-26T01:39:25.345560+0000","last_undegraded":"2023-10-26T01:39:25.345560+0000","last_fullsized":"2023-10-26T01:39:25.345560+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:01:31.862431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.339971+0000","last_change":"2023-10-26T01:39:25.339971+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.339971+0000","last_undegraded":"2023-10-26T01:39:25.339971+0000","last_fullsized":"2023-10-26T01:39:25.339971+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-26T01:39:25.355216+0000","last_change":"2023-10-26T01:39:25.355147+0000","last_active":"2023-10-26T01:39:25.355216+0000","last_peered":"2023-10-26T01:39:25.355216+0000","last_clean":"2023-10-26T01:39:25.355216+0000","last_became_active":"2023-10-26T01:39:25.354564+0000","last_became_peered":"2023-10-26T01:39:25.354564+0000","last_unstale":"2023-10-26T01:39:25.355216+0000","last_undegraded":"2023-10-26T01:39:25.355216+0000","last_fullsized":"2023-10-26T01:39:25.355216+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:08:33.196097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.340967+0000","last_change":"2023-10-26T01:39:25.340967+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.340967+0000","last_undegraded":"2023-10-26T01:39:25.340967+0000","last_fullsized":"2023-10-26T01:39:25.340967+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-26T01:39:25.354084+0000","last_change":"2023-10-26T01:39:25.353938+0000","last_active":"2023-10-26T01:39:25.354084+0000","last_peered":"2023-10-26T01:39:25.354084+0000","last_clean":"2023-10-26T01:39:25.354084+0000","last_became_active":"2023-10-26T01:39:25.353704+0000","last_became_peered":"2023-10-26T01:39:25.353704+0000","last_unstale":"2023-10-26T01:39:25.354084+0000","last_undegraded":"2023-10-26T01:39:25.354084+0000","last_fullsized":"2023-10-26T01:39:25.354084+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:28:46.712484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.344566+0000","last_change":"2023-10-26T01:39:25.344566+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.344566+0000","last_undegraded":"2023-10-26T01:39:25.344566+0000","last_fullsized":"2023-10-26T01:39:25.344566+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":11,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-26T01:39:26.333356+0000","last_change":"2023-10-26T01:39:25.353364+0000","last_active":"2023-10-26T01:39:26.333356+0000","last_peered":"2023-10-26T01:39:26.333356+0000","last_clean":"2023-10-26T01:39:26.333356+0000","last_became_active":"2023-10-26T01:39:25.353129+0000","last_became_peered":"2023-10-26T01:39:25.353129+0000","last_unstale":"2023-10-26T01:39:26.333356+0000","last_undegraded":"2023-10-26T01:39:26.333356+0000","last_fullsized":"2023-10-26T01:39:26.333356+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:07:24.731619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.332527+0000","last_change":"2023-10-26T01:39:25.332527+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.332527+0000","last_undegraded":"2023-10-26T01:39:25.332527+0000","last_fullsized":"2023-10-26T01:39:25.332527+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-26T01:39:25.353094+0000","last_change":"2023-10-26T01:39:25.352244+0000","last_active":"2023-10-26T01:39:25.353094+0000","last_peered":"2023-10-26T01:39:25.353094+0000","last_clean":"2023-10-26T01:39:25.353094+0000","last_became_active":"2023-10-26T01:39:25.351989+0000","last_became_peered":"2023-10-26T01:39:25.351989+0000","last_unstale":"2023-10-26T01:39:25.353094+0000","last_undegraded":"2023-10-26T01:39:25.353094+0000","last_fullsized":"2023-10-26T01:39:25.353094+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:56:13.935683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.333835+0000","last_change":"2023-10-26T01:39:25.333835+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.333835+0000","last_undegraded":"2023-10-26T01:39:25.333835+0000","last_fullsized":"2023-10-26T01:39:25.333835+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-26T01:39:25.354938+0000","last_change":"2023-10-26T01:39:25.354828+0000","last_active":"2023-10-26T01:39:25.354938+0000","last_peered":"2023-10-26T01:39:25.354938+0000","last_clean":"2023-10-26T01:39:25.354938+0000","last_became_active":"2023-10-26T01:39:25.354185+0000","last_became_peered":"2023-10-26T01:39:25.354185+0000","last_unstale":"2023-10-26T01:39:25.354938+0000","last_undegraded":"2023-10-26T01:39:25.354938+0000","last_fullsized":"2023-10-26T01:39:25.354938+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:32:50.379779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.344701+0000","last_change":"2023-10-26T01:39:25.344701+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.344701+0000","last_undegraded":"2023-10-26T01:39:25.344701+0000","last_fullsized":"2023-10-26T01:39:25.344701+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-26T01:39:25.353183+0000","last_change":"2023-10-26T01:39:25.350711+0000","last_active":"2023-10-26T01:39:25.353183+0000","last_peered":"2023-10-26T01:39:25.353183+0000","last_clean":"2023-10-26T01:39:25.353183+0000","last_became_active":"2023-10-26T01:39:25.349869+0000","last_became_peered":"2023-10-26T01:39:25.349869+0000","last_unstale":"2023-10-26T01:39:25.353183+0000","last_undegraded":"2023-10-26T01:39:25.353183+0000","last_fullsized":"2023-10-26T01:39:25.353183+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:30:24.885507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.340874+0000","last_change":"2023-10-26T01:39:25.340874+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.340874+0000","last_undegraded":"2023-10-26T01:39:25.340874+0000","last_fullsized":"2023-10-26T01:39:25.340874+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-26T01:39:25.342677+0000","last_change":"2023-10-26T01:39:25.342525+0000","last_active":"2023-10-26T01:39:25.342677+0000","last_peered":"2023-10-26T01:39:25.342677+0000","last_clean":"2023-10-26T01:39:25.342677+0000","last_became_active":"2023-10-26T01:39:25.341183+0000","last_became_peered":"2023-10-26T01:39:25.341183+0000","last_unstale":"2023-10-26T01:39:25.342677+0000","last_undegraded":"2023-10-26T01:39:25.342677+0000","last_fullsized":"2023-10-26T01:39:25.342677+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:07:38.898672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.344820+0000","last_change":"2023-10-26T01:39:25.344820+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.344820+0000","last_undegraded":"2023-10-26T01:39:25.344820+0000","last_fullsized":"2023-10-26T01:39:25.344820+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-26T01:39:25.355255+0000","last_change":"2023-10-26T01:39:25.355183+0000","last_active":"2023-10-26T01:39:25.355255+0000","last_peered":"2023-10-26T01:39:25.355255+0000","last_clean":"2023-10-26T01:39:25.355255+0000","last_became_active":"2023-10-26T01:39:25.354751+0000","last_became_peered":"2023-10-26T01:39:25.354751+0000","last_unstale":"2023-10-26T01:39:25.355255+0000","last_undegraded":"2023-10-26T01:39:25.355255+0000","last_fullsized":"2023-10-26T01:39:25.355255+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:39:10.557327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.343890+0000","last_change":"2023-10-26T01:39:25.343890+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.343890+0000","last_undegraded":"2023-10-26T01:39:25.343890+0000","last_fullsized":"2023-10-26T01:39:25.343890+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-26T01:39:25.350561+0000","last_change":"2023-10-26T01:39:25.350279+0000","last_active":"2023-10-26T01:39:25.350561+0000","last_peered":"2023-10-26T01:39:25.350561+0000","last_clean":"2023-10-26T01:39:25.350561+0000","last_became_active":"2023-10-26T01:39:25.349301+0000","last_became_peered":"2023-10-26T01:39:25.349301+0000","last_unstale":"2023-10-26T01:39:25.350561+0000","last_undegraded":"2023-10-26T01:39:25.350561+0000","last_fullsized":"2023-10-26T01:39:25.350561+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:02:52.785225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.342089+0000","last_change":"2023-10-26T01:39:25.342089+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.342089+0000","last_undegraded":"2023-10-26T01:39:25.342089+0000","last_fullsized":"2023-10-26T01:39:25.342089+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-26T01:39:25.355575+0000","last_change":"2023-10-26T01:39:25.355508+0000","last_active":"2023-10-26T01:39:25.355575+0000","last_peered":"2023-10-26T01:39:25.355575+0000","last_clean":"2023-10-26T01:39:25.355575+0000","last_became_active":"2023-10-26T01:39:25.355254+0000","last_became_peered":"2023-10-26T01:39:25.355254+0000","last_unstale":"2023-10-26T01:39:25.355575+0000","last_undegraded":"2023-10-26T01:39:25.355575+0000","last_fullsized":"2023-10-26T01:39:25.355575+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:12:53.520877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":9,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-26T01:39:26.354933+0000","last_change":"2023-10-26T01:39:26.354034+0000","last_active":"2023-10-26T01:39:26.354933+0000","last_peered":"2023-10-26T01:39:26.354933+0000","last_clean":"2023-10-26T01:39:26.354933+0000","last_became_active":"2023-10-26T01:39:26.353811+0000","last_became_peered":"2023-10-26T01:39:26.353811+0000","last_unstale":"2023-10-26T01:39:26.354933+0000","last_undegraded":"2023-10-26T01:39:26.354933+0000","last_fullsized":"2023-10-26T01:39:26.354933+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:53:55.836062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-26T01:39:25.350542+0000","last_change":"2023-10-26T01:39:25.348929+0000","last_active":"2023-10-26T01:39:25.350542+0000","last_peered":"2023-10-26T01:39:25.350542+0000","last_clean":"2023-10-26T01:39:25.350542+0000","last_became_active":"2023-10-26T01:39:25.348637+0000","last_became_peered":"2023-10-26T01:39:25.348637+0000","last_unstale":"2023-10-26T01:39:25.350542+0000","last_undegraded":"2023-10-26T01:39:25.350542+0000","last_fullsized":"2023-10-26T01:39:25.350542+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:16:32.808798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.340936+0000","last_change":"2023-10-26T01:39:25.340936+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.340936+0000","last_undegraded":"2023-10-26T01:39:25.340936+0000","last_fullsized":"2023-10-26T01:39:25.340936+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-26T01:39:25.353113+0000","last_change":"2023-10-26T01:39:25.352675+0000","last_active":"2023-10-26T01:39:25.353113+0000","last_peered":"2023-10-26T01:39:25.353113+0000","last_clean":"2023-10-26T01:39:25.353113+0000","last_became_active":"2023-10-26T01:39:25.352188+0000","last_became_peered":"2023-10-26T01:39:25.352188+0000","last_unstale":"2023-10-26T01:39:25.353113+0000","last_undegraded":"2023-10-26T01:39:25.353113+0000","last_fullsized":"2023-10-26T01:39:25.353113+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:07:38.898672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.346409+0000","last_change":"2023-10-26T01:39:25.346409+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.346409+0000","last_undegraded":"2023-10-26T01:39:25.346409+0000","last_fullsized":"2023-10-26T01:39:25.346409+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-26T01:39:25.354022+0000","last_change":"2023-10-26T01:39:25.353938+0000","last_active":"2023-10-26T01:39:25.354022+0000","last_peered":"2023-10-26T01:39:25.354022+0000","last_clean":"2023-10-26T01:39:25.354022+0000","last_became_active":"2023-10-26T01:39:25.353665+0000","last_became_peered":"2023-10-26T01:39:25.353665+0000","last_unstale":"2023-10-26T01:39:25.354022+0000","last_undegraded":"2023-10-26T01:39:25.354022+0000","last_fullsized":"2023-10-26T01:39:25.354022+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:32:20.357196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.342827+0000","last_change":"2023-10-26T01:39:25.342827+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.342827+0000","last_undegraded":"2023-10-26T01:39:25.342827+0000","last_fullsized":"2023-10-26T01:39:25.342827+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":11,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-26T01:39:26.328128+0000","last_change":"2023-10-26T01:39:25.355042+0000","last_active":"2023-10-26T01:39:26.328128+0000","last_peered":"2023-10-26T01:39:26.328128+0000","last_clean":"2023-10-26T01:39:26.328128+0000","last_became_active":"2023-10-26T01:39:25.354798+0000","last_became_peered":"2023-10-26T01:39:25.354798+0000","last_unstale":"2023-10-26T01:39:26.328128+0000","last_undegraded":"2023-10-26T01:39:26.328128+0000","last_fullsized":"2023-10-26T01:39:26.328128+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:34:40.255257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.340172+0000","last_change":"2023-10-26T01:39:25.340172+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.340172+0000","last_undegraded":"2023-10-26T01:39:25.340172+0000","last_fullsized":"2023-10-26T01:39:25.340172+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":11,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-26T01:39:26.342348+0000","last_change":"2023-10-26T01:39:25.355043+0000","last_active":"2023-10-26T01:39:26.342348+0000","last_peered":"2023-10-26T01:39:26.342348+0000","last_clean":"2023-10-26T01:39:26.342348+0000","last_became_active":"2023-10-26T01:39:25.354518+0000","last_became_peered":"2023-10-26T01:39:25.354518+0000","last_unstale":"2023-10-26T01:39:26.342348+0000","last_undegraded":"2023-10-26T01:39:26.342348+0000","last_fullsized":"2023-10-26T01:39:26.342348+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:37:06.664661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.335577+0000","last_change":"2023-10-26T01:39:25.335577+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.335577+0000","last_undegraded":"2023-10-26T01:39:25.335577+0000","last_fullsized":"2023-10-26T01:39:25.335577+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-26T01:39:25.355626+0000","last_change":"2023-10-26T01:39:25.355568+0000","last_active":"2023-10-26T01:39:25.355626+0000","last_peered":"2023-10-26T01:39:25.355626+0000","last_clean":"2023-10-26T01:39:25.355626+0000","last_became_active":"2023-10-26T01:39:25.354482+0000","last_became_peered":"2023-10-26T01:39:25.354482+0000","last_unstale":"2023-10-26T01:39:25.355626+0000","last_undegraded":"2023-10-26T01:39:25.355626+0000","last_fullsized":"2023-10-26T01:39:25.355626+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:10:56.447860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.333593+0000","last_change":"2023-10-26T01:39:25.333593+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.333593+0000","last_undegraded":"2023-10-26T01:39:25.333593+0000","last_fullsized":"2023-10-26T01:39:25.333593+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-26T01:39:25.350624+0000","last_change":"2023-10-26T01:39:25.350493+0000","last_active":"2023-10-26T01:39:25.350624+0000","last_peered":"2023-10-26T01:39:25.350624+0000","last_clean":"2023-10-26T01:39:25.350624+0000","last_became_active":"2023-10-26T01:39:25.349758+0000","last_became_peered":"2023-10-26T01:39:25.349758+0000","last_unstale":"2023-10-26T01:39:25.350624+0000","last_undegraded":"2023-10-26T01:39:25.350624+0000","last_fullsized":"2023-10-26T01:39:25.350624+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:23:21.057813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.339995+0000","last_change":"2023-10-26T01:39:25.339995+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.339995+0000","last_undegraded":"2023-10-26T01:39:25.339995+0000","last_fullsized":"2023-10-26T01:39:25.339995+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-26T01:39:25.347943+0000","last_change":"2023-10-26T01:39:25.347830+0000","last_active":"2023-10-26T01:39:25.347943+0000","last_peered":"2023-10-26T01:39:25.347943+0000","last_clean":"2023-10-26T01:39:25.347943+0000","last_became_active":"2023-10-26T01:39:25.346994+0000","last_became_peered":"2023-10-26T01:39:25.346994+0000","last_unstale":"2023-10-26T01:39:25.347943+0000","last_undegraded":"2023-10-26T01:39:25.347943+0000","last_fullsized":"2023-10-26T01:39:25.347943+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:16:32.808798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.340256+0000","last_change":"2023-10-26T01:39:25.340256+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.340256+0000","last_undegraded":"2023-10-26T01:39:25.340256+0000","last_fullsized":"2023-10-26T01:39:25.340256+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-26T01:39:25.354467+0000","last_change":"2023-10-26T01:39:25.354096+0000","last_active":"2023-10-26T01:39:25.354467+0000","last_peered":"2023-10-26T01:39:25.354467+0000","last_clean":"2023-10-26T01:39:25.354467+0000","last_became_active":"2023-10-26T01:39:25.353857+0000","last_became_peered":"2023-10-26T01:39:25.353857+0000","last_unstale":"2023-10-26T01:39:25.354467+0000","last_undegraded":"2023-10-26T01:39:25.354467+0000","last_fullsized":"2023-10-26T01:39:25.354467+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:28:46.712484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.335457+0000","last_change":"2023-10-26T01:39:25.335457+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.335457+0000","last_undegraded":"2023-10-26T01:39:25.335457+0000","last_fullsized":"2023-10-26T01:39:25.335457+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-26T01:39:25.350001+0000","last_change":"2023-10-26T01:39:25.349514+0000","last_active":"2023-10-26T01:39:25.350001+0000","last_peered":"2023-10-26T01:39:25.350001+0000","last_clean":"2023-10-26T01:39:25.350001+0000","last_became_active":"2023-10-26T01:39:25.349279+0000","last_became_peered":"2023-10-26T01:39:25.349279+0000","last_unstale":"2023-10-26T01:39:25.350001+0000","last_undegraded":"2023-10-26T01:39:25.350001+0000","last_fullsized":"2023-10-26T01:39:25.350001+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:51:35.449470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.337282+0000","last_change":"2023-10-26T01:39:25.337282+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.337282+0000","last_undegraded":"2023-10-26T01:39:25.337282+0000","last_fullsized":"2023-10-26T01:39:25.337282+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-26T01:39:25.348101+0000","last_change":"2023-10-26T01:39:25.347759+0000","last_active":"2023-10-26T01:39:25.348101+0000","last_peered":"2023-10-26T01:39:25.348101+0000","last_clean":"2023-10-26T01:39:25.348101+0000","last_became_active":"2023-10-26T01:39:25.346564+0000","last_became_peered":"2023-10-26T01:39:25.346564+0000","last_unstale":"2023-10-26T01:39:25.348101+0000","last_undegraded":"2023-10-26T01:39:25.348101+0000","last_fullsized":"2023-10-26T01:39:25.348101+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:12:27.843055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.343816+0000","last_change":"2023-10-26T01:39:25.343816+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.343816+0000","last_undegraded":"2023-10-26T01:39:25.343816+0000","last_fullsized":"2023-10-26T01:39:25.343816+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-26T01:39:25.344486+0000","last_change":"2023-10-26T01:39:25.343413+0000","last_active":"2023-10-26T01:39:25.344486+0000","last_peered":"2023-10-26T01:39:25.344486+0000","last_clean":"2023-10-26T01:39:25.344486+0000","last_became_active":"2023-10-26T01:39:25.342949+0000","last_became_peered":"2023-10-26T01:39:25.342949+0000","last_unstale":"2023-10-26T01:39:25.344486+0000","last_undegraded":"2023-10-26T01:39:25.344486+0000","last_fullsized":"2023-10-26T01:39:25.344486+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:11:10.758066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":9,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-26T01:39:26.336802+0000","last_change":"2023-10-26T01:39:26.335564+0000","last_active":"2023-10-26T01:39:26.336802+0000","last_peered":"2023-10-26T01:39:26.336802+0000","last_clean":"2023-10-26T01:39:26.336802+0000","last_became_active":"2023-10-26T01:39:26.335000+0000","last_became_peered":"2023-10-26T01:39:26.335000+0000","last_unstale":"2023-10-26T01:39:26.336802+0000","last_undegraded":"2023-10-26T01:39:26.336802+0000","last_fullsized":"2023-10-26T01:39:26.336802+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:05:15.173045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":11,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-26T01:39:26.338413+0000","last_change":"2023-10-26T01:39:25.356190+0000","last_active":"2023-10-26T01:39:26.338413+0000","last_peered":"2023-10-26T01:39:26.338413+0000","last_clean":"2023-10-26T01:39:26.338413+0000","last_became_active":"2023-10-26T01:39:25.355791+0000","last_became_peered":"2023-10-26T01:39:25.355791+0000","last_unstale":"2023-10-26T01:39:26.338413+0000","last_undegraded":"2023-10-26T01:39:26.338413+0000","last_fullsized":"2023-10-26T01:39:26.338413+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:24:21.026112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.339461+0000","last_change":"2023-10-26T01:39:25.339461+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.339461+0000","last_undegraded":"2023-10-26T01:39:25.339461+0000","last_fullsized":"2023-10-26T01:39:25.339461+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-26T01:39:25.349964+0000","last_change":"2023-10-26T01:39:25.347950+0000","last_active":"2023-10-26T01:39:25.349964+0000","last_peered":"2023-10-26T01:39:25.349964+0000","last_clean":"2023-10-26T01:39:25.349964+0000","last_became_active":"2023-10-26T01:39:25.347416+0000","last_became_peered":"2023-10-26T01:39:25.347416+0000","last_unstale":"2023-10-26T01:39:25.349964+0000","last_undegraded":"2023-10-26T01:39:25.349964+0000","last_fullsized":"2023-10-26T01:39:25.349964+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:22:18.250771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.334115+0000","last_change":"2023-10-26T01:39:25.334115+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.334115+0000","last_undegraded":"2023-10-26T01:39:25.334115+0000","last_fullsized":"2023-10-26T01:39:25.334115+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-26T01:39:25.353367+0000","last_change":"2023-10-26T01:39:25.352322+0000","last_active":"2023-10-26T01:39:25.353367+0000","last_peered":"2023-10-26T01:39:25.353367+0000","last_clean":"2023-10-26T01:39:25.353367+0000","last_became_active":"2023-10-26T01:39:25.351287+0000","last_became_peered":"2023-10-26T01:39:25.351287+0000","last_unstale":"2023-10-26T01:39:25.353367+0000","last_undegraded":"2023-10-26T01:39:25.353367+0000","last_fullsized":"2023-10-26T01:39:25.353367+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:03:57.007851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":292,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-26T01:39:24.316788+0000","last_change":"2023-10-26T01:30:34.883484+0000","last_active":"2023-10-26T01:39:24.316788+0000","last_peered":"2023-10-26T01:39:24.316788+0000","last_clean":"2023-10-26T01:39:24.316788+0000","last_became_active":"2023-10-26T01:30:34.883071+0000","last_became_peered":"2023-10-26T01:30:34.883071+0000","last_unstale":"2023-10-26T01:39:24.316788+0000","last_undegraded":"2023-10-26T01:39:24.316788+0000","last_fullsized":"2023-10-26T01:39:24.316788+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:32.890096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:32.890096+0000","last_clean_scrub_stamp":"2023-10-26T01:30:32.890096+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:26:31.879452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":14,"seq":60129542254,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":817,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194495,"statfs":{"total":200000000,"available":199163256,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:38:51 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.443,"5min":0.35599999999999998,"15min":0.33000000000000002},"max":{"1min":1.2549999999999999,"5min":1.6319999999999999,"15min":2.2549999999999999},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.92300000000000004,"15min":0.89300000000000002},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":1.752,"5min":1.752,"15min":1.8200000000000001},"last":0.67800000000000005}]},{"osd":1,"last update":"Thu Oct 26 01:38:51 2023","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.88400000000000001,"15min":0.92800000000000005},"min":{"1min":0.39400000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.383,"5min":1.444,"15min":4.4340000000000002},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.996,"5min":0.99199999999999999,"15min":0.995},"min":{"1min":0.56200000000000006,"5min":0.46700000000000003,"15min":0.374},"max":{"1min":1.353,"5min":1.538,"15min":2.3330000000000002},"last":0.70999999999999996}]},{"osd":2,"last update":"Thu Oct 26 01:38:51 2023","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":1.028,"15min":1.0700000000000001},"min":{"1min":0.54500000000000004,"5min":0.40999999999999998,"15min":0.39900000000000002},"max":{"1min":1.4279999999999999,"5min":1.927,"15min":4.2800000000000002},"last":0.79400000000000004},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.1479999999999999,"15min":1.0980000000000001},"min":{"1min":0.53300000000000003,"5min":0.495,"15min":0.49399999999999999},"max":{"1min":1.639,"5min":1.8680000000000001,"15min":2.302},"last":0.94399999999999995}]},{"osd":4,"last update":"Thu Oct 26 01:38:51 2023","interfaces":[{"interface":"back","average":{"1min":1.266,"5min":1.23,"15min":1.175},"min":{"1min":0.98199999999999998,"5min":0.755,"15min":0.66400000000000003},"max":{"1min":1.732,"5min":1.732,"15min":2.1070000000000002},"last":1.375},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.353,"15min":1.3859999999999999},"min":{"1min":0.91900000000000004,"5min":0.81599999999999995,"15min":0.79900000000000004},"max":{"1min":1.6220000000000001,"5min":2.4239999999999999,"15min":4.8250000000000002},"last":1.4019999999999999}]},{"osd":5,"last update":"Thu Oct 26 01:38:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.2929999999999999,"15min":1.3280000000000001},"min":{"1min":0.84899999999999998,"5min":0.76400000000000001,"15min":0.72499999999999998},"max":{"1min":1.6899999999999999,"5min":2.3380000000000001,"15min":4.9080000000000004},"last":1.2709999999999999},{"interface":"front","average":{"1min":1.331,"5min":1.2969999999999999,"15min":1.2789999999999999},"min":{"1min":0.82699999999999996,"5min":0.79900000000000004,"15min":0.79600000000000004},"max":{"1min":1.573,"5min":2.0310000000000001,"15min":2.8980000000000001},"last":1.385}]},{"osd":6,"last update":"Thu Oct 26 01:38:51 2023","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.351,"15min":1.329},"min":{"1min":0.75900000000000001,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.694,"5min":2.1499999999999999,"15min":2.8530000000000002},"last":1.448},{"interface":"front","average":{"1min":1.4159999999999999,"5min":1.3160000000000001,"15min":1.2769999999999999},"min":{"1min":0.84399999999999997,"5min":0.81699999999999995,"15min":0.73099999999999998},"max":{"1min":1.6899999999999999,"5min":2.048,"15min":2.181},"last":1.48}]},{"osd":7,"last update":"Thu Oct 26 01:38:51 2023","interfaces":[{"interface":"back","average":{"1min":1.4370000000000001,"5min":1.3660000000000001,"15min":1.3069999999999999},"min":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.71999999999999997},"max":{"1min":1.7589999999999999,"5min":1.958,"15min":2.0680000000000001},"last":1.3340000000000001},{"interface":"front","average":{"1min":1.385,"5min":1.393,"15min":1.411},"min":{"1min":0.73299999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.702,"5min":2.2370000000000001,"15min":4.8730000000000002},"last":1.595}]}]},{"osd":7,"up_from":13,"seq":55834574957,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1380,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193931,"statfs":{"total":200000000,"available":198586098,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:38:50 2023","interfaces":[{"interface":"back","average":{"1min":1.127,"5min":1.1379999999999999,"15min":1.097},"min":{"1min":0.68700000000000006,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.633,"5min":1.7,"15min":1.9410000000000001},"last":1.1879999999999999},{"interface":"front","average":{"1min":1.095,"5min":1.1100000000000001,"15min":1.0840000000000001},"min":{"1min":0.67300000000000004,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.6970000000000001,"5min":2.6779999999999999,"15min":2.6779999999999999},"last":1.294}]},{"osd":1,"last update":"Thu Oct 26 01:38:50 2023","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":1.1699999999999999,"15min":1.1020000000000001},"min":{"1min":0.82899999999999996,"5min":0.65800000000000003,"15min":0.65200000000000002},"max":{"1min":1.484,"5min":2.8260000000000001,"15min":2.8260000000000001},"last":0.879},{"interface":"front","average":{"1min":1.177,"5min":1.2,"15min":1.194},"min":{"1min":0.77200000000000002,"5min":0.66500000000000004,"15min":0.61199999999999999},"max":{"1min":1.76,"5min":3.2730000000000001,"15min":3.2730000000000001},"last":1.1240000000000001}]},{"osd":2,"last update":"Thu Oct 26 01:38:50 2023","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.2110000000000001,"15min":1.1299999999999999},"min":{"1min":0.873,"5min":0.69099999999999995,"15min":0.58499999999999996},"max":{"1min":1.851,"5min":2.218,"15min":2.218},"last":0.93400000000000005},{"interface":"front","average":{"1min":1.256,"5min":1.296,"15min":1.2430000000000001},"min":{"1min":0.82299999999999995,"5min":0.79400000000000004,"15min":0.73099999999999998},"max":{"1min":1.7350000000000001,"5min":3.0510000000000002,"15min":3.0510000000000002},"last":1.0229999999999999}]},{"osd":3,"last update":"Thu Oct 26 01:38:50 2023","interfaces":[{"interface":"back","average":{"1min":1.3120000000000001,"5min":1.296,"15min":1.2230000000000001},"min":{"1min":0.79500000000000004,"5min":0.79000000000000004,"15min":0.72099999999999997},"max":{"1min":1.8169999999999999,"5min":1.9830000000000001,"15min":1.9830000000000001},"last":0.79800000000000004},{"interface":"front","average":{"1min":1.224,"5min":1.373,"15min":1.3640000000000001},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.65700000000000003},"max":{"1min":1.5129999999999999,"5min":5.5039999999999996,"15min":5.5039999999999996},"last":1.651}]},{"osd":4,"last update":"Thu Oct 26 01:38:50 2023","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.371,"15min":1.351},"min":{"1min":0.81799999999999995,"5min":0.77300000000000002,"15min":0.76100000000000001},"max":{"1min":1.9339999999999999,"5min":2.77,"15min":2.77},"last":2.0390000000000001},{"interface":"front","average":{"1min":1.288,"5min":1.3919999999999999,"15min":1.393},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.6479999999999999,"5min":3.1419999999999999,"15min":3.1419999999999999},"last":2.536}]},{"osd":5,"last update":"Thu Oct 26 01:38:50 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.4410000000000001,"15min":1.4199999999999999},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.7809999999999999,"5min":5.734,"15min":5.734},"last":1.363},{"interface":"front","average":{"1min":1.4450000000000001,"5min":1.514,"15min":1.478},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.51900000000000002},"max":{"1min":2,"5min":2.8980000000000001,"15min":2.8980000000000001},"last":1.0920000000000001}]},{"osd":6,"last update":"Thu Oct 26 01:38:50 2023","interfaces":[{"interface":"back","average":{"1min":1.452,"5min":1.5800000000000001,"15min":1.8089999999999999},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.47399999999999998},"max":{"1min":1.8049999999999999,"5min":5.6340000000000003,"15min":11.577999999999999},"last":1.7769999999999999},{"interface":"front","average":{"1min":1.613,"5min":1.6719999999999999,"15min":1.8360000000000001},"min":{"1min":1.0640000000000001,"5min":0.91800000000000004,"15min":0.55100000000000005},"max":{"1min":2.0529999999999999,"5min":5.0919999999999996,"15min":10.704000000000001},"last":1.8779999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574957,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":803,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194508,"statfs":{"total":200000000,"available":199177080,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:38:46 2023","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.141,"15min":1.1519999999999999},"min":{"1min":0.66600000000000004,"5min":0.61699999999999999,"15min":0.60399999999999998},"max":{"1min":2.7370000000000001,"5min":2.7370000000000001,"15min":2.7370000000000001},"last":1.0760000000000001},{"interface":"front","average":{"1min":1.137,"5min":1.173,"15min":1.123},"min":{"1min":0.61699999999999999,"5min":0.61399999999999999,"15min":0.59199999999999997},"max":{"1min":4.4779999999999998,"5min":4.4779999999999998,"15min":4.4779999999999998},"last":1.0189999999999999}]},{"osd":1,"last update":"Thu Oct 26 01:38:46 2023","interfaces":[{"interface":"back","average":{"1min":1.3919999999999999,"5min":1.2729999999999999,"15min":1.1930000000000001},"min":{"1min":0.77100000000000002,"5min":0.72299999999999998,"15min":0.68300000000000005},"max":{"1min":5.5359999999999996,"5min":5.5359999999999996,"15min":5.5359999999999996},"last":1.246},{"interface":"front","average":{"1min":1.228,"5min":1.2190000000000001,"15min":1.204},"min":{"1min":0.74399999999999999,"5min":0.67900000000000005,"15min":0.64000000000000001},"max":{"1min":4.3579999999999997,"5min":4.3579999999999997,"15min":4.3579999999999997},"last":1.1279999999999999}]},{"osd":2,"last update":"Thu Oct 26 01:38:46 2023","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.2030000000000001,"15min":1.107},"min":{"1min":0.76300000000000001,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":2.367,"5min":3.7879999999999998,"15min":3.7879999999999998},"last":1.175},{"interface":"front","average":{"1min":1.371,"5min":1.3420000000000001,"15min":1.276},"min":{"1min":0.91200000000000003,"5min":0.75800000000000001,"15min":0.68899999999999995},"max":{"1min":3.0470000000000002,"5min":3.0470000000000002,"15min":3.0470000000000002},"last":1.4430000000000001}]},{"osd":3,"last update":"Thu Oct 26 01:38:46 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.331,"15min":1.22},"min":{"1min":0.76300000000000001,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":3.266,"5min":4.4560000000000004,"15min":4.4560000000000004},"last":1.601},{"interface":"front","average":{"1min":1.365,"5min":1.3500000000000001,"15min":1.3280000000000001},"min":{"1min":0.86399999999999999,"5min":0.78600000000000003,"15min":0.69499999999999995},"max":{"1min":2.6909999999999998,"5min":2.6909999999999998,"15min":2.6909999999999998},"last":1.55}]},{"osd":4,"last update":"Thu Oct 26 01:38:46 2023","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.24,"15min":1.22},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.48399999999999999},"max":{"1min":1.7410000000000001,"5min":2.9489999999999998,"15min":2.9489999999999998},"last":1.395},{"interface":"front","average":{"1min":1.3180000000000001,"5min":1.421,"15min":1.319},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.59299999999999997},"max":{"1min":1.698,"5min":6.609,"15min":6.609},"last":1.494}]},{"osd":5,"last update":"Thu Oct 26 01:38:46 2023","interfaces":[{"interface":"back","average":{"1min":1.464,"5min":1.5389999999999999,"15min":1.452},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.7949999999999999,"5min":5.4370000000000003,"15min":5.4370000000000003},"last":1.671},{"interface":"front","average":{"1min":1.492,"5min":1.4530000000000001,"15min":1.419},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":2.7759999999999998,"5min":2.7759999999999998,"15min":2.7759999999999998},"last":1.6459999999999999}]},{"osd":7,"last update":"Thu Oct 26 01:38:46 2023","interfaces":[{"interface":"back","average":{"1min":1.2869999999999999,"5min":1.484,"15min":1.405},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.54100000000000004},"max":{"1min":1.8200000000000001,"5min":11.804,"15min":11.804},"last":1.698},{"interface":"front","average":{"1min":1.577,"5min":1.6639999999999999,"15min":1.52},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":3.7650000000000001,"5min":11.541,"15min":11.541},"last":1.728}]}]},{"osd":5,"up_from":13,"seq":55834574957,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":810,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194502,"statfs":{"total":200000000,"available":199170191,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:38:48 2023","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.2310000000000001,"15min":1.147},"min":{"1min":0.69699999999999995,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":3.8210000000000002,"5min":3.8210000000000002,"15min":3.8210000000000002},"last":1.5549999999999999},{"interface":"front","average":{"1min":1.3480000000000001,"5min":1.302,"15min":1.1970000000000001},"min":{"1min":0.92600000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":2.9889999999999999,"5min":2.9889999999999999,"15min":2.9889999999999999},"last":1.48}]},{"osd":1,"last update":"Thu Oct 26 01:38:48 2023","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.1319999999999999,"15min":1.0940000000000001},"min":{"1min":0.83599999999999997,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.8120000000000001,"5min":1.8120000000000001,"15min":1.8120000000000001},"last":1.514},{"interface":"front","average":{"1min":1.272,"5min":1.1659999999999999,"15min":1.109},"min":{"1min":0.76700000000000002,"5min":0.629,"15min":0.60599999999999998},"max":{"1min":2.4340000000000002,"5min":2.4340000000000002,"15min":2.4340000000000002},"last":0.98199999999999998}]},{"osd":2,"last update":"Thu Oct 26 01:38:48 2023","interfaces":[{"interface":"back","average":{"1min":1.2130000000000001,"5min":1.206,"15min":1.1679999999999999},"min":{"1min":0.79700000000000004,"5min":0.71599999999999997,"15min":0.69299999999999995},"max":{"1min":1.5920000000000001,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":1.2270000000000001},{"interface":"front","average":{"1min":1.228,"5min":1.228,"15min":1.1850000000000001},"min":{"1min":0.91600000000000004,"5min":0.78400000000000003,"15min":0.63900000000000001},"max":{"1min":1.5900000000000001,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":1.6459999999999999}]},{"osd":3,"last update":"Thu Oct 26 01:38:48 2023","interfaces":[{"interface":"back","average":{"1min":1.389,"5min":1.2829999999999999,"15min":1.25},"min":{"1min":1.0009999999999999,"5min":0.72699999999999998,"15min":0.72299999999999998},"max":{"1min":1.7,"5min":1.7,"15min":1.835},"last":1.101},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.298,"15min":1.29},"min":{"1min":0.91300000000000003,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.52,"5min":1.788,"15min":2.9249999999999998},"last":0.74299999999999999}]},{"osd":4,"last update":"Thu Oct 26 01:38:48 2023","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":1.256,"15min":1.214},"min":{"1min":0.75,"5min":0.59599999999999997,"15min":0.49099999999999999},"max":{"1min":2.7770000000000001,"5min":2.7770000000000001,"15min":2.7770000000000001},"last":1.681},{"interface":"front","average":{"1min":1.3180000000000001,"5min":1.3979999999999999,"15min":1.355},"min":{"1min":0.67500000000000004,"5min":0.57299999999999995,"15min":0.56200000000000006},"max":{"1min":1.776,"5min":1.7849999999999999,"15min":1.867},"last":0.86699999999999999}]},{"osd":6,"last update":"Thu Oct 26 01:38:48 2023","interfaces":[{"interface":"back","average":{"1min":1.377,"5min":1.393,"15min":1.3300000000000001},"min":{"1min":0.79000000000000004,"5min":0.67500000000000004,"15min":0.51900000000000002},"max":{"1min":1.956,"5min":2.8250000000000002,"15min":2.8250000000000002},"last":1.4339999999999999},{"interface":"front","average":{"1min":1.4590000000000001,"5min":1.409,"15min":1.3400000000000001},"min":{"1min":0.89300000000000002,"5min":0.69499999999999995,"15min":0.61799999999999999},"max":{"1min":2.4830000000000001,"5min":2.4830000000000001,"15min":2.4830000000000001},"last":1.339}]},{"osd":7,"last update":"Thu Oct 26 01:38:48 2023","interfaces":[{"interface":"back","average":{"1min":1.6000000000000001,"5min":1.534,"15min":1.4379999999999999},"min":{"1min":0.81499999999999995,"5min":0.71099999999999997,"15min":0.61699999999999999},"max":{"1min":3.621,"5min":3.621,"15min":3.621},"last":1.609},{"interface":"front","average":{"1min":1.603,"5min":1.542,"15min":1.4470000000000001},"min":{"1min":0.82899999999999996,"5min":0.73899999999999999,"15min":0.59499999999999997},"max":{"1min":2.8010000000000002,"5min":2.8010000000000002,"15min":2.8010000000000002},"last":1.7889999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574957,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":804,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194508,"statfs":{"total":200000000,"available":199176466,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:38:43 2023","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.071,"15min":0.999},"min":{"1min":0.63500000000000001,"5min":0.59799999999999998,"15min":0.53100000000000003},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":2.0289999999999999},"last":1.7689999999999999},{"interface":"front","average":{"1min":1.123,"5min":1.077,"15min":1.0329999999999999},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.60599999999999998},"max":{"1min":1.784,"5min":1.784,"15min":2.6829999999999998},"last":1.921}]},{"osd":1,"last update":"Thu Oct 26 01:38:43 2023","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":1.161,"15min":1.1259999999999999},"min":{"1min":0.749,"5min":0.66700000000000004,"15min":0.56799999999999995},"max":{"1min":1.579,"5min":1.599,"15min":4.0519999999999996},"last":2.056},{"interface":"front","average":{"1min":1.2529999999999999,"5min":1.1779999999999999,"15min":1.121},"min":{"1min":0.70799999999999996,"5min":0.70399999999999996,"15min":0.60499999999999998},"max":{"1min":2.3919999999999999,"5min":2.3919999999999999,"15min":2.3919999999999999},"last":1.8919999999999999}]},{"osd":2,"last update":"Thu Oct 26 01:38:43 2023","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":1.304,"15min":1.2430000000000001},"min":{"1min":1.0429999999999999,"5min":0.66700000000000004,"15min":0.64300000000000002},"max":{"1min":1.6759999999999999,"5min":1.845,"15min":5.6749999999999998},"last":2.2040000000000002},{"interface":"front","average":{"1min":1.1799999999999999,"5min":1.218,"15min":1.1419999999999999},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.631,"5min":3.673,"15min":3.673},"last":1.2210000000000001}]},{"osd":3,"last update":"Thu Oct 26 01:38:43 2023","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.228,"15min":1.1919999999999999},"min":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.70399999999999996},"max":{"1min":1.7030000000000001,"5min":1.847,"15min":1.891},"last":1.9890000000000001},{"interface":"front","average":{"1min":1.1799999999999999,"5min":1.204,"15min":1.1619999999999999},"min":{"1min":0.86099999999999999,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.522,"5min":1.681,"15min":1.7210000000000001},"last":1.353}]},{"osd":5,"last update":"Thu Oct 26 01:38:43 2023","interfaces":[{"interface":"back","average":{"1min":1.236,"5min":1.258,"15min":1.2809999999999999},"min":{"1min":0.85699999999999998,"5min":0.52100000000000002,"15min":0.46300000000000002},"max":{"1min":1.681,"5min":2.1429999999999998,"15min":2.1429999999999998},"last":2.2469999999999999},{"interface":"front","average":{"1min":1.2270000000000001,"5min":1.2809999999999999,"15min":1.321},"min":{"1min":0.84899999999999998,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.754,"5min":1.9810000000000001,"15min":5.8579999999999997},"last":2.2850000000000001}]},{"osd":6,"last update":"Thu Oct 26 01:38:43 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.294,"15min":1.335},"min":{"1min":0.752,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.708,"5min":2.2149999999999999,"15min":12.343999999999999},"last":1.4419999999999999},{"interface":"front","average":{"1min":1.5129999999999999,"5min":1.5109999999999999,"15min":1.5189999999999999},"min":{"1min":1.0660000000000001,"5min":0.57999999999999996,"15min":0.55000000000000004},"max":{"1min":1.762,"5min":2.117,"15min":12.605},"last":2.1040000000000001}]},{"osd":7,"last update":"Thu Oct 26 01:38:43 2023","interfaces":[{"interface":"back","average":{"1min":1.4690000000000001,"5min":1.4670000000000001,"15min":1.49},"min":{"1min":1.1279999999999999,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.8200000000000001,"5min":2.762,"15min":5.774},"last":2.3330000000000002},{"interface":"front","average":{"1min":1.399,"5min":1.484,"15min":1.4590000000000001},"min":{"1min":1.008,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.7589999999999999,"5min":2.8330000000000002,"15min":4.6520000000000001},"last":1.5149999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574957,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":810,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194502,"statfs":{"total":200000000,"available":199170191,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:38:47 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.84499999999999997,"15min":0.85299999999999998},"min":{"1min":0.46600000000000003,"5min":0.41199999999999998,"15min":0.308},"max":{"1min":1.4079999999999999,"5min":1.5249999999999999,"15min":6.3200000000000003},"last":0.95999999999999996},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.97699999999999998,"15min":0.93300000000000005},"min":{"1min":0.44,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.476,"5min":1.9970000000000001,"15min":4.2619999999999996},"last":1.0129999999999999}]},{"osd":1,"last update":"Thu Oct 26 01:38:47 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.878,"15min":0.89100000000000001},"min":{"1min":0.48399999999999999,"5min":0.39300000000000002,"15min":0.34000000000000002},"max":{"1min":1.2829999999999999,"5min":1.629,"15min":5.1260000000000003},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.98499999999999999,"5min":1.0189999999999999,"15min":1.0029999999999999},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.36599999999999999},"max":{"1min":1.4870000000000001,"5min":1.873,"15min":6.1520000000000001},"last":1.286}]},{"osd":3,"last update":"Thu Oct 26 01:38:52 2023","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":1.004,"15min":0.94499999999999995},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.407,"5min":2.0640000000000001,"15min":2.0640000000000001},"last":1.3500000000000001},{"interface":"front","average":{"1min":1.0349999999999999,"5min":1.0860000000000001,"15min":1.0249999999999999},"min":{"1min":0.53800000000000003,"5min":0.51100000000000001,"15min":0.45300000000000001},"max":{"1min":1.665,"5min":1.9279999999999999,"15min":8.9019999999999992},"last":1.093}]},{"osd":4,"last update":"Thu Oct 26 01:38:47 2023","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.3280000000000001,"15min":1.2689999999999999},"min":{"1min":0.81599999999999995,"5min":0.80500000000000005,"15min":0.73899999999999999},"max":{"1min":1.6080000000000001,"5min":2.6190000000000002,"15min":6.2670000000000003},"last":1.4850000000000001},{"interface":"front","average":{"1min":1.2709999999999999,"5min":1.2969999999999999,"15min":1.266},"min":{"1min":0.81699999999999995,"5min":0.81100000000000005,"15min":0.78700000000000003},"max":{"1min":1.8380000000000001,"5min":2.4359999999999999,"15min":7.343},"last":1.1950000000000001}]},{"osd":5,"last update":"Thu Oct 26 01:38:47 2023","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":1.3500000000000001,"15min":1.29},"min":{"1min":0.85099999999999998,"5min":0.81699999999999995,"15min":0.71899999999999997},"max":{"1min":1.669,"5min":2.5790000000000002,"15min":3.7170000000000001},"last":1.55},{"interface":"front","average":{"1min":1.302,"5min":1.3120000000000001,"15min":1.2929999999999999},"min":{"1min":0.872,"5min":0.81899999999999995,"15min":0.65300000000000002},"max":{"1min":1.708,"5min":2.298,"15min":6.3899999999999997},"last":1.5780000000000001}]},{"osd":6,"last update":"Thu Oct 26 01:38:47 2023","interfaces":[{"interface":"back","average":{"1min":1.3280000000000001,"5min":1.3919999999999999,"15min":1.3380000000000001},"min":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.70599999999999996},"max":{"1min":1.714,"5min":2.73,"15min":6.0490000000000004},"last":1.643},{"interface":"front","average":{"1min":1.379,"5min":1.4139999999999999,"15min":1.3740000000000001},"min":{"1min":0.89200000000000002,"5min":0.78200000000000003,"15min":0.76800000000000002},"max":{"1min":2.0990000000000002,"5min":2.4089999999999998,"15min":5.0650000000000004},"last":1.4399999999999999}]},{"osd":7,"last update":"Thu Oct 26 01:38:47 2023","interfaces":[{"interface":"back","average":{"1min":1.373,"5min":1.3979999999999999,"15min":1.3939999999999999},"min":{"1min":0.79200000000000004,"5min":0.77900000000000003,"15min":0.69399999999999995},"max":{"1min":1.776,"5min":2.3370000000000002,"15min":6.4480000000000004},"last":1.7949999999999999},{"interface":"front","average":{"1min":1.4830000000000001,"5min":1.5169999999999999,"15min":1.458},"min":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.91800000000000004},"max":{"1min":2.1480000000000001,"5min":2.8290000000000002,"15min":2.8439999999999999},"last":1.734}]}]},{"osd":1,"up_from":13,"seq":55834574957,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":810,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194502,"statfs":{"total":200000000,"available":199170191,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:38:49 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.76900000000000002,"15min":0.72799999999999998},"min":{"1min":0.41799999999999998,"5min":0.34000000000000002,"15min":0.29499999999999998},"max":{"1min":0.92700000000000005,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.87,"15min":0.80200000000000005},"min":{"1min":0.52000000000000002,"5min":0.36299999999999999,"15min":0.36199999999999999},"max":{"1min":1.2769999999999999,"5min":1.6399999999999999,"15min":1.6399999999999999},"last":0.71699999999999997}]},{"osd":2,"last update":"Thu Oct 26 01:38:49 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.995,"15min":0.999},"min":{"1min":0.65300000000000002,"5min":0.39700000000000002,"15min":0.39600000000000002},"max":{"1min":1.286,"5min":1.8200000000000001,"15min":2.4830000000000001},"last":0.874},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.96199999999999997,"15min":0.88400000000000001},"min":{"1min":0.48099999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.569,"5min":1.8080000000000001,"15min":1.8080000000000001},"last":1.1339999999999999}]},{"osd":3,"last update":"Thu Oct 26 01:38:53 2023","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":0.996,"15min":0.95499999999999996},"min":{"1min":0.59999999999999998,"5min":0.42499999999999999,"15min":0.39500000000000002},"max":{"1min":1.5900000000000001,"5min":1.611,"15min":1.75},"last":1.292},{"interface":"front","average":{"1min":0.998,"5min":1.028,"15min":0.99199999999999999},"min":{"1min":0.55000000000000004,"5min":0.47799999999999998,"15min":0.41299999999999998},"max":{"1min":1.5049999999999999,"5min":1.673,"15min":1.802},"last":0.69799999999999995}]},{"osd":4,"last update":"Thu Oct 26 01:38:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":1.222,"15min":1.1990000000000001},"min":{"1min":0.90400000000000003,"5min":0.69099999999999995,"15min":0.56200000000000006},"max":{"1min":1.5249999999999999,"5min":1.756,"15min":2.0659999999999998},"last":1.024},{"interface":"front","average":{"1min":1.1659999999999999,"5min":1.1870000000000001,"15min":1.1399999999999999},"min":{"1min":0.81299999999999994,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.4390000000000001,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":1.0780000000000001}]},{"osd":5,"last update":"Thu Oct 26 01:38:49 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.2629999999999999,"15min":1.2290000000000001},"min":{"1min":0.84799999999999998,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.621,"5min":1.8779999999999999,"15min":1.8779999999999999},"last":1.1679999999999999},{"interface":"front","average":{"1min":1.359,"5min":1.375,"15min":1.339},"min":{"1min":1.052,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.7210000000000001,"5min":2.04,"15min":2.2200000000000002},"last":1.056}]},{"osd":6,"last update":"Thu Oct 26 01:38:49 2023","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.347,"15min":1.2849999999999999},"min":{"1min":0.90400000000000003,"5min":0.70199999999999996,"15min":0.68899999999999995},"max":{"1min":1.7709999999999999,"5min":2.0910000000000002,"15min":2.3780000000000001},"last":0.83399999999999996},{"interface":"front","average":{"1min":1.2889999999999999,"5min":1.2789999999999999,"15min":1.2490000000000001},"min":{"1min":0.92700000000000005,"5min":0.80700000000000005,"15min":0.70799999999999996},"max":{"1min":1.653,"5min":1.956,"15min":1.984},"last":1.22}]},{"osd":7,"last update":"Thu Oct 26 01:38:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.2749999999999999,"15min":1.262},"min":{"1min":0.79400000000000004,"5min":0.76100000000000001,"15min":0.69699999999999995},"max":{"1min":1.764,"5min":1.8380000000000001,"15min":2.6930000000000001},"last":1.256},{"interface":"front","average":{"1min":1.431,"5min":1.419,"15min":1.345},"min":{"1min":0.95499999999999996,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.835,"5min":2.0190000000000001,"15min":2.0190000000000001},"last":0.98699999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574957,"num_pgs":6,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1386,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193925,"statfs":{"total":200000000,"available":198579823,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Oct 26 01:38:46 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.79700000000000004,"15min":0.91100000000000003},"min":{"1min":0.47799999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.159,"5min":1.5049999999999999,"15min":19.425000000000001},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.93999999999999995,"15min":1.0369999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.462,"5min":1.462,"15min":15.162000000000001},"last":0.748}]},{"osd":2,"last update":"Thu Oct 26 01:38:46 2023","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":1.0149999999999999,"15min":1.175},"min":{"1min":0.64300000000000002,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.47,"5min":1.577,"15min":28.439},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.90000000000000002,"15min":0.91300000000000003},"min":{"1min":0.57299999999999995,"5min":0.37,"15min":0.37},"max":{"1min":1.2969999999999999,"5min":1.4810000000000001,"15min":4.9459999999999997},"last":1.0980000000000001}]},{"osd":3,"last update":"Thu Oct 26 01:38:46 2023","interfaces":[{"interface":"back","average":{"1min":1.1180000000000001,"5min":1.0640000000000001,"15min":1.085},"min":{"1min":0.66800000000000004,"5min":0.46000000000000002,"15min":0.44600000000000001},"max":{"1min":1.6539999999999999,"5min":1.6539999999999999,"15min":4.9800000000000004},"last":1.5549999999999999},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.123,"15min":1.1499999999999999},"min":{"1min":0.66000000000000003,"5min":0.438,"15min":0.438},"max":{"1min":1.4279999999999999,"5min":1.6559999999999999,"15min":5.016},"last":1.5069999999999999}]},{"osd":4,"last update":"Thu Oct 26 01:38:46 2023","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":1.169,"15min":1.2789999999999999},"min":{"1min":0.94499999999999995,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.391,"5min":1.681,"15min":19.329000000000001},"last":1.026},{"interface":"front","average":{"1min":1.23,"5min":1.262,"15min":1.292},"min":{"1min":0.86799999999999999,"5min":0.82099999999999995,"15min":0.67600000000000005},"max":{"1min":1.5149999999999999,"5min":1.718,"15min":5.5229999999999997},"last":1.208}]},{"osd":5,"last update":"Thu Oct 26 01:38:46 2023","interfaces":[{"interface":"back","average":{"1min":1.427,"5min":1.4019999999999999,"15min":1.4279999999999999},"min":{"1min":1.03,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.819,"5min":1.857,"15min":5.6360000000000001},"last":1.609},{"interface":"front","average":{"1min":1.2909999999999999,"5min":1.2749999999999999,"15min":1.395},"min":{"1min":0.80200000000000005,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.5880000000000001,"5min":1.8200000000000001,"15min":19.384},"last":1.1539999999999999}]},{"osd":6,"last update":"Thu Oct 26 01:38:46 2023","interfaces":[{"interface":"back","average":{"1min":1.306,"5min":1.296,"15min":1.3600000000000001},"min":{"1min":0.79300000000000004,"5min":0.76700000000000002,"15min":0.73799999999999999},"max":{"1min":1.736,"5min":1.736,"15min":8.2609999999999992},"last":1.3380000000000001},{"interface":"front","average":{"1min":1.3,"5min":1.335,"15min":1.3999999999999999},"min":{"1min":0.82999999999999996,"5min":0.77900000000000003,"15min":0.75900000000000001},"max":{"1min":1.581,"5min":1.9450000000000001,"15min":11.849},"last":1.373}]},{"osd":7,"last update":"Thu Oct 26 01:38:46 2023","interfaces":[{"interface":"back","average":{"1min":1.46,"5min":1.4319999999999999,"15min":1.4770000000000001},"min":{"1min":1.089,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.9239999999999999,"5min":1.9239999999999999,"15min":5.6689999999999996},"last":1.722},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.397,"15min":1.4930000000000001},"min":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.69399999999999995},"max":{"1min":1.661,"5min":1.9390000000000001,"15min":19.245999999999999},"last":1.4279999999999999}]}]}],"pool_statfs":[]}} 2023-10-26T01:39:30.757 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-26T01:39:31.015 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:31.012+0000 7f11bd887640 1 -- 172.21.15.169:0/3099882515 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11b80ac930 msgr2=0x7f11b80cd6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:31.015 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:31.012+0000 7f11bd887640 1 --2- 172.21.15.169:0/3099882515 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11b80ac930 0x7f11b80cd6f0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f11ac009830 tx=0x7f11ac02bd70 comp rx=0 tx=0).stop 2023-10-26T01:39:31.015 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:31.012+0000 7f11bd887640 1 -- 172.21.15.169:0/3099882515 shutdown_connections 2023-10-26T01:39:31.016 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:31.012+0000 7f11bd887640 1 --2- 172.21.15.169:0/3099882515 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11b80ac930 0x7f11b80cd6f0 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:31.016 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:31.012+0000 7f11bd887640 1 -- 172.21.15.169:0/3099882515 >> 172.21.15.169:0/3099882515 conn(0x7f11b8069930 msgr2=0x7f11b8069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:31.016 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:31.012+0000 7f11bd887640 1 -- 172.21.15.169:0/3099882515 shutdown_connections 2023-10-26T01:39:31.016 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:31.012+0000 7f11bd887640 1 -- 172.21.15.169:0/3099882515 wait complete. 2023-10-26T01:39:31.018 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:31.013+0000 7f11bd887640 1 Processor -- start 2023-10-26T01:39:31.018 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:31.014+0000 7f11bd887640 1 -- start start 2023-10-26T01:39:31.018 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:31.015+0000 7f11bd887640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f11b80ac930 0x7f11b8150800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:31.018 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:31.015+0000 7f11bd887640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11b80cdc30 0x7f11b80b02e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:31.019 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:31.015+0000 7f11bd887640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f11b81592d0 0x7f11b815baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:31.019 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:31.015+0000 7f11bd887640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11b8150d40 con 0x7f11b80cdc30 2023-10-26T01:39:31.019 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:31.015+0000 7f11bd887640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11b80c7a60 con 0x7f11b81592d0 2023-10-26T01:39:31.019 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:31.015+0000 7f11bd887640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11b80b0b10 con 0x7f11b80ac930 2023-10-26T01:39:31.019 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:31.015+0000 7f11b67fc640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11b80cdc30 0x7f11b80b02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:31.020 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:31.015+0000 7f11b67fc640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11b80cdc30 0x7f11b80b02e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:37434/0 (socket says 172.21.15.169:37434) 2023-10-26T01:39:31.020 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:31.015+0000 7f11b67fc640 1 -- 172.21.15.169:0/192127974 learned_addr learned my addr 172.21.15.169:0/192127974 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:39:31.020 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:31.015+0000 7f11b67fc640 1 -- 172.21.15.169:0/192127974 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f11b80ac930 msgr2=0x7f11b8150800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:31.020 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:31.015+0000 7f11b6ffd640 1 --2- 172.21.15.169:0/192127974 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f11b80ac930 0x7f11b8150800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:31.020 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:31.015+0000 7f11b67fc640 1 --2- 172.21.15.169:0/192127974 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f11b80ac930 0x7f11b8150800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:31.021 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:31.015+0000 7f11b67fc640 1 -- 172.21.15.169:0/192127974 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f11b81592d0 msgr2=0x7f11b815baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:31.021 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:31.015+0000 7f11b67fc640 1 --2- 172.21.15.169:0/192127974 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f11b81592d0 0x7f11b815baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:31.021 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:31.015+0000 7f11b67fc640 1 -- 172.21.15.169:0/192127974 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11ac009490 con 0x7f11b80cdc30 2023-10-26T01:39:31.022 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:31.016+0000 7f11b67fc640 1 --2- 172.21.15.169:0/192127974 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11b80cdc30 0x7f11b80b02e0 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f11ac009950 tx=0x7f11ac0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:31.023 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:31.016+0000 7f119ffff640 1 -- 172.21.15.169:0/192127974 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11ac003ea0 con 0x7f11b80cdc30 2023-10-26T01:39:31.023 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:31.016+0000 7f119ffff640 1 -- 172.21.15.169:0/192127974 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f11ac034440 con 0x7f11b80cdc30 2023-10-26T01:39:31.023 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:31.016+0000 7f11bd887640 1 -- 172.21.15.169:0/192127974 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11b80b0df0 con 0x7f11b80cdc30 2023-10-26T01:39:31.023 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:31.016+0000 7f119ffff640 1 -- 172.21.15.169:0/192127974 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11ac034850 con 0x7f11b80cdc30 2023-10-26T01:39:31.024 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:31.016+0000 7f11bd887640 1 -- 172.21.15.169:0/192127974 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11b80b13b0 con 0x7f11b80cdc30 2023-10-26T01:39:31.024 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:31.017+0000 7f11bd887640 1 -- 172.21.15.169:0/192127974 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11b804db10 con 0x7f11b80cdc30 2023-10-26T01:39:31.025 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:31.018+0000 7f119ffff640 1 -- 172.21.15.169:0/192127974 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f11ac03b040 con 0x7f11b80cdc30 2023-10-26T01:39:31.025 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:31.019+0000 7f119ffff640 1 --2- 172.21.15.169:0/192127974 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f11a406ae40 0x7f11a406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:31.025 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:31.019+0000 7f119ffff640 1 -- 172.21.15.169:0/192127974 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f11ac0abb20 con 0x7f11b80cdc30 2023-10-26T01:39:31.026 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:31.020+0000 7f11b6ffd640 1 --2- 172.21.15.169:0/192127974 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f11a406ae40 0x7f11a406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:31.027 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:31.021+0000 7f11b6ffd640 1 --2- 172.21.15.169:0/192127974 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f11a406ae40 0x7f11a406d300 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f11a8001270 tx=0x7f11a8009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:31.027 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:31.024+0000 7f119ffff640 1 -- 172.21.15.169:0/192127974 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f11ac076bc0 con 0x7f11b80cdc30 2023-10-26T01:39:31.198 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:31.195+0000 7f11bd887640 1 -- 172.21.15.169:0/192127974 --> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f11b80c7ba0 con 0x7f11a406ae40 2023-10-26T01:39:31.220 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:31.217+0000 7f119ffff640 1 -- 172.21.15.169:0/192127974 <== mgr.4112 v2:172.21.15.170:6832/1187156918 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+278073 (secure 0 0 0) 0x7f11b80c7ba0 con 0x7f11a406ae40 2023-10-26T01:39:31.223 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:39:31.225 INFO:teuthology.orchestra.run.smithi169.stderr:dumped all 2023-10-26T01:39:31.227 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:31.222+0000 7f11bd887640 1 -- 172.21.15.169:0/192127974 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f11a406ae40 msgr2=0x7f11a406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:31.227 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:31.222+0000 7f11bd887640 1 --2- 172.21.15.169:0/192127974 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f11a406ae40 0x7f11a406d300 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f11a8001270 tx=0x7f11a8009040 comp rx=0 tx=0).stop 2023-10-26T01:39:31.227 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:31.222+0000 7f11bd887640 1 -- 172.21.15.169:0/192127974 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11b80cdc30 msgr2=0x7f11b80b02e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:31.228 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:31.222+0000 7f11bd887640 1 --2- 172.21.15.169:0/192127974 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11b80cdc30 0x7f11b80b02e0 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f11ac009950 tx=0x7f11ac0038a0 comp rx=0 tx=0).stop 2023-10-26T01:39:31.228 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:31.222+0000 7f11bd887640 1 -- 172.21.15.169:0/192127974 shutdown_connections 2023-10-26T01:39:31.228 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:31.222+0000 7f11bd887640 1 --2- 172.21.15.169:0/192127974 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f11a406ae40 0x7f11a406d300 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:31.228 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:31.222+0000 7f11bd887640 1 --2- 172.21.15.169:0/192127974 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11b80cdc30 0x7f11b80b02e0 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:31.228 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:31.222+0000 7f11bd887640 1 --2- 172.21.15.169:0/192127974 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f11b81592d0 0x7f11b815baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:31.229 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:31.222+0000 7f11bd887640 1 --2- 172.21.15.169:0/192127974 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f11b80ac930 0x7f11b8150800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:31.229 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:31.222+0000 7f11bd887640 1 -- 172.21.15.169:0/192127974 >> 172.21.15.169:0/192127974 conn(0x7f11b8069930 msgr2=0x7f11b806ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:31.229 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:31.222+0000 7f11bd887640 1 -- 172.21.15.169:0/192127974 shutdown_connections 2023-10-26T01:39:31.229 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:31.222+0000 7f11bd887640 1 -- 172.21.15.169:0/192127974 wait complete. 2023-10-26T01:39:31.241 INFO:teuthology.orchestra.run.smithi169.stdout:{"pg_ready":true,"pg_map":{"version":414,"stamp":"2023-10-26T01:39:30.320024+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118,"ondisk_log_size":118,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":33,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":7623,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1554876,"statfs":{"total":1600000000,"available":1592193296,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.004175"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.339741+0000","last_change":"2023-10-26T01:39:25.339741+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.339741+0000","last_undegraded":"2023-10-26T01:39:25.339741+0000","last_fullsized":"2023-10-26T01:39:25.339741+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.342024+0000","last_change":"2023-10-26T01:39:25.342024+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.342024+0000","last_undegraded":"2023-10-26T01:39:25.342024+0000","last_fullsized":"2023-10-26T01:39:25.342024+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.349198+0000","last_change":"2023-10-26T01:39:25.349198+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.349198+0000","last_undegraded":"2023-10-26T01:39:25.349198+0000","last_fullsized":"2023-10-26T01:39:25.349198+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.347440+0000","last_change":"2023-10-26T01:39:25.347440+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.347440+0000","last_undegraded":"2023-10-26T01:39:25.347440+0000","last_fullsized":"2023-10-26T01:39:25.347440+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.337848+0000","last_change":"2023-10-26T01:39:25.337848+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.337848+0000","last_undegraded":"2023-10-26T01:39:25.337848+0000","last_fullsized":"2023-10-26T01:39:25.337848+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.343578+0000","last_change":"2023-10-26T01:39:25.343578+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.343578+0000","last_undegraded":"2023-10-26T01:39:25.343578+0000","last_fullsized":"2023-10-26T01:39:25.343578+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.345665+0000","last_change":"2023-10-26T01:39:25.345665+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.345665+0000","last_undegraded":"2023-10-26T01:39:25.345665+0000","last_fullsized":"2023-10-26T01:39:25.345665+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.348666+0000","last_change":"2023-10-26T01:39:25.348666+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.348666+0000","last_undegraded":"2023-10-26T01:39:25.348666+0000","last_fullsized":"2023-10-26T01:39:25.348666+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.349996+0000","last_change":"2023-10-26T01:39:25.349996+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.349996+0000","last_undegraded":"2023-10-26T01:39:25.349996+0000","last_fullsized":"2023-10-26T01:39:25.349996+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.342807+0000","last_change":"2023-10-26T01:39:25.342807+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.342807+0000","last_undegraded":"2023-10-26T01:39:25.342807+0000","last_fullsized":"2023-10-26T01:39:25.342807+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.340008+0000","last_change":"2023-10-26T01:39:25.340008+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.340008+0000","last_undegraded":"2023-10-26T01:39:25.340008+0000","last_fullsized":"2023-10-26T01:39:25.340008+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.348637+0000","last_change":"2023-10-26T01:39:25.348637+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.348637+0000","last_undegraded":"2023-10-26T01:39:25.348637+0000","last_fullsized":"2023-10-26T01:39:25.348637+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.343722+0000","last_change":"2023-10-26T01:39:25.343722+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.343722+0000","last_undegraded":"2023-10-26T01:39:25.343722+0000","last_fullsized":"2023-10-26T01:39:25.343722+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.343921+0000","last_change":"2023-10-26T01:39:25.343921+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.343921+0000","last_undegraded":"2023-10-26T01:39:25.343921+0000","last_fullsized":"2023-10-26T01:39:25.343921+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.345622+0000","last_change":"2023-10-26T01:39:25.345622+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.345622+0000","last_undegraded":"2023-10-26T01:39:25.345622+0000","last_fullsized":"2023-10-26T01:39:25.345622+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":9,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-26T01:39:26.356388+0000","last_change":"2023-10-26T01:39:26.355761+0000","last_active":"2023-10-26T01:39:26.356388+0000","last_peered":"2023-10-26T01:39:26.356388+0000","last_clean":"2023-10-26T01:39:26.356388+0000","last_became_active":"2023-10-26T01:39:26.355188+0000","last_became_peered":"2023-10-26T01:39:26.355188+0000","last_unstale":"2023-10-26T01:39:26.356388+0000","last_undegraded":"2023-10-26T01:39:26.356388+0000","last_fullsized":"2023-10-26T01:39:26.356388+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:19:25.353033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.341466+0000","last_change":"2023-10-26T01:39:25.341466+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.341466+0000","last_undegraded":"2023-10-26T01:39:25.341466+0000","last_fullsized":"2023-10-26T01:39:25.341466+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":9,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-26T01:39:26.348459+0000","last_change":"2023-10-26T01:39:26.348373+0000","last_active":"2023-10-26T01:39:26.348459+0000","last_peered":"2023-10-26T01:39:26.348459+0000","last_clean":"2023-10-26T01:39:26.348459+0000","last_became_active":"2023-10-26T01:39:26.346867+0000","last_became_peered":"2023-10-26T01:39:26.346867+0000","last_unstale":"2023-10-26T01:39:26.348459+0000","last_undegraded":"2023-10-26T01:39:26.348459+0000","last_fullsized":"2023-10-26T01:39:26.348459+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:21:37.836002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.341042+0000","last_change":"2023-10-26T01:39:25.341042+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.341042+0000","last_undegraded":"2023-10-26T01:39:25.341042+0000","last_fullsized":"2023-10-26T01:39:25.341042+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":9,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-26T01:39:26.351465+0000","last_change":"2023-10-26T01:39:26.351200+0000","last_active":"2023-10-26T01:39:26.351465+0000","last_peered":"2023-10-26T01:39:26.351465+0000","last_clean":"2023-10-26T01:39:26.351465+0000","last_became_active":"2023-10-26T01:39:26.350941+0000","last_became_peered":"2023-10-26T01:39:26.350941+0000","last_unstale":"2023-10-26T01:39:26.351465+0000","last_undegraded":"2023-10-26T01:39:26.351465+0000","last_fullsized":"2023-10-26T01:39:26.351465+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:27:04.484703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.338200+0000","last_change":"2023-10-26T01:39:25.338200+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.338200+0000","last_undegraded":"2023-10-26T01:39:25.338200+0000","last_fullsized":"2023-10-26T01:39:25.338200+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.339459+0000","last_change":"2023-10-26T01:39:25.339459+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.339459+0000","last_undegraded":"2023-10-26T01:39:25.339459+0000","last_fullsized":"2023-10-26T01:39:25.339459+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.347072+0000","last_change":"2023-10-26T01:39:25.347072+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.347072+0000","last_undegraded":"2023-10-26T01:39:25.347072+0000","last_fullsized":"2023-10-26T01:39:25.347072+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.340599+0000","last_change":"2023-10-26T01:39:25.340599+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.340599+0000","last_undegraded":"2023-10-26T01:39:25.340599+0000","last_fullsized":"2023-10-26T01:39:25.340599+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.342364+0000","last_change":"2023-10-26T01:39:25.342364+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.342364+0000","last_undegraded":"2023-10-26T01:39:25.342364+0000","last_fullsized":"2023-10-26T01:39:25.342364+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":9,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-26T01:39:26.356448+0000","last_change":"2023-10-26T01:39:26.355914+0000","last_active":"2023-10-26T01:39:26.356448+0000","last_peered":"2023-10-26T01:39:26.356448+0000","last_clean":"2023-10-26T01:39:26.356448+0000","last_became_active":"2023-10-26T01:39:26.355508+0000","last_became_peered":"2023-10-26T01:39:26.355508+0000","last_unstale":"2023-10-26T01:39:26.356448+0000","last_undegraded":"2023-10-26T01:39:26.356448+0000","last_fullsized":"2023-10-26T01:39:26.356448+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:15:35.499430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.341388+0000","last_change":"2023-10-26T01:39:25.341388+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.341388+0000","last_undegraded":"2023-10-26T01:39:25.341388+0000","last_fullsized":"2023-10-26T01:39:25.341388+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.340994+0000","last_change":"2023-10-26T01:39:25.340994+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.340994+0000","last_undegraded":"2023-10-26T01:39:25.340994+0000","last_fullsized":"2023-10-26T01:39:25.340994+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.346960+0000","last_change":"2023-10-26T01:39:25.346960+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.346960+0000","last_undegraded":"2023-10-26T01:39:25.346960+0000","last_fullsized":"2023-10-26T01:39:25.346960+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.345158+0000","last_change":"2023-10-26T01:39:25.345158+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.345158+0000","last_undegraded":"2023-10-26T01:39:25.345158+0000","last_fullsized":"2023-10-26T01:39:25.345158+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.337346+0000","last_change":"2023-10-26T01:39:25.337346+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.337346+0000","last_undegraded":"2023-10-26T01:39:25.337346+0000","last_fullsized":"2023-10-26T01:39:25.337346+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.340549+0000","last_change":"2023-10-26T01:39:25.340549+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.340549+0000","last_undegraded":"2023-10-26T01:39:25.340549+0000","last_fullsized":"2023-10-26T01:39:25.340549+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":9,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-26T01:39:26.344302+0000","last_change":"2023-10-26T01:39:26.344022+0000","last_active":"2023-10-26T01:39:26.344302+0000","last_peered":"2023-10-26T01:39:26.344302+0000","last_clean":"2023-10-26T01:39:26.344302+0000","last_became_active":"2023-10-26T01:39:26.341874+0000","last_became_peered":"2023-10-26T01:39:26.341874+0000","last_unstale":"2023-10-26T01:39:26.344302+0000","last_undegraded":"2023-10-26T01:39:26.344302+0000","last_fullsized":"2023-10-26T01:39:26.344302+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:34:17.840143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-26T01:39:25.347668+0000","last_change":"2023-10-26T01:39:25.347523+0000","last_active":"2023-10-26T01:39:25.347668+0000","last_peered":"2023-10-26T01:39:25.347668+0000","last_clean":"2023-10-26T01:39:25.347668+0000","last_became_active":"2023-10-26T01:39:25.346388+0000","last_became_peered":"2023-10-26T01:39:25.346388+0000","last_unstale":"2023-10-26T01:39:25.347668+0000","last_undegraded":"2023-10-26T01:39:25.347668+0000","last_fullsized":"2023-10-26T01:39:25.347668+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:01:31.862431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.346506+0000","last_change":"2023-10-26T01:39:25.346506+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.346506+0000","last_undegraded":"2023-10-26T01:39:25.346506+0000","last_fullsized":"2023-10-26T01:39:25.346506+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-26T01:39:25.348380+0000","last_change":"2023-10-26T01:39:25.347683+0000","last_active":"2023-10-26T01:39:25.348380+0000","last_peered":"2023-10-26T01:39:25.348380+0000","last_clean":"2023-10-26T01:39:25.348380+0000","last_became_active":"2023-10-26T01:39:25.346998+0000","last_became_peered":"2023-10-26T01:39:25.346998+0000","last_unstale":"2023-10-26T01:39:25.348380+0000","last_undegraded":"2023-10-26T01:39:25.348380+0000","last_fullsized":"2023-10-26T01:39:25.348380+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:08:33.196097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.335905+0000","last_change":"2023-10-26T01:39:25.335905+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.335905+0000","last_undegraded":"2023-10-26T01:39:25.335905+0000","last_fullsized":"2023-10-26T01:39:25.335905+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-26T01:39:25.352874+0000","last_change":"2023-10-26T01:39:25.352802+0000","last_active":"2023-10-26T01:39:25.352874+0000","last_peered":"2023-10-26T01:39:25.352874+0000","last_clean":"2023-10-26T01:39:25.352874+0000","last_became_active":"2023-10-26T01:39:25.352537+0000","last_became_peered":"2023-10-26T01:39:25.352537+0000","last_unstale":"2023-10-26T01:39:25.352874+0000","last_undegraded":"2023-10-26T01:39:25.352874+0000","last_fullsized":"2023-10-26T01:39:25.352874+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:43:42.148308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.336364+0000","last_change":"2023-10-26T01:39:25.336364+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.336364+0000","last_undegraded":"2023-10-26T01:39:25.336364+0000","last_fullsized":"2023-10-26T01:39:25.336364+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-26T01:39:25.348146+0000","last_change":"2023-10-26T01:39:25.347942+0000","last_active":"2023-10-26T01:39:25.348146+0000","last_peered":"2023-10-26T01:39:25.348146+0000","last_clean":"2023-10-26T01:39:25.348146+0000","last_became_active":"2023-10-26T01:39:25.347373+0000","last_became_peered":"2023-10-26T01:39:25.347373+0000","last_unstale":"2023-10-26T01:39:25.348146+0000","last_undegraded":"2023-10-26T01:39:25.348146+0000","last_fullsized":"2023-10-26T01:39:25.348146+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:56:13.935683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":9,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-26T01:39:26.348723+0000","last_change":"2023-10-26T01:39:26.348640+0000","last_active":"2023-10-26T01:39:26.348723+0000","last_peered":"2023-10-26T01:39:26.348723+0000","last_clean":"2023-10-26T01:39:26.348723+0000","last_became_active":"2023-10-26T01:39:26.348127+0000","last_became_peered":"2023-10-26T01:39:26.348127+0000","last_unstale":"2023-10-26T01:39:26.348723+0000","last_undegraded":"2023-10-26T01:39:26.348723+0000","last_fullsized":"2023-10-26T01:39:26.348723+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:35:50.504438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-26T01:39:25.351507+0000","last_change":"2023-10-26T01:39:25.351159+0000","last_active":"2023-10-26T01:39:25.351507+0000","last_peered":"2023-10-26T01:39:25.351507+0000","last_clean":"2023-10-26T01:39:25.351507+0000","last_became_active":"2023-10-26T01:39:25.349181+0000","last_became_peered":"2023-10-26T01:39:25.349181+0000","last_unstale":"2023-10-26T01:39:25.351507+0000","last_undegraded":"2023-10-26T01:39:25.351507+0000","last_fullsized":"2023-10-26T01:39:25.351507+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:32:20.357196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.339661+0000","last_change":"2023-10-26T01:39:25.339661+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.339661+0000","last_undegraded":"2023-10-26T01:39:25.339661+0000","last_fullsized":"2023-10-26T01:39:25.339661+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"150'1","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-26T01:39:26.362881+0000","last_change":"2023-10-26T01:39:25.352234+0000","last_active":"2023-10-26T01:39:26.362881+0000","last_peered":"2023-10-26T01:39:26.362881+0000","last_clean":"2023-10-26T01:39:26.362881+0000","last_became_active":"2023-10-26T01:39:25.351930+0000","last_became_peered":"2023-10-26T01:39:25.351930+0000","last_unstale":"2023-10-26T01:39:26.362881+0000","last_undegraded":"2023-10-26T01:39:26.362881+0000","last_fullsized":"2023-10-26T01:39:26.362881+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:23:47.193891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.345341+0000","last_change":"2023-10-26T01:39:25.345341+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.345341+0000","last_undegraded":"2023-10-26T01:39:25.345341+0000","last_fullsized":"2023-10-26T01:39:25.345341+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-26T01:39:25.345560+0000","last_change":"2023-10-26T01:39:25.345280+0000","last_active":"2023-10-26T01:39:25.345560+0000","last_peered":"2023-10-26T01:39:25.345560+0000","last_clean":"2023-10-26T01:39:25.345560+0000","last_became_active":"2023-10-26T01:39:25.344841+0000","last_became_peered":"2023-10-26T01:39:25.344841+0000","last_unstale":"2023-10-26T01:39:25.345560+0000","last_undegraded":"2023-10-26T01:39:25.345560+0000","last_fullsized":"2023-10-26T01:39:25.345560+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:01:31.862431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.339971+0000","last_change":"2023-10-26T01:39:25.339971+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.339971+0000","last_undegraded":"2023-10-26T01:39:25.339971+0000","last_fullsized":"2023-10-26T01:39:25.339971+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-26T01:39:25.355216+0000","last_change":"2023-10-26T01:39:25.355147+0000","last_active":"2023-10-26T01:39:25.355216+0000","last_peered":"2023-10-26T01:39:25.355216+0000","last_clean":"2023-10-26T01:39:25.355216+0000","last_became_active":"2023-10-26T01:39:25.354564+0000","last_became_peered":"2023-10-26T01:39:25.354564+0000","last_unstale":"2023-10-26T01:39:25.355216+0000","last_undegraded":"2023-10-26T01:39:25.355216+0000","last_fullsized":"2023-10-26T01:39:25.355216+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:08:33.196097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.340967+0000","last_change":"2023-10-26T01:39:25.340967+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.340967+0000","last_undegraded":"2023-10-26T01:39:25.340967+0000","last_fullsized":"2023-10-26T01:39:25.340967+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-26T01:39:25.354084+0000","last_change":"2023-10-26T01:39:25.353938+0000","last_active":"2023-10-26T01:39:25.354084+0000","last_peered":"2023-10-26T01:39:25.354084+0000","last_clean":"2023-10-26T01:39:25.354084+0000","last_became_active":"2023-10-26T01:39:25.353704+0000","last_became_peered":"2023-10-26T01:39:25.353704+0000","last_unstale":"2023-10-26T01:39:25.354084+0000","last_undegraded":"2023-10-26T01:39:25.354084+0000","last_fullsized":"2023-10-26T01:39:25.354084+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:28:46.712484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.344566+0000","last_change":"2023-10-26T01:39:25.344566+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.344566+0000","last_undegraded":"2023-10-26T01:39:25.344566+0000","last_fullsized":"2023-10-26T01:39:25.344566+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":11,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-26T01:39:26.333356+0000","last_change":"2023-10-26T01:39:25.353364+0000","last_active":"2023-10-26T01:39:26.333356+0000","last_peered":"2023-10-26T01:39:26.333356+0000","last_clean":"2023-10-26T01:39:26.333356+0000","last_became_active":"2023-10-26T01:39:25.353129+0000","last_became_peered":"2023-10-26T01:39:25.353129+0000","last_unstale":"2023-10-26T01:39:26.333356+0000","last_undegraded":"2023-10-26T01:39:26.333356+0000","last_fullsized":"2023-10-26T01:39:26.333356+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:07:24.731619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.332527+0000","last_change":"2023-10-26T01:39:25.332527+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.332527+0000","last_undegraded":"2023-10-26T01:39:25.332527+0000","last_fullsized":"2023-10-26T01:39:25.332527+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-26T01:39:25.353094+0000","last_change":"2023-10-26T01:39:25.352244+0000","last_active":"2023-10-26T01:39:25.353094+0000","last_peered":"2023-10-26T01:39:25.353094+0000","last_clean":"2023-10-26T01:39:25.353094+0000","last_became_active":"2023-10-26T01:39:25.351989+0000","last_became_peered":"2023-10-26T01:39:25.351989+0000","last_unstale":"2023-10-26T01:39:25.353094+0000","last_undegraded":"2023-10-26T01:39:25.353094+0000","last_fullsized":"2023-10-26T01:39:25.353094+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:56:13.935683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.333835+0000","last_change":"2023-10-26T01:39:25.333835+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.333835+0000","last_undegraded":"2023-10-26T01:39:25.333835+0000","last_fullsized":"2023-10-26T01:39:25.333835+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-26T01:39:25.354938+0000","last_change":"2023-10-26T01:39:25.354828+0000","last_active":"2023-10-26T01:39:25.354938+0000","last_peered":"2023-10-26T01:39:25.354938+0000","last_clean":"2023-10-26T01:39:25.354938+0000","last_became_active":"2023-10-26T01:39:25.354185+0000","last_became_peered":"2023-10-26T01:39:25.354185+0000","last_unstale":"2023-10-26T01:39:25.354938+0000","last_undegraded":"2023-10-26T01:39:25.354938+0000","last_fullsized":"2023-10-26T01:39:25.354938+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:32:50.379779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.344701+0000","last_change":"2023-10-26T01:39:25.344701+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.344701+0000","last_undegraded":"2023-10-26T01:39:25.344701+0000","last_fullsized":"2023-10-26T01:39:25.344701+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-26T01:39:25.353183+0000","last_change":"2023-10-26T01:39:25.350711+0000","last_active":"2023-10-26T01:39:25.353183+0000","last_peered":"2023-10-26T01:39:25.353183+0000","last_clean":"2023-10-26T01:39:25.353183+0000","last_became_active":"2023-10-26T01:39:25.349869+0000","last_became_peered":"2023-10-26T01:39:25.349869+0000","last_unstale":"2023-10-26T01:39:25.353183+0000","last_undegraded":"2023-10-26T01:39:25.353183+0000","last_fullsized":"2023-10-26T01:39:25.353183+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:30:24.885507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.340874+0000","last_change":"2023-10-26T01:39:25.340874+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.340874+0000","last_undegraded":"2023-10-26T01:39:25.340874+0000","last_fullsized":"2023-10-26T01:39:25.340874+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-26T01:39:25.342677+0000","last_change":"2023-10-26T01:39:25.342525+0000","last_active":"2023-10-26T01:39:25.342677+0000","last_peered":"2023-10-26T01:39:25.342677+0000","last_clean":"2023-10-26T01:39:25.342677+0000","last_became_active":"2023-10-26T01:39:25.341183+0000","last_became_peered":"2023-10-26T01:39:25.341183+0000","last_unstale":"2023-10-26T01:39:25.342677+0000","last_undegraded":"2023-10-26T01:39:25.342677+0000","last_fullsized":"2023-10-26T01:39:25.342677+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:07:38.898672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.344820+0000","last_change":"2023-10-26T01:39:25.344820+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.344820+0000","last_undegraded":"2023-10-26T01:39:25.344820+0000","last_fullsized":"2023-10-26T01:39:25.344820+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-26T01:39:25.355255+0000","last_change":"2023-10-26T01:39:25.355183+0000","last_active":"2023-10-26T01:39:25.355255+0000","last_peered":"2023-10-26T01:39:25.355255+0000","last_clean":"2023-10-26T01:39:25.355255+0000","last_became_active":"2023-10-26T01:39:25.354751+0000","last_became_peered":"2023-10-26T01:39:25.354751+0000","last_unstale":"2023-10-26T01:39:25.355255+0000","last_undegraded":"2023-10-26T01:39:25.355255+0000","last_fullsized":"2023-10-26T01:39:25.355255+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:39:10.557327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.343890+0000","last_change":"2023-10-26T01:39:25.343890+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.343890+0000","last_undegraded":"2023-10-26T01:39:25.343890+0000","last_fullsized":"2023-10-26T01:39:25.343890+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-26T01:39:25.350561+0000","last_change":"2023-10-26T01:39:25.350279+0000","last_active":"2023-10-26T01:39:25.350561+0000","last_peered":"2023-10-26T01:39:25.350561+0000","last_clean":"2023-10-26T01:39:25.350561+0000","last_became_active":"2023-10-26T01:39:25.349301+0000","last_became_peered":"2023-10-26T01:39:25.349301+0000","last_unstale":"2023-10-26T01:39:25.350561+0000","last_undegraded":"2023-10-26T01:39:25.350561+0000","last_fullsized":"2023-10-26T01:39:25.350561+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:02:52.785225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.342089+0000","last_change":"2023-10-26T01:39:25.342089+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.342089+0000","last_undegraded":"2023-10-26T01:39:25.342089+0000","last_fullsized":"2023-10-26T01:39:25.342089+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-26T01:39:25.355575+0000","last_change":"2023-10-26T01:39:25.355508+0000","last_active":"2023-10-26T01:39:25.355575+0000","last_peered":"2023-10-26T01:39:25.355575+0000","last_clean":"2023-10-26T01:39:25.355575+0000","last_became_active":"2023-10-26T01:39:25.355254+0000","last_became_peered":"2023-10-26T01:39:25.355254+0000","last_unstale":"2023-10-26T01:39:25.355575+0000","last_undegraded":"2023-10-26T01:39:25.355575+0000","last_fullsized":"2023-10-26T01:39:25.355575+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:12:53.520877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":9,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-26T01:39:26.354933+0000","last_change":"2023-10-26T01:39:26.354034+0000","last_active":"2023-10-26T01:39:26.354933+0000","last_peered":"2023-10-26T01:39:26.354933+0000","last_clean":"2023-10-26T01:39:26.354933+0000","last_became_active":"2023-10-26T01:39:26.353811+0000","last_became_peered":"2023-10-26T01:39:26.353811+0000","last_unstale":"2023-10-26T01:39:26.354933+0000","last_undegraded":"2023-10-26T01:39:26.354933+0000","last_fullsized":"2023-10-26T01:39:26.354933+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:53:55.836062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-26T01:39:25.350542+0000","last_change":"2023-10-26T01:39:25.348929+0000","last_active":"2023-10-26T01:39:25.350542+0000","last_peered":"2023-10-26T01:39:25.350542+0000","last_clean":"2023-10-26T01:39:25.350542+0000","last_became_active":"2023-10-26T01:39:25.348637+0000","last_became_peered":"2023-10-26T01:39:25.348637+0000","last_unstale":"2023-10-26T01:39:25.350542+0000","last_undegraded":"2023-10-26T01:39:25.350542+0000","last_fullsized":"2023-10-26T01:39:25.350542+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:16:32.808798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.340936+0000","last_change":"2023-10-26T01:39:25.340936+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.340936+0000","last_undegraded":"2023-10-26T01:39:25.340936+0000","last_fullsized":"2023-10-26T01:39:25.340936+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-26T01:39:25.353113+0000","last_change":"2023-10-26T01:39:25.352675+0000","last_active":"2023-10-26T01:39:25.353113+0000","last_peered":"2023-10-26T01:39:25.353113+0000","last_clean":"2023-10-26T01:39:25.353113+0000","last_became_active":"2023-10-26T01:39:25.352188+0000","last_became_peered":"2023-10-26T01:39:25.352188+0000","last_unstale":"2023-10-26T01:39:25.353113+0000","last_undegraded":"2023-10-26T01:39:25.353113+0000","last_fullsized":"2023-10-26T01:39:25.353113+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:07:38.898672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.346409+0000","last_change":"2023-10-26T01:39:25.346409+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.346409+0000","last_undegraded":"2023-10-26T01:39:25.346409+0000","last_fullsized":"2023-10-26T01:39:25.346409+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-26T01:39:25.354022+0000","last_change":"2023-10-26T01:39:25.353938+0000","last_active":"2023-10-26T01:39:25.354022+0000","last_peered":"2023-10-26T01:39:25.354022+0000","last_clean":"2023-10-26T01:39:25.354022+0000","last_became_active":"2023-10-26T01:39:25.353665+0000","last_became_peered":"2023-10-26T01:39:25.353665+0000","last_unstale":"2023-10-26T01:39:25.354022+0000","last_undegraded":"2023-10-26T01:39:25.354022+0000","last_fullsized":"2023-10-26T01:39:25.354022+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:32:20.357196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.342827+0000","last_change":"2023-10-26T01:39:25.342827+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.342827+0000","last_undegraded":"2023-10-26T01:39:25.342827+0000","last_fullsized":"2023-10-26T01:39:25.342827+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":11,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-26T01:39:26.328128+0000","last_change":"2023-10-26T01:39:25.355042+0000","last_active":"2023-10-26T01:39:26.328128+0000","last_peered":"2023-10-26T01:39:26.328128+0000","last_clean":"2023-10-26T01:39:26.328128+0000","last_became_active":"2023-10-26T01:39:25.354798+0000","last_became_peered":"2023-10-26T01:39:25.354798+0000","last_unstale":"2023-10-26T01:39:26.328128+0000","last_undegraded":"2023-10-26T01:39:26.328128+0000","last_fullsized":"2023-10-26T01:39:26.328128+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:34:40.255257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.340172+0000","last_change":"2023-10-26T01:39:25.340172+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.340172+0000","last_undegraded":"2023-10-26T01:39:25.340172+0000","last_fullsized":"2023-10-26T01:39:25.340172+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":11,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-26T01:39:26.342348+0000","last_change":"2023-10-26T01:39:25.355043+0000","last_active":"2023-10-26T01:39:26.342348+0000","last_peered":"2023-10-26T01:39:26.342348+0000","last_clean":"2023-10-26T01:39:26.342348+0000","last_became_active":"2023-10-26T01:39:25.354518+0000","last_became_peered":"2023-10-26T01:39:25.354518+0000","last_unstale":"2023-10-26T01:39:26.342348+0000","last_undegraded":"2023-10-26T01:39:26.342348+0000","last_fullsized":"2023-10-26T01:39:26.342348+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:37:06.664661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.335577+0000","last_change":"2023-10-26T01:39:25.335577+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.335577+0000","last_undegraded":"2023-10-26T01:39:25.335577+0000","last_fullsized":"2023-10-26T01:39:25.335577+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-26T01:39:25.355626+0000","last_change":"2023-10-26T01:39:25.355568+0000","last_active":"2023-10-26T01:39:25.355626+0000","last_peered":"2023-10-26T01:39:25.355626+0000","last_clean":"2023-10-26T01:39:25.355626+0000","last_became_active":"2023-10-26T01:39:25.354482+0000","last_became_peered":"2023-10-26T01:39:25.354482+0000","last_unstale":"2023-10-26T01:39:25.355626+0000","last_undegraded":"2023-10-26T01:39:25.355626+0000","last_fullsized":"2023-10-26T01:39:25.355626+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:10:56.447860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.333593+0000","last_change":"2023-10-26T01:39:25.333593+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.333593+0000","last_undegraded":"2023-10-26T01:39:25.333593+0000","last_fullsized":"2023-10-26T01:39:25.333593+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-26T01:39:25.350624+0000","last_change":"2023-10-26T01:39:25.350493+0000","last_active":"2023-10-26T01:39:25.350624+0000","last_peered":"2023-10-26T01:39:25.350624+0000","last_clean":"2023-10-26T01:39:25.350624+0000","last_became_active":"2023-10-26T01:39:25.349758+0000","last_became_peered":"2023-10-26T01:39:25.349758+0000","last_unstale":"2023-10-26T01:39:25.350624+0000","last_undegraded":"2023-10-26T01:39:25.350624+0000","last_fullsized":"2023-10-26T01:39:25.350624+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:23:21.057813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.339995+0000","last_change":"2023-10-26T01:39:25.339995+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.339995+0000","last_undegraded":"2023-10-26T01:39:25.339995+0000","last_fullsized":"2023-10-26T01:39:25.339995+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-26T01:39:25.347943+0000","last_change":"2023-10-26T01:39:25.347830+0000","last_active":"2023-10-26T01:39:25.347943+0000","last_peered":"2023-10-26T01:39:25.347943+0000","last_clean":"2023-10-26T01:39:25.347943+0000","last_became_active":"2023-10-26T01:39:25.346994+0000","last_became_peered":"2023-10-26T01:39:25.346994+0000","last_unstale":"2023-10-26T01:39:25.347943+0000","last_undegraded":"2023-10-26T01:39:25.347943+0000","last_fullsized":"2023-10-26T01:39:25.347943+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:16:32.808798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.340256+0000","last_change":"2023-10-26T01:39:25.340256+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.340256+0000","last_undegraded":"2023-10-26T01:39:25.340256+0000","last_fullsized":"2023-10-26T01:39:25.340256+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-26T01:39:25.354467+0000","last_change":"2023-10-26T01:39:25.354096+0000","last_active":"2023-10-26T01:39:25.354467+0000","last_peered":"2023-10-26T01:39:25.354467+0000","last_clean":"2023-10-26T01:39:25.354467+0000","last_became_active":"2023-10-26T01:39:25.353857+0000","last_became_peered":"2023-10-26T01:39:25.353857+0000","last_unstale":"2023-10-26T01:39:25.354467+0000","last_undegraded":"2023-10-26T01:39:25.354467+0000","last_fullsized":"2023-10-26T01:39:25.354467+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:28:46.712484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.335457+0000","last_change":"2023-10-26T01:39:25.335457+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.335457+0000","last_undegraded":"2023-10-26T01:39:25.335457+0000","last_fullsized":"2023-10-26T01:39:25.335457+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-26T01:39:25.350001+0000","last_change":"2023-10-26T01:39:25.349514+0000","last_active":"2023-10-26T01:39:25.350001+0000","last_peered":"2023-10-26T01:39:25.350001+0000","last_clean":"2023-10-26T01:39:25.350001+0000","last_became_active":"2023-10-26T01:39:25.349279+0000","last_became_peered":"2023-10-26T01:39:25.349279+0000","last_unstale":"2023-10-26T01:39:25.350001+0000","last_undegraded":"2023-10-26T01:39:25.350001+0000","last_fullsized":"2023-10-26T01:39:25.350001+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:51:35.449470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.337282+0000","last_change":"2023-10-26T01:39:25.337282+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.337282+0000","last_undegraded":"2023-10-26T01:39:25.337282+0000","last_fullsized":"2023-10-26T01:39:25.337282+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-26T01:39:25.348101+0000","last_change":"2023-10-26T01:39:25.347759+0000","last_active":"2023-10-26T01:39:25.348101+0000","last_peered":"2023-10-26T01:39:25.348101+0000","last_clean":"2023-10-26T01:39:25.348101+0000","last_became_active":"2023-10-26T01:39:25.346564+0000","last_became_peered":"2023-10-26T01:39:25.346564+0000","last_unstale":"2023-10-26T01:39:25.348101+0000","last_undegraded":"2023-10-26T01:39:25.348101+0000","last_fullsized":"2023-10-26T01:39:25.348101+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:12:27.843055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.343816+0000","last_change":"2023-10-26T01:39:25.343816+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.343816+0000","last_undegraded":"2023-10-26T01:39:25.343816+0000","last_fullsized":"2023-10-26T01:39:25.343816+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-26T01:39:25.344486+0000","last_change":"2023-10-26T01:39:25.343413+0000","last_active":"2023-10-26T01:39:25.344486+0000","last_peered":"2023-10-26T01:39:25.344486+0000","last_clean":"2023-10-26T01:39:25.344486+0000","last_became_active":"2023-10-26T01:39:25.342949+0000","last_became_peered":"2023-10-26T01:39:25.342949+0000","last_unstale":"2023-10-26T01:39:25.344486+0000","last_undegraded":"2023-10-26T01:39:25.344486+0000","last_fullsized":"2023-10-26T01:39:25.344486+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:11:10.758066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":9,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-26T01:39:26.336802+0000","last_change":"2023-10-26T01:39:26.335564+0000","last_active":"2023-10-26T01:39:26.336802+0000","last_peered":"2023-10-26T01:39:26.336802+0000","last_clean":"2023-10-26T01:39:26.336802+0000","last_became_active":"2023-10-26T01:39:26.335000+0000","last_became_peered":"2023-10-26T01:39:26.335000+0000","last_unstale":"2023-10-26T01:39:26.336802+0000","last_undegraded":"2023-10-26T01:39:26.336802+0000","last_fullsized":"2023-10-26T01:39:26.336802+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:05:15.173045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":11,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-26T01:39:26.338413+0000","last_change":"2023-10-26T01:39:25.356190+0000","last_active":"2023-10-26T01:39:26.338413+0000","last_peered":"2023-10-26T01:39:26.338413+0000","last_clean":"2023-10-26T01:39:26.338413+0000","last_became_active":"2023-10-26T01:39:25.355791+0000","last_became_peered":"2023-10-26T01:39:25.355791+0000","last_unstale":"2023-10-26T01:39:26.338413+0000","last_undegraded":"2023-10-26T01:39:26.338413+0000","last_fullsized":"2023-10-26T01:39:26.338413+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:24:21.026112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.339461+0000","last_change":"2023-10-26T01:39:25.339461+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.339461+0000","last_undegraded":"2023-10-26T01:39:25.339461+0000","last_fullsized":"2023-10-26T01:39:25.339461+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-26T01:39:25.349964+0000","last_change":"2023-10-26T01:39:25.347950+0000","last_active":"2023-10-26T01:39:25.349964+0000","last_peered":"2023-10-26T01:39:25.349964+0000","last_clean":"2023-10-26T01:39:25.349964+0000","last_became_active":"2023-10-26T01:39:25.347416+0000","last_became_peered":"2023-10-26T01:39:25.347416+0000","last_unstale":"2023-10-26T01:39:25.349964+0000","last_undegraded":"2023-10-26T01:39:25.349964+0000","last_fullsized":"2023-10-26T01:39:25.349964+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:22:18.250771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.334115+0000","last_change":"2023-10-26T01:39:25.334115+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.334115+0000","last_undegraded":"2023-10-26T01:39:25.334115+0000","last_fullsized":"2023-10-26T01:39:25.334115+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-26T01:39:25.353367+0000","last_change":"2023-10-26T01:39:25.352322+0000","last_active":"2023-10-26T01:39:25.353367+0000","last_peered":"2023-10-26T01:39:25.353367+0000","last_clean":"2023-10-26T01:39:25.353367+0000","last_became_active":"2023-10-26T01:39:25.351287+0000","last_became_peered":"2023-10-26T01:39:25.351287+0000","last_unstale":"2023-10-26T01:39:25.353367+0000","last_undegraded":"2023-10-26T01:39:25.353367+0000","last_fullsized":"2023-10-26T01:39:25.353367+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:03:57.007851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":292,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-26T01:39:24.316788+0000","last_change":"2023-10-26T01:30:34.883484+0000","last_active":"2023-10-26T01:39:24.316788+0000","last_peered":"2023-10-26T01:39:24.316788+0000","last_clean":"2023-10-26T01:39:24.316788+0000","last_became_active":"2023-10-26T01:30:34.883071+0000","last_became_peered":"2023-10-26T01:30:34.883071+0000","last_unstale":"2023-10-26T01:39:24.316788+0000","last_undegraded":"2023-10-26T01:39:24.316788+0000","last_fullsized":"2023-10-26T01:39:24.316788+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:32.890096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:32.890096+0000","last_clean_scrub_stamp":"2023-10-26T01:30:32.890096+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:26:31.879452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":14,"seq":60129542254,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":817,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194495,"statfs":{"total":200000000,"available":199163256,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:38:51 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.443,"5min":0.35599999999999998,"15min":0.33000000000000002},"max":{"1min":1.2549999999999999,"5min":1.6319999999999999,"15min":2.2549999999999999},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.92300000000000004,"15min":0.89300000000000002},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":1.752,"5min":1.752,"15min":1.8200000000000001},"last":0.67800000000000005}]},{"osd":1,"last update":"Thu Oct 26 01:38:51 2023","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.88400000000000001,"15min":0.92800000000000005},"min":{"1min":0.39400000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.383,"5min":1.444,"15min":4.4340000000000002},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.996,"5min":0.99199999999999999,"15min":0.995},"min":{"1min":0.56200000000000006,"5min":0.46700000000000003,"15min":0.374},"max":{"1min":1.353,"5min":1.538,"15min":2.3330000000000002},"last":0.70999999999999996}]},{"osd":2,"last update":"Thu Oct 26 01:38:51 2023","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":1.028,"15min":1.0700000000000001},"min":{"1min":0.54500000000000004,"5min":0.40999999999999998,"15min":0.39900000000000002},"max":{"1min":1.4279999999999999,"5min":1.927,"15min":4.2800000000000002},"last":0.79400000000000004},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.1479999999999999,"15min":1.0980000000000001},"min":{"1min":0.53300000000000003,"5min":0.495,"15min":0.49399999999999999},"max":{"1min":1.639,"5min":1.8680000000000001,"15min":2.302},"last":0.94399999999999995}]},{"osd":4,"last update":"Thu Oct 26 01:38:51 2023","interfaces":[{"interface":"back","average":{"1min":1.266,"5min":1.23,"15min":1.175},"min":{"1min":0.98199999999999998,"5min":0.755,"15min":0.66400000000000003},"max":{"1min":1.732,"5min":1.732,"15min":2.1070000000000002},"last":1.375},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.353,"15min":1.3859999999999999},"min":{"1min":0.91900000000000004,"5min":0.81599999999999995,"15min":0.79900000000000004},"max":{"1min":1.6220000000000001,"5min":2.4239999999999999,"15min":4.8250000000000002},"last":1.4019999999999999}]},{"osd":5,"last update":"Thu Oct 26 01:38:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.2929999999999999,"15min":1.3280000000000001},"min":{"1min":0.84899999999999998,"5min":0.76400000000000001,"15min":0.72499999999999998},"max":{"1min":1.6899999999999999,"5min":2.3380000000000001,"15min":4.9080000000000004},"last":1.2709999999999999},{"interface":"front","average":{"1min":1.331,"5min":1.2969999999999999,"15min":1.2789999999999999},"min":{"1min":0.82699999999999996,"5min":0.79900000000000004,"15min":0.79600000000000004},"max":{"1min":1.573,"5min":2.0310000000000001,"15min":2.8980000000000001},"last":1.385}]},{"osd":6,"last update":"Thu Oct 26 01:38:51 2023","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.351,"15min":1.329},"min":{"1min":0.75900000000000001,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.694,"5min":2.1499999999999999,"15min":2.8530000000000002},"last":1.448},{"interface":"front","average":{"1min":1.4159999999999999,"5min":1.3160000000000001,"15min":1.2769999999999999},"min":{"1min":0.84399999999999997,"5min":0.81699999999999995,"15min":0.73099999999999998},"max":{"1min":1.6899999999999999,"5min":2.048,"15min":2.181},"last":1.48}]},{"osd":7,"last update":"Thu Oct 26 01:38:51 2023","interfaces":[{"interface":"back","average":{"1min":1.4370000000000001,"5min":1.3660000000000001,"15min":1.3069999999999999},"min":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.71999999999999997},"max":{"1min":1.7589999999999999,"5min":1.958,"15min":2.0680000000000001},"last":1.3340000000000001},{"interface":"front","average":{"1min":1.385,"5min":1.393,"15min":1.411},"min":{"1min":0.73299999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.702,"5min":2.2370000000000001,"15min":4.8730000000000002},"last":1.595}]}]},{"osd":7,"up_from":13,"seq":55834574957,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1380,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193931,"statfs":{"total":200000000,"available":198586098,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:38:50 2023","interfaces":[{"interface":"back","average":{"1min":1.127,"5min":1.1379999999999999,"15min":1.097},"min":{"1min":0.68700000000000006,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.633,"5min":1.7,"15min":1.9410000000000001},"last":1.1879999999999999},{"interface":"front","average":{"1min":1.095,"5min":1.1100000000000001,"15min":1.0840000000000001},"min":{"1min":0.67300000000000004,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.6970000000000001,"5min":2.6779999999999999,"15min":2.6779999999999999},"last":1.294}]},{"osd":1,"last update":"Thu Oct 26 01:38:50 2023","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":1.1699999999999999,"15min":1.1020000000000001},"min":{"1min":0.82899999999999996,"5min":0.65800000000000003,"15min":0.65200000000000002},"max":{"1min":1.484,"5min":2.8260000000000001,"15min":2.8260000000000001},"last":0.879},{"interface":"front","average":{"1min":1.177,"5min":1.2,"15min":1.194},"min":{"1min":0.77200000000000002,"5min":0.66500000000000004,"15min":0.61199999999999999},"max":{"1min":1.76,"5min":3.2730000000000001,"15min":3.2730000000000001},"last":1.1240000000000001}]},{"osd":2,"last update":"Thu Oct 26 01:38:50 2023","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.2110000000000001,"15min":1.1299999999999999},"min":{"1min":0.873,"5min":0.69099999999999995,"15min":0.58499999999999996},"max":{"1min":1.851,"5min":2.218,"15min":2.218},"last":0.93400000000000005},{"interface":"front","average":{"1min":1.256,"5min":1.296,"15min":1.2430000000000001},"min":{"1min":0.82299999999999995,"5min":0.79400000000000004,"15min":0.73099999999999998},"max":{"1min":1.7350000000000001,"5min":3.0510000000000002,"15min":3.0510000000000002},"last":1.0229999999999999}]},{"osd":3,"last update":"Thu Oct 26 01:38:50 2023","interfaces":[{"interface":"back","average":{"1min":1.3120000000000001,"5min":1.296,"15min":1.2230000000000001},"min":{"1min":0.79500000000000004,"5min":0.79000000000000004,"15min":0.72099999999999997},"max":{"1min":1.8169999999999999,"5min":1.9830000000000001,"15min":1.9830000000000001},"last":0.79800000000000004},{"interface":"front","average":{"1min":1.224,"5min":1.373,"15min":1.3640000000000001},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.65700000000000003},"max":{"1min":1.5129999999999999,"5min":5.5039999999999996,"15min":5.5039999999999996},"last":1.651}]},{"osd":4,"last update":"Thu Oct 26 01:38:50 2023","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.371,"15min":1.351},"min":{"1min":0.81799999999999995,"5min":0.77300000000000002,"15min":0.76100000000000001},"max":{"1min":1.9339999999999999,"5min":2.77,"15min":2.77},"last":2.0390000000000001},{"interface":"front","average":{"1min":1.288,"5min":1.3919999999999999,"15min":1.393},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.6479999999999999,"5min":3.1419999999999999,"15min":3.1419999999999999},"last":2.536}]},{"osd":5,"last update":"Thu Oct 26 01:38:50 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.4410000000000001,"15min":1.4199999999999999},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.7809999999999999,"5min":5.734,"15min":5.734},"last":1.363},{"interface":"front","average":{"1min":1.4450000000000001,"5min":1.514,"15min":1.478},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.51900000000000002},"max":{"1min":2,"5min":2.8980000000000001,"15min":2.8980000000000001},"last":1.0920000000000001}]},{"osd":6,"last update":"Thu Oct 26 01:38:50 2023","interfaces":[{"interface":"back","average":{"1min":1.452,"5min":1.5800000000000001,"15min":1.8089999999999999},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.47399999999999998},"max":{"1min":1.8049999999999999,"5min":5.6340000000000003,"15min":11.577999999999999},"last":1.7769999999999999},{"interface":"front","average":{"1min":1.613,"5min":1.6719999999999999,"15min":1.8360000000000001},"min":{"1min":1.0640000000000001,"5min":0.91800000000000004,"15min":0.55100000000000005},"max":{"1min":2.0529999999999999,"5min":5.0919999999999996,"15min":10.704000000000001},"last":1.8779999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574957,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":803,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194508,"statfs":{"total":200000000,"available":199177080,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:38:46 2023","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.141,"15min":1.1519999999999999},"min":{"1min":0.66600000000000004,"5min":0.61699999999999999,"15min":0.60399999999999998},"max":{"1min":2.7370000000000001,"5min":2.7370000000000001,"15min":2.7370000000000001},"last":1.0760000000000001},{"interface":"front","average":{"1min":1.137,"5min":1.173,"15min":1.123},"min":{"1min":0.61699999999999999,"5min":0.61399999999999999,"15min":0.59199999999999997},"max":{"1min":4.4779999999999998,"5min":4.4779999999999998,"15min":4.4779999999999998},"last":1.0189999999999999}]},{"osd":1,"last update":"Thu Oct 26 01:38:46 2023","interfaces":[{"interface":"back","average":{"1min":1.3919999999999999,"5min":1.2729999999999999,"15min":1.1930000000000001},"min":{"1min":0.77100000000000002,"5min":0.72299999999999998,"15min":0.68300000000000005},"max":{"1min":5.5359999999999996,"5min":5.5359999999999996,"15min":5.5359999999999996},"last":1.246},{"interface":"front","average":{"1min":1.228,"5min":1.2190000000000001,"15min":1.204},"min":{"1min":0.74399999999999999,"5min":0.67900000000000005,"15min":0.64000000000000001},"max":{"1min":4.3579999999999997,"5min":4.3579999999999997,"15min":4.3579999999999997},"last":1.1279999999999999}]},{"osd":2,"last update":"Thu Oct 26 01:38:46 2023","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.2030000000000001,"15min":1.107},"min":{"1min":0.76300000000000001,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":2.367,"5min":3.7879999999999998,"15min":3.7879999999999998},"last":1.175},{"interface":"front","average":{"1min":1.371,"5min":1.3420000000000001,"15min":1.276},"min":{"1min":0.91200000000000003,"5min":0.75800000000000001,"15min":0.68899999999999995},"max":{"1min":3.0470000000000002,"5min":3.0470000000000002,"15min":3.0470000000000002},"last":1.4430000000000001}]},{"osd":3,"last update":"Thu Oct 26 01:38:46 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.331,"15min":1.22},"min":{"1min":0.76300000000000001,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":3.266,"5min":4.4560000000000004,"15min":4.4560000000000004},"last":1.601},{"interface":"front","average":{"1min":1.365,"5min":1.3500000000000001,"15min":1.3280000000000001},"min":{"1min":0.86399999999999999,"5min":0.78600000000000003,"15min":0.69499999999999995},"max":{"1min":2.6909999999999998,"5min":2.6909999999999998,"15min":2.6909999999999998},"last":1.55}]},{"osd":4,"last update":"Thu Oct 26 01:38:46 2023","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.24,"15min":1.22},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.48399999999999999},"max":{"1min":1.7410000000000001,"5min":2.9489999999999998,"15min":2.9489999999999998},"last":1.395},{"interface":"front","average":{"1min":1.3180000000000001,"5min":1.421,"15min":1.319},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.59299999999999997},"max":{"1min":1.698,"5min":6.609,"15min":6.609},"last":1.494}]},{"osd":5,"last update":"Thu Oct 26 01:38:46 2023","interfaces":[{"interface":"back","average":{"1min":1.464,"5min":1.5389999999999999,"15min":1.452},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.7949999999999999,"5min":5.4370000000000003,"15min":5.4370000000000003},"last":1.671},{"interface":"front","average":{"1min":1.492,"5min":1.4530000000000001,"15min":1.419},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":2.7759999999999998,"5min":2.7759999999999998,"15min":2.7759999999999998},"last":1.6459999999999999}]},{"osd":7,"last update":"Thu Oct 26 01:38:46 2023","interfaces":[{"interface":"back","average":{"1min":1.2869999999999999,"5min":1.484,"15min":1.405},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.54100000000000004},"max":{"1min":1.8200000000000001,"5min":11.804,"15min":11.804},"last":1.698},{"interface":"front","average":{"1min":1.577,"5min":1.6639999999999999,"15min":1.52},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":3.7650000000000001,"5min":11.541,"15min":11.541},"last":1.728}]}]},{"osd":5,"up_from":13,"seq":55834574957,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":810,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194502,"statfs":{"total":200000000,"available":199170191,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:38:48 2023","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.2310000000000001,"15min":1.147},"min":{"1min":0.69699999999999995,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":3.8210000000000002,"5min":3.8210000000000002,"15min":3.8210000000000002},"last":1.5549999999999999},{"interface":"front","average":{"1min":1.3480000000000001,"5min":1.302,"15min":1.1970000000000001},"min":{"1min":0.92600000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":2.9889999999999999,"5min":2.9889999999999999,"15min":2.9889999999999999},"last":1.48}]},{"osd":1,"last update":"Thu Oct 26 01:38:48 2023","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.1319999999999999,"15min":1.0940000000000001},"min":{"1min":0.83599999999999997,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.8120000000000001,"5min":1.8120000000000001,"15min":1.8120000000000001},"last":1.514},{"interface":"front","average":{"1min":1.272,"5min":1.1659999999999999,"15min":1.109},"min":{"1min":0.76700000000000002,"5min":0.629,"15min":0.60599999999999998},"max":{"1min":2.4340000000000002,"5min":2.4340000000000002,"15min":2.4340000000000002},"last":0.98199999999999998}]},{"osd":2,"last update":"Thu Oct 26 01:38:48 2023","interfaces":[{"interface":"back","average":{"1min":1.2130000000000001,"5min":1.206,"15min":1.1679999999999999},"min":{"1min":0.79700000000000004,"5min":0.71599999999999997,"15min":0.69299999999999995},"max":{"1min":1.5920000000000001,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":1.2270000000000001},{"interface":"front","average":{"1min":1.228,"5min":1.228,"15min":1.1850000000000001},"min":{"1min":0.91600000000000004,"5min":0.78400000000000003,"15min":0.63900000000000001},"max":{"1min":1.5900000000000001,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":1.6459999999999999}]},{"osd":3,"last update":"Thu Oct 26 01:38:48 2023","interfaces":[{"interface":"back","average":{"1min":1.389,"5min":1.2829999999999999,"15min":1.25},"min":{"1min":1.0009999999999999,"5min":0.72699999999999998,"15min":0.72299999999999998},"max":{"1min":1.7,"5min":1.7,"15min":1.835},"last":1.101},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.298,"15min":1.29},"min":{"1min":0.91300000000000003,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.52,"5min":1.788,"15min":2.9249999999999998},"last":0.74299999999999999}]},{"osd":4,"last update":"Thu Oct 26 01:38:48 2023","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":1.256,"15min":1.214},"min":{"1min":0.75,"5min":0.59599999999999997,"15min":0.49099999999999999},"max":{"1min":2.7770000000000001,"5min":2.7770000000000001,"15min":2.7770000000000001},"last":1.681},{"interface":"front","average":{"1min":1.3180000000000001,"5min":1.3979999999999999,"15min":1.355},"min":{"1min":0.67500000000000004,"5min":0.57299999999999995,"15min":0.56200000000000006},"max":{"1min":1.776,"5min":1.7849999999999999,"15min":1.867},"last":0.86699999999999999}]},{"osd":6,"last update":"Thu Oct 26 01:38:48 2023","interfaces":[{"interface":"back","average":{"1min":1.377,"5min":1.393,"15min":1.3300000000000001},"min":{"1min":0.79000000000000004,"5min":0.67500000000000004,"15min":0.51900000000000002},"max":{"1min":1.956,"5min":2.8250000000000002,"15min":2.8250000000000002},"last":1.4339999999999999},{"interface":"front","average":{"1min":1.4590000000000001,"5min":1.409,"15min":1.3400000000000001},"min":{"1min":0.89300000000000002,"5min":0.69499999999999995,"15min":0.61799999999999999},"max":{"1min":2.4830000000000001,"5min":2.4830000000000001,"15min":2.4830000000000001},"last":1.339}]},{"osd":7,"last update":"Thu Oct 26 01:38:48 2023","interfaces":[{"interface":"back","average":{"1min":1.6000000000000001,"5min":1.534,"15min":1.4379999999999999},"min":{"1min":0.81499999999999995,"5min":0.71099999999999997,"15min":0.61699999999999999},"max":{"1min":3.621,"5min":3.621,"15min":3.621},"last":1.609},{"interface":"front","average":{"1min":1.603,"5min":1.542,"15min":1.4470000000000001},"min":{"1min":0.82899999999999996,"5min":0.73899999999999999,"15min":0.59499999999999997},"max":{"1min":2.8010000000000002,"5min":2.8010000000000002,"15min":2.8010000000000002},"last":1.7889999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574957,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":804,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194508,"statfs":{"total":200000000,"available":199176466,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:38:43 2023","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.071,"15min":0.999},"min":{"1min":0.63500000000000001,"5min":0.59799999999999998,"15min":0.53100000000000003},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":2.0289999999999999},"last":1.7689999999999999},{"interface":"front","average":{"1min":1.123,"5min":1.077,"15min":1.0329999999999999},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.60599999999999998},"max":{"1min":1.784,"5min":1.784,"15min":2.6829999999999998},"last":1.921}]},{"osd":1,"last update":"Thu Oct 26 01:38:43 2023","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":1.161,"15min":1.1259999999999999},"min":{"1min":0.749,"5min":0.66700000000000004,"15min":0.56799999999999995},"max":{"1min":1.579,"5min":1.599,"15min":4.0519999999999996},"last":2.056},{"interface":"front","average":{"1min":1.2529999999999999,"5min":1.1779999999999999,"15min":1.121},"min":{"1min":0.70799999999999996,"5min":0.70399999999999996,"15min":0.60499999999999998},"max":{"1min":2.3919999999999999,"5min":2.3919999999999999,"15min":2.3919999999999999},"last":1.8919999999999999}]},{"osd":2,"last update":"Thu Oct 26 01:38:43 2023","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":1.304,"15min":1.2430000000000001},"min":{"1min":1.0429999999999999,"5min":0.66700000000000004,"15min":0.64300000000000002},"max":{"1min":1.6759999999999999,"5min":1.845,"15min":5.6749999999999998},"last":2.2040000000000002},{"interface":"front","average":{"1min":1.1799999999999999,"5min":1.218,"15min":1.1419999999999999},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.631,"5min":3.673,"15min":3.673},"last":1.2210000000000001}]},{"osd":3,"last update":"Thu Oct 26 01:38:43 2023","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.228,"15min":1.1919999999999999},"min":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.70399999999999996},"max":{"1min":1.7030000000000001,"5min":1.847,"15min":1.891},"last":1.9890000000000001},{"interface":"front","average":{"1min":1.1799999999999999,"5min":1.204,"15min":1.1619999999999999},"min":{"1min":0.86099999999999999,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.522,"5min":1.681,"15min":1.7210000000000001},"last":1.353}]},{"osd":5,"last update":"Thu Oct 26 01:38:43 2023","interfaces":[{"interface":"back","average":{"1min":1.236,"5min":1.258,"15min":1.2809999999999999},"min":{"1min":0.85699999999999998,"5min":0.52100000000000002,"15min":0.46300000000000002},"max":{"1min":1.681,"5min":2.1429999999999998,"15min":2.1429999999999998},"last":2.2469999999999999},{"interface":"front","average":{"1min":1.2270000000000001,"5min":1.2809999999999999,"15min":1.321},"min":{"1min":0.84899999999999998,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.754,"5min":1.9810000000000001,"15min":5.8579999999999997},"last":2.2850000000000001}]},{"osd":6,"last update":"Thu Oct 26 01:38:43 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.294,"15min":1.335},"min":{"1min":0.752,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.708,"5min":2.2149999999999999,"15min":12.343999999999999},"last":1.4419999999999999},{"interface":"front","average":{"1min":1.5129999999999999,"5min":1.5109999999999999,"15min":1.5189999999999999},"min":{"1min":1.0660000000000001,"5min":0.57999999999999996,"15min":0.55000000000000004},"max":{"1min":1.762,"5min":2.117,"15min":12.605},"last":2.1040000000000001}]},{"osd":7,"last update":"Thu Oct 26 01:38:43 2023","interfaces":[{"interface":"back","average":{"1min":1.4690000000000001,"5min":1.4670000000000001,"15min":1.49},"min":{"1min":1.1279999999999999,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.8200000000000001,"5min":2.762,"15min":5.774},"last":2.3330000000000002},{"interface":"front","average":{"1min":1.399,"5min":1.484,"15min":1.4590000000000001},"min":{"1min":1.008,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.7589999999999999,"5min":2.8330000000000002,"15min":4.6520000000000001},"last":1.5149999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574957,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":810,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194502,"statfs":{"total":200000000,"available":199170191,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:38:47 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.84499999999999997,"15min":0.85299999999999998},"min":{"1min":0.46600000000000003,"5min":0.41199999999999998,"15min":0.308},"max":{"1min":1.4079999999999999,"5min":1.5249999999999999,"15min":6.3200000000000003},"last":0.95999999999999996},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.97699999999999998,"15min":0.93300000000000005},"min":{"1min":0.44,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.476,"5min":1.9970000000000001,"15min":4.2619999999999996},"last":1.0129999999999999}]},{"osd":1,"last update":"Thu Oct 26 01:38:47 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.878,"15min":0.89100000000000001},"min":{"1min":0.48399999999999999,"5min":0.39300000000000002,"15min":0.34000000000000002},"max":{"1min":1.2829999999999999,"5min":1.629,"15min":5.1260000000000003},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.98499999999999999,"5min":1.0189999999999999,"15min":1.0029999999999999},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.36599999999999999},"max":{"1min":1.4870000000000001,"5min":1.873,"15min":6.1520000000000001},"last":1.286}]},{"osd":3,"last update":"Thu Oct 26 01:38:52 2023","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":1.004,"15min":0.94499999999999995},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.407,"5min":2.0640000000000001,"15min":2.0640000000000001},"last":1.3500000000000001},{"interface":"front","average":{"1min":1.0349999999999999,"5min":1.0860000000000001,"15min":1.0249999999999999},"min":{"1min":0.53800000000000003,"5min":0.51100000000000001,"15min":0.45300000000000001},"max":{"1min":1.665,"5min":1.9279999999999999,"15min":8.9019999999999992},"last":1.093}]},{"osd":4,"last update":"Thu Oct 26 01:38:47 2023","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.3280000000000001,"15min":1.2689999999999999},"min":{"1min":0.81599999999999995,"5min":0.80500000000000005,"15min":0.73899999999999999},"max":{"1min":1.6080000000000001,"5min":2.6190000000000002,"15min":6.2670000000000003},"last":1.4850000000000001},{"interface":"front","average":{"1min":1.2709999999999999,"5min":1.2969999999999999,"15min":1.266},"min":{"1min":0.81699999999999995,"5min":0.81100000000000005,"15min":0.78700000000000003},"max":{"1min":1.8380000000000001,"5min":2.4359999999999999,"15min":7.343},"last":1.1950000000000001}]},{"osd":5,"last update":"Thu Oct 26 01:38:47 2023","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":1.3500000000000001,"15min":1.29},"min":{"1min":0.85099999999999998,"5min":0.81699999999999995,"15min":0.71899999999999997},"max":{"1min":1.669,"5min":2.5790000000000002,"15min":3.7170000000000001},"last":1.55},{"interface":"front","average":{"1min":1.302,"5min":1.3120000000000001,"15min":1.2929999999999999},"min":{"1min":0.872,"5min":0.81899999999999995,"15min":0.65300000000000002},"max":{"1min":1.708,"5min":2.298,"15min":6.3899999999999997},"last":1.5780000000000001}]},{"osd":6,"last update":"Thu Oct 26 01:38:47 2023","interfaces":[{"interface":"back","average":{"1min":1.3280000000000001,"5min":1.3919999999999999,"15min":1.3380000000000001},"min":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.70599999999999996},"max":{"1min":1.714,"5min":2.73,"15min":6.0490000000000004},"last":1.643},{"interface":"front","average":{"1min":1.379,"5min":1.4139999999999999,"15min":1.3740000000000001},"min":{"1min":0.89200000000000002,"5min":0.78200000000000003,"15min":0.76800000000000002},"max":{"1min":2.0990000000000002,"5min":2.4089999999999998,"15min":5.0650000000000004},"last":1.4399999999999999}]},{"osd":7,"last update":"Thu Oct 26 01:38:47 2023","interfaces":[{"interface":"back","average":{"1min":1.373,"5min":1.3979999999999999,"15min":1.3939999999999999},"min":{"1min":0.79200000000000004,"5min":0.77900000000000003,"15min":0.69399999999999995},"max":{"1min":1.776,"5min":2.3370000000000002,"15min":6.4480000000000004},"last":1.7949999999999999},{"interface":"front","average":{"1min":1.4830000000000001,"5min":1.5169999999999999,"15min":1.458},"min":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.91800000000000004},"max":{"1min":2.1480000000000001,"5min":2.8290000000000002,"15min":2.8439999999999999},"last":1.734}]}]},{"osd":1,"up_from":13,"seq":55834574957,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":810,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194502,"statfs":{"total":200000000,"available":199170191,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:38:49 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.76900000000000002,"15min":0.72799999999999998},"min":{"1min":0.41799999999999998,"5min":0.34000000000000002,"15min":0.29499999999999998},"max":{"1min":0.92700000000000005,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.87,"15min":0.80200000000000005},"min":{"1min":0.52000000000000002,"5min":0.36299999999999999,"15min":0.36199999999999999},"max":{"1min":1.2769999999999999,"5min":1.6399999999999999,"15min":1.6399999999999999},"last":0.71699999999999997}]},{"osd":2,"last update":"Thu Oct 26 01:38:49 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.995,"15min":0.999},"min":{"1min":0.65300000000000002,"5min":0.39700000000000002,"15min":0.39600000000000002},"max":{"1min":1.286,"5min":1.8200000000000001,"15min":2.4830000000000001},"last":0.874},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.96199999999999997,"15min":0.88400000000000001},"min":{"1min":0.48099999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.569,"5min":1.8080000000000001,"15min":1.8080000000000001},"last":1.1339999999999999}]},{"osd":3,"last update":"Thu Oct 26 01:38:53 2023","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":0.996,"15min":0.95499999999999996},"min":{"1min":0.59999999999999998,"5min":0.42499999999999999,"15min":0.39500000000000002},"max":{"1min":1.5900000000000001,"5min":1.611,"15min":1.75},"last":1.292},{"interface":"front","average":{"1min":0.998,"5min":1.028,"15min":0.99199999999999999},"min":{"1min":0.55000000000000004,"5min":0.47799999999999998,"15min":0.41299999999999998},"max":{"1min":1.5049999999999999,"5min":1.673,"15min":1.802},"last":0.69799999999999995}]},{"osd":4,"last update":"Thu Oct 26 01:38:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":1.222,"15min":1.1990000000000001},"min":{"1min":0.90400000000000003,"5min":0.69099999999999995,"15min":0.56200000000000006},"max":{"1min":1.5249999999999999,"5min":1.756,"15min":2.0659999999999998},"last":1.024},{"interface":"front","average":{"1min":1.1659999999999999,"5min":1.1870000000000001,"15min":1.1399999999999999},"min":{"1min":0.81299999999999994,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.4390000000000001,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":1.0780000000000001}]},{"osd":5,"last update":"Thu Oct 26 01:38:49 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.2629999999999999,"15min":1.2290000000000001},"min":{"1min":0.84799999999999998,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.621,"5min":1.8779999999999999,"15min":1.8779999999999999},"last":1.1679999999999999},{"interface":"front","average":{"1min":1.359,"5min":1.375,"15min":1.339},"min":{"1min":1.052,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.7210000000000001,"5min":2.04,"15min":2.2200000000000002},"last":1.056}]},{"osd":6,"last update":"Thu Oct 26 01:38:49 2023","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.347,"15min":1.2849999999999999},"min":{"1min":0.90400000000000003,"5min":0.70199999999999996,"15min":0.68899999999999995},"max":{"1min":1.7709999999999999,"5min":2.0910000000000002,"15min":2.3780000000000001},"last":0.83399999999999996},{"interface":"front","average":{"1min":1.2889999999999999,"5min":1.2789999999999999,"15min":1.2490000000000001},"min":{"1min":0.92700000000000005,"5min":0.80700000000000005,"15min":0.70799999999999996},"max":{"1min":1.653,"5min":1.956,"15min":1.984},"last":1.22}]},{"osd":7,"last update":"Thu Oct 26 01:38:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.2749999999999999,"15min":1.262},"min":{"1min":0.79400000000000004,"5min":0.76100000000000001,"15min":0.69699999999999995},"max":{"1min":1.764,"5min":1.8380000000000001,"15min":2.6930000000000001},"last":1.256},{"interface":"front","average":{"1min":1.431,"5min":1.419,"15min":1.345},"min":{"1min":0.95499999999999996,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.835,"5min":2.0190000000000001,"15min":2.0190000000000001},"last":0.98699999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574957,"num_pgs":6,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1386,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193925,"statfs":{"total":200000000,"available":198579823,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Oct 26 01:38:46 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.79700000000000004,"15min":0.91100000000000003},"min":{"1min":0.47799999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.159,"5min":1.5049999999999999,"15min":19.425000000000001},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.93999999999999995,"15min":1.0369999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.462,"5min":1.462,"15min":15.162000000000001},"last":0.748}]},{"osd":2,"last update":"Thu Oct 26 01:38:46 2023","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":1.0149999999999999,"15min":1.175},"min":{"1min":0.64300000000000002,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.47,"5min":1.577,"15min":28.439},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.90000000000000002,"15min":0.91300000000000003},"min":{"1min":0.57299999999999995,"5min":0.37,"15min":0.37},"max":{"1min":1.2969999999999999,"5min":1.4810000000000001,"15min":4.9459999999999997},"last":1.0980000000000001}]},{"osd":3,"last update":"Thu Oct 26 01:38:46 2023","interfaces":[{"interface":"back","average":{"1min":1.1180000000000001,"5min":1.0640000000000001,"15min":1.085},"min":{"1min":0.66800000000000004,"5min":0.46000000000000002,"15min":0.44600000000000001},"max":{"1min":1.6539999999999999,"5min":1.6539999999999999,"15min":4.9800000000000004},"last":1.5549999999999999},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.123,"15min":1.1499999999999999},"min":{"1min":0.66000000000000003,"5min":0.438,"15min":0.438},"max":{"1min":1.4279999999999999,"5min":1.6559999999999999,"15min":5.016},"last":1.5069999999999999}]},{"osd":4,"last update":"Thu Oct 26 01:38:46 2023","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":1.169,"15min":1.2789999999999999},"min":{"1min":0.94499999999999995,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.391,"5min":1.681,"15min":19.329000000000001},"last":1.026},{"interface":"front","average":{"1min":1.23,"5min":1.262,"15min":1.292},"min":{"1min":0.86799999999999999,"5min":0.82099999999999995,"15min":0.67600000000000005},"max":{"1min":1.5149999999999999,"5min":1.718,"15min":5.5229999999999997},"last":1.208}]},{"osd":5,"last update":"Thu Oct 26 01:38:46 2023","interfaces":[{"interface":"back","average":{"1min":1.427,"5min":1.4019999999999999,"15min":1.4279999999999999},"min":{"1min":1.03,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.819,"5min":1.857,"15min":5.6360000000000001},"last":1.609},{"interface":"front","average":{"1min":1.2909999999999999,"5min":1.2749999999999999,"15min":1.395},"min":{"1min":0.80200000000000005,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.5880000000000001,"5min":1.8200000000000001,"15min":19.384},"last":1.1539999999999999}]},{"osd":6,"last update":"Thu Oct 26 01:38:46 2023","interfaces":[{"interface":"back","average":{"1min":1.306,"5min":1.296,"15min":1.3600000000000001},"min":{"1min":0.79300000000000004,"5min":0.76700000000000002,"15min":0.73799999999999999},"max":{"1min":1.736,"5min":1.736,"15min":8.2609999999999992},"last":1.3380000000000001},{"interface":"front","average":{"1min":1.3,"5min":1.335,"15min":1.3999999999999999},"min":{"1min":0.82999999999999996,"5min":0.77900000000000003,"15min":0.75900000000000001},"max":{"1min":1.581,"5min":1.9450000000000001,"15min":11.849},"last":1.373}]},{"osd":7,"last update":"Thu Oct 26 01:38:46 2023","interfaces":[{"interface":"back","average":{"1min":1.46,"5min":1.4319999999999999,"15min":1.4770000000000001},"min":{"1min":1.089,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.9239999999999999,"5min":1.9239999999999999,"15min":5.6689999999999996},"last":1.722},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.397,"15min":1.4930000000000001},"min":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.69399999999999995},"max":{"1min":1.661,"5min":1.9390000000000001,"15min":19.245999999999999},"last":1.4279999999999999}]}]}],"pool_statfs":[]}} 2023-10-26T01:39:31.248 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-26T01:39:31.503 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:31.500+0000 7f1a5167c640 1 -- 172.21.15.169:0/932284195 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a340051d0 msgr2=0x7f1a3400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:31.503 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:31.500+0000 7f1a5167c640 1 --2- 172.21.15.169:0/932284195 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a340051d0 0x7f1a3400b980 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f1a440043b0 tx=0x7f1a340057f0 comp rx=0 tx=0).stop 2023-10-26T01:39:31.503 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:31.500+0000 7f1a5167c640 1 -- 172.21.15.169:0/932284195 shutdown_connections 2023-10-26T01:39:31.503 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:31.500+0000 7f1a5167c640 1 --2- 172.21.15.169:0/932284195 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a340051d0 0x7f1a3400b980 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:31.504 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:31.500+0000 7f1a5167c640 1 -- 172.21.15.169:0/932284195 >> 172.21.15.169:0/932284195 conn(0x7f1a4c069930 msgr2=0x7f1a4c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:31.504 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:31.500+0000 7f1a5167c640 1 -- 172.21.15.169:0/932284195 shutdown_connections 2023-10-26T01:39:31.504 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:31.500+0000 7f1a5167c640 1 -- 172.21.15.169:0/932284195 wait complete. 2023-10-26T01:39:31.506 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:31.501+0000 7f1a5167c640 1 Processor -- start 2023-10-26T01:39:31.506 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:31.502+0000 7f1a5167c640 1 -- start start 2023-10-26T01:39:31.506 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:31.502+0000 7f1a5167c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a340051d0 0x7f1a4c097f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:31.506 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:31.503+0000 7f1a5167c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1a4c148930 0x7f1a4c14a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:31.507 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:31.503+0000 7f1a5167c640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1a4c1490f0 0x7f1a4c0701a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:31.507 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:31.503+0000 7f1a5167c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a4c14a5c0 con 0x7f1a340051d0 2023-10-26T01:39:31.507 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:31.503+0000 7f1a5167c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a4c14af20 con 0x7f1a4c148930 2023-10-26T01:39:31.507 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:31.503+0000 7f1a5167c640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a4c149940 con 0x7f1a4c1490f0 2023-10-26T01:39:31.507 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:31.503+0000 7f1a4affd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a340051d0 0x7f1a4c097f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:31.508 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:31.503+0000 7f1a4affd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a340051d0 0x7f1a4c097f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:37440/0 (socket says 172.21.15.169:37440) 2023-10-26T01:39:31.508 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:31.503+0000 7f1a4affd640 1 -- 172.21.15.169:0/3784479460 learned_addr learned my addr 172.21.15.169:0/3784479460 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:39:31.508 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:31.503+0000 7f1a4affd640 1 -- 172.21.15.169:0/3784479460 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1a4c1490f0 msgr2=0x7f1a4c0701a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:31.508 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:31.503+0000 7f1a4affd640 1 --2- 172.21.15.169:0/3784479460 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1a4c1490f0 0x7f1a4c0701a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:31.508 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:31.503+0000 7f1a4affd640 1 -- 172.21.15.169:0/3784479460 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1a4c148930 msgr2=0x7f1a4c14a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:31.509 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:31.503+0000 7f1a4affd640 1 --2- 172.21.15.169:0/3784479460 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1a4c148930 0x7f1a4c14a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:31.509 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:31.503+0000 7f1a4affd640 1 -- 172.21.15.169:0/3784479460 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a4c149c20 con 0x7f1a340051d0 2023-10-26T01:39:31.509 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:31.504+0000 7f1a4affd640 1 --2- 172.21.15.169:0/3784479460 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a340051d0 0x7f1a4c097f00 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c00f7d0 tx=0x7f1a3c00fc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:31.511 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:31.504+0000 7f1a337fe640 1 -- 172.21.15.169:0/3784479460 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a3c00fee0 con 0x7f1a340051d0 2023-10-26T01:39:31.511 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:31.504+0000 7f1a337fe640 1 -- 172.21.15.169:0/3784479460 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1a3c00bda0 con 0x7f1a340051d0 2023-10-26T01:39:31.511 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:31.504+0000 7f1a337fe640 1 -- 172.21.15.169:0/3784479460 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a3c004290 con 0x7f1a340051d0 2023-10-26T01:39:31.511 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:31.504+0000 7f1a5167c640 1 -- 172.21.15.169:0/3784479460 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a4c149f00 con 0x7f1a340051d0 2023-10-26T01:39:31.512 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:31.504+0000 7f1a5167c640 1 -- 172.21.15.169:0/3784479460 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a4c148db0 con 0x7f1a340051d0 2023-10-26T01:39:31.512 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:31.505+0000 7f1a5167c640 1 -- 172.21.15.169:0/3784479460 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a4c06bed0 con 0x7f1a340051d0 2023-10-26T01:39:31.512 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:31.506+0000 7f1a337fe640 1 -- 172.21.15.169:0/3784479460 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f1a3c0035f0 con 0x7f1a340051d0 2023-10-26T01:39:31.512 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:31.507+0000 7f1a337fe640 1 --2- 172.21.15.169:0/3784479460 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f1a3806ae40 0x7f1a3806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:31.513 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:31.507+0000 7f1a337fe640 1 -- 172.21.15.169:0/3784479460 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1a3c089ed0 con 0x7f1a340051d0 2023-10-26T01:39:31.513 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:31.508+0000 7f1a4a7fc640 1 --2- 172.21.15.169:0/3784479460 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f1a3806ae40 0x7f1a3806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:31.513 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:31.509+0000 7f1a4a7fc640 1 --2- 172.21.15.169:0/3784479460 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f1a3806ae40 0x7f1a3806d300 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c15a480 tx=0x7f1a40001ac0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:31.514 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:31.511+0000 7f1a337fe640 1 -- 172.21.15.169:0/3784479460 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1a3c0550c0 con 0x7f1a340051d0 2023-10-26T01:39:31.691 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:31.689+0000 7f1a5167c640 1 -- 172.21.15.169:0/3784479460 --> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1a4c04db10 con 0x7f1a3806ae40 2023-10-26T01:39:31.714 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:31.711+0000 7f1a337fe640 1 -- 172.21.15.169:0/3784479460 <== mgr.4112 v2:172.21.15.170:6832/1187156918 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+278073 (secure 0 0 0) 0x7f1a4c04db10 con 0x7f1a3806ae40 2023-10-26T01:39:31.718 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:39:31.719 INFO:teuthology.orchestra.run.smithi169.stderr:dumped all 2023-10-26T01:39:31.721 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:31.716+0000 7f1a5167c640 1 -- 172.21.15.169:0/3784479460 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f1a3806ae40 msgr2=0x7f1a3806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:31.722 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:31.716+0000 7f1a5167c640 1 --2- 172.21.15.169:0/3784479460 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f1a3806ae40 0x7f1a3806d300 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c15a480 tx=0x7f1a40001ac0 comp rx=0 tx=0).stop 2023-10-26T01:39:31.722 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:31.716+0000 7f1a5167c640 1 -- 172.21.15.169:0/3784479460 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a340051d0 msgr2=0x7f1a4c097f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:31.722 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:31.716+0000 7f1a5167c640 1 --2- 172.21.15.169:0/3784479460 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a340051d0 0x7f1a4c097f00 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c00f7d0 tx=0x7f1a3c00fc90 comp rx=0 tx=0).stop 2023-10-26T01:39:31.722 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:31.717+0000 7f1a5167c640 1 -- 172.21.15.169:0/3784479460 shutdown_connections 2023-10-26T01:39:31.723 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:31.717+0000 7f1a5167c640 1 --2- 172.21.15.169:0/3784479460 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f1a3806ae40 0x7f1a3806d300 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:31.723 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:31.717+0000 7f1a5167c640 1 --2- 172.21.15.169:0/3784479460 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1a4c1490f0 0x7f1a4c0701a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:31.723 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:31.717+0000 7f1a5167c640 1 --2- 172.21.15.169:0/3784479460 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1a4c148930 0x7f1a4c14a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:31.723 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:31.717+0000 7f1a5167c640 1 --2- 172.21.15.169:0/3784479460 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a340051d0 0x7f1a4c097f00 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:31.724 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:31.717+0000 7f1a5167c640 1 -- 172.21.15.169:0/3784479460 >> 172.21.15.169:0/3784479460 conn(0x7f1a4c069930 msgr2=0x7f1a4c0cc7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:31.724 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:31.717+0000 7f1a5167c640 1 -- 172.21.15.169:0/3784479460 shutdown_connections 2023-10-26T01:39:31.724 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:31.717+0000 7f1a5167c640 1 -- 172.21.15.169:0/3784479460 wait complete. 2023-10-26T01:39:31.736 INFO:teuthology.orchestra.run.smithi169.stdout:{"pg_ready":true,"pg_map":{"version":414,"stamp":"2023-10-26T01:39:30.320024+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118,"ondisk_log_size":118,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":33,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":7623,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1554876,"statfs":{"total":1600000000,"available":1592193296,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.004175"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.339741+0000","last_change":"2023-10-26T01:39:25.339741+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.339741+0000","last_undegraded":"2023-10-26T01:39:25.339741+0000","last_fullsized":"2023-10-26T01:39:25.339741+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.342024+0000","last_change":"2023-10-26T01:39:25.342024+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.342024+0000","last_undegraded":"2023-10-26T01:39:25.342024+0000","last_fullsized":"2023-10-26T01:39:25.342024+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.349198+0000","last_change":"2023-10-26T01:39:25.349198+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.349198+0000","last_undegraded":"2023-10-26T01:39:25.349198+0000","last_fullsized":"2023-10-26T01:39:25.349198+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.347440+0000","last_change":"2023-10-26T01:39:25.347440+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.347440+0000","last_undegraded":"2023-10-26T01:39:25.347440+0000","last_fullsized":"2023-10-26T01:39:25.347440+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.337848+0000","last_change":"2023-10-26T01:39:25.337848+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.337848+0000","last_undegraded":"2023-10-26T01:39:25.337848+0000","last_fullsized":"2023-10-26T01:39:25.337848+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.343578+0000","last_change":"2023-10-26T01:39:25.343578+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.343578+0000","last_undegraded":"2023-10-26T01:39:25.343578+0000","last_fullsized":"2023-10-26T01:39:25.343578+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.345665+0000","last_change":"2023-10-26T01:39:25.345665+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.345665+0000","last_undegraded":"2023-10-26T01:39:25.345665+0000","last_fullsized":"2023-10-26T01:39:25.345665+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.348666+0000","last_change":"2023-10-26T01:39:25.348666+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.348666+0000","last_undegraded":"2023-10-26T01:39:25.348666+0000","last_fullsized":"2023-10-26T01:39:25.348666+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.349996+0000","last_change":"2023-10-26T01:39:25.349996+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.349996+0000","last_undegraded":"2023-10-26T01:39:25.349996+0000","last_fullsized":"2023-10-26T01:39:25.349996+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.342807+0000","last_change":"2023-10-26T01:39:25.342807+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.342807+0000","last_undegraded":"2023-10-26T01:39:25.342807+0000","last_fullsized":"2023-10-26T01:39:25.342807+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.340008+0000","last_change":"2023-10-26T01:39:25.340008+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.340008+0000","last_undegraded":"2023-10-26T01:39:25.340008+0000","last_fullsized":"2023-10-26T01:39:25.340008+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.348637+0000","last_change":"2023-10-26T01:39:25.348637+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.348637+0000","last_undegraded":"2023-10-26T01:39:25.348637+0000","last_fullsized":"2023-10-26T01:39:25.348637+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.343722+0000","last_change":"2023-10-26T01:39:25.343722+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.343722+0000","last_undegraded":"2023-10-26T01:39:25.343722+0000","last_fullsized":"2023-10-26T01:39:25.343722+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.343921+0000","last_change":"2023-10-26T01:39:25.343921+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.343921+0000","last_undegraded":"2023-10-26T01:39:25.343921+0000","last_fullsized":"2023-10-26T01:39:25.343921+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.345622+0000","last_change":"2023-10-26T01:39:25.345622+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.345622+0000","last_undegraded":"2023-10-26T01:39:25.345622+0000","last_fullsized":"2023-10-26T01:39:25.345622+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":9,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-26T01:39:26.356388+0000","last_change":"2023-10-26T01:39:26.355761+0000","last_active":"2023-10-26T01:39:26.356388+0000","last_peered":"2023-10-26T01:39:26.356388+0000","last_clean":"2023-10-26T01:39:26.356388+0000","last_became_active":"2023-10-26T01:39:26.355188+0000","last_became_peered":"2023-10-26T01:39:26.355188+0000","last_unstale":"2023-10-26T01:39:26.356388+0000","last_undegraded":"2023-10-26T01:39:26.356388+0000","last_fullsized":"2023-10-26T01:39:26.356388+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:19:25.353033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.341466+0000","last_change":"2023-10-26T01:39:25.341466+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.341466+0000","last_undegraded":"2023-10-26T01:39:25.341466+0000","last_fullsized":"2023-10-26T01:39:25.341466+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":9,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-26T01:39:26.348459+0000","last_change":"2023-10-26T01:39:26.348373+0000","last_active":"2023-10-26T01:39:26.348459+0000","last_peered":"2023-10-26T01:39:26.348459+0000","last_clean":"2023-10-26T01:39:26.348459+0000","last_became_active":"2023-10-26T01:39:26.346867+0000","last_became_peered":"2023-10-26T01:39:26.346867+0000","last_unstale":"2023-10-26T01:39:26.348459+0000","last_undegraded":"2023-10-26T01:39:26.348459+0000","last_fullsized":"2023-10-26T01:39:26.348459+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:21:37.836002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.341042+0000","last_change":"2023-10-26T01:39:25.341042+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.341042+0000","last_undegraded":"2023-10-26T01:39:25.341042+0000","last_fullsized":"2023-10-26T01:39:25.341042+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":9,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-26T01:39:26.351465+0000","last_change":"2023-10-26T01:39:26.351200+0000","last_active":"2023-10-26T01:39:26.351465+0000","last_peered":"2023-10-26T01:39:26.351465+0000","last_clean":"2023-10-26T01:39:26.351465+0000","last_became_active":"2023-10-26T01:39:26.350941+0000","last_became_peered":"2023-10-26T01:39:26.350941+0000","last_unstale":"2023-10-26T01:39:26.351465+0000","last_undegraded":"2023-10-26T01:39:26.351465+0000","last_fullsized":"2023-10-26T01:39:26.351465+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:27:04.484703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.338200+0000","last_change":"2023-10-26T01:39:25.338200+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.338200+0000","last_undegraded":"2023-10-26T01:39:25.338200+0000","last_fullsized":"2023-10-26T01:39:25.338200+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.339459+0000","last_change":"2023-10-26T01:39:25.339459+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.339459+0000","last_undegraded":"2023-10-26T01:39:25.339459+0000","last_fullsized":"2023-10-26T01:39:25.339459+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.347072+0000","last_change":"2023-10-26T01:39:25.347072+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.347072+0000","last_undegraded":"2023-10-26T01:39:25.347072+0000","last_fullsized":"2023-10-26T01:39:25.347072+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.340599+0000","last_change":"2023-10-26T01:39:25.340599+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.340599+0000","last_undegraded":"2023-10-26T01:39:25.340599+0000","last_fullsized":"2023-10-26T01:39:25.340599+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.342364+0000","last_change":"2023-10-26T01:39:25.342364+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.342364+0000","last_undegraded":"2023-10-26T01:39:25.342364+0000","last_fullsized":"2023-10-26T01:39:25.342364+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":9,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-26T01:39:26.356448+0000","last_change":"2023-10-26T01:39:26.355914+0000","last_active":"2023-10-26T01:39:26.356448+0000","last_peered":"2023-10-26T01:39:26.356448+0000","last_clean":"2023-10-26T01:39:26.356448+0000","last_became_active":"2023-10-26T01:39:26.355508+0000","last_became_peered":"2023-10-26T01:39:26.355508+0000","last_unstale":"2023-10-26T01:39:26.356448+0000","last_undegraded":"2023-10-26T01:39:26.356448+0000","last_fullsized":"2023-10-26T01:39:26.356448+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:15:35.499430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.341388+0000","last_change":"2023-10-26T01:39:25.341388+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.341388+0000","last_undegraded":"2023-10-26T01:39:25.341388+0000","last_fullsized":"2023-10-26T01:39:25.341388+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.340994+0000","last_change":"2023-10-26T01:39:25.340994+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.340994+0000","last_undegraded":"2023-10-26T01:39:25.340994+0000","last_fullsized":"2023-10-26T01:39:25.340994+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.346960+0000","last_change":"2023-10-26T01:39:25.346960+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.346960+0000","last_undegraded":"2023-10-26T01:39:25.346960+0000","last_fullsized":"2023-10-26T01:39:25.346960+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.345158+0000","last_change":"2023-10-26T01:39:25.345158+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.345158+0000","last_undegraded":"2023-10-26T01:39:25.345158+0000","last_fullsized":"2023-10-26T01:39:25.345158+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.337346+0000","last_change":"2023-10-26T01:39:25.337346+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.337346+0000","last_undegraded":"2023-10-26T01:39:25.337346+0000","last_fullsized":"2023-10-26T01:39:25.337346+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.340549+0000","last_change":"2023-10-26T01:39:25.340549+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.340549+0000","last_undegraded":"2023-10-26T01:39:25.340549+0000","last_fullsized":"2023-10-26T01:39:25.340549+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":9,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-26T01:39:26.344302+0000","last_change":"2023-10-26T01:39:26.344022+0000","last_active":"2023-10-26T01:39:26.344302+0000","last_peered":"2023-10-26T01:39:26.344302+0000","last_clean":"2023-10-26T01:39:26.344302+0000","last_became_active":"2023-10-26T01:39:26.341874+0000","last_became_peered":"2023-10-26T01:39:26.341874+0000","last_unstale":"2023-10-26T01:39:26.344302+0000","last_undegraded":"2023-10-26T01:39:26.344302+0000","last_fullsized":"2023-10-26T01:39:26.344302+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:34:17.840143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-26T01:39:25.347668+0000","last_change":"2023-10-26T01:39:25.347523+0000","last_active":"2023-10-26T01:39:25.347668+0000","last_peered":"2023-10-26T01:39:25.347668+0000","last_clean":"2023-10-26T01:39:25.347668+0000","last_became_active":"2023-10-26T01:39:25.346388+0000","last_became_peered":"2023-10-26T01:39:25.346388+0000","last_unstale":"2023-10-26T01:39:25.347668+0000","last_undegraded":"2023-10-26T01:39:25.347668+0000","last_fullsized":"2023-10-26T01:39:25.347668+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:01:31.862431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.346506+0000","last_change":"2023-10-26T01:39:25.346506+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.346506+0000","last_undegraded":"2023-10-26T01:39:25.346506+0000","last_fullsized":"2023-10-26T01:39:25.346506+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-26T01:39:25.348380+0000","last_change":"2023-10-26T01:39:25.347683+0000","last_active":"2023-10-26T01:39:25.348380+0000","last_peered":"2023-10-26T01:39:25.348380+0000","last_clean":"2023-10-26T01:39:25.348380+0000","last_became_active":"2023-10-26T01:39:25.346998+0000","last_became_peered":"2023-10-26T01:39:25.346998+0000","last_unstale":"2023-10-26T01:39:25.348380+0000","last_undegraded":"2023-10-26T01:39:25.348380+0000","last_fullsized":"2023-10-26T01:39:25.348380+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:08:33.196097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.335905+0000","last_change":"2023-10-26T01:39:25.335905+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.335905+0000","last_undegraded":"2023-10-26T01:39:25.335905+0000","last_fullsized":"2023-10-26T01:39:25.335905+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-26T01:39:25.352874+0000","last_change":"2023-10-26T01:39:25.352802+0000","last_active":"2023-10-26T01:39:25.352874+0000","last_peered":"2023-10-26T01:39:25.352874+0000","last_clean":"2023-10-26T01:39:25.352874+0000","last_became_active":"2023-10-26T01:39:25.352537+0000","last_became_peered":"2023-10-26T01:39:25.352537+0000","last_unstale":"2023-10-26T01:39:25.352874+0000","last_undegraded":"2023-10-26T01:39:25.352874+0000","last_fullsized":"2023-10-26T01:39:25.352874+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:43:42.148308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.336364+0000","last_change":"2023-10-26T01:39:25.336364+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.336364+0000","last_undegraded":"2023-10-26T01:39:25.336364+0000","last_fullsized":"2023-10-26T01:39:25.336364+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-26T01:39:25.348146+0000","last_change":"2023-10-26T01:39:25.347942+0000","last_active":"2023-10-26T01:39:25.348146+0000","last_peered":"2023-10-26T01:39:25.348146+0000","last_clean":"2023-10-26T01:39:25.348146+0000","last_became_active":"2023-10-26T01:39:25.347373+0000","last_became_peered":"2023-10-26T01:39:25.347373+0000","last_unstale":"2023-10-26T01:39:25.348146+0000","last_undegraded":"2023-10-26T01:39:25.348146+0000","last_fullsized":"2023-10-26T01:39:25.348146+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:56:13.935683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":9,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-26T01:39:26.348723+0000","last_change":"2023-10-26T01:39:26.348640+0000","last_active":"2023-10-26T01:39:26.348723+0000","last_peered":"2023-10-26T01:39:26.348723+0000","last_clean":"2023-10-26T01:39:26.348723+0000","last_became_active":"2023-10-26T01:39:26.348127+0000","last_became_peered":"2023-10-26T01:39:26.348127+0000","last_unstale":"2023-10-26T01:39:26.348723+0000","last_undegraded":"2023-10-26T01:39:26.348723+0000","last_fullsized":"2023-10-26T01:39:26.348723+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:35:50.504438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-26T01:39:25.351507+0000","last_change":"2023-10-26T01:39:25.351159+0000","last_active":"2023-10-26T01:39:25.351507+0000","last_peered":"2023-10-26T01:39:25.351507+0000","last_clean":"2023-10-26T01:39:25.351507+0000","last_became_active":"2023-10-26T01:39:25.349181+0000","last_became_peered":"2023-10-26T01:39:25.349181+0000","last_unstale":"2023-10-26T01:39:25.351507+0000","last_undegraded":"2023-10-26T01:39:25.351507+0000","last_fullsized":"2023-10-26T01:39:25.351507+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:32:20.357196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.339661+0000","last_change":"2023-10-26T01:39:25.339661+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.339661+0000","last_undegraded":"2023-10-26T01:39:25.339661+0000","last_fullsized":"2023-10-26T01:39:25.339661+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"150'1","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-26T01:39:26.362881+0000","last_change":"2023-10-26T01:39:25.352234+0000","last_active":"2023-10-26T01:39:26.362881+0000","last_peered":"2023-10-26T01:39:26.362881+0000","last_clean":"2023-10-26T01:39:26.362881+0000","last_became_active":"2023-10-26T01:39:25.351930+0000","last_became_peered":"2023-10-26T01:39:25.351930+0000","last_unstale":"2023-10-26T01:39:26.362881+0000","last_undegraded":"2023-10-26T01:39:26.362881+0000","last_fullsized":"2023-10-26T01:39:26.362881+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:23:47.193891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.345341+0000","last_change":"2023-10-26T01:39:25.345341+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.345341+0000","last_undegraded":"2023-10-26T01:39:25.345341+0000","last_fullsized":"2023-10-26T01:39:25.345341+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-26T01:39:25.345560+0000","last_change":"2023-10-26T01:39:25.345280+0000","last_active":"2023-10-26T01:39:25.345560+0000","last_peered":"2023-10-26T01:39:25.345560+0000","last_clean":"2023-10-26T01:39:25.345560+0000","last_became_active":"2023-10-26T01:39:25.344841+0000","last_became_peered":"2023-10-26T01:39:25.344841+0000","last_unstale":"2023-10-26T01:39:25.345560+0000","last_undegraded":"2023-10-26T01:39:25.345560+0000","last_fullsized":"2023-10-26T01:39:25.345560+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:01:31.862431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.339971+0000","last_change":"2023-10-26T01:39:25.339971+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.339971+0000","last_undegraded":"2023-10-26T01:39:25.339971+0000","last_fullsized":"2023-10-26T01:39:25.339971+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-26T01:39:25.355216+0000","last_change":"2023-10-26T01:39:25.355147+0000","last_active":"2023-10-26T01:39:25.355216+0000","last_peered":"2023-10-26T01:39:25.355216+0000","last_clean":"2023-10-26T01:39:25.355216+0000","last_became_active":"2023-10-26T01:39:25.354564+0000","last_became_peered":"2023-10-26T01:39:25.354564+0000","last_unstale":"2023-10-26T01:39:25.355216+0000","last_undegraded":"2023-10-26T01:39:25.355216+0000","last_fullsized":"2023-10-26T01:39:25.355216+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:08:33.196097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.340967+0000","last_change":"2023-10-26T01:39:25.340967+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.340967+0000","last_undegraded":"2023-10-26T01:39:25.340967+0000","last_fullsized":"2023-10-26T01:39:25.340967+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-26T01:39:25.354084+0000","last_change":"2023-10-26T01:39:25.353938+0000","last_active":"2023-10-26T01:39:25.354084+0000","last_peered":"2023-10-26T01:39:25.354084+0000","last_clean":"2023-10-26T01:39:25.354084+0000","last_became_active":"2023-10-26T01:39:25.353704+0000","last_became_peered":"2023-10-26T01:39:25.353704+0000","last_unstale":"2023-10-26T01:39:25.354084+0000","last_undegraded":"2023-10-26T01:39:25.354084+0000","last_fullsized":"2023-10-26T01:39:25.354084+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:28:46.712484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.344566+0000","last_change":"2023-10-26T01:39:25.344566+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.344566+0000","last_undegraded":"2023-10-26T01:39:25.344566+0000","last_fullsized":"2023-10-26T01:39:25.344566+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":11,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-26T01:39:26.333356+0000","last_change":"2023-10-26T01:39:25.353364+0000","last_active":"2023-10-26T01:39:26.333356+0000","last_peered":"2023-10-26T01:39:26.333356+0000","last_clean":"2023-10-26T01:39:26.333356+0000","last_became_active":"2023-10-26T01:39:25.353129+0000","last_became_peered":"2023-10-26T01:39:25.353129+0000","last_unstale":"2023-10-26T01:39:26.333356+0000","last_undegraded":"2023-10-26T01:39:26.333356+0000","last_fullsized":"2023-10-26T01:39:26.333356+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:07:24.731619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.332527+0000","last_change":"2023-10-26T01:39:25.332527+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.332527+0000","last_undegraded":"2023-10-26T01:39:25.332527+0000","last_fullsized":"2023-10-26T01:39:25.332527+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-26T01:39:25.353094+0000","last_change":"2023-10-26T01:39:25.352244+0000","last_active":"2023-10-26T01:39:25.353094+0000","last_peered":"2023-10-26T01:39:25.353094+0000","last_clean":"2023-10-26T01:39:25.353094+0000","last_became_active":"2023-10-26T01:39:25.351989+0000","last_became_peered":"2023-10-26T01:39:25.351989+0000","last_unstale":"2023-10-26T01:39:25.353094+0000","last_undegraded":"2023-10-26T01:39:25.353094+0000","last_fullsized":"2023-10-26T01:39:25.353094+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:56:13.935683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.333835+0000","last_change":"2023-10-26T01:39:25.333835+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.333835+0000","last_undegraded":"2023-10-26T01:39:25.333835+0000","last_fullsized":"2023-10-26T01:39:25.333835+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-26T01:39:25.354938+0000","last_change":"2023-10-26T01:39:25.354828+0000","last_active":"2023-10-26T01:39:25.354938+0000","last_peered":"2023-10-26T01:39:25.354938+0000","last_clean":"2023-10-26T01:39:25.354938+0000","last_became_active":"2023-10-26T01:39:25.354185+0000","last_became_peered":"2023-10-26T01:39:25.354185+0000","last_unstale":"2023-10-26T01:39:25.354938+0000","last_undegraded":"2023-10-26T01:39:25.354938+0000","last_fullsized":"2023-10-26T01:39:25.354938+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:32:50.379779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.344701+0000","last_change":"2023-10-26T01:39:25.344701+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.344701+0000","last_undegraded":"2023-10-26T01:39:25.344701+0000","last_fullsized":"2023-10-26T01:39:25.344701+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-26T01:39:25.353183+0000","last_change":"2023-10-26T01:39:25.350711+0000","last_active":"2023-10-26T01:39:25.353183+0000","last_peered":"2023-10-26T01:39:25.353183+0000","last_clean":"2023-10-26T01:39:25.353183+0000","last_became_active":"2023-10-26T01:39:25.349869+0000","last_became_peered":"2023-10-26T01:39:25.349869+0000","last_unstale":"2023-10-26T01:39:25.353183+0000","last_undegraded":"2023-10-26T01:39:25.353183+0000","last_fullsized":"2023-10-26T01:39:25.353183+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:30:24.885507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.340874+0000","last_change":"2023-10-26T01:39:25.340874+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.340874+0000","last_undegraded":"2023-10-26T01:39:25.340874+0000","last_fullsized":"2023-10-26T01:39:25.340874+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-26T01:39:25.342677+0000","last_change":"2023-10-26T01:39:25.342525+0000","last_active":"2023-10-26T01:39:25.342677+0000","last_peered":"2023-10-26T01:39:25.342677+0000","last_clean":"2023-10-26T01:39:25.342677+0000","last_became_active":"2023-10-26T01:39:25.341183+0000","last_became_peered":"2023-10-26T01:39:25.341183+0000","last_unstale":"2023-10-26T01:39:25.342677+0000","last_undegraded":"2023-10-26T01:39:25.342677+0000","last_fullsized":"2023-10-26T01:39:25.342677+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:07:38.898672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.344820+0000","last_change":"2023-10-26T01:39:25.344820+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.344820+0000","last_undegraded":"2023-10-26T01:39:25.344820+0000","last_fullsized":"2023-10-26T01:39:25.344820+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-26T01:39:25.355255+0000","last_change":"2023-10-26T01:39:25.355183+0000","last_active":"2023-10-26T01:39:25.355255+0000","last_peered":"2023-10-26T01:39:25.355255+0000","last_clean":"2023-10-26T01:39:25.355255+0000","last_became_active":"2023-10-26T01:39:25.354751+0000","last_became_peered":"2023-10-26T01:39:25.354751+0000","last_unstale":"2023-10-26T01:39:25.355255+0000","last_undegraded":"2023-10-26T01:39:25.355255+0000","last_fullsized":"2023-10-26T01:39:25.355255+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:39:10.557327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.343890+0000","last_change":"2023-10-26T01:39:25.343890+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.343890+0000","last_undegraded":"2023-10-26T01:39:25.343890+0000","last_fullsized":"2023-10-26T01:39:25.343890+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-26T01:39:25.350561+0000","last_change":"2023-10-26T01:39:25.350279+0000","last_active":"2023-10-26T01:39:25.350561+0000","last_peered":"2023-10-26T01:39:25.350561+0000","last_clean":"2023-10-26T01:39:25.350561+0000","last_became_active":"2023-10-26T01:39:25.349301+0000","last_became_peered":"2023-10-26T01:39:25.349301+0000","last_unstale":"2023-10-26T01:39:25.350561+0000","last_undegraded":"2023-10-26T01:39:25.350561+0000","last_fullsized":"2023-10-26T01:39:25.350561+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:02:52.785225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.342089+0000","last_change":"2023-10-26T01:39:25.342089+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.342089+0000","last_undegraded":"2023-10-26T01:39:25.342089+0000","last_fullsized":"2023-10-26T01:39:25.342089+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-26T01:39:25.355575+0000","last_change":"2023-10-26T01:39:25.355508+0000","last_active":"2023-10-26T01:39:25.355575+0000","last_peered":"2023-10-26T01:39:25.355575+0000","last_clean":"2023-10-26T01:39:25.355575+0000","last_became_active":"2023-10-26T01:39:25.355254+0000","last_became_peered":"2023-10-26T01:39:25.355254+0000","last_unstale":"2023-10-26T01:39:25.355575+0000","last_undegraded":"2023-10-26T01:39:25.355575+0000","last_fullsized":"2023-10-26T01:39:25.355575+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:12:53.520877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":9,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-26T01:39:26.354933+0000","last_change":"2023-10-26T01:39:26.354034+0000","last_active":"2023-10-26T01:39:26.354933+0000","last_peered":"2023-10-26T01:39:26.354933+0000","last_clean":"2023-10-26T01:39:26.354933+0000","last_became_active":"2023-10-26T01:39:26.353811+0000","last_became_peered":"2023-10-26T01:39:26.353811+0000","last_unstale":"2023-10-26T01:39:26.354933+0000","last_undegraded":"2023-10-26T01:39:26.354933+0000","last_fullsized":"2023-10-26T01:39:26.354933+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:53:55.836062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-26T01:39:25.350542+0000","last_change":"2023-10-26T01:39:25.348929+0000","last_active":"2023-10-26T01:39:25.350542+0000","last_peered":"2023-10-26T01:39:25.350542+0000","last_clean":"2023-10-26T01:39:25.350542+0000","last_became_active":"2023-10-26T01:39:25.348637+0000","last_became_peered":"2023-10-26T01:39:25.348637+0000","last_unstale":"2023-10-26T01:39:25.350542+0000","last_undegraded":"2023-10-26T01:39:25.350542+0000","last_fullsized":"2023-10-26T01:39:25.350542+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:16:32.808798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.340936+0000","last_change":"2023-10-26T01:39:25.340936+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.340936+0000","last_undegraded":"2023-10-26T01:39:25.340936+0000","last_fullsized":"2023-10-26T01:39:25.340936+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-26T01:39:25.353113+0000","last_change":"2023-10-26T01:39:25.352675+0000","last_active":"2023-10-26T01:39:25.353113+0000","last_peered":"2023-10-26T01:39:25.353113+0000","last_clean":"2023-10-26T01:39:25.353113+0000","last_became_active":"2023-10-26T01:39:25.352188+0000","last_became_peered":"2023-10-26T01:39:25.352188+0000","last_unstale":"2023-10-26T01:39:25.353113+0000","last_undegraded":"2023-10-26T01:39:25.353113+0000","last_fullsized":"2023-10-26T01:39:25.353113+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:07:38.898672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.346409+0000","last_change":"2023-10-26T01:39:25.346409+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.346409+0000","last_undegraded":"2023-10-26T01:39:25.346409+0000","last_fullsized":"2023-10-26T01:39:25.346409+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-26T01:39:25.354022+0000","last_change":"2023-10-26T01:39:25.353938+0000","last_active":"2023-10-26T01:39:25.354022+0000","last_peered":"2023-10-26T01:39:25.354022+0000","last_clean":"2023-10-26T01:39:25.354022+0000","last_became_active":"2023-10-26T01:39:25.353665+0000","last_became_peered":"2023-10-26T01:39:25.353665+0000","last_unstale":"2023-10-26T01:39:25.354022+0000","last_undegraded":"2023-10-26T01:39:25.354022+0000","last_fullsized":"2023-10-26T01:39:25.354022+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:32:20.357196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.342827+0000","last_change":"2023-10-26T01:39:25.342827+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.342827+0000","last_undegraded":"2023-10-26T01:39:25.342827+0000","last_fullsized":"2023-10-26T01:39:25.342827+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":11,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-26T01:39:26.328128+0000","last_change":"2023-10-26T01:39:25.355042+0000","last_active":"2023-10-26T01:39:26.328128+0000","last_peered":"2023-10-26T01:39:26.328128+0000","last_clean":"2023-10-26T01:39:26.328128+0000","last_became_active":"2023-10-26T01:39:25.354798+0000","last_became_peered":"2023-10-26T01:39:25.354798+0000","last_unstale":"2023-10-26T01:39:26.328128+0000","last_undegraded":"2023-10-26T01:39:26.328128+0000","last_fullsized":"2023-10-26T01:39:26.328128+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:34:40.255257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.340172+0000","last_change":"2023-10-26T01:39:25.340172+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.340172+0000","last_undegraded":"2023-10-26T01:39:25.340172+0000","last_fullsized":"2023-10-26T01:39:25.340172+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":11,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-26T01:39:26.342348+0000","last_change":"2023-10-26T01:39:25.355043+0000","last_active":"2023-10-26T01:39:26.342348+0000","last_peered":"2023-10-26T01:39:26.342348+0000","last_clean":"2023-10-26T01:39:26.342348+0000","last_became_active":"2023-10-26T01:39:25.354518+0000","last_became_peered":"2023-10-26T01:39:25.354518+0000","last_unstale":"2023-10-26T01:39:26.342348+0000","last_undegraded":"2023-10-26T01:39:26.342348+0000","last_fullsized":"2023-10-26T01:39:26.342348+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:37:06.664661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.335577+0000","last_change":"2023-10-26T01:39:25.335577+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.335577+0000","last_undegraded":"2023-10-26T01:39:25.335577+0000","last_fullsized":"2023-10-26T01:39:25.335577+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-26T01:39:25.355626+0000","last_change":"2023-10-26T01:39:25.355568+0000","last_active":"2023-10-26T01:39:25.355626+0000","last_peered":"2023-10-26T01:39:25.355626+0000","last_clean":"2023-10-26T01:39:25.355626+0000","last_became_active":"2023-10-26T01:39:25.354482+0000","last_became_peered":"2023-10-26T01:39:25.354482+0000","last_unstale":"2023-10-26T01:39:25.355626+0000","last_undegraded":"2023-10-26T01:39:25.355626+0000","last_fullsized":"2023-10-26T01:39:25.355626+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:10:56.447860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.333593+0000","last_change":"2023-10-26T01:39:25.333593+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.333593+0000","last_undegraded":"2023-10-26T01:39:25.333593+0000","last_fullsized":"2023-10-26T01:39:25.333593+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-26T01:39:25.350624+0000","last_change":"2023-10-26T01:39:25.350493+0000","last_active":"2023-10-26T01:39:25.350624+0000","last_peered":"2023-10-26T01:39:25.350624+0000","last_clean":"2023-10-26T01:39:25.350624+0000","last_became_active":"2023-10-26T01:39:25.349758+0000","last_became_peered":"2023-10-26T01:39:25.349758+0000","last_unstale":"2023-10-26T01:39:25.350624+0000","last_undegraded":"2023-10-26T01:39:25.350624+0000","last_fullsized":"2023-10-26T01:39:25.350624+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:23:21.057813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.339995+0000","last_change":"2023-10-26T01:39:25.339995+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.339995+0000","last_undegraded":"2023-10-26T01:39:25.339995+0000","last_fullsized":"2023-10-26T01:39:25.339995+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-26T01:39:25.347943+0000","last_change":"2023-10-26T01:39:25.347830+0000","last_active":"2023-10-26T01:39:25.347943+0000","last_peered":"2023-10-26T01:39:25.347943+0000","last_clean":"2023-10-26T01:39:25.347943+0000","last_became_active":"2023-10-26T01:39:25.346994+0000","last_became_peered":"2023-10-26T01:39:25.346994+0000","last_unstale":"2023-10-26T01:39:25.347943+0000","last_undegraded":"2023-10-26T01:39:25.347943+0000","last_fullsized":"2023-10-26T01:39:25.347943+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:16:32.808798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.340256+0000","last_change":"2023-10-26T01:39:25.340256+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.340256+0000","last_undegraded":"2023-10-26T01:39:25.340256+0000","last_fullsized":"2023-10-26T01:39:25.340256+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-26T01:39:25.354467+0000","last_change":"2023-10-26T01:39:25.354096+0000","last_active":"2023-10-26T01:39:25.354467+0000","last_peered":"2023-10-26T01:39:25.354467+0000","last_clean":"2023-10-26T01:39:25.354467+0000","last_became_active":"2023-10-26T01:39:25.353857+0000","last_became_peered":"2023-10-26T01:39:25.353857+0000","last_unstale":"2023-10-26T01:39:25.354467+0000","last_undegraded":"2023-10-26T01:39:25.354467+0000","last_fullsized":"2023-10-26T01:39:25.354467+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:28:46.712484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.335457+0000","last_change":"2023-10-26T01:39:25.335457+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.335457+0000","last_undegraded":"2023-10-26T01:39:25.335457+0000","last_fullsized":"2023-10-26T01:39:25.335457+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-26T01:39:25.350001+0000","last_change":"2023-10-26T01:39:25.349514+0000","last_active":"2023-10-26T01:39:25.350001+0000","last_peered":"2023-10-26T01:39:25.350001+0000","last_clean":"2023-10-26T01:39:25.350001+0000","last_became_active":"2023-10-26T01:39:25.349279+0000","last_became_peered":"2023-10-26T01:39:25.349279+0000","last_unstale":"2023-10-26T01:39:25.350001+0000","last_undegraded":"2023-10-26T01:39:25.350001+0000","last_fullsized":"2023-10-26T01:39:25.350001+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:51:35.449470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.337282+0000","last_change":"2023-10-26T01:39:25.337282+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.337282+0000","last_undegraded":"2023-10-26T01:39:25.337282+0000","last_fullsized":"2023-10-26T01:39:25.337282+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-26T01:39:25.348101+0000","last_change":"2023-10-26T01:39:25.347759+0000","last_active":"2023-10-26T01:39:25.348101+0000","last_peered":"2023-10-26T01:39:25.348101+0000","last_clean":"2023-10-26T01:39:25.348101+0000","last_became_active":"2023-10-26T01:39:25.346564+0000","last_became_peered":"2023-10-26T01:39:25.346564+0000","last_unstale":"2023-10-26T01:39:25.348101+0000","last_undegraded":"2023-10-26T01:39:25.348101+0000","last_fullsized":"2023-10-26T01:39:25.348101+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:12:27.843055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.343816+0000","last_change":"2023-10-26T01:39:25.343816+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.343816+0000","last_undegraded":"2023-10-26T01:39:25.343816+0000","last_fullsized":"2023-10-26T01:39:25.343816+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-26T01:39:25.344486+0000","last_change":"2023-10-26T01:39:25.343413+0000","last_active":"2023-10-26T01:39:25.344486+0000","last_peered":"2023-10-26T01:39:25.344486+0000","last_clean":"2023-10-26T01:39:25.344486+0000","last_became_active":"2023-10-26T01:39:25.342949+0000","last_became_peered":"2023-10-26T01:39:25.342949+0000","last_unstale":"2023-10-26T01:39:25.344486+0000","last_undegraded":"2023-10-26T01:39:25.344486+0000","last_fullsized":"2023-10-26T01:39:25.344486+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:11:10.758066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":9,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-26T01:39:26.336802+0000","last_change":"2023-10-26T01:39:26.335564+0000","last_active":"2023-10-26T01:39:26.336802+0000","last_peered":"2023-10-26T01:39:26.336802+0000","last_clean":"2023-10-26T01:39:26.336802+0000","last_became_active":"2023-10-26T01:39:26.335000+0000","last_became_peered":"2023-10-26T01:39:26.335000+0000","last_unstale":"2023-10-26T01:39:26.336802+0000","last_undegraded":"2023-10-26T01:39:26.336802+0000","last_fullsized":"2023-10-26T01:39:26.336802+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:05:15.173045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":11,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-26T01:39:26.338413+0000","last_change":"2023-10-26T01:39:25.356190+0000","last_active":"2023-10-26T01:39:26.338413+0000","last_peered":"2023-10-26T01:39:26.338413+0000","last_clean":"2023-10-26T01:39:26.338413+0000","last_became_active":"2023-10-26T01:39:25.355791+0000","last_became_peered":"2023-10-26T01:39:25.355791+0000","last_unstale":"2023-10-26T01:39:26.338413+0000","last_undegraded":"2023-10-26T01:39:26.338413+0000","last_fullsized":"2023-10-26T01:39:26.338413+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:24:21.026112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.339461+0000","last_change":"2023-10-26T01:39:25.339461+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.339461+0000","last_undegraded":"2023-10-26T01:39:25.339461+0000","last_fullsized":"2023-10-26T01:39:25.339461+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-26T01:39:25.349964+0000","last_change":"2023-10-26T01:39:25.347950+0000","last_active":"2023-10-26T01:39:25.349964+0000","last_peered":"2023-10-26T01:39:25.349964+0000","last_clean":"2023-10-26T01:39:25.349964+0000","last_became_active":"2023-10-26T01:39:25.347416+0000","last_became_peered":"2023-10-26T01:39:25.347416+0000","last_unstale":"2023-10-26T01:39:25.349964+0000","last_undegraded":"2023-10-26T01:39:25.349964+0000","last_fullsized":"2023-10-26T01:39:25.349964+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:22:18.250771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2023-10-26T01:39:25.334115+0000","last_change":"2023-10-26T01:39:25.334115+0000","last_active":"2023-10-26T01:39:25.309143+0000","last_peered":"2023-10-26T01:39:25.309143+0000","last_clean":"2023-10-26T01:39:25.309143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T01:39:25.334115+0000","last_undegraded":"2023-10-26T01:39:25.334115+0000","last_fullsized":"2023-10-26T01:39:25.334115+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-26T01:39:25.353367+0000","last_change":"2023-10-26T01:39:25.352322+0000","last_active":"2023-10-26T01:39:25.353367+0000","last_peered":"2023-10-26T01:39:25.353367+0000","last_clean":"2023-10-26T01:39:25.353367+0000","last_became_active":"2023-10-26T01:39:25.351287+0000","last_became_peered":"2023-10-26T01:39:25.351287+0000","last_unstale":"2023-10-26T01:39:25.353367+0000","last_undegraded":"2023-10-26T01:39:25.353367+0000","last_fullsized":"2023-10-26T01:39:25.353367+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:03:57.007851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":292,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-26T01:39:24.316788+0000","last_change":"2023-10-26T01:30:34.883484+0000","last_active":"2023-10-26T01:39:24.316788+0000","last_peered":"2023-10-26T01:39:24.316788+0000","last_clean":"2023-10-26T01:39:24.316788+0000","last_became_active":"2023-10-26T01:30:34.883071+0000","last_became_peered":"2023-10-26T01:30:34.883071+0000","last_unstale":"2023-10-26T01:39:24.316788+0000","last_undegraded":"2023-10-26T01:39:24.316788+0000","last_fullsized":"2023-10-26T01:39:24.316788+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:32.890096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:32.890096+0000","last_clean_scrub_stamp":"2023-10-26T01:30:32.890096+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:26:31.879452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":14,"seq":60129542254,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":817,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194495,"statfs":{"total":200000000,"available":199163256,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:38:51 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.443,"5min":0.35599999999999998,"15min":0.33000000000000002},"max":{"1min":1.2549999999999999,"5min":1.6319999999999999,"15min":2.2549999999999999},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.92300000000000004,"15min":0.89300000000000002},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":1.752,"5min":1.752,"15min":1.8200000000000001},"last":0.67800000000000005}]},{"osd":1,"last update":"Thu Oct 26 01:38:51 2023","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.88400000000000001,"15min":0.92800000000000005},"min":{"1min":0.39400000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.383,"5min":1.444,"15min":4.4340000000000002},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.996,"5min":0.99199999999999999,"15min":0.995},"min":{"1min":0.56200000000000006,"5min":0.46700000000000003,"15min":0.374},"max":{"1min":1.353,"5min":1.538,"15min":2.3330000000000002},"last":0.70999999999999996}]},{"osd":2,"last update":"Thu Oct 26 01:38:51 2023","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":1.028,"15min":1.0700000000000001},"min":{"1min":0.54500000000000004,"5min":0.40999999999999998,"15min":0.39900000000000002},"max":{"1min":1.4279999999999999,"5min":1.927,"15min":4.2800000000000002},"last":0.79400000000000004},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.1479999999999999,"15min":1.0980000000000001},"min":{"1min":0.53300000000000003,"5min":0.495,"15min":0.49399999999999999},"max":{"1min":1.639,"5min":1.8680000000000001,"15min":2.302},"last":0.94399999999999995}]},{"osd":4,"last update":"Thu Oct 26 01:38:51 2023","interfaces":[{"interface":"back","average":{"1min":1.266,"5min":1.23,"15min":1.175},"min":{"1min":0.98199999999999998,"5min":0.755,"15min":0.66400000000000003},"max":{"1min":1.732,"5min":1.732,"15min":2.1070000000000002},"last":1.375},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.353,"15min":1.3859999999999999},"min":{"1min":0.91900000000000004,"5min":0.81599999999999995,"15min":0.79900000000000004},"max":{"1min":1.6220000000000001,"5min":2.4239999999999999,"15min":4.8250000000000002},"last":1.4019999999999999}]},{"osd":5,"last update":"Thu Oct 26 01:38:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.2929999999999999,"15min":1.3280000000000001},"min":{"1min":0.84899999999999998,"5min":0.76400000000000001,"15min":0.72499999999999998},"max":{"1min":1.6899999999999999,"5min":2.3380000000000001,"15min":4.9080000000000004},"last":1.2709999999999999},{"interface":"front","average":{"1min":1.331,"5min":1.2969999999999999,"15min":1.2789999999999999},"min":{"1min":0.82699999999999996,"5min":0.79900000000000004,"15min":0.79600000000000004},"max":{"1min":1.573,"5min":2.0310000000000001,"15min":2.8980000000000001},"last":1.385}]},{"osd":6,"last update":"Thu Oct 26 01:38:51 2023","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.351,"15min":1.329},"min":{"1min":0.75900000000000001,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.694,"5min":2.1499999999999999,"15min":2.8530000000000002},"last":1.448},{"interface":"front","average":{"1min":1.4159999999999999,"5min":1.3160000000000001,"15min":1.2769999999999999},"min":{"1min":0.84399999999999997,"5min":0.81699999999999995,"15min":0.73099999999999998},"max":{"1min":1.6899999999999999,"5min":2.048,"15min":2.181},"last":1.48}]},{"osd":7,"last update":"Thu Oct 26 01:38:51 2023","interfaces":[{"interface":"back","average":{"1min":1.4370000000000001,"5min":1.3660000000000001,"15min":1.3069999999999999},"min":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.71999999999999997},"max":{"1min":1.7589999999999999,"5min":1.958,"15min":2.0680000000000001},"last":1.3340000000000001},{"interface":"front","average":{"1min":1.385,"5min":1.393,"15min":1.411},"min":{"1min":0.73299999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.702,"5min":2.2370000000000001,"15min":4.8730000000000002},"last":1.595}]}]},{"osd":7,"up_from":13,"seq":55834574957,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1380,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193931,"statfs":{"total":200000000,"available":198586098,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:38:50 2023","interfaces":[{"interface":"back","average":{"1min":1.127,"5min":1.1379999999999999,"15min":1.097},"min":{"1min":0.68700000000000006,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.633,"5min":1.7,"15min":1.9410000000000001},"last":1.1879999999999999},{"interface":"front","average":{"1min":1.095,"5min":1.1100000000000001,"15min":1.0840000000000001},"min":{"1min":0.67300000000000004,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.6970000000000001,"5min":2.6779999999999999,"15min":2.6779999999999999},"last":1.294}]},{"osd":1,"last update":"Thu Oct 26 01:38:50 2023","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":1.1699999999999999,"15min":1.1020000000000001},"min":{"1min":0.82899999999999996,"5min":0.65800000000000003,"15min":0.65200000000000002},"max":{"1min":1.484,"5min":2.8260000000000001,"15min":2.8260000000000001},"last":0.879},{"interface":"front","average":{"1min":1.177,"5min":1.2,"15min":1.194},"min":{"1min":0.77200000000000002,"5min":0.66500000000000004,"15min":0.61199999999999999},"max":{"1min":1.76,"5min":3.2730000000000001,"15min":3.2730000000000001},"last":1.1240000000000001}]},{"osd":2,"last update":"Thu Oct 26 01:38:50 2023","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.2110000000000001,"15min":1.1299999999999999},"min":{"1min":0.873,"5min":0.69099999999999995,"15min":0.58499999999999996},"max":{"1min":1.851,"5min":2.218,"15min":2.218},"last":0.93400000000000005},{"interface":"front","average":{"1min":1.256,"5min":1.296,"15min":1.2430000000000001},"min":{"1min":0.82299999999999995,"5min":0.79400000000000004,"15min":0.73099999999999998},"max":{"1min":1.7350000000000001,"5min":3.0510000000000002,"15min":3.0510000000000002},"last":1.0229999999999999}]},{"osd":3,"last update":"Thu Oct 26 01:38:50 2023","interfaces":[{"interface":"back","average":{"1min":1.3120000000000001,"5min":1.296,"15min":1.2230000000000001},"min":{"1min":0.79500000000000004,"5min":0.79000000000000004,"15min":0.72099999999999997},"max":{"1min":1.8169999999999999,"5min":1.9830000000000001,"15min":1.9830000000000001},"last":0.79800000000000004},{"interface":"front","average":{"1min":1.224,"5min":1.373,"15min":1.3640000000000001},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.65700000000000003},"max":{"1min":1.5129999999999999,"5min":5.5039999999999996,"15min":5.5039999999999996},"last":1.651}]},{"osd":4,"last update":"Thu Oct 26 01:38:50 2023","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.371,"15min":1.351},"min":{"1min":0.81799999999999995,"5min":0.77300000000000002,"15min":0.76100000000000001},"max":{"1min":1.9339999999999999,"5min":2.77,"15min":2.77},"last":2.0390000000000001},{"interface":"front","average":{"1min":1.288,"5min":1.3919999999999999,"15min":1.393},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.6479999999999999,"5min":3.1419999999999999,"15min":3.1419999999999999},"last":2.536}]},{"osd":5,"last update":"Thu Oct 26 01:38:50 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.4410000000000001,"15min":1.4199999999999999},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.7809999999999999,"5min":5.734,"15min":5.734},"last":1.363},{"interface":"front","average":{"1min":1.4450000000000001,"5min":1.514,"15min":1.478},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.51900000000000002},"max":{"1min":2,"5min":2.8980000000000001,"15min":2.8980000000000001},"last":1.0920000000000001}]},{"osd":6,"last update":"Thu Oct 26 01:38:50 2023","interfaces":[{"interface":"back","average":{"1min":1.452,"5min":1.5800000000000001,"15min":1.8089999999999999},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.47399999999999998},"max":{"1min":1.8049999999999999,"5min":5.6340000000000003,"15min":11.577999999999999},"last":1.7769999999999999},{"interface":"front","average":{"1min":1.613,"5min":1.6719999999999999,"15min":1.8360000000000001},"min":{"1min":1.0640000000000001,"5min":0.91800000000000004,"15min":0.55100000000000005},"max":{"1min":2.0529999999999999,"5min":5.0919999999999996,"15min":10.704000000000001},"last":1.8779999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574957,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":803,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194508,"statfs":{"total":200000000,"available":199177080,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:38:46 2023","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.141,"15min":1.1519999999999999},"min":{"1min":0.66600000000000004,"5min":0.61699999999999999,"15min":0.60399999999999998},"max":{"1min":2.7370000000000001,"5min":2.7370000000000001,"15min":2.7370000000000001},"last":1.0760000000000001},{"interface":"front","average":{"1min":1.137,"5min":1.173,"15min":1.123},"min":{"1min":0.61699999999999999,"5min":0.61399999999999999,"15min":0.59199999999999997},"max":{"1min":4.4779999999999998,"5min":4.4779999999999998,"15min":4.4779999999999998},"last":1.0189999999999999}]},{"osd":1,"last update":"Thu Oct 26 01:38:46 2023","interfaces":[{"interface":"back","average":{"1min":1.3919999999999999,"5min":1.2729999999999999,"15min":1.1930000000000001},"min":{"1min":0.77100000000000002,"5min":0.72299999999999998,"15min":0.68300000000000005},"max":{"1min":5.5359999999999996,"5min":5.5359999999999996,"15min":5.5359999999999996},"last":1.246},{"interface":"front","average":{"1min":1.228,"5min":1.2190000000000001,"15min":1.204},"min":{"1min":0.74399999999999999,"5min":0.67900000000000005,"15min":0.64000000000000001},"max":{"1min":4.3579999999999997,"5min":4.3579999999999997,"15min":4.3579999999999997},"last":1.1279999999999999}]},{"osd":2,"last update":"Thu Oct 26 01:38:46 2023","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.2030000000000001,"15min":1.107},"min":{"1min":0.76300000000000001,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":2.367,"5min":3.7879999999999998,"15min":3.7879999999999998},"last":1.175},{"interface":"front","average":{"1min":1.371,"5min":1.3420000000000001,"15min":1.276},"min":{"1min":0.91200000000000003,"5min":0.75800000000000001,"15min":0.68899999999999995},"max":{"1min":3.0470000000000002,"5min":3.0470000000000002,"15min":3.0470000000000002},"last":1.4430000000000001}]},{"osd":3,"last update":"Thu Oct 26 01:38:46 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.331,"15min":1.22},"min":{"1min":0.76300000000000001,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":3.266,"5min":4.4560000000000004,"15min":4.4560000000000004},"last":1.601},{"interface":"front","average":{"1min":1.365,"5min":1.3500000000000001,"15min":1.3280000000000001},"min":{"1min":0.86399999999999999,"5min":0.78600000000000003,"15min":0.69499999999999995},"max":{"1min":2.6909999999999998,"5min":2.6909999999999998,"15min":2.6909999999999998},"last":1.55}]},{"osd":4,"last update":"Thu Oct 26 01:38:46 2023","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.24,"15min":1.22},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.48399999999999999},"max":{"1min":1.7410000000000001,"5min":2.9489999999999998,"15min":2.9489999999999998},"last":1.395},{"interface":"front","average":{"1min":1.3180000000000001,"5min":1.421,"15min":1.319},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.59299999999999997},"max":{"1min":1.698,"5min":6.609,"15min":6.609},"last":1.494}]},{"osd":5,"last update":"Thu Oct 26 01:38:46 2023","interfaces":[{"interface":"back","average":{"1min":1.464,"5min":1.5389999999999999,"15min":1.452},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.7949999999999999,"5min":5.4370000000000003,"15min":5.4370000000000003},"last":1.671},{"interface":"front","average":{"1min":1.492,"5min":1.4530000000000001,"15min":1.419},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":2.7759999999999998,"5min":2.7759999999999998,"15min":2.7759999999999998},"last":1.6459999999999999}]},{"osd":7,"last update":"Thu Oct 26 01:38:46 2023","interfaces":[{"interface":"back","average":{"1min":1.2869999999999999,"5min":1.484,"15min":1.405},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.54100000000000004},"max":{"1min":1.8200000000000001,"5min":11.804,"15min":11.804},"last":1.698},{"interface":"front","average":{"1min":1.577,"5min":1.6639999999999999,"15min":1.52},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":3.7650000000000001,"5min":11.541,"15min":11.541},"last":1.728}]}]},{"osd":5,"up_from":13,"seq":55834574957,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":810,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194502,"statfs":{"total":200000000,"available":199170191,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:38:48 2023","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.2310000000000001,"15min":1.147},"min":{"1min":0.69699999999999995,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":3.8210000000000002,"5min":3.8210000000000002,"15min":3.8210000000000002},"last":1.5549999999999999},{"interface":"front","average":{"1min":1.3480000000000001,"5min":1.302,"15min":1.1970000000000001},"min":{"1min":0.92600000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":2.9889999999999999,"5min":2.9889999999999999,"15min":2.9889999999999999},"last":1.48}]},{"osd":1,"last update":"Thu Oct 26 01:38:48 2023","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.1319999999999999,"15min":1.0940000000000001},"min":{"1min":0.83599999999999997,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.8120000000000001,"5min":1.8120000000000001,"15min":1.8120000000000001},"last":1.514},{"interface":"front","average":{"1min":1.272,"5min":1.1659999999999999,"15min":1.109},"min":{"1min":0.76700000000000002,"5min":0.629,"15min":0.60599999999999998},"max":{"1min":2.4340000000000002,"5min":2.4340000000000002,"15min":2.4340000000000002},"last":0.98199999999999998}]},{"osd":2,"last update":"Thu Oct 26 01:38:48 2023","interfaces":[{"interface":"back","average":{"1min":1.2130000000000001,"5min":1.206,"15min":1.1679999999999999},"min":{"1min":0.79700000000000004,"5min":0.71599999999999997,"15min":0.69299999999999995},"max":{"1min":1.5920000000000001,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":1.2270000000000001},{"interface":"front","average":{"1min":1.228,"5min":1.228,"15min":1.1850000000000001},"min":{"1min":0.91600000000000004,"5min":0.78400000000000003,"15min":0.63900000000000001},"max":{"1min":1.5900000000000001,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":1.6459999999999999}]},{"osd":3,"last update":"Thu Oct 26 01:38:48 2023","interfaces":[{"interface":"back","average":{"1min":1.389,"5min":1.2829999999999999,"15min":1.25},"min":{"1min":1.0009999999999999,"5min":0.72699999999999998,"15min":0.72299999999999998},"max":{"1min":1.7,"5min":1.7,"15min":1.835},"last":1.101},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.298,"15min":1.29},"min":{"1min":0.91300000000000003,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.52,"5min":1.788,"15min":2.9249999999999998},"last":0.74299999999999999}]},{"osd":4,"last update":"Thu Oct 26 01:38:48 2023","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":1.256,"15min":1.214},"min":{"1min":0.75,"5min":0.59599999999999997,"15min":0.49099999999999999},"max":{"1min":2.7770000000000001,"5min":2.7770000000000001,"15min":2.7770000000000001},"last":1.681},{"interface":"front","average":{"1min":1.3180000000000001,"5min":1.3979999999999999,"15min":1.355},"min":{"1min":0.67500000000000004,"5min":0.57299999999999995,"15min":0.56200000000000006},"max":{"1min":1.776,"5min":1.7849999999999999,"15min":1.867},"last":0.86699999999999999}]},{"osd":6,"last update":"Thu Oct 26 01:38:48 2023","interfaces":[{"interface":"back","average":{"1min":1.377,"5min":1.393,"15min":1.3300000000000001},"min":{"1min":0.79000000000000004,"5min":0.67500000000000004,"15min":0.51900000000000002},"max":{"1min":1.956,"5min":2.8250000000000002,"15min":2.8250000000000002},"last":1.4339999999999999},{"interface":"front","average":{"1min":1.4590000000000001,"5min":1.409,"15min":1.3400000000000001},"min":{"1min":0.89300000000000002,"5min":0.69499999999999995,"15min":0.61799999999999999},"max":{"1min":2.4830000000000001,"5min":2.4830000000000001,"15min":2.4830000000000001},"last":1.339}]},{"osd":7,"last update":"Thu Oct 26 01:38:48 2023","interfaces":[{"interface":"back","average":{"1min":1.6000000000000001,"5min":1.534,"15min":1.4379999999999999},"min":{"1min":0.81499999999999995,"5min":0.71099999999999997,"15min":0.61699999999999999},"max":{"1min":3.621,"5min":3.621,"15min":3.621},"last":1.609},{"interface":"front","average":{"1min":1.603,"5min":1.542,"15min":1.4470000000000001},"min":{"1min":0.82899999999999996,"5min":0.73899999999999999,"15min":0.59499999999999997},"max":{"1min":2.8010000000000002,"5min":2.8010000000000002,"15min":2.8010000000000002},"last":1.7889999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574957,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":804,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194508,"statfs":{"total":200000000,"available":199176466,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:38:43 2023","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.071,"15min":0.999},"min":{"1min":0.63500000000000001,"5min":0.59799999999999998,"15min":0.53100000000000003},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":2.0289999999999999},"last":1.7689999999999999},{"interface":"front","average":{"1min":1.123,"5min":1.077,"15min":1.0329999999999999},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.60599999999999998},"max":{"1min":1.784,"5min":1.784,"15min":2.6829999999999998},"last":1.921}]},{"osd":1,"last update":"Thu Oct 26 01:38:43 2023","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":1.161,"15min":1.1259999999999999},"min":{"1min":0.749,"5min":0.66700000000000004,"15min":0.56799999999999995},"max":{"1min":1.579,"5min":1.599,"15min":4.0519999999999996},"last":2.056},{"interface":"front","average":{"1min":1.2529999999999999,"5min":1.1779999999999999,"15min":1.121},"min":{"1min":0.70799999999999996,"5min":0.70399999999999996,"15min":0.60499999999999998},"max":{"1min":2.3919999999999999,"5min":2.3919999999999999,"15min":2.3919999999999999},"last":1.8919999999999999}]},{"osd":2,"last update":"Thu Oct 26 01:38:43 2023","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":1.304,"15min":1.2430000000000001},"min":{"1min":1.0429999999999999,"5min":0.66700000000000004,"15min":0.64300000000000002},"max":{"1min":1.6759999999999999,"5min":1.845,"15min":5.6749999999999998},"last":2.2040000000000002},{"interface":"front","average":{"1min":1.1799999999999999,"5min":1.218,"15min":1.1419999999999999},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.631,"5min":3.673,"15min":3.673},"last":1.2210000000000001}]},{"osd":3,"last update":"Thu Oct 26 01:38:43 2023","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.228,"15min":1.1919999999999999},"min":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.70399999999999996},"max":{"1min":1.7030000000000001,"5min":1.847,"15min":1.891},"last":1.9890000000000001},{"interface":"front","average":{"1min":1.1799999999999999,"5min":1.204,"15min":1.1619999999999999},"min":{"1min":0.86099999999999999,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.522,"5min":1.681,"15min":1.7210000000000001},"last":1.353}]},{"osd":5,"last update":"Thu Oct 26 01:38:43 2023","interfaces":[{"interface":"back","average":{"1min":1.236,"5min":1.258,"15min":1.2809999999999999},"min":{"1min":0.85699999999999998,"5min":0.52100000000000002,"15min":0.46300000000000002},"max":{"1min":1.681,"5min":2.1429999999999998,"15min":2.1429999999999998},"last":2.2469999999999999},{"interface":"front","average":{"1min":1.2270000000000001,"5min":1.2809999999999999,"15min":1.321},"min":{"1min":0.84899999999999998,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.754,"5min":1.9810000000000001,"15min":5.8579999999999997},"last":2.2850000000000001}]},{"osd":6,"last update":"Thu Oct 26 01:38:43 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.294,"15min":1.335},"min":{"1min":0.752,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.708,"5min":2.2149999999999999,"15min":12.343999999999999},"last":1.4419999999999999},{"interface":"front","average":{"1min":1.5129999999999999,"5min":1.5109999999999999,"15min":1.5189999999999999},"min":{"1min":1.0660000000000001,"5min":0.57999999999999996,"15min":0.55000000000000004},"max":{"1min":1.762,"5min":2.117,"15min":12.605},"last":2.1040000000000001}]},{"osd":7,"last update":"Thu Oct 26 01:38:43 2023","interfaces":[{"interface":"back","average":{"1min":1.4690000000000001,"5min":1.4670000000000001,"15min":1.49},"min":{"1min":1.1279999999999999,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.8200000000000001,"5min":2.762,"15min":5.774},"last":2.3330000000000002},{"interface":"front","average":{"1min":1.399,"5min":1.484,"15min":1.4590000000000001},"min":{"1min":1.008,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.7589999999999999,"5min":2.8330000000000002,"15min":4.6520000000000001},"last":1.5149999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574957,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":810,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194502,"statfs":{"total":200000000,"available":199170191,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:38:47 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.84499999999999997,"15min":0.85299999999999998},"min":{"1min":0.46600000000000003,"5min":0.41199999999999998,"15min":0.308},"max":{"1min":1.4079999999999999,"5min":1.5249999999999999,"15min":6.3200000000000003},"last":0.95999999999999996},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.97699999999999998,"15min":0.93300000000000005},"min":{"1min":0.44,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.476,"5min":1.9970000000000001,"15min":4.2619999999999996},"last":1.0129999999999999}]},{"osd":1,"last update":"Thu Oct 26 01:38:47 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.878,"15min":0.89100000000000001},"min":{"1min":0.48399999999999999,"5min":0.39300000000000002,"15min":0.34000000000000002},"max":{"1min":1.2829999999999999,"5min":1.629,"15min":5.1260000000000003},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.98499999999999999,"5min":1.0189999999999999,"15min":1.0029999999999999},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.36599999999999999},"max":{"1min":1.4870000000000001,"5min":1.873,"15min":6.1520000000000001},"last":1.286}]},{"osd":3,"last update":"Thu Oct 26 01:38:52 2023","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":1.004,"15min":0.94499999999999995},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.407,"5min":2.0640000000000001,"15min":2.0640000000000001},"last":1.3500000000000001},{"interface":"front","average":{"1min":1.0349999999999999,"5min":1.0860000000000001,"15min":1.0249999999999999},"min":{"1min":0.53800000000000003,"5min":0.51100000000000001,"15min":0.45300000000000001},"max":{"1min":1.665,"5min":1.9279999999999999,"15min":8.9019999999999992},"last":1.093}]},{"osd":4,"last update":"Thu Oct 26 01:38:47 2023","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.3280000000000001,"15min":1.2689999999999999},"min":{"1min":0.81599999999999995,"5min":0.80500000000000005,"15min":0.73899999999999999},"max":{"1min":1.6080000000000001,"5min":2.6190000000000002,"15min":6.2670000000000003},"last":1.4850000000000001},{"interface":"front","average":{"1min":1.2709999999999999,"5min":1.2969999999999999,"15min":1.266},"min":{"1min":0.81699999999999995,"5min":0.81100000000000005,"15min":0.78700000000000003},"max":{"1min":1.8380000000000001,"5min":2.4359999999999999,"15min":7.343},"last":1.1950000000000001}]},{"osd":5,"last update":"Thu Oct 26 01:38:47 2023","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":1.3500000000000001,"15min":1.29},"min":{"1min":0.85099999999999998,"5min":0.81699999999999995,"15min":0.71899999999999997},"max":{"1min":1.669,"5min":2.5790000000000002,"15min":3.7170000000000001},"last":1.55},{"interface":"front","average":{"1min":1.302,"5min":1.3120000000000001,"15min":1.2929999999999999},"min":{"1min":0.872,"5min":0.81899999999999995,"15min":0.65300000000000002},"max":{"1min":1.708,"5min":2.298,"15min":6.3899999999999997},"last":1.5780000000000001}]},{"osd":6,"last update":"Thu Oct 26 01:38:47 2023","interfaces":[{"interface":"back","average":{"1min":1.3280000000000001,"5min":1.3919999999999999,"15min":1.3380000000000001},"min":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.70599999999999996},"max":{"1min":1.714,"5min":2.73,"15min":6.0490000000000004},"last":1.643},{"interface":"front","average":{"1min":1.379,"5min":1.4139999999999999,"15min":1.3740000000000001},"min":{"1min":0.89200000000000002,"5min":0.78200000000000003,"15min":0.76800000000000002},"max":{"1min":2.0990000000000002,"5min":2.4089999999999998,"15min":5.0650000000000004},"last":1.4399999999999999}]},{"osd":7,"last update":"Thu Oct 26 01:38:47 2023","interfaces":[{"interface":"back","average":{"1min":1.373,"5min":1.3979999999999999,"15min":1.3939999999999999},"min":{"1min":0.79200000000000004,"5min":0.77900000000000003,"15min":0.69399999999999995},"max":{"1min":1.776,"5min":2.3370000000000002,"15min":6.4480000000000004},"last":1.7949999999999999},{"interface":"front","average":{"1min":1.4830000000000001,"5min":1.5169999999999999,"15min":1.458},"min":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.91800000000000004},"max":{"1min":2.1480000000000001,"5min":2.8290000000000002,"15min":2.8439999999999999},"last":1.734}]}]},{"osd":1,"up_from":13,"seq":55834574957,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":810,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194502,"statfs":{"total":200000000,"available":199170191,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:38:49 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.76900000000000002,"15min":0.72799999999999998},"min":{"1min":0.41799999999999998,"5min":0.34000000000000002,"15min":0.29499999999999998},"max":{"1min":0.92700000000000005,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.87,"15min":0.80200000000000005},"min":{"1min":0.52000000000000002,"5min":0.36299999999999999,"15min":0.36199999999999999},"max":{"1min":1.2769999999999999,"5min":1.6399999999999999,"15min":1.6399999999999999},"last":0.71699999999999997}]},{"osd":2,"last update":"Thu Oct 26 01:38:49 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.995,"15min":0.999},"min":{"1min":0.65300000000000002,"5min":0.39700000000000002,"15min":0.39600000000000002},"max":{"1min":1.286,"5min":1.8200000000000001,"15min":2.4830000000000001},"last":0.874},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.96199999999999997,"15min":0.88400000000000001},"min":{"1min":0.48099999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.569,"5min":1.8080000000000001,"15min":1.8080000000000001},"last":1.1339999999999999}]},{"osd":3,"last update":"Thu Oct 26 01:38:53 2023","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":0.996,"15min":0.95499999999999996},"min":{"1min":0.59999999999999998,"5min":0.42499999999999999,"15min":0.39500000000000002},"max":{"1min":1.5900000000000001,"5min":1.611,"15min":1.75},"last":1.292},{"interface":"front","average":{"1min":0.998,"5min":1.028,"15min":0.99199999999999999},"min":{"1min":0.55000000000000004,"5min":0.47799999999999998,"15min":0.41299999999999998},"max":{"1min":1.5049999999999999,"5min":1.673,"15min":1.802},"last":0.69799999999999995}]},{"osd":4,"last update":"Thu Oct 26 01:38:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":1.222,"15min":1.1990000000000001},"min":{"1min":0.90400000000000003,"5min":0.69099999999999995,"15min":0.56200000000000006},"max":{"1min":1.5249999999999999,"5min":1.756,"15min":2.0659999999999998},"last":1.024},{"interface":"front","average":{"1min":1.1659999999999999,"5min":1.1870000000000001,"15min":1.1399999999999999},"min":{"1min":0.81299999999999994,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.4390000000000001,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":1.0780000000000001}]},{"osd":5,"last update":"Thu Oct 26 01:38:49 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.2629999999999999,"15min":1.2290000000000001},"min":{"1min":0.84799999999999998,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.621,"5min":1.8779999999999999,"15min":1.8779999999999999},"last":1.1679999999999999},{"interface":"front","average":{"1min":1.359,"5min":1.375,"15min":1.339},"min":{"1min":1.052,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.7210000000000001,"5min":2.04,"15min":2.2200000000000002},"last":1.056}]},{"osd":6,"last update":"Thu Oct 26 01:38:49 2023","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.347,"15min":1.2849999999999999},"min":{"1min":0.90400000000000003,"5min":0.70199999999999996,"15min":0.68899999999999995},"max":{"1min":1.7709999999999999,"5min":2.0910000000000002,"15min":2.3780000000000001},"last":0.83399999999999996},{"interface":"front","average":{"1min":1.2889999999999999,"5min":1.2789999999999999,"15min":1.2490000000000001},"min":{"1min":0.92700000000000005,"5min":0.80700000000000005,"15min":0.70799999999999996},"max":{"1min":1.653,"5min":1.956,"15min":1.984},"last":1.22}]},{"osd":7,"last update":"Thu Oct 26 01:38:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.2749999999999999,"15min":1.262},"min":{"1min":0.79400000000000004,"5min":0.76100000000000001,"15min":0.69699999999999995},"max":{"1min":1.764,"5min":1.8380000000000001,"15min":2.6930000000000001},"last":1.256},{"interface":"front","average":{"1min":1.431,"5min":1.419,"15min":1.345},"min":{"1min":0.95499999999999996,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.835,"5min":2.0190000000000001,"15min":2.0190000000000001},"last":0.98699999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574957,"num_pgs":6,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1386,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193925,"statfs":{"total":200000000,"available":198579823,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Oct 26 01:38:46 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.79700000000000004,"15min":0.91100000000000003},"min":{"1min":0.47799999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.159,"5min":1.5049999999999999,"15min":19.425000000000001},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.93999999999999995,"15min":1.0369999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.462,"5min":1.462,"15min":15.162000000000001},"last":0.748}]},{"osd":2,"last update":"Thu Oct 26 01:38:46 2023","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":1.0149999999999999,"15min":1.175},"min":{"1min":0.64300000000000002,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.47,"5min":1.577,"15min":28.439},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.90000000000000002,"15min":0.91300000000000003},"min":{"1min":0.57299999999999995,"5min":0.37,"15min":0.37},"max":{"1min":1.2969999999999999,"5min":1.4810000000000001,"15min":4.9459999999999997},"last":1.0980000000000001}]},{"osd":3,"last update":"Thu Oct 26 01:38:46 2023","interfaces":[{"interface":"back","average":{"1min":1.1180000000000001,"5min":1.0640000000000001,"15min":1.085},"min":{"1min":0.66800000000000004,"5min":0.46000000000000002,"15min":0.44600000000000001},"max":{"1min":1.6539999999999999,"5min":1.6539999999999999,"15min":4.9800000000000004},"last":1.5549999999999999},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.123,"15min":1.1499999999999999},"min":{"1min":0.66000000000000003,"5min":0.438,"15min":0.438},"max":{"1min":1.4279999999999999,"5min":1.6559999999999999,"15min":5.016},"last":1.5069999999999999}]},{"osd":4,"last update":"Thu Oct 26 01:38:46 2023","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":1.169,"15min":1.2789999999999999},"min":{"1min":0.94499999999999995,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.391,"5min":1.681,"15min":19.329000000000001},"last":1.026},{"interface":"front","average":{"1min":1.23,"5min":1.262,"15min":1.292},"min":{"1min":0.86799999999999999,"5min":0.82099999999999995,"15min":0.67600000000000005},"max":{"1min":1.5149999999999999,"5min":1.718,"15min":5.5229999999999997},"last":1.208}]},{"osd":5,"last update":"Thu Oct 26 01:38:46 2023","interfaces":[{"interface":"back","average":{"1min":1.427,"5min":1.4019999999999999,"15min":1.4279999999999999},"min":{"1min":1.03,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.819,"5min":1.857,"15min":5.6360000000000001},"last":1.609},{"interface":"front","average":{"1min":1.2909999999999999,"5min":1.2749999999999999,"15min":1.395},"min":{"1min":0.80200000000000005,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.5880000000000001,"5min":1.8200000000000001,"15min":19.384},"last":1.1539999999999999}]},{"osd":6,"last update":"Thu Oct 26 01:38:46 2023","interfaces":[{"interface":"back","average":{"1min":1.306,"5min":1.296,"15min":1.3600000000000001},"min":{"1min":0.79300000000000004,"5min":0.76700000000000002,"15min":0.73799999999999999},"max":{"1min":1.736,"5min":1.736,"15min":8.2609999999999992},"last":1.3380000000000001},{"interface":"front","average":{"1min":1.3,"5min":1.335,"15min":1.3999999999999999},"min":{"1min":0.82999999999999996,"5min":0.77900000000000003,"15min":0.75900000000000001},"max":{"1min":1.581,"5min":1.9450000000000001,"15min":11.849},"last":1.373}]},{"osd":7,"last update":"Thu Oct 26 01:38:46 2023","interfaces":[{"interface":"back","average":{"1min":1.46,"5min":1.4319999999999999,"15min":1.4770000000000001},"min":{"1min":1.089,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.9239999999999999,"5min":1.9239999999999999,"15min":5.6689999999999996},"last":1.722},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.397,"15min":1.4930000000000001},"min":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.69399999999999995},"max":{"1min":1.661,"5min":1.9390000000000001,"15min":19.245999999999999},"last":1.4279999999999999}]}]}],"pool_statfs":[]}} 2023-10-26T01:39:31.743 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.3f is not active+clean 2023-10-26T01:39:31.743 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2023-10-26T01:39:25.339741+0000', 'last_change': '2023-10-26T01:39:25.339741+0000', 'last_active': '2023-10-26T01:39:25.309143+0000', 'last_peered': '2023-10-26T01:39:25.309143+0000', 'last_clean': '2023-10-26T01:39:25.309143+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T01:39:25.339741+0000', 'last_undegraded': '2023-10-26T01:39:25.339741+0000', 'last_fullsized': '2023-10-26T01:39:25.339741+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_clean_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-26T01:39:31.744 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.3e is not active+clean 2023-10-26T01:39:31.744 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2023-10-26T01:39:25.342024+0000', 'last_change': '2023-10-26T01:39:25.342024+0000', 'last_active': '2023-10-26T01:39:25.309143+0000', 'last_peered': '2023-10-26T01:39:25.309143+0000', 'last_clean': '2023-10-26T01:39:25.309143+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T01:39:25.342024+0000', 'last_undegraded': '2023-10-26T01:39:25.342024+0000', 'last_fullsized': '2023-10-26T01:39:25.342024+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_clean_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-26T01:39:31.745 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.3d is not active+clean 2023-10-26T01:39:31.745 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2023-10-26T01:39:25.349198+0000', 'last_change': '2023-10-26T01:39:25.349198+0000', 'last_active': '2023-10-26T01:39:25.309143+0000', 'last_peered': '2023-10-26T01:39:25.309143+0000', 'last_clean': '2023-10-26T01:39:25.309143+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T01:39:25.349198+0000', 'last_undegraded': '2023-10-26T01:39:25.349198+0000', 'last_fullsized': '2023-10-26T01:39:25.349198+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_clean_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-26T01:39:31.746 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.3c is not active+clean 2023-10-26T01:39:31.746 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2023-10-26T01:39:25.347440+0000', 'last_change': '2023-10-26T01:39:25.347440+0000', 'last_active': '2023-10-26T01:39:25.309143+0000', 'last_peered': '2023-10-26T01:39:25.309143+0000', 'last_clean': '2023-10-26T01:39:25.309143+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T01:39:25.347440+0000', 'last_undegraded': '2023-10-26T01:39:25.347440+0000', 'last_fullsized': '2023-10-26T01:39:25.347440+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_clean_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-26T01:39:31.747 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.3b is not active+clean 2023-10-26T01:39:31.747 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2023-10-26T01:39:25.337848+0000', 'last_change': '2023-10-26T01:39:25.337848+0000', 'last_active': '2023-10-26T01:39:25.309143+0000', 'last_peered': '2023-10-26T01:39:25.309143+0000', 'last_clean': '2023-10-26T01:39:25.309143+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T01:39:25.337848+0000', 'last_undegraded': '2023-10-26T01:39:25.337848+0000', 'last_fullsized': '2023-10-26T01:39:25.337848+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_clean_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-26T01:39:31.747 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.3a is not active+clean 2023-10-26T01:39:31.748 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2023-10-26T01:39:25.343578+0000', 'last_change': '2023-10-26T01:39:25.343578+0000', 'last_active': '2023-10-26T01:39:25.309143+0000', 'last_peered': '2023-10-26T01:39:25.309143+0000', 'last_clean': '2023-10-26T01:39:25.309143+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T01:39:25.343578+0000', 'last_undegraded': '2023-10-26T01:39:25.343578+0000', 'last_fullsized': '2023-10-26T01:39:25.343578+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_clean_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-26T01:39:31.748 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.39 is not active+clean 2023-10-26T01:39:31.749 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2023-10-26T01:39:25.345665+0000', 'last_change': '2023-10-26T01:39:25.345665+0000', 'last_active': '2023-10-26T01:39:25.309143+0000', 'last_peered': '2023-10-26T01:39:25.309143+0000', 'last_clean': '2023-10-26T01:39:25.309143+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T01:39:25.345665+0000', 'last_undegraded': '2023-10-26T01:39:25.345665+0000', 'last_fullsized': '2023-10-26T01:39:25.345665+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_clean_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-26T01:39:31.749 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.38 is not active+clean 2023-10-26T01:39:31.749 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2023-10-26T01:39:25.348666+0000', 'last_change': '2023-10-26T01:39:25.348666+0000', 'last_active': '2023-10-26T01:39:25.309143+0000', 'last_peered': '2023-10-26T01:39:25.309143+0000', 'last_clean': '2023-10-26T01:39:25.309143+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T01:39:25.348666+0000', 'last_undegraded': '2023-10-26T01:39:25.348666+0000', 'last_fullsized': '2023-10-26T01:39:25.348666+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_clean_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-26T01:39:31.750 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.37 is not active+clean 2023-10-26T01:39:31.750 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2023-10-26T01:39:25.349996+0000', 'last_change': '2023-10-26T01:39:25.349996+0000', 'last_active': '2023-10-26T01:39:25.309143+0000', 'last_peered': '2023-10-26T01:39:25.309143+0000', 'last_clean': '2023-10-26T01:39:25.309143+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T01:39:25.349996+0000', 'last_undegraded': '2023-10-26T01:39:25.349996+0000', 'last_fullsized': '2023-10-26T01:39:25.349996+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_clean_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-26T01:39:31.751 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.36 is not active+clean 2023-10-26T01:39:31.751 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2023-10-26T01:39:25.342807+0000', 'last_change': '2023-10-26T01:39:25.342807+0000', 'last_active': '2023-10-26T01:39:25.309143+0000', 'last_peered': '2023-10-26T01:39:25.309143+0000', 'last_clean': '2023-10-26T01:39:25.309143+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T01:39:25.342807+0000', 'last_undegraded': '2023-10-26T01:39:25.342807+0000', 'last_fullsized': '2023-10-26T01:39:25.342807+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_clean_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-26T01:39:31.752 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.35 is not active+clean 2023-10-26T01:39:31.752 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2023-10-26T01:39:25.340008+0000', 'last_change': '2023-10-26T01:39:25.340008+0000', 'last_active': '2023-10-26T01:39:25.309143+0000', 'last_peered': '2023-10-26T01:39:25.309143+0000', 'last_clean': '2023-10-26T01:39:25.309143+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T01:39:25.340008+0000', 'last_undegraded': '2023-10-26T01:39:25.340008+0000', 'last_fullsized': '2023-10-26T01:39:25.340008+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_clean_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-26T01:39:31.753 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.34 is not active+clean 2023-10-26T01:39:31.753 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2023-10-26T01:39:25.348637+0000', 'last_change': '2023-10-26T01:39:25.348637+0000', 'last_active': '2023-10-26T01:39:25.309143+0000', 'last_peered': '2023-10-26T01:39:25.309143+0000', 'last_clean': '2023-10-26T01:39:25.309143+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T01:39:25.348637+0000', 'last_undegraded': '2023-10-26T01:39:25.348637+0000', 'last_fullsized': '2023-10-26T01:39:25.348637+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_clean_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-26T01:39:31.754 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.33 is not active+clean 2023-10-26T01:39:31.754 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2023-10-26T01:39:25.343722+0000', 'last_change': '2023-10-26T01:39:25.343722+0000', 'last_active': '2023-10-26T01:39:25.309143+0000', 'last_peered': '2023-10-26T01:39:25.309143+0000', 'last_clean': '2023-10-26T01:39:25.309143+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T01:39:25.343722+0000', 'last_undegraded': '2023-10-26T01:39:25.343722+0000', 'last_fullsized': '2023-10-26T01:39:25.343722+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_clean_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-26T01:39:31.755 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.32 is not active+clean 2023-10-26T01:39:31.755 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2023-10-26T01:39:25.343921+0000', 'last_change': '2023-10-26T01:39:25.343921+0000', 'last_active': '2023-10-26T01:39:25.309143+0000', 'last_peered': '2023-10-26T01:39:25.309143+0000', 'last_clean': '2023-10-26T01:39:25.309143+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T01:39:25.343921+0000', 'last_undegraded': '2023-10-26T01:39:25.343921+0000', 'last_fullsized': '2023-10-26T01:39:25.343921+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_clean_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-26T01:39:31.756 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.31 is not active+clean 2023-10-26T01:39:31.756 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2023-10-26T01:39:25.345622+0000', 'last_change': '2023-10-26T01:39:25.345622+0000', 'last_active': '2023-10-26T01:39:25.309143+0000', 'last_peered': '2023-10-26T01:39:25.309143+0000', 'last_clean': '2023-10-26T01:39:25.309143+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T01:39:25.345622+0000', 'last_undegraded': '2023-10-26T01:39:25.345622+0000', 'last_fullsized': '2023-10-26T01:39:25.345622+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_clean_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-26T01:39:31.757 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.2f is not active+clean 2023-10-26T01:39:31.757 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2023-10-26T01:39:25.341466+0000', 'last_change': '2023-10-26T01:39:25.341466+0000', 'last_active': '2023-10-26T01:39:25.309143+0000', 'last_peered': '2023-10-26T01:39:25.309143+0000', 'last_clean': '2023-10-26T01:39:25.309143+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T01:39:25.341466+0000', 'last_undegraded': '2023-10-26T01:39:25.341466+0000', 'last_fullsized': '2023-10-26T01:39:25.341466+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_clean_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-26T01:39:31.757 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.2d is not active+clean 2023-10-26T01:39:31.758 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2023-10-26T01:39:25.341042+0000', 'last_change': '2023-10-26T01:39:25.341042+0000', 'last_active': '2023-10-26T01:39:25.309143+0000', 'last_peered': '2023-10-26T01:39:25.309143+0000', 'last_clean': '2023-10-26T01:39:25.309143+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T01:39:25.341042+0000', 'last_undegraded': '2023-10-26T01:39:25.341042+0000', 'last_fullsized': '2023-10-26T01:39:25.341042+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_clean_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-26T01:39:31.758 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.2b is not active+clean 2023-10-26T01:39:31.759 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2023-10-26T01:39:25.338200+0000', 'last_change': '2023-10-26T01:39:25.338200+0000', 'last_active': '2023-10-26T01:39:25.309143+0000', 'last_peered': '2023-10-26T01:39:25.309143+0000', 'last_clean': '2023-10-26T01:39:25.309143+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T01:39:25.338200+0000', 'last_undegraded': '2023-10-26T01:39:25.338200+0000', 'last_fullsized': '2023-10-26T01:39:25.338200+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_clean_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-26T01:39:31.759 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.2a is not active+clean 2023-10-26T01:39:31.759 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2023-10-26T01:39:25.339459+0000', 'last_change': '2023-10-26T01:39:25.339459+0000', 'last_active': '2023-10-26T01:39:25.309143+0000', 'last_peered': '2023-10-26T01:39:25.309143+0000', 'last_clean': '2023-10-26T01:39:25.309143+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T01:39:25.339459+0000', 'last_undegraded': '2023-10-26T01:39:25.339459+0000', 'last_fullsized': '2023-10-26T01:39:25.339459+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_clean_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-26T01:39:31.760 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.29 is not active+clean 2023-10-26T01:39:31.760 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2023-10-26T01:39:25.347072+0000', 'last_change': '2023-10-26T01:39:25.347072+0000', 'last_active': '2023-10-26T01:39:25.309143+0000', 'last_peered': '2023-10-26T01:39:25.309143+0000', 'last_clean': '2023-10-26T01:39:25.309143+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T01:39:25.347072+0000', 'last_undegraded': '2023-10-26T01:39:25.347072+0000', 'last_fullsized': '2023-10-26T01:39:25.347072+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_clean_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-26T01:39:31.761 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.28 is not active+clean 2023-10-26T01:39:31.761 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2023-10-26T01:39:25.340599+0000', 'last_change': '2023-10-26T01:39:25.340599+0000', 'last_active': '2023-10-26T01:39:25.309143+0000', 'last_peered': '2023-10-26T01:39:25.309143+0000', 'last_clean': '2023-10-26T01:39:25.309143+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T01:39:25.340599+0000', 'last_undegraded': '2023-10-26T01:39:25.340599+0000', 'last_fullsized': '2023-10-26T01:39:25.340599+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_clean_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-26T01:39:31.761 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.27 is not active+clean 2023-10-26T01:39:31.762 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2023-10-26T01:39:25.342364+0000', 'last_change': '2023-10-26T01:39:25.342364+0000', 'last_active': '2023-10-26T01:39:25.309143+0000', 'last_peered': '2023-10-26T01:39:25.309143+0000', 'last_clean': '2023-10-26T01:39:25.309143+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T01:39:25.342364+0000', 'last_undegraded': '2023-10-26T01:39:25.342364+0000', 'last_fullsized': '2023-10-26T01:39:25.342364+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_clean_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-26T01:39:31.762 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.25 is not active+clean 2023-10-26T01:39:31.763 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2023-10-26T01:39:25.341388+0000', 'last_change': '2023-10-26T01:39:25.341388+0000', 'last_active': '2023-10-26T01:39:25.309143+0000', 'last_peered': '2023-10-26T01:39:25.309143+0000', 'last_clean': '2023-10-26T01:39:25.309143+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T01:39:25.341388+0000', 'last_undegraded': '2023-10-26T01:39:25.341388+0000', 'last_fullsized': '2023-10-26T01:39:25.341388+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_clean_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-26T01:39:31.763 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.24 is not active+clean 2023-10-26T01:39:31.764 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2023-10-26T01:39:25.340994+0000', 'last_change': '2023-10-26T01:39:25.340994+0000', 'last_active': '2023-10-26T01:39:25.309143+0000', 'last_peered': '2023-10-26T01:39:25.309143+0000', 'last_clean': '2023-10-26T01:39:25.309143+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T01:39:25.340994+0000', 'last_undegraded': '2023-10-26T01:39:25.340994+0000', 'last_fullsized': '2023-10-26T01:39:25.340994+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_clean_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-26T01:39:31.764 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.23 is not active+clean 2023-10-26T01:39:31.764 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2023-10-26T01:39:25.346960+0000', 'last_change': '2023-10-26T01:39:25.346960+0000', 'last_active': '2023-10-26T01:39:25.309143+0000', 'last_peered': '2023-10-26T01:39:25.309143+0000', 'last_clean': '2023-10-26T01:39:25.309143+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T01:39:25.346960+0000', 'last_undegraded': '2023-10-26T01:39:25.346960+0000', 'last_fullsized': '2023-10-26T01:39:25.346960+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_clean_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-26T01:39:31.765 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.22 is not active+clean 2023-10-26T01:39:31.765 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2023-10-26T01:39:25.345158+0000', 'last_change': '2023-10-26T01:39:25.345158+0000', 'last_active': '2023-10-26T01:39:25.309143+0000', 'last_peered': '2023-10-26T01:39:25.309143+0000', 'last_clean': '2023-10-26T01:39:25.309143+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T01:39:25.345158+0000', 'last_undegraded': '2023-10-26T01:39:25.345158+0000', 'last_fullsized': '2023-10-26T01:39:25.345158+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_clean_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-26T01:39:31.766 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.21 is not active+clean 2023-10-26T01:39:31.766 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2023-10-26T01:39:25.337346+0000', 'last_change': '2023-10-26T01:39:25.337346+0000', 'last_active': '2023-10-26T01:39:25.309143+0000', 'last_peered': '2023-10-26T01:39:25.309143+0000', 'last_clean': '2023-10-26T01:39:25.309143+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T01:39:25.337346+0000', 'last_undegraded': '2023-10-26T01:39:25.337346+0000', 'last_fullsized': '2023-10-26T01:39:25.337346+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_clean_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-26T01:39:31.767 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.20 is not active+clean 2023-10-26T01:39:31.767 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2023-10-26T01:39:25.340549+0000', 'last_change': '2023-10-26T01:39:25.340549+0000', 'last_active': '2023-10-26T01:39:25.309143+0000', 'last_peered': '2023-10-26T01:39:25.309143+0000', 'last_clean': '2023-10-26T01:39:25.309143+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T01:39:25.340549+0000', 'last_undegraded': '2023-10-26T01:39:25.340549+0000', 'last_fullsized': '2023-10-26T01:39:25.340549+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_clean_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-26T01:39:31.768 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.1f is not active+clean 2023-10-26T01:39:31.768 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2023-10-26T01:39:25.346506+0000', 'last_change': '2023-10-26T01:39:25.346506+0000', 'last_active': '2023-10-26T01:39:25.309143+0000', 'last_peered': '2023-10-26T01:39:25.309143+0000', 'last_clean': '2023-10-26T01:39:25.309143+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T01:39:25.346506+0000', 'last_undegraded': '2023-10-26T01:39:25.346506+0000', 'last_fullsized': '2023-10-26T01:39:25.346506+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_clean_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-26T01:39:31.769 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.1c is not active+clean 2023-10-26T01:39:31.769 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2023-10-26T01:39:25.335905+0000', 'last_change': '2023-10-26T01:39:25.335905+0000', 'last_active': '2023-10-26T01:39:25.309143+0000', 'last_peered': '2023-10-26T01:39:25.309143+0000', 'last_clean': '2023-10-26T01:39:25.309143+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T01:39:25.335905+0000', 'last_undegraded': '2023-10-26T01:39:25.335905+0000', 'last_fullsized': '2023-10-26T01:39:25.335905+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_clean_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-26T01:39:31.770 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.1d is not active+clean 2023-10-26T01:39:31.770 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2023-10-26T01:39:25.336364+0000', 'last_change': '2023-10-26T01:39:25.336364+0000', 'last_active': '2023-10-26T01:39:25.309143+0000', 'last_peered': '2023-10-26T01:39:25.309143+0000', 'last_clean': '2023-10-26T01:39:25.309143+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T01:39:25.336364+0000', 'last_undegraded': '2023-10-26T01:39:25.336364+0000', 'last_fullsized': '2023-10-26T01:39:25.336364+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_clean_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-26T01:39:31.770 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.1b is not active+clean 2023-10-26T01:39:31.771 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2023-10-26T01:39:25.339661+0000', 'last_change': '2023-10-26T01:39:25.339661+0000', 'last_active': '2023-10-26T01:39:25.309143+0000', 'last_peered': '2023-10-26T01:39:25.309143+0000', 'last_clean': '2023-10-26T01:39:25.309143+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T01:39:25.339661+0000', 'last_undegraded': '2023-10-26T01:39:25.339661+0000', 'last_fullsized': '2023-10-26T01:39:25.339661+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_clean_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-26T01:39:31.771 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.18 is not active+clean 2023-10-26T01:39:31.772 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2023-10-26T01:39:25.345341+0000', 'last_change': '2023-10-26T01:39:25.345341+0000', 'last_active': '2023-10-26T01:39:25.309143+0000', 'last_peered': '2023-10-26T01:39:25.309143+0000', 'last_clean': '2023-10-26T01:39:25.309143+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T01:39:25.345341+0000', 'last_undegraded': '2023-10-26T01:39:25.345341+0000', 'last_fullsized': '2023-10-26T01:39:25.345341+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_clean_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-26T01:39:31.772 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.19 is not active+clean 2023-10-26T01:39:31.773 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2023-10-26T01:39:25.339971+0000', 'last_change': '2023-10-26T01:39:25.339971+0000', 'last_active': '2023-10-26T01:39:25.309143+0000', 'last_peered': '2023-10-26T01:39:25.309143+0000', 'last_clean': '2023-10-26T01:39:25.309143+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T01:39:25.339971+0000', 'last_undegraded': '2023-10-26T01:39:25.339971+0000', 'last_fullsized': '2023-10-26T01:39:25.339971+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_clean_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-26T01:39:31.773 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.16 is not active+clean 2023-10-26T01:39:31.774 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2023-10-26T01:39:25.340967+0000', 'last_change': '2023-10-26T01:39:25.340967+0000', 'last_active': '2023-10-26T01:39:25.309143+0000', 'last_peered': '2023-10-26T01:39:25.309143+0000', 'last_clean': '2023-10-26T01:39:25.309143+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T01:39:25.340967+0000', 'last_undegraded': '2023-10-26T01:39:25.340967+0000', 'last_fullsized': '2023-10-26T01:39:25.340967+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_clean_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-26T01:39:31.774 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.17 is not active+clean 2023-10-26T01:39:31.775 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2023-10-26T01:39:25.344566+0000', 'last_change': '2023-10-26T01:39:25.344566+0000', 'last_active': '2023-10-26T01:39:25.309143+0000', 'last_peered': '2023-10-26T01:39:25.309143+0000', 'last_clean': '2023-10-26T01:39:25.309143+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T01:39:25.344566+0000', 'last_undegraded': '2023-10-26T01:39:25.344566+0000', 'last_fullsized': '2023-10-26T01:39:25.344566+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_clean_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-26T01:39:31.775 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.14 is not active+clean 2023-10-26T01:39:31.776 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2023-10-26T01:39:25.332527+0000', 'last_change': '2023-10-26T01:39:25.332527+0000', 'last_active': '2023-10-26T01:39:25.309143+0000', 'last_peered': '2023-10-26T01:39:25.309143+0000', 'last_clean': '2023-10-26T01:39:25.309143+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T01:39:25.332527+0000', 'last_undegraded': '2023-10-26T01:39:25.332527+0000', 'last_fullsized': '2023-10-26T01:39:25.332527+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_clean_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-26T01:39:31.776 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.15 is not active+clean 2023-10-26T01:39:31.776 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2023-10-26T01:39:25.333835+0000', 'last_change': '2023-10-26T01:39:25.333835+0000', 'last_active': '2023-10-26T01:39:25.309143+0000', 'last_peered': '2023-10-26T01:39:25.309143+0000', 'last_clean': '2023-10-26T01:39:25.309143+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T01:39:25.333835+0000', 'last_undegraded': '2023-10-26T01:39:25.333835+0000', 'last_fullsized': '2023-10-26T01:39:25.333835+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_clean_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-26T01:39:31.777 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.13 is not active+clean 2023-10-26T01:39:31.777 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2023-10-26T01:39:25.344701+0000', 'last_change': '2023-10-26T01:39:25.344701+0000', 'last_active': '2023-10-26T01:39:25.309143+0000', 'last_peered': '2023-10-26T01:39:25.309143+0000', 'last_clean': '2023-10-26T01:39:25.309143+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T01:39:25.344701+0000', 'last_undegraded': '2023-10-26T01:39:25.344701+0000', 'last_fullsized': '2023-10-26T01:39:25.344701+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_clean_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-26T01:39:31.778 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.10 is not active+clean 2023-10-26T01:39:31.778 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2023-10-26T01:39:25.340874+0000', 'last_change': '2023-10-26T01:39:25.340874+0000', 'last_active': '2023-10-26T01:39:25.309143+0000', 'last_peered': '2023-10-26T01:39:25.309143+0000', 'last_clean': '2023-10-26T01:39:25.309143+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T01:39:25.340874+0000', 'last_undegraded': '2023-10-26T01:39:25.340874+0000', 'last_fullsized': '2023-10-26T01:39:25.340874+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_clean_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-26T01:39:31.779 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.11 is not active+clean 2023-10-26T01:39:31.779 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2023-10-26T01:39:25.344820+0000', 'last_change': '2023-10-26T01:39:25.344820+0000', 'last_active': '2023-10-26T01:39:25.309143+0000', 'last_peered': '2023-10-26T01:39:25.309143+0000', 'last_clean': '2023-10-26T01:39:25.309143+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T01:39:25.344820+0000', 'last_undegraded': '2023-10-26T01:39:25.344820+0000', 'last_fullsized': '2023-10-26T01:39:25.344820+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_clean_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-26T01:39:31.779 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.e is not active+clean 2023-10-26T01:39:31.780 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2023-10-26T01:39:25.343890+0000', 'last_change': '2023-10-26T01:39:25.343890+0000', 'last_active': '2023-10-26T01:39:25.309143+0000', 'last_peered': '2023-10-26T01:39:25.309143+0000', 'last_clean': '2023-10-26T01:39:25.309143+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T01:39:25.343890+0000', 'last_undegraded': '2023-10-26T01:39:25.343890+0000', 'last_fullsized': '2023-10-26T01:39:25.343890+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_clean_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-26T01:39:31.780 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.f is not active+clean 2023-10-26T01:39:31.781 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2023-10-26T01:39:25.342089+0000', 'last_change': '2023-10-26T01:39:25.342089+0000', 'last_active': '2023-10-26T01:39:25.309143+0000', 'last_peered': '2023-10-26T01:39:25.309143+0000', 'last_clean': '2023-10-26T01:39:25.309143+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T01:39:25.342089+0000', 'last_undegraded': '2023-10-26T01:39:25.342089+0000', 'last_fullsized': '2023-10-26T01:39:25.342089+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_clean_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-26T01:39:31.781 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.d is not active+clean 2023-10-26T01:39:31.782 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2023-10-26T01:39:25.340936+0000', 'last_change': '2023-10-26T01:39:25.340936+0000', 'last_active': '2023-10-26T01:39:25.309143+0000', 'last_peered': '2023-10-26T01:39:25.309143+0000', 'last_clean': '2023-10-26T01:39:25.309143+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T01:39:25.340936+0000', 'last_undegraded': '2023-10-26T01:39:25.340936+0000', 'last_fullsized': '2023-10-26T01:39:25.340936+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_clean_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-26T01:39:31.782 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.a is not active+clean 2023-10-26T01:39:31.782 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2023-10-26T01:39:25.346409+0000', 'last_change': '2023-10-26T01:39:25.346409+0000', 'last_active': '2023-10-26T01:39:25.309143+0000', 'last_peered': '2023-10-26T01:39:25.309143+0000', 'last_clean': '2023-10-26T01:39:25.309143+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T01:39:25.346409+0000', 'last_undegraded': '2023-10-26T01:39:25.346409+0000', 'last_fullsized': '2023-10-26T01:39:25.346409+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_clean_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-26T01:39:31.783 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.b is not active+clean 2023-10-26T01:39:31.783 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2023-10-26T01:39:25.342827+0000', 'last_change': '2023-10-26T01:39:25.342827+0000', 'last_active': '2023-10-26T01:39:25.309143+0000', 'last_peered': '2023-10-26T01:39:25.309143+0000', 'last_clean': '2023-10-26T01:39:25.309143+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T01:39:25.342827+0000', 'last_undegraded': '2023-10-26T01:39:25.342827+0000', 'last_fullsized': '2023-10-26T01:39:25.342827+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_clean_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-26T01:39:31.784 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.8 is not active+clean 2023-10-26T01:39:31.784 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2023-10-26T01:39:25.340172+0000', 'last_change': '2023-10-26T01:39:25.340172+0000', 'last_active': '2023-10-26T01:39:25.309143+0000', 'last_peered': '2023-10-26T01:39:25.309143+0000', 'last_clean': '2023-10-26T01:39:25.309143+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T01:39:25.340172+0000', 'last_undegraded': '2023-10-26T01:39:25.340172+0000', 'last_fullsized': '2023-10-26T01:39:25.340172+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_clean_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-26T01:39:31.785 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.9 is not active+clean 2023-10-26T01:39:31.785 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2023-10-26T01:39:25.335577+0000', 'last_change': '2023-10-26T01:39:25.335577+0000', 'last_active': '2023-10-26T01:39:25.309143+0000', 'last_peered': '2023-10-26T01:39:25.309143+0000', 'last_clean': '2023-10-26T01:39:25.309143+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T01:39:25.335577+0000', 'last_undegraded': '2023-10-26T01:39:25.335577+0000', 'last_fullsized': '2023-10-26T01:39:25.335577+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_clean_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-26T01:39:31.786 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.6 is not active+clean 2023-10-26T01:39:31.786 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2023-10-26T01:39:25.333593+0000', 'last_change': '2023-10-26T01:39:25.333593+0000', 'last_active': '2023-10-26T01:39:25.309143+0000', 'last_peered': '2023-10-26T01:39:25.309143+0000', 'last_clean': '2023-10-26T01:39:25.309143+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T01:39:25.333593+0000', 'last_undegraded': '2023-10-26T01:39:25.333593+0000', 'last_fullsized': '2023-10-26T01:39:25.333593+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_clean_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-26T01:39:31.787 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.7 is not active+clean 2023-10-26T01:39:31.787 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2023-10-26T01:39:25.339995+0000', 'last_change': '2023-10-26T01:39:25.339995+0000', 'last_active': '2023-10-26T01:39:25.309143+0000', 'last_peered': '2023-10-26T01:39:25.309143+0000', 'last_clean': '2023-10-26T01:39:25.309143+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T01:39:25.339995+0000', 'last_undegraded': '2023-10-26T01:39:25.339995+0000', 'last_fullsized': '2023-10-26T01:39:25.339995+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_clean_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-26T01:39:31.787 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.4 is not active+clean 2023-10-26T01:39:31.788 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2023-10-26T01:39:25.340256+0000', 'last_change': '2023-10-26T01:39:25.340256+0000', 'last_active': '2023-10-26T01:39:25.309143+0000', 'last_peered': '2023-10-26T01:39:25.309143+0000', 'last_clean': '2023-10-26T01:39:25.309143+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T01:39:25.340256+0000', 'last_undegraded': '2023-10-26T01:39:25.340256+0000', 'last_fullsized': '2023-10-26T01:39:25.340256+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_clean_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-26T01:39:31.789 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.5 is not active+clean 2023-10-26T01:39:31.789 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2023-10-26T01:39:25.335457+0000', 'last_change': '2023-10-26T01:39:25.335457+0000', 'last_active': '2023-10-26T01:39:25.309143+0000', 'last_peered': '2023-10-26T01:39:25.309143+0000', 'last_clean': '2023-10-26T01:39:25.309143+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T01:39:25.335457+0000', 'last_undegraded': '2023-10-26T01:39:25.335457+0000', 'last_fullsized': '2023-10-26T01:39:25.335457+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_clean_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-26T01:39:31.790 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.2 is not active+clean 2023-10-26T01:39:31.790 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2023-10-26T01:39:25.337282+0000', 'last_change': '2023-10-26T01:39:25.337282+0000', 'last_active': '2023-10-26T01:39:25.309143+0000', 'last_peered': '2023-10-26T01:39:25.309143+0000', 'last_clean': '2023-10-26T01:39:25.309143+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T01:39:25.337282+0000', 'last_undegraded': '2023-10-26T01:39:25.337282+0000', 'last_fullsized': '2023-10-26T01:39:25.337282+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_clean_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-26T01:39:31.791 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.3 is not active+clean 2023-10-26T01:39:31.791 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2023-10-26T01:39:25.343816+0000', 'last_change': '2023-10-26T01:39:25.343816+0000', 'last_active': '2023-10-26T01:39:25.309143+0000', 'last_peered': '2023-10-26T01:39:25.309143+0000', 'last_clean': '2023-10-26T01:39:25.309143+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T01:39:25.343816+0000', 'last_undegraded': '2023-10-26T01:39:25.343816+0000', 'last_fullsized': '2023-10-26T01:39:25.343816+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_clean_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-26T01:39:31.792 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.1 is not active+clean 2023-10-26T01:39:31.792 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2023-10-26T01:39:25.339461+0000', 'last_change': '2023-10-26T01:39:25.339461+0000', 'last_active': '2023-10-26T01:39:25.309143+0000', 'last_peered': '2023-10-26T01:39:25.309143+0000', 'last_clean': '2023-10-26T01:39:25.309143+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T01:39:25.339461+0000', 'last_undegraded': '2023-10-26T01:39:25.339461+0000', 'last_fullsized': '2023-10-26T01:39:25.339461+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_clean_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-26T01:39:31.792 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.12 is not active+clean 2023-10-26T01:39:31.793 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2023-10-26T01:39:25.334115+0000', 'last_change': '2023-10-26T01:39:25.334115+0000', 'last_active': '2023-10-26T01:39:25.309143+0000', 'last_peered': '2023-10-26T01:39:25.309143+0000', 'last_clean': '2023-10-26T01:39:25.309143+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T01:39:25.334115+0000', 'last_undegraded': '2023-10-26T01:39:25.334115+0000', 'last_fullsized': '2023-10-26T01:39:25.334115+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'last_clean_scrub_stamp': '2023-10-26T01:39:25.309143+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-26T01:39:31.794 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-26T01:39:32.015 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:32.012+0000 7f7582664640 1 -- 172.21.15.169:0/3349033413 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75680051d0 msgr2=0x7f756800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:32.016 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:32.012+0000 7f7582664640 1 --2- 172.21.15.169:0/3349033413 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75680051d0 0x7f756800b980 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f7574006ef0 tx=0x7f75680057f0 comp rx=0 tx=0).stop 2023-10-26T01:39:32.016 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:32.012+0000 7f7582664640 1 -- 172.21.15.169:0/3349033413 shutdown_connections 2023-10-26T01:39:32.016 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:32.012+0000 7f7582664640 1 --2- 172.21.15.169:0/3349033413 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75680051d0 0x7f756800b980 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:32.016 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:32.012+0000 7f7582664640 1 -- 172.21.15.169:0/3349033413 >> 172.21.15.169:0/3349033413 conn(0x7f757c0af8a0 msgr2=0x7f757c0afc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:32.017 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:32.012+0000 7f7582664640 1 -- 172.21.15.169:0/3349033413 shutdown_connections 2023-10-26T01:39:32.017 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:32.012+0000 7f7582664640 1 -- 172.21.15.169:0/3349033413 wait complete. 2023-10-26T01:39:32.019 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:32.014+0000 7f7582664640 1 Processor -- start 2023-10-26T01:39:32.019 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:32.014+0000 7f7582664640 1 -- start start 2023-10-26T01:39:32.020 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:32.015+0000 7f7582664640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f75680051d0 0x7f757c14f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:32.020 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:32.015+0000 7f7582664640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f757c153850 0x7f757c144450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:32.020 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:32.015+0000 7f7582664640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f757c1549f0 0x7f757c154e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:32.020 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:32.015+0000 7f7582664640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f757c14fe90 con 0x7f757c153850 2023-10-26T01:39:32.021 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:32.015+0000 7f7582664640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f757c14fbb0 con 0x7f75680051d0 2023-10-26T01:39:32.021 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:32.015+0000 7f7582664640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f757c153f10 con 0x7f757c1549f0 2023-10-26T01:39:32.021 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:32.016+0000 7f757b7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f757c153850 0x7f757c144450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:32.022 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:32.016+0000 7f757b7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f757c153850 0x7f757c144450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:37454/0 (socket says 172.21.15.169:37454) 2023-10-26T01:39:32.022 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:32.016+0000 7f757b7fe640 1 -- 172.21.15.169:0/1680533788 learned_addr learned my addr 172.21.15.169:0/1680533788 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:39:32.022 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:32.016+0000 7f7580c01640 1 --2- 172.21.15.169:0/1680533788 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f757c1549f0 0x7f757c154e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:32.022 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:32.016+0000 7f757bfff640 1 --2- 172.21.15.169:0/1680533788 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f75680051d0 0x7f757c14f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:32.023 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:32.016+0000 7f757b7fe640 1 -- 172.21.15.169:0/1680533788 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f757c1549f0 msgr2=0x7f757c154e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:32.023 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:32.016+0000 7f757b7fe640 1 --2- 172.21.15.169:0/1680533788 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f757c1549f0 0x7f757c154e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:32.023 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:32.016+0000 7f757b7fe640 1 -- 172.21.15.169:0/1680533788 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f75680051d0 msgr2=0x7f757c14f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:32.023 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:32.016+0000 7f757b7fe640 1 --2- 172.21.15.169:0/1680533788 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f75680051d0 0x7f757c14f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:32.024 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:32.016+0000 7f757b7fe640 1 -- 172.21.15.169:0/1680533788 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f757c0702c0 con 0x7f757c153850 2023-10-26T01:39:32.024 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:32.016+0000 7f757b7fe640 1 --2- 172.21.15.169:0/1680533788 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f757c153850 0x7f757c144450 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f756400d600 tx=0x7f756400da00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:32.024 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:32.016+0000 7f7578ff9640 1 -- 172.21.15.169:0/1680533788 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f756400f880 con 0x7f757c153850 2023-10-26T01:39:32.024 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:32.017+0000 7f7578ff9640 1 -- 172.21.15.169:0/1680533788 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f756400fad0 con 0x7f757c153850 2023-10-26T01:39:32.024 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:32.017+0000 7f7578ff9640 1 -- 172.21.15.169:0/1680533788 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7564010450 con 0x7f757c153850 2023-10-26T01:39:32.025 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:32.017+0000 7f7582664640 1 -- 172.21.15.169:0/1680533788 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f757c1541f0 con 0x7f757c153850 2023-10-26T01:39:32.025 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:32.017+0000 7f7582664640 1 -- 172.21.15.169:0/1680533788 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f757c14f7f0 con 0x7f757c153850 2023-10-26T01:39:32.026 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:32.018+0000 7f7582664640 1 -- 172.21.15.169:0/1680533788 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f757c153d70 con 0x7f757c153850 2023-10-26T01:39:32.026 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:32.019+0000 7f7578ff9640 1 -- 172.21.15.169:0/1680533788 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f75640105b0 con 0x7f757c153850 2023-10-26T01:39:32.026 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:32.019+0000 7f7578ff9640 1 --2- 172.21.15.169:0/1680533788 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f755c06ae40 0x7f755c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:32.027 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:32.020+0000 7f7578ff9640 1 -- 172.21.15.169:0/1680533788 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7564089960 con 0x7f757c153850 2023-10-26T01:39:32.027 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:32.020+0000 7f757bfff640 1 --2- 172.21.15.169:0/1680533788 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f755c06ae40 0x7f755c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:32.027 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:32.021+0000 7f757bfff640 1 --2- 172.21.15.169:0/1680533788 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f755c06ae40 0x7f755c06d300 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f756c009370 tx=0x7f756c0092c0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:32.028 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:32.024+0000 7f7578ff9640 1 -- 172.21.15.169:0/1680533788 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f757c153d70 con 0x7f757c153850 2023-10-26T01:39:32.287 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:32.285+0000 7f7582664640 1 -- 172.21.15.169:0/1680533788 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f757c154710 con 0x7f757c153850 2023-10-26T01:39:32.288 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:32.286+0000 7f7578ff9640 1 -- 172.21.15.169:0/1680533788 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1190 (secure 0 0 0) 0x7f757c154710 con 0x7f757c153850 2023-10-26T01:39:32.289 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:39:32.289 INFO:teuthology.orchestra.run.smithi169.stdout:{"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":549,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":151,"num_osds":8,"num_up_osds":8,"osd_up_since":1698283831,"num_in_osds":8,"osd_in_since":1698283827,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":56},{"state_name":"active+clean","count":41}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":7806704,"bytes_avail":1592193296,"bytes_total":1600000000,"inactive_pgs_ratio":0.57731956243515015},"fsmap":{"epoch":77,"id":9,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":9,"rank":0,"name":"c","status":"up:active","gid":8232}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":12,"modified":"2023-10-26T01:39:18.314476+0000","services":{"mds":{"daemons":{"summary":"","b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-10-26T01:39:32.291 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:32.288+0000 7f7582664640 1 -- 172.21.15.169:0/1680533788 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f755c06ae40 msgr2=0x7f755c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:32.291 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:32.288+0000 7f7582664640 1 --2- 172.21.15.169:0/1680533788 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f755c06ae40 0x7f755c06d300 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f756c009370 tx=0x7f756c0092c0 comp rx=0 tx=0).stop 2023-10-26T01:39:32.291 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:32.288+0000 7f7582664640 1 -- 172.21.15.169:0/1680533788 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f757c153850 msgr2=0x7f757c144450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:32.291 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:32.288+0000 7f7582664640 1 --2- 172.21.15.169:0/1680533788 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f757c153850 0x7f757c144450 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f756400d600 tx=0x7f756400da00 comp rx=0 tx=0).stop 2023-10-26T01:39:32.292 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:32.288+0000 7f7582664640 1 -- 172.21.15.169:0/1680533788 shutdown_connections 2023-10-26T01:39:32.292 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:32.288+0000 7f7582664640 1 --2- 172.21.15.169:0/1680533788 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f755c06ae40 0x7f755c06d300 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:32.293 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:32.288+0000 7f7582664640 1 --2- 172.21.15.169:0/1680533788 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f757c153850 0x7f757c144450 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:32.293 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:32.288+0000 7f7582664640 1 --2- 172.21.15.169:0/1680533788 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f757c1549f0 0x7f757c154e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:32.293 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:32.288+0000 7f7582664640 1 --2- 172.21.15.169:0/1680533788 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f75680051d0 0x7f757c14f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:32.293 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:32.288+0000 7f7582664640 1 -- 172.21.15.169:0/1680533788 >> 172.21.15.169:0/1680533788 conn(0x7f757c0af8a0 msgr2=0x7f757c156f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:32.294 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:32.289+0000 7f7582664640 1 -- 172.21.15.169:0/1680533788 shutdown_connections 2023-10-26T01:39:32.294 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:32.289+0000 7f7582664640 1 -- 172.21.15.169:0/1680533788 wait complete. 2023-10-26T01:39:32.294 INFO:tasks.ceph.mds.c.smithi169.stderr:2023-10-26T01:39:32.291+0000 7f814fac6640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-26T01:39:32.311 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-10-26T01:39:32.311 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-26T01:39:32.534 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:32.530+0000 7fbe13cc2640 1 -- 172.21.15.169:0/1230699442 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe0c1592d0 msgr2=0x7fbe0c0aa3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:32.534 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:32.530+0000 7fbe13cc2640 1 --2- 172.21.15.169:0/1230699442 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe0c1592d0 0x7fbe0c0aa3d0 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c0acdf0 tx=0x7fbe0802bcf0 comp rx=0 tx=0).stop 2023-10-26T01:39:32.534 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:32.531+0000 7fbe13cc2640 1 -- 172.21.15.169:0/1230699442 shutdown_connections 2023-10-26T01:39:32.534 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:32.531+0000 7fbe13cc2640 1 --2- 172.21.15.169:0/1230699442 >> v2:172.21.15.170:3300/0 conn(0x7fbe0c0cde00 0x7fbe0c0cbac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:32.535 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:32.531+0000 7fbe13cc2640 1 --2- 172.21.15.169:0/1230699442 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe0c1592d0 0x7fbe0c0aa3d0 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:32.535 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:32.531+0000 7fbe13cc2640 1 -- 172.21.15.169:0/1230699442 >> 172.21.15.169:0/1230699442 conn(0x7fbe0c069930 msgr2=0x7fbe0c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:32.535 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:32.531+0000 7fbe13cc2640 1 -- 172.21.15.169:0/1230699442 shutdown_connections 2023-10-26T01:39:32.535 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:32.531+0000 7fbe13cc2640 1 -- 172.21.15.169:0/1230699442 wait complete. 2023-10-26T01:39:32.538 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:32.533+0000 7fbe13cc2640 1 Processor -- start 2023-10-26T01:39:32.539 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:32.533+0000 7fbe13cc2640 1 -- start start 2023-10-26T01:39:32.539 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:32.534+0000 7fbe13cc2640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbe0c0ac930 0x7fbe0c06ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:32.539 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:32.534+0000 7fbe13cc2640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe0c0cde00 0x7fbe0c15a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:32.539 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:32.534+0000 7fbe13cc2640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbe0c1592d0 0x7fbe0c0c9070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:32.540 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:32.534+0000 7fbe13cc2640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe0c15ac20 con 0x7fbe0c0cde00 2023-10-26T01:39:32.540 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:32.534+0000 7fbe13cc2640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe0c070020 con 0x7fbe0c0ac930 2023-10-26T01:39:32.540 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:32.534+0000 7fbe13cc2640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe0c06f410 con 0x7fbe0c1592d0 2023-10-26T01:39:32.540 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:32.535+0000 7fbe1125d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe0c0cde00 0x7fbe0c15a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:32.541 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:32.535+0000 7fbe1125d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe0c0cde00 0x7fbe0c15a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:37466/0 (socket says 172.21.15.169:37466) 2023-10-26T01:39:32.541 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:32.535+0000 7fbe1125d640 1 -- 172.21.15.169:0/730781956 learned_addr learned my addr 172.21.15.169:0/730781956 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:39:32.541 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:32.535+0000 7fbe11a5e640 1 --2- 172.21.15.169:0/730781956 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbe0c0ac930 0x7fbe0c06ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:32.541 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:32.535+0000 7fbe1225f640 1 --2- 172.21.15.169:0/730781956 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbe0c1592d0 0x7fbe0c0c9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:32.542 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:32.535+0000 7fbe1125d640 1 -- 172.21.15.169:0/730781956 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbe0c1592d0 msgr2=0x7fbe0c0c9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:32.542 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:32.535+0000 7fbe1125d640 1 --2- 172.21.15.169:0/730781956 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbe0c1592d0 0x7fbe0c0c9070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:32.542 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:32.535+0000 7fbe1125d640 1 -- 172.21.15.169:0/730781956 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbe0c0ac930 msgr2=0x7fbe0c06ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:32.542 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:32.535+0000 7fbe1125d640 1 --2- 172.21.15.169:0/730781956 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbe0c0ac930 0x7fbe0c06ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:32.542 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:32.535+0000 7fbe1125d640 1 -- 172.21.15.169:0/730781956 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe08009490 con 0x7fbe0c0cde00 2023-10-26T01:39:32.543 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:32.535+0000 7fbe1125d640 1 --2- 172.21.15.169:0/730781956 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe0c0cde00 0x7fbe0c15a6e0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7fbdfc00c500 tx=0x7fbdfc00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:32.543 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:32.536+0000 7fbe027fc640 1 -- 172.21.15.169:0/730781956 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdfc012830 con 0x7fbe0c0cde00 2023-10-26T01:39:32.543 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:32.536+0000 7fbe027fc640 1 -- 172.21.15.169:0/730781956 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fbdfc012a80 con 0x7fbe0c0cde00 2023-10-26T01:39:32.543 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:32.536+0000 7fbe13cc2640 1 -- 172.21.15.169:0/730781956 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe0c06f6f0 con 0x7fbe0c0cde00 2023-10-26T01:39:32.544 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:32.536+0000 7fbe027fc640 1 -- 172.21.15.169:0/730781956 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdfc012e50 con 0x7fbe0c0cde00 2023-10-26T01:39:32.544 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:32.536+0000 7fbe13cc2640 1 -- 172.21.15.169:0/730781956 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe0c06fcb0 con 0x7fbe0c0cde00 2023-10-26T01:39:32.545 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:32.537+0000 7fbe13cc2640 1 -- 172.21.15.169:0/730781956 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe0c04db10 con 0x7fbe0c0cde00 2023-10-26T01:39:32.545 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:32.539+0000 7fbe027fc640 1 -- 172.21.15.169:0/730781956 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fbdfc01b400 con 0x7fbe0c0cde00 2023-10-26T01:39:32.545 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:32.540+0000 7fbe027fc640 1 --2- 172.21.15.169:0/730781956 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbdf006ae40 0x7fbdf006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:32.546 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:32.540+0000 7fbe027fc640 1 -- 172.21.15.169:0/730781956 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbdfc089aa0 con 0x7fbe0c0cde00 2023-10-26T01:39:32.546 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:32.540+0000 7fbe11a5e640 1 --2- 172.21.15.169:0/730781956 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbdf006ae40 0x7fbdf006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:32.546 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:32.541+0000 7fbe11a5e640 1 --2- 172.21.15.169:0/730781956 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbdf006ae40 0x7fbdf006d300 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fbe08036000 tx=0x7fbe08004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:32.548 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:32.545+0000 7fbe027fc640 1 -- 172.21.15.169:0/730781956 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbdfc054b60 con 0x7fbe0c0cde00 2023-10-26T01:39:32.734 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:32.731+0000 7fbe13cc2640 1 -- 172.21.15.169:0/730781956 --> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbe0c1e2230 con 0x7fbdf006ae40 2023-10-26T01:39:32.757 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:32.754+0000 7fbe027fc640 1 -- 172.21.15.169:0/730781956 <== mgr.4112 v2:172.21.15.170:6832/1187156918 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+282749 (secure 0 0 0) 0x7fbe0c1e2230 con 0x7fbdf006ae40 2023-10-26T01:39:32.761 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:39:32.763 INFO:teuthology.orchestra.run.smithi169.stderr:dumped all 2023-10-26T01:39:32.765 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:32.761+0000 7fbe13cc2640 1 -- 172.21.15.169:0/730781956 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbdf006ae40 msgr2=0x7fbdf006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:32.765 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:32.761+0000 7fbe13cc2640 1 --2- 172.21.15.169:0/730781956 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbdf006ae40 0x7fbdf006d300 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fbe08036000 tx=0x7fbe08004730 comp rx=0 tx=0).stop 2023-10-26T01:39:32.765 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:32.761+0000 7fbe13cc2640 1 -- 172.21.15.169:0/730781956 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe0c0cde00 msgr2=0x7fbe0c15a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:32.766 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:32.761+0000 7fbe13cc2640 1 --2- 172.21.15.169:0/730781956 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe0c0cde00 0x7fbe0c15a6e0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7fbdfc00c500 tx=0x7fbdfc00c9c0 comp rx=0 tx=0).stop 2023-10-26T01:39:32.766 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:32.761+0000 7fbe13cc2640 1 -- 172.21.15.169:0/730781956 shutdown_connections 2023-10-26T01:39:32.766 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:32.761+0000 7fbe13cc2640 1 --2- 172.21.15.169:0/730781956 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbdf006ae40 0x7fbdf006d300 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:32.766 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:32.761+0000 7fbe13cc2640 1 --2- 172.21.15.169:0/730781956 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe0c0cde00 0x7fbe0c15a6e0 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:32.767 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:32.761+0000 7fbe13cc2640 1 --2- 172.21.15.169:0/730781956 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbe0c1592d0 0x7fbe0c0c9070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:32.767 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:32.761+0000 7fbe13cc2640 1 --2- 172.21.15.169:0/730781956 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbe0c0ac930 0x7fbe0c06ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:32.767 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:32.761+0000 7fbe13cc2640 1 -- 172.21.15.169:0/730781956 >> 172.21.15.169:0/730781956 conn(0x7fbe0c069930 msgr2=0x7fbe0c157930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:32.767 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:32.761+0000 7fbe13cc2640 1 -- 172.21.15.169:0/730781956 shutdown_connections 2023-10-26T01:39:32.767 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:32.761+0000 7fbe13cc2640 1 -- 172.21.15.169:0/730781956 wait complete. 2023-10-26T01:39:32.785 INFO:teuthology.orchestra.run.smithi169.stdout:{"pg_ready":true,"pg_map":{"version":415,"stamp":"2023-10-26T01:39:32.320657+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":7742,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1554757,"statfs":{"total":1600000000,"available":1592071986,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.992088"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:27.328671+0000","last_change":"2023-10-26T01:39:26.351701+0000","last_active":"2023-10-26T01:39:27.328671+0000","last_peered":"2023-10-26T01:39:27.328671+0000","last_clean":"2023-10-26T01:39:27.328671+0000","last_became_active":"2023-10-26T01:39:26.351436+0000","last_became_peered":"2023-10-26T01:39:26.351436+0000","last_unstale":"2023-10-26T01:39:27.328671+0000","last_undegraded":"2023-10-26T01:39:27.328671+0000","last_fullsized":"2023-10-26T01:39:27.328671+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:12:28.860114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:28.142220+0000","last_change":"2023-10-26T01:39:26.341419+0000","last_active":"2023-10-26T01:39:28.142220+0000","last_peered":"2023-10-26T01:39:28.142220+0000","last_clean":"2023-10-26T01:39:28.142220+0000","last_became_active":"2023-10-26T01:39:26.340948+0000","last_became_peered":"2023-10-26T01:39:26.340948+0000","last_unstale":"2023-10-26T01:39:28.142220+0000","last_undegraded":"2023-10-26T01:39:28.142220+0000","last_fullsized":"2023-10-26T01:39:28.142220+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:09:11.583914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:27.638498+0000","last_change":"2023-10-26T01:39:26.350477+0000","last_active":"2023-10-26T01:39:27.638498+0000","last_peered":"2023-10-26T01:39:27.638498+0000","last_clean":"2023-10-26T01:39:27.638498+0000","last_became_active":"2023-10-26T01:39:26.349381+0000","last_became_peered":"2023-10-26T01:39:26.349381+0000","last_unstale":"2023-10-26T01:39:27.638498+0000","last_undegraded":"2023-10-26T01:39:27.638498+0000","last_fullsized":"2023-10-26T01:39:27.638498+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:23:48.210950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:27.327834+0000","last_change":"2023-10-26T01:39:26.354717+0000","last_active":"2023-10-26T01:39:27.327834+0000","last_peered":"2023-10-26T01:39:27.327834+0000","last_clean":"2023-10-26T01:39:27.327834+0000","last_became_active":"2023-10-26T01:39:26.352974+0000","last_became_peered":"2023-10-26T01:39:26.352974+0000","last_unstale":"2023-10-26T01:39:27.327834+0000","last_undegraded":"2023-10-26T01:39:27.327834+0000","last_fullsized":"2023-10-26T01:39:27.327834+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:02:53.802284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:28.140538+0000","last_change":"2023-10-26T01:39:26.345030+0000","last_active":"2023-10-26T01:39:28.140538+0000","last_peered":"2023-10-26T01:39:28.140538+0000","last_clean":"2023-10-26T01:39:28.140538+0000","last_became_active":"2023-10-26T01:39:26.344794+0000","last_became_peered":"2023-10-26T01:39:26.344794+0000","last_unstale":"2023-10-26T01:39:28.140538+0000","last_undegraded":"2023-10-26T01:39:28.140538+0000","last_fullsized":"2023-10-26T01:39:28.140538+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:41:16.234137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:27.330344+0000","last_change":"2023-10-26T01:39:26.353950+0000","last_active":"2023-10-26T01:39:27.330344+0000","last_peered":"2023-10-26T01:39:27.330344+0000","last_clean":"2023-10-26T01:39:27.330344+0000","last_became_active":"2023-10-26T01:39:26.353628+0000","last_became_peered":"2023-10-26T01:39:26.353628+0000","last_unstale":"2023-10-26T01:39:27.330344+0000","last_undegraded":"2023-10-26T01:39:27.330344+0000","last_fullsized":"2023-10-26T01:39:27.330344+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:21:57.499684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:27.330344+0000","last_change":"2023-10-26T01:39:26.341531+0000","last_active":"2023-10-26T01:39:27.330344+0000","last_peered":"2023-10-26T01:39:27.330344+0000","last_clean":"2023-10-26T01:39:27.330344+0000","last_became_active":"2023-10-26T01:39:26.341067+0000","last_became_peered":"2023-10-26T01:39:26.341067+0000","last_unstale":"2023-10-26T01:39:27.330344+0000","last_undegraded":"2023-10-26T01:39:27.330344+0000","last_fullsized":"2023-10-26T01:39:27.330344+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:43:29.618712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:27.638877+0000","last_change":"2023-10-26T01:39:26.350068+0000","last_active":"2023-10-26T01:39:27.638877+0000","last_peered":"2023-10-26T01:39:27.638877+0000","last_clean":"2023-10-26T01:39:27.638877+0000","last_became_active":"2023-10-26T01:39:26.347040+0000","last_became_peered":"2023-10-26T01:39:26.347040+0000","last_unstale":"2023-10-26T01:39:27.638877+0000","last_undegraded":"2023-10-26T01:39:27.638877+0000","last_fullsized":"2023-10-26T01:39:27.638877+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:53:50.829218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:27.638293+0000","last_change":"2023-10-26T01:39:26.338913+0000","last_active":"2023-10-26T01:39:27.638293+0000","last_peered":"2023-10-26T01:39:27.638293+0000","last_clean":"2023-10-26T01:39:27.638293+0000","last_became_active":"2023-10-26T01:39:26.337570+0000","last_became_peered":"2023-10-26T01:39:26.337570+0000","last_unstale":"2023-10-26T01:39:27.638293+0000","last_undegraded":"2023-10-26T01:39:27.638293+0000","last_fullsized":"2023-10-26T01:39:27.638293+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:35:50.504438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:28.142398+0000","last_change":"2023-10-26T01:39:26.348032+0000","last_active":"2023-10-26T01:39:28.142398+0000","last_peered":"2023-10-26T01:39:28.142398+0000","last_clean":"2023-10-26T01:39:28.142398+0000","last_became_active":"2023-10-26T01:39:26.347285+0000","last_became_peered":"2023-10-26T01:39:26.347285+0000","last_unstale":"2023-10-26T01:39:28.142398+0000","last_undegraded":"2023-10-26T01:39:28.142398+0000","last_fullsized":"2023-10-26T01:39:28.142398+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:36:46.428985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:28.142801+0000","last_change":"2023-10-26T01:39:26.342837+0000","last_active":"2023-10-26T01:39:28.142801+0000","last_peered":"2023-10-26T01:39:28.142801+0000","last_clean":"2023-10-26T01:39:28.142801+0000","last_became_active":"2023-10-26T01:39:26.340874+0000","last_became_peered":"2023-10-26T01:39:26.340874+0000","last_unstale":"2023-10-26T01:39:28.142801+0000","last_undegraded":"2023-10-26T01:39:28.142801+0000","last_fullsized":"2023-10-26T01:39:28.142801+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:15:06.099166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:27.638597+0000","last_change":"2023-10-26T01:39:26.348088+0000","last_active":"2023-10-26T01:39:27.638597+0000","last_peered":"2023-10-26T01:39:27.638597+0000","last_clean":"2023-10-26T01:39:27.638597+0000","last_became_active":"2023-10-26T01:39:26.344699+0000","last_became_peered":"2023-10-26T01:39:26.344699+0000","last_unstale":"2023-10-26T01:39:27.638597+0000","last_undegraded":"2023-10-26T01:39:27.638597+0000","last_fullsized":"2023-10-26T01:39:27.638597+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:24:22.043171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:27.327974+0000","last_change":"2023-10-26T01:39:26.352736+0000","last_active":"2023-10-26T01:39:27.327974+0000","last_peered":"2023-10-26T01:39:27.327974+0000","last_clean":"2023-10-26T01:39:27.327974+0000","last_became_active":"2023-10-26T01:39:26.352430+0000","last_became_peered":"2023-10-26T01:39:26.352430+0000","last_unstale":"2023-10-26T01:39:27.327974+0000","last_undegraded":"2023-10-26T01:39:27.327974+0000","last_fullsized":"2023-10-26T01:39:27.327974+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:32:21.374255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:28.139328+0000","last_change":"2023-10-26T01:39:26.354058+0000","last_active":"2023-10-26T01:39:28.139328+0000","last_peered":"2023-10-26T01:39:28.139328+0000","last_clean":"2023-10-26T01:39:28.139328+0000","last_became_active":"2023-10-26T01:39:26.353830+0000","last_became_peered":"2023-10-26T01:39:26.353830+0000","last_unstale":"2023-10-26T01:39:28.139328+0000","last_undegraded":"2023-10-26T01:39:28.139328+0000","last_fullsized":"2023-10-26T01:39:28.139328+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T01:41:39.459935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:27.329206+0000","last_change":"2023-10-26T01:39:26.355709+0000","last_active":"2023-10-26T01:39:27.329206+0000","last_peered":"2023-10-26T01:39:27.329206+0000","last_clean":"2023-10-26T01:39:27.329206+0000","last_became_active":"2023-10-26T01:39:26.355291+0000","last_became_peered":"2023-10-26T01:39:26.355291+0000","last_unstale":"2023-10-26T01:39:27.329206+0000","last_undegraded":"2023-10-26T01:39:27.329206+0000","last_fullsized":"2023-10-26T01:39:27.329206+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:11:54.780066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:28.141997+0000","last_change":"2023-10-26T01:39:26.355761+0000","last_active":"2023-10-26T01:39:28.141997+0000","last_peered":"2023-10-26T01:39:28.141997+0000","last_clean":"2023-10-26T01:39:28.141997+0000","last_became_active":"2023-10-26T01:39:26.355188+0000","last_became_peered":"2023-10-26T01:39:26.355188+0000","last_unstale":"2023-10-26T01:39:28.141997+0000","last_undegraded":"2023-10-26T01:39:28.141997+0000","last_fullsized":"2023-10-26T01:39:28.141997+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:16:33.825857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:27.331035+0000","last_change":"2023-10-26T01:39:26.344036+0000","last_active":"2023-10-26T01:39:27.331035+0000","last_peered":"2023-10-26T01:39:27.331035+0000","last_clean":"2023-10-26T01:39:27.331035+0000","last_became_active":"2023-10-26T01:39:26.343529+0000","last_became_peered":"2023-10-26T01:39:26.343529+0000","last_unstale":"2023-10-26T01:39:27.331035+0000","last_undegraded":"2023-10-26T01:39:27.331035+0000","last_fullsized":"2023-10-26T01:39:27.331035+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:18:22.694978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:28.140768+0000","last_change":"2023-10-26T01:39:26.348373+0000","last_active":"2023-10-26T01:39:28.140768+0000","last_peered":"2023-10-26T01:39:28.140768+0000","last_clean":"2023-10-26T01:39:28.140768+0000","last_became_active":"2023-10-26T01:39:26.346867+0000","last_became_peered":"2023-10-26T01:39:26.346867+0000","last_unstale":"2023-10-26T01:39:28.140768+0000","last_undegraded":"2023-10-26T01:39:28.140768+0000","last_fullsized":"2023-10-26T01:39:28.140768+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:06:57.001025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:28.140225+0000","last_change":"2023-10-26T01:39:26.346678+0000","last_active":"2023-10-26T01:39:28.140225+0000","last_peered":"2023-10-26T01:39:28.140225+0000","last_clean":"2023-10-26T01:39:28.140225+0000","last_became_active":"2023-10-26T01:39:26.345850+0000","last_became_peered":"2023-10-26T01:39:26.345850+0000","last_unstale":"2023-10-26T01:39:28.140225+0000","last_undegraded":"2023-10-26T01:39:28.140225+0000","last_fullsized":"2023-10-26T01:39:28.140225+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:30:53.479231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:28.140766+0000","last_change":"2023-10-26T01:39:26.351200+0000","last_active":"2023-10-26T01:39:28.140766+0000","last_peered":"2023-10-26T01:39:28.140766+0000","last_clean":"2023-10-26T01:39:28.140766+0000","last_became_active":"2023-10-26T01:39:26.350941+0000","last_became_peered":"2023-10-26T01:39:26.350941+0000","last_unstale":"2023-10-26T01:39:28.140766+0000","last_undegraded":"2023-10-26T01:39:28.140766+0000","last_fullsized":"2023-10-26T01:39:28.140766+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:32:30.699815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:28.139984+0000","last_change":"2023-10-26T01:39:26.345413+0000","last_active":"2023-10-26T01:39:28.139984+0000","last_peered":"2023-10-26T01:39:28.139984+0000","last_clean":"2023-10-26T01:39:28.139984+0000","last_became_active":"2023-10-26T01:39:26.344357+0000","last_became_peered":"2023-10-26T01:39:26.344357+0000","last_unstale":"2023-10-26T01:39:28.139984+0000","last_undegraded":"2023-10-26T01:39:28.139984+0000","last_fullsized":"2023-10-26T01:39:28.139984+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:23:37.929077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:28.142698+0000","last_change":"2023-10-26T01:39:26.340246+0000","last_active":"2023-10-26T01:39:28.142698+0000","last_peered":"2023-10-26T01:39:28.142698+0000","last_clean":"2023-10-26T01:39:28.142698+0000","last_became_active":"2023-10-26T01:39:26.339423+0000","last_became_peered":"2023-10-26T01:39:26.339423+0000","last_unstale":"2023-10-26T01:39:28.142698+0000","last_undegraded":"2023-10-26T01:39:28.142698+0000","last_fullsized":"2023-10-26T01:39:28.142698+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:57:08.525225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:27.639689+0000","last_change":"2023-10-26T01:39:26.349003+0000","last_active":"2023-10-26T01:39:27.639689+0000","last_peered":"2023-10-26T01:39:27.639689+0000","last_clean":"2023-10-26T01:39:27.639689+0000","last_became_active":"2023-10-26T01:39:26.346464+0000","last_became_peered":"2023-10-26T01:39:26.346464+0000","last_unstale":"2023-10-26T01:39:27.639689+0000","last_undegraded":"2023-10-26T01:39:27.639689+0000","last_fullsized":"2023-10-26T01:39:27.639689+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:32:30.699815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:28.140555+0000","last_change":"2023-10-26T01:39:26.346891+0000","last_active":"2023-10-26T01:39:28.140555+0000","last_peered":"2023-10-26T01:39:28.140555+0000","last_clean":"2023-10-26T01:39:28.140555+0000","last_became_active":"2023-10-26T01:39:26.346328+0000","last_became_peered":"2023-10-26T01:39:26.346328+0000","last_unstale":"2023-10-26T01:39:28.140555+0000","last_undegraded":"2023-10-26T01:39:28.140555+0000","last_fullsized":"2023-10-26T01:39:28.140555+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:45:55.590196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:28.143427+0000","last_change":"2023-10-26T01:39:26.347884+0000","last_active":"2023-10-26T01:39:28.143427+0000","last_peered":"2023-10-26T01:39:28.143427+0000","last_clean":"2023-10-26T01:39:28.143427+0000","last_became_active":"2023-10-26T01:39:26.345967+0000","last_became_peered":"2023-10-26T01:39:26.345967+0000","last_unstale":"2023-10-26T01:39:28.143427+0000","last_undegraded":"2023-10-26T01:39:28.143427+0000","last_fullsized":"2023-10-26T01:39:28.143427+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:31:25.915011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:28.142335+0000","last_change":"2023-10-26T01:39:26.355914+0000","last_active":"2023-10-26T01:39:28.142335+0000","last_peered":"2023-10-26T01:39:28.142335+0000","last_clean":"2023-10-26T01:39:28.142335+0000","last_became_active":"2023-10-26T01:39:26.355508+0000","last_became_peered":"2023-10-26T01:39:26.355508+0000","last_unstale":"2023-10-26T01:39:28.142335+0000","last_undegraded":"2023-10-26T01:39:28.142335+0000","last_fullsized":"2023-10-26T01:39:28.142335+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:11:11.775125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:27.328041+0000","last_change":"2023-10-26T01:39:26.350921+0000","last_active":"2023-10-26T01:39:27.328041+0000","last_peered":"2023-10-26T01:39:27.328041+0000","last_clean":"2023-10-26T01:39:27.328041+0000","last_became_active":"2023-10-26T01:39:26.350660+0000","last_became_peered":"2023-10-26T01:39:26.350660+0000","last_unstale":"2023-10-26T01:39:27.328041+0000","last_undegraded":"2023-10-26T01:39:27.328041+0000","last_fullsized":"2023-10-26T01:39:27.328041+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:56:14.952742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:27.328045+0000","last_change":"2023-10-26T01:39:26.342982+0000","last_active":"2023-10-26T01:39:27.328045+0000","last_peered":"2023-10-26T01:39:27.328045+0000","last_clean":"2023-10-26T01:39:27.328045+0000","last_became_active":"2023-10-26T01:39:26.342537+0000","last_became_peered":"2023-10-26T01:39:26.342537+0000","last_unstale":"2023-10-26T01:39:27.328045+0000","last_undegraded":"2023-10-26T01:39:27.328045+0000","last_fullsized":"2023-10-26T01:39:27.328045+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:46:02.179502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:27.328100+0000","last_change":"2023-10-26T01:39:26.355396+0000","last_active":"2023-10-26T01:39:27.328100+0000","last_peered":"2023-10-26T01:39:27.328100+0000","last_clean":"2023-10-26T01:39:27.328100+0000","last_became_active":"2023-10-26T01:39:26.354952+0000","last_became_peered":"2023-10-26T01:39:26.354952+0000","last_unstale":"2023-10-26T01:39:27.328100+0000","last_undegraded":"2023-10-26T01:39:27.328100+0000","last_fullsized":"2023-10-26T01:39:27.328100+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:08:34.213156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:27.329367+0000","last_change":"2023-10-26T01:39:26.350190+0000","last_active":"2023-10-26T01:39:27.329367+0000","last_peered":"2023-10-26T01:39:27.329367+0000","last_clean":"2023-10-26T01:39:27.329367+0000","last_became_active":"2023-10-26T01:39:26.349816+0000","last_became_peered":"2023-10-26T01:39:26.349816+0000","last_unstale":"2023-10-26T01:39:27.329367+0000","last_undegraded":"2023-10-26T01:39:27.329367+0000","last_fullsized":"2023-10-26T01:39:27.329367+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:01:41.045976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:28.139502+0000","last_change":"2023-10-26T01:39:26.352978+0000","last_active":"2023-10-26T01:39:28.139502+0000","last_peered":"2023-10-26T01:39:28.139502+0000","last_clean":"2023-10-26T01:39:28.139502+0000","last_became_active":"2023-10-26T01:39:26.352497+0000","last_became_peered":"2023-10-26T01:39:26.352497+0000","last_unstale":"2023-10-26T01:39:28.139502+0000","last_undegraded":"2023-10-26T01:39:28.139502+0000","last_fullsized":"2023-10-26T01:39:28.139502+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:18:22.694978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:27.330717+0000","last_change":"2023-10-26T01:39:26.336994+0000","last_active":"2023-10-26T01:39:27.330717+0000","last_peered":"2023-10-26T01:39:27.330717+0000","last_clean":"2023-10-26T01:39:27.330717+0000","last_became_active":"2023-10-26T01:39:26.336376+0000","last_became_peered":"2023-10-26T01:39:26.336376+0000","last_unstale":"2023-10-26T01:39:27.330717+0000","last_undegraded":"2023-10-26T01:39:27.330717+0000","last_fullsized":"2023-10-26T01:39:27.330717+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:49:45.366235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:28.140822+0000","last_change":"2023-10-26T01:39:26.344022+0000","last_active":"2023-10-26T01:39:28.140822+0000","last_peered":"2023-10-26T01:39:28.140822+0000","last_clean":"2023-10-26T01:39:28.140822+0000","last_became_active":"2023-10-26T01:39:26.341874+0000","last_became_peered":"2023-10-26T01:39:26.341874+0000","last_unstale":"2023-10-26T01:39:28.140822+0000","last_undegraded":"2023-10-26T01:39:28.140822+0000","last_fullsized":"2023-10-26T01:39:28.140822+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:33:00.360810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"150'4","reported_seq":16,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-26T01:39:26.363349+0000","last_change":"2023-10-26T01:39:25.347523+0000","last_active":"2023-10-26T01:39:26.363349+0000","last_peered":"2023-10-26T01:39:26.363349+0000","last_clean":"2023-10-26T01:39:26.363349+0000","last_became_active":"2023-10-26T01:39:25.346388+0000","last_became_peered":"2023-10-26T01:39:25.346388+0000","last_unstale":"2023-10-26T01:39:26.363349+0000","last_undegraded":"2023-10-26T01:39:26.363349+0000","last_fullsized":"2023-10-26T01:39:26.363349+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:03:43.273094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:27.639190+0000","last_change":"2023-10-26T01:39:26.354015+0000","last_active":"2023-10-26T01:39:27.639190+0000","last_peered":"2023-10-26T01:39:27.639190+0000","last_clean":"2023-10-26T01:39:27.639190+0000","last_became_active":"2023-10-26T01:39:26.353804+0000","last_became_peered":"2023-10-26T01:39:26.353804+0000","last_unstale":"2023-10-26T01:39:27.639190+0000","last_undegraded":"2023-10-26T01:39:27.639190+0000","last_fullsized":"2023-10-26T01:39:27.639190+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:06:57.001025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"150'1","reported_seq":13,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-26T01:39:26.364024+0000","last_change":"2023-10-26T01:39:25.347683+0000","last_active":"2023-10-26T01:39:26.364024+0000","last_peered":"2023-10-26T01:39:26.364024+0000","last_clean":"2023-10-26T01:39:26.364024+0000","last_became_active":"2023-10-26T01:39:25.346998+0000","last_became_peered":"2023-10-26T01:39:25.346998+0000","last_unstale":"2023-10-26T01:39:26.364024+0000","last_undegraded":"2023-10-26T01:39:26.364024+0000","last_fullsized":"2023-10-26T01:39:26.364024+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:06:50.840083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:28.139786+0000","last_change":"2023-10-26T01:39:26.345123+0000","last_active":"2023-10-26T01:39:28.139786+0000","last_peered":"2023-10-26T01:39:28.139786+0000","last_clean":"2023-10-26T01:39:28.139786+0000","last_became_active":"2023-10-26T01:39:26.343161+0000","last_became_peered":"2023-10-26T01:39:26.343161+0000","last_unstale":"2023-10-26T01:39:28.139786+0000","last_undegraded":"2023-10-26T01:39:28.139786+0000","last_fullsized":"2023-10-26T01:39:28.139786+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:03:19.428986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"150'2","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-26T01:39:26.364267+0000","last_change":"2023-10-26T01:39:25.352802+0000","last_active":"2023-10-26T01:39:26.364267+0000","last_peered":"2023-10-26T01:39:26.364267+0000","last_clean":"2023-10-26T01:39:26.364267+0000","last_became_active":"2023-10-26T01:39:25.352537+0000","last_became_peered":"2023-10-26T01:39:25.352537+0000","last_unstale":"2023-10-26T01:39:26.364267+0000","last_undegraded":"2023-10-26T01:39:26.364267+0000","last_fullsized":"2023-10-26T01:39:26.364267+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:44:22.214795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:28.140858+0000","last_change":"2023-10-26T01:39:26.347952+0000","last_active":"2023-10-26T01:39:28.140858+0000","last_peered":"2023-10-26T01:39:28.140858+0000","last_clean":"2023-10-26T01:39:28.140858+0000","last_became_active":"2023-10-26T01:39:26.347693+0000","last_became_peered":"2023-10-26T01:39:26.347693+0000","last_unstale":"2023-10-26T01:39:28.140858+0000","last_undegraded":"2023-10-26T01:39:28.140858+0000","last_fullsized":"2023-10-26T01:39:28.140858+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:11:20.000707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"150'2","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-26T01:39:26.362995+0000","last_change":"2023-10-26T01:39:25.347942+0000","last_active":"2023-10-26T01:39:26.362995+0000","last_peered":"2023-10-26T01:39:26.362995+0000","last_clean":"2023-10-26T01:39:26.362995+0000","last_became_active":"2023-10-26T01:39:25.347373+0000","last_became_peered":"2023-10-26T01:39:25.347373+0000","last_unstale":"2023-10-26T01:39:26.362995+0000","last_undegraded":"2023-10-26T01:39:26.362995+0000","last_fullsized":"2023-10-26T01:39:26.362995+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:52:46.723602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:28.141108+0000","last_change":"2023-10-26T01:39:26.348640+0000","last_active":"2023-10-26T01:39:28.141108+0000","last_peered":"2023-10-26T01:39:28.141108+0000","last_clean":"2023-10-26T01:39:28.141108+0000","last_became_active":"2023-10-26T01:39:26.348127+0000","last_became_peered":"2023-10-26T01:39:26.348127+0000","last_unstale":"2023-10-26T01:39:28.141108+0000","last_undegraded":"2023-10-26T01:39:28.141108+0000","last_fullsized":"2023-10-26T01:39:28.141108+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:45:34.051362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"150'1","reported_seq":13,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-26T01:39:26.362430+0000","last_change":"2023-10-26T01:39:25.351159+0000","last_active":"2023-10-26T01:39:26.362430+0000","last_peered":"2023-10-26T01:39:26.362430+0000","last_clean":"2023-10-26T01:39:26.362430+0000","last_became_active":"2023-10-26T01:39:25.349181+0000","last_became_peered":"2023-10-26T01:39:25.349181+0000","last_unstale":"2023-10-26T01:39:26.362430+0000","last_undegraded":"2023-10-26T01:39:26.362430+0000","last_fullsized":"2023-10-26T01:39:26.362430+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:01:46.130180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:27.331253+0000","last_change":"2023-10-26T01:39:26.341329+0000","last_active":"2023-10-26T01:39:27.331253+0000","last_peered":"2023-10-26T01:39:27.331253+0000","last_clean":"2023-10-26T01:39:27.331253+0000","last_became_active":"2023-10-26T01:39:26.340693+0000","last_became_peered":"2023-10-26T01:39:26.340693+0000","last_unstale":"2023-10-26T01:39:27.331253+0000","last_undegraded":"2023-10-26T01:39:27.331253+0000","last_fullsized":"2023-10-26T01:39:27.331253+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:25:55.556501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"150'1","reported_seq":13,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-26T01:39:26.955278+0000","last_change":"2023-10-26T01:39:25.352234+0000","last_active":"2023-10-26T01:39:26.955278+0000","last_peered":"2023-10-26T01:39:26.955278+0000","last_clean":"2023-10-26T01:39:26.955278+0000","last_became_active":"2023-10-26T01:39:25.351930+0000","last_became_peered":"2023-10-26T01:39:25.351930+0000","last_unstale":"2023-10-26T01:39:26.955278+0000","last_undegraded":"2023-10-26T01:39:26.955278+0000","last_fullsized":"2023-10-26T01:39:26.955278+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:23:47.193891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:27.638488+0000","last_change":"2023-10-26T01:39:26.347620+0000","last_active":"2023-10-26T01:39:27.638488+0000","last_peered":"2023-10-26T01:39:27.638488+0000","last_clean":"2023-10-26T01:39:27.638488+0000","last_became_active":"2023-10-26T01:39:26.347251+0000","last_became_peered":"2023-10-26T01:39:26.347251+0000","last_unstale":"2023-10-26T01:39:27.638488+0000","last_undegraded":"2023-10-26T01:39:27.638488+0000","last_fullsized":"2023-10-26T01:39:27.638488+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:15:35.499430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-26T01:39:26.322931+0000","last_change":"2023-10-26T01:39:25.345280+0000","last_active":"2023-10-26T01:39:26.322931+0000","last_peered":"2023-10-26T01:39:26.322931+0000","last_clean":"2023-10-26T01:39:26.322931+0000","last_became_active":"2023-10-26T01:39:25.344841+0000","last_became_peered":"2023-10-26T01:39:25.344841+0000","last_unstale":"2023-10-26T01:39:26.322931+0000","last_undegraded":"2023-10-26T01:39:26.322931+0000","last_fullsized":"2023-10-26T01:39:26.322931+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:08:42.045168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:28.139704+0000","last_change":"2023-10-26T01:39:26.337467+0000","last_active":"2023-10-26T01:39:28.139704+0000","last_peered":"2023-10-26T01:39:28.139704+0000","last_clean":"2023-10-26T01:39:28.139704+0000","last_became_active":"2023-10-26T01:39:26.336744+0000","last_became_peered":"2023-10-26T01:39:26.336744+0000","last_unstale":"2023-10-26T01:39:28.139704+0000","last_undegraded":"2023-10-26T01:39:28.139704+0000","last_fullsized":"2023-10-26T01:39:28.139704+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:31:48.033303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-26T01:39:26.333002+0000","last_change":"2023-10-26T01:39:25.355147+0000","last_active":"2023-10-26T01:39:26.333002+0000","last_peered":"2023-10-26T01:39:26.333002+0000","last_clean":"2023-10-26T01:39:26.333002+0000","last_became_active":"2023-10-26T01:39:25.354564+0000","last_became_peered":"2023-10-26T01:39:25.354564+0000","last_unstale":"2023-10-26T01:39:26.333002+0000","last_undegraded":"2023-10-26T01:39:26.333002+0000","last_fullsized":"2023-10-26T01:39:26.333002+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:48:12.023136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:27.328575+0000","last_change":"2023-10-26T01:39:26.354194+0000","last_active":"2023-10-26T01:39:27.328575+0000","last_peered":"2023-10-26T01:39:27.328575+0000","last_clean":"2023-10-26T01:39:27.328575+0000","last_became_active":"2023-10-26T01:39:26.353639+0000","last_became_peered":"2023-10-26T01:39:26.353639+0000","last_unstale":"2023-10-26T01:39:27.328575+0000","last_undegraded":"2023-10-26T01:39:27.328575+0000","last_fullsized":"2023-10-26T01:39:27.328575+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:07:39.915731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-26T01:39:26.321608+0000","last_change":"2023-10-26T01:39:25.353938+0000","last_active":"2023-10-26T01:39:26.321608+0000","last_peered":"2023-10-26T01:39:26.321608+0000","last_clean":"2023-10-26T01:39:26.321608+0000","last_became_active":"2023-10-26T01:39:25.353704+0000","last_became_peered":"2023-10-26T01:39:25.353704+0000","last_unstale":"2023-10-26T01:39:26.321608+0000","last_undegraded":"2023-10-26T01:39:26.321608+0000","last_fullsized":"2023-10-26T01:39:26.321608+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:01:46.130180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:27.328408+0000","last_change":"2023-10-26T01:39:26.345228+0000","last_active":"2023-10-26T01:39:27.328408+0000","last_peered":"2023-10-26T01:39:27.328408+0000","last_clean":"2023-10-26T01:39:27.328408+0000","last_became_active":"2023-10-26T01:39:26.344986+0000","last_became_peered":"2023-10-26T01:39:26.344986+0000","last_unstale":"2023-10-26T01:39:27.328408+0000","last_undegraded":"2023-10-26T01:39:27.328408+0000","last_fullsized":"2023-10-26T01:39:27.328408+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:30:57.828517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-26T01:39:26.955344+0000","last_change":"2023-10-26T01:39:25.353364+0000","last_active":"2023-10-26T01:39:26.955344+0000","last_peered":"2023-10-26T01:39:26.955344+0000","last_clean":"2023-10-26T01:39:26.955344+0000","last_became_active":"2023-10-26T01:39:25.353129+0000","last_became_peered":"2023-10-26T01:39:25.353129+0000","last_unstale":"2023-10-26T01:39:26.955344+0000","last_undegraded":"2023-10-26T01:39:26.955344+0000","last_fullsized":"2023-10-26T01:39:26.955344+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:07:24.731619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:28.140226+0000","last_change":"2023-10-26T01:39:26.336002+0000","last_active":"2023-10-26T01:39:28.140226+0000","last_peered":"2023-10-26T01:39:28.140226+0000","last_clean":"2023-10-26T01:39:28.140226+0000","last_became_active":"2023-10-26T01:39:26.335418+0000","last_became_peered":"2023-10-26T01:39:26.335418+0000","last_unstale":"2023-10-26T01:39:28.140226+0000","last_undegraded":"2023-10-26T01:39:28.140226+0000","last_fullsized":"2023-10-26T01:39:28.140226+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:17:44.949060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"150'3","reported_seq":15,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-26T01:39:26.364835+0000","last_change":"2023-10-26T01:39:25.352244+0000","last_active":"2023-10-26T01:39:26.364835+0000","last_peered":"2023-10-26T01:39:26.364835+0000","last_clean":"2023-10-26T01:39:26.364835+0000","last_became_active":"2023-10-26T01:39:25.351989+0000","last_became_peered":"2023-10-26T01:39:25.351989+0000","last_unstale":"2023-10-26T01:39:26.364835+0000","last_undegraded":"2023-10-26T01:39:26.364835+0000","last_fullsized":"2023-10-26T01:39:26.364835+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:02:04.396694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:28.139651+0000","last_change":"2023-10-26T01:39:26.348419+0000","last_active":"2023-10-26T01:39:28.139651+0000","last_peered":"2023-10-26T01:39:28.139651+0000","last_clean":"2023-10-26T01:39:28.139651+0000","last_became_active":"2023-10-26T01:39:26.348038+0000","last_became_peered":"2023-10-26T01:39:26.348038+0000","last_unstale":"2023-10-26T01:39:28.139651+0000","last_undegraded":"2023-10-26T01:39:28.139651+0000","last_fullsized":"2023-10-26T01:39:28.139651+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:07:52.564388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"150'6","reported_seq":18,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-26T01:39:26.367176+0000","last_change":"2023-10-26T01:39:25.354828+0000","last_active":"2023-10-26T01:39:26.367176+0000","last_peered":"2023-10-26T01:39:26.367176+0000","last_clean":"2023-10-26T01:39:26.367176+0000","last_became_active":"2023-10-26T01:39:25.354185+0000","last_became_peered":"2023-10-26T01:39:25.354185+0000","last_unstale":"2023-10-26T01:39:26.367176+0000","last_undegraded":"2023-10-26T01:39:26.367176+0000","last_fullsized":"2023-10-26T01:39:26.367176+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:19:26.298128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:27.329619+0000","last_change":"2023-10-26T01:39:26.350033+0000","last_active":"2023-10-26T01:39:27.329619+0000","last_peered":"2023-10-26T01:39:27.329619+0000","last_clean":"2023-10-26T01:39:27.329619+0000","last_became_active":"2023-10-26T01:39:26.349373+0000","last_became_peered":"2023-10-26T01:39:26.349373+0000","last_unstale":"2023-10-26T01:39:27.329619+0000","last_undegraded":"2023-10-26T01:39:27.329619+0000","last_fullsized":"2023-10-26T01:39:27.329619+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:06:51.857142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-26T01:39:26.341127+0000","last_change":"2023-10-26T01:39:25.350711+0000","last_active":"2023-10-26T01:39:26.341127+0000","last_peered":"2023-10-26T01:39:26.341127+0000","last_clean":"2023-10-26T01:39:26.341127+0000","last_became_active":"2023-10-26T01:39:25.349869+0000","last_became_peered":"2023-10-26T01:39:25.349869+0000","last_unstale":"2023-10-26T01:39:26.341127+0000","last_undegraded":"2023-10-26T01:39:26.341127+0000","last_fullsized":"2023-10-26T01:39:26.341127+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:08:07.790237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:28.139599+0000","last_change":"2023-10-26T01:39:26.351491+0000","last_active":"2023-10-26T01:39:28.139599+0000","last_peered":"2023-10-26T01:39:28.139599+0000","last_clean":"2023-10-26T01:39:28.139599+0000","last_became_active":"2023-10-26T01:39:26.351251+0000","last_became_peered":"2023-10-26T01:39:26.351251+0000","last_unstale":"2023-10-26T01:39:28.139599+0000","last_undegraded":"2023-10-26T01:39:28.139599+0000","last_fullsized":"2023-10-26T01:39:28.139599+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:28:48.296290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-26T01:39:26.329630+0000","last_change":"2023-10-26T01:39:25.342525+0000","last_active":"2023-10-26T01:39:26.329630+0000","last_peered":"2023-10-26T01:39:26.329630+0000","last_clean":"2023-10-26T01:39:26.329630+0000","last_became_active":"2023-10-26T01:39:25.341183+0000","last_became_peered":"2023-10-26T01:39:25.341183+0000","last_unstale":"2023-10-26T01:39:26.329630+0000","last_undegraded":"2023-10-26T01:39:26.329630+0000","last_fullsized":"2023-10-26T01:39:26.329630+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:46:01.162443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:27.638820+0000","last_change":"2023-10-26T01:39:26.342999+0000","last_active":"2023-10-26T01:39:27.638820+0000","last_peered":"2023-10-26T01:39:27.638820+0000","last_clean":"2023-10-26T01:39:27.638820+0000","last_became_active":"2023-10-26T01:39:26.342725+0000","last_became_peered":"2023-10-26T01:39:26.342725+0000","last_unstale":"2023-10-26T01:39:27.638820+0000","last_undegraded":"2023-10-26T01:39:27.638820+0000","last_fullsized":"2023-10-26T01:39:27.638820+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:19:43.588241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"150'8","reported_seq":20,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-26T01:39:26.366328+0000","last_change":"2023-10-26T01:39:25.355183+0000","last_active":"2023-10-26T01:39:26.366328+0000","last_peered":"2023-10-26T01:39:26.366328+0000","last_clean":"2023-10-26T01:39:26.366328+0000","last_became_active":"2023-10-26T01:39:25.354751+0000","last_became_peered":"2023-10-26T01:39:25.354751+0000","last_unstale":"2023-10-26T01:39:26.366328+0000","last_undegraded":"2023-10-26T01:39:26.366328+0000","last_fullsized":"2023-10-26T01:39:26.366328+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:07:24.731619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:27.638713+0000","last_change":"2023-10-26T01:39:26.354040+0000","last_active":"2023-10-26T01:39:27.638713+0000","last_peered":"2023-10-26T01:39:27.638713+0000","last_clean":"2023-10-26T01:39:27.638713+0000","last_became_active":"2023-10-26T01:39:26.353736+0000","last_became_peered":"2023-10-26T01:39:26.353736+0000","last_unstale":"2023-10-26T01:39:27.638713+0000","last_undegraded":"2023-10-26T01:39:27.638713+0000","last_fullsized":"2023-10-26T01:39:27.638713+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:37:07.681720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"150'1","reported_seq":13,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-26T01:39:26.362952+0000","last_change":"2023-10-26T01:39:25.350279+0000","last_active":"2023-10-26T01:39:26.362952+0000","last_peered":"2023-10-26T01:39:26.362952+0000","last_clean":"2023-10-26T01:39:26.362952+0000","last_became_active":"2023-10-26T01:39:25.349301+0000","last_became_peered":"2023-10-26T01:39:25.349301+0000","last_unstale":"2023-10-26T01:39:26.362952+0000","last_undegraded":"2023-10-26T01:39:26.362952+0000","last_fullsized":"2023-10-26T01:39:26.362952+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:09:55.848777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:27.328126+0000","last_change":"2023-10-26T01:39:26.340528+0000","last_active":"2023-10-26T01:39:27.328126+0000","last_peered":"2023-10-26T01:39:27.328126+0000","last_clean":"2023-10-26T01:39:27.328126+0000","last_became_active":"2023-10-26T01:39:26.339638+0000","last_became_peered":"2023-10-26T01:39:26.339638+0000","last_unstale":"2023-10-26T01:39:27.328126+0000","last_undegraded":"2023-10-26T01:39:27.328126+0000","last_fullsized":"2023-10-26T01:39:27.328126+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:52:47.740661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"150'2","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-26T01:39:26.366258+0000","last_change":"2023-10-26T01:39:25.355508+0000","last_active":"2023-10-26T01:39:26.366258+0000","last_peered":"2023-10-26T01:39:26.366258+0000","last_clean":"2023-10-26T01:39:26.366258+0000","last_became_active":"2023-10-26T01:39:25.355254+0000","last_became_peered":"2023-10-26T01:39:25.355254+0000","last_unstale":"2023-10-26T01:39:26.366258+0000","last_undegraded":"2023-10-26T01:39:26.366258+0000","last_fullsized":"2023-10-26T01:39:26.366258+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:17:27.317892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:28.141591+0000","last_change":"2023-10-26T01:39:26.354034+0000","last_active":"2023-10-26T01:39:28.141591+0000","last_peered":"2023-10-26T01:39:28.141591+0000","last_clean":"2023-10-26T01:39:28.141591+0000","last_became_active":"2023-10-26T01:39:26.353811+0000","last_became_peered":"2023-10-26T01:39:26.353811+0000","last_unstale":"2023-10-26T01:39:28.141591+0000","last_undegraded":"2023-10-26T01:39:28.141591+0000","last_fullsized":"2023-10-26T01:39:28.141591+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:01:32.879490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"150'1","reported_seq":13,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-26T01:39:26.364322+0000","last_change":"2023-10-26T01:39:25.348929+0000","last_active":"2023-10-26T01:39:26.364322+0000","last_peered":"2023-10-26T01:39:26.364322+0000","last_clean":"2023-10-26T01:39:26.364322+0000","last_became_active":"2023-10-26T01:39:25.348637+0000","last_became_peered":"2023-10-26T01:39:25.348637+0000","last_unstale":"2023-10-26T01:39:26.364322+0000","last_undegraded":"2023-10-26T01:39:26.364322+0000","last_fullsized":"2023-10-26T01:39:26.364322+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:29:59.562191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:28.140859+0000","last_change":"2023-10-26T01:39:26.347586+0000","last_active":"2023-10-26T01:39:28.140859+0000","last_peered":"2023-10-26T01:39:28.140859+0000","last_clean":"2023-10-26T01:39:28.140859+0000","last_became_active":"2023-10-26T01:39:26.345838+0000","last_became_peered":"2023-10-26T01:39:26.345838+0000","last_unstale":"2023-10-26T01:39:28.140859+0000","last_undegraded":"2023-10-26T01:39:28.140859+0000","last_fullsized":"2023-10-26T01:39:28.140859+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:38:29.849919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-26T01:39:26.319713+0000","last_change":"2023-10-26T01:39:25.352675+0000","last_active":"2023-10-26T01:39:26.319713+0000","last_peered":"2023-10-26T01:39:26.319713+0000","last_clean":"2023-10-26T01:39:26.319713+0000","last_became_active":"2023-10-26T01:39:25.352188+0000","last_became_peered":"2023-10-26T01:39:25.352188+0000","last_unstale":"2023-10-26T01:39:26.319713+0000","last_undegraded":"2023-10-26T01:39:26.319713+0000","last_fullsized":"2023-10-26T01:39:26.319713+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:12:27.843055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:27.328235+0000","last_change":"2023-10-26T01:39:26.353259+0000","last_active":"2023-10-26T01:39:27.328235+0000","last_peered":"2023-10-26T01:39:27.328235+0000","last_clean":"2023-10-26T01:39:27.328235+0000","last_became_active":"2023-10-26T01:39:26.352540+0000","last_became_peered":"2023-10-26T01:39:26.352540+0000","last_unstale":"2023-10-26T01:39:27.328235+0000","last_undegraded":"2023-10-26T01:39:27.328235+0000","last_fullsized":"2023-10-26T01:39:27.328235+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:28:47.729543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"150'2","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-26T01:39:26.364335+0000","last_change":"2023-10-26T01:39:25.353938+0000","last_active":"2023-10-26T01:39:26.364335+0000","last_peered":"2023-10-26T01:39:26.364335+0000","last_clean":"2023-10-26T01:39:26.364335+0000","last_became_active":"2023-10-26T01:39:25.353665+0000","last_became_peered":"2023-10-26T01:39:25.353665+0000","last_unstale":"2023-10-26T01:39:26.364335+0000","last_undegraded":"2023-10-26T01:39:26.364335+0000","last_fullsized":"2023-10-26T01:39:26.364335+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T01:51:26.281651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:27.330257+0000","last_change":"2023-10-26T01:39:26.350893+0000","last_active":"2023-10-26T01:39:27.330257+0000","last_peered":"2023-10-26T01:39:27.330257+0000","last_clean":"2023-10-26T01:39:27.330257+0000","last_became_active":"2023-10-26T01:39:26.350621+0000","last_became_peered":"2023-10-26T01:39:26.350621+0000","last_unstale":"2023-10-26T01:39:27.330257+0000","last_undegraded":"2023-10-26T01:39:27.330257+0000","last_fullsized":"2023-10-26T01:39:27.330257+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:21:44.926700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-26T01:39:26.955203+0000","last_change":"2023-10-26T01:39:25.355042+0000","last_active":"2023-10-26T01:39:26.955203+0000","last_peered":"2023-10-26T01:39:26.955203+0000","last_clean":"2023-10-26T01:39:26.955203+0000","last_became_active":"2023-10-26T01:39:25.354798+0000","last_became_peered":"2023-10-26T01:39:25.354798+0000","last_unstale":"2023-10-26T01:39:26.955203+0000","last_undegraded":"2023-10-26T01:39:26.955203+0000","last_fullsized":"2023-10-26T01:39:26.955203+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:34:40.255257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:28.140269+0000","last_change":"2023-10-26T01:39:26.347810+0000","last_active":"2023-10-26T01:39:28.140269+0000","last_peered":"2023-10-26T01:39:28.140269+0000","last_clean":"2023-10-26T01:39:28.140269+0000","last_became_active":"2023-10-26T01:39:26.347135+0000","last_became_peered":"2023-10-26T01:39:26.347135+0000","last_unstale":"2023-10-26T01:39:28.140269+0000","last_undegraded":"2023-10-26T01:39:28.140269+0000","last_fullsized":"2023-10-26T01:39:28.140269+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:13:17.935865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-26T01:39:26.955481+0000","last_change":"2023-10-26T01:39:25.355043+0000","last_active":"2023-10-26T01:39:26.955481+0000","last_peered":"2023-10-26T01:39:26.955481+0000","last_clean":"2023-10-26T01:39:26.955481+0000","last_became_active":"2023-10-26T01:39:25.354518+0000","last_became_peered":"2023-10-26T01:39:25.354518+0000","last_unstale":"2023-10-26T01:39:26.955481+0000","last_undegraded":"2023-10-26T01:39:26.955481+0000","last_fullsized":"2023-10-26T01:39:26.955481+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:37:06.664661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:28.140159+0000","last_change":"2023-10-26T01:39:26.352876+0000","last_active":"2023-10-26T01:39:28.140159+0000","last_peered":"2023-10-26T01:39:28.140159+0000","last_clean":"2023-10-26T01:39:28.140159+0000","last_became_active":"2023-10-26T01:39:26.352556+0000","last_became_peered":"2023-10-26T01:39:26.352556+0000","last_unstale":"2023-10-26T01:39:28.140159+0000","last_undegraded":"2023-10-26T01:39:28.140159+0000","last_fullsized":"2023-10-26T01:39:28.140159+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:48:10.843465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-26T01:39:26.335781+0000","last_change":"2023-10-26T01:39:25.355568+0000","last_active":"2023-10-26T01:39:26.335781+0000","last_peered":"2023-10-26T01:39:26.335781+0000","last_clean":"2023-10-26T01:39:26.335781+0000","last_became_active":"2023-10-26T01:39:25.354482+0000","last_became_peered":"2023-10-26T01:39:25.354482+0000","last_unstale":"2023-10-26T01:39:26.335781+0000","last_undegraded":"2023-10-26T01:39:26.335781+0000","last_fullsized":"2023-10-26T01:39:26.335781+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:54:16.002490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:27.330836+0000","last_change":"2023-10-26T01:39:26.329595+0000","last_active":"2023-10-26T01:39:27.330836+0000","last_peered":"2023-10-26T01:39:27.330836+0000","last_clean":"2023-10-26T01:39:27.330836+0000","last_became_active":"2023-10-26T01:39:26.329022+0000","last_became_peered":"2023-10-26T01:39:26.329022+0000","last_unstale":"2023-10-26T01:39:27.330836+0000","last_undegraded":"2023-10-26T01:39:27.330836+0000","last_fullsized":"2023-10-26T01:39:27.330836+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:23:29.662601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"150'2","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-26T01:39:26.362529+0000","last_change":"2023-10-26T01:39:25.350493+0000","last_active":"2023-10-26T01:39:26.362529+0000","last_peered":"2023-10-26T01:39:26.362529+0000","last_clean":"2023-10-26T01:39:26.362529+0000","last_became_active":"2023-10-26T01:39:25.349758+0000","last_became_peered":"2023-10-26T01:39:25.349758+0000","last_unstale":"2023-10-26T01:39:26.362529+0000","last_undegraded":"2023-10-26T01:39:26.362529+0000","last_fullsized":"2023-10-26T01:39:26.362529+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:00:32.016316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:27.328922+0000","last_change":"2023-10-26T01:39:26.354438+0000","last_active":"2023-10-26T01:39:27.328922+0000","last_peered":"2023-10-26T01:39:27.328922+0000","last_clean":"2023-10-26T01:39:27.328922+0000","last_became_active":"2023-10-26T01:39:26.353968+0000","last_became_peered":"2023-10-26T01:39:26.353968+0000","last_unstale":"2023-10-26T01:39:27.328922+0000","last_undegraded":"2023-10-26T01:39:27.328922+0000","last_fullsized":"2023-10-26T01:39:27.328922+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:49:50.298664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"151'1","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:28.140060+0000","last_change":"2023-10-26T01:39:25.347830+0000","last_active":"2023-10-26T01:39:28.140060+0000","last_peered":"2023-10-26T01:39:28.140060+0000","last_clean":"2023-10-26T01:39:28.140060+0000","last_became_active":"2023-10-26T01:39:25.346994+0000","last_became_peered":"2023-10-26T01:39:25.346994+0000","last_unstale":"2023-10-26T01:39:28.140060+0000","last_undegraded":"2023-10-26T01:39:28.140060+0000","last_fullsized":"2023-10-26T01:39:28.140060+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:08:42.045168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:28.143832+0000","last_change":"2023-10-26T01:39:26.348169+0000","last_active":"2023-10-26T01:39:28.143832+0000","last_peered":"2023-10-26T01:39:28.143832+0000","last_clean":"2023-10-26T01:39:28.143832+0000","last_became_active":"2023-10-26T01:39:26.347642+0000","last_became_peered":"2023-10-26T01:39:26.347642+0000","last_unstale":"2023-10-26T01:39:28.143832+0000","last_undegraded":"2023-10-26T01:39:28.143832+0000","last_fullsized":"2023-10-26T01:39:28.143832+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:26:01.227512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-26T01:39:26.327792+0000","last_change":"2023-10-26T01:39:25.354096+0000","last_active":"2023-10-26T01:39:26.327792+0000","last_peered":"2023-10-26T01:39:26.327792+0000","last_clean":"2023-10-26T01:39:26.327792+0000","last_became_active":"2023-10-26T01:39:25.353857+0000","last_became_peered":"2023-10-26T01:39:25.353857+0000","last_unstale":"2023-10-26T01:39:26.327792+0000","last_undegraded":"2023-10-26T01:39:26.327792+0000","last_fullsized":"2023-10-26T01:39:26.327792+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:37:52.208922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:27.330171+0000","last_change":"2023-10-26T01:39:26.354645+0000","last_active":"2023-10-26T01:39:27.330171+0000","last_peered":"2023-10-26T01:39:27.330171+0000","last_clean":"2023-10-26T01:39:27.330171+0000","last_became_active":"2023-10-26T01:39:26.354090+0000","last_became_peered":"2023-10-26T01:39:26.354090+0000","last_unstale":"2023-10-26T01:39:27.330171+0000","last_undegraded":"2023-10-26T01:39:27.330171+0000","last_fullsized":"2023-10-26T01:39:27.330171+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:07:26.972765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-26T01:39:26.339415+0000","last_change":"2023-10-26T01:39:25.349514+0000","last_active":"2023-10-26T01:39:26.339415+0000","last_peered":"2023-10-26T01:39:26.339415+0000","last_clean":"2023-10-26T01:39:26.339415+0000","last_became_active":"2023-10-26T01:39:25.349279+0000","last_became_peered":"2023-10-26T01:39:25.349279+0000","last_unstale":"2023-10-26T01:39:26.339415+0000","last_undegraded":"2023-10-26T01:39:26.339415+0000","last_fullsized":"2023-10-26T01:39:26.339415+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:33:13.388324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:28.139534+0000","last_change":"2023-10-26T01:39:26.341277+0000","last_active":"2023-10-26T01:39:28.139534+0000","last_peered":"2023-10-26T01:39:28.139534+0000","last_clean":"2023-10-26T01:39:28.139534+0000","last_became_active":"2023-10-26T01:39:26.340759+0000","last_became_peered":"2023-10-26T01:39:26.340759+0000","last_unstale":"2023-10-26T01:39:28.139534+0000","last_undegraded":"2023-10-26T01:39:28.139534+0000","last_fullsized":"2023-10-26T01:39:28.139534+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:25:55.556501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-26T01:39:26.331459+0000","last_change":"2023-10-26T01:39:25.347759+0000","last_active":"2023-10-26T01:39:26.331459+0000","last_peered":"2023-10-26T01:39:26.331459+0000","last_clean":"2023-10-26T01:39:26.331459+0000","last_became_active":"2023-10-26T01:39:25.346564+0000","last_became_peered":"2023-10-26T01:39:25.346564+0000","last_unstale":"2023-10-26T01:39:26.331459+0000","last_undegraded":"2023-10-26T01:39:26.331459+0000","last_fullsized":"2023-10-26T01:39:26.331459+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:47:41.429100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:27.328818+0000","last_change":"2023-10-26T01:39:26.354552+0000","last_active":"2023-10-26T01:39:27.328818+0000","last_peered":"2023-10-26T01:39:27.328818+0000","last_clean":"2023-10-26T01:39:27.328818+0000","last_became_active":"2023-10-26T01:39:26.354256+0000","last_became_peered":"2023-10-26T01:39:26.354256+0000","last_unstale":"2023-10-26T01:39:27.328818+0000","last_undegraded":"2023-10-26T01:39:27.328818+0000","last_fullsized":"2023-10-26T01:39:27.328818+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:20:22.556497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-26T01:39:26.328763+0000","last_change":"2023-10-26T01:39:25.343413+0000","last_active":"2023-10-26T01:39:26.328763+0000","last_peered":"2023-10-26T01:39:26.328763+0000","last_clean":"2023-10-26T01:39:26.328763+0000","last_became_active":"2023-10-26T01:39:25.342949+0000","last_became_peered":"2023-10-26T01:39:25.342949+0000","last_unstale":"2023-10-26T01:39:26.328763+0000","last_undegraded":"2023-10-26T01:39:26.328763+0000","last_fullsized":"2023-10-26T01:39:26.328763+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:29:59.562191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:28.141156+0000","last_change":"2023-10-26T01:39:26.335564+0000","last_active":"2023-10-26T01:39:28.141156+0000","last_peered":"2023-10-26T01:39:28.141156+0000","last_clean":"2023-10-26T01:39:28.141156+0000","last_became_active":"2023-10-26T01:39:26.335000+0000","last_became_peered":"2023-10-26T01:39:26.335000+0000","last_unstale":"2023-10-26T01:39:28.141156+0000","last_undegraded":"2023-10-26T01:39:28.141156+0000","last_fullsized":"2023-10-26T01:39:28.141156+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:47:31.962180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-26T01:39:26.955302+0000","last_change":"2023-10-26T01:39:25.356190+0000","last_active":"2023-10-26T01:39:26.955302+0000","last_peered":"2023-10-26T01:39:26.955302+0000","last_clean":"2023-10-26T01:39:26.955302+0000","last_became_active":"2023-10-26T01:39:25.355791+0000","last_became_peered":"2023-10-26T01:39:25.355791+0000","last_unstale":"2023-10-26T01:39:26.955302+0000","last_undegraded":"2023-10-26T01:39:26.955302+0000","last_fullsized":"2023-10-26T01:39:26.955302+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:24:21.026112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:28.139979+0000","last_change":"2023-10-26T01:39:26.339884+0000","last_active":"2023-10-26T01:39:28.139979+0000","last_peered":"2023-10-26T01:39:28.139979+0000","last_clean":"2023-10-26T01:39:28.139979+0000","last_became_active":"2023-10-26T01:39:26.339195+0000","last_became_peered":"2023-10-26T01:39:26.339195+0000","last_unstale":"2023-10-26T01:39:28.139979+0000","last_undegraded":"2023-10-26T01:39:28.139979+0000","last_fullsized":"2023-10-26T01:39:28.139979+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:48:03.859702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-26T01:39:26.334872+0000","last_change":"2023-10-26T01:39:25.347950+0000","last_active":"2023-10-26T01:39:26.334872+0000","last_peered":"2023-10-26T01:39:26.334872+0000","last_clean":"2023-10-26T01:39:26.334872+0000","last_became_active":"2023-10-26T01:39:25.347416+0000","last_became_peered":"2023-10-26T01:39:25.347416+0000","last_unstale":"2023-10-26T01:39:26.334872+0000","last_undegraded":"2023-10-26T01:39:26.334872+0000","last_fullsized":"2023-10-26T01:39:26.334872+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:33:22.964454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:28.140264+0000","last_change":"2023-10-26T01:39:26.353099+0000","last_active":"2023-10-26T01:39:28.140264+0000","last_peered":"2023-10-26T01:39:28.140264+0000","last_clean":"2023-10-26T01:39:28.140264+0000","last_became_active":"2023-10-26T01:39:26.352798+0000","last_became_peered":"2023-10-26T01:39:26.352798+0000","last_unstale":"2023-10-26T01:39:28.140264+0000","last_undegraded":"2023-10-26T01:39:28.140264+0000","last_fullsized":"2023-10-26T01:39:28.140264+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:51:09.620939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"150'2","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-26T01:39:26.362832+0000","last_change":"2023-10-26T01:39:25.352322+0000","last_active":"2023-10-26T01:39:26.362832+0000","last_peered":"2023-10-26T01:39:26.362832+0000","last_clean":"2023-10-26T01:39:26.362832+0000","last_became_active":"2023-10-26T01:39:25.351287+0000","last_became_peered":"2023-10-26T01:39:25.351287+0000","last_unstale":"2023-10-26T01:39:26.362832+0000","last_undegraded":"2023-10-26T01:39:26.362832+0000","last_fullsized":"2023-10-26T01:39:26.362832+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:49:44.349175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":294,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-26T01:39:26.326156+0000","last_change":"2023-10-26T01:30:34.883484+0000","last_active":"2023-10-26T01:39:26.326156+0000","last_peered":"2023-10-26T01:39:26.326156+0000","last_clean":"2023-10-26T01:39:26.326156+0000","last_became_active":"2023-10-26T01:30:34.883071+0000","last_became_peered":"2023-10-26T01:30:34.883071+0000","last_unstale":"2023-10-26T01:39:26.326156+0000","last_undegraded":"2023-10-26T01:39:26.326156+0000","last_fullsized":"2023-10-26T01:39:26.326156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:32.890096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:32.890096+0000","last_clean_scrub_stamp":"2023-10-26T01:30:32.890096+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:26:31.879452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":14,"seq":60129542255,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":823,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194489,"statfs":{"total":200000000,"available":199156956,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:38:51 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.443,"5min":0.35599999999999998,"15min":0.33000000000000002},"max":{"1min":1.2549999999999999,"5min":1.6319999999999999,"15min":2.2549999999999999},"last":0.88},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.92300000000000004,"15min":0.89300000000000002},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":1.752,"5min":1.752,"15min":1.8200000000000001},"last":0.93100000000000005}]},{"osd":1,"last update":"Thu Oct 26 01:38:51 2023","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.88400000000000001,"15min":0.92800000000000005},"min":{"1min":0.39400000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.383,"5min":1.444,"15min":4.4340000000000002},"last":1.052},{"interface":"front","average":{"1min":0.996,"5min":0.99199999999999999,"15min":0.995},"min":{"1min":0.56200000000000006,"5min":0.46700000000000003,"15min":0.374},"max":{"1min":1.353,"5min":1.538,"15min":2.3330000000000002},"last":0.77400000000000002}]},{"osd":2,"last update":"Thu Oct 26 01:38:51 2023","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":1.028,"15min":1.0700000000000001},"min":{"1min":0.54500000000000004,"5min":0.40999999999999998,"15min":0.39900000000000002},"max":{"1min":1.4279999999999999,"5min":1.927,"15min":4.2800000000000002},"last":0.99199999999999999},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.1479999999999999,"15min":1.0980000000000001},"min":{"1min":0.53300000000000003,"5min":0.495,"15min":0.49399999999999999},"max":{"1min":1.639,"5min":1.8680000000000001,"15min":2.302},"last":1.2490000000000001}]},{"osd":4,"last update":"Thu Oct 26 01:38:51 2023","interfaces":[{"interface":"back","average":{"1min":1.266,"5min":1.23,"15min":1.175},"min":{"1min":0.98199999999999998,"5min":0.755,"15min":0.66400000000000003},"max":{"1min":1.732,"5min":1.732,"15min":2.1070000000000002},"last":1.0369999999999999},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.353,"15min":1.3859999999999999},"min":{"1min":0.91900000000000004,"5min":0.81599999999999995,"15min":0.79900000000000004},"max":{"1min":1.6220000000000001,"5min":2.4239999999999999,"15min":4.8250000000000002},"last":1.1180000000000001}]},{"osd":5,"last update":"Thu Oct 26 01:38:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.2929999999999999,"15min":1.3280000000000001},"min":{"1min":0.84899999999999998,"5min":0.76400000000000001,"15min":0.72499999999999998},"max":{"1min":1.6899999999999999,"5min":2.3380000000000001,"15min":4.9080000000000004},"last":1.341},{"interface":"front","average":{"1min":1.331,"5min":1.2969999999999999,"15min":1.2789999999999999},"min":{"1min":0.82699999999999996,"5min":0.79900000000000004,"15min":0.79600000000000004},"max":{"1min":1.573,"5min":2.0310000000000001,"15min":2.8980000000000001},"last":1.375}]},{"osd":6,"last update":"Thu Oct 26 01:38:51 2023","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.351,"15min":1.329},"min":{"1min":0.75900000000000001,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.694,"5min":2.1499999999999999,"15min":2.8530000000000002},"last":1.4530000000000001},{"interface":"front","average":{"1min":1.4159999999999999,"5min":1.3160000000000001,"15min":1.2769999999999999},"min":{"1min":0.84399999999999997,"5min":0.81699999999999995,"15min":0.73099999999999998},"max":{"1min":1.6899999999999999,"5min":2.048,"15min":2.181},"last":1.093}]},{"osd":7,"last update":"Thu Oct 26 01:38:51 2023","interfaces":[{"interface":"back","average":{"1min":1.4370000000000001,"5min":1.3660000000000001,"15min":1.3069999999999999},"min":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.71999999999999997},"max":{"1min":1.7589999999999999,"5min":1.958,"15min":2.0680000000000001},"last":1.306},{"interface":"front","average":{"1min":1.385,"5min":1.393,"15min":1.411},"min":{"1min":0.73299999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.702,"5min":2.2370000000000001,"15min":4.8730000000000002},"last":1.2130000000000001}]}]},{"osd":7,"up_from":13,"seq":55834574958,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1401,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193910,"statfs":{"total":200000000,"available":198564498,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:38:50 2023","interfaces":[{"interface":"back","average":{"1min":1.127,"5min":1.1379999999999999,"15min":1.097},"min":{"1min":0.68700000000000006,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.633,"5min":1.7,"15min":1.9410000000000001},"last":1.034},{"interface":"front","average":{"1min":1.095,"5min":1.1100000000000001,"15min":1.0840000000000001},"min":{"1min":0.67300000000000004,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.6970000000000001,"5min":2.6779999999999999,"15min":2.6779999999999999},"last":1.417}]},{"osd":1,"last update":"Thu Oct 26 01:38:50 2023","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":1.1699999999999999,"15min":1.1020000000000001},"min":{"1min":0.82899999999999996,"5min":0.65800000000000003,"15min":0.65200000000000002},"max":{"1min":1.484,"5min":2.8260000000000001,"15min":2.8260000000000001},"last":0.875},{"interface":"front","average":{"1min":1.177,"5min":1.2,"15min":1.194},"min":{"1min":0.77200000000000002,"5min":0.66500000000000004,"15min":0.61199999999999999},"max":{"1min":1.76,"5min":3.2730000000000001,"15min":3.2730000000000001},"last":1.1679999999999999}]},{"osd":2,"last update":"Thu Oct 26 01:38:50 2023","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.2110000000000001,"15min":1.1299999999999999},"min":{"1min":0.873,"5min":0.69099999999999995,"15min":0.58499999999999996},"max":{"1min":1.851,"5min":2.218,"15min":2.218},"last":1.2529999999999999},{"interface":"front","average":{"1min":1.256,"5min":1.296,"15min":1.2430000000000001},"min":{"1min":0.82299999999999995,"5min":0.79400000000000004,"15min":0.73099999999999998},"max":{"1min":1.7350000000000001,"5min":3.0510000000000002,"15min":3.0510000000000002},"last":1.1259999999999999}]},{"osd":3,"last update":"Thu Oct 26 01:38:50 2023","interfaces":[{"interface":"back","average":{"1min":1.3120000000000001,"5min":1.296,"15min":1.2230000000000001},"min":{"1min":0.79500000000000004,"5min":0.79000000000000004,"15min":0.72099999999999997},"max":{"1min":1.8169999999999999,"5min":1.9830000000000001,"15min":1.9830000000000001},"last":1.073},{"interface":"front","average":{"1min":1.224,"5min":1.373,"15min":1.3640000000000001},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.65700000000000003},"max":{"1min":1.5129999999999999,"5min":5.5039999999999996,"15min":5.5039999999999996},"last":1.331}]},{"osd":4,"last update":"Thu Oct 26 01:38:50 2023","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.371,"15min":1.351},"min":{"1min":0.81799999999999995,"5min":0.77300000000000002,"15min":0.76100000000000001},"max":{"1min":1.9339999999999999,"5min":2.77,"15min":2.77},"last":2.0840000000000001},{"interface":"front","average":{"1min":1.288,"5min":1.3919999999999999,"15min":1.393},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.6479999999999999,"5min":3.1419999999999999,"15min":3.1419999999999999},"last":2.3929999999999998}]},{"osd":5,"last update":"Thu Oct 26 01:38:50 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.4410000000000001,"15min":1.4199999999999999},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.7809999999999999,"5min":5.734,"15min":5.734},"last":1.514},{"interface":"front","average":{"1min":1.4450000000000001,"5min":1.514,"15min":1.478},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.51900000000000002},"max":{"1min":2,"5min":2.8980000000000001,"15min":2.8980000000000001},"last":1.389}]},{"osd":6,"last update":"Thu Oct 26 01:38:50 2023","interfaces":[{"interface":"back","average":{"1min":1.452,"5min":1.5800000000000001,"15min":1.8089999999999999},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.47399999999999998},"max":{"1min":1.8049999999999999,"5min":5.6340000000000003,"15min":11.577999999999999},"last":1.583},{"interface":"front","average":{"1min":1.613,"5min":1.6719999999999999,"15min":1.8360000000000001},"min":{"1min":1.0640000000000001,"5min":0.91800000000000004,"15min":0.55100000000000005},"max":{"1min":2.0529999999999999,"5min":5.0919999999999996,"15min":10.704000000000001},"last":1.476}]}]},{"osd":6,"up_from":13,"seq":55834574958,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":824,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194488,"statfs":{"total":200000000,"available":199156118,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:38:46 2023","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.141,"15min":1.1519999999999999},"min":{"1min":0.66600000000000004,"5min":0.61699999999999999,"15min":0.60399999999999998},"max":{"1min":2.7370000000000001,"5min":2.7370000000000001,"15min":2.7370000000000001},"last":1.228},{"interface":"front","average":{"1min":1.137,"5min":1.173,"15min":1.123},"min":{"1min":0.61699999999999999,"5min":0.61399999999999999,"15min":0.59199999999999997},"max":{"1min":4.4779999999999998,"5min":4.4779999999999998,"15min":4.4779999999999998},"last":1.1819999999999999}]},{"osd":1,"last update":"Thu Oct 26 01:38:46 2023","interfaces":[{"interface":"back","average":{"1min":1.3919999999999999,"5min":1.2729999999999999,"15min":1.1930000000000001},"min":{"1min":0.77100000000000002,"5min":0.72299999999999998,"15min":0.68300000000000005},"max":{"1min":5.5359999999999996,"5min":5.5359999999999996,"15min":5.5359999999999996},"last":1.046},{"interface":"front","average":{"1min":1.228,"5min":1.2190000000000001,"15min":1.204},"min":{"1min":0.74399999999999999,"5min":0.67900000000000005,"15min":0.64000000000000001},"max":{"1min":4.3579999999999997,"5min":4.3579999999999997,"15min":4.3579999999999997},"last":1.5109999999999999}]},{"osd":2,"last update":"Thu Oct 26 01:38:46 2023","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.2030000000000001,"15min":1.107},"min":{"1min":0.76300000000000001,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":2.367,"5min":3.7879999999999998,"15min":3.7879999999999998},"last":1.302},{"interface":"front","average":{"1min":1.371,"5min":1.3420000000000001,"15min":1.276},"min":{"1min":0.91200000000000003,"5min":0.75800000000000001,"15min":0.68899999999999995},"max":{"1min":3.0470000000000002,"5min":3.0470000000000002,"15min":3.0470000000000002},"last":1.4259999999999999}]},{"osd":3,"last update":"Thu Oct 26 01:38:46 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.331,"15min":1.22},"min":{"1min":0.76300000000000001,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":3.266,"5min":4.4560000000000004,"15min":4.4560000000000004},"last":1.5800000000000001},{"interface":"front","average":{"1min":1.365,"5min":1.3500000000000001,"15min":1.3280000000000001},"min":{"1min":0.86399999999999999,"5min":0.78600000000000003,"15min":0.69499999999999995},"max":{"1min":2.6909999999999998,"5min":2.6909999999999998,"15min":2.6909999999999998},"last":1.3620000000000001}]},{"osd":4,"last update":"Thu Oct 26 01:38:46 2023","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.24,"15min":1.22},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.48399999999999999},"max":{"1min":1.7410000000000001,"5min":2.9489999999999998,"15min":2.9489999999999998},"last":1.262},{"interface":"front","average":{"1min":1.3180000000000001,"5min":1.421,"15min":1.319},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.59299999999999997},"max":{"1min":1.698,"5min":6.609,"15min":6.609},"last":0.98499999999999999}]},{"osd":5,"last update":"Thu Oct 26 01:38:46 2023","interfaces":[{"interface":"back","average":{"1min":1.464,"5min":1.5389999999999999,"15min":1.452},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.7949999999999999,"5min":5.4370000000000003,"15min":5.4370000000000003},"last":1.6539999999999999},{"interface":"front","average":{"1min":1.492,"5min":1.4530000000000001,"15min":1.419},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":2.7759999999999998,"5min":2.7759999999999998,"15min":2.7759999999999998},"last":1.5469999999999999}]},{"osd":7,"last update":"Thu Oct 26 01:38:46 2023","interfaces":[{"interface":"back","average":{"1min":1.2869999999999999,"5min":1.484,"15min":1.405},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.54100000000000004},"max":{"1min":1.8200000000000001,"5min":11.804,"15min":11.804},"last":1.3979999999999999},{"interface":"front","average":{"1min":1.577,"5min":1.6639999999999999,"15min":1.52},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":3.7650000000000001,"5min":11.541,"15min":11.541},"last":1.464}]}]},{"osd":5,"up_from":13,"seq":55834574958,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":823,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194489,"statfs":{"total":200000000,"available":199157004,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:38:48 2023","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.2310000000000001,"15min":1.147},"min":{"1min":0.69699999999999995,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":3.8210000000000002,"5min":3.8210000000000002,"15min":3.8210000000000002},"last":1.5800000000000001},{"interface":"front","average":{"1min":1.3480000000000001,"5min":1.302,"15min":1.1970000000000001},"min":{"1min":0.92600000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":2.9889999999999999,"5min":2.9889999999999999,"15min":2.9889999999999999},"last":1.673}]},{"osd":1,"last update":"Thu Oct 26 01:38:48 2023","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.1319999999999999,"15min":1.0940000000000001},"min":{"1min":0.83599999999999997,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.8120000000000001,"5min":1.8120000000000001,"15min":1.8120000000000001},"last":1.377},{"interface":"front","average":{"1min":1.272,"5min":1.1659999999999999,"15min":1.109},"min":{"1min":0.76700000000000002,"5min":0.629,"15min":0.60599999999999998},"max":{"1min":2.4340000000000002,"5min":2.4340000000000002,"15min":2.4340000000000002},"last":0.97499999999999998}]},{"osd":2,"last update":"Thu Oct 26 01:38:48 2023","interfaces":[{"interface":"back","average":{"1min":1.2130000000000001,"5min":1.206,"15min":1.1679999999999999},"min":{"1min":0.79700000000000004,"5min":0.71599999999999997,"15min":0.69299999999999995},"max":{"1min":1.5920000000000001,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":1.141},{"interface":"front","average":{"1min":1.228,"5min":1.228,"15min":1.1850000000000001},"min":{"1min":0.91600000000000004,"5min":0.78400000000000003,"15min":0.63900000000000001},"max":{"1min":1.5900000000000001,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":1.2230000000000001}]},{"osd":3,"last update":"Thu Oct 26 01:38:48 2023","interfaces":[{"interface":"back","average":{"1min":1.389,"5min":1.2829999999999999,"15min":1.25},"min":{"1min":1.0009999999999999,"5min":0.72699999999999998,"15min":0.72299999999999998},"max":{"1min":1.7,"5min":1.7,"15min":1.835},"last":1.629},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.298,"15min":1.29},"min":{"1min":0.91300000000000003,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.52,"5min":1.788,"15min":2.9249999999999998},"last":1.5}]},{"osd":4,"last update":"Thu Oct 26 01:38:48 2023","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":1.256,"15min":1.214},"min":{"1min":0.75,"5min":0.59599999999999997,"15min":0.49099999999999999},"max":{"1min":2.7770000000000001,"5min":2.7770000000000001,"15min":2.7770000000000001},"last":1.427},{"interface":"front","average":{"1min":1.3180000000000001,"5min":1.3979999999999999,"15min":1.355},"min":{"1min":0.67500000000000004,"5min":0.57299999999999995,"15min":0.56200000000000006},"max":{"1min":1.776,"5min":1.7849999999999999,"15min":1.867},"last":1.857}]},{"osd":6,"last update":"Thu Oct 26 01:38:48 2023","interfaces":[{"interface":"back","average":{"1min":1.377,"5min":1.393,"15min":1.3300000000000001},"min":{"1min":0.79000000000000004,"5min":0.67500000000000004,"15min":0.51900000000000002},"max":{"1min":1.956,"5min":2.8250000000000002,"15min":2.8250000000000002},"last":1.8069999999999999},{"interface":"front","average":{"1min":1.4590000000000001,"5min":1.409,"15min":1.3400000000000001},"min":{"1min":0.89300000000000002,"5min":0.69499999999999995,"15min":0.61799999999999999},"max":{"1min":2.4830000000000001,"5min":2.4830000000000001,"15min":2.4830000000000001},"last":1.7310000000000001}]},{"osd":7,"last update":"Thu Oct 26 01:38:48 2023","interfaces":[{"interface":"back","average":{"1min":1.6000000000000001,"5min":1.534,"15min":1.4379999999999999},"min":{"1min":0.81499999999999995,"5min":0.71099999999999997,"15min":0.61699999999999999},"max":{"1min":3.621,"5min":3.621,"15min":3.621},"last":1.9750000000000001},{"interface":"front","average":{"1min":1.603,"5min":1.542,"15min":1.4470000000000001},"min":{"1min":0.82899999999999996,"5min":0.73899999999999999,"15min":0.59499999999999997},"max":{"1min":2.8010000000000002,"5min":2.8010000000000002,"15min":2.8010000000000002},"last":1.915}]}]},{"osd":4,"up_from":13,"seq":55834574958,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":823,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194489,"statfs":{"total":200000000,"available":199157046,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:38:43 2023","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.071,"15min":0.999},"min":{"1min":0.63500000000000001,"5min":0.59799999999999998,"15min":0.53100000000000003},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":2.0289999999999999},"last":0.95099999999999996},{"interface":"front","average":{"1min":1.123,"5min":1.077,"15min":1.0329999999999999},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.60599999999999998},"max":{"1min":1.784,"5min":1.784,"15min":2.6829999999999998},"last":1.125}]},{"osd":1,"last update":"Thu Oct 26 01:38:43 2023","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":1.161,"15min":1.1259999999999999},"min":{"1min":0.749,"5min":0.66700000000000004,"15min":0.56799999999999995},"max":{"1min":1.579,"5min":1.599,"15min":4.0519999999999996},"last":1.018},{"interface":"front","average":{"1min":1.2529999999999999,"5min":1.1779999999999999,"15min":1.121},"min":{"1min":0.70799999999999996,"5min":0.70399999999999996,"15min":0.60499999999999998},"max":{"1min":2.3919999999999999,"5min":2.3919999999999999,"15min":2.3919999999999999},"last":1.3120000000000001}]},{"osd":2,"last update":"Thu Oct 26 01:38:43 2023","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":1.304,"15min":1.2430000000000001},"min":{"1min":1.0429999999999999,"5min":0.66700000000000004,"15min":0.64300000000000002},"max":{"1min":1.6759999999999999,"5min":1.845,"15min":5.6749999999999998},"last":1.3560000000000001},{"interface":"front","average":{"1min":1.1799999999999999,"5min":1.218,"15min":1.1419999999999999},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.631,"5min":3.673,"15min":3.673},"last":1.3819999999999999}]},{"osd":3,"last update":"Thu Oct 26 01:38:43 2023","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.228,"15min":1.1919999999999999},"min":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.70399999999999996},"max":{"1min":1.7030000000000001,"5min":1.847,"15min":1.891},"last":1.4350000000000001},{"interface":"front","average":{"1min":1.1799999999999999,"5min":1.204,"15min":1.1619999999999999},"min":{"1min":0.86099999999999999,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.522,"5min":1.681,"15min":1.7210000000000001},"last":1.0680000000000001}]},{"osd":5,"last update":"Thu Oct 26 01:38:43 2023","interfaces":[{"interface":"back","average":{"1min":1.236,"5min":1.258,"15min":1.2809999999999999},"min":{"1min":0.85699999999999998,"5min":0.52100000000000002,"15min":0.46300000000000002},"max":{"1min":1.681,"5min":2.1429999999999998,"15min":2.1429999999999998},"last":1.5569999999999999},{"interface":"front","average":{"1min":1.2270000000000001,"5min":1.2809999999999999,"15min":1.321},"min":{"1min":0.84899999999999998,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.754,"5min":1.9810000000000001,"15min":5.8579999999999997},"last":1.524}]},{"osd":6,"last update":"Thu Oct 26 01:38:43 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.294,"15min":1.335},"min":{"1min":0.752,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.708,"5min":2.2149999999999999,"15min":12.343999999999999},"last":1.486},{"interface":"front","average":{"1min":1.5129999999999999,"5min":1.5109999999999999,"15min":1.5189999999999999},"min":{"1min":1.0660000000000001,"5min":0.57999999999999996,"15min":0.55000000000000004},"max":{"1min":1.762,"5min":2.117,"15min":12.605},"last":1.6220000000000001}]},{"osd":7,"last update":"Thu Oct 26 01:38:43 2023","interfaces":[{"interface":"back","average":{"1min":1.4690000000000001,"5min":1.4670000000000001,"15min":1.49},"min":{"1min":1.1279999999999999,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.8200000000000001,"5min":2.762,"15min":5.774},"last":1.915},{"interface":"front","average":{"1min":1.399,"5min":1.484,"15min":1.4590000000000001},"min":{"1min":1.008,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.7589999999999999,"5min":2.8330000000000002,"15min":4.6520000000000001},"last":1.671}]}]},{"osd":2,"up_from":13,"seq":55834574958,"num_pgs":29,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":823,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194488,"statfs":{"total":200000000,"available":199156500,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:38:47 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.84499999999999997,"15min":0.85299999999999998},"min":{"1min":0.46600000000000003,"5min":0.41199999999999998,"15min":0.308},"max":{"1min":1.4079999999999999,"5min":1.5249999999999999,"15min":6.3200000000000003},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.97699999999999998,"15min":0.93300000000000005},"min":{"1min":0.44,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.476,"5min":1.9970000000000001,"15min":4.2619999999999996},"last":0.77100000000000002}]},{"osd":1,"last update":"Thu Oct 26 01:38:47 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.878,"15min":0.89100000000000001},"min":{"1min":0.48399999999999999,"5min":0.39300000000000002,"15min":0.34000000000000002},"max":{"1min":1.2829999999999999,"5min":1.629,"15min":5.1260000000000003},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.98499999999999999,"5min":1.0189999999999999,"15min":1.0029999999999999},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.36599999999999999},"max":{"1min":1.4870000000000001,"5min":1.873,"15min":6.1520000000000001},"last":1.2809999999999999}]},{"osd":3,"last update":"Thu Oct 26 01:38:52 2023","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":1.004,"15min":0.94499999999999995},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.407,"5min":2.0640000000000001,"15min":2.0640000000000001},"last":0.93500000000000005},{"interface":"front","average":{"1min":1.0349999999999999,"5min":1.0860000000000001,"15min":1.0249999999999999},"min":{"1min":0.53800000000000003,"5min":0.51100000000000001,"15min":0.45300000000000001},"max":{"1min":1.665,"5min":1.9279999999999999,"15min":8.9019999999999992},"last":0.999}]},{"osd":4,"last update":"Thu Oct 26 01:38:47 2023","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.3280000000000001,"15min":1.2689999999999999},"min":{"1min":0.81599999999999995,"5min":0.80500000000000005,"15min":0.73899999999999999},"max":{"1min":1.6080000000000001,"5min":2.6190000000000002,"15min":6.2670000000000003},"last":1.3460000000000001},{"interface":"front","average":{"1min":1.2709999999999999,"5min":1.2969999999999999,"15min":1.266},"min":{"1min":0.81699999999999995,"5min":0.81100000000000005,"15min":0.78700000000000003},"max":{"1min":1.8380000000000001,"5min":2.4359999999999999,"15min":7.343},"last":1.1200000000000001}]},{"osd":5,"last update":"Thu Oct 26 01:38:47 2023","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":1.3500000000000001,"15min":1.29},"min":{"1min":0.85099999999999998,"5min":0.81699999999999995,"15min":0.71899999999999997},"max":{"1min":1.669,"5min":2.5790000000000002,"15min":3.7170000000000001},"last":1.385},{"interface":"front","average":{"1min":1.302,"5min":1.3120000000000001,"15min":1.2929999999999999},"min":{"1min":0.872,"5min":0.81899999999999995,"15min":0.65300000000000002},"max":{"1min":1.708,"5min":2.298,"15min":6.3899999999999997},"last":1.421}]},{"osd":6,"last update":"Thu Oct 26 01:38:47 2023","interfaces":[{"interface":"back","average":{"1min":1.3280000000000001,"5min":1.3919999999999999,"15min":1.3380000000000001},"min":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.70599999999999996},"max":{"1min":1.714,"5min":2.73,"15min":6.0490000000000004},"last":1.1839999999999999},{"interface":"front","average":{"1min":1.379,"5min":1.4139999999999999,"15min":1.3740000000000001},"min":{"1min":0.89200000000000002,"5min":0.78200000000000003,"15min":0.76800000000000002},"max":{"1min":2.0990000000000002,"5min":2.4089999999999998,"15min":5.0650000000000004},"last":1.4750000000000001}]},{"osd":7,"last update":"Thu Oct 26 01:38:47 2023","interfaces":[{"interface":"back","average":{"1min":1.373,"5min":1.3979999999999999,"15min":1.3939999999999999},"min":{"1min":0.79200000000000004,"5min":0.77900000000000003,"15min":0.69399999999999995},"max":{"1min":1.776,"5min":2.3370000000000002,"15min":6.4480000000000004},"last":1.625},{"interface":"front","average":{"1min":1.4830000000000001,"5min":1.5169999999999999,"15min":1.458},"min":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.91800000000000004},"max":{"1min":2.1480000000000001,"5min":2.8290000000000002,"15min":2.8439999999999999},"last":1.5960000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574958,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":823,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194489,"statfs":{"total":200000000,"available":199157116,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:38:49 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.76900000000000002,"15min":0.72799999999999998},"min":{"1min":0.41799999999999998,"5min":0.34000000000000002,"15min":0.29499999999999998},"max":{"1min":0.92700000000000005,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.87,"15min":0.80200000000000005},"min":{"1min":0.52000000000000002,"5min":0.36299999999999999,"15min":0.36199999999999999},"max":{"1min":1.2769999999999999,"5min":1.6399999999999999,"15min":1.6399999999999999},"last":1.1910000000000001}]},{"osd":2,"last update":"Thu Oct 26 01:38:49 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.995,"15min":0.999},"min":{"1min":0.65300000000000002,"5min":0.39700000000000002,"15min":0.39600000000000002},"max":{"1min":1.286,"5min":1.8200000000000001,"15min":2.4830000000000001},"last":1.292},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.96199999999999997,"15min":0.88400000000000001},"min":{"1min":0.48099999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.569,"5min":1.8080000000000001,"15min":1.8080000000000001},"last":1.1499999999999999}]},{"osd":3,"last update":"Thu Oct 26 01:38:53 2023","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":0.996,"15min":0.95499999999999996},"min":{"1min":0.59999999999999998,"5min":0.42499999999999999,"15min":0.39500000000000002},"max":{"1min":1.5900000000000001,"5min":1.611,"15min":1.75},"last":0.872},{"interface":"front","average":{"1min":0.998,"5min":1.028,"15min":0.99199999999999999},"min":{"1min":0.55000000000000004,"5min":0.47799999999999998,"15min":0.41299999999999998},"max":{"1min":1.5049999999999999,"5min":1.673,"15min":1.802},"last":0.99099999999999999}]},{"osd":4,"last update":"Thu Oct 26 01:38:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":1.222,"15min":1.1990000000000001},"min":{"1min":0.90400000000000003,"5min":0.69099999999999995,"15min":0.56200000000000006},"max":{"1min":1.5249999999999999,"5min":1.756,"15min":2.0659999999999998},"last":1.538},{"interface":"front","average":{"1min":1.1659999999999999,"5min":1.1870000000000001,"15min":1.1399999999999999},"min":{"1min":0.81299999999999994,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.4390000000000001,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":1.351}]},{"osd":5,"last update":"Thu Oct 26 01:38:49 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.2629999999999999,"15min":1.2290000000000001},"min":{"1min":0.84799999999999998,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.621,"5min":1.8779999999999999,"15min":1.8779999999999999},"last":1.4890000000000001},{"interface":"front","average":{"1min":1.359,"5min":1.375,"15min":1.339},"min":{"1min":1.052,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.7210000000000001,"5min":2.04,"15min":2.2200000000000002},"last":1.45}]},{"osd":6,"last update":"Thu Oct 26 01:38:49 2023","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.347,"15min":1.2849999999999999},"min":{"1min":0.90400000000000003,"5min":0.70199999999999996,"15min":0.68899999999999995},"max":{"1min":1.7709999999999999,"5min":2.0910000000000002,"15min":2.3780000000000001},"last":1.5169999999999999},{"interface":"front","average":{"1min":1.2889999999999999,"5min":1.2789999999999999,"15min":1.2490000000000001},"min":{"1min":0.92700000000000005,"5min":0.80700000000000005,"15min":0.70799999999999996},"max":{"1min":1.653,"5min":1.956,"15min":1.984},"last":1.2529999999999999}]},{"osd":7,"last update":"Thu Oct 26 01:38:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.2749999999999999,"15min":1.262},"min":{"1min":0.79400000000000004,"5min":0.76100000000000001,"15min":0.69699999999999995},"max":{"1min":1.764,"5min":1.8380000000000001,"15min":2.6930000000000001},"last":1.4239999999999999},{"interface":"front","average":{"1min":1.431,"5min":1.419,"15min":1.345},"min":{"1min":0.95499999999999996,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.835,"5min":2.0190000000000001,"15min":2.0190000000000001},"last":1.569}]}]},{"osd":0,"up_from":13,"seq":55834574958,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1399,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193912,"statfs":{"total":200000000,"available":198566748,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Oct 26 01:38:46 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.79700000000000004,"15min":0.91100000000000003},"min":{"1min":0.47799999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.159,"5min":1.5049999999999999,"15min":19.425000000000001},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.93999999999999995,"15min":1.0369999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.462,"5min":1.462,"15min":15.162000000000001},"last":0.56200000000000006}]},{"osd":2,"last update":"Thu Oct 26 01:38:46 2023","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":1.0149999999999999,"15min":1.175},"min":{"1min":0.64300000000000002,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.47,"5min":1.577,"15min":28.439},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.90000000000000002,"15min":0.91300000000000003},"min":{"1min":0.57299999999999995,"5min":0.37,"15min":0.37},"max":{"1min":1.2969999999999999,"5min":1.4810000000000001,"15min":4.9459999999999997},"last":0.53900000000000003}]},{"osd":3,"last update":"Thu Oct 26 01:38:46 2023","interfaces":[{"interface":"back","average":{"1min":1.1180000000000001,"5min":1.0640000000000001,"15min":1.085},"min":{"1min":0.66800000000000004,"5min":0.46000000000000002,"15min":0.44600000000000001},"max":{"1min":1.6539999999999999,"5min":1.6539999999999999,"15min":4.9800000000000004},"last":0.66500000000000004},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.123,"15min":1.1499999999999999},"min":{"1min":0.66000000000000003,"5min":0.438,"15min":0.438},"max":{"1min":1.4279999999999999,"5min":1.6559999999999999,"15min":5.016},"last":0.873}]},{"osd":4,"last update":"Thu Oct 26 01:38:46 2023","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":1.169,"15min":1.2789999999999999},"min":{"1min":0.94499999999999995,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.391,"5min":1.681,"15min":19.329000000000001},"last":0.85699999999999998},{"interface":"front","average":{"1min":1.23,"5min":1.262,"15min":1.292},"min":{"1min":0.86799999999999999,"5min":0.82099999999999995,"15min":0.67600000000000005},"max":{"1min":1.5149999999999999,"5min":1.718,"15min":5.5229999999999997},"last":0.98999999999999999}]},{"osd":5,"last update":"Thu Oct 26 01:38:46 2023","interfaces":[{"interface":"back","average":{"1min":1.427,"5min":1.4019999999999999,"15min":1.4279999999999999},"min":{"1min":1.03,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.819,"5min":1.857,"15min":5.6360000000000001},"last":1.226},{"interface":"front","average":{"1min":1.2909999999999999,"5min":1.2749999999999999,"15min":1.395},"min":{"1min":0.80200000000000005,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.5880000000000001,"5min":1.8200000000000001,"15min":19.384},"last":0.94199999999999995}]},{"osd":6,"last update":"Thu Oct 26 01:38:46 2023","interfaces":[{"interface":"back","average":{"1min":1.306,"5min":1.296,"15min":1.3600000000000001},"min":{"1min":0.79300000000000004,"5min":0.76700000000000002,"15min":0.73799999999999999},"max":{"1min":1.736,"5min":1.736,"15min":8.2609999999999992},"last":0.97399999999999998},{"interface":"front","average":{"1min":1.3,"5min":1.335,"15min":1.3999999999999999},"min":{"1min":0.82999999999999996,"5min":0.77900000000000003,"15min":0.75900000000000001},"max":{"1min":1.581,"5min":1.9450000000000001,"15min":11.849},"last":1.0780000000000001}]},{"osd":7,"last update":"Thu Oct 26 01:38:46 2023","interfaces":[{"interface":"back","average":{"1min":1.46,"5min":1.4319999999999999,"15min":1.4770000000000001},"min":{"1min":1.089,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.9239999999999999,"5min":1.9239999999999999,"15min":5.6689999999999996},"last":1.117},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.397,"15min":1.4930000000000001},"min":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.69399999999999995},"max":{"1min":1.661,"5min":1.9390000000000001,"15min":19.245999999999999},"last":0.81299999999999994}]}]}],"pool_statfs":[]}} 2023-10-26T01:39:35.799 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-26T01:39:36.033 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:36.030+0000 7fd189bed640 1 -- 172.21.15.169:0/2333125737 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd174004d00 msgr2=0x7fd174005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:36.034 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:36.030+0000 7fd189bed640 1 --2- 172.21.15.169:0/2333125737 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd174004d00 0x7fd174005180 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7fd17c009e20 tx=0x7fd1740057f0 comp rx=0 tx=0).stop 2023-10-26T01:39:36.034 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:36.030+0000 7fd1893ec640 1 -- 172.21.15.169:0/2333125737 reap_dead start 2023-10-26T01:39:36.034 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:36.030+0000 7fd189bed640 1 -- 172.21.15.169:0/2333125737 shutdown_connections 2023-10-26T01:39:36.034 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:36.030+0000 7fd189bed640 1 -- 172.21.15.169:0/2333125737 >> 172.21.15.169:0/2333125737 conn(0x7fd1840aebf0 msgr2=0x7fd1840af000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:36.035 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:36.031+0000 7fd189bed640 1 -- 172.21.15.169:0/2333125737 shutdown_connections 2023-10-26T01:39:36.035 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:36.031+0000 7fd189bed640 1 -- 172.21.15.169:0/2333125737 wait complete. 2023-10-26T01:39:36.035 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:36.032+0000 7fd189bed640 1 Processor -- start 2023-10-26T01:39:36.036 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:36.032+0000 7fd189bed640 1 -- start start 2023-10-26T01:39:36.037 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:36.033+0000 7fd189bed640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd174004d00 0x7fd18415a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:36.037 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:36.033+0000 7fd189bed640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd184147840 0x7fd18415bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:36.037 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:36.033+0000 7fd189bed640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd18406f9e0 0x7fd18406fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:36.037 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:36.033+0000 7fd189bed640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1840703a0 con 0x7fd18406f9e0 2023-10-26T01:39:36.038 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:36.033+0000 7fd189bed640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd18415a8a0 con 0x7fd174004d00 2023-10-26T01:39:36.038 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:36.033+0000 7fd189bed640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd18415c4f0 con 0x7fd184147840 2023-10-26T01:39:36.038 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:36.034+0000 7fd1893ec640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd18406f9e0 0x7fd18406fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:36.038 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:36.034+0000 7fd1893ec640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd18406f9e0 0x7fd18406fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:33310/0 (socket says 172.21.15.169:33310) 2023-10-26T01:39:36.038 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:36.034+0000 7fd1893ec640 1 -- 172.21.15.169:0/3685508674 learned_addr learned my addr 172.21.15.169:0/3685508674 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:39:36.039 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:36.034+0000 7fd183fff640 1 --2- 172.21.15.169:0/3685508674 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd184147840 0x7fd18415bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:36.039 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:36.034+0000 7fd188beb640 1 --2- 172.21.15.169:0/3685508674 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd174004d00 0x7fd18415a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:36.039 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:36.034+0000 7fd1893ec640 1 -- 172.21.15.169:0/3685508674 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd184147840 msgr2=0x7fd18415bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:36.040 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:36.034+0000 7fd1893ec640 1 --2- 172.21.15.169:0/3685508674 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd184147840 0x7fd18415bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:36.040 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:36.034+0000 7fd1893ec640 1 -- 172.21.15.169:0/3685508674 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd174004d00 msgr2=0x7fd18415a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:36.040 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:36.034+0000 7fd1893ec640 1 --2- 172.21.15.169:0/3685508674 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd174004d00 0x7fd18415a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:36.040 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:36.034+0000 7fd1893ec640 1 -- 172.21.15.169:0/3685508674 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd18415af10 con 0x7fd18406f9e0 2023-10-26T01:39:36.040 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:36.034+0000 7fd188beb640 1 --2- 172.21.15.169:0/3685508674 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd174004d00 0x7fd18415a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:39:36.041 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:36.034+0000 7fd1893ec640 1 --2- 172.21.15.169:0/3685508674 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd18406f9e0 0x7fd18406fe60 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7fd17c009f40 tx=0x7fd17c033bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:36.042 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:36.035+0000 7fd1817fa640 1 -- 172.21.15.169:0/3685508674 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd17c047410 con 0x7fd18406f9e0 2023-10-26T01:39:36.042 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:36.035+0000 7fd1817fa640 1 -- 172.21.15.169:0/3685508674 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd17c047570 con 0x7fd18406f9e0 2023-10-26T01:39:36.042 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:36.035+0000 7fd1817fa640 1 -- 172.21.15.169:0/3685508674 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd17c047960 con 0x7fd18406f9e0 2023-10-26T01:39:36.043 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:36.035+0000 7fd189bed640 1 -- 172.21.15.169:0/3685508674 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd18415b1f0 con 0x7fd18406f9e0 2023-10-26T01:39:36.043 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:36.035+0000 7fd189bed640 1 -- 172.21.15.169:0/3685508674 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd18415b7b0 con 0x7fd18406f9e0 2023-10-26T01:39:36.043 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:36.036+0000 7fd189bed640 1 -- 172.21.15.169:0/3685508674 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1840675b0 con 0x7fd18406f9e0 2023-10-26T01:39:36.043 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:36.037+0000 7fd1817fa640 1 -- 172.21.15.169:0/3685508674 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fd17c03d410 con 0x7fd18406f9e0 2023-10-26T01:39:36.044 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:36.038+0000 7fd1817fa640 1 --2- 172.21.15.169:0/3685508674 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd16806ae40 0x7fd16806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:36.044 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:36.038+0000 7fd1817fa640 1 -- 172.21.15.169:0/3685508674 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd17c0abf70 con 0x7fd18406f9e0 2023-10-26T01:39:36.044 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:36.038+0000 7fd188beb640 1 --2- 172.21.15.169:0/3685508674 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd16806ae40 0x7fd16806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:36.044 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:36.039+0000 7fd188beb640 1 --2- 172.21.15.169:0/3685508674 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd16806ae40 0x7fd16806d300 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fd1780049a0 tx=0x7fd1780015c0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:36.045 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:36.042+0000 7fd1817fa640 1 -- 172.21.15.169:0/3685508674 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd17c077020 con 0x7fd18406f9e0 2023-10-26T01:39:36.233 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:36.230+0000 7fd189bed640 1 -- 172.21.15.169:0/3685508674 --> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd18406a8e0 con 0x7fd16806ae40 2023-10-26T01:39:36.255 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:36.252+0000 7fd1817fa640 1 -- 172.21.15.169:0/3685508674 <== mgr.4112 v2:172.21.15.170:6832/1187156918 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+282749 (secure 0 0 0) 0x7fd17800f970 con 0x7fd16806ae40 2023-10-26T01:39:36.259 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:39:36.260 INFO:teuthology.orchestra.run.smithi169.stderr:dumped all 2023-10-26T01:39:36.261 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:36.258+0000 7fd189bed640 1 -- 172.21.15.169:0/3685508674 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd16806ae40 msgr2=0x7fd16806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:36.262 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:36.258+0000 7fd189bed640 1 --2- 172.21.15.169:0/3685508674 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd16806ae40 0x7fd16806d300 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fd1780049a0 tx=0x7fd1780015c0 comp rx=0 tx=0).stop 2023-10-26T01:39:36.262 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:36.258+0000 7fd189bed640 1 -- 172.21.15.169:0/3685508674 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd18406f9e0 msgr2=0x7fd18406fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:36.262 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:36.258+0000 7fd189bed640 1 --2- 172.21.15.169:0/3685508674 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd18406f9e0 0x7fd18406fe60 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7fd17c009f40 tx=0x7fd17c033bd0 comp rx=0 tx=0).stop 2023-10-26T01:39:36.262 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:36.258+0000 7fd189bed640 1 -- 172.21.15.169:0/3685508674 shutdown_connections 2023-10-26T01:39:36.263 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:36.258+0000 7fd189bed640 1 --2- 172.21.15.169:0/3685508674 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd16806ae40 0x7fd16806d300 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:36.263 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:36.258+0000 7fd189bed640 1 --2- 172.21.15.169:0/3685508674 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd18406f9e0 0x7fd18406fe60 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:36.263 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:36.258+0000 7fd189bed640 1 --2- 172.21.15.169:0/3685508674 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd184147840 0x7fd18415bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:36.263 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:36.258+0000 7fd189bed640 1 --2- 172.21.15.169:0/3685508674 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd174004d00 0x7fd18415a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:36.263 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:36.258+0000 7fd189bed640 1 -- 172.21.15.169:0/3685508674 >> 172.21.15.169:0/3685508674 conn(0x7fd1840aebf0 msgr2=0x7fd1840ae480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:36.264 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:36.259+0000 7fd189bed640 1 -- 172.21.15.169:0/3685508674 shutdown_connections 2023-10-26T01:39:36.264 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:36.259+0000 7fd189bed640 1 -- 172.21.15.169:0/3685508674 wait complete. 2023-10-26T01:39:36.282 INFO:teuthology.orchestra.run.smithi169.stdout:{"pg_ready":true,"pg_map":{"version":416,"stamp":"2023-10-26T01:39:34.321702+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":7742,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1554757,"statfs":{"total":1600000000,"available":1592071986,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.003514"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:27.328671+0000","last_change":"2023-10-26T01:39:26.351701+0000","last_active":"2023-10-26T01:39:27.328671+0000","last_peered":"2023-10-26T01:39:27.328671+0000","last_clean":"2023-10-26T01:39:27.328671+0000","last_became_active":"2023-10-26T01:39:26.351436+0000","last_became_peered":"2023-10-26T01:39:26.351436+0000","last_unstale":"2023-10-26T01:39:27.328671+0000","last_undegraded":"2023-10-26T01:39:27.328671+0000","last_fullsized":"2023-10-26T01:39:27.328671+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:12:28.860114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:28.142220+0000","last_change":"2023-10-26T01:39:26.341419+0000","last_active":"2023-10-26T01:39:28.142220+0000","last_peered":"2023-10-26T01:39:28.142220+0000","last_clean":"2023-10-26T01:39:28.142220+0000","last_became_active":"2023-10-26T01:39:26.340948+0000","last_became_peered":"2023-10-26T01:39:26.340948+0000","last_unstale":"2023-10-26T01:39:28.142220+0000","last_undegraded":"2023-10-26T01:39:28.142220+0000","last_fullsized":"2023-10-26T01:39:28.142220+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:09:11.583914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:27.638498+0000","last_change":"2023-10-26T01:39:26.350477+0000","last_active":"2023-10-26T01:39:27.638498+0000","last_peered":"2023-10-26T01:39:27.638498+0000","last_clean":"2023-10-26T01:39:27.638498+0000","last_became_active":"2023-10-26T01:39:26.349381+0000","last_became_peered":"2023-10-26T01:39:26.349381+0000","last_unstale":"2023-10-26T01:39:27.638498+0000","last_undegraded":"2023-10-26T01:39:27.638498+0000","last_fullsized":"2023-10-26T01:39:27.638498+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:23:48.210950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:27.327834+0000","last_change":"2023-10-26T01:39:26.354717+0000","last_active":"2023-10-26T01:39:27.327834+0000","last_peered":"2023-10-26T01:39:27.327834+0000","last_clean":"2023-10-26T01:39:27.327834+0000","last_became_active":"2023-10-26T01:39:26.352974+0000","last_became_peered":"2023-10-26T01:39:26.352974+0000","last_unstale":"2023-10-26T01:39:27.327834+0000","last_undegraded":"2023-10-26T01:39:27.327834+0000","last_fullsized":"2023-10-26T01:39:27.327834+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:02:53.802284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:28.140538+0000","last_change":"2023-10-26T01:39:26.345030+0000","last_active":"2023-10-26T01:39:28.140538+0000","last_peered":"2023-10-26T01:39:28.140538+0000","last_clean":"2023-10-26T01:39:28.140538+0000","last_became_active":"2023-10-26T01:39:26.344794+0000","last_became_peered":"2023-10-26T01:39:26.344794+0000","last_unstale":"2023-10-26T01:39:28.140538+0000","last_undegraded":"2023-10-26T01:39:28.140538+0000","last_fullsized":"2023-10-26T01:39:28.140538+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:41:16.234137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:27.330344+0000","last_change":"2023-10-26T01:39:26.353950+0000","last_active":"2023-10-26T01:39:27.330344+0000","last_peered":"2023-10-26T01:39:27.330344+0000","last_clean":"2023-10-26T01:39:27.330344+0000","last_became_active":"2023-10-26T01:39:26.353628+0000","last_became_peered":"2023-10-26T01:39:26.353628+0000","last_unstale":"2023-10-26T01:39:27.330344+0000","last_undegraded":"2023-10-26T01:39:27.330344+0000","last_fullsized":"2023-10-26T01:39:27.330344+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:21:57.499684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:27.330344+0000","last_change":"2023-10-26T01:39:26.341531+0000","last_active":"2023-10-26T01:39:27.330344+0000","last_peered":"2023-10-26T01:39:27.330344+0000","last_clean":"2023-10-26T01:39:27.330344+0000","last_became_active":"2023-10-26T01:39:26.341067+0000","last_became_peered":"2023-10-26T01:39:26.341067+0000","last_unstale":"2023-10-26T01:39:27.330344+0000","last_undegraded":"2023-10-26T01:39:27.330344+0000","last_fullsized":"2023-10-26T01:39:27.330344+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:43:29.618712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:27.638877+0000","last_change":"2023-10-26T01:39:26.350068+0000","last_active":"2023-10-26T01:39:27.638877+0000","last_peered":"2023-10-26T01:39:27.638877+0000","last_clean":"2023-10-26T01:39:27.638877+0000","last_became_active":"2023-10-26T01:39:26.347040+0000","last_became_peered":"2023-10-26T01:39:26.347040+0000","last_unstale":"2023-10-26T01:39:27.638877+0000","last_undegraded":"2023-10-26T01:39:27.638877+0000","last_fullsized":"2023-10-26T01:39:27.638877+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:53:50.829218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:27.638293+0000","last_change":"2023-10-26T01:39:26.338913+0000","last_active":"2023-10-26T01:39:27.638293+0000","last_peered":"2023-10-26T01:39:27.638293+0000","last_clean":"2023-10-26T01:39:27.638293+0000","last_became_active":"2023-10-26T01:39:26.337570+0000","last_became_peered":"2023-10-26T01:39:26.337570+0000","last_unstale":"2023-10-26T01:39:27.638293+0000","last_undegraded":"2023-10-26T01:39:27.638293+0000","last_fullsized":"2023-10-26T01:39:27.638293+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:35:50.504438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:28.142398+0000","last_change":"2023-10-26T01:39:26.348032+0000","last_active":"2023-10-26T01:39:28.142398+0000","last_peered":"2023-10-26T01:39:28.142398+0000","last_clean":"2023-10-26T01:39:28.142398+0000","last_became_active":"2023-10-26T01:39:26.347285+0000","last_became_peered":"2023-10-26T01:39:26.347285+0000","last_unstale":"2023-10-26T01:39:28.142398+0000","last_undegraded":"2023-10-26T01:39:28.142398+0000","last_fullsized":"2023-10-26T01:39:28.142398+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:36:46.428985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:28.142801+0000","last_change":"2023-10-26T01:39:26.342837+0000","last_active":"2023-10-26T01:39:28.142801+0000","last_peered":"2023-10-26T01:39:28.142801+0000","last_clean":"2023-10-26T01:39:28.142801+0000","last_became_active":"2023-10-26T01:39:26.340874+0000","last_became_peered":"2023-10-26T01:39:26.340874+0000","last_unstale":"2023-10-26T01:39:28.142801+0000","last_undegraded":"2023-10-26T01:39:28.142801+0000","last_fullsized":"2023-10-26T01:39:28.142801+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:15:06.099166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:27.638597+0000","last_change":"2023-10-26T01:39:26.348088+0000","last_active":"2023-10-26T01:39:27.638597+0000","last_peered":"2023-10-26T01:39:27.638597+0000","last_clean":"2023-10-26T01:39:27.638597+0000","last_became_active":"2023-10-26T01:39:26.344699+0000","last_became_peered":"2023-10-26T01:39:26.344699+0000","last_unstale":"2023-10-26T01:39:27.638597+0000","last_undegraded":"2023-10-26T01:39:27.638597+0000","last_fullsized":"2023-10-26T01:39:27.638597+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:24:22.043171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:27.327974+0000","last_change":"2023-10-26T01:39:26.352736+0000","last_active":"2023-10-26T01:39:27.327974+0000","last_peered":"2023-10-26T01:39:27.327974+0000","last_clean":"2023-10-26T01:39:27.327974+0000","last_became_active":"2023-10-26T01:39:26.352430+0000","last_became_peered":"2023-10-26T01:39:26.352430+0000","last_unstale":"2023-10-26T01:39:27.327974+0000","last_undegraded":"2023-10-26T01:39:27.327974+0000","last_fullsized":"2023-10-26T01:39:27.327974+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:32:21.374255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:28.139328+0000","last_change":"2023-10-26T01:39:26.354058+0000","last_active":"2023-10-26T01:39:28.139328+0000","last_peered":"2023-10-26T01:39:28.139328+0000","last_clean":"2023-10-26T01:39:28.139328+0000","last_became_active":"2023-10-26T01:39:26.353830+0000","last_became_peered":"2023-10-26T01:39:26.353830+0000","last_unstale":"2023-10-26T01:39:28.139328+0000","last_undegraded":"2023-10-26T01:39:28.139328+0000","last_fullsized":"2023-10-26T01:39:28.139328+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T01:41:39.459935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:27.329206+0000","last_change":"2023-10-26T01:39:26.355709+0000","last_active":"2023-10-26T01:39:27.329206+0000","last_peered":"2023-10-26T01:39:27.329206+0000","last_clean":"2023-10-26T01:39:27.329206+0000","last_became_active":"2023-10-26T01:39:26.355291+0000","last_became_peered":"2023-10-26T01:39:26.355291+0000","last_unstale":"2023-10-26T01:39:27.329206+0000","last_undegraded":"2023-10-26T01:39:27.329206+0000","last_fullsized":"2023-10-26T01:39:27.329206+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:11:54.780066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:28.141997+0000","last_change":"2023-10-26T01:39:26.355761+0000","last_active":"2023-10-26T01:39:28.141997+0000","last_peered":"2023-10-26T01:39:28.141997+0000","last_clean":"2023-10-26T01:39:28.141997+0000","last_became_active":"2023-10-26T01:39:26.355188+0000","last_became_peered":"2023-10-26T01:39:26.355188+0000","last_unstale":"2023-10-26T01:39:28.141997+0000","last_undegraded":"2023-10-26T01:39:28.141997+0000","last_fullsized":"2023-10-26T01:39:28.141997+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:16:33.825857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:27.331035+0000","last_change":"2023-10-26T01:39:26.344036+0000","last_active":"2023-10-26T01:39:27.331035+0000","last_peered":"2023-10-26T01:39:27.331035+0000","last_clean":"2023-10-26T01:39:27.331035+0000","last_became_active":"2023-10-26T01:39:26.343529+0000","last_became_peered":"2023-10-26T01:39:26.343529+0000","last_unstale":"2023-10-26T01:39:27.331035+0000","last_undegraded":"2023-10-26T01:39:27.331035+0000","last_fullsized":"2023-10-26T01:39:27.331035+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:18:22.694978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:28.140768+0000","last_change":"2023-10-26T01:39:26.348373+0000","last_active":"2023-10-26T01:39:28.140768+0000","last_peered":"2023-10-26T01:39:28.140768+0000","last_clean":"2023-10-26T01:39:28.140768+0000","last_became_active":"2023-10-26T01:39:26.346867+0000","last_became_peered":"2023-10-26T01:39:26.346867+0000","last_unstale":"2023-10-26T01:39:28.140768+0000","last_undegraded":"2023-10-26T01:39:28.140768+0000","last_fullsized":"2023-10-26T01:39:28.140768+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:06:57.001025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:28.140225+0000","last_change":"2023-10-26T01:39:26.346678+0000","last_active":"2023-10-26T01:39:28.140225+0000","last_peered":"2023-10-26T01:39:28.140225+0000","last_clean":"2023-10-26T01:39:28.140225+0000","last_became_active":"2023-10-26T01:39:26.345850+0000","last_became_peered":"2023-10-26T01:39:26.345850+0000","last_unstale":"2023-10-26T01:39:28.140225+0000","last_undegraded":"2023-10-26T01:39:28.140225+0000","last_fullsized":"2023-10-26T01:39:28.140225+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:30:53.479231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:28.140766+0000","last_change":"2023-10-26T01:39:26.351200+0000","last_active":"2023-10-26T01:39:28.140766+0000","last_peered":"2023-10-26T01:39:28.140766+0000","last_clean":"2023-10-26T01:39:28.140766+0000","last_became_active":"2023-10-26T01:39:26.350941+0000","last_became_peered":"2023-10-26T01:39:26.350941+0000","last_unstale":"2023-10-26T01:39:28.140766+0000","last_undegraded":"2023-10-26T01:39:28.140766+0000","last_fullsized":"2023-10-26T01:39:28.140766+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:32:30.699815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:28.139984+0000","last_change":"2023-10-26T01:39:26.345413+0000","last_active":"2023-10-26T01:39:28.139984+0000","last_peered":"2023-10-26T01:39:28.139984+0000","last_clean":"2023-10-26T01:39:28.139984+0000","last_became_active":"2023-10-26T01:39:26.344357+0000","last_became_peered":"2023-10-26T01:39:26.344357+0000","last_unstale":"2023-10-26T01:39:28.139984+0000","last_undegraded":"2023-10-26T01:39:28.139984+0000","last_fullsized":"2023-10-26T01:39:28.139984+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:23:37.929077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:28.142698+0000","last_change":"2023-10-26T01:39:26.340246+0000","last_active":"2023-10-26T01:39:28.142698+0000","last_peered":"2023-10-26T01:39:28.142698+0000","last_clean":"2023-10-26T01:39:28.142698+0000","last_became_active":"2023-10-26T01:39:26.339423+0000","last_became_peered":"2023-10-26T01:39:26.339423+0000","last_unstale":"2023-10-26T01:39:28.142698+0000","last_undegraded":"2023-10-26T01:39:28.142698+0000","last_fullsized":"2023-10-26T01:39:28.142698+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:57:08.525225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:27.639689+0000","last_change":"2023-10-26T01:39:26.349003+0000","last_active":"2023-10-26T01:39:27.639689+0000","last_peered":"2023-10-26T01:39:27.639689+0000","last_clean":"2023-10-26T01:39:27.639689+0000","last_became_active":"2023-10-26T01:39:26.346464+0000","last_became_peered":"2023-10-26T01:39:26.346464+0000","last_unstale":"2023-10-26T01:39:27.639689+0000","last_undegraded":"2023-10-26T01:39:27.639689+0000","last_fullsized":"2023-10-26T01:39:27.639689+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:32:30.699815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:28.140555+0000","last_change":"2023-10-26T01:39:26.346891+0000","last_active":"2023-10-26T01:39:28.140555+0000","last_peered":"2023-10-26T01:39:28.140555+0000","last_clean":"2023-10-26T01:39:28.140555+0000","last_became_active":"2023-10-26T01:39:26.346328+0000","last_became_peered":"2023-10-26T01:39:26.346328+0000","last_unstale":"2023-10-26T01:39:28.140555+0000","last_undegraded":"2023-10-26T01:39:28.140555+0000","last_fullsized":"2023-10-26T01:39:28.140555+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:45:55.590196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:28.143427+0000","last_change":"2023-10-26T01:39:26.347884+0000","last_active":"2023-10-26T01:39:28.143427+0000","last_peered":"2023-10-26T01:39:28.143427+0000","last_clean":"2023-10-26T01:39:28.143427+0000","last_became_active":"2023-10-26T01:39:26.345967+0000","last_became_peered":"2023-10-26T01:39:26.345967+0000","last_unstale":"2023-10-26T01:39:28.143427+0000","last_undegraded":"2023-10-26T01:39:28.143427+0000","last_fullsized":"2023-10-26T01:39:28.143427+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:31:25.915011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:28.142335+0000","last_change":"2023-10-26T01:39:26.355914+0000","last_active":"2023-10-26T01:39:28.142335+0000","last_peered":"2023-10-26T01:39:28.142335+0000","last_clean":"2023-10-26T01:39:28.142335+0000","last_became_active":"2023-10-26T01:39:26.355508+0000","last_became_peered":"2023-10-26T01:39:26.355508+0000","last_unstale":"2023-10-26T01:39:28.142335+0000","last_undegraded":"2023-10-26T01:39:28.142335+0000","last_fullsized":"2023-10-26T01:39:28.142335+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:11:11.775125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:27.328041+0000","last_change":"2023-10-26T01:39:26.350921+0000","last_active":"2023-10-26T01:39:27.328041+0000","last_peered":"2023-10-26T01:39:27.328041+0000","last_clean":"2023-10-26T01:39:27.328041+0000","last_became_active":"2023-10-26T01:39:26.350660+0000","last_became_peered":"2023-10-26T01:39:26.350660+0000","last_unstale":"2023-10-26T01:39:27.328041+0000","last_undegraded":"2023-10-26T01:39:27.328041+0000","last_fullsized":"2023-10-26T01:39:27.328041+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:56:14.952742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:27.328045+0000","last_change":"2023-10-26T01:39:26.342982+0000","last_active":"2023-10-26T01:39:27.328045+0000","last_peered":"2023-10-26T01:39:27.328045+0000","last_clean":"2023-10-26T01:39:27.328045+0000","last_became_active":"2023-10-26T01:39:26.342537+0000","last_became_peered":"2023-10-26T01:39:26.342537+0000","last_unstale":"2023-10-26T01:39:27.328045+0000","last_undegraded":"2023-10-26T01:39:27.328045+0000","last_fullsized":"2023-10-26T01:39:27.328045+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:46:02.179502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:27.328100+0000","last_change":"2023-10-26T01:39:26.355396+0000","last_active":"2023-10-26T01:39:27.328100+0000","last_peered":"2023-10-26T01:39:27.328100+0000","last_clean":"2023-10-26T01:39:27.328100+0000","last_became_active":"2023-10-26T01:39:26.354952+0000","last_became_peered":"2023-10-26T01:39:26.354952+0000","last_unstale":"2023-10-26T01:39:27.328100+0000","last_undegraded":"2023-10-26T01:39:27.328100+0000","last_fullsized":"2023-10-26T01:39:27.328100+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:08:34.213156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:27.329367+0000","last_change":"2023-10-26T01:39:26.350190+0000","last_active":"2023-10-26T01:39:27.329367+0000","last_peered":"2023-10-26T01:39:27.329367+0000","last_clean":"2023-10-26T01:39:27.329367+0000","last_became_active":"2023-10-26T01:39:26.349816+0000","last_became_peered":"2023-10-26T01:39:26.349816+0000","last_unstale":"2023-10-26T01:39:27.329367+0000","last_undegraded":"2023-10-26T01:39:27.329367+0000","last_fullsized":"2023-10-26T01:39:27.329367+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:01:41.045976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:28.139502+0000","last_change":"2023-10-26T01:39:26.352978+0000","last_active":"2023-10-26T01:39:28.139502+0000","last_peered":"2023-10-26T01:39:28.139502+0000","last_clean":"2023-10-26T01:39:28.139502+0000","last_became_active":"2023-10-26T01:39:26.352497+0000","last_became_peered":"2023-10-26T01:39:26.352497+0000","last_unstale":"2023-10-26T01:39:28.139502+0000","last_undegraded":"2023-10-26T01:39:28.139502+0000","last_fullsized":"2023-10-26T01:39:28.139502+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:18:22.694978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:27.330717+0000","last_change":"2023-10-26T01:39:26.336994+0000","last_active":"2023-10-26T01:39:27.330717+0000","last_peered":"2023-10-26T01:39:27.330717+0000","last_clean":"2023-10-26T01:39:27.330717+0000","last_became_active":"2023-10-26T01:39:26.336376+0000","last_became_peered":"2023-10-26T01:39:26.336376+0000","last_unstale":"2023-10-26T01:39:27.330717+0000","last_undegraded":"2023-10-26T01:39:27.330717+0000","last_fullsized":"2023-10-26T01:39:27.330717+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:49:45.366235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:28.140822+0000","last_change":"2023-10-26T01:39:26.344022+0000","last_active":"2023-10-26T01:39:28.140822+0000","last_peered":"2023-10-26T01:39:28.140822+0000","last_clean":"2023-10-26T01:39:28.140822+0000","last_became_active":"2023-10-26T01:39:26.341874+0000","last_became_peered":"2023-10-26T01:39:26.341874+0000","last_unstale":"2023-10-26T01:39:28.140822+0000","last_undegraded":"2023-10-26T01:39:28.140822+0000","last_fullsized":"2023-10-26T01:39:28.140822+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:33:00.360810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"150'4","reported_seq":16,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-26T01:39:26.363349+0000","last_change":"2023-10-26T01:39:25.347523+0000","last_active":"2023-10-26T01:39:26.363349+0000","last_peered":"2023-10-26T01:39:26.363349+0000","last_clean":"2023-10-26T01:39:26.363349+0000","last_became_active":"2023-10-26T01:39:25.346388+0000","last_became_peered":"2023-10-26T01:39:25.346388+0000","last_unstale":"2023-10-26T01:39:26.363349+0000","last_undegraded":"2023-10-26T01:39:26.363349+0000","last_fullsized":"2023-10-26T01:39:26.363349+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:03:43.273094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:27.639190+0000","last_change":"2023-10-26T01:39:26.354015+0000","last_active":"2023-10-26T01:39:27.639190+0000","last_peered":"2023-10-26T01:39:27.639190+0000","last_clean":"2023-10-26T01:39:27.639190+0000","last_became_active":"2023-10-26T01:39:26.353804+0000","last_became_peered":"2023-10-26T01:39:26.353804+0000","last_unstale":"2023-10-26T01:39:27.639190+0000","last_undegraded":"2023-10-26T01:39:27.639190+0000","last_fullsized":"2023-10-26T01:39:27.639190+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:06:57.001025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"150'1","reported_seq":13,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-26T01:39:26.364024+0000","last_change":"2023-10-26T01:39:25.347683+0000","last_active":"2023-10-26T01:39:26.364024+0000","last_peered":"2023-10-26T01:39:26.364024+0000","last_clean":"2023-10-26T01:39:26.364024+0000","last_became_active":"2023-10-26T01:39:25.346998+0000","last_became_peered":"2023-10-26T01:39:25.346998+0000","last_unstale":"2023-10-26T01:39:26.364024+0000","last_undegraded":"2023-10-26T01:39:26.364024+0000","last_fullsized":"2023-10-26T01:39:26.364024+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:06:50.840083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:28.139786+0000","last_change":"2023-10-26T01:39:26.345123+0000","last_active":"2023-10-26T01:39:28.139786+0000","last_peered":"2023-10-26T01:39:28.139786+0000","last_clean":"2023-10-26T01:39:28.139786+0000","last_became_active":"2023-10-26T01:39:26.343161+0000","last_became_peered":"2023-10-26T01:39:26.343161+0000","last_unstale":"2023-10-26T01:39:28.139786+0000","last_undegraded":"2023-10-26T01:39:28.139786+0000","last_fullsized":"2023-10-26T01:39:28.139786+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:03:19.428986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"150'2","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-26T01:39:26.364267+0000","last_change":"2023-10-26T01:39:25.352802+0000","last_active":"2023-10-26T01:39:26.364267+0000","last_peered":"2023-10-26T01:39:26.364267+0000","last_clean":"2023-10-26T01:39:26.364267+0000","last_became_active":"2023-10-26T01:39:25.352537+0000","last_became_peered":"2023-10-26T01:39:25.352537+0000","last_unstale":"2023-10-26T01:39:26.364267+0000","last_undegraded":"2023-10-26T01:39:26.364267+0000","last_fullsized":"2023-10-26T01:39:26.364267+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:44:22.214795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:28.140858+0000","last_change":"2023-10-26T01:39:26.347952+0000","last_active":"2023-10-26T01:39:28.140858+0000","last_peered":"2023-10-26T01:39:28.140858+0000","last_clean":"2023-10-26T01:39:28.140858+0000","last_became_active":"2023-10-26T01:39:26.347693+0000","last_became_peered":"2023-10-26T01:39:26.347693+0000","last_unstale":"2023-10-26T01:39:28.140858+0000","last_undegraded":"2023-10-26T01:39:28.140858+0000","last_fullsized":"2023-10-26T01:39:28.140858+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:11:20.000707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"150'2","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-26T01:39:26.362995+0000","last_change":"2023-10-26T01:39:25.347942+0000","last_active":"2023-10-26T01:39:26.362995+0000","last_peered":"2023-10-26T01:39:26.362995+0000","last_clean":"2023-10-26T01:39:26.362995+0000","last_became_active":"2023-10-26T01:39:25.347373+0000","last_became_peered":"2023-10-26T01:39:25.347373+0000","last_unstale":"2023-10-26T01:39:26.362995+0000","last_undegraded":"2023-10-26T01:39:26.362995+0000","last_fullsized":"2023-10-26T01:39:26.362995+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:52:46.723602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:28.141108+0000","last_change":"2023-10-26T01:39:26.348640+0000","last_active":"2023-10-26T01:39:28.141108+0000","last_peered":"2023-10-26T01:39:28.141108+0000","last_clean":"2023-10-26T01:39:28.141108+0000","last_became_active":"2023-10-26T01:39:26.348127+0000","last_became_peered":"2023-10-26T01:39:26.348127+0000","last_unstale":"2023-10-26T01:39:28.141108+0000","last_undegraded":"2023-10-26T01:39:28.141108+0000","last_fullsized":"2023-10-26T01:39:28.141108+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:45:34.051362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"150'1","reported_seq":13,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-26T01:39:26.362430+0000","last_change":"2023-10-26T01:39:25.351159+0000","last_active":"2023-10-26T01:39:26.362430+0000","last_peered":"2023-10-26T01:39:26.362430+0000","last_clean":"2023-10-26T01:39:26.362430+0000","last_became_active":"2023-10-26T01:39:25.349181+0000","last_became_peered":"2023-10-26T01:39:25.349181+0000","last_unstale":"2023-10-26T01:39:26.362430+0000","last_undegraded":"2023-10-26T01:39:26.362430+0000","last_fullsized":"2023-10-26T01:39:26.362430+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:01:46.130180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:27.331253+0000","last_change":"2023-10-26T01:39:26.341329+0000","last_active":"2023-10-26T01:39:27.331253+0000","last_peered":"2023-10-26T01:39:27.331253+0000","last_clean":"2023-10-26T01:39:27.331253+0000","last_became_active":"2023-10-26T01:39:26.340693+0000","last_became_peered":"2023-10-26T01:39:26.340693+0000","last_unstale":"2023-10-26T01:39:27.331253+0000","last_undegraded":"2023-10-26T01:39:27.331253+0000","last_fullsized":"2023-10-26T01:39:27.331253+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:25:55.556501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"150'1","reported_seq":13,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-26T01:39:26.955278+0000","last_change":"2023-10-26T01:39:25.352234+0000","last_active":"2023-10-26T01:39:26.955278+0000","last_peered":"2023-10-26T01:39:26.955278+0000","last_clean":"2023-10-26T01:39:26.955278+0000","last_became_active":"2023-10-26T01:39:25.351930+0000","last_became_peered":"2023-10-26T01:39:25.351930+0000","last_unstale":"2023-10-26T01:39:26.955278+0000","last_undegraded":"2023-10-26T01:39:26.955278+0000","last_fullsized":"2023-10-26T01:39:26.955278+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:23:47.193891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:27.638488+0000","last_change":"2023-10-26T01:39:26.347620+0000","last_active":"2023-10-26T01:39:27.638488+0000","last_peered":"2023-10-26T01:39:27.638488+0000","last_clean":"2023-10-26T01:39:27.638488+0000","last_became_active":"2023-10-26T01:39:26.347251+0000","last_became_peered":"2023-10-26T01:39:26.347251+0000","last_unstale":"2023-10-26T01:39:27.638488+0000","last_undegraded":"2023-10-26T01:39:27.638488+0000","last_fullsized":"2023-10-26T01:39:27.638488+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:15:35.499430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-26T01:39:26.322931+0000","last_change":"2023-10-26T01:39:25.345280+0000","last_active":"2023-10-26T01:39:26.322931+0000","last_peered":"2023-10-26T01:39:26.322931+0000","last_clean":"2023-10-26T01:39:26.322931+0000","last_became_active":"2023-10-26T01:39:25.344841+0000","last_became_peered":"2023-10-26T01:39:25.344841+0000","last_unstale":"2023-10-26T01:39:26.322931+0000","last_undegraded":"2023-10-26T01:39:26.322931+0000","last_fullsized":"2023-10-26T01:39:26.322931+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:08:42.045168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:28.139704+0000","last_change":"2023-10-26T01:39:26.337467+0000","last_active":"2023-10-26T01:39:28.139704+0000","last_peered":"2023-10-26T01:39:28.139704+0000","last_clean":"2023-10-26T01:39:28.139704+0000","last_became_active":"2023-10-26T01:39:26.336744+0000","last_became_peered":"2023-10-26T01:39:26.336744+0000","last_unstale":"2023-10-26T01:39:28.139704+0000","last_undegraded":"2023-10-26T01:39:28.139704+0000","last_fullsized":"2023-10-26T01:39:28.139704+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:31:48.033303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-26T01:39:26.333002+0000","last_change":"2023-10-26T01:39:25.355147+0000","last_active":"2023-10-26T01:39:26.333002+0000","last_peered":"2023-10-26T01:39:26.333002+0000","last_clean":"2023-10-26T01:39:26.333002+0000","last_became_active":"2023-10-26T01:39:25.354564+0000","last_became_peered":"2023-10-26T01:39:25.354564+0000","last_unstale":"2023-10-26T01:39:26.333002+0000","last_undegraded":"2023-10-26T01:39:26.333002+0000","last_fullsized":"2023-10-26T01:39:26.333002+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:48:12.023136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:27.328575+0000","last_change":"2023-10-26T01:39:26.354194+0000","last_active":"2023-10-26T01:39:27.328575+0000","last_peered":"2023-10-26T01:39:27.328575+0000","last_clean":"2023-10-26T01:39:27.328575+0000","last_became_active":"2023-10-26T01:39:26.353639+0000","last_became_peered":"2023-10-26T01:39:26.353639+0000","last_unstale":"2023-10-26T01:39:27.328575+0000","last_undegraded":"2023-10-26T01:39:27.328575+0000","last_fullsized":"2023-10-26T01:39:27.328575+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:07:39.915731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-26T01:39:26.321608+0000","last_change":"2023-10-26T01:39:25.353938+0000","last_active":"2023-10-26T01:39:26.321608+0000","last_peered":"2023-10-26T01:39:26.321608+0000","last_clean":"2023-10-26T01:39:26.321608+0000","last_became_active":"2023-10-26T01:39:25.353704+0000","last_became_peered":"2023-10-26T01:39:25.353704+0000","last_unstale":"2023-10-26T01:39:26.321608+0000","last_undegraded":"2023-10-26T01:39:26.321608+0000","last_fullsized":"2023-10-26T01:39:26.321608+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:01:46.130180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:27.328408+0000","last_change":"2023-10-26T01:39:26.345228+0000","last_active":"2023-10-26T01:39:27.328408+0000","last_peered":"2023-10-26T01:39:27.328408+0000","last_clean":"2023-10-26T01:39:27.328408+0000","last_became_active":"2023-10-26T01:39:26.344986+0000","last_became_peered":"2023-10-26T01:39:26.344986+0000","last_unstale":"2023-10-26T01:39:27.328408+0000","last_undegraded":"2023-10-26T01:39:27.328408+0000","last_fullsized":"2023-10-26T01:39:27.328408+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:30:57.828517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-26T01:39:26.955344+0000","last_change":"2023-10-26T01:39:25.353364+0000","last_active":"2023-10-26T01:39:26.955344+0000","last_peered":"2023-10-26T01:39:26.955344+0000","last_clean":"2023-10-26T01:39:26.955344+0000","last_became_active":"2023-10-26T01:39:25.353129+0000","last_became_peered":"2023-10-26T01:39:25.353129+0000","last_unstale":"2023-10-26T01:39:26.955344+0000","last_undegraded":"2023-10-26T01:39:26.955344+0000","last_fullsized":"2023-10-26T01:39:26.955344+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:07:24.731619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:28.140226+0000","last_change":"2023-10-26T01:39:26.336002+0000","last_active":"2023-10-26T01:39:28.140226+0000","last_peered":"2023-10-26T01:39:28.140226+0000","last_clean":"2023-10-26T01:39:28.140226+0000","last_became_active":"2023-10-26T01:39:26.335418+0000","last_became_peered":"2023-10-26T01:39:26.335418+0000","last_unstale":"2023-10-26T01:39:28.140226+0000","last_undegraded":"2023-10-26T01:39:28.140226+0000","last_fullsized":"2023-10-26T01:39:28.140226+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:17:44.949060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"150'3","reported_seq":15,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-26T01:39:26.364835+0000","last_change":"2023-10-26T01:39:25.352244+0000","last_active":"2023-10-26T01:39:26.364835+0000","last_peered":"2023-10-26T01:39:26.364835+0000","last_clean":"2023-10-26T01:39:26.364835+0000","last_became_active":"2023-10-26T01:39:25.351989+0000","last_became_peered":"2023-10-26T01:39:25.351989+0000","last_unstale":"2023-10-26T01:39:26.364835+0000","last_undegraded":"2023-10-26T01:39:26.364835+0000","last_fullsized":"2023-10-26T01:39:26.364835+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:02:04.396694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:28.139651+0000","last_change":"2023-10-26T01:39:26.348419+0000","last_active":"2023-10-26T01:39:28.139651+0000","last_peered":"2023-10-26T01:39:28.139651+0000","last_clean":"2023-10-26T01:39:28.139651+0000","last_became_active":"2023-10-26T01:39:26.348038+0000","last_became_peered":"2023-10-26T01:39:26.348038+0000","last_unstale":"2023-10-26T01:39:28.139651+0000","last_undegraded":"2023-10-26T01:39:28.139651+0000","last_fullsized":"2023-10-26T01:39:28.139651+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:07:52.564388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"150'6","reported_seq":18,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-26T01:39:26.367176+0000","last_change":"2023-10-26T01:39:25.354828+0000","last_active":"2023-10-26T01:39:26.367176+0000","last_peered":"2023-10-26T01:39:26.367176+0000","last_clean":"2023-10-26T01:39:26.367176+0000","last_became_active":"2023-10-26T01:39:25.354185+0000","last_became_peered":"2023-10-26T01:39:25.354185+0000","last_unstale":"2023-10-26T01:39:26.367176+0000","last_undegraded":"2023-10-26T01:39:26.367176+0000","last_fullsized":"2023-10-26T01:39:26.367176+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:19:26.298128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:27.329619+0000","last_change":"2023-10-26T01:39:26.350033+0000","last_active":"2023-10-26T01:39:27.329619+0000","last_peered":"2023-10-26T01:39:27.329619+0000","last_clean":"2023-10-26T01:39:27.329619+0000","last_became_active":"2023-10-26T01:39:26.349373+0000","last_became_peered":"2023-10-26T01:39:26.349373+0000","last_unstale":"2023-10-26T01:39:27.329619+0000","last_undegraded":"2023-10-26T01:39:27.329619+0000","last_fullsized":"2023-10-26T01:39:27.329619+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:06:51.857142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-26T01:39:26.341127+0000","last_change":"2023-10-26T01:39:25.350711+0000","last_active":"2023-10-26T01:39:26.341127+0000","last_peered":"2023-10-26T01:39:26.341127+0000","last_clean":"2023-10-26T01:39:26.341127+0000","last_became_active":"2023-10-26T01:39:25.349869+0000","last_became_peered":"2023-10-26T01:39:25.349869+0000","last_unstale":"2023-10-26T01:39:26.341127+0000","last_undegraded":"2023-10-26T01:39:26.341127+0000","last_fullsized":"2023-10-26T01:39:26.341127+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:08:07.790237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:28.139599+0000","last_change":"2023-10-26T01:39:26.351491+0000","last_active":"2023-10-26T01:39:28.139599+0000","last_peered":"2023-10-26T01:39:28.139599+0000","last_clean":"2023-10-26T01:39:28.139599+0000","last_became_active":"2023-10-26T01:39:26.351251+0000","last_became_peered":"2023-10-26T01:39:26.351251+0000","last_unstale":"2023-10-26T01:39:28.139599+0000","last_undegraded":"2023-10-26T01:39:28.139599+0000","last_fullsized":"2023-10-26T01:39:28.139599+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:28:48.296290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-26T01:39:26.329630+0000","last_change":"2023-10-26T01:39:25.342525+0000","last_active":"2023-10-26T01:39:26.329630+0000","last_peered":"2023-10-26T01:39:26.329630+0000","last_clean":"2023-10-26T01:39:26.329630+0000","last_became_active":"2023-10-26T01:39:25.341183+0000","last_became_peered":"2023-10-26T01:39:25.341183+0000","last_unstale":"2023-10-26T01:39:26.329630+0000","last_undegraded":"2023-10-26T01:39:26.329630+0000","last_fullsized":"2023-10-26T01:39:26.329630+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:46:01.162443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:27.638820+0000","last_change":"2023-10-26T01:39:26.342999+0000","last_active":"2023-10-26T01:39:27.638820+0000","last_peered":"2023-10-26T01:39:27.638820+0000","last_clean":"2023-10-26T01:39:27.638820+0000","last_became_active":"2023-10-26T01:39:26.342725+0000","last_became_peered":"2023-10-26T01:39:26.342725+0000","last_unstale":"2023-10-26T01:39:27.638820+0000","last_undegraded":"2023-10-26T01:39:27.638820+0000","last_fullsized":"2023-10-26T01:39:27.638820+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:19:43.588241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"150'8","reported_seq":20,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-26T01:39:26.366328+0000","last_change":"2023-10-26T01:39:25.355183+0000","last_active":"2023-10-26T01:39:26.366328+0000","last_peered":"2023-10-26T01:39:26.366328+0000","last_clean":"2023-10-26T01:39:26.366328+0000","last_became_active":"2023-10-26T01:39:25.354751+0000","last_became_peered":"2023-10-26T01:39:25.354751+0000","last_unstale":"2023-10-26T01:39:26.366328+0000","last_undegraded":"2023-10-26T01:39:26.366328+0000","last_fullsized":"2023-10-26T01:39:26.366328+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:07:24.731619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:27.638713+0000","last_change":"2023-10-26T01:39:26.354040+0000","last_active":"2023-10-26T01:39:27.638713+0000","last_peered":"2023-10-26T01:39:27.638713+0000","last_clean":"2023-10-26T01:39:27.638713+0000","last_became_active":"2023-10-26T01:39:26.353736+0000","last_became_peered":"2023-10-26T01:39:26.353736+0000","last_unstale":"2023-10-26T01:39:27.638713+0000","last_undegraded":"2023-10-26T01:39:27.638713+0000","last_fullsized":"2023-10-26T01:39:27.638713+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:37:07.681720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"150'1","reported_seq":13,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-26T01:39:26.362952+0000","last_change":"2023-10-26T01:39:25.350279+0000","last_active":"2023-10-26T01:39:26.362952+0000","last_peered":"2023-10-26T01:39:26.362952+0000","last_clean":"2023-10-26T01:39:26.362952+0000","last_became_active":"2023-10-26T01:39:25.349301+0000","last_became_peered":"2023-10-26T01:39:25.349301+0000","last_unstale":"2023-10-26T01:39:26.362952+0000","last_undegraded":"2023-10-26T01:39:26.362952+0000","last_fullsized":"2023-10-26T01:39:26.362952+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:09:55.848777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:27.328126+0000","last_change":"2023-10-26T01:39:26.340528+0000","last_active":"2023-10-26T01:39:27.328126+0000","last_peered":"2023-10-26T01:39:27.328126+0000","last_clean":"2023-10-26T01:39:27.328126+0000","last_became_active":"2023-10-26T01:39:26.339638+0000","last_became_peered":"2023-10-26T01:39:26.339638+0000","last_unstale":"2023-10-26T01:39:27.328126+0000","last_undegraded":"2023-10-26T01:39:27.328126+0000","last_fullsized":"2023-10-26T01:39:27.328126+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:52:47.740661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"150'2","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-26T01:39:26.366258+0000","last_change":"2023-10-26T01:39:25.355508+0000","last_active":"2023-10-26T01:39:26.366258+0000","last_peered":"2023-10-26T01:39:26.366258+0000","last_clean":"2023-10-26T01:39:26.366258+0000","last_became_active":"2023-10-26T01:39:25.355254+0000","last_became_peered":"2023-10-26T01:39:25.355254+0000","last_unstale":"2023-10-26T01:39:26.366258+0000","last_undegraded":"2023-10-26T01:39:26.366258+0000","last_fullsized":"2023-10-26T01:39:26.366258+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:17:27.317892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:28.141591+0000","last_change":"2023-10-26T01:39:26.354034+0000","last_active":"2023-10-26T01:39:28.141591+0000","last_peered":"2023-10-26T01:39:28.141591+0000","last_clean":"2023-10-26T01:39:28.141591+0000","last_became_active":"2023-10-26T01:39:26.353811+0000","last_became_peered":"2023-10-26T01:39:26.353811+0000","last_unstale":"2023-10-26T01:39:28.141591+0000","last_undegraded":"2023-10-26T01:39:28.141591+0000","last_fullsized":"2023-10-26T01:39:28.141591+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:01:32.879490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"150'1","reported_seq":13,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-26T01:39:26.364322+0000","last_change":"2023-10-26T01:39:25.348929+0000","last_active":"2023-10-26T01:39:26.364322+0000","last_peered":"2023-10-26T01:39:26.364322+0000","last_clean":"2023-10-26T01:39:26.364322+0000","last_became_active":"2023-10-26T01:39:25.348637+0000","last_became_peered":"2023-10-26T01:39:25.348637+0000","last_unstale":"2023-10-26T01:39:26.364322+0000","last_undegraded":"2023-10-26T01:39:26.364322+0000","last_fullsized":"2023-10-26T01:39:26.364322+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:29:59.562191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:28.140859+0000","last_change":"2023-10-26T01:39:26.347586+0000","last_active":"2023-10-26T01:39:28.140859+0000","last_peered":"2023-10-26T01:39:28.140859+0000","last_clean":"2023-10-26T01:39:28.140859+0000","last_became_active":"2023-10-26T01:39:26.345838+0000","last_became_peered":"2023-10-26T01:39:26.345838+0000","last_unstale":"2023-10-26T01:39:28.140859+0000","last_undegraded":"2023-10-26T01:39:28.140859+0000","last_fullsized":"2023-10-26T01:39:28.140859+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:38:29.849919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-26T01:39:26.319713+0000","last_change":"2023-10-26T01:39:25.352675+0000","last_active":"2023-10-26T01:39:26.319713+0000","last_peered":"2023-10-26T01:39:26.319713+0000","last_clean":"2023-10-26T01:39:26.319713+0000","last_became_active":"2023-10-26T01:39:25.352188+0000","last_became_peered":"2023-10-26T01:39:25.352188+0000","last_unstale":"2023-10-26T01:39:26.319713+0000","last_undegraded":"2023-10-26T01:39:26.319713+0000","last_fullsized":"2023-10-26T01:39:26.319713+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:12:27.843055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:27.328235+0000","last_change":"2023-10-26T01:39:26.353259+0000","last_active":"2023-10-26T01:39:27.328235+0000","last_peered":"2023-10-26T01:39:27.328235+0000","last_clean":"2023-10-26T01:39:27.328235+0000","last_became_active":"2023-10-26T01:39:26.352540+0000","last_became_peered":"2023-10-26T01:39:26.352540+0000","last_unstale":"2023-10-26T01:39:27.328235+0000","last_undegraded":"2023-10-26T01:39:27.328235+0000","last_fullsized":"2023-10-26T01:39:27.328235+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:28:47.729543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"150'2","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-26T01:39:26.364335+0000","last_change":"2023-10-26T01:39:25.353938+0000","last_active":"2023-10-26T01:39:26.364335+0000","last_peered":"2023-10-26T01:39:26.364335+0000","last_clean":"2023-10-26T01:39:26.364335+0000","last_became_active":"2023-10-26T01:39:25.353665+0000","last_became_peered":"2023-10-26T01:39:25.353665+0000","last_unstale":"2023-10-26T01:39:26.364335+0000","last_undegraded":"2023-10-26T01:39:26.364335+0000","last_fullsized":"2023-10-26T01:39:26.364335+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T01:51:26.281651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:27.330257+0000","last_change":"2023-10-26T01:39:26.350893+0000","last_active":"2023-10-26T01:39:27.330257+0000","last_peered":"2023-10-26T01:39:27.330257+0000","last_clean":"2023-10-26T01:39:27.330257+0000","last_became_active":"2023-10-26T01:39:26.350621+0000","last_became_peered":"2023-10-26T01:39:26.350621+0000","last_unstale":"2023-10-26T01:39:27.330257+0000","last_undegraded":"2023-10-26T01:39:27.330257+0000","last_fullsized":"2023-10-26T01:39:27.330257+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:21:44.926700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-26T01:39:26.955203+0000","last_change":"2023-10-26T01:39:25.355042+0000","last_active":"2023-10-26T01:39:26.955203+0000","last_peered":"2023-10-26T01:39:26.955203+0000","last_clean":"2023-10-26T01:39:26.955203+0000","last_became_active":"2023-10-26T01:39:25.354798+0000","last_became_peered":"2023-10-26T01:39:25.354798+0000","last_unstale":"2023-10-26T01:39:26.955203+0000","last_undegraded":"2023-10-26T01:39:26.955203+0000","last_fullsized":"2023-10-26T01:39:26.955203+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:34:40.255257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:28.140269+0000","last_change":"2023-10-26T01:39:26.347810+0000","last_active":"2023-10-26T01:39:28.140269+0000","last_peered":"2023-10-26T01:39:28.140269+0000","last_clean":"2023-10-26T01:39:28.140269+0000","last_became_active":"2023-10-26T01:39:26.347135+0000","last_became_peered":"2023-10-26T01:39:26.347135+0000","last_unstale":"2023-10-26T01:39:28.140269+0000","last_undegraded":"2023-10-26T01:39:28.140269+0000","last_fullsized":"2023-10-26T01:39:28.140269+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:13:17.935865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-26T01:39:26.955481+0000","last_change":"2023-10-26T01:39:25.355043+0000","last_active":"2023-10-26T01:39:26.955481+0000","last_peered":"2023-10-26T01:39:26.955481+0000","last_clean":"2023-10-26T01:39:26.955481+0000","last_became_active":"2023-10-26T01:39:25.354518+0000","last_became_peered":"2023-10-26T01:39:25.354518+0000","last_unstale":"2023-10-26T01:39:26.955481+0000","last_undegraded":"2023-10-26T01:39:26.955481+0000","last_fullsized":"2023-10-26T01:39:26.955481+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:37:06.664661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:28.140159+0000","last_change":"2023-10-26T01:39:26.352876+0000","last_active":"2023-10-26T01:39:28.140159+0000","last_peered":"2023-10-26T01:39:28.140159+0000","last_clean":"2023-10-26T01:39:28.140159+0000","last_became_active":"2023-10-26T01:39:26.352556+0000","last_became_peered":"2023-10-26T01:39:26.352556+0000","last_unstale":"2023-10-26T01:39:28.140159+0000","last_undegraded":"2023-10-26T01:39:28.140159+0000","last_fullsized":"2023-10-26T01:39:28.140159+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:48:10.843465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-26T01:39:26.335781+0000","last_change":"2023-10-26T01:39:25.355568+0000","last_active":"2023-10-26T01:39:26.335781+0000","last_peered":"2023-10-26T01:39:26.335781+0000","last_clean":"2023-10-26T01:39:26.335781+0000","last_became_active":"2023-10-26T01:39:25.354482+0000","last_became_peered":"2023-10-26T01:39:25.354482+0000","last_unstale":"2023-10-26T01:39:26.335781+0000","last_undegraded":"2023-10-26T01:39:26.335781+0000","last_fullsized":"2023-10-26T01:39:26.335781+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:54:16.002490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:27.330836+0000","last_change":"2023-10-26T01:39:26.329595+0000","last_active":"2023-10-26T01:39:27.330836+0000","last_peered":"2023-10-26T01:39:27.330836+0000","last_clean":"2023-10-26T01:39:27.330836+0000","last_became_active":"2023-10-26T01:39:26.329022+0000","last_became_peered":"2023-10-26T01:39:26.329022+0000","last_unstale":"2023-10-26T01:39:27.330836+0000","last_undegraded":"2023-10-26T01:39:27.330836+0000","last_fullsized":"2023-10-26T01:39:27.330836+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:23:29.662601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"150'2","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-26T01:39:26.362529+0000","last_change":"2023-10-26T01:39:25.350493+0000","last_active":"2023-10-26T01:39:26.362529+0000","last_peered":"2023-10-26T01:39:26.362529+0000","last_clean":"2023-10-26T01:39:26.362529+0000","last_became_active":"2023-10-26T01:39:25.349758+0000","last_became_peered":"2023-10-26T01:39:25.349758+0000","last_unstale":"2023-10-26T01:39:26.362529+0000","last_undegraded":"2023-10-26T01:39:26.362529+0000","last_fullsized":"2023-10-26T01:39:26.362529+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:00:32.016316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:27.328922+0000","last_change":"2023-10-26T01:39:26.354438+0000","last_active":"2023-10-26T01:39:27.328922+0000","last_peered":"2023-10-26T01:39:27.328922+0000","last_clean":"2023-10-26T01:39:27.328922+0000","last_became_active":"2023-10-26T01:39:26.353968+0000","last_became_peered":"2023-10-26T01:39:26.353968+0000","last_unstale":"2023-10-26T01:39:27.328922+0000","last_undegraded":"2023-10-26T01:39:27.328922+0000","last_fullsized":"2023-10-26T01:39:27.328922+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:49:50.298664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"151'1","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:28.140060+0000","last_change":"2023-10-26T01:39:25.347830+0000","last_active":"2023-10-26T01:39:28.140060+0000","last_peered":"2023-10-26T01:39:28.140060+0000","last_clean":"2023-10-26T01:39:28.140060+0000","last_became_active":"2023-10-26T01:39:25.346994+0000","last_became_peered":"2023-10-26T01:39:25.346994+0000","last_unstale":"2023-10-26T01:39:28.140060+0000","last_undegraded":"2023-10-26T01:39:28.140060+0000","last_fullsized":"2023-10-26T01:39:28.140060+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:08:42.045168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:28.143832+0000","last_change":"2023-10-26T01:39:26.348169+0000","last_active":"2023-10-26T01:39:28.143832+0000","last_peered":"2023-10-26T01:39:28.143832+0000","last_clean":"2023-10-26T01:39:28.143832+0000","last_became_active":"2023-10-26T01:39:26.347642+0000","last_became_peered":"2023-10-26T01:39:26.347642+0000","last_unstale":"2023-10-26T01:39:28.143832+0000","last_undegraded":"2023-10-26T01:39:28.143832+0000","last_fullsized":"2023-10-26T01:39:28.143832+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:26:01.227512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-26T01:39:26.327792+0000","last_change":"2023-10-26T01:39:25.354096+0000","last_active":"2023-10-26T01:39:26.327792+0000","last_peered":"2023-10-26T01:39:26.327792+0000","last_clean":"2023-10-26T01:39:26.327792+0000","last_became_active":"2023-10-26T01:39:25.353857+0000","last_became_peered":"2023-10-26T01:39:25.353857+0000","last_unstale":"2023-10-26T01:39:26.327792+0000","last_undegraded":"2023-10-26T01:39:26.327792+0000","last_fullsized":"2023-10-26T01:39:26.327792+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:37:52.208922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:27.330171+0000","last_change":"2023-10-26T01:39:26.354645+0000","last_active":"2023-10-26T01:39:27.330171+0000","last_peered":"2023-10-26T01:39:27.330171+0000","last_clean":"2023-10-26T01:39:27.330171+0000","last_became_active":"2023-10-26T01:39:26.354090+0000","last_became_peered":"2023-10-26T01:39:26.354090+0000","last_unstale":"2023-10-26T01:39:27.330171+0000","last_undegraded":"2023-10-26T01:39:27.330171+0000","last_fullsized":"2023-10-26T01:39:27.330171+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:07:26.972765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-26T01:39:26.339415+0000","last_change":"2023-10-26T01:39:25.349514+0000","last_active":"2023-10-26T01:39:26.339415+0000","last_peered":"2023-10-26T01:39:26.339415+0000","last_clean":"2023-10-26T01:39:26.339415+0000","last_became_active":"2023-10-26T01:39:25.349279+0000","last_became_peered":"2023-10-26T01:39:25.349279+0000","last_unstale":"2023-10-26T01:39:26.339415+0000","last_undegraded":"2023-10-26T01:39:26.339415+0000","last_fullsized":"2023-10-26T01:39:26.339415+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:33:13.388324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:28.139534+0000","last_change":"2023-10-26T01:39:26.341277+0000","last_active":"2023-10-26T01:39:28.139534+0000","last_peered":"2023-10-26T01:39:28.139534+0000","last_clean":"2023-10-26T01:39:28.139534+0000","last_became_active":"2023-10-26T01:39:26.340759+0000","last_became_peered":"2023-10-26T01:39:26.340759+0000","last_unstale":"2023-10-26T01:39:28.139534+0000","last_undegraded":"2023-10-26T01:39:28.139534+0000","last_fullsized":"2023-10-26T01:39:28.139534+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:25:55.556501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-26T01:39:26.331459+0000","last_change":"2023-10-26T01:39:25.347759+0000","last_active":"2023-10-26T01:39:26.331459+0000","last_peered":"2023-10-26T01:39:26.331459+0000","last_clean":"2023-10-26T01:39:26.331459+0000","last_became_active":"2023-10-26T01:39:25.346564+0000","last_became_peered":"2023-10-26T01:39:25.346564+0000","last_unstale":"2023-10-26T01:39:26.331459+0000","last_undegraded":"2023-10-26T01:39:26.331459+0000","last_fullsized":"2023-10-26T01:39:26.331459+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:47:41.429100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:27.328818+0000","last_change":"2023-10-26T01:39:26.354552+0000","last_active":"2023-10-26T01:39:27.328818+0000","last_peered":"2023-10-26T01:39:27.328818+0000","last_clean":"2023-10-26T01:39:27.328818+0000","last_became_active":"2023-10-26T01:39:26.354256+0000","last_became_peered":"2023-10-26T01:39:26.354256+0000","last_unstale":"2023-10-26T01:39:27.328818+0000","last_undegraded":"2023-10-26T01:39:27.328818+0000","last_fullsized":"2023-10-26T01:39:27.328818+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:20:22.556497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-26T01:39:26.328763+0000","last_change":"2023-10-26T01:39:25.343413+0000","last_active":"2023-10-26T01:39:26.328763+0000","last_peered":"2023-10-26T01:39:26.328763+0000","last_clean":"2023-10-26T01:39:26.328763+0000","last_became_active":"2023-10-26T01:39:25.342949+0000","last_became_peered":"2023-10-26T01:39:25.342949+0000","last_unstale":"2023-10-26T01:39:26.328763+0000","last_undegraded":"2023-10-26T01:39:26.328763+0000","last_fullsized":"2023-10-26T01:39:26.328763+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:29:59.562191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:28.141156+0000","last_change":"2023-10-26T01:39:26.335564+0000","last_active":"2023-10-26T01:39:28.141156+0000","last_peered":"2023-10-26T01:39:28.141156+0000","last_clean":"2023-10-26T01:39:28.141156+0000","last_became_active":"2023-10-26T01:39:26.335000+0000","last_became_peered":"2023-10-26T01:39:26.335000+0000","last_unstale":"2023-10-26T01:39:28.141156+0000","last_undegraded":"2023-10-26T01:39:28.141156+0000","last_fullsized":"2023-10-26T01:39:28.141156+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:47:31.962180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-26T01:39:26.955302+0000","last_change":"2023-10-26T01:39:25.356190+0000","last_active":"2023-10-26T01:39:26.955302+0000","last_peered":"2023-10-26T01:39:26.955302+0000","last_clean":"2023-10-26T01:39:26.955302+0000","last_became_active":"2023-10-26T01:39:25.355791+0000","last_became_peered":"2023-10-26T01:39:25.355791+0000","last_unstale":"2023-10-26T01:39:26.955302+0000","last_undegraded":"2023-10-26T01:39:26.955302+0000","last_fullsized":"2023-10-26T01:39:26.955302+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:24:21.026112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:28.139979+0000","last_change":"2023-10-26T01:39:26.339884+0000","last_active":"2023-10-26T01:39:28.139979+0000","last_peered":"2023-10-26T01:39:28.139979+0000","last_clean":"2023-10-26T01:39:28.139979+0000","last_became_active":"2023-10-26T01:39:26.339195+0000","last_became_peered":"2023-10-26T01:39:26.339195+0000","last_unstale":"2023-10-26T01:39:28.139979+0000","last_undegraded":"2023-10-26T01:39:28.139979+0000","last_fullsized":"2023-10-26T01:39:28.139979+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:48:03.859702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-26T01:39:26.334872+0000","last_change":"2023-10-26T01:39:25.347950+0000","last_active":"2023-10-26T01:39:26.334872+0000","last_peered":"2023-10-26T01:39:26.334872+0000","last_clean":"2023-10-26T01:39:26.334872+0000","last_became_active":"2023-10-26T01:39:25.347416+0000","last_became_peered":"2023-10-26T01:39:25.347416+0000","last_unstale":"2023-10-26T01:39:26.334872+0000","last_undegraded":"2023-10-26T01:39:26.334872+0000","last_fullsized":"2023-10-26T01:39:26.334872+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:33:22.964454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-26T01:39:28.140264+0000","last_change":"2023-10-26T01:39:26.353099+0000","last_active":"2023-10-26T01:39:28.140264+0000","last_peered":"2023-10-26T01:39:28.140264+0000","last_clean":"2023-10-26T01:39:28.140264+0000","last_became_active":"2023-10-26T01:39:26.352798+0000","last_became_peered":"2023-10-26T01:39:26.352798+0000","last_unstale":"2023-10-26T01:39:28.140264+0000","last_undegraded":"2023-10-26T01:39:28.140264+0000","last_fullsized":"2023-10-26T01:39:28.140264+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:25.309143+0000","last_clean_scrub_stamp":"2023-10-26T01:39:25.309143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:51:09.620939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"150'2","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-26T01:39:26.362832+0000","last_change":"2023-10-26T01:39:25.352322+0000","last_active":"2023-10-26T01:39:26.362832+0000","last_peered":"2023-10-26T01:39:26.362832+0000","last_clean":"2023-10-26T01:39:26.362832+0000","last_became_active":"2023-10-26T01:39:25.351287+0000","last_became_peered":"2023-10-26T01:39:25.351287+0000","last_unstale":"2023-10-26T01:39:26.362832+0000","last_undegraded":"2023-10-26T01:39:26.362832+0000","last_fullsized":"2023-10-26T01:39:26.362832+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:39:24.292084+0000","last_clean_scrub_stamp":"2023-10-26T01:39:24.292084+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:49:44.349175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":294,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-26T01:39:26.326156+0000","last_change":"2023-10-26T01:30:34.883484+0000","last_active":"2023-10-26T01:39:26.326156+0000","last_peered":"2023-10-26T01:39:26.326156+0000","last_clean":"2023-10-26T01:39:26.326156+0000","last_became_active":"2023-10-26T01:30:34.883071+0000","last_became_peered":"2023-10-26T01:30:34.883071+0000","last_unstale":"2023-10-26T01:39:26.326156+0000","last_undegraded":"2023-10-26T01:39:26.326156+0000","last_fullsized":"2023-10-26T01:39:26.326156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:32.890096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:32.890096+0000","last_clean_scrub_stamp":"2023-10-26T01:30:32.890096+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:26:31.879452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":14,"seq":60129542255,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":823,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194489,"statfs":{"total":200000000,"available":199156956,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:38:51 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.443,"5min":0.35599999999999998,"15min":0.33000000000000002},"max":{"1min":1.2549999999999999,"5min":1.6319999999999999,"15min":2.2549999999999999},"last":0.88},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.92300000000000004,"15min":0.89300000000000002},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":1.752,"5min":1.752,"15min":1.8200000000000001},"last":0.93100000000000005}]},{"osd":1,"last update":"Thu Oct 26 01:38:51 2023","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.88400000000000001,"15min":0.92800000000000005},"min":{"1min":0.39400000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.383,"5min":1.444,"15min":4.4340000000000002},"last":1.052},{"interface":"front","average":{"1min":0.996,"5min":0.99199999999999999,"15min":0.995},"min":{"1min":0.56200000000000006,"5min":0.46700000000000003,"15min":0.374},"max":{"1min":1.353,"5min":1.538,"15min":2.3330000000000002},"last":0.77400000000000002}]},{"osd":2,"last update":"Thu Oct 26 01:38:51 2023","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":1.028,"15min":1.0700000000000001},"min":{"1min":0.54500000000000004,"5min":0.40999999999999998,"15min":0.39900000000000002},"max":{"1min":1.4279999999999999,"5min":1.927,"15min":4.2800000000000002},"last":0.99199999999999999},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.1479999999999999,"15min":1.0980000000000001},"min":{"1min":0.53300000000000003,"5min":0.495,"15min":0.49399999999999999},"max":{"1min":1.639,"5min":1.8680000000000001,"15min":2.302},"last":1.2490000000000001}]},{"osd":4,"last update":"Thu Oct 26 01:38:51 2023","interfaces":[{"interface":"back","average":{"1min":1.266,"5min":1.23,"15min":1.175},"min":{"1min":0.98199999999999998,"5min":0.755,"15min":0.66400000000000003},"max":{"1min":1.732,"5min":1.732,"15min":2.1070000000000002},"last":1.0369999999999999},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.353,"15min":1.3859999999999999},"min":{"1min":0.91900000000000004,"5min":0.81599999999999995,"15min":0.79900000000000004},"max":{"1min":1.6220000000000001,"5min":2.4239999999999999,"15min":4.8250000000000002},"last":1.1180000000000001}]},{"osd":5,"last update":"Thu Oct 26 01:38:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.2929999999999999,"15min":1.3280000000000001},"min":{"1min":0.84899999999999998,"5min":0.76400000000000001,"15min":0.72499999999999998},"max":{"1min":1.6899999999999999,"5min":2.3380000000000001,"15min":4.9080000000000004},"last":1.341},{"interface":"front","average":{"1min":1.331,"5min":1.2969999999999999,"15min":1.2789999999999999},"min":{"1min":0.82699999999999996,"5min":0.79900000000000004,"15min":0.79600000000000004},"max":{"1min":1.573,"5min":2.0310000000000001,"15min":2.8980000000000001},"last":1.375}]},{"osd":6,"last update":"Thu Oct 26 01:38:51 2023","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.351,"15min":1.329},"min":{"1min":0.75900000000000001,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.694,"5min":2.1499999999999999,"15min":2.8530000000000002},"last":1.4530000000000001},{"interface":"front","average":{"1min":1.4159999999999999,"5min":1.3160000000000001,"15min":1.2769999999999999},"min":{"1min":0.84399999999999997,"5min":0.81699999999999995,"15min":0.73099999999999998},"max":{"1min":1.6899999999999999,"5min":2.048,"15min":2.181},"last":1.093}]},{"osd":7,"last update":"Thu Oct 26 01:38:51 2023","interfaces":[{"interface":"back","average":{"1min":1.4370000000000001,"5min":1.3660000000000001,"15min":1.3069999999999999},"min":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.71999999999999997},"max":{"1min":1.7589999999999999,"5min":1.958,"15min":2.0680000000000001},"last":1.306},{"interface":"front","average":{"1min":1.385,"5min":1.393,"15min":1.411},"min":{"1min":0.73299999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.702,"5min":2.2370000000000001,"15min":4.8730000000000002},"last":1.2130000000000001}]}]},{"osd":7,"up_from":13,"seq":55834574958,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1401,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193910,"statfs":{"total":200000000,"available":198564498,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:38:50 2023","interfaces":[{"interface":"back","average":{"1min":1.127,"5min":1.1379999999999999,"15min":1.097},"min":{"1min":0.68700000000000006,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.633,"5min":1.7,"15min":1.9410000000000001},"last":1.034},{"interface":"front","average":{"1min":1.095,"5min":1.1100000000000001,"15min":1.0840000000000001},"min":{"1min":0.67300000000000004,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.6970000000000001,"5min":2.6779999999999999,"15min":2.6779999999999999},"last":1.417}]},{"osd":1,"last update":"Thu Oct 26 01:38:50 2023","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":1.1699999999999999,"15min":1.1020000000000001},"min":{"1min":0.82899999999999996,"5min":0.65800000000000003,"15min":0.65200000000000002},"max":{"1min":1.484,"5min":2.8260000000000001,"15min":2.8260000000000001},"last":0.875},{"interface":"front","average":{"1min":1.177,"5min":1.2,"15min":1.194},"min":{"1min":0.77200000000000002,"5min":0.66500000000000004,"15min":0.61199999999999999},"max":{"1min":1.76,"5min":3.2730000000000001,"15min":3.2730000000000001},"last":1.1679999999999999}]},{"osd":2,"last update":"Thu Oct 26 01:38:50 2023","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.2110000000000001,"15min":1.1299999999999999},"min":{"1min":0.873,"5min":0.69099999999999995,"15min":0.58499999999999996},"max":{"1min":1.851,"5min":2.218,"15min":2.218},"last":1.2529999999999999},{"interface":"front","average":{"1min":1.256,"5min":1.296,"15min":1.2430000000000001},"min":{"1min":0.82299999999999995,"5min":0.79400000000000004,"15min":0.73099999999999998},"max":{"1min":1.7350000000000001,"5min":3.0510000000000002,"15min":3.0510000000000002},"last":1.1259999999999999}]},{"osd":3,"last update":"Thu Oct 26 01:38:50 2023","interfaces":[{"interface":"back","average":{"1min":1.3120000000000001,"5min":1.296,"15min":1.2230000000000001},"min":{"1min":0.79500000000000004,"5min":0.79000000000000004,"15min":0.72099999999999997},"max":{"1min":1.8169999999999999,"5min":1.9830000000000001,"15min":1.9830000000000001},"last":1.073},{"interface":"front","average":{"1min":1.224,"5min":1.373,"15min":1.3640000000000001},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.65700000000000003},"max":{"1min":1.5129999999999999,"5min":5.5039999999999996,"15min":5.5039999999999996},"last":1.331}]},{"osd":4,"last update":"Thu Oct 26 01:38:50 2023","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.371,"15min":1.351},"min":{"1min":0.81799999999999995,"5min":0.77300000000000002,"15min":0.76100000000000001},"max":{"1min":1.9339999999999999,"5min":2.77,"15min":2.77},"last":2.0840000000000001},{"interface":"front","average":{"1min":1.288,"5min":1.3919999999999999,"15min":1.393},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.6479999999999999,"5min":3.1419999999999999,"15min":3.1419999999999999},"last":2.3929999999999998}]},{"osd":5,"last update":"Thu Oct 26 01:38:50 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.4410000000000001,"15min":1.4199999999999999},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.7809999999999999,"5min":5.734,"15min":5.734},"last":1.514},{"interface":"front","average":{"1min":1.4450000000000001,"5min":1.514,"15min":1.478},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.51900000000000002},"max":{"1min":2,"5min":2.8980000000000001,"15min":2.8980000000000001},"last":1.389}]},{"osd":6,"last update":"Thu Oct 26 01:38:50 2023","interfaces":[{"interface":"back","average":{"1min":1.452,"5min":1.5800000000000001,"15min":1.8089999999999999},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.47399999999999998},"max":{"1min":1.8049999999999999,"5min":5.6340000000000003,"15min":11.577999999999999},"last":1.583},{"interface":"front","average":{"1min":1.613,"5min":1.6719999999999999,"15min":1.8360000000000001},"min":{"1min":1.0640000000000001,"5min":0.91800000000000004,"15min":0.55100000000000005},"max":{"1min":2.0529999999999999,"5min":5.0919999999999996,"15min":10.704000000000001},"last":1.476}]}]},{"osd":6,"up_from":13,"seq":55834574958,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":824,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194488,"statfs":{"total":200000000,"available":199156118,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:38:46 2023","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.141,"15min":1.1519999999999999},"min":{"1min":0.66600000000000004,"5min":0.61699999999999999,"15min":0.60399999999999998},"max":{"1min":2.7370000000000001,"5min":2.7370000000000001,"15min":2.7370000000000001},"last":1.228},{"interface":"front","average":{"1min":1.137,"5min":1.173,"15min":1.123},"min":{"1min":0.61699999999999999,"5min":0.61399999999999999,"15min":0.59199999999999997},"max":{"1min":4.4779999999999998,"5min":4.4779999999999998,"15min":4.4779999999999998},"last":1.1819999999999999}]},{"osd":1,"last update":"Thu Oct 26 01:38:46 2023","interfaces":[{"interface":"back","average":{"1min":1.3919999999999999,"5min":1.2729999999999999,"15min":1.1930000000000001},"min":{"1min":0.77100000000000002,"5min":0.72299999999999998,"15min":0.68300000000000005},"max":{"1min":5.5359999999999996,"5min":5.5359999999999996,"15min":5.5359999999999996},"last":1.046},{"interface":"front","average":{"1min":1.228,"5min":1.2190000000000001,"15min":1.204},"min":{"1min":0.74399999999999999,"5min":0.67900000000000005,"15min":0.64000000000000001},"max":{"1min":4.3579999999999997,"5min":4.3579999999999997,"15min":4.3579999999999997},"last":1.5109999999999999}]},{"osd":2,"last update":"Thu Oct 26 01:38:46 2023","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.2030000000000001,"15min":1.107},"min":{"1min":0.76300000000000001,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":2.367,"5min":3.7879999999999998,"15min":3.7879999999999998},"last":1.302},{"interface":"front","average":{"1min":1.371,"5min":1.3420000000000001,"15min":1.276},"min":{"1min":0.91200000000000003,"5min":0.75800000000000001,"15min":0.68899999999999995},"max":{"1min":3.0470000000000002,"5min":3.0470000000000002,"15min":3.0470000000000002},"last":1.4259999999999999}]},{"osd":3,"last update":"Thu Oct 26 01:38:46 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.331,"15min":1.22},"min":{"1min":0.76300000000000001,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":3.266,"5min":4.4560000000000004,"15min":4.4560000000000004},"last":1.5800000000000001},{"interface":"front","average":{"1min":1.365,"5min":1.3500000000000001,"15min":1.3280000000000001},"min":{"1min":0.86399999999999999,"5min":0.78600000000000003,"15min":0.69499999999999995},"max":{"1min":2.6909999999999998,"5min":2.6909999999999998,"15min":2.6909999999999998},"last":1.3620000000000001}]},{"osd":4,"last update":"Thu Oct 26 01:38:46 2023","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.24,"15min":1.22},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.48399999999999999},"max":{"1min":1.7410000000000001,"5min":2.9489999999999998,"15min":2.9489999999999998},"last":1.262},{"interface":"front","average":{"1min":1.3180000000000001,"5min":1.421,"15min":1.319},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.59299999999999997},"max":{"1min":1.698,"5min":6.609,"15min":6.609},"last":0.98499999999999999}]},{"osd":5,"last update":"Thu Oct 26 01:38:46 2023","interfaces":[{"interface":"back","average":{"1min":1.464,"5min":1.5389999999999999,"15min":1.452},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.7949999999999999,"5min":5.4370000000000003,"15min":5.4370000000000003},"last":1.6539999999999999},{"interface":"front","average":{"1min":1.492,"5min":1.4530000000000001,"15min":1.419},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":2.7759999999999998,"5min":2.7759999999999998,"15min":2.7759999999999998},"last":1.5469999999999999}]},{"osd":7,"last update":"Thu Oct 26 01:38:46 2023","interfaces":[{"interface":"back","average":{"1min":1.2869999999999999,"5min":1.484,"15min":1.405},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.54100000000000004},"max":{"1min":1.8200000000000001,"5min":11.804,"15min":11.804},"last":1.3979999999999999},{"interface":"front","average":{"1min":1.577,"5min":1.6639999999999999,"15min":1.52},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":3.7650000000000001,"5min":11.541,"15min":11.541},"last":1.464}]}]},{"osd":5,"up_from":13,"seq":55834574958,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":823,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194489,"statfs":{"total":200000000,"available":199157004,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:38:48 2023","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.2310000000000001,"15min":1.147},"min":{"1min":0.69699999999999995,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":3.8210000000000002,"5min":3.8210000000000002,"15min":3.8210000000000002},"last":1.5800000000000001},{"interface":"front","average":{"1min":1.3480000000000001,"5min":1.302,"15min":1.1970000000000001},"min":{"1min":0.92600000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":2.9889999999999999,"5min":2.9889999999999999,"15min":2.9889999999999999},"last":1.673}]},{"osd":1,"last update":"Thu Oct 26 01:38:48 2023","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.1319999999999999,"15min":1.0940000000000001},"min":{"1min":0.83599999999999997,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.8120000000000001,"5min":1.8120000000000001,"15min":1.8120000000000001},"last":1.377},{"interface":"front","average":{"1min":1.272,"5min":1.1659999999999999,"15min":1.109},"min":{"1min":0.76700000000000002,"5min":0.629,"15min":0.60599999999999998},"max":{"1min":2.4340000000000002,"5min":2.4340000000000002,"15min":2.4340000000000002},"last":0.97499999999999998}]},{"osd":2,"last update":"Thu Oct 26 01:38:48 2023","interfaces":[{"interface":"back","average":{"1min":1.2130000000000001,"5min":1.206,"15min":1.1679999999999999},"min":{"1min":0.79700000000000004,"5min":0.71599999999999997,"15min":0.69299999999999995},"max":{"1min":1.5920000000000001,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":1.141},{"interface":"front","average":{"1min":1.228,"5min":1.228,"15min":1.1850000000000001},"min":{"1min":0.91600000000000004,"5min":0.78400000000000003,"15min":0.63900000000000001},"max":{"1min":1.5900000000000001,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":1.2230000000000001}]},{"osd":3,"last update":"Thu Oct 26 01:38:48 2023","interfaces":[{"interface":"back","average":{"1min":1.389,"5min":1.2829999999999999,"15min":1.25},"min":{"1min":1.0009999999999999,"5min":0.72699999999999998,"15min":0.72299999999999998},"max":{"1min":1.7,"5min":1.7,"15min":1.835},"last":1.629},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.298,"15min":1.29},"min":{"1min":0.91300000000000003,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.52,"5min":1.788,"15min":2.9249999999999998},"last":1.5}]},{"osd":4,"last update":"Thu Oct 26 01:38:48 2023","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":1.256,"15min":1.214},"min":{"1min":0.75,"5min":0.59599999999999997,"15min":0.49099999999999999},"max":{"1min":2.7770000000000001,"5min":2.7770000000000001,"15min":2.7770000000000001},"last":1.427},{"interface":"front","average":{"1min":1.3180000000000001,"5min":1.3979999999999999,"15min":1.355},"min":{"1min":0.67500000000000004,"5min":0.57299999999999995,"15min":0.56200000000000006},"max":{"1min":1.776,"5min":1.7849999999999999,"15min":1.867},"last":1.857}]},{"osd":6,"last update":"Thu Oct 26 01:38:48 2023","interfaces":[{"interface":"back","average":{"1min":1.377,"5min":1.393,"15min":1.3300000000000001},"min":{"1min":0.79000000000000004,"5min":0.67500000000000004,"15min":0.51900000000000002},"max":{"1min":1.956,"5min":2.8250000000000002,"15min":2.8250000000000002},"last":1.8069999999999999},{"interface":"front","average":{"1min":1.4590000000000001,"5min":1.409,"15min":1.3400000000000001},"min":{"1min":0.89300000000000002,"5min":0.69499999999999995,"15min":0.61799999999999999},"max":{"1min":2.4830000000000001,"5min":2.4830000000000001,"15min":2.4830000000000001},"last":1.7310000000000001}]},{"osd":7,"last update":"Thu Oct 26 01:38:48 2023","interfaces":[{"interface":"back","average":{"1min":1.6000000000000001,"5min":1.534,"15min":1.4379999999999999},"min":{"1min":0.81499999999999995,"5min":0.71099999999999997,"15min":0.61699999999999999},"max":{"1min":3.621,"5min":3.621,"15min":3.621},"last":1.9750000000000001},{"interface":"front","average":{"1min":1.603,"5min":1.542,"15min":1.4470000000000001},"min":{"1min":0.82899999999999996,"5min":0.73899999999999999,"15min":0.59499999999999997},"max":{"1min":2.8010000000000002,"5min":2.8010000000000002,"15min":2.8010000000000002},"last":1.915}]}]},{"osd":4,"up_from":13,"seq":55834574958,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":823,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194489,"statfs":{"total":200000000,"available":199157046,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:38:43 2023","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.071,"15min":0.999},"min":{"1min":0.63500000000000001,"5min":0.59799999999999998,"15min":0.53100000000000003},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":2.0289999999999999},"last":0.95099999999999996},{"interface":"front","average":{"1min":1.123,"5min":1.077,"15min":1.0329999999999999},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.60599999999999998},"max":{"1min":1.784,"5min":1.784,"15min":2.6829999999999998},"last":1.125}]},{"osd":1,"last update":"Thu Oct 26 01:38:43 2023","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":1.161,"15min":1.1259999999999999},"min":{"1min":0.749,"5min":0.66700000000000004,"15min":0.56799999999999995},"max":{"1min":1.579,"5min":1.599,"15min":4.0519999999999996},"last":1.018},{"interface":"front","average":{"1min":1.2529999999999999,"5min":1.1779999999999999,"15min":1.121},"min":{"1min":0.70799999999999996,"5min":0.70399999999999996,"15min":0.60499999999999998},"max":{"1min":2.3919999999999999,"5min":2.3919999999999999,"15min":2.3919999999999999},"last":1.3120000000000001}]},{"osd":2,"last update":"Thu Oct 26 01:38:43 2023","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":1.304,"15min":1.2430000000000001},"min":{"1min":1.0429999999999999,"5min":0.66700000000000004,"15min":0.64300000000000002},"max":{"1min":1.6759999999999999,"5min":1.845,"15min":5.6749999999999998},"last":1.3560000000000001},{"interface":"front","average":{"1min":1.1799999999999999,"5min":1.218,"15min":1.1419999999999999},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.631,"5min":3.673,"15min":3.673},"last":1.3819999999999999}]},{"osd":3,"last update":"Thu Oct 26 01:38:43 2023","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.228,"15min":1.1919999999999999},"min":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.70399999999999996},"max":{"1min":1.7030000000000001,"5min":1.847,"15min":1.891},"last":1.4350000000000001},{"interface":"front","average":{"1min":1.1799999999999999,"5min":1.204,"15min":1.1619999999999999},"min":{"1min":0.86099999999999999,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.522,"5min":1.681,"15min":1.7210000000000001},"last":1.0680000000000001}]},{"osd":5,"last update":"Thu Oct 26 01:38:43 2023","interfaces":[{"interface":"back","average":{"1min":1.236,"5min":1.258,"15min":1.2809999999999999},"min":{"1min":0.85699999999999998,"5min":0.52100000000000002,"15min":0.46300000000000002},"max":{"1min":1.681,"5min":2.1429999999999998,"15min":2.1429999999999998},"last":1.5569999999999999},{"interface":"front","average":{"1min":1.2270000000000001,"5min":1.2809999999999999,"15min":1.321},"min":{"1min":0.84899999999999998,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.754,"5min":1.9810000000000001,"15min":5.8579999999999997},"last":1.524}]},{"osd":6,"last update":"Thu Oct 26 01:38:43 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.294,"15min":1.335},"min":{"1min":0.752,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.708,"5min":2.2149999999999999,"15min":12.343999999999999},"last":1.486},{"interface":"front","average":{"1min":1.5129999999999999,"5min":1.5109999999999999,"15min":1.5189999999999999},"min":{"1min":1.0660000000000001,"5min":0.57999999999999996,"15min":0.55000000000000004},"max":{"1min":1.762,"5min":2.117,"15min":12.605},"last":1.6220000000000001}]},{"osd":7,"last update":"Thu Oct 26 01:38:43 2023","interfaces":[{"interface":"back","average":{"1min":1.4690000000000001,"5min":1.4670000000000001,"15min":1.49},"min":{"1min":1.1279999999999999,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.8200000000000001,"5min":2.762,"15min":5.774},"last":1.915},{"interface":"front","average":{"1min":1.399,"5min":1.484,"15min":1.4590000000000001},"min":{"1min":1.008,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.7589999999999999,"5min":2.8330000000000002,"15min":4.6520000000000001},"last":1.671}]}]},{"osd":2,"up_from":13,"seq":55834574958,"num_pgs":29,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":823,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194488,"statfs":{"total":200000000,"available":199156500,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:38:47 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.84499999999999997,"15min":0.85299999999999998},"min":{"1min":0.46600000000000003,"5min":0.41199999999999998,"15min":0.308},"max":{"1min":1.4079999999999999,"5min":1.5249999999999999,"15min":6.3200000000000003},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.97699999999999998,"15min":0.93300000000000005},"min":{"1min":0.44,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.476,"5min":1.9970000000000001,"15min":4.2619999999999996},"last":0.77100000000000002}]},{"osd":1,"last update":"Thu Oct 26 01:38:47 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.878,"15min":0.89100000000000001},"min":{"1min":0.48399999999999999,"5min":0.39300000000000002,"15min":0.34000000000000002},"max":{"1min":1.2829999999999999,"5min":1.629,"15min":5.1260000000000003},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.98499999999999999,"5min":1.0189999999999999,"15min":1.0029999999999999},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.36599999999999999},"max":{"1min":1.4870000000000001,"5min":1.873,"15min":6.1520000000000001},"last":1.2809999999999999}]},{"osd":3,"last update":"Thu Oct 26 01:38:52 2023","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":1.004,"15min":0.94499999999999995},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.407,"5min":2.0640000000000001,"15min":2.0640000000000001},"last":0.93500000000000005},{"interface":"front","average":{"1min":1.0349999999999999,"5min":1.0860000000000001,"15min":1.0249999999999999},"min":{"1min":0.53800000000000003,"5min":0.51100000000000001,"15min":0.45300000000000001},"max":{"1min":1.665,"5min":1.9279999999999999,"15min":8.9019999999999992},"last":0.999}]},{"osd":4,"last update":"Thu Oct 26 01:38:47 2023","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.3280000000000001,"15min":1.2689999999999999},"min":{"1min":0.81599999999999995,"5min":0.80500000000000005,"15min":0.73899999999999999},"max":{"1min":1.6080000000000001,"5min":2.6190000000000002,"15min":6.2670000000000003},"last":1.3460000000000001},{"interface":"front","average":{"1min":1.2709999999999999,"5min":1.2969999999999999,"15min":1.266},"min":{"1min":0.81699999999999995,"5min":0.81100000000000005,"15min":0.78700000000000003},"max":{"1min":1.8380000000000001,"5min":2.4359999999999999,"15min":7.343},"last":1.1200000000000001}]},{"osd":5,"last update":"Thu Oct 26 01:38:47 2023","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":1.3500000000000001,"15min":1.29},"min":{"1min":0.85099999999999998,"5min":0.81699999999999995,"15min":0.71899999999999997},"max":{"1min":1.669,"5min":2.5790000000000002,"15min":3.7170000000000001},"last":1.385},{"interface":"front","average":{"1min":1.302,"5min":1.3120000000000001,"15min":1.2929999999999999},"min":{"1min":0.872,"5min":0.81899999999999995,"15min":0.65300000000000002},"max":{"1min":1.708,"5min":2.298,"15min":6.3899999999999997},"last":1.421}]},{"osd":6,"last update":"Thu Oct 26 01:38:47 2023","interfaces":[{"interface":"back","average":{"1min":1.3280000000000001,"5min":1.3919999999999999,"15min":1.3380000000000001},"min":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.70599999999999996},"max":{"1min":1.714,"5min":2.73,"15min":6.0490000000000004},"last":1.1839999999999999},{"interface":"front","average":{"1min":1.379,"5min":1.4139999999999999,"15min":1.3740000000000001},"min":{"1min":0.89200000000000002,"5min":0.78200000000000003,"15min":0.76800000000000002},"max":{"1min":2.0990000000000002,"5min":2.4089999999999998,"15min":5.0650000000000004},"last":1.4750000000000001}]},{"osd":7,"last update":"Thu Oct 26 01:38:47 2023","interfaces":[{"interface":"back","average":{"1min":1.373,"5min":1.3979999999999999,"15min":1.3939999999999999},"min":{"1min":0.79200000000000004,"5min":0.77900000000000003,"15min":0.69399999999999995},"max":{"1min":1.776,"5min":2.3370000000000002,"15min":6.4480000000000004},"last":1.625},{"interface":"front","average":{"1min":1.4830000000000001,"5min":1.5169999999999999,"15min":1.458},"min":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.91800000000000004},"max":{"1min":2.1480000000000001,"5min":2.8290000000000002,"15min":2.8439999999999999},"last":1.5960000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574958,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":823,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194489,"statfs":{"total":200000000,"available":199157116,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:38:49 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.76900000000000002,"15min":0.72799999999999998},"min":{"1min":0.41799999999999998,"5min":0.34000000000000002,"15min":0.29499999999999998},"max":{"1min":0.92700000000000005,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.87,"15min":0.80200000000000005},"min":{"1min":0.52000000000000002,"5min":0.36299999999999999,"15min":0.36199999999999999},"max":{"1min":1.2769999999999999,"5min":1.6399999999999999,"15min":1.6399999999999999},"last":1.1910000000000001}]},{"osd":2,"last update":"Thu Oct 26 01:38:49 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.995,"15min":0.999},"min":{"1min":0.65300000000000002,"5min":0.39700000000000002,"15min":0.39600000000000002},"max":{"1min":1.286,"5min":1.8200000000000001,"15min":2.4830000000000001},"last":1.292},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.96199999999999997,"15min":0.88400000000000001},"min":{"1min":0.48099999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.569,"5min":1.8080000000000001,"15min":1.8080000000000001},"last":1.1499999999999999}]},{"osd":3,"last update":"Thu Oct 26 01:38:53 2023","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":0.996,"15min":0.95499999999999996},"min":{"1min":0.59999999999999998,"5min":0.42499999999999999,"15min":0.39500000000000002},"max":{"1min":1.5900000000000001,"5min":1.611,"15min":1.75},"last":0.872},{"interface":"front","average":{"1min":0.998,"5min":1.028,"15min":0.99199999999999999},"min":{"1min":0.55000000000000004,"5min":0.47799999999999998,"15min":0.41299999999999998},"max":{"1min":1.5049999999999999,"5min":1.673,"15min":1.802},"last":0.99099999999999999}]},{"osd":4,"last update":"Thu Oct 26 01:38:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":1.222,"15min":1.1990000000000001},"min":{"1min":0.90400000000000003,"5min":0.69099999999999995,"15min":0.56200000000000006},"max":{"1min":1.5249999999999999,"5min":1.756,"15min":2.0659999999999998},"last":1.538},{"interface":"front","average":{"1min":1.1659999999999999,"5min":1.1870000000000001,"15min":1.1399999999999999},"min":{"1min":0.81299999999999994,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.4390000000000001,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":1.351}]},{"osd":5,"last update":"Thu Oct 26 01:38:49 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.2629999999999999,"15min":1.2290000000000001},"min":{"1min":0.84799999999999998,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.621,"5min":1.8779999999999999,"15min":1.8779999999999999},"last":1.4890000000000001},{"interface":"front","average":{"1min":1.359,"5min":1.375,"15min":1.339},"min":{"1min":1.052,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.7210000000000001,"5min":2.04,"15min":2.2200000000000002},"last":1.45}]},{"osd":6,"last update":"Thu Oct 26 01:38:49 2023","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.347,"15min":1.2849999999999999},"min":{"1min":0.90400000000000003,"5min":0.70199999999999996,"15min":0.68899999999999995},"max":{"1min":1.7709999999999999,"5min":2.0910000000000002,"15min":2.3780000000000001},"last":1.5169999999999999},{"interface":"front","average":{"1min":1.2889999999999999,"5min":1.2789999999999999,"15min":1.2490000000000001},"min":{"1min":0.92700000000000005,"5min":0.80700000000000005,"15min":0.70799999999999996},"max":{"1min":1.653,"5min":1.956,"15min":1.984},"last":1.2529999999999999}]},{"osd":7,"last update":"Thu Oct 26 01:38:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.2749999999999999,"15min":1.262},"min":{"1min":0.79400000000000004,"5min":0.76100000000000001,"15min":0.69699999999999995},"max":{"1min":1.764,"5min":1.8380000000000001,"15min":2.6930000000000001},"last":1.4239999999999999},{"interface":"front","average":{"1min":1.431,"5min":1.419,"15min":1.345},"min":{"1min":0.95499999999999996,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.835,"5min":2.0190000000000001,"15min":2.0190000000000001},"last":1.569}]}]},{"osd":0,"up_from":13,"seq":55834574958,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1399,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193912,"statfs":{"total":200000000,"available":198566748,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Oct 26 01:38:46 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.79700000000000004,"15min":0.91100000000000003},"min":{"1min":0.47799999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.159,"5min":1.5049999999999999,"15min":19.425000000000001},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.93999999999999995,"15min":1.0369999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.462,"5min":1.462,"15min":15.162000000000001},"last":0.56200000000000006}]},{"osd":2,"last update":"Thu Oct 26 01:38:46 2023","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":1.0149999999999999,"15min":1.175},"min":{"1min":0.64300000000000002,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.47,"5min":1.577,"15min":28.439},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.90000000000000002,"15min":0.91300000000000003},"min":{"1min":0.57299999999999995,"5min":0.37,"15min":0.37},"max":{"1min":1.2969999999999999,"5min":1.4810000000000001,"15min":4.9459999999999997},"last":0.53900000000000003}]},{"osd":3,"last update":"Thu Oct 26 01:38:46 2023","interfaces":[{"interface":"back","average":{"1min":1.1180000000000001,"5min":1.0640000000000001,"15min":1.085},"min":{"1min":0.66800000000000004,"5min":0.46000000000000002,"15min":0.44600000000000001},"max":{"1min":1.6539999999999999,"5min":1.6539999999999999,"15min":4.9800000000000004},"last":0.66500000000000004},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.123,"15min":1.1499999999999999},"min":{"1min":0.66000000000000003,"5min":0.438,"15min":0.438},"max":{"1min":1.4279999999999999,"5min":1.6559999999999999,"15min":5.016},"last":0.873}]},{"osd":4,"last update":"Thu Oct 26 01:38:46 2023","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":1.169,"15min":1.2789999999999999},"min":{"1min":0.94499999999999995,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.391,"5min":1.681,"15min":19.329000000000001},"last":0.85699999999999998},{"interface":"front","average":{"1min":1.23,"5min":1.262,"15min":1.292},"min":{"1min":0.86799999999999999,"5min":0.82099999999999995,"15min":0.67600000000000005},"max":{"1min":1.5149999999999999,"5min":1.718,"15min":5.5229999999999997},"last":0.98999999999999999}]},{"osd":5,"last update":"Thu Oct 26 01:38:46 2023","interfaces":[{"interface":"back","average":{"1min":1.427,"5min":1.4019999999999999,"15min":1.4279999999999999},"min":{"1min":1.03,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.819,"5min":1.857,"15min":5.6360000000000001},"last":1.226},{"interface":"front","average":{"1min":1.2909999999999999,"5min":1.2749999999999999,"15min":1.395},"min":{"1min":0.80200000000000005,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.5880000000000001,"5min":1.8200000000000001,"15min":19.384},"last":0.94199999999999995}]},{"osd":6,"last update":"Thu Oct 26 01:38:46 2023","interfaces":[{"interface":"back","average":{"1min":1.306,"5min":1.296,"15min":1.3600000000000001},"min":{"1min":0.79300000000000004,"5min":0.76700000000000002,"15min":0.73799999999999999},"max":{"1min":1.736,"5min":1.736,"15min":8.2609999999999992},"last":0.97399999999999998},{"interface":"front","average":{"1min":1.3,"5min":1.335,"15min":1.3999999999999999},"min":{"1min":0.82999999999999996,"5min":0.77900000000000003,"15min":0.75900000000000001},"max":{"1min":1.581,"5min":1.9450000000000001,"15min":11.849},"last":1.0780000000000001}]},{"osd":7,"last update":"Thu Oct 26 01:38:46 2023","interfaces":[{"interface":"back","average":{"1min":1.46,"5min":1.4319999999999999,"15min":1.4770000000000001},"min":{"1min":1.089,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.9239999999999999,"5min":1.9239999999999999,"15min":5.6689999999999996},"last":1.117},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.397,"15min":1.4930000000000001},"min":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.69399999999999995},"max":{"1min":1.661,"5min":1.9390000000000001,"15min":19.245999999999999},"last":0.81299999999999994}]}]}],"pool_statfs":[]}} 2023-10-26T01:39:36.289 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-26T01:39:36.289 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-26T01:39:36.563 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:36.560+0000 7fba7bfff640 1 -- 172.21.15.169:0/3366307171 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fba7c0b4b10 msgr2=0x7fba7c0d0490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:36.564 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:36.560+0000 7fba7bfff640 1 --2- 172.21.15.169:0/3366307171 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fba7c0b4b10 0x7fba7c0d0490 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7fba7c04e000 tx=0x7fba7402bd10 comp rx=0 tx=0).stop 2023-10-26T01:39:36.564 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:36.560+0000 7fba7bfff640 1 -- 172.21.15.169:0/3366307171 shutdown_connections 2023-10-26T01:39:36.564 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:36.560+0000 7fba7bfff640 1 --2- 172.21.15.169:0/3366307171 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fba7c0b4b10 0x7fba7c0d0490 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:36.564 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:36.560+0000 7fba7bfff640 1 --2- 172.21.15.169:0/3366307171 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fba7c0cef30 0x7fba7c0b45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:36.564 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:36.560+0000 7fba7bfff640 1 --2- 172.21.15.169:0/3366307171 >> v2:172.21.15.170:3300/0 conn(0x7fba7c0b68d0 0x7fba7c065ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:36.565 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:36.560+0000 7fba7bfff640 1 -- 172.21.15.169:0/3366307171 >> 172.21.15.169:0/3366307171 conn(0x7fba7c0d1360 msgr2=0x7fba7c0d1770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:36.565 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:36.560+0000 7fba7bfff640 1 -- 172.21.15.169:0/3366307171 shutdown_connections 2023-10-26T01:39:36.565 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:36.560+0000 7fba7bfff640 1 -- 172.21.15.169:0/3366307171 wait complete. 2023-10-26T01:39:36.567 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:36.562+0000 7fba7bfff640 1 Processor -- start 2023-10-26T01:39:36.567 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:36.562+0000 7fba7bfff640 1 -- start start 2023-10-26T01:39:36.568 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:36.563+0000 7fba7bfff640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fba7c0b4b10 0x7fba7c15c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:36.568 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:36.563+0000 7fba7bfff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fba7c0b68d0 0x7fba7c15d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:36.568 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:36.563+0000 7fba7bfff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fba7c0cef30 0x7fba7c15dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:36.568 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:36.563+0000 7fba7bfff640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba7c1e5a50 con 0x7fba7c0cef30 2023-10-26T01:39:36.569 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:36.563+0000 7fba7bfff640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba7c1e4e40 con 0x7fba7c0b68d0 2023-10-26T01:39:36.569 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:36.563+0000 7fba7bfff640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba7c1e5120 con 0x7fba7c0b4b10 2023-10-26T01:39:36.569 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:36.563+0000 7fba7b7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fba7c0cef30 0x7fba7c15dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:36.570 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:36.563+0000 7fba7b7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fba7c0cef30 0x7fba7c15dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:33330/0 (socket says 172.21.15.169:33330) 2023-10-26T01:39:36.570 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:36.563+0000 7fba7b7fe640 1 -- 172.21.15.169:0/3137947483 learned_addr learned my addr 172.21.15.169:0/3137947483 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:39:36.571 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:36.564+0000 7fba7affd640 1 --2- 172.21.15.169:0/3137947483 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fba7c0b4b10 0x7fba7c15c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:36.571 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:36.564+0000 7fba7a7fc640 1 --2- 172.21.15.169:0/3137947483 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fba7c0b68d0 0x7fba7c15d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:36.571 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:36.564+0000 7fba7b7fe640 1 -- 172.21.15.169:0/3137947483 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fba7c0b4b10 msgr2=0x7fba7c15c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:36.572 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:36.564+0000 7fba7b7fe640 1 --2- 172.21.15.169:0/3137947483 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fba7c0b4b10 0x7fba7c15c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:36.572 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:36.564+0000 7fba7b7fe640 1 -- 172.21.15.169:0/3137947483 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fba7c0b68d0 msgr2=0x7fba7c15d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:36.572 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:36.564+0000 7fba7b7fe640 1 --2- 172.21.15.169:0/3137947483 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fba7c0b68d0 0x7fba7c15d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:36.572 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:36.564+0000 7fba7b7fe640 1 -- 172.21.15.169:0/3137947483 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba74009560 con 0x7fba7c0cef30 2023-10-26T01:39:36.573 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:36.564+0000 7fba7b7fe640 1 --2- 172.21.15.169:0/3137947483 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fba7c0cef30 0x7fba7c15dbb0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7fba74036040 tx=0x7fba7402da40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:36.573 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:36.564+0000 7fba737fe640 1 -- 172.21.15.169:0/3137947483 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba74003800 con 0x7fba7c0cef30 2023-10-26T01:39:36.573 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:36.565+0000 7fba737fe640 1 -- 172.21.15.169:0/3137947483 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fba74003a50 con 0x7fba7c0cef30 2023-10-26T01:39:36.574 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:36.565+0000 7fba737fe640 1 -- 172.21.15.169:0/3137947483 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba74003ea0 con 0x7fba7c0cef30 2023-10-26T01:39:36.574 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:36.565+0000 7fba7bfff640 1 -- 172.21.15.169:0/3137947483 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba7c1e5400 con 0x7fba7c0cef30 2023-10-26T01:39:36.574 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:36.565+0000 7fba7bfff640 1 -- 172.21.15.169:0/3137947483 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba7c15cdd0 con 0x7fba7c0cef30 2023-10-26T01:39:36.577 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:36.566+0000 7fba7bfff640 1 -- 172.21.15.169:0/3137947483 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba7c1e5bc0 con 0x7fba7c0cef30 2023-10-26T01:39:36.577 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:36.567+0000 7fba737fe640 1 -- 172.21.15.169:0/3137947483 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fba7403e450 con 0x7fba7c0cef30 2023-10-26T01:39:36.578 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:36.568+0000 7fba737fe640 1 --2- 172.21.15.169:0/3137947483 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fba5c06ae40 0x7fba5c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:36.578 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:36.568+0000 7fba737fe640 1 -- 172.21.15.169:0/3137947483 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fba740ac150 con 0x7fba7c0cef30 2023-10-26T01:39:36.578 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:36.569+0000 7fba7affd640 1 --2- 172.21.15.169:0/3137947483 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fba5c06ae40 0x7fba5c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:36.579 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:36.570+0000 7fba7affd640 1 --2- 172.21.15.169:0/3137947483 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fba5c06ae40 0x7fba5c06d300 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fba7c1f6320 tx=0x7fba6c0093c0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:36.579 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:36.573+0000 7fba737fe640 1 -- 172.21.15.169:0/3137947483 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fba74077310 con 0x7fba7c0cef30 2023-10-26T01:39:36.803 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T01:39:36.812 DEBUG:teuthology.orchestra.run.smithi170:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T01:39:36.829 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:36.826+0000 7fba7bfff640 1 -- 172.21.15.169:0/3137947483 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fba7c073c10 con 0x7fba7c0cef30 2023-10-26T01:39:36.832 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:36.830+0000 7fba737fe640 1 -- 172.21.15.169:0/3137947483 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v19) v1 ==== 190+0+147 (secure 0 0 0) 0x7fba7407fe60 con 0x7fba7c0cef30 2023-10-26T01:39:36.833 INFO:teuthology.orchestra.run.smithi169.stdout:[client.0] 2023-10-26T01:39:36.833 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQAgwTllKC7gOhAAPsvxGcuUA8/p8xqbUgGteA== 2023-10-26T01:39:36.833 INFO:teuthology.orchestra.run.smithi169.stdout: caps mds = "allow" 2023-10-26T01:39:36.833 INFO:teuthology.orchestra.run.smithi169.stdout: caps mon = "allow r" 2023-10-26T01:39:36.834 INFO:teuthology.orchestra.run.smithi169.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-10-26T01:39:36.834 INFO:teuthology.orchestra.run.smithi169.stderr:updated caps for client.0 2023-10-26T01:39:36.836 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:36.833+0000 7fba7bfff640 1 -- 172.21.15.169:0/3137947483 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fba5c06ae40 msgr2=0x7fba5c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:36.836 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:36.833+0000 7fba7bfff640 1 --2- 172.21.15.169:0/3137947483 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fba5c06ae40 0x7fba5c06d300 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fba7c1f6320 tx=0x7fba6c0093c0 comp rx=0 tx=0).stop 2023-10-26T01:39:36.836 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:36.833+0000 7fba7bfff640 1 -- 172.21.15.169:0/3137947483 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fba7c0cef30 msgr2=0x7fba7c15dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:36.837 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:36.833+0000 7fba7bfff640 1 --2- 172.21.15.169:0/3137947483 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fba7c0cef30 0x7fba7c15dbb0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7fba74036040 tx=0x7fba7402da40 comp rx=0 tx=0).stop 2023-10-26T01:39:36.837 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:36.833+0000 7fba7bfff640 1 -- 172.21.15.169:0/3137947483 shutdown_connections 2023-10-26T01:39:36.837 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:36.833+0000 7fba7bfff640 1 --2- 172.21.15.169:0/3137947483 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fba5c06ae40 0x7fba5c06d300 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:36.837 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:36.833+0000 7fba7bfff640 1 --2- 172.21.15.169:0/3137947483 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fba7c0cef30 0x7fba7c15dbb0 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:36.837 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:36.833+0000 7fba7bfff640 1 --2- 172.21.15.169:0/3137947483 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fba7c0b68d0 0x7fba7c15d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:36.838 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:36.833+0000 7fba7bfff640 1 --2- 172.21.15.169:0/3137947483 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fba7c0b4b10 0x7fba7c15c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:36.838 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:36.833+0000 7fba7bfff640 1 -- 172.21.15.169:0/3137947483 >> 172.21.15.169:0/3137947483 conn(0x7fba7c0d1360 msgr2=0x7fba7c065490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:36.838 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:36.833+0000 7fba7bfff640 1 -- 172.21.15.169:0/3137947483 shutdown_connections 2023-10-26T01:39:36.838 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:36.833+0000 7fba7bfff640 1 -- 172.21.15.169:0/3137947483 wait complete. 2023-10-26T01:39:36.853 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T01:39:37.093 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.090+0000 7fd5e2c26640 1 -- 172.21.15.169:0/178191228 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5c8005e10 msgr2=0x7fd5c8006290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:37.093 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.090+0000 7fd5e2c26640 1 --2- 172.21.15.169:0/178191228 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5c8005e10 0x7fd5c8006290 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7fd5d400b6b0 tx=0x7fd5c8006900 comp rx=0 tx=0).stop 2023-10-26T01:39:37.093 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.090+0000 7fd5e11c3640 1 -- 172.21.15.169:0/178191228 reap_dead start 2023-10-26T01:39:37.094 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.090+0000 7fd5e2c26640 1 -- 172.21.15.169:0/178191228 shutdown_connections 2023-10-26T01:39:37.094 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.090+0000 7fd5e2c26640 1 -- 172.21.15.169:0/178191228 >> 172.21.15.169:0/178191228 conn(0x7fd5dc069930 msgr2=0x7fd5dc069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:37.094 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.090+0000 7fd5e2c26640 1 -- 172.21.15.169:0/178191228 shutdown_connections 2023-10-26T01:39:37.094 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.090+0000 7fd5e2c26640 1 -- 172.21.15.169:0/178191228 wait complete. 2023-10-26T01:39:37.095 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.092+0000 7fd5e2c26640 1 Processor -- start 2023-10-26T01:39:37.095 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.092+0000 7fd5e2c26640 1 -- start start 2023-10-26T01:39:37.096 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.093+0000 7fd5e2c26640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5c8005e10 0x7fd5dc13f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:37.096 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.093+0000 7fd5e2c26640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd5dc1422a0 0x7fd5dc14abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:37.096 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.093+0000 7fd5e2c26640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd5dc14b0f0 0x7fd5dc14b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:37.096 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.093+0000 7fd5e2c26640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5dc14bab0 con 0x7fd5c8005e10 2023-10-26T01:39:37.096 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.093+0000 7fd5e2c26640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5dc14df40 con 0x7fd5dc1422a0 2023-10-26T01:39:37.096 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.093+0000 7fd5e2c26640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5dc14d7d0 con 0x7fd5dc14b0f0 2023-10-26T01:39:37.098 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.093+0000 7fd5e09c2640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5c8005e10 0x7fd5dc13f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:37.098 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.093+0000 7fd5e09c2640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5c8005e10 0x7fd5dc13f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:33344/0 (socket says 172.21.15.169:33344) 2023-10-26T01:39:37.098 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.093+0000 7fd5e09c2640 1 -- 172.21.15.169:0/1051543952 learned_addr learned my addr 172.21.15.169:0/1051543952 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:39:37.099 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.093+0000 7fd5e09c2640 1 -- 172.21.15.169:0/1051543952 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd5dc14b0f0 msgr2=0x7fd5dc14b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:37.099 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.093+0000 7fd5e09c2640 1 --2- 172.21.15.169:0/1051543952 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd5dc14b0f0 0x7fd5dc14b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:37.099 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.093+0000 7fd5e09c2640 1 -- 172.21.15.169:0/1051543952 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd5dc1422a0 msgr2=0x7fd5dc14abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:37.099 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.093+0000 7fd5e09c2640 1 --2- 172.21.15.169:0/1051543952 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd5dc1422a0 0x7fd5dc14abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:37.099 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.093+0000 7fd5e09c2640 1 -- 172.21.15.169:0/1051543952 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5dc14dab0 con 0x7fd5c8005e10 2023-10-26T01:39:37.100 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.094+0000 7fd5e09c2640 1 --2- 172.21.15.169:0/1051543952 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5c8005e10 0x7fd5dc13f470 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc001b20 tx=0x7fd5cc00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:37.100 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.094+0000 7fd5d97fa640 1 -- 172.21.15.169:0/1051543952 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5cc00cc60 con 0x7fd5c8005e10 2023-10-26T01:39:37.100 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.094+0000 7fd5d97fa640 1 -- 172.21.15.169:0/1051543952 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd5cc011460 con 0x7fd5c8005e10 2023-10-26T01:39:37.100 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.094+0000 7fd5d97fa640 1 -- 172.21.15.169:0/1051543952 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5cc011870 con 0x7fd5c8005e10 2023-10-26T01:39:37.100 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.094+0000 7fd5e2c26640 1 -- 172.21.15.169:0/1051543952 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5dc0718e0 con 0x7fd5c8005e10 2023-10-26T01:39:37.100 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.094+0000 7fd5e2c26640 1 -- 172.21.15.169:0/1051543952 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5dc142720 con 0x7fd5c8005e10 2023-10-26T01:39:37.101 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.095+0000 7fd5e2c26640 1 -- 172.21.15.169:0/1051543952 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5dc071e00 con 0x7fd5c8005e10 2023-10-26T01:39:37.101 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.096+0000 7fd5d97fa640 1 -- 172.21.15.169:0/1051543952 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fd5cc0119d0 con 0x7fd5c8005e10 2023-10-26T01:39:37.102 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.097+0000 7fd5d97fa640 1 --2- 172.21.15.169:0/1051543952 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd5bc06ae40 0x7fd5bc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:37.102 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.097+0000 7fd5d97fa640 1 -- 172.21.15.169:0/1051543952 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd5cc08a210 con 0x7fd5c8005e10 2023-10-26T01:39:37.102 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.097+0000 7fd5dbfff640 1 --2- 172.21.15.169:0/1051543952 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd5bc06ae40 0x7fd5bc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:37.102 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.099+0000 7fd5dbfff640 1 --2- 172.21.15.169:0/1051543952 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd5bc06ae40 0x7fd5bc06d300 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc1f1270 tx=0x7fd5c4006a00 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:37.104 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.101+0000 7fd5d97fa640 1 -- 172.21.15.169:0/1051543952 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd5cc055220 con 0x7fd5c8005e10 2023-10-26T01:39:37.311 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.309+0000 7fd5e2c26640 1 -- 172.21.15.169:0/1051543952 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd5dc04db10 con 0x7fd5c8005e10 2023-10-26T01:39:37.312 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.310+0000 7fd5d97fa640 1 -- 172.21.15.169:0/1051543952 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+4921 (secure 0 0 0) 0x7fd5cc058840 con 0x7fd5c8005e10 2023-10-26T01:39:37.313 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:39:37.313 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":77,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5711,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/3037106607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":3037106607},{"type":"v1","addr":"172.21.15.170:6836","nonce":3037106607}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72},{"gid":5714,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/1847866184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":1847866184},{"type":"v1","addr":"172.21.15.170:6837","nonce":1847866184}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72},{"gid":8229,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2707290348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2707290348},{"type":"v1","addr":"172.21.15.169:6833","nonce":2707290348}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T01:39:25.790062+0000","modified":"2023-10-26T01:39:28.841639+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8232},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8232":{"gid":8232,"name":"c","rank":0,"incarnation":73,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6835/66540264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":66540264},{"type":"v1","addr":"172.21.15.169:6835","nonce":66540264}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-10-26T01:39:37.313 INFO:teuthology.orchestra.run.smithi169.stderr:dumped fsmap epoch 77 2023-10-26T01:39:37.315 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.312+0000 7fd5e2c26640 1 -- 172.21.15.169:0/1051543952 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd5bc06ae40 msgr2=0x7fd5bc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:37.315 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.312+0000 7fd5e2c26640 1 --2- 172.21.15.169:0/1051543952 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd5bc06ae40 0x7fd5bc06d300 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc1f1270 tx=0x7fd5c4006a00 comp rx=0 tx=0).stop 2023-10-26T01:39:37.315 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.312+0000 7fd5e2c26640 1 -- 172.21.15.169:0/1051543952 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5c8005e10 msgr2=0x7fd5dc13f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:37.316 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.312+0000 7fd5e2c26640 1 --2- 172.21.15.169:0/1051543952 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5c8005e10 0x7fd5dc13f470 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc001b20 tx=0x7fd5cc00ee20 comp rx=0 tx=0).stop 2023-10-26T01:39:37.316 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.312+0000 7fd5e2c26640 1 -- 172.21.15.169:0/1051543952 shutdown_connections 2023-10-26T01:39:37.316 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.312+0000 7fd5e2c26640 1 --2- 172.21.15.169:0/1051543952 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd5bc06ae40 0x7fd5bc06d300 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:37.316 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.312+0000 7fd5e2c26640 1 --2- 172.21.15.169:0/1051543952 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd5dc14b0f0 0x7fd5dc14b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:37.316 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.312+0000 7fd5e2c26640 1 --2- 172.21.15.169:0/1051543952 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd5dc1422a0 0x7fd5dc14abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:37.316 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.312+0000 7fd5e2c26640 1 --2- 172.21.15.169:0/1051543952 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5c8005e10 0x7fd5dc13f470 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:37.317 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.312+0000 7fd5e2c26640 1 -- 172.21.15.169:0/1051543952 >> 172.21.15.169:0/1051543952 conn(0x7fd5dc069930 msgr2=0x7fd5dc0aab40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:37.317 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.312+0000 7fd5e2c26640 1 -- 172.21.15.169:0/1051543952 shutdown_connections 2023-10-26T01:39:37.317 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.312+0000 7fd5e2c26640 1 -- 172.21.15.169:0/1051543952 wait complete. 2023-10-26T01:39:37.736 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 77, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-26T01:39:25.790062+0000', 'modified': '2023-10-26T01:39:28.841639+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8232}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8232': {'gid': 8232, 'name': 'c', 'rank': 0, 'incarnation': 73, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.169:6835/66540264', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.169:6834', 'nonce': 66540264}, {'type': 'v1', 'addr': '172.21.15.169:6835', 'nonce': 66540264}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [19], 'metadata_pool': 18, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-26T01:39:37.736 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-26T01:39:37.737 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-26T01:39:37.964 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.961+0000 7f643d700640 1 -- 172.21.15.169:0/1562879354 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6420004d00 msgr2=0x7f6420005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:37.964 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.961+0000 7f643d700640 1 --2- 172.21.15.169:0/1562879354 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6420004d00 0x7f6420005180 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f6430009f00 tx=0x7f64200057f0 comp rx=0 tx=0).stop 2023-10-26T01:39:37.964 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.961+0000 7f64377fe640 1 -- 172.21.15.169:0/1562879354 reap_dead start 2023-10-26T01:39:37.965 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.961+0000 7f643d700640 1 -- 172.21.15.169:0/1562879354 shutdown_connections 2023-10-26T01:39:37.965 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.961+0000 7f643d700640 1 -- 172.21.15.169:0/1562879354 >> 172.21.15.169:0/1562879354 conn(0x7f6438069930 msgr2=0x7f6438069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:37.965 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.961+0000 7f643d700640 1 -- 172.21.15.169:0/1562879354 shutdown_connections 2023-10-26T01:39:37.965 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.961+0000 7f643d700640 1 -- 172.21.15.169:0/1562879354 wait complete. 2023-10-26T01:39:37.966 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.963+0000 7f643d700640 1 Processor -- start 2023-10-26T01:39:37.966 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.963+0000 7f643d700640 1 -- start start 2023-10-26T01:39:37.968 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.964+0000 7f643d700640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6420004d00 0x7f643814b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:37.968 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.965+0000 7f643d700640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f643806ed30 0x7f643806f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:37.968 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.965+0000 7f643d700640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6438158150 0x7f6438158600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:37.969 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.965+0000 7f643d700640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64380c93e0 con 0x7f6438158150 2023-10-26T01:39:37.969 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.965+0000 7f643d700640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64381570b0 con 0x7f643806ed30 2023-10-26T01:39:37.969 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.965+0000 7f643d700640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6438157390 con 0x7f6420004d00 2023-10-26T01:39:37.969 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.965+0000 7f64377fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6438158150 0x7f6438158600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:37.969 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.965+0000 7f64377fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6438158150 0x7f6438158600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:33376/0 (socket says 172.21.15.169:33376) 2023-10-26T01:39:37.970 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.965+0000 7f64377fe640 1 -- 172.21.15.169:0/2690068761 learned_addr learned my addr 172.21.15.169:0/2690068761 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:39:37.970 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.965+0000 7f64367fc640 1 --2- 172.21.15.169:0/2690068761 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f643806ed30 0x7f643806f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:37.970 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.965+0000 7f6436ffd640 1 --2- 172.21.15.169:0/2690068761 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6420004d00 0x7f643814b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:37.970 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.965+0000 7f64377fe640 1 -- 172.21.15.169:0/2690068761 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6420004d00 msgr2=0x7f643814b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:37.970 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.965+0000 7f64377fe640 1 --2- 172.21.15.169:0/2690068761 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6420004d00 0x7f643814b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:37.971 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.965+0000 7f64377fe640 1 -- 172.21.15.169:0/2690068761 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f643806ed30 msgr2=0x7f643806f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:37.971 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.965+0000 7f64377fe640 1 --2- 172.21.15.169:0/2690068761 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f643806ed30 0x7f643806f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:37.971 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.965+0000 7f64377fe640 1 -- 172.21.15.169:0/2690068761 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6438157670 con 0x7f6438158150 2023-10-26T01:39:37.971 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.966+0000 7f64377fe640 1 --2- 172.21.15.169:0/2690068761 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6438158150 0x7f6438158600 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f6430009f00 tx=0x7f6430004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:37.972 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.966+0000 7f641f7fe640 1 -- 172.21.15.169:0/2690068761 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6430003970 con 0x7f6438158150 2023-10-26T01:39:37.972 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.966+0000 7f641f7fe640 1 -- 172.21.15.169:0/2690068761 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6430003bc0 con 0x7f6438158150 2023-10-26T01:39:37.973 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.966+0000 7f643d700640 1 -- 172.21.15.169:0/2690068761 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6438157950 con 0x7f6438158150 2023-10-26T01:39:37.973 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.966+0000 7f641f7fe640 1 -- 172.21.15.169:0/2690068761 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6430004030 con 0x7f6438158150 2023-10-26T01:39:37.973 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.966+0000 7f643d700640 1 -- 172.21.15.169:0/2690068761 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f643814bc40 con 0x7f6438158150 2023-10-26T01:39:37.974 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.968+0000 7f641f7fe640 1 -- 172.21.15.169:0/2690068761 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f6430004190 con 0x7f6438158150 2023-10-26T01:39:37.983 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.970+0000 7f641f7fe640 1 --2- 172.21.15.169:0/2690068761 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f642406ae70 0x7f642406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:37.984 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.970+0000 7f641f7fe640 1 -- 172.21.15.169:0/2690068761 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f64300abc60 con 0x7f6438158150 2023-10-26T01:39:37.984 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.971+0000 7f6436ffd640 1 --2- 172.21.15.169:0/2690068761 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f642406ae70 0x7f642406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:37.984 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.971+0000 7f644036f740 1 Processor -- start 2023-10-26T01:39:37.985 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.971+0000 7f644036f740 1 -- start start 2023-10-26T01:39:37.985 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.972+0000 7f644036f740 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x55957da6e7d0 0x55957da41a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:37.985 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.972+0000 7f644036f740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55957da6ebb0 0x55957da71560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:37.986 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.972+0000 7f644036f740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55957da41f40 0x55957da71aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:37.986 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.972+0000 7f644036f740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55957d9b4740 con 0x55957da41f40 2023-10-26T01:39:37.986 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.972+0000 7f644036f740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55957d9b1e30 con 0x55957da6ebb0 2023-10-26T01:39:37.987 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.972+0000 7f644036f740 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x55957d9ad720 con 0x55957da6e7d0 2023-10-26T01:39:37.987 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.972+0000 7f6436ffd640 1 --2- 172.21.15.169:0/2690068761 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f642406ae70 0x7f642406d330 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f6428009a80 tx=0x7f6428005740 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:37.987 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.972+0000 7f64377fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55957da41f40 0x55957da71aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:37.987 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.972+0000 7f64377fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55957da41f40 0x55957da71aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:33378/0 (socket says 172.21.15.169:33378) 2023-10-26T01:39:37.988 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.972+0000 7f64377fe640 1 -- 172.21.15.169:0/4088618319 learned_addr learned my addr 172.21.15.169:0/4088618319 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:39:37.988 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.972+0000 7f6436ffd640 1 --2- 172.21.15.169:0/4088618319 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x55957da6e7d0 0x55957da41a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:37.988 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.972+0000 7f64367fc640 1 --2- 172.21.15.169:0/4088618319 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55957da6ebb0 0x55957da71560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:37.988 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.972+0000 7f64377fe640 1 -- 172.21.15.169:0/4088618319 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x55957da6e7d0 msgr2=0x55957da41a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T01:39:37.989 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.972+0000 7f64377fe640 1 --2- 172.21.15.169:0/4088618319 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x55957da6e7d0 0x55957da41a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:37.989 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.972+0000 7f64377fe640 1 -- 172.21.15.169:0/4088618319 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55957da6ebb0 msgr2=0x55957da71560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T01:39:37.989 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.972+0000 7f64377fe640 1 --2- 172.21.15.169:0/4088618319 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55957da6ebb0 0x55957da71560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:37.990 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.972+0000 7f64377fe640 1 -- 172.21.15.169:0/4088618319 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55957d971910 con 0x55957da41f40 2023-10-26T01:39:37.990 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.973+0000 7f64377fe640 1 --2- 172.21.15.169:0/4088618319 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55957da41f40 0x55957da71aa0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f6430005fd0 tx=0x7f643007a350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80d6b68b1d7c34e1 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:37.990 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.973+0000 7f641cff9640 1 -- 172.21.15.169:0/4088618319 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f643007ea80 con 0x55957da41f40 2023-10-26T01:39:37.990 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.973+0000 7f641cff9640 1 -- 172.21.15.169:0/4088618319 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6430082c30 con 0x55957da41f40 2023-10-26T01:39:37.991 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.973+0000 7f641cff9640 1 -- 172.21.15.169:0/4088618319 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6430009ad0 con 0x55957da41f40 2023-10-26T01:39:37.991 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.973+0000 7f644036f740 1 -- 172.21.15.169:0/4088618319 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55957da41f40 msgr2=0x55957da71aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:37.991 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.973+0000 7f644036f740 1 --2- 172.21.15.169:0/4088618319 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55957da41f40 0x55957da71aa0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f6430005fd0 tx=0x7f643007a350 comp rx=0 tx=0).stop 2023-10-26T01:39:37.991 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.974+0000 7f644036f740 1 -- 172.21.15.169:0/4088618319 shutdown_connections 2023-10-26T01:39:37.992 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.974+0000 7f644036f740 1 --2- 172.21.15.169:0/4088618319 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55957da41f40 0x55957da71aa0 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:37.992 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.974+0000 7f644036f740 1 --2- 172.21.15.169:0/4088618319 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55957da6ebb0 0x55957da71560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:37.992 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.974+0000 7f644036f740 1 --2- 172.21.15.169:0/4088618319 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x55957da6e7d0 0x55957da41a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:37.993 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.974+0000 7f644036f740 1 -- 172.21.15.169:0/4088618319 >> 172.21.15.169:0/4088618319 conn(0x55957da43c70 msgr2=0x55957da8c560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:37.993 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.974+0000 7f644036f740 1 -- 172.21.15.169:0/4088618319 shutdown_connections 2023-10-26T01:39:37.993 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.974+0000 7f644036f740 1 -- 172.21.15.169:0/4088618319 wait complete. 2023-10-26T01:39:37.994 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.974+0000 7f644036f740 1 -- start start 2023-10-26T01:39:37.994 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.975+0000 7f644036f740 1 Processor -- start 2023-10-26T01:39:37.995 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.976+0000 7f644036f740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55957da6e7d0 0x55957da74610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:37.995 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.976+0000 7f644036f740 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x55957da6ebb0 0x55957da74b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:37.995 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.976+0000 7f644036f740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55957da69980 0x55957da9f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:37.996 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.976+0000 7f644036f740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55957d94b970 con 0x55957da6e7d0 2023-10-26T01:39:37.996 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.976+0000 7f644036f740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55957d94b610 con 0x55957da69980 2023-10-26T01:39:37.997 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.976+0000 7f644036f740 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x55957d711f20 con 0x55957da6ebb0 2023-10-26T01:39:37.997 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.976+0000 7f644036f740 10 client.0 _ll_register_callbacks cb 0x55957d96f0c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T01:39:37.997 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.976+0000 7f6436ffd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55957da6e7d0 0x55957da74610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:37.998 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.976+0000 7f6436ffd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55957da6e7d0 0x55957da74610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:33392/0 (socket says 172.21.15.169:33392) 2023-10-26T01:39:37.998 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.976+0000 7f6436ffd640 1 -- 172.21.15.169:0/4136977600 learned_addr learned my addr 172.21.15.169:0/4136977600 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:39:37.999 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.976+0000 7f64367fc640 1 --2- 172.21.15.169:0/4136977600 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x55957da6ebb0 0x55957da74b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:37.999 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.976+0000 7f6436ffd640 1 -- 172.21.15.169:0/4136977600 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x55957da6ebb0 msgr2=0x55957da74b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T01:39:37.999 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.976+0000 7f6436ffd640 1 --2- 172.21.15.169:0/4136977600 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x55957da6ebb0 0x55957da74b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:38.000 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.976+0000 7f6436ffd640 1 -- 172.21.15.169:0/4136977600 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55957da69980 msgr2=0x55957da9f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T01:39:38.000 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.976+0000 7f64377fe640 1 --2- 172.21.15.169:0/4136977600 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55957da69980 0x55957da9f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:38.000 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.976+0000 7f6436ffd640 1 --2- 172.21.15.169:0/4136977600 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55957da69980 0x55957da9f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:38.001 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.976+0000 7f6436ffd640 1 -- 172.21.15.169:0/4136977600 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f643814bc40 con 0x55957da6e7d0 2023-10-26T01:39:38.001 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.977+0000 7f6436ffd640 1 --2- 172.21.15.169:0/4136977600 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55957da6e7d0 0x55957da74610 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f6428010fb0 tx=0x7f6428005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40a200b0152f8bc5 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:38.001 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.977+0000 7f6415ffb640 10 client.0 ms_handle_connect on v2:172.21.15.169:3300/0 2023-10-26T01:39:38.002 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.977+0000 7f6415ffb640 1 -- 172.21.15.169:0/4136977600 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6428020840 con 0x55957da6e7d0 2023-10-26T01:39:38.002 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.977+0000 7f6415ffb640 1 -- 172.21.15.169:0/4136977600 <== mon.0 v2:172.21.15.169:3300/0 2 ==== osd_map(151..151 src has 1..151) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f64280096d0 con 0x55957da6e7d0 2023-10-26T01:39:38.002 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.977+0000 7f644036f740 1 -- 172.21.15.169:0/4136977600 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55957d9a6700 con 0x55957da6e7d0 2023-10-26T01:39:38.003 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.977+0000 7f6415ffb640 1 -- 172.21.15.169:0/4136977600 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) v2 ==== 24+0+0 (secure 0 0 0) 0x7f64280209a0 con 0x55957da6e7d0 2023-10-26T01:39:38.003 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.977+0000 7f644036f740 10 client.8406 fetch_fsmap learned FSMap version 77 2023-10-26T01:39:38.003 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.977+0000 7f644036f740 1 -- 172.21.15.169:0/4136977600 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=2+}) v3 -- 0x55957d9abda0 con 0x55957da6e7d0 2023-10-26T01:39:38.004 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.978+0000 7f6415ffb640 1 -- 172.21.15.169:0/4136977600 <== mon.0 v2:172.21.15.169:3300/0 4 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x55957d9a6700 con 0x55957da6e7d0 2023-10-26T01:39:38.004 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.978+0000 7f6415ffb640 1 -- 172.21.15.169:0/4136977600 <== mon.0 v2:172.21.15.169:3300/0 5 ==== fsmap(e 77) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f6428026020 con 0x55957da6e7d0 2023-10-26T01:39:38.005 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.978+0000 7f644036f740 10 client.8406 fetch_fsmap finished waiting for FSMap version 77 2023-10-26T01:39:38.005 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.978+0000 7f644036f740 10 client.8406 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-26T01:39:38.005 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.978+0000 7f644036f740 20 client.8406 populate_metadata read hostname 'smithi169' 2023-10-26T01:39:38.006 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.978+0000 7f644036f740 1 --2- 172.21.15.169:0/4136977600 >> [v2:172.21.15.169:6834/66540264,v1:172.21.15.169:6835/66540264] conn(0x55957da6a970 0x55957da6ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:38.006 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.978+0000 7f644036f740 4 client.8406 mds_command: new command op to 8232 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-26T01:39:38.006 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.978+0000 7f644036f740 1 -- 172.21.15.169:0/4136977600 --> [v2:172.21.15.169:6834/66540264,v1:172.21.15.169:6835/66540264] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55957d9a6420 con 0x55957da6a970 2023-10-26T01:39:38.007 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.978+0000 7f64367fc640 1 --2- 172.21.15.169:0/4136977600 >> [v2:172.21.15.169:6834/66540264,v1:172.21.15.169:6835/66540264] conn(0x55957da6a970 0x55957da6ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:38.007 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.979+0000 7f64367fc640 1 --2- 172.21.15.169:0/4136977600 >> [v2:172.21.15.169:6834/66540264,v1:172.21.15.169:6835/66540264] conn(0x55957da6a970 0x55957da6ad70 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37887aa1e4351517 server_cookie=216df0b1ebf30d65 in_seq=0 out_seq=0 2023-10-26T01:39:38.007 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.979+0000 7f6415ffb640 10 client.8406 ms_handle_connect on v2:172.21.15.169:6834/66540264 2023-10-26T01:39:38.008 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.980+0000 7f6415ffb640 1 -- 172.21.15.169:0/4136977600 <== mds.0 v2:172.21.15.169:6834/66540264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55957d9a6420 con 0x55957da6a970 2023-10-26T01:39:38.008 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.980+0000 7f6415ffb640 10 client.8406 handle_command_reply: tid=0 2023-10-26T01:39:38.008 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.980+0000 7f644036f740 1 client.8406 shutdown 2023-10-26T01:39:38.009 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.980+0000 7f644036f740 1 -- 172.21.15.169:0/4136977600 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55957da6e7d0 msgr2=0x55957da74610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:38.009 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.980+0000 7f644036f740 1 --2- 172.21.15.169:0/4136977600 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55957da6e7d0 0x55957da74610 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f6428010fb0 tx=0x7f6428005f90 comp rx=0 tx=0).stop 2023-10-26T01:39:38.009 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.980+0000 7f644036f740 1 -- 172.21.15.169:0/4136977600 shutdown_connections 2023-10-26T01:39:38.010 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.980+0000 7f644036f740 1 --2- 172.21.15.169:0/4136977600 >> [v2:172.21.15.169:6834/66540264,v1:172.21.15.169:6835/66540264] conn(0x55957da6a970 0x55957da6ad70 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:38.010 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.980+0000 7f644036f740 1 --2- 172.21.15.169:0/4136977600 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55957da69980 0x55957da9f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:38.010 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.980+0000 7f644036f740 1 --2- 172.21.15.169:0/4136977600 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x55957da6ebb0 0x55957da74b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:38.011 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.980+0000 7f644036f740 1 --2- 172.21.15.169:0/4136977600 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55957da6e7d0 0x55957da74610 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:38.011 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.980+0000 7f6415ffb640 0 client.8406 ms_handle_reset on v2:172.21.15.169:6834/66540264 2023-10-26T01:39:38.011 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.980+0000 7f644036f740 1 -- 172.21.15.169:0/4136977600 >> 172.21.15.169:0/4136977600 conn(0x55957da41f40 msgr2=0x55957da45650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:38.012 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.980+0000 7f644036f740 1 -- 172.21.15.169:0/4136977600 shutdown_connections 2023-10-26T01:39:38.012 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.980+0000 7f644036f740 1 -- 172.21.15.169:0/4136977600 wait complete. 2023-10-26T01:39:38.013 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.980+0000 7f644036f740 20 client.8406 trim_cache size 0 max 16384 2023-10-26T01:39:38.018 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.996+0000 7f644036f740 1 Processor -- start 2023-10-26T01:39:38.018 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.996+0000 7f644036f740 1 -- start start 2023-10-26T01:39:38.018 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.997+0000 7f644036f740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55957da69980 0x55957da3d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:38.019 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.997+0000 7f644036f740 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x55957da6e7d0 0x55957da72a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:38.019 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.997+0000 7f644036f740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55957da6ebb0 0x55957da72f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:38.019 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.997+0000 7f644036f740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55957da71e80 con 0x55957da6ebb0 2023-10-26T01:39:38.019 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.997+0000 7f644036f740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55957da83da0 con 0x55957da69980 2023-10-26T01:39:38.020 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.997+0000 7f644036f740 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x55957da734a0 con 0x55957da6e7d0 2023-10-26T01:39:38.020 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.997+0000 7f64377fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55957da6ebb0 0x55957da72f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:38.020 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.997+0000 7f64377fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55957da6ebb0 0x55957da72f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:33400/0 (socket says 172.21.15.169:33400) 2023-10-26T01:39:38.020 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.997+0000 7f64377fe640 1 -- 172.21.15.169:0/2254807911 learned_addr learned my addr 172.21.15.169:0/2254807911 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:39:38.020 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.998+0000 7f64377fe640 1 -- 172.21.15.169:0/2254807911 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x55957da6e7d0 msgr2=0x55957da72a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T01:39:38.021 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.998+0000 7f64377fe640 1 --2- 172.21.15.169:0/2254807911 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x55957da6e7d0 0x55957da72a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:38.021 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.998+0000 7f64377fe640 1 -- 172.21.15.169:0/2254807911 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55957da69980 msgr2=0x55957da3d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T01:39:38.021 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.998+0000 7f64377fe640 1 --2- 172.21.15.169:0/2254807911 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55957da69980 0x55957da3d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:38.021 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.998+0000 7f64377fe640 1 -- 172.21.15.169:0/2254807911 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55957d94ab80 con 0x55957da6ebb0 2023-10-26T01:39:38.021 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.998+0000 7f64377fe640 1 --2- 172.21.15.169:0/2254807911 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55957da6ebb0 0x55957da72f60 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f643002cf30 tx=0x7f64300a9fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84a5d7ffb57a81f6 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:38.022 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.998+0000 7f641d7fa640 1 -- 172.21.15.169:0/2254807911 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f643007ea80 con 0x55957da6ebb0 2023-10-26T01:39:38.022 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.998+0000 7f641d7fa640 1 -- 172.21.15.169:0/2254807911 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6430009ad0 con 0x55957da6ebb0 2023-10-26T01:39:38.022 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.998+0000 7f641d7fa640 1 -- 172.21.15.169:0/2254807911 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6430092100 con 0x55957da6ebb0 2023-10-26T01:39:38.022 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.999+0000 7f644036f740 1 -- 172.21.15.169:0/2254807911 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55957da6ebb0 msgr2=0x55957da72f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:38.023 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.999+0000 7f644036f740 1 --2- 172.21.15.169:0/2254807911 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55957da6ebb0 0x55957da72f60 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f643002cf30 tx=0x7f64300a9fc0 comp rx=0 tx=0).stop 2023-10-26T01:39:38.023 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.999+0000 7f644036f740 1 -- 172.21.15.169:0/2254807911 shutdown_connections 2023-10-26T01:39:38.023 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.999+0000 7f644036f740 1 --2- 172.21.15.169:0/2254807911 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55957da6ebb0 0x55957da72f60 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:38.023 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.999+0000 7f644036f740 1 --2- 172.21.15.169:0/2254807911 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x55957da6e7d0 0x55957da72a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:38.023 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.999+0000 7f644036f740 1 --2- 172.21.15.169:0/2254807911 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55957da69980 0x55957da3d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:38.024 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.999+0000 7f644036f740 1 -- 172.21.15.169:0/2254807911 >> 172.21.15.169:0/2254807911 conn(0x55957da41f40 msgr2=0x55957da6c0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:38.024 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.999+0000 7f644036f740 1 -- 172.21.15.169:0/2254807911 shutdown_connections 2023-10-26T01:39:38.024 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.999+0000 7f644036f740 1 -- 172.21.15.169:0/2254807911 wait complete. 2023-10-26T01:39:38.024 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:37.999+0000 7f644036f740 1 -- start start 2023-10-26T01:39:38.024 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.000+0000 7f644036f740 1 Processor -- start 2023-10-26T01:39:38.025 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.001+0000 7f644036f740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55957da69980 0x55957daf3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:38.025 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.001+0000 7f644036f740 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x55957da6e7d0 0x55957da8d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:38.025 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.001+0000 7f644036f740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55957da6ebb0 0x55957da5cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:38.025 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.001+0000 7f644036f740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55957da5cad0 con 0x55957da6ebb0 2023-10-26T01:39:38.025 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.001+0000 7f644036f740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55957da8e0b0 con 0x55957da69980 2023-10-26T01:39:38.026 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.001+0000 7f644036f740 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x55957da8ebf0 con 0x55957da6e7d0 2023-10-26T01:39:38.026 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.001+0000 7f644036f740 10 client.0 _ll_register_callbacks cb 0x7f6430030f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T01:39:38.026 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.001+0000 7f64377fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55957da6ebb0 0x55957da5cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:38.026 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.001+0000 7f64377fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55957da6ebb0 0x55957da5cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:33406/0 (socket says 172.21.15.169:33406) 2023-10-26T01:39:38.026 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.001+0000 7f64377fe640 1 -- 172.21.15.169:0/1599843060 learned_addr learned my addr 172.21.15.169:0/1599843060 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:39:38.027 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.001+0000 7f64367fc640 1 --2- 172.21.15.169:0/1599843060 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x55957da6e7d0 0x55957da8d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:38.027 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.001+0000 7f64377fe640 1 -- 172.21.15.169:0/1599843060 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x55957da6e7d0 msgr2=0x55957da8d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T01:39:38.027 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.001+0000 7f64377fe640 1 --2- 172.21.15.169:0/1599843060 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x55957da6e7d0 0x55957da8d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:38.027 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.001+0000 7f64377fe640 1 -- 172.21.15.169:0/1599843060 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55957da69980 msgr2=0x55957daf3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T01:39:38.028 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.001+0000 7f6436ffd640 1 --2- 172.21.15.169:0/1599843060 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55957da69980 0x55957daf3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:38.028 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.001+0000 7f64377fe640 1 --2- 172.21.15.169:0/1599843060 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55957da69980 0x55957daf3d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:38.028 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.001+0000 7f64377fe640 1 -- 172.21.15.169:0/1599843060 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55957d971910 con 0x55957da6ebb0 2023-10-26T01:39:38.028 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.002+0000 7f64377fe640 1 --2- 172.21.15.169:0/1599843060 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55957da6ebb0 0x55957da5cef0 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f6430005fd0 tx=0x7f643007b170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ca0b216019621f8 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:38.028 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.002+0000 7f6415ffb640 10 client.0 ms_handle_connect on v2:172.21.15.169:3300/0 2023-10-26T01:39:38.029 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.002+0000 7f6415ffb640 1 -- 172.21.15.169:0/1599843060 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6430099310 con 0x55957da6ebb0 2023-10-26T01:39:38.029 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.002+0000 7f6415ffb640 1 -- 172.21.15.169:0/1599843060 <== mon.0 v2:172.21.15.169:3300/0 2 ==== osd_map(151..151 src has 1..151) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6430077040 con 0x55957da6ebb0 2023-10-26T01:39:38.029 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.002+0000 7f644036f740 1 -- 172.21.15.169:0/1599843060 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6430082c30 con 0x55957da6ebb0 2023-10-26T01:39:38.029 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.002+0000 7f6415ffb640 1 -- 172.21.15.169:0/1599843060 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6430077200 con 0x55957da6ebb0 2023-10-26T01:39:38.029 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.002+0000 7f644036f740 10 client.8412 fetch_fsmap learned FSMap version 77 2023-10-26T01:39:38.030 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.002+0000 7f644036f740 1 -- 172.21.15.169:0/1599843060 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=2+}) v3 -- 0x55957d970170 con 0x55957da6ebb0 2023-10-26T01:39:38.030 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.002+0000 7f6415ffb640 1 -- 172.21.15.169:0/1599843060 <== mon.0 v2:172.21.15.169:3300/0 4 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6430082c30 con 0x55957da6ebb0 2023-10-26T01:39:38.030 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.002+0000 7f6415ffb640 1 -- 172.21.15.169:0/1599843060 <== mon.0 v2:172.21.15.169:3300/0 5 ==== fsmap(e 77) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f6430099030 con 0x55957da6ebb0 2023-10-26T01:39:38.030 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.002+0000 7f644036f740 10 client.8412 fetch_fsmap finished waiting for FSMap version 77 2023-10-26T01:39:38.030 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.002+0000 7f644036f740 10 client.8412 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-26T01:39:38.031 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.002+0000 7f644036f740 20 client.8412 populate_metadata read hostname 'smithi169' 2023-10-26T01:39:38.031 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.002+0000 7f644036f740 1 --2- 172.21.15.169:0/1599843060 >> [v2:172.21.15.169:6834/66540264,v1:172.21.15.169:6835/66540264] conn(0x55957da6ad50 0x55957da6b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:38.031 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.002+0000 7f644036f740 4 client.8412 mds_command: new command op to 8232 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-26T01:39:38.031 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.002+0000 7f644036f740 1 -- 172.21.15.169:0/1599843060 --> [v2:172.21.15.169:6834/66540264,v1:172.21.15.169:6835/66540264] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55957d6af6c0 con 0x55957da6ad50 2023-10-26T01:39:38.032 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.003+0000 7f6436ffd640 1 --2- 172.21.15.169:0/1599843060 >> [v2:172.21.15.169:6834/66540264,v1:172.21.15.169:6835/66540264] conn(0x55957da6ad50 0x55957da6b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:38.032 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.003+0000 7f6436ffd640 1 --2- 172.21.15.169:0/1599843060 >> [v2:172.21.15.169:6834/66540264,v1:172.21.15.169:6835/66540264] conn(0x55957da6ad50 0x55957da6b130 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11de6e4b7cbcd84e server_cookie=cc4e22670a36161 in_seq=0 out_seq=0 2023-10-26T01:39:38.032 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.003+0000 7f6415ffb640 10 client.8412 ms_handle_connect on v2:172.21.15.169:6834/66540264 2023-10-26T01:39:38.032 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.003+0000 7f6415ffb640 1 -- 172.21.15.169:0/1599843060 <== mds.0 v2:172.21.15.169:6834/66540264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55957d6af6c0 con 0x55957da6ad50 2023-10-26T01:39:38.032 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.003+0000 7f6415ffb640 10 client.8412 handle_command_reply: tid=0 2023-10-26T01:39:38.033 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.003+0000 7f644036f740 1 client.8412 shutdown 2023-10-26T01:39:38.033 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.003+0000 7f644036f740 1 -- 172.21.15.169:0/1599843060 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55957da6ebb0 msgr2=0x55957da5cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:38.033 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.003+0000 7f644036f740 1 --2- 172.21.15.169:0/1599843060 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55957da6ebb0 0x55957da5cef0 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f6430005fd0 tx=0x7f643007b170 comp rx=0 tx=0).stop 2023-10-26T01:39:38.033 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.004+0000 7f644036f740 1 -- 172.21.15.169:0/1599843060 shutdown_connections 2023-10-26T01:39:38.033 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.004+0000 7f644036f740 1 --2- 172.21.15.169:0/1599843060 >> [v2:172.21.15.169:6834/66540264,v1:172.21.15.169:6835/66540264] conn(0x55957da6ad50 0x55957da6b130 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:38.034 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.004+0000 7f644036f740 1 --2- 172.21.15.169:0/1599843060 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55957da6ebb0 0x55957da5cef0 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:38.034 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.004+0000 7f644036f740 1 --2- 172.21.15.169:0/1599843060 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x55957da6e7d0 0x55957da8d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:38.034 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.004+0000 7f6415ffb640 0 client.8412 ms_handle_reset on v2:172.21.15.169:6834/66540264 2023-10-26T01:39:38.034 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.004+0000 7f644036f740 1 --2- 172.21.15.169:0/1599843060 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55957da69980 0x55957daf3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:38.034 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.004+0000 7f644036f740 1 -- 172.21.15.169:0/1599843060 >> 172.21.15.169:0/1599843060 conn(0x55957da41f40 msgr2=0x55957da6c070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:38.035 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.004+0000 7f644036f740 1 -- 172.21.15.169:0/1599843060 shutdown_connections 2023-10-26T01:39:38.035 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.004+0000 7f644036f740 1 -- 172.21.15.169:0/1599843060 wait complete. 2023-10-26T01:39:38.035 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.004+0000 7f644036f740 20 client.8412 trim_cache size 0 max 16384 2023-10-26T01:39:38.036 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2023-10-26T01:39:38.036 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_fsid": "30203b28-ed8a-44e8-a9a6-a9bf25f980f1", 2023-10-26T01:39:38.036 INFO:teuthology.orchestra.run.smithi169.stdout: "whoami": 0, 2023-10-26T01:39:38.036 INFO:teuthology.orchestra.run.smithi169.stdout: "id": 8232, 2023-10-26T01:39:38.037 INFO:teuthology.orchestra.run.smithi169.stdout: "want_state": "up:active", 2023-10-26T01:39:38.037 INFO:teuthology.orchestra.run.smithi169.stdout: "state": "up:active", 2023-10-26T01:39:38.037 INFO:teuthology.orchestra.run.smithi169.stdout: "fs_name": "cephfs", 2023-10-26T01:39:38.037 INFO:teuthology.orchestra.run.smithi169.stdout: "rank_uptime": 11.659371643, 2023-10-26T01:39:38.037 INFO:teuthology.orchestra.run.smithi169.stdout: "mdsmap_epoch": 77, 2023-10-26T01:39:38.038 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_epoch": 151, 2023-10-26T01:39:38.038 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_epoch_barrier": 151, 2023-10-26T01:39:38.038 INFO:teuthology.orchestra.run.smithi169.stdout: "uptime": 17.610946719000001 2023-10-26T01:39:38.038 INFO:teuthology.orchestra.run.smithi169.stdout:} 2023-10-26T01:39:38.040 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.004+0000 7f643d700640 1 -- 172.21.15.169:0/2690068761 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f642406ae70 msgr2=0x7f642406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:38.040 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.004+0000 7f643d700640 1 --2- 172.21.15.169:0/2690068761 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f642406ae70 0x7f642406d330 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f6428009a80 tx=0x7f6428005740 comp rx=0 tx=0).stop 2023-10-26T01:39:38.040 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.004+0000 7f643d700640 1 -- 172.21.15.169:0/2690068761 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6438158150 msgr2=0x7f6438158600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:38.040 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.004+0000 7f643d700640 1 --2- 172.21.15.169:0/2690068761 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6438158150 0x7f6438158600 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f6430009f00 tx=0x7f6430004ab0 comp rx=0 tx=0).stop 2023-10-26T01:39:38.041 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.004+0000 7f643d700640 1 -- 172.21.15.169:0/2690068761 shutdown_connections 2023-10-26T01:39:38.041 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.004+0000 7f643d700640 1 --2- 172.21.15.169:0/2690068761 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f642406ae70 0x7f642406d330 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:38.041 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.004+0000 7f643d700640 1 --2- 172.21.15.169:0/2690068761 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6438158150 0x7f6438158600 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:38.041 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.004+0000 7f643d700640 1 --2- 172.21.15.169:0/2690068761 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f643806ed30 0x7f643806f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:38.041 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.004+0000 7f643d700640 1 --2- 172.21.15.169:0/2690068761 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6420004d00 0x7f643814b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:38.042 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.004+0000 7f643d700640 1 -- 172.21.15.169:0/2690068761 >> 172.21.15.169:0/2690068761 conn(0x7f6438069930 msgr2=0x7f643815c540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:38.042 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.005+0000 7f643d700640 1 -- 172.21.15.169:0/2690068761 shutdown_connections 2023-10-26T01:39:38.042 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.005+0000 7f643d700640 1 -- 172.21.15.169:0/2690068761 wait complete. 2023-10-26T01:39:38.043 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-26T01:39:38.254 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.251+0000 7f0849007640 1 -- 172.21.15.169:0/3054129821 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0830003970 msgr2=0x7f0830003d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:38.254 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.251+0000 7f0849007640 1 --2- 172.21.15.169:0/3054129821 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0830003970 0x7f0830003d70 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f0834006fc0 tx=0x7f08300057f0 comp rx=0 tx=0).stop 2023-10-26T01:39:38.254 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.251+0000 7f0849007640 1 -- 172.21.15.169:0/3054129821 shutdown_connections 2023-10-26T01:39:38.255 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.251+0000 7f0849007640 1 --2- 172.21.15.169:0/3054129821 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0830003970 0x7f0830003d70 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:38.255 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.251+0000 7f0849007640 1 -- 172.21.15.169:0/3054129821 >> 172.21.15.169:0/3054129821 conn(0x7f0844069930 msgr2=0x7f0844069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:38.255 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.251+0000 7f0849007640 1 -- 172.21.15.169:0/3054129821 shutdown_connections 2023-10-26T01:39:38.255 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.251+0000 7f0849007640 1 -- 172.21.15.169:0/3054129821 wait complete. 2023-10-26T01:39:38.258 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.253+0000 7f0849007640 1 Processor -- start 2023-10-26T01:39:38.258 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.253+0000 7f0849007640 1 -- start start 2023-10-26T01:39:38.258 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.254+0000 7f0849007640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0830003970 0x7f08440c6660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:38.259 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.254+0000 7f0849007640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08440c8250 0x7f08441516a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:38.259 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.254+0000 7f0849007640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f08440b0890 0x7f08440b0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:38.259 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.254+0000 7f0849007640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08440c79e0 con 0x7f08440c8250 2023-10-26T01:39:38.260 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.254+0000 7f0849007640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08440b1250 con 0x7f08440b0890 2023-10-26T01:39:38.260 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.254+0000 7f0849007640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08440c6d60 con 0x7f0830003970 2023-10-26T01:39:38.260 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.254+0000 7f083bfff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08440c8250 0x7f08441516a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:38.260 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.254+0000 7f083bfff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08440c8250 0x7f08441516a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:33428/0 (socket says 172.21.15.169:33428) 2023-10-26T01:39:38.261 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.254+0000 7f083bfff640 1 -- 172.21.15.169:0/2484699490 learned_addr learned my addr 172.21.15.169:0/2484699490 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:39:38.261 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.254+0000 7f084359e640 1 --2- 172.21.15.169:0/2484699490 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f08440b0890 0x7f08440b0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:38.261 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.254+0000 7f0842d9d640 1 --2- 172.21.15.169:0/2484699490 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0830003970 0x7f08440c6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:38.262 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.254+0000 7f083bfff640 1 -- 172.21.15.169:0/2484699490 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0830003970 msgr2=0x7f08440c6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:38.262 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.254+0000 7f083bfff640 1 --2- 172.21.15.169:0/2484699490 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0830003970 0x7f08440c6660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:38.262 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.254+0000 7f083bfff640 1 -- 172.21.15.169:0/2484699490 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f08440b0890 msgr2=0x7f08440b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:38.262 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.254+0000 7f083bfff640 1 --2- 172.21.15.169:0/2484699490 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f08440b0890 0x7f08440b0d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:38.262 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.255+0000 7f083bfff640 1 -- 172.21.15.169:0/2484699490 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08440c7040 con 0x7f08440c8250 2023-10-26T01:39:38.263 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.255+0000 7f083bfff640 1 --2- 172.21.15.169:0/2484699490 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08440c8250 0x7f08441516a0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f082c00d440 tx=0x7f082c00d900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:38.263 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.255+0000 7f083b7fe640 1 -- 172.21.15.169:0/2484699490 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f082c005030 con 0x7f08440c8250 2023-10-26T01:39:38.263 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.255+0000 7f083b7fe640 1 -- 172.21.15.169:0/2484699490 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f082c0051b0 con 0x7f08440c8250 2023-10-26T01:39:38.263 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.255+0000 7f083b7fe640 1 -- 172.21.15.169:0/2484699490 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f082c0045e0 con 0x7f08440c8250 2023-10-26T01:39:38.264 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.255+0000 7f0849007640 1 -- 172.21.15.169:0/2484699490 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08440c7320 con 0x7f08440c8250 2023-10-26T01:39:38.264 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.255+0000 7f0849007640 1 -- 172.21.15.169:0/2484699490 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08440c9490 con 0x7f08440c8250 2023-10-26T01:39:38.265 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.256+0000 7f0849007640 1 -- 172.21.15.169:0/2484699490 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f084404db10 con 0x7f08440c8250 2023-10-26T01:39:38.265 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.257+0000 7f083b7fe640 1 -- 172.21.15.169:0/2484699490 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f082c004740 con 0x7f08440c8250 2023-10-26T01:39:38.265 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.258+0000 7f083b7fe640 1 --2- 172.21.15.169:0/2484699490 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f082406ae40 0x7f082406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:38.265 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.258+0000 7f083b7fe640 1 -- 172.21.15.169:0/2484699490 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f082c088be0 con 0x7f08440c8250 2023-10-26T01:39:38.266 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.259+0000 7f0842d9d640 1 --2- 172.21.15.169:0/2484699490 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f082406ae40 0x7f082406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:38.266 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.260+0000 7f0842d9d640 1 --2- 172.21.15.169:0/2484699490 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f082406ae40 0x7f082406d300 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f0834034000 tx=0x7f083402e000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:38.267 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.265+0000 7f083b7fe640 1 -- 172.21.15.169:0/2484699490 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f082c053c80 con 0x7f08440c8250 2023-10-26T01:39:38.495 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.492+0000 7f0849007640 1 -- 172.21.15.169:0/2484699490 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f08440b2410 con 0x7f08440c8250 2023-10-26T01:39:38.496 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.493+0000 7f083b7fe640 1 -- 172.21.15.169:0/2484699490 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v77) v1 ==== 53+0+73 (secure 0 0 0) 0x7f082c017070 con 0x7f08440c8250 2023-10-26T01:39:38.496 INFO:teuthology.orchestra.run.smithi169.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-26T01:39:38.499 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.496+0000 7f0849007640 1 -- 172.21.15.169:0/2484699490 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f082406ae40 msgr2=0x7f082406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:38.499 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.496+0000 7f0849007640 1 --2- 172.21.15.169:0/2484699490 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f082406ae40 0x7f082406d300 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f0834034000 tx=0x7f083402e000 comp rx=0 tx=0).stop 2023-10-26T01:39:38.499 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.496+0000 7f0849007640 1 -- 172.21.15.169:0/2484699490 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08440c8250 msgr2=0x7f08441516a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:38.499 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.496+0000 7f0849007640 1 --2- 172.21.15.169:0/2484699490 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08440c8250 0x7f08441516a0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f082c00d440 tx=0x7f082c00d900 comp rx=0 tx=0).stop 2023-10-26T01:39:38.499 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.496+0000 7f0849007640 1 -- 172.21.15.169:0/2484699490 shutdown_connections 2023-10-26T01:39:38.500 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.496+0000 7f0849007640 1 --2- 172.21.15.169:0/2484699490 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f082406ae40 0x7f082406d300 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:38.500 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.496+0000 7f0849007640 1 --2- 172.21.15.169:0/2484699490 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08440c8250 0x7f08441516a0 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:38.500 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.496+0000 7f0849007640 1 --2- 172.21.15.169:0/2484699490 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f08440b0890 0x7f08440b0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:38.500 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.496+0000 7f0849007640 1 --2- 172.21.15.169:0/2484699490 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0830003970 0x7f08440c6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:38.500 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.496+0000 7f0849007640 1 -- 172.21.15.169:0/2484699490 >> 172.21.15.169:0/2484699490 conn(0x7f0844069930 msgr2=0x7f084415de00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:38.501 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.496+0000 7f0849007640 1 -- 172.21.15.169:0/2484699490 shutdown_connections 2023-10-26T01:39:38.501 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:38.496+0000 7f0849007640 1 -- 172.21.15.169:0/2484699490 wait complete. 2023-10-26T01:39:38.516 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-26T01:39:38.517 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-26T01:39:38.517 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi169.front.sepia.ceph.com 2023-10-26T01:39:38.517 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-26T01:39:38.518 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-26T01:39:38.518 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-26T01:39:38.519 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-26T01:39:38.519 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-26T01:39:38.520 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-26T01:39:38.520 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:39:38.520 DEBUG:teuthology.orchestra.run.smithi169:> ip addr 2023-10-26T01:39:38.567 INFO:teuthology.orchestra.run.smithi169.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-26T01:39:38.568 INFO:teuthology.orchestra.run.smithi169.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-26T01:39:38.568 INFO:teuthology.orchestra.run.smithi169.stdout: inet 127.0.0.1/8 scope host lo 2023-10-26T01:39:38.568 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:39:38.568 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 ::1/128 scope host 2023-10-26T01:39:38.568 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:39:38.569 INFO:teuthology.orchestra.run.smithi169.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-26T01:39:38.569 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:d9:3b:b4 brd ff:ff:ff:ff:ff:ff 2023-10-26T01:39:38.569 INFO:teuthology.orchestra.run.smithi169.stdout: altname enp7s0f0 2023-10-26T01:39:38.569 INFO:teuthology.orchestra.run.smithi169.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-26T01:39:38.569 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:d9:3b:b5 brd ff:ff:ff:ff:ff:ff 2023-10-26T01:39:38.570 INFO:teuthology.orchestra.run.smithi169.stdout: altname enp7s0f1 2023-10-26T01:39:38.570 INFO:teuthology.orchestra.run.smithi169.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-26T01:39:38.570 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:88:7f:4c brd ff:ff:ff:ff:ff:ff 2023-10-26T01:39:38.570 INFO:teuthology.orchestra.run.smithi169.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-26T01:39:38.570 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:88:7f:4d brd ff:ff:ff:ff:ff:ff 2023-10-26T01:39:38.571 INFO:teuthology.orchestra.run.smithi169.stdout: inet 172.21.15.169/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-26T01:39:38.571 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft 41917sec preferred_lft 41917sec 2023-10-26T01:39:38.571 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 fe80::ec4:7aff:fe88:7f4d/64 scope link 2023-10-26T01:39:38.572 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:39:38.572 INFO:teuthology.orchestra.run.smithi169.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-26T01:39:38.572 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 22:a0:6b:43:7b:04 brd ff:ff:ff:ff:ff:ff 2023-10-26T01:39:38.572 INFO:teuthology.orchestra.run.smithi169.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-26T01:39:38.572 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:39:38.573 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 fe80::d815:31ff:fe87:d148/64 scope link 2023-10-26T01:39:38.573 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:39:38.573 INFO:teuthology.orchestra.run.smithi169.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-26T01:39:38.573 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 2a:80:f8:45:5f:9f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-26T01:39:38.573 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 fe80::2880:f8ff:fe45:5f9f/64 scope link 2023-10-26T01:39:38.574 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:39:38.574 INFO:teuthology.orchestra.run.smithi169.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-26T01:39:38.574 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 22:a0:6b:43:7b:04 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-26T01:39:38.574 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 fe80::20a0:6bff:fe43:7b04/64 scope link 2023-10-26T01:39:38.574 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:39:38.575 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:39:38.576 DEBUG:teuthology.orchestra.run.smithi169:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-26T01:39:38.711 INFO:teuthology.orchestra.run.smithi169.stdout:1 2023-10-26T01:39:38.715 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:39:38.716 DEBUG:teuthology.orchestra.run.smithi169:> route 2023-10-26T01:39:38.770 INFO:teuthology.orchestra.run.smithi169.stdout:Kernel IP routing table 2023-10-26T01:39:38.770 INFO:teuthology.orchestra.run.smithi169.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-26T01:39:38.770 INFO:teuthology.orchestra.run.smithi169.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-26T01:39:38.771 INFO:teuthology.orchestra.run.smithi169.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-26T01:39:38.771 INFO:teuthology.orchestra.run.smithi169.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-26T01:39:38.771 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:39:38.772 DEBUG:teuthology.orchestra.run.smithi169:> (cd / && exec bash -c ' 2023-10-26T01:39:38.772 DEBUG:teuthology.orchestra.run.smithi169:> set -e 2023-10-26T01:39:38.773 DEBUG:teuthology.orchestra.run.smithi169:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-26T01:39:38.773 DEBUG:teuthology.orchestra.run.smithi169:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-26T01:39:38.774 DEBUG:teuthology.orchestra.run.smithi169:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-26T01:39:38.774 DEBUG:teuthology.orchestra.run.smithi169:> ') 2023-10-26T01:39:38.956 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:39:38.956 DEBUG:teuthology.orchestra.run.smithi169:> ip netns list 2023-10-26T01:39:39.008 INFO:teuthology.orchestra.run.smithi169.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-26T01:39:39.008 INFO:teuthology.orchestra.run.smithi169.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-26T01:39:39.009 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-26T01:39:39.009 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:39:39.010 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link set brx.0 up 2023-10-26T01:39:39.108 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-26T01:39:39.109 DEBUG:teuthology.orchestra.run.smithi169:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-26T01:39:39.167 INFO:teuthology.orchestra.run.smithi169.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-26T01:39:39.167 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-26T01:39:39.168 DEBUG:teuthology.orchestra.run.smithi169:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-26T01:39:39.222 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-26T01:39:39.222 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:39:39.223 DEBUG:teuthology.orchestra.run.smithi169:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-26T01:39:39.364 INFO:teuthology.orchestra.run.smithi169.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-26T01:39:39.364 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-26T01:39:39.365 INFO:teuthology.orchestra.run.smithi169.stdout:invalid new device string format 2023-10-26T01:39:39.386 INFO:teuthology.orchestra.run.smithi169.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-26T01:39:39.386 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-26T01:39:39.386 INFO:teuthology.orchestra.run.smithi169.stdout:invalid new device string format 2023-10-26T01:39:39.386 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: resolved to: "172.21.15.169:3300,172.21.15.170:3300,172.21.15.170:3301" 2023-10-26T01:39:39.387 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-26T01:39:39.410 INFO:teuthology.orchestra.run.smithi169.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-26T01:39:39.410 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-26T01:39:39.410 INFO:teuthology.orchestra.run.smithi169.stdout:invalid new device string format 2023-10-26T01:39:39.410 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: resolved to: "172.21.15.169:3300,172.21.15.170:3300,172.21.15.170:3301" 2023-10-26T01:39:39.411 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-26T01:39:39.411 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: resolved to: "172.21.15.169:6789,172.21.15.170:6789,172.21.15.170:6790" 2023-10-26T01:39:39.411 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.169:6789,172.21.15.170:6789,172.21.15.170:6790:/ 2023-10-26T01:39:39.411 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=30203b28-ed8a-44e8-a9a6-a9bf25f980f1" will pass to kernel 2023-10-26T01:39:39.412 INFO:teuthology.orchestra.run.smithi169.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2023-10-26T01:39:39.412 INFO:teuthology.orchestra.run.smithi169.stdout: You just mounted a file system that supports labels which does not 2023-10-26T01:39:39.412 INFO:teuthology.orchestra.run.smithi169.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-10-26T01:39:39.412 INFO:teuthology.orchestra.run.smithi169.stdout: applications will generate AVC messages and not be allowed access to 2023-10-26T01:39:39.413 INFO:teuthology.orchestra.run.smithi169.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-10-26T01:39:39.415 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-26T01:39:39.416 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:39:39.416 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-26T01:39:39.496 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:39:39.497 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-26T01:39:39.556 INFO:teuthology.orchestra.run.smithi169.stdout:client8421 2023-10-26T01:39:39.556 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:39:39.557 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:39:39.611 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-26T01:39:39.828 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:39.824+0000 7fba3b59e640 1 -- 172.21.15.169:0/3031136924 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fba3c0ccb60 msgr2=0x7fba3c0b0d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:39.828 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:39.824+0000 7fba3b59e640 1 --2- 172.21.15.169:0/3031136924 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fba3c0ccb60 0x7fba3c0b0d60 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7fba3c0b1c30 tx=0x7fba2c02c250 comp rx=0 tx=0).stop 2023-10-26T01:39:39.828 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:39.825+0000 7fba3b59e640 1 -- 172.21.15.169:0/3031136924 shutdown_connections 2023-10-26T01:39:39.829 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:39.825+0000 7fba3b59e640 1 --2- 172.21.15.169:0/3031136924 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fba3c15be10 0x7fba3c15c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:39.829 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:39.825+0000 7fba3b59e640 1 --2- 172.21.15.169:0/3031136924 >> v2:172.21.15.170:3300/0 conn(0x7fba3c06c520 0x7fba3c06c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:39.829 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:39.825+0000 7fba3b59e640 1 --2- 172.21.15.169:0/3031136924 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fba3c0ccb60 0x7fba3c0b0d60 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:39.829 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:39.825+0000 7fba3b59e640 1 -- 172.21.15.169:0/3031136924 >> 172.21.15.169:0/3031136924 conn(0x7fba3c0af0c0 msgr2=0x7fba3c0af4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:39.829 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:39.825+0000 7fba3b59e640 1 -- 172.21.15.169:0/3031136924 shutdown_connections 2023-10-26T01:39:39.830 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:39.825+0000 7fba3b59e640 1 -- 172.21.15.169:0/3031136924 wait complete. 2023-10-26T01:39:39.831 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:39.826+0000 7fba3b59e640 1 Processor -- start 2023-10-26T01:39:39.832 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:39.827+0000 7fba3b59e640 1 -- start start 2023-10-26T01:39:39.832 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:39.827+0000 7fba3b59e640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fba3c06c520 0x7fba3c14a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:39.832 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:39.827+0000 7fba3b59e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fba3c0ccb60 0x7fba3c06f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:39.832 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:39.827+0000 7fba3b59e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fba3c15be10 0x7fba3c06f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:39.833 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:39.827+0000 7fba3b59e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba3c14acb0 con 0x7fba3c15be10 2023-10-26T01:39:39.833 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:39.827+0000 7fba3b59e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba3c1496f0 con 0x7fba3c0ccb60 2023-10-26T01:39:39.833 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:39.827+0000 7fba3b59e640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba3c1499d0 con 0x7fba3c06c520 2023-10-26T01:39:39.833 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:39.828+0000 7fba3ad9d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fba3c15be10 0x7fba3c06f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:39.833 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:39.828+0000 7fba3ad9d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fba3c15be10 0x7fba3c06f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:33448/0 (socket says 172.21.15.169:33448) 2023-10-26T01:39:39.834 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:39.828+0000 7fba3ad9d640 1 -- 172.21.15.169:0/3741350550 learned_addr learned my addr 172.21.15.169:0/3741350550 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:39:39.834 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:39.828+0000 7fba3a59c640 1 --2- 172.21.15.169:0/3741350550 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fba3c06c520 0x7fba3c14a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:39.834 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:39.828+0000 7fba3ad9d640 1 -- 172.21.15.169:0/3741350550 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fba3c06c520 msgr2=0x7fba3c14a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:39.834 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:39.828+0000 7fba3ad9d640 1 --2- 172.21.15.169:0/3741350550 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fba3c06c520 0x7fba3c14a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:39.834 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:39.828+0000 7fba3ad9d640 1 -- 172.21.15.169:0/3741350550 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fba3c0ccb60 msgr2=0x7fba3c06f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:39.835 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:39.828+0000 7fba3ad9d640 1 --2- 172.21.15.169:0/3741350550 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fba3c0ccb60 0x7fba3c06f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:39.835 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:39.828+0000 7fba3ad9d640 1 -- 172.21.15.169:0/3741350550 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba2c009900 con 0x7fba3c15be10 2023-10-26T01:39:39.835 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:39.828+0000 7fba3ad9d640 1 --2- 172.21.15.169:0/3741350550 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fba3c15be10 0x7fba3c06f560 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7fba3400c8f0 tx=0x7fba3400cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:39.835 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:39.829+0000 7fba32ffd640 1 -- 172.21.15.169:0/3741350550 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba34007bb0 con 0x7fba3c15be10 2023-10-26T01:39:39.835 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:39.829+0000 7fba32ffd640 1 -- 172.21.15.169:0/3741350550 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fba34007e00 con 0x7fba3c15be10 2023-10-26T01:39:39.836 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:39.829+0000 7fba32ffd640 1 -- 172.21.15.169:0/3741350550 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba34005280 con 0x7fba3c15be10 2023-10-26T01:39:39.836 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:39.829+0000 7fba3b59e640 1 -- 172.21.15.169:0/3741350550 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba3c149cb0 con 0x7fba3c15be10 2023-10-26T01:39:39.836 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:39.829+0000 7fba3b59e640 1 -- 172.21.15.169:0/3741350550 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba3c14a270 con 0x7fba3c15be10 2023-10-26T01:39:39.837 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:39.830+0000 7fba3b59e640 1 -- 172.21.15.169:0/3741350550 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba3c09b540 con 0x7fba3c15be10 2023-10-26T01:39:39.837 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:39.831+0000 7fba32ffd640 1 -- 172.21.15.169:0/3741350550 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fba340053e0 con 0x7fba3c15be10 2023-10-26T01:39:39.837 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:39.832+0000 7fba32ffd640 1 --2- 172.21.15.169:0/3741350550 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fba1c06ae40 0x7fba1c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:39.838 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:39.832+0000 7fba32ffd640 1 -- 172.21.15.169:0/3741350550 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fba34089e90 con 0x7fba3c15be10 2023-10-26T01:39:39.838 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:39.832+0000 7fba3a59c640 1 --2- 172.21.15.169:0/3741350550 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fba1c06ae40 0x7fba1c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:39.838 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:39.834+0000 7fba3a59c640 1 --2- 172.21.15.169:0/3741350550 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fba1c06ae40 0x7fba1c06d300 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fba2c036000 tx=0x7fba2c004ab0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:39.839 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:39.836+0000 7fba32ffd640 1 -- 172.21.15.169:0/3741350550 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fba34054ea0 con 0x7fba3c15be10 2023-10-26T01:39:40.067 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:40.064+0000 7fba3b59e640 1 -- 172.21.15.169:0/3741350550 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fba3c1463e0 con 0x7fba3c15be10 2023-10-26T01:39:40.067 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:40.065+0000 7fba32ffd640 1 -- 172.21.15.169:0/3741350550 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v77) v1 ==== 53+0+73 (secure 0 0 0) 0x7fba3408b020 con 0x7fba3c15be10 2023-10-26T01:39:40.068 INFO:teuthology.orchestra.run.smithi169.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-26T01:39:40.070 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:40.067+0000 7fba3b59e640 1 -- 172.21.15.169:0/3741350550 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fba1c06ae40 msgr2=0x7fba1c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:40.070 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:40.067+0000 7fba3b59e640 1 --2- 172.21.15.169:0/3741350550 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fba1c06ae40 0x7fba1c06d300 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fba2c036000 tx=0x7fba2c004ab0 comp rx=0 tx=0).stop 2023-10-26T01:39:40.071 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:40.067+0000 7fba3b59e640 1 -- 172.21.15.169:0/3741350550 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fba3c15be10 msgr2=0x7fba3c06f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:40.071 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:40.067+0000 7fba3b59e640 1 --2- 172.21.15.169:0/3741350550 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fba3c15be10 0x7fba3c06f560 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7fba3400c8f0 tx=0x7fba3400cdb0 comp rx=0 tx=0).stop 2023-10-26T01:39:40.071 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:40.067+0000 7fba3b59e640 1 -- 172.21.15.169:0/3741350550 shutdown_connections 2023-10-26T01:39:40.071 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:40.067+0000 7fba3b59e640 1 --2- 172.21.15.169:0/3741350550 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fba1c06ae40 0x7fba1c06d300 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:40.071 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:40.067+0000 7fba3b59e640 1 --2- 172.21.15.169:0/3741350550 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fba3c15be10 0x7fba3c06f560 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:40.072 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:40.067+0000 7fba3b59e640 1 --2- 172.21.15.169:0/3741350550 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fba3c0ccb60 0x7fba3c06f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:40.072 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:40.067+0000 7fba3b59e640 1 --2- 172.21.15.169:0/3741350550 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fba3c06c520 0x7fba3c14a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:40.072 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:40.067+0000 7fba3b59e640 1 -- 172.21.15.169:0/3741350550 >> 172.21.15.169:0/3741350550 conn(0x7fba3c0af0c0 msgr2=0x7fba3c0aab40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:40.072 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:40.068+0000 7fba3b59e640 1 -- 172.21.15.169:0/3741350550 shutdown_connections 2023-10-26T01:39:40.072 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:40.068+0000 7fba3b59e640 1 -- 172.21.15.169:0/3741350550 wait complete. 2023-10-26T01:39:40.089 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-26T01:39:40.089 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-26T01:39:40.089 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi170.front.sepia.ceph.com 2023-10-26T01:39:40.090 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-26T01:39:40.090 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-26T01:39:40.090 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-26T01:39:40.091 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-26T01:39:40.091 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-26T01:39:40.092 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-26T01:39:40.092 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:39:40.092 DEBUG:teuthology.orchestra.run.smithi170:> ip addr 2023-10-26T01:39:40.152 INFO:teuthology.orchestra.run.smithi170.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-26T01:39:40.153 INFO:teuthology.orchestra.run.smithi170.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-26T01:39:40.153 INFO:teuthology.orchestra.run.smithi170.stdout: inet 127.0.0.1/8 scope host lo 2023-10-26T01:39:40.153 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:39:40.153 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 ::1/128 scope host 2023-10-26T01:39:40.154 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:39:40.154 INFO:teuthology.orchestra.run.smithi170.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-26T01:39:40.154 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:d9:3b:54 brd ff:ff:ff:ff:ff:ff 2023-10-26T01:39:40.154 INFO:teuthology.orchestra.run.smithi170.stdout: altname enp7s0f0 2023-10-26T01:39:40.154 INFO:teuthology.orchestra.run.smithi170.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-26T01:39:40.155 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:d9:3b:55 brd ff:ff:ff:ff:ff:ff 2023-10-26T01:39:40.155 INFO:teuthology.orchestra.run.smithi170.stdout: altname enp7s0f1 2023-10-26T01:39:40.155 INFO:teuthology.orchestra.run.smithi170.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-26T01:39:40.155 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:88:72:54 brd ff:ff:ff:ff:ff:ff 2023-10-26T01:39:40.155 INFO:teuthology.orchestra.run.smithi170.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-26T01:39:40.155 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:88:72:55 brd ff:ff:ff:ff:ff:ff 2023-10-26T01:39:40.156 INFO:teuthology.orchestra.run.smithi170.stdout: inet 172.21.15.170/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-26T01:39:40.156 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft 41900sec preferred_lft 41900sec 2023-10-26T01:39:40.156 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::ec4:7aff:fe88:7255/64 scope link 2023-10-26T01:39:40.156 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:39:40.156 INFO:teuthology.orchestra.run.smithi170.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-26T01:39:40.157 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 32:b5:d1:ee:10:36 brd ff:ff:ff:ff:ff:ff 2023-10-26T01:39:40.157 INFO:teuthology.orchestra.run.smithi170.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-26T01:39:40.157 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:39:40.157 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::28ed:d1ff:fe37:70d/64 scope link 2023-10-26T01:39:40.157 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:39:40.158 INFO:teuthology.orchestra.run.smithi170.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-26T01:39:40.158 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 32:b5:d1:ee:10:36 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-26T01:39:40.158 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::30b5:d1ff:feee:1036/64 scope link 2023-10-26T01:39:40.158 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:39:40.158 INFO:teuthology.orchestra.run.smithi170.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-26T01:39:40.158 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether b2:c1:53:19:69:e4 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-26T01:39:40.159 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::b0c1:53ff:fe19:69e4/64 scope link 2023-10-26T01:39:40.159 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:39:40.160 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:39:40.160 DEBUG:teuthology.orchestra.run.smithi170:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-26T01:39:40.252 INFO:teuthology.orchestra.run.smithi170.stdout:1 2023-10-26T01:39:40.256 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:39:40.256 DEBUG:teuthology.orchestra.run.smithi170:> route 2023-10-26T01:39:40.313 INFO:teuthology.orchestra.run.smithi170.stdout:Kernel IP routing table 2023-10-26T01:39:40.313 INFO:teuthology.orchestra.run.smithi170.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-26T01:39:40.313 INFO:teuthology.orchestra.run.smithi170.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-26T01:39:40.313 INFO:teuthology.orchestra.run.smithi170.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-26T01:39:40.314 INFO:teuthology.orchestra.run.smithi170.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-26T01:39:40.314 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:39:40.314 DEBUG:teuthology.orchestra.run.smithi170:> (cd / && exec bash -c ' 2023-10-26T01:39:40.315 DEBUG:teuthology.orchestra.run.smithi170:> set -e 2023-10-26T01:39:40.315 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-26T01:39:40.315 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-26T01:39:40.316 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-26T01:39:40.316 DEBUG:teuthology.orchestra.run.smithi170:> ') 2023-10-26T01:39:40.496 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:39:40.496 DEBUG:teuthology.orchestra.run.smithi170:> ip netns list 2023-10-26T01:39:40.546 INFO:teuthology.orchestra.run.smithi170.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-26T01:39:40.547 INFO:teuthology.orchestra.run.smithi170.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-26T01:39:40.547 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-26T01:39:40.548 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:39:40.548 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip link set brx.0 up 2023-10-26T01:39:40.643 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-26T01:39:40.643 DEBUG:teuthology.orchestra.run.smithi170:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-26T01:39:40.701 INFO:teuthology.orchestra.run.smithi170.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-26T01:39:40.702 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-26T01:39:40.702 DEBUG:teuthology.orchestra.run.smithi170:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-26T01:39:40.756 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-26T01:39:40.756 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:39:40.757 DEBUG:teuthology.orchestra.run.smithi170:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-26T01:39:40.894 INFO:teuthology.orchestra.run.smithi170.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-26T01:39:40.895 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-26T01:39:40.895 INFO:teuthology.orchestra.run.smithi170.stdout:invalid new device string format 2023-10-26T01:39:40.916 INFO:teuthology.orchestra.run.smithi170.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-26T01:39:40.916 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-26T01:39:40.917 INFO:teuthology.orchestra.run.smithi170.stdout:invalid new device string format 2023-10-26T01:39:40.917 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: resolved to: "172.21.15.169:3300,172.21.15.170:3300,172.21.15.170:3301" 2023-10-26T01:39:40.917 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-26T01:39:40.940 INFO:teuthology.orchestra.run.smithi170.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-26T01:39:40.940 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-26T01:39:40.940 INFO:teuthology.orchestra.run.smithi170.stdout:invalid new device string format 2023-10-26T01:39:40.941 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: resolved to: "172.21.15.169:3300,172.21.15.170:3300,172.21.15.170:3301" 2023-10-26T01:39:40.941 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-26T01:39:40.941 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: resolved to: "172.21.15.169:6789,172.21.15.170:6789,172.21.15.170:6790" 2023-10-26T01:39:40.941 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.169:6789,172.21.15.170:6789,172.21.15.170:6790:/ 2023-10-26T01:39:40.942 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=30203b28-ed8a-44e8-a9a6-a9bf25f980f1" will pass to kernel 2023-10-26T01:39:40.942 INFO:teuthology.orchestra.run.smithi170.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2023-10-26T01:39:40.943 INFO:teuthology.orchestra.run.smithi170.stdout: You just mounted a file system that supports labels which does not 2023-10-26T01:39:40.943 INFO:teuthology.orchestra.run.smithi170.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-10-26T01:39:40.945 INFO:teuthology.orchestra.run.smithi170.stdout: applications will generate AVC messages and not be allowed access to 2023-10-26T01:39:40.945 INFO:teuthology.orchestra.run.smithi170.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-10-26T01:39:40.947 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-26T01:39:40.948 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:39:40.948 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-26T01:39:41.030 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:39:41.031 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-26T01:39:41.084 INFO:teuthology.orchestra.run.smithi170.stdout:client5792 2023-10-26T01:39:41.085 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-26T01:39:41.085 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:39:41.137 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T01:39:41.370 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:41.367+0000 7ffbb5333640 1 -- 172.21.15.169:0/1159667272 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffbb00cdc30 msgr2=0x7ffbb00cbac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:41.370 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:41.367+0000 7ffbb5333640 1 --2- 172.21.15.169:0/1159667272 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffbb00cdc30 0x7ffbb00cbac0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7ffba0009830 tx=0x7ffba002bcd0 comp rx=0 tx=0).stop 2023-10-26T01:39:41.371 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:41.367+0000 7ffbb5333640 1 -- 172.21.15.169:0/1159667272 shutdown_connections 2023-10-26T01:39:41.371 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:41.367+0000 7ffbb5333640 1 --2- 172.21.15.169:0/1159667272 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffbb00cdc30 0x7ffbb00cbac0 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:41.371 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:41.367+0000 7ffbb5333640 1 --2- 172.21.15.169:0/1159667272 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ffbb00ac930 0x7ffbb00cd6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:41.371 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:41.367+0000 7ffbb5333640 1 -- 172.21.15.169:0/1159667272 >> 172.21.15.169:0/1159667272 conn(0x7ffbb0069930 msgr2=0x7ffbb0069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:41.372 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:41.367+0000 7ffbb5333640 1 -- 172.21.15.169:0/1159667272 shutdown_connections 2023-10-26T01:39:41.372 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:41.367+0000 7ffbb5333640 1 -- 172.21.15.169:0/1159667272 wait complete. 2023-10-26T01:39:41.374 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:41.369+0000 7ffbb5333640 1 Processor -- start 2023-10-26T01:39:41.374 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:41.369+0000 7ffbb5333640 1 -- start start 2023-10-26T01:39:41.374 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:41.370+0000 7ffbb5333640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffbb00ac930 0x7ffbb0149c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:41.375 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:41.370+0000 7ffbb5333640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffbb00cdc30 0x7ffbb014a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:41.375 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:41.370+0000 7ffbb5333640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ffbb01592d0 0x7ffbb00c86e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:41.375 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:41.370+0000 7ffbb5333640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb0159ba0 con 0x7ffbb00ac930 2023-10-26T01:39:41.375 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:41.370+0000 7ffbb5333640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb015af60 con 0x7ffbb00cdc30 2023-10-26T01:39:41.375 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:41.370+0000 7ffbb5333640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb00c7440 con 0x7ffbb01592d0 2023-10-26T01:39:41.376 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:41.370+0000 7ffbaeffd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffbb00ac930 0x7ffbb0149c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:41.376 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:41.370+0000 7ffbaeffd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffbb00ac930 0x7ffbb0149c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:33472/0 (socket says 172.21.15.169:33472) 2023-10-26T01:39:41.376 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:41.370+0000 7ffbaeffd640 1 -- 172.21.15.169:0/257957300 learned_addr learned my addr 172.21.15.169:0/257957300 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:39:41.376 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:41.371+0000 7ffbaeffd640 1 -- 172.21.15.169:0/257957300 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ffbb01592d0 msgr2=0x7ffbb00c86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:41.376 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:41.371+0000 7ffbaeffd640 1 --2- 172.21.15.169:0/257957300 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ffbb01592d0 0x7ffbb00c86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:41.377 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:41.371+0000 7ffbaeffd640 1 -- 172.21.15.169:0/257957300 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffbb00cdc30 msgr2=0x7ffbb014a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:41.377 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:41.371+0000 7ffbaeffd640 1 --2- 172.21.15.169:0/257957300 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffbb00cdc30 0x7ffbb014a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:41.377 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:41.371+0000 7ffbaeffd640 1 -- 172.21.15.169:0/257957300 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffba0009490 con 0x7ffbb00ac930 2023-10-26T01:39:41.377 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:41.371+0000 7ffbaeffd640 1 --2- 172.21.15.169:0/257957300 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffbb00ac930 0x7ffbb0149c00 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7ffba800cae0 tx=0x7ffba800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:41.377 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:41.371+0000 7ffba77fe640 1 -- 172.21.15.169:0/257957300 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba800ac40 con 0x7ffbb00ac930 2023-10-26T01:39:41.378 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:41.371+0000 7ffba77fe640 1 -- 172.21.15.169:0/257957300 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ffba800ae90 con 0x7ffbb00ac930 2023-10-26T01:39:41.378 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:41.371+0000 7ffba77fe640 1 -- 172.21.15.169:0/257957300 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba8003b90 con 0x7ffbb00ac930 2023-10-26T01:39:41.378 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:41.371+0000 7ffbb5333640 1 -- 172.21.15.169:0/257957300 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbb00c7720 con 0x7ffbb00ac930 2023-10-26T01:39:41.378 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:41.371+0000 7ffbb5333640 1 -- 172.21.15.169:0/257957300 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbb00c7ce0 con 0x7ffbb00ac930 2023-10-26T01:39:41.379 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:41.373+0000 7ffbb5333640 1 -- 172.21.15.169:0/257957300 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbb015b120 con 0x7ffbb00ac930 2023-10-26T01:39:41.379 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:41.374+0000 7ffba77fe640 1 -- 172.21.15.169:0/257957300 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ffba8004030 con 0x7ffbb00ac930 2023-10-26T01:39:41.380 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:41.375+0000 7ffba77fe640 1 --2- 172.21.15.169:0/257957300 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ffb9006ae40 0x7ffb9006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:41.380 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:41.375+0000 7ffba77fe640 1 -- 172.21.15.169:0/257957300 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ffba8089e60 con 0x7ffbb00ac930 2023-10-26T01:39:41.380 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:41.375+0000 7ffbae7fc640 1 --2- 172.21.15.169:0/257957300 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ffb9006ae40 0x7ffb9006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:41.380 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:41.377+0000 7ffbae7fc640 1 --2- 172.21.15.169:0/257957300 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ffb9006ae40 0x7ffb9006d300 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7ffbb00aaab0 tx=0x7ffb98005e50 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:41.381 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:41.379+0000 7ffba77fe640 1 -- 172.21.15.169:0/257957300 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ffba8058a30 con 0x7ffbb00ac930 2023-10-26T01:39:41.607 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:41.605+0000 7ffbb5333640 1 -- 172.21.15.169:0/257957300 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffbb00ce080 con 0x7ffbb00ac930 2023-10-26T01:39:41.609 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:41.606+0000 7ffba77fe640 1 -- 172.21.15.169:0/257957300 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+4921 (secure 0 0 0) 0x7ffbb00ce080 con 0x7ffbb00ac930 2023-10-26T01:39:41.609 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:39:41.609 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":77,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5711,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/3037106607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":3037106607},{"type":"v1","addr":"172.21.15.170:6836","nonce":3037106607}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72},{"gid":5714,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/1847866184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":1847866184},{"type":"v1","addr":"172.21.15.170:6837","nonce":1847866184}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72},{"gid":8229,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2707290348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2707290348},{"type":"v1","addr":"172.21.15.169:6833","nonce":2707290348}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T01:39:25.790062+0000","modified":"2023-10-26T01:39:28.841639+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8232},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8232":{"gid":8232,"name":"c","rank":0,"incarnation":73,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6835/66540264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":66540264},{"type":"v1","addr":"172.21.15.169:6835","nonce":66540264}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-10-26T01:39:41.610 INFO:teuthology.orchestra.run.smithi169.stderr:dumped fsmap epoch 77 2023-10-26T01:39:41.612 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:41.609+0000 7ffbb5333640 1 -- 172.21.15.169:0/257957300 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ffb9006ae40 msgr2=0x7ffb9006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:41.612 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:41.609+0000 7ffbb5333640 1 --2- 172.21.15.169:0/257957300 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ffb9006ae40 0x7ffb9006d300 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7ffbb00aaab0 tx=0x7ffb98005e50 comp rx=0 tx=0).stop 2023-10-26T01:39:41.612 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:41.609+0000 7ffbb5333640 1 -- 172.21.15.169:0/257957300 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffbb00ac930 msgr2=0x7ffbb0149c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:41.612 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:41.609+0000 7ffbb5333640 1 --2- 172.21.15.169:0/257957300 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffbb00ac930 0x7ffbb0149c00 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7ffba800cae0 tx=0x7ffba800a060 comp rx=0 tx=0).stop 2023-10-26T01:39:41.613 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:41.609+0000 7ffbb5333640 1 -- 172.21.15.169:0/257957300 shutdown_connections 2023-10-26T01:39:41.613 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:41.609+0000 7ffbb5333640 1 --2- 172.21.15.169:0/257957300 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ffb9006ae40 0x7ffb9006d300 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:41.613 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:41.609+0000 7ffbb5333640 1 --2- 172.21.15.169:0/257957300 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ffbb01592d0 0x7ffbb00c86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:41.613 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:41.609+0000 7ffbb5333640 1 --2- 172.21.15.169:0/257957300 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffbb00cdc30 0x7ffbb014a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:41.613 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:41.609+0000 7ffbb5333640 1 --2- 172.21.15.169:0/257957300 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffbb00ac930 0x7ffbb0149c00 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:41.614 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:41.609+0000 7ffbb5333640 1 -- 172.21.15.169:0/257957300 >> 172.21.15.169:0/257957300 conn(0x7ffbb0069930 msgr2=0x7ffbb00ccaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:41.614 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:41.609+0000 7ffbb5333640 1 -- 172.21.15.169:0/257957300 shutdown_connections 2023-10-26T01:39:41.614 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:41.609+0000 7ffbb5333640 1 -- 172.21.15.169:0/257957300 wait complete. 2023-10-26T01:39:41.632 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2023-10-26T01:39:41.867 INFO:teuthology.orchestra.run.smithi169.stdout:{"cluster_fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","whoami":0,"id":8232,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":15.505201347,"mdsmap_epoch":77,"osdmap_epoch":151,"osdmap_epoch_barrier":151,"uptime":21.456789746999998} 2023-10-26T01:39:41.868 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "30203b28-ed8a-44e8-a9a6-a9bf25f980f1", "fs_name": "cephfs", "id": 8232, "mdsmap_epoch": 77, "osdmap_epoch": 151, "osdmap_epoch_barrier": 151, "rank_uptime": 15.505201347, "state": "up:active", "uptime": 21.456789747, "want_state": "up:active", "whoami": 0 } 2023-10-26T01:39:41.868 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set-quota cephfs_data max_bytes 33554432 2023-10-26T01:39:42.096 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:42.093+0000 7f27265d0640 1 -- 172.21.15.169:0/3386537941 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f272015d120 msgr2=0x7f272006bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:42.096 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:42.093+0000 7f27265d0640 1 --2- 172.21.15.169:0/3386537941 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f272015d120 0x7f272006bd80 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f272006e5e0 tx=0x7f271402bd30 comp rx=0 tx=0).stop 2023-10-26T01:39:42.096 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:42.093+0000 7f27265d0640 1 -- 172.21.15.169:0/3386537941 shutdown_connections 2023-10-26T01:39:42.097 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:42.093+0000 7f27265d0640 1 --2- 172.21.15.169:0/3386537941 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f272015d120 0x7f272006bd80 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:42.097 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:42.093+0000 7f27265d0640 1 --2- 172.21.15.169:0/3386537941 >> v2:172.21.15.170:3300/0 conn(0x7f27201595e0 0x7f27200c9bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:42.097 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:42.093+0000 7f27265d0640 1 -- 172.21.15.169:0/3386537941 >> 172.21.15.169:0/3386537941 conn(0x7f27200aeee0 msgr2=0x7f27200af2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:42.097 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:42.093+0000 7f27265d0640 1 -- 172.21.15.169:0/3386537941 shutdown_connections 2023-10-26T01:39:42.098 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:42.093+0000 7f27265d0640 1 -- 172.21.15.169:0/3386537941 wait complete. 2023-10-26T01:39:42.098 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:42.095+0000 7f27265d0640 1 Processor -- start 2023-10-26T01:39:42.099 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:42.095+0000 7f27265d0640 1 -- start start 2023-10-26T01:39:42.101 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:42.096+0000 7f27265d0640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f272006c2c0 0x7f27201416f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:42.101 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:42.096+0000 7f27265d0640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27201595e0 0x7f2720144190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:42.101 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:42.096+0000 7f27265d0640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f272015d120 0x7f27201f1c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:42.101 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:42.096+0000 7f27265d0640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2720151840 con 0x7f27201595e0 2023-10-26T01:39:42.102 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:42.096+0000 7f27265d0640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2720071bb0 con 0x7f272006c2c0 2023-10-26T01:39:42.102 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:42.096+0000 7f27265d0640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27201446d0 con 0x7f272015d120 2023-10-26T01:39:42.102 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:42.097+0000 7f271f7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27201595e0 0x7f2720144190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:42.102 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:42.097+0000 7f271f7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27201595e0 0x7f2720144190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:33486/0 (socket says 172.21.15.169:33486) 2023-10-26T01:39:42.102 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:42.097+0000 7f271f7fe640 1 -- 172.21.15.169:0/1553384203 learned_addr learned my addr 172.21.15.169:0/1553384203 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:39:42.103 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:42.097+0000 7f271ffff640 1 --2- 172.21.15.169:0/1553384203 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f272006c2c0 0x7f27201416f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:42.103 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:42.097+0000 7f2724b6d640 1 --2- 172.21.15.169:0/1553384203 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f272015d120 0x7f27201f1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:42.103 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:42.097+0000 7f271f7fe640 1 -- 172.21.15.169:0/1553384203 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f272015d120 msgr2=0x7f27201f1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:42.104 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:42.097+0000 7f271f7fe640 1 --2- 172.21.15.169:0/1553384203 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f272015d120 0x7f27201f1c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:42.104 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:42.097+0000 7f271f7fe640 1 -- 172.21.15.169:0/1553384203 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f272006c2c0 msgr2=0x7f27201416f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:42.105 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:42.097+0000 7f271f7fe640 1 --2- 172.21.15.169:0/1553384203 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f272006c2c0 0x7f27201416f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:42.105 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:42.097+0000 7f271f7fe640 1 -- 172.21.15.169:0/1553384203 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2714009560 con 0x7f27201595e0 2023-10-26T01:39:42.105 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:42.097+0000 7f271f7fe640 1 --2- 172.21.15.169:0/1553384203 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27201595e0 0x7f2720144190 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f2714000c00 tx=0x7f271402dad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:42.106 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:42.098+0000 7f271cff9640 1 -- 172.21.15.169:0/1553384203 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2714003800 con 0x7f27201595e0 2023-10-26T01:39:42.106 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:42.098+0000 7f271cff9640 1 -- 172.21.15.169:0/1553384203 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2714003a50 con 0x7f27201595e0 2023-10-26T01:39:42.107 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:42.098+0000 7f271cff9640 1 -- 172.21.15.169:0/1553384203 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2714003e60 con 0x7f27201595e0 2023-10-26T01:39:42.107 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:42.098+0000 7f27265d0640 1 -- 172.21.15.169:0/1553384203 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27200715f0 con 0x7f27201595e0 2023-10-26T01:39:42.107 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:42.098+0000 7f27265d0640 1 -- 172.21.15.169:0/1553384203 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2720151a70 con 0x7f27201595e0 2023-10-26T01:39:42.109 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:42.099+0000 7f27265d0640 1 -- 172.21.15.169:0/1553384203 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2720066af0 con 0x7f27201595e0 2023-10-26T01:39:42.110 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:42.100+0000 7f271cff9640 1 -- 172.21.15.169:0/1553384203 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f271403e450 con 0x7f27201595e0 2023-10-26T01:39:42.110 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:42.101+0000 7f271cff9640 1 --2- 172.21.15.169:0/1553384203 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f270c06ae40 0x7f270c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:39:42.110 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:42.102+0000 7f271cff9640 1 -- 172.21.15.169:0/1553384203 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f27140ac250 con 0x7f27201595e0 2023-10-26T01:39:42.111 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:42.102+0000 7f271ffff640 1 --2- 172.21.15.169:0/1553384203 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f270c06ae40 0x7f270c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:39:42.111 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:42.103+0000 7f271ffff640 1 --2- 172.21.15.169:0/1553384203 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f270c06ae40 0x7f270c06d300 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f2710005fd0 tx=0x7f2710005850 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:39:42.112 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:42.106+0000 7f271cff9640 1 -- 172.21.15.169:0/1553384203 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f27140771d0 con 0x7f27201595e0 2023-10-26T01:39:42.298 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:42.296+0000 7f27265d0640 1 -- 172.21.15.169:0/1553384203 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"} v 0) v1 -- 0x7f27200caa00 con 0x7f27201595e0 2023-10-26T01:39:43.140 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:43.137+0000 7f271cff9640 1 -- 172.21.15.169:0/1553384203 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"}]=0 set-quota max_bytes = 33554432 for pool cephfs_data v152) v1 ==== 181+0+0 (secure 0 0 0) 0x7f27140420c0 con 0x7f27201595e0 2023-10-26T01:39:43.141 INFO:teuthology.orchestra.run.smithi169.stderr:set-quota max_bytes = 33554432 for pool cephfs_data 2023-10-26T01:39:43.144 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:43.141+0000 7f27265d0640 1 -- 172.21.15.169:0/1553384203 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f270c06ae40 msgr2=0x7f270c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:43.145 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:43.141+0000 7f27265d0640 1 --2- 172.21.15.169:0/1553384203 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f270c06ae40 0x7f270c06d300 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f2710005fd0 tx=0x7f2710005850 comp rx=0 tx=0).stop 2023-10-26T01:39:43.145 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:43.141+0000 7f27265d0640 1 -- 172.21.15.169:0/1553384203 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27201595e0 msgr2=0x7f2720144190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:39:43.145 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:43.141+0000 7f27265d0640 1 --2- 172.21.15.169:0/1553384203 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27201595e0 0x7f2720144190 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f2714000c00 tx=0x7f271402dad0 comp rx=0 tx=0).stop 2023-10-26T01:39:43.145 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:43.142+0000 7f27265d0640 1 -- 172.21.15.169:0/1553384203 shutdown_connections 2023-10-26T01:39:43.146 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:43.142+0000 7f27265d0640 1 --2- 172.21.15.169:0/1553384203 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f270c06ae40 0x7f270c06d300 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:43.146 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:43.142+0000 7f27265d0640 1 --2- 172.21.15.169:0/1553384203 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27201595e0 0x7f2720144190 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:43.146 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:43.142+0000 7f27265d0640 1 --2- 172.21.15.169:0/1553384203 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f272015d120 0x7f27201f1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:43.146 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:43.142+0000 7f27265d0640 1 --2- 172.21.15.169:0/1553384203 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f272006c2c0 0x7f27201416f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:39:43.146 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:43.142+0000 7f27265d0640 1 -- 172.21.15.169:0/1553384203 >> 172.21.15.169:0/1553384203 conn(0x7f27200aeee0 msgr2=0x7f2720155aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:39:43.147 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:43.142+0000 7f27265d0640 1 -- 172.21.15.169:0/1553384203 shutdown_connections 2023-10-26T01:39:43.147 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:39:43.142+0000 7f27265d0640 1 -- 172.21.15.169:0/1553384203 wait complete. 2023-10-26T01:39:43.167 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2023-10-26T01:39:43.398 INFO:teuthology.orchestra.run.smithi169.stdout:{"osd_mon_report_interval":"5"} 2023-10-26T01:39:43.399 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2023-10-26T01:39:43.399 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mon_tick_interval 2023-10-26T01:39:43.618 INFO:teuthology.orchestra.run.smithi169.stdout:{"mon_tick_interval":"5"} 2023-10-26T01:39:43.618 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mon_tick_interval": "5" } 2023-10-26T01:39:43.619 INFO:tasks.cephfs.test_full:pool capacity 33554432, 32MB should be enough to fill it 2023-10-26T01:39:43.619 DEBUG:teuthology.orchestra.run.smithi169:> stdin-killer -- python3 -c ' 2023-10-26T01:39:43.619 DEBUG:teuthology.orchestra.run.smithi169:> import time 2023-10-26T01:39:43.620 DEBUG:teuthology.orchestra.run.smithi169:> import datetime 2023-10-26T01:39:43.620 DEBUG:teuthology.orchestra.run.smithi169:> import subprocess 2023-10-26T01:39:43.620 DEBUG:teuthology.orchestra.run.smithi169:> import os 2023-10-26T01:39:43.621 DEBUG:teuthology.orchestra.run.smithi169:> 2023-10-26T01:39:43.621 DEBUG:teuthology.orchestra.run.smithi169:> # Write some buffered data through before going full, all should be well 2023-10-26T01:39:43.621 DEBUG:teuthology.orchestra.run.smithi169:> print("writing some data through which we expect to succeed") 2023-10-26T01:39:43.622 DEBUG:teuthology.orchestra.run.smithi169:> bytes = 0 2023-10-26T01:39:43.622 DEBUG:teuthology.orchestra.run.smithi169:> f = os.open("/home/ubuntu/cephtest/mnt.0/full_test_file", os.O_WRONLY | os.O_CREAT) 2023-10-26T01:39:43.622 DEBUG:teuthology.orchestra.run.smithi169:> os.setxattr("/home/ubuntu/cephtest/mnt.0/full_test_file", '"'"'ceph.file.layout'"'"', b'"'"'stripe_unit=1048576 stripe_count=1 object_size=1048576'"'"') 2023-10-26T01:39:43.623 DEBUG:teuthology.orchestra.run.smithi169:> bytes += os.write(f, b'"'"'a'"'"' * 512 * 1024) 2023-10-26T01:39:43.623 DEBUG:teuthology.orchestra.run.smithi169:> os.fsync(f) 2023-10-26T01:39:43.623 DEBUG:teuthology.orchestra.run.smithi169:> print("fsync'"'"'ed data successfully, will now attempt to fill fs") 2023-10-26T01:39:43.624 DEBUG:teuthology.orchestra.run.smithi169:> 2023-10-26T01:39:43.624 DEBUG:teuthology.orchestra.run.smithi169:> # Okay, now we'"'"'re going to fill up the filesystem, and then keep 2023-10-26T01:39:43.625 DEBUG:teuthology.orchestra.run.smithi169:> # writing until we see an error from fsync. As long as we'"'"'re doing 2023-10-26T01:39:43.625 DEBUG:teuthology.orchestra.run.smithi169:> # buffered IO, the error should always only appear from fsync and not 2023-10-26T01:39:43.625 DEBUG:teuthology.orchestra.run.smithi169:> # from write 2023-10-26T01:39:43.626 DEBUG:teuthology.orchestra.run.smithi169:> full = False 2023-10-26T01:39:43.626 DEBUG:teuthology.orchestra.run.smithi169:> 2023-10-26T01:39:43.626 DEBUG:teuthology.orchestra.run.smithi169:> for n in range(0, int(32 * 0.9)): 2023-10-26T01:39:43.627 DEBUG:teuthology.orchestra.run.smithi169:> bytes += os.write(f, b'"'"'x'"'"' * 1024 * 1024) 2023-10-26T01:39:43.627 DEBUG:teuthology.orchestra.run.smithi169:> print("wrote {0} bytes via buffered write, may repeat".format(bytes)) 2023-10-26T01:39:43.627 DEBUG:teuthology.orchestra.run.smithi169:> print("done writing {0} bytes".format(bytes)) 2023-10-26T01:39:43.628 DEBUG:teuthology.orchestra.run.smithi169:> 2023-10-26T01:39:43.628 DEBUG:teuthology.orchestra.run.smithi169:> # OK, now we should sneak in under the full condition 2023-10-26T01:39:43.628 DEBUG:teuthology.orchestra.run.smithi169:> # due to the time it takes the OSDs to report to the 2023-10-26T01:39:43.629 DEBUG:teuthology.orchestra.run.smithi169:> # mons, and get a successful fsync on our full-making data 2023-10-26T01:39:43.629 DEBUG:teuthology.orchestra.run.smithi169:> os.fsync(f) 2023-10-26T01:39:43.629 DEBUG:teuthology.orchestra.run.smithi169:> print("successfully fsync'"'"'ed prior to getting full state reported") 2023-10-26T01:39:43.630 DEBUG:teuthology.orchestra.run.smithi169:> 2023-10-26T01:39:43.630 DEBUG:teuthology.orchestra.run.smithi169:> # buffered write, add more dirty data to the buffer 2023-10-26T01:39:43.631 DEBUG:teuthology.orchestra.run.smithi169:> print("starting buffered write") 2023-10-26T01:39:43.631 DEBUG:teuthology.orchestra.run.smithi169:> try: 2023-10-26T01:39:43.631 DEBUG:teuthology.orchestra.run.smithi169:> for n in range(0, int(32 * 0.2)): 2023-10-26T01:39:43.631 DEBUG:teuthology.orchestra.run.smithi169:> bytes += os.write(f, b'"'"'x'"'"' * 1024 * 1024) 2023-10-26T01:39:43.632 DEBUG:teuthology.orchestra.run.smithi169:> print("sleeping a bit as we'"'"'ve exceeded 90% of our expected full ratio") 2023-10-26T01:39:43.632 DEBUG:teuthology.orchestra.run.smithi169:> time.sleep(15.0) 2023-10-26T01:39:43.632 DEBUG:teuthology.orchestra.run.smithi169:> except OSError: 2023-10-26T01:39:43.633 DEBUG:teuthology.orchestra.run.smithi169:> pass; 2023-10-26T01:39:43.633 DEBUG:teuthology.orchestra.run.smithi169:> 2023-10-26T01:39:43.633 DEBUG:teuthology.orchestra.run.smithi169:> print("wrote, now waiting 30s and then doing a close we expect to fail") 2023-10-26T01:39:43.634 DEBUG:teuthology.orchestra.run.smithi169:> 2023-10-26T01:39:43.634 DEBUG:teuthology.orchestra.run.smithi169:> # Wait long enough for a background flush that should fail 2023-10-26T01:39:43.634 DEBUG:teuthology.orchestra.run.smithi169:> time.sleep(30) 2023-10-26T01:39:43.635 DEBUG:teuthology.orchestra.run.smithi169:> 2023-10-26T01:39:43.635 DEBUG:teuthology.orchestra.run.smithi169:> if False: 2023-10-26T01:39:43.636 DEBUG:teuthology.orchestra.run.smithi169:> # ...and check that the failed background flush is reflected in fclose 2023-10-26T01:39:43.636 DEBUG:teuthology.orchestra.run.smithi169:> try: 2023-10-26T01:39:43.636 DEBUG:teuthology.orchestra.run.smithi169:> os.close(f) 2023-10-26T01:39:43.637 DEBUG:teuthology.orchestra.run.smithi169:> except OSError: 2023-10-26T01:39:43.637 DEBUG:teuthology.orchestra.run.smithi169:> print("close() returned an error as expected") 2023-10-26T01:39:43.637 DEBUG:teuthology.orchestra.run.smithi169:> else: 2023-10-26T01:39:43.638 DEBUG:teuthology.orchestra.run.smithi169:> raise RuntimeError("close() failed to raise error") 2023-10-26T01:39:43.638 DEBUG:teuthology.orchestra.run.smithi169:> else: 2023-10-26T01:39:43.638 DEBUG:teuthology.orchestra.run.smithi169:> # The kernel cephfs client does not raise errors on fclose 2023-10-26T01:39:43.639 DEBUG:teuthology.orchestra.run.smithi169:> os.close(f) 2023-10-26T01:39:43.639 DEBUG:teuthology.orchestra.run.smithi169:> 2023-10-26T01:39:43.639 DEBUG:teuthology.orchestra.run.smithi169:> os.unlink("/home/ubuntu/cephtest/mnt.0/full_test_file") 2023-10-26T01:39:43.640 DEBUG:teuthology.orchestra.run.smithi169:> ' 2023-10-26T01:40:06.943 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T01:40:06.954 DEBUG:teuthology.orchestra.run.smithi170:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T01:40:37.090 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T01:40:37.101 DEBUG:teuthology.orchestra.run.smithi170:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T01:41:07.252 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T01:41:07.263 DEBUG:teuthology.orchestra.run.smithi170:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T01:41:37.409 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T01:41:37.419 DEBUG:teuthology.orchestra.run.smithi170:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T01:41:45.707 INFO:teuthology.orchestra.run.smithi169.stdout:writing some data through which we expect to succeed 2023-10-26T01:41:45.707 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed data successfully, will now attempt to fill fs 2023-10-26T01:41:45.708 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 1572864 bytes via buffered write, may repeat 2023-10-26T01:41:45.708 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 2621440 bytes via buffered write, may repeat 2023-10-26T01:41:45.708 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 3670016 bytes via buffered write, may repeat 2023-10-26T01:41:45.708 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 4718592 bytes via buffered write, may repeat 2023-10-26T01:41:45.708 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 5767168 bytes via buffered write, may repeat 2023-10-26T01:41:45.708 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 6815744 bytes via buffered write, may repeat 2023-10-26T01:41:45.709 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 7864320 bytes via buffered write, may repeat 2023-10-26T01:41:45.709 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 8912896 bytes via buffered write, may repeat 2023-10-26T01:41:45.709 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 9961472 bytes via buffered write, may repeat 2023-10-26T01:41:45.709 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 11010048 bytes via buffered write, may repeat 2023-10-26T01:41:45.709 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 12058624 bytes via buffered write, may repeat 2023-10-26T01:41:45.710 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 13107200 bytes via buffered write, may repeat 2023-10-26T01:41:45.710 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 14155776 bytes via buffered write, may repeat 2023-10-26T01:41:45.710 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 15204352 bytes via buffered write, may repeat 2023-10-26T01:41:45.710 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 16252928 bytes via buffered write, may repeat 2023-10-26T01:41:45.710 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 17301504 bytes via buffered write, may repeat 2023-10-26T01:41:45.711 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 18350080 bytes via buffered write, may repeat 2023-10-26T01:41:45.711 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 19398656 bytes via buffered write, may repeat 2023-10-26T01:41:45.711 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 20447232 bytes via buffered write, may repeat 2023-10-26T01:41:45.711 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 21495808 bytes via buffered write, may repeat 2023-10-26T01:41:45.711 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 22544384 bytes via buffered write, may repeat 2023-10-26T01:41:45.712 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 23592960 bytes via buffered write, may repeat 2023-10-26T01:41:45.712 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 24641536 bytes via buffered write, may repeat 2023-10-26T01:41:45.712 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 25690112 bytes via buffered write, may repeat 2023-10-26T01:41:45.712 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 26738688 bytes via buffered write, may repeat 2023-10-26T01:41:45.712 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 27787264 bytes via buffered write, may repeat 2023-10-26T01:41:45.713 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 28835840 bytes via buffered write, may repeat 2023-10-26T01:41:45.713 INFO:teuthology.orchestra.run.smithi169.stdout:wrote 29884416 bytes via buffered write, may repeat 2023-10-26T01:41:45.713 INFO:teuthology.orchestra.run.smithi169.stdout:done writing 29884416 bytes 2023-10-26T01:41:45.713 INFO:teuthology.orchestra.run.smithi169.stdout:successfully fsync'ed prior to getting full state reported 2023-10-26T01:41:45.713 INFO:teuthology.orchestra.run.smithi169.stdout:starting buffered write 2023-10-26T01:41:45.713 INFO:teuthology.orchestra.run.smithi169.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2023-10-26T01:41:45.714 INFO:teuthology.orchestra.run.smithi169.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2023-10-26T01:41:45.714 INFO:teuthology.orchestra.run.smithi169.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2023-10-26T01:41:45.714 INFO:teuthology.orchestra.run.smithi169.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2023-10-26T01:41:45.714 INFO:teuthology.orchestra.run.smithi169.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2023-10-26T01:41:45.714 INFO:teuthology.orchestra.run.smithi169.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2023-10-26T01:41:45.715 INFO:teuthology.orchestra.run.smithi169.stdout:wrote, now waiting 30s and then doing a close we expect to fail 2023-10-26T01:41:45.720 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T01:41:45.741 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-26T01:41:45.741 DEBUG:teuthology.orchestra.run.smithi169:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-26T01:41:45.864 DEBUG:teuthology.orchestra.run.smithi170:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-26T01:41:45.993 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-26T01:41:45.993 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:41:45.994 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:41:46.055 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:41:46.055 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:41:46.108 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:41:46.329 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.327+0000 7fedfaa6f640 1 -- 172.21.15.169:0/4108896058 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fede4003970 msgr2=0x7fede4003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:46.329 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.327+0000 7fedfaa6f640 1 --2- 172.21.15.169:0/4108896058 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fede4003970 0x7fede4003d90 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7fede800bbe0 tx=0x7fede40057f0 comp rx=0 tx=0).stop 2023-10-26T01:41:46.329 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.327+0000 7fedf900c640 1 -- 172.21.15.169:0/4108896058 reap_dead start 2023-10-26T01:41:46.329 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.327+0000 7fedfaa6f640 1 -- 172.21.15.169:0/4108896058 shutdown_connections 2023-10-26T01:41:46.330 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.327+0000 7fedfaa6f640 1 -- 172.21.15.169:0/4108896058 >> 172.21.15.169:0/4108896058 conn(0x7fedf4069930 msgr2=0x7fedf4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:41:46.330 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.327+0000 7fedfaa6f640 1 -- 172.21.15.169:0/4108896058 shutdown_connections 2023-10-26T01:41:46.330 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.327+0000 7fedfaa6f640 1 -- 172.21.15.169:0/4108896058 wait complete. 2023-10-26T01:41:46.331 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.329+0000 7fedfaa6f640 1 Processor -- start 2023-10-26T01:41:46.331 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.329+0000 7fedfaa6f640 1 -- start start 2023-10-26T01:41:46.333 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.330+0000 7fedfaa6f640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fede4003970 0x7fedf415ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:41:46.333 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.330+0000 7fedfaa6f640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fedf40c9470 0x7fedf415b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:41:46.333 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.330+0000 7fedfaa6f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fedf41f20d0 0x7fedf41f2550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:41:46.334 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.330+0000 7fedfaa6f640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedf40c8fe0 con 0x7fedf40c9470 2023-10-26T01:41:46.334 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.330+0000 7fedfaa6f640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedf40c6ec0 con 0x7fedf41f20d0 2023-10-26T01:41:46.334 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.330+0000 7fedfaa6f640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedf40c86b0 con 0x7fede4003970 2023-10-26T01:41:46.334 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.330+0000 7fedf3fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fedf40c9470 0x7fedf415b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:41:46.334 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.330+0000 7fedf3fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fedf40c9470 0x7fedf415b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:38186/0 (socket says 172.21.15.169:38186) 2023-10-26T01:41:46.335 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.330+0000 7fedf3fff640 1 -- 172.21.15.169:0/3182043952 learned_addr learned my addr 172.21.15.169:0/3182043952 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:41:46.335 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.330+0000 7fedf900c640 1 --2- 172.21.15.169:0/3182043952 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fedf41f20d0 0x7fedf41f2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:41:46.335 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.330+0000 7fedf880b640 1 --2- 172.21.15.169:0/3182043952 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fede4003970 0x7fedf415ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:41:46.335 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.331+0000 7fedf3fff640 1 -- 172.21.15.169:0/3182043952 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fede4003970 msgr2=0x7fedf415ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:46.335 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.331+0000 7fedf3fff640 1 --2- 172.21.15.169:0/3182043952 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fede4003970 0x7fedf415ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:46.336 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.331+0000 7fedf3fff640 1 -- 172.21.15.169:0/3182043952 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fedf41f20d0 msgr2=0x7fedf41f2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:46.336 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.331+0000 7fedf3fff640 1 --2- 172.21.15.169:0/3182043952 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fedf41f20d0 0x7fedf41f2550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:46.336 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.331+0000 7fedf3fff640 1 -- 172.21.15.169:0/3182043952 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedf40c8990 con 0x7fedf40c9470 2023-10-26T01:41:46.336 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.331+0000 7fedf900c640 1 --2- 172.21.15.169:0/3182043952 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fedf41f20d0 0x7fedf41f2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:41:46.336 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.331+0000 7fedf3fff640 1 --2- 172.21.15.169:0/3182043952 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fedf40c9470 0x7fedf415b3d0 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7fede000d730 tx=0x7fede000dbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:41:46.337 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.331+0000 7fedf17fa640 1 -- 172.21.15.169:0/3182043952 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fede000de40 con 0x7fedf40c9470 2023-10-26T01:41:46.337 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.331+0000 7fedf17fa640 1 -- 172.21.15.169:0/3182043952 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fede00052e0 con 0x7fedf40c9470 2023-10-26T01:41:46.337 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.331+0000 7fedfaa6f640 1 -- 172.21.15.169:0/3182043952 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedf40c8c70 con 0x7fedf40c9470 2023-10-26T01:41:46.337 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.331+0000 7fedf17fa640 1 -- 172.21.15.169:0/3182043952 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fede000b920 con 0x7fedf40c9470 2023-10-26T01:41:46.337 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.331+0000 7fedfaa6f640 1 -- 172.21.15.169:0/3182043952 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedf41f2a90 con 0x7fedf40c9470 2023-10-26T01:41:46.338 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.332+0000 7fedfaa6f640 1 -- 172.21.15.169:0/3182043952 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedf404db10 con 0x7fedf40c9470 2023-10-26T01:41:46.339 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.333+0000 7fedf17fa640 1 -- 172.21.15.169:0/3182043952 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fede000ba80 con 0x7fedf40c9470 2023-10-26T01:41:46.339 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.335+0000 7fedf17fa640 1 --2- 172.21.15.169:0/3182043952 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fedd806ae70 0x7fedd806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:41:46.339 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.335+0000 7fedf17fa640 1 -- 172.21.15.169:0/3182043952 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fede0089400 con 0x7fedf40c9470 2023-10-26T01:41:46.339 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.335+0000 7fedf17fa640 1 -- 172.21.15.169:0/3182043952 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=156}) v3 -- 0x7fedd8072a60 con 0x7fedf40c9470 2023-10-26T01:41:46.340 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.335+0000 7fedf880b640 1 --2- 172.21.15.169:0/3182043952 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fedd806ae70 0x7fedd806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:41:46.340 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.336+0000 7fedf880b640 1 --2- 172.21.15.169:0/3182043952 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fedd806ae70 0x7fedd806d330 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fede8036000 tx=0x7fede802e040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:41:46.341 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.340+0000 7fedf17fa640 1 -- 172.21.15.169:0/3182043952 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fede00543d0 con 0x7fedf40c9470 2023-10-26T01:41:46.537 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.535+0000 7fedfaa6f640 1 -- 172.21.15.169:0/3182043952 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fedf415c430 con 0x7fedf40c9470 2023-10-26T01:41:46.539 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.538+0000 7fedf17fa640 1 -- 172.21.15.169:0/3182043952 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v155) v1 ==== 74+0+16695 (secure 0 0 0) 0x7fede0019030 con 0x7fedf40c9470 2023-10-26T01:41:46.540 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:41:46.541 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":155,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:41:08.113301+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-10-26T01:39:23.646624+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"150","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-10-26T01:39:24.813678+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"19.5","mappings":[{"from":2,"to":0}]},{"pgid":"19.13","mappings":[{"from":2,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:41:46.542 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.540+0000 7fedfaa6f640 1 -- 172.21.15.169:0/3182043952 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fedd806ae70 msgr2=0x7fedd806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:46.542 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.540+0000 7fedfaa6f640 1 --2- 172.21.15.169:0/3182043952 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fedd806ae70 0x7fedd806d330 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fede8036000 tx=0x7fede802e040 comp rx=0 tx=0).stop 2023-10-26T01:41:46.542 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.540+0000 7fedfaa6f640 1 -- 172.21.15.169:0/3182043952 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fedf40c9470 msgr2=0x7fedf415b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:46.543 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.540+0000 7fedfaa6f640 1 --2- 172.21.15.169:0/3182043952 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fedf40c9470 0x7fedf415b3d0 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7fede000d730 tx=0x7fede000dbf0 comp rx=0 tx=0).stop 2023-10-26T01:41:46.543 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.540+0000 7fedfaa6f640 1 -- 172.21.15.169:0/3182043952 shutdown_connections 2023-10-26T01:41:46.543 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.540+0000 7fedfaa6f640 1 --2- 172.21.15.169:0/3182043952 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fedd806ae70 0x7fedd806d330 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:46.543 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.540+0000 7fedfaa6f640 1 --2- 172.21.15.169:0/3182043952 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fedf40c9470 0x7fedf415b3d0 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:46.543 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.540+0000 7fedfaa6f640 1 --2- 172.21.15.169:0/3182043952 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fedf41f20d0 0x7fedf41f2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:46.544 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.540+0000 7fedfaa6f640 1 --2- 172.21.15.169:0/3182043952 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fede4003970 0x7fedf415ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:46.544 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.540+0000 7fedfaa6f640 1 -- 172.21.15.169:0/3182043952 >> 172.21.15.169:0/3182043952 conn(0x7fedf4069930 msgr2=0x7fedf40c9930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:41:46.544 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.540+0000 7fedfaa6f640 1 -- 172.21.15.169:0/3182043952 shutdown_connections 2023-10-26T01:41:46.544 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.540+0000 7fedfaa6f640 1 -- 172.21.15.169:0/3182043952 wait complete. 2023-10-26T01:41:46.561 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T01:30:32.200808+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '109', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-26T01:39:23.646624+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '150', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_data', 'create_time': '2023-10-26T01:39:24.813678+0000', 'flags': 1027, 'flags_names': 'hashpspool,full,full_quota', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '155', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T01:41:46.562 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T01:41:46.778 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.775+0000 7f51bf86b640 1 -- 172.21.15.169:0/819399575 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f51b80b0820 msgr2=0x7f51b80aaf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:46.778 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.775+0000 7f51bf86b640 1 --2- 172.21.15.169:0/819399575 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f51b80b0820 0x7f51b80aaf80 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f51b80b20a0 tx=0x7f51a802bd90 comp rx=0 tx=0).stop 2023-10-26T01:41:46.778 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.776+0000 7f51bf86b640 1 -- 172.21.15.169:0/819399575 shutdown_connections 2023-10-26T01:41:46.779 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.776+0000 7f51bf86b640 1 --2- 172.21.15.169:0/819399575 >> v2:172.21.15.170:3300/0 conn(0x7f51b80ab4c0 0x7f51b80cd690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:46.779 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.776+0000 7f51bf86b640 1 --2- 172.21.15.169:0/819399575 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f51b80b0820 0x7f51b80aaf80 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:46.779 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.776+0000 7f51bf86b640 1 --2- 172.21.15.169:0/819399575 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f51b8159480 0x7f51b80b02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:46.779 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.776+0000 7f51bf86b640 1 -- 172.21.15.169:0/819399575 >> 172.21.15.169:0/819399575 conn(0x7f51b8069930 msgr2=0x7f51b8069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:41:46.779 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.776+0000 7f51bf86b640 1 -- 172.21.15.169:0/819399575 shutdown_connections 2023-10-26T01:41:46.780 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.776+0000 7f51bf86b640 1 -- 172.21.15.169:0/819399575 wait complete. 2023-10-26T01:41:46.781 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.777+0000 7f51bf86b640 1 Processor -- start 2023-10-26T01:41:46.782 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.778+0000 7f51bf86b640 1 -- start start 2023-10-26T01:41:46.782 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.778+0000 7f51bf86b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f51b80ab4c0 0x7f51b81f4980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:41:46.782 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.779+0000 7f51bf86b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f51b80b0820 0x7f51b81e5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:41:46.782 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.779+0000 7f51bf86b640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f51b8159480 0x7f51b81e59b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:41:46.783 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.779+0000 7f51bf86b640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51b81f4ec0 con 0x7f51b80ab4c0 2023-10-26T01:41:46.783 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.779+0000 7f51bf86b640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51b806f310 con 0x7f51b80b0820 2023-10-26T01:41:46.783 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.779+0000 7f51bf86b640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51b81f28b0 con 0x7f51b8159480 2023-10-26T01:41:46.783 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.779+0000 7f51bd607640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f51b80ab4c0 0x7f51b81f4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:41:46.783 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.779+0000 7f51bd607640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f51b80ab4c0 0x7f51b81f4980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:38218/0 (socket says 172.21.15.169:38218) 2023-10-26T01:41:46.784 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.779+0000 7f51bd607640 1 -- 172.21.15.169:0/2831017111 learned_addr learned my addr 172.21.15.169:0/2831017111 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:41:46.784 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.779+0000 7f51bce06640 1 --2- 172.21.15.169:0/2831017111 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f51b80b0820 0x7f51b81e5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:41:46.784 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.779+0000 7f51bde08640 1 --2- 172.21.15.169:0/2831017111 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f51b8159480 0x7f51b81e59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:41:46.784 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.779+0000 7f51bd607640 1 -- 172.21.15.169:0/2831017111 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f51b8159480 msgr2=0x7f51b81e59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:46.784 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.779+0000 7f51bd607640 1 --2- 172.21.15.169:0/2831017111 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f51b8159480 0x7f51b81e59b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:46.785 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.779+0000 7f51bd607640 1 -- 172.21.15.169:0/2831017111 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f51b80b0820 msgr2=0x7f51b81e5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:46.785 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.779+0000 7f51bd607640 1 --2- 172.21.15.169:0/2831017111 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f51b80b0820 0x7f51b81e5470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:46.785 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.779+0000 7f51bd607640 1 -- 172.21.15.169:0/2831017111 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51a8009560 con 0x7f51b80ab4c0 2023-10-26T01:41:46.785 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.780+0000 7f51bd607640 1 --2- 172.21.15.169:0/2831017111 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f51b80ab4c0 0x7f51b81f4980 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f51b400c970 tx=0x7f51b400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:41:46.785 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.780+0000 7f51adffb640 1 -- 172.21.15.169:0/2831017111 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b4007bb0 con 0x7f51b80ab4c0 2023-10-26T01:41:46.786 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.780+0000 7f51adffb640 1 -- 172.21.15.169:0/2831017111 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f51b4007e20 con 0x7f51b80ab4c0 2023-10-26T01:41:46.786 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.780+0000 7f51bf86b640 1 -- 172.21.15.169:0/2831017111 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51b81f2b90 con 0x7f51b80ab4c0 2023-10-26T01:41:46.786 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.780+0000 7f51adffb640 1 -- 172.21.15.169:0/2831017111 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b4005280 con 0x7f51b80ab4c0 2023-10-26T01:41:46.786 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.780+0000 7f51bf86b640 1 -- 172.21.15.169:0/2831017111 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51b806efa0 con 0x7f51b80ab4c0 2023-10-26T01:41:46.787 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.781+0000 7f51bf86b640 1 -- 172.21.15.169:0/2831017111 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51b8066af0 con 0x7f51b80ab4c0 2023-10-26T01:41:46.788 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.782+0000 7f51adffb640 1 -- 172.21.15.169:0/2831017111 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f51b40053e0 con 0x7f51b80ab4c0 2023-10-26T01:41:46.788 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.783+0000 7f51adffb640 1 --2- 172.21.15.169:0/2831017111 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f519c06ae40 0x7f519c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:41:46.788 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.783+0000 7f51adffb640 1 -- 172.21.15.169:0/2831017111 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f51b4089be0 con 0x7f51b80ab4c0 2023-10-26T01:41:46.788 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.783+0000 7f51adffb640 1 -- 172.21.15.169:0/2831017111 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=156}) v3 -- 0x7f519c072a30 con 0x7f51b80ab4c0 2023-10-26T01:41:46.788 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.784+0000 7f51bce06640 1 --2- 172.21.15.169:0/2831017111 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f519c06ae40 0x7f519c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:41:46.789 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.785+0000 7f51bce06640 1 --2- 172.21.15.169:0/2831017111 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f519c06ae40 0x7f519c06d300 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f51a8035000 tx=0x7f51a8004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:41:46.789 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.788+0000 7f51adffb640 1 -- 172.21.15.169:0/2831017111 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f51b401d030 con 0x7f51b80ab4c0 2023-10-26T01:41:46.977 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.976+0000 7f51bf86b640 1 -- 172.21.15.169:0/2831017111 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f51b804da70 con 0x7f51b80ab4c0 2023-10-26T01:41:46.978 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.977+0000 7f51adffb640 1 -- 172.21.15.169:0/2831017111 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v155) v1 ==== 93+0+10 (secure 0 0 0) 0x7f51b401d030 con 0x7f51b80ab4c0 2023-10-26T01:41:46.979 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 1 2023-10-26T01:41:46.981 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.979+0000 7f51bf86b640 1 -- 172.21.15.169:0/2831017111 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f519c06ae40 msgr2=0x7f519c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:46.981 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.979+0000 7f51bf86b640 1 --2- 172.21.15.169:0/2831017111 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f519c06ae40 0x7f519c06d300 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f51a8035000 tx=0x7f51a8004730 comp rx=0 tx=0).stop 2023-10-26T01:41:46.981 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.979+0000 7f51bf86b640 1 -- 172.21.15.169:0/2831017111 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f51b80ab4c0 msgr2=0x7f51b81f4980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:46.982 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.979+0000 7f51bf86b640 1 --2- 172.21.15.169:0/2831017111 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f51b80ab4c0 0x7f51b81f4980 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f51b400c970 tx=0x7f51b400ce30 comp rx=0 tx=0).stop 2023-10-26T01:41:46.982 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.979+0000 7f51bf86b640 1 -- 172.21.15.169:0/2831017111 shutdown_connections 2023-10-26T01:41:46.982 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.979+0000 7f51bf86b640 1 --2- 172.21.15.169:0/2831017111 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f519c06ae40 0x7f519c06d300 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:46.982 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.979+0000 7f51bf86b640 1 --2- 172.21.15.169:0/2831017111 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f51b8159480 0x7f51b81e59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:46.982 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.979+0000 7f51bf86b640 1 --2- 172.21.15.169:0/2831017111 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f51b80b0820 0x7f51b81e5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:46.983 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.979+0000 7f51bf86b640 1 --2- 172.21.15.169:0/2831017111 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f51b80ab4c0 0x7f51b81f4980 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:46.983 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.979+0000 7f51bf86b640 1 -- 172.21.15.169:0/2831017111 >> 172.21.15.169:0/2831017111 conn(0x7f51b8069930 msgr2=0x7f51b80af760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:41:46.983 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.979+0000 7f51bf86b640 1 -- 172.21.15.169:0/2831017111 shutdown_connections 2023-10-26T01:41:46.983 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:46.980+0000 7f51bf86b640 1 -- 172.21.15.169:0/2831017111 wait complete. 2023-10-26T01:41:46.999 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-26T01:41:47.211 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:47.209+0000 7f276cd6e640 1 -- 172.21.15.169:0/2220918441 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27681593e0 msgr2=0x7f27680ac200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:47.211 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:47.209+0000 7f276cd6e640 1 --2- 172.21.15.169:0/2220918441 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27681593e0 0x7f27680ac200 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f27680b2670 tx=0x7f275802be40 comp rx=0 tx=0).stop 2023-10-26T01:41:47.211 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:47.209+0000 7f276cd6e640 1 -- 172.21.15.169:0/2220918441 shutdown_connections 2023-10-26T01:41:47.211 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:47.209+0000 7f276cd6e640 1 --2- 172.21.15.169:0/2220918441 >> v2:172.21.15.170:3300/0 conn(0x7f27681597c0 0x7f276815dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:47.212 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:47.209+0000 7f276cd6e640 1 --2- 172.21.15.169:0/2220918441 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27681593e0 0x7f27680ac200 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:47.212 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:47.209+0000 7f276cd6e640 1 -- 172.21.15.169:0/2220918441 >> 172.21.15.169:0/2220918441 conn(0x7f2768069930 msgr2=0x7f2768069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:41:47.212 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:47.209+0000 7f276cd6e640 1 -- 172.21.15.169:0/2220918441 shutdown_connections 2023-10-26T01:41:47.212 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:47.209+0000 7f276cd6e640 1 -- 172.21.15.169:0/2220918441 wait complete. 2023-10-26T01:41:47.213 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:47.211+0000 7f276cd6e640 1 Processor -- start 2023-10-26T01:41:47.213 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:47.211+0000 7f276cd6e640 1 -- start start 2023-10-26T01:41:47.215 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:47.212+0000 7f276cd6e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27680b0c80 0x7f2768071960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:41:47.215 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:47.212+0000 7f276cd6e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f27681593e0 0x7f27681444d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:41:47.215 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:47.212+0000 7f276cd6e640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f27681597c0 0x7f27681ef810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:41:47.215 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:47.212+0000 7f276cd6e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27680ae2e0 con 0x7f27680b0c80 2023-10-26T01:41:47.215 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:47.212+0000 7f276cd6e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27680ae5c0 con 0x7f27681593e0 2023-10-26T01:41:47.216 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:47.212+0000 7f276cd6e640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27680ae8a0 con 0x7f27681597c0 2023-10-26T01:41:47.216 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:47.212+0000 7f276659c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27680b0c80 0x7f2768071960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:41:47.216 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:47.212+0000 7f276659c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27680b0c80 0x7f2768071960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:38228/0 (socket says 172.21.15.169:38228) 2023-10-26T01:41:47.216 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:47.212+0000 7f276659c640 1 -- 172.21.15.169:0/3488893571 learned_addr learned my addr 172.21.15.169:0/3488893571 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:41:47.216 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:47.213+0000 7f276659c640 1 -- 172.21.15.169:0/3488893571 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f27681597c0 msgr2=0x7f27681ef810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:47.217 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:47.213+0000 7f276659c640 1 --2- 172.21.15.169:0/3488893571 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f27681597c0 0x7f27681ef810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:47.217 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:47.213+0000 7f276659c640 1 -- 172.21.15.169:0/3488893571 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f27681593e0 msgr2=0x7f27681444d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:47.217 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:47.213+0000 7f275dd9b640 1 --2- 172.21.15.169:0/3488893571 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f27681593e0 0x7f27681444d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:41:47.217 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:47.213+0000 7f276659c640 1 --2- 172.21.15.169:0/3488893571 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f27681593e0 0x7f27681444d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:47.217 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:47.213+0000 7f276659c640 1 -- 172.21.15.169:0/3488893571 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2758009560 con 0x7f27680b0c80 2023-10-26T01:41:47.217 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:47.213+0000 7f276659c640 1 --2- 172.21.15.169:0/3488893571 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27680b0c80 0x7f2768071960 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f2758009850 tx=0x7f275802db70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:41:47.219 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:47.213+0000 7f275f7fe640 1 -- 172.21.15.169:0/3488893571 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2758003800 con 0x7f27680b0c80 2023-10-26T01:41:47.219 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:47.213+0000 7f275f7fe640 1 -- 172.21.15.169:0/3488893571 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2758003a50 con 0x7f27680b0c80 2023-10-26T01:41:47.219 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:47.214+0000 7f276cd6e640 1 -- 172.21.15.169:0/3488893571 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27680aeb80 con 0x7f27680b0c80 2023-10-26T01:41:47.219 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:47.214+0000 7f275f7fe640 1 -- 172.21.15.169:0/3488893571 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2758003e20 con 0x7f27680b0c80 2023-10-26T01:41:47.220 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:47.214+0000 7f276cd6e640 1 -- 172.21.15.169:0/3488893571 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2768071ea0 con 0x7f27680b0c80 2023-10-26T01:41:47.220 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:47.215+0000 7f276cd6e640 1 -- 172.21.15.169:0/3488893571 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27680af040 con 0x7f27680b0c80 2023-10-26T01:41:47.220 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:47.215+0000 7f275f7fe640 1 -- 172.21.15.169:0/3488893571 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f275803e450 con 0x7f27680b0c80 2023-10-26T01:41:47.220 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:47.216+0000 7f275f7fe640 1 --2- 172.21.15.169:0/3488893571 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f274806ae40 0x7f274806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:41:47.220 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:47.216+0000 7f275f7fe640 1 -- 172.21.15.169:0/3488893571 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f27580ac190 con 0x7f27680b0c80 2023-10-26T01:41:47.220 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:47.217+0000 7f275f7fe640 1 -- 172.21.15.169:0/3488893571 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=156}) v3 -- 0x7f2748072a30 con 0x7f27680b0c80 2023-10-26T01:41:47.221 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:47.217+0000 7f275dd9b640 1 --2- 172.21.15.169:0/3488893571 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f274806ae40 0x7f274806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:41:47.221 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:47.218+0000 7f275dd9b640 1 --2- 172.21.15.169:0/3488893571 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f274806ae40 0x7f274806d300 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f27500058c0 tx=0x7f2750005850 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:41:47.223 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:47.221+0000 7f275f7fe640 1 -- 172.21.15.169:0/3488893571 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f27580771f0 con 0x7f27680b0c80 2023-10-26T01:41:47.415 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:47.413+0000 7f276cd6e640 1 -- 172.21.15.169:0/3488893571 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f27680ce090 con 0x7f27680b0c80 2023-10-26T01:41:47.415 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:47.414+0000 7f275f7fe640 1 -- 172.21.15.169:0/3488893571 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v155) v1 ==== 104+0+11 (secure 0 0 0) 0x7f27580420a0 con 0x7f27680b0c80 2023-10-26T01:41:47.416 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 32 2023-10-26T01:41:47.418 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:47.416+0000 7f276cd6e640 1 -- 172.21.15.169:0/3488893571 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f274806ae40 msgr2=0x7f274806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:47.418 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:47.416+0000 7f276cd6e640 1 --2- 172.21.15.169:0/3488893571 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f274806ae40 0x7f274806d300 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f27500058c0 tx=0x7f2750005850 comp rx=0 tx=0).stop 2023-10-26T01:41:47.419 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:47.416+0000 7f276cd6e640 1 -- 172.21.15.169:0/3488893571 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27680b0c80 msgr2=0x7f2768071960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:47.419 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:47.416+0000 7f276cd6e640 1 --2- 172.21.15.169:0/3488893571 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27680b0c80 0x7f2768071960 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f2758009850 tx=0x7f275802db70 comp rx=0 tx=0).stop 2023-10-26T01:41:47.419 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:47.416+0000 7f276cd6e640 1 -- 172.21.15.169:0/3488893571 shutdown_connections 2023-10-26T01:41:47.419 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:47.416+0000 7f276cd6e640 1 --2- 172.21.15.169:0/3488893571 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f274806ae40 0x7f274806d300 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:47.419 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:47.416+0000 7f276cd6e640 1 --2- 172.21.15.169:0/3488893571 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f27681597c0 0x7f27681ef810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:47.419 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:47.416+0000 7f276cd6e640 1 --2- 172.21.15.169:0/3488893571 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f27681593e0 0x7f27681444d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:47.420 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:47.416+0000 7f276cd6e640 1 --2- 172.21.15.169:0/3488893571 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27680b0c80 0x7f2768071960 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:47.420 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:47.416+0000 7f276cd6e640 1 -- 172.21.15.169:0/3488893571 >> 172.21.15.169:0/3488893571 conn(0x7f2768069930 msgr2=0x7f27680aa720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:41:47.420 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:47.416+0000 7f276cd6e640 1 -- 172.21.15.169:0/3488893571 shutdown_connections 2023-10-26T01:41:47.420 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:47.417+0000 7f276cd6e640 1 -- 172.21.15.169:0/3488893571 wait complete. 2023-10-26T01:41:47.436 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-26T01:41:47.653 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:47.651+0000 7fd64404f640 1 -- 172.21.15.169:0/4205864658 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6280047c0 msgr2=0x7fd628004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:47.654 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:47.651+0000 7fd64404f640 1 --2- 172.21.15.169:0/4205864658 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6280047c0 0x7fd628004c40 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7fd63c159c30 tx=0x7fd6280057f0 comp rx=0 tx=0).stop 2023-10-26T01:41:47.654 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:47.651+0000 7fd64404f640 1 -- 172.21.15.169:0/4205864658 shutdown_connections 2023-10-26T01:41:47.654 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:47.652+0000 7fd64404f640 1 --2- 172.21.15.169:0/4205864658 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6280047c0 0x7fd628004c40 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:47.654 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:47.652+0000 7fd64404f640 1 -- 172.21.15.169:0/4205864658 >> 172.21.15.169:0/4205864658 conn(0x7fd63c069930 msgr2=0x7fd63c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:41:47.655 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:47.652+0000 7fd64404f640 1 -- 172.21.15.169:0/4205864658 shutdown_connections 2023-10-26T01:41:47.655 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:47.652+0000 7fd64404f640 1 -- 172.21.15.169:0/4205864658 wait complete. 2023-10-26T01:41:47.655 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:47.653+0000 7fd64404f640 1 Processor -- start 2023-10-26T01:41:47.656 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:47.654+0000 7fd64404f640 1 -- start start 2023-10-26T01:41:47.658 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:47.654+0000 7fd64404f640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd6280047c0 0x7fd63c142670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:41:47.658 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:47.655+0000 7fd64404f640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd63c0adf50 0x7fd63c0ae3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:41:47.658 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:47.655+0000 7fd64404f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd63c15b3d0 0x7fd63c15b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:41:47.658 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:47.655+0000 7fd64404f640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd63c15bd90 con 0x7fd63c0adf50 2023-10-26T01:41:47.659 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:47.655+0000 7fd64404f640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd63c0aeac0 con 0x7fd63c15b3d0 2023-10-26T01:41:47.659 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:47.655+0000 7fd64404f640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd63c0aeda0 con 0x7fd6280047c0 2023-10-26T01:41:47.659 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:47.655+0000 7fd6415ea640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd63c0adf50 0x7fd63c0ae3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:41:47.659 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:47.655+0000 7fd6415ea640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd63c0adf50 0x7fd63c0ae3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:38246/0 (socket says 172.21.15.169:38246) 2023-10-26T01:41:47.659 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:47.655+0000 7fd6415ea640 1 -- 172.21.15.169:0/373310380 learned_addr learned my addr 172.21.15.169:0/373310380 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:41:47.660 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:47.655+0000 7fd641deb640 1 --2- 172.21.15.169:0/373310380 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd6280047c0 0x7fd63c142670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:41:47.660 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:47.655+0000 7fd6425ec640 1 --2- 172.21.15.169:0/373310380 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd63c15b3d0 0x7fd63c15b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:41:47.660 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:47.655+0000 7fd6415ea640 1 -- 172.21.15.169:0/373310380 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd6280047c0 msgr2=0x7fd63c142670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:47.660 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:47.655+0000 7fd6415ea640 1 --2- 172.21.15.169:0/373310380 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd6280047c0 0x7fd63c142670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:47.660 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:47.655+0000 7fd6415ea640 1 -- 172.21.15.169:0/373310380 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd63c15b3d0 msgr2=0x7fd63c15b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:47.661 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:47.655+0000 7fd6415ea640 1 --2- 172.21.15.169:0/373310380 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd63c15b3d0 0x7fd63c15b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:47.661 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:47.655+0000 7fd6415ea640 1 -- 172.21.15.169:0/373310380 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd63c1558a0 con 0x7fd63c0adf50 2023-10-26T01:41:47.661 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:47.655+0000 7fd6425ec640 1 --2- 172.21.15.169:0/373310380 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd63c15b3d0 0x7fd63c15b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:41:47.661 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:47.656+0000 7fd6415ea640 1 --2- 172.21.15.169:0/373310380 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd63c0adf50 0x7fd63c0ae3d0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7fd62c001930 tx=0x7fd62c0065b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:41:47.661 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:47.656+0000 7fd6327fc640 1 -- 172.21.15.169:0/373310380 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd62c004a90 con 0x7fd63c0adf50 2023-10-26T01:41:47.662 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:47.656+0000 7fd6327fc640 1 -- 172.21.15.169:0/373310380 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd62c004ce0 con 0x7fd63c0adf50 2023-10-26T01:41:47.662 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:47.656+0000 7fd6327fc640 1 -- 172.21.15.169:0/373310380 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd62c0356d0 con 0x7fd63c0adf50 2023-10-26T01:41:47.662 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:47.656+0000 7fd64404f640 1 -- 172.21.15.169:0/373310380 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd63c155b80 con 0x7fd63c0adf50 2023-10-26T01:41:47.662 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:47.656+0000 7fd64404f640 1 -- 172.21.15.169:0/373310380 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd63c0afa70 con 0x7fd63c0adf50 2023-10-26T01:41:47.663 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:47.657+0000 7fd64404f640 1 -- 172.21.15.169:0/373310380 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd63c04db10 con 0x7fd63c0adf50 2023-10-26T01:41:47.664 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:47.658+0000 7fd6327fc640 1 -- 172.21.15.169:0/373310380 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fd62c032070 con 0x7fd63c0adf50 2023-10-26T01:41:47.664 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:47.659+0000 7fd6327fc640 1 --2- 172.21.15.169:0/373310380 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd62006ae40 0x7fd62006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:41:47.664 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:47.659+0000 7fd6327fc640 1 -- 172.21.15.169:0/373310380 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fd62c0aaf90 con 0x7fd63c0adf50 2023-10-26T01:41:47.664 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:47.659+0000 7fd6327fc640 1 -- 172.21.15.169:0/373310380 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=156}) v3 -- 0x7fd620072a30 con 0x7fd63c0adf50 2023-10-26T01:41:47.664 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:47.660+0000 7fd641deb640 1 --2- 172.21.15.169:0/373310380 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd62006ae40 0x7fd62006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:41:47.665 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:47.661+0000 7fd641deb640 1 --2- 172.21.15.169:0/373310380 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd62006ae40 0x7fd62006d300 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fd638000c00 tx=0x7fd638006b30 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:41:47.665 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:47.664+0000 7fd6327fc640 1 -- 172.21.15.169:0/373310380 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd62c075ff0 con 0x7fd63c0adf50 2023-10-26T01:41:47.854 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:47.852+0000 7fd64404f640 1 -- 172.21.15.169:0/373310380 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd63c0b2170 con 0x7fd63c0adf50 2023-10-26T01:41:47.854 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:47.853+0000 7fd6327fc640 1 -- 172.21.15.169:0/373310380 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v155) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd62c079610 con 0x7fd63c0adf50 2023-10-26T01:41:47.855 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 64 2023-10-26T01:41:47.857 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:47.855+0000 7fd64404f640 1 -- 172.21.15.169:0/373310380 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd62006ae40 msgr2=0x7fd62006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:47.858 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:47.855+0000 7fd64404f640 1 --2- 172.21.15.169:0/373310380 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd62006ae40 0x7fd62006d300 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fd638000c00 tx=0x7fd638006b30 comp rx=0 tx=0).stop 2023-10-26T01:41:47.858 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:47.855+0000 7fd64404f640 1 -- 172.21.15.169:0/373310380 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd63c0adf50 msgr2=0x7fd63c0ae3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:47.858 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:47.855+0000 7fd64404f640 1 --2- 172.21.15.169:0/373310380 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd63c0adf50 0x7fd63c0ae3d0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7fd62c001930 tx=0x7fd62c0065b0 comp rx=0 tx=0).stop 2023-10-26T01:41:47.858 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:47.856+0000 7fd64404f640 1 -- 172.21.15.169:0/373310380 shutdown_connections 2023-10-26T01:41:47.859 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:47.856+0000 7fd64404f640 1 --2- 172.21.15.169:0/373310380 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd62006ae40 0x7fd62006d300 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:47.859 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:47.856+0000 7fd64404f640 1 --2- 172.21.15.169:0/373310380 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd63c0adf50 0x7fd63c0ae3d0 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:47.859 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:47.856+0000 7fd64404f640 1 --2- 172.21.15.169:0/373310380 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd63c15b3d0 0x7fd63c15b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:47.859 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:47.856+0000 7fd64404f640 1 --2- 172.21.15.169:0/373310380 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd6280047c0 0x7fd63c142670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:47.859 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:47.856+0000 7fd64404f640 1 -- 172.21.15.169:0/373310380 >> 172.21.15.169:0/373310380 conn(0x7fd63c069930 msgr2=0x7fd63c151860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:41:47.860 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:47.856+0000 7fd64404f640 1 -- 172.21.15.169:0/373310380 shutdown_connections 2023-10-26T01:41:47.860 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:47.856+0000 7fd64404f640 1 -- 172.21.15.169:0/373310380 wait complete. 2023-10-26T01:41:47.876 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-26T01:41:48.096 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:48.094+0000 7fa243fff640 1 -- 172.21.15.169:0/1423593742 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa244156c50 msgr2=0x7fa2440699c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:48.096 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:48.094+0000 7fa243fff640 1 --2- 172.21.15.169:0/1423593742 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa244156c50 0x7fa2440699c0 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7fa244071f30 tx=0x7fa23402bcf0 comp rx=0 tx=0).stop 2023-10-26T01:41:48.097 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:48.094+0000 7fa243fff640 1 -- 172.21.15.169:0/1423593742 shutdown_connections 2023-10-26T01:41:48.097 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:48.094+0000 7fa243fff640 1 --2- 172.21.15.169:0/1423593742 >> v2:172.21.15.170:3300/0 conn(0x7fa244157030 0x7fa244069f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:48.097 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:48.094+0000 7fa243fff640 1 --2- 172.21.15.169:0/1423593742 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa244156c50 0x7fa2440699c0 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:48.097 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:48.094+0000 7fa243fff640 1 -- 172.21.15.169:0/1423593742 >> 172.21.15.169:0/1423593742 conn(0x7fa2440af110 msgr2=0x7fa2440af520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:41:48.097 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:48.094+0000 7fa243fff640 1 -- 172.21.15.169:0/1423593742 shutdown_connections 2023-10-26T01:41:48.097 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:48.094+0000 7fa243fff640 1 -- 172.21.15.169:0/1423593742 wait complete. 2023-10-26T01:41:48.098 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:48.096+0000 7fa243fff640 1 Processor -- start 2023-10-26T01:41:48.098 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:48.096+0000 7fa243fff640 1 -- start start 2023-10-26T01:41:48.100 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:48.097+0000 7fa243fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2440ac740 0x7fa24415bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:41:48.100 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:48.097+0000 7fa243fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa244156c50 0x7fa2441f1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:41:48.100 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:48.097+0000 7fa243fff640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa244157030 0x7fa2440c6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:41:48.100 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:48.097+0000 7fa243fff640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2441f2c80 con 0x7fa2440ac740 2023-10-26T01:41:48.100 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:48.097+0000 7fa243fff640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2441f26c0 con 0x7fa244156c50 2023-10-26T01:41:48.101 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:48.097+0000 7fa243fff640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2441f29a0 con 0x7fa244157030 2023-10-26T01:41:48.101 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:48.098+0000 7fa242ffd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2440ac740 0x7fa24415bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:41:48.101 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:48.098+0000 7fa242ffd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2440ac740 0x7fa24415bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:38264/0 (socket says 172.21.15.169:38264) 2023-10-26T01:41:48.101 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:48.098+0000 7fa242ffd640 1 -- 172.21.15.169:0/1117268091 learned_addr learned my addr 172.21.15.169:0/1117268091 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:41:48.101 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:48.098+0000 7fa23a7fc640 1 --2- 172.21.15.169:0/1117268091 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa244156c50 0x7fa2441f1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:41:48.102 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:48.098+0000 7fa242ffd640 1 -- 172.21.15.169:0/1117268091 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa244157030 msgr2=0x7fa2440c6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:48.102 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:48.098+0000 7fa242ffd640 1 --2- 172.21.15.169:0/1117268091 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa244157030 0x7fa2440c6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:48.102 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:48.098+0000 7fa242ffd640 1 -- 172.21.15.169:0/1117268091 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa244156c50 msgr2=0x7fa2441f1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:48.102 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:48.098+0000 7fa242ffd640 1 --2- 172.21.15.169:0/1117268091 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa244156c50 0x7fa2441f1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:48.102 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:48.098+0000 7fa242ffd640 1 -- 172.21.15.169:0/1117268091 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa234009490 con 0x7fa2440ac740 2023-10-26T01:41:48.103 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:48.098+0000 7fa242ffd640 1 --2- 172.21.15.169:0/1117268091 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2440ac740 0x7fa24415bf30 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7fa234009780 tx=0x7fa2340038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:41:48.104 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:48.099+0000 7fa23bfff640 1 -- 172.21.15.169:0/1117268091 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa234003ea0 con 0x7fa2440ac740 2023-10-26T01:41:48.104 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:48.099+0000 7fa23bfff640 1 -- 172.21.15.169:0/1117268091 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa234005020 con 0x7fa2440ac740 2023-10-26T01:41:48.104 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:48.099+0000 7fa243fff640 1 -- 172.21.15.169:0/1117268091 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2440c64a0 con 0x7fa2440ac740 2023-10-26T01:41:48.105 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:48.099+0000 7fa23bfff640 1 -- 172.21.15.169:0/1117268091 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2340053f0 con 0x7fa2440ac740 2023-10-26T01:41:48.105 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:48.099+0000 7fa243fff640 1 -- 172.21.15.169:0/1117268091 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa24415c470 con 0x7fa2440ac740 2023-10-26T01:41:48.105 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:48.100+0000 7fa243fff640 1 -- 172.21.15.169:0/1117268091 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2440c69c0 con 0x7fa2440ac740 2023-10-26T01:41:48.105 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:48.101+0000 7fa23bfff640 1 -- 172.21.15.169:0/1117268091 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fa2340047a0 con 0x7fa2440ac740 2023-10-26T01:41:48.105 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:48.102+0000 7fa23bfff640 1 --2- 172.21.15.169:0/1117268091 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa22406ae40 0x7fa22406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:41:48.106 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:48.102+0000 7fa23bfff640 1 -- 172.21.15.169:0/1117268091 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fa2340abb60 con 0x7fa2440ac740 2023-10-26T01:41:48.106 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:48.102+0000 7fa23bfff640 1 -- 172.21.15.169:0/1117268091 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=156}) v3 -- 0x7fa224072a30 con 0x7fa2440ac740 2023-10-26T01:41:48.106 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:48.102+0000 7fa23a7fc640 1 --2- 172.21.15.169:0/1117268091 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa22406ae40 0x7fa22406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:41:48.107 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:48.103+0000 7fa23a7fc640 1 --2- 172.21.15.169:0/1117268091 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa22406ae40 0x7fa22406d300 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fa22c005fd0 tx=0x7fa22c005850 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:41:48.108 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:48.106+0000 7fa23bfff640 1 -- 172.21.15.169:0/1117268091 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa234039060 con 0x7fa2440ac740 2023-10-26T01:41:48.316 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:48.315+0000 7fa243fff640 1 -- 172.21.15.169:0/1117268091 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fa244073c10 con 0x7fa2440ac740 2023-10-26T01:41:48.317 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:48.315+0000 7fa23bfff640 1 -- 172.21.15.169:0/1117268091 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v155) v1 ==== 80+0+0 (secure 0 0 0) 0x7fa234042070 con 0x7fa2440ac740 2023-10-26T01:41:48.317 INFO:teuthology.orchestra.run.smithi169.stderr:listed 0 entries 2023-10-26T01:41:48.321 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:48.319+0000 7fa243fff640 1 -- 172.21.15.169:0/1117268091 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa22406ae40 msgr2=0x7fa22406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:48.321 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:48.319+0000 7fa243fff640 1 --2- 172.21.15.169:0/1117268091 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa22406ae40 0x7fa22406d300 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fa22c005fd0 tx=0x7fa22c005850 comp rx=0 tx=0).stop 2023-10-26T01:41:48.321 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:48.319+0000 7fa243fff640 1 -- 172.21.15.169:0/1117268091 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2440ac740 msgr2=0x7fa24415bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:48.321 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:48.319+0000 7fa243fff640 1 --2- 172.21.15.169:0/1117268091 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2440ac740 0x7fa24415bf30 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7fa234009780 tx=0x7fa2340038a0 comp rx=0 tx=0).stop 2023-10-26T01:41:48.321 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:48.319+0000 7fa243fff640 1 -- 172.21.15.169:0/1117268091 shutdown_connections 2023-10-26T01:41:48.322 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:48.319+0000 7fa243fff640 1 --2- 172.21.15.169:0/1117268091 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa22406ae40 0x7fa22406d300 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:48.322 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:48.319+0000 7fa243fff640 1 --2- 172.21.15.169:0/1117268091 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa244157030 0x7fa2440c6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:48.322 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:48.319+0000 7fa243fff640 1 --2- 172.21.15.169:0/1117268091 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa244156c50 0x7fa2441f1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:48.322 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:48.319+0000 7fa243fff640 1 --2- 172.21.15.169:0/1117268091 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2440ac740 0x7fa24415bf30 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:48.322 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:48.319+0000 7fa243fff640 1 -- 172.21.15.169:0/1117268091 >> 172.21.15.169:0/1117268091 conn(0x7fa2440af110 msgr2=0x7fa244069b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:41:48.323 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:48.319+0000 7fa243fff640 1 -- 172.21.15.169:0/1117268091 shutdown_connections 2023-10-26T01:41:48.323 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:48.319+0000 7fa243fff640 1 -- 172.21.15.169:0/1117268091 wait complete. 2023-10-26T01:41:48.340 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-26T01:41:48.340 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:41:48.340 DEBUG:teuthology.orchestra.run.smithi169:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-26T01:41:48.445 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi169.front.sepia.ceph.com 2023-10-26T01:41:48.445 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:41:48.446 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-26T01:41:48.500 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-26T01:41:48.500 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:41:48.559 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-26T01:41:48.559 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:41:48.616 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:41:48.831 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:48.828+0000 7fdf5d200640 1 -- 172.21.15.169:0/1147240178 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdf580ceb70 msgr2=0x7fdf580d0030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:48.831 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:48.828+0000 7fdf5d200640 1 --2- 172.21.15.169:0/1147240178 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdf580ceb70 0x7fdf580d0030 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7fdf50005fd0 tx=0x7fdf5002bd90 comp rx=0 tx=0).stop 2023-10-26T01:41:48.831 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:48.829+0000 7fdf5d200640 1 -- 172.21.15.169:0/1147240178 shutdown_connections 2023-10-26T01:41:48.832 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:48.829+0000 7fdf5d200640 1 --2- 172.21.15.169:0/1147240178 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdf580ceb70 0x7fdf580d0030 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:48.832 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:48.829+0000 7fdf5d200640 1 --2- 172.21.15.169:0/1147240178 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdf5815f610 0x7fdf580ce630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:48.832 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:48.829+0000 7fdf5d200640 1 -- 172.21.15.169:0/1147240178 >> 172.21.15.169:0/1147240178 conn(0x7fdf58069e00 msgr2=0x7fdf5806a210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:41:48.832 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:48.829+0000 7fdf5d200640 1 -- 172.21.15.169:0/1147240178 shutdown_connections 2023-10-26T01:41:48.833 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:48.829+0000 7fdf5d200640 1 -- 172.21.15.169:0/1147240178 wait complete. 2023-10-26T01:41:48.835 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:48.830+0000 7fdf5d200640 1 Processor -- start 2023-10-26T01:41:48.835 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:48.831+0000 7fdf5d200640 1 -- start start 2023-10-26T01:41:48.835 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:48.831+0000 7fdf5d200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdf580ceb70 0x7fdf580b3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:41:48.836 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:48.831+0000 7fdf5d200640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdf5815b9d0 0x7fdf58157670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:41:48.836 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:48.831+0000 7fdf5d200640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdf5815f610 0x7fdf580ca960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:41:48.836 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:48.831+0000 7fdf5d200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf5815cfc0 con 0x7fdf580ceb70 2023-10-26T01:41:48.836 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:48.831+0000 7fdf5d200640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf580b4070 con 0x7fdf5815b9d0 2023-10-26T01:41:48.837 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:48.831+0000 7fdf5d200640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf58157bb0 con 0x7fdf5815f610 2023-10-26T01:41:48.837 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:48.832+0000 7fdf56d9d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdf580ceb70 0x7fdf580b3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:41:48.837 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:48.832+0000 7fdf56d9d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdf580ceb70 0x7fdf580b3b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:38284/0 (socket says 172.21.15.169:38284) 2023-10-26T01:41:48.837 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:48.832+0000 7fdf56d9d640 1 -- 172.21.15.169:0/1716151557 learned_addr learned my addr 172.21.15.169:0/1716151557 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:41:48.837 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:48.832+0000 7fdf5759e640 1 --2- 172.21.15.169:0/1716151557 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdf5815f610 0x7fdf580ca960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:41:48.838 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:48.832+0000 7fdf5659c640 1 --2- 172.21.15.169:0/1716151557 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdf5815b9d0 0x7fdf58157670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:41:48.838 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:48.832+0000 7fdf56d9d640 1 -- 172.21.15.169:0/1716151557 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdf5815f610 msgr2=0x7fdf580ca960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:48.838 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:48.832+0000 7fdf56d9d640 1 --2- 172.21.15.169:0/1716151557 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdf5815f610 0x7fdf580ca960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:48.838 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:48.832+0000 7fdf56d9d640 1 -- 172.21.15.169:0/1716151557 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdf5815b9d0 msgr2=0x7fdf58157670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:48.838 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:48.832+0000 7fdf56d9d640 1 --2- 172.21.15.169:0/1716151557 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdf5815b9d0 0x7fdf58157670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:48.839 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:48.832+0000 7fdf56d9d640 1 -- 172.21.15.169:0/1716151557 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf50009560 con 0x7fdf580ceb70 2023-10-26T01:41:48.839 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:48.833+0000 7fdf56d9d640 1 --2- 172.21.15.169:0/1716151557 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdf580ceb70 0x7fdf580b3b30 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7fdf4000c440 tx=0x7fdf4000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:41:48.839 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:48.833+0000 7fdf4f7fe640 1 -- 172.21.15.169:0/1716151557 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf40005480 con 0x7fdf580ceb70 2023-10-26T01:41:48.839 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:48.833+0000 7fdf4f7fe640 1 -- 172.21.15.169:0/1716151557 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fdf400035f0 con 0x7fdf580ceb70 2023-10-26T01:41:48.839 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:48.833+0000 7fdf4f7fe640 1 -- 172.21.15.169:0/1716151557 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf40003a20 con 0x7fdf580ceb70 2023-10-26T01:41:48.840 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:48.833+0000 7fdf5d200640 1 -- 172.21.15.169:0/1716151557 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf5815cce0 con 0x7fdf580ceb70 2023-10-26T01:41:48.840 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:48.833+0000 7fdf5d200640 1 -- 172.21.15.169:0/1716151557 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf5815c4e0 con 0x7fdf580ceb70 2023-10-26T01:41:48.841 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:48.834+0000 7fdf5d200640 1 -- 172.21.15.169:0/1716151557 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf5815bff0 con 0x7fdf580ceb70 2023-10-26T01:41:48.841 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:48.835+0000 7fdf4f7fe640 1 -- 172.21.15.169:0/1716151557 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fdf40003b80 con 0x7fdf580ceb70 2023-10-26T01:41:48.841 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:48.836+0000 7fdf4f7fe640 1 --2- 172.21.15.169:0/1716151557 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fdf3806ae40 0x7fdf3806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:41:48.842 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:48.836+0000 7fdf4f7fe640 1 -- 172.21.15.169:0/1716151557 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fdf40088b70 con 0x7fdf580ceb70 2023-10-26T01:41:48.842 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:48.836+0000 7fdf4f7fe640 1 -- 172.21.15.169:0/1716151557 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=156}) v3 -- 0x7fdf38072a30 con 0x7fdf580ceb70 2023-10-26T01:41:48.842 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:48.837+0000 7fdf5659c640 1 --2- 172.21.15.169:0/1716151557 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fdf3806ae40 0x7fdf3806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:41:48.842 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:48.838+0000 7fdf5659c640 1 --2- 172.21.15.169:0/1716151557 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fdf3806ae40 0x7fdf3806d300 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fdf5815fab0 tx=0x7fdf480072f0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:41:48.843 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:48.841+0000 7fdf4f7fe640 1 -- 172.21.15.169:0/1716151557 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdf40017070 con 0x7fdf580ceb70 2023-10-26T01:41:49.033 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.032+0000 7fdf5d200640 1 -- 172.21.15.169:0/1716151557 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdf581554d0 con 0x7fdf580ceb70 2023-10-26T01:41:49.035 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.034+0000 7fdf4f7fe640 1 -- 172.21.15.169:0/1716151557 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v155) v1 ==== 74+0+16695 (secure 0 0 0) 0x7fdf40053b40 con 0x7fdf580ceb70 2023-10-26T01:41:49.036 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:41:49.036 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":155,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:41:08.113301+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-10-26T01:39:23.646624+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"150","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-10-26T01:39:24.813678+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"19.5","mappings":[{"from":2,"to":0}]},{"pgid":"19.13","mappings":[{"from":2,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:41:49.039 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.037+0000 7fdf5d200640 1 -- 172.21.15.169:0/1716151557 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fdf3806ae40 msgr2=0x7fdf3806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:49.039 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.037+0000 7fdf5d200640 1 --2- 172.21.15.169:0/1716151557 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fdf3806ae40 0x7fdf3806d300 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fdf5815fab0 tx=0x7fdf480072f0 comp rx=0 tx=0).stop 2023-10-26T01:41:49.039 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.037+0000 7fdf5d200640 1 -- 172.21.15.169:0/1716151557 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdf580ceb70 msgr2=0x7fdf580b3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:49.039 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.037+0000 7fdf5d200640 1 --2- 172.21.15.169:0/1716151557 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdf580ceb70 0x7fdf580b3b30 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7fdf4000c440 tx=0x7fdf4000c900 comp rx=0 tx=0).stop 2023-10-26T01:41:49.040 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.037+0000 7fdf5d200640 1 -- 172.21.15.169:0/1716151557 shutdown_connections 2023-10-26T01:41:49.040 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.037+0000 7fdf5d200640 1 --2- 172.21.15.169:0/1716151557 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fdf3806ae40 0x7fdf3806d300 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:49.040 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.037+0000 7fdf5d200640 1 --2- 172.21.15.169:0/1716151557 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdf5815f610 0x7fdf580ca960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:49.040 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.037+0000 7fdf5d200640 1 --2- 172.21.15.169:0/1716151557 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdf5815b9d0 0x7fdf58157670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:49.040 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.037+0000 7fdf5d200640 1 --2- 172.21.15.169:0/1716151557 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdf580ceb70 0x7fdf580b3b30 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:49.041 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.037+0000 7fdf5d200640 1 -- 172.21.15.169:0/1716151557 >> 172.21.15.169:0/1716151557 conn(0x7fdf58069e00 msgr2=0x7fdf58158a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:41:49.041 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.037+0000 7fdf5d200640 1 -- 172.21.15.169:0/1716151557 shutdown_connections 2023-10-26T01:41:49.041 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.037+0000 7fdf5d200640 1 -- 172.21.15.169:0/1716151557 wait complete. 2023-10-26T01:41:49.058 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T01:30:32.200808+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '109', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-26T01:39:23.646624+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '150', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_data', 'create_time': '2023-10-26T01:39:24.813678+0000', 'flags': 1027, 'flags_names': 'hashpspool,full,full_quota', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '155', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T01:41:49.059 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T01:41:49.286 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.284+0000 7f464907c640 1 -- 172.21.15.169:0/578541901 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f46440aaf80 msgr2=0x7f46440ab3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:49.287 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.284+0000 7f464907c640 1 --2- 172.21.15.169:0/578541901 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f46440aaf80 0x7f46440ab3c0 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f462c005fd0 tx=0x7f462c02bd90 comp rx=0 tx=0).stop 2023-10-26T01:41:49.287 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.284+0000 7f464907c640 1 -- 172.21.15.169:0/578541901 shutdown_connections 2023-10-26T01:41:49.287 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.284+0000 7f464907c640 1 --2- 172.21.15.169:0/578541901 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f46440aaf80 0x7f46440ab3c0 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:49.287 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.284+0000 7f464907c640 1 -- 172.21.15.169:0/578541901 >> 172.21.15.169:0/578541901 conn(0x7f4644069930 msgr2=0x7f4644069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:41:49.288 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.284+0000 7f464907c640 1 -- 172.21.15.169:0/578541901 shutdown_connections 2023-10-26T01:41:49.288 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.284+0000 7f464907c640 1 -- 172.21.15.169:0/578541901 wait complete. 2023-10-26T01:41:49.290 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.286+0000 7f464907c640 1 Processor -- start 2023-10-26T01:41:49.290 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.286+0000 7f464907c640 1 -- start start 2023-10-26T01:41:49.290 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.287+0000 7f464907c640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f46440aaf80 0x7f46440af090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:41:49.291 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.287+0000 7f464907c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f46440cd690 0x7f4644148590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:41:49.291 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.287+0000 7f464907c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4644159480 0x7f46440c76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:41:49.291 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.287+0000 7f464907c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46440adda0 con 0x7f46440cd690 2023-10-26T01:41:49.291 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.287+0000 7f464907c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46440aec00 con 0x7f4644159480 2023-10-26T01:41:49.292 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.287+0000 7f464907c640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46440adff0 con 0x7f46440aaf80 2023-10-26T01:41:49.292 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.287+0000 7f463bfff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f46440cd690 0x7f4644148590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:41:49.292 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.287+0000 7f463bfff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f46440cd690 0x7f4644148590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:38300/0 (socket says 172.21.15.169:38300) 2023-10-26T01:41:49.292 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.287+0000 7f463bfff640 1 -- 172.21.15.169:0/3667867966 learned_addr learned my addr 172.21.15.169:0/3667867966 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:41:49.293 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.288+0000 7f463bfff640 1 -- 172.21.15.169:0/3667867966 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f46440aaf80 msgr2=0x7f46440af090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:49.293 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.288+0000 7f464359e640 1 --2- 172.21.15.169:0/3667867966 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4644159480 0x7f46440c76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:41:49.293 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.288+0000 7f4642d9d640 1 --2- 172.21.15.169:0/3667867966 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f46440aaf80 0x7f46440af090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:41:49.293 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.288+0000 7f463bfff640 1 --2- 172.21.15.169:0/3667867966 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f46440aaf80 0x7f46440af090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:49.294 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.288+0000 7f463bfff640 1 -- 172.21.15.169:0/3667867966 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4644159480 msgr2=0x7f46440c76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:49.294 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.288+0000 7f463bfff640 1 --2- 172.21.15.169:0/3667867966 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4644159480 0x7f46440c76b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:49.294 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.288+0000 7f463bfff640 1 -- 172.21.15.169:0/3667867966 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f462c009560 con 0x7f46440cd690 2023-10-26T01:41:49.294 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.288+0000 7f464359e640 1 --2- 172.21.15.169:0/3667867966 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4644159480 0x7f46440c76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:41:49.294 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.288+0000 7f463bfff640 1 --2- 172.21.15.169:0/3667867966 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f46440cd690 0x7f4644148590 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f462c036040 tx=0x7f462c02db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:41:49.295 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.288+0000 7f463b7fe640 1 -- 172.21.15.169:0/3667867966 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462c003800 con 0x7f46440cd690 2023-10-26T01:41:49.295 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.288+0000 7f463b7fe640 1 -- 172.21.15.169:0/3667867966 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f462c003a50 con 0x7f46440cd690 2023-10-26T01:41:49.295 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.288+0000 7f463b7fe640 1 -- 172.21.15.169:0/3667867966 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462c003e00 con 0x7f46440cd690 2023-10-26T01:41:49.295 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.288+0000 7f464907c640 1 -- 172.21.15.169:0/3667867966 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46440ae2d0 con 0x7f46440cd690 2023-10-26T01:41:49.296 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.289+0000 7f464907c640 1 -- 172.21.15.169:0/3667867966 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46440ae890 con 0x7f46440cd690 2023-10-26T01:41:49.297 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.290+0000 7f464907c640 1 -- 172.21.15.169:0/3667867966 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4644066af0 con 0x7f46440cd690 2023-10-26T01:41:49.297 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.291+0000 7f463b7fe640 1 -- 172.21.15.169:0/3667867966 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f462c03e450 con 0x7f46440cd690 2023-10-26T01:41:49.297 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.291+0000 7f463b7fe640 1 --2- 172.21.15.169:0/3667867966 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f462406ae40 0x7f462406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:41:49.298 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.292+0000 7f463b7fe640 1 -- 172.21.15.169:0/3667867966 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f462c0ac210 con 0x7f46440cd690 2023-10-26T01:41:49.298 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.292+0000 7f463b7fe640 1 -- 172.21.15.169:0/3667867966 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=156}) v3 -- 0x7f4624072a30 con 0x7f46440cd690 2023-10-26T01:41:49.298 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.292+0000 7f4642d9d640 1 --2- 172.21.15.169:0/3667867966 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f462406ae40 0x7f462406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:41:49.298 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.293+0000 7f4642d9d640 1 --2- 172.21.15.169:0/3667867966 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f462406ae40 0x7f462406d300 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f4634006fd0 tx=0x7f4634008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:41:49.299 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.296+0000 7f463b7fe640 1 -- 172.21.15.169:0/3667867966 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f462c042060 con 0x7f46440cd690 2023-10-26T01:41:49.491 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.489+0000 7f464907c640 1 -- 172.21.15.169:0/3667867966 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f464404da70 con 0x7f46440cd690 2023-10-26T01:41:49.491 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.490+0000 7f463b7fe640 1 -- 172.21.15.169:0/3667867966 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v155) v1 ==== 93+0+10 (secure 0 0 0) 0x7f462c042030 con 0x7f46440cd690 2023-10-26T01:41:49.492 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 1 2023-10-26T01:41:49.494 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.492+0000 7f464907c640 1 -- 172.21.15.169:0/3667867966 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f462406ae40 msgr2=0x7f462406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:49.495 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.492+0000 7f464907c640 1 --2- 172.21.15.169:0/3667867966 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f462406ae40 0x7f462406d300 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f4634006fd0 tx=0x7f4634008040 comp rx=0 tx=0).stop 2023-10-26T01:41:49.495 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.492+0000 7f464907c640 1 -- 172.21.15.169:0/3667867966 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f46440cd690 msgr2=0x7f4644148590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:49.495 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.492+0000 7f464907c640 1 --2- 172.21.15.169:0/3667867966 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f46440cd690 0x7f4644148590 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f462c036040 tx=0x7f462c02db10 comp rx=0 tx=0).stop 2023-10-26T01:41:49.495 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.492+0000 7f464907c640 1 -- 172.21.15.169:0/3667867966 shutdown_connections 2023-10-26T01:41:49.495 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.492+0000 7f464907c640 1 --2- 172.21.15.169:0/3667867966 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f462406ae40 0x7f462406d300 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:49.496 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.492+0000 7f464907c640 1 --2- 172.21.15.169:0/3667867966 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f46440cd690 0x7f4644148590 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:49.496 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.492+0000 7f464907c640 1 --2- 172.21.15.169:0/3667867966 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4644159480 0x7f46440c76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:49.496 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.492+0000 7f464907c640 1 --2- 172.21.15.169:0/3667867966 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f46440aaf80 0x7f46440af090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:49.496 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.492+0000 7f464907c640 1 -- 172.21.15.169:0/3667867966 >> 172.21.15.169:0/3667867966 conn(0x7f4644069930 msgr2=0x7f4644069d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:41:49.496 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.493+0000 7f464907c640 1 -- 172.21.15.169:0/3667867966 shutdown_connections 2023-10-26T01:41:49.497 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.493+0000 7f464907c640 1 -- 172.21.15.169:0/3667867966 wait complete. 2023-10-26T01:41:49.513 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-26T01:41:49.737 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.735+0000 7f3459e10640 1 -- 172.21.15.169:0/3460575951 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34540aabf0 msgr2=0x7f34540ab070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:49.738 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.735+0000 7f3459e10640 1 --2- 172.21.15.169:0/3460575951 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34540aabf0 0x7f34540ab070 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f345406a790 tx=0x7f344c02c1c0 comp rx=0 tx=0).stop 2023-10-26T01:41:49.738 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.736+0000 7f3459e10640 1 -- 172.21.15.169:0/3460575951 shutdown_connections 2023-10-26T01:41:49.738 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.736+0000 7f3459e10640 1 --2- 172.21.15.169:0/3460575951 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34540aabf0 0x7f34540ab070 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:49.739 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.736+0000 7f3459e10640 1 --2- 172.21.15.169:0/3460575951 >> v2:172.21.15.170:3300/0 conn(0x7f345406ce50 0x7f3454158bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:49.739 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.736+0000 7f3459e10640 1 --2- 172.21.15.169:0/3460575951 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3454157be0 0x7f345406b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:49.739 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.736+0000 7f3459e10640 1 -- 172.21.15.169:0/3460575951 >> 172.21.15.169:0/3460575951 conn(0x7f3454069930 msgr2=0x7f3454069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:41:49.739 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.736+0000 7f3459e10640 1 -- 172.21.15.169:0/3460575951 shutdown_connections 2023-10-26T01:41:49.739 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.736+0000 7f3459e10640 1 -- 172.21.15.169:0/3460575951 wait complete. 2023-10-26T01:41:49.741 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.737+0000 7f3459e10640 1 Processor -- start 2023-10-26T01:41:49.742 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.737+0000 7f3459e10640 1 -- start start 2023-10-26T01:41:49.742 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.738+0000 7f3459e10640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f345406ce50 0x7f3454141d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:41:49.742 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.738+0000 7f3459e10640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34540aabf0 0x7f34541424d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:41:49.742 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.738+0000 7f3459e10640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3454157be0 0x7f3454142a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:41:49.742 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.738+0000 7f3459e10640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34540af3b0 con 0x7f34540aabf0 2023-10-26T01:41:49.743 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.738+0000 7f3459e10640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3454159890 con 0x7f345406ce50 2023-10-26T01:41:49.743 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.738+0000 7f3459e10640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34540aecf0 con 0x7f3454157be0 2023-10-26T01:41:49.743 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.739+0000 7f344bfff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34540aabf0 0x7f34541424d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:41:49.743 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.739+0000 7f344bfff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34540aabf0 0x7f34541424d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:38324/0 (socket says 172.21.15.169:38324) 2023-10-26T01:41:49.743 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.739+0000 7f344bfff640 1 -- 172.21.15.169:0/877935709 learned_addr learned my addr 172.21.15.169:0/877935709 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:41:49.744 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.739+0000 7f34537fe640 1 --2- 172.21.15.169:0/877935709 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f345406ce50 0x7f3454141d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:41:49.744 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.739+0000 7f3453fff640 1 --2- 172.21.15.169:0/877935709 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3454157be0 0x7f3454142a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:41:49.744 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.739+0000 7f344bfff640 1 -- 172.21.15.169:0/877935709 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3454157be0 msgr2=0x7f3454142a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:49.744 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.739+0000 7f344bfff640 1 --2- 172.21.15.169:0/877935709 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3454157be0 0x7f3454142a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:49.745 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.739+0000 7f344bfff640 1 -- 172.21.15.169:0/877935709 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f345406ce50 msgr2=0x7f3454141d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:49.745 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.739+0000 7f344bfff640 1 --2- 172.21.15.169:0/877935709 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f345406ce50 0x7f3454141d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:49.745 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.739+0000 7f344bfff640 1 -- 172.21.15.169:0/877935709 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f344c009900 con 0x7f34540aabf0 2023-10-26T01:41:49.745 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.740+0000 7f344bfff640 1 --2- 172.21.15.169:0/877935709 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34540aabf0 0x7f34541424d0 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f343c00c940 tx=0x7f343c00ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:41:49.745 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.740+0000 7f3450ff9640 1 -- 172.21.15.169:0/877935709 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f343c007bb0 con 0x7f34540aabf0 2023-10-26T01:41:49.746 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.740+0000 7f3450ff9640 1 -- 172.21.15.169:0/877935709 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f343c007e00 con 0x7f34540aabf0 2023-10-26T01:41:49.746 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.740+0000 7f3450ff9640 1 -- 172.21.15.169:0/877935709 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f343c005280 con 0x7f34540aabf0 2023-10-26T01:41:49.746 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.740+0000 7f3459e10640 1 -- 172.21.15.169:0/877935709 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34540aefd0 con 0x7f34540aabf0 2023-10-26T01:41:49.746 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.740+0000 7f3459e10640 1 -- 172.21.15.169:0/877935709 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34541430e0 con 0x7f34540aabf0 2023-10-26T01:41:49.747 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.741+0000 7f3459e10640 1 -- 172.21.15.169:0/877935709 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f345404db10 con 0x7f34540aabf0 2023-10-26T01:41:49.747 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.742+0000 7f3450ff9640 1 -- 172.21.15.169:0/877935709 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f343c0054a0 con 0x7f34540aabf0 2023-10-26T01:41:49.748 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.743+0000 7f3450ff9640 1 --2- 172.21.15.169:0/877935709 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f343406ae40 0x7f343406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:41:49.748 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.743+0000 7f3450ff9640 1 -- 172.21.15.169:0/877935709 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f343c089ed0 con 0x7f34540aabf0 2023-10-26T01:41:49.748 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.743+0000 7f3450ff9640 1 -- 172.21.15.169:0/877935709 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=156}) v3 -- 0x7f3434072a30 con 0x7f34540aabf0 2023-10-26T01:41:49.748 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.744+0000 7f34537fe640 1 --2- 172.21.15.169:0/877935709 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f343406ae40 0x7f343406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:41:49.748 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.745+0000 7f34537fe640 1 --2- 172.21.15.169:0/877935709 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f343406ae40 0x7f343406d300 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f3444005fd0 tx=0x7f3444005d80 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:41:49.749 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.748+0000 7f3450ff9640 1 -- 172.21.15.169:0/877935709 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f343c01d070 con 0x7f34540aabf0 2023-10-26T01:41:49.933 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.932+0000 7f3459e10640 1 -- 172.21.15.169:0/877935709 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f34540cc740 con 0x7f34540aabf0 2023-10-26T01:41:49.934 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.933+0000 7f3450ff9640 1 -- 172.21.15.169:0/877935709 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v155) v1 ==== 104+0+11 (secure 0 0 0) 0x7f343c054f30 con 0x7f34540aabf0 2023-10-26T01:41:49.935 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 32 2023-10-26T01:41:49.937 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.935+0000 7f3459e10640 1 -- 172.21.15.169:0/877935709 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f343406ae40 msgr2=0x7f343406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:49.937 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.935+0000 7f3459e10640 1 --2- 172.21.15.169:0/877935709 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f343406ae40 0x7f343406d300 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f3444005fd0 tx=0x7f3444005d80 comp rx=0 tx=0).stop 2023-10-26T01:41:49.938 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.935+0000 7f3459e10640 1 -- 172.21.15.169:0/877935709 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34540aabf0 msgr2=0x7f34541424d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:49.938 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.935+0000 7f3459e10640 1 --2- 172.21.15.169:0/877935709 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34540aabf0 0x7f34541424d0 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f343c00c940 tx=0x7f343c00ce00 comp rx=0 tx=0).stop 2023-10-26T01:41:49.941 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.935+0000 7f3459e10640 1 -- 172.21.15.169:0/877935709 shutdown_connections 2023-10-26T01:41:49.941 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.935+0000 7f3459e10640 1 --2- 172.21.15.169:0/877935709 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f343406ae40 0x7f343406d300 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:49.942 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.935+0000 7f3459e10640 1 --2- 172.21.15.169:0/877935709 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34540aabf0 0x7f34541424d0 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:49.942 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.935+0000 7f3459e10640 1 --2- 172.21.15.169:0/877935709 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3454157be0 0x7f3454142a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:49.942 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.935+0000 7f3459e10640 1 --2- 172.21.15.169:0/877935709 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f345406ce50 0x7f3454141d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:49.942 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.935+0000 7f3459e10640 1 -- 172.21.15.169:0/877935709 >> 172.21.15.169:0/877935709 conn(0x7f3454069930 msgr2=0x7f345415a640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:41:49.942 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.935+0000 7f3459e10640 1 -- 172.21.15.169:0/877935709 shutdown_connections 2023-10-26T01:41:49.943 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:49.935+0000 7f3459e10640 1 -- 172.21.15.169:0/877935709 wait complete. 2023-10-26T01:41:49.956 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-26T01:41:50.165 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.163+0000 7fcfe5135640 1 -- 172.21.15.169:0/4007745271 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcfe01579d0 msgr2=0x7fcfe015b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:50.166 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.163+0000 7fcfe5135640 1 --2- 172.21.15.169:0/4007745271 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcfe01579d0 0x7fcfe015b650 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7fcfe0140f30 tx=0x7fcfd002bd70 comp rx=0 tx=0).stop 2023-10-26T01:41:50.166 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.163+0000 7fcfe5135640 1 -- 172.21.15.169:0/4007745271 shutdown_connections 2023-10-26T01:41:50.166 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.163+0000 7fcfe5135640 1 --2- 172.21.15.169:0/4007745271 >> v2:172.21.15.170:3300/0 conn(0x7fcfe006b9b0 0x7fcfe00afff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:50.166 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.163+0000 7fcfe5135640 1 --2- 172.21.15.169:0/4007745271 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcfe01579d0 0x7fcfe015b650 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:50.166 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.163+0000 7fcfe5135640 1 -- 172.21.15.169:0/4007745271 >> 172.21.15.169:0/4007745271 conn(0x7fcfe00af110 msgr2=0x7fcfe00af520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:41:50.167 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.163+0000 7fcfe5135640 1 -- 172.21.15.169:0/4007745271 shutdown_connections 2023-10-26T01:41:50.167 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.163+0000 7fcfe5135640 1 -- 172.21.15.169:0/4007745271 wait complete. 2023-10-26T01:41:50.168 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.165+0000 7fcfe5135640 1 Processor -- start 2023-10-26T01:41:50.168 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.165+0000 7fcfe5135640 1 -- start start 2023-10-26T01:41:50.168 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.166+0000 7fcfe5135640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcfe006b9b0 0x7fcfe014d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:41:50.168 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.166+0000 7fcfe5135640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcfe01579d0 0x7fcfe00c9260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:41:50.169 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.166+0000 7fcfe5135640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcfe015bb90 0x7fcfe00c97a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:41:50.169 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.166+0000 7fcfe5135640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe0095ce0 con 0x7fcfe015bb90 2023-10-26T01:41:50.169 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.166+0000 7fcfe5135640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe00cbb80 con 0x7fcfe01579d0 2023-10-26T01:41:50.169 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.166+0000 7fcfe5135640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe014de80 con 0x7fcfe006b9b0 2023-10-26T01:41:50.169 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.166+0000 7fcfe4934640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcfe015bb90 0x7fcfe00c97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:41:50.170 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.166+0000 7fcfe4934640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcfe015bb90 0x7fcfe00c97a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:38352/0 (socket says 172.21.15.169:38352) 2023-10-26T01:41:50.170 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.166+0000 7fcfe4934640 1 -- 172.21.15.169:0/506685503 learned_addr learned my addr 172.21.15.169:0/506685503 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:41:50.171 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.167+0000 7fcfe4934640 1 -- 172.21.15.169:0/506685503 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcfe006b9b0 msgr2=0x7fcfe014d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:50.171 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.167+0000 7fcfdffff640 1 --2- 172.21.15.169:0/506685503 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcfe006b9b0 0x7fcfe014d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:41:50.172 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.167+0000 7fcfd77fe640 1 --2- 172.21.15.169:0/506685503 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcfe01579d0 0x7fcfe00c9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:41:50.172 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.167+0000 7fcfe4934640 1 --2- 172.21.15.169:0/506685503 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcfe006b9b0 0x7fcfe014d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:50.172 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.167+0000 7fcfe4934640 1 -- 172.21.15.169:0/506685503 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcfe01579d0 msgr2=0x7fcfe00c9260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:50.172 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.167+0000 7fcfe4934640 1 --2- 172.21.15.169:0/506685503 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcfe01579d0 0x7fcfe00c9260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:50.172 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.167+0000 7fcfe4934640 1 -- 172.21.15.169:0/506685503 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfd0009490 con 0x7fcfe015bb90 2023-10-26T01:41:50.172 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.167+0000 7fcfd77fe640 1 --2- 172.21.15.169:0/506685503 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcfe01579d0 0x7fcfe00c9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:41:50.173 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.167+0000 7fcfe4934640 1 --2- 172.21.15.169:0/506685503 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcfe015bb90 0x7fcfe00c97a0 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7fcfd800e9d0 tx=0x7fcfd800ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:41:50.173 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.167+0000 7fcfdd7fa640 1 -- 172.21.15.169:0/506685503 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfd8004d20 con 0x7fcfe015bb90 2023-10-26T01:41:50.173 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.167+0000 7fcfdd7fa640 1 -- 172.21.15.169:0/506685503 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fcfd800cb20 con 0x7fcfe015bb90 2023-10-26T01:41:50.173 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.167+0000 7fcfdd7fa640 1 -- 172.21.15.169:0/506685503 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfd8005020 con 0x7fcfe015bb90 2023-10-26T01:41:50.174 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.167+0000 7fcfe5135640 1 -- 172.21.15.169:0/506685503 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfe014e120 con 0x7fcfe015bb90 2023-10-26T01:41:50.174 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.168+0000 7fcfe5135640 1 -- 172.21.15.169:0/506685503 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfe0095720 con 0x7fcfe015bb90 2023-10-26T01:41:50.174 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.169+0000 7fcfe5135640 1 -- 172.21.15.169:0/506685503 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfe0073c10 con 0x7fcfe015bb90 2023-10-26T01:41:50.174 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.170+0000 7fcfdd7fa640 1 -- 172.21.15.169:0/506685503 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fcfd8005180 con 0x7fcfe015bb90 2023-10-26T01:41:50.175 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.171+0000 7fcfdd7fa640 1 --2- 172.21.15.169:0/506685503 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcfc006ae70 0x7fcfc006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:41:50.175 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.172+0000 7fcfdd7fa640 1 -- 172.21.15.169:0/506685503 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fcfd8089f70 con 0x7fcfe015bb90 2023-10-26T01:41:50.175 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.172+0000 7fcfdd7fa640 1 -- 172.21.15.169:0/506685503 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=156}) v3 -- 0x7fcfc0072a60 con 0x7fcfe015bb90 2023-10-26T01:41:50.176 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.172+0000 7fcfdffff640 1 --2- 172.21.15.169:0/506685503 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcfc006ae70 0x7fcfc006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:41:50.176 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.173+0000 7fcfdffff640 1 --2- 172.21.15.169:0/506685503 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcfc006ae70 0x7fcfc006d330 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fcfd0036000 tx=0x7fcfd0004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:41:50.178 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.176+0000 7fcfdd7fa640 1 -- 172.21.15.169:0/506685503 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcfd801f020 con 0x7fcfe015bb90 2023-10-26T01:41:50.370 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.369+0000 7fcfe5135640 1 -- 172.21.15.169:0/506685503 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcfe006b080 con 0x7fcfe015bb90 2023-10-26T01:41:50.371 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.369+0000 7fcfdd7fa640 1 -- 172.21.15.169:0/506685503 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v155) v1 ==== 100+0+11 (secure 0 0 0) 0x7fcfd8054fd0 con 0x7fcfe015bb90 2023-10-26T01:41:50.372 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 64 2023-10-26T01:41:50.375 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.372+0000 7fcfe5135640 1 -- 172.21.15.169:0/506685503 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcfc006ae70 msgr2=0x7fcfc006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:50.375 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.372+0000 7fcfe5135640 1 --2- 172.21.15.169:0/506685503 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcfc006ae70 0x7fcfc006d330 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fcfd0036000 tx=0x7fcfd0004730 comp rx=0 tx=0).stop 2023-10-26T01:41:50.375 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.372+0000 7fcfe5135640 1 -- 172.21.15.169:0/506685503 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcfe015bb90 msgr2=0x7fcfe00c97a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:50.375 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.372+0000 7fcfe5135640 1 --2- 172.21.15.169:0/506685503 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcfe015bb90 0x7fcfe00c97a0 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7fcfd800e9d0 tx=0x7fcfd800ee90 comp rx=0 tx=0).stop 2023-10-26T01:41:50.375 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.372+0000 7fcfe5135640 1 -- 172.21.15.169:0/506685503 shutdown_connections 2023-10-26T01:41:50.376 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.372+0000 7fcfe5135640 1 --2- 172.21.15.169:0/506685503 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcfc006ae70 0x7fcfc006d330 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:50.376 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.372+0000 7fcfe5135640 1 --2- 172.21.15.169:0/506685503 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcfe015bb90 0x7fcfe00c97a0 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:50.376 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.372+0000 7fcfe5135640 1 --2- 172.21.15.169:0/506685503 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcfe01579d0 0x7fcfe00c9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:50.376 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.372+0000 7fcfe5135640 1 --2- 172.21.15.169:0/506685503 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcfe006b9b0 0x7fcfe014d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:50.376 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.372+0000 7fcfe5135640 1 -- 172.21.15.169:0/506685503 >> 172.21.15.169:0/506685503 conn(0x7fcfe00af110 msgr2=0x7fcfe00b1500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:41:50.377 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.373+0000 7fcfe5135640 1 -- 172.21.15.169:0/506685503 shutdown_connections 2023-10-26T01:41:50.377 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.373+0000 7fcfe5135640 1 -- 172.21.15.169:0/506685503 wait complete. 2023-10-26T01:41:50.393 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-26T01:41:50.597 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.595+0000 7fe12e667640 1 -- 172.21.15.169:0/2437928167 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe1140039a0 msgr2=0x7fe114003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:50.597 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.595+0000 7fe12e667640 1 --2- 172.21.15.169:0/2437928167 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe1140039a0 0x7fe114003da0 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7fe12400bb20 tx=0x7fe1140057f0 comp rx=0 tx=0).stop 2023-10-26T01:41:50.598 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.595+0000 7fe12de66640 1 -- 172.21.15.169:0/2437928167 reap_dead start 2023-10-26T01:41:50.598 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.595+0000 7fe12e667640 1 -- 172.21.15.169:0/2437928167 shutdown_connections 2023-10-26T01:41:50.598 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.595+0000 7fe12e667640 1 -- 172.21.15.169:0/2437928167 >> 172.21.15.169:0/2437928167 conn(0x7fe1280af110 msgr2=0x7fe1280af520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:41:50.598 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.595+0000 7fe12e667640 1 -- 172.21.15.169:0/2437928167 shutdown_connections 2023-10-26T01:41:50.598 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.595+0000 7fe12e667640 1 -- 172.21.15.169:0/2437928167 wait complete. 2023-10-26T01:41:50.600 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.597+0000 7fe12e667640 1 Processor -- start 2023-10-26T01:41:50.600 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.597+0000 7fe12e667640 1 -- start start 2023-10-26T01:41:50.600 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.598+0000 7fe12e667640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe1140039a0 0x7fe1280c71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:41:50.600 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.598+0000 7fe12e667640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe12806bf90 0x7fe12814e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:41:50.600 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.598+0000 7fe12e667640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe1280c8bf0 0x7fe1280c9070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:41:50.601 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.598+0000 7fe12e667640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1280c76f0 con 0x7fe1280c8bf0 2023-10-26T01:41:50.601 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.598+0000 7fe12e667640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe12806aef0 con 0x7fe1140039a0 2023-10-26T01:41:50.601 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.598+0000 7fe12e667640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe12806b1d0 con 0x7fe12806bf90 2023-10-26T01:41:50.602 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.598+0000 7fe12de66640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe1280c8bf0 0x7fe1280c9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:41:50.602 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.598+0000 7fe12de66640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe1280c8bf0 0x7fe1280c9070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:38370/0 (socket says 172.21.15.169:38370) 2023-10-26T01:41:50.602 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.598+0000 7fe12de66640 1 -- 172.21.15.169:0/3967617358 learned_addr learned my addr 172.21.15.169:0/3967617358 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:41:50.604 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.598+0000 7fe12de66640 1 -- 172.21.15.169:0/3967617358 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe12806bf90 msgr2=0x7fe12814e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:50.605 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.598+0000 7fe12ce64640 1 --2- 172.21.15.169:0/3967617358 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe12806bf90 0x7fe12814e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:41:50.605 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.599+0000 7fe12d665640 1 --2- 172.21.15.169:0/3967617358 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe1140039a0 0x7fe1280c71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:41:50.605 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.599+0000 7fe12de66640 1 --2- 172.21.15.169:0/3967617358 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe12806bf90 0x7fe12814e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:50.605 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.599+0000 7fe12de66640 1 -- 172.21.15.169:0/3967617358 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe1140039a0 msgr2=0x7fe1280c71b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:50.606 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.599+0000 7fe12de66640 1 --2- 172.21.15.169:0/3967617358 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe1140039a0 0x7fe1280c71b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:50.606 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.599+0000 7fe12de66640 1 -- 172.21.15.169:0/3967617358 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe12806b4b0 con 0x7fe1280c8bf0 2023-10-26T01:41:50.606 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.599+0000 7fe12ce64640 1 --2- 172.21.15.169:0/3967617358 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe12806bf90 0x7fe12814e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:41:50.606 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.599+0000 7fe12de66640 1 --2- 172.21.15.169:0/3967617358 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe1280c8bf0 0x7fe1280c9070 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7fe120001330 tx=0x7fe12000df00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:41:50.606 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.599+0000 7fe119ffb640 1 -- 172.21.15.169:0/3967617358 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe12000b8a0 con 0x7fe1280c8bf0 2023-10-26T01:41:50.607 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.599+0000 7fe119ffb640 1 -- 172.21.15.169:0/3967617358 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe12000ba20 con 0x7fe1280c8bf0 2023-10-26T01:41:50.607 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.599+0000 7fe119ffb640 1 -- 172.21.15.169:0/3967617358 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe12000be30 con 0x7fe1280c8bf0 2023-10-26T01:41:50.607 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.599+0000 7fe12e667640 1 -- 172.21.15.169:0/3967617358 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe12806b790 con 0x7fe1280c8bf0 2023-10-26T01:41:50.607 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.599+0000 7fe12e667640 1 -- 172.21.15.169:0/3967617358 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe12806c410 con 0x7fe1280c8bf0 2023-10-26T01:41:50.607 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.600+0000 7fe12e667640 1 -- 172.21.15.169:0/3967617358 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe12806bcb0 con 0x7fe1280c8bf0 2023-10-26T01:41:50.608 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.601+0000 7fe119ffb640 1 -- 172.21.15.169:0/3967617358 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fe12001c460 con 0x7fe1280c8bf0 2023-10-26T01:41:50.608 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.602+0000 7fe119ffb640 1 --2- 172.21.15.169:0/3967617358 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe10c06ae40 0x7fe10c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:41:50.608 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.602+0000 7fe119ffb640 1 -- 172.21.15.169:0/3967617358 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fe120089c30 con 0x7fe1280c8bf0 2023-10-26T01:41:50.608 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.602+0000 7fe119ffb640 1 -- 172.21.15.169:0/3967617358 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=156}) v3 -- 0x7fe10c072a30 con 0x7fe1280c8bf0 2023-10-26T01:41:50.608 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.603+0000 7fe12d665640 1 --2- 172.21.15.169:0/3967617358 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe10c06ae40 0x7fe10c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:41:50.609 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.604+0000 7fe12d665640 1 --2- 172.21.15.169:0/3967617358 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe10c06ae40 0x7fe10c06d300 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fe124037000 tx=0x7fe12402e040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:41:50.609 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.606+0000 7fe119ffb640 1 -- 172.21.15.169:0/3967617358 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe12806bcb0 con 0x7fe1280c8bf0 2023-10-26T01:41:50.813 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.811+0000 7fe12e667640 1 -- 172.21.15.169:0/3967617358 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fe12806bcb0 con 0x7fe1280c8bf0 2023-10-26T01:41:50.814 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.812+0000 7fe119ffb640 1 -- 172.21.15.169:0/3967617358 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v155) v1 ==== 80+0+0 (secure 0 0 0) 0x7fe12806bcb0 con 0x7fe1280c8bf0 2023-10-26T01:41:50.814 INFO:teuthology.orchestra.run.smithi169.stderr:listed 0 entries 2023-10-26T01:41:50.817 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.815+0000 7fe12e667640 1 -- 172.21.15.169:0/3967617358 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe10c06ae40 msgr2=0x7fe10c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:50.817 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.815+0000 7fe12e667640 1 --2- 172.21.15.169:0/3967617358 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe10c06ae40 0x7fe10c06d300 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fe124037000 tx=0x7fe12402e040 comp rx=0 tx=0).stop 2023-10-26T01:41:50.817 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.815+0000 7fe12e667640 1 -- 172.21.15.169:0/3967617358 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe1280c8bf0 msgr2=0x7fe1280c9070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:50.817 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.815+0000 7fe12e667640 1 --2- 172.21.15.169:0/3967617358 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe1280c8bf0 0x7fe1280c9070 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7fe120001330 tx=0x7fe12000df00 comp rx=0 tx=0).stop 2023-10-26T01:41:50.818 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.815+0000 7fe12e667640 1 -- 172.21.15.169:0/3967617358 shutdown_connections 2023-10-26T01:41:50.818 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.815+0000 7fe12e667640 1 --2- 172.21.15.169:0/3967617358 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe10c06ae40 0x7fe10c06d300 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:50.818 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.815+0000 7fe12e667640 1 --2- 172.21.15.169:0/3967617358 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe1280c8bf0 0x7fe1280c9070 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:50.818 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.815+0000 7fe12e667640 1 --2- 172.21.15.169:0/3967617358 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe12806bf90 0x7fe12814e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:50.818 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.815+0000 7fe12e667640 1 --2- 172.21.15.169:0/3967617358 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe1140039a0 0x7fe1280c71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:50.819 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.815+0000 7fe12e667640 1 -- 172.21.15.169:0/3967617358 >> 172.21.15.169:0/3967617358 conn(0x7fe1280af110 msgr2=0x7fe128156470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:41:50.819 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.815+0000 7fe12e667640 1 -- 172.21.15.169:0/3967617358 shutdown_connections 2023-10-26T01:41:50.819 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:50.815+0000 7fe12e667640 1 -- 172.21.15.169:0/3967617358 wait complete. 2023-10-26T01:41:50.837 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-26T01:41:50.837 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:41:50.838 DEBUG:teuthology.orchestra.run.smithi170:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-26T01:41:50.951 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi170.front.sepia.ceph.com 2023-10-26T01:41:50.951 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:41:50.952 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-26T01:41:51.005 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:41:51.005 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:41:51.064 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054301,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-26T01:41:51.065 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-26T01:41:51.065 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:41:51.126 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054303,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-26T01:41:51.127 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T01:41:51.359 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:51.356+0000 7f9babd6e640 1 -- 172.21.15.169:0/2126150170 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ba406c420 msgr2=0x7f9ba40b0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:51.359 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:51.356+0000 7f9babd6e640 1 --2- 172.21.15.169:0/2126150170 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ba406c420 0x7f9ba40b0910 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c005fd0 tx=0x7f9b9c02c2f0 comp rx=0 tx=0).stop 2023-10-26T01:41:51.359 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:51.357+0000 7f9babd6e640 1 -- 172.21.15.169:0/2126150170 shutdown_connections 2023-10-26T01:41:51.359 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:51.357+0000 7f9babd6e640 1 --2- 172.21.15.169:0/2126150170 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ba406c420 0x7f9ba40b0910 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:51.360 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:51.357+0000 7f9babd6e640 1 -- 172.21.15.169:0/2126150170 >> 172.21.15.169:0/2126150170 conn(0x7f9ba4069930 msgr2=0x7f9ba4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:41:51.360 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:51.357+0000 7f9babd6e640 1 -- 172.21.15.169:0/2126150170 shutdown_connections 2023-10-26T01:41:51.360 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:51.357+0000 7f9babd6e640 1 -- 172.21.15.169:0/2126150170 wait complete. 2023-10-26T01:41:51.361 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:51.358+0000 7f9babd6e640 1 Processor -- start 2023-10-26T01:41:51.361 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:51.358+0000 7f9babd6e640 1 -- start start 2023-10-26T01:41:51.361 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:51.359+0000 7f9babd6e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9ba406bf70 0x7f9ba414ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:41:51.362 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:51.359+0000 7f9babd6e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ba406c420 0x7f9ba4145440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:41:51.362 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:51.359+0000 7f9babd6e640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9ba414ece0 0x7f9ba414d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:41:51.362 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:51.359+0000 7f9babd6e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba4145980 con 0x7f9ba406c420 2023-10-26T01:41:51.362 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:51.359+0000 7f9babd6e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba40700d0 con 0x7f9ba406bf70 2023-10-26T01:41:51.363 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:51.359+0000 7f9babd6e640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba406f7e0 con 0x7f9ba414ece0 2023-10-26T01:41:51.363 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:51.360+0000 7f9ba9309640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ba406c420 0x7f9ba4145440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:41:51.363 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:51.360+0000 7f9ba9309640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ba406c420 0x7f9ba4145440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:38384/0 (socket says 172.21.15.169:38384) 2023-10-26T01:41:51.363 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:51.360+0000 7f9ba9309640 1 -- 172.21.15.169:0/1512349765 learned_addr learned my addr 172.21.15.169:0/1512349765 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:41:51.365 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:51.360+0000 7f9ba9b0a640 1 --2- 172.21.15.169:0/1512349765 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9ba406bf70 0x7f9ba414ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:41:51.365 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:51.360+0000 7f9ba9309640 1 -- 172.21.15.169:0/1512349765 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9ba414ece0 msgr2=0x7f9ba414d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:51.365 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:51.360+0000 7f9ba9309640 1 --2- 172.21.15.169:0/1512349765 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9ba414ece0 0x7f9ba414d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:51.366 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:51.360+0000 7f9ba9309640 1 -- 172.21.15.169:0/1512349765 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9ba406bf70 msgr2=0x7f9ba414ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:51.366 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:51.360+0000 7f9ba9309640 1 --2- 172.21.15.169:0/1512349765 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9ba406bf70 0x7f9ba414ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:51.366 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:51.360+0000 7f9ba9309640 1 -- 172.21.15.169:0/1512349765 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b9c009900 con 0x7f9ba406c420 2023-10-26T01:41:51.366 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:51.360+0000 7f9ba9b0a640 1 --2- 172.21.15.169:0/1512349765 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9ba406bf70 0x7f9ba414ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:41:51.366 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:51.360+0000 7f9ba9309640 1 --2- 172.21.15.169:0/1512349765 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ba406c420 0x7f9ba4145440 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f9b9400c450 tx=0x7f9b9400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:41:51.367 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:51.361+0000 7f9b9a7fc640 1 -- 172.21.15.169:0/1512349765 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b940052d0 con 0x7f9ba406c420 2023-10-26T01:41:51.367 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:51.361+0000 7f9b9a7fc640 1 -- 172.21.15.169:0/1512349765 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9b94005430 con 0x7f9ba406c420 2023-10-26T01:41:51.367 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:51.361+0000 7f9b9a7fc640 1 -- 172.21.15.169:0/1512349765 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b94012890 con 0x7f9ba406c420 2023-10-26T01:41:51.367 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:51.361+0000 7f9babd6e640 1 -- 172.21.15.169:0/1512349765 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ba414d890 con 0x7f9ba406c420 2023-10-26T01:41:51.367 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:51.361+0000 7f9babd6e640 1 -- 172.21.15.169:0/1512349765 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ba406fd60 con 0x7f9ba406c420 2023-10-26T01:41:51.368 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:51.362+0000 7f9babd6e640 1 -- 172.21.15.169:0/1512349765 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ba404db10 con 0x7f9ba406c420 2023-10-26T01:41:51.368 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:51.363+0000 7f9b9a7fc640 1 -- 172.21.15.169:0/1512349765 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f9b940035f0 con 0x7f9ba406c420 2023-10-26T01:41:51.369 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:51.364+0000 7f9b9a7fc640 1 --2- 172.21.15.169:0/1512349765 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9b8806ae40 0x7f9b8806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:41:51.369 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:51.364+0000 7f9b9a7fc640 1 -- 172.21.15.169:0/1512349765 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f9b94089380 con 0x7f9ba406c420 2023-10-26T01:41:51.369 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:51.364+0000 7f9b9a7fc640 1 -- 172.21.15.169:0/1512349765 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=156}) v3 -- 0x7f9b88072a30 con 0x7f9ba406c420 2023-10-26T01:41:51.369 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:51.364+0000 7f9ba9b0a640 1 --2- 172.21.15.169:0/1512349765 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9b8806ae40 0x7f9b8806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:41:51.369 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:51.365+0000 7f9ba9b0a640 1 --2- 172.21.15.169:0/1512349765 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9b8806ae40 0x7f9b8806d300 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f9ba000a810 tx=0x7f9ba0008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:41:51.370 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:51.368+0000 7f9b9a7fc640 1 -- 172.21.15.169:0/1512349765 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9b940543e0 con 0x7f9ba406c420 2023-10-26T01:41:51.596 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:51.594+0000 7f9babd6e640 1 -- 172.21.15.169:0/1512349765 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9ba4070210 con 0x7f9ba406c420 2023-10-26T01:41:51.597 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:51.595+0000 7f9b9a7fc640 1 -- 172.21.15.169:0/1512349765 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+4921 (secure 0 0 0) 0x7f9b94057a00 con 0x7f9ba406c420 2023-10-26T01:41:51.597 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:41:51.597 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":77,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5711,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/3037106607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":3037106607},{"type":"v1","addr":"172.21.15.170:6836","nonce":3037106607}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72},{"gid":5714,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/1847866184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":1847866184},{"type":"v1","addr":"172.21.15.170:6837","nonce":1847866184}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72},{"gid":8229,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2707290348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2707290348},{"type":"v1","addr":"172.21.15.169:6833","nonce":2707290348}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T01:39:25.790062+0000","modified":"2023-10-26T01:39:28.841639+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8232},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8232":{"gid":8232,"name":"c","rank":0,"incarnation":73,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6835/66540264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":66540264},{"type":"v1","addr":"172.21.15.169:6835","nonce":66540264}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-10-26T01:41:51.598 INFO:teuthology.orchestra.run.smithi169.stderr:dumped fsmap epoch 77 2023-10-26T01:41:51.599 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:51.598+0000 7f9babd6e640 1 -- 172.21.15.169:0/1512349765 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9b8806ae40 msgr2=0x7f9b8806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:51.600 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:51.598+0000 7f9babd6e640 1 --2- 172.21.15.169:0/1512349765 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9b8806ae40 0x7f9b8806d300 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f9ba000a810 tx=0x7f9ba0008040 comp rx=0 tx=0).stop 2023-10-26T01:41:51.600 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:51.598+0000 7f9babd6e640 1 -- 172.21.15.169:0/1512349765 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ba406c420 msgr2=0x7f9ba4145440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:51.600 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:51.598+0000 7f9babd6e640 1 --2- 172.21.15.169:0/1512349765 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ba406c420 0x7f9ba4145440 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f9b9400c450 tx=0x7f9b9400c910 comp rx=0 tx=0).stop 2023-10-26T01:41:51.600 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:51.598+0000 7f9babd6e640 1 -- 172.21.15.169:0/1512349765 shutdown_connections 2023-10-26T01:41:51.601 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:51.598+0000 7f9babd6e640 1 --2- 172.21.15.169:0/1512349765 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9b8806ae40 0x7f9b8806d300 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:51.601 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:51.598+0000 7f9babd6e640 1 --2- 172.21.15.169:0/1512349765 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ba406c420 0x7f9ba4145440 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:51.601 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:51.598+0000 7f9babd6e640 1 --2- 172.21.15.169:0/1512349765 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9ba414ece0 0x7f9ba414d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:51.601 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:51.598+0000 7f9babd6e640 1 --2- 172.21.15.169:0/1512349765 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9ba406bf70 0x7f9ba414ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:51.601 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:51.598+0000 7f9babd6e640 1 -- 172.21.15.169:0/1512349765 >> 172.21.15.169:0/1512349765 conn(0x7f9ba4069930 msgr2=0x7f9ba40cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:41:51.601 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:51.598+0000 7f9babd6e640 1 -- 172.21.15.169:0/1512349765 shutdown_connections 2023-10-26T01:41:51.602 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:51.598+0000 7f9babd6e640 1 -- 172.21.15.169:0/1512349765 wait complete. 2023-10-26T01:41:51.618 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:41:51.819 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:51.817+0000 7fdf2d3c7640 1 -- 172.21.15.169:0/202903770 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdf280ac4e0 msgr2=0x7fdf280cd670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:51.820 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:51.817+0000 7fdf2d3c7640 1 --2- 172.21.15.169:0/202903770 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdf280ac4e0 0x7fdf280cd670 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7fdf2806ce40 tx=0x7fdf1002bd90 comp rx=0 tx=0).stop 2023-10-26T01:41:51.820 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:51.817+0000 7fdf2d3c7640 1 -- 172.21.15.169:0/202903770 shutdown_connections 2023-10-26T01:41:51.820 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:51.817+0000 7fdf2d3c7640 1 --2- 172.21.15.169:0/202903770 >> v2:172.21.15.170:3300/0 conn(0x7fdf280ac940 0x7fdf280cdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:51.820 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:51.817+0000 7fdf2d3c7640 1 --2- 172.21.15.169:0/202903770 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdf280ac4e0 0x7fdf280cd670 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:51.821 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:51.817+0000 7fdf2d3c7640 1 --2- 172.21.15.169:0/202903770 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdf2815b610 0x7fdf2806ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:51.821 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:51.817+0000 7fdf2d3c7640 1 -- 172.21.15.169:0/202903770 >> 172.21.15.169:0/202903770 conn(0x7fdf28069930 msgr2=0x7fdf28069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:41:51.821 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:51.817+0000 7fdf2d3c7640 1 -- 172.21.15.169:0/202903770 shutdown_connections 2023-10-26T01:41:51.821 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:51.817+0000 7fdf2d3c7640 1 -- 172.21.15.169:0/202903770 wait complete. 2023-10-26T01:41:51.823 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:51.819+0000 7fdf2d3c7640 1 Processor -- start 2023-10-26T01:41:51.824 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:51.819+0000 7fdf2d3c7640 1 -- start start 2023-10-26T01:41:51.824 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:51.820+0000 7fdf2d3c7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdf280ac4e0 0x7fdf28156060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:41:51.824 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:51.820+0000 7fdf2d3c7640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdf280ac940 0x7fdf2815e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:41:51.824 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:51.820+0000 7fdf2d3c7640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdf2815b610 0x7fdf2814c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:41:51.825 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:51.820+0000 7fdf2d3c7640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf28156640 con 0x7fdf2815b610 2023-10-26T01:41:51.825 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:51.820+0000 7fdf2d3c7640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf28156920 con 0x7fdf280ac4e0 2023-10-26T01:41:51.825 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:51.820+0000 7fdf2d3c7640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf28156c00 con 0x7fdf280ac940 2023-10-26T01:41:51.825 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:51.820+0000 7fdf277fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdf2815b610 0x7fdf2814c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:41:51.825 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:51.820+0000 7fdf277fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdf2815b610 0x7fdf2814c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:38408/0 (socket says 172.21.15.169:38408) 2023-10-26T01:41:51.826 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:51.820+0000 7fdf277fe640 1 -- 172.21.15.169:0/1513952400 learned_addr learned my addr 172.21.15.169:0/1513952400 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:41:51.826 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:51.821+0000 7fdf277fe640 1 -- 172.21.15.169:0/1513952400 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdf280ac940 msgr2=0x7fdf2815e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:51.826 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:51.821+0000 7fdf277fe640 1 --2- 172.21.15.169:0/1513952400 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdf280ac940 0x7fdf2815e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:51.826 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:51.821+0000 7fdf277fe640 1 -- 172.21.15.169:0/1513952400 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdf280ac4e0 msgr2=0x7fdf28156060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:51.826 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:51.821+0000 7fdf26ffd640 1 --2- 172.21.15.169:0/1513952400 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdf280ac4e0 0x7fdf28156060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:41:51.827 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:51.821+0000 7fdf277fe640 1 --2- 172.21.15.169:0/1513952400 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdf280ac4e0 0x7fdf28156060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:51.827 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:51.821+0000 7fdf277fe640 1 -- 172.21.15.169:0/1513952400 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf10009560 con 0x7fdf2815b610 2023-10-26T01:41:51.827 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:51.821+0000 7fdf26ffd640 1 --2- 172.21.15.169:0/1513952400 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdf280ac4e0 0x7fdf28156060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:41:51.827 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:51.821+0000 7fdf277fe640 1 --2- 172.21.15.169:0/1513952400 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdf2815b610 0x7fdf2814c200 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7fdf2000ca40 tx=0x7fdf2000cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:41:51.827 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:51.821+0000 7fdf1f7fe640 1 -- 172.21.15.169:0/1513952400 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf2000adc0 con 0x7fdf2815b610 2023-10-26T01:41:51.828 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:51.822+0000 7fdf1f7fe640 1 -- 172.21.15.169:0/1513952400 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fdf20004c50 con 0x7fdf2815b610 2023-10-26T01:41:51.828 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:51.822+0000 7fdf2d3c7640 1 -- 172.21.15.169:0/1513952400 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf28156ee0 con 0x7fdf2815b610 2023-10-26T01:41:51.828 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:51.822+0000 7fdf1f7fe640 1 -- 172.21.15.169:0/1513952400 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf20005280 con 0x7fdf2815b610 2023-10-26T01:41:51.828 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:51.822+0000 7fdf2d3c7640 1 -- 172.21.15.169:0/1513952400 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf2815eae0 con 0x7fdf2815b610 2023-10-26T01:41:51.829 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:51.823+0000 7fdf2d3c7640 1 -- 172.21.15.169:0/1513952400 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf28157400 con 0x7fdf2815b610 2023-10-26T01:41:51.830 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:51.824+0000 7fdf1f7fe640 1 -- 172.21.15.169:0/1513952400 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fdf200053e0 con 0x7fdf2815b610 2023-10-26T01:41:51.830 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:51.825+0000 7fdf1f7fe640 1 --2- 172.21.15.169:0/1513952400 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fdf0806ae70 0x7fdf0806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:41:51.830 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:51.825+0000 7fdf1f7fe640 1 -- 172.21.15.169:0/1513952400 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fdf20089d50 con 0x7fdf2815b610 2023-10-26T01:41:51.830 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:51.825+0000 7fdf1f7fe640 1 -- 172.21.15.169:0/1513952400 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=156}) v3 -- 0x7fdf08072a60 con 0x7fdf2815b610 2023-10-26T01:41:51.831 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:51.825+0000 7fdf26ffd640 1 --2- 172.21.15.169:0/1513952400 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fdf0806ae70 0x7fdf0806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:41:51.831 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:51.827+0000 7fdf26ffd640 1 --2- 172.21.15.169:0/1513952400 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fdf0806ae70 0x7fdf0806d330 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fdf18009aa0 tx=0x7fdf18005db0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:41:51.831 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:51.829+0000 7fdf1f7fe640 1 -- 172.21.15.169:0/1513952400 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdf2001d070 con 0x7fdf2815b610 2023-10-26T01:41:52.023 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.021+0000 7fdf2d3c7640 1 -- 172.21.15.169:0/1513952400 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdf2804db10 con 0x7fdf2815b610 2023-10-26T01:41:52.024 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.023+0000 7fdf1f7fe640 1 -- 172.21.15.169:0/1513952400 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v155) v1 ==== 74+0+16695 (secure 0 0 0) 0x7fdf2008d030 con 0x7fdf2815b610 2023-10-26T01:41:52.025 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:41:52.026 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":155,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:41:08.113301+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-10-26T01:39:23.646624+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"150","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-10-26T01:39:24.813678+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"19.5","mappings":[{"from":2,"to":0}]},{"pgid":"19.13","mappings":[{"from":2,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:41:52.028 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.026+0000 7fdf2d3c7640 1 -- 172.21.15.169:0/1513952400 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fdf0806ae70 msgr2=0x7fdf0806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:52.028 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.026+0000 7fdf2d3c7640 1 --2- 172.21.15.169:0/1513952400 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fdf0806ae70 0x7fdf0806d330 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fdf18009aa0 tx=0x7fdf18005db0 comp rx=0 tx=0).stop 2023-10-26T01:41:52.028 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.026+0000 7fdf2d3c7640 1 -- 172.21.15.169:0/1513952400 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdf2815b610 msgr2=0x7fdf2814c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:52.029 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.026+0000 7fdf2d3c7640 1 --2- 172.21.15.169:0/1513952400 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdf2815b610 0x7fdf2814c200 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7fdf2000ca40 tx=0x7fdf2000cf00 comp rx=0 tx=0).stop 2023-10-26T01:41:52.029 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.026+0000 7fdf2d3c7640 1 -- 172.21.15.169:0/1513952400 shutdown_connections 2023-10-26T01:41:52.029 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.026+0000 7fdf2d3c7640 1 --2- 172.21.15.169:0/1513952400 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fdf0806ae70 0x7fdf0806d330 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:52.029 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.026+0000 7fdf2d3c7640 1 --2- 172.21.15.169:0/1513952400 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdf2815b610 0x7fdf2814c200 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:52.029 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.026+0000 7fdf2d3c7640 1 --2- 172.21.15.169:0/1513952400 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdf280ac940 0x7fdf2815e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:52.030 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.026+0000 7fdf2d3c7640 1 --2- 172.21.15.169:0/1513952400 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdf280ac4e0 0x7fdf28156060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:52.030 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.026+0000 7fdf2d3c7640 1 -- 172.21.15.169:0/1513952400 >> 172.21.15.169:0/1513952400 conn(0x7fdf28069930 msgr2=0x7fdf280b1010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:41:52.030 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.026+0000 7fdf2d3c7640 1 -- 172.21.15.169:0/1513952400 shutdown_connections 2023-10-26T01:41:52.030 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.026+0000 7fdf2d3c7640 1 -- 172.21.15.169:0/1513952400 wait complete. 2023-10-26T01:41:52.047 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T01:30:32.200808+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '109', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-26T01:39:23.646624+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '150', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_data', 'create_time': '2023-10-26T01:39:24.813678+0000', 'flags': 1027, 'flags_names': 'hashpspool,full,full_quota', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '155', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T01:41:52.048 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T01:41:52.269 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.266+0000 7f396aa1a640 1 -- 172.21.15.169:0/1595476922 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f39640ce450 msgr2=0x7f39640cc5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:52.269 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.266+0000 7f396aa1a640 1 --2- 172.21.15.169:0/1595476922 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f39640ce450 0x7f39640cc5f0 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f3950005fd0 tx=0x7f395002be20 comp rx=0 tx=0).stop 2023-10-26T01:41:52.269 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.266+0000 7f396aa1a640 1 -- 172.21.15.169:0/1595476922 shutdown_connections 2023-10-26T01:41:52.270 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.266+0000 7f396aa1a640 1 --2- 172.21.15.169:0/1595476922 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f39640ce450 0x7f39640cc5f0 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:52.270 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.266+0000 7f396aa1a640 1 -- 172.21.15.169:0/1595476922 >> 172.21.15.169:0/1595476922 conn(0x7f39640b2f10 msgr2=0x7f39640b3320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:41:52.270 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.266+0000 7f396aa1a640 1 -- 172.21.15.169:0/1595476922 shutdown_connections 2023-10-26T01:41:52.270 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.266+0000 7f396aa1a640 1 -- 172.21.15.169:0/1595476922 wait complete. 2023-10-26T01:41:52.271 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.268+0000 7f396aa1a640 1 Processor -- start 2023-10-26T01:41:52.271 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.268+0000 7f396aa1a640 1 -- start start 2023-10-26T01:41:52.272 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.270+0000 7f396aa1a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f39640ce450 0x7f39640c9210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:41:52.273 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.270+0000 7f396aa1a640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f39640d0a30 0x7f39640b6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:41:52.273 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.270+0000 7f396aa1a640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f396415b790 0x7f396415dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:41:52.273 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.270+0000 7f396aa1a640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39640c8df0 con 0x7f396415b790 2023-10-26T01:41:52.273 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.270+0000 7f396aa1a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f396415eab0 con 0x7f39640ce450 2023-10-26T01:41:52.273 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.270+0000 7f396aa1a640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39640c8450 con 0x7f39640d0a30 2023-10-26T01:41:52.274 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.270+0000 7f396a219640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f396415b790 0x7f396415dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:41:52.274 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.270+0000 7f396a219640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f396415b790 0x7f396415dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:38432/0 (socket says 172.21.15.169:38432) 2023-10-26T01:41:52.274 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.270+0000 7f396a219640 1 -- 172.21.15.169:0/266449873 learned_addr learned my addr 172.21.15.169:0/266449873 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:41:52.276 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.270+0000 7f3969a18640 1 --2- 172.21.15.169:0/266449873 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f39640ce450 0x7f39640c9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:41:52.276 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.270+0000 7f3969217640 1 --2- 172.21.15.169:0/266449873 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f39640d0a30 0x7f39640b6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:41:52.276 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.270+0000 7f396a219640 1 -- 172.21.15.169:0/266449873 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f39640d0a30 msgr2=0x7f39640b6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:52.276 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.270+0000 7f396a219640 1 --2- 172.21.15.169:0/266449873 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f39640d0a30 0x7f39640b6100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:52.276 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.270+0000 7f396a219640 1 -- 172.21.15.169:0/266449873 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f39640ce450 msgr2=0x7f39640c9210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:52.277 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.270+0000 7f396a219640 1 --2- 172.21.15.169:0/266449873 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f39640ce450 0x7f39640c9210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:52.277 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.270+0000 7f396a219640 1 -- 172.21.15.169:0/266449873 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3950009560 con 0x7f396415b790 2023-10-26T01:41:52.277 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.270+0000 7f3969a18640 1 --2- 172.21.15.169:0/266449873 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f39640ce450 0x7f39640c9210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:41:52.277 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.271+0000 7f396a219640 1 --2- 172.21.15.169:0/266449873 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f396415b790 0x7f396415dff0 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f395c00c410 tx=0x7f395c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:41:52.277 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.271+0000 7f39627fc640 1 -- 172.21.15.169:0/266449873 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f395c011790 con 0x7f396415b790 2023-10-26T01:41:52.278 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.271+0000 7f39627fc640 1 -- 172.21.15.169:0/266449873 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f395c0119e0 con 0x7f396415b790 2023-10-26T01:41:52.278 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.271+0000 7f396aa1a640 1 -- 172.21.15.169:0/266449873 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39640c8730 con 0x7f396415b790 2023-10-26T01:41:52.278 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.271+0000 7f39627fc640 1 -- 172.21.15.169:0/266449873 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f395c011e30 con 0x7f396415b790 2023-10-26T01:41:52.278 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.271+0000 7f396aa1a640 1 -- 172.21.15.169:0/266449873 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3964158870 con 0x7f396415b790 2023-10-26T01:41:52.278 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.272+0000 7f396aa1a640 1 -- 172.21.15.169:0/266449873 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39640c8f30 con 0x7f396415b790 2023-10-26T01:41:52.279 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.273+0000 7f39627fc640 1 -- 172.21.15.169:0/266449873 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f395c01c460 con 0x7f396415b790 2023-10-26T01:41:52.279 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.274+0000 7f39627fc640 1 --2- 172.21.15.169:0/266449873 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f394806ae40 0x7f394806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:41:52.280 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.274+0000 7f39627fc640 1 -- 172.21.15.169:0/266449873 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f395c089590 con 0x7f396415b790 2023-10-26T01:41:52.280 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.274+0000 7f39627fc640 1 -- 172.21.15.169:0/266449873 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=156}) v3 -- 0x7f3948072a30 con 0x7f396415b790 2023-10-26T01:41:52.280 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.275+0000 7f3969a18640 1 --2- 172.21.15.169:0/266449873 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f394806ae40 0x7f394806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:41:52.280 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.276+0000 7f3969a18640 1 --2- 172.21.15.169:0/266449873 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f394806ae40 0x7f394806d300 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f39580012e0 tx=0x7f3958009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:41:52.281 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.279+0000 7f39627fc640 1 -- 172.21.15.169:0/266449873 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f395c0545f0 con 0x7f396415b790 2023-10-26T01:41:52.473 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.470+0000 7f396aa1a640 1 -- 172.21.15.169:0/266449873 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f39640b0ad0 con 0x7f396415b790 2023-10-26T01:41:52.474 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.471+0000 7f39627fc640 1 -- 172.21.15.169:0/266449873 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v155) v1 ==== 93+0+10 (secure 0 0 0) 0x7f395c057c10 con 0x7f396415b790 2023-10-26T01:41:52.474 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 1 2023-10-26T01:41:52.477 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.474+0000 7f396aa1a640 1 -- 172.21.15.169:0/266449873 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f394806ae40 msgr2=0x7f394806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:52.477 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.474+0000 7f396aa1a640 1 --2- 172.21.15.169:0/266449873 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f394806ae40 0x7f394806d300 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f39580012e0 tx=0x7f3958009040 comp rx=0 tx=0).stop 2023-10-26T01:41:52.477 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.474+0000 7f396aa1a640 1 -- 172.21.15.169:0/266449873 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f396415b790 msgr2=0x7f396415dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:52.478 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.474+0000 7f396aa1a640 1 --2- 172.21.15.169:0/266449873 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f396415b790 0x7f396415dff0 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f395c00c410 tx=0x7f395c00c8d0 comp rx=0 tx=0).stop 2023-10-26T01:41:52.478 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.474+0000 7f396aa1a640 1 -- 172.21.15.169:0/266449873 shutdown_connections 2023-10-26T01:41:52.478 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.474+0000 7f396aa1a640 1 --2- 172.21.15.169:0/266449873 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f394806ae40 0x7f394806d300 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:52.478 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.474+0000 7f396aa1a640 1 --2- 172.21.15.169:0/266449873 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f396415b790 0x7f396415dff0 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:52.478 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.474+0000 7f396aa1a640 1 --2- 172.21.15.169:0/266449873 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f39640d0a30 0x7f39640b6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:52.479 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.474+0000 7f396aa1a640 1 --2- 172.21.15.169:0/266449873 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f39640ce450 0x7f39640c9210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:52.479 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.474+0000 7f396aa1a640 1 -- 172.21.15.169:0/266449873 >> 172.21.15.169:0/266449873 conn(0x7f39640b2f10 msgr2=0x7f39640b1240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:41:52.479 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.474+0000 7f396aa1a640 1 -- 172.21.15.169:0/266449873 shutdown_connections 2023-10-26T01:41:52.479 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.474+0000 7f396aa1a640 1 -- 172.21.15.169:0/266449873 wait complete. 2023-10-26T01:41:52.496 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-26T01:41:52.714 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.711+0000 7ff8ed49b640 1 -- 172.21.15.169:0/4138823044 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff8e815b9e0 msgr2=0x7ff8e806ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:52.714 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.711+0000 7ff8ed49b640 1 --2- 172.21.15.169:0/4138823044 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff8e815b9e0 0x7ff8e806ad90 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7ff8e80b0570 tx=0x7ff8e002c7c0 comp rx=0 tx=0).stop 2023-10-26T01:41:52.715 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.711+0000 7ff8ed49b640 1 -- 172.21.15.169:0/4138823044 shutdown_connections 2023-10-26T01:41:52.715 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.711+0000 7ff8ed49b640 1 --2- 172.21.15.169:0/4138823044 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff8e815b9e0 0x7ff8e806ad90 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:52.715 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.711+0000 7ff8ed49b640 1 --2- 172.21.15.169:0/4138823044 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff8e80cdd20 0x7ff8e815b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:52.715 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.711+0000 7ff8ed49b640 1 --2- 172.21.15.169:0/4138823044 >> v2:172.21.15.170:3300/0 conn(0x7ff8e8155360 0x7ff8e80cd7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:52.715 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.711+0000 7ff8ed49b640 1 -- 172.21.15.169:0/4138823044 >> 172.21.15.169:0/4138823044 conn(0x7ff8e8069930 msgr2=0x7ff8e8069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:41:52.715 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.711+0000 7ff8ed49b640 1 -- 172.21.15.169:0/4138823044 shutdown_connections 2023-10-26T01:41:52.716 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.711+0000 7ff8ed49b640 1 -- 172.21.15.169:0/4138823044 wait complete. 2023-10-26T01:41:52.717 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.713+0000 7ff8ed49b640 1 Processor -- start 2023-10-26T01:41:52.718 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.713+0000 7ff8ed49b640 1 -- start start 2023-10-26T01:41:52.718 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.714+0000 7ff8ed49b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff8e80cdd20 0x7ff8e8155b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:41:52.718 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.714+0000 7ff8ed49b640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff8e8155360 0x7ff8e81f0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:41:52.718 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.714+0000 7ff8ed49b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff8e815b9e0 0x7ff8e81f10f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:41:52.718 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.714+0000 7ff8ed49b640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e81f1630 con 0x7ff8e815b9e0 2023-10-26T01:41:52.719 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.714+0000 7ff8ed49b640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e80ade40 con 0x7ff8e80cdd20 2023-10-26T01:41:52.719 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.714+0000 7ff8ed49b640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e80ae120 con 0x7ff8e8155360 2023-10-26T01:41:52.719 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.714+0000 7ff8e77fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff8e815b9e0 0x7ff8e81f10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:41:52.719 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.714+0000 7ff8e77fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff8e815b9e0 0x7ff8e81f10f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:38450/0 (socket says 172.21.15.169:38450) 2023-10-26T01:41:52.719 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.714+0000 7ff8e77fe640 1 -- 172.21.15.169:0/865505104 learned_addr learned my addr 172.21.15.169:0/865505104 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:41:52.720 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.714+0000 7ff8e6ffd640 1 --2- 172.21.15.169:0/865505104 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff8e80cdd20 0x7ff8e8155b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:41:52.720 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.715+0000 7ff8e77fe640 1 -- 172.21.15.169:0/865505104 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff8e8155360 msgr2=0x7ff8e81f0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:52.720 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.715+0000 7ff8dffff640 1 --2- 172.21.15.169:0/865505104 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff8e8155360 0x7ff8e81f0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:41:52.720 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.715+0000 7ff8e77fe640 1 --2- 172.21.15.169:0/865505104 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff8e8155360 0x7ff8e81f0bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:52.720 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.715+0000 7ff8e77fe640 1 -- 172.21.15.169:0/865505104 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff8e80cdd20 msgr2=0x7ff8e8155b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:52.721 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.715+0000 7ff8e77fe640 1 --2- 172.21.15.169:0/865505104 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff8e80cdd20 0x7ff8e8155b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:52.721 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.715+0000 7ff8e77fe640 1 -- 172.21.15.169:0/865505104 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8e000c4a0 con 0x7ff8e815b9e0 2023-10-26T01:41:52.721 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.715+0000 7ff8e77fe640 1 --2- 172.21.15.169:0/865505104 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff8e815b9e0 0x7ff8e81f10f0 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7ff8e000c900 tx=0x7ff8e0032f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:41:52.722 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.715+0000 7ff8df7fe640 1 -- 172.21.15.169:0/865505104 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e0032520 con 0x7ff8e815b9e0 2023-10-26T01:41:52.722 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.715+0000 7ff8df7fe640 1 -- 172.21.15.169:0/865505104 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff8e0009570 con 0x7ff8e815b9e0 2023-10-26T01:41:52.723 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.715+0000 7ff8ed49b640 1 -- 172.21.15.169:0/865505104 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8e80ae400 con 0x7ff8e815b9e0 2023-10-26T01:41:52.723 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.715+0000 7ff8df7fe640 1 -- 172.21.15.169:0/865505104 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e0009940 con 0x7ff8e815b9e0 2023-10-26T01:41:52.723 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.716+0000 7ff8ed49b640 1 -- 172.21.15.169:0/865505104 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8e80ae9c0 con 0x7ff8e815b9e0 2023-10-26T01:41:52.723 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.717+0000 7ff8ed49b640 1 -- 172.21.15.169:0/865505104 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8e80aebb0 con 0x7ff8e815b9e0 2023-10-26T01:41:52.723 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.717+0000 7ff8df7fe640 1 -- 172.21.15.169:0/865505104 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ff8e000a040 con 0x7ff8e815b9e0 2023-10-26T01:41:52.724 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.719+0000 7ff8df7fe640 1 --2- 172.21.15.169:0/865505104 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff8c806ae40 0x7ff8c806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:41:52.724 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.719+0000 7ff8df7fe640 1 -- 172.21.15.169:0/865505104 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 5767+0+0 (secure 0 0 0) 0x7ff8e00abe60 con 0x7ff8e815b9e0 2023-10-26T01:41:52.724 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.719+0000 7ff8df7fe640 1 -- 172.21.15.169:0/865505104 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=156}) v3 -- 0x7ff8c8072a30 con 0x7ff8e815b9e0 2023-10-26T01:41:52.724 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.720+0000 7ff8e6ffd640 1 --2- 172.21.15.169:0/865505104 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff8c806ae40 0x7ff8c806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:41:52.725 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.721+0000 7ff8e6ffd640 1 --2- 172.21.15.169:0/865505104 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff8c806ae40 0x7ff8c806d300 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7ff8d8005e10 tx=0x7ff8d8005da0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:41:52.726 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.723+0000 7ff8df7fe640 1 -- 172.21.15.169:0/865505104 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff8e0076f50 con 0x7ff8e815b9e0 2023-10-26T01:41:52.920 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.917+0000 7ff8ed49b640 1 -- 172.21.15.169:0/865505104 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff8e804db10 con 0x7ff8e815b9e0 2023-10-26T01:41:52.920 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.918+0000 7ff8df7fe640 1 -- 172.21.15.169:0/865505104 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v155) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff8e004c020 con 0x7ff8e815b9e0 2023-10-26T01:41:52.921 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 32 2023-10-26T01:41:52.923 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.920+0000 7ff8ed49b640 1 -- 172.21.15.169:0/865505104 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff8c806ae40 msgr2=0x7ff8c806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:52.924 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.920+0000 7ff8ed49b640 1 --2- 172.21.15.169:0/865505104 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff8c806ae40 0x7ff8c806d300 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7ff8d8005e10 tx=0x7ff8d8005da0 comp rx=0 tx=0).stop 2023-10-26T01:41:52.924 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.920+0000 7ff8ed49b640 1 -- 172.21.15.169:0/865505104 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff8e815b9e0 msgr2=0x7ff8e81f10f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:52.924 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.920+0000 7ff8ed49b640 1 --2- 172.21.15.169:0/865505104 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff8e815b9e0 0x7ff8e81f10f0 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7ff8e000c900 tx=0x7ff8e0032f00 comp rx=0 tx=0).stop 2023-10-26T01:41:52.924 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.921+0000 7ff8ed49b640 1 -- 172.21.15.169:0/865505104 shutdown_connections 2023-10-26T01:41:52.925 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.921+0000 7ff8ed49b640 1 --2- 172.21.15.169:0/865505104 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff8c806ae40 0x7ff8c806d300 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:52.925 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.921+0000 7ff8ed49b640 1 --2- 172.21.15.169:0/865505104 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff8e815b9e0 0x7ff8e81f10f0 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:52.925 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.921+0000 7ff8ed49b640 1 --2- 172.21.15.169:0/865505104 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff8e8155360 0x7ff8e81f0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:52.925 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.921+0000 7ff8ed49b640 1 --2- 172.21.15.169:0/865505104 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff8e80cdd20 0x7ff8e8155b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:52.925 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.921+0000 7ff8ed49b640 1 -- 172.21.15.169:0/865505104 >> 172.21.15.169:0/865505104 conn(0x7ff8e8069930 msgr2=0x7ff8e8069d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:41:52.926 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.921+0000 7ff8ed49b640 1 -- 172.21.15.169:0/865505104 shutdown_connections 2023-10-26T01:41:52.926 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:52.921+0000 7ff8ed49b640 1 -- 172.21.15.169:0/865505104 wait complete. 2023-10-26T01:41:52.941 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-26T01:41:53.165 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:53.162+0000 7fa79b2da640 1 -- 172.21.15.169:0/3869497479 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa780009ea0 msgr2=0x7fa78000c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:53.193 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:53.162+0000 7fa79b2da640 1 --2- 172.21.15.169:0/3869497479 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa780009ea0 0x7fa78000c330 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7fa78c001680 tx=0x7fa7800037d0 comp rx=0 tx=0).stop 2023-10-26T01:41:53.193 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:53.162+0000 7fa799877640 1 -- 172.21.15.169:0/3869497479 reap_dead start 2023-10-26T01:41:53.193 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:53.162+0000 7fa79b2da640 1 -- 172.21.15.169:0/3869497479 shutdown_connections 2023-10-26T01:41:53.194 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:53.162+0000 7fa79b2da640 1 -- 172.21.15.169:0/3869497479 >> 172.21.15.169:0/3869497479 conn(0x7fa794069930 msgr2=0x7fa794069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:41:53.194 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:53.162+0000 7fa79b2da640 1 -- 172.21.15.169:0/3869497479 shutdown_connections 2023-10-26T01:41:53.194 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:53.162+0000 7fa79b2da640 1 -- 172.21.15.169:0/3869497479 wait complete. 2023-10-26T01:41:53.197 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:53.164+0000 7fa79b2da640 1 Processor -- start 2023-10-26T01:41:53.197 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:53.164+0000 7fa79b2da640 1 -- start start 2023-10-26T01:41:53.197 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:53.165+0000 7fa79b2da640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa780009ea0 0x7fa794070d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:41:53.197 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:53.165+0000 7fa79b2da640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa7940712b0 0x7fa7941f6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:41:53.198 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:53.165+0000 7fa79b2da640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa7941f69a0 0x7fa7941f6e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:41:53.198 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:53.165+0000 7fa79b2da640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa794071710 con 0x7fa7941f69a0 2023-10-26T01:41:53.198 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:53.165+0000 7fa79b2da640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa79406f060 con 0x7fa7940712b0 2023-10-26T01:41:53.198 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:53.165+0000 7fa79b2da640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa79406f340 con 0x7fa780009ea0 2023-10-26T01:41:53.198 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:53.165+0000 7fa799877640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa7941f69a0 0x7fa7941f6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:41:53.199 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:53.165+0000 7fa799877640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa7941f69a0 0x7fa7941f6e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:38462/0 (socket says 172.21.15.169:38462) 2023-10-26T01:41:53.199 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:53.165+0000 7fa799877640 1 -- 172.21.15.169:0/3139366782 learned_addr learned my addr 172.21.15.169:0/3139366782 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:41:53.199 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:53.166+0000 7fa799877640 1 -- 172.21.15.169:0/3139366782 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa780009ea0 msgr2=0x7fa794070d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:53.199 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:53.166+0000 7fa799877640 1 --2- 172.21.15.169:0/3139366782 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa780009ea0 0x7fa794070d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:53.199 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:53.166+0000 7fa799877640 1 -- 172.21.15.169:0/3139366782 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa7940712b0 msgr2=0x7fa7941f6460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:53.200 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:53.166+0000 7fa798875640 1 --2- 172.21.15.169:0/3139366782 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa7940712b0 0x7fa7941f6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:41:53.200 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:53.166+0000 7fa799877640 1 --2- 172.21.15.169:0/3139366782 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa7940712b0 0x7fa7941f6460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:53.200 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:53.166+0000 7fa799877640 1 -- 172.21.15.169:0/3139366782 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa79406f620 con 0x7fa7941f69a0 2023-10-26T01:41:53.200 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:53.166+0000 7fa799877640 1 --2- 172.21.15.169:0/3139366782 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa7941f69a0 0x7fa7941f6e20 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7fa78c02c1c0 tx=0x7fa78c0062e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:41:53.200 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:53.166+0000 7fa789ffb640 1 -- 172.21.15.169:0/3139366782 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa78c003a00 con 0x7fa7941f69a0 2023-10-26T01:41:53.201 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:53.166+0000 7fa789ffb640 1 -- 172.21.15.169:0/3139366782 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa78c003c50 con 0x7fa7941f69a0 2023-10-26T01:41:53.201 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:53.166+0000 7fa789ffb640 1 -- 172.21.15.169:0/3139366782 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa78c02c550 con 0x7fa7941f69a0 2023-10-26T01:41:53.201 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:53.166+0000 7fa79b2da640 1 -- 172.21.15.169:0/3139366782 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa79406f900 con 0x7fa7941f69a0 2023-10-26T01:41:53.201 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:53.166+0000 7fa79b2da640 1 -- 172.21.15.169:0/3139366782 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa79406ed30 con 0x7fa7941f69a0 2023-10-26T01:41:53.201 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:53.167+0000 7fa79b2da640 1 -- 172.21.15.169:0/3139366782 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa79406fe20 con 0x7fa7941f69a0 2023-10-26T01:41:53.202 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:53.168+0000 7fa789ffb640 1 -- 172.21.15.169:0/3139366782 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fa78c02c6b0 con 0x7fa7941f69a0 2023-10-26T01:41:53.202 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:53.169+0000 7fa789ffb640 1 --2- 172.21.15.169:0/3139366782 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa77806ae70 0x7fa77806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:41:53.202 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:53.169+0000 7fa789ffb640 1 -- 172.21.15.169:0/3139366782 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fa78c0a9fe0 con 0x7fa7941f69a0 2023-10-26T01:41:53.202 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:53.169+0000 7fa789ffb640 1 -- 172.21.15.169:0/3139366782 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=156}) v3 -- 0x7fa778072a60 con 0x7fa7941f69a0 2023-10-26T01:41:53.202 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:53.170+0000 7fa799076640 1 --2- 172.21.15.169:0/3139366782 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa77806ae70 0x7fa77806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:41:53.203 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:53.171+0000 7fa799076640 1 --2- 172.21.15.169:0/3139366782 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa77806ae70 0x7fa77806d330 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fa790001560 tx=0x7fa7900018c0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:41:53.203 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:53.174+0000 7fa789ffb640 1 -- 172.21.15.169:0/3139366782 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa79406fe20 con 0x7fa7941f69a0 2023-10-26T01:41:53.370 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:53.367+0000 7fa79b2da640 1 -- 172.21.15.169:0/3139366782 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa79406fe20 con 0x7fa7941f69a0 2023-10-26T01:41:53.371 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:53.368+0000 7fa789ffb640 1 -- 172.21.15.169:0/3139366782 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v155) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa79406fe20 con 0x7fa7941f69a0 2023-10-26T01:41:53.371 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 64 2023-10-26T01:41:53.374 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:53.371+0000 7fa79b2da640 1 -- 172.21.15.169:0/3139366782 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa77806ae70 msgr2=0x7fa77806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:53.374 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:53.371+0000 7fa79b2da640 1 --2- 172.21.15.169:0/3139366782 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa77806ae70 0x7fa77806d330 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fa790001560 tx=0x7fa7900018c0 comp rx=0 tx=0).stop 2023-10-26T01:41:53.375 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:53.371+0000 7fa79b2da640 1 -- 172.21.15.169:0/3139366782 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa7941f69a0 msgr2=0x7fa7941f6e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:53.375 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:53.371+0000 7fa79b2da640 1 --2- 172.21.15.169:0/3139366782 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa7941f69a0 0x7fa7941f6e20 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7fa78c02c1c0 tx=0x7fa78c0062e0 comp rx=0 tx=0).stop 2023-10-26T01:41:53.375 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:53.371+0000 7fa79b2da640 1 -- 172.21.15.169:0/3139366782 shutdown_connections 2023-10-26T01:41:53.375 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:53.371+0000 7fa79b2da640 1 --2- 172.21.15.169:0/3139366782 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa77806ae70 0x7fa77806d330 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:53.375 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:53.371+0000 7fa79b2da640 1 --2- 172.21.15.169:0/3139366782 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa7941f69a0 0x7fa7941f6e20 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:53.376 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:53.371+0000 7fa79b2da640 1 --2- 172.21.15.169:0/3139366782 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa7940712b0 0x7fa7941f6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:53.376 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:53.371+0000 7fa79b2da640 1 --2- 172.21.15.169:0/3139366782 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa780009ea0 0x7fa794070d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:53.376 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:53.371+0000 7fa79b2da640 1 -- 172.21.15.169:0/3139366782 >> 172.21.15.169:0/3139366782 conn(0x7fa794069930 msgr2=0x7fa79415a000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:41:53.376 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:53.372+0000 7fa79b2da640 1 -- 172.21.15.169:0/3139366782 shutdown_connections 2023-10-26T01:41:53.376 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:53.372+0000 7fa79b2da640 1 -- 172.21.15.169:0/3139366782 wait complete. 2023-10-26T01:41:53.391 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T01:41:53.613 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:53.610+0000 7f7dd1eaf640 1 -- 172.21.15.169:0/2613565759 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7dcc0b2d60 msgr2=0x7f7dcc0b3ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:53.613 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:53.610+0000 7f7dd1eaf640 1 --2- 172.21.15.169:0/2613565759 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7dcc0b2d60 0x7f7dcc0b3ba0 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f7db4009830 tx=0x7f7db402bd60 comp rx=0 tx=0).stop 2023-10-26T01:41:53.614 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:53.610+0000 7f7dd1eaf640 1 -- 172.21.15.169:0/2613565759 shutdown_connections 2023-10-26T01:41:53.614 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:53.610+0000 7f7dd1eaf640 1 --2- 172.21.15.169:0/2613565759 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7dcc15b990 0x7f7dcc15be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:53.614 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:53.610+0000 7f7dd1eaf640 1 --2- 172.21.15.169:0/2613565759 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7dcc0b2d60 0x7f7dcc0b3ba0 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:53.614 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:53.610+0000 7f7dd1eaf640 1 -- 172.21.15.169:0/2613565759 >> 172.21.15.169:0/2613565759 conn(0x7f7dcc098a60 msgr2=0x7f7dcc0b4a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:41:53.614 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:53.610+0000 7f7dd1eaf640 1 -- 172.21.15.169:0/2613565759 shutdown_connections 2023-10-26T01:41:53.615 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:53.610+0000 7f7dd1eaf640 1 -- 172.21.15.169:0/2613565759 wait complete. 2023-10-26T01:41:53.616 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:53.612+0000 7f7dd1eaf640 1 Processor -- start 2023-10-26T01:41:53.616 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:53.612+0000 7f7dd1eaf640 1 -- start start 2023-10-26T01:41:53.616 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:53.613+0000 7f7dd1eaf640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7dcc06d5b0 0x7f7dcc145030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:41:53.617 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:53.613+0000 7f7dd1eaf640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7dcc0b2d60 0x7f7dcc1e2620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:41:53.617 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:53.613+0000 7f7dd1eaf640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7dcc15b990 0x7f7dcc1e7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:41:53.617 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:53.613+0000 7f7dd1eaf640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dcc1e4a70 con 0x7f7dcc15b990 2023-10-26T01:41:53.617 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:53.613+0000 7f7dd1eaf640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dcc1e5030 con 0x7f7dcc06d5b0 2023-10-26T01:41:53.617 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:53.613+0000 7f7dd1eaf640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dcc1e30f0 con 0x7f7dcc0b2d60 2023-10-26T01:41:53.618 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:53.613+0000 7f7dcbfff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7dcc15b990 0x7f7dcc1e7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:41:53.618 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:53.613+0000 7f7dcbfff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7dcc15b990 0x7f7dcc1e7250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:38482/0 (socket says 172.21.15.169:38482) 2023-10-26T01:41:53.618 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:53.613+0000 7f7dcbfff640 1 -- 172.21.15.169:0/2473761091 learned_addr learned my addr 172.21.15.169:0/2473761091 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:41:53.618 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:53.613+0000 7f7dcbfff640 1 -- 172.21.15.169:0/2473761091 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7dcc0b2d60 msgr2=0x7f7dcc1e2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:53.618 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:53.613+0000 7f7dcbfff640 1 --2- 172.21.15.169:0/2473761091 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7dcc0b2d60 0x7f7dcc1e2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:53.619 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:53.613+0000 7f7dcb7fe640 1 --2- 172.21.15.169:0/2473761091 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7dcc06d5b0 0x7f7dcc145030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:41:53.619 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:53.613+0000 7f7dcbfff640 1 -- 172.21.15.169:0/2473761091 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7dcc06d5b0 msgr2=0x7f7dcc145030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:53.619 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:53.614+0000 7f7dcbfff640 1 --2- 172.21.15.169:0/2473761091 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7dcc06d5b0 0x7f7dcc145030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:53.619 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:53.614+0000 7f7dcbfff640 1 -- 172.21.15.169:0/2473761091 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7db4009490 con 0x7f7dcc15b990 2023-10-26T01:41:53.620 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:53.614+0000 7f7dcbfff640 1 --2- 172.21.15.169:0/2473761091 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7dcc15b990 0x7f7dcc1e7250 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f7dc400e960 tx=0x7f7dc400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:41:53.621 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:53.614+0000 7f7dc8ff9640 1 -- 172.21.15.169:0/2473761091 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc4004d20 con 0x7f7dcc15b990 2023-10-26T01:41:53.621 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:53.614+0000 7f7dc8ff9640 1 -- 172.21.15.169:0/2473761091 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7dc400cb20 con 0x7f7dcc15b990 2023-10-26T01:41:53.621 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:53.614+0000 7f7dc8ff9640 1 -- 172.21.15.169:0/2473761091 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc4005020 con 0x7f7dcc15b990 2023-10-26T01:41:53.621 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:53.614+0000 7f7dd1eaf640 1 -- 172.21.15.169:0/2473761091 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dcc1e33d0 con 0x7f7dcc15b990 2023-10-26T01:41:53.621 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:53.614+0000 7f7dd1eaf640 1 -- 172.21.15.169:0/2473761091 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dcc145570 con 0x7f7dcc15b990 2023-10-26T01:41:53.622 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:53.615+0000 7f7dd1eaf640 1 -- 172.21.15.169:0/2473761091 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7dcc1e51a0 con 0x7f7dcc15b990 2023-10-26T01:41:53.622 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:53.616+0000 7f7dc8ff9640 1 -- 172.21.15.169:0/2473761091 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f7dc4005180 con 0x7f7dcc15b990 2023-10-26T01:41:53.622 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:53.617+0000 7f7dc8ff9640 1 --2- 172.21.15.169:0/2473761091 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7dac06ae40 0x7f7dac06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:41:53.622 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:53.617+0000 7f7dc8ff9640 1 -- 172.21.15.169:0/2473761091 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f7dc4089fe0 con 0x7f7dcc15b990 2023-10-26T01:41:53.623 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:53.617+0000 7f7dc8ff9640 1 -- 172.21.15.169:0/2473761091 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=156}) v3 -- 0x7f7dac072a30 con 0x7f7dcc15b990 2023-10-26T01:41:53.623 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:53.618+0000 7f7dcb7fe640 1 --2- 172.21.15.169:0/2473761091 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7dac06ae40 0x7f7dac06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:41:53.623 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:53.619+0000 7f7dcb7fe640 1 --2- 172.21.15.169:0/2473761091 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7dac06ae40 0x7f7dac06d300 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc007fd0 tx=0x7f7dbc009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:41:53.624 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:53.622+0000 7f7dc8ff9640 1 -- 172.21.15.169:0/2473761091 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7dc401f020 con 0x7f7dcc15b990 2023-10-26T01:41:53.847 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:53.845+0000 7f7dd1eaf640 1 -- 172.21.15.169:0/2473761091 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7dcc04db10 con 0x7f7dcc15b990 2023-10-26T01:41:53.848 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:53.846+0000 7f7dc8ff9640 1 -- 172.21.15.169:0/2473761091 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+4921 (secure 0 0 0) 0x7f7dc40550a0 con 0x7f7dcc15b990 2023-10-26T01:41:53.848 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:41:53.849 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":77,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5711,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/3037106607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":3037106607},{"type":"v1","addr":"172.21.15.170:6836","nonce":3037106607}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72},{"gid":5714,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/1847866184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":1847866184},{"type":"v1","addr":"172.21.15.170:6837","nonce":1847866184}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72},{"gid":8229,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2707290348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2707290348},{"type":"v1","addr":"172.21.15.169:6833","nonce":2707290348}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T01:39:25.790062+0000","modified":"2023-10-26T01:39:28.841639+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8232},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8232":{"gid":8232,"name":"c","rank":0,"incarnation":73,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6835/66540264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":66540264},{"type":"v1","addr":"172.21.15.169:6835","nonce":66540264}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-10-26T01:41:53.849 INFO:teuthology.orchestra.run.smithi169.stderr:dumped fsmap epoch 77 2023-10-26T01:41:53.851 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:53.848+0000 7f7dd1eaf640 1 -- 172.21.15.169:0/2473761091 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7dac06ae40 msgr2=0x7f7dac06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:53.852 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:53.848+0000 7f7dd1eaf640 1 --2- 172.21.15.169:0/2473761091 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7dac06ae40 0x7f7dac06d300 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc007fd0 tx=0x7f7dbc009040 comp rx=0 tx=0).stop 2023-10-26T01:41:53.852 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:53.848+0000 7f7dd1eaf640 1 -- 172.21.15.169:0/2473761091 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7dcc15b990 msgr2=0x7f7dcc1e7250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:53.852 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:53.848+0000 7f7dd1eaf640 1 --2- 172.21.15.169:0/2473761091 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7dcc15b990 0x7f7dcc1e7250 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f7dc400e960 tx=0x7f7dc400ee20 comp rx=0 tx=0).stop 2023-10-26T01:41:53.852 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:53.848+0000 7f7dd1eaf640 1 -- 172.21.15.169:0/2473761091 shutdown_connections 2023-10-26T01:41:53.853 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:53.848+0000 7f7dd1eaf640 1 --2- 172.21.15.169:0/2473761091 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7dac06ae40 0x7f7dac06d300 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:53.853 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:53.848+0000 7f7dd1eaf640 1 --2- 172.21.15.169:0/2473761091 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7dcc15b990 0x7f7dcc1e7250 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:53.853 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:53.848+0000 7f7dd1eaf640 1 --2- 172.21.15.169:0/2473761091 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7dcc0b2d60 0x7f7dcc1e2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:53.853 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:53.848+0000 7f7dd1eaf640 1 --2- 172.21.15.169:0/2473761091 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7dcc06d5b0 0x7f7dcc145030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:53.853 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:53.848+0000 7f7dd1eaf640 1 -- 172.21.15.169:0/2473761091 >> 172.21.15.169:0/2473761091 conn(0x7f7dcc098a60 msgr2=0x7f7dcc0cfbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:41:53.854 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:53.849+0000 7f7dd1eaf640 1 -- 172.21.15.169:0/2473761091 shutdown_connections 2023-10-26T01:41:53.854 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:53.849+0000 7f7dd1eaf640 1 -- 172.21.15.169:0/2473761091 wait complete. 2023-10-26T01:41:53.869 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:41:54.090 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.087+0000 7fe9c9f06640 1 -- 172.21.15.169:0/1171419342 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9b40039a0 msgr2=0x7fe9b4003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:54.090 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.087+0000 7fe9c9f06640 1 --2- 172.21.15.169:0/1171419342 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9b40039a0 0x7fe9b4003dc0 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7fe9b800c290 tx=0x7fe9b40057f0 comp rx=0 tx=0).stop 2023-10-26T01:41:54.091 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.087+0000 7fe9c9705640 1 -- 172.21.15.169:0/1171419342 reap_dead start 2023-10-26T01:41:54.091 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.087+0000 7fe9c9f06640 1 -- 172.21.15.169:0/1171419342 shutdown_connections 2023-10-26T01:41:54.091 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.087+0000 7fe9c9f06640 1 -- 172.21.15.169:0/1171419342 >> 172.21.15.169:0/1171419342 conn(0x7fe9c40aedd0 msgr2=0x7fe9c40af1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:41:54.091 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.087+0000 7fe9c9f06640 1 -- 172.21.15.169:0/1171419342 shutdown_connections 2023-10-26T01:41:54.091 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.087+0000 7fe9c9f06640 1 -- 172.21.15.169:0/1171419342 wait complete. 2023-10-26T01:41:54.092 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.089+0000 7fe9c9f06640 1 Processor -- start 2023-10-26T01:41:54.092 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.089+0000 7fe9c9f06640 1 -- start start 2023-10-26T01:41:54.094 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.090+0000 7fe9c9f06640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9b40039a0 0x7fe9c4142ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:41:54.094 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.090+0000 7fe9c9f06640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe9c4156030 0x7fe9c41f5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:41:54.094 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.090+0000 7fe9c9f06640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe9c406f3f0 0x7fe9c40c9390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:41:54.095 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.090+0000 7fe9c9f06640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c4157b20 con 0x7fe9b40039a0 2023-10-26T01:41:54.095 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.090+0000 7fe9c9f06640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c41f6320 con 0x7fe9c4156030 2023-10-26T01:41:54.095 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.090+0000 7fe9c9f06640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c406f0e0 con 0x7fe9c406f3f0 2023-10-26T01:41:54.095 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.090+0000 7fe9c8f04640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9b40039a0 0x7fe9c4142ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:41:54.095 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.090+0000 7fe9c8f04640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9b40039a0 0x7fe9c4142ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:37550/0 (socket says 172.21.15.169:37550) 2023-10-26T01:41:54.096 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.090+0000 7fe9c8f04640 1 -- 172.21.15.169:0/3644306575 learned_addr learned my addr 172.21.15.169:0/3644306575 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:41:54.096 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.091+0000 7fe9c3fff640 1 --2- 172.21.15.169:0/3644306575 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe9c4156030 0x7fe9c41f5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:41:54.096 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.091+0000 7fe9c8f04640 1 -- 172.21.15.169:0/3644306575 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe9c406f3f0 msgr2=0x7fe9c40c9390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:54.096 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.091+0000 7fe9c8f04640 1 --2- 172.21.15.169:0/3644306575 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe9c406f3f0 0x7fe9c40c9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:54.096 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.091+0000 7fe9c8f04640 1 -- 172.21.15.169:0/3644306575 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe9c4156030 msgr2=0x7fe9c41f5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:54.096 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.091+0000 7fe9c8f04640 1 --2- 172.21.15.169:0/3644306575 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe9c4156030 0x7fe9c41f5de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:54.097 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.091+0000 7fe9c8f04640 1 -- 172.21.15.169:0/3644306575 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9c4157de0 con 0x7fe9b40039a0 2023-10-26T01:41:54.097 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.091+0000 7fe9c8f04640 1 --2- 172.21.15.169:0/3644306575 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9b40039a0 0x7fe9c4142ad0 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7fe9b802c910 tx=0x7fe9b80014b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:41:54.097 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.091+0000 7fe9c17fa640 1 -- 172.21.15.169:0/3644306575 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b8006250 con 0x7fe9b40039a0 2023-10-26T01:41:54.097 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.091+0000 7fe9c17fa640 1 -- 172.21.15.169:0/3644306575 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe9b80063b0 con 0x7fe9b40039a0 2023-10-26T01:41:54.097 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.091+0000 7fe9c9f06640 1 -- 172.21.15.169:0/3644306575 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9c406eb20 con 0x7fe9b40039a0 2023-10-26T01:41:54.098 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.091+0000 7fe9c17fa640 1 -- 172.21.15.169:0/3644306575 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b8003850 con 0x7fe9b40039a0 2023-10-26T01:41:54.098 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.092+0000 7fe9c9f06640 1 -- 172.21.15.169:0/3644306575 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9c406f870 con 0x7fe9b40039a0 2023-10-26T01:41:54.099 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.093+0000 7fe9c9f06640 1 -- 172.21.15.169:0/3644306575 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9c4073c10 con 0x7fe9b40039a0 2023-10-26T01:41:54.099 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.093+0000 7fe9c17fa640 1 -- 172.21.15.169:0/3644306575 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fe9b8031030 con 0x7fe9b40039a0 2023-10-26T01:41:54.099 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.094+0000 7fe9c17fa640 1 --2- 172.21.15.169:0/3644306575 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe9a806ae40 0x7fe9a806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:41:54.099 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.095+0000 7fe9c17fa640 1 -- 172.21.15.169:0/3644306575 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fe9b80aa9f0 con 0x7fe9b40039a0 2023-10-26T01:41:54.100 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.095+0000 7fe9c17fa640 1 -- 172.21.15.169:0/3644306575 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=156}) v3 -- 0x7fe9a8072a30 con 0x7fe9b40039a0 2023-10-26T01:41:54.100 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.095+0000 7fe9c3fff640 1 --2- 172.21.15.169:0/3644306575 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe9a806ae40 0x7fe9a806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:41:54.100 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.096+0000 7fe9c3fff640 1 --2- 172.21.15.169:0/3644306575 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe9a806ae40 0x7fe9a806d300 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fe9c40b3140 tx=0x7fe9b00069c0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:41:54.101 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.098+0000 7fe9c17fa640 1 -- 172.21.15.169:0/3644306575 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe9b8044060 con 0x7fe9b40039a0 2023-10-26T01:41:54.279 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.276+0000 7fe9c9f06640 1 -- 172.21.15.169:0/3644306575 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe9c40cc4d0 con 0x7fe9b40039a0 2023-10-26T01:41:54.281 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.279+0000 7fe9c17fa640 1 -- 172.21.15.169:0/3644306575 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v155) v1 ==== 74+0+16695 (secure 0 0 0) 0x7fe9b8079450 con 0x7fe9b40039a0 2023-10-26T01:41:54.282 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:41:54.282 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":155,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:41:08.113301+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-10-26T01:39:23.646624+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"150","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-10-26T01:39:24.813678+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"19.5","mappings":[{"from":2,"to":0}]},{"pgid":"19.13","mappings":[{"from":2,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:41:54.284 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.281+0000 7fe9c9f06640 1 -- 172.21.15.169:0/3644306575 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe9a806ae40 msgr2=0x7fe9a806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:54.285 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.281+0000 7fe9c9f06640 1 --2- 172.21.15.169:0/3644306575 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe9a806ae40 0x7fe9a806d300 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fe9c40b3140 tx=0x7fe9b00069c0 comp rx=0 tx=0).stop 2023-10-26T01:41:54.285 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.281+0000 7fe9c9f06640 1 -- 172.21.15.169:0/3644306575 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9b40039a0 msgr2=0x7fe9c4142ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:54.285 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.281+0000 7fe9c9f06640 1 --2- 172.21.15.169:0/3644306575 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9b40039a0 0x7fe9c4142ad0 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7fe9b802c910 tx=0x7fe9b80014b0 comp rx=0 tx=0).stop 2023-10-26T01:41:54.285 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.282+0000 7fe9c9f06640 1 -- 172.21.15.169:0/3644306575 shutdown_connections 2023-10-26T01:41:54.285 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.282+0000 7fe9c9f06640 1 --2- 172.21.15.169:0/3644306575 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe9a806ae40 0x7fe9a806d300 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:54.286 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.282+0000 7fe9c9f06640 1 --2- 172.21.15.169:0/3644306575 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe9c406f3f0 0x7fe9c40c9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:54.286 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.282+0000 7fe9c9f06640 1 --2- 172.21.15.169:0/3644306575 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe9c4156030 0x7fe9c41f5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:54.286 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.282+0000 7fe9c9f06640 1 --2- 172.21.15.169:0/3644306575 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9b40039a0 0x7fe9c4142ad0 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:54.286 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.282+0000 7fe9c9f06640 1 -- 172.21.15.169:0/3644306575 >> 172.21.15.169:0/3644306575 conn(0x7fe9c40aedd0 msgr2=0x7fe9c40adec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:41:54.286 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.282+0000 7fe9c9f06640 1 -- 172.21.15.169:0/3644306575 shutdown_connections 2023-10-26T01:41:54.287 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.282+0000 7fe9c9f06640 1 -- 172.21.15.169:0/3644306575 wait complete. 2023-10-26T01:41:54.303 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-26T01:41:54.304 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T01:41:54.518 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.515+0000 7f9a98a25640 1 -- 172.21.15.169:0/429403420 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a940ac6d0 msgr2=0x7f9a9415d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:54.518 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.515+0000 7f9a98a25640 1 --2- 172.21.15.169:0/429403420 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a940ac6d0 0x7f9a9415d840 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f9a84005fd0 tx=0x7f9a8402be20 comp rx=0 tx=0).stop 2023-10-26T01:41:54.519 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.515+0000 7f9a98a25640 1 -- 172.21.15.169:0/429403420 shutdown_connections 2023-10-26T01:41:54.519 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.515+0000 7f9a98a25640 1 --2- 172.21.15.169:0/429403420 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a940ac6d0 0x7f9a9415d840 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:54.519 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.515+0000 7f9a98a25640 1 -- 172.21.15.169:0/429403420 >> 172.21.15.169:0/429403420 conn(0x7f9a940aedd0 msgr2=0x7f9a940af1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:41:54.519 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.515+0000 7f9a98a25640 1 -- 172.21.15.169:0/429403420 shutdown_connections 2023-10-26T01:41:54.519 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.515+0000 7f9a98a25640 1 -- 172.21.15.169:0/429403420 wait complete. 2023-10-26T01:41:54.520 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.517+0000 7f9a98a25640 1 Processor -- start 2023-10-26T01:41:54.520 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.517+0000 7f9a98a25640 1 -- start start 2023-10-26T01:41:54.522 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.518+0000 7f9a98a25640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a9409a5c0 0x7f9a9413f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:41:54.522 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.518+0000 7f9a98a25640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9a940ac6d0 0x7f9a941f51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:41:54.523 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.518+0000 7f9a98a25640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9a940c9970 0x7f9a940aa0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:41:54.523 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.518+0000 7f9a98a25640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a9406e8f0 con 0x7f9a9409a5c0 2023-10-26T01:41:54.523 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.518+0000 7f9a98a25640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a941f60d0 con 0x7f9a940c9970 2023-10-26T01:41:54.523 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.518+0000 7f9a98a25640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a941f5b10 con 0x7f9a940ac6d0 2023-10-26T01:41:54.523 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.518+0000 7f9a92d9d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a9409a5c0 0x7f9a9413f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:41:54.524 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.518+0000 7f9a92d9d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a9409a5c0 0x7f9a9413f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:37570/0 (socket says 172.21.15.169:37570) 2023-10-26T01:41:54.524 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.518+0000 7f9a92d9d640 1 -- 172.21.15.169:0/1596016161 learned_addr learned my addr 172.21.15.169:0/1596016161 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:41:54.524 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.519+0000 7f9a92d9d640 1 -- 172.21.15.169:0/1596016161 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9a940ac6d0 msgr2=0x7f9a941f51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:54.524 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.519+0000 7f9a92d9d640 1 --2- 172.21.15.169:0/1596016161 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9a940ac6d0 0x7f9a941f51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:54.524 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.519+0000 7f9a9359e640 1 --2- 172.21.15.169:0/1596016161 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9a940c9970 0x7f9a940aa0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:41:54.524 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.519+0000 7f9a92d9d640 1 -- 172.21.15.169:0/1596016161 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9a940c9970 msgr2=0x7f9a940aa0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:54.525 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.519+0000 7f9a92d9d640 1 --2- 172.21.15.169:0/1596016161 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9a940c9970 0x7f9a940aa0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:54.525 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.519+0000 7f9a92d9d640 1 -- 172.21.15.169:0/1596016161 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a84009560 con 0x7f9a9409a5c0 2023-10-26T01:41:54.525 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.519+0000 7f9a9359e640 1 --2- 172.21.15.169:0/1596016161 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9a940c9970 0x7f9a940aa0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:41:54.525 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.519+0000 7f9a92d9d640 1 --2- 172.21.15.169:0/1596016161 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a9409a5c0 0x7f9a9413f200 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f9a84036040 tx=0x7f9a8402daf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:41:54.525 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.519+0000 7f9a8b7fe640 1 -- 172.21.15.169:0/1596016161 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a840035f0 con 0x7f9a9409a5c0 2023-10-26T01:41:54.526 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.519+0000 7f9a8b7fe640 1 -- 172.21.15.169:0/1596016161 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9a84003840 con 0x7f9a9409a5c0 2023-10-26T01:41:54.526 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.519+0000 7f9a8b7fe640 1 -- 172.21.15.169:0/1596016161 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a84003c10 con 0x7f9a9409a5c0 2023-10-26T01:41:54.526 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.520+0000 7f9a98a25640 1 -- 172.21.15.169:0/1596016161 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a941f5df0 con 0x7f9a9409a5c0 2023-10-26T01:41:54.526 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.520+0000 7f9a98a25640 1 -- 172.21.15.169:0/1596016161 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a94159db0 con 0x7f9a9409a5c0 2023-10-26T01:41:54.531 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.521+0000 7f9a98a25640 1 -- 172.21.15.169:0/1596016161 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a940995e0 con 0x7f9a9409a5c0 2023-10-26T01:41:54.531 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.522+0000 7f9a8b7fe640 1 -- 172.21.15.169:0/1596016161 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f9a8403e450 con 0x7f9a9409a5c0 2023-10-26T01:41:54.531 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.522+0000 7f9a8b7fe640 1 --2- 172.21.15.169:0/1596016161 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9a7406ae40 0x7f9a7406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:41:54.532 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.523+0000 7f9a8b7fe640 1 -- 172.21.15.169:0/1596016161 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f9a840ac230 con 0x7f9a9409a5c0 2023-10-26T01:41:54.532 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.523+0000 7f9a8b7fe640 1 -- 172.21.15.169:0/1596016161 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=156}) v3 -- 0x7f9a74072a30 con 0x7f9a9409a5c0 2023-10-26T01:41:54.532 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.523+0000 7f9a8bfff640 1 --2- 172.21.15.169:0/1596016161 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9a7406ae40 0x7f9a7406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:41:54.532 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.524+0000 7f9a8bfff640 1 --2- 172.21.15.169:0/1596016161 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9a7406ae40 0x7f9a7406d300 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c001270 tx=0x7f9a7c0077e0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:41:54.532 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.527+0000 7f9a8b7fe640 1 -- 172.21.15.169:0/1596016161 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9a84043060 con 0x7f9a9409a5c0 2023-10-26T01:41:54.754 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.752+0000 7f9a98a25640 1 -- 172.21.15.169:0/1596016161 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9a94073c10 con 0x7f9a9409a5c0 2023-10-26T01:41:54.755 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.753+0000 7f9a8b7fe640 1 -- 172.21.15.169:0/1596016161 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+4921 (secure 0 0 0) 0x7f9a84043030 con 0x7f9a9409a5c0 2023-10-26T01:41:54.756 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:41:54.756 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":77,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5711,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/3037106607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":3037106607},{"type":"v1","addr":"172.21.15.170:6836","nonce":3037106607}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72},{"gid":5714,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/1847866184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":1847866184},{"type":"v1","addr":"172.21.15.170:6837","nonce":1847866184}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72},{"gid":8229,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2707290348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2707290348},{"type":"v1","addr":"172.21.15.169:6833","nonce":2707290348}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T01:39:25.790062+0000","modified":"2023-10-26T01:39:28.841639+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8232},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8232":{"gid":8232,"name":"c","rank":0,"incarnation":73,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6835/66540264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":66540264},{"type":"v1","addr":"172.21.15.169:6835","nonce":66540264}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-10-26T01:41:54.757 INFO:teuthology.orchestra.run.smithi169.stderr:dumped fsmap epoch 77 2023-10-26T01:41:54.759 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.755+0000 7f9a98a25640 1 -- 172.21.15.169:0/1596016161 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9a7406ae40 msgr2=0x7f9a7406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:54.759 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.755+0000 7f9a98a25640 1 --2- 172.21.15.169:0/1596016161 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9a7406ae40 0x7f9a7406d300 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c001270 tx=0x7f9a7c0077e0 comp rx=0 tx=0).stop 2023-10-26T01:41:54.759 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.755+0000 7f9a98a25640 1 -- 172.21.15.169:0/1596016161 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a9409a5c0 msgr2=0x7f9a9413f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:54.759 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.755+0000 7f9a98a25640 1 --2- 172.21.15.169:0/1596016161 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a9409a5c0 0x7f9a9413f200 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f9a84036040 tx=0x7f9a8402daf0 comp rx=0 tx=0).stop 2023-10-26T01:41:54.759 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.756+0000 7f9a98a25640 1 -- 172.21.15.169:0/1596016161 shutdown_connections 2023-10-26T01:41:54.760 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.756+0000 7f9a98a25640 1 --2- 172.21.15.169:0/1596016161 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9a7406ae40 0x7f9a7406d300 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:54.760 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.756+0000 7f9a98a25640 1 --2- 172.21.15.169:0/1596016161 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9a940c9970 0x7f9a940aa0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:54.760 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.756+0000 7f9a98a25640 1 --2- 172.21.15.169:0/1596016161 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9a940ac6d0 0x7f9a941f51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:54.760 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.756+0000 7f9a98a25640 1 --2- 172.21.15.169:0/1596016161 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a9409a5c0 0x7f9a9413f200 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:54.760 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.756+0000 7f9a98a25640 1 -- 172.21.15.169:0/1596016161 >> 172.21.15.169:0/1596016161 conn(0x7f9a940aedd0 msgr2=0x7f9a941419c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:41:54.761 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.756+0000 7f9a98a25640 1 -- 172.21.15.169:0/1596016161 shutdown_connections 2023-10-26T01:41:54.761 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.756+0000 7f9a98a25640 1 -- 172.21.15.169:0/1596016161 wait complete. 2023-10-26T01:41:54.778 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T01:41:54.991 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.988+0000 7f64f9c94640 1 -- 172.21.15.169:0/2154733106 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64f415b610 msgr2=0x7f64f406ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:54.991 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.988+0000 7f64f9c94640 1 --2- 172.21.15.169:0/2154733106 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64f415b610 0x7f64f406ed30 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f64f406c960 tx=0x7f64e402be40 comp rx=0 tx=0).stop 2023-10-26T01:41:54.992 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.988+0000 7f64f9c94640 1 -- 172.21.15.169:0/2154733106 shutdown_connections 2023-10-26T01:41:54.992 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.988+0000 7f64f9c94640 1 --2- 172.21.15.169:0/2154733106 >> v2:172.21.15.170:3300/0 conn(0x7f64f40ac4e0 0x7f64f40cd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:54.992 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.988+0000 7f64f9c94640 1 --2- 172.21.15.169:0/2154733106 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64f415b610 0x7f64f406ed30 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:54.992 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.988+0000 7f64f9c94640 1 -- 172.21.15.169:0/2154733106 >> 172.21.15.169:0/2154733106 conn(0x7f64f4069930 msgr2=0x7f64f4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:41:54.992 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.988+0000 7f64f9c94640 1 -- 172.21.15.169:0/2154733106 shutdown_connections 2023-10-26T01:41:54.992 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.988+0000 7f64f9c94640 1 -- 172.21.15.169:0/2154733106 wait complete. 2023-10-26T01:41:54.993 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.990+0000 7f64f9c94640 1 Processor -- start 2023-10-26T01:41:54.993 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.990+0000 7f64f9c94640 1 -- start start 2023-10-26T01:41:54.995 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.992+0000 7f64f9c94640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f64f40ac4e0 0x7f64f41543a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:41:54.995 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.992+0000 7f64f9c94640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64f40ac940 0x7f64f4156e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:41:54.996 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.992+0000 7f64f9c94640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f64f415b610 0x7f64f4155a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:41:54.996 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.992+0000 7f64f9c94640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64f4156430 con 0x7f64f40ac940 2023-10-26T01:41:54.996 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.992+0000 7f64f9c94640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64f41548e0 con 0x7f64f415b610 2023-10-26T01:41:54.996 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.992+0000 7f64f9c94640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64f415c7d0 con 0x7f64f40ac4e0 2023-10-26T01:41:54.996 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.992+0000 7f64ebfff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64f40ac940 0x7f64f4156e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:41:54.997 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.992+0000 7f64ebfff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64f40ac940 0x7f64f4156e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:37588/0 (socket says 172.21.15.169:37588) 2023-10-26T01:41:54.997 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.992+0000 7f64ebfff640 1 -- 172.21.15.169:0/627546350 learned_addr learned my addr 172.21.15.169:0/627546350 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:41:54.997 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.992+0000 7f64f37fe640 1 --2- 172.21.15.169:0/627546350 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f64f40ac4e0 0x7f64f41543a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:41:54.997 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.992+0000 7f64f3fff640 1 --2- 172.21.15.169:0/627546350 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f64f415b610 0x7f64f4155a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:41:54.997 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.992+0000 7f64ebfff640 1 -- 172.21.15.169:0/627546350 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f64f40ac4e0 msgr2=0x7f64f41543a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:54.998 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.992+0000 7f64ebfff640 1 --2- 172.21.15.169:0/627546350 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f64f40ac4e0 0x7f64f41543a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:54.998 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.992+0000 7f64ebfff640 1 -- 172.21.15.169:0/627546350 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f64f415b610 msgr2=0x7f64f4155a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:54.998 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.992+0000 7f64ebfff640 1 --2- 172.21.15.169:0/627546350 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f64f415b610 0x7f64f4155a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:54.998 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.992+0000 7f64ebfff640 1 -- 172.21.15.169:0/627546350 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64e4009560 con 0x7f64f40ac940 2023-10-26T01:41:54.999 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.992+0000 7f64f3fff640 1 --2- 172.21.15.169:0/627546350 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f64f415b610 0x7f64f4155a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:41:54.999 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.993+0000 7f64ebfff640 1 --2- 172.21.15.169:0/627546350 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64f40ac940 0x7f64f4156e70 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f64dc00b4a0 tx=0x7f64dc00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:41:55.000 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.993+0000 7f64f0ff9640 1 -- 172.21.15.169:0/627546350 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64dc010610 con 0x7f64f40ac940 2023-10-26T01:41:55.000 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.993+0000 7f64f0ff9640 1 -- 172.21.15.169:0/627546350 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f64dc010860 con 0x7f64f40ac940 2023-10-26T01:41:55.001 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.993+0000 7f64f0ff9640 1 -- 172.21.15.169:0/627546350 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64dc010c70 con 0x7f64f40ac940 2023-10-26T01:41:55.001 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.993+0000 7f64f9c94640 1 -- 172.21.15.169:0/627546350 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64f415cab0 con 0x7f64f40ac940 2023-10-26T01:41:55.001 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.993+0000 7f64f9c94640 1 -- 172.21.15.169:0/627546350 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64f415d070 con 0x7f64f40ac940 2023-10-26T01:41:55.001 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.995+0000 7f64f9c94640 1 -- 172.21.15.169:0/627546350 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64f404db10 con 0x7f64f40ac940 2023-10-26T01:41:55.001 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.996+0000 7f64f0ff9640 1 -- 172.21.15.169:0/627546350 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f64dc017360 con 0x7f64f40ac940 2023-10-26T01:41:55.002 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.997+0000 7f64f0ff9640 1 --2- 172.21.15.169:0/627546350 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f64d406ae40 0x7f64d406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:41:55.002 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.997+0000 7f64f0ff9640 1 -- 172.21.15.169:0/627546350 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f64dc089530 con 0x7f64f40ac940 2023-10-26T01:41:55.002 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.997+0000 7f64f0ff9640 1 -- 172.21.15.169:0/627546350 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=156}) v3 -- 0x7f64d4072a30 con 0x7f64f40ac940 2023-10-26T01:41:55.002 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.998+0000 7f64f37fe640 1 --2- 172.21.15.169:0/627546350 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f64d406ae40 0x7f64d406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:41:55.003 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:54.999+0000 7f64f37fe640 1 --2- 172.21.15.169:0/627546350 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f64d406ae40 0x7f64d406d300 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f64e4035000 tx=0x7f64e4004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:41:55.004 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:55.002+0000 7f64f0ff9640 1 -- 172.21.15.169:0/627546350 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f64dc058030 con 0x7f64f40ac940 2023-10-26T01:41:55.229 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:55.227+0000 7f64f9c94640 1 -- 172.21.15.169:0/627546350 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f64f406c990 con 0x7f64f40ac940 2023-10-26T01:41:55.230 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:55.227+0000 7f64f0ff9640 1 -- 172.21.15.169:0/627546350 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+4921 (secure 0 0 0) 0x7f64dc014090 con 0x7f64f40ac940 2023-10-26T01:41:55.231 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:41:55.231 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":77,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5711,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/3037106607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":3037106607},{"type":"v1","addr":"172.21.15.170:6836","nonce":3037106607}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72},{"gid":5714,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/1847866184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":1847866184},{"type":"v1","addr":"172.21.15.170:6837","nonce":1847866184}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72},{"gid":8229,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2707290348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2707290348},{"type":"v1","addr":"172.21.15.169:6833","nonce":2707290348}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T01:39:25.790062+0000","modified":"2023-10-26T01:39:28.841639+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8232},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8232":{"gid":8232,"name":"c","rank":0,"incarnation":73,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6835/66540264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":66540264},{"type":"v1","addr":"172.21.15.169:6835","nonce":66540264}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-10-26T01:41:55.231 INFO:teuthology.orchestra.run.smithi169.stderr:dumped fsmap epoch 77 2023-10-26T01:41:55.233 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:55.230+0000 7f64f9c94640 1 -- 172.21.15.169:0/627546350 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f64d406ae40 msgr2=0x7f64d406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:55.233 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:55.230+0000 7f64f9c94640 1 --2- 172.21.15.169:0/627546350 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f64d406ae40 0x7f64d406d300 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f64e4035000 tx=0x7f64e4004730 comp rx=0 tx=0).stop 2023-10-26T01:41:55.233 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:55.230+0000 7f64f9c94640 1 -- 172.21.15.169:0/627546350 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64f40ac940 msgr2=0x7f64f4156e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:55.233 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:55.230+0000 7f64f9c94640 1 --2- 172.21.15.169:0/627546350 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64f40ac940 0x7f64f4156e70 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f64dc00b4a0 tx=0x7f64dc00b960 comp rx=0 tx=0).stop 2023-10-26T01:41:55.234 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:55.230+0000 7f64f9c94640 1 -- 172.21.15.169:0/627546350 shutdown_connections 2023-10-26T01:41:55.234 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:55.230+0000 7f64f9c94640 1 --2- 172.21.15.169:0/627546350 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f64d406ae40 0x7f64d406d300 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:55.234 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:55.230+0000 7f64f9c94640 1 --2- 172.21.15.169:0/627546350 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64f40ac940 0x7f64f4156e70 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:55.234 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:55.230+0000 7f64f9c94640 1 --2- 172.21.15.169:0/627546350 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f64f415b610 0x7f64f4155a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:55.235 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:55.230+0000 7f64f9c94640 1 --2- 172.21.15.169:0/627546350 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f64f40ac4e0 0x7f64f41543a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:55.235 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:55.230+0000 7f64f9c94640 1 -- 172.21.15.169:0/627546350 >> 172.21.15.169:0/627546350 conn(0x7f64f4069930 msgr2=0x7f64f40ace50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:41:55.235 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:55.230+0000 7f64f9c94640 1 -- 172.21.15.169:0/627546350 shutdown_connections 2023-10-26T01:41:55.235 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:55.230+0000 7f64f9c94640 1 -- 172.21.15.169:0/627546350 wait complete. 2023-10-26T01:41:55.249 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:41:55.448 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:55.445+0000 7fb8c4c97640 1 -- 172.21.15.169:0/95361222 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8c0158850 msgr2=0x7fb8c014e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:55.448 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:55.445+0000 7fb8c4c97640 1 --2- 172.21.15.169:0/95361222 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8c0158850 0x7fb8c014e790 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7fb8a8009830 tx=0x7fb8a802bcd0 comp rx=0 tx=0).stop 2023-10-26T01:41:55.449 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:55.445+0000 7fb8c4c97640 1 -- 172.21.15.169:0/95361222 shutdown_connections 2023-10-26T01:41:55.449 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:55.445+0000 7fb8c4c97640 1 --2- 172.21.15.169:0/95361222 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb8c0159e10 0x7fb8c015a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:55.449 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:55.445+0000 7fb8c4c97640 1 --2- 172.21.15.169:0/95361222 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8c0158850 0x7fb8c014e790 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:55.449 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:55.445+0000 7fb8c4c97640 1 -- 172.21.15.169:0/95361222 >> 172.21.15.169:0/95361222 conn(0x7fb8c0069930 msgr2=0x7fb8c0069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:41:55.449 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:55.445+0000 7fb8c4c97640 1 -- 172.21.15.169:0/95361222 shutdown_connections 2023-10-26T01:41:55.450 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:55.445+0000 7fb8c4c97640 1 -- 172.21.15.169:0/95361222 wait complete. 2023-10-26T01:41:55.450 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:55.447+0000 7fb8c4c97640 1 Processor -- start 2023-10-26T01:41:55.450 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:55.447+0000 7fb8c4c97640 1 -- start start 2023-10-26T01:41:55.451 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:55.448+0000 7fb8c4c97640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8c006ce20 0x7fb8c01f13c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:41:55.452 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:55.448+0000 7fb8c4c97640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb8c0158850 0x7fb8c0154480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:41:55.452 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:55.448+0000 7fb8c4c97640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb8c0159e10 0x7fb8c014c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:41:55.452 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:55.448+0000 7fb8c4c97640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c0157a40 con 0x7fb8c006ce20 2023-10-26T01:41:55.452 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:55.448+0000 7fb8c4c97640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c0153350 con 0x7fb8c0158850 2023-10-26T01:41:55.452 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:55.448+0000 7fb8c4c97640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c01535a0 con 0x7fb8c0159e10 2023-10-26T01:41:55.453 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:55.448+0000 7fb8be59c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8c006ce20 0x7fb8c01f13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:41:55.453 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:55.448+0000 7fb8be59c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8c006ce20 0x7fb8c01f13c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:37602/0 (socket says 172.21.15.169:37602) 2023-10-26T01:41:55.453 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:55.448+0000 7fb8be59c640 1 -- 172.21.15.169:0/2830746104 learned_addr learned my addr 172.21.15.169:0/2830746104 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:41:55.453 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:55.449+0000 7fb8be59c640 1 -- 172.21.15.169:0/2830746104 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb8c0159e10 msgr2=0x7fb8c014c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:55.453 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:55.449+0000 7fb8be59c640 1 --2- 172.21.15.169:0/2830746104 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb8c0159e10 0x7fb8c014c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:55.454 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:55.449+0000 7fb8bdd9b640 1 --2- 172.21.15.169:0/2830746104 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb8c0158850 0x7fb8c0154480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:41:55.454 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:55.449+0000 7fb8be59c640 1 -- 172.21.15.169:0/2830746104 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb8c0158850 msgr2=0x7fb8c0154480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:55.454 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:55.449+0000 7fb8be59c640 1 --2- 172.21.15.169:0/2830746104 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb8c0158850 0x7fb8c0154480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:55.454 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:55.449+0000 7fb8be59c640 1 -- 172.21.15.169:0/2830746104 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8a8009490 con 0x7fb8c006ce20 2023-10-26T01:41:55.454 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:55.449+0000 7fb8be59c640 1 --2- 172.21.15.169:0/2830746104 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8c006ce20 0x7fb8c01f13c0 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7fb8b000c430 tx=0x7fb8b000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:41:55.456 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:55.449+0000 7fb8b6ffd640 1 -- 172.21.15.169:0/2830746104 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b0004230 con 0x7fb8c006ce20 2023-10-26T01:41:55.456 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:55.449+0000 7fb8b6ffd640 1 -- 172.21.15.169:0/2830746104 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb8b00044a0 con 0x7fb8c006ce20 2023-10-26T01:41:55.456 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:55.449+0000 7fb8b6ffd640 1 -- 172.21.15.169:0/2830746104 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b0003850 con 0x7fb8c006ce20 2023-10-26T01:41:55.456 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:55.449+0000 7fb8c4c97640 1 -- 172.21.15.169:0/2830746104 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8c0153880 con 0x7fb8c006ce20 2023-10-26T01:41:55.457 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:55.450+0000 7fb8c4c97640 1 -- 172.21.15.169:0/2830746104 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8c0153e40 con 0x7fb8c006ce20 2023-10-26T01:41:55.457 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:55.451+0000 7fb8c4c97640 1 -- 172.21.15.169:0/2830746104 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8c004db10 con 0x7fb8c006ce20 2023-10-26T01:41:55.457 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:55.452+0000 7fb8b6ffd640 1 -- 172.21.15.169:0/2830746104 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fb8b00039f0 con 0x7fb8c006ce20 2023-10-26T01:41:55.457 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:55.452+0000 7fb8b6ffd640 1 --2- 172.21.15.169:0/2830746104 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb8a006ae40 0x7fb8a006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:41:55.457 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:55.453+0000 7fb8b6ffd640 1 -- 172.21.15.169:0/2830746104 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fb8b0088ec0 con 0x7fb8c006ce20 2023-10-26T01:41:55.458 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:55.453+0000 7fb8b6ffd640 1 -- 172.21.15.169:0/2830746104 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=156}) v3 -- 0x7fb8a0072a30 con 0x7fb8c006ce20 2023-10-26T01:41:55.458 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:55.453+0000 7fb8bdd9b640 1 --2- 172.21.15.169:0/2830746104 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb8a006ae40 0x7fb8a006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:41:55.458 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:55.454+0000 7fb8bdd9b640 1 --2- 172.21.15.169:0/2830746104 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb8a006ae40 0x7fb8a006d300 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fb8a8036000 tx=0x7fb8a8004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:41:55.459 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:55.457+0000 7fb8b6ffd640 1 -- 172.21.15.169:0/2830746104 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb8b0017060 con 0x7fb8c006ce20 2023-10-26T01:41:55.637 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:55.634+0000 7fb8c4c97640 1 -- 172.21.15.169:0/2830746104 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb8c00aca70 con 0x7fb8c006ce20 2023-10-26T01:41:55.639 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:55.636+0000 7fb8b6ffd640 1 -- 172.21.15.169:0/2830746104 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v155) v1 ==== 74+0+16695 (secure 0 0 0) 0x7fb8b0057a80 con 0x7fb8c006ce20 2023-10-26T01:41:55.639 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:41:55.640 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":155,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:41:08.113301+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-10-26T01:39:23.646624+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"150","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-10-26T01:39:24.813678+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"19.5","mappings":[{"from":2,"to":0}]},{"pgid":"19.13","mappings":[{"from":2,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:41:55.642 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:55.640+0000 7fb8c4c97640 1 -- 172.21.15.169:0/2830746104 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb8a006ae40 msgr2=0x7fb8a006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:55.643 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:55.640+0000 7fb8c4c97640 1 --2- 172.21.15.169:0/2830746104 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb8a006ae40 0x7fb8a006d300 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fb8a8036000 tx=0x7fb8a8004730 comp rx=0 tx=0).stop 2023-10-26T01:41:55.643 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:55.640+0000 7fb8c4c97640 1 -- 172.21.15.169:0/2830746104 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8c006ce20 msgr2=0x7fb8c01f13c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:55.643 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:55.640+0000 7fb8c4c97640 1 --2- 172.21.15.169:0/2830746104 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8c006ce20 0x7fb8c01f13c0 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7fb8b000c430 tx=0x7fb8b000c8f0 comp rx=0 tx=0).stop 2023-10-26T01:41:55.643 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:55.640+0000 7fb8c4c97640 1 -- 172.21.15.169:0/2830746104 shutdown_connections 2023-10-26T01:41:55.643 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:55.640+0000 7fb8c4c97640 1 --2- 172.21.15.169:0/2830746104 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb8a006ae40 0x7fb8a006d300 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:55.644 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:55.640+0000 7fb8c4c97640 1 --2- 172.21.15.169:0/2830746104 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb8c0159e10 0x7fb8c014c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:55.644 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:55.640+0000 7fb8c4c97640 1 --2- 172.21.15.169:0/2830746104 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb8c0158850 0x7fb8c0154480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:55.644 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:55.640+0000 7fb8c4c97640 1 --2- 172.21.15.169:0/2830746104 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8c006ce20 0x7fb8c01f13c0 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:55.644 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:55.640+0000 7fb8c4c97640 1 -- 172.21.15.169:0/2830746104 >> 172.21.15.169:0/2830746104 conn(0x7fb8c0069930 msgr2=0x7fb8c014d5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:41:55.644 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:55.640+0000 7fb8c4c97640 1 -- 172.21.15.169:0/2830746104 shutdown_connections 2023-10-26T01:41:55.644 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:55.640+0000 7fb8c4c97640 1 -- 172.21.15.169:0/2830746104 wait complete. 2023-10-26T01:41:55.661 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-26T01:41:55.864 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:55.861+0000 7f3d1b461640 1 -- 172.21.15.169:0/2105683917 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3d141597c0 msgr2=0x7f3d140ac200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:55.865 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:55.861+0000 7f3d1b461640 1 --2- 172.21.15.169:0/2105683917 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3d141597c0 0x7f3d140ac200 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f3d140cd2c0 tx=0x7f3d0402bdb0 comp rx=0 tx=0).stop 2023-10-26T01:41:55.865 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:55.861+0000 7f3d1b461640 1 -- 172.21.15.169:0/2105683917 shutdown_connections 2023-10-26T01:41:55.865 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:55.861+0000 7f3d1b461640 1 --2- 172.21.15.169:0/2105683917 >> v2:172.21.15.170:3300/0 conn(0x7f3d140b0c80 0x7f3d140cd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:55.865 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:55.861+0000 7f3d1b461640 1 --2- 172.21.15.169:0/2105683917 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3d141597c0 0x7f3d140ac200 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:55.865 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:55.861+0000 7f3d1b461640 1 -- 172.21.15.169:0/2105683917 >> 172.21.15.169:0/2105683917 conn(0x7f3d14069930 msgr2=0x7f3d14069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:41:55.866 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:55.861+0000 7f3d1b461640 1 -- 172.21.15.169:0/2105683917 shutdown_connections 2023-10-26T01:41:55.866 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:55.861+0000 7f3d1b461640 1 -- 172.21.15.169:0/2105683917 wait complete. 2023-10-26T01:41:55.867 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:55.863+0000 7f3d1b461640 1 Processor -- start 2023-10-26T01:41:55.867 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:55.863+0000 7f3d1b461640 1 -- start start 2023-10-26T01:41:55.867 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:55.864+0000 7f3d1b461640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3d140b0c80 0x7f3d14148740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:41:55.868 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:55.864+0000 7f3d1b461640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3d141593e0 0x7f3d140af920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:41:55.868 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:55.864+0000 7f3d1b461640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3d141597c0 0x7f3d140c6660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:41:55.868 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:55.864+0000 7f3d1b461640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d14148c80 con 0x7f3d141597c0 2023-10-26T01:41:55.868 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:55.864+0000 7f3d1b461640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d140ae880 con 0x7f3d141593e0 2023-10-26T01:41:55.868 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:55.864+0000 7f3d1b461640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d140aeb60 con 0x7f3d140b0c80 2023-10-26T01:41:55.868 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:55.864+0000 7f3d199fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3d141597c0 0x7f3d140c6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:41:55.869 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:55.864+0000 7f3d199fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3d141597c0 0x7f3d140c6660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:37624/0 (socket says 172.21.15.169:37624) 2023-10-26T01:41:55.869 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:55.864+0000 7f3d199fe640 1 -- 172.21.15.169:0/4224551520 learned_addr learned my addr 172.21.15.169:0/4224551520 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:41:55.870 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:55.865+0000 7f3d189fc640 1 --2- 172.21.15.169:0/4224551520 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3d141593e0 0x7f3d140af920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:41:55.870 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:55.865+0000 7f3d191fd640 1 --2- 172.21.15.169:0/4224551520 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3d140b0c80 0x7f3d14148740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:41:55.870 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:55.865+0000 7f3d199fe640 1 -- 172.21.15.169:0/4224551520 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3d140b0c80 msgr2=0x7f3d14148740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:55.871 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:55.865+0000 7f3d199fe640 1 --2- 172.21.15.169:0/4224551520 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3d140b0c80 0x7f3d14148740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:55.871 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:55.865+0000 7f3d199fe640 1 -- 172.21.15.169:0/4224551520 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3d141593e0 msgr2=0x7f3d140af920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:55.871 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:55.865+0000 7f3d199fe640 1 --2- 172.21.15.169:0/4224551520 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3d141593e0 0x7f3d140af920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:55.871 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:55.865+0000 7f3d199fe640 1 -- 172.21.15.169:0/4224551520 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d04009560 con 0x7f3d141597c0 2023-10-26T01:41:55.871 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:55.865+0000 7f3d199fe640 1 --2- 172.21.15.169:0/4224551520 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3d141597c0 0x7f3d140c6660 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c00eaa0 tx=0x7f3d0c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:41:55.871 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:55.865+0000 7f3d09ffb640 1 -- 172.21.15.169:0/4224551520 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d0c00cdb0 con 0x7f3d141597c0 2023-10-26T01:41:55.872 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:55.865+0000 7f3d09ffb640 1 -- 172.21.15.169:0/4224551520 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3d0c004b20 con 0x7f3d141597c0 2023-10-26T01:41:55.872 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:55.865+0000 7f3d09ffb640 1 -- 172.21.15.169:0/4224551520 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d0c005020 con 0x7f3d141597c0 2023-10-26T01:41:55.872 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:55.866+0000 7f3d1b461640 1 -- 172.21.15.169:0/4224551520 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d140aee40 con 0x7f3d141597c0 2023-10-26T01:41:55.872 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:55.866+0000 7f3d1b461640 1 -- 172.21.15.169:0/4224551520 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d140af400 con 0x7f3d141597c0 2023-10-26T01:41:55.872 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:55.867+0000 7f3d1b461640 1 -- 172.21.15.169:0/4224551520 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d1404db10 con 0x7f3d141597c0 2023-10-26T01:41:55.873 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:55.868+0000 7f3d09ffb640 1 -- 172.21.15.169:0/4224551520 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f3d0c005180 con 0x7f3d141597c0 2023-10-26T01:41:55.873 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:55.868+0000 7f3d09ffb640 1 --2- 172.21.15.169:0/4224551520 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3cf806ae40 0x7f3cf806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:41:55.874 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:55.869+0000 7f3d09ffb640 1 -- 172.21.15.169:0/4224551520 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f3d0c08a010 con 0x7f3d141597c0 2023-10-26T01:41:55.874 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:55.869+0000 7f3d09ffb640 1 -- 172.21.15.169:0/4224551520 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=156}) v3 -- 0x7f3cf8072a30 con 0x7f3d141597c0 2023-10-26T01:41:55.874 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:55.869+0000 7f3d191fd640 1 --2- 172.21.15.169:0/4224551520 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3cf806ae40 0x7f3cf806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:41:55.874 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:55.870+0000 7f3d191fd640 1 --2- 172.21.15.169:0/4224551520 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3cf806ae40 0x7f3cf806d300 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f3d10004ea0 tx=0x7f3d10009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:41:55.875 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:55.873+0000 7f3d09ffb640 1 -- 172.21.15.169:0/4224551520 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3d0c01f020 con 0x7f3d141597c0 2023-10-26T01:41:56.086 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:56.083+0000 7f3d1b461640 1 -- 172.21.15.169:0/4224551520 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f3d140af5f0 con 0x7f3d141597c0 2023-10-26T01:41:56.087 INFO:tasks.ceph.mon.a.smithi169.stderr:2023-10-26T01:41:56.084+0000 7f1588a74640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-26T01:41:56.090 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:56.088+0000 7f3d09ffb640 1 -- 172.21.15.169:0/4224551520 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v78) v1 ==== 161+0+0 (secure 0 0 0) 0x7f3d0c055110 con 0x7f3d141597c0 2023-10-26T01:41:56.091 INFO:teuthology.orchestra.run.smithi169.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-26T01:41:56.091 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:56.089+0000 7f3d09ffb640 1 -- 172.21.15.169:0/4224551520 <== mon.0 v2:172.21.15.169:3300/0 8 ==== osd_map(156..156 src has 1..156) v4 ==== 432+0+0 (secure 0 0 0) 0x7f3d0c0594d0 con 0x7f3d141597c0 2023-10-26T01:41:56.091 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:56.089+0000 7f3d09ffb640 1 -- 172.21.15.169:0/4224551520 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=157}) v3 -- 0x7f3cf80733d0 con 0x7f3d141597c0 2023-10-26T01:41:56.094 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:56.092+0000 7f3d1b461640 1 -- 172.21.15.169:0/4224551520 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3cf806ae40 msgr2=0x7f3cf806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:56.095 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:56.092+0000 7f3d1b461640 1 --2- 172.21.15.169:0/4224551520 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3cf806ae40 0x7f3cf806d300 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f3d10004ea0 tx=0x7f3d10009040 comp rx=0 tx=0).stop 2023-10-26T01:41:56.095 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:56.092+0000 7f3d1b461640 1 -- 172.21.15.169:0/4224551520 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3d141597c0 msgr2=0x7f3d140c6660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:56.095 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:56.092+0000 7f3d1b461640 1 --2- 172.21.15.169:0/4224551520 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3d141597c0 0x7f3d140c6660 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c00eaa0 tx=0x7f3d0c00ef60 comp rx=0 tx=0).stop 2023-10-26T01:41:56.097 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:56.095+0000 7f3d1b461640 1 -- 172.21.15.169:0/4224551520 shutdown_connections 2023-10-26T01:41:56.097 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:56.095+0000 7f3d1b461640 1 --2- 172.21.15.169:0/4224551520 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3cf806ae40 0x7f3cf806d300 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:56.098 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:56.095+0000 7f3d1b461640 1 --2- 172.21.15.169:0/4224551520 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3d141597c0 0x7f3d140c6660 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:56.098 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:56.095+0000 7f3d1b461640 1 --2- 172.21.15.169:0/4224551520 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3d141593e0 0x7f3d140af920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:56.098 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:56.095+0000 7f3d1b461640 1 --2- 172.21.15.169:0/4224551520 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3d140b0c80 0x7f3d14148740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:56.098 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:56.095+0000 7f3d1b461640 1 -- 172.21.15.169:0/4224551520 >> 172.21.15.169:0/4224551520 conn(0x7f3d14069930 msgr2=0x7f3d14159f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:41:56.099 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:56.095+0000 7f3d1b461640 1 -- 172.21.15.169:0/4224551520 shutdown_connections 2023-10-26T01:41:56.099 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:56.096+0000 7f3d1b461640 1 -- 172.21.15.169:0/4224551520 wait complete. 2023-10-26T01:41:56.118 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-26T01:41:56.176 INFO:tasks.ceph.mds.c.smithi169.stdout:starting mds.c at 2023-10-26T01:41:56.305 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:56.301+0000 7ff15d821640 1 -- 172.21.15.169:0/1347076189 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1440039c0 msgr2=0x7ff144003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:56.305 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:56.301+0000 7ff15d821640 1 --2- 172.21.15.169:0/1347076189 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1440039c0 0x7ff144003dc0 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7ff148006ef0 tx=0x7ff1440057f0 comp rx=0 tx=0).stop 2023-10-26T01:41:56.305 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:56.302+0000 7ff15d821640 1 -- 172.21.15.169:0/1347076189 shutdown_connections 2023-10-26T01:41:56.306 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:56.302+0000 7ff15d821640 1 --2- 172.21.15.169:0/1347076189 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1440039c0 0x7ff144003dc0 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:56.306 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:56.302+0000 7ff15d821640 1 -- 172.21.15.169:0/1347076189 >> 172.21.15.169:0/1347076189 conn(0x7ff158069930 msgr2=0x7ff158069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:41:56.306 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:56.302+0000 7ff15d821640 1 -- 172.21.15.169:0/1347076189 shutdown_connections 2023-10-26T01:41:56.306 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:56.302+0000 7ff15d821640 1 -- 172.21.15.169:0/1347076189 wait complete. 2023-10-26T01:41:56.309 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:56.303+0000 7ff15d821640 1 Processor -- start 2023-10-26T01:41:56.309 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:56.303+0000 7ff15d821640 1 -- start start 2023-10-26T01:41:56.309 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:56.304+0000 7ff15d821640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff1440039c0 0x7ff1580ca0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:41:56.309 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:56.304+0000 7ff15d821640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff15814f480 0x7ff158159620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:41:56.310 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:56.304+0000 7ff15d821640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff1580cb760 0x7ff1581e6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:41:56.310 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:56.304+0000 7ff15d821640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1580cb2d0 con 0x7ff15814f480 2023-10-26T01:41:56.310 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:56.304+0000 7ff15d821640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1580ca6c0 con 0x7ff1440039c0 2023-10-26T01:41:56.310 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:56.304+0000 7ff15d821640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1580ca9a0 con 0x7ff1580cb760 2023-10-26T01:41:56.310 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:56.305+0000 7ff14e7fc640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff15814f480 0x7ff158159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:41:56.311 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:56.305+0000 7ff14e7fc640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff15814f480 0x7ff158159620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:37668/0 (socket says 172.21.15.169:37668) 2023-10-26T01:41:56.311 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:56.305+0000 7ff14e7fc640 1 -- 172.21.15.169:0/2790422713 learned_addr learned my addr 172.21.15.169:0/2790422713 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:41:56.311 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:56.305+0000 7ff156ffd640 1 --2- 172.21.15.169:0/2790422713 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff1440039c0 0x7ff1580ca0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:41:56.311 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:56.305+0000 7ff1577fe640 1 --2- 172.21.15.169:0/2790422713 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff1580cb760 0x7ff1581e6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:41:56.312 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:56.305+0000 7ff14e7fc640 1 -- 172.21.15.169:0/2790422713 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff1580cb760 msgr2=0x7ff1581e6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:56.312 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:56.305+0000 7ff14e7fc640 1 --2- 172.21.15.169:0/2790422713 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff1580cb760 0x7ff1581e6260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:56.312 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:56.305+0000 7ff14e7fc640 1 -- 172.21.15.169:0/2790422713 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff1440039c0 msgr2=0x7ff1580ca0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:56.312 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:56.305+0000 7ff14e7fc640 1 --2- 172.21.15.169:0/2790422713 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff1440039c0 0x7ff1580ca0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:56.312 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:56.305+0000 7ff14e7fc640 1 -- 172.21.15.169:0/2790422713 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1580cac80 con 0x7ff15814f480 2023-10-26T01:41:56.313 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:56.305+0000 7ff156ffd640 1 --2- 172.21.15.169:0/2790422713 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff1440039c0 0x7ff1580ca0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:41:56.313 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:56.305+0000 7ff14e7fc640 1 --2- 172.21.15.169:0/2790422713 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff15814f480 0x7ff158159620 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7ff14000d730 tx=0x7ff14000dbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:41:56.313 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:56.306+0000 7ff154ff9640 1 -- 172.21.15.169:0/2790422713 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff14000de40 con 0x7ff15814f480 2023-10-26T01:41:56.313 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:56.306+0000 7ff154ff9640 1 -- 172.21.15.169:0/2790422713 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff14000b8d0 con 0x7ff15814f480 2023-10-26T01:41:56.314 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:56.306+0000 7ff154ff9640 1 -- 172.21.15.169:0/2790422713 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff14000bce0 con 0x7ff15814f480 2023-10-26T01:41:56.314 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:56.306+0000 7ff15d821640 1 -- 172.21.15.169:0/2790422713 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1580caf60 con 0x7ff15814f480 2023-10-26T01:41:56.314 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:56.306+0000 7ff15d821640 1 -- 172.21.15.169:0/2790422713 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1580cbbe0 con 0x7ff15814f480 2023-10-26T01:41:56.315 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:56.307+0000 7ff15d821640 1 -- 172.21.15.169:0/2790422713 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1580cb120 con 0x7ff15814f480 2023-10-26T01:41:56.315 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:56.308+0000 7ff154ff9640 1 -- 172.21.15.169:0/2790422713 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ff14001b400 con 0x7ff15814f480 2023-10-26T01:41:56.316 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:56.309+0000 7ff154ff9640 1 --2- 172.21.15.169:0/2790422713 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff13806ae40 0x7ff13806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:41:56.316 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:56.309+0000 7ff154ff9640 1 -- 172.21.15.169:0/2790422713 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 5853+0+0 (secure 0 0 0) 0x7ff140089b70 con 0x7ff15814f480 2023-10-26T01:41:56.316 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:56.309+0000 7ff154ff9640 1 -- 172.21.15.169:0/2790422713 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=157}) v3 -- 0x7ff138072b40 con 0x7ff15814f480 2023-10-26T01:41:56.316 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:56.310+0000 7ff156ffd640 1 --2- 172.21.15.169:0/2790422713 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff13806ae40 0x7ff13806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:41:56.316 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:56.311+0000 7ff156ffd640 1 --2- 172.21.15.169:0/2790422713 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff13806ae40 0x7ff13806d300 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7ff148034000 tx=0x7ff14802e070 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:41:56.317 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:56.314+0000 7ff154ff9640 1 -- 172.21.15.169:0/2790422713 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff140016020 con 0x7ff15814f480 2023-10-26T01:41:56.543 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:56.540+0000 7ff15d821640 1 -- 172.21.15.169:0/2790422713 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7ff15804db10 con 0x7ff15814f480 2023-10-26T01:41:57.108 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:57.105+0000 7ff154ff9640 1 -- 172.21.15.169:0/2790422713 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v79) v1 ==== 104+0+0 (secure 0 0 0) 0x7ff140054b90 con 0x7ff15814f480 2023-10-26T01:41:57.111 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:57.108+0000 7ff15d821640 1 -- 172.21.15.169:0/2790422713 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff13806ae40 msgr2=0x7ff13806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:57.112 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:57.108+0000 7ff15d821640 1 --2- 172.21.15.169:0/2790422713 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff13806ae40 0x7ff13806d300 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7ff148034000 tx=0x7ff14802e070 comp rx=0 tx=0).stop 2023-10-26T01:41:57.112 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:57.108+0000 7ff15d821640 1 -- 172.21.15.169:0/2790422713 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff15814f480 msgr2=0x7ff158159620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:57.112 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:57.108+0000 7ff15d821640 1 --2- 172.21.15.169:0/2790422713 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff15814f480 0x7ff158159620 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7ff14000d730 tx=0x7ff14000dbf0 comp rx=0 tx=0).stop 2023-10-26T01:41:57.112 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:57.108+0000 7ff15d821640 1 -- 172.21.15.169:0/2790422713 shutdown_connections 2023-10-26T01:41:57.112 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:57.108+0000 7ff15d821640 1 --2- 172.21.15.169:0/2790422713 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff13806ae40 0x7ff13806d300 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:57.113 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:57.108+0000 7ff15d821640 1 --2- 172.21.15.169:0/2790422713 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff15814f480 0x7ff158159620 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:57.113 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:57.108+0000 7ff15d821640 1 --2- 172.21.15.169:0/2790422713 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff1580cb760 0x7ff1581e6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:57.113 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:57.108+0000 7ff15d821640 1 --2- 172.21.15.169:0/2790422713 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff1440039c0 0x7ff1580ca0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:57.113 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:57.108+0000 7ff15d821640 1 -- 172.21.15.169:0/2790422713 >> 172.21.15.169:0/2790422713 conn(0x7ff158069930 msgr2=0x7ff15806ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:41:57.114 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:57.109+0000 7ff15d821640 1 -- 172.21.15.169:0/2790422713 shutdown_connections 2023-10-26T01:41:57.114 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:57.109+0000 7ff15d821640 1 -- 172.21.15.169:0/2790422713 wait complete. 2023-10-26T01:41:57.132 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-26T01:41:57.347 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:57.344+0000 7fecc27ce640 1 -- 172.21.15.169:0/3373655153 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feca8004ab0 msgr2=0x7feca8004eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:57.347 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:57.344+0000 7fecc27ce640 1 --2- 172.21.15.169:0/3373655153 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feca8004ab0 0x7feca8004eb0 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7fecac001d80 tx=0x7feca8006900 comp rx=0 tx=0).stop 2023-10-26T01:41:57.347 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:57.344+0000 7fecc0d6b640 1 -- 172.21.15.169:0/3373655153 reap_dead start 2023-10-26T01:41:57.348 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:57.344+0000 7fecc27ce640 1 -- 172.21.15.169:0/3373655153 shutdown_connections 2023-10-26T01:41:57.348 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:57.344+0000 7fecc27ce640 1 -- 172.21.15.169:0/3373655153 >> 172.21.15.169:0/3373655153 conn(0x7fecbc0aeee0 msgr2=0x7fecbc0af2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:41:57.348 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:57.344+0000 7fecc27ce640 1 -- 172.21.15.169:0/3373655153 shutdown_connections 2023-10-26T01:41:57.348 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:57.344+0000 7fecc27ce640 1 -- 172.21.15.169:0/3373655153 wait complete. 2023-10-26T01:41:57.349 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:57.346+0000 7fecc27ce640 1 Processor -- start 2023-10-26T01:41:57.349 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:57.346+0000 7fecc27ce640 1 -- start start 2023-10-26T01:41:57.350 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:57.347+0000 7fecc27ce640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feca8004ab0 0x7fecbc1f3410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:41:57.351 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:57.347+0000 7fecc27ce640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fecbc1e39c0 0x7fecbc14c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:41:57.351 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:57.347+0000 7fecc27ce640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fecbc06ecc0 0x7fecbc1f4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:41:57.351 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:57.347+0000 7fecc27ce640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecbc1f3c40 con 0x7feca8004ab0 2023-10-26T01:41:57.351 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:57.347+0000 7fecc27ce640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecbc1f3f20 con 0x7fecbc06ecc0 2023-10-26T01:41:57.351 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:57.347+0000 7fecc27ce640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecbc1f4200 con 0x7fecbc1e39c0 2023-10-26T01:41:57.352 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:57.348+0000 7fecbbfff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feca8004ab0 0x7fecbc1f3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:41:57.352 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:57.348+0000 7fecbbfff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feca8004ab0 0x7fecbc1f3410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:37692/0 (socket says 172.21.15.169:37692) 2023-10-26T01:41:57.352 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:57.348+0000 7fecbbfff640 1 -- 172.21.15.169:0/530079216 learned_addr learned my addr 172.21.15.169:0/530079216 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:41:57.352 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:57.348+0000 7fecc0d6b640 1 --2- 172.21.15.169:0/530079216 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fecbc06ecc0 0x7fecbc1f4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:41:57.352 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:57.348+0000 7fecbbfff640 1 -- 172.21.15.169:0/530079216 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fecbc1e39c0 msgr2=0x7fecbc14c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:57.353 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:57.348+0000 7fecbbfff640 1 --2- 172.21.15.169:0/530079216 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fecbc1e39c0 0x7fecbc14c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:57.353 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:57.348+0000 7fecbbfff640 1 -- 172.21.15.169:0/530079216 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fecbc06ecc0 msgr2=0x7fecbc1f4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:57.353 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:57.348+0000 7fecbbfff640 1 --2- 172.21.15.169:0/530079216 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fecbc06ecc0 0x7fecbc1f4ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:57.353 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:57.348+0000 7fecbbfff640 1 -- 172.21.15.169:0/530079216 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecbc1f44e0 con 0x7feca8004ab0 2023-10-26T01:41:57.355 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:57.348+0000 7fecbbfff640 1 --2- 172.21.15.169:0/530079216 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feca8004ab0 0x7fecbc1f3410 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7fecac001cd0 tx=0x7fecac02cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:41:57.355 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:57.348+0000 7fecb8ff9640 1 -- 172.21.15.169:0/530079216 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecac006440 con 0x7feca8004ab0 2023-10-26T01:41:57.355 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:57.349+0000 7fecb8ff9640 1 -- 172.21.15.169:0/530079216 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fecac006690 con 0x7feca8004ab0 2023-10-26T01:41:57.355 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:57.349+0000 7fecb8ff9640 1 -- 172.21.15.169:0/530079216 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecac02ce80 con 0x7feca8004ab0 2023-10-26T01:41:57.356 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:57.349+0000 7fecc27ce640 1 -- 172.21.15.169:0/530079216 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecbc1f47c0 con 0x7feca8004ab0 2023-10-26T01:41:57.356 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:57.349+0000 7fecc27ce640 1 -- 172.21.15.169:0/530079216 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecbc14cdc0 con 0x7feca8004ab0 2023-10-26T01:41:57.356 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:57.350+0000 7fecc27ce640 1 -- 172.21.15.169:0/530079216 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecbc066af0 con 0x7feca8004ab0 2023-10-26T01:41:57.356 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:57.351+0000 7fecb8ff9640 1 -- 172.21.15.169:0/530079216 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fecac03d070 con 0x7feca8004ab0 2023-10-26T01:41:57.356 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:57.352+0000 7fecb8ff9640 1 --2- 172.21.15.169:0/530079216 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fec9c06ae70 0x7fec9c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:41:57.356 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:57.352+0000 7fecb8ff9640 1 -- 172.21.15.169:0/530079216 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fecac0ab310 con 0x7feca8004ab0 2023-10-26T01:41:57.357 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:57.352+0000 7fecb8ff9640 1 -- 172.21.15.169:0/530079216 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=157}) v3 -- 0x7fec9c072b70 con 0x7feca8004ab0 2023-10-26T01:41:57.357 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:57.353+0000 7fecbb7fe640 1 --2- 172.21.15.169:0/530079216 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fec9c06ae70 0x7fec9c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:41:57.357 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:57.354+0000 7fecbb7fe640 1 --2- 172.21.15.169:0/530079216 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fec9c06ae70 0x7fec9c06d330 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fecbc0ab520 tx=0x7feca4001500 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:41:57.360 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:57.357+0000 7fecb8ff9640 1 -- 172.21.15.169:0/530079216 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fecac045060 con 0x7feca8004ab0 2023-10-26T01:41:57.552 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:57.549+0000 7fecc27ce640 1 -- 172.21.15.169:0/530079216 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fecbc04da70 con 0x7feca8004ab0 2023-10-26T01:41:58.140 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:58.137+0000 7fecb8ff9640 1 -- 172.21.15.169:0/530079216 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v157) v1 ==== 181+0+0 (secure 0 0 0) 0x7fecac045030 con 0x7feca8004ab0 2023-10-26T01:41:58.140 INFO:teuthology.orchestra.run.smithi169.stderr:pool 'cephfs_metadata' removed 2023-10-26T01:41:58.141 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:58.138+0000 7fecb8ff9640 1 -- 172.21.15.169:0/530079216 <== mon.0 v2:172.21.15.169:3300/0 8 ==== osd_map(157..157 src has 1..157) v4 ==== 745+0+0 (secure 0 0 0) 0x7fecac07a710 con 0x7feca8004ab0 2023-10-26T01:41:58.141 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:58.138+0000 7fecb8ff9640 1 -- 172.21.15.169:0/530079216 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=158}) v3 -- 0x7fec9c070fa0 con 0x7feca8004ab0 2023-10-26T01:41:58.152 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:58.149+0000 7fecc27ce640 1 -- 172.21.15.169:0/530079216 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fec9c06ae70 msgr2=0x7fec9c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:58.152 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:58.149+0000 7fecc27ce640 1 --2- 172.21.15.169:0/530079216 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fec9c06ae70 0x7fec9c06d330 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fecbc0ab520 tx=0x7feca4001500 comp rx=0 tx=0).stop 2023-10-26T01:41:58.153 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:58.149+0000 7fecc27ce640 1 -- 172.21.15.169:0/530079216 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feca8004ab0 msgr2=0x7fecbc1f3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:58.153 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:58.149+0000 7fecc27ce640 1 --2- 172.21.15.169:0/530079216 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feca8004ab0 0x7fecbc1f3410 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7fecac001cd0 tx=0x7fecac02cc90 comp rx=0 tx=0).stop 2023-10-26T01:41:58.153 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:58.149+0000 7fecc27ce640 1 -- 172.21.15.169:0/530079216 shutdown_connections 2023-10-26T01:41:58.153 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:58.149+0000 7fecc27ce640 1 --2- 172.21.15.169:0/530079216 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fec9c06ae70 0x7fec9c06d330 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:58.153 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:58.149+0000 7fecc27ce640 1 --2- 172.21.15.169:0/530079216 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fecbc06ecc0 0x7fecbc1f4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:58.154 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:58.149+0000 7fecc27ce640 1 --2- 172.21.15.169:0/530079216 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fecbc1e39c0 0x7fecbc14c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:58.154 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:58.149+0000 7fecc27ce640 1 --2- 172.21.15.169:0/530079216 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feca8004ab0 0x7fecbc1f3410 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:58.154 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:58.150+0000 7fecc27ce640 1 -- 172.21.15.169:0/530079216 >> 172.21.15.169:0/530079216 conn(0x7fecbc0aeee0 msgr2=0x7fecbc150b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:41:58.154 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:58.152+0000 7fecc27ce640 1 -- 172.21.15.169:0/530079216 shutdown_connections 2023-10-26T01:41:58.155 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:58.152+0000 7fecc27ce640 1 -- 172.21.15.169:0/530079216 wait complete. 2023-10-26T01:41:58.174 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-26T01:41:58.383 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:58.380+0000 7ffa46213640 1 -- 172.21.15.169:0/1737787034 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa40159370 msgr2=0x7ffa400d0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:58.383 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:58.380+0000 7ffa46213640 1 --2- 172.21.15.169:0/1737787034 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa40159370 0x7ffa400d0630 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c005fd0 tx=0x7ffa3c02bd10 comp rx=0 tx=0).stop 2023-10-26T01:41:58.384 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:58.380+0000 7ffa46213640 1 -- 172.21.15.169:0/1737787034 shutdown_connections 2023-10-26T01:41:58.384 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:58.380+0000 7ffa46213640 1 --2- 172.21.15.169:0/1737787034 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ffa4006ac80 0x7ffa4006b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:58.384 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:58.380+0000 7ffa46213640 1 --2- 172.21.15.169:0/1737787034 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa40159370 0x7ffa400d0630 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:58.384 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:58.380+0000 7ffa46213640 1 -- 172.21.15.169:0/1737787034 >> 172.21.15.169:0/1737787034 conn(0x7ffa400d13a0 msgr2=0x7ffa400d17b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:41:58.384 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:58.380+0000 7ffa46213640 1 -- 172.21.15.169:0/1737787034 shutdown_connections 2023-10-26T01:41:58.385 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:58.380+0000 7ffa46213640 1 -- 172.21.15.169:0/1737787034 wait complete. 2023-10-26T01:41:58.385 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:58.382+0000 7ffa46213640 1 Processor -- start 2023-10-26T01:41:58.385 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:58.383+0000 7ffa46213640 1 -- start start 2023-10-26T01:41:58.387 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:58.384+0000 7ffa46213640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ffa4006ac80 0x7ffa401520d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:41:58.387 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:58.384+0000 7ffa46213640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa40159370 0x7ffa400b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:41:58.387 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:58.384+0000 7ffa46213640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffa4015c6c0 0x7ffa400b2730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:41:58.387 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:58.384+0000 7ffa46213640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa401f1ad0 con 0x7ffa40159370 2023-10-26T01:41:58.387 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:58.384+0000 7ffa46213640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa401e3c20 con 0x7ffa4015c6c0 2023-10-26T01:41:58.388 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:58.384+0000 7ffa46213640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa401f1cc0 con 0x7ffa4006ac80 2023-10-26T01:41:58.388 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:58.384+0000 7ffa44a10640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa40159370 0x7ffa400b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:41:58.388 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:58.384+0000 7ffa44a10640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa40159370 0x7ffa400b21f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:37716/0 (socket says 172.21.15.169:37716) 2023-10-26T01:41:58.388 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:58.384+0000 7ffa44a10640 1 -- 172.21.15.169:0/950656450 learned_addr learned my addr 172.21.15.169:0/950656450 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:41:58.389 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:58.384+0000 7ffa45a12640 1 --2- 172.21.15.169:0/950656450 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffa4015c6c0 0x7ffa400b2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:41:58.389 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:58.385+0000 7ffa44a10640 1 -- 172.21.15.169:0/950656450 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ffa4006ac80 msgr2=0x7ffa401520d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:58.390 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:58.385+0000 7ffa45211640 1 --2- 172.21.15.169:0/950656450 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ffa4006ac80 0x7ffa401520d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:41:58.390 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:58.385+0000 7ffa44a10640 1 --2- 172.21.15.169:0/950656450 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ffa4006ac80 0x7ffa401520d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:58.390 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:58.385+0000 7ffa44a10640 1 -- 172.21.15.169:0/950656450 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffa4015c6c0 msgr2=0x7ffa400b2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:58.390 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:58.385+0000 7ffa44a10640 1 --2- 172.21.15.169:0/950656450 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffa4015c6c0 0x7ffa400b2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:58.391 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:58.385+0000 7ffa44a10640 1 -- 172.21.15.169:0/950656450 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa3c009560 con 0x7ffa40159370 2023-10-26T01:41:58.391 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:58.385+0000 7ffa45a12640 1 --2- 172.21.15.169:0/950656450 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffa4015c6c0 0x7ffa400b2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:41:58.391 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:58.385+0000 7ffa44a10640 1 --2- 172.21.15.169:0/950656450 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa40159370 0x7ffa400b21f0 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7ffa3400b3e0 tx=0x7ffa3400b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:41:58.391 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:58.385+0000 7ffa31ffb640 1 -- 172.21.15.169:0/950656450 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa34011830 con 0x7ffa40159370 2023-10-26T01:41:58.391 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:58.385+0000 7ffa31ffb640 1 -- 172.21.15.169:0/950656450 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ffa34011aa0 con 0x7ffa40159370 2023-10-26T01:41:58.391 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:58.385+0000 7ffa31ffb640 1 -- 172.21.15.169:0/950656450 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa34010380 con 0x7ffa40159370 2023-10-26T01:41:58.392 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:58.385+0000 7ffa46213640 1 -- 172.21.15.169:0/950656450 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa401e3380 con 0x7ffa40159370 2023-10-26T01:41:58.392 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:58.385+0000 7ffa46213640 1 -- 172.21.15.169:0/950656450 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa401e3940 con 0x7ffa40159370 2023-10-26T01:41:58.392 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:58.387+0000 7ffa46213640 1 -- 172.21.15.169:0/950656450 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa40073c10 con 0x7ffa40159370 2023-10-26T01:41:58.393 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:58.388+0000 7ffa31ffb640 1 -- 172.21.15.169:0/950656450 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ffa340104e0 con 0x7ffa40159370 2023-10-26T01:41:58.393 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:58.389+0000 7ffa31ffb640 1 --2- 172.21.15.169:0/950656450 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ffa2406ae40 0x7ffa2406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:41:58.393 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:58.389+0000 7ffa31ffb640 1 -- 172.21.15.169:0/950656450 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 5415+0+0 (secure 0 0 0) 0x7ffa34089950 con 0x7ffa40159370 2023-10-26T01:41:58.393 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:58.389+0000 7ffa45211640 1 --2- 172.21.15.169:0/950656450 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ffa2406ae40 0x7ffa2406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:41:58.394 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:58.390+0000 7ffa45211640 1 --2- 172.21.15.169:0/950656450 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ffa2406ae40 0x7ffa2406d300 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c035000 tx=0x7ffa3c004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:41:58.396 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:58.393+0000 7ffa31ffb640 1 -- 172.21.15.169:0/950656450 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ffa34054b10 con 0x7ffa40159370 2023-10-26T01:41:58.586 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:58.584+0000 7ffa46213640 1 -- 172.21.15.169:0/950656450 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ffa400b0890 con 0x7ffa40159370 2023-10-26T01:41:59.300 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:59.297+0000 7ffa31ffb640 1 -- 172.21.15.169:0/950656450 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v158) v1 ==== 169+0+0 (secure 0 0 0) 0x7ffa340150d0 con 0x7ffa40159370 2023-10-26T01:41:59.300 INFO:teuthology.orchestra.run.smithi169.stderr:pool 'cephfs_data' removed 2023-10-26T01:41:59.308 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:59.305+0000 7ffa46213640 1 -- 172.21.15.169:0/950656450 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ffa2406ae40 msgr2=0x7ffa2406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:59.308 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:59.305+0000 7ffa46213640 1 --2- 172.21.15.169:0/950656450 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ffa2406ae40 0x7ffa2406d300 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c035000 tx=0x7ffa3c004730 comp rx=0 tx=0).stop 2023-10-26T01:41:59.308 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:59.305+0000 7ffa46213640 1 -- 172.21.15.169:0/950656450 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa40159370 msgr2=0x7ffa400b21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:59.309 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:59.305+0000 7ffa46213640 1 --2- 172.21.15.169:0/950656450 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa40159370 0x7ffa400b21f0 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7ffa3400b3e0 tx=0x7ffa3400b8a0 comp rx=0 tx=0).stop 2023-10-26T01:41:59.309 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:59.305+0000 7ffa46213640 1 -- 172.21.15.169:0/950656450 shutdown_connections 2023-10-26T01:41:59.309 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:59.305+0000 7ffa46213640 1 --2- 172.21.15.169:0/950656450 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ffa2406ae40 0x7ffa2406d300 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:59.309 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:59.305+0000 7ffa46213640 1 --2- 172.21.15.169:0/950656450 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa40159370 0x7ffa400b21f0 secure :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7ffa3400b3e0 tx=0x7ffa3400b8a0 comp rx=0 tx=0).stop 2023-10-26T01:41:59.310 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:59.305+0000 7ffa46213640 1 --2- 172.21.15.169:0/950656450 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffa4015c6c0 0x7ffa400b2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:59.310 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:59.305+0000 7ffa46213640 1 --2- 172.21.15.169:0/950656450 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ffa4006ac80 0x7ffa401520d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:59.310 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:59.305+0000 7ffa46213640 1 -- 172.21.15.169:0/950656450 >> 172.21.15.169:0/950656450 conn(0x7ffa400d13a0 msgr2=0x7ffa4015ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:41:59.313 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:59.310+0000 7ffa46213640 1 -- 172.21.15.169:0/950656450 shutdown_connections 2023-10-26T01:41:59.313 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:59.311+0000 7ffa46213640 1 -- 172.21.15.169:0/950656450 wait complete. 2023-10-26T01:41:59.335 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 18 2023-10-26T01:41:59.536 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:59.533+0000 7f7ec5f73640 1 -- 172.21.15.169:0/3154505329 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ec01597c0 msgr2=0x7f7ec00ac200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:59.536 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:59.533+0000 7f7ec5f73640 1 --2- 172.21.15.169:0/3154505329 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ec01597c0 0x7f7ec00ac200 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f7ea8005fd0 tx=0x7f7ea802c2d0 comp rx=0 tx=0).stop 2023-10-26T01:41:59.537 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:59.533+0000 7f7ec5f73640 1 -- 172.21.15.169:0/3154505329 shutdown_connections 2023-10-26T01:41:59.537 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:59.533+0000 7f7ec5f73640 1 --2- 172.21.15.169:0/3154505329 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7ec00b0c80 0x7f7ec00cd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:59.537 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:59.533+0000 7f7ec5f73640 1 --2- 172.21.15.169:0/3154505329 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ec01597c0 0x7f7ec00ac200 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:59.537 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:59.533+0000 7f7ec5f73640 1 -- 172.21.15.169:0/3154505329 >> 172.21.15.169:0/3154505329 conn(0x7f7ec0069930 msgr2=0x7f7ec0069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:41:59.537 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:59.533+0000 7f7ec5f73640 1 -- 172.21.15.169:0/3154505329 shutdown_connections 2023-10-26T01:41:59.538 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:59.533+0000 7f7ec5f73640 1 -- 172.21.15.169:0/3154505329 wait complete. 2023-10-26T01:41:59.538 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:59.535+0000 7f7ec5f73640 1 Processor -- start 2023-10-26T01:41:59.538 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:59.536+0000 7f7ec5f73640 1 -- start start 2023-10-26T01:41:59.540 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:59.537+0000 7f7ec5f73640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ec00b0c80 0x7f7ec006c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:41:59.540 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:59.537+0000 7f7ec5f73640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ec01593e0 0x7f7ec00c8450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:41:59.540 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:59.537+0000 7f7ec5f73640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7ec01597c0 0x7f7ec0153510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:41:59.540 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:59.537+0000 7f7ec5f73640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec00c6b70 con 0x7f7ec00b0c80 2023-10-26T01:41:59.540 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:59.537+0000 7f7ec5f73640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec00c9120 con 0x7f7ec01593e0 2023-10-26T01:41:59.541 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:59.537+0000 7f7ec5f73640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec00c8b60 con 0x7f7ec01597c0 2023-10-26T01:41:59.541 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:59.537+0000 7f7ebf7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ec00b0c80 0x7f7ec006c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:41:59.541 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:59.537+0000 7f7ebf7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ec00b0c80 0x7f7ec006c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:37724/0 (socket says 172.21.15.169:37724) 2023-10-26T01:41:59.541 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:59.537+0000 7f7ebf7fe640 1 -- 172.21.15.169:0/2043035115 learned_addr learned my addr 172.21.15.169:0/2043035115 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:41:59.543 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:59.538+0000 7f7ebeffd640 1 --2- 172.21.15.169:0/2043035115 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ec01593e0 0x7f7ec00c8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:41:59.543 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:59.538+0000 7f7ebffff640 1 --2- 172.21.15.169:0/2043035115 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7ec01597c0 0x7f7ec0153510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:41:59.543 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:59.538+0000 7f7ebf7fe640 1 -- 172.21.15.169:0/2043035115 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7ec01597c0 msgr2=0x7f7ec0153510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:59.543 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:59.538+0000 7f7ebf7fe640 1 --2- 172.21.15.169:0/2043035115 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7ec01597c0 0x7f7ec0153510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:59.543 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:59.538+0000 7f7ebf7fe640 1 -- 172.21.15.169:0/2043035115 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ec01593e0 msgr2=0x7f7ec00c8450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:59.543 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:59.538+0000 7f7ebf7fe640 1 --2- 172.21.15.169:0/2043035115 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ec01593e0 0x7f7ec00c8450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:59.544 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:59.538+0000 7f7ebf7fe640 1 -- 172.21.15.169:0/2043035115 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ea8009900 con 0x7f7ec00b0c80 2023-10-26T01:41:59.544 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:59.538+0000 7f7ebffff640 1 --2- 172.21.15.169:0/2043035115 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7ec01597c0 0x7f7ec0153510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:41:59.544 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:59.538+0000 7f7ebf7fe640 1 --2- 172.21.15.169:0/2043035115 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ec00b0c80 0x7f7ec006c960 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f7eb000c400 tx=0x7f7eb000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:41:59.544 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:59.538+0000 7f7eb7fff640 1 -- 172.21.15.169:0/2043035115 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eb00052d0 con 0x7f7ec00b0c80 2023-10-26T01:41:59.544 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:59.538+0000 7f7eb7fff640 1 -- 172.21.15.169:0/2043035115 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7eb0007020 con 0x7f7ec00b0c80 2023-10-26T01:41:59.545 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:59.538+0000 7f7ec5f73640 1 -- 172.21.15.169:0/2043035115 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ec00c8e40 con 0x7f7ec00b0c80 2023-10-26T01:41:59.545 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:59.538+0000 7f7eb7fff640 1 -- 172.21.15.169:0/2043035115 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eb0007470 con 0x7f7ec00b0c80 2023-10-26T01:41:59.545 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:59.538+0000 7f7ec5f73640 1 -- 172.21.15.169:0/2043035115 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ec0153a50 con 0x7f7ec00b0c80 2023-10-26T01:41:59.545 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:59.540+0000 7f7ec5f73640 1 -- 172.21.15.169:0/2043035115 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ec004db10 con 0x7f7ec00b0c80 2023-10-26T01:41:59.546 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:59.541+0000 7f7eb7fff640 1 -- 172.21.15.169:0/2043035115 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f7eb00067d0 con 0x7f7ec00b0c80 2023-10-26T01:41:59.546 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:59.542+0000 7f7eb7fff640 1 --2- 172.21.15.169:0/2043035115 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7ea006ae70 0x7f7ea006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:41:59.546 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:59.542+0000 7f7eb7fff640 1 -- 172.21.15.169:0/2043035115 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f7eb00888d0 con 0x7f7ec00b0c80 2023-10-26T01:41:59.547 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:59.543+0000 7f7ebeffd640 1 --2- 172.21.15.169:0/2043035115 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7ea006ae70 0x7f7ea006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:41:59.547 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:59.544+0000 7f7ebeffd640 1 --2- 172.21.15.169:0/2043035115 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7ea006ae70 0x7f7ea006d330 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f7ea8036000 tx=0x7f7ea8004ab0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:41:59.549 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:59.547+0000 7f7eb7fff640 1 -- 172.21.15.169:0/2043035115 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7eb0053d10 con 0x7f7ec00b0c80 2023-10-26T01:41:59.740 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:59.737+0000 7f7ec5f73640 1 -- 172.21.15.169:0/2043035115 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "config reset", "num": 18} v 0) v1 -- 0x7f7ec00b2170 con 0x7f7ec00b0c80 2023-10-26T01:41:59.747 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:59.744+0000 7f7eb7fff640 1 -- 172.21.15.169:0/2043035115 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 18}]=0 v34) v1 ==== 71+0+0 (secure 0 0 0) 0x7f7eb0057330 con 0x7f7ec00b0c80 2023-10-26T01:41:59.751 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:59.747+0000 7f7ec5f73640 1 -- 172.21.15.169:0/2043035115 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7ea006ae70 msgr2=0x7f7ea006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:59.751 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:59.747+0000 7f7ec5f73640 1 --2- 172.21.15.169:0/2043035115 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7ea006ae70 0x7f7ea006d330 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f7ea8036000 tx=0x7f7ea8004ab0 comp rx=0 tx=0).stop 2023-10-26T01:41:59.752 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:59.747+0000 7f7ec5f73640 1 -- 172.21.15.169:0/2043035115 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ec00b0c80 msgr2=0x7f7ec006c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:59.752 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:59.747+0000 7f7ec5f73640 1 --2- 172.21.15.169:0/2043035115 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ec00b0c80 0x7f7ec006c960 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f7eb000c400 tx=0x7f7eb000c8c0 comp rx=0 tx=0).stop 2023-10-26T01:41:59.752 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:59.747+0000 7f7ec5f73640 1 -- 172.21.15.169:0/2043035115 shutdown_connections 2023-10-26T01:41:59.752 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:59.748+0000 7f7ec5f73640 1 --2- 172.21.15.169:0/2043035115 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7ea006ae70 0x7f7ea006d330 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:59.752 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:59.748+0000 7f7ec5f73640 1 --2- 172.21.15.169:0/2043035115 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7ec01597c0 0x7f7ec0153510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:59.753 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:59.748+0000 7f7ec5f73640 1 --2- 172.21.15.169:0/2043035115 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ec01593e0 0x7f7ec00c8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:59.753 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:59.748+0000 7f7ec5f73640 1 --2- 172.21.15.169:0/2043035115 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ec00b0c80 0x7f7ec006c960 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:59.753 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:59.748+0000 7f7ec5f73640 1 -- 172.21.15.169:0/2043035115 >> 172.21.15.169:0/2043035115 conn(0x7f7ec0069930 msgr2=0x7f7ec015bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:41:59.753 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:59.748+0000 7f7ec5f73640 1 -- 172.21.15.169:0/2043035115 shutdown_connections 2023-10-26T01:41:59.753 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:59.748+0000 7f7ec5f73640 1 -- 172.21.15.169:0/2043035115 wait complete. 2023-10-26T01:41:59.768 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_fclose' 2023-10-26T01:41:59.988 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:59.984+0000 7fce067e9640 1 -- 172.21.15.169:0/2115108278 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce0006cae0 msgr2=0x7fce000c7ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:59.988 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:59.984+0000 7fce067e9640 1 --2- 172.21.15.169:0/2115108278 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce0006cae0 0x7fce000c7ea0 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc005fd0 tx=0x7fcdfc02bd90 comp rx=0 tx=0).stop 2023-10-26T01:41:59.988 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:59.985+0000 7fce067e9640 1 -- 172.21.15.169:0/2115108278 shutdown_connections 2023-10-26T01:41:59.988 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:59.985+0000 7fce067e9640 1 --2- 172.21.15.169:0/2115108278 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce0006cae0 0x7fce000c7ea0 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:59.989 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:59.985+0000 7fce067e9640 1 -- 172.21.15.169:0/2115108278 >> 172.21.15.169:0/2115108278 conn(0x7fce000aedd0 msgr2=0x7fce000af1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:41:59.989 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:59.985+0000 7fce067e9640 1 -- 172.21.15.169:0/2115108278 shutdown_connections 2023-10-26T01:41:59.989 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:59.985+0000 7fce067e9640 1 -- 172.21.15.169:0/2115108278 wait complete. 2023-10-26T01:41:59.990 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:59.987+0000 7fce067e9640 1 Processor -- start 2023-10-26T01:41:59.990 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:59.987+0000 7fce067e9640 1 -- start start 2023-10-26T01:41:59.991 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:59.988+0000 7fce067e9640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce0006a990 0x7fce00147780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:41:59.991 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:59.988+0000 7fce067e9640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fce0006cae0 0x7fce00144370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:41:59.992 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:59.988+0000 7fce067e9640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce000c96f0 0x7fce00098a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:41:59.992 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:59.988+0000 7fce067e9640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce00147cc0 con 0x7fce0006a990 2023-10-26T01:41:59.992 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:59.988+0000 7fce067e9640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce0014b060 con 0x7fce000c96f0 2023-10-26T01:41:59.992 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:59.988+0000 7fce067e9640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce0015c660 con 0x7fce0006cae0 2023-10-26T01:41:59.992 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:59.989+0000 7fce057e7640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce0006a990 0x7fce00147780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:41:59.993 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:59.989+0000 7fce057e7640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce0006a990 0x7fce00147780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:37732/0 (socket says 172.21.15.169:37732) 2023-10-26T01:41:59.993 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:59.989+0000 7fce057e7640 1 -- 172.21.15.169:0/734310513 learned_addr learned my addr 172.21.15.169:0/734310513 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:41:59.994 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:59.989+0000 7fce05fe8640 1 --2- 172.21.15.169:0/734310513 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce000c96f0 0x7fce00098a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:41:59.994 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:59.989+0000 7fce057e7640 1 -- 172.21.15.169:0/734310513 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fce0006cae0 msgr2=0x7fce00144370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:59.995 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:59.989+0000 7fce057e7640 1 --2- 172.21.15.169:0/734310513 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fce0006cae0 0x7fce00144370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:59.995 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:59.989+0000 7fce057e7640 1 -- 172.21.15.169:0/734310513 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce000c96f0 msgr2=0x7fce00098a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:41:59.995 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:59.989+0000 7fce057e7640 1 --2- 172.21.15.169:0/734310513 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce000c96f0 0x7fce00098a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:41:59.995 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:59.989+0000 7fce057e7640 1 -- 172.21.15.169:0/734310513 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdfc009560 con 0x7fce0006a990 2023-10-26T01:41:59.995 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:59.990+0000 7fce057e7640 1 --2- 172.21.15.169:0/734310513 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce0006a990 0x7fce00147780 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc036040 tx=0x7fcdfc02dd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:41:59.996 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:59.990+0000 7fcdf5ffb640 1 -- 172.21.15.169:0/734310513 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdfc005650 con 0x7fce0006a990 2023-10-26T01:41:59.996 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:59.990+0000 7fcdf5ffb640 1 -- 172.21.15.169:0/734310513 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fcdfc0038f0 con 0x7fce0006a990 2023-10-26T01:41:59.997 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:59.990+0000 7fcdf5ffb640 1 -- 172.21.15.169:0/734310513 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdfc003ca0 con 0x7fce0006a990 2023-10-26T01:41:59.997 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:59.990+0000 7fce067e9640 1 -- 172.21.15.169:0/734310513 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce000ccd40 con 0x7fce0006a990 2023-10-26T01:41:59.997 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:59.990+0000 7fce067e9640 1 -- 172.21.15.169:0/734310513 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce0015cc20 con 0x7fce0006a990 2023-10-26T01:41:59.998 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:59.991+0000 7fce067e9640 1 -- 172.21.15.169:0/734310513 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce00073c10 con 0x7fce0006a990 2023-10-26T01:41:59.999 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:59.992+0000 7fcdf5ffb640 1 -- 172.21.15.169:0/734310513 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fcdfc03e450 con 0x7fce0006a990 2023-10-26T01:41:59.999 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:59.993+0000 7fcdf5ffb640 1 --2- 172.21.15.169:0/734310513 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcde406ae40 0x7fcde406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:41:59.999 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:59.993+0000 7fcdf5ffb640 1 -- 172.21.15.169:0/734310513 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fcdfc0abf50 con 0x7fce0006a990 2023-10-26T01:41:59.999 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:59.994+0000 7fce04fe6640 1 --2- 172.21.15.169:0/734310513 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcde406ae40 0x7fcde406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:41:59.999 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:59.995+0000 7fce04fe6640 1 --2- 172.21.15.169:0/734310513 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcde406ae40 0x7fcde406d300 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fcdf0000f60 tx=0x7fcdf00079b0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:42:00.000 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:41:59.997+0000 7fcdf5ffb640 1 -- 172.21.15.169:0/734310513 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcdfc077540 con 0x7fce0006a990 2023-10-26T01:42:00.259 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:00.256+0000 7fce067e9640 1 -- 172.21.15.169:0/734310513 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_fclose"]} v 0) v1 -- 0x7fce0006c790 con 0x7fce0006a990 2023-10-26T01:42:00.303 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:00.300+0000 7fcdf5ffb640 1 -- 172.21.15.169:0/734310513 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_fclose"]}]=0 v549) v1 ==== 132+0+0 (secure 0 0 0) 0x7fcdfc0430c0 con 0x7fce0006a990 2023-10-26T01:42:00.307 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:00.304+0000 7fce067e9640 1 -- 172.21.15.169:0/734310513 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcde406ae40 msgr2=0x7fcde406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:00.307 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:00.304+0000 7fce067e9640 1 --2- 172.21.15.169:0/734310513 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcde406ae40 0x7fcde406d300 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fcdf0000f60 tx=0x7fcdf00079b0 comp rx=0 tx=0).stop 2023-10-26T01:42:00.307 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:00.304+0000 7fce067e9640 1 -- 172.21.15.169:0/734310513 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce0006a990 msgr2=0x7fce00147780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:00.308 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:00.304+0000 7fce067e9640 1 --2- 172.21.15.169:0/734310513 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce0006a990 0x7fce00147780 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc036040 tx=0x7fcdfc02dd60 comp rx=0 tx=0).stop 2023-10-26T01:42:00.308 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:00.304+0000 7fce067e9640 1 -- 172.21.15.169:0/734310513 shutdown_connections 2023-10-26T01:42:00.308 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:00.304+0000 7fce067e9640 1 --2- 172.21.15.169:0/734310513 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcde406ae40 0x7fcde406d300 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:00.308 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:00.304+0000 7fce067e9640 1 --2- 172.21.15.169:0/734310513 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce000c96f0 0x7fce00098a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:00.309 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:00.304+0000 7fce067e9640 1 --2- 172.21.15.169:0/734310513 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fce0006cae0 0x7fce00144370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:00.309 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:00.304+0000 7fce067e9640 1 --2- 172.21.15.169:0/734310513 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce0006a990 0x7fce00147780 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:00.309 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:00.304+0000 7fce067e9640 1 -- 172.21.15.169:0/734310513 >> 172.21.15.169:0/734310513 conn(0x7fce000aedd0 msgr2=0x7fce0015b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:42:00.309 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:00.304+0000 7fce067e9640 1 -- 172.21.15.169:0/734310513 shutdown_connections 2023-10-26T01:42:00.309 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:00.304+0000 7fce067e9640 1 -- 172.21.15.169:0/734310513 wait complete. 2023-10-26T01:42:00.327 INFO:tasks.cephfs_test_runner:test_full_fclose (tasks.cephfs.test_full.TestQuotaFull) ... ok 2023-10-26T01:42:00.327 INFO:tasks.cephfs_test_runner:Starting test: test_full_fsync (tasks.cephfs.test_full.TestQuotaFull) That when the full flag is encountered during asynchronous 2023-10-26T01:42:00.327 INFO:tasks.cephfs_test_runner:test_full_fsync (tasks.cephfs.test_full.TestQuotaFull) 2023-10-26T01:42:00.328 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:42:00.544 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:00.540+0000 7f6ef736e640 1 -- 172.21.15.169:0/1619325889 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6ef00ac930 msgr2=0x7f6ef00cd6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:00.544 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:00.540+0000 7f6ef736e640 1 --2- 172.21.15.169:0/1619325889 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6ef00ac930 0x7f6ef00cd6f0 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f6ef00aab10 tx=0x7f6edc02bdc0 comp rx=0 tx=0).stop 2023-10-26T01:42:00.544 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:00.540+0000 7f6ef736e640 1 -- 172.21.15.169:0/1619325889 shutdown_connections 2023-10-26T01:42:00.545 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:00.540+0000 7f6ef736e640 1 --2- 172.21.15.169:0/1619325889 >> v2:172.21.15.170:3300/0 conn(0x7f6ef00cdc30 0x7f6ef00cbac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:00.545 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:00.540+0000 7f6ef736e640 1 --2- 172.21.15.169:0/1619325889 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6ef00ac930 0x7f6ef00cd6f0 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:00.545 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:00.540+0000 7f6ef736e640 1 -- 172.21.15.169:0/1619325889 >> 172.21.15.169:0/1619325889 conn(0x7f6ef0069930 msgr2=0x7f6ef0069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:42:00.545 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:00.541+0000 7f6ef736e640 1 -- 172.21.15.169:0/1619325889 shutdown_connections 2023-10-26T01:42:00.545 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:00.541+0000 7f6ef736e640 1 -- 172.21.15.169:0/1619325889 wait complete. 2023-10-26T01:42:00.547 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:00.542+0000 7f6ef736e640 1 Processor -- start 2023-10-26T01:42:00.548 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:00.542+0000 7f6ef736e640 1 -- start start 2023-10-26T01:42:00.548 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:00.543+0000 7f6ef736e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6ef00ac930 0x7f6ef0096930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:00.548 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:00.543+0000 7f6ef736e640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6ef00cdc30 0x7f6ef01521e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:00.548 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:00.543+0000 7f6ef736e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6ef01592d0 0x7f6ef0152720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:00.548 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:00.543+0000 7f6ef736e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef0071ec0 con 0x7f6ef00ac930 2023-10-26T01:42:00.549 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:00.543+0000 7f6ef736e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef0071240 con 0x7f6ef01592d0 2023-10-26T01:42:00.549 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:00.543+0000 7f6ef736e640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef0071520 con 0x7f6ef00cdc30 2023-10-26T01:42:00.549 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:00.543+0000 7f6ef510a640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6ef00ac930 0x7f6ef0096930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:42:00.549 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:00.544+0000 7f6ef510a640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6ef00ac930 0x7f6ef0096930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:37754/0 (socket says 172.21.15.169:37754) 2023-10-26T01:42:00.550 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:00.544+0000 7f6ef510a640 1 -- 172.21.15.169:0/3833140355 learned_addr learned my addr 172.21.15.169:0/3833140355 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:42:00.550 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:00.544+0000 7f6ef510a640 1 -- 172.21.15.169:0/3833140355 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6ef00cdc30 msgr2=0x7f6ef01521e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:00.550 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:00.544+0000 7f6ef510a640 1 --2- 172.21.15.169:0/3833140355 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6ef00cdc30 0x7f6ef01521e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:00.550 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:00.544+0000 7f6ef510a640 1 -- 172.21.15.169:0/3833140355 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6ef01592d0 msgr2=0x7f6ef0152720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:00.550 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:00.544+0000 7f6ef510a640 1 --2- 172.21.15.169:0/3833140355 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6ef01592d0 0x7f6ef0152720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:00.551 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:00.544+0000 7f6ef510a640 1 -- 172.21.15.169:0/3833140355 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6edc009560 con 0x7f6ef00ac930 2023-10-26T01:42:00.551 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:00.544+0000 7f6ef510a640 1 --2- 172.21.15.169:0/3833140355 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6ef00ac930 0x7f6ef0096930 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f6ee400c730 tx=0x7f6ee400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:42:00.551 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:00.544+0000 7f6eedffb640 1 -- 172.21.15.169:0/3833140355 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee4006d50 con 0x7f6ef00ac930 2023-10-26T01:42:00.551 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:00.544+0000 7f6eedffb640 1 -- 172.21.15.169:0/3833140355 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6ee400ce40 con 0x7f6ef00ac930 2023-10-26T01:42:00.551 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:00.544+0000 7f6eedffb640 1 -- 172.21.15.169:0/3833140355 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee4003bd0 con 0x7f6ef00ac930 2023-10-26T01:42:00.552 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:00.544+0000 7f6ef736e640 1 -- 172.21.15.169:0/3833140355 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ef0071800 con 0x7f6ef00ac930 2023-10-26T01:42:00.552 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:00.545+0000 7f6ef736e640 1 -- 172.21.15.169:0/3833140355 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ef0096e70 con 0x7f6ef00ac930 2023-10-26T01:42:00.553 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:00.546+0000 7f6ef736e640 1 -- 172.21.15.169:0/3833140355 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ef004db10 con 0x7f6ef00ac930 2023-10-26T01:42:00.553 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:00.547+0000 7f6eedffb640 1 -- 172.21.15.169:0/3833140355 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f6ee4003d30 con 0x7f6ef00ac930 2023-10-26T01:42:00.553 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:00.547+0000 7f6eedffb640 1 --2- 172.21.15.169:0/3833140355 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f6ed406ae40 0x7f6ed406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:00.553 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:00.548+0000 7f6eedffb640 1 -- 172.21.15.169:0/3833140355 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f6ee4088fc0 con 0x7f6ef00ac930 2023-10-26T01:42:00.554 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:00.548+0000 7f6ef4909640 1 --2- 172.21.15.169:0/3833140355 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f6ed406ae40 0x7f6ed406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:42:00.554 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:00.549+0000 7f6ef4909640 1 --2- 172.21.15.169:0/3833140355 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f6ed406ae40 0x7f6ed406d300 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f6edc035000 tx=0x7f6edc0046c0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:42:00.555 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:00.552+0000 7f6eedffb640 1 -- 172.21.15.169:0/3833140355 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6ee4057df0 con 0x7f6ef00ac930 2023-10-26T01:42:00.746 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:00.743+0000 7f6ef736e640 1 -- 172.21.15.169:0/3833140355 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6ef00ce080 con 0x7f6ef00ac930 2023-10-26T01:42:00.748 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:00.745+0000 7f6eedffb640 1 -- 172.21.15.169:0/3833140355 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v158) v1 ==== 74+0+13043 (secure 0 0 0) 0x7f6ef00ce080 con 0x7f6ef00ac930 2023-10-26T01:42:00.748 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:42:00.749 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":158,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:41:59.289086+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.169:6835/66540264":"2023-10-27T01:41:56.085825+0000","172.21.15.169:6834/66540264":"2023-10-27T01:41:56.085825+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:42:00.751 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:00.748+0000 7f6ef736e640 1 -- 172.21.15.169:0/3833140355 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f6ed406ae40 msgr2=0x7f6ed406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:00.751 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:00.748+0000 7f6ef736e640 1 --2- 172.21.15.169:0/3833140355 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f6ed406ae40 0x7f6ed406d300 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f6edc035000 tx=0x7f6edc0046c0 comp rx=0 tx=0).stop 2023-10-26T01:42:00.751 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:00.748+0000 7f6ef736e640 1 -- 172.21.15.169:0/3833140355 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6ef00ac930 msgr2=0x7f6ef0096930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:00.751 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:00.748+0000 7f6ef736e640 1 --2- 172.21.15.169:0/3833140355 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6ef00ac930 0x7f6ef0096930 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f6ee400c730 tx=0x7f6ee400cbf0 comp rx=0 tx=0).stop 2023-10-26T01:42:00.752 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:00.748+0000 7f6ef736e640 1 -- 172.21.15.169:0/3833140355 shutdown_connections 2023-10-26T01:42:00.752 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:00.748+0000 7f6ef736e640 1 --2- 172.21.15.169:0/3833140355 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f6ed406ae40 0x7f6ed406d300 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:00.752 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:00.748+0000 7f6ef736e640 1 --2- 172.21.15.169:0/3833140355 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6ef01592d0 0x7f6ef0152720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:00.752 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:00.748+0000 7f6ef736e640 1 --2- 172.21.15.169:0/3833140355 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6ef00cdc30 0x7f6ef01521e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:00.752 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:00.748+0000 7f6ef736e640 1 --2- 172.21.15.169:0/3833140355 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6ef00ac930 0x7f6ef0096930 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:00.752 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:00.748+0000 7f6ef736e640 1 -- 172.21.15.169:0/3833140355 >> 172.21.15.169:0/3833140355 conn(0x7f6ef0069930 msgr2=0x7f6ef0153900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:42:00.753 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:00.748+0000 7f6ef736e640 1 -- 172.21.15.169:0/3833140355 shutdown_connections 2023-10-26T01:42:00.753 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:00.748+0000 7f6ef736e640 1 -- 172.21.15.169:0/3833140355 wait complete. 2023-10-26T01:42:00.768 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T01:30:32.200808+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '109', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T01:42:00.769 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T01:42:00.987 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:00.983+0000 7f30de8e7640 1 -- 172.21.15.169:0/1423274335 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30d80aaf80 msgr2=0x7f30d80ab3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:00.987 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:00.983+0000 7f30de8e7640 1 --2- 172.21.15.169:0/1423274335 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30d80aaf80 0x7f30d80ab3c0 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f30c0009830 tx=0x7f30c002bcd0 comp rx=0 tx=0).stop 2023-10-26T01:42:00.987 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:00.984+0000 7f30de8e7640 1 -- 172.21.15.169:0/1423274335 shutdown_connections 2023-10-26T01:42:00.987 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:00.984+0000 7f30de8e7640 1 --2- 172.21.15.169:0/1423274335 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f30d80cd690 0x7f30d80cdb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:00.988 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:00.984+0000 7f30de8e7640 1 --2- 172.21.15.169:0/1423274335 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30d80aaf80 0x7f30d80ab3c0 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:00.988 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:00.984+0000 7f30de8e7640 1 -- 172.21.15.169:0/1423274335 >> 172.21.15.169:0/1423274335 conn(0x7f30d8069930 msgr2=0x7f30d8069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:42:00.988 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:00.984+0000 7f30de8e7640 1 -- 172.21.15.169:0/1423274335 shutdown_connections 2023-10-26T01:42:00.988 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:00.984+0000 7f30de8e7640 1 -- 172.21.15.169:0/1423274335 wait complete. 2023-10-26T01:42:00.989 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:00.985+0000 7f30de8e7640 1 Processor -- start 2023-10-26T01:42:00.990 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:00.986+0000 7f30de8e7640 1 -- start start 2023-10-26T01:42:00.990 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:00.986+0000 7f30de8e7640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f30d80aaf80 0x7f30d814b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:00.990 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:00.987+0000 7f30de8e7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f30d80cd690 0x7f30d8071100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:00.990 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:00.987+0000 7f30de8e7640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30d8159480 0x7f30d814a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:00.990 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:00.987+0000 7f30de8e7640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30d814bde0 con 0x7f30d8159480 2023-10-26T01:42:00.991 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:00.987+0000 7f30de8e7640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30d81efa40 con 0x7f30d80cd690 2023-10-26T01:42:00.991 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:00.987+0000 7f30de8e7640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30d81efd20 con 0x7f30d80aaf80 2023-10-26T01:42:00.991 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:00.987+0000 7f30dce84640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30d8159480 0x7f30d814a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:42:00.991 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:00.987+0000 7f30dce84640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30d8159480 0x7f30d814a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:37766/0 (socket says 172.21.15.169:37766) 2023-10-26T01:42:00.991 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:00.987+0000 7f30dce84640 1 -- 172.21.15.169:0/1913670247 learned_addr learned my addr 172.21.15.169:0/1913670247 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:42:00.992 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:00.987+0000 7f30cffff640 1 --2- 172.21.15.169:0/1913670247 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f30d80cd690 0x7f30d8071100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:42:00.992 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:00.987+0000 7f30d7fff640 1 --2- 172.21.15.169:0/1913670247 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f30d80aaf80 0x7f30d814b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:42:00.993 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:00.987+0000 7f30dce84640 1 -- 172.21.15.169:0/1913670247 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f30d80aaf80 msgr2=0x7f30d814b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:00.993 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:00.987+0000 7f30dce84640 1 --2- 172.21.15.169:0/1913670247 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f30d80aaf80 0x7f30d814b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:00.994 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:00.987+0000 7f30dce84640 1 -- 172.21.15.169:0/1913670247 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f30d80cd690 msgr2=0x7f30d8071100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:00.994 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:00.987+0000 7f30dce84640 1 --2- 172.21.15.169:0/1913670247 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f30d80cd690 0x7f30d8071100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:00.994 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:00.987+0000 7f30dce84640 1 -- 172.21.15.169:0/1913670247 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30c0009490 con 0x7f30d8159480 2023-10-26T01:42:00.994 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:00.988+0000 7f30dce84640 1 --2- 172.21.15.169:0/1913670247 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30d8159480 0x7f30d814a2b0 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f30d000eaa0 tx=0x7f30d000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:42:00.994 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:00.988+0000 7f30d57fa640 1 -- 172.21.15.169:0/1913670247 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d000cdb0 con 0x7f30d8159480 2023-10-26T01:42:00.995 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:00.988+0000 7f30d57fa640 1 -- 172.21.15.169:0/1913670247 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f30d0004b20 con 0x7f30d8159480 2023-10-26T01:42:00.995 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:00.988+0000 7f30d57fa640 1 -- 172.21.15.169:0/1913670247 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d0005020 con 0x7f30d8159480 2023-10-26T01:42:00.995 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:00.988+0000 7f30de8e7640 1 -- 172.21.15.169:0/1913670247 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30d81f0000 con 0x7f30d8159480 2023-10-26T01:42:00.995 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:00.988+0000 7f30de8e7640 1 -- 172.21.15.169:0/1913670247 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30d81f05c0 con 0x7f30d8159480 2023-10-26T01:42:00.995 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:00.989+0000 7f30de8e7640 1 -- 172.21.15.169:0/1913670247 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30d8066af0 con 0x7f30d8159480 2023-10-26T01:42:00.996 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:00.990+0000 7f30d57fa640 1 -- 172.21.15.169:0/1913670247 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f30d0005180 con 0x7f30d8159480 2023-10-26T01:42:00.996 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:00.991+0000 7f30d57fa640 1 --2- 172.21.15.169:0/1913670247 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f30b806ae40 0x7f30b806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:00.996 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:00.991+0000 7f30d57fa640 1 -- 172.21.15.169:0/1913670247 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f30d0089db0 con 0x7f30d8159480 2023-10-26T01:42:00.997 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:00.992+0000 7f30d7fff640 1 --2- 172.21.15.169:0/1913670247 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f30b806ae40 0x7f30b806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:42:00.997 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:00.993+0000 7f30d7fff640 1 --2- 172.21.15.169:0/1913670247 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f30b806ae40 0x7f30b806d300 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f30c80046b0 tx=0x7f30c8009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:42:00.998 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:00.996+0000 7f30d57fa640 1 -- 172.21.15.169:0/1913670247 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f30d0055160 con 0x7f30d8159480 2023-10-26T01:42:01.186 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:01.183+0000 7f30de8e7640 1 -- 172.21.15.169:0/1913670247 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f30d804da70 con 0x7f30d8159480 2023-10-26T01:42:01.187 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:01.184+0000 7f30d57fa640 1 -- 172.21.15.169:0/1913670247 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v158) v1 ==== 93+0+10 (secure 0 0 0) 0x7f30d001f020 con 0x7f30d8159480 2023-10-26T01:42:01.187 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 1 2023-10-26T01:42:01.189 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:01.186+0000 7f30de8e7640 1 -- 172.21.15.169:0/1913670247 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f30b806ae40 msgr2=0x7f30b806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:01.189 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:01.186+0000 7f30de8e7640 1 --2- 172.21.15.169:0/1913670247 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f30b806ae40 0x7f30b806d300 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f30c80046b0 tx=0x7f30c8009040 comp rx=0 tx=0).stop 2023-10-26T01:42:01.189 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:01.186+0000 7f30de8e7640 1 -- 172.21.15.169:0/1913670247 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30d8159480 msgr2=0x7f30d814a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:01.190 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:01.186+0000 7f30de8e7640 1 --2- 172.21.15.169:0/1913670247 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30d8159480 0x7f30d814a2b0 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f30d000eaa0 tx=0x7f30d000ef60 comp rx=0 tx=0).stop 2023-10-26T01:42:01.190 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:01.187+0000 7f30de8e7640 1 -- 172.21.15.169:0/1913670247 shutdown_connections 2023-10-26T01:42:01.191 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:01.187+0000 7f30de8e7640 1 --2- 172.21.15.169:0/1913670247 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f30b806ae40 0x7f30b806d300 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:01.191 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:01.187+0000 7f30de8e7640 1 --2- 172.21.15.169:0/1913670247 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30d8159480 0x7f30d814a2b0 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:01.191 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:01.187+0000 7f30de8e7640 1 --2- 172.21.15.169:0/1913670247 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f30d80cd690 0x7f30d8071100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:01.191 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:01.187+0000 7f30de8e7640 1 --2- 172.21.15.169:0/1913670247 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f30d80aaf80 0x7f30d814b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:01.191 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:01.187+0000 7f30de8e7640 1 -- 172.21.15.169:0/1913670247 >> 172.21.15.169:0/1913670247 conn(0x7f30d8069930 msgr2=0x7f30d80af8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:42:01.192 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:01.187+0000 7f30de8e7640 1 -- 172.21.15.169:0/1913670247 shutdown_connections 2023-10-26T01:42:01.192 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:01.187+0000 7f30de8e7640 1 -- 172.21.15.169:0/1913670247 wait complete. 2023-10-26T01:42:01.208 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_fsync' 2023-10-26T01:42:01.431 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:01.428+0000 7f42a792c640 1 -- 172.21.15.169:0/936035420 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f42a015b610 msgr2=0x7f42a006ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:01.431 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:01.428+0000 7f42a792c640 1 --2- 172.21.15.169:0/936035420 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f42a015b610 0x7f42a006ed30 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f429c005fd0 tx=0x7f429c02be20 comp rx=0 tx=0).stop 2023-10-26T01:42:01.432 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:01.428+0000 7f42a792c640 1 -- 172.21.15.169:0/936035420 shutdown_connections 2023-10-26T01:42:01.432 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:01.428+0000 7f42a792c640 1 --2- 172.21.15.169:0/936035420 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f42a015b610 0x7f42a006ed30 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:01.432 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:01.428+0000 7f42a792c640 1 -- 172.21.15.169:0/936035420 >> 172.21.15.169:0/936035420 conn(0x7f42a0069930 msgr2=0x7f42a0069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:42:01.432 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:01.428+0000 7f42a792c640 1 -- 172.21.15.169:0/936035420 shutdown_connections 2023-10-26T01:42:01.432 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:01.428+0000 7f42a792c640 1 -- 172.21.15.169:0/936035420 wait complete. 2023-10-26T01:42:01.433 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:01.430+0000 7f42a792c640 1 Processor -- start 2023-10-26T01:42:01.433 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:01.430+0000 7f42a792c640 1 -- start start 2023-10-26T01:42:01.435 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:01.431+0000 7f42a792c640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f42a00ac4e0 0x7f42a0148370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:01.435 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:01.431+0000 7f42a792c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f42a00cdd80 0x7f42a00aa1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:01.436 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:01.431+0000 7f42a792c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f42a015b610 0x7f42a00aa700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:01.436 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:01.431+0000 7f42a792c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42a014bc60 con 0x7f42a00cdd80 2023-10-26T01:42:01.436 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:01.431+0000 7f42a792c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42a01f5f80 con 0x7f42a015b610 2023-10-26T01:42:01.436 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:01.431+0000 7f42a792c640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42a01f5af0 con 0x7f42a00ac4e0 2023-10-26T01:42:01.436 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:01.431+0000 7f42a4ec7640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f42a00cdd80 0x7f42a00aa1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:42:01.437 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:01.431+0000 7f42a4ec7640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f42a00cdd80 0x7f42a00aa1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:37770/0 (socket says 172.21.15.169:37770) 2023-10-26T01:42:01.437 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:01.431+0000 7f42a4ec7640 1 -- 172.21.15.169:0/840516713 learned_addr learned my addr 172.21.15.169:0/840516713 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:42:01.437 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:01.432+0000 7f42a56c8640 1 --2- 172.21.15.169:0/840516713 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f42a00ac4e0 0x7f42a0148370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:42:01.437 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:01.432+0000 7f42a5ec9640 1 --2- 172.21.15.169:0/840516713 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f42a015b610 0x7f42a00aa700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:42:01.437 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:01.432+0000 7f42a4ec7640 1 -- 172.21.15.169:0/840516713 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f42a00ac4e0 msgr2=0x7f42a0148370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:01.437 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:01.432+0000 7f42a4ec7640 1 --2- 172.21.15.169:0/840516713 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f42a00ac4e0 0x7f42a0148370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:01.438 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:01.432+0000 7f42a4ec7640 1 -- 172.21.15.169:0/840516713 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f42a015b610 msgr2=0x7f42a00aa700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:01.438 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:01.432+0000 7f42a4ec7640 1 --2- 172.21.15.169:0/840516713 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f42a015b610 0x7f42a00aa700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:01.438 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:01.432+0000 7f42a4ec7640 1 -- 172.21.15.169:0/840516713 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f429c009560 con 0x7f42a00cdd80 2023-10-26T01:42:01.438 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:01.432+0000 7f42a5ec9640 1 --2- 172.21.15.169:0/840516713 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f42a015b610 0x7f42a00aa700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:42:01.439 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:01.432+0000 7f42a4ec7640 1 --2- 172.21.15.169:0/840516713 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f42a00cdd80 0x7f42a00aa1c0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f429400c4e0 tx=0x7f429400c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:42:01.439 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:01.432+0000 7f4291ffb640 1 -- 172.21.15.169:0/840516713 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4294005480 con 0x7f42a00cdd80 2023-10-26T01:42:01.439 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:01.432+0000 7f4291ffb640 1 -- 172.21.15.169:0/840516713 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4294003720 con 0x7f42a00cdd80 2023-10-26T01:42:01.439 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:01.432+0000 7f42a792c640 1 -- 172.21.15.169:0/840516713 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42a01f51c0 con 0x7f42a00cdd80 2023-10-26T01:42:01.439 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:01.432+0000 7f4291ffb640 1 -- 172.21.15.169:0/840516713 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4294003b30 con 0x7f42a00cdd80 2023-10-26T01:42:01.440 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:01.433+0000 7f42a792c640 1 -- 172.21.15.169:0/840516713 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42a01f5780 con 0x7f42a00cdd80 2023-10-26T01:42:01.441 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:01.434+0000 7f42a792c640 1 -- 172.21.15.169:0/840516713 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42a004db10 con 0x7f42a00cdd80 2023-10-26T01:42:01.441 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:01.435+0000 7f4291ffb640 1 -- 172.21.15.169:0/840516713 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f4294003c90 con 0x7f42a00cdd80 2023-10-26T01:42:01.441 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:01.435+0000 7f4291ffb640 1 --2- 172.21.15.169:0/840516713 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f428406ae40 0x7f428406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:01.441 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:01.436+0000 7f4291ffb640 1 -- 172.21.15.169:0/840516713 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f4294088a00 con 0x7f42a00cdd80 2023-10-26T01:42:01.441 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:01.436+0000 7f42a56c8640 1 --2- 172.21.15.169:0/840516713 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f428406ae40 0x7f428406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:42:01.442 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:01.437+0000 7f42a56c8640 1 --2- 172.21.15.169:0/840516713 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f428406ae40 0x7f428406d300 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f429c035000 tx=0x7f429c004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:42:01.442 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:01.440+0000 7f4291ffb640 1 -- 172.21.15.169:0/840516713 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4294053e60 con 0x7f42a00cdd80 2023-10-26T01:42:01.699 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:01.696+0000 7f42a792c640 1 -- 172.21.15.169:0/840516713 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_fsync"]} v 0) v1 -- 0x7f42a01f5c30 con 0x7f42a00cdd80 2023-10-26T01:42:02.317 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:02.312+0000 7f4291ffb640 1 -- 172.21.15.169:0/840516713 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_fsync"]}]=0 v551) v1 ==== 134+0+0 (secure 0 0 0) 0x7f4294017070 con 0x7f42a00cdd80 2023-10-26T01:42:02.319 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:02.316+0000 7f42a792c640 1 -- 172.21.15.169:0/840516713 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f428406ae40 msgr2=0x7f428406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:02.319 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:02.316+0000 7f42a792c640 1 --2- 172.21.15.169:0/840516713 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f428406ae40 0x7f428406d300 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f429c035000 tx=0x7f429c004730 comp rx=0 tx=0).stop 2023-10-26T01:42:02.320 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:02.316+0000 7f42a792c640 1 -- 172.21.15.169:0/840516713 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f42a00cdd80 msgr2=0x7f42a00aa1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:02.320 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:02.316+0000 7f42a792c640 1 --2- 172.21.15.169:0/840516713 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f42a00cdd80 0x7f42a00aa1c0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f429400c4e0 tx=0x7f429400c9a0 comp rx=0 tx=0).stop 2023-10-26T01:42:02.320 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:02.316+0000 7f42a792c640 1 -- 172.21.15.169:0/840516713 shutdown_connections 2023-10-26T01:42:02.321 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:02.316+0000 7f42a792c640 1 --2- 172.21.15.169:0/840516713 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f428406ae40 0x7f428406d300 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:02.321 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:02.316+0000 7f42a792c640 1 --2- 172.21.15.169:0/840516713 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f42a00cdd80 0x7f42a00aa1c0 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:02.322 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:02.316+0000 7f42a792c640 1 --2- 172.21.15.169:0/840516713 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f42a015b610 0x7f42a00aa700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:02.322 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:02.316+0000 7f42a792c640 1 --2- 172.21.15.169:0/840516713 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f42a00ac4e0 0x7f42a0148370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:02.323 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:02.316+0000 7f42a792c640 1 -- 172.21.15.169:0/840516713 >> 172.21.15.169:0/840516713 conn(0x7f42a0069930 msgr2=0x7f42a0159860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:42:02.323 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:02.316+0000 7f42a792c640 1 -- 172.21.15.169:0/840516713 shutdown_connections 2023-10-26T01:42:02.323 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:02.316+0000 7f42a792c640 1 -- 172.21.15.169:0/840516713 wait complete. 2023-10-26T01:42:02.340 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2023-10-26T01:42:02.559 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:02.556+0000 7fa3d1184640 1 -- 172.21.15.169:0/3028525305 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa3cc15bbb0 msgr2=0x7fa3cc162460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:02.559 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:02.556+0000 7fa3d1184640 1 --2- 172.21.15.169:0/3028525305 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa3cc15bbb0 0x7fa3cc162460 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7fa3c4005fd0 tx=0x7fa3c402d1e0 comp rx=0 tx=0).stop 2023-10-26T01:42:02.560 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:02.556+0000 7fa3d1184640 1 -- 172.21.15.169:0/3028525305 shutdown_connections 2023-10-26T01:42:02.560 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:02.556+0000 7fa3d1184640 1 --2- 172.21.15.169:0/3028525305 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa3cc15bbb0 0x7fa3cc162460 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:02.560 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:02.556+0000 7fa3d1184640 1 -- 172.21.15.169:0/3028525305 >> 172.21.15.169:0/3028525305 conn(0x7fa3cc069930 msgr2=0x7fa3cc069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:42:02.560 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:02.556+0000 7fa3d1184640 1 -- 172.21.15.169:0/3028525305 shutdown_connections 2023-10-26T01:42:02.560 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:02.556+0000 7fa3d1184640 1 -- 172.21.15.169:0/3028525305 wait complete. 2023-10-26T01:42:02.561 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:02.558+0000 7fa3d1184640 1 Processor -- start 2023-10-26T01:42:02.561 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:02.558+0000 7fa3d1184640 1 -- start start 2023-10-26T01:42:02.563 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:02.559+0000 7fa3d1184640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa3cc0cdef0 0x7fa3cc06ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:02.563 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:02.559+0000 7fa3d1184640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa3cc155360 0x7fa3cc0af810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:02.564 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:02.559+0000 7fa3d1184640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa3cc15bbb0 0x7fa3cc1542e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:02.564 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:02.559+0000 7fa3d1184640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3cc0704b0 con 0x7fa3cc15bbb0 2023-10-26T01:42:02.564 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:02.559+0000 7fa3d1184640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3cc06eed0 con 0x7fa3cc155360 2023-10-26T01:42:02.564 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:02.559+0000 7fa3d1184640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3cc06f1b0 con 0x7fa3cc0cdef0 2023-10-26T01:42:02.565 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:02.559+0000 7fa3cb59e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa3cc15bbb0 0x7fa3cc1542e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:42:02.565 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:02.559+0000 7fa3cb59e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa3cc15bbb0 0x7fa3cc1542e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:37784/0 (socket says 172.21.15.169:37784) 2023-10-26T01:42:02.565 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:02.559+0000 7fa3cb59e640 1 -- 172.21.15.169:0/2878212696 learned_addr learned my addr 172.21.15.169:0/2878212696 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:42:02.565 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:02.560+0000 7fa3cb59e640 1 -- 172.21.15.169:0/2878212696 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa3cc0cdef0 msgr2=0x7fa3cc06ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:02.566 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:02.560+0000 7fa3cb59e640 1 --2- 172.21.15.169:0/2878212696 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa3cc0cdef0 0x7fa3cc06ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:02.566 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:02.560+0000 7fa3cb59e640 1 -- 172.21.15.169:0/2878212696 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa3cc155360 msgr2=0x7fa3cc0af810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:02.566 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:02.560+0000 7fa3cb59e640 1 --2- 172.21.15.169:0/2878212696 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa3cc155360 0x7fa3cc0af810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:02.566 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:02.560+0000 7fa3cb59e640 1 -- 172.21.15.169:0/2878212696 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3c400aa10 con 0x7fa3cc15bbb0 2023-10-26T01:42:02.566 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:02.560+0000 7fa3cb59e640 1 --2- 172.21.15.169:0/2878212696 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa3cc15bbb0 0x7fa3cc1542e0 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7fa3c4038040 tx=0x7fa3c4036db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:42:02.567 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:02.560+0000 7fa3c37fe640 1 -- 172.21.15.169:0/2878212696 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c4035440 con 0x7fa3cc15bbb0 2023-10-26T01:42:02.567 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:02.560+0000 7fa3c37fe640 1 -- 172.21.15.169:0/2878212696 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa3c4004540 con 0x7fa3cc15bbb0 2023-10-26T01:42:02.567 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:02.560+0000 7fa3c37fe640 1 -- 172.21.15.169:0/2878212696 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c4040450 con 0x7fa3cc15bbb0 2023-10-26T01:42:02.567 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:02.560+0000 7fa3d1184640 1 -- 172.21.15.169:0/2878212696 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3cc06f490 con 0x7fa3cc15bbb0 2023-10-26T01:42:02.568 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:02.561+0000 7fa3d1184640 1 -- 172.21.15.169:0/2878212696 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3cc06fa50 con 0x7fa3cc15bbb0 2023-10-26T01:42:02.569 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:02.562+0000 7fa3d1184640 1 -- 172.21.15.169:0/2878212696 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3cc06fc40 con 0x7fa3cc15bbb0 2023-10-26T01:42:02.569 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:02.563+0000 7fa3c37fe640 1 -- 172.21.15.169:0/2878212696 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fa3c403c070 con 0x7fa3cc15bbb0 2023-10-26T01:42:02.569 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:02.564+0000 7fa3c37fe640 1 --2- 172.21.15.169:0/2878212696 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa3ac06ae40 0x7fa3ac06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:02.569 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:02.564+0000 7fa3c37fe640 1 -- 172.21.15.169:0/2878212696 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fa3c40ac820 con 0x7fa3cc15bbb0 2023-10-26T01:42:02.570 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:02.564+0000 7fa3cad9d640 1 --2- 172.21.15.169:0/2878212696 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa3ac06ae40 0x7fa3ac06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:42:02.570 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:02.566+0000 7fa3cad9d640 1 --2- 172.21.15.169:0/2878212696 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa3ac06ae40 0x7fa3ac06d300 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc000f60 tx=0x7fa3bc0079b0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:42:02.571 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:02.568+0000 7fa3c37fe640 1 -- 172.21.15.169:0/2878212696 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa3c4077bd0 con 0x7fa3cc15bbb0 2023-10-26T01:42:02.745 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:02.742+0000 7fa3d1184640 1 -- 172.21.15.169:0/2878212696 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "config dump"} v 0) v1 -- 0x7fa3cc04db10 con 0x7fa3cc15bbb0 2023-10-26T01:42:02.745 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:02.743+0000 7fa3c37fe640 1 -- 172.21.15.169:0/2878212696 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v34) v1 ==== 59+0+1116 (secure 0 0 0) 0x7fa3c4044080 con 0x7fa3cc15bbb0 2023-10-26T01:42:02.746 INFO:teuthology.orchestra.run.smithi169.stdout:WHO MASK LEVEL OPTION VALUE RO 2023-10-26T01:42:02.746 INFO:teuthology.orchestra.run.smithi169.stdout:mon advanced mon_op_complaint_time 120 2023-10-26T01:42:02.746 INFO:teuthology.orchestra.run.smithi169.stdout:osd advanced osd_op_complaint_time 180.000000 2023-10-26T01:42:02.746 INFO:teuthology.orchestra.run.smithi169.stdout:mds advanced debug_mds 20/20 2023-10-26T01:42:02.747 INFO:teuthology.orchestra.run.smithi169.stdout:mds advanced debug_mds_balancer 20/20 2023-10-26T01:42:02.747 INFO:teuthology.orchestra.run.smithi169.stdout:mds advanced debug_ms 1/1 2023-10-26T01:42:02.747 INFO:teuthology.orchestra.run.smithi169.stdout:mds dev mds_debug_frag true 2023-10-26T01:42:02.747 INFO:teuthology.orchestra.run.smithi169.stdout:mds dev mds_debug_scatterstat true 2023-10-26T01:42:02.747 INFO:teuthology.orchestra.run.smithi169.stdout:mds advanced mds_op_complaint_time 180.000000 2023-10-26T01:42:02.748 INFO:teuthology.orchestra.run.smithi169.stdout:mds dev mds_verify_scatter true 2023-10-26T01:42:02.748 INFO:teuthology.orchestra.run.smithi169.stdout:mds advanced osd_op_complaint_time 180.000000 2023-10-26T01:42:02.748 INFO:teuthology.orchestra.run.smithi169.stdout:mds advanced rados_mon_op_timeout 900 2023-10-26T01:42:02.748 INFO:teuthology.orchestra.run.smithi169.stdout:mds advanced rados_osd_op_timeout 900 2023-10-26T01:42:02.749 INFO:teuthology.orchestra.run.smithi169.stdout:client advanced client_mount_timeout 600 2023-10-26T01:42:02.749 INFO:teuthology.orchestra.run.smithi169.stdout:client advanced debug_client 20/20 2023-10-26T01:42:02.749 INFO:teuthology.orchestra.run.smithi169.stdout:client advanced debug_ms 1/1 2023-10-26T01:42:02.749 INFO:teuthology.orchestra.run.smithi169.stdout:client advanced rados_mon_op_timeout 900 2023-10-26T01:42:02.749 INFO:teuthology.orchestra.run.smithi169.stdout:client advanced rados_osd_op_timeout 900 2023-10-26T01:42:02.751 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:02.745+0000 7fa3d1184640 1 -- 172.21.15.169:0/2878212696 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa3ac06ae40 msgr2=0x7fa3ac06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:02.751 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:02.745+0000 7fa3d1184640 1 --2- 172.21.15.169:0/2878212696 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa3ac06ae40 0x7fa3ac06d300 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc000f60 tx=0x7fa3bc0079b0 comp rx=0 tx=0).stop 2023-10-26T01:42:02.751 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:02.745+0000 7fa3d1184640 1 -- 172.21.15.169:0/2878212696 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa3cc15bbb0 msgr2=0x7fa3cc1542e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:02.751 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:02.745+0000 7fa3d1184640 1 --2- 172.21.15.169:0/2878212696 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa3cc15bbb0 0x7fa3cc1542e0 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7fa3c4038040 tx=0x7fa3c4036db0 comp rx=0 tx=0).stop 2023-10-26T01:42:02.752 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:02.745+0000 7fa3d1184640 1 -- 172.21.15.169:0/2878212696 shutdown_connections 2023-10-26T01:42:02.752 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:02.745+0000 7fa3d1184640 1 --2- 172.21.15.169:0/2878212696 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa3ac06ae40 0x7fa3ac06d300 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:02.752 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:02.745+0000 7fa3d1184640 1 --2- 172.21.15.169:0/2878212696 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa3cc15bbb0 0x7fa3cc1542e0 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:02.752 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:02.745+0000 7fa3d1184640 1 --2- 172.21.15.169:0/2878212696 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa3cc155360 0x7fa3cc0af810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:02.752 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:02.745+0000 7fa3d1184640 1 --2- 172.21.15.169:0/2878212696 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa3cc0cdef0 0x7fa3cc06ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:02.753 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:02.745+0000 7fa3d1184640 1 -- 172.21.15.169:0/2878212696 >> 172.21.15.169:0/2878212696 conn(0x7fa3cc069930 msgr2=0x7fa3cc156ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:42:02.753 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:02.745+0000 7fa3d1184640 1 -- 172.21.15.169:0/2878212696 shutdown_connections 2023-10-26T01:42:02.753 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:02.745+0000 7fa3d1184640 1 -- 172.21.15.169:0/2878212696 wait complete. 2023-10-26T01:42:02.765 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-26T01:42:02.979 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:02.976+0000 7f743b114640 1 -- 172.21.15.169:0/3115517629 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f74340aac10 msgr2=0x7f74340b0300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:02.979 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:02.976+0000 7f743b114640 1 --2- 172.21.15.169:0/3115517629 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f74340aac10 0x7f74340b0300 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f743406e430 tx=0x7f742c02c080 comp rx=0 tx=0).stop 2023-10-26T01:42:02.979 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:02.976+0000 7f743b114640 1 -- 172.21.15.169:0/3115517629 shutdown_connections 2023-10-26T01:42:02.980 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:02.976+0000 7f743b114640 1 --2- 172.21.15.169:0/3115517629 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f74340aac10 0x7f74340b0300 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:02.980 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:02.976+0000 7f743b114640 1 --2- 172.21.15.169:0/3115517629 >> v2:172.21.15.170:3300/0 conn(0x7f743415b140 0x7f74340aa6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:02.980 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:02.976+0000 7f743b114640 1 -- 172.21.15.169:0/3115517629 >> 172.21.15.169:0/3115517629 conn(0x7f74340af110 msgr2=0x7f74340af520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:42:02.980 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:02.976+0000 7f743b114640 1 -- 172.21.15.169:0/3115517629 shutdown_connections 2023-10-26T01:42:02.980 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:02.976+0000 7f743b114640 1 -- 172.21.15.169:0/3115517629 wait complete. 2023-10-26T01:42:02.983 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:02.978+0000 7f743b114640 1 Processor -- start 2023-10-26T01:42:02.983 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:02.978+0000 7f743b114640 1 -- start start 2023-10-26T01:42:02.983 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:02.979+0000 7f743b114640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f74340aac10 0x7f7434145c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:02.983 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:02.979+0000 7f743b114640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7434157f80 0x7f7434145370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:02.983 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:02.979+0000 7f743b114640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f743415b140 0x7f743406eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:02.984 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:02.979+0000 7f743b114640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74340716f0 con 0x7f7434157f80 2023-10-26T01:42:02.984 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:02.979+0000 7f743b114640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f743414a630 con 0x7f743415b140 2023-10-26T01:42:02.984 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:02.979+0000 7f743b114640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f743414a910 con 0x7f74340aac10 2023-10-26T01:42:02.984 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:02.979+0000 7f7439911640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7434157f80 0x7f7434145370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:42:02.984 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:02.979+0000 7f7439911640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7434157f80 0x7f7434145370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:37798/0 (socket says 172.21.15.169:37798) 2023-10-26T01:42:02.985 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:02.979+0000 7f7439911640 1 -- 172.21.15.169:0/1236121105 learned_addr learned my addr 172.21.15.169:0/1236121105 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:42:02.985 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:02.979+0000 7f743a112640 1 --2- 172.21.15.169:0/1236121105 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f74340aac10 0x7f7434145c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:42:02.985 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:02.979+0000 7f7439911640 1 -- 172.21.15.169:0/1236121105 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f74340aac10 msgr2=0x7f7434145c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:02.985 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:02.979+0000 7f7439911640 1 --2- 172.21.15.169:0/1236121105 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f74340aac10 0x7f7434145c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:02.985 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:02.979+0000 7f7439911640 1 -- 172.21.15.169:0/1236121105 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f743415b140 msgr2=0x7f743406eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:02.986 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:02.979+0000 7f7439911640 1 --2- 172.21.15.169:0/1236121105 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f743415b140 0x7f743406eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:02.986 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:02.979+0000 7f7439911640 1 -- 172.21.15.169:0/1236121105 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f742c009840 con 0x7f7434157f80 2023-10-26T01:42:02.986 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:02.980+0000 7f7439911640 1 --2- 172.21.15.169:0/1236121105 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7434157f80 0x7f7434145370 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f742800d950 tx=0x7f742800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:42:02.986 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:02.980+0000 7f7426ffd640 1 -- 172.21.15.169:0/1236121105 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7428009c30 con 0x7f7434157f80 2023-10-26T01:42:02.986 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:02.980+0000 7f7426ffd640 1 -- 172.21.15.169:0/1236121105 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7428009e80 con 0x7f7434157f80 2023-10-26T01:42:02.987 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:02.980+0000 7f743b114640 1 -- 172.21.15.169:0/1236121105 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f743414abf0 con 0x7f7434157f80 2023-10-26T01:42:02.987 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:02.980+0000 7f7426ffd640 1 -- 172.21.15.169:0/1236121105 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74280106c0 con 0x7f7434157f80 2023-10-26T01:42:02.987 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:02.980+0000 7f743b114640 1 -- 172.21.15.169:0/1236121105 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7434146f30 con 0x7f7434157f80 2023-10-26T01:42:02.988 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:02.981+0000 7f743b114640 1 -- 172.21.15.169:0/1236121105 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7434073c10 con 0x7f7434157f80 2023-10-26T01:42:02.988 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:02.982+0000 7f7426ffd640 1 -- 172.21.15.169:0/1236121105 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f7428010820 con 0x7f7434157f80 2023-10-26T01:42:02.989 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:02.983+0000 7f7426ffd640 1 --2- 172.21.15.169:0/1236121105 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f741806ae40 0x7f741806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:02.989 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:02.983+0000 7f7426ffd640 1 -- 172.21.15.169:0/1236121105 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f7428089810 con 0x7f7434157f80 2023-10-26T01:42:02.989 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:02.984+0000 7f743a112640 1 --2- 172.21.15.169:0/1236121105 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f741806ae40 0x7f741806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:42:02.989 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:02.985+0000 7f743a112640 1 --2- 172.21.15.169:0/1236121105 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f741806ae40 0x7f741806d300 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f74300011e0 tx=0x7f7430009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:42:02.990 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:02.988+0000 7f7426ffd640 1 -- 172.21.15.169:0/1236121105 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7428054bb0 con 0x7f7434157f80 2023-10-26T01:42:03.179 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:03.176+0000 7f743b114640 1 -- 172.21.15.169:0/1236121105 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f7434158360 con 0x7f7434157f80 2023-10-26T01:42:03.185 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:03.183+0000 7f7426ffd640 1 -- 172.21.15.169:0/1236121105 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v35)=0 v35) v1 ==== 122+0+0 (secure 0 0 0) 0x7f74280200c0 con 0x7f7434157f80 2023-10-26T01:42:03.189 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:03.186+0000 7f743b114640 1 -- 172.21.15.169:0/1236121105 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f741806ae40 msgr2=0x7f741806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:03.189 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:03.186+0000 7f743b114640 1 --2- 172.21.15.169:0/1236121105 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f741806ae40 0x7f741806d300 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f74300011e0 tx=0x7f7430009040 comp rx=0 tx=0).stop 2023-10-26T01:42:03.189 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:03.186+0000 7f743b114640 1 -- 172.21.15.169:0/1236121105 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7434157f80 msgr2=0x7f7434145370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:03.189 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:03.186+0000 7f743b114640 1 --2- 172.21.15.169:0/1236121105 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7434157f80 0x7f7434145370 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f742800d950 tx=0x7f742800de10 comp rx=0 tx=0).stop 2023-10-26T01:42:03.190 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:03.186+0000 7f743b114640 1 -- 172.21.15.169:0/1236121105 shutdown_connections 2023-10-26T01:42:03.190 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:03.186+0000 7f743b114640 1 --2- 172.21.15.169:0/1236121105 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f741806ae40 0x7f741806d300 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:03.190 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:03.186+0000 7f743b114640 1 --2- 172.21.15.169:0/1236121105 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7434157f80 0x7f7434145370 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:03.190 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:03.186+0000 7f743b114640 1 --2- 172.21.15.169:0/1236121105 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f743415b140 0x7f743406eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:03.190 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:03.186+0000 7f743b114640 1 --2- 172.21.15.169:0/1236121105 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f74340aac10 0x7f7434145c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:03.191 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:03.186+0000 7f743b114640 1 -- 172.21.15.169:0/1236121105 >> 172.21.15.169:0/1236121105 conn(0x7f74340af110 msgr2=0x7f7434151190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:42:03.191 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:03.186+0000 7f743b114640 1 -- 172.21.15.169:0/1236121105 shutdown_connections 2023-10-26T01:42:03.191 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:03.186+0000 7f743b114640 1 -- 172.21.15.169:0/1236121105 wait complete. 2023-10-26T01:42:03.207 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-26T01:42:03.207 DEBUG:teuthology.orchestra.run.smithi169:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-26T01:42:03.333 DEBUG:teuthology.orchestra.run.smithi170:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-26T01:42:03.452 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-26T01:42:03.452 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:42:03.452 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:42:03.513 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054301,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-26T01:42:03.513 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-26T01:42:03.514 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:42:03.575 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054303,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-26T01:42:03.575 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:42:03.576 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:42:03.636 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054301,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-26T01:42:03.637 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-26T01:42:03.637 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:42:03.698 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054303,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-26T01:42:03.698 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-26T01:42:03.699 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T01:42:03.919 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:03.916+0000 7f40d769b640 1 -- 172.21.15.169:0/3656825906 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40d006ce20 msgr2=0x7f40d014e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:03.919 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:03.916+0000 7f40d769b640 1 --2- 172.21.15.169:0/3656825906 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40d006ce20 0x7f40d014e790 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f40cc009900 tx=0x7f40cc02bce0 comp rx=0 tx=0).stop 2023-10-26T01:42:03.920 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:03.916+0000 7f40d769b640 1 -- 172.21.15.169:0/3656825906 shutdown_connections 2023-10-26T01:42:03.920 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:03.916+0000 7f40d769b640 1 --2- 172.21.15.169:0/3656825906 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f40d0159e10 0x7f40d015a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:03.920 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:03.916+0000 7f40d769b640 1 --2- 172.21.15.169:0/3656825906 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40d006ce20 0x7f40d014e790 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:03.920 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:03.916+0000 7f40d769b640 1 -- 172.21.15.169:0/3656825906 >> 172.21.15.169:0/3656825906 conn(0x7f40d0069930 msgr2=0x7f40d0069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:42:03.920 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:03.916+0000 7f40d769b640 1 -- 172.21.15.169:0/3656825906 shutdown_connections 2023-10-26T01:42:03.921 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:03.916+0000 7f40d769b640 1 -- 172.21.15.169:0/3656825906 wait complete. 2023-10-26T01:42:03.922 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:03.918+0000 7f40d769b640 1 Processor -- start 2023-10-26T01:42:03.922 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:03.918+0000 7f40d769b640 1 -- start start 2023-10-26T01:42:03.922 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:03.919+0000 7f40d769b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40d006ce20 0x7f40d01e49e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:03.922 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:03.919+0000 7f40d769b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f40d0158850 0x7f40d00aff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:03.923 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:03.919+0000 7f40d769b640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f40d0159e10 0x7f40d014cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:03.923 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:03.919+0000 7f40d769b640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40d00b0e70 con 0x7f40d006ce20 2023-10-26T01:42:03.923 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:03.919+0000 7f40d769b640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40d014d490 con 0x7f40d0158850 2023-10-26T01:42:03.923 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:03.919+0000 7f40d769b640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40d01e21a0 con 0x7f40d0159e10 2023-10-26T01:42:03.924 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:03.919+0000 7f40d5437640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40d006ce20 0x7f40d01e49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:42:03.924 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:03.919+0000 7f40d5437640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40d006ce20 0x7f40d01e49e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:37818/0 (socket says 172.21.15.169:37818) 2023-10-26T01:42:03.924 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:03.919+0000 7f40d5437640 1 -- 172.21.15.169:0/1086205524 learned_addr learned my addr 172.21.15.169:0/1086205524 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:42:03.925 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:03.920+0000 7f40d4c36640 1 --2- 172.21.15.169:0/1086205524 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f40d0158850 0x7f40d00aff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:42:03.925 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:03.920+0000 7f40d5437640 1 -- 172.21.15.169:0/1086205524 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f40d0159e10 msgr2=0x7f40d014cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:03.926 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:03.920+0000 7f40d5437640 1 --2- 172.21.15.169:0/1086205524 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f40d0159e10 0x7f40d014cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:03.926 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:03.920+0000 7f40d5437640 1 -- 172.21.15.169:0/1086205524 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f40d0158850 msgr2=0x7f40d00aff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:03.926 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:03.920+0000 7f40d5437640 1 --2- 172.21.15.169:0/1086205524 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f40d0158850 0x7f40d00aff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:03.926 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:03.920+0000 7f40d5437640 1 -- 172.21.15.169:0/1086205524 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40cc009560 con 0x7f40d006ce20 2023-10-26T01:42:03.926 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:03.920+0000 7f40d4c36640 1 --2- 172.21.15.169:0/1086205524 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f40d0158850 0x7f40d00aff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:42:03.927 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:03.920+0000 7f40d5437640 1 --2- 172.21.15.169:0/1086205524 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40d006ce20 0x7f40d01e49e0 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f40cc036040 tx=0x7f40cc02dcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:42:03.927 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:03.920+0000 7f40c5ffb640 1 -- 172.21.15.169:0/1086205524 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40cc02dea0 con 0x7f40d006ce20 2023-10-26T01:42:03.927 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:03.920+0000 7f40c5ffb640 1 -- 172.21.15.169:0/1086205524 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f40cc003800 con 0x7f40d006ce20 2023-10-26T01:42:03.927 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:03.920+0000 7f40c5ffb640 1 -- 172.21.15.169:0/1086205524 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40cc003c10 con 0x7f40d006ce20 2023-10-26T01:42:03.928 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:03.920+0000 7f40d769b640 1 -- 172.21.15.169:0/1086205524 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40d01e3cc0 con 0x7f40d006ce20 2023-10-26T01:42:03.928 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:03.921+0000 7f40d769b640 1 -- 172.21.15.169:0/1086205524 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40d00b08b0 con 0x7f40d006ce20 2023-10-26T01:42:03.928 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:03.921+0000 7f40d769b640 1 -- 172.21.15.169:0/1086205524 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40d01e1f70 con 0x7f40d006ce20 2023-10-26T01:42:03.928 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:03.923+0000 7f40c5ffb640 1 -- 172.21.15.169:0/1086205524 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f40cc003d70 con 0x7f40d006ce20 2023-10-26T01:42:03.929 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:03.923+0000 7f40c5ffb640 1 --2- 172.21.15.169:0/1086205524 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f40b406ae70 0x7f40b406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:03.929 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:03.923+0000 7f40c5ffb640 1 -- 172.21.15.169:0/1086205524 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f40cc0aba90 con 0x7f40d006ce20 2023-10-26T01:42:03.929 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:03.924+0000 7f40d4c36640 1 --2- 172.21.15.169:0/1086205524 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f40b406ae70 0x7f40b406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:42:03.929 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:03.925+0000 7f40d4c36640 1 --2- 172.21.15.169:0/1086205524 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f40b406ae70 0x7f40b406d330 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f40d00cd220 tx=0x7f40c0009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:42:03.931 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:03.928+0000 7f40c5ffb640 1 -- 172.21.15.169:0/1086205524 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f40d01e1f70 con 0x7f40d006ce20 2023-10-26T01:42:04.142 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:04.139+0000 7f40d769b640 1 -- 172.21.15.169:0/1086205524 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f40d01e1f70 con 0x7f40d006ce20 2023-10-26T01:42:04.143 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:04.140+0000 7f40c5ffb640 1 -- 172.21.15.169:0/1086205524 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) v1 ==== 94+0+3711 (secure 0 0 0) 0x7f40cc042020 con 0x7f40d006ce20 2023-10-26T01:42:04.143 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:42:04.144 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":79,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5711,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/3037106607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":3037106607},{"type":"v1","addr":"172.21.15.170:6836","nonce":3037106607}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72},{"gid":5714,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/1847866184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":1847866184},{"type":"v1","addr":"172.21.15.170:6837","nonce":1847866184}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72},{"gid":8229,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2707290348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2707290348},{"type":"v1","addr":"172.21.15.169:6833","nonce":2707290348}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":71},{"gid":8607,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1036997813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1036997813},{"type":"v1","addr":"172.21.15.169:6835","nonce":1036997813}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":79}],"filesystems":[]} 2023-10-26T01:42:04.144 INFO:teuthology.orchestra.run.smithi169.stderr:dumped fsmap epoch 79 2023-10-26T01:42:04.146 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:04.143+0000 7f40d769b640 1 -- 172.21.15.169:0/1086205524 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f40b406ae70 msgr2=0x7f40b406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:04.146 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:04.143+0000 7f40d769b640 1 --2- 172.21.15.169:0/1086205524 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f40b406ae70 0x7f40b406d330 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f40d00cd220 tx=0x7f40c0009040 comp rx=0 tx=0).stop 2023-10-26T01:42:04.146 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:04.143+0000 7f40d769b640 1 -- 172.21.15.169:0/1086205524 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40d006ce20 msgr2=0x7f40d01e49e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:04.147 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:04.143+0000 7f40d769b640 1 --2- 172.21.15.169:0/1086205524 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40d006ce20 0x7f40d01e49e0 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f40cc036040 tx=0x7f40cc02dcb0 comp rx=0 tx=0).stop 2023-10-26T01:42:04.147 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:04.143+0000 7f40d769b640 1 -- 172.21.15.169:0/1086205524 shutdown_connections 2023-10-26T01:42:04.147 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:04.143+0000 7f40d769b640 1 --2- 172.21.15.169:0/1086205524 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f40b406ae70 0x7f40b406d330 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:04.147 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:04.143+0000 7f40d769b640 1 --2- 172.21.15.169:0/1086205524 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f40d0159e10 0x7f40d014cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:04.147 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:04.143+0000 7f40d769b640 1 --2- 172.21.15.169:0/1086205524 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f40d0158850 0x7f40d00aff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:04.148 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:04.143+0000 7f40d769b640 1 --2- 172.21.15.169:0/1086205524 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40d006ce20 0x7f40d01e49e0 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:04.148 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:04.143+0000 7f40d769b640 1 -- 172.21.15.169:0/1086205524 >> 172.21.15.169:0/1086205524 conn(0x7f40d0069930 msgr2=0x7f40d00cd9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:42:04.148 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:04.143+0000 7f40d769b640 1 -- 172.21.15.169:0/1086205524 shutdown_connections 2023-10-26T01:42:04.148 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:04.143+0000 7f40d769b640 1 -- 172.21.15.169:0/1086205524 wait complete. 2023-10-26T01:42:04.164 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-26T01:42:04.164 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-26T01:42:04.165 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-26T01:42:04.165 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T01:42:04.165 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-26T01:42:04.165 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-26T01:42:04.166 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-26T01:42:04.166 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T01:42:04.166 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-26T01:42:04.167 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-26T01:42:04.167 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-26T01:42:04.167 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T01:42:04.167 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-26T01:42:04.168 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-26T01:42:04.168 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-26T01:42:04.168 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T01:42:04.186 INFO:tasks.ceph.mds.c:Stopped 2023-10-26T01:42:04.187 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-26T01:42:04.189 INFO:tasks.ceph.mds.a:Stopped 2023-10-26T01:42:04.189 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-26T01:42:04.198 INFO:tasks.ceph.mds.b:Stopped 2023-10-26T01:42:04.198 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-26T01:42:04.202 INFO:tasks.ceph.mds.d:Stopped 2023-10-26T01:42:04.202 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-26T01:42:04.238 INFO:tasks.ceph.mds.c:Started 2023-10-26T01:42:04.238 DEBUG:teuthology.parallel:result is None 2023-10-26T01:42:04.245 INFO:tasks.ceph.mds.a:Started 2023-10-26T01:42:04.245 DEBUG:teuthology.parallel:result is None 2023-10-26T01:42:04.247 INFO:tasks.ceph.mds.b:Started 2023-10-26T01:42:04.248 DEBUG:teuthology.parallel:result is None 2023-10-26T01:42:04.252 INFO:tasks.ceph.mds.d:Started 2023-10-26T01:42:04.252 DEBUG:teuthology.parallel:result is None 2023-10-26T01:42:04.253 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-26T01:42:04.398 INFO:tasks.ceph.mds.c.smithi169.stdout:starting mds.c at 2023-10-26T01:42:04.398 INFO:tasks.ceph.mds.a.smithi169.stdout:starting mds.a at 2023-10-26T01:42:04.408 INFO:tasks.ceph.mds.b.smithi170.stdout:starting mds.b at 2023-10-26T01:42:04.408 INFO:tasks.ceph.mds.d.smithi170.stdout:starting mds.d at 2023-10-26T01:42:04.445 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:04.442+0000 7f0bb298e640 1 -- 172.21.15.169:0/2724763458 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0bac0cdef0 msgr2=0x7f0bac15b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:04.445 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:04.442+0000 7f0bb298e640 1 --2- 172.21.15.169:0/2724763458 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0bac0cdef0 0x7f0bac15b4a0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f0ba0009830 tx=0x7f0ba002bd50 comp rx=0 tx=0).stop 2023-10-26T01:42:04.445 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:04.442+0000 7f0bb298e640 1 -- 172.21.15.169:0/2724763458 shutdown_connections 2023-10-26T01:42:04.446 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:04.442+0000 7f0bb298e640 1 --2- 172.21.15.169:0/2724763458 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0bac15b9e0 0x7f0bac0b05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:04.446 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:04.442+0000 7f0bb298e640 1 --2- 172.21.15.169:0/2724763458 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0bac0cdef0 0x7f0bac15b4a0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:04.446 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:04.442+0000 7f0bb298e640 1 -- 172.21.15.169:0/2724763458 >> 172.21.15.169:0/2724763458 conn(0x7f0bac069930 msgr2=0x7f0bac069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:42:04.446 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:04.442+0000 7f0bb298e640 1 -- 172.21.15.169:0/2724763458 shutdown_connections 2023-10-26T01:42:04.446 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:04.442+0000 7f0bb298e640 1 -- 172.21.15.169:0/2724763458 wait complete. 2023-10-26T01:42:04.448 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:04.444+0000 7f0bb298e640 1 Processor -- start 2023-10-26T01:42:04.448 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:04.444+0000 7f0bb298e640 1 -- start start 2023-10-26T01:42:04.448 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:04.445+0000 7f0bb298e640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0bac0cdef0 0x7f0bac14adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:04.448 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:04.445+0000 7f0bb298e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0bac155360 0x7f0bac14daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:04.449 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:04.445+0000 7f0bb298e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0bac15b9e0 0x7f0bac14e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:04.449 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:04.445+0000 7f0bb298e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bac0c7450 con 0x7f0bac155360 2023-10-26T01:42:04.449 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:04.445+0000 7f0bb298e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bac0c6b40 con 0x7f0bac15b9e0 2023-10-26T01:42:04.449 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:04.445+0000 7f0bb298e640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bac1590e0 con 0x7f0bac0cdef0 2023-10-26T01:42:04.449 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:04.445+0000 7f0bab7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0bac155360 0x7f0bac14daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:42:04.449 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:04.445+0000 7f0bab7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0bac155360 0x7f0bac14daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:56090/0 (socket says 172.21.15.169:56090) 2023-10-26T01:42:04.450 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:04.445+0000 7f0bab7fe640 1 -- 172.21.15.169:0/117584829 learned_addr learned my addr 172.21.15.169:0/117584829 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:42:04.450 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:04.445+0000 7f0babfff640 1 --2- 172.21.15.169:0/117584829 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0bac0cdef0 0x7f0bac14adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:42:04.450 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:04.445+0000 7f0bb0f2b640 1 --2- 172.21.15.169:0/117584829 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0bac15b9e0 0x7f0bac14e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:42:04.450 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:04.445+0000 7f0bab7fe640 1 -- 172.21.15.169:0/117584829 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0bac0cdef0 msgr2=0x7f0bac14adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:04.450 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:04.445+0000 7f0bab7fe640 1 --2- 172.21.15.169:0/117584829 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0bac0cdef0 0x7f0bac14adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:04.451 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:04.445+0000 7f0bab7fe640 1 -- 172.21.15.169:0/117584829 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0bac15b9e0 msgr2=0x7f0bac14e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:04.451 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:04.445+0000 7f0bab7fe640 1 --2- 172.21.15.169:0/117584829 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0bac15b9e0 0x7f0bac14e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:04.451 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:04.445+0000 7f0bab7fe640 1 -- 172.21.15.169:0/117584829 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ba0009490 con 0x7f0bac155360 2023-10-26T01:42:04.451 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:04.445+0000 7f0babfff640 1 --2- 172.21.15.169:0/117584829 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0bac0cdef0 0x7f0bac14adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T01:42:04.452 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:04.446+0000 7f0bab7fe640 1 --2- 172.21.15.169:0/117584829 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0bac155360 0x7f0bac14daf0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f0ba0000c00 tx=0x7f0ba0003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:42:04.452 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:04.446+0000 7f0ba8ff9640 1 -- 172.21.15.169:0/117584829 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba0003e20 con 0x7f0bac155360 2023-10-26T01:42:04.453 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:04.446+0000 7f0ba8ff9640 1 -- 172.21.15.169:0/117584829 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0ba0034440 con 0x7f0bac155360 2023-10-26T01:42:04.453 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:04.446+0000 7f0ba8ff9640 1 -- 172.21.15.169:0/117584829 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba0034850 con 0x7f0bac155360 2023-10-26T01:42:04.453 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:04.446+0000 7f0bb298e640 1 -- 172.21.15.169:0/117584829 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bac0c6d90 con 0x7f0bac155360 2023-10-26T01:42:04.453 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:04.446+0000 7f0bb298e640 1 -- 172.21.15.169:0/117584829 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bac157320 con 0x7f0bac155360 2023-10-26T01:42:04.453 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:04.447+0000 7f0bb298e640 1 -- 172.21.15.169:0/117584829 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bac04db10 con 0x7f0bac155360 2023-10-26T01:42:04.454 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:04.448+0000 7f0ba8ff9640 1 -- 172.21.15.169:0/117584829 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f0ba00349b0 con 0x7f0bac155360 2023-10-26T01:42:04.454 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:04.449+0000 7f0ba8ff9640 1 --2- 172.21.15.169:0/117584829 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f0b9806ae40 0x7f0b9806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:04.454 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:04.449+0000 7f0ba8ff9640 1 -- 172.21.15.169:0/117584829 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f0ba00aba80 con 0x7f0bac155360 2023-10-26T01:42:04.454 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:04.449+0000 7f0babfff640 1 --2- 172.21.15.169:0/117584829 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f0b9806ae40 0x7f0b9806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:42:04.455 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:04.450+0000 7f0babfff640 1 --2- 172.21.15.169:0/117584829 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f0b9806ae40 0x7f0b9806d300 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c001710 tx=0x7f0b9c009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:42:04.456 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:04.453+0000 7f0ba8ff9640 1 -- 172.21.15.169:0/117584829 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0ba0076e20 con 0x7f0bac155360 2023-10-26T01:42:04.653 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:04.650+0000 7f0bb298e640 1 -- 172.21.15.169:0/117584829 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f0bac0c97f0 con 0x7f0bac155360 2023-10-26T01:42:05.344 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:05.341+0000 7f0ba8ff9640 1 -- 172.21.15.169:0/117584829 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v160) v1 ==== 97+0+0 (secure 0 0 0) 0x7f0ba0039020 con 0x7f0bac155360 2023-10-26T01:42:05.344 INFO:teuthology.orchestra.run.smithi169.stderr: removed all blocklist entries 2023-10-26T01:42:05.348 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:05.345+0000 7f0bb298e640 1 -- 172.21.15.169:0/117584829 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f0b9806ae40 msgr2=0x7f0b9806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:05.348 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:05.345+0000 7f0bb298e640 1 --2- 172.21.15.169:0/117584829 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f0b9806ae40 0x7f0b9806d300 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c001710 tx=0x7f0b9c009040 comp rx=0 tx=0).stop 2023-10-26T01:42:05.348 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:05.345+0000 7f0bb298e640 1 -- 172.21.15.169:0/117584829 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0bac155360 msgr2=0x7f0bac14daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:05.348 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:05.345+0000 7f0bb298e640 1 --2- 172.21.15.169:0/117584829 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0bac155360 0x7f0bac14daf0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f0ba0000c00 tx=0x7f0ba0003820 comp rx=0 tx=0).stop 2023-10-26T01:42:05.349 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:05.345+0000 7f0bb298e640 1 -- 172.21.15.169:0/117584829 shutdown_connections 2023-10-26T01:42:05.349 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:05.345+0000 7f0bb298e640 1 --2- 172.21.15.169:0/117584829 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f0b9806ae40 0x7f0b9806d300 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:05.349 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:05.345+0000 7f0bb298e640 1 --2- 172.21.15.169:0/117584829 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0bac155360 0x7f0bac14daf0 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:05.349 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:05.345+0000 7f0bb298e640 1 --2- 172.21.15.169:0/117584829 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0bac15b9e0 0x7f0bac14e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:05.349 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:05.345+0000 7f0bb298e640 1 --2- 172.21.15.169:0/117584829 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0bac0cdef0 0x7f0bac14adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:05.350 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:05.345+0000 7f0bb298e640 1 -- 172.21.15.169:0/117584829 >> 172.21.15.169:0/117584829 conn(0x7f0bac069930 msgr2=0x7f0bac152180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:42:05.350 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:05.345+0000 7f0bb298e640 1 -- 172.21.15.169:0/117584829 shutdown_connections 2023-10-26T01:42:05.350 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:05.345+0000 7f0bb298e640 1 -- 172.21.15.169:0/117584829 wait complete. 2023-10-26T01:42:05.370 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-26T01:42:05.576 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:05.573+0000 7f5f6a656640 1 -- 172.21.15.169:0/3712370370 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f4c0039a0 msgr2=0x7f5f4c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:05.576 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:05.573+0000 7f5f6a656640 1 --2- 172.21.15.169:0/3712370370 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f4c0039a0 0x7f5f4c003dc0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f5f54009b50 tx=0x7f5f4c0057f0 comp rx=0 tx=0).stop 2023-10-26T01:42:05.576 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:05.573+0000 7f5f68bf3640 1 -- 172.21.15.169:0/3712370370 reap_dead start 2023-10-26T01:42:05.577 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:05.573+0000 7f5f6a656640 1 -- 172.21.15.169:0/3712370370 shutdown_connections 2023-10-26T01:42:05.577 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:05.573+0000 7f5f6a656640 1 -- 172.21.15.169:0/3712370370 >> 172.21.15.169:0/3712370370 conn(0x7f5f64069930 msgr2=0x7f5f64069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:42:05.577 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:05.573+0000 7f5f6a656640 1 -- 172.21.15.169:0/3712370370 shutdown_connections 2023-10-26T01:42:05.577 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:05.573+0000 7f5f6a656640 1 -- 172.21.15.169:0/3712370370 wait complete. 2023-10-26T01:42:05.578 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:05.575+0000 7f5f6a656640 1 Processor -- start 2023-10-26T01:42:05.578 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:05.575+0000 7f5f6a656640 1 -- start start 2023-10-26T01:42:05.580 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:05.576+0000 7f5f6a656640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5f4c0039a0 0x7f5f6415a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:05.580 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:05.576+0000 7f5f6a656640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f64144160 0x7f5f641e2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:05.580 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:05.576+0000 7f5f6a656640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5f641445e0 0x7f5f641e3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:05.580 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:05.576+0000 7f5f6a656640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f64147be0 con 0x7f5f64144160 2023-10-26T01:42:05.581 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:05.576+0000 7f5f6a656640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f6414d280 con 0x7f5f641445e0 2023-10-26T01:42:05.581 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:05.576+0000 7f5f6a656640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f6414cfa0 con 0x7f5f4c0039a0 2023-10-26T01:42:05.581 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:05.576+0000 7f5f637fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f64144160 0x7f5f641e2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:42:05.581 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:05.576+0000 7f5f637fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f64144160 0x7f5f641e2ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:56104/0 (socket says 172.21.15.169:56104) 2023-10-26T01:42:05.581 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:05.576+0000 7f5f637fe640 1 -- 172.21.15.169:0/423193618 learned_addr learned my addr 172.21.15.169:0/423193618 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:42:05.582 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:05.577+0000 7f5f637fe640 1 -- 172.21.15.169:0/423193618 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5f4c0039a0 msgr2=0x7f5f6415a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:05.582 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:05.577+0000 7f5f637fe640 1 --2- 172.21.15.169:0/423193618 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5f4c0039a0 0x7f5f6415a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:05.582 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:05.577+0000 7f5f637fe640 1 -- 172.21.15.169:0/423193618 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5f641445e0 msgr2=0x7f5f641e3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:05.582 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:05.577+0000 7f5f637fe640 1 --2- 172.21.15.169:0/423193618 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5f641445e0 0x7f5f641e3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:05.582 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:05.577+0000 7f5f637fe640 1 -- 172.21.15.169:0/423193618 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f64147520 con 0x7f5f64144160 2023-10-26T01:42:05.583 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:05.577+0000 7f5f637fe640 1 --2- 172.21.15.169:0/423193618 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f64144160 0x7f5f641e2ff0 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f5f5800dc90 tx=0x7f5f5800b620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:42:05.583 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:05.577+0000 7f5f60ff9640 1 -- 172.21.15.169:0/423193618 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f5800bd30 con 0x7f5f64144160 2023-10-26T01:42:05.583 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:05.577+0000 7f5f60ff9640 1 -- 172.21.15.169:0/423193618 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5f58003970 con 0x7f5f64144160 2023-10-26T01:42:05.583 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:05.577+0000 7f5f60ff9640 1 -- 172.21.15.169:0/423193618 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f58003d00 con 0x7f5f64144160 2023-10-26T01:42:05.583 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:05.577+0000 7f5f6a656640 1 -- 172.21.15.169:0/423193618 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f64147800 con 0x7f5f64144160 2023-10-26T01:42:05.583 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:05.577+0000 7f5f6a656640 1 -- 172.21.15.169:0/423193618 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f641e3c50 con 0x7f5f64144160 2023-10-26T01:42:05.584 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:05.578+0000 7f5f6a656640 1 -- 172.21.15.169:0/423193618 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f640ab440 con 0x7f5f64144160 2023-10-26T01:42:05.584 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:05.579+0000 7f5f60ff9640 1 -- 172.21.15.169:0/423193618 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f5f580060d0 con 0x7f5f64144160 2023-10-26T01:42:05.585 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:05.581+0000 7f5f60ff9640 1 --2- 172.21.15.169:0/423193618 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5f5006ae40 0x7f5f5006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:05.585 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:05.581+0000 7f5f60ff9640 1 -- 172.21.15.169:0/423193618 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5f5808a100 con 0x7f5f64144160 2023-10-26T01:42:05.585 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:05.581+0000 7f5f63fff640 1 --2- 172.21.15.169:0/423193618 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5f5006ae40 0x7f5f5006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:42:05.585 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:05.582+0000 7f5f63fff640 1 --2- 172.21.15.169:0/423193618 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5f5006ae40 0x7f5f5006d300 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f5f54036000 tx=0x7f5f54035040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:42:05.588 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:05.585+0000 7f5f60ff9640 1 -- 172.21.15.169:0/423193618 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5f580555b0 con 0x7f5f64144160 2023-10-26T01:42:05.821 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:05.818+0000 7f5f6a656640 1 -- 172.21.15.169:0/423193618 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f5f641e1d50 con 0x7f5f64144160 2023-10-26T01:42:05.822 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:05.819+0000 7f5f60ff9640 1 -- 172.21.15.169:0/423193618 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v19) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f5f58014090 con 0x7f5f64144160 2023-10-26T01:42:05.822 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:42:05.823 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2023-10-26T01:42:05.823 INFO:teuthology.orchestra.run.smithi169.stdout: "auth_dump": [ 2023-10-26T01:42:05.823 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:42:05.823 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "mds.a", 2023-10-26T01:42:05.824 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAgwTllWtUfAhAA28+12VwWEF/BUEu+ErSVyA==", 2023-10-26T01:42:05.824 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:42:05.824 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow", 2023-10-26T01:42:05.824 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow *", 2023-10-26T01:42:05.824 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow *", 2023-10-26T01:42:05.825 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:42:05.825 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:42:05.825 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:42:05.825 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:42:05.826 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "mds.b", 2023-10-26T01:42:05.826 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAgwTlleA8lHxAA+jMcMo+QnYAGllpKJ6eYgQ==", 2023-10-26T01:42:05.826 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:42:05.826 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow", 2023-10-26T01:42:05.826 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow *", 2023-10-26T01:42:05.827 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow *", 2023-10-26T01:42:05.827 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:42:05.827 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:42:05.827 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:42:05.827 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:42:05.828 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "mds.c", 2023-10-26T01:42:05.828 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAgwTllaCCqEBAAokeHvwOyp/Jv2x0yHDwysw==", 2023-10-26T01:42:05.828 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:42:05.828 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow", 2023-10-26T01:42:05.828 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow *", 2023-10-26T01:42:05.829 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow *", 2023-10-26T01:42:05.829 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:42:05.829 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:42:05.829 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:42:05.829 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:42:05.830 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "mds.d", 2023-10-26T01:42:05.830 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAgwTll51JQLhAA4Wzso4zQj+Htv6cXqvwRmg==", 2023-10-26T01:42:05.830 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:42:05.830 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow", 2023-10-26T01:42:05.830 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow *", 2023-10-26T01:42:05.830 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow *", 2023-10-26T01:42:05.831 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:42:05.831 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:42:05.831 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:42:05.831 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:42:05.832 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "osd.0", 2023-10-26T01:42:05.832 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAlwTll8RZ+HRAAt/sBK+Mi91YmTyQYbhKHLg==", 2023-10-26T01:42:05.832 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:42:05.832 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow profile osd", 2023-10-26T01:42:05.832 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow profile osd", 2023-10-26T01:42:05.833 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:42:05.833 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:42:05.833 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:42:05.833 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:42:05.833 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "osd.1", 2023-10-26T01:42:05.834 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAlwTllLXwWLRAAuHB2lJ+oDJou/IJBtsYY4Q==", 2023-10-26T01:42:05.834 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:42:05.834 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow profile osd", 2023-10-26T01:42:05.834 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow profile osd", 2023-10-26T01:42:05.834 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:42:05.835 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:42:05.835 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:42:05.835 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:42:05.835 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "osd.2", 2023-10-26T01:42:05.835 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAmwTllj1nSARAA5Jl34Q0X7a71ySd4wz17lw==", 2023-10-26T01:42:05.836 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:42:05.836 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow profile osd", 2023-10-26T01:42:05.836 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow profile osd", 2023-10-26T01:42:05.836 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:42:05.836 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:42:05.837 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:42:05.837 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:42:05.837 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "osd.3", 2023-10-26T01:42:05.837 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAmwTllC8MeEhAAfGxRVSwD+YbU/U5aeenO3w==", 2023-10-26T01:42:05.837 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:42:05.838 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow profile osd", 2023-10-26T01:42:05.838 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow profile osd", 2023-10-26T01:42:05.838 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:42:05.838 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:42:05.838 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:42:05.839 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:42:05.839 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "osd.4", 2023-10-26T01:42:05.839 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAowTll4Md6DhAAuXfadUNAdouMP28MiX/D3Q==", 2023-10-26T01:42:05.839 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:42:05.839 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow profile osd", 2023-10-26T01:42:05.840 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow profile osd", 2023-10-26T01:42:05.840 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:42:05.840 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:42:05.840 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:42:05.840 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:42:05.841 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "osd.5", 2023-10-26T01:42:05.841 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAowTllDdViHRAAec9QyhZCsjJ2rgGhhcjwpQ==", 2023-10-26T01:42:05.841 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:42:05.841 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow profile osd", 2023-10-26T01:42:05.841 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow profile osd", 2023-10-26T01:42:05.841 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:42:05.842 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:42:05.842 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:42:05.842 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:42:05.842 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "osd.6", 2023-10-26T01:42:05.843 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAowTll23X5KxAAsmQvG0yq7ilkVG5fNiaCUg==", 2023-10-26T01:42:05.843 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:42:05.843 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow profile osd", 2023-10-26T01:42:05.843 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow profile osd", 2023-10-26T01:42:05.843 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:42:05.844 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:42:05.844 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:42:05.844 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:42:05.844 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "osd.7", 2023-10-26T01:42:05.844 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAowTllhcy/OhAAY5Yjheh76ibOVREqN6WBpQ==", 2023-10-26T01:42:05.844 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:42:05.845 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow profile osd", 2023-10-26T01:42:05.845 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow profile osd", 2023-10-26T01:42:05.845 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:42:05.845 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:42:05.845 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:42:05.846 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:42:05.846 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "client.0", 2023-10-26T01:42:05.846 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAgwTllKC7gOhAAPsvxGcuUA8/p8xqbUgGteA==", 2023-10-26T01:42:05.846 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:42:05.846 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow", 2023-10-26T01:42:05.847 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow r", 2023-10-26T01:42:05.847 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-26T01:42:05.847 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:42:05.847 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:42:05.847 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:42:05.848 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "client.1", 2023-10-26T01:42:05.848 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAhwTllhxb6ExAAeYTsurWpnjohWlrNIlLpsw==", 2023-10-26T01:42:05.848 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:42:05.848 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow", 2023-10-26T01:42:05.848 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow r", 2023-10-26T01:42:05.849 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow rw", 2023-10-26T01:42:05.849 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow rwx" 2023-10-26T01:42:05.849 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:42:05.849 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:42:05.849 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:42:05.850 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "client.2", 2023-10-26T01:42:05.850 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAhwTllLWMoChAAsUHzBc29GDqmEQAFnl7dHg==", 2023-10-26T01:42:05.850 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:42:05.850 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow", 2023-10-26T01:42:05.850 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow r", 2023-10-26T01:42:05.851 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow rw", 2023-10-26T01:42:05.851 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow rwx" 2023-10-26T01:42:05.851 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:42:05.851 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:42:05.851 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:42:05.852 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "client.3", 2023-10-26T01:42:05.852 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAhwTll875HHRAAGbwikoR/hDU8P/bj0OPWAw==", 2023-10-26T01:42:05.852 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:42:05.852 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow", 2023-10-26T01:42:05.852 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow r", 2023-10-26T01:42:05.853 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow rw", 2023-10-26T01:42:05.853 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow rwx" 2023-10-26T01:42:05.853 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:42:05.853 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:42:05.854 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:42:05.854 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "client.admin", 2023-10-26T01:42:05.854 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAewTll2yddMhAAO8bnXNevDGe/KEeNNa36pQ==", 2023-10-26T01:42:05.854 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:42:05.855 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow *", 2023-10-26T01:42:05.855 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow *", 2023-10-26T01:42:05.855 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow *", 2023-10-26T01:42:05.855 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:42:05.855 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:42:05.856 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:42:05.856 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:42:05.856 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "mgr.x", 2023-10-26T01:42:05.856 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAfwTllPu3FMxAAV8d5oRFXAeeZH7e1xzPEDg==", 2023-10-26T01:42:05.857 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:42:05.857 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow *", 2023-10-26T01:42:05.857 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow profile mgr", 2023-10-26T01:42:05.857 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:42:05.857 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:42:05.858 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:42:05.858 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:42:05.858 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "mgr.y", 2023-10-26T01:42:05.858 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAfwTllZc+DJxAAsZ2TWn8H7Kw/lNTc2qyKzA==", 2023-10-26T01:42:05.858 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:42:05.859 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow *", 2023-10-26T01:42:05.859 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow profile mgr", 2023-10-26T01:42:05.859 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:42:05.859 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:42:05.859 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:42:05.860 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:42:05.860 INFO:teuthology.orchestra.run.smithi169.stdout:} 2023-10-26T01:42:05.861 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:05.822+0000 7f5f6a656640 1 -- 172.21.15.169:0/423193618 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5f5006ae40 msgr2=0x7f5f5006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:05.862 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:05.822+0000 7f5f6a656640 1 --2- 172.21.15.169:0/423193618 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5f5006ae40 0x7f5f5006d300 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f5f54036000 tx=0x7f5f54035040 comp rx=0 tx=0).stop 2023-10-26T01:42:05.862 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:05.822+0000 7f5f6a656640 1 -- 172.21.15.169:0/423193618 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f64144160 msgr2=0x7f5f641e2ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:05.862 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:05.822+0000 7f5f6a656640 1 --2- 172.21.15.169:0/423193618 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f64144160 0x7f5f641e2ff0 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f5f5800dc90 tx=0x7f5f5800b620 comp rx=0 tx=0).stop 2023-10-26T01:42:05.862 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:05.822+0000 7f5f6a656640 1 -- 172.21.15.169:0/423193618 shutdown_connections 2023-10-26T01:42:05.863 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:05.822+0000 7f5f6a656640 1 --2- 172.21.15.169:0/423193618 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5f5006ae40 0x7f5f5006d300 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:05.863 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:05.822+0000 7f5f6a656640 1 --2- 172.21.15.169:0/423193618 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f64144160 0x7f5f641e2ff0 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:05.863 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:05.822+0000 7f5f6a656640 1 --2- 172.21.15.169:0/423193618 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5f641445e0 0x7f5f641e3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:05.863 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:05.822+0000 7f5f6a656640 1 --2- 172.21.15.169:0/423193618 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5f4c0039a0 0x7f5f6415a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:05.863 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:05.822+0000 7f5f6a656640 1 -- 172.21.15.169:0/423193618 >> 172.21.15.169:0/423193618 conn(0x7f5f64069930 msgr2=0x7f5f641dfda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:42:05.864 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:05.822+0000 7f5f6a656640 1 -- 172.21.15.169:0/423193618 shutdown_connections 2023-10-26T01:42:05.864 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:05.822+0000 7f5f6a656640 1 -- 172.21.15.169:0/423193618 wait complete. 2023-10-26T01:42:05.864 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:42:06.061 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.058+0000 7f5617c1a640 1 -- 172.21.15.169:0/2858464252 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f56100cdc60 msgr2=0x7f5610159e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:06.061 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.058+0000 7f5617c1a640 1 --2- 172.21.15.169:0/2858464252 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f56100cdc60 0x7f5610159e10 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f5608005fd0 tx=0x7f560802c050 comp rx=0 tx=0).stop 2023-10-26T01:42:06.062 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.058+0000 7f5617c1a640 1 -- 172.21.15.169:0/2858464252 shutdown_connections 2023-10-26T01:42:06.062 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.058+0000 7f5617c1a640 1 --2- 172.21.15.169:0/2858464252 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f56100cdc60 0x7f5610159e10 unknown :-1 s=CLOSED pgs=1667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:06.062 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.058+0000 7f5617c1a640 1 --2- 172.21.15.169:0/2858464252 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5610158850 0x7f56100cd720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:06.062 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.058+0000 7f5617c1a640 1 -- 172.21.15.169:0/2858464252 >> 172.21.15.169:0/2858464252 conn(0x7f5610069930 msgr2=0x7f5610069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:42:06.062 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.058+0000 7f5617c1a640 1 -- 172.21.15.169:0/2858464252 shutdown_connections 2023-10-26T01:42:06.063 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.058+0000 7f5617c1a640 1 -- 172.21.15.169:0/2858464252 wait complete. 2023-10-26T01:42:06.064 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.060+0000 7f5617c1a640 1 Processor -- start 2023-10-26T01:42:06.065 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.060+0000 7f5617c1a640 1 -- start start 2023-10-26T01:42:06.065 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.061+0000 7f5617c1a640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f56100cdc60 0x7f561014cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:06.065 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.061+0000 7f5617c1a640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f561014e790 0x7f56100705d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:06.065 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.061+0000 7f5617c1a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5610158850 0x7f5610070b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:06.066 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.061+0000 7f5617c1a640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5610153c80 con 0x7f561014e790 2023-10-26T01:42:06.066 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.061+0000 7f5617c1a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5610153970 con 0x7f5610158850 2023-10-26T01:42:06.066 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.061+0000 7f5617c1a640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f561009a930 con 0x7f56100cdc60 2023-10-26T01:42:06.066 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.061+0000 7f56151b5640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f561014e790 0x7f56100705d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:42:06.066 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.061+0000 7f56151b5640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f561014e790 0x7f56100705d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:56116/0 (socket says 172.21.15.169:56116) 2023-10-26T01:42:06.067 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.061+0000 7f56151b5640 1 -- 172.21.15.169:0/3445066307 learned_addr learned my addr 172.21.15.169:0/3445066307 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:42:06.067 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.061+0000 7f56151b5640 1 -- 172.21.15.169:0/3445066307 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f56100cdc60 msgr2=0x7f561014cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:06.067 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.061+0000 7f56151b5640 1 --2- 172.21.15.169:0/3445066307 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f56100cdc60 0x7f561014cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:06.067 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.061+0000 7f56151b5640 1 -- 172.21.15.169:0/3445066307 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5610158850 msgr2=0x7f5610070b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:06.067 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.061+0000 7f56151b5640 1 --2- 172.21.15.169:0/3445066307 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5610158850 0x7f5610070b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:06.068 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.061+0000 7f56151b5640 1 -- 172.21.15.169:0/3445066307 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5608009840 con 0x7f561014e790 2023-10-26T01:42:06.068 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.062+0000 7f56151b5640 1 --2- 172.21.15.169:0/3445066307 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f561014e790 0x7f56100705d0 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f560000d750 tx=0x7f560000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:42:06.068 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.062+0000 7f56067fc640 1 -- 172.21.15.169:0/3445066307 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5600004220 con 0x7f561014e790 2023-10-26T01:42:06.068 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.062+0000 7f56067fc640 1 -- 172.21.15.169:0/3445066307 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5600004470 con 0x7f561014e790 2023-10-26T01:42:06.068 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.062+0000 7f5617c1a640 1 -- 172.21.15.169:0/3445066307 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f561009ac10 con 0x7f561014e790 2023-10-26T01:42:06.069 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.062+0000 7f56067fc640 1 -- 172.21.15.169:0/3445066307 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5600009c30 con 0x7f561014e790 2023-10-26T01:42:06.069 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.062+0000 7f5617c1a640 1 -- 172.21.15.169:0/3445066307 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f561014d1c0 con 0x7f561014e790 2023-10-26T01:42:06.070 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.063+0000 7f5617c1a640 1 -- 172.21.15.169:0/3445066307 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f561014d380 con 0x7f561014e790 2023-10-26T01:42:06.070 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.064+0000 7f56067fc640 1 -- 172.21.15.169:0/3445066307 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f5600005020 con 0x7f561014e790 2023-10-26T01:42:06.070 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.065+0000 7f56067fc640 1 --2- 172.21.15.169:0/3445066307 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f55f406ae40 0x7f55f406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:06.070 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.065+0000 7f56067fc640 1 -- 172.21.15.169:0/3445066307 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5600089740 con 0x7f561014e790 2023-10-26T01:42:06.071 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.066+0000 7f56159b6640 1 --2- 172.21.15.169:0/3445066307 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f55f406ae40 0x7f55f406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:42:06.071 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.067+0000 7f56159b6640 1 --2- 172.21.15.169:0/3445066307 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f55f406ae40 0x7f55f406d300 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f560c007fd0 tx=0x7f560c0078e0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:42:06.072 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.070+0000 7f56067fc640 1 -- 172.21.15.169:0/3445066307 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f561014d380 con 0x7f561014e790 2023-10-26T01:42:06.265 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.262+0000 7f5617c1a640 1 -- 172.21.15.169:0/3445066307 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f561014d350 con 0x7f561014e790 2023-10-26T01:42:06.267 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.264+0000 7f56067fc640 1 -- 172.21.15.169:0/3445066307 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v162) v1 ==== 74+0+12916 (secure 0 0 0) 0x7f5600017020 con 0x7f561014e790 2023-10-26T01:42:06.267 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:42:06.268 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":162,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:42:05.353200+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:42:06.269 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.267+0000 7f5617c1a640 1 -- 172.21.15.169:0/3445066307 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f55f406ae40 msgr2=0x7f55f406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:06.270 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.267+0000 7f5617c1a640 1 --2- 172.21.15.169:0/3445066307 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f55f406ae40 0x7f55f406d300 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f560c007fd0 tx=0x7f560c0078e0 comp rx=0 tx=0).stop 2023-10-26T01:42:06.270 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.267+0000 7f5617c1a640 1 -- 172.21.15.169:0/3445066307 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f561014e790 msgr2=0x7f56100705d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:06.270 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.267+0000 7f5617c1a640 1 --2- 172.21.15.169:0/3445066307 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f561014e790 0x7f56100705d0 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f560000d750 tx=0x7f560000dc10 comp rx=0 tx=0).stop 2023-10-26T01:42:06.270 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.267+0000 7f5617c1a640 1 -- 172.21.15.169:0/3445066307 shutdown_connections 2023-10-26T01:42:06.270 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.267+0000 7f5617c1a640 1 --2- 172.21.15.169:0/3445066307 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f55f406ae40 0x7f55f406d300 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:06.271 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.267+0000 7f5617c1a640 1 --2- 172.21.15.169:0/3445066307 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f561014e790 0x7f56100705d0 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:06.271 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.267+0000 7f5617c1a640 1 --2- 172.21.15.169:0/3445066307 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5610158850 0x7f5610070b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:06.271 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.267+0000 7f5617c1a640 1 --2- 172.21.15.169:0/3445066307 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f56100cdc60 0x7f561014cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:06.271 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.267+0000 7f5617c1a640 1 -- 172.21.15.169:0/3445066307 >> 172.21.15.169:0/3445066307 conn(0x7f5610069930 msgr2=0x7f5610069d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:42:06.272 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.267+0000 7f5617c1a640 1 -- 172.21.15.169:0/3445066307 shutdown_connections 2023-10-26T01:42:06.272 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.267+0000 7f5617c1a640 1 -- 172.21.15.169:0/3445066307 wait complete. 2023-10-26T01:42:06.290 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T01:30:32.200808+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '109', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T01:42:06.290 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T01:42:06.518 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.514+0000 7fd4c166b640 1 -- 172.21.15.169:0/1071550051 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4bc0ab070 msgr2=0x7fd4bc0ac870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:06.518 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.514+0000 7fd4c166b640 1 --2- 172.21.15.169:0/1071550051 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4bc0ab070 0x7fd4bc0ac870 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc0cd240 tx=0x7fd4ac02c640 comp rx=0 tx=0).stop 2023-10-26T01:42:06.518 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.515+0000 7fd4c166b640 1 -- 172.21.15.169:0/1071550051 shutdown_connections 2023-10-26T01:42:06.519 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.515+0000 7fd4c166b640 1 --2- 172.21.15.169:0/1071550051 >> v2:172.21.15.170:3300/0 conn(0x7fd4bc0b0a80 0x7fd4bc158f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:06.519 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.515+0000 7fd4c166b640 1 --2- 172.21.15.169:0/1071550051 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4bc0ab070 0x7fd4bc0ac870 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:06.519 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.515+0000 7fd4c166b640 1 -- 172.21.15.169:0/1071550051 >> 172.21.15.169:0/1071550051 conn(0x7fd4bc069930 msgr2=0x7fd4bc069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:42:06.519 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.515+0000 7fd4c166b640 1 -- 172.21.15.169:0/1071550051 shutdown_connections 2023-10-26T01:42:06.519 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.515+0000 7fd4c166b640 1 -- 172.21.15.169:0/1071550051 wait complete. 2023-10-26T01:42:06.521 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.516+0000 7fd4c166b640 1 Processor -- start 2023-10-26T01:42:06.522 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.517+0000 7fd4c166b640 1 -- start start 2023-10-26T01:42:06.522 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.517+0000 7fd4c166b640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd4bc0ab070 0x7fd4bc14b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:06.522 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.517+0000 7fd4c166b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd4bc0cd760 0x7fd4bc14c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:06.522 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.518+0000 7fd4c166b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4bc148c40 0x7fd4bc0ae450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:06.522 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.518+0000 7fd4c166b640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4bc14c550 con 0x7fd4bc148c40 2023-10-26T01:42:06.523 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.518+0000 7fd4c166b640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4bc14ceb0 con 0x7fd4bc0cd760 2023-10-26T01:42:06.523 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.518+0000 7fd4c166b640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4bc14b8d0 con 0x7fd4bc0ab070 2023-10-26T01:42:06.523 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.518+0000 7fd4bb7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4bc148c40 0x7fd4bc0ae450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:42:06.523 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.518+0000 7fd4bb7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4bc148c40 0x7fd4bc0ae450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:56140/0 (socket says 172.21.15.169:56140) 2023-10-26T01:42:06.523 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.518+0000 7fd4bb7fe640 1 -- 172.21.15.169:0/573685925 learned_addr learned my addr 172.21.15.169:0/573685925 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:42:06.524 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.518+0000 7fd4b27fc640 1 --2- 172.21.15.169:0/573685925 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd4bc0cd760 0x7fd4bc14c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:42:06.524 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.518+0000 7fd4baffd640 1 --2- 172.21.15.169:0/573685925 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd4bc0ab070 0x7fd4bc14b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:42:06.524 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.518+0000 7fd4bb7fe640 1 -- 172.21.15.169:0/573685925 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd4bc0ab070 msgr2=0x7fd4bc14b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:06.524 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.518+0000 7fd4bb7fe640 1 --2- 172.21.15.169:0/573685925 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd4bc0ab070 0x7fd4bc14b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:06.524 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.518+0000 7fd4bb7fe640 1 -- 172.21.15.169:0/573685925 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd4bc0cd760 msgr2=0x7fd4bc14c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:06.525 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.518+0000 7fd4bb7fe640 1 --2- 172.21.15.169:0/573685925 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd4bc0cd760 0x7fd4bc14c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:06.525 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.518+0000 7fd4bb7fe640 1 -- 172.21.15.169:0/573685925 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4ac009df0 con 0x7fd4bc148c40 2023-10-26T01:42:06.525 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.519+0000 7fd4bb7fe640 1 --2- 172.21.15.169:0/573685925 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4bc148c40 0x7fd4bc0ae450 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7fd4b400b990 tx=0x7fd4b400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:42:06.525 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.519+0000 7fd4b3fff640 1 -- 172.21.15.169:0/573685925 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4b400cba0 con 0x7fd4bc148c40 2023-10-26T01:42:06.525 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.519+0000 7fd4b3fff640 1 -- 172.21.15.169:0/573685925 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd4b400cdf0 con 0x7fd4bc148c40 2023-10-26T01:42:06.526 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.519+0000 7fd4b3fff640 1 -- 172.21.15.169:0/573685925 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4b40146a0 con 0x7fd4bc148c40 2023-10-26T01:42:06.526 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.519+0000 7fd4c166b640 1 -- 172.21.15.169:0/573685925 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4bc14bbb0 con 0x7fd4bc148c40 2023-10-26T01:42:06.526 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.519+0000 7fd4c166b640 1 -- 172.21.15.169:0/573685925 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4bc14c170 con 0x7fd4bc148c40 2023-10-26T01:42:06.527 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.520+0000 7fd4c166b640 1 -- 172.21.15.169:0/573685925 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4bc066af0 con 0x7fd4bc148c40 2023-10-26T01:42:06.527 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.521+0000 7fd4b3fff640 1 -- 172.21.15.169:0/573685925 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fd4b401d3b0 con 0x7fd4bc148c40 2023-10-26T01:42:06.528 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.522+0000 7fd4b3fff640 1 --2- 172.21.15.169:0/573685925 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd49c06ae70 0x7fd49c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:06.528 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.523+0000 7fd4b3fff640 1 -- 172.21.15.169:0/573685925 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd4b4089520 con 0x7fd4bc148c40 2023-10-26T01:42:06.528 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.523+0000 7fd4baffd640 1 --2- 172.21.15.169:0/573685925 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd49c06ae70 0x7fd49c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:42:06.528 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.524+0000 7fd4baffd640 1 --2- 172.21.15.169:0/573685925 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd49c06ae70 0x7fd49c06d330 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac035000 tx=0x7fd4ac004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:42:06.529 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.527+0000 7fd4b3fff640 1 -- 172.21.15.169:0/573685925 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd4b4054880 con 0x7fd4bc148c40 2023-10-26T01:42:06.718 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.715+0000 7fd4c166b640 1 -- 172.21.15.169:0/573685925 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd4bc14c690 con 0x7fd4bc148c40 2023-10-26T01:42:06.718 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.716+0000 7fd4b3fff640 1 -- 172.21.15.169:0/573685925 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v162) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd4bc14c690 con 0x7fd4bc148c40 2023-10-26T01:42:06.719 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 1 2023-10-26T01:42:06.721 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.718+0000 7fd4c166b640 1 -- 172.21.15.169:0/573685925 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd49c06ae70 msgr2=0x7fd49c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:06.721 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.718+0000 7fd4c166b640 1 --2- 172.21.15.169:0/573685925 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd49c06ae70 0x7fd49c06d330 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac035000 tx=0x7fd4ac004730 comp rx=0 tx=0).stop 2023-10-26T01:42:06.721 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.718+0000 7fd4c166b640 1 -- 172.21.15.169:0/573685925 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4bc148c40 msgr2=0x7fd4bc0ae450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:06.722 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.718+0000 7fd4c166b640 1 --2- 172.21.15.169:0/573685925 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4bc148c40 0x7fd4bc0ae450 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7fd4b400b990 tx=0x7fd4b400be50 comp rx=0 tx=0).stop 2023-10-26T01:42:06.722 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.718+0000 7fd4c166b640 1 -- 172.21.15.169:0/573685925 shutdown_connections 2023-10-26T01:42:06.722 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.718+0000 7fd4c166b640 1 --2- 172.21.15.169:0/573685925 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd49c06ae70 0x7fd49c06d330 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:06.723 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.718+0000 7fd4c166b640 1 --2- 172.21.15.169:0/573685925 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4bc148c40 0x7fd4bc0ae450 unknown :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:06.723 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.718+0000 7fd4c166b640 1 --2- 172.21.15.169:0/573685925 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd4bc0cd760 0x7fd4bc14c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:06.723 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.718+0000 7fd4c166b640 1 --2- 172.21.15.169:0/573685925 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd4bc0ab070 0x7fd4bc14b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:06.723 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.718+0000 7fd4c166b640 1 -- 172.21.15.169:0/573685925 >> 172.21.15.169:0/573685925 conn(0x7fd4bc069930 msgr2=0x7fd4bc0aefc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:42:06.723 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.719+0000 7fd4c166b640 1 -- 172.21.15.169:0/573685925 shutdown_connections 2023-10-26T01:42:06.724 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.719+0000 7fd4c166b640 1 -- 172.21.15.169:0/573685925 wait complete. 2023-10-26T01:42:06.739 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-26T01:42:06.958 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.955+0000 7ff9d2ba9640 1 -- 172.21.15.169:0/166757294 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9bc0039a0 msgr2=0x7ff9bc003d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:06.958 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.955+0000 7ff9d2ba9640 1 --2- 172.21.15.169:0/166757294 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9bc0039a0 0x7ff9bc003d80 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7ff9c0001840 tx=0x7ff9bc0057f0 comp rx=0 tx=0).stop 2023-10-26T01:42:06.958 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.955+0000 7ff9d1146640 1 -- 172.21.15.169:0/166757294 reap_dead start 2023-10-26T01:42:06.959 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.955+0000 7ff9d2ba9640 1 -- 172.21.15.169:0/166757294 shutdown_connections 2023-10-26T01:42:06.959 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.955+0000 7ff9d2ba9640 1 -- 172.21.15.169:0/166757294 >> 172.21.15.169:0/166757294 conn(0x7ff9cc069930 msgr2=0x7ff9cc069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:42:06.959 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.955+0000 7ff9d2ba9640 1 -- 172.21.15.169:0/166757294 shutdown_connections 2023-10-26T01:42:06.959 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.955+0000 7ff9d2ba9640 1 -- 172.21.15.169:0/166757294 wait complete. 2023-10-26T01:42:06.961 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.957+0000 7ff9d2ba9640 1 Processor -- start 2023-10-26T01:42:06.961 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.957+0000 7ff9d2ba9640 1 -- start start 2023-10-26T01:42:06.962 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.958+0000 7ff9d2ba9640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff9bc0039a0 0x7ff9cc151cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:06.962 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.958+0000 7ff9d2ba9640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff9cc13e380 0x7ff9cc153940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:06.962 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.958+0000 7ff9d2ba9640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9cc13f920 0x7ff9cc13fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:06.962 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.958+0000 7ff9d2ba9640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9cc13f490 con 0x7ff9cc13f920 2023-10-26T01:42:06.962 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.958+0000 7ff9d2ba9640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9cc13e880 con 0x7ff9bc0039a0 2023-10-26T01:42:06.963 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.958+0000 7ff9d2ba9640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9cc13eb60 con 0x7ff9cc13e380 2023-10-26T01:42:06.963 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.958+0000 7ff9d1146640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9cc13f920 0x7ff9cc13fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:42:06.963 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.958+0000 7ff9d1146640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9cc13f920 0x7ff9cc13fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:56156/0 (socket says 172.21.15.169:56156) 2023-10-26T01:42:06.963 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.958+0000 7ff9d1146640 1 -- 172.21.15.169:0/1968673039 learned_addr learned my addr 172.21.15.169:0/1968673039 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:42:06.964 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.958+0000 7ff9d0945640 1 --2- 172.21.15.169:0/1968673039 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff9bc0039a0 0x7ff9cc151cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:42:06.964 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.958+0000 7ff9cbfff640 1 --2- 172.21.15.169:0/1968673039 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff9cc13e380 0x7ff9cc153940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:42:06.964 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.958+0000 7ff9d1146640 1 -- 172.21.15.169:0/1968673039 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff9cc13e380 msgr2=0x7ff9cc153940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:06.964 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.958+0000 7ff9d1146640 1 --2- 172.21.15.169:0/1968673039 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff9cc13e380 0x7ff9cc153940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:06.964 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.958+0000 7ff9d1146640 1 -- 172.21.15.169:0/1968673039 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff9bc0039a0 msgr2=0x7ff9cc151cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:06.965 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.958+0000 7ff9d1146640 1 --2- 172.21.15.169:0/1968673039 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff9bc0039a0 0x7ff9cc151cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:06.965 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.958+0000 7ff9d1146640 1 -- 172.21.15.169:0/1968673039 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9cc13ee40 con 0x7ff9cc13f920 2023-10-26T01:42:06.965 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.959+0000 7ff9d1146640 1 --2- 172.21.15.169:0/1968673039 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9cc13f920 0x7ff9cc13fda0 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc0ab9b0 tx=0x7ff9c400ee10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:42:06.967 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.959+0000 7ff9c97fa640 1 -- 172.21.15.169:0/1968673039 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c4018070 con 0x7ff9cc13f920 2023-10-26T01:42:06.967 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.959+0000 7ff9c97fa640 1 -- 172.21.15.169:0/1968673039 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff9c4004c10 con 0x7ff9cc13f920 2023-10-26T01:42:06.967 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.959+0000 7ff9c97fa640 1 -- 172.21.15.169:0/1968673039 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c400cad0 con 0x7ff9cc13f920 2023-10-26T01:42:06.967 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.959+0000 7ff9d2ba9640 1 -- 172.21.15.169:0/1968673039 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9cc13f120 con 0x7ff9cc13f920 2023-10-26T01:42:06.967 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.959+0000 7ff9d2ba9640 1 -- 172.21.15.169:0/1968673039 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9cc153e80 con 0x7ff9cc13f920 2023-10-26T01:42:06.968 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.960+0000 7ff9d2ba9640 1 -- 172.21.15.169:0/1968673039 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9cc1584e0 con 0x7ff9cc13f920 2023-10-26T01:42:06.968 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.961+0000 7ff9c97fa640 1 -- 172.21.15.169:0/1968673039 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ff9c400cc30 con 0x7ff9cc13f920 2023-10-26T01:42:06.968 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.962+0000 7ff9c97fa640 1 --2- 172.21.15.169:0/1968673039 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff9b006ae40 0x7ff9b006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:06.968 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.962+0000 7ff9c97fa640 1 -- 172.21.15.169:0/1968673039 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ff9c4088ec0 con 0x7ff9cc13f920 2023-10-26T01:42:06.968 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.963+0000 7ff9d0945640 1 --2- 172.21.15.169:0/1968673039 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff9b006ae40 0x7ff9b006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:42:06.969 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.964+0000 7ff9d0945640 1 --2- 172.21.15.169:0/1968673039 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff9b006ae40 0x7ff9b006d300 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7ff9c0034000 tx=0x7ff9c0039040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:42:06.969 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:06.967+0000 7ff9c97fa640 1 -- 172.21.15.169:0/1968673039 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff9c4054350 con 0x7ff9cc13f920 2023-10-26T01:42:07.161 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:07.158+0000 7ff9d2ba9640 1 -- 172.21.15.169:0/1968673039 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7ff9cc04db10 con 0x7ff9cc13f920 2023-10-26T01:42:07.162 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:07.159+0000 7ff9c97fa640 1 -- 172.21.15.169:0/1968673039 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v162) v1 ==== 84+0+65 (secure 0 0 0) 0x7ff9c4057970 con 0x7ff9cc13f920 2023-10-26T01:42:07.162 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:42:07.162 INFO:teuthology.orchestra.run.smithi169.stdout:[ 2023-10-26T01:42:07.163 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:42:07.163 INFO:teuthology.orchestra.run.smithi169.stdout: "poolnum": 1, 2023-10-26T01:42:07.163 INFO:teuthology.orchestra.run.smithi169.stdout: "poolname": ".mgr" 2023-10-26T01:42:07.163 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:42:07.163 INFO:teuthology.orchestra.run.smithi169.stdout:] 2023-10-26T01:42:07.164 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:07.162+0000 7ff9d2ba9640 1 -- 172.21.15.169:0/1968673039 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff9b006ae40 msgr2=0x7ff9b006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:07.165 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:07.162+0000 7ff9d2ba9640 1 --2- 172.21.15.169:0/1968673039 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff9b006ae40 0x7ff9b006d300 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7ff9c0034000 tx=0x7ff9c0039040 comp rx=0 tx=0).stop 2023-10-26T01:42:07.165 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:07.162+0000 7ff9d2ba9640 1 -- 172.21.15.169:0/1968673039 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9cc13f920 msgr2=0x7ff9cc13fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:07.165 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:07.162+0000 7ff9d2ba9640 1 --2- 172.21.15.169:0/1968673039 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9cc13f920 0x7ff9cc13fda0 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc0ab9b0 tx=0x7ff9c400ee10 comp rx=0 tx=0).stop 2023-10-26T01:42:07.165 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:07.162+0000 7ff9d2ba9640 1 -- 172.21.15.169:0/1968673039 shutdown_connections 2023-10-26T01:42:07.166 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:07.162+0000 7ff9d2ba9640 1 --2- 172.21.15.169:0/1968673039 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff9b006ae40 0x7ff9b006d300 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:07.166 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:07.162+0000 7ff9d2ba9640 1 --2- 172.21.15.169:0/1968673039 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9cc13f920 0x7ff9cc13fda0 unknown :-1 s=CLOSED pgs=1673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:07.166 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:07.162+0000 7ff9d2ba9640 1 --2- 172.21.15.169:0/1968673039 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff9cc13e380 0x7ff9cc153940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:07.166 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:07.162+0000 7ff9d2ba9640 1 --2- 172.21.15.169:0/1968673039 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff9bc0039a0 0x7ff9cc151cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:07.166 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:07.162+0000 7ff9d2ba9640 1 -- 172.21.15.169:0/1968673039 >> 172.21.15.169:0/1968673039 conn(0x7ff9cc069930 msgr2=0x7ff9cc157dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:42:07.167 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:07.162+0000 7ff9d2ba9640 1 -- 172.21.15.169:0/1968673039 shutdown_connections 2023-10-26T01:42:07.167 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:07.162+0000 7ff9d2ba9640 1 -- 172.21.15.169:0/1968673039 wait complete. 2023-10-26T01:42:07.182 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-26T01:42:07.183 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-26T01:42:07.406 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:07.403+0000 7f497c056640 1 -- 172.21.15.169:0/2525817708 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4960005180 msgr2=0x7f496000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:07.407 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:07.403+0000 7f497c056640 1 --2- 172.21.15.169:0/2525817708 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4960005180 0x7f496000b980 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f496c005fd0 tx=0x7f49600057f0 comp rx=0 tx=0).stop 2023-10-26T01:42:07.407 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:07.403+0000 7f497c056640 1 -- 172.21.15.169:0/2525817708 shutdown_connections 2023-10-26T01:42:07.407 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:07.403+0000 7f497c056640 1 --2- 172.21.15.169:0/2525817708 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4960005180 0x7f496000b980 unknown :-1 s=CLOSED pgs=1675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:07.407 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:07.403+0000 7f497c056640 1 -- 172.21.15.169:0/2525817708 >> 172.21.15.169:0/2525817708 conn(0x7f4974069930 msgr2=0x7f4974069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:42:07.408 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:07.404+0000 7f497c056640 1 -- 172.21.15.169:0/2525817708 shutdown_connections 2023-10-26T01:42:07.408 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:07.404+0000 7f497c056640 1 -- 172.21.15.169:0/2525817708 wait complete. 2023-10-26T01:42:07.408 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:07.405+0000 7f497c056640 1 Processor -- start 2023-10-26T01:42:07.409 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:07.405+0000 7f497c056640 1 -- start start 2023-10-26T01:42:07.411 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:07.406+0000 7f497c056640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4960005180 0x7f49740ade60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:07.411 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:07.406+0000 7f497c056640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f497406ad00 0x7f49741f1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:07.411 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:07.406+0000 7f497c056640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f497406b270 0x7f497406b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:07.411 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:07.406+0000 7f497c056640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49741f1b70 con 0x7f497406b270 2023-10-26T01:42:07.412 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:07.406+0000 7f497c056640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f497406bc30 con 0x7f497406ad00 2023-10-26T01:42:07.412 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:07.406+0000 7f497c056640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4974148040 con 0x7f4960005180 2023-10-26T01:42:07.412 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:07.407+0000 7f497a5f3640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f497406b270 0x7f497406b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:42:07.412 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:07.407+0000 7f497a5f3640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f497406b270 0x7f497406b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:56178/0 (socket says 172.21.15.169:56178) 2023-10-26T01:42:07.412 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:07.407+0000 7f497a5f3640 1 -- 172.21.15.169:0/1653612029 learned_addr learned my addr 172.21.15.169:0/1653612029 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:42:07.413 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:07.407+0000 7f49795f1640 1 --2- 172.21.15.169:0/1653612029 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f497406ad00 0x7f49741f1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:42:07.413 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:07.407+0000 7f497a5f3640 1 -- 172.21.15.169:0/1653612029 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4960005180 msgr2=0x7f49740ade60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:07.413 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:07.407+0000 7f4979df2640 1 --2- 172.21.15.169:0/1653612029 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4960005180 0x7f49740ade60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:42:07.413 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:07.407+0000 7f497a5f3640 1 --2- 172.21.15.169:0/1653612029 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4960005180 0x7f49740ade60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:07.414 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:07.407+0000 7f497a5f3640 1 -- 172.21.15.169:0/1653612029 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f497406ad00 msgr2=0x7f49741f1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:07.414 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:07.407+0000 7f497a5f3640 1 --2- 172.21.15.169:0/1653612029 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f497406ad00 0x7f49741f1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:07.414 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:07.407+0000 7f497a5f3640 1 -- 172.21.15.169:0/1653612029 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49740afdb0 con 0x7f497406b270 2023-10-26T01:42:07.414 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:07.407+0000 7f49795f1640 1 --2- 172.21.15.169:0/1653612029 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f497406ad00 0x7f49741f1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:42:07.414 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:07.407+0000 7f497a5f3640 1 --2- 172.21.15.169:0/1653612029 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f497406b270 0x7f497406b6f0 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f496c009f10 tx=0x7f496c033bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:42:07.415 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:07.408+0000 7f496a7fc640 1 -- 172.21.15.169:0/1653612029 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f496c043070 con 0x7f497406b270 2023-10-26T01:42:07.415 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:07.408+0000 7f496a7fc640 1 -- 172.21.15.169:0/1653612029 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f496c004c10 con 0x7f497406b270 2023-10-26T01:42:07.415 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:07.408+0000 7f496a7fc640 1 -- 172.21.15.169:0/1653612029 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f496c005020 con 0x7f497406b270 2023-10-26T01:42:07.415 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:07.408+0000 7f497c056640 1 -- 172.21.15.169:0/1653612029 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4974148290 con 0x7f497406b270 2023-10-26T01:42:07.415 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:07.408+0000 7f497c056640 1 -- 172.21.15.169:0/1653612029 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4974148850 con 0x7f497406b270 2023-10-26T01:42:07.416 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:07.409+0000 7f497c056640 1 -- 172.21.15.169:0/1653612029 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4974066af0 con 0x7f497406b270 2023-10-26T01:42:07.417 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:07.410+0000 7f496a7fc640 1 -- 172.21.15.169:0/1653612029 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f496c03a040 con 0x7f497406b270 2023-10-26T01:42:07.417 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:07.411+0000 7f496a7fc640 1 --2- 172.21.15.169:0/1653612029 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f495806ae70 0x7f495806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:07.417 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:07.411+0000 7f496a7fc640 1 -- 172.21.15.169:0/1653612029 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f496c0aab10 con 0x7f497406b270 2023-10-26T01:42:07.417 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:07.411+0000 7f4979df2640 1 --2- 172.21.15.169:0/1653612029 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f495806ae70 0x7f495806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:42:07.417 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:07.412+0000 7f4979df2640 1 --2- 172.21.15.169:0/1653612029 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f495806ae70 0x7f495806d330 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f4970000f60 tx=0x7f49700014f0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:42:07.418 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:07.415+0000 7f496a7fc640 1 -- 172.21.15.169:0/1653612029 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f496c075f00 con 0x7f497406b270 2023-10-26T01:42:07.556 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T01:42:07.565 DEBUG:teuthology.orchestra.run.smithi170:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T01:42:07.608 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:07.605+0000 7f497c056640 1 -- 172.21.15.169:0/1653612029 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f49740cad20 con 0x7f497406b270 2023-10-26T01:42:08.362 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:08.358+0000 7f496a7fc640 1 -- 172.21.15.169:0/1653612029 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v163) v1 ==== 138+0+0 (secure 0 0 0) 0x7f496c0380b0 con 0x7f497406b270 2023-10-26T01:42:08.362 INFO:teuthology.orchestra.run.smithi169.stderr:pool 'cephfs_metadata' created 2023-10-26T01:42:08.366 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:08.363+0000 7f497c056640 1 -- 172.21.15.169:0/1653612029 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f495806ae70 msgr2=0x7f495806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:08.367 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:08.363+0000 7f497c056640 1 --2- 172.21.15.169:0/1653612029 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f495806ae70 0x7f495806d330 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f4970000f60 tx=0x7f49700014f0 comp rx=0 tx=0).stop 2023-10-26T01:42:08.367 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:08.363+0000 7f497c056640 1 -- 172.21.15.169:0/1653612029 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f497406b270 msgr2=0x7f497406b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:08.367 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:08.363+0000 7f497c056640 1 --2- 172.21.15.169:0/1653612029 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f497406b270 0x7f497406b6f0 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f496c009f10 tx=0x7f496c033bd0 comp rx=0 tx=0).stop 2023-10-26T01:42:08.367 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:08.364+0000 7f497c056640 1 -- 172.21.15.169:0/1653612029 shutdown_connections 2023-10-26T01:42:08.367 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:08.364+0000 7f497c056640 1 --2- 172.21.15.169:0/1653612029 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f495806ae70 0x7f495806d330 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:08.368 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:08.364+0000 7f497c056640 1 --2- 172.21.15.169:0/1653612029 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f497406b270 0x7f497406b6f0 unknown :-1 s=CLOSED pgs=1676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:08.368 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:08.364+0000 7f497c056640 1 --2- 172.21.15.169:0/1653612029 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f497406ad00 0x7f49741f1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:08.368 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:08.364+0000 7f497c056640 1 --2- 172.21.15.169:0/1653612029 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4960005180 0x7f49740ade60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:08.368 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:08.364+0000 7f497c056640 1 -- 172.21.15.169:0/1653612029 >> 172.21.15.169:0/1653612029 conn(0x7f4974069930 msgr2=0x7f497415af70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:42:08.369 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:08.365+0000 7f497c056640 1 -- 172.21.15.169:0/1653612029 shutdown_connections 2023-10-26T01:42:08.369 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:08.365+0000 7f497c056640 1 -- 172.21.15.169:0/1653612029 wait complete. 2023-10-26T01:42:08.392 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-26T01:42:08.612 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:08.609+0000 7f375274c640 1 -- 172.21.15.169:0/3622013258 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37380047c0 msgr2=0x7f3738004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:08.612 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:08.609+0000 7f375274c640 1 --2- 172.21.15.169:0/3622013258 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37380047c0 0x7f3738004c40 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f373400bb20 tx=0x7f37380057f0 comp rx=0 tx=0).stop 2023-10-26T01:42:08.613 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:08.609+0000 7f375274c640 1 -- 172.21.15.169:0/3622013258 shutdown_connections 2023-10-26T01:42:08.613 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:08.609+0000 7f375274c640 1 --2- 172.21.15.169:0/3622013258 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37380047c0 0x7f3738004c40 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:08.613 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:08.609+0000 7f375274c640 1 -- 172.21.15.169:0/3622013258 >> 172.21.15.169:0/3622013258 conn(0x7f374c069930 msgr2=0x7f374c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:42:08.613 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:08.609+0000 7f375274c640 1 -- 172.21.15.169:0/3622013258 shutdown_connections 2023-10-26T01:42:08.613 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:08.609+0000 7f375274c640 1 -- 172.21.15.169:0/3622013258 wait complete. 2023-10-26T01:42:08.614 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:08.611+0000 7f375274c640 1 Processor -- start 2023-10-26T01:42:08.614 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:08.611+0000 7f375274c640 1 -- start start 2023-10-26T01:42:08.616 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:08.613+0000 7f375274c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f37380047c0 0x7f374c14e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:08.616 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:08.613+0000 7f375274c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f374c0c9040 0x7f374c0c75c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:08.617 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:08.613+0000 7f375274c640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f374c0c7b00 0x7f374c0b07c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:08.617 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:08.613+0000 7f375274c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f374c14edc0 con 0x7f374c0c9040 2023-10-26T01:42:08.617 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:08.613+0000 7f375274c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f374c0c7fa0 con 0x7f37380047c0 2023-10-26T01:42:08.617 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:08.613+0000 7f375274c640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f374c0c8280 con 0x7f374c0c7b00 2023-10-26T01:42:08.617 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:08.613+0000 7f3743fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f374c0c9040 0x7f374c0c75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:42:08.618 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:08.613+0000 7f3743fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f374c0c9040 0x7f374c0c75c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:56196/0 (socket says 172.21.15.169:56196) 2023-10-26T01:42:08.618 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:08.613+0000 7f3743fff640 1 -- 172.21.15.169:0/1157399305 learned_addr learned my addr 172.21.15.169:0/1157399305 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:42:08.618 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:08.613+0000 7f374bfff640 1 --2- 172.21.15.169:0/1157399305 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f37380047c0 0x7f374c14e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:42:08.618 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:08.613+0000 7f3743fff640 1 -- 172.21.15.169:0/1157399305 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f374c0c7b00 msgr2=0x7f374c0b07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:08.619 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:08.613+0000 7f3750ce9640 1 --2- 172.21.15.169:0/1157399305 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f374c0c7b00 0x7f374c0b07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:42:08.619 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:08.613+0000 7f3743fff640 1 --2- 172.21.15.169:0/1157399305 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f374c0c7b00 0x7f374c0b07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:08.619 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:08.613+0000 7f3743fff640 1 -- 172.21.15.169:0/1157399305 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f37380047c0 msgr2=0x7f374c14e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:08.619 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:08.613+0000 7f3743fff640 1 --2- 172.21.15.169:0/1157399305 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f37380047c0 0x7f374c14e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:08.619 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:08.613+0000 7f3743fff640 1 -- 172.21.15.169:0/1157399305 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f374c0c8560 con 0x7f374c0c9040 2023-10-26T01:42:08.620 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:08.613+0000 7f374bfff640 1 --2- 172.21.15.169:0/1157399305 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f37380047c0 0x7f374c14e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:42:08.620 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:08.613+0000 7f3750ce9640 1 --2- 172.21.15.169:0/1157399305 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f374c0c7b00 0x7f374c0b07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:42:08.620 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:08.613+0000 7f3743fff640 1 --2- 172.21.15.169:0/1157399305 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f374c0c9040 0x7f374c0c75c0 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f3734038040 tx=0x7f3734036720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:42:08.620 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:08.613+0000 7f37497fa640 1 -- 172.21.15.169:0/1157399305 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3734036d20 con 0x7f374c0c9040 2023-10-26T01:42:08.620 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:08.613+0000 7f37497fa640 1 -- 172.21.15.169:0/1157399305 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3734004030 con 0x7f374c0c9040 2023-10-26T01:42:08.621 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:08.613+0000 7f37497fa640 1 -- 172.21.15.169:0/1157399305 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3734004420 con 0x7f374c0c9040 2023-10-26T01:42:08.621 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:08.614+0000 7f375274c640 1 -- 172.21.15.169:0/1157399305 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f374c0c8840 con 0x7f374c0c9040 2023-10-26T01:42:08.621 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:08.614+0000 7f375274c640 1 -- 172.21.15.169:0/1157399305 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f374c0c94c0 con 0x7f374c0c9040 2023-10-26T01:42:08.622 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:08.615+0000 7f375274c640 1 -- 172.21.15.169:0/1157399305 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f374c04db10 con 0x7f374c0c9040 2023-10-26T01:42:08.622 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:08.616+0000 7f37497fa640 1 -- 172.21.15.169:0/1157399305 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f37340035f0 con 0x7f374c0c9040 2023-10-26T01:42:08.622 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:08.617+0000 7f37497fa640 1 --2- 172.21.15.169:0/1157399305 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f372c06ae70 0x7f372c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:08.622 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:08.617+0000 7f37497fa640 1 -- 172.21.15.169:0/1157399305 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f37340ab680 con 0x7f374c0c9040 2023-10-26T01:42:08.623 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:08.618+0000 7f374bfff640 1 --2- 172.21.15.169:0/1157399305 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f372c06ae70 0x7f372c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:42:08.623 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:08.619+0000 7f374bfff640 1 --2- 172.21.15.169:0/1157399305 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f372c06ae70 0x7f372c06d330 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f374c1f5480 tx=0x7f373c000ed0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:42:08.623 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:08.621+0000 7f37497fa640 1 -- 172.21.15.169:0/1157399305 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f37340768f0 con 0x7f374c0c9040 2023-10-26T01:42:08.812 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:08.809+0000 7f375274c640 1 -- 172.21.15.169:0/1157399305 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f374c0b2410 con 0x7f374c0c9040 2023-10-26T01:42:09.378 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:09.376+0000 7f37497fa640 1 -- 172.21.15.169:0/1157399305 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v164) v1 ==== 170+0+0 (secure 0 0 0) 0x7f37340480d0 con 0x7f374c0c9040 2023-10-26T01:42:09.379 INFO:teuthology.orchestra.run.smithi169.stderr:pool 'cephfs_data' created 2023-10-26T01:42:09.382 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:09.380+0000 7f375274c640 1 -- 172.21.15.169:0/1157399305 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f372c06ae70 msgr2=0x7f372c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:09.382 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:09.380+0000 7f375274c640 1 --2- 172.21.15.169:0/1157399305 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f372c06ae70 0x7f372c06d330 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f374c1f5480 tx=0x7f373c000ed0 comp rx=0 tx=0).stop 2023-10-26T01:42:09.382 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:09.380+0000 7f375274c640 1 -- 172.21.15.169:0/1157399305 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f374c0c9040 msgr2=0x7f374c0c75c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:09.382 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:09.380+0000 7f375274c640 1 --2- 172.21.15.169:0/1157399305 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f374c0c9040 0x7f374c0c75c0 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f3734038040 tx=0x7f3734036720 comp rx=0 tx=0).stop 2023-10-26T01:42:09.383 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:09.380+0000 7f375274c640 1 -- 172.21.15.169:0/1157399305 shutdown_connections 2023-10-26T01:42:09.383 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:09.381+0000 7f375274c640 1 --2- 172.21.15.169:0/1157399305 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f372c06ae70 0x7f372c06d330 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:09.384 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:09.381+0000 7f375274c640 1 --2- 172.21.15.169:0/1157399305 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f374c0c9040 0x7f374c0c75c0 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:09.384 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:09.381+0000 7f375274c640 1 --2- 172.21.15.169:0/1157399305 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f374c0c7b00 0x7f374c0b07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:09.384 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:09.381+0000 7f375274c640 1 --2- 172.21.15.169:0/1157399305 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f37380047c0 0x7f374c14e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:09.384 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:09.381+0000 7f375274c640 1 -- 172.21.15.169:0/1157399305 >> 172.21.15.169:0/1157399305 conn(0x7f374c069930 msgr2=0x7f374c069d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:42:09.385 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:09.381+0000 7f375274c640 1 -- 172.21.15.169:0/1157399305 shutdown_connections 2023-10-26T01:42:09.385 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:09.381+0000 7f375274c640 1 -- 172.21.15.169:0/1157399305 wait complete. 2023-10-26T01:42:09.414 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-26T01:42:09.613 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:09.611+0000 7fae28c4b640 1 -- 172.21.15.169:0/2222654909 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fae10004d00 msgr2=0x7fae10005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:09.613 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:09.611+0000 7fae28c4b640 1 --2- 172.21.15.169:0/2222654909 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fae10004d00 0x7fae10005180 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7fae1c00c610 tx=0x7fae100057f0 comp rx=0 tx=0).stop 2023-10-26T01:42:09.614 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:09.611+0000 7fae22d9d640 1 -- 172.21.15.169:0/2222654909 reap_dead start 2023-10-26T01:42:09.614 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:09.611+0000 7fae28c4b640 1 -- 172.21.15.169:0/2222654909 shutdown_connections 2023-10-26T01:42:09.614 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:09.611+0000 7fae28c4b640 1 -- 172.21.15.169:0/2222654909 >> 172.21.15.169:0/2222654909 conn(0x7fae24069930 msgr2=0x7fae24069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:42:09.614 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:09.611+0000 7fae28c4b640 1 -- 172.21.15.169:0/2222654909 shutdown_connections 2023-10-26T01:42:09.615 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:09.611+0000 7fae28c4b640 1 -- 172.21.15.169:0/2222654909 wait complete. 2023-10-26T01:42:09.617 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:09.613+0000 7fae28c4b640 1 Processor -- start 2023-10-26T01:42:09.617 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:09.613+0000 7fae28c4b640 1 -- start start 2023-10-26T01:42:09.618 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:09.614+0000 7fae28c4b640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fae10004d00 0x7fae2415a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:09.618 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:09.614+0000 7fae28c4b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fae24070790 0x7fae24157ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:09.618 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:09.614+0000 7fae28c4b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fae2414cbd0 0x7fae24158020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:09.618 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:09.614+0000 7fae28c4b640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae24071d30 con 0x7fae2414cbd0 2023-10-26T01:42:09.619 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:09.614+0000 7fae28c4b640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae24070c90 con 0x7fae24070790 2023-10-26T01:42:09.619 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:09.614+0000 7fae28c4b640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae24070f70 con 0x7fae10004d00 2023-10-26T01:42:09.619 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:09.614+0000 7fae22d9d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fae2414cbd0 0x7fae24158020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:42:09.619 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:09.614+0000 7fae22d9d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fae2414cbd0 0x7fae24158020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:56210/0 (socket says 172.21.15.169:56210) 2023-10-26T01:42:09.619 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:09.614+0000 7fae22d9d640 1 -- 172.21.15.169:0/4056340820 learned_addr learned my addr 172.21.15.169:0/4056340820 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:42:09.620 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:09.615+0000 7fae1bfff640 1 --2- 172.21.15.169:0/4056340820 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fae24070790 0x7fae24157ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:42:09.620 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:09.615+0000 7fae2259c640 1 --2- 172.21.15.169:0/4056340820 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fae10004d00 0x7fae2415a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:42:09.620 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:09.615+0000 7fae22d9d640 1 -- 172.21.15.169:0/4056340820 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fae10004d00 msgr2=0x7fae2415a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:09.620 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:09.615+0000 7fae22d9d640 1 --2- 172.21.15.169:0/4056340820 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fae10004d00 0x7fae2415a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:09.620 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:09.615+0000 7fae22d9d640 1 -- 172.21.15.169:0/4056340820 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fae24070790 msgr2=0x7fae24157ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:09.621 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:09.615+0000 7fae22d9d640 1 --2- 172.21.15.169:0/4056340820 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fae24070790 0x7fae24157ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:09.621 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:09.615+0000 7fae22d9d640 1 -- 172.21.15.169:0/4056340820 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae24071250 con 0x7fae2414cbd0 2023-10-26T01:42:09.621 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:09.615+0000 7fae22d9d640 1 --2- 172.21.15.169:0/4056340820 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fae2414cbd0 0x7fae24158020 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7fae1c000f80 tx=0x7fae1c002070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:42:09.621 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:09.615+0000 7fae1affd640 1 -- 172.21.15.169:0/4056340820 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae1c0018f0 con 0x7fae2414cbd0 2023-10-26T01:42:09.622 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:09.616+0000 7fae1affd640 1 -- 172.21.15.169:0/4056340820 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fae1c003970 con 0x7fae2414cbd0 2023-10-26T01:42:09.622 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:09.616+0000 7fae1affd640 1 -- 172.21.15.169:0/4056340820 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae1c003d40 con 0x7fae2414cbd0 2023-10-26T01:42:09.622 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:09.616+0000 7fae28c4b640 1 -- 172.21.15.169:0/4056340820 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae24071530 con 0x7fae2414cbd0 2023-10-26T01:42:09.622 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:09.616+0000 7fae28c4b640 1 -- 172.21.15.169:0/4056340820 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae240ca1c0 con 0x7fae2414cbd0 2023-10-26T01:42:09.623 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:09.617+0000 7fae28c4b640 1 -- 172.21.15.169:0/4056340820 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae24071a50 con 0x7fae2414cbd0 2023-10-26T01:42:09.623 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:09.618+0000 7fae1affd640 1 -- 172.21.15.169:0/4056340820 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fae1c031030 con 0x7fae2414cbd0 2023-10-26T01:42:09.624 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:09.619+0000 7fae1affd640 1 --2- 172.21.15.169:0/4056340820 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fae0406ae70 0x7fae0406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:09.624 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:09.619+0000 7fae1affd640 1 -- 172.21.15.169:0/4056340820 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fae1c0aa450 con 0x7fae2414cbd0 2023-10-26T01:42:09.624 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:09.620+0000 7fae2259c640 1 --2- 172.21.15.169:0/4056340820 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fae0406ae70 0x7fae0406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:42:09.624 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:09.621+0000 7fae2259c640 1 --2- 172.21.15.169:0/4056340820 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fae0406ae70 0x7fae0406d330 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fae14009a60 tx=0x7fae14005f90 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:42:09.625 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:09.624+0000 7fae1affd640 1 -- 172.21.15.169:0/4056340820 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fae24071a50 con 0x7fae2414cbd0 2023-10-26T01:42:09.853 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:09.851+0000 7fae28c4b640 1 -- 172.21.15.169:0/4056340820 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fae24071a50 con 0x7fae2414cbd0 2023-10-26T01:42:10.382 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:10.380+0000 7fae1affd640 1 -- 172.21.15.169:0/4056340820 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 20 and data pool 21 v82) v1 ==== 174+0+0 (secure 0 0 0) 0x7fae24071a50 con 0x7fae2414cbd0 2023-10-26T01:42:10.382 INFO:teuthology.orchestra.run.smithi169.stderr:new fs with metadata pool 20 and data pool 21 2023-10-26T01:42:10.386 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:10.384+0000 7fae28c4b640 1 -- 172.21.15.169:0/4056340820 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fae0406ae70 msgr2=0x7fae0406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:10.387 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:10.384+0000 7fae28c4b640 1 --2- 172.21.15.169:0/4056340820 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fae0406ae70 0x7fae0406d330 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fae14009a60 tx=0x7fae14005f90 comp rx=0 tx=0).stop 2023-10-26T01:42:10.387 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:10.384+0000 7fae28c4b640 1 -- 172.21.15.169:0/4056340820 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fae2414cbd0 msgr2=0x7fae24158020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:10.387 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:10.384+0000 7fae28c4b640 1 --2- 172.21.15.169:0/4056340820 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fae2414cbd0 0x7fae24158020 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7fae1c000f80 tx=0x7fae1c002070 comp rx=0 tx=0).stop 2023-10-26T01:42:10.387 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:10.384+0000 7fae28c4b640 1 -- 172.21.15.169:0/4056340820 shutdown_connections 2023-10-26T01:42:10.387 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:10.384+0000 7fae28c4b640 1 --2- 172.21.15.169:0/4056340820 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fae0406ae70 0x7fae0406d330 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:10.387 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:10.384+0000 7fae28c4b640 1 --2- 172.21.15.169:0/4056340820 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fae2414cbd0 0x7fae24158020 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:10.388 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:10.384+0000 7fae28c4b640 1 --2- 172.21.15.169:0/4056340820 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fae24070790 0x7fae24157ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:10.388 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:10.384+0000 7fae28c4b640 1 --2- 172.21.15.169:0/4056340820 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fae10004d00 0x7fae2415a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:10.388 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:10.384+0000 7fae28c4b640 1 -- 172.21.15.169:0/4056340820 >> 172.21.15.169:0/4056340820 conn(0x7fae24069930 msgr2=0x7fae240cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:42:10.388 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:10.384+0000 7fae28c4b640 1 -- 172.21.15.169:0/4056340820 shutdown_connections 2023-10-26T01:42:10.388 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:10.385+0000 7fae28c4b640 1 -- 172.21.15.169:0/4056340820 wait complete. 2023-10-26T01:42:10.419 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:42:10.609 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:10.607+0000 7fb53390f640 1 -- 172.21.15.169:0/494531847 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb52c15f610 msgr2=0x7fb52c0ce630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:10.609 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:10.607+0000 7fb53390f640 1 --2- 172.21.15.169:0/494531847 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb52c15f610 0x7fb52c0ce630 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7fb51c005fd0 tx=0x7fb51c02be20 comp rx=0 tx=0).stop 2023-10-26T01:42:10.609 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:10.607+0000 7fb53390f640 1 -- 172.21.15.169:0/494531847 shutdown_connections 2023-10-26T01:42:10.610 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:10.607+0000 7fb53390f640 1 --2- 172.21.15.169:0/494531847 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb52c15f610 0x7fb52c0ce630 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:10.610 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:10.607+0000 7fb53390f640 1 -- 172.21.15.169:0/494531847 >> 172.21.15.169:0/494531847 conn(0x7fb52c069e00 msgr2=0x7fb52c06a210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:42:10.610 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:10.607+0000 7fb53390f640 1 -- 172.21.15.169:0/494531847 shutdown_connections 2023-10-26T01:42:10.610 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:10.607+0000 7fb53390f640 1 -- 172.21.15.169:0/494531847 wait complete. 2023-10-26T01:42:10.611 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:10.609+0000 7fb53390f640 1 Processor -- start 2023-10-26T01:42:10.611 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:10.609+0000 7fb53390f640 1 -- start start 2023-10-26T01:42:10.613 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:10.610+0000 7fb53390f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb52c0ceb70 0x7fb52c0b1120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:10.613 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:10.610+0000 7fb53390f640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb52c15b9d0 0x7fb52c155520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:10.614 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:10.610+0000 7fb53390f640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb52c15f610 0x7fb52c0ca960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:10.614 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:10.610+0000 7fb53390f640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb52c0b2320 con 0x7fb52c15f610 2023-10-26T01:42:10.614 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:10.610+0000 7fb53390f640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb52c0b1970 con 0x7fb52c0ceb70 2023-10-26T01:42:10.614 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:10.610+0000 7fb53390f640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb52c0b1c50 con 0x7fb52c15b9d0 2023-10-26T01:42:10.614 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:10.610+0000 7fb531eac640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb52c15f610 0x7fb52c0ca960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:42:10.615 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:10.610+0000 7fb531eac640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb52c15f610 0x7fb52c0ca960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:56230/0 (socket says 172.21.15.169:56230) 2023-10-26T01:42:10.615 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:10.610+0000 7fb531eac640 1 -- 172.21.15.169:0/1643406297 learned_addr learned my addr 172.21.15.169:0/1643406297 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:42:10.615 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:10.611+0000 7fb5316ab640 1 --2- 172.21.15.169:0/1643406297 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb52c0ceb70 0x7fb52c0b1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:42:10.615 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:10.611+0000 7fb531eac640 1 -- 172.21.15.169:0/1643406297 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb52c15b9d0 msgr2=0x7fb52c155520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:10.615 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:10.611+0000 7fb531eac640 1 --2- 172.21.15.169:0/1643406297 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb52c15b9d0 0x7fb52c155520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:10.616 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:10.611+0000 7fb531eac640 1 -- 172.21.15.169:0/1643406297 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb52c0ceb70 msgr2=0x7fb52c0b1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:10.616 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:10.611+0000 7fb531eac640 1 --2- 172.21.15.169:0/1643406297 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb52c0ceb70 0x7fb52c0b1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:10.616 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:10.611+0000 7fb531eac640 1 -- 172.21.15.169:0/1643406297 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb51c009560 con 0x7fb52c15f610 2023-10-26T01:42:10.616 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:10.611+0000 7fb5316ab640 1 --2- 172.21.15.169:0/1643406297 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb52c0ceb70 0x7fb52c0b1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:42:10.616 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:10.611+0000 7fb531eac640 1 --2- 172.21.15.169:0/1643406297 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb52c15f610 0x7fb52c0ca960 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7fb52400c780 tx=0x7fb52400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:42:10.616 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:10.611+0000 7fb521ffb640 1 -- 172.21.15.169:0/1643406297 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb524003d80 con 0x7fb52c15f610 2023-10-26T01:42:10.617 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:10.611+0000 7fb521ffb640 1 -- 172.21.15.169:0/1643406297 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb52400ce90 con 0x7fb52c15f610 2023-10-26T01:42:10.617 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:10.611+0000 7fb53390f640 1 -- 172.21.15.169:0/1643406297 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb52c0b1f30 con 0x7fb52c15f610 2023-10-26T01:42:10.617 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:10.611+0000 7fb521ffb640 1 -- 172.21.15.169:0/1643406297 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5240052d0 con 0x7fb52c15f610 2023-10-26T01:42:10.617 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:10.611+0000 7fb53390f640 1 -- 172.21.15.169:0/1643406297 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb52c0b1660 con 0x7fb52c15f610 2023-10-26T01:42:10.618 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:10.612+0000 7fb53390f640 1 -- 172.21.15.169:0/1643406297 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb52c0b27b0 con 0x7fb52c15f610 2023-10-26T01:42:10.618 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:10.613+0000 7fb521ffb640 1 -- 172.21.15.169:0/1643406297 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fb524005430 con 0x7fb52c15f610 2023-10-26T01:42:10.619 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:10.614+0000 7fb521ffb640 1 --2- 172.21.15.169:0/1643406297 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb51006ae40 0x7fb51006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:10.619 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:10.614+0000 7fb521ffb640 1 -- 172.21.15.169:0/1643406297 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb5240891c0 con 0x7fb52c15f610 2023-10-26T01:42:10.619 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:10.615+0000 7fb5316ab640 1 --2- 172.21.15.169:0/1643406297 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb51006ae40 0x7fb51006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:42:10.619 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:10.616+0000 7fb5316ab640 1 --2- 172.21.15.169:0/1643406297 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb51006ae40 0x7fb51006d300 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fb528000f60 tx=0x7fb528009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:42:10.620 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:10.619+0000 7fb521ffb640 1 -- 172.21.15.169:0/1643406297 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb52c0b27b0 con 0x7fb52c15f610 2023-10-26T01:42:10.811 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:10.809+0000 7fb53390f640 1 -- 172.21.15.169:0/1643406297 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb52c0b27b0 con 0x7fb52c15f610 2023-10-26T01:42:10.814 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:10.812+0000 7fb521ffb640 1 -- 172.21.15.169:0/1643406297 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v165) v1 ==== 74+0+16641 (secure 0 0 0) 0x7fb52c0b27b0 con 0x7fb52c15f610 2023-10-26T01:42:10.814 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:42:10.815 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":165,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:42:09.853315+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2023-10-26T01:42:07.656531+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"165","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2023-10-26T01:42:08.866635+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"165","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:42:10.817 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:10.815+0000 7fb53390f640 1 -- 172.21.15.169:0/1643406297 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb51006ae40 msgr2=0x7fb51006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:10.817 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:10.815+0000 7fb53390f640 1 --2- 172.21.15.169:0/1643406297 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb51006ae40 0x7fb51006d300 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fb528000f60 tx=0x7fb528009040 comp rx=0 tx=0).stop 2023-10-26T01:42:10.817 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:10.815+0000 7fb53390f640 1 -- 172.21.15.169:0/1643406297 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb52c15f610 msgr2=0x7fb52c0ca960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:10.818 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:10.815+0000 7fb53390f640 1 --2- 172.21.15.169:0/1643406297 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb52c15f610 0x7fb52c0ca960 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7fb52400c780 tx=0x7fb52400cc40 comp rx=0 tx=0).stop 2023-10-26T01:42:10.818 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:10.815+0000 7fb53390f640 1 -- 172.21.15.169:0/1643406297 shutdown_connections 2023-10-26T01:42:10.818 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:10.815+0000 7fb53390f640 1 --2- 172.21.15.169:0/1643406297 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb51006ae40 0x7fb51006d300 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:10.818 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:10.815+0000 7fb53390f640 1 --2- 172.21.15.169:0/1643406297 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb52c15f610 0x7fb52c0ca960 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:10.819 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:10.815+0000 7fb53390f640 1 --2- 172.21.15.169:0/1643406297 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb52c15b9d0 0x7fb52c155520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:10.819 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:10.815+0000 7fb53390f640 1 --2- 172.21.15.169:0/1643406297 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb52c0ceb70 0x7fb52c0b1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:10.819 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:10.815+0000 7fb53390f640 1 -- 172.21.15.169:0/1643406297 >> 172.21.15.169:0/1643406297 conn(0x7fb52c069e00 msgr2=0x7fb52c072630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:42:10.819 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:10.815+0000 7fb53390f640 1 -- 172.21.15.169:0/1643406297 shutdown_connections 2023-10-26T01:42:10.819 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:10.815+0000 7fb53390f640 1 -- 172.21.15.169:0/1643406297 wait complete. 2023-10-26T01:42:10.835 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:42:11.046 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:11.044+0000 7fee9759e640 1 -- 172.21.15.169:0/4014488770 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fee84003970 msgr2=0x7fee84003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:11.046 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:11.044+0000 7fee9759e640 1 --2- 172.21.15.169:0/4014488770 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fee84003970 0x7fee84003d90 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7fee8800c260 tx=0x7fee840057f0 comp rx=0 tx=0).stop 2023-10-26T01:42:11.046 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:11.044+0000 7fee96d9d640 1 -- 172.21.15.169:0/4014488770 reap_dead start 2023-10-26T01:42:11.047 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:11.044+0000 7fee9759e640 1 -- 172.21.15.169:0/4014488770 shutdown_connections 2023-10-26T01:42:11.047 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:11.044+0000 7fee9759e640 1 -- 172.21.15.169:0/4014488770 >> 172.21.15.169:0/4014488770 conn(0x7fee980aedd0 msgr2=0x7fee980af1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:42:11.047 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:11.044+0000 7fee9759e640 1 -- 172.21.15.169:0/4014488770 shutdown_connections 2023-10-26T01:42:11.047 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:11.044+0000 7fee9759e640 1 -- 172.21.15.169:0/4014488770 wait complete. 2023-10-26T01:42:11.050 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:11.046+0000 7fee9759e640 1 Processor -- start 2023-10-26T01:42:11.050 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:11.046+0000 7fee9759e640 1 -- start start 2023-10-26T01:42:11.050 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:11.047+0000 7fee9759e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fee84003970 0x7fee981ef850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:11.050 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:11.047+0000 7fee9759e640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fee9814d470 0x7fee98151070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:11.050 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:11.047+0000 7fee9759e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fee9814cd00 0x7fee980b1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:11.051 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:11.047+0000 7fee9759e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee981efd90 con 0x7fee9814cd00 2023-10-26T01:42:11.051 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:11.047+0000 7fee9759e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee98070230 con 0x7fee84003970 2023-10-26T01:42:11.051 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:11.047+0000 7fee9759e640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee980707f0 con 0x7fee9814d470 2023-10-26T01:42:11.051 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:11.047+0000 7fee96d9d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fee9814cd00 0x7fee980b1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:42:11.051 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:11.047+0000 7fee96d9d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fee9814cd00 0x7fee980b1cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:56240/0 (socket says 172.21.15.169:56240) 2023-10-26T01:42:11.052 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:11.047+0000 7fee96d9d640 1 -- 172.21.15.169:0/2560947477 learned_addr learned my addr 172.21.15.169:0/2560947477 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:42:11.052 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:11.047+0000 7fee9659c640 1 --2- 172.21.15.169:0/2560947477 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fee84003970 0x7fee981ef850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:42:11.052 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:11.047+0000 7fee8ffff640 1 --2- 172.21.15.169:0/2560947477 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fee9814d470 0x7fee98151070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:42:11.052 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:11.047+0000 7fee96d9d640 1 -- 172.21.15.169:0/2560947477 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fee9814d470 msgr2=0x7fee98151070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:11.052 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:11.047+0000 7fee96d9d640 1 --2- 172.21.15.169:0/2560947477 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fee9814d470 0x7fee98151070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:11.052 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:11.047+0000 7fee96d9d640 1 -- 172.21.15.169:0/2560947477 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fee84003970 msgr2=0x7fee981ef850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:11.053 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:11.047+0000 7fee96d9d640 1 --2- 172.21.15.169:0/2560947477 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fee84003970 0x7fee981ef850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:11.053 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:11.047+0000 7fee96d9d640 1 -- 172.21.15.169:0/2560947477 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee9814dce0 con 0x7fee9814cd00 2023-10-26T01:42:11.053 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:11.047+0000 7fee9659c640 1 --2- 172.21.15.169:0/2560947477 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fee84003970 0x7fee981ef850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:42:11.053 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:11.048+0000 7fee96d9d640 1 --2- 172.21.15.169:0/2560947477 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fee9814cd00 0x7fee980b1cf0 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7fee980ac990 tx=0x7fee9000b360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:42:11.053 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:11.048+0000 7fee8effd640 1 -- 172.21.15.169:0/2560947477 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee9000beb0 con 0x7fee9814cd00 2023-10-26T01:42:11.053 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:11.048+0000 7fee8effd640 1 -- 172.21.15.169:0/2560947477 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fee90004c30 con 0x7fee9814cd00 2023-10-26T01:42:11.054 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:11.048+0000 7fee8effd640 1 -- 172.21.15.169:0/2560947477 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee90005020 con 0x7fee9814cd00 2023-10-26T01:42:11.054 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:11.048+0000 7fee9759e640 1 -- 172.21.15.169:0/2560947477 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee981515b0 con 0x7fee9814cd00 2023-10-26T01:42:11.054 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:11.048+0000 7fee9759e640 1 -- 172.21.15.169:0/2560947477 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee9806f680 con 0x7fee9814cd00 2023-10-26T01:42:11.055 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:11.049+0000 7fee9759e640 1 -- 172.21.15.169:0/2560947477 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee98073c10 con 0x7fee9814cd00 2023-10-26T01:42:11.055 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:11.050+0000 7fee8effd640 1 -- 172.21.15.169:0/2560947477 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fee90005180 con 0x7fee9814cd00 2023-10-26T01:42:11.055 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:11.051+0000 7fee8effd640 1 --2- 172.21.15.169:0/2560947477 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fee7806ae70 0x7fee7806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:11.056 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:11.052+0000 7fee8effd640 1 -- 172.21.15.169:0/2560947477 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fee90089fd0 con 0x7fee9814cd00 2023-10-26T01:42:11.056 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:11.052+0000 7fee9659c640 1 --2- 172.21.15.169:0/2560947477 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fee7806ae70 0x7fee7806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:42:11.056 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:11.053+0000 7fee9659c640 1 --2- 172.21.15.169:0/2560947477 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fee7806ae70 0x7fee7806d330 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fee88034000 tx=0x7fee88001a70 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:42:11.057 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:11.056+0000 7fee8effd640 1 -- 172.21.15.169:0/2560947477 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fee90055070 con 0x7fee9814cd00 2023-10-26T01:42:11.246 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:11.245+0000 7fee9759e640 1 -- 172.21.15.169:0/2560947477 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fee98070930 con 0x7fee9814cd00 2023-10-26T01:42:11.249 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:11.248+0000 7fee8effd640 1 -- 172.21.15.169:0/2560947477 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v165) v1 ==== 74+0+16641 (secure 0 0 0) 0x7fee90058690 con 0x7fee9814cd00 2023-10-26T01:42:11.250 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:42:11.250 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":165,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:42:09.853315+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2023-10-26T01:42:07.656531+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"165","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2023-10-26T01:42:08.866635+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"165","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:42:11.252 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:11.250+0000 7fee9759e640 1 -- 172.21.15.169:0/2560947477 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fee7806ae70 msgr2=0x7fee7806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:11.253 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:11.250+0000 7fee9759e640 1 --2- 172.21.15.169:0/2560947477 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fee7806ae70 0x7fee7806d330 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fee88034000 tx=0x7fee88001a70 comp rx=0 tx=0).stop 2023-10-26T01:42:11.253 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:11.250+0000 7fee9759e640 1 -- 172.21.15.169:0/2560947477 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fee9814cd00 msgr2=0x7fee980b1cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:11.253 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:11.250+0000 7fee9759e640 1 --2- 172.21.15.169:0/2560947477 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fee9814cd00 0x7fee980b1cf0 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7fee980ac990 tx=0x7fee9000b360 comp rx=0 tx=0).stop 2023-10-26T01:42:11.253 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:11.251+0000 7fee9759e640 1 -- 172.21.15.169:0/2560947477 shutdown_connections 2023-10-26T01:42:11.254 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:11.251+0000 7fee9759e640 1 --2- 172.21.15.169:0/2560947477 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fee7806ae70 0x7fee7806d330 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:11.254 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:11.251+0000 7fee9759e640 1 --2- 172.21.15.169:0/2560947477 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fee9814cd00 0x7fee980b1cf0 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:11.254 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:11.251+0000 7fee9759e640 1 --2- 172.21.15.169:0/2560947477 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fee9814d470 0x7fee98151070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:11.254 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:11.251+0000 7fee9759e640 1 --2- 172.21.15.169:0/2560947477 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fee84003970 0x7fee981ef850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:11.254 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:11.251+0000 7fee9759e640 1 -- 172.21.15.169:0/2560947477 >> 172.21.15.169:0/2560947477 conn(0x7fee980aedd0 msgr2=0x7fee980aced0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:42:11.254 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:11.251+0000 7fee9759e640 1 -- 172.21.15.169:0/2560947477 shutdown_connections 2023-10-26T01:42:11.255 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:11.251+0000 7fee9759e640 1 -- 172.21.15.169:0/2560947477 wait complete. 2023-10-26T01:42:11.271 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-26T01:42:11.490 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:11.488+0000 7fd40f81a640 1 -- 172.21.15.169:0/968123514 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd3f4005180 msgr2=0x7fd3f400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:11.491 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:11.488+0000 7fd40f81a640 1 --2- 172.21.15.169:0/968123514 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd3f4005180 0x7fd3f400b980 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7fd4000043e0 tx=0x7fd3f40057f0 comp rx=0 tx=0).stop 2023-10-26T01:42:11.491 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:11.488+0000 7fd40f81a640 1 -- 172.21.15.169:0/968123514 shutdown_connections 2023-10-26T01:42:11.491 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:11.488+0000 7fd40f81a640 1 --2- 172.21.15.169:0/968123514 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd3f4005180 0x7fd3f400b980 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:11.491 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:11.488+0000 7fd40f81a640 1 -- 172.21.15.169:0/968123514 >> 172.21.15.169:0/968123514 conn(0x7fd408069930 msgr2=0x7fd408069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:42:11.492 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:11.488+0000 7fd40f81a640 1 -- 172.21.15.169:0/968123514 shutdown_connections 2023-10-26T01:42:11.492 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:11.489+0000 7fd40f81a640 1 -- 172.21.15.169:0/968123514 wait complete. 2023-10-26T01:42:11.492 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:11.490+0000 7fd40f81a640 1 Processor -- start 2023-10-26T01:42:11.492 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:11.490+0000 7fd40f81a640 1 -- start start 2023-10-26T01:42:11.494 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:11.491+0000 7fd40f81a640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd3f4005180 0x7fd4080c6660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:11.494 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:11.491+0000 7fd40f81a640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd408147cf0 0x7fd40814f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:11.494 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:11.491+0000 7fd40f81a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd40814ff10 0x7fd40806ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:11.494 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:11.491+0000 7fd40f81a640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd408150390 con 0x7fd3f4005180 2023-10-26T01:42:11.494 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:11.491+0000 7fd40f81a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd40806fa20 con 0x7fd40814ff10 2023-10-26T01:42:11.495 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:11.491+0000 7fd40f81a640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd40806fd00 con 0x7fd408147cf0 2023-10-26T01:42:11.495 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:11.492+0000 7fd40d5b6640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd3f4005180 0x7fd4080c6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:42:11.495 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:11.492+0000 7fd40d5b6640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd3f4005180 0x7fd4080c6660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:56260/0 (socket says 172.21.15.169:56260) 2023-10-26T01:42:11.495 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:11.492+0000 7fd40d5b6640 1 -- 172.21.15.169:0/926253736 learned_addr learned my addr 172.21.15.169:0/926253736 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:42:11.495 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:11.492+0000 7fd40d5b6640 1 -- 172.21.15.169:0/926253736 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd408147cf0 msgr2=0x7fd40814f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:11.496 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:11.492+0000 7fd40d5b6640 1 --2- 172.21.15.169:0/926253736 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd408147cf0 0x7fd40814f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:11.496 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:11.492+0000 7fd40d5b6640 1 -- 172.21.15.169:0/926253736 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd40814ff10 msgr2=0x7fd40806ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:11.496 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:11.492+0000 7fd40d5b6640 1 --2- 172.21.15.169:0/926253736 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd40814ff10 0x7fd40806ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:11.496 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:11.492+0000 7fd40d5b6640 1 -- 172.21.15.169:0/926253736 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd408070770 con 0x7fd3f4005180 2023-10-26T01:42:11.496 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:11.492+0000 7fd40d5b6640 1 --2- 172.21.15.169:0/926253736 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd3f4005180 0x7fd4080c6660 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7fd40400e570 tx=0x7fd40400ea30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:42:11.497 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:11.492+0000 7fd3fdffb640 1 -- 172.21.15.169:0/926253736 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd40400c8f0 con 0x7fd3f4005180 2023-10-26T01:42:11.497 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:11.493+0000 7fd3fdffb640 1 -- 172.21.15.169:0/926253736 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd40400cb40 con 0x7fd3f4005180 2023-10-26T01:42:11.498 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:11.493+0000 7fd3fdffb640 1 -- 172.21.15.169:0/926253736 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd404011460 con 0x7fd3f4005180 2023-10-26T01:42:11.498 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:11.493+0000 7fd40f81a640 1 -- 172.21.15.169:0/926253736 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd408070a50 con 0x7fd3f4005180 2023-10-26T01:42:11.498 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:11.493+0000 7fd40f81a640 1 -- 172.21.15.169:0/926253736 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd40806f270 con 0x7fd3f4005180 2023-10-26T01:42:11.498 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:11.494+0000 7fd40f81a640 1 -- 172.21.15.169:0/926253736 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4080705d0 con 0x7fd3f4005180 2023-10-26T01:42:11.498 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:11.495+0000 7fd3fdffb640 1 -- 172.21.15.169:0/926253736 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fd4040115c0 con 0x7fd3f4005180 2023-10-26T01:42:11.499 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:11.496+0000 7fd3fdffb640 1 --2- 172.21.15.169:0/926253736 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd3ec06ae70 0x7fd3ec06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:11.499 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:11.496+0000 7fd3fdffb640 1 -- 172.21.15.169:0/926253736 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd404089ed0 con 0x7fd3f4005180 2023-10-26T01:42:11.500 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:11.497+0000 7fd40cdb5640 1 --2- 172.21.15.169:0/926253736 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd3ec06ae70 0x7fd3ec06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:42:11.500 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:11.498+0000 7fd40cdb5640 1 --2- 172.21.15.169:0/926253736 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd3ec06ae70 0x7fd3ec06d330 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fd4080c9260 tx=0x7fd3f8005740 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:42:11.502 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:11.501+0000 7fd3fdffb640 1 -- 172.21.15.169:0/926253736 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd4080705d0 con 0x7fd3f4005180 2023-10-26T01:42:11.733 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:11.731+0000 7fd40f81a640 1 -- 172.21.15.169:0/926253736 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fd408070c90 con 0x7fd3f4005180 2023-10-26T01:42:12.423 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:12.421+0000 7fd3fdffb640 1 -- 172.21.15.169:0/926253736 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v85) v1 ==== 118+0+0 (secure 0 0 0) 0x7fd408070c90 con 0x7fd3f4005180 2023-10-26T01:42:12.427 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:12.425+0000 7fd40f81a640 1 -- 172.21.15.169:0/926253736 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd3ec06ae70 msgr2=0x7fd3ec06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:12.427 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:12.425+0000 7fd40f81a640 1 --2- 172.21.15.169:0/926253736 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd3ec06ae70 0x7fd3ec06d330 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fd4080c9260 tx=0x7fd3f8005740 comp rx=0 tx=0).stop 2023-10-26T01:42:12.428 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:12.425+0000 7fd40f81a640 1 -- 172.21.15.169:0/926253736 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd3f4005180 msgr2=0x7fd4080c6660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:12.428 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:12.425+0000 7fd40f81a640 1 --2- 172.21.15.169:0/926253736 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd3f4005180 0x7fd4080c6660 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7fd40400e570 tx=0x7fd40400ea30 comp rx=0 tx=0).stop 2023-10-26T01:42:12.428 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:12.425+0000 7fd40f81a640 1 -- 172.21.15.169:0/926253736 shutdown_connections 2023-10-26T01:42:12.428 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:12.425+0000 7fd40f81a640 1 --2- 172.21.15.169:0/926253736 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd3ec06ae70 0x7fd3ec06d330 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:12.428 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:12.425+0000 7fd40f81a640 1 --2- 172.21.15.169:0/926253736 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd40814ff10 0x7fd40806ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:12.429 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:12.425+0000 7fd40f81a640 1 --2- 172.21.15.169:0/926253736 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd408147cf0 0x7fd40814f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:12.429 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:12.425+0000 7fd40f81a640 1 --2- 172.21.15.169:0/926253736 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd3f4005180 0x7fd4080c6660 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:12.429 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:12.425+0000 7fd40f81a640 1 -- 172.21.15.169:0/926253736 >> 172.21.15.169:0/926253736 conn(0x7fd408069930 msgr2=0x7fd40815e270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:42:12.429 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:12.425+0000 7fd40f81a640 1 -- 172.21.15.169:0/926253736 shutdown_connections 2023-10-26T01:42:12.429 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:12.426+0000 7fd40f81a640 1 -- 172.21.15.169:0/926253736 wait complete. 2023-10-26T01:42:12.448 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-26T01:42:12.449 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-26T01:42:12.678 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:12.676+0000 7f182084f640 1 -- 172.21.15.169:0/1210867901 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f181c0b0c80 msgr2=0x7f181c0cd710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:12.678 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:12.676+0000 7f182084f640 1 --2- 172.21.15.169:0/1210867901 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f181c0b0c80 0x7f181c0cd710 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f181c0b1ea0 tx=0x7f181402c300 comp rx=0 tx=0).stop 2023-10-26T01:42:12.678 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:12.676+0000 7f182084f640 1 -- 172.21.15.169:0/1210867901 shutdown_connections 2023-10-26T01:42:12.679 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:12.676+0000 7f182084f640 1 --2- 172.21.15.169:0/1210867901 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f181c0b0c80 0x7f181c0cd710 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:12.679 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:12.676+0000 7f182084f640 1 --2- 172.21.15.169:0/1210867901 >> v2:172.21.15.170:3300/0 conn(0x7f181c1593e0 0x7f181c0ac200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:12.679 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:12.676+0000 7f182084f640 1 -- 172.21.15.169:0/1210867901 >> 172.21.15.169:0/1210867901 conn(0x7f181c069930 msgr2=0x7f181c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:42:12.679 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:12.676+0000 7f182084f640 1 -- 172.21.15.169:0/1210867901 shutdown_connections 2023-10-26T01:42:12.679 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:12.676+0000 7f182084f640 1 -- 172.21.15.169:0/1210867901 wait complete. 2023-10-26T01:42:12.680 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:12.677+0000 7f182084f640 1 Processor -- start 2023-10-26T01:42:12.680 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:12.678+0000 7f182084f640 1 -- start start 2023-10-26T01:42:12.680 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:12.679+0000 7f182084f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f181c0b0c80 0x7f181c149750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:12.681 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:12.679+0000 7f182084f640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f181c1593e0 0x7f181c06ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:12.681 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:12.679+0000 7f182084f640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f181c1597c0 0x7f181c06b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:12.681 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:12.679+0000 7f182084f640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f181c0c6d30 con 0x7f181c1597c0 2023-10-26T01:42:12.681 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:12.679+0000 7f182084f640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f181c158ac0 con 0x7f181c0b0c80 2023-10-26T01:42:12.681 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:12.679+0000 7f182084f640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f181c0c6f80 con 0x7f181c1593e0 2023-10-26T01:42:12.683 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:12.679+0000 7f181ad9d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f181c1597c0 0x7f181c06b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:42:12.683 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:12.679+0000 7f181ad9d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f181c1597c0 0x7f181c06b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:56276/0 (socket says 172.21.15.169:56276) 2023-10-26T01:42:12.683 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:12.679+0000 7f181ad9d640 1 -- 172.21.15.169:0/3683199200 learned_addr learned my addr 172.21.15.169:0/3683199200 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:42:12.683 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:12.679+0000 7f1819d9b640 1 --2- 172.21.15.169:0/3683199200 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f181c1593e0 0x7f181c06ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:42:12.684 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:12.679+0000 7f181ad9d640 1 -- 172.21.15.169:0/3683199200 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f181c1593e0 msgr2=0x7f181c06ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:12.684 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:12.679+0000 7f181a59c640 1 --2- 172.21.15.169:0/3683199200 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f181c0b0c80 0x7f181c149750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:42:12.684 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:12.679+0000 7f181ad9d640 1 --2- 172.21.15.169:0/3683199200 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f181c1593e0 0x7f181c06ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:12.684 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:12.679+0000 7f181ad9d640 1 -- 172.21.15.169:0/3683199200 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f181c0b0c80 msgr2=0x7f181c149750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:12.684 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:12.679+0000 7f181ad9d640 1 --2- 172.21.15.169:0/3683199200 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f181c0b0c80 0x7f181c149750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:12.685 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:12.679+0000 7f181ad9d640 1 -- 172.21.15.169:0/3683199200 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1814009900 con 0x7f181c1597c0 2023-10-26T01:42:12.685 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:12.679+0000 7f1819d9b640 1 --2- 172.21.15.169:0/3683199200 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f181c1593e0 0x7f181c06ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:42:12.685 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:12.680+0000 7f181ad9d640 1 --2- 172.21.15.169:0/3683199200 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f181c1597c0 0x7f181c06b2d0 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f1814037040 tx=0x7f1814003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:42:12.685 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:12.680+0000 7f1812ffd640 1 -- 172.21.15.169:0/3683199200 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1814004250 con 0x7f181c1597c0 2023-10-26T01:42:12.685 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:12.680+0000 7f1812ffd640 1 -- 172.21.15.169:0/3683199200 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f18140044a0 con 0x7f181c1597c0 2023-10-26T01:42:12.686 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:12.680+0000 7f1812ffd640 1 -- 172.21.15.169:0/3683199200 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1814004890 con 0x7f181c1597c0 2023-10-26T01:42:12.686 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:12.680+0000 7f182084f640 1 -- 172.21.15.169:0/3683199200 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f181c0c7260 con 0x7f181c1597c0 2023-10-26T01:42:12.686 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:12.680+0000 7f182084f640 1 -- 172.21.15.169:0/3683199200 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f181c149c90 con 0x7f181c1597c0 2023-10-26T01:42:12.686 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:12.681+0000 7f182084f640 1 -- 172.21.15.169:0/3683199200 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f181c0c7780 con 0x7f181c1597c0 2023-10-26T01:42:12.687 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:12.682+0000 7f1812ffd640 1 -- 172.21.15.169:0/3683199200 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f181403b040 con 0x7f181c1597c0 2023-10-26T01:42:12.687 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:12.683+0000 7f1812ffd640 1 --2- 172.21.15.169:0/3683199200 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f17fc06ae40 0x7f17fc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:12.687 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:12.683+0000 7f1812ffd640 1 -- 172.21.15.169:0/3683199200 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f18140ac4b0 con 0x7f181c1597c0 2023-10-26T01:42:12.688 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:12.684+0000 7f181a59c640 1 --2- 172.21.15.169:0/3683199200 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f17fc06ae40 0x7f17fc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:42:12.688 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:12.685+0000 7f181a59c640 1 --2- 172.21.15.169:0/3683199200 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f17fc06ae40 0x7f17fc06d300 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f180c00ba40 tx=0x7f180c005cc0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:42:12.689 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:12.688+0000 7f1812ffd640 1 -- 172.21.15.169:0/3683199200 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f181c0c7780 con 0x7f181c1597c0 2023-10-26T01:42:12.922 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:12.920+0000 7f182084f640 1 -- 172.21.15.169:0/3683199200 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f181c0c7780 con 0x7f181c1597c0 2023-10-26T01:42:13.435 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:13.433+0000 7f1812ffd640 1 -- 172.21.15.169:0/3683199200 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v86) v1 ==== 122+0+0 (secure 0 0 0) 0x7f181c0c7780 con 0x7f181c1597c0 2023-10-26T01:42:13.439 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:13.437+0000 7f182084f640 1 -- 172.21.15.169:0/3683199200 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f17fc06ae40 msgr2=0x7f17fc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:13.439 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:13.437+0000 7f182084f640 1 --2- 172.21.15.169:0/3683199200 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f17fc06ae40 0x7f17fc06d300 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f180c00ba40 tx=0x7f180c005cc0 comp rx=0 tx=0).stop 2023-10-26T01:42:13.439 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:13.437+0000 7f182084f640 1 -- 172.21.15.169:0/3683199200 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f181c1597c0 msgr2=0x7f181c06b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:13.439 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:13.437+0000 7f182084f640 1 --2- 172.21.15.169:0/3683199200 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f181c1597c0 0x7f181c06b2d0 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f1814037040 tx=0x7f1814003c10 comp rx=0 tx=0).stop 2023-10-26T01:42:13.439 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:13.437+0000 7f182084f640 1 -- 172.21.15.169:0/3683199200 shutdown_connections 2023-10-26T01:42:13.440 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:13.437+0000 7f182084f640 1 --2- 172.21.15.169:0/3683199200 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f17fc06ae40 0x7f17fc06d300 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:13.440 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:13.437+0000 7f182084f640 1 --2- 172.21.15.169:0/3683199200 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f181c1597c0 0x7f181c06b2d0 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:13.440 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:13.437+0000 7f182084f640 1 --2- 172.21.15.169:0/3683199200 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f181c1593e0 0x7f181c06ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:13.440 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:13.437+0000 7f182084f640 1 --2- 172.21.15.169:0/3683199200 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f181c0b0c80 0x7f181c149750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:13.440 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:13.437+0000 7f182084f640 1 -- 172.21.15.169:0/3683199200 >> 172.21.15.169:0/3683199200 conn(0x7f181c069930 msgr2=0x7f181c0ad760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:42:13.441 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:13.437+0000 7f182084f640 1 -- 172.21.15.169:0/3683199200 shutdown_connections 2023-10-26T01:42:13.441 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:13.437+0000 7f182084f640 1 -- 172.21.15.169:0/3683199200 wait complete. 2023-10-26T01:42:13.459 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-26T01:42:13.460 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T01:42:13.680 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:13.678+0000 7fa995660640 1 -- 172.21.15.169:0/3740857913 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa990163210 msgr2=0x7fa990165600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:13.680 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:13.678+0000 7fa995660640 1 --2- 172.21.15.169:0/3740857913 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa990163210 0x7fa990165600 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fa988005fd0 tx=0x7fa98802c2d0 comp rx=0 tx=0).stop 2023-10-26T01:42:13.680 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:13.678+0000 7fa995660640 1 -- 172.21.15.169:0/3740857913 shutdown_connections 2023-10-26T01:42:13.681 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:13.678+0000 7fa995660640 1 --2- 172.21.15.169:0/3740857913 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa990163210 0x7fa990165600 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:13.681 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:13.678+0000 7fa995660640 1 --2- 172.21.15.169:0/3740857913 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa990065490 0x7fa99006cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:13.681 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:13.678+0000 7fa995660640 1 -- 172.21.15.169:0/3740857913 >> 172.21.15.169:0/3740857913 conn(0x7fa9900af650 msgr2=0x7fa9900afa30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:42:13.681 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:13.678+0000 7fa995660640 1 -- 172.21.15.169:0/3740857913 shutdown_connections 2023-10-26T01:42:13.681 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:13.678+0000 7fa995660640 1 -- 172.21.15.169:0/3740857913 wait complete. 2023-10-26T01:42:13.682 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:13.680+0000 7fa995660640 1 Processor -- start 2023-10-26T01:42:13.682 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:13.680+0000 7fa995660640 1 -- start start 2023-10-26T01:42:13.683 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:13.681+0000 7fa995660640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa990065490 0x7fa99015e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:13.683 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:13.681+0000 7fa995660640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa9900cba10 0x7fa990159820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:13.684 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:13.681+0000 7fa995660640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa990163210 0x7fa9901f6f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:13.684 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:13.681+0000 7fa995660640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa99014cc40 con 0x7fa990163210 2023-10-26T01:42:13.684 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:13.681+0000 7fa995660640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa99014d750 con 0x7fa9900cba10 2023-10-26T01:42:13.684 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:13.681+0000 7fa995660640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa99015eb40 con 0x7fa990065490 2023-10-26T01:42:13.684 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:13.681+0000 7fa994e5f640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa990163210 0x7fa9901f6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:42:13.684 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:13.681+0000 7fa994e5f640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa990163210 0x7fa9901f6f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:56290/0 (socket says 172.21.15.169:56290) 2023-10-26T01:42:13.685 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:13.681+0000 7fa994e5f640 1 -- 172.21.15.169:0/2515527778 learned_addr learned my addr 172.21.15.169:0/2515527778 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:42:13.685 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:13.682+0000 7fa98ffff640 1 --2- 172.21.15.169:0/2515527778 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa990065490 0x7fa99015e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:42:13.685 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:13.682+0000 7fa98f7fe640 1 --2- 172.21.15.169:0/2515527778 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa9900cba10 0x7fa990159820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:42:13.685 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:13.682+0000 7fa994e5f640 1 -- 172.21.15.169:0/2515527778 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa990065490 msgr2=0x7fa99015e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:13.685 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:13.682+0000 7fa994e5f640 1 --2- 172.21.15.169:0/2515527778 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa990065490 0x7fa99015e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:13.686 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:13.682+0000 7fa994e5f640 1 -- 172.21.15.169:0/2515527778 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa9900cba10 msgr2=0x7fa990159820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:13.686 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:13.682+0000 7fa994e5f640 1 --2- 172.21.15.169:0/2515527778 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa9900cba10 0x7fa990159820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:13.686 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:13.682+0000 7fa994e5f640 1 -- 172.21.15.169:0/2515527778 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa988009900 con 0x7fa990163210 2023-10-26T01:42:13.687 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:13.682+0000 7fa994e5f640 1 --2- 172.21.15.169:0/2515527778 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa990163210 0x7fa9901f6f10 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7fa988009bf0 tx=0x7fa988003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:42:13.687 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:13.682+0000 7fa98cff9640 1 -- 172.21.15.169:0/2515527778 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa988004450 con 0x7fa990163210 2023-10-26T01:42:13.688 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:13.683+0000 7fa98cff9640 1 -- 172.21.15.169:0/2515527778 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa98802de90 con 0x7fa990163210 2023-10-26T01:42:13.688 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:13.683+0000 7fa995660640 1 -- 172.21.15.169:0/2515527778 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa99014ce20 con 0x7fa990163210 2023-10-26T01:42:13.688 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:13.683+0000 7fa98cff9640 1 -- 172.21.15.169:0/2515527778 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa988003e10 con 0x7fa990163210 2023-10-26T01:42:13.688 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:13.683+0000 7fa995660640 1 -- 172.21.15.169:0/2515527778 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa99014d3e0 con 0x7fa990163210 2023-10-26T01:42:13.688 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:13.684+0000 7fa995660640 1 -- 172.21.15.169:0/2515527778 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa99006b900 con 0x7fa990163210 2023-10-26T01:42:13.689 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:13.685+0000 7fa98cff9640 1 -- 172.21.15.169:0/2515527778 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fa98800a040 con 0x7fa990163210 2023-10-26T01:42:13.689 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:13.686+0000 7fa98cff9640 1 --2- 172.21.15.169:0/2515527778 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa97c06ae40 0x7fa97c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:13.689 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:13.686+0000 7fa98cff9640 1 -- 172.21.15.169:0/2515527778 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa9880ac5e0 con 0x7fa990163210 2023-10-26T01:42:13.689 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:13.686+0000 7fa98ffff640 1 --2- 172.21.15.169:0/2515527778 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa97c06ae40 0x7fa97c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:42:13.690 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:13.687+0000 7fa98ffff640 1 --2- 172.21.15.169:0/2515527778 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa97c06ae40 0x7fa97c06d300 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fa980001640 tx=0x7fa980009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:42:13.692 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:13.690+0000 7fa98cff9640 1 -- 172.21.15.169:0/2515527778 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa988077690 con 0x7fa990163210 2023-10-26T01:42:13.919 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:13.917+0000 7fa995660640 1 -- 172.21.15.169:0/2515527778 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa990073c10 con 0x7fa990163210 2023-10-26T01:42:13.920 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:13.919+0000 7fa98cff9640 1 -- 172.21.15.169:0/2515527778 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) v1 ==== 94+0+4923 (secure 0 0 0) 0x7fa98807acb0 con 0x7fa990163210 2023-10-26T01:42:13.921 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:42:13.921 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":86,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5861,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/326391134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":326391134},{"type":"v1","addr":"172.21.15.170:6836","nonce":326391134}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":5864,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/3374486280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":3374486280},{"type":"v1","addr":"172.21.15.170:6837","nonce":3374486280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":8697,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/1445776478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":1445776478},{"type":"v1","addr":"172.21.15.169:6834","nonce":1445776478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T01:42:09.853788+0000","modified":"2023-10-26T01:42:12.922104+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8700},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8700":{"gid":8700,"name":"a","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6835/228678953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":228678953},{"type":"v1","addr":"172.21.15.169:6835","nonce":228678953}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-10-26T01:42:13.922 INFO:teuthology.orchestra.run.smithi169.stderr:dumped fsmap epoch 86 2023-10-26T01:42:13.924 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:13.921+0000 7fa995660640 1 -- 172.21.15.169:0/2515527778 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa97c06ae40 msgr2=0x7fa97c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:13.924 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:13.921+0000 7fa995660640 1 --2- 172.21.15.169:0/2515527778 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa97c06ae40 0x7fa97c06d300 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fa980001640 tx=0x7fa980009040 comp rx=0 tx=0).stop 2023-10-26T01:42:13.924 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:13.922+0000 7fa995660640 1 -- 172.21.15.169:0/2515527778 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa990163210 msgr2=0x7fa9901f6f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:13.924 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:13.922+0000 7fa995660640 1 --2- 172.21.15.169:0/2515527778 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa990163210 0x7fa9901f6f10 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7fa988009bf0 tx=0x7fa988003c20 comp rx=0 tx=0).stop 2023-10-26T01:42:13.924 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:13.922+0000 7fa995660640 1 -- 172.21.15.169:0/2515527778 shutdown_connections 2023-10-26T01:42:13.925 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:13.922+0000 7fa995660640 1 --2- 172.21.15.169:0/2515527778 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa97c06ae40 0x7fa97c06d300 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:13.925 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:13.922+0000 7fa995660640 1 --2- 172.21.15.169:0/2515527778 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa990163210 0x7fa9901f6f10 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:13.925 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:13.922+0000 7fa995660640 1 --2- 172.21.15.169:0/2515527778 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa9900cba10 0x7fa990159820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:13.925 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:13.922+0000 7fa995660640 1 --2- 172.21.15.169:0/2515527778 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa990065490 0x7fa99015e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:13.925 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:13.922+0000 7fa995660640 1 -- 172.21.15.169:0/2515527778 >> 172.21.15.169:0/2515527778 conn(0x7fa9900af650 msgr2=0x7fa9900ca160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:42:13.926 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:13.922+0000 7fa995660640 1 -- 172.21.15.169:0/2515527778 shutdown_connections 2023-10-26T01:42:13.926 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:13.922+0000 7fa995660640 1 -- 172.21.15.169:0/2515527778 wait complete. 2023-10-26T01:42:13.941 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:42:14.159 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:14.157+0000 7fe7e4c97640 1 -- 172.21.15.169:0/268249515 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7cc0047c0 msgr2=0x7fe7cc004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:14.159 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:14.157+0000 7fe7e4c97640 1 --2- 172.21.15.169:0/268249515 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7cc0047c0 0x7fe7cc004c40 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7fe7c8001680 tx=0x7fe7cc0057f0 comp rx=0 tx=0).stop 2023-10-26T01:42:14.160 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:14.157+0000 7fe7e4c97640 1 -- 172.21.15.169:0/268249515 shutdown_connections 2023-10-26T01:42:14.160 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:14.157+0000 7fe7e4c97640 1 --2- 172.21.15.169:0/268249515 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7cc0047c0 0x7fe7cc004c40 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:14.160 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:14.157+0000 7fe7e4c97640 1 -- 172.21.15.169:0/268249515 >> 172.21.15.169:0/268249515 conn(0x7fe7e0069930 msgr2=0x7fe7e0069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:42:14.160 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:14.157+0000 7fe7e4c97640 1 -- 172.21.15.169:0/268249515 shutdown_connections 2023-10-26T01:42:14.160 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:14.157+0000 7fe7e4c97640 1 -- 172.21.15.169:0/268249515 wait complete. 2023-10-26T01:42:14.161 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:14.159+0000 7fe7e4c97640 1 Processor -- start 2023-10-26T01:42:14.161 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:14.159+0000 7fe7e4c97640 1 -- start start 2023-10-26T01:42:14.163 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:14.160+0000 7fe7e4c97640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe7cc0047c0 0x7fe7e01411e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:14.163 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:14.160+0000 7fe7e4c97640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe7e0070140 0x7fe7e01e6770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:14.163 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:14.160+0000 7fe7e4c97640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7e006ad30 0x7fe7e006b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:14.163 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:14.160+0000 7fe7e4c97640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e006b6f0 con 0x7fe7e006ad30 2023-10-26T01:42:14.163 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:14.160+0000 7fe7e4c97640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e0071880 con 0x7fe7cc0047c0 2023-10-26T01:42:14.163 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:14.160+0000 7fe7e4c97640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e0141720 con 0x7fe7e0070140 2023-10-26T01:42:14.164 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:14.161+0000 7fe7ded9d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7e006ad30 0x7fe7e006b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:42:14.164 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:14.161+0000 7fe7ded9d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7e006ad30 0x7fe7e006b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:54100/0 (socket says 172.21.15.169:54100) 2023-10-26T01:42:14.164 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:14.161+0000 7fe7ded9d640 1 -- 172.21.15.169:0/252823608 learned_addr learned my addr 172.21.15.169:0/252823608 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:42:14.164 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:14.161+0000 7fe7de59c640 1 --2- 172.21.15.169:0/252823608 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe7cc0047c0 0x7fe7e01411e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:42:14.164 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:14.161+0000 7fe7d7fff640 1 --2- 172.21.15.169:0/252823608 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe7e0070140 0x7fe7e01e6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:42:14.165 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:14.161+0000 7fe7ded9d640 1 -- 172.21.15.169:0/252823608 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe7e0070140 msgr2=0x7fe7e01e6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:14.165 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:14.161+0000 7fe7ded9d640 1 --2- 172.21.15.169:0/252823608 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe7e0070140 0x7fe7e01e6770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:14.165 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:14.161+0000 7fe7ded9d640 1 -- 172.21.15.169:0/252823608 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe7cc0047c0 msgr2=0x7fe7e01411e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:14.165 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:14.161+0000 7fe7ded9d640 1 --2- 172.21.15.169:0/252823608 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe7cc0047c0 0x7fe7e01411e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:14.165 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:14.161+0000 7fe7ded9d640 1 -- 172.21.15.169:0/252823608 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7e0141bb0 con 0x7fe7e006ad30 2023-10-26T01:42:14.165 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:14.161+0000 7fe7d7fff640 1 --2- 172.21.15.169:0/252823608 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe7e0070140 0x7fe7e01e6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:42:14.167 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:14.161+0000 7fe7ded9d640 1 --2- 172.21.15.169:0/252823608 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7e006ad30 0x7fe7e006b1b0 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7fe7d8005fd0 tx=0x7fe7d800aa70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:42:14.167 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:14.162+0000 7fe7d6ffd640 1 -- 172.21.15.169:0/252823608 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d8011070 con 0x7fe7e006ad30 2023-10-26T01:42:14.167 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:14.162+0000 7fe7d6ffd640 1 -- 172.21.15.169:0/252823608 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe7d80092e0 con 0x7fe7e006ad30 2023-10-26T01:42:14.167 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:14.162+0000 7fe7e4c97640 1 -- 172.21.15.169:0/252823608 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7e006f0a0 con 0x7fe7e006ad30 2023-10-26T01:42:14.167 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:14.162+0000 7fe7d6ffd640 1 -- 172.21.15.169:0/252823608 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d8009490 con 0x7fe7e006ad30 2023-10-26T01:42:14.168 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:14.162+0000 7fe7e4c97640 1 -- 172.21.15.169:0/252823608 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7e006f660 con 0x7fe7e006ad30 2023-10-26T01:42:14.168 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:14.163+0000 7fe7e4c97640 1 -- 172.21.15.169:0/252823608 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7e004db10 con 0x7fe7e006ad30 2023-10-26T01:42:14.168 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:14.164+0000 7fe7d6ffd640 1 -- 172.21.15.169:0/252823608 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fe7d800c590 con 0x7fe7e006ad30 2023-10-26T01:42:14.168 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:14.165+0000 7fe7d6ffd640 1 --2- 172.21.15.169:0/252823608 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe7c006ae40 0x7fe7c006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:14.168 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:14.165+0000 7fe7d6ffd640 1 -- 172.21.15.169:0/252823608 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe7d8088a20 con 0x7fe7e006ad30 2023-10-26T01:42:14.169 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:14.166+0000 7fe7de59c640 1 --2- 172.21.15.169:0/252823608 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe7c006ae40 0x7fe7c006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:42:14.169 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:14.167+0000 7fe7de59c640 1 --2- 172.21.15.169:0/252823608 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe7c006ae40 0x7fe7c006d300 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fe7d00017a0 tx=0x7fe7d0001590 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:42:14.171 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:14.170+0000 7fe7d6ffd640 1 -- 172.21.15.169:0/252823608 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe7d8053ad0 con 0x7fe7e006ad30 2023-10-26T01:42:14.362 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:14.360+0000 7fe7e4c97640 1 -- 172.21.15.169:0/252823608 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe7e00aca70 con 0x7fe7e006ad30 2023-10-26T01:42:14.365 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:14.363+0000 7fe7d6ffd640 1 -- 172.21.15.169:0/252823608 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v166) v1 ==== 74+0+16628 (secure 0 0 0) 0x7fe7d80570f0 con 0x7fe7e006ad30 2023-10-26T01:42:14.365 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:42:14.365 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":166,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:42:11.381192+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2023-10-26T01:42:07.656531+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"165","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2023-10-26T01:42:08.866635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"166","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:42:14.368 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:14.366+0000 7fe7e4c97640 1 -- 172.21.15.169:0/252823608 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe7c006ae40 msgr2=0x7fe7c006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:14.368 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:14.366+0000 7fe7e4c97640 1 --2- 172.21.15.169:0/252823608 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe7c006ae40 0x7fe7c006d300 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fe7d00017a0 tx=0x7fe7d0001590 comp rx=0 tx=0).stop 2023-10-26T01:42:14.369 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:14.366+0000 7fe7e4c97640 1 -- 172.21.15.169:0/252823608 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7e006ad30 msgr2=0x7fe7e006b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:14.369 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:14.366+0000 7fe7e4c97640 1 --2- 172.21.15.169:0/252823608 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7e006ad30 0x7fe7e006b1b0 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7fe7d8005fd0 tx=0x7fe7d800aa70 comp rx=0 tx=0).stop 2023-10-26T01:42:14.369 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:14.366+0000 7fe7e4c97640 1 -- 172.21.15.169:0/252823608 shutdown_connections 2023-10-26T01:42:14.369 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:14.366+0000 7fe7e4c97640 1 --2- 172.21.15.169:0/252823608 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe7c006ae40 0x7fe7c006d300 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:14.369 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:14.366+0000 7fe7e4c97640 1 --2- 172.21.15.169:0/252823608 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7e006ad30 0x7fe7e006b1b0 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:14.369 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:14.366+0000 7fe7e4c97640 1 --2- 172.21.15.169:0/252823608 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe7e0070140 0x7fe7e01e6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:14.370 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:14.366+0000 7fe7e4c97640 1 --2- 172.21.15.169:0/252823608 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe7cc0047c0 0x7fe7e01411e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:14.370 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:14.366+0000 7fe7e4c97640 1 -- 172.21.15.169:0/252823608 >> 172.21.15.169:0/252823608 conn(0x7fe7e0069930 msgr2=0x7fe7e00acc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:42:14.370 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:14.366+0000 7fe7e4c97640 1 -- 172.21.15.169:0/252823608 shutdown_connections 2023-10-26T01:42:14.370 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:14.366+0000 7fe7e4c97640 1 -- 172.21.15.169:0/252823608 wait complete. 2023-10-26T01:42:14.389 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-26T01:42:14.389 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-26T01:42:14.609 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:14.607+0000 7f6ccdbf4640 1 -- 172.21.15.169:0/604119171 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6cb0003970 msgr2=0x7f6cb0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:14.610 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:14.607+0000 7f6ccdbf4640 1 --2- 172.21.15.169:0/604119171 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6cb0003970 0x7f6cb0003d90 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f6cc80cdff0 tx=0x7f6cb00057f0 comp rx=0 tx=0).stop 2023-10-26T01:42:14.610 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:14.607+0000 7f6ccd3f3640 1 -- 172.21.15.169:0/604119171 reap_dead start 2023-10-26T01:42:14.610 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:14.607+0000 7f6ccdbf4640 1 -- 172.21.15.169:0/604119171 shutdown_connections 2023-10-26T01:42:14.610 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:14.607+0000 7f6ccdbf4640 1 -- 172.21.15.169:0/604119171 >> 172.21.15.169:0/604119171 conn(0x7f6cc80aedd0 msgr2=0x7f6cc80af1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:42:14.610 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:14.607+0000 7f6ccdbf4640 1 -- 172.21.15.169:0/604119171 shutdown_connections 2023-10-26T01:42:14.611 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:14.607+0000 7f6ccdbf4640 1 -- 172.21.15.169:0/604119171 wait complete. 2023-10-26T01:42:14.611 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:14.609+0000 7f6ccdbf4640 1 Processor -- start 2023-10-26T01:42:14.611 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:14.609+0000 7f6ccdbf4640 1 -- start start 2023-10-26T01:42:14.613 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:14.610+0000 7f6ccdbf4640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6cb0003970 0x7f6cc81f5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:14.613 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:14.610+0000 7f6ccdbf4640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6cc80afce0 0x7f6cc8146650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:14.614 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:14.610+0000 7f6ccdbf4640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6cc81f5000 0x7f6cc81f5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:14.614 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:14.610+0000 7f6ccdbf4640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc80b2b80 con 0x7f6cc81f5000 2023-10-26T01:42:14.614 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:14.610+0000 7f6ccdbf4640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc81f59c0 con 0x7f6cb0003970 2023-10-26T01:42:14.614 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:14.610+0000 7f6ccdbf4640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc80b22e0 con 0x7f6cc80afce0 2023-10-26T01:42:14.614 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:14.611+0000 7f6ccd3f3640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6cc81f5000 0x7f6cc81f5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:42:14.615 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:14.611+0000 7f6ccd3f3640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6cc81f5000 0x7f6cc81f5480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:54116/0 (socket says 172.21.15.169:54116) 2023-10-26T01:42:14.615 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:14.611+0000 7f6ccd3f3640 1 -- 172.21.15.169:0/3727742521 learned_addr learned my addr 172.21.15.169:0/3727742521 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:42:14.615 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:14.611+0000 7f6cccbf2640 1 --2- 172.21.15.169:0/3727742521 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6cb0003970 0x7f6cc81f5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:42:14.615 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:14.611+0000 7f6ccd3f3640 1 -- 172.21.15.169:0/3727742521 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6cc80afce0 msgr2=0x7f6cc8146650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:14.615 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:14.611+0000 7f6ccd3f3640 1 --2- 172.21.15.169:0/3727742521 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6cc80afce0 0x7f6cc8146650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:14.615 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:14.611+0000 7f6ccd3f3640 1 -- 172.21.15.169:0/3727742521 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6cb0003970 msgr2=0x7f6cc81f5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:14.616 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:14.611+0000 7f6ccd3f3640 1 --2- 172.21.15.169:0/3727742521 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6cb0003970 0x7f6cc81f5ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:14.616 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:14.611+0000 7f6ccd3f3640 1 -- 172.21.15.169:0/3727742521 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cc80b25c0 con 0x7f6cc81f5000 2023-10-26T01:42:14.616 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:14.611+0000 7f6cccbf2640 1 --2- 172.21.15.169:0/3727742521 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6cb0003970 0x7f6cc81f5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:42:14.616 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:14.611+0000 7f6ccd3f3640 1 --2- 172.21.15.169:0/3727742521 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6cc81f5000 0x7f6cc81f5480 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f6cc0000f80 tx=0x7f6cc000dca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:42:14.616 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:14.612+0000 7f6cc57fa640 1 -- 172.21.15.169:0/3727742521 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc000b8a0 con 0x7f6cc81f5000 2023-10-26T01:42:14.616 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:14.612+0000 7f6cc57fa640 1 -- 172.21.15.169:0/3727742521 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6cc000ba00 con 0x7f6cc81f5000 2023-10-26T01:42:14.617 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:14.612+0000 7f6ccdbf4640 1 -- 172.21.15.169:0/3727742521 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cc80b28a0 con 0x7f6cc81f5000 2023-10-26T01:42:14.617 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:14.612+0000 7f6cc57fa640 1 -- 172.21.15.169:0/3727742521 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc000be10 con 0x7f6cc81f5000 2023-10-26T01:42:14.617 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:14.612+0000 7f6ccdbf4640 1 -- 172.21.15.169:0/3727742521 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cc80b0160 con 0x7f6cc81f5000 2023-10-26T01:42:14.618 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:14.613+0000 7f6ccdbf4640 1 -- 172.21.15.169:0/3727742521 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cc80b0320 con 0x7f6cc81f5000 2023-10-26T01:42:14.618 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:14.614+0000 7f6cc57fa640 1 -- 172.21.15.169:0/3727742521 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f6cc001c410 con 0x7f6cc81f5000 2023-10-26T01:42:14.618 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:14.615+0000 7f6cc57fa640 1 --2- 172.21.15.169:0/3727742521 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f6cb406ae70 0x7f6cb406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:14.619 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:14.615+0000 7f6cc57fa640 1 -- 172.21.15.169:0/3727742521 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6cc00890e0 con 0x7f6cc81f5000 2023-10-26T01:42:14.619 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:14.615+0000 7f6cccbf2640 1 --2- 172.21.15.169:0/3727742521 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f6cb406ae70 0x7f6cb406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:42:14.619 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:14.616+0000 7f6cccbf2640 1 --2- 172.21.15.169:0/3727742521 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f6cb406ae70 0x7f6cb406d330 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc037000 tx=0x7f6cbc02e040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:42:14.620 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:14.619+0000 7f6cc57fa640 1 -- 172.21.15.169:0/3727742521 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6cc00540f0 con 0x7f6cc81f5000 2023-10-26T01:42:14.812 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:14.810+0000 7f6ccdbf4640 1 -- 172.21.15.169:0/3727742521 --> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6cc806b830 con 0x7f6cb406ae70 2023-10-26T01:42:14.836 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:14.835+0000 7f6cc57fa640 1 -- 172.21.15.169:0/3727742521 <== mgr.4112 v2:172.21.15.170:6832/1187156918 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+282625 (secure 0 0 0) 0x7f6cbc02cc20 con 0x7f6cb406ae70 2023-10-26T01:42:14.840 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:42:14.842 INFO:teuthology.orchestra.run.smithi169.stderr:dumped all 2023-10-26T01:42:14.843 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:14.840+0000 7f6ccdbf4640 1 -- 172.21.15.169:0/3727742521 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f6cb406ae70 msgr2=0x7f6cb406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:14.844 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:14.840+0000 7f6ccdbf4640 1 --2- 172.21.15.169:0/3727742521 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f6cb406ae70 0x7f6cb406d330 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc037000 tx=0x7f6cbc02e040 comp rx=0 tx=0).stop 2023-10-26T01:42:14.844 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:14.840+0000 7f6ccdbf4640 1 -- 172.21.15.169:0/3727742521 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6cc81f5000 msgr2=0x7f6cc81f5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:14.844 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:14.840+0000 7f6ccdbf4640 1 --2- 172.21.15.169:0/3727742521 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6cc81f5000 0x7f6cc81f5480 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f6cc0000f80 tx=0x7f6cc000dca0 comp rx=0 tx=0).stop 2023-10-26T01:42:14.844 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:14.840+0000 7f6ccdbf4640 1 -- 172.21.15.169:0/3727742521 shutdown_connections 2023-10-26T01:42:14.844 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:14.840+0000 7f6ccdbf4640 1 --2- 172.21.15.169:0/3727742521 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f6cb406ae70 0x7f6cb406d330 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:14.844 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:14.840+0000 7f6ccdbf4640 1 --2- 172.21.15.169:0/3727742521 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6cc81f5000 0x7f6cc81f5480 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:14.845 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:14.840+0000 7f6ccdbf4640 1 --2- 172.21.15.169:0/3727742521 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6cc80afce0 0x7f6cc8146650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:14.845 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:14.840+0000 7f6ccdbf4640 1 --2- 172.21.15.169:0/3727742521 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6cb0003970 0x7f6cc81f5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:14.845 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:14.840+0000 7f6ccdbf4640 1 -- 172.21.15.169:0/3727742521 >> 172.21.15.169:0/3727742521 conn(0x7f6cc80aedd0 msgr2=0x7f6cc80af1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:42:14.845 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:14.840+0000 7f6ccdbf4640 1 -- 172.21.15.169:0/3727742521 shutdown_connections 2023-10-26T01:42:14.845 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:14.840+0000 7f6ccdbf4640 1 -- 172.21.15.169:0/3727742521 wait complete. 2023-10-26T01:42:14.860 INFO:teuthology.orchestra.run.smithi169.stdout:{"pg_ready":true,"pg_map":{"version":511,"stamp":"2023-10-26T01:42:14.382552+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":80,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":8369,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1554130,"statfs":{"total":1600000000,"available":1591429158,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002078"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.418672+0000","last_change":"2023-10-26T01:42:10.418587+0000","last_active":"2023-10-26T01:42:10.418672+0000","last_peered":"2023-10-26T01:42:10.418672+0000","last_clean":"2023-10-26T01:42:10.418672+0000","last_became_active":"2023-10-26T01:42:10.418333+0000","last_became_peered":"2023-10-26T01:42:10.418333+0000","last_unstale":"2023-10-26T01:42:10.418672+0000","last_undegraded":"2023-10-26T01:42:10.418672+0000","last_fullsized":"2023-10-26T01:42:10.418672+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:54:20.518649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":11,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-26T01:42:11.913772+0000","last_change":"2023-10-26T01:42:10.395338+0000","last_active":"2023-10-26T01:42:11.913772+0000","last_peered":"2023-10-26T01:42:11.913772+0000","last_clean":"2023-10-26T01:42:11.913772+0000","last_became_active":"2023-10-26T01:42:10.395079+0000","last_became_peered":"2023-10-26T01:42:10.395079+0000","last_unstale":"2023-10-26T01:42:11.913772+0000","last_undegraded":"2023-10-26T01:42:11.913772+0000","last_fullsized":"2023-10-26T01:42:11.913772+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:05:17.258140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.407471+0000","last_change":"2023-10-26T01:42:10.406941+0000","last_active":"2023-10-26T01:42:10.407471+0000","last_peered":"2023-10-26T01:42:10.407471+0000","last_clean":"2023-10-26T01:42:10.407471+0000","last_became_active":"2023-10-26T01:42:10.406338+0000","last_became_peered":"2023-10-26T01:42:10.406338+0000","last_unstale":"2023-10-26T01:42:10.407471+0000","last_undegraded":"2023-10-26T01:42:10.407471+0000","last_fullsized":"2023-10-26T01:42:10.407471+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:15:19.901066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.408843+0000","last_change":"2023-10-26T01:42:10.408057+0000","last_active":"2023-10-26T01:42:10.408843+0000","last_peered":"2023-10-26T01:42:10.408843+0000","last_clean":"2023-10-26T01:42:10.408843+0000","last_became_active":"2023-10-26T01:42:10.407815+0000","last_became_peered":"2023-10-26T01:42:10.407815+0000","last_unstale":"2023-10-26T01:42:10.408843+0000","last_undegraded":"2023-10-26T01:42:10.408843+0000","last_fullsized":"2023-10-26T01:42:10.408843+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:44:45.588628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.414596+0000","last_change":"2023-10-26T01:42:10.414294+0000","last_active":"2023-10-26T01:42:10.414596+0000","last_peered":"2023-10-26T01:42:10.414596+0000","last_clean":"2023-10-26T01:42:10.414596+0000","last_became_active":"2023-10-26T01:42:10.413979+0000","last_became_peered":"2023-10-26T01:42:10.413979+0000","last_unstale":"2023-10-26T01:42:10.414596+0000","last_undegraded":"2023-10-26T01:42:10.414596+0000","last_fullsized":"2023-10-26T01:42:10.414596+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:13:55.827244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.413899+0000","last_change":"2023-10-26T01:42:10.413817+0000","last_active":"2023-10-26T01:42:10.413899+0000","last_peered":"2023-10-26T01:42:10.413899+0000","last_clean":"2023-10-26T01:42:10.413899+0000","last_became_active":"2023-10-26T01:42:10.412562+0000","last_became_peered":"2023-10-26T01:42:10.412562+0000","last_unstale":"2023-10-26T01:42:10.413899+0000","last_undegraded":"2023-10-26T01:42:10.413899+0000","last_fullsized":"2023-10-26T01:42:10.413899+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:13:41.517039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.412170+0000","last_change":"2023-10-26T01:42:10.411551+0000","last_active":"2023-10-26T01:42:10.412170+0000","last_peered":"2023-10-26T01:42:10.412170+0000","last_clean":"2023-10-26T01:42:10.412170+0000","last_became_active":"2023-10-26T01:42:10.411310+0000","last_became_peered":"2023-10-26T01:42:10.411310+0000","last_unstale":"2023-10-26T01:42:10.412170+0000","last_undegraded":"2023-10-26T01:42:10.412170+0000","last_fullsized":"2023-10-26T01:42:10.412170+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:18:20.465446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.413991+0000","last_change":"2023-10-26T01:42:10.413912+0000","last_active":"2023-10-26T01:42:10.413991+0000","last_peered":"2023-10-26T01:42:10.413991+0000","last_clean":"2023-10-26T01:42:10.413991+0000","last_became_active":"2023-10-26T01:42:10.412939+0000","last_became_peered":"2023-10-26T01:42:10.412939+0000","last_unstale":"2023-10-26T01:42:10.413991+0000","last_undegraded":"2023-10-26T01:42:10.413991+0000","last_fullsized":"2023-10-26T01:42:10.413991+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:53:53.673059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":11,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-26T01:42:11.913515+0000","last_change":"2023-10-26T01:42:10.392593+0000","last_active":"2023-10-26T01:42:11.913515+0000","last_peered":"2023-10-26T01:42:11.913515+0000","last_clean":"2023-10-26T01:42:11.913515+0000","last_became_active":"2023-10-26T01:42:10.392343+0000","last_became_peered":"2023-10-26T01:42:10.392343+0000","last_unstale":"2023-10-26T01:42:11.913515+0000","last_undegraded":"2023-10-26T01:42:11.913515+0000","last_fullsized":"2023-10-26T01:42:11.913515+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:07:00.552083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.415480+0000","last_change":"2023-10-26T01:42:10.414646+0000","last_active":"2023-10-26T01:42:10.415480+0000","last_peered":"2023-10-26T01:42:10.415480+0000","last_clean":"2023-10-26T01:42:10.415480+0000","last_became_active":"2023-10-26T01:42:10.414039+0000","last_became_peered":"2023-10-26T01:42:10.414039+0000","last_unstale":"2023-10-26T01:42:10.415480+0000","last_undegraded":"2023-10-26T01:42:10.415480+0000","last_fullsized":"2023-10-26T01:42:10.415480+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:34:02.113140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.409468+0000","last_change":"2023-10-26T01:42:10.409369+0000","last_active":"2023-10-26T01:42:10.409468+0000","last_peered":"2023-10-26T01:42:10.409468+0000","last_clean":"2023-10-26T01:42:10.409468+0000","last_became_active":"2023-10-26T01:42:10.408982+0000","last_became_peered":"2023-10-26T01:42:10.408982+0000","last_unstale":"2023-10-26T01:42:10.409468+0000","last_undegraded":"2023-10-26T01:42:10.409468+0000","last_fullsized":"2023-10-26T01:42:10.409468+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:25:03.319950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.404213+0000","last_change":"2023-10-26T01:42:10.402771+0000","last_active":"2023-10-26T01:42:10.404213+0000","last_peered":"2023-10-26T01:42:10.404213+0000","last_clean":"2023-10-26T01:42:10.404213+0000","last_became_active":"2023-10-26T01:42:10.402145+0000","last_became_peered":"2023-10-26T01:42:10.402145+0000","last_unstale":"2023-10-26T01:42:10.404213+0000","last_undegraded":"2023-10-26T01:42:10.404213+0000","last_fullsized":"2023-10-26T01:42:10.404213+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:07:59.675904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.412071+0000","last_change":"2023-10-26T01:42:10.412000+0000","last_active":"2023-10-26T01:42:10.412071+0000","last_peered":"2023-10-26T01:42:10.412071+0000","last_clean":"2023-10-26T01:42:10.412071+0000","last_became_active":"2023-10-26T01:42:10.409415+0000","last_became_peered":"2023-10-26T01:42:10.409415+0000","last_unstale":"2023-10-26T01:42:10.412071+0000","last_undegraded":"2023-10-26T01:42:10.412071+0000","last_fullsized":"2023-10-26T01:42:10.412071+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:35:14.751935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.413394+0000","last_change":"2023-10-26T01:42:10.413303+0000","last_active":"2023-10-26T01:42:10.413394+0000","last_peered":"2023-10-26T01:42:10.413394+0000","last_clean":"2023-10-26T01:42:10.413394+0000","last_became_active":"2023-10-26T01:42:10.413039+0000","last_became_peered":"2023-10-26T01:42:10.413039+0000","last_unstale":"2023-10-26T01:42:10.413394+0000","last_undegraded":"2023-10-26T01:42:10.413394+0000","last_fullsized":"2023-10-26T01:42:10.413394+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:18:51.873615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.414532+0000","last_change":"2023-10-26T01:42:10.414266+0000","last_active":"2023-10-26T01:42:10.414532+0000","last_peered":"2023-10-26T01:42:10.414532+0000","last_clean":"2023-10-26T01:42:10.414532+0000","last_became_active":"2023-10-26T01:42:10.413243+0000","last_became_peered":"2023-10-26T01:42:10.413243+0000","last_unstale":"2023-10-26T01:42:10.414532+0000","last_undegraded":"2023-10-26T01:42:10.414532+0000","last_fullsized":"2023-10-26T01:42:10.414532+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:33:35.047366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.407349+0000","last_change":"2023-10-26T01:42:10.407281+0000","last_active":"2023-10-26T01:42:10.407349+0000","last_peered":"2023-10-26T01:42:10.407349+0000","last_clean":"2023-10-26T01:42:10.407349+0000","last_became_active":"2023-10-26T01:42:10.406994+0000","last_became_peered":"2023-10-26T01:42:10.406994+0000","last_unstale":"2023-10-26T01:42:10.407349+0000","last_undegraded":"2023-10-26T01:42:10.407349+0000","last_fullsized":"2023-10-26T01:42:10.407349+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:06:28.342273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.404232+0000","last_change":"2023-10-26T01:42:10.403011+0000","last_active":"2023-10-26T01:42:10.404232+0000","last_peered":"2023-10-26T01:42:10.404232+0000","last_clean":"2023-10-26T01:42:10.404232+0000","last_became_active":"2023-10-26T01:42:10.402510+0000","last_became_peered":"2023-10-26T01:42:10.402510+0000","last_unstale":"2023-10-26T01:42:10.404232+0000","last_undegraded":"2023-10-26T01:42:10.404232+0000","last_fullsized":"2023-10-26T01:42:10.404232+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:54:26.742574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.407451+0000","last_change":"2023-10-26T01:42:10.406780+0000","last_active":"2023-10-26T01:42:10.407451+0000","last_peered":"2023-10-26T01:42:10.407451+0000","last_clean":"2023-10-26T01:42:10.407451+0000","last_became_active":"2023-10-26T01:42:10.405395+0000","last_became_peered":"2023-10-26T01:42:10.405395+0000","last_unstale":"2023-10-26T01:42:10.407451+0000","last_undegraded":"2023-10-26T01:42:10.407451+0000","last_fullsized":"2023-10-26T01:42:10.407451+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:49:08.475248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.405126+0000","last_change":"2023-10-26T01:42:10.405058+0000","last_active":"2023-10-26T01:42:10.405126+0000","last_peered":"2023-10-26T01:42:10.405126+0000","last_clean":"2023-10-26T01:42:10.405126+0000","last_became_active":"2023-10-26T01:42:10.404775+0000","last_became_peered":"2023-10-26T01:42:10.404775+0000","last_unstale":"2023-10-26T01:42:10.405126+0000","last_undegraded":"2023-10-26T01:42:10.405126+0000","last_fullsized":"2023-10-26T01:42:10.405126+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:14:04.052827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.409374+0000","last_change":"2023-10-26T01:42:10.408742+0000","last_active":"2023-10-26T01:42:10.409374+0000","last_peered":"2023-10-26T01:42:10.409374+0000","last_clean":"2023-10-26T01:42:10.409374+0000","last_became_active":"2023-10-26T01:42:10.408480+0000","last_became_peered":"2023-10-26T01:42:10.408480+0000","last_unstale":"2023-10-26T01:42:10.409374+0000","last_undegraded":"2023-10-26T01:42:10.409374+0000","last_fullsized":"2023-10-26T01:42:10.409374+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:06:42.077030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.418147+0000","last_change":"2023-10-26T01:42:10.418040+0000","last_active":"2023-10-26T01:42:10.418147+0000","last_peered":"2023-10-26T01:42:10.418147+0000","last_clean":"2023-10-26T01:42:10.418147+0000","last_became_active":"2023-10-26T01:42:10.417382+0000","last_became_peered":"2023-10-26T01:42:10.417382+0000","last_unstale":"2023-10-26T01:42:10.418147+0000","last_undegraded":"2023-10-26T01:42:10.418147+0000","last_fullsized":"2023-10-26T01:42:10.418147+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:32:44.180501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.409495+0000","last_change":"2023-10-26T01:42:10.409084+0000","last_active":"2023-10-26T01:42:10.409495+0000","last_peered":"2023-10-26T01:42:10.409495+0000","last_clean":"2023-10-26T01:42:10.409495+0000","last_became_active":"2023-10-26T01:42:10.407811+0000","last_became_peered":"2023-10-26T01:42:10.407811+0000","last_unstale":"2023-10-26T01:42:10.409495+0000","last_undegraded":"2023-10-26T01:42:10.409495+0000","last_fullsized":"2023-10-26T01:42:10.409495+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:15:48.914492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.404658+0000","last_change":"2023-10-26T01:42:10.400644+0000","last_active":"2023-10-26T01:42:10.404658+0000","last_peered":"2023-10-26T01:42:10.404658+0000","last_clean":"2023-10-26T01:42:10.404658+0000","last_became_active":"2023-10-26T01:42:10.400297+0000","last_became_peered":"2023-10-26T01:42:10.400297+0000","last_unstale":"2023-10-26T01:42:10.404658+0000","last_undegraded":"2023-10-26T01:42:10.404658+0000","last_fullsized":"2023-10-26T01:42:10.404658+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:42:11.459436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.412492+0000","last_change":"2023-10-26T01:42:10.412094+0000","last_active":"2023-10-26T01:42:10.412492+0000","last_peered":"2023-10-26T01:42:10.412492+0000","last_clean":"2023-10-26T01:42:10.412492+0000","last_became_active":"2023-10-26T01:42:10.411765+0000","last_became_peered":"2023-10-26T01:42:10.411765+0000","last_unstale":"2023-10-26T01:42:10.412492+0000","last_undegraded":"2023-10-26T01:42:10.412492+0000","last_fullsized":"2023-10-26T01:42:10.412492+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:57:16.014827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.404491+0000","last_change":"2023-10-26T01:42:10.404414+0000","last_active":"2023-10-26T01:42:10.404491+0000","last_peered":"2023-10-26T01:42:10.404491+0000","last_clean":"2023-10-26T01:42:10.404491+0000","last_became_active":"2023-10-26T01:42:10.404167+0000","last_became_peered":"2023-10-26T01:42:10.404167+0000","last_unstale":"2023-10-26T01:42:10.404491+0000","last_undegraded":"2023-10-26T01:42:10.404491+0000","last_fullsized":"2023-10-26T01:42:10.404491+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:13:55.827244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.409430+0000","last_change":"2023-10-26T01:42:10.405918+0000","last_active":"2023-10-26T01:42:10.409430+0000","last_peered":"2023-10-26T01:42:10.409430+0000","last_clean":"2023-10-26T01:42:10.409430+0000","last_became_active":"2023-10-26T01:42:10.405198+0000","last_became_peered":"2023-10-26T01:42:10.405198+0000","last_unstale":"2023-10-26T01:42:10.409430+0000","last_undegraded":"2023-10-26T01:42:10.409430+0000","last_fullsized":"2023-10-26T01:42:10.409430+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:44:23.763936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.405342+0000","last_change":"2023-10-26T01:42:10.404482+0000","last_active":"2023-10-26T01:42:10.405342+0000","last_peered":"2023-10-26T01:42:10.405342+0000","last_clean":"2023-10-26T01:42:10.405342+0000","last_became_active":"2023-10-26T01:42:10.404133+0000","last_became_peered":"2023-10-26T01:42:10.404133+0000","last_unstale":"2023-10-26T01:42:10.405342+0000","last_undegraded":"2023-10-26T01:42:10.405342+0000","last_fullsized":"2023-10-26T01:42:10.405342+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:14:55.061732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.405927+0000","last_change":"2023-10-26T01:42:10.405213+0000","last_active":"2023-10-26T01:42:10.405927+0000","last_peered":"2023-10-26T01:42:10.405927+0000","last_clean":"2023-10-26T01:42:10.405927+0000","last_became_active":"2023-10-26T01:42:10.404895+0000","last_became_peered":"2023-10-26T01:42:10.404895+0000","last_unstale":"2023-10-26T01:42:10.405927+0000","last_undegraded":"2023-10-26T01:42:10.405927+0000","last_fullsized":"2023-10-26T01:42:10.405927+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:32:44.180501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.403780+0000","last_change":"2023-10-26T01:42:10.402030+0000","last_active":"2023-10-26T01:42:10.403780+0000","last_peered":"2023-10-26T01:42:10.403780+0000","last_clean":"2023-10-26T01:42:10.403780+0000","last_became_active":"2023-10-26T01:42:10.401794+0000","last_became_peered":"2023-10-26T01:42:10.401794+0000","last_unstale":"2023-10-26T01:42:10.403780+0000","last_undegraded":"2023-10-26T01:42:10.403780+0000","last_fullsized":"2023-10-26T01:42:10.403780+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:37:10.931971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.404685+0000","last_change":"2023-10-26T01:42:10.401647+0000","last_active":"2023-10-26T01:42:10.404685+0000","last_peered":"2023-10-26T01:42:10.404685+0000","last_clean":"2023-10-26T01:42:10.404685+0000","last_became_active":"2023-10-26T01:42:10.398276+0000","last_became_peered":"2023-10-26T01:42:10.398276+0000","last_unstale":"2023-10-26T01:42:10.404685+0000","last_undegraded":"2023-10-26T01:42:10.404685+0000","last_fullsized":"2023-10-26T01:42:10.404685+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T01:59:29.991970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":11,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-26T01:42:11.914521+0000","last_change":"2023-10-26T01:42:10.406731+0000","last_active":"2023-10-26T01:42:11.914521+0000","last_peered":"2023-10-26T01:42:11.914521+0000","last_clean":"2023-10-26T01:42:11.914521+0000","last_became_active":"2023-10-26T01:42:10.406419+0000","last_became_peered":"2023-10-26T01:42:10.406419+0000","last_unstale":"2023-10-26T01:42:11.914521+0000","last_undegraded":"2023-10-26T01:42:11.914521+0000","last_fullsized":"2023-10-26T01:42:11.914521+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:45:52.045328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.402335+0000","last_change":"2023-10-26T01:42:10.402260+0000","last_active":"2023-10-26T01:42:10.402335+0000","last_peered":"2023-10-26T01:42:10.402335+0000","last_clean":"2023-10-26T01:42:10.402335+0000","last_became_active":"2023-10-26T01:42:10.402034+0000","last_became_peered":"2023-10-26T01:42:10.402034+0000","last_unstale":"2023-10-26T01:42:10.402335+0000","last_undegraded":"2023-10-26T01:42:10.402335+0000","last_fullsized":"2023-10-26T01:42:10.402335+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:32:44.631370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.404756+0000","last_change":"2023-10-26T01:42:10.404673+0000","last_active":"2023-10-26T01:42:10.404756+0000","last_peered":"2023-10-26T01:42:10.404756+0000","last_clean":"2023-10-26T01:42:10.404756+0000","last_became_active":"2023-10-26T01:42:10.404309+0000","last_became_peered":"2023-10-26T01:42:10.404309+0000","last_unstale":"2023-10-26T01:42:10.404756+0000","last_undegraded":"2023-10-26T01:42:10.404756+0000","last_fullsized":"2023-10-26T01:42:10.404756+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:39:44.357515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1f","version":"165'4","reported_seq":15,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.420424+0000","last_change":"2023-10-26T01:42:09.405065+0000","last_active":"2023-10-26T01:42:10.420424+0000","last_peered":"2023-10-26T01:42:10.420424+0000","last_clean":"2023-10-26T01:42:10.420424+0000","last_became_active":"2023-10-26T01:42:09.404606+0000","last_became_peered":"2023-10-26T01:42:09.404606+0000","last_unstale":"2023-10-26T01:42:10.420424+0000","last_undegraded":"2023-10-26T01:42:10.420424+0000","last_fullsized":"2023-10-26T01:42:10.420424+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_clean_scrub_stamp":"2023-10-26T01:42:08.349732+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:53:10.091790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":11,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-26T01:42:11.914096+0000","last_change":"2023-10-26T01:42:10.392775+0000","last_active":"2023-10-26T01:42:11.914096+0000","last_peered":"2023-10-26T01:42:11.914096+0000","last_clean":"2023-10-26T01:42:11.914096+0000","last_became_active":"2023-10-26T01:42:10.392525+0000","last_became_peered":"2023-10-26T01:42:10.392525+0000","last_unstale":"2023-10-26T01:42:11.914096+0000","last_undegraded":"2023-10-26T01:42:11.914096+0000","last_fullsized":"2023-10-26T01:42:11.914096+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:42:11.459436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"165'1","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-26T01:42:11.913787+0000","last_change":"2023-10-26T01:42:09.402755+0000","last_active":"2023-10-26T01:42:11.913787+0000","last_peered":"2023-10-26T01:42:11.913787+0000","last_clean":"2023-10-26T01:42:11.913787+0000","last_became_active":"2023-10-26T01:42:09.402120+0000","last_became_peered":"2023-10-26T01:42:09.402120+0000","last_unstale":"2023-10-26T01:42:11.913787+0000","last_undegraded":"2023-10-26T01:42:11.913787+0000","last_fullsized":"2023-10-26T01:42:11.913787+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_clean_scrub_stamp":"2023-10-26T01:42:08.349732+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:13:26.673715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.413549+0000","last_change":"2023-10-26T01:42:10.412901+0000","last_active":"2023-10-26T01:42:10.413549+0000","last_peered":"2023-10-26T01:42:10.413549+0000","last_clean":"2023-10-26T01:42:10.413549+0000","last_became_active":"2023-10-26T01:42:10.411251+0000","last_became_peered":"2023-10-26T01:42:10.411251+0000","last_unstale":"2023-10-26T01:42:10.413549+0000","last_undegraded":"2023-10-26T01:42:10.413549+0000","last_fullsized":"2023-10-26T01:42:10.413549+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:23:53.405275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1d","version":"165'2","reported_seq":13,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.415861+0000","last_change":"2023-10-26T01:42:09.393430+0000","last_active":"2023-10-26T01:42:10.415861+0000","last_peered":"2023-10-26T01:42:10.415861+0000","last_clean":"2023-10-26T01:42:10.415861+0000","last_became_active":"2023-10-26T01:42:09.392624+0000","last_became_peered":"2023-10-26T01:42:09.392624+0000","last_unstale":"2023-10-26T01:42:10.415861+0000","last_undegraded":"2023-10-26T01:42:10.415861+0000","last_fullsized":"2023-10-26T01:42:10.415861+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_clean_scrub_stamp":"2023-10-26T01:42:08.349732+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:57:15.003296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.413645+0000","last_change":"2023-10-26T01:42:10.413497+0000","last_active":"2023-10-26T01:42:10.413645+0000","last_peered":"2023-10-26T01:42:10.413645+0000","last_clean":"2023-10-26T01:42:10.413645+0000","last_became_active":"2023-10-26T01:42:10.410617+0000","last_became_peered":"2023-10-26T01:42:10.410617+0000","last_unstale":"2023-10-26T01:42:10.413645+0000","last_undegraded":"2023-10-26T01:42:10.413645+0000","last_fullsized":"2023-10-26T01:42:10.413645+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:52:56.148787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"165'2","reported_seq":13,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.416536+0000","last_change":"2023-10-26T01:42:09.394789+0000","last_active":"2023-10-26T01:42:10.416536+0000","last_peered":"2023-10-26T01:42:10.416536+0000","last_clean":"2023-10-26T01:42:10.416536+0000","last_became_active":"2023-10-26T01:42:09.389841+0000","last_became_peered":"2023-10-26T01:42:09.389841+0000","last_unstale":"2023-10-26T01:42:10.416536+0000","last_undegraded":"2023-10-26T01:42:10.416536+0000","last_fullsized":"2023-10-26T01:42:10.416536+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_clean_scrub_stamp":"2023-10-26T01:42:08.349732+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:31:32.863018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":11,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-26T01:42:11.914434+0000","last_change":"2023-10-26T01:42:10.407075+0000","last_active":"2023-10-26T01:42:11.914434+0000","last_peered":"2023-10-26T01:42:11.914434+0000","last_clean":"2023-10-26T01:42:11.914434+0000","last_became_active":"2023-10-26T01:42:10.404574+0000","last_became_peered":"2023-10-26T01:42:10.404574+0000","last_unstale":"2023-10-26T01:42:11.914434+0000","last_undegraded":"2023-10-26T01:42:11.914434+0000","last_fullsized":"2023-10-26T01:42:11.914434+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:15:48.914492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1b","version":"165'1","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.417499+0000","last_change":"2023-10-26T01:42:09.405617+0000","last_active":"2023-10-26T01:42:10.417499+0000","last_peered":"2023-10-26T01:42:10.417499+0000","last_clean":"2023-10-26T01:42:10.417499+0000","last_became_active":"2023-10-26T01:42:09.405323+0000","last_became_peered":"2023-10-26T01:42:09.405323+0000","last_unstale":"2023-10-26T01:42:10.417499+0000","last_undegraded":"2023-10-26T01:42:10.417499+0000","last_fullsized":"2023-10-26T01:42:10.417499+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_clean_scrub_stamp":"2023-10-26T01:42:08.349732+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:42:15.440450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.410268+0000","last_change":"2023-10-26T01:42:10.410187+0000","last_active":"2023-10-26T01:42:10.410268+0000","last_peered":"2023-10-26T01:42:10.410268+0000","last_clean":"2023-10-26T01:42:10.410268+0000","last_became_active":"2023-10-26T01:42:10.408727+0000","last_became_peered":"2023-10-26T01:42:10.408727+0000","last_unstale":"2023-10-26T01:42:10.410268+0000","last_undegraded":"2023-10-26T01:42:10.410268+0000","last_fullsized":"2023-10-26T01:42:10.410268+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:09:20.791203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1a","version":"165'1","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.416333+0000","last_change":"2023-10-26T01:42:09.404429+0000","last_active":"2023-10-26T01:42:10.416333+0000","last_peered":"2023-10-26T01:42:10.416333+0000","last_clean":"2023-10-26T01:42:10.416333+0000","last_became_active":"2023-10-26T01:42:09.403818+0000","last_became_peered":"2023-10-26T01:42:09.403818+0000","last_unstale":"2023-10-26T01:42:10.416333+0000","last_undegraded":"2023-10-26T01:42:10.416333+0000","last_fullsized":"2023-10-26T01:42:10.416333+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_clean_scrub_stamp":"2023-10-26T01:42:08.349732+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:06:41.065499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":11,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-26T01:42:11.916564+0000","last_change":"2023-10-26T01:42:10.405535+0000","last_active":"2023-10-26T01:42:11.916564+0000","last_peered":"2023-10-26T01:42:11.916564+0000","last_clean":"2023-10-26T01:42:11.916564+0000","last_became_active":"2023-10-26T01:42:10.405239+0000","last_became_peered":"2023-10-26T01:42:10.405239+0000","last_unstale":"2023-10-26T01:42:11.916564+0000","last_undegraded":"2023-10-26T01:42:11.916564+0000","last_fullsized":"2023-10-26T01:42:11.916564+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:09:20.791203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":11,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.402971+0000","last_change":"2023-10-26T01:42:09.403856+0000","last_active":"2023-10-26T01:42:10.402971+0000","last_peered":"2023-10-26T01:42:10.402971+0000","last_clean":"2023-10-26T01:42:10.402971+0000","last_became_active":"2023-10-26T01:42:09.403458+0000","last_became_peered":"2023-10-26T01:42:09.403458+0000","last_unstale":"2023-10-26T01:42:10.402971+0000","last_undegraded":"2023-10-26T01:42:10.402971+0000","last_fullsized":"2023-10-26T01:42:10.402971+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_clean_scrub_stamp":"2023-10-26T01:42:08.349732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:58:33.483761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.412150+0000","last_change":"2023-10-26T01:42:10.411361+0000","last_active":"2023-10-26T01:42:10.412150+0000","last_peered":"2023-10-26T01:42:10.412150+0000","last_clean":"2023-10-26T01:42:10.412150+0000","last_became_active":"2023-10-26T01:42:10.406491+0000","last_became_peered":"2023-10-26T01:42:10.406491+0000","last_unstale":"2023-10-26T01:42:10.412150+0000","last_undegraded":"2023-10-26T01:42:10.412150+0000","last_fullsized":"2023-10-26T01:42:10.412150+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:56:34.881338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":11,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.394545+0000","last_change":"2023-10-26T01:42:09.405052+0000","last_active":"2023-10-26T01:42:10.394545+0000","last_peered":"2023-10-26T01:42:10.394545+0000","last_clean":"2023-10-26T01:42:10.394545+0000","last_became_active":"2023-10-26T01:42:09.404816+0000","last_became_peered":"2023-10-26T01:42:09.404816+0000","last_unstale":"2023-10-26T01:42:10.394545+0000","last_undegraded":"2023-10-26T01:42:10.394545+0000","last_fullsized":"2023-10-26T01:42:10.394545+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_clean_scrub_stamp":"2023-10-26T01:42:08.349732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:39:43.345984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.405063+0000","last_change":"2023-10-26T01:42:10.402404+0000","last_active":"2023-10-26T01:42:10.405063+0000","last_peered":"2023-10-26T01:42:10.405063+0000","last_clean":"2023-10-26T01:42:10.405063+0000","last_became_active":"2023-10-26T01:42:10.402174+0000","last_became_peered":"2023-10-26T01:42:10.402174+0000","last_unstale":"2023-10-26T01:42:10.405063+0000","last_undegraded":"2023-10-26T01:42:10.405063+0000","last_fullsized":"2023-10-26T01:42:10.405063+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:50:16.014300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":12,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-26T01:42:11.914775+0000","last_change":"2023-10-26T01:42:09.398301+0000","last_active":"2023-10-26T01:42:11.914775+0000","last_peered":"2023-10-26T01:42:11.914775+0000","last_clean":"2023-10-26T01:42:11.914775+0000","last_became_active":"2023-10-26T01:42:09.394569+0000","last_became_peered":"2023-10-26T01:42:09.394569+0000","last_unstale":"2023-10-26T01:42:11.914775+0000","last_undegraded":"2023-10-26T01:42:11.914775+0000","last_fullsized":"2023-10-26T01:42:11.914775+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_clean_scrub_stamp":"2023-10-26T01:42:08.349732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:15:18.889535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.417956+0000","last_change":"2023-10-26T01:42:10.417854+0000","last_active":"2023-10-26T01:42:10.417956+0000","last_peered":"2023-10-26T01:42:10.417956+0000","last_clean":"2023-10-26T01:42:10.417956+0000","last_became_active":"2023-10-26T01:42:10.417051+0000","last_became_peered":"2023-10-26T01:42:10.417051+0000","last_unstale":"2023-10-26T01:42:10.417956+0000","last_undegraded":"2023-10-26T01:42:10.417956+0000","last_fullsized":"2023-10-26T01:42:10.417956+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:59:00.597519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":11,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.394359+0000","last_change":"2023-10-26T01:42:09.405067+0000","last_active":"2023-10-26T01:42:10.394359+0000","last_peered":"2023-10-26T01:42:10.394359+0000","last_clean":"2023-10-26T01:42:10.394359+0000","last_became_active":"2023-10-26T01:42:09.404699+0000","last_became_peered":"2023-10-26T01:42:09.404699+0000","last_unstale":"2023-10-26T01:42:10.394359+0000","last_undegraded":"2023-10-26T01:42:10.394359+0000","last_fullsized":"2023-10-26T01:42:10.394359+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_clean_scrub_stamp":"2023-10-26T01:42:08.349732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:14:35.577148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.403154+0000","last_change":"2023-10-26T01:42:10.403079+0000","last_active":"2023-10-26T01:42:10.403154+0000","last_peered":"2023-10-26T01:42:10.403154+0000","last_clean":"2023-10-26T01:42:10.403154+0000","last_became_active":"2023-10-26T01:42:10.402817+0000","last_became_peered":"2023-10-26T01:42:10.402817+0000","last_unstale":"2023-10-26T01:42:10.403154+0000","last_undegraded":"2023-10-26T01:42:10.403154+0000","last_fullsized":"2023-10-26T01:42:10.403154+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:49:22.193228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.14","version":"165'6","reported_seq":17,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.419144+0000","last_change":"2023-10-26T01:42:09.398071+0000","last_active":"2023-10-26T01:42:10.419144+0000","last_peered":"2023-10-26T01:42:10.419144+0000","last_clean":"2023-10-26T01:42:10.419144+0000","last_became_active":"2023-10-26T01:42:09.397653+0000","last_became_peered":"2023-10-26T01:42:09.397653+0000","last_unstale":"2023-10-26T01:42:10.419144+0000","last_undegraded":"2023-10-26T01:42:10.419144+0000","last_fullsized":"2023-10-26T01:42:10.419144+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_clean_scrub_stamp":"2023-10-26T01:42:08.349732+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:25:44.055005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.409947+0000","last_change":"2023-10-26T01:42:10.409834+0000","last_active":"2023-10-26T01:42:10.409947+0000","last_peered":"2023-10-26T01:42:10.409947+0000","last_clean":"2023-10-26T01:42:10.409947+0000","last_became_active":"2023-10-26T01:42:10.409211+0000","last_became_peered":"2023-10-26T01:42:10.409211+0000","last_unstale":"2023-10-26T01:42:10.409947+0000","last_undegraded":"2023-10-26T01:42:10.409947+0000","last_fullsized":"2023-10-26T01:42:10.409947+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:01:03.396910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.13","version":"165'2","reported_seq":13,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.411134+0000","last_change":"2023-10-26T01:42:09.398259+0000","last_active":"2023-10-26T01:42:10.411134+0000","last_peered":"2023-10-26T01:42:10.411134+0000","last_clean":"2023-10-26T01:42:10.411134+0000","last_became_active":"2023-10-26T01:42:09.397928+0000","last_became_peered":"2023-10-26T01:42:09.397928+0000","last_unstale":"2023-10-26T01:42:10.411134+0000","last_undegraded":"2023-10-26T01:42:10.411134+0000","last_fullsized":"2023-10-26T01:42:10.411134+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_clean_scrub_stamp":"2023-10-26T01:42:08.349732+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:21:58.503627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":11,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-26T01:42:11.915918+0000","last_change":"2023-10-26T01:42:10.393830+0000","last_active":"2023-10-26T01:42:11.915918+0000","last_peered":"2023-10-26T01:42:11.915918+0000","last_clean":"2023-10-26T01:42:11.915918+0000","last_became_active":"2023-10-26T01:42:10.393326+0000","last_became_peered":"2023-10-26T01:42:10.393326+0000","last_unstale":"2023-10-26T01:42:11.915918+0000","last_undegraded":"2023-10-26T01:42:11.915918+0000","last_fullsized":"2023-10-26T01:42:11.915918+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:44:45.588628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":11,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.391177+0000","last_change":"2023-10-26T01:42:09.402031+0000","last_active":"2023-10-26T01:42:10.391177+0000","last_peered":"2023-10-26T01:42:10.391177+0000","last_clean":"2023-10-26T01:42:10.391177+0000","last_became_active":"2023-10-26T01:42:09.401444+0000","last_became_peered":"2023-10-26T01:42:09.401444+0000","last_unstale":"2023-10-26T01:42:10.391177+0000","last_undegraded":"2023-10-26T01:42:10.391177+0000","last_fullsized":"2023-10-26T01:42:10.391177+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_clean_scrub_stamp":"2023-10-26T01:42:08.349732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:13:26.673715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.413352+0000","last_change":"2023-10-26T01:42:10.413105+0000","last_active":"2023-10-26T01:42:10.413352+0000","last_peered":"2023-10-26T01:42:10.413352+0000","last_clean":"2023-10-26T01:42:10.413352+0000","last_became_active":"2023-10-26T01:42:10.409733+0000","last_became_peered":"2023-10-26T01:42:10.409733+0000","last_unstale":"2023-10-26T01:42:10.413352+0000","last_undegraded":"2023-10-26T01:42:10.413352+0000","last_fullsized":"2023-10-26T01:42:10.413352+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:35:44.412930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":11,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.392695+0000","last_change":"2023-10-26T01:42:09.389251+0000","last_active":"2023-10-26T01:42:10.392695+0000","last_peered":"2023-10-26T01:42:10.392695+0000","last_clean":"2023-10-26T01:42:10.392695+0000","last_became_active":"2023-10-26T01:42:09.388927+0000","last_became_peered":"2023-10-26T01:42:09.388927+0000","last_unstale":"2023-10-26T01:42:10.392695+0000","last_undegraded":"2023-10-26T01:42:10.392695+0000","last_fullsized":"2023-10-26T01:42:10.392695+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_clean_scrub_stamp":"2023-10-26T01:42:08.349732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:49:04.947399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.403260+0000","last_change":"2023-10-26T01:42:10.403097+0000","last_active":"2023-10-26T01:42:10.403260+0000","last_peered":"2023-10-26T01:42:10.403260+0000","last_clean":"2023-10-26T01:42:10.403260+0000","last_became_active":"2023-10-26T01:42:10.402206+0000","last_became_peered":"2023-10-26T01:42:10.402206+0000","last_unstale":"2023-10-26T01:42:10.403260+0000","last_undegraded":"2023-10-26T01:42:10.403260+0000","last_fullsized":"2023-10-26T01:42:10.403260+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:01:33.828985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"165'8","reported_seq":19,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.417426+0000","last_change":"2023-10-26T01:42:09.408422+0000","last_active":"2023-10-26T01:42:10.417426+0000","last_peered":"2023-10-26T01:42:10.417426+0000","last_clean":"2023-10-26T01:42:10.417426+0000","last_became_active":"2023-10-26T01:42:09.408022+0000","last_became_peered":"2023-10-26T01:42:09.408022+0000","last_unstale":"2023-10-26T01:42:10.417426+0000","last_undegraded":"2023-10-26T01:42:10.417426+0000","last_fullsized":"2023-10-26T01:42:10.417426+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_clean_scrub_stamp":"2023-10-26T01:42:08.349732+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:07:20.903387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.412585+0000","last_change":"2023-10-26T01:42:10.410896+0000","last_active":"2023-10-26T01:42:10.412585+0000","last_peered":"2023-10-26T01:42:10.412585+0000","last_clean":"2023-10-26T01:42:10.412585+0000","last_became_active":"2023-10-26T01:42:10.410191+0000","last_became_peered":"2023-10-26T01:42:10.410191+0000","last_unstale":"2023-10-26T01:42:10.412585+0000","last_undegraded":"2023-10-26T01:42:10.412585+0000","last_fullsized":"2023-10-26T01:42:10.412585+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:18:20.465446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"165'1","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.416450+0000","last_change":"2023-10-26T01:42:09.400503+0000","last_active":"2023-10-26T01:42:10.416450+0000","last_peered":"2023-10-26T01:42:10.416450+0000","last_clean":"2023-10-26T01:42:10.416450+0000","last_became_active":"2023-10-26T01:42:09.397318+0000","last_became_peered":"2023-10-26T01:42:09.397318+0000","last_unstale":"2023-10-26T01:42:10.416450+0000","last_undegraded":"2023-10-26T01:42:10.416450+0000","last_fullsized":"2023-10-26T01:42:10.416450+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_clean_scrub_stamp":"2023-10-26T01:42:08.349732+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:09:40.041614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.413458+0000","last_change":"2023-10-26T01:42:10.412058+0000","last_active":"2023-10-26T01:42:10.413458+0000","last_peered":"2023-10-26T01:42:10.413458+0000","last_clean":"2023-10-26T01:42:10.413458+0000","last_became_active":"2023-10-26T01:42:10.411703+0000","last_became_peered":"2023-10-26T01:42:10.411703+0000","last_unstale":"2023-10-26T01:42:10.413458+0000","last_undegraded":"2023-10-26T01:42:10.413458+0000","last_fullsized":"2023-10-26T01:42:10.413458+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:19:17.877977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"165'2","reported_seq":14,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-26T01:42:11.916832+0000","last_change":"2023-10-26T01:42:09.399534+0000","last_active":"2023-10-26T01:42:11.916832+0000","last_peered":"2023-10-26T01:42:11.916832+0000","last_clean":"2023-10-26T01:42:11.916832+0000","last_became_active":"2023-10-26T01:42:09.399276+0000","last_became_peered":"2023-10-26T01:42:09.399276+0000","last_unstale":"2023-10-26T01:42:11.916832+0000","last_undegraded":"2023-10-26T01:42:11.916832+0000","last_fullsized":"2023-10-26T01:42:11.916832+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_clean_scrub_stamp":"2023-10-26T01:42:08.349732+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:43:16.437117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.412871+0000","last_change":"2023-10-26T01:42:10.411633+0000","last_active":"2023-10-26T01:42:10.412871+0000","last_peered":"2023-10-26T01:42:10.412871+0000","last_clean":"2023-10-26T01:42:10.412871+0000","last_became_active":"2023-10-26T01:42:10.408927+0000","last_became_peered":"2023-10-26T01:42:10.408927+0000","last_unstale":"2023-10-26T01:42:10.412871+0000","last_undegraded":"2023-10-26T01:42:10.412871+0000","last_fullsized":"2023-10-26T01:42:10.412871+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:19:17.045067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.d","version":"165'1","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.418288+0000","last_change":"2023-10-26T01:42:09.401844+0000","last_active":"2023-10-26T01:42:10.418288+0000","last_peered":"2023-10-26T01:42:10.418288+0000","last_clean":"2023-10-26T01:42:10.418288+0000","last_became_active":"2023-10-26T01:42:09.401493+0000","last_became_peered":"2023-10-26T01:42:09.401493+0000","last_unstale":"2023-10-26T01:42:10.418288+0000","last_undegraded":"2023-10-26T01:42:10.418288+0000","last_fullsized":"2023-10-26T01:42:10.418288+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_clean_scrub_stamp":"2023-10-26T01:42:08.349732+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:47:22.909451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.409451+0000","last_change":"2023-10-26T01:42:10.409213+0000","last_active":"2023-10-26T01:42:10.409451+0000","last_peered":"2023-10-26T01:42:10.409451+0000","last_clean":"2023-10-26T01:42:10.409451+0000","last_became_active":"2023-10-26T01:42:10.408480+0000","last_became_peered":"2023-10-26T01:42:10.408480+0000","last_unstale":"2023-10-26T01:42:10.409451+0000","last_undegraded":"2023-10-26T01:42:10.409451+0000","last_fullsized":"2023-10-26T01:42:10.409451+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:00:02.415277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":12,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-26T01:42:11.914507+0000","last_change":"2023-10-26T01:42:09.397288+0000","last_active":"2023-10-26T01:42:11.914507+0000","last_peered":"2023-10-26T01:42:11.914507+0000","last_clean":"2023-10-26T01:42:11.914507+0000","last_became_active":"2023-10-26T01:42:09.397049+0000","last_became_peered":"2023-10-26T01:42:09.397049+0000","last_unstale":"2023-10-26T01:42:11.914507+0000","last_undegraded":"2023-10-26T01:42:11.914507+0000","last_fullsized":"2023-10-26T01:42:11.914507+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_clean_scrub_stamp":"2023-10-26T01:42:08.349732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:01:02.385379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.406071+0000","last_change":"2023-10-26T01:42:10.405363+0000","last_active":"2023-10-26T01:42:10.406071+0000","last_peered":"2023-10-26T01:42:10.406071+0000","last_clean":"2023-10-26T01:42:10.406071+0000","last_became_active":"2023-10-26T01:42:10.404945+0000","last_became_peered":"2023-10-26T01:42:10.404945+0000","last_unstale":"2023-10-26T01:42:10.406071+0000","last_undegraded":"2023-10-26T01:42:10.406071+0000","last_fullsized":"2023-10-26T01:42:10.406071+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:06:18.765851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.b","version":"165'2","reported_seq":13,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.413816+0000","last_change":"2023-10-26T01:42:09.408188+0000","last_active":"2023-10-26T01:42:10.413816+0000","last_peered":"2023-10-26T01:42:10.413816+0000","last_clean":"2023-10-26T01:42:10.413816+0000","last_became_active":"2023-10-26T01:42:09.407952+0000","last_became_peered":"2023-10-26T01:42:09.407952+0000","last_unstale":"2023-10-26T01:42:10.413816+0000","last_undegraded":"2023-10-26T01:42:10.413816+0000","last_fullsized":"2023-10-26T01:42:10.413816+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_clean_scrub_stamp":"2023-10-26T01:42:08.349732+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:03:16.727164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.401345+0000","last_change":"2023-10-26T01:42:10.400634+0000","last_active":"2023-10-26T01:42:10.401345+0000","last_peered":"2023-10-26T01:42:10.401345+0000","last_clean":"2023-10-26T01:42:10.401345+0000","last_became_active":"2023-10-26T01:42:10.400402+0000","last_became_peered":"2023-10-26T01:42:10.400402+0000","last_unstale":"2023-10-26T01:42:10.401345+0000","last_undegraded":"2023-10-26T01:42:10.401345+0000","last_fullsized":"2023-10-26T01:42:10.401345+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:59:00.597519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":11,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.399537+0000","last_change":"2023-10-26T01:42:09.403636+0000","last_active":"2023-10-26T01:42:10.399537+0000","last_peered":"2023-10-26T01:42:10.399537+0000","last_clean":"2023-10-26T01:42:10.399537+0000","last_became_active":"2023-10-26T01:42:09.403016+0000","last_became_peered":"2023-10-26T01:42:09.403016+0000","last_unstale":"2023-10-26T01:42:10.399537+0000","last_undegraded":"2023-10-26T01:42:10.399537+0000","last_fullsized":"2023-10-26T01:42:10.399537+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_clean_scrub_stamp":"2023-10-26T01:42:08.349732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:27:07.384772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.404626+0000","last_change":"2023-10-26T01:42:10.400073+0000","last_active":"2023-10-26T01:42:10.404626+0000","last_peered":"2023-10-26T01:42:10.404626+0000","last_clean":"2023-10-26T01:42:10.404626+0000","last_became_active":"2023-10-26T01:42:10.399836+0000","last_became_peered":"2023-10-26T01:42:10.399836+0000","last_unstale":"2023-10-26T01:42:10.404626+0000","last_undegraded":"2023-10-26T01:42:10.404626+0000","last_fullsized":"2023-10-26T01:42:10.404626+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:45:52.045328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":11,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.396335+0000","last_change":"2023-10-26T01:42:09.406759+0000","last_active":"2023-10-26T01:42:10.396335+0000","last_peered":"2023-10-26T01:42:10.396335+0000","last_clean":"2023-10-26T01:42:10.396335+0000","last_became_active":"2023-10-26T01:42:09.406519+0000","last_became_peered":"2023-10-26T01:42:09.406519+0000","last_unstale":"2023-10-26T01:42:10.396335+0000","last_undegraded":"2023-10-26T01:42:10.396335+0000","last_fullsized":"2023-10-26T01:42:10.396335+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_clean_scrub_stamp":"2023-10-26T01:42:08.349732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:30:06.187361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.412155+0000","last_change":"2023-10-26T01:42:10.411051+0000","last_active":"2023-10-26T01:42:10.412155+0000","last_peered":"2023-10-26T01:42:10.412155+0000","last_clean":"2023-10-26T01:42:10.412155+0000","last_became_active":"2023-10-26T01:42:10.408494+0000","last_became_peered":"2023-10-26T01:42:10.408494+0000","last_unstale":"2023-10-26T01:42:10.412155+0000","last_undegraded":"2023-10-26T01:42:10.412155+0000","last_fullsized":"2023-10-26T01:42:10.412155+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:27:08.396303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":11,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.392907+0000","last_change":"2023-10-26T01:42:09.402224+0000","last_active":"2023-10-26T01:42:10.392907+0000","last_peered":"2023-10-26T01:42:10.392907+0000","last_clean":"2023-10-26T01:42:10.392907+0000","last_became_active":"2023-10-26T01:42:09.401775+0000","last_became_peered":"2023-10-26T01:42:09.401775+0000","last_unstale":"2023-10-26T01:42:10.392907+0000","last_undegraded":"2023-10-26T01:42:10.392907+0000","last_fullsized":"2023-10-26T01:42:10.392907+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_clean_scrub_stamp":"2023-10-26T01:42:08.349732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:07:32.666411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.413584+0000","last_change":"2023-10-26T01:42:10.413516+0000","last_active":"2023-10-26T01:42:10.413584+0000","last_peered":"2023-10-26T01:42:10.413584+0000","last_clean":"2023-10-26T01:42:10.413584+0000","last_became_active":"2023-10-26T01:42:10.410033+0000","last_became_peered":"2023-10-26T01:42:10.410033+0000","last_unstale":"2023-10-26T01:42:10.413584+0000","last_undegraded":"2023-10-26T01:42:10.413584+0000","last_fullsized":"2023-10-26T01:42:10.413584+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:48:18.103481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.7","version":"165'2","reported_seq":13,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.419738+0000","last_change":"2023-10-26T01:42:09.408630+0000","last_active":"2023-10-26T01:42:10.419738+0000","last_peered":"2023-10-26T01:42:10.419738+0000","last_clean":"2023-10-26T01:42:10.419738+0000","last_became_active":"2023-10-26T01:42:09.408114+0000","last_became_peered":"2023-10-26T01:42:09.408114+0000","last_unstale":"2023-10-26T01:42:10.419738+0000","last_undegraded":"2023-10-26T01:42:10.419738+0000","last_fullsized":"2023-10-26T01:42:10.419738+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_clean_scrub_stamp":"2023-10-26T01:42:08.349732+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:52:58.503557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.412892+0000","last_change":"2023-10-26T01:42:10.412406+0000","last_active":"2023-10-26T01:42:10.412892+0000","last_peered":"2023-10-26T01:42:10.412892+0000","last_clean":"2023-10-26T01:42:10.412892+0000","last_became_active":"2023-10-26T01:42:10.409355+0000","last_became_peered":"2023-10-26T01:42:10.409355+0000","last_unstale":"2023-10-26T01:42:10.412892+0000","last_undegraded":"2023-10-26T01:42:10.412892+0000","last_fullsized":"2023-10-26T01:42:10.412892+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:22:49.493317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":11,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.389584+0000","last_change":"2023-10-26T01:42:09.403434+0000","last_active":"2023-10-26T01:42:10.389584+0000","last_peered":"2023-10-26T01:42:10.389584+0000","last_clean":"2023-10-26T01:42:10.389584+0000","last_became_active":"2023-10-26T01:42:09.403178+0000","last_became_peered":"2023-10-26T01:42:09.403178+0000","last_unstale":"2023-10-26T01:42:10.389584+0000","last_undegraded":"2023-10-26T01:42:10.389584+0000","last_fullsized":"2023-10-26T01:42:10.389584+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_clean_scrub_stamp":"2023-10-26T01:42:08.349732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:43:59.274726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":11,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-26T01:42:11.916246+0000","last_change":"2023-10-26T01:42:10.400259+0000","last_active":"2023-10-26T01:42:11.916246+0000","last_peered":"2023-10-26T01:42:11.916246+0000","last_clean":"2023-10-26T01:42:11.916246+0000","last_became_active":"2023-10-26T01:42:10.399855+0000","last_became_peered":"2023-10-26T01:42:10.399855+0000","last_unstale":"2023-10-26T01:42:11.916246+0000","last_undegraded":"2023-10-26T01:42:11.916246+0000","last_fullsized":"2023-10-26T01:42:11.916246+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:31:33.874549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":11,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.390843+0000","last_change":"2023-10-26T01:42:09.398662+0000","last_active":"2023-10-26T01:42:10.390843+0000","last_peered":"2023-10-26T01:42:10.390843+0000","last_clean":"2023-10-26T01:42:10.390843+0000","last_became_active":"2023-10-26T01:42:09.398290+0000","last_became_peered":"2023-10-26T01:42:09.398290+0000","last_unstale":"2023-10-26T01:42:10.390843+0000","last_undegraded":"2023-10-26T01:42:10.390843+0000","last_fullsized":"2023-10-26T01:42:10.390843+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_clean_scrub_stamp":"2023-10-26T01:42:08.349732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:00:43.327429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.410055+0000","last_change":"2023-10-26T01:42:10.408200+0000","last_active":"2023-10-26T01:42:10.410055+0000","last_peered":"2023-10-26T01:42:10.410055+0000","last_clean":"2023-10-26T01:42:10.410055+0000","last_became_active":"2023-10-26T01:42:10.407877+0000","last_became_peered":"2023-10-26T01:42:10.407877+0000","last_unstale":"2023-10-26T01:42:10.410055+0000","last_undegraded":"2023-10-26T01:42:10.410055+0000","last_fullsized":"2023-10-26T01:42:10.410055+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T01:52:06.939294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":11,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.390390+0000","last_change":"2023-10-26T01:42:09.398582+0000","last_active":"2023-10-26T01:42:10.390390+0000","last_peered":"2023-10-26T01:42:10.390390+0000","last_clean":"2023-10-26T01:42:10.390390+0000","last_became_active":"2023-10-26T01:42:09.398273+0000","last_became_peered":"2023-10-26T01:42:09.398273+0000","last_unstale":"2023-10-26T01:42:10.390390+0000","last_undegraded":"2023-10-26T01:42:10.390390+0000","last_fullsized":"2023-10-26T01:42:10.390390+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_clean_scrub_stamp":"2023-10-26T01:42:08.349732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:36:10.611662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.409391+0000","last_change":"2023-10-26T01:42:10.408929+0000","last_active":"2023-10-26T01:42:10.409391+0000","last_peered":"2023-10-26T01:42:10.409391+0000","last_clean":"2023-10-26T01:42:10.409391+0000","last_became_active":"2023-10-26T01:42:10.407830+0000","last_became_peered":"2023-10-26T01:42:10.407830+0000","last_unstale":"2023-10-26T01:42:10.409391+0000","last_undegraded":"2023-10-26T01:42:10.409391+0000","last_fullsized":"2023-10-26T01:42:10.409391+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:11:27.114346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":11,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.387828+0000","last_change":"2023-10-26T01:42:09.403023+0000","last_active":"2023-10-26T01:42:10.387828+0000","last_peered":"2023-10-26T01:42:10.387828+0000","last_clean":"2023-10-26T01:42:10.387828+0000","last_became_active":"2023-10-26T01:42:09.402490+0000","last_became_peered":"2023-10-26T01:42:09.402490+0000","last_unstale":"2023-10-26T01:42:10.387828+0000","last_undegraded":"2023-10-26T01:42:10.387828+0000","last_fullsized":"2023-10-26T01:42:10.387828+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_clean_scrub_stamp":"2023-10-26T01:42:08.349732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:52:58.503557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.408523+0000","last_change":"2023-10-26T01:42:10.407488+0000","last_active":"2023-10-26T01:42:10.408523+0000","last_peered":"2023-10-26T01:42:10.408523+0000","last_clean":"2023-10-26T01:42:10.408523+0000","last_became_active":"2023-10-26T01:42:10.407165+0000","last_became_peered":"2023-10-26T01:42:10.407165+0000","last_unstale":"2023-10-26T01:42:10.408523+0000","last_undegraded":"2023-10-26T01:42:10.408523+0000","last_fullsized":"2023-10-26T01:42:10.408523+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:04:16.931610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":11,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.395920+0000","last_change":"2023-10-26T01:42:09.408789+0000","last_active":"2023-10-26T01:42:10.395920+0000","last_peered":"2023-10-26T01:42:10.395920+0000","last_clean":"2023-10-26T01:42:10.395920+0000","last_became_active":"2023-10-26T01:42:09.408457+0000","last_became_peered":"2023-10-26T01:42:09.408457+0000","last_unstale":"2023-10-26T01:42:10.395920+0000","last_undegraded":"2023-10-26T01:42:10.395920+0000","last_fullsized":"2023-10-26T01:42:10.395920+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_clean_scrub_stamp":"2023-10-26T01:42:08.349732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:25:44.055005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.412416+0000","last_change":"2023-10-26T01:42:10.410701+0000","last_active":"2023-10-26T01:42:10.412416+0000","last_peered":"2023-10-26T01:42:10.412416+0000","last_clean":"2023-10-26T01:42:10.412416+0000","last_became_active":"2023-10-26T01:42:10.410464+0000","last_became_peered":"2023-10-26T01:42:10.410464+0000","last_unstale":"2023-10-26T01:42:10.412416+0000","last_undegraded":"2023-10-26T01:42:10.412416+0000","last_fullsized":"2023-10-26T01:42:10.412416+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:03:17.738695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":11,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.394814+0000","last_change":"2023-10-26T01:42:09.407254+0000","last_active":"2023-10-26T01:42:10.394814+0000","last_peered":"2023-10-26T01:42:10.394814+0000","last_clean":"2023-10-26T01:42:10.394814+0000","last_became_active":"2023-10-26T01:42:09.405035+0000","last_became_peered":"2023-10-26T01:42:09.405035+0000","last_unstale":"2023-10-26T01:42:10.394814+0000","last_undegraded":"2023-10-26T01:42:10.394814+0000","last_fullsized":"2023-10-26T01:42:10.394814+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_clean_scrub_stamp":"2023-10-26T01:42:08.349732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:39:55.222174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.412030+0000","last_change":"2023-10-26T01:42:10.410323+0000","last_active":"2023-10-26T01:42:10.412030+0000","last_peered":"2023-10-26T01:42:10.412030+0000","last_clean":"2023-10-26T01:42:10.412030+0000","last_became_active":"2023-10-26T01:42:10.409080+0000","last_became_peered":"2023-10-26T01:42:10.409080+0000","last_unstale":"2023-10-26T01:42:10.412030+0000","last_undegraded":"2023-10-26T01:42:10.412030+0000","last_fullsized":"2023-10-26T01:42:10.412030+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:19:17.877977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":12,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-26T01:42:11.913987+0000","last_change":"2023-10-26T01:42:09.402899+0000","last_active":"2023-10-26T01:42:11.913987+0000","last_peered":"2023-10-26T01:42:11.913987+0000","last_clean":"2023-10-26T01:42:11.913987+0000","last_became_active":"2023-10-26T01:42:09.402506+0000","last_became_peered":"2023-10-26T01:42:09.402506+0000","last_unstale":"2023-10-26T01:42:11.913987+0000","last_undegraded":"2023-10-26T01:42:11.913987+0000","last_fullsized":"2023-10-26T01:42:11.913987+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_clean_scrub_stamp":"2023-10-26T01:42:08.349732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:07:32.666411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.412133+0000","last_change":"2023-10-26T01:42:10.409711+0000","last_active":"2023-10-26T01:42:10.412133+0000","last_peered":"2023-10-26T01:42:10.412133+0000","last_clean":"2023-10-26T01:42:10.412133+0000","last_became_active":"2023-10-26T01:42:10.409103+0000","last_became_peered":"2023-10-26T01:42:10.409103+0000","last_unstale":"2023-10-26T01:42:10.412133+0000","last_undegraded":"2023-10-26T01:42:10.412133+0000","last_fullsized":"2023-10-26T01:42:10.412133+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:30:07.198892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.15","version":"165'3","reported_seq":14,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.413356+0000","last_change":"2023-10-26T01:42:09.407639+0000","last_active":"2023-10-26T01:42:10.413356+0000","last_peered":"2023-10-26T01:42:10.413356+0000","last_clean":"2023-10-26T01:42:10.413356+0000","last_became_active":"2023-10-26T01:42:09.405387+0000","last_became_peered":"2023-10-26T01:42:09.405387+0000","last_unstale":"2023-10-26T01:42:10.413356+0000","last_undegraded":"2023-10-26T01:42:10.413356+0000","last_fullsized":"2023-10-26T01:42:10.413356+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_clean_scrub_stamp":"2023-10-26T01:42:08.349732+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:36:32.565534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":308,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-26T01:42:09.401278+0000","last_change":"2023-10-26T01:30:34.883484+0000","last_active":"2023-10-26T01:42:09.401278+0000","last_peered":"2023-10-26T01:42:09.401278+0000","last_clean":"2023-10-26T01:42:09.401278+0000","last_became_active":"2023-10-26T01:30:34.883071+0000","last_became_peered":"2023-10-26T01:30:34.883071+0000","last_unstale":"2023-10-26T01:42:09.401278+0000","last_undegraded":"2023-10-26T01:42:09.401278+0000","last_fullsized":"2023-10-26T01:42:09.401278+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:32.890096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:32.890096+0000","last_clean_scrub_stamp":"2023-10-26T01:30:32.890096+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:26:31.879452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":14,"seq":60129542287,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":903,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194408,"statfs":{"total":200000000,"available":199074687,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:41:54 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.83799999999999997,"15min":0.80600000000000005},"min":{"1min":0.65100000000000002,"5min":0.443,"15min":0.33000000000000002},"max":{"1min":1.27,"5min":1.6319999999999999,"15min":2.2549999999999999},"last":0.86199999999999999},{"interface":"front","average":{"1min":1.0740000000000001,"5min":0.98699999999999999,"15min":0.91800000000000004},"min":{"1min":0.74199999999999999,"5min":0.309,"15min":0.309},"max":{"1min":1.619,"5min":1.919,"15min":1.919},"last":1.4810000000000001}]},{"osd":1,"last update":"Thu Oct 26 01:41:54 2023","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.89500000000000002,"15min":0.91300000000000003},"min":{"1min":0.58599999999999997,"5min":0.39400000000000002,"15min":0.32300000000000001},"max":{"1min":1.429,"5min":1.569,"15min":4.4340000000000002},"last":0.80400000000000005},{"interface":"front","average":{"1min":1.117,"5min":1.046,"15min":1.012},"min":{"1min":0.71899999999999997,"5min":0.51500000000000001,"15min":0.374},"max":{"1min":1.5269999999999999,"5min":1.7070000000000001,"15min":2.3330000000000002},"last":0.98999999999999999}]},{"osd":2,"last update":"Thu Oct 26 01:41:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":1.0569999999999999,"15min":1.0640000000000001},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.39900000000000002},"max":{"1min":1.5389999999999999,"5min":1.643,"15min":4.2800000000000002},"last":1.05},{"interface":"front","average":{"1min":1.131,"5min":1.1559999999999999,"15min":1.121},"min":{"1min":0.65200000000000002,"5min":0.50900000000000001,"15min":0.49399999999999999},"max":{"1min":1.5149999999999999,"5min":1.8129999999999999,"15min":2.302},"last":1.093}]},{"osd":4,"last update":"Thu Oct 26 01:41:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1970000000000001,"5min":1.2190000000000001,"15min":1.1890000000000001},"min":{"1min":0.94899999999999995,"5min":0.755,"15min":0.66400000000000003},"max":{"1min":1.4890000000000001,"5min":1.732,"15min":2.1070000000000002},"last":1.3},{"interface":"front","average":{"1min":1.4059999999999999,"5min":1.3540000000000001,"15min":1.3740000000000001},"min":{"1min":1.083,"5min":0.81599999999999995,"15min":0.79900000000000004},"max":{"1min":1.716,"5min":1.7509999999999999,"15min":4.8250000000000002},"last":1.4430000000000001}]},{"osd":5,"last update":"Thu Oct 26 01:41:54 2023","interfaces":[{"interface":"back","average":{"1min":1.4159999999999999,"5min":1.3140000000000001,"15min":1.3200000000000001},"min":{"1min":0.87,"5min":0.79300000000000004,"15min":0.72499999999999998},"max":{"1min":1.6539999999999999,"5min":1.6899999999999999,"15min":4.9080000000000004},"last":1.2390000000000001},{"interface":"front","average":{"1min":1.3300000000000001,"5min":1.3360000000000001,"15min":1.2969999999999999},"min":{"1min":0.93300000000000005,"5min":0.79100000000000004,"15min":0.79100000000000004},"max":{"1min":1.631,"5min":1.9279999999999999,"15min":2.8980000000000001},"last":1.1990000000000001}]},{"osd":6,"last update":"Thu Oct 26 01:41:54 2023","interfaces":[{"interface":"back","average":{"1min":1.4239999999999999,"5min":1.3740000000000001,"15min":1.351},"min":{"1min":1.012,"5min":0.74199999999999999,"15min":0.70199999999999996},"max":{"1min":1.6499999999999999,"5min":2.0310000000000001,"15min":2.8530000000000002},"last":1.381},{"interface":"front","average":{"1min":1.4399999999999999,"5min":1.361,"15min":1.3},"min":{"1min":1.1850000000000001,"5min":0.755,"15min":0.73099999999999998},"max":{"1min":1.6339999999999999,"5min":1.806,"15min":2.181},"last":1.4039999999999999}]},{"osd":7,"last update":"Thu Oct 26 01:41:54 2023","interfaces":[{"interface":"back","average":{"1min":1.4339999999999999,"5min":1.395,"15min":1.3340000000000001},"min":{"1min":1.1779999999999999,"5min":0.78800000000000003,"15min":0.71999999999999997},"max":{"1min":1.724,"5min":2.024,"15min":2.0680000000000001},"last":1.143},{"interface":"front","average":{"1min":1.472,"5min":1.395,"15min":1.4079999999999999},"min":{"1min":1.153,"5min":0.73299999999999998,"15min":0.72199999999999998},"max":{"1min":1.7889999999999999,"5min":1.7889999999999999,"15min":4.8730000000000002},"last":1.337}]}]},{"osd":7,"up_from":13,"seq":55834574990,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1473,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193839,"statfs":{"total":200000000,"available":198491184,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:41:59 2023","interfaces":[{"interface":"back","average":{"1min":1.169,"5min":1.1699999999999999,"15min":1.127},"min":{"1min":0.749,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.48,"5min":2.2269999999999999,"15min":2.2269999999999999},"last":0.68899999999999995},{"interface":"front","average":{"1min":1.133,"5min":1.1799999999999999,"15min":1.1279999999999999},"min":{"1min":0.79700000000000004,"5min":0.67300000000000004,"15min":0.58699999999999997},"max":{"1min":1.679,"5min":2.282,"15min":2.6779999999999999},"last":1.278}]},{"osd":1,"last update":"Thu Oct 26 01:41:59 2023","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":1.1579999999999999,"15min":1.1319999999999999},"min":{"1min":0.91100000000000003,"5min":0.65600000000000003,"15min":0.65200000000000002},"max":{"1min":1.5209999999999999,"5min":1.8,"15min":2.8260000000000001},"last":1.1140000000000001},{"interface":"front","average":{"1min":1.216,"5min":1.2310000000000001,"15min":1.2070000000000001},"min":{"1min":0.71699999999999997,"5min":0.66500000000000004,"15min":0.61199999999999999},"max":{"1min":1.681,"5min":2.0059999999999998,"15min":3.2730000000000001},"last":0.97399999999999998}]},{"osd":2,"last update":"Thu Oct 26 01:41:59 2023","interfaces":[{"interface":"back","average":{"1min":1.163,"5min":1.2030000000000001,"15min":1.1539999999999999},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.58499999999999996},"max":{"1min":1.4430000000000001,"5min":1.851,"15min":2.218},"last":1.397},{"interface":"front","average":{"1min":1.2,"5min":1.2789999999999999,"15min":1.26},"min":{"1min":0.86599999999999999,"5min":0.82299999999999995,"15min":0.73099999999999998},"max":{"1min":1.472,"5min":2.234,"15min":3.0510000000000002},"last":1.3320000000000001}]},{"osd":3,"last update":"Thu Oct 26 01:41:59 2023","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.3160000000000001,"15min":1.2549999999999999},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.5840000000000001,"5min":2.1379999999999999,"15min":2.1379999999999999},"last":1.177},{"interface":"front","average":{"1min":1.288,"5min":1.383,"15min":1.3859999999999999},"min":{"1min":0.92200000000000004,"5min":0.71099999999999997,"15min":0.65700000000000003},"max":{"1min":1.52,"5min":2.3050000000000002,"15min":5.5039999999999996},"last":1.825}]},{"osd":4,"last update":"Thu Oct 26 01:41:59 2023","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.4199999999999999,"15min":1.371},"min":{"1min":1.034,"5min":0.81799999999999995,"15min":0.76100000000000001},"max":{"1min":1.671,"5min":2.54,"15min":2.77},"last":1.7769999999999999},{"interface":"front","average":{"1min":1.302,"5min":1.4119999999999999,"15min":1.405},"min":{"1min":0.95199999999999996,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.605,"5min":2.8660000000000001,"15min":3.1419999999999999},"last":1.597}]},{"osd":5,"last update":"Thu Oct 26 01:41:59 2023","interfaces":[{"interface":"back","average":{"1min":1.377,"5min":1.423,"15min":1.429},"min":{"1min":0.98399999999999999,"5min":0.435,"15min":0.435},"max":{"1min":1.7609999999999999,"5min":2.3639999999999999,"15min":5.734},"last":1.6910000000000001},{"interface":"front","average":{"1min":1.4379999999999999,"5min":1.508,"15min":1.492},"min":{"1min":1.1120000000000001,"5min":0.76400000000000001,"15min":0.51900000000000002},"max":{"1min":1.756,"5min":2.5169999999999999,"15min":2.8980000000000001},"last":2.056}]},{"osd":6,"last update":"Thu Oct 26 01:41:59 2023","interfaces":[{"interface":"back","average":{"1min":1.5009999999999999,"5min":1.5469999999999999,"15min":1.7210000000000001},"min":{"1min":1.1739999999999999,"5min":0.51000000000000001,"15min":0.47399999999999998},"max":{"1min":1.774,"5min":2.5259999999999998,"15min":11.577999999999999},"last":1.502},{"interface":"front","average":{"1min":1.522,"5min":1.6619999999999999,"15min":1.7749999999999999},"min":{"1min":1.3799999999999999,"5min":1.0640000000000001,"15min":0.55100000000000005},"max":{"1min":1.6719999999999999,"5min":2.7250000000000001,"15min":10.704000000000001},"last":1.6399999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574990,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":903,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194409,"statfs":{"total":200000000,"available":199075245,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:41:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1339999999999999,"5min":1.1279999999999999,"15min":1.139},"min":{"1min":0.69299999999999995,"5min":0.66600000000000004,"15min":0.60399999999999998},"max":{"1min":1.706,"5min":2.7370000000000001,"15min":2.7370000000000001},"last":0.95299999999999996},{"interface":"front","average":{"1min":1.1850000000000001,"5min":1.1599999999999999,"15min":1.135},"min":{"1min":0.74099999999999999,"5min":0.61399999999999999,"15min":0.59199999999999997},"max":{"1min":1.5720000000000001,"5min":4.4779999999999998,"15min":4.4779999999999998},"last":0.85199999999999998}]},{"osd":1,"last update":"Thu Oct 26 01:41:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.2150000000000001,"15min":1.1850000000000001},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.889,"5min":5.5359999999999996,"15min":5.5359999999999996},"last":1.125},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.262,"15min":1.2130000000000001},"min":{"1min":0.90300000000000002,"5min":0.67900000000000005,"15min":0.64000000000000001},"max":{"1min":1.5960000000000001,"5min":4.3579999999999997,"15min":4.3579999999999997},"last":1.018}]},{"osd":2,"last update":"Thu Oct 26 01:41:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.1479999999999999,"15min":1.1259999999999999},"min":{"1min":0.77400000000000002,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.556,"5min":2.367,"15min":3.7879999999999998},"last":1.175},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.3300000000000001,"15min":1.2869999999999999},"min":{"1min":1.0169999999999999,"5min":0.86799999999999999,"15min":0.68899999999999995},"max":{"1min":1.645,"5min":3.0470000000000002,"15min":3.0470000000000002},"last":1.3360000000000001}]},{"osd":3,"last update":"Thu Oct 26 01:41:54 2023","interfaces":[{"interface":"back","average":{"1min":1.373,"5min":1.284,"15min":1.2490000000000001},"min":{"1min":1.016,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.8200000000000001,"5min":3.266,"15min":4.4560000000000004},"last":1.0780000000000001},{"interface":"front","average":{"1min":1.383,"5min":1.347,"15min":1.3360000000000001},"min":{"1min":1.0780000000000001,"5min":0.78600000000000003,"15min":0.69499999999999995},"max":{"1min":1.6719999999999999,"5min":2.6909999999999998,"15min":2.6909999999999998},"last":1.248}]},{"osd":4,"last update":"Thu Oct 26 01:41:54 2023","interfaces":[{"interface":"back","average":{"1min":1.302,"5min":1.2669999999999999,"15min":1.2410000000000001},"min":{"1min":0.81200000000000006,"5min":0.53400000000000003,"15min":0.48399999999999999},"max":{"1min":1.7430000000000001,"5min":1.825,"15min":2.9489999999999998},"last":0.59899999999999998},{"interface":"front","average":{"1min":1.3080000000000001,"5min":1.3280000000000001,"15min":1.3380000000000001},"min":{"1min":0.88600000000000001,"5min":0.60699999999999998,"15min":0.59299999999999997},"max":{"1min":2.169,"5min":2.169,"15min":6.609},"last":0.88400000000000001}]},{"osd":5,"last update":"Thu Oct 26 01:41:54 2023","interfaces":[{"interface":"back","average":{"1min":1.516,"5min":1.5169999999999999,"15min":1.4850000000000001},"min":{"1min":1.23,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.9399999999999999,"5min":2.04,"15min":5.4370000000000003},"last":1.2869999999999999},{"interface":"front","average":{"1min":1.4570000000000001,"5min":1.464,"15min":1.4339999999999999},"min":{"1min":0.92200000000000004,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.8520000000000001,"5min":2.7759999999999998,"15min":2.7759999999999998},"last":0.69999999999999996}]},{"osd":7,"last update":"Thu Oct 26 01:41:54 2023","interfaces":[{"interface":"back","average":{"1min":1.4450000000000001,"5min":1.3999999999999999,"15min":1.4259999999999999},"min":{"1min":0.93500000000000005,"5min":0.58899999999999997,"15min":0.54100000000000004},"max":{"1min":1.984,"5min":1.984,"15min":11.804},"last":0.72399999999999998},{"interface":"front","average":{"1min":1.54,"5min":1.5449999999999999,"15min":1.544},"min":{"1min":1.1539999999999999,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.909,"5min":3.7650000000000001,"15min":11.541},"last":1.3899999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574990,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":903,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194408,"statfs":{"total":200000000,"available":199074755,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:41:53 2023","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.238,"15min":1.171},"min":{"1min":0.68700000000000006,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.724,"5min":3.8210000000000002,"15min":3.8210000000000002},"last":0.90200000000000002},{"interface":"front","average":{"1min":1.371,"5min":1.3029999999999999,"15min":1.2350000000000001},"min":{"1min":0.94599999999999995,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.609,"5min":2.9889999999999999,"15min":2.9889999999999999},"last":0.97199999999999998}]},{"osd":1,"last update":"Thu Oct 26 01:41:53 2023","interfaces":[{"interface":"back","average":{"1min":1.2969999999999999,"5min":1.1919999999999999,"15min":1.133},"min":{"1min":0.90400000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.6850000000000001,"5min":1.899,"15min":1.899},"last":0.71299999999999997},{"interface":"front","average":{"1min":1.194,"5min":1.133,"15min":1.1180000000000001},"min":{"1min":0.88700000000000001,"5min":0.626,"15min":0.60599999999999998},"max":{"1min":1.5569999999999999,"5min":2.4340000000000002,"15min":2.4340000000000002},"last":1.01}]},{"osd":2,"last update":"Thu Oct 26 01:41:53 2023","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.2170000000000001,"15min":1.1870000000000001},"min":{"1min":0.94899999999999995,"5min":0.71599999999999997,"15min":0.69299999999999995},"max":{"1min":1.613,"5min":1.796,"15min":1.8220000000000001},"last":1.2270000000000001},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.226,"15min":1.2070000000000001},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.63900000000000001},"max":{"1min":1.637,"5min":1.875,"15min":1.875},"last":1.3140000000000001}]},{"osd":3,"last update":"Thu Oct 26 01:41:53 2023","interfaces":[{"interface":"back","average":{"1min":1.226,"5min":1.2629999999999999,"15min":1.262},"min":{"1min":0.85599999999999998,"5min":0.72399999999999998,"15min":0.72299999999999998},"max":{"1min":1.587,"5min":1.7,"15min":1.835},"last":1.1060000000000001},{"interface":"front","average":{"1min":1.5860000000000001,"5min":1.323,"15min":1.3089999999999999},"min":{"1min":1.139,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":2.6669999999999998,"5min":2.6669999999999998,"15min":2.9249999999999998},"last":0.83099999999999996}]},{"osd":4,"last update":"Thu Oct 26 01:41:53 2023","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.21,"15min":1.22},"min":{"1min":0.85799999999999998,"5min":0.59599999999999997,"15min":0.49099999999999999},"max":{"1min":1.47,"5min":2.7770000000000001,"15min":2.7770000000000001},"last":1.272},{"interface":"front","average":{"1min":1.347,"5min":1.353,"15min":1.369},"min":{"1min":0.93100000000000005,"5min":0.57299999999999995,"15min":0.56200000000000006},"max":{"1min":1.8029999999999999,"5min":1.857,"15min":1.867},"last":1.0589999999999999}]},{"osd":6,"last update":"Thu Oct 26 01:41:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.3979999999999999,"15min":1.3680000000000001},"min":{"1min":1.0129999999999999,"5min":0.67500000000000004,"15min":0.51900000000000002},"max":{"1min":1.794,"5min":2.4590000000000001,"15min":2.8250000000000002},"last":1.3959999999999999},{"interface":"front","average":{"1min":1.4279999999999999,"5min":1.4219999999999999,"15min":1.379},"min":{"1min":0.97699999999999998,"5min":0.69499999999999995,"15min":0.61799999999999999},"max":{"1min":1.831,"5min":2.4830000000000001,"15min":2.4830000000000001},"last":1.1739999999999999}]},{"osd":7,"last update":"Thu Oct 26 01:41:53 2023","interfaces":[{"interface":"back","average":{"1min":1.472,"5min":1.5229999999999999,"15min":1.474},"min":{"1min":1.0589999999999999,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.8919999999999999,"5min":3.621,"15min":3.621},"last":1.355},{"interface":"front","average":{"1min":1.5529999999999999,"5min":1.534,"15min":1.4870000000000001},"min":{"1min":1.3009999999999999,"5min":0.73899999999999999,"15min":0.59499999999999997},"max":{"1min":1.8819999999999999,"5min":2.8010000000000002,"15min":2.8010000000000002},"last":1.446}]}]},{"osd":4,"up_from":13,"seq":55834574990,"num_pgs":12,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":903,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194408,"statfs":{"total":200000000,"available":199074777,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:41:45 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.1779999999999999,"15min":1.0549999999999999},"min":{"1min":0.84499999999999997,"5min":0.63500000000000001,"15min":0.53100000000000003},"max":{"1min":1.702,"5min":1.78,"15min":2.0289999999999999},"last":1.3009999999999999},{"interface":"front","average":{"1min":1.1579999999999999,"5min":1.139,"15min":1.0680000000000001},"min":{"1min":0.747,"5min":0.66300000000000003,"15min":0.60599999999999998},"max":{"1min":1.6000000000000001,"5min":1.921,"15min":2.6829999999999998},"last":1.1060000000000001}]},{"osd":1,"last update":"Thu Oct 26 01:41:45 2023","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.2,"15min":1.1579999999999999},"min":{"1min":0.92500000000000004,"5min":0.66700000000000004,"15min":0.56799999999999995},"max":{"1min":1.716,"5min":2.056,"15min":4.0519999999999996},"last":1.369},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.2849999999999999,"15min":1.1719999999999999},"min":{"1min":0.98599999999999999,"5min":0.69699999999999995,"15min":0.60499999999999998},"max":{"1min":1.5089999999999999,"5min":2.4390000000000001,"15min":2.4390000000000001},"last":1.165}]},{"osd":2,"last update":"Thu Oct 26 01:41:45 2023","interfaces":[{"interface":"back","average":{"1min":1.401,"5min":1.379,"15min":1.292},"min":{"1min":0.98699999999999999,"5min":0.66700000000000004,"15min":0.64300000000000002},"max":{"1min":1.6519999999999999,"5min":2.2040000000000002,"15min":5.6749999999999998},"last":1.222},{"interface":"front","average":{"1min":1.246,"5min":1.2490000000000001,"15min":1.175},"min":{"1min":0.997,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.5249999999999999,"5min":3.673,"15min":3.673},"last":1.0569999999999999}]},{"osd":3,"last update":"Thu Oct 26 01:41:45 2023","interfaces":[{"interface":"back","average":{"1min":1.4299999999999999,"5min":1.3580000000000001,"15min":1.244},"min":{"1min":0.91200000000000003,"5min":0.78900000000000003,"15min":0.70399999999999996},"max":{"1min":1.9630000000000001,"5min":2.5960000000000001,"15min":2.5960000000000001},"last":1.0129999999999999},{"interface":"front","average":{"1min":1.2909999999999999,"5min":1.2410000000000001,"15min":1.1910000000000001},"min":{"1min":0.93300000000000005,"5min":0.85399999999999998,"15min":0.69699999999999995},"max":{"1min":1.639,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":1.1399999999999999}]},{"osd":5,"last update":"Thu Oct 26 01:41:45 2023","interfaces":[{"interface":"back","average":{"1min":1.3009999999999999,"5min":1.302,"15min":1.286},"min":{"1min":0.86399999999999999,"5min":0.52100000000000002,"15min":0.46300000000000002},"max":{"1min":1.8660000000000001,"5min":2.774,"15min":2.774},"last":0.88600000000000001},{"interface":"front","average":{"1min":1.276,"5min":1.349,"15min":1.335},"min":{"1min":0.84099999999999997,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.8280000000000001,"5min":2.2850000000000001,"15min":5.8579999999999997},"last":0.94599999999999995}]},{"osd":6,"last update":"Thu Oct 26 01:41:45 2023","interfaces":[{"interface":"back","average":{"1min":1.345,"5min":1.361,"15min":1.357},"min":{"1min":0.92900000000000005,"5min":0.69199999999999995,"15min":0.59299999999999997},"max":{"1min":1.7430000000000001,"5min":2.262,"15min":12.343999999999999},"last":1.2569999999999999},{"interface":"front","average":{"1min":1.5700000000000001,"5min":1.5820000000000001,"15min":1.554},"min":{"1min":1.1060000000000001,"5min":0.57999999999999996,"15min":0.55000000000000004},"max":{"1min":2.0790000000000002,"5min":2.9020000000000001,"15min":12.605},"last":1.526}]},{"osd":7,"last update":"Thu Oct 26 01:41:45 2023","interfaces":[{"interface":"back","average":{"1min":1.5109999999999999,"5min":1.556,"15min":1.5169999999999999},"min":{"1min":0.98699999999999999,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.921,"5min":2.7200000000000002,"15min":5.774},"last":1.4830000000000001},{"interface":"front","average":{"1min":1.538,"5min":1.5329999999999999,"15min":1.49},"min":{"1min":1.238,"5min":0.79000000000000004,"15min":0.68000000000000005},"max":{"1min":1.7689999999999999,"5min":2.8149999999999999,"15min":4.6520000000000001},"last":1.206}]}]},{"osd":2,"up_from":13,"seq":55834574990,"num_pgs":16,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":896,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194415,"statfs":{"total":200000000,"available":199081552,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:41:57 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.79200000000000004,"15min":0.83999999999999997},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.308},"max":{"1min":1.2010000000000001,"5min":1.4850000000000001,"15min":6.3200000000000003},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.92200000000000004,"15min":0.94599999999999995},"min":{"1min":0.45900000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.4379999999999999,"5min":1.476,"15min":4.2619999999999996},"last":1.101}]},{"osd":1,"last update":"Thu Oct 26 01:41:57 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.83499999999999996,"15min":0.88600000000000001},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.2729999999999999,"5min":1.504,"15min":5.1260000000000003},"last":1.0169999999999999},{"interface":"front","average":{"1min":0.97999999999999998,"5min":0.97499999999999998,"15min":1.008},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.36599999999999999},"max":{"1min":1.4159999999999999,"5min":1.4870000000000001,"15min":6.1520000000000001},"last":0.85099999999999998}]},{"osd":3,"last update":"Thu Oct 26 01:42:02 2023","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":1.0329999999999999,"15min":0.98699999999999999},"min":{"1min":0.46800000000000003,"5min":0.435,"15min":0.435},"max":{"1min":1.6719999999999999,"5min":1.909,"15min":2.0640000000000001},"last":1.2390000000000001},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.0629999999999999,"15min":1.0589999999999999},"min":{"1min":0.51400000000000001,"5min":0.503,"15min":0.45300000000000001},"max":{"1min":1.506,"5min":1.6659999999999999,"15min":8.9019999999999992},"last":1.1719999999999999}]},{"osd":4,"last update":"Thu Oct 26 01:41:57 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.292,"15min":1.292},"min":{"1min":0.77300000000000002,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.6890000000000001,"5min":2.109,"15min":6.2670000000000003},"last":1.341},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.244,"15min":1.274},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.53,"5min":1.923,"15min":7.343},"last":1.3089999999999999}]},{"osd":5,"last update":"Thu Oct 26 01:41:57 2023","interfaces":[{"interface":"back","average":{"1min":1.3140000000000001,"5min":1.292,"15min":1.304},"min":{"1min":0.79000000000000004,"5min":0.76900000000000002,"15min":0.71899999999999997},"max":{"1min":1.726,"5min":2.3849999999999998,"15min":3.7170000000000001},"last":1.395},{"interface":"front","average":{"1min":1.359,"5min":1.2949999999999999,"15min":1.3069999999999999},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.65300000000000002},"max":{"1min":1.728,"5min":1.8140000000000001,"15min":6.3899999999999997},"last":0.95599999999999996}]},{"osd":6,"last update":"Thu Oct 26 01:41:57 2023","interfaces":[{"interface":"back","average":{"1min":1.335,"5min":1.367,"15min":1.361},"min":{"1min":0.81100000000000005,"5min":0.76300000000000001,"15min":0.70599999999999996},"max":{"1min":1.73,"5min":2.097,"15min":6.0490000000000004},"last":1.5640000000000001},{"interface":"front","average":{"1min":1.3720000000000001,"5min":1.3620000000000001,"15min":1.381},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.6910000000000001,"5min":2.5230000000000001,"15min":5.0650000000000004},"last":1.516}]},{"osd":7,"last update":"Thu Oct 26 01:41:57 2023","interfaces":[{"interface":"back","average":{"1min":1.4510000000000001,"5min":1.391,"15min":1.4059999999999999},"min":{"1min":0.85699999999999998,"5min":0.77900000000000003,"15min":0.69399999999999995},"max":{"1min":1.722,"5min":1.7949999999999999,"15min":6.4480000000000004},"last":1.2649999999999999},{"interface":"front","average":{"1min":1.425,"5min":1.4670000000000001,"15min":1.47},"min":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"max":{"1min":1.7490000000000001,"5min":2.1480000000000001,"15min":2.8439999999999999},"last":1.454}]}]},{"osd":1,"up_from":13,"seq":55834574990,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":904,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194408,"statfs":{"total":200000000,"available":199074195,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:41:52 2023","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.81699999999999995,"15min":0.76300000000000001},"min":{"1min":0.64400000000000002,"5min":0.36099999999999999,"15min":0.29499999999999998},"max":{"1min":1.3080000000000001,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":1.0820000000000001},{"interface":"front","average":{"1min":1.022,"5min":0.92300000000000004,"15min":0.84699999999999998},"min":{"1min":0.70899999999999996,"5min":0.435,"15min":0.36199999999999999},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.6399999999999999},"last":0.84199999999999997}]},{"osd":2,"last update":"Thu Oct 26 01:41:52 2023","interfaces":[{"interface":"back","average":{"1min":1.167,"5min":1.103,"15min":1.022},"min":{"1min":0.88900000000000001,"5min":0.52200000000000002,"15min":0.39600000000000002},"max":{"1min":1.492,"5min":1.8200000000000001,"15min":2.4830000000000001},"last":1.1899999999999999},{"interface":"front","average":{"1min":1.097,"5min":1.054,"15min":0.93100000000000005},"min":{"1min":0.74399999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.5049999999999999,"5min":1.772,"15min":1.8080000000000001},"last":1.002}]},{"osd":3,"last update":"Thu Oct 26 01:41:57 2023","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":1.0529999999999999,"15min":0.98899999999999999},"min":{"1min":0.84799999999999998,"5min":0.52900000000000003,"15min":0.39500000000000002},"max":{"1min":1.6100000000000001,"5min":1.6100000000000001,"15min":1.75},"last":1.1619999999999999},{"interface":"front","average":{"1min":1.125,"5min":1.0669999999999999,"15min":1.022},"min":{"1min":0.73899999999999999,"5min":0.55000000000000004,"15min":0.41299999999999998},"max":{"1min":1.476,"5min":1.5049999999999999,"15min":1.802},"last":1.2889999999999999}]},{"osd":4,"last update":"Thu Oct 26 01:41:52 2023","interfaces":[{"interface":"back","average":{"1min":1.3009999999999999,"5min":1.2629999999999999,"15min":1.2230000000000001},"min":{"1min":1.1499999999999999,"5min":0.79800000000000004,"15min":0.56200000000000006},"max":{"1min":1.5649999999999999,"5min":1.756,"15min":2.0659999999999998},"last":1.444},{"interface":"front","average":{"1min":1.333,"5min":1.2210000000000001,"15min":1.175},"min":{"1min":1.004,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.8420000000000001,"5min":1.8420000000000001,"15min":2.0270000000000001},"last":1.4239999999999999}]},{"osd":5,"last update":"Thu Oct 26 01:41:52 2023","interfaces":[{"interface":"back","average":{"1min":1.3899999999999999,"5min":1.2769999999999999,"15min":1.252},"min":{"1min":0.85999999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":1.8779999999999999},"last":1.242},{"interface":"front","average":{"1min":1.401,"5min":1.399,"15min":1.3580000000000001},"min":{"1min":1.099,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.6679999999999999,"5min":2.04,"15min":2.2200000000000002},"last":1.52}]},{"osd":6,"last update":"Thu Oct 26 01:41:52 2023","interfaces":[{"interface":"back","average":{"1min":1.347,"5min":1.341,"15min":1.306},"min":{"1min":1.024,"5min":0.75,"15min":0.68899999999999995},"max":{"1min":1.651,"5min":1.7709999999999999,"15min":2.3780000000000001},"last":1.383},{"interface":"front","average":{"1min":1.361,"5min":1.3480000000000001,"15min":1.2829999999999999},"min":{"1min":1.224,"5min":0.80700000000000005,"15min":0.70799999999999996},"max":{"1min":1.534,"5min":1.8819999999999999,"15min":1.984},"last":1.3360000000000001}]},{"osd":7,"last update":"Thu Oct 26 01:41:52 2023","interfaces":[{"interface":"back","average":{"1min":1.407,"5min":1.319,"15min":1.284},"min":{"1min":1.1619999999999999,"5min":0.76100000000000001,"15min":0.69699999999999995},"max":{"1min":1.635,"5min":1.835,"15min":2.6930000000000001},"last":1.49},{"interface":"front","average":{"1min":1.4630000000000001,"5min":1.4359999999999999,"15min":1.377},"min":{"1min":1.0960000000000001,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.75,"5min":1.9370000000000001,"15min":2.0190000000000001},"last":1.593}]}]},{"osd":0,"up_from":13,"seq":55834574990,"num_pgs":13,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1481,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193830,"statfs":{"total":200000000,"available":198482763,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Oct 26 01:41:54 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.78700000000000003,"15min":0.89000000000000001},"min":{"1min":0.63200000000000001,"5min":0.32500000000000001,"15min":0.32000000000000001},"max":{"1min":1.1910000000000001,"5min":1.494,"15min":19.425000000000001},"last":0.745},{"interface":"front","average":{"1min":1.0009999999999999,"5min":0.95399999999999996,"15min":1.0249999999999999},"min":{"1min":0.58399999999999996,"5min":0.434,"15min":0.434},"max":{"1min":1.492,"5min":1.492,"15min":15.162000000000001},"last":1.1659999999999999}]},{"osd":2,"last update":"Thu Oct 26 01:41:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.0409999999999999,"15min":1.1599999999999999},"min":{"1min":0.69299999999999995,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.4750000000000001,"5min":1.556,"15min":28.439},"last":0.95399999999999996},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.93400000000000005,"15min":0.92000000000000004},"min":{"1min":0.72099999999999997,"5min":0.37,"15min":0.37},"max":{"1min":1.1539999999999999,"5min":1.851,"15min":4.9459999999999997},"last":0.90500000000000003}]},{"osd":3,"last update":"Thu Oct 26 01:41:54 2023","interfaces":[{"interface":"back","average":{"1min":1.163,"5min":1.113,"15min":1.0920000000000001},"min":{"1min":0.84899999999999998,"5min":0.48899999999999999,"15min":0.44600000000000001},"max":{"1min":1.3819999999999999,"5min":1.7509999999999999,"15min":4.9800000000000004},"last":1.1240000000000001},{"interface":"front","average":{"1min":1.28,"5min":1.1839999999999999,"15min":1.165},"min":{"1min":0.71599999999999997,"5min":0.438,"15min":0.438},"max":{"1min":1.546,"5min":1.774,"15min":5.016},"last":1.595}]},{"osd":4,"last update":"Thu Oct 26 01:41:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1559999999999999,"5min":1.1319999999999999,"15min":1.2589999999999999},"min":{"1min":0.86399999999999999,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.3680000000000001,"5min":1.5840000000000001,"15min":19.329000000000001},"last":1.0649999999999999},{"interface":"front","average":{"1min":1.2809999999999999,"5min":1.2509999999999999,"15min":1.2809999999999999},"min":{"1min":0.93200000000000005,"5min":0.86799999999999999,"15min":0.67600000000000005},"max":{"1min":1.7030000000000001,"5min":1.7270000000000001,"15min":5.5229999999999997},"last":1.3009999999999999}]},{"osd":5,"last update":"Thu Oct 26 01:41:54 2023","interfaces":[{"interface":"back","average":{"1min":1.4359999999999999,"5min":1.421,"15min":1.425},"min":{"1min":1.179,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.655,"5min":1.819,"15min":5.6360000000000001},"last":1.4430000000000001},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.262,"15min":1.375},"min":{"1min":0.91600000000000004,"5min":0.73599999999999999,"15min":0.57499999999999996},"max":{"1min":1.7390000000000001,"5min":1.742,"15min":19.384},"last":1.1910000000000001}]},{"osd":6,"last update":"Thu Oct 26 01:41:54 2023","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.319,"15min":1.355},"min":{"1min":1.018,"5min":0.76700000000000002,"15min":0.73799999999999999},"max":{"1min":1.8999999999999999,"5min":1.8999999999999999,"15min":8.2609999999999992},"last":1.353},{"interface":"front","average":{"1min":1.3799999999999999,"5min":1.3640000000000001,"15min":1.399},"min":{"1min":0.98299999999999998,"5min":0.82999999999999996,"15min":0.75900000000000001},"max":{"1min":1.8600000000000001,"5min":1.9399999999999999,"15min":11.849},"last":1.4099999999999999}]},{"osd":7,"last update":"Thu Oct 26 01:41:54 2023","interfaces":[{"interface":"back","average":{"1min":1.518,"5min":1.4750000000000001,"15min":1.4770000000000001},"min":{"1min":1.133,"5min":0.871,"15min":0.73699999999999999},"max":{"1min":1.768,"5min":1.9379999999999999,"15min":5.6689999999999996},"last":1.5369999999999999},{"interface":"front","average":{"1min":1.444,"5min":1.401,"15min":1.49},"min":{"1min":0.96499999999999997,"5min":0.81299999999999994,"15min":0.69399999999999995},"max":{"1min":1.6779999999999999,"5min":1.841,"15min":19.245999999999999},"last":1.492}]}]}],"pool_statfs":[]}} 2023-10-26T01:42:14.897 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-26T01:42:15.116 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:15.114+0000 7f302e8a3640 1 -- 172.21.15.169:0/3640702045 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f302806cae0 msgr2=0x7f30280c7ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:15.116 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:15.114+0000 7f302e8a3640 1 --2- 172.21.15.169:0/3640702045 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f302806cae0 0x7f30280c7ea0 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f3024009830 tx=0x7f302402bcd0 comp rx=0 tx=0).stop 2023-10-26T01:42:15.117 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:15.114+0000 7f302e8a3640 1 -- 172.21.15.169:0/3640702045 shutdown_connections 2023-10-26T01:42:15.117 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:15.114+0000 7f302e8a3640 1 --2- 172.21.15.169:0/3640702045 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f302806a990 0x7f30280c96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:15.117 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:15.114+0000 7f302e8a3640 1 --2- 172.21.15.169:0/3640702045 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f302806cae0 0x7f30280c7ea0 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:15.117 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:15.114+0000 7f302e8a3640 1 -- 172.21.15.169:0/3640702045 >> 172.21.15.169:0/3640702045 conn(0x7f30280aedd0 msgr2=0x7f30280af1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:42:15.117 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:15.114+0000 7f302e8a3640 1 -- 172.21.15.169:0/3640702045 shutdown_connections 2023-10-26T01:42:15.117 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:15.114+0000 7f302e8a3640 1 -- 172.21.15.169:0/3640702045 wait complete. 2023-10-26T01:42:15.118 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:15.116+0000 7f302e8a3640 1 Processor -- start 2023-10-26T01:42:15.118 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:15.116+0000 7f302e8a3640 1 -- start start 2023-10-26T01:42:15.119 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:15.117+0000 7f302e8a3640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f302806a990 0x7f302814bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:15.120 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:15.117+0000 7f302e8a3640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f302806cae0 0x7f302814e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:15.120 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:15.117+0000 7f302e8a3640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f30280c9c30 0x7f3028098a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:15.120 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:15.117+0000 7f302e8a3640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3028152bd0 con 0x7f302806a990 2023-10-26T01:42:15.120 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:15.117+0000 7f302e8a3640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f302814c0a0 con 0x7f30280c9c30 2023-10-26T01:42:15.120 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:15.117+0000 7f302e8a3640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3028152860 con 0x7f302806cae0 2023-10-26T01:42:15.121 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:15.117+0000 7f302d8a1640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f302806a990 0x7f302814bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:42:15.121 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:15.117+0000 7f302d8a1640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f302806a990 0x7f302814bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:54128/0 (socket says 172.21.15.169:54128) 2023-10-26T01:42:15.121 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:15.117+0000 7f302d8a1640 1 -- 172.21.15.169:0/894819545 learned_addr learned my addr 172.21.15.169:0/894819545 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:42:15.121 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:15.118+0000 7f302d8a1640 1 -- 172.21.15.169:0/894819545 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f302806cae0 msgr2=0x7f302814e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:15.121 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:15.118+0000 7f302d8a1640 1 --2- 172.21.15.169:0/894819545 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f302806cae0 0x7f302814e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:15.121 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:15.118+0000 7f302d8a1640 1 -- 172.21.15.169:0/894819545 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f30280c9c30 msgr2=0x7f3028098a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:15.122 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:15.118+0000 7f302d8a1640 1 --2- 172.21.15.169:0/894819545 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f30280c9c30 0x7f3028098a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:15.122 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:15.118+0000 7f302d8a1640 1 -- 172.21.15.169:0/894819545 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3024009490 con 0x7f302806a990 2023-10-26T01:42:15.122 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:15.118+0000 7f302d8a1640 1 --2- 172.21.15.169:0/894819545 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f302806a990 0x7f302814bb60 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f3024000c00 tx=0x7f30240038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:42:15.123 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:15.118+0000 7f301e7fc640 1 -- 172.21.15.169:0/894819545 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3024003ea0 con 0x7f302806a990 2023-10-26T01:42:15.124 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:15.118+0000 7f301e7fc640 1 -- 172.21.15.169:0/894819545 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3024034440 con 0x7f302806a990 2023-10-26T01:42:15.124 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:15.118+0000 7f302e8a3640 1 -- 172.21.15.169:0/894819545 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30281541e0 con 0x7f302806a990 2023-10-26T01:42:15.124 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:15.119+0000 7f301e7fc640 1 -- 172.21.15.169:0/894819545 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3024034850 con 0x7f302806a990 2023-10-26T01:42:15.124 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:15.119+0000 7f302e8a3640 1 -- 172.21.15.169:0/894819545 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30281547a0 con 0x7f302806a990 2023-10-26T01:42:15.124 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:15.120+0000 7f302e8a3640 1 -- 172.21.15.169:0/894819545 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3028152d10 con 0x7f302806a990 2023-10-26T01:42:15.124 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:15.121+0000 7f301e7fc640 1 -- 172.21.15.169:0/894819545 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f30240349b0 con 0x7f302806a990 2023-10-26T01:42:15.125 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:15.122+0000 7f301e7fc640 1 --2- 172.21.15.169:0/894819545 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f300c06ae70 0x7f300c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:15.125 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:15.122+0000 7f301e7fc640 1 -- 172.21.15.169:0/894819545 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f30240abe20 con 0x7f302806a990 2023-10-26T01:42:15.125 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:15.123+0000 7f302d0a0640 1 --2- 172.21.15.169:0/894819545 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f300c06ae70 0x7f300c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:42:15.126 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:15.124+0000 7f302d0a0640 1 --2- 172.21.15.169:0/894819545 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f300c06ae70 0x7f300c06d330 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f3018005fd0 tx=0x7f3018005850 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:42:15.128 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:15.126+0000 7f301e7fc640 1 -- 172.21.15.169:0/894819545 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3028152d10 con 0x7f302806a990 2023-10-26T01:42:15.315 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:15.313+0000 7f302e8a3640 1 -- 172.21.15.169:0/894819545 --> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3028157d40 con 0x7f300c06ae70 2023-10-26T01:42:15.336 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:15.334+0000 7f301e7fc640 1 -- 172.21.15.169:0/894819545 <== mgr.4112 v2:172.21.15.170:6832/1187156918 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+282625 (secure 0 0 0) 0x7f301800b6f0 con 0x7f300c06ae70 2023-10-26T01:42:15.338 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:42:15.342 INFO:teuthology.orchestra.run.smithi169.stderr:dumped all 2023-10-26T01:42:15.343 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:15.341+0000 7f302e8a3640 1 -- 172.21.15.169:0/894819545 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f300c06ae70 msgr2=0x7f300c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:15.343 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:15.341+0000 7f302e8a3640 1 --2- 172.21.15.169:0/894819545 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f300c06ae70 0x7f300c06d330 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f3018005fd0 tx=0x7f3018005850 comp rx=0 tx=0).stop 2023-10-26T01:42:15.343 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:15.341+0000 7f302e8a3640 1 -- 172.21.15.169:0/894819545 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f302806a990 msgr2=0x7f302814bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:15.344 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:15.341+0000 7f302e8a3640 1 --2- 172.21.15.169:0/894819545 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f302806a990 0x7f302814bb60 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f3024000c00 tx=0x7f30240038a0 comp rx=0 tx=0).stop 2023-10-26T01:42:15.344 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:15.341+0000 7f302e8a3640 1 -- 172.21.15.169:0/894819545 shutdown_connections 2023-10-26T01:42:15.344 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:15.341+0000 7f302e8a3640 1 --2- 172.21.15.169:0/894819545 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f300c06ae70 0x7f300c06d330 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:15.344 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:15.341+0000 7f302e8a3640 1 --2- 172.21.15.169:0/894819545 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f30280c9c30 0x7f3028098a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:15.344 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:15.341+0000 7f302e8a3640 1 --2- 172.21.15.169:0/894819545 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f302806cae0 0x7f302814e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:15.345 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:15.341+0000 7f302e8a3640 1 --2- 172.21.15.169:0/894819545 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f302806a990 0x7f302814bb60 unknown :-1 s=CLOSED pgs=1704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:15.345 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:15.341+0000 7f302e8a3640 1 -- 172.21.15.169:0/894819545 >> 172.21.15.169:0/894819545 conn(0x7f30280aedd0 msgr2=0x7f3028155750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:42:15.345 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:15.341+0000 7f302e8a3640 1 -- 172.21.15.169:0/894819545 shutdown_connections 2023-10-26T01:42:15.345 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:15.341+0000 7f302e8a3640 1 -- 172.21.15.169:0/894819545 wait complete. 2023-10-26T01:42:15.363 INFO:teuthology.orchestra.run.smithi169.stdout:{"pg_ready":true,"pg_map":{"version":511,"stamp":"2023-10-26T01:42:14.382552+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":80,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":8369,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1554130,"statfs":{"total":1600000000,"available":1591429158,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002078"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.418672+0000","last_change":"2023-10-26T01:42:10.418587+0000","last_active":"2023-10-26T01:42:10.418672+0000","last_peered":"2023-10-26T01:42:10.418672+0000","last_clean":"2023-10-26T01:42:10.418672+0000","last_became_active":"2023-10-26T01:42:10.418333+0000","last_became_peered":"2023-10-26T01:42:10.418333+0000","last_unstale":"2023-10-26T01:42:10.418672+0000","last_undegraded":"2023-10-26T01:42:10.418672+0000","last_fullsized":"2023-10-26T01:42:10.418672+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:54:20.518649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":11,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-26T01:42:11.913772+0000","last_change":"2023-10-26T01:42:10.395338+0000","last_active":"2023-10-26T01:42:11.913772+0000","last_peered":"2023-10-26T01:42:11.913772+0000","last_clean":"2023-10-26T01:42:11.913772+0000","last_became_active":"2023-10-26T01:42:10.395079+0000","last_became_peered":"2023-10-26T01:42:10.395079+0000","last_unstale":"2023-10-26T01:42:11.913772+0000","last_undegraded":"2023-10-26T01:42:11.913772+0000","last_fullsized":"2023-10-26T01:42:11.913772+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:05:17.258140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.407471+0000","last_change":"2023-10-26T01:42:10.406941+0000","last_active":"2023-10-26T01:42:10.407471+0000","last_peered":"2023-10-26T01:42:10.407471+0000","last_clean":"2023-10-26T01:42:10.407471+0000","last_became_active":"2023-10-26T01:42:10.406338+0000","last_became_peered":"2023-10-26T01:42:10.406338+0000","last_unstale":"2023-10-26T01:42:10.407471+0000","last_undegraded":"2023-10-26T01:42:10.407471+0000","last_fullsized":"2023-10-26T01:42:10.407471+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:15:19.901066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.408843+0000","last_change":"2023-10-26T01:42:10.408057+0000","last_active":"2023-10-26T01:42:10.408843+0000","last_peered":"2023-10-26T01:42:10.408843+0000","last_clean":"2023-10-26T01:42:10.408843+0000","last_became_active":"2023-10-26T01:42:10.407815+0000","last_became_peered":"2023-10-26T01:42:10.407815+0000","last_unstale":"2023-10-26T01:42:10.408843+0000","last_undegraded":"2023-10-26T01:42:10.408843+0000","last_fullsized":"2023-10-26T01:42:10.408843+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:44:45.588628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.414596+0000","last_change":"2023-10-26T01:42:10.414294+0000","last_active":"2023-10-26T01:42:10.414596+0000","last_peered":"2023-10-26T01:42:10.414596+0000","last_clean":"2023-10-26T01:42:10.414596+0000","last_became_active":"2023-10-26T01:42:10.413979+0000","last_became_peered":"2023-10-26T01:42:10.413979+0000","last_unstale":"2023-10-26T01:42:10.414596+0000","last_undegraded":"2023-10-26T01:42:10.414596+0000","last_fullsized":"2023-10-26T01:42:10.414596+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:13:55.827244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.413899+0000","last_change":"2023-10-26T01:42:10.413817+0000","last_active":"2023-10-26T01:42:10.413899+0000","last_peered":"2023-10-26T01:42:10.413899+0000","last_clean":"2023-10-26T01:42:10.413899+0000","last_became_active":"2023-10-26T01:42:10.412562+0000","last_became_peered":"2023-10-26T01:42:10.412562+0000","last_unstale":"2023-10-26T01:42:10.413899+0000","last_undegraded":"2023-10-26T01:42:10.413899+0000","last_fullsized":"2023-10-26T01:42:10.413899+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:13:41.517039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.412170+0000","last_change":"2023-10-26T01:42:10.411551+0000","last_active":"2023-10-26T01:42:10.412170+0000","last_peered":"2023-10-26T01:42:10.412170+0000","last_clean":"2023-10-26T01:42:10.412170+0000","last_became_active":"2023-10-26T01:42:10.411310+0000","last_became_peered":"2023-10-26T01:42:10.411310+0000","last_unstale":"2023-10-26T01:42:10.412170+0000","last_undegraded":"2023-10-26T01:42:10.412170+0000","last_fullsized":"2023-10-26T01:42:10.412170+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:18:20.465446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.413991+0000","last_change":"2023-10-26T01:42:10.413912+0000","last_active":"2023-10-26T01:42:10.413991+0000","last_peered":"2023-10-26T01:42:10.413991+0000","last_clean":"2023-10-26T01:42:10.413991+0000","last_became_active":"2023-10-26T01:42:10.412939+0000","last_became_peered":"2023-10-26T01:42:10.412939+0000","last_unstale":"2023-10-26T01:42:10.413991+0000","last_undegraded":"2023-10-26T01:42:10.413991+0000","last_fullsized":"2023-10-26T01:42:10.413991+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:53:53.673059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":11,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-26T01:42:11.913515+0000","last_change":"2023-10-26T01:42:10.392593+0000","last_active":"2023-10-26T01:42:11.913515+0000","last_peered":"2023-10-26T01:42:11.913515+0000","last_clean":"2023-10-26T01:42:11.913515+0000","last_became_active":"2023-10-26T01:42:10.392343+0000","last_became_peered":"2023-10-26T01:42:10.392343+0000","last_unstale":"2023-10-26T01:42:11.913515+0000","last_undegraded":"2023-10-26T01:42:11.913515+0000","last_fullsized":"2023-10-26T01:42:11.913515+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:07:00.552083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.415480+0000","last_change":"2023-10-26T01:42:10.414646+0000","last_active":"2023-10-26T01:42:10.415480+0000","last_peered":"2023-10-26T01:42:10.415480+0000","last_clean":"2023-10-26T01:42:10.415480+0000","last_became_active":"2023-10-26T01:42:10.414039+0000","last_became_peered":"2023-10-26T01:42:10.414039+0000","last_unstale":"2023-10-26T01:42:10.415480+0000","last_undegraded":"2023-10-26T01:42:10.415480+0000","last_fullsized":"2023-10-26T01:42:10.415480+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:34:02.113140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.409468+0000","last_change":"2023-10-26T01:42:10.409369+0000","last_active":"2023-10-26T01:42:10.409468+0000","last_peered":"2023-10-26T01:42:10.409468+0000","last_clean":"2023-10-26T01:42:10.409468+0000","last_became_active":"2023-10-26T01:42:10.408982+0000","last_became_peered":"2023-10-26T01:42:10.408982+0000","last_unstale":"2023-10-26T01:42:10.409468+0000","last_undegraded":"2023-10-26T01:42:10.409468+0000","last_fullsized":"2023-10-26T01:42:10.409468+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:25:03.319950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.404213+0000","last_change":"2023-10-26T01:42:10.402771+0000","last_active":"2023-10-26T01:42:10.404213+0000","last_peered":"2023-10-26T01:42:10.404213+0000","last_clean":"2023-10-26T01:42:10.404213+0000","last_became_active":"2023-10-26T01:42:10.402145+0000","last_became_peered":"2023-10-26T01:42:10.402145+0000","last_unstale":"2023-10-26T01:42:10.404213+0000","last_undegraded":"2023-10-26T01:42:10.404213+0000","last_fullsized":"2023-10-26T01:42:10.404213+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:07:59.675904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.412071+0000","last_change":"2023-10-26T01:42:10.412000+0000","last_active":"2023-10-26T01:42:10.412071+0000","last_peered":"2023-10-26T01:42:10.412071+0000","last_clean":"2023-10-26T01:42:10.412071+0000","last_became_active":"2023-10-26T01:42:10.409415+0000","last_became_peered":"2023-10-26T01:42:10.409415+0000","last_unstale":"2023-10-26T01:42:10.412071+0000","last_undegraded":"2023-10-26T01:42:10.412071+0000","last_fullsized":"2023-10-26T01:42:10.412071+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:35:14.751935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.413394+0000","last_change":"2023-10-26T01:42:10.413303+0000","last_active":"2023-10-26T01:42:10.413394+0000","last_peered":"2023-10-26T01:42:10.413394+0000","last_clean":"2023-10-26T01:42:10.413394+0000","last_became_active":"2023-10-26T01:42:10.413039+0000","last_became_peered":"2023-10-26T01:42:10.413039+0000","last_unstale":"2023-10-26T01:42:10.413394+0000","last_undegraded":"2023-10-26T01:42:10.413394+0000","last_fullsized":"2023-10-26T01:42:10.413394+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:18:51.873615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.414532+0000","last_change":"2023-10-26T01:42:10.414266+0000","last_active":"2023-10-26T01:42:10.414532+0000","last_peered":"2023-10-26T01:42:10.414532+0000","last_clean":"2023-10-26T01:42:10.414532+0000","last_became_active":"2023-10-26T01:42:10.413243+0000","last_became_peered":"2023-10-26T01:42:10.413243+0000","last_unstale":"2023-10-26T01:42:10.414532+0000","last_undegraded":"2023-10-26T01:42:10.414532+0000","last_fullsized":"2023-10-26T01:42:10.414532+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:33:35.047366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.407349+0000","last_change":"2023-10-26T01:42:10.407281+0000","last_active":"2023-10-26T01:42:10.407349+0000","last_peered":"2023-10-26T01:42:10.407349+0000","last_clean":"2023-10-26T01:42:10.407349+0000","last_became_active":"2023-10-26T01:42:10.406994+0000","last_became_peered":"2023-10-26T01:42:10.406994+0000","last_unstale":"2023-10-26T01:42:10.407349+0000","last_undegraded":"2023-10-26T01:42:10.407349+0000","last_fullsized":"2023-10-26T01:42:10.407349+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:06:28.342273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.404232+0000","last_change":"2023-10-26T01:42:10.403011+0000","last_active":"2023-10-26T01:42:10.404232+0000","last_peered":"2023-10-26T01:42:10.404232+0000","last_clean":"2023-10-26T01:42:10.404232+0000","last_became_active":"2023-10-26T01:42:10.402510+0000","last_became_peered":"2023-10-26T01:42:10.402510+0000","last_unstale":"2023-10-26T01:42:10.404232+0000","last_undegraded":"2023-10-26T01:42:10.404232+0000","last_fullsized":"2023-10-26T01:42:10.404232+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:54:26.742574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.407451+0000","last_change":"2023-10-26T01:42:10.406780+0000","last_active":"2023-10-26T01:42:10.407451+0000","last_peered":"2023-10-26T01:42:10.407451+0000","last_clean":"2023-10-26T01:42:10.407451+0000","last_became_active":"2023-10-26T01:42:10.405395+0000","last_became_peered":"2023-10-26T01:42:10.405395+0000","last_unstale":"2023-10-26T01:42:10.407451+0000","last_undegraded":"2023-10-26T01:42:10.407451+0000","last_fullsized":"2023-10-26T01:42:10.407451+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:49:08.475248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.405126+0000","last_change":"2023-10-26T01:42:10.405058+0000","last_active":"2023-10-26T01:42:10.405126+0000","last_peered":"2023-10-26T01:42:10.405126+0000","last_clean":"2023-10-26T01:42:10.405126+0000","last_became_active":"2023-10-26T01:42:10.404775+0000","last_became_peered":"2023-10-26T01:42:10.404775+0000","last_unstale":"2023-10-26T01:42:10.405126+0000","last_undegraded":"2023-10-26T01:42:10.405126+0000","last_fullsized":"2023-10-26T01:42:10.405126+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:14:04.052827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.409374+0000","last_change":"2023-10-26T01:42:10.408742+0000","last_active":"2023-10-26T01:42:10.409374+0000","last_peered":"2023-10-26T01:42:10.409374+0000","last_clean":"2023-10-26T01:42:10.409374+0000","last_became_active":"2023-10-26T01:42:10.408480+0000","last_became_peered":"2023-10-26T01:42:10.408480+0000","last_unstale":"2023-10-26T01:42:10.409374+0000","last_undegraded":"2023-10-26T01:42:10.409374+0000","last_fullsized":"2023-10-26T01:42:10.409374+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:06:42.077030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.418147+0000","last_change":"2023-10-26T01:42:10.418040+0000","last_active":"2023-10-26T01:42:10.418147+0000","last_peered":"2023-10-26T01:42:10.418147+0000","last_clean":"2023-10-26T01:42:10.418147+0000","last_became_active":"2023-10-26T01:42:10.417382+0000","last_became_peered":"2023-10-26T01:42:10.417382+0000","last_unstale":"2023-10-26T01:42:10.418147+0000","last_undegraded":"2023-10-26T01:42:10.418147+0000","last_fullsized":"2023-10-26T01:42:10.418147+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:32:44.180501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.409495+0000","last_change":"2023-10-26T01:42:10.409084+0000","last_active":"2023-10-26T01:42:10.409495+0000","last_peered":"2023-10-26T01:42:10.409495+0000","last_clean":"2023-10-26T01:42:10.409495+0000","last_became_active":"2023-10-26T01:42:10.407811+0000","last_became_peered":"2023-10-26T01:42:10.407811+0000","last_unstale":"2023-10-26T01:42:10.409495+0000","last_undegraded":"2023-10-26T01:42:10.409495+0000","last_fullsized":"2023-10-26T01:42:10.409495+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:15:48.914492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.404658+0000","last_change":"2023-10-26T01:42:10.400644+0000","last_active":"2023-10-26T01:42:10.404658+0000","last_peered":"2023-10-26T01:42:10.404658+0000","last_clean":"2023-10-26T01:42:10.404658+0000","last_became_active":"2023-10-26T01:42:10.400297+0000","last_became_peered":"2023-10-26T01:42:10.400297+0000","last_unstale":"2023-10-26T01:42:10.404658+0000","last_undegraded":"2023-10-26T01:42:10.404658+0000","last_fullsized":"2023-10-26T01:42:10.404658+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:42:11.459436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.412492+0000","last_change":"2023-10-26T01:42:10.412094+0000","last_active":"2023-10-26T01:42:10.412492+0000","last_peered":"2023-10-26T01:42:10.412492+0000","last_clean":"2023-10-26T01:42:10.412492+0000","last_became_active":"2023-10-26T01:42:10.411765+0000","last_became_peered":"2023-10-26T01:42:10.411765+0000","last_unstale":"2023-10-26T01:42:10.412492+0000","last_undegraded":"2023-10-26T01:42:10.412492+0000","last_fullsized":"2023-10-26T01:42:10.412492+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:57:16.014827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.404491+0000","last_change":"2023-10-26T01:42:10.404414+0000","last_active":"2023-10-26T01:42:10.404491+0000","last_peered":"2023-10-26T01:42:10.404491+0000","last_clean":"2023-10-26T01:42:10.404491+0000","last_became_active":"2023-10-26T01:42:10.404167+0000","last_became_peered":"2023-10-26T01:42:10.404167+0000","last_unstale":"2023-10-26T01:42:10.404491+0000","last_undegraded":"2023-10-26T01:42:10.404491+0000","last_fullsized":"2023-10-26T01:42:10.404491+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:13:55.827244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.409430+0000","last_change":"2023-10-26T01:42:10.405918+0000","last_active":"2023-10-26T01:42:10.409430+0000","last_peered":"2023-10-26T01:42:10.409430+0000","last_clean":"2023-10-26T01:42:10.409430+0000","last_became_active":"2023-10-26T01:42:10.405198+0000","last_became_peered":"2023-10-26T01:42:10.405198+0000","last_unstale":"2023-10-26T01:42:10.409430+0000","last_undegraded":"2023-10-26T01:42:10.409430+0000","last_fullsized":"2023-10-26T01:42:10.409430+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:44:23.763936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.405342+0000","last_change":"2023-10-26T01:42:10.404482+0000","last_active":"2023-10-26T01:42:10.405342+0000","last_peered":"2023-10-26T01:42:10.405342+0000","last_clean":"2023-10-26T01:42:10.405342+0000","last_became_active":"2023-10-26T01:42:10.404133+0000","last_became_peered":"2023-10-26T01:42:10.404133+0000","last_unstale":"2023-10-26T01:42:10.405342+0000","last_undegraded":"2023-10-26T01:42:10.405342+0000","last_fullsized":"2023-10-26T01:42:10.405342+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:14:55.061732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.405927+0000","last_change":"2023-10-26T01:42:10.405213+0000","last_active":"2023-10-26T01:42:10.405927+0000","last_peered":"2023-10-26T01:42:10.405927+0000","last_clean":"2023-10-26T01:42:10.405927+0000","last_became_active":"2023-10-26T01:42:10.404895+0000","last_became_peered":"2023-10-26T01:42:10.404895+0000","last_unstale":"2023-10-26T01:42:10.405927+0000","last_undegraded":"2023-10-26T01:42:10.405927+0000","last_fullsized":"2023-10-26T01:42:10.405927+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:32:44.180501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.403780+0000","last_change":"2023-10-26T01:42:10.402030+0000","last_active":"2023-10-26T01:42:10.403780+0000","last_peered":"2023-10-26T01:42:10.403780+0000","last_clean":"2023-10-26T01:42:10.403780+0000","last_became_active":"2023-10-26T01:42:10.401794+0000","last_became_peered":"2023-10-26T01:42:10.401794+0000","last_unstale":"2023-10-26T01:42:10.403780+0000","last_undegraded":"2023-10-26T01:42:10.403780+0000","last_fullsized":"2023-10-26T01:42:10.403780+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:37:10.931971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.404685+0000","last_change":"2023-10-26T01:42:10.401647+0000","last_active":"2023-10-26T01:42:10.404685+0000","last_peered":"2023-10-26T01:42:10.404685+0000","last_clean":"2023-10-26T01:42:10.404685+0000","last_became_active":"2023-10-26T01:42:10.398276+0000","last_became_peered":"2023-10-26T01:42:10.398276+0000","last_unstale":"2023-10-26T01:42:10.404685+0000","last_undegraded":"2023-10-26T01:42:10.404685+0000","last_fullsized":"2023-10-26T01:42:10.404685+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T01:59:29.991970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":11,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-26T01:42:11.914521+0000","last_change":"2023-10-26T01:42:10.406731+0000","last_active":"2023-10-26T01:42:11.914521+0000","last_peered":"2023-10-26T01:42:11.914521+0000","last_clean":"2023-10-26T01:42:11.914521+0000","last_became_active":"2023-10-26T01:42:10.406419+0000","last_became_peered":"2023-10-26T01:42:10.406419+0000","last_unstale":"2023-10-26T01:42:11.914521+0000","last_undegraded":"2023-10-26T01:42:11.914521+0000","last_fullsized":"2023-10-26T01:42:11.914521+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:45:52.045328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.402335+0000","last_change":"2023-10-26T01:42:10.402260+0000","last_active":"2023-10-26T01:42:10.402335+0000","last_peered":"2023-10-26T01:42:10.402335+0000","last_clean":"2023-10-26T01:42:10.402335+0000","last_became_active":"2023-10-26T01:42:10.402034+0000","last_became_peered":"2023-10-26T01:42:10.402034+0000","last_unstale":"2023-10-26T01:42:10.402335+0000","last_undegraded":"2023-10-26T01:42:10.402335+0000","last_fullsized":"2023-10-26T01:42:10.402335+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:32:44.631370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.404756+0000","last_change":"2023-10-26T01:42:10.404673+0000","last_active":"2023-10-26T01:42:10.404756+0000","last_peered":"2023-10-26T01:42:10.404756+0000","last_clean":"2023-10-26T01:42:10.404756+0000","last_became_active":"2023-10-26T01:42:10.404309+0000","last_became_peered":"2023-10-26T01:42:10.404309+0000","last_unstale":"2023-10-26T01:42:10.404756+0000","last_undegraded":"2023-10-26T01:42:10.404756+0000","last_fullsized":"2023-10-26T01:42:10.404756+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:39:44.357515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1f","version":"165'4","reported_seq":15,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.420424+0000","last_change":"2023-10-26T01:42:09.405065+0000","last_active":"2023-10-26T01:42:10.420424+0000","last_peered":"2023-10-26T01:42:10.420424+0000","last_clean":"2023-10-26T01:42:10.420424+0000","last_became_active":"2023-10-26T01:42:09.404606+0000","last_became_peered":"2023-10-26T01:42:09.404606+0000","last_unstale":"2023-10-26T01:42:10.420424+0000","last_undegraded":"2023-10-26T01:42:10.420424+0000","last_fullsized":"2023-10-26T01:42:10.420424+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_clean_scrub_stamp":"2023-10-26T01:42:08.349732+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:53:10.091790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":11,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-26T01:42:11.914096+0000","last_change":"2023-10-26T01:42:10.392775+0000","last_active":"2023-10-26T01:42:11.914096+0000","last_peered":"2023-10-26T01:42:11.914096+0000","last_clean":"2023-10-26T01:42:11.914096+0000","last_became_active":"2023-10-26T01:42:10.392525+0000","last_became_peered":"2023-10-26T01:42:10.392525+0000","last_unstale":"2023-10-26T01:42:11.914096+0000","last_undegraded":"2023-10-26T01:42:11.914096+0000","last_fullsized":"2023-10-26T01:42:11.914096+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:42:11.459436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"165'1","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-26T01:42:11.913787+0000","last_change":"2023-10-26T01:42:09.402755+0000","last_active":"2023-10-26T01:42:11.913787+0000","last_peered":"2023-10-26T01:42:11.913787+0000","last_clean":"2023-10-26T01:42:11.913787+0000","last_became_active":"2023-10-26T01:42:09.402120+0000","last_became_peered":"2023-10-26T01:42:09.402120+0000","last_unstale":"2023-10-26T01:42:11.913787+0000","last_undegraded":"2023-10-26T01:42:11.913787+0000","last_fullsized":"2023-10-26T01:42:11.913787+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_clean_scrub_stamp":"2023-10-26T01:42:08.349732+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:13:26.673715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.413549+0000","last_change":"2023-10-26T01:42:10.412901+0000","last_active":"2023-10-26T01:42:10.413549+0000","last_peered":"2023-10-26T01:42:10.413549+0000","last_clean":"2023-10-26T01:42:10.413549+0000","last_became_active":"2023-10-26T01:42:10.411251+0000","last_became_peered":"2023-10-26T01:42:10.411251+0000","last_unstale":"2023-10-26T01:42:10.413549+0000","last_undegraded":"2023-10-26T01:42:10.413549+0000","last_fullsized":"2023-10-26T01:42:10.413549+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:23:53.405275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1d","version":"165'2","reported_seq":13,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.415861+0000","last_change":"2023-10-26T01:42:09.393430+0000","last_active":"2023-10-26T01:42:10.415861+0000","last_peered":"2023-10-26T01:42:10.415861+0000","last_clean":"2023-10-26T01:42:10.415861+0000","last_became_active":"2023-10-26T01:42:09.392624+0000","last_became_peered":"2023-10-26T01:42:09.392624+0000","last_unstale":"2023-10-26T01:42:10.415861+0000","last_undegraded":"2023-10-26T01:42:10.415861+0000","last_fullsized":"2023-10-26T01:42:10.415861+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_clean_scrub_stamp":"2023-10-26T01:42:08.349732+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:57:15.003296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.413645+0000","last_change":"2023-10-26T01:42:10.413497+0000","last_active":"2023-10-26T01:42:10.413645+0000","last_peered":"2023-10-26T01:42:10.413645+0000","last_clean":"2023-10-26T01:42:10.413645+0000","last_became_active":"2023-10-26T01:42:10.410617+0000","last_became_peered":"2023-10-26T01:42:10.410617+0000","last_unstale":"2023-10-26T01:42:10.413645+0000","last_undegraded":"2023-10-26T01:42:10.413645+0000","last_fullsized":"2023-10-26T01:42:10.413645+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:52:56.148787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"165'2","reported_seq":13,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.416536+0000","last_change":"2023-10-26T01:42:09.394789+0000","last_active":"2023-10-26T01:42:10.416536+0000","last_peered":"2023-10-26T01:42:10.416536+0000","last_clean":"2023-10-26T01:42:10.416536+0000","last_became_active":"2023-10-26T01:42:09.389841+0000","last_became_peered":"2023-10-26T01:42:09.389841+0000","last_unstale":"2023-10-26T01:42:10.416536+0000","last_undegraded":"2023-10-26T01:42:10.416536+0000","last_fullsized":"2023-10-26T01:42:10.416536+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_clean_scrub_stamp":"2023-10-26T01:42:08.349732+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:31:32.863018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":11,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-26T01:42:11.914434+0000","last_change":"2023-10-26T01:42:10.407075+0000","last_active":"2023-10-26T01:42:11.914434+0000","last_peered":"2023-10-26T01:42:11.914434+0000","last_clean":"2023-10-26T01:42:11.914434+0000","last_became_active":"2023-10-26T01:42:10.404574+0000","last_became_peered":"2023-10-26T01:42:10.404574+0000","last_unstale":"2023-10-26T01:42:11.914434+0000","last_undegraded":"2023-10-26T01:42:11.914434+0000","last_fullsized":"2023-10-26T01:42:11.914434+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:15:48.914492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1b","version":"165'1","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.417499+0000","last_change":"2023-10-26T01:42:09.405617+0000","last_active":"2023-10-26T01:42:10.417499+0000","last_peered":"2023-10-26T01:42:10.417499+0000","last_clean":"2023-10-26T01:42:10.417499+0000","last_became_active":"2023-10-26T01:42:09.405323+0000","last_became_peered":"2023-10-26T01:42:09.405323+0000","last_unstale":"2023-10-26T01:42:10.417499+0000","last_undegraded":"2023-10-26T01:42:10.417499+0000","last_fullsized":"2023-10-26T01:42:10.417499+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_clean_scrub_stamp":"2023-10-26T01:42:08.349732+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:42:15.440450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.410268+0000","last_change":"2023-10-26T01:42:10.410187+0000","last_active":"2023-10-26T01:42:10.410268+0000","last_peered":"2023-10-26T01:42:10.410268+0000","last_clean":"2023-10-26T01:42:10.410268+0000","last_became_active":"2023-10-26T01:42:10.408727+0000","last_became_peered":"2023-10-26T01:42:10.408727+0000","last_unstale":"2023-10-26T01:42:10.410268+0000","last_undegraded":"2023-10-26T01:42:10.410268+0000","last_fullsized":"2023-10-26T01:42:10.410268+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:09:20.791203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1a","version":"165'1","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.416333+0000","last_change":"2023-10-26T01:42:09.404429+0000","last_active":"2023-10-26T01:42:10.416333+0000","last_peered":"2023-10-26T01:42:10.416333+0000","last_clean":"2023-10-26T01:42:10.416333+0000","last_became_active":"2023-10-26T01:42:09.403818+0000","last_became_peered":"2023-10-26T01:42:09.403818+0000","last_unstale":"2023-10-26T01:42:10.416333+0000","last_undegraded":"2023-10-26T01:42:10.416333+0000","last_fullsized":"2023-10-26T01:42:10.416333+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_clean_scrub_stamp":"2023-10-26T01:42:08.349732+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:06:41.065499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":11,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-26T01:42:11.916564+0000","last_change":"2023-10-26T01:42:10.405535+0000","last_active":"2023-10-26T01:42:11.916564+0000","last_peered":"2023-10-26T01:42:11.916564+0000","last_clean":"2023-10-26T01:42:11.916564+0000","last_became_active":"2023-10-26T01:42:10.405239+0000","last_became_peered":"2023-10-26T01:42:10.405239+0000","last_unstale":"2023-10-26T01:42:11.916564+0000","last_undegraded":"2023-10-26T01:42:11.916564+0000","last_fullsized":"2023-10-26T01:42:11.916564+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:09:20.791203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":11,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.402971+0000","last_change":"2023-10-26T01:42:09.403856+0000","last_active":"2023-10-26T01:42:10.402971+0000","last_peered":"2023-10-26T01:42:10.402971+0000","last_clean":"2023-10-26T01:42:10.402971+0000","last_became_active":"2023-10-26T01:42:09.403458+0000","last_became_peered":"2023-10-26T01:42:09.403458+0000","last_unstale":"2023-10-26T01:42:10.402971+0000","last_undegraded":"2023-10-26T01:42:10.402971+0000","last_fullsized":"2023-10-26T01:42:10.402971+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_clean_scrub_stamp":"2023-10-26T01:42:08.349732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:58:33.483761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.412150+0000","last_change":"2023-10-26T01:42:10.411361+0000","last_active":"2023-10-26T01:42:10.412150+0000","last_peered":"2023-10-26T01:42:10.412150+0000","last_clean":"2023-10-26T01:42:10.412150+0000","last_became_active":"2023-10-26T01:42:10.406491+0000","last_became_peered":"2023-10-26T01:42:10.406491+0000","last_unstale":"2023-10-26T01:42:10.412150+0000","last_undegraded":"2023-10-26T01:42:10.412150+0000","last_fullsized":"2023-10-26T01:42:10.412150+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:56:34.881338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":11,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.394545+0000","last_change":"2023-10-26T01:42:09.405052+0000","last_active":"2023-10-26T01:42:10.394545+0000","last_peered":"2023-10-26T01:42:10.394545+0000","last_clean":"2023-10-26T01:42:10.394545+0000","last_became_active":"2023-10-26T01:42:09.404816+0000","last_became_peered":"2023-10-26T01:42:09.404816+0000","last_unstale":"2023-10-26T01:42:10.394545+0000","last_undegraded":"2023-10-26T01:42:10.394545+0000","last_fullsized":"2023-10-26T01:42:10.394545+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_clean_scrub_stamp":"2023-10-26T01:42:08.349732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:39:43.345984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.405063+0000","last_change":"2023-10-26T01:42:10.402404+0000","last_active":"2023-10-26T01:42:10.405063+0000","last_peered":"2023-10-26T01:42:10.405063+0000","last_clean":"2023-10-26T01:42:10.405063+0000","last_became_active":"2023-10-26T01:42:10.402174+0000","last_became_peered":"2023-10-26T01:42:10.402174+0000","last_unstale":"2023-10-26T01:42:10.405063+0000","last_undegraded":"2023-10-26T01:42:10.405063+0000","last_fullsized":"2023-10-26T01:42:10.405063+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:50:16.014300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":12,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-26T01:42:11.914775+0000","last_change":"2023-10-26T01:42:09.398301+0000","last_active":"2023-10-26T01:42:11.914775+0000","last_peered":"2023-10-26T01:42:11.914775+0000","last_clean":"2023-10-26T01:42:11.914775+0000","last_became_active":"2023-10-26T01:42:09.394569+0000","last_became_peered":"2023-10-26T01:42:09.394569+0000","last_unstale":"2023-10-26T01:42:11.914775+0000","last_undegraded":"2023-10-26T01:42:11.914775+0000","last_fullsized":"2023-10-26T01:42:11.914775+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_clean_scrub_stamp":"2023-10-26T01:42:08.349732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:15:18.889535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.417956+0000","last_change":"2023-10-26T01:42:10.417854+0000","last_active":"2023-10-26T01:42:10.417956+0000","last_peered":"2023-10-26T01:42:10.417956+0000","last_clean":"2023-10-26T01:42:10.417956+0000","last_became_active":"2023-10-26T01:42:10.417051+0000","last_became_peered":"2023-10-26T01:42:10.417051+0000","last_unstale":"2023-10-26T01:42:10.417956+0000","last_undegraded":"2023-10-26T01:42:10.417956+0000","last_fullsized":"2023-10-26T01:42:10.417956+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:59:00.597519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":11,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.394359+0000","last_change":"2023-10-26T01:42:09.405067+0000","last_active":"2023-10-26T01:42:10.394359+0000","last_peered":"2023-10-26T01:42:10.394359+0000","last_clean":"2023-10-26T01:42:10.394359+0000","last_became_active":"2023-10-26T01:42:09.404699+0000","last_became_peered":"2023-10-26T01:42:09.404699+0000","last_unstale":"2023-10-26T01:42:10.394359+0000","last_undegraded":"2023-10-26T01:42:10.394359+0000","last_fullsized":"2023-10-26T01:42:10.394359+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_clean_scrub_stamp":"2023-10-26T01:42:08.349732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:14:35.577148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.403154+0000","last_change":"2023-10-26T01:42:10.403079+0000","last_active":"2023-10-26T01:42:10.403154+0000","last_peered":"2023-10-26T01:42:10.403154+0000","last_clean":"2023-10-26T01:42:10.403154+0000","last_became_active":"2023-10-26T01:42:10.402817+0000","last_became_peered":"2023-10-26T01:42:10.402817+0000","last_unstale":"2023-10-26T01:42:10.403154+0000","last_undegraded":"2023-10-26T01:42:10.403154+0000","last_fullsized":"2023-10-26T01:42:10.403154+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:49:22.193228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.14","version":"165'6","reported_seq":17,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.419144+0000","last_change":"2023-10-26T01:42:09.398071+0000","last_active":"2023-10-26T01:42:10.419144+0000","last_peered":"2023-10-26T01:42:10.419144+0000","last_clean":"2023-10-26T01:42:10.419144+0000","last_became_active":"2023-10-26T01:42:09.397653+0000","last_became_peered":"2023-10-26T01:42:09.397653+0000","last_unstale":"2023-10-26T01:42:10.419144+0000","last_undegraded":"2023-10-26T01:42:10.419144+0000","last_fullsized":"2023-10-26T01:42:10.419144+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_clean_scrub_stamp":"2023-10-26T01:42:08.349732+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:25:44.055005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.409947+0000","last_change":"2023-10-26T01:42:10.409834+0000","last_active":"2023-10-26T01:42:10.409947+0000","last_peered":"2023-10-26T01:42:10.409947+0000","last_clean":"2023-10-26T01:42:10.409947+0000","last_became_active":"2023-10-26T01:42:10.409211+0000","last_became_peered":"2023-10-26T01:42:10.409211+0000","last_unstale":"2023-10-26T01:42:10.409947+0000","last_undegraded":"2023-10-26T01:42:10.409947+0000","last_fullsized":"2023-10-26T01:42:10.409947+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:01:03.396910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.13","version":"165'2","reported_seq":13,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.411134+0000","last_change":"2023-10-26T01:42:09.398259+0000","last_active":"2023-10-26T01:42:10.411134+0000","last_peered":"2023-10-26T01:42:10.411134+0000","last_clean":"2023-10-26T01:42:10.411134+0000","last_became_active":"2023-10-26T01:42:09.397928+0000","last_became_peered":"2023-10-26T01:42:09.397928+0000","last_unstale":"2023-10-26T01:42:10.411134+0000","last_undegraded":"2023-10-26T01:42:10.411134+0000","last_fullsized":"2023-10-26T01:42:10.411134+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_clean_scrub_stamp":"2023-10-26T01:42:08.349732+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:21:58.503627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":11,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-26T01:42:11.915918+0000","last_change":"2023-10-26T01:42:10.393830+0000","last_active":"2023-10-26T01:42:11.915918+0000","last_peered":"2023-10-26T01:42:11.915918+0000","last_clean":"2023-10-26T01:42:11.915918+0000","last_became_active":"2023-10-26T01:42:10.393326+0000","last_became_peered":"2023-10-26T01:42:10.393326+0000","last_unstale":"2023-10-26T01:42:11.915918+0000","last_undegraded":"2023-10-26T01:42:11.915918+0000","last_fullsized":"2023-10-26T01:42:11.915918+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:44:45.588628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":11,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.391177+0000","last_change":"2023-10-26T01:42:09.402031+0000","last_active":"2023-10-26T01:42:10.391177+0000","last_peered":"2023-10-26T01:42:10.391177+0000","last_clean":"2023-10-26T01:42:10.391177+0000","last_became_active":"2023-10-26T01:42:09.401444+0000","last_became_peered":"2023-10-26T01:42:09.401444+0000","last_unstale":"2023-10-26T01:42:10.391177+0000","last_undegraded":"2023-10-26T01:42:10.391177+0000","last_fullsized":"2023-10-26T01:42:10.391177+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_clean_scrub_stamp":"2023-10-26T01:42:08.349732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:13:26.673715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.413352+0000","last_change":"2023-10-26T01:42:10.413105+0000","last_active":"2023-10-26T01:42:10.413352+0000","last_peered":"2023-10-26T01:42:10.413352+0000","last_clean":"2023-10-26T01:42:10.413352+0000","last_became_active":"2023-10-26T01:42:10.409733+0000","last_became_peered":"2023-10-26T01:42:10.409733+0000","last_unstale":"2023-10-26T01:42:10.413352+0000","last_undegraded":"2023-10-26T01:42:10.413352+0000","last_fullsized":"2023-10-26T01:42:10.413352+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:35:44.412930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":11,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.392695+0000","last_change":"2023-10-26T01:42:09.389251+0000","last_active":"2023-10-26T01:42:10.392695+0000","last_peered":"2023-10-26T01:42:10.392695+0000","last_clean":"2023-10-26T01:42:10.392695+0000","last_became_active":"2023-10-26T01:42:09.388927+0000","last_became_peered":"2023-10-26T01:42:09.388927+0000","last_unstale":"2023-10-26T01:42:10.392695+0000","last_undegraded":"2023-10-26T01:42:10.392695+0000","last_fullsized":"2023-10-26T01:42:10.392695+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_clean_scrub_stamp":"2023-10-26T01:42:08.349732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:49:04.947399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.403260+0000","last_change":"2023-10-26T01:42:10.403097+0000","last_active":"2023-10-26T01:42:10.403260+0000","last_peered":"2023-10-26T01:42:10.403260+0000","last_clean":"2023-10-26T01:42:10.403260+0000","last_became_active":"2023-10-26T01:42:10.402206+0000","last_became_peered":"2023-10-26T01:42:10.402206+0000","last_unstale":"2023-10-26T01:42:10.403260+0000","last_undegraded":"2023-10-26T01:42:10.403260+0000","last_fullsized":"2023-10-26T01:42:10.403260+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:01:33.828985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"165'8","reported_seq":19,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.417426+0000","last_change":"2023-10-26T01:42:09.408422+0000","last_active":"2023-10-26T01:42:10.417426+0000","last_peered":"2023-10-26T01:42:10.417426+0000","last_clean":"2023-10-26T01:42:10.417426+0000","last_became_active":"2023-10-26T01:42:09.408022+0000","last_became_peered":"2023-10-26T01:42:09.408022+0000","last_unstale":"2023-10-26T01:42:10.417426+0000","last_undegraded":"2023-10-26T01:42:10.417426+0000","last_fullsized":"2023-10-26T01:42:10.417426+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_clean_scrub_stamp":"2023-10-26T01:42:08.349732+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:07:20.903387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.412585+0000","last_change":"2023-10-26T01:42:10.410896+0000","last_active":"2023-10-26T01:42:10.412585+0000","last_peered":"2023-10-26T01:42:10.412585+0000","last_clean":"2023-10-26T01:42:10.412585+0000","last_became_active":"2023-10-26T01:42:10.410191+0000","last_became_peered":"2023-10-26T01:42:10.410191+0000","last_unstale":"2023-10-26T01:42:10.412585+0000","last_undegraded":"2023-10-26T01:42:10.412585+0000","last_fullsized":"2023-10-26T01:42:10.412585+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:18:20.465446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"165'1","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.416450+0000","last_change":"2023-10-26T01:42:09.400503+0000","last_active":"2023-10-26T01:42:10.416450+0000","last_peered":"2023-10-26T01:42:10.416450+0000","last_clean":"2023-10-26T01:42:10.416450+0000","last_became_active":"2023-10-26T01:42:09.397318+0000","last_became_peered":"2023-10-26T01:42:09.397318+0000","last_unstale":"2023-10-26T01:42:10.416450+0000","last_undegraded":"2023-10-26T01:42:10.416450+0000","last_fullsized":"2023-10-26T01:42:10.416450+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_clean_scrub_stamp":"2023-10-26T01:42:08.349732+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:09:40.041614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.413458+0000","last_change":"2023-10-26T01:42:10.412058+0000","last_active":"2023-10-26T01:42:10.413458+0000","last_peered":"2023-10-26T01:42:10.413458+0000","last_clean":"2023-10-26T01:42:10.413458+0000","last_became_active":"2023-10-26T01:42:10.411703+0000","last_became_peered":"2023-10-26T01:42:10.411703+0000","last_unstale":"2023-10-26T01:42:10.413458+0000","last_undegraded":"2023-10-26T01:42:10.413458+0000","last_fullsized":"2023-10-26T01:42:10.413458+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:19:17.877977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"165'2","reported_seq":14,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-26T01:42:11.916832+0000","last_change":"2023-10-26T01:42:09.399534+0000","last_active":"2023-10-26T01:42:11.916832+0000","last_peered":"2023-10-26T01:42:11.916832+0000","last_clean":"2023-10-26T01:42:11.916832+0000","last_became_active":"2023-10-26T01:42:09.399276+0000","last_became_peered":"2023-10-26T01:42:09.399276+0000","last_unstale":"2023-10-26T01:42:11.916832+0000","last_undegraded":"2023-10-26T01:42:11.916832+0000","last_fullsized":"2023-10-26T01:42:11.916832+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_clean_scrub_stamp":"2023-10-26T01:42:08.349732+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:43:16.437117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.412871+0000","last_change":"2023-10-26T01:42:10.411633+0000","last_active":"2023-10-26T01:42:10.412871+0000","last_peered":"2023-10-26T01:42:10.412871+0000","last_clean":"2023-10-26T01:42:10.412871+0000","last_became_active":"2023-10-26T01:42:10.408927+0000","last_became_peered":"2023-10-26T01:42:10.408927+0000","last_unstale":"2023-10-26T01:42:10.412871+0000","last_undegraded":"2023-10-26T01:42:10.412871+0000","last_fullsized":"2023-10-26T01:42:10.412871+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:19:17.045067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.d","version":"165'1","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.418288+0000","last_change":"2023-10-26T01:42:09.401844+0000","last_active":"2023-10-26T01:42:10.418288+0000","last_peered":"2023-10-26T01:42:10.418288+0000","last_clean":"2023-10-26T01:42:10.418288+0000","last_became_active":"2023-10-26T01:42:09.401493+0000","last_became_peered":"2023-10-26T01:42:09.401493+0000","last_unstale":"2023-10-26T01:42:10.418288+0000","last_undegraded":"2023-10-26T01:42:10.418288+0000","last_fullsized":"2023-10-26T01:42:10.418288+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_clean_scrub_stamp":"2023-10-26T01:42:08.349732+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:47:22.909451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.409451+0000","last_change":"2023-10-26T01:42:10.409213+0000","last_active":"2023-10-26T01:42:10.409451+0000","last_peered":"2023-10-26T01:42:10.409451+0000","last_clean":"2023-10-26T01:42:10.409451+0000","last_became_active":"2023-10-26T01:42:10.408480+0000","last_became_peered":"2023-10-26T01:42:10.408480+0000","last_unstale":"2023-10-26T01:42:10.409451+0000","last_undegraded":"2023-10-26T01:42:10.409451+0000","last_fullsized":"2023-10-26T01:42:10.409451+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:00:02.415277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":12,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-26T01:42:11.914507+0000","last_change":"2023-10-26T01:42:09.397288+0000","last_active":"2023-10-26T01:42:11.914507+0000","last_peered":"2023-10-26T01:42:11.914507+0000","last_clean":"2023-10-26T01:42:11.914507+0000","last_became_active":"2023-10-26T01:42:09.397049+0000","last_became_peered":"2023-10-26T01:42:09.397049+0000","last_unstale":"2023-10-26T01:42:11.914507+0000","last_undegraded":"2023-10-26T01:42:11.914507+0000","last_fullsized":"2023-10-26T01:42:11.914507+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_clean_scrub_stamp":"2023-10-26T01:42:08.349732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:01:02.385379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.406071+0000","last_change":"2023-10-26T01:42:10.405363+0000","last_active":"2023-10-26T01:42:10.406071+0000","last_peered":"2023-10-26T01:42:10.406071+0000","last_clean":"2023-10-26T01:42:10.406071+0000","last_became_active":"2023-10-26T01:42:10.404945+0000","last_became_peered":"2023-10-26T01:42:10.404945+0000","last_unstale":"2023-10-26T01:42:10.406071+0000","last_undegraded":"2023-10-26T01:42:10.406071+0000","last_fullsized":"2023-10-26T01:42:10.406071+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:06:18.765851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.b","version":"165'2","reported_seq":13,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.413816+0000","last_change":"2023-10-26T01:42:09.408188+0000","last_active":"2023-10-26T01:42:10.413816+0000","last_peered":"2023-10-26T01:42:10.413816+0000","last_clean":"2023-10-26T01:42:10.413816+0000","last_became_active":"2023-10-26T01:42:09.407952+0000","last_became_peered":"2023-10-26T01:42:09.407952+0000","last_unstale":"2023-10-26T01:42:10.413816+0000","last_undegraded":"2023-10-26T01:42:10.413816+0000","last_fullsized":"2023-10-26T01:42:10.413816+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_clean_scrub_stamp":"2023-10-26T01:42:08.349732+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:03:16.727164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.401345+0000","last_change":"2023-10-26T01:42:10.400634+0000","last_active":"2023-10-26T01:42:10.401345+0000","last_peered":"2023-10-26T01:42:10.401345+0000","last_clean":"2023-10-26T01:42:10.401345+0000","last_became_active":"2023-10-26T01:42:10.400402+0000","last_became_peered":"2023-10-26T01:42:10.400402+0000","last_unstale":"2023-10-26T01:42:10.401345+0000","last_undegraded":"2023-10-26T01:42:10.401345+0000","last_fullsized":"2023-10-26T01:42:10.401345+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:59:00.597519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":11,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.399537+0000","last_change":"2023-10-26T01:42:09.403636+0000","last_active":"2023-10-26T01:42:10.399537+0000","last_peered":"2023-10-26T01:42:10.399537+0000","last_clean":"2023-10-26T01:42:10.399537+0000","last_became_active":"2023-10-26T01:42:09.403016+0000","last_became_peered":"2023-10-26T01:42:09.403016+0000","last_unstale":"2023-10-26T01:42:10.399537+0000","last_undegraded":"2023-10-26T01:42:10.399537+0000","last_fullsized":"2023-10-26T01:42:10.399537+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_clean_scrub_stamp":"2023-10-26T01:42:08.349732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:27:07.384772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.404626+0000","last_change":"2023-10-26T01:42:10.400073+0000","last_active":"2023-10-26T01:42:10.404626+0000","last_peered":"2023-10-26T01:42:10.404626+0000","last_clean":"2023-10-26T01:42:10.404626+0000","last_became_active":"2023-10-26T01:42:10.399836+0000","last_became_peered":"2023-10-26T01:42:10.399836+0000","last_unstale":"2023-10-26T01:42:10.404626+0000","last_undegraded":"2023-10-26T01:42:10.404626+0000","last_fullsized":"2023-10-26T01:42:10.404626+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:45:52.045328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":11,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.396335+0000","last_change":"2023-10-26T01:42:09.406759+0000","last_active":"2023-10-26T01:42:10.396335+0000","last_peered":"2023-10-26T01:42:10.396335+0000","last_clean":"2023-10-26T01:42:10.396335+0000","last_became_active":"2023-10-26T01:42:09.406519+0000","last_became_peered":"2023-10-26T01:42:09.406519+0000","last_unstale":"2023-10-26T01:42:10.396335+0000","last_undegraded":"2023-10-26T01:42:10.396335+0000","last_fullsized":"2023-10-26T01:42:10.396335+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_clean_scrub_stamp":"2023-10-26T01:42:08.349732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:30:06.187361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.412155+0000","last_change":"2023-10-26T01:42:10.411051+0000","last_active":"2023-10-26T01:42:10.412155+0000","last_peered":"2023-10-26T01:42:10.412155+0000","last_clean":"2023-10-26T01:42:10.412155+0000","last_became_active":"2023-10-26T01:42:10.408494+0000","last_became_peered":"2023-10-26T01:42:10.408494+0000","last_unstale":"2023-10-26T01:42:10.412155+0000","last_undegraded":"2023-10-26T01:42:10.412155+0000","last_fullsized":"2023-10-26T01:42:10.412155+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:27:08.396303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":11,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.392907+0000","last_change":"2023-10-26T01:42:09.402224+0000","last_active":"2023-10-26T01:42:10.392907+0000","last_peered":"2023-10-26T01:42:10.392907+0000","last_clean":"2023-10-26T01:42:10.392907+0000","last_became_active":"2023-10-26T01:42:09.401775+0000","last_became_peered":"2023-10-26T01:42:09.401775+0000","last_unstale":"2023-10-26T01:42:10.392907+0000","last_undegraded":"2023-10-26T01:42:10.392907+0000","last_fullsized":"2023-10-26T01:42:10.392907+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_clean_scrub_stamp":"2023-10-26T01:42:08.349732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:07:32.666411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.413584+0000","last_change":"2023-10-26T01:42:10.413516+0000","last_active":"2023-10-26T01:42:10.413584+0000","last_peered":"2023-10-26T01:42:10.413584+0000","last_clean":"2023-10-26T01:42:10.413584+0000","last_became_active":"2023-10-26T01:42:10.410033+0000","last_became_peered":"2023-10-26T01:42:10.410033+0000","last_unstale":"2023-10-26T01:42:10.413584+0000","last_undegraded":"2023-10-26T01:42:10.413584+0000","last_fullsized":"2023-10-26T01:42:10.413584+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:48:18.103481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.7","version":"165'2","reported_seq":13,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.419738+0000","last_change":"2023-10-26T01:42:09.408630+0000","last_active":"2023-10-26T01:42:10.419738+0000","last_peered":"2023-10-26T01:42:10.419738+0000","last_clean":"2023-10-26T01:42:10.419738+0000","last_became_active":"2023-10-26T01:42:09.408114+0000","last_became_peered":"2023-10-26T01:42:09.408114+0000","last_unstale":"2023-10-26T01:42:10.419738+0000","last_undegraded":"2023-10-26T01:42:10.419738+0000","last_fullsized":"2023-10-26T01:42:10.419738+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_clean_scrub_stamp":"2023-10-26T01:42:08.349732+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:52:58.503557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.412892+0000","last_change":"2023-10-26T01:42:10.412406+0000","last_active":"2023-10-26T01:42:10.412892+0000","last_peered":"2023-10-26T01:42:10.412892+0000","last_clean":"2023-10-26T01:42:10.412892+0000","last_became_active":"2023-10-26T01:42:10.409355+0000","last_became_peered":"2023-10-26T01:42:10.409355+0000","last_unstale":"2023-10-26T01:42:10.412892+0000","last_undegraded":"2023-10-26T01:42:10.412892+0000","last_fullsized":"2023-10-26T01:42:10.412892+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:22:49.493317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":11,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.389584+0000","last_change":"2023-10-26T01:42:09.403434+0000","last_active":"2023-10-26T01:42:10.389584+0000","last_peered":"2023-10-26T01:42:10.389584+0000","last_clean":"2023-10-26T01:42:10.389584+0000","last_became_active":"2023-10-26T01:42:09.403178+0000","last_became_peered":"2023-10-26T01:42:09.403178+0000","last_unstale":"2023-10-26T01:42:10.389584+0000","last_undegraded":"2023-10-26T01:42:10.389584+0000","last_fullsized":"2023-10-26T01:42:10.389584+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_clean_scrub_stamp":"2023-10-26T01:42:08.349732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:43:59.274726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":11,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-26T01:42:11.916246+0000","last_change":"2023-10-26T01:42:10.400259+0000","last_active":"2023-10-26T01:42:11.916246+0000","last_peered":"2023-10-26T01:42:11.916246+0000","last_clean":"2023-10-26T01:42:11.916246+0000","last_became_active":"2023-10-26T01:42:10.399855+0000","last_became_peered":"2023-10-26T01:42:10.399855+0000","last_unstale":"2023-10-26T01:42:11.916246+0000","last_undegraded":"2023-10-26T01:42:11.916246+0000","last_fullsized":"2023-10-26T01:42:11.916246+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:31:33.874549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":11,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.390843+0000","last_change":"2023-10-26T01:42:09.398662+0000","last_active":"2023-10-26T01:42:10.390843+0000","last_peered":"2023-10-26T01:42:10.390843+0000","last_clean":"2023-10-26T01:42:10.390843+0000","last_became_active":"2023-10-26T01:42:09.398290+0000","last_became_peered":"2023-10-26T01:42:09.398290+0000","last_unstale":"2023-10-26T01:42:10.390843+0000","last_undegraded":"2023-10-26T01:42:10.390843+0000","last_fullsized":"2023-10-26T01:42:10.390843+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_clean_scrub_stamp":"2023-10-26T01:42:08.349732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:00:43.327429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.410055+0000","last_change":"2023-10-26T01:42:10.408200+0000","last_active":"2023-10-26T01:42:10.410055+0000","last_peered":"2023-10-26T01:42:10.410055+0000","last_clean":"2023-10-26T01:42:10.410055+0000","last_became_active":"2023-10-26T01:42:10.407877+0000","last_became_peered":"2023-10-26T01:42:10.407877+0000","last_unstale":"2023-10-26T01:42:10.410055+0000","last_undegraded":"2023-10-26T01:42:10.410055+0000","last_fullsized":"2023-10-26T01:42:10.410055+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T01:52:06.939294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":11,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.390390+0000","last_change":"2023-10-26T01:42:09.398582+0000","last_active":"2023-10-26T01:42:10.390390+0000","last_peered":"2023-10-26T01:42:10.390390+0000","last_clean":"2023-10-26T01:42:10.390390+0000","last_became_active":"2023-10-26T01:42:09.398273+0000","last_became_peered":"2023-10-26T01:42:09.398273+0000","last_unstale":"2023-10-26T01:42:10.390390+0000","last_undegraded":"2023-10-26T01:42:10.390390+0000","last_fullsized":"2023-10-26T01:42:10.390390+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_clean_scrub_stamp":"2023-10-26T01:42:08.349732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:36:10.611662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.409391+0000","last_change":"2023-10-26T01:42:10.408929+0000","last_active":"2023-10-26T01:42:10.409391+0000","last_peered":"2023-10-26T01:42:10.409391+0000","last_clean":"2023-10-26T01:42:10.409391+0000","last_became_active":"2023-10-26T01:42:10.407830+0000","last_became_peered":"2023-10-26T01:42:10.407830+0000","last_unstale":"2023-10-26T01:42:10.409391+0000","last_undegraded":"2023-10-26T01:42:10.409391+0000","last_fullsized":"2023-10-26T01:42:10.409391+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:11:27.114346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":11,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.387828+0000","last_change":"2023-10-26T01:42:09.403023+0000","last_active":"2023-10-26T01:42:10.387828+0000","last_peered":"2023-10-26T01:42:10.387828+0000","last_clean":"2023-10-26T01:42:10.387828+0000","last_became_active":"2023-10-26T01:42:09.402490+0000","last_became_peered":"2023-10-26T01:42:09.402490+0000","last_unstale":"2023-10-26T01:42:10.387828+0000","last_undegraded":"2023-10-26T01:42:10.387828+0000","last_fullsized":"2023-10-26T01:42:10.387828+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_clean_scrub_stamp":"2023-10-26T01:42:08.349732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:52:58.503557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.408523+0000","last_change":"2023-10-26T01:42:10.407488+0000","last_active":"2023-10-26T01:42:10.408523+0000","last_peered":"2023-10-26T01:42:10.408523+0000","last_clean":"2023-10-26T01:42:10.408523+0000","last_became_active":"2023-10-26T01:42:10.407165+0000","last_became_peered":"2023-10-26T01:42:10.407165+0000","last_unstale":"2023-10-26T01:42:10.408523+0000","last_undegraded":"2023-10-26T01:42:10.408523+0000","last_fullsized":"2023-10-26T01:42:10.408523+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:04:16.931610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":11,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.395920+0000","last_change":"2023-10-26T01:42:09.408789+0000","last_active":"2023-10-26T01:42:10.395920+0000","last_peered":"2023-10-26T01:42:10.395920+0000","last_clean":"2023-10-26T01:42:10.395920+0000","last_became_active":"2023-10-26T01:42:09.408457+0000","last_became_peered":"2023-10-26T01:42:09.408457+0000","last_unstale":"2023-10-26T01:42:10.395920+0000","last_undegraded":"2023-10-26T01:42:10.395920+0000","last_fullsized":"2023-10-26T01:42:10.395920+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_clean_scrub_stamp":"2023-10-26T01:42:08.349732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:25:44.055005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.412416+0000","last_change":"2023-10-26T01:42:10.410701+0000","last_active":"2023-10-26T01:42:10.412416+0000","last_peered":"2023-10-26T01:42:10.412416+0000","last_clean":"2023-10-26T01:42:10.412416+0000","last_became_active":"2023-10-26T01:42:10.410464+0000","last_became_peered":"2023-10-26T01:42:10.410464+0000","last_unstale":"2023-10-26T01:42:10.412416+0000","last_undegraded":"2023-10-26T01:42:10.412416+0000","last_fullsized":"2023-10-26T01:42:10.412416+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:03:17.738695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":11,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.394814+0000","last_change":"2023-10-26T01:42:09.407254+0000","last_active":"2023-10-26T01:42:10.394814+0000","last_peered":"2023-10-26T01:42:10.394814+0000","last_clean":"2023-10-26T01:42:10.394814+0000","last_became_active":"2023-10-26T01:42:09.405035+0000","last_became_peered":"2023-10-26T01:42:09.405035+0000","last_unstale":"2023-10-26T01:42:10.394814+0000","last_undegraded":"2023-10-26T01:42:10.394814+0000","last_fullsized":"2023-10-26T01:42:10.394814+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_clean_scrub_stamp":"2023-10-26T01:42:08.349732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:39:55.222174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.412030+0000","last_change":"2023-10-26T01:42:10.410323+0000","last_active":"2023-10-26T01:42:10.412030+0000","last_peered":"2023-10-26T01:42:10.412030+0000","last_clean":"2023-10-26T01:42:10.412030+0000","last_became_active":"2023-10-26T01:42:10.409080+0000","last_became_peered":"2023-10-26T01:42:10.409080+0000","last_unstale":"2023-10-26T01:42:10.412030+0000","last_undegraded":"2023-10-26T01:42:10.412030+0000","last_fullsized":"2023-10-26T01:42:10.412030+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:19:17.877977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":12,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-26T01:42:11.913987+0000","last_change":"2023-10-26T01:42:09.402899+0000","last_active":"2023-10-26T01:42:11.913987+0000","last_peered":"2023-10-26T01:42:11.913987+0000","last_clean":"2023-10-26T01:42:11.913987+0000","last_became_active":"2023-10-26T01:42:09.402506+0000","last_became_peered":"2023-10-26T01:42:09.402506+0000","last_unstale":"2023-10-26T01:42:11.913987+0000","last_undegraded":"2023-10-26T01:42:11.913987+0000","last_fullsized":"2023-10-26T01:42:11.913987+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_clean_scrub_stamp":"2023-10-26T01:42:08.349732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:07:32.666411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":9,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.412133+0000","last_change":"2023-10-26T01:42:10.409711+0000","last_active":"2023-10-26T01:42:10.412133+0000","last_peered":"2023-10-26T01:42:10.412133+0000","last_clean":"2023-10-26T01:42:10.412133+0000","last_became_active":"2023-10-26T01:42:10.409103+0000","last_became_peered":"2023-10-26T01:42:10.409103+0000","last_unstale":"2023-10-26T01:42:10.412133+0000","last_undegraded":"2023-10-26T01:42:10.412133+0000","last_fullsized":"2023-10-26T01:42:10.412133+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:09.361263+0000","last_clean_scrub_stamp":"2023-10-26T01:42:09.361263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:30:07.198892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.15","version":"165'3","reported_seq":14,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-26T01:42:10.413356+0000","last_change":"2023-10-26T01:42:09.407639+0000","last_active":"2023-10-26T01:42:10.413356+0000","last_peered":"2023-10-26T01:42:10.413356+0000","last_clean":"2023-10-26T01:42:10.413356+0000","last_became_active":"2023-10-26T01:42:09.405387+0000","last_became_peered":"2023-10-26T01:42:09.405387+0000","last_unstale":"2023-10-26T01:42:10.413356+0000","last_undegraded":"2023-10-26T01:42:10.413356+0000","last_fullsized":"2023-10-26T01:42:10.413356+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:42:08.349732+0000","last_clean_scrub_stamp":"2023-10-26T01:42:08.349732+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:36:32.565534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":308,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-26T01:42:09.401278+0000","last_change":"2023-10-26T01:30:34.883484+0000","last_active":"2023-10-26T01:42:09.401278+0000","last_peered":"2023-10-26T01:42:09.401278+0000","last_clean":"2023-10-26T01:42:09.401278+0000","last_became_active":"2023-10-26T01:30:34.883071+0000","last_became_peered":"2023-10-26T01:30:34.883071+0000","last_unstale":"2023-10-26T01:42:09.401278+0000","last_undegraded":"2023-10-26T01:42:09.401278+0000","last_fullsized":"2023-10-26T01:42:09.401278+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:32.890096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:32.890096+0000","last_clean_scrub_stamp":"2023-10-26T01:30:32.890096+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:26:31.879452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":14,"seq":60129542287,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":903,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194408,"statfs":{"total":200000000,"available":199074687,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:41:54 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.83799999999999997,"15min":0.80600000000000005},"min":{"1min":0.65100000000000002,"5min":0.443,"15min":0.33000000000000002},"max":{"1min":1.27,"5min":1.6319999999999999,"15min":2.2549999999999999},"last":0.86199999999999999},{"interface":"front","average":{"1min":1.0740000000000001,"5min":0.98699999999999999,"15min":0.91800000000000004},"min":{"1min":0.74199999999999999,"5min":0.309,"15min":0.309},"max":{"1min":1.619,"5min":1.919,"15min":1.919},"last":1.4810000000000001}]},{"osd":1,"last update":"Thu Oct 26 01:41:54 2023","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.89500000000000002,"15min":0.91300000000000003},"min":{"1min":0.58599999999999997,"5min":0.39400000000000002,"15min":0.32300000000000001},"max":{"1min":1.429,"5min":1.569,"15min":4.4340000000000002},"last":0.80400000000000005},{"interface":"front","average":{"1min":1.117,"5min":1.046,"15min":1.012},"min":{"1min":0.71899999999999997,"5min":0.51500000000000001,"15min":0.374},"max":{"1min":1.5269999999999999,"5min":1.7070000000000001,"15min":2.3330000000000002},"last":0.98999999999999999}]},{"osd":2,"last update":"Thu Oct 26 01:41:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":1.0569999999999999,"15min":1.0640000000000001},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.39900000000000002},"max":{"1min":1.5389999999999999,"5min":1.643,"15min":4.2800000000000002},"last":1.05},{"interface":"front","average":{"1min":1.131,"5min":1.1559999999999999,"15min":1.121},"min":{"1min":0.65200000000000002,"5min":0.50900000000000001,"15min":0.49399999999999999},"max":{"1min":1.5149999999999999,"5min":1.8129999999999999,"15min":2.302},"last":1.093}]},{"osd":4,"last update":"Thu Oct 26 01:41:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1970000000000001,"5min":1.2190000000000001,"15min":1.1890000000000001},"min":{"1min":0.94899999999999995,"5min":0.755,"15min":0.66400000000000003},"max":{"1min":1.4890000000000001,"5min":1.732,"15min":2.1070000000000002},"last":1.3},{"interface":"front","average":{"1min":1.4059999999999999,"5min":1.3540000000000001,"15min":1.3740000000000001},"min":{"1min":1.083,"5min":0.81599999999999995,"15min":0.79900000000000004},"max":{"1min":1.716,"5min":1.7509999999999999,"15min":4.8250000000000002},"last":1.4430000000000001}]},{"osd":5,"last update":"Thu Oct 26 01:41:54 2023","interfaces":[{"interface":"back","average":{"1min":1.4159999999999999,"5min":1.3140000000000001,"15min":1.3200000000000001},"min":{"1min":0.87,"5min":0.79300000000000004,"15min":0.72499999999999998},"max":{"1min":1.6539999999999999,"5min":1.6899999999999999,"15min":4.9080000000000004},"last":1.2390000000000001},{"interface":"front","average":{"1min":1.3300000000000001,"5min":1.3360000000000001,"15min":1.2969999999999999},"min":{"1min":0.93300000000000005,"5min":0.79100000000000004,"15min":0.79100000000000004},"max":{"1min":1.631,"5min":1.9279999999999999,"15min":2.8980000000000001},"last":1.1990000000000001}]},{"osd":6,"last update":"Thu Oct 26 01:41:54 2023","interfaces":[{"interface":"back","average":{"1min":1.4239999999999999,"5min":1.3740000000000001,"15min":1.351},"min":{"1min":1.012,"5min":0.74199999999999999,"15min":0.70199999999999996},"max":{"1min":1.6499999999999999,"5min":2.0310000000000001,"15min":2.8530000000000002},"last":1.381},{"interface":"front","average":{"1min":1.4399999999999999,"5min":1.361,"15min":1.3},"min":{"1min":1.1850000000000001,"5min":0.755,"15min":0.73099999999999998},"max":{"1min":1.6339999999999999,"5min":1.806,"15min":2.181},"last":1.4039999999999999}]},{"osd":7,"last update":"Thu Oct 26 01:41:54 2023","interfaces":[{"interface":"back","average":{"1min":1.4339999999999999,"5min":1.395,"15min":1.3340000000000001},"min":{"1min":1.1779999999999999,"5min":0.78800000000000003,"15min":0.71999999999999997},"max":{"1min":1.724,"5min":2.024,"15min":2.0680000000000001},"last":1.143},{"interface":"front","average":{"1min":1.472,"5min":1.395,"15min":1.4079999999999999},"min":{"1min":1.153,"5min":0.73299999999999998,"15min":0.72199999999999998},"max":{"1min":1.7889999999999999,"5min":1.7889999999999999,"15min":4.8730000000000002},"last":1.337}]}]},{"osd":7,"up_from":13,"seq":55834574990,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1473,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193839,"statfs":{"total":200000000,"available":198491184,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:41:59 2023","interfaces":[{"interface":"back","average":{"1min":1.169,"5min":1.1699999999999999,"15min":1.127},"min":{"1min":0.749,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.48,"5min":2.2269999999999999,"15min":2.2269999999999999},"last":0.68899999999999995},{"interface":"front","average":{"1min":1.133,"5min":1.1799999999999999,"15min":1.1279999999999999},"min":{"1min":0.79700000000000004,"5min":0.67300000000000004,"15min":0.58699999999999997},"max":{"1min":1.679,"5min":2.282,"15min":2.6779999999999999},"last":1.278}]},{"osd":1,"last update":"Thu Oct 26 01:41:59 2023","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":1.1579999999999999,"15min":1.1319999999999999},"min":{"1min":0.91100000000000003,"5min":0.65600000000000003,"15min":0.65200000000000002},"max":{"1min":1.5209999999999999,"5min":1.8,"15min":2.8260000000000001},"last":1.1140000000000001},{"interface":"front","average":{"1min":1.216,"5min":1.2310000000000001,"15min":1.2070000000000001},"min":{"1min":0.71699999999999997,"5min":0.66500000000000004,"15min":0.61199999999999999},"max":{"1min":1.681,"5min":2.0059999999999998,"15min":3.2730000000000001},"last":0.97399999999999998}]},{"osd":2,"last update":"Thu Oct 26 01:41:59 2023","interfaces":[{"interface":"back","average":{"1min":1.163,"5min":1.2030000000000001,"15min":1.1539999999999999},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.58499999999999996},"max":{"1min":1.4430000000000001,"5min":1.851,"15min":2.218},"last":1.397},{"interface":"front","average":{"1min":1.2,"5min":1.2789999999999999,"15min":1.26},"min":{"1min":0.86599999999999999,"5min":0.82299999999999995,"15min":0.73099999999999998},"max":{"1min":1.472,"5min":2.234,"15min":3.0510000000000002},"last":1.3320000000000001}]},{"osd":3,"last update":"Thu Oct 26 01:41:59 2023","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.3160000000000001,"15min":1.2549999999999999},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.5840000000000001,"5min":2.1379999999999999,"15min":2.1379999999999999},"last":1.177},{"interface":"front","average":{"1min":1.288,"5min":1.383,"15min":1.3859999999999999},"min":{"1min":0.92200000000000004,"5min":0.71099999999999997,"15min":0.65700000000000003},"max":{"1min":1.52,"5min":2.3050000000000002,"15min":5.5039999999999996},"last":1.825}]},{"osd":4,"last update":"Thu Oct 26 01:41:59 2023","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.4199999999999999,"15min":1.371},"min":{"1min":1.034,"5min":0.81799999999999995,"15min":0.76100000000000001},"max":{"1min":1.671,"5min":2.54,"15min":2.77},"last":1.7769999999999999},{"interface":"front","average":{"1min":1.302,"5min":1.4119999999999999,"15min":1.405},"min":{"1min":0.95199999999999996,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.605,"5min":2.8660000000000001,"15min":3.1419999999999999},"last":1.597}]},{"osd":5,"last update":"Thu Oct 26 01:41:59 2023","interfaces":[{"interface":"back","average":{"1min":1.377,"5min":1.423,"15min":1.429},"min":{"1min":0.98399999999999999,"5min":0.435,"15min":0.435},"max":{"1min":1.7609999999999999,"5min":2.3639999999999999,"15min":5.734},"last":1.6910000000000001},{"interface":"front","average":{"1min":1.4379999999999999,"5min":1.508,"15min":1.492},"min":{"1min":1.1120000000000001,"5min":0.76400000000000001,"15min":0.51900000000000002},"max":{"1min":1.756,"5min":2.5169999999999999,"15min":2.8980000000000001},"last":2.056}]},{"osd":6,"last update":"Thu Oct 26 01:41:59 2023","interfaces":[{"interface":"back","average":{"1min":1.5009999999999999,"5min":1.5469999999999999,"15min":1.7210000000000001},"min":{"1min":1.1739999999999999,"5min":0.51000000000000001,"15min":0.47399999999999998},"max":{"1min":1.774,"5min":2.5259999999999998,"15min":11.577999999999999},"last":1.502},{"interface":"front","average":{"1min":1.522,"5min":1.6619999999999999,"15min":1.7749999999999999},"min":{"1min":1.3799999999999999,"5min":1.0640000000000001,"15min":0.55100000000000005},"max":{"1min":1.6719999999999999,"5min":2.7250000000000001,"15min":10.704000000000001},"last":1.6399999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574990,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":903,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194409,"statfs":{"total":200000000,"available":199075245,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:41:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1339999999999999,"5min":1.1279999999999999,"15min":1.139},"min":{"1min":0.69299999999999995,"5min":0.66600000000000004,"15min":0.60399999999999998},"max":{"1min":1.706,"5min":2.7370000000000001,"15min":2.7370000000000001},"last":0.95299999999999996},{"interface":"front","average":{"1min":1.1850000000000001,"5min":1.1599999999999999,"15min":1.135},"min":{"1min":0.74099999999999999,"5min":0.61399999999999999,"15min":0.59199999999999997},"max":{"1min":1.5720000000000001,"5min":4.4779999999999998,"15min":4.4779999999999998},"last":0.85199999999999998}]},{"osd":1,"last update":"Thu Oct 26 01:41:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.2150000000000001,"15min":1.1850000000000001},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.889,"5min":5.5359999999999996,"15min":5.5359999999999996},"last":1.125},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.262,"15min":1.2130000000000001},"min":{"1min":0.90300000000000002,"5min":0.67900000000000005,"15min":0.64000000000000001},"max":{"1min":1.5960000000000001,"5min":4.3579999999999997,"15min":4.3579999999999997},"last":1.018}]},{"osd":2,"last update":"Thu Oct 26 01:41:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.1479999999999999,"15min":1.1259999999999999},"min":{"1min":0.77400000000000002,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.556,"5min":2.367,"15min":3.7879999999999998},"last":1.175},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.3300000000000001,"15min":1.2869999999999999},"min":{"1min":1.0169999999999999,"5min":0.86799999999999999,"15min":0.68899999999999995},"max":{"1min":1.645,"5min":3.0470000000000002,"15min":3.0470000000000002},"last":1.3360000000000001}]},{"osd":3,"last update":"Thu Oct 26 01:41:54 2023","interfaces":[{"interface":"back","average":{"1min":1.373,"5min":1.284,"15min":1.2490000000000001},"min":{"1min":1.016,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.8200000000000001,"5min":3.266,"15min":4.4560000000000004},"last":1.0780000000000001},{"interface":"front","average":{"1min":1.383,"5min":1.347,"15min":1.3360000000000001},"min":{"1min":1.0780000000000001,"5min":0.78600000000000003,"15min":0.69499999999999995},"max":{"1min":1.6719999999999999,"5min":2.6909999999999998,"15min":2.6909999999999998},"last":1.248}]},{"osd":4,"last update":"Thu Oct 26 01:41:54 2023","interfaces":[{"interface":"back","average":{"1min":1.302,"5min":1.2669999999999999,"15min":1.2410000000000001},"min":{"1min":0.81200000000000006,"5min":0.53400000000000003,"15min":0.48399999999999999},"max":{"1min":1.7430000000000001,"5min":1.825,"15min":2.9489999999999998},"last":0.59899999999999998},{"interface":"front","average":{"1min":1.3080000000000001,"5min":1.3280000000000001,"15min":1.3380000000000001},"min":{"1min":0.88600000000000001,"5min":0.60699999999999998,"15min":0.59299999999999997},"max":{"1min":2.169,"5min":2.169,"15min":6.609},"last":0.88400000000000001}]},{"osd":5,"last update":"Thu Oct 26 01:41:54 2023","interfaces":[{"interface":"back","average":{"1min":1.516,"5min":1.5169999999999999,"15min":1.4850000000000001},"min":{"1min":1.23,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.9399999999999999,"5min":2.04,"15min":5.4370000000000003},"last":1.2869999999999999},{"interface":"front","average":{"1min":1.4570000000000001,"5min":1.464,"15min":1.4339999999999999},"min":{"1min":0.92200000000000004,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.8520000000000001,"5min":2.7759999999999998,"15min":2.7759999999999998},"last":0.69999999999999996}]},{"osd":7,"last update":"Thu Oct 26 01:41:54 2023","interfaces":[{"interface":"back","average":{"1min":1.4450000000000001,"5min":1.3999999999999999,"15min":1.4259999999999999},"min":{"1min":0.93500000000000005,"5min":0.58899999999999997,"15min":0.54100000000000004},"max":{"1min":1.984,"5min":1.984,"15min":11.804},"last":0.72399999999999998},{"interface":"front","average":{"1min":1.54,"5min":1.5449999999999999,"15min":1.544},"min":{"1min":1.1539999999999999,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.909,"5min":3.7650000000000001,"15min":11.541},"last":1.3899999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574990,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":903,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194408,"statfs":{"total":200000000,"available":199074755,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:41:53 2023","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.238,"15min":1.171},"min":{"1min":0.68700000000000006,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.724,"5min":3.8210000000000002,"15min":3.8210000000000002},"last":0.90200000000000002},{"interface":"front","average":{"1min":1.371,"5min":1.3029999999999999,"15min":1.2350000000000001},"min":{"1min":0.94599999999999995,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.609,"5min":2.9889999999999999,"15min":2.9889999999999999},"last":0.97199999999999998}]},{"osd":1,"last update":"Thu Oct 26 01:41:53 2023","interfaces":[{"interface":"back","average":{"1min":1.2969999999999999,"5min":1.1919999999999999,"15min":1.133},"min":{"1min":0.90400000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.6850000000000001,"5min":1.899,"15min":1.899},"last":0.71299999999999997},{"interface":"front","average":{"1min":1.194,"5min":1.133,"15min":1.1180000000000001},"min":{"1min":0.88700000000000001,"5min":0.626,"15min":0.60599999999999998},"max":{"1min":1.5569999999999999,"5min":2.4340000000000002,"15min":2.4340000000000002},"last":1.01}]},{"osd":2,"last update":"Thu Oct 26 01:41:53 2023","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.2170000000000001,"15min":1.1870000000000001},"min":{"1min":0.94899999999999995,"5min":0.71599999999999997,"15min":0.69299999999999995},"max":{"1min":1.613,"5min":1.796,"15min":1.8220000000000001},"last":1.2270000000000001},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.226,"15min":1.2070000000000001},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.63900000000000001},"max":{"1min":1.637,"5min":1.875,"15min":1.875},"last":1.3140000000000001}]},{"osd":3,"last update":"Thu Oct 26 01:41:53 2023","interfaces":[{"interface":"back","average":{"1min":1.226,"5min":1.2629999999999999,"15min":1.262},"min":{"1min":0.85599999999999998,"5min":0.72399999999999998,"15min":0.72299999999999998},"max":{"1min":1.587,"5min":1.7,"15min":1.835},"last":1.1060000000000001},{"interface":"front","average":{"1min":1.5860000000000001,"5min":1.323,"15min":1.3089999999999999},"min":{"1min":1.139,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":2.6669999999999998,"5min":2.6669999999999998,"15min":2.9249999999999998},"last":0.83099999999999996}]},{"osd":4,"last update":"Thu Oct 26 01:41:53 2023","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.21,"15min":1.22},"min":{"1min":0.85799999999999998,"5min":0.59599999999999997,"15min":0.49099999999999999},"max":{"1min":1.47,"5min":2.7770000000000001,"15min":2.7770000000000001},"last":1.272},{"interface":"front","average":{"1min":1.347,"5min":1.353,"15min":1.369},"min":{"1min":0.93100000000000005,"5min":0.57299999999999995,"15min":0.56200000000000006},"max":{"1min":1.8029999999999999,"5min":1.857,"15min":1.867},"last":1.0589999999999999}]},{"osd":6,"last update":"Thu Oct 26 01:41:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.3979999999999999,"15min":1.3680000000000001},"min":{"1min":1.0129999999999999,"5min":0.67500000000000004,"15min":0.51900000000000002},"max":{"1min":1.794,"5min":2.4590000000000001,"15min":2.8250000000000002},"last":1.3959999999999999},{"interface":"front","average":{"1min":1.4279999999999999,"5min":1.4219999999999999,"15min":1.379},"min":{"1min":0.97699999999999998,"5min":0.69499999999999995,"15min":0.61799999999999999},"max":{"1min":1.831,"5min":2.4830000000000001,"15min":2.4830000000000001},"last":1.1739999999999999}]},{"osd":7,"last update":"Thu Oct 26 01:41:53 2023","interfaces":[{"interface":"back","average":{"1min":1.472,"5min":1.5229999999999999,"15min":1.474},"min":{"1min":1.0589999999999999,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.8919999999999999,"5min":3.621,"15min":3.621},"last":1.355},{"interface":"front","average":{"1min":1.5529999999999999,"5min":1.534,"15min":1.4870000000000001},"min":{"1min":1.3009999999999999,"5min":0.73899999999999999,"15min":0.59499999999999997},"max":{"1min":1.8819999999999999,"5min":2.8010000000000002,"15min":2.8010000000000002},"last":1.446}]}]},{"osd":4,"up_from":13,"seq":55834574990,"num_pgs":12,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":903,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194408,"statfs":{"total":200000000,"available":199074777,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:41:45 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.1779999999999999,"15min":1.0549999999999999},"min":{"1min":0.84499999999999997,"5min":0.63500000000000001,"15min":0.53100000000000003},"max":{"1min":1.702,"5min":1.78,"15min":2.0289999999999999},"last":1.3009999999999999},{"interface":"front","average":{"1min":1.1579999999999999,"5min":1.139,"15min":1.0680000000000001},"min":{"1min":0.747,"5min":0.66300000000000003,"15min":0.60599999999999998},"max":{"1min":1.6000000000000001,"5min":1.921,"15min":2.6829999999999998},"last":1.1060000000000001}]},{"osd":1,"last update":"Thu Oct 26 01:41:45 2023","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.2,"15min":1.1579999999999999},"min":{"1min":0.92500000000000004,"5min":0.66700000000000004,"15min":0.56799999999999995},"max":{"1min":1.716,"5min":2.056,"15min":4.0519999999999996},"last":1.369},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.2849999999999999,"15min":1.1719999999999999},"min":{"1min":0.98599999999999999,"5min":0.69699999999999995,"15min":0.60499999999999998},"max":{"1min":1.5089999999999999,"5min":2.4390000000000001,"15min":2.4390000000000001},"last":1.165}]},{"osd":2,"last update":"Thu Oct 26 01:41:45 2023","interfaces":[{"interface":"back","average":{"1min":1.401,"5min":1.379,"15min":1.292},"min":{"1min":0.98699999999999999,"5min":0.66700000000000004,"15min":0.64300000000000002},"max":{"1min":1.6519999999999999,"5min":2.2040000000000002,"15min":5.6749999999999998},"last":1.222},{"interface":"front","average":{"1min":1.246,"5min":1.2490000000000001,"15min":1.175},"min":{"1min":0.997,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.5249999999999999,"5min":3.673,"15min":3.673},"last":1.0569999999999999}]},{"osd":3,"last update":"Thu Oct 26 01:41:45 2023","interfaces":[{"interface":"back","average":{"1min":1.4299999999999999,"5min":1.3580000000000001,"15min":1.244},"min":{"1min":0.91200000000000003,"5min":0.78900000000000003,"15min":0.70399999999999996},"max":{"1min":1.9630000000000001,"5min":2.5960000000000001,"15min":2.5960000000000001},"last":1.0129999999999999},{"interface":"front","average":{"1min":1.2909999999999999,"5min":1.2410000000000001,"15min":1.1910000000000001},"min":{"1min":0.93300000000000005,"5min":0.85399999999999998,"15min":0.69699999999999995},"max":{"1min":1.639,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":1.1399999999999999}]},{"osd":5,"last update":"Thu Oct 26 01:41:45 2023","interfaces":[{"interface":"back","average":{"1min":1.3009999999999999,"5min":1.302,"15min":1.286},"min":{"1min":0.86399999999999999,"5min":0.52100000000000002,"15min":0.46300000000000002},"max":{"1min":1.8660000000000001,"5min":2.774,"15min":2.774},"last":0.88600000000000001},{"interface":"front","average":{"1min":1.276,"5min":1.349,"15min":1.335},"min":{"1min":0.84099999999999997,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.8280000000000001,"5min":2.2850000000000001,"15min":5.8579999999999997},"last":0.94599999999999995}]},{"osd":6,"last update":"Thu Oct 26 01:41:45 2023","interfaces":[{"interface":"back","average":{"1min":1.345,"5min":1.361,"15min":1.357},"min":{"1min":0.92900000000000005,"5min":0.69199999999999995,"15min":0.59299999999999997},"max":{"1min":1.7430000000000001,"5min":2.262,"15min":12.343999999999999},"last":1.2569999999999999},{"interface":"front","average":{"1min":1.5700000000000001,"5min":1.5820000000000001,"15min":1.554},"min":{"1min":1.1060000000000001,"5min":0.57999999999999996,"15min":0.55000000000000004},"max":{"1min":2.0790000000000002,"5min":2.9020000000000001,"15min":12.605},"last":1.526}]},{"osd":7,"last update":"Thu Oct 26 01:41:45 2023","interfaces":[{"interface":"back","average":{"1min":1.5109999999999999,"5min":1.556,"15min":1.5169999999999999},"min":{"1min":0.98699999999999999,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.921,"5min":2.7200000000000002,"15min":5.774},"last":1.4830000000000001},{"interface":"front","average":{"1min":1.538,"5min":1.5329999999999999,"15min":1.49},"min":{"1min":1.238,"5min":0.79000000000000004,"15min":0.68000000000000005},"max":{"1min":1.7689999999999999,"5min":2.8149999999999999,"15min":4.6520000000000001},"last":1.206}]}]},{"osd":2,"up_from":13,"seq":55834574990,"num_pgs":16,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":896,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194415,"statfs":{"total":200000000,"available":199081552,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:41:57 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.79200000000000004,"15min":0.83999999999999997},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.308},"max":{"1min":1.2010000000000001,"5min":1.4850000000000001,"15min":6.3200000000000003},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.92200000000000004,"15min":0.94599999999999995},"min":{"1min":0.45900000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.4379999999999999,"5min":1.476,"15min":4.2619999999999996},"last":1.101}]},{"osd":1,"last update":"Thu Oct 26 01:41:57 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.83499999999999996,"15min":0.88600000000000001},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.2729999999999999,"5min":1.504,"15min":5.1260000000000003},"last":1.0169999999999999},{"interface":"front","average":{"1min":0.97999999999999998,"5min":0.97499999999999998,"15min":1.008},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.36599999999999999},"max":{"1min":1.4159999999999999,"5min":1.4870000000000001,"15min":6.1520000000000001},"last":0.85099999999999998}]},{"osd":3,"last update":"Thu Oct 26 01:42:02 2023","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":1.0329999999999999,"15min":0.98699999999999999},"min":{"1min":0.46800000000000003,"5min":0.435,"15min":0.435},"max":{"1min":1.6719999999999999,"5min":1.909,"15min":2.0640000000000001},"last":1.2390000000000001},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.0629999999999999,"15min":1.0589999999999999},"min":{"1min":0.51400000000000001,"5min":0.503,"15min":0.45300000000000001},"max":{"1min":1.506,"5min":1.6659999999999999,"15min":8.9019999999999992},"last":1.1719999999999999}]},{"osd":4,"last update":"Thu Oct 26 01:41:57 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.292,"15min":1.292},"min":{"1min":0.77300000000000002,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.6890000000000001,"5min":2.109,"15min":6.2670000000000003},"last":1.341},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.244,"15min":1.274},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.53,"5min":1.923,"15min":7.343},"last":1.3089999999999999}]},{"osd":5,"last update":"Thu Oct 26 01:41:57 2023","interfaces":[{"interface":"back","average":{"1min":1.3140000000000001,"5min":1.292,"15min":1.304},"min":{"1min":0.79000000000000004,"5min":0.76900000000000002,"15min":0.71899999999999997},"max":{"1min":1.726,"5min":2.3849999999999998,"15min":3.7170000000000001},"last":1.395},{"interface":"front","average":{"1min":1.359,"5min":1.2949999999999999,"15min":1.3069999999999999},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.65300000000000002},"max":{"1min":1.728,"5min":1.8140000000000001,"15min":6.3899999999999997},"last":0.95599999999999996}]},{"osd":6,"last update":"Thu Oct 26 01:41:57 2023","interfaces":[{"interface":"back","average":{"1min":1.335,"5min":1.367,"15min":1.361},"min":{"1min":0.81100000000000005,"5min":0.76300000000000001,"15min":0.70599999999999996},"max":{"1min":1.73,"5min":2.097,"15min":6.0490000000000004},"last":1.5640000000000001},{"interface":"front","average":{"1min":1.3720000000000001,"5min":1.3620000000000001,"15min":1.381},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.6910000000000001,"5min":2.5230000000000001,"15min":5.0650000000000004},"last":1.516}]},{"osd":7,"last update":"Thu Oct 26 01:41:57 2023","interfaces":[{"interface":"back","average":{"1min":1.4510000000000001,"5min":1.391,"15min":1.4059999999999999},"min":{"1min":0.85699999999999998,"5min":0.77900000000000003,"15min":0.69399999999999995},"max":{"1min":1.722,"5min":1.7949999999999999,"15min":6.4480000000000004},"last":1.2649999999999999},{"interface":"front","average":{"1min":1.425,"5min":1.4670000000000001,"15min":1.47},"min":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"max":{"1min":1.7490000000000001,"5min":2.1480000000000001,"15min":2.8439999999999999},"last":1.454}]}]},{"osd":1,"up_from":13,"seq":55834574990,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":904,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194408,"statfs":{"total":200000000,"available":199074195,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:41:52 2023","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.81699999999999995,"15min":0.76300000000000001},"min":{"1min":0.64400000000000002,"5min":0.36099999999999999,"15min":0.29499999999999998},"max":{"1min":1.3080000000000001,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":1.0820000000000001},{"interface":"front","average":{"1min":1.022,"5min":0.92300000000000004,"15min":0.84699999999999998},"min":{"1min":0.70899999999999996,"5min":0.435,"15min":0.36199999999999999},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.6399999999999999},"last":0.84199999999999997}]},{"osd":2,"last update":"Thu Oct 26 01:41:52 2023","interfaces":[{"interface":"back","average":{"1min":1.167,"5min":1.103,"15min":1.022},"min":{"1min":0.88900000000000001,"5min":0.52200000000000002,"15min":0.39600000000000002},"max":{"1min":1.492,"5min":1.8200000000000001,"15min":2.4830000000000001},"last":1.1899999999999999},{"interface":"front","average":{"1min":1.097,"5min":1.054,"15min":0.93100000000000005},"min":{"1min":0.74399999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.5049999999999999,"5min":1.772,"15min":1.8080000000000001},"last":1.002}]},{"osd":3,"last update":"Thu Oct 26 01:41:57 2023","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":1.0529999999999999,"15min":0.98899999999999999},"min":{"1min":0.84799999999999998,"5min":0.52900000000000003,"15min":0.39500000000000002},"max":{"1min":1.6100000000000001,"5min":1.6100000000000001,"15min":1.75},"last":1.1619999999999999},{"interface":"front","average":{"1min":1.125,"5min":1.0669999999999999,"15min":1.022},"min":{"1min":0.73899999999999999,"5min":0.55000000000000004,"15min":0.41299999999999998},"max":{"1min":1.476,"5min":1.5049999999999999,"15min":1.802},"last":1.2889999999999999}]},{"osd":4,"last update":"Thu Oct 26 01:41:52 2023","interfaces":[{"interface":"back","average":{"1min":1.3009999999999999,"5min":1.2629999999999999,"15min":1.2230000000000001},"min":{"1min":1.1499999999999999,"5min":0.79800000000000004,"15min":0.56200000000000006},"max":{"1min":1.5649999999999999,"5min":1.756,"15min":2.0659999999999998},"last":1.444},{"interface":"front","average":{"1min":1.333,"5min":1.2210000000000001,"15min":1.175},"min":{"1min":1.004,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.8420000000000001,"5min":1.8420000000000001,"15min":2.0270000000000001},"last":1.4239999999999999}]},{"osd":5,"last update":"Thu Oct 26 01:41:52 2023","interfaces":[{"interface":"back","average":{"1min":1.3899999999999999,"5min":1.2769999999999999,"15min":1.252},"min":{"1min":0.85999999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":1.8779999999999999},"last":1.242},{"interface":"front","average":{"1min":1.401,"5min":1.399,"15min":1.3580000000000001},"min":{"1min":1.099,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.6679999999999999,"5min":2.04,"15min":2.2200000000000002},"last":1.52}]},{"osd":6,"last update":"Thu Oct 26 01:41:52 2023","interfaces":[{"interface":"back","average":{"1min":1.347,"5min":1.341,"15min":1.306},"min":{"1min":1.024,"5min":0.75,"15min":0.68899999999999995},"max":{"1min":1.651,"5min":1.7709999999999999,"15min":2.3780000000000001},"last":1.383},{"interface":"front","average":{"1min":1.361,"5min":1.3480000000000001,"15min":1.2829999999999999},"min":{"1min":1.224,"5min":0.80700000000000005,"15min":0.70799999999999996},"max":{"1min":1.534,"5min":1.8819999999999999,"15min":1.984},"last":1.3360000000000001}]},{"osd":7,"last update":"Thu Oct 26 01:41:52 2023","interfaces":[{"interface":"back","average":{"1min":1.407,"5min":1.319,"15min":1.284},"min":{"1min":1.1619999999999999,"5min":0.76100000000000001,"15min":0.69699999999999995},"max":{"1min":1.635,"5min":1.835,"15min":2.6930000000000001},"last":1.49},{"interface":"front","average":{"1min":1.4630000000000001,"5min":1.4359999999999999,"15min":1.377},"min":{"1min":1.0960000000000001,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.75,"5min":1.9370000000000001,"15min":2.0190000000000001},"last":1.593}]}]},{"osd":0,"up_from":13,"seq":55834574990,"num_pgs":13,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1481,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193830,"statfs":{"total":200000000,"available":198482763,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Oct 26 01:41:54 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.78700000000000003,"15min":0.89000000000000001},"min":{"1min":0.63200000000000001,"5min":0.32500000000000001,"15min":0.32000000000000001},"max":{"1min":1.1910000000000001,"5min":1.494,"15min":19.425000000000001},"last":0.745},{"interface":"front","average":{"1min":1.0009999999999999,"5min":0.95399999999999996,"15min":1.0249999999999999},"min":{"1min":0.58399999999999996,"5min":0.434,"15min":0.434},"max":{"1min":1.492,"5min":1.492,"15min":15.162000000000001},"last":1.1659999999999999}]},{"osd":2,"last update":"Thu Oct 26 01:41:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.0409999999999999,"15min":1.1599999999999999},"min":{"1min":0.69299999999999995,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.4750000000000001,"5min":1.556,"15min":28.439},"last":0.95399999999999996},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.93400000000000005,"15min":0.92000000000000004},"min":{"1min":0.72099999999999997,"5min":0.37,"15min":0.37},"max":{"1min":1.1539999999999999,"5min":1.851,"15min":4.9459999999999997},"last":0.90500000000000003}]},{"osd":3,"last update":"Thu Oct 26 01:41:54 2023","interfaces":[{"interface":"back","average":{"1min":1.163,"5min":1.113,"15min":1.0920000000000001},"min":{"1min":0.84899999999999998,"5min":0.48899999999999999,"15min":0.44600000000000001},"max":{"1min":1.3819999999999999,"5min":1.7509999999999999,"15min":4.9800000000000004},"last":1.1240000000000001},{"interface":"front","average":{"1min":1.28,"5min":1.1839999999999999,"15min":1.165},"min":{"1min":0.71599999999999997,"5min":0.438,"15min":0.438},"max":{"1min":1.546,"5min":1.774,"15min":5.016},"last":1.595}]},{"osd":4,"last update":"Thu Oct 26 01:41:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1559999999999999,"5min":1.1319999999999999,"15min":1.2589999999999999},"min":{"1min":0.86399999999999999,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.3680000000000001,"5min":1.5840000000000001,"15min":19.329000000000001},"last":1.0649999999999999},{"interface":"front","average":{"1min":1.2809999999999999,"5min":1.2509999999999999,"15min":1.2809999999999999},"min":{"1min":0.93200000000000005,"5min":0.86799999999999999,"15min":0.67600000000000005},"max":{"1min":1.7030000000000001,"5min":1.7270000000000001,"15min":5.5229999999999997},"last":1.3009999999999999}]},{"osd":5,"last update":"Thu Oct 26 01:41:54 2023","interfaces":[{"interface":"back","average":{"1min":1.4359999999999999,"5min":1.421,"15min":1.425},"min":{"1min":1.179,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.655,"5min":1.819,"15min":5.6360000000000001},"last":1.4430000000000001},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.262,"15min":1.375},"min":{"1min":0.91600000000000004,"5min":0.73599999999999999,"15min":0.57499999999999996},"max":{"1min":1.7390000000000001,"5min":1.742,"15min":19.384},"last":1.1910000000000001}]},{"osd":6,"last update":"Thu Oct 26 01:41:54 2023","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.319,"15min":1.355},"min":{"1min":1.018,"5min":0.76700000000000002,"15min":0.73799999999999999},"max":{"1min":1.8999999999999999,"5min":1.8999999999999999,"15min":8.2609999999999992},"last":1.353},{"interface":"front","average":{"1min":1.3799999999999999,"5min":1.3640000000000001,"15min":1.399},"min":{"1min":0.98299999999999998,"5min":0.82999999999999996,"15min":0.75900000000000001},"max":{"1min":1.8600000000000001,"5min":1.9399999999999999,"15min":11.849},"last":1.4099999999999999}]},{"osd":7,"last update":"Thu Oct 26 01:41:54 2023","interfaces":[{"interface":"back","average":{"1min":1.518,"5min":1.4750000000000001,"15min":1.4770000000000001},"min":{"1min":1.133,"5min":0.871,"15min":0.73699999999999999},"max":{"1min":1.768,"5min":1.9379999999999999,"15min":5.6689999999999996},"last":1.5369999999999999},{"interface":"front","average":{"1min":1.444,"5min":1.401,"15min":1.49},"min":{"1min":0.96499999999999997,"5min":0.81299999999999994,"15min":0.69399999999999995},"max":{"1min":1.6779999999999999,"5min":1.841,"15min":19.245999999999999},"last":1.492}]}]}],"pool_statfs":[]}} 2023-10-26T01:42:15.369 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-26T01:42:15.370 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-26T01:42:15.641 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:15.639+0000 7f3013690640 1 -- 172.21.15.169:0/3120841932 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f300c15bc00 msgr2=0x7f300c0d0db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:15.642 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:15.639+0000 7f3013690640 1 --2- 172.21.15.169:0/3120841932 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f300c15bc00 0x7f300c0d0db0 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f3004009830 tx=0x7f300402bde0 comp rx=0 tx=0).stop 2023-10-26T01:42:15.642 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:15.639+0000 7f3013690640 1 -- 172.21.15.169:0/3120841932 shutdown_connections 2023-10-26T01:42:15.642 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:15.639+0000 7f3013690640 1 --2- 172.21.15.169:0/3120841932 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f300c15bc00 0x7f300c0d0db0 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:15.642 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:15.639+0000 7f3013690640 1 --2- 172.21.15.169:0/3120841932 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f300c0cec50 0x7f300c0cc780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:15.642 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:15.639+0000 7f3013690640 1 -- 172.21.15.169:0/3120841932 >> 172.21.15.169:0/3120841932 conn(0x7f300c065670 msgr2=0x7f300c065a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:42:15.643 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:15.639+0000 7f3013690640 1 -- 172.21.15.169:0/3120841932 shutdown_connections 2023-10-26T01:42:15.643 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:15.639+0000 7f3013690640 1 -- 172.21.15.169:0/3120841932 wait complete. 2023-10-26T01:42:15.644 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:15.641+0000 7f3013690640 1 Processor -- start 2023-10-26T01:42:15.645 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:15.641+0000 7f3013690640 1 -- start start 2023-10-26T01:42:15.645 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:15.642+0000 7f3013690640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f300c0cec50 0x7f300c15c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:15.645 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:15.642+0000 7f3013690640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f300c15b820 0x7f300c1f4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:15.645 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:15.642+0000 7f3013690640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f300c15bc00 0x7f300c1f4b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:15.646 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:15.642+0000 7f3013690640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f300c1f50c0 con 0x7f300c15bc00 2023-10-26T01:42:15.646 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:15.642+0000 7f3013690640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f300c0cb190 con 0x7f300c15b820 2023-10-26T01:42:15.646 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:15.642+0000 7f3013690640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f300c0ca8f0 con 0x7f300c0cec50 2023-10-26T01:42:15.646 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:15.642+0000 7f3011c2d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f300c15bc00 0x7f300c1f4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:42:15.646 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:15.642+0000 7f3011c2d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f300c15bc00 0x7f300c1f4b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:54150/0 (socket says 172.21.15.169:54150) 2023-10-26T01:42:15.647 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:15.642+0000 7f3011c2d640 1 -- 172.21.15.169:0/508892283 learned_addr learned my addr 172.21.15.169:0/508892283 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:42:15.647 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:15.643+0000 7f3010c2b640 1 --2- 172.21.15.169:0/508892283 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f300c15b820 0x7f300c1f4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:42:15.647 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:15.643+0000 7f301142c640 1 --2- 172.21.15.169:0/508892283 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f300c0cec50 0x7f300c15c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:42:15.647 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:15.643+0000 7f3011c2d640 1 -- 172.21.15.169:0/508892283 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f300c0cec50 msgr2=0x7f300c15c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:15.647 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:15.643+0000 7f3011c2d640 1 --2- 172.21.15.169:0/508892283 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f300c0cec50 0x7f300c15c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:15.648 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:15.643+0000 7f3011c2d640 1 -- 172.21.15.169:0/508892283 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f300c15b820 msgr2=0x7f300c1f4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:15.648 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:15.643+0000 7f3011c2d640 1 --2- 172.21.15.169:0/508892283 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f300c15b820 0x7f300c1f4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:15.648 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:15.643+0000 7f3011c2d640 1 -- 172.21.15.169:0/508892283 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3004009490 con 0x7f300c15bc00 2023-10-26T01:42:15.648 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:15.643+0000 7f3010c2b640 1 --2- 172.21.15.169:0/508892283 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f300c15b820 0x7f300c1f4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:42:15.648 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:15.643+0000 7f3011c2d640 1 --2- 172.21.15.169:0/508892283 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f300c15bc00 0x7f300c1f4b80 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f3004009950 tx=0x7f3004005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:42:15.650 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:15.643+0000 7f3009ffb640 1 -- 172.21.15.169:0/508892283 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3004003e30 con 0x7f300c15bc00 2023-10-26T01:42:15.650 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:15.644+0000 7f3009ffb640 1 -- 172.21.15.169:0/508892283 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3004005020 con 0x7f300c15bc00 2023-10-26T01:42:15.650 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:15.644+0000 7f3009ffb640 1 -- 172.21.15.169:0/508892283 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3004005410 con 0x7f300c15bc00 2023-10-26T01:42:15.650 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:15.644+0000 7f3013690640 1 -- 172.21.15.169:0/508892283 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f300c0cabd0 con 0x7f300c15bc00 2023-10-26T01:42:15.650 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:15.644+0000 7f3013690640 1 -- 172.21.15.169:0/508892283 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f300c0c5850 con 0x7f300c15bc00 2023-10-26T01:42:15.651 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:15.645+0000 7f3013690640 1 -- 172.21.15.169:0/508892283 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f300c04db10 con 0x7f300c15bc00 2023-10-26T01:42:15.651 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:15.646+0000 7f3009ffb640 1 -- 172.21.15.169:0/508892283 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f300402dd50 con 0x7f300c15bc00 2023-10-26T01:42:15.651 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:15.647+0000 7f3009ffb640 1 --2- 172.21.15.169:0/508892283 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2ff806ae40 0x7f2ff806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:15.651 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:15.647+0000 7f3009ffb640 1 -- 172.21.15.169:0/508892283 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f30040ab7a0 con 0x7f300c15bc00 2023-10-26T01:42:15.651 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:15.647+0000 7f301142c640 1 --2- 172.21.15.169:0/508892283 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2ff806ae40 0x7f2ff806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:42:15.652 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:15.648+0000 7f301142c640 1 --2- 172.21.15.169:0/508892283 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2ff806ae40 0x7f2ff806d300 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f300c0b6ae0 tx=0x7f2ffc009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:42:15.652 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:15.651+0000 7f3009ffb640 1 -- 172.21.15.169:0/508892283 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3004076850 con 0x7f300c15bc00 2023-10-26T01:42:15.904 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:15.902+0000 7f3013690640 1 -- 172.21.15.169:0/508892283 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f300c0b40d0 con 0x7f300c15bc00 2023-10-26T01:42:15.908 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:15.906+0000 7f3009ffb640 1 -- 172.21.15.169:0/508892283 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v20) v1 ==== 190+0+147 (secure 0 0 0) 0x7f30040420b0 con 0x7f300c15bc00 2023-10-26T01:42:15.908 INFO:teuthology.orchestra.run.smithi169.stdout:[client.0] 2023-10-26T01:42:15.908 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQAgwTllKC7gOhAAPsvxGcuUA8/p8xqbUgGteA== 2023-10-26T01:42:15.909 INFO:teuthology.orchestra.run.smithi169.stdout: caps mds = "allow" 2023-10-26T01:42:15.909 INFO:teuthology.orchestra.run.smithi169.stdout: caps mon = "allow r" 2023-10-26T01:42:15.909 INFO:teuthology.orchestra.run.smithi169.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-10-26T01:42:15.909 INFO:teuthology.orchestra.run.smithi169.stderr:updated caps for client.0 2023-10-26T01:42:15.911 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:15.909+0000 7f3013690640 1 -- 172.21.15.169:0/508892283 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2ff806ae40 msgr2=0x7f2ff806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:15.911 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:15.909+0000 7f3013690640 1 --2- 172.21.15.169:0/508892283 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2ff806ae40 0x7f2ff806d300 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f300c0b6ae0 tx=0x7f2ffc009040 comp rx=0 tx=0).stop 2023-10-26T01:42:15.912 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:15.909+0000 7f3013690640 1 -- 172.21.15.169:0/508892283 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f300c15bc00 msgr2=0x7f300c1f4b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:15.912 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:15.909+0000 7f3013690640 1 --2- 172.21.15.169:0/508892283 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f300c15bc00 0x7f300c1f4b80 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f3004009950 tx=0x7f3004005dc0 comp rx=0 tx=0).stop 2023-10-26T01:42:15.912 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:15.909+0000 7f3013690640 1 -- 172.21.15.169:0/508892283 shutdown_connections 2023-10-26T01:42:15.912 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:15.909+0000 7f3013690640 1 --2- 172.21.15.169:0/508892283 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2ff806ae40 0x7f2ff806d300 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:15.912 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:15.909+0000 7f3013690640 1 --2- 172.21.15.169:0/508892283 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f300c15bc00 0x7f300c1f4b80 unknown :-1 s=CLOSED pgs=1707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:15.913 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:15.909+0000 7f3013690640 1 --2- 172.21.15.169:0/508892283 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f300c15b820 0x7f300c1f4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:15.913 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:15.909+0000 7f3013690640 1 --2- 172.21.15.169:0/508892283 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f300c0cec50 0x7f300c15c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:15.913 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:15.909+0000 7f3013690640 1 -- 172.21.15.169:0/508892283 >> 172.21.15.169:0/508892283 conn(0x7f300c065670 msgr2=0x7f300c0b45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:42:15.913 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:15.909+0000 7f3013690640 1 -- 172.21.15.169:0/508892283 shutdown_connections 2023-10-26T01:42:15.913 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:15.909+0000 7f3013690640 1 -- 172.21.15.169:0/508892283 wait complete. 2023-10-26T01:42:15.929 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T01:42:16.136 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:16.134+0000 7f0e4a133640 1 -- 172.21.15.169:0/2608495031 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e44158850 msgr2=0x7f0e440cd720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:16.137 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:16.134+0000 7f0e4a133640 1 --2- 172.21.15.169:0/2608495031 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e44158850 0x7f0e440cd720 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f0e38005fd0 tx=0x7f0e3802bcf0 comp rx=0 tx=0).stop 2023-10-26T01:42:16.137 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:16.135+0000 7f0e4a133640 1 -- 172.21.15.169:0/2608495031 shutdown_connections 2023-10-26T01:42:16.137 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:16.135+0000 7f0e4a133640 1 --2- 172.21.15.169:0/2608495031 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e44158850 0x7f0e440cd720 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:16.138 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:16.135+0000 7f0e4a133640 1 --2- 172.21.15.169:0/2608495031 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0e4414e790 0x7f0e440aab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:16.138 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:16.135+0000 7f0e4a133640 1 -- 172.21.15.169:0/2608495031 >> 172.21.15.169:0/2608495031 conn(0x7f0e44069930 msgr2=0x7f0e44069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:42:16.138 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:16.135+0000 7f0e4a133640 1 -- 172.21.15.169:0/2608495031 shutdown_connections 2023-10-26T01:42:16.138 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:16.135+0000 7f0e4a133640 1 -- 172.21.15.169:0/2608495031 wait complete. 2023-10-26T01:42:16.140 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:16.136+0000 7f0e4a133640 1 Processor -- start 2023-10-26T01:42:16.140 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:16.137+0000 7f0e4a133640 1 -- start start 2023-10-26T01:42:16.140 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:16.137+0000 7f0e4a133640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0e440cdc60 0x7f0e4406fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:16.141 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:16.138+0000 7f0e4a133640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e4414e790 0x7f0e4414ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:16.141 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:16.138+0000 7f0e4a133640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0e44158850 0x7f0e44096a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:16.141 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:16.138+0000 7f0e4a133640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e4406f0f0 con 0x7f0e4414e790 2023-10-26T01:42:16.141 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:16.138+0000 7f0e4a133640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e44096f50 con 0x7f0e440cdc60 2023-10-26T01:42:16.142 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:16.138+0000 7f0e4a133640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e44150070 con 0x7f0e44158850 2023-10-26T01:42:16.142 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:16.138+0000 7f0e42ffd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e4414e790 0x7f0e4414ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:42:16.142 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:16.138+0000 7f0e42ffd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e4414e790 0x7f0e4414ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:54158/0 (socket says 172.21.15.169:54158) 2023-10-26T01:42:16.142 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:16.138+0000 7f0e42ffd640 1 -- 172.21.15.169:0/2310868984 learned_addr learned my addr 172.21.15.169:0/2310868984 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:42:16.142 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:16.138+0000 7f0e437fe640 1 --2- 172.21.15.169:0/2310868984 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0e440cdc60 0x7f0e4406fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:42:16.143 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:16.138+0000 7f0e42ffd640 1 -- 172.21.15.169:0/2310868984 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0e44158850 msgr2=0x7f0e44096a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:16.143 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:16.138+0000 7f0e42ffd640 1 --2- 172.21.15.169:0/2310868984 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0e44158850 0x7f0e44096a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:16.143 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:16.138+0000 7f0e42ffd640 1 -- 172.21.15.169:0/2310868984 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0e440cdc60 msgr2=0x7f0e4406fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:16.143 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:16.138+0000 7f0e42ffd640 1 --2- 172.21.15.169:0/2310868984 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0e440cdc60 0x7f0e4406fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:16.143 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:16.138+0000 7f0e42ffd640 1 -- 172.21.15.169:0/2310868984 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e38009560 con 0x7f0e4414e790 2023-10-26T01:42:16.144 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:16.138+0000 7f0e437fe640 1 --2- 172.21.15.169:0/2310868984 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0e440cdc60 0x7f0e4406fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:42:16.144 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:16.139+0000 7f0e42ffd640 1 --2- 172.21.15.169:0/2310868984 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e4414e790 0x7f0e4414ed60 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f0e38036040 tx=0x7f0e3802dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:42:16.145 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:16.139+0000 7f0e2bfff640 1 -- 172.21.15.169:0/2310868984 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e38005650 con 0x7f0e4414e790 2023-10-26T01:42:16.145 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:16.139+0000 7f0e2bfff640 1 -- 172.21.15.169:0/2310868984 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0e38003800 con 0x7f0e4414e790 2023-10-26T01:42:16.145 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:16.139+0000 7f0e2bfff640 1 -- 172.21.15.169:0/2310868984 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e38003bb0 con 0x7f0e4414e790 2023-10-26T01:42:16.146 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:16.139+0000 7f0e4a133640 1 -- 172.21.15.169:0/2310868984 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e4414f460 con 0x7f0e4414e790 2023-10-26T01:42:16.146 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:16.139+0000 7f0e4a133640 1 -- 172.21.15.169:0/2310868984 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e4414fa20 con 0x7f0e4414e790 2023-10-26T01:42:16.146 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:16.140+0000 7f0e4a133640 1 -- 172.21.15.169:0/2310868984 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e44150200 con 0x7f0e4414e790 2023-10-26T01:42:16.146 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:16.141+0000 7f0e2bfff640 1 -- 172.21.15.169:0/2310868984 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f0e38003d10 con 0x7f0e4414e790 2023-10-26T01:42:16.146 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:16.142+0000 7f0e2bfff640 1 --2- 172.21.15.169:0/2310868984 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f0e3006ae70 0x7f0e3006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:16.147 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:16.142+0000 7f0e2bfff640 1 -- 172.21.15.169:0/2310868984 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0e380ac2a0 con 0x7f0e4414e790 2023-10-26T01:42:16.147 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:16.143+0000 7f0e437fe640 1 --2- 172.21.15.169:0/2310868984 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f0e3006ae70 0x7f0e3006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:42:16.147 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:16.144+0000 7f0e437fe640 1 --2- 172.21.15.169:0/2310868984 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f0e3006ae70 0x7f0e3006d330 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f0e3400a870 tx=0x7f0e34008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:42:16.148 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:16.146+0000 7f0e2bfff640 1 -- 172.21.15.169:0/2310868984 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0e380772b0 con 0x7f0e4414e790 2023-10-26T01:42:16.349 INFO:tasks.ceph.mds.a.smithi169.stderr:2023-10-26T01:42:16.347+0000 7fac77608640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-26T01:42:16.381 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:16.379+0000 7f0e4a133640 1 -- 172.21.15.169:0/2310868984 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0e440ab720 con 0x7f0e4414e790 2023-10-26T01:42:16.382 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:16.380+0000 7f0e2bfff640 1 -- 172.21.15.169:0/2310868984 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) v1 ==== 94+0+4923 (secure 0 0 0) 0x7f0e3807a8d0 con 0x7f0e4414e790 2023-10-26T01:42:16.382 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:42:16.382 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":86,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5861,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/326391134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":326391134},{"type":"v1","addr":"172.21.15.170:6836","nonce":326391134}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":5864,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/3374486280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":3374486280},{"type":"v1","addr":"172.21.15.170:6837","nonce":3374486280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":8697,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/1445776478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":1445776478},{"type":"v1","addr":"172.21.15.169:6834","nonce":1445776478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T01:42:09.853788+0000","modified":"2023-10-26T01:42:12.922104+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8700},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8700":{"gid":8700,"name":"a","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6835/228678953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":228678953},{"type":"v1","addr":"172.21.15.169:6835","nonce":228678953}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-10-26T01:42:16.383 INFO:teuthology.orchestra.run.smithi169.stderr:dumped fsmap epoch 86 2023-10-26T01:42:16.385 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:16.383+0000 7f0e4a133640 1 -- 172.21.15.169:0/2310868984 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f0e3006ae70 msgr2=0x7f0e3006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:16.385 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:16.383+0000 7f0e4a133640 1 --2- 172.21.15.169:0/2310868984 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f0e3006ae70 0x7f0e3006d330 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f0e3400a870 tx=0x7f0e34008040 comp rx=0 tx=0).stop 2023-10-26T01:42:16.385 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:16.383+0000 7f0e4a133640 1 -- 172.21.15.169:0/2310868984 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e4414e790 msgr2=0x7f0e4414ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:16.385 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:16.383+0000 7f0e4a133640 1 --2- 172.21.15.169:0/2310868984 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e4414e790 0x7f0e4414ed60 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f0e38036040 tx=0x7f0e3802dcc0 comp rx=0 tx=0).stop 2023-10-26T01:42:16.386 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:16.383+0000 7f0e4a133640 1 -- 172.21.15.169:0/2310868984 shutdown_connections 2023-10-26T01:42:16.386 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:16.383+0000 7f0e4a133640 1 --2- 172.21.15.169:0/2310868984 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f0e3006ae70 0x7f0e3006d330 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:16.386 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:16.383+0000 7f0e4a133640 1 --2- 172.21.15.169:0/2310868984 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e4414e790 0x7f0e4414ed60 unknown :-1 s=CLOSED pgs=1709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:16.386 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:16.383+0000 7f0e4a133640 1 --2- 172.21.15.169:0/2310868984 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0e44158850 0x7f0e44096a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:16.386 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:16.383+0000 7f0e4a133640 1 --2- 172.21.15.169:0/2310868984 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0e440cdc60 0x7f0e4406fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:16.387 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:16.383+0000 7f0e4a133640 1 -- 172.21.15.169:0/2310868984 >> 172.21.15.169:0/2310868984 conn(0x7f0e44069930 msgr2=0x7f0e44159d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:42:16.387 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:16.383+0000 7f0e4a133640 1 -- 172.21.15.169:0/2310868984 shutdown_connections 2023-10-26T01:42:16.387 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:16.383+0000 7f0e4a133640 1 -- 172.21.15.169:0/2310868984 wait complete. 2023-10-26T01:42:16.806 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 86, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-26T01:42:09.853788+0000', 'modified': '2023-10-26T01:42:12.922104+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8700}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8700': {'gid': 8700, 'name': 'a', 'rank': 0, 'incarnation': 82, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.169:6835/228678953', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.169:6833', 'nonce': 228678953}, {'type': 'v1', 'addr': '172.21.15.169:6835', 'nonce': 228678953}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-26T01:42:16.807 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-26T01:42:16.807 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-26T01:42:17.040 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.038+0000 7fbc16fa9640 1 -- 172.21.15.169:0/2163636359 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbc100713d0 msgr2=0x7fbc10070730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:17.040 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.038+0000 7fbc16fa9640 1 --2- 172.21.15.169:0/2163636359 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbc100713d0 0x7fbc10070730 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7fbc100b2fb0 tx=0x7fbbfc02bd90 comp rx=0 tx=0).stop 2023-10-26T01:42:17.041 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.038+0000 7fbc16fa9640 1 -- 172.21.15.169:0/2163636359 shutdown_connections 2023-10-26T01:42:17.041 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.038+0000 7fbc16fa9640 1 --2- 172.21.15.169:0/2163636359 >> v2:172.21.15.170:3300/0 conn(0x7fbc10070c70 0x7fbc100cb8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:17.041 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.038+0000 7fbc16fa9640 1 --2- 172.21.15.169:0/2163636359 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbc100713d0 0x7fbc10070730 unknown :-1 s=CLOSED pgs=1710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:17.041 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.038+0000 7fbc16fa9640 1 --2- 172.21.15.169:0/2163636359 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbc1015ca70 0x7fbc1015d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:17.041 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.038+0000 7fbc16fa9640 1 -- 172.21.15.169:0/2163636359 >> 172.21.15.169:0/2163636359 conn(0x7fbc100af8a0 msgr2=0x7fbc100afc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:42:17.041 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.038+0000 7fbc16fa9640 1 -- 172.21.15.169:0/2163636359 shutdown_connections 2023-10-26T01:42:17.042 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.038+0000 7fbc16fa9640 1 -- 172.21.15.169:0/2163636359 wait complete. 2023-10-26T01:42:17.043 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.040+0000 7fbc16fa9640 1 Processor -- start 2023-10-26T01:42:17.043 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.040+0000 7fbc16fa9640 1 -- start start 2023-10-26T01:42:17.043 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.041+0000 7fbc16fa9640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbc10070c70 0x7fbc1014d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:17.043 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.041+0000 7fbc16fa9640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbc100713d0 0x7fbc100ae9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:17.043 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.041+0000 7fbc16fa9640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbc1015ca70 0x7fbc1015b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:17.044 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.041+0000 7fbc16fa9640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc100cd4d0 con 0x7fbc100713d0 2023-10-26T01:42:17.044 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.041+0000 7fbc16fa9640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc100cc950 con 0x7fbc1015ca70 2023-10-26T01:42:17.044 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.041+0000 7fbc16fa9640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc100ccc30 con 0x7fbc10070c70 2023-10-26T01:42:17.045 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.041+0000 7fbc0ffff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbc100713d0 0x7fbc100ae9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:42:17.046 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.042+0000 7fbc0ffff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbc100713d0 0x7fbc100ae9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:54174/0 (socket says 172.21.15.169:54174) 2023-10-26T01:42:17.046 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.042+0000 7fbc0ffff640 1 -- 172.21.15.169:0/3433891558 learned_addr learned my addr 172.21.15.169:0/3433891558 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:42:17.046 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.042+0000 7fbc14d45640 1 --2- 172.21.15.169:0/3433891558 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbc10070c70 0x7fbc1014d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:42:17.046 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.042+0000 7fbc15546640 1 --2- 172.21.15.169:0/3433891558 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbc1015ca70 0x7fbc1015b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:42:17.046 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.042+0000 7fbc0ffff640 1 -- 172.21.15.169:0/3433891558 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbc10070c70 msgr2=0x7fbc1014d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:17.047 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.042+0000 7fbc0ffff640 1 --2- 172.21.15.169:0/3433891558 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbc10070c70 0x7fbc1014d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:17.047 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.042+0000 7fbc0ffff640 1 -- 172.21.15.169:0/3433891558 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbc1015ca70 msgr2=0x7fbc1015b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:17.047 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.042+0000 7fbc0ffff640 1 --2- 172.21.15.169:0/3433891558 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbc1015ca70 0x7fbc1015b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:17.047 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.042+0000 7fbc0ffff640 1 -- 172.21.15.169:0/3433891558 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbfc009560 con 0x7fbc100713d0 2023-10-26T01:42:17.047 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.042+0000 7fbc15546640 1 --2- 172.21.15.169:0/3433891558 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbc1015ca70 0x7fbc1015b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:42:17.049 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.042+0000 7fbc0ffff640 1 --2- 172.21.15.169:0/3433891558 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbc100713d0 0x7fbc100ae9f0 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc009850 tx=0x7fbbfc02dd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:42:17.049 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.043+0000 7fbc0d7fa640 1 -- 172.21.15.169:0/3433891558 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbfc005650 con 0x7fbc100713d0 2023-10-26T01:42:17.049 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.043+0000 7fbc0d7fa640 1 -- 172.21.15.169:0/3433891558 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fbbfc0038f0 con 0x7fbc100713d0 2023-10-26T01:42:17.050 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.043+0000 7fbc0d7fa640 1 -- 172.21.15.169:0/3433891558 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbfc003ce0 con 0x7fbc100713d0 2023-10-26T01:42:17.050 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.043+0000 7fbc16fa9640 1 -- 172.21.15.169:0/3433891558 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc100702c0 con 0x7fbc100713d0 2023-10-26T01:42:17.050 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.043+0000 7fbc16fa9640 1 -- 172.21.15.169:0/3433891558 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc100cd1f0 con 0x7fbc100713d0 2023-10-26T01:42:17.053 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.045+0000 7fbc0d7fa640 1 -- 172.21.15.169:0/3433891558 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fbbfc03e450 con 0x7fbc100713d0 2023-10-26T01:42:17.053 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.046+0000 7fbc0d7fa640 1 --2- 172.21.15.169:0/3433891558 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbbf406ae40 0x7fbbf406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:17.053 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.046+0000 7fbc0d7fa640 1 -- 172.21.15.169:0/3433891558 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbbfc0abf70 con 0x7fbc100713d0 2023-10-26T01:42:17.053 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.047+0000 7fbc14d45640 1 --2- 172.21.15.169:0/3433891558 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbbf406ae40 0x7fbbf406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:42:17.054 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.047+0000 7fbc19c11740 1 Processor -- start 2023-10-26T01:42:17.054 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.047+0000 7fbc19c11740 1 -- start start 2023-10-26T01:42:17.054 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.048+0000 7fbc14d45640 1 --2- 172.21.15.169:0/3433891558 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbbf406ae40 0x7fbbf406d300 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fbc0400ca40 tx=0x7fbc0400c420 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:42:17.054 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.048+0000 7fbc19c11740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56019ec897d0 0x56019ec5ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:17.054 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.048+0000 7fbc19c11740 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x56019ec89bb0 0x56019ec8c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:17.055 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.048+0000 7fbc19c11740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56019ec5cf40 0x56019ec8caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:17.055 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.048+0000 7fbc19c11740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x56019ebcf740 con 0x56019ec897d0 2023-10-26T01:42:17.055 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.048+0000 7fbc19c11740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56019ebcce30 con 0x56019ec5cf40 2023-10-26T01:42:17.055 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.048+0000 7fbc19c11740 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x56019ebc8720 con 0x56019ec89bb0 2023-10-26T01:42:17.055 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.048+0000 7fbc14d45640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56019ec897d0 0x56019ec5ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:42:17.055 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.048+0000 7fbc14d45640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56019ec897d0 0x56019ec5ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:54178/0 (socket says 172.21.15.169:54178) 2023-10-26T01:42:17.056 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.048+0000 7fbc14d45640 1 -- 172.21.15.169:0/2784968552 learned_addr learned my addr 172.21.15.169:0/2784968552 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:42:17.056 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.049+0000 7fbc15546640 1 --2- 172.21.15.169:0/2784968552 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56019ec5cf40 0x56019ec8caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:42:17.056 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.049+0000 7fbc0ffff640 1 --2- 172.21.15.169:0/2784968552 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x56019ec89bb0 0x56019ec8c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:42:17.056 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.049+0000 7fbc14d45640 1 -- 172.21.15.169:0/2784968552 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x56019ec89bb0 msgr2=0x56019ec8c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T01:42:17.056 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.049+0000 7fbc14d45640 1 --2- 172.21.15.169:0/2784968552 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x56019ec89bb0 0x56019ec8c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:17.056 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.049+0000 7fbc14d45640 1 -- 172.21.15.169:0/2784968552 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56019ec5cf40 msgr2=0x56019ec8caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T01:42:17.057 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.049+0000 7fbc14d45640 1 --2- 172.21.15.169:0/2784968552 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56019ec5cf40 0x56019ec8caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:17.057 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.049+0000 7fbc14d45640 1 -- 172.21.15.169:0/2784968552 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56019eb8c910 con 0x56019ec897d0 2023-10-26T01:42:17.057 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.049+0000 7fbc14d45640 1 --2- 172.21.15.169:0/2784968552 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56019ec897d0 0x56019ec5ca00 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7fbc0400ede0 tx=0x7fbc04011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ebe36d9b90dca34 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:42:17.057 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.049+0000 7fbbfa7fc640 1 -- 172.21.15.169:0/2784968552 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc0401d650 con 0x56019ec897d0 2023-10-26T01:42:17.057 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.049+0000 7fbbfa7fc640 1 -- 172.21.15.169:0/2784968552 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fbc0400eed0 con 0x56019ec897d0 2023-10-26T01:42:17.058 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.049+0000 7fbbfa7fc640 1 -- 172.21.15.169:0/2784968552 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc0401dac0 con 0x56019ec897d0 2023-10-26T01:42:17.058 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.050+0000 7fbc19c11740 1 -- 172.21.15.169:0/2784968552 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56019ec897d0 msgr2=0x56019ec5ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:17.058 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.050+0000 7fbc19c11740 1 --2- 172.21.15.169:0/2784968552 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56019ec897d0 0x56019ec5ca00 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7fbc0400ede0 tx=0x7fbc04011000 comp rx=0 tx=0).stop 2023-10-26T01:42:17.058 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.050+0000 7fbc19c11740 1 -- 172.21.15.169:0/2784968552 shutdown_connections 2023-10-26T01:42:17.058 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.050+0000 7fbc19c11740 1 --2- 172.21.15.169:0/2784968552 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56019ec5cf40 0x56019ec8caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:17.059 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.050+0000 7fbc19c11740 1 --2- 172.21.15.169:0/2784968552 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x56019ec89bb0 0x56019ec8c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:17.059 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.050+0000 7fbc19c11740 1 --2- 172.21.15.169:0/2784968552 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56019ec897d0 0x56019ec5ca00 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:17.059 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.050+0000 7fbc19c11740 1 -- 172.21.15.169:0/2784968552 >> 172.21.15.169:0/2784968552 conn(0x56019ec5ec70 msgr2=0x56019eca7560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:42:17.059 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.050+0000 7fbc19c11740 1 -- 172.21.15.169:0/2784968552 shutdown_connections 2023-10-26T01:42:17.059 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.050+0000 7fbc19c11740 1 -- 172.21.15.169:0/2784968552 wait complete. 2023-10-26T01:42:17.060 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.051+0000 7fbc19c11740 1 -- start start 2023-10-26T01:42:17.060 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.051+0000 7fbc19c11740 1 Processor -- start 2023-10-26T01:42:17.063 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.052+0000 7fbc19c11740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56019ec897d0 0x56019ec8fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:17.063 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.052+0000 7fbc19c11740 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x56019ec89bb0 0x56019ec9e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:17.063 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.052+0000 7fbc19c11740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56019ec76c50 0x56019ec771e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:17.063 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.052+0000 7fbc19c11740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x56019eb66970 con 0x56019ec76c50 2023-10-26T01:42:17.063 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.052+0000 7fbc19c11740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56019eb66610 con 0x56019ec897d0 2023-10-26T01:42:17.064 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.052+0000 7fbc19c11740 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x56019e92cf20 con 0x56019ec89bb0 2023-10-26T01:42:17.064 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.052+0000 7fbc19c11740 10 client.0 _ll_register_callbacks cb 0x56019eb8a0c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T01:42:17.064 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.052+0000 7fbc15546640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56019ec76c50 0x56019ec771e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:42:17.064 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.052+0000 7fbc15546640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56019ec76c50 0x56019ec771e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:54182/0 (socket says 172.21.15.169:54182) 2023-10-26T01:42:17.064 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.052+0000 7fbc15546640 1 -- 172.21.15.169:0/3801430847 learned_addr learned my addr 172.21.15.169:0/3801430847 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:42:17.065 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.053+0000 7fbc14d45640 1 --2- 172.21.15.169:0/3801430847 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56019ec897d0 0x56019ec8fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:42:17.065 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.053+0000 7fbc15546640 1 -- 172.21.15.169:0/3801430847 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x56019ec89bb0 msgr2=0x56019ec9e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T01:42:17.065 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.053+0000 7fbc15546640 1 --2- 172.21.15.169:0/3801430847 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x56019ec89bb0 0x56019ec9e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:17.065 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.053+0000 7fbc15546640 1 -- 172.21.15.169:0/3801430847 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56019ec897d0 msgr2=0x56019ec8fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T01:42:17.065 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.053+0000 7fbc15546640 1 --2- 172.21.15.169:0/3801430847 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56019ec897d0 0x56019ec8fab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:17.066 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.053+0000 7fbc15546640 1 -- 172.21.15.169:0/3801430847 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc0400ea90 con 0x56019ec76c50 2023-10-26T01:42:17.066 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.053+0000 7fbc15546640 1 --2- 172.21.15.169:0/3801430847 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56019ec76c50 0x56019ec771e0 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7fbc0800eab0 tx=0x7fbc0800ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca08d773ecd20342 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:42:17.066 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.053+0000 7fbbef7fe640 10 client.0 ms_handle_connect on v2:172.21.15.169:3300/0 2023-10-26T01:42:17.066 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.053+0000 7fbbef7fe640 1 -- 172.21.15.169:0/3801430847 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc0800ccc0 con 0x56019ec76c50 2023-10-26T01:42:17.067 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.053+0000 7fbbef7fe640 1 -- 172.21.15.169:0/3801430847 <== mon.0 v2:172.21.15.169:3300/0 2 ==== osd_map(166..166 src has 1..166) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbc0800ce20 con 0x56019ec76c50 2023-10-26T01:42:17.067 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.053+0000 7fbc19c11740 1 -- 172.21.15.169:0/3801430847 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56019ebc1700 con 0x56019ec76c50 2023-10-26T01:42:17.067 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.053+0000 7fbbef7fe640 1 -- 172.21.15.169:0/3801430847 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbc08017440 con 0x56019ec76c50 2023-10-26T01:42:17.067 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.053+0000 7fbc19c11740 10 client.8847 fetch_fsmap learned FSMap version 86 2023-10-26T01:42:17.067 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.053+0000 7fbc19c11740 1 -- 172.21.15.169:0/3801430847 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=2+}) v3 -- 0x56019ebc6da0 con 0x56019ec76c50 2023-10-26T01:42:17.068 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.054+0000 7fbbef7fe640 1 -- 172.21.15.169:0/3801430847 <== mon.0 v2:172.21.15.169:3300/0 4 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x56019ebc1700 con 0x56019ec76c50 2023-10-26T01:42:17.068 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.054+0000 7fbbef7fe640 1 -- 172.21.15.169:0/3801430847 <== mon.0 v2:172.21.15.169:3300/0 5 ==== fsmap(e 86) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fbc080175a0 con 0x56019ec76c50 2023-10-26T01:42:17.068 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.054+0000 7fbc19c11740 10 client.8847 fetch_fsmap finished waiting for FSMap version 86 2023-10-26T01:42:17.068 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.054+0000 7fbc19c11740 10 client.8847 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-26T01:42:17.068 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.054+0000 7fbc19c11740 20 client.8847 populate_metadata read hostname 'smithi169' 2023-10-26T01:42:17.069 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.054+0000 7fbc19c11740 1 --2- 172.21.15.169:0/3801430847 >> [v2:172.21.15.169:6833/228678953,v1:172.21.15.169:6835/228678953] conn(0x56019ec77720 0x56019ecc4c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:17.069 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.054+0000 7fbc19c11740 4 client.8847 mds_command: new command op to 8700 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-26T01:42:17.069 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.054+0000 7fbc19c11740 1 -- 172.21.15.169:0/3801430847 --> [v2:172.21.15.169:6833/228678953,v1:172.21.15.169:6835/228678953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56019ebc1420 con 0x56019ec77720 2023-10-26T01:42:17.069 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.054+0000 7fbc14d45640 1 --2- 172.21.15.169:0/3801430847 >> [v2:172.21.15.169:6833/228678953,v1:172.21.15.169:6835/228678953] conn(0x56019ec77720 0x56019ecc4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:42:17.069 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.055+0000 7fbc14d45640 1 --2- 172.21.15.169:0/3801430847 >> [v2:172.21.15.169:6833/228678953,v1:172.21.15.169:6835/228678953] conn(0x56019ec77720 0x56019ecc4c70 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=598393cdb4ac85f7 server_cookie=12a0194ccf3a0c04 in_seq=0 out_seq=0 2023-10-26T01:42:17.070 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.055+0000 7fbbef7fe640 10 client.8847 ms_handle_connect on v2:172.21.15.169:6833/228678953 2023-10-26T01:42:17.070 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.055+0000 7fbbef7fe640 1 -- 172.21.15.169:0/3801430847 <== mds.0 v2:172.21.15.169:6833/228678953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56019ebc1420 con 0x56019ec77720 2023-10-26T01:42:17.070 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.055+0000 7fbbef7fe640 10 client.8847 handle_command_reply: tid=0 2023-10-26T01:42:17.070 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.056+0000 7fbc19c11740 1 client.8847 shutdown 2023-10-26T01:42:17.070 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.056+0000 7fbc19c11740 1 -- 172.21.15.169:0/3801430847 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56019ec76c50 msgr2=0x56019ec771e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:17.071 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.056+0000 7fbc19c11740 1 --2- 172.21.15.169:0/3801430847 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56019ec76c50 0x56019ec771e0 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7fbc0800eab0 tx=0x7fbc0800ef70 comp rx=0 tx=0).stop 2023-10-26T01:42:17.071 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.056+0000 7fbc19c11740 1 -- 172.21.15.169:0/3801430847 shutdown_connections 2023-10-26T01:42:17.071 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.056+0000 7fbc19c11740 1 --2- 172.21.15.169:0/3801430847 >> [v2:172.21.15.169:6833/228678953,v1:172.21.15.169:6835/228678953] conn(0x56019ec77720 0x56019ecc4c70 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:17.071 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.056+0000 7fbc19c11740 1 --2- 172.21.15.169:0/3801430847 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56019ec76c50 0x56019ec771e0 unknown :-1 s=CLOSED pgs=1713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:17.071 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.056+0000 7fbc19c11740 1 --2- 172.21.15.169:0/3801430847 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x56019ec89bb0 0x56019ec9e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:17.071 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.056+0000 7fbc19c11740 1 --2- 172.21.15.169:0/3801430847 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56019ec897d0 0x56019ec8fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:17.072 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.056+0000 7fbbef7fe640 0 client.8847 ms_handle_reset on v2:172.21.15.169:6833/228678953 2023-10-26T01:42:17.072 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.056+0000 7fbc19c11740 1 -- 172.21.15.169:0/3801430847 >> 172.21.15.169:0/3801430847 conn(0x56019ec5cf40 msgr2=0x56019ec88e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:42:17.072 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.056+0000 7fbc19c11740 1 -- 172.21.15.169:0/3801430847 shutdown_connections 2023-10-26T01:42:17.072 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.056+0000 7fbc19c11740 1 -- 172.21.15.169:0/3801430847 wait complete. 2023-10-26T01:42:17.072 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.056+0000 7fbc19c11740 20 client.8847 trim_cache size 0 max 16384 2023-10-26T01:42:17.074 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.073+0000 7fbc19c11740 1 Processor -- start 2023-10-26T01:42:17.074 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.073+0000 7fbc19c11740 1 -- start start 2023-10-26T01:42:17.075 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.074+0000 7fbc19c11740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56019ec76c50 0x56019ec5be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:17.076 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.074+0000 7fbc19c11740 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x56019ec897d0 0x56019ec84310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:17.076 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.074+0000 7fbc19c11740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56019ec89bb0 0x56019ec58120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:17.076 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.074+0000 7fbc19c11740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x56019ec8db60 con 0x56019ec76c50 2023-10-26T01:42:17.076 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.074+0000 7fbc19c11740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56019ec8a270 con 0x56019ec89bb0 2023-10-26T01:42:17.076 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.074+0000 7fbc19c11740 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x56019ec9f7a0 con 0x56019ec897d0 2023-10-26T01:42:17.077 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.074+0000 7fbc14d45640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56019ec76c50 0x56019ec5be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:42:17.077 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.074+0000 7fbc14d45640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56019ec76c50 0x56019ec5be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:54190/0 (socket says 172.21.15.169:54190) 2023-10-26T01:42:17.077 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.074+0000 7fbc14d45640 1 -- 172.21.15.169:0/1082456442 learned_addr learned my addr 172.21.15.169:0/1082456442 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:42:17.078 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.074+0000 7fbc14d45640 1 -- 172.21.15.169:0/1082456442 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x56019ec897d0 msgr2=0x56019ec84310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T01:42:17.079 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.074+0000 7fbc14d45640 1 --2- 172.21.15.169:0/1082456442 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x56019ec897d0 0x56019ec84310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:17.079 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.074+0000 7fbc14d45640 1 -- 172.21.15.169:0/1082456442 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56019ec89bb0 msgr2=0x56019ec58120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T01:42:17.079 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.074+0000 7fbc14d45640 1 --2- 172.21.15.169:0/1082456442 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56019ec89bb0 0x56019ec58120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:17.079 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.074+0000 7fbc14d45640 1 -- 172.21.15.169:0/1082456442 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56019eb65b80 con 0x56019ec76c50 2023-10-26T01:42:17.079 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.075+0000 7fbc14d45640 1 --2- 172.21.15.169:0/1082456442 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56019ec76c50 0x56019ec5be20 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7fbc0400e720 tx=0x7fbc04004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6938c12e7547ee4 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:42:17.080 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.075+0000 7fbbfaffd640 1 -- 172.21.15.169:0/1082456442 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc0402d040 con 0x56019ec76c50 2023-10-26T01:42:17.080 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.075+0000 7fbbfaffd640 1 -- 172.21.15.169:0/1082456442 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fbc04033410 con 0x56019ec76c50 2023-10-26T01:42:17.080 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.075+0000 7fbbfaffd640 1 -- 172.21.15.169:0/1082456442 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc04033b90 con 0x56019ec76c50 2023-10-26T01:42:17.080 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.075+0000 7fbc19c11740 1 -- 172.21.15.169:0/1082456442 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56019ec76c50 msgr2=0x56019ec5be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:17.080 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.075+0000 7fbc19c11740 1 --2- 172.21.15.169:0/1082456442 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56019ec76c50 0x56019ec5be20 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7fbc0400e720 tx=0x7fbc04004730 comp rx=0 tx=0).stop 2023-10-26T01:42:17.081 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.075+0000 7fbc19c11740 1 -- 172.21.15.169:0/1082456442 shutdown_connections 2023-10-26T01:42:17.081 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.075+0000 7fbc19c11740 1 --2- 172.21.15.169:0/1082456442 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56019ec89bb0 0x56019ec58120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:17.081 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.075+0000 7fbc19c11740 1 --2- 172.21.15.169:0/1082456442 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x56019ec897d0 0x56019ec84310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:17.081 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.075+0000 7fbc19c11740 1 --2- 172.21.15.169:0/1082456442 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56019ec76c50 0x56019ec5be20 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:17.081 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.075+0000 7fbc19c11740 1 -- 172.21.15.169:0/1082456442 >> 172.21.15.169:0/1082456442 conn(0x56019ec5cf40 msgr2=0x56019ecaa2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:42:17.082 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.076+0000 7fbc19c11740 1 -- 172.21.15.169:0/1082456442 shutdown_connections 2023-10-26T01:42:17.082 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.076+0000 7fbc19c11740 1 -- 172.21.15.169:0/1082456442 wait complete. 2023-10-26T01:42:17.082 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.076+0000 7fbc19c11740 1 -- start start 2023-10-26T01:42:17.082 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.076+0000 7fbc19c11740 1 Processor -- start 2023-10-26T01:42:17.085 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.077+0000 7fbc19c11740 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x56019ec76c50 0x56019ed12200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:17.085 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.077+0000 7fbc19c11740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56019ec897d0 0x56019ed18210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:17.086 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.077+0000 7fbc19c11740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56019ec89bb0 0x56019eca8f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:17.086 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.077+0000 7fbc19c11740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x56019ec8e0b0 con 0x56019ec897d0 2023-10-26T01:42:17.086 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.077+0000 7fbc19c11740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56019ed12740 con 0x56019ec89bb0 2023-10-26T01:42:17.086 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.077+0000 7fbc19c11740 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x56019ed18a80 con 0x56019ec76c50 2023-10-26T01:42:17.086 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.078+0000 7fbc19c11740 10 client.0 _ll_register_callbacks cb 0x7fbc04014070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T01:42:17.087 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.078+0000 7fbc0ffff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56019ec897d0 0x56019ed18210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:42:17.087 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.078+0000 7fbc0ffff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56019ec897d0 0x56019ed18210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:54196/0 (socket says 172.21.15.169:54196) 2023-10-26T01:42:17.087 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.078+0000 7fbc0ffff640 1 -- 172.21.15.169:0/612933633 learned_addr learned my addr 172.21.15.169:0/612933633 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:42:17.087 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.078+0000 7fbc14d45640 1 --2- 172.21.15.169:0/612933633 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x56019ec76c50 0x56019ed12200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:42:17.087 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.078+0000 7fbc0ffff640 1 -- 172.21.15.169:0/612933633 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x56019ec76c50 msgr2=0x56019ed12200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T01:42:17.087 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.078+0000 7fbc0ffff640 1 --2- 172.21.15.169:0/612933633 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x56019ec76c50 0x56019ed12200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:17.088 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.078+0000 7fbc0ffff640 1 -- 172.21.15.169:0/612933633 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56019ec89bb0 msgr2=0x56019eca8f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T01:42:17.088 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.078+0000 7fbc0ffff640 1 --2- 172.21.15.169:0/612933633 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56019ec89bb0 0x56019eca8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:17.088 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.078+0000 7fbc0ffff640 1 -- 172.21.15.169:0/612933633 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56019eb8c910 con 0x56019ec897d0 2023-10-26T01:42:17.088 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.078+0000 7fbc14d45640 1 --2- 172.21.15.169:0/612933633 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x56019ec76c50 0x56019ed12200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:42:17.088 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.078+0000 7fbc0ffff640 1 --2- 172.21.15.169:0/612933633 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56019ec897d0 0x56019ed18210 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc0099c0 tx=0x7fbbfc003f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a0388f8f41eb7f7 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:42:17.089 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.078+0000 7fbbef7fe640 10 client.0 ms_handle_connect on v2:172.21.15.169:3300/0 2023-10-26T01:42:17.089 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.079+0000 7fbbef7fe640 1 -- 172.21.15.169:0/612933633 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbfc092070 con 0x56019ec897d0 2023-10-26T01:42:17.089 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.079+0000 7fbbef7fe640 1 -- 172.21.15.169:0/612933633 <== mon.0 v2:172.21.15.169:3300/0 2 ==== osd_map(166..166 src has 1..166) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbbfc07ed90 con 0x56019ec897d0 2023-10-26T01:42:17.089 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.079+0000 7fbc19c11740 1 -- 172.21.15.169:0/612933633 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbc0400eed0 con 0x56019ec897d0 2023-10-26T01:42:17.089 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.079+0000 7fbbef7fe640 1 -- 172.21.15.169:0/612933633 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbbfc082f40 con 0x56019ec897d0 2023-10-26T01:42:17.089 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.079+0000 7fbc19c11740 10 client.8853 fetch_fsmap learned FSMap version 86 2023-10-26T01:42:17.090 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.079+0000 7fbc19c11740 1 -- 172.21.15.169:0/612933633 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=2+}) v3 -- 0x56019eb8b170 con 0x56019ec897d0 2023-10-26T01:42:17.090 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.079+0000 7fbbef7fe640 1 -- 172.21.15.169:0/612933633 <== mon.0 v2:172.21.15.169:3300/0 4 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fbbfc096020 con 0x56019ec897d0 2023-10-26T01:42:17.090 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.079+0000 7fbbef7fe640 1 -- 172.21.15.169:0/612933633 <== mon.0 v2:172.21.15.169:3300/0 5 ==== fsmap(e 86) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fbbfc04e020 con 0x56019ec897d0 2023-10-26T01:42:17.090 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.079+0000 7fbc19c11740 10 client.8853 fetch_fsmap finished waiting for FSMap version 86 2023-10-26T01:42:17.090 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.079+0000 7fbc19c11740 10 client.8853 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-26T01:42:17.091 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.079+0000 7fbc19c11740 20 client.8853 populate_metadata read hostname 'smithi169' 2023-10-26T01:42:17.091 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.079+0000 7fbc19c11740 1 --2- 172.21.15.169:0/612933633 >> [v2:172.21.15.169:6833/228678953,v1:172.21.15.169:6835/228678953] conn(0x56019ec58120 0x56019ec58500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:17.091 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.079+0000 7fbc19c11740 4 client.8853 mds_command: new command op to 8700 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-26T01:42:17.091 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.079+0000 7fbc19c11740 1 -- 172.21.15.169:0/612933633 --> [v2:172.21.15.169:6833/228678953,v1:172.21.15.169:6835/228678953] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56019e8ca6c0 con 0x56019ec58120 2023-10-26T01:42:17.091 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.080+0000 7fbc14d45640 1 --2- 172.21.15.169:0/612933633 >> [v2:172.21.15.169:6833/228678953,v1:172.21.15.169:6835/228678953] conn(0x56019ec58120 0x56019ec58500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:42:17.091 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.080+0000 7fbc14d45640 1 --2- 172.21.15.169:0/612933633 >> [v2:172.21.15.169:6833/228678953,v1:172.21.15.169:6835/228678953] conn(0x56019ec58120 0x56019ec58500 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec176f62bcc95e47 server_cookie=4f2ed1b3d8d911da in_seq=0 out_seq=0 2023-10-26T01:42:17.092 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.080+0000 7fbbef7fe640 10 client.8853 ms_handle_connect on v2:172.21.15.169:6833/228678953 2023-10-26T01:42:17.092 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.080+0000 7fbbef7fe640 1 -- 172.21.15.169:0/612933633 <== mds.0 v2:172.21.15.169:6833/228678953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x56019e8ca6c0 con 0x56019ec58120 2023-10-26T01:42:17.092 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.080+0000 7fbbef7fe640 10 client.8853 handle_command_reply: tid=0 2023-10-26T01:42:17.092 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.080+0000 7fbc19c11740 1 client.8853 shutdown 2023-10-26T01:42:17.092 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.080+0000 7fbc19c11740 1 -- 172.21.15.169:0/612933633 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56019ec897d0 msgr2=0x56019ed18210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:17.092 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.080+0000 7fbc19c11740 1 --2- 172.21.15.169:0/612933633 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56019ec897d0 0x56019ed18210 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc0099c0 tx=0x7fbbfc003f00 comp rx=0 tx=0).stop 2023-10-26T01:42:17.093 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.081+0000 7fbc19c11740 1 -- 172.21.15.169:0/612933633 shutdown_connections 2023-10-26T01:42:17.093 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.081+0000 7fbc19c11740 1 --2- 172.21.15.169:0/612933633 >> [v2:172.21.15.169:6833/228678953,v1:172.21.15.169:6835/228678953] conn(0x56019ec58120 0x56019ec58500 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:17.093 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.081+0000 7fbc19c11740 1 --2- 172.21.15.169:0/612933633 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56019ec897d0 0x56019ed18210 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:17.093 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.081+0000 7fbc19c11740 1 --2- 172.21.15.169:0/612933633 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56019ec89bb0 0x56019eca8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:17.093 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.081+0000 7fbbef7fe640 0 client.8853 ms_handle_reset on v2:172.21.15.169:6833/228678953 2023-10-26T01:42:17.094 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.081+0000 7fbc19c11740 1 --2- 172.21.15.169:0/612933633 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x56019ec76c50 0x56019ed12200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:17.094 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.081+0000 7fbc19c11740 1 -- 172.21.15.169:0/612933633 >> 172.21.15.169:0/612933633 conn(0x56019ec5cf40 msgr2=0x56019eca7ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:42:17.094 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.081+0000 7fbc19c11740 1 -- 172.21.15.169:0/612933633 shutdown_connections 2023-10-26T01:42:17.094 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.081+0000 7fbc19c11740 1 -- 172.21.15.169:0/612933633 wait complete. 2023-10-26T01:42:17.094 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.081+0000 7fbc19c11740 20 client.8853 trim_cache size 0 max 16384 2023-10-26T01:42:17.095 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2023-10-26T01:42:17.095 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_fsid": "30203b28-ed8a-44e8-a9a6-a9bf25f980f1", 2023-10-26T01:42:17.095 INFO:teuthology.orchestra.run.smithi169.stdout: "whoami": 0, 2023-10-26T01:42:17.095 INFO:teuthology.orchestra.run.smithi169.stdout: "id": 8700, 2023-10-26T01:42:17.096 INFO:teuthology.orchestra.run.smithi169.stdout: "want_state": "up:active", 2023-10-26T01:42:17.096 INFO:teuthology.orchestra.run.smithi169.stdout: "state": "up:active", 2023-10-26T01:42:17.096 INFO:teuthology.orchestra.run.smithi169.stdout: "fs_name": "cephfs", 2023-10-26T01:42:17.096 INFO:teuthology.orchestra.run.smithi169.stdout: "rank_uptime": 6.6985407490000002, 2023-10-26T01:42:17.096 INFO:teuthology.orchestra.run.smithi169.stdout: "mdsmap_epoch": 86, 2023-10-26T01:42:17.096 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_epoch": 166, 2023-10-26T01:42:17.097 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_epoch_barrier": 166, 2023-10-26T01:42:17.097 INFO:teuthology.orchestra.run.smithi169.stdout: "uptime": 12.680243066999999 2023-10-26T01:42:17.097 INFO:teuthology.orchestra.run.smithi169.stdout:} 2023-10-26T01:42:17.098 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.081+0000 7fbc16fa9640 1 -- 172.21.15.169:0/3433891558 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbbf406ae40 msgr2=0x7fbbf406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:17.098 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.081+0000 7fbc16fa9640 1 --2- 172.21.15.169:0/3433891558 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbbf406ae40 0x7fbbf406d300 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fbc0400ca40 tx=0x7fbc0400c420 comp rx=0 tx=0).stop 2023-10-26T01:42:17.098 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.081+0000 7fbc16fa9640 1 -- 172.21.15.169:0/3433891558 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbc100713d0 msgr2=0x7fbc100ae9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:17.099 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.081+0000 7fbc16fa9640 1 --2- 172.21.15.169:0/3433891558 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbc100713d0 0x7fbc100ae9f0 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc009850 tx=0x7fbbfc02dd60 comp rx=0 tx=0).stop 2023-10-26T01:42:17.099 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.082+0000 7fbc16fa9640 1 -- 172.21.15.169:0/3433891558 shutdown_connections 2023-10-26T01:42:17.099 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.082+0000 7fbc16fa9640 1 --2- 172.21.15.169:0/3433891558 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbbf406ae40 0x7fbbf406d300 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:17.099 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.082+0000 7fbc16fa9640 1 --2- 172.21.15.169:0/3433891558 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbc100713d0 0x7fbc100ae9f0 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:17.099 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.082+0000 7fbc16fa9640 1 --2- 172.21.15.169:0/3433891558 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbc1015ca70 0x7fbc1015b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:17.100 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.082+0000 7fbc16fa9640 1 --2- 172.21.15.169:0/3433891558 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbc10070c70 0x7fbc1014d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:17.100 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.082+0000 7fbc16fa9640 1 -- 172.21.15.169:0/3433891558 >> 172.21.15.169:0/3433891558 conn(0x7fbc100af8a0 msgr2=0x7fbc1006bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:42:17.100 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.082+0000 7fbc16fa9640 1 -- 172.21.15.169:0/3433891558 shutdown_connections 2023-10-26T01:42:17.100 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.082+0000 7fbc16fa9640 1 -- 172.21.15.169:0/3433891558 wait complete. 2023-10-26T01:42:17.101 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-26T01:42:17.316 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.313+0000 7fa679901640 1 -- 172.21.15.169:0/3827822505 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa65c007330 msgr2=0x7fa65c009720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:17.316 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.313+0000 7fa679901640 1 --2- 172.21.15.169:0/3827822505 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa65c007330 0x7fa65c009720 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7fa66800bb20 tx=0x7fa65c0037d0 comp rx=0 tx=0).stop 2023-10-26T01:42:17.316 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.313+0000 7fa6737fe640 1 -- 172.21.15.169:0/3827822505 reap_dead start 2023-10-26T01:42:17.316 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.314+0000 7fa679901640 1 -- 172.21.15.169:0/3827822505 shutdown_connections 2023-10-26T01:42:17.317 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.314+0000 7fa679901640 1 -- 172.21.15.169:0/3827822505 >> 172.21.15.169:0/3827822505 conn(0x7fa674069930 msgr2=0x7fa674069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:42:17.317 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.314+0000 7fa679901640 1 -- 172.21.15.169:0/3827822505 shutdown_connections 2023-10-26T01:42:17.317 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.314+0000 7fa679901640 1 -- 172.21.15.169:0/3827822505 wait complete. 2023-10-26T01:42:17.318 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.315+0000 7fa679901640 1 Processor -- start 2023-10-26T01:42:17.318 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.316+0000 7fa679901640 1 -- start start 2023-10-26T01:42:17.319 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.316+0000 7fa679901640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa65c007330 0x7fa6740afa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:17.320 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.316+0000 7fa679901640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa6741e5570 0x7fa6740ade20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:17.320 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.317+0000 7fa679901640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa6741e59f0 0x7fa6740ae360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:17.320 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.317+0000 7fa679901640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa674071030 con 0x7fa6741e5570 2023-10-26T01:42:17.320 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.317+0000 7fa679901640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6740ae9e0 con 0x7fa65c007330 2023-10-26T01:42:17.321 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.317+0000 7fa679901640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6740aecc0 con 0x7fa6741e59f0 2023-10-26T01:42:17.321 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.317+0000 7fa6727fc640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa6741e5570 0x7fa6740ade20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:42:17.321 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.317+0000 7fa6727fc640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa6741e5570 0x7fa6740ade20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:54222/0 (socket says 172.21.15.169:54222) 2023-10-26T01:42:17.321 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.317+0000 7fa6727fc640 1 -- 172.21.15.169:0/3617894792 learned_addr learned my addr 172.21.15.169:0/3617894792 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:42:17.321 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.317+0000 7fa672ffd640 1 --2- 172.21.15.169:0/3617894792 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa65c007330 0x7fa6740afa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:42:17.321 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.317+0000 7fa6727fc640 1 -- 172.21.15.169:0/3617894792 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa6741e59f0 msgr2=0x7fa6740ae360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:17.322 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.317+0000 7fa6727fc640 1 --2- 172.21.15.169:0/3617894792 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa6741e59f0 0x7fa6740ae360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:17.322 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.317+0000 7fa6727fc640 1 -- 172.21.15.169:0/3617894792 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa65c007330 msgr2=0x7fa6740afa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:17.322 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.317+0000 7fa6727fc640 1 --2- 172.21.15.169:0/3617894792 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa65c007330 0x7fa6740afa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:17.322 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.317+0000 7fa6727fc640 1 -- 172.21.15.169:0/3617894792 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6740aefa0 con 0x7fa6741e5570 2023-10-26T01:42:17.323 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.318+0000 7fa6727fc640 1 --2- 172.21.15.169:0/3617894792 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa6741e5570 0x7fa6740ade20 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7fa668038040 tx=0x7fa6680366b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:42:17.323 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.318+0000 7fa65bfff640 1 -- 172.21.15.169:0/3617894792 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa668036d70 con 0x7fa6741e5570 2023-10-26T01:42:17.323 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.318+0000 7fa65bfff640 1 -- 172.21.15.169:0/3617894792 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa668004030 con 0x7fa6741e5570 2023-10-26T01:42:17.323 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.318+0000 7fa679901640 1 -- 172.21.15.169:0/3617894792 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6740af280 con 0x7fa6741e5570 2023-10-26T01:42:17.323 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.318+0000 7fa65bfff640 1 -- 172.21.15.169:0/3617894792 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa668004460 con 0x7fa6741e5570 2023-10-26T01:42:17.324 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.318+0000 7fa679901640 1 -- 172.21.15.169:0/3617894792 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6741e7110 con 0x7fa6741e5570 2023-10-26T01:42:17.325 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.319+0000 7fa679901640 1 -- 172.21.15.169:0/3617894792 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa67415b450 con 0x7fa6741e5570 2023-10-26T01:42:17.325 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.320+0000 7fa65bfff640 1 -- 172.21.15.169:0/3617894792 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fa6680035f0 con 0x7fa6741e5570 2023-10-26T01:42:17.325 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.321+0000 7fa65bfff640 1 --2- 172.21.15.169:0/3617894792 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa66006ae40 0x7fa66006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:17.325 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.321+0000 7fa65bfff640 1 -- 172.21.15.169:0/3617894792 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa6680ab720 con 0x7fa6741e5570 2023-10-26T01:42:17.325 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.322+0000 7fa672ffd640 1 --2- 172.21.15.169:0/3617894792 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa66006ae40 0x7fa66006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:42:17.326 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.323+0000 7fa672ffd640 1 --2- 172.21.15.169:0/3617894792 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa66006ae40 0x7fa66006d300 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fa664001820 tx=0x7fa664001db0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:42:17.327 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.325+0000 7fa65bfff640 1 -- 172.21.15.169:0/3617894792 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa668076730 con 0x7fa6741e5570 2023-10-26T01:42:17.557 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.556+0000 7fa679901640 1 -- 172.21.15.169:0/3617894792 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fa67404db10 con 0x7fa6741e5570 2023-10-26T01:42:17.558 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.557+0000 7fa65bfff640 1 -- 172.21.15.169:0/3617894792 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v86) v1 ==== 53+0+73 (secure 0 0 0) 0x7fa668079d50 con 0x7fa6741e5570 2023-10-26T01:42:17.559 INFO:teuthology.orchestra.run.smithi169.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-26T01:42:17.562 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.560+0000 7fa679901640 1 -- 172.21.15.169:0/3617894792 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa66006ae40 msgr2=0x7fa66006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:17.562 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.560+0000 7fa679901640 1 --2- 172.21.15.169:0/3617894792 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa66006ae40 0x7fa66006d300 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fa664001820 tx=0x7fa664001db0 comp rx=0 tx=0).stop 2023-10-26T01:42:17.562 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.560+0000 7fa679901640 1 -- 172.21.15.169:0/3617894792 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa6741e5570 msgr2=0x7fa6740ade20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:17.562 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.560+0000 7fa679901640 1 --2- 172.21.15.169:0/3617894792 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa6741e5570 0x7fa6740ade20 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7fa668038040 tx=0x7fa6680366b0 comp rx=0 tx=0).stop 2023-10-26T01:42:17.562 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.560+0000 7fa679901640 1 -- 172.21.15.169:0/3617894792 shutdown_connections 2023-10-26T01:42:17.563 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.560+0000 7fa679901640 1 --2- 172.21.15.169:0/3617894792 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa66006ae40 0x7fa66006d300 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:17.563 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.560+0000 7fa679901640 1 --2- 172.21.15.169:0/3617894792 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa6741e5570 0x7fa6740ade20 unknown :-1 s=CLOSED pgs=1718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:17.563 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.560+0000 7fa679901640 1 --2- 172.21.15.169:0/3617894792 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa6741e59f0 0x7fa6740ae360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:17.563 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.560+0000 7fa679901640 1 --2- 172.21.15.169:0/3617894792 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa65c007330 0x7fa6740afa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:17.563 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.560+0000 7fa679901640 1 -- 172.21.15.169:0/3617894792 >> 172.21.15.169:0/3617894792 conn(0x7fa674069930 msgr2=0x7fa67406b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:42:17.564 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.560+0000 7fa679901640 1 -- 172.21.15.169:0/3617894792 shutdown_connections 2023-10-26T01:42:17.564 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:17.560+0000 7fa679901640 1 -- 172.21.15.169:0/3617894792 wait complete. 2023-10-26T01:42:17.581 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-26T01:42:17.581 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-26T01:42:17.581 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi169.front.sepia.ceph.com 2023-10-26T01:42:17.582 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-26T01:42:17.582 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-26T01:42:17.582 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-26T01:42:17.582 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-26T01:42:17.583 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-26T01:42:17.583 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-26T01:42:17.583 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:42:17.584 DEBUG:teuthology.orchestra.run.smithi169:> ip addr 2023-10-26T01:42:17.637 INFO:teuthology.orchestra.run.smithi169.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-26T01:42:17.637 INFO:teuthology.orchestra.run.smithi169.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-26T01:42:17.637 INFO:teuthology.orchestra.run.smithi169.stdout: inet 127.0.0.1/8 scope host lo 2023-10-26T01:42:17.638 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:42:17.638 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 ::1/128 scope host 2023-10-26T01:42:17.638 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:42:17.638 INFO:teuthology.orchestra.run.smithi169.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-26T01:42:17.638 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:d9:3b:b4 brd ff:ff:ff:ff:ff:ff 2023-10-26T01:42:17.639 INFO:teuthology.orchestra.run.smithi169.stdout: altname enp7s0f0 2023-10-26T01:42:17.639 INFO:teuthology.orchestra.run.smithi169.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-26T01:42:17.639 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:d9:3b:b5 brd ff:ff:ff:ff:ff:ff 2023-10-26T01:42:17.639 INFO:teuthology.orchestra.run.smithi169.stdout: altname enp7s0f1 2023-10-26T01:42:17.639 INFO:teuthology.orchestra.run.smithi169.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-26T01:42:17.640 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:88:7f:4c brd ff:ff:ff:ff:ff:ff 2023-10-26T01:42:17.640 INFO:teuthology.orchestra.run.smithi169.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-26T01:42:17.640 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:88:7f:4d brd ff:ff:ff:ff:ff:ff 2023-10-26T01:42:17.640 INFO:teuthology.orchestra.run.smithi169.stdout: inet 172.21.15.169/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-26T01:42:17.640 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft 41758sec preferred_lft 41758sec 2023-10-26T01:42:17.640 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 fe80::ec4:7aff:fe88:7f4d/64 scope link 2023-10-26T01:42:17.641 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:42:17.641 INFO:teuthology.orchestra.run.smithi169.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-26T01:42:17.641 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 22:a0:6b:43:7b:04 brd ff:ff:ff:ff:ff:ff 2023-10-26T01:42:17.641 INFO:teuthology.orchestra.run.smithi169.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-26T01:42:17.641 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:42:17.642 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 fe80::d815:31ff:fe87:d148/64 scope link 2023-10-26T01:42:17.642 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:42:17.642 INFO:teuthology.orchestra.run.smithi169.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-26T01:42:17.642 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 2a:80:f8:45:5f:9f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-26T01:42:17.642 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 fe80::2880:f8ff:fe45:5f9f/64 scope link 2023-10-26T01:42:17.642 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:42:17.643 INFO:teuthology.orchestra.run.smithi169.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-26T01:42:17.643 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 22:a0:6b:43:7b:04 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-26T01:42:17.643 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 fe80::20a0:6bff:fe43:7b04/64 scope link 2023-10-26T01:42:17.643 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:42:17.644 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:42:17.644 DEBUG:teuthology.orchestra.run.smithi169:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-26T01:42:17.738 INFO:teuthology.orchestra.run.smithi169.stdout:1 2023-10-26T01:42:17.742 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:42:17.742 DEBUG:teuthology.orchestra.run.smithi169:> route 2023-10-26T01:42:17.798 INFO:teuthology.orchestra.run.smithi169.stdout:Kernel IP routing table 2023-10-26T01:42:17.798 INFO:teuthology.orchestra.run.smithi169.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-26T01:42:17.798 INFO:teuthology.orchestra.run.smithi169.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-26T01:42:17.799 INFO:teuthology.orchestra.run.smithi169.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-26T01:42:17.799 INFO:teuthology.orchestra.run.smithi169.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-26T01:42:17.800 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:42:17.800 DEBUG:teuthology.orchestra.run.smithi169:> (cd / && exec bash -c ' 2023-10-26T01:42:17.801 DEBUG:teuthology.orchestra.run.smithi169:> set -e 2023-10-26T01:42:17.801 DEBUG:teuthology.orchestra.run.smithi169:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-26T01:42:17.801 DEBUG:teuthology.orchestra.run.smithi169:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-26T01:42:17.802 DEBUG:teuthology.orchestra.run.smithi169:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-26T01:42:17.802 DEBUG:teuthology.orchestra.run.smithi169:> ') 2023-10-26T01:42:17.984 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:42:17.985 DEBUG:teuthology.orchestra.run.smithi169:> ip netns list 2023-10-26T01:42:18.034 INFO:teuthology.orchestra.run.smithi169.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-26T01:42:18.034 INFO:teuthology.orchestra.run.smithi169.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-26T01:42:18.035 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-26T01:42:18.035 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:42:18.035 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link set brx.0 up 2023-10-26T01:42:18.124 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-26T01:42:18.125 DEBUG:teuthology.orchestra.run.smithi169:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-26T01:42:18.180 INFO:teuthology.orchestra.run.smithi169.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-26T01:42:18.181 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-26T01:42:18.181 DEBUG:teuthology.orchestra.run.smithi169:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-26T01:42:18.233 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-26T01:42:18.234 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:42:18.234 DEBUG:teuthology.orchestra.run.smithi169:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-26T01:42:18.374 INFO:teuthology.orchestra.run.smithi169.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-26T01:42:18.374 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-26T01:42:18.374 INFO:teuthology.orchestra.run.smithi169.stdout:invalid new device string format 2023-10-26T01:42:18.398 INFO:teuthology.orchestra.run.smithi169.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-26T01:42:18.398 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-26T01:42:18.398 INFO:teuthology.orchestra.run.smithi169.stdout:invalid new device string format 2023-10-26T01:42:18.398 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: resolved to: "172.21.15.169:3300,172.21.15.170:3300,172.21.15.170:3301" 2023-10-26T01:42:18.399 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-26T01:42:18.423 INFO:teuthology.orchestra.run.smithi169.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-26T01:42:18.423 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-26T01:42:18.423 INFO:teuthology.orchestra.run.smithi169.stdout:invalid new device string format 2023-10-26T01:42:18.423 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: resolved to: "172.21.15.169:3300,172.21.15.170:3300,172.21.15.170:3301" 2023-10-26T01:42:18.424 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-26T01:42:18.424 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: resolved to: "172.21.15.169:6789,172.21.15.170:6789,172.21.15.170:6790" 2023-10-26T01:42:18.424 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.169:6789,172.21.15.170:6789,172.21.15.170:6790:/ 2023-10-26T01:42:18.424 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=30203b28-ed8a-44e8-a9a6-a9bf25f980f1" will pass to kernel 2023-10-26T01:42:18.427 INFO:teuthology.orchestra.run.smithi169.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2023-10-26T01:42:18.427 INFO:teuthology.orchestra.run.smithi169.stdout: You just mounted a file system that supports labels which does not 2023-10-26T01:42:18.427 INFO:teuthology.orchestra.run.smithi169.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-10-26T01:42:18.427 INFO:teuthology.orchestra.run.smithi169.stdout: applications will generate AVC messages and not be allowed access to 2023-10-26T01:42:18.428 INFO:teuthology.orchestra.run.smithi169.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-10-26T01:42:18.431 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-26T01:42:18.431 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:42:18.432 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-26T01:42:18.511 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:42:18.512 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-26T01:42:18.573 INFO:teuthology.orchestra.run.smithi169.stdout:client5909 2023-10-26T01:42:18.573 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:42:18.574 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:42:18.629 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-26T01:42:18.848 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:18.846+0000 7f7aff007640 1 -- 172.21.15.169:0/162761112 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7af815f4c0 msgr2=0x7f7af815f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:18.848 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:18.846+0000 7f7aff007640 1 --2- 172.21.15.169:0/162761112 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7af815f4c0 0x7f7af815f950 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f7af80b5000 tx=0x7f7af002be40 comp rx=0 tx=0).stop 2023-10-26T01:42:18.849 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:18.846+0000 7f7aff007640 1 -- 172.21.15.169:0/162761112 shutdown_connections 2023-10-26T01:42:18.849 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:18.846+0000 7f7aff007640 1 --2- 172.21.15.169:0/162761112 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7af815f4c0 0x7f7af815f950 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:18.849 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:18.846+0000 7f7aff007640 1 --2- 172.21.15.169:0/162761112 >> v2:172.21.15.170:3300/0 conn(0x7f7af80ccc10 0x7f7af80cbe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:18.849 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:18.846+0000 7f7aff007640 1 -- 172.21.15.169:0/162761112 >> 172.21.15.169:0/162761112 conn(0x7f7af80b4b90 msgr2=0x7f7af8098a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:42:18.850 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:18.846+0000 7f7aff007640 1 -- 172.21.15.169:0/162761112 shutdown_connections 2023-10-26T01:42:18.850 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:18.846+0000 7f7aff007640 1 -- 172.21.15.169:0/162761112 wait complete. 2023-10-26T01:42:18.852 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:18.848+0000 7f7aff007640 1 Processor -- start 2023-10-26T01:42:18.852 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:18.848+0000 7f7aff007640 1 -- start start 2023-10-26T01:42:18.852 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:18.849+0000 7f7aff007640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7af80ccc10 0x7f7af8151550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:18.853 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:18.849+0000 7f7aff007640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7af815f4c0 0x7f7af80b2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:18.853 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:18.849+0000 7f7aff007640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7af8160060 0x7f7af80b3370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:18.853 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:18.849+0000 7f7aff007640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7af80b4590 con 0x7f7af815f4c0 2023-10-26T01:42:18.853 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:18.849+0000 7f7aff007640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7af80b3980 con 0x7f7af80ccc10 2023-10-26T01:42:18.854 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:18.849+0000 7f7aff007640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7af80b3c60 con 0x7f7af8160060 2023-10-26T01:42:18.854 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:18.849+0000 7f7af7fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7af815f4c0 0x7f7af80b2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:42:18.854 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:18.849+0000 7f7af7fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7af815f4c0 0x7f7af80b2e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:54234/0 (socket says 172.21.15.169:54234) 2023-10-26T01:42:18.854 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:18.850+0000 7f7af7fff640 1 -- 172.21.15.169:0/4166010818 learned_addr learned my addr 172.21.15.169:0/4166010818 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:42:18.855 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:18.850+0000 7f7afd5a4640 1 --2- 172.21.15.169:0/4166010818 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7af8160060 0x7f7af80b3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:42:18.855 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:18.850+0000 7f7af7fff640 1 -- 172.21.15.169:0/4166010818 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7af8160060 msgr2=0x7f7af80b3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:18.855 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:18.850+0000 7f7af7fff640 1 --2- 172.21.15.169:0/4166010818 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7af8160060 0x7f7af80b3370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:18.855 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:18.850+0000 7f7af7fff640 1 -- 172.21.15.169:0/4166010818 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7af80ccc10 msgr2=0x7f7af8151550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:18.855 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:18.850+0000 7f7af7fff640 1 --2- 172.21.15.169:0/4166010818 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7af80ccc10 0x7f7af8151550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:18.856 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:18.850+0000 7f7af7fff640 1 -- 172.21.15.169:0/4166010818 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7af0009560 con 0x7f7af815f4c0 2023-10-26T01:42:18.856 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:18.850+0000 7f7af7fff640 1 --2- 172.21.15.169:0/4166010818 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7af815f4c0 0x7f7af80b2e30 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f7ae000c7f0 tx=0x7f7ae000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:42:18.856 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:18.850+0000 7f7af57fa640 1 -- 172.21.15.169:0/4166010818 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae0003da0 con 0x7f7af815f4c0 2023-10-26T01:42:18.856 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:18.850+0000 7f7af57fa640 1 -- 172.21.15.169:0/4166010818 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7ae0005070 con 0x7f7af815f4c0 2023-10-26T01:42:18.857 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:18.850+0000 7f7aff007640 1 -- 172.21.15.169:0/4166010818 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7af80b3f40 con 0x7f7af815f4c0 2023-10-26T01:42:18.857 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:18.850+0000 7f7af57fa640 1 -- 172.21.15.169:0/4166010818 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae0005460 con 0x7f7af815f4c0 2023-10-26T01:42:18.857 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:18.851+0000 7f7aff007640 1 -- 172.21.15.169:0/4166010818 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7af8155c90 con 0x7f7af815f4c0 2023-10-26T01:42:18.859 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:18.852+0000 7f7aff007640 1 -- 172.21.15.169:0/4166010818 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7af804db10 con 0x7f7af815f4c0 2023-10-26T01:42:18.859 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:18.853+0000 7f7af57fa640 1 -- 172.21.15.169:0/4166010818 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f7ae00055c0 con 0x7f7af815f4c0 2023-10-26T01:42:18.859 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:18.854+0000 7f7af57fa640 1 --2- 172.21.15.169:0/4166010818 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7ad806ae70 0x7f7ad806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:18.860 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:18.854+0000 7f7af57fa640 1 -- 172.21.15.169:0/4166010818 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7ae0089120 con 0x7f7af815f4c0 2023-10-26T01:42:18.860 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:18.854+0000 7f7afcda3640 1 --2- 172.21.15.169:0/4166010818 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7ad806ae70 0x7f7ad806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:42:18.860 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:18.855+0000 7f7afcda3640 1 --2- 172.21.15.169:0/4166010818 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7ad806ae70 0x7f7ad806d330 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f7ae8005fd0 tx=0x7f7ae8005850 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:42:18.861 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:18.859+0000 7f7af57fa640 1 -- 172.21.15.169:0/4166010818 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7ae00541c0 con 0x7f7af815f4c0 2023-10-26T01:42:19.092 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:19.090+0000 7f7aff007640 1 -- 172.21.15.169:0/4166010818 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f7af80b46d0 con 0x7f7af815f4c0 2023-10-26T01:42:19.093 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:19.091+0000 7f7af57fa640 1 -- 172.21.15.169:0/4166010818 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v86) v1 ==== 53+0+73 (secure 0 0 0) 0x7f7ae0017060 con 0x7f7af815f4c0 2023-10-26T01:42:19.093 INFO:teuthology.orchestra.run.smithi169.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-26T01:42:19.096 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:19.094+0000 7f7aff007640 1 -- 172.21.15.169:0/4166010818 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7ad806ae70 msgr2=0x7f7ad806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:19.096 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:19.094+0000 7f7aff007640 1 --2- 172.21.15.169:0/4166010818 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7ad806ae70 0x7f7ad806d330 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f7ae8005fd0 tx=0x7f7ae8005850 comp rx=0 tx=0).stop 2023-10-26T01:42:19.096 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:19.094+0000 7f7aff007640 1 -- 172.21.15.169:0/4166010818 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7af815f4c0 msgr2=0x7f7af80b2e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:19.097 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:19.094+0000 7f7aff007640 1 --2- 172.21.15.169:0/4166010818 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7af815f4c0 0x7f7af80b2e30 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f7ae000c7f0 tx=0x7f7ae000ccb0 comp rx=0 tx=0).stop 2023-10-26T01:42:19.097 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:19.094+0000 7f7aff007640 1 -- 172.21.15.169:0/4166010818 shutdown_connections 2023-10-26T01:42:19.097 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:19.094+0000 7f7aff007640 1 --2- 172.21.15.169:0/4166010818 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7ad806ae70 0x7f7ad806d330 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:19.097 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:19.094+0000 7f7aff007640 1 --2- 172.21.15.169:0/4166010818 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7af815f4c0 0x7f7af80b2e30 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:19.097 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:19.094+0000 7f7aff007640 1 --2- 172.21.15.169:0/4166010818 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7af8160060 0x7f7af80b3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:19.098 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:19.094+0000 7f7aff007640 1 --2- 172.21.15.169:0/4166010818 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7af80ccc10 0x7f7af8151550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:19.098 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:19.094+0000 7f7aff007640 1 -- 172.21.15.169:0/4166010818 >> 172.21.15.169:0/4166010818 conn(0x7f7af80b4b90 msgr2=0x7f7af8098a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:42:19.098 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:19.094+0000 7f7aff007640 1 -- 172.21.15.169:0/4166010818 shutdown_connections 2023-10-26T01:42:19.098 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:19.094+0000 7f7aff007640 1 -- 172.21.15.169:0/4166010818 wait complete. 2023-10-26T01:42:19.116 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-26T01:42:19.116 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-26T01:42:19.116 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi170.front.sepia.ceph.com 2023-10-26T01:42:19.117 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-26T01:42:19.117 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-26T01:42:19.117 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-26T01:42:19.118 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-26T01:42:19.118 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-26T01:42:19.119 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-26T01:42:19.119 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:42:19.119 DEBUG:teuthology.orchestra.run.smithi170:> ip addr 2023-10-26T01:42:19.181 INFO:teuthology.orchestra.run.smithi170.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-26T01:42:19.181 INFO:teuthology.orchestra.run.smithi170.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-26T01:42:19.181 INFO:teuthology.orchestra.run.smithi170.stdout: inet 127.0.0.1/8 scope host lo 2023-10-26T01:42:19.182 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:42:19.182 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 ::1/128 scope host 2023-10-26T01:42:19.182 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:42:19.182 INFO:teuthology.orchestra.run.smithi170.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-26T01:42:19.182 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:d9:3b:54 brd ff:ff:ff:ff:ff:ff 2023-10-26T01:42:19.183 INFO:teuthology.orchestra.run.smithi170.stdout: altname enp7s0f0 2023-10-26T01:42:19.183 INFO:teuthology.orchestra.run.smithi170.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-26T01:42:19.183 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:d9:3b:55 brd ff:ff:ff:ff:ff:ff 2023-10-26T01:42:19.183 INFO:teuthology.orchestra.run.smithi170.stdout: altname enp7s0f1 2023-10-26T01:42:19.183 INFO:teuthology.orchestra.run.smithi170.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-26T01:42:19.184 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:88:72:54 brd ff:ff:ff:ff:ff:ff 2023-10-26T01:42:19.184 INFO:teuthology.orchestra.run.smithi170.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-26T01:42:19.184 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:88:72:55 brd ff:ff:ff:ff:ff:ff 2023-10-26T01:42:19.184 INFO:teuthology.orchestra.run.smithi170.stdout: inet 172.21.15.170/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-26T01:42:19.185 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft 41741sec preferred_lft 41741sec 2023-10-26T01:42:19.185 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::ec4:7aff:fe88:7255/64 scope link 2023-10-26T01:42:19.185 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:42:19.185 INFO:teuthology.orchestra.run.smithi170.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-26T01:42:19.185 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 32:b5:d1:ee:10:36 brd ff:ff:ff:ff:ff:ff 2023-10-26T01:42:19.185 INFO:teuthology.orchestra.run.smithi170.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-26T01:42:19.186 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:42:19.186 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::28ed:d1ff:fe37:70d/64 scope link 2023-10-26T01:42:19.186 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:42:19.186 INFO:teuthology.orchestra.run.smithi170.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-26T01:42:19.186 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 32:b5:d1:ee:10:36 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-26T01:42:19.187 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::30b5:d1ff:feee:1036/64 scope link 2023-10-26T01:42:19.187 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:42:19.187 INFO:teuthology.orchestra.run.smithi170.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-26T01:42:19.187 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether b2:c1:53:19:69:e4 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-26T01:42:19.187 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::b0c1:53ff:fe19:69e4/64 scope link 2023-10-26T01:42:19.188 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:42:19.189 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:42:19.189 DEBUG:teuthology.orchestra.run.smithi170:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-26T01:42:19.284 INFO:teuthology.orchestra.run.smithi170.stdout:1 2023-10-26T01:42:19.288 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:42:19.289 DEBUG:teuthology.orchestra.run.smithi170:> route 2023-10-26T01:42:19.338 INFO:teuthology.orchestra.run.smithi170.stdout:Kernel IP routing table 2023-10-26T01:42:19.338 INFO:teuthology.orchestra.run.smithi170.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-26T01:42:19.338 INFO:teuthology.orchestra.run.smithi170.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-26T01:42:19.338 INFO:teuthology.orchestra.run.smithi170.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-26T01:42:19.339 INFO:teuthology.orchestra.run.smithi170.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-26T01:42:19.339 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:42:19.339 DEBUG:teuthology.orchestra.run.smithi170:> (cd / && exec bash -c ' 2023-10-26T01:42:19.340 DEBUG:teuthology.orchestra.run.smithi170:> set -e 2023-10-26T01:42:19.340 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-26T01:42:19.341 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-26T01:42:19.341 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-26T01:42:19.341 DEBUG:teuthology.orchestra.run.smithi170:> ') 2023-10-26T01:42:19.519 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:42:19.520 DEBUG:teuthology.orchestra.run.smithi170:> ip netns list 2023-10-26T01:42:19.571 INFO:teuthology.orchestra.run.smithi170.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-26T01:42:19.571 INFO:teuthology.orchestra.run.smithi170.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-26T01:42:19.572 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-26T01:42:19.573 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:42:19.573 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip link set brx.0 up 2023-10-26T01:42:19.660 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-26T01:42:19.661 DEBUG:teuthology.orchestra.run.smithi170:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-26T01:42:19.715 INFO:teuthology.orchestra.run.smithi170.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-26T01:42:19.715 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-26T01:42:19.716 DEBUG:teuthology.orchestra.run.smithi170:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-26T01:42:19.768 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-26T01:42:19.768 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:42:19.769 DEBUG:teuthology.orchestra.run.smithi170:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-26T01:42:19.909 INFO:teuthology.orchestra.run.smithi170.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-26T01:42:19.909 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-26T01:42:19.909 INFO:teuthology.orchestra.run.smithi170.stdout:invalid new device string format 2023-10-26T01:42:19.931 INFO:teuthology.orchestra.run.smithi170.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-26T01:42:19.931 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-26T01:42:19.931 INFO:teuthology.orchestra.run.smithi170.stdout:invalid new device string format 2023-10-26T01:42:19.931 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: resolved to: "172.21.15.169:3300,172.21.15.170:3300,172.21.15.170:3301" 2023-10-26T01:42:19.931 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-26T01:42:19.955 INFO:teuthology.orchestra.run.smithi170.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-26T01:42:19.955 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-26T01:42:19.956 INFO:teuthology.orchestra.run.smithi170.stdout:invalid new device string format 2023-10-26T01:42:19.956 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: resolved to: "172.21.15.169:3300,172.21.15.170:3300,172.21.15.170:3301" 2023-10-26T01:42:19.956 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-26T01:42:19.956 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: resolved to: "172.21.15.169:6789,172.21.15.170:6789,172.21.15.170:6790" 2023-10-26T01:42:19.956 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.169:6789,172.21.15.170:6789,172.21.15.170:6790:/ 2023-10-26T01:42:19.957 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=30203b28-ed8a-44e8-a9a6-a9bf25f980f1" will pass to kernel 2023-10-26T01:42:19.957 INFO:teuthology.orchestra.run.smithi170.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2023-10-26T01:42:19.958 INFO:teuthology.orchestra.run.smithi170.stdout: You just mounted a file system that supports labels which does not 2023-10-26T01:42:19.958 INFO:teuthology.orchestra.run.smithi170.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-10-26T01:42:19.958 INFO:teuthology.orchestra.run.smithi170.stdout: applications will generate AVC messages and not be allowed access to 2023-10-26T01:42:19.958 INFO:teuthology.orchestra.run.smithi170.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-10-26T01:42:19.962 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-26T01:42:19.962 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:42:19.963 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-26T01:42:20.039 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:42:20.039 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-26T01:42:20.097 INFO:teuthology.orchestra.run.smithi170.stdout:client5912 2023-10-26T01:42:20.098 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-26T01:42:20.098 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:42:20.149 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T01:42:20.376 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:20.374+0000 7f3ec78f2640 1 -- 172.21.15.169:0/1500944013 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ec0158850 msgr2=0x7f3ec014e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:20.376 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:20.374+0000 7f3ec78f2640 1 --2- 172.21.15.169:0/1500944013 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ec0158850 0x7f3ec014e790 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f3ec006eaf0 tx=0x7f3eb002bd30 comp rx=0 tx=0).stop 2023-10-26T01:42:20.376 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:20.374+0000 7f3ec78f2640 1 -- 172.21.15.169:0/1500944013 shutdown_connections 2023-10-26T01:42:20.377 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:20.374+0000 7f3ec78f2640 1 --2- 172.21.15.169:0/1500944013 >> v2:172.21.15.170:3300/0 conn(0x7f3ec0159e10 0x7f3ec015a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:20.377 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:20.374+0000 7f3ec78f2640 1 --2- 172.21.15.169:0/1500944013 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ec0158850 0x7f3ec014e790 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:20.377 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:20.374+0000 7f3ec78f2640 1 -- 172.21.15.169:0/1500944013 >> 172.21.15.169:0/1500944013 conn(0x7f3ec0069930 msgr2=0x7f3ec0069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:42:20.377 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:20.374+0000 7f3ec78f2640 1 -- 172.21.15.169:0/1500944013 shutdown_connections 2023-10-26T01:42:20.378 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:20.374+0000 7f3ec78f2640 1 -- 172.21.15.169:0/1500944013 wait complete. 2023-10-26T01:42:20.378 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:20.376+0000 7f3ec78f2640 1 Processor -- start 2023-10-26T01:42:20.379 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:20.376+0000 7f3ec78f2640 1 -- start start 2023-10-26T01:42:20.380 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:20.378+0000 7f3ec78f2640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3ec006ce20 0x7f3ec014db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:20.380 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:20.378+0000 7f3ec78f2640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ec0158850 0x7f3ec01475c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:20.381 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:20.378+0000 7f3ec78f2640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3ec0159e10 0x7f3ec00966c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:20.381 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:20.378+0000 7f3ec78f2640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ec0146560 con 0x7f3ec0158850 2023-10-26T01:42:20.381 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:20.378+0000 7f3ec78f2640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ec0147b00 con 0x7f3ec0159e10 2023-10-26T01:42:20.382 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:20.378+0000 7f3ec78f2640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ec0145950 con 0x7f3ec006ce20 2023-10-26T01:42:20.382 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:20.378+0000 7f3ec4e8d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ec0158850 0x7f3ec01475c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:42:20.382 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:20.378+0000 7f3ec4e8d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ec0158850 0x7f3ec01475c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:54252/0 (socket says 172.21.15.169:54252) 2023-10-26T01:42:20.382 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:20.378+0000 7f3ec4e8d640 1 -- 172.21.15.169:0/585737290 learned_addr learned my addr 172.21.15.169:0/585737290 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:42:20.383 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:20.378+0000 7f3ec568e640 1 --2- 172.21.15.169:0/585737290 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3ec006ce20 0x7f3ec014db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:42:20.383 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:20.379+0000 7f3ec5e8f640 1 --2- 172.21.15.169:0/585737290 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3ec0159e10 0x7f3ec00966c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:42:20.383 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:20.379+0000 7f3ec4e8d640 1 -- 172.21.15.169:0/585737290 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3ec006ce20 msgr2=0x7f3ec014db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:20.383 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:20.379+0000 7f3ec4e8d640 1 --2- 172.21.15.169:0/585737290 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3ec006ce20 0x7f3ec014db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:20.383 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:20.379+0000 7f3ec4e8d640 1 -- 172.21.15.169:0/585737290 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3ec0159e10 msgr2=0x7f3ec00966c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:20.384 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:20.379+0000 7f3ec4e8d640 1 --2- 172.21.15.169:0/585737290 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3ec0159e10 0x7f3ec00966c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:20.384 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:20.379+0000 7f3ec4e8d640 1 -- 172.21.15.169:0/585737290 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3eb0009560 con 0x7f3ec0158850 2023-10-26T01:42:20.385 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:20.379+0000 7f3ec4e8d640 1 --2- 172.21.15.169:0/585737290 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ec0158850 0x7f3ec01475c0 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f3eb0036040 tx=0x7f3eb002dd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:42:20.385 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:20.379+0000 7f3eb5ffb640 1 -- 172.21.15.169:0/585737290 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eb0003800 con 0x7f3ec0158850 2023-10-26T01:42:20.386 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:20.379+0000 7f3eb5ffb640 1 -- 172.21.15.169:0/585737290 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3eb0003a70 con 0x7f3ec0158850 2023-10-26T01:42:20.386 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:20.379+0000 7f3eb5ffb640 1 -- 172.21.15.169:0/585737290 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eb0003e80 con 0x7f3ec0158850 2023-10-26T01:42:20.386 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:20.379+0000 7f3ec78f2640 1 -- 172.21.15.169:0/585737290 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ec0145c30 con 0x7f3ec0158850 2023-10-26T01:42:20.386 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:20.380+0000 7f3ec78f2640 1 -- 172.21.15.169:0/585737290 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ec01461f0 con 0x7f3ec0158850 2023-10-26T01:42:20.386 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:20.381+0000 7f3ec78f2640 1 -- 172.21.15.169:0/585737290 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ec004db10 con 0x7f3ec0158850 2023-10-26T01:42:20.387 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:20.382+0000 7f3eb5ffb640 1 -- 172.21.15.169:0/585737290 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f3eb003e450 con 0x7f3ec0158850 2023-10-26T01:42:20.387 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:20.383+0000 7f3eb5ffb640 1 --2- 172.21.15.169:0/585737290 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3ea406ae40 0x7f3ea406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:20.387 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:20.383+0000 7f3eb5ffb640 1 -- 172.21.15.169:0/585737290 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3eb00abf20 con 0x7f3ec0158850 2023-10-26T01:42:20.387 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:20.383+0000 7f3ec568e640 1 --2- 172.21.15.169:0/585737290 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3ea406ae40 0x7f3ea406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:42:20.388 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:20.385+0000 7f3ec568e640 1 --2- 172.21.15.169:0/585737290 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3ea406ae40 0x7f3ea406d300 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc007fd0 tx=0x7f3ebc009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:42:20.389 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:20.387+0000 7f3eb5ffb640 1 -- 172.21.15.169:0/585737290 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3eb0077180 con 0x7f3ec0158850 2023-10-26T01:42:20.623 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:20.621+0000 7f3ec78f2640 1 -- 172.21.15.169:0/585737290 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3ec00aca70 con 0x7f3ec0158850 2023-10-26T01:42:20.624 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:20.622+0000 7f3eb5ffb640 1 -- 172.21.15.169:0/585737290 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) v1 ==== 94+0+4923 (secure 0 0 0) 0x7f3eb0042090 con 0x7f3ec0158850 2023-10-26T01:42:20.625 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:42:20.625 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":86,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5861,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/326391134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":326391134},{"type":"v1","addr":"172.21.15.170:6836","nonce":326391134}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":5864,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/3374486280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":3374486280},{"type":"v1","addr":"172.21.15.170:6837","nonce":3374486280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":8697,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/1445776478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":1445776478},{"type":"v1","addr":"172.21.15.169:6834","nonce":1445776478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T01:42:09.853788+0000","modified":"2023-10-26T01:42:12.922104+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8700},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8700":{"gid":8700,"name":"a","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6835/228678953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":228678953},{"type":"v1","addr":"172.21.15.169:6835","nonce":228678953}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-10-26T01:42:20.625 INFO:teuthology.orchestra.run.smithi169.stderr:dumped fsmap epoch 86 2023-10-26T01:42:20.627 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:20.625+0000 7f3ec78f2640 1 -- 172.21.15.169:0/585737290 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3ea406ae40 msgr2=0x7f3ea406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:20.627 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:20.625+0000 7f3ec78f2640 1 --2- 172.21.15.169:0/585737290 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3ea406ae40 0x7f3ea406d300 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc007fd0 tx=0x7f3ebc009040 comp rx=0 tx=0).stop 2023-10-26T01:42:20.628 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:20.625+0000 7f3ec78f2640 1 -- 172.21.15.169:0/585737290 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ec0158850 msgr2=0x7f3ec01475c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:20.628 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:20.625+0000 7f3ec78f2640 1 --2- 172.21.15.169:0/585737290 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ec0158850 0x7f3ec01475c0 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f3eb0036040 tx=0x7f3eb002dd00 comp rx=0 tx=0).stop 2023-10-26T01:42:20.628 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:20.625+0000 7f3ec78f2640 1 -- 172.21.15.169:0/585737290 shutdown_connections 2023-10-26T01:42:20.628 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:20.625+0000 7f3ec78f2640 1 --2- 172.21.15.169:0/585737290 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3ea406ae40 0x7f3ea406d300 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:20.629 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:20.625+0000 7f3ec78f2640 1 --2- 172.21.15.169:0/585737290 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ec0158850 0x7f3ec01475c0 unknown :-1 s=CLOSED pgs=1723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:20.629 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:20.625+0000 7f3ec78f2640 1 --2- 172.21.15.169:0/585737290 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3ec0159e10 0x7f3ec00966c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:20.629 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:20.625+0000 7f3ec78f2640 1 --2- 172.21.15.169:0/585737290 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3ec006ce20 0x7f3ec014db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:20.629 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:20.625+0000 7f3ec78f2640 1 -- 172.21.15.169:0/585737290 >> 172.21.15.169:0/585737290 conn(0x7f3ec0069930 msgr2=0x7f3ec014b410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:42:20.629 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:20.626+0000 7f3ec78f2640 1 -- 172.21.15.169:0/585737290 shutdown_connections 2023-10-26T01:42:20.630 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:20.626+0000 7f3ec78f2640 1 -- 172.21.15.169:0/585737290 wait complete. 2023-10-26T01:42:20.647 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2023-10-26T01:42:20.878 INFO:teuthology.orchestra.run.smithi169.stdout:{"cluster_fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","whoami":0,"id":8700,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":10.478533733000001,"mdsmap_epoch":86,"osdmap_epoch":166,"osdmap_epoch_barrier":166,"uptime":16.460243833} 2023-10-26T01:42:20.879 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "30203b28-ed8a-44e8-a9a6-a9bf25f980f1", "fs_name": "cephfs", "id": 8700, "mdsmap_epoch": 86, "osdmap_epoch": 166, "osdmap_epoch_barrier": 166, "rank_uptime": 10.478533733, "state": "up:active", "uptime": 16.460243833, "want_state": "up:active", "whoami": 0 } 2023-10-26T01:42:20.879 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set-quota cephfs_data max_bytes 33554432 2023-10-26T01:42:21.107 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:21.105+0000 7f789d939640 1 -- 172.21.15.169:0/722534620 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f78980aab40 msgr2=0x7f78980ac6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:21.108 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:21.105+0000 7f789d939640 1 --2- 172.21.15.169:0/722534620 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f78980aab40 0x7f78980ac6c0 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f7888005fd0 tx=0x7f788802bd90 comp rx=0 tx=0).stop 2023-10-26T01:42:21.108 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:21.105+0000 7f789d939640 1 -- 172.21.15.169:0/722534620 shutdown_connections 2023-10-26T01:42:21.108 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:21.105+0000 7f789d939640 1 --2- 172.21.15.169:0/722534620 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f78980aab40 0x7f78980ac6c0 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:21.108 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:21.105+0000 7f789d939640 1 -- 172.21.15.169:0/722534620 >> 172.21.15.169:0/722534620 conn(0x7f78980aeee0 msgr2=0x7f78980af2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:42:21.109 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:21.106+0000 7f789d939640 1 -- 172.21.15.169:0/722534620 shutdown_connections 2023-10-26T01:42:21.109 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:21.106+0000 7f789d939640 1 -- 172.21.15.169:0/722534620 wait complete. 2023-10-26T01:42:21.109 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:21.107+0000 7f789d939640 1 Processor -- start 2023-10-26T01:42:21.110 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:21.107+0000 7f789d939640 1 -- start start 2023-10-26T01:42:21.112 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:21.108+0000 7f789d939640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f789806cdb0 0x7f789814e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:21.112 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:21.108+0000 7f789d939640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7898157eb0 0x7f78980c9d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:21.112 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:21.108+0000 7f789d939640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f78980cd190 0x7f78980ae060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:21.112 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:21.108+0000 7f789d939640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78980caf50 con 0x7f7898157eb0 2023-10-26T01:42:21.112 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:21.108+0000 7f789d939640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78981f1320 con 0x7f789806cdb0 2023-10-26T01:42:21.113 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:21.108+0000 7f789d939640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78981f0fc0 con 0x7f78980cd190 2023-10-26T01:42:21.113 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:21.109+0000 7f78967fc640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7898157eb0 0x7f78980c9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:42:21.113 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:21.109+0000 7f78967fc640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7898157eb0 0x7f78980c9d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:54284/0 (socket says 172.21.15.169:54284) 2023-10-26T01:42:21.113 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:21.109+0000 7f78967fc640 1 -- 172.21.15.169:0/505620836 learned_addr learned my addr 172.21.15.169:0/505620836 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:42:21.114 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:21.109+0000 7f7896ffd640 1 --2- 172.21.15.169:0/505620836 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f789806cdb0 0x7f789814e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:42:21.114 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:21.109+0000 7f78977fe640 1 --2- 172.21.15.169:0/505620836 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f78980cd190 0x7f78980ae060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:42:21.114 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:21.109+0000 7f78967fc640 1 -- 172.21.15.169:0/505620836 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f78980cd190 msgr2=0x7f78980ae060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:21.114 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:21.109+0000 7f78967fc640 1 --2- 172.21.15.169:0/505620836 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f78980cd190 0x7f78980ae060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:21.114 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:21.109+0000 7f78967fc640 1 -- 172.21.15.169:0/505620836 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f789806cdb0 msgr2=0x7f789814e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:21.115 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:21.109+0000 7f78967fc640 1 --2- 172.21.15.169:0/505620836 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f789806cdb0 0x7f789814e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:21.115 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:21.109+0000 7f78967fc640 1 -- 172.21.15.169:0/505620836 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7888009560 con 0x7f7898157eb0 2023-10-26T01:42:21.115 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:21.109+0000 7f78977fe640 1 --2- 172.21.15.169:0/505620836 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f78980cd190 0x7f78980ae060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:42:21.115 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:21.110+0000 7f78967fc640 1 --2- 172.21.15.169:0/505620836 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7898157eb0 0x7f78980c9d60 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f788c00c440 tx=0x7f788c001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:42:21.115 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:21.110+0000 7f787ffff640 1 -- 172.21.15.169:0/505620836 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f788c0052d0 con 0x7f7898157eb0 2023-10-26T01:42:21.116 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:21.110+0000 7f787ffff640 1 -- 172.21.15.169:0/505620836 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f788c007020 con 0x7f7898157eb0 2023-10-26T01:42:21.116 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:21.110+0000 7f787ffff640 1 -- 172.21.15.169:0/505620836 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f788c007470 con 0x7f7898157eb0 2023-10-26T01:42:21.116 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:21.110+0000 7f789d939640 1 -- 172.21.15.169:0/505620836 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78980ca6b0 con 0x7f7898157eb0 2023-10-26T01:42:21.117 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:21.110+0000 7f789d939640 1 -- 172.21.15.169:0/505620836 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78980cac70 con 0x7f7898157eb0 2023-10-26T01:42:21.118 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:21.111+0000 7f789d939640 1 -- 172.21.15.169:0/505620836 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7898066af0 con 0x7f7898157eb0 2023-10-26T01:42:21.119 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:21.112+0000 7f787ffff640 1 -- 172.21.15.169:0/505620836 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f788c0067d0 con 0x7f7898157eb0 2023-10-26T01:42:21.119 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:21.113+0000 7f787ffff640 1 --2- 172.21.15.169:0/505620836 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f788406ae70 0x7f788406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:42:21.120 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:21.114+0000 7f787ffff640 1 -- 172.21.15.169:0/505620836 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f788c088cc0 con 0x7f7898157eb0 2023-10-26T01:42:21.120 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:21.114+0000 7f7896ffd640 1 --2- 172.21.15.169:0/505620836 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f788406ae70 0x7f788406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:42:21.120 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:21.115+0000 7f7896ffd640 1 --2- 172.21.15.169:0/505620836 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f788406ae70 0x7f788406d330 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f7888035000 tx=0x7f7888004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:42:21.121 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:21.118+0000 7f787ffff640 1 -- 172.21.15.169:0/505620836 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f788c08f080 con 0x7f7898157eb0 2023-10-26T01:42:21.315 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:21.313+0000 7f789d939640 1 -- 172.21.15.169:0/505620836 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"} v 0) v1 -- 0x7f789804da70 con 0x7f7898157eb0 2023-10-26T01:42:22.183 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:22.181+0000 7f787ffff640 1 -- 172.21.15.169:0/505620836 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"}]=0 set-quota max_bytes = 33554432 for pool cephfs_data v167) v1 ==== 181+0+0 (secure 0 0 0) 0x7f788c053cd0 con 0x7f7898157eb0 2023-10-26T01:42:22.184 INFO:teuthology.orchestra.run.smithi169.stderr:set-quota max_bytes = 33554432 for pool cephfs_data 2023-10-26T01:42:22.187 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:22.184+0000 7f789d939640 1 -- 172.21.15.169:0/505620836 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f788406ae70 msgr2=0x7f788406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:22.187 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:22.184+0000 7f789d939640 1 --2- 172.21.15.169:0/505620836 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f788406ae70 0x7f788406d330 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f7888035000 tx=0x7f7888004730 comp rx=0 tx=0).stop 2023-10-26T01:42:22.188 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:22.184+0000 7f789d939640 1 -- 172.21.15.169:0/505620836 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7898157eb0 msgr2=0x7f78980c9d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:42:22.188 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:22.184+0000 7f789d939640 1 --2- 172.21.15.169:0/505620836 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7898157eb0 0x7f78980c9d60 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f788c00c440 tx=0x7f788c001e70 comp rx=0 tx=0).stop 2023-10-26T01:42:22.188 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:22.184+0000 7f789d939640 1 -- 172.21.15.169:0/505620836 shutdown_connections 2023-10-26T01:42:22.189 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:22.184+0000 7f789d939640 1 --2- 172.21.15.169:0/505620836 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f788406ae70 0x7f788406d330 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:22.189 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:22.184+0000 7f789d939640 1 --2- 172.21.15.169:0/505620836 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7898157eb0 0x7f78980c9d60 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:22.189 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:22.184+0000 7f789d939640 1 --2- 172.21.15.169:0/505620836 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f78980cd190 0x7f78980ae060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:22.189 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:22.184+0000 7f789d939640 1 --2- 172.21.15.169:0/505620836 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f789806cdb0 0x7f789814e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:42:22.189 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:22.184+0000 7f789d939640 1 -- 172.21.15.169:0/505620836 >> 172.21.15.169:0/505620836 conn(0x7f78980aeee0 msgr2=0x7f789806bb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:42:22.190 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:22.185+0000 7f789d939640 1 -- 172.21.15.169:0/505620836 shutdown_connections 2023-10-26T01:42:22.190 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:42:22.185+0000 7f789d939640 1 -- 172.21.15.169:0/505620836 wait complete. 2023-10-26T01:42:22.208 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2023-10-26T01:42:22.439 INFO:teuthology.orchestra.run.smithi169.stdout:{"osd_mon_report_interval":"5"} 2023-10-26T01:42:22.440 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2023-10-26T01:42:22.440 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mon_tick_interval 2023-10-26T01:42:22.659 INFO:teuthology.orchestra.run.smithi169.stdout:{"mon_tick_interval":"5"} 2023-10-26T01:42:22.660 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mon_tick_interval": "5" } 2023-10-26T01:42:22.660 INFO:tasks.cephfs.test_full:pool capacity 33554432, 32MB should be enough to fill it 2023-10-26T01:42:22.661 DEBUG:teuthology.orchestra.run.smithi169:> stdin-killer -- python3 -c ' 2023-10-26T01:42:22.661 DEBUG:teuthology.orchestra.run.smithi169:> import time 2023-10-26T01:42:22.662 DEBUG:teuthology.orchestra.run.smithi169:> import datetime 2023-10-26T01:42:22.662 DEBUG:teuthology.orchestra.run.smithi169:> import subprocess 2023-10-26T01:42:22.662 DEBUG:teuthology.orchestra.run.smithi169:> import os 2023-10-26T01:42:22.663 DEBUG:teuthology.orchestra.run.smithi169:> 2023-10-26T01:42:22.663 DEBUG:teuthology.orchestra.run.smithi169:> # Write some buffered data through before going full, all should be well 2023-10-26T01:42:22.663 DEBUG:teuthology.orchestra.run.smithi169:> print("writing some data through which we expect to succeed") 2023-10-26T01:42:22.664 DEBUG:teuthology.orchestra.run.smithi169:> bytes = 0 2023-10-26T01:42:22.664 DEBUG:teuthology.orchestra.run.smithi169:> f = os.open("/home/ubuntu/cephtest/mnt.0/full_test_file", os.O_WRONLY | os.O_CREAT) 2023-10-26T01:42:22.664 DEBUG:teuthology.orchestra.run.smithi169:> os.setxattr("/home/ubuntu/cephtest/mnt.0/full_test_file", '"'"'ceph.file.layout'"'"', b'"'"'stripe_unit=1048576 stripe_count=1 object_size=1048576'"'"') 2023-10-26T01:42:22.665 DEBUG:teuthology.orchestra.run.smithi169:> bytes += os.write(f, b'"'"'a'"'"' * 4096) 2023-10-26T01:42:22.665 DEBUG:teuthology.orchestra.run.smithi169:> os.fsync(f) 2023-10-26T01:42:22.665 DEBUG:teuthology.orchestra.run.smithi169:> print("fsync'"'"'ed data successfully, will now attempt to fill fs") 2023-10-26T01:42:22.666 DEBUG:teuthology.orchestra.run.smithi169:> 2023-10-26T01:42:22.666 DEBUG:teuthology.orchestra.run.smithi169:> # Okay, now we'"'"'re going to fill up the filesystem, and then keep 2023-10-26T01:42:22.666 DEBUG:teuthology.orchestra.run.smithi169:> # writing until we see an error from fsync. As long as we'"'"'re doing 2023-10-26T01:42:22.667 DEBUG:teuthology.orchestra.run.smithi169:> # buffered IO, the error should always only appear from fsync and not 2023-10-26T01:42:22.667 DEBUG:teuthology.orchestra.run.smithi169:> # from write 2023-10-26T01:42:22.667 DEBUG:teuthology.orchestra.run.smithi169:> full = False 2023-10-26T01:42:22.668 DEBUG:teuthology.orchestra.run.smithi169:> 2023-10-26T01:42:22.668 DEBUG:teuthology.orchestra.run.smithi169:> for n in range(0, int(32 * 1.1)): 2023-10-26T01:42:22.668 DEBUG:teuthology.orchestra.run.smithi169:> try: 2023-10-26T01:42:22.669 DEBUG:teuthology.orchestra.run.smithi169:> bytes += os.write(f, b'"'"'x'"'"' * 1024 * 1024) 2023-10-26T01:42:22.669 DEBUG:teuthology.orchestra.run.smithi169:> print("wrote bytes via buffered write, moving on to fsync") 2023-10-26T01:42:22.669 DEBUG:teuthology.orchestra.run.smithi169:> except OSError as e: 2023-10-26T01:42:22.670 DEBUG:teuthology.orchestra.run.smithi169:> if False: 2023-10-26T01:42:22.670 DEBUG:teuthology.orchestra.run.smithi169:> print("Unexpected error %s from write() instead of fsync()" % e) 2023-10-26T01:42:22.670 DEBUG:teuthology.orchestra.run.smithi169:> raise 2023-10-26T01:42:22.671 DEBUG:teuthology.orchestra.run.smithi169:> else: 2023-10-26T01:42:22.671 DEBUG:teuthology.orchestra.run.smithi169:> print("Reached fullness after %.2f MB" % (bytes / (1024.0 * 1024.0))) 2023-10-26T01:42:22.671 DEBUG:teuthology.orchestra.run.smithi169:> full = True 2023-10-26T01:42:22.672 DEBUG:teuthology.orchestra.run.smithi169:> break 2023-10-26T01:42:22.672 DEBUG:teuthology.orchestra.run.smithi169:> 2023-10-26T01:42:22.672 DEBUG:teuthology.orchestra.run.smithi169:> try: 2023-10-26T01:42:22.673 DEBUG:teuthology.orchestra.run.smithi169:> os.fsync(f) 2023-10-26T01:42:22.673 DEBUG:teuthology.orchestra.run.smithi169:> print("fsync'"'"'ed successfully") 2023-10-26T01:42:22.673 DEBUG:teuthology.orchestra.run.smithi169:> except OSError as e: 2023-10-26T01:42:22.674 DEBUG:teuthology.orchestra.run.smithi169:> print("Reached fullness after %.2f MB" % (bytes / (1024.0 * 1024.0))) 2023-10-26T01:42:22.674 DEBUG:teuthology.orchestra.run.smithi169:> full = True 2023-10-26T01:42:22.674 DEBUG:teuthology.orchestra.run.smithi169:> break 2023-10-26T01:42:22.675 DEBUG:teuthology.orchestra.run.smithi169:> else: 2023-10-26T01:42:22.675 DEBUG:teuthology.orchestra.run.smithi169:> print("Not full yet after %.2f MB" % (bytes / (1024.0 * 1024.0))) 2023-10-26T01:42:22.675 DEBUG:teuthology.orchestra.run.smithi169:> 2023-10-26T01:42:22.675 DEBUG:teuthology.orchestra.run.smithi169:> if n > 32 * 0.9: 2023-10-26T01:42:22.676 DEBUG:teuthology.orchestra.run.smithi169:> # Be cautious in the last region where we expect to hit 2023-10-26T01:42:22.676 DEBUG:teuthology.orchestra.run.smithi169:> # the full condition, so that we don'"'"'t overshoot too dramatically 2023-10-26T01:42:22.676 DEBUG:teuthology.orchestra.run.smithi169:> print("sleeping a bit as we'"'"'ve exceeded 90% of our expected full ratio") 2023-10-26T01:42:22.677 DEBUG:teuthology.orchestra.run.smithi169:> time.sleep(15.0) 2023-10-26T01:42:22.677 DEBUG:teuthology.orchestra.run.smithi169:> 2023-10-26T01:42:22.677 DEBUG:teuthology.orchestra.run.smithi169:> if not full: 2023-10-26T01:42:22.678 DEBUG:teuthology.orchestra.run.smithi169:> raise RuntimeError("Failed to reach fullness after writing %d bytes" % bytes) 2023-10-26T01:42:22.678 DEBUG:teuthology.orchestra.run.smithi169:> 2023-10-26T01:42:22.678 DEBUG:teuthology.orchestra.run.smithi169:> # close() should not raise an error because we already caught it in 2023-10-26T01:42:22.679 DEBUG:teuthology.orchestra.run.smithi169:> # fsync. There shouldn'"'"'t have been any more writeback errors 2023-10-26T01:42:22.679 DEBUG:teuthology.orchestra.run.smithi169:> # since then because all IOs got cancelled on the full flag. 2023-10-26T01:42:22.679 DEBUG:teuthology.orchestra.run.smithi169:> print("calling close") 2023-10-26T01:42:22.680 DEBUG:teuthology.orchestra.run.smithi169:> os.close(f) 2023-10-26T01:42:22.680 DEBUG:teuthology.orchestra.run.smithi169:> print("close() did not raise error") 2023-10-26T01:42:22.680 DEBUG:teuthology.orchestra.run.smithi169:> 2023-10-26T01:42:22.681 DEBUG:teuthology.orchestra.run.smithi169:> os.unlink("/home/ubuntu/cephtest/mnt.0/full_test_file") 2023-10-26T01:42:22.681 DEBUG:teuthology.orchestra.run.smithi169:> ' 2023-10-26T01:42:37.729 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T01:42:37.739 DEBUG:teuthology.orchestra.run.smithi170:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T01:43:07.876 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T01:43:07.886 DEBUG:teuthology.orchestra.run.smithi170:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T01:43:24.911 INFO:teuthology.orchestra.run.smithi169.stdout:writing some data through which we expect to succeed 2023-10-26T01:43:24.911 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed data successfully, will now attempt to fill fs 2023-10-26T01:43:24.912 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:43:24.912 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:43:24.912 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 1.00 MB 2023-10-26T01:43:24.912 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:43:24.912 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:43:24.913 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 2.00 MB 2023-10-26T01:43:24.913 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:43:24.913 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:43:24.913 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 3.00 MB 2023-10-26T01:43:24.913 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:43:24.913 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:43:24.914 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 4.00 MB 2023-10-26T01:43:24.914 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:43:24.914 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:43:24.914 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 5.00 MB 2023-10-26T01:43:24.914 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:43:24.915 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:43:24.915 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 6.00 MB 2023-10-26T01:43:24.915 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:43:24.915 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:43:24.915 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 7.00 MB 2023-10-26T01:43:24.916 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:43:24.918 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:43:24.918 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 8.00 MB 2023-10-26T01:43:24.919 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:43:24.919 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:43:24.919 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 9.00 MB 2023-10-26T01:43:24.919 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:43:24.920 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:43:24.920 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 10.00 MB 2023-10-26T01:43:24.920 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:43:24.920 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:43:24.920 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 11.00 MB 2023-10-26T01:43:24.921 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:43:24.921 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:43:24.921 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 12.00 MB 2023-10-26T01:43:24.921 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:43:24.922 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:43:24.922 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 13.00 MB 2023-10-26T01:43:24.922 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:43:24.922 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:43:24.922 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 14.00 MB 2023-10-26T01:43:24.923 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:43:24.923 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:43:24.923 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 15.00 MB 2023-10-26T01:43:24.923 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:43:24.923 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:43:24.924 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 16.00 MB 2023-10-26T01:43:24.924 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:43:24.924 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:43:24.924 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 17.00 MB 2023-10-26T01:43:24.924 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:43:24.925 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:43:24.925 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 18.00 MB 2023-10-26T01:43:24.925 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:43:24.925 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:43:24.925 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 19.00 MB 2023-10-26T01:43:24.926 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:43:24.926 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:43:24.926 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 20.00 MB 2023-10-26T01:43:24.926 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:43:24.927 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:43:24.927 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 21.00 MB 2023-10-26T01:43:24.927 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:43:24.927 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:43:24.927 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 22.00 MB 2023-10-26T01:43:24.928 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:43:24.928 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:43:24.928 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 23.00 MB 2023-10-26T01:43:24.928 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:43:24.928 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:43:24.929 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 24.00 MB 2023-10-26T01:43:24.929 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:43:24.929 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:43:24.929 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 25.00 MB 2023-10-26T01:43:24.930 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:43:24.930 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:43:24.930 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 26.00 MB 2023-10-26T01:43:24.930 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:43:24.930 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:43:24.931 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 27.00 MB 2023-10-26T01:43:24.931 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:43:24.931 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:43:24.931 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 28.00 MB 2023-10-26T01:43:24.931 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:43:24.932 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:43:24.932 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 29.00 MB 2023-10-26T01:43:24.932 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:43:24.932 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:43:24.933 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 30.00 MB 2023-10-26T01:43:24.933 INFO:teuthology.orchestra.run.smithi169.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2023-10-26T01:43:24.933 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:43:24.933 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:43:24.934 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 31.00 MB 2023-10-26T01:43:24.934 INFO:teuthology.orchestra.run.smithi169.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2023-10-26T01:43:24.934 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:43:24.934 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:43:24.934 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 32.00 MB 2023-10-26T01:43:24.935 INFO:teuthology.orchestra.run.smithi169.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2023-10-26T01:43:24.935 INFO:teuthology.orchestra.run.smithi169.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T01:43:24.935 INFO:teuthology.orchestra.run.smithi169.stdout:fsync'ed successfully 2023-10-26T01:43:24.935 INFO:teuthology.orchestra.run.smithi169.stdout:Not full yet after 33.00 MB 2023-10-26T01:43:24.935 INFO:teuthology.orchestra.run.smithi169.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2023-10-26T01:43:24.936 INFO:teuthology.orchestra.run.smithi169.stdout:Reached fullness after 33.00 MB 2023-10-26T01:43:24.936 INFO:teuthology.orchestra.run.smithi169.stdout:calling close 2023-10-26T01:43:24.936 INFO:teuthology.orchestra.run.smithi169.stdout:close() did not raise error 2023-10-26T01:43:24.937 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T01:43:24.946 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-26T01:43:24.946 DEBUG:teuthology.orchestra.run.smithi169:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-26T01:43:25.079 DEBUG:teuthology.orchestra.run.smithi170:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-26T01:43:25.205 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-26T01:43:25.205 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:43:25.206 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:43:25.268 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:43:25.269 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:43:25.324 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:43:25.546 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:25.544+0000 7f3bf5d3c640 1 -- 172.21.15.169:0/4107858240 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3bd80047c0 msgr2=0x7f3bd8004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:25.547 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:25.544+0000 7f3bf5d3c640 1 --2- 172.21.15.169:0/4107858240 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3bd80047c0 0x7f3bd8004c40 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f3be0009b50 tx=0x7f3bd80057f0 comp rx=0 tx=0).stop 2023-10-26T01:43:25.547 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:25.544+0000 7f3bf553b640 1 -- 172.21.15.169:0/4107858240 reap_dead start 2023-10-26T01:43:25.547 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:25.545+0000 7f3bf5d3c640 1 -- 172.21.15.169:0/4107858240 shutdown_connections 2023-10-26T01:43:25.547 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:25.545+0000 7f3bf5d3c640 1 -- 172.21.15.169:0/4107858240 >> 172.21.15.169:0/4107858240 conn(0x7f3bf00aedd0 msgr2=0x7f3bf00af1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:25.548 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:25.545+0000 7f3bf5d3c640 1 -- 172.21.15.169:0/4107858240 shutdown_connections 2023-10-26T01:43:25.548 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:25.545+0000 7f3bf5d3c640 1 -- 172.21.15.169:0/4107858240 wait complete. 2023-10-26T01:43:25.548 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:25.546+0000 7f3bf5d3c640 1 Processor -- start 2023-10-26T01:43:25.549 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:25.547+0000 7f3bf5d3c640 1 -- start start 2023-10-26T01:43:25.551 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:25.547+0000 7f3bf5d3c640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3bd80047c0 0x7f3bf01f2870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:25.551 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:25.547+0000 7f3bf5d3c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3bf00b10f0 0x7f3bf0153820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:25.551 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:25.547+0000 7f3bf5d3c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3bf01567a0 0x7f3bf0156c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:25.551 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:25.548+0000 7f3bf5d3c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf0157160 con 0x7f3bf00b10f0 2023-10-26T01:43:25.551 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:25.548+0000 7f3bf5d3c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf01f2db0 con 0x7f3bf01567a0 2023-10-26T01:43:25.552 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:25.548+0000 7f3bf5d3c640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf00b0050 con 0x7f3bd80047c0 2023-10-26T01:43:25.552 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:25.548+0000 7f3beffff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3bf00b10f0 0x7f3bf0153820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:25.552 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:25.548+0000 7f3beffff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3bf00b10f0 0x7f3bf0153820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:49484/0 (socket says 172.21.15.169:49484) 2023-10-26T01:43:25.552 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:25.548+0000 7f3beffff640 1 -- 172.21.15.169:0/1503037042 learned_addr learned my addr 172.21.15.169:0/1503037042 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:43:25.552 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:25.548+0000 7f3bf4d3a640 1 --2- 172.21.15.169:0/1503037042 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3bd80047c0 0x7f3bf01f2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:25.553 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:25.548+0000 7f3bf553b640 1 --2- 172.21.15.169:0/1503037042 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3bf01567a0 0x7f3bf0156c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:25.553 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:25.548+0000 7f3beffff640 1 -- 172.21.15.169:0/1503037042 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3bd80047c0 msgr2=0x7f3bf01f2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:25.553 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:25.548+0000 7f3beffff640 1 --2- 172.21.15.169:0/1503037042 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3bd80047c0 0x7f3bf01f2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:25.553 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:25.548+0000 7f3beffff640 1 -- 172.21.15.169:0/1503037042 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3bf01567a0 msgr2=0x7f3bf0156c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:25.553 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:25.548+0000 7f3beffff640 1 --2- 172.21.15.169:0/1503037042 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3bf01567a0 0x7f3bf0156c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:25.554 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:25.548+0000 7f3beffff640 1 -- 172.21.15.169:0/1503037042 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bf00b0330 con 0x7f3bf00b10f0 2023-10-26T01:43:25.554 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:25.548+0000 7f3bf553b640 1 --2- 172.21.15.169:0/1503037042 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3bf01567a0 0x7f3bf0156c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:43:25.554 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:25.549+0000 7f3beffff640 1 --2- 172.21.15.169:0/1503037042 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3bf00b10f0 0x7f3bf0153820 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f3be0009a70 tx=0x7f3be0033c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:25.554 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:25.549+0000 7f3bed7fa640 1 -- 172.21.15.169:0/1503037042 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be0004220 con 0x7f3bf00b10f0 2023-10-26T01:43:25.554 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:25.549+0000 7f3bed7fa640 1 -- 172.21.15.169:0/1503037042 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3be0004380 con 0x7f3bf00b10f0 2023-10-26T01:43:25.555 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:25.549+0000 7f3bed7fa640 1 -- 172.21.15.169:0/1503037042 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be003d940 con 0x7f3bf00b10f0 2023-10-26T01:43:25.555 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:25.549+0000 7f3bf5d3c640 1 -- 172.21.15.169:0/1503037042 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bf00b0610 con 0x7f3bf00b10f0 2023-10-26T01:43:25.555 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:25.549+0000 7f3bf5d3c640 1 -- 172.21.15.169:0/1503037042 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bf00b0bd0 con 0x7f3bf00b10f0 2023-10-26T01:43:25.556 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:25.550+0000 7f3bf5d3c640 1 -- 172.21.15.169:0/1503037042 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bf00b0d60 con 0x7f3bf00b10f0 2023-10-26T01:43:25.556 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:25.551+0000 7f3bed7fa640 1 -- 172.21.15.169:0/1503037042 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f3be003daa0 con 0x7f3bf00b10f0 2023-10-26T01:43:25.557 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:25.552+0000 7f3bed7fa640 1 --2- 172.21.15.169:0/1503037042 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3bdc06ae70 0x7f3bdc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:25.557 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:25.553+0000 7f3bed7fa640 1 -- 172.21.15.169:0/1503037042 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f3be0033e40 con 0x7f3bf00b10f0 2023-10-26T01:43:25.557 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:25.553+0000 7f3bed7fa640 1 -- 172.21.15.169:0/1503037042 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=171}) v3 -- 0x7f3bdc072a00 con 0x7f3bf00b10f0 2023-10-26T01:43:25.557 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:25.553+0000 7f3bf4d3a640 1 --2- 172.21.15.169:0/1503037042 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3bdc06ae70 0x7f3bdc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:25.557 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:25.554+0000 7f3bf4d3a640 1 --2- 172.21.15.169:0/1503037042 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3bdc06ae70 0x7f3bdc06d330 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f3be4009bb0 tx=0x7f3be4005f90 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:25.559 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:25.557+0000 7f3bed7fa640 1 -- 172.21.15.169:0/1503037042 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3be004a060 con 0x7f3bf00b10f0 2023-10-26T01:43:25.748 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:25.746+0000 7f3bf5d3c640 1 -- 172.21.15.169:0/1503037042 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3bf01585c0 con 0x7f3bf00b10f0 2023-10-26T01:43:25.750 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:25.748+0000 7f3bed7fa640 1 -- 172.21.15.169:0/1503037042 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v170) v1 ==== 74+0+16704 (secure 0 0 0) 0x7f3be004a030 con 0x7f3bf00b10f0 2023-10-26T01:43:25.750 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:43:25.751 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":170,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:42:58.161990+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2023-10-26T01:42:07.656531+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"165","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2023-10-26T01:42:08.866635+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"170","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.39","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:43:25.753 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:25.751+0000 7f3bf5d3c640 1 -- 172.21.15.169:0/1503037042 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3bdc06ae70 msgr2=0x7f3bdc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:25.753 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:25.751+0000 7f3bf5d3c640 1 --2- 172.21.15.169:0/1503037042 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3bdc06ae70 0x7f3bdc06d330 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f3be4009bb0 tx=0x7f3be4005f90 comp rx=0 tx=0).stop 2023-10-26T01:43:25.754 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:25.751+0000 7f3bf5d3c640 1 -- 172.21.15.169:0/1503037042 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3bf00b10f0 msgr2=0x7f3bf0153820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:25.754 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:25.751+0000 7f3bf5d3c640 1 --2- 172.21.15.169:0/1503037042 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3bf00b10f0 0x7f3bf0153820 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f3be0009a70 tx=0x7f3be0033c50 comp rx=0 tx=0).stop 2023-10-26T01:43:25.754 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:25.751+0000 7f3bf5d3c640 1 -- 172.21.15.169:0/1503037042 shutdown_connections 2023-10-26T01:43:25.755 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:25.751+0000 7f3bf5d3c640 1 --2- 172.21.15.169:0/1503037042 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3bdc06ae70 0x7f3bdc06d330 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:25.755 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:25.751+0000 7f3bf5d3c640 1 --2- 172.21.15.169:0/1503037042 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3bf00b10f0 0x7f3bf0153820 unknown :-1 s=CLOSED pgs=1729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:25.755 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:25.751+0000 7f3bf5d3c640 1 --2- 172.21.15.169:0/1503037042 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3bf01567a0 0x7f3bf0156c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:25.755 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:25.751+0000 7f3bf5d3c640 1 --2- 172.21.15.169:0/1503037042 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3bd80047c0 0x7f3bf01f2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:25.755 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:25.751+0000 7f3bf5d3c640 1 -- 172.21.15.169:0/1503037042 >> 172.21.15.169:0/1503037042 conn(0x7f3bf00aedd0 msgr2=0x7f3bf00a9ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:25.756 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:25.751+0000 7f3bf5d3c640 1 -- 172.21.15.169:0/1503037042 shutdown_connections 2023-10-26T01:43:25.756 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:25.751+0000 7f3bf5d3c640 1 -- 172.21.15.169:0/1503037042 wait complete. 2023-10-26T01:43:25.771 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T01:30:32.200808+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '109', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 20, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-26T01:42:07.656531+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '165', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 21, 'pool_name': 'cephfs_data', 'create_time': '2023-10-26T01:42:08.866635+0000', 'flags': 1027, 'flags_names': 'hashpspool,full,full_quota', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '170', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T01:43:25.772 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T01:43:25.998 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:25.996+0000 7f29e5fb6640 1 -- 172.21.15.169:0/2503456062 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29e00acae0 msgr2=0x7f29e01665d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:25.998 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:25.996+0000 7f29e5fb6640 1 --2- 172.21.15.169:0/2503456062 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29e00acae0 0x7f29e01665d0 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f29d8005fd0 tx=0x7f29d802c260 comp rx=0 tx=0).stop 2023-10-26T01:43:25.998 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:25.996+0000 7f29e5fb6640 1 -- 172.21.15.169:0/2503456062 shutdown_connections 2023-10-26T01:43:25.999 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:25.996+0000 7f29e5fb6640 1 --2- 172.21.15.169:0/2503456062 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29e00acae0 0x7f29e01665d0 unknown :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:25.999 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:25.996+0000 7f29e5fb6640 1 -- 172.21.15.169:0/2503456062 >> 172.21.15.169:0/2503456062 conn(0x7f29e0069930 msgr2=0x7f29e0069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:25.999 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:25.996+0000 7f29e5fb6640 1 -- 172.21.15.169:0/2503456062 shutdown_connections 2023-10-26T01:43:25.999 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:25.996+0000 7f29e5fb6640 1 -- 172.21.15.169:0/2503456062 wait complete. 2023-10-26T01:43:26.000 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:25.998+0000 7f29e5fb6640 1 Processor -- start 2023-10-26T01:43:26.001 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:25.998+0000 7f29e5fb6640 1 -- start start 2023-10-26T01:43:26.001 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:25.999+0000 7f29e5fb6640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29e00acae0 0x7f29e0147380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:26.001 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:25.999+0000 7f29e5fb6640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f29e00cdda0 0x7f29e0148390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:26.001 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:25.999+0000 7f29e5fb6640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f29e015b6a0 0x7f29e014ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:26.001 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:25.999+0000 7f29e5fb6640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29e0149a60 con 0x7f29e00acae0 2023-10-26T01:43:26.002 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:25.999+0000 7f29e5fb6640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29e0148e50 con 0x7f29e00cdda0 2023-10-26T01:43:26.002 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:25.999+0000 7f29e5fb6640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29e0149130 con 0x7f29e015b6a0 2023-10-26T01:43:26.002 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:25.999+0000 7f29df7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29e00acae0 0x7f29e0147380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:26.002 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:25.999+0000 7f29df7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29e00acae0 0x7f29e0147380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:49490/0 (socket says 172.21.15.169:49490) 2023-10-26T01:43:26.002 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:25.999+0000 7f29df7fe640 1 -- 172.21.15.169:0/84571823 learned_addr learned my addr 172.21.15.169:0/84571823 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:43:26.004 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:25.999+0000 7f29d7fff640 1 --2- 172.21.15.169:0/84571823 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f29e00cdda0 0x7f29e0148390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:26.004 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:25.999+0000 7f29df7fe640 1 -- 172.21.15.169:0/84571823 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f29e015b6a0 msgr2=0x7f29e014ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:26.004 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:25.999+0000 7f29df7fe640 1 --2- 172.21.15.169:0/84571823 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f29e015b6a0 0x7f29e014ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:26.004 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:25.999+0000 7f29df7fe640 1 -- 172.21.15.169:0/84571823 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f29e00cdda0 msgr2=0x7f29e0148390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:26.005 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:25.999+0000 7f29df7fe640 1 --2- 172.21.15.169:0/84571823 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f29e00cdda0 0x7f29e0148390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:26.005 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:25.999+0000 7f29df7fe640 1 -- 172.21.15.169:0/84571823 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29d8009900 con 0x7f29e00acae0 2023-10-26T01:43:26.005 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:25.999+0000 7f29d7fff640 1 --2- 172.21.15.169:0/84571823 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f29e00cdda0 0x7f29e0148390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:43:26.005 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.000+0000 7f29df7fe640 1 --2- 172.21.15.169:0/84571823 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29e00acae0 0x7f29e0147380 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f29d000b3f0 tx=0x7f29d000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:26.005 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.000+0000 7f29dcff9640 1 -- 172.21.15.169:0/84571823 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d0011830 con 0x7f29e00acae0 2023-10-26T01:43:26.006 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.000+0000 7f29dcff9640 1 -- 172.21.15.169:0/84571823 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f29d0011a80 con 0x7f29e00acae0 2023-10-26T01:43:26.006 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.000+0000 7f29dcff9640 1 -- 172.21.15.169:0/84571823 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d0011e90 con 0x7f29e00acae0 2023-10-26T01:43:26.006 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.000+0000 7f29e5fb6640 1 -- 172.21.15.169:0/84571823 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29e0149410 con 0x7f29e00acae0 2023-10-26T01:43:26.006 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.000+0000 7f29e5fb6640 1 -- 172.21.15.169:0/84571823 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29e015dfa0 con 0x7f29e00acae0 2023-10-26T01:43:26.006 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.001+0000 7f29e5fb6640 1 -- 172.21.15.169:0/84571823 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29e0066af0 con 0x7f29e00acae0 2023-10-26T01:43:26.007 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.002+0000 7f29dcff9640 1 -- 172.21.15.169:0/84571823 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f29d001a400 con 0x7f29e00acae0 2023-10-26T01:43:26.008 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.003+0000 7f29dcff9640 1 --2- 172.21.15.169:0/84571823 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f29c006ae40 0x7f29c006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:26.008 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.003+0000 7f29dcff9640 1 -- 172.21.15.169:0/84571823 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f29d00896f0 con 0x7f29e00acae0 2023-10-26T01:43:26.008 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.003+0000 7f29dcff9640 1 -- 172.21.15.169:0/84571823 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=171}) v3 -- 0x7f29c00729d0 con 0x7f29e00acae0 2023-10-26T01:43:26.008 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.004+0000 7f29d7fff640 1 --2- 172.21.15.169:0/84571823 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f29c006ae40 0x7f29c006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:26.008 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.005+0000 7f29d7fff640 1 --2- 172.21.15.169:0/84571823 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f29c006ae40 0x7f29c006d300 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f29e01f5800 tx=0x7f29c8009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:26.009 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.008+0000 7f29dcff9640 1 -- 172.21.15.169:0/84571823 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f29d0015060 con 0x7f29e00acae0 2023-10-26T01:43:26.202 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.200+0000 7f29e5fb6640 1 -- 172.21.15.169:0/84571823 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f29e004da70 con 0x7f29e00acae0 2023-10-26T01:43:26.203 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.202+0000 7f29dcff9640 1 -- 172.21.15.169:0/84571823 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v170) v1 ==== 93+0+10 (secure 0 0 0) 0x7f29d0054780 con 0x7f29e00acae0 2023-10-26T01:43:26.204 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 1 2023-10-26T01:43:26.206 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.204+0000 7f29e5fb6640 1 -- 172.21.15.169:0/84571823 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f29c006ae40 msgr2=0x7f29c006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:26.206 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.204+0000 7f29e5fb6640 1 --2- 172.21.15.169:0/84571823 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f29c006ae40 0x7f29c006d300 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f29e01f5800 tx=0x7f29c8009040 comp rx=0 tx=0).stop 2023-10-26T01:43:26.206 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.204+0000 7f29e5fb6640 1 -- 172.21.15.169:0/84571823 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29e00acae0 msgr2=0x7f29e0147380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:26.207 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.204+0000 7f29e5fb6640 1 --2- 172.21.15.169:0/84571823 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29e00acae0 0x7f29e0147380 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f29d000b3f0 tx=0x7f29d000b8b0 comp rx=0 tx=0).stop 2023-10-26T01:43:26.207 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.204+0000 7f29e5fb6640 1 -- 172.21.15.169:0/84571823 shutdown_connections 2023-10-26T01:43:26.207 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.204+0000 7f29e5fb6640 1 --2- 172.21.15.169:0/84571823 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f29c006ae40 0x7f29c006d300 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:26.207 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.204+0000 7f29e5fb6640 1 --2- 172.21.15.169:0/84571823 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f29e015b6a0 0x7f29e014ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:26.207 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.204+0000 7f29e5fb6640 1 --2- 172.21.15.169:0/84571823 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f29e00cdda0 0x7f29e0148390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:26.208 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.204+0000 7f29e5fb6640 1 --2- 172.21.15.169:0/84571823 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29e00acae0 0x7f29e0147380 unknown :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:26.208 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.204+0000 7f29e5fb6640 1 -- 172.21.15.169:0/84571823 >> 172.21.15.169:0/84571823 conn(0x7f29e0069930 msgr2=0x7f29e00ac3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:26.208 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.204+0000 7f29e5fb6640 1 -- 172.21.15.169:0/84571823 shutdown_connections 2023-10-26T01:43:26.208 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.204+0000 7f29e5fb6640 1 -- 172.21.15.169:0/84571823 wait complete. 2023-10-26T01:43:26.226 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-26T01:43:26.449 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.447+0000 7fb9a4989640 1 -- 172.21.15.169:0/2822148639 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb98c0047c0 msgr2=0x7fb98c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:26.450 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.447+0000 7fb9a4989640 1 --2- 172.21.15.169:0/2822148639 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb98c0047c0 0x7fb98c004c40 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7fb988006fa0 tx=0x7fb98c0057f0 comp rx=0 tx=0).stop 2023-10-26T01:43:26.450 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.447+0000 7fb99f59e640 1 -- 172.21.15.169:0/2822148639 reap_dead start 2023-10-26T01:43:26.450 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.447+0000 7fb9a4989640 1 -- 172.21.15.169:0/2822148639 shutdown_connections 2023-10-26T01:43:26.450 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.447+0000 7fb9a4989640 1 -- 172.21.15.169:0/2822148639 >> 172.21.15.169:0/2822148639 conn(0x7fb9a00af110 msgr2=0x7fb9a00af520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:26.451 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.448+0000 7fb9a4989640 1 -- 172.21.15.169:0/2822148639 shutdown_connections 2023-10-26T01:43:26.451 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.448+0000 7fb9a4989640 1 -- 172.21.15.169:0/2822148639 wait complete. 2023-10-26T01:43:26.451 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.449+0000 7fb9a4989640 1 Processor -- start 2023-10-26T01:43:26.452 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.449+0000 7fb9a4989640 1 -- start start 2023-10-26T01:43:26.453 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.450+0000 7fb9a4989640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb98c0047c0 0x7fb9a006b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:26.454 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.450+0000 7fb9a4989640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9a00c65e0 0x7fb9a01f4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:26.454 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.450+0000 7fb9a4989640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb9a00c6a60 0x7fb9a01f5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:26.454 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.450+0000 7fb9a4989640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9a006c2a0 con 0x7fb9a00c65e0 2023-10-26T01:43:26.454 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.450+0000 7fb9a4989640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9a006ba60 con 0x7fb9a00c6a60 2023-10-26T01:43:26.455 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.450+0000 7fb9a4989640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9a006bce0 con 0x7fb98c0047c0 2023-10-26T01:43:26.455 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.451+0000 7fb997fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9a00c65e0 0x7fb9a01f4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:26.455 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.451+0000 7fb997fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9a00c65e0 0x7fb9a01f4d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:49508/0 (socket says 172.21.15.169:49508) 2023-10-26T01:43:26.455 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.451+0000 7fb997fff640 1 -- 172.21.15.169:0/2834226765 learned_addr learned my addr 172.21.15.169:0/2834226765 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:43:26.455 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.451+0000 7fb99ed9d640 1 --2- 172.21.15.169:0/2834226765 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb98c0047c0 0x7fb9a006b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:26.456 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.451+0000 7fb99f59e640 1 --2- 172.21.15.169:0/2834226765 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb9a00c6a60 0x7fb9a01f5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:26.456 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.451+0000 7fb997fff640 1 -- 172.21.15.169:0/2834226765 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb98c0047c0 msgr2=0x7fb9a006b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:26.456 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.451+0000 7fb997fff640 1 --2- 172.21.15.169:0/2834226765 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb98c0047c0 0x7fb9a006b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:26.456 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.451+0000 7fb997fff640 1 -- 172.21.15.169:0/2834226765 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb9a00c6a60 msgr2=0x7fb9a01f5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:26.456 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.451+0000 7fb997fff640 1 --2- 172.21.15.169:0/2834226765 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb9a00c6a60 0x7fb9a01f5240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:26.457 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.451+0000 7fb997fff640 1 -- 172.21.15.169:0/2834226765 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9a006bfc0 con 0x7fb9a00c65e0 2023-10-26T01:43:26.457 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.451+0000 7fb99f59e640 1 --2- 172.21.15.169:0/2834226765 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb9a00c6a60 0x7fb9a01f5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:43:26.457 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.451+0000 7fb997fff640 1 --2- 172.21.15.169:0/2834226765 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9a00c65e0 0x7fb9a01f4d00 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7fb988000f60 tx=0x7fb988001650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:26.457 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.452+0000 7fb9977fe640 1 -- 172.21.15.169:0/2834226765 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb988006230 con 0x7fb9a00c65e0 2023-10-26T01:43:26.457 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.452+0000 7fb9977fe640 1 -- 172.21.15.169:0/2834226765 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb988006390 con 0x7fb9a00c65e0 2023-10-26T01:43:26.458 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.452+0000 7fb9977fe640 1 -- 172.21.15.169:0/2834226765 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9880035f0 con 0x7fb9a00c65e0 2023-10-26T01:43:26.458 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.452+0000 7fb9a4989640 1 -- 172.21.15.169:0/2834226765 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9a01551b0 con 0x7fb9a00c65e0 2023-10-26T01:43:26.458 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.452+0000 7fb9a4989640 1 -- 172.21.15.169:0/2834226765 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9a00c9a10 con 0x7fb9a00c65e0 2023-10-26T01:43:26.459 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.453+0000 7fb9a4989640 1 -- 172.21.15.169:0/2834226765 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9a0155010 con 0x7fb9a00c65e0 2023-10-26T01:43:26.459 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.454+0000 7fb9977fe640 1 -- 172.21.15.169:0/2834226765 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fb988031030 con 0x7fb9a00c65e0 2023-10-26T01:43:26.459 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.455+0000 7fb9977fe640 1 --2- 172.21.15.169:0/2834226765 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb98006ae70 0x7fb98006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:26.460 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.455+0000 7fb9977fe640 1 -- 172.21.15.169:0/2834226765 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fb9880aa570 con 0x7fb9a00c65e0 2023-10-26T01:43:26.460 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.455+0000 7fb9977fe640 1 -- 172.21.15.169:0/2834226765 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=171}) v3 -- 0x7fb980072a00 con 0x7fb9a00c65e0 2023-10-26T01:43:26.460 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.456+0000 7fb99ed9d640 1 --2- 172.21.15.169:0/2834226765 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb98006ae70 0x7fb98006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:26.460 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.457+0000 7fb99ed9d640 1 --2- 172.21.15.169:0/2834226765 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb98006ae70 0x7fb98006d330 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fb9a00b17a0 tx=0x7fb990005ef0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:26.461 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.460+0000 7fb9977fe640 1 -- 172.21.15.169:0/2834226765 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb9a0155010 con 0x7fb9a00c65e0 2023-10-26T01:43:26.654 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.652+0000 7fb9a4989640 1 -- 172.21.15.169:0/2834226765 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb9a0155010 con 0x7fb9a00c65e0 2023-10-26T01:43:26.655 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.653+0000 7fb9977fe640 1 -- 172.21.15.169:0/2834226765 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v170) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb9a0155010 con 0x7fb9a00c65e0 2023-10-26T01:43:26.655 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 32 2023-10-26T01:43:26.658 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.655+0000 7fb9a4989640 1 -- 172.21.15.169:0/2834226765 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb98006ae70 msgr2=0x7fb98006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:26.658 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.655+0000 7fb9a4989640 1 --2- 172.21.15.169:0/2834226765 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb98006ae70 0x7fb98006d330 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fb9a00b17a0 tx=0x7fb990005ef0 comp rx=0 tx=0).stop 2023-10-26T01:43:26.658 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.655+0000 7fb9a4989640 1 -- 172.21.15.169:0/2834226765 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9a00c65e0 msgr2=0x7fb9a01f4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:26.659 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.655+0000 7fb9a4989640 1 --2- 172.21.15.169:0/2834226765 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9a00c65e0 0x7fb9a01f4d00 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7fb988000f60 tx=0x7fb988001650 comp rx=0 tx=0).stop 2023-10-26T01:43:26.659 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.656+0000 7fb9a4989640 1 -- 172.21.15.169:0/2834226765 shutdown_connections 2023-10-26T01:43:26.659 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.656+0000 7fb9a4989640 1 --2- 172.21.15.169:0/2834226765 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fb98006ae70 0x7fb98006d330 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:26.659 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.656+0000 7fb9a4989640 1 --2- 172.21.15.169:0/2834226765 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9a00c65e0 0x7fb9a01f4d00 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:26.659 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.656+0000 7fb9a4989640 1 --2- 172.21.15.169:0/2834226765 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb9a00c6a60 0x7fb9a01f5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:26.660 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.656+0000 7fb9a4989640 1 --2- 172.21.15.169:0/2834226765 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb98c0047c0 0x7fb9a006b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:26.660 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.656+0000 7fb9a4989640 1 -- 172.21.15.169:0/2834226765 >> 172.21.15.169:0/2834226765 conn(0x7fb9a00af110 msgr2=0x7fb9a00ac3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:26.660 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.656+0000 7fb9a4989640 1 -- 172.21.15.169:0/2834226765 shutdown_connections 2023-10-26T01:43:26.660 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.656+0000 7fb9a4989640 1 -- 172.21.15.169:0/2834226765 wait complete. 2023-10-26T01:43:26.675 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-26T01:43:26.899 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.897+0000 7ff7980b4640 1 -- 172.21.15.169:0/3177431157 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7901597c0 msgr2=0x7ff7900ac200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:26.899 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.897+0000 7ff7980b4640 1 --2- 172.21.15.169:0/3177431157 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7901597c0 0x7ff7900ac200 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7ff7900cd2c0 tx=0x7ff78002be40 comp rx=0 tx=0).stop 2023-10-26T01:43:26.899 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.897+0000 7ff7980b4640 1 -- 172.21.15.169:0/3177431157 shutdown_connections 2023-10-26T01:43:26.900 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.897+0000 7ff7980b4640 1 --2- 172.21.15.169:0/3177431157 >> v2:172.21.15.170:3300/0 conn(0x7ff7900b0c80 0x7ff7900cd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:26.900 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.897+0000 7ff7980b4640 1 --2- 172.21.15.169:0/3177431157 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7901597c0 0x7ff7900ac200 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:26.900 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.897+0000 7ff7980b4640 1 -- 172.21.15.169:0/3177431157 >> 172.21.15.169:0/3177431157 conn(0x7ff790069930 msgr2=0x7ff790069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:26.900 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.897+0000 7ff7980b4640 1 -- 172.21.15.169:0/3177431157 shutdown_connections 2023-10-26T01:43:26.901 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.897+0000 7ff7980b4640 1 -- 172.21.15.169:0/3177431157 wait complete. 2023-10-26T01:43:26.901 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.899+0000 7ff7980b4640 1 Processor -- start 2023-10-26T01:43:26.901 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.899+0000 7ff7980b4640 1 -- start start 2023-10-26T01:43:26.903 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.900+0000 7ff7980b4640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff7900b0c80 0x7ff790152520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:26.904 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.900+0000 7ff7980b4640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7901593e0 0x7ff79006c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:26.904 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.900+0000 7ff7980b4640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7901597c0 0x7ff7901e6520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:26.904 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.900+0000 7ff7980b4640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7901e3470 con 0x7ff7901593e0 2023-10-26T01:43:26.904 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.900+0000 7ff7980b4640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff790152a60 con 0x7ff7901597c0 2023-10-26T01:43:26.904 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.900+0000 7ff7980b4640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7901e2860 con 0x7ff7900b0c80 2023-10-26T01:43:26.905 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.900+0000 7ff79564f640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7901593e0 0x7ff79006c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:26.905 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.900+0000 7ff79564f640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7901593e0 0x7ff79006c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:49516/0 (socket says 172.21.15.169:49516) 2023-10-26T01:43:26.905 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.900+0000 7ff79564f640 1 -- 172.21.15.169:0/3396178791 learned_addr learned my addr 172.21.15.169:0/3396178791 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:43:26.905 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.901+0000 7ff795e50640 1 --2- 172.21.15.169:0/3396178791 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff7900b0c80 0x7ff790152520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:26.905 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.901+0000 7ff796651640 1 --2- 172.21.15.169:0/3396178791 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7901597c0 0x7ff7901e6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:26.906 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.901+0000 7ff79564f640 1 -- 172.21.15.169:0/3396178791 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff7900b0c80 msgr2=0x7ff790152520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:26.906 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.901+0000 7ff79564f640 1 --2- 172.21.15.169:0/3396178791 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff7900b0c80 0x7ff790152520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:26.906 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.901+0000 7ff79564f640 1 -- 172.21.15.169:0/3396178791 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7901597c0 msgr2=0x7ff7901e6520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:26.906 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.901+0000 7ff79564f640 1 --2- 172.21.15.169:0/3396178791 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7901597c0 0x7ff7901e6520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:26.907 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.901+0000 7ff79564f640 1 -- 172.21.15.169:0/3396178791 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff780009560 con 0x7ff7901593e0 2023-10-26T01:43:26.907 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.901+0000 7ff796651640 1 --2- 172.21.15.169:0/3396178791 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7901597c0 0x7ff7901e6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:43:26.907 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.901+0000 7ff79564f640 1 --2- 172.21.15.169:0/3396178791 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7901593e0 0x7ff79006c640 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7ff780009850 tx=0x7ff78002db70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:26.907 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.902+0000 7ff7867fc640 1 -- 172.21.15.169:0/3396178791 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff780003800 con 0x7ff7901593e0 2023-10-26T01:43:26.907 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.902+0000 7ff7867fc640 1 -- 172.21.15.169:0/3396178791 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff780003a50 con 0x7ff7901593e0 2023-10-26T01:43:26.908 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.902+0000 7ff7867fc640 1 -- 172.21.15.169:0/3396178791 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff780003e20 con 0x7ff7901593e0 2023-10-26T01:43:26.908 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.902+0000 7ff7980b4640 1 -- 172.21.15.169:0/3396178791 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7901e2b40 con 0x7ff7901593e0 2023-10-26T01:43:26.908 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.902+0000 7ff7980b4640 1 -- 172.21.15.169:0/3396178791 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7901e3100 con 0x7ff7901593e0 2023-10-26T01:43:26.909 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.903+0000 7ff7980b4640 1 -- 172.21.15.169:0/3396178791 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7901e32c0 con 0x7ff7901593e0 2023-10-26T01:43:26.909 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.904+0000 7ff7867fc640 1 -- 172.21.15.169:0/3396178791 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ff78003e450 con 0x7ff7901593e0 2023-10-26T01:43:26.909 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.905+0000 7ff7867fc640 1 --2- 172.21.15.169:0/3396178791 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff77406ae40 0x7ff77406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:26.910 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.905+0000 7ff7867fc640 1 -- 172.21.15.169:0/3396178791 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 5738+0+0 (secure 0 0 0) 0x7ff7800ac1f0 con 0x7ff7901593e0 2023-10-26T01:43:26.910 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.905+0000 7ff7867fc640 1 -- 172.21.15.169:0/3396178791 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=171}) v3 -- 0x7ff7740729d0 con 0x7ff7901593e0 2023-10-26T01:43:26.910 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.906+0000 7ff795e50640 1 --2- 172.21.15.169:0/3396178791 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff77406ae40 0x7ff77406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:26.910 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.907+0000 7ff795e50640 1 --2- 172.21.15.169:0/3396178791 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff77406ae40 0x7ff77406d300 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7ff78c000f60 tx=0x7ff78c009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:26.911 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:26.910+0000 7ff7867fc640 1 -- 172.21.15.169:0/3396178791 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff780042020 con 0x7ff7901593e0 2023-10-26T01:43:27.102 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:27.101+0000 7ff7980b4640 1 -- 172.21.15.169:0/3396178791 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff79004db10 con 0x7ff7901593e0 2023-10-26T01:43:27.103 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:27.101+0000 7ff7867fc640 1 -- 172.21.15.169:0/3396178791 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v170) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff7800771f0 con 0x7ff7901593e0 2023-10-26T01:43:27.103 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 64 2023-10-26T01:43:27.106 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:27.104+0000 7ff7980b4640 1 -- 172.21.15.169:0/3396178791 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff77406ae40 msgr2=0x7ff77406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:27.106 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:27.104+0000 7ff7980b4640 1 --2- 172.21.15.169:0/3396178791 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff77406ae40 0x7ff77406d300 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7ff78c000f60 tx=0x7ff78c009040 comp rx=0 tx=0).stop 2023-10-26T01:43:27.106 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:27.104+0000 7ff7980b4640 1 -- 172.21.15.169:0/3396178791 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7901593e0 msgr2=0x7ff79006c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:27.107 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:27.104+0000 7ff7980b4640 1 --2- 172.21.15.169:0/3396178791 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7901593e0 0x7ff79006c640 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7ff780009850 tx=0x7ff78002db70 comp rx=0 tx=0).stop 2023-10-26T01:43:27.107 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:27.104+0000 7ff7980b4640 1 -- 172.21.15.169:0/3396178791 shutdown_connections 2023-10-26T01:43:27.107 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:27.104+0000 7ff7980b4640 1 --2- 172.21.15.169:0/3396178791 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff77406ae40 0x7ff77406d300 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:27.107 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:27.104+0000 7ff7980b4640 1 --2- 172.21.15.169:0/3396178791 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7901593e0 0x7ff79006c640 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:27.107 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:27.104+0000 7ff7980b4640 1 --2- 172.21.15.169:0/3396178791 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7901597c0 0x7ff7901e6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:27.108 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:27.104+0000 7ff7980b4640 1 --2- 172.21.15.169:0/3396178791 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff7900b0c80 0x7ff790152520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:27.108 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:27.104+0000 7ff7980b4640 1 -- 172.21.15.169:0/3396178791 >> 172.21.15.169:0/3396178791 conn(0x7ff790069930 msgr2=0x7ff790153090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:27.108 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:27.104+0000 7ff7980b4640 1 -- 172.21.15.169:0/3396178791 shutdown_connections 2023-10-26T01:43:27.108 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:27.104+0000 7ff7980b4640 1 -- 172.21.15.169:0/3396178791 wait complete. 2023-10-26T01:43:27.125 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-26T01:43:27.346 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:27.344+0000 7fbcd0ff0640 1 -- 172.21.15.169:0/634994065 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbccc15b4a0 msgr2=0x7fbccc15b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:27.347 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:27.344+0000 7fbcd0ff0640 1 --2- 172.21.15.169:0/634994065 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbccc15b4a0 0x7fbccc15b910 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7fbcc4005fd0 tx=0x7fbcc402c060 comp rx=0 tx=0).stop 2023-10-26T01:43:27.347 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:27.345+0000 7fbcd0ff0640 1 -- 172.21.15.169:0/634994065 shutdown_connections 2023-10-26T01:43:27.347 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:27.345+0000 7fbcd0ff0640 1 --2- 172.21.15.169:0/634994065 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbccc15b4a0 0x7fbccc15b910 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:27.347 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:27.345+0000 7fbcd0ff0640 1 --2- 172.21.15.169:0/634994065 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbccc0cdef0 0x7fbccc155360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:27.348 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:27.345+0000 7fbcd0ff0640 1 -- 172.21.15.169:0/634994065 >> 172.21.15.169:0/634994065 conn(0x7fbccc069930 msgr2=0x7fbccc069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:27.348 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:27.345+0000 7fbcd0ff0640 1 -- 172.21.15.169:0/634994065 shutdown_connections 2023-10-26T01:43:27.348 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:27.345+0000 7fbcd0ff0640 1 -- 172.21.15.169:0/634994065 wait complete. 2023-10-26T01:43:27.349 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:27.346+0000 7fbcd0ff0640 1 Processor -- start 2023-10-26T01:43:27.350 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:27.346+0000 7fbcd0ff0640 1 -- start start 2023-10-26T01:43:27.350 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:27.347+0000 7fbcd0ff0640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbccc06ce00 0x7fbccc14a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:27.350 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:27.347+0000 7fbcd0ff0640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbccc0cdef0 0x7fbccc06b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:27.350 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:27.347+0000 7fbcd0ff0640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbccc15b4a0 0x7fbccc160820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:27.350 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:27.347+0000 7fbcd0ff0640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbccc154610 con 0x7fbccc15b4a0 2023-10-26T01:43:27.351 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:27.347+0000 7fbcd0ff0640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbccc14adf0 con 0x7fbccc0cdef0 2023-10-26T01:43:27.351 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:27.347+0000 7fbcd0ff0640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbccc06b9e0 con 0x7fbccc06ce00 2023-10-26T01:43:27.351 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:27.348+0000 7fbccad9d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbccc15b4a0 0x7fbccc160820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:27.351 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:27.348+0000 7fbccad9d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbccc15b4a0 0x7fbccc160820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:49520/0 (socket says 172.21.15.169:49520) 2023-10-26T01:43:27.351 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:27.348+0000 7fbccad9d640 1 -- 172.21.15.169:0/1213431838 learned_addr learned my addr 172.21.15.169:0/1213431838 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:43:27.352 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:27.348+0000 7fbcca59c640 1 --2- 172.21.15.169:0/1213431838 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbccc06ce00 0x7fbccc14a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:27.352 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:27.348+0000 7fbcc9d9b640 1 --2- 172.21.15.169:0/1213431838 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbccc0cdef0 0x7fbccc06b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:27.353 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:27.348+0000 7fbccad9d640 1 -- 172.21.15.169:0/1213431838 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbccc06ce00 msgr2=0x7fbccc14a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:27.353 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:27.348+0000 7fbccad9d640 1 --2- 172.21.15.169:0/1213431838 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbccc06ce00 0x7fbccc14a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:27.353 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:27.348+0000 7fbccad9d640 1 -- 172.21.15.169:0/1213431838 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbccc0cdef0 msgr2=0x7fbccc06b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:27.354 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:27.348+0000 7fbccad9d640 1 --2- 172.21.15.169:0/1213431838 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbccc0cdef0 0x7fbccc06b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:27.354 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:27.348+0000 7fbccad9d640 1 -- 172.21.15.169:0/1213431838 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcc4009840 con 0x7fbccc15b4a0 2023-10-26T01:43:27.354 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:27.349+0000 7fbccad9d640 1 --2- 172.21.15.169:0/1213431838 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbccc15b4a0 0x7fbccc160820 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7fbcc4009cd0 tx=0x7fbcc40042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:27.354 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:27.349+0000 7fbcbaffd640 1 -- 172.21.15.169:0/1213431838 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc403d430 con 0x7fbccc15b4a0 2023-10-26T01:43:27.354 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:27.349+0000 7fbcbaffd640 1 -- 172.21.15.169:0/1213431838 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fbcc403d680 con 0x7fbccc15b4a0 2023-10-26T01:43:27.355 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:27.349+0000 7fbcbaffd640 1 -- 172.21.15.169:0/1213431838 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc403da50 con 0x7fbccc15b4a0 2023-10-26T01:43:27.355 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:27.349+0000 7fbcd0ff0640 1 -- 172.21.15.169:0/1213431838 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbccc158cd0 con 0x7fbccc15b4a0 2023-10-26T01:43:27.355 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:27.349+0000 7fbcd0ff0640 1 -- 172.21.15.169:0/1213431838 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbccc153a00 con 0x7fbccc15b4a0 2023-10-26T01:43:27.355 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:27.350+0000 7fbcd0ff0640 1 -- 172.21.15.169:0/1213431838 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbccc1547c0 con 0x7fbccc15b4a0 2023-10-26T01:43:27.355 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:27.351+0000 7fbcbaffd640 1 -- 172.21.15.169:0/1213431838 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fbcc403dbb0 con 0x7fbccc15b4a0 2023-10-26T01:43:27.356 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:27.352+0000 7fbcbaffd640 1 --2- 172.21.15.169:0/1213431838 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbcac06ae40 0x7fbcac06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:27.356 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:27.353+0000 7fbcbaffd640 1 -- 172.21.15.169:0/1213431838 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fbcc40ac6f0 con 0x7fbccc15b4a0 2023-10-26T01:43:27.357 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:27.353+0000 7fbcbaffd640 1 -- 172.21.15.169:0/1213431838 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=171}) v3 -- 0x7fbcac0729d0 con 0x7fbccc15b4a0 2023-10-26T01:43:27.357 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:27.353+0000 7fbcca59c640 1 --2- 172.21.15.169:0/1213431838 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbcac06ae40 0x7fbcac06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:27.357 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:27.354+0000 7fbcca59c640 1 --2- 172.21.15.169:0/1213431838 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbcac06ae40 0x7fbcac06d300 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc0046b0 tx=0x7fbcbc009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:27.358 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:27.357+0000 7fbcbaffd640 1 -- 172.21.15.169:0/1213431838 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbcc404b060 con 0x7fbccc15b4a0 2023-10-26T01:43:27.562 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:27.559+0000 7fbcd0ff0640 1 -- 172.21.15.169:0/1213431838 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fbccc04db10 con 0x7fbccc15b4a0 2023-10-26T01:43:27.563 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:27.560+0000 7fbcbaffd640 1 -- 172.21.15.169:0/1213431838 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v170) v1 ==== 80+0+0 (secure 0 0 0) 0x7fbcc4041020 con 0x7fbccc15b4a0 2023-10-26T01:43:27.563 INFO:teuthology.orchestra.run.smithi169.stderr:listed 0 entries 2023-10-26T01:43:27.566 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:27.563+0000 7fbcd0ff0640 1 -- 172.21.15.169:0/1213431838 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbcac06ae40 msgr2=0x7fbcac06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:27.566 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:27.563+0000 7fbcd0ff0640 1 --2- 172.21.15.169:0/1213431838 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbcac06ae40 0x7fbcac06d300 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc0046b0 tx=0x7fbcbc009040 comp rx=0 tx=0).stop 2023-10-26T01:43:27.566 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:27.563+0000 7fbcd0ff0640 1 -- 172.21.15.169:0/1213431838 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbccc15b4a0 msgr2=0x7fbccc160820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:27.567 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:27.563+0000 7fbcd0ff0640 1 --2- 172.21.15.169:0/1213431838 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbccc15b4a0 0x7fbccc160820 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7fbcc4009cd0 tx=0x7fbcc40042e0 comp rx=0 tx=0).stop 2023-10-26T01:43:27.567 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:27.563+0000 7fbcd0ff0640 1 -- 172.21.15.169:0/1213431838 shutdown_connections 2023-10-26T01:43:27.567 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:27.563+0000 7fbcd0ff0640 1 --2- 172.21.15.169:0/1213431838 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbcac06ae40 0x7fbcac06d300 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:27.567 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:27.563+0000 7fbcd0ff0640 1 --2- 172.21.15.169:0/1213431838 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbccc15b4a0 0x7fbccc160820 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:27.568 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:27.563+0000 7fbcd0ff0640 1 --2- 172.21.15.169:0/1213431838 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbccc0cdef0 0x7fbccc06b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:27.568 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:27.563+0000 7fbcd0ff0640 1 --2- 172.21.15.169:0/1213431838 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbccc06ce00 0x7fbccc14a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:27.568 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:27.563+0000 7fbcd0ff0640 1 -- 172.21.15.169:0/1213431838 >> 172.21.15.169:0/1213431838 conn(0x7fbccc069930 msgr2=0x7fbccc06ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:27.568 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:27.563+0000 7fbcd0ff0640 1 -- 172.21.15.169:0/1213431838 shutdown_connections 2023-10-26T01:43:27.568 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:27.563+0000 7fbcd0ff0640 1 -- 172.21.15.169:0/1213431838 wait complete. 2023-10-26T01:43:27.584 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-26T01:43:27.584 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:43:27.584 DEBUG:teuthology.orchestra.run.smithi169:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-26T01:43:27.686 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi169.front.sepia.ceph.com 2023-10-26T01:43:27.686 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:43:27.687 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-26T01:43:27.740 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-26T01:43:27.740 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:43:27.803 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-26T01:43:27.803 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:43:27.861 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:43:28.090 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:28.087+0000 7f4f2f290640 1 -- 172.21.15.169:0/3440803117 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4f28158850 msgr2=0x7f4f280cd720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:28.090 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:28.087+0000 7f4f2f290640 1 --2- 172.21.15.169:0/3440803117 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4f28158850 0x7f4f280cd720 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f4f24005fd0 tx=0x7f4f2402c240 comp rx=0 tx=0).stop 2023-10-26T01:43:28.090 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:28.087+0000 7f4f2f290640 1 -- 172.21.15.169:0/3440803117 shutdown_connections 2023-10-26T01:43:28.090 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:28.087+0000 7f4f2f290640 1 --2- 172.21.15.169:0/3440803117 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4f28158850 0x7f4f280cd720 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:28.091 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:28.087+0000 7f4f2f290640 1 --2- 172.21.15.169:0/3440803117 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4f2814e790 0x7f4f280aab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:28.091 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:28.087+0000 7f4f2f290640 1 -- 172.21.15.169:0/3440803117 >> 172.21.15.169:0/3440803117 conn(0x7f4f28069930 msgr2=0x7f4f28069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:28.091 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:28.087+0000 7f4f2f290640 1 -- 172.21.15.169:0/3440803117 shutdown_connections 2023-10-26T01:43:28.091 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:28.087+0000 7f4f2f290640 1 -- 172.21.15.169:0/3440803117 wait complete. 2023-10-26T01:43:28.092 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:28.089+0000 7f4f2f290640 1 Processor -- start 2023-10-26T01:43:28.092 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:28.089+0000 7f4f2f290640 1 -- start start 2023-10-26T01:43:28.093 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:28.090+0000 7f4f2f290640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4f280cdc60 0x7f4f281ef190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:28.093 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:28.090+0000 7f4f2f290640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4f2814e790 0x7f4f280b0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:28.093 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:28.090+0000 7f4f2f290640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4f28158850 0x7f4f280b1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:28.093 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:28.090+0000 7f4f2f290640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f28142b70 con 0x7f4f280cdc60 2023-10-26T01:43:28.094 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:28.090+0000 7f4f2f290640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f281ef6d0 con 0x7f4f2814e790 2023-10-26T01:43:28.094 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:28.090+0000 7f4f2f290640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f28141f60 con 0x7f4f28158850 2023-10-26T01:43:28.095 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:28.091+0000 7f4f2d02c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4f280cdc60 0x7f4f281ef190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:28.096 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:28.091+0000 7f4f2d02c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4f280cdc60 0x7f4f281ef190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:49528/0 (socket says 172.21.15.169:49528) 2023-10-26T01:43:28.096 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:28.091+0000 7f4f2d02c640 1 -- 172.21.15.169:0/2736766885 learned_addr learned my addr 172.21.15.169:0/2736766885 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:43:28.096 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:28.091+0000 7f4f2c82b640 1 --2- 172.21.15.169:0/2736766885 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4f2814e790 0x7f4f280b0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:28.096 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:28.091+0000 7f4f2d82d640 1 --2- 172.21.15.169:0/2736766885 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4f28158850 0x7f4f280b1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:28.096 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:28.091+0000 7f4f2d02c640 1 -- 172.21.15.169:0/2736766885 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4f28158850 msgr2=0x7f4f280b1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:28.097 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:28.091+0000 7f4f2d02c640 1 --2- 172.21.15.169:0/2736766885 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4f28158850 0x7f4f280b1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:28.097 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:28.091+0000 7f4f2d02c640 1 -- 172.21.15.169:0/2736766885 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4f2814e790 msgr2=0x7f4f280b0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:28.097 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:28.091+0000 7f4f2d02c640 1 --2- 172.21.15.169:0/2736766885 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4f2814e790 0x7f4f280b0ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:28.097 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:28.091+0000 7f4f2d02c640 1 -- 172.21.15.169:0/2736766885 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f24009900 con 0x7f4f280cdc60 2023-10-26T01:43:28.097 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:28.091+0000 7f4f2d82d640 1 --2- 172.21.15.169:0/2736766885 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4f28158850 0x7f4f280b1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:43:28.098 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:28.092+0000 7f4f2d02c640 1 --2- 172.21.15.169:0/2736766885 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4f280cdc60 0x7f4f281ef190 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c00e9f0 tx=0x7f4f1c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:28.098 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:28.092+0000 7f4f19ffb640 1 -- 172.21.15.169:0/2736766885 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f1c009700 con 0x7f4f280cdc60 2023-10-26T01:43:28.098 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:28.092+0000 7f4f2f290640 1 -- 172.21.15.169:0/2736766885 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f28142240 con 0x7f4f280cdc60 2023-10-26T01:43:28.098 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:28.092+0000 7f4f19ffb640 1 -- 172.21.15.169:0/2736766885 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4f1c00ce70 con 0x7f4f280cdc60 2023-10-26T01:43:28.098 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:28.092+0000 7f4f19ffb640 1 -- 172.21.15.169:0/2736766885 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f1c010670 con 0x7f4f280cdc60 2023-10-26T01:43:28.099 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:28.092+0000 7f4f2f290640 1 -- 172.21.15.169:0/2736766885 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f28142800 con 0x7f4f280cdc60 2023-10-26T01:43:28.100 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:28.093+0000 7f4f2f290640 1 -- 172.21.15.169:0/2736766885 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f2804db10 con 0x7f4f280cdc60 2023-10-26T01:43:28.100 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:28.095+0000 7f4f19ffb640 1 -- 172.21.15.169:0/2736766885 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f4f1c0107d0 con 0x7f4f280cdc60 2023-10-26T01:43:28.100 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:28.095+0000 7f4f19ffb640 1 --2- 172.21.15.169:0/2736766885 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f4f0c06ae40 0x7f4f0c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:28.100 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:28.096+0000 7f4f19ffb640 1 -- 172.21.15.169:0/2736766885 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f4f1c014070 con 0x7f4f280cdc60 2023-10-26T01:43:28.100 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:28.096+0000 7f4f19ffb640 1 -- 172.21.15.169:0/2736766885 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=171}) v3 -- 0x7f4f0c0729d0 con 0x7f4f280cdc60 2023-10-26T01:43:28.101 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:28.096+0000 7f4f2c82b640 1 --2- 172.21.15.169:0/2736766885 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f4f0c06ae40 0x7f4f0c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:28.101 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:28.097+0000 7f4f2c82b640 1 --2- 172.21.15.169:0/2736766885 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f4f0c06ae40 0x7f4f0c06d300 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f4f24036000 tx=0x7f4f24004ab0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:28.104 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:28.101+0000 7f4f19ffb640 1 -- 172.21.15.169:0/2736766885 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4f1c051ca0 con 0x7f4f280cdc60 2023-10-26T01:43:28.299 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:28.296+0000 7f4f2f290640 1 -- 172.21.15.169:0/2736766885 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4f280aca70 con 0x7f4f280cdc60 2023-10-26T01:43:28.301 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:28.299+0000 7f4f19ffb640 1 -- 172.21.15.169:0/2736766885 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v170) v1 ==== 74+0+16704 (secure 0 0 0) 0x7f4f1c058e40 con 0x7f4f280cdc60 2023-10-26T01:43:28.302 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:43:28.302 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":170,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:42:58.161990+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2023-10-26T01:42:07.656531+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"165","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2023-10-26T01:42:08.866635+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"170","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.39","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:43:28.305 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:28.301+0000 7f4f2f290640 1 -- 172.21.15.169:0/2736766885 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f4f0c06ae40 msgr2=0x7f4f0c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:28.305 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:28.301+0000 7f4f2f290640 1 --2- 172.21.15.169:0/2736766885 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f4f0c06ae40 0x7f4f0c06d300 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f4f24036000 tx=0x7f4f24004ab0 comp rx=0 tx=0).stop 2023-10-26T01:43:28.306 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:28.301+0000 7f4f2f290640 1 -- 172.21.15.169:0/2736766885 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4f280cdc60 msgr2=0x7f4f281ef190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:28.306 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:28.301+0000 7f4f2f290640 1 --2- 172.21.15.169:0/2736766885 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4f280cdc60 0x7f4f281ef190 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c00e9f0 tx=0x7f4f1c00eeb0 comp rx=0 tx=0).stop 2023-10-26T01:43:28.306 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:28.302+0000 7f4f2f290640 1 -- 172.21.15.169:0/2736766885 shutdown_connections 2023-10-26T01:43:28.307 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:28.302+0000 7f4f2f290640 1 --2- 172.21.15.169:0/2736766885 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f4f0c06ae40 0x7f4f0c06d300 secure :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f4f24036000 tx=0x7f4f24004ab0 comp rx=0 tx=0).stop 2023-10-26T01:43:28.307 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:28.302+0000 7f4f2f290640 1 --2- 172.21.15.169:0/2736766885 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4f28158850 0x7f4f280b1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:28.307 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:28.302+0000 7f4f2f290640 1 --2- 172.21.15.169:0/2736766885 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4f2814e790 0x7f4f280b0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:28.308 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:28.302+0000 7f4f2f290640 1 --2- 172.21.15.169:0/2736766885 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4f280cdc60 0x7f4f281ef190 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:28.308 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:28.302+0000 7f4f2f290640 1 -- 172.21.15.169:0/2736766885 >> 172.21.15.169:0/2736766885 conn(0x7f4f28069930 msgr2=0x7f4f28069d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:28.308 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:28.302+0000 7f4f2f290640 1 -- 172.21.15.169:0/2736766885 shutdown_connections 2023-10-26T01:43:28.308 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:28.302+0000 7f4f2f290640 1 -- 172.21.15.169:0/2736766885 wait complete. 2023-10-26T01:43:28.325 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T01:30:32.200808+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '109', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 20, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-26T01:42:07.656531+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '165', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 21, 'pool_name': 'cephfs_data', 'create_time': '2023-10-26T01:42:08.866635+0000', 'flags': 1027, 'flags_names': 'hashpspool,full,full_quota', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '170', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T01:43:28.326 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T01:43:28.549 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:28.546+0000 7f37f8697640 1 -- 172.21.15.169:0/206734918 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37f006b7d0 msgr2=0x7f37f00b1ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:28.549 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:28.546+0000 7f37f8697640 1 --2- 172.21.15.169:0/206734918 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37f006b7d0 0x7f37f00b1ab0 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f37f00718b0 tx=0x7f37e002bdb0 comp rx=0 tx=0).stop 2023-10-26T01:43:28.549 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:28.546+0000 7f37f8697640 1 -- 172.21.15.169:0/206734918 shutdown_connections 2023-10-26T01:43:28.550 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:28.546+0000 7f37f8697640 1 --2- 172.21.15.169:0/206734918 >> v2:172.21.15.170:3300/0 conn(0x7f37f00b44f0 0x7f37f0165370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:28.550 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:28.546+0000 7f37f8697640 1 --2- 172.21.15.169:0/206734918 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37f006b7d0 0x7f37f00b1ab0 unknown :-1 s=CLOSED pgs=1743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:28.550 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:28.546+0000 7f37f8697640 1 -- 172.21.15.169:0/206734918 >> 172.21.15.169:0/206734918 conn(0x7f37f00b4ab0 msgr2=0x7f37f00b1070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:28.550 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:28.546+0000 7f37f8697640 1 -- 172.21.15.169:0/206734918 shutdown_connections 2023-10-26T01:43:28.550 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:28.546+0000 7f37f8697640 1 -- 172.21.15.169:0/206734918 wait complete. 2023-10-26T01:43:28.551 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:28.549+0000 7f37f8697640 1 Processor -- start 2023-10-26T01:43:28.552 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:28.549+0000 7f37f8697640 1 -- start start 2023-10-26T01:43:28.554 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:28.550+0000 7f37f8697640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37f006b7d0 0x7f37f0154060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:28.554 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:28.550+0000 7f37f8697640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f37f00b44f0 0x7f37f0158f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:28.554 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:28.550+0000 7f37f8697640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f37f015e1a0 0x7f37f0154d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:28.555 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:28.551+0000 7f37f8697640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37f009a750 con 0x7f37f006b7d0 2023-10-26T01:43:28.555 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:28.551+0000 7f37f8697640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37f01545a0 con 0x7f37f00b44f0 2023-10-26T01:43:28.555 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:28.551+0000 7f37f8697640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37f01598a0 con 0x7f37f015e1a0 2023-10-26T01:43:28.555 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:28.551+0000 7f37f6433640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37f006b7d0 0x7f37f0154060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:28.556 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:28.551+0000 7f37f6433640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37f006b7d0 0x7f37f0154060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:49556/0 (socket says 172.21.15.169:49556) 2023-10-26T01:43:28.556 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:28.551+0000 7f37f6433640 1 -- 172.21.15.169:0/1743367956 learned_addr learned my addr 172.21.15.169:0/1743367956 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:43:28.556 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:28.551+0000 7f37f6433640 1 -- 172.21.15.169:0/1743367956 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f37f015e1a0 msgr2=0x7f37f0154d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:28.556 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:28.551+0000 7f37f6433640 1 --2- 172.21.15.169:0/1743367956 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f37f015e1a0 0x7f37f0154d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:28.556 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:28.551+0000 7f37f6433640 1 -- 172.21.15.169:0/1743367956 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f37f00b44f0 msgr2=0x7f37f0158f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:28.557 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:28.551+0000 7f37f6433640 1 --2- 172.21.15.169:0/1743367956 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f37f00b44f0 0x7f37f0158f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:28.557 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:28.551+0000 7f37f6433640 1 -- 172.21.15.169:0/1743367956 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37e0009560 con 0x7f37f006b7d0 2023-10-26T01:43:28.558 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:28.552+0000 7f37f6433640 1 --2- 172.21.15.169:0/1743367956 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37f006b7d0 0x7f37f0154060 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f37ec00b3e0 tx=0x7f37ec00b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:28.558 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:28.552+0000 7f37e6ffd640 1 -- 172.21.15.169:0/1743367956 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ec011830 con 0x7f37f006b7d0 2023-10-26T01:43:28.558 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:28.552+0000 7f37e6ffd640 1 -- 172.21.15.169:0/1743367956 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f37ec011aa0 con 0x7f37f006b7d0 2023-10-26T01:43:28.559 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:28.552+0000 7f37e6ffd640 1 -- 172.21.15.169:0/1743367956 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ec010380 con 0x7f37f006b7d0 2023-10-26T01:43:28.559 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:28.552+0000 7f37f8697640 1 -- 172.21.15.169:0/1743367956 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37f0159b80 con 0x7f37f006b7d0 2023-10-26T01:43:28.559 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:28.552+0000 7f37f8697640 1 -- 172.21.15.169:0/1743367956 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37f009a370 con 0x7f37f006b7d0 2023-10-26T01:43:28.559 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:28.554+0000 7f37f8697640 1 -- 172.21.15.169:0/1743367956 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37f004da70 con 0x7f37f006b7d0 2023-10-26T01:43:28.560 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:28.555+0000 7f37e6ffd640 1 -- 172.21.15.169:0/1743367956 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f37ec0104e0 con 0x7f37f006b7d0 2023-10-26T01:43:28.561 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:28.556+0000 7f37e6ffd640 1 --2- 172.21.15.169:0/1743367956 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f37d406ae40 0x7f37d406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:28.561 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:28.556+0000 7f37e6ffd640 1 -- 172.21.15.169:0/1743367956 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f37ec089a20 con 0x7f37f006b7d0 2023-10-26T01:43:28.562 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:28.556+0000 7f37e6ffd640 1 -- 172.21.15.169:0/1743367956 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=171}) v3 -- 0x7f37d40729d0 con 0x7f37f006b7d0 2023-10-26T01:43:28.562 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:28.557+0000 7f37f5c32640 1 --2- 172.21.15.169:0/1743367956 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f37d406ae40 0x7f37d406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:28.562 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:28.558+0000 7f37f5c32640 1 --2- 172.21.15.169:0/1743367956 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f37d406ae40 0x7f37d406d300 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f37e0035000 tx=0x7f37e00046c0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:28.564 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:28.561+0000 7f37e6ffd640 1 -- 172.21.15.169:0/1743367956 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f37ec015060 con 0x7f37f006b7d0 2023-10-26T01:43:28.754 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:28.751+0000 7f37f8697640 1 -- 172.21.15.169:0/1743367956 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f37f0165330 con 0x7f37f006b7d0 2023-10-26T01:43:28.755 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:28.752+0000 7f37e6ffd640 1 -- 172.21.15.169:0/1743367956 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v170) v1 ==== 93+0+10 (secure 0 0 0) 0x7f37ec054b40 con 0x7f37f006b7d0 2023-10-26T01:43:28.755 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 1 2023-10-26T01:43:28.758 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:28.755+0000 7f37f8697640 1 -- 172.21.15.169:0/1743367956 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f37d406ae40 msgr2=0x7f37d406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:28.758 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:28.755+0000 7f37f8697640 1 --2- 172.21.15.169:0/1743367956 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f37d406ae40 0x7f37d406d300 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f37e0035000 tx=0x7f37e00046c0 comp rx=0 tx=0).stop 2023-10-26T01:43:28.759 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:28.755+0000 7f37f8697640 1 -- 172.21.15.169:0/1743367956 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37f006b7d0 msgr2=0x7f37f0154060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:28.759 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:28.755+0000 7f37f8697640 1 --2- 172.21.15.169:0/1743367956 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37f006b7d0 0x7f37f0154060 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f37ec00b3e0 tx=0x7f37ec00b8a0 comp rx=0 tx=0).stop 2023-10-26T01:43:28.759 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:28.755+0000 7f37f8697640 1 -- 172.21.15.169:0/1743367956 shutdown_connections 2023-10-26T01:43:28.759 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:28.755+0000 7f37f8697640 1 --2- 172.21.15.169:0/1743367956 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f37d406ae40 0x7f37d406d300 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:28.759 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:28.755+0000 7f37f8697640 1 --2- 172.21.15.169:0/1743367956 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f37f015e1a0 0x7f37f0154d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:28.760 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:28.755+0000 7f37f8697640 1 --2- 172.21.15.169:0/1743367956 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f37f00b44f0 0x7f37f0158f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:28.760 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:28.755+0000 7f37f8697640 1 --2- 172.21.15.169:0/1743367956 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37f006b7d0 0x7f37f0154060 unknown :-1 s=CLOSED pgs=1744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:28.760 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:28.755+0000 7f37f8697640 1 -- 172.21.15.169:0/1743367956 >> 172.21.15.169:0/1743367956 conn(0x7f37f00b4ab0 msgr2=0x7f37f00aea70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:28.760 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:28.756+0000 7f37f8697640 1 -- 172.21.15.169:0/1743367956 shutdown_connections 2023-10-26T01:43:28.760 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:28.756+0000 7f37f8697640 1 -- 172.21.15.169:0/1743367956 wait complete. 2023-10-26T01:43:28.776 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-26T01:43:29.006 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.003+0000 7fc1b7fff640 1 -- 172.21.15.169:0/1282748018 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1b815f450 msgr2=0x7fc1b815f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:29.006 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.003+0000 7fc1b7fff640 1 --2- 172.21.15.169:0/1282748018 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1b815f450 0x7fc1b815f8e0 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7fc1b0005fd0 tx=0x7fc1b002c2e0 comp rx=0 tx=0).stop 2023-10-26T01:43:29.007 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.003+0000 7fc1b7fff640 1 -- 172.21.15.169:0/1282748018 shutdown_connections 2023-10-26T01:43:29.007 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.003+0000 7fc1b7fff640 1 --2- 172.21.15.169:0/1282748018 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1b815f450 0x7fc1b815f8e0 unknown :-1 s=CLOSED pgs=1746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:29.007 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.003+0000 7fc1b7fff640 1 -- 172.21.15.169:0/1282748018 >> 172.21.15.169:0/1282748018 conn(0x7fc1b80af110 msgr2=0x7fc1b80af520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:29.007 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.003+0000 7fc1b7fff640 1 -- 172.21.15.169:0/1282748018 shutdown_connections 2023-10-26T01:43:29.007 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.003+0000 7fc1b7fff640 1 -- 172.21.15.169:0/1282748018 wait complete. 2023-10-26T01:43:29.008 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.005+0000 7fc1b7fff640 1 Processor -- start 2023-10-26T01:43:29.008 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.005+0000 7fc1b7fff640 1 -- start start 2023-10-26T01:43:29.009 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.007+0000 7fc1b7fff640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc1b806aef0 0x7fc1b8152780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:29.010 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.007+0000 7fc1b7fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1b80ac130 0x7fc1b8157bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:29.010 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.007+0000 7fc1b7fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1b815f450 0x7fc1b806fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:29.010 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.007+0000 7fc1b7fff640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b8152cc0 con 0x7fc1b815f450 2023-10-26T01:43:29.010 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.007+0000 7fc1b7fff640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b81f1cf0 con 0x7fc1b80ac130 2023-10-26T01:43:29.011 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.007+0000 7fc1b7fff640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b8158480 con 0x7fc1b806aef0 2023-10-26T01:43:29.011 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.007+0000 7fc1b77fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1b815f450 0x7fc1b806fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:29.011 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.007+0000 7fc1b77fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1b815f450 0x7fc1b806fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:49572/0 (socket says 172.21.15.169:49572) 2023-10-26T01:43:29.011 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.007+0000 7fc1b77fe640 1 -- 172.21.15.169:0/3608470387 learned_addr learned my addr 172.21.15.169:0/3608470387 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:43:29.012 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.007+0000 7fc1b6ffd640 1 --2- 172.21.15.169:0/3608470387 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc1b806aef0 0x7fc1b8152780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:29.013 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.007+0000 7fc1affff640 1 --2- 172.21.15.169:0/3608470387 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1b80ac130 0x7fc1b8157bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:29.013 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.007+0000 7fc1b77fe640 1 -- 172.21.15.169:0/3608470387 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc1b806aef0 msgr2=0x7fc1b8152780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:29.013 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.007+0000 7fc1b77fe640 1 --2- 172.21.15.169:0/3608470387 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc1b806aef0 0x7fc1b8152780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:29.013 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.007+0000 7fc1b77fe640 1 -- 172.21.15.169:0/3608470387 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1b80ac130 msgr2=0x7fc1b8157bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:29.013 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.007+0000 7fc1b77fe640 1 --2- 172.21.15.169:0/3608470387 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1b80ac130 0x7fc1b8157bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:29.014 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.007+0000 7fc1b77fe640 1 -- 172.21.15.169:0/3608470387 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1b0009900 con 0x7fc1b815f450 2023-10-26T01:43:29.014 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.007+0000 7fc1b6ffd640 1 --2- 172.21.15.169:0/3608470387 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc1b806aef0 0x7fc1b8152780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:43:29.014 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.007+0000 7fc1affff640 1 --2- 172.21.15.169:0/3608470387 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1b80ac130 0x7fc1b8157bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:43:29.014 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.008+0000 7fc1b77fe640 1 --2- 172.21.15.169:0/3608470387 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1b815f450 0x7fc1b806fa10 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7fc1b0037040 tx=0x7fc1b0035ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:29.014 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.008+0000 7fc1af7fe640 1 -- 172.21.15.169:0/3608470387 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b0004240 con 0x7fc1b815f450 2023-10-26T01:43:29.015 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.008+0000 7fc1af7fe640 1 -- 172.21.15.169:0/3608470387 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc1b0004490 con 0x7fc1b815f450 2023-10-26T01:43:29.015 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.008+0000 7fc1af7fe640 1 -- 172.21.15.169:0/3608470387 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b0004840 con 0x7fc1b815f450 2023-10-26T01:43:29.015 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.008+0000 7fc1b7fff640 1 -- 172.21.15.169:0/3608470387 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1b8158760 con 0x7fc1b815f450 2023-10-26T01:43:29.015 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.008+0000 7fc1b7fff640 1 -- 172.21.15.169:0/3608470387 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1b8158d20 con 0x7fc1b815f450 2023-10-26T01:43:29.015 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.010+0000 7fc1b7fff640 1 -- 172.21.15.169:0/3608470387 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1b8073c10 con 0x7fc1b815f450 2023-10-26T01:43:29.016 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.010+0000 7fc1af7fe640 1 -- 172.21.15.169:0/3608470387 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fc1b003b040 con 0x7fc1b815f450 2023-10-26T01:43:29.016 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.011+0000 7fc1af7fe640 1 --2- 172.21.15.169:0/3608470387 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc19806ae40 0x7fc19806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:29.017 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.011+0000 7fc1af7fe640 1 -- 172.21.15.169:0/3608470387 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fc1b00ac400 con 0x7fc1b815f450 2023-10-26T01:43:29.017 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.012+0000 7fc1af7fe640 1 -- 172.21.15.169:0/3608470387 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=171}) v3 -- 0x7fc1980729d0 con 0x7fc1b815f450 2023-10-26T01:43:29.017 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.012+0000 7fc1b6ffd640 1 --2- 172.21.15.169:0/3608470387 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc19806ae40 0x7fc19806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:29.017 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.013+0000 7fc1b6ffd640 1 --2- 172.21.15.169:0/3608470387 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc19806ae40 0x7fc19806d300 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fc1a8003eb0 tx=0x7fc1a8009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:29.020 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.017+0000 7fc1af7fe640 1 -- 172.21.15.169:0/3608470387 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc1b0077490 con 0x7fc1b815f450 2023-10-26T01:43:29.212 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.209+0000 7fc1b7fff640 1 -- 172.21.15.169:0/3608470387 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc1b8160030 con 0x7fc1b815f450 2023-10-26T01:43:29.213 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.210+0000 7fc1af7fe640 1 -- 172.21.15.169:0/3608470387 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v170) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc1b0043020 con 0x7fc1b815f450 2023-10-26T01:43:29.213 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 32 2023-10-26T01:43:29.216 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.213+0000 7fc1b7fff640 1 -- 172.21.15.169:0/3608470387 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc19806ae40 msgr2=0x7fc19806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:29.216 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.213+0000 7fc1b7fff640 1 --2- 172.21.15.169:0/3608470387 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc19806ae40 0x7fc19806d300 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fc1a8003eb0 tx=0x7fc1a8009040 comp rx=0 tx=0).stop 2023-10-26T01:43:29.216 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.213+0000 7fc1b7fff640 1 -- 172.21.15.169:0/3608470387 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1b815f450 msgr2=0x7fc1b806fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:29.216 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.213+0000 7fc1b7fff640 1 --2- 172.21.15.169:0/3608470387 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1b815f450 0x7fc1b806fa10 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7fc1b0037040 tx=0x7fc1b0035ad0 comp rx=0 tx=0).stop 2023-10-26T01:43:29.217 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.213+0000 7fc1b7fff640 1 -- 172.21.15.169:0/3608470387 shutdown_connections 2023-10-26T01:43:29.217 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.213+0000 7fc1b7fff640 1 --2- 172.21.15.169:0/3608470387 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc19806ae40 0x7fc19806d300 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:29.217 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.213+0000 7fc1b7fff640 1 --2- 172.21.15.169:0/3608470387 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1b815f450 0x7fc1b806fa10 unknown :-1 s=CLOSED pgs=1747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:29.217 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.213+0000 7fc1b7fff640 1 --2- 172.21.15.169:0/3608470387 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1b80ac130 0x7fc1b8157bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:29.218 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.213+0000 7fc1b7fff640 1 --2- 172.21.15.169:0/3608470387 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc1b806aef0 0x7fc1b8152780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:29.218 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.213+0000 7fc1b7fff640 1 -- 172.21.15.169:0/3608470387 >> 172.21.15.169:0/3608470387 conn(0x7fc1b80af110 msgr2=0x7fc1b80acc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:29.218 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.213+0000 7fc1b7fff640 1 -- 172.21.15.169:0/3608470387 shutdown_connections 2023-10-26T01:43:29.218 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.213+0000 7fc1b7fff640 1 -- 172.21.15.169:0/3608470387 wait complete. 2023-10-26T01:43:29.234 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-26T01:43:29.452 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.449+0000 7f2996b6c640 1 -- 172.21.15.169:0/3407891600 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f299006bd30 msgr2=0x7f29900aac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:29.452 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.449+0000 7f2996b6c640 1 --2- 172.21.15.169:0/3407891600 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f299006bd30 0x7f29900aac60 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f2988005fd0 tx=0x7f298802d1e0 comp rx=0 tx=0).stop 2023-10-26T01:43:29.452 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.449+0000 7f2996b6c640 1 -- 172.21.15.169:0/3407891600 shutdown_connections 2023-10-26T01:43:29.453 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.449+0000 7f2996b6c640 1 --2- 172.21.15.169:0/3407891600 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f299006bd30 0x7f29900aac60 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:29.453 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.449+0000 7f2996b6c640 1 -- 172.21.15.169:0/3407891600 >> 172.21.15.169:0/3407891600 conn(0x7f29900aefe0 msgr2=0x7f29900af3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:29.453 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.449+0000 7f2996b6c640 1 -- 172.21.15.169:0/3407891600 shutdown_connections 2023-10-26T01:43:29.453 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.449+0000 7f2996b6c640 1 -- 172.21.15.169:0/3407891600 wait complete. 2023-10-26T01:43:29.455 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.451+0000 7f2996b6c640 1 Processor -- start 2023-10-26T01:43:29.455 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.451+0000 7f2996b6c640 1 -- start start 2023-10-26T01:43:29.455 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.452+0000 7f2996b6c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f299006bd30 0x7f299014da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:29.455 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.452+0000 7f2996b6c640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f299006c650 0x7f2990156680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:29.456 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.452+0000 7f2996b6c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f299015ba30 0x7f29900c7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:29.456 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.452+0000 7f2996b6c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29900cc3d0 con 0x7f299006bd30 2023-10-26T01:43:29.456 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.452+0000 7f2996b6c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29900cb7c0 con 0x7f299015ba30 2023-10-26T01:43:29.456 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.452+0000 7f2996b6c640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29900cbaa0 con 0x7f299006c650 2023-10-26T01:43:29.457 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.452+0000 7f2994908640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f299006bd30 0x7f299014da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:29.457 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.452+0000 7f2994908640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f299006bd30 0x7f299014da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:49598/0 (socket says 172.21.15.169:49598) 2023-10-26T01:43:29.457 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.452+0000 7f2994908640 1 -- 172.21.15.169:0/813691697 learned_addr learned my addr 172.21.15.169:0/813691697 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:43:29.457 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.452+0000 7f2994908640 1 -- 172.21.15.169:0/813691697 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f299006c650 msgr2=0x7f2990156680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:29.457 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.452+0000 7f2994908640 1 --2- 172.21.15.169:0/813691697 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f299006c650 0x7f2990156680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:29.458 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.452+0000 7f2994908640 1 -- 172.21.15.169:0/813691697 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f299015ba30 msgr2=0x7f29900c7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:29.458 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.452+0000 7f2994908640 1 --2- 172.21.15.169:0/813691697 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f299015ba30 0x7f29900c7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:29.458 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.452+0000 7f2994908640 1 -- 172.21.15.169:0/813691697 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f298800aa10 con 0x7f299006bd30 2023-10-26T01:43:29.460 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.453+0000 7f2994908640 1 --2- 172.21.15.169:0/813691697 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f299006bd30 0x7f299014da60 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7f298400c750 tx=0x7f298400cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:29.460 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.453+0000 7f298d7fa640 1 -- 172.21.15.169:0/813691697 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2984005270 con 0x7f299006bd30 2023-10-26T01:43:29.460 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.453+0000 7f298d7fa640 1 -- 172.21.15.169:0/813691697 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f29840053f0 con 0x7f299006bd30 2023-10-26T01:43:29.460 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.453+0000 7f298d7fa640 1 -- 172.21.15.169:0/813691697 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f298400a8a0 con 0x7f299006bd30 2023-10-26T01:43:29.460 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.453+0000 7f2996b6c640 1 -- 172.21.15.169:0/813691697 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29900cbd80 con 0x7f299006bd30 2023-10-26T01:43:29.461 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.453+0000 7f2996b6c640 1 -- 172.21.15.169:0/813691697 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2990157420 con 0x7f299006bd30 2023-10-26T01:43:29.461 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.454+0000 7f2996b6c640 1 -- 172.21.15.169:0/813691697 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2990066af0 con 0x7f299006bd30 2023-10-26T01:43:29.461 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.455+0000 7f298d7fa640 1 -- 172.21.15.169:0/813691697 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f298400aa00 con 0x7f299006bd30 2023-10-26T01:43:29.461 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.456+0000 7f298d7fa640 1 --2- 172.21.15.169:0/813691697 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f297406ae70 0x7f297406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:29.461 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.456+0000 7f298d7fa640 1 -- 172.21.15.169:0/813691697 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f2984088c00 con 0x7f299006bd30 2023-10-26T01:43:29.462 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.456+0000 7f298d7fa640 1 -- 172.21.15.169:0/813691697 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=171}) v3 -- 0x7f2974072a00 con 0x7f299006bd30 2023-10-26T01:43:29.462 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.457+0000 7f298ffff640 1 --2- 172.21.15.169:0/813691697 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f297406ae70 0x7f297406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:29.462 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.458+0000 7f298ffff640 1 --2- 172.21.15.169:0/813691697 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f297406ae70 0x7f297406d330 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f297c006fd0 tx=0x7f297c006bc0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:29.463 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.460+0000 7f298d7fa640 1 -- 172.21.15.169:0/813691697 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2984017030 con 0x7f299006bd30 2023-10-26T01:43:29.655 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.652+0000 7f2996b6c640 1 -- 172.21.15.169:0/813691697 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f29900ab1a0 con 0x7f299006bd30 2023-10-26T01:43:29.655 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.653+0000 7f298d7fa640 1 -- 172.21.15.169:0/813691697 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v170) v1 ==== 100+0+11 (secure 0 0 0) 0x7f29900ab1a0 con 0x7f299006bd30 2023-10-26T01:43:29.656 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 64 2023-10-26T01:43:29.658 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.655+0000 7f2996b6c640 1 -- 172.21.15.169:0/813691697 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f297406ae70 msgr2=0x7f297406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:29.659 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.655+0000 7f2996b6c640 1 --2- 172.21.15.169:0/813691697 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f297406ae70 0x7f297406d330 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f297c006fd0 tx=0x7f297c006bc0 comp rx=0 tx=0).stop 2023-10-26T01:43:29.659 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.655+0000 7f2996b6c640 1 -- 172.21.15.169:0/813691697 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f299006bd30 msgr2=0x7f299014da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:29.659 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.655+0000 7f2996b6c640 1 --2- 172.21.15.169:0/813691697 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f299006bd30 0x7f299014da60 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7f298400c750 tx=0x7f298400cc10 comp rx=0 tx=0).stop 2023-10-26T01:43:29.659 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.656+0000 7f2996b6c640 1 -- 172.21.15.169:0/813691697 shutdown_connections 2023-10-26T01:43:29.660 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.656+0000 7f2996b6c640 1 --2- 172.21.15.169:0/813691697 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f297406ae70 0x7f297406d330 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:29.660 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.656+0000 7f2996b6c640 1 --2- 172.21.15.169:0/813691697 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f299015ba30 0x7f29900c7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:29.660 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.656+0000 7f2996b6c640 1 --2- 172.21.15.169:0/813691697 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f299006c650 0x7f2990156680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:29.660 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.656+0000 7f2996b6c640 1 --2- 172.21.15.169:0/813691697 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f299006bd30 0x7f299014da60 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:29.660 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.656+0000 7f2996b6c640 1 -- 172.21.15.169:0/813691697 >> 172.21.15.169:0/813691697 conn(0x7f29900aefe0 msgr2=0x7f29900ad730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:29.661 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.656+0000 7f2996b6c640 1 -- 172.21.15.169:0/813691697 shutdown_connections 2023-10-26T01:43:29.661 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.656+0000 7f2996b6c640 1 -- 172.21.15.169:0/813691697 wait complete. 2023-10-26T01:43:29.678 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-26T01:43:29.901 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.898+0000 7f7ab9613640 1 -- 172.21.15.169:0/701316211 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ab4069ff0 msgr2=0x7f7ab40aab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:29.902 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.898+0000 7f7ab9613640 1 --2- 172.21.15.169:0/701316211 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ab4069ff0 0x7f7ab40aab20 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7f7aac005fd0 tx=0x7f7aac02c050 comp rx=0 tx=0).stop 2023-10-26T01:43:29.902 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.898+0000 7f7ab9613640 1 -- 172.21.15.169:0/701316211 shutdown_connections 2023-10-26T01:43:29.902 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.898+0000 7f7ab9613640 1 --2- 172.21.15.169:0/701316211 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ab4069ff0 0x7f7ab40aab20 unknown :-1 s=CLOSED pgs=1752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:29.903 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.899+0000 7f7ab9613640 1 -- 172.21.15.169:0/701316211 >> 172.21.15.169:0/701316211 conn(0x7f7ab40af110 msgr2=0x7f7ab40af520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:29.903 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.899+0000 7f7ab9613640 1 -- 172.21.15.169:0/701316211 shutdown_connections 2023-10-26T01:43:29.903 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.899+0000 7f7ab9613640 1 -- 172.21.15.169:0/701316211 wait complete. 2023-10-26T01:43:29.903 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.900+0000 7f7ab9613640 1 Processor -- start 2023-10-26T01:43:29.904 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.901+0000 7f7ab9613640 1 -- start start 2023-10-26T01:43:29.905 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.901+0000 7f7ab9613640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ab4069c10 0x7f7ab409ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:29.905 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.902+0000 7f7ab9613640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7ab4069ff0 0x7f7ab41f3250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:29.906 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.902+0000 7f7ab9613640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ab4159640 0x7f7ab41f3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:29.906 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.902+0000 7f7ab9613640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab409b180 con 0x7f7ab4159640 2023-10-26T01:43:29.906 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.902+0000 7f7ab9613640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab4099ba0 con 0x7f7ab4069c10 2023-10-26T01:43:29.906 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.902+0000 7f7ab9613640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab4099e80 con 0x7f7ab4069ff0 2023-10-26T01:43:29.906 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.902+0000 7f7ab8e12640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ab4159640 0x7f7ab41f3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:29.907 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.902+0000 7f7ab8e12640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ab4159640 0x7f7ab41f3790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:49620/0 (socket says 172.21.15.169:49620) 2023-10-26T01:43:29.907 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.902+0000 7f7ab8e12640 1 -- 172.21.15.169:0/561773915 learned_addr learned my addr 172.21.15.169:0/561773915 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:43:29.907 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.902+0000 7f7ab3fff640 1 --2- 172.21.15.169:0/561773915 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ab4069c10 0x7f7ab409ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:29.907 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.902+0000 7f7aabfff640 1 --2- 172.21.15.169:0/561773915 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7ab4069ff0 0x7f7ab41f3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:29.907 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.902+0000 7f7ab8e12640 1 -- 172.21.15.169:0/561773915 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7ab4069ff0 msgr2=0x7f7ab41f3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:29.908 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.902+0000 7f7ab8e12640 1 --2- 172.21.15.169:0/561773915 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7ab4069ff0 0x7f7ab41f3250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:29.908 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.902+0000 7f7ab8e12640 1 -- 172.21.15.169:0/561773915 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ab4069c10 msgr2=0x7f7ab409ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:29.908 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.902+0000 7f7ab8e12640 1 --2- 172.21.15.169:0/561773915 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ab4069c10 0x7f7ab409ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:29.908 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.902+0000 7f7ab8e12640 1 -- 172.21.15.169:0/561773915 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7aac009840 con 0x7f7ab4159640 2023-10-26T01:43:29.908 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.903+0000 7f7ab8e12640 1 --2- 172.21.15.169:0/561773915 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ab4159640 0x7f7ab41f3790 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f7aac009b30 tx=0x7f7aac004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:29.910 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.903+0000 7f7ab17fa640 1 -- 172.21.15.169:0/561773915 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aac004870 con 0x7f7ab4159640 2023-10-26T01:43:29.910 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.903+0000 7f7ab17fa640 1 -- 172.21.15.169:0/561773915 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7aac03e550 con 0x7f7ab4159640 2023-10-26T01:43:29.910 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.903+0000 7f7ab17fa640 1 -- 172.21.15.169:0/561773915 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aac03e960 con 0x7f7ab4159640 2023-10-26T01:43:29.910 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.903+0000 7f7ab9613640 1 -- 172.21.15.169:0/561773915 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ab409a160 con 0x7f7ab4159640 2023-10-26T01:43:29.910 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.903+0000 7f7ab9613640 1 -- 172.21.15.169:0/561773915 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ab409a720 con 0x7f7ab4159640 2023-10-26T01:43:29.911 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.904+0000 7f7ab9613640 1 -- 172.21.15.169:0/561773915 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ab4073c10 con 0x7f7ab4159640 2023-10-26T01:43:29.911 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.905+0000 7f7ab17fa640 1 -- 172.21.15.169:0/561773915 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f7aac03eac0 con 0x7f7ab4159640 2023-10-26T01:43:29.911 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.906+0000 7f7ab17fa640 1 --2- 172.21.15.169:0/561773915 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7a9406ae40 0x7f7a9406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:29.911 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.906+0000 7f7ab17fa640 1 -- 172.21.15.169:0/561773915 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f7aac0ac730 con 0x7f7ab4159640 2023-10-26T01:43:29.911 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.906+0000 7f7ab17fa640 1 -- 172.21.15.169:0/561773915 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=171}) v3 -- 0x7f7a940729d0 con 0x7f7ab4159640 2023-10-26T01:43:29.912 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.907+0000 7f7ab3fff640 1 --2- 172.21.15.169:0/561773915 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7a9406ae40 0x7f7a9406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:29.912 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.908+0000 7f7ab3fff640 1 --2- 172.21.15.169:0/561773915 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7a9406ae40 0x7f7a9406d300 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f7aa4000f60 tx=0x7f7aa4009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:29.913 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:29.910+0000 7f7ab17fa640 1 -- 172.21.15.169:0/561773915 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7aac042060 con 0x7f7ab4159640 2023-10-26T01:43:30.116 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:30.113+0000 7f7ab9613640 1 -- 172.21.15.169:0/561773915 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f7ab4073c10 con 0x7f7ab4159640 2023-10-26T01:43:30.116 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:30.114+0000 7f7ab17fa640 1 -- 172.21.15.169:0/561773915 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v170) v1 ==== 80+0+0 (secure 0 0 0) 0x7f7aac077720 con 0x7f7ab4159640 2023-10-26T01:43:30.117 INFO:teuthology.orchestra.run.smithi169.stderr:listed 0 entries 2023-10-26T01:43:30.119 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:30.116+0000 7f7ab9613640 1 -- 172.21.15.169:0/561773915 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7a9406ae40 msgr2=0x7f7a9406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:30.120 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:30.116+0000 7f7ab9613640 1 --2- 172.21.15.169:0/561773915 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7a9406ae40 0x7f7a9406d300 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f7aa4000f60 tx=0x7f7aa4009040 comp rx=0 tx=0).stop 2023-10-26T01:43:30.120 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:30.116+0000 7f7ab9613640 1 -- 172.21.15.169:0/561773915 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ab4159640 msgr2=0x7f7ab41f3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:30.120 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:30.116+0000 7f7ab9613640 1 --2- 172.21.15.169:0/561773915 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ab4159640 0x7f7ab41f3790 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f7aac009b30 tx=0x7f7aac004270 comp rx=0 tx=0).stop 2023-10-26T01:43:30.120 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:30.117+0000 7f7ab9613640 1 -- 172.21.15.169:0/561773915 shutdown_connections 2023-10-26T01:43:30.121 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:30.117+0000 7f7ab9613640 1 --2- 172.21.15.169:0/561773915 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7a9406ae40 0x7f7a9406d300 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:30.121 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:30.117+0000 7f7ab9613640 1 --2- 172.21.15.169:0/561773915 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ab4159640 0x7f7ab41f3790 unknown :-1 s=CLOSED pgs=1753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:30.121 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:30.117+0000 7f7ab9613640 1 --2- 172.21.15.169:0/561773915 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7ab4069ff0 0x7f7ab41f3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:30.121 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:30.117+0000 7f7ab9613640 1 --2- 172.21.15.169:0/561773915 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ab4069c10 0x7f7ab409ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:30.121 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:30.117+0000 7f7ab9613640 1 -- 172.21.15.169:0/561773915 >> 172.21.15.169:0/561773915 conn(0x7f7ab40af110 msgr2=0x7f7ab4159e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:30.122 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:30.117+0000 7f7ab9613640 1 -- 172.21.15.169:0/561773915 shutdown_connections 2023-10-26T01:43:30.122 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:30.117+0000 7f7ab9613640 1 -- 172.21.15.169:0/561773915 wait complete. 2023-10-26T01:43:30.139 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-26T01:43:30.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:43:30.140 DEBUG:teuthology.orchestra.run.smithi170:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-26T01:43:30.255 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi170.front.sepia.ceph.com 2023-10-26T01:43:30.256 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:43:30.256 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-26T01:43:30.310 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:43:30.311 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:43:30.373 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054301,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-26T01:43:30.373 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-26T01:43:30.374 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:43:30.435 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054303,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-26T01:43:30.436 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T01:43:30.665 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:30.662+0000 7f34b6387640 1 -- 172.21.15.169:0/1534558160 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34b00ac6d0 msgr2=0x7f34b015d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:30.666 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:30.662+0000 7f34b6387640 1 --2- 172.21.15.169:0/1534558160 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34b00ac6d0 0x7f34b015d840 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f34b006ced0 tx=0x7f34ac02bd70 comp rx=0 tx=0).stop 2023-10-26T01:43:30.666 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:30.662+0000 7f34b6387640 1 -- 172.21.15.169:0/1534558160 shutdown_connections 2023-10-26T01:43:30.666 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:30.662+0000 7f34b6387640 1 --2- 172.21.15.169:0/1534558160 >> v2:172.21.15.170:3300/0 conn(0x7f34b009a5c0 0x7f34b009aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:30.666 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:30.662+0000 7f34b6387640 1 --2- 172.21.15.169:0/1534558160 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34b00ac6d0 0x7f34b015d840 unknown :-1 s=CLOSED pgs=1755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:30.666 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:30.662+0000 7f34b6387640 1 -- 172.21.15.169:0/1534558160 >> 172.21.15.169:0/1534558160 conn(0x7f34b00aedd0 msgr2=0x7f34b00af1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:30.667 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:30.662+0000 7f34b6387640 1 -- 172.21.15.169:0/1534558160 shutdown_connections 2023-10-26T01:43:30.667 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:30.662+0000 7f34b6387640 1 -- 172.21.15.169:0/1534558160 wait complete. 2023-10-26T01:43:30.669 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:30.664+0000 7f34b6387640 1 Processor -- start 2023-10-26T01:43:30.669 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:30.664+0000 7f34b6387640 1 -- start start 2023-10-26T01:43:30.670 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:30.665+0000 7f34b6387640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f34b009a5c0 0x7f34b0071300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:30.670 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:30.665+0000 7f34b6387640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34b00ac6d0 0x7f34b01f5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:30.670 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:30.665+0000 7f34b6387640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f34b00c9970 0x7f34b01f5890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:30.670 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:30.665+0000 7f34b6387640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34b0071840 con 0x7f34b00ac6d0 2023-10-26T01:43:30.670 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:30.665+0000 7f34b6387640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34b00b2210 con 0x7f34b00c9970 2023-10-26T01:43:30.671 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:30.665+0000 7f34b6387640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34b00b24f0 con 0x7f34b009a5c0 2023-10-26T01:43:30.671 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:30.665+0000 7f34b4b84640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34b00ac6d0 0x7f34b01f5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:30.671 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:30.666+0000 7f34b4b84640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34b00ac6d0 0x7f34b01f5350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:49632/0 (socket says 172.21.15.169:49632) 2023-10-26T01:43:30.672 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:30.666+0000 7f34b4b84640 1 -- 172.21.15.169:0/2439092050 learned_addr learned my addr 172.21.15.169:0/2439092050 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:43:30.672 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:30.666+0000 7f34b5385640 1 --2- 172.21.15.169:0/2439092050 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f34b009a5c0 0x7f34b0071300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:30.672 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:30.666+0000 7f34b5b86640 1 --2- 172.21.15.169:0/2439092050 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f34b00c9970 0x7f34b01f5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:30.672 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:30.666+0000 7f34b4b84640 1 -- 172.21.15.169:0/2439092050 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f34b009a5c0 msgr2=0x7f34b0071300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:30.672 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:30.666+0000 7f34b4b84640 1 --2- 172.21.15.169:0/2439092050 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f34b009a5c0 0x7f34b0071300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:30.672 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:30.666+0000 7f34b4b84640 1 -- 172.21.15.169:0/2439092050 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f34b00c9970 msgr2=0x7f34b01f5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:30.673 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:30.666+0000 7f34b4b84640 1 --2- 172.21.15.169:0/2439092050 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f34b00c9970 0x7f34b01f5890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:30.673 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:30.666+0000 7f34b4b84640 1 -- 172.21.15.169:0/2439092050 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34ac009490 con 0x7f34b00ac6d0 2023-10-26T01:43:30.673 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:30.666+0000 7f34b5385640 1 --2- 172.21.15.169:0/2439092050 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f34b009a5c0 0x7f34b0071300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:43:30.673 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:30.666+0000 7f34b5b86640 1 --2- 172.21.15.169:0/2439092050 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f34b00c9970 0x7f34b01f5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:43:30.673 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:30.666+0000 7f34b4b84640 1 --2- 172.21.15.169:0/2439092050 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34b00ac6d0 0x7f34b01f5350 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f34a000c3c0 tx=0x7f34a000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:30.674 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:30.667+0000 7f34a5ffb640 1 -- 172.21.15.169:0/2439092050 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34a0011790 con 0x7f34b00ac6d0 2023-10-26T01:43:30.674 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:30.667+0000 7f34a5ffb640 1 -- 172.21.15.169:0/2439092050 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f34a00119e0 con 0x7f34b00ac6d0 2023-10-26T01:43:30.674 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:30.667+0000 7f34a5ffb640 1 -- 172.21.15.169:0/2439092050 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34a0011e10 con 0x7f34b00ac6d0 2023-10-26T01:43:30.674 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:30.667+0000 7f34b6387640 1 -- 172.21.15.169:0/2439092050 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34b00b27d0 con 0x7f34b00ac6d0 2023-10-26T01:43:30.674 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:30.667+0000 7f34b6387640 1 -- 172.21.15.169:0/2439092050 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34b00b2d90 con 0x7f34b00ac6d0 2023-10-26T01:43:30.675 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:30.668+0000 7f34b6387640 1 -- 172.21.15.169:0/2439092050 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34b0073c10 con 0x7f34b00ac6d0 2023-10-26T01:43:30.676 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:30.669+0000 7f34a5ffb640 1 -- 172.21.15.169:0/2439092050 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f34a001c460 con 0x7f34b00ac6d0 2023-10-26T01:43:30.676 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:30.670+0000 7f34a5ffb640 1 --2- 172.21.15.169:0/2439092050 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f349406ae40 0x7f349406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:30.676 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:30.670+0000 7f34a5ffb640 1 -- 172.21.15.169:0/2439092050 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f34a0089500 con 0x7f34b00ac6d0 2023-10-26T01:43:30.676 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:30.670+0000 7f34a5ffb640 1 -- 172.21.15.169:0/2439092050 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=171}) v3 -- 0x7f34940729d0 con 0x7f34b00ac6d0 2023-10-26T01:43:30.676 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:30.671+0000 7f34b5385640 1 --2- 172.21.15.169:0/2439092050 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f349406ae40 0x7f349406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:30.677 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:30.672+0000 7f34b5385640 1 --2- 172.21.15.169:0/2439092050 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f349406ae40 0x7f349406d300 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f34ac036000 tx=0x7f34ac004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:30.677 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:30.674+0000 7f34a5ffb640 1 -- 172.21.15.169:0/2439092050 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f34a0054590 con 0x7f34b00ac6d0 2023-10-26T01:43:30.905 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:30.902+0000 7f34b6387640 1 -- 172.21.15.169:0/2439092050 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f34b00ca410 con 0x7f34b00ac6d0 2023-10-26T01:43:30.906 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:30.903+0000 7f34a5ffb640 1 -- 172.21.15.169:0/2439092050 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) v1 ==== 94+0+4923 (secure 0 0 0) 0x7f34b00ca410 con 0x7f34b00ac6d0 2023-10-26T01:43:30.906 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:43:30.906 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":86,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5861,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/326391134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":326391134},{"type":"v1","addr":"172.21.15.170:6836","nonce":326391134}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":5864,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/3374486280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":3374486280},{"type":"v1","addr":"172.21.15.170:6837","nonce":3374486280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":8697,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/1445776478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":1445776478},{"type":"v1","addr":"172.21.15.169:6834","nonce":1445776478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T01:42:09.853788+0000","modified":"2023-10-26T01:42:12.922104+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8700},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8700":{"gid":8700,"name":"a","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6835/228678953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":228678953},{"type":"v1","addr":"172.21.15.169:6835","nonce":228678953}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-10-26T01:43:30.907 INFO:teuthology.orchestra.run.smithi169.stderr:dumped fsmap epoch 86 2023-10-26T01:43:30.908 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:30.905+0000 7f34b6387640 1 -- 172.21.15.169:0/2439092050 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f349406ae40 msgr2=0x7f349406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:30.909 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:30.905+0000 7f34b6387640 1 --2- 172.21.15.169:0/2439092050 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f349406ae40 0x7f349406d300 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f34ac036000 tx=0x7f34ac004730 comp rx=0 tx=0).stop 2023-10-26T01:43:30.909 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:30.906+0000 7f34b6387640 1 -- 172.21.15.169:0/2439092050 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34b00ac6d0 msgr2=0x7f34b01f5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:30.909 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:30.906+0000 7f34b6387640 1 --2- 172.21.15.169:0/2439092050 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34b00ac6d0 0x7f34b01f5350 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f34a000c3c0 tx=0x7f34a000c880 comp rx=0 tx=0).stop 2023-10-26T01:43:30.909 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:30.906+0000 7f34b6387640 1 -- 172.21.15.169:0/2439092050 shutdown_connections 2023-10-26T01:43:30.910 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:30.906+0000 7f34b6387640 1 --2- 172.21.15.169:0/2439092050 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f349406ae40 0x7f349406d300 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:30.910 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:30.906+0000 7f34b6387640 1 --2- 172.21.15.169:0/2439092050 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34b00ac6d0 0x7f34b01f5350 unknown :-1 s=CLOSED pgs=1756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:30.910 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:30.906+0000 7f34b6387640 1 --2- 172.21.15.169:0/2439092050 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f34b00c9970 0x7f34b01f5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:30.910 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:30.906+0000 7f34b6387640 1 --2- 172.21.15.169:0/2439092050 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f34b009a5c0 0x7f34b0071300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:30.910 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:30.906+0000 7f34b6387640 1 -- 172.21.15.169:0/2439092050 >> 172.21.15.169:0/2439092050 conn(0x7f34b00aedd0 msgr2=0x7f34b01419e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:30.911 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:30.906+0000 7f34b6387640 1 -- 172.21.15.169:0/2439092050 shutdown_connections 2023-10-26T01:43:30.911 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:30.906+0000 7f34b6387640 1 -- 172.21.15.169:0/2439092050 wait complete. 2023-10-26T01:43:30.928 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:43:31.150 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:31.145+0000 7fd011c02640 1 -- 172.21.15.169:0/2447814106 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd00c14ece0 msgr2=0x7fd00c0aa1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:31.151 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:31.145+0000 7fd011c02640 1 --2- 172.21.15.169:0/2447814106 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd00c14ece0 0x7fd00c0aa1c0 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7fcffc005fd0 tx=0x7fcffc02bd90 comp rx=0 tx=0).stop 2023-10-26T01:43:31.151 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:31.145+0000 7fd011c02640 1 -- 172.21.15.169:0/2447814106 shutdown_connections 2023-10-26T01:43:31.151 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:31.145+0000 7fd011c02640 1 --2- 172.21.15.169:0/2447814106 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd00c06bf70 0x7fd00c0aab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:31.151 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:31.145+0000 7fd011c02640 1 --2- 172.21.15.169:0/2447814106 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd00c14ece0 0x7fd00c0aa1c0 unknown :-1 s=CLOSED pgs=1757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:31.152 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:31.145+0000 7fd011c02640 1 -- 172.21.15.169:0/2447814106 >> 172.21.15.169:0/2447814106 conn(0x7fd00c069930 msgr2=0x7fd00c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:31.152 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:31.145+0000 7fd011c02640 1 -- 172.21.15.169:0/2447814106 shutdown_connections 2023-10-26T01:43:31.152 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:31.145+0000 7fd011c02640 1 -- 172.21.15.169:0/2447814106 wait complete. 2023-10-26T01:43:31.152 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:31.147+0000 7fd011c02640 1 Processor -- start 2023-10-26T01:43:31.152 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:31.147+0000 7fd011c02640 1 -- start start 2023-10-26T01:43:31.155 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:31.148+0000 7fd011c02640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd00c06bf70 0x7fd00c06ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:31.155 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:31.148+0000 7fd011c02640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd00c06c420 0x7fd00c14c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:31.155 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:31.148+0000 7fd011c02640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd00c14ece0 0x7fd00c0cc920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:31.155 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:31.148+0000 7fd011c02640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd00c06f480 con 0x7fd00c06bf70 2023-10-26T01:43:31.156 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:31.148+0000 7fd011c02640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd00c14c7a0 con 0x7fd00c06c420 2023-10-26T01:43:31.156 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:31.148+0000 7fd011c02640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd00c06f670 con 0x7fd00c14ece0 2023-10-26T01:43:31.156 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:31.149+0000 7fd00b7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd00c06bf70 0x7fd00c06ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:31.156 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:31.149+0000 7fd00b7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd00c06bf70 0x7fd00c06ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:49644/0 (socket says 172.21.15.169:49644) 2023-10-26T01:43:31.156 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:31.149+0000 7fd00b7fe640 1 -- 172.21.15.169:0/3707558738 learned_addr learned my addr 172.21.15.169:0/3707558738 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:43:31.157 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:31.149+0000 7fd003fff640 1 --2- 172.21.15.169:0/3707558738 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd00c06c420 0x7fd00c14c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:31.157 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:31.149+0000 7fd00bfff640 1 --2- 172.21.15.169:0/3707558738 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd00c14ece0 0x7fd00c0cc920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:31.157 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:31.149+0000 7fd00b7fe640 1 -- 172.21.15.169:0/3707558738 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd00c14ece0 msgr2=0x7fd00c0cc920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:31.157 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:31.149+0000 7fd00b7fe640 1 --2- 172.21.15.169:0/3707558738 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd00c14ece0 0x7fd00c0cc920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:31.157 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:31.149+0000 7fd00b7fe640 1 -- 172.21.15.169:0/3707558738 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd00c06c420 msgr2=0x7fd00c14c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:31.158 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:31.149+0000 7fd00b7fe640 1 --2- 172.21.15.169:0/3707558738 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd00c06c420 0x7fd00c14c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:31.158 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:31.149+0000 7fd00b7fe640 1 -- 172.21.15.169:0/3707558738 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcffc009560 con 0x7fd00c06bf70 2023-10-26T01:43:31.158 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:31.150+0000 7fd00b7fe640 1 --2- 172.21.15.169:0/3707558738 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd00c06bf70 0x7fd00c06ef40 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7fcffc036040 tx=0x7fcffc02da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:31.158 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:31.150+0000 7fd008ff9640 1 -- 172.21.15.169:0/3707558738 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcffc003800 con 0x7fd00c06bf70 2023-10-26T01:43:31.158 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:31.150+0000 7fd008ff9640 1 -- 172.21.15.169:0/3707558738 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fcffc003a50 con 0x7fd00c06bf70 2023-10-26T01:43:31.159 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:31.150+0000 7fd011c02640 1 -- 172.21.15.169:0/3707558738 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd00c06f950 con 0x7fd00c06bf70 2023-10-26T01:43:31.159 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:31.150+0000 7fd008ff9640 1 -- 172.21.15.169:0/3707558738 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcffc003e00 con 0x7fd00c06bf70 2023-10-26T01:43:31.159 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:31.150+0000 7fd011c02640 1 -- 172.21.15.169:0/3707558738 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd00c06ff10 con 0x7fd00c06bf70 2023-10-26T01:43:31.159 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:31.151+0000 7fd011c02640 1 -- 172.21.15.169:0/3707558738 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd00c04db10 con 0x7fd00c06bf70 2023-10-26T01:43:31.159 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:31.152+0000 7fd008ff9640 1 -- 172.21.15.169:0/3707558738 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fcffc03e450 con 0x7fd00c06bf70 2023-10-26T01:43:31.160 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:31.153+0000 7fd008ff9640 1 --2- 172.21.15.169:0/3707558738 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcfec06ae40 0x7fcfec06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:31.160 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:31.153+0000 7fd008ff9640 1 -- 172.21.15.169:0/3707558738 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fcffc0ac1a0 con 0x7fd00c06bf70 2023-10-26T01:43:31.161 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:31.153+0000 7fd008ff9640 1 -- 172.21.15.169:0/3707558738 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=171}) v3 -- 0x7fcfec0729d0 con 0x7fd00c06bf70 2023-10-26T01:43:31.161 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:31.154+0000 7fd003fff640 1 --2- 172.21.15.169:0/3707558738 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcfec06ae40 0x7fcfec06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:31.161 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:31.155+0000 7fd003fff640 1 --2- 172.21.15.169:0/3707558738 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcfec06ae40 0x7fcfec06d300 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fcff4005fd0 tx=0x7fcff4005850 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:31.161 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:31.157+0000 7fd008ff9640 1 -- 172.21.15.169:0/3707558738 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcffc077220 con 0x7fd00c06bf70 2023-10-26T01:43:31.357 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:31.354+0000 7fd011c02640 1 -- 172.21.15.169:0/3707558738 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd00c0b06f0 con 0x7fd00c06bf70 2023-10-26T01:43:31.360 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:31.357+0000 7fd008ff9640 1 -- 172.21.15.169:0/3707558738 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v170) v1 ==== 74+0+16704 (secure 0 0 0) 0x7fcffc042070 con 0x7fd00c06bf70 2023-10-26T01:43:31.361 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:43:31.361 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":170,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:42:58.161990+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2023-10-26T01:42:07.656531+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"165","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2023-10-26T01:42:08.866635+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"170","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.39","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:43:31.363 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:31.360+0000 7fd011c02640 1 -- 172.21.15.169:0/3707558738 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcfec06ae40 msgr2=0x7fcfec06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:31.363 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:31.360+0000 7fd011c02640 1 --2- 172.21.15.169:0/3707558738 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcfec06ae40 0x7fcfec06d300 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fcff4005fd0 tx=0x7fcff4005850 comp rx=0 tx=0).stop 2023-10-26T01:43:31.364 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:31.360+0000 7fd011c02640 1 -- 172.21.15.169:0/3707558738 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd00c06bf70 msgr2=0x7fd00c06ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:31.364 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:31.360+0000 7fd011c02640 1 --2- 172.21.15.169:0/3707558738 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd00c06bf70 0x7fd00c06ef40 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7fcffc036040 tx=0x7fcffc02da60 comp rx=0 tx=0).stop 2023-10-26T01:43:31.364 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:31.360+0000 7fd011c02640 1 -- 172.21.15.169:0/3707558738 shutdown_connections 2023-10-26T01:43:31.364 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:31.360+0000 7fd011c02640 1 --2- 172.21.15.169:0/3707558738 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcfec06ae40 0x7fcfec06d300 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:31.364 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:31.360+0000 7fd011c02640 1 --2- 172.21.15.169:0/3707558738 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd00c14ece0 0x7fd00c0cc920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:31.365 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:31.360+0000 7fd011c02640 1 --2- 172.21.15.169:0/3707558738 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd00c06c420 0x7fd00c14c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:31.365 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:31.360+0000 7fd011c02640 1 --2- 172.21.15.169:0/3707558738 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd00c06bf70 0x7fd00c06ef40 unknown :-1 s=CLOSED pgs=1758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:31.365 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:31.360+0000 7fd011c02640 1 -- 172.21.15.169:0/3707558738 >> 172.21.15.169:0/3707558738 conn(0x7fd00c069930 msgr2=0x7fd00c0b0910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:31.365 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:31.360+0000 7fd011c02640 1 -- 172.21.15.169:0/3707558738 shutdown_connections 2023-10-26T01:43:31.365 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:31.360+0000 7fd011c02640 1 -- 172.21.15.169:0/3707558738 wait complete. 2023-10-26T01:43:31.382 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T01:30:32.200808+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '109', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 20, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-26T01:42:07.656531+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '165', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 21, 'pool_name': 'cephfs_data', 'create_time': '2023-10-26T01:42:08.866635+0000', 'flags': 1027, 'flags_names': 'hashpspool,full,full_quota', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '170', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T01:43:31.383 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T01:43:31.609 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:31.606+0000 7f6e43f4c640 1 -- 172.21.15.169:0/3655900229 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e3c0b0a80 msgr2=0x7f6e3c0ac870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:31.610 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:31.606+0000 7f6e43f4c640 1 --2- 172.21.15.169:0/3655900229 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e3c0b0a80 0x7f6e3c0ac870 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f6e38009860 tx=0x7f6e3802bd60 comp rx=0 tx=0).stop 2023-10-26T01:43:31.610 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:31.606+0000 7f6e43f4c640 1 -- 172.21.15.169:0/3655900229 shutdown_connections 2023-10-26T01:43:31.610 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:31.606+0000 7f6e43f4c640 1 --2- 172.21.15.169:0/3655900229 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6e3c06cdb0 0x7f6e3c0cd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:31.610 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:31.606+0000 7f6e43f4c640 1 --2- 172.21.15.169:0/3655900229 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e3c0b0a80 0x7f6e3c0ac870 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:31.611 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:31.606+0000 7f6e43f4c640 1 -- 172.21.15.169:0/3655900229 >> 172.21.15.169:0/3655900229 conn(0x7f6e3c069930 msgr2=0x7f6e3c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:31.611 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:31.607+0000 7f6e43f4c640 1 -- 172.21.15.169:0/3655900229 shutdown_connections 2023-10-26T01:43:31.611 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:31.607+0000 7f6e43f4c640 1 -- 172.21.15.169:0/3655900229 wait complete. 2023-10-26T01:43:31.612 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:31.609+0000 7f6e43f4c640 1 Processor -- start 2023-10-26T01:43:31.612 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:31.609+0000 7f6e43f4c640 1 -- start start 2023-10-26T01:43:31.614 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:31.610+0000 7f6e43f4c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6e3c06cdb0 0x7f6e3c096910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:31.614 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:31.610+0000 7f6e43f4c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e3c0ab070 0x7f6e3c152450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:31.614 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:31.610+0000 7f6e43f4c640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6e3c152990 0x7f6e3c14f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:31.615 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:31.610+0000 7f6e43f4c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e3c152e40 con 0x7f6e3c06cdb0 2023-10-26T01:43:31.615 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:31.610+0000 7f6e43f4c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e3c0ca320 con 0x7f6e3c0ab070 2023-10-26T01:43:31.615 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:31.610+0000 7f6e43f4c640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e3c0ca600 con 0x7f6e3c152990 2023-10-26T01:43:31.615 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:31.611+0000 7f6e41ce8640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6e3c06cdb0 0x7f6e3c096910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:31.615 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:31.611+0000 7f6e41ce8640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6e3c06cdb0 0x7f6e3c096910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:49656/0 (socket says 172.21.15.169:49656) 2023-10-26T01:43:31.616 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:31.611+0000 7f6e41ce8640 1 -- 172.21.15.169:0/1912092680 learned_addr learned my addr 172.21.15.169:0/1912092680 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:43:31.616 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:31.611+0000 7f6e414e7640 1 --2- 172.21.15.169:0/1912092680 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e3c0ab070 0x7f6e3c152450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:31.616 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:31.611+0000 7f6e424e9640 1 --2- 172.21.15.169:0/1912092680 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6e3c152990 0x7f6e3c14f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:31.616 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:31.611+0000 7f6e41ce8640 1 -- 172.21.15.169:0/1912092680 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6e3c152990 msgr2=0x7f6e3c14f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:31.616 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:31.611+0000 7f6e41ce8640 1 --2- 172.21.15.169:0/1912092680 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6e3c152990 0x7f6e3c14f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:31.617 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:31.611+0000 7f6e41ce8640 1 -- 172.21.15.169:0/1912092680 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e3c0ab070 msgr2=0x7f6e3c152450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:31.617 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:31.611+0000 7f6e41ce8640 1 --2- 172.21.15.169:0/1912092680 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e3c0ab070 0x7f6e3c152450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:31.617 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:31.611+0000 7f6e41ce8640 1 -- 172.21.15.169:0/1912092680 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e38009490 con 0x7f6e3c06cdb0 2023-10-26T01:43:31.617 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:31.612+0000 7f6e41ce8640 1 --2- 172.21.15.169:0/1912092680 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6e3c06cdb0 0x7f6e3c096910 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f6e38009980 tx=0x7f6e380038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:31.618 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:31.612+0000 7f6e327fc640 1 -- 172.21.15.169:0/1912092680 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e38003ea0 con 0x7f6e3c06cdb0 2023-10-26T01:43:31.619 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:31.612+0000 7f6e327fc640 1 -- 172.21.15.169:0/1912092680 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6e38034440 con 0x7f6e3c06cdb0 2023-10-26T01:43:31.619 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:31.612+0000 7f6e327fc640 1 -- 172.21.15.169:0/1912092680 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e38034850 con 0x7f6e3c06cdb0 2023-10-26T01:43:31.619 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:31.612+0000 7f6e43f4c640 1 -- 172.21.15.169:0/1912092680 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e3c0ca8e0 con 0x7f6e3c06cdb0 2023-10-26T01:43:31.619 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:31.612+0000 7f6e43f4c640 1 -- 172.21.15.169:0/1912092680 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e3c0caea0 con 0x7f6e3c06cdb0 2023-10-26T01:43:31.620 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:31.613+0000 7f6e43f4c640 1 -- 172.21.15.169:0/1912092680 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e3c066af0 con 0x7f6e3c06cdb0 2023-10-26T01:43:31.620 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:31.615+0000 7f6e327fc640 1 -- 172.21.15.169:0/1912092680 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f6e3803b040 con 0x7f6e3c06cdb0 2023-10-26T01:43:31.620 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:31.615+0000 7f6e327fc640 1 --2- 172.21.15.169:0/1912092680 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f6e2006ae40 0x7f6e2006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:31.620 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:31.616+0000 7f6e327fc640 1 -- 172.21.15.169:0/1912092680 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f6e380aba80 con 0x7f6e3c06cdb0 2023-10-26T01:43:31.620 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:31.616+0000 7f6e327fc640 1 -- 172.21.15.169:0/1912092680 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=171}) v3 -- 0x7f6e200729d0 con 0x7f6e3c06cdb0 2023-10-26T01:43:31.620 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:31.616+0000 7f6e414e7640 1 --2- 172.21.15.169:0/1912092680 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f6e2006ae40 0x7f6e2006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:31.621 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:31.618+0000 7f6e414e7640 1 --2- 172.21.15.169:0/1912092680 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f6e2006ae40 0x7f6e2006d300 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c071f20 tx=0x7f6e2c009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:31.623 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:31.620+0000 7f6e327fc640 1 -- 172.21.15.169:0/1912092680 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6e38039020 con 0x7f6e3c06cdb0 2023-10-26T01:43:31.810 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:31.808+0000 7f6e43f4c640 1 -- 172.21.15.169:0/1912092680 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6e3c04da70 con 0x7f6e3c06cdb0 2023-10-26T01:43:31.811 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:31.809+0000 7f6e327fc640 1 -- 172.21.15.169:0/1912092680 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v170) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6e38041030 con 0x7f6e3c06cdb0 2023-10-26T01:43:31.812 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 1 2023-10-26T01:43:31.814 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:31.811+0000 7f6e43f4c640 1 -- 172.21.15.169:0/1912092680 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f6e2006ae40 msgr2=0x7f6e2006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:31.814 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:31.811+0000 7f6e43f4c640 1 --2- 172.21.15.169:0/1912092680 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f6e2006ae40 0x7f6e2006d300 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c071f20 tx=0x7f6e2c009040 comp rx=0 tx=0).stop 2023-10-26T01:43:31.815 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:31.811+0000 7f6e43f4c640 1 -- 172.21.15.169:0/1912092680 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6e3c06cdb0 msgr2=0x7f6e3c096910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:31.815 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:31.811+0000 7f6e43f4c640 1 --2- 172.21.15.169:0/1912092680 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6e3c06cdb0 0x7f6e3c096910 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f6e38009980 tx=0x7f6e380038a0 comp rx=0 tx=0).stop 2023-10-26T01:43:31.815 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:31.811+0000 7f6e43f4c640 1 -- 172.21.15.169:0/1912092680 shutdown_connections 2023-10-26T01:43:31.815 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:31.811+0000 7f6e43f4c640 1 --2- 172.21.15.169:0/1912092680 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f6e2006ae40 0x7f6e2006d300 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:31.815 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:31.811+0000 7f6e43f4c640 1 --2- 172.21.15.169:0/1912092680 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6e3c152990 0x7f6e3c14f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:31.816 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:31.811+0000 7f6e43f4c640 1 --2- 172.21.15.169:0/1912092680 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e3c0ab070 0x7f6e3c152450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:31.816 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:31.812+0000 7f6e43f4c640 1 --2- 172.21.15.169:0/1912092680 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6e3c06cdb0 0x7f6e3c096910 unknown :-1 s=CLOSED pgs=1759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:31.816 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:31.812+0000 7f6e43f4c640 1 -- 172.21.15.169:0/1912092680 >> 172.21.15.169:0/1912092680 conn(0x7f6e3c069930 msgr2=0x7f6e3c159f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:31.816 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:31.812+0000 7f6e43f4c640 1 -- 172.21.15.169:0/1912092680 shutdown_connections 2023-10-26T01:43:31.816 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:31.812+0000 7f6e43f4c640 1 -- 172.21.15.169:0/1912092680 wait complete. 2023-10-26T01:43:31.833 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-26T01:43:32.054 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.051+0000 7f6629e56640 1 -- 172.21.15.169:0/1316082169 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f662406c650 msgr2=0x7f66240aac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:32.055 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.051+0000 7f6629e56640 1 --2- 172.21.15.169:0/1316082169 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f662406c650 0x7f66240aac60 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f6614005fd0 tx=0x7f661402c2d0 comp rx=0 tx=0).stop 2023-10-26T01:43:32.055 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.051+0000 7f6629e56640 1 -- 172.21.15.169:0/1316082169 shutdown_connections 2023-10-26T01:43:32.055 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.051+0000 7f6629e56640 1 --2- 172.21.15.169:0/1316082169 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f662415b320 0x7f662415b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:32.055 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.051+0000 7f6629e56640 1 --2- 172.21.15.169:0/1316082169 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f662406c650 0x7f66240aac60 unknown :-1 s=CLOSED pgs=1761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:32.055 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.051+0000 7f6629e56640 1 -- 172.21.15.169:0/1316082169 >> 172.21.15.169:0/1316082169 conn(0x7f66240aefe0 msgr2=0x7f66240af3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:32.056 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.051+0000 7f6629e56640 1 -- 172.21.15.169:0/1316082169 shutdown_connections 2023-10-26T01:43:32.056 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.052+0000 7f6629e56640 1 -- 172.21.15.169:0/1316082169 wait complete. 2023-10-26T01:43:32.058 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.053+0000 7f6629e56640 1 Processor -- start 2023-10-26T01:43:32.058 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.053+0000 7f6629e56640 1 -- start start 2023-10-26T01:43:32.058 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.054+0000 7f6629e56640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f662406c650 0x7f66241f1470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:32.058 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.054+0000 7f6629e56640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f662414ddf0 0x7f66240c9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:32.059 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.054+0000 7f6629e56640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f662415b320 0x7f66240ae240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:32.059 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.054+0000 7f6629e56640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66240ca780 con 0x7f662406c650 2023-10-26T01:43:32.059 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.054+0000 7f6629e56640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66240caa60 con 0x7f662415b320 2023-10-26T01:43:32.059 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.054+0000 7f6629e56640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66240cad40 con 0x7f662414ddf0 2023-10-26T01:43:32.059 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.055+0000 7f66237fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f662406c650 0x7f66241f1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:32.060 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.055+0000 7f66237fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f662406c650 0x7f66241f1470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:49670/0 (socket says 172.21.15.169:49670) 2023-10-26T01:43:32.060 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.055+0000 7f66237fe640 1 -- 172.21.15.169:0/3836114356 learned_addr learned my addr 172.21.15.169:0/3836114356 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:43:32.060 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.055+0000 7f66237fe640 1 -- 172.21.15.169:0/3836114356 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f662414ddf0 msgr2=0x7f66240c9cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:32.060 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.055+0000 7f66237fe640 1 --2- 172.21.15.169:0/3836114356 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f662414ddf0 0x7f66240c9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:32.060 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.055+0000 7f66237fe640 1 -- 172.21.15.169:0/3836114356 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f662415b320 msgr2=0x7f66240ae240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:32.061 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.055+0000 7f66237fe640 1 --2- 172.21.15.169:0/3836114356 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f662415b320 0x7f66240ae240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:32.061 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.055+0000 7f66237fe640 1 -- 172.21.15.169:0/3836114356 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6614009900 con 0x7f662406c650 2023-10-26T01:43:32.061 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.055+0000 7f66237fe640 1 --2- 172.21.15.169:0/3836114356 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f662406c650 0x7f66241f1470 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f6614006010 tx=0x7f6614003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:32.062 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.056+0000 7f661bfff640 1 -- 172.21.15.169:0/3836114356 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6614004450 con 0x7f662406c650 2023-10-26T01:43:32.063 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.056+0000 7f661bfff640 1 -- 172.21.15.169:0/3836114356 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f661402de90 con 0x7f662406c650 2023-10-26T01:43:32.063 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.056+0000 7f661bfff640 1 -- 172.21.15.169:0/3836114356 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6614003e10 con 0x7f662406c650 2023-10-26T01:43:32.063 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.056+0000 7f6629e56640 1 -- 172.21.15.169:0/3836114356 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66240cb020 con 0x7f662406c650 2023-10-26T01:43:32.063 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.056+0000 7f6629e56640 1 -- 172.21.15.169:0/3836114356 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f662414b640 con 0x7f662406c650 2023-10-26T01:43:32.063 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.057+0000 7f6629e56640 1 -- 172.21.15.169:0/3836114356 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6624066af0 con 0x7f662406c650 2023-10-26T01:43:32.064 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.058+0000 7f661bfff640 1 -- 172.21.15.169:0/3836114356 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f661400a040 con 0x7f662406c650 2023-10-26T01:43:32.064 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.059+0000 7f661bfff640 1 --2- 172.21.15.169:0/3836114356 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f660406ae40 0x7f660406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:32.064 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.059+0000 7f661bfff640 1 -- 172.21.15.169:0/3836114356 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f66140ac600 con 0x7f662406c650 2023-10-26T01:43:32.064 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.059+0000 7f661bfff640 1 -- 172.21.15.169:0/3836114356 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=171}) v3 -- 0x7f66040729d0 con 0x7f662406c650 2023-10-26T01:43:32.064 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.059+0000 7f6622ffd640 1 --2- 172.21.15.169:0/3836114356 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f660406ae40 0x7f660406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:32.065 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.060+0000 7f6622ffd640 1 --2- 172.21.15.169:0/3836114356 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f660406ae40 0x7f660406d300 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f660c0058c0 tx=0x7f660c005850 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:32.066 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.063+0000 7f661bfff640 1 -- 172.21.15.169:0/3836114356 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6614044060 con 0x7f662406c650 2023-10-26T01:43:32.257 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.255+0000 7f6629e56640 1 -- 172.21.15.169:0/3836114356 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f66240b1830 con 0x7f662406c650 2023-10-26T01:43:32.258 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.255+0000 7f661bfff640 1 -- 172.21.15.169:0/3836114356 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v170) v1 ==== 104+0+11 (secure 0 0 0) 0x7f66140390a0 con 0x7f662406c650 2023-10-26T01:43:32.258 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 32 2023-10-26T01:43:32.261 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.258+0000 7f6629e56640 1 -- 172.21.15.169:0/3836114356 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f660406ae40 msgr2=0x7f660406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:32.261 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.258+0000 7f6629e56640 1 --2- 172.21.15.169:0/3836114356 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f660406ae40 0x7f660406d300 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f660c0058c0 tx=0x7f660c005850 comp rx=0 tx=0).stop 2023-10-26T01:43:32.262 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.258+0000 7f6629e56640 1 -- 172.21.15.169:0/3836114356 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f662406c650 msgr2=0x7f66241f1470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:32.262 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.258+0000 7f6629e56640 1 --2- 172.21.15.169:0/3836114356 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f662406c650 0x7f66241f1470 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f6614006010 tx=0x7f6614003c20 comp rx=0 tx=0).stop 2023-10-26T01:43:32.262 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.258+0000 7f6629e56640 1 -- 172.21.15.169:0/3836114356 shutdown_connections 2023-10-26T01:43:32.262 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.258+0000 7f6629e56640 1 --2- 172.21.15.169:0/3836114356 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f660406ae40 0x7f660406d300 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:32.262 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.258+0000 7f6629e56640 1 --2- 172.21.15.169:0/3836114356 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f662415b320 0x7f66240ae240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:32.263 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.258+0000 7f6629e56640 1 --2- 172.21.15.169:0/3836114356 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f662414ddf0 0x7f66240c9cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:32.263 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.258+0000 7f6629e56640 1 --2- 172.21.15.169:0/3836114356 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f662406c650 0x7f66241f1470 unknown :-1 s=CLOSED pgs=1762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:32.263 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.258+0000 7f6629e56640 1 -- 172.21.15.169:0/3836114356 >> 172.21.15.169:0/3836114356 conn(0x7f66240aefe0 msgr2=0x7f66240acf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:32.263 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.259+0000 7f6629e56640 1 -- 172.21.15.169:0/3836114356 shutdown_connections 2023-10-26T01:43:32.263 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.259+0000 7f6629e56640 1 -- 172.21.15.169:0/3836114356 wait complete. 2023-10-26T01:43:32.281 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-26T01:43:32.499 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.496+0000 7fbd5935e640 1 -- 172.21.15.169:0/41092131 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbd5415b620 msgr2=0x7fbd540cd710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:32.499 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.496+0000 7fbd5935e640 1 --2- 172.21.15.169:0/41092131 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbd5415b620 0x7fbd540cd710 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7fbd44005fd0 tx=0x7fbd4402be20 comp rx=0 tx=0).stop 2023-10-26T01:43:32.499 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.496+0000 7fbd5935e640 1 -- 172.21.15.169:0/41092131 shutdown_connections 2023-10-26T01:43:32.499 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.496+0000 7fbd5935e640 1 --2- 172.21.15.169:0/41092131 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbd540c9d00 0x7fbd540cdc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:32.500 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.496+0000 7fbd5935e640 1 --2- 172.21.15.169:0/41092131 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbd5415b620 0x7fbd540cd710 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:32.500 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.496+0000 7fbd5935e640 1 -- 172.21.15.169:0/41092131 >> 172.21.15.169:0/41092131 conn(0x7fbd54069930 msgr2=0x7fbd54069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:32.500 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.496+0000 7fbd5935e640 1 -- 172.21.15.169:0/41092131 shutdown_connections 2023-10-26T01:43:32.500 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.496+0000 7fbd5935e640 1 -- 172.21.15.169:0/41092131 wait complete. 2023-10-26T01:43:32.503 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.497+0000 7fbd5935e640 1 Processor -- start 2023-10-26T01:43:32.503 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.498+0000 7fbd5935e640 1 -- start start 2023-10-26T01:43:32.503 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.499+0000 7fbd5935e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd540ac320 0x7fbd5414f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:32.503 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.499+0000 7fbd5935e640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbd540c9d00 0x7fbd540c6660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:32.504 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.499+0000 7fbd5935e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbd5415b620 0x7fbd540cc2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:32.504 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.499+0000 7fbd5935e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd540c6ba0 con 0x7fbd5415b620 2023-10-26T01:43:32.504 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.499+0000 7fbd5935e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd540c7390 con 0x7fbd540ac320 2023-10-26T01:43:32.504 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.499+0000 7fbd5935e640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd54157b80 con 0x7fbd540c9d00 2023-10-26T01:43:32.504 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.499+0000 7fbd537fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbd5415b620 0x7fbd540cc2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:32.505 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.499+0000 7fbd537fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbd5415b620 0x7fbd540cc2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:49688/0 (socket says 172.21.15.169:49688) 2023-10-26T01:43:32.505 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.499+0000 7fbd537fe640 1 -- 172.21.15.169:0/2899754871 learned_addr learned my addr 172.21.15.169:0/2899754871 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:43:32.505 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.499+0000 7fbd52ffd640 1 --2- 172.21.15.169:0/2899754871 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd540ac320 0x7fbd5414f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:32.505 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.499+0000 7fbd4bfff640 1 --2- 172.21.15.169:0/2899754871 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbd540c9d00 0x7fbd540c6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:32.505 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.499+0000 7fbd537fe640 1 -- 172.21.15.169:0/2899754871 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbd540c9d00 msgr2=0x7fbd540c6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:32.506 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.499+0000 7fbd537fe640 1 --2- 172.21.15.169:0/2899754871 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbd540c9d00 0x7fbd540c6660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:32.506 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.499+0000 7fbd537fe640 1 -- 172.21.15.169:0/2899754871 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd540ac320 msgr2=0x7fbd5414f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:32.506 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.499+0000 7fbd537fe640 1 --2- 172.21.15.169:0/2899754871 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd540ac320 0x7fbd5414f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:32.506 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.499+0000 7fbd537fe640 1 -- 172.21.15.169:0/2899754871 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd44009560 con 0x7fbd5415b620 2023-10-26T01:43:32.507 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.500+0000 7fbd537fe640 1 --2- 172.21.15.169:0/2899754871 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbd5415b620 0x7fbd540cc2b0 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c00c7a0 tx=0x7fbd4c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:32.507 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.500+0000 7fbd4b7fe640 1 -- 172.21.15.169:0/2899754871 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd4c003b70 con 0x7fbd5415b620 2023-10-26T01:43:32.507 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.500+0000 7fbd4b7fe640 1 -- 172.21.15.169:0/2899754871 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fbd4c003cf0 con 0x7fbd5415b620 2023-10-26T01:43:32.507 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.500+0000 7fbd5935e640 1 -- 172.21.15.169:0/2899754871 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd54157e60 con 0x7fbd5415b620 2023-10-26T01:43:32.508 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.500+0000 7fbd4b7fe640 1 -- 172.21.15.169:0/2899754871 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd4c00aab0 con 0x7fbd5415b620 2023-10-26T01:43:32.508 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.500+0000 7fbd5935e640 1 -- 172.21.15.169:0/2899754871 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd5415ade0 con 0x7fbd5415b620 2023-10-26T01:43:32.509 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.501+0000 7fbd5935e640 1 -- 172.21.15.169:0/2899754871 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd54158380 con 0x7fbd5415b620 2023-10-26T01:43:32.509 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.502+0000 7fbd4b7fe640 1 -- 172.21.15.169:0/2899754871 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fbd4c00ac10 con 0x7fbd5415b620 2023-10-26T01:43:32.509 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.503+0000 7fbd4b7fe640 1 --2- 172.21.15.169:0/2899754871 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbd3406ae40 0x7fbd3406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:32.509 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.503+0000 7fbd4b7fe640 1 -- 172.21.15.169:0/2899754871 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fbd4c0896f0 con 0x7fbd5415b620 2023-10-26T01:43:32.510 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.503+0000 7fbd4b7fe640 1 -- 172.21.15.169:0/2899754871 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=171}) v3 -- 0x7fbd340729d0 con 0x7fbd5415b620 2023-10-26T01:43:32.510 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.504+0000 7fbd52ffd640 1 --2- 172.21.15.169:0/2899754871 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbd3406ae40 0x7fbd3406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:32.510 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.505+0000 7fbd52ffd640 1 --2- 172.21.15.169:0/2899754871 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbd3406ae40 0x7fbd3406d300 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fbd44035000 tx=0x7fbd44004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:32.511 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.507+0000 7fbd4b7fe640 1 -- 172.21.15.169:0/2899754871 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbd4c0546e0 con 0x7fbd5415b620 2023-10-26T01:43:32.698 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.695+0000 7fbd5935e640 1 -- 172.21.15.169:0/2899754871 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbd5404db10 con 0x7fbd5415b620 2023-10-26T01:43:32.699 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.696+0000 7fbd4b7fe640 1 -- 172.21.15.169:0/2899754871 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v170) v1 ==== 100+0+11 (secure 0 0 0) 0x7fbd4c057d00 con 0x7fbd5415b620 2023-10-26T01:43:32.699 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 64 2023-10-26T01:43:32.702 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.699+0000 7fbd5935e640 1 -- 172.21.15.169:0/2899754871 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbd3406ae40 msgr2=0x7fbd3406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:32.702 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.699+0000 7fbd5935e640 1 --2- 172.21.15.169:0/2899754871 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbd3406ae40 0x7fbd3406d300 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fbd44035000 tx=0x7fbd44004730 comp rx=0 tx=0).stop 2023-10-26T01:43:32.702 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.699+0000 7fbd5935e640 1 -- 172.21.15.169:0/2899754871 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbd5415b620 msgr2=0x7fbd540cc2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:32.702 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.699+0000 7fbd5935e640 1 --2- 172.21.15.169:0/2899754871 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbd5415b620 0x7fbd540cc2b0 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c00c7a0 tx=0x7fbd4c00cc60 comp rx=0 tx=0).stop 2023-10-26T01:43:32.703 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.699+0000 7fbd5935e640 1 -- 172.21.15.169:0/2899754871 shutdown_connections 2023-10-26T01:43:32.703 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.699+0000 7fbd5935e640 1 --2- 172.21.15.169:0/2899754871 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbd3406ae40 0x7fbd3406d300 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:32.703 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.699+0000 7fbd5935e640 1 --2- 172.21.15.169:0/2899754871 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbd5415b620 0x7fbd540cc2b0 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:32.703 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.699+0000 7fbd5935e640 1 --2- 172.21.15.169:0/2899754871 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbd540c9d00 0x7fbd540c6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:32.703 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.699+0000 7fbd5935e640 1 --2- 172.21.15.169:0/2899754871 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd540ac320 0x7fbd5414f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:32.704 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.699+0000 7fbd5935e640 1 -- 172.21.15.169:0/2899754871 >> 172.21.15.169:0/2899754871 conn(0x7fbd54069930 msgr2=0x7fbd540aef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:32.704 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.699+0000 7fbd5935e640 1 -- 172.21.15.169:0/2899754871 shutdown_connections 2023-10-26T01:43:32.704 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.699+0000 7fbd5935e640 1 -- 172.21.15.169:0/2899754871 wait complete. 2023-10-26T01:43:32.721 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T01:43:32.932 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.929+0000 7f29b53a9640 1 -- 172.21.15.169:0/3193953996 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29b01592d0 msgr2=0x7f29b00aa3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:32.932 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.929+0000 7f29b53a9640 1 --2- 172.21.15.169:0/3193953996 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29b01592d0 0x7f29b00aa3d0 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f29a0005fd0 tx=0x7f29a002bcf0 comp rx=0 tx=0).stop 2023-10-26T01:43:32.933 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.929+0000 7f29b53a9640 1 -- 172.21.15.169:0/3193953996 shutdown_connections 2023-10-26T01:43:32.933 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.929+0000 7f29b53a9640 1 --2- 172.21.15.169:0/3193953996 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f29b00ac930 0x7f29b00cd6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:32.933 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.929+0000 7f29b53a9640 1 --2- 172.21.15.169:0/3193953996 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29b01592d0 0x7f29b00aa3d0 unknown :-1 s=CLOSED pgs=1766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:32.933 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.929+0000 7f29b53a9640 1 -- 172.21.15.169:0/3193953996 >> 172.21.15.169:0/3193953996 conn(0x7f29b0069930 msgr2=0x7f29b0069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:32.934 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.929+0000 7f29b53a9640 1 -- 172.21.15.169:0/3193953996 shutdown_connections 2023-10-26T01:43:32.934 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.929+0000 7f29b53a9640 1 -- 172.21.15.169:0/3193953996 wait complete. 2023-10-26T01:43:32.935 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.931+0000 7f29b53a9640 1 Processor -- start 2023-10-26T01:43:32.936 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.931+0000 7f29b53a9640 1 -- start start 2023-10-26T01:43:32.936 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.932+0000 7f29b53a9640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29b00ac930 0x7f29b015aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:32.936 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.932+0000 7f29b53a9640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f29b00cdc30 0x7f29b01f13b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:32.936 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.932+0000 7f29b53a9640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f29b01592d0 0x7f29b00c6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:32.936 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.932+0000 7f29b53a9640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29b01f18f0 con 0x7f29b00ac930 2023-10-26T01:43:32.937 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.932+0000 7f29b53a9640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29b015b150 con 0x7f29b00cdc30 2023-10-26T01:43:32.937 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.932+0000 7f29b53a9640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29b015b430 con 0x7f29b01592d0 2023-10-26T01:43:32.937 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.932+0000 7f29aeffd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29b00ac930 0x7f29b015aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:32.937 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.932+0000 7f29aeffd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29b00ac930 0x7f29b015aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:49698/0 (socket says 172.21.15.169:49698) 2023-10-26T01:43:32.938 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.932+0000 7f29aeffd640 1 -- 172.21.15.169:0/4106064604 learned_addr learned my addr 172.21.15.169:0/4106064604 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:43:32.938 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.933+0000 7f29a7fff640 1 --2- 172.21.15.169:0/4106064604 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f29b00cdc30 0x7f29b01f13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:32.938 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.933+0000 7f29aeffd640 1 -- 172.21.15.169:0/4106064604 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f29b01592d0 msgr2=0x7f29b00c6aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:32.938 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.933+0000 7f29aeffd640 1 --2- 172.21.15.169:0/4106064604 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f29b01592d0 0x7f29b00c6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:32.938 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.933+0000 7f29aeffd640 1 -- 172.21.15.169:0/4106064604 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f29b00cdc30 msgr2=0x7f29b01f13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:32.939 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.933+0000 7f29aeffd640 1 --2- 172.21.15.169:0/4106064604 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f29b00cdc30 0x7f29b01f13b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:32.939 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.933+0000 7f29aeffd640 1 -- 172.21.15.169:0/4106064604 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29a0009560 con 0x7f29b00ac930 2023-10-26T01:43:32.939 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.933+0000 7f29a7fff640 1 --2- 172.21.15.169:0/4106064604 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f29b00cdc30 0x7f29b01f13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:43:32.939 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.933+0000 7f29aeffd640 1 --2- 172.21.15.169:0/4106064604 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29b00ac930 0x7f29b015aa70 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f29a0006010 tx=0x7f29a002dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:32.941 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.933+0000 7f29a77fe640 1 -- 172.21.15.169:0/4106064604 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a0005650 con 0x7f29b00ac930 2023-10-26T01:43:32.941 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.933+0000 7f29a77fe640 1 -- 172.21.15.169:0/4106064604 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f29a0003800 con 0x7f29b00ac930 2023-10-26T01:43:32.941 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.933+0000 7f29b53a9640 1 -- 172.21.15.169:0/4106064604 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29b015b710 con 0x7f29b00ac930 2023-10-26T01:43:32.942 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.933+0000 7f29a77fe640 1 -- 172.21.15.169:0/4106064604 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a0003c10 con 0x7f29b00ac930 2023-10-26T01:43:32.942 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.933+0000 7f29b53a9640 1 -- 172.21.15.169:0/4106064604 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29b015bcd0 con 0x7f29b00ac930 2023-10-26T01:43:32.942 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.935+0000 7f29b53a9640 1 -- 172.21.15.169:0/4106064604 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29b004db10 con 0x7f29b00ac930 2023-10-26T01:43:32.942 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.936+0000 7f29a77fe640 1 -- 172.21.15.169:0/4106064604 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f29a003e450 con 0x7f29b00ac930 2023-10-26T01:43:32.943 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.936+0000 7f29a77fe640 1 --2- 172.21.15.169:0/4106064604 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f299006ae40 0x7f299006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:32.943 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.937+0000 7f29a77fe640 1 -- 172.21.15.169:0/4106064604 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f29a00ac160 con 0x7f29b00ac930 2023-10-26T01:43:32.943 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.937+0000 7f29a77fe640 1 -- 172.21.15.169:0/4106064604 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=171}) v3 -- 0x7f29900729d0 con 0x7f29b00ac930 2023-10-26T01:43:32.943 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.937+0000 7f29a7fff640 1 --2- 172.21.15.169:0/4106064604 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f299006ae40 0x7f299006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:32.943 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.938+0000 7f29a7fff640 1 --2- 172.21.15.169:0/4106064604 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f299006ae40 0x7f299006d300 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f29b0067420 tx=0x7f2998005d30 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:32.944 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:32.941+0000 7f29a77fe640 1 -- 172.21.15.169:0/4106064604 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f29a0077330 con 0x7f29b00ac930 2023-10-26T01:43:33.153 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:33.151+0000 7f29b53a9640 1 -- 172.21.15.169:0/4106064604 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f29b00ce080 con 0x7f29b00ac930 2023-10-26T01:43:33.154 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:33.152+0000 7f29a77fe640 1 -- 172.21.15.169:0/4106064604 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) v1 ==== 94+0+4923 (secure 0 0 0) 0x7f29b00ce080 con 0x7f29b00ac930 2023-10-26T01:43:33.155 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:43:33.155 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":86,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5861,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/326391134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":326391134},{"type":"v1","addr":"172.21.15.170:6836","nonce":326391134}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":5864,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/3374486280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":3374486280},{"type":"v1","addr":"172.21.15.170:6837","nonce":3374486280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":8697,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/1445776478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":1445776478},{"type":"v1","addr":"172.21.15.169:6834","nonce":1445776478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T01:42:09.853788+0000","modified":"2023-10-26T01:42:12.922104+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8700},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8700":{"gid":8700,"name":"a","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6835/228678953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":228678953},{"type":"v1","addr":"172.21.15.169:6835","nonce":228678953}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-10-26T01:43:33.155 INFO:teuthology.orchestra.run.smithi169.stderr:dumped fsmap epoch 86 2023-10-26T01:43:33.157 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:33.154+0000 7f29b53a9640 1 -- 172.21.15.169:0/4106064604 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f299006ae40 msgr2=0x7f299006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:33.157 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:33.154+0000 7f29b53a9640 1 --2- 172.21.15.169:0/4106064604 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f299006ae40 0x7f299006d300 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f29b0067420 tx=0x7f2998005d30 comp rx=0 tx=0).stop 2023-10-26T01:43:33.158 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:33.154+0000 7f29b53a9640 1 -- 172.21.15.169:0/4106064604 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29b00ac930 msgr2=0x7f29b015aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:33.158 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:33.154+0000 7f29b53a9640 1 --2- 172.21.15.169:0/4106064604 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29b00ac930 0x7f29b015aa70 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f29a0006010 tx=0x7f29a002dcc0 comp rx=0 tx=0).stop 2023-10-26T01:43:33.158 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:33.154+0000 7f29b53a9640 1 -- 172.21.15.169:0/4106064604 shutdown_connections 2023-10-26T01:43:33.158 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:33.154+0000 7f29b53a9640 1 --2- 172.21.15.169:0/4106064604 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f299006ae40 0x7f299006d300 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:33.158 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:33.154+0000 7f29b53a9640 1 --2- 172.21.15.169:0/4106064604 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f29b01592d0 0x7f29b00c6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:33.159 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:33.154+0000 7f29b53a9640 1 --2- 172.21.15.169:0/4106064604 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f29b00cdc30 0x7f29b01f13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:33.159 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:33.154+0000 7f29b53a9640 1 --2- 172.21.15.169:0/4106064604 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29b00ac930 0x7f29b015aa70 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:33.159 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:33.154+0000 7f29b53a9640 1 -- 172.21.15.169:0/4106064604 >> 172.21.15.169:0/4106064604 conn(0x7f29b0069930 msgr2=0x7f29b0155f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:33.159 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:33.155+0000 7f29b53a9640 1 -- 172.21.15.169:0/4106064604 shutdown_connections 2023-10-26T01:43:33.159 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:33.155+0000 7f29b53a9640 1 -- 172.21.15.169:0/4106064604 wait complete. 2023-10-26T01:43:33.176 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:43:33.403 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:33.396+0000 7f804775e640 1 -- 172.21.15.169:0/3330818832 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f802c0047c0 msgr2=0x7f802c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:33.403 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:33.396+0000 7f804775e640 1 --2- 172.21.15.169:0/3330818832 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f802c0047c0 0x7f802c004c40 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f8030006ed0 tx=0x7f802c0057f0 comp rx=0 tx=0).stop 2023-10-26T01:43:33.403 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:33.397+0000 7f804775e640 1 -- 172.21.15.169:0/3330818832 shutdown_connections 2023-10-26T01:43:33.404 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:33.397+0000 7f804775e640 1 --2- 172.21.15.169:0/3330818832 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f802c0047c0 0x7f802c004c40 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:33.404 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:33.397+0000 7f804775e640 1 -- 172.21.15.169:0/3330818832 >> 172.21.15.169:0/3330818832 conn(0x7f8040069930 msgr2=0x7f8040069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:33.404 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:33.397+0000 7f804775e640 1 -- 172.21.15.169:0/3330818832 shutdown_connections 2023-10-26T01:43:33.404 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:33.397+0000 7f804775e640 1 -- 172.21.15.169:0/3330818832 wait complete. 2023-10-26T01:43:33.404 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:33.398+0000 7f804775e640 1 Processor -- start 2023-10-26T01:43:33.405 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:33.399+0000 7f804775e640 1 -- start start 2023-10-26T01:43:33.405 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:33.400+0000 7f804775e640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f802c0047c0 0x7f804014f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:33.405 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:33.400+0000 7f804775e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8040153fd0 0x7f80400cba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:33.405 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:33.400+0000 7f804775e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f804009ad40 0x7f80400cbf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:33.405 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:33.400+0000 7f804775e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f804014fb00 con 0x7f8040153fd0 2023-10-26T01:43:33.406 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:33.400+0000 7f804775e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f804006ed30 con 0x7f804009ad40 2023-10-26T01:43:33.406 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:33.400+0000 7f804775e640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8040151bc0 con 0x7f802c0047c0 2023-10-26T01:43:33.406 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:33.400+0000 7f8044cf9640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8040153fd0 0x7f80400cba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:33.406 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:33.400+0000 7f8044cf9640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8040153fd0 0x7f80400cba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:49706/0 (socket says 172.21.15.169:49706) 2023-10-26T01:43:33.406 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:33.400+0000 7f8044cf9640 1 -- 172.21.15.169:0/3965900818 learned_addr learned my addr 172.21.15.169:0/3965900818 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:43:33.407 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:33.400+0000 7f8044cf9640 1 -- 172.21.15.169:0/3965900818 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f802c0047c0 msgr2=0x7f804014f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:33.407 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:33.400+0000 7f8044cf9640 1 --2- 172.21.15.169:0/3965900818 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f802c0047c0 0x7f804014f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:33.407 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:33.400+0000 7f8044cf9640 1 -- 172.21.15.169:0/3965900818 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f804009ad40 msgr2=0x7f80400cbf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:33.407 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:33.400+0000 7f8045cfb640 1 --2- 172.21.15.169:0/3965900818 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f804009ad40 0x7f80400cbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:33.407 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:33.400+0000 7f8044cf9640 1 --2- 172.21.15.169:0/3965900818 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f804009ad40 0x7f80400cbf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:33.408 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:33.400+0000 7f8044cf9640 1 -- 172.21.15.169:0/3965900818 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8040151ea0 con 0x7f8040153fd0 2023-10-26T01:43:33.409 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:33.401+0000 7f8044cf9640 1 --2- 172.21.15.169:0/3965900818 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8040153fd0 0x7f80400cba30 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f803002c700 tx=0x7f8030001720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:33.409 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:33.401+0000 7f8035ffb640 1 -- 172.21.15.169:0/3965900818 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f803003c040 con 0x7f8040153fd0 2023-10-26T01:43:33.410 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:33.401+0000 7f8035ffb640 1 -- 172.21.15.169:0/3965900818 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f80300037f0 con 0x7f8040153fd0 2023-10-26T01:43:33.410 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:33.401+0000 7f8035ffb640 1 -- 172.21.15.169:0/3965900818 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8030003be0 con 0x7f8040153fd0 2023-10-26T01:43:33.410 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:33.401+0000 7f804775e640 1 -- 172.21.15.169:0/3965900818 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f804006efd0 con 0x7f8040153fd0 2023-10-26T01:43:33.410 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:33.401+0000 7f804775e640 1 -- 172.21.15.169:0/3965900818 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8040151870 con 0x7f8040153fd0 2023-10-26T01:43:33.410 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:33.402+0000 7f804775e640 1 -- 172.21.15.169:0/3965900818 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f804004db10 con 0x7f8040153fd0 2023-10-26T01:43:33.411 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:33.403+0000 7f8035ffb640 1 -- 172.21.15.169:0/3965900818 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f8030032070 con 0x7f8040153fd0 2023-10-26T01:43:33.411 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:33.405+0000 7f8035ffb640 1 --2- 172.21.15.169:0/3965900818 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f802406ae70 0x7f802406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:33.411 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:33.405+0000 7f8035ffb640 1 -- 172.21.15.169:0/3965900818 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f80300aab10 con 0x7f8040153fd0 2023-10-26T01:43:33.411 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:33.405+0000 7f8035ffb640 1 -- 172.21.15.169:0/3965900818 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=171}) v3 -- 0x7f8024072a00 con 0x7f8040153fd0 2023-10-26T01:43:33.411 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:33.405+0000 7f80454fa640 1 --2- 172.21.15.169:0/3965900818 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f802406ae70 0x7f802406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:33.412 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:33.406+0000 7f80454fa640 1 --2- 172.21.15.169:0/3965900818 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f802406ae70 0x7f802406d330 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f803c001620 tx=0x7f803c006920 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:33.412 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:33.409+0000 7f8035ffb640 1 -- 172.21.15.169:0/3965900818 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8030046070 con 0x7f8040153fd0 2023-10-26T01:43:33.602 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:33.599+0000 7f804775e640 1 -- 172.21.15.169:0/3965900818 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8040152030 con 0x7f8040153fd0 2023-10-26T01:43:33.605 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:33.602+0000 7f8035ffb640 1 -- 172.21.15.169:0/3965900818 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v170) v1 ==== 74+0+16704 (secure 0 0 0) 0x7f8030075b00 con 0x7f8040153fd0 2023-10-26T01:43:33.606 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:43:33.606 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":170,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:42:58.161990+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2023-10-26T01:42:07.656531+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"165","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2023-10-26T01:42:08.866635+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"170","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.39","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:43:33.607 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:33.605+0000 7f804775e640 1 -- 172.21.15.169:0/3965900818 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f802406ae70 msgr2=0x7f802406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:33.608 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:33.605+0000 7f804775e640 1 --2- 172.21.15.169:0/3965900818 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f802406ae70 0x7f802406d330 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f803c001620 tx=0x7f803c006920 comp rx=0 tx=0).stop 2023-10-26T01:43:33.608 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:33.605+0000 7f804775e640 1 -- 172.21.15.169:0/3965900818 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8040153fd0 msgr2=0x7f80400cba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:33.608 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:33.605+0000 7f804775e640 1 --2- 172.21.15.169:0/3965900818 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8040153fd0 0x7f80400cba30 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f803002c700 tx=0x7f8030001720 comp rx=0 tx=0).stop 2023-10-26T01:43:33.609 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:33.605+0000 7f804775e640 1 -- 172.21.15.169:0/3965900818 shutdown_connections 2023-10-26T01:43:33.609 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:33.605+0000 7f804775e640 1 --2- 172.21.15.169:0/3965900818 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f802406ae70 0x7f802406d330 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:33.609 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:33.605+0000 7f804775e640 1 --2- 172.21.15.169:0/3965900818 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8040153fd0 0x7f80400cba30 unknown :-1 s=CLOSED pgs=1770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:33.609 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:33.605+0000 7f804775e640 1 --2- 172.21.15.169:0/3965900818 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f804009ad40 0x7f80400cbf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:33.610 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:33.605+0000 7f804775e640 1 --2- 172.21.15.169:0/3965900818 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f802c0047c0 0x7f804014f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:33.610 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:33.605+0000 7f804775e640 1 -- 172.21.15.169:0/3965900818 >> 172.21.15.169:0/3965900818 conn(0x7f8040069930 msgr2=0x7f80400adf40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:33.610 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:33.605+0000 7f804775e640 1 -- 172.21.15.169:0/3965900818 shutdown_connections 2023-10-26T01:43:33.610 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:33.605+0000 7f804775e640 1 -- 172.21.15.169:0/3965900818 wait complete. 2023-10-26T01:43:33.626 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-26T01:43:33.627 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T01:43:33.841 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:33.838+0000 7efc58124640 1 -- 172.21.15.169:0/3156456149 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efc5006b740 msgr2=0x7efc5015e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:33.842 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:33.838+0000 7efc58124640 1 --2- 172.21.15.169:0/3156456149 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efc5006b740 0x7efc5015e550 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7efc48005fd0 tx=0x7efc4802c260 comp rx=0 tx=0).stop 2023-10-26T01:43:33.842 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:33.838+0000 7efc58124640 1 -- 172.21.15.169:0/3156456149 shutdown_connections 2023-10-26T01:43:33.842 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:33.838+0000 7efc58124640 1 --2- 172.21.15.169:0/3156456149 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efc5006b740 0x7efc5015e550 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:33.842 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:33.838+0000 7efc58124640 1 -- 172.21.15.169:0/3156456149 >> 172.21.15.169:0/3156456149 conn(0x7efc50098a60 msgr2=0x7efc500b4a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:33.843 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:33.838+0000 7efc58124640 1 -- 172.21.15.169:0/3156456149 shutdown_connections 2023-10-26T01:43:33.843 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:33.839+0000 7efc58124640 1 -- 172.21.15.169:0/3156456149 wait complete. 2023-10-26T01:43:33.844 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:33.840+0000 7efc58124640 1 Processor -- start 2023-10-26T01:43:33.844 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:33.840+0000 7efc58124640 1 -- start start 2023-10-26T01:43:33.844 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:33.841+0000 7efc58124640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efc5006b740 0x7efc50148dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:33.844 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:33.841+0000 7efc58124640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7efc500b2c60 0x7efc50156450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:33.845 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:33.841+0000 7efc58124640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efc50155d90 0x7efc50152e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:33.845 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:33.841+0000 7efc58124640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc501581f0 con 0x7efc50155d90 2023-10-26T01:43:33.845 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:33.841+0000 7efc58124640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc501523c0 con 0x7efc5006b740 2023-10-26T01:43:33.845 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:33.841+0000 7efc58124640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc50156e80 con 0x7efc500b2c60 2023-10-26T01:43:33.847 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:33.842+0000 7efc566c1640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efc50155d90 0x7efc50152e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:33.847 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:33.842+0000 7efc566c1640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efc50155d90 0x7efc50152e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:49712/0 (socket says 172.21.15.169:49712) 2023-10-26T01:43:33.847 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:33.842+0000 7efc566c1640 1 -- 172.21.15.169:0/1988584634 learned_addr learned my addr 172.21.15.169:0/1988584634 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:43:33.847 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:33.842+0000 7efc55ec0640 1 --2- 172.21.15.169:0/1988584634 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efc5006b740 0x7efc50148dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:33.848 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:33.842+0000 7efc556bf640 1 --2- 172.21.15.169:0/1988584634 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7efc500b2c60 0x7efc50156450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:33.848 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:33.842+0000 7efc566c1640 1 -- 172.21.15.169:0/1988584634 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7efc500b2c60 msgr2=0x7efc50156450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:33.848 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:33.842+0000 7efc566c1640 1 --2- 172.21.15.169:0/1988584634 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7efc500b2c60 0x7efc50156450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:33.848 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:33.842+0000 7efc566c1640 1 -- 172.21.15.169:0/1988584634 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efc5006b740 msgr2=0x7efc50148dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:33.848 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:33.842+0000 7efc566c1640 1 --2- 172.21.15.169:0/1988584634 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efc5006b740 0x7efc50148dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:33.849 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:33.842+0000 7efc566c1640 1 -- 172.21.15.169:0/1988584634 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc48009900 con 0x7efc50155d90 2023-10-26T01:43:33.849 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:33.842+0000 7efc566c1640 1 --2- 172.21.15.169:0/1988584634 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efc50155d90 0x7efc50152e60 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7efc48009bf0 tx=0x7efc48003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:33.849 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:33.843+0000 7efc467fc640 1 -- 172.21.15.169:0/1988584634 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc48004450 con 0x7efc50155d90 2023-10-26T01:43:33.849 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:33.843+0000 7efc467fc640 1 -- 172.21.15.169:0/1988584634 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7efc480046a0 con 0x7efc50155d90 2023-10-26T01:43:33.849 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:33.843+0000 7efc467fc640 1 -- 172.21.15.169:0/1988584634 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc480406b0 con 0x7efc50155d90 2023-10-26T01:43:33.849 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:33.843+0000 7efc58124640 1 -- 172.21.15.169:0/1988584634 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc50152660 con 0x7efc50155d90 2023-10-26T01:43:33.850 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:33.843+0000 7efc58124640 1 -- 172.21.15.169:0/1988584634 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc50157160 con 0x7efc50155d90 2023-10-26T01:43:33.850 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:33.844+0000 7efc58124640 1 -- 172.21.15.169:0/1988584634 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc50152b80 con 0x7efc50155d90 2023-10-26T01:43:33.851 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:33.845+0000 7efc467fc640 1 -- 172.21.15.169:0/1988584634 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7efc4800a040 con 0x7efc50155d90 2023-10-26T01:43:33.851 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:33.846+0000 7efc467fc640 1 --2- 172.21.15.169:0/1988584634 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7efc3406ae40 0x7efc3406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:33.851 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:33.846+0000 7efc467fc640 1 -- 172.21.15.169:0/1988584634 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 5738+0+0 (secure 0 0 0) 0x7efc480ac750 con 0x7efc50155d90 2023-10-26T01:43:33.851 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:33.847+0000 7efc467fc640 1 -- 172.21.15.169:0/1988584634 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=171}) v3 -- 0x7efc340729d0 con 0x7efc50155d90 2023-10-26T01:43:33.852 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:33.847+0000 7efc55ec0640 1 --2- 172.21.15.169:0/1988584634 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7efc3406ae40 0x7efc3406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:33.852 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:33.848+0000 7efc55ec0640 1 --2- 172.21.15.169:0/1988584634 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7efc3406ae40 0x7efc3406d300 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7efc4c006fd0 tx=0x7efc4c008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:33.854 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:33.851+0000 7efc467fc640 1 -- 172.21.15.169:0/1988584634 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7efc48039060 con 0x7efc50155d90 2023-10-26T01:43:34.081 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.078+0000 7efc58124640 1 -- 172.21.15.169:0/1988584634 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efc50071be0 con 0x7efc50155d90 2023-10-26T01:43:34.082 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.079+0000 7efc467fc640 1 -- 172.21.15.169:0/1988584634 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) v1 ==== 94+0+4923 (secure 0 0 0) 0x7efc48044020 con 0x7efc50155d90 2023-10-26T01:43:34.083 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:43:34.083 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":86,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5861,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/326391134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":326391134},{"type":"v1","addr":"172.21.15.170:6836","nonce":326391134}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":5864,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/3374486280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":3374486280},{"type":"v1","addr":"172.21.15.170:6837","nonce":3374486280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":8697,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/1445776478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":1445776478},{"type":"v1","addr":"172.21.15.169:6834","nonce":1445776478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T01:42:09.853788+0000","modified":"2023-10-26T01:42:12.922104+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8700},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8700":{"gid":8700,"name":"a","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6835/228678953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":228678953},{"type":"v1","addr":"172.21.15.169:6835","nonce":228678953}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-10-26T01:43:34.083 INFO:teuthology.orchestra.run.smithi169.stderr:dumped fsmap epoch 86 2023-10-26T01:43:34.085 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.082+0000 7efc58124640 1 -- 172.21.15.169:0/1988584634 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7efc3406ae40 msgr2=0x7efc3406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:34.085 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.082+0000 7efc58124640 1 --2- 172.21.15.169:0/1988584634 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7efc3406ae40 0x7efc3406d300 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7efc4c006fd0 tx=0x7efc4c008040 comp rx=0 tx=0).stop 2023-10-26T01:43:34.085 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.082+0000 7efc58124640 1 -- 172.21.15.169:0/1988584634 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efc50155d90 msgr2=0x7efc50152e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:34.085 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.082+0000 7efc58124640 1 --2- 172.21.15.169:0/1988584634 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efc50155d90 0x7efc50152e60 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7efc48009bf0 tx=0x7efc48003c20 comp rx=0 tx=0).stop 2023-10-26T01:43:34.086 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.082+0000 7efc58124640 1 -- 172.21.15.169:0/1988584634 shutdown_connections 2023-10-26T01:43:34.086 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.082+0000 7efc58124640 1 --2- 172.21.15.169:0/1988584634 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7efc3406ae40 0x7efc3406d300 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:34.086 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.082+0000 7efc58124640 1 --2- 172.21.15.169:0/1988584634 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efc50155d90 0x7efc50152e60 unknown :-1 s=CLOSED pgs=1773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:34.087 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.082+0000 7efc58124640 1 --2- 172.21.15.169:0/1988584634 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7efc500b2c60 0x7efc50156450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:34.087 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.082+0000 7efc58124640 1 --2- 172.21.15.169:0/1988584634 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efc5006b740 0x7efc50148dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:34.087 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.082+0000 7efc58124640 1 -- 172.21.15.169:0/1988584634 >> 172.21.15.169:0/1988584634 conn(0x7efc50098a60 msgr2=0x7efc5015ef30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:34.088 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.083+0000 7efc58124640 1 -- 172.21.15.169:0/1988584634 shutdown_connections 2023-10-26T01:43:34.088 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.083+0000 7efc58124640 1 -- 172.21.15.169:0/1988584634 wait complete. 2023-10-26T01:43:34.103 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T01:43:34.309 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.306+0000 7fac6bff5640 1 -- 172.21.15.169:0/673210276 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fac640cde00 msgr2=0x7fac64164040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:34.310 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.306+0000 7fac6bff5640 1 --2- 172.21.15.169:0/673210276 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fac640cde00 0x7fac64164040 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7fac5c005fd0 tx=0x7fac5c02c060 comp rx=0 tx=0).stop 2023-10-26T01:43:34.310 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.306+0000 7fac6bff5640 1 -- 172.21.15.169:0/673210276 shutdown_connections 2023-10-26T01:43:34.310 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.306+0000 7fac6bff5640 1 --2- 172.21.15.169:0/673210276 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fac640cde00 0x7fac64164040 unknown :-1 s=CLOSED pgs=1775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:34.311 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.306+0000 7fac6bff5640 1 -- 172.21.15.169:0/673210276 >> 172.21.15.169:0/673210276 conn(0x7fac64069930 msgr2=0x7fac64069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:34.311 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.307+0000 7fac6bff5640 1 -- 172.21.15.169:0/673210276 shutdown_connections 2023-10-26T01:43:34.311 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.307+0000 7fac6bff5640 1 -- 172.21.15.169:0/673210276 wait complete. 2023-10-26T01:43:34.314 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.308+0000 7fac6bff5640 1 Processor -- start 2023-10-26T01:43:34.314 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.308+0000 7fac6bff5640 1 -- start start 2023-10-26T01:43:34.314 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.309+0000 7fac6bff5640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fac640ac930 0x7fac6406ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:34.314 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.309+0000 7fac6bff5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fac640cde00 0x7fac6415c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:34.314 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.309+0000 7fac6bff5640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fac641592d0 0x7fac64144430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:34.315 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.309+0000 7fac6bff5640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac640c76e0 con 0x7fac641592d0 2023-10-26T01:43:34.315 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.309+0000 7fac6bff5640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac640c6a60 con 0x7fac640cde00 2023-10-26T01:43:34.315 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.309+0000 7fac6bff5640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac640c6d40 con 0x7fac640ac930 2023-10-26T01:43:34.315 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.310+0000 7fac6a592640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fac641592d0 0x7fac64144430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:34.316 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.310+0000 7fac6a592640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fac641592d0 0x7fac64144430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:35682/0 (socket says 172.21.15.169:35682) 2023-10-26T01:43:34.316 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.310+0000 7fac6a592640 1 -- 172.21.15.169:0/3185282273 learned_addr learned my addr 172.21.15.169:0/3185282273 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:43:34.316 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.310+0000 7fac69d91640 1 --2- 172.21.15.169:0/3185282273 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fac640ac930 0x7fac6406ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:34.316 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.310+0000 7fac6a592640 1 -- 172.21.15.169:0/3185282273 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fac640ac930 msgr2=0x7fac6406ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:34.316 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.310+0000 7fac6a592640 1 --2- 172.21.15.169:0/3185282273 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fac640ac930 0x7fac6406ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:34.317 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.310+0000 7fac6a592640 1 -- 172.21.15.169:0/3185282273 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fac640cde00 msgr2=0x7fac6415c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:34.317 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.310+0000 7fac69590640 1 --2- 172.21.15.169:0/3185282273 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fac640cde00 0x7fac6415c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:34.317 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.310+0000 7fac6a592640 1 --2- 172.21.15.169:0/3185282273 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fac640cde00 0x7fac6415c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:34.317 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.310+0000 7fac6a592640 1 -- 172.21.15.169:0/3185282273 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac5c009840 con 0x7fac641592d0 2023-10-26T01:43:34.318 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.310+0000 7fac69590640 1 --2- 172.21.15.169:0/3185282273 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fac640cde00 0x7fac6415c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:43:34.318 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.310+0000 7fac6a592640 1 --2- 172.21.15.169:0/3185282273 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fac641592d0 0x7fac64144430 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7fac5c036040 tx=0x7fac5c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:34.318 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.311+0000 7fac5a7fc640 1 -- 172.21.15.169:0/3185282273 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac5c004870 con 0x7fac641592d0 2023-10-26T01:43:34.318 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.311+0000 7fac5a7fc640 1 -- 172.21.15.169:0/3185282273 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fac5c03d540 con 0x7fac641592d0 2023-10-26T01:43:34.318 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.311+0000 7fac5a7fc640 1 -- 172.21.15.169:0/3185282273 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac5c03d950 con 0x7fac641592d0 2023-10-26T01:43:34.319 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.311+0000 7fac6bff5640 1 -- 172.21.15.169:0/3185282273 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac640c7020 con 0x7fac641592d0 2023-10-26T01:43:34.319 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.311+0000 7fac6bff5640 1 -- 172.21.15.169:0/3185282273 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac6415c770 con 0x7fac641592d0 2023-10-26T01:43:34.320 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.312+0000 7fac6bff5640 1 -- 172.21.15.169:0/3185282273 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac6415c900 con 0x7fac641592d0 2023-10-26T01:43:34.320 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.313+0000 7fac5a7fc640 1 -- 172.21.15.169:0/3185282273 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fac5c03dab0 con 0x7fac641592d0 2023-10-26T01:43:34.321 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.314+0000 7fac5a7fc640 1 --2- 172.21.15.169:0/3185282273 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fac4806ae40 0x7fac4806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:34.321 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.314+0000 7fac5a7fc640 1 -- 172.21.15.169:0/3185282273 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fac5c0ac800 con 0x7fac641592d0 2023-10-26T01:43:34.321 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.314+0000 7fac5a7fc640 1 -- 172.21.15.169:0/3185282273 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=171}) v3 -- 0x7fac480729d0 con 0x7fac641592d0 2023-10-26T01:43:34.321 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.315+0000 7fac69d91640 1 --2- 172.21.15.169:0/3185282273 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fac4806ae40 0x7fac4806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:34.321 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.315+0000 7fac69d91640 1 --2- 172.21.15.169:0/3185282273 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fac4806ae40 0x7fac4806d300 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fac60007fd0 tx=0x7fac60009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:34.322 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.318+0000 7fac5a7fc640 1 -- 172.21.15.169:0/3185282273 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fac5c041060 con 0x7fac641592d0 2023-10-26T01:43:34.549 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.546+0000 7fac6bff5640 1 -- 172.21.15.169:0/3185282273 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fac640c7820 con 0x7fac641592d0 2023-10-26T01:43:34.550 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.548+0000 7fac5a7fc640 1 -- 172.21.15.169:0/3185282273 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) v1 ==== 94+0+4923 (secure 0 0 0) 0x7fac5c04b020 con 0x7fac641592d0 2023-10-26T01:43:34.551 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:43:34.551 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":86,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5861,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/326391134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":326391134},{"type":"v1","addr":"172.21.15.170:6836","nonce":326391134}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":5864,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/3374486280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":3374486280},{"type":"v1","addr":"172.21.15.170:6837","nonce":3374486280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":8697,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/1445776478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":1445776478},{"type":"v1","addr":"172.21.15.169:6834","nonce":1445776478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T01:42:09.853788+0000","modified":"2023-10-26T01:42:12.922104+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8700},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8700":{"gid":8700,"name":"a","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6835/228678953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":228678953},{"type":"v1","addr":"172.21.15.169:6835","nonce":228678953}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-10-26T01:43:34.551 INFO:teuthology.orchestra.run.smithi169.stderr:dumped fsmap epoch 86 2023-10-26T01:43:34.553 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.550+0000 7fac6bff5640 1 -- 172.21.15.169:0/3185282273 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fac4806ae40 msgr2=0x7fac4806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:34.553 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.550+0000 7fac6bff5640 1 --2- 172.21.15.169:0/3185282273 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fac4806ae40 0x7fac4806d300 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fac60007fd0 tx=0x7fac60009040 comp rx=0 tx=0).stop 2023-10-26T01:43:34.553 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.550+0000 7fac6bff5640 1 -- 172.21.15.169:0/3185282273 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fac641592d0 msgr2=0x7fac64144430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:34.553 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.550+0000 7fac6bff5640 1 --2- 172.21.15.169:0/3185282273 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fac641592d0 0x7fac64144430 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7fac5c036040 tx=0x7fac5c004270 comp rx=0 tx=0).stop 2023-10-26T01:43:34.554 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.550+0000 7fac6bff5640 1 -- 172.21.15.169:0/3185282273 shutdown_connections 2023-10-26T01:43:34.554 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.550+0000 7fac6bff5640 1 --2- 172.21.15.169:0/3185282273 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fac4806ae40 0x7fac4806d300 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:34.555 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.550+0000 7fac6bff5640 1 --2- 172.21.15.169:0/3185282273 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fac641592d0 0x7fac64144430 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:34.555 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.550+0000 7fac6bff5640 1 --2- 172.21.15.169:0/3185282273 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fac640cde00 0x7fac6415c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:34.555 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.550+0000 7fac6bff5640 1 --2- 172.21.15.169:0/3185282273 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fac640ac930 0x7fac6406ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:34.555 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.550+0000 7fac6bff5640 1 -- 172.21.15.169:0/3185282273 >> 172.21.15.169:0/3185282273 conn(0x7fac64069930 msgr2=0x7fac640af250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:34.555 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.551+0000 7fac6bff5640 1 -- 172.21.15.169:0/3185282273 shutdown_connections 2023-10-26T01:43:34.556 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.551+0000 7fac6bff5640 1 -- 172.21.15.169:0/3185282273 wait complete. 2023-10-26T01:43:34.572 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:43:34.779 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.776+0000 7ff728e9a640 1 -- 172.21.15.169:0/1466591464 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff70c004d00 msgr2=0x7ff70c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:34.780 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.776+0000 7ff728e9a640 1 --2- 172.21.15.169:0/1466591464 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff70c004d00 0x7ff70c005180 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7ff71c0043e0 tx=0x7ff70c0057f0 comp rx=0 tx=0).stop 2023-10-26T01:43:34.780 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.776+0000 7ff723fff640 1 -- 172.21.15.169:0/1466591464 reap_dead start 2023-10-26T01:43:34.780 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.776+0000 7ff728e9a640 1 -- 172.21.15.169:0/1466591464 shutdown_connections 2023-10-26T01:43:34.780 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.776+0000 7ff728e9a640 1 -- 172.21.15.169:0/1466591464 >> 172.21.15.169:0/1466591464 conn(0x7ff7240aedd0 msgr2=0x7ff7240af1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:34.780 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.777+0000 7ff728e9a640 1 -- 172.21.15.169:0/1466591464 shutdown_connections 2023-10-26T01:43:34.781 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.777+0000 7ff728e9a640 1 -- 172.21.15.169:0/1466591464 wait complete. 2023-10-26T01:43:34.781 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.778+0000 7ff728e9a640 1 Processor -- start 2023-10-26T01:43:34.781 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.779+0000 7ff728e9a640 1 -- start start 2023-10-26T01:43:34.783 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.779+0000 7ff728e9a640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff70c004d00 0x7ff72406f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:34.783 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.779+0000 7ff728e9a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7240c75b0 0x7ff7241f29d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:34.783 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.779+0000 7ff728e9a640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7241ef1c0 0x7ff7241ef600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:34.783 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.779+0000 7ff728e9a640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff724159b40 con 0x7ff7241ef1c0 2023-10-26T01:43:34.784 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.779+0000 7ff728e9a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7240c9390 con 0x7ff7240c75b0 2023-10-26T01:43:34.784 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.779+0000 7ff728e9a640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7240c7b70 con 0x7ff70c004d00 2023-10-26T01:43:34.784 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.780+0000 7ff723fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7241ef1c0 0x7ff7241ef600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:34.784 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.780+0000 7ff723fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7241ef1c0 0x7ff7241ef600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:35690/0 (socket says 172.21.15.169:35690) 2023-10-26T01:43:34.784 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.780+0000 7ff723fff640 1 -- 172.21.15.169:0/2422688515 learned_addr learned my addr 172.21.15.169:0/2422688515 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:43:34.785 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.780+0000 7ff722ffd640 1 --2- 172.21.15.169:0/2422688515 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7240c75b0 0x7ff7241f29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:34.785 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.780+0000 7ff723fff640 1 -- 172.21.15.169:0/2422688515 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff70c004d00 msgr2=0x7ff72406f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:34.785 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.780+0000 7ff723fff640 1 --2- 172.21.15.169:0/2422688515 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff70c004d00 0x7ff72406f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:34.785 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.780+0000 7ff723fff640 1 -- 172.21.15.169:0/2422688515 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7240c75b0 msgr2=0x7ff7241f29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:34.785 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.780+0000 7ff723fff640 1 --2- 172.21.15.169:0/2422688515 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7240c75b0 0x7ff7241f29d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:34.786 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.780+0000 7ff723fff640 1 -- 172.21.15.169:0/2422688515 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7240c7e50 con 0x7ff7241ef1c0 2023-10-26T01:43:34.786 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.780+0000 7ff723fff640 1 --2- 172.21.15.169:0/2422688515 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7241ef1c0 0x7ff7241ef600 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7ff71c02c380 tx=0x7ff71c006ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:34.787 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.781+0000 7ff70bfff640 1 -- 172.21.15.169:0/2422688515 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff71c006270 con 0x7ff7241ef1c0 2023-10-26T01:43:34.787 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.781+0000 7ff70bfff640 1 -- 172.21.15.169:0/2422688515 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff71c0064c0 con 0x7ff7241ef1c0 2023-10-26T01:43:34.787 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.781+0000 7ff70bfff640 1 -- 172.21.15.169:0/2422688515 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff71c006870 con 0x7ff7241ef1c0 2023-10-26T01:43:34.788 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.781+0000 7ff728e9a640 1 -- 172.21.15.169:0/2422688515 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7240c8130 con 0x7ff7241ef1c0 2023-10-26T01:43:34.788 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.781+0000 7ff728e9a640 1 -- 172.21.15.169:0/2422688515 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7240ad840 con 0x7ff7241ef1c0 2023-10-26T01:43:34.788 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.782+0000 7ff728e9a640 1 -- 172.21.15.169:0/2422688515 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff724159c80 con 0x7ff7241ef1c0 2023-10-26T01:43:34.788 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.783+0000 7ff70bfff640 1 -- 172.21.15.169:0/2422688515 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ff71c032070 con 0x7ff7241ef1c0 2023-10-26T01:43:34.788 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.784+0000 7ff70bfff640 1 --2- 172.21.15.169:0/2422688515 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff71006ae40 0x7ff71006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:34.789 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.784+0000 7ff70bfff640 1 -- 172.21.15.169:0/2422688515 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 5738+0+0 (secure 0 0 0) 0x7ff71c0aaff0 con 0x7ff7241ef1c0 2023-10-26T01:43:34.789 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.784+0000 7ff70bfff640 1 -- 172.21.15.169:0/2422688515 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=171}) v3 -- 0x7ff7100729d0 con 0x7ff7241ef1c0 2023-10-26T01:43:34.789 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.784+0000 7ff7237fe640 1 --2- 172.21.15.169:0/2422688515 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff71006ae40 0x7ff71006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:34.789 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.785+0000 7ff7237fe640 1 --2- 172.21.15.169:0/2422688515 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff71006ae40 0x7ff71006d300 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7ff714001390 tx=0x7ff714000fa0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:34.791 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.788+0000 7ff70bfff640 1 -- 172.21.15.169:0/2422688515 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff71c044020 con 0x7ff7241ef1c0 2023-10-26T01:43:34.980 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.978+0000 7ff728e9a640 1 -- 172.21.15.169:0/2422688515 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff724073c10 con 0x7ff7241ef1c0 2023-10-26T01:43:34.982 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.979+0000 7ff70bfff640 1 -- 172.21.15.169:0/2422688515 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v170) v1 ==== 74+0+16704 (secure 0 0 0) 0x7ff724159c80 con 0x7ff7241ef1c0 2023-10-26T01:43:34.983 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:43:34.983 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":170,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:42:58.161990+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2023-10-26T01:42:07.656531+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"165","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2023-10-26T01:42:08.866635+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"170","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.39","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:43:34.985 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.982+0000 7ff728e9a640 1 -- 172.21.15.169:0/2422688515 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff71006ae40 msgr2=0x7ff71006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:34.985 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.982+0000 7ff728e9a640 1 --2- 172.21.15.169:0/2422688515 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff71006ae40 0x7ff71006d300 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7ff714001390 tx=0x7ff714000fa0 comp rx=0 tx=0).stop 2023-10-26T01:43:34.985 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.982+0000 7ff728e9a640 1 -- 172.21.15.169:0/2422688515 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7241ef1c0 msgr2=0x7ff7241ef600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:34.985 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.982+0000 7ff728e9a640 1 --2- 172.21.15.169:0/2422688515 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7241ef1c0 0x7ff7241ef600 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7ff71c02c380 tx=0x7ff71c006ce0 comp rx=0 tx=0).stop 2023-10-26T01:43:34.986 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.983+0000 7ff728e9a640 1 -- 172.21.15.169:0/2422688515 shutdown_connections 2023-10-26T01:43:34.987 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.983+0000 7ff728e9a640 1 --2- 172.21.15.169:0/2422688515 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff71006ae40 0x7ff71006d300 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:34.987 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.983+0000 7ff728e9a640 1 --2- 172.21.15.169:0/2422688515 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7241ef1c0 0x7ff7241ef600 unknown :-1 s=CLOSED pgs=1779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:34.987 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.983+0000 7ff728e9a640 1 --2- 172.21.15.169:0/2422688515 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7240c75b0 0x7ff7241f29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:34.987 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.983+0000 7ff728e9a640 1 --2- 172.21.15.169:0/2422688515 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff70c004d00 0x7ff72406f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:34.987 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.983+0000 7ff728e9a640 1 -- 172.21.15.169:0/2422688515 >> 172.21.15.169:0/2422688515 conn(0x7ff7240aedd0 msgr2=0x7ff724141a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:34.988 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.983+0000 7ff728e9a640 1 -- 172.21.15.169:0/2422688515 shutdown_connections 2023-10-26T01:43:34.988 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:34.983+0000 7ff728e9a640 1 -- 172.21.15.169:0/2422688515 wait complete. 2023-10-26T01:43:35.005 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-26T01:43:35.226 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:35.223+0000 7f0c54183640 1 -- 172.21.15.169:0/1562059443 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c38003970 msgr2=0x7f0c38003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:35.227 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:35.223+0000 7f0c54183640 1 --2- 172.21.15.169:0/1562059443 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c38003970 0x7f0c38003d90 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7f0c40006fc0 tx=0x7f0c380057f0 comp rx=0 tx=0).stop 2023-10-26T01:43:35.227 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:35.223+0000 7f0c52720640 1 -- 172.21.15.169:0/1562059443 reap_dead start 2023-10-26T01:43:35.227 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:35.223+0000 7f0c54183640 1 -- 172.21.15.169:0/1562059443 shutdown_connections 2023-10-26T01:43:35.227 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:35.223+0000 7f0c54183640 1 -- 172.21.15.169:0/1562059443 >> 172.21.15.169:0/1562059443 conn(0x7f0c4c069930 msgr2=0x7f0c4c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:35.227 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:35.223+0000 7f0c54183640 1 -- 172.21.15.169:0/1562059443 shutdown_connections 2023-10-26T01:43:35.228 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:35.224+0000 7f0c54183640 1 -- 172.21.15.169:0/1562059443 wait complete. 2023-10-26T01:43:35.228 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:35.225+0000 7f0c54183640 1 Processor -- start 2023-10-26T01:43:35.228 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:35.225+0000 7f0c54183640 1 -- start start 2023-10-26T01:43:35.231 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:35.226+0000 7f0c54183640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0c38003970 0x7f0c4c152520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:35.231 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:35.226+0000 7f0c54183640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0c4c1f1ae0 0x7f0c4c0af870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:35.231 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:35.226+0000 7f0c54183640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c4c0cc320 0x7f0c4c0cc7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:35.231 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:35.226+0000 7f0c54183640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c4c0ccce0 con 0x7f0c4c0cc320 2023-10-26T01:43:35.232 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:35.226+0000 7f0c54183640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c4c152e00 con 0x7f0c38003970 2023-10-26T01:43:35.232 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:35.226+0000 7f0c54183640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c4c0afdb0 con 0x7f0c4c1f1ae0 2023-10-26T01:43:35.232 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:35.227+0000 7f0c52720640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c4c0cc320 0x7f0c4c0cc7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:35.232 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:35.227+0000 7f0c52720640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c4c0cc320 0x7f0c4c0cc7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:35706/0 (socket says 172.21.15.169:35706) 2023-10-26T01:43:35.232 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:35.227+0000 7f0c52720640 1 -- 172.21.15.169:0/2586855578 learned_addr learned my addr 172.21.15.169:0/2586855578 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:43:35.233 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:35.227+0000 7f0c51f1f640 1 --2- 172.21.15.169:0/2586855578 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0c38003970 0x7f0c4c152520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:35.233 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:35.227+0000 7f0c52720640 1 -- 172.21.15.169:0/2586855578 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0c4c1f1ae0 msgr2=0x7f0c4c0af870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:35.233 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:35.227+0000 7f0c5171e640 1 --2- 172.21.15.169:0/2586855578 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0c4c1f1ae0 0x7f0c4c0af870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:35.234 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:35.227+0000 7f0c52720640 1 --2- 172.21.15.169:0/2586855578 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0c4c1f1ae0 0x7f0c4c0af870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:35.234 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:35.227+0000 7f0c52720640 1 -- 172.21.15.169:0/2586855578 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0c38003970 msgr2=0x7f0c4c152520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:35.234 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:35.227+0000 7f0c52720640 1 --2- 172.21.15.169:0/2586855578 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0c38003970 0x7f0c4c152520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:35.234 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:35.227+0000 7f0c52720640 1 -- 172.21.15.169:0/2586855578 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c4c1f2090 con 0x7f0c4c0cc320 2023-10-26T01:43:35.234 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:35.227+0000 7f0c5171e640 1 --2- 172.21.15.169:0/2586855578 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0c4c1f1ae0 0x7f0c4c0af870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:43:35.235 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:35.227+0000 7f0c51f1f640 1 --2- 172.21.15.169:0/2586855578 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0c38003970 0x7f0c4c152520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:43:35.235 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:35.227+0000 7f0c52720640 1 --2- 172.21.15.169:0/2586855578 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c4c0cc320 0x7f0c4c0cc7a0 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f0c38004740 tx=0x7f0c4400eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:35.235 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:35.228+0000 7f0c3e7fc640 1 -- 172.21.15.169:0/2586855578 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c44004d20 con 0x7f0c4c0cc320 2023-10-26T01:43:35.235 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:35.228+0000 7f0c3e7fc640 1 -- 172.21.15.169:0/2586855578 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0c4400c820 con 0x7f0c4c0cc320 2023-10-26T01:43:35.236 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:35.228+0000 7f0c3e7fc640 1 -- 172.21.15.169:0/2586855578 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c4400cc30 con 0x7f0c4c0cc320 2023-10-26T01:43:35.236 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:35.228+0000 7f0c54183640 1 -- 172.21.15.169:0/2586855578 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c4c1f2370 con 0x7f0c4c0cc320 2023-10-26T01:43:35.236 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:35.228+0000 7f0c54183640 1 -- 172.21.15.169:0/2586855578 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c4c1f2930 con 0x7f0c4c0cc320 2023-10-26T01:43:35.237 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:35.229+0000 7f0c54183640 1 -- 172.21.15.169:0/2586855578 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c4c04db10 con 0x7f0c4c0cc320 2023-10-26T01:43:35.237 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:35.230+0000 7f0c3e7fc640 1 -- 172.21.15.169:0/2586855578 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f0c4400cd90 con 0x7f0c4c0cc320 2023-10-26T01:43:35.238 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:35.231+0000 7f0c3e7fc640 1 --2- 172.21.15.169:0/2586855578 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f0c3006ae70 0x7f0c3006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:35.238 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:35.231+0000 7f0c3e7fc640 1 -- 172.21.15.169:0/2586855578 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f0c44089910 con 0x7f0c4c0cc320 2023-10-26T01:43:35.238 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:35.231+0000 7f0c3e7fc640 1 -- 172.21.15.169:0/2586855578 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=171}) v3 -- 0x7f0c30072a00 con 0x7f0c4c0cc320 2023-10-26T01:43:35.238 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:35.232+0000 7f0c51f1f640 1 --2- 172.21.15.169:0/2586855578 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f0c3006ae70 0x7f0c3006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:35.239 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:35.233+0000 7f0c51f1f640 1 --2- 172.21.15.169:0/2586855578 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f0c3006ae70 0x7f0c3006d330 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f0c40036000 tx=0x7f0c40001a00 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:35.239 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:35.235+0000 7f0c3e7fc640 1 -- 172.21.15.169:0/2586855578 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0c44016060 con 0x7f0c4c0cc320 2023-10-26T01:43:35.468 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:35.465+0000 7f0c54183640 1 -- 172.21.15.169:0/2586855578 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f0c4c069fe0 con 0x7f0c4c0cc320 2023-10-26T01:43:35.469 INFO:tasks.ceph.mon.a.smithi169.stderr:2023-10-26T01:43:35.466+0000 7f1588a74640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-26T01:43:35.472 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:35.469+0000 7f0c3e7fc640 1 -- 172.21.15.169:0/2586855578 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v87) v1 ==== 161+0+0 (secure 0 0 0) 0x7f0c44016030 con 0x7f0c4c0cc320 2023-10-26T01:43:35.472 INFO:teuthology.orchestra.run.smithi169.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-26T01:43:35.475 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:35.471+0000 7f0c3e7fc640 1 -- 172.21.15.169:0/2586855578 <== mon.0 v2:172.21.15.169:3300/0 8 ==== osd_map(171..171 src has 1..171) v4 ==== 403+0+0 (secure 0 0 0) 0x7f0c44058d50 con 0x7f0c4c0cc320 2023-10-26T01:43:35.475 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:35.471+0000 7f0c3e7fc640 1 -- 172.21.15.169:0/2586855578 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=172}) v3 -- 0x7f0c300733a0 con 0x7f0c4c0cc320 2023-10-26T01:43:35.478 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:35.476+0000 7f0c54183640 1 -- 172.21.15.169:0/2586855578 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f0c3006ae70 msgr2=0x7f0c3006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:35.479 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:35.476+0000 7f0c54183640 1 --2- 172.21.15.169:0/2586855578 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f0c3006ae70 0x7f0c3006d330 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f0c40036000 tx=0x7f0c40001a00 comp rx=0 tx=0).stop 2023-10-26T01:43:35.479 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:35.476+0000 7f0c54183640 1 -- 172.21.15.169:0/2586855578 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c4c0cc320 msgr2=0x7f0c4c0cc7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:35.479 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:35.476+0000 7f0c54183640 1 --2- 172.21.15.169:0/2586855578 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c4c0cc320 0x7f0c4c0cc7a0 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f0c38004740 tx=0x7f0c4400eb90 comp rx=0 tx=0).stop 2023-10-26T01:43:35.480 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:35.476+0000 7f0c54183640 1 -- 172.21.15.169:0/2586855578 shutdown_connections 2023-10-26T01:43:35.480 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:35.476+0000 7f0c54183640 1 --2- 172.21.15.169:0/2586855578 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f0c3006ae70 0x7f0c3006d330 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:35.480 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:35.476+0000 7f0c54183640 1 --2- 172.21.15.169:0/2586855578 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c4c0cc320 0x7f0c4c0cc7a0 unknown :-1 s=CLOSED pgs=1782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:35.480 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:35.476+0000 7f0c54183640 1 --2- 172.21.15.169:0/2586855578 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0c4c1f1ae0 0x7f0c4c0af870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:35.481 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:35.476+0000 7f0c54183640 1 --2- 172.21.15.169:0/2586855578 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0c38003970 0x7f0c4c152520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:35.481 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:35.477+0000 7f0c54183640 1 -- 172.21.15.169:0/2586855578 >> 172.21.15.169:0/2586855578 conn(0x7f0c4c069930 msgr2=0x7f0c4c157240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:35.481 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:35.477+0000 7f0c54183640 1 -- 172.21.15.169:0/2586855578 shutdown_connections 2023-10-26T01:43:35.481 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:35.477+0000 7f0c54183640 1 -- 172.21.15.169:0/2586855578 wait complete. 2023-10-26T01:43:35.498 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-26T01:43:35.549 INFO:tasks.ceph.mds.a.smithi169.stdout:starting mds.a at 2023-10-26T01:43:35.698 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:35.695+0000 7f030754c640 1 -- 172.21.15.169:0/2260871247 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f03000cdd20 msgr2=0x7f030015b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:35.698 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:35.695+0000 7f030754c640 1 --2- 172.21.15.169:0/2260871247 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f03000cdd20 0x7f030015b4a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f02f4009830 tx=0x7f02f402bd50 comp rx=0 tx=0).stop 2023-10-26T01:43:35.699 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:35.695+0000 7f030754c640 1 -- 172.21.15.169:0/2260871247 shutdown_connections 2023-10-26T01:43:35.699 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:35.695+0000 7f030754c640 1 --2- 172.21.15.169:0/2260871247 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f03000cdd20 0x7f030015b4a0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:35.699 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:35.695+0000 7f030754c640 1 --2- 172.21.15.169:0/2260871247 >> v2:172.21.15.170:3300/0 conn(0x7f0300155360 0x7f03000cd7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:35.699 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:35.695+0000 7f030754c640 1 -- 172.21.15.169:0/2260871247 >> 172.21.15.169:0/2260871247 conn(0x7f0300069930 msgr2=0x7f0300069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:35.699 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:35.695+0000 7f030754c640 1 -- 172.21.15.169:0/2260871247 shutdown_connections 2023-10-26T01:43:35.700 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:35.695+0000 7f030754c640 1 -- 172.21.15.169:0/2260871247 wait complete. 2023-10-26T01:43:35.700 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:35.697+0000 7f030754c640 1 Processor -- start 2023-10-26T01:43:35.700 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:35.697+0000 7f030754c640 1 -- start start 2023-10-26T01:43:35.703 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:35.698+0000 7f030754c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03000cdd20 0x7f0300146c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:35.703 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:35.698+0000 7f030754c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0300155360 0x7f03001505f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:35.703 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:35.698+0000 7f030754c640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f030015b9e0 0x7f03000b0050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:35.703 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:35.698+0000 7f030754c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0300150b30 con 0x7f03000cdd20 2023-10-26T01:43:35.703 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:35.698+0000 7f030754c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0300150160 con 0x7f0300155360 2023-10-26T01:43:35.704 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:35.698+0000 7f030754c640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03001f1b40 con 0x7f030015b9e0 2023-10-26T01:43:35.704 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:35.698+0000 7f03052e8640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03000cdd20 0x7f0300146c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:35.704 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:35.698+0000 7f03052e8640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03000cdd20 0x7f0300146c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:35728/0 (socket says 172.21.15.169:35728) 2023-10-26T01:43:35.704 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:35.698+0000 7f03052e8640 1 -- 172.21.15.169:0/1116070673 learned_addr learned my addr 172.21.15.169:0/1116070673 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:43:35.704 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:35.699+0000 7f0304ae7640 1 --2- 172.21.15.169:0/1116070673 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0300155360 0x7f03001505f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:35.705 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:35.699+0000 7f03052e8640 1 -- 172.21.15.169:0/1116070673 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f030015b9e0 msgr2=0x7f03000b0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:35.705 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:35.699+0000 7f0305ae9640 1 --2- 172.21.15.169:0/1116070673 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f030015b9e0 0x7f03000b0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:35.705 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:35.699+0000 7f03052e8640 1 --2- 172.21.15.169:0/1116070673 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f030015b9e0 0x7f03000b0050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:35.705 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:35.699+0000 7f03052e8640 1 -- 172.21.15.169:0/1116070673 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0300155360 msgr2=0x7f03001505f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:35.705 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:35.699+0000 7f03052e8640 1 --2- 172.21.15.169:0/1116070673 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0300155360 0x7f03001505f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:35.706 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:35.699+0000 7f03052e8640 1 -- 172.21.15.169:0/1116070673 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02fc009490 con 0x7f03000cdd20 2023-10-26T01:43:35.706 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:35.699+0000 7f0304ae7640 1 --2- 172.21.15.169:0/1116070673 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0300155360 0x7f03001505f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:43:35.706 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:35.699+0000 7f03052e8640 1 --2- 172.21.15.169:0/1116070673 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03000cdd20 0x7f0300146c20 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f02fc00ea90 tx=0x7f02fc00ef50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:35.706 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:35.699+0000 7f02f1ffb640 1 -- 172.21.15.169:0/1116070673 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02fc00cde0 con 0x7f03000cdd20 2023-10-26T01:43:35.706 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:35.699+0000 7f02f1ffb640 1 -- 172.21.15.169:0/1116070673 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f02fc004020 con 0x7f03000cdd20 2023-10-26T01:43:35.707 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:35.699+0000 7f030754c640 1 -- 172.21.15.169:0/1116070673 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02f4009490 con 0x7f03000cdd20 2023-10-26T01:43:35.707 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:35.700+0000 7f02f1ffb640 1 -- 172.21.15.169:0/1116070673 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02fc004430 con 0x7f03000cdd20 2023-10-26T01:43:35.707 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:35.700+0000 7f030754c640 1 -- 172.21.15.169:0/1116070673 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f030014f550 con 0x7f03000cdd20 2023-10-26T01:43:35.708 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:35.701+0000 7f030754c640 1 -- 172.21.15.169:0/1116070673 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f030004db10 con 0x7f03000cdd20 2023-10-26T01:43:35.708 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:35.702+0000 7f02f1ffb640 1 -- 172.21.15.169:0/1116070673 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f02fc0035f0 con 0x7f03000cdd20 2023-10-26T01:43:35.708 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:35.703+0000 7f02f1ffb640 1 --2- 172.21.15.169:0/1116070673 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f02e406ae40 0x7f02e406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:35.709 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:35.703+0000 7f02f1ffb640 1 -- 172.21.15.169:0/1116070673 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f02fc014070 con 0x7f03000cdd20 2023-10-26T01:43:35.709 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:35.703+0000 7f02f1ffb640 1 -- 172.21.15.169:0/1116070673 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=172}) v3 -- 0x7f02e4072ae0 con 0x7f03000cdd20 2023-10-26T01:43:35.709 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:35.703+0000 7f0304ae7640 1 --2- 172.21.15.169:0/1116070673 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f02e406ae40 0x7f02e406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:35.709 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:35.704+0000 7f0304ae7640 1 --2- 172.21.15.169:0/1116070673 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f02e406ae40 0x7f02e406d300 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f02f4036000 tx=0x7f02f4004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:35.710 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:35.707+0000 7f02f1ffb640 1 -- 172.21.15.169:0/1116070673 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f02fc055290 con 0x7f03000cdd20 2023-10-26T01:43:35.941 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:35.938+0000 7f030754c640 1 -- 172.21.15.169:0/1116070673 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f03000cda90 con 0x7f03000cdd20 2023-10-26T01:43:36.476 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:36.473+0000 7f02f1ffb640 1 -- 172.21.15.169:0/1116070673 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v88) v1 ==== 104+0+0 (secure 0 0 0) 0x7f02fc0588b0 con 0x7f03000cdd20 2023-10-26T01:43:36.482 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:36.478+0000 7f030754c640 1 -- 172.21.15.169:0/1116070673 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f02e406ae40 msgr2=0x7f02e406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:36.483 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:36.478+0000 7f030754c640 1 --2- 172.21.15.169:0/1116070673 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f02e406ae40 0x7f02e406d300 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f02f4036000 tx=0x7f02f4004730 comp rx=0 tx=0).stop 2023-10-26T01:43:36.483 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:36.478+0000 7f030754c640 1 -- 172.21.15.169:0/1116070673 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03000cdd20 msgr2=0x7f0300146c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:36.483 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:36.478+0000 7f030754c640 1 --2- 172.21.15.169:0/1116070673 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03000cdd20 0x7f0300146c20 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f02fc00ea90 tx=0x7f02fc00ef50 comp rx=0 tx=0).stop 2023-10-26T01:43:36.483 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:36.478+0000 7f030754c640 1 -- 172.21.15.169:0/1116070673 shutdown_connections 2023-10-26T01:43:36.483 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:36.478+0000 7f030754c640 1 --2- 172.21.15.169:0/1116070673 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f02e406ae40 0x7f02e406d300 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:36.484 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:36.478+0000 7f030754c640 1 --2- 172.21.15.169:0/1116070673 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f030015b9e0 0x7f03000b0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:36.484 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:36.478+0000 7f030754c640 1 --2- 172.21.15.169:0/1116070673 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0300155360 0x7f03001505f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:36.484 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:36.478+0000 7f030754c640 1 --2- 172.21.15.169:0/1116070673 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03000cdd20 0x7f0300146c20 unknown :-1 s=CLOSED pgs=1785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:36.484 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:36.478+0000 7f030754c640 1 -- 172.21.15.169:0/1116070673 >> 172.21.15.169:0/1116070673 conn(0x7f0300069930 msgr2=0x7f030014cd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:36.484 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:36.478+0000 7f030754c640 1 -- 172.21.15.169:0/1116070673 shutdown_connections 2023-10-26T01:43:36.485 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:36.478+0000 7f030754c640 1 -- 172.21.15.169:0/1116070673 wait complete. 2023-10-26T01:43:36.503 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-26T01:43:36.727 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:36.724+0000 7ff92c861640 1 -- 172.21.15.169:0/39409121 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9140039a0 msgr2=0x7ff914003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:36.728 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:36.724+0000 7ff92c861640 1 --2- 172.21.15.169:0/39409121 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9140039a0 0x7ff914003dc0 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7ff918001680 tx=0x7ff9140057f0 comp rx=0 tx=0).stop 2023-10-26T01:43:36.728 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:36.724+0000 7ff926d9d640 1 -- 172.21.15.169:0/39409121 reap_dead start 2023-10-26T01:43:36.728 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:36.724+0000 7ff92c861640 1 -- 172.21.15.169:0/39409121 shutdown_connections 2023-10-26T01:43:36.728 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:36.724+0000 7ff92c861640 1 -- 172.21.15.169:0/39409121 >> 172.21.15.169:0/39409121 conn(0x7ff9280aeee0 msgr2=0x7ff9280af2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:36.728 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:36.724+0000 7ff92c861640 1 -- 172.21.15.169:0/39409121 shutdown_connections 2023-10-26T01:43:36.729 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:36.724+0000 7ff92c861640 1 -- 172.21.15.169:0/39409121 wait complete. 2023-10-26T01:43:36.729 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:36.726+0000 7ff92c861640 1 Processor -- start 2023-10-26T01:43:36.729 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:36.726+0000 7ff92c861640 1 -- start start 2023-10-26T01:43:36.732 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:36.727+0000 7ff92c861640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9140039a0 0x7ff92806fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:36.732 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:36.727+0000 7ff92c861640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff9280c7ed0 0x7ff9280c7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:36.732 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:36.727+0000 7ff92c861640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff92814eda0 0x7ff92806be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:36.732 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:36.727+0000 7ff92c861640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9280c7580 con 0x7ff9140039a0 2023-10-26T01:43:36.732 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:36.727+0000 7ff92c861640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9280703e0 con 0x7ff92814eda0 2023-10-26T01:43:36.733 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:36.727+0000 7ff92c861640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9280706c0 con 0x7ff9280c7ed0 2023-10-26T01:43:36.733 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:36.727+0000 7ff92659c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9140039a0 0x7ff92806fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:36.733 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:36.728+0000 7ff92659c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9140039a0 0x7ff92806fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:35740/0 (socket says 172.21.15.169:35740) 2023-10-26T01:43:36.733 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:36.728+0000 7ff92659c640 1 -- 172.21.15.169:0/3801432941 learned_addr learned my addr 172.21.15.169:0/3801432941 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:43:36.734 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:36.728+0000 7ff926d9d640 1 --2- 172.21.15.169:0/3801432941 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff92814eda0 0x7ff92806be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:36.734 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:36.728+0000 7ff91ffff640 1 --2- 172.21.15.169:0/3801432941 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff9280c7ed0 0x7ff9280c7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:36.734 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:36.728+0000 7ff92659c640 1 -- 172.21.15.169:0/3801432941 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff9280c7ed0 msgr2=0x7ff9280c7040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:36.734 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:36.728+0000 7ff92659c640 1 --2- 172.21.15.169:0/3801432941 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff9280c7ed0 0x7ff9280c7040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:36.734 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:36.728+0000 7ff92659c640 1 -- 172.21.15.169:0/3801432941 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff92814eda0 msgr2=0x7ff92806be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:36.735 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:36.728+0000 7ff92659c640 1 --2- 172.21.15.169:0/3801432941 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff92814eda0 0x7ff92806be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:36.735 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:36.728+0000 7ff92659c640 1 -- 172.21.15.169:0/3801432941 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9280c6560 con 0x7ff9140039a0 2023-10-26T01:43:36.735 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:36.728+0000 7ff92659c640 1 --2- 172.21.15.169:0/3801432941 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9140039a0 0x7ff92806fd00 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7ff9180017a0 tx=0x7ff9180035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:36.735 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:36.729+0000 7ff91effd640 1 -- 172.21.15.169:0/3801432941 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff918003bf0 con 0x7ff9140039a0 2023-10-26T01:43:36.735 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:36.729+0000 7ff91effd640 1 -- 172.21.15.169:0/3801432941 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff918003e40 con 0x7ff9140039a0 2023-10-26T01:43:36.736 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:36.729+0000 7ff92c861640 1 -- 172.21.15.169:0/3801432941 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9280c6840 con 0x7ff9140039a0 2023-10-26T01:43:36.736 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:36.729+0000 7ff91effd640 1 -- 172.21.15.169:0/3801432941 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff91802c7b0 con 0x7ff9140039a0 2023-10-26T01:43:36.736 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:36.729+0000 7ff92c861640 1 -- 172.21.15.169:0/3801432941 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9280c8330 con 0x7ff9140039a0 2023-10-26T01:43:36.737 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:36.730+0000 7ff92c861640 1 -- 172.21.15.169:0/3801432941 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff928066af0 con 0x7ff9140039a0 2023-10-26T01:43:36.737 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:36.731+0000 7ff91effd640 1 -- 172.21.15.169:0/3801432941 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ff91802c910 con 0x7ff9140039a0 2023-10-26T01:43:36.738 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:36.732+0000 7ff91effd640 1 --2- 172.21.15.169:0/3801432941 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff90806ae70 0x7ff90806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:36.738 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:36.732+0000 7ff91effd640 1 -- 172.21.15.169:0/3801432941 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 5824+0+0 (secure 0 0 0) 0x7ff9180aa990 con 0x7ff9140039a0 2023-10-26T01:43:36.738 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:36.732+0000 7ff91effd640 1 -- 172.21.15.169:0/3801432941 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=172}) v3 -- 0x7ff908072b10 con 0x7ff9140039a0 2023-10-26T01:43:36.738 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:36.732+0000 7ff91ffff640 1 --2- 172.21.15.169:0/3801432941 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff90806ae70 0x7ff90806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:36.739 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:36.733+0000 7ff91ffff640 1 --2- 172.21.15.169:0/3801432941 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff90806ae70 0x7ff90806d330 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7ff9281f5630 tx=0x7ff910005ed0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:36.739 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:36.736+0000 7ff91effd640 1 -- 172.21.15.169:0/3801432941 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff918043020 con 0x7ff9140039a0 2023-10-26T01:43:36.928 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:36.925+0000 7ff92c861640 1 -- 172.21.15.169:0/3801432941 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff92804da70 con 0x7ff9140039a0 2023-10-26T01:43:37.479 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:37.476+0000 7ff91effd640 1 -- 172.21.15.169:0/3801432941 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v172) v1 ==== 181+0+0 (secure 0 0 0) 0x7ff918079470 con 0x7ff9140039a0 2023-10-26T01:43:37.480 INFO:teuthology.orchestra.run.smithi169.stderr:pool 'cephfs_metadata' removed 2023-10-26T01:43:37.481 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:37.478+0000 7ff91effd640 1 -- 172.21.15.169:0/3801432941 <== mon.0 v2:172.21.15.169:3300/0 8 ==== osd_map(172..172 src has 1..172) v4 ==== 325+0+0 (secure 0 0 0) 0x7ff918079da0 con 0x7ff9140039a0 2023-10-26T01:43:37.481 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:37.478+0000 7ff91effd640 1 -- 172.21.15.169:0/3801432941 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=173}) v3 -- 0x7ff908070f40 con 0x7ff9140039a0 2023-10-26T01:43:37.489 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:37.487+0000 7ff92c861640 1 -- 172.21.15.169:0/3801432941 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff90806ae70 msgr2=0x7ff90806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:37.490 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:37.487+0000 7ff92c861640 1 --2- 172.21.15.169:0/3801432941 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff90806ae70 0x7ff90806d330 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7ff9281f5630 tx=0x7ff910005ed0 comp rx=0 tx=0).stop 2023-10-26T01:43:37.490 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:37.487+0000 7ff92c861640 1 -- 172.21.15.169:0/3801432941 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9140039a0 msgr2=0x7ff92806fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:37.490 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:37.487+0000 7ff92c861640 1 --2- 172.21.15.169:0/3801432941 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9140039a0 0x7ff92806fd00 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7ff9180017a0 tx=0x7ff9180035f0 comp rx=0 tx=0).stop 2023-10-26T01:43:37.491 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:37.488+0000 7ff92c861640 1 -- 172.21.15.169:0/3801432941 shutdown_connections 2023-10-26T01:43:37.491 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:37.488+0000 7ff92c861640 1 --2- 172.21.15.169:0/3801432941 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff90806ae70 0x7ff90806d330 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:37.491 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:37.488+0000 7ff92c861640 1 --2- 172.21.15.169:0/3801432941 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff92814eda0 0x7ff92806be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:37.493 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:37.488+0000 7ff92c861640 1 --2- 172.21.15.169:0/3801432941 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff9280c7ed0 0x7ff9280c7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:37.494 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:37.488+0000 7ff92c861640 1 --2- 172.21.15.169:0/3801432941 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9140039a0 0x7ff92806fd00 unknown :-1 s=CLOSED pgs=1788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:37.494 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:37.488+0000 7ff92c861640 1 -- 172.21.15.169:0/3801432941 >> 172.21.15.169:0/3801432941 conn(0x7ff9280aeee0 msgr2=0x7ff92806b570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:37.495 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:37.489+0000 7ff92c861640 1 -- 172.21.15.169:0/3801432941 shutdown_connections 2023-10-26T01:43:37.495 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:37.489+0000 7ff92c861640 1 -- 172.21.15.169:0/3801432941 wait complete. 2023-10-26T01:43:37.517 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-26T01:43:37.729 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:37.726+0000 7f5e0f59e640 1 -- 172.21.15.169:0/2703007186 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5e101538c0 msgr2=0x7f5e1009a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:37.730 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:37.726+0000 7f5e0f59e640 1 --2- 172.21.15.169:0/2703007186 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5e101538c0 0x7f5e1009a920 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f5e1006cc50 tx=0x7f5e0002bc60 comp rx=0 tx=0).stop 2023-10-26T01:43:37.730 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:37.726+0000 7f5e0f59e640 1 -- 172.21.15.169:0/2703007186 shutdown_connections 2023-10-26T01:43:37.730 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:37.726+0000 7f5e0f59e640 1 --2- 172.21.15.169:0/2703007186 >> v2:172.21.15.170:3300/0 conn(0x7f5e100715f0 0x7f5e100b0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:37.730 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:37.726+0000 7f5e0f59e640 1 --2- 172.21.15.169:0/2703007186 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5e1015a430 0x7f5e100710b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:37.731 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:37.726+0000 7f5e0f59e640 1 --2- 172.21.15.169:0/2703007186 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5e101538c0 0x7f5e1009a920 unknown :-1 s=CLOSED pgs=1789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:37.731 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:37.726+0000 7f5e0f59e640 1 -- 172.21.15.169:0/2703007186 >> 172.21.15.169:0/2703007186 conn(0x7f5e10069b40 msgr2=0x7f5e100aeb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:37.731 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:37.726+0000 7f5e0f59e640 1 -- 172.21.15.169:0/2703007186 shutdown_connections 2023-10-26T01:43:37.731 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:37.727+0000 7f5e0f59e640 1 -- 172.21.15.169:0/2703007186 wait complete. 2023-10-26T01:43:37.733 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:37.728+0000 7f5e0f59e640 1 Processor -- start 2023-10-26T01:43:37.733 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:37.728+0000 7f5e0f59e640 1 -- start start 2023-10-26T01:43:37.733 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:37.729+0000 7f5e0f59e640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5e100715f0 0x7f5e10150b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:37.733 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:37.729+0000 7f5e0f59e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e101538c0 0x7f5e100adb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:37.734 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:37.729+0000 7f5e0f59e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5e1015a430 0x7f5e100ae040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:37.734 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:37.729+0000 7f5e0f59e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e1006aaa0 con 0x7f5e1015a430 2023-10-26T01:43:37.734 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:37.729+0000 7f5e0f59e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e1006ad80 con 0x7f5e101538c0 2023-10-26T01:43:37.734 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:37.729+0000 7f5e0f59e640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e1006b060 con 0x7f5e100715f0 2023-10-26T01:43:37.734 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:37.730+0000 7f5e0ed9d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5e1015a430 0x7f5e100ae040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:37.735 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:37.730+0000 7f5e0ed9d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5e1015a430 0x7f5e100ae040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:35770/0 (socket says 172.21.15.169:35770) 2023-10-26T01:43:37.735 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:37.730+0000 7f5e0ed9d640 1 -- 172.21.15.169:0/2398668126 learned_addr learned my addr 172.21.15.169:0/2398668126 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:43:37.735 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:37.730+0000 7f5e07fff640 1 --2- 172.21.15.169:0/2398668126 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e101538c0 0x7f5e100adb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:37.735 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:37.730+0000 7f5e0e59c640 1 --2- 172.21.15.169:0/2398668126 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5e100715f0 0x7f5e10150b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:37.735 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:37.730+0000 7f5e0ed9d640 1 -- 172.21.15.169:0/2398668126 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5e100715f0 msgr2=0x7f5e10150b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:37.736 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:37.730+0000 7f5e0ed9d640 1 --2- 172.21.15.169:0/2398668126 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5e100715f0 0x7f5e10150b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:37.736 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:37.730+0000 7f5e0ed9d640 1 -- 172.21.15.169:0/2398668126 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e101538c0 msgr2=0x7f5e100adb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:37.736 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:37.730+0000 7f5e0ed9d640 1 --2- 172.21.15.169:0/2398668126 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e101538c0 0x7f5e100adb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:37.736 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:37.730+0000 7f5e0ed9d640 1 -- 172.21.15.169:0/2398668126 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e00009560 con 0x7f5e1015a430 2023-10-26T01:43:37.736 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:37.730+0000 7f5e0e59c640 1 --2- 172.21.15.169:0/2398668126 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5e100715f0 0x7f5e10150b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T01:43:37.737 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:37.730+0000 7f5e0ed9d640 1 --2- 172.21.15.169:0/2398668126 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5e1015a430 0x7f5e100ae040 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f5e0800cce0 tx=0x7f5e0800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:37.738 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:37.731+0000 7f5e06ffd640 1 -- 172.21.15.169:0/2398668126 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e0800f450 con 0x7f5e1015a430 2023-10-26T01:43:37.738 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:37.731+0000 7f5e06ffd640 1 -- 172.21.15.169:0/2398668126 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5e0800f6a0 con 0x7f5e1015a430 2023-10-26T01:43:37.738 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:37.731+0000 7f5e0f59e640 1 -- 172.21.15.169:0/2398668126 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e1006b340 con 0x7f5e1015a430 2023-10-26T01:43:37.739 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:37.731+0000 7f5e06ffd640 1 -- 172.21.15.169:0/2398668126 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e0800fab0 con 0x7f5e1015a430 2023-10-26T01:43:37.739 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:37.731+0000 7f5e0f59e640 1 -- 172.21.15.169:0/2398668126 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e10151090 con 0x7f5e1015a430 2023-10-26T01:43:37.739 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:37.732+0000 7f5e0f59e640 1 -- 172.21.15.169:0/2398668126 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e10073c10 con 0x7f5e1015a430 2023-10-26T01:43:37.739 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:37.733+0000 7f5e06ffd640 1 -- 172.21.15.169:0/2398668126 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f5e0800fc10 con 0x7f5e1015a430 2023-10-26T01:43:37.740 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:37.734+0000 7f5e06ffd640 1 --2- 172.21.15.169:0/2398668126 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5df006ae40 0x7f5df006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:37.740 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:37.734+0000 7f5e06ffd640 1 -- 172.21.15.169:0/2398668126 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5386+0+0 (secure 0 0 0) 0x7f5e0808a120 con 0x7f5e1015a430 2023-10-26T01:43:37.740 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:37.734+0000 7f5e06ffd640 1 -- 172.21.15.169:0/2398668126 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=173}) v3 -- 0x7f5df0071c70 con 0x7f5e1015a430 2023-10-26T01:43:37.740 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:37.735+0000 7f5e0e59c640 1 --2- 172.21.15.169:0/2398668126 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5df006ae40 0x7f5df006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:37.741 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:37.736+0000 7f5e0e59c640 1 --2- 172.21.15.169:0/2398668126 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5df006ae40 0x7f5df006d300 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f5e00035000 tx=0x7f5e00004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:37.741 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:37.739+0000 7f5e06ffd640 1 -- 172.21.15.169:0/2398668126 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5e08055270 con 0x7f5e1015a430 2023-10-26T01:43:37.931 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:37.928+0000 7f5e0f59e640 1 -- 172.21.15.169:0/2398668126 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f5e100b1fb0 con 0x7f5e1015a430 2023-10-26T01:43:38.032 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T01:43:38.041 DEBUG:teuthology.orchestra.run.smithi170:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T01:43:38.183 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:38.180+0000 7f5e06ffd640 1 -- 172.21.15.169:0/2398668126 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v173) v1 ==== 169+0+0 (secure 0 0 0) 0x7f5e08058890 con 0x7f5e1015a430 2023-10-26T01:43:38.183 INFO:teuthology.orchestra.run.smithi169.stderr:pool 'cephfs_data' removed 2023-10-26T01:43:38.184 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:38.181+0000 7f5e06ffd640 1 -- 172.21.15.169:0/2398668126 <== mon.0 v2:172.21.15.169:3300/0 8 ==== osd_map(173..173 src has 1..173) v4 ==== 704+0+0 (secure 0 0 0) 0x7f5e08059630 con 0x7f5e1015a430 2023-10-26T01:43:38.184 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:38.181+0000 7f5e06ffd640 1 -- 172.21.15.169:0/2398668126 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=174}) v3 -- 0x7f5df0070a00 con 0x7f5e1015a430 2023-10-26T01:43:38.190 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:38.187+0000 7f5e0f59e640 1 -- 172.21.15.169:0/2398668126 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5df006ae40 msgr2=0x7f5df006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:38.190 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:38.187+0000 7f5e0f59e640 1 --2- 172.21.15.169:0/2398668126 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5df006ae40 0x7f5df006d300 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f5e00035000 tx=0x7f5e00004730 comp rx=0 tx=0).stop 2023-10-26T01:43:38.191 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:38.187+0000 7f5e0f59e640 1 -- 172.21.15.169:0/2398668126 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5e1015a430 msgr2=0x7f5e100ae040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:38.191 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:38.187+0000 7f5e0f59e640 1 --2- 172.21.15.169:0/2398668126 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5e1015a430 0x7f5e100ae040 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f5e0800cce0 tx=0x7f5e0800a620 comp rx=0 tx=0).stop 2023-10-26T01:43:38.192 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:38.189+0000 7f5e0f59e640 1 -- 172.21.15.169:0/2398668126 shutdown_connections 2023-10-26T01:43:38.193 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:38.189+0000 7f5e0f59e640 1 --2- 172.21.15.169:0/2398668126 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5df006ae40 0x7f5df006d300 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:38.193 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:38.189+0000 7f5e0f59e640 1 --2- 172.21.15.169:0/2398668126 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5e1015a430 0x7f5e100ae040 unknown :-1 s=CLOSED pgs=1790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:38.193 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:38.189+0000 7f5e0f59e640 1 --2- 172.21.15.169:0/2398668126 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e101538c0 0x7f5e100adb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:38.193 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:38.189+0000 7f5e0f59e640 1 --2- 172.21.15.169:0/2398668126 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5e100715f0 0x7f5e10150b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:38.194 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:38.189+0000 7f5e0f59e640 1 -- 172.21.15.169:0/2398668126 >> 172.21.15.169:0/2398668126 conn(0x7f5e10069b40 msgr2=0x7f5e10099a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:38.194 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:38.190+0000 7f5e0f59e640 1 -- 172.21.15.169:0/2398668126 shutdown_connections 2023-10-26T01:43:38.194 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:38.190+0000 7f5e0f59e640 1 -- 172.21.15.169:0/2398668126 wait complete. 2023-10-26T01:43:38.219 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 18 2023-10-26T01:43:38.478 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:38.475+0000 7f2325839640 1 -- 172.21.15.169:0/3434734342 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f232015b620 msgr2=0x7f23200cd710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:38.478 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:38.475+0000 7f2325839640 1 --2- 172.21.15.169:0/3434734342 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f232015b620 0x7f23200cd710 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f23200abb80 tx=0x7f231002bd10 comp rx=0 tx=0).stop 2023-10-26T01:43:38.479 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:38.475+0000 7f2325839640 1 -- 172.21.15.169:0/3434734342 shutdown_connections 2023-10-26T01:43:38.479 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:38.475+0000 7f2325839640 1 --2- 172.21.15.169:0/3434734342 >> v2:172.21.15.170:3300/0 conn(0x7f23200aca30 0x7f23200cdc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:38.479 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:38.475+0000 7f2325839640 1 --2- 172.21.15.169:0/3434734342 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f232015b620 0x7f23200cd710 unknown :-1 s=CLOSED pgs=1791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:38.479 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:38.475+0000 7f2325839640 1 -- 172.21.15.169:0/3434734342 >> 172.21.15.169:0/3434734342 conn(0x7f2320069930 msgr2=0x7f2320069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:38.479 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:38.475+0000 7f2325839640 1 -- 172.21.15.169:0/3434734342 shutdown_connections 2023-10-26T01:43:38.479 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:38.475+0000 7f2325839640 1 -- 172.21.15.169:0/3434734342 wait complete. 2023-10-26T01:43:38.481 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:38.477+0000 7f2325839640 1 Processor -- start 2023-10-26T01:43:38.481 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:38.477+0000 7f2325839640 1 -- start start 2023-10-26T01:43:38.481 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:38.478+0000 7f2325839640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f23200aca30 0x7f23201477c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:38.482 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:38.478+0000 7f2325839640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f23200c9d00 0x7f2320070af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:38.482 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:38.478+0000 7f2325839640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f232015b620 0x7f2320071030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:38.482 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:38.478+0000 7f2325839640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23200c86c0 con 0x7f23200c9d00 2023-10-26T01:43:38.482 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:38.478+0000 7f2325839640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2320147d00 con 0x7f232015b620 2023-10-26T01:43:38.482 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:38.478+0000 7f2325839640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23200c7ad0 con 0x7f23200aca30 2023-10-26T01:43:38.483 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:38.478+0000 7f23167fc640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f23200c9d00 0x7f2320070af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:38.483 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:38.478+0000 7f23167fc640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f23200c9d00 0x7f2320070af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:35780/0 (socket says 172.21.15.169:35780) 2023-10-26T01:43:38.483 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:38.478+0000 7f23167fc640 1 -- 172.21.15.169:0/207045062 learned_addr learned my addr 172.21.15.169:0/207045062 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:43:38.484 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:38.479+0000 7f23167fc640 1 -- 172.21.15.169:0/207045062 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f23200aca30 msgr2=0x7f23201477c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:38.485 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:38.479+0000 7f231effd640 1 --2- 172.21.15.169:0/207045062 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f23200aca30 0x7f23201477c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:38.485 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:38.479+0000 7f23167fc640 1 --2- 172.21.15.169:0/207045062 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f23200aca30 0x7f23201477c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:38.486 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:38.479+0000 7f23167fc640 1 -- 172.21.15.169:0/207045062 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f232015b620 msgr2=0x7f2320071030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:38.486 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:38.479+0000 7f231f7fe640 1 --2- 172.21.15.169:0/207045062 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f232015b620 0x7f2320071030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:38.486 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:38.479+0000 7f23167fc640 1 --2- 172.21.15.169:0/207045062 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f232015b620 0x7f2320071030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:38.486 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:38.479+0000 7f23167fc640 1 -- 172.21.15.169:0/207045062 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2310009560 con 0x7f23200c9d00 2023-10-26T01:43:38.487 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:38.479+0000 7f231f7fe640 1 --2- 172.21.15.169:0/207045062 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f232015b620 0x7f2320071030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:43:38.487 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:38.479+0000 7f23167fc640 1 --2- 172.21.15.169:0/207045062 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f23200c9d00 0x7f2320070af0 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f230800c410 tx=0x7f230800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:38.487 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:38.481+0000 7f231cff9640 1 -- 172.21.15.169:0/207045062 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23080035f0 con 0x7f23200c9d00 2023-10-26T01:43:38.487 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:38.481+0000 7f231cff9640 1 -- 172.21.15.169:0/207045062 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2308003750 con 0x7f23200c9d00 2023-10-26T01:43:38.488 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:38.481+0000 7f231cff9640 1 -- 172.21.15.169:0/207045062 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2308003ba0 con 0x7f23200c9d00 2023-10-26T01:43:38.488 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:38.481+0000 7f2325839640 1 -- 172.21.15.169:0/207045062 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23200c7d90 con 0x7f23200c9d00 2023-10-26T01:43:38.488 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:38.481+0000 7f2325839640 1 -- 172.21.15.169:0/207045062 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23200c8350 con 0x7f23200c9d00 2023-10-26T01:43:38.489 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:38.483+0000 7f2325839640 1 -- 172.21.15.169:0/207045062 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23200c8830 con 0x7f23200c9d00 2023-10-26T01:43:38.490 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:38.484+0000 7f231cff9640 1 -- 172.21.15.169:0/207045062 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f2308003d00 con 0x7f23200c9d00 2023-10-26T01:43:38.490 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:38.484+0000 7f231cff9640 1 --2- 172.21.15.169:0/207045062 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f230006ae70 0x7f230006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:38.490 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:38.485+0000 7f231cff9640 1 -- 172.21.15.169:0/207045062 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f23080885f0 con 0x7f23200c9d00 2023-10-26T01:43:38.490 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:38.485+0000 7f231effd640 1 --2- 172.21.15.169:0/207045062 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f230006ae70 0x7f230006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:38.491 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:38.486+0000 7f231effd640 1 --2- 172.21.15.169:0/207045062 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f230006ae70 0x7f230006d330 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f2310035000 tx=0x7f2310004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:38.492 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:38.489+0000 7f231cff9640 1 -- 172.21.15.169:0/207045062 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2308053a30 con 0x7f23200c9d00 2023-10-26T01:43:38.683 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:38.680+0000 7f2325839640 1 -- 172.21.15.169:0/207045062 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "config reset", "num": 18} v 0) v1 -- 0x7f232004db10 con 0x7f23200c9d00 2023-10-26T01:43:38.690 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:38.688+0000 7f231cff9640 1 -- 172.21.15.169:0/207045062 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 18}]=0 v36) v1 ==== 71+0+0 (secure 0 0 0) 0x7f2308057050 con 0x7f23200c9d00 2023-10-26T01:43:38.694 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:38.690+0000 7f2325839640 1 -- 172.21.15.169:0/207045062 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f230006ae70 msgr2=0x7f230006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:38.694 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:38.690+0000 7f2325839640 1 --2- 172.21.15.169:0/207045062 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f230006ae70 0x7f230006d330 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f2310035000 tx=0x7f2310004730 comp rx=0 tx=0).stop 2023-10-26T01:43:38.694 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:38.690+0000 7f2325839640 1 -- 172.21.15.169:0/207045062 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f23200c9d00 msgr2=0x7f2320070af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:38.695 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:38.690+0000 7f2325839640 1 --2- 172.21.15.169:0/207045062 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f23200c9d00 0x7f2320070af0 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f230800c410 tx=0x7f230800c8d0 comp rx=0 tx=0).stop 2023-10-26T01:43:38.695 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:38.691+0000 7f2325839640 1 -- 172.21.15.169:0/207045062 shutdown_connections 2023-10-26T01:43:38.695 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:38.691+0000 7f2325839640 1 --2- 172.21.15.169:0/207045062 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f230006ae70 0x7f230006d330 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:38.696 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:38.691+0000 7f2325839640 1 --2- 172.21.15.169:0/207045062 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f23200c9d00 0x7f2320070af0 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:38.696 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:38.691+0000 7f2325839640 1 --2- 172.21.15.169:0/207045062 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f232015b620 0x7f2320071030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:38.697 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:38.691+0000 7f2325839640 1 --2- 172.21.15.169:0/207045062 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f23200aca30 0x7f23201477c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:38.697 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:38.691+0000 7f2325839640 1 -- 172.21.15.169:0/207045062 >> 172.21.15.169:0/207045062 conn(0x7f2320069930 msgr2=0x7f2320150950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:38.697 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:38.691+0000 7f2325839640 1 -- 172.21.15.169:0/207045062 shutdown_connections 2023-10-26T01:43:38.697 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:38.691+0000 7f2325839640 1 -- 172.21.15.169:0/207045062 wait complete. 2023-10-26T01:43:38.713 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_fsync' 2023-10-26T01:43:38.933 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:38.930+0000 7f6411576640 1 -- 172.21.15.169:0/967751594 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63f8003970 msgr2=0x7f63f8003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:38.934 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:38.930+0000 7f6411576640 1 --2- 172.21.15.169:0/967751594 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63f8003970 0x7f63f8003d90 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f63fc009b80 tx=0x7f63f80057f0 comp rx=0 tx=0).stop 2023-10-26T01:43:38.935 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:38.930+0000 7f640b7fe640 1 -- 172.21.15.169:0/967751594 reap_dead start 2023-10-26T01:43:38.935 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:38.931+0000 7f6411576640 1 -- 172.21.15.169:0/967751594 shutdown_connections 2023-10-26T01:43:38.935 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:38.931+0000 7f6411576640 1 -- 172.21.15.169:0/967751594 >> 172.21.15.169:0/967751594 conn(0x7f640c069930 msgr2=0x7f640c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:38.935 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:38.931+0000 7f6411576640 1 -- 172.21.15.169:0/967751594 shutdown_connections 2023-10-26T01:43:38.935 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:38.931+0000 7f6411576640 1 -- 172.21.15.169:0/967751594 wait complete. 2023-10-26T01:43:38.937 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:38.932+0000 7f6411576640 1 Processor -- start 2023-10-26T01:43:38.937 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:38.933+0000 7f6411576640 1 -- start start 2023-10-26T01:43:38.937 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:38.933+0000 7f6411576640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f63f8003970 0x7f640c1f1f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:38.938 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:38.934+0000 7f6411576640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f640c0b0ed0 0x7f640c0b1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:38.938 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:38.934+0000 7f6411576640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f640c15bdc0 0x7f640c15c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:38.938 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:38.934+0000 7f6411576640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f640c0c7c40 con 0x7f640c15bdc0 2023-10-26T01:43:38.938 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:38.934+0000 7f6411576640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f640c0c7030 con 0x7f640c0b0ed0 2023-10-26T01:43:38.938 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:38.934+0000 7f6411576640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f640c0c7310 con 0x7f63f8003970 2023-10-26T01:43:38.939 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:38.934+0000 7f640b7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f640c15bdc0 0x7f640c15c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:38.939 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:38.934+0000 7f640b7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f640c15bdc0 0x7f640c15c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:35794/0 (socket says 172.21.15.169:35794) 2023-10-26T01:43:38.939 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:38.934+0000 7f640b7fe640 1 -- 172.21.15.169:0/930018834 learned_addr learned my addr 172.21.15.169:0/930018834 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:43:38.939 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:38.934+0000 7f6403fff640 1 --2- 172.21.15.169:0/930018834 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f640c0b0ed0 0x7f640c0b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:38.939 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:38.934+0000 7f640affd640 1 --2- 172.21.15.169:0/930018834 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f63f8003970 0x7f640c1f1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:38.940 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:38.934+0000 7f640b7fe640 1 -- 172.21.15.169:0/930018834 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f63f8003970 msgr2=0x7f640c1f1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:38.940 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:38.934+0000 7f640b7fe640 1 --2- 172.21.15.169:0/930018834 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f63f8003970 0x7f640c1f1f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:38.940 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:38.934+0000 7f640b7fe640 1 -- 172.21.15.169:0/930018834 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f640c0b0ed0 msgr2=0x7f640c0b1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:38.940 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:38.934+0000 7f640b7fe640 1 --2- 172.21.15.169:0/930018834 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f640c0b0ed0 0x7f640c0b1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:38.940 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:38.934+0000 7f640b7fe640 1 -- 172.21.15.169:0/930018834 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f640c1598f0 con 0x7f640c15bdc0 2023-10-26T01:43:38.941 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:38.934+0000 7f6403fff640 1 --2- 172.21.15.169:0/930018834 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f640c0b0ed0 0x7f640c0b1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:43:38.942 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:38.935+0000 7f640b7fe640 1 --2- 172.21.15.169:0/930018834 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f640c15bdc0 0x7f640c15c240 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f640c0b26b0 tx=0x7f640400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:38.942 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:38.935+0000 7f64037fe640 1 -- 172.21.15.169:0/930018834 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f640400ac70 con 0x7f640c15bdc0 2023-10-26T01:43:38.942 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:38.935+0000 7f64037fe640 1 -- 172.21.15.169:0/930018834 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f640400aec0 con 0x7f640c15bdc0 2023-10-26T01:43:38.943 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:38.935+0000 7f64037fe640 1 -- 172.21.15.169:0/930018834 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64040072d0 con 0x7f640c15bdc0 2023-10-26T01:43:38.943 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:38.935+0000 7f6411576640 1 -- 172.21.15.169:0/930018834 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f640c0c75f0 con 0x7f640c15bdc0 2023-10-26T01:43:38.943 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:38.935+0000 7f6411576640 1 -- 172.21.15.169:0/930018834 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f640c14ee30 con 0x7f640c15bdc0 2023-10-26T01:43:38.943 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:38.937+0000 7f6411576640 1 -- 172.21.15.169:0/930018834 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f640c0cb040 con 0x7f640c15bdc0 2023-10-26T01:43:38.943 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:38.937+0000 7f64037fe640 1 -- 172.21.15.169:0/930018834 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f6404007430 con 0x7f640c15bdc0 2023-10-26T01:43:38.944 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:38.939+0000 7f64037fe640 1 --2- 172.21.15.169:0/930018834 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f63ec06ae70 0x7f63ec06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:38.944 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:38.939+0000 7f64037fe640 1 -- 172.21.15.169:0/930018834 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f6404089230 con 0x7f640c15bdc0 2023-10-26T01:43:38.944 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:38.939+0000 7f640affd640 1 --2- 172.21.15.169:0/930018834 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f63ec06ae70 0x7f63ec06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:38.944 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:38.940+0000 7f640affd640 1 --2- 172.21.15.169:0/930018834 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f63ec06ae70 0x7f63ec06d330 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f63fc036000 tx=0x7f63fc035040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:38.946 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:38.943+0000 7f64037fe640 1 -- 172.21.15.169:0/930018834 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6404054670 con 0x7f640c15bdc0 2023-10-26T01:43:39.205 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:39.202+0000 7f6411576640 1 -- 172.21.15.169:0/930018834 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_fsync"]} v 0) v1 -- 0x7f640c0ce080 con 0x7f640c15bdc0 2023-10-26T01:43:39.485 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:39.482+0000 7f64037fe640 1 -- 172.21.15.169:0/930018834 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_fsync"]}]=0 v617) v1 ==== 131+0+0 (secure 0 0 0) 0x7f640c0ce080 con 0x7f640c15bdc0 2023-10-26T01:43:39.488 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:39.485+0000 7f6411576640 1 -- 172.21.15.169:0/930018834 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f63ec06ae70 msgr2=0x7f63ec06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:39.488 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:39.485+0000 7f6411576640 1 --2- 172.21.15.169:0/930018834 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f63ec06ae70 0x7f63ec06d330 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f63fc036000 tx=0x7f63fc035040 comp rx=0 tx=0).stop 2023-10-26T01:43:39.488 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:39.485+0000 7f6411576640 1 -- 172.21.15.169:0/930018834 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f640c15bdc0 msgr2=0x7f640c15c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:39.488 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:39.485+0000 7f6411576640 1 --2- 172.21.15.169:0/930018834 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f640c15bdc0 0x7f640c15c240 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f640c0b26b0 tx=0x7f640400a060 comp rx=0 tx=0).stop 2023-10-26T01:43:39.489 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:39.485+0000 7f6411576640 1 -- 172.21.15.169:0/930018834 shutdown_connections 2023-10-26T01:43:39.489 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:39.485+0000 7f6411576640 1 --2- 172.21.15.169:0/930018834 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f63ec06ae70 0x7f63ec06d330 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:39.489 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:39.485+0000 7f6411576640 1 --2- 172.21.15.169:0/930018834 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f640c15bdc0 0x7f640c15c240 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:39.490 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:39.485+0000 7f6411576640 1 --2- 172.21.15.169:0/930018834 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f640c0b0ed0 0x7f640c0b1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:39.490 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:39.485+0000 7f6411576640 1 --2- 172.21.15.169:0/930018834 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f63f8003970 0x7f640c1f1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:39.490 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:39.485+0000 7f6411576640 1 -- 172.21.15.169:0/930018834 >> 172.21.15.169:0/930018834 conn(0x7f640c069930 msgr2=0x7f640c0acfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:39.490 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:39.486+0000 7f6411576640 1 -- 172.21.15.169:0/930018834 shutdown_connections 2023-10-26T01:43:39.490 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:39.486+0000 7f6411576640 1 -- 172.21.15.169:0/930018834 wait complete. 2023-10-26T01:43:39.511 INFO:tasks.cephfs_test_runner:That when the full flag is encountered during asynchronous ... ok 2023-10-26T01:43:39.512 INFO:tasks.cephfs_test_runner:Starting test: test_full_same_file (tasks.cephfs.test_full.TestQuotaFull) 2023-10-26T01:43:39.512 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:43:39.733 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:39.730+0000 7efe05559640 1 -- 172.21.15.169:0/2967098910 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe0009ab50 msgr2=0x7efe000c8280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:39.734 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:39.730+0000 7efe05559640 1 --2- 172.21.15.169:0/2967098910 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe0009ab50 0x7efe000c8280 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7efe000c7d90 tx=0x7efdf402bd90 comp rx=0 tx=0).stop 2023-10-26T01:43:39.734 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:39.730+0000 7efe05559640 1 -- 172.21.15.169:0/2967098910 shutdown_connections 2023-10-26T01:43:39.734 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:39.730+0000 7efe05559640 1 --2- 172.21.15.169:0/2967098910 >> v2:172.21.15.170:3300/0 conn(0x7efe000c87c0 0x7efe0006b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:39.735 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:39.730+0000 7efe05559640 1 --2- 172.21.15.169:0/2967098910 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe0009ab50 0x7efe000c8280 unknown :-1 s=CLOSED pgs=1796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:39.735 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:39.730+0000 7efe05559640 1 --2- 172.21.15.169:0/2967098910 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7efe00158070 0x7efe0009a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:39.735 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:39.730+0000 7efe05559640 1 -- 172.21.15.169:0/2967098910 >> 172.21.15.169:0/2967098910 conn(0x7efe000aedd0 msgr2=0x7efe000af1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:39.735 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:39.730+0000 7efe05559640 1 -- 172.21.15.169:0/2967098910 shutdown_connections 2023-10-26T01:43:39.735 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:39.731+0000 7efe05559640 1 -- 172.21.15.169:0/2967098910 wait complete. 2023-10-26T01:43:39.738 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:39.732+0000 7efe05559640 1 Processor -- start 2023-10-26T01:43:39.738 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:39.732+0000 7efe05559640 1 -- start start 2023-10-26T01:43:39.738 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:39.733+0000 7efe05559640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe0009ab50 0x7efe00147340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:39.738 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:39.733+0000 7efe05559640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe000c87c0 0x7efe0014c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:39.738 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:39.733+0000 7efe05559640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7efe00158070 0x7efe00146120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:39.739 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:39.733+0000 7efe05559640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe00146660 con 0x7efe0009ab50 2023-10-26T01:43:39.739 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:39.733+0000 7efe05559640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe00147a40 con 0x7efe000c87c0 2023-10-26T01:43:39.739 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:39.733+0000 7efe05559640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe00147d20 con 0x7efe00158070 2023-10-26T01:43:39.739 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:39.734+0000 7efdfffff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe0009ab50 0x7efe00147340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:39.739 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:39.734+0000 7efdfffff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe0009ab50 0x7efe00147340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:35804/0 (socket says 172.21.15.169:35804) 2023-10-26T01:43:39.740 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:39.734+0000 7efdfffff640 1 -- 172.21.15.169:0/3930409010 learned_addr learned my addr 172.21.15.169:0/3930409010 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:43:39.740 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:39.734+0000 7efdff7fe640 1 --2- 172.21.15.169:0/3930409010 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe000c87c0 0x7efe0014c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:39.740 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:39.734+0000 7efdfffff640 1 -- 172.21.15.169:0/3930409010 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7efe00158070 msgr2=0x7efe00146120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:39.740 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:39.734+0000 7efdfffff640 1 --2- 172.21.15.169:0/3930409010 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7efe00158070 0x7efe00146120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:39.740 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:39.734+0000 7efdfffff640 1 -- 172.21.15.169:0/3930409010 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe000c87c0 msgr2=0x7efe0014c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:39.741 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:39.734+0000 7efdfffff640 1 --2- 172.21.15.169:0/3930409010 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe000c87c0 0x7efe0014c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:39.741 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:39.734+0000 7efdfffff640 1 -- 172.21.15.169:0/3930409010 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdf4009560 con 0x7efe0009ab50 2023-10-26T01:43:39.741 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:39.734+0000 7efdfffff640 1 --2- 172.21.15.169:0/3930409010 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe0009ab50 0x7efe00147340 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7efdf000b410 tx=0x7efdf000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:39.741 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:39.735+0000 7efdfcff9640 1 -- 172.21.15.169:0/3930409010 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf0011830 con 0x7efe0009ab50 2023-10-26T01:43:39.741 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:39.735+0000 7efdfcff9640 1 -- 172.21.15.169:0/3930409010 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7efdf0011aa0 con 0x7efe0009ab50 2023-10-26T01:43:39.742 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:39.735+0000 7efdfcff9640 1 -- 172.21.15.169:0/3930409010 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf0010380 con 0x7efe0009ab50 2023-10-26T01:43:39.742 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:39.735+0000 7efe05559640 1 -- 172.21.15.169:0/3930409010 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe00148000 con 0x7efe0009ab50 2023-10-26T01:43:39.742 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:39.735+0000 7efe05559640 1 -- 172.21.15.169:0/3930409010 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe000c8cd0 con 0x7efe0009ab50 2023-10-26T01:43:39.743 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:39.736+0000 7efe05559640 1 -- 172.21.15.169:0/3930409010 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe00073c10 con 0x7efe0009ab50 2023-10-26T01:43:39.743 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:39.737+0000 7efdfcff9640 1 -- 172.21.15.169:0/3930409010 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7efdf00104e0 con 0x7efe0009ab50 2023-10-26T01:43:39.743 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:39.738+0000 7efdfcff9640 1 --2- 172.21.15.169:0/3930409010 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7efdec06ae40 0x7efdec06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:39.743 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:39.738+0000 7efdfcff9640 1 -- 172.21.15.169:0/3930409010 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 4943+0+0 (secure 0 0 0) 0x7efdf00896d0 con 0x7efe0009ab50 2023-10-26T01:43:39.744 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:39.739+0000 7efdff7fe640 1 --2- 172.21.15.169:0/3930409010 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7efdec06ae40 0x7efdec06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:39.744 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:39.740+0000 7efdff7fe640 1 --2- 172.21.15.169:0/3930409010 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7efdec06ae40 0x7efdec06d300 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7efdf4035000 tx=0x7efdf4004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:39.745 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:39.743+0000 7efdfcff9640 1 -- 172.21.15.169:0/3930409010 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7efdf0054b10 con 0x7efe0009ab50 2023-10-26T01:43:39.928 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:39.925+0000 7efe05559640 1 -- 172.21.15.169:0/3930409010 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efe001585f0 con 0x7efe0009ab50 2023-10-26T01:43:39.930 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:39.927+0000 7efdfcff9640 1 -- 172.21.15.169:0/3930409010 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v173) v1 ==== 74+0+13045 (secure 0 0 0) 0x7efdf0058130 con 0x7efe0009ab50 2023-10-26T01:43:39.930 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:43:39.931 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":173,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:43:38.178294+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.169:6835/228678953":"2023-10-27T01:43:35.467825+0000","172.21.15.169:6833/228678953":"2023-10-27T01:43:35.467825+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:43:39.933 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:39.930+0000 7efe05559640 1 -- 172.21.15.169:0/3930409010 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7efdec06ae40 msgr2=0x7efdec06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:39.933 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:39.930+0000 7efe05559640 1 --2- 172.21.15.169:0/3930409010 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7efdec06ae40 0x7efdec06d300 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7efdf4035000 tx=0x7efdf4004730 comp rx=0 tx=0).stop 2023-10-26T01:43:39.933 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:39.930+0000 7efe05559640 1 -- 172.21.15.169:0/3930409010 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe0009ab50 msgr2=0x7efe00147340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:39.933 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:39.930+0000 7efe05559640 1 --2- 172.21.15.169:0/3930409010 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe0009ab50 0x7efe00147340 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7efdf000b410 tx=0x7efdf000b8d0 comp rx=0 tx=0).stop 2023-10-26T01:43:39.934 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:39.930+0000 7efe05559640 1 -- 172.21.15.169:0/3930409010 shutdown_connections 2023-10-26T01:43:39.934 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:39.930+0000 7efe05559640 1 --2- 172.21.15.169:0/3930409010 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7efdec06ae40 0x7efdec06d300 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:39.934 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:39.930+0000 7efe05559640 1 --2- 172.21.15.169:0/3930409010 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7efe00158070 0x7efe00146120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:39.934 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:39.930+0000 7efe05559640 1 --2- 172.21.15.169:0/3930409010 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe000c87c0 0x7efe0014c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:39.934 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:39.930+0000 7efe05559640 1 --2- 172.21.15.169:0/3930409010 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe0009ab50 0x7efe00147340 unknown :-1 s=CLOSED pgs=1797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:39.935 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:39.930+0000 7efe05559640 1 -- 172.21.15.169:0/3930409010 >> 172.21.15.169:0/3930409010 conn(0x7efe000aedd0 msgr2=0x7efe000ab860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:39.935 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:39.930+0000 7efe05559640 1 -- 172.21.15.169:0/3930409010 shutdown_connections 2023-10-26T01:43:39.935 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:39.930+0000 7efe05559640 1 -- 172.21.15.169:0/3930409010 wait complete. 2023-10-26T01:43:39.951 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T01:30:32.200808+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '109', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T01:43:39.952 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T01:43:40.161 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:40.158+0000 7f7abcce5640 1 -- 172.21.15.169:0/356894743 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ab80cd690 msgr2=0x7f7ab80cdb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:40.162 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:40.158+0000 7f7abcce5640 1 --2- 172.21.15.169:0/356894743 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ab80cd690 0x7f7ab80cdb20 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f7ab0005fd0 tx=0x7f7ab002c240 comp rx=0 tx=0).stop 2023-10-26T01:43:40.162 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:40.158+0000 7f7abcce5640 1 -- 172.21.15.169:0/356894743 shutdown_connections 2023-10-26T01:43:40.162 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:40.158+0000 7f7abcce5640 1 --2- 172.21.15.169:0/356894743 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ab80cd690 0x7f7ab80cdb20 unknown :-1 s=CLOSED pgs=1799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:40.163 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:40.158+0000 7f7abcce5640 1 --2- 172.21.15.169:0/356894743 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7ab80aaf80 0x7f7ab80ab3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:40.163 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:40.158+0000 7f7abcce5640 1 -- 172.21.15.169:0/356894743 >> 172.21.15.169:0/356894743 conn(0x7f7ab8069930 msgr2=0x7f7ab8069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:40.163 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:40.159+0000 7f7abcce5640 1 -- 172.21.15.169:0/356894743 shutdown_connections 2023-10-26T01:43:40.163 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:40.159+0000 7f7abcce5640 1 -- 172.21.15.169:0/356894743 wait complete. 2023-10-26T01:43:40.166 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:40.160+0000 7f7abcce5640 1 Processor -- start 2023-10-26T01:43:40.166 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:40.160+0000 7f7abcce5640 1 -- start start 2023-10-26T01:43:40.166 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:40.161+0000 7f7abcce5640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7ab80aaf80 0x7f7ab814cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:40.166 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:40.161+0000 7f7abcce5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ab80cd690 0x7f7ab806ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:40.167 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:40.161+0000 7f7abcce5640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ab8159480 0x7f7ab81e2f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:40.167 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:40.161+0000 7f7abcce5640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab806be10 con 0x7f7ab8159480 2023-10-26T01:43:40.167 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:40.161+0000 7f7abcce5640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab814d060 con 0x7f7ab80cd690 2023-10-26T01:43:40.167 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:40.161+0000 7f7abcce5640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab806b190 con 0x7f7ab80aaf80 2023-10-26T01:43:40.167 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:40.162+0000 7f7ab6d9d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ab8159480 0x7f7ab81e2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:40.168 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:40.162+0000 7f7ab6d9d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ab8159480 0x7f7ab81e2f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:35820/0 (socket says 172.21.15.169:35820) 2023-10-26T01:43:40.168 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:40.162+0000 7f7ab6d9d640 1 -- 172.21.15.169:0/2034448165 learned_addr learned my addr 172.21.15.169:0/2034448165 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:43:40.168 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:40.162+0000 7f7ab5d9b640 1 --2- 172.21.15.169:0/2034448165 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ab80cd690 0x7f7ab806ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:40.168 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:40.162+0000 7f7ab6d9d640 1 -- 172.21.15.169:0/2034448165 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7ab80aaf80 msgr2=0x7f7ab814cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:40.168 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:40.162+0000 7f7ab6d9d640 1 --2- 172.21.15.169:0/2034448165 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7ab80aaf80 0x7f7ab814cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:40.169 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:40.162+0000 7f7ab6d9d640 1 -- 172.21.15.169:0/2034448165 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ab80cd690 msgr2=0x7f7ab806ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:40.169 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:40.162+0000 7f7ab6d9d640 1 --2- 172.21.15.169:0/2034448165 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ab80cd690 0x7f7ab806ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:40.169 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:40.162+0000 7f7ab6d9d640 1 -- 172.21.15.169:0/2034448165 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ab0009900 con 0x7f7ab8159480 2023-10-26T01:43:40.169 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:40.162+0000 7f7ab5d9b640 1 --2- 172.21.15.169:0/2034448165 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ab80cd690 0x7f7ab806ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:43:40.169 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:40.162+0000 7f7ab6d9d640 1 --2- 172.21.15.169:0/2034448165 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ab8159480 0x7f7ab81e2f60 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7f7ab0037040 tx=0x7f7ab0035a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:40.169 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:40.163+0000 7f7aa6ffd640 1 -- 172.21.15.169:0/2034448165 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab0034440 con 0x7f7ab8159480 2023-10-26T01:43:40.170 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:40.163+0000 7f7aa6ffd640 1 -- 172.21.15.169:0/2034448165 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7ab0004330 con 0x7f7ab8159480 2023-10-26T01:43:40.170 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:40.163+0000 7f7aa6ffd640 1 -- 172.21.15.169:0/2034448165 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab00046e0 con 0x7f7ab8159480 2023-10-26T01:43:40.170 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:40.163+0000 7f7abcce5640 1 -- 172.21.15.169:0/2034448165 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ab806b470 con 0x7f7ab8159480 2023-10-26T01:43:40.171 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:40.163+0000 7f7abcce5640 1 -- 172.21.15.169:0/2034448165 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ab806ba30 con 0x7f7ab8159480 2023-10-26T01:43:40.172 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:40.164+0000 7f7abcce5640 1 -- 172.21.15.169:0/2034448165 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ab8066af0 con 0x7f7ab8159480 2023-10-26T01:43:40.172 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:40.165+0000 7f7aa6ffd640 1 -- 172.21.15.169:0/2034448165 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f7ab003b040 con 0x7f7ab8159480 2023-10-26T01:43:40.172 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:40.166+0000 7f7aa6ffd640 1 --2- 172.21.15.169:0/2034448165 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7a9806ae40 0x7f7a9806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:40.172 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:40.166+0000 7f7aa6ffd640 1 -- 172.21.15.169:0/2034448165 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f7ab00abfb0 con 0x7f7ab8159480 2023-10-26T01:43:40.172 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:40.167+0000 7f7ab659c640 1 --2- 172.21.15.169:0/2034448165 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7a9806ae40 0x7f7a9806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:40.173 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:40.168+0000 7f7ab659c640 1 --2- 172.21.15.169:0/2034448165 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7a9806ae40 0x7f7a9806d300 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f7aa8000f60 tx=0x7f7aa80079b0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:40.174 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:40.171+0000 7f7aa6ffd640 1 -- 172.21.15.169:0/2034448165 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7ab0077350 con 0x7f7ab8159480 2023-10-26T01:43:40.361 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:40.358+0000 7f7abcce5640 1 -- 172.21.15.169:0/2034448165 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7ab806bf50 con 0x7f7ab8159480 2023-10-26T01:43:40.362 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:40.359+0000 7f7aa6ffd640 1 -- 172.21.15.169:0/2034448165 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v173) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7ab806bf50 con 0x7f7ab8159480 2023-10-26T01:43:40.362 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 1 2023-10-26T01:43:40.364 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:40.361+0000 7f7abcce5640 1 -- 172.21.15.169:0/2034448165 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7a9806ae40 msgr2=0x7f7a9806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:40.364 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:40.361+0000 7f7abcce5640 1 --2- 172.21.15.169:0/2034448165 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7a9806ae40 0x7f7a9806d300 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f7aa8000f60 tx=0x7f7aa80079b0 comp rx=0 tx=0).stop 2023-10-26T01:43:40.364 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:40.361+0000 7f7abcce5640 1 -- 172.21.15.169:0/2034448165 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ab8159480 msgr2=0x7f7ab81e2f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:40.365 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:40.361+0000 7f7abcce5640 1 --2- 172.21.15.169:0/2034448165 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ab8159480 0x7f7ab81e2f60 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7f7ab0037040 tx=0x7f7ab0035a40 comp rx=0 tx=0).stop 2023-10-26T01:43:40.365 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:40.361+0000 7f7abcce5640 1 -- 172.21.15.169:0/2034448165 shutdown_connections 2023-10-26T01:43:40.365 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:40.361+0000 7f7abcce5640 1 --2- 172.21.15.169:0/2034448165 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7a9806ae40 0x7f7a9806d300 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:40.365 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:40.361+0000 7f7abcce5640 1 --2- 172.21.15.169:0/2034448165 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ab8159480 0x7f7ab81e2f60 unknown :-1 s=CLOSED pgs=1800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:40.365 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:40.361+0000 7f7abcce5640 1 --2- 172.21.15.169:0/2034448165 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ab80cd690 0x7f7ab806ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:40.366 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:40.361+0000 7f7abcce5640 1 --2- 172.21.15.169:0/2034448165 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7ab80aaf80 0x7f7ab814cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:40.366 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:40.361+0000 7f7abcce5640 1 -- 172.21.15.169:0/2034448165 >> 172.21.15.169:0/2034448165 conn(0x7f7ab8069930 msgr2=0x7f7ab80cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:40.366 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:40.361+0000 7f7abcce5640 1 -- 172.21.15.169:0/2034448165 shutdown_connections 2023-10-26T01:43:40.366 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:40.361+0000 7f7abcce5640 1 -- 172.21.15.169:0/2034448165 wait complete. 2023-10-26T01:43:40.383 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_same_file' 2023-10-26T01:43:40.601 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:40.598+0000 7f2ea759e640 1 -- 172.21.15.169:0/2390183357 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ea80ac6d0 msgr2=0x7f2ea815d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:40.601 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:40.598+0000 7f2ea759e640 1 --2- 172.21.15.169:0/2390183357 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ea80ac6d0 0x7f2ea815d840 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f2e98009830 tx=0x7f2e9802bd60 comp rx=0 tx=0).stop 2023-10-26T01:43:40.602 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:40.598+0000 7f2ea759e640 1 -- 172.21.15.169:0/2390183357 shutdown_connections 2023-10-26T01:43:40.602 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:40.598+0000 7f2ea759e640 1 --2- 172.21.15.169:0/2390183357 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2ea809a5c0 0x7f2ea809aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:40.602 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:40.598+0000 7f2ea759e640 1 --2- 172.21.15.169:0/2390183357 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ea80ac6d0 0x7f2ea815d840 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:40.602 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:40.598+0000 7f2ea759e640 1 -- 172.21.15.169:0/2390183357 >> 172.21.15.169:0/2390183357 conn(0x7f2ea80aedd0 msgr2=0x7f2ea80af1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:40.602 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:40.598+0000 7f2ea759e640 1 -- 172.21.15.169:0/2390183357 shutdown_connections 2023-10-26T01:43:40.603 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:40.598+0000 7f2ea759e640 1 -- 172.21.15.169:0/2390183357 wait complete. 2023-10-26T01:43:40.603 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:40.600+0000 7f2ea759e640 1 Processor -- start 2023-10-26T01:43:40.603 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:40.601+0000 7f2ea759e640 1 -- start start 2023-10-26T01:43:40.604 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:40.602+0000 7f2ea759e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2ea809a5c0 0x7f2ea81e36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:40.605 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:40.602+0000 7f2ea759e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ea80ac6d0 0x7f2ea80b27a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:40.605 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:40.602+0000 7f2ea759e640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2ea80c9970 0x7f2ea80b0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:40.605 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:40.602+0000 7f2ea759e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea80b2ce0 con 0x7f2ea809a5c0 2023-10-26T01:43:40.605 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:40.602+0000 7f2ea759e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea81e3d40 con 0x7f2ea80ac6d0 2023-10-26T01:43:40.606 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:40.602+0000 7f2ea759e640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea81e4020 con 0x7f2ea80c9970 2023-10-26T01:43:40.606 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:40.602+0000 7f2ea659c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2ea809a5c0 0x7f2ea81e36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:40.606 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:40.602+0000 7f2ea659c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2ea809a5c0 0x7f2ea81e36b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:35826/0 (socket says 172.21.15.169:35826) 2023-10-26T01:43:40.606 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:40.602+0000 7f2ea659c640 1 -- 172.21.15.169:0/2752974984 learned_addr learned my addr 172.21.15.169:0/2752974984 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:43:40.607 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:40.603+0000 7f2e9ffff640 1 --2- 172.21.15.169:0/2752974984 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ea80ac6d0 0x7f2ea80b27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:40.608 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:40.603+0000 7f2ea6d9d640 1 --2- 172.21.15.169:0/2752974984 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2ea80c9970 0x7f2ea80b0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:40.608 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:40.603+0000 7f2ea659c640 1 -- 172.21.15.169:0/2752974984 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2ea80c9970 msgr2=0x7f2ea80b0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:40.608 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:40.603+0000 7f2ea659c640 1 --2- 172.21.15.169:0/2752974984 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2ea80c9970 0x7f2ea80b0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:40.608 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:40.603+0000 7f2ea659c640 1 -- 172.21.15.169:0/2752974984 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ea80ac6d0 msgr2=0x7f2ea80b27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:40.609 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:40.603+0000 7f2ea659c640 1 --2- 172.21.15.169:0/2752974984 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ea80ac6d0 0x7f2ea80b27a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:40.609 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:40.603+0000 7f2ea659c640 1 -- 172.21.15.169:0/2752974984 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e98009490 con 0x7f2ea809a5c0 2023-10-26T01:43:40.609 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:40.603+0000 7f2ea6d9d640 1 --2- 172.21.15.169:0/2752974984 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2ea80c9970 0x7f2ea80b0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:43:40.609 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:40.603+0000 7f2ea659c640 1 --2- 172.21.15.169:0/2752974984 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2ea809a5c0 0x7f2ea81e36b0 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f2e98009780 tx=0x7f2e980038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:40.609 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:40.603+0000 7f2e9effd640 1 -- 172.21.15.169:0/2752974984 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e98003ea0 con 0x7f2ea809a5c0 2023-10-26T01:43:40.609 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:40.603+0000 7f2e9effd640 1 -- 172.21.15.169:0/2752974984 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2e98034440 con 0x7f2ea809a5c0 2023-10-26T01:43:40.610 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:40.603+0000 7f2e9effd640 1 -- 172.21.15.169:0/2752974984 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e98034850 con 0x7f2ea809a5c0 2023-10-26T01:43:40.610 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:40.603+0000 7f2ea759e640 1 -- 172.21.15.169:0/2752974984 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ea81e4300 con 0x7f2ea809a5c0 2023-10-26T01:43:40.610 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:40.604+0000 7f2ea759e640 1 -- 172.21.15.169:0/2752974984 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ea813e400 con 0x7f2ea809a5c0 2023-10-26T01:43:40.610 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:40.605+0000 7f2ea759e640 1 -- 172.21.15.169:0/2752974984 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ea80ccb10 con 0x7f2ea809a5c0 2023-10-26T01:43:40.611 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:40.606+0000 7f2e9effd640 1 -- 172.21.15.169:0/2752974984 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f2e9803b040 con 0x7f2ea809a5c0 2023-10-26T01:43:40.611 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:40.606+0000 7f2e9effd640 1 --2- 172.21.15.169:0/2752974984 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2e8806ae40 0x7f2e8806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:40.611 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:40.607+0000 7f2e9effd640 1 -- 172.21.15.169:0/2752974984 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f2e980ab860 con 0x7f2ea809a5c0 2023-10-26T01:43:40.612 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:40.607+0000 7f2e9ffff640 1 --2- 172.21.15.169:0/2752974984 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2e8806ae40 0x7f2e8806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:40.612 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:40.608+0000 7f2e9ffff640 1 --2- 172.21.15.169:0/2752974984 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2e8806ae40 0x7f2e8806d300 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f2e90000f60 tx=0x7f2e90009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:40.613 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:40.611+0000 7f2e9effd640 1 -- 172.21.15.169:0/2752974984 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2e98076c10 con 0x7f2ea809a5c0 2023-10-26T01:43:40.880 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:40.877+0000 7f2ea759e640 1 -- 172.21.15.169:0/2752974984 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_same_file"]} v 0) v1 -- 0x7f2ea8073c10 con 0x7f2ea809a5c0 2023-10-26T01:43:41.496 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:41.493+0000 7f2e9effd640 1 -- 172.21.15.169:0/2752974984 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_same_file"]}]=0 v619) v1 ==== 138+0+0 (secure 0 0 0) 0x7f2e98039020 con 0x7f2ea809a5c0 2023-10-26T01:43:41.500 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:41.497+0000 7f2ea759e640 1 -- 172.21.15.169:0/2752974984 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2e8806ae40 msgr2=0x7f2e8806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:41.500 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:41.497+0000 7f2ea759e640 1 --2- 172.21.15.169:0/2752974984 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2e8806ae40 0x7f2e8806d300 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f2e90000f60 tx=0x7f2e90009040 comp rx=0 tx=0).stop 2023-10-26T01:43:41.501 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:41.497+0000 7f2ea759e640 1 -- 172.21.15.169:0/2752974984 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2ea809a5c0 msgr2=0x7f2ea81e36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:41.501 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:41.497+0000 7f2ea759e640 1 --2- 172.21.15.169:0/2752974984 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2ea809a5c0 0x7f2ea81e36b0 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f2e98009780 tx=0x7f2e980038a0 comp rx=0 tx=0).stop 2023-10-26T01:43:41.502 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:41.497+0000 7f2ea759e640 1 -- 172.21.15.169:0/2752974984 shutdown_connections 2023-10-26T01:43:41.502 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:41.497+0000 7f2ea759e640 1 --2- 172.21.15.169:0/2752974984 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2e8806ae40 0x7f2e8806d300 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:41.502 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:41.497+0000 7f2ea759e640 1 --2- 172.21.15.169:0/2752974984 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2ea80c9970 0x7f2ea80b0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:41.503 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:41.497+0000 7f2ea759e640 1 --2- 172.21.15.169:0/2752974984 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ea80ac6d0 0x7f2ea80b27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:41.503 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:41.497+0000 7f2ea759e640 1 --2- 172.21.15.169:0/2752974984 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2ea809a5c0 0x7f2ea81e36b0 unknown :-1 s=CLOSED pgs=1801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:41.503 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:41.497+0000 7f2ea759e640 1 -- 172.21.15.169:0/2752974984 >> 172.21.15.169:0/2752974984 conn(0x7f2ea80aedd0 msgr2=0x7f2ea80c6270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:41.504 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:41.497+0000 7f2ea759e640 1 -- 172.21.15.169:0/2752974984 shutdown_connections 2023-10-26T01:43:41.504 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:41.497+0000 7f2ea759e640 1 -- 172.21.15.169:0/2752974984 wait complete. 2023-10-26T01:43:41.522 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2023-10-26T01:43:41.737 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:41.734+0000 7f9051cc2640 1 -- 172.21.15.169:0/1533578161 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f90380047c0 msgr2=0x7f9038004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:41.737 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:41.734+0000 7f9051cc2640 1 --2- 172.21.15.169:0/1533578161 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f90380047c0 0x7f9038004c40 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7f903400baf0 tx=0x7f90380057f0 comp rx=0 tx=0).stop 2023-10-26T01:43:41.738 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:41.734+0000 7f9051cc2640 1 -- 172.21.15.169:0/1533578161 shutdown_connections 2023-10-26T01:43:41.738 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:41.734+0000 7f9051cc2640 1 --2- 172.21.15.169:0/1533578161 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f90380047c0 0x7f9038004c40 unknown :-1 s=CLOSED pgs=1803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:41.738 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:41.734+0000 7f9051cc2640 1 -- 172.21.15.169:0/1533578161 >> 172.21.15.169:0/1533578161 conn(0x7f904c069930 msgr2=0x7f904c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:41.738 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:41.734+0000 7f9051cc2640 1 -- 172.21.15.169:0/1533578161 shutdown_connections 2023-10-26T01:43:41.739 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:41.734+0000 7f9051cc2640 1 -- 172.21.15.169:0/1533578161 wait complete. 2023-10-26T01:43:41.740 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:41.736+0000 7f9051cc2640 1 Processor -- start 2023-10-26T01:43:41.740 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:41.736+0000 7f9051cc2640 1 -- start start 2023-10-26T01:43:41.741 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:41.737+0000 7f9051cc2640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f90380047c0 0x7f904c0aecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:41.741 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:41.737+0000 7f9051cc2640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f904c0af710 0x7f904c14ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:41.741 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:41.737+0000 7f9051cc2640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f904c070400 0x7f904c070880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:41.742 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:41.737+0000 7f9051cc2640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f904c0af320 con 0x7f904c0af710 2023-10-26T01:43:41.742 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:41.737+0000 7f9051cc2640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f904c070dc0 con 0x7f90380047c0 2023-10-26T01:43:41.742 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:41.737+0000 7f9051cc2640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f904c0ae6c0 con 0x7f904c070400 2023-10-26T01:43:41.742 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:41.737+0000 7f9043fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f904c0af710 0x7f904c14ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:41.743 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:41.737+0000 7f9043fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f904c0af710 0x7f904c14ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:35838/0 (socket says 172.21.15.169:35838) 2023-10-26T01:43:41.743 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:41.737+0000 7f9043fff640 1 -- 172.21.15.169:0/2548752949 learned_addr learned my addr 172.21.15.169:0/2548752949 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:43:41.743 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:41.738+0000 7f904b7fe640 1 --2- 172.21.15.169:0/2548752949 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f90380047c0 0x7f904c0aecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:41.743 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:41.738+0000 7f9043fff640 1 -- 172.21.15.169:0/2548752949 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f904c070400 msgr2=0x7f904c070880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:41.743 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:41.738+0000 7f9043fff640 1 --2- 172.21.15.169:0/2548752949 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f904c070400 0x7f904c070880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:41.744 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:41.738+0000 7f9043fff640 1 -- 172.21.15.169:0/2548752949 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f90380047c0 msgr2=0x7f904c0aecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:41.744 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:41.738+0000 7f9043fff640 1 --2- 172.21.15.169:0/2548752949 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f90380047c0 0x7f904c0aecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:41.744 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:41.738+0000 7f9043fff640 1 -- 172.21.15.169:0/2548752949 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f904c0ad900 con 0x7f904c0af710 2023-10-26T01:43:41.744 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:41.738+0000 7f9043fff640 1 --2- 172.21.15.169:0/2548752949 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f904c0af710 0x7f904c14ee90 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f903400baf0 tx=0x7f9034036700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:41.746 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:41.738+0000 7f9048ff9640 1 -- 172.21.15.169:0/2548752949 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9034036d00 con 0x7f904c0af710 2023-10-26T01:43:41.746 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:41.738+0000 7f9048ff9640 1 -- 172.21.15.169:0/2548752949 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9034004030 con 0x7f904c0af710 2023-10-26T01:43:41.746 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:41.738+0000 7f9048ff9640 1 -- 172.21.15.169:0/2548752949 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9034004420 con 0x7f904c0af710 2023-10-26T01:43:41.747 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:41.738+0000 7f9051cc2640 1 -- 172.21.15.169:0/2548752949 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f904c0adbe0 con 0x7f904c0af710 2023-10-26T01:43:41.747 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:41.738+0000 7f9051cc2640 1 -- 172.21.15.169:0/2548752949 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f904c0ae1a0 con 0x7f904c0af710 2023-10-26T01:43:41.747 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:41.740+0000 7f9051cc2640 1 -- 172.21.15.169:0/2548752949 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f904c0ae330 con 0x7f904c0af710 2023-10-26T01:43:41.747 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:41.741+0000 7f9048ff9640 1 -- 172.21.15.169:0/2548752949 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f90340035f0 con 0x7f904c0af710 2023-10-26T01:43:41.747 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:41.741+0000 7f9048ff9640 1 --2- 172.21.15.169:0/2548752949 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f902c06ae70 0x7f902c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:41.748 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:41.741+0000 7f9048ff9640 1 -- 172.21.15.169:0/2548752949 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f90340ab4b0 con 0x7f904c0af710 2023-10-26T01:43:41.748 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:41.742+0000 7f904b7fe640 1 --2- 172.21.15.169:0/2548752949 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f902c06ae70 0x7f902c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:41.748 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:41.743+0000 7f904b7fe640 1 --2- 172.21.15.169:0/2548752949 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f902c06ae70 0x7f902c06d330 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f903c001ce0 tx=0x7f903c001dd0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:41.749 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:41.746+0000 7f9048ff9640 1 -- 172.21.15.169:0/2548752949 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9034076860 con 0x7f904c0af710 2023-10-26T01:43:41.937 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:41.934+0000 7f9051cc2640 1 -- 172.21.15.169:0/2548752949 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "config dump"} v 0) v1 -- 0x7f904c04db10 con 0x7f904c0af710 2023-10-26T01:43:41.938 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:41.935+0000 7f9048ff9640 1 -- 172.21.15.169:0/2548752949 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v36) v1 ==== 59+0+1116 (secure 0 0 0) 0x7f9034032470 con 0x7f904c0af710 2023-10-26T01:43:41.938 INFO:teuthology.orchestra.run.smithi169.stdout:WHO MASK LEVEL OPTION VALUE RO 2023-10-26T01:43:41.938 INFO:teuthology.orchestra.run.smithi169.stdout:mon advanced mon_op_complaint_time 120 2023-10-26T01:43:41.938 INFO:teuthology.orchestra.run.smithi169.stdout:osd advanced osd_op_complaint_time 180.000000 2023-10-26T01:43:41.939 INFO:teuthology.orchestra.run.smithi169.stdout:mds advanced debug_mds 20/20 2023-10-26T01:43:41.939 INFO:teuthology.orchestra.run.smithi169.stdout:mds advanced debug_mds_balancer 20/20 2023-10-26T01:43:41.939 INFO:teuthology.orchestra.run.smithi169.stdout:mds advanced debug_ms 1/1 2023-10-26T01:43:41.939 INFO:teuthology.orchestra.run.smithi169.stdout:mds dev mds_debug_frag true 2023-10-26T01:43:41.939 INFO:teuthology.orchestra.run.smithi169.stdout:mds dev mds_debug_scatterstat true 2023-10-26T01:43:41.940 INFO:teuthology.orchestra.run.smithi169.stdout:mds advanced mds_op_complaint_time 180.000000 2023-10-26T01:43:41.940 INFO:teuthology.orchestra.run.smithi169.stdout:mds dev mds_verify_scatter true 2023-10-26T01:43:41.940 INFO:teuthology.orchestra.run.smithi169.stdout:mds advanced osd_op_complaint_time 180.000000 2023-10-26T01:43:41.940 INFO:teuthology.orchestra.run.smithi169.stdout:mds advanced rados_mon_op_timeout 900 2023-10-26T01:43:41.940 INFO:teuthology.orchestra.run.smithi169.stdout:mds advanced rados_osd_op_timeout 900 2023-10-26T01:43:41.941 INFO:teuthology.orchestra.run.smithi169.stdout:client advanced client_mount_timeout 600 2023-10-26T01:43:41.941 INFO:teuthology.orchestra.run.smithi169.stdout:client advanced debug_client 20/20 2023-10-26T01:43:41.941 INFO:teuthology.orchestra.run.smithi169.stdout:client advanced debug_ms 1/1 2023-10-26T01:43:41.941 INFO:teuthology.orchestra.run.smithi169.stdout:client advanced rados_mon_op_timeout 900 2023-10-26T01:43:41.941 INFO:teuthology.orchestra.run.smithi169.stdout:client advanced rados_osd_op_timeout 900 2023-10-26T01:43:41.943 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:41.937+0000 7f9051cc2640 1 -- 172.21.15.169:0/2548752949 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f902c06ae70 msgr2=0x7f902c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:41.943 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:41.937+0000 7f9051cc2640 1 --2- 172.21.15.169:0/2548752949 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f902c06ae70 0x7f902c06d330 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f903c001ce0 tx=0x7f903c001dd0 comp rx=0 tx=0).stop 2023-10-26T01:43:41.943 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:41.937+0000 7f9051cc2640 1 -- 172.21.15.169:0/2548752949 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f904c0af710 msgr2=0x7f904c14ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:41.943 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:41.937+0000 7f9051cc2640 1 --2- 172.21.15.169:0/2548752949 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f904c0af710 0x7f904c14ee90 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f903400baf0 tx=0x7f9034036700 comp rx=0 tx=0).stop 2023-10-26T01:43:41.944 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:41.938+0000 7f9051cc2640 1 -- 172.21.15.169:0/2548752949 shutdown_connections 2023-10-26T01:43:41.944 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:41.938+0000 7f9051cc2640 1 --2- 172.21.15.169:0/2548752949 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f902c06ae70 0x7f902c06d330 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:41.944 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:41.938+0000 7f9051cc2640 1 --2- 172.21.15.169:0/2548752949 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f904c0af710 0x7f904c14ee90 unknown :-1 s=CLOSED pgs=1804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:41.944 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:41.938+0000 7f9051cc2640 1 --2- 172.21.15.169:0/2548752949 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f904c070400 0x7f904c070880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:41.944 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:41.938+0000 7f9051cc2640 1 --2- 172.21.15.169:0/2548752949 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f90380047c0 0x7f904c0aecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:41.945 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:41.938+0000 7f9051cc2640 1 -- 172.21.15.169:0/2548752949 >> 172.21.15.169:0/2548752949 conn(0x7f904c069930 msgr2=0x7f904c155910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:41.945 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:41.938+0000 7f9051cc2640 1 -- 172.21.15.169:0/2548752949 shutdown_connections 2023-10-26T01:43:41.945 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:41.938+0000 7f9051cc2640 1 -- 172.21.15.169:0/2548752949 wait complete. 2023-10-26T01:43:41.958 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-26T01:43:42.179 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:42.176+0000 7fe4124e5640 1 -- 172.21.15.169:0/2570356289 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe40c15b9e0 msgr2=0x7fe40c06ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:42.179 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:42.176+0000 7fe4124e5640 1 --2- 172.21.15.169:0/2570356289 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe40c15b9e0 0x7fe40c06ad90 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7fe40c0b3500 tx=0x7fe40402bf20 comp rx=0 tx=0).stop 2023-10-26T01:43:42.180 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:42.176+0000 7fe4124e5640 1 -- 172.21.15.169:0/2570356289 shutdown_connections 2023-10-26T01:43:42.180 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:42.176+0000 7fe4124e5640 1 --2- 172.21.15.169:0/2570356289 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe40c15b9e0 0x7fe40c06ad90 unknown :-1 s=CLOSED pgs=1805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:42.180 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:42.176+0000 7fe4124e5640 1 --2- 172.21.15.169:0/2570356289 >> v2:172.21.15.170:3300/0 conn(0x7fe40c0cdd20 0x7fe40c15b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:42.180 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:42.176+0000 7fe4124e5640 1 --2- 172.21.15.169:0/2570356289 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe40c155360 0x7fe40c0cd7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:42.181 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:42.176+0000 7fe4124e5640 1 -- 172.21.15.169:0/2570356289 >> 172.21.15.169:0/2570356289 conn(0x7fe40c069930 msgr2=0x7fe40c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:42.181 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:42.176+0000 7fe4124e5640 1 -- 172.21.15.169:0/2570356289 shutdown_connections 2023-10-26T01:43:42.181 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:42.176+0000 7fe4124e5640 1 -- 172.21.15.169:0/2570356289 wait complete. 2023-10-26T01:43:42.183 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:42.178+0000 7fe4124e5640 1 Processor -- start 2023-10-26T01:43:42.183 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:42.178+0000 7fe4124e5640 1 -- start start 2023-10-26T01:43:42.183 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:42.179+0000 7fe4124e5640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe40c0cdd20 0x7fe40c14fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:42.183 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:42.179+0000 7fe4124e5640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe40c155360 0x7fe40c1503b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:42.183 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:42.179+0000 7fe4124e5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe40c15b9e0 0x7fe40c0aecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:42.184 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:42.179+0000 7fe4124e5640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe40c1ef650 con 0x7fe40c155360 2023-10-26T01:43:42.184 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:42.179+0000 7fe4124e5640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe40c0715f0 con 0x7fe40c15b9e0 2023-10-26T01:43:42.184 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:42.179+0000 7fe4124e5640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe40c14f440 con 0x7fe40c0cdd20 2023-10-26T01:43:42.184 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:42.179+0000 7fe403fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe40c155360 0x7fe40c1503b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:42.184 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:42.179+0000 7fe403fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe40c155360 0x7fe40c1503b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:35860/0 (socket says 172.21.15.169:35860) 2023-10-26T01:43:42.185 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:42.179+0000 7fe403fff640 1 -- 172.21.15.169:0/3375404296 learned_addr learned my addr 172.21.15.169:0/3375404296 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:43:42.185 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:42.180+0000 7fe40bfff640 1 --2- 172.21.15.169:0/3375404296 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe40c0cdd20 0x7fe40c14fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:42.185 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:42.180+0000 7fe410a82640 1 --2- 172.21.15.169:0/3375404296 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe40c15b9e0 0x7fe40c0aecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:42.185 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:42.180+0000 7fe403fff640 1 -- 172.21.15.169:0/3375404296 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe40c0cdd20 msgr2=0x7fe40c14fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:42.185 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:42.180+0000 7fe403fff640 1 --2- 172.21.15.169:0/3375404296 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe40c0cdd20 0x7fe40c14fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:42.186 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:42.180+0000 7fe403fff640 1 -- 172.21.15.169:0/3375404296 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe40c15b9e0 msgr2=0x7fe40c0aecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:42.186 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:42.180+0000 7fe403fff640 1 --2- 172.21.15.169:0/3375404296 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe40c15b9e0 0x7fe40c0aecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:42.186 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:42.180+0000 7fe403fff640 1 -- 172.21.15.169:0/3375404296 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe404009460 con 0x7fe40c155360 2023-10-26T01:43:42.186 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:42.180+0000 7fe403fff640 1 --2- 172.21.15.169:0/3375404296 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe40c155360 0x7fe40c1503b0 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7fe3f400d770 tx=0x7fe3f400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:42.187 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:42.181+0000 7fe4097fa640 1 -- 172.21.15.169:0/3375404296 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f4009c30 con 0x7fe40c155360 2023-10-26T01:43:42.188 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:42.181+0000 7fe4097fa640 1 -- 172.21.15.169:0/3375404296 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe3f4009e80 con 0x7fe40c155360 2023-10-26T01:43:42.188 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:42.181+0000 7fe4124e5640 1 -- 172.21.15.169:0/3375404296 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe40c14f720 con 0x7fe40c155360 2023-10-26T01:43:42.188 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:42.181+0000 7fe4097fa640 1 -- 172.21.15.169:0/3375404296 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f40106a0 con 0x7fe40c155360 2023-10-26T01:43:42.188 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:42.181+0000 7fe4124e5640 1 -- 172.21.15.169:0/3375404296 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe40c071280 con 0x7fe40c155360 2023-10-26T01:43:42.188 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:42.182+0000 7fe4124e5640 1 -- 172.21.15.169:0/3375404296 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe40c071760 con 0x7fe40c155360 2023-10-26T01:43:42.189 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:42.183+0000 7fe4097fa640 1 -- 172.21.15.169:0/3375404296 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fe3f4010800 con 0x7fe40c155360 2023-10-26T01:43:42.189 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:42.184+0000 7fe4097fa640 1 --2- 172.21.15.169:0/3375404296 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe3ec06ae40 0x7fe3ec06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:42.189 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:42.184+0000 7fe4097fa640 1 -- 172.21.15.169:0/3375404296 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fe3f4089870 con 0x7fe40c155360 2023-10-26T01:43:42.189 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:42.184+0000 7fe40bfff640 1 --2- 172.21.15.169:0/3375404296 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe3ec06ae40 0x7fe3ec06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:42.190 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:42.185+0000 7fe40bfff640 1 --2- 172.21.15.169:0/3375404296 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe3ec06ae40 0x7fe3ec06d300 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc005fd0 tx=0x7fe3fc005850 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:42.191 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:42.188+0000 7fe4097fa640 1 -- 172.21.15.169:0/3375404296 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe3f40586a0 con 0x7fe40c155360 2023-10-26T01:43:42.385 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:42.382+0000 7fe4124e5640 1 -- 172.21.15.169:0/3375404296 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fe40c04db10 con 0x7fe40c155360 2023-10-26T01:43:42.390 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:42.387+0000 7fe4097fa640 1 -- 172.21.15.169:0/3375404296 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v37)=0 v37) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe3f40170c0 con 0x7fe40c155360 2023-10-26T01:43:42.393 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:42.390+0000 7fe4124e5640 1 -- 172.21.15.169:0/3375404296 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe3ec06ae40 msgr2=0x7fe3ec06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:42.394 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:42.390+0000 7fe4124e5640 1 --2- 172.21.15.169:0/3375404296 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe3ec06ae40 0x7fe3ec06d300 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc005fd0 tx=0x7fe3fc005850 comp rx=0 tx=0).stop 2023-10-26T01:43:42.394 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:42.390+0000 7fe4124e5640 1 -- 172.21.15.169:0/3375404296 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe40c155360 msgr2=0x7fe40c1503b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:42.394 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:42.390+0000 7fe4124e5640 1 --2- 172.21.15.169:0/3375404296 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe40c155360 0x7fe40c1503b0 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7fe3f400d770 tx=0x7fe3f400dc30 comp rx=0 tx=0).stop 2023-10-26T01:43:42.394 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:42.391+0000 7fe4124e5640 1 -- 172.21.15.169:0/3375404296 shutdown_connections 2023-10-26T01:43:42.394 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:42.391+0000 7fe4124e5640 1 --2- 172.21.15.169:0/3375404296 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe3ec06ae40 0x7fe3ec06d300 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:42.395 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:42.391+0000 7fe4124e5640 1 --2- 172.21.15.169:0/3375404296 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe40c155360 0x7fe40c1503b0 unknown :-1 s=CLOSED pgs=1806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:42.395 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:42.391+0000 7fe4124e5640 1 --2- 172.21.15.169:0/3375404296 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe40c15b9e0 0x7fe40c0aecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:42.395 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:42.391+0000 7fe4124e5640 1 --2- 172.21.15.169:0/3375404296 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe40c0cdd20 0x7fe40c14fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:42.395 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:42.391+0000 7fe4124e5640 1 -- 172.21.15.169:0/3375404296 >> 172.21.15.169:0/3375404296 conn(0x7fe40c069930 msgr2=0x7fe40c06b360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:42.395 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:42.391+0000 7fe4124e5640 1 -- 172.21.15.169:0/3375404296 shutdown_connections 2023-10-26T01:43:42.396 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:42.391+0000 7fe4124e5640 1 -- 172.21.15.169:0/3375404296 wait complete. 2023-10-26T01:43:42.412 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-26T01:43:42.413 DEBUG:teuthology.orchestra.run.smithi169:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-26T01:43:42.524 DEBUG:teuthology.orchestra.run.smithi170:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-26T01:43:42.644 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-26T01:43:42.645 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:43:42.645 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:43:42.705 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054301,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-26T01:43:42.705 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-26T01:43:42.706 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:43:42.764 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054303,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-26T01:43:42.765 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:43:42.765 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:43:42.825 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054301,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-26T01:43:42.826 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-26T01:43:42.827 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:43:42.888 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054303,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-26T01:43:42.889 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-26T01:43:42.890 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T01:43:43.121 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:43.118+0000 7f596a1ae640 1 -- 172.21.15.169:0/2212890784 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f596406b8b0 msgr2=0x7f596406bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:43.122 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:43.118+0000 7f596a1ae640 1 --2- 172.21.15.169:0/2212890784 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f596406b8b0 0x7f596406bd20 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f595c005fd0 tx=0x7f595c02c2d0 comp rx=0 tx=0).stop 2023-10-26T01:43:43.122 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:43.118+0000 7f596a1ae640 1 -- 172.21.15.169:0/2212890784 shutdown_connections 2023-10-26T01:43:43.122 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:43.118+0000 7f596a1ae640 1 --2- 172.21.15.169:0/2212890784 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f596406b8b0 0x7f596406bd20 unknown :-1 s=CLOSED pgs=1807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:43.122 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:43.118+0000 7f596a1ae640 1 --2- 172.21.15.169:0/2212890784 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5964158070 0x7f596409a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:43.123 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:43.118+0000 7f596a1ae640 1 -- 172.21.15.169:0/2212890784 >> 172.21.15.169:0/2212890784 conn(0x7f59640aedd0 msgr2=0x7f59640af1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:43.123 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:43.119+0000 7f596a1ae640 1 -- 172.21.15.169:0/2212890784 shutdown_connections 2023-10-26T01:43:43.123 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:43.119+0000 7f596a1ae640 1 -- 172.21.15.169:0/2212890784 wait complete. 2023-10-26T01:43:43.125 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:43.120+0000 7f596a1ae640 1 Processor -- start 2023-10-26T01:43:43.126 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:43.120+0000 7f596a1ae640 1 -- start start 2023-10-26T01:43:43.126 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:43.121+0000 7f596a1ae640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f596406b8b0 0x7f59641587a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:43.126 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:43.121+0000 7f596a1ae640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f596409ab50 0x7f5964071110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:43.126 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:43.121+0000 7f596a1ae640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5964158070 0x7f596415bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:43.127 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:43.121+0000 7f596a1ae640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59640b1b10 con 0x7f5964158070 2023-10-26T01:43:43.127 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:43.121+0000 7f596a1ae640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f596415c5d0 con 0x7f596406b8b0 2023-10-26T01:43:43.127 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:43.121+0000 7f596a1ae640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f596415c8b0 con 0x7f596409ab50 2023-10-26T01:43:43.127 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:43.122+0000 7f59699ad640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5964158070 0x7f596415bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:43.127 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:43.122+0000 7f59699ad640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5964158070 0x7f596415bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:35888/0 (socket says 172.21.15.169:35888) 2023-10-26T01:43:43.128 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:43.122+0000 7f59699ad640 1 -- 172.21.15.169:0/1210674160 learned_addr learned my addr 172.21.15.169:0/1210674160 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:43:43.128 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:43.122+0000 7f59691ac640 1 --2- 172.21.15.169:0/1210674160 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f596406b8b0 0x7f59641587a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:43.128 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:43.122+0000 7f59689ab640 1 --2- 172.21.15.169:0/1210674160 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f596409ab50 0x7f5964071110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:43.128 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:43.122+0000 7f59699ad640 1 -- 172.21.15.169:0/1210674160 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f596409ab50 msgr2=0x7f5964071110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:43.129 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:43.122+0000 7f59699ad640 1 --2- 172.21.15.169:0/1210674160 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f596409ab50 0x7f5964071110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:43.129 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:43.122+0000 7f59699ad640 1 -- 172.21.15.169:0/1210674160 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f596406b8b0 msgr2=0x7f59641587a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:43.129 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:43.122+0000 7f59699ad640 1 --2- 172.21.15.169:0/1210674160 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f596406b8b0 0x7f59641587a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:43.129 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:43.122+0000 7f59699ad640 1 -- 172.21.15.169:0/1210674160 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f595c009900 con 0x7f5964158070 2023-10-26T01:43:43.129 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:43.122+0000 7f59689ab640 1 --2- 172.21.15.169:0/1210674160 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f596409ab50 0x7f5964071110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:43:43.130 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:43.122+0000 7f59699ad640 1 --2- 172.21.15.169:0/1210674160 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5964158070 0x7f596415bb60 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f595c037040 tx=0x7f595c003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:43.130 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:43.123+0000 7f5959ffb640 1 -- 172.21.15.169:0/1210674160 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f595c004240 con 0x7f5964158070 2023-10-26T01:43:43.130 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:43.123+0000 7f5959ffb640 1 -- 172.21.15.169:0/1210674160 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f595c004490 con 0x7f5964158070 2023-10-26T01:43:43.130 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:43.123+0000 7f596a1ae640 1 -- 172.21.15.169:0/1210674160 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59640b1db0 con 0x7f5964158070 2023-10-26T01:43:43.130 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:43.123+0000 7f5959ffb640 1 -- 172.21.15.169:0/1210674160 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f595c004820 con 0x7f5964158070 2023-10-26T01:43:43.131 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:43.123+0000 7f596a1ae640 1 -- 172.21.15.169:0/1210674160 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5964071650 con 0x7f5964158070 2023-10-26T01:43:43.131 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:43.124+0000 7f596a1ae640 1 -- 172.21.15.169:0/1210674160 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59640b1fc0 con 0x7f5964158070 2023-10-26T01:43:43.132 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:43.125+0000 7f5959ffb640 1 -- 172.21.15.169:0/1210674160 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f595c03b040 con 0x7f5964158070 2023-10-26T01:43:43.132 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:43.126+0000 7f5959ffb640 1 --2- 172.21.15.169:0/1210674160 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f594806ae40 0x7f594806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:43.132 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:43.126+0000 7f5959ffb640 1 -- 172.21.15.169:0/1210674160 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f595c0ac230 con 0x7f5964158070 2023-10-26T01:43:43.132 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:43.127+0000 7f59691ac640 1 --2- 172.21.15.169:0/1210674160 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f594806ae40 0x7f594806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:43.133 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:43.128+0000 7f59691ac640 1 --2- 172.21.15.169:0/1210674160 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f594806ae40 0x7f594806d300 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f5960005fd0 tx=0x7f5960005850 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:43.133 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:43.130+0000 7f5959ffb640 1 -- 172.21.15.169:0/1210674160 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f595c0775d0 con 0x7f5964158070 2023-10-26T01:43:43.362 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:43.359+0000 7f596a1ae640 1 -- 172.21.15.169:0/1210674160 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f59641585c0 con 0x7f5964158070 2023-10-26T01:43:43.363 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:43.360+0000 7f5959ffb640 1 -- 172.21.15.169:0/1210674160 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) v1 ==== 94+0+3708 (secure 0 0 0) 0x7f595c043e90 con 0x7f5964158070 2023-10-26T01:43:43.363 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:43:43.364 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":88,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5861,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/326391134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":326391134},{"type":"v1","addr":"172.21.15.170:6836","nonce":326391134}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":5864,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/3374486280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":3374486280},{"type":"v1","addr":"172.21.15.170:6837","nonce":3374486280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":8697,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/1445776478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":1445776478},{"type":"v1","addr":"172.21.15.169:6834","nonce":1445776478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":80},{"gid":9042,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/2120527361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":2120527361},{"type":"v1","addr":"172.21.15.169:6835","nonce":2120527361}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":88}],"filesystems":[]} 2023-10-26T01:43:43.364 INFO:teuthology.orchestra.run.smithi169.stderr:dumped fsmap epoch 88 2023-10-26T01:43:43.366 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:43.363+0000 7f596a1ae640 1 -- 172.21.15.169:0/1210674160 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f594806ae40 msgr2=0x7f594806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:43.366 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:43.363+0000 7f596a1ae640 1 --2- 172.21.15.169:0/1210674160 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f594806ae40 0x7f594806d300 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f5960005fd0 tx=0x7f5960005850 comp rx=0 tx=0).stop 2023-10-26T01:43:43.366 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:43.363+0000 7f596a1ae640 1 -- 172.21.15.169:0/1210674160 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5964158070 msgr2=0x7f596415bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:43.366 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:43.363+0000 7f596a1ae640 1 --2- 172.21.15.169:0/1210674160 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5964158070 0x7f596415bb60 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f595c037040 tx=0x7f595c003ba0 comp rx=0 tx=0).stop 2023-10-26T01:43:43.366 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:43.363+0000 7f596a1ae640 1 -- 172.21.15.169:0/1210674160 shutdown_connections 2023-10-26T01:43:43.367 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:43.363+0000 7f596a1ae640 1 --2- 172.21.15.169:0/1210674160 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f594806ae40 0x7f594806d300 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:43.367 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:43.363+0000 7f596a1ae640 1 --2- 172.21.15.169:0/1210674160 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5964158070 0x7f596415bb60 unknown :-1 s=CLOSED pgs=1808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:43.367 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:43.363+0000 7f596a1ae640 1 --2- 172.21.15.169:0/1210674160 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f596409ab50 0x7f5964071110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:43.367 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:43.363+0000 7f596a1ae640 1 --2- 172.21.15.169:0/1210674160 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f596406b8b0 0x7f59641587a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:43.367 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:43.363+0000 7f596a1ae640 1 -- 172.21.15.169:0/1210674160 >> 172.21.15.169:0/1210674160 conn(0x7f59640aedd0 msgr2=0x7f59640cab30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:43.368 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:43.363+0000 7f596a1ae640 1 -- 172.21.15.169:0/1210674160 shutdown_connections 2023-10-26T01:43:43.368 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:43.363+0000 7f596a1ae640 1 -- 172.21.15.169:0/1210674160 wait complete. 2023-10-26T01:43:43.386 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-26T01:43:43.386 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-26T01:43:43.386 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-26T01:43:43.387 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T01:43:43.387 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-26T01:43:43.387 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-26T01:43:43.387 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-26T01:43:43.388 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T01:43:43.388 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-26T01:43:43.388 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-26T01:43:43.389 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-26T01:43:43.389 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T01:43:43.389 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-26T01:43:43.389 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-26T01:43:43.390 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-26T01:43:43.390 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T01:43:43.405 INFO:tasks.ceph.mds.a:Stopped 2023-10-26T01:43:43.405 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-26T01:43:43.409 INFO:tasks.ceph.mds.c:Stopped 2023-10-26T01:43:43.409 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-26T01:43:43.411 INFO:tasks.ceph.mds.b:Stopped 2023-10-26T01:43:43.411 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-26T01:43:43.416 INFO:tasks.ceph.mds.d:Stopped 2023-10-26T01:43:43.416 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-26T01:43:43.457 INFO:tasks.ceph.mds.a:Started 2023-10-26T01:43:43.458 DEBUG:teuthology.parallel:result is None 2023-10-26T01:43:43.464 INFO:tasks.ceph.mds.b:Started 2023-10-26T01:43:43.465 DEBUG:teuthology.parallel:result is None 2023-10-26T01:43:43.466 INFO:tasks.ceph.mds.c:Started 2023-10-26T01:43:43.466 DEBUG:teuthology.parallel:result is None 2023-10-26T01:43:43.472 INFO:tasks.ceph.mds.d:Started 2023-10-26T01:43:43.473 DEBUG:teuthology.parallel:result is None 2023-10-26T01:43:43.473 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-26T01:43:43.619 INFO:tasks.ceph.mds.a.smithi169.stdout:starting mds.a at 2023-10-26T01:43:43.619 INFO:tasks.ceph.mds.c.smithi169.stdout:starting mds.c at 2023-10-26T01:43:43.630 INFO:tasks.ceph.mds.b.smithi170.stdout:starting mds.b at 2023-10-26T01:43:43.631 INFO:tasks.ceph.mds.d.smithi170.stdout:starting mds.d at 2023-10-26T01:43:43.664 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:43.661+0000 7f80ecaea640 1 -- 172.21.15.169:0/2820402943 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f80e80b0e50 msgr2=0x7f80e815d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:43.665 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:43.661+0000 7f80ecaea640 1 --2- 172.21.15.169:0/2820402943 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f80e80b0e50 0x7f80e815d7e0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f80e0005fd0 tx=0x7f80e002c170 comp rx=0 tx=0).stop 2023-10-26T01:43:43.665 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:43.661+0000 7f80ecaea640 1 -- 172.21.15.169:0/2820402943 shutdown_connections 2023-10-26T01:43:43.666 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:43.661+0000 7f80ecaea640 1 --2- 172.21.15.169:0/2820402943 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f80e80b0e50 0x7f80e815d7e0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:43.666 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:43.661+0000 7f80ecaea640 1 -- 172.21.15.169:0/2820402943 >> 172.21.15.169:0/2820402943 conn(0x7f80e80af110 msgr2=0x7f80e80af520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:43.666 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:43.661+0000 7f80ecaea640 1 -- 172.21.15.169:0/2820402943 shutdown_connections 2023-10-26T01:43:43.667 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:43.661+0000 7f80ecaea640 1 -- 172.21.15.169:0/2820402943 wait complete. 2023-10-26T01:43:43.671 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:43.663+0000 7f80ecaea640 1 Processor -- start 2023-10-26T01:43:43.672 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:43.663+0000 7f80ecaea640 1 -- start start 2023-10-26T01:43:43.672 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:43.664+0000 7f80ecaea640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f80e806a010 0x7f80e8099a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:43.673 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:43.664+0000 7f80ecaea640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f80e80b0e50 0x7f80e8158270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:43.673 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:43.664+0000 7f80ecaea640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f80e8153b90 0x7f80e81587b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:43.673 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:43.664+0000 7f80ecaea640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80e81f21d0 con 0x7f80e806a010 2023-10-26T01:43:43.674 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:43.664+0000 7f80ecaea640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80e8099fa0 con 0x7f80e8153b90 2023-10-26T01:43:43.674 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:43.664+0000 7f80ecaea640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80e8155190 con 0x7f80e80b0e50 2023-10-26T01:43:43.674 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:43.664+0000 7f80e77fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f80e806a010 0x7f80e8099a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:43.675 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:43.664+0000 7f80e77fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f80e806a010 0x7f80e8099a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:35912/0 (socket says 172.21.15.169:35912) 2023-10-26T01:43:43.675 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:43.664+0000 7f80e77fe640 1 -- 172.21.15.169:0/2048966315 learned_addr learned my addr 172.21.15.169:0/2048966315 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:43:43.676 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:43.664+0000 7f80e7fff640 1 --2- 172.21.15.169:0/2048966315 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f80e8153b90 0x7f80e81587b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:43.676 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:43.664+0000 7f80e6ffd640 1 --2- 172.21.15.169:0/2048966315 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f80e80b0e50 0x7f80e8158270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:43.676 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:43.665+0000 7f80e77fe640 1 -- 172.21.15.169:0/2048966315 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f80e80b0e50 msgr2=0x7f80e8158270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:43.677 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:43.665+0000 7f80e77fe640 1 --2- 172.21.15.169:0/2048966315 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f80e80b0e50 0x7f80e8158270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:43.677 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:43.665+0000 7f80e77fe640 1 -- 172.21.15.169:0/2048966315 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f80e8153b90 msgr2=0x7f80e81587b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:43.678 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:43.665+0000 7f80e77fe640 1 --2- 172.21.15.169:0/2048966315 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f80e8153b90 0x7f80e81587b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:43.678 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:43.665+0000 7f80e77fe640 1 -- 172.21.15.169:0/2048966315 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80e0009840 con 0x7f80e806a010 2023-10-26T01:43:43.678 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:43.665+0000 7f80e77fe640 1 --2- 172.21.15.169:0/2048966315 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f80e806a010 0x7f80e8099a60 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f80d800c460 tx=0x7f80d800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:43.679 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:43.665+0000 7f80cffff640 1 -- 172.21.15.169:0/2048966315 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d80052d0 con 0x7f80e806a010 2023-10-26T01:43:43.679 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:43.665+0000 7f80cffff640 1 -- 172.21.15.169:0/2048966315 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f80d80035f0 con 0x7f80e806a010 2023-10-26T01:43:43.679 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:43.665+0000 7f80ecaea640 1 -- 172.21.15.169:0/2048966315 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80e8140cc0 con 0x7f80e806a010 2023-10-26T01:43:43.680 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:43.665+0000 7f80cffff640 1 -- 172.21.15.169:0/2048966315 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d8003a00 con 0x7f80e806a010 2023-10-26T01:43:43.680 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:43.665+0000 7f80ecaea640 1 -- 172.21.15.169:0/2048966315 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80e8155430 con 0x7f80e806a010 2023-10-26T01:43:43.680 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:43.666+0000 7f80ecaea640 1 -- 172.21.15.169:0/2048966315 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80e8073c10 con 0x7f80e806a010 2023-10-26T01:43:43.681 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:43.667+0000 7f80cffff640 1 -- 172.21.15.169:0/2048966315 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f80d8003b60 con 0x7f80e806a010 2023-10-26T01:43:43.681 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:43.668+0000 7f80cffff640 1 --2- 172.21.15.169:0/2048966315 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f80d406ae40 0x7f80d406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:43.681 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:43.668+0000 7f80cffff640 1 -- 172.21.15.169:0/2048966315 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f80d8088b70 con 0x7f80e806a010 2023-10-26T01:43:43.682 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:43.669+0000 7f80e6ffd640 1 --2- 172.21.15.169:0/2048966315 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f80d406ae40 0x7f80d406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:43.683 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:43.669+0000 7f80e6ffd640 1 --2- 172.21.15.169:0/2048966315 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f80d406ae40 0x7f80d406d300 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f80e80b1690 tx=0x7f80dc009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:43.683 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:43.673+0000 7f80cffff640 1 -- 172.21.15.169:0/2048966315 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f80d8053f30 con 0x7f80e806a010 2023-10-26T01:43:43.870 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:43.867+0000 7f80ecaea640 1 -- 172.21.15.169:0/2048966315 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f80e8069900 con 0x7f80e806a010 2023-10-26T01:43:45.222 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:45.220+0000 7f80cffff640 1 -- 172.21.15.169:0/2048966315 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v178) v1 ==== 97+0+0 (secure 0 0 0) 0x7f80d8057550 con 0x7f80e806a010 2023-10-26T01:43:45.222 INFO:teuthology.orchestra.run.smithi169.stderr: removed all blocklist entries 2023-10-26T01:43:45.227 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:45.224+0000 7f80ecaea640 1 -- 172.21.15.169:0/2048966315 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f80d406ae40 msgr2=0x7f80d406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:45.227 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:45.224+0000 7f80ecaea640 1 --2- 172.21.15.169:0/2048966315 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f80d406ae40 0x7f80d406d300 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f80e80b1690 tx=0x7f80dc009040 comp rx=0 tx=0).stop 2023-10-26T01:43:45.228 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:45.224+0000 7f80ecaea640 1 -- 172.21.15.169:0/2048966315 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f80e806a010 msgr2=0x7f80e8099a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:45.228 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:45.224+0000 7f80ecaea640 1 --2- 172.21.15.169:0/2048966315 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f80e806a010 0x7f80e8099a60 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f80d800c460 tx=0x7f80d800c920 comp rx=0 tx=0).stop 2023-10-26T01:43:45.228 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:45.224+0000 7f80ecaea640 1 -- 172.21.15.169:0/2048966315 shutdown_connections 2023-10-26T01:43:45.229 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:45.224+0000 7f80ecaea640 1 --2- 172.21.15.169:0/2048966315 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f80d406ae40 0x7f80d406d300 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:45.229 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:45.224+0000 7f80ecaea640 1 --2- 172.21.15.169:0/2048966315 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f80e8153b90 0x7f80e81587b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:45.229 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:45.224+0000 7f80ecaea640 1 --2- 172.21.15.169:0/2048966315 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f80e80b0e50 0x7f80e8158270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:45.230 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:45.224+0000 7f80ecaea640 1 --2- 172.21.15.169:0/2048966315 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f80e806a010 0x7f80e8099a60 unknown :-1 s=CLOSED pgs=1814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:45.230 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:45.224+0000 7f80ecaea640 1 -- 172.21.15.169:0/2048966315 >> 172.21.15.169:0/2048966315 conn(0x7f80e80af110 msgr2=0x7f80e806b710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:45.231 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:45.224+0000 7f80ecaea640 1 -- 172.21.15.169:0/2048966315 shutdown_connections 2023-10-26T01:43:45.231 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:45.224+0000 7f80ecaea640 1 -- 172.21.15.169:0/2048966315 wait complete. 2023-10-26T01:43:45.246 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-26T01:43:45.451 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:45.448+0000 7f072538e640 1 -- 172.21.15.169:0/1935369863 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0708003970 msgr2=0x7f0708003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:45.451 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:45.448+0000 7f072538e640 1 --2- 172.21.15.169:0/1935369863 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0708003970 0x7f0708003d90 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f0710009b30 tx=0x7f07080057f0 comp rx=0 tx=0).stop 2023-10-26T01:43:45.451 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:45.448+0000 7f072538e640 1 -- 172.21.15.169:0/1935369863 shutdown_connections 2023-10-26T01:43:45.451 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:45.448+0000 7f072538e640 1 --2- 172.21.15.169:0/1935369863 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0708003970 0x7f0708003d90 unknown :-1 s=CLOSED pgs=1816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:45.452 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:45.448+0000 7f072538e640 1 -- 172.21.15.169:0/1935369863 >> 172.21.15.169:0/1935369863 conn(0x7f0720069930 msgr2=0x7f0720069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:45.452 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:45.448+0000 7f072538e640 1 -- 172.21.15.169:0/1935369863 shutdown_connections 2023-10-26T01:43:45.452 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:45.448+0000 7f072538e640 1 -- 172.21.15.169:0/1935369863 wait complete. 2023-10-26T01:43:45.454 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:45.450+0000 7f072538e640 1 Processor -- start 2023-10-26T01:43:45.454 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:45.450+0000 7f072538e640 1 -- start start 2023-10-26T01:43:45.455 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:45.451+0000 7f072538e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0708003970 0x7f07200c8c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:45.455 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:45.451+0000 7f072538e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0720151cc0 0x7f07201f4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:45.455 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:45.451+0000 7f072538e640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f07200c6a20 0x7f07200c6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:45.455 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:45.451+0000 7f072538e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07200b0840 con 0x7f0708003970 2023-10-26T01:43:45.455 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:45.451+0000 7f072538e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07200c73e0 con 0x7f0720151cc0 2023-10-26T01:43:45.456 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:45.451+0000 7f072538e640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0720151120 con 0x7f07200c6a20 2023-10-26T01:43:45.456 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:45.451+0000 7f071effd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0708003970 0x7f07200c8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:45.456 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:45.451+0000 7f071effd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0708003970 0x7f07200c8c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:47764/0 (socket says 172.21.15.169:47764) 2023-10-26T01:43:45.456 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:45.451+0000 7f071effd640 1 -- 172.21.15.169:0/832067610 learned_addr learned my addr 172.21.15.169:0/832067610 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:43:45.456 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:45.452+0000 7f071e7fc640 1 --2- 172.21.15.169:0/832067610 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0720151cc0 0x7f07201f4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:45.457 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:45.452+0000 7f071effd640 1 -- 172.21.15.169:0/832067610 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f07200c6a20 msgr2=0x7f07200c6ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:45.457 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:45.452+0000 7f071effd640 1 --2- 172.21.15.169:0/832067610 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f07200c6a20 0x7f07200c6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:45.457 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:45.452+0000 7f071effd640 1 -- 172.21.15.169:0/832067610 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0720151cc0 msgr2=0x7f07201f4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:45.457 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:45.452+0000 7f071effd640 1 --2- 172.21.15.169:0/832067610 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0720151cc0 0x7f07201f4b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:45.458 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:45.452+0000 7f071effd640 1 -- 172.21.15.169:0/832067610 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07201513c0 con 0x7f0708003970 2023-10-26T01:43:45.458 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:45.452+0000 7f071effd640 1 --2- 172.21.15.169:0/832067610 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0708003970 0x7f07200c8c50 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f071002cb70 tx=0x7f0710004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:45.458 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:45.452+0000 7f07077fe640 1 -- 172.21.15.169:0/832067610 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f071003e670 con 0x7f0708003970 2023-10-26T01:43:45.458 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:45.452+0000 7f07077fe640 1 -- 172.21.15.169:0/832067610 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f071003e8c0 con 0x7f0708003970 2023-10-26T01:43:45.458 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:45.452+0000 7f07077fe640 1 -- 172.21.15.169:0/832067610 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f071003ecd0 con 0x7f0708003970 2023-10-26T01:43:45.459 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:45.452+0000 7f072538e640 1 -- 172.21.15.169:0/832067610 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07201516a0 con 0x7f0708003970 2023-10-26T01:43:45.459 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:45.452+0000 7f072538e640 1 -- 172.21.15.169:0/832067610 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07201e1ca0 con 0x7f0708003970 2023-10-26T01:43:45.460 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:45.453+0000 7f072538e640 1 -- 172.21.15.169:0/832067610 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07201518e0 con 0x7f0708003970 2023-10-26T01:43:45.460 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:45.454+0000 7f07077fe640 1 -- 172.21.15.169:0/832067610 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f0710033400 con 0x7f0708003970 2023-10-26T01:43:45.460 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:45.456+0000 7f07077fe640 1 --2- 172.21.15.169:0/832067610 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f070c06ae40 0x7f070c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:45.460 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:45.456+0000 7f07077fe640 1 -- 172.21.15.169:0/832067610 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f07100ac320 con 0x7f0708003970 2023-10-26T01:43:45.461 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:45.456+0000 7f071e7fc640 1 --2- 172.21.15.169:0/832067610 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f070c06ae40 0x7f070c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:45.461 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:45.457+0000 7f071e7fc640 1 --2- 172.21.15.169:0/832067610 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f070c06ae40 0x7f070c06d300 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f07200b0110 tx=0x7f0714009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:45.462 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:45.461+0000 7f07077fe640 1 -- 172.21.15.169:0/832067610 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0710077790 con 0x7f0708003970 2023-10-26T01:43:45.698 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:45.696+0000 7f072538e640 1 -- 172.21.15.169:0/832067610 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f07200b0260 con 0x7f0708003970 2023-10-26T01:43:45.699 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:45.697+0000 7f07077fe640 1 -- 172.21.15.169:0/832067610 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v21) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f0710042090 con 0x7f0708003970 2023-10-26T01:43:45.699 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:43:45.700 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2023-10-26T01:43:45.700 INFO:teuthology.orchestra.run.smithi169.stdout: "auth_dump": [ 2023-10-26T01:43:45.700 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:43:45.701 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "mds.a", 2023-10-26T01:43:45.701 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAgwTllWtUfAhAA28+12VwWEF/BUEu+ErSVyA==", 2023-10-26T01:43:45.701 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:43:45.701 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow", 2023-10-26T01:43:45.701 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow *", 2023-10-26T01:43:45.702 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow *", 2023-10-26T01:43:45.702 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:43:45.702 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:43:45.702 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:43:45.703 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:43:45.703 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "mds.b", 2023-10-26T01:43:45.703 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAgwTlleA8lHxAA+jMcMo+QnYAGllpKJ6eYgQ==", 2023-10-26T01:43:45.703 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:43:45.705 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow", 2023-10-26T01:43:45.705 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow *", 2023-10-26T01:43:45.706 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow *", 2023-10-26T01:43:45.706 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:43:45.706 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:43:45.707 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:43:45.707 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:43:45.707 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "mds.c", 2023-10-26T01:43:45.707 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAgwTllaCCqEBAAokeHvwOyp/Jv2x0yHDwysw==", 2023-10-26T01:43:45.707 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:43:45.708 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow", 2023-10-26T01:43:45.708 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow *", 2023-10-26T01:43:45.708 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow *", 2023-10-26T01:43:45.708 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:43:45.709 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:43:45.709 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:43:45.709 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:43:45.709 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "mds.d", 2023-10-26T01:43:45.709 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAgwTll51JQLhAA4Wzso4zQj+Htv6cXqvwRmg==", 2023-10-26T01:43:45.710 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:43:45.710 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow", 2023-10-26T01:43:45.710 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow *", 2023-10-26T01:43:45.710 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow *", 2023-10-26T01:43:45.710 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:43:45.711 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:43:45.711 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:43:45.711 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:43:45.711 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "osd.0", 2023-10-26T01:43:45.711 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAlwTll8RZ+HRAAt/sBK+Mi91YmTyQYbhKHLg==", 2023-10-26T01:43:45.712 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:43:45.712 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow profile osd", 2023-10-26T01:43:45.712 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow profile osd", 2023-10-26T01:43:45.712 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:43:45.712 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:43:45.713 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:43:45.713 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:43:45.713 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "osd.1", 2023-10-26T01:43:45.713 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAlwTllLXwWLRAAuHB2lJ+oDJou/IJBtsYY4Q==", 2023-10-26T01:43:45.714 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:43:45.714 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow profile osd", 2023-10-26T01:43:45.714 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow profile osd", 2023-10-26T01:43:45.714 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:43:45.714 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:43:45.715 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:43:45.715 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:43:45.715 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "osd.2", 2023-10-26T01:43:45.715 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAmwTllj1nSARAA5Jl34Q0X7a71ySd4wz17lw==", 2023-10-26T01:43:45.715 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:43:45.716 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow profile osd", 2023-10-26T01:43:45.716 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow profile osd", 2023-10-26T01:43:45.716 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:43:45.716 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:43:45.716 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:43:45.717 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:43:45.717 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "osd.3", 2023-10-26T01:43:45.717 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAmwTllC8MeEhAAfGxRVSwD+YbU/U5aeenO3w==", 2023-10-26T01:43:45.718 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:43:45.718 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow profile osd", 2023-10-26T01:43:45.718 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow profile osd", 2023-10-26T01:43:45.718 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:43:45.718 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:43:45.719 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:43:45.719 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:43:45.719 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "osd.4", 2023-10-26T01:43:45.719 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAowTll4Md6DhAAuXfadUNAdouMP28MiX/D3Q==", 2023-10-26T01:43:45.720 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:43:45.720 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow profile osd", 2023-10-26T01:43:45.720 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow profile osd", 2023-10-26T01:43:45.720 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:43:45.720 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:43:45.721 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:43:45.721 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:43:45.721 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "osd.5", 2023-10-26T01:43:45.721 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAowTllDdViHRAAec9QyhZCsjJ2rgGhhcjwpQ==", 2023-10-26T01:43:45.721 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:43:45.722 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow profile osd", 2023-10-26T01:43:45.722 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow profile osd", 2023-10-26T01:43:45.722 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:43:45.722 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:43:45.722 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:43:45.723 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:43:45.723 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "osd.6", 2023-10-26T01:43:45.723 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAowTll23X5KxAAsmQvG0yq7ilkVG5fNiaCUg==", 2023-10-26T01:43:45.723 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:43:45.724 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow profile osd", 2023-10-26T01:43:45.724 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow profile osd", 2023-10-26T01:43:45.724 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:43:45.724 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:43:45.724 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:43:45.725 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:43:45.725 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "osd.7", 2023-10-26T01:43:45.725 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAowTllhcy/OhAAY5Yjheh76ibOVREqN6WBpQ==", 2023-10-26T01:43:45.725 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:43:45.725 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow profile osd", 2023-10-26T01:43:45.726 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow profile osd", 2023-10-26T01:43:45.726 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:43:45.726 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:43:45.726 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:43:45.726 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:43:45.727 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "client.0", 2023-10-26T01:43:45.727 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAgwTllKC7gOhAAPsvxGcuUA8/p8xqbUgGteA==", 2023-10-26T01:43:45.727 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:43:45.727 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow", 2023-10-26T01:43:45.728 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow r", 2023-10-26T01:43:45.728 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-26T01:43:45.728 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:43:45.728 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:43:45.728 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:43:45.729 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "client.1", 2023-10-26T01:43:45.729 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAhwTllhxb6ExAAeYTsurWpnjohWlrNIlLpsw==", 2023-10-26T01:43:45.729 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:43:45.729 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow", 2023-10-26T01:43:45.729 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow r", 2023-10-26T01:43:45.730 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow rw", 2023-10-26T01:43:45.730 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow rwx" 2023-10-26T01:43:45.730 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:43:45.730 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:43:45.730 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:43:45.731 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "client.2", 2023-10-26T01:43:45.731 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAhwTllLWMoChAAsUHzBc29GDqmEQAFnl7dHg==", 2023-10-26T01:43:45.731 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:43:45.731 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow", 2023-10-26T01:43:45.731 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow r", 2023-10-26T01:43:45.732 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow rw", 2023-10-26T01:43:45.732 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow rwx" 2023-10-26T01:43:45.732 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:43:45.732 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:43:45.732 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:43:45.733 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "client.3", 2023-10-26T01:43:45.733 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAhwTll875HHRAAGbwikoR/hDU8P/bj0OPWAw==", 2023-10-26T01:43:45.733 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:43:45.733 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow", 2023-10-26T01:43:45.733 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow r", 2023-10-26T01:43:45.734 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow rw", 2023-10-26T01:43:45.734 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow rwx" 2023-10-26T01:43:45.734 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:43:45.734 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:43:45.735 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:43:45.735 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "client.admin", 2023-10-26T01:43:45.735 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAewTll2yddMhAAO8bnXNevDGe/KEeNNa36pQ==", 2023-10-26T01:43:45.735 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:43:45.735 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow *", 2023-10-26T01:43:45.736 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": "allow *", 2023-10-26T01:43:45.736 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow *", 2023-10-26T01:43:45.736 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:43:45.736 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:43:45.736 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:43:45.737 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:43:45.737 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "mgr.x", 2023-10-26T01:43:45.737 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAfwTllPu3FMxAAV8d5oRFXAeeZH7e1xzPEDg==", 2023-10-26T01:43:45.737 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:43:45.737 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow *", 2023-10-26T01:43:45.738 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow profile mgr", 2023-10-26T01:43:45.738 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:43:45.738 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:43:45.738 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:43:45.739 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:43:45.739 INFO:teuthology.orchestra.run.smithi169.stdout: "entity": "mgr.y", 2023-10-26T01:43:45.739 INFO:teuthology.orchestra.run.smithi169.stdout: "key": "AQAfwTllZc+DJxAAsZ2TWn8H7Kw/lNTc2qyKzA==", 2023-10-26T01:43:45.739 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": { 2023-10-26T01:43:45.739 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": "allow *", 2023-10-26T01:43:45.740 INFO:teuthology.orchestra.run.smithi169.stdout: "mon": "allow profile mgr", 2023-10-26T01:43:45.740 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": "allow *" 2023-10-26T01:43:45.740 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:43:45.740 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:43:45.740 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:43:45.741 INFO:teuthology.orchestra.run.smithi169.stdout:} 2023-10-26T01:43:45.742 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:45.700+0000 7f072538e640 1 -- 172.21.15.169:0/832067610 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f070c06ae40 msgr2=0x7f070c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:45.743 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:45.700+0000 7f072538e640 1 --2- 172.21.15.169:0/832067610 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f070c06ae40 0x7f070c06d300 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f07200b0110 tx=0x7f0714009040 comp rx=0 tx=0).stop 2023-10-26T01:43:45.743 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:45.700+0000 7f072538e640 1 -- 172.21.15.169:0/832067610 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0708003970 msgr2=0x7f07200c8c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:45.743 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:45.700+0000 7f072538e640 1 --2- 172.21.15.169:0/832067610 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0708003970 0x7f07200c8c50 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f071002cb70 tx=0x7f0710004a50 comp rx=0 tx=0).stop 2023-10-26T01:43:45.743 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:45.700+0000 7f072538e640 1 -- 172.21.15.169:0/832067610 shutdown_connections 2023-10-26T01:43:45.744 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:45.700+0000 7f072538e640 1 --2- 172.21.15.169:0/832067610 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f070c06ae40 0x7f070c06d300 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:45.744 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:45.700+0000 7f072538e640 1 --2- 172.21.15.169:0/832067610 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f07200c6a20 0x7f07200c6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:45.744 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:45.700+0000 7f072538e640 1 --2- 172.21.15.169:0/832067610 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0720151cc0 0x7f07201f4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:45.744 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:45.700+0000 7f072538e640 1 --2- 172.21.15.169:0/832067610 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0708003970 0x7f07200c8c50 unknown :-1 s=CLOSED pgs=1817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:45.744 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:45.700+0000 7f072538e640 1 -- 172.21.15.169:0/832067610 >> 172.21.15.169:0/832067610 conn(0x7f0720069930 msgr2=0x7f0720153870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:45.745 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:45.700+0000 7f072538e640 1 -- 172.21.15.169:0/832067610 shutdown_connections 2023-10-26T01:43:45.745 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:45.700+0000 7f072538e640 1 -- 172.21.15.169:0/832067610 wait complete. 2023-10-26T01:43:45.746 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:43:45.969 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:45.966+0000 7f8a4dd54640 1 -- 172.21.15.169:0/796222159 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a4809a5c0 msgr2=0x7f8a4809aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:45.969 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:45.966+0000 7f8a4dd54640 1 --2- 172.21.15.169:0/796222159 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a4809a5c0 0x7f8a4809aa70 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f8a480c98c0 tx=0x7f8a4002c250 comp rx=0 tx=0).stop 2023-10-26T01:43:45.970 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:45.966+0000 7f8a4dd54640 1 -- 172.21.15.169:0/796222159 shutdown_connections 2023-10-26T01:43:45.970 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:45.966+0000 7f8a4dd54640 1 --2- 172.21.15.169:0/796222159 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a4809a5c0 0x7f8a4809aa70 unknown :-1 s=CLOSED pgs=1818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:45.971 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:45.966+0000 7f8a4dd54640 1 --2- 172.21.15.169:0/796222159 >> v2:172.21.15.170:3300/0 conn(0x7f8a480c9970 0x7f8a48098a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:45.971 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:45.966+0000 7f8a4dd54640 1 --2- 172.21.15.169:0/796222159 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f8a480ac6d0 0x7f8a4815d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:45.971 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:45.966+0000 7f8a4dd54640 1 -- 172.21.15.169:0/796222159 >> 172.21.15.169:0/796222159 conn(0x7f8a480aedd0 msgr2=0x7f8a480af1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:45.972 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:45.966+0000 7f8a4dd54640 1 -- 172.21.15.169:0/796222159 shutdown_connections 2023-10-26T01:43:45.972 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:45.966+0000 7f8a4dd54640 1 -- 172.21.15.169:0/796222159 wait complete. 2023-10-26T01:43:45.978 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:45.968+0000 7f8a4dd54640 1 Processor -- start 2023-10-26T01:43:45.979 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:45.968+0000 7f8a4dd54640 1 -- start start 2023-10-26T01:43:45.979 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:45.969+0000 7f8a4dd54640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f8a4809a5c0 0x7f8a48140c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:45.980 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:45.969+0000 7f8a4dd54640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a480ac6d0 0x7f8a480afce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:45.980 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:45.969+0000 7f8a4dd54640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a480c9970 0x7f8a480ad2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:45.981 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:45.969+0000 7f8a4dd54640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a4815c940 con 0x7f8a480c9970 2023-10-26T01:43:45.981 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:45.969+0000 7f8a4dd54640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a4806efe0 con 0x7f8a480ac6d0 2023-10-26T01:43:45.981 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:45.969+0000 7f8a4dd54640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a48141180 con 0x7f8a4809a5c0 2023-10-26T01:43:45.982 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:45.970+0000 7f8a4d553640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a480c9970 0x7f8a480ad2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:45.982 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:45.970+0000 7f8a4d553640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a480c9970 0x7f8a480ad2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:47790/0 (socket says 172.21.15.169:47790) 2023-10-26T01:43:45.983 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:45.970+0000 7f8a4d553640 1 -- 172.21.15.169:0/407102934 learned_addr learned my addr 172.21.15.169:0/407102934 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:43:45.983 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:45.970+0000 7f8a4cd52640 1 --2- 172.21.15.169:0/407102934 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f8a4809a5c0 0x7f8a48140c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:45.983 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:45.970+0000 7f8a47fff640 1 --2- 172.21.15.169:0/407102934 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a480ac6d0 0x7f8a480afce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:45.984 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:45.970+0000 7f8a4d553640 1 -- 172.21.15.169:0/407102934 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f8a4809a5c0 msgr2=0x7f8a48140c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:45.984 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:45.970+0000 7f8a4d553640 1 --2- 172.21.15.169:0/407102934 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f8a4809a5c0 0x7f8a48140c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:45.984 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:45.970+0000 7f8a4d553640 1 -- 172.21.15.169:0/407102934 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a480ac6d0 msgr2=0x7f8a480afce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:45.985 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:45.970+0000 7f8a4d553640 1 --2- 172.21.15.169:0/407102934 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a480ac6d0 0x7f8a480afce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:45.985 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:45.970+0000 7f8a4d553640 1 -- 172.21.15.169:0/407102934 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a40009900 con 0x7f8a480c9970 2023-10-26T01:43:45.986 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:45.970+0000 7f8a4cd52640 1 --2- 172.21.15.169:0/407102934 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f8a4809a5c0 0x7f8a48140c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:43:45.986 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:45.970+0000 7f8a47fff640 1 --2- 172.21.15.169:0/407102934 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a480ac6d0 0x7f8a480afce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:43:45.987 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:45.970+0000 7f8a4d553640 1 --2- 172.21.15.169:0/407102934 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a480c9970 0x7f8a480ad2d0 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f8a40037040 tx=0x7f8a40035db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:45.987 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:45.971+0000 7f8a457fa640 1 -- 172.21.15.169:0/407102934 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a40034440 con 0x7f8a480c9970 2023-10-26T01:43:45.987 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:45.971+0000 7f8a457fa640 1 -- 172.21.15.169:0/407102934 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8a40004250 con 0x7f8a480c9970 2023-10-26T01:43:45.988 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:45.971+0000 7f8a457fa640 1 -- 172.21.15.169:0/407102934 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a40004640 con 0x7f8a480c9970 2023-10-26T01:43:45.988 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:45.971+0000 7f8a4dd54640 1 -- 172.21.15.169:0/407102934 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a4806f230 con 0x7f8a480c9970 2023-10-26T01:43:45.988 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:45.971+0000 7f8a4dd54640 1 -- 172.21.15.169:0/407102934 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a4806f7f0 con 0x7f8a480c9970 2023-10-26T01:43:45.988 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:45.972+0000 7f8a4dd54640 1 -- 172.21.15.169:0/407102934 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a48098a60 con 0x7f8a480c9970 2023-10-26T01:43:45.989 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:45.973+0000 7f8a457fa640 1 -- 172.21.15.169:0/407102934 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f8a4003b040 con 0x7f8a480c9970 2023-10-26T01:43:45.989 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:45.974+0000 7f8a457fa640 1 --2- 172.21.15.169:0/407102934 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f8a2c06ae40 0x7f8a2c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:45.989 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:45.974+0000 7f8a457fa640 1 -- 172.21.15.169:0/407102934 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f8a400ac000 con 0x7f8a480c9970 2023-10-26T01:43:45.989 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:45.975+0000 7f8a4cd52640 1 --2- 172.21.15.169:0/407102934 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f8a2c06ae40 0x7f8a2c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:45.990 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:45.976+0000 7f8a4cd52640 1 --2- 172.21.15.169:0/407102934 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f8a2c06ae40 0x7f8a2c06d300 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c00baa0 tx=0x7f8a3c005d90 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:45.990 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:45.978+0000 7f8a457fa640 1 -- 172.21.15.169:0/407102934 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8a40077360 con 0x7f8a480c9970 2023-10-26T01:43:46.156 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:46.154+0000 7f8a4dd54640 1 -- 172.21.15.169:0/407102934 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8a4815d3b0 con 0x7f8a480c9970 2023-10-26T01:43:46.157 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:46.156+0000 7f8a457fa640 1 -- 172.21.15.169:0/407102934 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v178) v1 ==== 74+0+12916 (secure 0 0 0) 0x7f8a4007a980 con 0x7f8a480c9970 2023-10-26T01:43:46.158 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:43:46.158 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":178,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:43:45.216533+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:43:46.160 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:46.158+0000 7f8a4dd54640 1 -- 172.21.15.169:0/407102934 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f8a2c06ae40 msgr2=0x7f8a2c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:46.161 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:46.158+0000 7f8a4dd54640 1 --2- 172.21.15.169:0/407102934 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f8a2c06ae40 0x7f8a2c06d300 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c00baa0 tx=0x7f8a3c005d90 comp rx=0 tx=0).stop 2023-10-26T01:43:46.161 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:46.158+0000 7f8a4dd54640 1 -- 172.21.15.169:0/407102934 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a480c9970 msgr2=0x7f8a480ad2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:46.161 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:46.158+0000 7f8a4dd54640 1 --2- 172.21.15.169:0/407102934 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a480c9970 0x7f8a480ad2d0 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f8a40037040 tx=0x7f8a40035db0 comp rx=0 tx=0).stop 2023-10-26T01:43:46.162 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:46.158+0000 7f8a4dd54640 1 -- 172.21.15.169:0/407102934 shutdown_connections 2023-10-26T01:43:46.162 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:46.158+0000 7f8a4dd54640 1 --2- 172.21.15.169:0/407102934 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f8a2c06ae40 0x7f8a2c06d300 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:46.163 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:46.158+0000 7f8a4dd54640 1 --2- 172.21.15.169:0/407102934 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a480c9970 0x7f8a480ad2d0 unknown :-1 s=CLOSED pgs=1819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:46.163 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:46.158+0000 7f8a4dd54640 1 --2- 172.21.15.169:0/407102934 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a480ac6d0 0x7f8a480afce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:46.163 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:46.158+0000 7f8a4dd54640 1 --2- 172.21.15.169:0/407102934 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f8a4809a5c0 0x7f8a48140c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:46.164 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:46.158+0000 7f8a4dd54640 1 -- 172.21.15.169:0/407102934 >> 172.21.15.169:0/407102934 conn(0x7f8a480aedd0 msgr2=0x7f8a480ab4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:46.164 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:46.158+0000 7f8a4dd54640 1 -- 172.21.15.169:0/407102934 shutdown_connections 2023-10-26T01:43:46.165 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:46.158+0000 7f8a4dd54640 1 -- 172.21.15.169:0/407102934 wait complete. 2023-10-26T01:43:46.178 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T01:30:32.200808+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '109', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T01:43:46.179 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T01:43:46.400 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:46.397+0000 7f5b5f59e640 1 -- 172.21.15.169:0/87985907 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5b600cd410 msgr2=0x7f5b600b1cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:46.400 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:46.397+0000 7f5b5f59e640 1 --2- 172.21.15.169:0/87985907 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5b600cd410 0x7f5b600b1cd0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f5b48005fd0 tx=0x7f5b4802be10 comp rx=0 tx=0).stop 2023-10-26T01:43:46.400 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:46.397+0000 7f5b5f59e640 1 -- 172.21.15.169:0/87985907 shutdown_connections 2023-10-26T01:43:46.400 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:46.397+0000 7f5b5f59e640 1 --2- 172.21.15.169:0/87985907 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5b600cb850 0x7f5b600cbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:46.401 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:46.398+0000 7f5b5f59e640 1 --2- 172.21.15.169:0/87985907 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5b600cd410 0x7f5b600b1cd0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:46.401 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:46.398+0000 7f5b5f59e640 1 -- 172.21.15.169:0/87985907 >> 172.21.15.169:0/87985907 conn(0x7f5b60069510 msgr2=0x7f5b60069920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:46.401 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:46.398+0000 7f5b5f59e640 1 -- 172.21.15.169:0/87985907 shutdown_connections 2023-10-26T01:43:46.401 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:46.398+0000 7f5b5f59e640 1 -- 172.21.15.169:0/87985907 wait complete. 2023-10-26T01:43:46.402 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:46.400+0000 7f5b5f59e640 1 Processor -- start 2023-10-26T01:43:46.402 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:46.400+0000 7f5b5f59e640 1 -- start start 2023-10-26T01:43:46.404 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:46.401+0000 7f5b5f59e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5b600cb850 0x7f5b601509a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:46.404 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:46.401+0000 7f5b5f59e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b600cd410 0x7f5b600acee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:46.404 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:46.401+0000 7f5b5f59e640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5b60155cf0 0x7f5b6006ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:46.404 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:46.401+0000 7f5b5f59e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b60143da0 con 0x7f5b600cd410 2023-10-26T01:43:46.404 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:46.401+0000 7f5b5f59e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b600ad9a0 con 0x7f5b600cb850 2023-10-26T01:43:46.405 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:46.401+0000 7f5b5f59e640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b600adc80 con 0x7f5b60155cf0 2023-10-26T01:43:46.405 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:46.402+0000 7f5b57fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b600cd410 0x7f5b600acee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:46.405 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:46.402+0000 7f5b57fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b600cd410 0x7f5b600acee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:47792/0 (socket says 172.21.15.169:47792) 2023-10-26T01:43:46.405 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:46.402+0000 7f5b57fff640 1 -- 172.21.15.169:0/3880308248 learned_addr learned my addr 172.21.15.169:0/3880308248 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:43:46.405 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:46.402+0000 7f5b5e59c640 1 --2- 172.21.15.169:0/3880308248 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5b600cb850 0x7f5b601509a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:46.406 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:46.402+0000 7f5b5ed9d640 1 --2- 172.21.15.169:0/3880308248 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5b60155cf0 0x7f5b6006ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:46.406 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:46.402+0000 7f5b57fff640 1 -- 172.21.15.169:0/3880308248 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5b60155cf0 msgr2=0x7f5b6006ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:46.406 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:46.402+0000 7f5b57fff640 1 --2- 172.21.15.169:0/3880308248 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5b60155cf0 0x7f5b6006ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:46.407 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:46.402+0000 7f5b57fff640 1 -- 172.21.15.169:0/3880308248 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5b600cb850 msgr2=0x7f5b601509a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:46.407 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:46.402+0000 7f5b57fff640 1 --2- 172.21.15.169:0/3880308248 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5b600cb850 0x7f5b601509a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:46.407 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:46.402+0000 7f5b57fff640 1 -- 172.21.15.169:0/3880308248 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b48009560 con 0x7f5b600cd410 2023-10-26T01:43:46.407 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:46.402+0000 7f5b5ed9d640 1 --2- 172.21.15.169:0/3880308248 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5b60155cf0 0x7f5b6006ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:43:46.409 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:46.402+0000 7f5b57fff640 1 --2- 172.21.15.169:0/3880308248 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b600cd410 0x7f5b600acee0 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f5b48009850 tx=0x7f5b4802da50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:46.409 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:46.403+0000 7f5b56ffd640 1 -- 172.21.15.169:0/3880308248 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b4802de50 con 0x7f5b600cd410 2023-10-26T01:43:46.409 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:46.403+0000 7f5b56ffd640 1 -- 172.21.15.169:0/3880308248 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5b480036e0 con 0x7f5b600cd410 2023-10-26T01:43:46.409 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:46.403+0000 7f5b56ffd640 1 -- 172.21.15.169:0/3880308248 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b48003a90 con 0x7f5b600cd410 2023-10-26T01:43:46.409 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:46.403+0000 7f5b5f59e640 1 -- 172.21.15.169:0/3880308248 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b600adf60 con 0x7f5b600cd410 2023-10-26T01:43:46.410 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:46.403+0000 7f5b5f59e640 1 -- 172.21.15.169:0/3880308248 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b600ae520 con 0x7f5b600cd410 2023-10-26T01:43:46.410 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:46.404+0000 7f5b5f59e640 1 -- 172.21.15.169:0/3880308248 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b600ae740 con 0x7f5b600cd410 2023-10-26T01:43:46.410 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:46.405+0000 7f5b56ffd640 1 -- 172.21.15.169:0/3880308248 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f5b48003bf0 con 0x7f5b600cd410 2023-10-26T01:43:46.411 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:46.406+0000 7f5b56ffd640 1 --2- 172.21.15.169:0/3880308248 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5b4006ae70 0x7f5b4006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:46.411 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:46.406+0000 7f5b56ffd640 1 -- 172.21.15.169:0/3880308248 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5b480abe30 con 0x7f5b600cd410 2023-10-26T01:43:46.411 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:46.407+0000 7f5b5e59c640 1 --2- 172.21.15.169:0/3880308248 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5b4006ae70 0x7f5b4006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:46.411 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:46.408+0000 7f5b5e59c640 1 --2- 172.21.15.169:0/3880308248 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5b4006ae70 0x7f5b4006d330 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f5b50000f60 tx=0x7f5b50009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:46.413 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:46.411+0000 7f5b56ffd640 1 -- 172.21.15.169:0/3880308248 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5b480772c0 con 0x7f5b600cd410 2023-10-26T01:43:46.594 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:46.592+0000 7f5b5f59e640 1 -- 172.21.15.169:0/3880308248 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5b60073c10 con 0x7f5b600cd410 2023-10-26T01:43:46.595 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:46.593+0000 7f5b56ffd640 1 -- 172.21.15.169:0/3880308248 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v178) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5b48043070 con 0x7f5b600cd410 2023-10-26T01:43:46.595 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 1 2023-10-26T01:43:46.598 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:46.596+0000 7f5b5f59e640 1 -- 172.21.15.169:0/3880308248 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5b4006ae70 msgr2=0x7f5b4006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:46.598 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:46.596+0000 7f5b5f59e640 1 --2- 172.21.15.169:0/3880308248 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5b4006ae70 0x7f5b4006d330 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f5b50000f60 tx=0x7f5b50009040 comp rx=0 tx=0).stop 2023-10-26T01:43:46.598 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:46.596+0000 7f5b5f59e640 1 -- 172.21.15.169:0/3880308248 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b600cd410 msgr2=0x7f5b600acee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:46.599 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:46.596+0000 7f5b5f59e640 1 --2- 172.21.15.169:0/3880308248 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b600cd410 0x7f5b600acee0 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f5b48009850 tx=0x7f5b4802da50 comp rx=0 tx=0).stop 2023-10-26T01:43:46.599 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:46.596+0000 7f5b5f59e640 1 -- 172.21.15.169:0/3880308248 shutdown_connections 2023-10-26T01:43:46.599 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:46.596+0000 7f5b5f59e640 1 --2- 172.21.15.169:0/3880308248 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5b4006ae70 0x7f5b4006d330 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:46.599 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:46.596+0000 7f5b5f59e640 1 --2- 172.21.15.169:0/3880308248 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b600cd410 0x7f5b600acee0 unknown :-1 s=CLOSED pgs=1820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:46.599 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:46.596+0000 7f5b5f59e640 1 --2- 172.21.15.169:0/3880308248 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5b60155cf0 0x7f5b6006ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:46.600 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:46.596+0000 7f5b5f59e640 1 --2- 172.21.15.169:0/3880308248 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5b600cb850 0x7f5b601509a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:46.600 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:46.596+0000 7f5b5f59e640 1 -- 172.21.15.169:0/3880308248 >> 172.21.15.169:0/3880308248 conn(0x7f5b60069510 msgr2=0x7f5b600698f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:46.600 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:46.596+0000 7f5b5f59e640 1 -- 172.21.15.169:0/3880308248 shutdown_connections 2023-10-26T01:43:46.600 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:46.596+0000 7f5b5f59e640 1 -- 172.21.15.169:0/3880308248 wait complete. 2023-10-26T01:43:46.615 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-26T01:43:46.828 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:46.826+0000 7f3686bb8640 1 -- 172.21.15.169:0/1328228901 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3680155c40 msgr2=0x7f368006ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:46.828 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:46.826+0000 7f3686bb8640 1 --2- 172.21.15.169:0/1328228901 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3680155c40 0x7f368006ba10 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f36800c6d70 tx=0x7f367002bd90 comp rx=0 tx=0).stop 2023-10-26T01:43:46.829 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:46.826+0000 7f3686bb8640 1 -- 172.21.15.169:0/1328228901 shutdown_connections 2023-10-26T01:43:46.829 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:46.826+0000 7f3686bb8640 1 --2- 172.21.15.169:0/1328228901 >> v2:172.21.15.170:3300/0 conn(0x7f36800c6270 0x7f36800c6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:46.829 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:46.826+0000 7f3686bb8640 1 --2- 172.21.15.169:0/1328228901 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3680155c40 0x7f368006ba10 unknown :-1 s=CLOSED pgs=1821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:46.829 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:46.826+0000 7f3686bb8640 1 --2- 172.21.15.169:0/1328228901 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3680155860 0x7f368006b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:46.829 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:46.826+0000 7f3686bb8640 1 -- 172.21.15.169:0/1328228901 >> 172.21.15.169:0/1328228901 conn(0x7f36800aedd0 msgr2=0x7f36800af1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:46.830 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:46.826+0000 7f3686bb8640 1 -- 172.21.15.169:0/1328228901 shutdown_connections 2023-10-26T01:43:46.830 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:46.826+0000 7f3686bb8640 1 -- 172.21.15.169:0/1328228901 wait complete. 2023-10-26T01:43:46.832 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:46.828+0000 7f3686bb8640 1 Processor -- start 2023-10-26T01:43:46.832 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:46.828+0000 7f3686bb8640 1 -- start start 2023-10-26T01:43:46.832 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:46.829+0000 7f3686bb8640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f36800c6270 0x7f368015e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:46.832 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:46.829+0000 7f3686bb8640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3680155860 0x7f3680156c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:46.832 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:46.829+0000 7f3686bb8640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3680157180 0x7f36800ab4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:46.833 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:46.829+0000 7f3686bb8640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36801567b0 con 0x7f36800c6270 2023-10-26T01:43:46.833 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:46.829+0000 7f3686bb8640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36800ac260 con 0x7f3680155860 2023-10-26T01:43:46.833 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:46.829+0000 7f3686bb8640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36800abf80 con 0x7f3680157180 2023-10-26T01:43:46.833 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:46.829+0000 7f3685bb6640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f36800c6270 0x7f368015e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:46.834 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:46.829+0000 7f3685bb6640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f36800c6270 0x7f368015e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:47808/0 (socket says 172.21.15.169:47808) 2023-10-26T01:43:46.834 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:46.829+0000 7f3685bb6640 1 -- 172.21.15.169:0/4151632587 learned_addr learned my addr 172.21.15.169:0/4151632587 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:43:46.834 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:46.829+0000 7f3685bb6640 1 -- 172.21.15.169:0/4151632587 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3680157180 msgr2=0x7f36800ab4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:46.834 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:46.829+0000 7f3685bb6640 1 --2- 172.21.15.169:0/4151632587 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3680157180 0x7f36800ab4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:46.834 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:46.829+0000 7f3685bb6640 1 -- 172.21.15.169:0/4151632587 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3680155860 msgr2=0x7f3680156c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:46.835 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:46.829+0000 7f3685bb6640 1 --2- 172.21.15.169:0/4151632587 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3680155860 0x7f3680156c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:46.835 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:46.829+0000 7f3685bb6640 1 -- 172.21.15.169:0/4151632587 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3670009560 con 0x7f36800c6270 2023-10-26T01:43:46.835 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:46.830+0000 7f3685bb6640 1 --2- 172.21.15.169:0/4151632587 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f36800c6270 0x7f368015e300 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f367c00d770 tx=0x7f367c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:46.835 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:46.830+0000 7f36767fc640 1 -- 172.21.15.169:0/4151632587 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f367c009c30 con 0x7f36800c6270 2023-10-26T01:43:46.835 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:46.830+0000 7f36767fc640 1 -- 172.21.15.169:0/4151632587 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f367c00de80 con 0x7f36800c6270 2023-10-26T01:43:46.836 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:46.830+0000 7f3686bb8640 1 -- 172.21.15.169:0/4151632587 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3680156160 con 0x7f36800c6270 2023-10-26T01:43:46.836 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:46.830+0000 7f36767fc640 1 -- 172.21.15.169:0/4151632587 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f367c005280 con 0x7f36800c6270 2023-10-26T01:43:46.836 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:46.830+0000 7f3686bb8640 1 -- 172.21.15.169:0/4151632587 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3680148050 con 0x7f36800c6270 2023-10-26T01:43:46.837 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:46.831+0000 7f3686bb8640 1 -- 172.21.15.169:0/4151632587 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3680073c10 con 0x7f36800c6270 2023-10-26T01:43:46.837 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:46.832+0000 7f36767fc640 1 -- 172.21.15.169:0/4151632587 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f367c0053e0 con 0x7f36800c6270 2023-10-26T01:43:46.837 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:46.833+0000 7f36767fc640 1 --2- 172.21.15.169:0/4151632587 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f366406ae40 0x7f366406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:46.837 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:46.834+0000 7f36767fc640 1 -- 172.21.15.169:0/4151632587 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f367c089400 con 0x7f36800c6270 2023-10-26T01:43:46.838 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:46.834+0000 7f36853b5640 1 --2- 172.21.15.169:0/4151632587 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f366406ae40 0x7f366406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:46.838 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:46.835+0000 7f36853b5640 1 --2- 172.21.15.169:0/4151632587 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f366406ae40 0x7f366406d300 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f3670035000 tx=0x7f36700046c0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:46.839 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:46.838+0000 7f36767fc640 1 -- 172.21.15.169:0/4151632587 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f367c054890 con 0x7f36800c6270 2023-10-26T01:43:47.028 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:47.026+0000 7f3686bb8640 1 -- 172.21.15.169:0/4151632587 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f36800c9170 con 0x7f36800c6270 2023-10-26T01:43:47.029 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:47.027+0000 7f36767fc640 1 -- 172.21.15.169:0/4151632587 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v178) v1 ==== 84+0+65 (secure 0 0 0) 0x7f367c017090 con 0x7f36800c6270 2023-10-26T01:43:47.030 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:43:47.030 INFO:teuthology.orchestra.run.smithi169.stdout:[ 2023-10-26T01:43:47.031 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:43:47.031 INFO:teuthology.orchestra.run.smithi169.stdout: "poolnum": 1, 2023-10-26T01:43:47.032 INFO:teuthology.orchestra.run.smithi169.stdout: "poolname": ".mgr" 2023-10-26T01:43:47.032 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:43:47.032 INFO:teuthology.orchestra.run.smithi169.stdout:] 2023-10-26T01:43:47.035 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:47.030+0000 7f3686bb8640 1 -- 172.21.15.169:0/4151632587 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f366406ae40 msgr2=0x7f366406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:47.035 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:47.030+0000 7f3686bb8640 1 --2- 172.21.15.169:0/4151632587 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f366406ae40 0x7f366406d300 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f3670035000 tx=0x7f36700046c0 comp rx=0 tx=0).stop 2023-10-26T01:43:47.036 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:47.030+0000 7f3686bb8640 1 -- 172.21.15.169:0/4151632587 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f36800c6270 msgr2=0x7f368015e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:47.036 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:47.030+0000 7f3686bb8640 1 --2- 172.21.15.169:0/4151632587 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f36800c6270 0x7f368015e300 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f367c00d770 tx=0x7f367c00dc30 comp rx=0 tx=0).stop 2023-10-26T01:43:47.037 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:47.030+0000 7f3686bb8640 1 -- 172.21.15.169:0/4151632587 shutdown_connections 2023-10-26T01:43:47.037 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:47.030+0000 7f3686bb8640 1 --2- 172.21.15.169:0/4151632587 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f366406ae40 0x7f366406d300 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:47.037 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:47.030+0000 7f3686bb8640 1 --2- 172.21.15.169:0/4151632587 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3680157180 0x7f36800ab4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:47.038 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:47.030+0000 7f3686bb8640 1 --2- 172.21.15.169:0/4151632587 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3680155860 0x7f3680156c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:47.038 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:47.030+0000 7f3686bb8640 1 --2- 172.21.15.169:0/4151632587 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f36800c6270 0x7f368015e300 unknown :-1 s=CLOSED pgs=1822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:47.038 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:47.030+0000 7f3686bb8640 1 -- 172.21.15.169:0/4151632587 >> 172.21.15.169:0/4151632587 conn(0x7f36800aedd0 msgr2=0x7f36800c9a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:47.039 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:47.031+0000 7f3686bb8640 1 -- 172.21.15.169:0/4151632587 shutdown_connections 2023-10-26T01:43:47.039 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:47.031+0000 7f3686bb8640 1 -- 172.21.15.169:0/4151632587 wait complete. 2023-10-26T01:43:47.051 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-26T01:43:47.052 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-26T01:43:47.267 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:47.265+0000 7fe3fae7c640 1 -- 172.21.15.169:0/1751910458 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe3f40b08f0 msgr2=0x7fe3f40cdb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:47.268 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:47.265+0000 7fe3fae7c640 1 --2- 172.21.15.169:0/1751910458 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe3f40b08f0 0x7fe3f40cdb30 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec009ca0 tx=0x7fe3ec02c240 comp rx=0 tx=0).stop 2023-10-26T01:43:47.268 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:47.265+0000 7fe3fae7c640 1 -- 172.21.15.169:0/1751910458 shutdown_connections 2023-10-26T01:43:47.268 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:47.265+0000 7fe3fae7c640 1 --2- 172.21.15.169:0/1751910458 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe3f40b08f0 0x7fe3f40cdb30 unknown :-1 s=CLOSED pgs=1824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:47.268 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:47.265+0000 7fe3fae7c640 1 --2- 172.21.15.169:0/1751910458 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe3f41539d0 0x7fe3f40cd5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:47.269 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:47.265+0000 7fe3fae7c640 1 -- 172.21.15.169:0/1751910458 >> 172.21.15.169:0/1751910458 conn(0x7fe3f4069930 msgr2=0x7fe3f4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:47.269 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:47.265+0000 7fe3fae7c640 1 -- 172.21.15.169:0/1751910458 shutdown_connections 2023-10-26T01:43:47.269 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:47.265+0000 7fe3fae7c640 1 -- 172.21.15.169:0/1751910458 wait complete. 2023-10-26T01:43:47.271 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:47.267+0000 7fe3fae7c640 1 Processor -- start 2023-10-26T01:43:47.272 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:47.267+0000 7fe3fae7c640 1 -- start start 2023-10-26T01:43:47.272 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:47.268+0000 7fe3fae7c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe3f40b08f0 0x7fe3f4152090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:47.272 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:47.268+0000 7fe3fae7c640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe3f41539d0 0x7fe3f41f1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:47.272 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:47.268+0000 7fe3fae7c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe3f415aae0 0x7fe3f4070e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:47.273 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:47.268+0000 7fe3fae7c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f40c9420 con 0x7fe3f415aae0 2023-10-26T01:43:47.273 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:47.268+0000 7fe3fae7c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f40ad310 con 0x7fe3f40b08f0 2023-10-26T01:43:47.273 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:47.268+0000 7fe3fae7c640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f40c8e60 con 0x7fe3f41539d0 2023-10-26T01:43:47.273 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:47.269+0000 7fe3f9419640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe3f415aae0 0x7fe3f4070e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:47.273 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:47.269+0000 7fe3f9419640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe3f415aae0 0x7fe3f4070e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:47826/0 (socket says 172.21.15.169:47826) 2023-10-26T01:43:47.274 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:47.269+0000 7fe3f9419640 1 -- 172.21.15.169:0/1112149588 learned_addr learned my addr 172.21.15.169:0/1112149588 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:43:47.274 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:47.269+0000 7fe3f8c18640 1 --2- 172.21.15.169:0/1112149588 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe3f40b08f0 0x7fe3f4152090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:47.274 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:47.269+0000 7fe3f3fff640 1 --2- 172.21.15.169:0/1112149588 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe3f41539d0 0x7fe3f41f1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:47.274 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:47.269+0000 7fe3f9419640 1 -- 172.21.15.169:0/1112149588 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe3f41539d0 msgr2=0x7fe3f41f1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:47.275 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:47.269+0000 7fe3f9419640 1 --2- 172.21.15.169:0/1112149588 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe3f41539d0 0x7fe3f41f1cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:47.275 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:47.269+0000 7fe3f9419640 1 -- 172.21.15.169:0/1112149588 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe3f40b08f0 msgr2=0x7fe3f4152090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:47.275 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:47.269+0000 7fe3f9419640 1 --2- 172.21.15.169:0/1112149588 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe3f40b08f0 0x7fe3f4152090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:47.275 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:47.269+0000 7fe3f9419640 1 -- 172.21.15.169:0/1112149588 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3ec009900 con 0x7fe3f415aae0 2023-10-26T01:43:47.276 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:47.269+0000 7fe3f9419640 1 --2- 172.21.15.169:0/1112149588 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe3f415aae0 0x7fe3f4070e90 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec006010 tx=0x7fe3ec035a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:47.276 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:47.270+0000 7fe3f17fa640 1 -- 172.21.15.169:0/1112149588 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3ec034440 con 0x7fe3f415aae0 2023-10-26T01:43:47.276 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:47.270+0000 7fe3f17fa640 1 -- 172.21.15.169:0/1112149588 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe3ec004330 con 0x7fe3f415aae0 2023-10-26T01:43:47.276 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:47.270+0000 7fe3f17fa640 1 -- 172.21.15.169:0/1112149588 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3ec0046c0 con 0x7fe3f415aae0 2023-10-26T01:43:47.276 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:47.270+0000 7fe3fae7c640 1 -- 172.21.15.169:0/1112149588 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3f40c9140 con 0x7fe3f415aae0 2023-10-26T01:43:47.277 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:47.270+0000 7fe3fae7c640 1 -- 172.21.15.169:0/1112149588 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3f40ac2f0 con 0x7fe3f415aae0 2023-10-26T01:43:47.278 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:47.271+0000 7fe3fae7c640 1 -- 172.21.15.169:0/1112149588 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3f4066af0 con 0x7fe3f415aae0 2023-10-26T01:43:47.278 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:47.272+0000 7fe3f17fa640 1 -- 172.21.15.169:0/1112149588 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fe3ec03b040 con 0x7fe3f415aae0 2023-10-26T01:43:47.278 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:47.273+0000 7fe3f17fa640 1 --2- 172.21.15.169:0/1112149588 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe3d806ae40 0x7fe3d806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:47.279 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:47.273+0000 7fe3f17fa640 1 -- 172.21.15.169:0/1112149588 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fe3ec0abea0 con 0x7fe3f415aae0 2023-10-26T01:43:47.279 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:47.273+0000 7fe3f8c18640 1 --2- 172.21.15.169:0/1112149588 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe3d806ae40 0x7fe3d806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:47.279 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:47.275+0000 7fe3f8c18640 1 --2- 172.21.15.169:0/1112149588 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe3d806ae40 0x7fe3d806d300 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fe3e8005fd0 tx=0x7fe3e8005850 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:47.280 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:47.277+0000 7fe3f17fa640 1 -- 172.21.15.169:0/1112149588 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe3ec077290 con 0x7fe3f415aae0 2023-10-26T01:43:47.451 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:47.449+0000 7fe3fae7c640 1 -- 172.21.15.169:0/1112149588 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fe3f40acde0 con 0x7fe3f415aae0 2023-10-26T01:43:48.239 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:48.237+0000 7fe3f17fa640 1 -- 172.21.15.169:0/1112149588 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v179) v1 ==== 138+0+0 (secure 0 0 0) 0x7fe3ec07a8b0 con 0x7fe3f415aae0 2023-10-26T01:43:48.239 INFO:teuthology.orchestra.run.smithi169.stderr:pool 'cephfs_metadata' created 2023-10-26T01:43:48.243 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:48.241+0000 7fe3fae7c640 1 -- 172.21.15.169:0/1112149588 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe3d806ae40 msgr2=0x7fe3d806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:48.243 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:48.241+0000 7fe3fae7c640 1 --2- 172.21.15.169:0/1112149588 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe3d806ae40 0x7fe3d806d300 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fe3e8005fd0 tx=0x7fe3e8005850 comp rx=0 tx=0).stop 2023-10-26T01:43:48.243 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:48.241+0000 7fe3fae7c640 1 -- 172.21.15.169:0/1112149588 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe3f415aae0 msgr2=0x7fe3f4070e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:48.244 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:48.241+0000 7fe3fae7c640 1 --2- 172.21.15.169:0/1112149588 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe3f415aae0 0x7fe3f4070e90 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec006010 tx=0x7fe3ec035a40 comp rx=0 tx=0).stop 2023-10-26T01:43:48.244 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:48.241+0000 7fe3fae7c640 1 -- 172.21.15.169:0/1112149588 shutdown_connections 2023-10-26T01:43:48.244 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:48.241+0000 7fe3fae7c640 1 --2- 172.21.15.169:0/1112149588 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe3d806ae40 0x7fe3d806d300 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:48.244 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:48.241+0000 7fe3fae7c640 1 --2- 172.21.15.169:0/1112149588 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe3f415aae0 0x7fe3f4070e90 unknown :-1 s=CLOSED pgs=1825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:48.244 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:48.241+0000 7fe3fae7c640 1 --2- 172.21.15.169:0/1112149588 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe3f41539d0 0x7fe3f41f1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:48.245 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:48.241+0000 7fe3fae7c640 1 --2- 172.21.15.169:0/1112149588 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe3f40b08f0 0x7fe3f4152090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:48.245 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:48.241+0000 7fe3fae7c640 1 -- 172.21.15.169:0/1112149588 >> 172.21.15.169:0/1112149588 conn(0x7fe3f4069930 msgr2=0x7fe3f4158f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:48.245 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:48.241+0000 7fe3fae7c640 1 -- 172.21.15.169:0/1112149588 shutdown_connections 2023-10-26T01:43:48.245 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:48.241+0000 7fe3fae7c640 1 -- 172.21.15.169:0/1112149588 wait complete. 2023-10-26T01:43:48.267 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-26T01:43:48.485 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:48.483+0000 7fc73a528640 1 -- 172.21.15.169:0/2037177940 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7340b68d0 msgr2=0x7fc734065ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:48.486 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:48.483+0000 7fc73a528640 1 --2- 172.21.15.169:0/2037177940 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7340b68d0 0x7fc734065ea0 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7fc7340cd630 tx=0x7fc73002bd10 comp rx=0 tx=0).stop 2023-10-26T01:43:48.486 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:48.483+0000 7fc73a528640 1 -- 172.21.15.169:0/2037177940 shutdown_connections 2023-10-26T01:43:48.486 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:48.483+0000 7fc73a528640 1 --2- 172.21.15.169:0/2037177940 >> v2:172.21.15.170:3300/0 conn(0x7fc7340d0490 0x7fc7340d0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:48.487 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:48.483+0000 7fc73a528640 1 --2- 172.21.15.169:0/2037177940 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7340b68d0 0x7fc734065ea0 unknown :-1 s=CLOSED pgs=1826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:48.487 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:48.483+0000 7fc73a528640 1 -- 172.21.15.169:0/2037177940 >> 172.21.15.169:0/2037177940 conn(0x7fc7340d1360 msgr2=0x7fc7340d1770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:48.487 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:48.483+0000 7fc73a528640 1 -- 172.21.15.169:0/2037177940 shutdown_connections 2023-10-26T01:43:48.487 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:48.483+0000 7fc73a528640 1 -- 172.21.15.169:0/2037177940 wait complete. 2023-10-26T01:43:48.489 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:48.485+0000 7fc73a528640 1 Processor -- start 2023-10-26T01:43:48.489 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:48.485+0000 7fc73a528640 1 -- start start 2023-10-26T01:43:48.490 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:48.486+0000 7fc73a528640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc7340b68d0 0x7fc734156f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:48.490 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:48.486+0000 7fc73a528640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc7340cef30 0x7fc734159dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:48.490 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:48.486+0000 7fc73a528640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7340d0490 0x7fc734153480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:48.491 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:48.486+0000 7fc73a528640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7341e6ec0 con 0x7fc7340d0490 2023-10-26T01:43:48.491 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:48.486+0000 7fc73a528640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc734157f80 con 0x7fc7340cef30 2023-10-26T01:43:48.491 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:48.486+0000 7fc73a528640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc734158220 con 0x7fc7340b68d0 2023-10-26T01:43:48.491 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:48.486+0000 7fc739d27640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7340d0490 0x7fc734153480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:48.492 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:48.486+0000 7fc739d27640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7340d0490 0x7fc734153480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:47834/0 (socket says 172.21.15.169:47834) 2023-10-26T01:43:48.492 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:48.486+0000 7fc739d27640 1 -- 172.21.15.169:0/444064397 learned_addr learned my addr 172.21.15.169:0/444064397 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:43:48.492 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:48.487+0000 7fc739526640 1 --2- 172.21.15.169:0/444064397 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc7340b68d0 0x7fc734156f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:48.492 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:48.487+0000 7fc739d27640 1 -- 172.21.15.169:0/444064397 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc7340b68d0 msgr2=0x7fc734156f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:48.492 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:48.487+0000 7fc738d25640 1 --2- 172.21.15.169:0/444064397 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc7340cef30 0x7fc734159dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:48.493 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:48.487+0000 7fc739d27640 1 --2- 172.21.15.169:0/444064397 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc7340b68d0 0x7fc734156f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:48.493 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:48.487+0000 7fc739d27640 1 -- 172.21.15.169:0/444064397 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc7340cef30 msgr2=0x7fc734159dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:48.493 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:48.487+0000 7fc739d27640 1 --2- 172.21.15.169:0/444064397 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc7340cef30 0x7fc734159dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:48.493 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:48.487+0000 7fc739d27640 1 -- 172.21.15.169:0/444064397 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc730009560 con 0x7fc7340d0490 2023-10-26T01:43:48.494 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:48.487+0000 7fc738d25640 1 --2- 172.21.15.169:0/444064397 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc7340cef30 0x7fc734159dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:43:48.494 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:48.487+0000 7fc739d27640 1 --2- 172.21.15.169:0/444064397 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7340d0490 0x7fc734153480 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7fc72c00cab0 tx=0x7fc72c00cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:48.494 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:48.487+0000 7fc729ffb640 1 -- 172.21.15.169:0/444064397 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc72c007d90 con 0x7fc7340d0490 2023-10-26T01:43:48.494 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:48.487+0000 7fc729ffb640 1 -- 172.21.15.169:0/444064397 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc72c00f450 con 0x7fc7340d0490 2023-10-26T01:43:48.494 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:48.487+0000 7fc73a528640 1 -- 172.21.15.169:0/444064397 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7341e4ee0 con 0x7fc7340d0490 2023-10-26T01:43:48.495 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:48.487+0000 7fc729ffb640 1 -- 172.21.15.169:0/444064397 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc72c00f840 con 0x7fc7340d0490 2023-10-26T01:43:48.495 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:48.487+0000 7fc73a528640 1 -- 172.21.15.169:0/444064397 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc734158500 con 0x7fc7340d0490 2023-10-26T01:43:48.496 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:48.488+0000 7fc73a528640 1 -- 172.21.15.169:0/444064397 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc734073c10 con 0x7fc7340d0490 2023-10-26T01:43:48.496 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:48.490+0000 7fc729ffb640 1 -- 172.21.15.169:0/444064397 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fc72c004b20 con 0x7fc7340d0490 2023-10-26T01:43:48.496 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:48.491+0000 7fc729ffb640 1 --2- 172.21.15.169:0/444064397 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc71806ae70 0x7fc71806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:48.496 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:48.491+0000 7fc729ffb640 1 -- 172.21.15.169:0/444064397 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fc72c08a290 con 0x7fc7340d0490 2023-10-26T01:43:48.497 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:48.492+0000 7fc739526640 1 --2- 172.21.15.169:0/444064397 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc71806ae70 0x7fc71806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:48.497 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:48.493+0000 7fc739526640 1 --2- 172.21.15.169:0/444064397 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc71806ae70 0x7fc71806d330 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fc730035000 tx=0x7fc730004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:48.497 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:48.495+0000 7fc729ffb640 1 -- 172.21.15.169:0/444064397 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc72c055590 con 0x7fc7340d0490 2023-10-26T01:43:48.683 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:48.681+0000 7fc73a528640 1 -- 172.21.15.169:0/444064397 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fc7340d1ee0 con 0x7fc7340d0490 2023-10-26T01:43:49.246 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:49.244+0000 7fc729ffb640 1 -- 172.21.15.169:0/444064397 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v180) v1 ==== 170+0+0 (secure 0 0 0) 0x7fc72c01d0d0 con 0x7fc7340d0490 2023-10-26T01:43:49.246 INFO:teuthology.orchestra.run.smithi169.stderr:pool 'cephfs_data' created 2023-10-26T01:43:49.250 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:49.247+0000 7fc73a528640 1 -- 172.21.15.169:0/444064397 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc71806ae70 msgr2=0x7fc71806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:49.250 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:49.247+0000 7fc73a528640 1 --2- 172.21.15.169:0/444064397 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc71806ae70 0x7fc71806d330 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fc730035000 tx=0x7fc730004730 comp rx=0 tx=0).stop 2023-10-26T01:43:49.250 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:49.247+0000 7fc73a528640 1 -- 172.21.15.169:0/444064397 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7340d0490 msgr2=0x7fc734153480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:49.250 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:49.247+0000 7fc73a528640 1 --2- 172.21.15.169:0/444064397 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7340d0490 0x7fc734153480 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7fc72c00cab0 tx=0x7fc72c00cf70 comp rx=0 tx=0).stop 2023-10-26T01:43:49.251 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:49.248+0000 7fc73a528640 1 -- 172.21.15.169:0/444064397 shutdown_connections 2023-10-26T01:43:49.251 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:49.248+0000 7fc73a528640 1 --2- 172.21.15.169:0/444064397 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc71806ae70 0x7fc71806d330 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:49.251 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:49.248+0000 7fc73a528640 1 --2- 172.21.15.169:0/444064397 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7340d0490 0x7fc734153480 unknown :-1 s=CLOSED pgs=1827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:49.251 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:49.248+0000 7fc73a528640 1 --2- 172.21.15.169:0/444064397 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc7340cef30 0x7fc734159dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:49.252 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:49.248+0000 7fc73a528640 1 --2- 172.21.15.169:0/444064397 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc7340b68d0 0x7fc734156f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:49.252 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:49.248+0000 7fc73a528640 1 -- 172.21.15.169:0/444064397 >> 172.21.15.169:0/444064397 conn(0x7fc7340d1360 msgr2=0x7fc7340b28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:49.252 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:49.248+0000 7fc73a528640 1 -- 172.21.15.169:0/444064397 shutdown_connections 2023-10-26T01:43:49.252 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:49.248+0000 7fc73a528640 1 -- 172.21.15.169:0/444064397 wait complete. 2023-10-26T01:43:49.279 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-26T01:43:49.473 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:49.470+0000 7f3c696f2640 1 -- 172.21.15.169:0/2943308396 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3c640cb6c0 msgr2=0x7f3c640cbb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:49.474 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:49.470+0000 7f3c696f2640 1 --2- 172.21.15.169:0/2943308396 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3c640cb6c0 0x7f3c640cbb20 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f3c640cb510 tx=0x7f3c4c02bd90 comp rx=0 tx=0).stop 2023-10-26T01:43:49.474 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:49.470+0000 7f3c696f2640 1 -- 172.21.15.169:0/2943308396 shutdown_connections 2023-10-26T01:43:49.474 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:49.470+0000 7f3c696f2640 1 --2- 172.21.15.169:0/2943308396 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3c640cb6c0 0x7f3c640cbb20 unknown :-1 s=CLOSED pgs=1829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:49.474 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:49.470+0000 7f3c696f2640 1 --2- 172.21.15.169:0/2943308396 >> v2:172.21.15.170:3300/0 conn(0x7f3c6415b760 0x7f3c64070420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:49.475 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:49.470+0000 7f3c696f2640 1 -- 172.21.15.169:0/2943308396 >> 172.21.15.169:0/2943308396 conn(0x7f3c640af570 msgr2=0x7f3c640af980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:49.475 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:49.470+0000 7f3c696f2640 1 -- 172.21.15.169:0/2943308396 shutdown_connections 2023-10-26T01:43:49.475 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:49.470+0000 7f3c696f2640 1 -- 172.21.15.169:0/2943308396 wait complete. 2023-10-26T01:43:49.477 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:49.472+0000 7f3c696f2640 1 Processor -- start 2023-10-26T01:43:49.478 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:49.472+0000 7f3c696f2640 1 -- start start 2023-10-26T01:43:49.478 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:49.473+0000 7f3c696f2640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3c640cb6c0 0x7f3c640aace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:49.478 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:49.473+0000 7f3c696f2640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c640cc060 0x7f3c640afbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:49.478 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:49.473+0000 7f3c696f2640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3c6415b760 0x7f3c641f9a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:49.479 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:49.473+0000 7f3c696f2640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c640ab220 con 0x7f3c640cb6c0 2023-10-26T01:43:49.479 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:49.473+0000 7f3c696f2640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c6415df10 con 0x7f3c640cc060 2023-10-26T01:43:49.479 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:49.473+0000 7f3c696f2640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c6415e1f0 con 0x7f3c6415b760 2023-10-26T01:43:49.479 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:49.473+0000 7f3c62ffd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3c640cb6c0 0x7f3c640aace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:49.480 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:49.473+0000 7f3c62ffd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3c640cb6c0 0x7f3c640aace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:47852/0 (socket says 172.21.15.169:47852) 2023-10-26T01:43:49.480 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:49.473+0000 7f3c62ffd640 1 -- 172.21.15.169:0/3890515777 learned_addr learned my addr 172.21.15.169:0/3890515777 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:43:49.480 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:49.474+0000 7f3c627fc640 1 --2- 172.21.15.169:0/3890515777 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c640cc060 0x7f3c640afbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:49.480 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:49.474+0000 7f3c62ffd640 1 -- 172.21.15.169:0/3890515777 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3c6415b760 msgr2=0x7f3c641f9a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:49.480 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:49.474+0000 7f3c62ffd640 1 --2- 172.21.15.169:0/3890515777 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3c6415b760 0x7f3c641f9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:49.481 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:49.474+0000 7f3c62ffd640 1 -- 172.21.15.169:0/3890515777 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c640cc060 msgr2=0x7f3c640afbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:49.481 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:49.474+0000 7f3c62ffd640 1 --2- 172.21.15.169:0/3890515777 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c640cc060 0x7f3c640afbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:49.481 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:49.474+0000 7f3c62ffd640 1 -- 172.21.15.169:0/3890515777 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c4c009560 con 0x7f3c640cb6c0 2023-10-26T01:43:49.481 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:49.474+0000 7f3c62ffd640 1 --2- 172.21.15.169:0/3890515777 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3c640cb6c0 0x7f3c640aace0 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f3c5400c970 tx=0x7f3c5400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:49.482 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:49.474+0000 7f3c5b7fe640 1 -- 172.21.15.169:0/3890515777 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c54007bb0 con 0x7f3c640cb6c0 2023-10-26T01:43:49.482 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:49.474+0000 7f3c5b7fe640 1 -- 172.21.15.169:0/3890515777 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3c54007e00 con 0x7f3c640cb6c0 2023-10-26T01:43:49.482 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:49.474+0000 7f3c5b7fe640 1 -- 172.21.15.169:0/3890515777 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c5400f6b0 con 0x7f3c640cb6c0 2023-10-26T01:43:49.482 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:49.474+0000 7f3c696f2640 1 -- 172.21.15.169:0/3890515777 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c6415e4d0 con 0x7f3c640cb6c0 2023-10-26T01:43:49.482 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:49.474+0000 7f3c696f2640 1 -- 172.21.15.169:0/3890515777 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c640ab580 con 0x7f3c640cb6c0 2023-10-26T01:43:49.483 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:49.475+0000 7f3c696f2640 1 -- 172.21.15.169:0/3890515777 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c6415e710 con 0x7f3c640cb6c0 2023-10-26T01:43:49.484 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:49.476+0000 7f3c5b7fe640 1 -- 172.21.15.169:0/3890515777 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f3c5400f810 con 0x7f3c640cb6c0 2023-10-26T01:43:49.484 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:49.477+0000 7f3c5b7fe640 1 --2- 172.21.15.169:0/3890515777 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3c4406ae40 0x7f3c4406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:49.484 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:49.477+0000 7f3c5b7fe640 1 -- 172.21.15.169:0/3890515777 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f3c54089d30 con 0x7f3c640cb6c0 2023-10-26T01:43:49.484 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:49.478+0000 7f3c627fc640 1 --2- 172.21.15.169:0/3890515777 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3c4406ae40 0x7f3c4406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:49.485 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:49.479+0000 7f3c627fc640 1 --2- 172.21.15.169:0/3890515777 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3c4406ae40 0x7f3c4406d300 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c035000 tx=0x7f3c4c004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:49.485 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:49.482+0000 7f3c5b7fe640 1 -- 172.21.15.169:0/3890515777 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3c6415e710 con 0x7f3c640cb6c0 2023-10-26T01:43:49.710 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:49.708+0000 7f3c696f2640 1 -- 172.21.15.169:0/3890515777 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f3c641f9370 con 0x7f3c640cb6c0 2023-10-26T01:43:50.312 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:50.310+0000 7f3c5b7fe640 1 -- 172.21.15.169:0/3890515777 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 22 and data pool 23 v90) v1 ==== 174+0+0 (secure 0 0 0) 0x7f3c641f9370 con 0x7f3c640cb6c0 2023-10-26T01:43:50.313 INFO:teuthology.orchestra.run.smithi169.stderr:new fs with metadata pool 22 and data pool 23 2023-10-26T01:43:50.319 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:50.317+0000 7f3c696f2640 1 -- 172.21.15.169:0/3890515777 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3c4406ae40 msgr2=0x7f3c4406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:50.319 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:50.317+0000 7f3c696f2640 1 --2- 172.21.15.169:0/3890515777 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3c4406ae40 0x7f3c4406d300 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c035000 tx=0x7f3c4c004730 comp rx=0 tx=0).stop 2023-10-26T01:43:50.319 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:50.317+0000 7f3c696f2640 1 -- 172.21.15.169:0/3890515777 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3c640cb6c0 msgr2=0x7f3c640aace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:50.320 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:50.317+0000 7f3c696f2640 1 --2- 172.21.15.169:0/3890515777 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3c640cb6c0 0x7f3c640aace0 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f3c5400c970 tx=0x7f3c5400ce30 comp rx=0 tx=0).stop 2023-10-26T01:43:50.321 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:50.319+0000 7f3c696f2640 1 -- 172.21.15.169:0/3890515777 shutdown_connections 2023-10-26T01:43:50.321 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:50.319+0000 7f3c696f2640 1 --2- 172.21.15.169:0/3890515777 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3c4406ae40 0x7f3c4406d300 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:50.321 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:50.319+0000 7f3c696f2640 1 --2- 172.21.15.169:0/3890515777 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3c6415b760 0x7f3c641f9a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:50.321 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:50.319+0000 7f3c696f2640 1 --2- 172.21.15.169:0/3890515777 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c640cc060 0x7f3c640afbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:50.322 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:50.319+0000 7f3c696f2640 1 --2- 172.21.15.169:0/3890515777 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3c640cb6c0 0x7f3c640aace0 unknown :-1 s=CLOSED pgs=1830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:50.322 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:50.319+0000 7f3c696f2640 1 -- 172.21.15.169:0/3890515777 >> 172.21.15.169:0/3890515777 conn(0x7f3c640af570 msgr2=0x7f3c641571c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:50.323 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:50.321+0000 7f3c696f2640 1 -- 172.21.15.169:0/3890515777 shutdown_connections 2023-10-26T01:43:50.323 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:50.321+0000 7f3c696f2640 1 -- 172.21.15.169:0/3890515777 wait complete. 2023-10-26T01:43:50.344 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:43:50.533 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:50.531+0000 7fd91e7b6640 1 -- 172.21.15.169:0/2405117937 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9180ac940 msgr2=0x7fd9180cdbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:50.534 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:50.531+0000 7fd91e7b6640 1 --2- 172.21.15.169:0/2405117937 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9180ac940 0x7fd9180cdbb0 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7fd91806bff0 tx=0x7fd91002c260 comp rx=0 tx=0).stop 2023-10-26T01:43:50.534 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:50.531+0000 7fd91e7b6640 1 -- 172.21.15.169:0/2405117937 shutdown_connections 2023-10-26T01:43:50.534 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:50.531+0000 7fd91e7b6640 1 --2- 172.21.15.169:0/2405117937 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9180ac940 0x7fd9180cdbb0 unknown :-1 s=CLOSED pgs=1831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:50.535 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:50.531+0000 7fd91e7b6640 1 --2- 172.21.15.169:0/2405117937 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd9180ac4e0 0x7fd9180cd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:50.535 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:50.531+0000 7fd91e7b6640 1 --2- 172.21.15.169:0/2405117937 >> v2:172.21.15.170:3300/0 conn(0x7fd91815b610 0x7fd91806ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:50.535 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:50.531+0000 7fd91e7b6640 1 -- 172.21.15.169:0/2405117937 >> 172.21.15.169:0/2405117937 conn(0x7fd918069930 msgr2=0x7fd918069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:50.535 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:50.531+0000 7fd91e7b6640 1 -- 172.21.15.169:0/2405117937 shutdown_connections 2023-10-26T01:43:50.535 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:50.531+0000 7fd91e7b6640 1 -- 172.21.15.169:0/2405117937 wait complete. 2023-10-26T01:43:50.537 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:50.533+0000 7fd91e7b6640 1 Processor -- start 2023-10-26T01:43:50.538 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:50.533+0000 7fd91e7b6640 1 -- start start 2023-10-26T01:43:50.538 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:50.534+0000 7fd91e7b6640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd9180ac4e0 0x7fd91814f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:50.538 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:50.534+0000 7fd91e7b6640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9180ac940 0x7fd918154310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:50.538 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:50.534+0000 7fd91e7b6640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd91815b610 0x7fd918154c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:50.539 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:50.534+0000 7fd91e7b6640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd91814fdc0 con 0x7fd9180ac940 2023-10-26T01:43:50.539 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:50.534+0000 7fd91e7b6640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd918155680 con 0x7fd91815b610 2023-10-26T01:43:50.539 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:50.534+0000 7fd91e7b6640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd918153550 con 0x7fd9180ac4e0 2023-10-26T01:43:50.539 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:50.535+0000 7fd90ffff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9180ac940 0x7fd918154310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:50.539 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:50.535+0000 7fd90ffff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9180ac940 0x7fd918154310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:47876/0 (socket says 172.21.15.169:47876) 2023-10-26T01:43:50.540 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:50.535+0000 7fd90ffff640 1 -- 172.21.15.169:0/2561363846 learned_addr learned my addr 172.21.15.169:0/2561363846 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:43:50.540 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:50.535+0000 7fd90ffff640 1 -- 172.21.15.169:0/2561363846 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd9180ac4e0 msgr2=0x7fd91814f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:50.540 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:50.535+0000 7fd917fff640 1 --2- 172.21.15.169:0/2561363846 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd9180ac4e0 0x7fd91814f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:50.540 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:50.535+0000 7fd91cd53640 1 --2- 172.21.15.169:0/2561363846 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd91815b610 0x7fd918154c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:50.540 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:50.535+0000 7fd90ffff640 1 --2- 172.21.15.169:0/2561363846 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd9180ac4e0 0x7fd91814f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:50.541 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:50.535+0000 7fd90ffff640 1 -- 172.21.15.169:0/2561363846 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd91815b610 msgr2=0x7fd918154c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:50.541 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:50.535+0000 7fd90ffff640 1 --2- 172.21.15.169:0/2561363846 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd91815b610 0x7fd918154c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:50.541 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:50.535+0000 7fd90ffff640 1 -- 172.21.15.169:0/2561363846 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd910009900 con 0x7fd9180ac940 2023-10-26T01:43:50.541 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:50.535+0000 7fd90ffff640 1 --2- 172.21.15.169:0/2561363846 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9180ac940 0x7fd918154310 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7fd90000c940 tx=0x7fd90000ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:50.542 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:50.536+0000 7fd9157fa640 1 -- 172.21.15.169:0/2561363846 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd900007bb0 con 0x7fd9180ac940 2023-10-26T01:43:50.542 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:50.536+0000 7fd9157fa640 1 -- 172.21.15.169:0/2561363846 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd900007e20 con 0x7fd9180ac940 2023-10-26T01:43:50.542 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:50.536+0000 7fd9157fa640 1 -- 172.21.15.169:0/2561363846 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd900005280 con 0x7fd9180ac940 2023-10-26T01:43:50.542 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:50.536+0000 7fd91e7b6640 1 -- 172.21.15.169:0/2561363846 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd918153830 con 0x7fd9180ac940 2023-10-26T01:43:50.543 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:50.536+0000 7fd91e7b6640 1 -- 172.21.15.169:0/2561363846 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd918153df0 con 0x7fd9180ac940 2023-10-26T01:43:50.544 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:50.537+0000 7fd91e7b6640 1 -- 172.21.15.169:0/2561363846 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd91804db10 con 0x7fd9180ac940 2023-10-26T01:43:50.544 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:50.538+0000 7fd9157fa640 1 -- 172.21.15.169:0/2561363846 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fd9000053e0 con 0x7fd9180ac940 2023-10-26T01:43:50.544 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:50.539+0000 7fd9157fa640 1 --2- 172.21.15.169:0/2561363846 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd8fc06ae40 0x7fd8fc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:50.544 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:50.539+0000 7fd9157fa640 1 -- 172.21.15.169:0/2561363846 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd900089b60 con 0x7fd9180ac940 2023-10-26T01:43:50.545 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:50.540+0000 7fd917fff640 1 --2- 172.21.15.169:0/2561363846 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd8fc06ae40 0x7fd8fc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:50.545 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:50.541+0000 7fd917fff640 1 --2- 172.21.15.169:0/2561363846 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd8fc06ae40 0x7fd8fc06d300 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fd9080058c0 tx=0x7fd908005be0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:50.545 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:50.543+0000 7fd9157fa640 1 -- 172.21.15.169:0/2561363846 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd900054c00 con 0x7fd9180ac940 2023-10-26T01:43:50.736 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:50.734+0000 7fd91e7b6640 1 -- 172.21.15.169:0/2561363846 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd918071fc0 con 0x7fd9180ac940 2023-10-26T01:43:50.739 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:50.737+0000 7fd9157fa640 1 -- 172.21.15.169:0/2561363846 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v181) v1 ==== 74+0+16633 (secure 0 0 0) 0x7fd90001d070 con 0x7fd9180ac940 2023-10-26T01:43:50.740 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:43:50.740 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":181,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:43:49.710266+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2023-10-26T01:43:47.501296+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"181","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2023-10-26T01:43:48.737507+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"181","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:43:50.742 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:50.739+0000 7fd91e7b6640 1 -- 172.21.15.169:0/2561363846 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd8fc06ae40 msgr2=0x7fd8fc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:50.742 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:50.739+0000 7fd91e7b6640 1 --2- 172.21.15.169:0/2561363846 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd8fc06ae40 0x7fd8fc06d300 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fd9080058c0 tx=0x7fd908005be0 comp rx=0 tx=0).stop 2023-10-26T01:43:50.743 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:50.739+0000 7fd91e7b6640 1 -- 172.21.15.169:0/2561363846 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9180ac940 msgr2=0x7fd918154310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:50.743 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:50.739+0000 7fd91e7b6640 1 --2- 172.21.15.169:0/2561363846 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9180ac940 0x7fd918154310 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7fd90000c940 tx=0x7fd90000ce00 comp rx=0 tx=0).stop 2023-10-26T01:43:50.743 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:50.740+0000 7fd91e7b6640 1 -- 172.21.15.169:0/2561363846 shutdown_connections 2023-10-26T01:43:50.744 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:50.740+0000 7fd91e7b6640 1 --2- 172.21.15.169:0/2561363846 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fd8fc06ae40 0x7fd8fc06d300 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:50.744 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:50.740+0000 7fd91e7b6640 1 --2- 172.21.15.169:0/2561363846 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9180ac940 0x7fd918154310 unknown :-1 s=CLOSED pgs=1832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:50.744 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:50.740+0000 7fd91e7b6640 1 --2- 172.21.15.169:0/2561363846 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd91815b610 0x7fd918154c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:50.745 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:50.740+0000 7fd91e7b6640 1 --2- 172.21.15.169:0/2561363846 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd9180ac4e0 0x7fd91814f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:50.745 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:50.740+0000 7fd91e7b6640 1 -- 172.21.15.169:0/2561363846 >> 172.21.15.169:0/2561363846 conn(0x7fd918069930 msgr2=0x7fd9180cc7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:50.745 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:50.740+0000 7fd91e7b6640 1 -- 172.21.15.169:0/2561363846 shutdown_connections 2023-10-26T01:43:50.746 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:50.740+0000 7fd91e7b6640 1 -- 172.21.15.169:0/2561363846 wait complete. 2023-10-26T01:43:50.761 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:43:50.968 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:50.966+0000 7fe7f01ce640 1 -- 172.21.15.169:0/180954557 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7e80cdc30 msgr2=0x7fe7e80cbac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:50.968 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:50.966+0000 7fe7f01ce640 1 --2- 172.21.15.169:0/180954557 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7e80cdc30 0x7fe7e80cbac0 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7fe7e0005fd0 tx=0x7fe7e002c1b0 comp rx=0 tx=0).stop 2023-10-26T01:43:50.968 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:50.966+0000 7fe7f01ce640 1 -- 172.21.15.169:0/180954557 shutdown_connections 2023-10-26T01:43:50.969 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:50.966+0000 7fe7f01ce640 1 --2- 172.21.15.169:0/180954557 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7e80cdc30 0x7fe7e80cbac0 unknown :-1 s=CLOSED pgs=1833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:50.969 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:50.966+0000 7fe7f01ce640 1 --2- 172.21.15.169:0/180954557 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe7e80ac930 0x7fe7e80cd6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:50.969 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:50.966+0000 7fe7f01ce640 1 -- 172.21.15.169:0/180954557 >> 172.21.15.169:0/180954557 conn(0x7fe7e8069930 msgr2=0x7fe7e8069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:50.969 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:50.966+0000 7fe7f01ce640 1 -- 172.21.15.169:0/180954557 shutdown_connections 2023-10-26T01:43:50.969 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:50.966+0000 7fe7f01ce640 1 -- 172.21.15.169:0/180954557 wait complete. 2023-10-26T01:43:50.971 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:50.968+0000 7fe7f01ce640 1 Processor -- start 2023-10-26T01:43:50.971 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:50.968+0000 7fe7f01ce640 1 -- start start 2023-10-26T01:43:50.971 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:50.969+0000 7fe7f01ce640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe7e80ac930 0x7fe7e81e5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:50.972 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:50.969+0000 7fe7f01ce640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7e80cdc30 0x7fe7e814ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:50.972 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:50.969+0000 7fe7f01ce640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe7e81592d0 0x7fe7e80c8d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:50.972 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:50.969+0000 7fe7f01ce640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e81e47e0 con 0x7fe7e80cdc30 2023-10-26T01:43:50.972 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:50.969+0000 7fe7f01ce640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e81e4a30 con 0x7fe7e80ac930 2023-10-26T01:43:50.972 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:50.969+0000 7fe7f01ce640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e81e4d10 con 0x7fe7e81592d0 2023-10-26T01:43:50.973 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:50.969+0000 7fe7ed769640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7e80cdc30 0x7fe7e814ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:50.973 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:50.969+0000 7fe7ed769640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7e80cdc30 0x7fe7e814ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:47898/0 (socket says 172.21.15.169:47898) 2023-10-26T01:43:50.973 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:50.969+0000 7fe7ed769640 1 -- 172.21.15.169:0/1789785598 learned_addr learned my addr 172.21.15.169:0/1789785598 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:43:50.973 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:50.969+0000 7fe7ed769640 1 -- 172.21.15.169:0/1789785598 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe7e81592d0 msgr2=0x7fe7e80c8d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:50.973 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:50.969+0000 7fe7ed769640 1 --2- 172.21.15.169:0/1789785598 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe7e81592d0 0x7fe7e80c8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:50.974 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:50.969+0000 7fe7edf6a640 1 --2- 172.21.15.169:0/1789785598 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe7e80ac930 0x7fe7e81e5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:50.974 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:50.969+0000 7fe7ed769640 1 -- 172.21.15.169:0/1789785598 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe7e80ac930 msgr2=0x7fe7e81e5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:50.974 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:50.969+0000 7fe7ed769640 1 --2- 172.21.15.169:0/1789785598 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe7e80ac930 0x7fe7e81e5ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:50.974 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:50.969+0000 7fe7ed769640 1 -- 172.21.15.169:0/1789785598 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7e0009900 con 0x7fe7e80cdc30 2023-10-26T01:43:50.974 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:50.970+0000 7fe7ed769640 1 --2- 172.21.15.169:0/1789785598 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7e80cdc30 0x7fe7e814ce00 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7fe7d800c890 tx=0x7fe7d800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:50.975 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:50.970+0000 7fe7de7fc640 1 -- 172.21.15.169:0/1789785598 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d8004220 con 0x7fe7e80cdc30 2023-10-26T01:43:50.976 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:50.970+0000 7fe7de7fc640 1 -- 172.21.15.169:0/1789785598 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe7d8004470 con 0x7fe7e80cdc30 2023-10-26T01:43:50.976 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:50.970+0000 7fe7de7fc640 1 -- 172.21.15.169:0/1789785598 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d8007c10 con 0x7fe7e80cdc30 2023-10-26T01:43:50.976 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:50.970+0000 7fe7f01ce640 1 -- 172.21.15.169:0/1789785598 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7e81e4ff0 con 0x7fe7e80cdc30 2023-10-26T01:43:50.976 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:50.970+0000 7fe7f01ce640 1 -- 172.21.15.169:0/1789785598 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7e81e55b0 con 0x7fe7e80cdc30 2023-10-26T01:43:50.977 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:50.971+0000 7fe7f01ce640 1 -- 172.21.15.169:0/1789785598 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7e81e57d0 con 0x7fe7e80cdc30 2023-10-26T01:43:50.977 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:50.972+0000 7fe7de7fc640 1 -- 172.21.15.169:0/1789785598 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fe7d8005020 con 0x7fe7e80cdc30 2023-10-26T01:43:50.977 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:50.973+0000 7fe7de7fc640 1 --2- 172.21.15.169:0/1789785598 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe7cc06ae40 0x7fe7cc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:50.977 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:50.973+0000 7fe7de7fc640 1 -- 172.21.15.169:0/1789785598 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe7d8089cf0 con 0x7fe7e80cdc30 2023-10-26T01:43:50.977 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:50.974+0000 7fe7edf6a640 1 --2- 172.21.15.169:0/1789785598 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe7cc06ae40 0x7fe7cc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:50.978 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:50.975+0000 7fe7edf6a640 1 --2- 172.21.15.169:0/1789785598 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe7cc06ae40 0x7fe7cc06d300 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fe7e400a750 tx=0x7fe7e4008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:50.979 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:50.977+0000 7fe7de7fc640 1 -- 172.21.15.169:0/1789785598 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe7d8054c70 con 0x7fe7e80cdc30 2023-10-26T01:43:51.156 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:51.154+0000 7fe7f01ce640 1 -- 172.21.15.169:0/1789785598 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe7e804db10 con 0x7fe7e80cdc30 2023-10-26T01:43:51.158 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:51.157+0000 7fe7de7fc640 1 -- 172.21.15.169:0/1789785598 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v181) v1 ==== 74+0+16633 (secure 0 0 0) 0x7fe7d801d030 con 0x7fe7e80cdc30 2023-10-26T01:43:51.159 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:43:51.159 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":181,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:43:49.710266+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2023-10-26T01:43:47.501296+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"181","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2023-10-26T01:43:48.737507+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"181","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:43:51.162 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:51.159+0000 7fe7f01ce640 1 -- 172.21.15.169:0/1789785598 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe7cc06ae40 msgr2=0x7fe7cc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:51.162 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:51.159+0000 7fe7f01ce640 1 --2- 172.21.15.169:0/1789785598 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe7cc06ae40 0x7fe7cc06d300 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fe7e400a750 tx=0x7fe7e4008040 comp rx=0 tx=0).stop 2023-10-26T01:43:51.162 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:51.159+0000 7fe7f01ce640 1 -- 172.21.15.169:0/1789785598 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7e80cdc30 msgr2=0x7fe7e814ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:51.162 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:51.159+0000 7fe7f01ce640 1 --2- 172.21.15.169:0/1789785598 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7e80cdc30 0x7fe7e814ce00 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7fe7d800c890 tx=0x7fe7d800cd50 comp rx=0 tx=0).stop 2023-10-26T01:43:51.162 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:51.159+0000 7fe7f01ce640 1 -- 172.21.15.169:0/1789785598 shutdown_connections 2023-10-26T01:43:51.163 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:51.159+0000 7fe7f01ce640 1 --2- 172.21.15.169:0/1789785598 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe7cc06ae40 0x7fe7cc06d300 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:51.163 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:51.159+0000 7fe7f01ce640 1 --2- 172.21.15.169:0/1789785598 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7e80cdc30 0x7fe7e814ce00 unknown :-1 s=CLOSED pgs=1834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:51.163 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:51.159+0000 7fe7f01ce640 1 --2- 172.21.15.169:0/1789785598 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe7e81592d0 0x7fe7e80c8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:51.163 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:51.159+0000 7fe7f01ce640 1 --2- 172.21.15.169:0/1789785598 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe7e80ac930 0x7fe7e81e5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:51.163 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:51.159+0000 7fe7f01ce640 1 -- 172.21.15.169:0/1789785598 >> 172.21.15.169:0/1789785598 conn(0x7fe7e8069930 msgr2=0x7fe7e8158140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:51.164 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:51.160+0000 7fe7f01ce640 1 -- 172.21.15.169:0/1789785598 shutdown_connections 2023-10-26T01:43:51.164 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:51.160+0000 7fe7f01ce640 1 -- 172.21.15.169:0/1789785598 wait complete. 2023-10-26T01:43:51.179 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-26T01:43:51.404 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:51.402+0000 7f3bcc026640 1 -- 172.21.15.169:0/2587858363 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3bb00047c0 msgr2=0x7f3bb0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:51.404 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:51.402+0000 7f3bcc026640 1 --2- 172.21.15.169:0/2587858363 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3bb00047c0 0x7f3bb0004c40 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f3bb4006fd0 tx=0x7f3bb00057f0 comp rx=0 tx=0).stop 2023-10-26T01:43:51.405 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:51.402+0000 7f3bca5c3640 1 -- 172.21.15.169:0/2587858363 reap_dead start 2023-10-26T01:43:51.405 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:51.402+0000 7f3bcc026640 1 -- 172.21.15.169:0/2587858363 shutdown_connections 2023-10-26T01:43:51.405 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:51.402+0000 7f3bcc026640 1 -- 172.21.15.169:0/2587858363 >> 172.21.15.169:0/2587858363 conn(0x7f3bc4069930 msgr2=0x7f3bc4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:51.405 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:51.402+0000 7f3bcc026640 1 -- 172.21.15.169:0/2587858363 shutdown_connections 2023-10-26T01:43:51.405 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:51.402+0000 7f3bcc026640 1 -- 172.21.15.169:0/2587858363 wait complete. 2023-10-26T01:43:51.407 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:51.404+0000 7f3bcc026640 1 Processor -- start 2023-10-26T01:43:51.407 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:51.404+0000 7f3bcc026640 1 -- start start 2023-10-26T01:43:51.407 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:51.405+0000 7f3bcc026640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3bb00047c0 0x7f3bc414e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:51.407 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:51.405+0000 7f3bcc026640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3bc40c6660 0x7f3bc40c6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:51.407 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:51.405+0000 7f3bcc026640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3bc415a010 0x7f3bc415a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:51.408 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:51.405+0000 7f3bcc026640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc40c7e50 con 0x7f3bc40c6660 2023-10-26T01:43:51.408 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:51.405+0000 7f3bcc026640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc415a9d0 con 0x7f3bc415a010 2023-10-26T01:43:51.408 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:51.405+0000 7f3bcc026640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc40c71d0 con 0x7f3bb00047c0 2023-10-26T01:43:51.408 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:51.405+0000 7f3bc95c1640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3bc40c6660 0x7f3bc40c6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:51.408 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:51.405+0000 7f3bc95c1640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3bc40c6660 0x7f3bc40c6ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:47914/0 (socket says 172.21.15.169:47914) 2023-10-26T01:43:51.409 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:51.405+0000 7f3bc95c1640 1 -- 172.21.15.169:0/300900986 learned_addr learned my addr 172.21.15.169:0/300900986 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:43:51.410 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:51.406+0000 7f3bc95c1640 1 -- 172.21.15.169:0/300900986 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3bb00047c0 msgr2=0x7f3bc414e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:51.410 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:51.406+0000 7f3bc95c1640 1 --2- 172.21.15.169:0/300900986 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3bb00047c0 0x7f3bc414e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:51.410 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:51.406+0000 7f3bca5c3640 1 --2- 172.21.15.169:0/300900986 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3bc415a010 0x7f3bc415a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:51.411 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:51.406+0000 7f3bc95c1640 1 -- 172.21.15.169:0/300900986 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3bc415a010 msgr2=0x7f3bc415a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:51.411 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:51.406+0000 7f3bc95c1640 1 --2- 172.21.15.169:0/300900986 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3bc415a010 0x7f3bc415a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:51.411 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:51.406+0000 7f3bc95c1640 1 -- 172.21.15.169:0/300900986 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bc40c74b0 con 0x7f3bc40c6660 2023-10-26T01:43:51.411 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:51.406+0000 7f3bc95c1640 1 --2- 172.21.15.169:0/300900986 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3bc40c6660 0x7f3bc40c6ae0 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f3bb402c900 tx=0x7f3bb40015f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:51.411 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:51.406+0000 7f3bba7fc640 1 -- 172.21.15.169:0/300900986 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb4006230 con 0x7f3bc40c6660 2023-10-26T01:43:51.412 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:51.406+0000 7f3bba7fc640 1 -- 172.21.15.169:0/300900986 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3bb4006390 con 0x7f3bc40c6660 2023-10-26T01:43:51.412 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:51.406+0000 7f3bba7fc640 1 -- 172.21.15.169:0/300900986 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb40035f0 con 0x7f3bc40c6660 2023-10-26T01:43:51.412 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:51.406+0000 7f3bcc026640 1 -- 172.21.15.169:0/300900986 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bc40c7790 con 0x7f3bc40c6660 2023-10-26T01:43:51.412 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:51.407+0000 7f3bcc026640 1 -- 172.21.15.169:0/300900986 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bc41596c0 con 0x7f3bc40c6660 2023-10-26T01:43:51.412 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:51.408+0000 7f3bcc026640 1 -- 172.21.15.169:0/300900986 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bc406feb0 con 0x7f3bc40c6660 2023-10-26T01:43:51.413 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:51.409+0000 7f3bba7fc640 1 -- 172.21.15.169:0/300900986 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f3bb4031030 con 0x7f3bc40c6660 2023-10-26T01:43:51.413 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:51.409+0000 7f3bba7fc640 1 --2- 172.21.15.169:0/300900986 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3ba806ae70 0x7f3ba806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:51.414 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:51.410+0000 7f3bba7fc640 1 -- 172.21.15.169:0/300900986 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3bb40aa520 con 0x7f3bc40c6660 2023-10-26T01:43:51.414 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:51.410+0000 7f3bc9dc2640 1 --2- 172.21.15.169:0/300900986 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3ba806ae70 0x7f3ba806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:51.414 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:51.411+0000 7f3bc9dc2640 1 --2- 172.21.15.169:0/300900986 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3ba806ae70 0x7f3ba806d330 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f3bc0005d00 tx=0x7f3bc00094b0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:51.415 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:51.414+0000 7f3bba7fc640 1 -- 172.21.15.169:0/300900986 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3bb40755d0 con 0x7f3bc40c6660 2023-10-26T01:43:51.640 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:51.638+0000 7f3bcc026640 1 -- 172.21.15.169:0/300900986 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f3bc404db10 con 0x7f3bc40c6660 2023-10-26T01:43:52.335 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:52.333+0000 7f3bba7fc640 1 -- 172.21.15.169:0/300900986 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v93) v1 ==== 118+0+0 (secure 0 0 0) 0x7f3bb4078bf0 con 0x7f3bc40c6660 2023-10-26T01:43:52.339 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:52.336+0000 7f3bcc026640 1 -- 172.21.15.169:0/300900986 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3ba806ae70 msgr2=0x7f3ba806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:52.339 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:52.336+0000 7f3bcc026640 1 --2- 172.21.15.169:0/300900986 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3ba806ae70 0x7f3ba806d330 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f3bc0005d00 tx=0x7f3bc00094b0 comp rx=0 tx=0).stop 2023-10-26T01:43:52.339 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:52.336+0000 7f3bcc026640 1 -- 172.21.15.169:0/300900986 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3bc40c6660 msgr2=0x7f3bc40c6ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:52.339 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:52.336+0000 7f3bcc026640 1 --2- 172.21.15.169:0/300900986 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3bc40c6660 0x7f3bc40c6ae0 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f3bb402c900 tx=0x7f3bb40015f0 comp rx=0 tx=0).stop 2023-10-26T01:43:52.339 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:52.336+0000 7f3bcc026640 1 -- 172.21.15.169:0/300900986 shutdown_connections 2023-10-26T01:43:52.340 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:52.336+0000 7f3bcc026640 1 --2- 172.21.15.169:0/300900986 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3ba806ae70 0x7f3ba806d330 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:52.340 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:52.336+0000 7f3bcc026640 1 --2- 172.21.15.169:0/300900986 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3bc40c6660 0x7f3bc40c6ae0 unknown :-1 s=CLOSED pgs=1837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:52.340 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:52.336+0000 7f3bcc026640 1 --2- 172.21.15.169:0/300900986 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3bc415a010 0x7f3bc415a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:52.340 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:52.336+0000 7f3bcc026640 1 --2- 172.21.15.169:0/300900986 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3bb00047c0 0x7f3bc414e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:52.340 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:52.336+0000 7f3bcc026640 1 -- 172.21.15.169:0/300900986 >> 172.21.15.169:0/300900986 conn(0x7f3bc4069930 msgr2=0x7f3bc4150640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:52.341 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:52.337+0000 7f3bcc026640 1 -- 172.21.15.169:0/300900986 shutdown_connections 2023-10-26T01:43:52.341 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:52.337+0000 7f3bcc026640 1 -- 172.21.15.169:0/300900986 wait complete. 2023-10-26T01:43:52.358 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-26T01:43:52.359 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-26T01:43:52.582 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:52.580+0000 7ff2b027c640 1 -- 172.21.15.169:0/530306958 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff294004d00 msgr2=0x7ff294005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:52.583 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:52.580+0000 7ff2b027c640 1 --2- 172.21.15.169:0/530306958 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff294004d00 0x7ff294005180 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7ff2a000c610 tx=0x7ff2940057f0 comp rx=0 tx=0).stop 2023-10-26T01:43:52.583 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:52.580+0000 7ff2ae819640 1 -- 172.21.15.169:0/530306958 reap_dead start 2023-10-26T01:43:52.583 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:52.580+0000 7ff2b027c640 1 -- 172.21.15.169:0/530306958 shutdown_connections 2023-10-26T01:43:52.583 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:52.580+0000 7ff2b027c640 1 -- 172.21.15.169:0/530306958 >> 172.21.15.169:0/530306958 conn(0x7ff2a8069930 msgr2=0x7ff2a8069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:52.584 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:52.580+0000 7ff2b027c640 1 -- 172.21.15.169:0/530306958 shutdown_connections 2023-10-26T01:43:52.584 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:52.581+0000 7ff2b027c640 1 -- 172.21.15.169:0/530306958 wait complete. 2023-10-26T01:43:52.584 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:52.582+0000 7ff2b027c640 1 Processor -- start 2023-10-26T01:43:52.585 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:52.582+0000 7ff2b027c640 1 -- start start 2023-10-26T01:43:52.586 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:52.583+0000 7ff2b027c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff294004d00 0x7ff2a81f1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:52.586 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:52.583+0000 7ff2b027c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff2a815a5e0 0x7ff2a80acf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:52.587 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:52.583+0000 7ff2b027c640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff2a80ad440 0x7ff2a8157dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:52.587 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:52.583+0000 7ff2b027c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a80aff90 con 0x7ff294004d00 2023-10-26T01:43:52.587 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:52.583+0000 7ff2b027c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a80af9d0 con 0x7ff2a815a5e0 2023-10-26T01:43:52.587 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:52.583+0000 7ff2b027c640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a80afcb0 con 0x7ff2a80ad440 2023-10-26T01:43:52.587 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:52.584+0000 7ff2ae018640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff294004d00 0x7ff2a81f1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:52.588 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:52.584+0000 7ff2ae018640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff294004d00 0x7ff2a81f1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:47926/0 (socket says 172.21.15.169:47926) 2023-10-26T01:43:52.588 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:52.584+0000 7ff2ae018640 1 -- 172.21.15.169:0/704860021 learned_addr learned my addr 172.21.15.169:0/704860021 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:43:52.588 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:52.584+0000 7ff2ae819640 1 --2- 172.21.15.169:0/704860021 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff2a80ad440 0x7ff2a8157dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:52.588 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:52.584+0000 7ff2ad817640 1 --2- 172.21.15.169:0/704860021 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff2a815a5e0 0x7ff2a80acf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:52.588 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:52.584+0000 7ff2ae018640 1 -- 172.21.15.169:0/704860021 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff2a80ad440 msgr2=0x7ff2a8157dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:52.589 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:52.584+0000 7ff2ae018640 1 --2- 172.21.15.169:0/704860021 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff2a80ad440 0x7ff2a8157dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:52.589 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:52.584+0000 7ff2ae018640 1 -- 172.21.15.169:0/704860021 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff2a815a5e0 msgr2=0x7ff2a80acf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:52.589 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:52.584+0000 7ff2ae018640 1 --2- 172.21.15.169:0/704860021 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff2a815a5e0 0x7ff2a80acf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:52.589 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:52.584+0000 7ff2ae018640 1 -- 172.21.15.169:0/704860021 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2a8155dc0 con 0x7ff294004d00 2023-10-26T01:43:52.589 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:52.584+0000 7ff2ae018640 1 --2- 172.21.15.169:0/704860021 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff294004d00 0x7ff2a81f1580 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7ff2a400e990 tx=0x7ff2a400ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:52.590 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:52.585+0000 7ff29effd640 1 -- 172.21.15.169:0/704860021 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a400cbe0 con 0x7ff294004d00 2023-10-26T01:43:52.590 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:52.585+0000 7ff29effd640 1 -- 172.21.15.169:0/704860021 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff2a400ce30 con 0x7ff294004d00 2023-10-26T01:43:52.590 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:52.585+0000 7ff29effd640 1 -- 172.21.15.169:0/704860021 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a4005280 con 0x7ff294004d00 2023-10-26T01:43:52.590 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:52.585+0000 7ff2b027c640 1 -- 172.21.15.169:0/704860021 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2a81560a0 con 0x7ff294004d00 2023-10-26T01:43:52.590 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:52.585+0000 7ff2b027c640 1 -- 172.21.15.169:0/704860021 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2a80af610 con 0x7ff294004d00 2023-10-26T01:43:52.591 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:52.586+0000 7ff2b027c640 1 -- 172.21.15.169:0/704860021 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2a804db10 con 0x7ff294004d00 2023-10-26T01:43:52.592 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:52.587+0000 7ff29effd640 1 -- 172.21.15.169:0/704860021 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ff2a40053e0 con 0x7ff294004d00 2023-10-26T01:43:52.592 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:52.588+0000 7ff29effd640 1 --2- 172.21.15.169:0/704860021 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff28c06ae70 0x7ff28c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:52.592 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:52.588+0000 7ff29effd640 1 -- 172.21.15.169:0/704860021 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff2a4089f10 con 0x7ff294004d00 2023-10-26T01:43:52.592 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:52.589+0000 7ff2ad817640 1 --2- 172.21.15.169:0/704860021 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff28c06ae70 0x7ff28c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:52.592 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:52.590+0000 7ff2ad817640 1 --2- 172.21.15.169:0/704860021 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff28c06ae70 0x7ff28c06d330 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7ff298005fd0 tx=0x7ff29800c4e0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:52.594 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:52.593+0000 7ff29effd640 1 -- 172.21.15.169:0/704860021 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff2a4054fb0 con 0x7ff294004d00 2023-10-26T01:43:52.817 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:52.815+0000 7ff2b027c640 1 -- 172.21.15.169:0/704860021 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7ff2a80b2170 con 0x7ff294004d00 2023-10-26T01:43:53.347 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:53.345+0000 7ff29effd640 1 -- 172.21.15.169:0/704860021 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v94) v1 ==== 122+0+0 (secure 0 0 0) 0x7ff2a401f070 con 0x7ff294004d00 2023-10-26T01:43:53.350 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:53.348+0000 7ff2b027c640 1 -- 172.21.15.169:0/704860021 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff28c06ae70 msgr2=0x7ff28c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:53.351 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:53.348+0000 7ff2b027c640 1 --2- 172.21.15.169:0/704860021 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff28c06ae70 0x7ff28c06d330 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7ff298005fd0 tx=0x7ff29800c4e0 comp rx=0 tx=0).stop 2023-10-26T01:43:53.351 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:53.348+0000 7ff2b027c640 1 -- 172.21.15.169:0/704860021 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff294004d00 msgr2=0x7ff2a81f1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:53.351 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:53.348+0000 7ff2b027c640 1 --2- 172.21.15.169:0/704860021 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff294004d00 0x7ff2a81f1580 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7ff2a400e990 tx=0x7ff2a400ee50 comp rx=0 tx=0).stop 2023-10-26T01:43:53.351 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:53.349+0000 7ff2b027c640 1 -- 172.21.15.169:0/704860021 shutdown_connections 2023-10-26T01:43:53.352 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:53.349+0000 7ff2b027c640 1 --2- 172.21.15.169:0/704860021 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff28c06ae70 0x7ff28c06d330 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:53.352 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:53.349+0000 7ff2b027c640 1 --2- 172.21.15.169:0/704860021 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff2a80ad440 0x7ff2a8157dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:53.352 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:53.349+0000 7ff2b027c640 1 --2- 172.21.15.169:0/704860021 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff2a815a5e0 0x7ff2a80acf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:53.352 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:53.349+0000 7ff2b027c640 1 --2- 172.21.15.169:0/704860021 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff294004d00 0x7ff2a81f1580 unknown :-1 s=CLOSED pgs=1840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:53.352 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:53.349+0000 7ff2b027c640 1 -- 172.21.15.169:0/704860021 >> 172.21.15.169:0/704860021 conn(0x7ff2a8069930 msgr2=0x7ff2a806c500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:53.352 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:53.349+0000 7ff2b027c640 1 -- 172.21.15.169:0/704860021 shutdown_connections 2023-10-26T01:43:53.353 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:53.349+0000 7ff2b027c640 1 -- 172.21.15.169:0/704860021 wait complete. 2023-10-26T01:43:53.370 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-26T01:43:53.370 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T01:43:53.589 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:53.587+0000 7f33e3e61640 1 -- 172.21.15.169:0/679420457 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33dc14ece0 msgr2=0x7f33dc0aa1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:53.590 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:53.587+0000 7f33e3e61640 1 --2- 172.21.15.169:0/679420457 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33dc14ece0 0x7f33dc0aa1c0 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f33dc0acde0 tx=0x7f33d802bda0 comp rx=0 tx=0).stop 2023-10-26T01:43:53.590 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:53.587+0000 7f33e3e61640 1 -- 172.21.15.169:0/679420457 shutdown_connections 2023-10-26T01:43:53.591 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:53.587+0000 7f33e3e61640 1 --2- 172.21.15.169:0/679420457 >> v2:172.21.15.170:3300/0 conn(0x7f33dc06c420 0x7f33dc0ab050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:53.591 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:53.587+0000 7f33e3e61640 1 --2- 172.21.15.169:0/679420457 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f33dc06bf70 0x7f33dc0aab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:53.591 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:53.587+0000 7f33e3e61640 1 --2- 172.21.15.169:0/679420457 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33dc14ece0 0x7f33dc0aa1c0 unknown :-1 s=CLOSED pgs=1841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:53.591 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:53.587+0000 7f33e3e61640 1 -- 172.21.15.169:0/679420457 >> 172.21.15.169:0/679420457 conn(0x7f33dc069930 msgr2=0x7f33dc069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:53.592 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:53.587+0000 7f33e3e61640 1 -- 172.21.15.169:0/679420457 shutdown_connections 2023-10-26T01:43:53.592 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:53.587+0000 7f33e3e61640 1 -- 172.21.15.169:0/679420457 wait complete. 2023-10-26T01:43:53.594 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:53.589+0000 7f33e3e61640 1 Processor -- start 2023-10-26T01:43:53.595 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:53.589+0000 7f33e3e61640 1 -- start start 2023-10-26T01:43:53.595 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:53.590+0000 7f33e3e61640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33dc06bf70 0x7f33dc144b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:53.595 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:53.590+0000 7f33e3e61640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33dc06c420 0x7f33dc0b1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:53.595 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:53.590+0000 7f33e3e61640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f33dc14ece0 0x7f33dc147fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:53.596 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:53.590+0000 7f33e3e61640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33dc152d50 con 0x7f33dc06bf70 2023-10-26T01:43:53.596 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:53.590+0000 7f33e3e61640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33dc1464c0 con 0x7f33dc06c420 2023-10-26T01:43:53.596 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:53.590+0000 7f33e3e61640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33dc145f00 con 0x7f33dc14ece0 2023-10-26T01:43:53.596 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:53.591+0000 7f33e1bfd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33dc06bf70 0x7f33dc144b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:53.596 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:53.591+0000 7f33e1bfd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33dc06bf70 0x7f33dc144b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:47932/0 (socket says 172.21.15.169:47932) 2023-10-26T01:43:53.597 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:53.591+0000 7f33e1bfd640 1 -- 172.21.15.169:0/1586791152 learned_addr learned my addr 172.21.15.169:0/1586791152 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:43:53.597 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:53.591+0000 7f33e13fc640 1 --2- 172.21.15.169:0/1586791152 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33dc06c420 0x7f33dc0b1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:53.597 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:53.591+0000 7f33e1bfd640 1 -- 172.21.15.169:0/1586791152 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f33dc14ece0 msgr2=0x7f33dc147fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:53.597 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:53.591+0000 7f33e1bfd640 1 --2- 172.21.15.169:0/1586791152 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f33dc14ece0 0x7f33dc147fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:53.597 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:53.591+0000 7f33e1bfd640 1 -- 172.21.15.169:0/1586791152 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33dc06c420 msgr2=0x7f33dc0b1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:53.598 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:53.591+0000 7f33e1bfd640 1 --2- 172.21.15.169:0/1586791152 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33dc06c420 0x7f33dc0b1dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:53.598 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:53.591+0000 7f33e1bfd640 1 -- 172.21.15.169:0/1586791152 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33d8009560 con 0x7f33dc06bf70 2023-10-26T01:43:53.598 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:53.591+0000 7f33e1bfd640 1 --2- 172.21.15.169:0/1586791152 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33dc06bf70 0x7f33dc144b20 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f33d8036040 tx=0x7f33d802dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:53.599 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:53.592+0000 7f33d27fc640 1 -- 172.21.15.169:0/1586791152 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33d8005650 con 0x7f33dc06bf70 2023-10-26T01:43:53.599 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:53.592+0000 7f33d27fc640 1 -- 172.21.15.169:0/1586791152 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f33d8003930 con 0x7f33dc06bf70 2023-10-26T01:43:53.599 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:53.592+0000 7f33e3e61640 1 -- 172.21.15.169:0/1586791152 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33dc1461e0 con 0x7f33dc06bf70 2023-10-26T01:43:53.599 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:53.592+0000 7f33d27fc640 1 -- 172.21.15.169:0/1586791152 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33d8003d40 con 0x7f33dc06bf70 2023-10-26T01:43:53.599 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:53.592+0000 7f33e3e61640 1 -- 172.21.15.169:0/1586791152 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33dc146890 con 0x7f33dc06bf70 2023-10-26T01:43:53.600 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:53.593+0000 7f33e3e61640 1 -- 172.21.15.169:0/1586791152 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33dc1527d0 con 0x7f33dc06bf70 2023-10-26T01:43:53.601 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:53.594+0000 7f33d27fc640 1 -- 172.21.15.169:0/1586791152 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f33d803e450 con 0x7f33dc06bf70 2023-10-26T01:43:53.601 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:53.595+0000 7f33d27fc640 1 --2- 172.21.15.169:0/1586791152 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f33c006ae40 0x7f33c006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:53.601 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:53.595+0000 7f33d27fc640 1 -- 172.21.15.169:0/1586791152 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f33d80abfc0 con 0x7f33dc06bf70 2023-10-26T01:43:53.601 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:53.595+0000 7f33e13fc640 1 --2- 172.21.15.169:0/1586791152 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f33c006ae40 0x7f33c006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:53.602 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:53.596+0000 7f33e13fc640 1 --2- 172.21.15.169:0/1586791152 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f33c006ae40 0x7f33c006d300 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f33dc15adb0 tx=0x7f33cc005c60 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:53.602 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:53.599+0000 7f33d27fc640 1 -- 172.21.15.169:0/1586791152 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f33dc1527d0 con 0x7f33dc06bf70 2023-10-26T01:43:53.827 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:53.826+0000 7f33e3e61640 1 -- 172.21.15.169:0/1586791152 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f33dc1527d0 con 0x7f33dc06bf70 2023-10-26T01:43:53.829 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:53.827+0000 7f33d27fc640 1 -- 172.21.15.169:0/1586791152 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) v1 ==== 94+0+4920 (secure 0 0 0) 0x7f33dc1527d0 con 0x7f33dc06bf70 2023-10-26T01:43:53.829 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:43:53.829 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":94,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6484,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/195700049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":195700049},{"type":"v1","addr":"172.21.15.169:6835","nonce":195700049}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":89},{"gid":6493,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/1167696300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1167696300},{"type":"v1","addr":"172.21.15.170:6836","nonce":1167696300}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":89},{"gid":9111,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/22729060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":22729060},{"type":"v1","addr":"172.21.15.169:6834","nonce":22729060}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":89}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T01:43:49.711084+0000","modified":"2023-10-26T01:43:52.816785+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9123},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9123":{"gid":9123,"name":"d","rank":0,"incarnation":90,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6837/2205518515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":2205518515},{"type":"v1","addr":"172.21.15.170:6837","nonce":2205518515}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2023-10-26T01:43:53.830 INFO:teuthology.orchestra.run.smithi169.stderr:dumped fsmap epoch 94 2023-10-26T01:43:53.832 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:53.830+0000 7f33e3e61640 1 -- 172.21.15.169:0/1586791152 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f33c006ae40 msgr2=0x7f33c006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:53.832 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:53.830+0000 7f33e3e61640 1 --2- 172.21.15.169:0/1586791152 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f33c006ae40 0x7f33c006d300 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f33dc15adb0 tx=0x7f33cc005c60 comp rx=0 tx=0).stop 2023-10-26T01:43:53.832 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:53.830+0000 7f33e3e61640 1 -- 172.21.15.169:0/1586791152 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33dc06bf70 msgr2=0x7f33dc144b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:53.833 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:53.830+0000 7f33e3e61640 1 --2- 172.21.15.169:0/1586791152 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33dc06bf70 0x7f33dc144b20 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f33d8036040 tx=0x7f33d802dd70 comp rx=0 tx=0).stop 2023-10-26T01:43:53.833 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:53.830+0000 7f33e3e61640 1 -- 172.21.15.169:0/1586791152 shutdown_connections 2023-10-26T01:43:53.833 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:53.830+0000 7f33e3e61640 1 --2- 172.21.15.169:0/1586791152 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f33c006ae40 0x7f33c006d300 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:53.833 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:53.830+0000 7f33e3e61640 1 --2- 172.21.15.169:0/1586791152 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f33dc14ece0 0x7f33dc147fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:53.833 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:53.830+0000 7f33e3e61640 1 --2- 172.21.15.169:0/1586791152 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33dc06c420 0x7f33dc0b1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:53.834 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:53.830+0000 7f33e3e61640 1 --2- 172.21.15.169:0/1586791152 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33dc06bf70 0x7f33dc144b20 unknown :-1 s=CLOSED pgs=1842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:53.834 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:53.830+0000 7f33e3e61640 1 -- 172.21.15.169:0/1586791152 >> 172.21.15.169:0/1586791152 conn(0x7f33dc069930 msgr2=0x7f33dc0cc3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:53.834 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:53.830+0000 7f33e3e61640 1 -- 172.21.15.169:0/1586791152 shutdown_connections 2023-10-26T01:43:53.834 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:53.830+0000 7f33e3e61640 1 -- 172.21.15.169:0/1586791152 wait complete. 2023-10-26T01:43:53.850 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:43:54.071 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:54.069+0000 7f2ca2b4c640 1 -- 172.21.15.169:0/929636402 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c9c15b610 msgr2=0x7f2c9c06ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:54.072 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:54.069+0000 7f2ca2b4c640 1 --2- 172.21.15.169:0/929636402 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c9c15b610 0x7f2c9c06ed30 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c06c960 tx=0x7f2c8402be40 comp rx=0 tx=0).stop 2023-10-26T01:43:54.072 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:54.069+0000 7f2ca2b4c640 1 -- 172.21.15.169:0/929636402 shutdown_connections 2023-10-26T01:43:54.072 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:54.069+0000 7f2ca2b4c640 1 --2- 172.21.15.169:0/929636402 >> v2:172.21.15.170:3300/0 conn(0x7f2c9c0ac4e0 0x7f2c9c0cd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:54.072 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:54.069+0000 7f2ca2b4c640 1 --2- 172.21.15.169:0/929636402 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c9c15b610 0x7f2c9c06ed30 unknown :-1 s=CLOSED pgs=1844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:54.072 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:54.069+0000 7f2ca2b4c640 1 -- 172.21.15.169:0/929636402 >> 172.21.15.169:0/929636402 conn(0x7f2c9c069930 msgr2=0x7f2c9c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:54.073 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:54.069+0000 7f2ca2b4c640 1 -- 172.21.15.169:0/929636402 shutdown_connections 2023-10-26T01:43:54.073 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:54.069+0000 7f2ca2b4c640 1 -- 172.21.15.169:0/929636402 wait complete. 2023-10-26T01:43:54.075 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:54.071+0000 7f2ca2b4c640 1 Processor -- start 2023-10-26T01:43:54.075 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:54.071+0000 7f2ca2b4c640 1 -- start start 2023-10-26T01:43:54.075 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:54.072+0000 7f2ca2b4c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c9c0ac4e0 0x7f2c9c099ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:54.075 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:54.072+0000 7f2ca2b4c640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2c9c0ac940 0x7f2c9c159bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:54.075 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:54.072+0000 7f2ca2b4c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2c9c15b610 0x7f2c9c0c7790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:54.076 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:54.072+0000 7f2ca2b4c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c9c09b140 con 0x7f2c9c0ac4e0 2023-10-26T01:43:54.076 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:54.072+0000 7f2ca2b4c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c9c09a530 con 0x7f2c9c15b610 2023-10-26T01:43:54.076 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:54.072+0000 7f2ca2b4c640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c9c09a810 con 0x7f2c9c0ac940 2023-10-26T01:43:54.076 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:54.072+0000 7f2ca08e8640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c9c0ac4e0 0x7f2c9c099ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:54.076 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:54.072+0000 7f2ca08e8640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c9c0ac4e0 0x7f2c9c099ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:46130/0 (socket says 172.21.15.169:46130) 2023-10-26T01:43:54.077 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:54.072+0000 7f2ca08e8640 1 -- 172.21.15.169:0/1173021686 learned_addr learned my addr 172.21.15.169:0/1173021686 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:43:54.077 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:54.073+0000 7f2ca08e8640 1 -- 172.21.15.169:0/1173021686 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2c9c0ac940 msgr2=0x7f2c9c159bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:54.077 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:54.073+0000 7f2ca08e8640 1 --2- 172.21.15.169:0/1173021686 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2c9c0ac940 0x7f2c9c159bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:54.077 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:54.073+0000 7f2ca10e9640 1 --2- 172.21.15.169:0/1173021686 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2c9c15b610 0x7f2c9c0c7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:54.077 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:54.073+0000 7f2ca08e8640 1 -- 172.21.15.169:0/1173021686 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2c9c15b610 msgr2=0x7f2c9c0c7790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:54.078 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:54.073+0000 7f2ca08e8640 1 --2- 172.21.15.169:0/1173021686 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2c9c15b610 0x7f2c9c0c7790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:54.078 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:54.073+0000 7f2ca08e8640 1 -- 172.21.15.169:0/1173021686 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c84009560 con 0x7f2c9c0ac4e0 2023-10-26T01:43:54.078 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:54.073+0000 7f2ca08e8640 1 --2- 172.21.15.169:0/1173021686 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c9c0ac4e0 0x7f2c9c099ce0 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f2c84009850 tx=0x7f2c8402db70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:54.079 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:54.073+0000 7f2c997fa640 1 -- 172.21.15.169:0/1173021686 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c840038c0 con 0x7f2c9c0ac4e0 2023-10-26T01:43:54.080 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:54.074+0000 7f2c997fa640 1 -- 172.21.15.169:0/1173021686 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2c84003b10 con 0x7f2c9c0ac4e0 2023-10-26T01:43:54.080 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:54.074+0000 7f2c997fa640 1 -- 172.21.15.169:0/1173021686 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c8403e450 con 0x7f2c9c0ac4e0 2023-10-26T01:43:54.080 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:54.074+0000 7f2ca2b4c640 1 -- 172.21.15.169:0/1173021686 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c9c09aaf0 con 0x7f2c9c0ac4e0 2023-10-26T01:43:54.080 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:54.074+0000 7f2ca2b4c640 1 -- 172.21.15.169:0/1173021686 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c9c09a220 con 0x7f2c9c0ac4e0 2023-10-26T01:43:54.080 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:54.075+0000 7f2ca2b4c640 1 -- 172.21.15.169:0/1173021686 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c9c0972d0 con 0x7f2c9c0ac4e0 2023-10-26T01:43:54.081 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:54.076+0000 7f2c997fa640 1 -- 172.21.15.169:0/1173021686 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f2c8403e5b0 con 0x7f2c9c0ac4e0 2023-10-26T01:43:54.081 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:54.077+0000 7f2c997fa640 1 --2- 172.21.15.169:0/1173021686 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2c7c06ae70 0x7f2c7c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:54.081 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:54.077+0000 7f2c997fa640 1 -- 172.21.15.169:0/1173021686 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2c840ac150 con 0x7f2c9c0ac4e0 2023-10-26T01:43:54.081 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:54.077+0000 7f2c9bfff640 1 --2- 172.21.15.169:0/1173021686 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2c7c06ae70 0x7f2c7c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:54.082 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:54.078+0000 7f2c9bfff640 1 --2- 172.21.15.169:0/1173021686 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2c7c06ae70 0x7f2c7c06d330 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c0afcb0 tx=0x7f2c8c009710 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:54.082 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:54.081+0000 7f2c997fa640 1 -- 172.21.15.169:0/1173021686 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2c840771f0 con 0x7f2c9c0ac4e0 2023-10-26T01:43:54.275 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:54.273+0000 7f2ca2b4c640 1 -- 172.21.15.169:0/1173021686 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2c9c04db10 con 0x7f2c9c0ac4e0 2023-10-26T01:43:54.276 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:54.274+0000 7f2c997fa640 1 -- 172.21.15.169:0/1173021686 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v182) v1 ==== 74+0+16620 (secure 0 0 0) 0x7f2c8407a810 con 0x7f2c9c0ac4e0 2023-10-26T01:43:54.277 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:43:54.277 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":182,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:43:51.284286+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2023-10-26T01:43:47.501296+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"181","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2023-10-26T01:43:48.737507+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"182","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:43:54.279 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:54.277+0000 7f2ca2b4c640 1 -- 172.21.15.169:0/1173021686 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2c7c06ae70 msgr2=0x7f2c7c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:54.279 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:54.277+0000 7f2ca2b4c640 1 --2- 172.21.15.169:0/1173021686 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2c7c06ae70 0x7f2c7c06d330 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c0afcb0 tx=0x7f2c8c009710 comp rx=0 tx=0).stop 2023-10-26T01:43:54.279 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:54.277+0000 7f2ca2b4c640 1 -- 172.21.15.169:0/1173021686 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c9c0ac4e0 msgr2=0x7f2c9c099ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:54.279 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:54.277+0000 7f2ca2b4c640 1 --2- 172.21.15.169:0/1173021686 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c9c0ac4e0 0x7f2c9c099ce0 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f2c84009850 tx=0x7f2c8402db70 comp rx=0 tx=0).stop 2023-10-26T01:43:54.280 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:54.277+0000 7f2ca2b4c640 1 -- 172.21.15.169:0/1173021686 shutdown_connections 2023-10-26T01:43:54.280 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:54.277+0000 7f2ca2b4c640 1 --2- 172.21.15.169:0/1173021686 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2c7c06ae70 0x7f2c7c06d330 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:54.280 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:54.277+0000 7f2ca2b4c640 1 --2- 172.21.15.169:0/1173021686 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2c9c15b610 0x7f2c9c0c7790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:54.280 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:54.277+0000 7f2ca2b4c640 1 --2- 172.21.15.169:0/1173021686 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2c9c0ac940 0x7f2c9c159bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:54.280 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:54.277+0000 7f2ca2b4c640 1 --2- 172.21.15.169:0/1173021686 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c9c0ac4e0 0x7f2c9c099ce0 unknown :-1 s=CLOSED pgs=1845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:54.280 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:54.277+0000 7f2ca2b4c640 1 -- 172.21.15.169:0/1173021686 >> 172.21.15.169:0/1173021686 conn(0x7f2c9c069930 msgr2=0x7f2c9c0ae760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:54.281 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:54.277+0000 7f2ca2b4c640 1 -- 172.21.15.169:0/1173021686 shutdown_connections 2023-10-26T01:43:54.281 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:54.277+0000 7f2ca2b4c640 1 -- 172.21.15.169:0/1173021686 wait complete. 2023-10-26T01:43:54.298 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-26T01:43:54.299 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-26T01:43:54.411 INFO:tasks.ceph.mds.d.smithi170.stderr:2023-10-26T01:43:54.409+0000 7fc2f725d640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-26T01:43:54.508 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:54.506+0000 7f294ca40640 1 -- 172.21.15.169:0/25525456 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f294814ece0 msgr2=0x7f29480aa1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:54.508 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:54.506+0000 7f294ca40640 1 --2- 172.21.15.169:0/25525456 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f294814ece0 0x7f29480aa1c0 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7f2938005fd0 tx=0x7f293802be10 comp rx=0 tx=0).stop 2023-10-26T01:43:54.509 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:54.506+0000 7f294ca40640 1 -- 172.21.15.169:0/25525456 shutdown_connections 2023-10-26T01:43:54.509 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:54.506+0000 7f294ca40640 1 --2- 172.21.15.169:0/25525456 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f294806c420 0x7f29480b0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:54.509 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:54.506+0000 7f294ca40640 1 --2- 172.21.15.169:0/25525456 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f294814ece0 0x7f29480aa1c0 unknown :-1 s=CLOSED pgs=1847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:54.509 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:54.506+0000 7f294ca40640 1 -- 172.21.15.169:0/25525456 >> 172.21.15.169:0/25525456 conn(0x7f2948069930 msgr2=0x7f2948069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:54.510 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:54.506+0000 7f294ca40640 1 -- 172.21.15.169:0/25525456 shutdown_connections 2023-10-26T01:43:54.510 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:54.506+0000 7f294ca40640 1 -- 172.21.15.169:0/25525456 wait complete. 2023-10-26T01:43:54.512 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:54.507+0000 7f294ca40640 1 Processor -- start 2023-10-26T01:43:54.512 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:54.508+0000 7f294ca40640 1 -- start start 2023-10-26T01:43:54.512 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:54.508+0000 7f294ca40640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f294806bf70 0x7f2948154b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:54.512 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:54.508+0000 7f294ca40640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f294806c420 0x7f2948155050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:54.513 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:54.508+0000 7f294ca40640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f294814ece0 0x7f2948147e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:54.513 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:54.508+0000 7f294ca40640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2948150c60 con 0x7f294814ece0 2023-10-26T01:43:54.513 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:54.508+0000 7f294ca40640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29481562d0 con 0x7f294806c420 2023-10-26T01:43:54.513 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:54.508+0000 7f294ca40640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29481565b0 con 0x7f294806bf70 2023-10-26T01:43:54.513 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:54.509+0000 7f2946d9d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f294814ece0 0x7f2948147e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:54.514 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:54.509+0000 7f2946d9d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f294814ece0 0x7f2948147e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:46152/0 (socket says 172.21.15.169:46152) 2023-10-26T01:43:54.514 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:54.509+0000 7f2946d9d640 1 -- 172.21.15.169:0/3178710576 learned_addr learned my addr 172.21.15.169:0/3178710576 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:43:54.514 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:54.509+0000 7f294659c640 1 --2- 172.21.15.169:0/3178710576 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f294806bf70 0x7f2948154b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:54.514 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:54.509+0000 7f293ffff640 1 --2- 172.21.15.169:0/3178710576 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f294806c420 0x7f2948155050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:54.515 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:54.509+0000 7f2946d9d640 1 -- 172.21.15.169:0/3178710576 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f294806bf70 msgr2=0x7f2948154b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:54.515 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:54.509+0000 7f2946d9d640 1 --2- 172.21.15.169:0/3178710576 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f294806bf70 0x7f2948154b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:54.515 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:54.509+0000 7f2946d9d640 1 -- 172.21.15.169:0/3178710576 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f294806c420 msgr2=0x7f2948155050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:54.515 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:54.509+0000 7f2946d9d640 1 --2- 172.21.15.169:0/3178710576 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f294806c420 0x7f2948155050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:54.515 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:54.509+0000 7f2946d9d640 1 -- 172.21.15.169:0/3178710576 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2938009560 con 0x7f294814ece0 2023-10-26T01:43:54.516 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:54.510+0000 7f2946d9d640 1 --2- 172.21.15.169:0/3178710576 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f294814ece0 0x7f2948147e70 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f294000eaa0 tx=0x7f294000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:54.516 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:54.510+0000 7f293effd640 1 -- 172.21.15.169:0/3178710576 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f294000cdb0 con 0x7f294814ece0 2023-10-26T01:43:54.516 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:54.510+0000 7f293effd640 1 -- 172.21.15.169:0/3178710576 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2940004b20 con 0x7f294814ece0 2023-10-26T01:43:54.516 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:54.510+0000 7f294ca40640 1 -- 172.21.15.169:0/3178710576 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2948150eb0 con 0x7f294814ece0 2023-10-26T01:43:54.516 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:54.510+0000 7f293effd640 1 -- 172.21.15.169:0/3178710576 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2940005020 con 0x7f294814ece0 2023-10-26T01:43:54.517 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:54.510+0000 7f294ca40640 1 -- 172.21.15.169:0/3178710576 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2948151470 con 0x7f294814ece0 2023-10-26T01:43:54.518 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:54.511+0000 7f294ca40640 1 -- 172.21.15.169:0/3178710576 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2948151690 con 0x7f294814ece0 2023-10-26T01:43:54.518 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:54.512+0000 7f293effd640 1 -- 172.21.15.169:0/3178710576 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f2940005180 con 0x7f294814ece0 2023-10-26T01:43:54.518 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:54.513+0000 7f293effd640 1 --2- 172.21.15.169:0/3178710576 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f292806ae40 0x7f292806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:54.518 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:54.513+0000 7f293effd640 1 -- 172.21.15.169:0/3178710576 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f294008a080 con 0x7f294814ece0 2023-10-26T01:43:54.519 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:54.514+0000 7f294659c640 1 --2- 172.21.15.169:0/3178710576 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f292806ae40 0x7f292806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:54.519 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:54.515+0000 7f294659c640 1 --2- 172.21.15.169:0/3178710576 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f292806ae40 0x7f292806d300 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f2938035000 tx=0x7f2938004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:54.519 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:54.518+0000 7f293effd640 1 -- 172.21.15.169:0/3178710576 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f29400bcca0 con 0x7f294814ece0 2023-10-26T01:43:54.702 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:54.701+0000 7f294ca40640 1 -- 172.21.15.169:0/3178710576 --> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f29480ab220 con 0x7f292806ae40 2023-10-26T01:43:54.726 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:54.725+0000 7f293effd640 1 -- 172.21.15.169:0/3178710576 <== mgr.4112 v2:172.21.15.170:6832/1187156918 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+282400 (secure 0 0 0) 0x7f29380042f0 con 0x7f292806ae40 2023-10-26T01:43:54.729 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:43:54.732 INFO:teuthology.orchestra.run.smithi169.stderr:dumped all 2023-10-26T01:43:54.734 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:54.732+0000 7f294ca40640 1 -- 172.21.15.169:0/3178710576 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f292806ae40 msgr2=0x7f292806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:54.734 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:54.732+0000 7f294ca40640 1 --2- 172.21.15.169:0/3178710576 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f292806ae40 0x7f292806d300 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f2938035000 tx=0x7f2938004730 comp rx=0 tx=0).stop 2023-10-26T01:43:54.734 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:54.732+0000 7f294ca40640 1 -- 172.21.15.169:0/3178710576 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f294814ece0 msgr2=0x7f2948147e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:54.735 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:54.732+0000 7f294ca40640 1 --2- 172.21.15.169:0/3178710576 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f294814ece0 0x7f2948147e70 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f294000eaa0 tx=0x7f294000ef60 comp rx=0 tx=0).stop 2023-10-26T01:43:54.735 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:54.732+0000 7f294ca40640 1 -- 172.21.15.169:0/3178710576 shutdown_connections 2023-10-26T01:43:54.735 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:54.732+0000 7f294ca40640 1 --2- 172.21.15.169:0/3178710576 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f292806ae40 0x7f292806d300 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:54.735 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:54.732+0000 7f294ca40640 1 --2- 172.21.15.169:0/3178710576 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f294814ece0 0x7f2948147e70 unknown :-1 s=CLOSED pgs=1848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:54.736 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:54.732+0000 7f294ca40640 1 --2- 172.21.15.169:0/3178710576 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f294806c420 0x7f2948155050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:54.736 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:54.732+0000 7f294ca40640 1 --2- 172.21.15.169:0/3178710576 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f294806bf70 0x7f2948154b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:54.736 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:54.732+0000 7f294ca40640 1 -- 172.21.15.169:0/3178710576 >> 172.21.15.169:0/3178710576 conn(0x7f2948069930 msgr2=0x7f2948070f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:54.736 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:54.732+0000 7f294ca40640 1 -- 172.21.15.169:0/3178710576 shutdown_connections 2023-10-26T01:43:54.736 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:54.732+0000 7f294ca40640 1 -- 172.21.15.169:0/3178710576 wait complete. 2023-10-26T01:43:54.754 INFO:teuthology.orchestra.run.smithi169.stdout:{"pg_ready":true,"pg_map":{"version":577,"stamp":"2023-10-26T01:43:54.420532+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":78,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":9086,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1553413,"statfs":{"total":1600000000,"available":1590695388,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002337"},"pg_stats":[{"pgid":"23.3f","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.308770+0000","last_change":"2023-10-26T01:43:50.308500+0000","last_active":"2023-10-26T01:43:50.308770+0000","last_peered":"2023-10-26T01:43:50.308770+0000","last_clean":"2023-10-26T01:43:50.308770+0000","last_became_active":"2023-10-26T01:43:50.308142+0000","last_became_peered":"2023-10-26T01:43:50.308142+0000","last_unstale":"2023-10-26T01:43:50.308770+0000","last_undegraded":"2023-10-26T01:43:50.308770+0000","last_fullsized":"2023-10-26T01:43:50.308770+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:30:34.655863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.3e","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.311684+0000","last_change":"2023-10-26T01:43:50.310257+0000","last_active":"2023-10-26T01:43:50.311684+0000","last_peered":"2023-10-26T01:43:50.311684+0000","last_clean":"2023-10-26T01:43:50.311684+0000","last_became_active":"2023-10-26T01:43:50.309527+0000","last_became_peered":"2023-10-26T01:43:50.309527+0000","last_unstale":"2023-10-26T01:43:50.311684+0000","last_undegraded":"2023-10-26T01:43:50.311684+0000","last_fullsized":"2023-10-26T01:43:50.311684+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:37:58.399376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3d","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.309657+0000","last_change":"2023-10-26T01:43:50.308312+0000","last_active":"2023-10-26T01:43:50.309657+0000","last_peered":"2023-10-26T01:43:50.309657+0000","last_clean":"2023-10-26T01:43:50.309657+0000","last_became_active":"2023-10-26T01:43:50.307837+0000","last_became_peered":"2023-10-26T01:43:50.307837+0000","last_unstale":"2023-10-26T01:43:50.309657+0000","last_undegraded":"2023-10-26T01:43:50.309657+0000","last_fullsized":"2023-10-26T01:43:50.309657+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:07:17.983568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.3c","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.322982+0000","last_change":"2023-10-26T01:43:50.322354+0000","last_active":"2023-10-26T01:43:50.322982+0000","last_peered":"2023-10-26T01:43:50.322982+0000","last_clean":"2023-10-26T01:43:50.322982+0000","last_became_active":"2023-10-26T01:43:50.322091+0000","last_became_peered":"2023-10-26T01:43:50.322091+0000","last_unstale":"2023-10-26T01:43:50.322982+0000","last_undegraded":"2023-10-26T01:43:50.322982+0000","last_fullsized":"2023-10-26T01:43:50.322982+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:35:41.989570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3b","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.319844+0000","last_change":"2023-10-26T01:43:50.319762+0000","last_active":"2023-10-26T01:43:50.319844+0000","last_peered":"2023-10-26T01:43:50.319844+0000","last_clean":"2023-10-26T01:43:50.319844+0000","last_became_active":"2023-10-26T01:43:50.318893+0000","last_became_peered":"2023-10-26T01:43:50.318893+0000","last_unstale":"2023-10-26T01:43:50.319844+0000","last_undegraded":"2023-10-26T01:43:50.319844+0000","last_fullsized":"2023-10-26T01:43:50.319844+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:05:12.398879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3a","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.318840+0000","last_change":"2023-10-26T01:43:50.318313+0000","last_active":"2023-10-26T01:43:50.318840+0000","last_peered":"2023-10-26T01:43:50.318840+0000","last_clean":"2023-10-26T01:43:50.318840+0000","last_became_active":"2023-10-26T01:43:50.317326+0000","last_became_peered":"2023-10-26T01:43:50.317326+0000","last_unstale":"2023-10-26T01:43:50.318840+0000","last_undegraded":"2023-10-26T01:43:50.318840+0000","last_fullsized":"2023-10-26T01:43:50.318840+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:30:41.575163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.39","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.321750+0000","last_change":"2023-10-26T01:43:50.321636+0000","last_active":"2023-10-26T01:43:50.321750+0000","last_peered":"2023-10-26T01:43:50.321750+0000","last_clean":"2023-10-26T01:43:50.321750+0000","last_became_active":"2023-10-26T01:43:50.321121+0000","last_became_peered":"2023-10-26T01:43:50.321121+0000","last_unstale":"2023-10-26T01:43:50.321750+0000","last_undegraded":"2023-10-26T01:43:50.321750+0000","last_fullsized":"2023-10-26T01:43:50.321750+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T01:56:12.485221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.38","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.311762+0000","last_change":"2023-10-26T01:43:50.311066+0000","last_active":"2023-10-26T01:43:50.311762+0000","last_peered":"2023-10-26T01:43:50.311762+0000","last_clean":"2023-10-26T01:43:50.311762+0000","last_became_active":"2023-10-26T01:43:50.310772+0000","last_became_peered":"2023-10-26T01:43:50.310772+0000","last_unstale":"2023-10-26T01:43:50.311762+0000","last_undegraded":"2023-10-26T01:43:50.311762+0000","last_fullsized":"2023-10-26T01:43:50.311762+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:46:02.839349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.37","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.317642+0000","last_change":"2023-10-26T01:43:50.314888+0000","last_active":"2023-10-26T01:43:50.317642+0000","last_peered":"2023-10-26T01:43:50.317642+0000","last_clean":"2023-10-26T01:43:50.317642+0000","last_became_active":"2023-10-26T01:43:50.314609+0000","last_became_peered":"2023-10-26T01:43:50.314609+0000","last_unstale":"2023-10-26T01:43:50.317642+0000","last_undegraded":"2023-10-26T01:43:50.317642+0000","last_fullsized":"2023-10-26T01:43:50.317642+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:31:00.739794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.36","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.317684+0000","last_change":"2023-10-26T01:43:50.316577+0000","last_active":"2023-10-26T01:43:50.317684+0000","last_peered":"2023-10-26T01:43:50.317684+0000","last_clean":"2023-10-26T01:43:50.317684+0000","last_became_active":"2023-10-26T01:43:50.314127+0000","last_became_peered":"2023-10-26T01:43:50.314127+0000","last_unstale":"2023-10-26T01:43:50.317684+0000","last_undegraded":"2023-10-26T01:43:50.317684+0000","last_fullsized":"2023-10-26T01:43:50.317684+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:20:13.363860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.35","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.320405+0000","last_change":"2023-10-26T01:43:50.320327+0000","last_active":"2023-10-26T01:43:50.320405+0000","last_peered":"2023-10-26T01:43:50.320405+0000","last_clean":"2023-10-26T01:43:50.320405+0000","last_became_active":"2023-10-26T01:43:50.320045+0000","last_became_peered":"2023-10-26T01:43:50.320045+0000","last_unstale":"2023-10-26T01:43:50.320405+0000","last_undegraded":"2023-10-26T01:43:50.320405+0000","last_fullsized":"2023-10-26T01:43:50.320405+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:45:38.848472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.34","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.311707+0000","last_change":"2023-10-26T01:43:50.310463+0000","last_active":"2023-10-26T01:43:50.311707+0000","last_peered":"2023-10-26T01:43:50.311707+0000","last_clean":"2023-10-26T01:43:50.311707+0000","last_became_active":"2023-10-26T01:43:50.309959+0000","last_became_peered":"2023-10-26T01:43:50.309959+0000","last_unstale":"2023-10-26T01:43:50.311707+0000","last_undegraded":"2023-10-26T01:43:50.311707+0000","last_fullsized":"2023-10-26T01:43:50.311707+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:14:09.503559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.33","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.314172+0000","last_change":"2023-10-26T01:43:50.311920+0000","last_active":"2023-10-26T01:43:50.314172+0000","last_peered":"2023-10-26T01:43:50.314172+0000","last_clean":"2023-10-26T01:43:50.314172+0000","last_became_active":"2023-10-26T01:43:50.311207+0000","last_became_peered":"2023-10-26T01:43:50.311207+0000","last_unstale":"2023-10-26T01:43:50.314172+0000","last_undegraded":"2023-10-26T01:43:50.314172+0000","last_fullsized":"2023-10-26T01:43:50.314172+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:05:49.047185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.32","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.314157+0000","last_change":"2023-10-26T01:43:50.311679+0000","last_active":"2023-10-26T01:43:50.314157+0000","last_peered":"2023-10-26T01:43:50.314157+0000","last_clean":"2023-10-26T01:43:50.314157+0000","last_became_active":"2023-10-26T01:43:50.311405+0000","last_became_peered":"2023-10-26T01:43:50.311405+0000","last_unstale":"2023-10-26T01:43:50.314157+0000","last_undegraded":"2023-10-26T01:43:50.314157+0000","last_fullsized":"2023-10-26T01:43:50.314157+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:24:24.024882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.31","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.312073+0000","last_change":"2023-10-26T01:43:50.311517+0000","last_active":"2023-10-26T01:43:50.312073+0000","last_peered":"2023-10-26T01:43:50.312073+0000","last_clean":"2023-10-26T01:43:50.312073+0000","last_became_active":"2023-10-26T01:43:50.310199+0000","last_became_peered":"2023-10-26T01:43:50.310199+0000","last_unstale":"2023-10-26T01:43:50.312073+0000","last_undegraded":"2023-10-26T01:43:50.312073+0000","last_fullsized":"2023-10-26T01:43:50.312073+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:12:29.214521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.30","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.312908+0000","last_change":"2023-10-26T01:43:50.312478+0000","last_active":"2023-10-26T01:43:50.312908+0000","last_peered":"2023-10-26T01:43:50.312908+0000","last_clean":"2023-10-26T01:43:50.312908+0000","last_became_active":"2023-10-26T01:43:50.312231+0000","last_became_peered":"2023-10-26T01:43:50.312231+0000","last_unstale":"2023-10-26T01:43:50.312908+0000","last_undegraded":"2023-10-26T01:43:50.312908+0000","last_fullsized":"2023-10-26T01:43:50.312908+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:22:45.891664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.2f","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.317854+0000","last_change":"2023-10-26T01:43:50.317655+0000","last_active":"2023-10-26T01:43:50.317854+0000","last_peered":"2023-10-26T01:43:50.317854+0000","last_clean":"2023-10-26T01:43:50.317854+0000","last_became_active":"2023-10-26T01:43:50.317043+0000","last_became_peered":"2023-10-26T01:43:50.317043+0000","last_unstale":"2023-10-26T01:43:50.317854+0000","last_undegraded":"2023-10-26T01:43:50.317854+0000","last_fullsized":"2023-10-26T01:43:50.317854+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:17:55.522661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2e","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.322382+0000","last_change":"2023-10-26T01:43:50.322147+0000","last_active":"2023-10-26T01:43:50.322382+0000","last_peered":"2023-10-26T01:43:50.322382+0000","last_clean":"2023-10-26T01:43:50.322382+0000","last_became_active":"2023-10-26T01:43:50.321724+0000","last_became_peered":"2023-10-26T01:43:50.321724+0000","last_unstale":"2023-10-26T01:43:50.322382+0000","last_undegraded":"2023-10-26T01:43:50.322382+0000","last_fullsized":"2023-10-26T01:43:50.322382+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:22:26.970184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2d","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.319888+0000","last_change":"2023-10-26T01:43:50.319709+0000","last_active":"2023-10-26T01:43:50.319888+0000","last_peered":"2023-10-26T01:43:50.319888+0000","last_clean":"2023-10-26T01:43:50.319888+0000","last_became_active":"2023-10-26T01:43:50.319444+0000","last_became_peered":"2023-10-26T01:43:50.319444+0000","last_unstale":"2023-10-26T01:43:50.319888+0000","last_undegraded":"2023-10-26T01:43:50.319888+0000","last_fullsized":"2023-10-26T01:43:50.319888+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:07:17.983568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2c","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.316685+0000","last_change":"2023-10-26T01:43:50.316602+0000","last_active":"2023-10-26T01:43:50.316685+0000","last_peered":"2023-10-26T01:43:50.316685+0000","last_clean":"2023-10-26T01:43:50.316685+0000","last_became_active":"2023-10-26T01:43:50.315889+0000","last_became_peered":"2023-10-26T01:43:50.315889+0000","last_unstale":"2023-10-26T01:43:50.316685+0000","last_undegraded":"2023-10-26T01:43:50.316685+0000","last_fullsized":"2023-10-26T01:43:50.316685+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:10:53.556092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.2b","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.317832+0000","last_change":"2023-10-26T01:43:50.317596+0000","last_active":"2023-10-26T01:43:50.317832+0000","last_peered":"2023-10-26T01:43:50.317832+0000","last_clean":"2023-10-26T01:43:50.317832+0000","last_became_active":"2023-10-26T01:43:50.316967+0000","last_became_peered":"2023-10-26T01:43:50.316967+0000","last_unstale":"2023-10-26T01:43:50.317832+0000","last_undegraded":"2023-10-26T01:43:50.317832+0000","last_fullsized":"2023-10-26T01:43:50.317832+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:24:03.603238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2a","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.319668+0000","last_change":"2023-10-26T01:43:50.319586+0000","last_active":"2023-10-26T01:43:50.319668+0000","last_peered":"2023-10-26T01:43:50.319668+0000","last_clean":"2023-10-26T01:43:50.319668+0000","last_became_active":"2023-10-26T01:43:50.319108+0000","last_became_peered":"2023-10-26T01:43:50.319108+0000","last_unstale":"2023-10-26T01:43:50.319668+0000","last_undegraded":"2023-10-26T01:43:50.319668+0000","last_fullsized":"2023-10-26T01:43:50.319668+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:39:02.127957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.29","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.316493+0000","last_change":"2023-10-26T01:43:50.316179+0000","last_active":"2023-10-26T01:43:50.316493+0000","last_peered":"2023-10-26T01:43:50.316493+0000","last_clean":"2023-10-26T01:43:50.316493+0000","last_became_active":"2023-10-26T01:43:50.315941+0000","last_became_peered":"2023-10-26T01:43:50.315941+0000","last_unstale":"2023-10-26T01:43:50.316493+0000","last_undegraded":"2023-10-26T01:43:50.316493+0000","last_fullsized":"2023-10-26T01:43:50.316493+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:46:23.366905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.28","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.309174+0000","last_change":"2023-10-26T01:43:50.309071+0000","last_active":"2023-10-26T01:43:50.309174+0000","last_peered":"2023-10-26T01:43:50.309174+0000","last_clean":"2023-10-26T01:43:50.309174+0000","last_became_active":"2023-10-26T01:43:50.308627+0000","last_became_peered":"2023-10-26T01:43:50.308627+0000","last_unstale":"2023-10-26T01:43:50.309174+0000","last_undegraded":"2023-10-26T01:43:50.309174+0000","last_fullsized":"2023-10-26T01:43:50.309174+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:40:14.136503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.27","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.317024+0000","last_change":"2023-10-26T01:43:50.316772+0000","last_active":"2023-10-26T01:43:50.317024+0000","last_peered":"2023-10-26T01:43:50.317024+0000","last_clean":"2023-10-26T01:43:50.317024+0000","last_became_active":"2023-10-26T01:43:50.316273+0000","last_became_peered":"2023-10-26T01:43:50.316273+0000","last_unstale":"2023-10-26T01:43:50.317024+0000","last_undegraded":"2023-10-26T01:43:50.317024+0000","last_fullsized":"2023-10-26T01:43:50.317024+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:44:08.964253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.26","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.314452+0000","last_change":"2023-10-26T01:43:50.314358+0000","last_active":"2023-10-26T01:43:50.314452+0000","last_peered":"2023-10-26T01:43:50.314452+0000","last_clean":"2023-10-26T01:43:50.314452+0000","last_became_active":"2023-10-26T01:43:50.314040+0000","last_became_peered":"2023-10-26T01:43:50.314040+0000","last_unstale":"2023-10-26T01:43:50.314452+0000","last_undegraded":"2023-10-26T01:43:50.314452+0000","last_fullsized":"2023-10-26T01:43:50.314452+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:26:32.296617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.25","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.314139+0000","last_change":"2023-10-26T01:43:50.310537+0000","last_active":"2023-10-26T01:43:50.314139+0000","last_peered":"2023-10-26T01:43:50.314139+0000","last_clean":"2023-10-26T01:43:50.314139+0000","last_became_active":"2023-10-26T01:43:50.310244+0000","last_became_peered":"2023-10-26T01:43:50.310244+0000","last_unstale":"2023-10-26T01:43:50.314139+0000","last_undegraded":"2023-10-26T01:43:50.314139+0000","last_fullsized":"2023-10-26T01:43:50.314139+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:22:13.378905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.24","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.312298+0000","last_change":"2023-10-26T01:43:50.312178+0000","last_active":"2023-10-26T01:43:50.312298+0000","last_peered":"2023-10-26T01:43:50.312298+0000","last_clean":"2023-10-26T01:43:50.312298+0000","last_became_active":"2023-10-26T01:43:50.310962+0000","last_became_peered":"2023-10-26T01:43:50.310962+0000","last_unstale":"2023-10-26T01:43:50.312298+0000","last_undegraded":"2023-10-26T01:43:50.312298+0000","last_fullsized":"2023-10-26T01:43:50.312298+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:27:24.942966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.23","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.309679+0000","last_change":"2023-10-26T01:43:50.308620+0000","last_active":"2023-10-26T01:43:50.309679+0000","last_peered":"2023-10-26T01:43:50.309679+0000","last_clean":"2023-10-26T01:43:50.309679+0000","last_became_active":"2023-10-26T01:43:50.307934+0000","last_became_peered":"2023-10-26T01:43:50.307934+0000","last_unstale":"2023-10-26T01:43:50.309679+0000","last_undegraded":"2023-10-26T01:43:50.309679+0000","last_fullsized":"2023-10-26T01:43:50.309679+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:59:34.197770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.22","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.295358+0000","last_change":"2023-10-26T01:43:50.295164+0000","last_active":"2023-10-26T01:43:50.295358+0000","last_peered":"2023-10-26T01:43:50.295358+0000","last_clean":"2023-10-26T01:43:50.295358+0000","last_became_active":"2023-10-26T01:43:50.293769+0000","last_became_peered":"2023-10-26T01:43:50.293769+0000","last_unstale":"2023-10-26T01:43:50.295358+0000","last_undegraded":"2023-10-26T01:43:50.295358+0000","last_fullsized":"2023-10-26T01:43:50.295358+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:28:36.730001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.21","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.317881+0000","last_change":"2023-10-26T01:43:50.317761+0000","last_active":"2023-10-26T01:43:50.317881+0000","last_peered":"2023-10-26T01:43:50.317881+0000","last_clean":"2023-10-26T01:43:50.317881+0000","last_became_active":"2023-10-26T01:43:50.316880+0000","last_became_peered":"2023-10-26T01:43:50.316880+0000","last_unstale":"2023-10-26T01:43:50.317881+0000","last_undegraded":"2023-10-26T01:43:50.317881+0000","last_fullsized":"2023-10-26T01:43:50.317881+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T01:56:12.485221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.20","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.321512+0000","last_change":"2023-10-26T01:43:50.319722+0000","last_active":"2023-10-26T01:43:50.321512+0000","last_peered":"2023-10-26T01:43:50.321512+0000","last_clean":"2023-10-26T01:43:50.321512+0000","last_became_active":"2023-10-26T01:43:50.319490+0000","last_became_peered":"2023-10-26T01:43:50.319490+0000","last_unstale":"2023-10-26T01:43:50.321512+0000","last_undegraded":"2023-10-26T01:43:50.321512+0000","last_fullsized":"2023-10-26T01:43:50.321512+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:05:12.398879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.314889+0000","last_change":"2023-10-26T01:43:50.314795+0000","last_active":"2023-10-26T01:43:50.314889+0000","last_peered":"2023-10-26T01:43:50.314889+0000","last_clean":"2023-10-26T01:43:50.314889+0000","last_became_active":"2023-10-26T01:43:50.314264+0000","last_became_peered":"2023-10-26T01:43:50.314264+0000","last_unstale":"2023-10-26T01:43:50.314889+0000","last_undegraded":"2023-10-26T01:43:50.314889+0000","last_fullsized":"2023-10-26T01:43:50.314889+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:28:36.730001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1f","version":"181'4","reported_seq":15,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.328923+0000","last_change":"2023-10-26T01:43:49.263896+0000","last_active":"2023-10-26T01:43:50.328923+0000","last_peered":"2023-10-26T01:43:50.328923+0000","last_clean":"2023-10-26T01:43:50.328923+0000","last_became_active":"2023-10-26T01:43:49.263284+0000","last_became_peered":"2023-10-26T01:43:49.263284+0000","last_unstale":"2023-10-26T01:43:50.328923+0000","last_undegraded":"2023-10-26T01:43:50.328923+0000","last_fullsized":"2023-10-26T01:43:50.328923+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_clean_scrub_stamp":"2023-10-26T01:43:48.232806+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:06:13.311992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.297013+0000","last_change":"2023-10-26T01:43:50.296698+0000","last_active":"2023-10-26T01:43:50.297013+0000","last_peered":"2023-10-26T01:43:50.297013+0000","last_clean":"2023-10-26T01:43:50.297013+0000","last_became_active":"2023-10-26T01:43:50.296065+0000","last_became_peered":"2023-10-26T01:43:50.296065+0000","last_unstale":"2023-10-26T01:43:50.297013+0000","last_undegraded":"2023-10-26T01:43:50.297013+0000","last_fullsized":"2023-10-26T01:43:50.297013+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:22:45.891664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1e","version":"181'1","reported_seq":12,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.329965+0000","last_change":"2023-10-26T01:43:49.275818+0000","last_active":"2023-10-26T01:43:50.329965+0000","last_peered":"2023-10-26T01:43:50.329965+0000","last_clean":"2023-10-26T01:43:50.329965+0000","last_became_active":"2023-10-26T01:43:49.275556+0000","last_became_peered":"2023-10-26T01:43:49.275556+0000","last_unstale":"2023-10-26T01:43:50.329965+0000","last_undegraded":"2023-10-26T01:43:50.329965+0000","last_fullsized":"2023-10-26T01:43:50.329965+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_clean_scrub_stamp":"2023-10-26T01:43:48.232806+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:08:08.571789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.318863+0000","last_change":"2023-10-26T01:43:50.318624+0000","last_active":"2023-10-26T01:43:50.318863+0000","last_peered":"2023-10-26T01:43:50.318863+0000","last_clean":"2023-10-26T01:43:50.318863+0000","last_became_active":"2023-10-26T01:43:50.317875+0000","last_became_peered":"2023-10-26T01:43:50.317875+0000","last_unstale":"2023-10-26T01:43:50.318863+0000","last_undegraded":"2023-10-26T01:43:50.318863+0000","last_fullsized":"2023-10-26T01:43:50.318863+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:11:45.355135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1d","version":"181'2","reported_seq":13,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.329000+0000","last_change":"2023-10-26T01:43:49.271734+0000","last_active":"2023-10-26T01:43:50.329000+0000","last_peered":"2023-10-26T01:43:50.329000+0000","last_clean":"2023-10-26T01:43:50.329000+0000","last_became_active":"2023-10-26T01:43:49.271511+0000","last_became_peered":"2023-10-26T01:43:49.271511+0000","last_unstale":"2023-10-26T01:43:50.329000+0000","last_undegraded":"2023-10-26T01:43:50.329000+0000","last_fullsized":"2023-10-26T01:43:50.329000+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_clean_scrub_stamp":"2023-10-26T01:43:48.232806+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:28:47.267845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.320688+0000","last_change":"2023-10-26T01:43:50.320568+0000","last_active":"2023-10-26T01:43:50.320688+0000","last_peered":"2023-10-26T01:43:50.320688+0000","last_clean":"2023-10-26T01:43:50.320688+0000","last_became_active":"2023-10-26T01:43:50.320115+0000","last_became_peered":"2023-10-26T01:43:50.320115+0000","last_unstale":"2023-10-26T01:43:50.320688+0000","last_undegraded":"2023-10-26T01:43:50.320688+0000","last_fullsized":"2023-10-26T01:43:50.320688+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:46:23.366905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1c","version":"181'2","reported_seq":13,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.329510+0000","last_change":"2023-10-26T01:43:49.271198+0000","last_active":"2023-10-26T01:43:50.329510+0000","last_peered":"2023-10-26T01:43:50.329510+0000","last_clean":"2023-10-26T01:43:50.329510+0000","last_became_active":"2023-10-26T01:43:49.270976+0000","last_became_peered":"2023-10-26T01:43:49.270976+0000","last_unstale":"2023-10-26T01:43:50.329510+0000","last_undegraded":"2023-10-26T01:43:50.329510+0000","last_fullsized":"2023-10-26T01:43:50.329510+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_clean_scrub_stamp":"2023-10-26T01:43:48.232806+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:34:23.052043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.311863+0000","last_change":"2023-10-26T01:43:50.309347+0000","last_active":"2023-10-26T01:43:50.311863+0000","last_peered":"2023-10-26T01:43:50.311863+0000","last_clean":"2023-10-26T01:43:50.311863+0000","last_became_active":"2023-10-26T01:43:50.308714+0000","last_became_peered":"2023-10-26T01:43:50.308714+0000","last_unstale":"2023-10-26T01:43:50.311863+0000","last_undegraded":"2023-10-26T01:43:50.311863+0000","last_fullsized":"2023-10-26T01:43:50.311863+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:18:13.356291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1b","version":"181'1","reported_seq":12,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.330571+0000","last_change":"2023-10-26T01:43:49.268691+0000","last_active":"2023-10-26T01:43:50.330571+0000","last_peered":"2023-10-26T01:43:50.330571+0000","last_clean":"2023-10-26T01:43:50.330571+0000","last_became_active":"2023-10-26T01:43:49.268207+0000","last_became_peered":"2023-10-26T01:43:49.268207+0000","last_unstale":"2023-10-26T01:43:50.330571+0000","last_undegraded":"2023-10-26T01:43:50.330571+0000","last_fullsized":"2023-10-26T01:43:50.330571+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_clean_scrub_stamp":"2023-10-26T01:43:48.232806+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:30:40.570275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.314094+0000","last_change":"2023-10-26T01:43:50.313380+0000","last_active":"2023-10-26T01:43:50.314094+0000","last_peered":"2023-10-26T01:43:50.314094+0000","last_clean":"2023-10-26T01:43:50.314094+0000","last_became_active":"2023-10-26T01:43:50.312423+0000","last_became_peered":"2023-10-26T01:43:50.312423+0000","last_unstale":"2023-10-26T01:43:50.314094+0000","last_undegraded":"2023-10-26T01:43:50.314094+0000","last_fullsized":"2023-10-26T01:43:50.314094+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:16:16.465109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.1a","version":"181'1","reported_seq":12,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.329119+0000","last_change":"2023-10-26T01:43:49.273232+0000","last_active":"2023-10-26T01:43:50.329119+0000","last_peered":"2023-10-26T01:43:50.329119+0000","last_clean":"2023-10-26T01:43:50.329119+0000","last_became_active":"2023-10-26T01:43:49.272568+0000","last_became_peered":"2023-10-26T01:43:49.272568+0000","last_unstale":"2023-10-26T01:43:50.329119+0000","last_undegraded":"2023-10-26T01:43:50.329119+0000","last_fullsized":"2023-10-26T01:43:50.329119+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_clean_scrub_stamp":"2023-10-26T01:43:48.232806+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:19:59.336988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.317217+0000","last_change":"2023-10-26T01:43:50.315725+0000","last_active":"2023-10-26T01:43:50.317217+0000","last_peered":"2023-10-26T01:43:50.317217+0000","last_clean":"2023-10-26T01:43:50.317217+0000","last_became_active":"2023-10-26T01:43:50.314793+0000","last_became_peered":"2023-10-26T01:43:50.314793+0000","last_unstale":"2023-10-26T01:43:50.317217+0000","last_undegraded":"2023-10-26T01:43:50.317217+0000","last_fullsized":"2023-10-26T01:43:50.317217+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:00:40.473949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.299633+0000","last_change":"2023-10-26T01:43:49.274596+0000","last_active":"2023-10-26T01:43:50.299633+0000","last_peered":"2023-10-26T01:43:50.299633+0000","last_clean":"2023-10-26T01:43:50.299633+0000","last_became_active":"2023-10-26T01:43:49.274039+0000","last_became_peered":"2023-10-26T01:43:49.274039+0000","last_unstale":"2023-10-26T01:43:50.299633+0000","last_undegraded":"2023-10-26T01:43:50.299633+0000","last_fullsized":"2023-10-26T01:43:50.299633+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_clean_scrub_stamp":"2023-10-26T01:43:48.232806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:51:03.285440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.317113+0000","last_change":"2023-10-26T01:43:50.317041+0000","last_active":"2023-10-26T01:43:50.317113+0000","last_peered":"2023-10-26T01:43:50.317113+0000","last_clean":"2023-10-26T01:43:50.317113+0000","last_became_active":"2023-10-26T01:43:50.316751+0000","last_became_peered":"2023-10-26T01:43:50.316751+0000","last_unstale":"2023-10-26T01:43:50.317113+0000","last_undegraded":"2023-10-26T01:43:50.317113+0000","last_fullsized":"2023-10-26T01:43:50.317113+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:59:34.197770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.300137+0000","last_change":"2023-10-26T01:43:49.264095+0000","last_active":"2023-10-26T01:43:50.300137+0000","last_peered":"2023-10-26T01:43:50.300137+0000","last_clean":"2023-10-26T01:43:50.300137+0000","last_became_active":"2023-10-26T01:43:49.263822+0000","last_became_peered":"2023-10-26T01:43:49.263822+0000","last_unstale":"2023-10-26T01:43:50.300137+0000","last_undegraded":"2023-10-26T01:43:50.300137+0000","last_fullsized":"2023-10-26T01:43:50.300137+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_clean_scrub_stamp":"2023-10-26T01:43:48.232806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:10:22.173425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.313471+0000","last_change":"2023-10-26T01:43:50.313351+0000","last_active":"2023-10-26T01:43:50.313471+0000","last_peered":"2023-10-26T01:43:50.313471+0000","last_clean":"2023-10-26T01:43:50.313471+0000","last_became_active":"2023-10-26T01:43:50.313121+0000","last_became_peered":"2023-10-26T01:43:50.313121+0000","last_unstale":"2023-10-26T01:43:50.313471+0000","last_undegraded":"2023-10-26T01:43:50.313471+0000","last_fullsized":"2023-10-26T01:43:50.313471+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:00:10.505388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.313270+0000","last_change":"2023-10-26T01:43:49.259137+0000","last_active":"2023-10-26T01:43:50.313270+0000","last_peered":"2023-10-26T01:43:50.313270+0000","last_clean":"2023-10-26T01:43:50.313270+0000","last_became_active":"2023-10-26T01:43:49.256376+0000","last_became_peered":"2023-10-26T01:43:49.256376+0000","last_unstale":"2023-10-26T01:43:50.313270+0000","last_undegraded":"2023-10-26T01:43:50.313270+0000","last_fullsized":"2023-10-26T01:43:50.313270+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_clean_scrub_stamp":"2023-10-26T01:43:48.232806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:18:40.463970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.316447+0000","last_change":"2023-10-26T01:43:50.314842+0000","last_active":"2023-10-26T01:43:50.316447+0000","last_peered":"2023-10-26T01:43:50.316447+0000","last_clean":"2023-10-26T01:43:50.316447+0000","last_became_active":"2023-10-26T01:43:50.306052+0000","last_became_peered":"2023-10-26T01:43:50.306052+0000","last_unstale":"2023-10-26T01:43:50.316447+0000","last_undegraded":"2023-10-26T01:43:50.316447+0000","last_fullsized":"2023-10-26T01:43:50.316447+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:45:38.848472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.15","version":"181'3","reported_seq":14,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.328469+0000","last_change":"2023-10-26T01:43:49.265714+0000","last_active":"2023-10-26T01:43:50.328469+0000","last_peered":"2023-10-26T01:43:50.328469+0000","last_clean":"2023-10-26T01:43:50.328469+0000","last_became_active":"2023-10-26T01:43:49.262146+0000","last_became_peered":"2023-10-26T01:43:49.262146+0000","last_unstale":"2023-10-26T01:43:50.328469+0000","last_undegraded":"2023-10-26T01:43:50.328469+0000","last_fullsized":"2023-10-26T01:43:50.328469+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_clean_scrub_stamp":"2023-10-26T01:43:48.232806+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:54:31.893883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.305531+0000","last_change":"2023-10-26T01:43:50.304110+0000","last_active":"2023-10-26T01:43:50.305531+0000","last_peered":"2023-10-26T01:43:50.305531+0000","last_clean":"2023-10-26T01:43:50.305531+0000","last_became_active":"2023-10-26T01:43:50.303740+0000","last_became_peered":"2023-10-26T01:43:50.303740+0000","last_unstale":"2023-10-26T01:43:50.305531+0000","last_undegraded":"2023-10-26T01:43:50.305531+0000","last_fullsized":"2023-10-26T01:43:50.305531+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:45:14.355821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.14","version":"181'6","reported_seq":17,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.331673+0000","last_change":"2023-10-26T01:43:49.262181+0000","last_active":"2023-10-26T01:43:50.331673+0000","last_peered":"2023-10-26T01:43:50.331673+0000","last_clean":"2023-10-26T01:43:50.331673+0000","last_became_active":"2023-10-26T01:43:49.261557+0000","last_became_peered":"2023-10-26T01:43:49.261557+0000","last_unstale":"2023-10-26T01:43:50.331673+0000","last_undegraded":"2023-10-26T01:43:50.331673+0000","last_fullsized":"2023-10-26T01:43:50.331673+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_clean_scrub_stamp":"2023-10-26T01:43:48.232806+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:26:31.291729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.316474+0000","last_change":"2023-10-26T01:43:50.315983+0000","last_active":"2023-10-26T01:43:50.316474+0000","last_peered":"2023-10-26T01:43:50.316474+0000","last_clean":"2023-10-26T01:43:50.316474+0000","last_became_active":"2023-10-26T01:43:50.315759+0000","last_became_peered":"2023-10-26T01:43:50.315759+0000","last_unstale":"2023-10-26T01:43:50.316474+0000","last_undegraded":"2023-10-26T01:43:50.316474+0000","last_fullsized":"2023-10-26T01:43:50.316474+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:07:51.331877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.13","version":"181'2","reported_seq":13,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.329556+0000","last_change":"2023-10-26T01:43:49.268936+0000","last_active":"2023-10-26T01:43:50.329556+0000","last_peered":"2023-10-26T01:43:50.329556+0000","last_clean":"2023-10-26T01:43:50.329556+0000","last_became_active":"2023-10-26T01:43:49.268702+0000","last_became_peered":"2023-10-26T01:43:49.268702+0000","last_unstale":"2023-10-26T01:43:50.329556+0000","last_undegraded":"2023-10-26T01:43:50.329556+0000","last_fullsized":"2023-10-26T01:43:50.329556+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_clean_scrub_stamp":"2023-10-26T01:43:48.232806+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:58:26.341483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.319024+0000","last_change":"2023-10-26T01:43:50.318945+0000","last_active":"2023-10-26T01:43:50.319024+0000","last_peered":"2023-10-26T01:43:50.319024+0000","last_clean":"2023-10-26T01:43:50.319024+0000","last_became_active":"2023-10-26T01:43:50.318580+0000","last_became_peered":"2023-10-26T01:43:50.318580+0000","last_unstale":"2023-10-26T01:43:50.319024+0000","last_undegraded":"2023-10-26T01:43:50.319024+0000","last_fullsized":"2023-10-26T01:43:50.319024+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:08:09.576677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.288968+0000","last_change":"2023-10-26T01:43:49.270191+0000","last_active":"2023-10-26T01:43:50.288968+0000","last_peered":"2023-10-26T01:43:50.288968+0000","last_clean":"2023-10-26T01:43:50.288968+0000","last_became_active":"2023-10-26T01:43:49.269164+0000","last_became_peered":"2023-10-26T01:43:49.269164+0000","last_unstale":"2023-10-26T01:43:50.288968+0000","last_undegraded":"2023-10-26T01:43:50.288968+0000","last_fullsized":"2023-10-26T01:43:50.288968+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_clean_scrub_stamp":"2023-10-26T01:43:48.232806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:56:57.372727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.311555+0000","last_change":"2023-10-26T01:43:50.311196+0000","last_active":"2023-10-26T01:43:50.311555+0000","last_peered":"2023-10-26T01:43:50.311555+0000","last_clean":"2023-10-26T01:43:50.311555+0000","last_became_active":"2023-10-26T01:43:50.310155+0000","last_became_peered":"2023-10-26T01:43:50.310155+0000","last_unstale":"2023-10-26T01:43:50.311555+0000","last_undegraded":"2023-10-26T01:43:50.311555+0000","last_fullsized":"2023-10-26T01:43:50.311555+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:56:06.619004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.300762+0000","last_change":"2023-10-26T01:43:49.272431+0000","last_active":"2023-10-26T01:43:50.300762+0000","last_peered":"2023-10-26T01:43:50.300762+0000","last_clean":"2023-10-26T01:43:50.300762+0000","last_became_active":"2023-10-26T01:43:49.272134+0000","last_became_peered":"2023-10-26T01:43:49.272134+0000","last_unstale":"2023-10-26T01:43:50.300762+0000","last_undegraded":"2023-10-26T01:43:50.300762+0000","last_fullsized":"2023-10-26T01:43:50.300762+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_clean_scrub_stamp":"2023-10-26T01:43:48.232806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:54:35.020329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.315161+0000","last_change":"2023-10-26T01:43:50.314690+0000","last_active":"2023-10-26T01:43:50.315161+0000","last_peered":"2023-10-26T01:43:50.315161+0000","last_clean":"2023-10-26T01:43:50.315161+0000","last_became_active":"2023-10-26T01:43:50.314394+0000","last_became_peered":"2023-10-26T01:43:50.314394+0000","last_unstale":"2023-10-26T01:43:50.315161+0000","last_undegraded":"2023-10-26T01:43:50.315161+0000","last_fullsized":"2023-10-26T01:43:50.315161+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:24:03.603238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.10","version":"181'8","reported_seq":19,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.330719+0000","last_change":"2023-10-26T01:43:49.266389+0000","last_active":"2023-10-26T01:43:50.330719+0000","last_peered":"2023-10-26T01:43:50.330719+0000","last_clean":"2023-10-26T01:43:50.330719+0000","last_became_active":"2023-10-26T01:43:49.265889+0000","last_became_peered":"2023-10-26T01:43:49.265889+0000","last_unstale":"2023-10-26T01:43:50.330719+0000","last_undegraded":"2023-10-26T01:43:50.330719+0000","last_fullsized":"2023-10-26T01:43:50.330719+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_clean_scrub_stamp":"2023-10-26T01:43:48.232806+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:33:46.344343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.318129+0000","last_change":"2023-10-26T01:43:50.317869+0000","last_active":"2023-10-26T01:43:50.318129+0000","last_peered":"2023-10-26T01:43:50.318129+0000","last_clean":"2023-10-26T01:43:50.318129+0000","last_became_active":"2023-10-26T01:43:50.317638+0000","last_became_peered":"2023-10-26T01:43:50.317638+0000","last_unstale":"2023-10-26T01:43:50.318129+0000","last_undegraded":"2023-10-26T01:43:50.318129+0000","last_fullsized":"2023-10-26T01:43:50.318129+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:04:57.615125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"181'1","reported_seq":12,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.328550+0000","last_change":"2023-10-26T01:43:49.275222+0000","last_active":"2023-10-26T01:43:50.328550+0000","last_peered":"2023-10-26T01:43:50.328550+0000","last_clean":"2023-10-26T01:43:50.328550+0000","last_became_active":"2023-10-26T01:43:49.274973+0000","last_became_peered":"2023-10-26T01:43:49.274973+0000","last_unstale":"2023-10-26T01:43:50.328550+0000","last_undegraded":"2023-10-26T01:43:50.328550+0000","last_fullsized":"2023-10-26T01:43:50.328550+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_clean_scrub_stamp":"2023-10-26T01:43:48.232806+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:40:46.207474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.294182+0000","last_change":"2023-10-26T01:43:50.293891+0000","last_active":"2023-10-26T01:43:50.294182+0000","last_peered":"2023-10-26T01:43:50.294182+0000","last_clean":"2023-10-26T01:43:50.294182+0000","last_became_active":"2023-10-26T01:43:50.293145+0000","last_became_peered":"2023-10-26T01:43:50.293145+0000","last_unstale":"2023-10-26T01:43:50.294182+0000","last_undegraded":"2023-10-26T01:43:50.294182+0000","last_fullsized":"2023-10-26T01:43:50.294182+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:38:17.000674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.e","version":"181'2","reported_seq":13,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.327192+0000","last_change":"2023-10-26T01:43:49.275173+0000","last_active":"2023-10-26T01:43:50.327192+0000","last_peered":"2023-10-26T01:43:50.327192+0000","last_clean":"2023-10-26T01:43:50.327192+0000","last_became_active":"2023-10-26T01:43:49.274005+0000","last_became_peered":"2023-10-26T01:43:49.274005+0000","last_unstale":"2023-10-26T01:43:50.327192+0000","last_undegraded":"2023-10-26T01:43:50.327192+0000","last_fullsized":"2023-10-26T01:43:50.327192+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_clean_scrub_stamp":"2023-10-26T01:43:48.232806+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:54:31.893883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.315679+0000","last_change":"2023-10-26T01:43:50.315594+0000","last_active":"2023-10-26T01:43:50.315679+0000","last_peered":"2023-10-26T01:43:50.315679+0000","last_clean":"2023-10-26T01:43:50.315679+0000","last_became_active":"2023-10-26T01:43:50.314593+0000","last_became_peered":"2023-10-26T01:43:50.314593+0000","last_unstale":"2023-10-26T01:43:50.315679+0000","last_undegraded":"2023-10-26T01:43:50.315679+0000","last_fullsized":"2023-10-26T01:43:50.315679+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:35:14.923796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.d","version":"181'1","reported_seq":12,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.326056+0000","last_change":"2023-10-26T01:43:49.273840+0000","last_active":"2023-10-26T01:43:50.326056+0000","last_peered":"2023-10-26T01:43:50.326056+0000","last_clean":"2023-10-26T01:43:50.326056+0000","last_became_active":"2023-10-26T01:43:49.272623+0000","last_became_peered":"2023-10-26T01:43:49.272623+0000","last_unstale":"2023-10-26T01:43:50.326056+0000","last_undegraded":"2023-10-26T01:43:50.326056+0000","last_fullsized":"2023-10-26T01:43:50.326056+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_clean_scrub_stamp":"2023-10-26T01:43:48.232806+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:26:45.190756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.320516+0000","last_change":"2023-10-26T01:43:50.320362+0000","last_active":"2023-10-26T01:43:50.320516+0000","last_peered":"2023-10-26T01:43:50.320516+0000","last_clean":"2023-10-26T01:43:50.320516+0000","last_became_active":"2023-10-26T01:43:50.320018+0000","last_became_peered":"2023-10-26T01:43:50.320018+0000","last_unstale":"2023-10-26T01:43:50.320516+0000","last_undegraded":"2023-10-26T01:43:50.320516+0000","last_fullsized":"2023-10-26T01:43:50.320516+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:33:47.349231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.288666+0000","last_change":"2023-10-26T01:43:49.268357+0000","last_active":"2023-10-26T01:43:50.288666+0000","last_peered":"2023-10-26T01:43:50.288666+0000","last_clean":"2023-10-26T01:43:50.288666+0000","last_became_active":"2023-10-26T01:43:49.267600+0000","last_became_peered":"2023-10-26T01:43:49.267600+0000","last_unstale":"2023-10-26T01:43:50.288666+0000","last_undegraded":"2023-10-26T01:43:50.288666+0000","last_fullsized":"2023-10-26T01:43:50.288666+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_clean_scrub_stamp":"2023-10-26T01:43:48.232806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:40:32.580321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.319381+0000","last_change":"2023-10-26T01:43:50.318926+0000","last_active":"2023-10-26T01:43:50.319381+0000","last_peered":"2023-10-26T01:43:50.319381+0000","last_clean":"2023-10-26T01:43:50.319381+0000","last_became_active":"2023-10-26T01:43:50.318645+0000","last_became_peered":"2023-10-26T01:43:50.318645+0000","last_unstale":"2023-10-26T01:43:50.319381+0000","last_undegraded":"2023-10-26T01:43:50.319381+0000","last_fullsized":"2023-10-26T01:43:50.319381+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:11:45.355135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.b","version":"181'2","reported_seq":13,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.328965+0000","last_change":"2023-10-26T01:43:49.273491+0000","last_active":"2023-10-26T01:43:50.328965+0000","last_peered":"2023-10-26T01:43:50.328965+0000","last_clean":"2023-10-26T01:43:50.328965+0000","last_became_active":"2023-10-26T01:43:49.272989+0000","last_became_peered":"2023-10-26T01:43:49.272989+0000","last_unstale":"2023-10-26T01:43:50.328965+0000","last_undegraded":"2023-10-26T01:43:50.328965+0000","last_fullsized":"2023-10-26T01:43:50.328965+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_clean_scrub_stamp":"2023-10-26T01:43:48.232806+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:31:46.070435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.316677+0000","last_change":"2023-10-26T01:43:50.316597+0000","last_active":"2023-10-26T01:43:50.316677+0000","last_peered":"2023-10-26T01:43:50.316677+0000","last_clean":"2023-10-26T01:43:50.316677+0000","last_became_active":"2023-10-26T01:43:50.316298+0000","last_became_peered":"2023-10-26T01:43:50.316298+0000","last_unstale":"2023-10-26T01:43:50.316677+0000","last_undegraded":"2023-10-26T01:43:50.316677+0000","last_fullsized":"2023-10-26T01:43:50.316677+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:30:07.397807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.290601+0000","last_change":"2023-10-26T01:43:49.264581+0000","last_active":"2023-10-26T01:43:50.290601+0000","last_peered":"2023-10-26T01:43:50.290601+0000","last_clean":"2023-10-26T01:43:50.290601+0000","last_became_active":"2023-10-26T01:43:49.263942+0000","last_became_peered":"2023-10-26T01:43:49.263942+0000","last_unstale":"2023-10-26T01:43:50.290601+0000","last_undegraded":"2023-10-26T01:43:50.290601+0000","last_fullsized":"2023-10-26T01:43:50.290601+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_clean_scrub_stamp":"2023-10-26T01:43:48.232806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:41:57.698469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.315298+0000","last_change":"2023-10-26T01:43:50.313736+0000","last_active":"2023-10-26T01:43:50.315298+0000","last_peered":"2023-10-26T01:43:50.315298+0000","last_clean":"2023-10-26T01:43:50.315298+0000","last_became_active":"2023-10-26T01:43:50.313374+0000","last_became_peered":"2023-10-26T01:43:50.313374+0000","last_unstale":"2023-10-26T01:43:50.315298+0000","last_undegraded":"2023-10-26T01:43:50.315298+0000","last_fullsized":"2023-10-26T01:43:50.315298+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:58:47.374635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.306832+0000","last_change":"2023-10-26T01:43:49.265844+0000","last_active":"2023-10-26T01:43:50.306832+0000","last_peered":"2023-10-26T01:43:50.306832+0000","last_clean":"2023-10-26T01:43:50.306832+0000","last_became_active":"2023-10-26T01:43:49.265590+0000","last_became_peered":"2023-10-26T01:43:49.265590+0000","last_unstale":"2023-10-26T01:43:50.306832+0000","last_undegraded":"2023-10-26T01:43:50.306832+0000","last_fullsized":"2023-10-26T01:43:50.306832+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_clean_scrub_stamp":"2023-10-26T01:43:48.232806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:06:04.697622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.320607+0000","last_change":"2023-10-26T01:43:50.320400+0000","last_active":"2023-10-26T01:43:50.320607+0000","last_peered":"2023-10-26T01:43:50.320607+0000","last_clean":"2023-10-26T01:43:50.320607+0000","last_became_active":"2023-10-26T01:43:50.320163+0000","last_became_peered":"2023-10-26T01:43:50.320163+0000","last_unstale":"2023-10-26T01:43:50.320607+0000","last_undegraded":"2023-10-26T01:43:50.320607+0000","last_fullsized":"2023-10-26T01:43:50.320607+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:22:13.378905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.292680+0000","last_change":"2023-10-26T01:43:49.274338+0000","last_active":"2023-10-26T01:43:50.292680+0000","last_peered":"2023-10-26T01:43:50.292680+0000","last_clean":"2023-10-26T01:43:50.292680+0000","last_became_active":"2023-10-26T01:43:49.273491+0000","last_became_peered":"2023-10-26T01:43:49.273491+0000","last_unstale":"2023-10-26T01:43:50.292680+0000","last_undegraded":"2023-10-26T01:43:50.292680+0000","last_fullsized":"2023-10-26T01:43:50.292680+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_clean_scrub_stamp":"2023-10-26T01:43:48.232806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:41:29.428670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.314229+0000","last_change":"2023-10-26T01:43:50.314153+0000","last_active":"2023-10-26T01:43:50.314229+0000","last_peered":"2023-10-26T01:43:50.314229+0000","last_clean":"2023-10-26T01:43:50.314229+0000","last_became_active":"2023-10-26T01:43:50.313921+0000","last_became_peered":"2023-10-26T01:43:50.313921+0000","last_unstale":"2023-10-26T01:43:50.314229+0000","last_undegraded":"2023-10-26T01:43:50.314229+0000","last_fullsized":"2023-10-26T01:43:50.314229+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:34:37.310319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.7","version":"181'2","reported_seq":13,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.329969+0000","last_change":"2023-10-26T01:43:49.275634+0000","last_active":"2023-10-26T01:43:50.329969+0000","last_peered":"2023-10-26T01:43:50.329969+0000","last_clean":"2023-10-26T01:43:50.329969+0000","last_became_active":"2023-10-26T01:43:49.275430+0000","last_became_peered":"2023-10-26T01:43:49.275430+0000","last_unstale":"2023-10-26T01:43:50.329969+0000","last_undegraded":"2023-10-26T01:43:50.329969+0000","last_fullsized":"2023-10-26T01:43:50.329969+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_clean_scrub_stamp":"2023-10-26T01:43:48.232806+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:40:49.361777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.315314+0000","last_change":"2023-10-26T01:43:50.315243+0000","last_active":"2023-10-26T01:43:50.315314+0000","last_peered":"2023-10-26T01:43:50.315314+0000","last_clean":"2023-10-26T01:43:50.315314+0000","last_became_active":"2023-10-26T01:43:50.314818+0000","last_became_peered":"2023-10-26T01:43:50.314818+0000","last_unstale":"2023-10-26T01:43:50.315314+0000","last_undegraded":"2023-10-26T01:43:50.315314+0000","last_fullsized":"2023-10-26T01:43:50.315314+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:53:06.157549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.301429+0000","last_change":"2023-10-26T01:43:49.269365+0000","last_active":"2023-10-26T01:43:50.301429+0000","last_peered":"2023-10-26T01:43:50.301429+0000","last_clean":"2023-10-26T01:43:50.301429+0000","last_became_active":"2023-10-26T01:43:49.269032+0000","last_became_peered":"2023-10-26T01:43:49.269032+0000","last_unstale":"2023-10-26T01:43:50.301429+0000","last_undegraded":"2023-10-26T01:43:50.301429+0000","last_fullsized":"2023-10-26T01:43:50.301429+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_clean_scrub_stamp":"2023-10-26T01:43:48.232806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:17:54.517773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.319868+0000","last_change":"2023-10-26T01:43:50.319382+0000","last_active":"2023-10-26T01:43:50.319868+0000","last_peered":"2023-10-26T01:43:50.319868+0000","last_clean":"2023-10-26T01:43:50.319868+0000","last_became_active":"2023-10-26T01:43:50.318147+0000","last_became_peered":"2023-10-26T01:43:50.318147+0000","last_unstale":"2023-10-26T01:43:50.319868+0000","last_undegraded":"2023-10-26T01:43:50.319868+0000","last_fullsized":"2023-10-26T01:43:50.319868+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:40:47.212361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.303574+0000","last_change":"2023-10-26T01:43:49.274381+0000","last_active":"2023-10-26T01:43:50.303574+0000","last_peered":"2023-10-26T01:43:50.303574+0000","last_clean":"2023-10-26T01:43:50.303574+0000","last_became_active":"2023-10-26T01:43:49.273694+0000","last_became_peered":"2023-10-26T01:43:49.273694+0000","last_unstale":"2023-10-26T01:43:50.303574+0000","last_undegraded":"2023-10-26T01:43:50.303574+0000","last_fullsized":"2023-10-26T01:43:50.303574+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_clean_scrub_stamp":"2023-10-26T01:43:48.232806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:00:09.500500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.308586+0000","last_change":"2023-10-26T01:43:50.308290+0000","last_active":"2023-10-26T01:43:50.308586+0000","last_peered":"2023-10-26T01:43:50.308586+0000","last_clean":"2023-10-26T01:43:50.308586+0000","last_became_active":"2023-10-26T01:43:50.302771+0000","last_became_peered":"2023-10-26T01:43:50.302771+0000","last_unstale":"2023-10-26T01:43:50.308586+0000","last_undegraded":"2023-10-26T01:43:50.308586+0000","last_fullsized":"2023-10-26T01:43:50.308586+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:13:16.853273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.287508+0000","last_change":"2023-10-26T01:43:49.275138+0000","last_active":"2023-10-26T01:43:50.287508+0000","last_peered":"2023-10-26T01:43:50.287508+0000","last_clean":"2023-10-26T01:43:50.287508+0000","last_became_active":"2023-10-26T01:43:49.274913+0000","last_became_peered":"2023-10-26T01:43:49.274913+0000","last_unstale":"2023-10-26T01:43:50.287508+0000","last_undegraded":"2023-10-26T01:43:50.287508+0000","last_fullsized":"2023-10-26T01:43:50.287508+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_clean_scrub_stamp":"2023-10-26T01:43:48.232806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:49:52.800645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.318160+0000","last_change":"2023-10-26T01:43:50.317537+0000","last_active":"2023-10-26T01:43:50.318160+0000","last_peered":"2023-10-26T01:43:50.318160+0000","last_clean":"2023-10-26T01:43:50.318160+0000","last_became_active":"2023-10-26T01:43:50.317268+0000","last_became_peered":"2023-10-26T01:43:50.317268+0000","last_unstale":"2023-10-26T01:43:50.318160+0000","last_undegraded":"2023-10-26T01:43:50.318160+0000","last_fullsized":"2023-10-26T01:43:50.318160+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:21:57.042490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.294298+0000","last_change":"2023-10-26T01:43:49.271052+0000","last_active":"2023-10-26T01:43:50.294298+0000","last_peered":"2023-10-26T01:43:50.294298+0000","last_clean":"2023-10-26T01:43:50.294298+0000","last_became_active":"2023-10-26T01:43:49.267753+0000","last_became_peered":"2023-10-26T01:43:49.267753+0000","last_unstale":"2023-10-26T01:43:50.294298+0000","last_undegraded":"2023-10-26T01:43:50.294298+0000","last_fullsized":"2023-10-26T01:43:50.294298+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_clean_scrub_stamp":"2023-10-26T01:43:48.232806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:28:59.234606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.317321+0000","last_change":"2023-10-26T01:43:50.317111+0000","last_active":"2023-10-26T01:43:50.317321+0000","last_peered":"2023-10-26T01:43:50.317321+0000","last_clean":"2023-10-26T01:43:50.317321+0000","last_became_active":"2023-10-26T01:43:50.315676+0000","last_became_peered":"2023-10-26T01:43:50.315676+0000","last_unstale":"2023-10-26T01:43:50.317321+0000","last_undegraded":"2023-10-26T01:43:50.317321+0000","last_fullsized":"2023-10-26T01:43:50.317321+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:30:07.397807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.301086+0000","last_change":"2023-10-26T01:43:49.263842+0000","last_active":"2023-10-26T01:43:50.301086+0000","last_peered":"2023-10-26T01:43:50.301086+0000","last_clean":"2023-10-26T01:43:50.301086+0000","last_became_active":"2023-10-26T01:43:49.263479+0000","last_became_peered":"2023-10-26T01:43:49.263479+0000","last_unstale":"2023-10-26T01:43:50.301086+0000","last_undegraded":"2023-10-26T01:43:50.301086+0000","last_fullsized":"2023-10-26T01:43:50.301086+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_clean_scrub_stamp":"2023-10-26T01:43:48.232806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:54:38.386630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.314637+0000","last_change":"2023-10-26T01:43:50.314535+0000","last_active":"2023-10-26T01:43:50.314637+0000","last_peered":"2023-10-26T01:43:50.314637+0000","last_clean":"2023-10-26T01:43:50.314637+0000","last_became_active":"2023-10-26T01:43:50.313916+0000","last_became_peered":"2023-10-26T01:43:50.313916+0000","last_unstale":"2023-10-26T01:43:50.314637+0000","last_undegraded":"2023-10-26T01:43:50.314637+0000","last_fullsized":"2023-10-26T01:43:50.314637+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:45:14.355821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.299832+0000","last_change":"2023-10-26T01:43:49.272389+0000","last_active":"2023-10-26T01:43:50.299832+0000","last_peered":"2023-10-26T01:43:50.299832+0000","last_clean":"2023-10-26T01:43:50.299832+0000","last_became_active":"2023-10-26T01:43:49.272152+0000","last_became_peered":"2023-10-26T01:43:49.272152+0000","last_unstale":"2023-10-26T01:43:50.299832+0000","last_undegraded":"2023-10-26T01:43:50.299832+0000","last_fullsized":"2023-10-26T01:43:50.299832+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_clean_scrub_stamp":"2023-10-26T01:43:48.232806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:29:37.539714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.319840+0000","last_change":"2023-10-26T01:43:50.319770+0000","last_active":"2023-10-26T01:43:50.319840+0000","last_peered":"2023-10-26T01:43:50.319840+0000","last_clean":"2023-10-26T01:43:50.319840+0000","last_became_active":"2023-10-26T01:43:50.319515+0000","last_became_peered":"2023-10-26T01:43:50.319515+0000","last_unstale":"2023-10-26T01:43:50.319840+0000","last_undegraded":"2023-10-26T01:43:50.319840+0000","last_fullsized":"2023-10-26T01:43:50.319840+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:52:37.600083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.290196+0000","last_change":"2023-10-26T01:43:49.256981+0000","last_active":"2023-10-26T01:43:50.290196+0000","last_peered":"2023-10-26T01:43:50.290196+0000","last_clean":"2023-10-26T01:43:50.290196+0000","last_became_active":"2023-10-26T01:43:49.256295+0000","last_became_peered":"2023-10-26T01:43:49.256295+0000","last_unstale":"2023-10-26T01:43:50.290196+0000","last_undegraded":"2023-10-26T01:43:50.290196+0000","last_fullsized":"2023-10-26T01:43:50.290196+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_clean_scrub_stamp":"2023-10-26T01:43:48.232806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:10:22.173425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.312632+0000","last_change":"2023-10-26T01:43:50.312485+0000","last_active":"2023-10-26T01:43:50.312632+0000","last_peered":"2023-10-26T01:43:50.312632+0000","last_clean":"2023-10-26T01:43:50.312632+0000","last_became_active":"2023-10-26T01:43:50.307903+0000","last_became_peered":"2023-10-26T01:43:50.307903+0000","last_unstale":"2023-10-26T01:43:50.312632+0000","last_undegraded":"2023-10-26T01:43:50.312632+0000","last_fullsized":"2023-10-26T01:43:50.312632+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:25:08.147968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.293974+0000","last_change":"2023-10-26T01:43:49.266363+0000","last_active":"2023-10-26T01:43:50.293974+0000","last_peered":"2023-10-26T01:43:50.293974+0000","last_clean":"2023-10-26T01:43:50.293974+0000","last_became_active":"2023-10-26T01:43:49.265947+0000","last_became_peered":"2023-10-26T01:43:49.265947+0000","last_unstale":"2023-10-26T01:43:50.293974+0000","last_undegraded":"2023-10-26T01:43:50.293974+0000","last_fullsized":"2023-10-26T01:43:50.293974+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_clean_scrub_stamp":"2023-10-26T01:43:48.232806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:53:05.152661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":324,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-26T01:43:49.255543+0000","last_change":"2023-10-26T01:30:34.883484+0000","last_active":"2023-10-26T01:43:49.255543+0000","last_peered":"2023-10-26T01:43:49.255543+0000","last_clean":"2023-10-26T01:43:49.255543+0000","last_became_active":"2023-10-26T01:30:34.883071+0000","last_became_peered":"2023-10-26T01:30:34.883071+0000","last_unstale":"2023-10-26T01:43:49.255543+0000","last_undegraded":"2023-10-26T01:43:49.255543+0000","last_fullsized":"2023-10-26T01:43:49.255543+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:32.890096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:32.890096+0000","last_clean_scrub_stamp":"2023-10-26T01:30:32.890096+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:26:31.879452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":23,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":22,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":14,"seq":60129542307,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":995,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194317,"statfs":{"total":200000000,"available":198981104,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:42:55 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.84499999999999997,"15min":0.80800000000000005},"min":{"1min":0.49199999999999999,"5min":0.443,"15min":0.33000000000000002},"max":{"1min":1.278,"5min":1.571,"15min":2.2549999999999999},"last":0.377},{"interface":"front","average":{"1min":0.92700000000000005,"5min":1.004,"15min":0.91900000000000004},"min":{"1min":0.40799999999999997,"5min":0.309,"15min":0.309},"max":{"1min":1.49,"5min":1.919,"15min":1.919},"last":0.50600000000000001}]},{"osd":1,"last update":"Thu Oct 26 01:42:55 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.91700000000000004,"15min":0.91000000000000003},"min":{"1min":0.44900000000000001,"5min":0.39400000000000002,"15min":0.32300000000000001},"max":{"1min":1.357,"5min":1.569,"15min":4.4340000000000002},"last":0.48899999999999999},{"interface":"front","average":{"1min":1.028,"5min":1.0649999999999999,"15min":1.012},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.374},"max":{"1min":1.617,"5min":1.7070000000000001,"15min":2.3330000000000002},"last":0.53200000000000003}]},{"osd":2,"last update":"Thu Oct 26 01:42:55 2023","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.081,"15min":1.0640000000000001},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.39900000000000002},"max":{"1min":1.6819999999999999,"5min":1.6819999999999999,"15min":4.2800000000000002},"last":0.44},{"interface":"front","average":{"1min":1.117,"5min":1.173,"15min":1.123},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.49399999999999999},"max":{"1min":1.7410000000000001,"5min":1.8129999999999999,"15min":2.302},"last":0.46700000000000003}]},{"osd":4,"last update":"Thu Oct 26 01:42:55 2023","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.236,"15min":1.196},"min":{"1min":0.89900000000000002,"5min":0.78700000000000003,"15min":0.66400000000000003},"max":{"1min":1.516,"5min":1.732,"15min":2.1070000000000002},"last":0.66000000000000003},{"interface":"front","average":{"1min":1.365,"5min":1.3759999999999999,"15min":1.3700000000000001},"min":{"1min":1.046,"5min":0.91900000000000004,"15min":0.79900000000000004},"max":{"1min":1.653,"5min":1.7509999999999999,"15min":4.8250000000000002},"last":0.61099999999999999}]},{"osd":5,"last update":"Thu Oct 26 01:42:55 2023","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.3220000000000001,"15min":1.3129999999999999},"min":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.72499999999999998},"max":{"1min":1.8109999999999999,"5min":1.8109999999999999,"15min":4.9080000000000004},"last":0.73899999999999999},{"interface":"front","average":{"1min":1.3280000000000001,"5min":1.3520000000000001,"15min":1.3},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.9279999999999999,"5min":1.9279999999999999,"15min":2.8980000000000001},"last":0.81299999999999994}]},{"osd":6,"last update":"Thu Oct 26 01:42:55 2023","interfaces":[{"interface":"back","average":{"1min":1.5900000000000001,"5min":1.4359999999999999,"15min":1.369},"min":{"1min":0.79700000000000004,"5min":0.75900000000000001,"15min":0.70199999999999996},"max":{"1min":3.5800000000000001,"5min":3.5800000000000001,"15min":3.5800000000000001},"last":0.85099999999999998},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.381,"15min":1.302},"min":{"1min":0.80800000000000005,"5min":0.755,"15min":0.73099999999999998},"max":{"1min":1.5609999999999999,"5min":1.806,"15min":2.181},"last":0.79100000000000004}]},{"osd":7,"last update":"Thu Oct 26 01:42:55 2023","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.411,"15min":1.3380000000000001},"min":{"1min":0.94899999999999995,"5min":0.78800000000000003,"15min":0.71999999999999997},"max":{"1min":1.647,"5min":2.024,"15min":2.0680000000000001},"last":0.87},{"interface":"front","average":{"1min":1.4059999999999999,"5min":1.419,"15min":1.405},"min":{"1min":0.84399999999999997,"5min":0.73299999999999998,"15min":0.72199999999999998},"max":{"1min":1.8560000000000001,"5min":1.8560000000000001,"15min":4.8730000000000002},"last":0.70399999999999996}]}]},{"osd":7,"up_from":13,"seq":55834575010,"num_pgs":13,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1564,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193747,"statfs":{"total":200000000,"available":198397545,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:43:01 2023","interfaces":[{"interface":"back","average":{"1min":1.2090000000000001,"5min":1.1930000000000001,"15min":1.1359999999999999},"min":{"1min":0.63200000000000001,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.6970000000000001,"5min":2.2269999999999999,"15min":2.2269999999999999},"last":0.83899999999999997},{"interface":"front","average":{"1min":1.181,"5min":1.218,"15min":1.137},"min":{"1min":0.79800000000000004,"5min":0.67300000000000004,"15min":0.58699999999999997},"max":{"1min":1.738,"5min":2.282,"15min":2.6779999999999999},"last":0.92700000000000005}]},{"osd":1,"last update":"Thu Oct 26 01:43:01 2023","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":1.165,"15min":1.1399999999999999},"min":{"1min":0.755,"5min":0.65600000000000003,"15min":0.65200000000000002},"max":{"1min":1.51,"5min":1.8,"15min":2.8260000000000001},"last":0.97399999999999998},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.2310000000000001,"15min":1.2130000000000001},"min":{"1min":0.97399999999999998,"5min":0.71699999999999997,"15min":0.61199999999999999},"max":{"1min":1.736,"5min":2.0059999999999998,"15min":3.2730000000000001},"last":1.1910000000000001}]},{"osd":2,"last update":"Thu Oct 26 01:43:01 2023","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.2250000000000001,"15min":1.165},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.58499999999999996},"max":{"1min":1.679,"5min":1.851,"15min":2.218},"last":1.6100000000000001},{"interface":"front","average":{"1min":1.264,"5min":1.2769999999999999,"15min":1.264},"min":{"1min":0.90300000000000002,"5min":0.82299999999999995,"15min":0.73099999999999998},"max":{"1min":1.879,"5min":2.234,"15min":3.0510000000000002},"last":1.248}]},{"osd":3,"last update":"Thu Oct 26 01:43:01 2023","interfaces":[{"interface":"back","average":{"1min":1.319,"5min":1.327,"15min":1.264},"min":{"1min":1.101,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.8149999999999999,"5min":2.1379999999999999,"15min":2.1379999999999999},"last":1.5449999999999999},{"interface":"front","average":{"1min":1.421,"5min":1.4079999999999999,"15min":1.3899999999999999},"min":{"1min":1.008,"5min":0.71099999999999997,"15min":0.65700000000000003},"max":{"1min":1.825,"5min":2.3050000000000002,"15min":5.5039999999999996},"last":1.0529999999999999}]},{"osd":4,"last update":"Thu Oct 26 01:43:01 2023","interfaces":[{"interface":"back","average":{"1min":1.359,"5min":1.4039999999999999,"15min":1.3720000000000001},"min":{"1min":0.92700000000000005,"5min":0.81799999999999995,"15min":0.76100000000000001},"max":{"1min":2.0899999999999999,"5min":2.54,"15min":2.77},"last":1.1439999999999999},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.3919999999999999,"15min":1.4039999999999999},"min":{"1min":0.85399999999999998,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.7470000000000001,"5min":2.8660000000000001,"15min":3.1419999999999999},"last":1.423}]},{"osd":5,"last update":"Thu Oct 26 01:43:01 2023","interfaces":[{"interface":"back","average":{"1min":1.3839999999999999,"5min":1.3999999999999999,"15min":1.4279999999999999},"min":{"1min":0.94899999999999995,"5min":0.435,"15min":0.435},"max":{"1min":1.8180000000000001,"5min":2.3639999999999999,"15min":5.734},"last":1.375},{"interface":"front","average":{"1min":1.5309999999999999,"5min":1.5149999999999999,"15min":1.4970000000000001},"min":{"1min":1.119,"5min":0.76400000000000001,"15min":0.51900000000000002},"max":{"1min":2.2109999999999999,"5min":2.5169999999999999,"15min":2.8980000000000001},"last":1.3140000000000001}]},{"osd":6,"last update":"Thu Oct 26 01:43:01 2023","interfaces":[{"interface":"back","average":{"1min":1.542,"5min":1.5429999999999999,"15min":1.6890000000000001},"min":{"1min":1.1299999999999999,"5min":0.51000000000000001,"15min":0.47399999999999998},"max":{"1min":1.925,"5min":2.5259999999999998,"15min":11.577999999999999},"last":1.292},{"interface":"front","average":{"1min":1.5780000000000001,"5min":1.655,"15min":1.7470000000000001},"min":{"1min":1.2989999999999999,"5min":1.0640000000000001,"15min":0.55100000000000005},"max":{"1min":2.302,"5min":2.7250000000000001,"15min":10.704000000000001},"last":1.474}]}]},{"osd":6,"up_from":13,"seq":55834575010,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":994,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194318,"statfs":{"total":200000000,"available":198981730,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:42:56 2023","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":1.1279999999999999,"15min":1.133},"min":{"1min":0.70099999999999996,"5min":0.66600000000000004,"15min":0.60399999999999998},"max":{"1min":1.6060000000000001,"5min":2.7370000000000001,"15min":2.7370000000000001},"last":1.222},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.1830000000000001,"15min":1.1439999999999999},"min":{"1min":0.85199999999999998,"5min":0.61699999999999999,"15min":0.59199999999999997},"max":{"1min":1.833,"5min":4.4779999999999998,"15min":4.4779999999999998},"last":1.375}]},{"osd":1,"last update":"Thu Oct 26 01:42:56 2023","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":1.179,"15min":1.179},"min":{"1min":0.751,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.4490000000000001,"5min":5.5359999999999996,"15min":5.5359999999999996},"last":1.335},{"interface":"front","average":{"1min":1.169,"5min":1.2430000000000001,"15min":1.2090000000000001},"min":{"1min":0.76400000000000001,"5min":0.74399999999999999,"15min":0.64000000000000001},"max":{"1min":1.6759999999999999,"5min":4.3579999999999997,"15min":4.3579999999999997},"last":1.012}]},{"osd":2,"last update":"Thu Oct 26 01:42:56 2023","interfaces":[{"interface":"back","average":{"1min":1.24,"5min":1.1799999999999999,"15min":1.1379999999999999},"min":{"1min":0.82999999999999996,"5min":0.69799999999999995,"15min":0.59799999999999998},"max":{"1min":1.651,"5min":2.367,"15min":3.7879999999999998},"last":0.97699999999999998},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.3140000000000001,"15min":1.29},"min":{"1min":0.93400000000000005,"5min":0.86799999999999999,"15min":0.68899999999999995},"max":{"1min":1.704,"5min":3.0470000000000002,"15min":3.0470000000000002},"last":1.409}]},{"osd":3,"last update":"Thu Oct 26 01:42:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.2749999999999999,"15min":1.254},"min":{"1min":0.93500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.7569999999999999,"5min":3.266,"15min":4.4560000000000004},"last":1.177},{"interface":"front","average":{"1min":1.399,"5min":1.369,"15min":1.341},"min":{"1min":0.93700000000000006,"5min":0.86399999999999999,"15min":0.69499999999999995},"max":{"1min":1.8009999999999999,"5min":2.6909999999999998,"15min":2.6909999999999998},"last":1.1359999999999999}]},{"osd":4,"last update":"Thu Oct 26 01:42:56 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.262,"15min":1.2430000000000001},"min":{"1min":0.59899999999999998,"5min":0.53400000000000003,"15min":0.48399999999999999},"max":{"1min":1.6439999999999999,"5min":1.825,"15min":2.9489999999999998},"last":0.874},{"interface":"front","average":{"1min":1.357,"5min":1.345,"15min":1.345},"min":{"1min":0.88400000000000001,"5min":0.60699999999999998,"15min":0.59299999999999997},"max":{"1min":1.8959999999999999,"5min":2.169,"15min":6.609},"last":1.29}]},{"osd":5,"last update":"Thu Oct 26 01:42:56 2023","interfaces":[{"interface":"back","average":{"1min":1.556,"5min":1.5369999999999999,"15min":1.496},"min":{"1min":1.2769999999999999,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.9119999999999999,"5min":2.04,"15min":5.4370000000000003},"last":1.4330000000000001},{"interface":"front","average":{"1min":1.4379999999999999,"5min":1.464,"15min":1.4370000000000001},"min":{"1min":0.69999999999999996,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.7010000000000001,"5min":2.7759999999999998,"15min":2.7759999999999998},"last":1.4910000000000001}]},{"osd":7,"last update":"Thu Oct 26 01:42:56 2023","interfaces":[{"interface":"back","average":{"1min":1.389,"5min":1.4099999999999999,"15min":1.4279999999999999},"min":{"1min":0.72399999999999998,"5min":0.58899999999999997,"15min":0.54100000000000004},"max":{"1min":1.887,"5min":1.984,"15min":11.804},"last":1.2390000000000001},{"interface":"front","average":{"1min":1.492,"5min":1.5429999999999999,"15min":1.548},"min":{"1min":0.96399999999999997,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.7470000000000001,"5min":3.7650000000000001,"15min":11.541},"last":1.095}]}]},{"osd":5,"up_from":13,"seq":55834575010,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":995,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194316,"statfs":{"total":200000000,"available":198980534,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:42:57 2023","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.2310000000000001,"15min":1.173},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53300000000000003},"max":{"1min":1.651,"5min":3.8210000000000002,"15min":3.8210000000000002},"last":1.3109999999999999},{"interface":"front","average":{"1min":1.2390000000000001,"5min":1.3100000000000001,"15min":1.242},"min":{"1min":0.82099999999999995,"5min":0.68600000000000005,"15min":0.67900000000000005},"max":{"1min":1.585,"5min":2.9889999999999999,"15min":2.9889999999999999},"last":1.4950000000000001}]},{"osd":1,"last update":"Thu Oct 26 01:42:57 2023","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.22,"15min":1.137},"min":{"1min":0.71299999999999997,"5min":0.70999999999999996,"15min":0.65900000000000003},"max":{"1min":1.6220000000000001,"5min":1.899,"15min":1.899},"last":1.5349999999999999},{"interface":"front","average":{"1min":1.141,"5min":1.147,"15min":1.1240000000000001},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.5600000000000001,"5min":2.4340000000000002,"15min":2.4340000000000002},"last":1.456}]},{"osd":2,"last update":"Thu Oct 26 01:42:57 2023","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.204,"15min":1.1839999999999999},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.69299999999999995},"max":{"1min":1.577,"5min":1.796,"15min":1.8220000000000001},"last":1.272},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.2549999999999999,"15min":1.212},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.673,"5min":1.875,"15min":1.875},"last":1.3460000000000001}]},{"osd":3,"last update":"Thu Oct 26 01:42:57 2023","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":1.3069999999999999,"15min":1.2689999999999999},"min":{"1min":0.91200000000000003,"5min":0.72399999999999998,"15min":0.72299999999999998},"max":{"1min":1.6899999999999999,"5min":1.7,"15min":1.835},"last":1.647},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.3149999999999999,"15min":1.304},"min":{"1min":0.67600000000000005,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.655,"5min":2.6669999999999998,"15min":2.9249999999999998},"last":1.573}]},{"osd":4,"last update":"Thu Oct 26 01:42:57 2023","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.2250000000000001,"15min":1.224},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.49099999999999999},"max":{"1min":1.611,"5min":2.7770000000000001,"15min":2.7770000000000001},"last":1.083},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.343,"15min":1.367},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.647,"5min":1.857,"15min":1.867},"last":1.399}]},{"osd":6,"last update":"Thu Oct 26 01:42:57 2023","interfaces":[{"interface":"back","average":{"1min":1.383,"5min":1.4199999999999999,"15min":1.3759999999999999},"min":{"1min":0.80000000000000004,"5min":0.79000000000000004,"15min":0.51900000000000002},"max":{"1min":1.6819999999999999,"5min":2.4590000000000001,"15min":2.8250000000000002},"last":1.1419999999999999},{"interface":"front","average":{"1min":1.355,"5min":1.4370000000000001,"15min":1.385},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.61799999999999999},"max":{"1min":1.7210000000000001,"5min":2.4830000000000001,"15min":2.4830000000000001},"last":1.208}]},{"osd":7,"last update":"Thu Oct 26 01:42:57 2023","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":1.5109999999999999,"15min":1.474},"min":{"1min":0.58799999999999997,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.766,"5min":3.621,"15min":3.621},"last":1.6240000000000001},{"interface":"front","average":{"1min":1.534,"5min":1.5629999999999999,"15min":1.4990000000000001},"min":{"1min":0.85299999999999998,"5min":0.82899999999999996,"15min":0.59499999999999997},"max":{"1min":1.9970000000000001,"5min":2.8010000000000002,"15min":2.8010000000000002},"last":1.601}]}]},{"osd":4,"up_from":13,"seq":55834575010,"num_pgs":6,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":988,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194324,"statfs":{"total":200000000,"available":198987913,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:43:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.1850000000000001,"15min":1.079},"min":{"1min":0.57699999999999996,"5min":0.503,"15min":0.503},"max":{"1min":1.6499999999999999,"5min":1.78,"15min":2.0289999999999999},"last":1.21},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.1639999999999999,"15min":1.087},"min":{"1min":0.85799999999999998,"5min":0.72599999999999998,"15min":0.60599999999999998},"max":{"1min":1.6639999999999999,"5min":1.921,"15min":2.6829999999999998},"last":1.3340000000000001}]},{"osd":1,"last update":"Thu Oct 26 01:43:49 2023","interfaces":[{"interface":"back","average":{"1min":1.167,"5min":1.2270000000000001,"15min":1.1699999999999999},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.56799999999999995},"max":{"1min":1.804,"5min":2.056,"15min":4.0519999999999996},"last":1.3899999999999999},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.274,"15min":1.1839999999999999},"min":{"1min":0.77800000000000002,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.599,"5min":2.4390000000000001,"15min":2.4390000000000001},"last":1.3080000000000001}]},{"osd":2,"last update":"Thu Oct 26 01:43:49 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.3839999999999999,"15min":1.3149999999999999},"min":{"1min":0.88100000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":2.048,"5min":2.2040000000000002,"15min":5.6749999999999998},"last":1.27},{"interface":"front","average":{"1min":1.196,"5min":1.224,"15min":1.1870000000000001},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.7190000000000001,"5min":1.986,"15min":3.673},"last":1.248}]},{"osd":3,"last update":"Thu Oct 26 01:43:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.357,"15min":1.2529999999999999},"min":{"1min":0.64700000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.708,"5min":2.5960000000000001,"15min":2.5960000000000001},"last":1.054},{"interface":"front","average":{"1min":1.196,"5min":1.2509999999999999,"15min":1.202},"min":{"1min":0.78600000000000003,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.589,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":1.0860000000000001}]},{"osd":5,"last update":"Thu Oct 26 01:43:49 2023","interfaces":[{"interface":"back","average":{"1min":1.228,"5min":1.2949999999999999,"15min":1.278},"min":{"1min":0.876,"5min":0.71099999999999997,"15min":0.46300000000000002},"max":{"1min":1.8640000000000001,"5min":2.774,"15min":2.774},"last":0.876},{"interface":"front","average":{"1min":1.272,"5min":1.355,"15min":1.3320000000000001},"min":{"1min":0.93000000000000005,"5min":0.77700000000000002,"15min":0.55200000000000005},"max":{"1min":1.6539999999999999,"5min":2.3109999999999999,"15min":5.8579999999999997},"last":0.97899999999999998}]},{"osd":6,"last update":"Thu Oct 26 01:43:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3180000000000001,"5min":1.371,"15min":1.3620000000000001},"min":{"1min":0.94899999999999995,"5min":0.69499999999999995,"15min":0.59299999999999997},"max":{"1min":1.76,"5min":2.3780000000000001,"15min":12.343999999999999},"last":1.4510000000000001},{"interface":"front","average":{"1min":1.5389999999999999,"5min":1.6000000000000001,"15min":1.5660000000000001},"min":{"1min":1.113,"5min":0.80000000000000004,"15min":0.55000000000000004},"max":{"1min":2.2799999999999998,"5min":2.9020000000000001,"15min":12.605},"last":1.4199999999999999}]},{"osd":7,"last update":"Thu Oct 26 01:43:49 2023","interfaces":[{"interface":"back","average":{"1min":1.466,"5min":1.5740000000000001,"15min":1.522},"min":{"1min":1.0649999999999999,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":2.044,"5min":2.7200000000000002,"15min":5.774},"last":1.157},{"interface":"front","average":{"1min":1.468,"5min":1.542,"15min":1.4990000000000001},"min":{"1min":0.79500000000000004,"5min":0.79000000000000004,"15min":0.68000000000000005},"max":{"1min":1.8500000000000001,"5min":2.8149999999999999,"15min":4.6520000000000001},"last":1.363}]}]},{"osd":2,"up_from":13,"seq":55834575010,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":988,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194324,"statfs":{"total":200000000,"available":198987913,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:43:02 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.80000000000000004,"15min":0.83699999999999997},"min":{"1min":0.441,"5min":0.38500000000000001,"15min":0.308},"max":{"1min":1.0660000000000001,"5min":1.4079999999999999,"15min":6.3200000000000003},"last":1.363},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.94599999999999995,"15min":0.94899999999999995},"min":{"1min":0.46400000000000002,"5min":0.432,"15min":0.38600000000000001},"max":{"1min":1.4359999999999999,"5min":1.476,"15min":4.2619999999999996},"last":0.84199999999999997}]},{"osd":1,"last update":"Thu Oct 26 01:43:02 2023","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.872,"15min":0.89000000000000001},"min":{"1min":0.48799999999999999,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.4339999999999999,"5min":1.504,"15min":5.1260000000000003},"last":0.73499999999999999},{"interface":"front","average":{"1min":1.087,"5min":1.0209999999999999,"15min":1.0149999999999999},"min":{"1min":0.58199999999999996,"5min":0.42699999999999999,"15min":0.36599999999999999},"max":{"1min":1.5469999999999999,"5min":1.5469999999999999,"15min":6.1520000000000001},"last":1.103}]},{"osd":3,"last update":"Thu Oct 26 01:43:02 2023","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.089,"15min":1.004},"min":{"1min":0.55800000000000005,"5min":0.435,"15min":0.435},"max":{"1min":1.5760000000000001,"5min":1.909,"15min":2.0640000000000001},"last":0.98699999999999999},{"interface":"front","average":{"1min":1.1439999999999999,"5min":1.105,"15min":1.0720000000000001},"min":{"1min":0.52600000000000002,"5min":0.503,"15min":0.45300000000000001},"max":{"1min":1.6639999999999999,"5min":1.6659999999999999,"15min":8.9019999999999992},"last":1.0880000000000001}]},{"osd":4,"last update":"Thu Oct 26 01:43:02 2023","interfaces":[{"interface":"back","average":{"1min":1.363,"5min":1.3200000000000001,"15min":1.302},"min":{"1min":1.04,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.9910000000000001,"5min":2.109,"15min":6.2670000000000003},"last":1.544},{"interface":"front","average":{"1min":1.3129999999999999,"5min":1.2689999999999999,"15min":1.2809999999999999},"min":{"1min":0.77500000000000002,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.659,"5min":1.923,"15min":7.343},"last":1.266}]},{"osd":5,"last update":"Thu Oct 26 01:43:02 2023","interfaces":[{"interface":"back","average":{"1min":1.405,"5min":1.333,"15min":1.3140000000000001},"min":{"1min":0.81799999999999995,"5min":0.76900000000000002,"15min":0.71899999999999997},"max":{"1min":1.784,"5min":2.3849999999999998,"15min":3.7170000000000001},"last":1.4390000000000001},{"interface":"front","average":{"1min":1.292,"5min":1.3260000000000001,"15min":1.3089999999999999},"min":{"1min":0.90300000000000002,"5min":0.69499999999999995,"15min":0.65300000000000002},"max":{"1min":1.591,"5min":1.8140000000000001,"15min":6.3899999999999997},"last":1.3080000000000001}]},{"osd":6,"last update":"Thu Oct 26 01:43:02 2023","interfaces":[{"interface":"back","average":{"1min":1.3540000000000001,"5min":1.3779999999999999,"15min":1.365},"min":{"1min":0.84799999999999998,"5min":0.76300000000000001,"15min":0.70599999999999996},"max":{"1min":1.6759999999999999,"5min":2.097,"15min":6.0490000000000004},"last":1.6399999999999999},{"interface":"front","average":{"1min":1.464,"5min":1.395,"15min":1.389},"min":{"1min":1.0720000000000001,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":2.036,"5min":2.5230000000000001,"15min":5.0650000000000004},"last":1.6659999999999999}]},{"osd":7,"last update":"Thu Oct 26 01:43:02 2023","interfaces":[{"interface":"back","average":{"1min":1.363,"5min":1.4159999999999999,"15min":1.4039999999999999},"min":{"1min":0.92400000000000004,"5min":0.79200000000000004,"15min":0.69399999999999995},"max":{"1min":1.7569999999999999,"5min":1.7949999999999999,"15min":6.4480000000000004},"last":1.476},{"interface":"front","average":{"1min":1.476,"5min":1.484,"15min":1.474},"min":{"1min":1.0569999999999999,"5min":0.91000000000000003,"15min":0.91000000000000003},"max":{"1min":1.804,"5min":2.1480000000000001,"15min":2.8439999999999999},"last":1.706}]}]},{"osd":1,"up_from":13,"seq":55834575010,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":995,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194317,"statfs":{"total":200000000,"available":198981104,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:42:53 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.82599999999999996,"15min":0.77000000000000002},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.29499999999999998},"max":{"1min":1.177,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.95699999999999996,"15min":0.86099999999999999},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.36199999999999999},"max":{"1min":1.591,"5min":1.591,"15min":1.6399999999999999},"last":0.66400000000000003}]},{"osd":2,"last update":"Thu Oct 26 01:42:53 2023","interfaces":[{"interface":"back","average":{"1min":1.0840000000000001,"5min":1.095,"15min":1.026},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.39600000000000002},"max":{"1min":1.5800000000000001,"5min":1.655,"15min":2.4830000000000001},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.98099999999999998,"5min":1.0389999999999999,"15min":0.93999999999999995},"min":{"1min":0.51000000000000001,"5min":0.48099999999999998,"15min":0.41199999999999998},"max":{"1min":1.2330000000000001,"5min":1.752,"15min":1.8080000000000001},"last":1.0980000000000001}]},{"osd":3,"last update":"Thu Oct 26 01:42:58 2023","interfaces":[{"interface":"back","average":{"1min":1.0569999999999999,"5min":1.073,"15min":0.997},"min":{"1min":0.64100000000000001,"5min":0.57099999999999995,"15min":0.39500000000000002},"max":{"1min":1.3879999999999999,"5min":1.6100000000000001,"15min":1.75},"last":1.022},{"interface":"front","average":{"1min":1.1299999999999999,"5min":1.0980000000000001,"15min":1.0329999999999999},"min":{"1min":0.57499999999999996,"5min":0.55000000000000004,"15min":0.41299999999999998},"max":{"1min":1.556,"5min":1.556,"15min":1.802},"last":1.054}]},{"osd":4,"last update":"Thu Oct 26 01:42:53 2023","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":1.2749999999999999,"15min":1.2310000000000001},"min":{"1min":0.95899999999999996,"5min":0.79800000000000004,"15min":0.56200000000000006},"max":{"1min":1.591,"5min":1.726,"15min":2.0659999999999998},"last":1.2090000000000001},{"interface":"front","average":{"1min":1.248,"5min":1.2509999999999999,"15min":1.1839999999999999},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.65800000000000003},"max":{"1min":1.7450000000000001,"5min":1.8420000000000001,"15min":2.0270000000000001},"last":1.6850000000000001}]},{"osd":5,"last update":"Thu Oct 26 01:42:53 2023","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.3120000000000001,"15min":1.2589999999999999},"min":{"1min":0.629,"5min":0.629,"15min":0.61799999999999999},"max":{"1min":1.7,"5min":1.7050000000000001,"15min":1.8779999999999999},"last":1.464},{"interface":"front","average":{"1min":1.3480000000000001,"5min":1.395,"15min":1.359},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.59999999999999998},"max":{"1min":1.639,"5min":1.8280000000000001,"15min":2.2200000000000002},"last":1.349}]},{"osd":6,"last update":"Thu Oct 26 01:42:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3460000000000001,"5min":1.3520000000000001,"15min":1.3120000000000001},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.8,"5min":1.8,"15min":2.3780000000000001},"last":1.258},{"interface":"front","average":{"1min":1.3009999999999999,"5min":1.3560000000000001,"15min":1.288},"min":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.70799999999999996},"max":{"1min":1.611,"5min":1.8819999999999999,"15min":1.984},"last":1.3109999999999999}]},{"osd":7,"last update":"Thu Oct 26 01:42:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3180000000000001,"5min":1.3460000000000001,"15min":1.2869999999999999},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.772,"5min":1.835,"15min":2.6930000000000001},"last":1.409},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.4359999999999999,"15min":1.381},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.7,"5min":1.903,"15min":2.0190000000000001},"last":1.645}]}]},{"osd":0,"up_from":13,"seq":55834575010,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1564,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193747,"statfs":{"total":200000000,"available":198397545,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Oct 26 01:42:57 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.79100000000000004,"15min":0.88400000000000001},"min":{"1min":0.52800000000000002,"5min":0.32500000000000001,"15min":0.32000000000000001},"max":{"1min":1.048,"5min":1.494,"15min":19.425000000000001},"last":0.79200000000000004},{"interface":"front","average":{"1min":1.0389999999999999,"5min":0.99399999999999999,"15min":1.024},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.478,"5min":1.492,"15min":15.162000000000001},"last":1.044}]},{"osd":2,"last update":"Thu Oct 26 01:42:57 2023","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.0469999999999999,"15min":1.151},"min":{"1min":0.51000000000000001,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.4410000000000001,"5min":1.556,"15min":28.439},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.93799999999999994,"15min":0.91900000000000004},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.37},"max":{"1min":1.365,"5min":1.851,"15min":4.9459999999999997},"last":1.302}]},{"osd":3,"last update":"Thu Oct 26 01:42:57 2023","interfaces":[{"interface":"back","average":{"1min":1.0900000000000001,"5min":1.131,"15min":1.0900000000000001},"min":{"1min":0.629,"5min":0.56999999999999995,"15min":0.44600000000000001},"max":{"1min":1.597,"5min":1.7509999999999999,"15min":4.9800000000000004},"last":1.4179999999999999},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.216,"15min":1.169},"min":{"1min":0.69799999999999995,"5min":0.63500000000000001,"15min":0.438},"max":{"1min":1.661,"5min":1.774,"15min":5.016},"last":1.48}]},{"osd":4,"last update":"Thu Oct 26 01:42:57 2023","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":1.1439999999999999,"15min":1.252},"min":{"1min":0.875,"5min":0.76800000000000002,"15min":0.52900000000000003},"max":{"1min":1.5569999999999999,"5min":1.5840000000000001,"15min":19.329000000000001},"last":1.0940000000000001},{"interface":"front","average":{"1min":1.2210000000000001,"5min":1.2509999999999999,"15min":1.2749999999999999},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.67600000000000005},"max":{"1min":1.48,"5min":1.7270000000000001,"15min":5.5229999999999997},"last":1.1850000000000001}]},{"osd":5,"last update":"Thu Oct 26 01:42:57 2023","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.4279999999999999,"15min":1.4219999999999999},"min":{"1min":1.0189999999999999,"5min":1,"15min":0.64900000000000002},"max":{"1min":1.8660000000000001,"5min":1.8660000000000001,"15min":5.6360000000000001},"last":1.5960000000000001},{"interface":"front","average":{"1min":1.306,"5min":1.2829999999999999,"15min":1.371},"min":{"1min":0.91200000000000003,"5min":0.80200000000000005,"15min":0.57499999999999996},"max":{"1min":1.6259999999999999,"5min":1.742,"15min":19.384},"last":1.288}]},{"osd":6,"last update":"Thu Oct 26 01:42:57 2023","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.3180000000000001,"15min":1.347},"min":{"1min":0.83999999999999997,"5min":0.79300000000000004,"15min":0.73799999999999999},"max":{"1min":1.6899999999999999,"5min":1.8999999999999999,"15min":8.2609999999999992},"last":1.226},{"interface":"front","average":{"1min":1.321,"5min":1.3600000000000001,"15min":1.3939999999999999},"min":{"1min":0.746,"5min":0.746,"15min":0.746},"max":{"1min":1.6000000000000001,"5min":1.9399999999999999,"15min":11.849},"last":1.361}]},{"osd":7,"last update":"Thu Oct 26 01:42:57 2023","interfaces":[{"interface":"back","average":{"1min":1.458,"5min":1.4870000000000001,"15min":1.4730000000000001},"min":{"1min":1.071,"5min":0.999,"15min":0.73699999999999999},"max":{"1min":1.9350000000000001,"5min":1.9379999999999999,"15min":5.6689999999999996},"last":1.8520000000000001},{"interface":"front","average":{"1min":1.4139999999999999,"5min":1.411,"15min":1.4870000000000001},"min":{"1min":1.079,"5min":0.81299999999999994,"15min":0.69399999999999995},"max":{"1min":1.6339999999999999,"5min":1.8149999999999999,"15min":19.245999999999999},"last":1.337}]}]}],"pool_statfs":[]}} 2023-10-26T01:43:54.761 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-26T01:43:55.023 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:55.021+0000 7f2c69733640 1 -- 172.21.15.169:0/3292435174 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c641592d0 msgr2=0x7f2c640aa3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:55.023 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:55.021+0000 7f2c69733640 1 --2- 172.21.15.169:0/3292435174 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c641592d0 0x7f2c640aa3d0 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7f2c640acdf0 tx=0x7f2c5402bda0 comp rx=0 tx=0).stop 2023-10-26T01:43:55.023 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:55.021+0000 7f2c69733640 1 -- 172.21.15.169:0/3292435174 shutdown_connections 2023-10-26T01:43:55.023 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:55.021+0000 7f2c69733640 1 --2- 172.21.15.169:0/3292435174 >> v2:172.21.15.170:3300/0 conn(0x7f2c640cde00 0x7f2c640cbac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:55.024 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:55.021+0000 7f2c69733640 1 --2- 172.21.15.169:0/3292435174 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c641592d0 0x7f2c640aa3d0 unknown :-1 s=CLOSED pgs=1850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:55.024 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:55.021+0000 7f2c69733640 1 -- 172.21.15.169:0/3292435174 >> 172.21.15.169:0/3292435174 conn(0x7f2c64069930 msgr2=0x7f2c64069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:55.024 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:55.021+0000 7f2c69733640 1 -- 172.21.15.169:0/3292435174 shutdown_connections 2023-10-26T01:43:55.024 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:55.021+0000 7f2c69733640 1 -- 172.21.15.169:0/3292435174 wait complete. 2023-10-26T01:43:55.025 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:55.023+0000 7f2c69733640 1 Processor -- start 2023-10-26T01:43:55.025 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:55.023+0000 7f2c69733640 1 -- start start 2023-10-26T01:43:55.026 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:55.024+0000 7f2c69733640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c640ac930 0x7f2c6414e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:55.026 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:55.024+0000 7f2c69733640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2c640cde00 0x7f2c640aeec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:55.027 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:55.024+0000 7f2c69733640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2c641592d0 0x7f2c6415afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:55.027 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:55.024+0000 7f2c69733640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c6414ecb0 con 0x7f2c640ac930 2023-10-26T01:43:55.027 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:55.024+0000 7f2c69733640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c640b0140 con 0x7f2c641592d0 2023-10-26T01:43:55.027 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:55.024+0000 7f2c69733640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c640af8a0 con 0x7f2c640cde00 2023-10-26T01:43:55.027 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:55.024+0000 7f2c62ffd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c640ac930 0x7f2c6414e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:55.028 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:55.024+0000 7f2c62ffd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c640ac930 0x7f2c6414e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:46178/0 (socket says 172.21.15.169:46178) 2023-10-26T01:43:55.028 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:55.024+0000 7f2c62ffd640 1 -- 172.21.15.169:0/932241974 learned_addr learned my addr 172.21.15.169:0/932241974 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:43:55.028 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:55.025+0000 7f2c62ffd640 1 -- 172.21.15.169:0/932241974 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2c640cde00 msgr2=0x7f2c640aeec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:55.028 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:55.025+0000 7f2c62ffd640 1 --2- 172.21.15.169:0/932241974 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2c640cde00 0x7f2c640aeec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:55.028 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:55.025+0000 7f2c62ffd640 1 -- 172.21.15.169:0/932241974 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2c641592d0 msgr2=0x7f2c6415afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:55.029 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:55.025+0000 7f2c62ffd640 1 --2- 172.21.15.169:0/932241974 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2c641592d0 0x7f2c6415afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:55.029 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:55.025+0000 7f2c62ffd640 1 -- 172.21.15.169:0/932241974 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c54009560 con 0x7f2c640ac930 2023-10-26T01:43:55.029 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:55.025+0000 7f2c62ffd640 1 --2- 172.21.15.169:0/932241974 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c640ac930 0x7f2c6414e770 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f2c54009850 tx=0x7f2c5402dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:55.030 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:55.025+0000 7f2c5b7fe640 1 -- 172.21.15.169:0/932241974 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c54005650 con 0x7f2c640ac930 2023-10-26T01:43:55.030 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:55.025+0000 7f2c5b7fe640 1 -- 172.21.15.169:0/932241974 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2c540038f0 con 0x7f2c640ac930 2023-10-26T01:43:55.030 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:55.025+0000 7f2c5b7fe640 1 -- 172.21.15.169:0/932241974 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c54003ce0 con 0x7f2c640ac930 2023-10-26T01:43:55.031 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:55.025+0000 7f2c69733640 1 -- 172.21.15.169:0/932241974 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c640afb80 con 0x7f2c640ac930 2023-10-26T01:43:55.031 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:55.025+0000 7f2c69733640 1 -- 172.21.15.169:0/932241974 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c641527c0 con 0x7f2c640ac930 2023-10-26T01:43:55.031 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:55.026+0000 7f2c69733640 1 -- 172.21.15.169:0/932241974 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c64152620 con 0x7f2c640ac930 2023-10-26T01:43:55.031 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:55.028+0000 7f2c5b7fe640 1 -- 172.21.15.169:0/932241974 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f2c5403e450 con 0x7f2c640ac930 2023-10-26T01:43:55.032 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:55.029+0000 7f2c5b7fe640 1 --2- 172.21.15.169:0/932241974 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2c4406ae40 0x7f2c4406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:55.032 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:55.029+0000 7f2c5b7fe640 1 -- 172.21.15.169:0/932241974 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2c540abfd0 con 0x7f2c640ac930 2023-10-26T01:43:55.032 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:55.030+0000 7f2c5bfff640 1 --2- 172.21.15.169:0/932241974 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2c4406ae40 0x7f2c4406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:55.033 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:55.031+0000 7f2c5bfff640 1 --2- 172.21.15.169:0/932241974 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2c4406ae40 0x7f2c4406d300 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c000f60 tx=0x7f2c4c0079b0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:55.035 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:55.034+0000 7f2c5b7fe640 1 -- 172.21.15.169:0/932241974 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2c64152620 con 0x7f2c640ac930 2023-10-26T01:43:55.221 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:55.219+0000 7f2c69733640 1 -- 172.21.15.169:0/932241974 --> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2c64152620 con 0x7f2c4406ae40 2023-10-26T01:43:55.243 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:55.242+0000 7f2c5b7fe640 1 -- 172.21.15.169:0/932241974 <== mgr.4112 v2:172.21.15.170:6832/1187156918 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+282400 (secure 0 0 0) 0x7f2c64152620 con 0x7f2c4406ae40 2023-10-26T01:43:55.247 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:43:55.249 INFO:teuthology.orchestra.run.smithi169.stderr:dumped all 2023-10-26T01:43:55.250 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:55.248+0000 7f2c69733640 1 -- 172.21.15.169:0/932241974 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2c4406ae40 msgr2=0x7f2c4406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:55.251 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:55.248+0000 7f2c69733640 1 --2- 172.21.15.169:0/932241974 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2c4406ae40 0x7f2c4406d300 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c000f60 tx=0x7f2c4c0079b0 comp rx=0 tx=0).stop 2023-10-26T01:43:55.251 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:55.248+0000 7f2c69733640 1 -- 172.21.15.169:0/932241974 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c640ac930 msgr2=0x7f2c6414e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:55.251 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:55.248+0000 7f2c69733640 1 --2- 172.21.15.169:0/932241974 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c640ac930 0x7f2c6414e770 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f2c54009850 tx=0x7f2c5402dd70 comp rx=0 tx=0).stop 2023-10-26T01:43:55.251 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:55.249+0000 7f2c69733640 1 -- 172.21.15.169:0/932241974 shutdown_connections 2023-10-26T01:43:55.251 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:55.249+0000 7f2c69733640 1 --2- 172.21.15.169:0/932241974 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2c4406ae40 0x7f2c4406d300 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:55.252 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:55.249+0000 7f2c69733640 1 --2- 172.21.15.169:0/932241974 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2c641592d0 0x7f2c6415afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:55.252 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:55.249+0000 7f2c69733640 1 --2- 172.21.15.169:0/932241974 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2c640cde00 0x7f2c640aeec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:55.252 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:55.249+0000 7f2c69733640 1 --2- 172.21.15.169:0/932241974 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c640ac930 0x7f2c6414e770 unknown :-1 s=CLOSED pgs=1851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:55.252 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:55.249+0000 7f2c69733640 1 -- 172.21.15.169:0/932241974 >> 172.21.15.169:0/932241974 conn(0x7f2c64069930 msgr2=0x7f2c640716f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:55.252 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:55.249+0000 7f2c69733640 1 -- 172.21.15.169:0/932241974 shutdown_connections 2023-10-26T01:43:55.252 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:55.249+0000 7f2c69733640 1 -- 172.21.15.169:0/932241974 wait complete. 2023-10-26T01:43:55.269 INFO:teuthology.orchestra.run.smithi169.stdout:{"pg_ready":true,"pg_map":{"version":577,"stamp":"2023-10-26T01:43:54.420532+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":78,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":9086,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1553413,"statfs":{"total":1600000000,"available":1590695388,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002337"},"pg_stats":[{"pgid":"23.3f","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.308770+0000","last_change":"2023-10-26T01:43:50.308500+0000","last_active":"2023-10-26T01:43:50.308770+0000","last_peered":"2023-10-26T01:43:50.308770+0000","last_clean":"2023-10-26T01:43:50.308770+0000","last_became_active":"2023-10-26T01:43:50.308142+0000","last_became_peered":"2023-10-26T01:43:50.308142+0000","last_unstale":"2023-10-26T01:43:50.308770+0000","last_undegraded":"2023-10-26T01:43:50.308770+0000","last_fullsized":"2023-10-26T01:43:50.308770+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:30:34.655863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.3e","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.311684+0000","last_change":"2023-10-26T01:43:50.310257+0000","last_active":"2023-10-26T01:43:50.311684+0000","last_peered":"2023-10-26T01:43:50.311684+0000","last_clean":"2023-10-26T01:43:50.311684+0000","last_became_active":"2023-10-26T01:43:50.309527+0000","last_became_peered":"2023-10-26T01:43:50.309527+0000","last_unstale":"2023-10-26T01:43:50.311684+0000","last_undegraded":"2023-10-26T01:43:50.311684+0000","last_fullsized":"2023-10-26T01:43:50.311684+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:37:58.399376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3d","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.309657+0000","last_change":"2023-10-26T01:43:50.308312+0000","last_active":"2023-10-26T01:43:50.309657+0000","last_peered":"2023-10-26T01:43:50.309657+0000","last_clean":"2023-10-26T01:43:50.309657+0000","last_became_active":"2023-10-26T01:43:50.307837+0000","last_became_peered":"2023-10-26T01:43:50.307837+0000","last_unstale":"2023-10-26T01:43:50.309657+0000","last_undegraded":"2023-10-26T01:43:50.309657+0000","last_fullsized":"2023-10-26T01:43:50.309657+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:07:17.983568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.3c","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.322982+0000","last_change":"2023-10-26T01:43:50.322354+0000","last_active":"2023-10-26T01:43:50.322982+0000","last_peered":"2023-10-26T01:43:50.322982+0000","last_clean":"2023-10-26T01:43:50.322982+0000","last_became_active":"2023-10-26T01:43:50.322091+0000","last_became_peered":"2023-10-26T01:43:50.322091+0000","last_unstale":"2023-10-26T01:43:50.322982+0000","last_undegraded":"2023-10-26T01:43:50.322982+0000","last_fullsized":"2023-10-26T01:43:50.322982+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:35:41.989570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3b","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.319844+0000","last_change":"2023-10-26T01:43:50.319762+0000","last_active":"2023-10-26T01:43:50.319844+0000","last_peered":"2023-10-26T01:43:50.319844+0000","last_clean":"2023-10-26T01:43:50.319844+0000","last_became_active":"2023-10-26T01:43:50.318893+0000","last_became_peered":"2023-10-26T01:43:50.318893+0000","last_unstale":"2023-10-26T01:43:50.319844+0000","last_undegraded":"2023-10-26T01:43:50.319844+0000","last_fullsized":"2023-10-26T01:43:50.319844+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:05:12.398879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3a","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.318840+0000","last_change":"2023-10-26T01:43:50.318313+0000","last_active":"2023-10-26T01:43:50.318840+0000","last_peered":"2023-10-26T01:43:50.318840+0000","last_clean":"2023-10-26T01:43:50.318840+0000","last_became_active":"2023-10-26T01:43:50.317326+0000","last_became_peered":"2023-10-26T01:43:50.317326+0000","last_unstale":"2023-10-26T01:43:50.318840+0000","last_undegraded":"2023-10-26T01:43:50.318840+0000","last_fullsized":"2023-10-26T01:43:50.318840+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:30:41.575163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.39","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.321750+0000","last_change":"2023-10-26T01:43:50.321636+0000","last_active":"2023-10-26T01:43:50.321750+0000","last_peered":"2023-10-26T01:43:50.321750+0000","last_clean":"2023-10-26T01:43:50.321750+0000","last_became_active":"2023-10-26T01:43:50.321121+0000","last_became_peered":"2023-10-26T01:43:50.321121+0000","last_unstale":"2023-10-26T01:43:50.321750+0000","last_undegraded":"2023-10-26T01:43:50.321750+0000","last_fullsized":"2023-10-26T01:43:50.321750+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T01:56:12.485221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.38","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.311762+0000","last_change":"2023-10-26T01:43:50.311066+0000","last_active":"2023-10-26T01:43:50.311762+0000","last_peered":"2023-10-26T01:43:50.311762+0000","last_clean":"2023-10-26T01:43:50.311762+0000","last_became_active":"2023-10-26T01:43:50.310772+0000","last_became_peered":"2023-10-26T01:43:50.310772+0000","last_unstale":"2023-10-26T01:43:50.311762+0000","last_undegraded":"2023-10-26T01:43:50.311762+0000","last_fullsized":"2023-10-26T01:43:50.311762+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:46:02.839349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.37","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.317642+0000","last_change":"2023-10-26T01:43:50.314888+0000","last_active":"2023-10-26T01:43:50.317642+0000","last_peered":"2023-10-26T01:43:50.317642+0000","last_clean":"2023-10-26T01:43:50.317642+0000","last_became_active":"2023-10-26T01:43:50.314609+0000","last_became_peered":"2023-10-26T01:43:50.314609+0000","last_unstale":"2023-10-26T01:43:50.317642+0000","last_undegraded":"2023-10-26T01:43:50.317642+0000","last_fullsized":"2023-10-26T01:43:50.317642+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:31:00.739794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.36","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.317684+0000","last_change":"2023-10-26T01:43:50.316577+0000","last_active":"2023-10-26T01:43:50.317684+0000","last_peered":"2023-10-26T01:43:50.317684+0000","last_clean":"2023-10-26T01:43:50.317684+0000","last_became_active":"2023-10-26T01:43:50.314127+0000","last_became_peered":"2023-10-26T01:43:50.314127+0000","last_unstale":"2023-10-26T01:43:50.317684+0000","last_undegraded":"2023-10-26T01:43:50.317684+0000","last_fullsized":"2023-10-26T01:43:50.317684+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:20:13.363860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.35","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.320405+0000","last_change":"2023-10-26T01:43:50.320327+0000","last_active":"2023-10-26T01:43:50.320405+0000","last_peered":"2023-10-26T01:43:50.320405+0000","last_clean":"2023-10-26T01:43:50.320405+0000","last_became_active":"2023-10-26T01:43:50.320045+0000","last_became_peered":"2023-10-26T01:43:50.320045+0000","last_unstale":"2023-10-26T01:43:50.320405+0000","last_undegraded":"2023-10-26T01:43:50.320405+0000","last_fullsized":"2023-10-26T01:43:50.320405+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:45:38.848472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.34","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.311707+0000","last_change":"2023-10-26T01:43:50.310463+0000","last_active":"2023-10-26T01:43:50.311707+0000","last_peered":"2023-10-26T01:43:50.311707+0000","last_clean":"2023-10-26T01:43:50.311707+0000","last_became_active":"2023-10-26T01:43:50.309959+0000","last_became_peered":"2023-10-26T01:43:50.309959+0000","last_unstale":"2023-10-26T01:43:50.311707+0000","last_undegraded":"2023-10-26T01:43:50.311707+0000","last_fullsized":"2023-10-26T01:43:50.311707+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:14:09.503559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.33","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.314172+0000","last_change":"2023-10-26T01:43:50.311920+0000","last_active":"2023-10-26T01:43:50.314172+0000","last_peered":"2023-10-26T01:43:50.314172+0000","last_clean":"2023-10-26T01:43:50.314172+0000","last_became_active":"2023-10-26T01:43:50.311207+0000","last_became_peered":"2023-10-26T01:43:50.311207+0000","last_unstale":"2023-10-26T01:43:50.314172+0000","last_undegraded":"2023-10-26T01:43:50.314172+0000","last_fullsized":"2023-10-26T01:43:50.314172+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:05:49.047185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.32","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.314157+0000","last_change":"2023-10-26T01:43:50.311679+0000","last_active":"2023-10-26T01:43:50.314157+0000","last_peered":"2023-10-26T01:43:50.314157+0000","last_clean":"2023-10-26T01:43:50.314157+0000","last_became_active":"2023-10-26T01:43:50.311405+0000","last_became_peered":"2023-10-26T01:43:50.311405+0000","last_unstale":"2023-10-26T01:43:50.314157+0000","last_undegraded":"2023-10-26T01:43:50.314157+0000","last_fullsized":"2023-10-26T01:43:50.314157+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:24:24.024882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.31","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.312073+0000","last_change":"2023-10-26T01:43:50.311517+0000","last_active":"2023-10-26T01:43:50.312073+0000","last_peered":"2023-10-26T01:43:50.312073+0000","last_clean":"2023-10-26T01:43:50.312073+0000","last_became_active":"2023-10-26T01:43:50.310199+0000","last_became_peered":"2023-10-26T01:43:50.310199+0000","last_unstale":"2023-10-26T01:43:50.312073+0000","last_undegraded":"2023-10-26T01:43:50.312073+0000","last_fullsized":"2023-10-26T01:43:50.312073+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:12:29.214521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.30","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.312908+0000","last_change":"2023-10-26T01:43:50.312478+0000","last_active":"2023-10-26T01:43:50.312908+0000","last_peered":"2023-10-26T01:43:50.312908+0000","last_clean":"2023-10-26T01:43:50.312908+0000","last_became_active":"2023-10-26T01:43:50.312231+0000","last_became_peered":"2023-10-26T01:43:50.312231+0000","last_unstale":"2023-10-26T01:43:50.312908+0000","last_undegraded":"2023-10-26T01:43:50.312908+0000","last_fullsized":"2023-10-26T01:43:50.312908+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:22:45.891664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.2f","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.317854+0000","last_change":"2023-10-26T01:43:50.317655+0000","last_active":"2023-10-26T01:43:50.317854+0000","last_peered":"2023-10-26T01:43:50.317854+0000","last_clean":"2023-10-26T01:43:50.317854+0000","last_became_active":"2023-10-26T01:43:50.317043+0000","last_became_peered":"2023-10-26T01:43:50.317043+0000","last_unstale":"2023-10-26T01:43:50.317854+0000","last_undegraded":"2023-10-26T01:43:50.317854+0000","last_fullsized":"2023-10-26T01:43:50.317854+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:17:55.522661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2e","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.322382+0000","last_change":"2023-10-26T01:43:50.322147+0000","last_active":"2023-10-26T01:43:50.322382+0000","last_peered":"2023-10-26T01:43:50.322382+0000","last_clean":"2023-10-26T01:43:50.322382+0000","last_became_active":"2023-10-26T01:43:50.321724+0000","last_became_peered":"2023-10-26T01:43:50.321724+0000","last_unstale":"2023-10-26T01:43:50.322382+0000","last_undegraded":"2023-10-26T01:43:50.322382+0000","last_fullsized":"2023-10-26T01:43:50.322382+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:22:26.970184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2d","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.319888+0000","last_change":"2023-10-26T01:43:50.319709+0000","last_active":"2023-10-26T01:43:50.319888+0000","last_peered":"2023-10-26T01:43:50.319888+0000","last_clean":"2023-10-26T01:43:50.319888+0000","last_became_active":"2023-10-26T01:43:50.319444+0000","last_became_peered":"2023-10-26T01:43:50.319444+0000","last_unstale":"2023-10-26T01:43:50.319888+0000","last_undegraded":"2023-10-26T01:43:50.319888+0000","last_fullsized":"2023-10-26T01:43:50.319888+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:07:17.983568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2c","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.316685+0000","last_change":"2023-10-26T01:43:50.316602+0000","last_active":"2023-10-26T01:43:50.316685+0000","last_peered":"2023-10-26T01:43:50.316685+0000","last_clean":"2023-10-26T01:43:50.316685+0000","last_became_active":"2023-10-26T01:43:50.315889+0000","last_became_peered":"2023-10-26T01:43:50.315889+0000","last_unstale":"2023-10-26T01:43:50.316685+0000","last_undegraded":"2023-10-26T01:43:50.316685+0000","last_fullsized":"2023-10-26T01:43:50.316685+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:10:53.556092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.2b","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.317832+0000","last_change":"2023-10-26T01:43:50.317596+0000","last_active":"2023-10-26T01:43:50.317832+0000","last_peered":"2023-10-26T01:43:50.317832+0000","last_clean":"2023-10-26T01:43:50.317832+0000","last_became_active":"2023-10-26T01:43:50.316967+0000","last_became_peered":"2023-10-26T01:43:50.316967+0000","last_unstale":"2023-10-26T01:43:50.317832+0000","last_undegraded":"2023-10-26T01:43:50.317832+0000","last_fullsized":"2023-10-26T01:43:50.317832+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:24:03.603238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2a","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.319668+0000","last_change":"2023-10-26T01:43:50.319586+0000","last_active":"2023-10-26T01:43:50.319668+0000","last_peered":"2023-10-26T01:43:50.319668+0000","last_clean":"2023-10-26T01:43:50.319668+0000","last_became_active":"2023-10-26T01:43:50.319108+0000","last_became_peered":"2023-10-26T01:43:50.319108+0000","last_unstale":"2023-10-26T01:43:50.319668+0000","last_undegraded":"2023-10-26T01:43:50.319668+0000","last_fullsized":"2023-10-26T01:43:50.319668+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:39:02.127957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.29","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.316493+0000","last_change":"2023-10-26T01:43:50.316179+0000","last_active":"2023-10-26T01:43:50.316493+0000","last_peered":"2023-10-26T01:43:50.316493+0000","last_clean":"2023-10-26T01:43:50.316493+0000","last_became_active":"2023-10-26T01:43:50.315941+0000","last_became_peered":"2023-10-26T01:43:50.315941+0000","last_unstale":"2023-10-26T01:43:50.316493+0000","last_undegraded":"2023-10-26T01:43:50.316493+0000","last_fullsized":"2023-10-26T01:43:50.316493+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:46:23.366905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.28","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.309174+0000","last_change":"2023-10-26T01:43:50.309071+0000","last_active":"2023-10-26T01:43:50.309174+0000","last_peered":"2023-10-26T01:43:50.309174+0000","last_clean":"2023-10-26T01:43:50.309174+0000","last_became_active":"2023-10-26T01:43:50.308627+0000","last_became_peered":"2023-10-26T01:43:50.308627+0000","last_unstale":"2023-10-26T01:43:50.309174+0000","last_undegraded":"2023-10-26T01:43:50.309174+0000","last_fullsized":"2023-10-26T01:43:50.309174+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:40:14.136503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.27","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.317024+0000","last_change":"2023-10-26T01:43:50.316772+0000","last_active":"2023-10-26T01:43:50.317024+0000","last_peered":"2023-10-26T01:43:50.317024+0000","last_clean":"2023-10-26T01:43:50.317024+0000","last_became_active":"2023-10-26T01:43:50.316273+0000","last_became_peered":"2023-10-26T01:43:50.316273+0000","last_unstale":"2023-10-26T01:43:50.317024+0000","last_undegraded":"2023-10-26T01:43:50.317024+0000","last_fullsized":"2023-10-26T01:43:50.317024+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:44:08.964253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.26","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.314452+0000","last_change":"2023-10-26T01:43:50.314358+0000","last_active":"2023-10-26T01:43:50.314452+0000","last_peered":"2023-10-26T01:43:50.314452+0000","last_clean":"2023-10-26T01:43:50.314452+0000","last_became_active":"2023-10-26T01:43:50.314040+0000","last_became_peered":"2023-10-26T01:43:50.314040+0000","last_unstale":"2023-10-26T01:43:50.314452+0000","last_undegraded":"2023-10-26T01:43:50.314452+0000","last_fullsized":"2023-10-26T01:43:50.314452+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:26:32.296617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.25","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.314139+0000","last_change":"2023-10-26T01:43:50.310537+0000","last_active":"2023-10-26T01:43:50.314139+0000","last_peered":"2023-10-26T01:43:50.314139+0000","last_clean":"2023-10-26T01:43:50.314139+0000","last_became_active":"2023-10-26T01:43:50.310244+0000","last_became_peered":"2023-10-26T01:43:50.310244+0000","last_unstale":"2023-10-26T01:43:50.314139+0000","last_undegraded":"2023-10-26T01:43:50.314139+0000","last_fullsized":"2023-10-26T01:43:50.314139+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:22:13.378905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.24","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.312298+0000","last_change":"2023-10-26T01:43:50.312178+0000","last_active":"2023-10-26T01:43:50.312298+0000","last_peered":"2023-10-26T01:43:50.312298+0000","last_clean":"2023-10-26T01:43:50.312298+0000","last_became_active":"2023-10-26T01:43:50.310962+0000","last_became_peered":"2023-10-26T01:43:50.310962+0000","last_unstale":"2023-10-26T01:43:50.312298+0000","last_undegraded":"2023-10-26T01:43:50.312298+0000","last_fullsized":"2023-10-26T01:43:50.312298+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:27:24.942966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.23","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.309679+0000","last_change":"2023-10-26T01:43:50.308620+0000","last_active":"2023-10-26T01:43:50.309679+0000","last_peered":"2023-10-26T01:43:50.309679+0000","last_clean":"2023-10-26T01:43:50.309679+0000","last_became_active":"2023-10-26T01:43:50.307934+0000","last_became_peered":"2023-10-26T01:43:50.307934+0000","last_unstale":"2023-10-26T01:43:50.309679+0000","last_undegraded":"2023-10-26T01:43:50.309679+0000","last_fullsized":"2023-10-26T01:43:50.309679+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:59:34.197770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.22","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.295358+0000","last_change":"2023-10-26T01:43:50.295164+0000","last_active":"2023-10-26T01:43:50.295358+0000","last_peered":"2023-10-26T01:43:50.295358+0000","last_clean":"2023-10-26T01:43:50.295358+0000","last_became_active":"2023-10-26T01:43:50.293769+0000","last_became_peered":"2023-10-26T01:43:50.293769+0000","last_unstale":"2023-10-26T01:43:50.295358+0000","last_undegraded":"2023-10-26T01:43:50.295358+0000","last_fullsized":"2023-10-26T01:43:50.295358+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:28:36.730001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.21","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.317881+0000","last_change":"2023-10-26T01:43:50.317761+0000","last_active":"2023-10-26T01:43:50.317881+0000","last_peered":"2023-10-26T01:43:50.317881+0000","last_clean":"2023-10-26T01:43:50.317881+0000","last_became_active":"2023-10-26T01:43:50.316880+0000","last_became_peered":"2023-10-26T01:43:50.316880+0000","last_unstale":"2023-10-26T01:43:50.317881+0000","last_undegraded":"2023-10-26T01:43:50.317881+0000","last_fullsized":"2023-10-26T01:43:50.317881+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T01:56:12.485221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.20","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.321512+0000","last_change":"2023-10-26T01:43:50.319722+0000","last_active":"2023-10-26T01:43:50.321512+0000","last_peered":"2023-10-26T01:43:50.321512+0000","last_clean":"2023-10-26T01:43:50.321512+0000","last_became_active":"2023-10-26T01:43:50.319490+0000","last_became_peered":"2023-10-26T01:43:50.319490+0000","last_unstale":"2023-10-26T01:43:50.321512+0000","last_undegraded":"2023-10-26T01:43:50.321512+0000","last_fullsized":"2023-10-26T01:43:50.321512+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:05:12.398879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.314889+0000","last_change":"2023-10-26T01:43:50.314795+0000","last_active":"2023-10-26T01:43:50.314889+0000","last_peered":"2023-10-26T01:43:50.314889+0000","last_clean":"2023-10-26T01:43:50.314889+0000","last_became_active":"2023-10-26T01:43:50.314264+0000","last_became_peered":"2023-10-26T01:43:50.314264+0000","last_unstale":"2023-10-26T01:43:50.314889+0000","last_undegraded":"2023-10-26T01:43:50.314889+0000","last_fullsized":"2023-10-26T01:43:50.314889+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:28:36.730001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1f","version":"181'4","reported_seq":15,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.328923+0000","last_change":"2023-10-26T01:43:49.263896+0000","last_active":"2023-10-26T01:43:50.328923+0000","last_peered":"2023-10-26T01:43:50.328923+0000","last_clean":"2023-10-26T01:43:50.328923+0000","last_became_active":"2023-10-26T01:43:49.263284+0000","last_became_peered":"2023-10-26T01:43:49.263284+0000","last_unstale":"2023-10-26T01:43:50.328923+0000","last_undegraded":"2023-10-26T01:43:50.328923+0000","last_fullsized":"2023-10-26T01:43:50.328923+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_clean_scrub_stamp":"2023-10-26T01:43:48.232806+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:06:13.311992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.297013+0000","last_change":"2023-10-26T01:43:50.296698+0000","last_active":"2023-10-26T01:43:50.297013+0000","last_peered":"2023-10-26T01:43:50.297013+0000","last_clean":"2023-10-26T01:43:50.297013+0000","last_became_active":"2023-10-26T01:43:50.296065+0000","last_became_peered":"2023-10-26T01:43:50.296065+0000","last_unstale":"2023-10-26T01:43:50.297013+0000","last_undegraded":"2023-10-26T01:43:50.297013+0000","last_fullsized":"2023-10-26T01:43:50.297013+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:22:45.891664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1e","version":"181'1","reported_seq":12,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.329965+0000","last_change":"2023-10-26T01:43:49.275818+0000","last_active":"2023-10-26T01:43:50.329965+0000","last_peered":"2023-10-26T01:43:50.329965+0000","last_clean":"2023-10-26T01:43:50.329965+0000","last_became_active":"2023-10-26T01:43:49.275556+0000","last_became_peered":"2023-10-26T01:43:49.275556+0000","last_unstale":"2023-10-26T01:43:50.329965+0000","last_undegraded":"2023-10-26T01:43:50.329965+0000","last_fullsized":"2023-10-26T01:43:50.329965+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_clean_scrub_stamp":"2023-10-26T01:43:48.232806+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:08:08.571789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.318863+0000","last_change":"2023-10-26T01:43:50.318624+0000","last_active":"2023-10-26T01:43:50.318863+0000","last_peered":"2023-10-26T01:43:50.318863+0000","last_clean":"2023-10-26T01:43:50.318863+0000","last_became_active":"2023-10-26T01:43:50.317875+0000","last_became_peered":"2023-10-26T01:43:50.317875+0000","last_unstale":"2023-10-26T01:43:50.318863+0000","last_undegraded":"2023-10-26T01:43:50.318863+0000","last_fullsized":"2023-10-26T01:43:50.318863+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:11:45.355135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1d","version":"181'2","reported_seq":13,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.329000+0000","last_change":"2023-10-26T01:43:49.271734+0000","last_active":"2023-10-26T01:43:50.329000+0000","last_peered":"2023-10-26T01:43:50.329000+0000","last_clean":"2023-10-26T01:43:50.329000+0000","last_became_active":"2023-10-26T01:43:49.271511+0000","last_became_peered":"2023-10-26T01:43:49.271511+0000","last_unstale":"2023-10-26T01:43:50.329000+0000","last_undegraded":"2023-10-26T01:43:50.329000+0000","last_fullsized":"2023-10-26T01:43:50.329000+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_clean_scrub_stamp":"2023-10-26T01:43:48.232806+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:28:47.267845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.320688+0000","last_change":"2023-10-26T01:43:50.320568+0000","last_active":"2023-10-26T01:43:50.320688+0000","last_peered":"2023-10-26T01:43:50.320688+0000","last_clean":"2023-10-26T01:43:50.320688+0000","last_became_active":"2023-10-26T01:43:50.320115+0000","last_became_peered":"2023-10-26T01:43:50.320115+0000","last_unstale":"2023-10-26T01:43:50.320688+0000","last_undegraded":"2023-10-26T01:43:50.320688+0000","last_fullsized":"2023-10-26T01:43:50.320688+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:46:23.366905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1c","version":"181'2","reported_seq":13,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.329510+0000","last_change":"2023-10-26T01:43:49.271198+0000","last_active":"2023-10-26T01:43:50.329510+0000","last_peered":"2023-10-26T01:43:50.329510+0000","last_clean":"2023-10-26T01:43:50.329510+0000","last_became_active":"2023-10-26T01:43:49.270976+0000","last_became_peered":"2023-10-26T01:43:49.270976+0000","last_unstale":"2023-10-26T01:43:50.329510+0000","last_undegraded":"2023-10-26T01:43:50.329510+0000","last_fullsized":"2023-10-26T01:43:50.329510+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_clean_scrub_stamp":"2023-10-26T01:43:48.232806+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:34:23.052043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.311863+0000","last_change":"2023-10-26T01:43:50.309347+0000","last_active":"2023-10-26T01:43:50.311863+0000","last_peered":"2023-10-26T01:43:50.311863+0000","last_clean":"2023-10-26T01:43:50.311863+0000","last_became_active":"2023-10-26T01:43:50.308714+0000","last_became_peered":"2023-10-26T01:43:50.308714+0000","last_unstale":"2023-10-26T01:43:50.311863+0000","last_undegraded":"2023-10-26T01:43:50.311863+0000","last_fullsized":"2023-10-26T01:43:50.311863+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:18:13.356291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1b","version":"181'1","reported_seq":12,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.330571+0000","last_change":"2023-10-26T01:43:49.268691+0000","last_active":"2023-10-26T01:43:50.330571+0000","last_peered":"2023-10-26T01:43:50.330571+0000","last_clean":"2023-10-26T01:43:50.330571+0000","last_became_active":"2023-10-26T01:43:49.268207+0000","last_became_peered":"2023-10-26T01:43:49.268207+0000","last_unstale":"2023-10-26T01:43:50.330571+0000","last_undegraded":"2023-10-26T01:43:50.330571+0000","last_fullsized":"2023-10-26T01:43:50.330571+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_clean_scrub_stamp":"2023-10-26T01:43:48.232806+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:30:40.570275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.314094+0000","last_change":"2023-10-26T01:43:50.313380+0000","last_active":"2023-10-26T01:43:50.314094+0000","last_peered":"2023-10-26T01:43:50.314094+0000","last_clean":"2023-10-26T01:43:50.314094+0000","last_became_active":"2023-10-26T01:43:50.312423+0000","last_became_peered":"2023-10-26T01:43:50.312423+0000","last_unstale":"2023-10-26T01:43:50.314094+0000","last_undegraded":"2023-10-26T01:43:50.314094+0000","last_fullsized":"2023-10-26T01:43:50.314094+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:16:16.465109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.1a","version":"181'1","reported_seq":12,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.329119+0000","last_change":"2023-10-26T01:43:49.273232+0000","last_active":"2023-10-26T01:43:50.329119+0000","last_peered":"2023-10-26T01:43:50.329119+0000","last_clean":"2023-10-26T01:43:50.329119+0000","last_became_active":"2023-10-26T01:43:49.272568+0000","last_became_peered":"2023-10-26T01:43:49.272568+0000","last_unstale":"2023-10-26T01:43:50.329119+0000","last_undegraded":"2023-10-26T01:43:50.329119+0000","last_fullsized":"2023-10-26T01:43:50.329119+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_clean_scrub_stamp":"2023-10-26T01:43:48.232806+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:19:59.336988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.317217+0000","last_change":"2023-10-26T01:43:50.315725+0000","last_active":"2023-10-26T01:43:50.317217+0000","last_peered":"2023-10-26T01:43:50.317217+0000","last_clean":"2023-10-26T01:43:50.317217+0000","last_became_active":"2023-10-26T01:43:50.314793+0000","last_became_peered":"2023-10-26T01:43:50.314793+0000","last_unstale":"2023-10-26T01:43:50.317217+0000","last_undegraded":"2023-10-26T01:43:50.317217+0000","last_fullsized":"2023-10-26T01:43:50.317217+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:00:40.473949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.299633+0000","last_change":"2023-10-26T01:43:49.274596+0000","last_active":"2023-10-26T01:43:50.299633+0000","last_peered":"2023-10-26T01:43:50.299633+0000","last_clean":"2023-10-26T01:43:50.299633+0000","last_became_active":"2023-10-26T01:43:49.274039+0000","last_became_peered":"2023-10-26T01:43:49.274039+0000","last_unstale":"2023-10-26T01:43:50.299633+0000","last_undegraded":"2023-10-26T01:43:50.299633+0000","last_fullsized":"2023-10-26T01:43:50.299633+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_clean_scrub_stamp":"2023-10-26T01:43:48.232806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:51:03.285440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.317113+0000","last_change":"2023-10-26T01:43:50.317041+0000","last_active":"2023-10-26T01:43:50.317113+0000","last_peered":"2023-10-26T01:43:50.317113+0000","last_clean":"2023-10-26T01:43:50.317113+0000","last_became_active":"2023-10-26T01:43:50.316751+0000","last_became_peered":"2023-10-26T01:43:50.316751+0000","last_unstale":"2023-10-26T01:43:50.317113+0000","last_undegraded":"2023-10-26T01:43:50.317113+0000","last_fullsized":"2023-10-26T01:43:50.317113+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:59:34.197770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.300137+0000","last_change":"2023-10-26T01:43:49.264095+0000","last_active":"2023-10-26T01:43:50.300137+0000","last_peered":"2023-10-26T01:43:50.300137+0000","last_clean":"2023-10-26T01:43:50.300137+0000","last_became_active":"2023-10-26T01:43:49.263822+0000","last_became_peered":"2023-10-26T01:43:49.263822+0000","last_unstale":"2023-10-26T01:43:50.300137+0000","last_undegraded":"2023-10-26T01:43:50.300137+0000","last_fullsized":"2023-10-26T01:43:50.300137+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_clean_scrub_stamp":"2023-10-26T01:43:48.232806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:10:22.173425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.313471+0000","last_change":"2023-10-26T01:43:50.313351+0000","last_active":"2023-10-26T01:43:50.313471+0000","last_peered":"2023-10-26T01:43:50.313471+0000","last_clean":"2023-10-26T01:43:50.313471+0000","last_became_active":"2023-10-26T01:43:50.313121+0000","last_became_peered":"2023-10-26T01:43:50.313121+0000","last_unstale":"2023-10-26T01:43:50.313471+0000","last_undegraded":"2023-10-26T01:43:50.313471+0000","last_fullsized":"2023-10-26T01:43:50.313471+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:00:10.505388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.313270+0000","last_change":"2023-10-26T01:43:49.259137+0000","last_active":"2023-10-26T01:43:50.313270+0000","last_peered":"2023-10-26T01:43:50.313270+0000","last_clean":"2023-10-26T01:43:50.313270+0000","last_became_active":"2023-10-26T01:43:49.256376+0000","last_became_peered":"2023-10-26T01:43:49.256376+0000","last_unstale":"2023-10-26T01:43:50.313270+0000","last_undegraded":"2023-10-26T01:43:50.313270+0000","last_fullsized":"2023-10-26T01:43:50.313270+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_clean_scrub_stamp":"2023-10-26T01:43:48.232806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:18:40.463970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.316447+0000","last_change":"2023-10-26T01:43:50.314842+0000","last_active":"2023-10-26T01:43:50.316447+0000","last_peered":"2023-10-26T01:43:50.316447+0000","last_clean":"2023-10-26T01:43:50.316447+0000","last_became_active":"2023-10-26T01:43:50.306052+0000","last_became_peered":"2023-10-26T01:43:50.306052+0000","last_unstale":"2023-10-26T01:43:50.316447+0000","last_undegraded":"2023-10-26T01:43:50.316447+0000","last_fullsized":"2023-10-26T01:43:50.316447+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:45:38.848472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.15","version":"181'3","reported_seq":14,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.328469+0000","last_change":"2023-10-26T01:43:49.265714+0000","last_active":"2023-10-26T01:43:50.328469+0000","last_peered":"2023-10-26T01:43:50.328469+0000","last_clean":"2023-10-26T01:43:50.328469+0000","last_became_active":"2023-10-26T01:43:49.262146+0000","last_became_peered":"2023-10-26T01:43:49.262146+0000","last_unstale":"2023-10-26T01:43:50.328469+0000","last_undegraded":"2023-10-26T01:43:50.328469+0000","last_fullsized":"2023-10-26T01:43:50.328469+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_clean_scrub_stamp":"2023-10-26T01:43:48.232806+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:54:31.893883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.305531+0000","last_change":"2023-10-26T01:43:50.304110+0000","last_active":"2023-10-26T01:43:50.305531+0000","last_peered":"2023-10-26T01:43:50.305531+0000","last_clean":"2023-10-26T01:43:50.305531+0000","last_became_active":"2023-10-26T01:43:50.303740+0000","last_became_peered":"2023-10-26T01:43:50.303740+0000","last_unstale":"2023-10-26T01:43:50.305531+0000","last_undegraded":"2023-10-26T01:43:50.305531+0000","last_fullsized":"2023-10-26T01:43:50.305531+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:45:14.355821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.14","version":"181'6","reported_seq":17,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.331673+0000","last_change":"2023-10-26T01:43:49.262181+0000","last_active":"2023-10-26T01:43:50.331673+0000","last_peered":"2023-10-26T01:43:50.331673+0000","last_clean":"2023-10-26T01:43:50.331673+0000","last_became_active":"2023-10-26T01:43:49.261557+0000","last_became_peered":"2023-10-26T01:43:49.261557+0000","last_unstale":"2023-10-26T01:43:50.331673+0000","last_undegraded":"2023-10-26T01:43:50.331673+0000","last_fullsized":"2023-10-26T01:43:50.331673+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_clean_scrub_stamp":"2023-10-26T01:43:48.232806+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:26:31.291729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.316474+0000","last_change":"2023-10-26T01:43:50.315983+0000","last_active":"2023-10-26T01:43:50.316474+0000","last_peered":"2023-10-26T01:43:50.316474+0000","last_clean":"2023-10-26T01:43:50.316474+0000","last_became_active":"2023-10-26T01:43:50.315759+0000","last_became_peered":"2023-10-26T01:43:50.315759+0000","last_unstale":"2023-10-26T01:43:50.316474+0000","last_undegraded":"2023-10-26T01:43:50.316474+0000","last_fullsized":"2023-10-26T01:43:50.316474+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:07:51.331877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.13","version":"181'2","reported_seq":13,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.329556+0000","last_change":"2023-10-26T01:43:49.268936+0000","last_active":"2023-10-26T01:43:50.329556+0000","last_peered":"2023-10-26T01:43:50.329556+0000","last_clean":"2023-10-26T01:43:50.329556+0000","last_became_active":"2023-10-26T01:43:49.268702+0000","last_became_peered":"2023-10-26T01:43:49.268702+0000","last_unstale":"2023-10-26T01:43:50.329556+0000","last_undegraded":"2023-10-26T01:43:50.329556+0000","last_fullsized":"2023-10-26T01:43:50.329556+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_clean_scrub_stamp":"2023-10-26T01:43:48.232806+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:58:26.341483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.319024+0000","last_change":"2023-10-26T01:43:50.318945+0000","last_active":"2023-10-26T01:43:50.319024+0000","last_peered":"2023-10-26T01:43:50.319024+0000","last_clean":"2023-10-26T01:43:50.319024+0000","last_became_active":"2023-10-26T01:43:50.318580+0000","last_became_peered":"2023-10-26T01:43:50.318580+0000","last_unstale":"2023-10-26T01:43:50.319024+0000","last_undegraded":"2023-10-26T01:43:50.319024+0000","last_fullsized":"2023-10-26T01:43:50.319024+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:08:09.576677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.288968+0000","last_change":"2023-10-26T01:43:49.270191+0000","last_active":"2023-10-26T01:43:50.288968+0000","last_peered":"2023-10-26T01:43:50.288968+0000","last_clean":"2023-10-26T01:43:50.288968+0000","last_became_active":"2023-10-26T01:43:49.269164+0000","last_became_peered":"2023-10-26T01:43:49.269164+0000","last_unstale":"2023-10-26T01:43:50.288968+0000","last_undegraded":"2023-10-26T01:43:50.288968+0000","last_fullsized":"2023-10-26T01:43:50.288968+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_clean_scrub_stamp":"2023-10-26T01:43:48.232806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:56:57.372727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.311555+0000","last_change":"2023-10-26T01:43:50.311196+0000","last_active":"2023-10-26T01:43:50.311555+0000","last_peered":"2023-10-26T01:43:50.311555+0000","last_clean":"2023-10-26T01:43:50.311555+0000","last_became_active":"2023-10-26T01:43:50.310155+0000","last_became_peered":"2023-10-26T01:43:50.310155+0000","last_unstale":"2023-10-26T01:43:50.311555+0000","last_undegraded":"2023-10-26T01:43:50.311555+0000","last_fullsized":"2023-10-26T01:43:50.311555+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:56:06.619004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.300762+0000","last_change":"2023-10-26T01:43:49.272431+0000","last_active":"2023-10-26T01:43:50.300762+0000","last_peered":"2023-10-26T01:43:50.300762+0000","last_clean":"2023-10-26T01:43:50.300762+0000","last_became_active":"2023-10-26T01:43:49.272134+0000","last_became_peered":"2023-10-26T01:43:49.272134+0000","last_unstale":"2023-10-26T01:43:50.300762+0000","last_undegraded":"2023-10-26T01:43:50.300762+0000","last_fullsized":"2023-10-26T01:43:50.300762+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_clean_scrub_stamp":"2023-10-26T01:43:48.232806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:54:35.020329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.315161+0000","last_change":"2023-10-26T01:43:50.314690+0000","last_active":"2023-10-26T01:43:50.315161+0000","last_peered":"2023-10-26T01:43:50.315161+0000","last_clean":"2023-10-26T01:43:50.315161+0000","last_became_active":"2023-10-26T01:43:50.314394+0000","last_became_peered":"2023-10-26T01:43:50.314394+0000","last_unstale":"2023-10-26T01:43:50.315161+0000","last_undegraded":"2023-10-26T01:43:50.315161+0000","last_fullsized":"2023-10-26T01:43:50.315161+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:24:03.603238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.10","version":"181'8","reported_seq":19,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.330719+0000","last_change":"2023-10-26T01:43:49.266389+0000","last_active":"2023-10-26T01:43:50.330719+0000","last_peered":"2023-10-26T01:43:50.330719+0000","last_clean":"2023-10-26T01:43:50.330719+0000","last_became_active":"2023-10-26T01:43:49.265889+0000","last_became_peered":"2023-10-26T01:43:49.265889+0000","last_unstale":"2023-10-26T01:43:50.330719+0000","last_undegraded":"2023-10-26T01:43:50.330719+0000","last_fullsized":"2023-10-26T01:43:50.330719+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_clean_scrub_stamp":"2023-10-26T01:43:48.232806+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:33:46.344343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.318129+0000","last_change":"2023-10-26T01:43:50.317869+0000","last_active":"2023-10-26T01:43:50.318129+0000","last_peered":"2023-10-26T01:43:50.318129+0000","last_clean":"2023-10-26T01:43:50.318129+0000","last_became_active":"2023-10-26T01:43:50.317638+0000","last_became_peered":"2023-10-26T01:43:50.317638+0000","last_unstale":"2023-10-26T01:43:50.318129+0000","last_undegraded":"2023-10-26T01:43:50.318129+0000","last_fullsized":"2023-10-26T01:43:50.318129+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:04:57.615125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"181'1","reported_seq":12,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.328550+0000","last_change":"2023-10-26T01:43:49.275222+0000","last_active":"2023-10-26T01:43:50.328550+0000","last_peered":"2023-10-26T01:43:50.328550+0000","last_clean":"2023-10-26T01:43:50.328550+0000","last_became_active":"2023-10-26T01:43:49.274973+0000","last_became_peered":"2023-10-26T01:43:49.274973+0000","last_unstale":"2023-10-26T01:43:50.328550+0000","last_undegraded":"2023-10-26T01:43:50.328550+0000","last_fullsized":"2023-10-26T01:43:50.328550+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_clean_scrub_stamp":"2023-10-26T01:43:48.232806+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:40:46.207474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.294182+0000","last_change":"2023-10-26T01:43:50.293891+0000","last_active":"2023-10-26T01:43:50.294182+0000","last_peered":"2023-10-26T01:43:50.294182+0000","last_clean":"2023-10-26T01:43:50.294182+0000","last_became_active":"2023-10-26T01:43:50.293145+0000","last_became_peered":"2023-10-26T01:43:50.293145+0000","last_unstale":"2023-10-26T01:43:50.294182+0000","last_undegraded":"2023-10-26T01:43:50.294182+0000","last_fullsized":"2023-10-26T01:43:50.294182+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:38:17.000674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.e","version":"181'2","reported_seq":13,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.327192+0000","last_change":"2023-10-26T01:43:49.275173+0000","last_active":"2023-10-26T01:43:50.327192+0000","last_peered":"2023-10-26T01:43:50.327192+0000","last_clean":"2023-10-26T01:43:50.327192+0000","last_became_active":"2023-10-26T01:43:49.274005+0000","last_became_peered":"2023-10-26T01:43:49.274005+0000","last_unstale":"2023-10-26T01:43:50.327192+0000","last_undegraded":"2023-10-26T01:43:50.327192+0000","last_fullsized":"2023-10-26T01:43:50.327192+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_clean_scrub_stamp":"2023-10-26T01:43:48.232806+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:54:31.893883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.315679+0000","last_change":"2023-10-26T01:43:50.315594+0000","last_active":"2023-10-26T01:43:50.315679+0000","last_peered":"2023-10-26T01:43:50.315679+0000","last_clean":"2023-10-26T01:43:50.315679+0000","last_became_active":"2023-10-26T01:43:50.314593+0000","last_became_peered":"2023-10-26T01:43:50.314593+0000","last_unstale":"2023-10-26T01:43:50.315679+0000","last_undegraded":"2023-10-26T01:43:50.315679+0000","last_fullsized":"2023-10-26T01:43:50.315679+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:35:14.923796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.d","version":"181'1","reported_seq":12,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.326056+0000","last_change":"2023-10-26T01:43:49.273840+0000","last_active":"2023-10-26T01:43:50.326056+0000","last_peered":"2023-10-26T01:43:50.326056+0000","last_clean":"2023-10-26T01:43:50.326056+0000","last_became_active":"2023-10-26T01:43:49.272623+0000","last_became_peered":"2023-10-26T01:43:49.272623+0000","last_unstale":"2023-10-26T01:43:50.326056+0000","last_undegraded":"2023-10-26T01:43:50.326056+0000","last_fullsized":"2023-10-26T01:43:50.326056+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_clean_scrub_stamp":"2023-10-26T01:43:48.232806+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:26:45.190756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.320516+0000","last_change":"2023-10-26T01:43:50.320362+0000","last_active":"2023-10-26T01:43:50.320516+0000","last_peered":"2023-10-26T01:43:50.320516+0000","last_clean":"2023-10-26T01:43:50.320516+0000","last_became_active":"2023-10-26T01:43:50.320018+0000","last_became_peered":"2023-10-26T01:43:50.320018+0000","last_unstale":"2023-10-26T01:43:50.320516+0000","last_undegraded":"2023-10-26T01:43:50.320516+0000","last_fullsized":"2023-10-26T01:43:50.320516+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:33:47.349231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.288666+0000","last_change":"2023-10-26T01:43:49.268357+0000","last_active":"2023-10-26T01:43:50.288666+0000","last_peered":"2023-10-26T01:43:50.288666+0000","last_clean":"2023-10-26T01:43:50.288666+0000","last_became_active":"2023-10-26T01:43:49.267600+0000","last_became_peered":"2023-10-26T01:43:49.267600+0000","last_unstale":"2023-10-26T01:43:50.288666+0000","last_undegraded":"2023-10-26T01:43:50.288666+0000","last_fullsized":"2023-10-26T01:43:50.288666+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_clean_scrub_stamp":"2023-10-26T01:43:48.232806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:40:32.580321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.319381+0000","last_change":"2023-10-26T01:43:50.318926+0000","last_active":"2023-10-26T01:43:50.319381+0000","last_peered":"2023-10-26T01:43:50.319381+0000","last_clean":"2023-10-26T01:43:50.319381+0000","last_became_active":"2023-10-26T01:43:50.318645+0000","last_became_peered":"2023-10-26T01:43:50.318645+0000","last_unstale":"2023-10-26T01:43:50.319381+0000","last_undegraded":"2023-10-26T01:43:50.319381+0000","last_fullsized":"2023-10-26T01:43:50.319381+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:11:45.355135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.b","version":"181'2","reported_seq":13,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.328965+0000","last_change":"2023-10-26T01:43:49.273491+0000","last_active":"2023-10-26T01:43:50.328965+0000","last_peered":"2023-10-26T01:43:50.328965+0000","last_clean":"2023-10-26T01:43:50.328965+0000","last_became_active":"2023-10-26T01:43:49.272989+0000","last_became_peered":"2023-10-26T01:43:49.272989+0000","last_unstale":"2023-10-26T01:43:50.328965+0000","last_undegraded":"2023-10-26T01:43:50.328965+0000","last_fullsized":"2023-10-26T01:43:50.328965+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_clean_scrub_stamp":"2023-10-26T01:43:48.232806+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:31:46.070435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.316677+0000","last_change":"2023-10-26T01:43:50.316597+0000","last_active":"2023-10-26T01:43:50.316677+0000","last_peered":"2023-10-26T01:43:50.316677+0000","last_clean":"2023-10-26T01:43:50.316677+0000","last_became_active":"2023-10-26T01:43:50.316298+0000","last_became_peered":"2023-10-26T01:43:50.316298+0000","last_unstale":"2023-10-26T01:43:50.316677+0000","last_undegraded":"2023-10-26T01:43:50.316677+0000","last_fullsized":"2023-10-26T01:43:50.316677+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:30:07.397807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.290601+0000","last_change":"2023-10-26T01:43:49.264581+0000","last_active":"2023-10-26T01:43:50.290601+0000","last_peered":"2023-10-26T01:43:50.290601+0000","last_clean":"2023-10-26T01:43:50.290601+0000","last_became_active":"2023-10-26T01:43:49.263942+0000","last_became_peered":"2023-10-26T01:43:49.263942+0000","last_unstale":"2023-10-26T01:43:50.290601+0000","last_undegraded":"2023-10-26T01:43:50.290601+0000","last_fullsized":"2023-10-26T01:43:50.290601+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_clean_scrub_stamp":"2023-10-26T01:43:48.232806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:41:57.698469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.315298+0000","last_change":"2023-10-26T01:43:50.313736+0000","last_active":"2023-10-26T01:43:50.315298+0000","last_peered":"2023-10-26T01:43:50.315298+0000","last_clean":"2023-10-26T01:43:50.315298+0000","last_became_active":"2023-10-26T01:43:50.313374+0000","last_became_peered":"2023-10-26T01:43:50.313374+0000","last_unstale":"2023-10-26T01:43:50.315298+0000","last_undegraded":"2023-10-26T01:43:50.315298+0000","last_fullsized":"2023-10-26T01:43:50.315298+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:58:47.374635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.306832+0000","last_change":"2023-10-26T01:43:49.265844+0000","last_active":"2023-10-26T01:43:50.306832+0000","last_peered":"2023-10-26T01:43:50.306832+0000","last_clean":"2023-10-26T01:43:50.306832+0000","last_became_active":"2023-10-26T01:43:49.265590+0000","last_became_peered":"2023-10-26T01:43:49.265590+0000","last_unstale":"2023-10-26T01:43:50.306832+0000","last_undegraded":"2023-10-26T01:43:50.306832+0000","last_fullsized":"2023-10-26T01:43:50.306832+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_clean_scrub_stamp":"2023-10-26T01:43:48.232806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:06:04.697622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.320607+0000","last_change":"2023-10-26T01:43:50.320400+0000","last_active":"2023-10-26T01:43:50.320607+0000","last_peered":"2023-10-26T01:43:50.320607+0000","last_clean":"2023-10-26T01:43:50.320607+0000","last_became_active":"2023-10-26T01:43:50.320163+0000","last_became_peered":"2023-10-26T01:43:50.320163+0000","last_unstale":"2023-10-26T01:43:50.320607+0000","last_undegraded":"2023-10-26T01:43:50.320607+0000","last_fullsized":"2023-10-26T01:43:50.320607+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T04:22:13.378905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.292680+0000","last_change":"2023-10-26T01:43:49.274338+0000","last_active":"2023-10-26T01:43:50.292680+0000","last_peered":"2023-10-26T01:43:50.292680+0000","last_clean":"2023-10-26T01:43:50.292680+0000","last_became_active":"2023-10-26T01:43:49.273491+0000","last_became_peered":"2023-10-26T01:43:49.273491+0000","last_unstale":"2023-10-26T01:43:50.292680+0000","last_undegraded":"2023-10-26T01:43:50.292680+0000","last_fullsized":"2023-10-26T01:43:50.292680+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_clean_scrub_stamp":"2023-10-26T01:43:48.232806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:41:29.428670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.314229+0000","last_change":"2023-10-26T01:43:50.314153+0000","last_active":"2023-10-26T01:43:50.314229+0000","last_peered":"2023-10-26T01:43:50.314229+0000","last_clean":"2023-10-26T01:43:50.314229+0000","last_became_active":"2023-10-26T01:43:50.313921+0000","last_became_peered":"2023-10-26T01:43:50.313921+0000","last_unstale":"2023-10-26T01:43:50.314229+0000","last_undegraded":"2023-10-26T01:43:50.314229+0000","last_fullsized":"2023-10-26T01:43:50.314229+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:34:37.310319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.7","version":"181'2","reported_seq":13,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.329969+0000","last_change":"2023-10-26T01:43:49.275634+0000","last_active":"2023-10-26T01:43:50.329969+0000","last_peered":"2023-10-26T01:43:50.329969+0000","last_clean":"2023-10-26T01:43:50.329969+0000","last_became_active":"2023-10-26T01:43:49.275430+0000","last_became_peered":"2023-10-26T01:43:49.275430+0000","last_unstale":"2023-10-26T01:43:50.329969+0000","last_undegraded":"2023-10-26T01:43:50.329969+0000","last_fullsized":"2023-10-26T01:43:50.329969+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_clean_scrub_stamp":"2023-10-26T01:43:48.232806+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:40:49.361777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.315314+0000","last_change":"2023-10-26T01:43:50.315243+0000","last_active":"2023-10-26T01:43:50.315314+0000","last_peered":"2023-10-26T01:43:50.315314+0000","last_clean":"2023-10-26T01:43:50.315314+0000","last_became_active":"2023-10-26T01:43:50.314818+0000","last_became_peered":"2023-10-26T01:43:50.314818+0000","last_unstale":"2023-10-26T01:43:50.315314+0000","last_undegraded":"2023-10-26T01:43:50.315314+0000","last_fullsized":"2023-10-26T01:43:50.315314+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:53:06.157549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.301429+0000","last_change":"2023-10-26T01:43:49.269365+0000","last_active":"2023-10-26T01:43:50.301429+0000","last_peered":"2023-10-26T01:43:50.301429+0000","last_clean":"2023-10-26T01:43:50.301429+0000","last_became_active":"2023-10-26T01:43:49.269032+0000","last_became_peered":"2023-10-26T01:43:49.269032+0000","last_unstale":"2023-10-26T01:43:50.301429+0000","last_undegraded":"2023-10-26T01:43:50.301429+0000","last_fullsized":"2023-10-26T01:43:50.301429+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_clean_scrub_stamp":"2023-10-26T01:43:48.232806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:17:54.517773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.319868+0000","last_change":"2023-10-26T01:43:50.319382+0000","last_active":"2023-10-26T01:43:50.319868+0000","last_peered":"2023-10-26T01:43:50.319868+0000","last_clean":"2023-10-26T01:43:50.319868+0000","last_became_active":"2023-10-26T01:43:50.318147+0000","last_became_peered":"2023-10-26T01:43:50.318147+0000","last_unstale":"2023-10-26T01:43:50.319868+0000","last_undegraded":"2023-10-26T01:43:50.319868+0000","last_fullsized":"2023-10-26T01:43:50.319868+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:40:47.212361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.303574+0000","last_change":"2023-10-26T01:43:49.274381+0000","last_active":"2023-10-26T01:43:50.303574+0000","last_peered":"2023-10-26T01:43:50.303574+0000","last_clean":"2023-10-26T01:43:50.303574+0000","last_became_active":"2023-10-26T01:43:49.273694+0000","last_became_peered":"2023-10-26T01:43:49.273694+0000","last_unstale":"2023-10-26T01:43:50.303574+0000","last_undegraded":"2023-10-26T01:43:50.303574+0000","last_fullsized":"2023-10-26T01:43:50.303574+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_clean_scrub_stamp":"2023-10-26T01:43:48.232806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:00:09.500500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.308586+0000","last_change":"2023-10-26T01:43:50.308290+0000","last_active":"2023-10-26T01:43:50.308586+0000","last_peered":"2023-10-26T01:43:50.308586+0000","last_clean":"2023-10-26T01:43:50.308586+0000","last_became_active":"2023-10-26T01:43:50.302771+0000","last_became_peered":"2023-10-26T01:43:50.302771+0000","last_unstale":"2023-10-26T01:43:50.308586+0000","last_undegraded":"2023-10-26T01:43:50.308586+0000","last_fullsized":"2023-10-26T01:43:50.308586+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T02:13:16.853273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.287508+0000","last_change":"2023-10-26T01:43:49.275138+0000","last_active":"2023-10-26T01:43:50.287508+0000","last_peered":"2023-10-26T01:43:50.287508+0000","last_clean":"2023-10-26T01:43:50.287508+0000","last_became_active":"2023-10-26T01:43:49.274913+0000","last_became_peered":"2023-10-26T01:43:49.274913+0000","last_unstale":"2023-10-26T01:43:50.287508+0000","last_undegraded":"2023-10-26T01:43:50.287508+0000","last_fullsized":"2023-10-26T01:43:50.287508+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_clean_scrub_stamp":"2023-10-26T01:43:48.232806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:49:52.800645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.318160+0000","last_change":"2023-10-26T01:43:50.317537+0000","last_active":"2023-10-26T01:43:50.318160+0000","last_peered":"2023-10-26T01:43:50.318160+0000","last_clean":"2023-10-26T01:43:50.318160+0000","last_became_active":"2023-10-26T01:43:50.317268+0000","last_became_peered":"2023-10-26T01:43:50.317268+0000","last_unstale":"2023-10-26T01:43:50.318160+0000","last_undegraded":"2023-10-26T01:43:50.318160+0000","last_fullsized":"2023-10-26T01:43:50.318160+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:21:57.042490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.294298+0000","last_change":"2023-10-26T01:43:49.271052+0000","last_active":"2023-10-26T01:43:50.294298+0000","last_peered":"2023-10-26T01:43:50.294298+0000","last_clean":"2023-10-26T01:43:50.294298+0000","last_became_active":"2023-10-26T01:43:49.267753+0000","last_became_peered":"2023-10-26T01:43:49.267753+0000","last_unstale":"2023-10-26T01:43:50.294298+0000","last_undegraded":"2023-10-26T01:43:50.294298+0000","last_fullsized":"2023-10-26T01:43:50.294298+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_clean_scrub_stamp":"2023-10-26T01:43:48.232806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:28:59.234606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.317321+0000","last_change":"2023-10-26T01:43:50.317111+0000","last_active":"2023-10-26T01:43:50.317321+0000","last_peered":"2023-10-26T01:43:50.317321+0000","last_clean":"2023-10-26T01:43:50.317321+0000","last_became_active":"2023-10-26T01:43:50.315676+0000","last_became_peered":"2023-10-26T01:43:50.315676+0000","last_unstale":"2023-10-26T01:43:50.317321+0000","last_undegraded":"2023-10-26T01:43:50.317321+0000","last_fullsized":"2023-10-26T01:43:50.317321+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:30:07.397807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.301086+0000","last_change":"2023-10-26T01:43:49.263842+0000","last_active":"2023-10-26T01:43:50.301086+0000","last_peered":"2023-10-26T01:43:50.301086+0000","last_clean":"2023-10-26T01:43:50.301086+0000","last_became_active":"2023-10-26T01:43:49.263479+0000","last_became_peered":"2023-10-26T01:43:49.263479+0000","last_unstale":"2023-10-26T01:43:50.301086+0000","last_undegraded":"2023-10-26T01:43:50.301086+0000","last_fullsized":"2023-10-26T01:43:50.301086+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_clean_scrub_stamp":"2023-10-26T01:43:48.232806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:54:38.386630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.314637+0000","last_change":"2023-10-26T01:43:50.314535+0000","last_active":"2023-10-26T01:43:50.314637+0000","last_peered":"2023-10-26T01:43:50.314637+0000","last_clean":"2023-10-26T01:43:50.314637+0000","last_became_active":"2023-10-26T01:43:50.313916+0000","last_became_peered":"2023-10-26T01:43:50.313916+0000","last_unstale":"2023-10-26T01:43:50.314637+0000","last_undegraded":"2023-10-26T01:43:50.314637+0000","last_fullsized":"2023-10-26T01:43:50.314637+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:45:14.355821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.299832+0000","last_change":"2023-10-26T01:43:49.272389+0000","last_active":"2023-10-26T01:43:50.299832+0000","last_peered":"2023-10-26T01:43:50.299832+0000","last_clean":"2023-10-26T01:43:50.299832+0000","last_became_active":"2023-10-26T01:43:49.272152+0000","last_became_peered":"2023-10-26T01:43:49.272152+0000","last_unstale":"2023-10-26T01:43:50.299832+0000","last_undegraded":"2023-10-26T01:43:50.299832+0000","last_fullsized":"2023-10-26T01:43:50.299832+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_clean_scrub_stamp":"2023-10-26T01:43:48.232806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:29:37.539714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.319840+0000","last_change":"2023-10-26T01:43:50.319770+0000","last_active":"2023-10-26T01:43:50.319840+0000","last_peered":"2023-10-26T01:43:50.319840+0000","last_clean":"2023-10-26T01:43:50.319840+0000","last_became_active":"2023-10-26T01:43:50.319515+0000","last_became_peered":"2023-10-26T01:43:50.319515+0000","last_unstale":"2023-10-26T01:43:50.319840+0000","last_undegraded":"2023-10-26T01:43:50.319840+0000","last_fullsized":"2023-10-26T01:43:50.319840+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:52:37.600083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.290196+0000","last_change":"2023-10-26T01:43:49.256981+0000","last_active":"2023-10-26T01:43:50.290196+0000","last_peered":"2023-10-26T01:43:50.290196+0000","last_clean":"2023-10-26T01:43:50.290196+0000","last_became_active":"2023-10-26T01:43:49.256295+0000","last_became_peered":"2023-10-26T01:43:49.256295+0000","last_unstale":"2023-10-26T01:43:50.290196+0000","last_undegraded":"2023-10-26T01:43:50.290196+0000","last_fullsized":"2023-10-26T01:43:50.290196+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_clean_scrub_stamp":"2023-10-26T01:43:48.232806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:10:22.173425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.312632+0000","last_change":"2023-10-26T01:43:50.312485+0000","last_active":"2023-10-26T01:43:50.312632+0000","last_peered":"2023-10-26T01:43:50.312632+0000","last_clean":"2023-10-26T01:43:50.312632+0000","last_became_active":"2023-10-26T01:43:50.307903+0000","last_became_peered":"2023-10-26T01:43:50.307903+0000","last_unstale":"2023-10-26T01:43:50.312632+0000","last_undegraded":"2023-10-26T01:43:50.312632+0000","last_fullsized":"2023-10-26T01:43:50.312632+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:49.237693+0000","last_clean_scrub_stamp":"2023-10-26T01:43:49.237693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:25:08.147968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-26T01:43:50.293974+0000","last_change":"2023-10-26T01:43:49.266363+0000","last_active":"2023-10-26T01:43:50.293974+0000","last_peered":"2023-10-26T01:43:50.293974+0000","last_clean":"2023-10-26T01:43:50.293974+0000","last_became_active":"2023-10-26T01:43:49.265947+0000","last_became_peered":"2023-10-26T01:43:49.265947+0000","last_unstale":"2023-10-26T01:43:50.293974+0000","last_undegraded":"2023-10-26T01:43:50.293974+0000","last_fullsized":"2023-10-26T01:43:50.293974+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:43:48.232806+0000","last_clean_scrub_stamp":"2023-10-26T01:43:48.232806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:53:05.152661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":324,"reported_epoch":180,"state":"active+clean","last_fresh":"2023-10-26T01:43:49.255543+0000","last_change":"2023-10-26T01:30:34.883484+0000","last_active":"2023-10-26T01:43:49.255543+0000","last_peered":"2023-10-26T01:43:49.255543+0000","last_clean":"2023-10-26T01:43:49.255543+0000","last_became_active":"2023-10-26T01:30:34.883071+0000","last_became_peered":"2023-10-26T01:30:34.883071+0000","last_unstale":"2023-10-26T01:43:49.255543+0000","last_undegraded":"2023-10-26T01:43:49.255543+0000","last_fullsized":"2023-10-26T01:43:49.255543+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:32.890096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:32.890096+0000","last_clean_scrub_stamp":"2023-10-26T01:30:32.890096+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:26:31.879452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":23,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":22,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":14,"seq":60129542307,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":995,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194317,"statfs":{"total":200000000,"available":198981104,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:42:55 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.84499999999999997,"15min":0.80800000000000005},"min":{"1min":0.49199999999999999,"5min":0.443,"15min":0.33000000000000002},"max":{"1min":1.278,"5min":1.571,"15min":2.2549999999999999},"last":0.377},{"interface":"front","average":{"1min":0.92700000000000005,"5min":1.004,"15min":0.91900000000000004},"min":{"1min":0.40799999999999997,"5min":0.309,"15min":0.309},"max":{"1min":1.49,"5min":1.919,"15min":1.919},"last":0.50600000000000001}]},{"osd":1,"last update":"Thu Oct 26 01:42:55 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.91700000000000004,"15min":0.91000000000000003},"min":{"1min":0.44900000000000001,"5min":0.39400000000000002,"15min":0.32300000000000001},"max":{"1min":1.357,"5min":1.569,"15min":4.4340000000000002},"last":0.48899999999999999},{"interface":"front","average":{"1min":1.028,"5min":1.0649999999999999,"15min":1.012},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.374},"max":{"1min":1.617,"5min":1.7070000000000001,"15min":2.3330000000000002},"last":0.53200000000000003}]},{"osd":2,"last update":"Thu Oct 26 01:42:55 2023","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.081,"15min":1.0640000000000001},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.39900000000000002},"max":{"1min":1.6819999999999999,"5min":1.6819999999999999,"15min":4.2800000000000002},"last":0.44},{"interface":"front","average":{"1min":1.117,"5min":1.173,"15min":1.123},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.49399999999999999},"max":{"1min":1.7410000000000001,"5min":1.8129999999999999,"15min":2.302},"last":0.46700000000000003}]},{"osd":4,"last update":"Thu Oct 26 01:42:55 2023","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.236,"15min":1.196},"min":{"1min":0.89900000000000002,"5min":0.78700000000000003,"15min":0.66400000000000003},"max":{"1min":1.516,"5min":1.732,"15min":2.1070000000000002},"last":0.66000000000000003},{"interface":"front","average":{"1min":1.365,"5min":1.3759999999999999,"15min":1.3700000000000001},"min":{"1min":1.046,"5min":0.91900000000000004,"15min":0.79900000000000004},"max":{"1min":1.653,"5min":1.7509999999999999,"15min":4.8250000000000002},"last":0.61099999999999999}]},{"osd":5,"last update":"Thu Oct 26 01:42:55 2023","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.3220000000000001,"15min":1.3129999999999999},"min":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.72499999999999998},"max":{"1min":1.8109999999999999,"5min":1.8109999999999999,"15min":4.9080000000000004},"last":0.73899999999999999},{"interface":"front","average":{"1min":1.3280000000000001,"5min":1.3520000000000001,"15min":1.3},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.9279999999999999,"5min":1.9279999999999999,"15min":2.8980000000000001},"last":0.81299999999999994}]},{"osd":6,"last update":"Thu Oct 26 01:42:55 2023","interfaces":[{"interface":"back","average":{"1min":1.5900000000000001,"5min":1.4359999999999999,"15min":1.369},"min":{"1min":0.79700000000000004,"5min":0.75900000000000001,"15min":0.70199999999999996},"max":{"1min":3.5800000000000001,"5min":3.5800000000000001,"15min":3.5800000000000001},"last":0.85099999999999998},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.381,"15min":1.302},"min":{"1min":0.80800000000000005,"5min":0.755,"15min":0.73099999999999998},"max":{"1min":1.5609999999999999,"5min":1.806,"15min":2.181},"last":0.79100000000000004}]},{"osd":7,"last update":"Thu Oct 26 01:42:55 2023","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.411,"15min":1.3380000000000001},"min":{"1min":0.94899999999999995,"5min":0.78800000000000003,"15min":0.71999999999999997},"max":{"1min":1.647,"5min":2.024,"15min":2.0680000000000001},"last":0.87},{"interface":"front","average":{"1min":1.4059999999999999,"5min":1.419,"15min":1.405},"min":{"1min":0.84399999999999997,"5min":0.73299999999999998,"15min":0.72199999999999998},"max":{"1min":1.8560000000000001,"5min":1.8560000000000001,"15min":4.8730000000000002},"last":0.70399999999999996}]}]},{"osd":7,"up_from":13,"seq":55834575010,"num_pgs":13,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1564,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193747,"statfs":{"total":200000000,"available":198397545,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:43:01 2023","interfaces":[{"interface":"back","average":{"1min":1.2090000000000001,"5min":1.1930000000000001,"15min":1.1359999999999999},"min":{"1min":0.63200000000000001,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.6970000000000001,"5min":2.2269999999999999,"15min":2.2269999999999999},"last":0.83899999999999997},{"interface":"front","average":{"1min":1.181,"5min":1.218,"15min":1.137},"min":{"1min":0.79800000000000004,"5min":0.67300000000000004,"15min":0.58699999999999997},"max":{"1min":1.738,"5min":2.282,"15min":2.6779999999999999},"last":0.92700000000000005}]},{"osd":1,"last update":"Thu Oct 26 01:43:01 2023","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":1.165,"15min":1.1399999999999999},"min":{"1min":0.755,"5min":0.65600000000000003,"15min":0.65200000000000002},"max":{"1min":1.51,"5min":1.8,"15min":2.8260000000000001},"last":0.97399999999999998},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.2310000000000001,"15min":1.2130000000000001},"min":{"1min":0.97399999999999998,"5min":0.71699999999999997,"15min":0.61199999999999999},"max":{"1min":1.736,"5min":2.0059999999999998,"15min":3.2730000000000001},"last":1.1910000000000001}]},{"osd":2,"last update":"Thu Oct 26 01:43:01 2023","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.2250000000000001,"15min":1.165},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.58499999999999996},"max":{"1min":1.679,"5min":1.851,"15min":2.218},"last":1.6100000000000001},{"interface":"front","average":{"1min":1.264,"5min":1.2769999999999999,"15min":1.264},"min":{"1min":0.90300000000000002,"5min":0.82299999999999995,"15min":0.73099999999999998},"max":{"1min":1.879,"5min":2.234,"15min":3.0510000000000002},"last":1.248}]},{"osd":3,"last update":"Thu Oct 26 01:43:01 2023","interfaces":[{"interface":"back","average":{"1min":1.319,"5min":1.327,"15min":1.264},"min":{"1min":1.101,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.8149999999999999,"5min":2.1379999999999999,"15min":2.1379999999999999},"last":1.5449999999999999},{"interface":"front","average":{"1min":1.421,"5min":1.4079999999999999,"15min":1.3899999999999999},"min":{"1min":1.008,"5min":0.71099999999999997,"15min":0.65700000000000003},"max":{"1min":1.825,"5min":2.3050000000000002,"15min":5.5039999999999996},"last":1.0529999999999999}]},{"osd":4,"last update":"Thu Oct 26 01:43:01 2023","interfaces":[{"interface":"back","average":{"1min":1.359,"5min":1.4039999999999999,"15min":1.3720000000000001},"min":{"1min":0.92700000000000005,"5min":0.81799999999999995,"15min":0.76100000000000001},"max":{"1min":2.0899999999999999,"5min":2.54,"15min":2.77},"last":1.1439999999999999},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.3919999999999999,"15min":1.4039999999999999},"min":{"1min":0.85399999999999998,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.7470000000000001,"5min":2.8660000000000001,"15min":3.1419999999999999},"last":1.423}]},{"osd":5,"last update":"Thu Oct 26 01:43:01 2023","interfaces":[{"interface":"back","average":{"1min":1.3839999999999999,"5min":1.3999999999999999,"15min":1.4279999999999999},"min":{"1min":0.94899999999999995,"5min":0.435,"15min":0.435},"max":{"1min":1.8180000000000001,"5min":2.3639999999999999,"15min":5.734},"last":1.375},{"interface":"front","average":{"1min":1.5309999999999999,"5min":1.5149999999999999,"15min":1.4970000000000001},"min":{"1min":1.119,"5min":0.76400000000000001,"15min":0.51900000000000002},"max":{"1min":2.2109999999999999,"5min":2.5169999999999999,"15min":2.8980000000000001},"last":1.3140000000000001}]},{"osd":6,"last update":"Thu Oct 26 01:43:01 2023","interfaces":[{"interface":"back","average":{"1min":1.542,"5min":1.5429999999999999,"15min":1.6890000000000001},"min":{"1min":1.1299999999999999,"5min":0.51000000000000001,"15min":0.47399999999999998},"max":{"1min":1.925,"5min":2.5259999999999998,"15min":11.577999999999999},"last":1.292},{"interface":"front","average":{"1min":1.5780000000000001,"5min":1.655,"15min":1.7470000000000001},"min":{"1min":1.2989999999999999,"5min":1.0640000000000001,"15min":0.55100000000000005},"max":{"1min":2.302,"5min":2.7250000000000001,"15min":10.704000000000001},"last":1.474}]}]},{"osd":6,"up_from":13,"seq":55834575010,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":994,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194318,"statfs":{"total":200000000,"available":198981730,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:42:56 2023","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":1.1279999999999999,"15min":1.133},"min":{"1min":0.70099999999999996,"5min":0.66600000000000004,"15min":0.60399999999999998},"max":{"1min":1.6060000000000001,"5min":2.7370000000000001,"15min":2.7370000000000001},"last":1.222},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.1830000000000001,"15min":1.1439999999999999},"min":{"1min":0.85199999999999998,"5min":0.61699999999999999,"15min":0.59199999999999997},"max":{"1min":1.833,"5min":4.4779999999999998,"15min":4.4779999999999998},"last":1.375}]},{"osd":1,"last update":"Thu Oct 26 01:42:56 2023","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":1.179,"15min":1.179},"min":{"1min":0.751,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.4490000000000001,"5min":5.5359999999999996,"15min":5.5359999999999996},"last":1.335},{"interface":"front","average":{"1min":1.169,"5min":1.2430000000000001,"15min":1.2090000000000001},"min":{"1min":0.76400000000000001,"5min":0.74399999999999999,"15min":0.64000000000000001},"max":{"1min":1.6759999999999999,"5min":4.3579999999999997,"15min":4.3579999999999997},"last":1.012}]},{"osd":2,"last update":"Thu Oct 26 01:42:56 2023","interfaces":[{"interface":"back","average":{"1min":1.24,"5min":1.1799999999999999,"15min":1.1379999999999999},"min":{"1min":0.82999999999999996,"5min":0.69799999999999995,"15min":0.59799999999999998},"max":{"1min":1.651,"5min":2.367,"15min":3.7879999999999998},"last":0.97699999999999998},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.3140000000000001,"15min":1.29},"min":{"1min":0.93400000000000005,"5min":0.86799999999999999,"15min":0.68899999999999995},"max":{"1min":1.704,"5min":3.0470000000000002,"15min":3.0470000000000002},"last":1.409}]},{"osd":3,"last update":"Thu Oct 26 01:42:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.2749999999999999,"15min":1.254},"min":{"1min":0.93500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.7569999999999999,"5min":3.266,"15min":4.4560000000000004},"last":1.177},{"interface":"front","average":{"1min":1.399,"5min":1.369,"15min":1.341},"min":{"1min":0.93700000000000006,"5min":0.86399999999999999,"15min":0.69499999999999995},"max":{"1min":1.8009999999999999,"5min":2.6909999999999998,"15min":2.6909999999999998},"last":1.1359999999999999}]},{"osd":4,"last update":"Thu Oct 26 01:42:56 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.262,"15min":1.2430000000000001},"min":{"1min":0.59899999999999998,"5min":0.53400000000000003,"15min":0.48399999999999999},"max":{"1min":1.6439999999999999,"5min":1.825,"15min":2.9489999999999998},"last":0.874},{"interface":"front","average":{"1min":1.357,"5min":1.345,"15min":1.345},"min":{"1min":0.88400000000000001,"5min":0.60699999999999998,"15min":0.59299999999999997},"max":{"1min":1.8959999999999999,"5min":2.169,"15min":6.609},"last":1.29}]},{"osd":5,"last update":"Thu Oct 26 01:42:56 2023","interfaces":[{"interface":"back","average":{"1min":1.556,"5min":1.5369999999999999,"15min":1.496},"min":{"1min":1.2769999999999999,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.9119999999999999,"5min":2.04,"15min":5.4370000000000003},"last":1.4330000000000001},{"interface":"front","average":{"1min":1.4379999999999999,"5min":1.464,"15min":1.4370000000000001},"min":{"1min":0.69999999999999996,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.7010000000000001,"5min":2.7759999999999998,"15min":2.7759999999999998},"last":1.4910000000000001}]},{"osd":7,"last update":"Thu Oct 26 01:42:56 2023","interfaces":[{"interface":"back","average":{"1min":1.389,"5min":1.4099999999999999,"15min":1.4279999999999999},"min":{"1min":0.72399999999999998,"5min":0.58899999999999997,"15min":0.54100000000000004},"max":{"1min":1.887,"5min":1.984,"15min":11.804},"last":1.2390000000000001},{"interface":"front","average":{"1min":1.492,"5min":1.5429999999999999,"15min":1.548},"min":{"1min":0.96399999999999997,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.7470000000000001,"5min":3.7650000000000001,"15min":11.541},"last":1.095}]}]},{"osd":5,"up_from":13,"seq":55834575010,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":995,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194316,"statfs":{"total":200000000,"available":198980534,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:42:57 2023","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.2310000000000001,"15min":1.173},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53300000000000003},"max":{"1min":1.651,"5min":3.8210000000000002,"15min":3.8210000000000002},"last":1.3109999999999999},{"interface":"front","average":{"1min":1.2390000000000001,"5min":1.3100000000000001,"15min":1.242},"min":{"1min":0.82099999999999995,"5min":0.68600000000000005,"15min":0.67900000000000005},"max":{"1min":1.585,"5min":2.9889999999999999,"15min":2.9889999999999999},"last":1.4950000000000001}]},{"osd":1,"last update":"Thu Oct 26 01:42:57 2023","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.22,"15min":1.137},"min":{"1min":0.71299999999999997,"5min":0.70999999999999996,"15min":0.65900000000000003},"max":{"1min":1.6220000000000001,"5min":1.899,"15min":1.899},"last":1.5349999999999999},{"interface":"front","average":{"1min":1.141,"5min":1.147,"15min":1.1240000000000001},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.5600000000000001,"5min":2.4340000000000002,"15min":2.4340000000000002},"last":1.456}]},{"osd":2,"last update":"Thu Oct 26 01:42:57 2023","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.204,"15min":1.1839999999999999},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.69299999999999995},"max":{"1min":1.577,"5min":1.796,"15min":1.8220000000000001},"last":1.272},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.2549999999999999,"15min":1.212},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.673,"5min":1.875,"15min":1.875},"last":1.3460000000000001}]},{"osd":3,"last update":"Thu Oct 26 01:42:57 2023","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":1.3069999999999999,"15min":1.2689999999999999},"min":{"1min":0.91200000000000003,"5min":0.72399999999999998,"15min":0.72299999999999998},"max":{"1min":1.6899999999999999,"5min":1.7,"15min":1.835},"last":1.647},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.3149999999999999,"15min":1.304},"min":{"1min":0.67600000000000005,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.655,"5min":2.6669999999999998,"15min":2.9249999999999998},"last":1.573}]},{"osd":4,"last update":"Thu Oct 26 01:42:57 2023","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.2250000000000001,"15min":1.224},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.49099999999999999},"max":{"1min":1.611,"5min":2.7770000000000001,"15min":2.7770000000000001},"last":1.083},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.343,"15min":1.367},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.647,"5min":1.857,"15min":1.867},"last":1.399}]},{"osd":6,"last update":"Thu Oct 26 01:42:57 2023","interfaces":[{"interface":"back","average":{"1min":1.383,"5min":1.4199999999999999,"15min":1.3759999999999999},"min":{"1min":0.80000000000000004,"5min":0.79000000000000004,"15min":0.51900000000000002},"max":{"1min":1.6819999999999999,"5min":2.4590000000000001,"15min":2.8250000000000002},"last":1.1419999999999999},{"interface":"front","average":{"1min":1.355,"5min":1.4370000000000001,"15min":1.385},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.61799999999999999},"max":{"1min":1.7210000000000001,"5min":2.4830000000000001,"15min":2.4830000000000001},"last":1.208}]},{"osd":7,"last update":"Thu Oct 26 01:42:57 2023","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":1.5109999999999999,"15min":1.474},"min":{"1min":0.58799999999999997,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.766,"5min":3.621,"15min":3.621},"last":1.6240000000000001},{"interface":"front","average":{"1min":1.534,"5min":1.5629999999999999,"15min":1.4990000000000001},"min":{"1min":0.85299999999999998,"5min":0.82899999999999996,"15min":0.59499999999999997},"max":{"1min":1.9970000000000001,"5min":2.8010000000000002,"15min":2.8010000000000002},"last":1.601}]}]},{"osd":4,"up_from":13,"seq":55834575010,"num_pgs":6,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":988,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194324,"statfs":{"total":200000000,"available":198987913,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:43:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.1850000000000001,"15min":1.079},"min":{"1min":0.57699999999999996,"5min":0.503,"15min":0.503},"max":{"1min":1.6499999999999999,"5min":1.78,"15min":2.0289999999999999},"last":1.21},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.1639999999999999,"15min":1.087},"min":{"1min":0.85799999999999998,"5min":0.72599999999999998,"15min":0.60599999999999998},"max":{"1min":1.6639999999999999,"5min":1.921,"15min":2.6829999999999998},"last":1.3340000000000001}]},{"osd":1,"last update":"Thu Oct 26 01:43:49 2023","interfaces":[{"interface":"back","average":{"1min":1.167,"5min":1.2270000000000001,"15min":1.1699999999999999},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.56799999999999995},"max":{"1min":1.804,"5min":2.056,"15min":4.0519999999999996},"last":1.3899999999999999},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.274,"15min":1.1839999999999999},"min":{"1min":0.77800000000000002,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.599,"5min":2.4390000000000001,"15min":2.4390000000000001},"last":1.3080000000000001}]},{"osd":2,"last update":"Thu Oct 26 01:43:49 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.3839999999999999,"15min":1.3149999999999999},"min":{"1min":0.88100000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":2.048,"5min":2.2040000000000002,"15min":5.6749999999999998},"last":1.27},{"interface":"front","average":{"1min":1.196,"5min":1.224,"15min":1.1870000000000001},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.7190000000000001,"5min":1.986,"15min":3.673},"last":1.248}]},{"osd":3,"last update":"Thu Oct 26 01:43:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.357,"15min":1.2529999999999999},"min":{"1min":0.64700000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.708,"5min":2.5960000000000001,"15min":2.5960000000000001},"last":1.054},{"interface":"front","average":{"1min":1.196,"5min":1.2509999999999999,"15min":1.202},"min":{"1min":0.78600000000000003,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.589,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":1.0860000000000001}]},{"osd":5,"last update":"Thu Oct 26 01:43:49 2023","interfaces":[{"interface":"back","average":{"1min":1.228,"5min":1.2949999999999999,"15min":1.278},"min":{"1min":0.876,"5min":0.71099999999999997,"15min":0.46300000000000002},"max":{"1min":1.8640000000000001,"5min":2.774,"15min":2.774},"last":0.876},{"interface":"front","average":{"1min":1.272,"5min":1.355,"15min":1.3320000000000001},"min":{"1min":0.93000000000000005,"5min":0.77700000000000002,"15min":0.55200000000000005},"max":{"1min":1.6539999999999999,"5min":2.3109999999999999,"15min":5.8579999999999997},"last":0.97899999999999998}]},{"osd":6,"last update":"Thu Oct 26 01:43:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3180000000000001,"5min":1.371,"15min":1.3620000000000001},"min":{"1min":0.94899999999999995,"5min":0.69499999999999995,"15min":0.59299999999999997},"max":{"1min":1.76,"5min":2.3780000000000001,"15min":12.343999999999999},"last":1.4510000000000001},{"interface":"front","average":{"1min":1.5389999999999999,"5min":1.6000000000000001,"15min":1.5660000000000001},"min":{"1min":1.113,"5min":0.80000000000000004,"15min":0.55000000000000004},"max":{"1min":2.2799999999999998,"5min":2.9020000000000001,"15min":12.605},"last":1.4199999999999999}]},{"osd":7,"last update":"Thu Oct 26 01:43:49 2023","interfaces":[{"interface":"back","average":{"1min":1.466,"5min":1.5740000000000001,"15min":1.522},"min":{"1min":1.0649999999999999,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":2.044,"5min":2.7200000000000002,"15min":5.774},"last":1.157},{"interface":"front","average":{"1min":1.468,"5min":1.542,"15min":1.4990000000000001},"min":{"1min":0.79500000000000004,"5min":0.79000000000000004,"15min":0.68000000000000005},"max":{"1min":1.8500000000000001,"5min":2.8149999999999999,"15min":4.6520000000000001},"last":1.363}]}]},{"osd":2,"up_from":13,"seq":55834575010,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":988,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194324,"statfs":{"total":200000000,"available":198987913,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:43:02 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.80000000000000004,"15min":0.83699999999999997},"min":{"1min":0.441,"5min":0.38500000000000001,"15min":0.308},"max":{"1min":1.0660000000000001,"5min":1.4079999999999999,"15min":6.3200000000000003},"last":1.363},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.94599999999999995,"15min":0.94899999999999995},"min":{"1min":0.46400000000000002,"5min":0.432,"15min":0.38600000000000001},"max":{"1min":1.4359999999999999,"5min":1.476,"15min":4.2619999999999996},"last":0.84199999999999997}]},{"osd":1,"last update":"Thu Oct 26 01:43:02 2023","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.872,"15min":0.89000000000000001},"min":{"1min":0.48799999999999999,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.4339999999999999,"5min":1.504,"15min":5.1260000000000003},"last":0.73499999999999999},{"interface":"front","average":{"1min":1.087,"5min":1.0209999999999999,"15min":1.0149999999999999},"min":{"1min":0.58199999999999996,"5min":0.42699999999999999,"15min":0.36599999999999999},"max":{"1min":1.5469999999999999,"5min":1.5469999999999999,"15min":6.1520000000000001},"last":1.103}]},{"osd":3,"last update":"Thu Oct 26 01:43:02 2023","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.089,"15min":1.004},"min":{"1min":0.55800000000000005,"5min":0.435,"15min":0.435},"max":{"1min":1.5760000000000001,"5min":1.909,"15min":2.0640000000000001},"last":0.98699999999999999},{"interface":"front","average":{"1min":1.1439999999999999,"5min":1.105,"15min":1.0720000000000001},"min":{"1min":0.52600000000000002,"5min":0.503,"15min":0.45300000000000001},"max":{"1min":1.6639999999999999,"5min":1.6659999999999999,"15min":8.9019999999999992},"last":1.0880000000000001}]},{"osd":4,"last update":"Thu Oct 26 01:43:02 2023","interfaces":[{"interface":"back","average":{"1min":1.363,"5min":1.3200000000000001,"15min":1.302},"min":{"1min":1.04,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.9910000000000001,"5min":2.109,"15min":6.2670000000000003},"last":1.544},{"interface":"front","average":{"1min":1.3129999999999999,"5min":1.2689999999999999,"15min":1.2809999999999999},"min":{"1min":0.77500000000000002,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.659,"5min":1.923,"15min":7.343},"last":1.266}]},{"osd":5,"last update":"Thu Oct 26 01:43:02 2023","interfaces":[{"interface":"back","average":{"1min":1.405,"5min":1.333,"15min":1.3140000000000001},"min":{"1min":0.81799999999999995,"5min":0.76900000000000002,"15min":0.71899999999999997},"max":{"1min":1.784,"5min":2.3849999999999998,"15min":3.7170000000000001},"last":1.4390000000000001},{"interface":"front","average":{"1min":1.292,"5min":1.3260000000000001,"15min":1.3089999999999999},"min":{"1min":0.90300000000000002,"5min":0.69499999999999995,"15min":0.65300000000000002},"max":{"1min":1.591,"5min":1.8140000000000001,"15min":6.3899999999999997},"last":1.3080000000000001}]},{"osd":6,"last update":"Thu Oct 26 01:43:02 2023","interfaces":[{"interface":"back","average":{"1min":1.3540000000000001,"5min":1.3779999999999999,"15min":1.365},"min":{"1min":0.84799999999999998,"5min":0.76300000000000001,"15min":0.70599999999999996},"max":{"1min":1.6759999999999999,"5min":2.097,"15min":6.0490000000000004},"last":1.6399999999999999},{"interface":"front","average":{"1min":1.464,"5min":1.395,"15min":1.389},"min":{"1min":1.0720000000000001,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":2.036,"5min":2.5230000000000001,"15min":5.0650000000000004},"last":1.6659999999999999}]},{"osd":7,"last update":"Thu Oct 26 01:43:02 2023","interfaces":[{"interface":"back","average":{"1min":1.363,"5min":1.4159999999999999,"15min":1.4039999999999999},"min":{"1min":0.92400000000000004,"5min":0.79200000000000004,"15min":0.69399999999999995},"max":{"1min":1.7569999999999999,"5min":1.7949999999999999,"15min":6.4480000000000004},"last":1.476},{"interface":"front","average":{"1min":1.476,"5min":1.484,"15min":1.474},"min":{"1min":1.0569999999999999,"5min":0.91000000000000003,"15min":0.91000000000000003},"max":{"1min":1.804,"5min":2.1480000000000001,"15min":2.8439999999999999},"last":1.706}]}]},{"osd":1,"up_from":13,"seq":55834575010,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":995,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194317,"statfs":{"total":200000000,"available":198981104,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:42:53 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.82599999999999996,"15min":0.77000000000000002},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.29499999999999998},"max":{"1min":1.177,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.95699999999999996,"15min":0.86099999999999999},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.36199999999999999},"max":{"1min":1.591,"5min":1.591,"15min":1.6399999999999999},"last":0.66400000000000003}]},{"osd":2,"last update":"Thu Oct 26 01:42:53 2023","interfaces":[{"interface":"back","average":{"1min":1.0840000000000001,"5min":1.095,"15min":1.026},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.39600000000000002},"max":{"1min":1.5800000000000001,"5min":1.655,"15min":2.4830000000000001},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.98099999999999998,"5min":1.0389999999999999,"15min":0.93999999999999995},"min":{"1min":0.51000000000000001,"5min":0.48099999999999998,"15min":0.41199999999999998},"max":{"1min":1.2330000000000001,"5min":1.752,"15min":1.8080000000000001},"last":1.0980000000000001}]},{"osd":3,"last update":"Thu Oct 26 01:42:58 2023","interfaces":[{"interface":"back","average":{"1min":1.0569999999999999,"5min":1.073,"15min":0.997},"min":{"1min":0.64100000000000001,"5min":0.57099999999999995,"15min":0.39500000000000002},"max":{"1min":1.3879999999999999,"5min":1.6100000000000001,"15min":1.75},"last":1.022},{"interface":"front","average":{"1min":1.1299999999999999,"5min":1.0980000000000001,"15min":1.0329999999999999},"min":{"1min":0.57499999999999996,"5min":0.55000000000000004,"15min":0.41299999999999998},"max":{"1min":1.556,"5min":1.556,"15min":1.802},"last":1.054}]},{"osd":4,"last update":"Thu Oct 26 01:42:53 2023","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":1.2749999999999999,"15min":1.2310000000000001},"min":{"1min":0.95899999999999996,"5min":0.79800000000000004,"15min":0.56200000000000006},"max":{"1min":1.591,"5min":1.726,"15min":2.0659999999999998},"last":1.2090000000000001},{"interface":"front","average":{"1min":1.248,"5min":1.2509999999999999,"15min":1.1839999999999999},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.65800000000000003},"max":{"1min":1.7450000000000001,"5min":1.8420000000000001,"15min":2.0270000000000001},"last":1.6850000000000001}]},{"osd":5,"last update":"Thu Oct 26 01:42:53 2023","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.3120000000000001,"15min":1.2589999999999999},"min":{"1min":0.629,"5min":0.629,"15min":0.61799999999999999},"max":{"1min":1.7,"5min":1.7050000000000001,"15min":1.8779999999999999},"last":1.464},{"interface":"front","average":{"1min":1.3480000000000001,"5min":1.395,"15min":1.359},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.59999999999999998},"max":{"1min":1.639,"5min":1.8280000000000001,"15min":2.2200000000000002},"last":1.349}]},{"osd":6,"last update":"Thu Oct 26 01:42:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3460000000000001,"5min":1.3520000000000001,"15min":1.3120000000000001},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.8,"5min":1.8,"15min":2.3780000000000001},"last":1.258},{"interface":"front","average":{"1min":1.3009999999999999,"5min":1.3560000000000001,"15min":1.288},"min":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.70799999999999996},"max":{"1min":1.611,"5min":1.8819999999999999,"15min":1.984},"last":1.3109999999999999}]},{"osd":7,"last update":"Thu Oct 26 01:42:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3180000000000001,"5min":1.3460000000000001,"15min":1.2869999999999999},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.772,"5min":1.835,"15min":2.6930000000000001},"last":1.409},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.4359999999999999,"15min":1.381},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.7,"5min":1.903,"15min":2.0190000000000001},"last":1.645}]}]},{"osd":0,"up_from":13,"seq":55834575010,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1564,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193747,"statfs":{"total":200000000,"available":198397545,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Oct 26 01:42:57 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.79100000000000004,"15min":0.88400000000000001},"min":{"1min":0.52800000000000002,"5min":0.32500000000000001,"15min":0.32000000000000001},"max":{"1min":1.048,"5min":1.494,"15min":19.425000000000001},"last":0.79200000000000004},{"interface":"front","average":{"1min":1.0389999999999999,"5min":0.99399999999999999,"15min":1.024},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.478,"5min":1.492,"15min":15.162000000000001},"last":1.044}]},{"osd":2,"last update":"Thu Oct 26 01:42:57 2023","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.0469999999999999,"15min":1.151},"min":{"1min":0.51000000000000001,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.4410000000000001,"5min":1.556,"15min":28.439},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.93799999999999994,"15min":0.91900000000000004},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.37},"max":{"1min":1.365,"5min":1.851,"15min":4.9459999999999997},"last":1.302}]},{"osd":3,"last update":"Thu Oct 26 01:42:57 2023","interfaces":[{"interface":"back","average":{"1min":1.0900000000000001,"5min":1.131,"15min":1.0900000000000001},"min":{"1min":0.629,"5min":0.56999999999999995,"15min":0.44600000000000001},"max":{"1min":1.597,"5min":1.7509999999999999,"15min":4.9800000000000004},"last":1.4179999999999999},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.216,"15min":1.169},"min":{"1min":0.69799999999999995,"5min":0.63500000000000001,"15min":0.438},"max":{"1min":1.661,"5min":1.774,"15min":5.016},"last":1.48}]},{"osd":4,"last update":"Thu Oct 26 01:42:57 2023","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":1.1439999999999999,"15min":1.252},"min":{"1min":0.875,"5min":0.76800000000000002,"15min":0.52900000000000003},"max":{"1min":1.5569999999999999,"5min":1.5840000000000001,"15min":19.329000000000001},"last":1.0940000000000001},{"interface":"front","average":{"1min":1.2210000000000001,"5min":1.2509999999999999,"15min":1.2749999999999999},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.67600000000000005},"max":{"1min":1.48,"5min":1.7270000000000001,"15min":5.5229999999999997},"last":1.1850000000000001}]},{"osd":5,"last update":"Thu Oct 26 01:42:57 2023","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.4279999999999999,"15min":1.4219999999999999},"min":{"1min":1.0189999999999999,"5min":1,"15min":0.64900000000000002},"max":{"1min":1.8660000000000001,"5min":1.8660000000000001,"15min":5.6360000000000001},"last":1.5960000000000001},{"interface":"front","average":{"1min":1.306,"5min":1.2829999999999999,"15min":1.371},"min":{"1min":0.91200000000000003,"5min":0.80200000000000005,"15min":0.57499999999999996},"max":{"1min":1.6259999999999999,"5min":1.742,"15min":19.384},"last":1.288}]},{"osd":6,"last update":"Thu Oct 26 01:42:57 2023","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.3180000000000001,"15min":1.347},"min":{"1min":0.83999999999999997,"5min":0.79300000000000004,"15min":0.73799999999999999},"max":{"1min":1.6899999999999999,"5min":1.8999999999999999,"15min":8.2609999999999992},"last":1.226},{"interface":"front","average":{"1min":1.321,"5min":1.3600000000000001,"15min":1.3939999999999999},"min":{"1min":0.746,"5min":0.746,"15min":0.746},"max":{"1min":1.6000000000000001,"5min":1.9399999999999999,"15min":11.849},"last":1.361}]},{"osd":7,"last update":"Thu Oct 26 01:42:57 2023","interfaces":[{"interface":"back","average":{"1min":1.458,"5min":1.4870000000000001,"15min":1.4730000000000001},"min":{"1min":1.071,"5min":0.999,"15min":0.73699999999999999},"max":{"1min":1.9350000000000001,"5min":1.9379999999999999,"15min":5.6689999999999996},"last":1.8520000000000001},{"interface":"front","average":{"1min":1.4139999999999999,"5min":1.411,"15min":1.4870000000000001},"min":{"1min":1.079,"5min":0.81299999999999994,"15min":0.69399999999999995},"max":{"1min":1.6339999999999999,"5min":1.8149999999999999,"15min":19.245999999999999},"last":1.337}]}]}],"pool_statfs":[]}} 2023-10-26T01:43:55.275 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-26T01:43:55.276 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-26T01:43:55.541 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:55.539+0000 7f351e467640 1 -- 172.21.15.169:0/1750234307 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35000039c0 msgr2=0x7f3500003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:55.542 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:55.539+0000 7f351e467640 1 --2- 172.21.15.169:0/1750234307 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35000039c0 0x7f3500003de0 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f350800bb20 tx=0x7f35000057f0 comp rx=0 tx=0).stop 2023-10-26T01:43:55.542 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:55.539+0000 7f351e467640 1 -- 172.21.15.169:0/1750234307 shutdown_connections 2023-10-26T01:43:55.542 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:55.539+0000 7f351e467640 1 --2- 172.21.15.169:0/1750234307 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35000039c0 0x7f3500003de0 unknown :-1 s=CLOSED pgs=1853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:55.542 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:55.539+0000 7f351e467640 1 -- 172.21.15.169:0/1750234307 >> 172.21.15.169:0/1750234307 conn(0x7f3518069b70 msgr2=0x7f35180aebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:55.542 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:55.539+0000 7f351e467640 1 -- 172.21.15.169:0/1750234307 shutdown_connections 2023-10-26T01:43:55.543 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:55.539+0000 7f351e467640 1 -- 172.21.15.169:0/1750234307 wait complete. 2023-10-26T01:43:55.543 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:55.541+0000 7f351e467640 1 Processor -- start 2023-10-26T01:43:55.544 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:55.541+0000 7f351e467640 1 -- start start 2023-10-26T01:43:55.545 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:55.542+0000 7f351e467640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35000039c0 0x7f35180c8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:55.545 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:55.542+0000 7f351e467640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3518143500 0x7f35181f1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:55.545 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:55.542+0000 7f351e467640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f35181f1c50 0x7f35181f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:55.546 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:55.542+0000 7f351e467640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35180c8050 con 0x7f35000039c0 2023-10-26T01:43:55.546 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:55.542+0000 7f351e467640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35180c6fb0 con 0x7f35181f1c50 2023-10-26T01:43:55.546 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:55.542+0000 7f351e467640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35180c7290 con 0x7f3518143500 2023-10-26T01:43:55.546 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:55.542+0000 7f351d465640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35000039c0 0x7f35180c8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:55.546 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:55.542+0000 7f351d465640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35000039c0 0x7f35180c8b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:46186/0 (socket says 172.21.15.169:46186) 2023-10-26T01:43:55.547 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:55.543+0000 7f351d465640 1 -- 172.21.15.169:0/2508901817 learned_addr learned my addr 172.21.15.169:0/2508901817 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:43:55.547 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:55.543+0000 7f351d465640 1 -- 172.21.15.169:0/2508901817 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3518143500 msgr2=0x7f35181f1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:55.547 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:55.543+0000 7f351d465640 1 --2- 172.21.15.169:0/2508901817 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3518143500 0x7f35181f1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:55.547 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:55.543+0000 7f351d465640 1 -- 172.21.15.169:0/2508901817 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f35181f1c50 msgr2=0x7f35181f20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:55.547 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:55.543+0000 7f351d465640 1 --2- 172.21.15.169:0/2508901817 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f35181f1c50 0x7f35181f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:55.548 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:55.543+0000 7f351d465640 1 -- 172.21.15.169:0/2508901817 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35180c7570 con 0x7f35000039c0 2023-10-26T01:43:55.548 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:55.543+0000 7f351d465640 1 --2- 172.21.15.169:0/2508901817 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35000039c0 0x7f35180c8b90 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f3508038040 tx=0x7f3508036720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:55.548 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:55.543+0000 7f350dffb640 1 -- 172.21.15.169:0/2508901817 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3508036d20 con 0x7f35000039c0 2023-10-26T01:43:55.548 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:55.544+0000 7f350dffb640 1 -- 172.21.15.169:0/2508901817 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3508004030 con 0x7f35000039c0 2023-10-26T01:43:55.548 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:55.544+0000 7f351e467640 1 -- 172.21.15.169:0/2508901817 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35180c7850 con 0x7f35000039c0 2023-10-26T01:43:55.549 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:55.544+0000 7f350dffb640 1 -- 172.21.15.169:0/2508901817 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3508004420 con 0x7f35000039c0 2023-10-26T01:43:55.549 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:55.544+0000 7f351e467640 1 -- 172.21.15.169:0/2508901817 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35180c90d0 con 0x7f35000039c0 2023-10-26T01:43:55.550 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:55.545+0000 7f351e467640 1 -- 172.21.15.169:0/2508901817 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3518066ef0 con 0x7f35000039c0 2023-10-26T01:43:55.550 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:55.546+0000 7f350dffb640 1 -- 172.21.15.169:0/2508901817 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f35080035f0 con 0x7f35000039c0 2023-10-26T01:43:55.550 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:55.547+0000 7f350dffb640 1 --2- 172.21.15.169:0/2508901817 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f350406ae40 0x7f350406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:55.550 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:55.547+0000 7f350dffb640 1 -- 172.21.15.169:0/2508901817 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f35080ac300 con 0x7f35000039c0 2023-10-26T01:43:55.551 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:55.547+0000 7f351cc64640 1 --2- 172.21.15.169:0/2508901817 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f350406ae40 0x7f350406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:55.551 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:55.549+0000 7f351cc64640 1 --2- 172.21.15.169:0/2508901817 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f350406ae40 0x7f350406d300 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f35140017a0 tx=0x7f3514000ef0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:55.552 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:55.551+0000 7f350dffb640 1 -- 172.21.15.169:0/2508901817 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3508077310 con 0x7f35000039c0 2023-10-26T01:43:55.813 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:55.811+0000 7f351e467640 1 -- 172.21.15.169:0/2508901817 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f3518073c10 con 0x7f35000039c0 2023-10-26T01:43:55.817 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:55.816+0000 7f350dffb640 1 -- 172.21.15.169:0/2508901817 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v22) v1 ==== 190+0+147 (secure 0 0 0) 0x7f350807fe60 con 0x7f35000039c0 2023-10-26T01:43:55.818 INFO:teuthology.orchestra.run.smithi169.stdout:[client.0] 2023-10-26T01:43:55.818 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQAgwTllKC7gOhAAPsvxGcuUA8/p8xqbUgGteA== 2023-10-26T01:43:55.818 INFO:teuthology.orchestra.run.smithi169.stdout: caps mds = "allow" 2023-10-26T01:43:55.818 INFO:teuthology.orchestra.run.smithi169.stdout: caps mon = "allow r" 2023-10-26T01:43:55.819 INFO:teuthology.orchestra.run.smithi169.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-10-26T01:43:55.819 INFO:teuthology.orchestra.run.smithi169.stderr:updated caps for client.0 2023-10-26T01:43:55.820 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:55.818+0000 7f351e467640 1 -- 172.21.15.169:0/2508901817 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f350406ae40 msgr2=0x7f350406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:55.820 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:55.818+0000 7f351e467640 1 --2- 172.21.15.169:0/2508901817 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f350406ae40 0x7f350406d300 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f35140017a0 tx=0x7f3514000ef0 comp rx=0 tx=0).stop 2023-10-26T01:43:55.820 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:55.818+0000 7f351e467640 1 -- 172.21.15.169:0/2508901817 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35000039c0 msgr2=0x7f35180c8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:55.820 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:55.818+0000 7f351e467640 1 --2- 172.21.15.169:0/2508901817 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35000039c0 0x7f35180c8b90 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f3508038040 tx=0x7f3508036720 comp rx=0 tx=0).stop 2023-10-26T01:43:55.821 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:55.818+0000 7f351e467640 1 -- 172.21.15.169:0/2508901817 shutdown_connections 2023-10-26T01:43:55.821 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:55.818+0000 7f351e467640 1 --2- 172.21.15.169:0/2508901817 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f350406ae40 0x7f350406d300 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:55.821 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:55.818+0000 7f351e467640 1 --2- 172.21.15.169:0/2508901817 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f35181f1c50 0x7f35181f20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:55.821 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:55.818+0000 7f351e467640 1 --2- 172.21.15.169:0/2508901817 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3518143500 0x7f35181f1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:55.821 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:55.818+0000 7f351e467640 1 --2- 172.21.15.169:0/2508901817 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35000039c0 0x7f35180c8b90 unknown :-1 s=CLOSED pgs=1854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:55.822 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:55.818+0000 7f351e467640 1 -- 172.21.15.169:0/2508901817 >> 172.21.15.169:0/2508901817 conn(0x7f3518069b70 msgr2=0x7f35180ab620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:55.822 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:55.818+0000 7f351e467640 1 -- 172.21.15.169:0/2508901817 shutdown_connections 2023-10-26T01:43:55.822 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:55.818+0000 7f351e467640 1 -- 172.21.15.169:0/2508901817 wait complete. 2023-10-26T01:43:55.838 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T01:43:56.059 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.057+0000 7f6f858d7640 1 -- 172.21.15.169:0/3574067427 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f800cde00 msgr2=0x7f6f800cbac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:56.060 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.057+0000 7f6f858d7640 1 --2- 172.21.15.169:0/3574067427 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f800cde00 0x7f6f800cbac0 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f6f8006be60 tx=0x7f6f7802c1d0 comp rx=0 tx=0).stop 2023-10-26T01:43:56.060 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.057+0000 7f6f858d7640 1 -- 172.21.15.169:0/3574067427 shutdown_connections 2023-10-26T01:43:56.060 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.057+0000 7f6f858d7640 1 --2- 172.21.15.169:0/3574067427 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f800cde00 0x7f6f800cbac0 unknown :-1 s=CLOSED pgs=1855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:56.060 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.057+0000 7f6f858d7640 1 --2- 172.21.15.169:0/3574067427 >> v2:172.21.15.170:3300/0 conn(0x7f6f801592d0 0x7f6f800aa3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:56.060 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.057+0000 7f6f858d7640 1 -- 172.21.15.169:0/3574067427 >> 172.21.15.169:0/3574067427 conn(0x7f6f80069930 msgr2=0x7f6f80069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:56.061 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.057+0000 7f6f858d7640 1 -- 172.21.15.169:0/3574067427 shutdown_connections 2023-10-26T01:43:56.061 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.057+0000 7f6f858d7640 1 -- 172.21.15.169:0/3574067427 wait complete. 2023-10-26T01:43:56.062 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.059+0000 7f6f858d7640 1 Processor -- start 2023-10-26T01:43:56.062 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.059+0000 7f6f858d7640 1 -- start start 2023-10-26T01:43:56.062 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.060+0000 7f6f858d7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f800ac930 0x7f6f8015b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:56.062 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.060+0000 7f6f858d7640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f800cde00 0x7f6f8014a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:56.063 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.060+0000 7f6f858d7640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6f801592d0 0x7f6f800adfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:56.063 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.060+0000 7f6f858d7640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f8015b840 con 0x7f6f800cde00 2023-10-26T01:43:56.063 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.060+0000 7f6f858d7640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f800b0680 con 0x7f6f800ac930 2023-10-26T01:43:56.063 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.060+0000 7f6f858d7640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f800b0960 con 0x7f6f801592d0 2023-10-26T01:43:56.065 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.061+0000 7f6f77fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f800cde00 0x7f6f8014a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:56.065 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.061+0000 7f6f77fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f800cde00 0x7f6f8014a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:46210/0 (socket says 172.21.15.169:46210) 2023-10-26T01:43:56.065 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.061+0000 7f6f77fff640 1 -- 172.21.15.169:0/3597458283 learned_addr learned my addr 172.21.15.169:0/3597458283 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:43:56.065 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.061+0000 7f6f7effd640 1 --2- 172.21.15.169:0/3597458283 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f800ac930 0x7f6f8015b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:56.066 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.061+0000 7f6f7f7fe640 1 --2- 172.21.15.169:0/3597458283 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6f801592d0 0x7f6f800adfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:56.066 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.061+0000 7f6f77fff640 1 -- 172.21.15.169:0/3597458283 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6f801592d0 msgr2=0x7f6f800adfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:56.066 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.061+0000 7f6f77fff640 1 --2- 172.21.15.169:0/3597458283 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6f801592d0 0x7f6f800adfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:56.066 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.061+0000 7f6f77fff640 1 -- 172.21.15.169:0/3597458283 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f800ac930 msgr2=0x7f6f8015b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:56.066 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.061+0000 7f6f77fff640 1 --2- 172.21.15.169:0/3597458283 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f800ac930 0x7f6f8015b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:56.067 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.061+0000 7f6f77fff640 1 -- 172.21.15.169:0/3597458283 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f78009900 con 0x7f6f800cde00 2023-10-26T01:43:56.067 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.061+0000 7f6f7f7fe640 1 --2- 172.21.15.169:0/3597458283 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6f801592d0 0x7f6f800adfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:43:56.067 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.061+0000 7f6f7effd640 1 --2- 172.21.15.169:0/3597458283 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f800ac930 0x7f6f8015b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T01:43:56.067 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.061+0000 7f6f77fff640 1 --2- 172.21.15.169:0/3597458283 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f800cde00 0x7f6f8014a7f0 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f6f6800c430 tx=0x7f6f6800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:56.067 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.062+0000 7f6f7cff9640 1 -- 172.21.15.169:0/3597458283 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f68011790 con 0x7f6f800cde00 2023-10-26T01:43:56.068 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.062+0000 7f6f7cff9640 1 -- 172.21.15.169:0/3597458283 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6f680119e0 con 0x7f6f800cde00 2023-10-26T01:43:56.068 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.062+0000 7f6f7cff9640 1 -- 172.21.15.169:0/3597458283 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f68011e30 con 0x7f6f800cde00 2023-10-26T01:43:56.068 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.062+0000 7f6f858d7640 1 -- 172.21.15.169:0/3597458283 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f800b0c40 con 0x7f6f800cde00 2023-10-26T01:43:56.068 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.062+0000 7f6f858d7640 1 -- 172.21.15.169:0/3597458283 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f800b1200 con 0x7f6f800cde00 2023-10-26T01:43:56.068 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.063+0000 7f6f858d7640 1 -- 172.21.15.169:0/3597458283 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f800b13c0 con 0x7f6f800cde00 2023-10-26T01:43:56.069 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.064+0000 7f6f7cff9640 1 -- 172.21.15.169:0/3597458283 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f6f6801c460 con 0x7f6f800cde00 2023-10-26T01:43:56.070 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.065+0000 7f6f7cff9640 1 --2- 172.21.15.169:0/3597458283 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f6f6006ae40 0x7f6f6006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:56.070 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.065+0000 7f6f7cff9640 1 -- 172.21.15.169:0/3597458283 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6f680897e0 con 0x7f6f800cde00 2023-10-26T01:43:56.070 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.066+0000 7f6f7effd640 1 --2- 172.21.15.169:0/3597458283 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f6f6006ae40 0x7f6f6006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:56.070 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.067+0000 7f6f7effd640 1 --2- 172.21.15.169:0/3597458283 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f6f6006ae40 0x7f6f6006d300 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f6f70005fd0 tx=0x7f6f70005e20 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:56.071 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.070+0000 7f6f7cff9640 1 -- 172.21.15.169:0/3597458283 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6f68054920 con 0x7f6f800cde00 2023-10-26T01:43:56.298 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.296+0000 7f6f858d7640 1 -- 172.21.15.169:0/3597458283 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6f80153360 con 0x7f6f800cde00 2023-10-26T01:43:56.298 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.297+0000 7f6f7cff9640 1 -- 172.21.15.169:0/3597458283 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) v1 ==== 94+0+4920 (secure 0 0 0) 0x7f6f68057f40 con 0x7f6f800cde00 2023-10-26T01:43:56.299 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:43:56.299 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":94,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6484,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/195700049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":195700049},{"type":"v1","addr":"172.21.15.169:6835","nonce":195700049}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":89},{"gid":6493,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/1167696300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1167696300},{"type":"v1","addr":"172.21.15.170:6836","nonce":1167696300}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":89},{"gid":9111,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/22729060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":22729060},{"type":"v1","addr":"172.21.15.169:6834","nonce":22729060}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":89}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T01:43:49.711084+0000","modified":"2023-10-26T01:43:52.816785+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9123},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9123":{"gid":9123,"name":"d","rank":0,"incarnation":90,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6837/2205518515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":2205518515},{"type":"v1","addr":"172.21.15.170:6837","nonce":2205518515}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2023-10-26T01:43:56.299 INFO:teuthology.orchestra.run.smithi169.stderr:dumped fsmap epoch 94 2023-10-26T01:43:56.301 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.299+0000 7f6f858d7640 1 -- 172.21.15.169:0/3597458283 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f6f6006ae40 msgr2=0x7f6f6006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:56.301 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.299+0000 7f6f858d7640 1 --2- 172.21.15.169:0/3597458283 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f6f6006ae40 0x7f6f6006d300 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f6f70005fd0 tx=0x7f6f70005e20 comp rx=0 tx=0).stop 2023-10-26T01:43:56.301 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.299+0000 7f6f858d7640 1 -- 172.21.15.169:0/3597458283 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f800cde00 msgr2=0x7f6f8014a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:56.302 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.299+0000 7f6f858d7640 1 --2- 172.21.15.169:0/3597458283 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f800cde00 0x7f6f8014a7f0 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f6f6800c430 tx=0x7f6f6800c8f0 comp rx=0 tx=0).stop 2023-10-26T01:43:56.302 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.299+0000 7f6f858d7640 1 -- 172.21.15.169:0/3597458283 shutdown_connections 2023-10-26T01:43:56.302 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.299+0000 7f6f858d7640 1 --2- 172.21.15.169:0/3597458283 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f6f6006ae40 0x7f6f6006d300 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:56.302 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.299+0000 7f6f858d7640 1 --2- 172.21.15.169:0/3597458283 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f800cde00 0x7f6f8014a7f0 unknown :-1 s=CLOSED pgs=1856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:56.302 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.299+0000 7f6f858d7640 1 --2- 172.21.15.169:0/3597458283 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6f801592d0 0x7f6f800adfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:56.303 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.299+0000 7f6f858d7640 1 --2- 172.21.15.169:0/3597458283 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f800ac930 0x7f6f8015b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:56.303 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.299+0000 7f6f858d7640 1 -- 172.21.15.169:0/3597458283 >> 172.21.15.169:0/3597458283 conn(0x7f6f80069930 msgr2=0x7f6f800aeec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:56.303 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.299+0000 7f6f858d7640 1 -- 172.21.15.169:0/3597458283 shutdown_connections 2023-10-26T01:43:56.303 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.299+0000 7f6f858d7640 1 -- 172.21.15.169:0/3597458283 wait complete. 2023-10-26T01:43:56.720 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 94, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-26T01:43:49.711084+0000', 'modified': '2023-10-26T01:43:52.816785+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9123}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9123': {'gid': 9123, 'name': 'd', 'rank': 0, 'incarnation': 90, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.170:6837/2205518515', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6835', 'nonce': 2205518515}, {'type': 'v1', 'addr': '172.21.15.170:6837', 'nonce': 2205518515}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [23], 'metadata_pool': 22, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-26T01:43:56.721 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-26T01:43:56.721 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-10-26T01:43:56.958 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.955+0000 7f3771d75640 1 -- 172.21.15.169:0/3949245463 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f376c06cba0 msgr2=0x7f376c15b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:56.958 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.955+0000 7f3771d75640 1 --2- 172.21.15.169:0/3949245463 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f376c06cba0 0x7f376c15b140 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f3760005fd0 tx=0x7f376002bd90 comp rx=0 tx=0).stop 2023-10-26T01:43:56.958 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.956+0000 7f3771d75640 1 -- 172.21.15.169:0/3949245463 shutdown_connections 2023-10-26T01:43:56.958 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.956+0000 7f3771d75640 1 --2- 172.21.15.169:0/3949245463 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f376c06cba0 0x7f376c15b140 unknown :-1 s=CLOSED pgs=1858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:56.959 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.956+0000 7f3771d75640 1 -- 172.21.15.169:0/3949245463 >> 172.21.15.169:0/3949245463 conn(0x7f376c0af110 msgr2=0x7f376c0af520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:56.959 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.956+0000 7f3771d75640 1 -- 172.21.15.169:0/3949245463 shutdown_connections 2023-10-26T01:43:56.959 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.956+0000 7f3771d75640 1 -- 172.21.15.169:0/3949245463 wait complete. 2023-10-26T01:43:56.961 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.957+0000 7f3771d75640 1 Processor -- start 2023-10-26T01:43:56.962 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.957+0000 7f3771d75640 1 -- start start 2023-10-26T01:43:56.962 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.958+0000 7f3771d75640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f376c06a120 0x7f376c145210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:56.962 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.958+0000 7f3771d75640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f376c06cba0 0x7f376c06f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:56.962 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.958+0000 7f3771d75640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f376c0b0300 0x7f376c06fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:56.963 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.958+0000 7f3771d75640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f376c14e550 con 0x7f376c06cba0 2023-10-26T01:43:56.963 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.958+0000 7f3771d75640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f376c14df40 con 0x7f376c06a120 2023-10-26T01:43:56.963 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.958+0000 7f3771d75640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f376c148ef0 con 0x7f376c0b0300 2023-10-26T01:43:56.963 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.959+0000 7f376bfff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f376c06cba0 0x7f376c06f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:56.963 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.959+0000 7f376bfff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f376c06cba0 0x7f376c06f500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:46226/0 (socket says 172.21.15.169:46226) 2023-10-26T01:43:56.964 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.959+0000 7f376bfff640 1 -- 172.21.15.169:0/1019983631 learned_addr learned my addr 172.21.15.169:0/1019983631 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:43:56.964 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.959+0000 7f3770d73640 1 --2- 172.21.15.169:0/1019983631 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f376c06a120 0x7f376c145210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:56.965 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.959+0000 7f376bfff640 1 -- 172.21.15.169:0/1019983631 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f376c0b0300 msgr2=0x7f376c06fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:56.966 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.959+0000 7f3771574640 1 --2- 172.21.15.169:0/1019983631 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f376c0b0300 0x7f376c06fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:56.966 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.959+0000 7f376bfff640 1 --2- 172.21.15.169:0/1019983631 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f376c0b0300 0x7f376c06fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:56.966 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.959+0000 7f376bfff640 1 -- 172.21.15.169:0/1019983631 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f376c06a120 msgr2=0x7f376c145210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:56.966 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.959+0000 7f376bfff640 1 --2- 172.21.15.169:0/1019983631 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f376c06a120 0x7f376c145210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:56.967 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.959+0000 7f376bfff640 1 -- 172.21.15.169:0/1019983631 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3760009560 con 0x7f376c06cba0 2023-10-26T01:43:56.967 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.959+0000 7f3770d73640 1 --2- 172.21.15.169:0/1019983631 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f376c06a120 0x7f376c145210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:43:56.967 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.959+0000 7f3771574640 1 --2- 172.21.15.169:0/1019983631 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f376c0b0300 0x7f376c06fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:43:56.967 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.959+0000 7f376bfff640 1 --2- 172.21.15.169:0/1019983631 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f376c06cba0 0x7f376c06f500 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7f375800c410 tx=0x7f375800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:56.967 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.960+0000 7f37697fa640 1 -- 172.21.15.169:0/1019983631 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37580035f0 con 0x7f376c06cba0 2023-10-26T01:43:56.968 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.960+0000 7f37697fa640 1 -- 172.21.15.169:0/1019983631 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3758003750 con 0x7f376c06cba0 2023-10-26T01:43:56.968 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.960+0000 7f37697fa640 1 -- 172.21.15.169:0/1019983631 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3758003ba0 con 0x7f376c06cba0 2023-10-26T01:43:56.968 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.960+0000 7f3771d75640 1 -- 172.21.15.169:0/1019983631 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f376c1491d0 con 0x7f376c06cba0 2023-10-26T01:43:56.968 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.960+0000 7f3771d75640 1 -- 172.21.15.169:0/1019983631 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f376c148ba0 con 0x7f376c06cba0 2023-10-26T01:43:56.973 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.962+0000 7f37697fa640 1 -- 172.21.15.169:0/1019983631 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f3758003d00 con 0x7f376c06cba0 2023-10-26T01:43:56.973 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.963+0000 7f37697fa640 1 --2- 172.21.15.169:0/1019983631 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f375006ae40 0x7f375006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:56.973 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.963+0000 7f37697fa640 1 -- 172.21.15.169:0/1019983631 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3758088a70 con 0x7f376c06cba0 2023-10-26T01:43:56.974 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.963+0000 7f3775b69740 1 Processor -- start 2023-10-26T01:43:56.974 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.964+0000 7f3775b69740 1 -- start start 2023-10-26T01:43:56.974 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.964+0000 7f3770d73640 1 --2- 172.21.15.169:0/1019983631 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f375006ae40 0x7f375006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:56.974 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.964+0000 7f3775b69740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df1fb09970 0x55df1fadcba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:56.975 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.964+0000 7f3775b69740 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x55df1fb09d50 0x55df1fb0c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:56.975 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.964+0000 7f3775b69740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df1fadd0e0 0x55df1fb0cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:56.975 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.964+0000 7f3775b69740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df1fa4f740 con 0x55df1fb09970 2023-10-26T01:43:56.976 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.964+0000 7f3775b69740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df1fa4ce30 con 0x55df1fadd0e0 2023-10-26T01:43:56.976 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.964+0000 7f3775b69740 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x55df1fa48720 con 0x55df1fb09d50 2023-10-26T01:43:56.976 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.965+0000 7f3770d73640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df1fb09970 0x55df1fadcba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:56.976 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.965+0000 7f3770d73640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df1fb09970 0x55df1fadcba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:46236/0 (socket says 172.21.15.169:46236) 2023-10-26T01:43:56.976 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.965+0000 7f3770d73640 1 -- 172.21.15.169:0/4237657989 learned_addr learned my addr 172.21.15.169:0/4237657989 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:43:56.977 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.965+0000 7f3770d73640 1 --2- 172.21.15.169:0/1019983631 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f375006ae40 0x7f375006d300 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f3760035000 tx=0x7f3760004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:56.977 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.965+0000 7f376bfff640 1 --2- 172.21.15.169:0/4237657989 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x55df1fb09d50 0x55df1fb0c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:56.977 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.965+0000 7f3771574640 1 --2- 172.21.15.169:0/4237657989 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df1fadd0e0 0x55df1fb0cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:56.978 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.965+0000 7f3770d73640 1 -- 172.21.15.169:0/4237657989 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x55df1fb09d50 msgr2=0x55df1fb0c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T01:43:56.978 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.965+0000 7f3770d73640 1 --2- 172.21.15.169:0/4237657989 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x55df1fb09d50 0x55df1fb0c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:56.978 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.965+0000 7f3770d73640 1 -- 172.21.15.169:0/4237657989 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df1fadd0e0 msgr2=0x55df1fb0cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T01:43:56.978 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.965+0000 7f3770d73640 1 --2- 172.21.15.169:0/4237657989 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df1fadd0e0 0x55df1fb0cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:56.979 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.965+0000 7f3770d73640 1 -- 172.21.15.169:0/4237657989 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df1fa0c910 con 0x55df1fb09970 2023-10-26T01:43:56.979 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.966+0000 7f3770d73640 1 --2- 172.21.15.169:0/4237657989 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df1fb09970 0x55df1fadcba0 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7f3760000c00 tx=0x7f376003d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e95823950db3498f server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:56.979 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.966+0000 7f37567fc640 1 -- 172.21.15.169:0/4237657989 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f376002ce80 con 0x55df1fb09970 2023-10-26T01:43:56.979 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.966+0000 7f37567fc640 1 -- 172.21.15.169:0/4237657989 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3760044070 con 0x55df1fb09970 2023-10-26T01:43:56.980 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.966+0000 7f37567fc640 1 -- 172.21.15.169:0/4237657989 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37600037f0 con 0x55df1fb09970 2023-10-26T01:43:56.980 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.966+0000 7f3775b69740 1 -- 172.21.15.169:0/4237657989 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df1fb09970 msgr2=0x55df1fadcba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:56.980 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.966+0000 7f3775b69740 1 --2- 172.21.15.169:0/4237657989 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df1fb09970 0x55df1fadcba0 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7f3760000c00 tx=0x7f376003d000 comp rx=0 tx=0).stop 2023-10-26T01:43:56.980 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.966+0000 7f3775b69740 1 -- 172.21.15.169:0/4237657989 shutdown_connections 2023-10-26T01:43:56.981 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.966+0000 7f3775b69740 1 --2- 172.21.15.169:0/4237657989 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df1fadd0e0 0x55df1fb0cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:56.981 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.966+0000 7f3775b69740 1 --2- 172.21.15.169:0/4237657989 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x55df1fb09d50 0x55df1fb0c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:56.981 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.966+0000 7f3775b69740 1 --2- 172.21.15.169:0/4237657989 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df1fb09970 0x55df1fadcba0 unknown :-1 s=CLOSED pgs=1860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:56.981 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.966+0000 7f3775b69740 1 -- 172.21.15.169:0/4237657989 >> 172.21.15.169:0/4237657989 conn(0x55df1fadee10 msgr2=0x55df1fb27700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:56.982 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.967+0000 7f3775b69740 1 -- 172.21.15.169:0/4237657989 shutdown_connections 2023-10-26T01:43:56.982 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.967+0000 7f3775b69740 1 -- 172.21.15.169:0/4237657989 wait complete. 2023-10-26T01:43:56.982 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.967+0000 7f3775b69740 1 -- start start 2023-10-26T01:43:56.983 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.967+0000 7f3775b69740 1 Processor -- start 2023-10-26T01:43:56.983 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.968+0000 7f3775b69740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df1fb09970 0x55df1fb00cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:56.983 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.968+0000 7f3775b69740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df1fb09d50 0x55df1faf3470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:56.983 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.968+0000 7f3775b69740 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x55df1faf39b0 0x55df1fafaf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:56.984 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.968+0000 7f3775b69740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df1f9e6970 con 0x55df1fb09d50 2023-10-26T01:43:56.984 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.968+0000 7f3775b69740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df1f9e6610 con 0x55df1fb09970 2023-10-26T01:43:56.984 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.968+0000 7f3775b69740 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x55df1f7acf20 con 0x55df1faf39b0 2023-10-26T01:43:56.984 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.969+0000 7f3775b69740 10 client.0 _ll_register_callbacks cb 0x55df1fa0a0c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T01:43:56.985 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.969+0000 7f376bfff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df1fb09d50 0x55df1faf3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:56.985 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.969+0000 7f376bfff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df1fb09d50 0x55df1faf3470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:46250/0 (socket says 172.21.15.169:46250) 2023-10-26T01:43:56.985 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.969+0000 7f376bfff640 1 -- 172.21.15.169:0/3511137234 learned_addr learned my addr 172.21.15.169:0/3511137234 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:43:56.985 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.969+0000 7f3770d73640 1 --2- 172.21.15.169:0/3511137234 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df1fb09970 0x55df1fb00cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:56.986 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.969+0000 7f3771574640 1 --2- 172.21.15.169:0/3511137234 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x55df1faf39b0 0x55df1fafaf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:56.986 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.969+0000 7f376bfff640 1 -- 172.21.15.169:0/3511137234 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x55df1faf39b0 msgr2=0x55df1fafaf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T01:43:56.986 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.969+0000 7f376bfff640 1 --2- 172.21.15.169:0/3511137234 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x55df1faf39b0 0x55df1fafaf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:56.986 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.969+0000 7f376bfff640 1 -- 172.21.15.169:0/3511137234 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df1fb09970 msgr2=0x55df1fb00cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T01:43:56.987 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.969+0000 7f376bfff640 1 --2- 172.21.15.169:0/3511137234 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df1fb09970 0x55df1fb00cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:56.987 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.969+0000 7f376bfff640 1 -- 172.21.15.169:0/3511137234 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f376c15b8e0 con 0x55df1fb09d50 2023-10-26T01:43:56.987 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.969+0000 7f3770d73640 1 --2- 172.21.15.169:0/3511137234 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df1fb09970 0x55df1fb00cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:43:56.987 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.969+0000 7f376bfff640 1 --2- 172.21.15.169:0/3511137234 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df1fb09d50 0x55df1faf3470 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f3758053c40 tx=0x7f3758057880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e294dd822c654821 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:56.988 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.969+0000 7f374b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.169:3300/0 2023-10-26T01:43:56.988 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.970+0000 7f374b7fe640 1 -- 172.21.15.169:0/3511137234 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37580870d0 con 0x55df1fb09d50 2023-10-26T01:43:56.988 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.970+0000 7f374b7fe640 1 -- 172.21.15.169:0/3511137234 <== mon.0 v2:172.21.15.169:3300/0 2 ==== osd_map(182..182 src has 1..182) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3758053530 con 0x55df1fb09d50 2023-10-26T01:43:56.988 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.970+0000 7f3775b69740 1 -- 172.21.15.169:0/3511137234 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55df1fa41700 con 0x55df1fb09d50 2023-10-26T01:43:56.988 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.970+0000 7f374b7fe640 1 -- 172.21.15.169:0/3511137234 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_get_version_reply(handle=1 version=94) v2 ==== 24+0+0 (secure 0 0 0) 0x7f37580536f0 con 0x55df1fb09d50 2023-10-26T01:43:56.989 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.970+0000 7f3775b69740 10 client.9258 fetch_fsmap learned FSMap version 94 2023-10-26T01:43:56.989 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.970+0000 7f3775b69740 1 -- 172.21.15.169:0/3511137234 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,fsmap=94,monmap=2+}) v3 -- 0x55df1fa46da0 con 0x55df1fb09d50 2023-10-26T01:43:56.989 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.970+0000 7f374b7fe640 1 -- 172.21.15.169:0/3511137234 <== mon.0 v2:172.21.15.169:3300/0 4 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x55df1fa41700 con 0x55df1fb09d50 2023-10-26T01:43:56.989 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.970+0000 7f374b7fe640 1 -- 172.21.15.169:0/3511137234 <== mon.0 v2:172.21.15.169:3300/0 5 ==== fsmap(e 94) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f37580566a0 con 0x55df1fb09d50 2023-10-26T01:43:56.989 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.970+0000 7f3775b69740 10 client.9258 fetch_fsmap finished waiting for FSMap version 94 2023-10-26T01:43:56.990 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.970+0000 7f3775b69740 10 client.9258 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-26T01:43:56.990 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.970+0000 7f3775b69740 20 client.9258 populate_metadata read hostname 'smithi169' 2023-10-26T01:43:56.990 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.970+0000 7f3775b69740 1 --2- 172.21.15.169:0/3511137234 >> [v2:172.21.15.170:6835/2205518515,v1:172.21.15.170:6837/2205518515] conn(0x55df1fb0fea0 0x55df1fb0a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:56.990 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.970+0000 7f3775b69740 4 client.9258 mds_command: new command op to 9123 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-26T01:43:56.991 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.970+0000 7f3775b69740 1 -- 172.21.15.169:0/3511137234 --> [v2:172.21.15.170:6835/2205518515,v1:172.21.15.170:6837/2205518515] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55df1fa41420 con 0x55df1fb0fea0 2023-10-26T01:43:56.991 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.971+0000 7f3770d73640 1 --2- 172.21.15.169:0/3511137234 >> [v2:172.21.15.170:6835/2205518515,v1:172.21.15.170:6837/2205518515] conn(0x55df1fb0fea0 0x55df1fb0a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:56.992 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.972+0000 7f3770d73640 1 --2- 172.21.15.169:0/3511137234 >> [v2:172.21.15.170:6835/2205518515,v1:172.21.15.170:6837/2205518515] conn(0x55df1fb0fea0 0x55df1fb0a890 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7646dc1a7356d82 server_cookie=5baccad16bbd39b1 in_seq=0 out_seq=0 2023-10-26T01:43:56.992 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.972+0000 7f374b7fe640 10 client.9258 ms_handle_connect on v2:172.21.15.170:6835/2205518515 2023-10-26T01:43:56.992 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.974+0000 7f374b7fe640 1 -- 172.21.15.169:0/3511137234 <== mds.0 v2:172.21.15.170:6835/2205518515 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55df1fa41420 con 0x55df1fb0fea0 2023-10-26T01:43:56.993 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.974+0000 7f374b7fe640 10 client.9258 handle_command_reply: tid=0 2023-10-26T01:43:56.993 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.974+0000 7f3775b69740 1 client.9258 shutdown 2023-10-26T01:43:56.993 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.974+0000 7f3775b69740 1 -- 172.21.15.169:0/3511137234 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df1fb09d50 msgr2=0x55df1faf3470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:56.993 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.974+0000 7f3775b69740 1 --2- 172.21.15.169:0/3511137234 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df1fb09d50 0x55df1faf3470 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f3758053c40 tx=0x7f3758057880 comp rx=0 tx=0).stop 2023-10-26T01:43:56.993 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.974+0000 7f3775b69740 1 -- 172.21.15.169:0/3511137234 shutdown_connections 2023-10-26T01:43:56.994 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.974+0000 7f3775b69740 1 --2- 172.21.15.169:0/3511137234 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df1fb09d50 0x55df1faf3470 unknown :-1 s=CLOSED pgs=1861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:56.994 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.974+0000 7f3775b69740 1 --2- 172.21.15.169:0/3511137234 >> [v2:172.21.15.170:6835/2205518515,v1:172.21.15.170:6837/2205518515] conn(0x55df1fb0fea0 0x55df1fb0a890 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:56.994 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.974+0000 7f3775b69740 1 --2- 172.21.15.169:0/3511137234 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x55df1faf39b0 0x55df1fafaf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:56.994 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.974+0000 7f3775b69740 1 --2- 172.21.15.169:0/3511137234 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df1fb09970 0x55df1fb00cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:56.994 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.974+0000 7f374b7fe640 0 client.9258 ms_handle_reset on v2:172.21.15.170:6835/2205518515 2023-10-26T01:43:56.995 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.974+0000 7f3775b69740 1 -- 172.21.15.169:0/3511137234 >> 172.21.15.169:0/3511137234 conn(0x55df1fadd0e0 msgr2=0x55df1fae0930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:56.995 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.974+0000 7f3775b69740 1 -- 172.21.15.169:0/3511137234 shutdown_connections 2023-10-26T01:43:56.995 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.975+0000 7f3775b69740 1 -- 172.21.15.169:0/3511137234 wait complete. 2023-10-26T01:43:56.995 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.975+0000 7f3775b69740 20 client.9258 trim_cache size 0 max 16384 2023-10-26T01:43:56.998 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.991+0000 7f3775b69740 1 Processor -- start 2023-10-26T01:43:56.999 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.991+0000 7f3775b69740 1 -- start start 2023-10-26T01:43:56.999 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.992+0000 7f3775b69740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df1faf39b0 0x55df1fb01280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:56.999 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.992+0000 7f3775b69740 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x55df1fb09970 0x55df1fb2ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:56.999 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.992+0000 7f3775b69740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df1fb09d50 0x55df1fb2b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:56.999 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.992+0000 7f3775b69740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df1fb4e900 con 0x55df1faf39b0 2023-10-26T01:43:57.000 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.992+0000 7f3775b69740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df1fb0e630 con 0x55df1fb09d50 2023-10-26T01:43:57.000 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.992+0000 7f3775b69740 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x55df1f9e5950 con 0x55df1fb09970 2023-10-26T01:43:57.000 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.992+0000 7f3770d73640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df1faf39b0 0x55df1fb01280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:57.000 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.992+0000 7f3770d73640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df1faf39b0 0x55df1fb01280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:46258/0 (socket says 172.21.15.169:46258) 2023-10-26T01:43:57.000 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.992+0000 7f3770d73640 1 -- 172.21.15.169:0/574775474 learned_addr learned my addr 172.21.15.169:0/574775474 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:43:57.001 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.993+0000 7f3770d73640 1 -- 172.21.15.169:0/574775474 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x55df1fb09970 msgr2=0x55df1fb2ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T01:43:57.001 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.993+0000 7f3770d73640 1 --2- 172.21.15.169:0/574775474 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x55df1fb09970 0x55df1fb2ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:57.001 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.993+0000 7f3770d73640 1 -- 172.21.15.169:0/574775474 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df1fb09d50 msgr2=0x55df1fb2b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T01:43:57.001 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.993+0000 7f3770d73640 1 --2- 172.21.15.169:0/574775474 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df1fb09d50 0x55df1fb2b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:57.001 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.993+0000 7f3770d73640 1 -- 172.21.15.169:0/574775474 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df1f9e5b80 con 0x55df1faf39b0 2023-10-26T01:43:57.002 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.993+0000 7f3770d73640 1 --2- 172.21.15.169:0/574775474 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df1faf39b0 0x55df1fb01280 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7f376004d040 tx=0x7f376003f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cbcceb0ec58b14b server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:57.002 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.993+0000 7f3756ffd640 1 -- 172.21.15.169:0/574775474 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f376005d5a0 con 0x55df1faf39b0 2023-10-26T01:43:57.002 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.993+0000 7f3756ffd640 1 -- 172.21.15.169:0/574775474 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3760003690 con 0x55df1faf39b0 2023-10-26T01:43:57.002 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.993+0000 7f3756ffd640 1 -- 172.21.15.169:0/574775474 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3760003c30 con 0x55df1faf39b0 2023-10-26T01:43:57.002 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.994+0000 7f3775b69740 1 -- 172.21.15.169:0/574775474 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df1faf39b0 msgr2=0x55df1fb01280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:57.003 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.994+0000 7f3775b69740 1 --2- 172.21.15.169:0/574775474 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df1faf39b0 0x55df1fb01280 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7f376004d040 tx=0x7f376003f000 comp rx=0 tx=0).stop 2023-10-26T01:43:57.003 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.994+0000 7f3775b69740 1 -- 172.21.15.169:0/574775474 shutdown_connections 2023-10-26T01:43:57.003 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.994+0000 7f3775b69740 1 --2- 172.21.15.169:0/574775474 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df1fb09d50 0x55df1fb2b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:57.003 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.994+0000 7f3775b69740 1 --2- 172.21.15.169:0/574775474 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x55df1fb09970 0x55df1fb2ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:57.003 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.994+0000 7f3775b69740 1 --2- 172.21.15.169:0/574775474 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df1faf39b0 0x55df1fb01280 unknown :-1 s=CLOSED pgs=1862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:57.004 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.994+0000 7f3775b69740 1 -- 172.21.15.169:0/574775474 >> 172.21.15.169:0/574775474 conn(0x55df1fadd0e0 msgr2=0x55df1fb0d6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:57.004 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.994+0000 7f3775b69740 1 -- 172.21.15.169:0/574775474 shutdown_connections 2023-10-26T01:43:57.004 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.994+0000 7f3775b69740 1 -- 172.21.15.169:0/574775474 wait complete. 2023-10-26T01:43:57.004 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.995+0000 7f3775b69740 1 -- start start 2023-10-26T01:43:57.004 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.995+0000 7f3775b69740 1 Processor -- start 2023-10-26T01:43:57.004 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.996+0000 7f3775b69740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df1faf39b0 0x55df1fad8520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:57.005 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.996+0000 7f3775b69740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df1fb09970 0x55df1fb2ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:57.005 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.996+0000 7f3775b69740 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x55df1fb09d50 0x55df1fb2b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:57.005 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.996+0000 7f3775b69740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df1fb3aac0 con 0x55df1faf39b0 2023-10-26T01:43:57.005 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.996+0000 7f3775b69740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df1faffb70 con 0x55df1fb09970 2023-10-26T01:43:57.005 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.996+0000 7f3775b69740 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x55df1faffe50 con 0x55df1fb09d50 2023-10-26T01:43:57.006 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.996+0000 7f3775b69740 10 client.0 _ll_register_callbacks cb 0x7f3760009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T01:43:57.006 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.996+0000 7f3770d73640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df1faf39b0 0x55df1fad8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:57.006 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.996+0000 7f3770d73640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df1faf39b0 0x55df1fad8520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:46274/0 (socket says 172.21.15.169:46274) 2023-10-26T01:43:57.006 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.996+0000 7f3770d73640 1 -- 172.21.15.169:0/4053360601 learned_addr learned my addr 172.21.15.169:0/4053360601 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:43:57.007 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.997+0000 7f3770d73640 1 -- 172.21.15.169:0/4053360601 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x55df1fb09d50 msgr2=0x55df1fb2b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T01:43:57.007 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.997+0000 7f3770d73640 1 --2- 172.21.15.169:0/4053360601 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x55df1fb09d50 0x55df1fb2b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:57.007 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.997+0000 7f376bfff640 1 --2- 172.21.15.169:0/4053360601 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df1fb09970 0x55df1fb2ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:57.007 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.997+0000 7f3770d73640 1 -- 172.21.15.169:0/4053360601 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df1fb09970 msgr2=0x55df1fb2ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T01:43:57.007 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.997+0000 7f3770d73640 1 --2- 172.21.15.169:0/4053360601 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df1fb09970 0x55df1fb2ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:57.008 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.997+0000 7f3770d73640 1 -- 172.21.15.169:0/4053360601 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55df1fa0c910 con 0x55df1faf39b0 2023-10-26T01:43:57.008 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.997+0000 7f3770d73640 1 --2- 172.21.15.169:0/4053360601 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df1faf39b0 0x55df1fad8520 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f376002e010 tx=0x7f376005d500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f174c69e8d7bcc7 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:57.008 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.997+0000 7f374b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.169:3300/0 2023-10-26T01:43:57.011 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.997+0000 7f374b7fe640 1 -- 172.21.15.169:0/4053360601 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3760003de0 con 0x55df1faf39b0 2023-10-26T01:43:57.011 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.997+0000 7f374b7fe640 1 -- 172.21.15.169:0/4053360601 <== mon.0 v2:172.21.15.169:3300/0 2 ==== osd_map(182..182 src has 1..182) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f376005a040 con 0x55df1faf39b0 2023-10-26T01:43:57.011 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.997+0000 7f3775b69740 1 -- 172.21.15.169:0/4053360601 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3760044070 con 0x55df1faf39b0 2023-10-26T01:43:57.011 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.998+0000 7f374b7fe640 1 -- 172.21.15.169:0/4053360601 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_get_version_reply(handle=1 version=94) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3760003f40 con 0x55df1faf39b0 2023-10-26T01:43:57.011 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.998+0000 7f3775b69740 10 client.9264 fetch_fsmap learned FSMap version 94 2023-10-26T01:43:57.012 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.998+0000 7f3775b69740 1 -- 172.21.15.169:0/4053360601 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,fsmap=94,monmap=2+}) v3 -- 0x55df1fa0b170 con 0x55df1faf39b0 2023-10-26T01:43:57.012 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.998+0000 7f374b7fe640 1 -- 172.21.15.169:0/4053360601 <== mon.0 v2:172.21.15.169:3300/0 4 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3760044070 con 0x55df1faf39b0 2023-10-26T01:43:57.012 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.998+0000 7f374b7fe640 1 -- 172.21.15.169:0/4053360601 <== mon.0 v2:172.21.15.169:3300/0 5 ==== fsmap(e 94) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f376003c040 con 0x55df1faf39b0 2023-10-26T01:43:57.012 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.998+0000 7f3775b69740 10 client.9264 fetch_fsmap finished waiting for FSMap version 94 2023-10-26T01:43:57.012 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.998+0000 7f3775b69740 10 client.9264 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-26T01:43:57.013 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.998+0000 7f3775b69740 20 client.9264 populate_metadata read hostname 'smithi169' 2023-10-26T01:43:57.013 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.998+0000 7f3775b69740 1 --2- 172.21.15.169:0/4053360601 >> [v2:172.21.15.170:6835/2205518515,v1:172.21.15.170:6837/2205518515] conn(0x55df1fb456b0 0x55df1fb45a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:57.013 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.998+0000 7f3775b69740 4 client.9264 mds_command: new command op to 9123 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-26T01:43:57.013 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.998+0000 7f3775b69740 1 -- 172.21.15.169:0/4053360601 --> [v2:172.21.15.170:6835/2205518515,v1:172.21.15.170:6837/2205518515] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55df1f74a6c0 con 0x55df1fb456b0 2023-10-26T01:43:57.014 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:56.999+0000 7f376bfff640 1 --2- 172.21.15.169:0/4053360601 >> [v2:172.21.15.170:6835/2205518515,v1:172.21.15.170:6837/2205518515] conn(0x55df1fb456b0 0x55df1fb45a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:57.014 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:57.000+0000 7f376bfff640 1 --2- 172.21.15.169:0/4053360601 >> [v2:172.21.15.170:6835/2205518515,v1:172.21.15.170:6837/2205518515] conn(0x55df1fb456b0 0x55df1fb45a90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c181acc14afda17 server_cookie=2033ac3b40129e62 in_seq=0 out_seq=0 2023-10-26T01:43:57.014 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:57.000+0000 7f374b7fe640 10 client.9264 ms_handle_connect on v2:172.21.15.170:6835/2205518515 2023-10-26T01:43:57.014 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:57.000+0000 7f374b7fe640 1 -- 172.21.15.169:0/4053360601 <== mds.0 v2:172.21.15.170:6835/2205518515 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55df1f74a6c0 con 0x55df1fb456b0 2023-10-26T01:43:57.014 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:57.000+0000 7f374b7fe640 10 client.9264 handle_command_reply: tid=0 2023-10-26T01:43:57.015 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:57.000+0000 7f3775b69740 1 client.9264 shutdown 2023-10-26T01:43:57.015 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:57.001+0000 7f3775b69740 1 -- 172.21.15.169:0/4053360601 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df1faf39b0 msgr2=0x55df1fad8520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:57.015 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:57.001+0000 7f3775b69740 1 --2- 172.21.15.169:0/4053360601 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df1faf39b0 0x55df1fad8520 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f376002e010 tx=0x7f376005d500 comp rx=0 tx=0).stop 2023-10-26T01:43:57.016 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:57.001+0000 7f3775b69740 1 -- 172.21.15.169:0/4053360601 shutdown_connections 2023-10-26T01:43:57.016 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:57.001+0000 7f3775b69740 1 --2- 172.21.15.169:0/4053360601 >> [v2:172.21.15.170:6835/2205518515,v1:172.21.15.170:6837/2205518515] conn(0x55df1fb456b0 0x55df1fb45a90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:57.016 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:57.001+0000 7f3775b69740 1 --2- 172.21.15.169:0/4053360601 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x55df1fb09d50 0x55df1fb2b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:57.016 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:57.001+0000 7f3775b69740 1 --2- 172.21.15.169:0/4053360601 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df1fb09970 0x55df1fb2ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:57.016 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:57.001+0000 7f374b7fe640 0 client.9264 ms_handle_reset on v2:172.21.15.170:6835/2205518515 2023-10-26T01:43:57.017 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:57.001+0000 7f3775b69740 1 --2- 172.21.15.169:0/4053360601 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55df1faf39b0 0x55df1fad8520 unknown :-1 s=CLOSED pgs=1863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:57.017 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:57.001+0000 7f3775b69740 1 -- 172.21.15.169:0/4053360601 >> 172.21.15.169:0/4053360601 conn(0x55df1fadd0e0 msgr2=0x55df1fb0cc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:57.017 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:57.001+0000 7f3775b69740 1 -- 172.21.15.169:0/4053360601 shutdown_connections 2023-10-26T01:43:57.017 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:57.001+0000 7f3775b69740 1 -- 172.21.15.169:0/4053360601 wait complete. 2023-10-26T01:43:57.018 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:57.001+0000 7f3775b69740 20 client.9264 trim_cache size 0 max 16384 2023-10-26T01:43:57.018 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2023-10-26T01:43:57.018 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_fsid": "30203b28-ed8a-44e8-a9a6-a9bf25f980f1", 2023-10-26T01:43:57.019 INFO:teuthology.orchestra.run.smithi169.stdout: "whoami": 0, 2023-10-26T01:43:57.019 INFO:teuthology.orchestra.run.smithi169.stdout: "id": 9123, 2023-10-26T01:43:57.019 INFO:teuthology.orchestra.run.smithi169.stdout: "want_state": "up:active", 2023-10-26T01:43:57.020 INFO:teuthology.orchestra.run.smithi169.stdout: "state": "up:active", 2023-10-26T01:43:57.020 INFO:teuthology.orchestra.run.smithi169.stdout: "fs_name": "cephfs", 2023-10-26T01:43:57.020 INFO:teuthology.orchestra.run.smithi169.stdout: "rank_uptime": 6.6891216489999996, 2023-10-26T01:43:57.020 INFO:teuthology.orchestra.run.smithi169.stdout: "mdsmap_epoch": 94, 2023-10-26T01:43:57.021 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_epoch": 182, 2023-10-26T01:43:57.021 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_epoch_barrier": 182, 2023-10-26T01:43:57.021 INFO:teuthology.orchestra.run.smithi169.stdout: "uptime": 13.366520693 2023-10-26T01:43:57.021 INFO:teuthology.orchestra.run.smithi169.stdout:} 2023-10-26T01:43:57.023 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:57.002+0000 7f3771d75640 1 -- 172.21.15.169:0/1019983631 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f375006ae40 msgr2=0x7f375006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:57.023 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:57.002+0000 7f3771d75640 1 --2- 172.21.15.169:0/1019983631 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f375006ae40 0x7f375006d300 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f3760035000 tx=0x7f3760004730 comp rx=0 tx=0).stop 2023-10-26T01:43:57.023 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:57.002+0000 7f3771d75640 1 -- 172.21.15.169:0/1019983631 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f376c06cba0 msgr2=0x7f376c06f500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:57.024 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:57.002+0000 7f3771d75640 1 --2- 172.21.15.169:0/1019983631 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f376c06cba0 0x7f376c06f500 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7f375800c410 tx=0x7f375800c8d0 comp rx=0 tx=0).stop 2023-10-26T01:43:57.024 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:57.002+0000 7f3771d75640 1 -- 172.21.15.169:0/1019983631 shutdown_connections 2023-10-26T01:43:57.024 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:57.002+0000 7f3771d75640 1 --2- 172.21.15.169:0/1019983631 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f375006ae40 0x7f375006d300 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:57.024 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:57.002+0000 7f3771d75640 1 --2- 172.21.15.169:0/1019983631 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f376c06cba0 0x7f376c06f500 unknown :-1 s=CLOSED pgs=1859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:57.024 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:57.002+0000 7f3771d75640 1 --2- 172.21.15.169:0/1019983631 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f376c0b0300 0x7f376c06fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:57.025 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:57.002+0000 7f3771d75640 1 --2- 172.21.15.169:0/1019983631 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f376c06a120 0x7f376c145210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:57.025 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:57.002+0000 7f3771d75640 1 -- 172.21.15.169:0/1019983631 >> 172.21.15.169:0/1019983631 conn(0x7f376c0af110 msgr2=0x7f376c0af4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:57.025 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:57.002+0000 7f3771d75640 1 -- 172.21.15.169:0/1019983631 shutdown_connections 2023-10-26T01:43:57.025 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:57.002+0000 7f3771d75640 1 -- 172.21.15.169:0/1019983631 wait complete. 2023-10-26T01:43:57.026 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-26T01:43:57.240 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:57.238+0000 7f139ad5a640 1 -- 172.21.15.169:0/1365019973 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1384005e10 msgr2=0x7f1384006290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:57.240 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:57.238+0000 7f139ad5a640 1 --2- 172.21.15.169:0/1365019973 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1384005e10 0x7f1384006290 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f138c009ed0 tx=0x7f1384006900 comp rx=0 tx=0).stop 2023-10-26T01:43:57.240 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:57.238+0000 7f13992f7640 1 -- 172.21.15.169:0/1365019973 reap_dead start 2023-10-26T01:43:57.241 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:57.238+0000 7f139ad5a640 1 -- 172.21.15.169:0/1365019973 shutdown_connections 2023-10-26T01:43:57.241 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:57.238+0000 7f139ad5a640 1 -- 172.21.15.169:0/1365019973 >> 172.21.15.169:0/1365019973 conn(0x7f1394069930 msgr2=0x7f1394069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:57.241 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:57.238+0000 7f139ad5a640 1 -- 172.21.15.169:0/1365019973 shutdown_connections 2023-10-26T01:43:57.241 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:57.238+0000 7f139ad5a640 1 -- 172.21.15.169:0/1365019973 wait complete. 2023-10-26T01:43:57.242 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:57.240+0000 7f139ad5a640 1 Processor -- start 2023-10-26T01:43:57.243 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:57.240+0000 7f139ad5a640 1 -- start start 2023-10-26T01:43:57.243 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:57.241+0000 7f139ad5a640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1384005e10 0x7f13940afa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:57.243 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:57.241+0000 7f139ad5a640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f13940cca20 0x7f13941eefa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:57.243 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:57.241+0000 7f139ad5a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f139415a1c0 0x7f13941520e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:57.244 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:57.241+0000 7f139ad5a640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13940af050 con 0x7f13940cca20 2023-10-26T01:43:57.244 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:57.241+0000 7f139ad5a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13941560d0 con 0x7f139415a1c0 2023-10-26T01:43:57.244 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:57.241+0000 7f139ad5a640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13940af5d0 con 0x7f1384005e10 2023-10-26T01:43:57.244 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:57.241+0000 7f1393fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f13940cca20 0x7f13941eefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:57.244 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:57.241+0000 7f1393fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f13940cca20 0x7f13941eefa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:46290/0 (socket says 172.21.15.169:46290) 2023-10-26T01:43:57.245 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:57.241+0000 7f1393fff640 1 -- 172.21.15.169:0/4129280442 learned_addr learned my addr 172.21.15.169:0/4129280442 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:43:57.246 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:57.241+0000 7f1393fff640 1 -- 172.21.15.169:0/4129280442 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1384005e10 msgr2=0x7f13940afa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:57.247 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:57.241+0000 7f1398af6640 1 --2- 172.21.15.169:0/4129280442 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1384005e10 0x7f13940afa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:57.247 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:57.241+0000 7f13992f7640 1 --2- 172.21.15.169:0/4129280442 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f139415a1c0 0x7f13941520e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:57.247 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:57.242+0000 7f1393fff640 1 --2- 172.21.15.169:0/4129280442 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1384005e10 0x7f13940afa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:57.248 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:57.242+0000 7f1393fff640 1 -- 172.21.15.169:0/4129280442 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f139415a1c0 msgr2=0x7f13941520e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:57.248 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:57.242+0000 7f1393fff640 1 --2- 172.21.15.169:0/4129280442 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f139415a1c0 0x7f13941520e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:57.248 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:57.242+0000 7f1393fff640 1 -- 172.21.15.169:0/4129280442 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1394153fa0 con 0x7f13940cca20 2023-10-26T01:43:57.248 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:57.242+0000 7f1393fff640 1 --2- 172.21.15.169:0/4129280442 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f13940cca20 0x7f13941eefa0 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7f138000e9a0 tx=0x7f138000ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:57.248 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:57.242+0000 7f13917fa640 1 -- 172.21.15.169:0/4129280442 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f138000ccc0 con 0x7f13940cca20 2023-10-26T01:43:57.249 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:57.242+0000 7f13917fa640 1 -- 172.21.15.169:0/4129280442 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f138000ce20 con 0x7f13940cca20 2023-10-26T01:43:57.249 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:57.242+0000 7f139ad5a640 1 -- 172.21.15.169:0/4129280442 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1394153d00 con 0x7f13940cca20 2023-10-26T01:43:57.249 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:57.242+0000 7f13917fa640 1 -- 172.21.15.169:0/4129280442 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1380005280 con 0x7f13940cca20 2023-10-26T01:43:57.250 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:57.242+0000 7f139ad5a640 1 -- 172.21.15.169:0/4129280442 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1394156320 con 0x7f13940cca20 2023-10-26T01:43:57.250 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:57.243+0000 7f139ad5a640 1 -- 172.21.15.169:0/4129280442 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f139404db10 con 0x7f13940cca20 2023-10-26T01:43:57.250 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:57.244+0000 7f13917fa640 1 -- 172.21.15.169:0/4129280442 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f13800053e0 con 0x7f13940cca20 2023-10-26T01:43:57.251 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:57.245+0000 7f13917fa640 1 --2- 172.21.15.169:0/4129280442 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f137806ae70 0x7f137806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:57.251 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:57.245+0000 7f13917fa640 1 -- 172.21.15.169:0/4129280442 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1380014070 con 0x7f13940cca20 2023-10-26T01:43:57.251 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:57.246+0000 7f1398af6640 1 --2- 172.21.15.169:0/4129280442 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f137806ae70 0x7f137806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:57.252 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:57.247+0000 7f1398af6640 1 --2- 172.21.15.169:0/4129280442 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f137806ae70 0x7f137806d330 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f139406c5c0 tx=0x7f1388000ed0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:57.252 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:57.250+0000 7f13917fa640 1 -- 172.21.15.169:0/4129280442 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1380055030 con 0x7f13940cca20 2023-10-26T01:43:57.483 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:57.481+0000 7f139ad5a640 1 -- 172.21.15.169:0/4129280442 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f1394158c70 con 0x7f13940cca20 2023-10-26T01:43:57.483 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:57.481+0000 7f13917fa640 1 -- 172.21.15.169:0/4129280442 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v94) v1 ==== 53+0+73 (secure 0 0 0) 0x7f1380058650 con 0x7f13940cca20 2023-10-26T01:43:57.483 INFO:teuthology.orchestra.run.smithi169.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-26T01:43:57.486 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:57.484+0000 7f139ad5a640 1 -- 172.21.15.169:0/4129280442 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f137806ae70 msgr2=0x7f137806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:57.486 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:57.484+0000 7f139ad5a640 1 --2- 172.21.15.169:0/4129280442 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f137806ae70 0x7f137806d330 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f139406c5c0 tx=0x7f1388000ed0 comp rx=0 tx=0).stop 2023-10-26T01:43:57.486 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:57.484+0000 7f139ad5a640 1 -- 172.21.15.169:0/4129280442 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f13940cca20 msgr2=0x7f13941eefa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:57.487 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:57.484+0000 7f139ad5a640 1 --2- 172.21.15.169:0/4129280442 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f13940cca20 0x7f13941eefa0 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7f138000e9a0 tx=0x7f138000ee60 comp rx=0 tx=0).stop 2023-10-26T01:43:57.487 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:57.484+0000 7f139ad5a640 1 -- 172.21.15.169:0/4129280442 shutdown_connections 2023-10-26T01:43:57.487 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:57.484+0000 7f139ad5a640 1 --2- 172.21.15.169:0/4129280442 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f137806ae70 0x7f137806d330 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:57.487 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:57.484+0000 7f139ad5a640 1 --2- 172.21.15.169:0/4129280442 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f13940cca20 0x7f13941eefa0 unknown :-1 s=CLOSED pgs=1866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:57.488 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:57.484+0000 7f139ad5a640 1 --2- 172.21.15.169:0/4129280442 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f139415a1c0 0x7f13941520e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:57.488 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:57.484+0000 7f139ad5a640 1 --2- 172.21.15.169:0/4129280442 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1384005e10 0x7f13940afa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:57.488 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:57.484+0000 7f139ad5a640 1 -- 172.21.15.169:0/4129280442 >> 172.21.15.169:0/4129280442 conn(0x7f1394069930 msgr2=0x7f1394156960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:57.489 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:57.484+0000 7f139ad5a640 1 -- 172.21.15.169:0/4129280442 shutdown_connections 2023-10-26T01:43:57.489 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:57.484+0000 7f139ad5a640 1 -- 172.21.15.169:0/4129280442 wait complete. 2023-10-26T01:43:57.504 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-26T01:43:57.504 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-26T01:43:57.504 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi169.front.sepia.ceph.com 2023-10-26T01:43:57.505 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-26T01:43:57.505 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-26T01:43:57.505 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-26T01:43:57.506 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-26T01:43:57.506 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-26T01:43:57.507 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-26T01:43:57.507 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:43:57.507 DEBUG:teuthology.orchestra.run.smithi169:> ip addr 2023-10-26T01:43:57.561 INFO:teuthology.orchestra.run.smithi169.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-26T01:43:57.561 INFO:teuthology.orchestra.run.smithi169.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-26T01:43:57.561 INFO:teuthology.orchestra.run.smithi169.stdout: inet 127.0.0.1/8 scope host lo 2023-10-26T01:43:57.562 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:43:57.562 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 ::1/128 scope host 2023-10-26T01:43:57.562 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:43:57.562 INFO:teuthology.orchestra.run.smithi169.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-26T01:43:57.562 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:d9:3b:b4 brd ff:ff:ff:ff:ff:ff 2023-10-26T01:43:57.563 INFO:teuthology.orchestra.run.smithi169.stdout: altname enp7s0f0 2023-10-26T01:43:57.563 INFO:teuthology.orchestra.run.smithi169.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-26T01:43:57.563 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:d9:3b:b5 brd ff:ff:ff:ff:ff:ff 2023-10-26T01:43:57.563 INFO:teuthology.orchestra.run.smithi169.stdout: altname enp7s0f1 2023-10-26T01:43:57.563 INFO:teuthology.orchestra.run.smithi169.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-26T01:43:57.564 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:88:7f:4c brd ff:ff:ff:ff:ff:ff 2023-10-26T01:43:57.564 INFO:teuthology.orchestra.run.smithi169.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-26T01:43:57.564 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:88:7f:4d brd ff:ff:ff:ff:ff:ff 2023-10-26T01:43:57.564 INFO:teuthology.orchestra.run.smithi169.stdout: inet 172.21.15.169/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-26T01:43:57.564 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft 41658sec preferred_lft 41658sec 2023-10-26T01:43:57.565 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 fe80::ec4:7aff:fe88:7f4d/64 scope link 2023-10-26T01:43:57.565 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:43:57.565 INFO:teuthology.orchestra.run.smithi169.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-26T01:43:57.565 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 22:a0:6b:43:7b:04 brd ff:ff:ff:ff:ff:ff 2023-10-26T01:43:57.565 INFO:teuthology.orchestra.run.smithi169.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-26T01:43:57.566 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:43:57.566 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 fe80::d815:31ff:fe87:d148/64 scope link 2023-10-26T01:43:57.566 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:43:57.566 INFO:teuthology.orchestra.run.smithi169.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-26T01:43:57.566 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 2a:80:f8:45:5f:9f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-26T01:43:57.567 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 fe80::2880:f8ff:fe45:5f9f/64 scope link 2023-10-26T01:43:57.567 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:43:57.567 INFO:teuthology.orchestra.run.smithi169.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-26T01:43:57.567 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 22:a0:6b:43:7b:04 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-26T01:43:57.567 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 fe80::20a0:6bff:fe43:7b04/64 scope link 2023-10-26T01:43:57.568 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:43:57.569 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:43:57.569 DEBUG:teuthology.orchestra.run.smithi169:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-26T01:43:57.663 INFO:teuthology.orchestra.run.smithi169.stdout:1 2023-10-26T01:43:57.667 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:43:57.668 DEBUG:teuthology.orchestra.run.smithi169:> route 2023-10-26T01:43:57.724 INFO:teuthology.orchestra.run.smithi169.stdout:Kernel IP routing table 2023-10-26T01:43:57.724 INFO:teuthology.orchestra.run.smithi169.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-26T01:43:57.724 INFO:teuthology.orchestra.run.smithi169.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-26T01:43:57.725 INFO:teuthology.orchestra.run.smithi169.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-26T01:43:57.725 INFO:teuthology.orchestra.run.smithi169.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-26T01:43:57.726 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:43:57.727 DEBUG:teuthology.orchestra.run.smithi169:> (cd / && exec bash -c ' 2023-10-26T01:43:57.728 DEBUG:teuthology.orchestra.run.smithi169:> set -e 2023-10-26T01:43:57.728 DEBUG:teuthology.orchestra.run.smithi169:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-26T01:43:57.729 DEBUG:teuthology.orchestra.run.smithi169:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-26T01:43:57.729 DEBUG:teuthology.orchestra.run.smithi169:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-26T01:43:57.730 DEBUG:teuthology.orchestra.run.smithi169:> ') 2023-10-26T01:43:57.919 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:43:57.920 DEBUG:teuthology.orchestra.run.smithi169:> ip netns list 2023-10-26T01:43:57.973 INFO:teuthology.orchestra.run.smithi169.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-26T01:43:57.974 INFO:teuthology.orchestra.run.smithi169.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-26T01:43:57.974 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-26T01:43:57.974 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:43:57.975 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link set brx.0 up 2023-10-26T01:43:58.066 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-26T01:43:58.067 DEBUG:teuthology.orchestra.run.smithi169:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-26T01:43:58.121 INFO:teuthology.orchestra.run.smithi169.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-26T01:43:58.122 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-26T01:43:58.122 DEBUG:teuthology.orchestra.run.smithi169:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-26T01:43:58.173 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-26T01:43:58.174 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:43:58.174 DEBUG:teuthology.orchestra.run.smithi169:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-26T01:43:58.315 INFO:teuthology.orchestra.run.smithi169.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-26T01:43:58.316 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-26T01:43:58.316 INFO:teuthology.orchestra.run.smithi169.stdout:invalid new device string format 2023-10-26T01:43:58.336 INFO:teuthology.orchestra.run.smithi169.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-26T01:43:58.336 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-26T01:43:58.337 INFO:teuthology.orchestra.run.smithi169.stdout:invalid new device string format 2023-10-26T01:43:58.337 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: resolved to: "172.21.15.169:3300,172.21.15.170:3300,172.21.15.170:3301" 2023-10-26T01:43:58.337 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-26T01:43:58.361 INFO:teuthology.orchestra.run.smithi169.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-26T01:43:58.361 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-26T01:43:58.362 INFO:teuthology.orchestra.run.smithi169.stdout:invalid new device string format 2023-10-26T01:43:58.362 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: resolved to: "172.21.15.169:3300,172.21.15.170:3300,172.21.15.170:3301" 2023-10-26T01:43:58.362 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-26T01:43:58.363 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: resolved to: "172.21.15.169:6789,172.21.15.170:6789,172.21.15.170:6790" 2023-10-26T01:43:58.363 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.169:6789,172.21.15.170:6789,172.21.15.170:6790:/ 2023-10-26T01:43:58.363 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=30203b28-ed8a-44e8-a9a6-a9bf25f980f1" will pass to kernel 2023-10-26T01:43:58.364 INFO:teuthology.orchestra.run.smithi169.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2023-10-26T01:43:58.364 INFO:teuthology.orchestra.run.smithi169.stdout: You just mounted a file system that supports labels which does not 2023-10-26T01:43:58.364 INFO:teuthology.orchestra.run.smithi169.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-10-26T01:43:58.364 INFO:teuthology.orchestra.run.smithi169.stdout: applications will generate AVC messages and not be allowed access to 2023-10-26T01:43:58.365 INFO:teuthology.orchestra.run.smithi169.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-10-26T01:43:58.367 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-26T01:43:58.367 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:43:58.368 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-26T01:43:58.446 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:43:58.447 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-26T01:43:58.516 INFO:teuthology.orchestra.run.smithi169.stdout:client6044 2023-10-26T01:43:58.517 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:43:58.518 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:43:58.569 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-26T01:43:58.836 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:58.834+0000 7fdd78105640 1 -- 172.21.15.169:0/1854689566 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd701597c0 msgr2=0x7fdd7015dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:58.836 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:58.834+0000 7fdd78105640 1 --2- 172.21.15.169:0/1854689566 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd701597c0 0x7fdd7015dbf0 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7fdd700b1ea0 tx=0x7fdd6002bd70 comp rx=0 tx=0).stop 2023-10-26T01:43:58.836 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:58.834+0000 7fdd78105640 1 -- 172.21.15.169:0/1854689566 shutdown_connections 2023-10-26T01:43:58.837 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:58.834+0000 7fdd78105640 1 --2- 172.21.15.169:0/1854689566 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd701597c0 0x7fdd7015dbf0 unknown :-1 s=CLOSED pgs=1868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:58.837 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:58.834+0000 7fdd78105640 1 --2- 172.21.15.169:0/1854689566 >> v2:172.21.15.170:3300/0 conn(0x7fdd701593e0 0x7fdd700ac200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:58.837 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:58.834+0000 7fdd78105640 1 -- 172.21.15.169:0/1854689566 >> 172.21.15.169:0/1854689566 conn(0x7fdd70069930 msgr2=0x7fdd70069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:58.837 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:58.834+0000 7fdd78105640 1 -- 172.21.15.169:0/1854689566 shutdown_connections 2023-10-26T01:43:58.837 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:58.834+0000 7fdd78105640 1 -- 172.21.15.169:0/1854689566 wait complete. 2023-10-26T01:43:58.839 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:58.836+0000 7fdd78105640 1 Processor -- start 2023-10-26T01:43:58.839 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:58.836+0000 7fdd78105640 1 -- start start 2023-10-26T01:43:58.839 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:58.837+0000 7fdd78105640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdd700b0c80 0x7fdd70148410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:58.840 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:58.837+0000 7fdd78105640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd701593e0 0x7fdd701f4da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:58.840 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:58.837+0000 7fdd78105640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd701597c0 0x7fdd701f52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:58.840 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:58.837+0000 7fdd78105640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd70158030 con 0x7fdd701593e0 2023-10-26T01:43:58.840 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:58.837+0000 7fdd78105640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd7014fc80 con 0x7fdd701597c0 2023-10-26T01:43:58.840 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:58.837+0000 7fdd78105640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd700ae5a0 con 0x7fdd700b0c80 2023-10-26T01:43:58.841 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:58.837+0000 7fdd756a0640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd701593e0 0x7fdd701f4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:58.841 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:58.837+0000 7fdd756a0640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd701593e0 0x7fdd701f4da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:46296/0 (socket says 172.21.15.169:46296) 2023-10-26T01:43:58.841 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:58.837+0000 7fdd756a0640 1 -- 172.21.15.169:0/1241214851 learned_addr learned my addr 172.21.15.169:0/1241214851 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:43:58.841 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:58.837+0000 7fdd756a0640 1 -- 172.21.15.169:0/1241214851 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdd700b0c80 msgr2=0x7fdd70148410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:58.841 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:58.837+0000 7fdd756a0640 1 --2- 172.21.15.169:0/1241214851 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdd700b0c80 0x7fdd70148410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:58.842 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:58.837+0000 7fdd756a0640 1 -- 172.21.15.169:0/1241214851 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd701597c0 msgr2=0x7fdd701f52e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:58.842 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:58.837+0000 7fdd766a2640 1 --2- 172.21.15.169:0/1241214851 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd701597c0 0x7fdd701f52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:58.842 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:58.837+0000 7fdd756a0640 1 --2- 172.21.15.169:0/1241214851 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd701597c0 0x7fdd701f52e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:58.842 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:58.837+0000 7fdd756a0640 1 -- 172.21.15.169:0/1241214851 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd60009490 con 0x7fdd701593e0 2023-10-26T01:43:58.843 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:58.838+0000 7fdd756a0640 1 --2- 172.21.15.169:0/1241214851 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd701593e0 0x7fdd701f4da0 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7fdd60009780 tx=0x7fdd600038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:58.844 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:58.838+0000 7fdd667fc640 1 -- 172.21.15.169:0/1241214851 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd60003ea0 con 0x7fdd701593e0 2023-10-26T01:43:58.844 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:58.838+0000 7fdd667fc640 1 -- 172.21.15.169:0/1241214851 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fdd60005020 con 0x7fdd701593e0 2023-10-26T01:43:58.844 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:58.838+0000 7fdd667fc640 1 -- 172.21.15.169:0/1241214851 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd600053f0 con 0x7fdd701593e0 2023-10-26T01:43:58.844 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:58.838+0000 7fdd78105640 1 -- 172.21.15.169:0/1241214851 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd70158280 con 0x7fdd701593e0 2023-10-26T01:43:58.844 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:58.838+0000 7fdd78105640 1 -- 172.21.15.169:0/1241214851 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd700ae820 con 0x7fdd701593e0 2023-10-26T01:43:58.844 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:58.839+0000 7fdd78105640 1 -- 172.21.15.169:0/1241214851 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd701587a0 con 0x7fdd701593e0 2023-10-26T01:43:58.845 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:58.840+0000 7fdd667fc640 1 -- 172.21.15.169:0/1241214851 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fdd600047a0 con 0x7fdd701593e0 2023-10-26T01:43:58.845 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:58.842+0000 7fdd667fc640 1 --2- 172.21.15.169:0/1241214851 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fdd5406ae70 0x7fdd5406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:43:58.845 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:58.842+0000 7fdd667fc640 1 -- 172.21.15.169:0/1241214851 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdd600abbd0 con 0x7fdd701593e0 2023-10-26T01:43:58.846 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:58.842+0000 7fdd75ea1640 1 --2- 172.21.15.169:0/1241214851 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fdd5406ae70 0x7fdd5406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:43:58.846 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:58.843+0000 7fdd75ea1640 1 --2- 172.21.15.169:0/1241214851 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fdd5406ae70 0x7fdd5406d330 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c005fd0 tx=0x7fdd6c005850 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:43:58.848 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:58.847+0000 7fdd667fc640 1 -- 172.21.15.169:0/1241214851 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdd60076b60 con 0x7fdd701593e0 2023-10-26T01:43:59.080 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:59.078+0000 7fdd78105640 1 -- 172.21.15.169:0/1241214851 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fdd700ce040 con 0x7fdd701593e0 2023-10-26T01:43:59.080 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:59.079+0000 7fdd667fc640 1 -- 172.21.15.169:0/1241214851 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v94) v1 ==== 53+0+73 (secure 0 0 0) 0x7fdd6007a180 con 0x7fdd701593e0 2023-10-26T01:43:59.081 INFO:teuthology.orchestra.run.smithi169.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-26T01:43:59.084 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:59.081+0000 7fdd78105640 1 -- 172.21.15.169:0/1241214851 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fdd5406ae70 msgr2=0x7fdd5406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:59.084 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:59.081+0000 7fdd78105640 1 --2- 172.21.15.169:0/1241214851 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fdd5406ae70 0x7fdd5406d330 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c005fd0 tx=0x7fdd6c005850 comp rx=0 tx=0).stop 2023-10-26T01:43:59.084 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:59.081+0000 7fdd78105640 1 -- 172.21.15.169:0/1241214851 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd701593e0 msgr2=0x7fdd701f4da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:43:59.084 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:59.081+0000 7fdd78105640 1 --2- 172.21.15.169:0/1241214851 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd701593e0 0x7fdd701f4da0 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7fdd60009780 tx=0x7fdd600038a0 comp rx=0 tx=0).stop 2023-10-26T01:43:59.085 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:59.082+0000 7fdd78105640 1 -- 172.21.15.169:0/1241214851 shutdown_connections 2023-10-26T01:43:59.085 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:59.082+0000 7fdd78105640 1 --2- 172.21.15.169:0/1241214851 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fdd5406ae70 0x7fdd5406d330 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:59.085 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:59.082+0000 7fdd78105640 1 --2- 172.21.15.169:0/1241214851 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd701593e0 0x7fdd701f4da0 unknown :-1 s=CLOSED pgs=1869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:59.085 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:59.082+0000 7fdd78105640 1 --2- 172.21.15.169:0/1241214851 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd701597c0 0x7fdd701f52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:59.085 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:59.082+0000 7fdd78105640 1 --2- 172.21.15.169:0/1241214851 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdd700b0c80 0x7fdd70148410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:43:59.086 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:59.082+0000 7fdd78105640 1 -- 172.21.15.169:0/1241214851 >> 172.21.15.169:0/1241214851 conn(0x7fdd70069930 msgr2=0x7fdd7006c0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:43:59.086 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:59.082+0000 7fdd78105640 1 -- 172.21.15.169:0/1241214851 shutdown_connections 2023-10-26T01:43:59.086 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:43:59.082+0000 7fdd78105640 1 -- 172.21.15.169:0/1241214851 wait complete. 2023-10-26T01:43:59.102 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-26T01:43:59.103 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-26T01:43:59.103 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi170.front.sepia.ceph.com 2023-10-26T01:43:59.103 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-26T01:43:59.104 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-26T01:43:59.104 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-26T01:43:59.104 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-26T01:43:59.105 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-26T01:43:59.105 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-26T01:43:59.105 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:43:59.106 DEBUG:teuthology.orchestra.run.smithi170:> ip addr 2023-10-26T01:43:59.168 INFO:teuthology.orchestra.run.smithi170.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-26T01:43:59.168 INFO:teuthology.orchestra.run.smithi170.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-26T01:43:59.169 INFO:teuthology.orchestra.run.smithi170.stdout: inet 127.0.0.1/8 scope host lo 2023-10-26T01:43:59.169 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:43:59.169 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 ::1/128 scope host 2023-10-26T01:43:59.169 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:43:59.169 INFO:teuthology.orchestra.run.smithi170.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-26T01:43:59.169 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:d9:3b:54 brd ff:ff:ff:ff:ff:ff 2023-10-26T01:43:59.170 INFO:teuthology.orchestra.run.smithi170.stdout: altname enp7s0f0 2023-10-26T01:43:59.170 INFO:teuthology.orchestra.run.smithi170.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-26T01:43:59.170 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:d9:3b:55 brd ff:ff:ff:ff:ff:ff 2023-10-26T01:43:59.170 INFO:teuthology.orchestra.run.smithi170.stdout: altname enp7s0f1 2023-10-26T01:43:59.170 INFO:teuthology.orchestra.run.smithi170.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-26T01:43:59.171 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:88:72:54 brd ff:ff:ff:ff:ff:ff 2023-10-26T01:43:59.171 INFO:teuthology.orchestra.run.smithi170.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-26T01:43:59.171 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:88:72:55 brd ff:ff:ff:ff:ff:ff 2023-10-26T01:43:59.171 INFO:teuthology.orchestra.run.smithi170.stdout: inet 172.21.15.170/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-26T01:43:59.171 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft 41641sec preferred_lft 41641sec 2023-10-26T01:43:59.172 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::ec4:7aff:fe88:7255/64 scope link 2023-10-26T01:43:59.172 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:43:59.172 INFO:teuthology.orchestra.run.smithi170.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-26T01:43:59.172 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 32:b5:d1:ee:10:36 brd ff:ff:ff:ff:ff:ff 2023-10-26T01:43:59.172 INFO:teuthology.orchestra.run.smithi170.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-26T01:43:59.172 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:43:59.173 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::28ed:d1ff:fe37:70d/64 scope link 2023-10-26T01:43:59.173 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:43:59.173 INFO:teuthology.orchestra.run.smithi170.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-26T01:43:59.173 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 32:b5:d1:ee:10:36 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-26T01:43:59.173 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::30b5:d1ff:feee:1036/64 scope link 2023-10-26T01:43:59.174 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:43:59.174 INFO:teuthology.orchestra.run.smithi170.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-26T01:43:59.174 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether b2:c1:53:19:69:e4 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-26T01:43:59.174 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::b0c1:53ff:fe19:69e4/64 scope link 2023-10-26T01:43:59.174 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-26T01:43:59.175 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:43:59.176 DEBUG:teuthology.orchestra.run.smithi170:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-26T01:43:59.268 INFO:teuthology.orchestra.run.smithi170.stdout:1 2023-10-26T01:43:59.273 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:43:59.273 DEBUG:teuthology.orchestra.run.smithi170:> route 2023-10-26T01:43:59.329 INFO:teuthology.orchestra.run.smithi170.stdout:Kernel IP routing table 2023-10-26T01:43:59.329 INFO:teuthology.orchestra.run.smithi170.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-26T01:43:59.329 INFO:teuthology.orchestra.run.smithi170.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-26T01:43:59.329 INFO:teuthology.orchestra.run.smithi170.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-26T01:43:59.330 INFO:teuthology.orchestra.run.smithi170.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-26T01:43:59.330 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:43:59.331 DEBUG:teuthology.orchestra.run.smithi170:> (cd / && exec bash -c ' 2023-10-26T01:43:59.331 DEBUG:teuthology.orchestra.run.smithi170:> set -e 2023-10-26T01:43:59.331 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-26T01:43:59.332 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-26T01:43:59.332 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-26T01:43:59.332 DEBUG:teuthology.orchestra.run.smithi170:> ') 2023-10-26T01:43:59.514 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:43:59.515 DEBUG:teuthology.orchestra.run.smithi170:> ip netns list 2023-10-26T01:43:59.566 INFO:teuthology.orchestra.run.smithi170.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-26T01:43:59.566 INFO:teuthology.orchestra.run.smithi170.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-26T01:43:59.566 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-26T01:43:59.567 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:43:59.567 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip link set brx.0 up 2023-10-26T01:43:59.659 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-26T01:43:59.660 DEBUG:teuthology.orchestra.run.smithi170:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-26T01:43:59.717 INFO:teuthology.orchestra.run.smithi170.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-26T01:43:59.717 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-26T01:43:59.718 DEBUG:teuthology.orchestra.run.smithi170:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-26T01:43:59.769 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-26T01:43:59.769 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:43:59.770 DEBUG:teuthology.orchestra.run.smithi170:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-26T01:43:59.907 INFO:teuthology.orchestra.run.smithi170.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-26T01:43:59.907 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-26T01:43:59.908 INFO:teuthology.orchestra.run.smithi170.stdout:invalid new device string format 2023-10-26T01:43:59.929 INFO:teuthology.orchestra.run.smithi170.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-26T01:43:59.930 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-26T01:43:59.930 INFO:teuthology.orchestra.run.smithi170.stdout:invalid new device string format 2023-10-26T01:43:59.930 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: resolved to: "172.21.15.169:3300,172.21.15.170:3300,172.21.15.170:3301" 2023-10-26T01:43:59.930 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-26T01:43:59.952 INFO:teuthology.orchestra.run.smithi170.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-26T01:43:59.952 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-26T01:43:59.952 INFO:teuthology.orchestra.run.smithi170.stdout:invalid new device string format 2023-10-26T01:43:59.953 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: resolved to: "172.21.15.169:3300,172.21.15.170:3300,172.21.15.170:3301" 2023-10-26T01:43:59.953 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-26T01:43:59.953 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: resolved to: "172.21.15.169:6789,172.21.15.170:6789,172.21.15.170:6790" 2023-10-26T01:43:59.953 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.169:6789,172.21.15.170:6789,172.21.15.170:6790:/ 2023-10-26T01:43:59.953 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=30203b28-ed8a-44e8-a9a6-a9bf25f980f1" will pass to kernel 2023-10-26T01:43:59.954 INFO:teuthology.orchestra.run.smithi170.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2023-10-26T01:43:59.954 INFO:teuthology.orchestra.run.smithi170.stdout: You just mounted a file system that supports labels which does not 2023-10-26T01:43:59.954 INFO:teuthology.orchestra.run.smithi170.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-10-26T01:43:59.954 INFO:teuthology.orchestra.run.smithi170.stdout: applications will generate AVC messages and not be allowed access to 2023-10-26T01:43:59.955 INFO:teuthology.orchestra.run.smithi170.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-10-26T01:43:59.958 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-26T01:43:59.959 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:43:59.959 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-26T01:44:00.043 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:44:00.044 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-26T01:44:00.103 INFO:teuthology.orchestra.run.smithi170.stdout:client6047 2023-10-26T01:44:00.104 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-26T01:44:00.104 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:44:00.156 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T01:44:00.387 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:00.385+0000 7f5f348aa640 1 -- 172.21.15.169:0/300834853 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f300cc4f0 msgr2=0x7f5f300cc970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:00.388 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:00.385+0000 7f5f348aa640 1 --2- 172.21.15.169:0/300834853 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f300cc4f0 0x7f5f300cc970 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7f5f180092b0 tx=0x7f5f1802bcb0 comp rx=0 tx=0).stop 2023-10-26T01:44:00.388 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:00.385+0000 7f5f348aa640 1 -- 172.21.15.169:0/300834853 shutdown_connections 2023-10-26T01:44:00.388 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:00.385+0000 7f5f348aa640 1 --2- 172.21.15.169:0/300834853 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f300cc4f0 0x7f5f300cc970 unknown :-1 s=CLOSED pgs=1871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:00.388 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:00.385+0000 7f5f348aa640 1 --2- 172.21.15.169:0/300834853 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5f300c7ea0 0x7f5f3006a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:00.388 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:00.385+0000 7f5f348aa640 1 -- 172.21.15.169:0/300834853 >> 172.21.15.169:0/300834853 conn(0x7f5f300aedd0 msgr2=0x7f5f300af1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:00.389 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:00.385+0000 7f5f348aa640 1 -- 172.21.15.169:0/300834853 shutdown_connections 2023-10-26T01:44:00.389 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:00.385+0000 7f5f348aa640 1 -- 172.21.15.169:0/300834853 wait complete. 2023-10-26T01:44:00.389 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:00.387+0000 7f5f348aa640 1 Processor -- start 2023-10-26T01:44:00.390 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:00.387+0000 7f5f348aa640 1 -- start start 2023-10-26T01:44:00.391 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:00.388+0000 7f5f348aa640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5f300c7ea0 0x7f5f30146850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:00.392 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:00.388+0000 7f5f348aa640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5f300c96f0 0x7f5f300b28a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:00.392 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:00.388+0000 7f5f348aa640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f300cc4f0 0x7f5f300abd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:00.392 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:00.388+0000 7f5f348aa640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f300ab950 con 0x7f5f300cc4f0 2023-10-26T01:44:00.392 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:00.388+0000 7f5f348aa640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f30146d90 con 0x7f5f300c7ea0 2023-10-26T01:44:00.392 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:00.388+0000 7f5f348aa640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f300aacd0 con 0x7f5f300c96f0 2023-10-26T01:44:00.393 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:00.389+0000 7f5f2ffff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f300cc4f0 0x7f5f300abd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:00.393 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:00.389+0000 7f5f2ffff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f300cc4f0 0x7f5f300abd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:46316/0 (socket says 172.21.15.169:46316) 2023-10-26T01:44:00.393 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:00.389+0000 7f5f2ffff640 1 -- 172.21.15.169:0/1014701577 learned_addr learned my addr 172.21.15.169:0/1014701577 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:44:00.393 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:00.389+0000 7f5f2ffff640 1 -- 172.21.15.169:0/1014701577 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5f300c96f0 msgr2=0x7f5f300b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:00.393 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:00.389+0000 7f5f2f7fe640 1 --2- 172.21.15.169:0/1014701577 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5f300c7ea0 0x7f5f30146850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:00.394 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:00.389+0000 7f5f2ffff640 1 --2- 172.21.15.169:0/1014701577 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5f300c96f0 0x7f5f300b28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:00.394 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:00.389+0000 7f5f2ffff640 1 -- 172.21.15.169:0/1014701577 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5f300c7ea0 msgr2=0x7f5f30146850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:00.394 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:00.389+0000 7f5f2ffff640 1 --2- 172.21.15.169:0/1014701577 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5f300c7ea0 0x7f5f30146850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:00.394 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:00.389+0000 7f5f2ffff640 1 -- 172.21.15.169:0/1014701577 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f18009d20 con 0x7f5f300cc4f0 2023-10-26T01:44:00.394 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:00.389+0000 7f5f2f7fe640 1 --2- 172.21.15.169:0/1014701577 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5f300c7ea0 0x7f5f30146850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:44:00.395 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:00.389+0000 7f5f2ffff640 1 --2- 172.21.15.169:0/1014701577 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f300cc4f0 0x7f5f300abd70 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f5f2800c410 tx=0x7f5f2800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:00.395 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:00.390+0000 7f5f27fff640 1 -- 172.21.15.169:0/1014701577 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f28011790 con 0x7f5f300cc4f0 2023-10-26T01:44:00.395 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:00.390+0000 7f5f27fff640 1 -- 172.21.15.169:0/1014701577 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5f280119e0 con 0x7f5f300cc4f0 2023-10-26T01:44:00.395 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:00.390+0000 7f5f27fff640 1 -- 172.21.15.169:0/1014701577 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f28011df0 con 0x7f5f300cc4f0 2023-10-26T01:44:00.395 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:00.390+0000 7f5f348aa640 1 -- 172.21.15.169:0/1014701577 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f300aafb0 con 0x7f5f300cc4f0 2023-10-26T01:44:00.396 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:00.390+0000 7f5f348aa640 1 -- 172.21.15.169:0/1014701577 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f300ab570 con 0x7f5f300cc4f0 2023-10-26T01:44:00.396 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:00.391+0000 7f5f348aa640 1 -- 172.21.15.169:0/1014701577 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f300aba90 con 0x7f5f300cc4f0 2023-10-26T01:44:00.397 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:00.392+0000 7f5f27fff640 1 -- 172.21.15.169:0/1014701577 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f5f2801c460 con 0x7f5f300cc4f0 2023-10-26T01:44:00.397 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:00.393+0000 7f5f27fff640 1 --2- 172.21.15.169:0/1014701577 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5f1006ae40 0x7f5f1006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:00.397 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:00.393+0000 7f5f27fff640 1 -- 172.21.15.169:0/1014701577 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5f28089870 con 0x7f5f300cc4f0 2023-10-26T01:44:00.397 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:00.394+0000 7f5f2f7fe640 1 --2- 172.21.15.169:0/1014701577 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5f1006ae40 0x7f5f1006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:00.397 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:00.395+0000 7f5f2f7fe640 1 --2- 172.21.15.169:0/1014701577 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5f1006ae40 0x7f5f1006d300 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f5f20005fd0 tx=0x7f5f200093e0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:00.399 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:00.397+0000 7f5f27fff640 1 -- 172.21.15.169:0/1014701577 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5f300aba90 con 0x7f5f300cc4f0 2023-10-26T01:44:00.625 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:00.623+0000 7f5f348aa640 1 -- 172.21.15.169:0/1014701577 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5f300aba90 con 0x7f5f300cc4f0 2023-10-26T01:44:00.626 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:00.624+0000 7f5f27fff640 1 -- 172.21.15.169:0/1014701577 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) v1 ==== 94+0+4920 (secure 0 0 0) 0x7f5f300aba90 con 0x7f5f300cc4f0 2023-10-26T01:44:00.627 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:44:00.627 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":94,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6484,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/195700049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":195700049},{"type":"v1","addr":"172.21.15.169:6835","nonce":195700049}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":89},{"gid":6493,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/1167696300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1167696300},{"type":"v1","addr":"172.21.15.170:6836","nonce":1167696300}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":89},{"gid":9111,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/22729060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":22729060},{"type":"v1","addr":"172.21.15.169:6834","nonce":22729060}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":89}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T01:43:49.711084+0000","modified":"2023-10-26T01:43:52.816785+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9123},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9123":{"gid":9123,"name":"d","rank":0,"incarnation":90,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6837/2205518515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":2205518515},{"type":"v1","addr":"172.21.15.170:6837","nonce":2205518515}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2023-10-26T01:44:00.627 INFO:teuthology.orchestra.run.smithi169.stderr:dumped fsmap epoch 94 2023-10-26T01:44:00.629 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:00.627+0000 7f5f348aa640 1 -- 172.21.15.169:0/1014701577 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5f1006ae40 msgr2=0x7f5f1006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:00.629 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:00.627+0000 7f5f348aa640 1 --2- 172.21.15.169:0/1014701577 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5f1006ae40 0x7f5f1006d300 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f5f20005fd0 tx=0x7f5f200093e0 comp rx=0 tx=0).stop 2023-10-26T01:44:00.629 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:00.627+0000 7f5f348aa640 1 -- 172.21.15.169:0/1014701577 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f300cc4f0 msgr2=0x7f5f300abd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:00.630 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:00.627+0000 7f5f348aa640 1 --2- 172.21.15.169:0/1014701577 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f300cc4f0 0x7f5f300abd70 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f5f2800c410 tx=0x7f5f2800c8d0 comp rx=0 tx=0).stop 2023-10-26T01:44:00.630 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:00.627+0000 7f5f348aa640 1 -- 172.21.15.169:0/1014701577 shutdown_connections 2023-10-26T01:44:00.630 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:00.627+0000 7f5f348aa640 1 --2- 172.21.15.169:0/1014701577 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5f1006ae40 0x7f5f1006d300 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:00.630 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:00.627+0000 7f5f348aa640 1 --2- 172.21.15.169:0/1014701577 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f300cc4f0 0x7f5f300abd70 unknown :-1 s=CLOSED pgs=1872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:00.630 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:00.627+0000 7f5f348aa640 1 --2- 172.21.15.169:0/1014701577 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5f300c96f0 0x7f5f300b28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:00.631 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:00.627+0000 7f5f348aa640 1 --2- 172.21.15.169:0/1014701577 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5f300c7ea0 0x7f5f30146850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:00.631 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:00.627+0000 7f5f348aa640 1 -- 172.21.15.169:0/1014701577 >> 172.21.15.169:0/1014701577 conn(0x7f5f300aedd0 msgr2=0x7f5f30158cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:00.631 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:00.628+0000 7f5f348aa640 1 -- 172.21.15.169:0/1014701577 shutdown_connections 2023-10-26T01:44:00.631 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:00.628+0000 7f5f348aa640 1 -- 172.21.15.169:0/1014701577 wait complete. 2023-10-26T01:44:00.648 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json status 2023-10-26T01:44:00.887 INFO:teuthology.orchestra.run.smithi170.stdout:{"cluster_fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","whoami":0,"id":9123,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":10.558708425000001,"mdsmap_epoch":94,"osdmap_epoch":182,"osdmap_epoch_barrier":182,"uptime":17.236112951999999} 2023-10-26T01:44:00.888 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "30203b28-ed8a-44e8-a9a6-a9bf25f980f1", "fs_name": "cephfs", "id": 9123, "mdsmap_epoch": 94, "osdmap_epoch": 182, "osdmap_epoch_barrier": 182, "rank_uptime": 10.558708425, "state": "up:active", "uptime": 17.236112952, "want_state": "up:active", "whoami": 0 } 2023-10-26T01:44:00.889 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set-quota cephfs_data max_bytes 33554432 2023-10-26T01:44:01.122 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:01.120+0000 7f0d80e2e640 1 -- 172.21.15.169:0/533911779 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0d7c06ac80 msgr2=0x7f0d7c06b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:01.122 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:01.120+0000 7f0d80e2e640 1 --2- 172.21.15.169:0/533911779 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0d7c06ac80 0x7f0d7c06b090 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c06a850 tx=0x7f0d7402bd30 comp rx=0 tx=0).stop 2023-10-26T01:44:01.123 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:01.120+0000 7f0d80e2e640 1 -- 172.21.15.169:0/533911779 shutdown_connections 2023-10-26T01:44:01.123 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:01.120+0000 7f0d80e2e640 1 --2- 172.21.15.169:0/533911779 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0d7c06ac80 0x7f0d7c06b090 unknown :-1 s=CLOSED pgs=1873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:01.123 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:01.120+0000 7f0d80e2e640 1 --2- 172.21.15.169:0/533911779 >> v2:172.21.15.170:3300/0 conn(0x7f0d7c0d0630 0x7f0d7c15c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:01.123 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:01.120+0000 7f0d80e2e640 1 -- 172.21.15.169:0/533911779 >> 172.21.15.169:0/533911779 conn(0x7f0d7c0d13a0 msgr2=0x7f0d7c0d17b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:01.123 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:01.120+0000 7f0d80e2e640 1 -- 172.21.15.169:0/533911779 shutdown_connections 2023-10-26T01:44:01.124 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:01.120+0000 7f0d80e2e640 1 -- 172.21.15.169:0/533911779 wait complete. 2023-10-26T01:44:01.125 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:01.122+0000 7f0d80e2e640 1 Processor -- start 2023-10-26T01:44:01.125 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:01.122+0000 7f0d80e2e640 1 -- start start 2023-10-26T01:44:01.125 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:01.123+0000 7f0d80e2e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0d7c06ac80 0x7f0d7c157360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:01.125 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:01.123+0000 7f0d80e2e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0d7c0d0630 0x7f0d7c14e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:01.126 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:01.123+0000 7f0d80e2e640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0d7c159370 0x7f0d7c150e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:01.126 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:01.123+0000 7f0d80e2e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d7c1513d0 con 0x7f0d7c0d0630 2023-10-26T01:44:01.126 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:01.123+0000 7f0d80e2e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d7c1f27e0 con 0x7f0d7c06ac80 2023-10-26T01:44:01.126 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:01.123+0000 7f0d80e2e640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d7c1515c0 con 0x7f0d7c159370 2023-10-26T01:44:01.126 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:01.123+0000 7f0d7affd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0d7c0d0630 0x7f0d7c14e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:01.127 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:01.123+0000 7f0d7affd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0d7c0d0630 0x7f0d7c14e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:46342/0 (socket says 172.21.15.169:46342) 2023-10-26T01:44:01.127 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:01.123+0000 7f0d7affd640 1 -- 172.21.15.169:0/2278276788 learned_addr learned my addr 172.21.15.169:0/2278276788 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:44:01.128 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:01.124+0000 7f0d7b7fe640 1 --2- 172.21.15.169:0/2278276788 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0d7c06ac80 0x7f0d7c157360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:01.128 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:01.124+0000 7f0d7bfff640 1 --2- 172.21.15.169:0/2278276788 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0d7c159370 0x7f0d7c150e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:01.129 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:01.124+0000 7f0d7affd640 1 -- 172.21.15.169:0/2278276788 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0d7c159370 msgr2=0x7f0d7c150e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:01.129 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:01.124+0000 7f0d7affd640 1 --2- 172.21.15.169:0/2278276788 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0d7c159370 0x7f0d7c150e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:01.129 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:01.124+0000 7f0d7affd640 1 -- 172.21.15.169:0/2278276788 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0d7c06ac80 msgr2=0x7f0d7c157360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:01.129 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:01.124+0000 7f0d7affd640 1 --2- 172.21.15.169:0/2278276788 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0d7c06ac80 0x7f0d7c157360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:01.129 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:01.124+0000 7f0d7affd640 1 -- 172.21.15.169:0/2278276788 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d74009560 con 0x7f0d7c0d0630 2023-10-26T01:44:01.130 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:01.124+0000 7f0d7affd640 1 --2- 172.21.15.169:0/2278276788 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0d7c0d0630 0x7f0d7c14e220 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c00ea80 tx=0x7f0d6c00ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:01.130 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:01.124+0000 7f0d73fff640 1 -- 172.21.15.169:0/2278276788 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d6c00ce40 con 0x7f0d7c0d0630 2023-10-26T01:44:01.130 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:01.124+0000 7f0d73fff640 1 -- 172.21.15.169:0/2278276788 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0d6c004c10 con 0x7f0d7c0d0630 2023-10-26T01:44:01.130 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:01.124+0000 7f0d73fff640 1 -- 172.21.15.169:0/2278276788 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d6c005020 con 0x7f0d7c0d0630 2023-10-26T01:44:01.131 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:01.124+0000 7f0d80e2e640 1 -- 172.21.15.169:0/2278276788 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d7c1518a0 con 0x7f0d7c0d0630 2023-10-26T01:44:01.131 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:01.125+0000 7f0d80e2e640 1 -- 172.21.15.169:0/2278276788 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d7c154af0 con 0x7f0d7c0d0630 2023-10-26T01:44:01.131 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:01.126+0000 7f0d80e2e640 1 -- 172.21.15.169:0/2278276788 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d7c0cf970 con 0x7f0d7c0d0630 2023-10-26T01:44:01.131 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:01.127+0000 7f0d73fff640 1 -- 172.21.15.169:0/2278276788 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f0d6c005180 con 0x7f0d7c0d0630 2023-10-26T01:44:01.132 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:01.128+0000 7f0d73fff640 1 --2- 172.21.15.169:0/2278276788 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f0d5c06ae40 0x7f0d5c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:01.132 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:01.128+0000 7f0d73fff640 1 -- 172.21.15.169:0/2278276788 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0d6c08a120 con 0x7f0d7c0d0630 2023-10-26T01:44:01.132 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:01.128+0000 7f0d7b7fe640 1 --2- 172.21.15.169:0/2278276788 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f0d5c06ae40 0x7f0d5c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:01.133 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:01.129+0000 7f0d7b7fe640 1 --2- 172.21.15.169:0/2278276788 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f0d5c06ae40 0x7f0d5c06d300 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f0d64006fd0 tx=0x7f0d64008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:01.133 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:01.132+0000 7f0d73fff640 1 -- 172.21.15.169:0/2278276788 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0d6c055130 con 0x7f0d7c0d0630 2023-10-26T01:44:01.324 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:01.322+0000 7f0d80e2e640 1 -- 172.21.15.169:0/2278276788 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"} v 0) v1 -- 0x7f0d7c073c10 con 0x7f0d7c0d0630 2023-10-26T01:44:01.491 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:01.489+0000 7f0d73fff640 1 -- 172.21.15.169:0/2278276788 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"}]=0 set-quota max_bytes = 33554432 for pool cephfs_data v183) v1 ==== 181+0+0 (secure 0 0 0) 0x7f0d6c058750 con 0x7f0d7c0d0630 2023-10-26T01:44:01.555 INFO:teuthology.orchestra.run.smithi169.stderr:set-quota max_bytes = 33554432 for pool cephfs_data 2023-10-26T01:44:01.558 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:01.493+0000 7f0d80e2e640 1 -- 172.21.15.169:0/2278276788 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f0d5c06ae40 msgr2=0x7f0d5c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:01.559 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:01.493+0000 7f0d80e2e640 1 --2- 172.21.15.169:0/2278276788 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f0d5c06ae40 0x7f0d5c06d300 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f0d64006fd0 tx=0x7f0d64008040 comp rx=0 tx=0).stop 2023-10-26T01:44:01.559 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:01.493+0000 7f0d80e2e640 1 -- 172.21.15.169:0/2278276788 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0d7c0d0630 msgr2=0x7f0d7c14e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:01.559 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:01.493+0000 7f0d80e2e640 1 --2- 172.21.15.169:0/2278276788 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0d7c0d0630 0x7f0d7c14e220 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c00ea80 tx=0x7f0d6c00ef40 comp rx=0 tx=0).stop 2023-10-26T01:44:01.560 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:01.495+0000 7f0d80e2e640 1 -- 172.21.15.169:0/2278276788 shutdown_connections 2023-10-26T01:44:01.560 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:01.495+0000 7f0d80e2e640 1 --2- 172.21.15.169:0/2278276788 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f0d5c06ae40 0x7f0d5c06d300 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:01.560 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:01.495+0000 7f0d80e2e640 1 --2- 172.21.15.169:0/2278276788 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0d7c0d0630 0x7f0d7c14e220 unknown :-1 s=CLOSED pgs=1874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:01.561 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:01.495+0000 7f0d80e2e640 1 --2- 172.21.15.169:0/2278276788 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0d7c159370 0x7f0d7c150e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:01.561 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:01.495+0000 7f0d80e2e640 1 --2- 172.21.15.169:0/2278276788 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0d7c06ac80 0x7f0d7c157360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:01.561 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:01.495+0000 7f0d80e2e640 1 -- 172.21.15.169:0/2278276788 >> 172.21.15.169:0/2278276788 conn(0x7f0d7c0d13a0 msgr2=0x7f0d7c0d1780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:01.562 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:01.495+0000 7f0d80e2e640 1 -- 172.21.15.169:0/2278276788 shutdown_connections 2023-10-26T01:44:01.562 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:01.495+0000 7f0d80e2e640 1 -- 172.21.15.169:0/2278276788 wait complete. 2023-10-26T01:44:01.563 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2023-10-26T01:44:01.800 INFO:teuthology.orchestra.run.smithi169.stdout:{"osd_mon_report_interval":"5"} 2023-10-26T01:44:01.800 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2023-10-26T01:44:01.801 INFO:tasks.cephfs.test_full:Writing 32MB should fill this cluster 2023-10-26T01:44:01.801 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:44:01.801 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:44:01.850 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:44:01.851 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_a bs=1M conv=fdatasync count=16 seek=0) 2023-10-26T01:44:02.175 INFO:teuthology.orchestra.run.smithi169.stderr:16+0 records in 2023-10-26T01:44:02.175 INFO:teuthology.orchestra.run.smithi169.stderr:16+0 records out 2023-10-26T01:44:02.175 INFO:teuthology.orchestra.run.smithi169.stderr:16777216 bytes (17 MB, 16 MiB) copied, 0.217666 s, 77.1 MB/s 2023-10-26T01:44:02.176 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:44:02.177 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:44:02.231 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:44:02.231 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=17 seek=0) 2023-10-26T01:44:02.474 INFO:teuthology.orchestra.run.smithi169.stderr:17+0 records in 2023-10-26T01:44:02.475 INFO:teuthology.orchestra.run.smithi169.stderr:17+0 records out 2023-10-26T01:44:02.475 INFO:teuthology.orchestra.run.smithi169.stderr:17825792 bytes (18 MB, 17 MiB) copied, 0.189013 s, 94.3 MB/s 2023-10-26T01:44:02.476 INFO:tasks.cephfs.test_full:Writing file B succeeded (full status will happen soon) 2023-10-26T01:44:02.477 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-26T01:44:02.684 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:02.682+0000 7f3cdac1a640 1 -- 172.21.15.169:0/3346703648 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3cd41592d0 msgr2=0x7f3cd40aa3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:02.685 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:02.682+0000 7f3cdac1a640 1 --2- 172.21.15.169:0/3346703648 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3cd41592d0 0x7f3cd40aa3d0 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7f3cc8005fd0 tx=0x7f3cc802bd10 comp rx=0 tx=0).stop 2023-10-26T01:44:02.685 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:02.683+0000 7f3cdac1a640 1 -- 172.21.15.169:0/3346703648 shutdown_connections 2023-10-26T01:44:02.685 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:02.683+0000 7f3cdac1a640 1 --2- 172.21.15.169:0/3346703648 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3cd40cde00 0x7f3cd40cbac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:02.686 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:02.683+0000 7f3cdac1a640 1 --2- 172.21.15.169:0/3346703648 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3cd41592d0 0x7f3cd40aa3d0 unknown :-1 s=CLOSED pgs=1875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:02.686 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:02.683+0000 7f3cdac1a640 1 -- 172.21.15.169:0/3346703648 >> 172.21.15.169:0/3346703648 conn(0x7f3cd4069930 msgr2=0x7f3cd4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:02.686 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:02.683+0000 7f3cdac1a640 1 -- 172.21.15.169:0/3346703648 shutdown_connections 2023-10-26T01:44:02.686 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:02.683+0000 7f3cdac1a640 1 -- 172.21.15.169:0/3346703648 wait complete. 2023-10-26T01:44:02.687 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:02.684+0000 7f3cdac1a640 1 Processor -- start 2023-10-26T01:44:02.687 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:02.685+0000 7f3cdac1a640 1 -- start start 2023-10-26T01:44:02.688 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:02.685+0000 7f3cdac1a640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3cd40ac930 0x7f3cd41598c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:02.688 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:02.685+0000 7f3cdac1a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3cd40cde00 0x7f3cd40b0050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:02.688 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:02.686+0000 7f3cdac1a640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3cd41592d0 0x7f3cd40b0590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:02.688 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:02.686+0000 7f3cdac1a640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd415c280 con 0x7f3cd41592d0 2023-10-26T01:44:02.688 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:02.686+0000 7f3cdac1a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd40b2040 con 0x7f3cd40cde00 2023-10-26T01:44:02.689 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:02.686+0000 7f3cdac1a640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd415bc30 con 0x7f3cd40ac930 2023-10-26T01:44:02.689 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:02.686+0000 7f3cd91b7640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3cd41592d0 0x7f3cd40b0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:02.689 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:02.686+0000 7f3cd91b7640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3cd41592d0 0x7f3cd40b0590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:46354/0 (socket says 172.21.15.169:46354) 2023-10-26T01:44:02.689 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:02.686+0000 7f3cd91b7640 1 -- 172.21.15.169:0/3708967166 learned_addr learned my addr 172.21.15.169:0/3708967166 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:44:02.691 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:02.686+0000 7f3cd89b6640 1 --2- 172.21.15.169:0/3708967166 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3cd40ac930 0x7f3cd41598c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:02.691 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:02.686+0000 7f3cd91b7640 1 -- 172.21.15.169:0/3708967166 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3cd40ac930 msgr2=0x7f3cd41598c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:02.691 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:02.686+0000 7f3cd91b7640 1 --2- 172.21.15.169:0/3708967166 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3cd40ac930 0x7f3cd41598c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:02.692 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:02.686+0000 7f3cd3fff640 1 --2- 172.21.15.169:0/3708967166 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3cd40cde00 0x7f3cd40b0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:02.692 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:02.686+0000 7f3cd91b7640 1 -- 172.21.15.169:0/3708967166 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3cd40cde00 msgr2=0x7f3cd40b0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:02.692 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:02.686+0000 7f3cd91b7640 1 --2- 172.21.15.169:0/3708967166 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3cd40cde00 0x7f3cd40b0050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:02.692 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:02.686+0000 7f3cd91b7640 1 -- 172.21.15.169:0/3708967166 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cc8009560 con 0x7f3cd41592d0 2023-10-26T01:44:02.692 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:02.686+0000 7f3cd3fff640 1 --2- 172.21.15.169:0/3708967166 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3cd40cde00 0x7f3cd40b0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:44:02.693 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:02.687+0000 7f3cd91b7640 1 --2- 172.21.15.169:0/3708967166 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3cd41592d0 0x7f3cd40b0590 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc00b9a0 tx=0x7f3ccc00be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:02.693 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:02.687+0000 7f3cd17fa640 1 -- 172.21.15.169:0/3708967166 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ccc00cba0 con 0x7f3cd41592d0 2023-10-26T01:44:02.693 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:02.687+0000 7f3cd17fa640 1 -- 172.21.15.169:0/3708967166 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3ccc00ce10 con 0x7f3cd41592d0 2023-10-26T01:44:02.693 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:02.687+0000 7f3cd17fa640 1 -- 172.21.15.169:0/3708967166 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ccc0146a0 con 0x7f3cd41592d0 2023-10-26T01:44:02.693 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:02.687+0000 7f3cdac1a640 1 -- 172.21.15.169:0/3708967166 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cd4153c90 con 0x7f3cd41592d0 2023-10-26T01:44:02.694 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:02.687+0000 7f3cdac1a640 1 -- 172.21.15.169:0/3708967166 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cd4153980 con 0x7f3cd41592d0 2023-10-26T01:44:02.694 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:02.688+0000 7f3cdac1a640 1 -- 172.21.15.169:0/3708967166 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cd415c3c0 con 0x7f3cd41592d0 2023-10-26T01:44:02.694 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:02.689+0000 7f3cd17fa640 1 -- 172.21.15.169:0/3708967166 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f3ccc01d3b0 con 0x7f3cd41592d0 2023-10-26T01:44:02.695 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:02.690+0000 7f3cd17fa640 1 --2- 172.21.15.169:0/3708967166 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3cb806ae40 0x7f3cb806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:02.695 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:02.690+0000 7f3cd17fa640 1 -- 172.21.15.169:0/3708967166 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3ccc0896c0 con 0x7f3cd41592d0 2023-10-26T01:44:02.695 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:02.691+0000 7f3cd89b6640 1 --2- 172.21.15.169:0/3708967166 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3cb806ae40 0x7f3cb806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:02.695 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:02.692+0000 7f3cd89b6640 1 --2- 172.21.15.169:0/3708967166 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3cb806ae40 0x7f3cb806d300 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f3cc8035000 tx=0x7f3cc8004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:02.696 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:02.695+0000 7f3cd17fa640 1 -- 172.21.15.169:0/3708967166 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3cd415c3c0 con 0x7f3cd41592d0 2023-10-26T01:44:02.887 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:02.886+0000 7f3cdac1a640 1 -- 172.21.15.169:0/3708967166 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f3cd415c3c0 con 0x7f3cd41592d0 2023-10-26T01:44:02.890 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:02.889+0000 7f3cd17fa640 1 -- 172.21.15.169:0/3708967166 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v183) v1 ==== 81+0+34062 (secure 0 0 0) 0x7f3cd415c3c0 con 0x7f3cd41592d0 2023-10-26T01:44:02.891 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:44:02.891 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2023-10-26T01:44:02.891 INFO:teuthology.orchestra.run.smithi169.stdout: "epoch": 183, 2023-10-26T01:44:02.892 INFO:teuthology.orchestra.run.smithi169.stdout: "fsid": "30203b28-ed8a-44e8-a9a6-a9bf25f980f1", 2023-10-26T01:44:02.892 INFO:teuthology.orchestra.run.smithi169.stdout: "created": "2023-10-26T01:30:22.897155+0000", 2023-10-26T01:44:02.892 INFO:teuthology.orchestra.run.smithi169.stdout: "modified": "2023-10-26T01:44:01.477790+0000", 2023-10-26T01:44:02.892 INFO:teuthology.orchestra.run.smithi169.stdout: "last_up_change": "2023-10-26T01:30:31.875281+0000", 2023-10-26T01:44:02.892 INFO:teuthology.orchestra.run.smithi169.stdout: "last_in_change": "2023-10-26T01:30:27.637941+0000", 2023-10-26T01:44:02.892 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-26T01:44:02.893 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_num": 5799936, 2023-10-26T01:44:02.893 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_set": [ 2023-10-26T01:44:02.893 INFO:teuthology.orchestra.run.smithi169.stdout: "pglog_hardlimit", 2023-10-26T01:44:02.893 INFO:teuthology.orchestra.run.smithi169.stdout: "purged_snapdirs", 2023-10-26T01:44:02.893 INFO:teuthology.orchestra.run.smithi169.stdout: "recovery_deletes", 2023-10-26T01:44:02.894 INFO:teuthology.orchestra.run.smithi169.stdout: "sortbitwise" 2023-10-26T01:44:02.894 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:44:02.894 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_version": 4, 2023-10-26T01:44:02.894 INFO:teuthology.orchestra.run.smithi169.stdout: "full_ratio": 0.69999998807907104, 2023-10-26T01:44:02.894 INFO:teuthology.orchestra.run.smithi169.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-26T01:44:02.895 INFO:teuthology.orchestra.run.smithi169.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-26T01:44:02.895 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_snapshot": "", 2023-10-26T01:44:02.895 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_max": 23, 2023-10-26T01:44:02.895 INFO:teuthology.orchestra.run.smithi169.stdout: "max_osd": 8, 2023-10-26T01:44:02.895 INFO:teuthology.orchestra.run.smithi169.stdout: "require_min_compat_client": "luminous", 2023-10-26T01:44:02.896 INFO:teuthology.orchestra.run.smithi169.stdout: "min_compat_client": "jewel", 2023-10-26T01:44:02.896 INFO:teuthology.orchestra.run.smithi169.stdout: "require_osd_release": "reef", 2023-10-26T01:44:02.896 INFO:teuthology.orchestra.run.smithi169.stdout: "allow_crimson": false, 2023-10-26T01:44:02.896 INFO:teuthology.orchestra.run.smithi169.stdout: "pools": [ 2023-10-26T01:44:02.896 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:02.897 INFO:teuthology.orchestra.run.smithi169.stdout: "pool": 1, 2023-10-26T01:44:02.897 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_name": ".mgr", 2023-10-26T01:44:02.897 INFO:teuthology.orchestra.run.smithi169.stdout: "create_time": "2023-10-26T01:30:32.200808+0000", 2023-10-26T01:44:02.897 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": 1, 2023-10-26T01:44:02.898 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_names": "hashpspool", 2023-10-26T01:44:02.898 INFO:teuthology.orchestra.run.smithi169.stdout: "type": 1, 2023-10-26T01:44:02.898 INFO:teuthology.orchestra.run.smithi169.stdout: "size": 2, 2023-10-26T01:44:02.898 INFO:teuthology.orchestra.run.smithi169.stdout: "min_size": 1, 2023-10-26T01:44:02.899 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_rule": 0, 2023-10-26T01:44:02.899 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_count": 0, 2023-10-26T01:44:02.899 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_target": 0, 2023-10-26T01:44:02.899 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T01:44:02.899 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T01:44:02.899 INFO:teuthology.orchestra.run.smithi169.stdout: "object_hash": 2, 2023-10-26T01:44:02.900 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_mode": "off", 2023-10-26T01:44:02.900 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num": 1, 2023-10-26T01:44:02.900 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num": 1, 2023-10-26T01:44:02.900 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num_target": 1, 2023-10-26T01:44:02.900 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_target": 1, 2023-10-26T01:44:02.900 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_pending": 1, 2023-10-26T01:44:02.901 INFO:teuthology.orchestra.run.smithi169.stdout: "last_pg_merge_meta": { 2023-10-26T01:44:02.901 INFO:teuthology.orchestra.run.smithi169.stdout: "source_pgid": "0.0", 2023-10-26T01:44:02.901 INFO:teuthology.orchestra.run.smithi169.stdout: "ready_epoch": 0, 2023-10-26T01:44:02.901 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_started": 0, 2023-10-26T01:44:02.901 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_clean": 0, 2023-10-26T01:44:02.902 INFO:teuthology.orchestra.run.smithi169.stdout: "source_version": "0'0", 2023-10-26T01:44:02.902 INFO:teuthology.orchestra.run.smithi169.stdout: "target_version": "0'0" 2023-10-26T01:44:02.902 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:02.902 INFO:teuthology.orchestra.run.smithi169.stdout: "last_change": "109", 2023-10-26T01:44:02.902 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend": "0", 2023-10-26T01:44:02.902 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T01:44:02.903 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T01:44:02.903 INFO:teuthology.orchestra.run.smithi169.stdout: "auid": 0, 2023-10-26T01:44:02.903 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_mode": "selfmanaged", 2023-10-26T01:44:02.903 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_seq": 0, 2023-10-26T01:44:02.903 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_epoch": 0, 2023-10-26T01:44:02.904 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_snaps": [], 2023-10-26T01:44:02.904 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps": "[]", 2023-10-26T01:44:02.904 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_bytes": 0, 2023-10-26T01:44:02.904 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_objects": 0, 2023-10-26T01:44:02.904 INFO:teuthology.orchestra.run.smithi169.stdout: "tiers": [], 2023-10-26T01:44:02.904 INFO:teuthology.orchestra.run.smithi169.stdout: "tier_of": -1, 2023-10-26T01:44:02.905 INFO:teuthology.orchestra.run.smithi169.stdout: "read_tier": -1, 2023-10-26T01:44:02.905 INFO:teuthology.orchestra.run.smithi169.stdout: "write_tier": -1, 2023-10-26T01:44:02.905 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_mode": "none", 2023-10-26T01:44:02.905 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_bytes": 0, 2023-10-26T01:44:02.905 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_objects": 0, 2023-10-26T01:44:02.906 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T01:44:02.906 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T01:44:02.906 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T01:44:02.906 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_flush_age": 0, 2023-10-26T01:44:02.906 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_evict_age": 0, 2023-10-26T01:44:02.907 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profile": "", 2023-10-26T01:44:02.907 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_params": { 2023-10-26T01:44:02.907 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "none" 2023-10-26T01:44:02.907 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:02.907 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_period": 0, 2023-10-26T01:44:02.907 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_count": 0, 2023-10-26T01:44:02.908 INFO:teuthology.orchestra.run.smithi169.stdout: "use_gmt_hitset": true, 2023-10-26T01:44:02.908 INFO:teuthology.orchestra.run.smithi169.stdout: "min_read_recency_for_promote": 0, 2023-10-26T01:44:02.908 INFO:teuthology.orchestra.run.smithi169.stdout: "min_write_recency_for_promote": 0, 2023-10-26T01:44:02.908 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T01:44:02.908 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_search_last_n": 0, 2023-10-26T01:44:02.909 INFO:teuthology.orchestra.run.smithi169.stdout: "grade_table": [], 2023-10-26T01:44:02.909 INFO:teuthology.orchestra.run.smithi169.stdout: "stripe_width": 0, 2023-10-26T01:44:02.909 INFO:teuthology.orchestra.run.smithi169.stdout: "expected_num_objects": 0, 2023-10-26T01:44:02.909 INFO:teuthology.orchestra.run.smithi169.stdout: "fast_read": false, 2023-10-26T01:44:02.909 INFO:teuthology.orchestra.run.smithi169.stdout: "options": { 2023-10-26T01:44:02.910 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_max": 32, 2023-10-26T01:44:02.910 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_min": 1 2023-10-26T01:44:02.910 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:02.910 INFO:teuthology.orchestra.run.smithi169.stdout: "application_metadata": { 2023-10-26T01:44:02.910 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": {} 2023-10-26T01:44:02.911 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:02.911 INFO:teuthology.orchestra.run.smithi169.stdout: "read_balance": { 2023-10-26T01:44:02.911 INFO:teuthology.orchestra.run.smithi169.stdout: "score_acting": 8, 2023-10-26T01:44:02.911 INFO:teuthology.orchestra.run.smithi169.stdout: "score_stable": 8, 2023-10-26T01:44:02.911 INFO:teuthology.orchestra.run.smithi169.stdout: "optimal_score": 0.25, 2023-10-26T01:44:02.911 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_acting": 2, 2023-10-26T01:44:02.912 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_stable": 2, 2023-10-26T01:44:02.912 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity_weighted": 1, 2023-10-26T01:44:02.912 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity": 1, 2023-10-26T01:44:02.912 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity_weighted": 1 2023-10-26T01:44:02.912 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:02.913 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:02.913 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:02.913 INFO:teuthology.orchestra.run.smithi169.stdout: "pool": 22, 2023-10-26T01:44:02.913 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_name": "cephfs_metadata", 2023-10-26T01:44:02.913 INFO:teuthology.orchestra.run.smithi169.stdout: "create_time": "2023-10-26T01:43:47.501296+0000", 2023-10-26T01:44:02.913 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": 1, 2023-10-26T01:44:02.914 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_names": "hashpspool", 2023-10-26T01:44:02.914 INFO:teuthology.orchestra.run.smithi169.stdout: "type": 1, 2023-10-26T01:44:02.914 INFO:teuthology.orchestra.run.smithi169.stdout: "size": 2, 2023-10-26T01:44:02.914 INFO:teuthology.orchestra.run.smithi169.stdout: "min_size": 1, 2023-10-26T01:44:02.914 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_rule": 0, 2023-10-26T01:44:02.915 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_count": 0, 2023-10-26T01:44:02.915 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_target": 0, 2023-10-26T01:44:02.915 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T01:44:02.915 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T01:44:02.915 INFO:teuthology.orchestra.run.smithi169.stdout: "object_hash": 2, 2023-10-26T01:44:02.916 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_mode": "off", 2023-10-26T01:44:02.916 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num": 32, 2023-10-26T01:44:02.916 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num": 32, 2023-10-26T01:44:02.916 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num_target": 32, 2023-10-26T01:44:02.916 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_target": 32, 2023-10-26T01:44:02.916 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_pending": 32, 2023-10-26T01:44:02.917 INFO:teuthology.orchestra.run.smithi169.stdout: "last_pg_merge_meta": { 2023-10-26T01:44:02.917 INFO:teuthology.orchestra.run.smithi169.stdout: "source_pgid": "0.0", 2023-10-26T01:44:02.917 INFO:teuthology.orchestra.run.smithi169.stdout: "ready_epoch": 0, 2023-10-26T01:44:02.917 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_started": 0, 2023-10-26T01:44:02.917 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_clean": 0, 2023-10-26T01:44:02.918 INFO:teuthology.orchestra.run.smithi169.stdout: "source_version": "0'0", 2023-10-26T01:44:02.918 INFO:teuthology.orchestra.run.smithi169.stdout: "target_version": "0'0" 2023-10-26T01:44:02.918 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:02.918 INFO:teuthology.orchestra.run.smithi169.stdout: "last_change": "181", 2023-10-26T01:44:02.918 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend": "0", 2023-10-26T01:44:02.919 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T01:44:02.919 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T01:44:02.919 INFO:teuthology.orchestra.run.smithi169.stdout: "auid": 0, 2023-10-26T01:44:02.919 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_mode": "selfmanaged", 2023-10-26T01:44:02.919 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_seq": 0, 2023-10-26T01:44:02.920 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_epoch": 0, 2023-10-26T01:44:02.920 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_snaps": [], 2023-10-26T01:44:02.920 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps": "[]", 2023-10-26T01:44:02.920 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_bytes": 0, 2023-10-26T01:44:02.920 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_objects": 0, 2023-10-26T01:44:02.920 INFO:teuthology.orchestra.run.smithi169.stdout: "tiers": [], 2023-10-26T01:44:02.921 INFO:teuthology.orchestra.run.smithi169.stdout: "tier_of": -1, 2023-10-26T01:44:02.921 INFO:teuthology.orchestra.run.smithi169.stdout: "read_tier": -1, 2023-10-26T01:44:02.921 INFO:teuthology.orchestra.run.smithi169.stdout: "write_tier": -1, 2023-10-26T01:44:02.921 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_mode": "none", 2023-10-26T01:44:02.921 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_bytes": 0, 2023-10-26T01:44:02.922 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_objects": 0, 2023-10-26T01:44:02.922 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T01:44:02.922 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T01:44:02.922 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T01:44:02.922 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_flush_age": 0, 2023-10-26T01:44:02.923 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_evict_age": 0, 2023-10-26T01:44:02.923 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profile": "", 2023-10-26T01:44:02.923 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_params": { 2023-10-26T01:44:02.923 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "none" 2023-10-26T01:44:02.923 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:02.923 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_period": 0, 2023-10-26T01:44:02.924 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_count": 0, 2023-10-26T01:44:02.924 INFO:teuthology.orchestra.run.smithi169.stdout: "use_gmt_hitset": true, 2023-10-26T01:44:02.924 INFO:teuthology.orchestra.run.smithi169.stdout: "min_read_recency_for_promote": 0, 2023-10-26T01:44:02.924 INFO:teuthology.orchestra.run.smithi169.stdout: "min_write_recency_for_promote": 0, 2023-10-26T01:44:02.925 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T01:44:02.925 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_search_last_n": 0, 2023-10-26T01:44:02.925 INFO:teuthology.orchestra.run.smithi169.stdout: "grade_table": [], 2023-10-26T01:44:02.925 INFO:teuthology.orchestra.run.smithi169.stdout: "stripe_width": 0, 2023-10-26T01:44:02.925 INFO:teuthology.orchestra.run.smithi169.stdout: "expected_num_objects": 0, 2023-10-26T01:44:02.925 INFO:teuthology.orchestra.run.smithi169.stdout: "fast_read": false, 2023-10-26T01:44:02.926 INFO:teuthology.orchestra.run.smithi169.stdout: "options": { 2023-10-26T01:44:02.926 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_bias": 4, 2023-10-26T01:44:02.926 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_min": 16, 2023-10-26T01:44:02.926 INFO:teuthology.orchestra.run.smithi169.stdout: "recovery_priority": 5 2023-10-26T01:44:02.926 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:02.927 INFO:teuthology.orchestra.run.smithi169.stdout: "application_metadata": { 2023-10-26T01:44:02.927 INFO:teuthology.orchestra.run.smithi169.stdout: "cephfs": { 2023-10-26T01:44:02.927 INFO:teuthology.orchestra.run.smithi169.stdout: "metadata": "cephfs" 2023-10-26T01:44:02.927 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:02.927 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:02.928 INFO:teuthology.orchestra.run.smithi169.stdout: "read_balance": { 2023-10-26T01:44:02.928 INFO:teuthology.orchestra.run.smithi169.stdout: "score_acting": 2, 2023-10-26T01:44:02.928 INFO:teuthology.orchestra.run.smithi169.stdout: "score_stable": 2, 2023-10-26T01:44:02.928 INFO:teuthology.orchestra.run.smithi169.stdout: "optimal_score": 1, 2023-10-26T01:44:02.928 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_acting": 2, 2023-10-26T01:44:02.928 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_stable": 2, 2023-10-26T01:44:02.929 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity_weighted": 1, 2023-10-26T01:44:02.929 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity": 1, 2023-10-26T01:44:02.929 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity_weighted": 1 2023-10-26T01:44:02.929 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:02.929 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:02.930 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:02.930 INFO:teuthology.orchestra.run.smithi169.stdout: "pool": 23, 2023-10-26T01:44:02.930 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_name": "cephfs_data", 2023-10-26T01:44:02.930 INFO:teuthology.orchestra.run.smithi169.stdout: "create_time": "2023-10-26T01:43:48.737507+0000", 2023-10-26T01:44:02.930 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": 1, 2023-10-26T01:44:02.931 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_names": "hashpspool", 2023-10-26T01:44:02.931 INFO:teuthology.orchestra.run.smithi169.stdout: "type": 1, 2023-10-26T01:44:02.931 INFO:teuthology.orchestra.run.smithi169.stdout: "size": 2, 2023-10-26T01:44:02.931 INFO:teuthology.orchestra.run.smithi169.stdout: "min_size": 1, 2023-10-26T01:44:02.931 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_rule": 0, 2023-10-26T01:44:02.931 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_count": 0, 2023-10-26T01:44:02.932 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_target": 0, 2023-10-26T01:44:02.932 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T01:44:02.932 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T01:44:02.932 INFO:teuthology.orchestra.run.smithi169.stdout: "object_hash": 2, 2023-10-26T01:44:02.932 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_mode": "off", 2023-10-26T01:44:02.933 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num": 64, 2023-10-26T01:44:02.933 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num": 64, 2023-10-26T01:44:02.933 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num_target": 64, 2023-10-26T01:44:02.933 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_target": 64, 2023-10-26T01:44:02.933 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_pending": 64, 2023-10-26T01:44:02.933 INFO:teuthology.orchestra.run.smithi169.stdout: "last_pg_merge_meta": { 2023-10-26T01:44:02.934 INFO:teuthology.orchestra.run.smithi169.stdout: "source_pgid": "0.0", 2023-10-26T01:44:02.934 INFO:teuthology.orchestra.run.smithi169.stdout: "ready_epoch": 0, 2023-10-26T01:44:02.934 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_started": 0, 2023-10-26T01:44:02.934 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_clean": 0, 2023-10-26T01:44:02.934 INFO:teuthology.orchestra.run.smithi169.stdout: "source_version": "0'0", 2023-10-26T01:44:02.935 INFO:teuthology.orchestra.run.smithi169.stdout: "target_version": "0'0" 2023-10-26T01:44:02.935 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:02.935 INFO:teuthology.orchestra.run.smithi169.stdout: "last_change": "183", 2023-10-26T01:44:02.935 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend": "0", 2023-10-26T01:44:02.935 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T01:44:02.936 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T01:44:02.936 INFO:teuthology.orchestra.run.smithi169.stdout: "auid": 0, 2023-10-26T01:44:02.936 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_mode": "selfmanaged", 2023-10-26T01:44:02.936 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_seq": 0, 2023-10-26T01:44:02.936 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_epoch": 0, 2023-10-26T01:44:02.936 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_snaps": [], 2023-10-26T01:44:02.937 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps": "[]", 2023-10-26T01:44:02.937 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_bytes": 33554432, 2023-10-26T01:44:02.937 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_objects": 0, 2023-10-26T01:44:02.937 INFO:teuthology.orchestra.run.smithi169.stdout: "tiers": [], 2023-10-26T01:44:02.937 INFO:teuthology.orchestra.run.smithi169.stdout: "tier_of": -1, 2023-10-26T01:44:02.938 INFO:teuthology.orchestra.run.smithi169.stdout: "read_tier": -1, 2023-10-26T01:44:02.938 INFO:teuthology.orchestra.run.smithi169.stdout: "write_tier": -1, 2023-10-26T01:44:02.938 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_mode": "none", 2023-10-26T01:44:02.938 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_bytes": 0, 2023-10-26T01:44:02.938 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_objects": 0, 2023-10-26T01:44:02.939 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T01:44:02.939 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T01:44:02.939 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T01:44:02.939 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_flush_age": 0, 2023-10-26T01:44:02.939 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_evict_age": 0, 2023-10-26T01:44:02.939 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profile": "", 2023-10-26T01:44:02.940 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_params": { 2023-10-26T01:44:02.940 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "none" 2023-10-26T01:44:02.940 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:02.940 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_period": 0, 2023-10-26T01:44:02.940 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_count": 0, 2023-10-26T01:44:02.941 INFO:teuthology.orchestra.run.smithi169.stdout: "use_gmt_hitset": true, 2023-10-26T01:44:02.941 INFO:teuthology.orchestra.run.smithi169.stdout: "min_read_recency_for_promote": 0, 2023-10-26T01:44:02.941 INFO:teuthology.orchestra.run.smithi169.stdout: "min_write_recency_for_promote": 0, 2023-10-26T01:44:02.941 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T01:44:02.941 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_search_last_n": 0, 2023-10-26T01:44:02.942 INFO:teuthology.orchestra.run.smithi169.stdout: "grade_table": [], 2023-10-26T01:44:02.942 INFO:teuthology.orchestra.run.smithi169.stdout: "stripe_width": 0, 2023-10-26T01:44:02.942 INFO:teuthology.orchestra.run.smithi169.stdout: "expected_num_objects": 0, 2023-10-26T01:44:02.942 INFO:teuthology.orchestra.run.smithi169.stdout: "fast_read": false, 2023-10-26T01:44:02.942 INFO:teuthology.orchestra.run.smithi169.stdout: "options": { 2023-10-26T01:44:02.942 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_min": 64, 2023-10-26T01:44:02.943 INFO:teuthology.orchestra.run.smithi169.stdout: "target_size_ratio": 0.89999997615814209 2023-10-26T01:44:02.943 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:02.943 INFO:teuthology.orchestra.run.smithi169.stdout: "application_metadata": { 2023-10-26T01:44:02.943 INFO:teuthology.orchestra.run.smithi169.stdout: "cephfs": { 2023-10-26T01:44:02.943 INFO:teuthology.orchestra.run.smithi169.stdout: "data": "cephfs" 2023-10-26T01:44:02.944 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:02.944 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:02.944 INFO:teuthology.orchestra.run.smithi169.stdout: "read_balance": { 2023-10-26T01:44:02.944 INFO:teuthology.orchestra.run.smithi169.stdout: "score_acting": 1.3799999952316284, 2023-10-26T01:44:02.944 INFO:teuthology.orchestra.run.smithi169.stdout: "score_stable": 1.3799999952316284, 2023-10-26T01:44:02.944 INFO:teuthology.orchestra.run.smithi169.stdout: "optimal_score": 1, 2023-10-26T01:44:02.945 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_acting": 1.3799999952316284, 2023-10-26T01:44:02.945 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_stable": 1.3799999952316284, 2023-10-26T01:44:02.945 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity_weighted": 1, 2023-10-26T01:44:02.945 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity": 1, 2023-10-26T01:44:02.945 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity_weighted": 1 2023-10-26T01:44:02.946 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:02.946 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:02.946 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:44:02.946 INFO:teuthology.orchestra.run.smithi169.stdout: "osds": [ 2023-10-26T01:44:02.946 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:02.946 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 0, 2023-10-26T01:44:02.947 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:44:02.947 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:44:02.947 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:44:02.947 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:44:02.947 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:44:02.948 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:44:02.948 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:44:02.948 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:44:02.948 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 180, 2023-10-26T01:44:02.948 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:44:02.949 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:44:02.949 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:44:02.949 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:02.949 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:02.949 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:02.950 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6816", 2023-10-26T01:44:02.950 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:44:02.950 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:02.950 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:02.950 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:02.950 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6817", 2023-10-26T01:44:02.951 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:44:02.951 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:02.951 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:02.951 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:02.951 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:44:02.952 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:02.952 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:02.952 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:02.952 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6818", 2023-10-26T01:44:02.952 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:44:02.953 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:02.953 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:02.953 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:02.953 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6819", 2023-10-26T01:44:02.953 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:44:02.953 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:02.954 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:02.954 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:02.954 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:44:02.954 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:02.954 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:02.955 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:02.955 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6822", 2023-10-26T01:44:02.955 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:44:02.955 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:02.955 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:02.956 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:02.956 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6823", 2023-10-26T01:44:02.956 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:44:02.956 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:02.956 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:02.956 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:02.957 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:44:02.957 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:02.957 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:02.957 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:02.957 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6820", 2023-10-26T01:44:02.958 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:44:02.958 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:02.958 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:02.958 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:02.958 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6821", 2023-10-26T01:44:02.959 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:44:02.959 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:02.959 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:02.959 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:02.959 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6817/4204255864", 2023-10-26T01:44:02.960 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6819/4204255864", 2023-10-26T01:44:02.960 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6823/4204255864", 2023-10-26T01:44:02.960 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6821/4204255864", 2023-10-26T01:44:02.960 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:44:02.960 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:44:02.960 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:44:02.961 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:02.961 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:02.961 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:02.961 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 1, 2023-10-26T01:44:02.961 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:44:02.961 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:44:02.962 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:44:02.962 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:44:02.962 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:44:02.962 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:44:02.962 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:44:02.963 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:44:02.963 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 180, 2023-10-26T01:44:02.963 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:44:02.963 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:44:02.963 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:44:02.963 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:02.964 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:02.964 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:02.964 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6808", 2023-10-26T01:44:02.964 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:44:02.964 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:02.965 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:02.965 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:02.965 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6809", 2023-10-26T01:44:02.965 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:44:02.965 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:02.965 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:02.966 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:02.966 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:44:02.966 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:02.966 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:02.966 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:02.967 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6810", 2023-10-26T01:44:02.967 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:44:02.967 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:02.967 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:02.967 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:02.968 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6811", 2023-10-26T01:44:02.968 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:44:02.968 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:02.968 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:02.968 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:02.968 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:44:02.969 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:02.969 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:02.969 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:02.969 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6814", 2023-10-26T01:44:02.969 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:44:02.970 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:02.970 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:02.970 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:02.970 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6815", 2023-10-26T01:44:02.970 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:44:02.970 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:02.971 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:02.971 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:02.971 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:44:02.971 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:02.971 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:02.972 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:02.972 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6812", 2023-10-26T01:44:02.972 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:44:02.972 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:02.972 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:02.972 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:02.973 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6813", 2023-10-26T01:44:02.973 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:44:02.973 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:02.973 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:02.973 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:02.974 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6809/238206046", 2023-10-26T01:44:02.974 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6811/238206046", 2023-10-26T01:44:02.974 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6815/238206046", 2023-10-26T01:44:02.974 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6813/238206046", 2023-10-26T01:44:02.974 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:44:02.974 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:44:02.975 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:44:02.975 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:02.975 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:02.975 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:02.975 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 2, 2023-10-26T01:44:02.976 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:44:02.976 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:44:02.976 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:44:02.976 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:44:02.976 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:44:02.977 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:44:02.977 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:44:02.977 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:44:02.977 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 180, 2023-10-26T01:44:02.977 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:44:02.977 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:44:02.978 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:44:02.978 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:02.978 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:02.978 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:02.978 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6800", 2023-10-26T01:44:02.979 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:44:02.979 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:02.979 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:02.979 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:02.979 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6801", 2023-10-26T01:44:02.980 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:44:02.980 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:02.980 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:02.980 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:02.980 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:44:02.980 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:02.981 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:02.981 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:02.981 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6802", 2023-10-26T01:44:02.982 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:44:02.982 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:02.982 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:02.983 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:02.983 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6803", 2023-10-26T01:44:02.983 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:44:02.983 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:02.983 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:02.983 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:02.984 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:44:02.984 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:02.984 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:02.984 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:02.984 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6806", 2023-10-26T01:44:02.985 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:44:02.985 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:02.985 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:02.985 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:02.985 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6807", 2023-10-26T01:44:02.985 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:44:02.986 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:02.986 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:02.986 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:02.986 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:44:02.986 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:02.987 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:02.987 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:02.987 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6804", 2023-10-26T01:44:02.987 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:44:02.987 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:02.988 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:02.988 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:02.988 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6805", 2023-10-26T01:44:02.988 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:44:02.988 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:02.988 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:02.989 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:02.989 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6801/3215791846", 2023-10-26T01:44:02.989 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6803/3215791846", 2023-10-26T01:44:02.989 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6807/3215791846", 2023-10-26T01:44:02.989 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6805/3215791846", 2023-10-26T01:44:02.990 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:44:02.990 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:44:02.990 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:44:02.990 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:02.990 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:02.991 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:02.991 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 3, 2023-10-26T01:44:02.991 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:44:02.991 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:44:02.991 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:44:02.991 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:44:02.992 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:44:02.992 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:44:02.992 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:44:02.992 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 14, 2023-10-26T01:44:02.992 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 180, 2023-10-26T01:44:02.993 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:44:02.993 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:44:02.993 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:44:02.993 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:02.993 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:02.993 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:02.994 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6824", 2023-10-26T01:44:02.994 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:44:02.994 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:02.994 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:02.994 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:02.994 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6825", 2023-10-26T01:44:02.995 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:44:02.995 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:02.995 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:02.995 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:02.995 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:44:02.996 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:02.996 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:02.996 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:02.996 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6826", 2023-10-26T01:44:02.996 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:44:02.996 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:02.997 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:02.997 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:02.997 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6827", 2023-10-26T01:44:02.997 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:44:02.997 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:02.998 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:02.998 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:02.998 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:44:02.998 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:02.998 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:02.999 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:02.999 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6830", 2023-10-26T01:44:02.999 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:44:02.999 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:02.999 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:03.000 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:03.000 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6831", 2023-10-26T01:44:03.000 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:44:03.000 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:03.000 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:03.000 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:03.001 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:44:03.001 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:03.001 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:03.001 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:03.001 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6828", 2023-10-26T01:44:03.001 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:44:03.002 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:03.002 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:03.002 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:03.002 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6829", 2023-10-26T01:44:03.002 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:44:03.003 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:03.003 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:03.003 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:03.003 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6825/2256701226", 2023-10-26T01:44:03.003 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6827/2256701226", 2023-10-26T01:44:03.003 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6831/2256701226", 2023-10-26T01:44:03.004 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6829/2256701226", 2023-10-26T01:44:03.004 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:44:03.004 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:44:03.004 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:44:03.004 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:03.005 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:03.005 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:03.005 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 4, 2023-10-26T01:44:03.005 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:44:03.005 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:44:03.006 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:44:03.006 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:44:03.006 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:44:03.006 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:44:03.006 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:44:03.006 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:44:03.007 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 180, 2023-10-26T01:44:03.007 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:44:03.007 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:44:03.007 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:44:03.007 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:03.008 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:03.008 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:03.008 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6816", 2023-10-26T01:44:03.008 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:44:03.008 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:03.009 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:03.009 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:03.009 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6817", 2023-10-26T01:44:03.009 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:44:03.009 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:03.009 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:03.010 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:03.010 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:44:03.010 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:03.010 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:03.010 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:03.011 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6818", 2023-10-26T01:44:03.011 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:44:03.011 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:03.011 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:03.011 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:03.011 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6819", 2023-10-26T01:44:03.012 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:44:03.012 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:03.012 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:03.012 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:03.012 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:44:03.013 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:03.013 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:03.013 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:03.013 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6825", 2023-10-26T01:44:03.013 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:44:03.013 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:03.014 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:03.014 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:03.014 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6828", 2023-10-26T01:44:03.014 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:44:03.014 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:03.015 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:03.015 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:03.015 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:44:03.015 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:03.015 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:03.015 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:03.016 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6820", 2023-10-26T01:44:03.016 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:44:03.016 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:03.016 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:03.016 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:03.017 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6823", 2023-10-26T01:44:03.017 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:44:03.017 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:03.017 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:03.017 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:03.017 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6817/1082279874", 2023-10-26T01:44:03.018 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6819/1082279874", 2023-10-26T01:44:03.018 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6828/1082279874", 2023-10-26T01:44:03.018 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6823/1082279874", 2023-10-26T01:44:03.018 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:44:03.018 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:44:03.019 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:44:03.019 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:03.019 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:03.019 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:03.019 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 5, 2023-10-26T01:44:03.020 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:44:03.020 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:44:03.020 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:44:03.020 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:44:03.020 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:44:03.020 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:44:03.021 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:44:03.021 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:44:03.021 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 180, 2023-10-26T01:44:03.021 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:44:03.021 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:44:03.021 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:44:03.022 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:03.022 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:03.022 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:03.022 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6808", 2023-10-26T01:44:03.022 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:44:03.022 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:03.023 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:03.023 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:03.023 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6809", 2023-10-26T01:44:03.023 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:44:03.023 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:03.024 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:03.024 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:03.024 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:44:03.024 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:03.024 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:03.024 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:03.025 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6810", 2023-10-26T01:44:03.025 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:44:03.025 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:03.025 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:03.025 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:03.025 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6811", 2023-10-26T01:44:03.026 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:44:03.026 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:03.026 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:03.026 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:03.026 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:44:03.027 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:03.027 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:03.027 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:03.027 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6814", 2023-10-26T01:44:03.027 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:44:03.027 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:03.028 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:03.028 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:03.028 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6815", 2023-10-26T01:44:03.028 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:44:03.028 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:03.029 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:03.029 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:03.029 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:44:03.029 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:03.029 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:03.030 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:03.030 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6812", 2023-10-26T01:44:03.030 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:44:03.030 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:03.030 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:03.031 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:03.031 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6813", 2023-10-26T01:44:03.031 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:44:03.031 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:03.031 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:03.031 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:03.032 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6809/3335654351", 2023-10-26T01:44:03.032 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6811/3335654351", 2023-10-26T01:44:03.032 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6815/3335654351", 2023-10-26T01:44:03.032 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6813/3335654351", 2023-10-26T01:44:03.032 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:44:03.033 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:44:03.033 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:44:03.033 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:03.033 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:03.033 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:03.033 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 6, 2023-10-26T01:44:03.034 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:44:03.034 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:44:03.034 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:44:03.034 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:44:03.034 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:44:03.035 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:44:03.035 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:44:03.035 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:44:03.035 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 180, 2023-10-26T01:44:03.035 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:44:03.036 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:44:03.036 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:44:03.036 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:03.036 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:03.036 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:03.036 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6800", 2023-10-26T01:44:03.037 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:44:03.037 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:03.037 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:03.037 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:03.037 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6801", 2023-10-26T01:44:03.038 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:44:03.038 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:03.038 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:03.038 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:03.038 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:44:03.039 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:03.039 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:03.039 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:03.039 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6802", 2023-10-26T01:44:03.039 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:44:03.040 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:03.040 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:03.040 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:03.040 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6803", 2023-10-26T01:44:03.040 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:44:03.040 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:03.041 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:03.041 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:03.041 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:44:03.041 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:03.041 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:03.041 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:03.042 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6806", 2023-10-26T01:44:03.042 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:44:03.042 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:03.042 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:03.042 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:03.043 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6807", 2023-10-26T01:44:03.043 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:44:03.043 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:03.043 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:03.043 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:03.043 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:44:03.044 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:03.044 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:03.044 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:03.044 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6804", 2023-10-26T01:44:03.044 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:44:03.045 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:03.045 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:03.045 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:03.045 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6805", 2023-10-26T01:44:03.045 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:44:03.045 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:03.046 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:03.046 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:03.046 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6801/2520460697", 2023-10-26T01:44:03.046 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6803/2520460697", 2023-10-26T01:44:03.046 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6807/2520460697", 2023-10-26T01:44:03.047 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6805/2520460697", 2023-10-26T01:44:03.047 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:44:03.047 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:44:03.047 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:44:03.047 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:03.048 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:03.048 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:03.048 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 7, 2023-10-26T01:44:03.048 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:44:03.048 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:44:03.049 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:44:03.049 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:44:03.049 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:44:03.049 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:44:03.049 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:44:03.049 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:44:03.050 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 180, 2023-10-26T01:44:03.050 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:44:03.050 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:44:03.050 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:44:03.050 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:03.050 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:03.051 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:03.051 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6821", 2023-10-26T01:44:03.051 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:44:03.051 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:03.051 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:03.052 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:03.052 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6822", 2023-10-26T01:44:03.052 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:44:03.052 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:03.052 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:03.052 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:03.053 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:44:03.053 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:03.053 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:03.053 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:03.053 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6824", 2023-10-26T01:44:03.053 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:44:03.054 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:03.054 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:03.054 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:03.054 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6826", 2023-10-26T01:44:03.054 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:44:03.055 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:03.055 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:03.055 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:03.055 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:44:03.055 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:03.055 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:03.056 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:03.056 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6830", 2023-10-26T01:44:03.056 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:44:03.056 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:03.056 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:03.057 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:03.057 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6831", 2023-10-26T01:44:03.057 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:44:03.057 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:03.057 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:03.057 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:03.058 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:44:03.058 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:03.058 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:03.058 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:03.058 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6827", 2023-10-26T01:44:03.059 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:44:03.059 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:03.059 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:03.059 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:03.059 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6829", 2023-10-26T01:44:03.060 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:44:03.060 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:03.060 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:03.060 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:03.060 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6822/1115930619", 2023-10-26T01:44:03.060 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6826/1115930619", 2023-10-26T01:44:03.061 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6831/1115930619", 2023-10-26T01:44:03.061 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6829/1115930619", 2023-10-26T01:44:03.061 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:44:03.061 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:44:03.061 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:44:03.061 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:03.062 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:03.062 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:44:03.062 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_xinfo": [ 2023-10-26T01:44:03.062 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:03.062 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 0, 2023-10-26T01:44:03.063 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:44:03.063 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:44:03.063 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:44:03.063 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:44:03.063 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:44:03.064 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.889675+0000", 2023-10-26T01:44:03.064 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:44:03.064 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:03.064 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:03.064 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 1, 2023-10-26T01:44:03.064 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:44:03.065 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:44:03.065 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:44:03.065 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:44:03.065 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:44:03.065 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.864157+0000", 2023-10-26T01:44:03.066 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:44:03.066 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:03.066 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:03.066 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 2, 2023-10-26T01:44:03.066 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:44:03.067 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:44:03.067 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:44:03.067 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:44:03.067 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:44:03.067 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.876919+0000", 2023-10-26T01:44:03.067 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:44:03.068 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:03.068 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:03.068 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 3, 2023-10-26T01:44:03.068 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:44:03.068 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:44:03.069 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:44:03.069 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:44:03.069 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:44:03.069 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.974541+0000", 2023-10-26T01:44:03.069 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:44:03.070 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:03.070 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:03.070 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 4, 2023-10-26T01:44:03.070 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:44:03.070 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:44:03.070 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:44:03.071 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:44:03.071 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:44:03.071 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.969696+0000", 2023-10-26T01:44:03.071 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:44:03.071 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:03.072 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:03.072 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 5, 2023-10-26T01:44:03.072 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:44:03.072 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:44:03.072 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:44:03.072 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:44:03.073 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:44:03.073 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.859703+0000", 2023-10-26T01:44:03.073 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:44:03.073 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:03.073 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:03.073 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 6, 2023-10-26T01:44:03.074 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:44:03.074 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:44:03.074 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:44:03.074 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:44:03.074 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:44:03.075 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.867917+0000", 2023-10-26T01:44:03.075 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:44:03.075 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:03.075 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:03.075 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 7, 2023-10-26T01:44:03.075 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:44:03.076 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:44:03.076 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:44:03.076 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:44:03.076 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:44:03.076 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.898605+0000", 2023-10-26T01:44:03.077 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:44:03.077 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:03.077 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:44:03.077 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_upmap": [], 2023-10-26T01:44:03.077 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_upmap_items": [], 2023-10-26T01:44:03.077 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_upmap_primaries": [], 2023-10-26T01:44:03.078 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_temp": [], 2023-10-26T01:44:03.078 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_temp": [], 2023-10-26T01:44:03.078 INFO:teuthology.orchestra.run.smithi169.stdout: "blocklist": {}, 2023-10-26T01:44:03.078 INFO:teuthology.orchestra.run.smithi169.stdout: "range_blocklist": {}, 2023-10-26T01:44:03.078 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profiles": { 2023-10-26T01:44:03.079 INFO:teuthology.orchestra.run.smithi169.stdout: "default": { 2023-10-26T01:44:03.079 INFO:teuthology.orchestra.run.smithi169.stdout: "crush-failure-domain": "osd", 2023-10-26T01:44:03.079 INFO:teuthology.orchestra.run.smithi169.stdout: "k": "2", 2023-10-26T01:44:03.079 INFO:teuthology.orchestra.run.smithi169.stdout: "m": "1", 2023-10-26T01:44:03.079 INFO:teuthology.orchestra.run.smithi169.stdout: "plugin": "jerasure", 2023-10-26T01:44:03.079 INFO:teuthology.orchestra.run.smithi169.stdout: "technique": "reed_sol_van" 2023-10-26T01:44:03.080 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:03.080 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:03.080 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps_queue": [], 2023-10-26T01:44:03.080 INFO:teuthology.orchestra.run.smithi169.stdout: "new_removed_snaps": [], 2023-10-26T01:44:03.080 INFO:teuthology.orchestra.run.smithi169.stdout: "new_purged_snaps": [], 2023-10-26T01:44:03.080 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_node_flags": {}, 2023-10-26T01:44:03.081 INFO:teuthology.orchestra.run.smithi169.stdout: "device_class_flags": {}, 2023-10-26T01:44:03.081 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_mode": { 2023-10-26T01:44:03.081 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_mode_enabled": false, 2023-10-26T01:44:03.081 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_bucket_count": 0, 2023-10-26T01:44:03.081 INFO:teuthology.orchestra.run.smithi169.stdout: "degraded_stretch_mode": 0, 2023-10-26T01:44:03.082 INFO:teuthology.orchestra.run.smithi169.stdout: "recovering_stretch_mode": 0, 2023-10-26T01:44:03.082 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_mode_bucket": 0 2023-10-26T01:44:03.082 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:03.082 INFO:teuthology.orchestra.run.smithi169.stdout:} 2023-10-26T01:44:03.084 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:02.891+0000 7f3cdac1a640 1 -- 172.21.15.169:0/3708967166 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3cb806ae40 msgr2=0x7f3cb806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:03.084 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:02.891+0000 7f3cdac1a640 1 --2- 172.21.15.169:0/3708967166 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3cb806ae40 0x7f3cb806d300 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f3cc8035000 tx=0x7f3cc8004730 comp rx=0 tx=0).stop 2023-10-26T01:44:03.084 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:02.891+0000 7f3cdac1a640 1 -- 172.21.15.169:0/3708967166 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3cd41592d0 msgr2=0x7f3cd40b0590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:03.084 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:02.891+0000 7f3cdac1a640 1 --2- 172.21.15.169:0/3708967166 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3cd41592d0 0x7f3cd40b0590 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc00b9a0 tx=0x7f3ccc00be60 comp rx=0 tx=0).stop 2023-10-26T01:44:03.084 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:02.892+0000 7f3cdac1a640 1 -- 172.21.15.169:0/3708967166 shutdown_connections 2023-10-26T01:44:03.085 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:02.892+0000 7f3cdac1a640 1 --2- 172.21.15.169:0/3708967166 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3cb806ae40 0x7f3cb806d300 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:03.085 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:02.892+0000 7f3cdac1a640 1 --2- 172.21.15.169:0/3708967166 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3cd41592d0 0x7f3cd40b0590 unknown :-1 s=CLOSED pgs=1876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:03.085 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:02.892+0000 7f3cdac1a640 1 --2- 172.21.15.169:0/3708967166 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3cd40cde00 0x7f3cd40b0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:03.085 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:02.892+0000 7f3cdac1a640 1 --2- 172.21.15.169:0/3708967166 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3cd40ac930 0x7f3cd41598c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:03.085 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:02.892+0000 7f3cdac1a640 1 -- 172.21.15.169:0/3708967166 >> 172.21.15.169:0/3708967166 conn(0x7f3cd4069930 msgr2=0x7f3cd406ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:03.086 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:02.892+0000 7f3cdac1a640 1 -- 172.21.15.169:0/3708967166 shutdown_connections 2023-10-26T01:44:03.086 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:02.892+0000 7f3cdac1a640 1 -- 172.21.15.169:0/3708967166 wait complete. 2023-10-26T01:44:03.086 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2023-10-26T01:44:08.088 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-26T01:44:08.192 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T01:44:08.198 DEBUG:teuthology.orchestra.run.smithi170:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T01:44:08.311 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:08.309+0000 7f707e9d7640 1 -- 172.21.15.169:0/1068152087 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f707806ce20 msgr2=0x7f707814e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:08.311 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:08.309+0000 7f707e9d7640 1 --2- 172.21.15.169:0/1068152087 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f707806ce20 0x7f707814e790 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7f707806eaf0 tx=0x7f706802c1c0 comp rx=0 tx=0).stop 2023-10-26T01:44:08.311 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:08.309+0000 7f707e9d7640 1 -- 172.21.15.169:0/1068152087 shutdown_connections 2023-10-26T01:44:08.311 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:08.309+0000 7f707e9d7640 1 --2- 172.21.15.169:0/1068152087 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f707806ad30 0x7f707806b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:08.312 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:08.309+0000 7f707e9d7640 1 --2- 172.21.15.169:0/1068152087 >> v2:172.21.15.170:3300/0 conn(0x7f7078158850 0x7f70780aab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:08.312 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:08.309+0000 7f707e9d7640 1 --2- 172.21.15.169:0/1068152087 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f707806ce20 0x7f707814e790 unknown :-1 s=CLOSED pgs=1877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:08.312 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:08.309+0000 7f707e9d7640 1 -- 172.21.15.169:0/1068152087 >> 172.21.15.169:0/1068152087 conn(0x7f7078069930 msgr2=0x7f7078069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:08.312 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:08.309+0000 7f707e9d7640 1 -- 172.21.15.169:0/1068152087 shutdown_connections 2023-10-26T01:44:08.313 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:08.309+0000 7f707e9d7640 1 -- 172.21.15.169:0/1068152087 wait complete. 2023-10-26T01:44:08.315 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:08.311+0000 7f707e9d7640 1 Processor -- start 2023-10-26T01:44:08.315 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:08.311+0000 7f707e9d7640 1 -- start start 2023-10-26T01:44:08.316 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:08.312+0000 7f707e9d7640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f707806ad30 0x7f7078096610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:08.316 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:08.312+0000 7f707e9d7640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f707806ce20 0x7f707814ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:08.316 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:08.312+0000 7f707e9d7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7078158850 0x7f7078151020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:08.316 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:08.312+0000 7f707e9d7640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70780aeec0 con 0x7f707806ad30 2023-10-26T01:44:08.317 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:08.312+0000 7f707e9d7640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7078096330 con 0x7f7078158850 2023-10-26T01:44:08.317 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:08.312+0000 7f707e9d7640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70780af0d0 con 0x7f707806ce20 2023-10-26T01:44:08.317 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:08.312+0000 7f7077fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f707806ad30 0x7f7078096610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:08.317 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:08.312+0000 7f7077fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f707806ad30 0x7f7078096610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:52192/0 (socket says 172.21.15.169:52192) 2023-10-26T01:44:08.317 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:08.312+0000 7f7077fff640 1 -- 172.21.15.169:0/2828880711 learned_addr learned my addr 172.21.15.169:0/2828880711 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:44:08.318 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:08.312+0000 7f707cf74640 1 --2- 172.21.15.169:0/2828880711 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7078158850 0x7f7078151020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:08.318 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:08.312+0000 7f70777fe640 1 --2- 172.21.15.169:0/2828880711 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f707806ce20 0x7f707814ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:08.318 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:08.312+0000 7f7077fff640 1 -- 172.21.15.169:0/2828880711 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f707806ce20 msgr2=0x7f707814ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:08.318 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:08.312+0000 7f7077fff640 1 --2- 172.21.15.169:0/2828880711 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f707806ce20 0x7f707814ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:08.319 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:08.312+0000 7f7077fff640 1 -- 172.21.15.169:0/2828880711 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7078158850 msgr2=0x7f7078151020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:08.319 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:08.312+0000 7f7077fff640 1 --2- 172.21.15.169:0/2828880711 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7078158850 0x7f7078151020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:08.319 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:08.313+0000 7f7077fff640 1 -- 172.21.15.169:0/2828880711 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7068009900 con 0x7f707806ad30 2023-10-26T01:44:08.320 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:08.313+0000 7f7077fff640 1 --2- 172.21.15.169:0/2828880711 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f707806ad30 0x7f7078096610 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7f7068000f80 tx=0x7f7068035e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:08.320 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:08.313+0000 7f7074ff9640 1 -- 172.21.15.169:0/2828880711 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7068034440 con 0x7f707806ad30 2023-10-26T01:44:08.320 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:08.313+0000 7f7074ff9640 1 -- 172.21.15.169:0/2828880711 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7068004540 con 0x7f707806ad30 2023-10-26T01:44:08.320 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:08.313+0000 7f7074ff9640 1 -- 172.21.15.169:0/2828880711 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7068003e60 con 0x7f707806ad30 2023-10-26T01:44:08.320 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:08.313+0000 7f707e9d7640 1 -- 172.21.15.169:0/2828880711 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f707814c790 con 0x7f707806ad30 2023-10-26T01:44:08.321 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:08.313+0000 7f707e9d7640 1 -- 172.21.15.169:0/2828880711 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7078096d50 con 0x7f707806ad30 2023-10-26T01:44:08.322 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:08.314+0000 7f707e9d7640 1 -- 172.21.15.169:0/2828880711 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f707804db10 con 0x7f707806ad30 2023-10-26T01:44:08.322 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:08.315+0000 7f7074ff9640 1 -- 172.21.15.169:0/2828880711 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f706803b040 con 0x7f707806ad30 2023-10-26T01:44:08.322 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:08.317+0000 7f7074ff9640 1 --2- 172.21.15.169:0/2828880711 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f705806ae40 0x7f705806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:08.322 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:08.317+0000 7f7074ff9640 1 -- 172.21.15.169:0/2828880711 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f70680ac350 con 0x7f707806ad30 2023-10-26T01:44:08.323 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:08.317+0000 7f70777fe640 1 --2- 172.21.15.169:0/2828880711 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f705806ae40 0x7f705806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:08.323 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:08.318+0000 7f70777fe640 1 --2- 172.21.15.169:0/2828880711 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f705806ae40 0x7f705806d300 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f707814d830 tx=0x7f7060005e30 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:08.323 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:08.322+0000 7f7074ff9640 1 -- 172.21.15.169:0/2828880711 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f70680773f0 con 0x7f707806ad30 2023-10-26T01:44:08.516 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:08.514+0000 7f707e9d7640 1 -- 172.21.15.169:0/2828880711 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f707814e8b0 con 0x7f707806ad30 2023-10-26T01:44:08.519 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:08.517+0000 7f7074ff9640 1 -- 172.21.15.169:0/2828880711 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v183) v1 ==== 81+0+34062 (secure 0 0 0) 0x7f706807aa10 con 0x7f707806ad30 2023-10-26T01:44:08.520 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:44:08.520 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2023-10-26T01:44:08.520 INFO:teuthology.orchestra.run.smithi169.stdout: "epoch": 183, 2023-10-26T01:44:08.520 INFO:teuthology.orchestra.run.smithi169.stdout: "fsid": "30203b28-ed8a-44e8-a9a6-a9bf25f980f1", 2023-10-26T01:44:08.521 INFO:teuthology.orchestra.run.smithi169.stdout: "created": "2023-10-26T01:30:22.897155+0000", 2023-10-26T01:44:08.521 INFO:teuthology.orchestra.run.smithi169.stdout: "modified": "2023-10-26T01:44:01.477790+0000", 2023-10-26T01:44:08.521 INFO:teuthology.orchestra.run.smithi169.stdout: "last_up_change": "2023-10-26T01:30:31.875281+0000", 2023-10-26T01:44:08.521 INFO:teuthology.orchestra.run.smithi169.stdout: "last_in_change": "2023-10-26T01:30:27.637941+0000", 2023-10-26T01:44:08.521 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-26T01:44:08.522 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_num": 5799936, 2023-10-26T01:44:08.522 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_set": [ 2023-10-26T01:44:08.522 INFO:teuthology.orchestra.run.smithi169.stdout: "pglog_hardlimit", 2023-10-26T01:44:08.522 INFO:teuthology.orchestra.run.smithi169.stdout: "purged_snapdirs", 2023-10-26T01:44:08.522 INFO:teuthology.orchestra.run.smithi169.stdout: "recovery_deletes", 2023-10-26T01:44:08.523 INFO:teuthology.orchestra.run.smithi169.stdout: "sortbitwise" 2023-10-26T01:44:08.523 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:44:08.523 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_version": 4, 2023-10-26T01:44:08.523 INFO:teuthology.orchestra.run.smithi169.stdout: "full_ratio": 0.69999998807907104, 2023-10-26T01:44:08.523 INFO:teuthology.orchestra.run.smithi169.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-26T01:44:08.524 INFO:teuthology.orchestra.run.smithi169.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-26T01:44:08.524 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_snapshot": "", 2023-10-26T01:44:08.524 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_max": 23, 2023-10-26T01:44:08.524 INFO:teuthology.orchestra.run.smithi169.stdout: "max_osd": 8, 2023-10-26T01:44:08.524 INFO:teuthology.orchestra.run.smithi169.stdout: "require_min_compat_client": "luminous", 2023-10-26T01:44:08.525 INFO:teuthology.orchestra.run.smithi169.stdout: "min_compat_client": "jewel", 2023-10-26T01:44:08.525 INFO:teuthology.orchestra.run.smithi169.stdout: "require_osd_release": "reef", 2023-10-26T01:44:08.525 INFO:teuthology.orchestra.run.smithi169.stdout: "allow_crimson": false, 2023-10-26T01:44:08.525 INFO:teuthology.orchestra.run.smithi169.stdout: "pools": [ 2023-10-26T01:44:08.525 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:08.525 INFO:teuthology.orchestra.run.smithi169.stdout: "pool": 1, 2023-10-26T01:44:08.526 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_name": ".mgr", 2023-10-26T01:44:08.526 INFO:teuthology.orchestra.run.smithi169.stdout: "create_time": "2023-10-26T01:30:32.200808+0000", 2023-10-26T01:44:08.526 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": 1, 2023-10-26T01:44:08.526 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_names": "hashpspool", 2023-10-26T01:44:08.526 INFO:teuthology.orchestra.run.smithi169.stdout: "type": 1, 2023-10-26T01:44:08.527 INFO:teuthology.orchestra.run.smithi169.stdout: "size": 2, 2023-10-26T01:44:08.527 INFO:teuthology.orchestra.run.smithi169.stdout: "min_size": 1, 2023-10-26T01:44:08.527 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_rule": 0, 2023-10-26T01:44:08.527 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_count": 0, 2023-10-26T01:44:08.527 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_target": 0, 2023-10-26T01:44:08.528 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T01:44:08.528 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T01:44:08.528 INFO:teuthology.orchestra.run.smithi169.stdout: "object_hash": 2, 2023-10-26T01:44:08.528 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_mode": "off", 2023-10-26T01:44:08.528 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num": 1, 2023-10-26T01:44:08.529 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num": 1, 2023-10-26T01:44:08.529 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num_target": 1, 2023-10-26T01:44:08.529 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_target": 1, 2023-10-26T01:44:08.529 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_pending": 1, 2023-10-26T01:44:08.529 INFO:teuthology.orchestra.run.smithi169.stdout: "last_pg_merge_meta": { 2023-10-26T01:44:08.529 INFO:teuthology.orchestra.run.smithi169.stdout: "source_pgid": "0.0", 2023-10-26T01:44:08.530 INFO:teuthology.orchestra.run.smithi169.stdout: "ready_epoch": 0, 2023-10-26T01:44:08.530 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_started": 0, 2023-10-26T01:44:08.530 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_clean": 0, 2023-10-26T01:44:08.530 INFO:teuthology.orchestra.run.smithi169.stdout: "source_version": "0'0", 2023-10-26T01:44:08.531 INFO:teuthology.orchestra.run.smithi169.stdout: "target_version": "0'0" 2023-10-26T01:44:08.531 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:08.531 INFO:teuthology.orchestra.run.smithi169.stdout: "last_change": "109", 2023-10-26T01:44:08.531 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend": "0", 2023-10-26T01:44:08.531 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T01:44:08.531 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T01:44:08.532 INFO:teuthology.orchestra.run.smithi169.stdout: "auid": 0, 2023-10-26T01:44:08.532 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_mode": "selfmanaged", 2023-10-26T01:44:08.532 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_seq": 0, 2023-10-26T01:44:08.532 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_epoch": 0, 2023-10-26T01:44:08.532 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_snaps": [], 2023-10-26T01:44:08.533 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps": "[]", 2023-10-26T01:44:08.533 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_bytes": 0, 2023-10-26T01:44:08.533 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_objects": 0, 2023-10-26T01:44:08.533 INFO:teuthology.orchestra.run.smithi169.stdout: "tiers": [], 2023-10-26T01:44:08.533 INFO:teuthology.orchestra.run.smithi169.stdout: "tier_of": -1, 2023-10-26T01:44:08.534 INFO:teuthology.orchestra.run.smithi169.stdout: "read_tier": -1, 2023-10-26T01:44:08.534 INFO:teuthology.orchestra.run.smithi169.stdout: "write_tier": -1, 2023-10-26T01:44:08.534 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_mode": "none", 2023-10-26T01:44:08.534 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_bytes": 0, 2023-10-26T01:44:08.534 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_objects": 0, 2023-10-26T01:44:08.535 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T01:44:08.535 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T01:44:08.535 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T01:44:08.535 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_flush_age": 0, 2023-10-26T01:44:08.536 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_evict_age": 0, 2023-10-26T01:44:08.536 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profile": "", 2023-10-26T01:44:08.536 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_params": { 2023-10-26T01:44:08.536 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "none" 2023-10-26T01:44:08.536 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:08.536 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_period": 0, 2023-10-26T01:44:08.537 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_count": 0, 2023-10-26T01:44:08.537 INFO:teuthology.orchestra.run.smithi169.stdout: "use_gmt_hitset": true, 2023-10-26T01:44:08.537 INFO:teuthology.orchestra.run.smithi169.stdout: "min_read_recency_for_promote": 0, 2023-10-26T01:44:08.537 INFO:teuthology.orchestra.run.smithi169.stdout: "min_write_recency_for_promote": 0, 2023-10-26T01:44:08.538 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T01:44:08.538 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_search_last_n": 0, 2023-10-26T01:44:08.538 INFO:teuthology.orchestra.run.smithi169.stdout: "grade_table": [], 2023-10-26T01:44:08.538 INFO:teuthology.orchestra.run.smithi169.stdout: "stripe_width": 0, 2023-10-26T01:44:08.538 INFO:teuthology.orchestra.run.smithi169.stdout: "expected_num_objects": 0, 2023-10-26T01:44:08.539 INFO:teuthology.orchestra.run.smithi169.stdout: "fast_read": false, 2023-10-26T01:44:08.539 INFO:teuthology.orchestra.run.smithi169.stdout: "options": { 2023-10-26T01:44:08.539 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_max": 32, 2023-10-26T01:44:08.539 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_min": 1 2023-10-26T01:44:08.540 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:08.540 INFO:teuthology.orchestra.run.smithi169.stdout: "application_metadata": { 2023-10-26T01:44:08.540 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": {} 2023-10-26T01:44:08.540 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:08.541 INFO:teuthology.orchestra.run.smithi169.stdout: "read_balance": { 2023-10-26T01:44:08.541 INFO:teuthology.orchestra.run.smithi169.stdout: "score_acting": 8, 2023-10-26T01:44:08.541 INFO:teuthology.orchestra.run.smithi169.stdout: "score_stable": 8, 2023-10-26T01:44:08.541 INFO:teuthology.orchestra.run.smithi169.stdout: "optimal_score": 0.25, 2023-10-26T01:44:08.541 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_acting": 2, 2023-10-26T01:44:08.542 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_stable": 2, 2023-10-26T01:44:08.542 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity_weighted": 1, 2023-10-26T01:44:08.542 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity": 1, 2023-10-26T01:44:08.542 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity_weighted": 1 2023-10-26T01:44:08.542 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:08.543 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:08.543 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:08.543 INFO:teuthology.orchestra.run.smithi169.stdout: "pool": 22, 2023-10-26T01:44:08.543 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_name": "cephfs_metadata", 2023-10-26T01:44:08.544 INFO:teuthology.orchestra.run.smithi169.stdout: "create_time": "2023-10-26T01:43:47.501296+0000", 2023-10-26T01:44:08.544 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": 1, 2023-10-26T01:44:08.544 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_names": "hashpspool", 2023-10-26T01:44:08.544 INFO:teuthology.orchestra.run.smithi169.stdout: "type": 1, 2023-10-26T01:44:08.544 INFO:teuthology.orchestra.run.smithi169.stdout: "size": 2, 2023-10-26T01:44:08.545 INFO:teuthology.orchestra.run.smithi169.stdout: "min_size": 1, 2023-10-26T01:44:08.545 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_rule": 0, 2023-10-26T01:44:08.545 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_count": 0, 2023-10-26T01:44:08.545 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_target": 0, 2023-10-26T01:44:08.546 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T01:44:08.546 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T01:44:08.546 INFO:teuthology.orchestra.run.smithi169.stdout: "object_hash": 2, 2023-10-26T01:44:08.546 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_mode": "off", 2023-10-26T01:44:08.546 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num": 32, 2023-10-26T01:44:08.547 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num": 32, 2023-10-26T01:44:08.547 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num_target": 32, 2023-10-26T01:44:08.547 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_target": 32, 2023-10-26T01:44:08.547 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_pending": 32, 2023-10-26T01:44:08.547 INFO:teuthology.orchestra.run.smithi169.stdout: "last_pg_merge_meta": { 2023-10-26T01:44:08.547 INFO:teuthology.orchestra.run.smithi169.stdout: "source_pgid": "0.0", 2023-10-26T01:44:08.548 INFO:teuthology.orchestra.run.smithi169.stdout: "ready_epoch": 0, 2023-10-26T01:44:08.548 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_started": 0, 2023-10-26T01:44:08.548 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_clean": 0, 2023-10-26T01:44:08.548 INFO:teuthology.orchestra.run.smithi169.stdout: "source_version": "0'0", 2023-10-26T01:44:08.548 INFO:teuthology.orchestra.run.smithi169.stdout: "target_version": "0'0" 2023-10-26T01:44:08.549 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:08.549 INFO:teuthology.orchestra.run.smithi169.stdout: "last_change": "181", 2023-10-26T01:44:08.549 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend": "0", 2023-10-26T01:44:08.549 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T01:44:08.549 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T01:44:08.550 INFO:teuthology.orchestra.run.smithi169.stdout: "auid": 0, 2023-10-26T01:44:08.550 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_mode": "selfmanaged", 2023-10-26T01:44:08.550 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_seq": 0, 2023-10-26T01:44:08.550 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_epoch": 0, 2023-10-26T01:44:08.550 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_snaps": [], 2023-10-26T01:44:08.551 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps": "[]", 2023-10-26T01:44:08.551 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_bytes": 0, 2023-10-26T01:44:08.551 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_objects": 0, 2023-10-26T01:44:08.551 INFO:teuthology.orchestra.run.smithi169.stdout: "tiers": [], 2023-10-26T01:44:08.551 INFO:teuthology.orchestra.run.smithi169.stdout: "tier_of": -1, 2023-10-26T01:44:08.552 INFO:teuthology.orchestra.run.smithi169.stdout: "read_tier": -1, 2023-10-26T01:44:08.552 INFO:teuthology.orchestra.run.smithi169.stdout: "write_tier": -1, 2023-10-26T01:44:08.552 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_mode": "none", 2023-10-26T01:44:08.552 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_bytes": 0, 2023-10-26T01:44:08.552 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_objects": 0, 2023-10-26T01:44:08.553 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T01:44:08.553 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T01:44:08.553 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T01:44:08.553 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_flush_age": 0, 2023-10-26T01:44:08.553 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_evict_age": 0, 2023-10-26T01:44:08.554 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profile": "", 2023-10-26T01:44:08.554 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_params": { 2023-10-26T01:44:08.554 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "none" 2023-10-26T01:44:08.554 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:08.554 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_period": 0, 2023-10-26T01:44:08.555 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_count": 0, 2023-10-26T01:44:08.555 INFO:teuthology.orchestra.run.smithi169.stdout: "use_gmt_hitset": true, 2023-10-26T01:44:08.555 INFO:teuthology.orchestra.run.smithi169.stdout: "min_read_recency_for_promote": 0, 2023-10-26T01:44:08.555 INFO:teuthology.orchestra.run.smithi169.stdout: "min_write_recency_for_promote": 0, 2023-10-26T01:44:08.555 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T01:44:08.555 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_search_last_n": 0, 2023-10-26T01:44:08.556 INFO:teuthology.orchestra.run.smithi169.stdout: "grade_table": [], 2023-10-26T01:44:08.556 INFO:teuthology.orchestra.run.smithi169.stdout: "stripe_width": 0, 2023-10-26T01:44:08.556 INFO:teuthology.orchestra.run.smithi169.stdout: "expected_num_objects": 0, 2023-10-26T01:44:08.557 INFO:teuthology.orchestra.run.smithi169.stdout: "fast_read": false, 2023-10-26T01:44:08.557 INFO:teuthology.orchestra.run.smithi169.stdout: "options": { 2023-10-26T01:44:08.557 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_bias": 4, 2023-10-26T01:44:08.557 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_min": 16, 2023-10-26T01:44:08.557 INFO:teuthology.orchestra.run.smithi169.stdout: "recovery_priority": 5 2023-10-26T01:44:08.558 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:08.558 INFO:teuthology.orchestra.run.smithi169.stdout: "application_metadata": { 2023-10-26T01:44:08.558 INFO:teuthology.orchestra.run.smithi169.stdout: "cephfs": { 2023-10-26T01:44:08.558 INFO:teuthology.orchestra.run.smithi169.stdout: "metadata": "cephfs" 2023-10-26T01:44:08.558 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:08.559 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:08.559 INFO:teuthology.orchestra.run.smithi169.stdout: "read_balance": { 2023-10-26T01:44:08.559 INFO:teuthology.orchestra.run.smithi169.stdout: "score_acting": 2, 2023-10-26T01:44:08.559 INFO:teuthology.orchestra.run.smithi169.stdout: "score_stable": 2, 2023-10-26T01:44:08.559 INFO:teuthology.orchestra.run.smithi169.stdout: "optimal_score": 1, 2023-10-26T01:44:08.560 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_acting": 2, 2023-10-26T01:44:08.560 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_stable": 2, 2023-10-26T01:44:08.560 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity_weighted": 1, 2023-10-26T01:44:08.560 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity": 1, 2023-10-26T01:44:08.560 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity_weighted": 1 2023-10-26T01:44:08.561 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:08.561 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:08.561 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:08.561 INFO:teuthology.orchestra.run.smithi169.stdout: "pool": 23, 2023-10-26T01:44:08.561 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_name": "cephfs_data", 2023-10-26T01:44:08.562 INFO:teuthology.orchestra.run.smithi169.stdout: "create_time": "2023-10-26T01:43:48.737507+0000", 2023-10-26T01:44:08.562 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": 1, 2023-10-26T01:44:08.562 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_names": "hashpspool", 2023-10-26T01:44:08.562 INFO:teuthology.orchestra.run.smithi169.stdout: "type": 1, 2023-10-26T01:44:08.562 INFO:teuthology.orchestra.run.smithi169.stdout: "size": 2, 2023-10-26T01:44:08.563 INFO:teuthology.orchestra.run.smithi169.stdout: "min_size": 1, 2023-10-26T01:44:08.563 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_rule": 0, 2023-10-26T01:44:08.563 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_count": 0, 2023-10-26T01:44:08.563 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_target": 0, 2023-10-26T01:44:08.563 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T01:44:08.563 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T01:44:08.564 INFO:teuthology.orchestra.run.smithi169.stdout: "object_hash": 2, 2023-10-26T01:44:08.564 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_mode": "off", 2023-10-26T01:44:08.564 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num": 64, 2023-10-26T01:44:08.564 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num": 64, 2023-10-26T01:44:08.564 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num_target": 64, 2023-10-26T01:44:08.565 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_target": 64, 2023-10-26T01:44:08.565 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_pending": 64, 2023-10-26T01:44:08.565 INFO:teuthology.orchestra.run.smithi169.stdout: "last_pg_merge_meta": { 2023-10-26T01:44:08.565 INFO:teuthology.orchestra.run.smithi169.stdout: "source_pgid": "0.0", 2023-10-26T01:44:08.565 INFO:teuthology.orchestra.run.smithi169.stdout: "ready_epoch": 0, 2023-10-26T01:44:08.566 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_started": 0, 2023-10-26T01:44:08.566 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_clean": 0, 2023-10-26T01:44:08.566 INFO:teuthology.orchestra.run.smithi169.stdout: "source_version": "0'0", 2023-10-26T01:44:08.566 INFO:teuthology.orchestra.run.smithi169.stdout: "target_version": "0'0" 2023-10-26T01:44:08.567 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:08.567 INFO:teuthology.orchestra.run.smithi169.stdout: "last_change": "183", 2023-10-26T01:44:08.567 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend": "0", 2023-10-26T01:44:08.567 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T01:44:08.567 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T01:44:08.568 INFO:teuthology.orchestra.run.smithi169.stdout: "auid": 0, 2023-10-26T01:44:08.568 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_mode": "selfmanaged", 2023-10-26T01:44:08.568 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_seq": 0, 2023-10-26T01:44:08.568 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_epoch": 0, 2023-10-26T01:44:08.568 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_snaps": [], 2023-10-26T01:44:08.569 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps": "[]", 2023-10-26T01:44:08.569 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_bytes": 33554432, 2023-10-26T01:44:08.569 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_objects": 0, 2023-10-26T01:44:08.569 INFO:teuthology.orchestra.run.smithi169.stdout: "tiers": [], 2023-10-26T01:44:08.569 INFO:teuthology.orchestra.run.smithi169.stdout: "tier_of": -1, 2023-10-26T01:44:08.569 INFO:teuthology.orchestra.run.smithi169.stdout: "read_tier": -1, 2023-10-26T01:44:08.570 INFO:teuthology.orchestra.run.smithi169.stdout: "write_tier": -1, 2023-10-26T01:44:08.570 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_mode": "none", 2023-10-26T01:44:08.570 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_bytes": 0, 2023-10-26T01:44:08.570 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_objects": 0, 2023-10-26T01:44:08.570 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T01:44:08.571 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T01:44:08.571 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T01:44:08.571 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_flush_age": 0, 2023-10-26T01:44:08.571 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_evict_age": 0, 2023-10-26T01:44:08.571 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profile": "", 2023-10-26T01:44:08.572 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_params": { 2023-10-26T01:44:08.572 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "none" 2023-10-26T01:44:08.572 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:08.572 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_period": 0, 2023-10-26T01:44:08.572 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_count": 0, 2023-10-26T01:44:08.572 INFO:teuthology.orchestra.run.smithi169.stdout: "use_gmt_hitset": true, 2023-10-26T01:44:08.573 INFO:teuthology.orchestra.run.smithi169.stdout: "min_read_recency_for_promote": 0, 2023-10-26T01:44:08.573 INFO:teuthology.orchestra.run.smithi169.stdout: "min_write_recency_for_promote": 0, 2023-10-26T01:44:08.573 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T01:44:08.573 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_search_last_n": 0, 2023-10-26T01:44:08.573 INFO:teuthology.orchestra.run.smithi169.stdout: "grade_table": [], 2023-10-26T01:44:08.574 INFO:teuthology.orchestra.run.smithi169.stdout: "stripe_width": 0, 2023-10-26T01:44:08.574 INFO:teuthology.orchestra.run.smithi169.stdout: "expected_num_objects": 0, 2023-10-26T01:44:08.574 INFO:teuthology.orchestra.run.smithi169.stdout: "fast_read": false, 2023-10-26T01:44:08.574 INFO:teuthology.orchestra.run.smithi169.stdout: "options": { 2023-10-26T01:44:08.574 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_min": 64, 2023-10-26T01:44:08.575 INFO:teuthology.orchestra.run.smithi169.stdout: "target_size_ratio": 0.89999997615814209 2023-10-26T01:44:08.575 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:08.575 INFO:teuthology.orchestra.run.smithi169.stdout: "application_metadata": { 2023-10-26T01:44:08.575 INFO:teuthology.orchestra.run.smithi169.stdout: "cephfs": { 2023-10-26T01:44:08.575 INFO:teuthology.orchestra.run.smithi169.stdout: "data": "cephfs" 2023-10-26T01:44:08.576 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:08.576 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:08.576 INFO:teuthology.orchestra.run.smithi169.stdout: "read_balance": { 2023-10-26T01:44:08.576 INFO:teuthology.orchestra.run.smithi169.stdout: "score_acting": 1.3799999952316284, 2023-10-26T01:44:08.576 INFO:teuthology.orchestra.run.smithi169.stdout: "score_stable": 1.3799999952316284, 2023-10-26T01:44:08.577 INFO:teuthology.orchestra.run.smithi169.stdout: "optimal_score": 1, 2023-10-26T01:44:08.577 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_acting": 1.3799999952316284, 2023-10-26T01:44:08.577 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_stable": 1.3799999952316284, 2023-10-26T01:44:08.577 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity_weighted": 1, 2023-10-26T01:44:08.577 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity": 1, 2023-10-26T01:44:08.578 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity_weighted": 1 2023-10-26T01:44:08.578 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:08.578 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:08.578 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:44:08.578 INFO:teuthology.orchestra.run.smithi169.stdout: "osds": [ 2023-10-26T01:44:08.579 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:08.579 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 0, 2023-10-26T01:44:08.579 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:44:08.579 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:44:08.579 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:44:08.580 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:44:08.580 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:44:08.580 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:44:08.580 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:44:08.580 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:44:08.580 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 180, 2023-10-26T01:44:08.581 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:44:08.581 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:44:08.581 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:44:08.581 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:08.581 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:08.582 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:08.582 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6816", 2023-10-26T01:44:08.582 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:44:08.582 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:08.582 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:08.583 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:08.583 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6817", 2023-10-26T01:44:08.583 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:44:08.583 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:08.583 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:08.583 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:08.584 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:44:08.584 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:08.584 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:08.584 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:08.584 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6818", 2023-10-26T01:44:08.585 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:44:08.585 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:08.585 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:08.585 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:08.585 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6819", 2023-10-26T01:44:08.586 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:44:08.586 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:08.586 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:08.586 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:08.586 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:44:08.587 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:08.587 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:08.587 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:08.587 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6822", 2023-10-26T01:44:08.588 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:44:08.588 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:08.588 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:08.588 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:08.589 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6823", 2023-10-26T01:44:08.589 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:44:08.589 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:08.589 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:08.590 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:08.590 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:44:08.590 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:08.590 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:08.590 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:08.591 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6820", 2023-10-26T01:44:08.591 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:44:08.591 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:08.591 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:08.592 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:08.592 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6821", 2023-10-26T01:44:08.592 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:44:08.592 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:08.592 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:08.593 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:08.593 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6817/4204255864", 2023-10-26T01:44:08.593 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6819/4204255864", 2023-10-26T01:44:08.593 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6823/4204255864", 2023-10-26T01:44:08.593 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6821/4204255864", 2023-10-26T01:44:08.593 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:44:08.594 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:44:08.594 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:44:08.594 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:08.594 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:08.595 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:08.595 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 1, 2023-10-26T01:44:08.595 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:44:08.595 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:44:08.595 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:44:08.595 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:44:08.596 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:44:08.596 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:44:08.596 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:44:08.596 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:44:08.596 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 180, 2023-10-26T01:44:08.596 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:44:08.597 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:44:08.597 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:44:08.597 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:08.597 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:08.597 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:08.598 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6808", 2023-10-26T01:44:08.598 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:44:08.598 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:08.598 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:08.598 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:08.599 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6809", 2023-10-26T01:44:08.599 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:44:08.599 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:08.599 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:08.599 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:08.600 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:44:08.600 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:08.600 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:08.600 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:08.600 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6810", 2023-10-26T01:44:08.601 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:44:08.601 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:08.601 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:08.601 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:08.601 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6811", 2023-10-26T01:44:08.602 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:44:08.602 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:08.602 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:08.602 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:08.602 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:44:08.603 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:08.603 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:08.603 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:08.603 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6814", 2023-10-26T01:44:08.603 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:44:08.604 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:08.604 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:08.604 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:08.604 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6815", 2023-10-26T01:44:08.604 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:44:08.604 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:08.605 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:08.605 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:08.605 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:44:08.605 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:08.605 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:08.606 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:08.606 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6812", 2023-10-26T01:44:08.606 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:44:08.606 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:08.606 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:08.607 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:08.607 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6813", 2023-10-26T01:44:08.607 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:44:08.608 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:08.608 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:08.608 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:08.608 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6809/238206046", 2023-10-26T01:44:08.608 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6811/238206046", 2023-10-26T01:44:08.609 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6815/238206046", 2023-10-26T01:44:08.609 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6813/238206046", 2023-10-26T01:44:08.609 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:44:08.609 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:44:08.610 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:44:08.610 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:08.610 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:08.610 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:08.610 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 2, 2023-10-26T01:44:08.611 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:44:08.611 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:44:08.611 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:44:08.611 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:44:08.612 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:44:08.612 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:44:08.612 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:44:08.612 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:44:08.612 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 180, 2023-10-26T01:44:08.613 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:44:08.613 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:44:08.613 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:44:08.613 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:08.614 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:08.614 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:08.614 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6800", 2023-10-26T01:44:08.614 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:44:08.614 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:08.615 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:08.615 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:08.615 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6801", 2023-10-26T01:44:08.615 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:44:08.616 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:08.616 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:08.616 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:08.616 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:44:08.617 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:08.617 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:08.617 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:08.617 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6802", 2023-10-26T01:44:08.618 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:44:08.618 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:08.618 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:08.618 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:08.619 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6803", 2023-10-26T01:44:08.619 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:44:08.619 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:08.619 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:08.620 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:08.620 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:44:08.620 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:08.620 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:08.620 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:08.621 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6806", 2023-10-26T01:44:08.621 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:44:08.621 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:08.621 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:08.621 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:08.621 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6807", 2023-10-26T01:44:08.622 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:44:08.622 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:08.622 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:08.622 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:08.623 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:44:08.623 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:08.623 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:08.623 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:08.624 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6804", 2023-10-26T01:44:08.624 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:44:08.624 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:08.624 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:08.624 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:08.625 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6805", 2023-10-26T01:44:08.625 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:44:08.625 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:08.625 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:08.625 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:08.625 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6801/3215791846", 2023-10-26T01:44:08.626 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6803/3215791846", 2023-10-26T01:44:08.626 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6807/3215791846", 2023-10-26T01:44:08.626 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6805/3215791846", 2023-10-26T01:44:08.626 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:44:08.626 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:44:08.627 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:44:08.627 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:08.627 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:08.627 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:08.627 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 3, 2023-10-26T01:44:08.628 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:44:08.628 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:44:08.628 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:44:08.628 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:44:08.629 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:44:08.629 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:44:08.629 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:44:08.629 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 14, 2023-10-26T01:44:08.629 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 180, 2023-10-26T01:44:08.630 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:44:08.630 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:44:08.630 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:44:08.630 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:08.630 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:08.631 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:08.631 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6824", 2023-10-26T01:44:08.631 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:44:08.631 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:08.631 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:08.632 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:08.632 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6825", 2023-10-26T01:44:08.632 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:44:08.632 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:08.632 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:08.632 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:08.633 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:44:08.633 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:08.633 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:08.633 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:08.633 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6826", 2023-10-26T01:44:08.633 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:44:08.634 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:08.634 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:08.634 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:08.634 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6827", 2023-10-26T01:44:08.634 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:44:08.635 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:08.635 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:08.635 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:08.635 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:44:08.635 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:08.635 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:08.637 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:08.637 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6830", 2023-10-26T01:44:08.637 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:44:08.638 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:08.638 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:08.638 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:08.638 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6831", 2023-10-26T01:44:08.638 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:44:08.639 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:08.639 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:08.639 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:08.639 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:44:08.639 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:08.639 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:08.640 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:08.640 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6828", 2023-10-26T01:44:08.640 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:44:08.640 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:08.640 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:08.640 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:08.641 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6829", 2023-10-26T01:44:08.641 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:44:08.641 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:08.641 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:08.641 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:08.642 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6825/2256701226", 2023-10-26T01:44:08.642 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6827/2256701226", 2023-10-26T01:44:08.642 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6831/2256701226", 2023-10-26T01:44:08.642 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6829/2256701226", 2023-10-26T01:44:08.642 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:44:08.643 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:44:08.643 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:44:08.643 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:08.643 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:08.643 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:08.644 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 4, 2023-10-26T01:44:08.644 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:44:08.644 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:44:08.644 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:44:08.644 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:44:08.645 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:44:08.645 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:44:08.645 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:44:08.645 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:44:08.646 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 180, 2023-10-26T01:44:08.646 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:44:08.646 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:44:08.646 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:44:08.646 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:08.647 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:08.647 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:08.647 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6816", 2023-10-26T01:44:08.647 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:44:08.648 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:08.648 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:08.648 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:08.648 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6817", 2023-10-26T01:44:08.648 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:44:08.649 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:08.649 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:08.649 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:08.649 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:44:08.650 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:08.650 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:08.650 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:08.650 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6818", 2023-10-26T01:44:08.650 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:44:08.651 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:08.651 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:08.651 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:08.651 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6819", 2023-10-26T01:44:08.652 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:44:08.652 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:08.652 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:08.652 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:08.652 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:44:08.653 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:08.653 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:08.653 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:08.653 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6825", 2023-10-26T01:44:08.653 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:44:08.654 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:08.654 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:08.654 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:08.654 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6828", 2023-10-26T01:44:08.655 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:44:08.655 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:08.655 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:08.656 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:08.656 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:44:08.656 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:08.656 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:08.657 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:08.657 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6820", 2023-10-26T01:44:08.657 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:44:08.657 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:08.658 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:08.658 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:08.658 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6823", 2023-10-26T01:44:08.658 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:44:08.658 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:08.659 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:08.659 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:08.659 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6817/1082279874", 2023-10-26T01:44:08.659 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6819/1082279874", 2023-10-26T01:44:08.659 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6828/1082279874", 2023-10-26T01:44:08.660 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6823/1082279874", 2023-10-26T01:44:08.660 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:44:08.660 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:44:08.660 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:44:08.660 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:08.660 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:08.661 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:08.661 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 5, 2023-10-26T01:44:08.661 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:44:08.661 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:44:08.661 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:44:08.661 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:44:08.662 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:44:08.662 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:44:08.662 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:44:08.662 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:44:08.662 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 180, 2023-10-26T01:44:08.663 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:44:08.663 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:44:08.663 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:44:08.663 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:08.663 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:08.664 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:08.664 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6808", 2023-10-26T01:44:08.664 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:44:08.664 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:08.664 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:08.664 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:08.665 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6809", 2023-10-26T01:44:08.665 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:44:08.665 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:08.665 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:08.665 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:08.665 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:44:08.666 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:08.666 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:08.666 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:08.666 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6810", 2023-10-26T01:44:08.666 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:44:08.667 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:08.667 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:08.667 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:08.667 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6811", 2023-10-26T01:44:08.667 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:44:08.668 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:08.668 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:08.668 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:08.668 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:44:08.668 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:08.669 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:08.669 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:08.669 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6814", 2023-10-26T01:44:08.669 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:44:08.669 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:08.669 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:08.670 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:08.670 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6815", 2023-10-26T01:44:08.670 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:44:08.670 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:08.671 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:08.671 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:08.671 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:44:08.671 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:08.671 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:08.672 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:08.672 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6812", 2023-10-26T01:44:08.672 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:44:08.672 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:08.672 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:08.672 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:08.673 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6813", 2023-10-26T01:44:08.673 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:44:08.673 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:08.673 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:08.674 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:08.674 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6809/3335654351", 2023-10-26T01:44:08.674 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6811/3335654351", 2023-10-26T01:44:08.674 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6815/3335654351", 2023-10-26T01:44:08.674 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6813/3335654351", 2023-10-26T01:44:08.675 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:44:08.675 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:44:08.675 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:44:08.675 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:08.675 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:08.676 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:08.676 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 6, 2023-10-26T01:44:08.676 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:44:08.676 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:44:08.676 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:44:08.676 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:44:08.677 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:44:08.677 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:44:08.677 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:44:08.677 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:44:08.678 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 180, 2023-10-26T01:44:08.678 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:44:08.678 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:44:08.678 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:44:08.679 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:08.679 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:08.679 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:08.679 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6800", 2023-10-26T01:44:08.679 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:44:08.679 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:08.680 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:08.680 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:08.680 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6801", 2023-10-26T01:44:08.680 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:44:08.681 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:08.681 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:08.681 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:08.681 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:44:08.681 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:08.681 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:08.682 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:08.682 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6802", 2023-10-26T01:44:08.682 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:44:08.683 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:08.683 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:08.683 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:08.683 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6803", 2023-10-26T01:44:08.683 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:44:08.683 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:08.684 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:08.684 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:08.684 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:44:08.684 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:08.684 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:08.685 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:08.685 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6806", 2023-10-26T01:44:08.685 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:44:08.685 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:08.685 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:08.685 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:08.686 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6807", 2023-10-26T01:44:08.686 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:44:08.686 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:08.686 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:08.686 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:08.687 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:44:08.687 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:08.687 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:08.687 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:08.687 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6804", 2023-10-26T01:44:08.688 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:44:08.688 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:08.688 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:08.688 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:08.689 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6805", 2023-10-26T01:44:08.689 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:44:08.689 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:08.689 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:08.689 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:08.689 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6801/2520460697", 2023-10-26T01:44:08.690 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6803/2520460697", 2023-10-26T01:44:08.690 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6807/2520460697", 2023-10-26T01:44:08.690 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6805/2520460697", 2023-10-26T01:44:08.690 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:44:08.690 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:44:08.691 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:44:08.691 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:08.691 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:08.691 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:08.691 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 7, 2023-10-26T01:44:08.692 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:44:08.692 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:44:08.692 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:44:08.692 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:44:08.692 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:44:08.692 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:44:08.693 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:44:08.693 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:44:08.693 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 180, 2023-10-26T01:44:08.693 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:44:08.693 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:44:08.693 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:44:08.694 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:08.694 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:08.694 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:08.694 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6821", 2023-10-26T01:44:08.694 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:44:08.695 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:08.695 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:08.695 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:08.695 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6822", 2023-10-26T01:44:08.695 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:44:08.696 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:08.696 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:08.696 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:08.696 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:44:08.696 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:08.696 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:08.697 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:08.697 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6824", 2023-10-26T01:44:08.697 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:44:08.697 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:08.697 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:08.698 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:08.698 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6826", 2023-10-26T01:44:08.698 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:44:08.698 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:08.698 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:08.698 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:08.699 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:44:08.699 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:08.699 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:08.699 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:08.699 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6830", 2023-10-26T01:44:08.700 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:44:08.700 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:08.700 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:08.701 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:08.701 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6831", 2023-10-26T01:44:08.701 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:44:08.701 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:08.702 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:08.702 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:08.702 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:44:08.703 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:08.703 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:08.703 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:08.704 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6827", 2023-10-26T01:44:08.704 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:44:08.704 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:08.704 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:08.705 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:08.705 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6829", 2023-10-26T01:44:08.705 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:44:08.706 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:08.706 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:08.706 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:08.706 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6822/1115930619", 2023-10-26T01:44:08.706 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6826/1115930619", 2023-10-26T01:44:08.707 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6831/1115930619", 2023-10-26T01:44:08.707 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6829/1115930619", 2023-10-26T01:44:08.707 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:44:08.707 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:44:08.707 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:44:08.708 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:08.708 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:08.708 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:44:08.708 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_xinfo": [ 2023-10-26T01:44:08.708 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:08.708 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 0, 2023-10-26T01:44:08.709 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:44:08.709 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:44:08.709 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:44:08.709 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:44:08.709 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:44:08.710 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.889675+0000", 2023-10-26T01:44:08.710 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:44:08.710 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:08.710 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:08.710 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 1, 2023-10-26T01:44:08.711 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:44:08.711 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:44:08.711 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:44:08.711 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:44:08.711 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:44:08.712 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.864157+0000", 2023-10-26T01:44:08.712 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:44:08.712 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:08.712 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:08.713 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 2, 2023-10-26T01:44:08.713 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:44:08.713 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:44:08.713 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:44:08.713 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:44:08.714 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:44:08.714 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.876919+0000", 2023-10-26T01:44:08.714 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:44:08.714 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:08.715 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:08.715 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 3, 2023-10-26T01:44:08.715 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:44:08.715 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:44:08.715 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:44:08.716 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:44:08.716 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:44:08.716 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.974541+0000", 2023-10-26T01:44:08.716 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:44:08.716 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:08.717 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:08.717 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 4, 2023-10-26T01:44:08.717 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:44:08.717 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:44:08.717 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:44:08.717 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:44:08.718 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:44:08.718 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.969696+0000", 2023-10-26T01:44:08.718 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:44:08.718 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:08.718 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:08.719 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 5, 2023-10-26T01:44:08.719 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:44:08.719 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:44:08.719 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:44:08.719 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:44:08.720 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:44:08.720 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.859703+0000", 2023-10-26T01:44:08.720 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:44:08.720 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:08.720 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:08.720 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 6, 2023-10-26T01:44:08.721 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:44:08.721 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:44:08.721 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:44:08.721 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:44:08.721 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:44:08.722 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.867917+0000", 2023-10-26T01:44:08.722 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:44:08.722 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:08.722 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:08.722 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 7, 2023-10-26T01:44:08.723 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:44:08.723 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:44:08.723 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:44:08.723 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:44:08.723 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:44:08.724 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.898605+0000", 2023-10-26T01:44:08.724 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:44:08.724 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:08.724 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:44:08.725 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_upmap": [], 2023-10-26T01:44:08.725 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_upmap_items": [], 2023-10-26T01:44:08.725 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_upmap_primaries": [], 2023-10-26T01:44:08.725 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_temp": [], 2023-10-26T01:44:08.725 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_temp": [], 2023-10-26T01:44:08.726 INFO:teuthology.orchestra.run.smithi169.stdout: "blocklist": {}, 2023-10-26T01:44:08.726 INFO:teuthology.orchestra.run.smithi169.stdout: "range_blocklist": {}, 2023-10-26T01:44:08.726 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profiles": { 2023-10-26T01:44:08.726 INFO:teuthology.orchestra.run.smithi169.stdout: "default": { 2023-10-26T01:44:08.727 INFO:teuthology.orchestra.run.smithi169.stdout: "crush-failure-domain": "osd", 2023-10-26T01:44:08.727 INFO:teuthology.orchestra.run.smithi169.stdout: "k": "2", 2023-10-26T01:44:08.727 INFO:teuthology.orchestra.run.smithi169.stdout: "m": "1", 2023-10-26T01:44:08.727 INFO:teuthology.orchestra.run.smithi169.stdout: "plugin": "jerasure", 2023-10-26T01:44:08.727 INFO:teuthology.orchestra.run.smithi169.stdout: "technique": "reed_sol_van" 2023-10-26T01:44:08.728 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:08.728 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:08.728 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps_queue": [], 2023-10-26T01:44:08.728 INFO:teuthology.orchestra.run.smithi169.stdout: "new_removed_snaps": [], 2023-10-26T01:44:08.728 INFO:teuthology.orchestra.run.smithi169.stdout: "new_purged_snaps": [], 2023-10-26T01:44:08.729 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_node_flags": {}, 2023-10-26T01:44:08.729 INFO:teuthology.orchestra.run.smithi169.stdout: "device_class_flags": {}, 2023-10-26T01:44:08.729 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_mode": { 2023-10-26T01:44:08.729 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_mode_enabled": false, 2023-10-26T01:44:08.729 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_bucket_count": 0, 2023-10-26T01:44:08.730 INFO:teuthology.orchestra.run.smithi169.stdout: "degraded_stretch_mode": 0, 2023-10-26T01:44:08.730 INFO:teuthology.orchestra.run.smithi169.stdout: "recovering_stretch_mode": 0, 2023-10-26T01:44:08.730 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_mode_bucket": 0 2023-10-26T01:44:08.730 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:08.731 INFO:teuthology.orchestra.run.smithi169.stdout:} 2023-10-26T01:44:08.732 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:08.520+0000 7f707e9d7640 1 -- 172.21.15.169:0/2828880711 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f705806ae40 msgr2=0x7f705806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:08.733 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:08.520+0000 7f707e9d7640 1 --2- 172.21.15.169:0/2828880711 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f705806ae40 0x7f705806d300 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f707814d830 tx=0x7f7060005e30 comp rx=0 tx=0).stop 2023-10-26T01:44:08.733 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:08.520+0000 7f707e9d7640 1 -- 172.21.15.169:0/2828880711 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f707806ad30 msgr2=0x7f7078096610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:08.733 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:08.520+0000 7f707e9d7640 1 --2- 172.21.15.169:0/2828880711 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f707806ad30 0x7f7078096610 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7f7068000f80 tx=0x7f7068035e30 comp rx=0 tx=0).stop 2023-10-26T01:44:08.734 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:08.520+0000 7f707e9d7640 1 -- 172.21.15.169:0/2828880711 shutdown_connections 2023-10-26T01:44:08.734 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:08.520+0000 7f707e9d7640 1 --2- 172.21.15.169:0/2828880711 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f705806ae40 0x7f705806d300 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:08.734 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:08.520+0000 7f707e9d7640 1 --2- 172.21.15.169:0/2828880711 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7078158850 0x7f7078151020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:08.734 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:08.520+0000 7f707e9d7640 1 --2- 172.21.15.169:0/2828880711 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f707806ce20 0x7f707814ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:08.735 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:08.520+0000 7f707e9d7640 1 --2- 172.21.15.169:0/2828880711 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f707806ad30 0x7f7078096610 unknown :-1 s=CLOSED pgs=1878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:08.735 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:08.520+0000 7f707e9d7640 1 -- 172.21.15.169:0/2828880711 >> 172.21.15.169:0/2828880711 conn(0x7f7078069930 msgr2=0x7f7078150800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:08.735 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:08.520+0000 7f707e9d7640 1 -- 172.21.15.169:0/2828880711 shutdown_connections 2023-10-26T01:44:08.735 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:08.521+0000 7f707e9d7640 1 -- 172.21.15.169:0/2828880711 wait complete. 2023-10-26T01:44:08.736 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2023-10-26T01:44:13.738 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-26T01:44:13.972 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:13.970+0000 7f0876165640 1 -- 172.21.15.169:0/1532725518 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08700ce1c0 msgr2=0x7f087006b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:13.972 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:13.970+0000 7f0876165640 1 --2- 172.21.15.169:0/1532725518 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08700ce1c0 0x7f087006b590 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7f0868005fd0 tx=0x7f086802c240 comp rx=0 tx=0).stop 2023-10-26T01:44:13.973 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:13.970+0000 7f0876165640 1 -- 172.21.15.169:0/1532725518 shutdown_connections 2023-10-26T01:44:13.973 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:13.970+0000 7f0876165640 1 --2- 172.21.15.169:0/1532725518 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08700ce1c0 0x7f087006b590 unknown :-1 s=CLOSED pgs=1879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:13.973 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:13.970+0000 7f0876165640 1 --2- 172.21.15.169:0/1532725518 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f08700b22b0 0x7f08700b28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:13.973 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:13.970+0000 7f0876165640 1 -- 172.21.15.169:0/1532725518 >> 172.21.15.169:0/1532725518 conn(0x7f0870069510 msgr2=0x7f08700698f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:13.974 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:13.970+0000 7f0876165640 1 -- 172.21.15.169:0/1532725518 shutdown_connections 2023-10-26T01:44:13.974 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:13.970+0000 7f0876165640 1 -- 172.21.15.169:0/1532725518 wait complete. 2023-10-26T01:44:13.975 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:13.972+0000 7f0876165640 1 Processor -- start 2023-10-26T01:44:13.975 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:13.972+0000 7f0876165640 1 -- start start 2023-10-26T01:44:13.976 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:13.973+0000 7f0876165640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f08700b22b0 0x7f087013d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:13.976 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:13.973+0000 7f0876165640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f08700ce1c0 0x7f08700c7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:13.976 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:13.973+0000 7f0876165640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f087013e8d0 0x7f0870158b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:13.976 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:13.973+0000 7f0876165640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08701f2a00 con 0x7f087013e8d0 2023-10-26T01:44:13.977 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:13.973+0000 7f0876165640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08701f2080 con 0x7f08700ce1c0 2023-10-26T01:44:13.977 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:13.973+0000 7f0876165640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08700c8760 con 0x7f08700b22b0 2023-10-26T01:44:13.977 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:13.973+0000 7f086ffff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f087013e8d0 0x7f0870158b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:13.977 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:13.973+0000 7f086ffff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f087013e8d0 0x7f0870158b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:52212/0 (socket says 172.21.15.169:52212) 2023-10-26T01:44:13.977 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:13.973+0000 7f086ffff640 1 -- 172.21.15.169:0/1236310514 learned_addr learned my addr 172.21.15.169:0/1236310514 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:44:13.978 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:13.974+0000 7f086ffff640 1 -- 172.21.15.169:0/1236310514 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f08700b22b0 msgr2=0x7f087013d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:13.978 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:13.974+0000 7f086f7fe640 1 --2- 172.21.15.169:0/1236310514 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f08700b22b0 0x7f087013d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:13.978 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:13.974+0000 7f086effd640 1 --2- 172.21.15.169:0/1236310514 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f08700ce1c0 0x7f08700c7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:13.978 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:13.974+0000 7f086ffff640 1 --2- 172.21.15.169:0/1236310514 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f08700b22b0 0x7f087013d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:13.979 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:13.974+0000 7f086ffff640 1 -- 172.21.15.169:0/1236310514 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f08700ce1c0 msgr2=0x7f08700c7dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:13.979 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:13.974+0000 7f086ffff640 1 --2- 172.21.15.169:0/1236310514 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f08700ce1c0 0x7f08700c7dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:13.979 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:13.974+0000 7f086ffff640 1 -- 172.21.15.169:0/1236310514 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0868009900 con 0x7f087013e8d0 2023-10-26T01:44:13.979 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:13.974+0000 7f086effd640 1 --2- 172.21.15.169:0/1236310514 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f08700ce1c0 0x7f08700c7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:44:13.981 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:13.974+0000 7f086ffff640 1 --2- 172.21.15.169:0/1236310514 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f087013e8d0 0x7f0870158b70 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7f0868037040 tx=0x7f0868035a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:13.981 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:13.974+0000 7f0867fff640 1 -- 172.21.15.169:0/1236310514 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0868034440 con 0x7f087013e8d0 2023-10-26T01:44:13.981 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:13.974+0000 7f0867fff640 1 -- 172.21.15.169:0/1236310514 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0868004330 con 0x7f087013e8d0 2023-10-26T01:44:13.981 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:13.974+0000 7f0867fff640 1 -- 172.21.15.169:0/1236310514 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08680046e0 con 0x7f087013e8d0 2023-10-26T01:44:13.982 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:13.975+0000 7f0876165640 1 -- 172.21.15.169:0/1236310514 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08701f2340 con 0x7f087013e8d0 2023-10-26T01:44:13.982 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:13.975+0000 7f0876165640 1 -- 172.21.15.169:0/1236310514 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08701f2e20 con 0x7f087013e8d0 2023-10-26T01:44:13.982 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:13.976+0000 7f0876165640 1 -- 172.21.15.169:0/1236310514 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08701f2b40 con 0x7f087013e8d0 2023-10-26T01:44:13.982 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:13.977+0000 7f0867fff640 1 -- 172.21.15.169:0/1236310514 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f086803b040 con 0x7f087013e8d0 2023-10-26T01:44:13.983 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:13.978+0000 7f0867fff640 1 --2- 172.21.15.169:0/1236310514 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f085006ae40 0x7f085006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:13.983 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:13.978+0000 7f0867fff640 1 -- 172.21.15.169:0/1236310514 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f08680ac140 con 0x7f087013e8d0 2023-10-26T01:44:13.983 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:13.978+0000 7f0867fff640 1 -- 172.21.15.169:0/1236310514 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=185}) v3 -- 0x7f0850072970 con 0x7f087013e8d0 2023-10-26T01:44:13.983 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:13.979+0000 7f086f7fe640 1 --2- 172.21.15.169:0/1236310514 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f085006ae40 0x7f085006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:13.984 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:13.980+0000 7f086f7fe640 1 --2- 172.21.15.169:0/1236310514 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f085006ae40 0x7f085006d300 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f087006c8d0 tx=0x7f0860008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:13.984 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:13.983+0000 7f0867fff640 1 -- 172.21.15.169:0/1236310514 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f08701f2b40 con 0x7f087013e8d0 2023-10-26T01:44:14.177 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:14.175+0000 7f0876165640 1 -- 172.21.15.169:0/1236310514 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f08700b2e00 con 0x7f087013e8d0 2023-10-26T01:44:14.180 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:14.178+0000 7f0867fff640 1 -- 172.21.15.169:0/1236310514 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v184) v1 ==== 81+0+34081 (secure 0 0 0) 0x7f08700b2e00 con 0x7f087013e8d0 2023-10-26T01:44:14.181 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:44:14.181 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2023-10-26T01:44:14.181 INFO:teuthology.orchestra.run.smithi169.stdout: "epoch": 184, 2023-10-26T01:44:14.181 INFO:teuthology.orchestra.run.smithi169.stdout: "fsid": "30203b28-ed8a-44e8-a9a6-a9bf25f980f1", 2023-10-26T01:44:14.182 INFO:teuthology.orchestra.run.smithi169.stdout: "created": "2023-10-26T01:30:22.897155+0000", 2023-10-26T01:44:14.182 INFO:teuthology.orchestra.run.smithi169.stdout: "modified": "2023-10-26T01:44:13.194893+0000", 2023-10-26T01:44:14.182 INFO:teuthology.orchestra.run.smithi169.stdout: "last_up_change": "2023-10-26T01:30:31.875281+0000", 2023-10-26T01:44:14.182 INFO:teuthology.orchestra.run.smithi169.stdout: "last_in_change": "2023-10-26T01:30:27.637941+0000", 2023-10-26T01:44:14.182 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-26T01:44:14.183 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_num": 5799936, 2023-10-26T01:44:14.183 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_set": [ 2023-10-26T01:44:14.183 INFO:teuthology.orchestra.run.smithi169.stdout: "pglog_hardlimit", 2023-10-26T01:44:14.183 INFO:teuthology.orchestra.run.smithi169.stdout: "purged_snapdirs", 2023-10-26T01:44:14.183 INFO:teuthology.orchestra.run.smithi169.stdout: "recovery_deletes", 2023-10-26T01:44:14.184 INFO:teuthology.orchestra.run.smithi169.stdout: "sortbitwise" 2023-10-26T01:44:14.184 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:44:14.184 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_version": 4, 2023-10-26T01:44:14.184 INFO:teuthology.orchestra.run.smithi169.stdout: "full_ratio": 0.69999998807907104, 2023-10-26T01:44:14.185 INFO:teuthology.orchestra.run.smithi169.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-26T01:44:14.185 INFO:teuthology.orchestra.run.smithi169.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-26T01:44:14.185 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_snapshot": "", 2023-10-26T01:44:14.185 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_max": 23, 2023-10-26T01:44:14.185 INFO:teuthology.orchestra.run.smithi169.stdout: "max_osd": 8, 2023-10-26T01:44:14.185 INFO:teuthology.orchestra.run.smithi169.stdout: "require_min_compat_client": "luminous", 2023-10-26T01:44:14.186 INFO:teuthology.orchestra.run.smithi169.stdout: "min_compat_client": "jewel", 2023-10-26T01:44:14.186 INFO:teuthology.orchestra.run.smithi169.stdout: "require_osd_release": "reef", 2023-10-26T01:44:14.186 INFO:teuthology.orchestra.run.smithi169.stdout: "allow_crimson": false, 2023-10-26T01:44:14.186 INFO:teuthology.orchestra.run.smithi169.stdout: "pools": [ 2023-10-26T01:44:14.186 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:14.187 INFO:teuthology.orchestra.run.smithi169.stdout: "pool": 1, 2023-10-26T01:44:14.187 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_name": ".mgr", 2023-10-26T01:44:14.187 INFO:teuthology.orchestra.run.smithi169.stdout: "create_time": "2023-10-26T01:30:32.200808+0000", 2023-10-26T01:44:14.187 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": 1, 2023-10-26T01:44:14.187 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_names": "hashpspool", 2023-10-26T01:44:14.188 INFO:teuthology.orchestra.run.smithi169.stdout: "type": 1, 2023-10-26T01:44:14.188 INFO:teuthology.orchestra.run.smithi169.stdout: "size": 2, 2023-10-26T01:44:14.188 INFO:teuthology.orchestra.run.smithi169.stdout: "min_size": 1, 2023-10-26T01:44:14.188 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_rule": 0, 2023-10-26T01:44:14.188 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_count": 0, 2023-10-26T01:44:14.189 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_target": 0, 2023-10-26T01:44:14.189 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T01:44:14.189 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T01:44:14.189 INFO:teuthology.orchestra.run.smithi169.stdout: "object_hash": 2, 2023-10-26T01:44:14.189 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_mode": "off", 2023-10-26T01:44:14.189 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num": 1, 2023-10-26T01:44:14.190 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num": 1, 2023-10-26T01:44:14.190 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num_target": 1, 2023-10-26T01:44:14.190 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_target": 1, 2023-10-26T01:44:14.190 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_pending": 1, 2023-10-26T01:44:14.190 INFO:teuthology.orchestra.run.smithi169.stdout: "last_pg_merge_meta": { 2023-10-26T01:44:14.191 INFO:teuthology.orchestra.run.smithi169.stdout: "source_pgid": "0.0", 2023-10-26T01:44:14.191 INFO:teuthology.orchestra.run.smithi169.stdout: "ready_epoch": 0, 2023-10-26T01:44:14.191 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_started": 0, 2023-10-26T01:44:14.191 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_clean": 0, 2023-10-26T01:44:14.191 INFO:teuthology.orchestra.run.smithi169.stdout: "source_version": "0'0", 2023-10-26T01:44:14.191 INFO:teuthology.orchestra.run.smithi169.stdout: "target_version": "0'0" 2023-10-26T01:44:14.192 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:14.192 INFO:teuthology.orchestra.run.smithi169.stdout: "last_change": "109", 2023-10-26T01:44:14.192 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend": "0", 2023-10-26T01:44:14.192 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T01:44:14.192 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T01:44:14.193 INFO:teuthology.orchestra.run.smithi169.stdout: "auid": 0, 2023-10-26T01:44:14.193 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_mode": "selfmanaged", 2023-10-26T01:44:14.193 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_seq": 0, 2023-10-26T01:44:14.193 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_epoch": 0, 2023-10-26T01:44:14.193 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_snaps": [], 2023-10-26T01:44:14.194 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps": "[]", 2023-10-26T01:44:14.194 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_bytes": 0, 2023-10-26T01:44:14.194 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_objects": 0, 2023-10-26T01:44:14.194 INFO:teuthology.orchestra.run.smithi169.stdout: "tiers": [], 2023-10-26T01:44:14.194 INFO:teuthology.orchestra.run.smithi169.stdout: "tier_of": -1, 2023-10-26T01:44:14.195 INFO:teuthology.orchestra.run.smithi169.stdout: "read_tier": -1, 2023-10-26T01:44:14.195 INFO:teuthology.orchestra.run.smithi169.stdout: "write_tier": -1, 2023-10-26T01:44:14.195 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_mode": "none", 2023-10-26T01:44:14.195 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_bytes": 0, 2023-10-26T01:44:14.195 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_objects": 0, 2023-10-26T01:44:14.195 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T01:44:14.196 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T01:44:14.196 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T01:44:14.196 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_flush_age": 0, 2023-10-26T01:44:14.196 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_evict_age": 0, 2023-10-26T01:44:14.196 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profile": "", 2023-10-26T01:44:14.197 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_params": { 2023-10-26T01:44:14.197 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "none" 2023-10-26T01:44:14.197 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:14.197 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_period": 0, 2023-10-26T01:44:14.197 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_count": 0, 2023-10-26T01:44:14.197 INFO:teuthology.orchestra.run.smithi169.stdout: "use_gmt_hitset": true, 2023-10-26T01:44:14.198 INFO:teuthology.orchestra.run.smithi169.stdout: "min_read_recency_for_promote": 0, 2023-10-26T01:44:14.198 INFO:teuthology.orchestra.run.smithi169.stdout: "min_write_recency_for_promote": 0, 2023-10-26T01:44:14.198 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T01:44:14.198 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_search_last_n": 0, 2023-10-26T01:44:14.198 INFO:teuthology.orchestra.run.smithi169.stdout: "grade_table": [], 2023-10-26T01:44:14.199 INFO:teuthology.orchestra.run.smithi169.stdout: "stripe_width": 0, 2023-10-26T01:44:14.199 INFO:teuthology.orchestra.run.smithi169.stdout: "expected_num_objects": 0, 2023-10-26T01:44:14.199 INFO:teuthology.orchestra.run.smithi169.stdout: "fast_read": false, 2023-10-26T01:44:14.199 INFO:teuthology.orchestra.run.smithi169.stdout: "options": { 2023-10-26T01:44:14.199 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_max": 32, 2023-10-26T01:44:14.200 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_min": 1 2023-10-26T01:44:14.200 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:14.200 INFO:teuthology.orchestra.run.smithi169.stdout: "application_metadata": { 2023-10-26T01:44:14.200 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": {} 2023-10-26T01:44:14.200 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:14.201 INFO:teuthology.orchestra.run.smithi169.stdout: "read_balance": { 2023-10-26T01:44:14.201 INFO:teuthology.orchestra.run.smithi169.stdout: "score_acting": 8, 2023-10-26T01:44:14.201 INFO:teuthology.orchestra.run.smithi169.stdout: "score_stable": 8, 2023-10-26T01:44:14.201 INFO:teuthology.orchestra.run.smithi169.stdout: "optimal_score": 0.25, 2023-10-26T01:44:14.201 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_acting": 2, 2023-10-26T01:44:14.201 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_stable": 2, 2023-10-26T01:44:14.202 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity_weighted": 1, 2023-10-26T01:44:14.202 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity": 1, 2023-10-26T01:44:14.202 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity_weighted": 1 2023-10-26T01:44:14.202 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:14.203 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:14.203 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:14.203 INFO:teuthology.orchestra.run.smithi169.stdout: "pool": 22, 2023-10-26T01:44:14.203 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_name": "cephfs_metadata", 2023-10-26T01:44:14.203 INFO:teuthology.orchestra.run.smithi169.stdout: "create_time": "2023-10-26T01:43:47.501296+0000", 2023-10-26T01:44:14.203 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": 1, 2023-10-26T01:44:14.204 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_names": "hashpspool", 2023-10-26T01:44:14.204 INFO:teuthology.orchestra.run.smithi169.stdout: "type": 1, 2023-10-26T01:44:14.204 INFO:teuthology.orchestra.run.smithi169.stdout: "size": 2, 2023-10-26T01:44:14.204 INFO:teuthology.orchestra.run.smithi169.stdout: "min_size": 1, 2023-10-26T01:44:14.204 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_rule": 0, 2023-10-26T01:44:14.205 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_count": 0, 2023-10-26T01:44:14.205 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_target": 0, 2023-10-26T01:44:14.205 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T01:44:14.205 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T01:44:14.205 INFO:teuthology.orchestra.run.smithi169.stdout: "object_hash": 2, 2023-10-26T01:44:14.206 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_mode": "off", 2023-10-26T01:44:14.206 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num": 32, 2023-10-26T01:44:14.206 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num": 32, 2023-10-26T01:44:14.206 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num_target": 32, 2023-10-26T01:44:14.206 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_target": 32, 2023-10-26T01:44:14.207 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_pending": 32, 2023-10-26T01:44:14.207 INFO:teuthology.orchestra.run.smithi169.stdout: "last_pg_merge_meta": { 2023-10-26T01:44:14.207 INFO:teuthology.orchestra.run.smithi169.stdout: "source_pgid": "0.0", 2023-10-26T01:44:14.207 INFO:teuthology.orchestra.run.smithi169.stdout: "ready_epoch": 0, 2023-10-26T01:44:14.207 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_started": 0, 2023-10-26T01:44:14.207 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_clean": 0, 2023-10-26T01:44:14.208 INFO:teuthology.orchestra.run.smithi169.stdout: "source_version": "0'0", 2023-10-26T01:44:14.208 INFO:teuthology.orchestra.run.smithi169.stdout: "target_version": "0'0" 2023-10-26T01:44:14.208 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:14.208 INFO:teuthology.orchestra.run.smithi169.stdout: "last_change": "181", 2023-10-26T01:44:14.208 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend": "0", 2023-10-26T01:44:14.208 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T01:44:14.209 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T01:44:14.209 INFO:teuthology.orchestra.run.smithi169.stdout: "auid": 0, 2023-10-26T01:44:14.209 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_mode": "selfmanaged", 2023-10-26T01:44:14.209 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_seq": 0, 2023-10-26T01:44:14.209 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_epoch": 0, 2023-10-26T01:44:14.210 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_snaps": [], 2023-10-26T01:44:14.210 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps": "[]", 2023-10-26T01:44:14.210 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_bytes": 0, 2023-10-26T01:44:14.210 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_objects": 0, 2023-10-26T01:44:14.210 INFO:teuthology.orchestra.run.smithi169.stdout: "tiers": [], 2023-10-26T01:44:14.210 INFO:teuthology.orchestra.run.smithi169.stdout: "tier_of": -1, 2023-10-26T01:44:14.211 INFO:teuthology.orchestra.run.smithi169.stdout: "read_tier": -1, 2023-10-26T01:44:14.211 INFO:teuthology.orchestra.run.smithi169.stdout: "write_tier": -1, 2023-10-26T01:44:14.211 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_mode": "none", 2023-10-26T01:44:14.211 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_bytes": 0, 2023-10-26T01:44:14.211 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_objects": 0, 2023-10-26T01:44:14.212 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T01:44:14.212 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T01:44:14.212 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T01:44:14.212 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_flush_age": 0, 2023-10-26T01:44:14.212 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_evict_age": 0, 2023-10-26T01:44:14.213 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profile": "", 2023-10-26T01:44:14.213 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_params": { 2023-10-26T01:44:14.213 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "none" 2023-10-26T01:44:14.213 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:14.213 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_period": 0, 2023-10-26T01:44:14.213 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_count": 0, 2023-10-26T01:44:14.214 INFO:teuthology.orchestra.run.smithi169.stdout: "use_gmt_hitset": true, 2023-10-26T01:44:14.214 INFO:teuthology.orchestra.run.smithi169.stdout: "min_read_recency_for_promote": 0, 2023-10-26T01:44:14.214 INFO:teuthology.orchestra.run.smithi169.stdout: "min_write_recency_for_promote": 0, 2023-10-26T01:44:14.214 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T01:44:14.214 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_search_last_n": 0, 2023-10-26T01:44:14.215 INFO:teuthology.orchestra.run.smithi169.stdout: "grade_table": [], 2023-10-26T01:44:14.215 INFO:teuthology.orchestra.run.smithi169.stdout: "stripe_width": 0, 2023-10-26T01:44:14.215 INFO:teuthology.orchestra.run.smithi169.stdout: "expected_num_objects": 0, 2023-10-26T01:44:14.215 INFO:teuthology.orchestra.run.smithi169.stdout: "fast_read": false, 2023-10-26T01:44:14.215 INFO:teuthology.orchestra.run.smithi169.stdout: "options": { 2023-10-26T01:44:14.215 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_bias": 4, 2023-10-26T01:44:14.216 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_min": 16, 2023-10-26T01:44:14.216 INFO:teuthology.orchestra.run.smithi169.stdout: "recovery_priority": 5 2023-10-26T01:44:14.216 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:14.216 INFO:teuthology.orchestra.run.smithi169.stdout: "application_metadata": { 2023-10-26T01:44:14.216 INFO:teuthology.orchestra.run.smithi169.stdout: "cephfs": { 2023-10-26T01:44:14.217 INFO:teuthology.orchestra.run.smithi169.stdout: "metadata": "cephfs" 2023-10-26T01:44:14.217 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:14.217 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:14.217 INFO:teuthology.orchestra.run.smithi169.stdout: "read_balance": { 2023-10-26T01:44:14.217 INFO:teuthology.orchestra.run.smithi169.stdout: "score_acting": 2, 2023-10-26T01:44:14.217 INFO:teuthology.orchestra.run.smithi169.stdout: "score_stable": 2, 2023-10-26T01:44:14.218 INFO:teuthology.orchestra.run.smithi169.stdout: "optimal_score": 1, 2023-10-26T01:44:14.218 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_acting": 2, 2023-10-26T01:44:14.218 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_stable": 2, 2023-10-26T01:44:14.218 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity_weighted": 1, 2023-10-26T01:44:14.218 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity": 1, 2023-10-26T01:44:14.219 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity_weighted": 1 2023-10-26T01:44:14.219 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:14.219 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:14.219 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:14.219 INFO:teuthology.orchestra.run.smithi169.stdout: "pool": 23, 2023-10-26T01:44:14.220 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_name": "cephfs_data", 2023-10-26T01:44:14.220 INFO:teuthology.orchestra.run.smithi169.stdout: "create_time": "2023-10-26T01:43:48.737507+0000", 2023-10-26T01:44:14.220 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": 1027, 2023-10-26T01:44:14.220 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_names": "hashpspool,full,full_quota", 2023-10-26T01:44:14.220 INFO:teuthology.orchestra.run.smithi169.stdout: "type": 1, 2023-10-26T01:44:14.220 INFO:teuthology.orchestra.run.smithi169.stdout: "size": 2, 2023-10-26T01:44:14.221 INFO:teuthology.orchestra.run.smithi169.stdout: "min_size": 1, 2023-10-26T01:44:14.221 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_rule": 0, 2023-10-26T01:44:14.221 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_count": 0, 2023-10-26T01:44:14.221 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_target": 0, 2023-10-26T01:44:14.221 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T01:44:14.222 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T01:44:14.222 INFO:teuthology.orchestra.run.smithi169.stdout: "object_hash": 2, 2023-10-26T01:44:14.222 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_mode": "off", 2023-10-26T01:44:14.222 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num": 64, 2023-10-26T01:44:14.222 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num": 64, 2023-10-26T01:44:14.223 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num_target": 64, 2023-10-26T01:44:14.223 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_target": 64, 2023-10-26T01:44:14.223 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_pending": 64, 2023-10-26T01:44:14.223 INFO:teuthology.orchestra.run.smithi169.stdout: "last_pg_merge_meta": { 2023-10-26T01:44:14.223 INFO:teuthology.orchestra.run.smithi169.stdout: "source_pgid": "0.0", 2023-10-26T01:44:14.223 INFO:teuthology.orchestra.run.smithi169.stdout: "ready_epoch": 0, 2023-10-26T01:44:14.224 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_started": 0, 2023-10-26T01:44:14.224 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_clean": 0, 2023-10-26T01:44:14.224 INFO:teuthology.orchestra.run.smithi169.stdout: "source_version": "0'0", 2023-10-26T01:44:14.224 INFO:teuthology.orchestra.run.smithi169.stdout: "target_version": "0'0" 2023-10-26T01:44:14.224 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:14.225 INFO:teuthology.orchestra.run.smithi169.stdout: "last_change": "184", 2023-10-26T01:44:14.225 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend": "0", 2023-10-26T01:44:14.225 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T01:44:14.225 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T01:44:14.225 INFO:teuthology.orchestra.run.smithi169.stdout: "auid": 0, 2023-10-26T01:44:14.226 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_mode": "selfmanaged", 2023-10-26T01:44:14.226 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_seq": 0, 2023-10-26T01:44:14.226 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_epoch": 0, 2023-10-26T01:44:14.226 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_snaps": [], 2023-10-26T01:44:14.226 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps": "[]", 2023-10-26T01:44:14.226 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_bytes": 33554432, 2023-10-26T01:44:14.227 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_objects": 0, 2023-10-26T01:44:14.227 INFO:teuthology.orchestra.run.smithi169.stdout: "tiers": [], 2023-10-26T01:44:14.227 INFO:teuthology.orchestra.run.smithi169.stdout: "tier_of": -1, 2023-10-26T01:44:14.227 INFO:teuthology.orchestra.run.smithi169.stdout: "read_tier": -1, 2023-10-26T01:44:14.227 INFO:teuthology.orchestra.run.smithi169.stdout: "write_tier": -1, 2023-10-26T01:44:14.228 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_mode": "none", 2023-10-26T01:44:14.228 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_bytes": 0, 2023-10-26T01:44:14.228 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_objects": 0, 2023-10-26T01:44:14.228 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T01:44:14.228 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T01:44:14.228 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T01:44:14.229 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_flush_age": 0, 2023-10-26T01:44:14.229 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_evict_age": 0, 2023-10-26T01:44:14.229 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profile": "", 2023-10-26T01:44:14.229 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_params": { 2023-10-26T01:44:14.229 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "none" 2023-10-26T01:44:14.230 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:14.230 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_period": 0, 2023-10-26T01:44:14.230 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_count": 0, 2023-10-26T01:44:14.230 INFO:teuthology.orchestra.run.smithi169.stdout: "use_gmt_hitset": true, 2023-10-26T01:44:14.230 INFO:teuthology.orchestra.run.smithi169.stdout: "min_read_recency_for_promote": 0, 2023-10-26T01:44:14.231 INFO:teuthology.orchestra.run.smithi169.stdout: "min_write_recency_for_promote": 0, 2023-10-26T01:44:14.231 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T01:44:14.231 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_search_last_n": 0, 2023-10-26T01:44:14.231 INFO:teuthology.orchestra.run.smithi169.stdout: "grade_table": [], 2023-10-26T01:44:14.231 INFO:teuthology.orchestra.run.smithi169.stdout: "stripe_width": 0, 2023-10-26T01:44:14.231 INFO:teuthology.orchestra.run.smithi169.stdout: "expected_num_objects": 0, 2023-10-26T01:44:14.232 INFO:teuthology.orchestra.run.smithi169.stdout: "fast_read": false, 2023-10-26T01:44:14.232 INFO:teuthology.orchestra.run.smithi169.stdout: "options": { 2023-10-26T01:44:14.232 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_min": 64, 2023-10-26T01:44:14.232 INFO:teuthology.orchestra.run.smithi169.stdout: "target_size_ratio": 0.89999997615814209 2023-10-26T01:44:14.232 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:14.232 INFO:teuthology.orchestra.run.smithi169.stdout: "application_metadata": { 2023-10-26T01:44:14.233 INFO:teuthology.orchestra.run.smithi169.stdout: "cephfs": { 2023-10-26T01:44:14.233 INFO:teuthology.orchestra.run.smithi169.stdout: "data": "cephfs" 2023-10-26T01:44:14.233 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:14.233 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:14.233 INFO:teuthology.orchestra.run.smithi169.stdout: "read_balance": { 2023-10-26T01:44:14.234 INFO:teuthology.orchestra.run.smithi169.stdout: "score_acting": 1.3799999952316284, 2023-10-26T01:44:14.234 INFO:teuthology.orchestra.run.smithi169.stdout: "score_stable": 1.3799999952316284, 2023-10-26T01:44:14.234 INFO:teuthology.orchestra.run.smithi169.stdout: "optimal_score": 1, 2023-10-26T01:44:14.234 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_acting": 1.3799999952316284, 2023-10-26T01:44:14.234 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_stable": 1.3799999952316284, 2023-10-26T01:44:14.235 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity_weighted": 1, 2023-10-26T01:44:14.235 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity": 1, 2023-10-26T01:44:14.235 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity_weighted": 1 2023-10-26T01:44:14.235 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:14.235 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:14.235 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:44:14.236 INFO:teuthology.orchestra.run.smithi169.stdout: "osds": [ 2023-10-26T01:44:14.236 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:14.236 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 0, 2023-10-26T01:44:14.236 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:44:14.236 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:44:14.236 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:44:14.237 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:44:14.237 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:44:14.237 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:44:14.237 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:44:14.237 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:44:14.238 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 180, 2023-10-26T01:44:14.238 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:44:14.238 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:44:14.238 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:44:14.238 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:14.238 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:14.239 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:14.239 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6816", 2023-10-26T01:44:14.239 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:44:14.239 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:14.239 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:14.240 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:14.240 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6817", 2023-10-26T01:44:14.240 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:44:14.240 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:14.240 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:14.240 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:14.241 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:44:14.241 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:14.241 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:14.241 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:14.241 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6818", 2023-10-26T01:44:14.242 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:44:14.242 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:14.242 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:14.242 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:14.242 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6819", 2023-10-26T01:44:14.243 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:44:14.243 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:14.243 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:14.243 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:14.243 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:44:14.243 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:14.244 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:14.244 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:14.244 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6822", 2023-10-26T01:44:14.244 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:44:14.244 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:14.245 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:14.245 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:14.245 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6823", 2023-10-26T01:44:14.245 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:44:14.245 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:14.245 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:14.246 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:14.246 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:44:14.246 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:14.246 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:14.246 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:14.247 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6820", 2023-10-26T01:44:14.247 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:44:14.247 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:14.247 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:14.247 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:14.247 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6821", 2023-10-26T01:44:14.248 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:44:14.248 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:14.248 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:14.248 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:14.248 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6817/4204255864", 2023-10-26T01:44:14.249 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6819/4204255864", 2023-10-26T01:44:14.249 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6823/4204255864", 2023-10-26T01:44:14.249 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6821/4204255864", 2023-10-26T01:44:14.249 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:44:14.249 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:44:14.250 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:44:14.250 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:14.250 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:14.250 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:14.250 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 1, 2023-10-26T01:44:14.251 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:44:14.251 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:44:14.251 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:44:14.251 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:44:14.251 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:44:14.251 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:44:14.252 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:44:14.252 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:44:14.252 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 180, 2023-10-26T01:44:14.252 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:44:14.252 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:44:14.253 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:44:14.253 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:14.253 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:14.253 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:14.253 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6808", 2023-10-26T01:44:14.254 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:44:14.254 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:14.254 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:14.254 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:14.254 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6809", 2023-10-26T01:44:14.255 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:44:14.255 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:14.255 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:14.255 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:14.255 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:44:14.255 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:14.256 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:14.256 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:14.256 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6810", 2023-10-26T01:44:14.256 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:44:14.256 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:14.257 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:14.257 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:14.257 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6811", 2023-10-26T01:44:14.257 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:44:14.257 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:14.257 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:14.258 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:14.258 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:44:14.258 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:14.258 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:14.258 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:14.259 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6814", 2023-10-26T01:44:14.259 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:44:14.259 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:14.259 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:14.259 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:14.260 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6815", 2023-10-26T01:44:14.260 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:44:14.260 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:14.260 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:14.260 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:14.260 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:44:14.261 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:14.261 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:14.261 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:14.261 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6812", 2023-10-26T01:44:14.261 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:44:14.262 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:14.262 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:14.262 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:14.262 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6813", 2023-10-26T01:44:14.262 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:44:14.262 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:14.263 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:14.263 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:14.263 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6809/238206046", 2023-10-26T01:44:14.263 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6811/238206046", 2023-10-26T01:44:14.263 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6815/238206046", 2023-10-26T01:44:14.264 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6813/238206046", 2023-10-26T01:44:14.264 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:44:14.264 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:44:14.264 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:44:14.264 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:14.264 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:14.265 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:14.265 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 2, 2023-10-26T01:44:14.265 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:44:14.265 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:44:14.265 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:44:14.266 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:44:14.266 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:44:14.266 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:44:14.266 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:44:14.266 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:44:14.266 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 180, 2023-10-26T01:44:14.267 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:44:14.267 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:44:14.267 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:44:14.267 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:14.267 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:14.268 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:14.268 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6800", 2023-10-26T01:44:14.268 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:44:14.268 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:14.268 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:14.268 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:14.269 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6801", 2023-10-26T01:44:14.269 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:44:14.269 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:14.269 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:14.269 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:14.270 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:44:14.270 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:14.270 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:14.270 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:14.270 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6802", 2023-10-26T01:44:14.271 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:44:14.271 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:14.271 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:14.271 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:14.271 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6803", 2023-10-26T01:44:14.271 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:44:14.272 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:14.272 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:14.272 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:14.272 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:44:14.272 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:14.272 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:14.273 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:14.273 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6806", 2023-10-26T01:44:14.273 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:44:14.273 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:14.273 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:14.274 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:14.274 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6807", 2023-10-26T01:44:14.274 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:44:14.274 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:14.274 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:14.275 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:14.275 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:44:14.275 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:14.275 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:14.275 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:14.276 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6804", 2023-10-26T01:44:14.276 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:44:14.276 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:14.276 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:14.276 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:14.276 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6805", 2023-10-26T01:44:14.277 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:44:14.277 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:14.277 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:14.277 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:14.277 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6801/3215791846", 2023-10-26T01:44:14.278 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6803/3215791846", 2023-10-26T01:44:14.278 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6807/3215791846", 2023-10-26T01:44:14.278 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6805/3215791846", 2023-10-26T01:44:14.278 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:44:14.278 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:44:14.278 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:44:14.279 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:14.279 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:14.279 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:14.279 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 3, 2023-10-26T01:44:14.279 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:44:14.280 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:44:14.280 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:44:14.280 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:44:14.280 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:44:14.280 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:44:14.281 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:44:14.281 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 14, 2023-10-26T01:44:14.281 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 180, 2023-10-26T01:44:14.281 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:44:14.281 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:44:14.281 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:44:14.282 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:14.282 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:14.282 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:14.282 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6824", 2023-10-26T01:44:14.282 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:44:14.283 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:14.283 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:14.283 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:14.283 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6825", 2023-10-26T01:44:14.283 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:44:14.283 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:14.284 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:14.284 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:14.284 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:44:14.284 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:14.284 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:14.285 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:14.285 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6826", 2023-10-26T01:44:14.285 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:44:14.285 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:14.285 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:14.285 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:14.286 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6827", 2023-10-26T01:44:14.286 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:44:14.286 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:14.286 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:14.286 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:14.287 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:44:14.287 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:14.287 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:14.287 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:14.287 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6830", 2023-10-26T01:44:14.287 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:44:14.288 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:14.288 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:14.288 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:14.288 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6831", 2023-10-26T01:44:14.288 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:44:14.289 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:14.289 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:14.289 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:14.289 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:44:14.289 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:14.289 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:14.290 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:14.290 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6828", 2023-10-26T01:44:14.290 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:44:14.290 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:14.290 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:14.291 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:14.291 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6829", 2023-10-26T01:44:14.291 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:44:14.291 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:14.291 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:14.292 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:14.292 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6825/2256701226", 2023-10-26T01:44:14.292 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6827/2256701226", 2023-10-26T01:44:14.292 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6831/2256701226", 2023-10-26T01:44:14.292 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6829/2256701226", 2023-10-26T01:44:14.292 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:44:14.293 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:44:14.293 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:44:14.293 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:14.293 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:14.293 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:14.294 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 4, 2023-10-26T01:44:14.294 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:44:14.294 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:44:14.294 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:44:14.294 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:44:14.295 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:44:14.295 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:44:14.295 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:44:14.295 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:44:14.295 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 180, 2023-10-26T01:44:14.295 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:44:14.296 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:44:14.296 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:44:14.296 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:14.296 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:14.296 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:14.297 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6816", 2023-10-26T01:44:14.297 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:44:14.297 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:14.297 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:14.297 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:14.297 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6817", 2023-10-26T01:44:14.298 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:44:14.298 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:14.298 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:14.298 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:14.298 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:44:14.299 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:14.299 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:14.299 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:14.299 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6818", 2023-10-26T01:44:14.299 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:44:14.300 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:14.300 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:14.300 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:14.300 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6819", 2023-10-26T01:44:14.300 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:44:14.300 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:14.301 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:14.301 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:14.301 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:44:14.301 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:14.301 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:14.302 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:14.302 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6825", 2023-10-26T01:44:14.302 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:44:14.302 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:14.302 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:14.303 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:14.303 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6828", 2023-10-26T01:44:14.303 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:44:14.303 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:14.303 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:14.303 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:14.304 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:44:14.304 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:14.304 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:14.304 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:14.304 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6820", 2023-10-26T01:44:14.304 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:44:14.305 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:14.305 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:14.305 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:14.305 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6823", 2023-10-26T01:44:14.305 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:44:14.306 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:14.306 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:14.306 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:14.306 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6817/1082279874", 2023-10-26T01:44:14.306 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6819/1082279874", 2023-10-26T01:44:14.307 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6828/1082279874", 2023-10-26T01:44:14.307 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6823/1082279874", 2023-10-26T01:44:14.307 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:44:14.307 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:44:14.307 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:44:14.307 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:14.308 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:14.308 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:14.308 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 5, 2023-10-26T01:44:14.308 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:44:14.308 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:44:14.309 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:44:14.309 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:44:14.309 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:44:14.309 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:44:14.309 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:44:14.309 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:44:14.310 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 180, 2023-10-26T01:44:14.310 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:44:14.310 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:44:14.310 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:44:14.310 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:14.311 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:14.311 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:14.311 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6808", 2023-10-26T01:44:14.311 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:44:14.311 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:14.312 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:14.312 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:14.312 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6809", 2023-10-26T01:44:14.312 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:44:14.312 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:14.312 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:14.313 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:14.313 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:44:14.313 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:14.313 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:14.313 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:14.314 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6810", 2023-10-26T01:44:14.314 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:44:14.314 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:14.314 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:14.314 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:14.315 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6811", 2023-10-26T01:44:14.315 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:44:14.315 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:14.315 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:14.315 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:14.315 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:44:14.316 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:14.316 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:14.316 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:14.316 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6814", 2023-10-26T01:44:14.316 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:44:14.317 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:14.317 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:14.317 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:14.317 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6815", 2023-10-26T01:44:14.317 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:44:14.317 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:14.318 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:14.318 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:14.318 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:44:14.318 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:14.318 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:14.319 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:14.319 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6812", 2023-10-26T01:44:14.319 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:44:14.319 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:14.319 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:14.320 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:14.320 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6813", 2023-10-26T01:44:14.320 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:44:14.320 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:14.320 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:14.321 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:14.321 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6809/3335654351", 2023-10-26T01:44:14.321 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6811/3335654351", 2023-10-26T01:44:14.321 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6815/3335654351", 2023-10-26T01:44:14.321 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6813/3335654351", 2023-10-26T01:44:14.321 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:44:14.322 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:44:14.322 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:44:14.322 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:14.322 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:14.322 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:14.323 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 6, 2023-10-26T01:44:14.323 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:44:14.323 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:44:14.323 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:44:14.323 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:44:14.324 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:44:14.324 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:44:14.324 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:44:14.324 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:44:14.324 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 180, 2023-10-26T01:44:14.325 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:44:14.325 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:44:14.325 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:44:14.325 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:14.325 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:14.326 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:14.326 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6800", 2023-10-26T01:44:14.326 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:44:14.326 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:14.326 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:14.327 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:14.327 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6801", 2023-10-26T01:44:14.327 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:44:14.327 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:14.327 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:14.327 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:14.328 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:44:14.328 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:14.328 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:14.328 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:14.328 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6802", 2023-10-26T01:44:14.329 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:44:14.329 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:14.329 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:14.329 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:14.329 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6803", 2023-10-26T01:44:14.329 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:44:14.330 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:14.330 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:14.330 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:14.330 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:44:14.330 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:14.331 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:14.331 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:14.331 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6806", 2023-10-26T01:44:14.331 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:44:14.331 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:14.331 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:14.332 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:14.332 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6807", 2023-10-26T01:44:14.332 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:44:14.332 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:14.332 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:14.333 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:14.333 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:44:14.333 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:14.333 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:14.333 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:14.333 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6804", 2023-10-26T01:44:14.334 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:44:14.334 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:14.334 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:14.334 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:14.334 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6805", 2023-10-26T01:44:14.335 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:44:14.335 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:14.335 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:14.335 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:14.335 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6801/2520460697", 2023-10-26T01:44:14.336 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6803/2520460697", 2023-10-26T01:44:14.336 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6807/2520460697", 2023-10-26T01:44:14.336 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6805/2520460697", 2023-10-26T01:44:14.336 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:44:14.336 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:44:14.336 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:44:14.337 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:14.337 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:14.337 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:14.337 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 7, 2023-10-26T01:44:14.337 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:44:14.337 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:44:14.338 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:44:14.338 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:44:14.338 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:44:14.338 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:44:14.338 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:44:14.339 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:44:14.339 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 180, 2023-10-26T01:44:14.339 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:44:14.339 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:44:14.339 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:44:14.339 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:14.340 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:14.340 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:14.340 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6821", 2023-10-26T01:44:14.340 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:44:14.340 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:14.341 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:14.341 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:14.341 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6822", 2023-10-26T01:44:14.341 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:44:14.341 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:14.342 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:14.342 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:14.342 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:44:14.342 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:14.342 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:14.343 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:14.343 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6824", 2023-10-26T01:44:14.343 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:44:14.343 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:14.343 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:14.344 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:14.344 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6826", 2023-10-26T01:44:14.344 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:44:14.344 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:14.345 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:14.345 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:14.345 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:44:14.345 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:14.345 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:14.345 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:14.346 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6830", 2023-10-26T01:44:14.346 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:44:14.346 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:14.346 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:14.346 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:14.347 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6831", 2023-10-26T01:44:14.347 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:44:14.347 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:14.347 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:14.347 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:14.348 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:44:14.348 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:14.348 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:14.348 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:14.348 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6827", 2023-10-26T01:44:14.349 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:44:14.349 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:14.349 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:14.349 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:14.349 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6829", 2023-10-26T01:44:14.349 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:44:14.350 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:14.350 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:14.350 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:14.350 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6822/1115930619", 2023-10-26T01:44:14.350 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6826/1115930619", 2023-10-26T01:44:14.351 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6831/1115930619", 2023-10-26T01:44:14.351 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6829/1115930619", 2023-10-26T01:44:14.351 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:44:14.351 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:44:14.351 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:44:14.352 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:14.352 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:14.352 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:44:14.352 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_xinfo": [ 2023-10-26T01:44:14.352 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:14.352 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 0, 2023-10-26T01:44:14.353 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:44:14.353 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:44:14.353 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:44:14.353 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:44:14.353 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:44:14.354 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.889675+0000", 2023-10-26T01:44:14.354 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:44:14.354 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:14.354 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:14.354 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 1, 2023-10-26T01:44:14.355 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:44:14.355 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:44:14.355 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:44:14.355 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:44:14.355 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:44:14.355 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.864157+0000", 2023-10-26T01:44:14.356 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:44:14.356 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:14.356 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:14.356 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 2, 2023-10-26T01:44:14.356 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:44:14.357 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:44:14.357 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:44:14.357 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:44:14.357 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:44:14.357 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.876919+0000", 2023-10-26T01:44:14.357 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:44:14.358 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:14.358 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:14.358 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 3, 2023-10-26T01:44:14.358 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:44:14.358 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:44:14.359 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:44:14.359 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:44:14.359 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:44:14.359 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.974541+0000", 2023-10-26T01:44:14.359 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:44:14.359 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:14.360 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:14.360 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 4, 2023-10-26T01:44:14.360 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:44:14.360 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:44:14.360 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:44:14.361 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:44:14.361 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:44:14.361 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.969696+0000", 2023-10-26T01:44:14.361 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:44:14.361 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:14.362 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:14.362 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 5, 2023-10-26T01:44:14.362 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:44:14.362 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:44:14.362 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:44:14.362 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:44:14.363 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:44:14.363 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.859703+0000", 2023-10-26T01:44:14.363 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:44:14.363 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:14.363 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:14.364 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 6, 2023-10-26T01:44:14.364 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:44:14.365 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:44:14.366 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:44:14.366 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:44:14.366 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:44:14.366 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.867917+0000", 2023-10-26T01:44:14.366 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:44:14.367 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:14.367 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:14.367 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 7, 2023-10-26T01:44:14.367 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:44:14.367 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:44:14.367 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:44:14.368 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:44:14.368 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:44:14.368 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.898605+0000", 2023-10-26T01:44:14.368 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:44:14.368 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:14.369 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:44:14.369 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_upmap": [], 2023-10-26T01:44:14.369 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_upmap_items": [], 2023-10-26T01:44:14.369 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_upmap_primaries": [], 2023-10-26T01:44:14.369 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_temp": [], 2023-10-26T01:44:14.369 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_temp": [], 2023-10-26T01:44:14.370 INFO:teuthology.orchestra.run.smithi169.stdout: "blocklist": {}, 2023-10-26T01:44:14.370 INFO:teuthology.orchestra.run.smithi169.stdout: "range_blocklist": {}, 2023-10-26T01:44:14.370 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profiles": { 2023-10-26T01:44:14.370 INFO:teuthology.orchestra.run.smithi169.stdout: "default": { 2023-10-26T01:44:14.370 INFO:teuthology.orchestra.run.smithi169.stdout: "crush-failure-domain": "osd", 2023-10-26T01:44:14.371 INFO:teuthology.orchestra.run.smithi169.stdout: "k": "2", 2023-10-26T01:44:14.371 INFO:teuthology.orchestra.run.smithi169.stdout: "m": "1", 2023-10-26T01:44:14.371 INFO:teuthology.orchestra.run.smithi169.stdout: "plugin": "jerasure", 2023-10-26T01:44:14.371 INFO:teuthology.orchestra.run.smithi169.stdout: "technique": "reed_sol_van" 2023-10-26T01:44:14.371 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:14.371 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:14.372 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps_queue": [], 2023-10-26T01:44:14.372 INFO:teuthology.orchestra.run.smithi169.stdout: "new_removed_snaps": [], 2023-10-26T01:44:14.372 INFO:teuthology.orchestra.run.smithi169.stdout: "new_purged_snaps": [], 2023-10-26T01:44:14.372 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_node_flags": {}, 2023-10-26T01:44:14.372 INFO:teuthology.orchestra.run.smithi169.stdout: "device_class_flags": {}, 2023-10-26T01:44:14.373 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_mode": { 2023-10-26T01:44:14.373 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_mode_enabled": false, 2023-10-26T01:44:14.373 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_bucket_count": 0, 2023-10-26T01:44:14.373 INFO:teuthology.orchestra.run.smithi169.stdout: "degraded_stretch_mode": 0, 2023-10-26T01:44:14.373 INFO:teuthology.orchestra.run.smithi169.stdout: "recovering_stretch_mode": 0, 2023-10-26T01:44:14.373 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_mode_bucket": 0 2023-10-26T01:44:14.374 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:14.374 INFO:teuthology.orchestra.run.smithi169.stdout:} 2023-10-26T01:44:14.375 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:14.181+0000 7f0876165640 1 -- 172.21.15.169:0/1236310514 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f085006ae40 msgr2=0x7f085006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:14.376 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:14.181+0000 7f0876165640 1 --2- 172.21.15.169:0/1236310514 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f085006ae40 0x7f085006d300 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f087006c8d0 tx=0x7f0860008040 comp rx=0 tx=0).stop 2023-10-26T01:44:14.376 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:14.181+0000 7f0876165640 1 -- 172.21.15.169:0/1236310514 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f087013e8d0 msgr2=0x7f0870158b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:14.376 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:14.181+0000 7f0876165640 1 --2- 172.21.15.169:0/1236310514 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f087013e8d0 0x7f0870158b70 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7f0868037040 tx=0x7f0868035a40 comp rx=0 tx=0).stop 2023-10-26T01:44:14.376 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:14.181+0000 7f0876165640 1 -- 172.21.15.169:0/1236310514 shutdown_connections 2023-10-26T01:44:14.377 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:14.181+0000 7f0876165640 1 --2- 172.21.15.169:0/1236310514 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f085006ae40 0x7f085006d300 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:14.377 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:14.181+0000 7f0876165640 1 --2- 172.21.15.169:0/1236310514 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f087013e8d0 0x7f0870158b70 unknown :-1 s=CLOSED pgs=1880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:14.377 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:14.181+0000 7f0876165640 1 --2- 172.21.15.169:0/1236310514 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f08700ce1c0 0x7f08700c7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:14.377 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:14.181+0000 7f0876165640 1 --2- 172.21.15.169:0/1236310514 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f08700b22b0 0x7f087013d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:14.377 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:14.181+0000 7f0876165640 1 -- 172.21.15.169:0/1236310514 >> 172.21.15.169:0/1236310514 conn(0x7f0870069510 msgr2=0x7f087015c690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:14.378 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:14.181+0000 7f0876165640 1 -- 172.21.15.169:0/1236310514 shutdown_connections 2023-10-26T01:44:14.378 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:14.181+0000 7f0876165640 1 -- 172.21.15.169:0/1236310514 wait complete. 2023-10-26T01:44:14.378 DEBUG:tasks.ceph_test_case:wait_until_true: success in 10s and 0 retries 2023-10-26T01:44:14.379 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:44:14.379 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:44:14.439 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:44:14.440 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=50 seek=16) 2023-10-26T01:44:14.882 INFO:teuthology.orchestra.run.smithi169.stderr:dd: fdatasync failed for 'large_file_b': No space left on device 2023-10-26T01:44:14.882 INFO:teuthology.orchestra.run.smithi169.stderr:50+0 records in 2023-10-26T01:44:14.883 INFO:teuthology.orchestra.run.smithi169.stderr:50+0 records out 2023-10-26T01:44:14.883 INFO:teuthology.orchestra.run.smithi169.stderr:52428800 bytes (52 MB, 50 MiB) copied, 0.381357 s, 137 MB/s 2023-10-26T01:44:14.883 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-26T01:44:14.884 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-26T01:44:15.091 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:15.089+0000 7f38af4c2640 1 -- 172.21.15.169:0/2177692662 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f38a80ce1c0 msgr2=0x7f38a806b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:15.092 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:15.089+0000 7f38af4c2640 1 --2- 172.21.15.169:0/2177692662 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f38a80ce1c0 0x7f38a806b590 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f38a0005fd0 tx=0x7f38a002c2e0 comp rx=0 tx=0).stop 2023-10-26T01:44:15.092 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:15.089+0000 7f38af4c2640 1 -- 172.21.15.169:0/2177692662 shutdown_connections 2023-10-26T01:44:15.092 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:15.089+0000 7f38af4c2640 1 --2- 172.21.15.169:0/2177692662 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f38a80ce1c0 0x7f38a806b590 unknown :-1 s=CLOSED pgs=1882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:15.093 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:15.089+0000 7f38af4c2640 1 --2- 172.21.15.169:0/2177692662 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f38a813e8d0 0x7f38a80b28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:15.093 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:15.089+0000 7f38af4c2640 1 -- 172.21.15.169:0/2177692662 >> 172.21.15.169:0/2177692662 conn(0x7f38a8069510 msgr2=0x7f38a80698f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:15.093 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:15.089+0000 7f38af4c2640 1 -- 172.21.15.169:0/2177692662 shutdown_connections 2023-10-26T01:44:15.093 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:15.089+0000 7f38af4c2640 1 -- 172.21.15.169:0/2177692662 wait complete. 2023-10-26T01:44:15.096 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:15.091+0000 7f38af4c2640 1 Processor -- start 2023-10-26T01:44:15.096 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:15.091+0000 7f38af4c2640 1 -- start start 2023-10-26T01:44:15.096 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:15.092+0000 7f38af4c2640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f38a80b22b0 0x7f38a809b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:15.096 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:15.092+0000 7f38af4c2640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f38a80ce1c0 0x7f38a80ae790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:15.097 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:15.092+0000 7f38af4c2640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f38a813e8d0 0x7f38a80aecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:15.097 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:15.092+0000 7f38af4c2640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a80cbc60 con 0x7f38a80ce1c0 2023-10-26T01:44:15.097 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:15.092+0000 7f38af4c2640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a80cbf40 con 0x7f38a80b22b0 2023-10-26T01:44:15.097 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:15.092+0000 7f38af4c2640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38a80cc220 con 0x7f38a813e8d0 2023-10-26T01:44:15.098 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:15.093+0000 7f38aca5d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f38a80ce1c0 0x7f38a80ae790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:15.098 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:15.093+0000 7f38aca5d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f38a80ce1c0 0x7f38a80ae790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:40758/0 (socket says 172.21.15.169:40758) 2023-10-26T01:44:15.098 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:15.093+0000 7f38aca5d640 1 -- 172.21.15.169:0/478958297 learned_addr learned my addr 172.21.15.169:0/478958297 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:44:15.098 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:15.093+0000 7f38ada5f640 1 --2- 172.21.15.169:0/478958297 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f38a813e8d0 0x7f38a80aecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:15.099 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:15.093+0000 7f38ad25e640 1 --2- 172.21.15.169:0/478958297 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f38a80b22b0 0x7f38a809b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:15.099 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:15.093+0000 7f38aca5d640 1 -- 172.21.15.169:0/478958297 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f38a813e8d0 msgr2=0x7f38a80aecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:15.099 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:15.093+0000 7f38aca5d640 1 --2- 172.21.15.169:0/478958297 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f38a813e8d0 0x7f38a80aecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:15.099 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:15.093+0000 7f38aca5d640 1 -- 172.21.15.169:0/478958297 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f38a80b22b0 msgr2=0x7f38a809b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:15.100 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:15.093+0000 7f38aca5d640 1 --2- 172.21.15.169:0/478958297 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f38a80b22b0 0x7f38a809b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:15.100 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:15.093+0000 7f38aca5d640 1 -- 172.21.15.169:0/478958297 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38a0009900 con 0x7f38a80ce1c0 2023-10-26T01:44:15.100 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:15.093+0000 7f38aca5d640 1 --2- 172.21.15.169:0/478958297 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f38a80ce1c0 0x7f38a80ae790 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7f389c00c750 tx=0x7f389c00cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:15.100 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:15.093+0000 7f38a5ffb640 1 -- 172.21.15.169:0/478958297 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f389c005270 con 0x7f38a80ce1c0 2023-10-26T01:44:15.100 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:15.094+0000 7f38a5ffb640 1 -- 172.21.15.169:0/478958297 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f389c0053f0 con 0x7f38a80ce1c0 2023-10-26T01:44:15.101 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:15.094+0000 7f38af4c2640 1 -- 172.21.15.169:0/478958297 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38a80cc500 con 0x7f38a80ce1c0 2023-10-26T01:44:15.101 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:15.094+0000 7f38a5ffb640 1 -- 172.21.15.169:0/478958297 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f389c00a8a0 con 0x7f38a80ce1c0 2023-10-26T01:44:15.101 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:15.094+0000 7f38af4c2640 1 -- 172.21.15.169:0/478958297 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38a809b840 con 0x7f38a80ce1c0 2023-10-26T01:44:15.102 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:15.095+0000 7f38af4c2640 1 -- 172.21.15.169:0/478958297 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38a8098fb0 con 0x7f38a80ce1c0 2023-10-26T01:44:15.103 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:15.096+0000 7f38a5ffb640 1 -- 172.21.15.169:0/478958297 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f389c00aac0 con 0x7f38a80ce1c0 2023-10-26T01:44:15.103 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:15.097+0000 7f38a5ffb640 1 --2- 172.21.15.169:0/478958297 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f389406ae40 0x7f389406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:15.104 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:15.097+0000 7f38a5ffb640 1 -- 172.21.15.169:0/478958297 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f389c088de0 con 0x7f38a80ce1c0 2023-10-26T01:44:15.104 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:15.098+0000 7f38a5ffb640 1 -- 172.21.15.169:0/478958297 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=185}) v3 -- 0x7f3894072970 con 0x7f38a80ce1c0 2023-10-26T01:44:15.104 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:15.098+0000 7f38ad25e640 1 --2- 172.21.15.169:0/478958297 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f389406ae40 0x7f389406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:15.105 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:15.099+0000 7f38ad25e640 1 --2- 172.21.15.169:0/478958297 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f389406ae40 0x7f389406d300 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f38980058c0 tx=0x7f3898005850 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:15.105 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:15.102+0000 7f38a5ffb640 1 -- 172.21.15.169:0/478958297 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f389c017030 con 0x7f38a80ce1c0 2023-10-26T01:44:15.294 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:15.293+0000 7f38af4c2640 1 -- 172.21.15.169:0/478958297 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f38a80b2e00 con 0x7f38a80ce1c0 2023-10-26T01:44:15.298 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:15.296+0000 7f38a5ffb640 1 -- 172.21.15.169:0/478958297 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v184) v1 ==== 81+0+34081 (secure 0 0 0) 0x7f38a80b2e00 con 0x7f38a80ce1c0 2023-10-26T01:44:15.298 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:44:15.299 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2023-10-26T01:44:15.299 INFO:teuthology.orchestra.run.smithi169.stdout: "epoch": 184, 2023-10-26T01:44:15.299 INFO:teuthology.orchestra.run.smithi169.stdout: "fsid": "30203b28-ed8a-44e8-a9a6-a9bf25f980f1", 2023-10-26T01:44:15.299 INFO:teuthology.orchestra.run.smithi169.stdout: "created": "2023-10-26T01:30:22.897155+0000", 2023-10-26T01:44:15.300 INFO:teuthology.orchestra.run.smithi169.stdout: "modified": "2023-10-26T01:44:13.194893+0000", 2023-10-26T01:44:15.300 INFO:teuthology.orchestra.run.smithi169.stdout: "last_up_change": "2023-10-26T01:30:31.875281+0000", 2023-10-26T01:44:15.300 INFO:teuthology.orchestra.run.smithi169.stdout: "last_in_change": "2023-10-26T01:30:27.637941+0000", 2023-10-26T01:44:15.300 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-26T01:44:15.300 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_num": 5799936, 2023-10-26T01:44:15.300 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_set": [ 2023-10-26T01:44:15.301 INFO:teuthology.orchestra.run.smithi169.stdout: "pglog_hardlimit", 2023-10-26T01:44:15.301 INFO:teuthology.orchestra.run.smithi169.stdout: "purged_snapdirs", 2023-10-26T01:44:15.301 INFO:teuthology.orchestra.run.smithi169.stdout: "recovery_deletes", 2023-10-26T01:44:15.301 INFO:teuthology.orchestra.run.smithi169.stdout: "sortbitwise" 2023-10-26T01:44:15.301 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:44:15.301 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_version": 4, 2023-10-26T01:44:15.302 INFO:teuthology.orchestra.run.smithi169.stdout: "full_ratio": 0.69999998807907104, 2023-10-26T01:44:15.302 INFO:teuthology.orchestra.run.smithi169.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-26T01:44:15.302 INFO:teuthology.orchestra.run.smithi169.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-26T01:44:15.302 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_snapshot": "", 2023-10-26T01:44:15.302 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_max": 23, 2023-10-26T01:44:15.303 INFO:teuthology.orchestra.run.smithi169.stdout: "max_osd": 8, 2023-10-26T01:44:15.303 INFO:teuthology.orchestra.run.smithi169.stdout: "require_min_compat_client": "luminous", 2023-10-26T01:44:15.303 INFO:teuthology.orchestra.run.smithi169.stdout: "min_compat_client": "jewel", 2023-10-26T01:44:15.303 INFO:teuthology.orchestra.run.smithi169.stdout: "require_osd_release": "reef", 2023-10-26T01:44:15.303 INFO:teuthology.orchestra.run.smithi169.stdout: "allow_crimson": false, 2023-10-26T01:44:15.304 INFO:teuthology.orchestra.run.smithi169.stdout: "pools": [ 2023-10-26T01:44:15.304 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:15.304 INFO:teuthology.orchestra.run.smithi169.stdout: "pool": 1, 2023-10-26T01:44:15.304 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_name": ".mgr", 2023-10-26T01:44:15.304 INFO:teuthology.orchestra.run.smithi169.stdout: "create_time": "2023-10-26T01:30:32.200808+0000", 2023-10-26T01:44:15.304 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": 1, 2023-10-26T01:44:15.305 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_names": "hashpspool", 2023-10-26T01:44:15.305 INFO:teuthology.orchestra.run.smithi169.stdout: "type": 1, 2023-10-26T01:44:15.305 INFO:teuthology.orchestra.run.smithi169.stdout: "size": 2, 2023-10-26T01:44:15.305 INFO:teuthology.orchestra.run.smithi169.stdout: "min_size": 1, 2023-10-26T01:44:15.305 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_rule": 0, 2023-10-26T01:44:15.306 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_count": 0, 2023-10-26T01:44:15.306 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_target": 0, 2023-10-26T01:44:15.306 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T01:44:15.306 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T01:44:15.306 INFO:teuthology.orchestra.run.smithi169.stdout: "object_hash": 2, 2023-10-26T01:44:15.307 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_mode": "off", 2023-10-26T01:44:15.307 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num": 1, 2023-10-26T01:44:15.307 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num": 1, 2023-10-26T01:44:15.307 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num_target": 1, 2023-10-26T01:44:15.307 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_target": 1, 2023-10-26T01:44:15.308 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_pending": 1, 2023-10-26T01:44:15.308 INFO:teuthology.orchestra.run.smithi169.stdout: "last_pg_merge_meta": { 2023-10-26T01:44:15.308 INFO:teuthology.orchestra.run.smithi169.stdout: "source_pgid": "0.0", 2023-10-26T01:44:15.308 INFO:teuthology.orchestra.run.smithi169.stdout: "ready_epoch": 0, 2023-10-26T01:44:15.308 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_started": 0, 2023-10-26T01:44:15.308 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_clean": 0, 2023-10-26T01:44:15.309 INFO:teuthology.orchestra.run.smithi169.stdout: "source_version": "0'0", 2023-10-26T01:44:15.309 INFO:teuthology.orchestra.run.smithi169.stdout: "target_version": "0'0" 2023-10-26T01:44:15.309 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:15.309 INFO:teuthology.orchestra.run.smithi169.stdout: "last_change": "109", 2023-10-26T01:44:15.309 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend": "0", 2023-10-26T01:44:15.310 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T01:44:15.310 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T01:44:15.310 INFO:teuthology.orchestra.run.smithi169.stdout: "auid": 0, 2023-10-26T01:44:15.310 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_mode": "selfmanaged", 2023-10-26T01:44:15.310 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_seq": 0, 2023-10-26T01:44:15.311 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_epoch": 0, 2023-10-26T01:44:15.311 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_snaps": [], 2023-10-26T01:44:15.311 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps": "[]", 2023-10-26T01:44:15.311 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_bytes": 0, 2023-10-26T01:44:15.311 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_objects": 0, 2023-10-26T01:44:15.311 INFO:teuthology.orchestra.run.smithi169.stdout: "tiers": [], 2023-10-26T01:44:15.312 INFO:teuthology.orchestra.run.smithi169.stdout: "tier_of": -1, 2023-10-26T01:44:15.312 INFO:teuthology.orchestra.run.smithi169.stdout: "read_tier": -1, 2023-10-26T01:44:15.312 INFO:teuthology.orchestra.run.smithi169.stdout: "write_tier": -1, 2023-10-26T01:44:15.312 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_mode": "none", 2023-10-26T01:44:15.312 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_bytes": 0, 2023-10-26T01:44:15.312 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_objects": 0, 2023-10-26T01:44:15.313 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T01:44:15.313 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T01:44:15.313 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T01:44:15.313 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_flush_age": 0, 2023-10-26T01:44:15.313 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_evict_age": 0, 2023-10-26T01:44:15.314 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profile": "", 2023-10-26T01:44:15.314 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_params": { 2023-10-26T01:44:15.314 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "none" 2023-10-26T01:44:15.314 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:15.314 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_period": 0, 2023-10-26T01:44:15.315 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_count": 0, 2023-10-26T01:44:15.315 INFO:teuthology.orchestra.run.smithi169.stdout: "use_gmt_hitset": true, 2023-10-26T01:44:15.315 INFO:teuthology.orchestra.run.smithi169.stdout: "min_read_recency_for_promote": 0, 2023-10-26T01:44:15.315 INFO:teuthology.orchestra.run.smithi169.stdout: "min_write_recency_for_promote": 0, 2023-10-26T01:44:15.315 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T01:44:15.316 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_search_last_n": 0, 2023-10-26T01:44:15.316 INFO:teuthology.orchestra.run.smithi169.stdout: "grade_table": [], 2023-10-26T01:44:15.316 INFO:teuthology.orchestra.run.smithi169.stdout: "stripe_width": 0, 2023-10-26T01:44:15.316 INFO:teuthology.orchestra.run.smithi169.stdout: "expected_num_objects": 0, 2023-10-26T01:44:15.316 INFO:teuthology.orchestra.run.smithi169.stdout: "fast_read": false, 2023-10-26T01:44:15.316 INFO:teuthology.orchestra.run.smithi169.stdout: "options": { 2023-10-26T01:44:15.317 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_max": 32, 2023-10-26T01:44:15.317 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_min": 1 2023-10-26T01:44:15.317 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:15.317 INFO:teuthology.orchestra.run.smithi169.stdout: "application_metadata": { 2023-10-26T01:44:15.317 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": {} 2023-10-26T01:44:15.318 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:15.318 INFO:teuthology.orchestra.run.smithi169.stdout: "read_balance": { 2023-10-26T01:44:15.318 INFO:teuthology.orchestra.run.smithi169.stdout: "score_acting": 8, 2023-10-26T01:44:15.318 INFO:teuthology.orchestra.run.smithi169.stdout: "score_stable": 8, 2023-10-26T01:44:15.318 INFO:teuthology.orchestra.run.smithi169.stdout: "optimal_score": 0.25, 2023-10-26T01:44:15.319 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_acting": 2, 2023-10-26T01:44:15.319 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_stable": 2, 2023-10-26T01:44:15.319 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity_weighted": 1, 2023-10-26T01:44:15.319 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity": 1, 2023-10-26T01:44:15.319 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity_weighted": 1 2023-10-26T01:44:15.320 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:15.320 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:15.320 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:15.320 INFO:teuthology.orchestra.run.smithi169.stdout: "pool": 22, 2023-10-26T01:44:15.320 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_name": "cephfs_metadata", 2023-10-26T01:44:15.320 INFO:teuthology.orchestra.run.smithi169.stdout: "create_time": "2023-10-26T01:43:47.501296+0000", 2023-10-26T01:44:15.321 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": 1, 2023-10-26T01:44:15.321 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_names": "hashpspool", 2023-10-26T01:44:15.321 INFO:teuthology.orchestra.run.smithi169.stdout: "type": 1, 2023-10-26T01:44:15.321 INFO:teuthology.orchestra.run.smithi169.stdout: "size": 2, 2023-10-26T01:44:15.321 INFO:teuthology.orchestra.run.smithi169.stdout: "min_size": 1, 2023-10-26T01:44:15.322 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_rule": 0, 2023-10-26T01:44:15.322 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_count": 0, 2023-10-26T01:44:15.322 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_target": 0, 2023-10-26T01:44:15.322 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T01:44:15.322 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T01:44:15.323 INFO:teuthology.orchestra.run.smithi169.stdout: "object_hash": 2, 2023-10-26T01:44:15.323 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_mode": "off", 2023-10-26T01:44:15.323 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num": 32, 2023-10-26T01:44:15.323 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num": 32, 2023-10-26T01:44:15.323 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num_target": 32, 2023-10-26T01:44:15.324 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_target": 32, 2023-10-26T01:44:15.324 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_pending": 32, 2023-10-26T01:44:15.324 INFO:teuthology.orchestra.run.smithi169.stdout: "last_pg_merge_meta": { 2023-10-26T01:44:15.324 INFO:teuthology.orchestra.run.smithi169.stdout: "source_pgid": "0.0", 2023-10-26T01:44:15.324 INFO:teuthology.orchestra.run.smithi169.stdout: "ready_epoch": 0, 2023-10-26T01:44:15.324 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_started": 0, 2023-10-26T01:44:15.325 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_clean": 0, 2023-10-26T01:44:15.325 INFO:teuthology.orchestra.run.smithi169.stdout: "source_version": "0'0", 2023-10-26T01:44:15.325 INFO:teuthology.orchestra.run.smithi169.stdout: "target_version": "0'0" 2023-10-26T01:44:15.325 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:15.325 INFO:teuthology.orchestra.run.smithi169.stdout: "last_change": "181", 2023-10-26T01:44:15.326 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend": "0", 2023-10-26T01:44:15.326 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T01:44:15.326 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T01:44:15.326 INFO:teuthology.orchestra.run.smithi169.stdout: "auid": 0, 2023-10-26T01:44:15.326 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_mode": "selfmanaged", 2023-10-26T01:44:15.327 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_seq": 0, 2023-10-26T01:44:15.327 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_epoch": 0, 2023-10-26T01:44:15.327 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_snaps": [], 2023-10-26T01:44:15.327 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps": "[]", 2023-10-26T01:44:15.327 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_bytes": 0, 2023-10-26T01:44:15.328 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_objects": 0, 2023-10-26T01:44:15.328 INFO:teuthology.orchestra.run.smithi169.stdout: "tiers": [], 2023-10-26T01:44:15.328 INFO:teuthology.orchestra.run.smithi169.stdout: "tier_of": -1, 2023-10-26T01:44:15.328 INFO:teuthology.orchestra.run.smithi169.stdout: "read_tier": -1, 2023-10-26T01:44:15.328 INFO:teuthology.orchestra.run.smithi169.stdout: "write_tier": -1, 2023-10-26T01:44:15.329 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_mode": "none", 2023-10-26T01:44:15.329 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_bytes": 0, 2023-10-26T01:44:15.329 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_objects": 0, 2023-10-26T01:44:15.329 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T01:44:15.329 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T01:44:15.329 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T01:44:15.330 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_flush_age": 0, 2023-10-26T01:44:15.330 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_evict_age": 0, 2023-10-26T01:44:15.330 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profile": "", 2023-10-26T01:44:15.330 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_params": { 2023-10-26T01:44:15.330 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "none" 2023-10-26T01:44:15.331 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:15.331 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_period": 0, 2023-10-26T01:44:15.331 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_count": 0, 2023-10-26T01:44:15.331 INFO:teuthology.orchestra.run.smithi169.stdout: "use_gmt_hitset": true, 2023-10-26T01:44:15.331 INFO:teuthology.orchestra.run.smithi169.stdout: "min_read_recency_for_promote": 0, 2023-10-26T01:44:15.332 INFO:teuthology.orchestra.run.smithi169.stdout: "min_write_recency_for_promote": 0, 2023-10-26T01:44:15.332 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T01:44:15.332 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_search_last_n": 0, 2023-10-26T01:44:15.332 INFO:teuthology.orchestra.run.smithi169.stdout: "grade_table": [], 2023-10-26T01:44:15.332 INFO:teuthology.orchestra.run.smithi169.stdout: "stripe_width": 0, 2023-10-26T01:44:15.333 INFO:teuthology.orchestra.run.smithi169.stdout: "expected_num_objects": 0, 2023-10-26T01:44:15.333 INFO:teuthology.orchestra.run.smithi169.stdout: "fast_read": false, 2023-10-26T01:44:15.333 INFO:teuthology.orchestra.run.smithi169.stdout: "options": { 2023-10-26T01:44:15.333 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_bias": 4, 2023-10-26T01:44:15.333 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_min": 16, 2023-10-26T01:44:15.333 INFO:teuthology.orchestra.run.smithi169.stdout: "recovery_priority": 5 2023-10-26T01:44:15.334 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:15.334 INFO:teuthology.orchestra.run.smithi169.stdout: "application_metadata": { 2023-10-26T01:44:15.334 INFO:teuthology.orchestra.run.smithi169.stdout: "cephfs": { 2023-10-26T01:44:15.334 INFO:teuthology.orchestra.run.smithi169.stdout: "metadata": "cephfs" 2023-10-26T01:44:15.334 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:15.335 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:15.335 INFO:teuthology.orchestra.run.smithi169.stdout: "read_balance": { 2023-10-26T01:44:15.335 INFO:teuthology.orchestra.run.smithi169.stdout: "score_acting": 2, 2023-10-26T01:44:15.335 INFO:teuthology.orchestra.run.smithi169.stdout: "score_stable": 2, 2023-10-26T01:44:15.335 INFO:teuthology.orchestra.run.smithi169.stdout: "optimal_score": 1, 2023-10-26T01:44:15.336 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_acting": 2, 2023-10-26T01:44:15.336 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_stable": 2, 2023-10-26T01:44:15.336 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity_weighted": 1, 2023-10-26T01:44:15.336 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity": 1, 2023-10-26T01:44:15.336 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity_weighted": 1 2023-10-26T01:44:15.337 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:15.337 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:15.337 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:15.337 INFO:teuthology.orchestra.run.smithi169.stdout: "pool": 23, 2023-10-26T01:44:15.337 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_name": "cephfs_data", 2023-10-26T01:44:15.338 INFO:teuthology.orchestra.run.smithi169.stdout: "create_time": "2023-10-26T01:43:48.737507+0000", 2023-10-26T01:44:15.338 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": 1027, 2023-10-26T01:44:15.338 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_names": "hashpspool,full,full_quota", 2023-10-26T01:44:15.338 INFO:teuthology.orchestra.run.smithi169.stdout: "type": 1, 2023-10-26T01:44:15.338 INFO:teuthology.orchestra.run.smithi169.stdout: "size": 2, 2023-10-26T01:44:15.339 INFO:teuthology.orchestra.run.smithi169.stdout: "min_size": 1, 2023-10-26T01:44:15.339 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_rule": 0, 2023-10-26T01:44:15.339 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_count": 0, 2023-10-26T01:44:15.339 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_target": 0, 2023-10-26T01:44:15.339 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T01:44:15.339 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T01:44:15.340 INFO:teuthology.orchestra.run.smithi169.stdout: "object_hash": 2, 2023-10-26T01:44:15.340 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_mode": "off", 2023-10-26T01:44:15.340 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num": 64, 2023-10-26T01:44:15.340 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num": 64, 2023-10-26T01:44:15.340 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num_target": 64, 2023-10-26T01:44:15.341 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_target": 64, 2023-10-26T01:44:15.341 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_pending": 64, 2023-10-26T01:44:15.341 INFO:teuthology.orchestra.run.smithi169.stdout: "last_pg_merge_meta": { 2023-10-26T01:44:15.341 INFO:teuthology.orchestra.run.smithi169.stdout: "source_pgid": "0.0", 2023-10-26T01:44:15.341 INFO:teuthology.orchestra.run.smithi169.stdout: "ready_epoch": 0, 2023-10-26T01:44:15.341 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_started": 0, 2023-10-26T01:44:15.342 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_clean": 0, 2023-10-26T01:44:15.342 INFO:teuthology.orchestra.run.smithi169.stdout: "source_version": "0'0", 2023-10-26T01:44:15.342 INFO:teuthology.orchestra.run.smithi169.stdout: "target_version": "0'0" 2023-10-26T01:44:15.342 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:15.342 INFO:teuthology.orchestra.run.smithi169.stdout: "last_change": "184", 2023-10-26T01:44:15.343 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend": "0", 2023-10-26T01:44:15.343 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T01:44:15.343 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T01:44:15.343 INFO:teuthology.orchestra.run.smithi169.stdout: "auid": 0, 2023-10-26T01:44:15.343 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_mode": "selfmanaged", 2023-10-26T01:44:15.344 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_seq": 0, 2023-10-26T01:44:15.344 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_epoch": 0, 2023-10-26T01:44:15.344 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_snaps": [], 2023-10-26T01:44:15.344 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps": "[]", 2023-10-26T01:44:15.344 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_bytes": 33554432, 2023-10-26T01:44:15.344 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_objects": 0, 2023-10-26T01:44:15.345 INFO:teuthology.orchestra.run.smithi169.stdout: "tiers": [], 2023-10-26T01:44:15.345 INFO:teuthology.orchestra.run.smithi169.stdout: "tier_of": -1, 2023-10-26T01:44:15.345 INFO:teuthology.orchestra.run.smithi169.stdout: "read_tier": -1, 2023-10-26T01:44:15.345 INFO:teuthology.orchestra.run.smithi169.stdout: "write_tier": -1, 2023-10-26T01:44:15.345 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_mode": "none", 2023-10-26T01:44:15.346 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_bytes": 0, 2023-10-26T01:44:15.346 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_objects": 0, 2023-10-26T01:44:15.346 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T01:44:15.346 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T01:44:15.346 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T01:44:15.347 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_flush_age": 0, 2023-10-26T01:44:15.347 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_evict_age": 0, 2023-10-26T01:44:15.347 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profile": "", 2023-10-26T01:44:15.347 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_params": { 2023-10-26T01:44:15.347 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "none" 2023-10-26T01:44:15.348 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:15.348 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_period": 0, 2023-10-26T01:44:15.348 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_count": 0, 2023-10-26T01:44:15.348 INFO:teuthology.orchestra.run.smithi169.stdout: "use_gmt_hitset": true, 2023-10-26T01:44:15.348 INFO:teuthology.orchestra.run.smithi169.stdout: "min_read_recency_for_promote": 0, 2023-10-26T01:44:15.348 INFO:teuthology.orchestra.run.smithi169.stdout: "min_write_recency_for_promote": 0, 2023-10-26T01:44:15.349 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T01:44:15.349 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_search_last_n": 0, 2023-10-26T01:44:15.349 INFO:teuthology.orchestra.run.smithi169.stdout: "grade_table": [], 2023-10-26T01:44:15.349 INFO:teuthology.orchestra.run.smithi169.stdout: "stripe_width": 0, 2023-10-26T01:44:15.349 INFO:teuthology.orchestra.run.smithi169.stdout: "expected_num_objects": 0, 2023-10-26T01:44:15.350 INFO:teuthology.orchestra.run.smithi169.stdout: "fast_read": false, 2023-10-26T01:44:15.350 INFO:teuthology.orchestra.run.smithi169.stdout: "options": { 2023-10-26T01:44:15.350 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_min": 64, 2023-10-26T01:44:15.350 INFO:teuthology.orchestra.run.smithi169.stdout: "target_size_ratio": 0.89999997615814209 2023-10-26T01:44:15.350 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:15.351 INFO:teuthology.orchestra.run.smithi169.stdout: "application_metadata": { 2023-10-26T01:44:15.351 INFO:teuthology.orchestra.run.smithi169.stdout: "cephfs": { 2023-10-26T01:44:15.351 INFO:teuthology.orchestra.run.smithi169.stdout: "data": "cephfs" 2023-10-26T01:44:15.351 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:15.351 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:15.352 INFO:teuthology.orchestra.run.smithi169.stdout: "read_balance": { 2023-10-26T01:44:15.352 INFO:teuthology.orchestra.run.smithi169.stdout: "score_acting": 1.3799999952316284, 2023-10-26T01:44:15.352 INFO:teuthology.orchestra.run.smithi169.stdout: "score_stable": 1.3799999952316284, 2023-10-26T01:44:15.352 INFO:teuthology.orchestra.run.smithi169.stdout: "optimal_score": 1, 2023-10-26T01:44:15.352 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_acting": 1.3799999952316284, 2023-10-26T01:44:15.353 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_stable": 1.3799999952316284, 2023-10-26T01:44:15.353 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity_weighted": 1, 2023-10-26T01:44:15.353 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity": 1, 2023-10-26T01:44:15.353 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity_weighted": 1 2023-10-26T01:44:15.353 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:15.353 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:15.354 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:44:15.354 INFO:teuthology.orchestra.run.smithi169.stdout: "osds": [ 2023-10-26T01:44:15.354 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:15.354 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 0, 2023-10-26T01:44:15.354 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:44:15.355 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:44:15.355 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:44:15.355 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:44:15.355 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:44:15.355 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:44:15.355 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:44:15.356 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:44:15.356 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 180, 2023-10-26T01:44:15.356 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:44:15.356 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:44:15.356 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:44:15.357 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:15.357 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:15.357 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:15.357 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6816", 2023-10-26T01:44:15.357 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:44:15.357 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:15.358 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:15.358 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:15.358 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6817", 2023-10-26T01:44:15.358 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:44:15.358 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:15.359 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:15.359 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:15.359 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:44:15.359 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:15.359 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:15.360 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:15.360 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6818", 2023-10-26T01:44:15.360 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:44:15.360 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:15.360 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:15.360 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:15.361 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6819", 2023-10-26T01:44:15.361 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:44:15.361 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:15.361 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:15.361 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:15.362 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:44:15.362 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:15.362 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:15.362 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:15.362 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6822", 2023-10-26T01:44:15.363 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:44:15.363 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:15.363 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:15.363 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:15.363 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6823", 2023-10-26T01:44:15.363 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:44:15.364 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:15.364 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:15.364 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:15.364 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:44:15.364 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:15.365 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:15.365 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:15.365 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6820", 2023-10-26T01:44:15.365 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:44:15.365 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:15.365 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:15.366 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:15.366 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6821", 2023-10-26T01:44:15.366 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:44:15.366 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:15.366 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:15.367 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:15.367 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6817/4204255864", 2023-10-26T01:44:15.367 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6819/4204255864", 2023-10-26T01:44:15.367 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6823/4204255864", 2023-10-26T01:44:15.367 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6821/4204255864", 2023-10-26T01:44:15.368 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:44:15.368 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:44:15.368 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:44:15.368 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:15.368 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:15.369 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:15.369 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 1, 2023-10-26T01:44:15.369 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:44:15.369 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:44:15.369 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:44:15.370 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:44:15.370 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:44:15.370 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:44:15.370 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:44:15.370 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:44:15.371 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 180, 2023-10-26T01:44:15.371 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:44:15.371 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:44:15.371 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:44:15.371 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:15.372 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:15.372 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:15.372 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6808", 2023-10-26T01:44:15.372 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:44:15.372 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:15.372 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:15.373 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:15.373 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6809", 2023-10-26T01:44:15.373 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:44:15.373 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:15.373 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:15.374 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:15.374 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:44:15.374 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:15.374 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:15.374 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:15.375 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6810", 2023-10-26T01:44:15.375 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:44:15.375 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:15.375 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:15.375 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:15.376 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6811", 2023-10-26T01:44:15.376 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:44:15.376 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:15.376 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:15.376 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:15.377 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:44:15.377 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:15.377 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:15.377 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:15.377 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6814", 2023-10-26T01:44:15.377 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:44:15.378 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:15.378 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:15.378 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:15.378 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6815", 2023-10-26T01:44:15.379 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:44:15.379 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:15.379 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:15.379 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:15.379 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:44:15.379 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:15.380 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:15.380 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:15.380 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6812", 2023-10-26T01:44:15.380 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:44:15.380 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:15.381 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:15.381 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:15.381 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6813", 2023-10-26T01:44:15.381 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:44:15.381 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:15.382 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:15.382 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:15.382 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6809/238206046", 2023-10-26T01:44:15.382 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6811/238206046", 2023-10-26T01:44:15.382 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6815/238206046", 2023-10-26T01:44:15.383 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6813/238206046", 2023-10-26T01:44:15.383 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:44:15.383 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:44:15.383 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:44:15.383 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:15.384 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:15.384 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:15.384 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 2, 2023-10-26T01:44:15.384 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:44:15.384 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:44:15.384 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:44:15.385 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:44:15.385 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:44:15.385 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:44:15.385 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:44:15.385 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:44:15.386 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 180, 2023-10-26T01:44:15.386 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:44:15.386 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:44:15.386 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:44:15.386 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:15.387 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:15.387 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:15.387 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6800", 2023-10-26T01:44:15.387 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:44:15.387 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:15.387 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:15.388 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:15.388 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6801", 2023-10-26T01:44:15.388 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:44:15.388 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:15.388 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:15.389 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:15.389 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:44:15.389 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:15.389 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:15.389 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:15.390 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6802", 2023-10-26T01:44:15.390 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:44:15.390 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:15.390 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:15.390 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:15.391 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6803", 2023-10-26T01:44:15.391 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:44:15.391 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:15.391 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:15.391 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:15.392 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:44:15.392 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:15.392 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:15.392 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:15.392 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6806", 2023-10-26T01:44:15.393 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:44:15.393 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:15.393 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:15.393 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:15.393 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6807", 2023-10-26T01:44:15.394 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:44:15.394 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:15.394 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:15.394 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:15.394 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:44:15.395 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:15.395 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:15.395 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:15.395 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6804", 2023-10-26T01:44:15.395 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:44:15.396 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:15.396 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:15.396 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:15.396 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6805", 2023-10-26T01:44:15.396 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:44:15.396 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:15.397 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:15.397 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:15.397 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6801/3215791846", 2023-10-26T01:44:15.397 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6803/3215791846", 2023-10-26T01:44:15.397 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6807/3215791846", 2023-10-26T01:44:15.398 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6805/3215791846", 2023-10-26T01:44:15.398 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:44:15.398 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:44:15.398 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:44:15.398 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:15.399 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:15.399 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:15.399 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 3, 2023-10-26T01:44:15.399 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:44:15.399 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:44:15.400 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:44:15.400 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:44:15.400 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:44:15.400 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:44:15.400 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:44:15.400 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 14, 2023-10-26T01:44:15.401 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 180, 2023-10-26T01:44:15.401 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:44:15.401 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:44:15.401 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:44:15.401 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:15.402 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:15.402 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:15.402 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6824", 2023-10-26T01:44:15.402 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:44:15.402 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:15.403 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:15.403 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:15.403 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6825", 2023-10-26T01:44:15.403 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:44:15.403 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:15.403 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:15.404 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:15.404 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:44:15.404 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:15.404 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:15.404 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:15.405 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6826", 2023-10-26T01:44:15.405 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:44:15.405 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:15.405 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:15.405 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:15.406 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6827", 2023-10-26T01:44:15.406 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:44:15.406 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:15.406 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:15.406 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:15.407 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:44:15.407 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:15.407 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:15.407 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:15.407 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6830", 2023-10-26T01:44:15.407 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:44:15.408 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:15.408 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:15.408 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:15.408 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6831", 2023-10-26T01:44:15.408 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:44:15.409 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:15.409 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:15.409 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:15.409 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:44:15.409 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:15.410 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:15.410 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:15.410 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6828", 2023-10-26T01:44:15.410 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:44:15.410 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:15.411 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:15.411 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:15.411 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6829", 2023-10-26T01:44:15.411 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:44:15.411 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:15.411 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:15.412 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:15.412 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6825/2256701226", 2023-10-26T01:44:15.412 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6827/2256701226", 2023-10-26T01:44:15.412 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6831/2256701226", 2023-10-26T01:44:15.412 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6829/2256701226", 2023-10-26T01:44:15.413 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:44:15.413 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:44:15.413 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:44:15.413 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:15.413 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:15.413 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:15.414 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 4, 2023-10-26T01:44:15.414 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:44:15.414 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:44:15.414 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:44:15.414 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:44:15.415 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:44:15.415 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:44:15.415 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:44:15.415 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:44:15.415 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 180, 2023-10-26T01:44:15.416 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:44:15.416 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:44:15.416 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:44:15.416 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:15.416 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:15.416 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:15.417 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6816", 2023-10-26T01:44:15.417 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:44:15.417 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:15.417 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:15.417 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:15.418 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6817", 2023-10-26T01:44:15.418 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:44:15.418 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:15.418 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:15.418 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:15.419 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:44:15.419 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:15.419 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:15.419 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:15.419 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6818", 2023-10-26T01:44:15.419 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:44:15.420 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:15.420 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:15.420 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:15.420 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6819", 2023-10-26T01:44:15.420 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:44:15.421 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:15.421 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:15.421 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:15.421 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:44:15.421 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:15.422 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:15.422 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:15.422 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6825", 2023-10-26T01:44:15.422 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:44:15.422 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:15.423 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:15.423 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:15.423 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6828", 2023-10-26T01:44:15.423 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:44:15.423 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:15.423 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:15.424 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:15.424 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:44:15.424 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:15.424 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:15.424 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:15.425 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6820", 2023-10-26T01:44:15.425 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:44:15.425 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:15.425 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:15.425 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:15.426 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6823", 2023-10-26T01:44:15.426 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:44:15.426 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:15.426 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:15.426 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:15.427 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6817/1082279874", 2023-10-26T01:44:15.427 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6819/1082279874", 2023-10-26T01:44:15.427 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6828/1082279874", 2023-10-26T01:44:15.427 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6823/1082279874", 2023-10-26T01:44:15.427 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:44:15.427 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:44:15.428 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:44:15.428 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:15.428 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:15.428 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:15.428 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 5, 2023-10-26T01:44:15.429 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:44:15.429 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:44:15.429 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:44:15.429 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:44:15.429 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:44:15.429 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:44:15.430 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:44:15.430 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:44:15.430 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 180, 2023-10-26T01:44:15.430 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:44:15.430 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:44:15.431 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:44:15.431 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:15.431 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:15.431 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:15.431 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6808", 2023-10-26T01:44:15.432 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:44:15.432 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:15.432 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:15.432 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:15.432 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6809", 2023-10-26T01:44:15.433 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:44:15.433 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:15.433 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:15.433 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:15.433 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:44:15.433 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:15.434 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:15.434 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:15.434 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6810", 2023-10-26T01:44:15.434 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:44:15.434 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:15.435 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:15.435 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:15.435 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6811", 2023-10-26T01:44:15.435 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:44:15.435 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:15.436 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:15.436 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:15.436 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:44:15.436 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:15.436 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:15.437 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:15.437 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6814", 2023-10-26T01:44:15.437 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:44:15.437 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:15.437 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:15.438 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:15.438 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6815", 2023-10-26T01:44:15.438 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:44:15.438 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:15.438 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:15.439 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:15.439 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:44:15.439 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:15.439 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:15.439 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:15.439 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6812", 2023-10-26T01:44:15.440 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:44:15.440 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:15.440 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:15.440 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:15.440 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6813", 2023-10-26T01:44:15.441 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:44:15.441 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:15.441 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:15.441 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:15.441 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6809/3335654351", 2023-10-26T01:44:15.442 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6811/3335654351", 2023-10-26T01:44:15.442 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6815/3335654351", 2023-10-26T01:44:15.442 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6813/3335654351", 2023-10-26T01:44:15.442 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:44:15.442 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:44:15.443 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:44:15.443 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:15.443 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:15.443 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:15.443 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 6, 2023-10-26T01:44:15.443 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:44:15.444 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:44:15.444 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:44:15.444 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:44:15.444 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:44:15.444 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:44:15.445 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:44:15.445 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:44:15.445 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 180, 2023-10-26T01:44:15.445 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:44:15.445 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:44:15.445 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:44:15.446 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:15.446 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:15.446 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:15.446 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6800", 2023-10-26T01:44:15.446 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:44:15.447 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:15.447 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:15.447 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:15.447 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6801", 2023-10-26T01:44:15.447 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:44:15.448 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:15.448 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:15.448 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:15.448 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:44:15.448 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:15.449 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:15.449 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:15.449 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6802", 2023-10-26T01:44:15.449 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:44:15.449 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:15.449 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:15.450 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:15.450 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6803", 2023-10-26T01:44:15.450 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:44:15.450 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:15.450 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:15.451 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:15.451 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:44:15.451 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:15.451 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:15.451 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:15.452 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6806", 2023-10-26T01:44:15.452 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:44:15.452 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:15.452 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:15.452 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:15.452 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6807", 2023-10-26T01:44:15.453 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:44:15.453 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:15.453 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:15.453 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:15.453 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:44:15.454 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:15.454 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:15.454 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:15.454 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6804", 2023-10-26T01:44:15.454 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:44:15.455 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:15.455 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:15.455 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:15.455 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6805", 2023-10-26T01:44:15.455 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:44:15.455 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:15.456 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:15.456 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:15.456 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6801/2520460697", 2023-10-26T01:44:15.456 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6803/2520460697", 2023-10-26T01:44:15.456 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6807/2520460697", 2023-10-26T01:44:15.457 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6805/2520460697", 2023-10-26T01:44:15.457 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:44:15.457 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:44:15.457 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:44:15.457 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:15.457 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:15.458 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:15.458 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 7, 2023-10-26T01:44:15.458 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:44:15.458 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:44:15.458 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:44:15.459 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:44:15.459 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:44:15.459 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:44:15.459 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:44:15.459 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:44:15.460 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 180, 2023-10-26T01:44:15.460 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:44:15.460 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:44:15.460 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:44:15.460 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:15.460 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:15.461 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:15.461 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6821", 2023-10-26T01:44:15.461 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:44:15.461 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:15.461 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:15.462 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:15.462 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6822", 2023-10-26T01:44:15.462 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:44:15.462 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:15.462 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:15.463 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:15.463 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:44:15.463 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:15.463 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:15.463 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:15.464 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6824", 2023-10-26T01:44:15.464 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:44:15.464 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:15.464 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:15.464 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:15.464 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6826", 2023-10-26T01:44:15.465 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:44:15.465 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:15.465 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:15.465 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:15.465 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:44:15.466 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:15.466 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:15.466 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:15.466 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6830", 2023-10-26T01:44:15.466 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:44:15.467 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:15.467 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:15.467 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:15.467 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6831", 2023-10-26T01:44:15.467 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:44:15.468 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:15.468 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:15.468 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:15.468 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:44:15.468 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:15.469 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:15.469 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:15.469 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6827", 2023-10-26T01:44:15.469 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:44:15.469 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:15.469 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:15.470 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:15.470 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6829", 2023-10-26T01:44:15.470 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:44:15.470 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:15.470 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:15.471 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:15.471 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6822/1115930619", 2023-10-26T01:44:15.471 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6826/1115930619", 2023-10-26T01:44:15.471 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6831/1115930619", 2023-10-26T01:44:15.471 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6829/1115930619", 2023-10-26T01:44:15.472 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:44:15.472 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:44:15.472 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:44:15.472 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:15.472 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:15.473 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:44:15.473 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_xinfo": [ 2023-10-26T01:44:15.473 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:15.473 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 0, 2023-10-26T01:44:15.473 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:44:15.473 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:44:15.474 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:44:15.474 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:44:15.474 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:44:15.474 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.889675+0000", 2023-10-26T01:44:15.475 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:44:15.475 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:15.475 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:15.475 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 1, 2023-10-26T01:44:15.475 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:44:15.475 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:44:15.476 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:44:15.476 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:44:15.476 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:44:15.476 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.864157+0000", 2023-10-26T01:44:15.476 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:44:15.477 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:15.477 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:15.477 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 2, 2023-10-26T01:44:15.477 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:44:15.477 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:44:15.478 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:44:15.478 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:44:15.478 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:44:15.478 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.876919+0000", 2023-10-26T01:44:15.478 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:44:15.479 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:15.479 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:15.479 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 3, 2023-10-26T01:44:15.479 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:44:15.479 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:44:15.480 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:44:15.480 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:44:15.480 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:44:15.480 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.974541+0000", 2023-10-26T01:44:15.480 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:44:15.481 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:15.481 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:15.481 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 4, 2023-10-26T01:44:15.481 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:44:15.481 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:44:15.482 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:44:15.482 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:44:15.482 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:44:15.482 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.969696+0000", 2023-10-26T01:44:15.482 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:44:15.483 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:15.483 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:15.483 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 5, 2023-10-26T01:44:15.483 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:44:15.483 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:44:15.483 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:44:15.484 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:44:15.484 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:44:15.484 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.859703+0000", 2023-10-26T01:44:15.484 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:44:15.484 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:15.485 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:15.485 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 6, 2023-10-26T01:44:15.485 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:44:15.485 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:44:15.485 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:44:15.486 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:44:15.486 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:44:15.486 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.867917+0000", 2023-10-26T01:44:15.486 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:44:15.486 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:15.487 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:15.487 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 7, 2023-10-26T01:44:15.487 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:44:15.487 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:44:15.487 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:44:15.488 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:44:15.488 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:44:15.488 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.898605+0000", 2023-10-26T01:44:15.488 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:44:15.488 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:15.488 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:44:15.489 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_upmap": [], 2023-10-26T01:44:15.489 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_upmap_items": [], 2023-10-26T01:44:15.489 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_upmap_primaries": [], 2023-10-26T01:44:15.489 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_temp": [], 2023-10-26T01:44:15.489 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_temp": [], 2023-10-26T01:44:15.490 INFO:teuthology.orchestra.run.smithi169.stdout: "blocklist": {}, 2023-10-26T01:44:15.490 INFO:teuthology.orchestra.run.smithi169.stdout: "range_blocklist": {}, 2023-10-26T01:44:15.490 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profiles": { 2023-10-26T01:44:15.490 INFO:teuthology.orchestra.run.smithi169.stdout: "default": { 2023-10-26T01:44:15.490 INFO:teuthology.orchestra.run.smithi169.stdout: "crush-failure-domain": "osd", 2023-10-26T01:44:15.491 INFO:teuthology.orchestra.run.smithi169.stdout: "k": "2", 2023-10-26T01:44:15.491 INFO:teuthology.orchestra.run.smithi169.stdout: "m": "1", 2023-10-26T01:44:15.491 INFO:teuthology.orchestra.run.smithi169.stdout: "plugin": "jerasure", 2023-10-26T01:44:15.491 INFO:teuthology.orchestra.run.smithi169.stdout: "technique": "reed_sol_van" 2023-10-26T01:44:15.491 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:15.491 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:15.492 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps_queue": [], 2023-10-26T01:44:15.492 INFO:teuthology.orchestra.run.smithi169.stdout: "new_removed_snaps": [], 2023-10-26T01:44:15.492 INFO:teuthology.orchestra.run.smithi169.stdout: "new_purged_snaps": [], 2023-10-26T01:44:15.492 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_node_flags": {}, 2023-10-26T01:44:15.492 INFO:teuthology.orchestra.run.smithi169.stdout: "device_class_flags": {}, 2023-10-26T01:44:15.492 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_mode": { 2023-10-26T01:44:15.493 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_mode_enabled": false, 2023-10-26T01:44:15.493 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_bucket_count": 0, 2023-10-26T01:44:15.493 INFO:teuthology.orchestra.run.smithi169.stdout: "degraded_stretch_mode": 0, 2023-10-26T01:44:15.493 INFO:teuthology.orchestra.run.smithi169.stdout: "recovering_stretch_mode": 0, 2023-10-26T01:44:15.493 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_mode_bucket": 0 2023-10-26T01:44:15.493 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:15.494 INFO:teuthology.orchestra.run.smithi169.stdout:} 2023-10-26T01:44:15.495 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:15.298+0000 7f38af4c2640 1 -- 172.21.15.169:0/478958297 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f389406ae40 msgr2=0x7f389406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:15.495 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:15.298+0000 7f38af4c2640 1 --2- 172.21.15.169:0/478958297 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f389406ae40 0x7f389406d300 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f38980058c0 tx=0x7f3898005850 comp rx=0 tx=0).stop 2023-10-26T01:44:15.496 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:15.299+0000 7f38af4c2640 1 -- 172.21.15.169:0/478958297 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f38a80ce1c0 msgr2=0x7f38a80ae790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:15.496 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:15.299+0000 7f38af4c2640 1 --2- 172.21.15.169:0/478958297 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f38a80ce1c0 0x7f38a80ae790 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7f389c00c750 tx=0x7f389c00cc10 comp rx=0 tx=0).stop 2023-10-26T01:44:15.496 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:15.299+0000 7f38af4c2640 1 -- 172.21.15.169:0/478958297 shutdown_connections 2023-10-26T01:44:15.496 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:15.299+0000 7f38af4c2640 1 --2- 172.21.15.169:0/478958297 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f389406ae40 0x7f389406d300 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:15.496 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:15.299+0000 7f38af4c2640 1 --2- 172.21.15.169:0/478958297 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f38a80ce1c0 0x7f38a80ae790 unknown :-1 s=CLOSED pgs=1883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:15.497 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:15.299+0000 7f38af4c2640 1 --2- 172.21.15.169:0/478958297 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f38a813e8d0 0x7f38a80aecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:15.497 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:15.299+0000 7f38af4c2640 1 --2- 172.21.15.169:0/478958297 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f38a80b22b0 0x7f38a809b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:15.497 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:15.299+0000 7f38af4c2640 1 -- 172.21.15.169:0/478958297 >> 172.21.15.169:0/478958297 conn(0x7f38a8069510 msgr2=0x7f38a813da30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:15.497 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:15.299+0000 7f38af4c2640 1 -- 172.21.15.169:0/478958297 shutdown_connections 2023-10-26T01:44:15.497 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:15.299+0000 7f38af4c2640 1 -- 172.21.15.169:0/478958297 wait complete. 2023-10-26T01:44:15.498 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T01:44:15.724 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:15.721+0000 7fcd60bc0640 1 -- 172.21.15.169:0/3584733234 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd5c1592d0 msgr2=0x7fcd5c0aa3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:15.724 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:15.721+0000 7fcd60bc0640 1 --2- 172.21.15.169:0/3584733234 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd5c1592d0 0x7fcd5c0aa3d0 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c0b2650 tx=0x7fcd4c02bce0 comp rx=0 tx=0).stop 2023-10-26T01:44:15.724 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:15.721+0000 7fcd60bc0640 1 -- 172.21.15.169:0/3584733234 shutdown_connections 2023-10-26T01:44:15.724 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:15.721+0000 7fcd60bc0640 1 --2- 172.21.15.169:0/3584733234 >> v2:172.21.15.170:3300/0 conn(0x7fcd5c0ac930 0x7fcd5c0cd6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:15.725 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:15.721+0000 7fcd60bc0640 1 --2- 172.21.15.169:0/3584733234 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd5c1592d0 0x7fcd5c0aa3d0 unknown :-1 s=CLOSED pgs=1885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:15.725 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:15.721+0000 7fcd60bc0640 1 -- 172.21.15.169:0/3584733234 >> 172.21.15.169:0/3584733234 conn(0x7fcd5c069930 msgr2=0x7fcd5c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:15.725 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:15.721+0000 7fcd60bc0640 1 -- 172.21.15.169:0/3584733234 shutdown_connections 2023-10-26T01:44:15.725 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:15.721+0000 7fcd60bc0640 1 -- 172.21.15.169:0/3584733234 wait complete. 2023-10-26T01:44:15.726 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:15.723+0000 7fcd60bc0640 1 Processor -- start 2023-10-26T01:44:15.726 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:15.723+0000 7fcd60bc0640 1 -- start start 2023-10-26T01:44:15.728 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:15.724+0000 7fcd60bc0640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd5c0ac930 0x7fcd5c154840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:15.728 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:15.724+0000 7fcd60bc0640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcd5c0cdc30 0x7fcd5c159a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:15.728 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:15.724+0000 7fcd60bc0640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcd5c1592d0 0x7fcd5c0c7b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:15.728 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:15.724+0000 7fcd60bc0640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd5c15c330 con 0x7fcd5c0ac930 2023-10-26T01:44:15.729 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:15.724+0000 7fcd60bc0640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd5c154d80 con 0x7fcd5c1592d0 2023-10-26T01:44:15.729 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:15.724+0000 7fcd60bc0640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd5c15b760 con 0x7fcd5c0cdc30 2023-10-26T01:44:15.729 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:15.724+0000 7fcd5a59c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd5c0ac930 0x7fcd5c154840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:15.729 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:15.724+0000 7fcd5a59c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd5c0ac930 0x7fcd5c154840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:40776/0 (socket says 172.21.15.169:40776) 2023-10-26T01:44:15.729 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:15.724+0000 7fcd5a59c640 1 -- 172.21.15.169:0/4183630790 learned_addr learned my addr 172.21.15.169:0/4183630790 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:44:15.729 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:15.724+0000 7fcd5a59c640 1 -- 172.21.15.169:0/4183630790 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcd5c0cdc30 msgr2=0x7fcd5c159a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:15.730 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:15.724+0000 7fcd5a59c640 1 --2- 172.21.15.169:0/4183630790 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcd5c0cdc30 0x7fcd5c159a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:15.730 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:15.724+0000 7fcd5a59c640 1 -- 172.21.15.169:0/4183630790 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcd5c1592d0 msgr2=0x7fcd5c0c7b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:15.730 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:15.724+0000 7fcd5a59c640 1 --2- 172.21.15.169:0/4183630790 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcd5c1592d0 0x7fcd5c0c7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:15.730 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:15.724+0000 7fcd5a59c640 1 -- 172.21.15.169:0/4183630790 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd4c009490 con 0x7fcd5c0ac930 2023-10-26T01:44:15.730 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:15.725+0000 7fcd5a59c640 1 --2- 172.21.15.169:0/4183630790 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd5c0ac930 0x7fcd5c154840 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c009920 tx=0x7fcd4c0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:15.731 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:15.725+0000 7fcd52ffd640 1 -- 172.21.15.169:0/4183630790 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd4c003ea0 con 0x7fcd5c0ac930 2023-10-26T01:44:15.731 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:15.725+0000 7fcd52ffd640 1 -- 172.21.15.169:0/4183630790 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fcd4c034440 con 0x7fcd5c0ac930 2023-10-26T01:44:15.731 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:15.725+0000 7fcd52ffd640 1 -- 172.21.15.169:0/4183630790 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd4c034850 con 0x7fcd5c0ac930 2023-10-26T01:44:15.731 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:15.725+0000 7fcd60bc0640 1 -- 172.21.15.169:0/4183630790 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd5c15ba00 con 0x7fcd5c0ac930 2023-10-26T01:44:15.731 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:15.725+0000 7fcd60bc0640 1 -- 172.21.15.169:0/4183630790 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd5c15bfc0 con 0x7fcd5c0ac930 2023-10-26T01:44:15.732 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:15.727+0000 7fcd60bc0640 1 -- 172.21.15.169:0/4183630790 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd5c15c150 con 0x7fcd5c0ac930 2023-10-26T01:44:15.733 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:15.727+0000 7fcd52ffd640 1 -- 172.21.15.169:0/4183630790 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fcd4c03b040 con 0x7fcd5c0ac930 2023-10-26T01:44:15.733 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:15.728+0000 7fcd52ffd640 1 --2- 172.21.15.169:0/4183630790 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcd3c06ae40 0x7fcd3c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:15.733 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:15.729+0000 7fcd52ffd640 1 -- 172.21.15.169:0/4183630790 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcd4c0abb20 con 0x7fcd5c0ac930 2023-10-26T01:44:15.733 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:15.729+0000 7fcd52ffd640 1 -- 172.21.15.169:0/4183630790 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=185}) v3 -- 0x7fcd3c072970 con 0x7fcd5c0ac930 2023-10-26T01:44:15.733 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:15.729+0000 7fcd53fff640 1 --2- 172.21.15.169:0/4183630790 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcd3c06ae40 0x7fcd3c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:15.734 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:15.730+0000 7fcd53fff640 1 --2- 172.21.15.169:0/4183630790 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcd3c06ae40 0x7fcd3c06d300 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fcd44005fd0 tx=0x7fcd44005850 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:15.735 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:15.733+0000 7fcd52ffd640 1 -- 172.21.15.169:0/4183630790 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcd4c039020 con 0x7fcd5c0ac930 2023-10-26T01:44:15.964 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:15.961+0000 7fcd60bc0640 1 -- 172.21.15.169:0/4183630790 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcd5c04db10 con 0x7fcd5c0ac930 2023-10-26T01:44:15.965 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:15.963+0000 7fcd52ffd640 1 -- 172.21.15.169:0/4183630790 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) v1 ==== 94+0+4920 (secure 0 0 0) 0x7fcd4c041070 con 0x7fcd5c0ac930 2023-10-26T01:44:15.966 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:44:15.966 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":94,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6484,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/195700049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":195700049},{"type":"v1","addr":"172.21.15.169:6835","nonce":195700049}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":89},{"gid":6493,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/1167696300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1167696300},{"type":"v1","addr":"172.21.15.170:6836","nonce":1167696300}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":89},{"gid":9111,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/22729060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":22729060},{"type":"v1","addr":"172.21.15.169:6834","nonce":22729060}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":89}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T01:43:49.711084+0000","modified":"2023-10-26T01:43:52.816785+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9123},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9123":{"gid":9123,"name":"d","rank":0,"incarnation":90,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6837/2205518515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":2205518515},{"type":"v1","addr":"172.21.15.170:6837","nonce":2205518515}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2023-10-26T01:44:15.966 INFO:teuthology.orchestra.run.smithi169.stderr:dumped fsmap epoch 94 2023-10-26T01:44:15.969 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:15.966+0000 7fcd60bc0640 1 -- 172.21.15.169:0/4183630790 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcd3c06ae40 msgr2=0x7fcd3c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:15.969 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:15.966+0000 7fcd60bc0640 1 --2- 172.21.15.169:0/4183630790 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcd3c06ae40 0x7fcd3c06d300 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fcd44005fd0 tx=0x7fcd44005850 comp rx=0 tx=0).stop 2023-10-26T01:44:15.969 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:15.966+0000 7fcd60bc0640 1 -- 172.21.15.169:0/4183630790 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd5c0ac930 msgr2=0x7fcd5c154840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:15.969 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:15.966+0000 7fcd60bc0640 1 --2- 172.21.15.169:0/4183630790 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd5c0ac930 0x7fcd5c154840 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c009920 tx=0x7fcd4c0038a0 comp rx=0 tx=0).stop 2023-10-26T01:44:15.969 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:15.966+0000 7fcd60bc0640 1 -- 172.21.15.169:0/4183630790 shutdown_connections 2023-10-26T01:44:15.970 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:15.966+0000 7fcd60bc0640 1 --2- 172.21.15.169:0/4183630790 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcd3c06ae40 0x7fcd3c06d300 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:15.970 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:15.966+0000 7fcd60bc0640 1 --2- 172.21.15.169:0/4183630790 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcd5c1592d0 0x7fcd5c0c7b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:15.970 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:15.966+0000 7fcd60bc0640 1 --2- 172.21.15.169:0/4183630790 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcd5c0cdc30 0x7fcd5c159a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:15.970 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:15.966+0000 7fcd60bc0640 1 --2- 172.21.15.169:0/4183630790 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd5c0ac930 0x7fcd5c154840 unknown :-1 s=CLOSED pgs=1886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:15.970 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:15.966+0000 7fcd60bc0640 1 -- 172.21.15.169:0/4183630790 >> 172.21.15.169:0/4183630790 conn(0x7fcd5c069930 msgr2=0x7fcd5c069d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:15.971 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:15.966+0000 7fcd60bc0640 1 -- 172.21.15.169:0/4183630790 shutdown_connections 2023-10-26T01:44:15.971 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:15.966+0000 7fcd60bc0640 1 -- 172.21.15.169:0/4183630790 wait complete. 2023-10-26T01:44:15.988 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json status 2023-10-26T01:44:16.228 INFO:teuthology.orchestra.run.smithi170.stdout:{"cluster_fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","whoami":0,"id":9123,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":25.900438388000001,"mdsmap_epoch":94,"osdmap_epoch":184,"osdmap_epoch_barrier":184,"uptime":32.577836294999997} 2023-10-26T01:44:16.229 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "30203b28-ed8a-44e8-a9a6-a9bf25f980f1", "fs_name": "cephfs", "id": 9123, "mdsmap_epoch": 94, "osdmap_epoch": 184, "osdmap_epoch_barrier": 184, "rank_uptime": 25.900438388, "state": "up:active", "uptime": 32.577836295, "want_state": "up:active", "whoami": 0 } 2023-10-26T01:44:16.229 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-26T01:44:16.230 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:44:16.230 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_b) 2023-10-26T01:44:16.298 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:44:16.298 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_a) 2023-10-26T01:44:16.382 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-26T01:44:16.607 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:16.603+0000 7fabfc516640 1 -- 172.21.15.169:0/509779160 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fabe0003fb0 msgr2=0x7fabe00047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:16.607 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:16.603+0000 7fabfc516640 1 --2- 172.21.15.169:0/509779160 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fabe0003fb0 0x7fabe00047c0 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7fabe400c6e0 tx=0x7fabe00057f0 comp rx=0 tx=0).stop 2023-10-26T01:44:16.607 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:16.604+0000 7fabfaab3640 1 -- 172.21.15.169:0/509779160 reap_dead start 2023-10-26T01:44:16.607 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:16.604+0000 7fabfc516640 1 -- 172.21.15.169:0/509779160 shutdown_connections 2023-10-26T01:44:16.608 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:16.604+0000 7fabfc516640 1 -- 172.21.15.169:0/509779160 >> 172.21.15.169:0/509779160 conn(0x7fabf4069930 msgr2=0x7fabf4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:16.608 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:16.604+0000 7fabfc516640 1 -- 172.21.15.169:0/509779160 shutdown_connections 2023-10-26T01:44:16.608 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:16.604+0000 7fabfc516640 1 -- 172.21.15.169:0/509779160 wait complete. 2023-10-26T01:44:16.609 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:16.606+0000 7fabfc516640 1 Processor -- start 2023-10-26T01:44:16.609 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:16.606+0000 7fabfc516640 1 -- start start 2023-10-26T01:44:16.611 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:16.607+0000 7fabfc516640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fabe0003fb0 0x7fabf414b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:16.611 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:16.607+0000 7fabfc516640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fabf41e5030 0x7fabf41e54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:16.611 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:16.607+0000 7fabfc516640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fabf40b1090 0x7fabf40b1510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:16.611 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:16.607+0000 7fabfc516640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabf40b1a50 con 0x7fabf40b1090 2023-10-26T01:44:16.612 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:16.607+0000 7fabfc516640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabf41e5af0 con 0x7fabe0003fb0 2023-10-26T01:44:16.612 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:16.607+0000 7fabfc516640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabf41e5dd0 con 0x7fabf41e5030 2023-10-26T01:44:16.612 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:16.607+0000 7fabfaab3640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fabf40b1090 0x7fabf40b1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:16.612 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:16.607+0000 7fabfaab3640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fabf40b1090 0x7fabf40b1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:40798/0 (socket says 172.21.15.169:40798) 2023-10-26T01:44:16.612 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:16.607+0000 7fabfaab3640 1 -- 172.21.15.169:0/4094137766 learned_addr learned my addr 172.21.15.169:0/4094137766 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:44:16.613 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:16.607+0000 7fabfa2b2640 1 --2- 172.21.15.169:0/4094137766 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fabe0003fb0 0x7fabf414b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:16.613 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:16.607+0000 7fabf9ab1640 1 --2- 172.21.15.169:0/4094137766 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fabf41e5030 0x7fabf41e54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:16.613 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:16.608+0000 7fabfaab3640 1 -- 172.21.15.169:0/4094137766 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fabf41e5030 msgr2=0x7fabf41e54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:16.613 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:16.608+0000 7fabfaab3640 1 --2- 172.21.15.169:0/4094137766 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fabf41e5030 0x7fabf41e54b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:16.613 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:16.608+0000 7fabfaab3640 1 -- 172.21.15.169:0/4094137766 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fabe0003fb0 msgr2=0x7fabf414b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:16.614 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:16.608+0000 7fabfaab3640 1 --2- 172.21.15.169:0/4094137766 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fabe0003fb0 0x7fabf414b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:16.614 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:16.608+0000 7fabfaab3640 1 -- 172.21.15.169:0/4094137766 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabf41e60b0 con 0x7fabf40b1090 2023-10-26T01:44:16.614 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:16.608+0000 7fabfa2b2640 1 --2- 172.21.15.169:0/4094137766 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fabe0003fb0 0x7fabf414b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:44:16.614 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:16.608+0000 7fabfaab3640 1 --2- 172.21.15.169:0/4094137766 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fabf40b1090 0x7fabf40b1510 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7fabf40b3420 tx=0x7fabec00ef30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:16.614 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:16.608+0000 7fabeaffd640 1 -- 172.21.15.169:0/4094137766 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabec004d20 con 0x7fabf40b1090 2023-10-26T01:44:16.615 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:16.608+0000 7fabeaffd640 1 -- 172.21.15.169:0/4094137766 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fabec00caa0 con 0x7fabf40b1090 2023-10-26T01:44:16.615 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:16.608+0000 7fabeaffd640 1 -- 172.21.15.169:0/4094137766 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabec00ce70 con 0x7fabf40b1090 2023-10-26T01:44:16.615 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:16.608+0000 7fabfc516640 1 -- 172.21.15.169:0/4094137766 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabf41e6390 con 0x7fabf40b1090 2023-10-26T01:44:16.615 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:16.608+0000 7fabfc516640 1 -- 172.21.15.169:0/4094137766 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabf406f6c0 con 0x7fabf40b1090 2023-10-26T01:44:16.616 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:16.610+0000 7fabfc516640 1 -- 172.21.15.169:0/4094137766 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabf41e68b0 con 0x7fabf40b1090 2023-10-26T01:44:16.616 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:16.610+0000 7fabeaffd640 1 -- 172.21.15.169:0/4094137766 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fabec016030 con 0x7fabf40b1090 2023-10-26T01:44:16.617 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:16.611+0000 7fabeaffd640 1 --2- 172.21.15.169:0/4094137766 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fabd806ae40 0x7fabd806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:16.617 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:16.611+0000 7fabeaffd640 1 -- 172.21.15.169:0/4094137766 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fabec089820 con 0x7fabf40b1090 2023-10-26T01:44:16.617 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:16.612+0000 7fabeaffd640 1 -- 172.21.15.169:0/4094137766 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=185}) v3 -- 0x7fabd8072970 con 0x7fabf40b1090 2023-10-26T01:44:16.617 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:16.612+0000 7fabfa2b2640 1 --2- 172.21.15.169:0/4094137766 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fabd806ae40 0x7fabd806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:16.618 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:16.613+0000 7fabfa2b2640 1 --2- 172.21.15.169:0/4094137766 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fabd806ae40 0x7fabd806d300 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fabf00017a0 tx=0x7fabf0001bf0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:16.618 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:16.616+0000 7fabeaffd640 1 -- 172.21.15.169:0/4094137766 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fabec089020 con 0x7fabf40b1090 2023-10-26T01:44:16.814 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:16.811+0000 7fabfc516640 1 -- 172.21.15.169:0/4094137766 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fabf404db10 con 0x7fabf40b1090 2023-10-26T01:44:16.817 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:16.814+0000 7fabeaffd640 1 -- 172.21.15.169:0/4094137766 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v184) v1 ==== 81+0+34081 (secure 0 0 0) 0x7fabf41e68b0 con 0x7fabf40b1090 2023-10-26T01:44:16.818 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:44:16.819 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2023-10-26T01:44:16.819 INFO:teuthology.orchestra.run.smithi169.stdout: "epoch": 184, 2023-10-26T01:44:16.819 INFO:teuthology.orchestra.run.smithi169.stdout: "fsid": "30203b28-ed8a-44e8-a9a6-a9bf25f980f1", 2023-10-26T01:44:16.819 INFO:teuthology.orchestra.run.smithi169.stdout: "created": "2023-10-26T01:30:22.897155+0000", 2023-10-26T01:44:16.820 INFO:teuthology.orchestra.run.smithi169.stdout: "modified": "2023-10-26T01:44:13.194893+0000", 2023-10-26T01:44:16.820 INFO:teuthology.orchestra.run.smithi169.stdout: "last_up_change": "2023-10-26T01:30:31.875281+0000", 2023-10-26T01:44:16.820 INFO:teuthology.orchestra.run.smithi169.stdout: "last_in_change": "2023-10-26T01:30:27.637941+0000", 2023-10-26T01:44:16.820 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-26T01:44:16.820 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_num": 5799936, 2023-10-26T01:44:16.821 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_set": [ 2023-10-26T01:44:16.821 INFO:teuthology.orchestra.run.smithi169.stdout: "pglog_hardlimit", 2023-10-26T01:44:16.821 INFO:teuthology.orchestra.run.smithi169.stdout: "purged_snapdirs", 2023-10-26T01:44:16.821 INFO:teuthology.orchestra.run.smithi169.stdout: "recovery_deletes", 2023-10-26T01:44:16.821 INFO:teuthology.orchestra.run.smithi169.stdout: "sortbitwise" 2023-10-26T01:44:16.821 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:44:16.822 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_version": 4, 2023-10-26T01:44:16.822 INFO:teuthology.orchestra.run.smithi169.stdout: "full_ratio": 0.69999998807907104, 2023-10-26T01:44:16.822 INFO:teuthology.orchestra.run.smithi169.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-26T01:44:16.822 INFO:teuthology.orchestra.run.smithi169.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-26T01:44:16.823 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_snapshot": "", 2023-10-26T01:44:16.823 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_max": 23, 2023-10-26T01:44:16.823 INFO:teuthology.orchestra.run.smithi169.stdout: "max_osd": 8, 2023-10-26T01:44:16.823 INFO:teuthology.orchestra.run.smithi169.stdout: "require_min_compat_client": "luminous", 2023-10-26T01:44:16.823 INFO:teuthology.orchestra.run.smithi169.stdout: "min_compat_client": "jewel", 2023-10-26T01:44:16.824 INFO:teuthology.orchestra.run.smithi169.stdout: "require_osd_release": "reef", 2023-10-26T01:44:16.824 INFO:teuthology.orchestra.run.smithi169.stdout: "allow_crimson": false, 2023-10-26T01:44:16.824 INFO:teuthology.orchestra.run.smithi169.stdout: "pools": [ 2023-10-26T01:44:16.824 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:16.824 INFO:teuthology.orchestra.run.smithi169.stdout: "pool": 1, 2023-10-26T01:44:16.825 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_name": ".mgr", 2023-10-26T01:44:16.825 INFO:teuthology.orchestra.run.smithi169.stdout: "create_time": "2023-10-26T01:30:32.200808+0000", 2023-10-26T01:44:16.825 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": 1, 2023-10-26T01:44:16.825 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_names": "hashpspool", 2023-10-26T01:44:16.825 INFO:teuthology.orchestra.run.smithi169.stdout: "type": 1, 2023-10-26T01:44:16.826 INFO:teuthology.orchestra.run.smithi169.stdout: "size": 2, 2023-10-26T01:44:16.826 INFO:teuthology.orchestra.run.smithi169.stdout: "min_size": 1, 2023-10-26T01:44:16.826 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_rule": 0, 2023-10-26T01:44:16.826 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_count": 0, 2023-10-26T01:44:16.826 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_target": 0, 2023-10-26T01:44:16.827 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T01:44:16.827 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T01:44:16.827 INFO:teuthology.orchestra.run.smithi169.stdout: "object_hash": 2, 2023-10-26T01:44:16.827 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_mode": "off", 2023-10-26T01:44:16.827 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num": 1, 2023-10-26T01:44:16.827 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num": 1, 2023-10-26T01:44:16.828 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num_target": 1, 2023-10-26T01:44:16.828 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_target": 1, 2023-10-26T01:44:16.828 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_pending": 1, 2023-10-26T01:44:16.828 INFO:teuthology.orchestra.run.smithi169.stdout: "last_pg_merge_meta": { 2023-10-26T01:44:16.828 INFO:teuthology.orchestra.run.smithi169.stdout: "source_pgid": "0.0", 2023-10-26T01:44:16.829 INFO:teuthology.orchestra.run.smithi169.stdout: "ready_epoch": 0, 2023-10-26T01:44:16.829 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_started": 0, 2023-10-26T01:44:16.829 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_clean": 0, 2023-10-26T01:44:16.829 INFO:teuthology.orchestra.run.smithi169.stdout: "source_version": "0'0", 2023-10-26T01:44:16.829 INFO:teuthology.orchestra.run.smithi169.stdout: "target_version": "0'0" 2023-10-26T01:44:16.830 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:16.830 INFO:teuthology.orchestra.run.smithi169.stdout: "last_change": "109", 2023-10-26T01:44:16.830 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend": "0", 2023-10-26T01:44:16.830 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T01:44:16.831 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T01:44:16.831 INFO:teuthology.orchestra.run.smithi169.stdout: "auid": 0, 2023-10-26T01:44:16.831 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_mode": "selfmanaged", 2023-10-26T01:44:16.831 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_seq": 0, 2023-10-26T01:44:16.831 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_epoch": 0, 2023-10-26T01:44:16.832 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_snaps": [], 2023-10-26T01:44:16.832 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps": "[]", 2023-10-26T01:44:16.832 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_bytes": 0, 2023-10-26T01:44:16.832 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_objects": 0, 2023-10-26T01:44:16.832 INFO:teuthology.orchestra.run.smithi169.stdout: "tiers": [], 2023-10-26T01:44:16.833 INFO:teuthology.orchestra.run.smithi169.stdout: "tier_of": -1, 2023-10-26T01:44:16.833 INFO:teuthology.orchestra.run.smithi169.stdout: "read_tier": -1, 2023-10-26T01:44:16.833 INFO:teuthology.orchestra.run.smithi169.stdout: "write_tier": -1, 2023-10-26T01:44:16.833 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_mode": "none", 2023-10-26T01:44:16.833 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_bytes": 0, 2023-10-26T01:44:16.834 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_objects": 0, 2023-10-26T01:44:16.834 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T01:44:16.834 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T01:44:16.834 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T01:44:16.835 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_flush_age": 0, 2023-10-26T01:44:16.835 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_evict_age": 0, 2023-10-26T01:44:16.835 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profile": "", 2023-10-26T01:44:16.835 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_params": { 2023-10-26T01:44:16.835 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "none" 2023-10-26T01:44:16.836 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:16.836 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_period": 0, 2023-10-26T01:44:16.836 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_count": 0, 2023-10-26T01:44:16.836 INFO:teuthology.orchestra.run.smithi169.stdout: "use_gmt_hitset": true, 2023-10-26T01:44:16.836 INFO:teuthology.orchestra.run.smithi169.stdout: "min_read_recency_for_promote": 0, 2023-10-26T01:44:16.837 INFO:teuthology.orchestra.run.smithi169.stdout: "min_write_recency_for_promote": 0, 2023-10-26T01:44:16.837 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T01:44:16.837 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_search_last_n": 0, 2023-10-26T01:44:16.837 INFO:teuthology.orchestra.run.smithi169.stdout: "grade_table": [], 2023-10-26T01:44:16.837 INFO:teuthology.orchestra.run.smithi169.stdout: "stripe_width": 0, 2023-10-26T01:44:16.837 INFO:teuthology.orchestra.run.smithi169.stdout: "expected_num_objects": 0, 2023-10-26T01:44:16.838 INFO:teuthology.orchestra.run.smithi169.stdout: "fast_read": false, 2023-10-26T01:44:16.838 INFO:teuthology.orchestra.run.smithi169.stdout: "options": { 2023-10-26T01:44:16.838 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_max": 32, 2023-10-26T01:44:16.838 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_min": 1 2023-10-26T01:44:16.838 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:16.839 INFO:teuthology.orchestra.run.smithi169.stdout: "application_metadata": { 2023-10-26T01:44:16.839 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": {} 2023-10-26T01:44:16.839 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:16.839 INFO:teuthology.orchestra.run.smithi169.stdout: "read_balance": { 2023-10-26T01:44:16.839 INFO:teuthology.orchestra.run.smithi169.stdout: "score_acting": 8, 2023-10-26T01:44:16.840 INFO:teuthology.orchestra.run.smithi169.stdout: "score_stable": 8, 2023-10-26T01:44:16.840 INFO:teuthology.orchestra.run.smithi169.stdout: "optimal_score": 0.25, 2023-10-26T01:44:16.840 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_acting": 2, 2023-10-26T01:44:16.840 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_stable": 2, 2023-10-26T01:44:16.840 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity_weighted": 1, 2023-10-26T01:44:16.841 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity": 1, 2023-10-26T01:44:16.841 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity_weighted": 1 2023-10-26T01:44:16.841 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:16.841 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:16.842 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:16.842 INFO:teuthology.orchestra.run.smithi169.stdout: "pool": 22, 2023-10-26T01:44:16.842 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_name": "cephfs_metadata", 2023-10-26T01:44:16.842 INFO:teuthology.orchestra.run.smithi169.stdout: "create_time": "2023-10-26T01:43:47.501296+0000", 2023-10-26T01:44:16.842 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": 1, 2023-10-26T01:44:16.843 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_names": "hashpspool", 2023-10-26T01:44:16.843 INFO:teuthology.orchestra.run.smithi169.stdout: "type": 1, 2023-10-26T01:44:16.843 INFO:teuthology.orchestra.run.smithi169.stdout: "size": 2, 2023-10-26T01:44:16.843 INFO:teuthology.orchestra.run.smithi169.stdout: "min_size": 1, 2023-10-26T01:44:16.843 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_rule": 0, 2023-10-26T01:44:16.844 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_count": 0, 2023-10-26T01:44:16.844 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_target": 0, 2023-10-26T01:44:16.844 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T01:44:16.844 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T01:44:16.844 INFO:teuthology.orchestra.run.smithi169.stdout: "object_hash": 2, 2023-10-26T01:44:16.845 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_mode": "off", 2023-10-26T01:44:16.845 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num": 32, 2023-10-26T01:44:16.845 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num": 32, 2023-10-26T01:44:16.845 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num_target": 32, 2023-10-26T01:44:16.845 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_target": 32, 2023-10-26T01:44:16.846 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_pending": 32, 2023-10-26T01:44:16.846 INFO:teuthology.orchestra.run.smithi169.stdout: "last_pg_merge_meta": { 2023-10-26T01:44:16.846 INFO:teuthology.orchestra.run.smithi169.stdout: "source_pgid": "0.0", 2023-10-26T01:44:16.846 INFO:teuthology.orchestra.run.smithi169.stdout: "ready_epoch": 0, 2023-10-26T01:44:16.846 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_started": 0, 2023-10-26T01:44:16.847 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_clean": 0, 2023-10-26T01:44:16.847 INFO:teuthology.orchestra.run.smithi169.stdout: "source_version": "0'0", 2023-10-26T01:44:16.847 INFO:teuthology.orchestra.run.smithi169.stdout: "target_version": "0'0" 2023-10-26T01:44:16.847 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:16.847 INFO:teuthology.orchestra.run.smithi169.stdout: "last_change": "181", 2023-10-26T01:44:16.848 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend": "0", 2023-10-26T01:44:16.848 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T01:44:16.848 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T01:44:16.848 INFO:teuthology.orchestra.run.smithi169.stdout: "auid": 0, 2023-10-26T01:44:16.848 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_mode": "selfmanaged", 2023-10-26T01:44:16.848 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_seq": 0, 2023-10-26T01:44:16.849 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_epoch": 0, 2023-10-26T01:44:16.849 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_snaps": [], 2023-10-26T01:44:16.849 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps": "[]", 2023-10-26T01:44:16.849 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_bytes": 0, 2023-10-26T01:44:16.849 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_objects": 0, 2023-10-26T01:44:16.849 INFO:teuthology.orchestra.run.smithi169.stdout: "tiers": [], 2023-10-26T01:44:16.850 INFO:teuthology.orchestra.run.smithi169.stdout: "tier_of": -1, 2023-10-26T01:44:16.850 INFO:teuthology.orchestra.run.smithi169.stdout: "read_tier": -1, 2023-10-26T01:44:16.850 INFO:teuthology.orchestra.run.smithi169.stdout: "write_tier": -1, 2023-10-26T01:44:16.850 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_mode": "none", 2023-10-26T01:44:16.851 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_bytes": 0, 2023-10-26T01:44:16.851 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_objects": 0, 2023-10-26T01:44:16.851 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T01:44:16.851 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T01:44:16.851 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T01:44:16.852 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_flush_age": 0, 2023-10-26T01:44:16.852 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_evict_age": 0, 2023-10-26T01:44:16.852 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profile": "", 2023-10-26T01:44:16.852 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_params": { 2023-10-26T01:44:16.852 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "none" 2023-10-26T01:44:16.852 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:16.853 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_period": 0, 2023-10-26T01:44:16.853 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_count": 0, 2023-10-26T01:44:16.853 INFO:teuthology.orchestra.run.smithi169.stdout: "use_gmt_hitset": true, 2023-10-26T01:44:16.853 INFO:teuthology.orchestra.run.smithi169.stdout: "min_read_recency_for_promote": 0, 2023-10-26T01:44:16.853 INFO:teuthology.orchestra.run.smithi169.stdout: "min_write_recency_for_promote": 0, 2023-10-26T01:44:16.854 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T01:44:16.854 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_search_last_n": 0, 2023-10-26T01:44:16.854 INFO:teuthology.orchestra.run.smithi169.stdout: "grade_table": [], 2023-10-26T01:44:16.854 INFO:teuthology.orchestra.run.smithi169.stdout: "stripe_width": 0, 2023-10-26T01:44:16.854 INFO:teuthology.orchestra.run.smithi169.stdout: "expected_num_objects": 0, 2023-10-26T01:44:16.855 INFO:teuthology.orchestra.run.smithi169.stdout: "fast_read": false, 2023-10-26T01:44:16.855 INFO:teuthology.orchestra.run.smithi169.stdout: "options": { 2023-10-26T01:44:16.855 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_bias": 4, 2023-10-26T01:44:16.855 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_min": 16, 2023-10-26T01:44:16.855 INFO:teuthology.orchestra.run.smithi169.stdout: "recovery_priority": 5 2023-10-26T01:44:16.855 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:16.856 INFO:teuthology.orchestra.run.smithi169.stdout: "application_metadata": { 2023-10-26T01:44:16.856 INFO:teuthology.orchestra.run.smithi169.stdout: "cephfs": { 2023-10-26T01:44:16.856 INFO:teuthology.orchestra.run.smithi169.stdout: "metadata": "cephfs" 2023-10-26T01:44:16.856 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:16.857 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:16.857 INFO:teuthology.orchestra.run.smithi169.stdout: "read_balance": { 2023-10-26T01:44:16.857 INFO:teuthology.orchestra.run.smithi169.stdout: "score_acting": 2, 2023-10-26T01:44:16.857 INFO:teuthology.orchestra.run.smithi169.stdout: "score_stable": 2, 2023-10-26T01:44:16.857 INFO:teuthology.orchestra.run.smithi169.stdout: "optimal_score": 1, 2023-10-26T01:44:16.857 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_acting": 2, 2023-10-26T01:44:16.858 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_stable": 2, 2023-10-26T01:44:16.858 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity_weighted": 1, 2023-10-26T01:44:16.858 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity": 1, 2023-10-26T01:44:16.858 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity_weighted": 1 2023-10-26T01:44:16.858 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:16.859 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:16.859 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:16.859 INFO:teuthology.orchestra.run.smithi169.stdout: "pool": 23, 2023-10-26T01:44:16.859 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_name": "cephfs_data", 2023-10-26T01:44:16.859 INFO:teuthology.orchestra.run.smithi169.stdout: "create_time": "2023-10-26T01:43:48.737507+0000", 2023-10-26T01:44:16.860 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": 1027, 2023-10-26T01:44:16.860 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_names": "hashpspool,full,full_quota", 2023-10-26T01:44:16.860 INFO:teuthology.orchestra.run.smithi169.stdout: "type": 1, 2023-10-26T01:44:16.860 INFO:teuthology.orchestra.run.smithi169.stdout: "size": 2, 2023-10-26T01:44:16.860 INFO:teuthology.orchestra.run.smithi169.stdout: "min_size": 1, 2023-10-26T01:44:16.861 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_rule": 0, 2023-10-26T01:44:16.861 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_count": 0, 2023-10-26T01:44:16.861 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_target": 0, 2023-10-26T01:44:16.861 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T01:44:16.861 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T01:44:16.861 INFO:teuthology.orchestra.run.smithi169.stdout: "object_hash": 2, 2023-10-26T01:44:16.862 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_mode": "off", 2023-10-26T01:44:16.862 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num": 64, 2023-10-26T01:44:16.862 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num": 64, 2023-10-26T01:44:16.862 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num_target": 64, 2023-10-26T01:44:16.862 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_target": 64, 2023-10-26T01:44:16.863 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_pending": 64, 2023-10-26T01:44:16.863 INFO:teuthology.orchestra.run.smithi169.stdout: "last_pg_merge_meta": { 2023-10-26T01:44:16.863 INFO:teuthology.orchestra.run.smithi169.stdout: "source_pgid": "0.0", 2023-10-26T01:44:16.863 INFO:teuthology.orchestra.run.smithi169.stdout: "ready_epoch": 0, 2023-10-26T01:44:16.863 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_started": 0, 2023-10-26T01:44:16.864 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_clean": 0, 2023-10-26T01:44:16.864 INFO:teuthology.orchestra.run.smithi169.stdout: "source_version": "0'0", 2023-10-26T01:44:16.864 INFO:teuthology.orchestra.run.smithi169.stdout: "target_version": "0'0" 2023-10-26T01:44:16.864 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:16.864 INFO:teuthology.orchestra.run.smithi169.stdout: "last_change": "184", 2023-10-26T01:44:16.864 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend": "0", 2023-10-26T01:44:16.865 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T01:44:16.865 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T01:44:16.865 INFO:teuthology.orchestra.run.smithi169.stdout: "auid": 0, 2023-10-26T01:44:16.865 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_mode": "selfmanaged", 2023-10-26T01:44:16.865 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_seq": 0, 2023-10-26T01:44:16.866 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_epoch": 0, 2023-10-26T01:44:16.866 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_snaps": [], 2023-10-26T01:44:16.866 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps": "[]", 2023-10-26T01:44:16.866 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_bytes": 33554432, 2023-10-26T01:44:16.866 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_objects": 0, 2023-10-26T01:44:16.867 INFO:teuthology.orchestra.run.smithi169.stdout: "tiers": [], 2023-10-26T01:44:16.867 INFO:teuthology.orchestra.run.smithi169.stdout: "tier_of": -1, 2023-10-26T01:44:16.867 INFO:teuthology.orchestra.run.smithi169.stdout: "read_tier": -1, 2023-10-26T01:44:16.867 INFO:teuthology.orchestra.run.smithi169.stdout: "write_tier": -1, 2023-10-26T01:44:16.867 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_mode": "none", 2023-10-26T01:44:16.867 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_bytes": 0, 2023-10-26T01:44:16.868 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_objects": 0, 2023-10-26T01:44:16.868 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T01:44:16.868 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T01:44:16.868 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T01:44:16.868 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_flush_age": 0, 2023-10-26T01:44:16.869 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_evict_age": 0, 2023-10-26T01:44:16.869 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profile": "", 2023-10-26T01:44:16.869 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_params": { 2023-10-26T01:44:16.869 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "none" 2023-10-26T01:44:16.869 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:16.870 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_period": 0, 2023-10-26T01:44:16.870 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_count": 0, 2023-10-26T01:44:16.870 INFO:teuthology.orchestra.run.smithi169.stdout: "use_gmt_hitset": true, 2023-10-26T01:44:16.870 INFO:teuthology.orchestra.run.smithi169.stdout: "min_read_recency_for_promote": 0, 2023-10-26T01:44:16.870 INFO:teuthology.orchestra.run.smithi169.stdout: "min_write_recency_for_promote": 0, 2023-10-26T01:44:16.870 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T01:44:16.871 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_search_last_n": 0, 2023-10-26T01:44:16.871 INFO:teuthology.orchestra.run.smithi169.stdout: "grade_table": [], 2023-10-26T01:44:16.871 INFO:teuthology.orchestra.run.smithi169.stdout: "stripe_width": 0, 2023-10-26T01:44:16.871 INFO:teuthology.orchestra.run.smithi169.stdout: "expected_num_objects": 0, 2023-10-26T01:44:16.871 INFO:teuthology.orchestra.run.smithi169.stdout: "fast_read": false, 2023-10-26T01:44:16.872 INFO:teuthology.orchestra.run.smithi169.stdout: "options": { 2023-10-26T01:44:16.872 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_min": 64, 2023-10-26T01:44:16.872 INFO:teuthology.orchestra.run.smithi169.stdout: "target_size_ratio": 0.89999997615814209 2023-10-26T01:44:16.872 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:16.872 INFO:teuthology.orchestra.run.smithi169.stdout: "application_metadata": { 2023-10-26T01:44:16.873 INFO:teuthology.orchestra.run.smithi169.stdout: "cephfs": { 2023-10-26T01:44:16.873 INFO:teuthology.orchestra.run.smithi169.stdout: "data": "cephfs" 2023-10-26T01:44:16.873 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:16.873 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:16.873 INFO:teuthology.orchestra.run.smithi169.stdout: "read_balance": { 2023-10-26T01:44:16.874 INFO:teuthology.orchestra.run.smithi169.stdout: "score_acting": 1.3799999952316284, 2023-10-26T01:44:16.874 INFO:teuthology.orchestra.run.smithi169.stdout: "score_stable": 1.3799999952316284, 2023-10-26T01:44:16.874 INFO:teuthology.orchestra.run.smithi169.stdout: "optimal_score": 1, 2023-10-26T01:44:16.874 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_acting": 1.3799999952316284, 2023-10-26T01:44:16.874 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_stable": 1.3799999952316284, 2023-10-26T01:44:16.875 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity_weighted": 1, 2023-10-26T01:44:16.875 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity": 1, 2023-10-26T01:44:16.875 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity_weighted": 1 2023-10-26T01:44:16.875 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:16.875 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:16.875 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:44:16.876 INFO:teuthology.orchestra.run.smithi169.stdout: "osds": [ 2023-10-26T01:44:16.876 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:16.876 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 0, 2023-10-26T01:44:16.876 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:44:16.876 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:44:16.877 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:44:16.877 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:44:16.877 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:44:16.877 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:44:16.877 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:44:16.878 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:44:16.878 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 180, 2023-10-26T01:44:16.878 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:44:16.878 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:44:16.878 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:44:16.879 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:16.879 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:16.879 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:16.879 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6816", 2023-10-26T01:44:16.879 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:44:16.880 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:16.880 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:16.880 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:16.880 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6817", 2023-10-26T01:44:16.880 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:44:16.881 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:16.881 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:16.881 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:16.881 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:44:16.881 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:16.881 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:16.882 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:16.882 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6818", 2023-10-26T01:44:16.882 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:44:16.882 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:16.882 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:16.883 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:16.883 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6819", 2023-10-26T01:44:16.883 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:44:16.883 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:16.883 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:16.884 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:16.884 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:44:16.884 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:16.884 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:16.884 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:16.885 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6822", 2023-10-26T01:44:16.885 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:44:16.885 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:16.885 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:16.885 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:16.885 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6823", 2023-10-26T01:44:16.886 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:44:16.886 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:16.886 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:16.886 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:16.886 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:44:16.887 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:16.887 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:16.887 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:16.887 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6820", 2023-10-26T01:44:16.887 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:44:16.888 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:16.888 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:16.888 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:16.888 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6821", 2023-10-26T01:44:16.888 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:44:16.889 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:16.889 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:16.889 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:16.889 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6817/4204255864", 2023-10-26T01:44:16.889 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6819/4204255864", 2023-10-26T01:44:16.890 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6823/4204255864", 2023-10-26T01:44:16.890 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6821/4204255864", 2023-10-26T01:44:16.890 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:44:16.890 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:44:16.890 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:44:16.891 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:16.891 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:16.891 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:16.891 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 1, 2023-10-26T01:44:16.891 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:44:16.892 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:44:16.892 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:44:16.892 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:44:16.892 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:44:16.892 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:44:16.893 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:44:16.893 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:44:16.893 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 180, 2023-10-26T01:44:16.893 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:44:16.893 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:44:16.894 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:44:16.894 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:16.894 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:16.894 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:16.894 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6808", 2023-10-26T01:44:16.895 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:44:16.895 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:16.895 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:16.895 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:16.895 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6809", 2023-10-26T01:44:16.896 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:44:16.896 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:16.896 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:16.896 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:16.896 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:44:16.897 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:16.897 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:16.897 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:16.897 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6810", 2023-10-26T01:44:16.897 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:44:16.898 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:16.898 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:16.898 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:16.898 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6811", 2023-10-26T01:44:16.898 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:44:16.899 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:16.899 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:16.899 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:16.899 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:44:16.899 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:16.900 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:16.900 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:16.900 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6814", 2023-10-26T01:44:16.900 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:44:16.900 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:16.901 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:16.901 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:16.901 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6815", 2023-10-26T01:44:16.901 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:44:16.901 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:16.902 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:16.902 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:16.902 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:44:16.902 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:16.902 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:16.903 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:16.903 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6812", 2023-10-26T01:44:16.903 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:44:16.903 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:16.903 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:16.904 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:16.904 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6813", 2023-10-26T01:44:16.904 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:44:16.904 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:16.904 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:16.905 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:16.905 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6809/238206046", 2023-10-26T01:44:16.905 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6811/238206046", 2023-10-26T01:44:16.905 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6815/238206046", 2023-10-26T01:44:16.905 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6813/238206046", 2023-10-26T01:44:16.905 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:44:16.906 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:44:16.906 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:44:16.906 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:16.906 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:16.906 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:16.907 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 2, 2023-10-26T01:44:16.907 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:44:16.907 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:44:16.907 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:44:16.907 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:44:16.907 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:44:16.908 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:44:16.908 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:44:16.908 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:44:16.908 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 180, 2023-10-26T01:44:16.908 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:44:16.909 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:44:16.909 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:44:16.909 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:16.909 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:16.909 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:16.910 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6800", 2023-10-26T01:44:16.910 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:44:16.910 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:16.910 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:16.910 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:16.911 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6801", 2023-10-26T01:44:16.911 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:44:16.911 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:16.911 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:16.911 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:16.911 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:44:16.912 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:16.912 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:16.912 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:16.912 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6802", 2023-10-26T01:44:16.912 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:44:16.913 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:16.913 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:16.913 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:16.913 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6803", 2023-10-26T01:44:16.913 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:44:16.914 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:16.914 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:16.914 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:16.914 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:44:16.914 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:16.915 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:16.915 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:16.915 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6806", 2023-10-26T01:44:16.915 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:44:16.915 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:16.915 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:16.916 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:16.916 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6807", 2023-10-26T01:44:16.916 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:44:16.916 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:16.916 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:16.917 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:16.917 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:44:16.917 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:16.917 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:16.917 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:16.917 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6804", 2023-10-26T01:44:16.918 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:44:16.918 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:16.918 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:16.918 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:16.918 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6805", 2023-10-26T01:44:16.919 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:44:16.919 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:16.919 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:16.919 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:16.919 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6801/3215791846", 2023-10-26T01:44:16.920 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6803/3215791846", 2023-10-26T01:44:16.920 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6807/3215791846", 2023-10-26T01:44:16.920 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6805/3215791846", 2023-10-26T01:44:16.920 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:44:16.920 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:44:16.921 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:44:16.921 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:16.921 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:16.921 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:16.921 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 3, 2023-10-26T01:44:16.922 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:44:16.922 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:44:16.922 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:44:16.922 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:44:16.923 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:44:16.923 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:44:16.923 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:44:16.923 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 14, 2023-10-26T01:44:16.923 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 180, 2023-10-26T01:44:16.924 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:44:16.924 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:44:16.924 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:44:16.924 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:16.924 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:16.925 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:16.925 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6824", 2023-10-26T01:44:16.925 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:44:16.925 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:16.925 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:16.925 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:16.926 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6825", 2023-10-26T01:44:16.926 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:44:16.926 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:16.926 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:16.927 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:16.927 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:44:16.927 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:16.927 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:16.927 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:16.928 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6826", 2023-10-26T01:44:16.928 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:44:16.928 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:16.928 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:16.928 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:16.928 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6827", 2023-10-26T01:44:16.929 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:44:16.929 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:16.929 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:16.929 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:16.929 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:44:16.930 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:16.930 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:16.930 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:16.930 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6830", 2023-10-26T01:44:16.930 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:44:16.931 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:16.931 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:16.931 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:16.931 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6831", 2023-10-26T01:44:16.931 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:44:16.932 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:16.932 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:16.932 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:16.932 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:44:16.932 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:16.932 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:16.933 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:16.933 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6828", 2023-10-26T01:44:16.933 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:44:16.933 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:16.933 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:16.934 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:16.934 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6829", 2023-10-26T01:44:16.934 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:44:16.934 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:16.934 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:16.935 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:16.935 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6825/2256701226", 2023-10-26T01:44:16.935 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6827/2256701226", 2023-10-26T01:44:16.935 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6831/2256701226", 2023-10-26T01:44:16.935 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6829/2256701226", 2023-10-26T01:44:16.936 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:44:16.936 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:44:16.936 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:44:16.936 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:16.936 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:16.936 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:16.937 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 4, 2023-10-26T01:44:16.937 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:44:16.937 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:44:16.937 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:44:16.937 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:44:16.938 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:44:16.938 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:44:16.938 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:44:16.938 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:44:16.938 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 180, 2023-10-26T01:44:16.939 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:44:16.939 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:44:16.939 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:44:16.939 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:16.939 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:16.940 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:16.940 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6816", 2023-10-26T01:44:16.940 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:44:16.940 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:16.940 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:16.940 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:16.941 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6817", 2023-10-26T01:44:16.941 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:44:16.941 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:16.941 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:16.941 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:16.942 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:44:16.942 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:16.942 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:16.942 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:16.942 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6818", 2023-10-26T01:44:16.943 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:44:16.943 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:16.943 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:16.943 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:16.943 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6819", 2023-10-26T01:44:16.944 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:44:16.944 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:16.944 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:16.944 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:16.944 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:44:16.945 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:16.945 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:16.945 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:16.945 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6825", 2023-10-26T01:44:16.945 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:44:16.946 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:16.946 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:16.946 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:16.946 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6828", 2023-10-26T01:44:16.946 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:44:16.946 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:16.947 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:16.947 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:16.947 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:44:16.947 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:16.947 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:16.948 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:16.948 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6820", 2023-10-26T01:44:16.948 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:44:16.948 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:16.948 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:16.949 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:16.949 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6823", 2023-10-26T01:44:16.949 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:44:16.949 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:16.949 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:16.949 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:16.950 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6817/1082279874", 2023-10-26T01:44:16.950 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6819/1082279874", 2023-10-26T01:44:16.950 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6828/1082279874", 2023-10-26T01:44:16.950 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6823/1082279874", 2023-10-26T01:44:16.950 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:44:16.951 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:44:16.951 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:44:16.951 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:16.951 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:16.951 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:16.952 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 5, 2023-10-26T01:44:16.952 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:44:16.952 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:44:16.952 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:44:16.952 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:44:16.953 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:44:16.953 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:44:16.953 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:44:16.953 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:44:16.953 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 180, 2023-10-26T01:44:16.953 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:44:16.954 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:44:16.954 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:44:16.954 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:16.954 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:16.955 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:16.955 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6808", 2023-10-26T01:44:16.955 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:44:16.955 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:16.955 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:16.956 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:16.956 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6809", 2023-10-26T01:44:16.956 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:44:16.956 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:16.956 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:16.957 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:16.957 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:44:16.957 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:16.957 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:16.957 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:16.958 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6810", 2023-10-26T01:44:16.958 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:44:16.958 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:16.958 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:16.958 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:16.959 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6811", 2023-10-26T01:44:16.959 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:44:16.959 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:16.959 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:16.960 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:16.960 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:44:16.960 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:16.961 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:16.961 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:16.961 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6814", 2023-10-26T01:44:16.961 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:44:16.961 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:16.962 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:16.962 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:16.962 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6815", 2023-10-26T01:44:16.962 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:44:16.963 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:16.963 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:16.963 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:16.963 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:44:16.963 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:16.964 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:16.964 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:16.964 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6812", 2023-10-26T01:44:16.964 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:44:16.965 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:16.965 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:16.965 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:16.965 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6813", 2023-10-26T01:44:16.965 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:44:16.966 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:16.966 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:16.966 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:16.966 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6809/3335654351", 2023-10-26T01:44:16.966 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6811/3335654351", 2023-10-26T01:44:16.967 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6815/3335654351", 2023-10-26T01:44:16.967 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6813/3335654351", 2023-10-26T01:44:16.967 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:44:16.967 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:44:16.967 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:44:16.968 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:16.968 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:16.968 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:16.968 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 6, 2023-10-26T01:44:16.968 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:44:16.968 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:44:16.969 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:44:16.969 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:44:16.969 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:44:16.969 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:44:16.969 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:44:16.970 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:44:16.970 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 180, 2023-10-26T01:44:16.970 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:44:16.970 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:44:16.970 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:44:16.971 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:16.971 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:16.971 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:16.971 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6800", 2023-10-26T01:44:16.971 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:44:16.972 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:16.972 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:16.972 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:16.972 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6801", 2023-10-26T01:44:16.972 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:44:16.972 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:16.973 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:16.973 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:16.973 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:44:16.973 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:16.973 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:16.974 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:16.974 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6802", 2023-10-26T01:44:16.974 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:44:16.974 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:16.974 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:16.974 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:16.975 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6803", 2023-10-26T01:44:16.975 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:44:16.975 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:16.975 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:16.975 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:16.976 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:44:16.976 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:16.976 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:16.976 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:16.976 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6806", 2023-10-26T01:44:16.977 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:44:16.977 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:16.977 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:16.977 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:16.977 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6807", 2023-10-26T01:44:16.978 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:44:16.978 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:16.978 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:16.978 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:16.978 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:44:16.978 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:16.979 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:16.979 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:16.979 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6804", 2023-10-26T01:44:16.979 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:44:16.980 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:16.980 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:16.980 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:16.980 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6805", 2023-10-26T01:44:16.980 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:44:16.981 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:16.981 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:16.981 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:16.981 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6801/2520460697", 2023-10-26T01:44:16.981 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6803/2520460697", 2023-10-26T01:44:16.982 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6807/2520460697", 2023-10-26T01:44:16.982 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6805/2520460697", 2023-10-26T01:44:16.982 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:44:16.982 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:44:16.982 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:44:16.983 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:16.983 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:16.983 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:16.983 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 7, 2023-10-26T01:44:16.983 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:44:16.984 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:44:16.984 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:44:16.984 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:44:16.984 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:44:16.984 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:44:16.984 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:44:16.985 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:44:16.985 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 180, 2023-10-26T01:44:16.985 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:44:16.985 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:44:16.985 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:44:16.986 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:16.986 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:16.986 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:16.986 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6821", 2023-10-26T01:44:16.987 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:44:16.987 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:16.987 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:16.987 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:16.987 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6822", 2023-10-26T01:44:16.987 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:44:16.988 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:16.988 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:16.988 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:16.988 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:44:16.988 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:16.989 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:16.989 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:16.989 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6824", 2023-10-26T01:44:16.989 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:44:16.989 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:16.990 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:16.990 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:16.990 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6826", 2023-10-26T01:44:16.990 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:44:16.990 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:16.991 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:16.991 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:16.991 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:44:16.991 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:16.991 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:16.992 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:16.992 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6830", 2023-10-26T01:44:16.992 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:44:16.992 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:16.992 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:16.992 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:16.993 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6831", 2023-10-26T01:44:16.993 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:44:16.993 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:16.993 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:16.993 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:16.994 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:44:16.994 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:16.994 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:16.994 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:16.994 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6827", 2023-10-26T01:44:16.995 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:44:16.995 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:16.995 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:16.995 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:16.995 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6829", 2023-10-26T01:44:16.996 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:44:16.996 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:16.996 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:16.996 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:16.996 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6822/1115930619", 2023-10-26T01:44:16.997 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6826/1115930619", 2023-10-26T01:44:16.997 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6831/1115930619", 2023-10-26T01:44:16.997 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6829/1115930619", 2023-10-26T01:44:16.997 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:44:16.998 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:44:16.998 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:44:16.998 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:16.999 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:16.999 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:44:16.999 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_xinfo": [ 2023-10-26T01:44:16.999 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:16.999 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 0, 2023-10-26T01:44:17.000 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:44:17.000 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:44:17.000 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:44:17.000 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:44:17.000 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:44:17.001 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.889675+0000", 2023-10-26T01:44:17.001 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:44:17.001 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:17.001 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:17.001 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 1, 2023-10-26T01:44:17.002 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:44:17.002 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:44:17.002 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:44:17.002 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:44:17.002 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:44:17.003 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.864157+0000", 2023-10-26T01:44:17.003 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:44:17.003 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:17.003 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:17.003 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 2, 2023-10-26T01:44:17.004 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:44:17.004 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:44:17.004 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:44:17.004 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:44:17.004 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:44:17.005 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.876919+0000", 2023-10-26T01:44:17.005 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:44:17.005 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:17.005 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:17.005 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 3, 2023-10-26T01:44:17.005 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:44:17.006 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:44:17.006 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:44:17.006 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:44:17.006 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:44:17.006 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.974541+0000", 2023-10-26T01:44:17.007 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:44:17.007 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:17.007 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:17.007 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 4, 2023-10-26T01:44:17.007 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:44:17.008 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:44:17.008 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:44:17.008 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:44:17.008 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:44:17.008 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.969696+0000", 2023-10-26T01:44:17.008 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:44:17.009 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:17.009 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:17.009 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 5, 2023-10-26T01:44:17.009 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:44:17.009 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:44:17.010 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:44:17.010 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:44:17.010 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:44:17.010 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.859703+0000", 2023-10-26T01:44:17.010 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:44:17.011 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:17.011 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:17.011 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 6, 2023-10-26T01:44:17.011 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:44:17.011 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:44:17.012 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:44:17.012 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:44:17.012 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:44:17.012 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.867917+0000", 2023-10-26T01:44:17.012 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:44:17.012 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:17.013 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:17.013 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 7, 2023-10-26T01:44:17.013 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:44:17.013 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:44:17.013 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:44:17.014 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:44:17.014 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:44:17.014 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.898605+0000", 2023-10-26T01:44:17.014 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:44:17.014 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:17.014 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:44:17.015 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_upmap": [], 2023-10-26T01:44:17.015 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_upmap_items": [], 2023-10-26T01:44:17.015 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_upmap_primaries": [], 2023-10-26T01:44:17.015 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_temp": [], 2023-10-26T01:44:17.015 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_temp": [], 2023-10-26T01:44:17.016 INFO:teuthology.orchestra.run.smithi169.stdout: "blocklist": {}, 2023-10-26T01:44:17.016 INFO:teuthology.orchestra.run.smithi169.stdout: "range_blocklist": {}, 2023-10-26T01:44:17.016 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profiles": { 2023-10-26T01:44:17.016 INFO:teuthology.orchestra.run.smithi169.stdout: "default": { 2023-10-26T01:44:17.016 INFO:teuthology.orchestra.run.smithi169.stdout: "crush-failure-domain": "osd", 2023-10-26T01:44:17.017 INFO:teuthology.orchestra.run.smithi169.stdout: "k": "2", 2023-10-26T01:44:17.017 INFO:teuthology.orchestra.run.smithi169.stdout: "m": "1", 2023-10-26T01:44:17.017 INFO:teuthology.orchestra.run.smithi169.stdout: "plugin": "jerasure", 2023-10-26T01:44:17.017 INFO:teuthology.orchestra.run.smithi169.stdout: "technique": "reed_sol_van" 2023-10-26T01:44:17.017 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:17.017 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:17.018 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps_queue": [], 2023-10-26T01:44:17.018 INFO:teuthology.orchestra.run.smithi169.stdout: "new_removed_snaps": [], 2023-10-26T01:44:17.018 INFO:teuthology.orchestra.run.smithi169.stdout: "new_purged_snaps": [], 2023-10-26T01:44:17.018 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_node_flags": {}, 2023-10-26T01:44:17.018 INFO:teuthology.orchestra.run.smithi169.stdout: "device_class_flags": {}, 2023-10-26T01:44:17.019 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_mode": { 2023-10-26T01:44:17.019 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_mode_enabled": false, 2023-10-26T01:44:17.019 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_bucket_count": 0, 2023-10-26T01:44:17.019 INFO:teuthology.orchestra.run.smithi169.stdout: "degraded_stretch_mode": 0, 2023-10-26T01:44:17.019 INFO:teuthology.orchestra.run.smithi169.stdout: "recovering_stretch_mode": 0, 2023-10-26T01:44:17.019 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_mode_bucket": 0 2023-10-26T01:44:17.020 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:17.020 INFO:teuthology.orchestra.run.smithi169.stdout:} 2023-10-26T01:44:17.021 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:16.817+0000 7fabfc516640 1 -- 172.21.15.169:0/4094137766 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fabd806ae40 msgr2=0x7fabd806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:17.022 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:16.818+0000 7fabfc516640 1 --2- 172.21.15.169:0/4094137766 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fabd806ae40 0x7fabd806d300 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fabf00017a0 tx=0x7fabf0001bf0 comp rx=0 tx=0).stop 2023-10-26T01:44:17.022 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:16.818+0000 7fabfc516640 1 -- 172.21.15.169:0/4094137766 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fabf40b1090 msgr2=0x7fabf40b1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:17.022 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:16.818+0000 7fabfc516640 1 --2- 172.21.15.169:0/4094137766 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fabf40b1090 0x7fabf40b1510 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7fabf40b3420 tx=0x7fabec00ef30 comp rx=0 tx=0).stop 2023-10-26T01:44:17.022 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:16.818+0000 7fabfc516640 1 -- 172.21.15.169:0/4094137766 shutdown_connections 2023-10-26T01:44:17.023 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:16.818+0000 7fabfc516640 1 --2- 172.21.15.169:0/4094137766 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fabd806ae40 0x7fabd806d300 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:17.023 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:16.818+0000 7fabfc516640 1 --2- 172.21.15.169:0/4094137766 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fabf40b1090 0x7fabf40b1510 unknown :-1 s=CLOSED pgs=1889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:17.023 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:16.818+0000 7fabfc516640 1 --2- 172.21.15.169:0/4094137766 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fabf41e5030 0x7fabf41e54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:17.023 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:16.818+0000 7fabfc516640 1 --2- 172.21.15.169:0/4094137766 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fabe0003fb0 0x7fabf414b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:17.023 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:16.818+0000 7fabfc516640 1 -- 172.21.15.169:0/4094137766 >> 172.21.15.169:0/4094137766 conn(0x7fabf4069930 msgr2=0x7fabf40aab40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:17.024 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:16.818+0000 7fabfc516640 1 -- 172.21.15.169:0/4094137766 shutdown_connections 2023-10-26T01:44:17.024 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:16.818+0000 7fabfc516640 1 -- 172.21.15.169:0/4094137766 wait complete. 2023-10-26T01:44:17.024 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2023-10-26T01:44:22.025 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-26T01:44:22.256 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:22.253+0000 7fef1a900640 1 -- 172.21.15.169:0/1436654750 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feefc0047c0 msgr2=0x7feefc004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:22.256 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:22.253+0000 7fef1a900640 1 --2- 172.21.15.169:0/1436654750 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feefc0047c0 0x7feefc004c40 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7fef0800ac90 tx=0x7feefc0057f0 comp rx=0 tx=0).stop 2023-10-26T01:44:22.256 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:22.253+0000 7fef18e9d640 1 -- 172.21.15.169:0/1436654750 reap_dead start 2023-10-26T01:44:22.257 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:22.253+0000 7fef1a900640 1 -- 172.21.15.169:0/1436654750 shutdown_connections 2023-10-26T01:44:22.257 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:22.253+0000 7fef1a900640 1 -- 172.21.15.169:0/1436654750 >> 172.21.15.169:0/1436654750 conn(0x7fef14069930 msgr2=0x7fef14069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:22.257 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:22.253+0000 7fef1a900640 1 -- 172.21.15.169:0/1436654750 shutdown_connections 2023-10-26T01:44:22.257 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:22.253+0000 7fef1a900640 1 -- 172.21.15.169:0/1436654750 wait complete. 2023-10-26T01:44:22.258 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:22.255+0000 7fef1a900640 1 Processor -- start 2023-10-26T01:44:22.258 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:22.255+0000 7fef1a900640 1 -- start start 2023-10-26T01:44:22.260 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:22.256+0000 7fef1a900640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feefc0047c0 0x7fef14155bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:22.260 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:22.256+0000 7fef1a900640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fef1409a850 0x7fef1409acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:22.260 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:22.256+0000 7fef1a900640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fef1414c340 0x7fef1414c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:22.260 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:22.256+0000 7fef1a900640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef140afab0 con 0x7feefc0047c0 2023-10-26T01:44:22.261 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:22.256+0000 7fef1a900640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef140aeec0 con 0x7fef1409a850 2023-10-26T01:44:22.261 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:22.256+0000 7fef1a900640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef1414cd00 con 0x7fef1414c340 2023-10-26T01:44:22.261 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:22.256+0000 7fef13fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feefc0047c0 0x7fef14155bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:22.261 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:22.256+0000 7fef13fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feefc0047c0 0x7fef14155bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:40816/0 (socket says 172.21.15.169:40816) 2023-10-26T01:44:22.261 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:22.256+0000 7fef13fff640 1 -- 172.21.15.169:0/3748570732 learned_addr learned my addr 172.21.15.169:0/3748570732 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:44:22.262 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:22.256+0000 7fef13fff640 1 -- 172.21.15.169:0/3748570732 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fef1414c340 msgr2=0x7fef1414c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:22.262 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:22.256+0000 7fef13fff640 1 --2- 172.21.15.169:0/3748570732 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fef1414c340 0x7fef1414c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:22.262 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:22.256+0000 7fef137fe640 1 --2- 172.21.15.169:0/3748570732 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fef1409a850 0x7fef1409acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:22.262 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:22.256+0000 7fef13fff640 1 -- 172.21.15.169:0/3748570732 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fef1409a850 msgr2=0x7fef1409acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:22.262 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:22.256+0000 7fef13fff640 1 --2- 172.21.15.169:0/3748570732 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fef1409a850 0x7fef1409acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:22.263 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:22.256+0000 7fef13fff640 1 -- 172.21.15.169:0/3748570732 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef140af110 con 0x7feefc0047c0 2023-10-26T01:44:22.263 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:22.257+0000 7fef13fff640 1 --2- 172.21.15.169:0/3748570732 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feefc0047c0 0x7fef14155bf0 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7fef0400d720 tx=0x7fef0400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:22.263 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:22.257+0000 7fef10ff9640 1 -- 172.21.15.169:0/3748570732 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef040050d0 con 0x7feefc0047c0 2023-10-26T01:44:22.263 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:22.257+0000 7fef10ff9640 1 -- 172.21.15.169:0/3748570732 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fef04005230 con 0x7feefc0047c0 2023-10-26T01:44:22.263 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:22.257+0000 7fef1a900640 1 -- 172.21.15.169:0/3748570732 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef140af3f0 con 0x7feefc0047c0 2023-10-26T01:44:22.264 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:22.257+0000 7fef10ff9640 1 -- 172.21.15.169:0/3748570732 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef0400b6c0 con 0x7feefc0047c0 2023-10-26T01:44:22.264 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:22.257+0000 7fef1a900640 1 -- 172.21.15.169:0/3748570732 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef14071be0 con 0x7feefc0047c0 2023-10-26T01:44:22.265 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:22.258+0000 7fef1a900640 1 -- 172.21.15.169:0/3748570732 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef1404db10 con 0x7feefc0047c0 2023-10-26T01:44:22.265 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:22.259+0000 7fef10ff9640 1 -- 172.21.15.169:0/3748570732 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fef0400b820 con 0x7feefc0047c0 2023-10-26T01:44:22.265 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:22.260+0000 7fef10ff9640 1 --2- 172.21.15.169:0/3748570732 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fef0006ae40 0x7fef0006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:22.265 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:22.260+0000 7fef10ff9640 1 -- 172.21.15.169:0/3748570732 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fef04088d40 con 0x7feefc0047c0 2023-10-26T01:44:22.266 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:22.260+0000 7fef10ff9640 1 -- 172.21.15.169:0/3748570732 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=185}) v3 -- 0x7fef00072970 con 0x7feefc0047c0 2023-10-26T01:44:22.266 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:22.261+0000 7fef137fe640 1 --2- 172.21.15.169:0/3748570732 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fef0006ae40 0x7fef0006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:22.266 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:22.262+0000 7fef137fe640 1 --2- 172.21.15.169:0/3748570732 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fef0006ae40 0x7fef0006d300 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fef08038000 tx=0x7fef080046c0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:22.267 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:22.265+0000 7fef10ff9640 1 -- 172.21.15.169:0/3748570732 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fef04019020 con 0x7feefc0047c0 2023-10-26T01:44:22.466 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:22.463+0000 7fef1a900640 1 -- 172.21.15.169:0/3748570732 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fef140aad80 con 0x7feefc0047c0 2023-10-26T01:44:22.468 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:22.465+0000 7fef10ff9640 1 -- 172.21.15.169:0/3748570732 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v184) v1 ==== 81+0+34081 (secure 0 0 0) 0x7fef04053ea0 con 0x7feefc0047c0 2023-10-26T01:44:22.469 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:44:22.469 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2023-10-26T01:44:22.469 INFO:teuthology.orchestra.run.smithi169.stdout: "epoch": 184, 2023-10-26T01:44:22.469 INFO:teuthology.orchestra.run.smithi169.stdout: "fsid": "30203b28-ed8a-44e8-a9a6-a9bf25f980f1", 2023-10-26T01:44:22.470 INFO:teuthology.orchestra.run.smithi169.stdout: "created": "2023-10-26T01:30:22.897155+0000", 2023-10-26T01:44:22.470 INFO:teuthology.orchestra.run.smithi169.stdout: "modified": "2023-10-26T01:44:13.194893+0000", 2023-10-26T01:44:22.470 INFO:teuthology.orchestra.run.smithi169.stdout: "last_up_change": "2023-10-26T01:30:31.875281+0000", 2023-10-26T01:44:22.470 INFO:teuthology.orchestra.run.smithi169.stdout: "last_in_change": "2023-10-26T01:30:27.637941+0000", 2023-10-26T01:44:22.471 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-26T01:44:22.471 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_num": 5799936, 2023-10-26T01:44:22.471 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_set": [ 2023-10-26T01:44:22.471 INFO:teuthology.orchestra.run.smithi169.stdout: "pglog_hardlimit", 2023-10-26T01:44:22.471 INFO:teuthology.orchestra.run.smithi169.stdout: "purged_snapdirs", 2023-10-26T01:44:22.472 INFO:teuthology.orchestra.run.smithi169.stdout: "recovery_deletes", 2023-10-26T01:44:22.472 INFO:teuthology.orchestra.run.smithi169.stdout: "sortbitwise" 2023-10-26T01:44:22.472 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:44:22.472 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_version": 4, 2023-10-26T01:44:22.472 INFO:teuthology.orchestra.run.smithi169.stdout: "full_ratio": 0.69999998807907104, 2023-10-26T01:44:22.473 INFO:teuthology.orchestra.run.smithi169.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-26T01:44:22.473 INFO:teuthology.orchestra.run.smithi169.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-26T01:44:22.473 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_snapshot": "", 2023-10-26T01:44:22.473 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_max": 23, 2023-10-26T01:44:22.474 INFO:teuthology.orchestra.run.smithi169.stdout: "max_osd": 8, 2023-10-26T01:44:22.474 INFO:teuthology.orchestra.run.smithi169.stdout: "require_min_compat_client": "luminous", 2023-10-26T01:44:22.474 INFO:teuthology.orchestra.run.smithi169.stdout: "min_compat_client": "jewel", 2023-10-26T01:44:22.474 INFO:teuthology.orchestra.run.smithi169.stdout: "require_osd_release": "reef", 2023-10-26T01:44:22.474 INFO:teuthology.orchestra.run.smithi169.stdout: "allow_crimson": false, 2023-10-26T01:44:22.475 INFO:teuthology.orchestra.run.smithi169.stdout: "pools": [ 2023-10-26T01:44:22.475 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:22.475 INFO:teuthology.orchestra.run.smithi169.stdout: "pool": 1, 2023-10-26T01:44:22.475 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_name": ".mgr", 2023-10-26T01:44:22.475 INFO:teuthology.orchestra.run.smithi169.stdout: "create_time": "2023-10-26T01:30:32.200808+0000", 2023-10-26T01:44:22.476 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": 1, 2023-10-26T01:44:22.476 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_names": "hashpspool", 2023-10-26T01:44:22.476 INFO:teuthology.orchestra.run.smithi169.stdout: "type": 1, 2023-10-26T01:44:22.476 INFO:teuthology.orchestra.run.smithi169.stdout: "size": 2, 2023-10-26T01:44:22.476 INFO:teuthology.orchestra.run.smithi169.stdout: "min_size": 1, 2023-10-26T01:44:22.477 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_rule": 0, 2023-10-26T01:44:22.477 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_count": 0, 2023-10-26T01:44:22.477 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_target": 0, 2023-10-26T01:44:22.477 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T01:44:22.477 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T01:44:22.478 INFO:teuthology.orchestra.run.smithi169.stdout: "object_hash": 2, 2023-10-26T01:44:22.478 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_mode": "off", 2023-10-26T01:44:22.478 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num": 1, 2023-10-26T01:44:22.478 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num": 1, 2023-10-26T01:44:22.478 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num_target": 1, 2023-10-26T01:44:22.479 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_target": 1, 2023-10-26T01:44:22.479 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_pending": 1, 2023-10-26T01:44:22.479 INFO:teuthology.orchestra.run.smithi169.stdout: "last_pg_merge_meta": { 2023-10-26T01:44:22.479 INFO:teuthology.orchestra.run.smithi169.stdout: "source_pgid": "0.0", 2023-10-26T01:44:22.479 INFO:teuthology.orchestra.run.smithi169.stdout: "ready_epoch": 0, 2023-10-26T01:44:22.480 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_started": 0, 2023-10-26T01:44:22.480 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_clean": 0, 2023-10-26T01:44:22.480 INFO:teuthology.orchestra.run.smithi169.stdout: "source_version": "0'0", 2023-10-26T01:44:22.480 INFO:teuthology.orchestra.run.smithi169.stdout: "target_version": "0'0" 2023-10-26T01:44:22.480 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:22.480 INFO:teuthology.orchestra.run.smithi169.stdout: "last_change": "109", 2023-10-26T01:44:22.481 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend": "0", 2023-10-26T01:44:22.481 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T01:44:22.481 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T01:44:22.481 INFO:teuthology.orchestra.run.smithi169.stdout: "auid": 0, 2023-10-26T01:44:22.481 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_mode": "selfmanaged", 2023-10-26T01:44:22.482 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_seq": 0, 2023-10-26T01:44:22.482 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_epoch": 0, 2023-10-26T01:44:22.482 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_snaps": [], 2023-10-26T01:44:22.482 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps": "[]", 2023-10-26T01:44:22.482 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_bytes": 0, 2023-10-26T01:44:22.483 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_objects": 0, 2023-10-26T01:44:22.483 INFO:teuthology.orchestra.run.smithi169.stdout: "tiers": [], 2023-10-26T01:44:22.483 INFO:teuthology.orchestra.run.smithi169.stdout: "tier_of": -1, 2023-10-26T01:44:22.483 INFO:teuthology.orchestra.run.smithi169.stdout: "read_tier": -1, 2023-10-26T01:44:22.483 INFO:teuthology.orchestra.run.smithi169.stdout: "write_tier": -1, 2023-10-26T01:44:22.484 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_mode": "none", 2023-10-26T01:44:22.484 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_bytes": 0, 2023-10-26T01:44:22.484 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_objects": 0, 2023-10-26T01:44:22.484 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T01:44:22.484 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T01:44:22.484 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T01:44:22.485 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_flush_age": 0, 2023-10-26T01:44:22.485 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_evict_age": 0, 2023-10-26T01:44:22.485 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profile": "", 2023-10-26T01:44:22.485 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_params": { 2023-10-26T01:44:22.485 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "none" 2023-10-26T01:44:22.486 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:22.486 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_period": 0, 2023-10-26T01:44:22.486 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_count": 0, 2023-10-26T01:44:22.486 INFO:teuthology.orchestra.run.smithi169.stdout: "use_gmt_hitset": true, 2023-10-26T01:44:22.486 INFO:teuthology.orchestra.run.smithi169.stdout: "min_read_recency_for_promote": 0, 2023-10-26T01:44:22.487 INFO:teuthology.orchestra.run.smithi169.stdout: "min_write_recency_for_promote": 0, 2023-10-26T01:44:22.487 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T01:44:22.487 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_search_last_n": 0, 2023-10-26T01:44:22.487 INFO:teuthology.orchestra.run.smithi169.stdout: "grade_table": [], 2023-10-26T01:44:22.487 INFO:teuthology.orchestra.run.smithi169.stdout: "stripe_width": 0, 2023-10-26T01:44:22.488 INFO:teuthology.orchestra.run.smithi169.stdout: "expected_num_objects": 0, 2023-10-26T01:44:22.488 INFO:teuthology.orchestra.run.smithi169.stdout: "fast_read": false, 2023-10-26T01:44:22.488 INFO:teuthology.orchestra.run.smithi169.stdout: "options": { 2023-10-26T01:44:22.488 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_max": 32, 2023-10-26T01:44:22.488 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_min": 1 2023-10-26T01:44:22.489 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:22.489 INFO:teuthology.orchestra.run.smithi169.stdout: "application_metadata": { 2023-10-26T01:44:22.489 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": {} 2023-10-26T01:44:22.489 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:22.489 INFO:teuthology.orchestra.run.smithi169.stdout: "read_balance": { 2023-10-26T01:44:22.490 INFO:teuthology.orchestra.run.smithi169.stdout: "score_acting": 8, 2023-10-26T01:44:22.490 INFO:teuthology.orchestra.run.smithi169.stdout: "score_stable": 8, 2023-10-26T01:44:22.490 INFO:teuthology.orchestra.run.smithi169.stdout: "optimal_score": 0.25, 2023-10-26T01:44:22.490 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_acting": 2, 2023-10-26T01:44:22.490 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_stable": 2, 2023-10-26T01:44:22.491 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity_weighted": 1, 2023-10-26T01:44:22.491 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity": 1, 2023-10-26T01:44:22.491 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity_weighted": 1 2023-10-26T01:44:22.491 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:22.491 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:22.492 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:22.492 INFO:teuthology.orchestra.run.smithi169.stdout: "pool": 22, 2023-10-26T01:44:22.492 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_name": "cephfs_metadata", 2023-10-26T01:44:22.492 INFO:teuthology.orchestra.run.smithi169.stdout: "create_time": "2023-10-26T01:43:47.501296+0000", 2023-10-26T01:44:22.492 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": 1, 2023-10-26T01:44:22.493 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_names": "hashpspool", 2023-10-26T01:44:22.493 INFO:teuthology.orchestra.run.smithi169.stdout: "type": 1, 2023-10-26T01:44:22.493 INFO:teuthology.orchestra.run.smithi169.stdout: "size": 2, 2023-10-26T01:44:22.493 INFO:teuthology.orchestra.run.smithi169.stdout: "min_size": 1, 2023-10-26T01:44:22.493 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_rule": 0, 2023-10-26T01:44:22.493 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_count": 0, 2023-10-26T01:44:22.494 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_target": 0, 2023-10-26T01:44:22.494 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T01:44:22.494 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T01:44:22.494 INFO:teuthology.orchestra.run.smithi169.stdout: "object_hash": 2, 2023-10-26T01:44:22.494 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_mode": "off", 2023-10-26T01:44:22.495 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num": 32, 2023-10-26T01:44:22.495 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num": 32, 2023-10-26T01:44:22.495 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num_target": 32, 2023-10-26T01:44:22.495 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_target": 32, 2023-10-26T01:44:22.495 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_pending": 32, 2023-10-26T01:44:22.496 INFO:teuthology.orchestra.run.smithi169.stdout: "last_pg_merge_meta": { 2023-10-26T01:44:22.496 INFO:teuthology.orchestra.run.smithi169.stdout: "source_pgid": "0.0", 2023-10-26T01:44:22.496 INFO:teuthology.orchestra.run.smithi169.stdout: "ready_epoch": 0, 2023-10-26T01:44:22.496 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_started": 0, 2023-10-26T01:44:22.496 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_clean": 0, 2023-10-26T01:44:22.497 INFO:teuthology.orchestra.run.smithi169.stdout: "source_version": "0'0", 2023-10-26T01:44:22.497 INFO:teuthology.orchestra.run.smithi169.stdout: "target_version": "0'0" 2023-10-26T01:44:22.497 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:22.497 INFO:teuthology.orchestra.run.smithi169.stdout: "last_change": "181", 2023-10-26T01:44:22.497 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend": "0", 2023-10-26T01:44:22.498 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T01:44:22.498 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T01:44:22.498 INFO:teuthology.orchestra.run.smithi169.stdout: "auid": 0, 2023-10-26T01:44:22.498 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_mode": "selfmanaged", 2023-10-26T01:44:22.498 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_seq": 0, 2023-10-26T01:44:22.498 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_epoch": 0, 2023-10-26T01:44:22.499 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_snaps": [], 2023-10-26T01:44:22.499 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps": "[]", 2023-10-26T01:44:22.499 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_bytes": 0, 2023-10-26T01:44:22.499 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_objects": 0, 2023-10-26T01:44:22.499 INFO:teuthology.orchestra.run.smithi169.stdout: "tiers": [], 2023-10-26T01:44:22.500 INFO:teuthology.orchestra.run.smithi169.stdout: "tier_of": -1, 2023-10-26T01:44:22.500 INFO:teuthology.orchestra.run.smithi169.stdout: "read_tier": -1, 2023-10-26T01:44:22.500 INFO:teuthology.orchestra.run.smithi169.stdout: "write_tier": -1, 2023-10-26T01:44:22.500 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_mode": "none", 2023-10-26T01:44:22.500 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_bytes": 0, 2023-10-26T01:44:22.501 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_objects": 0, 2023-10-26T01:44:22.501 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T01:44:22.501 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T01:44:22.501 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T01:44:22.501 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_flush_age": 0, 2023-10-26T01:44:22.502 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_evict_age": 0, 2023-10-26T01:44:22.502 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profile": "", 2023-10-26T01:44:22.502 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_params": { 2023-10-26T01:44:22.502 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "none" 2023-10-26T01:44:22.502 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:22.503 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_period": 0, 2023-10-26T01:44:22.503 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_count": 0, 2023-10-26T01:44:22.503 INFO:teuthology.orchestra.run.smithi169.stdout: "use_gmt_hitset": true, 2023-10-26T01:44:22.503 INFO:teuthology.orchestra.run.smithi169.stdout: "min_read_recency_for_promote": 0, 2023-10-26T01:44:22.503 INFO:teuthology.orchestra.run.smithi169.stdout: "min_write_recency_for_promote": 0, 2023-10-26T01:44:22.504 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T01:44:22.504 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_search_last_n": 0, 2023-10-26T01:44:22.504 INFO:teuthology.orchestra.run.smithi169.stdout: "grade_table": [], 2023-10-26T01:44:22.504 INFO:teuthology.orchestra.run.smithi169.stdout: "stripe_width": 0, 2023-10-26T01:44:22.504 INFO:teuthology.orchestra.run.smithi169.stdout: "expected_num_objects": 0, 2023-10-26T01:44:22.505 INFO:teuthology.orchestra.run.smithi169.stdout: "fast_read": false, 2023-10-26T01:44:22.505 INFO:teuthology.orchestra.run.smithi169.stdout: "options": { 2023-10-26T01:44:22.505 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_bias": 4, 2023-10-26T01:44:22.505 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_min": 16, 2023-10-26T01:44:22.505 INFO:teuthology.orchestra.run.smithi169.stdout: "recovery_priority": 5 2023-10-26T01:44:22.505 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:22.506 INFO:teuthology.orchestra.run.smithi169.stdout: "application_metadata": { 2023-10-26T01:44:22.506 INFO:teuthology.orchestra.run.smithi169.stdout: "cephfs": { 2023-10-26T01:44:22.506 INFO:teuthology.orchestra.run.smithi169.stdout: "metadata": "cephfs" 2023-10-26T01:44:22.506 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:22.506 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:22.507 INFO:teuthology.orchestra.run.smithi169.stdout: "read_balance": { 2023-10-26T01:44:22.507 INFO:teuthology.orchestra.run.smithi169.stdout: "score_acting": 2, 2023-10-26T01:44:22.507 INFO:teuthology.orchestra.run.smithi169.stdout: "score_stable": 2, 2023-10-26T01:44:22.507 INFO:teuthology.orchestra.run.smithi169.stdout: "optimal_score": 1, 2023-10-26T01:44:22.507 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_acting": 2, 2023-10-26T01:44:22.508 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_stable": 2, 2023-10-26T01:44:22.508 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity_weighted": 1, 2023-10-26T01:44:22.508 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity": 1, 2023-10-26T01:44:22.508 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity_weighted": 1 2023-10-26T01:44:22.508 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:22.509 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:22.509 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:22.509 INFO:teuthology.orchestra.run.smithi169.stdout: "pool": 23, 2023-10-26T01:44:22.509 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_name": "cephfs_data", 2023-10-26T01:44:22.509 INFO:teuthology.orchestra.run.smithi169.stdout: "create_time": "2023-10-26T01:43:48.737507+0000", 2023-10-26T01:44:22.509 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": 1027, 2023-10-26T01:44:22.510 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_names": "hashpspool,full,full_quota", 2023-10-26T01:44:22.510 INFO:teuthology.orchestra.run.smithi169.stdout: "type": 1, 2023-10-26T01:44:22.510 INFO:teuthology.orchestra.run.smithi169.stdout: "size": 2, 2023-10-26T01:44:22.510 INFO:teuthology.orchestra.run.smithi169.stdout: "min_size": 1, 2023-10-26T01:44:22.510 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_rule": 0, 2023-10-26T01:44:22.511 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_count": 0, 2023-10-26T01:44:22.511 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_target": 0, 2023-10-26T01:44:22.511 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T01:44:22.511 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T01:44:22.511 INFO:teuthology.orchestra.run.smithi169.stdout: "object_hash": 2, 2023-10-26T01:44:22.511 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_mode": "off", 2023-10-26T01:44:22.512 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num": 64, 2023-10-26T01:44:22.512 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num": 64, 2023-10-26T01:44:22.512 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num_target": 64, 2023-10-26T01:44:22.512 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_target": 64, 2023-10-26T01:44:22.512 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_pending": 64, 2023-10-26T01:44:22.513 INFO:teuthology.orchestra.run.smithi169.stdout: "last_pg_merge_meta": { 2023-10-26T01:44:22.513 INFO:teuthology.orchestra.run.smithi169.stdout: "source_pgid": "0.0", 2023-10-26T01:44:22.513 INFO:teuthology.orchestra.run.smithi169.stdout: "ready_epoch": 0, 2023-10-26T01:44:22.513 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_started": 0, 2023-10-26T01:44:22.513 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_clean": 0, 2023-10-26T01:44:22.514 INFO:teuthology.orchestra.run.smithi169.stdout: "source_version": "0'0", 2023-10-26T01:44:22.514 INFO:teuthology.orchestra.run.smithi169.stdout: "target_version": "0'0" 2023-10-26T01:44:22.514 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:22.514 INFO:teuthology.orchestra.run.smithi169.stdout: "last_change": "184", 2023-10-26T01:44:22.514 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend": "0", 2023-10-26T01:44:22.515 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T01:44:22.515 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T01:44:22.515 INFO:teuthology.orchestra.run.smithi169.stdout: "auid": 0, 2023-10-26T01:44:22.515 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_mode": "selfmanaged", 2023-10-26T01:44:22.515 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_seq": 0, 2023-10-26T01:44:22.516 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_epoch": 0, 2023-10-26T01:44:22.516 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_snaps": [], 2023-10-26T01:44:22.516 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps": "[]", 2023-10-26T01:44:22.516 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_bytes": 33554432, 2023-10-26T01:44:22.516 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_objects": 0, 2023-10-26T01:44:22.517 INFO:teuthology.orchestra.run.smithi169.stdout: "tiers": [], 2023-10-26T01:44:22.517 INFO:teuthology.orchestra.run.smithi169.stdout: "tier_of": -1, 2023-10-26T01:44:22.517 INFO:teuthology.orchestra.run.smithi169.stdout: "read_tier": -1, 2023-10-26T01:44:22.517 INFO:teuthology.orchestra.run.smithi169.stdout: "write_tier": -1, 2023-10-26T01:44:22.517 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_mode": "none", 2023-10-26T01:44:22.517 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_bytes": 0, 2023-10-26T01:44:22.518 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_objects": 0, 2023-10-26T01:44:22.518 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T01:44:22.518 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T01:44:22.518 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T01:44:22.518 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_flush_age": 0, 2023-10-26T01:44:22.519 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_evict_age": 0, 2023-10-26T01:44:22.519 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profile": "", 2023-10-26T01:44:22.519 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_params": { 2023-10-26T01:44:22.519 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "none" 2023-10-26T01:44:22.519 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:22.519 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_period": 0, 2023-10-26T01:44:22.520 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_count": 0, 2023-10-26T01:44:22.520 INFO:teuthology.orchestra.run.smithi169.stdout: "use_gmt_hitset": true, 2023-10-26T01:44:22.520 INFO:teuthology.orchestra.run.smithi169.stdout: "min_read_recency_for_promote": 0, 2023-10-26T01:44:22.520 INFO:teuthology.orchestra.run.smithi169.stdout: "min_write_recency_for_promote": 0, 2023-10-26T01:44:22.520 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T01:44:22.521 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_search_last_n": 0, 2023-10-26T01:44:22.521 INFO:teuthology.orchestra.run.smithi169.stdout: "grade_table": [], 2023-10-26T01:44:22.521 INFO:teuthology.orchestra.run.smithi169.stdout: "stripe_width": 0, 2023-10-26T01:44:22.521 INFO:teuthology.orchestra.run.smithi169.stdout: "expected_num_objects": 0, 2023-10-26T01:44:22.521 INFO:teuthology.orchestra.run.smithi169.stdout: "fast_read": false, 2023-10-26T01:44:22.522 INFO:teuthology.orchestra.run.smithi169.stdout: "options": { 2023-10-26T01:44:22.522 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_min": 64, 2023-10-26T01:44:22.522 INFO:teuthology.orchestra.run.smithi169.stdout: "target_size_ratio": 0.89999997615814209 2023-10-26T01:44:22.522 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:22.522 INFO:teuthology.orchestra.run.smithi169.stdout: "application_metadata": { 2023-10-26T01:44:22.522 INFO:teuthology.orchestra.run.smithi169.stdout: "cephfs": { 2023-10-26T01:44:22.523 INFO:teuthology.orchestra.run.smithi169.stdout: "data": "cephfs" 2023-10-26T01:44:22.523 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:22.523 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:22.523 INFO:teuthology.orchestra.run.smithi169.stdout: "read_balance": { 2023-10-26T01:44:22.523 INFO:teuthology.orchestra.run.smithi169.stdout: "score_acting": 1.3799999952316284, 2023-10-26T01:44:22.524 INFO:teuthology.orchestra.run.smithi169.stdout: "score_stable": 1.3799999952316284, 2023-10-26T01:44:22.524 INFO:teuthology.orchestra.run.smithi169.stdout: "optimal_score": 1, 2023-10-26T01:44:22.524 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_acting": 1.3799999952316284, 2023-10-26T01:44:22.524 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_stable": 1.3799999952316284, 2023-10-26T01:44:22.524 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity_weighted": 1, 2023-10-26T01:44:22.525 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity": 1, 2023-10-26T01:44:22.525 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity_weighted": 1 2023-10-26T01:44:22.525 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:22.525 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:22.525 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:44:22.526 INFO:teuthology.orchestra.run.smithi169.stdout: "osds": [ 2023-10-26T01:44:22.526 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:22.526 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 0, 2023-10-26T01:44:22.526 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:44:22.526 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:44:22.526 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:44:22.527 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:44:22.527 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:44:22.527 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:44:22.527 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:44:22.527 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:44:22.528 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 180, 2023-10-26T01:44:22.528 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:44:22.528 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:44:22.528 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:44:22.528 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:22.529 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:22.529 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:22.529 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6816", 2023-10-26T01:44:22.529 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:44:22.529 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:22.529 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:22.530 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:22.530 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6817", 2023-10-26T01:44:22.530 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:44:22.530 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:22.530 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:22.531 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:22.531 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:44:22.531 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:22.531 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:22.531 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:22.532 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6818", 2023-10-26T01:44:22.532 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:44:22.532 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:22.532 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:22.532 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:22.533 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6819", 2023-10-26T01:44:22.533 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:44:22.533 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:22.533 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:22.533 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:22.534 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:44:22.534 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:22.534 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:22.534 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:22.534 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6822", 2023-10-26T01:44:22.534 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:44:22.535 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:22.535 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:22.535 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:22.535 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6823", 2023-10-26T01:44:22.535 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:44:22.536 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:22.536 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:22.536 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:22.536 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:44:22.536 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:22.536 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:22.537 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:22.537 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6820", 2023-10-26T01:44:22.537 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:44:22.537 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:22.537 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:22.538 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:22.538 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6821", 2023-10-26T01:44:22.538 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:44:22.538 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:22.538 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:22.538 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:22.539 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6817/4204255864", 2023-10-26T01:44:22.539 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6819/4204255864", 2023-10-26T01:44:22.539 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6823/4204255864", 2023-10-26T01:44:22.539 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6821/4204255864", 2023-10-26T01:44:22.539 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:44:22.540 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:44:22.540 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:44:22.540 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:22.540 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:22.540 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:22.541 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 1, 2023-10-26T01:44:22.541 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:44:22.541 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:44:22.541 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:44:22.541 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:44:22.542 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:44:22.542 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:44:22.542 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:44:22.542 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:44:22.542 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 180, 2023-10-26T01:44:22.543 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:44:22.543 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:44:22.543 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:44:22.543 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:22.543 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:22.543 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:22.544 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6808", 2023-10-26T01:44:22.544 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:44:22.544 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:22.544 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:22.544 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:22.545 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6809", 2023-10-26T01:44:22.545 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:44:22.545 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:22.545 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:22.545 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:22.545 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:44:22.546 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:22.546 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:22.546 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:22.546 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6810", 2023-10-26T01:44:22.546 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:44:22.547 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:22.547 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:22.547 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:22.547 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6811", 2023-10-26T01:44:22.547 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:44:22.548 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:22.548 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:22.548 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:22.548 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:44:22.548 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:22.548 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:22.549 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:22.549 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6814", 2023-10-26T01:44:22.549 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:44:22.549 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:22.549 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:22.550 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:22.550 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6815", 2023-10-26T01:44:22.550 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:44:22.550 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:22.550 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:22.551 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:22.551 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:44:22.551 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:22.551 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:22.551 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:22.551 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6812", 2023-10-26T01:44:22.552 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:44:22.552 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:22.552 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:22.552 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:22.552 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6813", 2023-10-26T01:44:22.553 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:44:22.553 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:22.553 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:22.553 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:22.553 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6809/238206046", 2023-10-26T01:44:22.553 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6811/238206046", 2023-10-26T01:44:22.554 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6815/238206046", 2023-10-26T01:44:22.554 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6813/238206046", 2023-10-26T01:44:22.554 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:44:22.554 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:44:22.554 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:44:22.555 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:22.555 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:22.555 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:22.555 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 2, 2023-10-26T01:44:22.555 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:44:22.556 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:44:22.556 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:44:22.556 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:44:22.556 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:44:22.556 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:44:22.556 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:44:22.557 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:44:22.557 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 180, 2023-10-26T01:44:22.557 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:44:22.557 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:44:22.557 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:44:22.558 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:22.558 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:22.558 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:22.558 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6800", 2023-10-26T01:44:22.558 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:44:22.558 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:22.559 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:22.559 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:22.559 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6801", 2023-10-26T01:44:22.559 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:44:22.559 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:22.560 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:22.560 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:22.560 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:44:22.560 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:22.560 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:22.560 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:22.561 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6802", 2023-10-26T01:44:22.561 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:44:22.561 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:22.561 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:22.561 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:22.562 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6803", 2023-10-26T01:44:22.562 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:44:22.562 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:22.562 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:22.562 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:22.563 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:44:22.563 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:22.563 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:22.563 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:22.563 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6806", 2023-10-26T01:44:22.564 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:44:22.564 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:22.564 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:22.564 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:22.564 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6807", 2023-10-26T01:44:22.565 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:44:22.565 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:22.565 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:22.565 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:22.565 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:44:22.565 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:22.566 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:22.566 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:22.566 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6804", 2023-10-26T01:44:22.566 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:44:22.566 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:22.567 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:22.567 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:22.567 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6805", 2023-10-26T01:44:22.567 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:44:22.567 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:22.568 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:22.568 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:22.568 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6801/3215791846", 2023-10-26T01:44:22.568 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6803/3215791846", 2023-10-26T01:44:22.568 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6807/3215791846", 2023-10-26T01:44:22.569 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6805/3215791846", 2023-10-26T01:44:22.569 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:44:22.569 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:44:22.569 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:44:22.569 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:22.569 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:22.569 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:22.570 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 3, 2023-10-26T01:44:22.570 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:44:22.570 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:44:22.570 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:44:22.570 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:44:22.571 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:44:22.571 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:44:22.571 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:44:22.571 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 14, 2023-10-26T01:44:22.571 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 180, 2023-10-26T01:44:22.571 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:44:22.572 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:44:22.572 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:44:22.572 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:22.572 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:22.572 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:22.573 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6824", 2023-10-26T01:44:22.573 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:44:22.573 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:22.573 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:22.573 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:22.574 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6825", 2023-10-26T01:44:22.574 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:44:22.574 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:22.574 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:22.574 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:22.574 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:44:22.575 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:22.575 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:22.575 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:22.575 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6826", 2023-10-26T01:44:22.575 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:44:22.576 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:22.576 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:22.576 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:22.576 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6827", 2023-10-26T01:44:22.576 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:44:22.576 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:22.577 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:22.577 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:22.577 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:44:22.577 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:22.577 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:22.578 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:22.578 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6830", 2023-10-26T01:44:22.578 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:44:22.578 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:22.578 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:22.579 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:22.579 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6831", 2023-10-26T01:44:22.579 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:44:22.579 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:22.579 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:22.579 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:22.580 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:44:22.580 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:22.580 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:22.580 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:22.580 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6828", 2023-10-26T01:44:22.581 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:44:22.581 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:22.581 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:22.581 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:22.581 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6829", 2023-10-26T01:44:22.582 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:44:22.582 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:22.582 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:22.582 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:22.582 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6825/2256701226", 2023-10-26T01:44:22.583 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6827/2256701226", 2023-10-26T01:44:22.583 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6831/2256701226", 2023-10-26T01:44:22.583 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6829/2256701226", 2023-10-26T01:44:22.583 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:44:22.583 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:44:22.584 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:44:22.584 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:22.584 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:22.584 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:22.584 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 4, 2023-10-26T01:44:22.585 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:44:22.585 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:44:22.585 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:44:22.585 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:44:22.585 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:44:22.586 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:44:22.586 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:44:22.586 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:44:22.586 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 180, 2023-10-26T01:44:22.586 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:44:22.587 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:44:22.587 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:44:22.587 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:22.587 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:22.587 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:22.587 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6816", 2023-10-26T01:44:22.588 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:44:22.588 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:22.588 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:22.588 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:22.588 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6817", 2023-10-26T01:44:22.589 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:44:22.589 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:22.589 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:22.589 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:22.589 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:44:22.589 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:22.590 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:22.590 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:22.590 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6818", 2023-10-26T01:44:22.590 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:44:22.590 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:22.591 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:22.591 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:22.591 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6819", 2023-10-26T01:44:22.591 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:44:22.591 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:22.592 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:22.592 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:22.592 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:44:22.592 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:22.592 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:22.592 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:22.593 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6825", 2023-10-26T01:44:22.593 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:44:22.593 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:22.593 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:22.593 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:22.594 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6828", 2023-10-26T01:44:22.594 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:44:22.594 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:22.594 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:22.594 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:22.595 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:44:22.595 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:22.595 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:22.595 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:22.595 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6820", 2023-10-26T01:44:22.596 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:44:22.596 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:22.596 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:22.596 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:22.596 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6823", 2023-10-26T01:44:22.597 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:44:22.597 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:22.597 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:22.597 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:22.597 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6817/1082279874", 2023-10-26T01:44:22.598 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6819/1082279874", 2023-10-26T01:44:22.598 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6828/1082279874", 2023-10-26T01:44:22.598 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6823/1082279874", 2023-10-26T01:44:22.598 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:44:22.598 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:44:22.599 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:44:22.599 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:22.599 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:22.599 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:22.599 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 5, 2023-10-26T01:44:22.599 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:44:22.600 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:44:22.600 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:44:22.600 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:44:22.600 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:44:22.600 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:44:22.601 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:44:22.601 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:44:22.601 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 180, 2023-10-26T01:44:22.601 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:44:22.601 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:44:22.601 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:44:22.602 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:22.602 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:22.602 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:22.602 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6808", 2023-10-26T01:44:22.602 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:44:22.603 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:22.603 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:22.603 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:22.603 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6809", 2023-10-26T01:44:22.603 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:44:22.604 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:22.604 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:22.604 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:22.604 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:44:22.604 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:22.604 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:22.605 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:22.605 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6810", 2023-10-26T01:44:22.605 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:44:22.605 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:22.605 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:22.606 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:22.606 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6811", 2023-10-26T01:44:22.606 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:44:22.606 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:22.606 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:22.606 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:22.607 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:44:22.607 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:22.607 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:22.607 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:22.607 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6814", 2023-10-26T01:44:22.607 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:44:22.608 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:22.608 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:22.608 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:22.608 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6815", 2023-10-26T01:44:22.609 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:44:22.609 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:22.609 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:22.609 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:22.609 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:44:22.610 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:22.610 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:22.610 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:22.610 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6812", 2023-10-26T01:44:22.610 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:44:22.610 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:22.611 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:22.611 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:22.611 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6813", 2023-10-26T01:44:22.611 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:44:22.611 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:22.612 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:22.612 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:22.612 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6809/3335654351", 2023-10-26T01:44:22.612 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6811/3335654351", 2023-10-26T01:44:22.612 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6815/3335654351", 2023-10-26T01:44:22.612 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6813/3335654351", 2023-10-26T01:44:22.613 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:44:22.613 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:44:22.613 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:44:22.613 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:22.613 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:22.614 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:22.614 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 6, 2023-10-26T01:44:22.614 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:44:22.614 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:44:22.614 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:44:22.614 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:44:22.615 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:44:22.615 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:44:22.615 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:44:22.615 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:44:22.615 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 180, 2023-10-26T01:44:22.616 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:44:22.616 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:44:22.616 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:44:22.616 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:22.616 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:22.617 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:22.617 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6800", 2023-10-26T01:44:22.617 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:44:22.617 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:22.617 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:22.618 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:22.618 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6801", 2023-10-26T01:44:22.618 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:44:22.618 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:22.618 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:22.618 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:22.619 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:44:22.619 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:22.619 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:22.619 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:22.619 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6802", 2023-10-26T01:44:22.620 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:44:22.620 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:22.620 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:22.620 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:22.620 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6803", 2023-10-26T01:44:22.620 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:44:22.621 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:22.621 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:22.621 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:22.621 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:44:22.621 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:22.622 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:22.622 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:22.622 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6806", 2023-10-26T01:44:22.622 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:44:22.622 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:22.623 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:22.623 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:22.623 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6807", 2023-10-26T01:44:22.623 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:44:22.623 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:22.624 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:22.624 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:22.624 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:44:22.624 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:22.624 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:22.624 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:22.625 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6804", 2023-10-26T01:44:22.625 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:44:22.625 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:22.625 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:22.625 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:22.626 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6805", 2023-10-26T01:44:22.626 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:44:22.626 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:22.626 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:22.626 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:22.627 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6801/2520460697", 2023-10-26T01:44:22.627 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6803/2520460697", 2023-10-26T01:44:22.627 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6807/2520460697", 2023-10-26T01:44:22.627 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6805/2520460697", 2023-10-26T01:44:22.627 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:44:22.628 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:44:22.628 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:44:22.628 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:22.628 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:22.628 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:22.629 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 7, 2023-10-26T01:44:22.629 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:44:22.629 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:44:22.629 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:44:22.629 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:44:22.629 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:44:22.630 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:44:22.630 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:44:22.630 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:44:22.630 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 180, 2023-10-26T01:44:22.631 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:44:22.631 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:44:22.631 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:44:22.631 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:22.631 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:22.631 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:22.632 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6821", 2023-10-26T01:44:22.632 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:44:22.632 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:22.632 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:22.632 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:22.633 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6822", 2023-10-26T01:44:22.633 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:44:22.633 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:22.633 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:22.633 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:22.633 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:44:22.634 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:22.634 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:22.634 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:22.634 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6824", 2023-10-26T01:44:22.634 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:44:22.635 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:22.635 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:22.635 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:22.635 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6826", 2023-10-26T01:44:22.635 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:44:22.636 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:22.636 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:22.636 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:22.636 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:44:22.636 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:22.637 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:22.637 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:22.637 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6830", 2023-10-26T01:44:22.637 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:44:22.637 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:22.637 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:22.638 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:22.638 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6831", 2023-10-26T01:44:22.638 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:44:22.638 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:22.638 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:22.639 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:22.639 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:44:22.639 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:22.639 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:22.639 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:22.640 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6827", 2023-10-26T01:44:22.640 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:44:22.640 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:22.640 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:22.640 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:22.640 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6829", 2023-10-26T01:44:22.641 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:44:22.641 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:22.641 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:22.641 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:22.641 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6822/1115930619", 2023-10-26T01:44:22.642 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6826/1115930619", 2023-10-26T01:44:22.642 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6831/1115930619", 2023-10-26T01:44:22.642 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6829/1115930619", 2023-10-26T01:44:22.642 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:44:22.642 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:44:22.643 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:44:22.643 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:22.643 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:22.643 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:44:22.643 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_xinfo": [ 2023-10-26T01:44:22.644 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:22.644 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 0, 2023-10-26T01:44:22.644 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:44:22.644 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:44:22.644 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:44:22.645 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:44:22.645 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:44:22.645 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.889675+0000", 2023-10-26T01:44:22.645 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:44:22.645 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:22.645 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:22.646 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 1, 2023-10-26T01:44:22.646 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:44:22.646 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:44:22.646 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:44:22.646 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:44:22.647 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:44:22.647 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.864157+0000", 2023-10-26T01:44:22.647 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:44:22.647 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:22.647 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:22.648 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 2, 2023-10-26T01:44:22.648 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:44:22.648 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:44:22.648 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:44:22.648 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:44:22.649 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:44:22.649 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.876919+0000", 2023-10-26T01:44:22.649 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:44:22.649 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:22.649 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:22.649 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 3, 2023-10-26T01:44:22.650 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:44:22.650 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:44:22.650 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:44:22.650 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:44:22.650 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:44:22.651 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.974541+0000", 2023-10-26T01:44:22.651 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:44:22.651 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:22.651 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:22.651 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 4, 2023-10-26T01:44:22.652 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:44:22.652 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:44:22.652 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:44:22.652 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:44:22.652 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:44:22.652 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.969696+0000", 2023-10-26T01:44:22.653 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:44:22.653 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:22.653 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:22.653 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 5, 2023-10-26T01:44:22.653 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:44:22.654 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:44:22.654 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:44:22.654 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:44:22.654 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:44:22.654 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.859703+0000", 2023-10-26T01:44:22.655 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:44:22.655 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:22.655 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:22.655 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 6, 2023-10-26T01:44:22.655 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:44:22.656 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:44:22.656 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:44:22.656 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:44:22.656 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:44:22.656 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.867917+0000", 2023-10-26T01:44:22.656 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:44:22.657 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:22.657 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:22.657 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 7, 2023-10-26T01:44:22.657 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:44:22.657 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:44:22.658 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:44:22.658 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:44:22.658 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:44:22.658 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.898605+0000", 2023-10-26T01:44:22.658 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:44:22.659 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:22.659 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:44:22.659 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_upmap": [], 2023-10-26T01:44:22.659 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_upmap_items": [], 2023-10-26T01:44:22.659 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_upmap_primaries": [], 2023-10-26T01:44:22.659 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_temp": [], 2023-10-26T01:44:22.660 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_temp": [], 2023-10-26T01:44:22.660 INFO:teuthology.orchestra.run.smithi169.stdout: "blocklist": {}, 2023-10-26T01:44:22.660 INFO:teuthology.orchestra.run.smithi169.stdout: "range_blocklist": {}, 2023-10-26T01:44:22.660 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profiles": { 2023-10-26T01:44:22.660 INFO:teuthology.orchestra.run.smithi169.stdout: "default": { 2023-10-26T01:44:22.661 INFO:teuthology.orchestra.run.smithi169.stdout: "crush-failure-domain": "osd", 2023-10-26T01:44:22.661 INFO:teuthology.orchestra.run.smithi169.stdout: "k": "2", 2023-10-26T01:44:22.661 INFO:teuthology.orchestra.run.smithi169.stdout: "m": "1", 2023-10-26T01:44:22.661 INFO:teuthology.orchestra.run.smithi169.stdout: "plugin": "jerasure", 2023-10-26T01:44:22.661 INFO:teuthology.orchestra.run.smithi169.stdout: "technique": "reed_sol_van" 2023-10-26T01:44:22.661 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:22.662 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:22.662 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps_queue": [], 2023-10-26T01:44:22.662 INFO:teuthology.orchestra.run.smithi169.stdout: "new_removed_snaps": [], 2023-10-26T01:44:22.662 INFO:teuthology.orchestra.run.smithi169.stdout: "new_purged_snaps": [], 2023-10-26T01:44:22.662 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_node_flags": {}, 2023-10-26T01:44:22.663 INFO:teuthology.orchestra.run.smithi169.stdout: "device_class_flags": {}, 2023-10-26T01:44:22.663 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_mode": { 2023-10-26T01:44:22.663 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_mode_enabled": false, 2023-10-26T01:44:22.663 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_bucket_count": 0, 2023-10-26T01:44:22.663 INFO:teuthology.orchestra.run.smithi169.stdout: "degraded_stretch_mode": 0, 2023-10-26T01:44:22.664 INFO:teuthology.orchestra.run.smithi169.stdout: "recovering_stretch_mode": 0, 2023-10-26T01:44:22.664 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_mode_bucket": 0 2023-10-26T01:44:22.664 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:22.664 INFO:teuthology.orchestra.run.smithi169.stdout:} 2023-10-26T01:44:22.666 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:22.468+0000 7fef1a900640 1 -- 172.21.15.169:0/3748570732 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fef0006ae40 msgr2=0x7fef0006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:22.666 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:22.468+0000 7fef1a900640 1 --2- 172.21.15.169:0/3748570732 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fef0006ae40 0x7fef0006d300 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fef08038000 tx=0x7fef080046c0 comp rx=0 tx=0).stop 2023-10-26T01:44:22.666 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:22.468+0000 7fef1a900640 1 -- 172.21.15.169:0/3748570732 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feefc0047c0 msgr2=0x7fef14155bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:22.666 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:22.468+0000 7fef1a900640 1 --2- 172.21.15.169:0/3748570732 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feefc0047c0 0x7fef14155bf0 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7fef0400d720 tx=0x7fef0400dbe0 comp rx=0 tx=0).stop 2023-10-26T01:44:22.667 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:22.468+0000 7fef1a900640 1 -- 172.21.15.169:0/3748570732 shutdown_connections 2023-10-26T01:44:22.667 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:22.468+0000 7fef1a900640 1 --2- 172.21.15.169:0/3748570732 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fef0006ae40 0x7fef0006d300 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:22.667 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:22.468+0000 7fef1a900640 1 --2- 172.21.15.169:0/3748570732 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fef1414c340 0x7fef1414c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:22.667 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:22.468+0000 7fef1a900640 1 --2- 172.21.15.169:0/3748570732 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fef1409a850 0x7fef1409acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:22.667 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:22.468+0000 7fef1a900640 1 --2- 172.21.15.169:0/3748570732 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feefc0047c0 0x7fef14155bf0 unknown :-1 s=CLOSED pgs=1892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:22.668 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:22.468+0000 7fef1a900640 1 -- 172.21.15.169:0/3748570732 >> 172.21.15.169:0/3748570732 conn(0x7fef14069930 msgr2=0x7fef14157400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:22.668 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:22.468+0000 7fef1a900640 1 -- 172.21.15.169:0/3748570732 shutdown_connections 2023-10-26T01:44:22.668 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:22.468+0000 7fef1a900640 1 -- 172.21.15.169:0/3748570732 wait complete. 2023-10-26T01:44:22.669 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2023-10-26T01:44:27.670 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-26T01:44:27.900 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:27.897+0000 7f269f90c640 1 -- 172.21.15.169:0/3672551248 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26840047c0 msgr2=0x7f2684004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:27.900 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:27.897+0000 7f269f90c640 1 --2- 172.21.15.169:0/3672551248 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26840047c0 0x7f2684004c40 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7f2688009b20 tx=0x7f26840057f0 comp rx=0 tx=0).stop 2023-10-26T01:44:27.900 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:27.897+0000 7f269dea9640 1 -- 172.21.15.169:0/3672551248 reap_dead start 2023-10-26T01:44:27.901 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:27.897+0000 7f269f90c640 1 -- 172.21.15.169:0/3672551248 shutdown_connections 2023-10-26T01:44:27.901 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:27.897+0000 7f269f90c640 1 -- 172.21.15.169:0/3672551248 >> 172.21.15.169:0/3672551248 conn(0x7f2698069930 msgr2=0x7f2698069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:27.901 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:27.897+0000 7f269f90c640 1 -- 172.21.15.169:0/3672551248 shutdown_connections 2023-10-26T01:44:27.901 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:27.897+0000 7f269f90c640 1 -- 172.21.15.169:0/3672551248 wait complete. 2023-10-26T01:44:27.902 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:27.899+0000 7f269f90c640 1 Processor -- start 2023-10-26T01:44:27.902 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:27.899+0000 7f269f90c640 1 -- start start 2023-10-26T01:44:27.903 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:27.900+0000 7f269f90c640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f26840047c0 0x7f269814f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:27.904 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:27.900+0000 7f269f90c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26980c6560 0x7f26980c93d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:27.904 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:27.900+0000 7f269f90c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f26980c69c0 0x7f26980c9910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:27.904 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:27.900+0000 7f269f90c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26981f5580 con 0x7f26980c6560 2023-10-26T01:44:27.904 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:27.900+0000 7f269f90c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26981f5860 con 0x7f26980c69c0 2023-10-26T01:44:27.905 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:27.900+0000 7f269f90c640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26981f5b40 con 0x7f26840047c0 2023-10-26T01:44:27.905 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:27.900+0000 7f269cea7640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26980c6560 0x7f26980c93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:27.905 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:27.900+0000 7f269cea7640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26980c6560 0x7f26980c93d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:48664/0 (socket says 172.21.15.169:48664) 2023-10-26T01:44:27.905 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:27.900+0000 7f269cea7640 1 -- 172.21.15.169:0/3626523380 learned_addr learned my addr 172.21.15.169:0/3626523380 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:44:27.905 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:27.901+0000 7f269d6a8640 1 --2- 172.21.15.169:0/3626523380 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f26840047c0 0x7f269814f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:27.906 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:27.901+0000 7f269dea9640 1 --2- 172.21.15.169:0/3626523380 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f26980c69c0 0x7f26980c9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:27.906 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:27.901+0000 7f269cea7640 1 -- 172.21.15.169:0/3626523380 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f26840047c0 msgr2=0x7f269814f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:27.906 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:27.901+0000 7f269cea7640 1 --2- 172.21.15.169:0/3626523380 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f26840047c0 0x7f269814f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:27.906 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:27.901+0000 7f269cea7640 1 -- 172.21.15.169:0/3626523380 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f26980c69c0 msgr2=0x7f26980c9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:27.906 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:27.901+0000 7f269cea7640 1 --2- 172.21.15.169:0/3626523380 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f26980c69c0 0x7f26980c9910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:27.907 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:27.901+0000 7f269cea7640 1 -- 172.21.15.169:0/3626523380 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26981f5e20 con 0x7f26980c6560 2023-10-26T01:44:27.907 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:27.901+0000 7f269cea7640 1 --2- 172.21.15.169:0/3626523380 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26980c6560 0x7f26980c93d0 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7f268802cb60 tx=0x7f2688033c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:27.908 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:27.901+0000 7f268dffb640 1 -- 172.21.15.169:0/3626523380 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2688004220 con 0x7f26980c6560 2023-10-26T01:44:27.908 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:27.902+0000 7f268dffb640 1 -- 172.21.15.169:0/3626523380 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2688004380 con 0x7f26980c6560 2023-10-26T01:44:27.908 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:27.902+0000 7f268dffb640 1 -- 172.21.15.169:0/3626523380 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f268803d880 con 0x7f26980c6560 2023-10-26T01:44:27.909 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:27.902+0000 7f269f90c640 1 -- 172.21.15.169:0/3626523380 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26981f6100 con 0x7f26980c6560 2023-10-26T01:44:27.909 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:27.902+0000 7f269f90c640 1 -- 172.21.15.169:0/3626523380 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f269814fcc0 con 0x7f26980c6560 2023-10-26T01:44:27.909 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:27.903+0000 7f269f90c640 1 -- 172.21.15.169:0/3626523380 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f269804db10 con 0x7f26980c6560 2023-10-26T01:44:27.909 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:27.904+0000 7f268dffb640 1 -- 172.21.15.169:0/3626523380 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f268803d9e0 con 0x7f26980c6560 2023-10-26T01:44:27.909 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:27.905+0000 7f268dffb640 1 --2- 172.21.15.169:0/3626523380 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f267c06ae40 0x7f267c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:27.910 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:27.905+0000 7f268dffb640 1 -- 172.21.15.169:0/3626523380 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f26880ac320 con 0x7f26980c6560 2023-10-26T01:44:27.910 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:27.905+0000 7f269d6a8640 1 --2- 172.21.15.169:0/3626523380 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f267c06ae40 0x7f267c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:27.910 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:27.907+0000 7f269d6a8640 1 --2- 172.21.15.169:0/3626523380 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f267c06ae40 0x7f267c06d300 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f26940019f0 tx=0x7f2694001ba0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:27.912 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:27.910+0000 7f268dffb640 1 -- 172.21.15.169:0/3626523380 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f26880773c0 con 0x7f26980c6560 2023-10-26T01:44:28.103 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:28.101+0000 7f269f90c640 1 -- 172.21.15.169:0/3626523380 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f269806cfd0 con 0x7f26980c6560 2023-10-26T01:44:28.107 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:28.104+0000 7f268dffb640 1 -- 172.21.15.169:0/3626523380 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v185) v1 ==== 81+0+34062 (secure 0 0 0) 0x7f268807a9e0 con 0x7f26980c6560 2023-10-26T01:44:28.108 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:44:28.108 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2023-10-26T01:44:28.108 INFO:teuthology.orchestra.run.smithi169.stdout: "epoch": 185, 2023-10-26T01:44:28.109 INFO:teuthology.orchestra.run.smithi169.stdout: "fsid": "30203b28-ed8a-44e8-a9a6-a9bf25f980f1", 2023-10-26T01:44:28.109 INFO:teuthology.orchestra.run.smithi169.stdout: "created": "2023-10-26T01:30:22.897155+0000", 2023-10-26T01:44:28.109 INFO:teuthology.orchestra.run.smithi169.stdout: "modified": "2023-10-26T01:44:23.200104+0000", 2023-10-26T01:44:28.109 INFO:teuthology.orchestra.run.smithi169.stdout: "last_up_change": "2023-10-26T01:30:31.875281+0000", 2023-10-26T01:44:28.110 INFO:teuthology.orchestra.run.smithi169.stdout: "last_in_change": "2023-10-26T01:30:27.637941+0000", 2023-10-26T01:44:28.110 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-26T01:44:28.110 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_num": 5799936, 2023-10-26T01:44:28.110 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_set": [ 2023-10-26T01:44:28.111 INFO:teuthology.orchestra.run.smithi169.stdout: "pglog_hardlimit", 2023-10-26T01:44:28.111 INFO:teuthology.orchestra.run.smithi169.stdout: "purged_snapdirs", 2023-10-26T01:44:28.111 INFO:teuthology.orchestra.run.smithi169.stdout: "recovery_deletes", 2023-10-26T01:44:28.112 INFO:teuthology.orchestra.run.smithi169.stdout: "sortbitwise" 2023-10-26T01:44:28.112 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:44:28.112 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_version": 4, 2023-10-26T01:44:28.112 INFO:teuthology.orchestra.run.smithi169.stdout: "full_ratio": 0.69999998807907104, 2023-10-26T01:44:28.113 INFO:teuthology.orchestra.run.smithi169.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-26T01:44:28.113 INFO:teuthology.orchestra.run.smithi169.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-26T01:44:28.113 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_snapshot": "", 2023-10-26T01:44:28.113 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_max": 23, 2023-10-26T01:44:28.114 INFO:teuthology.orchestra.run.smithi169.stdout: "max_osd": 8, 2023-10-26T01:44:28.114 INFO:teuthology.orchestra.run.smithi169.stdout: "require_min_compat_client": "luminous", 2023-10-26T01:44:28.114 INFO:teuthology.orchestra.run.smithi169.stdout: "min_compat_client": "jewel", 2023-10-26T01:44:28.114 INFO:teuthology.orchestra.run.smithi169.stdout: "require_osd_release": "reef", 2023-10-26T01:44:28.114 INFO:teuthology.orchestra.run.smithi169.stdout: "allow_crimson": false, 2023-10-26T01:44:28.115 INFO:teuthology.orchestra.run.smithi169.stdout: "pools": [ 2023-10-26T01:44:28.115 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.115 INFO:teuthology.orchestra.run.smithi169.stdout: "pool": 1, 2023-10-26T01:44:28.115 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_name": ".mgr", 2023-10-26T01:44:28.115 INFO:teuthology.orchestra.run.smithi169.stdout: "create_time": "2023-10-26T01:30:32.200808+0000", 2023-10-26T01:44:28.116 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": 1, 2023-10-26T01:44:28.116 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_names": "hashpspool", 2023-10-26T01:44:28.116 INFO:teuthology.orchestra.run.smithi169.stdout: "type": 1, 2023-10-26T01:44:28.116 INFO:teuthology.orchestra.run.smithi169.stdout: "size": 2, 2023-10-26T01:44:28.116 INFO:teuthology.orchestra.run.smithi169.stdout: "min_size": 1, 2023-10-26T01:44:28.117 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_rule": 0, 2023-10-26T01:44:28.117 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_count": 0, 2023-10-26T01:44:28.117 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_target": 0, 2023-10-26T01:44:28.117 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T01:44:28.117 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T01:44:28.118 INFO:teuthology.orchestra.run.smithi169.stdout: "object_hash": 2, 2023-10-26T01:44:28.118 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_mode": "off", 2023-10-26T01:44:28.118 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num": 1, 2023-10-26T01:44:28.118 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num": 1, 2023-10-26T01:44:28.118 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num_target": 1, 2023-10-26T01:44:28.119 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_target": 1, 2023-10-26T01:44:28.119 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_pending": 1, 2023-10-26T01:44:28.119 INFO:teuthology.orchestra.run.smithi169.stdout: "last_pg_merge_meta": { 2023-10-26T01:44:28.119 INFO:teuthology.orchestra.run.smithi169.stdout: "source_pgid": "0.0", 2023-10-26T01:44:28.119 INFO:teuthology.orchestra.run.smithi169.stdout: "ready_epoch": 0, 2023-10-26T01:44:28.120 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_started": 0, 2023-10-26T01:44:28.120 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_clean": 0, 2023-10-26T01:44:28.120 INFO:teuthology.orchestra.run.smithi169.stdout: "source_version": "0'0", 2023-10-26T01:44:28.120 INFO:teuthology.orchestra.run.smithi169.stdout: "target_version": "0'0" 2023-10-26T01:44:28.120 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.120 INFO:teuthology.orchestra.run.smithi169.stdout: "last_change": "109", 2023-10-26T01:44:28.121 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend": "0", 2023-10-26T01:44:28.121 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T01:44:28.121 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T01:44:28.121 INFO:teuthology.orchestra.run.smithi169.stdout: "auid": 0, 2023-10-26T01:44:28.121 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_mode": "selfmanaged", 2023-10-26T01:44:28.122 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_seq": 0, 2023-10-26T01:44:28.122 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_epoch": 0, 2023-10-26T01:44:28.122 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_snaps": [], 2023-10-26T01:44:28.122 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps": "[]", 2023-10-26T01:44:28.122 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_bytes": 0, 2023-10-26T01:44:28.123 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_objects": 0, 2023-10-26T01:44:28.123 INFO:teuthology.orchestra.run.smithi169.stdout: "tiers": [], 2023-10-26T01:44:28.123 INFO:teuthology.orchestra.run.smithi169.stdout: "tier_of": -1, 2023-10-26T01:44:28.123 INFO:teuthology.orchestra.run.smithi169.stdout: "read_tier": -1, 2023-10-26T01:44:28.123 INFO:teuthology.orchestra.run.smithi169.stdout: "write_tier": -1, 2023-10-26T01:44:28.124 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_mode": "none", 2023-10-26T01:44:28.124 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_bytes": 0, 2023-10-26T01:44:28.124 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_objects": 0, 2023-10-26T01:44:28.124 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T01:44:28.124 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T01:44:28.124 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T01:44:28.125 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_flush_age": 0, 2023-10-26T01:44:28.125 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_evict_age": 0, 2023-10-26T01:44:28.125 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profile": "", 2023-10-26T01:44:28.125 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_params": { 2023-10-26T01:44:28.125 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "none" 2023-10-26T01:44:28.126 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.126 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_period": 0, 2023-10-26T01:44:28.126 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_count": 0, 2023-10-26T01:44:28.126 INFO:teuthology.orchestra.run.smithi169.stdout: "use_gmt_hitset": true, 2023-10-26T01:44:28.126 INFO:teuthology.orchestra.run.smithi169.stdout: "min_read_recency_for_promote": 0, 2023-10-26T01:44:28.127 INFO:teuthology.orchestra.run.smithi169.stdout: "min_write_recency_for_promote": 0, 2023-10-26T01:44:28.127 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T01:44:28.127 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_search_last_n": 0, 2023-10-26T01:44:28.127 INFO:teuthology.orchestra.run.smithi169.stdout: "grade_table": [], 2023-10-26T01:44:28.127 INFO:teuthology.orchestra.run.smithi169.stdout: "stripe_width": 0, 2023-10-26T01:44:28.128 INFO:teuthology.orchestra.run.smithi169.stdout: "expected_num_objects": 0, 2023-10-26T01:44:28.128 INFO:teuthology.orchestra.run.smithi169.stdout: "fast_read": false, 2023-10-26T01:44:28.128 INFO:teuthology.orchestra.run.smithi169.stdout: "options": { 2023-10-26T01:44:28.128 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_max": 32, 2023-10-26T01:44:28.128 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_min": 1 2023-10-26T01:44:28.128 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.129 INFO:teuthology.orchestra.run.smithi169.stdout: "application_metadata": { 2023-10-26T01:44:28.129 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": {} 2023-10-26T01:44:28.129 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.129 INFO:teuthology.orchestra.run.smithi169.stdout: "read_balance": { 2023-10-26T01:44:28.129 INFO:teuthology.orchestra.run.smithi169.stdout: "score_acting": 8, 2023-10-26T01:44:28.130 INFO:teuthology.orchestra.run.smithi169.stdout: "score_stable": 8, 2023-10-26T01:44:28.130 INFO:teuthology.orchestra.run.smithi169.stdout: "optimal_score": 0.25, 2023-10-26T01:44:28.130 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_acting": 2, 2023-10-26T01:44:28.130 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_stable": 2, 2023-10-26T01:44:28.130 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity_weighted": 1, 2023-10-26T01:44:28.131 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity": 1, 2023-10-26T01:44:28.131 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity_weighted": 1 2023-10-26T01:44:28.131 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:28.131 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.131 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.132 INFO:teuthology.orchestra.run.smithi169.stdout: "pool": 22, 2023-10-26T01:44:28.132 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_name": "cephfs_metadata", 2023-10-26T01:44:28.132 INFO:teuthology.orchestra.run.smithi169.stdout: "create_time": "2023-10-26T01:43:47.501296+0000", 2023-10-26T01:44:28.132 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": 1, 2023-10-26T01:44:28.132 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_names": "hashpspool", 2023-10-26T01:44:28.133 INFO:teuthology.orchestra.run.smithi169.stdout: "type": 1, 2023-10-26T01:44:28.133 INFO:teuthology.orchestra.run.smithi169.stdout: "size": 2, 2023-10-26T01:44:28.133 INFO:teuthology.orchestra.run.smithi169.stdout: "min_size": 1, 2023-10-26T01:44:28.133 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_rule": 0, 2023-10-26T01:44:28.133 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_count": 0, 2023-10-26T01:44:28.134 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_target": 0, 2023-10-26T01:44:28.134 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T01:44:28.134 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T01:44:28.134 INFO:teuthology.orchestra.run.smithi169.stdout: "object_hash": 2, 2023-10-26T01:44:28.134 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_mode": "off", 2023-10-26T01:44:28.135 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num": 32, 2023-10-26T01:44:28.135 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num": 32, 2023-10-26T01:44:28.135 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num_target": 32, 2023-10-26T01:44:28.135 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_target": 32, 2023-10-26T01:44:28.135 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_pending": 32, 2023-10-26T01:44:28.136 INFO:teuthology.orchestra.run.smithi169.stdout: "last_pg_merge_meta": { 2023-10-26T01:44:28.136 INFO:teuthology.orchestra.run.smithi169.stdout: "source_pgid": "0.0", 2023-10-26T01:44:28.136 INFO:teuthology.orchestra.run.smithi169.stdout: "ready_epoch": 0, 2023-10-26T01:44:28.136 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_started": 0, 2023-10-26T01:44:28.136 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_clean": 0, 2023-10-26T01:44:28.136 INFO:teuthology.orchestra.run.smithi169.stdout: "source_version": "0'0", 2023-10-26T01:44:28.137 INFO:teuthology.orchestra.run.smithi169.stdout: "target_version": "0'0" 2023-10-26T01:44:28.137 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.137 INFO:teuthology.orchestra.run.smithi169.stdout: "last_change": "181", 2023-10-26T01:44:28.137 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend": "0", 2023-10-26T01:44:28.137 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T01:44:28.138 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T01:44:28.138 INFO:teuthology.orchestra.run.smithi169.stdout: "auid": 0, 2023-10-26T01:44:28.138 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_mode": "selfmanaged", 2023-10-26T01:44:28.138 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_seq": 0, 2023-10-26T01:44:28.138 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_epoch": 0, 2023-10-26T01:44:28.139 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_snaps": [], 2023-10-26T01:44:28.139 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps": "[]", 2023-10-26T01:44:28.139 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_bytes": 0, 2023-10-26T01:44:28.139 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_objects": 0, 2023-10-26T01:44:28.139 INFO:teuthology.orchestra.run.smithi169.stdout: "tiers": [], 2023-10-26T01:44:28.140 INFO:teuthology.orchestra.run.smithi169.stdout: "tier_of": -1, 2023-10-26T01:44:28.140 INFO:teuthology.orchestra.run.smithi169.stdout: "read_tier": -1, 2023-10-26T01:44:28.140 INFO:teuthology.orchestra.run.smithi169.stdout: "write_tier": -1, 2023-10-26T01:44:28.140 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_mode": "none", 2023-10-26T01:44:28.140 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_bytes": 0, 2023-10-26T01:44:28.140 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_objects": 0, 2023-10-26T01:44:28.141 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T01:44:28.141 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T01:44:28.141 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T01:44:28.141 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_flush_age": 0, 2023-10-26T01:44:28.141 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_evict_age": 0, 2023-10-26T01:44:28.142 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profile": "", 2023-10-26T01:44:28.142 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_params": { 2023-10-26T01:44:28.142 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "none" 2023-10-26T01:44:28.142 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.142 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_period": 0, 2023-10-26T01:44:28.143 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_count": 0, 2023-10-26T01:44:28.143 INFO:teuthology.orchestra.run.smithi169.stdout: "use_gmt_hitset": true, 2023-10-26T01:44:28.143 INFO:teuthology.orchestra.run.smithi169.stdout: "min_read_recency_for_promote": 0, 2023-10-26T01:44:28.143 INFO:teuthology.orchestra.run.smithi169.stdout: "min_write_recency_for_promote": 0, 2023-10-26T01:44:28.143 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T01:44:28.144 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_search_last_n": 0, 2023-10-26T01:44:28.144 INFO:teuthology.orchestra.run.smithi169.stdout: "grade_table": [], 2023-10-26T01:44:28.144 INFO:teuthology.orchestra.run.smithi169.stdout: "stripe_width": 0, 2023-10-26T01:44:28.144 INFO:teuthology.orchestra.run.smithi169.stdout: "expected_num_objects": 0, 2023-10-26T01:44:28.144 INFO:teuthology.orchestra.run.smithi169.stdout: "fast_read": false, 2023-10-26T01:44:28.145 INFO:teuthology.orchestra.run.smithi169.stdout: "options": { 2023-10-26T01:44:28.145 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_bias": 4, 2023-10-26T01:44:28.145 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_min": 16, 2023-10-26T01:44:28.145 INFO:teuthology.orchestra.run.smithi169.stdout: "recovery_priority": 5 2023-10-26T01:44:28.145 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.145 INFO:teuthology.orchestra.run.smithi169.stdout: "application_metadata": { 2023-10-26T01:44:28.146 INFO:teuthology.orchestra.run.smithi169.stdout: "cephfs": { 2023-10-26T01:44:28.146 INFO:teuthology.orchestra.run.smithi169.stdout: "metadata": "cephfs" 2023-10-26T01:44:28.146 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:28.146 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.146 INFO:teuthology.orchestra.run.smithi169.stdout: "read_balance": { 2023-10-26T01:44:28.147 INFO:teuthology.orchestra.run.smithi169.stdout: "score_acting": 2, 2023-10-26T01:44:28.147 INFO:teuthology.orchestra.run.smithi169.stdout: "score_stable": 2, 2023-10-26T01:44:28.147 INFO:teuthology.orchestra.run.smithi169.stdout: "optimal_score": 1, 2023-10-26T01:44:28.147 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_acting": 2, 2023-10-26T01:44:28.147 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_stable": 2, 2023-10-26T01:44:28.148 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity_weighted": 1, 2023-10-26T01:44:28.148 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity": 1, 2023-10-26T01:44:28.148 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity_weighted": 1 2023-10-26T01:44:28.148 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:28.148 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.149 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.149 INFO:teuthology.orchestra.run.smithi169.stdout: "pool": 23, 2023-10-26T01:44:28.149 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_name": "cephfs_data", 2023-10-26T01:44:28.149 INFO:teuthology.orchestra.run.smithi169.stdout: "create_time": "2023-10-26T01:43:48.737507+0000", 2023-10-26T01:44:28.149 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": 1, 2023-10-26T01:44:28.150 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_names": "hashpspool", 2023-10-26T01:44:28.150 INFO:teuthology.orchestra.run.smithi169.stdout: "type": 1, 2023-10-26T01:44:28.150 INFO:teuthology.orchestra.run.smithi169.stdout: "size": 2, 2023-10-26T01:44:28.150 INFO:teuthology.orchestra.run.smithi169.stdout: "min_size": 1, 2023-10-26T01:44:28.150 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_rule": 0, 2023-10-26T01:44:28.151 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_count": 0, 2023-10-26T01:44:28.151 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_target": 0, 2023-10-26T01:44:28.151 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T01:44:28.151 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T01:44:28.151 INFO:teuthology.orchestra.run.smithi169.stdout: "object_hash": 2, 2023-10-26T01:44:28.152 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_mode": "off", 2023-10-26T01:44:28.152 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num": 64, 2023-10-26T01:44:28.152 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num": 64, 2023-10-26T01:44:28.152 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num_target": 64, 2023-10-26T01:44:28.152 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_target": 64, 2023-10-26T01:44:28.152 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_pending": 64, 2023-10-26T01:44:28.153 INFO:teuthology.orchestra.run.smithi169.stdout: "last_pg_merge_meta": { 2023-10-26T01:44:28.153 INFO:teuthology.orchestra.run.smithi169.stdout: "source_pgid": "0.0", 2023-10-26T01:44:28.153 INFO:teuthology.orchestra.run.smithi169.stdout: "ready_epoch": 0, 2023-10-26T01:44:28.153 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_started": 0, 2023-10-26T01:44:28.153 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_clean": 0, 2023-10-26T01:44:28.154 INFO:teuthology.orchestra.run.smithi169.stdout: "source_version": "0'0", 2023-10-26T01:44:28.154 INFO:teuthology.orchestra.run.smithi169.stdout: "target_version": "0'0" 2023-10-26T01:44:28.154 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.154 INFO:teuthology.orchestra.run.smithi169.stdout: "last_change": "185", 2023-10-26T01:44:28.154 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend": "0", 2023-10-26T01:44:28.155 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T01:44:28.155 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T01:44:28.155 INFO:teuthology.orchestra.run.smithi169.stdout: "auid": 0, 2023-10-26T01:44:28.155 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_mode": "selfmanaged", 2023-10-26T01:44:28.155 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_seq": 0, 2023-10-26T01:44:28.156 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_epoch": 0, 2023-10-26T01:44:28.156 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_snaps": [], 2023-10-26T01:44:28.156 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps": "[]", 2023-10-26T01:44:28.156 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_bytes": 33554432, 2023-10-26T01:44:28.156 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_objects": 0, 2023-10-26T01:44:28.156 INFO:teuthology.orchestra.run.smithi169.stdout: "tiers": [], 2023-10-26T01:44:28.157 INFO:teuthology.orchestra.run.smithi169.stdout: "tier_of": -1, 2023-10-26T01:44:28.157 INFO:teuthology.orchestra.run.smithi169.stdout: "read_tier": -1, 2023-10-26T01:44:28.157 INFO:teuthology.orchestra.run.smithi169.stdout: "write_tier": -1, 2023-10-26T01:44:28.157 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_mode": "none", 2023-10-26T01:44:28.157 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_bytes": 0, 2023-10-26T01:44:28.158 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_objects": 0, 2023-10-26T01:44:28.158 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T01:44:28.158 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T01:44:28.158 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T01:44:28.159 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_flush_age": 0, 2023-10-26T01:44:28.159 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_evict_age": 0, 2023-10-26T01:44:28.159 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profile": "", 2023-10-26T01:44:28.159 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_params": { 2023-10-26T01:44:28.159 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "none" 2023-10-26T01:44:28.159 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.160 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_period": 0, 2023-10-26T01:44:28.160 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_count": 0, 2023-10-26T01:44:28.160 INFO:teuthology.orchestra.run.smithi169.stdout: "use_gmt_hitset": true, 2023-10-26T01:44:28.160 INFO:teuthology.orchestra.run.smithi169.stdout: "min_read_recency_for_promote": 0, 2023-10-26T01:44:28.160 INFO:teuthology.orchestra.run.smithi169.stdout: "min_write_recency_for_promote": 0, 2023-10-26T01:44:28.161 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T01:44:28.161 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_search_last_n": 0, 2023-10-26T01:44:28.161 INFO:teuthology.orchestra.run.smithi169.stdout: "grade_table": [], 2023-10-26T01:44:28.161 INFO:teuthology.orchestra.run.smithi169.stdout: "stripe_width": 0, 2023-10-26T01:44:28.161 INFO:teuthology.orchestra.run.smithi169.stdout: "expected_num_objects": 0, 2023-10-26T01:44:28.162 INFO:teuthology.orchestra.run.smithi169.stdout: "fast_read": false, 2023-10-26T01:44:28.162 INFO:teuthology.orchestra.run.smithi169.stdout: "options": { 2023-10-26T01:44:28.162 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_min": 64, 2023-10-26T01:44:28.162 INFO:teuthology.orchestra.run.smithi169.stdout: "target_size_ratio": 0.89999997615814209 2023-10-26T01:44:28.163 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.163 INFO:teuthology.orchestra.run.smithi169.stdout: "application_metadata": { 2023-10-26T01:44:28.163 INFO:teuthology.orchestra.run.smithi169.stdout: "cephfs": { 2023-10-26T01:44:28.163 INFO:teuthology.orchestra.run.smithi169.stdout: "data": "cephfs" 2023-10-26T01:44:28.163 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:28.163 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.164 INFO:teuthology.orchestra.run.smithi169.stdout: "read_balance": { 2023-10-26T01:44:28.164 INFO:teuthology.orchestra.run.smithi169.stdout: "score_acting": 1.3799999952316284, 2023-10-26T01:44:28.164 INFO:teuthology.orchestra.run.smithi169.stdout: "score_stable": 1.3799999952316284, 2023-10-26T01:44:28.164 INFO:teuthology.orchestra.run.smithi169.stdout: "optimal_score": 1, 2023-10-26T01:44:28.164 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_acting": 1.3799999952316284, 2023-10-26T01:44:28.165 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_stable": 1.3799999952316284, 2023-10-26T01:44:28.165 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity_weighted": 1, 2023-10-26T01:44:28.165 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity": 1, 2023-10-26T01:44:28.165 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity_weighted": 1 2023-10-26T01:44:28.165 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:28.166 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:28.166 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:44:28.166 INFO:teuthology.orchestra.run.smithi169.stdout: "osds": [ 2023-10-26T01:44:28.166 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.166 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 0, 2023-10-26T01:44:28.167 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:44:28.167 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:44:28.167 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:44:28.167 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:44:28.167 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:44:28.168 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:44:28.168 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:44:28.168 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:44:28.168 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 180, 2023-10-26T01:44:28.168 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:44:28.168 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:44:28.169 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:44:28.169 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:28.169 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.169 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:28.169 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6816", 2023-10-26T01:44:28.170 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:44:28.170 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.170 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.170 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:28.170 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6817", 2023-10-26T01:44:28.171 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:44:28.171 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:28.171 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:28.171 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.171 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:44:28.171 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:28.172 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.172 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:28.172 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6818", 2023-10-26T01:44:28.172 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:44:28.172 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.173 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.173 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:28.173 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6819", 2023-10-26T01:44:28.173 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:44:28.173 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:28.174 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:28.174 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.174 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:44:28.174 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:28.174 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.175 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:28.175 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6822", 2023-10-26T01:44:28.175 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:44:28.175 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.175 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.176 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:28.176 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6823", 2023-10-26T01:44:28.176 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:44:28.176 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:28.176 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:28.177 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.177 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:44:28.177 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:28.177 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.177 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:28.178 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6820", 2023-10-26T01:44:28.178 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:44:28.178 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.178 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.178 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:28.179 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6821", 2023-10-26T01:44:28.179 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:44:28.179 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:28.179 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:28.179 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.180 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6817/4204255864", 2023-10-26T01:44:28.180 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6819/4204255864", 2023-10-26T01:44:28.180 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6823/4204255864", 2023-10-26T01:44:28.180 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6821/4204255864", 2023-10-26T01:44:28.180 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:44:28.181 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:44:28.181 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:44:28.181 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:28.181 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.181 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.181 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 1, 2023-10-26T01:44:28.182 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:44:28.182 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:44:28.182 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:44:28.182 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:44:28.182 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:44:28.183 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:44:28.183 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:44:28.183 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:44:28.183 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 180, 2023-10-26T01:44:28.183 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:44:28.184 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:44:28.184 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:44:28.184 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:28.184 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.184 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:28.185 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6808", 2023-10-26T01:44:28.185 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:44:28.185 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.185 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.185 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:28.186 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6809", 2023-10-26T01:44:28.186 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:44:28.186 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:28.186 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:28.186 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.187 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:44:28.187 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:28.187 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.187 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:28.187 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6810", 2023-10-26T01:44:28.187 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:44:28.188 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.188 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.188 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:28.188 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6811", 2023-10-26T01:44:28.188 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:44:28.189 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:28.189 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:28.189 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.189 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:44:28.189 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:28.190 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.190 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:28.190 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6814", 2023-10-26T01:44:28.190 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:44:28.190 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.190 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.191 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:28.191 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6815", 2023-10-26T01:44:28.191 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:44:28.191 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:28.191 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:28.192 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.192 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:44:28.192 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:28.192 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.192 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:28.193 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6812", 2023-10-26T01:44:28.193 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:44:28.193 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.193 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.193 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:28.194 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6813", 2023-10-26T01:44:28.194 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:44:28.194 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:28.194 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:28.194 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.195 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6809/238206046", 2023-10-26T01:44:28.195 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6811/238206046", 2023-10-26T01:44:28.195 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6815/238206046", 2023-10-26T01:44:28.195 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6813/238206046", 2023-10-26T01:44:28.195 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:44:28.196 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:44:28.196 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:44:28.196 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:28.196 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.196 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.196 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 2, 2023-10-26T01:44:28.197 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:44:28.197 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:44:28.197 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:44:28.197 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:44:28.197 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:44:28.198 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:44:28.198 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:44:28.198 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:44:28.198 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 180, 2023-10-26T01:44:28.198 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:44:28.199 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:44:28.199 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:44:28.199 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:28.199 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.199 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:28.200 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6800", 2023-10-26T01:44:28.200 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:44:28.200 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.200 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.200 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:28.201 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6801", 2023-10-26T01:44:28.201 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:44:28.201 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:28.201 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:28.201 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.202 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:44:28.202 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:28.202 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.202 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:28.202 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6802", 2023-10-26T01:44:28.203 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:44:28.203 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.203 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.203 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:28.203 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6803", 2023-10-26T01:44:28.203 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:44:28.204 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:28.204 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:28.204 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.204 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:44:28.204 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:28.205 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.205 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:28.205 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6806", 2023-10-26T01:44:28.205 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:44:28.205 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.205 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.206 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:28.206 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6807", 2023-10-26T01:44:28.206 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:44:28.206 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:28.206 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:28.207 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.207 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:44:28.207 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:28.207 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.207 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:28.208 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6804", 2023-10-26T01:44:28.208 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:44:28.208 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.208 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.208 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:28.209 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6805", 2023-10-26T01:44:28.209 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:44:28.209 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:28.209 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:28.209 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.209 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6801/3215791846", 2023-10-26T01:44:28.210 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6803/3215791846", 2023-10-26T01:44:28.210 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6807/3215791846", 2023-10-26T01:44:28.210 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6805/3215791846", 2023-10-26T01:44:28.210 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:44:28.210 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:44:28.211 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:44:28.211 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:28.211 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.211 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.212 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 3, 2023-10-26T01:44:28.212 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:44:28.212 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:44:28.212 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:44:28.212 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:44:28.213 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:44:28.213 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:44:28.213 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:44:28.213 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 14, 2023-10-26T01:44:28.213 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 180, 2023-10-26T01:44:28.214 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:44:28.214 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:44:28.214 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:44:28.214 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:28.214 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.215 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:28.215 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6824", 2023-10-26T01:44:28.215 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:44:28.215 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.215 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.216 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:28.216 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6825", 2023-10-26T01:44:28.216 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:44:28.216 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:28.216 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:28.217 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.217 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:44:28.217 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:28.217 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.217 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:28.218 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6826", 2023-10-26T01:44:28.218 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:44:28.218 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.218 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.218 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:28.219 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6827", 2023-10-26T01:44:28.219 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:44:28.219 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:28.219 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:28.219 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.220 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:44:28.220 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:28.220 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.220 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:28.220 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6830", 2023-10-26T01:44:28.221 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:44:28.221 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.221 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.221 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:28.221 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6831", 2023-10-26T01:44:28.222 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:44:28.222 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:28.222 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:28.222 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.222 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:44:28.223 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:28.223 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.223 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:28.223 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6828", 2023-10-26T01:44:28.223 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:44:28.223 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.224 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.224 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:28.224 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6829", 2023-10-26T01:44:28.224 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:44:28.224 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:28.225 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:28.225 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.225 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6825/2256701226", 2023-10-26T01:44:28.225 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6827/2256701226", 2023-10-26T01:44:28.225 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6831/2256701226", 2023-10-26T01:44:28.225 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6829/2256701226", 2023-10-26T01:44:28.226 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:44:28.226 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:44:28.226 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:44:28.226 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:28.226 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.227 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.227 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 4, 2023-10-26T01:44:28.227 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:44:28.227 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:44:28.227 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:44:28.228 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:44:28.228 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:44:28.228 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:44:28.228 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:44:28.228 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:44:28.228 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 180, 2023-10-26T01:44:28.229 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:44:28.229 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:44:28.229 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:44:28.229 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:28.229 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.230 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:28.230 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6816", 2023-10-26T01:44:28.230 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:44:28.230 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.230 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.231 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:28.231 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6817", 2023-10-26T01:44:28.231 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:44:28.231 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:28.231 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:28.231 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.232 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:44:28.232 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:28.232 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.232 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:28.232 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6818", 2023-10-26T01:44:28.233 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:44:28.233 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.233 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.233 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:28.233 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6819", 2023-10-26T01:44:28.234 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:44:28.234 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:28.234 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:28.234 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.234 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:44:28.235 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:28.235 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.235 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:28.235 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6825", 2023-10-26T01:44:28.235 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:44:28.235 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.236 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.236 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:28.236 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6828", 2023-10-26T01:44:28.236 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:44:28.236 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:28.237 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:28.237 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.237 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:44:28.237 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:28.237 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.237 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:28.238 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6820", 2023-10-26T01:44:28.238 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:44:28.238 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.238 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.238 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:28.239 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6823", 2023-10-26T01:44:28.239 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:44:28.239 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:28.239 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:28.239 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.240 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6817/1082279874", 2023-10-26T01:44:28.240 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6819/1082279874", 2023-10-26T01:44:28.240 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6828/1082279874", 2023-10-26T01:44:28.240 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6823/1082279874", 2023-10-26T01:44:28.240 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:44:28.240 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:44:28.241 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:44:28.241 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:28.241 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.241 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.241 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 5, 2023-10-26T01:44:28.242 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:44:28.242 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:44:28.242 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:44:28.242 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:44:28.242 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:44:28.243 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:44:28.243 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:44:28.243 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:44:28.243 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 180, 2023-10-26T01:44:28.243 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:44:28.244 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:44:28.244 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:44:28.244 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:28.244 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.244 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:28.244 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6808", 2023-10-26T01:44:28.245 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:44:28.245 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.245 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.245 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:28.245 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6809", 2023-10-26T01:44:28.246 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:44:28.246 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:28.246 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:28.246 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.246 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:44:28.247 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:28.247 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.247 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:28.247 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6810", 2023-10-26T01:44:28.247 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:44:28.248 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.248 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.248 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:28.248 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6811", 2023-10-26T01:44:28.248 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:44:28.249 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:28.249 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:28.249 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.249 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:44:28.249 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:28.250 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.250 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:28.250 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6814", 2023-10-26T01:44:28.250 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:44:28.250 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.250 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.251 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:28.251 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6815", 2023-10-26T01:44:28.251 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:44:28.251 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:28.251 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:28.252 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.252 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:44:28.252 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:28.252 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.252 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:28.253 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6812", 2023-10-26T01:44:28.253 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:44:28.253 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.253 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.253 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:28.253 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6813", 2023-10-26T01:44:28.254 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:44:28.254 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:28.254 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:28.254 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.254 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6809/3335654351", 2023-10-26T01:44:28.255 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6811/3335654351", 2023-10-26T01:44:28.255 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6815/3335654351", 2023-10-26T01:44:28.255 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6813/3335654351", 2023-10-26T01:44:28.255 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:44:28.255 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:44:28.256 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:44:28.256 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:28.256 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.256 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.256 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 6, 2023-10-26T01:44:28.256 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:44:28.257 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:44:28.257 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:44:28.257 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:44:28.257 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:44:28.257 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:44:28.258 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:44:28.258 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:44:28.258 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 180, 2023-10-26T01:44:28.258 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:44:28.258 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:44:28.259 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:44:28.259 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:28.259 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.259 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:28.259 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6800", 2023-10-26T01:44:28.260 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:44:28.260 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.260 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.260 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:28.260 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6801", 2023-10-26T01:44:28.260 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:44:28.261 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:28.261 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:28.261 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.261 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:44:28.261 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:28.262 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.262 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:28.262 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6802", 2023-10-26T01:44:28.262 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:44:28.262 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.263 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.263 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:28.263 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6803", 2023-10-26T01:44:28.263 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:44:28.263 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:28.263 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:28.264 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.264 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:44:28.264 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:28.264 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.264 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:28.265 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6806", 2023-10-26T01:44:28.265 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:44:28.265 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.265 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.265 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:28.266 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6807", 2023-10-26T01:44:28.266 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:44:28.266 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:28.266 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:28.266 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.267 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:44:28.267 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:28.267 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.267 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:28.267 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6804", 2023-10-26T01:44:28.267 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:44:28.268 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.268 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.268 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:28.268 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6805", 2023-10-26T01:44:28.268 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:44:28.269 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:28.269 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:28.269 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.269 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6801/2520460697", 2023-10-26T01:44:28.269 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6803/2520460697", 2023-10-26T01:44:28.270 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6807/2520460697", 2023-10-26T01:44:28.270 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6805/2520460697", 2023-10-26T01:44:28.270 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:44:28.270 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:44:28.270 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:44:28.271 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:28.271 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.271 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.271 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 7, 2023-10-26T01:44:28.271 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:44:28.272 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:44:28.272 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:44:28.272 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:44:28.272 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:44:28.272 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:44:28.273 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:44:28.273 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:44:28.273 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 180, 2023-10-26T01:44:28.273 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:44:28.273 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:44:28.274 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:44:28.274 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:28.274 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.274 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:28.274 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6821", 2023-10-26T01:44:28.275 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:44:28.275 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.275 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.275 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:28.275 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6822", 2023-10-26T01:44:28.276 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:44:28.276 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:28.276 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:28.276 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.276 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:44:28.277 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:28.277 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.277 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:28.277 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6824", 2023-10-26T01:44:28.277 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:44:28.278 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.278 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.278 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:28.278 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6826", 2023-10-26T01:44:28.278 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:44:28.279 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:28.279 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:28.279 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.279 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:44:28.279 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:28.280 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.280 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:28.280 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6830", 2023-10-26T01:44:28.280 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:44:28.280 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.281 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.281 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:28.281 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6831", 2023-10-26T01:44:28.281 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:44:28.281 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:28.282 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:28.282 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.282 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:44:28.282 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:28.282 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.282 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:28.283 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6827", 2023-10-26T01:44:28.283 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:44:28.283 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.283 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.283 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:28.284 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6829", 2023-10-26T01:44:28.284 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:44:28.284 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:28.284 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:28.285 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.285 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6822/1115930619", 2023-10-26T01:44:28.285 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6826/1115930619", 2023-10-26T01:44:28.285 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6831/1115930619", 2023-10-26T01:44:28.285 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6829/1115930619", 2023-10-26T01:44:28.286 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:44:28.286 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:44:28.286 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:44:28.286 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:28.286 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:28.287 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:44:28.287 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_xinfo": [ 2023-10-26T01:44:28.287 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.287 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 0, 2023-10-26T01:44:28.287 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:44:28.288 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:44:28.288 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:44:28.288 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:44:28.288 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:44:28.288 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.889675+0000", 2023-10-26T01:44:28.289 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:44:28.289 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.289 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.289 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 1, 2023-10-26T01:44:28.289 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:44:28.290 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:44:28.290 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:44:28.290 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:44:28.290 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:44:28.290 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.864157+0000", 2023-10-26T01:44:28.290 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:44:28.291 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.291 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.291 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 2, 2023-10-26T01:44:28.291 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:44:28.291 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:44:28.292 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:44:28.292 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:44:28.292 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:44:28.292 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.876919+0000", 2023-10-26T01:44:28.292 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:44:28.293 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.293 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.293 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 3, 2023-10-26T01:44:28.293 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:44:28.293 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:44:28.294 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:44:28.294 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:44:28.294 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:44:28.294 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.974541+0000", 2023-10-26T01:44:28.294 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:44:28.294 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.295 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.295 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 4, 2023-10-26T01:44:28.295 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:44:28.295 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:44:28.295 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:44:28.296 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:44:28.296 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:44:28.296 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.969696+0000", 2023-10-26T01:44:28.296 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:44:28.296 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.297 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.297 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 5, 2023-10-26T01:44:28.297 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:44:28.297 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:44:28.297 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:44:28.298 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:44:28.298 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:44:28.298 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.859703+0000", 2023-10-26T01:44:28.298 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:44:28.298 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.299 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.299 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 6, 2023-10-26T01:44:28.299 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:44:28.299 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:44:28.299 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:44:28.300 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:44:28.300 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:44:28.300 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.867917+0000", 2023-10-26T01:44:28.300 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:44:28.300 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.301 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.301 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 7, 2023-10-26T01:44:28.301 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:44:28.301 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:44:28.301 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:44:28.302 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:44:28.302 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:44:28.302 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.898605+0000", 2023-10-26T01:44:28.302 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:44:28.302 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:28.302 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:44:28.303 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_upmap": [], 2023-10-26T01:44:28.303 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_upmap_items": [], 2023-10-26T01:44:28.303 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_upmap_primaries": [], 2023-10-26T01:44:28.303 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_temp": [], 2023-10-26T01:44:28.303 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_temp": [], 2023-10-26T01:44:28.304 INFO:teuthology.orchestra.run.smithi169.stdout: "blocklist": {}, 2023-10-26T01:44:28.304 INFO:teuthology.orchestra.run.smithi169.stdout: "range_blocklist": {}, 2023-10-26T01:44:28.304 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profiles": { 2023-10-26T01:44:28.304 INFO:teuthology.orchestra.run.smithi169.stdout: "default": { 2023-10-26T01:44:28.304 INFO:teuthology.orchestra.run.smithi169.stdout: "crush-failure-domain": "osd", 2023-10-26T01:44:28.304 INFO:teuthology.orchestra.run.smithi169.stdout: "k": "2", 2023-10-26T01:44:28.305 INFO:teuthology.orchestra.run.smithi169.stdout: "m": "1", 2023-10-26T01:44:28.305 INFO:teuthology.orchestra.run.smithi169.stdout: "plugin": "jerasure", 2023-10-26T01:44:28.305 INFO:teuthology.orchestra.run.smithi169.stdout: "technique": "reed_sol_van" 2023-10-26T01:44:28.305 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:28.305 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.306 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps_queue": [], 2023-10-26T01:44:28.306 INFO:teuthology.orchestra.run.smithi169.stdout: "new_removed_snaps": [], 2023-10-26T01:44:28.306 INFO:teuthology.orchestra.run.smithi169.stdout: "new_purged_snaps": [], 2023-10-26T01:44:28.306 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_node_flags": {}, 2023-10-26T01:44:28.306 INFO:teuthology.orchestra.run.smithi169.stdout: "device_class_flags": {}, 2023-10-26T01:44:28.307 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_mode": { 2023-10-26T01:44:28.307 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_mode_enabled": false, 2023-10-26T01:44:28.307 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_bucket_count": 0, 2023-10-26T01:44:28.307 INFO:teuthology.orchestra.run.smithi169.stdout: "degraded_stretch_mode": 0, 2023-10-26T01:44:28.307 INFO:teuthology.orchestra.run.smithi169.stdout: "recovering_stretch_mode": 0, 2023-10-26T01:44:28.308 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_mode_bucket": 0 2023-10-26T01:44:28.308 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:28.308 INFO:teuthology.orchestra.run.smithi169.stdout:} 2023-10-26T01:44:28.309 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:28.107+0000 7f269f90c640 1 -- 172.21.15.169:0/3626523380 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f267c06ae40 msgr2=0x7f267c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:28.310 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:28.107+0000 7f269f90c640 1 --2- 172.21.15.169:0/3626523380 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f267c06ae40 0x7f267c06d300 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f26940019f0 tx=0x7f2694001ba0 comp rx=0 tx=0).stop 2023-10-26T01:44:28.310 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:28.107+0000 7f269f90c640 1 -- 172.21.15.169:0/3626523380 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26980c6560 msgr2=0x7f26980c93d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:28.310 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:28.107+0000 7f269f90c640 1 --2- 172.21.15.169:0/3626523380 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26980c6560 0x7f26980c93d0 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7f268802cb60 tx=0x7f2688033c30 comp rx=0 tx=0).stop 2023-10-26T01:44:28.310 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:28.107+0000 7f269f90c640 1 -- 172.21.15.169:0/3626523380 shutdown_connections 2023-10-26T01:44:28.311 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:28.107+0000 7f269f90c640 1 --2- 172.21.15.169:0/3626523380 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f267c06ae40 0x7f267c06d300 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:28.311 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:28.107+0000 7f269f90c640 1 --2- 172.21.15.169:0/3626523380 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26980c6560 0x7f26980c93d0 unknown :-1 s=CLOSED pgs=1895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:28.311 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:28.107+0000 7f269f90c640 1 --2- 172.21.15.169:0/3626523380 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f26980c69c0 0x7f26980c9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:28.311 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:28.107+0000 7f269f90c640 1 --2- 172.21.15.169:0/3626523380 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f26840047c0 0x7f269814f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:28.311 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:28.107+0000 7f269f90c640 1 -- 172.21.15.169:0/3626523380 >> 172.21.15.169:0/3626523380 conn(0x7f2698069930 msgr2=0x7f2698155f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:28.311 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:28.107+0000 7f269f90c640 1 -- 172.21.15.169:0/3626523380 shutdown_connections 2023-10-26T01:44:28.312 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:28.107+0000 7f269f90c640 1 -- 172.21.15.169:0/3626523380 wait complete. 2023-10-26T01:44:28.312 DEBUG:tasks.ceph_test_case:wait_until_true: success in 10s and 0 retries 2023-10-26T01:44:28.313 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-26T01:44:28.542 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:28.539+0000 7ff868ad7640 1 -- 172.21.15.169:0/1074756361 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff86406cae0 msgr2=0x7ff8640c7ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:28.543 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:28.539+0000 7ff868ad7640 1 --2- 172.21.15.169:0/1074756361 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff86406cae0 0x7ff8640c7ea0 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7ff854009830 tx=0x7ff85402bcd0 comp rx=0 tx=0).stop 2023-10-26T01:44:28.543 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:28.539+0000 7ff868ad7640 1 -- 172.21.15.169:0/1074756361 shutdown_connections 2023-10-26T01:44:28.543 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:28.539+0000 7ff868ad7640 1 --2- 172.21.15.169:0/1074756361 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff86406a990 0x7ff8640cc4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:28.543 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:28.539+0000 7ff868ad7640 1 --2- 172.21.15.169:0/1074756361 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff86406cae0 0x7ff8640c7ea0 unknown :-1 s=CLOSED pgs=1897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:28.544 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:28.539+0000 7ff868ad7640 1 -- 172.21.15.169:0/1074756361 >> 172.21.15.169:0/1074756361 conn(0x7ff8640aedd0 msgr2=0x7ff8640af1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:28.544 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:28.539+0000 7ff868ad7640 1 -- 172.21.15.169:0/1074756361 shutdown_connections 2023-10-26T01:44:28.544 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:28.540+0000 7ff868ad7640 1 -- 172.21.15.169:0/1074756361 wait complete. 2023-10-26T01:44:28.545 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:28.541+0000 7ff868ad7640 1 Processor -- start 2023-10-26T01:44:28.545 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:28.541+0000 7ff868ad7640 1 -- start start 2023-10-26T01:44:28.547 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:28.542+0000 7ff868ad7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff86406a990 0x7ff864146ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:28.547 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:28.542+0000 7ff868ad7640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff86406cae0 0x7ff8640b0030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:28.547 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:28.543+0000 7ff868ad7640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff8640cca30 0x7ff8640c6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:28.548 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:28.543+0000 7ff868ad7640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff864098c00 con 0x7ff86406cae0 2023-10-26T01:44:28.548 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:28.543+0000 7ff868ad7640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff864098ee0 con 0x7ff86406a990 2023-10-26T01:44:28.548 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:28.543+0000 7ff868ad7640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8640991c0 con 0x7ff8640cca30 2023-10-26T01:44:28.548 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:28.543+0000 7ff85bfff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff86406cae0 0x7ff8640b0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:28.549 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:28.543+0000 7ff85bfff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff86406cae0 0x7ff8640b0030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:48674/0 (socket says 172.21.15.169:48674) 2023-10-26T01:44:28.549 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:28.543+0000 7ff85bfff640 1 -- 172.21.15.169:0/1032266273 learned_addr learned my addr 172.21.15.169:0/1032266273 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:44:28.549 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:28.543+0000 7ff8637fe640 1 --2- 172.21.15.169:0/1032266273 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff86406a990 0x7ff864146ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:28.549 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:28.543+0000 7ff85bfff640 1 -- 172.21.15.169:0/1032266273 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff8640cca30 msgr2=0x7ff8640c6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:28.549 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:28.543+0000 7ff863fff640 1 --2- 172.21.15.169:0/1032266273 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff8640cca30 0x7ff8640c6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:28.550 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:28.543+0000 7ff85bfff640 1 --2- 172.21.15.169:0/1032266273 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff8640cca30 0x7ff8640c6270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:28.550 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:28.543+0000 7ff85bfff640 1 -- 172.21.15.169:0/1032266273 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff86406a990 msgr2=0x7ff864146ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:28.550 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:28.543+0000 7ff85bfff640 1 --2- 172.21.15.169:0/1032266273 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff86406a990 0x7ff864146ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:28.550 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:28.543+0000 7ff85bfff640 1 -- 172.21.15.169:0/1032266273 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff854009490 con 0x7ff86406cae0 2023-10-26T01:44:28.550 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:28.543+0000 7ff8637fe640 1 --2- 172.21.15.169:0/1032266273 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff86406a990 0x7ff864146ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:44:28.551 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:28.543+0000 7ff863fff640 1 --2- 172.21.15.169:0/1032266273 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff8640cca30 0x7ff8640c6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:44:28.551 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:28.544+0000 7ff85bfff640 1 --2- 172.21.15.169:0/1032266273 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff86406cae0 0x7ff8640b0030 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7ff84c00d950 tx=0x7ff84c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:28.551 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:28.544+0000 7ff860ff9640 1 -- 172.21.15.169:0/1032266273 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff84c009c30 con 0x7ff86406cae0 2023-10-26T01:44:28.551 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:28.544+0000 7ff860ff9640 1 -- 172.21.15.169:0/1032266273 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff84c009e80 con 0x7ff86406cae0 2023-10-26T01:44:28.551 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:28.544+0000 7ff868ad7640 1 -- 172.21.15.169:0/1032266273 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8640ca190 con 0x7ff86406cae0 2023-10-26T01:44:28.551 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:28.544+0000 7ff860ff9640 1 -- 172.21.15.169:0/1032266273 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff84c0106c0 con 0x7ff86406cae0 2023-10-26T01:44:28.552 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:28.544+0000 7ff868ad7640 1 -- 172.21.15.169:0/1032266273 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff864099780 con 0x7ff86406cae0 2023-10-26T01:44:28.553 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:28.546+0000 7ff868ad7640 1 -- 172.21.15.169:0/1032266273 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8640cb370 con 0x7ff86406cae0 2023-10-26T01:44:28.553 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:28.546+0000 7ff860ff9640 1 -- 172.21.15.169:0/1032266273 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ff84c010820 con 0x7ff86406cae0 2023-10-26T01:44:28.553 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:28.547+0000 7ff860ff9640 1 --2- 172.21.15.169:0/1032266273 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff84406ae40 0x7ff84406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:28.553 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:28.548+0000 7ff860ff9640 1 -- 172.21.15.169:0/1032266273 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff84c0899f0 con 0x7ff86406cae0 2023-10-26T01:44:28.553 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:28.548+0000 7ff8637fe640 1 --2- 172.21.15.169:0/1032266273 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff84406ae40 0x7ff84406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:28.554 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:28.549+0000 7ff8637fe640 1 --2- 172.21.15.169:0/1032266273 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff84406ae40 0x7ff84406d300 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7ff854036000 tx=0x7ff854004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:28.554 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:28.552+0000 7ff860ff9640 1 -- 172.21.15.169:0/1032266273 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff8640cb370 con 0x7ff86406cae0 2023-10-26T01:44:28.746 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:28.743+0000 7ff868ad7640 1 -- 172.21.15.169:0/1032266273 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7ff8640cb370 con 0x7ff86406cae0 2023-10-26T01:44:28.749 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:28.747+0000 7ff860ff9640 1 -- 172.21.15.169:0/1032266273 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v185) v1 ==== 81+0+34062 (secure 0 0 0) 0x7ff8640cb370 con 0x7ff86406cae0 2023-10-26T01:44:28.750 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:44:28.751 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2023-10-26T01:44:28.751 INFO:teuthology.orchestra.run.smithi169.stdout: "epoch": 185, 2023-10-26T01:44:28.751 INFO:teuthology.orchestra.run.smithi169.stdout: "fsid": "30203b28-ed8a-44e8-a9a6-a9bf25f980f1", 2023-10-26T01:44:28.751 INFO:teuthology.orchestra.run.smithi169.stdout: "created": "2023-10-26T01:30:22.897155+0000", 2023-10-26T01:44:28.752 INFO:teuthology.orchestra.run.smithi169.stdout: "modified": "2023-10-26T01:44:23.200104+0000", 2023-10-26T01:44:28.752 INFO:teuthology.orchestra.run.smithi169.stdout: "last_up_change": "2023-10-26T01:30:31.875281+0000", 2023-10-26T01:44:28.752 INFO:teuthology.orchestra.run.smithi169.stdout: "last_in_change": "2023-10-26T01:30:27.637941+0000", 2023-10-26T01:44:28.752 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-26T01:44:28.752 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_num": 5799936, 2023-10-26T01:44:28.753 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_set": [ 2023-10-26T01:44:28.753 INFO:teuthology.orchestra.run.smithi169.stdout: "pglog_hardlimit", 2023-10-26T01:44:28.753 INFO:teuthology.orchestra.run.smithi169.stdout: "purged_snapdirs", 2023-10-26T01:44:28.753 INFO:teuthology.orchestra.run.smithi169.stdout: "recovery_deletes", 2023-10-26T01:44:28.753 INFO:teuthology.orchestra.run.smithi169.stdout: "sortbitwise" 2023-10-26T01:44:28.754 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:44:28.754 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_version": 4, 2023-10-26T01:44:28.754 INFO:teuthology.orchestra.run.smithi169.stdout: "full_ratio": 0.69999998807907104, 2023-10-26T01:44:28.754 INFO:teuthology.orchestra.run.smithi169.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-26T01:44:28.755 INFO:teuthology.orchestra.run.smithi169.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-26T01:44:28.755 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_snapshot": "", 2023-10-26T01:44:28.755 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_max": 23, 2023-10-26T01:44:28.755 INFO:teuthology.orchestra.run.smithi169.stdout: "max_osd": 8, 2023-10-26T01:44:28.755 INFO:teuthology.orchestra.run.smithi169.stdout: "require_min_compat_client": "luminous", 2023-10-26T01:44:28.756 INFO:teuthology.orchestra.run.smithi169.stdout: "min_compat_client": "jewel", 2023-10-26T01:44:28.756 INFO:teuthology.orchestra.run.smithi169.stdout: "require_osd_release": "reef", 2023-10-26T01:44:28.756 INFO:teuthology.orchestra.run.smithi169.stdout: "allow_crimson": false, 2023-10-26T01:44:28.756 INFO:teuthology.orchestra.run.smithi169.stdout: "pools": [ 2023-10-26T01:44:28.756 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.757 INFO:teuthology.orchestra.run.smithi169.stdout: "pool": 1, 2023-10-26T01:44:28.757 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_name": ".mgr", 2023-10-26T01:44:28.757 INFO:teuthology.orchestra.run.smithi169.stdout: "create_time": "2023-10-26T01:30:32.200808+0000", 2023-10-26T01:44:28.757 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": 1, 2023-10-26T01:44:28.757 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_names": "hashpspool", 2023-10-26T01:44:28.758 INFO:teuthology.orchestra.run.smithi169.stdout: "type": 1, 2023-10-26T01:44:28.758 INFO:teuthology.orchestra.run.smithi169.stdout: "size": 2, 2023-10-26T01:44:28.758 INFO:teuthology.orchestra.run.smithi169.stdout: "min_size": 1, 2023-10-26T01:44:28.758 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_rule": 0, 2023-10-26T01:44:28.758 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_count": 0, 2023-10-26T01:44:28.759 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_target": 0, 2023-10-26T01:44:28.759 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T01:44:28.759 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T01:44:28.759 INFO:teuthology.orchestra.run.smithi169.stdout: "object_hash": 2, 2023-10-26T01:44:28.759 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_mode": "off", 2023-10-26T01:44:28.760 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num": 1, 2023-10-26T01:44:28.760 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num": 1, 2023-10-26T01:44:28.760 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num_target": 1, 2023-10-26T01:44:28.760 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_target": 1, 2023-10-26T01:44:28.760 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_pending": 1, 2023-10-26T01:44:28.761 INFO:teuthology.orchestra.run.smithi169.stdout: "last_pg_merge_meta": { 2023-10-26T01:44:28.761 INFO:teuthology.orchestra.run.smithi169.stdout: "source_pgid": "0.0", 2023-10-26T01:44:28.761 INFO:teuthology.orchestra.run.smithi169.stdout: "ready_epoch": 0, 2023-10-26T01:44:28.761 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_started": 0, 2023-10-26T01:44:28.761 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_clean": 0, 2023-10-26T01:44:28.761 INFO:teuthology.orchestra.run.smithi169.stdout: "source_version": "0'0", 2023-10-26T01:44:28.762 INFO:teuthology.orchestra.run.smithi169.stdout: "target_version": "0'0" 2023-10-26T01:44:28.762 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.762 INFO:teuthology.orchestra.run.smithi169.stdout: "last_change": "109", 2023-10-26T01:44:28.762 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend": "0", 2023-10-26T01:44:28.762 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T01:44:28.763 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T01:44:28.763 INFO:teuthology.orchestra.run.smithi169.stdout: "auid": 0, 2023-10-26T01:44:28.763 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_mode": "selfmanaged", 2023-10-26T01:44:28.763 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_seq": 0, 2023-10-26T01:44:28.763 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_epoch": 0, 2023-10-26T01:44:28.764 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_snaps": [], 2023-10-26T01:44:28.764 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps": "[]", 2023-10-26T01:44:28.764 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_bytes": 0, 2023-10-26T01:44:28.764 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_objects": 0, 2023-10-26T01:44:28.764 INFO:teuthology.orchestra.run.smithi169.stdout: "tiers": [], 2023-10-26T01:44:28.765 INFO:teuthology.orchestra.run.smithi169.stdout: "tier_of": -1, 2023-10-26T01:44:28.765 INFO:teuthology.orchestra.run.smithi169.stdout: "read_tier": -1, 2023-10-26T01:44:28.765 INFO:teuthology.orchestra.run.smithi169.stdout: "write_tier": -1, 2023-10-26T01:44:28.765 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_mode": "none", 2023-10-26T01:44:28.765 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_bytes": 0, 2023-10-26T01:44:28.766 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_objects": 0, 2023-10-26T01:44:28.766 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T01:44:28.766 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T01:44:28.766 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T01:44:28.766 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_flush_age": 0, 2023-10-26T01:44:28.766 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_evict_age": 0, 2023-10-26T01:44:28.767 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profile": "", 2023-10-26T01:44:28.767 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_params": { 2023-10-26T01:44:28.767 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "none" 2023-10-26T01:44:28.767 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.767 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_period": 0, 2023-10-26T01:44:28.768 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_count": 0, 2023-10-26T01:44:28.768 INFO:teuthology.orchestra.run.smithi169.stdout: "use_gmt_hitset": true, 2023-10-26T01:44:28.768 INFO:teuthology.orchestra.run.smithi169.stdout: "min_read_recency_for_promote": 0, 2023-10-26T01:44:28.768 INFO:teuthology.orchestra.run.smithi169.stdout: "min_write_recency_for_promote": 0, 2023-10-26T01:44:28.768 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T01:44:28.769 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_search_last_n": 0, 2023-10-26T01:44:28.769 INFO:teuthology.orchestra.run.smithi169.stdout: "grade_table": [], 2023-10-26T01:44:28.769 INFO:teuthology.orchestra.run.smithi169.stdout: "stripe_width": 0, 2023-10-26T01:44:28.769 INFO:teuthology.orchestra.run.smithi169.stdout: "expected_num_objects": 0, 2023-10-26T01:44:28.769 INFO:teuthology.orchestra.run.smithi169.stdout: "fast_read": false, 2023-10-26T01:44:28.770 INFO:teuthology.orchestra.run.smithi169.stdout: "options": { 2023-10-26T01:44:28.770 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_max": 32, 2023-10-26T01:44:28.770 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_min": 1 2023-10-26T01:44:28.770 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.770 INFO:teuthology.orchestra.run.smithi169.stdout: "application_metadata": { 2023-10-26T01:44:28.771 INFO:teuthology.orchestra.run.smithi169.stdout: "mgr": {} 2023-10-26T01:44:28.771 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.771 INFO:teuthology.orchestra.run.smithi169.stdout: "read_balance": { 2023-10-26T01:44:28.771 INFO:teuthology.orchestra.run.smithi169.stdout: "score_acting": 8, 2023-10-26T01:44:28.771 INFO:teuthology.orchestra.run.smithi169.stdout: "score_stable": 8, 2023-10-26T01:44:28.772 INFO:teuthology.orchestra.run.smithi169.stdout: "optimal_score": 0.25, 2023-10-26T01:44:28.772 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_acting": 2, 2023-10-26T01:44:28.772 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_stable": 2, 2023-10-26T01:44:28.772 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity_weighted": 1, 2023-10-26T01:44:28.772 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity": 1, 2023-10-26T01:44:28.772 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity_weighted": 1 2023-10-26T01:44:28.773 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:28.773 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.773 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.773 INFO:teuthology.orchestra.run.smithi169.stdout: "pool": 22, 2023-10-26T01:44:28.773 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_name": "cephfs_metadata", 2023-10-26T01:44:28.774 INFO:teuthology.orchestra.run.smithi169.stdout: "create_time": "2023-10-26T01:43:47.501296+0000", 2023-10-26T01:44:28.774 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": 1, 2023-10-26T01:44:28.774 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_names": "hashpspool", 2023-10-26T01:44:28.774 INFO:teuthology.orchestra.run.smithi169.stdout: "type": 1, 2023-10-26T01:44:28.774 INFO:teuthology.orchestra.run.smithi169.stdout: "size": 2, 2023-10-26T01:44:28.775 INFO:teuthology.orchestra.run.smithi169.stdout: "min_size": 1, 2023-10-26T01:44:28.775 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_rule": 0, 2023-10-26T01:44:28.775 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_count": 0, 2023-10-26T01:44:28.775 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_target": 0, 2023-10-26T01:44:28.775 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T01:44:28.776 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T01:44:28.776 INFO:teuthology.orchestra.run.smithi169.stdout: "object_hash": 2, 2023-10-26T01:44:28.776 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_mode": "off", 2023-10-26T01:44:28.776 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num": 32, 2023-10-26T01:44:28.776 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num": 32, 2023-10-26T01:44:28.776 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num_target": 32, 2023-10-26T01:44:28.777 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_target": 32, 2023-10-26T01:44:28.777 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_pending": 32, 2023-10-26T01:44:28.777 INFO:teuthology.orchestra.run.smithi169.stdout: "last_pg_merge_meta": { 2023-10-26T01:44:28.777 INFO:teuthology.orchestra.run.smithi169.stdout: "source_pgid": "0.0", 2023-10-26T01:44:28.777 INFO:teuthology.orchestra.run.smithi169.stdout: "ready_epoch": 0, 2023-10-26T01:44:28.778 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_started": 0, 2023-10-26T01:44:28.778 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_clean": 0, 2023-10-26T01:44:28.778 INFO:teuthology.orchestra.run.smithi169.stdout: "source_version": "0'0", 2023-10-26T01:44:28.778 INFO:teuthology.orchestra.run.smithi169.stdout: "target_version": "0'0" 2023-10-26T01:44:28.778 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.779 INFO:teuthology.orchestra.run.smithi169.stdout: "last_change": "181", 2023-10-26T01:44:28.779 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend": "0", 2023-10-26T01:44:28.779 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T01:44:28.779 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T01:44:28.779 INFO:teuthology.orchestra.run.smithi169.stdout: "auid": 0, 2023-10-26T01:44:28.780 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_mode": "selfmanaged", 2023-10-26T01:44:28.780 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_seq": 0, 2023-10-26T01:44:28.780 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_epoch": 0, 2023-10-26T01:44:28.780 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_snaps": [], 2023-10-26T01:44:28.780 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps": "[]", 2023-10-26T01:44:28.781 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_bytes": 0, 2023-10-26T01:44:28.781 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_objects": 0, 2023-10-26T01:44:28.781 INFO:teuthology.orchestra.run.smithi169.stdout: "tiers": [], 2023-10-26T01:44:28.781 INFO:teuthology.orchestra.run.smithi169.stdout: "tier_of": -1, 2023-10-26T01:44:28.781 INFO:teuthology.orchestra.run.smithi169.stdout: "read_tier": -1, 2023-10-26T01:44:28.782 INFO:teuthology.orchestra.run.smithi169.stdout: "write_tier": -1, 2023-10-26T01:44:28.782 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_mode": "none", 2023-10-26T01:44:28.782 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_bytes": 0, 2023-10-26T01:44:28.782 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_objects": 0, 2023-10-26T01:44:28.782 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T01:44:28.783 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T01:44:28.783 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T01:44:28.783 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_flush_age": 0, 2023-10-26T01:44:28.783 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_evict_age": 0, 2023-10-26T01:44:28.783 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profile": "", 2023-10-26T01:44:28.784 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_params": { 2023-10-26T01:44:28.784 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "none" 2023-10-26T01:44:28.784 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.784 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_period": 0, 2023-10-26T01:44:28.784 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_count": 0, 2023-10-26T01:44:28.784 INFO:teuthology.orchestra.run.smithi169.stdout: "use_gmt_hitset": true, 2023-10-26T01:44:28.785 INFO:teuthology.orchestra.run.smithi169.stdout: "min_read_recency_for_promote": 0, 2023-10-26T01:44:28.785 INFO:teuthology.orchestra.run.smithi169.stdout: "min_write_recency_for_promote": 0, 2023-10-26T01:44:28.785 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T01:44:28.785 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_search_last_n": 0, 2023-10-26T01:44:28.785 INFO:teuthology.orchestra.run.smithi169.stdout: "grade_table": [], 2023-10-26T01:44:28.786 INFO:teuthology.orchestra.run.smithi169.stdout: "stripe_width": 0, 2023-10-26T01:44:28.786 INFO:teuthology.orchestra.run.smithi169.stdout: "expected_num_objects": 0, 2023-10-26T01:44:28.786 INFO:teuthology.orchestra.run.smithi169.stdout: "fast_read": false, 2023-10-26T01:44:28.786 INFO:teuthology.orchestra.run.smithi169.stdout: "options": { 2023-10-26T01:44:28.786 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_bias": 4, 2023-10-26T01:44:28.787 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_min": 16, 2023-10-26T01:44:28.787 INFO:teuthology.orchestra.run.smithi169.stdout: "recovery_priority": 5 2023-10-26T01:44:28.787 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.787 INFO:teuthology.orchestra.run.smithi169.stdout: "application_metadata": { 2023-10-26T01:44:28.787 INFO:teuthology.orchestra.run.smithi169.stdout: "cephfs": { 2023-10-26T01:44:28.788 INFO:teuthology.orchestra.run.smithi169.stdout: "metadata": "cephfs" 2023-10-26T01:44:28.788 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:28.788 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.788 INFO:teuthology.orchestra.run.smithi169.stdout: "read_balance": { 2023-10-26T01:44:28.788 INFO:teuthology.orchestra.run.smithi169.stdout: "score_acting": 2, 2023-10-26T01:44:28.789 INFO:teuthology.orchestra.run.smithi169.stdout: "score_stable": 2, 2023-10-26T01:44:28.789 INFO:teuthology.orchestra.run.smithi169.stdout: "optimal_score": 1, 2023-10-26T01:44:28.789 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_acting": 2, 2023-10-26T01:44:28.789 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_stable": 2, 2023-10-26T01:44:28.789 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity_weighted": 1, 2023-10-26T01:44:28.789 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity": 1, 2023-10-26T01:44:28.790 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity_weighted": 1 2023-10-26T01:44:28.790 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:28.790 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.790 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.790 INFO:teuthology.orchestra.run.smithi169.stdout: "pool": 23, 2023-10-26T01:44:28.791 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_name": "cephfs_data", 2023-10-26T01:44:28.791 INFO:teuthology.orchestra.run.smithi169.stdout: "create_time": "2023-10-26T01:43:48.737507+0000", 2023-10-26T01:44:28.791 INFO:teuthology.orchestra.run.smithi169.stdout: "flags": 1, 2023-10-26T01:44:28.791 INFO:teuthology.orchestra.run.smithi169.stdout: "flags_names": "hashpspool", 2023-10-26T01:44:28.791 INFO:teuthology.orchestra.run.smithi169.stdout: "type": 1, 2023-10-26T01:44:28.792 INFO:teuthology.orchestra.run.smithi169.stdout: "size": 2, 2023-10-26T01:44:28.792 INFO:teuthology.orchestra.run.smithi169.stdout: "min_size": 1, 2023-10-26T01:44:28.792 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_rule": 0, 2023-10-26T01:44:28.792 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_count": 0, 2023-10-26T01:44:28.792 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_target": 0, 2023-10-26T01:44:28.793 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T01:44:28.793 INFO:teuthology.orchestra.run.smithi169.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T01:44:28.793 INFO:teuthology.orchestra.run.smithi169.stdout: "object_hash": 2, 2023-10-26T01:44:28.793 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_autoscale_mode": "off", 2023-10-26T01:44:28.793 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num": 64, 2023-10-26T01:44:28.794 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num": 64, 2023-10-26T01:44:28.794 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_placement_num_target": 64, 2023-10-26T01:44:28.794 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_target": 64, 2023-10-26T01:44:28.794 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_pending": 64, 2023-10-26T01:44:28.794 INFO:teuthology.orchestra.run.smithi169.stdout: "last_pg_merge_meta": { 2023-10-26T01:44:28.795 INFO:teuthology.orchestra.run.smithi169.stdout: "source_pgid": "0.0", 2023-10-26T01:44:28.795 INFO:teuthology.orchestra.run.smithi169.stdout: "ready_epoch": 0, 2023-10-26T01:44:28.795 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_started": 0, 2023-10-26T01:44:28.795 INFO:teuthology.orchestra.run.smithi169.stdout: "last_epoch_clean": 0, 2023-10-26T01:44:28.795 INFO:teuthology.orchestra.run.smithi169.stdout: "source_version": "0'0", 2023-10-26T01:44:28.796 INFO:teuthology.orchestra.run.smithi169.stdout: "target_version": "0'0" 2023-10-26T01:44:28.796 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.796 INFO:teuthology.orchestra.run.smithi169.stdout: "last_change": "185", 2023-10-26T01:44:28.796 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend": "0", 2023-10-26T01:44:28.796 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T01:44:28.796 INFO:teuthology.orchestra.run.smithi169.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T01:44:28.797 INFO:teuthology.orchestra.run.smithi169.stdout: "auid": 0, 2023-10-26T01:44:28.797 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_mode": "selfmanaged", 2023-10-26T01:44:28.797 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_seq": 0, 2023-10-26T01:44:28.797 INFO:teuthology.orchestra.run.smithi169.stdout: "snap_epoch": 0, 2023-10-26T01:44:28.797 INFO:teuthology.orchestra.run.smithi169.stdout: "pool_snaps": [], 2023-10-26T01:44:28.798 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps": "[]", 2023-10-26T01:44:28.798 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_bytes": 33554432, 2023-10-26T01:44:28.798 INFO:teuthology.orchestra.run.smithi169.stdout: "quota_max_objects": 0, 2023-10-26T01:44:28.798 INFO:teuthology.orchestra.run.smithi169.stdout: "tiers": [], 2023-10-26T01:44:28.798 INFO:teuthology.orchestra.run.smithi169.stdout: "tier_of": -1, 2023-10-26T01:44:28.799 INFO:teuthology.orchestra.run.smithi169.stdout: "read_tier": -1, 2023-10-26T01:44:28.799 INFO:teuthology.orchestra.run.smithi169.stdout: "write_tier": -1, 2023-10-26T01:44:28.799 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_mode": "none", 2023-10-26T01:44:28.799 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_bytes": 0, 2023-10-26T01:44:28.799 INFO:teuthology.orchestra.run.smithi169.stdout: "target_max_objects": 0, 2023-10-26T01:44:28.800 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T01:44:28.800 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T01:44:28.800 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T01:44:28.800 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_flush_age": 0, 2023-10-26T01:44:28.800 INFO:teuthology.orchestra.run.smithi169.stdout: "cache_min_evict_age": 0, 2023-10-26T01:44:28.801 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profile": "", 2023-10-26T01:44:28.801 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_params": { 2023-10-26T01:44:28.801 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "none" 2023-10-26T01:44:28.801 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.801 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_period": 0, 2023-10-26T01:44:28.802 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_count": 0, 2023-10-26T01:44:28.802 INFO:teuthology.orchestra.run.smithi169.stdout: "use_gmt_hitset": true, 2023-10-26T01:44:28.802 INFO:teuthology.orchestra.run.smithi169.stdout: "min_read_recency_for_promote": 0, 2023-10-26T01:44:28.802 INFO:teuthology.orchestra.run.smithi169.stdout: "min_write_recency_for_promote": 0, 2023-10-26T01:44:28.803 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T01:44:28.803 INFO:teuthology.orchestra.run.smithi169.stdout: "hit_set_search_last_n": 0, 2023-10-26T01:44:28.803 INFO:teuthology.orchestra.run.smithi169.stdout: "grade_table": [], 2023-10-26T01:44:28.803 INFO:teuthology.orchestra.run.smithi169.stdout: "stripe_width": 0, 2023-10-26T01:44:28.803 INFO:teuthology.orchestra.run.smithi169.stdout: "expected_num_objects": 0, 2023-10-26T01:44:28.804 INFO:teuthology.orchestra.run.smithi169.stdout: "fast_read": false, 2023-10-26T01:44:28.804 INFO:teuthology.orchestra.run.smithi169.stdout: "options": { 2023-10-26T01:44:28.804 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_num_min": 64, 2023-10-26T01:44:28.804 INFO:teuthology.orchestra.run.smithi169.stdout: "target_size_ratio": 0.89999997615814209 2023-10-26T01:44:28.804 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.805 INFO:teuthology.orchestra.run.smithi169.stdout: "application_metadata": { 2023-10-26T01:44:28.805 INFO:teuthology.orchestra.run.smithi169.stdout: "cephfs": { 2023-10-26T01:44:28.805 INFO:teuthology.orchestra.run.smithi169.stdout: "data": "cephfs" 2023-10-26T01:44:28.805 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:28.805 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.806 INFO:teuthology.orchestra.run.smithi169.stdout: "read_balance": { 2023-10-26T01:44:28.806 INFO:teuthology.orchestra.run.smithi169.stdout: "score_acting": 1.3799999952316284, 2023-10-26T01:44:28.806 INFO:teuthology.orchestra.run.smithi169.stdout: "score_stable": 1.3799999952316284, 2023-10-26T01:44:28.806 INFO:teuthology.orchestra.run.smithi169.stdout: "optimal_score": 1, 2023-10-26T01:44:28.806 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_acting": 1.3799999952316284, 2023-10-26T01:44:28.807 INFO:teuthology.orchestra.run.smithi169.stdout: "raw_score_stable": 1.3799999952316284, 2023-10-26T01:44:28.807 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity_weighted": 1, 2023-10-26T01:44:28.807 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity": 1, 2023-10-26T01:44:28.807 INFO:teuthology.orchestra.run.smithi169.stdout: "average_primary_affinity_weighted": 1 2023-10-26T01:44:28.807 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:28.808 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:28.808 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:44:28.808 INFO:teuthology.orchestra.run.smithi169.stdout: "osds": [ 2023-10-26T01:44:28.808 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.808 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 0, 2023-10-26T01:44:28.809 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:44:28.809 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:44:28.809 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:44:28.809 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:44:28.809 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:44:28.810 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:44:28.810 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:44:28.810 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:44:28.810 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 180, 2023-10-26T01:44:28.810 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:44:28.810 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:44:28.811 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:44:28.811 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:28.811 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.811 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:28.811 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6816", 2023-10-26T01:44:28.812 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:44:28.812 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.812 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.812 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:28.812 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6817", 2023-10-26T01:44:28.813 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:44:28.813 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:28.813 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:28.813 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.813 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:44:28.813 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:28.814 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.814 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:28.814 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6818", 2023-10-26T01:44:28.814 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:44:28.814 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.815 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.815 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:28.815 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6819", 2023-10-26T01:44:28.815 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:44:28.815 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:28.816 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:28.816 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.816 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:44:28.816 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:28.816 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.816 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:28.817 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6822", 2023-10-26T01:44:28.817 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:44:28.817 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.817 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.817 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:28.818 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6823", 2023-10-26T01:44:28.818 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:44:28.818 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:28.818 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:28.818 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.819 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:44:28.819 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:28.819 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.819 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:28.819 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6820", 2023-10-26T01:44:28.820 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:44:28.820 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.820 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.820 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:28.820 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6821", 2023-10-26T01:44:28.820 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 4204255864 2023-10-26T01:44:28.821 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:28.821 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:28.821 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.821 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6817/4204255864", 2023-10-26T01:44:28.821 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6819/4204255864", 2023-10-26T01:44:28.822 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6823/4204255864", 2023-10-26T01:44:28.822 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6821/4204255864", 2023-10-26T01:44:28.822 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:44:28.822 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:44:28.822 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:44:28.822 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:28.823 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.823 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.823 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 1, 2023-10-26T01:44:28.823 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:44:28.823 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:44:28.824 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:44:28.824 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:44:28.824 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:44:28.824 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:44:28.824 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:44:28.825 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:44:28.825 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 180, 2023-10-26T01:44:28.825 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:44:28.825 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:44:28.825 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:44:28.826 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:28.826 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.826 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:28.826 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6808", 2023-10-26T01:44:28.826 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:44:28.827 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.827 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.827 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:28.827 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6809", 2023-10-26T01:44:28.827 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:44:28.828 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:28.828 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:28.828 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.828 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:44:28.828 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:28.829 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.829 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:28.829 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6810", 2023-10-26T01:44:28.829 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:44:28.829 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.830 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.830 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:28.830 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6811", 2023-10-26T01:44:28.830 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:44:28.830 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:28.831 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:28.831 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.831 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:44:28.831 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:28.831 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.832 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:28.832 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6814", 2023-10-26T01:44:28.832 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:44:28.832 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.832 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.833 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:28.833 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6815", 2023-10-26T01:44:28.833 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:44:28.833 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:28.833 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:28.833 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.834 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:44:28.834 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:28.834 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.834 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:28.835 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6812", 2023-10-26T01:44:28.835 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:44:28.835 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.835 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.835 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:28.835 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6813", 2023-10-26T01:44:28.836 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 238206046 2023-10-26T01:44:28.836 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:28.836 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:28.836 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.837 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6809/238206046", 2023-10-26T01:44:28.837 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6811/238206046", 2023-10-26T01:44:28.837 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6815/238206046", 2023-10-26T01:44:28.837 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6813/238206046", 2023-10-26T01:44:28.837 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:44:28.838 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:44:28.838 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:44:28.838 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:28.838 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.838 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.839 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 2, 2023-10-26T01:44:28.839 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:44:28.839 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:44:28.839 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:44:28.839 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:44:28.840 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:44:28.840 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:44:28.840 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:44:28.840 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:44:28.840 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 180, 2023-10-26T01:44:28.841 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:44:28.841 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:44:28.841 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:44:28.841 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:28.841 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.841 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:28.842 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6800", 2023-10-26T01:44:28.842 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:44:28.842 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.842 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.842 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:28.843 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6801", 2023-10-26T01:44:28.843 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:44:28.843 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:28.843 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:28.843 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.843 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:44:28.844 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:28.844 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.844 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:28.844 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6802", 2023-10-26T01:44:28.844 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:44:28.845 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.845 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.845 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:28.845 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6803", 2023-10-26T01:44:28.845 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:44:28.846 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:28.846 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:28.846 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.846 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:44:28.846 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:28.846 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.847 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:28.847 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6806", 2023-10-26T01:44:28.847 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:44:28.847 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.847 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.848 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:28.848 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6807", 2023-10-26T01:44:28.848 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:44:28.848 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:28.848 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:28.848 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.849 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:44:28.849 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:28.849 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.849 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:28.849 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6804", 2023-10-26T01:44:28.850 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:44:28.850 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.850 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.850 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:28.850 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6805", 2023-10-26T01:44:28.851 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3215791846 2023-10-26T01:44:28.851 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:28.851 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:28.851 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.851 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6801/3215791846", 2023-10-26T01:44:28.851 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6803/3215791846", 2023-10-26T01:44:28.852 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6807/3215791846", 2023-10-26T01:44:28.852 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6805/3215791846", 2023-10-26T01:44:28.852 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:44:28.852 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:44:28.852 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:44:28.853 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:28.853 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.853 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.853 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 3, 2023-10-26T01:44:28.853 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:44:28.854 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:44:28.854 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:44:28.854 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:44:28.854 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:44:28.854 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:44:28.854 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:44:28.855 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 14, 2023-10-26T01:44:28.855 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 180, 2023-10-26T01:44:28.855 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:44:28.855 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:44:28.855 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:44:28.855 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:28.856 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.856 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:28.856 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6824", 2023-10-26T01:44:28.856 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:44:28.856 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.857 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.857 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:28.857 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6825", 2023-10-26T01:44:28.857 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:44:28.857 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:28.857 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:28.858 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.858 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:44:28.858 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:28.858 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.858 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:28.859 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6826", 2023-10-26T01:44:28.859 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:44:28.859 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.859 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.859 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:28.860 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6827", 2023-10-26T01:44:28.860 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:44:28.860 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:28.860 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:28.860 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.861 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:44:28.861 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:28.861 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.861 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:28.861 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6830", 2023-10-26T01:44:28.861 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:44:28.862 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.862 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.862 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:28.862 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6831", 2023-10-26T01:44:28.862 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:44:28.863 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:28.863 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:28.863 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.863 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:44:28.863 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:28.864 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.864 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:28.864 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6828", 2023-10-26T01:44:28.864 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:44:28.864 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.865 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.865 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:28.865 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.169:6829", 2023-10-26T01:44:28.865 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2256701226 2023-10-26T01:44:28.865 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:28.866 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:28.866 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.866 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.169:6825/2256701226", 2023-10-26T01:44:28.866 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.169:6827/2256701226", 2023-10-26T01:44:28.866 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.169:6831/2256701226", 2023-10-26T01:44:28.867 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.169:6829/2256701226", 2023-10-26T01:44:28.867 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:44:28.867 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:44:28.867 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:44:28.867 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:28.867 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.868 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.868 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 4, 2023-10-26T01:44:28.868 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:44:28.868 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:44:28.868 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:44:28.869 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:44:28.869 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:44:28.869 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:44:28.869 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:44:28.869 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:44:28.870 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 180, 2023-10-26T01:44:28.870 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:44:28.870 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:44:28.870 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:44:28.870 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:28.871 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.871 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:28.871 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6816", 2023-10-26T01:44:28.871 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:44:28.871 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.872 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.872 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:28.872 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6817", 2023-10-26T01:44:28.872 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:44:28.872 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:28.873 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:28.873 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.873 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:44:28.873 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:28.873 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.873 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:28.874 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6818", 2023-10-26T01:44:28.874 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:44:28.874 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.874 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.874 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:28.875 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6819", 2023-10-26T01:44:28.875 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:44:28.875 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:28.875 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:28.875 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.875 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:44:28.876 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:28.876 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.876 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:28.876 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6825", 2023-10-26T01:44:28.876 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:44:28.877 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.877 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.877 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:28.877 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6828", 2023-10-26T01:44:28.877 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:44:28.878 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:28.878 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:28.878 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.878 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:44:28.878 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:28.879 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.879 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:28.879 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6820", 2023-10-26T01:44:28.879 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:44:28.879 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.880 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.880 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:28.880 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6823", 2023-10-26T01:44:28.880 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1082279874 2023-10-26T01:44:28.880 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:28.880 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:28.881 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.881 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6817/1082279874", 2023-10-26T01:44:28.881 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6819/1082279874", 2023-10-26T01:44:28.881 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6828/1082279874", 2023-10-26T01:44:28.881 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6823/1082279874", 2023-10-26T01:44:28.882 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:44:28.882 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:44:28.882 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:44:28.882 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:28.882 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.883 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.883 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 5, 2023-10-26T01:44:28.883 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:44:28.883 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:44:28.883 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:44:28.884 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:44:28.884 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:44:28.884 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:44:28.884 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:44:28.884 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:44:28.885 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 180, 2023-10-26T01:44:28.885 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:44:28.885 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:44:28.885 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:44:28.885 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:28.886 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.886 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:28.886 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6808", 2023-10-26T01:44:28.886 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:44:28.887 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.887 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.887 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:28.887 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6809", 2023-10-26T01:44:28.887 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:44:28.887 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:28.888 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:28.888 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.888 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:44:28.888 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:28.888 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.889 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:28.889 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6810", 2023-10-26T01:44:28.889 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:44:28.889 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.889 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.890 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:28.890 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6811", 2023-10-26T01:44:28.890 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:44:28.890 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:28.890 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:28.890 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.891 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:44:28.891 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:28.891 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.891 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:28.894 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6814", 2023-10-26T01:44:28.894 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:44:28.894 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.894 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.895 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:28.895 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6815", 2023-10-26T01:44:28.895 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:44:28.895 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:28.896 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:28.896 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.896 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:44:28.896 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:28.896 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.896 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:28.897 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6812", 2023-10-26T01:44:28.897 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:44:28.897 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.897 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.897 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:28.898 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6813", 2023-10-26T01:44:28.898 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 3335654351 2023-10-26T01:44:28.898 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:28.898 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:28.898 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.899 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6809/3335654351", 2023-10-26T01:44:28.899 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6811/3335654351", 2023-10-26T01:44:28.899 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6815/3335654351", 2023-10-26T01:44:28.899 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6813/3335654351", 2023-10-26T01:44:28.899 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:44:28.899 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:44:28.900 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:44:28.900 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:28.900 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.900 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.900 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 6, 2023-10-26T01:44:28.901 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:44:28.901 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:44:28.901 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:44:28.901 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:44:28.901 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:44:28.902 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:44:28.902 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:44:28.902 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:44:28.902 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 180, 2023-10-26T01:44:28.902 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:44:28.903 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:44:28.903 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:44:28.903 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:28.903 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.904 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:28.904 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6800", 2023-10-26T01:44:28.904 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:44:28.904 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.904 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.904 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:28.905 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6801", 2023-10-26T01:44:28.905 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:44:28.905 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:28.905 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:28.905 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.906 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:44:28.906 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:28.906 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.906 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:28.906 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6802", 2023-10-26T01:44:28.907 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:44:28.907 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.907 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.907 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:28.907 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6803", 2023-10-26T01:44:28.908 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:44:28.908 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:28.908 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:28.908 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.908 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:44:28.909 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:28.909 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.909 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:28.909 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6806", 2023-10-26T01:44:28.909 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:44:28.910 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.910 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.910 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:28.910 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6807", 2023-10-26T01:44:28.910 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:44:28.910 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:28.911 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:28.911 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.911 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:44:28.911 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:28.911 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.912 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:28.912 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6804", 2023-10-26T01:44:28.912 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:44:28.912 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.913 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.913 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:28.913 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6805", 2023-10-26T01:44:28.913 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2520460697 2023-10-26T01:44:28.913 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:28.914 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:28.914 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.914 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6801/2520460697", 2023-10-26T01:44:28.914 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6803/2520460697", 2023-10-26T01:44:28.914 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6807/2520460697", 2023-10-26T01:44:28.915 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6805/2520460697", 2023-10-26T01:44:28.915 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:44:28.915 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:44:28.915 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:44:28.915 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:28.915 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.916 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.916 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 7, 2023-10-26T01:44:28.916 INFO:teuthology.orchestra.run.smithi169.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T01:44:28.916 INFO:teuthology.orchestra.run.smithi169.stdout: "up": 1, 2023-10-26T01:44:28.916 INFO:teuthology.orchestra.run.smithi169.stdout: "in": 1, 2023-10-26T01:44:28.917 INFO:teuthology.orchestra.run.smithi169.stdout: "weight": 1, 2023-10-26T01:44:28.917 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_affinity": 1, 2023-10-26T01:44:28.917 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_begin": 0, 2023-10-26T01:44:28.917 INFO:teuthology.orchestra.run.smithi169.stdout: "last_clean_end": 0, 2023-10-26T01:44:28.917 INFO:teuthology.orchestra.run.smithi169.stdout: "up_from": 13, 2023-10-26T01:44:28.918 INFO:teuthology.orchestra.run.smithi169.stdout: "up_thru": 180, 2023-10-26T01:44:28.918 INFO:teuthology.orchestra.run.smithi169.stdout: "down_at": 0, 2023-10-26T01:44:28.918 INFO:teuthology.orchestra.run.smithi169.stdout: "lost_at": 0, 2023-10-26T01:44:28.918 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addrs": { 2023-10-26T01:44:28.919 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:28.919 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.919 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:28.919 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6821", 2023-10-26T01:44:28.919 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:44:28.920 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.920 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.920 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:28.920 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6822", 2023-10-26T01:44:28.921 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:44:28.921 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:28.921 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:28.921 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.921 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addrs": { 2023-10-26T01:44:28.922 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:28.922 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.922 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:28.922 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6824", 2023-10-26T01:44:28.922 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:44:28.923 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.923 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.923 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:28.923 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6826", 2023-10-26T01:44:28.923 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:44:28.924 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:28.924 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:28.924 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.924 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addrs": { 2023-10-26T01:44:28.924 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:28.925 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.925 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:28.925 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6830", 2023-10-26T01:44:28.925 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:44:28.925 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.925 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.926 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:28.926 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6831", 2023-10-26T01:44:28.926 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:44:28.926 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:28.926 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:28.927 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.927 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addrs": { 2023-10-26T01:44:28.927 INFO:teuthology.orchestra.run.smithi169.stdout: "addrvec": [ 2023-10-26T01:44:28.927 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.927 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v2", 2023-10-26T01:44:28.928 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6827", 2023-10-26T01:44:28.928 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:44:28.928 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.928 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.928 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2023-10-26T01:44:28.929 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "172.21.15.170:6829", 2023-10-26T01:44:28.929 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 1115930619 2023-10-26T01:44:28.929 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:28.929 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:28.929 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.930 INFO:teuthology.orchestra.run.smithi169.stdout: "public_addr": "172.21.15.170:6822/1115930619", 2023-10-26T01:44:28.930 INFO:teuthology.orchestra.run.smithi169.stdout: "cluster_addr": "172.21.15.170:6826/1115930619", 2023-10-26T01:44:28.930 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_back_addr": "172.21.15.170:6831/1115930619", 2023-10-26T01:44:28.930 INFO:teuthology.orchestra.run.smithi169.stdout: "heartbeat_front_addr": "172.21.15.170:6829/1115930619", 2023-10-26T01:44:28.930 INFO:teuthology.orchestra.run.smithi169.stdout: "state": [ 2023-10-26T01:44:28.931 INFO:teuthology.orchestra.run.smithi169.stdout: "exists", 2023-10-26T01:44:28.931 INFO:teuthology.orchestra.run.smithi169.stdout: "up" 2023-10-26T01:44:28.931 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2023-10-26T01:44:28.931 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:28.931 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:44:28.931 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_xinfo": [ 2023-10-26T01:44:28.932 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.932 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 0, 2023-10-26T01:44:28.932 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:44:28.932 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:44:28.932 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:44:28.933 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:44:28.933 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:44:28.933 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.889675+0000", 2023-10-26T01:44:28.933 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:44:28.933 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.934 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.934 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 1, 2023-10-26T01:44:28.934 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:44:28.934 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:44:28.934 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:44:28.935 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:44:28.935 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:44:28.935 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.864157+0000", 2023-10-26T01:44:28.935 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:44:28.935 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.935 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.936 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 2, 2023-10-26T01:44:28.936 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:44:28.936 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:44:28.936 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:44:28.936 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:44:28.937 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:44:28.937 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.876919+0000", 2023-10-26T01:44:28.937 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:44:28.937 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.937 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.938 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 3, 2023-10-26T01:44:28.938 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:44:28.938 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:44:28.938 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:44:28.938 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:44:28.939 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:44:28.939 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.974541+0000", 2023-10-26T01:44:28.939 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:44:28.939 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.939 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.939 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 4, 2023-10-26T01:44:28.940 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:44:28.940 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:44:28.940 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:44:28.940 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:44:28.940 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:44:28.941 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.969696+0000", 2023-10-26T01:44:28.941 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:44:28.941 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.941 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.941 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 5, 2023-10-26T01:44:28.941 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:44:28.942 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:44:28.942 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:44:28.942 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:44:28.942 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:44:28.942 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.859703+0000", 2023-10-26T01:44:28.943 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:44:28.943 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.943 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.943 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 6, 2023-10-26T01:44:28.943 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:44:28.944 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:44:28.944 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:44:28.944 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:44:28.944 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:44:28.944 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.867917+0000", 2023-10-26T01:44:28.945 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:44:28.945 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.945 INFO:teuthology.orchestra.run.smithi169.stdout: { 2023-10-26T01:44:28.945 INFO:teuthology.orchestra.run.smithi169.stdout: "osd": 7, 2023-10-26T01:44:28.945 INFO:teuthology.orchestra.run.smithi169.stdout: "down_stamp": "0.000000", 2023-10-26T01:44:28.946 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_probability": 0, 2023-10-26T01:44:28.946 INFO:teuthology.orchestra.run.smithi169.stdout: "laggy_interval": 0, 2023-10-26T01:44:28.946 INFO:teuthology.orchestra.run.smithi169.stdout: "features": 4540138322906710015, 2023-10-26T01:44:28.946 INFO:teuthology.orchestra.run.smithi169.stdout: "old_weight": 0, 2023-10-26T01:44:28.946 INFO:teuthology.orchestra.run.smithi169.stdout: "last_purged_snaps_scrub": "2023-10-26T01:30:28.898605+0000", 2023-10-26T01:44:28.947 INFO:teuthology.orchestra.run.smithi169.stdout: "dead_epoch": 0 2023-10-26T01:44:28.947 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:28.947 INFO:teuthology.orchestra.run.smithi169.stdout: ], 2023-10-26T01:44:28.947 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_upmap": [], 2023-10-26T01:44:28.947 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_upmap_items": [], 2023-10-26T01:44:28.948 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_upmap_primaries": [], 2023-10-26T01:44:28.948 INFO:teuthology.orchestra.run.smithi169.stdout: "pg_temp": [], 2023-10-26T01:44:28.948 INFO:teuthology.orchestra.run.smithi169.stdout: "primary_temp": [], 2023-10-26T01:44:28.948 INFO:teuthology.orchestra.run.smithi169.stdout: "blocklist": {}, 2023-10-26T01:44:28.948 INFO:teuthology.orchestra.run.smithi169.stdout: "range_blocklist": {}, 2023-10-26T01:44:28.949 INFO:teuthology.orchestra.run.smithi169.stdout: "erasure_code_profiles": { 2023-10-26T01:44:28.949 INFO:teuthology.orchestra.run.smithi169.stdout: "default": { 2023-10-26T01:44:28.949 INFO:teuthology.orchestra.run.smithi169.stdout: "crush-failure-domain": "osd", 2023-10-26T01:44:28.949 INFO:teuthology.orchestra.run.smithi169.stdout: "k": "2", 2023-10-26T01:44:28.949 INFO:teuthology.orchestra.run.smithi169.stdout: "m": "1", 2023-10-26T01:44:28.950 INFO:teuthology.orchestra.run.smithi169.stdout: "plugin": "jerasure", 2023-10-26T01:44:28.950 INFO:teuthology.orchestra.run.smithi169.stdout: "technique": "reed_sol_van" 2023-10-26T01:44:28.950 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:28.950 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2023-10-26T01:44:28.951 INFO:teuthology.orchestra.run.smithi169.stdout: "removed_snaps_queue": [], 2023-10-26T01:44:28.951 INFO:teuthology.orchestra.run.smithi169.stdout: "new_removed_snaps": [], 2023-10-26T01:44:28.951 INFO:teuthology.orchestra.run.smithi169.stdout: "new_purged_snaps": [], 2023-10-26T01:44:28.951 INFO:teuthology.orchestra.run.smithi169.stdout: "crush_node_flags": {}, 2023-10-26T01:44:28.951 INFO:teuthology.orchestra.run.smithi169.stdout: "device_class_flags": {}, 2023-10-26T01:44:28.952 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_mode": { 2023-10-26T01:44:28.952 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_mode_enabled": false, 2023-10-26T01:44:28.952 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_bucket_count": 0, 2023-10-26T01:44:28.952 INFO:teuthology.orchestra.run.smithi169.stdout: "degraded_stretch_mode": 0, 2023-10-26T01:44:28.952 INFO:teuthology.orchestra.run.smithi169.stdout: "recovering_stretch_mode": 0, 2023-10-26T01:44:28.952 INFO:teuthology.orchestra.run.smithi169.stdout: "stretch_mode_bucket": 0 2023-10-26T01:44:28.953 INFO:teuthology.orchestra.run.smithi169.stdout: } 2023-10-26T01:44:28.953 INFO:teuthology.orchestra.run.smithi169.stdout:} 2023-10-26T01:44:28.955 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:28.750+0000 7ff868ad7640 1 -- 172.21.15.169:0/1032266273 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff84406ae40 msgr2=0x7ff84406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:28.955 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:28.750+0000 7ff868ad7640 1 --2- 172.21.15.169:0/1032266273 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff84406ae40 0x7ff84406d300 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7ff854036000 tx=0x7ff854004730 comp rx=0 tx=0).stop 2023-10-26T01:44:28.955 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:28.750+0000 7ff868ad7640 1 -- 172.21.15.169:0/1032266273 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff86406cae0 msgr2=0x7ff8640b0030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:28.955 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:28.750+0000 7ff868ad7640 1 --2- 172.21.15.169:0/1032266273 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff86406cae0 0x7ff8640b0030 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7ff84c00d950 tx=0x7ff84c00de10 comp rx=0 tx=0).stop 2023-10-26T01:44:28.955 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:28.750+0000 7ff868ad7640 1 -- 172.21.15.169:0/1032266273 shutdown_connections 2023-10-26T01:44:28.956 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:28.750+0000 7ff868ad7640 1 --2- 172.21.15.169:0/1032266273 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff84406ae40 0x7ff84406d300 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:28.956 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:28.750+0000 7ff868ad7640 1 --2- 172.21.15.169:0/1032266273 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff86406cae0 0x7ff8640b0030 unknown :-1 s=CLOSED pgs=1898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:28.956 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:28.750+0000 7ff868ad7640 1 --2- 172.21.15.169:0/1032266273 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff8640cca30 0x7ff8640c6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:28.956 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:28.750+0000 7ff868ad7640 1 --2- 172.21.15.169:0/1032266273 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff86406a990 0x7ff864146ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:28.956 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:28.750+0000 7ff868ad7640 1 -- 172.21.15.169:0/1032266273 >> 172.21.15.169:0/1032266273 conn(0x7ff8640aedd0 msgr2=0x7ff864158cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:28.957 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:28.750+0000 7ff868ad7640 1 -- 172.21.15.169:0/1032266273 shutdown_connections 2023-10-26T01:44:28.957 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:28.750+0000 7ff868ad7640 1 -- 172.21.15.169:0/1032266273 wait complete. 2023-10-26T01:44:28.958 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T01:44:29.186 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:29.183+0000 7fee75e90640 1 -- 172.21.15.169:0/1869575594 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fee700cdbb0 msgr2=0x7fee700b07d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:29.186 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:29.183+0000 7fee75e90640 1 --2- 172.21.15.169:0/1869575594 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fee700cdbb0 0x7fee700b07d0 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7fee7006c960 tx=0x7fee6802c2f0 comp rx=0 tx=0).stop 2023-10-26T01:44:29.186 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:29.183+0000 7fee75e90640 1 -- 172.21.15.169:0/1869575594 shutdown_connections 2023-10-26T01:44:29.187 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:29.183+0000 7fee75e90640 1 --2- 172.21.15.169:0/1869575594 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fee700cdbb0 0x7fee700b07d0 unknown :-1 s=CLOSED pgs=1900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:29.187 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:29.183+0000 7fee75e90640 1 --2- 172.21.15.169:0/1869575594 >> v2:172.21.15.170:3300/0 conn(0x7fee7006ed30 0x7fee700cd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:29.187 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:29.183+0000 7fee75e90640 1 -- 172.21.15.169:0/1869575594 >> 172.21.15.169:0/1869575594 conn(0x7fee70069930 msgr2=0x7fee70069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:29.187 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:29.183+0000 7fee75e90640 1 -- 172.21.15.169:0/1869575594 shutdown_connections 2023-10-26T01:44:29.187 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:29.183+0000 7fee75e90640 1 -- 172.21.15.169:0/1869575594 wait complete. 2023-10-26T01:44:29.188 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:29.185+0000 7fee75e90640 1 Processor -- start 2023-10-26T01:44:29.188 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:29.185+0000 7fee75e90640 1 -- start start 2023-10-26T01:44:29.190 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:29.186+0000 7fee75e90640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fee7006ed30 0x7fee7014a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:29.190 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:29.186+0000 7fee75e90640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fee700cdbb0 0x7fee701482b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:29.190 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:29.186+0000 7fee75e90640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fee7015b610 0x7fee70158d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:29.190 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:29.186+0000 7fee75e90640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee70158910 con 0x7fee7006ed30 2023-10-26T01:44:29.191 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:29.186+0000 7fee75e90640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee7014b270 con 0x7fee700cdbb0 2023-10-26T01:44:29.191 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:29.186+0000 7fee75e90640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee7014acb0 con 0x7fee7015b610 2023-10-26T01:44:29.191 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:29.187+0000 7fee6f7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fee7006ed30 0x7fee7014a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:29.191 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:29.187+0000 7fee6f7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fee7006ed30 0x7fee7014a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:48700/0 (socket says 172.21.15.169:48700) 2023-10-26T01:44:29.192 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:29.187+0000 7fee6f7fe640 1 -- 172.21.15.169:0/3352501122 learned_addr learned my addr 172.21.15.169:0/3352501122 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:44:29.192 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:29.187+0000 7fee6ffff640 1 --2- 172.21.15.169:0/3352501122 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fee7015b610 0x7fee70158d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:29.192 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:29.187+0000 7fee67fff640 1 --2- 172.21.15.169:0/3352501122 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fee700cdbb0 0x7fee701482b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:29.192 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:29.187+0000 7fee6f7fe640 1 -- 172.21.15.169:0/3352501122 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fee7015b610 msgr2=0x7fee70158d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:29.192 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:29.187+0000 7fee6f7fe640 1 --2- 172.21.15.169:0/3352501122 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fee7015b610 0x7fee70158d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:29.193 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:29.187+0000 7fee6f7fe640 1 -- 172.21.15.169:0/3352501122 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fee700cdbb0 msgr2=0x7fee701482b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:29.193 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:29.187+0000 7fee6f7fe640 1 --2- 172.21.15.169:0/3352501122 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fee700cdbb0 0x7fee701482b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:29.193 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:29.187+0000 7fee6f7fe640 1 -- 172.21.15.169:0/3352501122 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee68009900 con 0x7fee7006ed30 2023-10-26T01:44:29.193 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:29.187+0000 7fee67fff640 1 --2- 172.21.15.169:0/3352501122 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fee700cdbb0 0x7fee701482b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:44:29.194 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:29.188+0000 7fee6f7fe640 1 --2- 172.21.15.169:0/3352501122 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fee7006ed30 0x7fee7014a2f0 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7fee6000c400 tx=0x7fee6000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:29.195 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:29.188+0000 7fee6cff9640 1 -- 172.21.15.169:0/3352501122 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee600052d0 con 0x7fee7006ed30 2023-10-26T01:44:29.195 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:29.188+0000 7fee6cff9640 1 -- 172.21.15.169:0/3352501122 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fee600035f0 con 0x7fee7006ed30 2023-10-26T01:44:29.195 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:29.188+0000 7fee75e90640 1 -- 172.21.15.169:0/3352501122 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee7014af90 con 0x7fee7006ed30 2023-10-26T01:44:29.195 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:29.188+0000 7fee6cff9640 1 -- 172.21.15.169:0/3352501122 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee60003a00 con 0x7fee7006ed30 2023-10-26T01:44:29.195 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:29.188+0000 7fee75e90640 1 -- 172.21.15.169:0/3352501122 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee7014a830 con 0x7fee7006ed30 2023-10-26T01:44:29.196 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:29.189+0000 7fee75e90640 1 -- 172.21.15.169:0/3352501122 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee7014aa20 con 0x7fee7006ed30 2023-10-26T01:44:29.196 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:29.190+0000 7fee6cff9640 1 -- 172.21.15.169:0/3352501122 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fee60003b60 con 0x7fee7006ed30 2023-10-26T01:44:29.196 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:29.191+0000 7fee6cff9640 1 --2- 172.21.15.169:0/3352501122 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fee5006ae70 0x7fee5006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:29.196 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:29.191+0000 7fee6cff9640 1 -- 172.21.15.169:0/3352501122 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fee60088d10 con 0x7fee7006ed30 2023-10-26T01:44:29.196 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:29.192+0000 7fee67fff640 1 --2- 172.21.15.169:0/3352501122 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fee5006ae70 0x7fee5006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:29.197 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:29.193+0000 7fee67fff640 1 --2- 172.21.15.169:0/3352501122 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fee5006ae70 0x7fee5006d330 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fee700b34c0 tx=0x7fee58005d30 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:29.198 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:29.196+0000 7fee6cff9640 1 -- 172.21.15.169:0/3352501122 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fee60053e50 con 0x7fee7006ed30 2023-10-26T01:44:29.419 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:29.417+0000 7fee75e90640 1 -- 172.21.15.169:0/3352501122 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fee7004db10 con 0x7fee7006ed30 2023-10-26T01:44:29.420 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:29.418+0000 7fee6cff9640 1 -- 172.21.15.169:0/3352501122 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) v1 ==== 94+0+4920 (secure 0 0 0) 0x7fee60057470 con 0x7fee7006ed30 2023-10-26T01:44:29.421 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:44:29.421 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":94,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6484,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/195700049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":195700049},{"type":"v1","addr":"172.21.15.169:6835","nonce":195700049}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":89},{"gid":6493,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/1167696300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1167696300},{"type":"v1","addr":"172.21.15.170:6836","nonce":1167696300}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":89},{"gid":9111,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/22729060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":22729060},{"type":"v1","addr":"172.21.15.169:6834","nonce":22729060}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":89}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T01:43:49.711084+0000","modified":"2023-10-26T01:43:52.816785+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9123},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9123":{"gid":9123,"name":"d","rank":0,"incarnation":90,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6837/2205518515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":2205518515},{"type":"v1","addr":"172.21.15.170:6837","nonce":2205518515}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2023-10-26T01:44:29.421 INFO:teuthology.orchestra.run.smithi169.stderr:dumped fsmap epoch 94 2023-10-26T01:44:29.423 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:29.420+0000 7fee75e90640 1 -- 172.21.15.169:0/3352501122 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fee5006ae70 msgr2=0x7fee5006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:29.423 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:29.420+0000 7fee75e90640 1 --2- 172.21.15.169:0/3352501122 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fee5006ae70 0x7fee5006d330 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fee700b34c0 tx=0x7fee58005d30 comp rx=0 tx=0).stop 2023-10-26T01:44:29.423 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:29.420+0000 7fee75e90640 1 -- 172.21.15.169:0/3352501122 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fee7006ed30 msgr2=0x7fee7014a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:29.424 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:29.420+0000 7fee75e90640 1 --2- 172.21.15.169:0/3352501122 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fee7006ed30 0x7fee7014a2f0 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7fee6000c400 tx=0x7fee6000c8c0 comp rx=0 tx=0).stop 2023-10-26T01:44:29.424 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:29.420+0000 7fee75e90640 1 -- 172.21.15.169:0/3352501122 shutdown_connections 2023-10-26T01:44:29.424 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:29.420+0000 7fee75e90640 1 --2- 172.21.15.169:0/3352501122 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fee5006ae70 0x7fee5006d330 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:29.424 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:29.420+0000 7fee75e90640 1 --2- 172.21.15.169:0/3352501122 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fee7015b610 0x7fee70158d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:29.424 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:29.420+0000 7fee75e90640 1 --2- 172.21.15.169:0/3352501122 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fee700cdbb0 0x7fee701482b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:29.425 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:29.420+0000 7fee75e90640 1 --2- 172.21.15.169:0/3352501122 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fee7006ed30 0x7fee7014a2f0 unknown :-1 s=CLOSED pgs=1901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:29.425 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:29.420+0000 7fee75e90640 1 -- 172.21.15.169:0/3352501122 >> 172.21.15.169:0/3352501122 conn(0x7fee70069930 msgr2=0x7fee700b1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:29.425 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:29.420+0000 7fee75e90640 1 -- 172.21.15.169:0/3352501122 shutdown_connections 2023-10-26T01:44:29.425 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:29.421+0000 7fee75e90640 1 -- 172.21.15.169:0/3352501122 wait complete. 2023-10-26T01:44:29.441 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json status 2023-10-26T01:44:29.681 INFO:teuthology.orchestra.run.smithi170.stdout:{"cluster_fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","whoami":0,"id":9123,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":39.351341513999998,"mdsmap_epoch":94,"osdmap_epoch":185,"osdmap_epoch_barrier":184,"uptime":46.028770887} 2023-10-26T01:44:29.681 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "30203b28-ed8a-44e8-a9a6-a9bf25f980f1", "fs_name": "cephfs", "id": 9123, "mdsmap_epoch": 94, "osdmap_epoch": 185, "osdmap_epoch_barrier": 184, "rank_uptime": 39.351341514, "state": "up:active", "uptime": 46.028770887, "want_state": "up:active", "whoami": 0 } 2023-10-26T01:44:29.682 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-26T01:44:29.682 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:44:29.683 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:44:29.743 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:44:29.743 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file bs=1M conv=fdatasync count=50 seek=0) 2023-10-26T01:44:30.220 INFO:teuthology.orchestra.run.smithi169.stderr:50+0 records in 2023-10-26T01:44:30.220 INFO:teuthology.orchestra.run.smithi169.stderr:50+0 records out 2023-10-26T01:44:30.220 INFO:teuthology.orchestra.run.smithi169.stderr:52428800 bytes (52 MB, 50 MiB) copied, 0.417129 s, 126 MB/s 2023-10-26T01:44:30.221 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-26T01:44:30.222 DEBUG:teuthology.orchestra.run.smithi169:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-26T01:44:30.326 DEBUG:teuthology.orchestra.run.smithi170:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-26T01:44:30.443 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-26T01:44:30.444 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:44:30.444 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:44:30.504 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:44:30.505 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:44:30.561 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:44:30.782 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:30.779+0000 7fe3ba000640 1 -- 172.21.15.169:0/125153785 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe3b4158850 msgr2=0x7fe3b40cd720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:30.783 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:30.779+0000 7fe3ba000640 1 --2- 172.21.15.169:0/125153785 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe3b4158850 0x7fe3b40cd720 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7fe3b40accf0 tx=0x7fe39c02bd90 comp rx=0 tx=0).stop 2023-10-26T01:44:30.783 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:30.779+0000 7fe3ba000640 1 -- 172.21.15.169:0/125153785 shutdown_connections 2023-10-26T01:44:30.783 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:30.779+0000 7fe3ba000640 1 --2- 172.21.15.169:0/125153785 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe3b40cdc60 0x7fe3b406ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:30.783 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:30.779+0000 7fe3ba000640 1 --2- 172.21.15.169:0/125153785 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe3b4158850 0x7fe3b40cd720 unknown :-1 s=CLOSED pgs=1902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:30.783 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:30.779+0000 7fe3ba000640 1 --2- 172.21.15.169:0/125153785 >> v2:172.21.15.170:3300/0 conn(0x7fe3b414e790 0x7fe3b40aab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:30.784 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:30.779+0000 7fe3ba000640 1 -- 172.21.15.169:0/125153785 >> 172.21.15.169:0/125153785 conn(0x7fe3b4069930 msgr2=0x7fe3b4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:30.784 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:30.779+0000 7fe3ba000640 1 -- 172.21.15.169:0/125153785 shutdown_connections 2023-10-26T01:44:30.784 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:30.779+0000 7fe3ba000640 1 -- 172.21.15.169:0/125153785 wait complete. 2023-10-26T01:44:30.785 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:30.781+0000 7fe3ba000640 1 Processor -- start 2023-10-26T01:44:30.786 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:30.781+0000 7fe3ba000640 1 -- start start 2023-10-26T01:44:30.786 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:30.782+0000 7fe3ba000640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe3b40cdc60 0x7fe3b406ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:30.786 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:30.782+0000 7fe3ba000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe3b414e790 0x7fe3b4143ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:30.786 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:30.782+0000 7fe3ba000640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe3b4158850 0x7fe3b41569f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:30.786 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:30.782+0000 7fe3ba000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b4146cb0 con 0x7fe3b414e790 2023-10-26T01:44:30.787 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:30.782+0000 7fe3ba000640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b40aeec0 con 0x7fe3b40cdc60 2023-10-26T01:44:30.787 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:30.782+0000 7fe3ba000640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b40afc90 con 0x7fe3b4158850 2023-10-26T01:44:30.787 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:30.782+0000 7fe3b2ffd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe3b414e790 0x7fe3b4143ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:30.787 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:30.782+0000 7fe3b2ffd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe3b414e790 0x7fe3b4143ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:48712/0 (socket says 172.21.15.169:48712) 2023-10-26T01:44:30.787 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:30.782+0000 7fe3b2ffd640 1 -- 172.21.15.169:0/1479672250 learned_addr learned my addr 172.21.15.169:0/1479672250 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:44:30.788 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:30.783+0000 7fe3b37fe640 1 --2- 172.21.15.169:0/1479672250 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe3b40cdc60 0x7fe3b406ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:30.788 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:30.783+0000 7fe3b3fff640 1 --2- 172.21.15.169:0/1479672250 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe3b4158850 0x7fe3b41569f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:30.788 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:30.783+0000 7fe3b2ffd640 1 -- 172.21.15.169:0/1479672250 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe3b4158850 msgr2=0x7fe3b41569f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:30.788 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:30.783+0000 7fe3b2ffd640 1 --2- 172.21.15.169:0/1479672250 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe3b4158850 0x7fe3b41569f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:30.788 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:30.783+0000 7fe3b2ffd640 1 -- 172.21.15.169:0/1479672250 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe3b40cdc60 msgr2=0x7fe3b406ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:30.789 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:30.783+0000 7fe3b2ffd640 1 --2- 172.21.15.169:0/1479672250 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe3b40cdc60 0x7fe3b406ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:30.789 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:30.783+0000 7fe3b2ffd640 1 -- 172.21.15.169:0/1479672250 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe39c009560 con 0x7fe3b414e790 2023-10-26T01:44:30.790 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:30.783+0000 7fe3b2ffd640 1 --2- 172.21.15.169:0/1479672250 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe3b414e790 0x7fe3b4143ba0 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7fe39c0099c0 tx=0x7fe39c02dd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:30.790 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:30.784+0000 7fe3abfff640 1 -- 172.21.15.169:0/1479672250 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe39c003800 con 0x7fe3b414e790 2023-10-26T01:44:30.790 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:30.784+0000 7fe3abfff640 1 -- 172.21.15.169:0/1479672250 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe39c003a70 con 0x7fe3b414e790 2023-10-26T01:44:30.791 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:30.784+0000 7fe3ba000640 1 -- 172.21.15.169:0/1479672250 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3b41440e0 con 0x7fe3b414e790 2023-10-26T01:44:30.791 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:30.784+0000 7fe3abfff640 1 -- 172.21.15.169:0/1479672250 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe39c003e80 con 0x7fe3b414e790 2023-10-26T01:44:30.791 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:30.784+0000 7fe3ba000640 1 -- 172.21.15.169:0/1479672250 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3b40af3f0 con 0x7fe3b414e790 2023-10-26T01:44:30.791 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:30.785+0000 7fe3ba000640 1 -- 172.21.15.169:0/1479672250 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3b40aa1c0 con 0x7fe3b414e790 2023-10-26T01:44:30.791 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:30.786+0000 7fe3abfff640 1 -- 172.21.15.169:0/1479672250 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fe39c03e450 con 0x7fe3b414e790 2023-10-26T01:44:30.792 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:30.787+0000 7fe3abfff640 1 --2- 172.21.15.169:0/1479672250 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe39406ae40 0x7fe39406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:30.792 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:30.788+0000 7fe3abfff640 1 -- 172.21.15.169:0/1479672250 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe39c0ac280 con 0x7fe3b414e790 2023-10-26T01:44:30.792 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:30.788+0000 7fe3b37fe640 1 --2- 172.21.15.169:0/1479672250 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe39406ae40 0x7fe39406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:30.793 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:30.789+0000 7fe3b37fe640 1 --2- 172.21.15.169:0/1479672250 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe39406ae40 0x7fe39406d300 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fe3a4005c40 tx=0x7fe3a400a780 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:30.794 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:30.792+0000 7fe3abfff640 1 -- 172.21.15.169:0/1479672250 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe39c077440 con 0x7fe3b414e790 2023-10-26T01:44:30.989 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:30.987+0000 7fe3ba000640 1 -- 172.21.15.169:0/1479672250 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe3b404db10 con 0x7fe3b414e790 2023-10-26T01:44:30.991 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:30.988+0000 7fe3abfff640 1 -- 172.21.15.169:0/1479672250 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v185) v1 ==== 74+0+16627 (secure 0 0 0) 0x7fe39c07aa60 con 0x7fe3b414e790 2023-10-26T01:44:30.992 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:44:30.992 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":185,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:44:23.200104+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2023-10-26T01:43:47.501296+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"181","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2023-10-26T01:43:48.737507+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"185","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:44:30.994 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:30.992+0000 7fe3ba000640 1 -- 172.21.15.169:0/1479672250 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe39406ae40 msgr2=0x7fe39406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:30.994 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:30.992+0000 7fe3ba000640 1 --2- 172.21.15.169:0/1479672250 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe39406ae40 0x7fe39406d300 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fe3a4005c40 tx=0x7fe3a400a780 comp rx=0 tx=0).stop 2023-10-26T01:44:30.994 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:30.992+0000 7fe3ba000640 1 -- 172.21.15.169:0/1479672250 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe3b414e790 msgr2=0x7fe3b4143ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:30.994 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:30.992+0000 7fe3ba000640 1 --2- 172.21.15.169:0/1479672250 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe3b414e790 0x7fe3b4143ba0 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7fe39c0099c0 tx=0x7fe39c02dd60 comp rx=0 tx=0).stop 2023-10-26T01:44:30.995 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:30.992+0000 7fe3ba000640 1 -- 172.21.15.169:0/1479672250 shutdown_connections 2023-10-26T01:44:30.995 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:30.992+0000 7fe3ba000640 1 --2- 172.21.15.169:0/1479672250 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fe39406ae40 0x7fe39406d300 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:30.995 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:30.992+0000 7fe3ba000640 1 --2- 172.21.15.169:0/1479672250 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe3b414e790 0x7fe3b4143ba0 unknown :-1 s=CLOSED pgs=1903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:30.996 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:30.992+0000 7fe3ba000640 1 --2- 172.21.15.169:0/1479672250 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe3b4158850 0x7fe3b41569f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:30.996 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:30.992+0000 7fe3ba000640 1 --2- 172.21.15.169:0/1479672250 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe3b40cdc60 0x7fe3b406ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:30.996 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:30.992+0000 7fe3ba000640 1 -- 172.21.15.169:0/1479672250 >> 172.21.15.169:0/1479672250 conn(0x7fe3b4069930 msgr2=0x7fe3b40adbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:30.996 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:30.992+0000 7fe3ba000640 1 -- 172.21.15.169:0/1479672250 shutdown_connections 2023-10-26T01:44:30.996 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:30.992+0000 7fe3ba000640 1 -- 172.21.15.169:0/1479672250 wait complete. 2023-10-26T01:44:31.014 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T01:30:32.200808+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '109', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 22, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-26T01:43:47.501296+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '181', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 23, 'pool_name': 'cephfs_data', 'create_time': '2023-10-26T01:43:48.737507+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '185', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T01:44:31.015 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T01:44:31.238 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:31.235+0000 7f7d05875640 1 -- 172.21.15.169:0/3364250849 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d000cde70 msgr2=0x7f7d00158300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:31.239 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:31.235+0000 7f7d05875640 1 --2- 172.21.15.169:0/3364250849 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d000cde70 0x7f7d00158300 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7f7cf8005fd0 tx=0x7f7cf802c2d0 comp rx=0 tx=0).stop 2023-10-26T01:44:31.239 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:31.235+0000 7f7d05875640 1 -- 172.21.15.169:0/3364250849 shutdown_connections 2023-10-26T01:44:31.239 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:31.235+0000 7f7d05875640 1 --2- 172.21.15.169:0/3364250849 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d000cde70 0x7f7d00158300 unknown :-1 s=CLOSED pgs=1905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:31.239 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:31.235+0000 7f7d05875640 1 --2- 172.21.15.169:0/3364250849 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7d000ab070 0x7f7d000ac870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:31.240 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:31.235+0000 7f7d05875640 1 -- 172.21.15.169:0/3364250849 >> 172.21.15.169:0/3364250849 conn(0x7f7d00069930 msgr2=0x7f7d00069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:31.240 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:31.236+0000 7f7d05875640 1 -- 172.21.15.169:0/3364250849 shutdown_connections 2023-10-26T01:44:31.240 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:31.236+0000 7f7d05875640 1 -- 172.21.15.169:0/3364250849 wait complete. 2023-10-26T01:44:31.241 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:31.237+0000 7f7d05875640 1 Processor -- start 2023-10-26T01:44:31.241 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:31.238+0000 7f7d05875640 1 -- start start 2023-10-26T01:44:31.243 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:31.238+0000 7f7d05875640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d000ab070 0x7f7d0006eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:31.243 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:31.239+0000 7f7d05875640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7d000cde70 0x7f7d001f1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:31.243 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:31.239+0000 7f7d05875640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d00071390 0x7f7d001468c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:31.243 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:31.239+0000 7f7d05875640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d001529b0 con 0x7f7d00071390 2023-10-26T01:44:31.244 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:31.239+0000 7f7d05875640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d00152f70 con 0x7f7d000ab070 2023-10-26T01:44:31.244 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:31.239+0000 7f7d05875640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d001f1bc0 con 0x7f7d000cde70 2023-10-26T01:44:31.244 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:31.239+0000 7f7cff7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d00071390 0x7f7d001468c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:31.244 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:31.239+0000 7f7cff7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d00071390 0x7f7d001468c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:48730/0 (socket says 172.21.15.169:48730) 2023-10-26T01:44:31.244 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:31.239+0000 7f7cff7fe640 1 -- 172.21.15.169:0/3085382344 learned_addr learned my addr 172.21.15.169:0/3085382344 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:44:31.245 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:31.239+0000 7f7cfeffd640 1 --2- 172.21.15.169:0/3085382344 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d000ab070 0x7f7d0006eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:31.245 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:31.239+0000 7f7cff7fe640 1 -- 172.21.15.169:0/3085382344 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7d000cde70 msgr2=0x7f7d001f1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:31.245 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:31.239+0000 7f7cff7fe640 1 --2- 172.21.15.169:0/3085382344 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7d000cde70 0x7f7d001f1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:31.245 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:31.239+0000 7f7cff7fe640 1 -- 172.21.15.169:0/3085382344 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d000ab070 msgr2=0x7f7d0006eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:31.245 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:31.239+0000 7f7cff7fe640 1 --2- 172.21.15.169:0/3085382344 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d000ab070 0x7f7d0006eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:31.245 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:31.239+0000 7f7cff7fe640 1 -- 172.21.15.169:0/3085382344 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cf8009900 con 0x7f7d00071390 2023-10-26T01:44:31.246 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:31.240+0000 7f7cff7fe640 1 --2- 172.21.15.169:0/3085382344 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d00071390 0x7f7d001468c0 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7f7cf8037040 tx=0x7f7cf8003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:31.246 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:31.240+0000 7f7ce7fff640 1 -- 172.21.15.169:0/3085382344 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf8004240 con 0x7f7d00071390 2023-10-26T01:44:31.246 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:31.240+0000 7f7d05875640 1 -- 172.21.15.169:0/3085382344 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d00071bc0 con 0x7f7d00071390 2023-10-26T01:44:31.246 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:31.240+0000 7f7ce7fff640 1 -- 172.21.15.169:0/3085382344 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7cf8004490 con 0x7f7d00071390 2023-10-26T01:44:31.247 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:31.240+0000 7f7ce7fff640 1 -- 172.21.15.169:0/3085382344 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf8004860 con 0x7f7d00071390 2023-10-26T01:44:31.247 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:31.240+0000 7f7d05875640 1 -- 172.21.15.169:0/3085382344 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d00152c00 con 0x7f7d00071390 2023-10-26T01:44:31.248 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:31.242+0000 7f7d05875640 1 -- 172.21.15.169:0/3085382344 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d00066af0 con 0x7f7d00071390 2023-10-26T01:44:31.248 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:31.242+0000 7f7ce7fff640 1 -- 172.21.15.169:0/3085382344 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f7cf803b040 con 0x7f7d00071390 2023-10-26T01:44:31.248 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:31.243+0000 7f7ce7fff640 1 --2- 172.21.15.169:0/3085382344 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7cec06ae40 0x7f7cec06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:31.248 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:31.243+0000 7f7ce7fff640 1 -- 172.21.15.169:0/3085382344 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7cf80ac4d0 con 0x7f7d00071390 2023-10-26T01:44:31.248 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:31.244+0000 7f7cfeffd640 1 --2- 172.21.15.169:0/3085382344 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7cec06ae40 0x7f7cec06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:31.249 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:31.245+0000 7f7cfeffd640 1 --2- 172.21.15.169:0/3085382344 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7cec06ae40 0x7f7cec06d300 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f7cf0000f60 tx=0x7f7cf0009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:31.250 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:31.248+0000 7f7ce7fff640 1 -- 172.21.15.169:0/3085382344 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7cf8077590 con 0x7f7d00071390 2023-10-26T01:44:31.443 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:31.440+0000 7f7d05875640 1 -- 172.21.15.169:0/3085382344 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7d0004da70 con 0x7f7d00071390 2023-10-26T01:44:31.444 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:31.441+0000 7f7ce7fff640 1 -- 172.21.15.169:0/3085382344 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v185) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7cf807abb0 con 0x7f7d00071390 2023-10-26T01:44:31.444 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 1 2023-10-26T01:44:31.447 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:31.443+0000 7f7d05875640 1 -- 172.21.15.169:0/3085382344 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7cec06ae40 msgr2=0x7f7cec06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:31.447 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:31.444+0000 7f7d05875640 1 --2- 172.21.15.169:0/3085382344 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7cec06ae40 0x7f7cec06d300 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f7cf0000f60 tx=0x7f7cf0009040 comp rx=0 tx=0).stop 2023-10-26T01:44:31.447 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:31.444+0000 7f7d05875640 1 -- 172.21.15.169:0/3085382344 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d00071390 msgr2=0x7f7d001468c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:31.447 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:31.444+0000 7f7d05875640 1 --2- 172.21.15.169:0/3085382344 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d00071390 0x7f7d001468c0 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7f7cf8037040 tx=0x7f7cf8003ba0 comp rx=0 tx=0).stop 2023-10-26T01:44:31.448 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:31.444+0000 7f7d05875640 1 -- 172.21.15.169:0/3085382344 shutdown_connections 2023-10-26T01:44:31.448 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:31.444+0000 7f7d05875640 1 --2- 172.21.15.169:0/3085382344 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f7cec06ae40 0x7f7cec06d300 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:31.448 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:31.444+0000 7f7d05875640 1 --2- 172.21.15.169:0/3085382344 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d00071390 0x7f7d001468c0 unknown :-1 s=CLOSED pgs=1906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:31.448 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:31.444+0000 7f7d05875640 1 --2- 172.21.15.169:0/3085382344 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7d000cde70 0x7f7d001f1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:31.448 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:31.444+0000 7f7d05875640 1 --2- 172.21.15.169:0/3085382344 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d000ab070 0x7f7d0006eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:31.449 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:31.444+0000 7f7d05875640 1 -- 172.21.15.169:0/3085382344 >> 172.21.15.169:0/3085382344 conn(0x7f7d00069930 msgr2=0x7f7d00156e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:31.449 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:31.444+0000 7f7d05875640 1 -- 172.21.15.169:0/3085382344 shutdown_connections 2023-10-26T01:44:31.449 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:31.444+0000 7f7d05875640 1 -- 172.21.15.169:0/3085382344 wait complete. 2023-10-26T01:44:31.464 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-26T01:44:31.687 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:31.684+0000 7f9d168bb640 1 -- 172.21.15.169:0/918106465 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9cfc003970 msgr2=0x7f9cfc003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:31.687 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:31.684+0000 7f9d168bb640 1 --2- 172.21.15.169:0/918106465 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9cfc003970 0x7f9cfc003d90 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7f9d00005fd0 tx=0x7f9cfc0057f0 comp rx=0 tx=0).stop 2023-10-26T01:44:31.687 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:31.684+0000 7f9d14e58640 1 -- 172.21.15.169:0/918106465 reap_dead start 2023-10-26T01:44:31.688 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:31.684+0000 7f9d168bb640 1 -- 172.21.15.169:0/918106465 shutdown_connections 2023-10-26T01:44:31.688 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:31.684+0000 7f9d168bb640 1 -- 172.21.15.169:0/918106465 >> 172.21.15.169:0/918106465 conn(0x7f9d10069930 msgr2=0x7f9d10069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:31.688 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:31.684+0000 7f9d168bb640 1 -- 172.21.15.169:0/918106465 shutdown_connections 2023-10-26T01:44:31.688 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:31.684+0000 7f9d168bb640 1 -- 172.21.15.169:0/918106465 wait complete. 2023-10-26T01:44:31.689 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:31.686+0000 7f9d168bb640 1 Processor -- start 2023-10-26T01:44:31.689 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:31.686+0000 7f9d168bb640 1 -- start start 2023-10-26T01:44:31.691 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:31.687+0000 7f9d168bb640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9cfc003970 0x7f9d10151900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:31.691 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:31.687+0000 7f9d168bb640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9d101e2040 0x7f9d101e24c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:31.691 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:31.687+0000 7f9d168bb640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d100c8cb0 0x7f9d100c9140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:31.691 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:31.687+0000 7f9d168bb640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d100c9680 con 0x7f9d100c8cb0 2023-10-26T01:44:31.692 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:31.687+0000 7f9d168bb640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d101e2e20 con 0x7f9d101e2040 2023-10-26T01:44:31.692 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:31.687+0000 7f9d168bb640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d100c6660 con 0x7f9cfc003970 2023-10-26T01:44:31.692 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:31.687+0000 7f9d14e58640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d100c8cb0 0x7f9d100c9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:31.692 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:31.687+0000 7f9d14e58640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d100c8cb0 0x7f9d100c9140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:48748/0 (socket says 172.21.15.169:48748) 2023-10-26T01:44:31.692 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:31.687+0000 7f9d14e58640 1 -- 172.21.15.169:0/1636459994 learned_addr learned my addr 172.21.15.169:0/1636459994 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:44:31.693 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:31.688+0000 7f9d07fff640 1 --2- 172.21.15.169:0/1636459994 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9d101e2040 0x7f9d101e24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:31.693 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:31.688+0000 7f9d14e58640 1 -- 172.21.15.169:0/1636459994 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9cfc003970 msgr2=0x7f9d10151900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:31.693 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:31.688+0000 7f9d14e58640 1 --2- 172.21.15.169:0/1636459994 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9cfc003970 0x7f9d10151900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:31.693 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:31.688+0000 7f9d14e58640 1 -- 172.21.15.169:0/1636459994 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9d101e2040 msgr2=0x7f9d101e24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:31.693 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:31.688+0000 7f9d14e58640 1 --2- 172.21.15.169:0/1636459994 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9d101e2040 0x7f9d101e24c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:31.693 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:31.688+0000 7f9d14e58640 1 -- 172.21.15.169:0/1636459994 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d1006bc10 con 0x7f9d100c8cb0 2023-10-26T01:44:31.694 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:31.688+0000 7f9d14e58640 1 --2- 172.21.15.169:0/1636459994 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d100c8cb0 0x7f9d100c9140 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7f9d08001350 tx=0x7f9d0800dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:31.694 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:31.688+0000 7f9d0d7fa640 1 -- 172.21.15.169:0/1636459994 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d08015030 con 0x7f9d100c8cb0 2023-10-26T01:44:31.694 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:31.688+0000 7f9d0d7fa640 1 -- 172.21.15.169:0/1636459994 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9d08003e90 con 0x7f9d100c8cb0 2023-10-26T01:44:31.694 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:31.688+0000 7f9d0d7fa640 1 -- 172.21.15.169:0/1636459994 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d0800ba00 con 0x7f9d100c8cb0 2023-10-26T01:44:31.695 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:31.688+0000 7f9d168bb640 1 -- 172.21.15.169:0/1636459994 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d10151e40 con 0x7f9d100c8cb0 2023-10-26T01:44:31.695 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:31.688+0000 7f9d168bb640 1 -- 172.21.15.169:0/1636459994 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d100c68b0 con 0x7f9d100c8cb0 2023-10-26T01:44:31.696 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:31.689+0000 7f9d168bb640 1 -- 172.21.15.169:0/1636459994 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d1004db10 con 0x7f9d100c8cb0 2023-10-26T01:44:31.696 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:31.690+0000 7f9d0d7fa640 1 -- 172.21.15.169:0/1636459994 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f9d08018070 con 0x7f9d100c8cb0 2023-10-26T01:44:31.696 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:31.691+0000 7f9d0d7fa640 1 --2- 172.21.15.169:0/1636459994 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9cf006ae70 0x7f9cf006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:31.696 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:31.691+0000 7f9d0d7fa640 1 -- 172.21.15.169:0/1636459994 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9d08016070 con 0x7f9d100c8cb0 2023-10-26T01:44:31.696 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:31.692+0000 7f9d0ffff640 1 --2- 172.21.15.169:0/1636459994 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9cf006ae70 0x7f9cf006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:31.697 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:31.693+0000 7f9d0ffff640 1 --2- 172.21.15.169:0/1636459994 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9cf006ae70 0x7f9cf006d330 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f9d00036000 tx=0x7f9d00035040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:31.698 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:31.696+0000 7f9d0d7fa640 1 -- 172.21.15.169:0/1636459994 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9d08058370 con 0x7f9d100c8cb0 2023-10-26T01:44:31.889 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:31.886+0000 7f9d168bb640 1 -- 172.21.15.169:0/1636459994 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9d10069f50 con 0x7f9d100c8cb0 2023-10-26T01:44:31.889 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:31.887+0000 7f9d0d7fa640 1 -- 172.21.15.169:0/1636459994 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v185) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9d08057f00 con 0x7f9d100c8cb0 2023-10-26T01:44:31.890 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 32 2023-10-26T01:44:31.892 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:31.889+0000 7f9d168bb640 1 -- 172.21.15.169:0/1636459994 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9cf006ae70 msgr2=0x7f9cf006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:31.892 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:31.889+0000 7f9d168bb640 1 --2- 172.21.15.169:0/1636459994 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9cf006ae70 0x7f9cf006d330 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f9d00036000 tx=0x7f9d00035040 comp rx=0 tx=0).stop 2023-10-26T01:44:31.893 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:31.889+0000 7f9d168bb640 1 -- 172.21.15.169:0/1636459994 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d100c8cb0 msgr2=0x7f9d100c9140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:31.893 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:31.889+0000 7f9d168bb640 1 --2- 172.21.15.169:0/1636459994 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d100c8cb0 0x7f9d100c9140 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7f9d08001350 tx=0x7f9d0800dd90 comp rx=0 tx=0).stop 2023-10-26T01:44:31.893 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:31.890+0000 7f9d168bb640 1 -- 172.21.15.169:0/1636459994 shutdown_connections 2023-10-26T01:44:31.893 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:31.890+0000 7f9d168bb640 1 --2- 172.21.15.169:0/1636459994 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9cf006ae70 0x7f9cf006d330 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:31.893 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:31.890+0000 7f9d168bb640 1 --2- 172.21.15.169:0/1636459994 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d100c8cb0 0x7f9d100c9140 unknown :-1 s=CLOSED pgs=1909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:31.894 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:31.890+0000 7f9d168bb640 1 --2- 172.21.15.169:0/1636459994 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9d101e2040 0x7f9d101e24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:31.894 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:31.890+0000 7f9d168bb640 1 --2- 172.21.15.169:0/1636459994 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9cfc003970 0x7f9d10151900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:31.894 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:31.890+0000 7f9d168bb640 1 -- 172.21.15.169:0/1636459994 >> 172.21.15.169:0/1636459994 conn(0x7f9d10069930 msgr2=0x7f9d10159dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:31.894 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:31.890+0000 7f9d168bb640 1 -- 172.21.15.169:0/1636459994 shutdown_connections 2023-10-26T01:44:31.895 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:31.890+0000 7f9d168bb640 1 -- 172.21.15.169:0/1636459994 wait complete. 2023-10-26T01:44:31.911 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-26T01:44:32.137 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:32.134+0000 7f7068cb4640 1 -- 172.21.15.169:0/571831174 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f70500047c0 msgr2=0x7f7050004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:32.137 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:32.134+0000 7f7068cb4640 1 --2- 172.21.15.169:0/571831174 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f70500047c0 0x7f7050004c40 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f704c006fa0 tx=0x7f70500057f0 comp rx=0 tx=0).stop 2023-10-26T01:44:32.137 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:32.134+0000 7f7062d9d640 1 -- 172.21.15.169:0/571831174 reap_dead start 2023-10-26T01:44:32.138 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:32.134+0000 7f7068cb4640 1 -- 172.21.15.169:0/571831174 shutdown_connections 2023-10-26T01:44:32.138 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:32.134+0000 7f7068cb4640 1 -- 172.21.15.169:0/571831174 >> 172.21.15.169:0/571831174 conn(0x7f70640aefe0 msgr2=0x7f70640af3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:32.138 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:32.134+0000 7f7068cb4640 1 -- 172.21.15.169:0/571831174 shutdown_connections 2023-10-26T01:44:32.138 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:32.134+0000 7f7068cb4640 1 -- 172.21.15.169:0/571831174 wait complete. 2023-10-26T01:44:32.139 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:32.136+0000 7f7068cb4640 1 Processor -- start 2023-10-26T01:44:32.139 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:32.136+0000 7f7068cb4640 1 -- start start 2023-10-26T01:44:32.141 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:32.137+0000 7f7068cb4640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f70500047c0 0x7f706414ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:32.141 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:32.137+0000 7f7068cb4640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f70640b1de0 0x7f70640aa930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:32.141 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:32.137+0000 7f7068cb4640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f70640c8cd0 0x7f70640aae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:32.141 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:32.137+0000 7f7068cb4640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70640c6cc0 con 0x7f70500047c0 2023-10-26T01:44:32.142 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:32.137+0000 7f7068cb4640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7064159dd0 con 0x7f70640b1de0 2023-10-26T01:44:32.142 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:32.137+0000 7f7068cb4640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70640c6700 con 0x7f70640c8cd0 2023-10-26T01:44:32.142 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:32.137+0000 7f706259c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f70500047c0 0x7f706414ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:32.142 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:32.137+0000 7f706259c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f70500047c0 0x7f706414ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:48768/0 (socket says 172.21.15.169:48768) 2023-10-26T01:44:32.142 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:32.137+0000 7f706259c640 1 -- 172.21.15.169:0/3636564921 learned_addr learned my addr 172.21.15.169:0/3636564921 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:44:32.143 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:32.137+0000 7f706259c640 1 -- 172.21.15.169:0/3636564921 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f70640c8cd0 msgr2=0x7f70640aae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:32.143 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:32.137+0000 7f706259c640 1 --2- 172.21.15.169:0/3636564921 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f70640c8cd0 0x7f70640aae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:32.143 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:32.137+0000 7f706259c640 1 -- 172.21.15.169:0/3636564921 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f70640b1de0 msgr2=0x7f70640aa930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:32.143 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:32.137+0000 7f706259c640 1 --2- 172.21.15.169:0/3636564921 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f70640b1de0 0x7f70640aa930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:32.143 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:32.137+0000 7f706259c640 1 -- 172.21.15.169:0/3636564921 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70640c69e0 con 0x7f70500047c0 2023-10-26T01:44:32.144 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:32.138+0000 7f706259c640 1 --2- 172.21.15.169:0/3636564921 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f70500047c0 0x7f706414ebc0 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f705400fa00 tx=0x7f705400fec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:32.144 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:32.138+0000 7f705affd640 1 -- 172.21.15.169:0/3636564921 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f705400bcb0 con 0x7f70500047c0 2023-10-26T01:44:32.144 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:32.138+0000 7f705affd640 1 -- 172.21.15.169:0/3636564921 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7054011440 con 0x7f70500047c0 2023-10-26T01:44:32.144 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:32.138+0000 7f7068cb4640 1 -- 172.21.15.169:0/3636564921 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70640b2350 con 0x7f70500047c0 2023-10-26T01:44:32.144 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:32.138+0000 7f705affd640 1 -- 172.21.15.169:0/3636564921 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7054011830 con 0x7f70500047c0 2023-10-26T01:44:32.145 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:32.138+0000 7f7068cb4640 1 -- 172.21.15.169:0/3636564921 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70641e2ff0 con 0x7f70500047c0 2023-10-26T01:44:32.145 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:32.139+0000 7f7068cb4640 1 -- 172.21.15.169:0/3636564921 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7064066af0 con 0x7f70500047c0 2023-10-26T01:44:32.146 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:32.140+0000 7f705affd640 1 -- 172.21.15.169:0/3636564921 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f7054011990 con 0x7f70500047c0 2023-10-26T01:44:32.146 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:32.141+0000 7f705affd640 1 --2- 172.21.15.169:0/3636564921 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f704406ae70 0x7f704406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:32.146 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:32.141+0000 7f705affd640 1 -- 172.21.15.169:0/3636564921 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f705408a350 con 0x7f70500047c0 2023-10-26T01:44:32.146 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:32.142+0000 7f705bfff640 1 --2- 172.21.15.169:0/3636564921 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f704406ae70 0x7f704406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:32.147 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:32.143+0000 7f705bfff640 1 --2- 172.21.15.169:0/3636564921 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f704406ae70 0x7f704406d330 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f704c034000 tx=0x7f704c039040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:32.148 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:32.145+0000 7f705affd640 1 -- 172.21.15.169:0/3636564921 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7054055490 con 0x7f70500047c0 2023-10-26T01:44:32.340 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:32.338+0000 7f7068cb4640 1 -- 172.21.15.169:0/3636564921 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f706404da70 con 0x7f70500047c0 2023-10-26T01:44:32.341 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:32.340+0000 7f705affd640 1 -- 172.21.15.169:0/3636564921 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v185) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7054058ab0 con 0x7f70500047c0 2023-10-26T01:44:32.342 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 64 2023-10-26T01:44:32.344 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:32.341+0000 7f7068cb4640 1 -- 172.21.15.169:0/3636564921 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f704406ae70 msgr2=0x7f704406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:32.345 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:32.341+0000 7f7068cb4640 1 --2- 172.21.15.169:0/3636564921 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f704406ae70 0x7f704406d330 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f704c034000 tx=0x7f704c039040 comp rx=0 tx=0).stop 2023-10-26T01:44:32.345 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:32.341+0000 7f7068cb4640 1 -- 172.21.15.169:0/3636564921 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f70500047c0 msgr2=0x7f706414ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:32.345 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:32.341+0000 7f7068cb4640 1 --2- 172.21.15.169:0/3636564921 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f70500047c0 0x7f706414ebc0 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f705400fa00 tx=0x7f705400fec0 comp rx=0 tx=0).stop 2023-10-26T01:44:32.345 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:32.342+0000 7f7068cb4640 1 -- 172.21.15.169:0/3636564921 shutdown_connections 2023-10-26T01:44:32.346 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:32.342+0000 7f7068cb4640 1 --2- 172.21.15.169:0/3636564921 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f704406ae70 0x7f704406d330 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:32.346 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:32.342+0000 7f7068cb4640 1 --2- 172.21.15.169:0/3636564921 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f70640c8cd0 0x7f70640aae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:32.346 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:32.342+0000 7f7068cb4640 1 --2- 172.21.15.169:0/3636564921 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f70640b1de0 0x7f70640aa930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:32.346 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:32.342+0000 7f7068cb4640 1 --2- 172.21.15.169:0/3636564921 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f70500047c0 0x7f706414ebc0 unknown :-1 s=CLOSED pgs=1912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:32.346 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:32.342+0000 7f7068cb4640 1 -- 172.21.15.169:0/3636564921 >> 172.21.15.169:0/3636564921 conn(0x7f70640aefe0 msgr2=0x7f7064151830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:32.347 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:32.342+0000 7f7068cb4640 1 -- 172.21.15.169:0/3636564921 shutdown_connections 2023-10-26T01:44:32.347 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:32.342+0000 7f7068cb4640 1 -- 172.21.15.169:0/3636564921 wait complete. 2023-10-26T01:44:32.364 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-26T01:44:32.590 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:32.579+0000 7fbb800bb640 1 -- 172.21.15.169:0/3823608735 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb7815b9e0 msgr2=0x7fbb7806ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:32.591 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:32.579+0000 7fbb800bb640 1 --2- 172.21.15.169:0/3823608735 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb7815b9e0 0x7fbb7806ad90 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7fbb780b3500 tx=0x7fbb7002bf20 comp rx=0 tx=0).stop 2023-10-26T01:44:32.592 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:32.579+0000 7fbb800bb640 1 -- 172.21.15.169:0/3823608735 shutdown_connections 2023-10-26T01:44:32.592 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:32.579+0000 7fbb800bb640 1 --2- 172.21.15.169:0/3823608735 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb7815b9e0 0x7fbb7806ad90 unknown :-1 s=CLOSED pgs=1913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:32.592 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:32.579+0000 7fbb800bb640 1 --2- 172.21.15.169:0/3823608735 >> v2:172.21.15.170:3300/0 conn(0x7fbb780cdd20 0x7fbb7815b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:32.592 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:32.579+0000 7fbb800bb640 1 --2- 172.21.15.169:0/3823608735 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbb78155360 0x7fbb780cd7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:32.592 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:32.579+0000 7fbb800bb640 1 -- 172.21.15.169:0/3823608735 >> 172.21.15.169:0/3823608735 conn(0x7fbb78069930 msgr2=0x7fbb78069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:32.593 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:32.579+0000 7fbb800bb640 1 -- 172.21.15.169:0/3823608735 shutdown_connections 2023-10-26T01:44:32.593 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:32.579+0000 7fbb800bb640 1 -- 172.21.15.169:0/3823608735 wait complete. 2023-10-26T01:44:32.593 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:32.581+0000 7fbb800bb640 1 Processor -- start 2023-10-26T01:44:32.594 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:32.581+0000 7fbb800bb640 1 -- start start 2023-10-26T01:44:32.594 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:32.582+0000 7fbb800bb640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbb780cdd20 0x7fbb7809ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:32.594 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:32.582+0000 7fbb800bb640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb78155360 0x7fbb780973d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:32.595 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:32.582+0000 7fbb800bb640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb7815b9e0 0x7fbb78150f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:32.595 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:32.582+0000 7fbb800bb640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb78097910 con 0x7fbb7815b9e0 2023-10-26T01:44:32.595 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:32.582+0000 7fbb800bb640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb78151bc0 con 0x7fbb78155360 2023-10-26T01:44:32.595 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:32.582+0000 7fbb800bb640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb78099de0 con 0x7fbb780cdd20 2023-10-26T01:44:32.596 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:32.582+0000 7fbb7e658640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb7815b9e0 0x7fbb78150f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:32.596 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:32.582+0000 7fbb7e658640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb7815b9e0 0x7fbb78150f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:48778/0 (socket says 172.21.15.169:48778) 2023-10-26T01:44:32.596 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:32.582+0000 7fbb7e658640 1 -- 172.21.15.169:0/145456083 learned_addr learned my addr 172.21.15.169:0/145456083 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:44:32.596 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:32.582+0000 7fbb7d656640 1 --2- 172.21.15.169:0/145456083 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb78155360 0x7fbb780973d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:32.596 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:32.582+0000 7fbb7e658640 1 -- 172.21.15.169:0/145456083 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbb780cdd20 msgr2=0x7fbb7809ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:32.597 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:32.582+0000 7fbb7e658640 1 --2- 172.21.15.169:0/145456083 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbb780cdd20 0x7fbb7809ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:32.597 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:32.582+0000 7fbb7e658640 1 -- 172.21.15.169:0/145456083 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb78155360 msgr2=0x7fbb780973d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:32.597 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:32.582+0000 7fbb7e658640 1 --2- 172.21.15.169:0/145456083 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb78155360 0x7fbb780973d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:32.597 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:32.582+0000 7fbb7e658640 1 -- 172.21.15.169:0/145456083 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb70009460 con 0x7fbb7815b9e0 2023-10-26T01:44:32.597 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:32.583+0000 7fbb7e658640 1 --2- 172.21.15.169:0/145456083 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb7815b9e0 0x7fbb78150f60 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7fbb70037040 tx=0x7fbb70004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:32.598 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:32.583+0000 7fbb6a7fc640 1 -- 172.21.15.169:0/145456083 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb700047f0 con 0x7fbb7815b9e0 2023-10-26T01:44:32.598 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:32.583+0000 7fbb6a7fc640 1 -- 172.21.15.169:0/145456083 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fbb700345b0 con 0x7fbb7815b9e0 2023-10-26T01:44:32.598 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:32.583+0000 7fbb800bb640 1 -- 172.21.15.169:0/145456083 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb7809a0c0 con 0x7fbb7815b9e0 2023-10-26T01:44:32.598 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:32.583+0000 7fbb6a7fc640 1 -- 172.21.15.169:0/145456083 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb700349a0 con 0x7fbb7815b9e0 2023-10-26T01:44:32.599 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:32.583+0000 7fbb800bb640 1 -- 172.21.15.169:0/145456083 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb7809a680 con 0x7fbb7815b9e0 2023-10-26T01:44:32.599 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:32.584+0000 7fbb800bb640 1 -- 172.21.15.169:0/145456083 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb7804db10 con 0x7fbb7815b9e0 2023-10-26T01:44:32.599 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:32.585+0000 7fbb6a7fc640 1 -- 172.21.15.169:0/145456083 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fbb7003b040 con 0x7fbb7815b9e0 2023-10-26T01:44:32.599 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:32.587+0000 7fbb6a7fc640 1 --2- 172.21.15.169:0/145456083 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbb5c06ae40 0x7fbb5c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:32.599 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:32.587+0000 7fbb6a7fc640 1 -- 172.21.15.169:0/145456083 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbb700abcc0 con 0x7fbb7815b9e0 2023-10-26T01:44:32.600 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:32.587+0000 7fbb7de57640 1 --2- 172.21.15.169:0/145456083 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbb5c06ae40 0x7fbb5c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:32.600 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:32.589+0000 7fbb7de57640 1 --2- 172.21.15.169:0/145456083 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbb5c06ae40 0x7fbb5c06d300 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fbb74005fd0 tx=0x7fbb74005850 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:32.600 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:32.591+0000 7fbb6a7fc640 1 -- 172.21.15.169:0/145456083 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbb70076d70 con 0x7fbb7815b9e0 2023-10-26T01:44:32.799 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:32.798+0000 7fbb800bb640 1 -- 172.21.15.169:0/145456083 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fbb78071f70 con 0x7fbb7815b9e0 2023-10-26T01:44:32.800 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:32.799+0000 7fbb6a7fc640 1 -- 172.21.15.169:0/145456083 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v185) v1 ==== 80+0+0 (secure 0 0 0) 0x7fbb70041080 con 0x7fbb7815b9e0 2023-10-26T01:44:32.801 INFO:teuthology.orchestra.run.smithi169.stderr:listed 0 entries 2023-10-26T01:44:32.803 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:32.801+0000 7fbb800bb640 1 -- 172.21.15.169:0/145456083 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbb5c06ae40 msgr2=0x7fbb5c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:32.803 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:32.801+0000 7fbb800bb640 1 --2- 172.21.15.169:0/145456083 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbb5c06ae40 0x7fbb5c06d300 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fbb74005fd0 tx=0x7fbb74005850 comp rx=0 tx=0).stop 2023-10-26T01:44:32.804 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:32.801+0000 7fbb800bb640 1 -- 172.21.15.169:0/145456083 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb7815b9e0 msgr2=0x7fbb78150f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:32.804 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:32.801+0000 7fbb800bb640 1 --2- 172.21.15.169:0/145456083 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb7815b9e0 0x7fbb78150f60 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7fbb70037040 tx=0x7fbb70004270 comp rx=0 tx=0).stop 2023-10-26T01:44:32.804 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:32.801+0000 7fbb800bb640 1 -- 172.21.15.169:0/145456083 shutdown_connections 2023-10-26T01:44:32.804 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:32.801+0000 7fbb800bb640 1 --2- 172.21.15.169:0/145456083 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbb5c06ae40 0x7fbb5c06d300 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:32.805 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:32.801+0000 7fbb800bb640 1 --2- 172.21.15.169:0/145456083 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb7815b9e0 0x7fbb78150f60 unknown :-1 s=CLOSED pgs=1914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:32.805 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:32.801+0000 7fbb800bb640 1 --2- 172.21.15.169:0/145456083 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb78155360 0x7fbb780973d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:32.805 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:32.801+0000 7fbb800bb640 1 --2- 172.21.15.169:0/145456083 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbb780cdd20 0x7fbb7809ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:32.805 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:32.801+0000 7fbb800bb640 1 -- 172.21.15.169:0/145456083 >> 172.21.15.169:0/145456083 conn(0x7fbb78069930 msgr2=0x7fbb780715d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:32.806 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:32.802+0000 7fbb800bb640 1 -- 172.21.15.169:0/145456083 shutdown_connections 2023-10-26T01:44:32.806 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:32.802+0000 7fbb800bb640 1 -- 172.21.15.169:0/145456083 wait complete. 2023-10-26T01:44:32.821 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-26T01:44:32.821 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:44:32.822 DEBUG:teuthology.orchestra.run.smithi169:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-26T01:44:32.990 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi169.front.sepia.ceph.com 2023-10-26T01:44:32.990 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:44:32.991 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-26T01:44:33.031 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-26T01:44:33.032 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:44:33.075 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-26T01:44:33.076 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:44:33.171 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:44:33.393 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:33.391+0000 7fefe256c640 1 -- 172.21.15.169:0/3378251997 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fefdc09a5c0 msgr2=0x7fefdc09aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:33.393 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:33.391+0000 7fefe256c640 1 --2- 172.21.15.169:0/3378251997 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fefdc09a5c0 0x7fefdc09aa70 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7fefd4005fd0 tx=0x7fefd402c0f0 comp rx=0 tx=0).stop 2023-10-26T01:44:33.394 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:33.391+0000 7fefe256c640 1 -- 172.21.15.169:0/3378251997 shutdown_connections 2023-10-26T01:44:33.394 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:33.391+0000 7fefe256c640 1 --2- 172.21.15.169:0/3378251997 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fefdc09a5c0 0x7fefdc09aa70 unknown :-1 s=CLOSED pgs=1916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:33.394 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:33.391+0000 7fefe256c640 1 -- 172.21.15.169:0/3378251997 >> 172.21.15.169:0/3378251997 conn(0x7fefdc0aedd0 msgr2=0x7fefdc0af1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:33.394 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:33.391+0000 7fefe256c640 1 -- 172.21.15.169:0/3378251997 shutdown_connections 2023-10-26T01:44:33.394 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:33.391+0000 7fefe256c640 1 -- 172.21.15.169:0/3378251997 wait complete. 2023-10-26T01:44:33.396 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:33.392+0000 7fefe256c640 1 Processor -- start 2023-10-26T01:44:33.397 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:33.393+0000 7fefe256c640 1 -- start start 2023-10-26T01:44:33.397 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:33.393+0000 7fefe256c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fefdc09a5c0 0x7fefdc06e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:33.397 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:33.394+0000 7fefe256c640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fefdc0ac6d0 0x7fefdc070f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:33.397 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:33.394+0000 7fefe256c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fefdc0c9970 0x7fefdc158990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:33.397 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:33.394+0000 7fefe256c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefdc06f770 con 0x7fefdc09a5c0 2023-10-26T01:44:33.398 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:33.394+0000 7fefe256c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefdc15a5b0 con 0x7fefdc0c9970 2023-10-26T01:44:33.399 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:33.394+0000 7fefe256c640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefdc158ed0 con 0x7fefdc0ac6d0 2023-10-26T01:44:33.399 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:33.394+0000 7fefe156a640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fefdc09a5c0 0x7fefdc06e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:33.399 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:33.394+0000 7fefe156a640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fefdc09a5c0 0x7fefdc06e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:48792/0 (socket says 172.21.15.169:48792) 2023-10-26T01:44:33.399 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:33.394+0000 7fefe156a640 1 -- 172.21.15.169:0/3493479453 learned_addr learned my addr 172.21.15.169:0/3493479453 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:44:33.400 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:33.394+0000 7fefe156a640 1 -- 172.21.15.169:0/3493479453 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fefdc0ac6d0 msgr2=0x7fefdc070f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:33.400 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:33.394+0000 7fefe156a640 1 --2- 172.21.15.169:0/3493479453 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fefdc0ac6d0 0x7fefdc070f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:33.400 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:33.394+0000 7fefe156a640 1 -- 172.21.15.169:0/3493479453 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fefdc0c9970 msgr2=0x7fefdc158990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:33.400 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:33.394+0000 7fefe156a640 1 --2- 172.21.15.169:0/3493479453 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fefdc0c9970 0x7fefdc158990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:33.400 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:33.394+0000 7fefe156a640 1 -- 172.21.15.169:0/3493479453 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefd4009840 con 0x7fefdc09a5c0 2023-10-26T01:44:33.401 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:33.394+0000 7fefe156a640 1 --2- 172.21.15.169:0/3493479453 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fefdc09a5c0 0x7fefdc06e8f0 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7fefd800c460 tx=0x7fefd800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:33.401 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:33.395+0000 7fefd1ffb640 1 -- 172.21.15.169:0/3493479453 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd8012830 con 0x7fefdc09a5c0 2023-10-26T01:44:33.401 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:33.395+0000 7fefd1ffb640 1 -- 172.21.15.169:0/3493479453 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fefd8012ac0 con 0x7fefdc09a5c0 2023-10-26T01:44:33.401 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:33.395+0000 7fefe256c640 1 -- 172.21.15.169:0/3493479453 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefdc070ab0 con 0x7fefdc09a5c0 2023-10-26T01:44:33.401 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:33.395+0000 7fefd1ffb640 1 -- 172.21.15.169:0/3493479453 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd8011380 con 0x7fefdc09a5c0 2023-10-26T01:44:33.402 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:33.395+0000 7fefe256c640 1 -- 172.21.15.169:0/3493479453 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefdc06f1b0 con 0x7fefdc09a5c0 2023-10-26T01:44:33.403 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:33.396+0000 7fefe256c640 1 -- 172.21.15.169:0/3493479453 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefdc15a360 con 0x7fefdc09a5c0 2023-10-26T01:44:33.403 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:33.397+0000 7fefd1ffb640 1 -- 172.21.15.169:0/3493479453 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fefd80114e0 con 0x7fefdc09a5c0 2023-10-26T01:44:33.403 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:33.398+0000 7fefd1ffb640 1 --2- 172.21.15.169:0/3493479453 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fefc006ae70 0x7fefc006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:33.403 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:33.398+0000 7fefd1ffb640 1 -- 172.21.15.169:0/3493479453 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fefd8089a00 con 0x7fefdc09a5c0 2023-10-26T01:44:33.404 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:33.398+0000 7fefd1ffb640 1 -- 172.21.15.169:0/3493479453 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=187}) v3 -- 0x7fefc00729a0 con 0x7fefdc09a5c0 2023-10-26T01:44:33.404 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:33.399+0000 7fefe0d69640 1 --2- 172.21.15.169:0/3493479453 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fefc006ae70 0x7fefc006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:33.404 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:33.400+0000 7fefe0d69640 1 --2- 172.21.15.169:0/3493479453 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fefc006ae70 0x7fefc006d330 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fefdc0cc050 tx=0x7fefcc007930 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:33.404 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:33.403+0000 7fefd1ffb640 1 -- 172.21.15.169:0/3493479453 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fefd8054b40 con 0x7fefdc09a5c0 2023-10-26T01:44:33.596 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:33.594+0000 7fefe256c640 1 -- 172.21.15.169:0/3493479453 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fefdc073c10 con 0x7fefdc09a5c0 2023-10-26T01:44:33.599 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:33.597+0000 7fefd1ffb640 1 -- 172.21.15.169:0/3493479453 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v186) v1 ==== 74+0+16646 (secure 0 0 0) 0x7fefd8058160 con 0x7fefdc09a5c0 2023-10-26T01:44:33.599 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:44:33.600 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":186,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:44:33.204279+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2023-10-26T01:43:47.501296+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"181","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2023-10-26T01:43:48.737507+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"186","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:44:33.602 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:33.600+0000 7fefe256c640 1 -- 172.21.15.169:0/3493479453 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fefc006ae70 msgr2=0x7fefc006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:33.603 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:33.600+0000 7fefe256c640 1 --2- 172.21.15.169:0/3493479453 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fefc006ae70 0x7fefc006d330 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fefdc0cc050 tx=0x7fefcc007930 comp rx=0 tx=0).stop 2023-10-26T01:44:33.603 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:33.600+0000 7fefe256c640 1 -- 172.21.15.169:0/3493479453 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fefdc09a5c0 msgr2=0x7fefdc06e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:33.603 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:33.600+0000 7fefe256c640 1 --2- 172.21.15.169:0/3493479453 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fefdc09a5c0 0x7fefdc06e8f0 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7fefd800c460 tx=0x7fefd800c920 comp rx=0 tx=0).stop 2023-10-26T01:44:33.603 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:33.600+0000 7fefe256c640 1 -- 172.21.15.169:0/3493479453 shutdown_connections 2023-10-26T01:44:33.603 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:33.600+0000 7fefe256c640 1 --2- 172.21.15.169:0/3493479453 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fefc006ae70 0x7fefc006d330 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:33.604 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:33.600+0000 7fefe256c640 1 --2- 172.21.15.169:0/3493479453 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fefdc0c9970 0x7fefdc158990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:33.604 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:33.600+0000 7fefe256c640 1 --2- 172.21.15.169:0/3493479453 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fefdc0ac6d0 0x7fefdc070f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:33.604 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:33.600+0000 7fefe256c640 1 --2- 172.21.15.169:0/3493479453 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fefdc09a5c0 0x7fefdc06e8f0 unknown :-1 s=CLOSED pgs=1917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:33.604 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:33.600+0000 7fefe256c640 1 -- 172.21.15.169:0/3493479453 >> 172.21.15.169:0/3493479453 conn(0x7fefdc0aedd0 msgr2=0x7fefdc141720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:33.604 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:33.601+0000 7fefe256c640 1 -- 172.21.15.169:0/3493479453 shutdown_connections 2023-10-26T01:44:33.605 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:33.601+0000 7fefe256c640 1 -- 172.21.15.169:0/3493479453 wait complete. 2023-10-26T01:44:33.622 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T01:30:32.200808+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '109', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 22, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-26T01:43:47.501296+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '181', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 23, 'pool_name': 'cephfs_data', 'create_time': '2023-10-26T01:43:48.737507+0000', 'flags': 1027, 'flags_names': 'hashpspool,full,full_quota', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '186', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T01:44:33.623 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T01:44:33.845 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:33.843+0000 7fdb0ffff640 1 -- 172.21.15.169:0/106078313 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdaf80039a0 msgr2=0x7fdaf8003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:33.845 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:33.843+0000 7fdb0ffff640 1 --2- 172.21.15.169:0/106078313 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdaf80039a0 0x7fdaf8003dc0 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7fdb0000c610 tx=0x7fdaf80057f0 comp rx=0 tx=0).stop 2023-10-26T01:44:33.846 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:33.843+0000 7fdb0f7fe640 1 -- 172.21.15.169:0/106078313 reap_dead start 2023-10-26T01:44:33.846 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:33.843+0000 7fdb0ffff640 1 -- 172.21.15.169:0/106078313 shutdown_connections 2023-10-26T01:44:33.846 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:33.843+0000 7fdb0ffff640 1 -- 172.21.15.169:0/106078313 >> 172.21.15.169:0/106078313 conn(0x7fdb100b10c0 msgr2=0x7fdb100b14a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:33.846 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:33.843+0000 7fdb0ffff640 1 -- 172.21.15.169:0/106078313 shutdown_connections 2023-10-26T01:44:33.846 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:33.843+0000 7fdb0ffff640 1 -- 172.21.15.169:0/106078313 wait complete. 2023-10-26T01:44:33.847 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:33.845+0000 7fdb0ffff640 1 Processor -- start 2023-10-26T01:44:33.847 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:33.845+0000 7fdb0ffff640 1 -- start start 2023-10-26T01:44:33.849 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:33.846+0000 7fdb0ffff640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdaf80039a0 0x7fdb1006a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:33.849 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:33.846+0000 7fdb0ffff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdb10154e80 0x7fdb10141ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:33.850 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:33.846+0000 7fdb0ffff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdb101f1970 0x7fdb101f1e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:33.850 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:33.846+0000 7fdb0ffff640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb1006ba10 con 0x7fdb10154e80 2023-10-26T01:44:33.850 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:33.846+0000 7fdb0ffff640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb1006ab60 con 0x7fdb101f1970 2023-10-26T01:44:33.850 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:33.846+0000 7fdb0ffff640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb1006ae00 con 0x7fdaf80039a0 2023-10-26T01:44:33.850 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:33.846+0000 7fdb0e7fc640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdb10154e80 0x7fdb10141ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:33.851 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:33.846+0000 7fdb0e7fc640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdb10154e80 0x7fdb10141ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:48806/0 (socket says 172.21.15.169:48806) 2023-10-26T01:44:33.851 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:33.846+0000 7fdb0e7fc640 1 -- 172.21.15.169:0/630922268 learned_addr learned my addr 172.21.15.169:0/630922268 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:44:33.851 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:33.847+0000 7fdb0effd640 1 --2- 172.21.15.169:0/630922268 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdaf80039a0 0x7fdb1006a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:33.851 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:33.847+0000 7fdb0f7fe640 1 --2- 172.21.15.169:0/630922268 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdb101f1970 0x7fdb101f1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:33.851 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:33.847+0000 7fdb0e7fc640 1 -- 172.21.15.169:0/630922268 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdaf80039a0 msgr2=0x7fdb1006a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:33.851 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:33.847+0000 7fdb0e7fc640 1 --2- 172.21.15.169:0/630922268 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdaf80039a0 0x7fdb1006a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:33.852 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:33.847+0000 7fdb0e7fc640 1 -- 172.21.15.169:0/630922268 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdb101f1970 msgr2=0x7fdb101f1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:33.852 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:33.847+0000 7fdb0e7fc640 1 --2- 172.21.15.169:0/630922268 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdb101f1970 0x7fdb101f1e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:33.852 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:33.847+0000 7fdb0e7fc640 1 -- 172.21.15.169:0/630922268 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb1006b0e0 con 0x7fdb10154e80 2023-10-26T01:44:33.852 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:33.847+0000 7fdb0f7fe640 1 --2- 172.21.15.169:0/630922268 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdb101f1970 0x7fdb101f1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:44:33.852 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:33.847+0000 7fdb0e7fc640 1 --2- 172.21.15.169:0/630922268 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdb10154e80 0x7fdb10141ec0 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7fdb04010a10 tx=0x7fdb04010ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:33.853 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:33.847+0000 7fdaf77fe640 1 -- 172.21.15.169:0/630922268 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb040095e0 con 0x7fdb10154e80 2023-10-26T01:44:33.853 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:33.847+0000 7fdaf77fe640 1 -- 172.21.15.169:0/630922268 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fdb0400ece0 con 0x7fdb10154e80 2023-10-26T01:44:33.853 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:33.847+0000 7fdaf77fe640 1 -- 172.21.15.169:0/630922268 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb04005280 con 0x7fdb10154e80 2023-10-26T01:44:33.853 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:33.847+0000 7fdb0ffff640 1 -- 172.21.15.169:0/630922268 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb1006b3c0 con 0x7fdb10154e80 2023-10-26T01:44:33.853 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:33.848+0000 7fdb0ffff640 1 -- 172.21.15.169:0/630922268 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb10142400 con 0x7fdb10154e80 2023-10-26T01:44:33.854 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:33.849+0000 7fdb0ffff640 1 -- 172.21.15.169:0/630922268 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb10066af0 con 0x7fdb10154e80 2023-10-26T01:44:33.855 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:33.850+0000 7fdaf77fe640 1 -- 172.21.15.169:0/630922268 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fdb040053e0 con 0x7fdb10154e80 2023-10-26T01:44:33.855 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:33.851+0000 7fdaf77fe640 1 --2- 172.21.15.169:0/630922268 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fdafc06ae70 0x7fdafc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:33.855 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:33.851+0000 7fdaf77fe640 1 -- 172.21.15.169:0/630922268 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdb04016070 con 0x7fdb10154e80 2023-10-26T01:44:33.855 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:33.851+0000 7fdaf77fe640 1 -- 172.21.15.169:0/630922268 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=187}) v3 -- 0x7fdafc0729a0 con 0x7fdb10154e80 2023-10-26T01:44:33.855 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:33.852+0000 7fdb0effd640 1 --2- 172.21.15.169:0/630922268 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fdafc06ae70 0x7fdafc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:33.856 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:33.853+0000 7fdb0effd640 1 --2- 172.21.15.169:0/630922268 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fdafc06ae70 0x7fdafc06d330 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fdb00034000 tx=0x7fdb0002e000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:33.857 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:33.855+0000 7fdaf77fe640 1 -- 172.21.15.169:0/630922268 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdb04055300 con 0x7fdb10154e80 2023-10-26T01:44:34.047 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.045+0000 7fdb0ffff640 1 -- 172.21.15.169:0/630922268 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdb10073c10 con 0x7fdb10154e80 2023-10-26T01:44:34.048 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.046+0000 7fdaf77fe640 1 -- 172.21.15.169:0/630922268 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v186) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdb04058920 con 0x7fdb10154e80 2023-10-26T01:44:34.049 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 1 2023-10-26T01:44:34.051 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.049+0000 7fdb0ffff640 1 -- 172.21.15.169:0/630922268 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fdafc06ae70 msgr2=0x7fdafc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:34.052 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.050+0000 7fdb0ffff640 1 --2- 172.21.15.169:0/630922268 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fdafc06ae70 0x7fdafc06d330 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fdb00034000 tx=0x7fdb0002e000 comp rx=0 tx=0).stop 2023-10-26T01:44:34.052 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.050+0000 7fdb0ffff640 1 -- 172.21.15.169:0/630922268 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdb10154e80 msgr2=0x7fdb10141ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:34.052 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.050+0000 7fdb0ffff640 1 --2- 172.21.15.169:0/630922268 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdb10154e80 0x7fdb10141ec0 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7fdb04010a10 tx=0x7fdb04010ed0 comp rx=0 tx=0).stop 2023-10-26T01:44:34.052 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.050+0000 7fdb0ffff640 1 -- 172.21.15.169:0/630922268 shutdown_connections 2023-10-26T01:44:34.053 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.050+0000 7fdb0ffff640 1 --2- 172.21.15.169:0/630922268 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fdafc06ae70 0x7fdafc06d330 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:34.053 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.050+0000 7fdb0ffff640 1 --2- 172.21.15.169:0/630922268 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdb10154e80 0x7fdb10141ec0 unknown :-1 s=CLOSED pgs=1920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:34.053 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.050+0000 7fdb0ffff640 1 --2- 172.21.15.169:0/630922268 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdb101f1970 0x7fdb101f1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:34.053 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.050+0000 7fdb0ffff640 1 --2- 172.21.15.169:0/630922268 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdaf80039a0 0x7fdb1006a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:34.053 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.050+0000 7fdb0ffff640 1 -- 172.21.15.169:0/630922268 >> 172.21.15.169:0/630922268 conn(0x7fdb100b10c0 msgr2=0x7fdb1015b170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:34.054 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.050+0000 7fdb0ffff640 1 -- 172.21.15.169:0/630922268 shutdown_connections 2023-10-26T01:44:34.054 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.050+0000 7fdb0ffff640 1 -- 172.21.15.169:0/630922268 wait complete. 2023-10-26T01:44:34.071 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-26T01:44:34.296 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.294+0000 7f98670e1640 1 -- 172.21.15.169:0/3777052120 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98600cd7e0 msgr2=0x7f98600cdc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:34.296 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.294+0000 7f98670e1640 1 --2- 172.21.15.169:0/3777052120 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98600cd7e0 0x7f98600cdc90 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7f98600b0570 tx=0x7f984c02bd10 comp rx=0 tx=0).stop 2023-10-26T01:44:34.296 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.294+0000 7f98670e1640 1 -- 172.21.15.169:0/3777052120 shutdown_connections 2023-10-26T01:44:34.297 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.294+0000 7f98670e1640 1 --2- 172.21.15.169:0/3777052120 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98600cd7e0 0x7f98600cdc90 unknown :-1 s=CLOSED pgs=1921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:34.297 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.294+0000 7f98670e1640 1 --2- 172.21.15.169:0/3777052120 >> v2:172.21.15.170:3300/0 conn(0x7f986006ce00 0x7f9860155360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:34.297 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.294+0000 7f98670e1640 1 -- 172.21.15.169:0/3777052120 >> 172.21.15.169:0/3777052120 conn(0x7f9860069930 msgr2=0x7f9860069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:34.297 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.294+0000 7f98670e1640 1 -- 172.21.15.169:0/3777052120 shutdown_connections 2023-10-26T01:44:34.297 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.294+0000 7f98670e1640 1 -- 172.21.15.169:0/3777052120 wait complete. 2023-10-26T01:44:34.298 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.296+0000 7f98670e1640 1 Processor -- start 2023-10-26T01:44:34.298 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.296+0000 7f98670e1640 1 -- start start 2023-10-26T01:44:34.300 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.297+0000 7f98670e1640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f986006ce00 0x7f986014d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:34.300 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.297+0000 7f98670e1640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f98600cd7e0 0x7f986014d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:34.301 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.297+0000 7f98670e1640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f986015b4a0 0x7f986014c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:34.301 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.297+0000 7f98670e1640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f986014ce10 con 0x7f986006ce00 2023-10-26T01:44:34.301 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.297+0000 7f98670e1640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98600cabc0 con 0x7f986015b4a0 2023-10-26T01:44:34.301 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.297+0000 7f98670e1640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f986014ebd0 con 0x7f98600cd7e0 2023-10-26T01:44:34.301 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.297+0000 7f9864e7d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f986006ce00 0x7f986014d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:34.301 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.297+0000 7f9864e7d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f986006ce00 0x7f986014d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:36138/0 (socket says 172.21.15.169:36138) 2023-10-26T01:44:34.302 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.297+0000 7f9864e7d640 1 -- 172.21.15.169:0/471013464 learned_addr learned my addr 172.21.15.169:0/471013464 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:44:34.302 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.298+0000 7f986567e640 1 --2- 172.21.15.169:0/471013464 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f986015b4a0 0x7f986014c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:34.302 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.298+0000 7f9864e7d640 1 -- 172.21.15.169:0/471013464 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f98600cd7e0 msgr2=0x7f986014d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:34.302 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.298+0000 7f985ffff640 1 --2- 172.21.15.169:0/471013464 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f98600cd7e0 0x7f986014d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:34.302 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.298+0000 7f9864e7d640 1 --2- 172.21.15.169:0/471013464 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f98600cd7e0 0x7f986014d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:34.303 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.298+0000 7f9864e7d640 1 -- 172.21.15.169:0/471013464 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f986015b4a0 msgr2=0x7f986014c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:34.303 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.298+0000 7f9864e7d640 1 --2- 172.21.15.169:0/471013464 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f986015b4a0 0x7f986014c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:34.303 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.298+0000 7f9864e7d640 1 -- 172.21.15.169:0/471013464 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f984c009560 con 0x7f986006ce00 2023-10-26T01:44:34.303 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.298+0000 7f9864e7d640 1 --2- 172.21.15.169:0/471013464 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f986006ce00 0x7f986014d400 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f985400d950 tx=0x7f985400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:34.303 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.298+0000 7f985d7fa640 1 -- 172.21.15.169:0/471013464 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9854009c30 con 0x7f986006ce00 2023-10-26T01:44:34.304 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.299+0000 7f985d7fa640 1 -- 172.21.15.169:0/471013464 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9854009e80 con 0x7f986006ce00 2023-10-26T01:44:34.304 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.299+0000 7f985d7fa640 1 -- 172.21.15.169:0/471013464 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98540106c0 con 0x7f986006ce00 2023-10-26T01:44:34.304 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.299+0000 7f98670e1640 1 -- 172.21.15.169:0/471013464 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9860151f90 con 0x7f986006ce00 2023-10-26T01:44:34.304 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.299+0000 7f98670e1640 1 -- 172.21.15.169:0/471013464 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9860156140 con 0x7f986006ce00 2023-10-26T01:44:34.305 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.300+0000 7f98670e1640 1 -- 172.21.15.169:0/471013464 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f986004db10 con 0x7f986006ce00 2023-10-26T01:44:34.305 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.301+0000 7f985d7fa640 1 -- 172.21.15.169:0/471013464 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f9854010820 con 0x7f986006ce00 2023-10-26T01:44:34.306 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.302+0000 7f985d7fa640 1 --2- 172.21.15.169:0/471013464 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f984406ae40 0x7f984406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:34.306 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.302+0000 7f985d7fa640 1 -- 172.21.15.169:0/471013464 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9854089b10 con 0x7f986006ce00 2023-10-26T01:44:34.306 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.302+0000 7f985d7fa640 1 -- 172.21.15.169:0/471013464 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=187}) v3 -- 0x7f9844072970 con 0x7f986006ce00 2023-10-26T01:44:34.306 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.303+0000 7f985ffff640 1 --2- 172.21.15.169:0/471013464 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f984406ae40 0x7f984406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:34.306 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.304+0000 7f985ffff640 1 --2- 172.21.15.169:0/471013464 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f984406ae40 0x7f984406d300 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f984c035000 tx=0x7f984c004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:34.308 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.306+0000 7f985d7fa640 1 -- 172.21.15.169:0/471013464 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9854017060 con 0x7f986006ce00 2023-10-26T01:44:34.496 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.495+0000 7f98670e1640 1 -- 172.21.15.169:0/471013464 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f98600662e0 con 0x7f986006ce00 2023-10-26T01:44:34.497 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.495+0000 7f985d7fa640 1 -- 172.21.15.169:0/471013464 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v186) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9854017020 con 0x7f986006ce00 2023-10-26T01:44:34.497 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 32 2023-10-26T01:44:34.499 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.498+0000 7f98670e1640 1 -- 172.21.15.169:0/471013464 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f984406ae40 msgr2=0x7f984406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:34.500 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.498+0000 7f98670e1640 1 --2- 172.21.15.169:0/471013464 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f984406ae40 0x7f984406d300 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f984c035000 tx=0x7f984c004730 comp rx=0 tx=0).stop 2023-10-26T01:44:34.500 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.498+0000 7f98670e1640 1 -- 172.21.15.169:0/471013464 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f986006ce00 msgr2=0x7f986014d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:34.500 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.498+0000 7f98670e1640 1 --2- 172.21.15.169:0/471013464 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f986006ce00 0x7f986014d400 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f985400d950 tx=0x7f985400de10 comp rx=0 tx=0).stop 2023-10-26T01:44:34.500 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.498+0000 7f98670e1640 1 -- 172.21.15.169:0/471013464 shutdown_connections 2023-10-26T01:44:34.501 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.498+0000 7f98670e1640 1 --2- 172.21.15.169:0/471013464 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f984406ae40 0x7f984406d300 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:34.501 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.498+0000 7f98670e1640 1 --2- 172.21.15.169:0/471013464 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f986015b4a0 0x7f986014c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:34.501 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.498+0000 7f98670e1640 1 --2- 172.21.15.169:0/471013464 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f98600cd7e0 0x7f986014d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:34.501 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.498+0000 7f98670e1640 1 --2- 172.21.15.169:0/471013464 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f986006ce00 0x7f986014d400 unknown :-1 s=CLOSED pgs=1922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:34.501 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.498+0000 7f98670e1640 1 -- 172.21.15.169:0/471013464 >> 172.21.15.169:0/471013464 conn(0x7f9860069930 msgr2=0x7f986014e410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:34.501 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.498+0000 7f98670e1640 1 -- 172.21.15.169:0/471013464 shutdown_connections 2023-10-26T01:44:34.502 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.498+0000 7f98670e1640 1 -- 172.21.15.169:0/471013464 wait complete. 2023-10-26T01:44:34.518 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-26T01:44:34.728 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.726+0000 7f190ef25640 1 -- 172.21.15.169:0/4085491259 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19080ac200 msgr2=0x7f19080cd710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:34.728 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.726+0000 7f190ef25640 1 --2- 172.21.15.169:0/4085491259 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19080ac200 0x7f19080cd710 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7f1900005fd0 tx=0x7f190002c0f0 comp rx=0 tx=0).stop 2023-10-26T01:44:34.729 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.726+0000 7f190ef25640 1 -- 172.21.15.169:0/4085491259 shutdown_connections 2023-10-26T01:44:34.729 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.726+0000 7f190ef25640 1 --2- 172.21.15.169:0/4085491259 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19080ac200 0x7f19080cd710 unknown :-1 s=CLOSED pgs=1924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:34.729 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.726+0000 7f190ef25640 1 -- 172.21.15.169:0/4085491259 >> 172.21.15.169:0/4085491259 conn(0x7f1908069930 msgr2=0x7f1908069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:34.729 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.726+0000 7f190ef25640 1 -- 172.21.15.169:0/4085491259 shutdown_connections 2023-10-26T01:44:34.729 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.726+0000 7f190ef25640 1 -- 172.21.15.169:0/4085491259 wait complete. 2023-10-26T01:44:34.730 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.728+0000 7f190ef25640 1 Processor -- start 2023-10-26T01:44:34.730 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.728+0000 7f190ef25640 1 -- start start 2023-10-26T01:44:34.732 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.729+0000 7f190ef25640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19080ac200 0x7f19081486d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:34.732 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.729+0000 7f190ef25640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f19081593e0 0x7f19081558a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:34.732 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.729+0000 7f190ef25640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f19081597c0 0x7f1908150ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:34.733 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.729+0000 7f190ef25640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19080acfa0 con 0x7f19080ac200 2023-10-26T01:44:34.733 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.729+0000 7f190ef25640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19080ad280 con 0x7f19081597c0 2023-10-26T01:44:34.733 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.729+0000 7f190ef25640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19080ad560 con 0x7f19081593e0 2023-10-26T01:44:34.733 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.729+0000 7f190ccc1640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19080ac200 0x7f19081486d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:34.733 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.730+0000 7f190ccc1640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19080ac200 0x7f19081486d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:36154/0 (socket says 172.21.15.169:36154) 2023-10-26T01:44:34.734 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.730+0000 7f190ccc1640 1 -- 172.21.15.169:0/2130307628 learned_addr learned my addr 172.21.15.169:0/2130307628 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:44:34.734 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.730+0000 7f190d4c2640 1 --2- 172.21.15.169:0/2130307628 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f19081597c0 0x7f1908150ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:34.734 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.730+0000 7f1907fff640 1 --2- 172.21.15.169:0/2130307628 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f19081593e0 0x7f19081558a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:34.734 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.730+0000 7f190ccc1640 1 -- 172.21.15.169:0/2130307628 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f19081593e0 msgr2=0x7f19081558a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:34.735 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.730+0000 7f190ccc1640 1 --2- 172.21.15.169:0/2130307628 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f19081593e0 0x7f19081558a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:34.735 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.730+0000 7f190ccc1640 1 -- 172.21.15.169:0/2130307628 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f19081597c0 msgr2=0x7f1908150ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:34.735 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.730+0000 7f190ccc1640 1 --2- 172.21.15.169:0/2130307628 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f19081597c0 0x7f1908150ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:34.735 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.730+0000 7f190ccc1640 1 -- 172.21.15.169:0/2130307628 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1900009840 con 0x7f19080ac200 2023-10-26T01:44:34.735 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.730+0000 7f190d4c2640 1 --2- 172.21.15.169:0/2130307628 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f19081597c0 0x7f1908150ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:44:34.736 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.730+0000 7f190ccc1640 1 --2- 172.21.15.169:0/2130307628 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19080ac200 0x7f19081486d0 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7f18fc00c430 tx=0x7f18fc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:34.736 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.730+0000 7f19057fa640 1 -- 172.21.15.169:0/2130307628 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18fc011790 con 0x7f19080ac200 2023-10-26T01:44:34.736 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.731+0000 7f19057fa640 1 -- 172.21.15.169:0/2130307628 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f18fc0119e0 con 0x7f19080ac200 2023-10-26T01:44:34.736 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.731+0000 7f19057fa640 1 -- 172.21.15.169:0/2130307628 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18fc011e30 con 0x7f19080ac200 2023-10-26T01:44:34.736 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.731+0000 7f190ef25640 1 -- 172.21.15.169:0/2130307628 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19080ad840 con 0x7f19080ac200 2023-10-26T01:44:34.736 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.731+0000 7f190ef25640 1 -- 172.21.15.169:0/2130307628 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1908158580 con 0x7f19080ac200 2023-10-26T01:44:34.738 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.732+0000 7f190ef25640 1 -- 172.21.15.169:0/2130307628 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19080add60 con 0x7f19080ac200 2023-10-26T01:44:34.738 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.733+0000 7f19057fa640 1 -- 172.21.15.169:0/2130307628 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f18fc01c460 con 0x7f19080ac200 2023-10-26T01:44:34.738 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.734+0000 7f19057fa640 1 --2- 172.21.15.169:0/2130307628 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f18ec06ae40 0x7f18ec06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:34.741 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.734+0000 7f19057fa640 1 -- 172.21.15.169:0/2130307628 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f18fc089860 con 0x7f19080ac200 2023-10-26T01:44:34.741 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.734+0000 7f19057fa640 1 -- 172.21.15.169:0/2130307628 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=187}) v3 -- 0x7f18ec072970 con 0x7f19080ac200 2023-10-26T01:44:34.741 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.735+0000 7f1907fff640 1 --2- 172.21.15.169:0/2130307628 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f18ec06ae40 0x7f18ec06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:34.741 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.736+0000 7f1907fff640 1 --2- 172.21.15.169:0/2130307628 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f18ec06ae40 0x7f18ec06d300 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f19080cc930 tx=0x7f18f4009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:34.742 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.739+0000 7f19057fa640 1 -- 172.21.15.169:0/2130307628 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f19080add60 con 0x7f19080ac200 2023-10-26T01:44:34.934 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.932+0000 7f190ef25640 1 -- 172.21.15.169:0/2130307628 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f19080add60 con 0x7f19080ac200 2023-10-26T01:44:34.935 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.933+0000 7f19057fa640 1 -- 172.21.15.169:0/2130307628 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v186) v1 ==== 100+0+11 (secure 0 0 0) 0x7f19080add60 con 0x7f19080ac200 2023-10-26T01:44:34.935 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 64 2023-10-26T01:44:34.939 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.937+0000 7f190ef25640 1 -- 172.21.15.169:0/2130307628 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f18ec06ae40 msgr2=0x7f18ec06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:34.939 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.937+0000 7f190ef25640 1 --2- 172.21.15.169:0/2130307628 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f18ec06ae40 0x7f18ec06d300 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f19080cc930 tx=0x7f18f4009040 comp rx=0 tx=0).stop 2023-10-26T01:44:34.939 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.937+0000 7f190ef25640 1 -- 172.21.15.169:0/2130307628 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19080ac200 msgr2=0x7f19081486d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:34.939 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.937+0000 7f190ef25640 1 --2- 172.21.15.169:0/2130307628 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19080ac200 0x7f19081486d0 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7f18fc00c430 tx=0x7f18fc00c8f0 comp rx=0 tx=0).stop 2023-10-26T01:44:34.940 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.937+0000 7f190ef25640 1 -- 172.21.15.169:0/2130307628 shutdown_connections 2023-10-26T01:44:34.940 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.937+0000 7f190ef25640 1 --2- 172.21.15.169:0/2130307628 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f18ec06ae40 0x7f18ec06d300 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:34.940 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.937+0000 7f190ef25640 1 --2- 172.21.15.169:0/2130307628 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f19081597c0 0x7f1908150ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:34.940 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.937+0000 7f190ef25640 1 --2- 172.21.15.169:0/2130307628 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f19081593e0 0x7f19081558a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:34.940 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.937+0000 7f190ef25640 1 --2- 172.21.15.169:0/2130307628 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19080ac200 0x7f19081486d0 unknown :-1 s=CLOSED pgs=1925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:34.940 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.937+0000 7f190ef25640 1 -- 172.21.15.169:0/2130307628 >> 172.21.15.169:0/2130307628 conn(0x7f1908069930 msgr2=0x7f1908154560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:34.941 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.937+0000 7f190ef25640 1 -- 172.21.15.169:0/2130307628 shutdown_connections 2023-10-26T01:44:34.941 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:34.937+0000 7f190ef25640 1 -- 172.21.15.169:0/2130307628 wait complete. 2023-10-26T01:44:34.958 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-26T01:44:35.178 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:35.176+0000 7f15ff14c640 1 -- 172.21.15.169:0/2720984863 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15f80ccda0 msgr2=0x7f15f815ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:35.179 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:35.176+0000 7f15ff14c640 1 --2- 172.21.15.169:0/2720984863 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15f80ccda0 0x7f15f815ca20 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7f15ec005fd0 tx=0x7f15ec02be20 comp rx=0 tx=0).stop 2023-10-26T01:44:35.179 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:35.176+0000 7f15ff14c640 1 -- 172.21.15.169:0/2720984863 shutdown_connections 2023-10-26T01:44:35.179 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:35.176+0000 7f15ff14c640 1 --2- 172.21.15.169:0/2720984863 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f15f8141cb0 0x7f15f80b14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:35.179 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:35.176+0000 7f15ff14c640 1 --2- 172.21.15.169:0/2720984863 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15f80ccda0 0x7f15f815ca20 unknown :-1 s=CLOSED pgs=1927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:35.179 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:35.176+0000 7f15ff14c640 1 -- 172.21.15.169:0/2720984863 >> 172.21.15.169:0/2720984863 conn(0x7f15f8069930 msgr2=0x7f15f8069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:35.180 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:35.176+0000 7f15ff14c640 1 -- 172.21.15.169:0/2720984863 shutdown_connections 2023-10-26T01:44:35.180 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:35.176+0000 7f15ff14c640 1 -- 172.21.15.169:0/2720984863 wait complete. 2023-10-26T01:44:35.181 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:35.178+0000 7f15ff14c640 1 Processor -- start 2023-10-26T01:44:35.181 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:35.178+0000 7f15ff14c640 1 -- start start 2023-10-26T01:44:35.182 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:35.179+0000 7f15ff14c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f15f80ccda0 0x7f15f806ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:35.182 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:35.179+0000 7f15ff14c640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f15f80cd7e0 0x7f15f8153050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:35.182 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:35.179+0000 7f15ff14c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15f8141cb0 0x7f15f8154fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:35.182 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:35.179+0000 7f15ff14c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15f806fb00 con 0x7f15f8141cb0 2023-10-26T01:44:35.182 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:35.179+0000 7f15ff14c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15f8150f00 con 0x7f15f80ccda0 2023-10-26T01:44:35.183 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:35.179+0000 7f15ff14c640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15f806f270 con 0x7f15f80cd7e0 2023-10-26T01:44:35.183 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:35.180+0000 7f15fd6e9640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15f8141cb0 0x7f15f8154fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:35.183 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:35.180+0000 7f15fd6e9640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15f8141cb0 0x7f15f8154fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:36180/0 (socket says 172.21.15.169:36180) 2023-10-26T01:44:35.183 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:35.180+0000 7f15fd6e9640 1 -- 172.21.15.169:0/1730197705 learned_addr learned my addr 172.21.15.169:0/1730197705 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:44:35.184 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:35.180+0000 7f15f7fff640 1 --2- 172.21.15.169:0/1730197705 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f15f80cd7e0 0x7f15f8153050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:35.185 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:35.180+0000 7f15fcee8640 1 --2- 172.21.15.169:0/1730197705 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f15f80ccda0 0x7f15f806ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:35.185 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:35.180+0000 7f15fd6e9640 1 -- 172.21.15.169:0/1730197705 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f15f80cd7e0 msgr2=0x7f15f8153050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:35.185 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:35.180+0000 7f15fd6e9640 1 --2- 172.21.15.169:0/1730197705 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f15f80cd7e0 0x7f15f8153050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:35.185 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:35.180+0000 7f15fd6e9640 1 -- 172.21.15.169:0/1730197705 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f15f80ccda0 msgr2=0x7f15f806ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:35.186 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:35.180+0000 7f15fd6e9640 1 --2- 172.21.15.169:0/1730197705 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f15f80ccda0 0x7f15f806ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:35.186 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:35.180+0000 7f15fd6e9640 1 -- 172.21.15.169:0/1730197705 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15ec009560 con 0x7f15f8141cb0 2023-10-26T01:44:35.186 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:35.180+0000 7f15fd6e9640 1 --2- 172.21.15.169:0/1730197705 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15f8141cb0 0x7f15f8154fc0 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7f15f0001560 tx=0x7f15f000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:35.186 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:35.181+0000 7f15f57fa640 1 -- 172.21.15.169:0/1730197705 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f0003da0 con 0x7f15f8141cb0 2023-10-26T01:44:35.186 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:35.181+0000 7f15f57fa640 1 -- 172.21.15.169:0/1730197705 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f15f0005070 con 0x7f15f8141cb0 2023-10-26T01:44:35.187 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:35.181+0000 7f15f57fa640 1 -- 172.21.15.169:0/1730197705 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f0005460 con 0x7f15f8141cb0 2023-10-26T01:44:35.187 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:35.181+0000 7f15ff14c640 1 -- 172.21.15.169:0/1730197705 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15f806f540 con 0x7f15f8141cb0 2023-10-26T01:44:35.187 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:35.181+0000 7f15ff14c640 1 -- 172.21.15.169:0/1730197705 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15f8150b90 con 0x7f15f8141cb0 2023-10-26T01:44:35.187 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:35.182+0000 7f15ff14c640 1 -- 172.21.15.169:0/1730197705 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15f81509f0 con 0x7f15f8141cb0 2023-10-26T01:44:35.187 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:35.183+0000 7f15f57fa640 1 -- 172.21.15.169:0/1730197705 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f15f00055c0 con 0x7f15f8141cb0 2023-10-26T01:44:35.188 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:35.184+0000 7f15f57fa640 1 --2- 172.21.15.169:0/1730197705 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f15dc06ae40 0x7f15dc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:35.189 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:35.184+0000 7f15f57fa640 1 -- 172.21.15.169:0/1730197705 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f15f00891d0 con 0x7f15f8141cb0 2023-10-26T01:44:35.189 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:35.184+0000 7f15f57fa640 1 -- 172.21.15.169:0/1730197705 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=187}) v3 -- 0x7f15dc072970 con 0x7f15f8141cb0 2023-10-26T01:44:35.189 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:35.185+0000 7f15fcee8640 1 --2- 172.21.15.169:0/1730197705 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f15dc06ae40 0x7f15dc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:35.189 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:35.186+0000 7f15fcee8640 1 --2- 172.21.15.169:0/1730197705 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f15dc06ae40 0x7f15dc06d300 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f15ec035000 tx=0x7f15ec004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:35.190 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:35.188+0000 7f15f57fa640 1 -- 172.21.15.169:0/1730197705 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f15f0017070 con 0x7f15f8141cb0 2023-10-26T01:44:35.385 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:35.383+0000 7f15ff14c640 1 -- 172.21.15.169:0/1730197705 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f15f80ac180 con 0x7f15f8141cb0 2023-10-26T01:44:35.386 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:35.384+0000 7f15f57fa640 1 -- 172.21.15.169:0/1730197705 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v186) v1 ==== 80+0+0 (secure 0 0 0) 0x7f15f81509f0 con 0x7f15f8141cb0 2023-10-26T01:44:35.386 INFO:teuthology.orchestra.run.smithi169.stderr:listed 0 entries 2023-10-26T01:44:35.388 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:35.386+0000 7f15ff14c640 1 -- 172.21.15.169:0/1730197705 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f15dc06ae40 msgr2=0x7f15dc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:35.389 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:35.386+0000 7f15ff14c640 1 --2- 172.21.15.169:0/1730197705 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f15dc06ae40 0x7f15dc06d300 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f15ec035000 tx=0x7f15ec004730 comp rx=0 tx=0).stop 2023-10-26T01:44:35.389 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:35.386+0000 7f15ff14c640 1 -- 172.21.15.169:0/1730197705 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15f8141cb0 msgr2=0x7f15f8154fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:35.389 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:35.386+0000 7f15ff14c640 1 --2- 172.21.15.169:0/1730197705 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15f8141cb0 0x7f15f8154fc0 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7f15f0001560 tx=0x7f15f000cc70 comp rx=0 tx=0).stop 2023-10-26T01:44:35.389 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:35.387+0000 7f15ff14c640 1 -- 172.21.15.169:0/1730197705 shutdown_connections 2023-10-26T01:44:35.389 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:35.387+0000 7f15ff14c640 1 --2- 172.21.15.169:0/1730197705 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f15dc06ae40 0x7f15dc06d300 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:35.390 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:35.387+0000 7f15ff14c640 1 --2- 172.21.15.169:0/1730197705 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15f8141cb0 0x7f15f8154fc0 unknown :-1 s=CLOSED pgs=1928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:35.390 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:35.387+0000 7f15ff14c640 1 --2- 172.21.15.169:0/1730197705 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f15f80cd7e0 0x7f15f8153050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:35.390 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:35.387+0000 7f15ff14c640 1 --2- 172.21.15.169:0/1730197705 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f15f80ccda0 0x7f15f806ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:35.390 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:35.387+0000 7f15ff14c640 1 -- 172.21.15.169:0/1730197705 >> 172.21.15.169:0/1730197705 conn(0x7f15f8069930 msgr2=0x7f15f80acf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:35.390 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:35.387+0000 7f15ff14c640 1 -- 172.21.15.169:0/1730197705 shutdown_connections 2023-10-26T01:44:35.391 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:35.387+0000 7f15ff14c640 1 -- 172.21.15.169:0/1730197705 wait complete. 2023-10-26T01:44:35.406 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-26T01:44:35.407 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:44:35.407 DEBUG:teuthology.orchestra.run.smithi170:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-26T01:44:35.524 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi170.front.sepia.ceph.com 2023-10-26T01:44:35.525 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:44:35.525 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-26T01:44:35.579 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:44:35.579 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:44:35.638 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054301,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-26T01:44:35.639 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-26T01:44:35.639 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:44:35.694 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054303,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-26T01:44:35.695 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T01:44:35.922 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:35.920+0000 7f9472887640 1 -- 172.21.15.169:0/1275060994 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f946c158850 msgr2=0x7f946c14e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:35.923 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:35.920+0000 7f9472887640 1 --2- 172.21.15.169:0/1275060994 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f946c158850 0x7f946c14e790 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7f946c06eaf0 tx=0x7f945402bd30 comp rx=0 tx=0).stop 2023-10-26T01:44:35.923 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:35.920+0000 7f9472887640 1 -- 172.21.15.169:0/1275060994 shutdown_connections 2023-10-26T01:44:35.923 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:35.920+0000 7f9472887640 1 --2- 172.21.15.169:0/1275060994 >> v2:172.21.15.170:3300/0 conn(0x7f946c0cd720 0x7f946c0cdb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:35.923 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:35.920+0000 7f9472887640 1 --2- 172.21.15.169:0/1275060994 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f946c158850 0x7f946c14e790 unknown :-1 s=CLOSED pgs=1929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:35.924 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:35.920+0000 7f9472887640 1 -- 172.21.15.169:0/1275060994 >> 172.21.15.169:0/1275060994 conn(0x7f946c069930 msgr2=0x7f946c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:35.924 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:35.921+0000 7f9472887640 1 -- 172.21.15.169:0/1275060994 shutdown_connections 2023-10-26T01:44:35.924 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:35.921+0000 7f9472887640 1 -- 172.21.15.169:0/1275060994 wait complete. 2023-10-26T01:44:35.925 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:35.923+0000 7f9472887640 1 Processor -- start 2023-10-26T01:44:35.925 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:35.923+0000 7f9472887640 1 -- start start 2023-10-26T01:44:35.926 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:35.924+0000 7f9472887640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f946c06ce20 0x7f946c156930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:35.926 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:35.924+0000 7f9472887640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f946c0cd720 0x7f946c0adfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:35.926 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:35.924+0000 7f9472887640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f946c158850 0x7f946c149bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:35.927 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:35.924+0000 7f9472887640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f946c1564e0 con 0x7f946c06ce20 2023-10-26T01:44:35.927 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:35.924+0000 7f9472887640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f946c1570c0 con 0x7f946c0cd720 2023-10-26T01:44:35.927 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:35.924+0000 7f9472887640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f946c157680 con 0x7f946c158850 2023-10-26T01:44:35.927 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:35.924+0000 7f946bfff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f946c06ce20 0x7f946c156930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:35.927 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:35.924+0000 7f946bfff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f946c06ce20 0x7f946c156930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:36192/0 (socket says 172.21.15.169:36192) 2023-10-26T01:44:35.928 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:35.924+0000 7f946bfff640 1 -- 172.21.15.169:0/4081189756 learned_addr learned my addr 172.21.15.169:0/4081189756 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:44:35.929 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:35.925+0000 7f946bfff640 1 -- 172.21.15.169:0/4081189756 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f946c158850 msgr2=0x7f946c149bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:35.929 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:35.925+0000 7f946bfff640 1 --2- 172.21.15.169:0/4081189756 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f946c158850 0x7f946c149bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:35.929 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:35.925+0000 7f946bfff640 1 -- 172.21.15.169:0/4081189756 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f946c0cd720 msgr2=0x7f946c0adfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:35.930 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:35.925+0000 7f946bfff640 1 --2- 172.21.15.169:0/4081189756 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f946c0cd720 0x7f946c0adfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:35.930 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:35.925+0000 7f946bfff640 1 -- 172.21.15.169:0/4081189756 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9454009560 con 0x7f946c06ce20 2023-10-26T01:44:35.930 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:35.925+0000 7f946bfff640 1 --2- 172.21.15.169:0/4081189756 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f946c06ce20 0x7f946c156930 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7f945c00c410 tx=0x7f945c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:35.930 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:35.925+0000 7f94697fa640 1 -- 172.21.15.169:0/4081189756 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f945c0035f0 con 0x7f946c06ce20 2023-10-26T01:44:35.930 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:35.925+0000 7f94697fa640 1 -- 172.21.15.169:0/4081189756 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f945c003750 con 0x7f946c06ce20 2023-10-26T01:44:35.931 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:35.925+0000 7f9472887640 1 -- 172.21.15.169:0/4081189756 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f946c157960 con 0x7f946c06ce20 2023-10-26T01:44:35.931 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:35.925+0000 7f94697fa640 1 -- 172.21.15.169:0/4081189756 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f945c003ba0 con 0x7f946c06ce20 2023-10-26T01:44:35.931 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:35.925+0000 7f9472887640 1 -- 172.21.15.169:0/4081189756 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f946c0ae520 con 0x7f946c06ce20 2023-10-26T01:44:35.931 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:35.927+0000 7f9472887640 1 -- 172.21.15.169:0/4081189756 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f946c1574e0 con 0x7f946c06ce20 2023-10-26T01:44:35.931 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:35.927+0000 7f94697fa640 1 -- 172.21.15.169:0/4081189756 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f945c003d00 con 0x7f946c06ce20 2023-10-26T01:44:35.932 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:35.928+0000 7f94697fa640 1 --2- 172.21.15.169:0/4081189756 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f944c06ae40 0x7f944c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:35.932 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:35.928+0000 7f94697fa640 1 -- 172.21.15.169:0/4081189756 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f945c088e60 con 0x7f946c06ce20 2023-10-26T01:44:35.933 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:35.929+0000 7f94697fa640 1 -- 172.21.15.169:0/4081189756 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=187}) v3 -- 0x7f944c072970 con 0x7f946c06ce20 2023-10-26T01:44:35.933 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:35.929+0000 7f9463fff640 1 --2- 172.21.15.169:0/4081189756 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f944c06ae40 0x7f944c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:35.933 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:35.930+0000 7f9463fff640 1 --2- 172.21.15.169:0/4081189756 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f944c06ae40 0x7f944c06d300 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f9454035000 tx=0x7f94540046c0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:35.934 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:35.933+0000 7f94697fa640 1 -- 172.21.15.169:0/4081189756 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f945c017070 con 0x7f946c06ce20 2023-10-26T01:44:36.163 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.161+0000 7f9472887640 1 -- 172.21.15.169:0/4081189756 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f946c04db10 con 0x7f946c06ce20 2023-10-26T01:44:36.164 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.162+0000 7f94697fa640 1 -- 172.21.15.169:0/4081189756 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) v1 ==== 94+0+4920 (secure 0 0 0) 0x7f946c1574e0 con 0x7f946c06ce20 2023-10-26T01:44:36.164 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:44:36.165 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":94,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6484,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/195700049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":195700049},{"type":"v1","addr":"172.21.15.169:6835","nonce":195700049}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":89},{"gid":6493,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/1167696300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1167696300},{"type":"v1","addr":"172.21.15.170:6836","nonce":1167696300}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":89},{"gid":9111,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/22729060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":22729060},{"type":"v1","addr":"172.21.15.169:6834","nonce":22729060}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":89}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T01:43:49.711084+0000","modified":"2023-10-26T01:43:52.816785+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9123},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9123":{"gid":9123,"name":"d","rank":0,"incarnation":90,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6837/2205518515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":2205518515},{"type":"v1","addr":"172.21.15.170:6837","nonce":2205518515}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2023-10-26T01:44:36.165 INFO:teuthology.orchestra.run.smithi169.stderr:dumped fsmap epoch 94 2023-10-26T01:44:36.167 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.165+0000 7f9472887640 1 -- 172.21.15.169:0/4081189756 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f944c06ae40 msgr2=0x7f944c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:36.167 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.165+0000 7f9472887640 1 --2- 172.21.15.169:0/4081189756 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f944c06ae40 0x7f944c06d300 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f9454035000 tx=0x7f94540046c0 comp rx=0 tx=0).stop 2023-10-26T01:44:36.168 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.165+0000 7f9472887640 1 -- 172.21.15.169:0/4081189756 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f946c06ce20 msgr2=0x7f946c156930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:36.168 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.165+0000 7f9472887640 1 --2- 172.21.15.169:0/4081189756 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f946c06ce20 0x7f946c156930 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7f945c00c410 tx=0x7f945c00c8d0 comp rx=0 tx=0).stop 2023-10-26T01:44:36.168 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.165+0000 7f9472887640 1 -- 172.21.15.169:0/4081189756 shutdown_connections 2023-10-26T01:44:36.168 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.165+0000 7f9472887640 1 --2- 172.21.15.169:0/4081189756 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f944c06ae40 0x7f944c06d300 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:36.168 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.165+0000 7f9472887640 1 --2- 172.21.15.169:0/4081189756 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f946c158850 0x7f946c149bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:36.169 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.165+0000 7f9472887640 1 --2- 172.21.15.169:0/4081189756 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f946c0cd720 0x7f946c0adfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:36.169 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.165+0000 7f9472887640 1 --2- 172.21.15.169:0/4081189756 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f946c06ce20 0x7f946c156930 unknown :-1 s=CLOSED pgs=1930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:36.169 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.165+0000 7f9472887640 1 -- 172.21.15.169:0/4081189756 >> 172.21.15.169:0/4081189756 conn(0x7f946c069930 msgr2=0x7f946c142450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:36.169 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.165+0000 7f9472887640 1 -- 172.21.15.169:0/4081189756 shutdown_connections 2023-10-26T01:44:36.169 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.166+0000 7f9472887640 1 -- 172.21.15.169:0/4081189756 wait complete. 2023-10-26T01:44:36.186 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:44:36.411 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.409+0000 7fc482bc0640 1 -- 172.21.15.169:0/829266561 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc468003970 msgr2=0x7fc468003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:36.411 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.409+0000 7fc482bc0640 1 --2- 172.21.15.169:0/829266561 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc468003970 0x7fc468003d90 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7fc464006fd0 tx=0x7fc4680057f0 comp rx=0 tx=0).stop 2023-10-26T01:44:36.412 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.409+0000 7fc482bc0640 1 -- 172.21.15.169:0/829266561 shutdown_connections 2023-10-26T01:44:36.412 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.409+0000 7fc482bc0640 1 --2- 172.21.15.169:0/829266561 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc468003970 0x7fc468003d90 unknown :-1 s=CLOSED pgs=1932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:36.412 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.409+0000 7fc482bc0640 1 -- 172.21.15.169:0/829266561 >> 172.21.15.169:0/829266561 conn(0x7fc47c069e00 msgr2=0x7fc47c06a210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:36.412 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.409+0000 7fc482bc0640 1 -- 172.21.15.169:0/829266561 shutdown_connections 2023-10-26T01:44:36.412 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.409+0000 7fc482bc0640 1 -- 172.21.15.169:0/829266561 wait complete. 2023-10-26T01:44:36.413 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.411+0000 7fc482bc0640 1 Processor -- start 2023-10-26T01:44:36.413 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.411+0000 7fc482bc0640 1 -- start start 2023-10-26T01:44:36.415 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.412+0000 7fc482bc0640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc468003970 0x7fc47c1542a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:36.415 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.412+0000 7fc482bc0640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc47c1547e0 0x7fc47c0ca250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:36.416 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.412+0000 7fc482bc0640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc47c0ca790 0x7fc47c15d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:36.416 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.412+0000 7fc482bc0640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc47c15cb40 con 0x7fc47c0ca790 2023-10-26T01:44:36.416 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.412+0000 7fc482bc0640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc47c0cabf0 con 0x7fc47c1547e0 2023-10-26T01:44:36.416 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.412+0000 7fc482bc0640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc47c15bec0 con 0x7fc468003970 2023-10-26T01:44:36.416 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.412+0000 7fc48115d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc47c0ca790 0x7fc47c15d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:36.417 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.412+0000 7fc48115d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc47c0ca790 0x7fc47c15d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:36208/0 (socket says 172.21.15.169:36208) 2023-10-26T01:44:36.417 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.412+0000 7fc48115d640 1 -- 172.21.15.169:0/3122267181 learned_addr learned my addr 172.21.15.169:0/3122267181 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:44:36.417 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.413+0000 7fc48095c640 1 --2- 172.21.15.169:0/3122267181 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc468003970 0x7fc47c1542a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:36.417 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.413+0000 7fc47bfff640 1 --2- 172.21.15.169:0/3122267181 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc47c1547e0 0x7fc47c0ca250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:36.417 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.413+0000 7fc48115d640 1 -- 172.21.15.169:0/3122267181 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc468003970 msgr2=0x7fc47c1542a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:36.418 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.413+0000 7fc48115d640 1 --2- 172.21.15.169:0/3122267181 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc468003970 0x7fc47c1542a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:36.418 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.413+0000 7fc48115d640 1 -- 172.21.15.169:0/3122267181 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc47c1547e0 msgr2=0x7fc47c0ca250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:36.418 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.413+0000 7fc48115d640 1 --2- 172.21.15.169:0/3122267181 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc47c1547e0 0x7fc47c0ca250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:36.418 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.413+0000 7fc48115d640 1 -- 172.21.15.169:0/3122267181 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc47c15c1a0 con 0x7fc47c0ca790 2023-10-26T01:44:36.418 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.413+0000 7fc48095c640 1 --2- 172.21.15.169:0/3122267181 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc468003970 0x7fc47c1542a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:44:36.419 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.413+0000 7fc47bfff640 1 --2- 172.21.15.169:0/3122267181 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc47c1547e0 0x7fc47c0ca250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:44:36.419 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.413+0000 7fc48115d640 1 --2- 172.21.15.169:0/3122267181 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc47c0ca790 0x7fc47c15d2a0 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7fc47c0b10f0 tx=0x7fc47400c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:36.419 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.413+0000 7fc4797fa640 1 -- 172.21.15.169:0/3122267181 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4740052d0 con 0x7fc47c0ca790 2023-10-26T01:44:36.419 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.414+0000 7fc4797fa640 1 -- 172.21.15.169:0/3122267181 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc4740035f0 con 0x7fc47c0ca790 2023-10-26T01:44:36.419 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.414+0000 7fc482bc0640 1 -- 172.21.15.169:0/3122267181 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc47c15c480 con 0x7fc47c0ca790 2023-10-26T01:44:36.420 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.414+0000 7fc4797fa640 1 -- 172.21.15.169:0/3122267181 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc474003a00 con 0x7fc47c0ca790 2023-10-26T01:44:36.420 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.414+0000 7fc482bc0640 1 -- 172.21.15.169:0/3122267181 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc47c15cf40 con 0x7fc47c0ca790 2023-10-26T01:44:36.421 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.415+0000 7fc482bc0640 1 -- 172.21.15.169:0/3122267181 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc47c04db10 con 0x7fc47c0ca790 2023-10-26T01:44:36.421 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.416+0000 7fc4797fa640 1 -- 172.21.15.169:0/3122267181 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fc474003b60 con 0x7fc47c0ca790 2023-10-26T01:44:36.421 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.417+0000 7fc4797fa640 1 --2- 172.21.15.169:0/3122267181 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc45c06ae40 0x7fc45c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:36.421 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.417+0000 7fc4797fa640 1 -- 172.21.15.169:0/3122267181 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc474088ac0 con 0x7fc47c0ca790 2023-10-26T01:44:36.422 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.418+0000 7fc4797fa640 1 -- 172.21.15.169:0/3122267181 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=187}) v3 -- 0x7fc45c072970 con 0x7fc47c0ca790 2023-10-26T01:44:36.422 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.418+0000 7fc48095c640 1 --2- 172.21.15.169:0/3122267181 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc45c06ae40 0x7fc45c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:36.422 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.419+0000 7fc48095c640 1 --2- 172.21.15.169:0/3122267181 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc45c06ae40 0x7fc45c06d300 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fc464034000 tx=0x7fc46402e000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:36.423 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.422+0000 7fc4797fa640 1 -- 172.21.15.169:0/3122267181 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc474017060 con 0x7fc47c0ca790 2023-10-26T01:44:36.616 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.614+0000 7fc482bc0640 1 -- 172.21.15.169:0/3122267181 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc47c0cf4b0 con 0x7fc47c0ca790 2023-10-26T01:44:36.617 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.616+0000 7fc4797fa640 1 -- 172.21.15.169:0/3122267181 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v186) v1 ==== 74+0+16646 (secure 0 0 0) 0x7fc474053ad0 con 0x7fc47c0ca790 2023-10-26T01:44:36.618 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:44:36.618 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":186,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:44:33.204279+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2023-10-26T01:43:47.501296+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"181","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2023-10-26T01:43:48.737507+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"186","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:44:36.621 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.619+0000 7fc482bc0640 1 -- 172.21.15.169:0/3122267181 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc45c06ae40 msgr2=0x7fc45c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:36.621 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.619+0000 7fc482bc0640 1 --2- 172.21.15.169:0/3122267181 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc45c06ae40 0x7fc45c06d300 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fc464034000 tx=0x7fc46402e000 comp rx=0 tx=0).stop 2023-10-26T01:44:36.621 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.619+0000 7fc482bc0640 1 -- 172.21.15.169:0/3122267181 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc47c0ca790 msgr2=0x7fc47c15d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:36.621 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.619+0000 7fc482bc0640 1 --2- 172.21.15.169:0/3122267181 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc47c0ca790 0x7fc47c15d2a0 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7fc47c0b10f0 tx=0x7fc47400c820 comp rx=0 tx=0).stop 2023-10-26T01:44:36.622 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.619+0000 7fc482bc0640 1 -- 172.21.15.169:0/3122267181 shutdown_connections 2023-10-26T01:44:36.622 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.619+0000 7fc482bc0640 1 --2- 172.21.15.169:0/3122267181 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fc45c06ae40 0x7fc45c06d300 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:36.622 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.619+0000 7fc482bc0640 1 --2- 172.21.15.169:0/3122267181 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc47c0ca790 0x7fc47c15d2a0 unknown :-1 s=CLOSED pgs=1933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:36.622 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.619+0000 7fc482bc0640 1 --2- 172.21.15.169:0/3122267181 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc47c1547e0 0x7fc47c0ca250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:36.622 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.619+0000 7fc482bc0640 1 --2- 172.21.15.169:0/3122267181 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc468003970 0x7fc47c1542a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:36.623 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.619+0000 7fc482bc0640 1 -- 172.21.15.169:0/3122267181 >> 172.21.15.169:0/3122267181 conn(0x7fc47c069e00 msgr2=0x7fc47c15aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:36.623 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.619+0000 7fc482bc0640 1 -- 172.21.15.169:0/3122267181 shutdown_connections 2023-10-26T01:44:36.623 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.619+0000 7fc482bc0640 1 -- 172.21.15.169:0/3122267181 wait complete. 2023-10-26T01:44:36.638 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T01:30:32.200808+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '109', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 22, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-26T01:43:47.501296+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '181', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 23, 'pool_name': 'cephfs_data', 'create_time': '2023-10-26T01:43:48.737507+0000', 'flags': 1027, 'flags_names': 'hashpspool,full,full_quota', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '186', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T01:44:36.639 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T01:44:36.865 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.863+0000 7f2cd4dd7640 1 -- 172.21.15.169:0/2858110591 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2cbc0058d0 msgr2=0x7f2cbc005d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:36.866 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.863+0000 7f2cd4dd7640 1 --2- 172.21.15.169:0/2858110591 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2cbc0058d0 0x7f2cbc005d50 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7f2cb8001650 tx=0x7f2cbc006900 comp rx=0 tx=0).stop 2023-10-26T01:44:36.866 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.863+0000 7f2cd4dd7640 1 -- 172.21.15.169:0/2858110591 shutdown_connections 2023-10-26T01:44:36.866 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.863+0000 7f2cd4dd7640 1 --2- 172.21.15.169:0/2858110591 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2cbc0058d0 0x7f2cbc005d50 unknown :-1 s=CLOSED pgs=1935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:36.866 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.863+0000 7f2cd4dd7640 1 -- 172.21.15.169:0/2858110591 >> 172.21.15.169:0/2858110591 conn(0x7f2cd0069510 msgr2=0x7f2cd0069920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:36.867 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.863+0000 7f2cd4dd7640 1 -- 172.21.15.169:0/2858110591 shutdown_connections 2023-10-26T01:44:36.867 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.863+0000 7f2cd4dd7640 1 -- 172.21.15.169:0/2858110591 wait complete. 2023-10-26T01:44:36.869 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.865+0000 7f2cd4dd7640 1 Processor -- start 2023-10-26T01:44:36.869 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.865+0000 7f2cd4dd7640 1 -- start start 2023-10-26T01:44:36.869 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.866+0000 7f2cd4dd7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2cbc0058d0 0x7f2cd0150690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:36.869 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.866+0000 7f2cd4dd7640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2cd00af580 0x7f2cd0149f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:36.870 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.866+0000 7f2cd4dd7640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2cd01e5910 0x7f2cd01e5d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:36.870 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.866+0000 7f2cd4dd7640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd0151910 con 0x7f2cd00af580 2023-10-26T01:44:36.870 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.866+0000 7f2cd4dd7640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd01e62d0 con 0x7f2cbc0058d0 2023-10-26T01:44:36.870 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.866+0000 7f2cd4dd7640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd0150c90 con 0x7f2cd01e5910 2023-10-26T01:44:36.870 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.866+0000 7f2cc7fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2cd00af580 0x7f2cd0149f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:36.871 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.866+0000 7f2cc7fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2cd00af580 0x7f2cd0149f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:36228/0 (socket says 172.21.15.169:36228) 2023-10-26T01:44:36.871 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.866+0000 7f2cc7fff640 1 -- 172.21.15.169:0/3538005087 learned_addr learned my addr 172.21.15.169:0/3538005087 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:44:36.871 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.867+0000 7f2ccf7fe640 1 --2- 172.21.15.169:0/3538005087 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2cbc0058d0 0x7f2cd0150690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:36.871 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.867+0000 7f2cc7fff640 1 -- 172.21.15.169:0/3538005087 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2cd01e5910 msgr2=0x7f2cd01e5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:36.871 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.867+0000 7f2ccffff640 1 --2- 172.21.15.169:0/3538005087 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2cd01e5910 0x7f2cd01e5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:36.872 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.867+0000 7f2cc7fff640 1 --2- 172.21.15.169:0/3538005087 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2cd01e5910 0x7f2cd01e5d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:36.872 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.867+0000 7f2cc7fff640 1 -- 172.21.15.169:0/3538005087 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2cbc0058d0 msgr2=0x7f2cd0150690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:36.872 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.867+0000 7f2cc7fff640 1 --2- 172.21.15.169:0/3538005087 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2cbc0058d0 0x7f2cd0150690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:36.872 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.867+0000 7f2cc7fff640 1 -- 172.21.15.169:0/3538005087 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cd0150f70 con 0x7f2cd00af580 2023-10-26T01:44:36.872 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.867+0000 7f2ccf7fe640 1 --2- 172.21.15.169:0/3538005087 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2cbc0058d0 0x7f2cd0150690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:44:36.873 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.867+0000 7f2cc7fff640 1 --2- 172.21.15.169:0/3538005087 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2cd00af580 0x7f2cd0149f60 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7f2cb802c090 tx=0x7f2cb8003de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:36.874 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.868+0000 7f2cccff9640 1 -- 172.21.15.169:0/3538005087 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cb803b070 con 0x7f2cd00af580 2023-10-26T01:44:36.874 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.868+0000 7f2cccff9640 1 -- 172.21.15.169:0/3538005087 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2cb8006240 con 0x7f2cd00af580 2023-10-26T01:44:36.874 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.868+0000 7f2cccff9640 1 -- 172.21.15.169:0/3538005087 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cb802d700 con 0x7f2cd00af580 2023-10-26T01:44:36.875 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.868+0000 7f2cd4dd7640 1 -- 172.21.15.169:0/3538005087 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cd0151250 con 0x7f2cd00af580 2023-10-26T01:44:36.875 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.868+0000 7f2cd4dd7640 1 -- 172.21.15.169:0/3538005087 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cd014a4a0 con 0x7f2cd00af580 2023-10-26T01:44:36.875 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.869+0000 7f2cd4dd7640 1 -- 172.21.15.169:0/3538005087 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cd014a660 con 0x7f2cd00af580 2023-10-26T01:44:36.875 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.870+0000 7f2cccff9640 1 -- 172.21.15.169:0/3538005087 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f2cb802d860 con 0x7f2cd00af580 2023-10-26T01:44:36.876 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.871+0000 7f2cccff9640 1 --2- 172.21.15.169:0/3538005087 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2cb006ae70 0x7f2cb006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:36.876 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.871+0000 7f2cccff9640 1 -- 172.21.15.169:0/3538005087 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2cb80aad10 con 0x7f2cd00af580 2023-10-26T01:44:36.876 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.871+0000 7f2cccff9640 1 -- 172.21.15.169:0/3538005087 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=187}) v3 -- 0x7f2cb00729a0 con 0x7f2cd00af580 2023-10-26T01:44:36.876 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.871+0000 7f2ccf7fe640 1 --2- 172.21.15.169:0/3538005087 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2cb006ae70 0x7f2cb006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:36.876 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.873+0000 7f2ccf7fe640 1 --2- 172.21.15.169:0/3538005087 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2cb006ae70 0x7f2cb006d330 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f2cc0009b10 tx=0x7f2cc0005f90 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:36.877 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:36.875+0000 7f2cccff9640 1 -- 172.21.15.169:0/3538005087 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2cb8045070 con 0x7f2cd00af580 2023-10-26T01:44:37.066 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.064+0000 7f2cd4dd7640 1 -- 172.21.15.169:0/3538005087 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2cd0073c10 con 0x7f2cd00af580 2023-10-26T01:44:37.066 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.065+0000 7f2cccff9640 1 -- 172.21.15.169:0/3538005087 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v186) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2cd014a660 con 0x7f2cd00af580 2023-10-26T01:44:37.067 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 1 2023-10-26T01:44:37.069 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.067+0000 7f2cd4dd7640 1 -- 172.21.15.169:0/3538005087 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2cb006ae70 msgr2=0x7f2cb006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:37.069 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.067+0000 7f2cd4dd7640 1 --2- 172.21.15.169:0/3538005087 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2cb006ae70 0x7f2cb006d330 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f2cc0009b10 tx=0x7f2cc0005f90 comp rx=0 tx=0).stop 2023-10-26T01:44:37.070 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.067+0000 7f2cd4dd7640 1 -- 172.21.15.169:0/3538005087 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2cd00af580 msgr2=0x7f2cd0149f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:37.070 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.067+0000 7f2cd4dd7640 1 --2- 172.21.15.169:0/3538005087 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2cd00af580 0x7f2cd0149f60 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7f2cb802c090 tx=0x7f2cb8003de0 comp rx=0 tx=0).stop 2023-10-26T01:44:37.070 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.067+0000 7f2cd4dd7640 1 -- 172.21.15.169:0/3538005087 shutdown_connections 2023-10-26T01:44:37.070 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.067+0000 7f2cd4dd7640 1 --2- 172.21.15.169:0/3538005087 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2cb006ae70 0x7f2cb006d330 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:37.070 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.067+0000 7f2cd4dd7640 1 --2- 172.21.15.169:0/3538005087 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2cd00af580 0x7f2cd0149f60 unknown :-1 s=CLOSED pgs=1936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:37.071 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.067+0000 7f2cd4dd7640 1 --2- 172.21.15.169:0/3538005087 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2cd01e5910 0x7f2cd01e5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:37.071 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.067+0000 7f2cd4dd7640 1 --2- 172.21.15.169:0/3538005087 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2cbc0058d0 0x7f2cd0150690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:37.071 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.067+0000 7f2cd4dd7640 1 -- 172.21.15.169:0/3538005087 >> 172.21.15.169:0/3538005087 conn(0x7f2cd0069510 msgr2=0x7f2cd0152120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:37.071 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.068+0000 7f2cd4dd7640 1 -- 172.21.15.169:0/3538005087 shutdown_connections 2023-10-26T01:44:37.071 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.068+0000 7f2cd4dd7640 1 -- 172.21.15.169:0/3538005087 wait complete. 2023-10-26T01:44:37.089 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-26T01:44:37.317 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.315+0000 7fcc189d0640 1 -- 172.21.15.169:0/2582997096 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc00003970 msgr2=0x7fcc00003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:37.318 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.315+0000 7fcc189d0640 1 --2- 172.21.15.169:0/2582997096 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc00003970 0x7fcc00003d90 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7fcc04009ed0 tx=0x7fcc000057f0 comp rx=0 tx=0).stop 2023-10-26T01:44:37.318 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.315+0000 7fcc12d9d640 1 -- 172.21.15.169:0/2582997096 reap_dead start 2023-10-26T01:44:37.318 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.315+0000 7fcc189d0640 1 -- 172.21.15.169:0/2582997096 shutdown_connections 2023-10-26T01:44:37.318 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.315+0000 7fcc189d0640 1 -- 172.21.15.169:0/2582997096 >> 172.21.15.169:0/2582997096 conn(0x7fcc14069930 msgr2=0x7fcc14069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:37.318 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.315+0000 7fcc189d0640 1 -- 172.21.15.169:0/2582997096 shutdown_connections 2023-10-26T01:44:37.319 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.315+0000 7fcc189d0640 1 -- 172.21.15.169:0/2582997096 wait complete. 2023-10-26T01:44:37.320 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.317+0000 7fcc189d0640 1 Processor -- start 2023-10-26T01:44:37.320 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.317+0000 7fcc189d0640 1 -- start start 2023-10-26T01:44:37.321 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.318+0000 7fcc189d0640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcc00003970 0x7fcc140af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:37.321 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.318+0000 7fcc189d0640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc1414fa10 0x7fcc141596d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:37.321 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.318+0000 7fcc189d0640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc14159c10 0x7fcc140aded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:37.321 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.318+0000 7fcc189d0640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc1415a090 con 0x7fcc1414fa10 2023-10-26T01:44:37.321 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.318+0000 7fcc189d0640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc1406fc20 con 0x7fcc14159c10 2023-10-26T01:44:37.322 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.318+0000 7fcc189d0640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc140afe10 con 0x7fcc00003970 2023-10-26T01:44:37.322 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.318+0000 7fcc11d9b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc1414fa10 0x7fcc141596d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:37.322 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.318+0000 7fcc11d9b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc1414fa10 0x7fcc141596d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:36238/0 (socket says 172.21.15.169:36238) 2023-10-26T01:44:37.322 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.318+0000 7fcc11d9b640 1 -- 172.21.15.169:0/2997590713 learned_addr learned my addr 172.21.15.169:0/2997590713 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:44:37.324 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.319+0000 7fcc1259c640 1 --2- 172.21.15.169:0/2997590713 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcc00003970 0x7fcc140af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:37.324 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.319+0000 7fcc11d9b640 1 -- 172.21.15.169:0/2997590713 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcc00003970 msgr2=0x7fcc140af8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:37.324 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.319+0000 7fcc12d9d640 1 --2- 172.21.15.169:0/2997590713 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc14159c10 0x7fcc140aded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:37.325 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.319+0000 7fcc11d9b640 1 --2- 172.21.15.169:0/2997590713 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcc00003970 0x7fcc140af8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:37.325 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.319+0000 7fcc11d9b640 1 -- 172.21.15.169:0/2997590713 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc14159c10 msgr2=0x7fcc140aded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:37.325 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.319+0000 7fcc11d9b640 1 --2- 172.21.15.169:0/2997590713 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc14159c10 0x7fcc140aded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:37.325 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.319+0000 7fcc11d9b640 1 -- 172.21.15.169:0/2997590713 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc1406ed30 con 0x7fcc1414fa10 2023-10-26T01:44:37.325 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.319+0000 7fcc1259c640 1 --2- 172.21.15.169:0/2997590713 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcc00003970 0x7fcc140af8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:44:37.326 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.319+0000 7fcc11d9b640 1 --2- 172.21.15.169:0/2997590713 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc1414fa10 0x7fcc141596d0 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc00e5e0 tx=0x7fcbfc00eaa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:37.326 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.319+0000 7fcc0affd640 1 -- 172.21.15.169:0/2997590713 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbfc00c970 con 0x7fcc1414fa10 2023-10-26T01:44:37.326 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.319+0000 7fcc0affd640 1 -- 172.21.15.169:0/2997590713 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fcbfc00cbc0 con 0x7fcc1414fa10 2023-10-26T01:44:37.326 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.319+0000 7fcc189d0640 1 -- 172.21.15.169:0/2997590713 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc1406f010 con 0x7fcc1414fa10 2023-10-26T01:44:37.326 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.320+0000 7fcc0affd640 1 -- 172.21.15.169:0/2997590713 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbfc011460 con 0x7fcc1414fa10 2023-10-26T01:44:37.327 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.320+0000 7fcc189d0640 1 -- 172.21.15.169:0/2997590713 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc1406f5d0 con 0x7fcc1414fa10 2023-10-26T01:44:37.327 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.321+0000 7fcc189d0640 1 -- 172.21.15.169:0/2997590713 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc1404db10 con 0x7fcc1414fa10 2023-10-26T01:44:37.327 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.322+0000 7fcc0affd640 1 -- 172.21.15.169:0/2997590713 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fcbfc0115c0 con 0x7fcc1414fa10 2023-10-26T01:44:37.327 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.322+0000 7fcc0affd640 1 --2- 172.21.15.169:0/2997590713 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcbf406ae40 0x7fcbf406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:37.327 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.323+0000 7fcc0affd640 1 -- 172.21.15.169:0/2997590713 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcbfc089e70 con 0x7fcc1414fa10 2023-10-26T01:44:37.328 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.323+0000 7fcc0affd640 1 -- 172.21.15.169:0/2997590713 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=187}) v3 -- 0x7fcbf4072970 con 0x7fcc1414fa10 2023-10-26T01:44:37.328 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.323+0000 7fcc1259c640 1 --2- 172.21.15.169:0/2997590713 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcbf406ae40 0x7fcbf406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:37.329 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.324+0000 7fcc1259c640 1 --2- 172.21.15.169:0/2997590713 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcbf406ae40 0x7fcbf406d300 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fcc04037000 tx=0x7fcc04033040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:37.329 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.327+0000 7fcc0affd640 1 -- 172.21.15.169:0/2997590713 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcbfc01f020 con 0x7fcc1414fa10 2023-10-26T01:44:37.516 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.515+0000 7fcc189d0640 1 -- 172.21.15.169:0/2997590713 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcc141540a0 con 0x7fcc1414fa10 2023-10-26T01:44:37.517 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.515+0000 7fcc0affd640 1 -- 172.21.15.169:0/2997590713 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v186) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcbfc015070 con 0x7fcc1414fa10 2023-10-26T01:44:37.518 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 32 2023-10-26T01:44:37.520 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.518+0000 7fcc189d0640 1 -- 172.21.15.169:0/2997590713 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcbf406ae40 msgr2=0x7fcbf406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:37.521 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.518+0000 7fcc189d0640 1 --2- 172.21.15.169:0/2997590713 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcbf406ae40 0x7fcbf406d300 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fcc04037000 tx=0x7fcc04033040 comp rx=0 tx=0).stop 2023-10-26T01:44:37.521 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.518+0000 7fcc189d0640 1 -- 172.21.15.169:0/2997590713 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc1414fa10 msgr2=0x7fcc141596d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:37.521 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.518+0000 7fcc189d0640 1 --2- 172.21.15.169:0/2997590713 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc1414fa10 0x7fcc141596d0 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc00e5e0 tx=0x7fcbfc00eaa0 comp rx=0 tx=0).stop 2023-10-26T01:44:37.521 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.518+0000 7fcc189d0640 1 -- 172.21.15.169:0/2997590713 shutdown_connections 2023-10-26T01:44:37.522 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.518+0000 7fcc189d0640 1 --2- 172.21.15.169:0/2997590713 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcbf406ae40 0x7fcbf406d300 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:37.522 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.518+0000 7fcc189d0640 1 --2- 172.21.15.169:0/2997590713 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc1414fa10 0x7fcc141596d0 unknown :-1 s=CLOSED pgs=1939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:37.522 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.518+0000 7fcc189d0640 1 --2- 172.21.15.169:0/2997590713 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc14159c10 0x7fcc140aded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:37.522 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.518+0000 7fcc189d0640 1 --2- 172.21.15.169:0/2997590713 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcc00003970 0x7fcc140af8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:37.522 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.518+0000 7fcc189d0640 1 -- 172.21.15.169:0/2997590713 >> 172.21.15.169:0/2997590713 conn(0x7fcc14069930 msgr2=0x7fcc1415a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:37.523 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.518+0000 7fcc189d0640 1 -- 172.21.15.169:0/2997590713 shutdown_connections 2023-10-26T01:44:37.523 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.519+0000 7fcc189d0640 1 -- 172.21.15.169:0/2997590713 wait complete. 2023-10-26T01:44:37.538 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-26T01:44:37.761 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.759+0000 7f4604100640 1 -- 172.21.15.169:0/2721418727 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f45fc15b4a0 msgr2=0x7f45fc15b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:37.762 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.759+0000 7f4604100640 1 --2- 172.21.15.169:0/2721418727 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f45fc15b4a0 0x7f45fc15b910 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7f45fc0b3500 tx=0x7f45f402c1d0 comp rx=0 tx=0).stop 2023-10-26T01:44:37.762 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.759+0000 7f4604100640 1 -- 172.21.15.169:0/2721418727 shutdown_connections 2023-10-26T01:44:37.762 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.759+0000 7f4604100640 1 --2- 172.21.15.169:0/2721418727 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f45fc15b4a0 0x7f45fc15b910 unknown :-1 s=CLOSED pgs=1940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:37.763 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.759+0000 7f4604100640 1 --2- 172.21.15.169:0/2721418727 >> v2:172.21.15.170:3300/0 conn(0x7f45fc0cd7e0 0x7f45fc0cdc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:37.763 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.759+0000 7f4604100640 1 --2- 172.21.15.169:0/2721418727 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f45fc06ce00 0x7f45fc155360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:37.763 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.759+0000 7f4604100640 1 -- 172.21.15.169:0/2721418727 >> 172.21.15.169:0/2721418727 conn(0x7f45fc069930 msgr2=0x7f45fc069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:37.763 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.760+0000 7f4604100640 1 -- 172.21.15.169:0/2721418727 shutdown_connections 2023-10-26T01:44:37.763 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.760+0000 7f4604100640 1 -- 172.21.15.169:0/2721418727 wait complete. 2023-10-26T01:44:37.766 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.761+0000 7f4604100640 1 Processor -- start 2023-10-26T01:44:37.766 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.761+0000 7f4604100640 1 -- start start 2023-10-26T01:44:37.766 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.762+0000 7f4604100640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f45fc06ce00 0x7f45fc164090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:37.766 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.762+0000 7f4604100640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f45fc0cd7e0 0x7f45fc14cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:37.767 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.762+0000 7f4604100640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f45fc15b4a0 0x7f45fc14d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:37.767 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.762+0000 7f4604100640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45fc14d260 con 0x7f45fc06ce00 2023-10-26T01:44:37.767 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.762+0000 7f4604100640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45fc14f010 con 0x7f45fc0cd7e0 2023-10-26T01:44:37.767 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.762+0000 7f4604100640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45fc14f2b0 con 0x7f45fc15b4a0 2023-10-26T01:44:37.767 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.763+0000 7f4601e9c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f45fc06ce00 0x7f45fc164090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:37.768 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.763+0000 7f4601e9c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f45fc06ce00 0x7f45fc164090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:36258/0 (socket says 172.21.15.169:36258) 2023-10-26T01:44:37.768 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.763+0000 7f4601e9c640 1 -- 172.21.15.169:0/3764957427 learned_addr learned my addr 172.21.15.169:0/3764957427 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:44:37.768 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.763+0000 7f460269d640 1 --2- 172.21.15.169:0/3764957427 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f45fc15b4a0 0x7f45fc14d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:37.768 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.763+0000 7f4601e9c640 1 -- 172.21.15.169:0/3764957427 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f45fc15b4a0 msgr2=0x7f45fc14d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:37.768 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.763+0000 7f4601e9c640 1 --2- 172.21.15.169:0/3764957427 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f45fc15b4a0 0x7f45fc14d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:37.769 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.763+0000 7f4601e9c640 1 -- 172.21.15.169:0/3764957427 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f45fc0cd7e0 msgr2=0x7f45fc14cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:37.769 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.763+0000 7f4601e9c640 1 --2- 172.21.15.169:0/3764957427 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f45fc0cd7e0 0x7f45fc14cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:37.769 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.763+0000 7f4601e9c640 1 -- 172.21.15.169:0/3764957427 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45f4009900 con 0x7f45fc06ce00 2023-10-26T01:44:37.769 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.763+0000 7f4601e9c640 1 --2- 172.21.15.169:0/3764957427 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f45fc06ce00 0x7f45fc164090 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f45f800d770 tx=0x7f45f800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:37.769 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.764+0000 7f45f27fc640 1 -- 172.21.15.169:0/3764957427 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f8009c30 con 0x7f45fc06ce00 2023-10-26T01:44:37.770 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.764+0000 7f45f27fc640 1 -- 172.21.15.169:0/3764957427 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f45f8009e80 con 0x7f45fc06ce00 2023-10-26T01:44:37.770 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.764+0000 7f45f27fc640 1 -- 172.21.15.169:0/3764957427 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f80106c0 con 0x7f45fc06ce00 2023-10-26T01:44:37.770 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.764+0000 7f4604100640 1 -- 172.21.15.169:0/3764957427 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45fc14f590 con 0x7f45fc06ce00 2023-10-26T01:44:37.770 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.764+0000 7f4604100640 1 -- 172.21.15.169:0/3764957427 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45fc14c800 con 0x7f45fc06ce00 2023-10-26T01:44:37.772 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.765+0000 7f4604100640 1 -- 172.21.15.169:0/3764957427 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45fc14c9c0 con 0x7f45fc06ce00 2023-10-26T01:44:37.772 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.766+0000 7f45f27fc640 1 -- 172.21.15.169:0/3764957427 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f45f8010820 con 0x7f45fc06ce00 2023-10-26T01:44:37.772 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.767+0000 7f45f27fc640 1 --2- 172.21.15.169:0/3764957427 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f45e006ae70 0x7f45e006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:37.772 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.767+0000 7f45f27fc640 1 -- 172.21.15.169:0/3764957427 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f45f8089c70 con 0x7f45fc06ce00 2023-10-26T01:44:37.773 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.767+0000 7f45f27fc640 1 -- 172.21.15.169:0/3764957427 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=187}) v3 -- 0x7f45e00729a0 con 0x7f45fc06ce00 2023-10-26T01:44:37.773 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.768+0000 7f460169b640 1 --2- 172.21.15.169:0/3764957427 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f45e006ae70 0x7f45e006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:37.773 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.769+0000 7f460169b640 1 --2- 172.21.15.169:0/3764957427 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f45e006ae70 0x7f45e006d330 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f45ec005fd0 tx=0x7f45ec00b410 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:37.774 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.771+0000 7f45f27fc640 1 -- 172.21.15.169:0/3764957427 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f45f8021020 con 0x7f45fc06ce00 2023-10-26T01:44:37.964 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.962+0000 7f4604100640 1 -- 172.21.15.169:0/3764957427 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f45fc14c380 con 0x7f45fc06ce00 2023-10-26T01:44:37.965 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.963+0000 7f45f27fc640 1 -- 172.21.15.169:0/3764957427 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v186) v1 ==== 100+0+11 (secure 0 0 0) 0x7f45fc14c380 con 0x7f45fc06ce00 2023-10-26T01:44:37.965 INFO:teuthology.orchestra.run.smithi169.stdout:pg_num: 64 2023-10-26T01:44:37.968 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.965+0000 7f4604100640 1 -- 172.21.15.169:0/3764957427 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f45e006ae70 msgr2=0x7f45e006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:37.968 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.965+0000 7f4604100640 1 --2- 172.21.15.169:0/3764957427 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f45e006ae70 0x7f45e006d330 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f45ec005fd0 tx=0x7f45ec00b410 comp rx=0 tx=0).stop 2023-10-26T01:44:37.968 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.966+0000 7f4604100640 1 -- 172.21.15.169:0/3764957427 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f45fc06ce00 msgr2=0x7f45fc164090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:37.968 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.966+0000 7f4604100640 1 --2- 172.21.15.169:0/3764957427 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f45fc06ce00 0x7f45fc164090 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f45f800d770 tx=0x7f45f800dc30 comp rx=0 tx=0).stop 2023-10-26T01:44:37.969 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.966+0000 7f4604100640 1 -- 172.21.15.169:0/3764957427 shutdown_connections 2023-10-26T01:44:37.969 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.966+0000 7f4604100640 1 --2- 172.21.15.169:0/3764957427 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f45e006ae70 0x7f45e006d330 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:37.969 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.966+0000 7f4604100640 1 --2- 172.21.15.169:0/3764957427 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f45fc15b4a0 0x7f45fc14d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:37.969 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.966+0000 7f4604100640 1 --2- 172.21.15.169:0/3764957427 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f45fc0cd7e0 0x7f45fc14cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:37.970 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.966+0000 7f4604100640 1 --2- 172.21.15.169:0/3764957427 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f45fc06ce00 0x7f45fc164090 unknown :-1 s=CLOSED pgs=1941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:37.970 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.966+0000 7f4604100640 1 -- 172.21.15.169:0/3764957427 >> 172.21.15.169:0/3764957427 conn(0x7f45fc069930 msgr2=0x7f45fc156890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:37.970 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.966+0000 7f4604100640 1 -- 172.21.15.169:0/3764957427 shutdown_connections 2023-10-26T01:44:37.970 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:37.966+0000 7f4604100640 1 -- 172.21.15.169:0/3764957427 wait complete. 2023-10-26T01:44:37.988 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T01:44:38.204 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.202+0000 7f89a68a2640 1 -- 172.21.15.169:0/2644858078 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f898c0047c0 msgr2=0x7f898c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:38.205 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.202+0000 7f89a68a2640 1 --2- 172.21.15.169:0/2644858078 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f898c0047c0 0x7f898c004c40 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7f8988009b50 tx=0x7f898c0057f0 comp rx=0 tx=0).stop 2023-10-26T01:44:38.205 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.202+0000 7f89a4e3f640 1 -- 172.21.15.169:0/2644858078 reap_dead start 2023-10-26T01:44:38.206 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.202+0000 7f89a68a2640 1 -- 172.21.15.169:0/2644858078 shutdown_connections 2023-10-26T01:44:38.206 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.202+0000 7f89a68a2640 1 -- 172.21.15.169:0/2644858078 >> 172.21.15.169:0/2644858078 conn(0x7f89a0069930 msgr2=0x7f89a0069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:38.206 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.202+0000 7f89a68a2640 1 -- 172.21.15.169:0/2644858078 shutdown_connections 2023-10-26T01:44:38.207 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.202+0000 7f89a68a2640 1 -- 172.21.15.169:0/2644858078 wait complete. 2023-10-26T01:44:38.210 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.204+0000 7f89a68a2640 1 Processor -- start 2023-10-26T01:44:38.210 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.204+0000 7f89a68a2640 1 -- start start 2023-10-26T01:44:38.211 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.205+0000 7f89a68a2640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f898c0047c0 0x7f89a015c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:38.211 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.205+0000 7f89a68a2640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f89a013fd50 0x7f89a013e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:38.211 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.205+0000 7f89a68a2640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89a013e730 0x7f89a0140280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:38.211 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.205+0000 7f89a68a2640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89a013f8c0 con 0x7f898c0047c0 2023-10-26T01:44:38.212 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.205+0000 7f89a68a2640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89a013ecb0 con 0x7f89a013e730 2023-10-26T01:44:38.212 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.205+0000 7f89a68a2640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89a013ef90 con 0x7f89a013fd50 2023-10-26T01:44:38.212 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.205+0000 7f899ffff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f898c0047c0 0x7f89a015c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:38.213 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.205+0000 7f899ffff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f898c0047c0 0x7f89a015c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:36280/0 (socket says 172.21.15.169:36280) 2023-10-26T01:44:38.213 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.206+0000 7f899ffff640 1 -- 172.21.15.169:0/3503132318 learned_addr learned my addr 172.21.15.169:0/3503132318 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:44:38.213 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.206+0000 7f8997fff640 1 --2- 172.21.15.169:0/3503132318 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f89a013fd50 0x7f89a013e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:38.213 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.206+0000 7f89a4e3f640 1 --2- 172.21.15.169:0/3503132318 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89a013e730 0x7f89a0140280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:38.214 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.206+0000 7f899ffff640 1 -- 172.21.15.169:0/3503132318 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f89a013fd50 msgr2=0x7f89a013e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:38.214 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.206+0000 7f899ffff640 1 --2- 172.21.15.169:0/3503132318 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f89a013fd50 0x7f89a013e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:38.214 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.206+0000 7f899ffff640 1 -- 172.21.15.169:0/3503132318 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89a013e730 msgr2=0x7f89a0140280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:38.214 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.206+0000 7f899ffff640 1 --2- 172.21.15.169:0/3503132318 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89a013e730 0x7f89a0140280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:38.215 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.206+0000 7f899ffff640 1 -- 172.21.15.169:0/3503132318 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89a013f270 con 0x7f898c0047c0 2023-10-26T01:44:38.215 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.206+0000 7f89a4e3f640 1 --2- 172.21.15.169:0/3503132318 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89a013e730 0x7f89a0140280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:44:38.215 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.206+0000 7f899ffff640 1 --2- 172.21.15.169:0/3503132318 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f898c0047c0 0x7f89a015c2b0 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7f899000d7d0 tx=0x7f899000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:38.216 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.206+0000 7f899d7fa640 1 -- 172.21.15.169:0/3503132318 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89900052e0 con 0x7f898c0047c0 2023-10-26T01:44:38.216 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.207+0000 7f899d7fa640 1 -- 172.21.15.169:0/3503132318 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8990005440 con 0x7f898c0047c0 2023-10-26T01:44:38.216 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.207+0000 7f899d7fa640 1 -- 172.21.15.169:0/3503132318 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f899000b920 con 0x7f898c0047c0 2023-10-26T01:44:38.217 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.207+0000 7f89a68a2640 1 -- 172.21.15.169:0/3503132318 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89a013f550 con 0x7f898c0047c0 2023-10-26T01:44:38.217 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.207+0000 7f89a68a2640 1 -- 172.21.15.169:0/3503132318 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89a0151a70 con 0x7f898c0047c0 2023-10-26T01:44:38.217 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.208+0000 7f89a68a2640 1 -- 172.21.15.169:0/3503132318 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89a013f6e0 con 0x7f898c0047c0 2023-10-26T01:44:38.219 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.209+0000 7f899d7fa640 1 -- 172.21.15.169:0/3503132318 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f899000ba80 con 0x7f898c0047c0 2023-10-26T01:44:38.219 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.211+0000 7f899d7fa640 1 --2- 172.21.15.169:0/3503132318 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f898006ae40 0x7f898006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:38.219 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.211+0000 7f899d7fa640 1 -- 172.21.15.169:0/3503132318 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8990088e10 con 0x7f898c0047c0 2023-10-26T01:44:38.220 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.211+0000 7f899d7fa640 1 -- 172.21.15.169:0/3503132318 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=187}) v3 -- 0x7f8980072970 con 0x7f898c0047c0 2023-10-26T01:44:38.220 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.211+0000 7f8997fff640 1 --2- 172.21.15.169:0/3503132318 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f898006ae40 0x7f898006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:38.220 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.212+0000 7f8997fff640 1 --2- 172.21.15.169:0/3503132318 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f898006ae40 0x7f898006d300 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f8988036000 tx=0x7f8988035040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:38.220 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.216+0000 7f899d7fa640 1 -- 172.21.15.169:0/3503132318 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8990019020 con 0x7f898c0047c0 2023-10-26T01:44:38.334 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T01:44:38.345 DEBUG:teuthology.orchestra.run.smithi170:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T01:44:38.450 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.448+0000 7f89a68a2640 1 -- 172.21.15.169:0/3503132318 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f89a00ce080 con 0x7f898c0047c0 2023-10-26T01:44:38.451 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.449+0000 7f899d7fa640 1 -- 172.21.15.169:0/3503132318 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) v1 ==== 94+0+4920 (secure 0 0 0) 0x7f89a00ce080 con 0x7f898c0047c0 2023-10-26T01:44:38.451 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:44:38.451 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":94,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6484,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/195700049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":195700049},{"type":"v1","addr":"172.21.15.169:6835","nonce":195700049}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":89},{"gid":6493,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/1167696300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1167696300},{"type":"v1","addr":"172.21.15.170:6836","nonce":1167696300}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":89},{"gid":9111,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/22729060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":22729060},{"type":"v1","addr":"172.21.15.169:6834","nonce":22729060}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":89}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T01:43:49.711084+0000","modified":"2023-10-26T01:43:52.816785+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9123},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9123":{"gid":9123,"name":"d","rank":0,"incarnation":90,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6837/2205518515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":2205518515},{"type":"v1","addr":"172.21.15.170:6837","nonce":2205518515}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2023-10-26T01:44:38.452 INFO:teuthology.orchestra.run.smithi169.stderr:dumped fsmap epoch 94 2023-10-26T01:44:38.454 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.451+0000 7f89a68a2640 1 -- 172.21.15.169:0/3503132318 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f898006ae40 msgr2=0x7f898006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:38.454 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.451+0000 7f89a68a2640 1 --2- 172.21.15.169:0/3503132318 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f898006ae40 0x7f898006d300 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f8988036000 tx=0x7f8988035040 comp rx=0 tx=0).stop 2023-10-26T01:44:38.454 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.451+0000 7f89a68a2640 1 -- 172.21.15.169:0/3503132318 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f898c0047c0 msgr2=0x7f89a015c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:38.454 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.451+0000 7f89a68a2640 1 --2- 172.21.15.169:0/3503132318 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f898c0047c0 0x7f89a015c2b0 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7f899000d7d0 tx=0x7f899000dc90 comp rx=0 tx=0).stop 2023-10-26T01:44:38.455 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.451+0000 7f89a68a2640 1 -- 172.21.15.169:0/3503132318 shutdown_connections 2023-10-26T01:44:38.455 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.451+0000 7f89a68a2640 1 --2- 172.21.15.169:0/3503132318 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f898006ae40 0x7f898006d300 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:38.455 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.451+0000 7f89a68a2640 1 --2- 172.21.15.169:0/3503132318 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89a013e730 0x7f89a0140280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:38.455 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.451+0000 7f89a68a2640 1 --2- 172.21.15.169:0/3503132318 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f89a013fd50 0x7f89a013e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:38.456 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.451+0000 7f89a68a2640 1 --2- 172.21.15.169:0/3503132318 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f898c0047c0 0x7f89a015c2b0 unknown :-1 s=CLOSED pgs=1944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:38.456 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.451+0000 7f89a68a2640 1 -- 172.21.15.169:0/3503132318 >> 172.21.15.169:0/3503132318 conn(0x7f89a0069930 msgr2=0x7f89a0157d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:38.456 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.452+0000 7f89a68a2640 1 -- 172.21.15.169:0/3503132318 shutdown_connections 2023-10-26T01:44:38.456 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.452+0000 7f89a68a2640 1 -- 172.21.15.169:0/3503132318 wait complete. 2023-10-26T01:44:38.470 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:44:38.731 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.729+0000 7f0b92c83640 1 -- 172.21.15.169:0/1311233722 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0b8c0ac6d0 msgr2=0x7f0b8c15d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:38.731 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.729+0000 7f0b92c83640 1 --2- 172.21.15.169:0/1311233722 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0b8c0ac6d0 0x7f0b8c15d840 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c0cbd50 tx=0x7f0b8802bda0 comp rx=0 tx=0).stop 2023-10-26T01:44:38.732 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.729+0000 7f0b92c83640 1 -- 172.21.15.169:0/1311233722 shutdown_connections 2023-10-26T01:44:38.732 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.729+0000 7f0b92c83640 1 --2- 172.21.15.169:0/1311233722 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0b8c09a5c0 0x7f0b8c09aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:38.732 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.729+0000 7f0b92c83640 1 --2- 172.21.15.169:0/1311233722 >> v2:172.21.15.170:3300/0 conn(0x7f0b8c0c9970 0x7f0b8c098a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:38.732 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.729+0000 7f0b92c83640 1 --2- 172.21.15.169:0/1311233722 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0b8c0ac6d0 0x7f0b8c15d840 unknown :-1 s=CLOSED pgs=1945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:38.733 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.729+0000 7f0b92c83640 1 -- 172.21.15.169:0/1311233722 >> 172.21.15.169:0/1311233722 conn(0x7f0b8c0aedd0 msgr2=0x7f0b8c0af1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:38.733 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.729+0000 7f0b92c83640 1 -- 172.21.15.169:0/1311233722 shutdown_connections 2023-10-26T01:44:38.733 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.729+0000 7f0b92c83640 1 -- 172.21.15.169:0/1311233722 wait complete. 2023-10-26T01:44:38.735 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.731+0000 7f0b92c83640 1 Processor -- start 2023-10-26T01:44:38.735 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.731+0000 7f0b92c83640 1 -- start start 2023-10-26T01:44:38.735 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.732+0000 7f0b92c83640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0b8c09a5c0 0x7f0b8c0c7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:38.735 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.732+0000 7f0b92c83640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0b8c0ac6d0 0x7f0b8c15ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:38.736 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.732+0000 7f0b92c83640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0b8c0c9970 0x7f0b8c071300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:38.736 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.732+0000 7f0b92c83640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b8c0c8c50 con 0x7f0b8c0c9970 2023-10-26T01:44:38.736 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.732+0000 7f0b92c83640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b8c0c7dc0 con 0x7f0b8c0ac6d0 2023-10-26T01:44:38.736 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.732+0000 7f0b92c83640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b8c15b080 con 0x7f0b8c09a5c0 2023-10-26T01:44:38.736 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.732+0000 7f0b92482640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0b8c0c9970 0x7f0b8c071300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:38.736 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.733+0000 7f0b92482640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0b8c0c9970 0x7f0b8c071300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:36308/0 (socket says 172.21.15.169:36308) 2023-10-26T01:44:38.737 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.733+0000 7f0b92482640 1 -- 172.21.15.169:0/2543382422 learned_addr learned my addr 172.21.15.169:0/2543382422 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:44:38.737 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.733+0000 7f0b91c81640 1 --2- 172.21.15.169:0/2543382422 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0b8c09a5c0 0x7f0b8c0c7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:38.737 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.733+0000 7f0b91480640 1 --2- 172.21.15.169:0/2543382422 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0b8c0ac6d0 0x7f0b8c15ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:38.737 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.733+0000 7f0b92482640 1 -- 172.21.15.169:0/2543382422 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0b8c09a5c0 msgr2=0x7f0b8c0c7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:38.737 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.733+0000 7f0b92482640 1 --2- 172.21.15.169:0/2543382422 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0b8c09a5c0 0x7f0b8c0c7880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:38.738 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.733+0000 7f0b92482640 1 -- 172.21.15.169:0/2543382422 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0b8c0ac6d0 msgr2=0x7f0b8c15ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:38.738 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.733+0000 7f0b92482640 1 --2- 172.21.15.169:0/2543382422 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0b8c0ac6d0 0x7f0b8c15ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:38.738 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.733+0000 7f0b92482640 1 -- 172.21.15.169:0/2543382422 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b88009560 con 0x7f0b8c0c9970 2023-10-26T01:44:38.738 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.733+0000 7f0b92482640 1 --2- 172.21.15.169:0/2543382422 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0b8c0c9970 0x7f0b8c071300 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7f0b8400b970 tx=0x7f0b8400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:38.740 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.734+0000 7f0b827fc640 1 -- 172.21.15.169:0/2543382422 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b8400cb40 con 0x7f0b8c0c9970 2023-10-26T01:44:38.740 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.734+0000 7f0b827fc640 1 -- 172.21.15.169:0/2543382422 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0b8400cdb0 con 0x7f0b8c0c9970 2023-10-26T01:44:38.740 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.734+0000 7f0b92c83640 1 -- 172.21.15.169:0/2543382422 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b8c15d3d0 con 0x7f0b8c0c9970 2023-10-26T01:44:38.740 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.734+0000 7f0b827fc640 1 -- 172.21.15.169:0/2543382422 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b840146a0 con 0x7f0b8c0c9970 2023-10-26T01:44:38.740 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.734+0000 7f0b92c83640 1 -- 172.21.15.169:0/2543382422 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b8c0c82b0 con 0x7f0b8c0c9970 2023-10-26T01:44:38.741 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.735+0000 7f0b92c83640 1 -- 172.21.15.169:0/2543382422 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b8c0c8d90 con 0x7f0b8c0c9970 2023-10-26T01:44:38.741 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.736+0000 7f0b827fc640 1 -- 172.21.15.169:0/2543382422 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f0b8401d3b0 con 0x7f0b8c0c9970 2023-10-26T01:44:38.741 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.737+0000 7f0b827fc640 1 --2- 172.21.15.169:0/2543382422 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f0b7006ae40 0x7f0b7006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:38.741 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.737+0000 7f0b827fc640 1 -- 172.21.15.169:0/2543382422 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0b840897a0 con 0x7f0b8c0c9970 2023-10-26T01:44:38.741 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.738+0000 7f0b827fc640 1 -- 172.21.15.169:0/2543382422 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=187}) v3 -- 0x7f0b70072970 con 0x7f0b8c0c9970 2023-10-26T01:44:38.741 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.738+0000 7f0b91c81640 1 --2- 172.21.15.169:0/2543382422 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f0b7006ae40 0x7f0b7006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:38.742 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.739+0000 7f0b91c81640 1 --2- 172.21.15.169:0/2543382422 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f0b7006ae40 0x7f0b7006d300 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f0b88035000 tx=0x7f0b88004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:38.744 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.742+0000 7f0b827fc640 1 -- 172.21.15.169:0/2543382422 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0b84019070 con 0x7f0b8c0c9970 2023-10-26T01:44:38.923 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.921+0000 7f0b92c83640 1 -- 172.21.15.169:0/2543382422 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0b8c098f20 con 0x7f0b8c0c9970 2023-10-26T01:44:38.926 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.924+0000 7f0b827fc640 1 -- 172.21.15.169:0/2543382422 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v186) v1 ==== 74+0+16646 (secure 0 0 0) 0x7f0b84054840 con 0x7f0b8c0c9970 2023-10-26T01:44:38.926 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:44:38.927 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":186,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:44:33.204279+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2023-10-26T01:43:47.501296+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"181","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2023-10-26T01:43:48.737507+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"186","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:44:38.928 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.926+0000 7f0b92c83640 1 -- 172.21.15.169:0/2543382422 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f0b7006ae40 msgr2=0x7f0b7006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:38.929 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.926+0000 7f0b92c83640 1 --2- 172.21.15.169:0/2543382422 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f0b7006ae40 0x7f0b7006d300 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f0b88035000 tx=0x7f0b88004730 comp rx=0 tx=0).stop 2023-10-26T01:44:38.929 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.927+0000 7f0b92c83640 1 -- 172.21.15.169:0/2543382422 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0b8c0c9970 msgr2=0x7f0b8c071300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:38.929 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.927+0000 7f0b92c83640 1 --2- 172.21.15.169:0/2543382422 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0b8c0c9970 0x7f0b8c071300 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7f0b8400b970 tx=0x7f0b8400be30 comp rx=0 tx=0).stop 2023-10-26T01:44:38.929 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.927+0000 7f0b92c83640 1 -- 172.21.15.169:0/2543382422 shutdown_connections 2023-10-26T01:44:38.930 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.927+0000 7f0b92c83640 1 --2- 172.21.15.169:0/2543382422 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f0b7006ae40 0x7f0b7006d300 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:38.930 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.927+0000 7f0b92c83640 1 --2- 172.21.15.169:0/2543382422 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0b8c0c9970 0x7f0b8c071300 unknown :-1 s=CLOSED pgs=1946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:38.930 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.927+0000 7f0b92c83640 1 --2- 172.21.15.169:0/2543382422 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0b8c0ac6d0 0x7f0b8c15ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:38.930 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.927+0000 7f0b92c83640 1 --2- 172.21.15.169:0/2543382422 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0b8c09a5c0 0x7f0b8c0c7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:38.930 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.927+0000 7f0b92c83640 1 -- 172.21.15.169:0/2543382422 >> 172.21.15.169:0/2543382422 conn(0x7f0b8c0aedd0 msgr2=0x7f0b8c0ab4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:38.931 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.927+0000 7f0b92c83640 1 -- 172.21.15.169:0/2543382422 shutdown_connections 2023-10-26T01:44:38.931 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:38.927+0000 7f0b92c83640 1 -- 172.21.15.169:0/2543382422 wait complete. 2023-10-26T01:44:38.946 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-26T01:44:38.947 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T01:44:39.165 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.163+0000 7f2b0b097640 1 -- 172.21.15.169:0/3231721005 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2b0406ce20 msgr2=0x7f2b0414e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:39.165 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.163+0000 7f2b0b097640 1 --2- 172.21.15.169:0/3231721005 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2b0406ce20 0x7f2b0414e790 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7f2af8009900 tx=0x7f2af802bce0 comp rx=0 tx=0).stop 2023-10-26T01:44:39.166 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.163+0000 7f2b0b097640 1 -- 172.21.15.169:0/3231721005 shutdown_connections 2023-10-26T01:44:39.166 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.163+0000 7f2b0b097640 1 --2- 172.21.15.169:0/3231721005 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2b04159e10 0x7f2b0415a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:39.166 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.163+0000 7f2b0b097640 1 --2- 172.21.15.169:0/3231721005 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2b0406ce20 0x7f2b0414e790 unknown :-1 s=CLOSED pgs=1947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:39.166 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.163+0000 7f2b0b097640 1 -- 172.21.15.169:0/3231721005 >> 172.21.15.169:0/3231721005 conn(0x7f2b04069930 msgr2=0x7f2b04069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:39.167 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.163+0000 7f2b0b097640 1 -- 172.21.15.169:0/3231721005 shutdown_connections 2023-10-26T01:44:39.167 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.163+0000 7f2b0b097640 1 -- 172.21.15.169:0/3231721005 wait complete. 2023-10-26T01:44:39.168 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.165+0000 7f2b0b097640 1 Processor -- start 2023-10-26T01:44:39.168 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.165+0000 7f2b0b097640 1 -- start start 2023-10-26T01:44:39.169 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.166+0000 7f2b0b097640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2b0406ce20 0x7f2b041ed940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:39.169 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.166+0000 7f2b0b097640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2b04158850 0x7f2b0406f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:39.169 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.166+0000 7f2b0b097640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2b04159e10 0x7f2b040aeec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:39.169 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.166+0000 7f2b0b097640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b04070020 con 0x7f2b0406ce20 2023-10-26T01:44:39.169 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.166+0000 7f2b0b097640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b04070300 con 0x7f2b04158850 2023-10-26T01:44:39.170 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.166+0000 7f2b0b097640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b040705e0 con 0x7f2b04159e10 2023-10-26T01:44:39.170 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.166+0000 7f2b08e33640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2b0406ce20 0x7f2b041ed940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:39.170 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.166+0000 7f2b08e33640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2b0406ce20 0x7f2b041ed940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:36334/0 (socket says 172.21.15.169:36334) 2023-10-26T01:44:39.170 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.166+0000 7f2b08e33640 1 -- 172.21.15.169:0/4066049308 learned_addr learned my addr 172.21.15.169:0/4066049308 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:44:39.171 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.167+0000 7f2b03fff640 1 --2- 172.21.15.169:0/4066049308 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2b04158850 0x7f2b0406f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:39.171 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.167+0000 7f2b09634640 1 --2- 172.21.15.169:0/4066049308 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2b04159e10 0x7f2b040aeec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:39.172 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.167+0000 7f2b08e33640 1 -- 172.21.15.169:0/4066049308 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2b04159e10 msgr2=0x7f2b040aeec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:39.172 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.167+0000 7f2b08e33640 1 --2- 172.21.15.169:0/4066049308 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2b04159e10 0x7f2b040aeec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:39.173 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.167+0000 7f2b08e33640 1 -- 172.21.15.169:0/4066049308 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2b04158850 msgr2=0x7f2b0406f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:39.173 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.167+0000 7f2b08e33640 1 --2- 172.21.15.169:0/4066049308 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2b04158850 0x7f2b0406f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:39.173 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.167+0000 7f2b08e33640 1 -- 172.21.15.169:0/4066049308 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2af8009560 con 0x7f2b0406ce20 2023-10-26T01:44:39.173 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.167+0000 7f2b03fff640 1 --2- 172.21.15.169:0/4066049308 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2b04158850 0x7f2b0406f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T01:44:39.173 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.167+0000 7f2b08e33640 1 --2- 172.21.15.169:0/4066049308 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2b0406ce20 0x7f2b041ed940 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7f2af8036040 tx=0x7f2af802ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:39.174 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.168+0000 7f2b017fa640 1 -- 172.21.15.169:0/4066049308 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af802ee00 con 0x7f2b0406ce20 2023-10-26T01:44:39.174 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.168+0000 7f2b0b097640 1 -- 172.21.15.169:0/4066049308 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b040708c0 con 0x7f2b0406ce20 2023-10-26T01:44:39.174 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.168+0000 7f2b017fa640 1 -- 172.21.15.169:0/4066049308 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2af80038f0 con 0x7f2b0406ce20 2023-10-26T01:44:39.174 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.168+0000 7f2b017fa640 1 -- 172.21.15.169:0/4066049308 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af8003cc0 con 0x7f2b0406ce20 2023-10-26T01:44:39.174 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.168+0000 7f2b0b097640 1 -- 172.21.15.169:0/4066049308 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b041ede80 con 0x7f2b0406ce20 2023-10-26T01:44:39.175 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.169+0000 7f2b0b097640 1 -- 172.21.15.169:0/4066049308 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b0404db10 con 0x7f2b0406ce20 2023-10-26T01:44:39.175 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.170+0000 7f2b017fa640 1 -- 172.21.15.169:0/4066049308 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f2af8034440 con 0x7f2b0406ce20 2023-10-26T01:44:39.175 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.171+0000 7f2b017fa640 1 --2- 172.21.15.169:0/4066049308 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2ae806ae70 0x7f2ae806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:39.175 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.171+0000 7f2b017fa640 1 -- 172.21.15.169:0/4066049308 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2af80ae3d0 con 0x7f2b0406ce20 2023-10-26T01:44:39.176 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.171+0000 7f2b017fa640 1 -- 172.21.15.169:0/4066049308 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=187}) v3 -- 0x7f2ae80729a0 con 0x7f2b0406ce20 2023-10-26T01:44:39.176 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.171+0000 7f2b03fff640 1 --2- 172.21.15.169:0/4066049308 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2ae806ae70 0x7f2ae806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:39.176 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.172+0000 7f2b03fff640 1 --2- 172.21.15.169:0/4066049308 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2ae806ae70 0x7f2ae806d330 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f2b0414f7f0 tx=0x7f2af0009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:39.177 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.175+0000 7f2b017fa640 1 -- 172.21.15.169:0/4066049308 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2af80793e0 con 0x7f2b0406ce20 2023-10-26T01:44:39.406 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.405+0000 7f2b0b097640 1 -- 172.21.15.169:0/4066049308 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2b040aca70 con 0x7f2b0406ce20 2023-10-26T01:44:39.407 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.406+0000 7f2b017fa640 1 -- 172.21.15.169:0/4066049308 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) v1 ==== 94+0+4920 (secure 0 0 0) 0x7f2af804d020 con 0x7f2b0406ce20 2023-10-26T01:44:39.408 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:44:39.408 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":94,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6484,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/195700049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":195700049},{"type":"v1","addr":"172.21.15.169:6835","nonce":195700049}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":89},{"gid":6493,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/1167696300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1167696300},{"type":"v1","addr":"172.21.15.170:6836","nonce":1167696300}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":89},{"gid":9111,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/22729060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":22729060},{"type":"v1","addr":"172.21.15.169:6834","nonce":22729060}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":89}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T01:43:49.711084+0000","modified":"2023-10-26T01:43:52.816785+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9123},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9123":{"gid":9123,"name":"d","rank":0,"incarnation":90,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6837/2205518515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":2205518515},{"type":"v1","addr":"172.21.15.170:6837","nonce":2205518515}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2023-10-26T01:44:39.409 INFO:teuthology.orchestra.run.smithi169.stderr:dumped fsmap epoch 94 2023-10-26T01:44:39.411 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.408+0000 7f2b0b097640 1 -- 172.21.15.169:0/4066049308 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2ae806ae70 msgr2=0x7f2ae806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:39.411 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.408+0000 7f2b0b097640 1 --2- 172.21.15.169:0/4066049308 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2ae806ae70 0x7f2ae806d330 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f2b0414f7f0 tx=0x7f2af0009040 comp rx=0 tx=0).stop 2023-10-26T01:44:39.411 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.408+0000 7f2b0b097640 1 -- 172.21.15.169:0/4066049308 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2b0406ce20 msgr2=0x7f2b041ed940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:39.411 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.408+0000 7f2b0b097640 1 --2- 172.21.15.169:0/4066049308 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2b0406ce20 0x7f2b041ed940 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7f2af8036040 tx=0x7f2af802ec10 comp rx=0 tx=0).stop 2023-10-26T01:44:39.411 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.409+0000 7f2b0b097640 1 -- 172.21.15.169:0/4066049308 shutdown_connections 2023-10-26T01:44:39.412 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.409+0000 7f2b0b097640 1 --2- 172.21.15.169:0/4066049308 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f2ae806ae70 0x7f2ae806d330 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:39.412 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.409+0000 7f2b0b097640 1 --2- 172.21.15.169:0/4066049308 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2b04159e10 0x7f2b040aeec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:39.412 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.409+0000 7f2b0b097640 1 --2- 172.21.15.169:0/4066049308 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2b04158850 0x7f2b0406f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:39.412 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.409+0000 7f2b0b097640 1 --2- 172.21.15.169:0/4066049308 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2b0406ce20 0x7f2b041ed940 unknown :-1 s=CLOSED pgs=1948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:39.412 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.409+0000 7f2b0b097640 1 -- 172.21.15.169:0/4066049308 >> 172.21.15.169:0/4066049308 conn(0x7f2b04069930 msgr2=0x7f2b0414d5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:39.413 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.409+0000 7f2b0b097640 1 -- 172.21.15.169:0/4066049308 shutdown_connections 2023-10-26T01:44:39.414 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.409+0000 7f2b0b097640 1 -- 172.21.15.169:0/4066049308 wait complete. 2023-10-26T01:44:39.428 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T01:44:39.650 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.648+0000 7f9de52e0640 1 -- 172.21.15.169:0/1517108891 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9de00ac940 msgr2=0x7f9de00cdbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:39.651 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.648+0000 7f9de52e0640 1 --2- 172.21.15.169:0/1517108891 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9de00ac940 0x7f9de00cdbb0 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7f9de006bff0 tx=0x7f9dd802c260 comp rx=0 tx=0).stop 2023-10-26T01:44:39.651 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.648+0000 7f9de52e0640 1 -- 172.21.15.169:0/1517108891 shutdown_connections 2023-10-26T01:44:39.651 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.648+0000 7f9de52e0640 1 --2- 172.21.15.169:0/1517108891 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9de00ac940 0x7f9de00cdbb0 unknown :-1 s=CLOSED pgs=1949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:39.651 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.648+0000 7f9de52e0640 1 --2- 172.21.15.169:0/1517108891 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9de00ac4e0 0x7f9de00cd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:39.652 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.648+0000 7f9de52e0640 1 --2- 172.21.15.169:0/1517108891 >> v2:172.21.15.170:3300/0 conn(0x7f9de015b610 0x7f9de006ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:39.652 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.648+0000 7f9de52e0640 1 -- 172.21.15.169:0/1517108891 >> 172.21.15.169:0/1517108891 conn(0x7f9de0069930 msgr2=0x7f9de0069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:39.652 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.648+0000 7f9de52e0640 1 -- 172.21.15.169:0/1517108891 shutdown_connections 2023-10-26T01:44:39.652 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.648+0000 7f9de52e0640 1 -- 172.21.15.169:0/1517108891 wait complete. 2023-10-26T01:44:39.654 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.650+0000 7f9de52e0640 1 Processor -- start 2023-10-26T01:44:39.654 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.650+0000 7f9de52e0640 1 -- start start 2023-10-26T01:44:39.654 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.651+0000 7f9de52e0640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9de00ac4e0 0x7f9de0141320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:39.654 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.651+0000 7f9de52e0640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9de00ac940 0x7f9de0143210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:39.654 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.651+0000 7f9de52e0640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9de015b610 0x7f9de0070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:39.655 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.651+0000 7f9de52e0640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9de0143750 con 0x7f9de015b610 2023-10-26T01:44:39.655 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.651+0000 7f9de52e0640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9de0153790 con 0x7f9de00ac940 2023-10-26T01:44:39.655 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.651+0000 7f9de52e0640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9de01f5f10 con 0x7f9de00ac4e0 2023-10-26T01:44:39.655 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.652+0000 7f9ddf7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9de015b610 0x7f9de0070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:39.655 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.652+0000 7f9ddf7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9de015b610 0x7f9de0070470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:36350/0 (socket says 172.21.15.169:36350) 2023-10-26T01:44:39.656 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.652+0000 7f9ddf7fe640 1 -- 172.21.15.169:0/824284516 learned_addr learned my addr 172.21.15.169:0/824284516 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:44:39.656 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.652+0000 7f9dde7fc640 1 --2- 172.21.15.169:0/824284516 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9de00ac940 0x7f9de0143210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:39.656 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.652+0000 7f9ddeffd640 1 --2- 172.21.15.169:0/824284516 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9de00ac4e0 0x7f9de0141320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:39.656 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.652+0000 7f9ddf7fe640 1 -- 172.21.15.169:0/824284516 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9de00ac4e0 msgr2=0x7f9de0141320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:39.656 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.652+0000 7f9ddf7fe640 1 --2- 172.21.15.169:0/824284516 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9de00ac4e0 0x7f9de0141320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:39.656 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.652+0000 7f9ddf7fe640 1 -- 172.21.15.169:0/824284516 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9de00ac940 msgr2=0x7f9de0143210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:39.657 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.652+0000 7f9ddf7fe640 1 --2- 172.21.15.169:0/824284516 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9de00ac940 0x7f9de0143210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:39.657 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.652+0000 7f9ddf7fe640 1 -- 172.21.15.169:0/824284516 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dd8009900 con 0x7f9de015b610 2023-10-26T01:44:39.658 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.652+0000 7f9ddf7fe640 1 --2- 172.21.15.169:0/824284516 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9de015b610 0x7f9de0070470 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7f9dd8037040 tx=0x7f9dd8003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:39.658 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.653+0000 7f9dc77fe640 1 -- 172.21.15.169:0/824284516 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dd8004250 con 0x7f9de015b610 2023-10-26T01:44:39.659 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.653+0000 7f9dc77fe640 1 -- 172.21.15.169:0/824284516 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9dd80044a0 con 0x7f9de015b610 2023-10-26T01:44:39.659 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.653+0000 7f9dc77fe640 1 -- 172.21.15.169:0/824284516 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dd8004890 con 0x7f9de015b610 2023-10-26T01:44:39.659 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.653+0000 7f9de52e0640 1 -- 172.21.15.169:0/824284516 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9de0153140 con 0x7f9de015b610 2023-10-26T01:44:39.659 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.653+0000 7f9de52e0640 1 -- 172.21.15.169:0/824284516 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9de0152da0 con 0x7f9de015b610 2023-10-26T01:44:39.659 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.654+0000 7f9de52e0640 1 -- 172.21.15.169:0/824284516 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9de0153900 con 0x7f9de015b610 2023-10-26T01:44:39.660 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.655+0000 7f9dc77fe640 1 -- 172.21.15.169:0/824284516 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f9dd803b040 con 0x7f9de015b610 2023-10-26T01:44:39.660 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.656+0000 7f9dc77fe640 1 --2- 172.21.15.169:0/824284516 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9dcc06ae40 0x7f9dcc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:39.660 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.656+0000 7f9dc77fe640 1 -- 172.21.15.169:0/824284516 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9dd80ac4c0 con 0x7f9de015b610 2023-10-26T01:44:39.660 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.656+0000 7f9dc77fe640 1 -- 172.21.15.169:0/824284516 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=187}) v3 -- 0x7f9dcc072970 con 0x7f9de015b610 2023-10-26T01:44:39.660 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.657+0000 7f9ddeffd640 1 --2- 172.21.15.169:0/824284516 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9dcc06ae40 0x7f9dcc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:39.661 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.658+0000 7f9ddeffd640 1 --2- 172.21.15.169:0/824284516 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9dcc06ae40 0x7f9dcc06d300 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f9dd000b810 tx=0x7f9dd0008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:39.662 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.660+0000 7f9dc77fe640 1 -- 172.21.15.169:0/824284516 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9dd8077560 con 0x7f9de015b610 2023-10-26T01:44:39.888 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.887+0000 7f9de52e0640 1 -- 172.21.15.169:0/824284516 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9de004db10 con 0x7f9de015b610 2023-10-26T01:44:39.889 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.888+0000 7f9dc77fe640 1 -- 172.21.15.169:0/824284516 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) v1 ==== 94+0+4920 (secure 0 0 0) 0x7f9dd8039020 con 0x7f9de015b610 2023-10-26T01:44:39.890 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:44:39.890 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":94,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6484,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/195700049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":195700049},{"type":"v1","addr":"172.21.15.169:6835","nonce":195700049}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":89},{"gid":6493,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/1167696300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1167696300},{"type":"v1","addr":"172.21.15.170:6836","nonce":1167696300}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":89},{"gid":9111,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/22729060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":22729060},{"type":"v1","addr":"172.21.15.169:6834","nonce":22729060}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":89}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T01:43:49.711084+0000","modified":"2023-10-26T01:43:52.816785+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9123},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9123":{"gid":9123,"name":"d","rank":0,"incarnation":90,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6837/2205518515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":2205518515},{"type":"v1","addr":"172.21.15.170:6837","nonce":2205518515}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2023-10-26T01:44:39.890 INFO:teuthology.orchestra.run.smithi169.stderr:dumped fsmap epoch 94 2023-10-26T01:44:39.892 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.890+0000 7f9de52e0640 1 -- 172.21.15.169:0/824284516 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9dcc06ae40 msgr2=0x7f9dcc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:39.892 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.890+0000 7f9de52e0640 1 --2- 172.21.15.169:0/824284516 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9dcc06ae40 0x7f9dcc06d300 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f9dd000b810 tx=0x7f9dd0008040 comp rx=0 tx=0).stop 2023-10-26T01:44:39.893 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.890+0000 7f9de52e0640 1 -- 172.21.15.169:0/824284516 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9de015b610 msgr2=0x7f9de0070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:39.893 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.890+0000 7f9de52e0640 1 --2- 172.21.15.169:0/824284516 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9de015b610 0x7f9de0070470 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7f9dd8037040 tx=0x7f9dd8003c10 comp rx=0 tx=0).stop 2023-10-26T01:44:39.893 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.890+0000 7f9de52e0640 1 -- 172.21.15.169:0/824284516 shutdown_connections 2023-10-26T01:44:39.893 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.890+0000 7f9de52e0640 1 --2- 172.21.15.169:0/824284516 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9dcc06ae40 0x7f9dcc06d300 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:39.893 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.890+0000 7f9de52e0640 1 --2- 172.21.15.169:0/824284516 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9de015b610 0x7f9de0070470 unknown :-1 s=CLOSED pgs=1950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:39.893 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.890+0000 7f9de52e0640 1 --2- 172.21.15.169:0/824284516 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9de00ac940 0x7f9de0143210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:39.894 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.890+0000 7f9de52e0640 1 --2- 172.21.15.169:0/824284516 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9de00ac4e0 0x7f9de0141320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:39.894 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.890+0000 7f9de52e0640 1 -- 172.21.15.169:0/824284516 >> 172.21.15.169:0/824284516 conn(0x7f9de0069930 msgr2=0x7f9de00b1070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:39.894 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.891+0000 7f9de52e0640 1 -- 172.21.15.169:0/824284516 shutdown_connections 2023-10-26T01:44:39.894 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:39.891+0000 7f9de52e0640 1 -- 172.21.15.169:0/824284516 wait complete. 2023-10-26T01:44:39.912 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:44:40.128 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:40.126+0000 7fdea9755640 1 -- 172.21.15.169:0/3108823021 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fde90004d00 msgr2=0x7fde90005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:40.129 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:40.126+0000 7fdea9755640 1 --2- 172.21.15.169:0/3108823021 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fde90004d00 0x7fde90005180 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7fde9c009e20 tx=0x7fde900057f0 comp rx=0 tx=0).stop 2023-10-26T01:44:40.129 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:40.126+0000 7fdea37fe640 1 -- 172.21.15.169:0/3108823021 reap_dead start 2023-10-26T01:44:40.129 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:40.126+0000 7fdea9755640 1 -- 172.21.15.169:0/3108823021 shutdown_connections 2023-10-26T01:44:40.129 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:40.126+0000 7fdea9755640 1 -- 172.21.15.169:0/3108823021 >> 172.21.15.169:0/3108823021 conn(0x7fdea4069930 msgr2=0x7fdea4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:40.129 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:40.126+0000 7fdea9755640 1 -- 172.21.15.169:0/3108823021 shutdown_connections 2023-10-26T01:44:40.130 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:40.126+0000 7fdea9755640 1 -- 172.21.15.169:0/3108823021 wait complete. 2023-10-26T01:44:40.131 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:40.128+0000 7fdea9755640 1 Processor -- start 2023-10-26T01:44:40.131 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:40.128+0000 7fdea9755640 1 -- start start 2023-10-26T01:44:40.131 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:40.129+0000 7fdea9755640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fde90004d00 0x7fdea4145540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:40.132 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:40.129+0000 7fdea9755640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdea4157390 0x7fdea41577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:40.132 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:40.129+0000 7fdea9755640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdea40700b0 0x7fdea4070540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:40.132 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:40.129+0000 7fdea9755640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdea4157d30 con 0x7fdea40700b0 2023-10-26T01:44:40.132 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:40.129+0000 7fdea9755640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdea41518f0 con 0x7fde90004d00 2023-10-26T01:44:40.132 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:40.129+0000 7fdea9755640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdea406ed30 con 0x7fdea4157390 2023-10-26T01:44:40.133 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:40.129+0000 7fdea37fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdea40700b0 0x7fdea4070540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:40.133 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:40.129+0000 7fdea37fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdea40700b0 0x7fdea4070540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:36382/0 (socket says 172.21.15.169:36382) 2023-10-26T01:44:40.133 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:40.129+0000 7fdea37fe640 1 -- 172.21.15.169:0/2974692150 learned_addr learned my addr 172.21.15.169:0/2974692150 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:44:40.134 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:40.130+0000 7fdea37fe640 1 -- 172.21.15.169:0/2974692150 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdea4157390 msgr2=0x7fdea41577f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:40.135 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:40.130+0000 7fdea37fe640 1 --2- 172.21.15.169:0/2974692150 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdea4157390 0x7fdea41577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:40.135 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:40.130+0000 7fdea2ffd640 1 --2- 172.21.15.169:0/2974692150 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fde90004d00 0x7fdea4145540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:40.135 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:40.130+0000 7fdea37fe640 1 -- 172.21.15.169:0/2974692150 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fde90004d00 msgr2=0x7fdea4145540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:40.135 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:40.130+0000 7fdea37fe640 1 --2- 172.21.15.169:0/2974692150 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fde90004d00 0x7fdea4145540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:40.136 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:40.130+0000 7fdea37fe640 1 -- 172.21.15.169:0/2974692150 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdea406f010 con 0x7fdea40700b0 2023-10-26T01:44:40.136 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:40.130+0000 7fdea37fe640 1 --2- 172.21.15.169:0/2974692150 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdea40700b0 0x7fdea4070540 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7fde9c002230 tx=0x7fde9c035bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:40.136 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:40.130+0000 7fde9bfff640 1 -- 172.21.15.169:0/2974692150 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde9c035270 con 0x7fdea40700b0 2023-10-26T01:44:40.136 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:40.130+0000 7fde9bfff640 1 -- 172.21.15.169:0/2974692150 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fde9c03d410 con 0x7fdea40700b0 2023-10-26T01:44:40.136 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:40.130+0000 7fdea9755640 1 -- 172.21.15.169:0/2974692150 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdea406f2f0 con 0x7fdea40700b0 2023-10-26T01:44:40.137 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:40.131+0000 7fde9bfff640 1 -- 172.21.15.169:0/2974692150 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde9c03d7a0 con 0x7fdea40700b0 2023-10-26T01:44:40.137 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:40.131+0000 7fdea9755640 1 -- 172.21.15.169:0/2974692150 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdea406f8b0 con 0x7fdea40700b0 2023-10-26T01:44:40.137 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:40.132+0000 7fdea9755640 1 -- 172.21.15.169:0/2974692150 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdea404db10 con 0x7fdea40700b0 2023-10-26T01:44:40.137 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:40.133+0000 7fde9bfff640 1 -- 172.21.15.169:0/2974692150 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fde9c03d900 con 0x7fdea40700b0 2023-10-26T01:44:40.138 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:40.134+0000 7fde9bfff640 1 --2- 172.21.15.169:0/2974692150 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fde8406ae70 0x7fde8406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:40.138 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:40.134+0000 7fde9bfff640 1 -- 172.21.15.169:0/2974692150 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fde9c0ac270 con 0x7fdea40700b0 2023-10-26T01:44:40.139 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:40.134+0000 7fde9bfff640 1 -- 172.21.15.169:0/2974692150 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=187}) v3 -- 0x7fde840729a0 con 0x7fdea40700b0 2023-10-26T01:44:40.139 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:40.135+0000 7fdea2ffd640 1 --2- 172.21.15.169:0/2974692150 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fde8406ae70 0x7fde8406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:40.139 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:40.136+0000 7fdea2ffd640 1 --2- 172.21.15.169:0/2974692150 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fde8406ae70 0x7fde8406d330 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fde940017c0 tx=0x7fde94001110 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:40.140 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:40.138+0000 7fde9bfff640 1 -- 172.21.15.169:0/2974692150 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fde9c07fe60 con 0x7fdea40700b0 2023-10-26T01:44:40.333 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:40.331+0000 7fdea9755640 1 -- 172.21.15.169:0/2974692150 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdea4156b90 con 0x7fdea40700b0 2023-10-26T01:44:40.337 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:40.335+0000 7fde9bfff640 1 -- 172.21.15.169:0/2974692150 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v186) v1 ==== 74+0+16646 (secure 0 0 0) 0x7fde9c07fe60 con 0x7fdea40700b0 2023-10-26T01:44:40.337 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:44:40.337 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":186,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:44:33.204279+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2023-10-26T01:43:47.501296+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"181","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2023-10-26T01:43:48.737507+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"186","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:44:40.339 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:40.337+0000 7fdea9755640 1 -- 172.21.15.169:0/2974692150 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fde8406ae70 msgr2=0x7fde8406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:40.340 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:40.337+0000 7fdea9755640 1 --2- 172.21.15.169:0/2974692150 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fde8406ae70 0x7fde8406d330 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fde940017c0 tx=0x7fde94001110 comp rx=0 tx=0).stop 2023-10-26T01:44:40.340 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:40.337+0000 7fdea9755640 1 -- 172.21.15.169:0/2974692150 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdea40700b0 msgr2=0x7fdea4070540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:40.340 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:40.337+0000 7fdea9755640 1 --2- 172.21.15.169:0/2974692150 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdea40700b0 0x7fdea4070540 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7fde9c002230 tx=0x7fde9c035bf0 comp rx=0 tx=0).stop 2023-10-26T01:44:40.340 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:40.338+0000 7fdea9755640 1 -- 172.21.15.169:0/2974692150 shutdown_connections 2023-10-26T01:44:40.340 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:40.338+0000 7fdea9755640 1 --2- 172.21.15.169:0/2974692150 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fde8406ae70 0x7fde8406d330 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:40.341 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:40.338+0000 7fdea9755640 1 --2- 172.21.15.169:0/2974692150 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdea40700b0 0x7fdea4070540 unknown :-1 s=CLOSED pgs=1953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:40.341 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:40.338+0000 7fdea9755640 1 --2- 172.21.15.169:0/2974692150 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdea4157390 0x7fdea41577f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:40.341 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:40.338+0000 7fdea9755640 1 --2- 172.21.15.169:0/2974692150 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fde90004d00 0x7fdea4145540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:40.341 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:40.338+0000 7fdea9755640 1 -- 172.21.15.169:0/2974692150 >> 172.21.15.169:0/2974692150 conn(0x7fdea4069930 msgr2=0x7fdea41610d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:40.341 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:40.338+0000 7fdea9755640 1 -- 172.21.15.169:0/2974692150 shutdown_connections 2023-10-26T01:44:40.342 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:40.338+0000 7fdea9755640 1 -- 172.21.15.169:0/2974692150 wait complete. 2023-10-26T01:44:40.359 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-26T01:44:40.580 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:40.578+0000 7fbb35890640 1 -- 172.21.15.169:0/1556731604 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb30153b90 msgr2=0x7fbb30069900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:40.580 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:40.578+0000 7fbb35890640 1 --2- 172.21.15.169:0/1556731604 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb30153b90 0x7fbb30069900 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7fbb3013fc00 tx=0x7fbb2402bd70 comp rx=0 tx=0).stop 2023-10-26T01:44:40.580 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:40.578+0000 7fbb35890640 1 -- 172.21.15.169:0/1556731604 shutdown_connections 2023-10-26T01:44:40.581 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:40.578+0000 7fbb35890640 1 --2- 172.21.15.169:0/1556731604 >> v2:172.21.15.170:3300/0 conn(0x7fbb300b0e50 0x7fbb30069e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:40.581 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:40.578+0000 7fbb35890640 1 --2- 172.21.15.169:0/1556731604 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb30153b90 0x7fbb30069900 unknown :-1 s=CLOSED pgs=1955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:40.581 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:40.578+0000 7fbb35890640 1 -- 172.21.15.169:0/1556731604 >> 172.21.15.169:0/1556731604 conn(0x7fbb300af110 msgr2=0x7fbb300af520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:40.581 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:40.578+0000 7fbb35890640 1 -- 172.21.15.169:0/1556731604 shutdown_connections 2023-10-26T01:44:40.581 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:40.578+0000 7fbb35890640 1 -- 172.21.15.169:0/1556731604 wait complete. 2023-10-26T01:44:40.583 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:40.580+0000 7fbb35890640 1 Processor -- start 2023-10-26T01:44:40.583 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:40.580+0000 7fbb35890640 1 -- start start 2023-10-26T01:44:40.583 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:40.581+0000 7fbb35890640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbb300b0e50 0x7fbb300c65a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:40.583 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:40.581+0000 7fbb35890640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb3015d0d0 0x7fbb301f14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:40.584 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:40.581+0000 7fbb35890640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb30158d00 0x7fbb3006bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:40.584 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:40.581+0000 7fbb35890640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb3006b4e0 con 0x7fbb30158d00 2023-10-26T01:44:40.584 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:40.581+0000 7fbb35890640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb30159200 con 0x7fbb3015d0d0 2023-10-26T01:44:40.584 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:40.581+0000 7fbb35890640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb301f1a30 con 0x7fbb300b0e50 2023-10-26T01:44:40.585 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:40.581+0000 7fbb3508f640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb30158d00 0x7fbb3006bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:40.585 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:40.581+0000 7fbb3508f640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb30158d00 0x7fbb3006bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:36400/0 (socket says 172.21.15.169:36400) 2023-10-26T01:44:40.585 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:40.581+0000 7fbb3508f640 1 -- 172.21.15.169:0/3603432344 learned_addr learned my addr 172.21.15.169:0/3603432344 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:44:40.586 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:40.582+0000 7fbb3508f640 1 -- 172.21.15.169:0/3603432344 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbb300b0e50 msgr2=0x7fbb300c65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:40.587 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:40.582+0000 7fbb3508f640 1 --2- 172.21.15.169:0/3603432344 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbb300b0e50 0x7fbb300c65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:40.587 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:40.582+0000 7fbb3508f640 1 -- 172.21.15.169:0/3603432344 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb3015d0d0 msgr2=0x7fbb301f14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:40.587 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:40.582+0000 7fbb3508f640 1 --2- 172.21.15.169:0/3603432344 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb3015d0d0 0x7fbb301f14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:40.587 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:40.582+0000 7fbb3508f640 1 -- 172.21.15.169:0/3603432344 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb24009490 con 0x7fbb30158d00 2023-10-26T01:44:40.587 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:40.582+0000 7fbb3508f640 1 --2- 172.21.15.169:0/3603432344 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb30158d00 0x7fbb3006bc50 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7fbb2800cb10 tx=0x7fbb2800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:40.588 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:40.582+0000 7fbb2d7fa640 1 -- 172.21.15.169:0/3603432344 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb2800ac70 con 0x7fbb30158d00 2023-10-26T01:44:40.588 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:40.582+0000 7fbb2d7fa640 1 -- 172.21.15.169:0/3603432344 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fbb28012420 con 0x7fbb30158d00 2023-10-26T01:44:40.588 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:40.582+0000 7fbb2d7fa640 1 -- 172.21.15.169:0/3603432344 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb28012830 con 0x7fbb30158d00 2023-10-26T01:44:40.588 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:40.582+0000 7fbb35890640 1 -- 172.21.15.169:0/3603432344 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb30071430 con 0x7fbb30158d00 2023-10-26T01:44:40.589 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:40.582+0000 7fbb35890640 1 -- 172.21.15.169:0/3603432344 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb3006b730 con 0x7fbb30158d00 2023-10-26T01:44:40.589 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:40.583+0000 7fbb35890640 1 -- 172.21.15.169:0/3603432344 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb3006b950 con 0x7fbb30158d00 2023-10-26T01:44:40.589 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:40.584+0000 7fbb2d7fa640 1 -- 172.21.15.169:0/3603432344 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fbb2801b400 con 0x7fbb30158d00 2023-10-26T01:44:40.590 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:40.585+0000 7fbb2d7fa640 1 --2- 172.21.15.169:0/3603432344 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbb1406ae40 0x7fbb1406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:40.590 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:40.585+0000 7fbb2d7fa640 1 -- 172.21.15.169:0/3603432344 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbb2808a100 con 0x7fbb30158d00 2023-10-26T01:44:40.591 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:40.586+0000 7fbb2d7fa640 1 -- 172.21.15.169:0/3603432344 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=187}) v3 -- 0x7fbb14072970 con 0x7fbb30158d00 2023-10-26T01:44:40.591 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:40.586+0000 7fbb3488e640 1 --2- 172.21.15.169:0/3603432344 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbb1406ae40 0x7fbb1406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:40.591 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:40.587+0000 7fbb3488e640 1 --2- 172.21.15.169:0/3603432344 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbb1406ae40 0x7fbb1406d300 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fbb24036000 tx=0x7fbb240046c0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:40.592 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:40.589+0000 7fbb2d7fa640 1 -- 172.21.15.169:0/3603432344 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbb280551a0 con 0x7fbb30158d00 2023-10-26T01:44:40.812 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:40.810+0000 7fbb35890640 1 -- 172.21.15.169:0/3603432344 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fbb30073c10 con 0x7fbb30158d00 2023-10-26T01:44:40.813 INFO:tasks.ceph.mon.a.smithi169.stderr:2023-10-26T01:44:40.812+0000 7f1588a74640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-26T01:44:40.817 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:40.816+0000 7fbb2d7fa640 1 -- 172.21.15.169:0/3603432344 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v95) v1 ==== 161+0+0 (secure 0 0 0) 0x7fbb28016020 con 0x7fbb30158d00 2023-10-26T01:44:40.818 INFO:teuthology.orchestra.run.smithi169.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-26T01:44:40.819 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:40.817+0000 7fbb2d7fa640 1 -- 172.21.15.169:0/3603432344 <== mon.0 v2:172.21.15.169:3300/0 8 ==== osd_map(187..187 src has 1..187) v4 ==== 374+0+0 (secure 0 0 0) 0x7fbb28059560 con 0x7fbb30158d00 2023-10-26T01:44:40.819 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:40.818+0000 7fbb2d7fa640 1 -- 172.21.15.169:0/3603432344 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=188}) v3 -- 0x7fbb14073370 con 0x7fbb30158d00 2023-10-26T01:44:40.822 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:40.819+0000 7fbb35890640 1 -- 172.21.15.169:0/3603432344 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbb1406ae40 msgr2=0x7fbb1406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:40.823 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:40.819+0000 7fbb35890640 1 --2- 172.21.15.169:0/3603432344 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbb1406ae40 0x7fbb1406d300 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fbb24036000 tx=0x7fbb240046c0 comp rx=0 tx=0).stop 2023-10-26T01:44:40.823 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:40.819+0000 7fbb35890640 1 -- 172.21.15.169:0/3603432344 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb30158d00 msgr2=0x7fbb3006bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:40.823 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:40.819+0000 7fbb35890640 1 --2- 172.21.15.169:0/3603432344 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb30158d00 0x7fbb3006bc50 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7fbb2800cb10 tx=0x7fbb2800a060 comp rx=0 tx=0).stop 2023-10-26T01:44:40.823 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:40.821+0000 7fbb35890640 1 -- 172.21.15.169:0/3603432344 shutdown_connections 2023-10-26T01:44:40.823 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:40.821+0000 7fbb35890640 1 --2- 172.21.15.169:0/3603432344 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbb1406ae40 0x7fbb1406d300 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:40.824 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:40.821+0000 7fbb35890640 1 --2- 172.21.15.169:0/3603432344 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb30158d00 0x7fbb3006bc50 unknown :-1 s=CLOSED pgs=1956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:40.824 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:40.821+0000 7fbb35890640 1 --2- 172.21.15.169:0/3603432344 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb3015d0d0 0x7fbb301f14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:40.824 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:40.821+0000 7fbb35890640 1 --2- 172.21.15.169:0/3603432344 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbb300b0e50 0x7fbb300c65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:40.824 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:40.821+0000 7fbb35890640 1 -- 172.21.15.169:0/3603432344 >> 172.21.15.169:0/3603432344 conn(0x7fbb300af110 msgr2=0x7fbb3009ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:40.824 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:40.821+0000 7fbb35890640 1 -- 172.21.15.169:0/3603432344 shutdown_connections 2023-10-26T01:44:40.825 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:40.821+0000 7fbb35890640 1 -- 172.21.15.169:0/3603432344 wait complete. 2023-10-26T01:44:40.840 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-26T01:44:40.859 INFO:tasks.ceph.mds.d.smithi170.stderr: -9397> 2023-10-26T01:43:54.409+0000 7fc2f725d640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-26T01:44:40.940 INFO:tasks.ceph.mds.d.smithi170.stdout:starting mds.d at 2023-10-26T01:44:41.035 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:41.033+0000 7ff767bf5640 1 -- 172.21.15.169:0/2616610312 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7600b0c80 msgr2=0x7ff7600cd710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:41.036 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:41.033+0000 7ff767bf5640 1 --2- 172.21.15.169:0/2616610312 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7600b0c80 0x7ff7600cd710 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7ff758005fd0 tx=0x7ff75802c240 comp rx=0 tx=0).stop 2023-10-26T01:44:41.036 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:41.033+0000 7ff767bf5640 1 -- 172.21.15.169:0/2616610312 shutdown_connections 2023-10-26T01:44:41.036 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:41.033+0000 7ff767bf5640 1 --2- 172.21.15.169:0/2616610312 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7600b0c80 0x7ff7600cd710 unknown :-1 s=CLOSED pgs=1957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:41.036 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:41.033+0000 7ff767bf5640 1 --2- 172.21.15.169:0/2616610312 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff7601593e0 0x7ff7600ac200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:41.037 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:41.033+0000 7ff767bf5640 1 -- 172.21.15.169:0/2616610312 >> 172.21.15.169:0/2616610312 conn(0x7ff760069930 msgr2=0x7ff760069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:41.037 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:41.033+0000 7ff767bf5640 1 -- 172.21.15.169:0/2616610312 shutdown_connections 2023-10-26T01:44:41.037 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:41.033+0000 7ff767bf5640 1 -- 172.21.15.169:0/2616610312 wait complete. 2023-10-26T01:44:41.039 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:41.035+0000 7ff767bf5640 1 Processor -- start 2023-10-26T01:44:41.039 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:41.035+0000 7ff767bf5640 1 -- start start 2023-10-26T01:44:41.039 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:41.036+0000 7ff767bf5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7600b0c80 0x7ff7601f24d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:41.040 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:41.036+0000 7ff767bf5640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff7601593e0 0x7ff76006ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:41.040 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:41.036+0000 7ff767bf5640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7601597c0 0x7ff76006bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:41.040 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:41.036+0000 7ff767bf5640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff76006c100 con 0x7ff7601597c0 2023-10-26T01:44:41.040 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:41.036+0000 7ff767bf5640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7601f2b20 con 0x7ff7600b0c80 2023-10-26T01:44:41.040 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:41.036+0000 7ff767bf5640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7601f2e00 con 0x7ff7601593e0 2023-10-26T01:44:41.041 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:41.036+0000 7ff766192640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7601597c0 0x7ff76006bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:41.041 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:41.036+0000 7ff766192640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7601597c0 0x7ff76006bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:36424/0 (socket says 172.21.15.169:36424) 2023-10-26T01:44:41.041 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:41.036+0000 7ff766192640 1 -- 172.21.15.169:0/3215547749 learned_addr learned my addr 172.21.15.169:0/3215547749 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:44:41.041 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:41.037+0000 7ff765991640 1 --2- 172.21.15.169:0/3215547749 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7600b0c80 0x7ff7601f24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:41.041 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:41.037+0000 7ff766192640 1 -- 172.21.15.169:0/3215547749 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff7601593e0 msgr2=0x7ff76006ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:41.042 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:41.037+0000 7ff765190640 1 --2- 172.21.15.169:0/3215547749 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff7601593e0 0x7ff76006ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:41.042 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:41.037+0000 7ff766192640 1 --2- 172.21.15.169:0/3215547749 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff7601593e0 0x7ff76006ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:41.042 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:41.037+0000 7ff766192640 1 -- 172.21.15.169:0/3215547749 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7600b0c80 msgr2=0x7ff7601f24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:41.042 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:41.037+0000 7ff766192640 1 --2- 172.21.15.169:0/3215547749 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7600b0c80 0x7ff7601f24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:41.042 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:41.037+0000 7ff766192640 1 -- 172.21.15.169:0/3215547749 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff758009900 con 0x7ff7601597c0 2023-10-26T01:44:41.043 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:41.037+0000 7ff765991640 1 --2- 172.21.15.169:0/3215547749 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7600b0c80 0x7ff7601f24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:44:41.043 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:41.037+0000 7ff766192640 1 --2- 172.21.15.169:0/3215547749 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7601597c0 0x7ff76006bbc0 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7ff758037040 tx=0x7ff758035a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:41.043 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:41.037+0000 7ff7527fc640 1 -- 172.21.15.169:0/3215547749 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff758034440 con 0x7ff7601597c0 2023-10-26T01:44:41.043 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:41.037+0000 7ff7527fc640 1 -- 172.21.15.169:0/3215547749 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff758004330 con 0x7ff7601597c0 2023-10-26T01:44:41.043 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:41.037+0000 7ff767bf5640 1 -- 172.21.15.169:0/3215547749 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7601f30e0 con 0x7ff7601597c0 2023-10-26T01:44:41.044 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:41.038+0000 7ff7527fc640 1 -- 172.21.15.169:0/3215547749 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7580046e0 con 0x7ff7601597c0 2023-10-26T01:44:41.044 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:41.038+0000 7ff767bf5640 1 -- 172.21.15.169:0/3215547749 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7600c8650 con 0x7ff7601597c0 2023-10-26T01:44:41.045 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:41.039+0000 7ff767bf5640 1 -- 172.21.15.169:0/3215547749 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7600c84b0 con 0x7ff7601597c0 2023-10-26T01:44:41.045 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:41.040+0000 7ff7527fc640 1 -- 172.21.15.169:0/3215547749 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ff75803b040 con 0x7ff7601597c0 2023-10-26T01:44:41.045 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:41.041+0000 7ff7527fc640 1 --2- 172.21.15.169:0/3215547749 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff74406ae70 0x7ff74406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:41.045 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:41.041+0000 7ff7527fc640 1 -- 172.21.15.169:0/3215547749 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 5795+0+0 (secure 0 0 0) 0x7ff7580ac330 con 0x7ff7601597c0 2023-10-26T01:44:41.045 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:41.041+0000 7ff7527fc640 1 -- 172.21.15.169:0/3215547749 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=188}) v3 -- 0x7ff744072ab0 con 0x7ff7601597c0 2023-10-26T01:44:41.046 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:41.041+0000 7ff765991640 1 --2- 172.21.15.169:0/3215547749 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff74406ae70 0x7ff74406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:41.046 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:41.042+0000 7ff765991640 1 --2- 172.21.15.169:0/3215547749 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff74406ae70 0x7ff74406d330 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7ff75c0058c0 tx=0x7ff75c005d30 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:41.047 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:41.045+0000 7ff7527fc640 1 -- 172.21.15.169:0/3215547749 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff7600c84b0 con 0x7ff7601597c0 2023-10-26T01:44:41.274 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:41.272+0000 7ff767bf5640 1 -- 172.21.15.169:0/3215547749 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7ff7600c84b0 con 0x7ff7601597c0 2023-10-26T01:44:41.821 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:41.819+0000 7ff7527fc640 1 -- 172.21.15.169:0/3215547749 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v96) v1 ==== 104+0+0 (secure 0 0 0) 0x7ff7600c84b0 con 0x7ff7601597c0 2023-10-26T01:44:41.825 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:41.823+0000 7ff767bf5640 1 -- 172.21.15.169:0/3215547749 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff74406ae70 msgr2=0x7ff74406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:41.825 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:41.823+0000 7ff767bf5640 1 --2- 172.21.15.169:0/3215547749 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff74406ae70 0x7ff74406d330 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7ff75c0058c0 tx=0x7ff75c005d30 comp rx=0 tx=0).stop 2023-10-26T01:44:41.825 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:41.823+0000 7ff767bf5640 1 -- 172.21.15.169:0/3215547749 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7601597c0 msgr2=0x7ff76006bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:41.825 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:41.823+0000 7ff767bf5640 1 --2- 172.21.15.169:0/3215547749 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7601597c0 0x7ff76006bbc0 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7ff758037040 tx=0x7ff758035a40 comp rx=0 tx=0).stop 2023-10-26T01:44:41.826 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:41.823+0000 7ff767bf5640 1 -- 172.21.15.169:0/3215547749 shutdown_connections 2023-10-26T01:44:41.826 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:41.823+0000 7ff767bf5640 1 --2- 172.21.15.169:0/3215547749 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7ff74406ae70 0x7ff74406d330 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:41.826 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:41.823+0000 7ff767bf5640 1 --2- 172.21.15.169:0/3215547749 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7601597c0 0x7ff76006bbc0 unknown :-1 s=CLOSED pgs=1958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:41.826 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:41.823+0000 7ff767bf5640 1 --2- 172.21.15.169:0/3215547749 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff7601593e0 0x7ff76006ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:41.827 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:41.823+0000 7ff767bf5640 1 --2- 172.21.15.169:0/3215547749 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7600b0c80 0x7ff7601f24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:41.827 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:41.823+0000 7ff767bf5640 1 -- 172.21.15.169:0/3215547749 >> 172.21.15.169:0/3215547749 conn(0x7ff760069930 msgr2=0x7ff76015c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:41.827 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:41.823+0000 7ff767bf5640 1 -- 172.21.15.169:0/3215547749 shutdown_connections 2023-10-26T01:44:41.828 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:41.823+0000 7ff767bf5640 1 -- 172.21.15.169:0/3215547749 wait complete. 2023-10-26T01:44:41.844 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-26T01:44:42.057 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:42.055+0000 7f47a5226640 1 -- 172.21.15.169:0/2508934080 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f47a00ac2f0 msgr2=0x7f47a00ac7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:42.058 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:42.055+0000 7f47a5226640 1 --2- 172.21.15.169:0/2508934080 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f47a00ac2f0 0x7f47a00ac7a0 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7f4798005fd0 tx=0x7f479802c1b0 comp rx=0 tx=0).stop 2023-10-26T01:44:42.058 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:42.055+0000 7f47a5226640 1 -- 172.21.15.169:0/2508934080 shutdown_connections 2023-10-26T01:44:42.058 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:42.055+0000 7f47a5226640 1 --2- 172.21.15.169:0/2508934080 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f47a00ac2f0 0x7f47a00ac7a0 unknown :-1 s=CLOSED pgs=1959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:42.059 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:42.055+0000 7f47a5226640 1 --2- 172.21.15.169:0/2508934080 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f47a015d120 0x7f47a00c9bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:42.059 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:42.055+0000 7f47a5226640 1 -- 172.21.15.169:0/2508934080 >> 172.21.15.169:0/2508934080 conn(0x7f47a00aeee0 msgr2=0x7f47a00af2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:42.059 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:42.056+0000 7f47a5226640 1 -- 172.21.15.169:0/2508934080 shutdown_connections 2023-10-26T01:44:42.059 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:42.056+0000 7f47a5226640 1 -- 172.21.15.169:0/2508934080 wait complete. 2023-10-26T01:44:42.061 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:42.057+0000 7f47a5226640 1 Processor -- start 2023-10-26T01:44:42.061 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:42.057+0000 7f47a5226640 1 -- start start 2023-10-26T01:44:42.061 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:42.058+0000 7f47a5226640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f47a00ac2f0 0x7f47a00c7640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:42.062 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:42.058+0000 7f47a5226640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f47a01595e0 0x7f47a00aabe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:42.062 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:42.058+0000 7f47a5226640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f47a015d120 0x7f47a015c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:42.062 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:42.058+0000 7f47a5226640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47a00aa9e0 con 0x7f47a01595e0 2023-10-26T01:44:42.062 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:42.058+0000 7f47a5226640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47a01f29b0 con 0x7f47a015d120 2023-10-26T01:44:42.062 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:42.058+0000 7f47a5226640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47a01f2c90 con 0x7f47a00ac2f0 2023-10-26T01:44:42.063 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:42.059+0000 7f4797fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f47a01595e0 0x7f47a00aabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:42.063 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:42.059+0000 7f4797fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f47a01595e0 0x7f47a00aabe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:36442/0 (socket says 172.21.15.169:36442) 2023-10-26T01:44:42.063 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:42.059+0000 7f4797fff640 1 -- 172.21.15.169:0/3329500399 learned_addr learned my addr 172.21.15.169:0/3329500399 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:44:42.063 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:42.059+0000 7f479ed9d640 1 --2- 172.21.15.169:0/3329500399 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f47a00ac2f0 0x7f47a00c7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:42.064 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:42.059+0000 7f479f59e640 1 --2- 172.21.15.169:0/3329500399 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f47a015d120 0x7f47a015c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:42.064 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:42.059+0000 7f4797fff640 1 -- 172.21.15.169:0/3329500399 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f47a00ac2f0 msgr2=0x7f47a00c7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:42.064 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:42.059+0000 7f4797fff640 1 --2- 172.21.15.169:0/3329500399 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f47a00ac2f0 0x7f47a00c7640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:42.064 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:42.059+0000 7f4797fff640 1 -- 172.21.15.169:0/3329500399 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f47a015d120 msgr2=0x7f47a015c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:42.064 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:42.059+0000 7f4797fff640 1 --2- 172.21.15.169:0/3329500399 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f47a015d120 0x7f47a015c020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:42.065 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:42.059+0000 7f4797fff640 1 -- 172.21.15.169:0/3329500399 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4798009900 con 0x7f47a01595e0 2023-10-26T01:44:42.065 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:42.060+0000 7f4797fff640 1 --2- 172.21.15.169:0/3329500399 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f47a01595e0 0x7f47a00aabe0 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7f478800c970 tx=0x7f478800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:42.066 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:42.060+0000 7f47977fe640 1 -- 172.21.15.169:0/3329500399 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4788007bb0 con 0x7f47a01595e0 2023-10-26T01:44:42.066 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:42.060+0000 7f47977fe640 1 -- 172.21.15.169:0/3329500399 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4788007e20 con 0x7f47a01595e0 2023-10-26T01:44:42.067 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:42.060+0000 7f47977fe640 1 -- 172.21.15.169:0/3329500399 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4788005280 con 0x7f47a01595e0 2023-10-26T01:44:42.067 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:42.060+0000 7f47a5226640 1 -- 172.21.15.169:0/3329500399 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47a01f2f70 con 0x7f47a01595e0 2023-10-26T01:44:42.067 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:42.060+0000 7f47a5226640 1 -- 172.21.15.169:0/3329500399 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47a01f3530 con 0x7f47a01595e0 2023-10-26T01:44:42.067 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:42.061+0000 7f47a5226640 1 -- 172.21.15.169:0/3329500399 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47a0066af0 con 0x7f47a01595e0 2023-10-26T01:44:42.067 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:42.062+0000 7f47977fe640 1 -- 172.21.15.169:0/3329500399 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f47880053e0 con 0x7f47a01595e0 2023-10-26T01:44:42.068 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:42.063+0000 7f47977fe640 1 --2- 172.21.15.169:0/3329500399 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f478006ae40 0x7f478006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:42.068 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:42.063+0000 7f47977fe640 1 -- 172.21.15.169:0/3329500399 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f4788089be0 con 0x7f47a01595e0 2023-10-26T01:44:42.068 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:42.063+0000 7f47977fe640 1 -- 172.21.15.169:0/3329500399 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=188}) v3 -- 0x7f4780072a80 con 0x7f47a01595e0 2023-10-26T01:44:42.068 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:42.064+0000 7f479ed9d640 1 --2- 172.21.15.169:0/3329500399 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f478006ae40 0x7f478006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:42.069 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:42.065+0000 7f479ed9d640 1 --2- 172.21.15.169:0/3329500399 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f478006ae40 0x7f478006d300 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f47a00aecb0 tx=0x7f4790009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:42.069 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:42.068+0000 7f47977fe640 1 -- 172.21.15.169:0/3329500399 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f478801d030 con 0x7f47a01595e0 2023-10-26T01:44:42.244 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:42.242+0000 7f47a5226640 1 -- 172.21.15.169:0/3329500399 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f47a00b0470 con 0x7f47a01595e0 2023-10-26T01:44:42.825 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:42.823+0000 7f47977fe640 1 -- 172.21.15.169:0/3329500399 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v188) v1 ==== 181+0+0 (secure 0 0 0) 0x7f47a00b0470 con 0x7f47a01595e0 2023-10-26T01:44:42.825 INFO:teuthology.orchestra.run.smithi169.stderr:pool 'cephfs_metadata' removed 2023-10-26T01:44:42.826 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:42.824+0000 7f47977fe640 1 -- 172.21.15.169:0/3329500399 <== mon.0 v2:172.21.15.169:3300/0 8 ==== osd_map(188..188 src has 1..188) v4 ==== 296+0+0 (secure 0 0 0) 0x7f4788005690 con 0x7f47a01595e0 2023-10-26T01:44:42.826 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:42.824+0000 7f47977fe640 1 -- 172.21.15.169:0/3329500399 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=189}) v3 -- 0x7f4780070eb0 con 0x7f47a01595e0 2023-10-26T01:44:42.831 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:42.829+0000 7f47a5226640 1 -- 172.21.15.169:0/3329500399 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f478006ae40 msgr2=0x7f478006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:42.834 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:42.829+0000 7f47a5226640 1 --2- 172.21.15.169:0/3329500399 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f478006ae40 0x7f478006d300 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f47a00aecb0 tx=0x7f4790009040 comp rx=0 tx=0).stop 2023-10-26T01:44:42.834 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:42.829+0000 7f47a5226640 1 -- 172.21.15.169:0/3329500399 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f47a01595e0 msgr2=0x7f47a00aabe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:42.834 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:42.829+0000 7f47a5226640 1 --2- 172.21.15.169:0/3329500399 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f47a01595e0 0x7f47a00aabe0 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7f478800c970 tx=0x7f478800ce30 comp rx=0 tx=0).stop 2023-10-26T01:44:42.834 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:42.830+0000 7f47a5226640 1 -- 172.21.15.169:0/3329500399 shutdown_connections 2023-10-26T01:44:42.835 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:42.830+0000 7f47a5226640 1 --2- 172.21.15.169:0/3329500399 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f478006ae40 0x7f478006d300 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:42.835 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:42.830+0000 7f47a5226640 1 --2- 172.21.15.169:0/3329500399 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f47a01595e0 0x7f47a00aabe0 unknown :-1 s=CLOSED pgs=1960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:42.835 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:42.830+0000 7f47a5226640 1 --2- 172.21.15.169:0/3329500399 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f47a015d120 0x7f47a015c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:42.835 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:42.830+0000 7f47a5226640 1 --2- 172.21.15.169:0/3329500399 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f47a00ac2f0 0x7f47a00c7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:42.836 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:42.830+0000 7f47a5226640 1 -- 172.21.15.169:0/3329500399 >> 172.21.15.169:0/3329500399 conn(0x7f47a00aeee0 msgr2=0x7f47a015d670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:42.836 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:42.831+0000 7f47a5226640 1 -- 172.21.15.169:0/3329500399 shutdown_connections 2023-10-26T01:44:42.836 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:42.831+0000 7f47a5226640 1 -- 172.21.15.169:0/3329500399 wait complete. 2023-10-26T01:44:42.855 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-26T01:44:43.066 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:43.064+0000 7f4e00f8a640 1 -- 172.21.15.169:0/2748997014 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4dfc1577a0 msgr2=0x7f4dfc157b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:43.067 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:43.064+0000 7f4e00f8a640 1 --2- 172.21.15.169:0/2748997014 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4dfc1577a0 0x7f4dfc157b80 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc0b0550 tx=0x7f4dec02bca0 comp rx=0 tx=0).stop 2023-10-26T01:44:43.067 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:43.064+0000 7f4e00f8a640 1 -- 172.21.15.169:0/2748997014 shutdown_connections 2023-10-26T01:44:43.067 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:43.064+0000 7f4e00f8a640 1 --2- 172.21.15.169:0/2748997014 >> v2:172.21.15.170:3300/0 conn(0x7f4dfc0aab40 0x7f4dfc0ac6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:43.067 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:43.064+0000 7f4e00f8a640 1 --2- 172.21.15.169:0/2748997014 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4dfc1577a0 0x7f4dfc157b80 unknown :-1 s=CLOSED pgs=1961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:43.068 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:43.064+0000 7f4e00f8a640 1 -- 172.21.15.169:0/2748997014 >> 172.21.15.169:0/2748997014 conn(0x7f4dfc0aeee0 msgr2=0x7f4dfc0af2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:43.068 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:43.064+0000 7f4e00f8a640 1 -- 172.21.15.169:0/2748997014 shutdown_connections 2023-10-26T01:44:43.068 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:43.065+0000 7f4e00f8a640 1 -- 172.21.15.169:0/2748997014 wait complete. 2023-10-26T01:44:43.069 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:43.066+0000 7f4e00f8a640 1 Processor -- start 2023-10-26T01:44:43.069 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:43.066+0000 7f4e00f8a640 1 -- start start 2023-10-26T01:44:43.070 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:43.067+0000 7f4e00f8a640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4dfc06cdb0 0x7f4dfc0713d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:43.070 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:43.067+0000 7f4e00f8a640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4dfc1577a0 0x7f4dfc150e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:43.070 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:43.067+0000 7f4e00f8a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4dfc151340 0x7f4dfc1f1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:43.070 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:43.067+0000 7f4e00f8a640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dfc071bb0 con 0x7f4dfc06cdb0 2023-10-26T01:44:43.070 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:43.067+0000 7f4e00f8a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dfc06f800 con 0x7f4dfc151340 2023-10-26T01:44:43.071 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:43.067+0000 7f4e00f8a640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dfc071910 con 0x7f4dfc1577a0 2023-10-26T01:44:43.071 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:43.067+0000 7f4dfa59c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4dfc06cdb0 0x7f4dfc0713d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:43.071 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:43.068+0000 7f4dfa59c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4dfc06cdb0 0x7f4dfc0713d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:36458/0 (socket says 172.21.15.169:36458) 2023-10-26T01:44:43.071 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:43.068+0000 7f4dfa59c640 1 -- 172.21.15.169:0/2501809272 learned_addr learned my addr 172.21.15.169:0/2501809272 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:44:43.073 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:43.068+0000 7f4dfad9d640 1 --2- 172.21.15.169:0/2501809272 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4dfc151340 0x7f4dfc1f1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:43.073 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:43.068+0000 7f4dfa59c640 1 -- 172.21.15.169:0/2501809272 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4dfc1577a0 msgr2=0x7f4dfc150e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:43.073 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:43.068+0000 7f4df3fff640 1 --2- 172.21.15.169:0/2501809272 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4dfc1577a0 0x7f4dfc150e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:43.073 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:43.068+0000 7f4dfa59c640 1 --2- 172.21.15.169:0/2501809272 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4dfc1577a0 0x7f4dfc150e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:43.073 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:43.068+0000 7f4dfa59c640 1 -- 172.21.15.169:0/2501809272 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4dfc151340 msgr2=0x7f4dfc1f1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:43.074 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:43.068+0000 7f4dfa59c640 1 --2- 172.21.15.169:0/2501809272 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4dfc151340 0x7f4dfc1f1730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:43.074 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:43.068+0000 7f4dfa59c640 1 -- 172.21.15.169:0/2501809272 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dec009560 con 0x7f4dfc06cdb0 2023-10-26T01:44:43.074 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:43.068+0000 7f4dfa59c640 1 --2- 172.21.15.169:0/2501809272 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4dfc06cdb0 0x7f4dfc0713d0 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7f4dec009850 tx=0x7f4dec02dc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:43.074 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:43.069+0000 7f4df2ffd640 1 -- 172.21.15.169:0/2501809272 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dec005650 con 0x7f4dfc06cdb0 2023-10-26T01:44:43.075 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:43.069+0000 7f4df2ffd640 1 -- 172.21.15.169:0/2501809272 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4dec0038f0 con 0x7f4dfc06cdb0 2023-10-26T01:44:43.075 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:43.069+0000 7f4df2ffd640 1 -- 172.21.15.169:0/2501809272 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dec003cc0 con 0x7f4dfc06cdb0 2023-10-26T01:44:43.075 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:43.069+0000 7f4e00f8a640 1 -- 172.21.15.169:0/2501809272 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4dfc06ee60 con 0x7f4dfc06cdb0 2023-10-26T01:44:43.075 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:43.069+0000 7f4e00f8a640 1 -- 172.21.15.169:0/2501809272 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dfc06f420 con 0x7f4dfc06cdb0 2023-10-26T01:44:43.075 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:43.070+0000 7f4e00f8a640 1 -- 172.21.15.169:0/2501809272 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4dfc066af0 con 0x7f4dfc06cdb0 2023-10-26T01:44:43.075 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:43.071+0000 7f4df2ffd640 1 -- 172.21.15.169:0/2501809272 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f4dec03e450 con 0x7f4dfc06cdb0 2023-10-26T01:44:43.076 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:43.072+0000 7f4df2ffd640 1 --2- 172.21.15.169:0/2501809272 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f4ddc06ae40 0x7f4ddc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:43.076 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:43.072+0000 7f4df2ffd640 1 -- 172.21.15.169:0/2501809272 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 5357+0+0 (secure 0 0 0) 0x7f4dec0abce0 con 0x7f4dfc06cdb0 2023-10-26T01:44:43.077 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:43.073+0000 7f4df2ffd640 1 -- 172.21.15.169:0/2501809272 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=189}) v3 -- 0x7f4ddc071c10 con 0x7f4dfc06cdb0 2023-10-26T01:44:43.077 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:43.073+0000 7f4df3fff640 1 --2- 172.21.15.169:0/2501809272 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f4ddc06ae40 0x7f4ddc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:43.077 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:43.074+0000 7f4df3fff640 1 --2- 172.21.15.169:0/2501809272 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f4ddc06ae40 0x7f4ddc06d300 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc15b8e0 tx=0x7f4de4008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:43.078 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:43.077+0000 7f4df2ffd640 1 -- 172.21.15.169:0/2501809272 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4dec042020 con 0x7f4dfc06cdb0 2023-10-26T01:44:43.273 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:43.271+0000 7f4e00f8a640 1 -- 172.21.15.169:0/2501809272 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f4dfc04da70 con 0x7f4dfc06cdb0 2023-10-26T01:44:44.216 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:44.214+0000 7f4df2ffd640 1 -- 172.21.15.169:0/2501809272 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v189) v1 ==== 169+0+0 (secure 0 0 0) 0x7f4dec076ee0 con 0x7f4dfc06cdb0 2023-10-26T01:44:44.216 INFO:teuthology.orchestra.run.smithi169.stderr:pool 'cephfs_data' removed 2023-10-26T01:44:44.220 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:44.218+0000 7f4df2ffd640 1 -- 172.21.15.169:0/2501809272 <== mon.0 v2:172.21.15.169:3300/0 8 ==== osd_map(189..189 src has 1..189) v4 ==== 296+0+0 (secure 0 0 0) 0x7f4dec0b2080 con 0x7f4dfc06cdb0 2023-10-26T01:44:44.220 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:44.218+0000 7f4df2ffd640 1 -- 172.21.15.169:0/2501809272 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=190}) v3 -- 0x7f4ddc0709a0 con 0x7f4dfc06cdb0 2023-10-26T01:44:44.232 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:44.231+0000 7f4e00f8a640 1 -- 172.21.15.169:0/2501809272 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f4ddc06ae40 msgr2=0x7f4ddc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:44.233 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:44.231+0000 7f4e00f8a640 1 --2- 172.21.15.169:0/2501809272 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f4ddc06ae40 0x7f4ddc06d300 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc15b8e0 tx=0x7f4de4008040 comp rx=0 tx=0).stop 2023-10-26T01:44:44.233 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:44.231+0000 7f4e00f8a640 1 -- 172.21.15.169:0/2501809272 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4dfc06cdb0 msgr2=0x7f4dfc0713d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:44.233 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:44.231+0000 7f4e00f8a640 1 --2- 172.21.15.169:0/2501809272 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4dfc06cdb0 0x7f4dfc0713d0 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7f4dec009850 tx=0x7f4dec02dc70 comp rx=0 tx=0).stop 2023-10-26T01:44:44.234 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:44.232+0000 7f4e00f8a640 1 -- 172.21.15.169:0/2501809272 shutdown_connections 2023-10-26T01:44:44.235 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:44.232+0000 7f4e00f8a640 1 --2- 172.21.15.169:0/2501809272 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f4ddc06ae40 0x7f4ddc06d300 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:44.235 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:44.232+0000 7f4e00f8a640 1 --2- 172.21.15.169:0/2501809272 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4dfc151340 0x7f4dfc1f1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:44.235 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:44.232+0000 7f4e00f8a640 1 --2- 172.21.15.169:0/2501809272 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4dfc1577a0 0x7f4dfc150e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:44.235 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:44.232+0000 7f4e00f8a640 1 --2- 172.21.15.169:0/2501809272 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4dfc06cdb0 0x7f4dfc0713d0 unknown :-1 s=CLOSED pgs=1962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:44.236 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:44.232+0000 7f4e00f8a640 1 -- 172.21.15.169:0/2501809272 >> 172.21.15.169:0/2501809272 conn(0x7f4dfc0aeee0 msgr2=0x7f4dfc1582f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:44.236 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:44.233+0000 7f4e00f8a640 1 -- 172.21.15.169:0/2501809272 shutdown_connections 2023-10-26T01:44:44.236 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:44.233+0000 7f4e00f8a640 1 -- 172.21.15.169:0/2501809272 wait complete. 2023-10-26T01:44:44.258 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 18 2023-10-26T01:44:44.459 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:44.457+0000 7fa5a5125640 1 -- 172.21.15.169:0/611297842 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa58c003970 msgr2=0x7fa58c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:44.459 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:44.457+0000 7fa5a5125640 1 --2- 172.21.15.169:0/611297842 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa58c003970 0x7fa58c003d90 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7fa590009b50 tx=0x7fa58c0057f0 comp rx=0 tx=0).stop 2023-10-26T01:44:44.460 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:44.457+0000 7fa59f59e640 1 -- 172.21.15.169:0/611297842 reap_dead start 2023-10-26T01:44:44.460 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:44.457+0000 7fa5a5125640 1 -- 172.21.15.169:0/611297842 shutdown_connections 2023-10-26T01:44:44.460 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:44.457+0000 7fa5a5125640 1 -- 172.21.15.169:0/611297842 >> 172.21.15.169:0/611297842 conn(0x7fa5a0069930 msgr2=0x7fa5a0069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:44.460 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:44.457+0000 7fa5a5125640 1 -- 172.21.15.169:0/611297842 shutdown_connections 2023-10-26T01:44:44.460 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:44.457+0000 7fa5a5125640 1 -- 172.21.15.169:0/611297842 wait complete. 2023-10-26T01:44:44.463 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:44.459+0000 7fa5a5125640 1 Processor -- start 2023-10-26T01:44:44.463 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:44.459+0000 7fa5a5125640 1 -- start start 2023-10-26T01:44:44.463 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:44.460+0000 7fa5a5125640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa58c003970 0x7fa5a014e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:44.464 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:44.460+0000 7fa5a5125640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa5a0071560 0x7fa5a0159c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:44.464 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:44.460+0000 7fa5a5125640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa5a015a1a0 0x7fa5a00703f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:44.464 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:44.460+0000 7fa5a5125640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a014e660 con 0x7fa5a0071560 2023-10-26T01:44:44.464 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:44.460+0000 7fa5a5125640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a0158830 con 0x7fa5a015a1a0 2023-10-26T01:44:44.465 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:44.460+0000 7fa5a5125640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a006ff60 con 0x7fa58c003970 2023-10-26T01:44:44.465 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:44.460+0000 7fa597fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa5a0071560 0x7fa5a0159c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:44.465 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:44.460+0000 7fa597fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa5a0071560 0x7fa5a0159c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:57034/0 (socket says 172.21.15.169:57034) 2023-10-26T01:44:44.465 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:44.460+0000 7fa597fff640 1 -- 172.21.15.169:0/972436947 learned_addr learned my addr 172.21.15.169:0/972436947 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:44:44.466 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:44.460+0000 7fa597fff640 1 -- 172.21.15.169:0/972436947 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa58c003970 msgr2=0x7fa5a014e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:44.466 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:44.460+0000 7fa597fff640 1 --2- 172.21.15.169:0/972436947 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa58c003970 0x7fa5a014e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:44.466 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:44.460+0000 7fa597fff640 1 -- 172.21.15.169:0/972436947 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa5a015a1a0 msgr2=0x7fa5a00703f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:44.466 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:44.460+0000 7fa597fff640 1 --2- 172.21.15.169:0/972436947 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa5a015a1a0 0x7fa5a00703f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:44.466 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:44.460+0000 7fa597fff640 1 -- 172.21.15.169:0/972436947 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5a00afaf0 con 0x7fa5a0071560 2023-10-26T01:44:44.467 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:44.461+0000 7fa597fff640 1 --2- 172.21.15.169:0/972436947 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa5a0071560 0x7fa5a0159c60 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7fa58800e9a0 tx=0x7fa58800ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:44.467 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:44.461+0000 7fa5977fe640 1 -- 172.21.15.169:0/972436947 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa58800ccc0 con 0x7fa5a0071560 2023-10-26T01:44:44.467 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:44.461+0000 7fa5977fe640 1 -- 172.21.15.169:0/972436947 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa58800ce20 con 0x7fa5a0071560 2023-10-26T01:44:44.467 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:44.461+0000 7fa5a5125640 1 -- 172.21.15.169:0/972436947 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5a006f630 con 0x7fa5a0071560 2023-10-26T01:44:44.468 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:44.461+0000 7fa5977fe640 1 -- 172.21.15.169:0/972436947 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa588005280 con 0x7fa5a0071560 2023-10-26T01:44:44.468 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:44.461+0000 7fa5a5125640 1 -- 172.21.15.169:0/972436947 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5a006fbf0 con 0x7fa5a0071560 2023-10-26T01:44:44.469 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:44.462+0000 7fa5a5125640 1 -- 172.21.15.169:0/972436947 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5a004db10 con 0x7fa5a0071560 2023-10-26T01:44:44.469 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:44.463+0000 7fa5977fe640 1 -- 172.21.15.169:0/972436947 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fa5880053e0 con 0x7fa5a0071560 2023-10-26T01:44:44.469 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:44.464+0000 7fa5977fe640 1 --2- 172.21.15.169:0/972436947 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa58006ae70 0x7fa58006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:44.470 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:44.464+0000 7fa5977fe640 1 -- 172.21.15.169:0/972436947 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fa588014070 con 0x7fa5a0071560 2023-10-26T01:44:44.470 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:44.465+0000 7fa59ed9d640 1 --2- 172.21.15.169:0/972436947 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa58006ae70 0x7fa58006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:44.470 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:44.466+0000 7fa59ed9d640 1 --2- 172.21.15.169:0/972436947 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa58006ae70 0x7fa58006d330 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fa590036000 tx=0x7fa590035040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:44.471 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:44.469+0000 7fa5977fe640 1 -- 172.21.15.169:0/972436947 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa588055260 con 0x7fa5a0071560 2023-10-26T01:44:44.659 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:44.657+0000 7fa5a5125640 1 -- 172.21.15.169:0/972436947 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "config reset", "num": 18} v 0) v1 -- 0x7fa5a00700a0 con 0x7fa5a0071560 2023-10-26T01:44:44.665 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:44.663+0000 7fa5977fe640 1 -- 172.21.15.169:0/972436947 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 18}]=0 v38) v1 ==== 71+0+0 (secure 0 0 0) 0x7fa588058880 con 0x7fa5a0071560 2023-10-26T01:44:44.668 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:44.666+0000 7fa5a5125640 1 -- 172.21.15.169:0/972436947 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa58006ae70 msgr2=0x7fa58006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:44.668 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:44.666+0000 7fa5a5125640 1 --2- 172.21.15.169:0/972436947 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa58006ae70 0x7fa58006d330 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fa590036000 tx=0x7fa590035040 comp rx=0 tx=0).stop 2023-10-26T01:44:44.669 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:44.666+0000 7fa5a5125640 1 -- 172.21.15.169:0/972436947 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa5a0071560 msgr2=0x7fa5a0159c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:44.669 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:44.666+0000 7fa5a5125640 1 --2- 172.21.15.169:0/972436947 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa5a0071560 0x7fa5a0159c60 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7fa58800e9a0 tx=0x7fa58800ee60 comp rx=0 tx=0).stop 2023-10-26T01:44:44.669 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:44.666+0000 7fa5a5125640 1 -- 172.21.15.169:0/972436947 shutdown_connections 2023-10-26T01:44:44.669 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:44.666+0000 7fa5a5125640 1 --2- 172.21.15.169:0/972436947 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa58006ae70 0x7fa58006d330 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:44.669 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:44.666+0000 7fa5a5125640 1 --2- 172.21.15.169:0/972436947 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa5a0071560 0x7fa5a0159c60 unknown :-1 s=CLOSED pgs=1965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:44.670 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:44.666+0000 7fa5a5125640 1 --2- 172.21.15.169:0/972436947 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa5a015a1a0 0x7fa5a00703f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:44.670 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:44.666+0000 7fa5a5125640 1 --2- 172.21.15.169:0/972436947 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa58c003970 0x7fa5a014e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:44.670 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:44.666+0000 7fa5a5125640 1 -- 172.21.15.169:0/972436947 >> 172.21.15.169:0/972436947 conn(0x7fa5a0069930 msgr2=0x7fa5a0153b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:44.671 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:44.667+0000 7fa5a5125640 1 -- 172.21.15.169:0/972436947 shutdown_connections 2023-10-26T01:44:44.671 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:44.667+0000 7fa5a5125640 1 -- 172.21.15.169:0/972436947 wait complete. 2023-10-26T01:44:44.687 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_same_file' 2023-10-26T01:44:44.903 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:44.901+0000 7f97d6ded640 1 -- 172.21.15.169:0/4226163890 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97d006cae0 msgr2=0x7f97d00c7ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:44.903 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:44.901+0000 7f97d6ded640 1 --2- 172.21.15.169:0/4226163890 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97d006cae0 0x7f97d00c7ea0 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7f97cc005fd0 tx=0x7f97cc02d020 comp rx=0 tx=0).stop 2023-10-26T01:44:44.903 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:44.901+0000 7f97d6ded640 1 -- 172.21.15.169:0/4226163890 shutdown_connections 2023-10-26T01:44:44.904 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:44.901+0000 7f97d6ded640 1 --2- 172.21.15.169:0/4226163890 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f97d00c9e00 0x7f97d00cb390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:44.904 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:44.901+0000 7f97d6ded640 1 --2- 172.21.15.169:0/4226163890 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97d006cae0 0x7f97d00c7ea0 unknown :-1 s=CLOSED pgs=1966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:44.904 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:44.901+0000 7f97d6ded640 1 -- 172.21.15.169:0/4226163890 >> 172.21.15.169:0/4226163890 conn(0x7f97d00aedd0 msgr2=0x7f97d00af1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:44.904 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:44.901+0000 7f97d6ded640 1 -- 172.21.15.169:0/4226163890 shutdown_connections 2023-10-26T01:44:44.904 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:44.901+0000 7f97d6ded640 1 -- 172.21.15.169:0/4226163890 wait complete. 2023-10-26T01:44:44.905 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:44.903+0000 7f97d6ded640 1 Processor -- start 2023-10-26T01:44:44.905 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:44.903+0000 7f97d6ded640 1 -- start start 2023-10-26T01:44:44.907 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:44.904+0000 7f97d6ded640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f97d006a990 0x7f97d0145c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:44.907 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:44.904+0000 7f97d6ded640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97d006cae0 0x7f97d00b1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:44.907 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:44.904+0000 7f97d6ded640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f97d00c9e00 0x7f97d00b20f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:44.907 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:44.904+0000 7f97d6ded640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97d01431f0 con 0x7f97d006cae0 2023-10-26T01:44:44.908 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:44.904+0000 7f97d6ded640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97d0098e10 con 0x7f97d00c9e00 2023-10-26T01:44:44.908 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:44.904+0000 7f97d6ded640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97d0146140 con 0x7f97d006a990 2023-10-26T01:44:44.908 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:44.905+0000 7f97d55ea640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97d006cae0 0x7f97d00b1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:44.908 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:44.905+0000 7f97d55ea640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97d006cae0 0x7f97d00b1bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:57044/0 (socket says 172.21.15.169:57044) 2023-10-26T01:44:44.908 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:44.905+0000 7f97d55ea640 1 -- 172.21.15.169:0/285958408 learned_addr learned my addr 172.21.15.169:0/285958408 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:44:44.909 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:44.905+0000 7f97d5deb640 1 --2- 172.21.15.169:0/285958408 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f97d006a990 0x7f97d0145c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:44.909 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:44.905+0000 7f97d65ec640 1 --2- 172.21.15.169:0/285958408 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f97d00c9e00 0x7f97d00b20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:44.909 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:44.905+0000 7f97d55ea640 1 -- 172.21.15.169:0/285958408 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f97d006a990 msgr2=0x7f97d0145c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:44.909 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:44.905+0000 7f97d55ea640 1 --2- 172.21.15.169:0/285958408 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f97d006a990 0x7f97d0145c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:44.909 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:44.905+0000 7f97d55ea640 1 -- 172.21.15.169:0/285958408 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f97d00c9e00 msgr2=0x7f97d00b20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:44.910 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:44.905+0000 7f97d55ea640 1 --2- 172.21.15.169:0/285958408 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f97d00c9e00 0x7f97d00b20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:44.910 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:44.905+0000 7f97d55ea640 1 -- 172.21.15.169:0/285958408 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97cc00a670 con 0x7f97d006cae0 2023-10-26T01:44:44.910 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:44.905+0000 7f97d5deb640 1 --2- 172.21.15.169:0/285958408 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f97d006a990 0x7f97d0145c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T01:44:44.911 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:44.906+0000 7f97d55ea640 1 --2- 172.21.15.169:0/285958408 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97d006cae0 0x7f97d00b1bb0 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7f97c000c430 tx=0x7f97c000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:44.912 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:44.906+0000 7f97c67fc640 1 -- 172.21.15.169:0/285958408 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97c0011790 con 0x7f97d006cae0 2023-10-26T01:44:44.912 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:44.906+0000 7f97c67fc640 1 -- 172.21.15.169:0/285958408 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f97c00119e0 con 0x7f97d006cae0 2023-10-26T01:44:44.912 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:44.906+0000 7f97c67fc640 1 -- 172.21.15.169:0/285958408 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97c0011e30 con 0x7f97d006cae0 2023-10-26T01:44:44.912 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:44.906+0000 7f97d6ded640 1 -- 172.21.15.169:0/285958408 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97d00b2770 con 0x7f97d006cae0 2023-10-26T01:44:44.913 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:44.906+0000 7f97d6ded640 1 -- 172.21.15.169:0/285958408 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97d0099060 con 0x7f97d006cae0 2023-10-26T01:44:44.913 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:44.907+0000 7f97d6ded640 1 -- 172.21.15.169:0/285958408 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97d009a9e0 con 0x7f97d006cae0 2023-10-26T01:44:44.913 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:44.908+0000 7f97c67fc640 1 -- 172.21.15.169:0/285958408 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f97c001c460 con 0x7f97d006cae0 2023-10-26T01:44:44.913 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:44.909+0000 7f97c67fc640 1 --2- 172.21.15.169:0/285958408 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f97b406ae40 0x7f97b406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:44.913 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:44.909+0000 7f97c67fc640 1 -- 172.21.15.169:0/285958408 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f97c0089520 con 0x7f97d006cae0 2023-10-26T01:44:44.914 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:44.910+0000 7f97d5deb640 1 --2- 172.21.15.169:0/285958408 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f97b406ae40 0x7f97b406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:44.914 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:44.911+0000 7f97d5deb640 1 --2- 172.21.15.169:0/285958408 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f97b406ae40 0x7f97b406d300 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f97cc036000 tx=0x7f97cc004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:44.917 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:44.915+0000 7f97c67fc640 1 -- 172.21.15.169:0/285958408 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f97c00548d0 con 0x7f97d006cae0 2023-10-26T01:44:45.175 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:45.174+0000 7f97d6ded640 1 -- 172.21.15.169:0/285958408 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_same_file"]} v 0) v1 -- 0x7f97d0073c10 con 0x7f97d006cae0 2023-10-26T01:44:45.833 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:45.831+0000 7f97c67fc640 1 -- 172.21.15.169:0/285958408 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_same_file"]}]=0 v672) v1 ==== 135+0+0 (secure 0 0 0) 0x7f97c0089020 con 0x7f97d006cae0 2023-10-26T01:44:45.837 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:45.835+0000 7f97d6ded640 1 -- 172.21.15.169:0/285958408 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f97b406ae40 msgr2=0x7f97b406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:45.837 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:45.835+0000 7f97d6ded640 1 --2- 172.21.15.169:0/285958408 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f97b406ae40 0x7f97b406d300 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f97cc036000 tx=0x7f97cc004730 comp rx=0 tx=0).stop 2023-10-26T01:44:45.837 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:45.835+0000 7f97d6ded640 1 -- 172.21.15.169:0/285958408 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97d006cae0 msgr2=0x7f97d00b1bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:45.838 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:45.835+0000 7f97d6ded640 1 --2- 172.21.15.169:0/285958408 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97d006cae0 0x7f97d00b1bb0 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7f97c000c430 tx=0x7f97c000c8f0 comp rx=0 tx=0).stop 2023-10-26T01:44:45.838 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:45.835+0000 7f97d6ded640 1 -- 172.21.15.169:0/285958408 shutdown_connections 2023-10-26T01:44:45.838 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:45.835+0000 7f97d6ded640 1 --2- 172.21.15.169:0/285958408 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f97b406ae40 0x7f97b406d300 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:45.838 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:45.835+0000 7f97d6ded640 1 --2- 172.21.15.169:0/285958408 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97d006cae0 0x7f97d00b1bb0 unknown :-1 s=CLOSED pgs=1967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:45.839 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:45.835+0000 7f97d6ded640 1 --2- 172.21.15.169:0/285958408 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f97d00c9e00 0x7f97d00b20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:45.839 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:45.835+0000 7f97d6ded640 1 --2- 172.21.15.169:0/285958408 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f97d006a990 0x7f97d0145c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:45.839 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:45.835+0000 7f97d6ded640 1 -- 172.21.15.169:0/285958408 >> 172.21.15.169:0/285958408 conn(0x7f97d00aedd0 msgr2=0x7f97d0158cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:45.839 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:45.835+0000 7f97d6ded640 1 -- 172.21.15.169:0/285958408 shutdown_connections 2023-10-26T01:44:45.839 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:45.835+0000 7f97d6ded640 1 -- 172.21.15.169:0/285958408 wait complete. 2023-10-26T01:44:45.857 INFO:tasks.cephfs_test_runner:test_full_same_file (tasks.cephfs.test_full.TestQuotaFull) ... ok 2023-10-26T01:44:45.857 INFO:tasks.cephfs_test_runner: 2023-10-26T01:44:45.858 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-10-26T01:44:45.858 INFO:tasks.cephfs_test_runner:Ran 10 tests in 800.620s 2023-10-26T01:44:45.858 INFO:tasks.cephfs_test_runner: 2023-10-26T01:44:45.859 INFO:tasks.cephfs_test_runner:OK 2023-10-26T01:44:45.859 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2023-10-26T01:44:45.870 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2023-10-26T01:44:45.889 INFO:tasks.kclient:Unmounting kernel clients... 2023-10-26T01:44:45.889 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:44:45.889 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:44:45.949 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054301,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-26T01:44:45.950 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-10-26T01:44:45.950 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:44:46.003 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054301,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-26T01:44:46.004 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-26T01:44:46.004 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:44:46.064 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054303,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-26T01:44:46.065 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-26T01:44:46.065 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T01:44:46.119 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054303,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-26T01:44:46.119 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:44:46.119 DEBUG:teuthology.orchestra.run.smithi169:> ip netns list 2023-10-26T01:44:46.180 INFO:teuthology.orchestra.run.smithi169.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-26T01:44:46.180 INFO:teuthology.orchestra.run.smithi169.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-26T01:44:46.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:44:46.182 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-26T01:44:46.281 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:44:46.282 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-26T01:44:46.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:44:46.373 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link delete ceph-brx 2023-10-26T01:44:46.527 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:44:46.528 DEBUG:teuthology.orchestra.run.smithi169:> ip netns list 2023-10-26T01:44:46.574 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:44:46.574 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link delete ceph-brx 2023-10-26T01:44:46.721 INFO:teuthology.orchestra.run.smithi169.stderr:Cannot find device "ceph-brx" 2023-10-26T01:44:46.724 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-26T01:44:46.725 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:44:46.725 DEBUG:teuthology.orchestra.run.smithi170:> ip netns list 2023-10-26T01:44:46.784 INFO:teuthology.orchestra.run.smithi170.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-26T01:44:46.785 INFO:teuthology.orchestra.run.smithi170.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-26T01:44:46.786 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:44:46.786 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-26T01:44:46.885 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:44:46.886 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-26T01:44:46.974 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:44:46.974 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip link delete ceph-brx 2023-10-26T01:44:47.102 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:44:47.103 DEBUG:teuthology.orchestra.run.smithi170:> ip netns list 2023-10-26T01:44:47.148 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T01:44:47.148 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip link delete ceph-brx 2023-10-26T01:44:47.290 INFO:teuthology.orchestra.run.smithi170.stderr:Cannot find device "ceph-brx" 2023-10-26T01:44:47.293 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-26T01:44:47.293 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-10-26T01:44:47.303 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-10-26T01:44:47.304 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-26T01:44:47.530 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:47.528+0000 7fdc7a84b640 1 -- 172.21.15.169:0/2900794720 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdc7406c420 msgr2=0x7fdc740b0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:47.530 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:47.528+0000 7fdc7a84b640 1 --2- 172.21.15.169:0/2900794720 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdc7406c420 0x7fdc740b0910 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7fdc7406cd20 tx=0x7fdc6c02c110 comp rx=0 tx=0).stop 2023-10-26T01:44:47.531 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:47.528+0000 7fdc7a84b640 1 -- 172.21.15.169:0/2900794720 shutdown_connections 2023-10-26T01:44:47.531 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:47.528+0000 7fdc7a84b640 1 --2- 172.21.15.169:0/2900794720 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdc7406c420 0x7fdc740b0910 unknown :-1 s=CLOSED pgs=1969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:47.531 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:47.528+0000 7fdc7a84b640 1 --2- 172.21.15.169:0/2900794720 >> v2:172.21.15.170:3300/0 conn(0x7fdc7414ece0 0x7fdc740aa1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:47.531 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:47.528+0000 7fdc7a84b640 1 -- 172.21.15.169:0/2900794720 >> 172.21.15.169:0/2900794720 conn(0x7fdc74069930 msgr2=0x7fdc74069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:47.532 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:47.528+0000 7fdc7a84b640 1 -- 172.21.15.169:0/2900794720 shutdown_connections 2023-10-26T01:44:47.532 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:47.528+0000 7fdc7a84b640 1 -- 172.21.15.169:0/2900794720 wait complete. 2023-10-26T01:44:47.533 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:47.530+0000 7fdc7a84b640 1 Processor -- start 2023-10-26T01:44:47.534 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:47.530+0000 7fdc7a84b640 1 -- start start 2023-10-26T01:44:47.534 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:47.531+0000 7fdc7a84b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdc7406bf70 0x7fdc7414de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:47.534 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:47.531+0000 7fdc7a84b640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdc7406c420 0x7fdc74153b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:47.534 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:47.531+0000 7fdc7a84b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdc7414ece0 0x7fdc741540c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:47.535 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:47.531+0000 7fdc7a84b640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc7414e370 con 0x7fdc7414ece0 2023-10-26T01:44:47.535 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:47.531+0000 7fdc7a84b640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc74149720 con 0x7fdc7406bf70 2023-10-26T01:44:47.535 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:47.531+0000 7fdc7a84b640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc74152ae0 con 0x7fdc7406c420 2023-10-26T01:44:47.535 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:47.531+0000 7fdc78de8640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdc7414ece0 0x7fdc741540c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:47.535 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:47.531+0000 7fdc78de8640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdc7414ece0 0x7fdc741540c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:57066/0 (socket says 172.21.15.169:57066) 2023-10-26T01:44:47.536 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:47.531+0000 7fdc78de8640 1 -- 172.21.15.169:0/963740602 learned_addr learned my addr 172.21.15.169:0/963740602 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:44:47.536 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:47.532+0000 7fdc78de8640 1 -- 172.21.15.169:0/963740602 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdc7406c420 msgr2=0x7fdc74153b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:47.536 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:47.532+0000 7fdc78de8640 1 --2- 172.21.15.169:0/963740602 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdc7406c420 0x7fdc74153b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:47.536 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:47.532+0000 7fdc78de8640 1 -- 172.21.15.169:0/963740602 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdc7406bf70 msgr2=0x7fdc7414de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:47.536 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:47.532+0000 7fdc78de8640 1 --2- 172.21.15.169:0/963740602 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdc7406bf70 0x7fdc7414de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:47.537 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:47.532+0000 7fdc78de8640 1 -- 172.21.15.169:0/963740602 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc6c009840 con 0x7fdc7414ece0 2023-10-26T01:44:47.537 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:47.532+0000 7fdc78de8640 1 --2- 172.21.15.169:0/963740602 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdc7414ece0 0x7fdc741540c0 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c036040 tx=0x7fdc6c0042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:47.538 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:47.532+0000 7fdc717fa640 1 -- 172.21.15.169:0/963740602 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc6c0046e0 con 0x7fdc7414ece0 2023-10-26T01:44:47.538 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:47.533+0000 7fdc717fa640 1 -- 172.21.15.169:0/963740602 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fdc6c004840 con 0x7fdc7414ece0 2023-10-26T01:44:47.538 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:47.533+0000 7fdc717fa640 1 -- 172.21.15.169:0/963740602 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc6c034800 con 0x7fdc7414ece0 2023-10-26T01:44:47.539 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:47.533+0000 7fdc7a84b640 1 -- 172.21.15.169:0/963740602 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc74152dc0 con 0x7fdc7414ece0 2023-10-26T01:44:47.539 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:47.533+0000 7fdc7a84b640 1 -- 172.21.15.169:0/963740602 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc74153380 con 0x7fdc7414ece0 2023-10-26T01:44:47.539 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:47.534+0000 7fdc7a84b640 1 -- 172.21.15.169:0/963740602 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc7404db10 con 0x7fdc7414ece0 2023-10-26T01:44:47.539 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:47.535+0000 7fdc717fa640 1 -- 172.21.15.169:0/963740602 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fdc6c034960 con 0x7fdc7414ece0 2023-10-26T01:44:47.539 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:47.536+0000 7fdc717fa640 1 --2- 172.21.15.169:0/963740602 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fdc5406ae40 0x7fdc5406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:47.540 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:47.536+0000 7fdc717fa640 1 -- 172.21.15.169:0/963740602 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fdc6c0ac6f0 con 0x7fdc7414ece0 2023-10-26T01:44:47.540 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:47.537+0000 7fdc73fff640 1 --2- 172.21.15.169:0/963740602 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fdc5406ae40 0x7fdc5406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:47.540 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:47.538+0000 7fdc73fff640 1 --2- 172.21.15.169:0/963740602 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fdc5406ae40 0x7fdc5406d300 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fdc64005fd0 tx=0x7fdc64005850 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:47.542 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:47.540+0000 7fdc717fa640 1 -- 172.21.15.169:0/963740602 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdc6c077970 con 0x7fdc7414ece0 2023-10-26T01:44:47.727 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:47.726+0000 7fdc7a84b640 1 -- 172.21.15.169:0/963740602 --> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdc740ab220 con 0x7fdc5406ae40 2023-10-26T01:44:47.734 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:47.732+0000 7fdc717fa640 1 -- 172.21.15.169:0/963740602 <== mgr.4112 v2:172.21.15.170:6832/1187156918 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+47124 (secure 0 0 0) 0x7fdc64012070 con 0x7fdc5406ae40 2023-10-26T01:44:47.734 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:44:47.735 INFO:teuthology.orchestra.run.smithi169.stderr:dumped all 2023-10-26T01:44:47.736 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:47.734+0000 7fdc7a84b640 1 -- 172.21.15.169:0/963740602 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fdc5406ae40 msgr2=0x7fdc5406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:47.736 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:47.734+0000 7fdc7a84b640 1 --2- 172.21.15.169:0/963740602 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fdc5406ae40 0x7fdc5406d300 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fdc64005fd0 tx=0x7fdc64005850 comp rx=0 tx=0).stop 2023-10-26T01:44:47.736 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:47.734+0000 7fdc7a84b640 1 -- 172.21.15.169:0/963740602 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdc7414ece0 msgr2=0x7fdc741540c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:47.737 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:47.734+0000 7fdc7a84b640 1 --2- 172.21.15.169:0/963740602 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdc7414ece0 0x7fdc741540c0 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c036040 tx=0x7fdc6c0042e0 comp rx=0 tx=0).stop 2023-10-26T01:44:47.737 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:47.735+0000 7fdc7a84b640 1 -- 172.21.15.169:0/963740602 shutdown_connections 2023-10-26T01:44:47.737 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:47.735+0000 7fdc7a84b640 1 --2- 172.21.15.169:0/963740602 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fdc5406ae40 0x7fdc5406d300 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:47.738 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:47.735+0000 7fdc7a84b640 1 --2- 172.21.15.169:0/963740602 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdc7414ece0 0x7fdc741540c0 unknown :-1 s=CLOSED pgs=1970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:47.738 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:47.735+0000 7fdc7a84b640 1 --2- 172.21.15.169:0/963740602 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdc7406c420 0x7fdc74153b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:47.738 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:47.735+0000 7fdc7a84b640 1 --2- 172.21.15.169:0/963740602 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdc7406bf70 0x7fdc7414de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:47.738 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:47.735+0000 7fdc7a84b640 1 -- 172.21.15.169:0/963740602 >> 172.21.15.169:0/963740602 conn(0x7fdc74069930 msgr2=0x7fdc74159250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:47.738 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:47.735+0000 7fdc7a84b640 1 -- 172.21.15.169:0/963740602 shutdown_connections 2023-10-26T01:44:47.739 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:47.735+0000 7fdc7a84b640 1 -- 172.21.15.169:0/963740602 wait complete. 2023-10-26T01:44:47.754 INFO:teuthology.orchestra.run.smithi169.stdout:{"pg_ready":true,"pg_map":{"version":610,"stamp":"2023-10-26T01:44:46.440527+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":106,"num_osds":8,"num_per_pool_osds":5,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":17677,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1544822,"statfs":{"total":1600000000,"available":1581898514,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-52490893,"num_objects":-36,"num_object_clones":0,"num_object_copies":-72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-36,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-124,"num_write_kb":-85083,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002210"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":333,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T01:44:42.841516+0000","last_change":"2023-10-26T01:30:34.883484+0000","last_active":"2023-10-26T01:44:42.841516+0000","last_peered":"2023-10-26T01:44:42.841516+0000","last_clean":"2023-10-26T01:44:42.841516+0000","last_became_active":"2023-10-26T01:30:34.883071+0000","last_became_peered":"2023-10-26T01:30:34.883071+0000","last_unstale":"2023-10-26T01:44:42.841516+0000","last_undegraded":"2023-10-26T01:44:42.841516+0000","last_fullsized":"2023-10-26T01:44:42.841516+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:32.890096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:32.890096+0000","last_clean_scrub_stamp":"2023-10-26T01:30:32.890096+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:26:31.879452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":14,"seq":60129542317,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":9223,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":186088,"statfs":{"total":200000000,"available":190554713,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:43:57 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.84599999999999997,"15min":0.80700000000000005},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.33000000000000002},"max":{"1min":1.391,"5min":1.571,"15min":2.2549999999999999},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.98799999999999999,"15min":0.91900000000000004},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.309},"max":{"1min":1.607,"5min":1.919,"15min":1.919},"last":0.68799999999999994}]},{"osd":1,"last update":"Thu Oct 26 01:43:57 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.90800000000000003,"15min":0.89800000000000002},"min":{"1min":0.38,"5min":0.38,"15min":0.32300000000000001},"max":{"1min":1.411,"5min":1.569,"15min":4.4340000000000002},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.99099999999999999,"5min":1.0640000000000001,"15min":1.01},"min":{"1min":0.46800000000000003,"5min":0.38600000000000001,"15min":0.374},"max":{"1min":1.3440000000000001,"5min":1.7070000000000001,"15min":2.3330000000000002},"last":0.92400000000000004}]},{"osd":2,"last update":"Thu Oct 26 01:43:57 2023","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":1.0629999999999999,"15min":1.0529999999999999},"min":{"1min":0.44,"5min":0.44,"15min":0.39900000000000002},"max":{"1min":1.3049999999999999,"5min":1.6819999999999999,"15min":4.2800000000000002},"last":0.752},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.159,"15min":1.1259999999999999},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.8009999999999999,"5min":1.8129999999999999,"15min":2.302},"last":0.873}]},{"osd":4,"last update":"Thu Oct 26 01:43:57 2023","interfaces":[{"interface":"back","average":{"1min":1.133,"5min":1.2090000000000001,"15min":1.1950000000000001},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.7150000000000001,"5min":1.7150000000000001,"15min":2.1070000000000002},"last":1.387},{"interface":"front","average":{"1min":1.3160000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.7290000000000001,"5min":1.7509999999999999,"15min":4.8250000000000002},"last":1.421}]},{"osd":5,"last update":"Thu Oct 26 01:43:57 2023","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.3169999999999999,"15min":1.306},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.72499999999999998},"max":{"1min":1.998,"5min":1.998,"15min":4.9080000000000004},"last":1.21},{"interface":"front","average":{"1min":1.325,"5min":1.351,"15min":1.3029999999999999},"min":{"1min":0.81299999999999994,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.778,"5min":1.9279999999999999,"15min":2.8980000000000001},"last":1.4510000000000001}]},{"osd":6,"last update":"Thu Oct 26 01:43:57 2023","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.448,"15min":1.371},"min":{"1min":0.751,"5min":0.751,"15min":0.70199999999999996},"max":{"1min":1.845,"5min":3.5800000000000001,"15min":3.5800000000000001},"last":1.246},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.3480000000000001,"15min":1.3},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.7769999999999999,"5min":1.806,"15min":2.181},"last":1.097}]},{"osd":7,"last update":"Thu Oct 26 01:43:57 2023","interfaces":[{"interface":"back","average":{"1min":1.3140000000000001,"5min":1.3859999999999999,"15min":1.3400000000000001},"min":{"1min":0.745,"5min":0.745,"15min":0.71999999999999997},"max":{"1min":1.845,"5min":2.024,"15min":2.0680000000000001},"last":1.298},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.419,"15min":1.401},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":2.0579999999999998,"5min":2.0579999999999998,"15min":4.8730000000000002},"last":1.3320000000000001}]}]},{"osd":7,"up_from":13,"seq":55834575021,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1619,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193692,"statfs":{"total":200000000,"available":198341621,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:44:03 2023","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":1.171,"15min":1.133},"min":{"1min":0.60599999999999998,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.5740000000000001,"5min":2.2269999999999999,"15min":2.2269999999999999},"last":1.165},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.2270000000000001,"15min":1.143},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.58699999999999997},"max":{"1min":1.6020000000000001,"5min":2.282,"15min":2.6779999999999999},"last":0.80200000000000005}]},{"osd":1,"last update":"Thu Oct 26 01:44:03 2023","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.1679999999999999,"15min":1.147},"min":{"1min":0.83599999999999997,"5min":0.65600000000000003,"15min":0.65200000000000002},"max":{"1min":1.5800000000000001,"5min":1.8,"15min":2.8260000000000001},"last":0.88200000000000001},{"interface":"front","average":{"1min":1.1759999999999999,"5min":1.23,"15min":1.2130000000000001},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.61199999999999999},"max":{"1min":1.5129999999999999,"5min":2.0059999999999998,"15min":3.2730000000000001},"last":1.6020000000000001}]},{"osd":2,"last update":"Thu Oct 26 01:44:03 2023","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.2150000000000001,"15min":1.1739999999999999},"min":{"1min":0.77900000000000003,"5min":0.68999999999999995,"15min":0.58499999999999996},"max":{"1min":1.794,"5min":1.794,"15min":2.218},"last":0.83599999999999997},{"interface":"front","average":{"1min":1.319,"5min":1.29,"15min":1.2709999999999999},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.73099999999999998},"max":{"1min":1.8160000000000001,"5min":2.234,"15min":3.0510000000000002},"last":1.0149999999999999}]},{"osd":3,"last update":"Thu Oct 26 01:44:03 2023","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.325,"15min":1.2729999999999999},"min":{"1min":0.91400000000000003,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.6819999999999999,"5min":2.1379999999999999,"15min":2.1379999999999999},"last":1.5389999999999999},{"interface":"front","average":{"1min":1.391,"5min":1.4419999999999999,"15min":1.3919999999999999},"min":{"1min":0.98899999999999999,"5min":0.92200000000000004,"15min":0.65700000000000003},"max":{"1min":1.768,"5min":2.3050000000000002,"15min":5.5039999999999996},"last":1.446}]},{"osd":4,"last update":"Thu Oct 26 01:44:03 2023","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.399,"15min":1.369},"min":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.76100000000000001},"max":{"1min":2.0259999999999998,"5min":2.54,"15min":2.77},"last":1.659},{"interface":"front","average":{"1min":1.3180000000000001,"5min":1.3979999999999999,"15min":1.399},"min":{"1min":0.89400000000000002,"5min":0.75700000000000001,"15min":0.46500000000000002},"max":{"1min":1.9099999999999999,"5min":2.8660000000000001,"15min":3.1419999999999999},"last":1.266}]},{"osd":5,"last update":"Thu Oct 26 01:44:03 2023","interfaces":[{"interface":"back","average":{"1min":1.4219999999999999,"5min":1.4239999999999999,"15min":1.4299999999999999},"min":{"1min":0.94899999999999995,"5min":0.86599999999999999,"15min":0.435},"max":{"1min":1.9379999999999999,"5min":2.3639999999999999,"15min":5.734},"last":1.7490000000000001},{"interface":"front","average":{"1min":1.508,"5min":1.528,"15min":1.5009999999999999},"min":{"1min":1.1519999999999999,"5min":1.0920000000000001,"15min":0.51900000000000002},"max":{"1min":1.9810000000000001,"5min":2.5169999999999999,"15min":2.8980000000000001},"last":1.7210000000000001}]},{"osd":6,"last update":"Thu Oct 26 01:44:03 2023","interfaces":[{"interface":"back","average":{"1min":1.5860000000000001,"5min":1.5700000000000001,"15min":1.661},"min":{"1min":1.292,"5min":1.038,"15min":0.47399999999999998},"max":{"1min":2.0459999999999998,"5min":2.5259999999999998,"15min":11.577999999999999},"last":1.6930000000000001},{"interface":"front","average":{"1min":1.631,"5min":1.6579999999999999,"15min":1.722},"min":{"1min":1.214,"5min":1.214,"15min":0.55100000000000005},"max":{"1min":2.0489999999999999,"5min":2.7250000000000001,"15min":10.704000000000001},"last":1.7729999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575021,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1042,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194270,"statfs":{"total":200000000,"available":198932603,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:43:58 2023","interfaces":[{"interface":"back","average":{"1min":1.1200000000000001,"5min":1.121,"15min":1.1279999999999999},"min":{"1min":0.747,"5min":0.69299999999999995,"15min":0.60399999999999998},"max":{"1min":1.536,"5min":2.2509999999999999,"15min":2.7370000000000001},"last":1.01},{"interface":"front","average":{"1min":1.101,"5min":1.1759999999999999,"15min":1.143},"min":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.59199999999999997},"max":{"1min":1.53,"5min":2.2170000000000001,"15min":4.4779999999999998},"last":1.3440000000000001}]},{"osd":1,"last update":"Thu Oct 26 01:43:58 2023","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":1.1359999999999999,"15min":1.179},"min":{"1min":0.85499999999999998,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.6679999999999999,"5min":1.889,"15min":5.5359999999999996},"last":1.2589999999999999},{"interface":"front","average":{"1min":1.208,"5min":1.2390000000000001,"15min":1.208},"min":{"1min":0.93400000000000005,"5min":0.76400000000000001,"15min":0.64000000000000001},"max":{"1min":1.726,"5min":1.726,"15min":4.3579999999999997},"last":0.95699999999999996}]},{"osd":2,"last update":"Thu Oct 26 01:43:58 2023","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.153,"15min":1.141},"min":{"1min":0.73599999999999999,"5min":0.69799999999999995,"15min":0.59799999999999998},"max":{"1min":1.329,"5min":1.651,"15min":3.7879999999999998},"last":1.5289999999999999},{"interface":"front","average":{"1min":1.272,"5min":1.2949999999999999,"15min":1.292},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.68899999999999995},"max":{"1min":1.7829999999999999,"5min":2.0110000000000001,"15min":3.0470000000000002},"last":1.474}]},{"osd":3,"last update":"Thu Oct 26 01:43:58 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.266,"15min":1.2569999999999999},"min":{"1min":0.85299999999999998,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.4690000000000001,"5min":1.8200000000000001,"15min":4.4560000000000004},"last":1.113},{"interface":"front","average":{"1min":1.3080000000000001,"5min":1.3580000000000001,"15min":1.341},"min":{"1min":1.038,"5min":0.93700000000000006,"15min":0.69499999999999995},"max":{"1min":1.8560000000000001,"5min":1.871,"15min":2.6909999999999998},"last":1.6020000000000001}]},{"osd":4,"last update":"Thu Oct 26 01:43:58 2023","interfaces":[{"interface":"back","average":{"1min":1.1899999999999999,"5min":1.272,"15min":1.2430000000000001},"min":{"1min":0.84699999999999998,"5min":0.59899999999999998,"15min":0.48399999999999999},"max":{"1min":1.5580000000000001,"5min":1.825,"15min":2.9489999999999998},"last":1.5649999999999999},{"interface":"front","average":{"1min":1.288,"5min":1.339,"15min":1.347},"min":{"1min":0.94199999999999995,"5min":0.83499999999999996,"15min":0.59299999999999997},"max":{"1min":1.573,"5min":2.169,"15min":6.609},"last":1.407}]},{"osd":5,"last update":"Thu Oct 26 01:43:58 2023","interfaces":[{"interface":"back","average":{"1min":1.4670000000000001,"5min":1.5369999999999999,"15min":1.5009999999999999},"min":{"1min":1.202,"5min":0.90400000000000003,"15min":0.63700000000000001},"max":{"1min":1.677,"5min":2.04,"15min":5.4370000000000003},"last":1.6699999999999999},{"interface":"front","average":{"1min":1.4370000000000001,"5min":1.4530000000000001,"15min":1.4399999999999999},"min":{"1min":1.052,"5min":0.69999999999999996,"15min":0.55200000000000005},"max":{"1min":1.921,"5min":2.1190000000000002,"15min":2.7759999999999998},"last":1.7090000000000001}]},{"osd":7,"last update":"Thu Oct 26 01:43:58 2023","interfaces":[{"interface":"back","average":{"1min":1.3320000000000001,"5min":1.419,"15min":1.427},"min":{"1min":0.92000000000000004,"5min":0.72399999999999998,"15min":0.54100000000000004},"max":{"1min":1.655,"5min":1.984,"15min":11.804},"last":1.643},{"interface":"front","average":{"1min":1.4650000000000001,"5min":1.52,"15min":1.55},"min":{"1min":1.095,"5min":0.96399999999999997,"15min":0.56499999999999995},"max":{"1min":1.7270000000000001,"5min":1.9510000000000001,"15min":11.541},"last":1.742}]}]},{"osd":5,"up_from":13,"seq":55834575021,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1042,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194269,"statfs":{"total":200000000,"available":198931989,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:43:58 2023","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.1950000000000001,"15min":1.179},"min":{"1min":0.628,"5min":0.53400000000000003,"15min":0.53300000000000003},"max":{"1min":1.5940000000000001,"5min":1.724,"15min":3.8210000000000002},"last":1.135},{"interface":"front","average":{"1min":1.284,"5min":1.298,"15min":1.252},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.67900000000000005},"max":{"1min":1.8140000000000001,"5min":1.8140000000000001,"15min":2.9889999999999999},"last":1.4299999999999999}]},{"osd":1,"last update":"Thu Oct 26 01:43:58 2023","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":1.2290000000000001,"15min":1.149},"min":{"1min":0.752,"5min":0.70999999999999996,"15min":0.65900000000000003},"max":{"1min":1.605,"5min":1.899,"15min":1.899},"last":1.0860000000000001},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.1200000000000001,"15min":1.1279999999999999},"min":{"1min":0.69499999999999995,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.514,"5min":1.708,"15min":2.4340000000000002},"last":0.86899999999999999}]},{"osd":2,"last update":"Thu Oct 26 01:43:58 2023","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.2170000000000001,"15min":1.1930000000000001},"min":{"1min":0.89700000000000002,"5min":0.76700000000000002,"15min":0.69299999999999995},"max":{"1min":1.7030000000000001,"5min":1.796,"15min":1.8220000000000001},"last":1.5860000000000001},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.248,"15min":1.214},"min":{"1min":0.85599999999999998,"5min":0.62,"15min":0.62},"max":{"1min":1.6000000000000001,"5min":1.875,"15min":1.875},"last":0.78200000000000003}]},{"osd":3,"last update":"Thu Oct 26 01:43:58 2023","interfaces":[{"interface":"back","average":{"1min":1.3759999999999999,"5min":1.304,"15min":1.28},"min":{"1min":0.94299999999999995,"5min":0.72399999999999998,"15min":0.72299999999999998},"max":{"1min":1.7569999999999999,"5min":1.7569999999999999,"15min":1.835},"last":1.2290000000000001},{"interface":"front","average":{"1min":1.4119999999999999,"5min":1.3520000000000001,"15min":1.3129999999999999},"min":{"1min":1.03,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.9410000000000001,"5min":2.6669999999999998,"15min":2.9249999999999998},"last":1.1879999999999999}]},{"osd":4,"last update":"Thu Oct 26 01:43:58 2023","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":1.2350000000000001,"15min":1.2370000000000001},"min":{"1min":0.94599999999999995,"5min":0.52000000000000002,"15min":0.49099999999999999},"max":{"1min":1.758,"5min":1.79,"15min":2.7770000000000001},"last":1.0309999999999999},{"interface":"front","average":{"1min":1.5620000000000001,"5min":1.3919999999999999,"15min":1.3839999999999999},"min":{"1min":0.98099999999999998,"5min":0.504,"15min":0.504},"max":{"1min":2.125,"5min":2.125,"15min":2.125},"last":1.5349999999999999}]},{"osd":6,"last update":"Thu Oct 26 01:43:58 2023","interfaces":[{"interface":"back","average":{"1min":1.51,"5min":1.4470000000000001,"15min":1.3919999999999999},"min":{"1min":1.1419999999999999,"5min":0.80000000000000004,"15min":0.51900000000000002},"max":{"1min":1.9590000000000001,"5min":2.4590000000000001,"15min":2.8250000000000002},"last":1.3879999999999999},{"interface":"front","average":{"1min":1.5189999999999999,"5min":1.4490000000000001,"15min":1.4019999999999999},"min":{"1min":1.208,"5min":0.65500000000000003,"15min":0.61799999999999999},"max":{"1min":1.8759999999999999,"5min":2.3530000000000002,"15min":2.4830000000000001},"last":1.3400000000000001}]},{"osd":7,"last update":"Thu Oct 26 01:43:58 2023","interfaces":[{"interface":"back","average":{"1min":1.663,"5min":1.5229999999999999,"15min":1.4950000000000001},"min":{"1min":1.119,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":2.0409999999999999,"5min":2.0409999999999999,"15min":3.621},"last":1.2989999999999999},{"interface":"front","average":{"1min":1.6890000000000001,"5min":1.5800000000000001,"15min":1.52},"min":{"1min":1.4670000000000001,"5min":0.85299999999999998,"15min":0.59499999999999997},"max":{"1min":2.0539999999999998,"5min":2.0539999999999998,"15min":2.8010000000000002},"last":1.486}]}]},{"osd":4,"up_from":13,"seq":55834575021,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1042,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194269,"statfs":{"total":200000000,"available":198931989,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:43:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.1850000000000001,"15min":1.079},"min":{"1min":0.57699999999999996,"5min":0.503,"15min":0.503},"max":{"1min":1.6499999999999999,"5min":1.78,"15min":2.0289999999999999},"last":1.2709999999999999},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.1639999999999999,"15min":1.087},"min":{"1min":0.85799999999999998,"5min":0.72599999999999998,"15min":0.60599999999999998},"max":{"1min":1.6639999999999999,"5min":1.921,"15min":2.6829999999999998},"last":1.206}]},{"osd":1,"last update":"Thu Oct 26 01:43:49 2023","interfaces":[{"interface":"back","average":{"1min":1.167,"5min":1.2270000000000001,"15min":1.1699999999999999},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.56799999999999995},"max":{"1min":1.804,"5min":2.056,"15min":4.0519999999999996},"last":1.085},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.274,"15min":1.1839999999999999},"min":{"1min":0.77800000000000002,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.599,"5min":2.4390000000000001,"15min":2.4390000000000001},"last":1.1459999999999999}]},{"osd":2,"last update":"Thu Oct 26 01:43:49 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.3839999999999999,"15min":1.3149999999999999},"min":{"1min":0.88100000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":2.048,"5min":2.2040000000000002,"15min":5.6749999999999998},"last":0.82399999999999995},{"interface":"front","average":{"1min":1.196,"5min":1.224,"15min":1.1870000000000001},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.7190000000000001,"5min":1.986,"15min":3.673},"last":0.98899999999999999}]},{"osd":3,"last update":"Thu Oct 26 01:43:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.357,"15min":1.2529999999999999},"min":{"1min":0.64700000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.708,"5min":2.5960000000000001,"15min":2.5960000000000001},"last":1.423},{"interface":"front","average":{"1min":1.196,"5min":1.2509999999999999,"15min":1.202},"min":{"1min":0.78600000000000003,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.589,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":1.3260000000000001}]},{"osd":5,"last update":"Thu Oct 26 01:43:49 2023","interfaces":[{"interface":"back","average":{"1min":1.228,"5min":1.2949999999999999,"15min":1.278},"min":{"1min":0.876,"5min":0.71099999999999997,"15min":0.46300000000000002},"max":{"1min":1.8640000000000001,"5min":2.774,"15min":2.774},"last":1.365},{"interface":"front","average":{"1min":1.272,"5min":1.355,"15min":1.3320000000000001},"min":{"1min":0.93000000000000005,"5min":0.77700000000000002,"15min":0.55200000000000005},"max":{"1min":1.6539999999999999,"5min":2.3109999999999999,"15min":5.8579999999999997},"last":1.5049999999999999}]},{"osd":6,"last update":"Thu Oct 26 01:43:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3180000000000001,"5min":1.371,"15min":1.3620000000000001},"min":{"1min":0.94899999999999995,"5min":0.69499999999999995,"15min":0.59299999999999997},"max":{"1min":1.76,"5min":2.3780000000000001,"15min":12.343999999999999},"last":1.4630000000000001},{"interface":"front","average":{"1min":1.5389999999999999,"5min":1.6000000000000001,"15min":1.5660000000000001},"min":{"1min":1.113,"5min":0.80000000000000004,"15min":0.55000000000000004},"max":{"1min":2.2799999999999998,"5min":2.9020000000000001,"15min":12.605},"last":1.5449999999999999}]},{"osd":7,"last update":"Thu Oct 26 01:43:49 2023","interfaces":[{"interface":"back","average":{"1min":1.466,"5min":1.5740000000000001,"15min":1.522},"min":{"1min":1.0649999999999999,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":2.044,"5min":2.7200000000000002,"15min":5.774},"last":1.3839999999999999},{"interface":"front","average":{"1min":1.468,"5min":1.542,"15min":1.4990000000000001},"min":{"1min":0.79500000000000004,"5min":0.79000000000000004,"15min":0.68000000000000005},"max":{"1min":1.8500000000000001,"5min":2.8149999999999999,"15min":4.6520000000000001},"last":1.571}]}]},{"osd":2,"up_from":13,"seq":55834575021,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1042,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194269,"statfs":{"total":200000000,"available":198931989,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:44:05 2023","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.81899999999999995,"15min":0.84099999999999997},"min":{"1min":0.53300000000000003,"5min":0.38500000000000001,"15min":0.308},"max":{"1min":1.363,"5min":1.363,"15min":6.3200000000000003},"last":1.1459999999999999},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.94399999999999995,"15min":0.95099999999999996},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38600000000000001},"max":{"1min":1.4119999999999999,"5min":1.4379999999999999,"15min":4.2619999999999996},"last":0.70199999999999996}]},{"osd":1,"last update":"Thu Oct 26 01:44:05 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.877,"15min":0.88600000000000001},"min":{"1min":0.56399999999999995,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.387,"5min":1.504,"15min":5.1260000000000003},"last":0.65000000000000002},{"interface":"front","average":{"1min":1.052,"5min":1.0349999999999999,"15min":1.0189999999999999},"min":{"1min":0.439,"5min":0.42699999999999999,"15min":0.36599999999999999},"max":{"1min":1.409,"5min":1.5469999999999999,"15min":6.1520000000000001},"last":1.022}]},{"osd":3,"last update":"Thu Oct 26 01:44:05 2023","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.1140000000000001,"15min":1.018},"min":{"1min":0.48699999999999999,"5min":0.46800000000000003,"15min":0.435},"max":{"1min":1.6259999999999999,"5min":1.909,"15min":2.0640000000000001},"last":0.81999999999999995},{"interface":"front","average":{"1min":1.002,"5min":1.0980000000000001,"15min":1.0760000000000001},"min":{"1min":0.621,"5min":0.503,"15min":0.45300000000000001},"max":{"1min":1.2849999999999999,"5min":1.6659999999999999,"15min":8.9019999999999992},"last":1.095}]},{"osd":4,"last update":"Thu Oct 26 01:44:05 2023","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.3340000000000001,"15min":1.3069999999999999},"min":{"1min":0.78700000000000003,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.5649999999999999,"5min":2.109,"15min":6.2670000000000003},"last":0.92600000000000005},{"interface":"front","average":{"1min":1.274,"5min":1.27,"15min":1.2849999999999999},"min":{"1min":0.79800000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.6419999999999999,"5min":1.923,"15min":7.343},"last":1.321}]},{"osd":5,"last update":"Thu Oct 26 01:44:05 2023","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.3400000000000001,"15min":1.3200000000000001},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.71899999999999997},"max":{"1min":1.796,"5min":2.3849999999999998,"15min":3.7170000000000001},"last":1.4330000000000001},{"interface":"front","average":{"1min":1.278,"5min":1.321,"15min":1.3089999999999999},"min":{"1min":0.95199999999999996,"5min":0.69499999999999995,"15min":0.65300000000000002},"max":{"1min":1.5649999999999999,"5min":1.8140000000000001,"15min":6.3899999999999997},"last":1.2549999999999999}]},{"osd":6,"last update":"Thu Oct 26 01:44:05 2023","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":1.3879999999999999,"15min":1.371},"min":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.70599999999999996},"max":{"1min":1.7010000000000001,"5min":2.097,"15min":6.0490000000000004},"last":1.2789999999999999},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.3899999999999999,"15min":1.3899999999999999},"min":{"1min":0.874,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.8080000000000001,"5min":2.5230000000000001,"15min":5.0650000000000004},"last":1.49}]},{"osd":7,"last update":"Thu Oct 26 01:44:05 2023","interfaces":[{"interface":"back","average":{"1min":1.367,"5min":1.415,"15min":1.403},"min":{"1min":1.073,"5min":0.83399999999999996,"15min":0.69399999999999995},"max":{"1min":1.6599999999999999,"5min":1.7949999999999999,"15min":6.4480000000000004},"last":1.464},{"interface":"front","average":{"1min":1.478,"5min":1.4830000000000001,"15min":1.4770000000000001},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":1.706,"5min":2.0169999999999999,"15min":2.8439999999999999},"last":1.4079999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575021,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1042,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194269,"statfs":{"total":200000000,"available":198931989,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:43:56 2023","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.86599999999999999,"15min":0.78600000000000003},"min":{"1min":0.46999999999999997,"5min":0.39300000000000002,"15min":0.29499999999999998},"max":{"1min":1.8009999999999999,"5min":1.8009999999999999,"15min":1.8009999999999999},"last":0.95099999999999996},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.96899999999999997,"15min":0.87},"min":{"1min":0.46999999999999997,"5min":0.40300000000000002,"15min":0.36199999999999999},"max":{"1min":1.4510000000000001,"5min":1.591,"15min":1.6399999999999999},"last":0.63700000000000001}]},{"osd":2,"last update":"Thu Oct 26 01:43:56 2023","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.115,"15min":1.032},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.39600000000000002},"max":{"1min":1.4930000000000001,"5min":1.655,"15min":2.4830000000000001},"last":1.097},{"interface":"front","average":{"1min":0.96299999999999997,"5min":1.036,"15min":0.94799999999999995},"min":{"1min":0.55000000000000004,"5min":0.51000000000000001,"15min":0.41199999999999998},"max":{"1min":1.3109999999999999,"5min":1.752,"15min":1.8080000000000001},"last":1.034}]},{"osd":3,"last update":"Thu Oct 26 01:44:00 2023","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":1.0720000000000001,"15min":1.0029999999999999},"min":{"1min":0.495,"5min":0.495,"15min":0.39500000000000002},"max":{"1min":1.575,"5min":1.6100000000000001,"15min":1.75},"last":0.89800000000000002},{"interface":"front","average":{"1min":1.1200000000000001,"5min":1.1220000000000001,"15min":1.0429999999999999},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.41299999999999998},"max":{"1min":1.458,"5min":1.556,"15min":1.802},"last":1.2529999999999999}]},{"osd":4,"last update":"Thu Oct 26 01:43:56 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.294,"15min":1.2370000000000001},"min":{"1min":0.95399999999999996,"5min":0.79800000000000004,"15min":0.56200000000000006},"max":{"1min":1.728,"5min":1.728,"15min":2.0659999999999998},"last":1.214},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.26,"15min":1.1910000000000001},"min":{"1min":0.90300000000000002,"5min":0.70999999999999996,"15min":0.65800000000000003},"max":{"1min":1.6850000000000001,"5min":1.8420000000000001,"15min":2.0270000000000001},"last":1.347}]},{"osd":5,"last update":"Thu Oct 26 01:43:56 2023","interfaces":[{"interface":"back","average":{"1min":1.302,"5min":1.3240000000000001,"15min":1.2649999999999999},"min":{"1min":0.70899999999999996,"5min":0.629,"15min":0.61799999999999999},"max":{"1min":1.776,"5min":1.776,"15min":1.8779999999999999},"last":1.286},{"interface":"front","average":{"1min":1.3919999999999999,"5min":1.401,"15min":1.3640000000000001},"min":{"1min":0.94599999999999995,"5min":0.72899999999999998,"15min":0.59999999999999998},"max":{"1min":1.859,"5min":1.859,"15min":2.2200000000000002},"last":1.3779999999999999}]},{"osd":6,"last update":"Thu Oct 26 01:43:56 2023","interfaces":[{"interface":"back","average":{"1min":1.391,"5min":1.3640000000000001,"15min":1.3200000000000001},"min":{"1min":1.1100000000000001,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.6890000000000001,"5min":1.8,"15min":2.3780000000000001},"last":1.4319999999999999},{"interface":"front","average":{"1min":1.3129999999999999,"5min":1.361,"15min":1.294},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.70799999999999996},"max":{"1min":1.6579999999999999,"5min":1.8819999999999999,"15min":1.984},"last":1.131}]},{"osd":7,"last update":"Thu Oct 26 01:43:56 2023","interfaces":[{"interface":"back","average":{"1min":1.3320000000000001,"5min":1.357,"15min":1.2909999999999999},"min":{"1min":0.872,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.802,"5min":1.835,"15min":2.6930000000000001},"last":1.1579999999999999},{"interface":"front","average":{"1min":1.472,"5min":1.4450000000000001,"15min":1.393},"min":{"1min":1.0509999999999999,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.863,"5min":1.903,"15min":2.0190000000000001},"last":1.3240000000000001}]}]},{"osd":0,"up_from":13,"seq":55834575021,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1619,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193692,"statfs":{"total":200000000,"available":198341621,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Oct 26 01:43:58 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.79000000000000004,"15min":0.879},"min":{"1min":0.46200000000000002,"5min":0.32500000000000001,"15min":0.32000000000000001},"max":{"1min":1.1879999999999999,"5min":1.494,"15min":19.425000000000001},"last":0.62},{"interface":"front","average":{"1min":1.0229999999999999,"5min":1.004,"15min":1.0229999999999999},"min":{"1min":0.54600000000000004,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.5289999999999999,"5min":1.5289999999999999,"15min":15.162000000000001},"last":0.57099999999999995}]},{"osd":2,"last update":"Thu Oct 26 01:43:58 2023","interfaces":[{"interface":"back","average":{"1min":1.0329999999999999,"5min":1.0489999999999999,"15min":1.1439999999999999},"min":{"1min":0.63700000000000001,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.3879999999999999,"5min":1.556,"15min":28.439},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.93100000000000005,"15min":0.91800000000000004},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.37},"max":{"1min":1.494,"5min":1.851,"15min":4.9459999999999997},"last":0.91800000000000004}]},{"osd":3,"last update":"Thu Oct 26 01:43:58 2023","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.1359999999999999,"15min":1.0920000000000001},"min":{"1min":0.58799999999999997,"5min":0.56999999999999995,"15min":0.44600000000000001},"max":{"1min":1.5009999999999999,"5min":1.7509999999999999,"15min":4.9800000000000004},"last":1.0860000000000001},{"interface":"front","average":{"1min":1.163,"5min":1.2290000000000001,"15min":1.1679999999999999},"min":{"1min":0.67300000000000004,"5min":0.63500000000000001,"15min":0.438},"max":{"1min":1.645,"5min":1.774,"15min":5.016},"last":0.81100000000000005}]},{"osd":4,"last update":"Thu Oct 26 01:43:58 2023","interfaces":[{"interface":"back","average":{"1min":1.169,"5min":1.159,"15min":1.246},"min":{"1min":0.84299999999999997,"5min":0.76800000000000002,"15min":0.52900000000000003},"max":{"1min":1.5389999999999999,"5min":1.5840000000000001,"15min":19.329000000000001},"last":0.876},{"interface":"front","average":{"1min":1.242,"5min":1.254,"15min":1.2689999999999999},"min":{"1min":1.01,"5min":0.78500000000000003,"15min":0.67600000000000005},"max":{"1min":1.5960000000000001,"5min":1.7270000000000001,"15min":5.5229999999999997},"last":0.84599999999999997}]},{"osd":5,"last update":"Thu Oct 26 01:43:58 2023","interfaces":[{"interface":"back","average":{"1min":1.4430000000000001,"5min":1.4319999999999999,"15min":1.4219999999999999},"min":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.64900000000000002},"max":{"1min":2.605,"5min":2.605,"15min":5.6360000000000001},"last":0.97199999999999998},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.286,"15min":1.367},"min":{"1min":0.92400000000000004,"5min":0.80800000000000005,"15min":0.57499999999999996},"max":{"1min":2.036,"5min":2.036,"15min":19.384},"last":0.999}]},{"osd":6,"last update":"Thu Oct 26 01:43:58 2023","interfaces":[{"interface":"back","average":{"1min":1.3120000000000001,"5min":1.319,"15min":1.343},"min":{"1min":0.875,"5min":0.81100000000000005,"15min":0.73799999999999999},"max":{"1min":1.611,"5min":1.8999999999999999,"15min":8.2609999999999992},"last":1.274},{"interface":"front","average":{"1min":1.3320000000000001,"5min":1.3660000000000001,"15min":1.3899999999999999},"min":{"1min":0.76100000000000001,"5min":0.746,"15min":0.746},"max":{"1min":1.7110000000000001,"5min":1.9399999999999999,"15min":11.849},"last":1.0649999999999999}]},{"osd":7,"last update":"Thu Oct 26 01:43:58 2023","interfaces":[{"interface":"back","average":{"1min":1.4650000000000001,"5min":1.488,"15min":1.47},"min":{"1min":1.016,"5min":0.999,"15min":0.73699999999999999},"max":{"1min":1.954,"5min":1.954,"15min":5.6689999999999996},"last":1.016},{"interface":"front","average":{"1min":1.419,"5min":1.4330000000000001,"15min":1.484},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.69399999999999995},"max":{"1min":1.71,"5min":1.8149999999999999,"15min":19.245999999999999},"last":1.228}]}]}],"pool_statfs":[]}} 2023-10-26T01:44:47.756 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-26T01:44:47.977 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:47.975+0000 7fa60a0c9640 1 -- 172.21.15.169:0/3616760672 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa6040acfe0 msgr2=0x7fa60415bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:47.978 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:47.975+0000 7fa60a0c9640 1 --2- 172.21.15.169:0/3616760672 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa6040acfe0 0x7fa60415bc50 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7fa5ec009830 tx=0x7fa5ec02bd60 comp rx=0 tx=0).stop 2023-10-26T01:44:47.978 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:47.975+0000 7fa60a0c9640 1 -- 172.21.15.169:0/3616760672 shutdown_connections 2023-10-26T01:44:47.978 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:47.975+0000 7fa60a0c9640 1 --2- 172.21.15.169:0/3616760672 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa6040acfe0 0x7fa60415bc50 unknown :-1 s=CLOSED pgs=1972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:47.978 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:47.975+0000 7fa60a0c9640 1 -- 172.21.15.169:0/3616760672 >> 172.21.15.169:0/3616760672 conn(0x7fa6040b07c0 msgr2=0x7fa60406a620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:47.978 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:47.976+0000 7fa60a0c9640 1 -- 172.21.15.169:0/3616760672 shutdown_connections 2023-10-26T01:44:47.979 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:47.976+0000 7fa60a0c9640 1 -- 172.21.15.169:0/3616760672 wait complete. 2023-10-26T01:44:47.979 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:47.977+0000 7fa60a0c9640 1 Processor -- start 2023-10-26T01:44:47.980 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:47.977+0000 7fa60a0c9640 1 -- start start 2023-10-26T01:44:47.981 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:47.978+0000 7fa60a0c9640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa60406ad30 0x7fa6041f5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:47.981 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:47.978+0000 7fa60a0c9640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa6040acfe0 0x7fa60412dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:47.982 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:47.978+0000 7fa60a0c9640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa60415ce10 0x7fa6041e5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:47.982 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:47.978+0000 7fa60a0c9640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6041340d0 con 0x7fa6040acfe0 2023-10-26T01:44:47.982 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:47.978+0000 7fa60a0c9640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6041e4fb0 con 0x7fa60415ce10 2023-10-26T01:44:47.982 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:47.978+0000 7fa60a0c9640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6041e43a0 con 0x7fa60406ad30 2023-10-26T01:44:47.982 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:47.979+0000 7fa5fbfff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa6040acfe0 0x7fa60412dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:47.983 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:47.979+0000 7fa5fbfff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa6040acfe0 0x7fa60412dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:57094/0 (socket says 172.21.15.169:57094) 2023-10-26T01:44:47.983 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:47.979+0000 7fa5fbfff640 1 -- 172.21.15.169:0/8845453 learned_addr learned my addr 172.21.15.169:0/8845453 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:44:47.983 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:47.979+0000 7fa5fbfff640 1 -- 172.21.15.169:0/8845453 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa60406ad30 msgr2=0x7fa6041f5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:47.983 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:47.979+0000 7fa5fbfff640 1 --2- 172.21.15.169:0/8845453 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa60406ad30 0x7fa6041f5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:47.983 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:47.979+0000 7fa5fbfff640 1 -- 172.21.15.169:0/8845453 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa60415ce10 msgr2=0x7fa6041e5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:47.984 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:47.979+0000 7fa5fbfff640 1 --2- 172.21.15.169:0/8845453 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa60415ce10 0x7fa6041e5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:47.984 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:47.979+0000 7fa5fbfff640 1 -- 172.21.15.169:0/8845453 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5ec009490 con 0x7fa6040acfe0 2023-10-26T01:44:47.984 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:47.979+0000 7fa5fbfff640 1 --2- 172.21.15.169:0/8845453 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa6040acfe0 0x7fa60412dd00 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7fa5ec009950 tx=0x7fa5ec0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:47.984 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:47.980+0000 7fa600ff9640 1 -- 172.21.15.169:0/8845453 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5ec003ea0 con 0x7fa6040acfe0 2023-10-26T01:44:47.985 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:47.980+0000 7fa600ff9640 1 -- 172.21.15.169:0/8845453 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa5ec005020 con 0x7fa6040acfe0 2023-10-26T01:44:47.985 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:47.980+0000 7fa60a0c9640 1 -- 172.21.15.169:0/8845453 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6041e4680 con 0x7fa6040acfe0 2023-10-26T01:44:47.985 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:47.980+0000 7fa600ff9640 1 -- 172.21.15.169:0/8845453 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5ec005410 con 0x7fa6040acfe0 2023-10-26T01:44:47.985 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:47.980+0000 7fa60a0c9640 1 -- 172.21.15.169:0/8845453 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6041e4c40 con 0x7fa6040acfe0 2023-10-26T01:44:47.986 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:47.981+0000 7fa60a0c9640 1 -- 172.21.15.169:0/8845453 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa60404db10 con 0x7fa6040acfe0 2023-10-26T01:44:47.986 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:47.982+0000 7fa600ff9640 1 -- 172.21.15.169:0/8845453 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fa5ec0047a0 con 0x7fa6040acfe0 2023-10-26T01:44:47.987 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:47.983+0000 7fa600ff9640 1 --2- 172.21.15.169:0/8845453 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa5e406ae40 0x7fa5e406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:47.987 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:47.983+0000 7fa600ff9640 1 -- 172.21.15.169:0/8845453 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fa5ec0ac0f0 con 0x7fa6040acfe0 2023-10-26T01:44:47.987 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:47.984+0000 7fa6037fe640 1 --2- 172.21.15.169:0/8845453 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa5e406ae40 0x7fa5e406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:47.987 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:47.985+0000 7fa6037fe640 1 --2- 172.21.15.169:0/8845453 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa5e406ae40 0x7fa5e406d300 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fa5f4003cc0 tx=0x7fa5f40079b0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:47.989 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:47.987+0000 7fa600ff9640 1 -- 172.21.15.169:0/8845453 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa5ec077400 con 0x7fa6040acfe0 2023-10-26T01:44:48.174 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:48.172+0000 7fa60a0c9640 1 -- 172.21.15.169:0/8845453 --> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa60415d280 con 0x7fa5e406ae40 2023-10-26T01:44:48.180 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:48.179+0000 7fa600ff9640 1 -- 172.21.15.169:0/8845453 <== mgr.4112 v2:172.21.15.170:6832/1187156918 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+47124 (secure 0 0 0) 0x7fa5f4014400 con 0x7fa5e406ae40 2023-10-26T01:44:48.181 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:44:48.181 INFO:teuthology.orchestra.run.smithi169.stderr:dumped all 2023-10-26T01:44:48.183 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:48.181+0000 7fa60a0c9640 1 -- 172.21.15.169:0/8845453 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa5e406ae40 msgr2=0x7fa5e406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:48.183 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:48.181+0000 7fa60a0c9640 1 --2- 172.21.15.169:0/8845453 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa5e406ae40 0x7fa5e406d300 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fa5f4003cc0 tx=0x7fa5f40079b0 comp rx=0 tx=0).stop 2023-10-26T01:44:48.183 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:48.181+0000 7fa60a0c9640 1 -- 172.21.15.169:0/8845453 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa6040acfe0 msgr2=0x7fa60412dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:48.184 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:48.181+0000 7fa60a0c9640 1 --2- 172.21.15.169:0/8845453 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa6040acfe0 0x7fa60412dd00 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7fa5ec009950 tx=0x7fa5ec0038a0 comp rx=0 tx=0).stop 2023-10-26T01:44:48.184 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:48.181+0000 7fa60a0c9640 1 -- 172.21.15.169:0/8845453 shutdown_connections 2023-10-26T01:44:48.184 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:48.181+0000 7fa60a0c9640 1 --2- 172.21.15.169:0/8845453 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fa5e406ae40 0x7fa5e406d300 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:48.184 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:48.181+0000 7fa60a0c9640 1 --2- 172.21.15.169:0/8845453 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa6040acfe0 0x7fa60412dd00 unknown :-1 s=CLOSED pgs=1973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:48.184 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:48.181+0000 7fa60a0c9640 1 --2- 172.21.15.169:0/8845453 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa60415ce10 0x7fa6041e5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:48.185 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:48.181+0000 7fa60a0c9640 1 --2- 172.21.15.169:0/8845453 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa60406ad30 0x7fa6041f5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:48.185 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:48.181+0000 7fa60a0c9640 1 -- 172.21.15.169:0/8845453 >> 172.21.15.169:0/8845453 conn(0x7fa6040b07c0 msgr2=0x7fa604142e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:48.185 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:48.181+0000 7fa60a0c9640 1 -- 172.21.15.169:0/8845453 shutdown_connections 2023-10-26T01:44:48.185 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:48.181+0000 7fa60a0c9640 1 -- 172.21.15.169:0/8845453 wait complete. 2023-10-26T01:44:48.203 INFO:teuthology.orchestra.run.smithi169.stdout:{"pg_ready":true,"pg_map":{"version":610,"stamp":"2023-10-26T01:44:46.440527+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":106,"num_osds":8,"num_per_pool_osds":5,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":17677,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1544822,"statfs":{"total":1600000000,"available":1581898514,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-52490893,"num_objects":-36,"num_object_clones":0,"num_object_copies":-72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-36,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-124,"num_write_kb":-85083,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002210"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":333,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T01:44:42.841516+0000","last_change":"2023-10-26T01:30:34.883484+0000","last_active":"2023-10-26T01:44:42.841516+0000","last_peered":"2023-10-26T01:44:42.841516+0000","last_clean":"2023-10-26T01:44:42.841516+0000","last_became_active":"2023-10-26T01:30:34.883071+0000","last_became_peered":"2023-10-26T01:30:34.883071+0000","last_unstale":"2023-10-26T01:44:42.841516+0000","last_undegraded":"2023-10-26T01:44:42.841516+0000","last_fullsized":"2023-10-26T01:44:42.841516+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:32.890096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:32.890096+0000","last_clean_scrub_stamp":"2023-10-26T01:30:32.890096+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:26:31.879452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":14,"seq":60129542317,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":9223,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":186088,"statfs":{"total":200000000,"available":190554713,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:43:57 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.84599999999999997,"15min":0.80700000000000005},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.33000000000000002},"max":{"1min":1.391,"5min":1.571,"15min":2.2549999999999999},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.98799999999999999,"15min":0.91900000000000004},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.309},"max":{"1min":1.607,"5min":1.919,"15min":1.919},"last":0.68799999999999994}]},{"osd":1,"last update":"Thu Oct 26 01:43:57 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.90800000000000003,"15min":0.89800000000000002},"min":{"1min":0.38,"5min":0.38,"15min":0.32300000000000001},"max":{"1min":1.411,"5min":1.569,"15min":4.4340000000000002},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.99099999999999999,"5min":1.0640000000000001,"15min":1.01},"min":{"1min":0.46800000000000003,"5min":0.38600000000000001,"15min":0.374},"max":{"1min":1.3440000000000001,"5min":1.7070000000000001,"15min":2.3330000000000002},"last":0.92400000000000004}]},{"osd":2,"last update":"Thu Oct 26 01:43:57 2023","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":1.0629999999999999,"15min":1.0529999999999999},"min":{"1min":0.44,"5min":0.44,"15min":0.39900000000000002},"max":{"1min":1.3049999999999999,"5min":1.6819999999999999,"15min":4.2800000000000002},"last":0.752},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.159,"15min":1.1259999999999999},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.8009999999999999,"5min":1.8129999999999999,"15min":2.302},"last":0.873}]},{"osd":4,"last update":"Thu Oct 26 01:43:57 2023","interfaces":[{"interface":"back","average":{"1min":1.133,"5min":1.2090000000000001,"15min":1.1950000000000001},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.7150000000000001,"5min":1.7150000000000001,"15min":2.1070000000000002},"last":1.387},{"interface":"front","average":{"1min":1.3160000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.7290000000000001,"5min":1.7509999999999999,"15min":4.8250000000000002},"last":1.421}]},{"osd":5,"last update":"Thu Oct 26 01:43:57 2023","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.3169999999999999,"15min":1.306},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.72499999999999998},"max":{"1min":1.998,"5min":1.998,"15min":4.9080000000000004},"last":1.21},{"interface":"front","average":{"1min":1.325,"5min":1.351,"15min":1.3029999999999999},"min":{"1min":0.81299999999999994,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.778,"5min":1.9279999999999999,"15min":2.8980000000000001},"last":1.4510000000000001}]},{"osd":6,"last update":"Thu Oct 26 01:43:57 2023","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.448,"15min":1.371},"min":{"1min":0.751,"5min":0.751,"15min":0.70199999999999996},"max":{"1min":1.845,"5min":3.5800000000000001,"15min":3.5800000000000001},"last":1.246},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.3480000000000001,"15min":1.3},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.7769999999999999,"5min":1.806,"15min":2.181},"last":1.097}]},{"osd":7,"last update":"Thu Oct 26 01:43:57 2023","interfaces":[{"interface":"back","average":{"1min":1.3140000000000001,"5min":1.3859999999999999,"15min":1.3400000000000001},"min":{"1min":0.745,"5min":0.745,"15min":0.71999999999999997},"max":{"1min":1.845,"5min":2.024,"15min":2.0680000000000001},"last":1.298},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.419,"15min":1.401},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":2.0579999999999998,"5min":2.0579999999999998,"15min":4.8730000000000002},"last":1.3320000000000001}]}]},{"osd":7,"up_from":13,"seq":55834575021,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1619,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193692,"statfs":{"total":200000000,"available":198341621,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:44:03 2023","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":1.171,"15min":1.133},"min":{"1min":0.60599999999999998,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.5740000000000001,"5min":2.2269999999999999,"15min":2.2269999999999999},"last":1.165},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.2270000000000001,"15min":1.143},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.58699999999999997},"max":{"1min":1.6020000000000001,"5min":2.282,"15min":2.6779999999999999},"last":0.80200000000000005}]},{"osd":1,"last update":"Thu Oct 26 01:44:03 2023","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.1679999999999999,"15min":1.147},"min":{"1min":0.83599999999999997,"5min":0.65600000000000003,"15min":0.65200000000000002},"max":{"1min":1.5800000000000001,"5min":1.8,"15min":2.8260000000000001},"last":0.88200000000000001},{"interface":"front","average":{"1min":1.1759999999999999,"5min":1.23,"15min":1.2130000000000001},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.61199999999999999},"max":{"1min":1.5129999999999999,"5min":2.0059999999999998,"15min":3.2730000000000001},"last":1.6020000000000001}]},{"osd":2,"last update":"Thu Oct 26 01:44:03 2023","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.2150000000000001,"15min":1.1739999999999999},"min":{"1min":0.77900000000000003,"5min":0.68999999999999995,"15min":0.58499999999999996},"max":{"1min":1.794,"5min":1.794,"15min":2.218},"last":0.83599999999999997},{"interface":"front","average":{"1min":1.319,"5min":1.29,"15min":1.2709999999999999},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.73099999999999998},"max":{"1min":1.8160000000000001,"5min":2.234,"15min":3.0510000000000002},"last":1.0149999999999999}]},{"osd":3,"last update":"Thu Oct 26 01:44:03 2023","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.325,"15min":1.2729999999999999},"min":{"1min":0.91400000000000003,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.6819999999999999,"5min":2.1379999999999999,"15min":2.1379999999999999},"last":1.5389999999999999},{"interface":"front","average":{"1min":1.391,"5min":1.4419999999999999,"15min":1.3919999999999999},"min":{"1min":0.98899999999999999,"5min":0.92200000000000004,"15min":0.65700000000000003},"max":{"1min":1.768,"5min":2.3050000000000002,"15min":5.5039999999999996},"last":1.446}]},{"osd":4,"last update":"Thu Oct 26 01:44:03 2023","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.399,"15min":1.369},"min":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.76100000000000001},"max":{"1min":2.0259999999999998,"5min":2.54,"15min":2.77},"last":1.659},{"interface":"front","average":{"1min":1.3180000000000001,"5min":1.3979999999999999,"15min":1.399},"min":{"1min":0.89400000000000002,"5min":0.75700000000000001,"15min":0.46500000000000002},"max":{"1min":1.9099999999999999,"5min":2.8660000000000001,"15min":3.1419999999999999},"last":1.266}]},{"osd":5,"last update":"Thu Oct 26 01:44:03 2023","interfaces":[{"interface":"back","average":{"1min":1.4219999999999999,"5min":1.4239999999999999,"15min":1.4299999999999999},"min":{"1min":0.94899999999999995,"5min":0.86599999999999999,"15min":0.435},"max":{"1min":1.9379999999999999,"5min":2.3639999999999999,"15min":5.734},"last":1.7490000000000001},{"interface":"front","average":{"1min":1.508,"5min":1.528,"15min":1.5009999999999999},"min":{"1min":1.1519999999999999,"5min":1.0920000000000001,"15min":0.51900000000000002},"max":{"1min":1.9810000000000001,"5min":2.5169999999999999,"15min":2.8980000000000001},"last":1.7210000000000001}]},{"osd":6,"last update":"Thu Oct 26 01:44:03 2023","interfaces":[{"interface":"back","average":{"1min":1.5860000000000001,"5min":1.5700000000000001,"15min":1.661},"min":{"1min":1.292,"5min":1.038,"15min":0.47399999999999998},"max":{"1min":2.0459999999999998,"5min":2.5259999999999998,"15min":11.577999999999999},"last":1.6930000000000001},{"interface":"front","average":{"1min":1.631,"5min":1.6579999999999999,"15min":1.722},"min":{"1min":1.214,"5min":1.214,"15min":0.55100000000000005},"max":{"1min":2.0489999999999999,"5min":2.7250000000000001,"15min":10.704000000000001},"last":1.7729999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575021,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1042,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194270,"statfs":{"total":200000000,"available":198932603,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:43:58 2023","interfaces":[{"interface":"back","average":{"1min":1.1200000000000001,"5min":1.121,"15min":1.1279999999999999},"min":{"1min":0.747,"5min":0.69299999999999995,"15min":0.60399999999999998},"max":{"1min":1.536,"5min":2.2509999999999999,"15min":2.7370000000000001},"last":1.01},{"interface":"front","average":{"1min":1.101,"5min":1.1759999999999999,"15min":1.143},"min":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.59199999999999997},"max":{"1min":1.53,"5min":2.2170000000000001,"15min":4.4779999999999998},"last":1.3440000000000001}]},{"osd":1,"last update":"Thu Oct 26 01:43:58 2023","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":1.1359999999999999,"15min":1.179},"min":{"1min":0.85499999999999998,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.6679999999999999,"5min":1.889,"15min":5.5359999999999996},"last":1.2589999999999999},{"interface":"front","average":{"1min":1.208,"5min":1.2390000000000001,"15min":1.208},"min":{"1min":0.93400000000000005,"5min":0.76400000000000001,"15min":0.64000000000000001},"max":{"1min":1.726,"5min":1.726,"15min":4.3579999999999997},"last":0.95699999999999996}]},{"osd":2,"last update":"Thu Oct 26 01:43:58 2023","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.153,"15min":1.141},"min":{"1min":0.73599999999999999,"5min":0.69799999999999995,"15min":0.59799999999999998},"max":{"1min":1.329,"5min":1.651,"15min":3.7879999999999998},"last":1.5289999999999999},{"interface":"front","average":{"1min":1.272,"5min":1.2949999999999999,"15min":1.292},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.68899999999999995},"max":{"1min":1.7829999999999999,"5min":2.0110000000000001,"15min":3.0470000000000002},"last":1.474}]},{"osd":3,"last update":"Thu Oct 26 01:43:58 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.266,"15min":1.2569999999999999},"min":{"1min":0.85299999999999998,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.4690000000000001,"5min":1.8200000000000001,"15min":4.4560000000000004},"last":1.113},{"interface":"front","average":{"1min":1.3080000000000001,"5min":1.3580000000000001,"15min":1.341},"min":{"1min":1.038,"5min":0.93700000000000006,"15min":0.69499999999999995},"max":{"1min":1.8560000000000001,"5min":1.871,"15min":2.6909999999999998},"last":1.6020000000000001}]},{"osd":4,"last update":"Thu Oct 26 01:43:58 2023","interfaces":[{"interface":"back","average":{"1min":1.1899999999999999,"5min":1.272,"15min":1.2430000000000001},"min":{"1min":0.84699999999999998,"5min":0.59899999999999998,"15min":0.48399999999999999},"max":{"1min":1.5580000000000001,"5min":1.825,"15min":2.9489999999999998},"last":1.5649999999999999},{"interface":"front","average":{"1min":1.288,"5min":1.339,"15min":1.347},"min":{"1min":0.94199999999999995,"5min":0.83499999999999996,"15min":0.59299999999999997},"max":{"1min":1.573,"5min":2.169,"15min":6.609},"last":1.407}]},{"osd":5,"last update":"Thu Oct 26 01:43:58 2023","interfaces":[{"interface":"back","average":{"1min":1.4670000000000001,"5min":1.5369999999999999,"15min":1.5009999999999999},"min":{"1min":1.202,"5min":0.90400000000000003,"15min":0.63700000000000001},"max":{"1min":1.677,"5min":2.04,"15min":5.4370000000000003},"last":1.6699999999999999},{"interface":"front","average":{"1min":1.4370000000000001,"5min":1.4530000000000001,"15min":1.4399999999999999},"min":{"1min":1.052,"5min":0.69999999999999996,"15min":0.55200000000000005},"max":{"1min":1.921,"5min":2.1190000000000002,"15min":2.7759999999999998},"last":1.7090000000000001}]},{"osd":7,"last update":"Thu Oct 26 01:43:58 2023","interfaces":[{"interface":"back","average":{"1min":1.3320000000000001,"5min":1.419,"15min":1.427},"min":{"1min":0.92000000000000004,"5min":0.72399999999999998,"15min":0.54100000000000004},"max":{"1min":1.655,"5min":1.984,"15min":11.804},"last":1.643},{"interface":"front","average":{"1min":1.4650000000000001,"5min":1.52,"15min":1.55},"min":{"1min":1.095,"5min":0.96399999999999997,"15min":0.56499999999999995},"max":{"1min":1.7270000000000001,"5min":1.9510000000000001,"15min":11.541},"last":1.742}]}]},{"osd":5,"up_from":13,"seq":55834575021,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1042,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194269,"statfs":{"total":200000000,"available":198931989,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:43:58 2023","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.1950000000000001,"15min":1.179},"min":{"1min":0.628,"5min":0.53400000000000003,"15min":0.53300000000000003},"max":{"1min":1.5940000000000001,"5min":1.724,"15min":3.8210000000000002},"last":1.135},{"interface":"front","average":{"1min":1.284,"5min":1.298,"15min":1.252},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.67900000000000005},"max":{"1min":1.8140000000000001,"5min":1.8140000000000001,"15min":2.9889999999999999},"last":1.4299999999999999}]},{"osd":1,"last update":"Thu Oct 26 01:43:58 2023","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":1.2290000000000001,"15min":1.149},"min":{"1min":0.752,"5min":0.70999999999999996,"15min":0.65900000000000003},"max":{"1min":1.605,"5min":1.899,"15min":1.899},"last":1.0860000000000001},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.1200000000000001,"15min":1.1279999999999999},"min":{"1min":0.69499999999999995,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.514,"5min":1.708,"15min":2.4340000000000002},"last":0.86899999999999999}]},{"osd":2,"last update":"Thu Oct 26 01:43:58 2023","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.2170000000000001,"15min":1.1930000000000001},"min":{"1min":0.89700000000000002,"5min":0.76700000000000002,"15min":0.69299999999999995},"max":{"1min":1.7030000000000001,"5min":1.796,"15min":1.8220000000000001},"last":1.5860000000000001},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.248,"15min":1.214},"min":{"1min":0.85599999999999998,"5min":0.62,"15min":0.62},"max":{"1min":1.6000000000000001,"5min":1.875,"15min":1.875},"last":0.78200000000000003}]},{"osd":3,"last update":"Thu Oct 26 01:43:58 2023","interfaces":[{"interface":"back","average":{"1min":1.3759999999999999,"5min":1.304,"15min":1.28},"min":{"1min":0.94299999999999995,"5min":0.72399999999999998,"15min":0.72299999999999998},"max":{"1min":1.7569999999999999,"5min":1.7569999999999999,"15min":1.835},"last":1.2290000000000001},{"interface":"front","average":{"1min":1.4119999999999999,"5min":1.3520000000000001,"15min":1.3129999999999999},"min":{"1min":1.03,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.9410000000000001,"5min":2.6669999999999998,"15min":2.9249999999999998},"last":1.1879999999999999}]},{"osd":4,"last update":"Thu Oct 26 01:43:58 2023","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":1.2350000000000001,"15min":1.2370000000000001},"min":{"1min":0.94599999999999995,"5min":0.52000000000000002,"15min":0.49099999999999999},"max":{"1min":1.758,"5min":1.79,"15min":2.7770000000000001},"last":1.0309999999999999},{"interface":"front","average":{"1min":1.5620000000000001,"5min":1.3919999999999999,"15min":1.3839999999999999},"min":{"1min":0.98099999999999998,"5min":0.504,"15min":0.504},"max":{"1min":2.125,"5min":2.125,"15min":2.125},"last":1.5349999999999999}]},{"osd":6,"last update":"Thu Oct 26 01:43:58 2023","interfaces":[{"interface":"back","average":{"1min":1.51,"5min":1.4470000000000001,"15min":1.3919999999999999},"min":{"1min":1.1419999999999999,"5min":0.80000000000000004,"15min":0.51900000000000002},"max":{"1min":1.9590000000000001,"5min":2.4590000000000001,"15min":2.8250000000000002},"last":1.3879999999999999},{"interface":"front","average":{"1min":1.5189999999999999,"5min":1.4490000000000001,"15min":1.4019999999999999},"min":{"1min":1.208,"5min":0.65500000000000003,"15min":0.61799999999999999},"max":{"1min":1.8759999999999999,"5min":2.3530000000000002,"15min":2.4830000000000001},"last":1.3400000000000001}]},{"osd":7,"last update":"Thu Oct 26 01:43:58 2023","interfaces":[{"interface":"back","average":{"1min":1.663,"5min":1.5229999999999999,"15min":1.4950000000000001},"min":{"1min":1.119,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":2.0409999999999999,"5min":2.0409999999999999,"15min":3.621},"last":1.2989999999999999},{"interface":"front","average":{"1min":1.6890000000000001,"5min":1.5800000000000001,"15min":1.52},"min":{"1min":1.4670000000000001,"5min":0.85299999999999998,"15min":0.59499999999999997},"max":{"1min":2.0539999999999998,"5min":2.0539999999999998,"15min":2.8010000000000002},"last":1.486}]}]},{"osd":4,"up_from":13,"seq":55834575021,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1042,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194269,"statfs":{"total":200000000,"available":198931989,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:43:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.1850000000000001,"15min":1.079},"min":{"1min":0.57699999999999996,"5min":0.503,"15min":0.503},"max":{"1min":1.6499999999999999,"5min":1.78,"15min":2.0289999999999999},"last":1.2709999999999999},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.1639999999999999,"15min":1.087},"min":{"1min":0.85799999999999998,"5min":0.72599999999999998,"15min":0.60599999999999998},"max":{"1min":1.6639999999999999,"5min":1.921,"15min":2.6829999999999998},"last":1.206}]},{"osd":1,"last update":"Thu Oct 26 01:43:49 2023","interfaces":[{"interface":"back","average":{"1min":1.167,"5min":1.2270000000000001,"15min":1.1699999999999999},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.56799999999999995},"max":{"1min":1.804,"5min":2.056,"15min":4.0519999999999996},"last":1.085},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.274,"15min":1.1839999999999999},"min":{"1min":0.77800000000000002,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.599,"5min":2.4390000000000001,"15min":2.4390000000000001},"last":1.1459999999999999}]},{"osd":2,"last update":"Thu Oct 26 01:43:49 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.3839999999999999,"15min":1.3149999999999999},"min":{"1min":0.88100000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":2.048,"5min":2.2040000000000002,"15min":5.6749999999999998},"last":0.82399999999999995},{"interface":"front","average":{"1min":1.196,"5min":1.224,"15min":1.1870000000000001},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.7190000000000001,"5min":1.986,"15min":3.673},"last":0.98899999999999999}]},{"osd":3,"last update":"Thu Oct 26 01:43:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.357,"15min":1.2529999999999999},"min":{"1min":0.64700000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.708,"5min":2.5960000000000001,"15min":2.5960000000000001},"last":1.423},{"interface":"front","average":{"1min":1.196,"5min":1.2509999999999999,"15min":1.202},"min":{"1min":0.78600000000000003,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.589,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":1.3260000000000001}]},{"osd":5,"last update":"Thu Oct 26 01:43:49 2023","interfaces":[{"interface":"back","average":{"1min":1.228,"5min":1.2949999999999999,"15min":1.278},"min":{"1min":0.876,"5min":0.71099999999999997,"15min":0.46300000000000002},"max":{"1min":1.8640000000000001,"5min":2.774,"15min":2.774},"last":1.365},{"interface":"front","average":{"1min":1.272,"5min":1.355,"15min":1.3320000000000001},"min":{"1min":0.93000000000000005,"5min":0.77700000000000002,"15min":0.55200000000000005},"max":{"1min":1.6539999999999999,"5min":2.3109999999999999,"15min":5.8579999999999997},"last":1.5049999999999999}]},{"osd":6,"last update":"Thu Oct 26 01:43:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3180000000000001,"5min":1.371,"15min":1.3620000000000001},"min":{"1min":0.94899999999999995,"5min":0.69499999999999995,"15min":0.59299999999999997},"max":{"1min":1.76,"5min":2.3780000000000001,"15min":12.343999999999999},"last":1.4630000000000001},{"interface":"front","average":{"1min":1.5389999999999999,"5min":1.6000000000000001,"15min":1.5660000000000001},"min":{"1min":1.113,"5min":0.80000000000000004,"15min":0.55000000000000004},"max":{"1min":2.2799999999999998,"5min":2.9020000000000001,"15min":12.605},"last":1.5449999999999999}]},{"osd":7,"last update":"Thu Oct 26 01:43:49 2023","interfaces":[{"interface":"back","average":{"1min":1.466,"5min":1.5740000000000001,"15min":1.522},"min":{"1min":1.0649999999999999,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":2.044,"5min":2.7200000000000002,"15min":5.774},"last":1.3839999999999999},{"interface":"front","average":{"1min":1.468,"5min":1.542,"15min":1.4990000000000001},"min":{"1min":0.79500000000000004,"5min":0.79000000000000004,"15min":0.68000000000000005},"max":{"1min":1.8500000000000001,"5min":2.8149999999999999,"15min":4.6520000000000001},"last":1.571}]}]},{"osd":2,"up_from":13,"seq":55834575021,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1042,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194269,"statfs":{"total":200000000,"available":198931989,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:44:05 2023","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.81899999999999995,"15min":0.84099999999999997},"min":{"1min":0.53300000000000003,"5min":0.38500000000000001,"15min":0.308},"max":{"1min":1.363,"5min":1.363,"15min":6.3200000000000003},"last":1.1459999999999999},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.94399999999999995,"15min":0.95099999999999996},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38600000000000001},"max":{"1min":1.4119999999999999,"5min":1.4379999999999999,"15min":4.2619999999999996},"last":0.70199999999999996}]},{"osd":1,"last update":"Thu Oct 26 01:44:05 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.877,"15min":0.88600000000000001},"min":{"1min":0.56399999999999995,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.387,"5min":1.504,"15min":5.1260000000000003},"last":0.65000000000000002},{"interface":"front","average":{"1min":1.052,"5min":1.0349999999999999,"15min":1.0189999999999999},"min":{"1min":0.439,"5min":0.42699999999999999,"15min":0.36599999999999999},"max":{"1min":1.409,"5min":1.5469999999999999,"15min":6.1520000000000001},"last":1.022}]},{"osd":3,"last update":"Thu Oct 26 01:44:05 2023","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.1140000000000001,"15min":1.018},"min":{"1min":0.48699999999999999,"5min":0.46800000000000003,"15min":0.435},"max":{"1min":1.6259999999999999,"5min":1.909,"15min":2.0640000000000001},"last":0.81999999999999995},{"interface":"front","average":{"1min":1.002,"5min":1.0980000000000001,"15min":1.0760000000000001},"min":{"1min":0.621,"5min":0.503,"15min":0.45300000000000001},"max":{"1min":1.2849999999999999,"5min":1.6659999999999999,"15min":8.9019999999999992},"last":1.095}]},{"osd":4,"last update":"Thu Oct 26 01:44:05 2023","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.3340000000000001,"15min":1.3069999999999999},"min":{"1min":0.78700000000000003,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.5649999999999999,"5min":2.109,"15min":6.2670000000000003},"last":0.92600000000000005},{"interface":"front","average":{"1min":1.274,"5min":1.27,"15min":1.2849999999999999},"min":{"1min":0.79800000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.6419999999999999,"5min":1.923,"15min":7.343},"last":1.321}]},{"osd":5,"last update":"Thu Oct 26 01:44:05 2023","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.3400000000000001,"15min":1.3200000000000001},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.71899999999999997},"max":{"1min":1.796,"5min":2.3849999999999998,"15min":3.7170000000000001},"last":1.4330000000000001},{"interface":"front","average":{"1min":1.278,"5min":1.321,"15min":1.3089999999999999},"min":{"1min":0.95199999999999996,"5min":0.69499999999999995,"15min":0.65300000000000002},"max":{"1min":1.5649999999999999,"5min":1.8140000000000001,"15min":6.3899999999999997},"last":1.2549999999999999}]},{"osd":6,"last update":"Thu Oct 26 01:44:05 2023","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":1.3879999999999999,"15min":1.371},"min":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.70599999999999996},"max":{"1min":1.7010000000000001,"5min":2.097,"15min":6.0490000000000004},"last":1.2789999999999999},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.3899999999999999,"15min":1.3899999999999999},"min":{"1min":0.874,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.8080000000000001,"5min":2.5230000000000001,"15min":5.0650000000000004},"last":1.49}]},{"osd":7,"last update":"Thu Oct 26 01:44:05 2023","interfaces":[{"interface":"back","average":{"1min":1.367,"5min":1.415,"15min":1.403},"min":{"1min":1.073,"5min":0.83399999999999996,"15min":0.69399999999999995},"max":{"1min":1.6599999999999999,"5min":1.7949999999999999,"15min":6.4480000000000004},"last":1.464},{"interface":"front","average":{"1min":1.478,"5min":1.4830000000000001,"15min":1.4770000000000001},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":1.706,"5min":2.0169999999999999,"15min":2.8439999999999999},"last":1.4079999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575021,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1042,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194269,"statfs":{"total":200000000,"available":198931989,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:43:56 2023","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.86599999999999999,"15min":0.78600000000000003},"min":{"1min":0.46999999999999997,"5min":0.39300000000000002,"15min":0.29499999999999998},"max":{"1min":1.8009999999999999,"5min":1.8009999999999999,"15min":1.8009999999999999},"last":0.95099999999999996},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.96899999999999997,"15min":0.87},"min":{"1min":0.46999999999999997,"5min":0.40300000000000002,"15min":0.36199999999999999},"max":{"1min":1.4510000000000001,"5min":1.591,"15min":1.6399999999999999},"last":0.63700000000000001}]},{"osd":2,"last update":"Thu Oct 26 01:43:56 2023","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.115,"15min":1.032},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.39600000000000002},"max":{"1min":1.4930000000000001,"5min":1.655,"15min":2.4830000000000001},"last":1.097},{"interface":"front","average":{"1min":0.96299999999999997,"5min":1.036,"15min":0.94799999999999995},"min":{"1min":0.55000000000000004,"5min":0.51000000000000001,"15min":0.41199999999999998},"max":{"1min":1.3109999999999999,"5min":1.752,"15min":1.8080000000000001},"last":1.034}]},{"osd":3,"last update":"Thu Oct 26 01:44:00 2023","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":1.0720000000000001,"15min":1.0029999999999999},"min":{"1min":0.495,"5min":0.495,"15min":0.39500000000000002},"max":{"1min":1.575,"5min":1.6100000000000001,"15min":1.75},"last":0.89800000000000002},{"interface":"front","average":{"1min":1.1200000000000001,"5min":1.1220000000000001,"15min":1.0429999999999999},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.41299999999999998},"max":{"1min":1.458,"5min":1.556,"15min":1.802},"last":1.2529999999999999}]},{"osd":4,"last update":"Thu Oct 26 01:43:56 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.294,"15min":1.2370000000000001},"min":{"1min":0.95399999999999996,"5min":0.79800000000000004,"15min":0.56200000000000006},"max":{"1min":1.728,"5min":1.728,"15min":2.0659999999999998},"last":1.214},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.26,"15min":1.1910000000000001},"min":{"1min":0.90300000000000002,"5min":0.70999999999999996,"15min":0.65800000000000003},"max":{"1min":1.6850000000000001,"5min":1.8420000000000001,"15min":2.0270000000000001},"last":1.347}]},{"osd":5,"last update":"Thu Oct 26 01:43:56 2023","interfaces":[{"interface":"back","average":{"1min":1.302,"5min":1.3240000000000001,"15min":1.2649999999999999},"min":{"1min":0.70899999999999996,"5min":0.629,"15min":0.61799999999999999},"max":{"1min":1.776,"5min":1.776,"15min":1.8779999999999999},"last":1.286},{"interface":"front","average":{"1min":1.3919999999999999,"5min":1.401,"15min":1.3640000000000001},"min":{"1min":0.94599999999999995,"5min":0.72899999999999998,"15min":0.59999999999999998},"max":{"1min":1.859,"5min":1.859,"15min":2.2200000000000002},"last":1.3779999999999999}]},{"osd":6,"last update":"Thu Oct 26 01:43:56 2023","interfaces":[{"interface":"back","average":{"1min":1.391,"5min":1.3640000000000001,"15min":1.3200000000000001},"min":{"1min":1.1100000000000001,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.6890000000000001,"5min":1.8,"15min":2.3780000000000001},"last":1.4319999999999999},{"interface":"front","average":{"1min":1.3129999999999999,"5min":1.361,"15min":1.294},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.70799999999999996},"max":{"1min":1.6579999999999999,"5min":1.8819999999999999,"15min":1.984},"last":1.131}]},{"osd":7,"last update":"Thu Oct 26 01:43:56 2023","interfaces":[{"interface":"back","average":{"1min":1.3320000000000001,"5min":1.357,"15min":1.2909999999999999},"min":{"1min":0.872,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.802,"5min":1.835,"15min":2.6930000000000001},"last":1.1579999999999999},{"interface":"front","average":{"1min":1.472,"5min":1.4450000000000001,"15min":1.393},"min":{"1min":1.0509999999999999,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.863,"5min":1.903,"15min":2.0190000000000001},"last":1.3240000000000001}]}]},{"osd":0,"up_from":13,"seq":55834575021,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1619,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193692,"statfs":{"total":200000000,"available":198341621,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Oct 26 01:43:58 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.79000000000000004,"15min":0.879},"min":{"1min":0.46200000000000002,"5min":0.32500000000000001,"15min":0.32000000000000001},"max":{"1min":1.1879999999999999,"5min":1.494,"15min":19.425000000000001},"last":0.62},{"interface":"front","average":{"1min":1.0229999999999999,"5min":1.004,"15min":1.0229999999999999},"min":{"1min":0.54600000000000004,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.5289999999999999,"5min":1.5289999999999999,"15min":15.162000000000001},"last":0.57099999999999995}]},{"osd":2,"last update":"Thu Oct 26 01:43:58 2023","interfaces":[{"interface":"back","average":{"1min":1.0329999999999999,"5min":1.0489999999999999,"15min":1.1439999999999999},"min":{"1min":0.63700000000000001,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.3879999999999999,"5min":1.556,"15min":28.439},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.93100000000000005,"15min":0.91800000000000004},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.37},"max":{"1min":1.494,"5min":1.851,"15min":4.9459999999999997},"last":0.91800000000000004}]},{"osd":3,"last update":"Thu Oct 26 01:43:58 2023","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.1359999999999999,"15min":1.0920000000000001},"min":{"1min":0.58799999999999997,"5min":0.56999999999999995,"15min":0.44600000000000001},"max":{"1min":1.5009999999999999,"5min":1.7509999999999999,"15min":4.9800000000000004},"last":1.0860000000000001},{"interface":"front","average":{"1min":1.163,"5min":1.2290000000000001,"15min":1.1679999999999999},"min":{"1min":0.67300000000000004,"5min":0.63500000000000001,"15min":0.438},"max":{"1min":1.645,"5min":1.774,"15min":5.016},"last":0.81100000000000005}]},{"osd":4,"last update":"Thu Oct 26 01:43:58 2023","interfaces":[{"interface":"back","average":{"1min":1.169,"5min":1.159,"15min":1.246},"min":{"1min":0.84299999999999997,"5min":0.76800000000000002,"15min":0.52900000000000003},"max":{"1min":1.5389999999999999,"5min":1.5840000000000001,"15min":19.329000000000001},"last":0.876},{"interface":"front","average":{"1min":1.242,"5min":1.254,"15min":1.2689999999999999},"min":{"1min":1.01,"5min":0.78500000000000003,"15min":0.67600000000000005},"max":{"1min":1.5960000000000001,"5min":1.7270000000000001,"15min":5.5229999999999997},"last":0.84599999999999997}]},{"osd":5,"last update":"Thu Oct 26 01:43:58 2023","interfaces":[{"interface":"back","average":{"1min":1.4430000000000001,"5min":1.4319999999999999,"15min":1.4219999999999999},"min":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.64900000000000002},"max":{"1min":2.605,"5min":2.605,"15min":5.6360000000000001},"last":0.97199999999999998},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.286,"15min":1.367},"min":{"1min":0.92400000000000004,"5min":0.80800000000000005,"15min":0.57499999999999996},"max":{"1min":2.036,"5min":2.036,"15min":19.384},"last":0.999}]},{"osd":6,"last update":"Thu Oct 26 01:43:58 2023","interfaces":[{"interface":"back","average":{"1min":1.3120000000000001,"5min":1.319,"15min":1.343},"min":{"1min":0.875,"5min":0.81100000000000005,"15min":0.73799999999999999},"max":{"1min":1.611,"5min":1.8999999999999999,"15min":8.2609999999999992},"last":1.274},{"interface":"front","average":{"1min":1.3320000000000001,"5min":1.3660000000000001,"15min":1.3899999999999999},"min":{"1min":0.76100000000000001,"5min":0.746,"15min":0.746},"max":{"1min":1.7110000000000001,"5min":1.9399999999999999,"15min":11.849},"last":1.0649999999999999}]},{"osd":7,"last update":"Thu Oct 26 01:43:58 2023","interfaces":[{"interface":"back","average":{"1min":1.4650000000000001,"5min":1.488,"15min":1.47},"min":{"1min":1.016,"5min":0.999,"15min":0.73699999999999999},"max":{"1min":1.954,"5min":1.954,"15min":5.6689999999999996},"last":1.016},{"interface":"front","average":{"1min":1.419,"5min":1.4330000000000001,"15min":1.484},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.69399999999999995},"max":{"1min":1.71,"5min":1.8149999999999999,"15min":19.245999999999999},"last":1.228}]}]}],"pool_statfs":[]}} 2023-10-26T01:44:48.205 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-10-26T01:44:48.206 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-26T01:44:48.422 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:48.420+0000 7f8a93fff640 1 -- 172.21.15.169:0/558587045 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a94152150 msgr2=0x7f8a940ac420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:48.423 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:48.420+0000 7f8a93fff640 1 --2- 172.21.15.169:0/558587045 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a94152150 0x7f8a940ac420 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7f8a88009860 tx=0x7f8a8802bd40 comp rx=0 tx=0).stop 2023-10-26T01:44:48.423 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:48.420+0000 7f8a93fff640 1 -- 172.21.15.169:0/558587045 shutdown_connections 2023-10-26T01:44:48.423 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:48.420+0000 7f8a93fff640 1 --2- 172.21.15.169:0/558587045 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f8a940cb240 0x7f8a940c91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:48.423 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:48.420+0000 7f8a93fff640 1 --2- 172.21.15.169:0/558587045 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a94152150 0x7f8a940ac420 unknown :-1 s=CLOSED pgs=1975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:48.423 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:48.420+0000 7f8a93fff640 1 -- 172.21.15.169:0/558587045 >> 172.21.15.169:0/558587045 conn(0x7f8a940aedd0 msgr2=0x7f8a940af1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:48.424 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:48.420+0000 7f8a93fff640 1 -- 172.21.15.169:0/558587045 shutdown_connections 2023-10-26T01:44:48.424 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:48.420+0000 7f8a93fff640 1 -- 172.21.15.169:0/558587045 wait complete. 2023-10-26T01:44:48.424 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:48.422+0000 7f8a93fff640 1 Processor -- start 2023-10-26T01:44:48.425 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:48.422+0000 7f8a93fff640 1 -- start start 2023-10-26T01:44:48.426 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:48.423+0000 7f8a93fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a940cb240 0x7f8a941ef720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:48.427 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:48.423+0000 7f8a93fff640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f8a94151d70 0x7f8a9406f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:48.427 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:48.423+0000 7f8a93fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a94152150 0x7f8a940716b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:48.427 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:48.423+0000 7f8a93fff640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a9406fe40 con 0x7f8a94152150 2023-10-26T01:44:48.427 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:48.423+0000 7f8a93fff640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a941efdb0 con 0x7f8a940cb240 2023-10-26T01:44:48.427 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:48.423+0000 7f8a93fff640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a941f0090 con 0x7f8a94151d70 2023-10-26T01:44:48.428 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:48.424+0000 7f8a937fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a94152150 0x7f8a940716b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:48.428 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:48.424+0000 7f8a937fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a94152150 0x7f8a940716b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:57118/0 (socket says 172.21.15.169:57118) 2023-10-26T01:44:48.428 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:48.424+0000 7f8a937fe640 1 -- 172.21.15.169:0/3291495475 learned_addr learned my addr 172.21.15.169:0/3291495475 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:44:48.428 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:48.424+0000 7f8a927fc640 1 --2- 172.21.15.169:0/3291495475 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f8a94151d70 0x7f8a9406f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:48.428 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:48.424+0000 7f8a92ffd640 1 --2- 172.21.15.169:0/3291495475 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a940cb240 0x7f8a941ef720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:48.429 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:48.424+0000 7f8a937fe640 1 -- 172.21.15.169:0/3291495475 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f8a94151d70 msgr2=0x7f8a9406f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:48.429 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:48.424+0000 7f8a937fe640 1 --2- 172.21.15.169:0/3291495475 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f8a94151d70 0x7f8a9406f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:48.429 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:48.424+0000 7f8a937fe640 1 -- 172.21.15.169:0/3291495475 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a940cb240 msgr2=0x7f8a941ef720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:48.429 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:48.424+0000 7f8a937fe640 1 --2- 172.21.15.169:0/3291495475 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a940cb240 0x7f8a941ef720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:48.429 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:48.424+0000 7f8a937fe640 1 -- 172.21.15.169:0/3291495475 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a88009490 con 0x7f8a94152150 2023-10-26T01:44:48.430 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:48.425+0000 7f8a937fe640 1 --2- 172.21.15.169:0/3291495475 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a94152150 0x7f8a940716b0 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c00cad0 tx=0x7f8a8c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:48.430 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:48.425+0000 7f8a7b7fe640 1 -- 172.21.15.169:0/3291495475 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a8c00ae20 con 0x7f8a94152150 2023-10-26T01:44:48.430 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:48.425+0000 7f8a7b7fe640 1 -- 172.21.15.169:0/3291495475 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8a8c004b20 con 0x7f8a94152150 2023-10-26T01:44:48.430 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:48.425+0000 7f8a7b7fe640 1 -- 172.21.15.169:0/3291495475 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a8c005020 con 0x7f8a94152150 2023-10-26T01:44:48.430 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:48.425+0000 7f8a93fff640 1 -- 172.21.15.169:0/3291495475 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a941f0370 con 0x7f8a94152150 2023-10-26T01:44:48.431 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:48.425+0000 7f8a93fff640 1 -- 172.21.15.169:0/3291495475 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a94071bf0 con 0x7f8a94152150 2023-10-26T01:44:48.432 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:48.426+0000 7f8a93fff640 1 -- 172.21.15.169:0/3291495475 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a941f0bc0 con 0x7f8a94152150 2023-10-26T01:44:48.432 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:48.427+0000 7f8a7b7fe640 1 -- 172.21.15.169:0/3291495475 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f8a8c005180 con 0x7f8a94152150 2023-10-26T01:44:48.432 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:48.428+0000 7f8a7b7fe640 1 --2- 172.21.15.169:0/3291495475 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f8a8006ae40 0x7f8a8006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:48.432 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:48.428+0000 7f8a7b7fe640 1 -- 172.21.15.169:0/3291495475 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f8a8c089c30 con 0x7f8a94152150 2023-10-26T01:44:48.432 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:48.429+0000 7f8a92ffd640 1 --2- 172.21.15.169:0/3291495475 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f8a8006ae40 0x7f8a8006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:48.433 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:48.430+0000 7f8a92ffd640 1 --2- 172.21.15.169:0/3291495475 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f8a8006ae40 0x7f8a8006d300 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f8a84007fd0 tx=0x7f8a84009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:48.434 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:48.433+0000 7f8a7b7fe640 1 -- 172.21.15.169:0/3291495475 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8a941f0bc0 con 0x7f8a94152150 2023-10-26T01:44:48.624 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:48.622+0000 7f8a93fff640 1 -- 172.21.15.169:0/3291495475 --> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8a941f0bc0 con 0x7f8a8006ae40 2023-10-26T01:44:48.630 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:48.628+0000 7f8a7b7fe640 1 -- 172.21.15.169:0/3291495475 <== mgr.4112 v2:172.21.15.170:6832/1187156918 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+47141 (secure 0 0 0) 0x7f8a941f0bc0 con 0x7f8a8006ae40 2023-10-26T01:44:48.631 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:44:48.631 INFO:teuthology.orchestra.run.smithi169.stderr:dumped all 2023-10-26T01:44:48.633 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:48.631+0000 7f8a93fff640 1 -- 172.21.15.169:0/3291495475 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f8a8006ae40 msgr2=0x7f8a8006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:48.633 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:48.631+0000 7f8a93fff640 1 --2- 172.21.15.169:0/3291495475 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f8a8006ae40 0x7f8a8006d300 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f8a84007fd0 tx=0x7f8a84009040 comp rx=0 tx=0).stop 2023-10-26T01:44:48.633 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:48.631+0000 7f8a93fff640 1 -- 172.21.15.169:0/3291495475 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a94152150 msgr2=0x7f8a940716b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:48.633 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:48.631+0000 7f8a93fff640 1 --2- 172.21.15.169:0/3291495475 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a94152150 0x7f8a940716b0 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c00cad0 tx=0x7f8a8c00cf90 comp rx=0 tx=0).stop 2023-10-26T01:44:48.634 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:48.631+0000 7f8a93fff640 1 -- 172.21.15.169:0/3291495475 shutdown_connections 2023-10-26T01:44:48.634 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:48.631+0000 7f8a93fff640 1 --2- 172.21.15.169:0/3291495475 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f8a8006ae40 0x7f8a8006d300 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:48.634 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:48.631+0000 7f8a93fff640 1 --2- 172.21.15.169:0/3291495475 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a94152150 0x7f8a940716b0 unknown :-1 s=CLOSED pgs=1976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:48.634 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:48.631+0000 7f8a93fff640 1 --2- 172.21.15.169:0/3291495475 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f8a94151d70 0x7f8a9406f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:48.635 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:48.631+0000 7f8a93fff640 1 --2- 172.21.15.169:0/3291495475 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a940cb240 0x7f8a941ef720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:48.635 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:48.631+0000 7f8a93fff640 1 -- 172.21.15.169:0/3291495475 >> 172.21.15.169:0/3291495475 conn(0x7f8a940aedd0 msgr2=0x7f8a940cb890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:48.635 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:48.631+0000 7f8a93fff640 1 -- 172.21.15.169:0/3291495475 shutdown_connections 2023-10-26T01:44:48.635 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:48.631+0000 7f8a93fff640 1 -- 172.21.15.169:0/3291495475 wait complete. 2023-10-26T01:44:48.652 INFO:teuthology.orchestra.run.smithi169.stdout:{"pg_ready":true,"pg_map":{"version":611,"stamp":"2023-10-26T01:44:48.441045+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":6,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":9496,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1553003,"statfs":{"total":1600000000,"available":1590275790,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-52490893,"num_objects":-36,"num_object_clones":0,"num_object_copies":-72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-36,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-124,"num_write_kb":-85083,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.622626"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":333,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T01:44:42.841516+0000","last_change":"2023-10-26T01:30:34.883484+0000","last_active":"2023-10-26T01:44:42.841516+0000","last_peered":"2023-10-26T01:44:42.841516+0000","last_clean":"2023-10-26T01:44:42.841516+0000","last_became_active":"2023-10-26T01:30:34.883071+0000","last_became_peered":"2023-10-26T01:30:34.883071+0000","last_unstale":"2023-10-26T01:44:42.841516+0000","last_undegraded":"2023-10-26T01:44:42.841516+0000","last_fullsized":"2023-10-26T01:44:42.841516+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:32.890096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:32.890096+0000","last_clean_scrub_stamp":"2023-10-26T01:30:32.890096+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:26:31.879452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":14,"seq":60129542318,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1042,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194269,"statfs":{"total":200000000,"available":198931989,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:43:57 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.84599999999999997,"15min":0.80700000000000005},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.33000000000000002},"max":{"1min":1.391,"5min":1.571,"15min":2.2549999999999999},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.98799999999999999,"15min":0.91900000000000004},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.309},"max":{"1min":1.607,"5min":1.919,"15min":1.919},"last":1.1850000000000001}]},{"osd":1,"last update":"Thu Oct 26 01:43:57 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.90800000000000003,"15min":0.89800000000000002},"min":{"1min":0.38,"5min":0.38,"15min":0.32300000000000001},"max":{"1min":1.411,"5min":1.569,"15min":4.4340000000000002},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.99099999999999999,"5min":1.0640000000000001,"15min":1.01},"min":{"1min":0.46800000000000003,"5min":0.38600000000000001,"15min":0.374},"max":{"1min":1.3440000000000001,"5min":1.7070000000000001,"15min":2.3330000000000002},"last":1}]},{"osd":2,"last update":"Thu Oct 26 01:43:57 2023","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":1.0629999999999999,"15min":1.0529999999999999},"min":{"1min":0.44,"5min":0.44,"15min":0.39900000000000002},"max":{"1min":1.3049999999999999,"5min":1.6819999999999999,"15min":4.2800000000000002},"last":1.083},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.159,"15min":1.1259999999999999},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.8009999999999999,"5min":1.8129999999999999,"15min":2.302},"last":1.28}]},{"osd":4,"last update":"Thu Oct 26 01:43:57 2023","interfaces":[{"interface":"back","average":{"1min":1.133,"5min":1.2090000000000001,"15min":1.1950000000000001},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.7150000000000001,"5min":1.7150000000000001,"15min":2.1070000000000002},"last":1.333},{"interface":"front","average":{"1min":1.3160000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.7290000000000001,"5min":1.7509999999999999,"15min":4.8250000000000002},"last":1.5089999999999999}]},{"osd":5,"last update":"Thu Oct 26 01:43:57 2023","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.3169999999999999,"15min":1.306},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.72499999999999998},"max":{"1min":1.998,"5min":1.998,"15min":4.9080000000000004},"last":1.373},{"interface":"front","average":{"1min":1.325,"5min":1.351,"15min":1.3029999999999999},"min":{"1min":0.81299999999999994,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.778,"5min":1.9279999999999999,"15min":2.8980000000000001},"last":1.4299999999999999}]},{"osd":6,"last update":"Thu Oct 26 01:43:57 2023","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.448,"15min":1.371},"min":{"1min":0.751,"5min":0.751,"15min":0.70199999999999996},"max":{"1min":1.845,"5min":3.5800000000000001,"15min":3.5800000000000001},"last":1.5429999999999999},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.3480000000000001,"15min":1.3},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.7769999999999999,"5min":1.806,"15min":2.181},"last":1.4790000000000001}]},{"osd":7,"last update":"Thu Oct 26 01:43:57 2023","interfaces":[{"interface":"back","average":{"1min":1.3140000000000001,"5min":1.3859999999999999,"15min":1.3400000000000001},"min":{"1min":0.745,"5min":0.745,"15min":0.71999999999999997},"max":{"1min":1.845,"5min":2.024,"15min":2.0680000000000001},"last":1.4039999999999999},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.419,"15min":1.401},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":2.0579999999999998,"5min":2.0579999999999998,"15min":4.8730000000000002},"last":1.583}]}]},{"osd":7,"up_from":13,"seq":55834575021,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1619,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193692,"statfs":{"total":200000000,"available":198341621,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:44:03 2023","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":1.171,"15min":1.133},"min":{"1min":0.60599999999999998,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.5740000000000001,"5min":2.2269999999999999,"15min":2.2269999999999999},"last":1.165},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.2270000000000001,"15min":1.143},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.58699999999999997},"max":{"1min":1.6020000000000001,"5min":2.282,"15min":2.6779999999999999},"last":0.80200000000000005}]},{"osd":1,"last update":"Thu Oct 26 01:44:03 2023","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.1679999999999999,"15min":1.147},"min":{"1min":0.83599999999999997,"5min":0.65600000000000003,"15min":0.65200000000000002},"max":{"1min":1.5800000000000001,"5min":1.8,"15min":2.8260000000000001},"last":0.88200000000000001},{"interface":"front","average":{"1min":1.1759999999999999,"5min":1.23,"15min":1.2130000000000001},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.61199999999999999},"max":{"1min":1.5129999999999999,"5min":2.0059999999999998,"15min":3.2730000000000001},"last":1.6020000000000001}]},{"osd":2,"last update":"Thu Oct 26 01:44:03 2023","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.2150000000000001,"15min":1.1739999999999999},"min":{"1min":0.77900000000000003,"5min":0.68999999999999995,"15min":0.58499999999999996},"max":{"1min":1.794,"5min":1.794,"15min":2.218},"last":0.83599999999999997},{"interface":"front","average":{"1min":1.319,"5min":1.29,"15min":1.2709999999999999},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.73099999999999998},"max":{"1min":1.8160000000000001,"5min":2.234,"15min":3.0510000000000002},"last":1.0149999999999999}]},{"osd":3,"last update":"Thu Oct 26 01:44:03 2023","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.325,"15min":1.2729999999999999},"min":{"1min":0.91400000000000003,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.6819999999999999,"5min":2.1379999999999999,"15min":2.1379999999999999},"last":1.5389999999999999},{"interface":"front","average":{"1min":1.391,"5min":1.4419999999999999,"15min":1.3919999999999999},"min":{"1min":0.98899999999999999,"5min":0.92200000000000004,"15min":0.65700000000000003},"max":{"1min":1.768,"5min":2.3050000000000002,"15min":5.5039999999999996},"last":1.446}]},{"osd":4,"last update":"Thu Oct 26 01:44:03 2023","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.399,"15min":1.369},"min":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.76100000000000001},"max":{"1min":2.0259999999999998,"5min":2.54,"15min":2.77},"last":1.659},{"interface":"front","average":{"1min":1.3180000000000001,"5min":1.3979999999999999,"15min":1.399},"min":{"1min":0.89400000000000002,"5min":0.75700000000000001,"15min":0.46500000000000002},"max":{"1min":1.9099999999999999,"5min":2.8660000000000001,"15min":3.1419999999999999},"last":1.266}]},{"osd":5,"last update":"Thu Oct 26 01:44:03 2023","interfaces":[{"interface":"back","average":{"1min":1.4219999999999999,"5min":1.4239999999999999,"15min":1.4299999999999999},"min":{"1min":0.94899999999999995,"5min":0.86599999999999999,"15min":0.435},"max":{"1min":1.9379999999999999,"5min":2.3639999999999999,"15min":5.734},"last":1.7490000000000001},{"interface":"front","average":{"1min":1.508,"5min":1.528,"15min":1.5009999999999999},"min":{"1min":1.1519999999999999,"5min":1.0920000000000001,"15min":0.51900000000000002},"max":{"1min":1.9810000000000001,"5min":2.5169999999999999,"15min":2.8980000000000001},"last":1.7210000000000001}]},{"osd":6,"last update":"Thu Oct 26 01:44:03 2023","interfaces":[{"interface":"back","average":{"1min":1.5860000000000001,"5min":1.5700000000000001,"15min":1.661},"min":{"1min":1.292,"5min":1.038,"15min":0.47399999999999998},"max":{"1min":2.0459999999999998,"5min":2.5259999999999998,"15min":11.577999999999999},"last":1.6930000000000001},{"interface":"front","average":{"1min":1.631,"5min":1.6579999999999999,"15min":1.722},"min":{"1min":1.214,"5min":1.214,"15min":0.55100000000000005},"max":{"1min":2.0489999999999999,"5min":2.7250000000000001,"15min":10.704000000000001},"last":1.7729999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575021,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1042,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194270,"statfs":{"total":200000000,"available":198932603,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:43:58 2023","interfaces":[{"interface":"back","average":{"1min":1.1200000000000001,"5min":1.121,"15min":1.1279999999999999},"min":{"1min":0.747,"5min":0.69299999999999995,"15min":0.60399999999999998},"max":{"1min":1.536,"5min":2.2509999999999999,"15min":2.7370000000000001},"last":1.01},{"interface":"front","average":{"1min":1.101,"5min":1.1759999999999999,"15min":1.143},"min":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.59199999999999997},"max":{"1min":1.53,"5min":2.2170000000000001,"15min":4.4779999999999998},"last":1.3440000000000001}]},{"osd":1,"last update":"Thu Oct 26 01:43:58 2023","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":1.1359999999999999,"15min":1.179},"min":{"1min":0.85499999999999998,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.6679999999999999,"5min":1.889,"15min":5.5359999999999996},"last":1.2589999999999999},{"interface":"front","average":{"1min":1.208,"5min":1.2390000000000001,"15min":1.208},"min":{"1min":0.93400000000000005,"5min":0.76400000000000001,"15min":0.64000000000000001},"max":{"1min":1.726,"5min":1.726,"15min":4.3579999999999997},"last":0.95699999999999996}]},{"osd":2,"last update":"Thu Oct 26 01:43:58 2023","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.153,"15min":1.141},"min":{"1min":0.73599999999999999,"5min":0.69799999999999995,"15min":0.59799999999999998},"max":{"1min":1.329,"5min":1.651,"15min":3.7879999999999998},"last":1.5289999999999999},{"interface":"front","average":{"1min":1.272,"5min":1.2949999999999999,"15min":1.292},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.68899999999999995},"max":{"1min":1.7829999999999999,"5min":2.0110000000000001,"15min":3.0470000000000002},"last":1.474}]},{"osd":3,"last update":"Thu Oct 26 01:43:58 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.266,"15min":1.2569999999999999},"min":{"1min":0.85299999999999998,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.4690000000000001,"5min":1.8200000000000001,"15min":4.4560000000000004},"last":1.113},{"interface":"front","average":{"1min":1.3080000000000001,"5min":1.3580000000000001,"15min":1.341},"min":{"1min":1.038,"5min":0.93700000000000006,"15min":0.69499999999999995},"max":{"1min":1.8560000000000001,"5min":1.871,"15min":2.6909999999999998},"last":1.6020000000000001}]},{"osd":4,"last update":"Thu Oct 26 01:43:58 2023","interfaces":[{"interface":"back","average":{"1min":1.1899999999999999,"5min":1.272,"15min":1.2430000000000001},"min":{"1min":0.84699999999999998,"5min":0.59899999999999998,"15min":0.48399999999999999},"max":{"1min":1.5580000000000001,"5min":1.825,"15min":2.9489999999999998},"last":1.5649999999999999},{"interface":"front","average":{"1min":1.288,"5min":1.339,"15min":1.347},"min":{"1min":0.94199999999999995,"5min":0.83499999999999996,"15min":0.59299999999999997},"max":{"1min":1.573,"5min":2.169,"15min":6.609},"last":1.407}]},{"osd":5,"last update":"Thu Oct 26 01:43:58 2023","interfaces":[{"interface":"back","average":{"1min":1.4670000000000001,"5min":1.5369999999999999,"15min":1.5009999999999999},"min":{"1min":1.202,"5min":0.90400000000000003,"15min":0.63700000000000001},"max":{"1min":1.677,"5min":2.04,"15min":5.4370000000000003},"last":1.6699999999999999},{"interface":"front","average":{"1min":1.4370000000000001,"5min":1.4530000000000001,"15min":1.4399999999999999},"min":{"1min":1.052,"5min":0.69999999999999996,"15min":0.55200000000000005},"max":{"1min":1.921,"5min":2.1190000000000002,"15min":2.7759999999999998},"last":1.7090000000000001}]},{"osd":7,"last update":"Thu Oct 26 01:43:58 2023","interfaces":[{"interface":"back","average":{"1min":1.3320000000000001,"5min":1.419,"15min":1.427},"min":{"1min":0.92000000000000004,"5min":0.72399999999999998,"15min":0.54100000000000004},"max":{"1min":1.655,"5min":1.984,"15min":11.804},"last":1.643},{"interface":"front","average":{"1min":1.4650000000000001,"5min":1.52,"15min":1.55},"min":{"1min":1.095,"5min":0.96399999999999997,"15min":0.56499999999999995},"max":{"1min":1.7270000000000001,"5min":1.9510000000000001,"15min":11.541},"last":1.742}]}]},{"osd":5,"up_from":13,"seq":55834575021,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1042,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194269,"statfs":{"total":200000000,"available":198931989,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:43:58 2023","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.1950000000000001,"15min":1.179},"min":{"1min":0.628,"5min":0.53400000000000003,"15min":0.53300000000000003},"max":{"1min":1.5940000000000001,"5min":1.724,"15min":3.8210000000000002},"last":1.135},{"interface":"front","average":{"1min":1.284,"5min":1.298,"15min":1.252},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.67900000000000005},"max":{"1min":1.8140000000000001,"5min":1.8140000000000001,"15min":2.9889999999999999},"last":1.4299999999999999}]},{"osd":1,"last update":"Thu Oct 26 01:43:58 2023","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":1.2290000000000001,"15min":1.149},"min":{"1min":0.752,"5min":0.70999999999999996,"15min":0.65900000000000003},"max":{"1min":1.605,"5min":1.899,"15min":1.899},"last":1.0860000000000001},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.1200000000000001,"15min":1.1279999999999999},"min":{"1min":0.69499999999999995,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.514,"5min":1.708,"15min":2.4340000000000002},"last":0.86899999999999999}]},{"osd":2,"last update":"Thu Oct 26 01:43:58 2023","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.2170000000000001,"15min":1.1930000000000001},"min":{"1min":0.89700000000000002,"5min":0.76700000000000002,"15min":0.69299999999999995},"max":{"1min":1.7030000000000001,"5min":1.796,"15min":1.8220000000000001},"last":1.5860000000000001},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.248,"15min":1.214},"min":{"1min":0.85599999999999998,"5min":0.62,"15min":0.62},"max":{"1min":1.6000000000000001,"5min":1.875,"15min":1.875},"last":0.78200000000000003}]},{"osd":3,"last update":"Thu Oct 26 01:43:58 2023","interfaces":[{"interface":"back","average":{"1min":1.3759999999999999,"5min":1.304,"15min":1.28},"min":{"1min":0.94299999999999995,"5min":0.72399999999999998,"15min":0.72299999999999998},"max":{"1min":1.7569999999999999,"5min":1.7569999999999999,"15min":1.835},"last":1.2290000000000001},{"interface":"front","average":{"1min":1.4119999999999999,"5min":1.3520000000000001,"15min":1.3129999999999999},"min":{"1min":1.03,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.9410000000000001,"5min":2.6669999999999998,"15min":2.9249999999999998},"last":1.1879999999999999}]},{"osd":4,"last update":"Thu Oct 26 01:43:58 2023","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":1.2350000000000001,"15min":1.2370000000000001},"min":{"1min":0.94599999999999995,"5min":0.52000000000000002,"15min":0.49099999999999999},"max":{"1min":1.758,"5min":1.79,"15min":2.7770000000000001},"last":1.0309999999999999},{"interface":"front","average":{"1min":1.5620000000000001,"5min":1.3919999999999999,"15min":1.3839999999999999},"min":{"1min":0.98099999999999998,"5min":0.504,"15min":0.504},"max":{"1min":2.125,"5min":2.125,"15min":2.125},"last":1.5349999999999999}]},{"osd":6,"last update":"Thu Oct 26 01:43:58 2023","interfaces":[{"interface":"back","average":{"1min":1.51,"5min":1.4470000000000001,"15min":1.3919999999999999},"min":{"1min":1.1419999999999999,"5min":0.80000000000000004,"15min":0.51900000000000002},"max":{"1min":1.9590000000000001,"5min":2.4590000000000001,"15min":2.8250000000000002},"last":1.3879999999999999},{"interface":"front","average":{"1min":1.5189999999999999,"5min":1.4490000000000001,"15min":1.4019999999999999},"min":{"1min":1.208,"5min":0.65500000000000003,"15min":0.61799999999999999},"max":{"1min":1.8759999999999999,"5min":2.3530000000000002,"15min":2.4830000000000001},"last":1.3400000000000001}]},{"osd":7,"last update":"Thu Oct 26 01:43:58 2023","interfaces":[{"interface":"back","average":{"1min":1.663,"5min":1.5229999999999999,"15min":1.4950000000000001},"min":{"1min":1.119,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":2.0409999999999999,"5min":2.0409999999999999,"15min":3.621},"last":1.2989999999999999},{"interface":"front","average":{"1min":1.6890000000000001,"5min":1.5800000000000001,"15min":1.52},"min":{"1min":1.4670000000000001,"5min":0.85299999999999998,"15min":0.59499999999999997},"max":{"1min":2.0539999999999998,"5min":2.0539999999999998,"15min":2.8010000000000002},"last":1.486}]}]},{"osd":4,"up_from":13,"seq":55834575021,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1042,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194269,"statfs":{"total":200000000,"available":198931989,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:43:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.1850000000000001,"15min":1.079},"min":{"1min":0.57699999999999996,"5min":0.503,"15min":0.503},"max":{"1min":1.6499999999999999,"5min":1.78,"15min":2.0289999999999999},"last":1.2709999999999999},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.1639999999999999,"15min":1.087},"min":{"1min":0.85799999999999998,"5min":0.72599999999999998,"15min":0.60599999999999998},"max":{"1min":1.6639999999999999,"5min":1.921,"15min":2.6829999999999998},"last":1.206}]},{"osd":1,"last update":"Thu Oct 26 01:43:49 2023","interfaces":[{"interface":"back","average":{"1min":1.167,"5min":1.2270000000000001,"15min":1.1699999999999999},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.56799999999999995},"max":{"1min":1.804,"5min":2.056,"15min":4.0519999999999996},"last":1.085},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.274,"15min":1.1839999999999999},"min":{"1min":0.77800000000000002,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.599,"5min":2.4390000000000001,"15min":2.4390000000000001},"last":1.1459999999999999}]},{"osd":2,"last update":"Thu Oct 26 01:43:49 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.3839999999999999,"15min":1.3149999999999999},"min":{"1min":0.88100000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":2.048,"5min":2.2040000000000002,"15min":5.6749999999999998},"last":0.82399999999999995},{"interface":"front","average":{"1min":1.196,"5min":1.224,"15min":1.1870000000000001},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.7190000000000001,"5min":1.986,"15min":3.673},"last":0.98899999999999999}]},{"osd":3,"last update":"Thu Oct 26 01:43:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.357,"15min":1.2529999999999999},"min":{"1min":0.64700000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.708,"5min":2.5960000000000001,"15min":2.5960000000000001},"last":1.423},{"interface":"front","average":{"1min":1.196,"5min":1.2509999999999999,"15min":1.202},"min":{"1min":0.78600000000000003,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.589,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":1.3260000000000001}]},{"osd":5,"last update":"Thu Oct 26 01:43:49 2023","interfaces":[{"interface":"back","average":{"1min":1.228,"5min":1.2949999999999999,"15min":1.278},"min":{"1min":0.876,"5min":0.71099999999999997,"15min":0.46300000000000002},"max":{"1min":1.8640000000000001,"5min":2.774,"15min":2.774},"last":1.365},{"interface":"front","average":{"1min":1.272,"5min":1.355,"15min":1.3320000000000001},"min":{"1min":0.93000000000000005,"5min":0.77700000000000002,"15min":0.55200000000000005},"max":{"1min":1.6539999999999999,"5min":2.3109999999999999,"15min":5.8579999999999997},"last":1.5049999999999999}]},{"osd":6,"last update":"Thu Oct 26 01:43:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3180000000000001,"5min":1.371,"15min":1.3620000000000001},"min":{"1min":0.94899999999999995,"5min":0.69499999999999995,"15min":0.59299999999999997},"max":{"1min":1.76,"5min":2.3780000000000001,"15min":12.343999999999999},"last":1.4630000000000001},{"interface":"front","average":{"1min":1.5389999999999999,"5min":1.6000000000000001,"15min":1.5660000000000001},"min":{"1min":1.113,"5min":0.80000000000000004,"15min":0.55000000000000004},"max":{"1min":2.2799999999999998,"5min":2.9020000000000001,"15min":12.605},"last":1.5449999999999999}]},{"osd":7,"last update":"Thu Oct 26 01:43:49 2023","interfaces":[{"interface":"back","average":{"1min":1.466,"5min":1.5740000000000001,"15min":1.522},"min":{"1min":1.0649999999999999,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":2.044,"5min":2.7200000000000002,"15min":5.774},"last":1.3839999999999999},{"interface":"front","average":{"1min":1.468,"5min":1.542,"15min":1.4990000000000001},"min":{"1min":0.79500000000000004,"5min":0.79000000000000004,"15min":0.68000000000000005},"max":{"1min":1.8500000000000001,"5min":2.8149999999999999,"15min":4.6520000000000001},"last":1.571}]}]},{"osd":2,"up_from":13,"seq":55834575021,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1042,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194269,"statfs":{"total":200000000,"available":198931989,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:44:05 2023","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.81899999999999995,"15min":0.84099999999999997},"min":{"1min":0.53300000000000003,"5min":0.38500000000000001,"15min":0.308},"max":{"1min":1.363,"5min":1.363,"15min":6.3200000000000003},"last":1.1459999999999999},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.94399999999999995,"15min":0.95099999999999996},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38600000000000001},"max":{"1min":1.4119999999999999,"5min":1.4379999999999999,"15min":4.2619999999999996},"last":0.70199999999999996}]},{"osd":1,"last update":"Thu Oct 26 01:44:05 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.877,"15min":0.88600000000000001},"min":{"1min":0.56399999999999995,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.387,"5min":1.504,"15min":5.1260000000000003},"last":0.65000000000000002},{"interface":"front","average":{"1min":1.052,"5min":1.0349999999999999,"15min":1.0189999999999999},"min":{"1min":0.439,"5min":0.42699999999999999,"15min":0.36599999999999999},"max":{"1min":1.409,"5min":1.5469999999999999,"15min":6.1520000000000001},"last":1.022}]},{"osd":3,"last update":"Thu Oct 26 01:44:05 2023","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.1140000000000001,"15min":1.018},"min":{"1min":0.48699999999999999,"5min":0.46800000000000003,"15min":0.435},"max":{"1min":1.6259999999999999,"5min":1.909,"15min":2.0640000000000001},"last":0.81999999999999995},{"interface":"front","average":{"1min":1.002,"5min":1.0980000000000001,"15min":1.0760000000000001},"min":{"1min":0.621,"5min":0.503,"15min":0.45300000000000001},"max":{"1min":1.2849999999999999,"5min":1.6659999999999999,"15min":8.9019999999999992},"last":1.095}]},{"osd":4,"last update":"Thu Oct 26 01:44:05 2023","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.3340000000000001,"15min":1.3069999999999999},"min":{"1min":0.78700000000000003,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.5649999999999999,"5min":2.109,"15min":6.2670000000000003},"last":0.92600000000000005},{"interface":"front","average":{"1min":1.274,"5min":1.27,"15min":1.2849999999999999},"min":{"1min":0.79800000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.6419999999999999,"5min":1.923,"15min":7.343},"last":1.321}]},{"osd":5,"last update":"Thu Oct 26 01:44:05 2023","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.3400000000000001,"15min":1.3200000000000001},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.71899999999999997},"max":{"1min":1.796,"5min":2.3849999999999998,"15min":3.7170000000000001},"last":1.4330000000000001},{"interface":"front","average":{"1min":1.278,"5min":1.321,"15min":1.3089999999999999},"min":{"1min":0.95199999999999996,"5min":0.69499999999999995,"15min":0.65300000000000002},"max":{"1min":1.5649999999999999,"5min":1.8140000000000001,"15min":6.3899999999999997},"last":1.2549999999999999}]},{"osd":6,"last update":"Thu Oct 26 01:44:05 2023","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":1.3879999999999999,"15min":1.371},"min":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.70599999999999996},"max":{"1min":1.7010000000000001,"5min":2.097,"15min":6.0490000000000004},"last":1.2789999999999999},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.3899999999999999,"15min":1.3899999999999999},"min":{"1min":0.874,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.8080000000000001,"5min":2.5230000000000001,"15min":5.0650000000000004},"last":1.49}]},{"osd":7,"last update":"Thu Oct 26 01:44:05 2023","interfaces":[{"interface":"back","average":{"1min":1.367,"5min":1.415,"15min":1.403},"min":{"1min":1.073,"5min":0.83399999999999996,"15min":0.69399999999999995},"max":{"1min":1.6599999999999999,"5min":1.7949999999999999,"15min":6.4480000000000004},"last":1.464},{"interface":"front","average":{"1min":1.478,"5min":1.4830000000000001,"15min":1.4770000000000001},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":1.706,"5min":2.0169999999999999,"15min":2.8439999999999999},"last":1.4079999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575021,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1042,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194269,"statfs":{"total":200000000,"available":198931989,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:43:56 2023","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.86599999999999999,"15min":0.78600000000000003},"min":{"1min":0.46999999999999997,"5min":0.39300000000000002,"15min":0.29499999999999998},"max":{"1min":1.8009999999999999,"5min":1.8009999999999999,"15min":1.8009999999999999},"last":0.95099999999999996},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.96899999999999997,"15min":0.87},"min":{"1min":0.46999999999999997,"5min":0.40300000000000002,"15min":0.36199999999999999},"max":{"1min":1.4510000000000001,"5min":1.591,"15min":1.6399999999999999},"last":0.63700000000000001}]},{"osd":2,"last update":"Thu Oct 26 01:43:56 2023","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.115,"15min":1.032},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.39600000000000002},"max":{"1min":1.4930000000000001,"5min":1.655,"15min":2.4830000000000001},"last":1.097},{"interface":"front","average":{"1min":0.96299999999999997,"5min":1.036,"15min":0.94799999999999995},"min":{"1min":0.55000000000000004,"5min":0.51000000000000001,"15min":0.41199999999999998},"max":{"1min":1.3109999999999999,"5min":1.752,"15min":1.8080000000000001},"last":1.034}]},{"osd":3,"last update":"Thu Oct 26 01:44:00 2023","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":1.0720000000000001,"15min":1.0029999999999999},"min":{"1min":0.495,"5min":0.495,"15min":0.39500000000000002},"max":{"1min":1.575,"5min":1.6100000000000001,"15min":1.75},"last":0.89800000000000002},{"interface":"front","average":{"1min":1.1200000000000001,"5min":1.1220000000000001,"15min":1.0429999999999999},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.41299999999999998},"max":{"1min":1.458,"5min":1.556,"15min":1.802},"last":1.2529999999999999}]},{"osd":4,"last update":"Thu Oct 26 01:43:56 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.294,"15min":1.2370000000000001},"min":{"1min":0.95399999999999996,"5min":0.79800000000000004,"15min":0.56200000000000006},"max":{"1min":1.728,"5min":1.728,"15min":2.0659999999999998},"last":1.214},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.26,"15min":1.1910000000000001},"min":{"1min":0.90300000000000002,"5min":0.70999999999999996,"15min":0.65800000000000003},"max":{"1min":1.6850000000000001,"5min":1.8420000000000001,"15min":2.0270000000000001},"last":1.347}]},{"osd":5,"last update":"Thu Oct 26 01:43:56 2023","interfaces":[{"interface":"back","average":{"1min":1.302,"5min":1.3240000000000001,"15min":1.2649999999999999},"min":{"1min":0.70899999999999996,"5min":0.629,"15min":0.61799999999999999},"max":{"1min":1.776,"5min":1.776,"15min":1.8779999999999999},"last":1.286},{"interface":"front","average":{"1min":1.3919999999999999,"5min":1.401,"15min":1.3640000000000001},"min":{"1min":0.94599999999999995,"5min":0.72899999999999998,"15min":0.59999999999999998},"max":{"1min":1.859,"5min":1.859,"15min":2.2200000000000002},"last":1.3779999999999999}]},{"osd":6,"last update":"Thu Oct 26 01:43:56 2023","interfaces":[{"interface":"back","average":{"1min":1.391,"5min":1.3640000000000001,"15min":1.3200000000000001},"min":{"1min":1.1100000000000001,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.6890000000000001,"5min":1.8,"15min":2.3780000000000001},"last":1.4319999999999999},{"interface":"front","average":{"1min":1.3129999999999999,"5min":1.361,"15min":1.294},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.70799999999999996},"max":{"1min":1.6579999999999999,"5min":1.8819999999999999,"15min":1.984},"last":1.131}]},{"osd":7,"last update":"Thu Oct 26 01:43:56 2023","interfaces":[{"interface":"back","average":{"1min":1.3320000000000001,"5min":1.357,"15min":1.2909999999999999},"min":{"1min":0.872,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.802,"5min":1.835,"15min":2.6930000000000001},"last":1.1579999999999999},{"interface":"front","average":{"1min":1.472,"5min":1.4450000000000001,"15min":1.393},"min":{"1min":1.0509999999999999,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.863,"5min":1.903,"15min":2.0190000000000001},"last":1.3240000000000001}]}]},{"osd":0,"up_from":13,"seq":55834575021,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1619,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193692,"statfs":{"total":200000000,"available":198341621,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Oct 26 01:43:58 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.79000000000000004,"15min":0.879},"min":{"1min":0.46200000000000002,"5min":0.32500000000000001,"15min":0.32000000000000001},"max":{"1min":1.1879999999999999,"5min":1.494,"15min":19.425000000000001},"last":0.62},{"interface":"front","average":{"1min":1.0229999999999999,"5min":1.004,"15min":1.0229999999999999},"min":{"1min":0.54600000000000004,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.5289999999999999,"5min":1.5289999999999999,"15min":15.162000000000001},"last":0.57099999999999995}]},{"osd":2,"last update":"Thu Oct 26 01:43:58 2023","interfaces":[{"interface":"back","average":{"1min":1.0329999999999999,"5min":1.0489999999999999,"15min":1.1439999999999999},"min":{"1min":0.63700000000000001,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.3879999999999999,"5min":1.556,"15min":28.439},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.93100000000000005,"15min":0.91800000000000004},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.37},"max":{"1min":1.494,"5min":1.851,"15min":4.9459999999999997},"last":0.91800000000000004}]},{"osd":3,"last update":"Thu Oct 26 01:43:58 2023","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.1359999999999999,"15min":1.0920000000000001},"min":{"1min":0.58799999999999997,"5min":0.56999999999999995,"15min":0.44600000000000001},"max":{"1min":1.5009999999999999,"5min":1.7509999999999999,"15min":4.9800000000000004},"last":1.0860000000000001},{"interface":"front","average":{"1min":1.163,"5min":1.2290000000000001,"15min":1.1679999999999999},"min":{"1min":0.67300000000000004,"5min":0.63500000000000001,"15min":0.438},"max":{"1min":1.645,"5min":1.774,"15min":5.016},"last":0.81100000000000005}]},{"osd":4,"last update":"Thu Oct 26 01:43:58 2023","interfaces":[{"interface":"back","average":{"1min":1.169,"5min":1.159,"15min":1.246},"min":{"1min":0.84299999999999997,"5min":0.76800000000000002,"15min":0.52900000000000003},"max":{"1min":1.5389999999999999,"5min":1.5840000000000001,"15min":19.329000000000001},"last":0.876},{"interface":"front","average":{"1min":1.242,"5min":1.254,"15min":1.2689999999999999},"min":{"1min":1.01,"5min":0.78500000000000003,"15min":0.67600000000000005},"max":{"1min":1.5960000000000001,"5min":1.7270000000000001,"15min":5.5229999999999997},"last":0.84599999999999997}]},{"osd":5,"last update":"Thu Oct 26 01:43:58 2023","interfaces":[{"interface":"back","average":{"1min":1.4430000000000001,"5min":1.4319999999999999,"15min":1.4219999999999999},"min":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.64900000000000002},"max":{"1min":2.605,"5min":2.605,"15min":5.6360000000000001},"last":0.97199999999999998},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.286,"15min":1.367},"min":{"1min":0.92400000000000004,"5min":0.80800000000000005,"15min":0.57499999999999996},"max":{"1min":2.036,"5min":2.036,"15min":19.384},"last":0.999}]},{"osd":6,"last update":"Thu Oct 26 01:43:58 2023","interfaces":[{"interface":"back","average":{"1min":1.3120000000000001,"5min":1.319,"15min":1.343},"min":{"1min":0.875,"5min":0.81100000000000005,"15min":0.73799999999999999},"max":{"1min":1.611,"5min":1.8999999999999999,"15min":8.2609999999999992},"last":1.274},{"interface":"front","average":{"1min":1.3320000000000001,"5min":1.3660000000000001,"15min":1.3899999999999999},"min":{"1min":0.76100000000000001,"5min":0.746,"15min":0.746},"max":{"1min":1.7110000000000001,"5min":1.9399999999999999,"15min":11.849},"last":1.0649999999999999}]},{"osd":7,"last update":"Thu Oct 26 01:43:58 2023","interfaces":[{"interface":"back","average":{"1min":1.4650000000000001,"5min":1.488,"15min":1.47},"min":{"1min":1.016,"5min":0.999,"15min":0.73699999999999999},"max":{"1min":1.954,"5min":1.954,"15min":5.6689999999999996},"last":1.016},{"interface":"front","average":{"1min":1.419,"5min":1.4330000000000001,"15min":1.484},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.69399999999999995},"max":{"1min":1.71,"5min":1.8149999999999999,"15min":19.245999999999999},"last":1.228}]}]}],"pool_statfs":[]}} 2023-10-26T01:44:48.654 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T01:44:48.878 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:48.876+0000 7f1ddbb9e640 1 -- 172.21.15.169:0/277683820 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1dc00039a0 msgr2=0x7f1dc0003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:48.879 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:48.876+0000 7f1ddbb9e640 1 --2- 172.21.15.169:0/277683820 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1dc00039a0 0x7f1dc0003dc0 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7f1dd000bb20 tx=0x7f1dc00057f0 comp rx=0 tx=0).stop 2023-10-26T01:44:48.879 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:48.876+0000 7f1dda13b640 1 -- 172.21.15.169:0/277683820 reap_dead start 2023-10-26T01:44:48.880 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:48.876+0000 7f1ddbb9e640 1 -- 172.21.15.169:0/277683820 shutdown_connections 2023-10-26T01:44:48.880 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:48.876+0000 7f1ddbb9e640 1 -- 172.21.15.169:0/277683820 >> 172.21.15.169:0/277683820 conn(0x7f1dd4069930 msgr2=0x7f1dd4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:48.880 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:48.876+0000 7f1ddbb9e640 1 -- 172.21.15.169:0/277683820 shutdown_connections 2023-10-26T01:44:48.880 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:48.876+0000 7f1ddbb9e640 1 -- 172.21.15.169:0/277683820 wait complete. 2023-10-26T01:44:48.882 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:48.878+0000 7f1ddbb9e640 1 Processor -- start 2023-10-26T01:44:48.882 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:48.878+0000 7f1ddbb9e640 1 -- start start 2023-10-26T01:44:48.882 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:48.879+0000 7f1ddbb9e640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1dc00039a0 0x7f1dd4157440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:48.882 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:48.879+0000 7f1ddbb9e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1dd4156350 0x7f1dd4155010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:48.883 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:48.879+0000 7f1ddbb9e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1dd4155550 0x7f1dd41559d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:48.883 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:48.879+0000 7f1ddbb9e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd41f1e30 con 0x7f1dd4155550 2023-10-26T01:44:48.883 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:48.879+0000 7f1ddbb9e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd4155f10 con 0x7f1dd4156350 2023-10-26T01:44:48.883 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:48.879+0000 7f1ddbb9e640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd4157b40 con 0x7f1dc00039a0 2023-10-26T01:44:48.883 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:48.879+0000 7f1dda13b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1dd4155550 0x7f1dd41559d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:48.884 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:48.880+0000 7f1dda13b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1dd4155550 0x7f1dd41559d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:57142/0 (socket says 172.21.15.169:57142) 2023-10-26T01:44:48.884 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:48.880+0000 7f1dda13b640 1 -- 172.21.15.169:0/3749110627 learned_addr learned my addr 172.21.15.169:0/3749110627 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:44:48.884 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:48.880+0000 7f1dd993a640 1 --2- 172.21.15.169:0/3749110627 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1dc00039a0 0x7f1dd4157440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:48.884 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:48.880+0000 7f1dd9139640 1 --2- 172.21.15.169:0/3749110627 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1dd4156350 0x7f1dd4155010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:48.884 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:48.880+0000 7f1dda13b640 1 -- 172.21.15.169:0/3749110627 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1dc00039a0 msgr2=0x7f1dd4157440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:48.885 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:48.880+0000 7f1dda13b640 1 --2- 172.21.15.169:0/3749110627 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1dc00039a0 0x7f1dd4157440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:48.885 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:48.880+0000 7f1dda13b640 1 -- 172.21.15.169:0/3749110627 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1dd4156350 msgr2=0x7f1dd4155010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:48.885 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:48.880+0000 7f1dda13b640 1 --2- 172.21.15.169:0/3749110627 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1dd4156350 0x7f1dd4155010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:48.885 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:48.880+0000 7f1dda13b640 1 -- 172.21.15.169:0/3749110627 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dd4157e20 con 0x7f1dd4155550 2023-10-26T01:44:48.885 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:48.880+0000 7f1dd993a640 1 --2- 172.21.15.169:0/3749110627 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1dc00039a0 0x7f1dd4157440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:44:48.886 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:48.880+0000 7f1dd9139640 1 --2- 172.21.15.169:0/3749110627 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1dd4156350 0x7f1dd4155010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:44:48.886 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:48.880+0000 7f1dda13b640 1 --2- 172.21.15.169:0/3749110627 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1dd4155550 0x7f1dd41559d0 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7f1dd406bff0 tx=0x7f1dcc00e970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:48.888 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:48.880+0000 7f1dca7fc640 1 -- 172.21.15.169:0/3749110627 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dcc00c070 con 0x7f1dd4155550 2023-10-26T01:44:48.888 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:48.881+0000 7f1dca7fc640 1 -- 172.21.15.169:0/3749110627 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1dcc0135f0 con 0x7f1dd4155550 2023-10-26T01:44:48.889 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:48.881+0000 7f1ddbb9e640 1 -- 172.21.15.169:0/3749110627 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dd4158100 con 0x7f1dd4155550 2023-10-26T01:44:48.889 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:48.881+0000 7f1dca7fc640 1 -- 172.21.15.169:0/3749110627 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dcc013a00 con 0x7f1dd4155550 2023-10-26T01:44:48.889 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:48.881+0000 7f1ddbb9e640 1 -- 172.21.15.169:0/3749110627 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dd41567d0 con 0x7f1dd4155550 2023-10-26T01:44:48.889 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:48.882+0000 7f1ddbb9e640 1 -- 172.21.15.169:0/3749110627 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1dd4158900 con 0x7f1dd4155550 2023-10-26T01:44:48.890 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:48.883+0000 7f1dca7fc640 1 -- 172.21.15.169:0/3749110627 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f1dcc01c3b0 con 0x7f1dd4155550 2023-10-26T01:44:48.890 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:48.884+0000 7f1dca7fc640 1 --2- 172.21.15.169:0/3749110627 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f1db806ae70 0x7f1db806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:48.890 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:48.884+0000 7f1dca7fc640 1 -- 172.21.15.169:0/3749110627 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f1dcc088740 con 0x7f1dd4155550 2023-10-26T01:44:48.891 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:48.884+0000 7f1dd993a640 1 --2- 172.21.15.169:0/3749110627 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f1db806ae70 0x7f1db806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:48.891 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:48.886+0000 7f1dd993a640 1 --2- 172.21.15.169:0/3749110627 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f1db806ae70 0x7f1db806d330 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f1dd0037000 tx=0x7f1dd002e040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:48.892 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:48.888+0000 7f1dca7fc640 1 -- 172.21.15.169:0/3749110627 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1dcc053ae0 con 0x7f1dd4155550 2023-10-26T01:44:49.080 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:49.078+0000 7f1ddbb9e640 1 -- 172.21.15.169:0/3749110627 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1dd415f340 con 0x7f1dd4155550 2023-10-26T01:44:49.081 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:49.079+0000 7f1dca7fc640 1 -- 172.21.15.169:0/3749110627 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v189) v1 ==== 74+0+13047 (secure 0 0 0) 0x7f1dcc018020 con 0x7f1dd4155550 2023-10-26T01:44:49.081 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:44:49.082 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":189,"fsid":"30203b28-ed8a-44e8-a9a6-a9bf25f980f1","created":"2023-10-26T01:30:22.897155+0000","modified":"2023-10-26T01:44:44.211179+0000","last_up_change":"2023-10-26T01:30:31.875281+0000","last_in_change":"2023-10-26T01:30:27.637941+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T01:30:32.200808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6817","nonce":4204255864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6819","nonce":4204255864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6823","nonce":4204255864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":4204255864},{"type":"v1","addr":"172.21.15.169:6821","nonce":4204255864}]},"public_addr":"172.21.15.169:6817/4204255864","cluster_addr":"172.21.15.169:6819/4204255864","heartbeat_back_addr":"172.21.15.169:6823/4204255864","heartbeat_front_addr":"172.21.15.169:6821/4204255864","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6809","nonce":238206046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6811","nonce":238206046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6815","nonce":238206046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":238206046},{"type":"v1","addr":"172.21.15.169:6813","nonce":238206046}]},"public_addr":"172.21.15.169:6809/238206046","cluster_addr":"172.21.15.169:6811/238206046","heartbeat_back_addr":"172.21.15.169:6815/238206046","heartbeat_front_addr":"172.21.15.169:6813/238206046","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6801","nonce":3215791846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6803","nonce":3215791846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6807","nonce":3215791846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3215791846},{"type":"v1","addr":"172.21.15.169:6805","nonce":3215791846}]},"public_addr":"172.21.15.169:6801/3215791846","cluster_addr":"172.21.15.169:6803/3215791846","heartbeat_back_addr":"172.21.15.169:6807/3215791846","heartbeat_front_addr":"172.21.15.169:6805/3215791846","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6825","nonce":2256701226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6827","nonce":2256701226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6831","nonce":2256701226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2256701226},{"type":"v1","addr":"172.21.15.169:6829","nonce":2256701226}]},"public_addr":"172.21.15.169:6825/2256701226","cluster_addr":"172.21.15.169:6827/2256701226","heartbeat_back_addr":"172.21.15.169:6831/2256701226","heartbeat_front_addr":"172.21.15.169:6829/2256701226","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6817","nonce":1082279874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6819","nonce":1082279874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6828","nonce":1082279874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":1082279874},{"type":"v1","addr":"172.21.15.170:6823","nonce":1082279874}]},"public_addr":"172.21.15.170:6817/1082279874","cluster_addr":"172.21.15.170:6819/1082279874","heartbeat_back_addr":"172.21.15.170:6828/1082279874","heartbeat_front_addr":"172.21.15.170:6823/1082279874","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6809","nonce":3335654351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6811","nonce":3335654351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6815","nonce":3335654351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3335654351},{"type":"v1","addr":"172.21.15.170:6813","nonce":3335654351}]},"public_addr":"172.21.15.170:6809/3335654351","cluster_addr":"172.21.15.170:6811/3335654351","heartbeat_back_addr":"172.21.15.170:6815/3335654351","heartbeat_front_addr":"172.21.15.170:6813/3335654351","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6801","nonce":2520460697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6803","nonce":2520460697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6807","nonce":2520460697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2520460697},{"type":"v1","addr":"172.21.15.170:6805","nonce":2520460697}]},"public_addr":"172.21.15.170:6801/2520460697","cluster_addr":"172.21.15.170:6803/2520460697","heartbeat_back_addr":"172.21.15.170:6807/2520460697","heartbeat_front_addr":"172.21.15.170:6805/2520460697","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6821","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6822","nonce":1115930619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6826","nonce":1115930619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6831","nonce":1115930619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6827","nonce":1115930619},{"type":"v1","addr":"172.21.15.170:6829","nonce":1115930619}]},"public_addr":"172.21.15.170:6822/1115930619","cluster_addr":"172.21.15.170:6826/1115930619","heartbeat_back_addr":"172.21.15.170:6831/1115930619","heartbeat_front_addr":"172.21.15.170:6829/1115930619","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.889675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.864157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.876919+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.974541+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.969696+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.859703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.867917+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T01:30:28.898605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.170:6837/2205518515":"2023-10-27T01:44:40.812405+0000","172.21.15.170:6835/2205518515":"2023-10-27T01:44:40.812405+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T01:44:49.084 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:49.082+0000 7f1ddbb9e640 1 -- 172.21.15.169:0/3749110627 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f1db806ae70 msgr2=0x7f1db806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:49.084 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:49.082+0000 7f1ddbb9e640 1 --2- 172.21.15.169:0/3749110627 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f1db806ae70 0x7f1db806d330 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f1dd0037000 tx=0x7f1dd002e040 comp rx=0 tx=0).stop 2023-10-26T01:44:49.084 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:49.082+0000 7f1ddbb9e640 1 -- 172.21.15.169:0/3749110627 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1dd4155550 msgr2=0x7f1dd41559d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:49.084 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:49.082+0000 7f1ddbb9e640 1 --2- 172.21.15.169:0/3749110627 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1dd4155550 0x7f1dd41559d0 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7f1dd406bff0 tx=0x7f1dcc00e970 comp rx=0 tx=0).stop 2023-10-26T01:44:49.085 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:49.082+0000 7f1ddbb9e640 1 -- 172.21.15.169:0/3749110627 shutdown_connections 2023-10-26T01:44:49.085 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:49.082+0000 7f1ddbb9e640 1 --2- 172.21.15.169:0/3749110627 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f1db806ae70 0x7f1db806d330 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:49.085 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:49.082+0000 7f1ddbb9e640 1 --2- 172.21.15.169:0/3749110627 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1dd4155550 0x7f1dd41559d0 unknown :-1 s=CLOSED pgs=1979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:49.085 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:49.082+0000 7f1ddbb9e640 1 --2- 172.21.15.169:0/3749110627 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1dd4156350 0x7f1dd4155010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:49.085 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:49.082+0000 7f1ddbb9e640 1 --2- 172.21.15.169:0/3749110627 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1dc00039a0 0x7f1dd4157440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:49.086 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:49.082+0000 7f1ddbb9e640 1 -- 172.21.15.169:0/3749110627 >> 172.21.15.169:0/3749110627 conn(0x7f1dd4069930 msgr2=0x7f1dd415a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:49.086 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:49.082+0000 7f1ddbb9e640 1 -- 172.21.15.169:0/3749110627 shutdown_connections 2023-10-26T01:44:49.086 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:49.082+0000 7f1ddbb9e640 1 -- 172.21.15.169:0/3749110627 wait complete. 2023-10-26T01:44:50.104 INFO:tasks.ceph:Scrubbing osd.0 2023-10-26T01:44:50.104 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-10-26T01:44:50.339 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.337+0000 7f1f3317d640 1 -- 172.21.15.169:0/393161912 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1f2c1595e0 msgr2=0x7f1f2c0c9bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:50.340 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.337+0000 7f1f3317d640 1 --2- 172.21.15.169:0/393161912 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1f2c1595e0 0x7f1f2c0c9bd0 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c0cb1b0 tx=0x7f1f1c02c1d0 comp rx=0 tx=0).stop 2023-10-26T01:44:50.340 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.337+0000 7f1f3317d640 1 -- 172.21.15.169:0/393161912 shutdown_connections 2023-10-26T01:44:50.340 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.337+0000 7f1f3317d640 1 --2- 172.21.15.169:0/393161912 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1f2c06c2c0 0x7f1f2c0ac2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:50.340 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.337+0000 7f1f3317d640 1 --2- 172.21.15.169:0/393161912 >> v2:172.21.15.170:3300/0 conn(0x7f1f2c15d120 0x7f1f2c06bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:50.341 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.337+0000 7f1f3317d640 1 --2- 172.21.15.169:0/393161912 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1f2c1595e0 0x7f1f2c0c9bd0 unknown :-1 s=CLOSED pgs=1980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:50.341 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.337+0000 7f1f3317d640 1 -- 172.21.15.169:0/393161912 >> 172.21.15.169:0/393161912 conn(0x7f1f2c0aeee0 msgr2=0x7f1f2c0af2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:50.341 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.337+0000 7f1f3317d640 1 -- 172.21.15.169:0/393161912 shutdown_connections 2023-10-26T01:44:50.341 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.337+0000 7f1f3317d640 1 -- 172.21.15.169:0/393161912 wait complete. 2023-10-26T01:44:50.343 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.339+0000 7f1f3317d640 1 Processor -- start 2023-10-26T01:44:50.344 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.339+0000 7f1f3317d640 1 -- start start 2023-10-26T01:44:50.344 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.340+0000 7f1f3317d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f2c06c2c0 0x7f1f2c070420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:50.344 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.340+0000 7f1f3317d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1f2c1595e0 0x7f1f2c156440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:50.344 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.340+0000 7f1f3317d640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1f2c15d120 0x7f1f2c157720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:50.344 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.340+0000 7f1f3317d640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f2c1f2f80 con 0x7f1f2c1595e0 2023-10-26T01:44:50.345 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.340+0000 7f1f3317d640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f2c14ebb0 con 0x7f1f2c06c2c0 2023-10-26T01:44:50.345 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.340+0000 7f1f3317d640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f2c070960 con 0x7f1f2c15d120 2023-10-26T01:44:50.345 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.341+0000 7f1f2bfff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1f2c1595e0 0x7f1f2c156440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:50.345 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.341+0000 7f1f2bfff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1f2c1595e0 0x7f1f2c156440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:57168/0 (socket says 172.21.15.169:57168) 2023-10-26T01:44:50.345 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.341+0000 7f1f2bfff640 1 -- 172.21.15.169:0/3988941144 learned_addr learned my addr 172.21.15.169:0/3988941144 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:44:50.346 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.341+0000 7f1f3171a640 1 --2- 172.21.15.169:0/3988941144 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1f2c15d120 0x7f1f2c157720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:50.346 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.341+0000 7f1f30f19640 1 --2- 172.21.15.169:0/3988941144 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f2c06c2c0 0x7f1f2c070420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:50.346 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.341+0000 7f1f2bfff640 1 -- 172.21.15.169:0/3988941144 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1f2c15d120 msgr2=0x7f1f2c157720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:50.346 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.341+0000 7f1f2bfff640 1 --2- 172.21.15.169:0/3988941144 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1f2c15d120 0x7f1f2c157720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:50.346 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.341+0000 7f1f2bfff640 1 -- 172.21.15.169:0/3988941144 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f2c06c2c0 msgr2=0x7f1f2c070420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:50.347 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.341+0000 7f1f2bfff640 1 --2- 172.21.15.169:0/3988941144 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f2c06c2c0 0x7f1f2c070420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:50.347 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.341+0000 7f1f2bfff640 1 -- 172.21.15.169:0/3988941144 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f1c009900 con 0x7f1f2c1595e0 2023-10-26T01:44:50.347 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.341+0000 7f1f30f19640 1 --2- 172.21.15.169:0/3988941144 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f2c06c2c0 0x7f1f2c070420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:44:50.347 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.341+0000 7f1f2bfff640 1 --2- 172.21.15.169:0/3988941144 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1f2c1595e0 0x7f1f2c156440 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7f1f2000c970 tx=0x7f1f2000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:50.347 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.342+0000 7f1f297fa640 1 -- 172.21.15.169:0/3988941144 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f20007bb0 con 0x7f1f2c1595e0 2023-10-26T01:44:50.348 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.342+0000 7f1f297fa640 1 -- 172.21.15.169:0/3988941144 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1f20007d10 con 0x7f1f2c1595e0 2023-10-26T01:44:50.348 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.342+0000 7f1f297fa640 1 -- 172.21.15.169:0/3988941144 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f20005280 con 0x7f1f2c1595e0 2023-10-26T01:44:50.348 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.342+0000 7f1f3317d640 1 -- 172.21.15.169:0/3988941144 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f2c1f29c0 con 0x7f1f2c1595e0 2023-10-26T01:44:50.348 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.342+0000 7f1f3317d640 1 -- 172.21.15.169:0/3988941144 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f2c155c40 con 0x7f1f2c1595e0 2023-10-26T01:44:50.349 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.343+0000 7f1f3317d640 1 -- 172.21.15.169:0/3988941144 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1f2c04da70 con 0x7f1f2c1595e0 2023-10-26T01:44:50.350 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.344+0000 7f1f297fa640 1 -- 172.21.15.169:0/3988941144 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f1f200053e0 con 0x7f1f2c1595e0 2023-10-26T01:44:50.350 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.345+0000 7f1f297fa640 1 --2- 172.21.15.169:0/3988941144 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f1f1806ae40 0x7f1f1806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:50.350 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.345+0000 7f1f297fa640 1 -- 172.21.15.169:0/3988941144 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f1f200898b0 con 0x7f1f2c1595e0 2023-10-26T01:44:50.350 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.345+0000 7f1f297fa640 1 --2- 172.21.15.169:0/3988941144 >> [v2:172.21.15.169:6816/4204255864,v1:172.21.15.169:6817/4204255864] conn(0x7f1f18070ca0 0x7f1f180730c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:50.350 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.345+0000 7f1f297fa640 1 -- 172.21.15.169:0/3988941144 --> [v2:172.21.15.169:6816/4204255864,v1:172.21.15.169:6817/4204255864] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1f18073770 con 0x7f1f18070ca0 2023-10-26T01:44:50.351 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.345+0000 7f1f297fa640 1 -- 172.21.15.169:0/3988941144 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_get_version_reply(handle=1 version=189) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1f2008be30 con 0x7f1f2c1595e0 2023-10-26T01:44:50.351 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.345+0000 7f1f3171a640 1 --2- 172.21.15.169:0/3988941144 >> [v2:172.21.15.169:6816/4204255864,v1:172.21.15.169:6817/4204255864] conn(0x7f1f18070ca0 0x7f1f180730c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:50.351 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.346+0000 7f1f30f19640 1 --2- 172.21.15.169:0/3988941144 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f1f1806ae40 0x7f1f1806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:50.351 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.346+0000 7f1f3171a640 1 --2- 172.21.15.169:0/3988941144 >> [v2:172.21.15.169:6816/4204255864,v1:172.21.15.169:6817/4204255864] conn(0x7f1f18070ca0 0x7f1f180730c0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:50.351 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.347+0000 7f1f30f19640 1 --2- 172.21.15.169:0/3988941144 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f1f1806ae40 0x7f1f1806d300 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c036000 tx=0x7f1f1c004ab0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:50.352 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.347+0000 7f1f297fa640 1 -- 172.21.15.169:0/3988941144 <== osd.0 v2:172.21.15.169:6816/4204255864 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23595 (crc 0 0 0) 0x7f1f18073770 con 0x7f1f18070ca0 2023-10-26T01:44:50.361 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.359+0000 7f1f3317d640 1 -- 172.21.15.169:0/3988941144 --> [v2:172.21.15.169:6816/4204255864,v1:172.21.15.169:6817/4204255864] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f1f2c066af0 con 0x7f1f18070ca0 2023-10-26T01:44:50.361 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.360+0000 7f1f297fa640 1 -- 172.21.15.169:0/3988941144 <== osd.0 v2:172.21.15.169:6816/4204255864 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+632 (crc 0 0 0) 0x7f1f2c066af0 con 0x7f1f18070ca0 2023-10-26T01:44:50.362 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2023-10-26T01:44:50.362 INFO:teuthology.orchestra.run.smithi169.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_pg_delete_cost = '' (not observed, change may require restart) osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2023-10-26T01:44:50.362 INFO:teuthology.orchestra.run.smithi169.stdout:} 2023-10-26T01:44:50.363 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.360+0000 7f1f3317d640 1 -- 172.21.15.169:0/3988941144 >> [v2:172.21.15.169:6816/4204255864,v1:172.21.15.169:6817/4204255864] conn(0x7f1f18070ca0 msgr2=0x7f1f180730c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:50.363 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.360+0000 7f1f3317d640 1 --2- 172.21.15.169:0/3988941144 >> [v2:172.21.15.169:6816/4204255864,v1:172.21.15.169:6817/4204255864] conn(0x7f1f18070ca0 0x7f1f180730c0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:50.364 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.360+0000 7f1f3317d640 1 -- 172.21.15.169:0/3988941144 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f1f1806ae40 msgr2=0x7f1f1806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:50.364 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.360+0000 7f1f3317d640 1 --2- 172.21.15.169:0/3988941144 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f1f1806ae40 0x7f1f1806d300 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c036000 tx=0x7f1f1c004ab0 comp rx=0 tx=0).stop 2023-10-26T01:44:50.364 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.360+0000 7f1f3317d640 1 -- 172.21.15.169:0/3988941144 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1f2c1595e0 msgr2=0x7f1f2c156440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:50.364 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.360+0000 7f1f3317d640 1 --2- 172.21.15.169:0/3988941144 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1f2c1595e0 0x7f1f2c156440 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7f1f2000c970 tx=0x7f1f2000ce30 comp rx=0 tx=0).stop 2023-10-26T01:44:50.364 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.361+0000 7f1f3171a640 1 -- 172.21.15.169:0/3988941144 reap_dead start 2023-10-26T01:44:50.365 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.361+0000 7f1f3317d640 1 -- 172.21.15.169:0/3988941144 shutdown_connections 2023-10-26T01:44:50.365 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.361+0000 7f1f3317d640 1 -- 172.21.15.169:0/3988941144 >> 172.21.15.169:0/3988941144 conn(0x7f1f2c0aeee0 msgr2=0x7f1f2c14f580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:50.365 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.361+0000 7f1f3317d640 1 -- 172.21.15.169:0/3988941144 shutdown_connections 2023-10-26T01:44:50.365 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.361+0000 7f1f3317d640 1 -- 172.21.15.169:0/3988941144 wait complete. 2023-10-26T01:44:50.380 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-10-26T01:44:50.602 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.600+0000 7f05ad6c9640 1 -- 172.21.15.169:0/1134780231 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f05a80c9d00 msgr2=0x7f05a80ac320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:50.603 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.600+0000 7f05ad6c9640 1 --2- 172.21.15.169:0/1134780231 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f05a80c9d00 0x7f05a80ac320 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7f0590009830 tx=0x7f059002bd50 comp rx=0 tx=0).stop 2023-10-26T01:44:50.603 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.600+0000 7f05ad6c9640 1 -- 172.21.15.169:0/1134780231 shutdown_connections 2023-10-26T01:44:50.603 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.600+0000 7f05ad6c9640 1 --2- 172.21.15.169:0/1134780231 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f05a80cde20 0x7f05a80ac860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:50.603 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.600+0000 7f05ad6c9640 1 --2- 172.21.15.169:0/1134780231 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f05a80c9d00 0x7f05a80ac320 unknown :-1 s=CLOSED pgs=1983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:50.603 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.600+0000 7f05ad6c9640 1 -- 172.21.15.169:0/1134780231 >> 172.21.15.169:0/1134780231 conn(0x7f05a8069930 msgr2=0x7f05a8069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:50.604 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.600+0000 7f05ad6c9640 1 -- 172.21.15.169:0/1134780231 shutdown_connections 2023-10-26T01:44:50.604 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.600+0000 7f05ad6c9640 1 -- 172.21.15.169:0/1134780231 wait complete. 2023-10-26T01:44:50.605 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.602+0000 7f05ad6c9640 1 Processor -- start 2023-10-26T01:44:50.605 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.602+0000 7f05ad6c9640 1 -- start start 2023-10-26T01:44:50.605 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.603+0000 7f05ad6c9640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f05a80c9d00 0x7f05a8150970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:50.606 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.603+0000 7f05ad6c9640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05a80cde20 0x7f05a814f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:50.606 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.603+0000 7f05ad6c9640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f05a815b620 0x7f05a815d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:50.606 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.603+0000 7f05ad6c9640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05a80cbc60 con 0x7f05a80c9d00 2023-10-26T01:44:50.606 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.603+0000 7f05ad6c9640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05a814ff60 con 0x7f05a80cde20 2023-10-26T01:44:50.606 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.603+0000 7f05ad6c9640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05a8150eb0 con 0x7f05a815b620 2023-10-26T01:44:50.607 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.603+0000 7f05a6ffd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f05a80c9d00 0x7f05a8150970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:50.607 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.603+0000 7f05a6ffd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f05a80c9d00 0x7f05a8150970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:57190/0 (socket says 172.21.15.169:57190) 2023-10-26T01:44:50.607 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.603+0000 7f05a6ffd640 1 -- 172.21.15.169:0/3192953262 learned_addr learned my addr 172.21.15.169:0/3192953262 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:44:50.608 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.604+0000 7f05a77fe640 1 --2- 172.21.15.169:0/3192953262 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f05a815b620 0x7f05a815d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:50.608 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.604+0000 7f05a6ffd640 1 -- 172.21.15.169:0/3192953262 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f05a815b620 msgr2=0x7f05a815d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:50.609 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.604+0000 7f05a67fc640 1 --2- 172.21.15.169:0/3192953262 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05a80cde20 0x7f05a814f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:50.609 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.604+0000 7f05a6ffd640 1 --2- 172.21.15.169:0/3192953262 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f05a815b620 0x7f05a815d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:50.609 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.604+0000 7f05a6ffd640 1 -- 172.21.15.169:0/3192953262 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05a80cde20 msgr2=0x7f05a814f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:50.609 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.604+0000 7f05a6ffd640 1 --2- 172.21.15.169:0/3192953262 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05a80cde20 0x7f05a814f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:50.609 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.604+0000 7f05a6ffd640 1 -- 172.21.15.169:0/3192953262 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0590009490 con 0x7f05a80c9d00 2023-10-26T01:44:50.610 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.604+0000 7f05a6ffd640 1 --2- 172.21.15.169:0/3192953262 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f05a80c9d00 0x7f05a8150970 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7f059800cae0 tx=0x7f059800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:50.610 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.604+0000 7f059f7fe640 1 -- 172.21.15.169:0/3192953262 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f059800ac40 con 0x7f05a80c9d00 2023-10-26T01:44:50.610 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.604+0000 7f059f7fe640 1 -- 172.21.15.169:0/3192953262 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0598003970 con 0x7f05a80c9d00 2023-10-26T01:44:50.610 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.604+0000 7f05ad6c9640 1 -- 172.21.15.169:0/3192953262 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05a81504e0 con 0x7f05a80c9d00 2023-10-26T01:44:50.610 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.604+0000 7f059f7fe640 1 -- 172.21.15.169:0/3192953262 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0598003d80 con 0x7f05a80c9d00 2023-10-26T01:44:50.611 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.605+0000 7f05ad6c9640 1 -- 172.21.15.169:0/3192953262 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05a814f8a0 con 0x7f05a80c9d00 2023-10-26T01:44:50.611 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.606+0000 7f05ad6c9640 1 -- 172.21.15.169:0/3192953262 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05a804db10 con 0x7f05a80c9d00 2023-10-26T01:44:50.611 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.607+0000 7f059f7fe640 1 -- 172.21.15.169:0/3192953262 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f0598004030 con 0x7f05a80c9d00 2023-10-26T01:44:50.612 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.607+0000 7f059f7fe640 1 --2- 172.21.15.169:0/3192953262 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f058806ae40 0x7f058806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:50.612 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.608+0000 7f059f7fe640 1 -- 172.21.15.169:0/3192953262 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f0598089c50 con 0x7f05a80c9d00 2023-10-26T01:44:50.612 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.608+0000 7f05a67fc640 1 --2- 172.21.15.169:0/3192953262 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f058806ae40 0x7f058806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:50.612 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.609+0000 7f05a67fc640 1 --2- 172.21.15.169:0/3192953262 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f058806ae40 0x7f058806d300 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f0590036000 tx=0x7f0590004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:50.614 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.612+0000 7f059f7fe640 1 -- 172.21.15.169:0/3192953262 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0598055070 con 0x7f05a80c9d00 2023-10-26T01:44:50.805 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.803+0000 7f05ad6c9640 1 -- 172.21.15.169:0/3192953262 --> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f05a80b2410 con 0x7f058806ae40 2023-10-26T01:44:50.806 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.804+0000 7f059f7fe640 1 -- 172.21.15.169:0/3192953262 <== mgr.4112 v2:172.21.15.170:6832/1187156918 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f05a80b2410 con 0x7f058806ae40 2023-10-26T01:44:50.806 INFO:teuthology.orchestra.run.smithi169.stderr:instructed osd(s) 0 to deep-scrub 2023-10-26T01:44:50.809 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.807+0000 7f05ad6c9640 1 -- 172.21.15.169:0/3192953262 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f058806ae40 msgr2=0x7f058806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:50.809 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.807+0000 7f05ad6c9640 1 --2- 172.21.15.169:0/3192953262 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f058806ae40 0x7f058806d300 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f0590036000 tx=0x7f0590004730 comp rx=0 tx=0).stop 2023-10-26T01:44:50.809 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.807+0000 7f05ad6c9640 1 -- 172.21.15.169:0/3192953262 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f05a80c9d00 msgr2=0x7f05a8150970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:50.809 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.807+0000 7f05ad6c9640 1 --2- 172.21.15.169:0/3192953262 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f05a80c9d00 0x7f05a8150970 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7f059800cae0 tx=0x7f059800a060 comp rx=0 tx=0).stop 2023-10-26T01:44:50.810 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.807+0000 7f05ad6c9640 1 -- 172.21.15.169:0/3192953262 shutdown_connections 2023-10-26T01:44:50.810 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.807+0000 7f05ad6c9640 1 --2- 172.21.15.169:0/3192953262 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f058806ae40 0x7f058806d300 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:50.810 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.807+0000 7f05ad6c9640 1 --2- 172.21.15.169:0/3192953262 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f05a815b620 0x7f05a815d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:50.810 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.807+0000 7f05ad6c9640 1 --2- 172.21.15.169:0/3192953262 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05a80cde20 0x7f05a814f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:50.811 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.807+0000 7f05ad6c9640 1 --2- 172.21.15.169:0/3192953262 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f05a80c9d00 0x7f05a8150970 unknown :-1 s=CLOSED pgs=1984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:50.811 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.807+0000 7f05ad6c9640 1 -- 172.21.15.169:0/3192953262 >> 172.21.15.169:0/3192953262 conn(0x7f05a8069930 msgr2=0x7f05a81516e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:50.811 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.807+0000 7f05ad6c9640 1 -- 172.21.15.169:0/3192953262 shutdown_connections 2023-10-26T01:44:50.811 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:50.807+0000 7f05ad6c9640 1 -- 172.21.15.169:0/3192953262 wait complete. 2023-10-26T01:44:50.829 INFO:tasks.ceph:Scrubbing osd.1 2023-10-26T01:44:50.829 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-10-26T01:44:51.035 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.033+0000 7f3475613640 1 -- 172.21.15.169:0/2323357742 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f345c0047c0 msgr2=0x7f345c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:51.035 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.033+0000 7f3475613640 1 --2- 172.21.15.169:0/2323357742 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f345c0047c0 0x7f345c004c40 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7f3458009b20 tx=0x7f345c0057f0 comp rx=0 tx=0).stop 2023-10-26T01:44:51.035 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.033+0000 7f3474e12640 1 -- 172.21.15.169:0/2323357742 reap_dead start 2023-10-26T01:44:51.036 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.033+0000 7f3475613640 1 -- 172.21.15.169:0/2323357742 shutdown_connections 2023-10-26T01:44:51.036 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.033+0000 7f3475613640 1 -- 172.21.15.169:0/2323357742 >> 172.21.15.169:0/2323357742 conn(0x7f34700d13a0 msgr2=0x7f34700d17b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:51.036 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.033+0000 7f3475613640 1 -- 172.21.15.169:0/2323357742 shutdown_connections 2023-10-26T01:44:51.036 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.033+0000 7f3475613640 1 -- 172.21.15.169:0/2323357742 wait complete. 2023-10-26T01:44:51.037 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.035+0000 7f3475613640 1 Processor -- start 2023-10-26T01:44:51.037 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.035+0000 7f3475613640 1 -- start start 2023-10-26T01:44:51.039 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.036+0000 7f3475613640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f345c0047c0 0x7f34700c61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:51.039 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.036+0000 7f3475613640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34700c76a0 0x7f34700c6700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:51.040 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.036+0000 7f3475613640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f34701f4ad0 0x7f347015ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:51.040 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.036+0000 7f3475613640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34700cbd30 con 0x7f34700c76a0 2023-10-26T01:44:51.040 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.036+0000 7f3475613640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34700cb720 con 0x7f345c0047c0 2023-10-26T01:44:51.040 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.036+0000 7f3475613640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34700c72d0 con 0x7f34701f4ad0 2023-10-26T01:44:51.040 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.036+0000 7f3467fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34700c76a0 0x7f34700c6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:51.041 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.036+0000 7f3467fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34700c76a0 0x7f34700c6700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:57214/0 (socket says 172.21.15.169:57214) 2023-10-26T01:44:51.041 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.036+0000 7f3467fff640 1 -- 172.21.15.169:0/1422024301 learned_addr learned my addr 172.21.15.169:0/1422024301 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:44:51.041 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.036+0000 7f346ffff640 1 --2- 172.21.15.169:0/1422024301 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f345c0047c0 0x7f34700c61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:51.041 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.037+0000 7f3467fff640 1 -- 172.21.15.169:0/1422024301 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f34701f4ad0 msgr2=0x7f347015ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:51.041 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.037+0000 7f3474e12640 1 --2- 172.21.15.169:0/1422024301 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f34701f4ad0 0x7f347015ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:51.042 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.037+0000 7f3467fff640 1 --2- 172.21.15.169:0/1422024301 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f34701f4ad0 0x7f347015ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:51.042 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.037+0000 7f3467fff640 1 -- 172.21.15.169:0/1422024301 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f345c0047c0 msgr2=0x7f34700c61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:51.042 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.037+0000 7f3467fff640 1 --2- 172.21.15.169:0/1422024301 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f345c0047c0 0x7f34700c61c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:51.042 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.037+0000 7f3467fff640 1 -- 172.21.15.169:0/1422024301 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34700c6d10 con 0x7f34700c76a0 2023-10-26T01:44:51.043 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.037+0000 7f346ffff640 1 --2- 172.21.15.169:0/1422024301 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f345c0047c0 0x7f34700c61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:44:51.043 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.037+0000 7f3467fff640 1 --2- 172.21.15.169:0/1422024301 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34700c76a0 0x7f34700c6700 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7f345802cab0 tx=0x7f3458033ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:51.043 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.037+0000 7f346d7fa640 1 -- 172.21.15.169:0/1422024301 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3458004220 con 0x7f34700c76a0 2023-10-26T01:44:51.043 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.037+0000 7f346d7fa640 1 -- 172.21.15.169:0/1422024301 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3458004380 con 0x7f34700c76a0 2023-10-26T01:44:51.043 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.037+0000 7f3475613640 1 -- 172.21.15.169:0/1422024301 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34700c6ff0 con 0x7f34700c76a0 2023-10-26T01:44:51.044 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.037+0000 7f346d7fa640 1 -- 172.21.15.169:0/1422024301 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f345803d940 con 0x7f34700c76a0 2023-10-26T01:44:51.044 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.037+0000 7f3475613640 1 -- 172.21.15.169:0/1422024301 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34700b2b90 con 0x7f34700c76a0 2023-10-26T01:44:51.045 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.038+0000 7f3475613640 1 -- 172.21.15.169:0/1422024301 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f34700cbd30 con 0x7f34700c76a0 2023-10-26T01:44:51.045 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.039+0000 7f346d7fa640 1 -- 172.21.15.169:0/1422024301 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f345803daa0 con 0x7f34700c76a0 2023-10-26T01:44:51.046 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.040+0000 7f346d7fa640 1 --2- 172.21.15.169:0/1422024301 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f345006ae40 0x7f345006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:51.046 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.040+0000 7f346d7fa640 1 -- 172.21.15.169:0/1422024301 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f34580ac140 con 0x7f34700c76a0 2023-10-26T01:44:51.046 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.040+0000 7f346d7fa640 1 --2- 172.21.15.169:0/1422024301 >> [v2:172.21.15.169:6808/238206046,v1:172.21.15.169:6809/238206046] conn(0x7f3450070ca0 0x7f34500730c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:51.046 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.040+0000 7f346d7fa640 1 -- 172.21.15.169:0/1422024301 --> [v2:172.21.15.169:6808/238206046,v1:172.21.15.169:6809/238206046] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3450073770 con 0x7f3450070ca0 2023-10-26T01:44:51.047 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.040+0000 7f346d7fa640 1 -- 172.21.15.169:0/1422024301 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_get_version_reply(handle=1 version=189) v2 ==== 24+0+0 (secure 0 0 0) 0x7f34580ae720 con 0x7f34700c76a0 2023-10-26T01:44:51.047 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.041+0000 7f3474e12640 1 --2- 172.21.15.169:0/1422024301 >> [v2:172.21.15.169:6808/238206046,v1:172.21.15.169:6809/238206046] conn(0x7f3450070ca0 0x7f34500730c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:51.047 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.041+0000 7f346ffff640 1 --2- 172.21.15.169:0/1422024301 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f345006ae40 0x7f345006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:51.047 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.041+0000 7f3474e12640 1 --2- 172.21.15.169:0/1422024301 >> [v2:172.21.15.169:6808/238206046,v1:172.21.15.169:6809/238206046] conn(0x7f3450070ca0 0x7f34500730c0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:51.047 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.042+0000 7f346ffff640 1 --2- 172.21.15.169:0/1422024301 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f345006ae40 0x7f345006d300 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f3460001970 tx=0x7f3460001240 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:51.048 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.042+0000 7f346d7fa640 1 -- 172.21.15.169:0/1422024301 <== osd.1 v2:172.21.15.169:6808/238206046 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23595 (crc 0 0 0) 0x7f3450073770 con 0x7f3450070ca0 2023-10-26T01:44:51.055 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.054+0000 7f3475613640 1 -- 172.21.15.169:0/1422024301 --> [v2:172.21.15.169:6808/238206046,v1:172.21.15.169:6809/238206046] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f3470073c10 con 0x7f3450070ca0 2023-10-26T01:44:51.056 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.055+0000 7f346d7fa640 1 -- 172.21.15.169:0/1422024301 <== osd.1 v2:172.21.15.169:6808/238206046 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+632 (crc 0 0 0) 0x7f3470073c10 con 0x7f3450070ca0 2023-10-26T01:44:51.056 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2023-10-26T01:44:51.057 INFO:teuthology.orchestra.run.smithi169.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_pg_delete_cost = '' (not observed, change may require restart) osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2023-10-26T01:44:51.057 INFO:teuthology.orchestra.run.smithi169.stdout:} 2023-10-26T01:44:51.058 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.055+0000 7f3475613640 1 -- 172.21.15.169:0/1422024301 >> [v2:172.21.15.169:6808/238206046,v1:172.21.15.169:6809/238206046] conn(0x7f3450070ca0 msgr2=0x7f34500730c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:51.058 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.055+0000 7f3475613640 1 --2- 172.21.15.169:0/1422024301 >> [v2:172.21.15.169:6808/238206046,v1:172.21.15.169:6809/238206046] conn(0x7f3450070ca0 0x7f34500730c0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:51.058 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.055+0000 7f3475613640 1 -- 172.21.15.169:0/1422024301 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f345006ae40 msgr2=0x7f345006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:51.059 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.055+0000 7f3475613640 1 --2- 172.21.15.169:0/1422024301 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f345006ae40 0x7f345006d300 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f3460001970 tx=0x7f3460001240 comp rx=0 tx=0).stop 2023-10-26T01:44:51.059 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.055+0000 7f3475613640 1 -- 172.21.15.169:0/1422024301 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34700c76a0 msgr2=0x7f34700c6700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:51.059 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.055+0000 7f3475613640 1 --2- 172.21.15.169:0/1422024301 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34700c76a0 0x7f34700c6700 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7f345802cab0 tx=0x7f3458033ad0 comp rx=0 tx=0).stop 2023-10-26T01:44:51.059 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.055+0000 7f3474e12640 1 -- 172.21.15.169:0/1422024301 reap_dead start 2023-10-26T01:44:51.059 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.055+0000 7f3475613640 1 -- 172.21.15.169:0/1422024301 shutdown_connections 2023-10-26T01:44:51.060 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.055+0000 7f3475613640 1 -- 172.21.15.169:0/1422024301 >> 172.21.15.169:0/1422024301 conn(0x7f34700d13a0 msgr2=0x7f3470158070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:51.060 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.055+0000 7f3475613640 1 -- 172.21.15.169:0/1422024301 shutdown_connections 2023-10-26T01:44:51.060 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.056+0000 7f3475613640 1 -- 172.21.15.169:0/1422024301 wait complete. 2023-10-26T01:44:51.072 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-10-26T01:44:51.292 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.289+0000 7f261f339640 1 -- 172.21.15.169:0/2500587140 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f261806ce00 msgr2=0x7f2618155360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:51.292 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.289+0000 7f261f339640 1 --2- 172.21.15.169:0/2500587140 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f261806ce00 0x7f2618155360 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f2614005fd0 tx=0x7f261402c240 comp rx=0 tx=0).stop 2023-10-26T01:44:51.292 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.290+0000 7f261f339640 1 -- 172.21.15.169:0/2500587140 shutdown_connections 2023-10-26T01:44:51.293 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.290+0000 7f261f339640 1 --2- 172.21.15.169:0/2500587140 >> v2:172.21.15.170:3300/0 conn(0x7f261815b4a0 0x7f261815b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:51.293 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.290+0000 7f261f339640 1 --2- 172.21.15.169:0/2500587140 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f261806ce00 0x7f2618155360 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:51.293 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.290+0000 7f261f339640 1 -- 172.21.15.169:0/2500587140 >> 172.21.15.169:0/2500587140 conn(0x7f2618069930 msgr2=0x7f2618069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:51.293 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.290+0000 7f261f339640 1 -- 172.21.15.169:0/2500587140 shutdown_connections 2023-10-26T01:44:51.293 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.290+0000 7f261f339640 1 -- 172.21.15.169:0/2500587140 wait complete. 2023-10-26T01:44:51.296 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.291+0000 7f261f339640 1 Processor -- start 2023-10-26T01:44:51.296 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.292+0000 7f261f339640 1 -- start start 2023-10-26T01:44:51.296 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.293+0000 7f261f339640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f261806ad90 0x7f26181eeee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:51.297 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.293+0000 7f261f339640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f261806ce00 0x7f26181558a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:51.297 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.293+0000 7f261f339640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f261815b4a0 0x7f2618155de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:51.297 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.293+0000 7f261f339640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26181f16e0 con 0x7f261815b4a0 2023-10-26T01:44:51.297 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.293+0000 7f261f339640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26181f19c0 con 0x7f261806ad90 2023-10-26T01:44:51.298 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.293+0000 7f261f339640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26181f1ca0 con 0x7f261806ce00 2023-10-26T01:44:51.298 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.293+0000 7f261d8d6640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f261815b4a0 0x7f2618155de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:51.298 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.293+0000 7f261d8d6640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f261815b4a0 0x7f2618155de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:57230/0 (socket says 172.21.15.169:57230) 2023-10-26T01:44:51.298 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.293+0000 7f261d8d6640 1 -- 172.21.15.169:0/134915075 learned_addr learned my addr 172.21.15.169:0/134915075 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:44:51.298 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.293+0000 7f261d0d5640 1 --2- 172.21.15.169:0/134915075 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f261806ad90 0x7f26181eeee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:51.299 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.293+0000 7f261c8d4640 1 --2- 172.21.15.169:0/134915075 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f261806ce00 0x7f26181558a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:51.299 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.293+0000 7f261d8d6640 1 -- 172.21.15.169:0/134915075 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f261806ce00 msgr2=0x7f26181558a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:51.299 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.293+0000 7f261d8d6640 1 --2- 172.21.15.169:0/134915075 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f261806ce00 0x7f26181558a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:51.299 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.293+0000 7f261d8d6640 1 -- 172.21.15.169:0/134915075 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f261806ad90 msgr2=0x7f26181eeee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:51.299 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.293+0000 7f261d8d6640 1 --2- 172.21.15.169:0/134915075 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f261806ad90 0x7f26181eeee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:51.300 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.293+0000 7f261d8d6640 1 -- 172.21.15.169:0/134915075 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2614009900 con 0x7f261815b4a0 2023-10-26T01:44:51.300 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.293+0000 7f261d0d5640 1 --2- 172.21.15.169:0/134915075 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f261806ad90 0x7f26181eeee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:44:51.300 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.294+0000 7f261d8d6640 1 --2- 172.21.15.169:0/134915075 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f261815b4a0 0x7f2618155de0 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7f261000c3c0 tx=0x7f261000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:51.300 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.294+0000 7f260dffb640 1 -- 172.21.15.169:0/134915075 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26100052d0 con 0x7f261815b4a0 2023-10-26T01:44:51.301 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.294+0000 7f260dffb640 1 -- 172.21.15.169:0/134915075 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2610003610 con 0x7f261815b4a0 2023-10-26T01:44:51.301 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.294+0000 7f260dffb640 1 -- 172.21.15.169:0/134915075 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2610003aa0 con 0x7f261815b4a0 2023-10-26T01:44:51.301 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.294+0000 7f261f339640 1 -- 172.21.15.169:0/134915075 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26181f1f80 con 0x7f261815b4a0 2023-10-26T01:44:51.301 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.294+0000 7f261f339640 1 -- 172.21.15.169:0/134915075 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26181ef420 con 0x7f261815b4a0 2023-10-26T01:44:51.302 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.295+0000 7f261f339640 1 -- 172.21.15.169:0/134915075 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26181f24a0 con 0x7f261815b4a0 2023-10-26T01:44:51.302 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.296+0000 7f260dffb640 1 -- 172.21.15.169:0/134915075 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f2610003c00 con 0x7f261815b4a0 2023-10-26T01:44:51.303 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.297+0000 7f260dffb640 1 --2- 172.21.15.169:0/134915075 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f25fc06ae40 0x7f25fc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:51.303 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.297+0000 7f260dffb640 1 -- 172.21.15.169:0/134915075 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f2610088ac0 con 0x7f261815b4a0 2023-10-26T01:44:51.303 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.298+0000 7f261d0d5640 1 --2- 172.21.15.169:0/134915075 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f25fc06ae40 0x7f25fc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:51.303 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.299+0000 7f261d0d5640 1 --2- 172.21.15.169:0/134915075 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f25fc06ae40 0x7f25fc06d300 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f2614036000 tx=0x7f2614004ab0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:51.304 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.301+0000 7f260dffb640 1 -- 172.21.15.169:0/134915075 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2610053e70 con 0x7f261815b4a0 2023-10-26T01:44:51.477 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.476+0000 7f261f339640 1 -- 172.21.15.169:0/134915075 --> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f26181ef640 con 0x7f25fc06ae40 2023-10-26T01:44:51.479 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.477+0000 7f260dffb640 1 -- 172.21.15.169:0/134915075 <== mgr.4112 v2:172.21.15.170:6832/1187156918 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f26181ef640 con 0x7f25fc06ae40 2023-10-26T01:44:51.479 INFO:teuthology.orchestra.run.smithi169.stderr:instructed osd(s) 1 to deep-scrub 2023-10-26T01:44:51.482 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.480+0000 7f261f339640 1 -- 172.21.15.169:0/134915075 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f25fc06ae40 msgr2=0x7f25fc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:51.482 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.480+0000 7f261f339640 1 --2- 172.21.15.169:0/134915075 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f25fc06ae40 0x7f25fc06d300 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f2614036000 tx=0x7f2614004ab0 comp rx=0 tx=0).stop 2023-10-26T01:44:51.482 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.480+0000 7f261f339640 1 -- 172.21.15.169:0/134915075 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f261815b4a0 msgr2=0x7f2618155de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:51.483 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.480+0000 7f261f339640 1 --2- 172.21.15.169:0/134915075 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f261815b4a0 0x7f2618155de0 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7f261000c3c0 tx=0x7f261000c880 comp rx=0 tx=0).stop 2023-10-26T01:44:51.483 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.480+0000 7f261f339640 1 -- 172.21.15.169:0/134915075 shutdown_connections 2023-10-26T01:44:51.483 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.480+0000 7f261f339640 1 --2- 172.21.15.169:0/134915075 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f25fc06ae40 0x7f25fc06d300 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:51.483 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.480+0000 7f261f339640 1 --2- 172.21.15.169:0/134915075 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f261815b4a0 0x7f2618155de0 unknown :-1 s=CLOSED pgs=1988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:51.483 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.480+0000 7f261f339640 1 --2- 172.21.15.169:0/134915075 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f261806ce00 0x7f26181558a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:51.484 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.480+0000 7f261f339640 1 --2- 172.21.15.169:0/134915075 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f261806ad90 0x7f26181eeee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:51.484 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.480+0000 7f261f339640 1 -- 172.21.15.169:0/134915075 >> 172.21.15.169:0/134915075 conn(0x7f2618069930 msgr2=0x7f26180b0920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:51.484 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.480+0000 7f261f339640 1 -- 172.21.15.169:0/134915075 shutdown_connections 2023-10-26T01:44:51.484 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.480+0000 7f261f339640 1 -- 172.21.15.169:0/134915075 wait complete. 2023-10-26T01:44:51.499 INFO:tasks.ceph:Scrubbing osd.2 2023-10-26T01:44:51.500 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-10-26T01:44:51.722 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.720+0000 7fec3b59e640 1 -- 172.21.15.169:0/135474773 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec24003970 msgr2=0x7fec24003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:51.723 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.720+0000 7fec3b59e640 1 --2- 172.21.15.169:0/135474773 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec24003970 0x7fec24003d90 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7fec2c009f20 tx=0x7fec240057f0 comp rx=0 tx=0).stop 2023-10-26T01:44:51.723 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.720+0000 7fec3ad9d640 1 -- 172.21.15.169:0/135474773 reap_dead start 2023-10-26T01:44:51.723 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.720+0000 7fec3b59e640 1 -- 172.21.15.169:0/135474773 shutdown_connections 2023-10-26T01:44:51.723 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.721+0000 7fec3b59e640 1 -- 172.21.15.169:0/135474773 >> 172.21.15.169:0/135474773 conn(0x7fec3c069b40 msgr2=0x7fec3c0aeb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:51.724 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.721+0000 7fec3b59e640 1 -- 172.21.15.169:0/135474773 shutdown_connections 2023-10-26T01:44:51.724 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.721+0000 7fec3b59e640 1 -- 172.21.15.169:0/135474773 wait complete. 2023-10-26T01:44:51.724 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.722+0000 7fec3b59e640 1 Processor -- start 2023-10-26T01:44:51.725 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.723+0000 7fec3b59e640 1 -- start start 2023-10-26T01:44:51.726 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.723+0000 7fec3b59e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec24003970 0x7fec3c152070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:51.727 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.724+0000 7fec3b59e640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fec3c14f610 0x7fec3c14eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:51.727 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.724+0000 7fec3b59e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fec3c15af00 0x7fec3c15b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:51.727 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.724+0000 7fec3b59e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec3c14fe20 con 0x7fec24003970 2023-10-26T01:44:51.727 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.724+0000 7fec3b59e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec3c1509a0 con 0x7fec3c15af00 2023-10-26T01:44:51.727 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.724+0000 7fec3b59e640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec3c150070 con 0x7fec3c14f610 2023-10-26T01:44:51.728 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.724+0000 7fec3a59c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec24003970 0x7fec3c152070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:51.728 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.724+0000 7fec3a59c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec24003970 0x7fec3c152070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:57246/0 (socket says 172.21.15.169:57246) 2023-10-26T01:44:51.728 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.724+0000 7fec3a59c640 1 -- 172.21.15.169:0/4178811561 learned_addr learned my addr 172.21.15.169:0/4178811561 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:44:51.728 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.724+0000 7fec3a59c640 1 -- 172.21.15.169:0/4178811561 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fec3c14f610 msgr2=0x7fec3c14eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:51.728 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.724+0000 7fec3a59c640 1 --2- 172.21.15.169:0/4178811561 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fec3c14f610 0x7fec3c14eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:51.729 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.724+0000 7fec3a59c640 1 -- 172.21.15.169:0/4178811561 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fec3c15af00 msgr2=0x7fec3c15b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:51.729 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.724+0000 7fec3a59c640 1 --2- 172.21.15.169:0/4178811561 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fec3c15af00 0x7fec3c15b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:51.729 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.724+0000 7fec3a59c640 1 -- 172.21.15.169:0/4178811561 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec3c150350 con 0x7fec24003970 2023-10-26T01:44:51.729 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.724+0000 7fec3a59c640 1 --2- 172.21.15.169:0/4178811561 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec24003970 0x7fec3c152070 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7fec2c038040 tx=0x7fec2c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:51.729 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.725+0000 7fec2affd640 1 -- 172.21.15.169:0/4178811561 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec2c004670 con 0x7fec24003970 2023-10-26T01:44:51.730 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.725+0000 7fec2affd640 1 -- 172.21.15.169:0/4178811561 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fec2c040470 con 0x7fec24003970 2023-10-26T01:44:51.730 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.725+0000 7fec2affd640 1 -- 172.21.15.169:0/4178811561 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec2c040820 con 0x7fec24003970 2023-10-26T01:44:51.730 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.725+0000 7fec3b59e640 1 -- 172.21.15.169:0/4178811561 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec3c150630 con 0x7fec24003970 2023-10-26T01:44:51.730 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.725+0000 7fec3b59e640 1 -- 172.21.15.169:0/4178811561 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec3c150e30 con 0x7fec24003970 2023-10-26T01:44:51.731 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.726+0000 7fec3b59e640 1 -- 172.21.15.169:0/4178811561 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fec3c1501e0 con 0x7fec24003970 2023-10-26T01:44:51.732 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.727+0000 7fec2affd640 1 -- 172.21.15.169:0/4178811561 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fec2c03c070 con 0x7fec24003970 2023-10-26T01:44:51.732 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.728+0000 7fec2affd640 1 --2- 172.21.15.169:0/4178811561 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fec1c06ae70 0x7fec1c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:51.732 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.728+0000 7fec2affd640 1 -- 172.21.15.169:0/4178811561 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fec2c0abca0 con 0x7fec24003970 2023-10-26T01:44:51.732 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.728+0000 7fec2affd640 1 --2- 172.21.15.169:0/4178811561 >> [v2:172.21.15.169:6800/3215791846,v1:172.21.15.169:6801/3215791846] conn(0x7fec1c070cd0 0x7fec1c0730f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:51.732 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.728+0000 7fec2affd640 1 -- 172.21.15.169:0/4178811561 --> [v2:172.21.15.169:6800/3215791846,v1:172.21.15.169:6801/3215791846] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fec1c0737a0 con 0x7fec1c070cd0 2023-10-26T01:44:51.733 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.728+0000 7fec2affd640 1 -- 172.21.15.169:0/4178811561 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_get_version_reply(handle=1 version=189) v2 ==== 24+0+0 (secure 0 0 0) 0x7fec2c0ae300 con 0x7fec24003970 2023-10-26T01:44:51.733 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.728+0000 7fec3ad9d640 1 --2- 172.21.15.169:0/4178811561 >> [v2:172.21.15.169:6800/3215791846,v1:172.21.15.169:6801/3215791846] conn(0x7fec1c070cd0 0x7fec1c0730f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:51.733 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.729+0000 7fec39d9b640 1 --2- 172.21.15.169:0/4178811561 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fec1c06ae70 0x7fec1c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:51.733 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.729+0000 7fec3ad9d640 1 --2- 172.21.15.169:0/4178811561 >> [v2:172.21.15.169:6800/3215791846,v1:172.21.15.169:6801/3215791846] conn(0x7fec1c070cd0 0x7fec1c0730f0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:51.733 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.730+0000 7fec39d9b640 1 --2- 172.21.15.169:0/4178811561 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fec1c06ae70 0x7fec1c06d330 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fec30001eb0 tx=0x7fec30001130 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:51.734 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.730+0000 7fec2affd640 1 -- 172.21.15.169:0/4178811561 <== osd.2 v2:172.21.15.169:6800/3215791846 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23595 (crc 0 0 0) 0x7fec1c0737a0 con 0x7fec1c070cd0 2023-10-26T01:44:51.744 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.742+0000 7fec3b59e640 1 -- 172.21.15.169:0/4178811561 --> [v2:172.21.15.169:6800/3215791846,v1:172.21.15.169:6801/3215791846] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fec3c0b0e90 con 0x7fec1c070cd0 2023-10-26T01:44:51.744 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.743+0000 7fec2affd640 1 -- 172.21.15.169:0/4178811561 <== osd.2 v2:172.21.15.169:6800/3215791846 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+632 (crc 0 0 0) 0x7fec3400d800 con 0x7fec1c070cd0 2023-10-26T01:44:51.745 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2023-10-26T01:44:51.745 INFO:teuthology.orchestra.run.smithi169.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_pg_delete_cost = '' (not observed, change may require restart) osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2023-10-26T01:44:51.745 INFO:teuthology.orchestra.run.smithi169.stdout:} 2023-10-26T01:44:51.746 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.743+0000 7fec3b59e640 1 -- 172.21.15.169:0/4178811561 >> [v2:172.21.15.169:6800/3215791846,v1:172.21.15.169:6801/3215791846] conn(0x7fec1c070cd0 msgr2=0x7fec1c0730f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:51.747 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.743+0000 7fec3b59e640 1 --2- 172.21.15.169:0/4178811561 >> [v2:172.21.15.169:6800/3215791846,v1:172.21.15.169:6801/3215791846] conn(0x7fec1c070cd0 0x7fec1c0730f0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:51.747 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.744+0000 7fec3b59e640 1 -- 172.21.15.169:0/4178811561 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fec1c06ae70 msgr2=0x7fec1c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:51.747 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.744+0000 7fec3b59e640 1 --2- 172.21.15.169:0/4178811561 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fec1c06ae70 0x7fec1c06d330 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fec30001eb0 tx=0x7fec30001130 comp rx=0 tx=0).stop 2023-10-26T01:44:51.747 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.744+0000 7fec3b59e640 1 -- 172.21.15.169:0/4178811561 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec24003970 msgr2=0x7fec3c152070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:51.747 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.744+0000 7fec3b59e640 1 --2- 172.21.15.169:0/4178811561 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec24003970 0x7fec3c152070 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7fec2c038040 tx=0x7fec2c0042f0 comp rx=0 tx=0).stop 2023-10-26T01:44:51.748 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.744+0000 7fec3ad9d640 1 -- 172.21.15.169:0/4178811561 reap_dead start 2023-10-26T01:44:51.748 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.744+0000 7fec3b59e640 1 -- 172.21.15.169:0/4178811561 shutdown_connections 2023-10-26T01:44:51.748 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.744+0000 7fec3b59e640 1 -- 172.21.15.169:0/4178811561 >> 172.21.15.169:0/4178811561 conn(0x7fec3c069b40 msgr2=0x7fec3c0b13a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:51.748 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.744+0000 7fec3b59e640 1 -- 172.21.15.169:0/4178811561 shutdown_connections 2023-10-26T01:44:51.748 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.744+0000 7fec3b59e640 1 -- 172.21.15.169:0/4178811561 wait complete. 2023-10-26T01:44:51.762 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-10-26T01:44:51.979 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.977+0000 7f3b040bb640 1 -- 172.21.15.169:0/3296212604 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ae8004d00 msgr2=0x7f3ae8005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:51.980 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.977+0000 7f3b040bb640 1 --2- 172.21.15.169:0/3296212604 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ae8004d00 0x7f3ae8005180 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7f3af4001d80 tx=0x7f3ae80057f0 comp rx=0 tx=0).stop 2023-10-26T01:44:51.980 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.977+0000 7f3b02658640 1 -- 172.21.15.169:0/3296212604 reap_dead start 2023-10-26T01:44:51.980 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.977+0000 7f3b040bb640 1 -- 172.21.15.169:0/3296212604 shutdown_connections 2023-10-26T01:44:51.980 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.977+0000 7f3b040bb640 1 -- 172.21.15.169:0/3296212604 >> 172.21.15.169:0/3296212604 conn(0x7f3afc069930 msgr2=0x7f3afc069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:51.981 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.977+0000 7f3b040bb640 1 -- 172.21.15.169:0/3296212604 shutdown_connections 2023-10-26T01:44:51.981 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.977+0000 7f3b040bb640 1 -- 172.21.15.169:0/3296212604 wait complete. 2023-10-26T01:44:51.982 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.979+0000 7f3b040bb640 1 Processor -- start 2023-10-26T01:44:51.982 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.980+0000 7f3b040bb640 1 -- start start 2023-10-26T01:44:51.984 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.981+0000 7f3b040bb640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3ae8004d00 0x7f3afc1ef2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:51.984 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.981+0000 7f3b040bb640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3afc15ba90 0x7f3afc15acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:51.984 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.981+0000 7f3b040bb640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3afc15bf10 0x7f3afc15b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:51.985 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.981+0000 7f3b040bb640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3afc06fae0 con 0x7f3afc15bf10 2023-10-26T01:44:51.985 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.981+0000 7f3b040bb640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3afc06eed0 con 0x7f3ae8004d00 2023-10-26T01:44:51.985 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.981+0000 7f3b040bb640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3afc06f1b0 con 0x7f3afc15ba90 2023-10-26T01:44:51.985 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.981+0000 7f3b02658640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3afc15bf10 0x7f3afc15b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:51.985 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.981+0000 7f3b02658640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3afc15bf10 0x7f3afc15b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:57274/0 (socket says 172.21.15.169:57274) 2023-10-26T01:44:51.986 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.981+0000 7f3b02658640 1 -- 172.21.15.169:0/487225099 learned_addr learned my addr 172.21.15.169:0/487225099 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:44:51.986 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.982+0000 7f3b02658640 1 -- 172.21.15.169:0/487225099 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3afc15ba90 msgr2=0x7f3afc15acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:51.986 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.982+0000 7f3b02658640 1 --2- 172.21.15.169:0/487225099 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3afc15ba90 0x7f3afc15acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:51.986 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.982+0000 7f3b02658640 1 -- 172.21.15.169:0/487225099 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3ae8004d00 msgr2=0x7f3afc1ef2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:51.986 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.982+0000 7f3b01e57640 1 --2- 172.21.15.169:0/487225099 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3ae8004d00 0x7f3afc1ef2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:51.987 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.982+0000 7f3b02658640 1 --2- 172.21.15.169:0/487225099 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3ae8004d00 0x7f3afc1ef2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:51.987 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.982+0000 7f3b02658640 1 -- 172.21.15.169:0/487225099 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3afc06f490 con 0x7f3afc15bf10 2023-10-26T01:44:51.987 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.982+0000 7f3b02658640 1 --2- 172.21.15.169:0/487225099 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3afc15bf10 0x7f3afc15b1f0 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7f3af4001ea0 tx=0x7f3af4003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:51.987 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.982+0000 7f3af27fc640 1 -- 172.21.15.169:0/487225099 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3af402ccd0 con 0x7f3afc15bf10 2023-10-26T01:44:51.987 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.982+0000 7f3af27fc640 1 -- 172.21.15.169:0/487225099 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3af40060b0 con 0x7f3afc15bf10 2023-10-26T01:44:51.988 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.982+0000 7f3af27fc640 1 -- 172.21.15.169:0/487225099 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3af4006480 con 0x7f3afc15bf10 2023-10-26T01:44:51.988 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.982+0000 7f3b040bb640 1 -- 172.21.15.169:0/487225099 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3afc06f770 con 0x7f3afc15bf10 2023-10-26T01:44:51.988 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.983+0000 7f3b040bb640 1 -- 172.21.15.169:0/487225099 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3afc149bb0 con 0x7f3afc15bf10 2023-10-26T01:44:51.989 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.984+0000 7f3b040bb640 1 -- 172.21.15.169:0/487225099 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3afc04db10 con 0x7f3afc15bf10 2023-10-26T01:44:51.989 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.985+0000 7f3af27fc640 1 -- 172.21.15.169:0/487225099 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f3af40065e0 con 0x7f3afc15bf10 2023-10-26T01:44:51.990 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.985+0000 7f3af27fc640 1 --2- 172.21.15.169:0/487225099 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3ae006ae40 0x7f3ae006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:51.990 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.986+0000 7f3af27fc640 1 -- 172.21.15.169:0/487225099 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f3af40aa380 con 0x7f3afc15bf10 2023-10-26T01:44:51.990 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.986+0000 7f3b01e57640 1 --2- 172.21.15.169:0/487225099 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3ae006ae40 0x7f3ae006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:51.990 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.987+0000 7f3b01e57640 1 --2- 172.21.15.169:0/487225099 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3ae006ae40 0x7f3ae006d300 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f3af80069f0 tx=0x7f3af80014f0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:51.991 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:51.990+0000 7f3af27fc640 1 -- 172.21.15.169:0/487225099 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3af4075730 con 0x7f3afc15bf10 2023-10-26T01:44:52.183 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.181+0000 7f3b040bb640 1 -- 172.21.15.169:0/487225099 --> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f3afc0b1340 con 0x7f3ae006ae40 2023-10-26T01:44:52.184 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.182+0000 7f3af27fc640 1 -- 172.21.15.169:0/487225099 <== mgr.4112 v2:172.21.15.170:6832/1187156918 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f3af800c050 con 0x7f3ae006ae40 2023-10-26T01:44:52.184 INFO:teuthology.orchestra.run.smithi169.stderr:instructed osd(s) 2 to deep-scrub 2023-10-26T01:44:52.187 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.184+0000 7f3b040bb640 1 -- 172.21.15.169:0/487225099 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3ae006ae40 msgr2=0x7f3ae006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:52.187 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.184+0000 7f3b040bb640 1 --2- 172.21.15.169:0/487225099 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3ae006ae40 0x7f3ae006d300 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f3af80069f0 tx=0x7f3af80014f0 comp rx=0 tx=0).stop 2023-10-26T01:44:52.187 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.185+0000 7f3b040bb640 1 -- 172.21.15.169:0/487225099 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3afc15bf10 msgr2=0x7f3afc15b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:52.188 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.185+0000 7f3b040bb640 1 --2- 172.21.15.169:0/487225099 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3afc15bf10 0x7f3afc15b1f0 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7f3af4001ea0 tx=0x7f3af4003ba0 comp rx=0 tx=0).stop 2023-10-26T01:44:52.188 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.185+0000 7f3b040bb640 1 -- 172.21.15.169:0/487225099 shutdown_connections 2023-10-26T01:44:52.188 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.185+0000 7f3b040bb640 1 --2- 172.21.15.169:0/487225099 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f3ae006ae40 0x7f3ae006d300 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:52.188 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.185+0000 7f3b040bb640 1 --2- 172.21.15.169:0/487225099 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3afc15bf10 0x7f3afc15b1f0 unknown :-1 s=CLOSED pgs=1994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:52.189 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.185+0000 7f3b040bb640 1 --2- 172.21.15.169:0/487225099 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3afc15ba90 0x7f3afc15acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:52.189 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.185+0000 7f3b040bb640 1 --2- 172.21.15.169:0/487225099 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3ae8004d00 0x7f3afc1ef2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:52.189 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.185+0000 7f3b040bb640 1 -- 172.21.15.169:0/487225099 >> 172.21.15.169:0/487225099 conn(0x7f3afc069930 msgr2=0x7f3afc154820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:52.189 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.185+0000 7f3b040bb640 1 -- 172.21.15.169:0/487225099 shutdown_connections 2023-10-26T01:44:52.190 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.185+0000 7f3b040bb640 1 -- 172.21.15.169:0/487225099 wait complete. 2023-10-26T01:44:52.205 INFO:tasks.ceph:Scrubbing osd.3 2023-10-26T01:44:52.206 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-10-26T01:44:52.426 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.424+0000 7f235db4c640 1 -- 172.21.15.169:0/1900015442 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f235806cdb0 msgr2=0x7f235815b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:52.426 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.424+0000 7f235db4c640 1 --2- 172.21.15.169:0/1900015442 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f235806cdb0 0x7f235815b2c0 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7f235806ccb0 tx=0x7f234002bd10 comp rx=0 tx=0).stop 2023-10-26T01:44:52.427 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.424+0000 7f235db4c640 1 -- 172.21.15.169:0/1900015442 shutdown_connections 2023-10-26T01:44:52.427 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.424+0000 7f235db4c640 1 --2- 172.21.15.169:0/1900015442 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f235806cdb0 0x7f235815b2c0 unknown :-1 s=CLOSED pgs=1995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:52.427 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.424+0000 7f235db4c640 1 --2- 172.21.15.169:0/1900015442 >> v2:172.21.15.170:3300/0 conn(0x7f23580aab40 0x7f23580ac6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:52.427 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.424+0000 7f235db4c640 1 -- 172.21.15.169:0/1900015442 >> 172.21.15.169:0/1900015442 conn(0x7f23580aeee0 msgr2=0x7f23580af2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:52.428 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.424+0000 7f235db4c640 1 -- 172.21.15.169:0/1900015442 shutdown_connections 2023-10-26T01:44:52.428 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.424+0000 7f235db4c640 1 -- 172.21.15.169:0/1900015442 wait complete. 2023-10-26T01:44:52.428 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.426+0000 7f235db4c640 1 Processor -- start 2023-10-26T01:44:52.429 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.426+0000 7f235db4c640 1 -- start start 2023-10-26T01:44:52.431 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.427+0000 7f235db4c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f235806cdb0 0x7f2358150b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:52.431 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.427+0000 7f235db4c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f235815b800 0x7f23580ad090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:52.431 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.427+0000 7f235db4c640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f23580ad5d0 0x7f23581532a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:52.432 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.427+0000 7f235db4c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23580c6700 con 0x7f235815b800 2023-10-26T01:44:52.432 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.427+0000 7f235db4c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23580c69e0 con 0x7f235806cdb0 2023-10-26T01:44:52.432 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.427+0000 7f235db4c640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23580c6cc0 con 0x7f23580ad5d0 2023-10-26T01:44:52.432 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.428+0000 7f234ffff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f235815b800 0x7f23580ad090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:52.432 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.428+0000 7f234ffff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f235815b800 0x7f23580ad090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:57294/0 (socket says 172.21.15.169:57294) 2023-10-26T01:44:52.433 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.428+0000 7f234ffff640 1 -- 172.21.15.169:0/1617047553 learned_addr learned my addr 172.21.15.169:0/1617047553 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:44:52.433 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.428+0000 7f23577fe640 1 --2- 172.21.15.169:0/1617047553 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f235806cdb0 0x7f2358150b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:52.433 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.428+0000 7f2357fff640 1 --2- 172.21.15.169:0/1617047553 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f23580ad5d0 0x7f23581532a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:52.433 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.428+0000 7f234ffff640 1 -- 172.21.15.169:0/1617047553 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f23580ad5d0 msgr2=0x7f23581532a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:52.434 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.428+0000 7f234ffff640 1 --2- 172.21.15.169:0/1617047553 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f23580ad5d0 0x7f23581532a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:52.434 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.428+0000 7f234ffff640 1 -- 172.21.15.169:0/1617047553 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f235806cdb0 msgr2=0x7f2358150b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:52.434 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.428+0000 7f234ffff640 1 --2- 172.21.15.169:0/1617047553 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f235806cdb0 0x7f2358150b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:52.434 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.428+0000 7f234ffff640 1 -- 172.21.15.169:0/1617047553 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2340009560 con 0x7f235815b800 2023-10-26T01:44:52.434 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.429+0000 7f234ffff640 1 --2- 172.21.15.169:0/1617047553 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f235815b800 0x7f23580ad090 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7f2340009850 tx=0x7f234002dce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:52.435 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.429+0000 7f2354ff9640 1 -- 172.21.15.169:0/1617047553 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2340005650 con 0x7f235815b800 2023-10-26T01:44:52.435 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.429+0000 7f2354ff9640 1 -- 172.21.15.169:0/1617047553 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f23400038f0 con 0x7f235815b800 2023-10-26T01:44:52.435 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.429+0000 7f2354ff9640 1 -- 172.21.15.169:0/1617047553 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2340003ce0 con 0x7f235815b800 2023-10-26T01:44:52.435 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.429+0000 7f235db4c640 1 -- 172.21.15.169:0/1617047553 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23580c6fa0 con 0x7f235815b800 2023-10-26T01:44:52.436 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.429+0000 7f235db4c640 1 -- 172.21.15.169:0/1617047553 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f235814a610 con 0x7f235815b800 2023-10-26T01:44:52.437 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.430+0000 7f235db4c640 1 -- 172.21.15.169:0/1617047553 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f23580c6b50 con 0x7f235815b800 2023-10-26T01:44:52.437 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.431+0000 7f2354ff9640 1 -- 172.21.15.169:0/1617047553 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f234003e450 con 0x7f235815b800 2023-10-26T01:44:52.438 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.432+0000 7f2354ff9640 1 --2- 172.21.15.169:0/1617047553 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f233806ae70 0x7f233806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:52.438 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.432+0000 7f2354ff9640 1 -- 172.21.15.169:0/1617047553 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f23400abc40 con 0x7f235815b800 2023-10-26T01:44:52.438 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.433+0000 7f2354ff9640 1 --2- 172.21.15.169:0/1617047553 >> [v2:172.21.15.169:6824/2256701226,v1:172.21.15.169:6825/2256701226] conn(0x7f2338070cd0 0x7f23380730f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:52.438 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.433+0000 7f2354ff9640 1 -- 172.21.15.169:0/1617047553 --> [v2:172.21.15.169:6824/2256701226,v1:172.21.15.169:6825/2256701226] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f23380737a0 con 0x7f2338070cd0 2023-10-26T01:44:52.439 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.433+0000 7f2354ff9640 1 -- 172.21.15.169:0/1617047553 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_get_version_reply(handle=1 version=189) v2 ==== 24+0+0 (secure 0 0 0) 0x7f23400b02c0 con 0x7f235815b800 2023-10-26T01:44:52.439 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.433+0000 7f23577fe640 1 --2- 172.21.15.169:0/1617047553 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f233806ae70 0x7f233806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:52.439 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.433+0000 7f2357fff640 1 --2- 172.21.15.169:0/1617047553 >> [v2:172.21.15.169:6824/2256701226,v1:172.21.15.169:6825/2256701226] conn(0x7f2338070cd0 0x7f23380730f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:52.439 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.433+0000 7f2357fff640 1 --2- 172.21.15.169:0/1617047553 >> [v2:172.21.15.169:6824/2256701226,v1:172.21.15.169:6825/2256701226] conn(0x7f2338070cd0 0x7f23380730f0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:52.440 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.434+0000 7f23577fe640 1 --2- 172.21.15.169:0/1617047553 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f233806ae70 0x7f233806d330 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f2348005e10 tx=0x7f2348005da0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:52.440 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.434+0000 7f2354ff9640 1 -- 172.21.15.169:0/1617047553 <== osd.3 v2:172.21.15.169:6824/2256701226 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23595 (crc 0 0 0) 0x7f23380737a0 con 0x7f2338070cd0 2023-10-26T01:44:52.448 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.447+0000 7f235db4c640 1 -- 172.21.15.169:0/1617047553 --> [v2:172.21.15.169:6824/2256701226,v1:172.21.15.169:6825/2256701226] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f23580b19e0 con 0x7f2338070cd0 2023-10-26T01:44:52.449 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.447+0000 7f2354ff9640 1 -- 172.21.15.169:0/1617047553 <== osd.3 v2:172.21.15.169:6824/2256701226 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+632 (crc 0 0 0) 0x7f2350009b70 con 0x7f2338070cd0 2023-10-26T01:44:52.449 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2023-10-26T01:44:52.450 INFO:teuthology.orchestra.run.smithi169.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_pg_delete_cost = '' (not observed, change may require restart) osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2023-10-26T01:44:52.450 INFO:teuthology.orchestra.run.smithi169.stdout:} 2023-10-26T01:44:52.451 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.448+0000 7f235db4c640 1 -- 172.21.15.169:0/1617047553 >> [v2:172.21.15.169:6824/2256701226,v1:172.21.15.169:6825/2256701226] conn(0x7f2338070cd0 msgr2=0x7f23380730f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:52.451 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.448+0000 7f235db4c640 1 --2- 172.21.15.169:0/1617047553 >> [v2:172.21.15.169:6824/2256701226,v1:172.21.15.169:6825/2256701226] conn(0x7f2338070cd0 0x7f23380730f0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:52.452 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.448+0000 7f235db4c640 1 -- 172.21.15.169:0/1617047553 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f233806ae70 msgr2=0x7f233806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:52.452 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.448+0000 7f235db4c640 1 --2- 172.21.15.169:0/1617047553 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f233806ae70 0x7f233806d330 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f2348005e10 tx=0x7f2348005da0 comp rx=0 tx=0).stop 2023-10-26T01:44:52.452 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.448+0000 7f235db4c640 1 -- 172.21.15.169:0/1617047553 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f235815b800 msgr2=0x7f23580ad090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:52.452 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.448+0000 7f235db4c640 1 --2- 172.21.15.169:0/1617047553 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f235815b800 0x7f23580ad090 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7f2340009850 tx=0x7f234002dce0 comp rx=0 tx=0).stop 2023-10-26T01:44:52.453 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.448+0000 7f2357fff640 1 -- 172.21.15.169:0/1617047553 reap_dead start 2023-10-26T01:44:52.453 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.448+0000 7f235db4c640 1 -- 172.21.15.169:0/1617047553 shutdown_connections 2023-10-26T01:44:52.453 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.448+0000 7f235db4c640 1 -- 172.21.15.169:0/1617047553 >> 172.21.15.169:0/1617047553 conn(0x7f23580aeee0 msgr2=0x7f2358071240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:52.453 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.448+0000 7f235db4c640 1 -- 172.21.15.169:0/1617047553 shutdown_connections 2023-10-26T01:44:52.453 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.448+0000 7f235db4c640 1 -- 172.21.15.169:0/1617047553 wait complete. 2023-10-26T01:44:52.468 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-10-26T01:44:52.689 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.687+0000 7f9acc4d7640 1 -- 172.21.15.169:0/2139623188 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ac415b620 msgr2=0x7f9ac40cd710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:52.689 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.687+0000 7f9acc4d7640 1 --2- 172.21.15.169:0/2139623188 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ac415b620 0x7f9ac40cd710 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7f9ac0009830 tx=0x7f9ac002bd50 comp rx=0 tx=0).stop 2023-10-26T01:44:52.690 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.687+0000 7f9acc4d7640 1 -- 172.21.15.169:0/2139623188 shutdown_connections 2023-10-26T01:44:52.690 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.687+0000 7f9acc4d7640 1 --2- 172.21.15.169:0/2139623188 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9ac40c9d00 0x7f9ac40cdc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:52.690 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.687+0000 7f9acc4d7640 1 --2- 172.21.15.169:0/2139623188 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ac415b620 0x7f9ac40cd710 unknown :-1 s=CLOSED pgs=1998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:52.690 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.687+0000 7f9acc4d7640 1 -- 172.21.15.169:0/2139623188 >> 172.21.15.169:0/2139623188 conn(0x7f9ac4069930 msgr2=0x7f9ac4069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:52.690 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.687+0000 7f9acc4d7640 1 -- 172.21.15.169:0/2139623188 shutdown_connections 2023-10-26T01:44:52.691 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.687+0000 7f9acc4d7640 1 -- 172.21.15.169:0/2139623188 wait complete. 2023-10-26T01:44:52.692 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.689+0000 7f9acc4d7640 1 Processor -- start 2023-10-26T01:44:52.693 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.689+0000 7f9acc4d7640 1 -- start start 2023-10-26T01:44:52.693 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.690+0000 7f9acc4d7640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9ac40ac320 0x7f9ac40c8f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:52.693 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.690+0000 7f9acc4d7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9ac40c9d00 0x7f9ac415e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:52.693 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.690+0000 7f9acc4d7640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ac415b620 0x7f9ac40c71c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:52.693 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.690+0000 7f9acc4d7640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac40af140 con 0x7f9ac415b620 2023-10-26T01:44:52.694 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.690+0000 7f9acc4d7640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac40c95e0 con 0x7f9ac40c9d00 2023-10-26T01:44:52.694 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.690+0000 7f9acc4d7640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac40c7b70 con 0x7f9ac40ac320 2023-10-26T01:44:52.694 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.690+0000 7f9acaa74640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ac415b620 0x7f9ac40c71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:52.694 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.690+0000 7f9acaa74640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ac415b620 0x7f9ac40c71c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:57310/0 (socket says 172.21.15.169:57310) 2023-10-26T01:44:52.694 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.690+0000 7f9acaa74640 1 -- 172.21.15.169:0/119114265 learned_addr learned my addr 172.21.15.169:0/119114265 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:44:52.695 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.691+0000 7f9aca273640 1 --2- 172.21.15.169:0/119114265 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9ac40ac320 0x7f9ac40c8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:52.695 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.691+0000 7f9ac9a72640 1 --2- 172.21.15.169:0/119114265 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9ac40c9d00 0x7f9ac415e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:52.695 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.691+0000 7f9acaa74640 1 -- 172.21.15.169:0/119114265 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9ac40ac320 msgr2=0x7f9ac40c8f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:52.695 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.691+0000 7f9acaa74640 1 --2- 172.21.15.169:0/119114265 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9ac40ac320 0x7f9ac40c8f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:52.695 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.691+0000 7f9acaa74640 1 -- 172.21.15.169:0/119114265 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9ac40c9d00 msgr2=0x7f9ac415e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:52.696 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.691+0000 7f9acaa74640 1 --2- 172.21.15.169:0/119114265 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9ac40c9d00 0x7f9ac415e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:52.696 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.691+0000 7f9acaa74640 1 -- 172.21.15.169:0/119114265 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ac0009490 con 0x7f9ac415b620 2023-10-26T01:44:52.696 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.691+0000 7f9acaa74640 1 --2- 172.21.15.169:0/119114265 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ac415b620 0x7f9ac40c71c0 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7f9abc001560 tx=0x7f9abc00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:52.697 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.691+0000 7f9abaffd640 1 -- 172.21.15.169:0/119114265 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9abc003da0 con 0x7f9ac415b620 2023-10-26T01:44:52.697 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.691+0000 7f9abaffd640 1 -- 172.21.15.169:0/119114265 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9abc005070 con 0x7f9ac415b620 2023-10-26T01:44:52.698 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.691+0000 7f9abaffd640 1 -- 172.21.15.169:0/119114265 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9abc005460 con 0x7f9ac415b620 2023-10-26T01:44:52.698 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.691+0000 7f9acc4d7640 1 -- 172.21.15.169:0/119114265 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ac40c7e50 con 0x7f9ac415b620 2023-10-26T01:44:52.698 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.692+0000 7f9acc4d7640 1 -- 172.21.15.169:0/119114265 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ac40c8410 con 0x7f9ac415b620 2023-10-26T01:44:52.698 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.693+0000 7f9acc4d7640 1 -- 172.21.15.169:0/119114265 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ac404db10 con 0x7f9ac415b620 2023-10-26T01:44:52.698 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.694+0000 7f9abaffd640 1 -- 172.21.15.169:0/119114265 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f9abc0055c0 con 0x7f9ac415b620 2023-10-26T01:44:52.699 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.695+0000 7f9abaffd640 1 --2- 172.21.15.169:0/119114265 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9aa806ae40 0x7f9aa806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:52.699 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.695+0000 7f9abaffd640 1 -- 172.21.15.169:0/119114265 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f9abc088db0 con 0x7f9ac415b620 2023-10-26T01:44:52.699 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.695+0000 7f9aca273640 1 --2- 172.21.15.169:0/119114265 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9aa806ae40 0x7f9aa806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:52.699 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.696+0000 7f9aca273640 1 --2- 172.21.15.169:0/119114265 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9aa806ae40 0x7f9aa806d300 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f9ac0036000 tx=0x7f9ac0004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:52.700 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.699+0000 7f9abaffd640 1 -- 172.21.15.169:0/119114265 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9abc0541f0 con 0x7f9ac415b620 2023-10-26T01:44:52.890 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.888+0000 7f9acc4d7640 1 -- 172.21.15.169:0/119114265 --> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f9ac40b2410 con 0x7f9aa806ae40 2023-10-26T01:44:52.914 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.889+0000 7f9abaffd640 1 -- 172.21.15.169:0/119114265 <== mgr.4112 v2:172.21.15.170:6832/1187156918 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f9ac40b2410 con 0x7f9aa806ae40 2023-10-26T01:44:52.914 INFO:teuthology.orchestra.run.smithi169.stderr:instructed osd(s) 3 to deep-scrub 2023-10-26T01:44:52.915 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.891+0000 7f9acc4d7640 1 -- 172.21.15.169:0/119114265 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9aa806ae40 msgr2=0x7f9aa806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:52.915 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.891+0000 7f9acc4d7640 1 --2- 172.21.15.169:0/119114265 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9aa806ae40 0x7f9aa806d300 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f9ac0036000 tx=0x7f9ac0004730 comp rx=0 tx=0).stop 2023-10-26T01:44:52.915 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.891+0000 7f9acc4d7640 1 -- 172.21.15.169:0/119114265 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ac415b620 msgr2=0x7f9ac40c71c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:52.915 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.891+0000 7f9acc4d7640 1 --2- 172.21.15.169:0/119114265 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ac415b620 0x7f9ac40c71c0 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7f9abc001560 tx=0x7f9abc00cc70 comp rx=0 tx=0).stop 2023-10-26T01:44:52.916 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.891+0000 7f9acc4d7640 1 -- 172.21.15.169:0/119114265 shutdown_connections 2023-10-26T01:44:52.916 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.891+0000 7f9acc4d7640 1 --2- 172.21.15.169:0/119114265 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f9aa806ae40 0x7f9aa806d300 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:52.916 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.891+0000 7f9acc4d7640 1 --2- 172.21.15.169:0/119114265 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ac415b620 0x7f9ac40c71c0 unknown :-1 s=CLOSED pgs=1999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:52.916 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.891+0000 7f9acc4d7640 1 --2- 172.21.15.169:0/119114265 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9ac40c9d00 0x7f9ac415e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:52.916 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.891+0000 7f9acc4d7640 1 --2- 172.21.15.169:0/119114265 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9ac40ac320 0x7f9ac40c8f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:52.917 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.891+0000 7f9acc4d7640 1 -- 172.21.15.169:0/119114265 >> 172.21.15.169:0/119114265 conn(0x7f9ac4069930 msgr2=0x7f9ac4150f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:52.917 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.891+0000 7f9acc4d7640 1 -- 172.21.15.169:0/119114265 shutdown_connections 2023-10-26T01:44:52.917 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:52.892+0000 7f9acc4d7640 1 -- 172.21.15.169:0/119114265 wait complete. 2023-10-26T01:44:52.917 INFO:tasks.ceph:Scrubbing osd.4 2023-10-26T01:44:52.918 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-10-26T01:44:53.148 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.146+0000 7fbfda368640 1 -- 172.21.15.169:0/760837656 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbfd40aab40 msgr2=0x7fbfd40ac6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:53.148 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.146+0000 7fbfda368640 1 --2- 172.21.15.169:0/760837656 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbfd40aab40 0x7fbfd40ac6c0 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7fbfd415c3e0 tx=0x7fbfc402bd10 comp rx=0 tx=0).stop 2023-10-26T01:44:53.148 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.146+0000 7fbfda368640 1 -- 172.21.15.169:0/760837656 shutdown_connections 2023-10-26T01:44:53.149 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.146+0000 7fbfda368640 1 --2- 172.21.15.169:0/760837656 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbfd4157ce0 0x7fbfd40b19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:53.149 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.146+0000 7fbfda368640 1 --2- 172.21.15.169:0/760837656 >> v2:172.21.15.170:3300/0 conn(0x7fbfd406cdb0 0x7fbfd41577a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:53.149 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.146+0000 7fbfda368640 1 --2- 172.21.15.169:0/760837656 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbfd40aab40 0x7fbfd40ac6c0 unknown :-1 s=CLOSED pgs=2000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:53.150 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.146+0000 7fbfda368640 1 -- 172.21.15.169:0/760837656 >> 172.21.15.169:0/760837656 conn(0x7fbfd40aeee0 msgr2=0x7fbfd40af2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:53.150 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.146+0000 7fbfda368640 1 -- 172.21.15.169:0/760837656 shutdown_connections 2023-10-26T01:44:53.150 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.146+0000 7fbfda368640 1 -- 172.21.15.169:0/760837656 wait complete. 2023-10-26T01:44:53.151 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.148+0000 7fbfda368640 1 Processor -- start 2023-10-26T01:44:53.151 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.148+0000 7fbfda368640 1 -- start start 2023-10-26T01:44:53.154 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.150+0000 7fbfda368640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbfd406cdb0 0x7fbfd4159f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:53.154 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.150+0000 7fbfda368640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbfd4157ce0 0x7fbfd4154430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:53.154 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.150+0000 7fbfda368640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbfd413d910 0x7fbfd4152630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:53.154 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.150+0000 7fbfda368640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd414cc30 con 0x7fbfd4157ce0 2023-10-26T01:44:53.155 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.150+0000 7fbfda368640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd4153210 con 0x7fbfd406cdb0 2023-10-26T01:44:53.155 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.150+0000 7fbfda368640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd406f010 con 0x7fbfd413d910 2023-10-26T01:44:53.155 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.150+0000 7fbfd37fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbfd4157ce0 0x7fbfd4154430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:53.156 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.150+0000 7fbfd37fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbfd4157ce0 0x7fbfd4154430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:57340/0 (socket says 172.21.15.169:57340) 2023-10-26T01:44:53.156 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.150+0000 7fbfd37fe640 1 -- 172.21.15.169:0/3578484994 learned_addr learned my addr 172.21.15.169:0/3578484994 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:44:53.156 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.150+0000 7fbfd3fff640 1 --2- 172.21.15.169:0/3578484994 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbfd406cdb0 0x7fbfd4159f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:53.156 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.151+0000 7fbfd37fe640 1 -- 172.21.15.169:0/3578484994 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbfd413d910 msgr2=0x7fbfd4152630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:53.157 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.151+0000 7fbfd8905640 1 --2- 172.21.15.169:0/3578484994 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbfd413d910 0x7fbfd4152630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:53.157 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.151+0000 7fbfd37fe640 1 --2- 172.21.15.169:0/3578484994 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbfd413d910 0x7fbfd4152630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:53.157 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.151+0000 7fbfd37fe640 1 -- 172.21.15.169:0/3578484994 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbfd406cdb0 msgr2=0x7fbfd4159f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:53.157 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.151+0000 7fbfd37fe640 1 --2- 172.21.15.169:0/3578484994 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbfd406cdb0 0x7fbfd4159f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:53.157 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.151+0000 7fbfd37fe640 1 -- 172.21.15.169:0/3578484994 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfc4009560 con 0x7fbfd4157ce0 2023-10-26T01:44:53.158 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.151+0000 7fbfd3fff640 1 --2- 172.21.15.169:0/3578484994 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbfd406cdb0 0x7fbfd4159f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:44:53.158 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.151+0000 7fbfd8905640 1 --2- 172.21.15.169:0/3578484994 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbfd413d910 0x7fbfd4152630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:44:53.158 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.151+0000 7fbfd37fe640 1 --2- 172.21.15.169:0/3578484994 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbfd4157ce0 0x7fbfd4154430 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7fbfc800c970 tx=0x7fbfc800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:53.158 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.151+0000 7fbfd0ff9640 1 -- 172.21.15.169:0/3578484994 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc8007bb0 con 0x7fbfd4157ce0 2023-10-26T01:44:53.158 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.151+0000 7fbfd0ff9640 1 -- 172.21.15.169:0/3578484994 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fbfc8007e00 con 0x7fbfd4157ce0 2023-10-26T01:44:53.159 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.151+0000 7fbfda368640 1 -- 172.21.15.169:0/3578484994 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfd406ecc0 con 0x7fbfd4157ce0 2023-10-26T01:44:53.159 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.151+0000 7fbfd0ff9640 1 -- 172.21.15.169:0/3578484994 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc800f6b0 con 0x7fbfd4157ce0 2023-10-26T01:44:53.159 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.152+0000 7fbfda368640 1 -- 172.21.15.169:0/3578484994 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfd4153740 con 0x7fbfd4157ce0 2023-10-26T01:44:53.161 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.153+0000 7fbfda368640 1 -- 172.21.15.169:0/3578484994 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbfd404da70 con 0x7fbfd4157ce0 2023-10-26T01:44:53.161 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.154+0000 7fbfd0ff9640 1 -- 172.21.15.169:0/3578484994 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fbfc800f810 con 0x7fbfd4157ce0 2023-10-26T01:44:53.161 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.154+0000 7fbfd0ff9640 1 --2- 172.21.15.169:0/3578484994 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbfc006ae70 0x7fbfc006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:53.161 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.155+0000 7fbfd0ff9640 1 -- 172.21.15.169:0/3578484994 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fbfc8089bd0 con 0x7fbfd4157ce0 2023-10-26T01:44:53.161 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.155+0000 7fbfd0ff9640 1 --2- 172.21.15.169:0/3578484994 >> [v2:172.21.15.170:6816/1082279874,v1:172.21.15.170:6817/1082279874] conn(0x7fbfc0070cd0 0x7fbfc00730f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:53.162 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.155+0000 7fbfd0ff9640 1 -- 172.21.15.169:0/3578484994 --> [v2:172.21.15.170:6816/1082279874,v1:172.21.15.170:6817/1082279874] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbfc00737a0 con 0x7fbfc0070cd0 2023-10-26T01:44:53.162 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.155+0000 7fbfd0ff9640 1 -- 172.21.15.169:0/3578484994 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_get_version_reply(handle=1 version=189) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbfc808c290 con 0x7fbfd4157ce0 2023-10-26T01:44:53.162 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.155+0000 7fbfd3fff640 1 --2- 172.21.15.169:0/3578484994 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbfc006ae70 0x7fbfc006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:53.162 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.156+0000 7fbfd8905640 1 --2- 172.21.15.169:0/3578484994 >> [v2:172.21.15.170:6816/1082279874,v1:172.21.15.170:6817/1082279874] conn(0x7fbfc0070cd0 0x7fbfc00730f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:53.162 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.157+0000 7fbfd3fff640 1 --2- 172.21.15.169:0/3578484994 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbfc006ae70 0x7fbfc006d330 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fbfc4035000 tx=0x7fbfc4004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:53.163 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.157+0000 7fbfd8905640 1 --2- 172.21.15.169:0/3578484994 >> [v2:172.21.15.170:6816/1082279874,v1:172.21.15.170:6817/1082279874] conn(0x7fbfc0070cd0 0x7fbfc00730f0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:53.163 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.158+0000 7fbfd0ff9640 1 -- 172.21.15.169:0/3578484994 <== osd.4 v2:172.21.15.170:6816/1082279874 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23595 (crc 0 0 0) 0x7fbfc00737a0 con 0x7fbfc0070cd0 2023-10-26T01:44:53.172 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.171+0000 7fbfda368640 1 -- 172.21.15.169:0/3578484994 --> [v2:172.21.15.170:6816/1082279874,v1:172.21.15.170:6817/1082279874] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fbfd4066af0 con 0x7fbfc0070cd0 2023-10-26T01:44:53.173 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.172+0000 7fbfd0ff9640 1 -- 172.21.15.169:0/3578484994 <== osd.4 v2:172.21.15.170:6816/1082279874 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+632 (crc 0 0 0) 0x7fbfd4066af0 con 0x7fbfc0070cd0 2023-10-26T01:44:53.174 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2023-10-26T01:44:53.174 INFO:teuthology.orchestra.run.smithi169.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_pg_delete_cost = '' (not observed, change may require restart) osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2023-10-26T01:44:53.175 INFO:teuthology.orchestra.run.smithi169.stdout:} 2023-10-26T01:44:53.176 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.172+0000 7fbfda368640 1 -- 172.21.15.169:0/3578484994 >> [v2:172.21.15.170:6816/1082279874,v1:172.21.15.170:6817/1082279874] conn(0x7fbfc0070cd0 msgr2=0x7fbfc00730f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:53.176 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.172+0000 7fbfda368640 1 --2- 172.21.15.169:0/3578484994 >> [v2:172.21.15.170:6816/1082279874,v1:172.21.15.170:6817/1082279874] conn(0x7fbfc0070cd0 0x7fbfc00730f0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:53.176 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.173+0000 7fbfda368640 1 -- 172.21.15.169:0/3578484994 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbfc006ae70 msgr2=0x7fbfc006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:53.176 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.173+0000 7fbfda368640 1 --2- 172.21.15.169:0/3578484994 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbfc006ae70 0x7fbfc006d330 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fbfc4035000 tx=0x7fbfc4004730 comp rx=0 tx=0).stop 2023-10-26T01:44:53.177 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.173+0000 7fbfda368640 1 -- 172.21.15.169:0/3578484994 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbfd4157ce0 msgr2=0x7fbfd4154430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:53.177 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.173+0000 7fbfda368640 1 --2- 172.21.15.169:0/3578484994 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbfd4157ce0 0x7fbfd4154430 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7fbfc800c970 tx=0x7fbfc800ce30 comp rx=0 tx=0).stop 2023-10-26T01:44:53.177 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.173+0000 7fbfd8905640 1 -- 172.21.15.169:0/3578484994 reap_dead start 2023-10-26T01:44:53.177 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.173+0000 7fbfda368640 1 -- 172.21.15.169:0/3578484994 shutdown_connections 2023-10-26T01:44:53.177 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.173+0000 7fbfda368640 1 -- 172.21.15.169:0/3578484994 >> 172.21.15.169:0/3578484994 conn(0x7fbfd40aeee0 msgr2=0x7fbfd40ae370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:53.178 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.173+0000 7fbfda368640 1 -- 172.21.15.169:0/3578484994 shutdown_connections 2023-10-26T01:44:53.178 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.173+0000 7fbfda368640 1 -- 172.21.15.169:0/3578484994 wait complete. 2023-10-26T01:44:53.191 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-10-26T01:44:53.409 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.407+0000 7f6b9cb2c640 1 -- 172.21.15.169:0/4208500290 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b9815bbb0 msgr2=0x7f6b98155360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:53.409 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.407+0000 7f6b9cb2c640 1 --2- 172.21.15.169:0/4208500290 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b9815bbb0 0x7f6b98155360 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7f6b80009830 tx=0x7f6b8002bcd0 comp rx=0 tx=0).stop 2023-10-26T01:44:53.409 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.407+0000 7f6b9cb2c640 1 -- 172.21.15.169:0/4208500290 shutdown_connections 2023-10-26T01:44:53.410 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.407+0000 7f6b9cb2c640 1 --2- 172.21.15.169:0/4208500290 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6b980b05a0 0x7f6b980b09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:53.410 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.407+0000 7f6b9cb2c640 1 --2- 172.21.15.169:0/4208500290 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b9815bbb0 0x7f6b98155360 unknown :-1 s=CLOSED pgs=2003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:53.410 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.407+0000 7f6b9cb2c640 1 -- 172.21.15.169:0/4208500290 >> 172.21.15.169:0/4208500290 conn(0x7f6b98069930 msgr2=0x7f6b98069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:53.410 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.407+0000 7f6b9cb2c640 1 -- 172.21.15.169:0/4208500290 shutdown_connections 2023-10-26T01:44:53.411 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.407+0000 7f6b9cb2c640 1 -- 172.21.15.169:0/4208500290 wait complete. 2023-10-26T01:44:53.411 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.409+0000 7f6b9cb2c640 1 Processor -- start 2023-10-26T01:44:53.411 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.409+0000 7f6b9cb2c640 1 -- start start 2023-10-26T01:44:53.413 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.410+0000 7f6b9cb2c640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6b9806ce00 0x7f6b9806f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:53.413 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.410+0000 7f6b9cb2c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b980b05a0 0x7f6b9815a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:53.414 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.410+0000 7f6b9cb2c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b9815bbb0 0x7f6b9814c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:53.414 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.410+0000 7f6b9cb2c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b98151e80 con 0x7f6b980b05a0 2023-10-26T01:44:53.414 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.410+0000 7f6b9cb2c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b9806fd20 con 0x7f6b9815bbb0 2023-10-26T01:44:53.414 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.410+0000 7f6b9cb2c640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b98070000 con 0x7f6b9806ce00 2023-10-26T01:44:53.414 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.410+0000 7f6b8dd9b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b980b05a0 0x7f6b9815a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:53.415 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.410+0000 7f6b8dd9b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b980b05a0 0x7f6b9815a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:57350/0 (socket says 172.21.15.169:57350) 2023-10-26T01:44:53.415 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.410+0000 7f6b8dd9b640 1 -- 172.21.15.169:0/1347196418 learned_addr learned my addr 172.21.15.169:0/1347196418 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:44:53.415 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.411+0000 7f6b8dd9b640 1 -- 172.21.15.169:0/1347196418 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6b9806ce00 msgr2=0x7f6b9806f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:53.415 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.411+0000 7f6b8dd9b640 1 --2- 172.21.15.169:0/1347196418 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6b9806ce00 0x7f6b9806f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:53.415 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.411+0000 7f6b8dd9b640 1 -- 172.21.15.169:0/1347196418 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b9815bbb0 msgr2=0x7f6b9814c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:53.416 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.411+0000 7f6b8dd9b640 1 --2- 172.21.15.169:0/1347196418 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b9815bbb0 0x7f6b9814c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:53.416 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.411+0000 7f6b8dd9b640 1 -- 172.21.15.169:0/1347196418 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b80009490 con 0x7f6b980b05a0 2023-10-26T01:44:53.416 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.411+0000 7f6b8dd9b640 1 --2- 172.21.15.169:0/1347196418 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b980b05a0 0x7f6b9815a940 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7f6b80000c00 tx=0x7f6b800038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:53.416 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.411+0000 7f6b8f7fe640 1 -- 172.21.15.169:0/1347196418 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b80003ea0 con 0x7f6b980b05a0 2023-10-26T01:44:53.416 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.411+0000 7f6b8f7fe640 1 -- 172.21.15.169:0/1347196418 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6b80034440 con 0x7f6b980b05a0 2023-10-26T01:44:53.417 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.411+0000 7f6b9cb2c640 1 -- 172.21.15.169:0/1347196418 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b980702e0 con 0x7f6b980b05a0 2023-10-26T01:44:53.417 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.411+0000 7f6b8f7fe640 1 -- 172.21.15.169:0/1347196418 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b80034850 con 0x7f6b980b05a0 2023-10-26T01:44:53.417 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.412+0000 7f6b9cb2c640 1 -- 172.21.15.169:0/1347196418 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b980708a0 con 0x7f6b980b05a0 2023-10-26T01:44:53.418 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.413+0000 7f6b9cb2c640 1 -- 172.21.15.169:0/1347196418 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b9804db10 con 0x7f6b980b05a0 2023-10-26T01:44:53.418 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.414+0000 7f6b8f7fe640 1 -- 172.21.15.169:0/1347196418 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f6b800349b0 con 0x7f6b980b05a0 2023-10-26T01:44:53.418 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.414+0000 7f6b8f7fe640 1 --2- 172.21.15.169:0/1347196418 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f6b7806ae40 0x7f6b7806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:53.418 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.415+0000 7f6b8f7fe640 1 -- 172.21.15.169:0/1347196418 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f6b800abbd0 con 0x7f6b980b05a0 2023-10-26T01:44:53.419 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.415+0000 7f6b9659c640 1 --2- 172.21.15.169:0/1347196418 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f6b7806ae40 0x7f6b7806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:53.419 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.416+0000 7f6b9659c640 1 --2- 172.21.15.169:0/1347196418 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f6b7806ae40 0x7f6b7806d300 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f6b88004a30 tx=0x7f6b880016b0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:53.421 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.420+0000 7f6b8f7fe640 1 -- 172.21.15.169:0/1347196418 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6b80076f70 con 0x7f6b980b05a0 2023-10-26T01:44:53.610 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.607+0000 7f6b9cb2c640 1 -- 172.21.15.169:0/1347196418 --> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f6b98069db0 con 0x7f6b7806ae40 2023-10-26T01:44:53.611 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.608+0000 7f6b8f7fe640 1 -- 172.21.15.169:0/1347196418 <== mgr.4112 v2:172.21.15.170:6832/1187156918 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f6b98069db0 con 0x7f6b7806ae40 2023-10-26T01:44:53.611 INFO:teuthology.orchestra.run.smithi169.stderr:instructed osd(s) 4 to deep-scrub 2023-10-26T01:44:53.613 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.611+0000 7f6b9cb2c640 1 -- 172.21.15.169:0/1347196418 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f6b7806ae40 msgr2=0x7f6b7806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:53.613 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.611+0000 7f6b9cb2c640 1 --2- 172.21.15.169:0/1347196418 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f6b7806ae40 0x7f6b7806d300 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f6b88004a30 tx=0x7f6b880016b0 comp rx=0 tx=0).stop 2023-10-26T01:44:53.614 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.611+0000 7f6b9cb2c640 1 -- 172.21.15.169:0/1347196418 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b980b05a0 msgr2=0x7f6b9815a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:53.614 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.611+0000 7f6b9cb2c640 1 --2- 172.21.15.169:0/1347196418 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b980b05a0 0x7f6b9815a940 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7f6b80000c00 tx=0x7f6b800038a0 comp rx=0 tx=0).stop 2023-10-26T01:44:53.614 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.611+0000 7f6b9cb2c640 1 -- 172.21.15.169:0/1347196418 shutdown_connections 2023-10-26T01:44:53.614 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.611+0000 7f6b9cb2c640 1 --2- 172.21.15.169:0/1347196418 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f6b7806ae40 0x7f6b7806d300 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:53.614 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.611+0000 7f6b9cb2c640 1 --2- 172.21.15.169:0/1347196418 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b980b05a0 0x7f6b9815a940 unknown :-1 s=CLOSED pgs=2004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:53.615 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.611+0000 7f6b9cb2c640 1 --2- 172.21.15.169:0/1347196418 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b9815bbb0 0x7f6b9814c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:53.615 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.611+0000 7f6b9cb2c640 1 --2- 172.21.15.169:0/1347196418 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6b9806ce00 0x7f6b9806f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:53.615 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.611+0000 7f6b9cb2c640 1 -- 172.21.15.169:0/1347196418 >> 172.21.15.169:0/1347196418 conn(0x7f6b98069930 msgr2=0x7f6b981561b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:53.615 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.611+0000 7f6b9cb2c640 1 -- 172.21.15.169:0/1347196418 shutdown_connections 2023-10-26T01:44:53.615 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.612+0000 7f6b9cb2c640 1 -- 172.21.15.169:0/1347196418 wait complete. 2023-10-26T01:44:53.631 INFO:tasks.ceph:Scrubbing osd.5 2023-10-26T01:44:53.631 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-10-26T01:44:53.841 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.839+0000 7fbb3359e640 1 -- 172.21.15.169:0/492290919 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb340b1820 msgr2=0x7fbb34098a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:53.842 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.839+0000 7fbb3359e640 1 --2- 172.21.15.169:0/492290919 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb340b1820 0x7fbb34098a60 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7fbb340b20c0 tx=0x7fbb1c02bda0 comp rx=0 tx=0).stop 2023-10-26T01:44:53.842 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.839+0000 7fbb3359e640 1 -- 172.21.15.169:0/492290919 shutdown_connections 2023-10-26T01:44:53.842 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.839+0000 7fbb3359e640 1 --2- 172.21.15.169:0/492290919 >> v2:172.21.15.170:3300/0 conn(0x7fbb3415c710 0x7fbb34098fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:53.843 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.839+0000 7fbb3359e640 1 --2- 172.21.15.169:0/492290919 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb340b1820 0x7fbb34098a60 unknown :-1 s=CLOSED pgs=2005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:53.843 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.839+0000 7fbb3359e640 1 --2- 172.21.15.169:0/492290919 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbb340cc890 0x7fbb340afe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:53.843 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.839+0000 7fbb3359e640 1 -- 172.21.15.169:0/492290919 >> 172.21.15.169:0/492290919 conn(0x7fbb34069b40 msgr2=0x7fbb340aeb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:53.843 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.839+0000 7fbb3359e640 1 -- 172.21.15.169:0/492290919 shutdown_connections 2023-10-26T01:44:53.843 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.839+0000 7fbb3359e640 1 -- 172.21.15.169:0/492290919 wait complete. 2023-10-26T01:44:53.845 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.841+0000 7fbb3359e640 1 Processor -- start 2023-10-26T01:44:53.845 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.841+0000 7fbb3359e640 1 -- start start 2023-10-26T01:44:53.846 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.842+0000 7fbb3359e640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbb340b1820 0x7fbb3414a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:53.846 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.842+0000 7fbb3359e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb340cc890 0x7fbb34156940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:53.846 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.842+0000 7fbb3359e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb3415c710 0x7fbb340ae780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:53.846 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.842+0000 7fbb3359e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb340ad6e0 con 0x7fbb3415c710 2023-10-26T01:44:53.846 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.842+0000 7fbb3359e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb340ad9c0 con 0x7fbb340cc890 2023-10-26T01:44:53.847 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.842+0000 7fbb3359e640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb340adca0 con 0x7fbb340b1820 2023-10-26T01:44:53.847 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.842+0000 7fbb32d9d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb3415c710 0x7fbb340ae780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:53.847 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.842+0000 7fbb32d9d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb3415c710 0x7fbb340ae780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:57368/0 (socket says 172.21.15.169:57368) 2023-10-26T01:44:53.847 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.842+0000 7fbb32d9d640 1 -- 172.21.15.169:0/1485214837 learned_addr learned my addr 172.21.15.169:0/1485214837 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:44:53.847 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.843+0000 7fbb2bfff640 1 --2- 172.21.15.169:0/1485214837 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb340cc890 0x7fbb34156940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:53.848 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.843+0000 7fbb32d9d640 1 -- 172.21.15.169:0/1485214837 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbb340b1820 msgr2=0x7fbb3414a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:53.848 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.843+0000 7fbb32d9d640 1 --2- 172.21.15.169:0/1485214837 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbb340b1820 0x7fbb3414a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:53.848 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.843+0000 7fbb32d9d640 1 -- 172.21.15.169:0/1485214837 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb340cc890 msgr2=0x7fbb34156940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:53.848 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.843+0000 7fbb32d9d640 1 --2- 172.21.15.169:0/1485214837 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb340cc890 0x7fbb34156940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:53.848 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.843+0000 7fbb32d9d640 1 -- 172.21.15.169:0/1485214837 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb1c009560 con 0x7fbb3415c710 2023-10-26T01:44:53.849 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.843+0000 7fbb32d9d640 1 --2- 172.21.15.169:0/1485214837 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb3415c710 0x7fbb340ae780 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c00cab0 tx=0x7fbb2c00cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:53.849 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.843+0000 7fbb2affd640 1 -- 172.21.15.169:0/1485214837 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb2c007d90 con 0x7fbb3415c710 2023-10-26T01:44:53.849 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.843+0000 7fbb2affd640 1 -- 172.21.15.169:0/1485214837 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fbb2c00f450 con 0x7fbb3415c710 2023-10-26T01:44:53.849 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.843+0000 7fbb2affd640 1 -- 172.21.15.169:0/1485214837 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb2c00f840 con 0x7fbb3415c710 2023-10-26T01:44:53.849 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.843+0000 7fbb3359e640 1 -- 172.21.15.169:0/1485214837 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb340adf80 con 0x7fbb3415c710 2023-10-26T01:44:53.850 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.844+0000 7fbb3359e640 1 -- 172.21.15.169:0/1485214837 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb34156e80 con 0x7fbb3415c710 2023-10-26T01:44:53.851 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.845+0000 7fbb3359e640 1 -- 172.21.15.169:0/1485214837 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbb34157070 con 0x7fbb3415c710 2023-10-26T01:44:53.851 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.846+0000 7fbb2affd640 1 -- 172.21.15.169:0/1485214837 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fbb2c00f9a0 con 0x7fbb3415c710 2023-10-26T01:44:53.851 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.847+0000 7fbb2affd640 1 --2- 172.21.15.169:0/1485214837 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbb1406ae40 0x7fbb1406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:53.852 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.847+0000 7fbb2affd640 1 -- 172.21.15.169:0/1485214837 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fbb2c089dc0 con 0x7fbb3415c710 2023-10-26T01:44:53.852 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.847+0000 7fbb2affd640 1 --2- 172.21.15.169:0/1485214837 >> [v2:172.21.15.170:6808/3335654351,v1:172.21.15.170:6809/3335654351] conn(0x7fbb14070ca0 0x7fbb140730c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:53.852 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.847+0000 7fbb2affd640 1 -- 172.21.15.169:0/1485214837 --> [v2:172.21.15.170:6808/3335654351,v1:172.21.15.170:6809/3335654351] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbb14073770 con 0x7fbb14070ca0 2023-10-26T01:44:53.852 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.847+0000 7fbb2affd640 1 -- 172.21.15.169:0/1485214837 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_get_version_reply(handle=1 version=189) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbb2c08c3a0 con 0x7fbb3415c710 2023-10-26T01:44:53.852 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.847+0000 7fbb3259c640 1 --2- 172.21.15.169:0/1485214837 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbb1406ae40 0x7fbb1406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:53.853 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.848+0000 7fbb2bfff640 1 --2- 172.21.15.169:0/1485214837 >> [v2:172.21.15.170:6808/3335654351,v1:172.21.15.170:6809/3335654351] conn(0x7fbb14070ca0 0x7fbb140730c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:53.853 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.849+0000 7fbb2bfff640 1 --2- 172.21.15.169:0/1485214837 >> [v2:172.21.15.170:6808/3335654351,v1:172.21.15.170:6809/3335654351] conn(0x7fbb14070ca0 0x7fbb140730c0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:53.853 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.849+0000 7fbb3259c640 1 --2- 172.21.15.169:0/1485214837 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbb1406ae40 0x7fbb1406d300 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fbb2400a990 tx=0x7fbb24005c70 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:53.853 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.850+0000 7fbb2affd640 1 -- 172.21.15.169:0/1485214837 <== osd.5 v2:172.21.15.170:6808/3335654351 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23595 (crc 0 0 0) 0x7fbb14073770 con 0x7fbb14070ca0 2023-10-26T01:44:53.864 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.863+0000 7fbb3359e640 1 -- 172.21.15.169:0/1485214837 --> [v2:172.21.15.170:6808/3335654351,v1:172.21.15.170:6809/3335654351] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fbb340ae4a0 con 0x7fbb14070ca0 2023-10-26T01:44:53.865 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.864+0000 7fbb2affd640 1 -- 172.21.15.169:0/1485214837 <== osd.5 v2:172.21.15.170:6808/3335654351 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+632 (crc 0 0 0) 0x7fbb340ae4a0 con 0x7fbb14070ca0 2023-10-26T01:44:53.866 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2023-10-26T01:44:53.866 INFO:teuthology.orchestra.run.smithi169.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_pg_delete_cost = '' (not observed, change may require restart) osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2023-10-26T01:44:53.866 INFO:teuthology.orchestra.run.smithi169.stdout:} 2023-10-26T01:44:53.867 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.864+0000 7fbb3359e640 1 -- 172.21.15.169:0/1485214837 >> [v2:172.21.15.170:6808/3335654351,v1:172.21.15.170:6809/3335654351] conn(0x7fbb14070ca0 msgr2=0x7fbb140730c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:53.867 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.864+0000 7fbb3359e640 1 --2- 172.21.15.169:0/1485214837 >> [v2:172.21.15.170:6808/3335654351,v1:172.21.15.170:6809/3335654351] conn(0x7fbb14070ca0 0x7fbb140730c0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:53.868 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.864+0000 7fbb3359e640 1 -- 172.21.15.169:0/1485214837 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbb1406ae40 msgr2=0x7fbb1406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:53.868 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.864+0000 7fbb3359e640 1 --2- 172.21.15.169:0/1485214837 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbb1406ae40 0x7fbb1406d300 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fbb2400a990 tx=0x7fbb24005c70 comp rx=0 tx=0).stop 2023-10-26T01:44:53.868 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.864+0000 7fbb3359e640 1 -- 172.21.15.169:0/1485214837 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb3415c710 msgr2=0x7fbb340ae780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:53.868 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.864+0000 7fbb3359e640 1 --2- 172.21.15.169:0/1485214837 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb3415c710 0x7fbb340ae780 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c00cab0 tx=0x7fbb2c00cf70 comp rx=0 tx=0).stop 2023-10-26T01:44:53.868 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.864+0000 7fbb32d9d640 1 -- 172.21.15.169:0/1485214837 reap_dead start 2023-10-26T01:44:53.869 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.865+0000 7fbb3359e640 1 -- 172.21.15.169:0/1485214837 shutdown_connections 2023-10-26T01:44:53.869 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.865+0000 7fbb3359e640 1 -- 172.21.15.169:0/1485214837 >> 172.21.15.169:0/1485214837 conn(0x7fbb34069b40 msgr2=0x7fbb340abe70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:53.869 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.865+0000 7fbb3359e640 1 -- 172.21.15.169:0/1485214837 shutdown_connections 2023-10-26T01:44:53.869 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:53.865+0000 7fbb3359e640 1 -- 172.21.15.169:0/1485214837 wait complete. 2023-10-26T01:44:53.884 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-10-26T01:44:54.101 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.099+0000 7f84f0600640 1 -- 172.21.15.169:0/3806762259 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84e80cdef0 msgr2=0x7f84e815b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:54.101 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.099+0000 7f84f0600640 1 --2- 172.21.15.169:0/3806762259 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84e80cdef0 0x7f84e815b4a0 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7f84d8005fd0 tx=0x7f84d802bd90 comp rx=0 tx=0).stop 2023-10-26T01:44:54.101 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.099+0000 7f84f0600640 1 -- 172.21.15.169:0/3806762259 shutdown_connections 2023-10-26T01:44:54.102 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.099+0000 7f84f0600640 1 --2- 172.21.15.169:0/3806762259 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f84e815b9e0 0x7f84e80b05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:54.102 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.099+0000 7f84f0600640 1 --2- 172.21.15.169:0/3806762259 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84e80cdef0 0x7f84e815b4a0 unknown :-1 s=CLOSED pgs=2007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:54.102 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.099+0000 7f84f0600640 1 -- 172.21.15.169:0/3806762259 >> 172.21.15.169:0/3806762259 conn(0x7f84e8069930 msgr2=0x7f84e8069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:54.102 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.099+0000 7f84f0600640 1 -- 172.21.15.169:0/3806762259 shutdown_connections 2023-10-26T01:44:54.102 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.099+0000 7f84f0600640 1 -- 172.21.15.169:0/3806762259 wait complete. 2023-10-26T01:44:54.104 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.101+0000 7f84f0600640 1 Processor -- start 2023-10-26T01:44:54.104 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.101+0000 7f84f0600640 1 -- start start 2023-10-26T01:44:54.104 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.102+0000 7f84f0600640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f84e80cdef0 0x7f84e8096910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:54.104 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.102+0000 7f84f0600640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f84e8155360 0x7f84e814c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:54.105 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.102+0000 7f84f0600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84e815b9e0 0x7f84e814ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:54.105 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.102+0000 7f84f0600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84e80afeb0 con 0x7f84e815b9e0 2023-10-26T01:44:54.105 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.102+0000 7f84f0600640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84e815b790 con 0x7f84e80cdef0 2023-10-26T01:44:54.105 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.102+0000 7f84f0600640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84e8151ba0 con 0x7f84e8155360 2023-10-26T01:44:54.105 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.102+0000 7f84eeb9d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84e815b9e0 0x7f84e814ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:54.105 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.102+0000 7f84eeb9d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84e815b9e0 0x7f84e814ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:51642/0 (socket says 172.21.15.169:51642) 2023-10-26T01:44:54.106 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.102+0000 7f84eeb9d640 1 -- 172.21.15.169:0/4171165451 learned_addr learned my addr 172.21.15.169:0/4171165451 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:44:54.106 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.102+0000 7f84ee39c640 1 --2- 172.21.15.169:0/4171165451 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f84e80cdef0 0x7f84e8096910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:54.107 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.103+0000 7f84edb9b640 1 --2- 172.21.15.169:0/4171165451 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f84e8155360 0x7f84e814c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:54.107 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.103+0000 7f84eeb9d640 1 -- 172.21.15.169:0/4171165451 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f84e8155360 msgr2=0x7f84e814c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:54.108 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.103+0000 7f84eeb9d640 1 --2- 172.21.15.169:0/4171165451 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f84e8155360 0x7f84e814c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:54.108 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.103+0000 7f84eeb9d640 1 -- 172.21.15.169:0/4171165451 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f84e80cdef0 msgr2=0x7f84e8096910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:54.108 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.103+0000 7f84eeb9d640 1 --2- 172.21.15.169:0/4171165451 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f84e80cdef0 0x7f84e8096910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:54.108 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.103+0000 7f84eeb9d640 1 -- 172.21.15.169:0/4171165451 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84d8009560 con 0x7f84e815b9e0 2023-10-26T01:44:54.108 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.103+0000 7f84eeb9d640 1 --2- 172.21.15.169:0/4171165451 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84e815b9e0 0x7f84e814ce90 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7f84e000b970 tx=0x7f84e000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:54.109 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.103+0000 7f84deffd640 1 -- 172.21.15.169:0/4171165451 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e000cb40 con 0x7f84e815b9e0 2023-10-26T01:44:54.109 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.103+0000 7f84deffd640 1 -- 172.21.15.169:0/4171165451 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f84e000cdb0 con 0x7f84e815b9e0 2023-10-26T01:44:54.109 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.103+0000 7f84deffd640 1 -- 172.21.15.169:0/4171165451 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e00146a0 con 0x7f84e815b9e0 2023-10-26T01:44:54.109 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.103+0000 7f84f0600640 1 -- 172.21.15.169:0/4171165451 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84e81518c0 con 0x7f84e815b9e0 2023-10-26T01:44:54.109 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.103+0000 7f84f0600640 1 -- 172.21.15.169:0/4171165451 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84e80afbd0 con 0x7f84e815b9e0 2023-10-26T01:44:54.109 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.104+0000 7f84f0600640 1 -- 172.21.15.169:0/4171165451 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84e804db10 con 0x7f84e815b9e0 2023-10-26T01:44:54.110 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.106+0000 7f84deffd640 1 -- 172.21.15.169:0/4171165451 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f84e001d3b0 con 0x7f84e815b9e0 2023-10-26T01:44:54.110 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.107+0000 7f84deffd640 1 --2- 172.21.15.169:0/4171165451 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f84cc06ae40 0x7f84cc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:54.111 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.107+0000 7f84deffd640 1 -- 172.21.15.169:0/4171165451 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f84e0089520 con 0x7f84e815b9e0 2023-10-26T01:44:54.111 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.107+0000 7f84ee39c640 1 --2- 172.21.15.169:0/4171165451 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f84cc06ae40 0x7f84cc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:54.111 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.108+0000 7f84ee39c640 1 --2- 172.21.15.169:0/4171165451 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f84cc06ae40 0x7f84cc06d300 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f84e4007fd0 tx=0x7f84e4009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:54.112 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.111+0000 7f84deffd640 1 -- 172.21.15.169:0/4171165451 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f84e00548d0 con 0x7f84e815b9e0 2023-10-26T01:44:54.301 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.299+0000 7f84f0600640 1 -- 172.21.15.169:0/4171165451 --> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f84e80af770 con 0x7f84cc06ae40 2023-10-26T01:44:54.302 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.300+0000 7f84deffd640 1 -- 172.21.15.169:0/4171165451 <== mgr.4112 v2:172.21.15.170:6832/1187156918 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f84e4001c60 con 0x7f84cc06ae40 2023-10-26T01:44:54.302 INFO:teuthology.orchestra.run.smithi169.stderr:instructed osd(s) 5 to deep-scrub 2023-10-26T01:44:54.304 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.302+0000 7f84f0600640 1 -- 172.21.15.169:0/4171165451 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f84cc06ae40 msgr2=0x7f84cc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:54.305 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.302+0000 7f84f0600640 1 --2- 172.21.15.169:0/4171165451 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f84cc06ae40 0x7f84cc06d300 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f84e4007fd0 tx=0x7f84e4009040 comp rx=0 tx=0).stop 2023-10-26T01:44:54.305 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.302+0000 7f84f0600640 1 -- 172.21.15.169:0/4171165451 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84e815b9e0 msgr2=0x7f84e814ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:54.305 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.302+0000 7f84f0600640 1 --2- 172.21.15.169:0/4171165451 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84e815b9e0 0x7f84e814ce90 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7f84e000b970 tx=0x7f84e000be30 comp rx=0 tx=0).stop 2023-10-26T01:44:54.305 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.303+0000 7f84f0600640 1 -- 172.21.15.169:0/4171165451 shutdown_connections 2023-10-26T01:44:54.306 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.303+0000 7f84f0600640 1 --2- 172.21.15.169:0/4171165451 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f84cc06ae40 0x7f84cc06d300 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:54.306 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.303+0000 7f84f0600640 1 --2- 172.21.15.169:0/4171165451 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84e815b9e0 0x7f84e814ce90 unknown :-1 s=CLOSED pgs=2008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:54.306 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.303+0000 7f84f0600640 1 --2- 172.21.15.169:0/4171165451 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f84e8155360 0x7f84e814c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:54.306 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.303+0000 7f84f0600640 1 --2- 172.21.15.169:0/4171165451 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f84e80cdef0 0x7f84e8096910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:54.306 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.303+0000 7f84f0600640 1 -- 172.21.15.169:0/4171165451 >> 172.21.15.169:0/4171165451 conn(0x7f84e8069930 msgr2=0x7f84e814d9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:54.307 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.303+0000 7f84f0600640 1 -- 172.21.15.169:0/4171165451 shutdown_connections 2023-10-26T01:44:54.307 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.303+0000 7f84f0600640 1 -- 172.21.15.169:0/4171165451 wait complete. 2023-10-26T01:44:54.321 INFO:tasks.ceph:Scrubbing osd.6 2023-10-26T01:44:54.322 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-10-26T01:44:54.535 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.533+0000 7f0922397640 1 -- 172.21.15.169:0/3356407904 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f091c1595e0 msgr2=0x7f091c0c9bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:54.536 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.533+0000 7f0922397640 1 --2- 172.21.15.169:0/3356407904 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f091c1595e0 0x7f091c0c9bd0 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7f091c0c9950 tx=0x7f090c02bce0 comp rx=0 tx=0).stop 2023-10-26T01:44:54.536 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.533+0000 7f0922397640 1 -- 172.21.15.169:0/3356407904 shutdown_connections 2023-10-26T01:44:54.536 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.533+0000 7f0922397640 1 --2- 172.21.15.169:0/3356407904 >> v2:172.21.15.170:3300/0 conn(0x7f091c0ac2f0 0x7f091c0ac720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:54.536 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.533+0000 7f0922397640 1 --2- 172.21.15.169:0/3356407904 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f091c1595e0 0x7f091c0c9bd0 unknown :-1 s=CLOSED pgs=2010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:54.537 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.533+0000 7f0922397640 1 -- 172.21.15.169:0/3356407904 >> 172.21.15.169:0/3356407904 conn(0x7f091c0aeee0 msgr2=0x7f091c0af2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:54.537 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.533+0000 7f0922397640 1 -- 172.21.15.169:0/3356407904 shutdown_connections 2023-10-26T01:44:54.537 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.533+0000 7f0922397640 1 -- 172.21.15.169:0/3356407904 wait complete. 2023-10-26T01:44:54.539 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.535+0000 7f0922397640 1 Processor -- start 2023-10-26T01:44:54.539 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.535+0000 7f0922397640 1 -- start start 2023-10-26T01:44:54.540 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.536+0000 7f0922397640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f091c0ac2f0 0x7f091c1f6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:54.540 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.536+0000 7f0922397640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f091c1595e0 0x7f091c1584a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:54.540 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.536+0000 7f0922397640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f091c15d120 0x7f091c1f19c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:54.540 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.536+0000 7f0922397640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f091c1f66a0 con 0x7f091c1595e0 2023-10-26T01:44:54.540 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.536+0000 7f0922397640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f091c156b80 con 0x7f091c15d120 2023-10-26T01:44:54.541 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.536+0000 7f0922397640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f091c157010 con 0x7f091c0ac2f0 2023-10-26T01:44:54.541 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.537+0000 7f091b7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f091c1595e0 0x7f091c1584a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:54.541 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.537+0000 7f091b7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f091c1595e0 0x7f091c1584a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:51668/0 (socket says 172.21.15.169:51668) 2023-10-26T01:44:54.541 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.537+0000 7f091b7fe640 1 -- 172.21.15.169:0/3024124303 learned_addr learned my addr 172.21.15.169:0/3024124303 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:44:54.541 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.537+0000 7f091bfff640 1 --2- 172.21.15.169:0/3024124303 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f091c0ac2f0 0x7f091c1f6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:54.542 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.537+0000 7f0920934640 1 --2- 172.21.15.169:0/3024124303 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f091c15d120 0x7f091c1f19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:54.542 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.537+0000 7f091b7fe640 1 -- 172.21.15.169:0/3024124303 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f091c0ac2f0 msgr2=0x7f091c1f6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:54.542 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.537+0000 7f091b7fe640 1 --2- 172.21.15.169:0/3024124303 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f091c0ac2f0 0x7f091c1f6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:54.542 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.537+0000 7f091b7fe640 1 -- 172.21.15.169:0/3024124303 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f091c15d120 msgr2=0x7f091c1f19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:54.542 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.537+0000 7f091b7fe640 1 --2- 172.21.15.169:0/3024124303 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f091c15d120 0x7f091c1f19c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:54.543 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.537+0000 7f091b7fe640 1 -- 172.21.15.169:0/3024124303 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f090c009490 con 0x7f091c1595e0 2023-10-26T01:44:54.543 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.537+0000 7f0920934640 1 --2- 172.21.15.169:0/3024124303 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f091c15d120 0x7f091c1f19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:44:54.543 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.537+0000 7f091b7fe640 1 --2- 172.21.15.169:0/3024124303 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f091c1595e0 0x7f091c1584a0 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7f091000c3e0 tx=0x7f091000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:54.543 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.538+0000 7f0918ff9640 1 -- 172.21.15.169:0/3024124303 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09100035f0 con 0x7f091c1595e0 2023-10-26T01:44:54.543 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.538+0000 7f0918ff9640 1 -- 172.21.15.169:0/3024124303 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0910003750 con 0x7f091c1595e0 2023-10-26T01:44:54.544 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.538+0000 7f0918ff9640 1 -- 172.21.15.169:0/3024124303 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0910003b80 con 0x7f091c1595e0 2023-10-26T01:44:54.544 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.538+0000 7f0922397640 1 -- 172.21.15.169:0/3024124303 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f091c1589e0 con 0x7f091c1595e0 2023-10-26T01:44:54.544 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.538+0000 7f0922397640 1 -- 172.21.15.169:0/3024124303 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f091c156250 con 0x7f091c1595e0 2023-10-26T01:44:54.545 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.539+0000 7f0922397640 1 -- 172.21.15.169:0/3024124303 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f091c0cb3e0 con 0x7f091c1595e0 2023-10-26T01:44:54.546 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.540+0000 7f0918ff9640 1 -- 172.21.15.169:0/3024124303 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f0910003ce0 con 0x7f091c1595e0 2023-10-26T01:44:54.546 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.541+0000 7f0918ff9640 1 --2- 172.21.15.169:0/3024124303 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f090806ae40 0x7f090806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:54.546 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.541+0000 7f0918ff9640 1 -- 172.21.15.169:0/3024124303 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f0910088b80 con 0x7f091c1595e0 2023-10-26T01:44:54.546 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.541+0000 7f0918ff9640 1 --2- 172.21.15.169:0/3024124303 >> [v2:172.21.15.170:6800/2520460697,v1:172.21.15.170:6801/2520460697] conn(0x7f0908070ca0 0x7f09080730c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:54.546 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.541+0000 7f0918ff9640 1 -- 172.21.15.169:0/3024124303 --> [v2:172.21.15.170:6800/2520460697,v1:172.21.15.170:6801/2520460697] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0908073770 con 0x7f0908070ca0 2023-10-26T01:44:54.547 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.541+0000 7f0918ff9640 1 -- 172.21.15.169:0/3024124303 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_get_version_reply(handle=1 version=189) v2 ==== 24+0+0 (secure 0 0 0) 0x7f091008d2a0 con 0x7f091c1595e0 2023-10-26T01:44:54.547 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.541+0000 7f091bfff640 1 --2- 172.21.15.169:0/3024124303 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f090806ae40 0x7f090806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:54.547 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.542+0000 7f0920934640 1 --2- 172.21.15.169:0/3024124303 >> [v2:172.21.15.170:6800/2520460697,v1:172.21.15.170:6801/2520460697] conn(0x7f0908070ca0 0x7f09080730c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:54.547 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.543+0000 7f091bfff640 1 --2- 172.21.15.169:0/3024124303 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f090806ae40 0x7f090806d300 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f090c036000 tx=0x7f090c004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:54.547 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.543+0000 7f0920934640 1 --2- 172.21.15.169:0/3024124303 >> [v2:172.21.15.170:6800/2520460697,v1:172.21.15.170:6801/2520460697] conn(0x7f0908070ca0 0x7f09080730c0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:54.548 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.544+0000 7f0918ff9640 1 -- 172.21.15.169:0/3024124303 <== osd.6 v2:172.21.15.170:6800/2520460697 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23595 (crc 0 0 0) 0x7f0908073770 con 0x7f0908070ca0 2023-10-26T01:44:54.558 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.556+0000 7f0922397640 1 -- 172.21.15.169:0/3024124303 --> [v2:172.21.15.170:6800/2520460697,v1:172.21.15.170:6801/2520460697] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f091c066af0 con 0x7f0908070ca0 2023-10-26T01:44:54.559 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.558+0000 7f0918ff9640 1 -- 172.21.15.169:0/3024124303 <== osd.6 v2:172.21.15.170:6800/2520460697 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+632 (crc 0 0 0) 0x7f091c066af0 con 0x7f0908070ca0 2023-10-26T01:44:54.560 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2023-10-26T01:44:54.560 INFO:teuthology.orchestra.run.smithi169.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_pg_delete_cost = '' (not observed, change may require restart) osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2023-10-26T01:44:54.560 INFO:teuthology.orchestra.run.smithi169.stdout:} 2023-10-26T01:44:54.561 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.558+0000 7f0922397640 1 -- 172.21.15.169:0/3024124303 >> [v2:172.21.15.170:6800/2520460697,v1:172.21.15.170:6801/2520460697] conn(0x7f0908070ca0 msgr2=0x7f09080730c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:54.561 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.558+0000 7f0922397640 1 --2- 172.21.15.169:0/3024124303 >> [v2:172.21.15.170:6800/2520460697,v1:172.21.15.170:6801/2520460697] conn(0x7f0908070ca0 0x7f09080730c0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:54.561 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.558+0000 7f0922397640 1 -- 172.21.15.169:0/3024124303 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f090806ae40 msgr2=0x7f090806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:54.562 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.558+0000 7f0922397640 1 --2- 172.21.15.169:0/3024124303 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f090806ae40 0x7f090806d300 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f090c036000 tx=0x7f090c004730 comp rx=0 tx=0).stop 2023-10-26T01:44:54.562 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.558+0000 7f0922397640 1 -- 172.21.15.169:0/3024124303 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f091c1595e0 msgr2=0x7f091c1584a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:54.562 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.558+0000 7f0922397640 1 --2- 172.21.15.169:0/3024124303 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f091c1595e0 0x7f091c1584a0 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7f091000c3e0 tx=0x7f091000c8a0 comp rx=0 tx=0).stop 2023-10-26T01:44:54.562 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.558+0000 7f0920934640 1 -- 172.21.15.169:0/3024124303 reap_dead start 2023-10-26T01:44:54.562 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.558+0000 7f0922397640 1 -- 172.21.15.169:0/3024124303 shutdown_connections 2023-10-26T01:44:54.563 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.558+0000 7f0922397640 1 -- 172.21.15.169:0/3024124303 >> 172.21.15.169:0/3024124303 conn(0x7f091c0aeee0 msgr2=0x7f091c0c99a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:54.563 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.558+0000 7f0922397640 1 -- 172.21.15.169:0/3024124303 shutdown_connections 2023-10-26T01:44:54.563 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.558+0000 7f0922397640 1 -- 172.21.15.169:0/3024124303 wait complete. 2023-10-26T01:44:54.576 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-10-26T01:44:54.795 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.793+0000 7f0272051640 1 -- 172.21.15.169:0/3448048223 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f026c157be0 msgr2=0x7f026c06b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:54.795 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.793+0000 7f0272051640 1 --2- 172.21.15.169:0/3448048223 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f026c157be0 0x7f026c06b740 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7f026c0aa9d0 tx=0x7f025c02bcf0 comp rx=0 tx=0).stop 2023-10-26T01:44:54.795 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.793+0000 7f0272051640 1 -- 172.21.15.169:0/3448048223 shutdown_connections 2023-10-26T01:44:54.796 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.793+0000 7f0272051640 1 --2- 172.21.15.169:0/3448048223 >> v2:172.21.15.170:3300/0 conn(0x7f026c06ce50 0x7f026c158bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:54.796 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.793+0000 7f0272051640 1 --2- 172.21.15.169:0/3448048223 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f026c157be0 0x7f026c06b740 unknown :-1 s=CLOSED pgs=2013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:54.796 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.793+0000 7f0272051640 1 -- 172.21.15.169:0/3448048223 >> 172.21.15.169:0/3448048223 conn(0x7f026c069930 msgr2=0x7f026c069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:54.796 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.793+0000 7f0272051640 1 -- 172.21.15.169:0/3448048223 shutdown_connections 2023-10-26T01:44:54.796 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.793+0000 7f0272051640 1 -- 172.21.15.169:0/3448048223 wait complete. 2023-10-26T01:44:54.798 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.795+0000 7f0272051640 1 Processor -- start 2023-10-26T01:44:54.798 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.795+0000 7f0272051640 1 -- start start 2023-10-26T01:44:54.798 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.796+0000 7f0272051640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f026c06ce50 0x7f026c143fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:54.799 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.796+0000 7f0272051640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f026c0aabf0 0x7f026c1437d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:54.799 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.796+0000 7f0272051640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f026c157be0 0x7f026c14ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:54.799 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.796+0000 7f0272051640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f026c1448b0 con 0x7f026c06ce50 2023-10-26T01:44:54.799 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.796+0000 7f0272051640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f026c149b70 con 0x7f026c157be0 2023-10-26T01:44:54.799 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.796+0000 7f0272051640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f026c14a380 con 0x7f026c0aabf0 2023-10-26T01:44:54.800 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.796+0000 7f026b7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f026c06ce50 0x7f026c143fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:54.800 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.796+0000 7f026b7fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f026c06ce50 0x7f026c143fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:51688/0 (socket says 172.21.15.169:51688) 2023-10-26T01:44:54.800 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.796+0000 7f026b7fe640 1 -- 172.21.15.169:0/2565576220 learned_addr learned my addr 172.21.15.169:0/2565576220 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:44:54.800 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.796+0000 7f026b7fe640 1 -- 172.21.15.169:0/2565576220 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f026c0aabf0 msgr2=0x7f026c1437d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:54.800 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.796+0000 7f026b7fe640 1 --2- 172.21.15.169:0/2565576220 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f026c0aabf0 0x7f026c1437d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:54.801 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.796+0000 7f026b7fe640 1 -- 172.21.15.169:0/2565576220 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f026c157be0 msgr2=0x7f026c14ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:54.801 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.796+0000 7f026b7fe640 1 --2- 172.21.15.169:0/2565576220 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f026c157be0 0x7f026c14ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:54.801 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.796+0000 7f026b7fe640 1 -- 172.21.15.169:0/2565576220 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f025c009490 con 0x7f026c06ce50 2023-10-26T01:44:54.801 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.797+0000 7f026b7fe640 1 --2- 172.21.15.169:0/2565576220 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f026c06ce50 0x7f026c143fa0 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7f025c000c00 tx=0x7f025c02c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:54.802 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.797+0000 7f0268ff9640 1 -- 172.21.15.169:0/2565576220 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f025c003b80 con 0x7f026c06ce50 2023-10-26T01:44:54.803 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.797+0000 7f0268ff9640 1 -- 172.21.15.169:0/2565576220 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f025c003df0 con 0x7f026c06ce50 2023-10-26T01:44:54.803 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.797+0000 7f0272051640 1 -- 172.21.15.169:0/2565576220 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f026c149dc0 con 0x7f026c06ce50 2023-10-26T01:44:54.803 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.797+0000 7f0268ff9640 1 -- 172.21.15.169:0/2565576220 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f025c0346a0 con 0x7f026c06ce50 2023-10-26T01:44:54.803 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.797+0000 7f0272051640 1 -- 172.21.15.169:0/2565576220 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f026c146160 con 0x7f026c06ce50 2023-10-26T01:44:54.803 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.798+0000 7f0272051640 1 -- 172.21.15.169:0/2565576220 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f026c0ccb80 con 0x7f026c06ce50 2023-10-26T01:44:54.804 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.799+0000 7f0268ff9640 1 -- 172.21.15.169:0/2565576220 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f025c03b040 con 0x7f026c06ce50 2023-10-26T01:44:54.804 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.800+0000 7f0268ff9640 1 --2- 172.21.15.169:0/2565576220 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f024c06ae40 0x7f024c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:54.804 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.800+0000 7f0268ff9640 1 -- 172.21.15.169:0/2565576220 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f025c0ab5c0 con 0x7f026c06ce50 2023-10-26T01:44:54.804 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.801+0000 7f0263fff640 1 --2- 172.21.15.169:0/2565576220 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f024c06ae40 0x7f024c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:54.805 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.802+0000 7f0263fff640 1 --2- 172.21.15.169:0/2565576220 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f024c06ae40 0x7f024c06d300 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f026c0b1290 tx=0x7f0254005bf0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:54.806 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.804+0000 7f0268ff9640 1 -- 172.21.15.169:0/2565576220 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f025c076a00 con 0x7f026c06ce50 2023-10-26T01:44:54.995 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.993+0000 7f0272051640 1 -- 172.21.15.169:0/2565576220 --> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f026c0ccfd0 con 0x7f024c06ae40 2023-10-26T01:44:54.996 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.994+0000 7f0268ff9640 1 -- 172.21.15.169:0/2565576220 <== mgr.4112 v2:172.21.15.170:6832/1187156918 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f026c0ccfd0 con 0x7f024c06ae40 2023-10-26T01:44:54.996 INFO:teuthology.orchestra.run.smithi169.stderr:instructed osd(s) 6 to deep-scrub 2023-10-26T01:44:54.999 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.997+0000 7f0272051640 1 -- 172.21.15.169:0/2565576220 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f024c06ae40 msgr2=0x7f024c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:54.999 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.997+0000 7f0272051640 1 --2- 172.21.15.169:0/2565576220 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f024c06ae40 0x7f024c06d300 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f026c0b1290 tx=0x7f0254005bf0 comp rx=0 tx=0).stop 2023-10-26T01:44:54.999 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.997+0000 7f0272051640 1 -- 172.21.15.169:0/2565576220 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f026c06ce50 msgr2=0x7f026c143fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:54.999 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.997+0000 7f0272051640 1 --2- 172.21.15.169:0/2565576220 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f026c06ce50 0x7f026c143fa0 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7f025c000c00 tx=0x7f025c02c920 comp rx=0 tx=0).stop 2023-10-26T01:44:54.999 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.997+0000 7f0272051640 1 -- 172.21.15.169:0/2565576220 shutdown_connections 2023-10-26T01:44:55.000 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.997+0000 7f0272051640 1 --2- 172.21.15.169:0/2565576220 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f024c06ae40 0x7f024c06d300 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:55.000 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.997+0000 7f0272051640 1 --2- 172.21.15.169:0/2565576220 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f026c157be0 0x7f026c14ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:55.000 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.997+0000 7f0272051640 1 --2- 172.21.15.169:0/2565576220 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f026c0aabf0 0x7f026c1437d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:55.000 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.997+0000 7f0272051640 1 --2- 172.21.15.169:0/2565576220 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f026c06ce50 0x7f026c143fa0 unknown :-1 s=CLOSED pgs=2014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:55.000 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.997+0000 7f0272051640 1 -- 172.21.15.169:0/2565576220 >> 172.21.15.169:0/2565576220 conn(0x7f026c069930 msgr2=0x7f026c0b0040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:55.001 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.997+0000 7f0272051640 1 -- 172.21.15.169:0/2565576220 shutdown_connections 2023-10-26T01:44:55.001 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:54.997+0000 7f0272051640 1 -- 172.21.15.169:0/2565576220 wait complete. 2023-10-26T01:44:55.018 INFO:tasks.ceph:Scrubbing osd.7 2023-10-26T01:44:55.018 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-10-26T01:44:55.228 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.226+0000 7f40c52e7640 1 -- 172.21.15.169:0/4094408158 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40c006caf0 msgr2=0x7f40c00b08f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:55.228 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.226+0000 7f40c52e7640 1 --2- 172.21.15.169:0/4094408158 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40c006caf0 0x7f40c00b08f0 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7f40c006e5b0 tx=0x7f40b802bcf0 comp rx=0 tx=0).stop 2023-10-26T01:44:55.228 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.226+0000 7f40c52e7640 1 -- 172.21.15.169:0/4094408158 shutdown_connections 2023-10-26T01:44:55.229 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.226+0000 7f40c52e7640 1 --2- 172.21.15.169:0/4094408158 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40c006caf0 0x7f40c00b08f0 unknown :-1 s=CLOSED pgs=2016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:55.229 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.226+0000 7f40c52e7640 1 --2- 172.21.15.169:0/4094408158 >> v2:172.21.15.170:3300/0 conn(0x7f40c0158060 0x7f40c00aad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:55.229 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.226+0000 7f40c52e7640 1 -- 172.21.15.169:0/4094408158 >> 172.21.15.169:0/4094408158 conn(0x7f40c00aeee0 msgr2=0x7f40c00af2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:55.229 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.226+0000 7f40c52e7640 1 -- 172.21.15.169:0/4094408158 shutdown_connections 2023-10-26T01:44:55.230 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.226+0000 7f40c52e7640 1 -- 172.21.15.169:0/4094408158 wait complete. 2023-10-26T01:44:55.232 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.228+0000 7f40c52e7640 1 Processor -- start 2023-10-26T01:44:55.232 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.228+0000 7f40c52e7640 1 -- start start 2023-10-26T01:44:55.232 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.229+0000 7f40c52e7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f40c006c680 0x7f40c014c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:55.232 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.229+0000 7f40c52e7640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40c006caf0 0x7f40c0152e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:55.233 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.229+0000 7f40c52e7640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f40c0158060 0x7f40c0153350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:55.233 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.229+0000 7f40c52e7640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40c0153890 con 0x7f40c006caf0 2023-10-26T01:44:55.233 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.229+0000 7f40c52e7640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40c015b5f0 con 0x7f40c006c680 2023-10-26T01:44:55.233 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.229+0000 7f40c52e7640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40c015aa70 con 0x7f40c0158060 2023-10-26T01:44:55.233 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.229+0000 7f40b67fc640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40c006caf0 0x7f40c0152e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:55.234 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.229+0000 7f40b67fc640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40c006caf0 0x7f40c0152e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:51710/0 (socket says 172.21.15.169:51710) 2023-10-26T01:44:55.234 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.229+0000 7f40b67fc640 1 -- 172.21.15.169:0/861444227 learned_addr learned my addr 172.21.15.169:0/861444227 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:44:55.234 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.229+0000 7f40beffd640 1 --2- 172.21.15.169:0/861444227 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f40c006c680 0x7f40c014c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:55.234 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.229+0000 7f40bf7fe640 1 --2- 172.21.15.169:0/861444227 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f40c0158060 0x7f40c0153350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:55.234 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.229+0000 7f40b67fc640 1 -- 172.21.15.169:0/861444227 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f40c0158060 msgr2=0x7f40c0153350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:55.235 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.229+0000 7f40b67fc640 1 --2- 172.21.15.169:0/861444227 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f40c0158060 0x7f40c0153350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:55.235 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.229+0000 7f40b67fc640 1 -- 172.21.15.169:0/861444227 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f40c006c680 msgr2=0x7f40c014c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:55.235 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.229+0000 7f40b67fc640 1 --2- 172.21.15.169:0/861444227 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f40c006c680 0x7f40c014c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:55.235 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.229+0000 7f40b67fc640 1 -- 172.21.15.169:0/861444227 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40b8009490 con 0x7f40c006caf0 2023-10-26T01:44:55.235 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.229+0000 7f40beffd640 1 --2- 172.21.15.169:0/861444227 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f40c006c680 0x7f40c014c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:44:55.236 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.230+0000 7f40b67fc640 1 --2- 172.21.15.169:0/861444227 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40c006caf0 0x7f40c0152e10 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x7f40a800c430 tx=0x7f40a800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:55.236 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.230+0000 7f40b7fff640 1 -- 172.21.15.169:0/861444227 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a8011790 con 0x7f40c006caf0 2023-10-26T01:44:55.236 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.230+0000 7f40b7fff640 1 -- 172.21.15.169:0/861444227 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f40a8011a00 con 0x7f40c006caf0 2023-10-26T01:44:55.236 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.230+0000 7f40b7fff640 1 -- 172.21.15.169:0/861444227 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a8011e10 con 0x7f40c006caf0 2023-10-26T01:44:55.236 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.230+0000 7f40c52e7640 1 -- 172.21.15.169:0/861444227 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40c015ad50 con 0x7f40c006caf0 2023-10-26T01:44:55.237 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.230+0000 7f40c52e7640 1 -- 172.21.15.169:0/861444227 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40c015b310 con 0x7f40c006caf0 2023-10-26T01:44:55.238 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.231+0000 7f40c52e7640 1 -- 172.21.15.169:0/861444227 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f40c004da70 con 0x7f40c006caf0 2023-10-26T01:44:55.238 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.232+0000 7f40b7fff640 1 -- 172.21.15.169:0/861444227 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f40a801c460 con 0x7f40c006caf0 2023-10-26T01:44:55.239 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.233+0000 7f40b7fff640 1 --2- 172.21.15.169:0/861444227 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f40a006ae40 0x7f40a006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:55.239 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.233+0000 7f40b7fff640 1 -- 172.21.15.169:0/861444227 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f40a8089ee0 con 0x7f40c006caf0 2023-10-26T01:44:55.239 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.233+0000 7f40b7fff640 1 --2- 172.21.15.169:0/861444227 >> [v2:172.21.15.170:6821/1115930619,v1:172.21.15.170:6822/1115930619] conn(0x7f40a0070ca0 0x7f40a00730c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:55.239 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.233+0000 7f40b7fff640 1 -- 172.21.15.169:0/861444227 --> [v2:172.21.15.170:6821/1115930619,v1:172.21.15.170:6822/1115930619] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f40a0073770 con 0x7f40a0070ca0 2023-10-26T01:44:55.239 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.233+0000 7f40b7fff640 1 -- 172.21.15.169:0/861444227 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_get_version_reply(handle=1 version=189) v2 ==== 24+0+0 (secure 0 0 0) 0x7f40a808a250 con 0x7f40c006caf0 2023-10-26T01:44:55.240 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.234+0000 7f40beffd640 1 --2- 172.21.15.169:0/861444227 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f40a006ae40 0x7f40a006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:55.240 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.234+0000 7f40bf7fe640 1 --2- 172.21.15.169:0/861444227 >> [v2:172.21.15.170:6821/1115930619,v1:172.21.15.170:6822/1115930619] conn(0x7f40a0070ca0 0x7f40a00730c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:55.240 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.235+0000 7f40bf7fe640 1 --2- 172.21.15.169:0/861444227 >> [v2:172.21.15.170:6821/1115930619,v1:172.21.15.170:6822/1115930619] conn(0x7f40a0070ca0 0x7f40a00730c0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:55.240 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.235+0000 7f40beffd640 1 --2- 172.21.15.169:0/861444227 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f40a006ae40 0x7f40a006d300 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f40b0005fd0 tx=0x7f40b00093e0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:55.241 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.237+0000 7f40b7fff640 1 -- 172.21.15.169:0/861444227 <== osd.7 v2:172.21.15.170:6821/1115930619 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23595 (crc 0 0 0) 0x7f40a0073770 con 0x7f40a0070ca0 2023-10-26T01:44:55.252 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.250+0000 7f40c52e7640 1 -- 172.21.15.169:0/861444227 --> [v2:172.21.15.170:6821/1115930619,v1:172.21.15.170:6822/1115930619] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f40c0066af0 con 0x7f40a0070ca0 2023-10-26T01:44:55.253 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.251+0000 7f40b7fff640 1 -- 172.21.15.169:0/861444227 <== osd.7 v2:172.21.15.170:6821/1115930619 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+632 (crc 0 0 0) 0x7f40c0066af0 con 0x7f40a0070ca0 2023-10-26T01:44:55.253 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2023-10-26T01:44:55.254 INFO:teuthology.orchestra.run.smithi169.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_pg_delete_cost = '' (not observed, change may require restart) osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2023-10-26T01:44:55.254 INFO:teuthology.orchestra.run.smithi169.stdout:} 2023-10-26T01:44:55.255 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.252+0000 7f40c52e7640 1 -- 172.21.15.169:0/861444227 >> [v2:172.21.15.170:6821/1115930619,v1:172.21.15.170:6822/1115930619] conn(0x7f40a0070ca0 msgr2=0x7f40a00730c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:55.255 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.252+0000 7f40c52e7640 1 --2- 172.21.15.169:0/861444227 >> [v2:172.21.15.170:6821/1115930619,v1:172.21.15.170:6822/1115930619] conn(0x7f40a0070ca0 0x7f40a00730c0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:55.255 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.252+0000 7f40c52e7640 1 -- 172.21.15.169:0/861444227 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f40a006ae40 msgr2=0x7f40a006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:55.256 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.252+0000 7f40c52e7640 1 --2- 172.21.15.169:0/861444227 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f40a006ae40 0x7f40a006d300 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f40b0005fd0 tx=0x7f40b00093e0 comp rx=0 tx=0).stop 2023-10-26T01:44:55.256 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.252+0000 7f40c52e7640 1 -- 172.21.15.169:0/861444227 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40c006caf0 msgr2=0x7f40c0152e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:55.256 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.252+0000 7f40c52e7640 1 --2- 172.21.15.169:0/861444227 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40c006caf0 0x7f40c0152e10 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x7f40a800c430 tx=0x7f40a800c8f0 comp rx=0 tx=0).stop 2023-10-26T01:44:55.256 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.252+0000 7f40bf7fe640 1 -- 172.21.15.169:0/861444227 reap_dead start 2023-10-26T01:44:55.256 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.252+0000 7f40c52e7640 1 -- 172.21.15.169:0/861444227 shutdown_connections 2023-10-26T01:44:55.257 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.252+0000 7f40c52e7640 1 -- 172.21.15.169:0/861444227 >> 172.21.15.169:0/861444227 conn(0x7f40c00aeee0 msgr2=0x7f40c006b130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:55.257 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.252+0000 7f40c52e7640 1 -- 172.21.15.169:0/861444227 shutdown_connections 2023-10-26T01:44:55.257 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.252+0000 7f40c52e7640 1 -- 172.21.15.169:0/861444227 wait complete. 2023-10-26T01:44:55.271 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-10-26T01:44:55.486 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.484+0000 7fbbdd485640 1 -- 172.21.15.169:0/2544283577 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbbc40039a0 msgr2=0x7fbbc4003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:55.487 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.484+0000 7fbbdd485640 1 --2- 172.21.15.169:0/2544283577 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbbc40039a0 0x7fbbc4003dc0 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 crypto rx=0x7fbbc8009b50 tx=0x7fbbc40057f0 comp rx=0 tx=0).stop 2023-10-26T01:44:55.487 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.484+0000 7fbbdcc84640 1 -- 172.21.15.169:0/2544283577 reap_dead start 2023-10-26T01:44:55.487 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.484+0000 7fbbdd485640 1 -- 172.21.15.169:0/2544283577 shutdown_connections 2023-10-26T01:44:55.487 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.484+0000 7fbbdd485640 1 -- 172.21.15.169:0/2544283577 >> 172.21.15.169:0/2544283577 conn(0x7fbbd806ccc0 msgr2=0x7fbbd806d0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:55.487 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.484+0000 7fbbdd485640 1 -- 172.21.15.169:0/2544283577 shutdown_connections 2023-10-26T01:44:55.488 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.484+0000 7fbbdd485640 1 -- 172.21.15.169:0/2544283577 wait complete. 2023-10-26T01:44:55.489 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.486+0000 7fbbdd485640 1 Processor -- start 2023-10-26T01:44:55.489 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.486+0000 7fbbdd485640 1 -- start start 2023-10-26T01:44:55.489 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.487+0000 7fbbdd485640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbbc40039a0 0x7fbbd81f1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:55.490 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.487+0000 7fbbdd485640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbbd8140c70 0x7fbbd80b11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:55.490 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.487+0000 7fbbdd485640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbbd81410f0 0x7fbbd80b16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:55.490 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.487+0000 7fbbdd485640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbd81407e0 con 0x7fbbd81410f0 2023-10-26T01:44:55.490 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.487+0000 7fbbdd485640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbd81f6440 con 0x7fbbd8140c70 2023-10-26T01:44:55.491 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.487+0000 7fbbdd485640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbd81e69c0 con 0x7fbbc40039a0 2023-10-26T01:44:55.491 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.487+0000 7fbbdcc84640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbbd81410f0 0x7fbbd80b16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:55.491 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.488+0000 7fbbdcc84640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbbd81410f0 0x7fbbd80b16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:51718/0 (socket says 172.21.15.169:51718) 2023-10-26T01:44:55.491 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.488+0000 7fbbdcc84640 1 -- 172.21.15.169:0/3051541409 learned_addr learned my addr 172.21.15.169:0/3051541409 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:44:55.493 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.488+0000 7fbbd7fff640 1 --2- 172.21.15.169:0/3051541409 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbbc40039a0 0x7fbbd81f1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:55.494 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.488+0000 7fbbcffff640 1 --2- 172.21.15.169:0/3051541409 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbbd8140c70 0x7fbbd80b11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:55.494 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.488+0000 7fbbdcc84640 1 -- 172.21.15.169:0/3051541409 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbbc40039a0 msgr2=0x7fbbd81f1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:55.494 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.488+0000 7fbbdcc84640 1 --2- 172.21.15.169:0/3051541409 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbbc40039a0 0x7fbbd81f1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:55.494 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.488+0000 7fbbdcc84640 1 -- 172.21.15.169:0/3051541409 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbbd8140c70 msgr2=0x7fbbd80b11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:55.495 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.488+0000 7fbbdcc84640 1 --2- 172.21.15.169:0/3051541409 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbbd8140c70 0x7fbbd80b11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:55.495 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.488+0000 7fbbdcc84640 1 -- 172.21.15.169:0/3051541409 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbd8140190 con 0x7fbbd81410f0 2023-10-26T01:44:55.495 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.488+0000 7fbbd7fff640 1 --2- 172.21.15.169:0/3051541409 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbbc40039a0 0x7fbbd81f1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:44:55.495 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.488+0000 7fbbcffff640 1 --2- 172.21.15.169:0/3051541409 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbbd8140c70 0x7fbbd80b11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:44:55.495 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.488+0000 7fbbdcc84640 1 --2- 172.21.15.169:0/3051541409 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbbd81410f0 0x7fbbd80b16e0 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7fbbd80b4210 tx=0x7fbbd000e890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:55.496 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.488+0000 7fbbd57fa640 1 -- 172.21.15.169:0/3051541409 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd000c070 con 0x7fbbd81410f0 2023-10-26T01:44:55.496 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.489+0000 7fbbd57fa640 1 -- 172.21.15.169:0/3051541409 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fbbd0013610 con 0x7fbbd81410f0 2023-10-26T01:44:55.496 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.489+0000 7fbbd57fa640 1 -- 172.21.15.169:0/3051541409 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd0013a20 con 0x7fbbd81410f0 2023-10-26T01:44:55.496 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.489+0000 7fbbdd485640 1 -- 172.21.15.169:0/3051541409 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbd8140470 con 0x7fbbd81410f0 2023-10-26T01:44:55.497 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.489+0000 7fbbdd485640 1 -- 172.21.15.169:0/3051541409 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbd80c42c0 con 0x7fbbd81410f0 2023-10-26T01:44:55.497 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.490+0000 7fbbdd485640 1 -- 172.21.15.169:0/3051541409 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbd8066af0 con 0x7fbbd81410f0 2023-10-26T01:44:55.497 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.491+0000 7fbbd57fa640 1 -- 172.21.15.169:0/3051541409 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fbbd001c3b0 con 0x7fbbd81410f0 2023-10-26T01:44:55.497 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.492+0000 7fbbd57fa640 1 --2- 172.21.15.169:0/3051541409 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbbb806ae70 0x7fbbb806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:55.497 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.492+0000 7fbbd57fa640 1 -- 172.21.15.169:0/3051541409 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fbbd00885e0 con 0x7fbbd81410f0 2023-10-26T01:44:55.498 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.493+0000 7fbbd7fff640 1 --2- 172.21.15.169:0/3051541409 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbbb806ae70 0x7fbbb806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:55.498 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.494+0000 7fbbd7fff640 1 --2- 172.21.15.169:0/3051541409 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbbb806ae70 0x7fbbb806d330 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fbbc8009b50 tx=0x7fbbc8035040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:55.499 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.496+0000 7fbbd57fa640 1 -- 172.21.15.169:0/3051541409 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbbd0053990 con 0x7fbbd81410f0 2023-10-26T01:44:55.686 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.684+0000 7fbbdd485640 1 -- 172.21.15.169:0/3051541409 --> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7fbbd80cbde0 con 0x7fbbb806ae70 2023-10-26T01:44:55.686 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.685+0000 7fbbd57fa640 1 -- 172.21.15.169:0/3051541409 <== mgr.4112 v2:172.21.15.170:6832/1187156918 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fbbc8004020 con 0x7fbbb806ae70 2023-10-26T01:44:55.687 INFO:teuthology.orchestra.run.smithi169.stderr:instructed osd(s) 7 to deep-scrub 2023-10-26T01:44:55.689 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.687+0000 7fbbdd485640 1 -- 172.21.15.169:0/3051541409 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbbb806ae70 msgr2=0x7fbbb806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:55.690 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.687+0000 7fbbdd485640 1 --2- 172.21.15.169:0/3051541409 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbbb806ae70 0x7fbbb806d330 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fbbc8009b50 tx=0x7fbbc8035040 comp rx=0 tx=0).stop 2023-10-26T01:44:55.690 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.687+0000 7fbbdd485640 1 -- 172.21.15.169:0/3051541409 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbbd81410f0 msgr2=0x7fbbd80b16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:55.690 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.687+0000 7fbbdd485640 1 --2- 172.21.15.169:0/3051541409 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbbd81410f0 0x7fbbd80b16e0 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7fbbd80b4210 tx=0x7fbbd000e890 comp rx=0 tx=0).stop 2023-10-26T01:44:55.690 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.688+0000 7fbbdd485640 1 -- 172.21.15.169:0/3051541409 shutdown_connections 2023-10-26T01:44:55.691 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.688+0000 7fbbdd485640 1 --2- 172.21.15.169:0/3051541409 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fbbb806ae70 0x7fbbb806d330 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:55.691 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.688+0000 7fbbdd485640 1 --2- 172.21.15.169:0/3051541409 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbbd81410f0 0x7fbbd80b16e0 unknown :-1 s=CLOSED pgs=2020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:55.691 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.688+0000 7fbbdd485640 1 --2- 172.21.15.169:0/3051541409 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbbd8140c70 0x7fbbd80b11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:55.691 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.688+0000 7fbbdd485640 1 --2- 172.21.15.169:0/3051541409 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbbc40039a0 0x7fbbd81f1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:55.691 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.688+0000 7fbbdd485640 1 -- 172.21.15.169:0/3051541409 >> 172.21.15.169:0/3051541409 conn(0x7fbbd806ccc0 msgr2=0x7fbbd80cce90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:55.692 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.688+0000 7fbbdd485640 1 -- 172.21.15.169:0/3051541409 shutdown_connections 2023-10-26T01:44:55.692 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.688+0000 7fbbdd485640 1 -- 172.21.15.169:0/3051541409 wait complete. 2023-10-26T01:44:55.708 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-26T01:44:55.922 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.919+0000 7f264edc0640 1 -- 172.21.15.169:0/2060876063 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26480cdc30 msgr2=0x7f26480cbac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:55.922 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.919+0000 7f264edc0640 1 --2- 172.21.15.169:0/2060876063 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26480cdc30 0x7f26480cbac0 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7f2640005fd0 tx=0x7f264002c060 comp rx=0 tx=0).stop 2023-10-26T01:44:55.922 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.920+0000 7f264edc0640 1 -- 172.21.15.169:0/2060876063 shutdown_connections 2023-10-26T01:44:55.923 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.920+0000 7f264edc0640 1 --2- 172.21.15.169:0/2060876063 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26480cdc30 0x7f26480cbac0 unknown :-1 s=CLOSED pgs=2022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:55.923 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.920+0000 7f264edc0640 1 --2- 172.21.15.169:0/2060876063 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f26480ac930 0x7f26480cd6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:55.923 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.920+0000 7f264edc0640 1 -- 172.21.15.169:0/2060876063 >> 172.21.15.169:0/2060876063 conn(0x7f2648069930 msgr2=0x7f2648069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:55.923 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.920+0000 7f264edc0640 1 -- 172.21.15.169:0/2060876063 shutdown_connections 2023-10-26T01:44:55.923 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.920+0000 7f264edc0640 1 -- 172.21.15.169:0/2060876063 wait complete. 2023-10-26T01:44:55.925 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.922+0000 7f264edc0640 1 Processor -- start 2023-10-26T01:44:55.925 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.922+0000 7f264edc0640 1 -- start start 2023-10-26T01:44:55.925 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.923+0000 7f264edc0640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26480ac930 0x7f26481512c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:55.926 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.923+0000 7f264edc0640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f26480cdc30 0x7f26480c6f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:55.926 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.923+0000 7f264edc0640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f26481592d0 0x7f26480c74c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:55.926 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.923+0000 7f264edc0640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26481e6cb0 con 0x7f26480ac930 2023-10-26T01:44:55.926 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.923+0000 7f264edc0640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26481e69d0 con 0x7f26481592d0 2023-10-26T01:44:55.926 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.923+0000 7f264edc0640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26480af060 con 0x7f26480cdc30 2023-10-26T01:44:55.927 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.923+0000 7f264cb5c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26480ac930 0x7f26481512c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:55.927 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.923+0000 7f264cb5c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26480ac930 0x7f26481512c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:51732/0 (socket says 172.21.15.169:51732) 2023-10-26T01:44:55.927 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.923+0000 7f264cb5c640 1 -- 172.21.15.169:0/2835388448 learned_addr learned my addr 172.21.15.169:0/2835388448 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:44:55.927 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.923+0000 7f264cb5c640 1 -- 172.21.15.169:0/2835388448 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f26480cdc30 msgr2=0x7f26480c6f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:55.928 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.923+0000 7f264cb5c640 1 --2- 172.21.15.169:0/2835388448 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f26480cdc30 0x7f26480c6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:55.928 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.923+0000 7f264cb5c640 1 -- 172.21.15.169:0/2835388448 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f26481592d0 msgr2=0x7f26480c74c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:55.928 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.923+0000 7f264cb5c640 1 --2- 172.21.15.169:0/2835388448 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f26481592d0 0x7f26480c74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:55.928 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.923+0000 7f264cb5c640 1 -- 172.21.15.169:0/2835388448 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2640009840 con 0x7f26480ac930 2023-10-26T01:44:55.929 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.924+0000 7f264cb5c640 1 --2- 172.21.15.169:0/2835388448 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26480ac930 0x7f26481512c0 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x7f263c00c3e0 tx=0x7f263c00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:55.930 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.924+0000 7f26457fa640 1 -- 172.21.15.169:0/2835388448 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f263c011790 con 0x7f26480ac930 2023-10-26T01:44:55.930 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.924+0000 7f26457fa640 1 -- 172.21.15.169:0/2835388448 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f263c0119e0 con 0x7f26480ac930 2023-10-26T01:44:55.930 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.924+0000 7f264edc0640 1 -- 172.21.15.169:0/2835388448 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26480af340 con 0x7f26480ac930 2023-10-26T01:44:55.931 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.924+0000 7f26457fa640 1 -- 172.21.15.169:0/2835388448 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f263c011e30 con 0x7f26480ac930 2023-10-26T01:44:55.931 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.924+0000 7f264edc0640 1 -- 172.21.15.169:0/2835388448 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2648151800 con 0x7f26480ac930 2023-10-26T01:44:55.931 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.925+0000 7f264edc0640 1 -- 172.21.15.169:0/2835388448 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26480aeec0 con 0x7f26480ac930 2023-10-26T01:44:55.931 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.926+0000 7f26457fa640 1 -- 172.21.15.169:0/2835388448 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f263c01c460 con 0x7f26480ac930 2023-10-26T01:44:55.932 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.927+0000 7f26457fa640 1 --2- 172.21.15.169:0/2835388448 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f262c06ae70 0x7f262c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:44:55.932 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.927+0000 7f26457fa640 1 -- 172.21.15.169:0/2835388448 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f263c089380 con 0x7f26480ac930 2023-10-26T01:44:55.932 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.928+0000 7f2647fff640 1 --2- 172.21.15.169:0/2835388448 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f262c06ae70 0x7f262c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:44:55.932 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.929+0000 7f2647fff640 1 --2- 172.21.15.169:0/2835388448 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f262c06ae70 0x7f262c06d330 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f2634005fd0 tx=0x7f2634005c70 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:44:55.933 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:55.931+0000 7f26457fa640 1 -- 172.21.15.169:0/2835388448 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f26480aeec0 con 0x7f26480ac930 2023-10-26T01:44:56.120 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:56.119+0000 7f264edc0640 1 -- 172.21.15.169:0/2835388448 --> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f26480af860 con 0x7f262c06ae70 2023-10-26T01:44:56.127 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:56.125+0000 7f26457fa640 1 -- 172.21.15.169:0/2835388448 <== mgr.4112 v2:172.21.15.170:6832/1187156918 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+47063 (secure 0 0 0) 0x7f26480af860 con 0x7f262c06ae70 2023-10-26T01:44:56.128 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:44:56.128 INFO:teuthology.orchestra.run.smithi169.stderr:dumped all 2023-10-26T01:44:56.130 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:56.128+0000 7f264edc0640 1 -- 172.21.15.169:0/2835388448 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f262c06ae70 msgr2=0x7f262c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:56.130 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:56.128+0000 7f264edc0640 1 --2- 172.21.15.169:0/2835388448 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f262c06ae70 0x7f262c06d330 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f2634005fd0 tx=0x7f2634005c70 comp rx=0 tx=0).stop 2023-10-26T01:44:56.130 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:56.128+0000 7f264edc0640 1 -- 172.21.15.169:0/2835388448 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26480ac930 msgr2=0x7f26481512c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:44:56.130 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:56.128+0000 7f264edc0640 1 --2- 172.21.15.169:0/2835388448 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26480ac930 0x7f26481512c0 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x7f263c00c3e0 tx=0x7f263c00c8a0 comp rx=0 tx=0).stop 2023-10-26T01:44:56.131 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:56.128+0000 7f264edc0640 1 -- 172.21.15.169:0/2835388448 shutdown_connections 2023-10-26T01:44:56.131 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:56.128+0000 7f264edc0640 1 --2- 172.21.15.169:0/2835388448 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f262c06ae70 0x7f262c06d330 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:56.131 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:56.128+0000 7f264edc0640 1 --2- 172.21.15.169:0/2835388448 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f26481592d0 0x7f26480c74c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:56.131 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:56.128+0000 7f264edc0640 1 --2- 172.21.15.169:0/2835388448 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f26480cdc30 0x7f26480c6f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:56.131 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:56.128+0000 7f264edc0640 1 --2- 172.21.15.169:0/2835388448 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26480ac930 0x7f26481512c0 unknown :-1 s=CLOSED pgs=2023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:44:56.132 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:56.128+0000 7f264edc0640 1 -- 172.21.15.169:0/2835388448 >> 172.21.15.169:0/2835388448 conn(0x7f2648069930 msgr2=0x7f26480aa3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:44:56.132 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:56.128+0000 7f264edc0640 1 -- 172.21.15.169:0/2835388448 shutdown_connections 2023-10-26T01:44:56.132 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:44:56.128+0000 7f264edc0640 1 -- 172.21.15.169:0/2835388448 wait complete. 2023-10-26T01:44:56.148 INFO:teuthology.orchestra.run.smithi169.stdout:{"pg_ready":true,"pg_map":{"version":614,"stamp":"2023-10-26T01:44:54.442582+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":6,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":9496,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1553003,"statfs":{"total":1600000000,"available":1590275790,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.223611"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":333,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T01:44:42.841516+0000","last_change":"2023-10-26T01:30:34.883484+0000","last_active":"2023-10-26T01:44:42.841516+0000","last_peered":"2023-10-26T01:44:42.841516+0000","last_clean":"2023-10-26T01:44:42.841516+0000","last_became_active":"2023-10-26T01:30:34.883071+0000","last_became_peered":"2023-10-26T01:30:34.883071+0000","last_unstale":"2023-10-26T01:44:42.841516+0000","last_undegraded":"2023-10-26T01:44:42.841516+0000","last_fullsized":"2023-10-26T01:44:42.841516+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T01:30:32.890096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T01:30:32.890096+0000","last_clean_scrub_stamp":"2023-10-26T01:30:32.890096+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:26:31.879452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":14,"seq":60129542319,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1042,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194269,"statfs":{"total":200000000,"available":198931989,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:43:57 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.84599999999999997,"15min":0.80700000000000005},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.33000000000000002},"max":{"1min":1.391,"5min":1.571,"15min":2.2549999999999999},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.98799999999999999,"15min":0.91900000000000004},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.309},"max":{"1min":1.607,"5min":1.919,"15min":1.919},"last":0.879}]},{"osd":1,"last update":"Thu Oct 26 01:43:57 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.90800000000000003,"15min":0.89800000000000002},"min":{"1min":0.38,"5min":0.38,"15min":0.32300000000000001},"max":{"1min":1.411,"5min":1.569,"15min":4.4340000000000002},"last":1.2210000000000001},{"interface":"front","average":{"1min":0.99099999999999999,"5min":1.0640000000000001,"15min":1.01},"min":{"1min":0.46800000000000003,"5min":0.38600000000000001,"15min":0.374},"max":{"1min":1.3440000000000001,"5min":1.7070000000000001,"15min":2.3330000000000002},"last":1.1890000000000001}]},{"osd":2,"last update":"Thu Oct 26 01:43:57 2023","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":1.0629999999999999,"15min":1.0529999999999999},"min":{"1min":0.44,"5min":0.44,"15min":0.39900000000000002},"max":{"1min":1.3049999999999999,"5min":1.6819999999999999,"15min":4.2800000000000002},"last":0.75700000000000001},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.159,"15min":1.1259999999999999},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.8009999999999999,"5min":1.8129999999999999,"15min":2.302},"last":1.3160000000000001}]},{"osd":4,"last update":"Thu Oct 26 01:43:57 2023","interfaces":[{"interface":"back","average":{"1min":1.133,"5min":1.2090000000000001,"15min":1.1950000000000001},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.7150000000000001,"5min":1.7150000000000001,"15min":2.1070000000000002},"last":1.45},{"interface":"front","average":{"1min":1.3160000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.7290000000000001,"5min":1.7509999999999999,"15min":4.8250000000000002},"last":1.4870000000000001}]},{"osd":5,"last update":"Thu Oct 26 01:43:57 2023","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.3169999999999999,"15min":1.306},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.72499999999999998},"max":{"1min":1.998,"5min":1.998,"15min":4.9080000000000004},"last":1.4199999999999999},{"interface":"front","average":{"1min":1.325,"5min":1.351,"15min":1.3029999999999999},"min":{"1min":0.81299999999999994,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.778,"5min":1.9279999999999999,"15min":2.8980000000000001},"last":1.651}]},{"osd":6,"last update":"Thu Oct 26 01:43:57 2023","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.448,"15min":1.371},"min":{"1min":0.751,"5min":0.751,"15min":0.70199999999999996},"max":{"1min":1.845,"5min":3.5800000000000001,"15min":3.5800000000000001},"last":1.548},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.3480000000000001,"15min":1.3},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.7769999999999999,"5min":1.806,"15min":2.181},"last":1.2729999999999999}]},{"osd":7,"last update":"Thu Oct 26 01:43:57 2023","interfaces":[{"interface":"back","average":{"1min":1.3140000000000001,"5min":1.3859999999999999,"15min":1.3400000000000001},"min":{"1min":0.745,"5min":0.745,"15min":0.71999999999999997},"max":{"1min":1.845,"5min":2.024,"15min":2.0680000000000001},"last":1.3620000000000001},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.419,"15min":1.401},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":2.0579999999999998,"5min":2.0579999999999998,"15min":4.8730000000000002},"last":1.577}]}]},{"osd":7,"up_from":13,"seq":55834575022,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1619,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193692,"statfs":{"total":200000000,"available":198341621,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:44:03 2023","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":1.171,"15min":1.133},"min":{"1min":0.60599999999999998,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.5740000000000001,"5min":2.2269999999999999,"15min":2.2269999999999999},"last":0.751},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.2270000000000001,"15min":1.143},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.58699999999999997},"max":{"1min":1.6020000000000001,"5min":2.282,"15min":2.6779999999999999},"last":0.80000000000000004}]},{"osd":1,"last update":"Thu Oct 26 01:44:03 2023","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.1679999999999999,"15min":1.147},"min":{"1min":0.83599999999999997,"5min":0.65600000000000003,"15min":0.65200000000000002},"max":{"1min":1.5800000000000001,"5min":1.8,"15min":2.8260000000000001},"last":0.877},{"interface":"front","average":{"1min":1.1759999999999999,"5min":1.23,"15min":1.2130000000000001},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.61199999999999999},"max":{"1min":1.5129999999999999,"5min":2.0059999999999998,"15min":3.2730000000000001},"last":1.1850000000000001}]},{"osd":2,"last update":"Thu Oct 26 01:44:03 2023","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.2150000000000001,"15min":1.1739999999999999},"min":{"1min":0.77900000000000003,"5min":0.68999999999999995,"15min":0.58499999999999996},"max":{"1min":1.794,"5min":1.794,"15min":2.218},"last":0.84199999999999997},{"interface":"front","average":{"1min":1.319,"5min":1.29,"15min":1.2709999999999999},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.73099999999999998},"max":{"1min":1.8160000000000001,"5min":2.234,"15min":3.0510000000000002},"last":0.98399999999999999}]},{"osd":3,"last update":"Thu Oct 26 01:44:03 2023","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.325,"15min":1.2729999999999999},"min":{"1min":0.91400000000000003,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.6819999999999999,"5min":2.1379999999999999,"15min":2.1379999999999999},"last":0.95099999999999996},{"interface":"front","average":{"1min":1.391,"5min":1.4419999999999999,"15min":1.3919999999999999},"min":{"1min":0.98899999999999999,"5min":0.92200000000000004,"15min":0.65700000000000003},"max":{"1min":1.768,"5min":2.3050000000000002,"15min":5.5039999999999996},"last":1.0329999999999999}]},{"osd":4,"last update":"Thu Oct 26 01:44:03 2023","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.399,"15min":1.369},"min":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.76100000000000001},"max":{"1min":2.0259999999999998,"5min":2.54,"15min":2.77},"last":1.458},{"interface":"front","average":{"1min":1.3180000000000001,"5min":1.3979999999999999,"15min":1.399},"min":{"1min":0.89400000000000002,"5min":0.75700000000000001,"15min":0.46500000000000002},"max":{"1min":1.9099999999999999,"5min":2.8660000000000001,"15min":3.1419999999999999},"last":1.121}]},{"osd":5,"last update":"Thu Oct 26 01:44:03 2023","interfaces":[{"interface":"back","average":{"1min":1.4219999999999999,"5min":1.4239999999999999,"15min":1.4299999999999999},"min":{"1min":0.94899999999999995,"5min":0.86599999999999999,"15min":0.435},"max":{"1min":1.9379999999999999,"5min":2.3639999999999999,"15min":5.734},"last":1.27},{"interface":"front","average":{"1min":1.508,"5min":1.528,"15min":1.5009999999999999},"min":{"1min":1.1519999999999999,"5min":1.0920000000000001,"15min":0.51900000000000002},"max":{"1min":1.9810000000000001,"5min":2.5169999999999999,"15min":2.8980000000000001},"last":1.343}]},{"osd":6,"last update":"Thu Oct 26 01:44:03 2023","interfaces":[{"interface":"back","average":{"1min":1.5860000000000001,"5min":1.5700000000000001,"15min":1.661},"min":{"1min":1.292,"5min":1.038,"15min":0.47399999999999998},"max":{"1min":2.0459999999999998,"5min":2.5259999999999998,"15min":11.577999999999999},"last":1.4330000000000001},{"interface":"front","average":{"1min":1.631,"5min":1.6579999999999999,"15min":1.722},"min":{"1min":1.214,"5min":1.214,"15min":0.55100000000000005},"max":{"1min":2.0489999999999999,"5min":2.7250000000000001,"15min":10.704000000000001},"last":1.5169999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575022,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1042,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194270,"statfs":{"total":200000000,"available":198932603,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:43:58 2023","interfaces":[{"interface":"back","average":{"1min":1.1200000000000001,"5min":1.121,"15min":1.1279999999999999},"min":{"1min":0.747,"5min":0.69299999999999995,"15min":0.60399999999999998},"max":{"1min":1.536,"5min":2.2509999999999999,"15min":2.7370000000000001},"last":1.0309999999999999},{"interface":"front","average":{"1min":1.101,"5min":1.1759999999999999,"15min":1.143},"min":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.59199999999999997},"max":{"1min":1.53,"5min":2.2170000000000001,"15min":4.4779999999999998},"last":0.98699999999999999}]},{"osd":1,"last update":"Thu Oct 26 01:43:58 2023","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":1.1359999999999999,"15min":1.179},"min":{"1min":0.85499999999999998,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.6679999999999999,"5min":1.889,"15min":5.5359999999999996},"last":1.3460000000000001},{"interface":"front","average":{"1min":1.208,"5min":1.2390000000000001,"15min":1.208},"min":{"1min":0.93400000000000005,"5min":0.76400000000000001,"15min":0.64000000000000001},"max":{"1min":1.726,"5min":1.726,"15min":4.3579999999999997},"last":1.2849999999999999}]},{"osd":2,"last update":"Thu Oct 26 01:43:58 2023","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.153,"15min":1.141},"min":{"1min":0.73599999999999999,"5min":0.69799999999999995,"15min":0.59799999999999998},"max":{"1min":1.329,"5min":1.651,"15min":3.7879999999999998},"last":1.397},{"interface":"front","average":{"1min":1.272,"5min":1.2949999999999999,"15min":1.292},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.68899999999999995},"max":{"1min":1.7829999999999999,"5min":2.0110000000000001,"15min":3.0470000000000002},"last":1.2410000000000001}]},{"osd":3,"last update":"Thu Oct 26 01:43:58 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.266,"15min":1.2569999999999999},"min":{"1min":0.85299999999999998,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.4690000000000001,"5min":1.8200000000000001,"15min":4.4560000000000004},"last":1.1719999999999999},{"interface":"front","average":{"1min":1.3080000000000001,"5min":1.3580000000000001,"15min":1.341},"min":{"1min":1.038,"5min":0.93700000000000006,"15min":0.69499999999999995},"max":{"1min":1.8560000000000001,"5min":1.871,"15min":2.6909999999999998},"last":1.452}]},{"osd":4,"last update":"Thu Oct 26 01:43:58 2023","interfaces":[{"interface":"back","average":{"1min":1.1899999999999999,"5min":1.272,"15min":1.2430000000000001},"min":{"1min":0.84699999999999998,"5min":0.59899999999999998,"15min":0.48399999999999999},"max":{"1min":1.5580000000000001,"5min":1.825,"15min":2.9489999999999998},"last":1.6519999999999999},{"interface":"front","average":{"1min":1.288,"5min":1.339,"15min":1.347},"min":{"1min":0.94199999999999995,"5min":0.83499999999999996,"15min":0.59299999999999997},"max":{"1min":1.573,"5min":2.169,"15min":6.609},"last":1.492}]},{"osd":5,"last update":"Thu Oct 26 01:43:58 2023","interfaces":[{"interface":"back","average":{"1min":1.4670000000000001,"5min":1.5369999999999999,"15min":1.5009999999999999},"min":{"1min":1.202,"5min":0.90400000000000003,"15min":0.63700000000000001},"max":{"1min":1.677,"5min":2.04,"15min":5.4370000000000003},"last":1.593},{"interface":"front","average":{"1min":1.4370000000000001,"5min":1.4530000000000001,"15min":1.4399999999999999},"min":{"1min":1.052,"5min":0.69999999999999996,"15min":0.55200000000000005},"max":{"1min":1.921,"5min":2.1190000000000002,"15min":2.7759999999999998},"last":1.5509999999999999}]},{"osd":7,"last update":"Thu Oct 26 01:43:58 2023","interfaces":[{"interface":"back","average":{"1min":1.3320000000000001,"5min":1.419,"15min":1.427},"min":{"1min":0.92000000000000004,"5min":0.72399999999999998,"15min":0.54100000000000004},"max":{"1min":1.655,"5min":1.984,"15min":11.804},"last":1.7909999999999999},{"interface":"front","average":{"1min":1.4650000000000001,"5min":1.52,"15min":1.55},"min":{"1min":1.095,"5min":0.96399999999999997,"15min":0.56499999999999995},"max":{"1min":1.7270000000000001,"5min":1.9510000000000001,"15min":11.541},"last":1.7310000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575022,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1042,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194269,"statfs":{"total":200000000,"available":198931989,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:43:58 2023","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.1950000000000001,"15min":1.179},"min":{"1min":0.628,"5min":0.53400000000000003,"15min":0.53300000000000003},"max":{"1min":1.5940000000000001,"5min":1.724,"15min":3.8210000000000002},"last":0.74299999999999999},{"interface":"front","average":{"1min":1.284,"5min":1.298,"15min":1.252},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.67900000000000005},"max":{"1min":1.8140000000000001,"5min":1.8140000000000001,"15min":2.9889999999999999},"last":0.90600000000000003}]},{"osd":1,"last update":"Thu Oct 26 01:43:58 2023","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":1.2290000000000001,"15min":1.149},"min":{"1min":0.752,"5min":0.70999999999999996,"15min":0.65900000000000003},"max":{"1min":1.605,"5min":1.899,"15min":1.899},"last":0.79600000000000004},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.1200000000000001,"15min":1.1279999999999999},"min":{"1min":0.69499999999999995,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.514,"5min":1.708,"15min":2.4340000000000002},"last":0.95499999999999996}]},{"osd":2,"last update":"Thu Oct 26 01:43:58 2023","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.2170000000000001,"15min":1.1930000000000001},"min":{"1min":0.89700000000000002,"5min":0.76700000000000002,"15min":0.69299999999999995},"max":{"1min":1.7030000000000001,"5min":1.796,"15min":1.8220000000000001},"last":1.131},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.248,"15min":1.214},"min":{"1min":0.85599999999999998,"5min":0.62,"15min":0.62},"max":{"1min":1.6000000000000001,"5min":1.875,"15min":1.875},"last":1.087}]},{"osd":3,"last update":"Thu Oct 26 01:43:58 2023","interfaces":[{"interface":"back","average":{"1min":1.3759999999999999,"5min":1.304,"15min":1.28},"min":{"1min":0.94299999999999995,"5min":0.72399999999999998,"15min":0.72299999999999998},"max":{"1min":1.7569999999999999,"5min":1.7569999999999999,"15min":1.835},"last":1.496},{"interface":"front","average":{"1min":1.4119999999999999,"5min":1.3520000000000001,"15min":1.3129999999999999},"min":{"1min":1.03,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.9410000000000001,"5min":2.6669999999999998,"15min":2.9249999999999998},"last":1.038}]},{"osd":4,"last update":"Thu Oct 26 01:43:58 2023","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":1.2350000000000001,"15min":1.2370000000000001},"min":{"1min":0.94599999999999995,"5min":0.52000000000000002,"15min":0.49099999999999999},"max":{"1min":1.758,"5min":1.79,"15min":2.7770000000000001},"last":1.3740000000000001},{"interface":"front","average":{"1min":1.5620000000000001,"5min":1.3919999999999999,"15min":1.3839999999999999},"min":{"1min":0.98099999999999998,"5min":0.504,"15min":0.504},"max":{"1min":2.125,"5min":2.125,"15min":2.125},"last":1.4239999999999999}]},{"osd":6,"last update":"Thu Oct 26 01:43:58 2023","interfaces":[{"interface":"back","average":{"1min":1.51,"5min":1.4470000000000001,"15min":1.3919999999999999},"min":{"1min":1.1419999999999999,"5min":0.80000000000000004,"15min":0.51900000000000002},"max":{"1min":1.9590000000000001,"5min":2.4590000000000001,"15min":2.8250000000000002},"last":1.633},{"interface":"front","average":{"1min":1.5189999999999999,"5min":1.4490000000000001,"15min":1.4019999999999999},"min":{"1min":1.208,"5min":0.65500000000000003,"15min":0.61799999999999999},"max":{"1min":1.8759999999999999,"5min":2.3530000000000002,"15min":2.4830000000000001},"last":1.5740000000000001}]},{"osd":7,"last update":"Thu Oct 26 01:43:58 2023","interfaces":[{"interface":"back","average":{"1min":1.663,"5min":1.5229999999999999,"15min":1.4950000000000001},"min":{"1min":1.119,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":2.0409999999999999,"5min":2.0409999999999999,"15min":3.621},"last":1.6619999999999999},{"interface":"front","average":{"1min":1.6890000000000001,"5min":1.5800000000000001,"15min":1.52},"min":{"1min":1.4670000000000001,"5min":0.85299999999999998,"15min":0.59499999999999997},"max":{"1min":2.0539999999999998,"5min":2.0539999999999998,"15min":2.8010000000000002},"last":1.736}]}]},{"osd":4,"up_from":13,"seq":55834575022,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1042,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194269,"statfs":{"total":200000000,"available":198931989,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:43:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.1850000000000001,"15min":1.079},"min":{"1min":0.57699999999999996,"5min":0.503,"15min":0.503},"max":{"1min":1.6499999999999999,"5min":1.78,"15min":2.0289999999999999},"last":1.377},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.1639999999999999,"15min":1.087},"min":{"1min":0.85799999999999998,"5min":0.72599999999999998,"15min":0.60599999999999998},"max":{"1min":1.6639999999999999,"5min":1.921,"15min":2.6829999999999998},"last":1.0129999999999999}]},{"osd":1,"last update":"Thu Oct 26 01:43:49 2023","interfaces":[{"interface":"back","average":{"1min":1.167,"5min":1.2270000000000001,"15min":1.1699999999999999},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.56799999999999995},"max":{"1min":1.804,"5min":2.056,"15min":4.0519999999999996},"last":0.81399999999999995},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.274,"15min":1.1839999999999999},"min":{"1min":0.77800000000000002,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.599,"5min":2.4390000000000001,"15min":2.4390000000000001},"last":1.073}]},{"osd":2,"last update":"Thu Oct 26 01:43:49 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.3839999999999999,"15min":1.3149999999999999},"min":{"1min":0.88100000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":2.048,"5min":2.2040000000000002,"15min":5.6749999999999998},"last":1.27},{"interface":"front","average":{"1min":1.196,"5min":1.224,"15min":1.1870000000000001},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.7190000000000001,"5min":1.986,"15min":3.673},"last":1.135}]},{"osd":3,"last update":"Thu Oct 26 01:43:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.357,"15min":1.2529999999999999},"min":{"1min":0.64700000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.708,"5min":2.5960000000000001,"15min":2.5960000000000001},"last":1.181},{"interface":"front","average":{"1min":1.196,"5min":1.2509999999999999,"15min":1.202},"min":{"1min":0.78600000000000003,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.589,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":0.94899999999999995}]},{"osd":5,"last update":"Thu Oct 26 01:43:49 2023","interfaces":[{"interface":"back","average":{"1min":1.228,"5min":1.2949999999999999,"15min":1.278},"min":{"1min":0.876,"5min":0.71099999999999997,"15min":0.46300000000000002},"max":{"1min":1.8640000000000001,"5min":2.774,"15min":2.774},"last":1.5049999999999999},{"interface":"front","average":{"1min":1.272,"5min":1.355,"15min":1.3320000000000001},"min":{"1min":0.93000000000000005,"5min":0.77700000000000002,"15min":0.55200000000000005},"max":{"1min":1.6539999999999999,"5min":2.3109999999999999,"15min":5.8579999999999997},"last":1.1040000000000001}]},{"osd":6,"last update":"Thu Oct 26 01:43:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3180000000000001,"5min":1.371,"15min":1.3620000000000001},"min":{"1min":0.94899999999999995,"5min":0.69499999999999995,"15min":0.59299999999999997},"max":{"1min":1.76,"5min":2.3780000000000001,"15min":12.343999999999999},"last":1.3340000000000001},{"interface":"front","average":{"1min":1.5389999999999999,"5min":1.6000000000000001,"15min":1.5660000000000001},"min":{"1min":1.113,"5min":0.80000000000000004,"15min":0.55000000000000004},"max":{"1min":2.2799999999999998,"5min":2.9020000000000001,"15min":12.605},"last":1.5669999999999999}]},{"osd":7,"last update":"Thu Oct 26 01:43:49 2023","interfaces":[{"interface":"back","average":{"1min":1.466,"5min":1.5740000000000001,"15min":1.522},"min":{"1min":1.0649999999999999,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":2.044,"5min":2.7200000000000002,"15min":5.774},"last":1.454},{"interface":"front","average":{"1min":1.468,"5min":1.542,"15min":1.4990000000000001},"min":{"1min":0.79500000000000004,"5min":0.79000000000000004,"15min":0.68000000000000005},"max":{"1min":1.8500000000000001,"5min":2.8149999999999999,"15min":4.6520000000000001},"last":1.421}]}]},{"osd":2,"up_from":13,"seq":55834575022,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1042,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194269,"statfs":{"total":200000000,"available":198931989,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:44:05 2023","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.81899999999999995,"15min":0.84099999999999997},"min":{"1min":0.53300000000000003,"5min":0.38500000000000001,"15min":0.308},"max":{"1min":1.363,"5min":1.363,"15min":6.3200000000000003},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.94399999999999995,"15min":0.95099999999999996},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38600000000000001},"max":{"1min":1.4119999999999999,"5min":1.4379999999999999,"15min":4.2619999999999996},"last":0.61299999999999999}]},{"osd":1,"last update":"Thu Oct 26 01:44:05 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.877,"15min":0.88600000000000001},"min":{"1min":0.56399999999999995,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.387,"5min":1.504,"15min":5.1260000000000003},"last":0.76400000000000001},{"interface":"front","average":{"1min":1.052,"5min":1.0349999999999999,"15min":1.0189999999999999},"min":{"1min":0.439,"5min":0.42699999999999999,"15min":0.36599999999999999},"max":{"1min":1.409,"5min":1.5469999999999999,"15min":6.1520000000000001},"last":0.94299999999999995}]},{"osd":3,"last update":"Thu Oct 26 01:44:05 2023","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.1140000000000001,"15min":1.018},"min":{"1min":0.48699999999999999,"5min":0.46800000000000003,"15min":0.435},"max":{"1min":1.6259999999999999,"5min":1.909,"15min":2.0640000000000001},"last":1.046},{"interface":"front","average":{"1min":1.002,"5min":1.0980000000000001,"15min":1.0760000000000001},"min":{"1min":0.621,"5min":0.503,"15min":0.45300000000000001},"max":{"1min":1.2849999999999999,"5min":1.6659999999999999,"15min":8.9019999999999992},"last":1.087}]},{"osd":4,"last update":"Thu Oct 26 01:44:05 2023","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.3340000000000001,"15min":1.3069999999999999},"min":{"1min":0.78700000000000003,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.5649999999999999,"5min":2.109,"15min":6.2670000000000003},"last":1.405},{"interface":"front","average":{"1min":1.274,"5min":1.27,"15min":1.2849999999999999},"min":{"1min":0.79800000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.6419999999999999,"5min":1.923,"15min":7.343},"last":1.2430000000000001}]},{"osd":5,"last update":"Thu Oct 26 01:44:05 2023","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.3400000000000001,"15min":1.3200000000000001},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.71899999999999997},"max":{"1min":1.796,"5min":2.3849999999999998,"15min":3.7170000000000001},"last":1.488},{"interface":"front","average":{"1min":1.278,"5min":1.321,"15min":1.3089999999999999},"min":{"1min":0.95199999999999996,"5min":0.69499999999999995,"15min":0.65300000000000002},"max":{"1min":1.5649999999999999,"5min":1.8140000000000001,"15min":6.3899999999999997},"last":1.429}]},{"osd":6,"last update":"Thu Oct 26 01:44:05 2023","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":1.3879999999999999,"15min":1.371},"min":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.70599999999999996},"max":{"1min":1.7010000000000001,"5min":2.097,"15min":6.0490000000000004},"last":1.46},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.3899999999999999,"15min":1.3899999999999999},"min":{"1min":0.874,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.8080000000000001,"5min":2.5230000000000001,"15min":5.0650000000000004},"last":1.6519999999999999}]},{"osd":7,"last update":"Thu Oct 26 01:44:05 2023","interfaces":[{"interface":"back","average":{"1min":1.367,"5min":1.415,"15min":1.403},"min":{"1min":1.073,"5min":0.83399999999999996,"15min":0.69399999999999995},"max":{"1min":1.6599999999999999,"5min":1.7949999999999999,"15min":6.4480000000000004},"last":1.617},{"interface":"front","average":{"1min":1.478,"5min":1.4830000000000001,"15min":1.4770000000000001},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":1.706,"5min":2.0169999999999999,"15min":2.8439999999999999},"last":1.627}]}]},{"osd":1,"up_from":13,"seq":55834575022,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1042,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194269,"statfs":{"total":200000000,"available":198931989,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:43:56 2023","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.86599999999999999,"15min":0.78600000000000003},"min":{"1min":0.46999999999999997,"5min":0.39300000000000002,"15min":0.29499999999999998},"max":{"1min":1.8009999999999999,"5min":1.8009999999999999,"15min":1.8009999999999999},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.96899999999999997,"15min":0.87},"min":{"1min":0.46999999999999997,"5min":0.40300000000000002,"15min":0.36199999999999999},"max":{"1min":1.4510000000000001,"5min":1.591,"15min":1.6399999999999999},"last":1.0700000000000001}]},{"osd":2,"last update":"Thu Oct 26 01:43:56 2023","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.115,"15min":1.032},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.39600000000000002},"max":{"1min":1.4930000000000001,"5min":1.655,"15min":2.4830000000000001},"last":1.4430000000000001},{"interface":"front","average":{"1min":0.96299999999999997,"5min":1.036,"15min":0.94799999999999995},"min":{"1min":0.55000000000000004,"5min":0.51000000000000001,"15min":0.41199999999999998},"max":{"1min":1.3109999999999999,"5min":1.752,"15min":1.8080000000000001},"last":1.014}]},{"osd":3,"last update":"Thu Oct 26 01:44:00 2023","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":1.0720000000000001,"15min":1.0029999999999999},"min":{"1min":0.495,"5min":0.495,"15min":0.39500000000000002},"max":{"1min":1.575,"5min":1.6100000000000001,"15min":1.75},"last":1.1619999999999999},{"interface":"front","average":{"1min":1.1200000000000001,"5min":1.1220000000000001,"15min":1.0429999999999999},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.41299999999999998},"max":{"1min":1.458,"5min":1.556,"15min":1.802},"last":1.2889999999999999}]},{"osd":4,"last update":"Thu Oct 26 01:43:56 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.294,"15min":1.2370000000000001},"min":{"1min":0.95399999999999996,"5min":0.79800000000000004,"15min":0.56200000000000006},"max":{"1min":1.728,"5min":1.728,"15min":2.0659999999999998},"last":1.2549999999999999},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.26,"15min":1.1910000000000001},"min":{"1min":0.90300000000000002,"5min":0.70999999999999996,"15min":0.65800000000000003},"max":{"1min":1.6850000000000001,"5min":1.8420000000000001,"15min":2.0270000000000001},"last":1.5629999999999999}]},{"osd":5,"last update":"Thu Oct 26 01:43:56 2023","interfaces":[{"interface":"back","average":{"1min":1.302,"5min":1.3240000000000001,"15min":1.2649999999999999},"min":{"1min":0.70899999999999996,"5min":0.629,"15min":0.61799999999999999},"max":{"1min":1.776,"5min":1.776,"15min":1.8779999999999999},"last":1.2050000000000001},{"interface":"front","average":{"1min":1.3919999999999999,"5min":1.401,"15min":1.3640000000000001},"min":{"1min":0.94599999999999995,"5min":0.72899999999999998,"15min":0.59999999999999998},"max":{"1min":1.859,"5min":1.859,"15min":2.2200000000000002},"last":1.623}]},{"osd":6,"last update":"Thu Oct 26 01:43:56 2023","interfaces":[{"interface":"back","average":{"1min":1.391,"5min":1.3640000000000001,"15min":1.3200000000000001},"min":{"1min":1.1100000000000001,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.6890000000000001,"5min":1.8,"15min":2.3780000000000001},"last":1.6819999999999999},{"interface":"front","average":{"1min":1.3129999999999999,"5min":1.361,"15min":1.294},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.70799999999999996},"max":{"1min":1.6579999999999999,"5min":1.8819999999999999,"15min":1.984},"last":1.375}]},{"osd":7,"last update":"Thu Oct 26 01:43:56 2023","interfaces":[{"interface":"back","average":{"1min":1.3320000000000001,"5min":1.357,"15min":1.2909999999999999},"min":{"1min":0.872,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.802,"5min":1.835,"15min":2.6930000000000001},"last":1.4139999999999999},{"interface":"front","average":{"1min":1.472,"5min":1.4450000000000001,"15min":1.393},"min":{"1min":1.0509999999999999,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.863,"5min":1.903,"15min":2.0190000000000001},"last":1.518}]}]},{"osd":0,"up_from":13,"seq":55834575022,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1619,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193692,"statfs":{"total":200000000,"available":198341621,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Oct 26 01:43:58 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.79000000000000004,"15min":0.879},"min":{"1min":0.46200000000000002,"5min":0.32500000000000001,"15min":0.32000000000000001},"max":{"1min":1.1879999999999999,"5min":1.494,"15min":19.425000000000001},"last":0.499},{"interface":"front","average":{"1min":1.0229999999999999,"5min":1.004,"15min":1.0229999999999999},"min":{"1min":0.54600000000000004,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.5289999999999999,"5min":1.5289999999999999,"15min":15.162000000000001},"last":0.65000000000000002}]},{"osd":2,"last update":"Thu Oct 26 01:43:58 2023","interfaces":[{"interface":"back","average":{"1min":1.0329999999999999,"5min":1.0489999999999999,"15min":1.1439999999999999},"min":{"1min":0.63700000000000001,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.3879999999999999,"5min":1.556,"15min":28.439},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.93100000000000005,"15min":0.91800000000000004},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.37},"max":{"1min":1.494,"5min":1.851,"15min":4.9459999999999997},"last":1.1259999999999999}]},{"osd":3,"last update":"Thu Oct 26 01:43:58 2023","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.1359999999999999,"15min":1.0920000000000001},"min":{"1min":0.58799999999999997,"5min":0.56999999999999995,"15min":0.44600000000000001},"max":{"1min":1.5009999999999999,"5min":1.7509999999999999,"15min":4.9800000000000004},"last":1.0600000000000001},{"interface":"front","average":{"1min":1.163,"5min":1.2290000000000001,"15min":1.1679999999999999},"min":{"1min":0.67300000000000004,"5min":0.63500000000000001,"15min":0.438},"max":{"1min":1.645,"5min":1.774,"15min":5.016},"last":1.21}]},{"osd":4,"last update":"Thu Oct 26 01:43:58 2023","interfaces":[{"interface":"back","average":{"1min":1.169,"5min":1.159,"15min":1.246},"min":{"1min":0.84299999999999997,"5min":0.76800000000000002,"15min":0.52900000000000003},"max":{"1min":1.5389999999999999,"5min":1.5840000000000001,"15min":19.329000000000001},"last":0.70799999999999996},{"interface":"front","average":{"1min":1.242,"5min":1.254,"15min":1.2689999999999999},"min":{"1min":1.01,"5min":0.78500000000000003,"15min":0.67600000000000005},"max":{"1min":1.5960000000000001,"5min":1.7270000000000001,"15min":5.5229999999999997},"last":0.92200000000000004}]},{"osd":5,"last update":"Thu Oct 26 01:43:58 2023","interfaces":[{"interface":"back","average":{"1min":1.4430000000000001,"5min":1.4319999999999999,"15min":1.4219999999999999},"min":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.64900000000000002},"max":{"1min":2.605,"5min":2.605,"15min":5.6360000000000001},"last":1.798},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.286,"15min":1.367},"min":{"1min":0.92400000000000004,"5min":0.80800000000000005,"15min":0.57499999999999996},"max":{"1min":2.036,"5min":2.036,"15min":19.384},"last":0.75800000000000001}]},{"osd":6,"last update":"Thu Oct 26 01:43:58 2023","interfaces":[{"interface":"back","average":{"1min":1.3120000000000001,"5min":1.319,"15min":1.343},"min":{"1min":0.875,"5min":0.81100000000000005,"15min":0.73799999999999999},"max":{"1min":1.611,"5min":1.8999999999999999,"15min":8.2609999999999992},"last":0.80600000000000005},{"interface":"front","average":{"1min":1.3320000000000001,"5min":1.3660000000000001,"15min":1.3899999999999999},"min":{"1min":0.76100000000000001,"5min":0.746,"15min":0.746},"max":{"1min":1.7110000000000001,"5min":1.9399999999999999,"15min":11.849},"last":0.99299999999999999}]},{"osd":7,"last update":"Thu Oct 26 01:43:58 2023","interfaces":[{"interface":"back","average":{"1min":1.4650000000000001,"5min":1.488,"15min":1.47},"min":{"1min":1.016,"5min":0.999,"15min":0.73699999999999999},"max":{"1min":1.954,"5min":1.954,"15min":5.6689999999999996},"last":1.8700000000000001},{"interface":"front","average":{"1min":1.419,"5min":1.4330000000000001,"15min":1.484},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.69399999999999995},"max":{"1min":1.71,"5min":1.8149999999999999,"15min":19.245999999999999},"last":1.02}]}]}],"pool_statfs":[]}} 2023-10-26T01:44:56.152 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-10-26T01:30:32.890096+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=26, tm_hour=1, tm_min=30, tm_sec=32, tm_wday=3, tm_yday=299, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=26, tm_hour=1, tm_min=44, tm_sec=49, tm_wday=3, tm_yday=299, tm_isdst=0) 2023-10-26T01:44:56.153 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-26T01:45:08.476 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T01:45:08.485 DEBUG:teuthology.orchestra.run.smithi170:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T01:45:16.154 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-26T01:45:16.386 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:16.383+0000 7fcf655d7640 1 -- 172.21.15.169:0/2657533832 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf600cd720 msgr2=0x7fcf600cdbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:45:16.386 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:16.383+0000 7fcf655d7640 1 --2- 172.21.15.169:0/2657533832 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf600cd720 0x7fcf600cdbb0 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7fcf58005fd0 tx=0x7fcf5802bfc0 comp rx=0 tx=0).stop 2023-10-26T01:45:16.387 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:16.383+0000 7fcf655d7640 1 -- 172.21.15.169:0/2657533832 shutdown_connections 2023-10-26T01:45:16.387 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:16.383+0000 7fcf655d7640 1 --2- 172.21.15.169:0/2657533832 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf600cd720 0x7fcf600cdbb0 unknown :-1 s=CLOSED pgs=2025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:45:16.387 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:16.383+0000 7fcf655d7640 1 --2- 172.21.15.169:0/2657533832 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcf60158850 0x7fcf6014e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:45:16.387 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:16.383+0000 7fcf655d7640 1 -- 172.21.15.169:0/2657533832 >> 172.21.15.169:0/2657533832 conn(0x7fcf60069930 msgr2=0x7fcf60069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:45:16.387 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:16.383+0000 7fcf655d7640 1 -- 172.21.15.169:0/2657533832 shutdown_connections 2023-10-26T01:45:16.388 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:16.383+0000 7fcf655d7640 1 -- 172.21.15.169:0/2657533832 wait complete. 2023-10-26T01:45:16.390 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:16.385+0000 7fcf655d7640 1 Processor -- start 2023-10-26T01:45:16.390 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:16.385+0000 7fcf655d7640 1 -- start start 2023-10-26T01:45:16.390 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:16.386+0000 7fcf655d7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf6006ce20 0x7fcf600c9e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:45:16.390 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:16.386+0000 7fcf655d7640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf600cd720 0x7fcf6015fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:45:16.391 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:16.386+0000 7fcf655d7640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcf60158850 0x7fcf60160200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:45:16.391 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:16.386+0000 7fcf655d7640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf600cc310 con 0x7fcf600cd720 2023-10-26T01:45:16.391 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:16.386+0000 7fcf655d7640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf601608f0 con 0x7fcf6006ce20 2023-10-26T01:45:16.391 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:16.386+0000 7fcf655d7640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf60160bd0 con 0x7fcf60158850 2023-10-26T01:45:16.392 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:16.386+0000 7fcf5e7fc640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf600cd720 0x7fcf6015fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:45:16.392 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:16.386+0000 7fcf5e7fc640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf600cd720 0x7fcf6015fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:60752/0 (socket says 172.21.15.169:60752) 2023-10-26T01:45:16.392 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:16.386+0000 7fcf5e7fc640 1 -- 172.21.15.169:0/227086384 learned_addr learned my addr 172.21.15.169:0/227086384 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:45:16.392 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:16.386+0000 7fcf5effd640 1 --2- 172.21.15.169:0/227086384 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf6006ce20 0x7fcf600c9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:45:16.392 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:16.386+0000 7fcf5e7fc640 1 -- 172.21.15.169:0/227086384 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcf60158850 msgr2=0x7fcf60160200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:45:16.392 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:16.386+0000 7fcf5e7fc640 1 --2- 172.21.15.169:0/227086384 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcf60158850 0x7fcf60160200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:45:16.393 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:16.386+0000 7fcf5e7fc640 1 -- 172.21.15.169:0/227086384 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf6006ce20 msgr2=0x7fcf600c9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:45:16.393 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:16.386+0000 7fcf5e7fc640 1 --2- 172.21.15.169:0/227086384 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf6006ce20 0x7fcf600c9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:45:16.393 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:16.386+0000 7fcf5e7fc640 1 -- 172.21.15.169:0/227086384 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf58009840 con 0x7fcf600cd720 2023-10-26T01:45:16.393 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:16.386+0000 7fcf5effd640 1 --2- 172.21.15.169:0/227086384 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf6006ce20 0x7fcf600c9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:45:16.393 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:16.387+0000 7fcf5e7fc640 1 --2- 172.21.15.169:0/227086384 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf600cd720 0x7fcf6015fcc0 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x7fcf5400d740 tx=0x7fcf5400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:45:16.394 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:16.387+0000 7fcf477fe640 1 -- 172.21.15.169:0/227086384 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf54009c30 con 0x7fcf600cd720 2023-10-26T01:45:16.394 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:16.387+0000 7fcf477fe640 1 -- 172.21.15.169:0/227086384 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fcf54009e80 con 0x7fcf600cd720 2023-10-26T01:45:16.394 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:16.387+0000 7fcf655d7640 1 -- 172.21.15.169:0/227086384 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf60160eb0 con 0x7fcf600cd720 2023-10-26T01:45:16.394 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:16.387+0000 7fcf477fe640 1 -- 172.21.15.169:0/227086384 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf540106c0 con 0x7fcf600cd720 2023-10-26T01:45:16.395 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:16.387+0000 7fcf655d7640 1 -- 172.21.15.169:0/227086384 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf60161470 con 0x7fcf600cd720 2023-10-26T01:45:16.395 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:16.388+0000 7fcf655d7640 1 -- 172.21.15.169:0/227086384 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf6004db10 con 0x7fcf600cd720 2023-10-26T01:45:16.396 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:16.389+0000 7fcf477fe640 1 -- 172.21.15.169:0/227086384 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fcf54010820 con 0x7fcf600cd720 2023-10-26T01:45:16.396 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:16.390+0000 7fcf477fe640 1 --2- 172.21.15.169:0/227086384 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcf4c06ae40 0x7fcf4c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:45:16.396 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:16.390+0000 7fcf477fe640 1 -- 172.21.15.169:0/227086384 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fcf540898e0 con 0x7fcf600cd720 2023-10-26T01:45:16.396 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:16.391+0000 7fcf5effd640 1 --2- 172.21.15.169:0/227086384 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcf4c06ae40 0x7fcf4c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:45:16.397 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:16.392+0000 7fcf5effd640 1 --2- 172.21.15.169:0/227086384 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcf4c06ae40 0x7fcf4c06d300 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fcf500011e0 tx=0x7fcf50009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:45:16.397 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:16.395+0000 7fcf477fe640 1 -- 172.21.15.169:0/227086384 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcf54054bf0 con 0x7fcf600cd720 2023-10-26T01:45:16.584 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:16.581+0000 7fcf655d7640 1 -- 172.21.15.169:0/227086384 --> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcf600aca70 con 0x7fcf4c06ae40 2023-10-26T01:45:16.590 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:16.587+0000 7fcf477fe640 1 -- 172.21.15.169:0/227086384 <== mgr.4112 v2:172.21.15.170:6832/1187156918 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+46913 (secure 0 0 0) 0x7fcf600aca70 con 0x7fcf4c06ae40 2023-10-26T01:45:16.590 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:45:16.591 INFO:teuthology.orchestra.run.smithi169.stderr:dumped all 2023-10-26T01:45:16.593 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:16.590+0000 7fcf655d7640 1 -- 172.21.15.169:0/227086384 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcf4c06ae40 msgr2=0x7fcf4c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:45:16.593 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:16.590+0000 7fcf655d7640 1 --2- 172.21.15.169:0/227086384 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcf4c06ae40 0x7fcf4c06d300 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fcf500011e0 tx=0x7fcf50009040 comp rx=0 tx=0).stop 2023-10-26T01:45:16.593 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:16.590+0000 7fcf655d7640 1 -- 172.21.15.169:0/227086384 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf600cd720 msgr2=0x7fcf6015fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:45:16.593 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:16.590+0000 7fcf655d7640 1 --2- 172.21.15.169:0/227086384 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf600cd720 0x7fcf6015fcc0 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x7fcf5400d740 tx=0x7fcf5400dc00 comp rx=0 tx=0).stop 2023-10-26T01:45:16.594 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:16.590+0000 7fcf655d7640 1 -- 172.21.15.169:0/227086384 shutdown_connections 2023-10-26T01:45:16.594 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:16.590+0000 7fcf655d7640 1 --2- 172.21.15.169:0/227086384 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fcf4c06ae40 0x7fcf4c06d300 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:45:16.594 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:16.590+0000 7fcf655d7640 1 --2- 172.21.15.169:0/227086384 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf600cd720 0x7fcf6015fcc0 unknown :-1 s=CLOSED pgs=2026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:45:16.594 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:16.590+0000 7fcf655d7640 1 --2- 172.21.15.169:0/227086384 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcf60158850 0x7fcf60160200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:45:16.594 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:16.590+0000 7fcf655d7640 1 --2- 172.21.15.169:0/227086384 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf6006ce20 0x7fcf600c9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:45:16.595 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:16.590+0000 7fcf655d7640 1 -- 172.21.15.169:0/227086384 >> 172.21.15.169:0/227086384 conn(0x7fcf60069930 msgr2=0x7fcf60157300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:45:16.595 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:16.590+0000 7fcf655d7640 1 -- 172.21.15.169:0/227086384 shutdown_connections 2023-10-26T01:45:16.595 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:16.590+0000 7fcf655d7640 1 -- 172.21.15.169:0/227086384 wait complete. 2023-10-26T01:45:16.612 INFO:teuthology.orchestra.run.smithi169.stdout:{"pg_ready":true,"pg_map":{"version":625,"stamp":"2023-10-26T01:45:16.448989+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":6,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":9496,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1553003,"statfs":{"total":1600000000,"available":1590275790,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003574"},"pg_stats":[{"pgid":"1.0","version":"189'119","reported_seq":344,"reported_epoch":189,"state":"active+clean","last_fresh":"2023-10-26T01:44:59.425672+0000","last_change":"2023-10-26T01:44:59.425791+0000","last_active":"2023-10-26T01:44:59.425672+0000","last_peered":"2023-10-26T01:44:59.425672+0000","last_clean":"2023-10-26T01:44:59.425672+0000","last_became_active":"2023-10-26T01:30:34.883071+0000","last_became_peered":"2023-10-26T01:30:34.883071+0000","last_unstale":"2023-10-26T01:44:59.425672+0000","last_undegraded":"2023-10-26T01:44:59.425672+0000","last_fullsized":"2023-10-26T01:44:59.425672+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"189'119","last_scrub_stamp":"2023-10-26T01:44:59.425576+0000","last_deep_scrub":"189'119","last_deep_scrub_stamp":"2023-10-26T01:44:59.425576+0000","last_clean_scrub_stamp":"2023-10-26T01:44:59.425576+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T03:46:27.298803+0000","scrub_duration":0.0073866690000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":14,"seq":60129542323,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1042,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194269,"statfs":{"total":200000000,"available":198931989,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:45:01 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.82299999999999995,"15min":0.80600000000000005},"min":{"1min":0.45400000000000001,"5min":0.35399999999999998,"15min":0.33000000000000002},"max":{"1min":1.123,"5min":1.571,"15min":2.2549999999999999},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.94399999999999995,"5min":0.99299999999999999,"15min":0.92200000000000004},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.309},"max":{"1min":1.5589999999999999,"5min":1.919,"15min":1.919},"last":1.0609999999999999}]},{"osd":1,"last update":"Thu Oct 26 01:45:01 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.89400000000000002,"15min":0.88400000000000001},"min":{"1min":0.44400000000000001,"5min":0.38,"15min":0.32300000000000001},"max":{"1min":1.264,"5min":1.569,"15min":4.4340000000000002},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.96199999999999997,"5min":1.0609999999999999,"15min":1.0069999999999999},"min":{"1min":0.40699999999999997,"5min":0.38600000000000001,"15min":0.374},"max":{"1min":1.5169999999999999,"5min":1.7070000000000001,"15min":2.3330000000000002},"last":0.95999999999999996}]},{"osd":2,"last update":"Thu Oct 26 01:45:01 2023","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":1.0469999999999999,"15min":1.04},"min":{"1min":0.47599999999999998,"5min":0.44,"15min":0.39900000000000002},"max":{"1min":1.462,"5min":1.6819999999999999,"15min":4.2800000000000002},"last":1.206},{"interface":"front","average":{"1min":1.073,"5min":1.1499999999999999,"15min":1.1259999999999999},"min":{"1min":0.46000000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.5609999999999999,"5min":1.8129999999999999,"15min":2.302},"last":1.4019999999999999}]},{"osd":4,"last update":"Thu Oct 26 01:45:01 2023","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":1.2030000000000001,"15min":1.1950000000000001},"min":{"1min":0.63,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.6759999999999999,"5min":1.7150000000000001,"15min":2.1070000000000002},"last":1.1850000000000001},{"interface":"front","average":{"1min":1.335,"5min":1.363,"15min":1.355},"min":{"1min":0.90000000000000002,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.6899999999999999,"5min":1.7509999999999999,"15min":4.8250000000000002},"last":1.4390000000000001}]},{"osd":5,"last update":"Thu Oct 26 01:45:01 2023","interfaces":[{"interface":"back","average":{"1min":1.2410000000000001,"5min":1.3120000000000001,"15min":1.2969999999999999},"min":{"1min":0.74299999999999999,"5min":0.73899999999999999,"15min":0.72499999999999998},"max":{"1min":1.694,"5min":1.998,"15min":4.9080000000000004},"last":1.04},{"interface":"front","average":{"1min":1.298,"5min":1.347,"15min":1.304},"min":{"1min":0.82399999999999995,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.651,"5min":1.9279999999999999,"15min":2.8980000000000001},"last":1.135}]},{"osd":6,"last update":"Thu Oct 26 01:45:01 2023","interfaces":[{"interface":"back","average":{"1min":1.286,"5min":1.4350000000000001,"15min":1.369},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.70199999999999996},"max":{"1min":1.6599999999999999,"5min":3.5800000000000001,"15min":3.5800000000000001},"last":1.248},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.349,"15min":1.3},"min":{"1min":0.88,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.569,"5min":1.782,"15min":2.181},"last":1.2949999999999999}]},{"osd":7,"last update":"Thu Oct 26 01:45:01 2023","interfaces":[{"interface":"back","average":{"1min":1.3620000000000001,"5min":1.3919999999999999,"15min":1.345},"min":{"1min":1.099,"5min":0.745,"15min":0.71999999999999997},"max":{"1min":1.623,"5min":2.024,"15min":2.0680000000000001},"last":1.3720000000000001},{"interface":"front","average":{"1min":1.3180000000000001,"5min":1.397,"15min":1.3919999999999999},"min":{"1min":0.71699999999999997,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.903,"5min":2.0579999999999998,"15min":4.8730000000000002},"last":1.339}]}]},{"osd":7,"up_from":13,"seq":55834575027,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1619,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193692,"statfs":{"total":200000000,"available":198341621,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:45:05 2023","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.1279999999999999,"15min":1.139},"min":{"1min":0.751,"5min":0.60599999999999998,"15min":0.59599999999999997},"max":{"1min":1.615,"5min":1.6970000000000001,"15min":2.2269999999999999},"last":1.1200000000000001},{"interface":"front","average":{"1min":1.149,"5min":1.1639999999999999,"15min":1.1499999999999999},"min":{"1min":0.80000000000000004,"5min":0.69399999999999995,"15min":0.58699999999999997},"max":{"1min":1.6359999999999999,"5min":1.738,"15min":2.6779999999999999},"last":1.147}]},{"osd":1,"last update":"Thu Oct 26 01:45:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.165,"15min":1.1539999999999999},"min":{"1min":0.76300000000000001,"5min":0.71499999999999997,"15min":0.65200000000000002},"max":{"1min":1.54,"5min":1.8,"15min":2.8260000000000001},"last":1.2130000000000001},{"interface":"front","average":{"1min":1.302,"5min":1.2450000000000001,"15min":1.222},"min":{"1min":0.93700000000000006,"5min":0.66400000000000003,"15min":0.61199999999999999},"max":{"1min":1.6020000000000001,"5min":1.736,"15min":3.2730000000000001},"last":1.484}]},{"osd":2,"last update":"Thu Oct 26 01:45:05 2023","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.212,"15min":1.1839999999999999},"min":{"1min":0.77700000000000002,"5min":0.68999999999999995,"15min":0.58499999999999996},"max":{"1min":1.6859999999999999,"5min":1.794,"15min":2.218},"last":1.0800000000000001},{"interface":"front","average":{"1min":1.23,"5min":1.248,"15min":1.2729999999999999},"min":{"1min":0.88400000000000001,"5min":0.77000000000000002,"15min":0.73099999999999998},"max":{"1min":1.796,"5min":1.879,"15min":3.0510000000000002},"last":1.288}]},{"osd":3,"last update":"Thu Oct 26 01:45:05 2023","interfaces":[{"interface":"back","average":{"1min":1.359,"5min":1.329,"15min":1.2849999999999999},"min":{"1min":0.93700000000000006,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.6140000000000001,"5min":2.1379999999999999,"15min":2.1379999999999999},"last":1.337},{"interface":"front","average":{"1min":1.2889999999999999,"5min":1.377,"15min":1.3879999999999999},"min":{"1min":0.93400000000000005,"5min":0.92200000000000004,"15min":0.65700000000000003},"max":{"1min":1.651,"5min":2.2639999999999998,"15min":5.5039999999999996},"last":1.3879999999999999}]},{"osd":4,"last update":"Thu Oct 26 01:45:05 2023","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.3280000000000001,"15min":1.371},"min":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.76100000000000001},"max":{"1min":1.6699999999999999,"5min":2.0899999999999999,"15min":2.77},"last":1.4239999999999999},{"interface":"front","average":{"1min":1.321,"5min":1.304,"15min":1.3959999999999999},"min":{"1min":0.94099999999999995,"5min":0.85399999999999998,"15min":0.46500000000000002},"max":{"1min":1.7410000000000001,"5min":1.9099999999999999,"15min":3.1419999999999999},"last":1.012}]},{"osd":5,"last update":"Thu Oct 26 01:45:05 2023","interfaces":[{"interface":"back","average":{"1min":1.399,"5min":1.3819999999999999,"15min":1.4299999999999999},"min":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.435},"max":{"1min":1.7490000000000001,"5min":1.9379999999999999,"15min":5.734},"last":1.591},{"interface":"front","average":{"1min":1.5209999999999999,"5min":1.494,"15min":1.5049999999999999},"min":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":0.51900000000000002},"max":{"1min":1.839,"5min":2.2109999999999999,"15min":2.8980000000000001},"last":1.1859999999999999}]},{"osd":6,"last update":"Thu Oct 26 01:45:05 2023","interfaces":[{"interface":"back","average":{"1min":1.5589999999999999,"5min":1.536,"15min":1.631},"min":{"1min":1.21,"5min":1.038,"15min":0.47399999999999998},"max":{"1min":1.847,"5min":2.0459999999999998,"15min":11.577999999999999},"last":1.2490000000000001},{"interface":"front","average":{"1min":1.615,"5min":1.599,"15min":1.696},"min":{"1min":1.3680000000000001,"5min":1.214,"15min":0.55100000000000005},"max":{"1min":1.919,"5min":2.4300000000000002,"15min":10.704000000000001},"last":1.5329999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575027,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1042,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194270,"statfs":{"total":200000000,"available":198932603,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:45:02 2023","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":1.1399999999999999,"15min":1.125},"min":{"1min":0.79600000000000004,"5min":0.69299999999999995,"15min":0.60399999999999998},"max":{"1min":1.5580000000000001,"5min":2.2509999999999999,"15min":2.7370000000000001},"last":1.0589999999999999},{"interface":"front","average":{"1min":1.097,"5min":1.1870000000000001,"15min":1.1419999999999999},"min":{"1min":0.627,"5min":0.627,"15min":0.59199999999999997},"max":{"1min":1.4339999999999999,"5min":2.2170000000000001,"15min":4.4779999999999998},"last":1.196}]},{"osd":1,"last update":"Thu Oct 26 01:45:02 2023","interfaces":[{"interface":"back","average":{"1min":1.177,"5min":1.151,"15min":1.179},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.637,"5min":1.889,"15min":5.5359999999999996},"last":1.498},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.21,"15min":1.202},"min":{"1min":0.88300000000000001,"5min":0.76400000000000001,"15min":0.64000000000000001},"max":{"1min":1.4299999999999999,"5min":1.726,"15min":4.3579999999999997},"last":1.087}]},{"osd":2,"last update":"Thu Oct 26 01:45:02 2023","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":1.1739999999999999,"15min":1.1499999999999999},"min":{"1min":0.78700000000000003,"5min":0.73599999999999999,"15min":0.59799999999999998},"max":{"1min":1.5289999999999999,"5min":1.651,"15min":3.7879999999999998},"last":0.91100000000000003},{"interface":"front","average":{"1min":1.2749999999999999,"5min":1.2929999999999999,"15min":1.294},"min":{"1min":0.94499999999999995,"5min":0.77300000000000002,"15min":0.68899999999999995},"max":{"1min":1.72,"5min":2.0110000000000001,"15min":3.0470000000000002},"last":1.1479999999999999}]},{"osd":3,"last update":"Thu Oct 26 01:45:02 2023","interfaces":[{"interface":"back","average":{"1min":1.21,"5min":1.2450000000000001,"15min":1.26},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.68500000000000005},"max":{"1min":1.597,"5min":1.8200000000000001,"15min":4.4560000000000004},"last":1.3720000000000001},{"interface":"front","average":{"1min":1.325,"5min":1.369,"15min":1.341},"min":{"1min":0.98199999999999998,"5min":0.93700000000000006,"15min":0.69499999999999995},"max":{"1min":1.6850000000000001,"5min":1.871,"15min":2.6909999999999998},"last":1.2430000000000001}]},{"osd":4,"last update":"Thu Oct 26 01:45:02 2023","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.2609999999999999,"15min":1.246},"min":{"1min":0.91900000000000004,"5min":0.59899999999999998,"15min":0.48399999999999999},"max":{"1min":1.6519999999999999,"5min":1.825,"15min":2.9489999999999998},"last":1.419},{"interface":"front","average":{"1min":1.351,"5min":1.329,"15min":1.3540000000000001},"min":{"1min":0.92900000000000005,"5min":0.88400000000000001,"15min":0.59299999999999997},"max":{"1min":1.706,"5min":2.169,"15min":6.609},"last":1.0289999999999999}]},{"osd":5,"last update":"Thu Oct 26 01:45:02 2023","interfaces":[{"interface":"back","average":{"1min":1.5660000000000001,"5min":1.5249999999999999,"15min":1.5129999999999999},"min":{"1min":1.28,"5min":1.121,"15min":0.63700000000000001},"max":{"1min":1.7989999999999999,"5min":1.9399999999999999,"15min":5.4370000000000003},"last":1.284},{"interface":"front","average":{"1min":1.516,"5min":1.468,"15min":1.448},"min":{"1min":1.0429999999999999,"5min":0.69999999999999996,"15min":0.55200000000000005},"max":{"1min":1.8200000000000001,"5min":2.1190000000000002,"15min":2.7759999999999998},"last":1.329}]},{"osd":7,"last update":"Thu Oct 26 01:45:02 2023","interfaces":[{"interface":"back","average":{"1min":1.429,"5min":1.407,"15min":1.431},"min":{"1min":1.0349999999999999,"5min":0.72399999999999998,"15min":0.54100000000000004},"max":{"1min":1.7909999999999999,"5min":1.984,"15min":11.804},"last":1.5649999999999999},{"interface":"front","average":{"1min":1.5129999999999999,"5min":1.5049999999999999,"15min":1.556},"min":{"1min":1.075,"5min":0.96399999999999997,"15min":0.56499999999999995},"max":{"1min":1.742,"5min":1.921,"15min":11.541},"last":1.46}]}]},{"osd":5,"up_from":13,"seq":55834575027,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1042,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194269,"statfs":{"total":200000000,"available":198931989,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:45:02 2023","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.1850000000000001,"15min":1.181},"min":{"1min":0.624,"5min":0.53400000000000003,"15min":0.53300000000000003},"max":{"1min":1.593,"5min":1.724,"15min":3.8210000000000002},"last":1.3320000000000001},{"interface":"front","average":{"1min":1.175,"5min":1.286,"15min":1.2549999999999999},"min":{"1min":0.78500000000000003,"5min":0.68200000000000005,"15min":0.67900000000000005},"max":{"1min":1.4870000000000001,"5min":1.8140000000000001,"15min":2.9889999999999999},"last":1.5569999999999999}]},{"osd":1,"last update":"Thu Oct 26 01:45:02 2023","interfaces":[{"interface":"back","average":{"1min":1.1020000000000001,"5min":1.228,"15min":1.1519999999999999},"min":{"1min":0.79600000000000004,"5min":0.71299999999999997,"15min":0.65900000000000003},"max":{"1min":1.4299999999999999,"5min":1.899,"15min":1.899},"last":0.97099999999999997},{"interface":"front","average":{"1min":1.1579999999999999,"5min":1.147,"15min":1.135},"min":{"1min":0.85999999999999999,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.577,"5min":1.708,"15min":2.4340000000000002},"last":1.4930000000000001}]},{"osd":2,"last update":"Thu Oct 26 01:45:02 2023","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.202,"15min":1.194},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.69299999999999995},"max":{"1min":1.5860000000000001,"5min":1.7030000000000001,"15min":1.8220000000000001},"last":1.399},{"interface":"front","average":{"1min":1.169,"5min":1.236,"15min":1.2150000000000001},"min":{"1min":0.78200000000000003,"5min":0.62,"15min":0.62},"max":{"1min":1.49,"5min":1.875,"15min":1.875},"last":1.367}]},{"osd":3,"last update":"Thu Oct 26 01:45:02 2023","interfaces":[{"interface":"back","average":{"1min":1.2809999999999999,"5min":1.302,"15min":1.284},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.72299999999999998},"max":{"1min":1.6140000000000001,"5min":1.7569999999999999,"15min":1.835},"last":1.1699999999999999},{"interface":"front","average":{"1min":1.2769999999999999,"5min":1.371,"15min":1.3120000000000001},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.64000000000000001},"max":{"1min":1.6060000000000001,"5min":2.6669999999999998,"15min":2.9249999999999998},"last":1.6639999999999999}]},{"osd":4,"last update":"Thu Oct 26 01:45:02 2023","interfaces":[{"interface":"back","average":{"1min":1.3009999999999999,"5min":1.2589999999999999,"15min":1.246},"min":{"1min":0.53200000000000003,"5min":0.52000000000000002,"15min":0.49099999999999999},"max":{"1min":1.762,"5min":1.79,"15min":2.7770000000000001},"last":1.204},{"interface":"front","average":{"1min":1.333,"5min":1.3759999999999999,"15min":1.3859999999999999},"min":{"1min":0.58699999999999997,"5min":0.504,"15min":0.504},"max":{"1min":1.718,"5min":2.125,"15min":2.125},"last":1.8009999999999999}]},{"osd":6,"last update":"Thu Oct 26 01:45:02 2023","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":1.4379999999999999,"15min":1.3999999999999999},"min":{"1min":0.95599999999999996,"5min":0.80000000000000004,"15min":0.51900000000000002},"max":{"1min":1.7430000000000001,"5min":2.4590000000000001,"15min":2.8250000000000002},"last":1.2709999999999999},{"interface":"front","average":{"1min":1.48,"5min":1.46,"15min":1.4159999999999999},"min":{"1min":0.621,"5min":0.621,"15min":0.61799999999999999},"max":{"1min":1.9419999999999999,"5min":2.3530000000000002,"15min":2.4830000000000001},"last":1.6180000000000001}]},{"osd":7,"last update":"Thu Oct 26 01:45:02 2023","interfaces":[{"interface":"back","average":{"1min":1.4970000000000001,"5min":1.516,"15min":1.504},"min":{"1min":0.77200000000000002,"5min":0.58799999999999997,"15min":0.57299999999999995},"max":{"1min":1.673,"5min":2.0409999999999999,"15min":3.621},"last":1.72},{"interface":"front","average":{"1min":1.5389999999999999,"5min":1.575,"15min":1.532},"min":{"1min":1.214,"5min":0.85299999999999998,"15min":0.59499999999999997},"max":{"1min":1.7569999999999999,"5min":2.0539999999999998,"15min":2.8010000000000002},"last":1.4590000000000001}]}]},{"osd":4,"up_from":13,"seq":55834575027,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1042,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194269,"statfs":{"total":200000000,"available":198931989,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:44:51 2023","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":1.161,"15min":1.0900000000000001},"min":{"1min":0.61199999999999999,"5min":0.503,"15min":0.503},"max":{"1min":1.649,"5min":1.702,"15min":2.0289999999999999},"last":1.25},{"interface":"front","average":{"1min":1.1799999999999999,"5min":1.149,"15min":1.099},"min":{"1min":0.90300000000000002,"5min":0.72599999999999998,"15min":0.60599999999999998},"max":{"1min":1.5509999999999999,"5min":1.722,"15min":2.6829999999999998},"last":1.3069999999999999}]},{"osd":1,"last update":"Thu Oct 26 01:44:51 2023","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":1.1910000000000001,"15min":1.1759999999999999},"min":{"1min":0.81399999999999995,"5min":0.74399999999999999,"15min":0.56799999999999995},"max":{"1min":1.782,"5min":1.804,"15min":4.0519999999999996},"last":1.446},{"interface":"front","average":{"1min":1.169,"5min":1.226,"15min":1.1899999999999999},"min":{"1min":0.65300000000000002,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.659,"5min":1.667,"15min":2.4390000000000001},"last":1.1220000000000001}]},{"osd":2,"last update":"Thu Oct 26 01:44:51 2023","interfaces":[{"interface":"back","average":{"1min":1.335,"5min":1.3620000000000001,"15min":1.3260000000000001},"min":{"1min":0.751,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.8859999999999999,"5min":2.1160000000000001,"15min":5.6749999999999998},"last":1.5049999999999999},{"interface":"front","average":{"1min":1.1140000000000001,"5min":1.1970000000000001,"15min":1.1890000000000001},"min":{"1min":0.71699999999999997,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.5920000000000001,"5min":1.7190000000000001,"15min":3.673},"last":1.2709999999999999}]},{"osd":3,"last update":"Thu Oct 26 01:44:51 2023","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.2929999999999999,"15min":1.2549999999999999},"min":{"1min":0.80300000000000005,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.587,"5min":1.9630000000000001,"15min":2.5960000000000001},"last":1.3480000000000001},{"interface":"front","average":{"1min":1.2470000000000001,"5min":1.23,"15min":1.21},"min":{"1min":0.94899999999999995,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.589,"5min":1.641,"15min":1.9570000000000001},"last":0.96099999999999997}]},{"osd":5,"last update":"Thu Oct 26 01:44:51 2023","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.256,"15min":1.2809999999999999},"min":{"1min":0.92100000000000004,"5min":0.71099999999999997,"15min":0.46300000000000002},"max":{"1min":1.673,"5min":2.6080000000000001,"15min":2.774},"last":0.91700000000000004},{"interface":"front","average":{"1min":1.363,"5min":1.3160000000000001,"15min":1.3340000000000001},"min":{"1min":0.93999999999999995,"5min":0.78000000000000003,"15min":0.55200000000000005},"max":{"1min":2.097,"5min":2.3109999999999999,"15min":5.8579999999999997},"last":1.016}]},{"osd":6,"last update":"Thu Oct 26 01:44:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3069999999999999,"5min":1.3440000000000001,"15min":1.363},"min":{"1min":0.89900000000000002,"5min":0.69499999999999995,"15min":0.59299999999999997},"max":{"1min":1.9350000000000001,"5min":2.3780000000000001,"15min":12.343999999999999},"last":1.1659999999999999},{"interface":"front","average":{"1min":1.554,"5min":1.554,"15min":1.5720000000000001},"min":{"1min":1.266,"5min":0.80000000000000004,"15min":0.55000000000000004},"max":{"1min":1.8069999999999999,"5min":2.7480000000000002,"15min":12.605},"last":1.478}]},{"osd":7,"last update":"Thu Oct 26 01:44:51 2023","interfaces":[{"interface":"back","average":{"1min":1.546,"5min":1.526,"15min":1.526},"min":{"1min":1.083,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":2.242,"5min":2.544,"15min":5.774},"last":1.21},{"interface":"front","average":{"1min":1.484,"5min":1.5,"15min":1.5029999999999999},"min":{"1min":1.2190000000000001,"5min":0.79000000000000004,"15min":0.68000000000000005},"max":{"1min":1.833,"5min":1.9690000000000001,"15min":4.6520000000000001},"last":1.3999999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575027,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1042,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194269,"statfs":{"total":200000000,"available":198931989,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:45:06 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.82699999999999996,"15min":0.83699999999999997},"min":{"1min":0.40999999999999998,"5min":0.38500000000000001,"15min":0.308},"max":{"1min":1.298,"5min":1.363,"15min":6.3200000000000003},"last":0.745},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.93600000000000005,"15min":0.94899999999999995},"min":{"1min":0.497,"5min":0.38700000000000001,"15min":0.38600000000000001},"max":{"1min":1.349,"5min":1.4379999999999999,"15min":4.2619999999999996},"last":0.93600000000000005}]},{"osd":1,"last update":"Thu Oct 26 01:45:06 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.88700000000000001,"15min":0.88200000000000001},"min":{"1min":0.442,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.327,"5min":1.504,"15min":5.1260000000000003},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.98199999999999998,"5min":1.0449999999999999,"15min":1.0189999999999999},"min":{"1min":0.63500000000000001,"5min":0.42699999999999999,"15min":0.36599999999999999},"max":{"1min":1.3879999999999999,"5min":1.5469999999999999,"15min":6.1520000000000001},"last":1.145}]},{"osd":3,"last update":"Thu Oct 26 01:45:06 2023","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":1.1020000000000001,"15min":1.0249999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.3700000000000001,"5min":1.6719999999999999,"15min":2.0640000000000001},"last":1.099},{"interface":"front","average":{"1min":1.0780000000000001,"5min":1.0940000000000001,"15min":1.085},"min":{"1min":0.57799999999999996,"5min":0.51400000000000001,"15min":0.45300000000000001},"max":{"1min":1.5149999999999999,"5min":1.6659999999999999,"15min":8.9019999999999992},"last":1.2310000000000001}]},{"osd":4,"last update":"Thu Oct 26 01:45:06 2023","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.3149999999999999,"15min":1.3080000000000001},"min":{"1min":0.79300000000000004,"5min":0.77300000000000002,"15min":0.64000000000000001},"max":{"1min":1.623,"5min":1.9910000000000001,"15min":6.2670000000000003},"last":1.274},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.274,"15min":1.284},"min":{"1min":0.82599999999999996,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.4930000000000001,"5min":1.6639999999999999,"15min":7.343},"last":1.3600000000000001}]},{"osd":5,"last update":"Thu Oct 26 01:45:06 2023","interfaces":[{"interface":"back","average":{"1min":1.3069999999999999,"5min":1.335,"15min":1.323},"min":{"1min":0.92700000000000005,"5min":0.76800000000000002,"15min":0.71899999999999997},"max":{"1min":1.671,"5min":1.796,"15min":3.7170000000000001},"last":1.444},{"interface":"front","average":{"1min":1.24,"5min":1.319,"15min":1.3069999999999999},"min":{"1min":0.86199999999999999,"5min":0.69499999999999995,"15min":0.65300000000000002},"max":{"1min":1.7010000000000001,"5min":1.8100000000000001,"15min":6.3899999999999997},"last":1.397}]},{"osd":6,"last update":"Thu Oct 26 01:45:06 2023","interfaces":[{"interface":"back","average":{"1min":1.2909999999999999,"5min":1.371,"15min":1.371},"min":{"1min":0.77500000000000002,"5min":0.72399999999999998,"15min":0.70599999999999996},"max":{"1min":1.7629999999999999,"5min":2.097,"15min":6.0490000000000004},"last":1.5740000000000001},{"interface":"front","average":{"1min":1.391,"5min":1.399,"15min":1.393},"min":{"1min":1.0029999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.73,"5min":2.036,"15min":5.0650000000000004},"last":1.526}]},{"osd":7,"last update":"Thu Oct 26 01:45:06 2023","interfaces":[{"interface":"back","average":{"1min":1.357,"5min":1.4159999999999999,"15min":1.401},"min":{"1min":0.99399999999999999,"5min":0.85699999999999998,"15min":0.69399999999999995},"max":{"1min":1.704,"5min":1.7569999999999999,"15min":6.4480000000000004},"last":1.321},{"interface":"front","average":{"1min":1.391,"5min":1.4710000000000001,"15min":1.4750000000000001},"min":{"1min":0.92300000000000004,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":1.8779999999999999,"5min":1.986,"15min":2.8439999999999999},"last":1.6539999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575027,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1042,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194269,"statfs":{"total":200000000,"available":198931989,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 01:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.85999999999999999,"15min":0.79300000000000004},"min":{"1min":0.58599999999999997,"5min":0.39300000000000002,"15min":0.29499999999999998},"max":{"1min":1.4059999999999999,"5min":1.8009999999999999,"15min":1.8009999999999999},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.94799999999999995,"15min":0.874},"min":{"1min":0.47899999999999998,"5min":0.40300000000000002,"15min":0.36199999999999999},"max":{"1min":1.4379999999999999,"5min":1.591,"15min":1.6399999999999999},"last":0.96699999999999997}]},{"osd":2,"last update":"Thu Oct 26 01:44:59 2023","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":1.137,"15min":1.036},"min":{"1min":0.72099999999999997,"5min":0.52500000000000002,"15min":0.39600000000000002},"max":{"1min":1.4430000000000001,"5min":1.655,"15min":2.4830000000000001},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.93200000000000005,"5min":1.016,"15min":0.95399999999999996},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.41199999999999998},"max":{"1min":1.248,"5min":1.623,"15min":1.8080000000000001},"last":1.0620000000000001}]},{"osd":3,"last update":"Thu Oct 26 01:45:03 2023","interfaces":[{"interface":"back","average":{"1min":1.022,"5min":1.0580000000000001,"15min":1.0089999999999999},"min":{"1min":0.63300000000000001,"5min":0.495,"15min":0.39500000000000002},"max":{"1min":1.776,"5min":1.776,"15min":1.776},"last":1.1659999999999999},{"interface":"front","average":{"1min":1.103,"5min":1.133,"15min":1.0509999999999999},"min":{"1min":0.54000000000000004,"5min":0.51600000000000001,"15min":0.41299999999999998},"max":{"1min":1.7190000000000001,"5min":1.7190000000000001,"15min":1.802},"last":1.1990000000000001}]},{"osd":4,"last update":"Thu Oct 26 01:44:59 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.288,"15min":1.2410000000000001},"min":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.56200000000000006},"max":{"1min":1.589,"5min":1.728,"15min":2.0659999999999998},"last":1.266},{"interface":"front","average":{"1min":1.236,"5min":1.27,"15min":1.1990000000000001},"min":{"1min":0.73899999999999999,"5min":0.70999999999999996,"15min":0.65800000000000003},"max":{"1min":1.6759999999999999,"5min":1.8420000000000001,"15min":2.0270000000000001},"last":1.298}]},{"osd":5,"last update":"Thu Oct 26 01:44:59 2023","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":1.321,"15min":1.27},"min":{"1min":0.79300000000000004,"5min":0.629,"15min":0.61799999999999999},"max":{"1min":1.7829999999999999,"5min":1.7829999999999999,"15min":1.8779999999999999},"last":1.387},{"interface":"front","average":{"1min":1.411,"5min":1.413,"15min":1.369},"min":{"1min":0.80100000000000005,"5min":0.72899999999999998,"15min":0.59999999999999998},"max":{"1min":1.7569999999999999,"5min":1.859,"15min":2.2200000000000002},"last":1.5149999999999999}]},{"osd":6,"last update":"Thu Oct 26 01:44:59 2023","interfaces":[{"interface":"back","average":{"1min":1.3819999999999999,"5min":1.3799999999999999,"15min":1.329},"min":{"1min":1.0760000000000001,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.7949999999999999,"5min":1.8,"15min":2.3780000000000001},"last":1.407},{"interface":"front","average":{"1min":1.2529999999999999,"5min":1.3360000000000001,"15min":1.296},"min":{"1min":0.80400000000000005,"5min":0.77700000000000002,"15min":0.70799999999999996},"max":{"1min":1.694,"5min":1.8819999999999999,"15min":1.984},"last":1.3280000000000001}]},{"osd":7,"last update":"Thu Oct 26 01:44:59 2023","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":1.3480000000000001,"15min":1.2929999999999999},"min":{"1min":0.91600000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":2.0219999999999998,"5min":2.0219999999999998,"15min":2.6930000000000001},"last":1.4590000000000001},{"interface":"front","average":{"1min":1.4119999999999999,"5min":1.448,"15min":1.3999999999999999},"min":{"1min":1.0840000000000001,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.6479999999999999,"5min":1.903,"15min":2.0190000000000001},"last":1.5960000000000001}]}]},{"osd":0,"up_from":13,"seq":55834575027,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1619,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193692,"statfs":{"total":200000000,"available":198341621,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Oct 26 01:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.81000000000000005,"15min":0.872},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.32000000000000001},"max":{"1min":1.1919999999999999,"5min":1.494,"15min":19.425000000000001},"last":0.747},{"interface":"front","average":{"1min":0.92100000000000004,"5min":1.0109999999999999,"15min":1.014},"min":{"1min":0.40799999999999997,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.6859999999999999,"5min":1.6859999999999999,"15min":15.162000000000001},"last":1.228}]},{"osd":2,"last update":"Thu Oct 26 01:44:59 2023","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":1.05,"15min":1.1279999999999999},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.5900000000000001,"5min":1.5900000000000001,"15min":28.439},"last":1.149},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.96199999999999997,"15min":0.91800000000000004},"min":{"1min":0.53000000000000003,"5min":0.40600000000000003,"15min":0.37},"max":{"1min":2.2789999999999999,"5min":2.2789999999999999,"15min":4.9459999999999997},"last":1.127}]},{"osd":3,"last update":"Thu Oct 26 01:44:59 2023","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":1.1499999999999999,"15min":1.089},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.44600000000000001},"max":{"1min":2.1970000000000001,"5min":2.1970000000000001,"15min":4.9800000000000004},"last":1.1910000000000001},{"interface":"front","average":{"1min":1.25,"5min":1.264,"15min":1.173},"min":{"1min":0.499,"5min":0.499,"15min":0.438},"max":{"1min":3.4049999999999998,"5min":3.4049999999999998,"15min":5.016},"last":1.014}]},{"osd":4,"last update":"Thu Oct 26 01:44:59 2023","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.1719999999999999,"15min":1.238},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.52900000000000003},"max":{"1min":1.4650000000000001,"5min":1.5840000000000001,"15min":19.329000000000001},"last":1.0700000000000001},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.2849999999999999,"15min":1.27},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.67600000000000005},"max":{"1min":3.8860000000000001,"5min":3.8860000000000001,"15min":5.5229999999999997},"last":1.4610000000000001}]},{"osd":5,"last update":"Thu Oct 26 01:44:59 2023","interfaces":[{"interface":"back","average":{"1min":1.4179999999999999,"5min":1.4470000000000001,"15min":1.4199999999999999},"min":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.64900000000000002},"max":{"1min":2.395,"5min":2.605,"15min":5.6360000000000001},"last":1.3},{"interface":"front","average":{"1min":1.2210000000000001,"5min":1.286,"15min":1.3560000000000001},"min":{"1min":0.752,"5min":0.752,"15min":0.57499999999999996},"max":{"1min":1.8420000000000001,"5min":2.036,"15min":19.384},"last":1.5800000000000001}]},{"osd":6,"last update":"Thu Oct 26 01:44:59 2023","interfaces":[{"interface":"back","average":{"1min":1.2829999999999999,"5min":1.3220000000000001,"15min":1.337},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.73799999999999999},"max":{"1min":1.8899999999999999,"5min":1.8999999999999999,"15min":8.2609999999999992},"last":1.407},{"interface":"front","average":{"1min":1.3939999999999999,"5min":1.3700000000000001,"15min":1.3899999999999999},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":3.774,"5min":3.774,"15min":11.849},"last":1.355}]},{"osd":7,"last update":"Thu Oct 26 01:44:59 2023","interfaces":[{"interface":"back","average":{"1min":1.4650000000000001,"5min":1.51,"15min":1.4670000000000001},"min":{"1min":1,"5min":1,"15min":0.73699999999999999},"max":{"1min":2.101,"5min":2.101,"15min":5.6689999999999996},"last":1.256},{"interface":"front","average":{"1min":1.3999999999999999,"5min":1.4470000000000001,"15min":1.48},"min":{"1min":0.91000000000000003,"5min":0.79000000000000004,"15min":0.69399999999999995},"max":{"1min":1.972,"5min":1.972,"15min":19.245999999999999},"last":1.5089999999999999}]}]}],"pool_statfs":[]}} 2023-10-26T01:45:16.614 DEBUG:teuthology.orchestra.run.smithi169:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-10-26T01:45:16.820 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:16.817+0000 7f5a35e64640 1 -- 172.21.15.169:0/4100550132 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a1c004d00 msgr2=0x7f5a1c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:45:16.820 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:16.817+0000 7f5a35e64640 1 --2- 172.21.15.169:0/4100550132 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a1c004d00 0x7f5a1c005180 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x7f5a300cf110 tx=0x7f5a1c0057f0 comp rx=0 tx=0).stop 2023-10-26T01:45:16.820 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:16.817+0000 7f5a2ffff640 1 -- 172.21.15.169:0/4100550132 reap_dead start 2023-10-26T01:45:16.820 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:16.817+0000 7f5a35e64640 1 -- 172.21.15.169:0/4100550132 shutdown_connections 2023-10-26T01:45:16.821 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:16.817+0000 7f5a35e64640 1 -- 172.21.15.169:0/4100550132 >> 172.21.15.169:0/4100550132 conn(0x7f5a30065670 msgr2=0x7f5a30065a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:45:16.821 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:16.817+0000 7f5a35e64640 1 -- 172.21.15.169:0/4100550132 shutdown_connections 2023-10-26T01:45:16.821 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:16.817+0000 7f5a35e64640 1 -- 172.21.15.169:0/4100550132 wait complete. 2023-10-26T01:45:16.822 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:16.819+0000 7f5a35e64640 1 Processor -- start 2023-10-26T01:45:16.822 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:16.819+0000 7f5a35e64640 1 -- start start 2023-10-26T01:45:16.824 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:16.820+0000 7f5a35e64640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5a1c004d00 0x7f5a300aa050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:45:16.824 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:16.820+0000 7f5a35e64640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5a30159e10 0x7f5a300ab990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:45:16.824 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:16.820+0000 7f5a35e64640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a3015a270 0x7f5a301f5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:45:16.824 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:16.820+0000 7f5a35e64640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a300a9bc0 con 0x7f5a3015a270 2023-10-26T01:45:16.825 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:16.820+0000 7f5a35e64640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a300a8fb0 con 0x7f5a30159e10 2023-10-26T01:45:16.825 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:16.820+0000 7f5a35e64640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a300a9290 con 0x7f5a1c004d00 2023-10-26T01:45:16.825 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:16.820+0000 7f5a2ffff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a3015a270 0x7f5a301f5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:45:16.825 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:16.820+0000 7f5a2ffff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a3015a270 0x7f5a301f5380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:60768/0 (socket says 172.21.15.169:60768) 2023-10-26T01:45:16.825 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:16.820+0000 7f5a2ffff640 1 -- 172.21.15.169:0/810693921 learned_addr learned my addr 172.21.15.169:0/810693921 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:45:16.826 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:16.820+0000 7f5a2f7fe640 1 --2- 172.21.15.169:0/810693921 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5a1c004d00 0x7f5a300aa050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:45:16.826 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:16.820+0000 7f5a2effd640 1 --2- 172.21.15.169:0/810693921 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5a30159e10 0x7f5a300ab990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:45:16.826 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:16.820+0000 7f5a2ffff640 1 -- 172.21.15.169:0/810693921 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5a1c004d00 msgr2=0x7f5a300aa050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:45:16.826 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:16.820+0000 7f5a2ffff640 1 --2- 172.21.15.169:0/810693921 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5a1c004d00 0x7f5a300aa050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:45:16.826 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:16.820+0000 7f5a2ffff640 1 -- 172.21.15.169:0/810693921 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5a30159e10 msgr2=0x7f5a300ab990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:45:16.827 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:16.820+0000 7f5a2ffff640 1 --2- 172.21.15.169:0/810693921 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5a30159e10 0x7f5a300ab990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:45:16.827 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:16.821+0000 7f5a2ffff640 1 -- 172.21.15.169:0/810693921 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a300a9570 con 0x7f5a3015a270 2023-10-26T01:45:16.827 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:16.821+0000 7f5a2f7fe640 1 --2- 172.21.15.169:0/810693921 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5a1c004d00 0x7f5a300aa050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:45:16.827 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:16.821+0000 7f5a2effd640 1 --2- 172.21.15.169:0/810693921 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5a30159e10 0x7f5a300ab990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:45:16.827 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:16.821+0000 7f5a2ffff640 1 --2- 172.21.15.169:0/810693921 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a3015a270 0x7f5a301f5380 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x7f5a28006fa0 tx=0x7f5a2802dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:45:16.827 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:16.821+0000 7f5a27fff640 1 -- 172.21.15.169:0/810693921 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a2803b070 con 0x7f5a3015a270 2023-10-26T01:45:16.828 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:16.821+0000 7f5a27fff640 1 -- 172.21.15.169:0/810693921 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5a280063f0 con 0x7f5a3015a270 2023-10-26T01:45:16.828 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:16.821+0000 7f5a35e64640 1 -- 172.21.15.169:0/810693921 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a300a9850 con 0x7f5a3015a270 2023-10-26T01:45:16.828 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:16.821+0000 7f5a27fff640 1 -- 172.21.15.169:0/810693921 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a2802df10 con 0x7f5a3015a270 2023-10-26T01:45:16.828 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:16.821+0000 7f5a35e64640 1 -- 172.21.15.169:0/810693921 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a300abed0 con 0x7f5a3015a270 2023-10-26T01:45:16.829 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:16.823+0000 7f5a35e64640 1 -- 172.21.15.169:0/810693921 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a300a9d30 con 0x7f5a3015a270 2023-10-26T01:45:16.829 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:16.823+0000 7f5a27fff640 1 -- 172.21.15.169:0/810693921 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f5a28032070 con 0x7f5a3015a270 2023-10-26T01:45:16.830 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:16.825+0000 7f5a27fff640 1 --2- 172.21.15.169:0/810693921 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5a1006ae40 0x7f5a1006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:45:16.830 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:16.825+0000 7f5a27fff640 1 -- 172.21.15.169:0/810693921 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f5a280a9f60 con 0x7f5a3015a270 2023-10-26T01:45:16.830 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:16.826+0000 7f5a2f7fe640 1 --2- 172.21.15.169:0/810693921 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5a1006ae40 0x7f5a1006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:45:16.830 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:16.827+0000 7f5a2f7fe640 1 --2- 172.21.15.169:0/810693921 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5a1006ae40 0x7f5a1006d300 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f5a20000f60 tx=0x7f5a20006a80 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:45:16.832 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:16.830+0000 7f5a27fff640 1 -- 172.21.15.169:0/810693921 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5a280753a0 con 0x7f5a3015a270 2023-10-26T01:45:17.021 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:17.019+0000 7f5a35e64640 1 -- 172.21.15.169:0/810693921 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f5a3004db10 con 0x7f5a3015a270 2023-10-26T01:45:17.027 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:17.024+0000 7f5a27fff640 1 -- 172.21.15.169:0/810693921 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v39)=0 v39) v1 ==== 123+0+0 (secure 0 0 0) 0x7f5a280789c0 con 0x7f5a3015a270 2023-10-26T01:45:17.028 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:17.025+0000 7f5a27fff640 1 -- 172.21.15.169:0/810693921 <== mon.0 v2:172.21.15.169:3300/0 8 ==== config(6 keys) v1 ==== 172+0+0 (secure 0 0 0) 0x7f5a280753a0 con 0x7f5a3015a270 2023-10-26T01:45:17.030 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:17.027+0000 7f5a35e64640 1 -- 172.21.15.169:0/810693921 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5a1006ae40 msgr2=0x7f5a1006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:45:17.031 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:17.027+0000 7f5a35e64640 1 --2- 172.21.15.169:0/810693921 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5a1006ae40 0x7f5a1006d300 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f5a20000f60 tx=0x7f5a20006a80 comp rx=0 tx=0).stop 2023-10-26T01:45:17.031 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:17.027+0000 7f5a35e64640 1 -- 172.21.15.169:0/810693921 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a3015a270 msgr2=0x7f5a301f5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:45:17.031 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:17.027+0000 7f5a35e64640 1 --2- 172.21.15.169:0/810693921 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a3015a270 0x7f5a301f5380 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x7f5a28006fa0 tx=0x7f5a2802dde0 comp rx=0 tx=0).stop 2023-10-26T01:45:17.031 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:17.027+0000 7f5a35e64640 1 -- 172.21.15.169:0/810693921 shutdown_connections 2023-10-26T01:45:17.031 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:17.027+0000 7f5a35e64640 1 --2- 172.21.15.169:0/810693921 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7f5a1006ae40 0x7f5a1006d300 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:45:17.032 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:17.027+0000 7f5a35e64640 1 --2- 172.21.15.169:0/810693921 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a3015a270 0x7f5a301f5380 unknown :-1 s=CLOSED pgs=2029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:45:17.032 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:17.027+0000 7f5a35e64640 1 --2- 172.21.15.169:0/810693921 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5a30159e10 0x7f5a300ab990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:45:17.032 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:17.027+0000 7f5a35e64640 1 --2- 172.21.15.169:0/810693921 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5a1c004d00 0x7f5a300aa050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:45:17.032 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:17.027+0000 7f5a35e64640 1 -- 172.21.15.169:0/810693921 >> 172.21.15.169:0/810693921 conn(0x7f5a30065670 msgr2=0x7f5a300d1760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:45:17.032 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:17.028+0000 7f5a35e64640 1 -- 172.21.15.169:0/810693921 shutdown_connections 2023-10-26T01:45:17.032 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:17.028+0000 7f5a35e64640 1 -- 172.21.15.169:0/810693921 wait complete. 2023-10-26T01:45:17.049 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-26T01:45:17.050 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T01:45:17.273 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:17.270+0000 7ff0019a0640 1 -- 172.21.15.169:0/813867547 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feffc0c7ea0 msgr2=0x7feffc06a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:45:17.273 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:17.270+0000 7ff0019a0640 1 --2- 172.21.15.169:0/813867547 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feffc0c7ea0 0x7feffc06a990 secure :-1 s=READY pgs=2030 cs=0 l=1 rev1=1 crypto rx=0x7feff8005fd0 tx=0x7feff802c610 comp rx=0 tx=0).stop 2023-10-26T01:45:17.273 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:17.270+0000 7ff0019a0640 1 -- 172.21.15.169:0/813867547 shutdown_connections 2023-10-26T01:45:17.274 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:17.270+0000 7ff0019a0640 1 --2- 172.21.15.169:0/813867547 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7feffc0cc4f0 0x7feffc0cc970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:45:17.274 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:17.270+0000 7ff0019a0640 1 --2- 172.21.15.169:0/813867547 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feffc0c7ea0 0x7feffc06a990 unknown :-1 s=CLOSED pgs=2030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:45:17.274 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:17.270+0000 7ff0019a0640 1 -- 172.21.15.169:0/813867547 >> 172.21.15.169:0/813867547 conn(0x7feffc0aedd0 msgr2=0x7feffc0af1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:45:17.274 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:17.270+0000 7ff0019a0640 1 -- 172.21.15.169:0/813867547 shutdown_connections 2023-10-26T01:45:17.275 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:17.270+0000 7ff0019a0640 1 -- 172.21.15.169:0/813867547 wait complete. 2023-10-26T01:45:17.275 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:17.272+0000 7ff0019a0640 1 Processor -- start 2023-10-26T01:45:17.275 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:17.272+0000 7ff0019a0640 1 -- start start 2023-10-26T01:45:17.277 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:17.273+0000 7ff0019a0640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feffc0c7ea0 0x7feffc14f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:45:17.277 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:17.273+0000 7ff0019a0640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feffc0c96f0 0x7feffc150ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:45:17.277 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:17.273+0000 7ff0019a0640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7feffc0cc4f0 0x7feffc14dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:45:17.277 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:17.273+0000 7ff0019a0640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feffc14fa20 con 0x7feffc0c7ea0 2023-10-26T01:45:17.277 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:17.273+0000 7ff0019a0640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feffc14fd00 con 0x7feffc0c96f0 2023-10-26T01:45:17.278 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:17.273+0000 7ff0019a0640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feffc14ffe0 con 0x7feffc0cc4f0 2023-10-26T01:45:17.278 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:17.273+0000 7ff00099e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feffc0c7ea0 0x7feffc14f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:45:17.278 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:17.274+0000 7ff00099e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feffc0c7ea0 0x7feffc14f3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:60792/0 (socket says 172.21.15.169:60792) 2023-10-26T01:45:17.278 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:17.274+0000 7ff00099e640 1 -- 172.21.15.169:0/3883080757 learned_addr learned my addr 172.21.15.169:0/3883080757 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-10-26T01:45:17.278 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:17.274+0000 7ff00119f640 1 --2- 172.21.15.169:0/3883080757 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7feffc0cc4f0 0x7feffc14dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:45:17.279 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:17.274+0000 7feff3fff640 1 --2- 172.21.15.169:0/3883080757 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feffc0c96f0 0x7feffc150ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:45:17.279 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:17.274+0000 7ff00099e640 1 -- 172.21.15.169:0/3883080757 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7feffc0cc4f0 msgr2=0x7feffc14dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:45:17.279 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:17.274+0000 7ff00099e640 1 --2- 172.21.15.169:0/3883080757 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7feffc0cc4f0 0x7feffc14dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:45:17.279 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:17.274+0000 7ff00099e640 1 -- 172.21.15.169:0/3883080757 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feffc0c96f0 msgr2=0x7feffc150ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:45:17.279 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:17.274+0000 7ff00099e640 1 --2- 172.21.15.169:0/3883080757 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feffc0c96f0 0x7feffc150ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:45:17.280 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:17.274+0000 7ff00099e640 1 -- 172.21.15.169:0/3883080757 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feff8009df0 con 0x7feffc0c7ea0 2023-10-26T01:45:17.280 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:17.274+0000 7feff3fff640 1 --2- 172.21.15.169:0/3883080757 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feffc0c96f0 0x7feffc150ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T01:45:17.280 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:17.274+0000 7ff00099e640 1 --2- 172.21.15.169:0/3883080757 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feffc0c7ea0 0x7feffc14f3d0 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 crypto rx=0x7feff8000c00 tx=0x7feff80346c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:45:17.281 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:17.275+0000 7feff17fa640 1 -- 172.21.15.169:0/3883080757 <== mon.0 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff8034cc0 con 0x7feffc0c7ea0 2023-10-26T01:45:17.281 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:17.275+0000 7feff17fa640 1 -- 172.21.15.169:0/3883080757 <== mon.0 v2:172.21.15.169:3300/0 2 ==== config(6 keys) v1 ==== 172+0+0 (secure 0 0 0) 0x7feff802ce90 con 0x7feffc0c7ea0 2023-10-26T01:45:17.282 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:17.275+0000 7feff17fa640 1 -- 172.21.15.169:0/3883080757 <== mon.0 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff8004290 con 0x7feffc0c7ea0 2023-10-26T01:45:17.282 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:17.275+0000 7ff0019a0640 1 -- 172.21.15.169:0/3883080757 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feffc157d50 con 0x7feffc0c7ea0 2023-10-26T01:45:17.282 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:17.275+0000 7ff0019a0640 1 -- 172.21.15.169:0/3883080757 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feffc1505a0 con 0x7feffc0c7ea0 2023-10-26T01:45:17.282 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:17.276+0000 7ff0019a0640 1 -- 172.21.15.169:0/3883080757 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feffc150790 con 0x7feffc0c7ea0 2023-10-26T01:45:17.282 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:17.277+0000 7feff17fa640 1 -- 172.21.15.169:0/3883080757 <== mon.0 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7feff80035f0 con 0x7feffc0c7ea0 2023-10-26T01:45:17.283 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:17.278+0000 7feff17fa640 1 --2- 172.21.15.169:0/3883080757 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fefe806ae40 0x7fefe806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T01:45:17.283 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:17.278+0000 7feff17fa640 1 -- 172.21.15.169:0/3883080757 <== mon.0 v2:172.21.15.169:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 4943+0+0 (secure 0 0 0) 0x7feff80ac330 con 0x7feffc0c7ea0 2023-10-26T01:45:17.283 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:17.279+0000 7feff3fff640 1 --2- 172.21.15.169:0/3883080757 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fefe806ae40 0x7fefe806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T01:45:17.284 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:17.280+0000 7feff3fff640 1 --2- 172.21.15.169:0/3883080757 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fefe806ae40 0x7fefe806d300 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fefec0058c0 tx=0x7fefec005c90 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T01:45:17.284 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:17.282+0000 7feff17fa640 1 -- 172.21.15.169:0/3883080757 <== mon.0 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7feff8077820 con 0x7feffc0c7ea0 2023-10-26T01:45:17.511 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:17.509+0000 7ff0019a0640 1 -- 172.21.15.169:0/3883080757 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feffc073c10 con 0x7feffc0c7ea0 2023-10-26T01:45:17.513 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:17.510+0000 7feff17fa640 1 -- 172.21.15.169:0/3883080757 <== mon.0 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) v1 ==== 94+0+3702 (secure 0 0 0) 0x7feff804ce90 con 0x7feffc0c7ea0 2023-10-26T01:45:17.513 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-10-26T01:45:17.514 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":96,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6125,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/1491206818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":1491206818},{"type":"v1","addr":"172.21.15.170:6837","nonce":1491206818}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":96},{"gid":6484,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/195700049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":195700049},{"type":"v1","addr":"172.21.15.169:6835","nonce":195700049}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":89},{"gid":6493,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/1167696300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1167696300},{"type":"v1","addr":"172.21.15.170:6836","nonce":1167696300}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":89},{"gid":9111,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/22729060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":22729060},{"type":"v1","addr":"172.21.15.169:6834","nonce":22729060}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":89}],"filesystems":[]} 2023-10-26T01:45:17.514 INFO:teuthology.orchestra.run.smithi169.stderr:dumped fsmap epoch 96 2023-10-26T01:45:17.516 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:17.513+0000 7ff0019a0640 1 -- 172.21.15.169:0/3883080757 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fefe806ae40 msgr2=0x7fefe806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:45:17.516 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:17.513+0000 7ff0019a0640 1 --2- 172.21.15.169:0/3883080757 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fefe806ae40 0x7fefe806d300 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fefec0058c0 tx=0x7fefec005c90 comp rx=0 tx=0).stop 2023-10-26T01:45:17.516 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:17.513+0000 7ff0019a0640 1 -- 172.21.15.169:0/3883080757 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feffc0c7ea0 msgr2=0x7feffc14f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T01:45:17.516 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:17.513+0000 7ff0019a0640 1 --2- 172.21.15.169:0/3883080757 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feffc0c7ea0 0x7feffc14f3d0 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 crypto rx=0x7feff8000c00 tx=0x7feff80346c0 comp rx=0 tx=0).stop 2023-10-26T01:45:17.517 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:17.513+0000 7ff0019a0640 1 -- 172.21.15.169:0/3883080757 shutdown_connections 2023-10-26T01:45:17.517 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:17.513+0000 7ff0019a0640 1 --2- 172.21.15.169:0/3883080757 >> [v2:172.21.15.170:6832/1187156918,v1:172.21.15.170:6833/1187156918] conn(0x7fefe806ae40 0x7fefe806d300 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:45:17.517 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:17.513+0000 7ff0019a0640 1 --2- 172.21.15.169:0/3883080757 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7feffc0cc4f0 0x7feffc14dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:45:17.517 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:17.513+0000 7ff0019a0640 1 --2- 172.21.15.169:0/3883080757 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feffc0c96f0 0x7feffc150ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:45:17.517 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:17.513+0000 7ff0019a0640 1 --2- 172.21.15.169:0/3883080757 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feffc0c7ea0 0x7feffc14f3d0 unknown :-1 s=CLOSED pgs=2031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T01:45:17.518 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:17.513+0000 7ff0019a0640 1 -- 172.21.15.169:0/3883080757 >> 172.21.15.169:0/3883080757 conn(0x7feffc0aedd0 msgr2=0x7feffc154ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T01:45:17.518 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:17.513+0000 7ff0019a0640 1 -- 172.21.15.169:0/3883080757 shutdown_connections 2023-10-26T01:45:17.518 INFO:teuthology.orchestra.run.smithi169.stderr:2023-10-26T01:45:17.513+0000 7ff0019a0640 1 -- 172.21.15.169:0/3883080757 wait complete. 2023-10-26T01:45:17.536 DEBUG:tasks.cephfs.filesystem:already dead... 2023-10-26T01:45:17.536 INFO:teuthology.misc:Shutting down mds daemons... 2023-10-26T01:45:17.537 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-26T01:45:17.537 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T01:45:17.553 INFO:tasks.ceph.mds.a:Stopped 2023-10-26T01:45:17.553 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-26T01:45:17.554 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T01:45:17.618 INFO:tasks.ceph.mds.c:Stopped 2023-10-26T01:45:17.619 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-26T01:45:17.619 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T01:45:17.640 INFO:tasks.ceph.mds.b:Stopped 2023-10-26T01:45:17.641 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-26T01:45:17.641 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T01:45:17.711 INFO:tasks.ceph.mds.d:Stopped 2023-10-26T01:45:17.711 INFO:teuthology.misc:Shutting down osd daemons... 2023-10-26T01:45:17.712 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-10-26T01:45:17.712 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T01:45:17.780 INFO:tasks.ceph.osd.0:Stopped 2023-10-26T01:45:17.781 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-10-26T01:45:17.781 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T01:45:19.906 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~0s 2023-10-26T01:45:23.883 INFO:tasks.ceph.osd.1:Stopped 2023-10-26T01:45:23.884 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-10-26T01:45:23.884 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T01:45:25.913 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2023-10-26T01:45:25.913 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 has been restored 2023-10-26T01:45:29.986 INFO:tasks.ceph.osd.2:Stopped 2023-10-26T01:45:29.987 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-10-26T01:45:29.988 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T01:45:31.820 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2023-10-26T01:45:31.821 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 has been restored 2023-10-26T01:45:36.090 INFO:tasks.ceph.osd.3:Stopped 2023-10-26T01:45:36.090 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-10-26T01:45:36.091 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T01:45:36.168 INFO:tasks.ceph.osd.4:Stopped 2023-10-26T01:45:36.169 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-10-26T01:45:36.169 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T01:45:36.238 INFO:tasks.ceph.osd.5:Stopped 2023-10-26T01:45:36.239 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-10-26T01:45:36.239 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T01:45:36.302 INFO:tasks.ceph.osd.6:Stopped 2023-10-26T01:45:36.303 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-10-26T01:45:36.303 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T01:45:36.370 INFO:tasks.ceph.osd.7:Stopped 2023-10-26T01:45:36.371 INFO:teuthology.misc:Shutting down mgr daemons... 2023-10-26T01:45:36.372 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-10-26T01:45:36.372 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T01:45:36.446 INFO:tasks.ceph.mgr.y:Stopped 2023-10-26T01:45:36.447 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-10-26T01:45:36.447 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T01:45:36.526 INFO:tasks.ceph.mgr.x:Stopped 2023-10-26T01:45:36.526 INFO:teuthology.misc:Shutting down mon daemons... 2023-10-26T01:45:36.527 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-10-26T01:45:36.527 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T01:45:36.581 INFO:tasks.ceph.mon.a:Stopped 2023-10-26T01:45:36.581 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-10-26T01:45:36.581 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T01:45:36.619 INFO:tasks.ceph.mon.b:Stopped 2023-10-26T01:45:36.620 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-10-26T01:45:36.620 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T01:45:36.706 INFO:tasks.ceph.mon.c:Stopped 2023-10-26T01:45:36.706 INFO:tasks.ceph:Checking cluster log for badness... 2023-10-26T01:45:36.707 DEBUG:teuthology.orchestra.run.smithi169:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | egrep -v PG_RECOVERY_FULL | egrep -v PG_DEGRADED | head -n 1 2023-10-26T01:45:36.821 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 has been restored 2023-10-26T01:45:36.823 INFO:teuthology.orchestra.run.smithi169.stdout:2023-10-26T01:30:29.910396+0000 osd.1 (osd.1) 3 : cluster [WRN] OSD bench result of 97444.837208 IOPS exceeded the threshold limit of 500.000000 IOPS for osd.1. IOPS capacity is unchanged at 315.000000 IOPS. The recommendation is to establish the osd's IOPS capacity using other benchmark tools (e.g. Fio) and then override osd_mclock_max_capacity_iops_[hdd|ssd]. 2023-10-26T01:45:36.824 WARNING:tasks.ceph:Found errors (ERR|WRN|SEC) in cluster log 2023-10-26T01:45:36.825 DEBUG:teuthology.orchestra.run.smithi169:> sudo egrep '\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | egrep -v PG_RECOVERY_FULL | egrep -v PG_DEGRADED | head -n 1 2023-10-26T01:45:36.921 DEBUG:teuthology.orchestra.run.smithi169:> sudo egrep '\[ERR\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | egrep -v PG_RECOVERY_FULL | egrep -v PG_DEGRADED | head -n 1 2023-10-26T01:45:37.011 DEBUG:teuthology.orchestra.run.smithi169:> sudo egrep '\[WRN\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | egrep -v PG_RECOVERY_FULL | egrep -v PG_DEGRADED | head -n 1 2023-10-26T01:45:37.103 INFO:teuthology.orchestra.run.smithi169.stdout:2023-10-26T01:30:29.910396+0000 osd.1 (osd.1) 3 : cluster [WRN] OSD bench result of 97444.837208 IOPS exceeded the threshold limit of 500.000000 IOPS for osd.1. IOPS capacity is unchanged at 315.000000 IOPS. The recommendation is to establish the osd's IOPS capacity using other benchmark tools (e.g. Fio) and then override osd_mclock_max_capacity_iops_[hdd|ssd]. 2023-10-26T01:45:37.105 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi169.front.sepia.ceph.com 2023-10-26T01:45:37.105 DEBUG:teuthology.orchestra.run.smithi169:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-10-26T01:45:37.306 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi169.front.sepia.ceph.com 2023-10-26T01:45:37.307 DEBUG:teuthology.orchestra.run.smithi169:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-10-26T01:45:37.454 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi169.front.sepia.ceph.com 2023-10-26T01:45:37.454 DEBUG:teuthology.orchestra.run.smithi169:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-10-26T01:45:37.618 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi169.front.sepia.ceph.com 2023-10-26T01:45:37.618 DEBUG:teuthology.orchestra.run.smithi169:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-10-26T01:45:37.786 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi170.front.sepia.ceph.com 2023-10-26T01:45:37.786 DEBUG:teuthology.orchestra.run.smithi170:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-10-26T01:45:38.043 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi170.front.sepia.ceph.com 2023-10-26T01:45:38.043 DEBUG:teuthology.orchestra.run.smithi170:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-10-26T01:45:38.202 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi170.front.sepia.ceph.com 2023-10-26T01:45:38.203 DEBUG:teuthology.orchestra.run.smithi170:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-10-26T01:45:38.360 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi170.front.sepia.ceph.com 2023-10-26T01:45:38.360 DEBUG:teuthology.orchestra.run.smithi170:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-10-26T01:45:38.509 INFO:tasks.ceph:Archiving mon data... 2023-10-26T01:45:38.510 DEBUG:teuthology.misc:Transferring archived files from smithi169:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/pdonnell-2023-10-24_19:52:40-fs-wip-batrick-testing-20231024.144545-distro-default-smithi/7436434/data/mon.a.tgz 2023-10-26T01:45:38.511 DEBUG:teuthology.orchestra.run.smithi169:> mktemp 2023-10-26T01:45:38.571 INFO:teuthology.orchestra.run.smithi169.stdout:/tmp/tmp.HGQCdGaSj3 2023-10-26T01:45:38.571 DEBUG:teuthology.orchestra.run.smithi169:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.HGQCdGaSj3 2023-10-26T01:45:38.622 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T01:45:38.628 DEBUG:teuthology.orchestra.run.smithi170:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T01:45:38.785 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod 0666 /tmp/tmp.HGQCdGaSj3 2023-10-26T01:45:38.934 DEBUG:teuthology.orchestra.remote:smithi169:/tmp/tmp.HGQCdGaSj3 is 503KB 2023-10-26T01:45:39.043 DEBUG:teuthology.orchestra.run.smithi169:> rm -fr /tmp/tmp.HGQCdGaSj3 2023-10-26T01:45:39.105 DEBUG:teuthology.misc:Transferring archived files from smithi170:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/pdonnell-2023-10-24_19:52:40-fs-wip-batrick-testing-20231024.144545-distro-default-smithi/7436434/data/mon.b.tgz 2023-10-26T01:45:39.105 DEBUG:teuthology.orchestra.run.smithi170:> mktemp 2023-10-26T01:45:39.165 INFO:teuthology.orchestra.run.smithi170.stdout:/tmp/tmp.kiDgDaKMh4 2023-10-26T01:45:39.166 DEBUG:teuthology.orchestra.run.smithi170:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.kiDgDaKMh4 2023-10-26T01:45:39.399 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod 0666 /tmp/tmp.kiDgDaKMh4 2023-10-26T01:45:39.555 DEBUG:teuthology.orchestra.remote:smithi170:/tmp/tmp.kiDgDaKMh4 is 533KB 2023-10-26T01:45:39.666 DEBUG:teuthology.orchestra.run.smithi170:> rm -fr /tmp/tmp.kiDgDaKMh4 2023-10-26T01:45:39.726 DEBUG:teuthology.misc:Transferring archived files from smithi170:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/pdonnell-2023-10-24_19:52:40-fs-wip-batrick-testing-20231024.144545-distro-default-smithi/7436434/data/mon.c.tgz 2023-10-26T01:45:39.726 DEBUG:teuthology.orchestra.run.smithi170:> mktemp 2023-10-26T01:45:39.780 INFO:teuthology.orchestra.run.smithi170.stdout:/tmp/tmp.qO4lq81tud 2023-10-26T01:45:39.781 DEBUG:teuthology.orchestra.run.smithi170:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.qO4lq81tud 2023-10-26T01:45:39.992 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod 0666 /tmp/tmp.qO4lq81tud 2023-10-26T01:45:40.146 DEBUG:teuthology.orchestra.remote:smithi170:/tmp/tmp.qO4lq81tud is 533KB 2023-10-26T01:45:40.256 DEBUG:teuthology.orchestra.run.smithi170:> rm -fr /tmp/tmp.qO4lq81tud 2023-10-26T01:45:40.318 INFO:tasks.ceph:Cleaning ceph cluster... 2023-10-26T01:45:40.318 DEBUG:teuthology.orchestra.run.smithi169:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-10-26T01:45:40.327 DEBUG:teuthology.orchestra.run.smithi170:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-10-26T01:45:40.430 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-10-26T01:45:40.431 DEBUG:teuthology.orchestra.run.smithi169:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-10-26T01:45:40.440 DEBUG:teuthology.orchestra.run.smithi170:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-10-26T01:45:40.541 INFO:teuthology.orchestra.run.smithi169.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-10-26T01:45:40.545 INFO:teuthology.orchestra.run.smithi170.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-10-26T01:45:40.559 INFO:tasks.ceph:Archiving crash dumps... 2023-10-26T01:45:40.561 DEBUG:teuthology.misc:Transferring archived files from smithi169:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2023-10-24_19:52:40-fs-wip-batrick-testing-20231024.144545-distro-default-smithi/7436434/remote/smithi169/crash 2023-10-26T01:45:40.562 DEBUG:teuthology.orchestra.run.smithi169:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-10-26T01:45:40.652 DEBUG:teuthology.misc:Transferring archived files from smithi170:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2023-10-24_19:52:40-fs-wip-batrick-testing-20231024.144545-distro-default-smithi/7436434/remote/smithi170/crash 2023-10-26T01:45:40.653 DEBUG:teuthology.orchestra.run.smithi170:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-10-26T01:45:40.754 INFO:tasks.ceph:Shutting down logrotate 2023-10-26T01:45:40.755 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T01:45:40.764 DEBUG:teuthology.orchestra.run.smithi170:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T01:45:40.889 DEBUG:teuthology.orchestra.run.smithi169:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-10-26T01:45:40.991 DEBUG:teuthology.orchestra.run.smithi170:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-10-26T01:45:41.093 INFO:tasks.ceph:Compressing logs... 2023-10-26T01:45:41.094 DEBUG:teuthology.orchestra.run.smithi169:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-10-26T01:45:41.103 DEBUG:teuthology.orchestra.run.smithi170:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-10-26T01:45:41.195 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2023-10-26T01:45:41.196 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98305.log 2023-10-26T01:45:41.196 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.83048.log 2023-10-26T01:45:41.197 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80372.log 2023-10-26T01:45:41.197 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.84583.log 2023-10-26T01:45:41.198 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.98305.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95715.log 2023-10-26T01:45:41.198 INFO:teuthology.orchestra.run.smithi169.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.98305.log.gz 2023-10-26T01:45:41.200 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.1.83048.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.1.83048.log.gz -5 2023-10-26T01:45:41.200 INFO:teuthology.orchestra.run.smithi170.stderr: --verbose -- /var/log/ceph/ceph-client.1.84587.log 2023-10-26T01:45:41.200 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.1.84583.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.84583.log.gz 2023-10-26T01:45:41.200 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.82060.log 2023-10-26T01:45:41.201 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.80372.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.92635.log 2023-10-26T01:45:41.201 INFO:teuthology.orchestra.run.smithi169.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.80372.log.gz 2023-10-26T01:45:41.203 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.95715.log: gzip -5 --verbose -- 86.9% /var/log/ceph/ceph-client.admin.103677.log 2023-10-26T01:45:41.203 INFO:teuthology.orchestra.run.smithi169.stderr: -- replaced with /var/log/ceph/ceph-client.admin.95715.log.gz 2023-10-26T01:45:41.203 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.0.92635.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.92635.log.gz 2023-10-26T01:45:41.203 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99877.log 2023-10-26T01:45:41.204 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.1.84587.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79904.log 2023-10-26T01:45:41.204 INFO:teuthology.orchestra.run.smithi170.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.84587.log.gz 2023-10-26T01:45:41.204 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.1.82060.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.82060.log.gz 2023-10-26T01:45:41.205 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.86224.log 2023-10-26T01:45:41.205 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.79904.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.86228.log 2023-10-26T01:45:41.205 INFO:teuthology.orchestra.run.smithi170.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.79904.log.gz 2023-10-26T01:45:41.205 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.1.86224.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.86224.log.gz 2023-10-26T01:45:41.205 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.82554.log 2023-10-26T01:45:41.208 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.1.86228.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.86228.log.gz 2023-10-26T01:45:41.208 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79944.log 2023-10-26T01:45:41.208 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.1.82554.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.84082.log 2023-10-26T01:45:41.208 INFO:teuthology.orchestra.run.smithi170.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.82554.log.gz 2023-10-26T01:45:41.208 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.79944.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.79944.log.gz 2023-10-26T01:45:41.209 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-10-26T01:45:41.210 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.103677.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114748.log 2023-10-26T01:45:41.210 INFO:teuthology.orchestra.run.smithi169.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.103677.log.gz 2023-10-26T01:45:41.210 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.99877.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105736.log 2023-10-26T01:45:41.211 INFO:teuthology.orchestra.run.smithi169.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.99877.log.gz 2023-10-26T01:45:41.211 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.114748.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91906.log 2023-10-26T01:45:41.211 INFO:teuthology.orchestra.run.smithi169.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.114748.log.gz 2023-10-26T01:45:41.211 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.105736.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.105736.log.gz 2023-10-26T01:45:41.211 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99777.log 2023-10-26T01:45:41.212 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.91906.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80602.log 2023-10-26T01:45:41.212 INFO:teuthology.orchestra.run.smithi169.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.91906.log.gz 2023-10-26T01:45:41.212 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.99777.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83741.log 2023-10-26T01:45:41.212 INFO:teuthology.orchestra.run.smithi169.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.99777.log.gz 2023-10-26T01:45:41.212 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.80602.log: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.80602.log.gz 2023-10-26T01:45:41.213 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119868.log 2023-10-26T01:45:41.216 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.83741.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88151.log 87.9% 2023-10-26T01:45:41.216 INFO:teuthology.orchestra.run.smithi169.stderr: -- replaced with /var/log/ceph/ceph-client.admin.83741.log.gz 2023-10-26T01:45:41.216 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.119868.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88667.log 2023-10-26T01:45:41.216 INFO:teuthology.orchestra.run.smithi169.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.119868.log.gz 2023-10-26T01:45:41.217 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.88151.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83657.log 2023-10-26T01:45:41.217 INFO:teuthology.orchestra.run.smithi169.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.88151.log.gz 2023-10-26T01:45:41.217 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.88667.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2023-10-26T01:45:41.218 INFO:teuthology.orchestra.run.smithi169.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.88667.log.gz 2023-10-26T01:45:41.218 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.83657.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95757.log 2023-10-26T01:45:41.218 INFO:teuthology.orchestra.run.smithi169.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.83657.log.gz 2023-10-26T01:45:41.221 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.1.84082.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2023-10-26T01:45:41.221 INFO:teuthology.orchestra.run.smithi170.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.84082.log.gz 2023-10-26T01:45:41.221 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.85673.log 2023-10-26T01:45:41.221 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.85073.log 2023-10-26T01:45:41.221 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.1.85673.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.85673.log.gz 2023-10-26T01:45:41.222 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79824.log 2023-10-26T01:45:41.222 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.1.85073.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.85073.log.gz 2023-10-26T01:45:41.222 INFO:teuthology.orchestra.run.smithi170.stderr: 94.0% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-10-26T01:45:41.222 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.81448.log 2023-10-26T01:45:41.222 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2023-10-26T01:45:41.223 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.79824.log: /var/log/ceph/ceph-client.3.81448.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.81448.log.gz 2023-10-26T01:45:41.223 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80068.log 2023-10-26T01:45:41.223 INFO:teuthology.orchestra.run.smithi170.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.79824.log.gz 2023-10-26T01:45:41.223 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.81057.log 2023-10-26T01:45:41.223 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.80068.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.80068.log.gz 2023-10-26T01:45:41.224 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.86750.log 2023-10-26T01:45:41.224 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.1.81057.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.81057.log.gz 2023-10-26T01:45:41.224 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.82550.log 2023-10-26T01:45:41.231 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114473.log 2023-10-26T01:45:41.231 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.95757.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87274.log 2023-10-26T01:45:41.231 INFO:teuthology.orchestra.run.smithi169.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.95757.log.gz 2023-10-26T01:45:41.232 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.114473.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83041.log 2023-10-26T01:45:41.232 INFO:teuthology.orchestra.run.smithi169.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.114473.log.gz 2023-10-26T01:45:41.232 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.87274.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.87274.log.gz 2023-10-26T01:45:41.232 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100652.log 2023-10-26T01:45:41.233 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.83041.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91677.log 2023-10-26T01:45:41.233 INFO:teuthology.orchestra.run.smithi169.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.83041.log.gz 2023-10-26T01:45:41.233 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.100652.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87527.log 2023-10-26T01:45:41.233 INFO:teuthology.orchestra.run.smithi169.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.100652.log.gz 2023-10-26T01:45:41.233 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.91677.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121088.log 2023-10-26T01:45:41.234 INFO:teuthology.orchestra.run.smithi169.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.91677.log.gz 2023-10-26T01:45:41.234 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.87527.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.87527.log.gz 2023-10-26T01:45:41.234 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100442.log 2023-10-26T01:45:41.234 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.121088.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.121088.log.gz 2023-10-26T01:45:41.234 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115974.log 2023-10-26T01:45:41.235 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.100442.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.100442.log.gz 2023-10-26T01:45:41.235 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84119.log 2023-10-26T01:45:41.235 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.115974.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.115974.log.gz 2023-10-26T01:45:41.235 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101031.log 2023-10-26T01:45:41.235 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.84119.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.84119.log.gz 2023-10-26T01:45:41.235 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115286.log 2023-10-26T01:45:41.238 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.1.86750.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.86750.log.gz 2023-10-26T01:45:41.238 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.86754.log 2023-10-26T01:45:41.238 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.1.82550.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.82550.log.gz 2023-10-26T01:45:41.238 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2023-10-26T01:45:41.239 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.1.86754.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.86754.log.gz 2023-10-26T01:45:41.239 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2023-10-26T01:45:41.239 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80115.log 2023-10-26T01:45:41.239 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.85669.log 2023-10-26T01:45:41.239 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.80115.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.83584.log 2023-10-26T01:45:41.240 INFO:teuthology.orchestra.run.smithi170.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.80115.log.gz 2023-10-26T01:45:41.240 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.1.85669.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.85669.log.gz 2023-10-26T01:45:41.240 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.82056.log 2023-10-26T01:45:41.240 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.1.83584.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.83584.log.gz 2023-10-26T01:45:41.241 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.81048.log 2023-10-26T01:45:41.241 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.1.82056.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.82056.log.gz 2023-10-26T01:45:41.241 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.83588.log 2023-10-26T01:45:41.241 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.1.81048.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.81048.log.gz 2023-10-26T01:45:41.241 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80024.log 2023-10-26T01:45:41.242 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.1.83588.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.83588.log.gz 2023-10-26T01:45:41.242 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2023-10-26T01:45:41.247 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.80024.log: 86.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.1.85077.log 2023-10-26T01:45:41.247 INFO:teuthology.orchestra.run.smithi170.stderr: -- replaced with /var/log/ceph/ceph-client.admin.80024.log.gz 2023-10-26T01:45:41.247 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79864.log 2023-10-26T01:45:41.247 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.1.85077.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.85077.log.gz 2023-10-26T01:45:41.248 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.79864.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.79864.log.gz 2023-10-26T01:45:41.248 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-10-26T01:45:41.248 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.83052.log 2023-10-26T01:45:41.250 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph.log: /var/log/ceph/ceph-client.1.83052.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.83052.log.gz 2023-10-26T01:45:41.250 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2023-10-26T01:45:41.252 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.101031.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.101031.log.gz 2023-10-26T01:45:41.252 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119952.log 2023-10-26T01:45:41.252 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.115286.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.115286.log.gz 2023-10-26T01:45:41.252 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91734.log 2023-10-26T01:45:41.252 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.119952.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.119952.log.gz 2023-10-26T01:45:41.253 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119826.log 2023-10-26T01:45:41.253 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.91734.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.91734.log.gz 2023-10-26T01:45:41.253 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83040.log 2023-10-26T01:45:41.253 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.119826.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.119826.log.gz 2023-10-26T01:45:41.253 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111772.log 2023-10-26T01:45:41.254 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.83040.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.83040.log.gz 2023-10-26T01:45:41.254 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87443.log 2023-10-26T01:45:41.254 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.111772.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.111772.log.gz 2023-10-26T01:45:41.254 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97480.log 2023-10-26T01:45:41.254 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.87443.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.87443.log.gz 2023-10-26T01:45:41.255 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104295.log 2023-10-26T01:45:41.255 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.97480.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.97480.log.gz 2023-10-26T01:45:41.255 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120138.log 2023-10-26T01:45:41.255 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.104295.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.104295.log.gz 2023-10-26T01:45:41.255 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82061.log 2023-10-26T01:45:41.256 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.120138.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.120138.log.gz 2023-10-26T01:45:41.256 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104506.log 2023-10-26T01:45:41.256 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.82061.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.82061.log.gz 2023-10-26T01:45:41.256 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.105456.log 2023-10-26T01:45:41.256 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.104506.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.104506.log.gz 2023-10-26T01:45:41.257 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86154.log 2023-10-26T01:45:41.257 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.0.105456.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.105456.log.gz 2023-10-26T01:45:41.257 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96259.log 2023-10-26T01:45:41.257 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.86154.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.86154.log.gz 2023-10-26T01:45:41.257 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90916.log 2023-10-26T01:45:41.258 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.96259.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.96259.log.gz 2023-10-26T01:45:41.258 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82695.log 2023-10-26T01:45:41.258 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.90916.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.90916.log.gz 2023-10-26T01:45:41.258 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80124.log 2023-10-26T01:45:41.260 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-mon.b.log: 92.7% -- replaced with /var/log/ceph/ceph.log.gz 2023-10-26T01:45:41.260 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.84086.log 2023-10-26T01:45:41.261 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2023-10-26T01:45:41.261 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.1.84086.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.84086.log.gz 2023-10-26T01:45:41.261 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.81444.log 2023-10-26T01:45:41.261 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79984.log 2023-10-26T01:45:41.261 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2023-10-26T01:45:41.264 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.82695.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.82695.log.gz 2023-10-26T01:45:41.264 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115178.log 2023-10-26T01:45:41.265 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.80124.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.80124.log.gz 2023-10-26T01:45:41.265 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117920.log 2023-10-26T01:45:41.265 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.115178.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.115178.log.gz 2023-10-26T01:45:41.265 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93195.log 2023-10-26T01:45:41.265 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.117920.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.117920.log.gz 2023-10-26T01:45:41.266 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104781.log 2023-10-26T01:45:41.266 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.93195.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.93195.log.gz 2023-10-26T01:45:41.266 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113174.log 2023-10-26T01:45:41.266 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.104781.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.104781.log.gz 2023-10-26T01:45:41.266 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117454.log 2023-10-26T01:45:41.267 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.113174.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.113174.log.gz 2023-10-26T01:45:41.267 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102135.log 2023-10-26T01:45:41.267 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.117454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88487.log 2023-10-26T01:45:41.267 INFO:teuthology.orchestra.run.smithi169.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.117454.log.gz 2023-10-26T01:45:41.268 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.102135.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.102135.log.gz 2023-10-26T01:45:41.268 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90664.log 2023-10-26T01:45:41.268 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.88487.log: gzip 87.5% -- replaced with /var/log/ceph/ceph-client.admin.88487.log.gz 2023-10-26T01:45:41.268 INFO:teuthology.orchestra.run.smithi169.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.81470.log 2023-10-26T01:45:41.271 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-mds.d.log: /var/log/ceph/ceph-client.3.81444.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.81444.log.gz 2023-10-26T01:45:41.271 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-mon.c.log: /var/log/ceph/ceph-client.admin.79984.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.79984.log.gz 2023-10-26T01:45:41.273 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.90664.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.90664.log.gz 2023-10-26T01:45:41.273 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90706.log 2023-10-26T01:45:41.273 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.81470.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.81470.log.gz 2023-10-26T01:45:41.274 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104731.log 2023-10-26T01:45:41.274 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.90706.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115220.log 2023-10-26T01:45:41.274 INFO:teuthology.orchestra.run.smithi169.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.90706.log.gz 2023-10-26T01:45:41.274 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.104731.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.104731.log.gz 2023-10-26T01:45:41.274 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119480.log 2023-10-26T01:45:41.275 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.115220.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88573.log 2023-10-26T01:45:41.275 INFO:teuthology.orchestra.run.smithi169.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.115220.log.gz 2023-10-26T01:45:41.275 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.119480.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.119480.log.gz 2023-10-26T01:45:41.275 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91776.log 2023-10-26T01:45:41.276 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.88573.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.88573.log.gz 2023-10-26T01:45:41.276 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88837.log 2023-10-26T01:45:41.277 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.91776.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.91776.log.gz 2023-10-26T01:45:41.278 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101591.log 2023-10-26T01:45:41.278 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.88837.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.88837.log.gz 2023-10-26T01:45:41.278 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111134.log 2023-10-26T01:45:41.278 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.101591.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.101591.log.gz 2023-10-26T01:45:41.278 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86825.log 2023-10-26T01:45:41.279 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.111134.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.111134.log.gz 2023-10-26T01:45:41.279 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105161.log 2023-10-26T01:45:41.280 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.86825.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.86825.log.gz 2023-10-26T01:45:41.280 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98821.log 2023-10-26T01:45:41.280 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.105161.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81593.log 2023-10-26T01:45:41.281 INFO:teuthology.orchestra.run.smithi169.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.105161.log.gz 2023-10-26T01:45:41.281 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.98821.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.98821.log.gz 2023-10-26T01:45:41.282 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94880.log 2023-10-26T01:45:41.282 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.81593.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.81593.log.gz 2023-10-26T01:45:41.283 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93859.log 2023-10-26T01:45:41.283 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.94880.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.94880.log.gz 2023-10-26T01:45:41.284 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81807.log 2023-10-26T01:45:41.285 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.93859.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91271.log 2023-10-26T01:45:41.285 INFO:teuthology.orchestra.run.smithi169.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.93859.log.gz 2023-10-26T01:45:41.286 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.81807.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.81807.log.gz 2023-10-26T01:45:41.286 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83357.log 2023-10-26T01:45:41.287 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.91271.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.91271.log.gz 2023-10-26T01:45:41.287 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106161.log 2023-10-26T01:45:41.288 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.83357.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.83357.log.gz 2023-10-26T01:45:41.288 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113778.log 2023-10-26T01:45:41.289 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.106161.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114558.log 2023-10-26T01:45:41.290 INFO:teuthology.orchestra.run.smithi169.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.106161.log.gz 2023-10-26T01:45:41.290 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.113778.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.113778.log.gz 2023-10-26T01:45:41.291 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87695.log 2023-10-26T01:45:41.292 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.114558.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.114558.log.gz 2023-10-26T01:45:41.292 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107342.log 2023-10-26T01:45:41.293 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.87695.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.87695.log.gz 2023-10-26T01:45:41.293 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98093.log 2023-10-26T01:45:41.294 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.107342.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.107342.log.gz 2023-10-26T01:45:41.295 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114882.log 2023-10-26T01:45:41.295 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.98093.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.98093.log.gz 2023-10-26T01:45:41.296 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106759.log 2023-10-26T01:45:41.297 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.114882.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.114882.log.gz 2023-10-26T01:45:41.297 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104032.log 2023-10-26T01:45:41.298 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.106759.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.106759.log.gz 2023-10-26T01:45:41.298 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80245.log 2023-10-26T01:45:41.299 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.104032.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.104032.log.gz 2023-10-26T01:45:41.300 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113216.log 2023-10-26T01:45:41.300 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.80245.log: 78.3% -- replaced with /var/log/ceph/ceph-client.admin.80245.log.gz 2023-10-26T01:45:41.301 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111644.log 2023-10-26T01:45:41.302 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.113216.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.113216.log.gz 2023-10-26T01:45:41.302 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90580.log 2023-10-26T01:45:41.303 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.111644.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.111644.log.gz 2023-10-26T01:45:41.303 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106533.log 2023-10-26T01:45:41.304 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.90580.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.90580.log.gz 2023-10-26T01:45:41.305 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80859.log 2023-10-26T01:45:41.306 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.106533.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.78579.log 2023-10-26T01:45:41.306 INFO:teuthology.orchestra.run.smithi169.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.106533.log.gz 2023-10-26T01:45:41.306 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.80859.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.80859.log.gz 2023-10-26T01:45:41.307 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87611.log 2023-10-26T01:45:41.308 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph.tmp-client.admin.78579.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.78579.log.gz 2023-10-26T01:45:41.308 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121004.log 2023-10-26T01:45:41.310 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.87611.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.87611.log.gz 2023-10-26T01:45:41.310 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87017.log 2023-10-26T01:45:41.311 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.121004.log: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.121004.log.gz 2023-10-26T01:45:41.311 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99735.log 2023-10-26T01:45:41.313 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.87017.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.87017.log.gz 2023-10-26T01:45:41.313 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98653.log 2023-10-26T01:45:41.315 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.99735.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.99735.log.gz 2023-10-26T01:45:41.315 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112886.log 2023-10-26T01:45:41.316 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.98653.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.98653.log.gz 2023-10-26T01:45:41.316 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90312.log 2023-10-26T01:45:41.318 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.112886.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.112886.log.gz 2023-10-26T01:45:41.319 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109792.log 2023-10-26T01:45:41.319 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.90312.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.90312.log.gz 2023-10-26T01:45:41.319 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109177.log 2023-10-26T01:45:41.325 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.109792.log: gzip 87.2% -- replaced with /var/log/ceph/ceph-client.admin.109792.log.gz -5 2023-10-26T01:45:41.325 INFO:teuthology.orchestra.run.smithi169.stderr: --verbose -- /var/log/ceph/ceph-client.admin.118939.log 2023-10-26T01:45:41.325 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.109177.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.109177.log.gz 2023-10-26T01:45:41.326 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107937.log 2023-10-26T01:45:41.326 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.118939.log: gzip -5 --verbose 87.0% -- -- replaced with /var/log/ceph/ceph-client.admin.118939.log.gz /var/log/ceph/ceph-client.admin.103551.log 2023-10-26T01:45:41.326 INFO:teuthology.orchestra.run.smithi169.stderr: 2023-10-26T01:45:41.326 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.107937.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.107937.log.gz 2023-10-26T01:45:41.327 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96981.log 2023-10-26T01:45:41.327 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.103551.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.103551.log.gz 2023-10-26T01:45:41.327 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121130.log 2023-10-26T01:45:41.416 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.96981.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.96981.log.gz 2023-10-26T01:45:41.416 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82481.log 2023-10-26T01:45:41.418 INFO:teuthology.orchestra.run.smithi170.stderr: 93.1% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2023-10-26T01:45:41.418 INFO:teuthology.orchestra.run.smithi170.stderr: 92.5% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2023-10-26T01:45:41.456 INFO:teuthology.orchestra.run.smithi170.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2023-10-26T01:45:41.461 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.121130.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.121130.log.gz 2023-10-26T01:45:41.462 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117372.log 2023-10-26T01:45:41.462 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.82481.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.82481.log.gz 2023-10-26T01:45:41.462 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101825.log 2023-10-26T01:45:41.463 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.117372.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.117372.log.gz 2023-10-26T01:45:41.463 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94169.log 2023-10-26T01:45:41.463 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.101825.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.101825.log.gz 2023-10-26T01:45:41.464 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100869.log 2023-10-26T01:45:41.464 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.94169.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.94169.log.gz 2023-10-26T01:45:41.464 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101287.log 2023-10-26T01:45:41.464 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.100869.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.100869.log.gz 2023-10-26T01:45:41.465 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120709.log 2023-10-26T01:45:41.465 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.101287.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.101287.log.gz 2023-10-26T01:45:41.465 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113610.log 2023-10-26T01:45:41.465 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.120709.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.120709.log.gz 2023-10-26T01:45:41.466 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106843.log 2023-10-26T01:45:41.466 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.113610.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.113610.log.gz 2023-10-26T01:45:41.466 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97522.log 2023-10-26T01:45:41.466 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.106843.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.106843.log.gz 2023-10-26T01:45:41.467 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88751.log 2023-10-26T01:45:41.467 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.97522.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.97522.log.gz 2023-10-26T01:45:41.467 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81678.log 2023-10-26T01:45:41.467 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.88751.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.88751.log.gz 2023-10-26T01:45:41.468 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114840.log 2023-10-26T01:45:41.468 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.81678.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.81678.log.gz 2023-10-26T01:45:41.468 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95411.log 2023-10-26T01:45:41.468 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.114840.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.114840.log.gz 2023-10-26T01:45:41.469 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120667.log 2023-10-26T01:45:41.469 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.95411.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.95411.log.gz 2023-10-26T01:45:41.469 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105820.log 2023-10-26T01:45:41.469 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.120667.log: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.120667.log.gz 2023-10-26T01:45:41.470 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90790.log 2023-10-26T01:45:41.470 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.105820.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.105820.log.gz 2023-10-26T01:45:41.470 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119438.log 2023-10-26T01:45:41.470 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.90790.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.90790.log.gz 2023-10-26T01:45:41.470 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117197.log 2023-10-26T01:45:41.471 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.119438.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.119438.log.gz 2023-10-26T01:45:41.471 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92168.log 2023-10-26T01:45:41.471 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.117197.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.117197.log.gz 2023-10-26T01:45:41.471 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112586.log 2023-10-26T01:45:41.472 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.92168.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.92168.log.gz 2023-10-26T01:45:41.472 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91998.log 2023-10-26T01:45:41.472 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.112586.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.112586.log.gz 2023-10-26T01:45:41.472 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106801.log 2023-10-26T01:45:41.473 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.91998.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.91998.log.gz 2023-10-26T01:45:41.473 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103821.log 2023-10-26T01:45:41.473 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.106801.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.106801.log.gz 2023-10-26T01:45:41.473 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.105460.log 2023-10-26T01:45:41.473 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.103821.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.103821.log.gz 2023-10-26T01:45:41.474 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84839.log 2023-10-26T01:45:41.474 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.0.105460.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.105460.log.gz 2023-10-26T01:45:41.474 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120624.log 2023-10-26T01:45:41.474 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.84839.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.84839.log.gz 2023-10-26T01:45:41.475 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92949.log 2023-10-26T01:45:41.475 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.120624.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.120624.log.gz 2023-10-26T01:45:41.475 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119726.log 2023-10-26T01:45:41.475 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.92949.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.92949.log.gz 2023-10-26T01:45:41.475 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119184.log 2023-10-26T01:45:41.476 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.119726.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.119726.log.gz 2023-10-26T01:45:41.477 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114926.log 2023-10-26T01:45:41.478 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.119184.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.119184.log.gz 2023-10-26T01:45:41.478 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96635.log 2023-10-26T01:45:41.478 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.114926.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.114926.log.gz 2023-10-26T01:45:41.478 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108485.log 2023-10-26T01:45:41.479 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.96635.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.96635.log.gz 2023-10-26T01:45:41.479 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108987.log 2023-10-26T01:45:41.479 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.108485.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.108485.log.gz 2023-10-26T01:45:41.479 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93066.log 2023-10-26T01:45:41.480 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.108987.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.108987.log.gz 2023-10-26T01:45:41.480 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99573.log 2023-10-26T01:45:41.480 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.93066.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.93066.log.gz 2023-10-26T01:45:41.480 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86699.log 2023-10-26T01:45:41.480 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.99573.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.99573.log.gz 2023-10-26T01:45:41.481 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91956.log 2023-10-26T01:45:41.481 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.86699.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.86699.log.gz 2023-10-26T01:45:41.481 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83029.log 2023-10-26T01:45:41.482 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.91956.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.91956.log.gz 2023-10-26T01:45:41.482 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115094.log 2023-10-26T01:45:41.482 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.83029.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.83029.log.gz 2023-10-26T01:45:41.482 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107216.log 2023-10-26T01:45:41.483 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.115094.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.115094.log.gz 2023-10-26T01:45:41.483 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116800.log 2023-10-26T01:45:41.483 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.107216.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.107216.log.gz 2023-10-26T01:45:41.483 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95323.log 2023-10-26T01:45:41.483 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.116800.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.116800.log.gz 2023-10-26T01:45:41.484 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87485.log 2023-10-26T01:45:41.484 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.95323.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.95323.log.gz 2023-10-26T01:45:41.484 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97732.log 2023-10-26T01:45:41.484 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.87485.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.87485.log.gz 2023-10-26T01:45:41.485 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93024.log 2023-10-26T01:45:41.485 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.97732.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.97732.log.gz 2023-10-26T01:45:41.485 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102595.log 2023-10-26T01:45:41.485 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.93024.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.93024.log.gz 2023-10-26T01:45:41.486 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.84701.log 2023-10-26T01:45:41.486 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.102595.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.102595.log.gz 2023-10-26T01:45:41.486 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100400.log 2023-10-26T01:45:41.486 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.0.84701.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.84701.log.gz 2023-10-26T01:45:41.487 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92126.log 2023-10-26T01:45:41.487 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.100400.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.100400.log.gz 2023-10-26T01:45:41.487 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117710.log 2023-10-26T01:45:41.487 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.92126.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.92126.log.gz 2023-10-26T01:45:41.488 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90024.log 2023-10-26T01:45:41.488 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.117710.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.117710.log.gz 2023-10-26T01:45:41.488 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83699.log 2023-10-26T01:45:41.488 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.90024.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.90024.log.gz 2023-10-26T01:45:41.489 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116530.log 2023-10-26T01:45:41.489 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.83699.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.83699.log.gz 2023-10-26T01:45:41.489 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116987.log 2023-10-26T01:45:41.489 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.116530.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.116530.log.gz 2023-10-26T01:45:41.489 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120390.log 2023-10-26T01:45:41.490 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.116987.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.116987.log.gz 2023-10-26T01:45:41.490 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116945.log 2023-10-26T01:45:41.490 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.120390.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.120390.log.gz 2023-10-26T01:45:41.490 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88319.log 2023-10-26T01:45:41.491 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.116945.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.116945.log.gz 2023-10-26T01:45:41.491 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102466.log 2023-10-26T01:45:41.491 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.88319.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.88319.log.gz 2023-10-26T01:45:41.491 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121466.log 2023-10-26T01:45:41.492 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.102466.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114217.log 2023-10-26T01:45:41.492 INFO:teuthology.orchestra.run.smithi169.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.102466.log.gz 2023-10-26T01:45:41.492 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.121466.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.121466.log.gz 2023-10-26T01:45:41.492 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113258.log 2023-10-26T01:45:41.492 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.114217.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.114217.log.gz 2023-10-26T01:45:41.493 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103157.log 2023-10-26T01:45:41.493 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.113258.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.113258.log.gz 2023-10-26T01:45:41.493 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98219.log 2023-10-26T01:45:41.493 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.103157.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.103157.log.gz 2023-10-26T01:45:41.494 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103425.log 2023-10-26T01:45:41.494 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.98219.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.98219.log.gz 2023-10-26T01:45:41.494 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111730.log 2023-10-26T01:45:41.494 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.103425.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.103425.log.gz 2023-10-26T01:45:41.495 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111982.log 2023-10-26T01:45:41.495 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.111730.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.111730.log.gz 2023-10-26T01:45:41.495 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85692.log 2023-10-26T01:45:41.496 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.111982.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.111982.log.gz 2023-10-26T01:45:41.496 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107011.log 2023-10-26T01:45:41.496 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.85692.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.85692.log.gz 2023-10-26T01:45:41.496 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106449.log 2023-10-26T01:45:41.497 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.107011.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.107011.log.gz 2023-10-26T01:45:41.497 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105915.log 2023-10-26T01:45:41.497 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.106449.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.106449.log.gz 2023-10-26T01:45:41.498 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100484.log 2023-10-26T01:45:41.498 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.105915.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.105915.log.gz 2023-10-26T01:45:41.498 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118821.log 2023-10-26T01:45:41.498 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.100484.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.100484.log.gz 2023-10-26T01:45:41.499 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116136.log 2023-10-26T01:45:41.499 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.118821.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.118821.log.gz 2023-10-26T01:45:41.499 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119558.log 2023-10-26T01:45:41.499 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.116136.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.116136.log.gz 2023-10-26T01:45:41.500 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106245.log 2023-10-26T01:45:41.500 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.119558.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.119558.log.gz 2023-10-26T01:45:41.500 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117496.log 2023-10-26T01:45:41.500 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.106245.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.106245.log.gz 2023-10-26T01:45:41.501 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107089.log 2023-10-26T01:45:41.501 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.117496.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105119.log 87.9% 2023-10-26T01:45:41.501 INFO:teuthology.orchestra.run.smithi169.stderr: -- replaced with /var/log/ceph/ceph-client.admin.117496.log.gz 2023-10-26T01:45:41.502 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.107089.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.107089.log.gz 2023-10-26T01:45:41.502 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109378.log 2023-10-26T01:45:41.502 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.105119.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.105119.log.gz 2023-10-26T01:45:41.502 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100357.log 2023-10-26T01:45:41.503 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.109378.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.109378.log.gz 2023-10-26T01:45:41.503 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109454.log 2023-10-26T01:45:41.503 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.100357.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.100357.log.gz 2023-10-26T01:45:41.503 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110038.log 2023-10-26T01:45:41.504 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.109454.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.109454.log.gz 2023-10-26T01:45:41.504 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96217.log 2023-10-26T01:45:41.504 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.110038.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.110038.log.gz 2023-10-26T01:45:41.504 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114392.log 2023-10-26T01:45:41.505 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.96217.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.96217.log.gz 2023-10-26T01:45:41.505 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83867.log 2023-10-26T01:45:41.505 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.114392.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.114392.log.gz 2023-10-26T01:45:41.505 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92210.log 2023-10-26T01:45:41.505 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.83867.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110492.log 87.3% 2023-10-26T01:45:41.506 INFO:teuthology.orchestra.run.smithi169.stderr: -- replaced with /var/log/ceph/ceph-client.admin.83867.log.gz 2023-10-26T01:45:41.506 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.92210.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.92210.log.gz 2023-10-26T01:45:41.506 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101717.log 2023-10-26T01:45:41.506 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.110492.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.110492.log.gz 2023-10-26T01:45:41.507 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102721.log 2023-10-26T01:45:41.507 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.101717.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.101717.log.gz 2023-10-26T01:45:41.507 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95673.log 2023-10-26T01:45:41.507 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.102721.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.102721.log.gz 2023-10-26T01:45:41.508 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109954.log 2023-10-26T01:45:41.508 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.95673.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.95673.log.gz 2023-10-26T01:45:41.508 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87317.log 2023-10-26T01:45:41.508 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.109954.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.109954.log.gz 2023-10-26T01:45:41.508 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107851.log 2023-10-26T01:45:41.509 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.87317.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.87317.log.gz 2023-10-26T01:45:41.509 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119910.log 2023-10-26T01:45:41.509 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.107851.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.107851.log.gz 2023-10-26T01:45:41.510 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95281.log 2023-10-26T01:45:41.510 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.119910.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.119910.log.gz 2023-10-26T01:45:41.510 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113965.log 2023-10-26T01:45:41.510 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.95281.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.95281.log.gz 2023-10-26T01:45:41.510 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82145.log 2023-10-26T01:45:41.511 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.113965.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.113965.log.gz 2023-10-26T01:45:41.511 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80646.log 2023-10-26T01:45:41.511 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.82145.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.82145.log.gz 2023-10-26T01:45:41.511 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101073.log 2023-10-26T01:45:41.511 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.80646.log: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.80646.log.gz 2023-10-26T01:45:41.511 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98441.log 2023-10-26T01:45:41.512 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.101073.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.101073.log.gz 2023-10-26T01:45:41.512 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100171.log 2023-10-26T01:45:41.512 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.98441.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.98441.log.gz 2023-10-26T01:45:41.512 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111602.log 2023-10-26T01:45:41.512 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.100171.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.100171.log.gz 2023-10-26T01:45:41.513 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81935.log 2023-10-26T01:45:41.513 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.111602.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.111602.log.gz 2023-10-26T01:45:41.513 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85818.log 2023-10-26T01:45:41.513 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.81935.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.81935.log.gz 2023-10-26T01:45:41.513 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106717.log 2023-10-26T01:45:41.513 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.85818.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.85818.log.gz 2023-10-26T01:45:41.514 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120793.log 2023-10-26T01:45:41.514 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.106717.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.106717.log.gz 2023-10-26T01:45:41.514 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120432.log 2023-10-26T01:45:41.514 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.120793.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116698.log 2023-10-26T01:45:41.514 INFO:teuthology.orchestra.run.smithi169.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.120793.log.gz 2023-10-26T01:45:41.515 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.120432.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.120432.log.gz 2023-10-26T01:45:41.515 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102995.log 2023-10-26T01:45:41.515 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.116698.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.116698.log.gz 2023-10-26T01:45:41.515 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107174.log 2023-10-26T01:45:41.515 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.102995.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.102995.log.gz 2023-10-26T01:45:41.516 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121566.log 2023-10-26T01:45:41.538 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.107174.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.107174.log.gz 2023-10-26T01:45:41.538 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120096.log 2023-10-26T01:45:41.538 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.121566.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.121566.log.gz 2023-10-26T01:45:41.539 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81851.log 2023-10-26T01:45:41.539 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.120096.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.120096.log.gz 2023-10-26T01:45:41.539 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118761.log 2023-10-26T01:45:41.539 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.81851.log: gzip 86.7% -5 -- replaced with /var/log/ceph/ceph-client.admin.81851.log.gz --verbose 2023-10-26T01:45:41.540 INFO:teuthology.orchestra.run.smithi169.stderr: -- /var/log/ceph/ceph-client.admin.91061.log 2023-10-26T01:45:41.540 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.118761.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.118761.log.gz 2023-10-26T01:45:41.540 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83345.log 2023-10-26T01:45:41.540 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.91061.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.91061.log.gz 2023-10-26T01:45:41.540 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93655.log 2023-10-26T01:45:41.541 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.83345.log: gzip -5 --verbose -- 87.0% /var/log/ceph/ceph-client.admin.116262.log 2023-10-26T01:45:41.541 INFO:teuthology.orchestra.run.smithi169.stderr: -- replaced with /var/log/ceph/ceph-client.admin.83345.log.gz 2023-10-26T01:45:41.541 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.93655.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.93655.log.gz 2023-10-26T01:45:41.541 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118238.log 2023-10-26T01:45:41.541 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.116262.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.116262.log.gz 2023-10-26T01:45:41.542 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.85139.log 2023-10-26T01:45:41.542 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.118238.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.118238.log.gz 2023-10-26T01:45:41.542 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80416.log 2023-10-26T01:45:41.542 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.2.85139.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.85139.log.gz 2023-10-26T01:45:41.542 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121298.log 2023-10-26T01:45:41.543 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.80416.log: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.80416.log.gz 2023-10-26T01:45:41.543 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100568.log 2023-10-26T01:45:41.543 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.121298.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.121298.log.gz 2023-10-26T01:45:41.543 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90270.log 2023-10-26T01:45:41.544 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.100568.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.100568.log.gz 2023-10-26T01:45:41.544 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86975.log 2023-10-26T01:45:41.544 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.90270.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.90270.log.gz 2023-10-26T01:45:41.544 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104375.log 2023-10-26T01:45:41.544 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.86975.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.86975.log.gz 2023-10-26T01:45:41.545 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116016.log 2023-10-26T01:45:41.545 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.104375.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.104375.log.gz 2023-10-26T01:45:41.545 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111814.log 2023-10-26T01:45:41.545 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.116016.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.116016.log.gz 2023-10-26T01:45:41.546 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120264.log 2023-10-26T01:45:41.546 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.111814.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87863.log 2023-10-26T01:45:41.546 INFO:teuthology.orchestra.run.smithi169.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.111814.log.gz 2023-10-26T01:45:41.546 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.120264.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.120264.log.gz 2023-10-26T01:45:41.546 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100045.log 2023-10-26T01:45:41.547 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.87863.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.87863.log.gz 2023-10-26T01:45:41.547 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85986.log 2023-10-26T01:45:41.547 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.100045.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.100045.log.gz 2023-10-26T01:45:41.547 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90538.log 2023-10-26T01:45:41.547 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.85986.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.85986.log.gz 2023-10-26T01:45:41.548 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82271.log 2023-10-26T01:45:41.548 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.90538.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.90538.log.gz 2023-10-26T01:45:41.548 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94379.log 2023-10-26T01:45:41.548 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.82271.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.82271.log.gz 2023-10-26T01:45:41.549 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95461.log 2023-10-26T01:45:41.549 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.94379.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.94379.log.gz 2023-10-26T01:45:41.549 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99651.log 2023-10-26T01:45:41.549 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.95461.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.95461.log.gz 2023-10-26T01:45:41.550 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89640.log 2023-10-26T01:45:41.550 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.99651.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.99651.log.gz 2023-10-26T01:45:41.550 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105778.log 2023-10-26T01:45:41.551 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.89640.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.89640.log.gz 2023-10-26T01:45:41.551 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110366.log 2023-10-26T01:45:41.551 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.105778.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.105778.log.gz 2023-10-26T01:45:41.552 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99489.log 2023-10-26T01:45:41.552 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.110366.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.110366.log.gz 2023-10-26T01:45:41.552 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118172.log 2023-10-26T01:45:41.553 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.99489.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.99489.log.gz 2023-10-26T01:45:41.553 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113568.log 2023-10-26T01:45:41.553 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.118172.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121340.log 2023-10-26T01:45:41.553 INFO:teuthology.orchestra.run.smithi169.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.118172.log.gz 2023-10-26T01:45:41.553 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.113568.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.113568.log.gz 2023-10-26T01:45:41.554 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109047.log 2023-10-26T01:45:41.554 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.121340.log: gzip 86.5% -- replaced with /var/log/ceph/ceph-client.admin.121340.log.gz -5 2023-10-26T01:45:41.554 INFO:teuthology.orchestra.run.smithi169.stderr: --verbose -- /var/log/ceph/ceph-client.admin.97427.log 2023-10-26T01:45:41.554 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.109047.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.109047.log.gz 2023-10-26T01:45:41.555 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89107.log 2023-10-26T01:45:41.555 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.97427.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.97427.log.gz 2023-10-26T01:45:41.555 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102424.log 2023-10-26T01:45:41.555 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.89107.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85944.log 2023-10-26T01:45:41.555 INFO:teuthology.orchestra.run.smithi169.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.89107.log.gz 2023-10-26T01:45:41.555 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.102424.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.102424.log.gz 2023-10-26T01:45:41.556 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86028.log 2023-10-26T01:45:41.556 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.85944.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.85944.log.gz 2023-10-26T01:45:41.556 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87737.log 2023-10-26T01:45:41.556 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.86028.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.86028.log.gz 2023-10-26T01:45:41.556 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117071.log 2023-10-26T01:45:41.557 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.87737.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.87737.log.gz 2023-10-26T01:45:41.557 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88027.log 2023-10-26T01:45:41.557 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.117071.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.117071.log.gz 2023-10-26T01:45:41.557 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97836.log 2023-10-26T01:45:41.557 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.88027.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.88027.log.gz 2023-10-26T01:45:41.558 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90396.log 2023-10-26T01:45:41.558 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.97836.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.97836.log.gz 2023-10-26T01:45:41.558 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90438.log 2023-10-26T01:45:41.558 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.90396.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.90396.log.gz 2023-10-26T01:45:41.559 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96939.log 2023-10-26T01:45:41.559 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.90438.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.90438.log.gz 2023-10-26T01:45:41.559 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96509.log 2023-10-26T01:45:41.559 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.96939.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.96939.log.gz 2023-10-26T01:45:41.559 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88193.log 2023-10-26T01:45:41.560 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.96509.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.96509.log.gz 2023-10-26T01:45:41.560 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97293.log 2023-10-26T01:45:41.560 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.88193.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.88193.log.gz 2023-10-26T01:45:41.560 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100297.log 2023-10-26T01:45:41.560 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.97293.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.97293.log.gz 2023-10-26T01:45:41.561 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81977.log 2023-10-26T01:45:41.561 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.100297.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.100297.log.gz 2023-10-26T01:45:41.561 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116362.log 2023-10-26T01:45:41.561 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.81977.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.81977.log.gz 2023-10-26T01:45:41.562 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116843.log 2023-10-26T01:45:41.562 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.116362.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.116362.log.gz 2023-10-26T01:45:41.562 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118046.log 2023-10-26T01:45:41.562 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.116843.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.116843.log.gz 2023-10-26T01:45:41.563 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110240.log 2023-10-26T01:45:41.563 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.118046.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.118046.log.gz 2023-10-26T01:45:41.563 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93529.log 2023-10-26T01:45:41.563 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.110240.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.110240.log.gz 2023-10-26T01:45:41.564 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97065.log 2023-10-26T01:45:41.564 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.93529.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.93529.log.gz 2023-10-26T01:45:41.564 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93817.log 2023-10-26T01:45:41.564 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.97065.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.97065.log.gz 2023-10-26T01:45:41.565 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121172.log 2023-10-26T01:45:41.565 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.93817.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.93817.log.gz 2023-10-26T01:45:41.565 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-10-26T01:45:41.579 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.121172.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.121172.log.gz 2023-10-26T01:45:41.580 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104643.log 2023-10-26T01:45:41.580 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86196.log 2023-10-26T01:45:41.580 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.104643.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.104643.log.gz 2023-10-26T01:45:41.581 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107643.log 2023-10-26T01:45:41.581 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.86196.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.86196.log.gz 2023-10-26T01:45:41.581 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91818.log 2023-10-26T01:45:41.581 INFO:teuthology.orchestra.run.smithi169.stderr: 89.8% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-10-26T01:45:41.581 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.107643.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.107643.log.gz 2023-10-26T01:45:41.582 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101759.log 2023-10-26T01:45:41.582 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.91818.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.91818.log.gz 2023-10-26T01:45:41.582 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91862.log 2023-10-26T01:45:41.583 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.101759.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80635.log 2023-10-26T01:45:41.583 INFO:teuthology.orchestra.run.smithi169.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.101759.log.gz 2023-10-26T01:45:41.583 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.91862.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.91862.log.gz 2023-10-26T01:45:41.583 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91187.log 2023-10-26T01:45:41.584 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.80635.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.80635.log.gz 2023-10-26T01:45:41.584 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86741.log 2023-10-26T01:45:41.584 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.91187.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.91187.log.gz 2023-10-26T01:45:41.584 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101507.log 2023-10-26T01:45:41.584 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.86741.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.86741.log.gz 2023-10-26T01:45:41.585 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116220.log 2023-10-26T01:45:41.585 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.101507.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.101507.log.gz 2023-10-26T01:45:41.585 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99961.log 2023-10-26T01:45:41.585 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.116220.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.116220.log.gz 2023-10-26T01:45:41.585 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104559.log 2023-10-26T01:45:41.586 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.99961.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.99961.log.gz 2023-10-26T01:45:41.586 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106407.log 2023-10-26T01:45:41.586 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.104559.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.104559.log.gz 2023-10-26T01:45:41.586 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95545.log 2023-10-26T01:45:41.586 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.106407.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.106407.log.gz 2023-10-26T01:45:41.587 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83299.log 2023-10-26T01:45:41.587 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.95545.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.95545.log.gz 2023-10-26T01:45:41.587 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106927.log 2023-10-26T01:45:41.587 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.83299.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.83299.log.gz 2023-10-26T01:45:41.587 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107981.log 2023-10-26T01:45:41.588 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.106927.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.106927.log.gz 2023-10-26T01:45:41.588 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84035.log 2023-10-26T01:45:41.588 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.107981.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.107981.log.gz 2023-10-26T01:45:41.588 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107132.log 2023-10-26T01:45:41.588 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.84035.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.84035.log.gz 2023-10-26T01:45:41.589 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103761.log 2023-10-26T01:45:41.589 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.107132.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.107132.log.gz 2023-10-26T01:45:41.589 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82688.log 2023-10-26T01:45:41.589 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.103761.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.103761.log.gz 2023-10-26T01:45:41.590 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96881.log 2023-10-26T01:45:41.590 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.82688.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.82688.log.gz 2023-10-26T01:45:41.590 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83036.log 2023-10-26T01:45:41.590 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.96881.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89173.log 2023-10-26T01:45:41.591 INFO:teuthology.orchestra.run.smithi169.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.96881.log.gz 2023-10-26T01:45:41.591 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.83036.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.83036.log.gz 2023-10-26T01:45:41.591 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100774.log 2023-10-26T01:45:41.591 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.89173.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.89173.log.gz 2023-10-26T01:45:41.592 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113694.log 2023-10-26T01:45:41.592 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.100774.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.100774.log.gz 2023-10-26T01:45:41.592 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106365.log 2023-10-26T01:45:41.592 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.113694.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.113694.log.gz 2023-10-26T01:45:41.593 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85776.log 2023-10-26T01:45:41.593 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.106365.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.106365.log.gz 2023-10-26T01:45:41.593 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95073.log 2023-10-26T01:45:41.593 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.85776.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.85776.log.gz 2023-10-26T01:45:41.593 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103509.log 2023-10-26T01:45:41.594 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.95073.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.95073.log.gz 2023-10-26T01:45:41.594 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.96136.log 2023-10-26T01:45:41.594 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.103509.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.103509.log.gz 2023-10-26T01:45:41.594 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98177.log 2023-10-26T01:45:41.594 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.0.96136.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.96136.log.gz 2023-10-26T01:45:41.595 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116404.log 2023-10-26T01:45:41.595 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.98177.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106675.log 2023-10-26T01:45:41.595 INFO:teuthology.orchestra.run.smithi169.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.98177.log.gz 2023-10-26T01:45:41.595 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.116404.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88879.log 2023-10-26T01:45:41.596 INFO:teuthology.orchestra.run.smithi169.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.116404.log.gz 2023-10-26T01:45:41.596 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.106675.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81763.log 2023-10-26T01:45:41.596 INFO:teuthology.orchestra.run.smithi169.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.106675.log.gz 2023-10-26T01:45:41.596 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.88879.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.88879.log.gz 2023-10-26T01:45:41.596 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80633.log 2023-10-26T01:45:41.604 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.81763.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82397.log 2023-10-26T01:45:41.605 INFO:teuthology.orchestra.run.smithi169.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.81763.log.gz 2023-10-26T01:45:41.605 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.80633.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92040.log 2023-10-26T01:45:41.605 INFO:teuthology.orchestra.run.smithi169.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.80633.log.gz 2023-10-26T01:45:41.605 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.82397.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94566.log 2023-10-26T01:45:41.605 INFO:teuthology.orchestra.run.smithi169.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.82397.log.gz 2023-10-26T01:45:41.606 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.92040.log: gzip 87.5% -- replaced with /var/log/ceph/ceph-client.admin.92040.log.gz -5 2023-10-26T01:45:41.606 INFO:teuthology.orchestra.run.smithi169.stderr: --verbose -- /var/log/ceph/ceph-client.admin.120222.log 2023-10-26T01:45:41.606 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.94566.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107809.log 2023-10-26T01:45:41.606 INFO:teuthology.orchestra.run.smithi169.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.94566.log.gz 2023-10-26T01:45:41.607 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.120222.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91145.log 2023-10-26T01:45:41.607 INFO:teuthology.orchestra.run.smithi169.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.120222.log.gz 2023-10-26T01:45:41.607 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.107809.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87779.log 2023-10-26T01:45:41.607 INFO:teuthology.orchestra.run.smithi169.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.107809.log.gz 2023-10-26T01:45:41.608 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.91145.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.91145.log.gz 2023-10-26T01:45:41.608 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107724.log 2023-10-26T01:45:41.608 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.87779.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102093.log 2023-10-26T01:45:41.608 INFO:teuthology.orchestra.run.smithi169.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.87779.log.gz 2023-10-26T01:45:41.608 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.107724.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116740.log 2023-10-26T01:45:41.609 INFO:teuthology.orchestra.run.smithi169.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.107724.log.gz 2023-10-26T01:45:41.609 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.102093.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89005.log 2023-10-26T01:45:41.609 INFO:teuthology.orchestra.run.smithi169.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.102093.log.gz 2023-10-26T01:45:41.609 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.116740.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92906.log 2023-10-26T01:45:41.609 INFO:teuthology.orchestra.run.smithi169.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.116740.log.gz 2023-10-26T01:45:41.610 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.89005.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90108.log 2023-10-26T01:45:41.610 INFO:teuthology.orchestra.run.smithi169.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.89005.log.gz 2023-10-26T01:45:41.610 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.92906.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96593.log 2023-10-26T01:45:41.610 INFO:teuthology.orchestra.run.smithi169.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.92906.log.gz 2023-10-26T01:45:41.610 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.90108.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.90108.log.gz 2023-10-26T01:45:41.611 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103719.log 2023-10-26T01:45:41.611 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.96593.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112108.log 2023-10-26T01:45:41.611 INFO:teuthology.orchestra.run.smithi169.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.96593.log.gz 2023-10-26T01:45:41.611 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.103719.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89047.log 2023-10-26T01:45:41.611 INFO:teuthology.orchestra.run.smithi169.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.103719.log.gz 2023-10-26T01:45:41.612 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.112108.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109834.log 2023-10-26T01:45:41.612 INFO:teuthology.orchestra.run.smithi169.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.112108.log.gz 2023-10-26T01:45:41.612 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.89047.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88109.log 2023-10-26T01:45:41.612 INFO:teuthology.orchestra.run.smithi169.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.89047.log.gz 2023-10-26T01:45:41.613 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.109834.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119994.log 2023-10-26T01:45:41.613 INFO:teuthology.orchestra.run.smithi169.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.109834.log.gz 2023-10-26T01:45:41.613 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.88109.log: gzip -5 --verbose 87.3% -- -- replaced with /var/log/ceph/ceph-client.admin.88109.log.gz /var/log/ceph/ceph-client.admin.82699.log 2023-10-26T01:45:41.613 INFO:teuthology.orchestra.run.smithi169.stderr: 2023-10-26T01:45:41.614 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.119994.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82523.log 2023-10-26T01:45:41.614 INFO:teuthology.orchestra.run.smithi169.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.119994.log.gz 2023-10-26T01:45:41.619 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.82699.log: gzip -5 87.2% --verbose -- replaced with /var/log/ceph/ceph-client.admin.82699.log.gz -- 2023-10-26T01:45:41.620 INFO:teuthology.orchestra.run.smithi169.stderr: /var/log/ceph/ceph-client.admin.118130.log 2023-10-26T01:45:41.620 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.82523.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116320.log 2023-10-26T01:45:41.620 INFO:teuthology.orchestra.run.smithi169.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.82523.log.gz 2023-10-26T01:45:41.620 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.118130.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.118130.log.gz 2023-10-26T01:45:41.620 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108233.log 2023-10-26T01:45:41.621 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.116320.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92754.log 2023-10-26T01:45:41.621 INFO:teuthology.orchestra.run.smithi169.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.116320.log.gz 2023-10-26T01:45:41.621 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.108233.log: 87.2%gzip -- replaced with /var/log/ceph/ceph-client.admin.108233.log.gz 2023-10-26T01:45:41.621 INFO:teuthology.orchestra.run.smithi169.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.120751.log 2023-10-26T01:45:41.622 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.92754.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98527.log 2023-10-26T01:45:41.622 INFO:teuthology.orchestra.run.smithi169.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.92754.log.gz 2023-10-26T01:45:41.623 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.120751.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.120751.log.gz 2023-10-26T01:45:41.623 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.102012.log 2023-10-26T01:45:41.623 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.98527.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92294.log 2023-10-26T01:45:41.623 INFO:teuthology.orchestra.run.smithi169.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.98527.log.gz 2023-10-26T01:45:41.624 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.0.102012.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.102012.log.gz 2023-10-26T01:45:41.624 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93297.log 2023-10-26T01:45:41.624 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.92294.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.84710.log 2023-10-26T01:45:41.624 INFO:teuthology.orchestra.run.smithi169.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.92294.log.gz 2023-10-26T01:45:41.624 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.93297.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104951.log 2023-10-26T01:45:41.625 INFO:teuthology.orchestra.run.smithi169.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.93297.log.gz 2023-10-26T01:45:41.625 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.0.84710.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.84710.log.gz 2023-10-26T01:45:41.625 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85608.log 2023-10-26T01:45:41.625 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.104951.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83034.log 2023-10-26T01:45:41.626 INFO:teuthology.orchestra.run.smithi169.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.104951.log.gz 2023-10-26T01:45:41.626 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.85608.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.85608.log.gz 2023-10-26T01:45:41.626 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108317.log 2023-10-26T01:45:41.627 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.83034.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121424.log 2023-10-26T01:45:41.627 INFO:teuthology.orchestra.run.smithi169.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.83034.log.gz 2023-10-26T01:45:41.627 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.108317.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110702.log 2023-10-26T01:45:41.627 INFO:teuthology.orchestra.run.smithi169.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.108317.log.gz 2023-10-26T01:45:41.632 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.121424.log: gzip 87.1% -- replaced with /var/log/ceph/ceph-client.admin.121424.log.gz 2023-10-26T01:45:41.632 INFO:teuthology.orchestra.run.smithi169.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.109261.log 2023-10-26T01:45:41.633 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.110702.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106119.log 2023-10-26T01:45:41.633 INFO:teuthology.orchestra.run.smithi169.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.110702.log.gz 2023-10-26T01:45:41.633 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.109261.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.109261.log.gz 2023-10-26T01:45:41.633 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116446.log 2023-10-26T01:45:41.633 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.106119.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.106119.log.gz 2023-10-26T01:45:41.634 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111039.log 2023-10-26T01:45:41.634 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.116446.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.116446.log.gz 2023-10-26T01:45:41.634 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101675.log 2023-10-26T01:45:41.634 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.111039.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90066.log 2023-10-26T01:45:41.634 INFO:teuthology.orchestra.run.smithi169.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.111039.log.gz 2023-10-26T01:45:41.635 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.101675.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.101675.log.gz 2023-10-26T01:45:41.635 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86112.log 2023-10-26T01:45:41.635 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.90066.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.90066.log.gz 2023-10-26T01:45:41.635 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89724.log 2023-10-26T01:45:41.636 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.86112.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.86112.log.gz 2023-10-26T01:45:41.636 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108065.log 2023-10-26T01:45:41.636 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.89724.log: 87.5%gzip -- replaced with /var/log/ceph/ceph-client.admin.89724.log.gz 2023-10-26T01:45:41.636 INFO:teuthology.orchestra.run.smithi169.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.114968.log 2023-10-26T01:45:41.637 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.108065.log: gzip -5 --verbose -- 87.5% /var/log/ceph/ceph-client.admin.113012.log -- replaced with /var/log/ceph/ceph-client.admin.108065.log.gz 2023-10-26T01:45:41.637 INFO:teuthology.orchestra.run.smithi169.stderr: 2023-10-26T01:45:41.637 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.114968.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.114968.log.gz 2023-10-26T01:45:41.637 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98737.log 2023-10-26T01:45:41.640 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.113012.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.113012.log.gz 2023-10-26T01:45:41.641 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120054.log 2023-10-26T01:45:41.641 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.98737.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.98737.log.gz 2023-10-26T01:45:41.641 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88277.log 2023-10-26T01:45:41.641 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.120054.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.120054.log.gz 2023-10-26T01:45:41.642 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86783.log 2023-10-26T01:45:41.642 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.88277.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.88277.log.gz 2023-10-26T01:45:41.642 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110282.log 2023-10-26T01:45:41.642 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.86783.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.86783.log.gz 2023-10-26T01:45:41.643 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104601.log 2023-10-26T01:45:41.643 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.110282.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.110282.log.gz 2023-10-26T01:45:41.643 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113358.log 2023-10-26T01:45:41.643 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.104601.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.104601.log.gz 2023-10-26T01:45:41.643 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103948.log 2023-10-26T01:45:41.645 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.113358.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85860.log 87.1% 2023-10-26T01:45:41.646 INFO:teuthology.orchestra.run.smithi169.stderr: -- replaced with /var/log/ceph/ceph-client.admin.113358.log.gz 2023-10-26T01:45:41.646 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.103948.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.103948.log.gz 2023-10-26T01:45:41.646 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108107.log 2023-10-26T01:45:41.646 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.85860.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.85860.log.gz 2023-10-26T01:45:41.647 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80477.log 2023-10-26T01:45:41.647 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.108107.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.108107.log.gz 2023-10-26T01:45:41.647 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115010.log 2023-10-26T01:45:41.649 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.80477.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.80477.log.gz 2023-10-26T01:45:41.649 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110955.log 2023-10-26T01:45:41.649 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.115010.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109750.log 2023-10-26T01:45:41.650 INFO:teuthology.orchestra.run.smithi169.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.115010.log.gz 2023-10-26T01:45:41.650 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.110955.log: gzip 87.3% -- replaced with /var/log/ceph/ceph-client.admin.110955.log.gz 2023-10-26T01:45:41.650 INFO:teuthology.orchestra.run.smithi169.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.80503.log 2023-10-26T01:45:41.652 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.109750.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.109750.log.gz 2023-10-26T01:45:41.652 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82704.log 2023-10-26T01:45:41.652 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.80503.log: gzip -5 86.8% --verbose -- replaced with /var/log/ceph/ceph-client.admin.80503.log.gz -- 2023-10-26T01:45:41.652 INFO:teuthology.orchestra.run.smithi169.stderr: /var/log/ceph/ceph-client.admin.109219.log 2023-10-26T01:45:41.652 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.82704.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80630.log 2023-10-26T01:45:41.653 INFO:teuthology.orchestra.run.smithi169.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.82704.log.gz 2023-10-26T01:45:41.654 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.109219.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.109219.log.gz 2023-10-26T01:45:41.654 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92252.log 2023-10-26T01:45:41.654 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.80630.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96839.log 2023-10-26T01:45:41.654 INFO:teuthology.orchestra.run.smithi169.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.80630.log.gz 2023-10-26T01:45:41.655 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.92252.log: gzip -5 --verbose -- 87.1% /var/log/ceph/ceph-client.admin.85902.log -- replaced with /var/log/ceph/ceph-client.admin.92252.log.gz 2023-10-26T01:45:41.655 INFO:teuthology.orchestra.run.smithi169.stderr: 2023-10-26T01:45:41.656 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.96839.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81721.log 2023-10-26T01:45:41.657 INFO:teuthology.orchestra.run.smithi169.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.96839.log.gz 2023-10-26T01:45:41.657 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.85902.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.85902.log.gz 2023-10-26T01:45:41.658 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120919.log 2023-10-26T01:45:41.658 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.81721.log: 87.1%gzip -- replaced with /var/log/ceph/ceph-client.admin.81721.log.gz 2023-10-26T01:45:41.659 INFO:teuthology.orchestra.run.smithi169.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.88963.log 2023-10-26T01:45:41.659 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.120919.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.120919.log.gz 2023-10-26T01:45:41.660 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104909.log 2023-10-26T01:45:41.660 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.88963.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.88963.log.gz 2023-10-26T01:45:41.661 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100610.log 2023-10-26T01:45:41.661 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.104909.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119354.log 2023-10-26T01:45:41.662 INFO:teuthology.orchestra.run.smithi169.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.104909.log.gz 2023-10-26T01:45:41.662 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.100610.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101549.log 2023-10-26T01:45:41.663 INFO:teuthology.orchestra.run.smithi169.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.100610.log.gz 2023-10-26T01:45:41.663 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.119354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82705.log 2023-10-26T01:45:41.664 INFO:teuthology.orchestra.run.smithi169.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.119354.log.gz 2023-10-26T01:45:41.664 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.101549.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117292.log 2023-10-26T01:45:41.665 INFO:teuthology.orchestra.run.smithi169.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.101549.log.gz 2023-10-26T01:45:41.665 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.82705.log: gzip -5 87.3% --verbose -- replaced with /var/log/ceph/ceph-client.admin.82705.log.gz 2023-10-26T01:45:41.666 INFO:teuthology.orchestra.run.smithi169.stderr: -- /var/log/ceph/ceph-client.admin.83783.log 2023-10-26T01:45:41.667 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.117292.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.117292.log.gz 2023-10-26T01:45:41.667 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98349.log 2023-10-26T01:45:41.668 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.83783.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.83783.log.gz 2023-10-26T01:45:41.668 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101157.log 2023-10-26T01:45:41.669 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.98349.log: gzip -5 87.0% --verbose -- replaced with /var/log/ceph/ceph-client.admin.98349.log.gz -- 2023-10-26T01:45:41.669 INFO:teuthology.orchestra.run.smithi169.stderr: /var/log/ceph/ceph-client.admin.99693.log 2023-10-26T01:45:41.670 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.101157.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.101157.log.gz 2023-10-26T01:45:41.670 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117538.log 2023-10-26T01:45:41.671 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.99693.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.99693.log.gz 2023-10-26T01:45:41.671 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112928.log 2023-10-26T01:45:41.672 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.117538.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.117538.log.gz 2023-10-26T01:45:41.672 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2023-10-26T01:45:41.673 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.112928.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100255.log 2023-10-26T01:45:41.673 INFO:teuthology.orchestra.run.smithi169.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.112928.log.gz 2023-10-26T01:45:41.674 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94481.log 2023-10-26T01:45:41.675 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.100255.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120348.log 87.5% 2023-10-26T01:45:41.675 INFO:teuthology.orchestra.run.smithi169.stderr: -- replaced with /var/log/ceph/ceph-client.admin.100255.log.gz 2023-10-26T01:45:41.677 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.94481.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103990.log 2023-10-26T01:45:41.677 INFO:teuthology.orchestra.run.smithi169.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.94481.log.gz 2023-10-26T01:45:41.678 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.120348.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.120348.log.gz 2023-10-26T01:45:41.678 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113442.log 2023-10-26T01:45:41.679 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.103990.log: gzip -5 87.7% --verbose -- replaced with /var/log/ceph/ceph-client.admin.103990.log.gz -- 2023-10-26T01:45:41.679 INFO:teuthology.orchestra.run.smithi169.stderr: /var/log/ceph/ceph-client.0.102016.log 2023-10-26T01:45:41.680 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.113442.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.113442.log.gz 2023-10-26T01:45:41.681 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86495.log 2023-10-26T01:45:41.682 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.0.102016.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.102016.log.gz 2023-10-26T01:45:41.683 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95883.log 2023-10-26T01:45:41.683 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.86495.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.86495.log.gz 2023-10-26T01:45:41.684 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103864.log 2023-10-26T01:45:41.684 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.95883.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97931.log 2023-10-26T01:45:41.685 INFO:teuthology.orchestra.run.smithi169.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.95883.log.gz 2023-10-26T01:45:41.686 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.103864.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.103864.log.gz 2023-10-26T01:45:41.686 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112544.log 2023-10-26T01:45:41.687 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.97931.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.97931.log.gz 2023-10-26T01:45:41.688 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110787.log 2023-10-26T01:45:41.688 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.112544.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.112544.log.gz 2023-10-26T01:45:41.689 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82703.log 2023-10-26T01:45:41.689 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.110787.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.110787.log.gz 2023-10-26T01:45:41.690 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.108868.log 2023-10-26T01:45:41.691 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.82703.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.82703.log.gz 2023-10-26T01:45:41.691 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109538.log 2023-10-26T01:45:41.692 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.0.108868.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.108868.log.gz 2023-10-26T01:45:41.692 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103199.log 2023-10-26T01:45:41.693 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.109538.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.109538.log.gz 2023-10-26T01:45:41.694 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119642.log 2023-10-26T01:45:41.694 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.103199.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.103199.log.gz 2023-10-26T01:45:41.695 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106885.log 2023-10-26T01:45:41.695 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.119642.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.119642.log.gz 2023-10-26T01:45:41.696 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105641.log 2023-10-26T01:45:41.697 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.106885.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.106885.log.gz 2023-10-26T01:45:41.697 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91333.log 2023-10-26T01:45:41.698 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.105641.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.105641.log.gz 2023-10-26T01:45:41.698 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80329.log 2023-10-26T01:45:41.699 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.91333.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.91333.log.gz 2023-10-26T01:45:41.700 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110324.log 2023-10-26T01:45:41.701 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.80329.log: gzip 86.9% -5 -- replaced with /var/log/ceph/ceph-client.admin.80329.log.gz --verbose 2023-10-26T01:45:41.701 INFO:teuthology.orchestra.run.smithi169.stderr: -- /var/log/ceph/ceph-client.admin.111552.log 2023-10-26T01:45:41.702 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.110324.log: 87.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.110324.log.gz 2023-10-26T01:45:41.702 INFO:teuthology.orchestra.run.smithi169.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.107468.log 2023-10-26T01:45:41.703 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.111552.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.111552.log.gz 2023-10-26T01:45:41.703 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90622.log 2023-10-26T01:45:41.704 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.107468.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.107468.log.gz 2023-10-26T01:45:41.705 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116614.log 2023-10-26T01:45:41.706 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.90622.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.90622.log.gz 2023-10-26T01:45:41.706 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110576.log 2023-10-26T01:45:41.707 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.116614.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.116614.log.gz 2023-10-26T01:45:41.708 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113400.log 2023-10-26T01:45:41.709 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.110576.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.110576.log.gz 2023-10-26T01:45:41.709 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112168.log 2023-10-26T01:45:41.710 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.113400.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.113400.log.gz 2023-10-26T01:45:41.710 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86579.log 2023-10-26T01:45:41.711 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.112168.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.112168.log.gz 2023-10-26T01:45:41.711 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84077.log 2023-10-26T01:45:41.712 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.86579.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.86579.log.gz 2023-10-26T01:45:41.712 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119684.log 2023-10-26T01:45:41.713 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.84077.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.84077.log.gz 2023-10-26T01:45:41.713 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94337.log 2023-10-26T01:45:41.714 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.119684.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.119684.log.gz 2023-10-26T01:45:41.714 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121214.log 2023-10-26T01:45:41.715 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.94337.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.94337.log.gz 2023-10-26T01:45:41.716 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108443.log 2023-10-26T01:45:41.716 INFO:teuthology.orchestra.run.smithi170.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2023-10-26T01:45:41.717 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.121214.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.121214.log.gz 2023-10-26T01:45:41.717 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98031.log 2023-10-26T01:45:41.718 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.108443.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.108443.log.gz 2023-10-26T01:45:41.718 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83951.log 2023-10-26T01:45:41.719 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.98031.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.98031.log.gz 2023-10-26T01:45:41.719 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82696.log 2023-10-26T01:45:41.720 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.83951.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.83951.log.gz 2023-10-26T01:45:41.720 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.92631.log 2023-10-26T01:45:41.722 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.82696.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.82696.log.gz 2023-10-26T01:45:41.722 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114091.log 2023-10-26T01:45:41.723 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.0.92631.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.92631.log.gz 2023-10-26T01:45:41.723 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96467.log 2023-10-26T01:45:41.724 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.114091.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.114091.log.gz 2023-10-26T01:45:41.724 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108401.log 2023-10-26T01:45:41.725 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.96467.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.96467.log.gz 2023-10-26T01:45:41.725 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87101.log 2023-10-26T01:45:41.726 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.108401.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.108401.log.gz 2023-10-26T01:45:41.727 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103467.log 2023-10-26T01:45:41.728 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.87101.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.87101.log.gz 2023-10-26T01:45:41.728 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120306.log 2023-10-26T01:45:41.729 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.103467.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.103467.log.gz 2023-10-26T01:45:41.729 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94001.log 2023-10-26T01:45:41.730 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.120306.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86258.log 2023-10-26T01:45:41.730 INFO:teuthology.orchestra.run.smithi169.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.120306.log.gz 2023-10-26T01:45:41.731 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.94001.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.94001.log.gz 2023-10-26T01:45:41.731 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86537.log 2023-10-26T01:45:41.732 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.86258.log: 87.7%gzip -- replaced with /var/log/ceph/ceph-client.admin.86258.log.gz -5 2023-10-26T01:45:41.732 INFO:teuthology.orchestra.run.smithi169.stderr: --verbose -- /var/log/ceph/ceph-client.admin.119016.log 2023-10-26T01:45:41.733 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.86537.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.86537.log.gz 2023-10-26T01:45:41.734 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110871.log 2023-10-26T01:45:41.735 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.119016.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94211.log 2023-10-26T01:45:41.735 INFO:teuthology.orchestra.run.smithi169.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.119016.log.gz 2023-10-26T01:45:41.736 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.110871.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.110871.log.gz 2023-10-26T01:45:41.736 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90832.log 2023-10-26T01:45:41.737 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.94211.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.94211.log.gz 2023-10-26T01:45:41.737 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117752.log 2023-10-26T01:45:41.738 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.90832.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.90832.log.gz 2023-10-26T01:45:41.739 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113484.log 2023-10-26T01:45:41.740 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.117752.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87653.log 2023-10-26T01:45:41.740 INFO:teuthology.orchestra.run.smithi169.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.117752.log.gz 2023-10-26T01:45:41.740 INFO:teuthology.orchestra.run.smithi170.stderr: 91.9% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2023-10-26T01:45:41.741 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.113484.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.113484.log.gz 2023-10-26T01:45:41.741 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106591.log 2023-10-26T01:45:41.742 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.87653.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95503.log 2023-10-26T01:45:41.742 INFO:teuthology.orchestra.run.smithi169.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.87653.log.gz 2023-10-26T01:45:41.743 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.106591.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.106591.log.gz 2023-10-26T01:45:41.743 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114007.log 2023-10-26T01:45:41.744 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.95503.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.95503.log.gz 2023-10-26T01:45:41.744 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108275.log 2023-10-26T01:45:41.746 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.114007.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.114007.log.gz 2023-10-26T01:45:41.747 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91375.log 2023-10-26T01:45:41.748 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.108275.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81374.log 2023-10-26T01:45:41.748 INFO:teuthology.orchestra.run.smithi169.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.108275.log.gz 2023-10-26T01:45:41.749 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.91375.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.91375.log.gz 2023-10-26T01:45:41.749 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109996.log 2023-10-26T01:45:41.751 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.81374.log: gzip 87.4% -- replaced with /var/log/ceph/ceph-client.admin.81374.log.gz -5 2023-10-26T01:45:41.751 INFO:teuthology.orchestra.run.smithi169.stderr: --verbose -- /var/log/ceph/ceph-client.admin.97794.log 2023-10-26T01:45:41.751 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.109996.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.109996.log.gz 2023-10-26T01:45:41.752 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104993.log 2023-10-26T01:45:41.754 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.97794.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102911.log 2023-10-26T01:45:41.754 INFO:teuthology.orchestra.run.smithi169.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.97794.log.gz 2023-10-26T01:45:41.755 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.104993.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.104993.log.gz 2023-10-26T01:45:41.755 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113736.log 2023-10-26T01:45:41.758 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.102911.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83019.log 2023-10-26T01:45:41.759 INFO:teuthology.orchestra.run.smithi169.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.102911.log.gz 2023-10-26T01:45:41.759 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.113736.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.113736.log.gz 2023-10-26T01:45:41.759 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91550.log 2023-10-26T01:45:41.759 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.83019.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100003.log 2023-10-26T01:45:41.760 INFO:teuthology.orchestra.run.smithi169.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.83019.log.gz 2023-10-26T01:45:41.760 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.91550.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.91550.log.gz 2023-10-26T01:45:41.760 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115596.log 2023-10-26T01:45:41.761 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.100003.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94838.log 2023-10-26T01:45:41.762 INFO:teuthology.orchestra.run.smithi169.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.100003.log.gz 2023-10-26T01:45:41.762 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.115596.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.115596.log.gz 2023-10-26T01:45:41.762 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111214.log 2023-10-26T01:45:41.762 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.94838.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.94838.log.gz 2023-10-26T01:45:41.763 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106203.log 2023-10-26T01:45:41.763 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.111214.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.111214.log.gz 2023-10-26T01:45:41.764 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101199.log 2023-10-26T01:45:41.765 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.106203.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91103.log 2023-10-26T01:45:41.765 INFO:teuthology.orchestra.run.smithi169.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.106203.log.gz 2023-10-26T01:45:41.766 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.101199.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.101199.log.gz 2023-10-26T01:45:41.766 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94608.log 2023-10-26T01:45:41.767 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.91103.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.91103.log.gz 2023-10-26T01:45:41.767 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104687.log 2023-10-26T01:45:41.768 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.94608.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.94608.log.gz 2023-10-26T01:45:41.768 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118088.log 2023-10-26T01:45:41.769 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.104687.log: gzip -5 87.2% --verbose -- replaced with /var/log/ceph/ceph-client.admin.104687.log.gz -- 2023-10-26T01:45:41.769 INFO:teuthology.orchestra.run.smithi169.stderr: /var/log/ceph/ceph-client.admin.88403.log 2023-10-26T01:45:41.770 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.118088.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.118088.log.gz 2023-10-26T01:45:41.770 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121046.log 2023-10-26T01:45:41.771 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.88403.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107384.log 2023-10-26T01:45:41.771 INFO:teuthology.orchestra.run.smithi169.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.88403.log.gz 2023-10-26T01:45:41.772 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.121046.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.121046.log.gz 2023-10-26T01:45:41.773 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113132.log 2023-10-26T01:45:41.774 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.107384.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114600.log 2023-10-26T01:45:41.774 INFO:teuthology.orchestra.run.smithi169.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.107384.log.gz 2023-10-26T01:45:41.775 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.113132.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.113132.log.gz 2023-10-26T01:45:41.775 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120180.log 2023-10-26T01:45:41.776 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.114600.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98135.log 2023-10-26T01:45:41.776 INFO:teuthology.orchestra.run.smithi169.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.114600.log.gz 2023-10-26T01:45:41.777 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.120180.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.120180.log.gz 2023-10-26T01:45:41.777 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108527.log 2023-10-26T01:45:41.778 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.98135.log: gzip -5 87.1% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.98135.log.gz /var/log/ceph/ceph-client.admin.81893.log 2023-10-26T01:45:41.779 INFO:teuthology.orchestra.run.smithi169.stderr: 2023-10-26T01:45:41.779 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.108527.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.108527.log.gz 2023-10-26T01:45:41.780 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103635.log 2023-10-26T01:45:41.781 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.81893.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118719.log 87.6% 2023-10-26T01:45:41.781 INFO:teuthology.orchestra.run.smithi169.stderr: -- replaced with /var/log/ceph/ceph-client.admin.81893.log.gz 2023-10-26T01:45:41.782 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.103635.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.103635.log.gz 2023-10-26T01:45:41.782 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80529.log 2023-10-26T01:45:41.783 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.118719.log: gzip 87.6% -5 -- replaced with /var/log/ceph/ceph-client.admin.118719.log.gz 2023-10-26T01:45:41.783 INFO:teuthology.orchestra.run.smithi169.stderr: --verbose -- /var/log/ceph/ceph-client.admin.111898.log 2023-10-26T01:45:41.784 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.80529.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.80529.log.gz 2023-10-26T01:45:41.784 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118548.log 2023-10-26T01:45:41.785 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.111898.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104865.log 2023-10-26T01:45:41.786 INFO:teuthology.orchestra.run.smithi169.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.111898.log.gz 2023-10-26T01:45:41.787 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.118548.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.118548.log.gz 2023-10-26T01:45:41.787 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2023-10-26T01:45:41.788 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.104865.log: gzip 87.0% -- replaced with /var/log/ceph/ceph-client.admin.104865.log.gz -5 2023-10-26T01:45:41.788 INFO:teuthology.orchestra.run.smithi169.stderr: --verbose -- /var/log/ceph/ceph-client.admin.96797.log 2023-10-26T01:45:41.791 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110829.log 2023-10-26T01:45:41.791 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.96797.log: gzip -5 87.2% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.96797.log.gz /var/log/ceph/ceph-client.admin.99819.log 2023-10-26T01:45:41.791 INFO:teuthology.orchestra.run.smithi169.stderr: 2023-10-26T01:45:41.792 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.110829.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.110829.log.gz 2023-10-26T01:45:41.792 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96551.log 2023-10-26T01:45:41.794 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.99819.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104456.log 2023-10-26T01:45:41.794 INFO:teuthology.orchestra.run.smithi169.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.99819.log.gz 2023-10-26T01:45:41.795 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.96551.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.96551.log.gz 2023-10-26T01:45:41.795 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95631.log 2023-10-26T01:45:41.796 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.104456.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.104456.log.gz 2023-10-26T01:45:41.797 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112276.log 2023-10-26T01:45:41.798 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.95631.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86933.log 2023-10-26T01:45:41.798 INFO:teuthology.orchestra.run.smithi169.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.95631.log.gz 2023-10-26T01:45:41.800 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.112276.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.112276.log.gz 2023-10-26T01:45:41.800 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99215.log 2023-10-26T01:45:41.802 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.86933.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.86933.log.gz 2023-10-26T01:45:41.802 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85524.log 2023-10-26T01:45:41.803 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.99215.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98483.log 2023-10-26T01:45:41.804 INFO:teuthology.orchestra.run.smithi169.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.99215.log.gz 2023-10-26T01:45:41.805 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.85524.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.85524.log.gz 2023-10-26T01:45:41.805 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104116.log 2023-10-26T01:45:41.807 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.98483.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101421.log 2023-10-26T01:45:41.807 INFO:teuthology.orchestra.run.smithi169.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.98483.log.gz 2023-10-26T01:45:41.809 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.104116.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110997.log 2023-10-26T01:45:41.809 INFO:teuthology.orchestra.run.smithi169.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.104116.log.gz 2023-10-26T01:45:41.810 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.101421.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.101421.log.gz 2023-10-26T01:45:41.810 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113838.log 2023-10-26T01:45:41.811 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.110997.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113526.log 2023-10-26T01:45:41.812 INFO:teuthology.orchestra.run.smithi169.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.110997.log.gz 2023-10-26T01:45:41.813 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.113838.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.113838.log.gz 2023-10-26T01:45:41.813 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105269.log 2023-10-26T01:45:41.814 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.113526.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.113526.log.gz 2023-10-26T01:45:41.815 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.112467.log 2023-10-26T01:45:41.816 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.105269.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112648.log 2023-10-26T01:45:41.816 INFO:teuthology.orchestra.run.smithi169.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.105269.log.gz 2023-10-26T01:45:41.817 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.0.112467.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.112467.log.gz 2023-10-26T01:45:41.818 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102289.log 2023-10-26T01:45:41.818 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.112648.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.112648.log.gz 2023-10-26T01:45:41.819 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116178.log 2023-10-26T01:45:41.820 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.102289.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103593.log 2023-10-26T01:45:41.820 INFO:teuthology.orchestra.run.smithi169.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.102289.log.gz 2023-10-26T01:45:41.822 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.116178.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.116178.log.gz 2023-10-26T01:45:41.822 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80637.log 2023-10-26T01:45:41.823 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.103593.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.103593.log.gz 2023-10-26T01:45:41.824 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114312.log 2023-10-26T01:45:41.825 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.80637.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89483.log 2023-10-26T01:45:41.825 INFO:teuthology.orchestra.run.smithi169.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.80637.log.gz 2023-10-26T01:45:41.826 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.114312.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.114312.log.gz 2023-10-26T01:45:41.827 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98261.log 2023-10-26T01:45:41.828 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.89483.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.89483.log.gz 2023-10-26T01:45:41.828 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89682.log 2023-10-26T01:45:41.829 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.98261.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86411.log 2023-10-26T01:45:41.830 INFO:teuthology.orchestra.run.smithi169.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.98261.log.gz 2023-10-26T01:45:41.831 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.89682.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.89682.log.gz 2023-10-26T01:45:41.831 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109496.log 2023-10-26T01:45:41.833 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.86411.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107258.log 2023-10-26T01:45:41.833 INFO:teuthology.orchestra.run.smithi169.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.86411.log.gz 2023-10-26T01:45:41.834 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.109496.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.109496.log.gz 2023-10-26T01:45:41.834 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93255.log 2023-10-26T01:45:41.835 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.107258.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87569.log 2023-10-26T01:45:41.835 INFO:teuthology.orchestra.run.smithi169.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.107258.log.gz 2023-10-26T01:45:41.837 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.93255.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.93255.log.gz 2023-10-26T01:45:41.837 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82439.log 2023-10-26T01:45:41.838 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.87569.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91632.log 2023-10-26T01:45:41.839 INFO:teuthology.orchestra.run.smithi169.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.87569.log.gz 2023-10-26T01:45:41.840 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.82439.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.82439.log.gz 2023-10-26T01:45:41.840 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121526.log 2023-10-26T01:45:41.841 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.91632.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95841.log 2023-10-26T01:45:41.842 INFO:teuthology.orchestra.run.smithi169.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.91632.log.gz 2023-10-26T01:45:41.842 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.121526.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.121526.log.gz 2023-10-26T01:45:41.843 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.118429.log 2023-10-26T01:45:41.844 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.95841.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102953.log 2023-10-26T01:45:41.844 INFO:teuthology.orchestra.run.smithi169.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.95841.log.gz 2023-10-26T01:45:41.845 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.0.118429.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.118429.log.gz 2023-10-26T01:45:41.846 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109708.log 2023-10-26T01:45:41.847 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.102953.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.102953.log.gz 2023-10-26T01:45:41.847 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113652.log 2023-10-26T01:45:41.848 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.109708.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96713.log 2023-10-26T01:45:41.849 INFO:teuthology.orchestra.run.smithi169.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.109708.log.gz 2023-10-26T01:45:41.850 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.113652.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.113652.log.gz 2023-10-26T01:45:41.851 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102637.log 2023-10-26T01:45:41.851 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.96713.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.96713.log.gz 2023-10-26T01:45:41.852 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.118425.log 2023-10-26T01:45:41.852 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.102637.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85272.log 2023-10-26T01:45:41.853 INFO:teuthology.orchestra.run.smithi169.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.102637.log.gz 2023-10-26T01:45:41.854 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.0.118425.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.118425.log.gz 2023-10-26T01:45:41.854 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114049.log 2023-10-26T01:45:41.855 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.85272.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.85272.log.gz 2023-10-26T01:45:41.856 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121608.log 2023-10-26T01:45:41.857 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.114049.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93775.log 2023-10-26T01:45:41.858 INFO:teuthology.orchestra.run.smithi169.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.114049.log.gz 2023-10-26T01:45:41.859 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.121608.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.121608.log.gz 2023-10-26T01:45:41.859 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85398.log 2023-10-26T01:45:41.860 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.93775.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.93775.log.gz 2023-10-26T01:45:41.860 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.108864.log 2023-10-26T01:45:41.861 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.85398.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112970.log 2023-10-26T01:45:41.862 INFO:teuthology.orchestra.run.smithi169.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.85398.log.gz 2023-10-26T01:45:41.863 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.0.108864.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.108864.log.gz 2023-10-26T01:45:41.863 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107300.log 2023-10-26T01:45:41.865 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.112970.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.112970.log.gz 2023-10-26T01:45:41.865 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103241.log 2023-10-26T01:45:41.866 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.107300.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.89360.log 2023-10-26T01:45:41.867 INFO:teuthology.orchestra.run.smithi169.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.107300.log.gz 2023-10-26T01:45:41.868 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.103241.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89819.log 2023-10-26T01:45:41.869 INFO:teuthology.orchestra.run.smithi169.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.103241.log.gz 2023-10-26T01:45:41.870 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.0.89360.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.89360.log.gz 2023-10-26T01:45:41.870 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97648.log 2023-10-26T01:45:41.871 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.89819.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93943.log 2023-10-26T01:45:41.871 INFO:teuthology.orchestra.run.smithi169.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.89819.log.gz 2023-10-26T01:45:41.872 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.97648.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.97648.log.gz 2023-10-26T01:45:41.872 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100526.log 2023-10-26T01:45:41.873 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.93943.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.93943.log.gz 2023-10-26T01:45:41.874 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104158.log 2023-10-26T01:45:41.875 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.100526.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105035.log 2023-10-26T01:45:41.875 INFO:teuthology.orchestra.run.smithi169.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.100526.log.gz 2023-10-26T01:45:41.876 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.104158.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.104158.log.gz 2023-10-26T01:45:41.876 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109336.log 2023-10-26T01:45:41.877 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.105035.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.105035.log.gz 2023-10-26T01:45:41.878 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99447.log 2023-10-26T01:45:41.879 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.109336.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98779.log 2023-10-26T01:45:41.879 INFO:teuthology.orchestra.run.smithi169.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.109336.log.gz 2023-10-26T01:45:41.880 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.99447.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.99447.log.gz 2023-10-26T01:45:41.881 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88361.log 2023-10-26T01:45:41.881 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.98779.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.98779.log.gz 2023-10-26T01:45:41.882 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110198.log 2023-10-26T01:45:41.883 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.88361.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117622.log 2023-10-26T01:45:41.884 INFO:teuthology.orchestra.run.smithi169.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.88361.log.gz 2023-10-26T01:45:41.884 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.110198.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.110198.log.gz 2023-10-26T01:45:41.885 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95196.log 2023-10-26T01:45:41.885 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.117622.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.117622.log.gz 2023-10-26T01:45:41.886 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108023.log 2023-10-26T01:45:41.887 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.95196.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110618.log 2023-10-26T01:45:41.887 INFO:teuthology.orchestra.run.smithi169.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.95196.log.gz 2023-10-26T01:45:41.888 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.108023.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.108023.log.gz 2023-10-26T01:45:41.889 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100714.log 2023-10-26T01:45:41.890 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.110618.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.110618.log.gz 2023-10-26T01:45:41.890 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105203.log 2023-10-26T01:45:41.891 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.100714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83338.log 2023-10-26T01:45:41.892 INFO:teuthology.orchestra.run.smithi169.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.100714.log.gz 2023-10-26T01:45:41.893 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.105203.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.105203.log.gz 2023-10-26T01:45:41.893 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82229.log 2023-10-26T01:45:41.894 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.83338.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80215.log 2023-10-26T01:45:41.894 INFO:teuthology.orchestra.run.smithi169.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.83338.log.gz 2023-10-26T01:45:41.895 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.82229.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.82229.log.gz 2023-10-26T01:45:41.896 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106491.log 2023-10-26T01:45:41.896 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.80215.log: 77.8% -- replaced with /var/log/ceph/ceph-client.admin.80215.log.gz 2023-10-26T01:45:41.897 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94734.log 2023-10-26T01:45:41.898 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.106491.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.106491.log.gz 2023-10-26T01:45:41.898 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113881.log 2023-10-26T01:45:41.899 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.94734.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111380.log 2023-10-26T01:45:41.900 INFO:teuthology.orchestra.run.smithi169.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.94734.log.gz 2023-10-26T01:45:41.900 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.113881.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.113881.log.gz 2023-10-26T01:45:41.901 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99173.log 2023-10-26T01:45:41.902 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.111380.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.111380.log.gz 2023-10-26T01:45:41.902 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115554.log 2023-10-26T01:45:41.903 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.99173.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.99173.log.gz 2023-10-26T01:45:41.904 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115136.log 2023-10-26T01:45:41.905 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.115554.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88445.log 2023-10-26T01:45:41.905 INFO:teuthology.orchestra.run.smithi169.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.115554.log.gz 2023-10-26T01:45:41.906 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.115136.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.115136.log.gz 2023-10-26T01:45:41.906 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83038.log 2023-10-26T01:45:41.907 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.88445.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.88445.log.gz 2023-10-26T01:45:41.908 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108359.log 2023-10-26T01:45:41.909 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.83038.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87230.log 2023-10-26T01:45:41.909 INFO:teuthology.orchestra.run.smithi169.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.83038.log.gz 2023-10-26T01:45:41.910 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.108359.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.108359.log.gz 2023-10-26T01:45:41.911 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119396.log 2023-10-26T01:45:41.911 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.87230.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.87230.log.gz 2023-10-26T01:45:41.912 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82103.log 2023-10-26T01:45:41.913 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.119396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94043.log 2023-10-26T01:45:41.913 INFO:teuthology.orchestra.run.smithi169.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.119396.log.gz 2023-10-26T01:45:41.914 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.82103.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.82103.log.gz 2023-10-26T01:45:41.914 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99405.log 2023-10-26T01:45:41.915 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.94043.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.94043.log.gz 2023-10-26T01:45:41.917 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114175.log 2023-10-26T01:45:41.918 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.99405.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97335.log 2023-10-26T01:45:41.918 INFO:teuthology.orchestra.run.smithi169.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.99405.log.gz 2023-10-26T01:45:41.919 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.114175.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.114175.log.gz 2023-10-26T01:45:41.919 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2023-10-26T01:45:41.920 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.97335.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.97335.log.gz 2023-10-26T01:45:41.921 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113300.log 2023-10-26T01:45:41.922 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117580.log 2023-10-26T01:45:41.923 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.113300.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.113300.log.gz 2023-10-26T01:45:41.923 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86070.log 2023-10-26T01:45:41.924 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.117580.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.117580.log.gz 2023-10-26T01:45:41.925 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110408.log 2023-10-26T01:45:41.926 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.86070.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109876.log 2023-10-26T01:45:41.926 INFO:teuthology.orchestra.run.smithi169.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.86070.log.gz 2023-10-26T01:45:41.927 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.110408.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.110408.log.gz 2023-10-26T01:45:41.928 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108677.log 2023-10-26T01:45:41.928 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.109876.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101337.log 2023-10-26T01:45:41.929 INFO:teuthology.orchestra.run.smithi169.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.109876.log.gz 2023-10-26T01:45:41.930 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.108677.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.108677.log.gz 2023-10-26T01:45:41.930 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80141.log 2023-10-26T01:45:41.931 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.101337.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116094.log 2023-10-26T01:45:41.932 INFO:teuthology.orchestra.run.smithi169.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.101337.log.gz 2023-10-26T01:45:41.932 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.80141.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.80141.log.gz 2023-10-26T01:45:41.932 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117666.log 2023-10-26T01:45:41.933 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.116094.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98569.log 2023-10-26T01:45:41.934 INFO:teuthology.orchestra.run.smithi169.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.116094.log.gz 2023-10-26T01:45:41.935 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.117666.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.117666.log.gz 2023-10-26T01:45:41.935 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88529.log 2023-10-26T01:45:41.936 INFO:teuthology.orchestra.run.smithi169.stderr: 94.5% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2023-10-26T01:45:41.936 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.98569.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110744.log 2023-10-26T01:45:41.937 INFO:teuthology.orchestra.run.smithi169.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.98569.log.gz 2023-10-26T01:45:41.937 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.88529.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.88529.log.gz 2023-10-26T01:45:41.937 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88709.log 2023-10-26T01:45:41.938 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83573.log/var/log/ceph/ceph-client.admin.110744.log: 2023-10-26T01:45:41.939 INFO:teuthology.orchestra.run.smithi169.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.110744.log.gz 2023-10-26T01:45:41.940 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.88709.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.88709.log.gz 2023-10-26T01:45:41.940 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86300.log 2023-10-26T01:45:41.941 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83825.log 2023-10-26T01:45:41.941 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.83573.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.83573.log.gz 2023-10-26T01:45:41.942 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.86300.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.86300.log.gz 2023-10-26T01:45:41.942 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110080.log 2023-10-26T01:45:41.943 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.83825.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108569.log 2023-10-26T01:45:41.944 INFO:teuthology.orchestra.run.smithi169.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.83825.log.gz 2023-10-26T01:45:41.944 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.110080.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.110080.log.gz 2023-10-26T01:45:41.944 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113054.log 2023-10-26T01:45:41.945 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81551.log 2023-10-26T01:45:41.946 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.108569.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.108569.log.gz 2023-10-26T01:45:41.947 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.113054.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.113054.log.gz 2023-10-26T01:45:41.947 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80290.log 2023-10-26T01:45:41.948 INFO:teuthology.orchestra.run.smithi169.stderr:gzip/var/log/ceph/ceph-client.admin.81551.log: -5 --verbose -- /var/log/ceph/ceph-client.0.115477.log 2023-10-26T01:45:41.948 INFO:teuthology.orchestra.run.smithi169.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.81551.log.gz 2023-10-26T01:45:41.949 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.80290.log: 77.6% -- replaced with /var/log/ceph/ceph-client.admin.80290.log.gz 2023-10-26T01:45:41.949 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97149.log 2023-10-26T01:45:41.950 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.0.115477.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98399.log 2023-10-26T01:45:41.951 INFO:teuthology.orchestra.run.smithi169.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.115477.log.gz 2023-10-26T01:45:41.951 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.97149.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.97149.log.gz 2023-10-26T01:45:41.951 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87059.log 2023-10-26T01:45:41.952 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118004.log 2023-10-26T01:45:41.953 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.98399.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.98399.log.gz 2023-10-26T01:45:41.953 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.87059.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.87059.log.gz 2023-10-26T01:45:41.954 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103906.log 2023-10-26T01:45:41.955 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.118004.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115848.log 2023-10-26T01:45:41.955 INFO:teuthology.orchestra.run.smithi169.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.118004.log.gz 2023-10-26T01:45:41.956 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.103906.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.103906.log.gz 2023-10-26T01:45:41.956 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88795.log 2023-10-26T01:45:41.957 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.115848.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118590.log 2023-10-26T01:45:41.958 INFO:teuthology.orchestra.run.smithi169.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.115848.log.gz 2023-10-26T01:45:41.958 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.88795.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.88795.log.gz 2023-10-26T01:45:41.959 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89545.log 2023-10-26T01:45:41.960 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.118590.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.118590.log.gz 2023-10-26T01:45:41.961 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98695.log 2023-10-26T01:45:41.961 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.89545.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.89545.log.gz 2023-10-26T01:45:41.962 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103115.log 2023-10-26T01:45:41.963 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.98695.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90150.log 2023-10-26T01:45:41.963 INFO:teuthology.orchestra.run.smithi169.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.98695.log.gz 2023-10-26T01:45:41.964 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.103115.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.103115.log.gz 2023-10-26T01:45:41.964 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103037.log 2023-10-26T01:45:41.965 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.90150.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.90150.log.gz 2023-10-26T01:45:41.965 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118506.log 2023-10-26T01:45:41.967 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.103037.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94650.log 2023-10-26T01:45:41.967 INFO:teuthology.orchestra.run.smithi169.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.103037.log.gz 2023-10-26T01:45:41.968 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.118506.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.118506.log.gz 2023-10-26T01:45:41.968 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90874.log 2023-10-26T01:45:41.969 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.94650.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.94650.log.gz 2023-10-26T01:45:41.970 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91019.log 2023-10-26T01:45:41.971 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.90874.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119100.log 2023-10-26T01:45:41.971 INFO:teuthology.orchestra.run.smithi169.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.90874.log.gz 2023-10-26T01:45:41.972 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.91019.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.91019.log.gz 2023-10-26T01:45:41.972 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82019.log 2023-10-26T01:45:41.973 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.119100.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.119100.log.gz 2023-10-26T01:45:41.974 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117155.log 2023-10-26T01:45:41.975 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.82019.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121382.log 2023-10-26T01:45:41.975 INFO:teuthology.orchestra.run.smithi169.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.82019.log.gz 2023-10-26T01:45:41.976 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.117155.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.117155.log.gz 2023-10-26T01:45:41.976 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108611.log 2023-10-26T01:45:41.977 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.121382.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.121382.log.gz 2023-10-26T01:45:41.978 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117029.log 2023-10-26T01:45:41.979 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.108611.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83376.log 2023-10-26T01:45:41.980 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.117029.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.108611.log.gz 2023-10-26T01:45:41.980 INFO:teuthology.orchestra.run.smithi169.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.117029.log.gz 2023-10-26T01:45:41.980 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85482.log 2023-10-26T01:45:41.981 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.83376.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.83376.log.gz 2023-10-26T01:45:41.982 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94253.log 2023-10-26T01:45:41.983 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.85482.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.85482.log.gz 2023-10-26T01:45:41.984 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85566.log 2023-10-26T01:45:41.984 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.94253.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.94253.log.gz 2023-10-26T01:45:41.985 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97564.log 2023-10-26T01:45:41.986 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.85566.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88235.log 2023-10-26T01:45:41.986 INFO:teuthology.orchestra.run.smithi169.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.85566.log.gz 2023-10-26T01:45:41.987 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.97564.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.97564.log.gz 2023-10-26T01:45:41.987 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100213.log 2023-10-26T01:45:41.988 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.88235.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.88235.log.gz 2023-10-26T01:45:41.989 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88617.log 2023-10-26T01:45:41.990 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.100213.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.100213.log.gz 2023-10-26T01:45:41.990 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115932.log 2023-10-26T01:45:41.991 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.88617.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116656.log 2023-10-26T01:45:41.992 INFO:teuthology.orchestra.run.smithi169.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.88617.log.gz 2023-10-26T01:45:41.992 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.115932.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.115932.log.gz 2023-10-26T01:45:41.993 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97690.log 2023-10-26T01:45:41.994 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.116656.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.116656.log.gz 2023-10-26T01:45:41.995 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82187.log 2023-10-26T01:45:41.996 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.97690.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.97690.log.gz 2023-10-26T01:45:41.996 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84359.log 2023-10-26T01:45:41.997 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.82187.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.82187.log.gz 2023-10-26T01:45:41.997 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111295.log 2023-10-26T01:45:41.998 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.84359.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114642.log 2023-10-26T01:45:41.999 INFO:teuthology.orchestra.run.smithi169.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.84359.log.gz 2023-10-26T01:45:42.000 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.111295.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.111295.log.gz 2023-10-26T01:45:42.000 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87947.log 2023-10-26T01:45:42.001 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.114642.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.114642.log.gz 2023-10-26T01:45:42.002 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117794.log 2023-10-26T01:45:42.002 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.87947.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.87947.log.gz 2023-10-26T01:45:42.003 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81636.log 2023-10-26T01:45:42.004 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.117794.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115658.log 2023-10-26T01:45:42.005 INFO:teuthology.orchestra.run.smithi169.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.117794.log.gz 2023-10-26T01:45:42.005 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.81636.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.81636.log.gz 2023-10-26T01:45:42.006 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94127.log 2023-10-26T01:45:42.006 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.115658.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.115658.log.gz 2023-10-26T01:45:42.007 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110450.log 2023-10-26T01:45:42.008 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.94127.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120877.log 2023-10-26T01:45:42.008 INFO:teuthology.orchestra.run.smithi169.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.94127.log.gz 2023-10-26T01:45:42.009 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.110450.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.110450.log.gz 2023-10-26T01:45:42.010 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98611.log 2023-10-26T01:45:42.011 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.120877.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.120877.log.gz 2023-10-26T01:45:42.011 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82355.log 2023-10-26T01:45:42.012 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.98611.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.98611.log.gz 2023-10-26T01:45:42.013 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117836.log 2023-10-26T01:45:42.014 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.82355.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.82355.log.gz 2023-10-26T01:45:42.014 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97377.log 2023-10-26T01:45:42.015 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.117836.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.117836.log.gz 2023-10-26T01:45:42.016 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2023-10-26T01:45:42.016 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.97377.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.97377.log.gz 2023-10-26T01:45:42.017 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117878.log 2023-10-26T01:45:42.018 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81307.log 2023-10-26T01:45:42.020 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.117878.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104823.log 2023-10-26T01:45:42.020 INFO:teuthology.orchestra.run.smithi169.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.117878.log.gz 2023-10-26T01:45:42.021 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.81307.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83375.log 2023-10-26T01:45:42.022 INFO:teuthology.orchestra.run.smithi169.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.81307.log.gz 2023-10-26T01:45:42.023 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.104823.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.104823.log.gz 2023-10-26T01:45:42.023 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82313.log 2023-10-26T01:45:42.025 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.83375.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99531.log 2023-10-26T01:45:42.025 INFO:teuthology.orchestra.run.smithi169.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.83375.log.gz 2023-10-26T01:45:42.026 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.82313.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108149.log 2023-10-26T01:45:42.026 INFO:teuthology.orchestra.run.smithi169.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.82313.log.gz 2023-10-26T01:45:42.027 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.99531.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94776.log 2023-10-26T01:45:42.027 INFO:teuthology.orchestra.run.smithi169.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.99531.log.gz 2023-10-26T01:45:42.029 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.108149.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105077.log 2023-10-26T01:45:42.029 INFO:teuthology.orchestra.run.smithi169.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.108149.log.gz 2023-10-26T01:45:42.030 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.94776.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101465.log 2023-10-26T01:45:42.031 INFO:teuthology.orchestra.run.smithi169.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.94776.log.gz 2023-10-26T01:45:42.031 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.105077.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116572.log 2023-10-26T01:45:42.032 INFO:teuthology.orchestra.run.smithi169.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.105077.log.gz 2023-10-26T01:45:42.033 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.101465.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93697.log 2023-10-26T01:45:42.033 INFO:teuthology.orchestra.run.smithi169.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.101465.log.gz 2023-10-26T01:45:42.035 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.116572.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94975.log 2023-10-26T01:45:42.035 INFO:teuthology.orchestra.run.smithi169.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.116572.log.gz 2023-10-26T01:45:42.035 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.93697.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87359.log 2023-10-26T01:45:42.036 INFO:teuthology.orchestra.run.smithi169.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.93697.log.gz 2023-10-26T01:45:42.037 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.94975.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99919.log 2023-10-26T01:45:42.037 INFO:teuthology.orchestra.run.smithi169.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.94975.log.gz 2023-10-26T01:45:42.039 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.87359.log: gzip 87.5% -- replaced with /var/log/ceph/ceph-client.admin.87359.log.gz -5 2023-10-26T01:45:42.039 INFO:teuthology.orchestra.run.smithi169.stderr: --verbose -- /var/log/ceph/ceph-client.admin.85734.log 2023-10-26T01:45:42.040 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.99919.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119142.log 2023-10-26T01:45:42.040 INFO:teuthology.orchestra.run.smithi169.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.99919.log.gz 2023-10-26T01:45:42.041 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.85734.log: gzip 87.1% -- replaced with /var/log/ceph/ceph-client.admin.85734.log.gz -5 2023-10-26T01:45:42.042 INFO:teuthology.orchestra.run.smithi169.stderr: --verbose -- /var/log/ceph/ceph-client.admin.94421.log 2023-10-26T01:45:42.043 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.119142.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92378.log 2023-10-26T01:45:42.043 INFO:teuthology.orchestra.run.smithi169.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.119142.log.gz 2023-10-26T01:45:42.044 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.94421.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95799.log 2023-10-26T01:45:42.045 INFO:teuthology.orchestra.run.smithi169.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.94421.log.gz 2023-10-26T01:45:42.046 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.92378.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114798.log 2023-10-26T01:45:42.046 INFO:teuthology.orchestra.run.smithi169.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.92378.log.gz 2023-10-26T01:45:42.048 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.95799.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105537.log 2023-10-26T01:45:42.048 INFO:teuthology.orchestra.run.smithi169.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.95799.log.gz 2023-10-26T01:45:42.049 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.114798.log: 87.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92336.log 2023-10-26T01:45:42.049 INFO:teuthology.orchestra.run.smithi169.stderr: -- replaced with /var/log/ceph/ceph-client.admin.114798.log.gz 2023-10-26T01:45:42.050 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.105537.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114704.log 2023-10-26T01:45:42.050 INFO:teuthology.orchestra.run.smithi169.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.105537.log.gz 2023-10-26T01:45:42.052 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.92336.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111856.log 2023-10-26T01:45:42.052 INFO:teuthology.orchestra.run.smithi169.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.92336.log.gz 2023-10-26T01:45:42.053 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.114704.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97209.log 2023-10-26T01:45:42.053 INFO:teuthology.orchestra.run.smithi169.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.114704.log.gz 2023-10-26T01:45:42.054 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.111856.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.85143.log 2023-10-26T01:45:42.055 INFO:teuthology.orchestra.run.smithi169.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.111856.log.gz 2023-10-26T01:45:42.056 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.97209.log: gzip -5 --verbose -- 87.4% /var/log/ceph/ceph-client.admin.119600.log -- replaced with /var/log/ceph/ceph-client.admin.97209.log.gz 2023-10-26T01:45:42.056 INFO:teuthology.orchestra.run.smithi169.stderr: 2023-10-26T01:45:42.057 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.2.85143.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.85143.log.gz 2023-10-26T01:45:42.057 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83909.log 2023-10-26T01:45:42.058 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.119600.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116885.log 2023-10-26T01:45:42.059 INFO:teuthology.orchestra.run.smithi169.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.119600.log.gz 2023-10-26T01:45:42.060 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.83909.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94295.log 2023-10-26T01:45:42.060 INFO:teuthology.orchestra.run.smithi169.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.83909.log.gz 2023-10-26T01:45:42.062 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.116885.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-10-26T01:45:42.062 INFO:teuthology.orchestra.run.smithi169.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.116885.log.gz 2023-10-26T01:45:42.063 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.94295.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100949.log 2023-10-26T01:45:42.063 INFO:teuthology.orchestra.run.smithi169.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.94295.log.gz 2023-10-26T01:45:42.064 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119312.log 2023-10-26T01:45:42.066 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.100949.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110660.log 2023-10-26T01:45:42.066 INFO:teuthology.orchestra.run.smithi169.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.100949.log.gz 2023-10-26T01:45:42.067 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.119312.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2023-10-26T01:45:42.068 INFO:teuthology.orchestra.run.smithi169.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.119312.log.gz 2023-10-26T01:45:42.068 INFO:teuthology.orchestra.run.smithi169.stderr: 87.0% -- replaced with /var/log/ceph/ceph.log.gz 2023-10-26T01:45:42.069 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.110660.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.110660.log.gz 2023-10-26T01:45:42.069 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103283.log 2023-10-26T01:45:42.070 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85314.log 2023-10-26T01:45:42.072 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.103283.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90192.log 2023-10-26T01:45:42.072 INFO:teuthology.orchestra.run.smithi169.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.103283.log.gz 2023-10-26T01:45:42.073 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.85314.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97107.log 2023-10-26T01:45:42.074 INFO:teuthology.orchestra.run.smithi169.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.85314.log.gz 2023-10-26T01:45:42.074 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.90192.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.90192.log.gz 2023-10-26T01:45:42.075 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86657.log 2023-10-26T01:45:42.076 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.97107.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94692.log 2023-10-26T01:45:42.077 INFO:teuthology.orchestra.run.smithi169.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.97107.log.gz 2023-10-26T01:45:42.078 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.86657.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91470.log 2023-10-26T01:45:42.078 INFO:teuthology.orchestra.run.smithi169.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.86657.log.gz 2023-10-26T01:45:42.079 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.94692.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87905.log 2023-10-26T01:45:42.079 INFO:teuthology.orchestra.run.smithi169.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.94692.log.gz 2023-10-26T01:45:42.081 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.91470.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2023-10-26T01:45:42.081 INFO:teuthology.orchestra.run.smithi169.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.91470.log.gz 2023-10-26T01:45:42.081 INFO:teuthology.orchestra.run.smithi170.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2023-10-26T01:45:42.082 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.87905.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.87905.log.gz 2023-10-26T01:45:42.083 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112066.log 2023-10-26T01:45:42.084 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113923.log 2023-10-26T01:45:42.086 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.112066.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.112066.log.gz 2023-10-26T01:45:42.086 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112210.log 2023-10-26T01:45:42.092 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.113923.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.113923.log.gz 2023-10-26T01:45:42.092 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104200.log 2023-10-26T01:45:42.092 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.112210.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.112210.log.gz 2023-10-26T01:45:42.092 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88921.log 2023-10-26T01:45:42.092 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.104200.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.104200.log.gz 2023-10-26T01:45:42.093 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111422.log 2023-10-26T01:45:42.094 INFO:teuthology.orchestra.run.smithi170.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2023-10-26T01:45:42.096 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.88921.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.88921.log.gz 2023-10-26T01:45:42.096 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117113.log 2023-10-26T01:45:42.096 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.111422.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.111422.log.gz 2023-10-26T01:45:42.096 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90976.log 2023-10-26T01:45:42.098 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.117113.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.117113.log.gz 2023-10-26T01:45:42.098 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115890.log 2023-10-26T01:45:42.099 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.90976.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.90976.log.gz 2023-10-26T01:45:42.100 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107893.log 2023-10-26T01:45:42.101 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.115890.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.115890.log.gz 2023-10-26T01:45:42.102 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110913.log 2023-10-26T01:45:42.103 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.107893.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.107893.log.gz 2023-10-26T01:45:42.104 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111508.log 2023-10-26T01:45:42.105 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.110913.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.110913.log.gz 2023-10-26T01:45:42.106 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.112463.log 2023-10-26T01:45:42.106 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.111508.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.111508.log.gz 2023-10-26T01:45:42.107 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83377.log 2023-10-26T01:45:42.108 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.0.112463.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.112463.log.gz 2023-10-26T01:45:42.109 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110534.log 2023-10-26T01:45:42.110 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.83377.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.83377.log.gz 2023-10-26T01:45:42.111 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118897.log 2023-10-26T01:45:42.112 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.110534.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.110534.log.gz 2023-10-26T01:45:42.113 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83025.log 2023-10-26T01:45:42.115 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.118897.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.118897.log.gz 2023-10-26T01:45:42.115 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103383.log 2023-10-26T01:45:42.117 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.83025.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.83025.log.gz 2023-10-26T01:45:42.117 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111940.log 2023-10-26T01:45:42.118 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.103383.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.103383.log.gz 2023-10-26T01:45:42.119 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87401.log 2023-10-26T01:45:42.121 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.111940.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.111940.log.gz 2023-10-26T01:45:42.121 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5/var/log/ceph/ceph-client.admin.87401.log: --verbose -- /var/log/ceph/ceph-client.admin.84221.log 2023-10-26T01:45:42.123 INFO:teuthology.orchestra.run.smithi169.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.87401.log.gz 2023-10-26T01:45:42.123 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111464.log 2023-10-26T01:45:42.124 INFO:teuthology.orchestra.run.smithi170.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2023-10-26T01:45:42.126 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.84221.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.84221.log.gz 2023-10-26T01:45:42.127 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5/var/log/ceph/ceph-client.admin.111464.log: --verbose -- /var/log/ceph/ceph-client.admin.117962.log 2023-10-26T01:45:42.127 INFO:teuthology.orchestra.run.smithi169.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.111464.log.gz 2023-10-26T01:45:42.128 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103341.log 2023-10-26T01:45:42.130 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.117962.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.117962.log.gz 2023-10-26T01:45:42.130 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86453.log 2023-10-26T01:45:42.131 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.103341.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.103341.log.gz 2023-10-26T01:45:42.132 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92712.log 2023-10-26T01:45:42.133 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.86453.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.86453.log.gz 2023-10-26T01:45:42.134 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81062.log 2023-10-26T01:45:42.135 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.92712.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.92712.log.gz 2023-10-26T01:45:42.135 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93613.log 2023-10-26T01:45:42.136 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.81062.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.81062.log.gz 2023-10-26T01:45:42.137 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93381.log 2023-10-26T01:45:42.138 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.93613.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.93613.log.gz 2023-10-26T01:45:42.139 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83615.log 2023-10-26T01:45:42.140 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.93381.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.93381.log.gz 2023-10-26T01:45:42.141 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108191.log 2023-10-26T01:45:42.142 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.83615.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.83615.log.gz 2023-10-26T01:45:42.143 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90748.log 2023-10-26T01:45:42.144 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.108191.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.108191.log.gz 2023-10-26T01:45:42.145 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87180.log 2023-10-26T01:45:42.147 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.90748.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.90748.log.gz 2023-10-26T01:45:42.147 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.99096.log 2023-10-26T01:45:42.148 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.87180.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.87180.log.gz 2023-10-26T01:45:42.149 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102869.log 2023-10-26T01:45:42.150 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.0.99096.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.99096.log.gz 2023-10-26T01:45:42.151 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94524.log 2023-10-26T01:45:42.152 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.102869.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.102869.log.gz 2023-10-26T01:45:42.153 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5/var/log/ceph/ceph-client.admin.94524.log: --verbose -- /var/log/ceph/ceph-client.admin.102331.log 2023-10-26T01:45:42.154 INFO:teuthology.orchestra.run.smithi169.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.94524.log.gz 2023-10-26T01:45:42.155 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101115.log 2023-10-26T01:45:42.156 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.102331.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.102331.log.gz 2023-10-26T01:45:42.157 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114515.log 2023-10-26T01:45:42.158 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.101115.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.101115.log.gz 2023-10-26T01:45:42.158 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101379.log 2023-10-26T01:45:42.160 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.114515.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.114515.log.gz 2023-10-26T01:45:42.161 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102679.log 2023-10-26T01:45:42.162 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.101379.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.101379.log.gz 2023-10-26T01:45:42.163 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111686.log 2023-10-26T01:45:42.164 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.102679.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.102679.log.gz 2023-10-26T01:45:42.164 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.96140.log 2023-10-26T01:45:42.165 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.111686.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.111686.log.gz 2023-10-26T01:45:42.167 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97251.log 2023-10-26T01:45:42.167 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.0.96140.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.96140.log.gz 2023-10-26T01:45:42.168 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121256.log 2023-10-26T01:45:42.169 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.97251.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.97251.log.gz 2023-10-26T01:45:42.170 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94085.log 2023-10-26T01:45:42.171 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.121256.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.121256.log.gz 2023-10-26T01:45:42.172 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95367.log 2023-10-26T01:45:42.173 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.94085.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.94085.log.gz 2023-10-26T01:45:42.174 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96755.log 2023-10-26T01:45:42.175 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.95367.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.95367.log.gz 2023-10-26T01:45:42.176 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89441.log 2023-10-26T01:45:42.177 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.96755.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.96755.log.gz 2023-10-26T01:45:42.178 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95154.log 2023-10-26T01:45:42.179 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.89441.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.89441.log.gz 2023-10-26T01:45:42.180 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105579.log 2023-10-26T01:45:42.181 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.95154.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.95154.log.gz 2023-10-26T01:45:42.182 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93571.log 2023-10-26T01:45:42.184 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.105579.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.105579.log.gz 2023-10-26T01:45:42.184 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106633.log 2023-10-26T01:45:42.185 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.93571.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.93571.log.gz 2023-10-26T01:45:42.186 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106287.log 2023-10-26T01:45:42.189 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.106633.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.106633.log.gz 2023-10-26T01:45:42.189 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107563.log/var/log/ceph/ceph-client.admin.106287.log: 2023-10-26T01:45:42.189 INFO:teuthology.orchestra.run.smithi169.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.106287.log.gz 2023-10-26T01:45:42.191 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.89364.log 2023-10-26T01:45:42.192 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.107563.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.107563.log.gz 2023-10-26T01:45:42.192 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97606.log 2023-10-26T01:45:42.193 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.0.89364.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.89364.log.gz 2023-10-26T01:45:42.194 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104074.log 2023-10-26T01:45:42.195 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.97606.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.97606.log.gz 2023-10-26T01:45:42.196 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82692.log 2023-10-26T01:45:42.197 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.104074.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.104074.log.gz 2023-10-26T01:45:42.198 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120961.log 2023-10-26T01:45:42.199 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.82692.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.82692.log.gz 2023-10-26T01:45:42.200 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97023.log 2023-10-26T01:45:42.202 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.120961.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.120961.log.gz 2023-10-26T01:45:42.203 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100129.log 2023-10-26T01:45:42.204 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.97023.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.97023.log.gz 2023-10-26T01:45:42.204 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110122.log 2023-10-26T01:45:42.205 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.100129.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.100129.log.gz 2023-10-26T01:45:42.206 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95949.log 2023-10-26T01:45:42.207 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.110122.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.110122.log.gz 2023-10-26T01:45:42.208 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109580.log 2023-10-26T01:45:42.209 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.95949.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.95949.log.gz 2023-10-26T01:45:42.210 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93339.log 2023-10-26T01:45:42.212 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.109580.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.109580.log.gz 2023-10-26T01:45:42.212 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101633.log 2023-10-26T01:45:42.213 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.93339.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.93339.log.gz 2023-10-26T01:45:42.214 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83993.log 2023-10-26T01:45:42.215 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.101633.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.101633.log.gz 2023-10-26T01:45:42.216 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119058.log 2023-10-26T01:45:42.217 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.83993.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.83993.log.gz 2023-10-26T01:45:42.218 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115052.log 2023-10-26T01:45:42.219 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.119058.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.119058.log.gz 2023-10-26T01:45:42.220 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90354.log 2023-10-26T01:45:42.221 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.115052.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.115052.log.gz 2023-10-26T01:45:42.222 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100087.log 2023-10-26T01:45:42.222 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.90354.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.90354.log.gz 2023-10-26T01:45:42.223 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85440.log 2023-10-26T01:45:42.224 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.100087.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.100087.log.gz 2023-10-26T01:45:42.225 INFO:teuthology.orchestra.run.smithi169.stderr: 90.5% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2023-10-26T01:45:42.226 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106969.log 2023-10-26T01:45:42.226 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.85440.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.85440.log.gz 2023-10-26T01:45:42.226 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90496.log 2023-10-26T01:45:42.227 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.106969.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116488.log 2023-10-26T01:45:42.228 INFO:teuthology.orchestra.run.smithi169.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.106969.log.gz 2023-10-26T01:45:42.229 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.90496.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.90496.log.gz 2023-10-26T01:45:42.229 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.99092.log 2023-10-26T01:45:42.230 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.116488.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.116488.log.gz 2023-10-26T01:45:42.230 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84161.log 2023-10-26T01:45:42.232 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95239.log 2023-10-26T01:45:42.232 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.0.99092.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.99092.log.gz 2023-10-26T01:45:42.233 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.84161.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.84161.log.gz 2023-10-26T01:45:42.234 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93901.log 2023-10-26T01:45:42.234 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.95239.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.95239.log.gz 2023-10-26T01:45:42.236 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119784.log 2023-10-26T01:45:42.236 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.93901.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.93901.log.gz 2023-10-26T01:45:42.237 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108945.log 2023-10-26T01:45:42.238 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.119784.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112024.log 2023-10-26T01:45:42.238 INFO:teuthology.orchestra.run.smithi169.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.119784.log.gz 2023-10-26T01:45:42.239 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.108945.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.108945.log.gz 2023-10-26T01:45:42.240 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85650.log 2023-10-26T01:45:42.241 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.112024.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.112024.log.gz 2023-10-26T01:45:42.241 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107426.log 2023-10-26T01:45:42.242 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.85650.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.115473.log 2023-10-26T01:45:42.243 INFO:teuthology.orchestra.run.smithi169.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.85650.log.gz 2023-10-26T01:45:42.243 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.107426.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.107426.log.gz 2023-10-26T01:45:42.244 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120835.log 2023-10-26T01:45:42.245 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.0.115473.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.115473.log.gz 2023-10-26T01:45:42.246 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2023-10-26T01:45:42.247 INFO:teuthology.orchestra.run.smithi169.stderr:gzip/var/log/ceph/ceph-client.admin.120835.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.85356.log 2023-10-26T01:45:42.248 INFO:teuthology.orchestra.run.smithi169.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.120835.log.gz 2023-10-26T01:45:42.251 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-mds.a.log: /var/log/ceph/ceph-client.admin.85356.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95589.log 2023-10-26T01:45:42.251 INFO:teuthology.orchestra.run.smithi169.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.85356.log.gz 2023-10-26T01:45:42.252 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107766.log 2023-10-26T01:45:42.253 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.95589.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.95589.log.gz 2023-10-26T01:45:42.254 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.107766.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114133.log 2023-10-26T01:45:42.254 INFO:teuthology.orchestra.run.smithi169.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.107766.log.gz 2023-10-26T01:45:42.255 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83365.log 2023-10-26T01:45:42.256 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.114133.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.114133.log.gz 2023-10-26T01:45:42.257 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101243.log 2023-10-26T01:45:42.258 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.83365.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.83365.log.gz 2023-10-26T01:45:42.259 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98887.log 2023-10-26T01:45:42.260 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.101243.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.101243.log.gz 2023-10-26T01:45:42.261 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111337.log 2023-10-26T01:45:42.262 INFO:teuthology.orchestra.run.smithi170.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2023-10-26T01:45:42.263 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.98887.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.98887.log.gz 2023-10-26T01:45:42.263 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87821.log 2023-10-26T01:45:42.265 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.111337.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.111337.log.gz 2023-10-26T01:45:42.265 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92084.log 2023-10-26T01:45:42.266 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.87821.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.87821.log.gz 2023-10-26T01:45:42.266 INFO:teuthology.orchestra.run.smithi170.stderr: 2023-10-26T01:45:42.267 INFO:teuthology.orchestra.run.smithi170.stderr:real 0m1.112s 2023-10-26T01:45:42.267 INFO:teuthology.orchestra.run.smithi170.stderr:user 0m4.645s 2023-10-26T01:45:42.267 INFO:teuthology.orchestra.run.smithi170.stderr:sys 0m0.584s 2023-10-26T01:45:42.271 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.92084.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.92084.log.gz 2023-10-26T01:45:42.271 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91229.log 2023-10-26T01:45:42.271 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92444.log 2023-10-26T01:45:42.271 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.91229.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.91229.log.gz 2023-10-26T01:45:42.272 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.92444.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.92444.log.gz 2023-10-26T01:45:42.331 INFO:teuthology.orchestra.run.smithi169.stderr: 92.4% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2023-10-26T01:45:42.565 INFO:teuthology.orchestra.run.smithi169.stderr: 94.1% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2023-10-26T01:45:42.685 INFO:teuthology.orchestra.run.smithi169.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2023-10-26T01:45:42.752 INFO:teuthology.orchestra.run.smithi169.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2023-10-26T01:45:43.044 INFO:teuthology.orchestra.run.smithi169.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2023-10-26T01:45:43.058 INFO:teuthology.orchestra.run.smithi169.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2023-10-26T01:45:43.061 INFO:teuthology.orchestra.run.smithi169.stderr: 2023-10-26T01:45:43.061 INFO:teuthology.orchestra.run.smithi169.stderr:real 0m1.909s 2023-10-26T01:45:43.062 INFO:teuthology.orchestra.run.smithi169.stderr:user 0m5.397s 2023-10-26T01:45:43.062 INFO:teuthology.orchestra.run.smithi169.stderr:sys 0m2.510s 2023-10-26T01:45:43.063 INFO:tasks.ceph:Archiving logs... 2023-10-26T01:45:43.064 DEBUG:teuthology.misc:Transferring archived files from smithi169:/var/log/ceph to /home/teuthworker/archive/pdonnell-2023-10-24_19:52:40-fs-wip-batrick-testing-20231024.144545-distro-default-smithi/7436434/remote/smithi169/log 2023-10-26T01:45:43.066 DEBUG:teuthology.orchestra.run.smithi169:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-26T01:45:44.453 DEBUG:teuthology.misc:Transferring archived files from smithi170:/var/log/ceph to /home/teuthworker/archive/pdonnell-2023-10-24_19:52:40-fs-wip-batrick-testing-20231024.144545-distro-default-smithi/7436434/remote/smithi170/log 2023-10-26T01:45:44.454 DEBUG:teuthology.orchestra.run.smithi170:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-26T01:45:45.159 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-10-26T01:45:45.172 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-10-26T01:45:45.173 DEBUG:teuthology.orchestra.run.smithi169:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-26T01:45:45.182 DEBUG:teuthology.orchestra.run.smithi170:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-26T01:45:45.337 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-10-26T01:45:45.347 INFO:teuthology.task.clock:Checking final clock skew... 2023-10-26T01:45:45.347 DEBUG:teuthology.orchestra.run.smithi169:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-26T01:45:45.357 DEBUG:teuthology.orchestra.run.smithi170:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-26T01:45:45.406 INFO:teuthology.orchestra.run.smithi169.stderr:bash: line 1: ntpq: command not found 2023-10-26T01:45:45.413 INFO:teuthology.orchestra.run.smithi170.stderr:bash: line 1: ntpq: command not found 2023-10-26T01:45:45.415 INFO:teuthology.orchestra.run.smithi169.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-26T01:45:45.416 INFO:teuthology.orchestra.run.smithi169.stdout:=============================================================================== 2023-10-26T01:45:45.416 INFO:teuthology.orchestra.run.smithi169.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 11 +380us[ +389us] +/- 89ms 2023-10-26T01:45:45.416 INFO:teuthology.orchestra.run.smithi169.stdout:^* hv02.front.sepia.ceph.com 3 6 377 8 +14us[ +22us] +/- 56ms 2023-10-26T01:45:45.416 INFO:teuthology.orchestra.run.smithi169.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 10 -279us[ -270us] +/- 64ms 2023-10-26T01:45:45.417 INFO:teuthology.orchestra.run.smithi169.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-10-26T01:45:45.423 INFO:teuthology.orchestra.run.smithi170.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-26T01:45:45.423 INFO:teuthology.orchestra.run.smithi170.stdout:=============================================================================== 2023-10-26T01:45:45.423 INFO:teuthology.orchestra.run.smithi170.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 11 +357us[ +360us] +/- 89ms 2023-10-26T01:45:45.424 INFO:teuthology.orchestra.run.smithi170.stdout:^* hv02.front.sepia.ceph.com 3 6 377 11 +21us[ +23us] +/- 56ms 2023-10-26T01:45:45.424 INFO:teuthology.orchestra.run.smithi170.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 12 -304us[ -302us] +/- 64ms 2023-10-26T01:45:45.424 INFO:teuthology.orchestra.run.smithi170.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-10-26T01:45:45.425 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-10-26T01:45:45.435 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-10-26T01:45:45.436 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-10-26T01:45:45.449 DEBUG:teuthology.orchestra.run.smithi169:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-10-26T01:45:45.703 DEBUG:teuthology.orchestra.run.smithi170:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-10-26T01:45:45.964 DEBUG:teuthology.orchestra.run.smithi169:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-26T01:45:46.077 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-26T01:45:46.078 DEBUG:teuthology.orchestra.run.smithi170:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-26T01:45:46.186 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-26T01:45:46.187 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-10-26T01:45:46.197 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-10-26T01:45:46.249 INFO:teuthology.task.internal:Duration was 1630.325536 seconds 2023-10-26T01:45:46.249 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-10-26T01:45:46.301 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-10-26T01:45:46.302 DEBUG:teuthology.orchestra.run.smithi169:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-26T01:45:46.311 DEBUG:teuthology.orchestra.run.smithi170:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-26T01:45:46.473 INFO:teuthology.orchestra.run.smithi169.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-26T01:45:46.483 INFO:teuthology.orchestra.run.smithi170.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-26T01:45:47.026 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-10-26T01:45:47.027 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi169.front.sepia.ceph.com 2023-10-26T01:45:47.028 DEBUG:teuthology.orchestra.run.smithi169:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-26T01:45:47.104 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi170.front.sepia.ceph.com 2023-10-26T01:45:47.105 DEBUG:teuthology.orchestra.run.smithi170:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-26T01:45:47.193 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-10-26T01:45:47.194 DEBUG:teuthology.orchestra.run.smithi169:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-26T01:45:47.204 DEBUG:teuthology.orchestra.run.smithi170:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-26T01:45:47.301 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-10-26T01:45:47.302 DEBUG:teuthology.orchestra.run.smithi169:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-26T01:45:47.308 DEBUG:teuthology.orchestra.run.smithi170:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-26T01:45:47.518 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-10-26T01:45:47.540 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-10-26T01:45:47.540 DEBUG:teuthology.orchestra.run.smithi169:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-26T01:45:47.643 DEBUG:teuthology.orchestra.run.smithi170:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-26T01:45:47.746 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-10-26T01:45:47.761 DEBUG:teuthology.orchestra.run.smithi169:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-26T01:45:47.770 DEBUG:teuthology.orchestra.run.smithi170:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-26T01:45:47.873 INFO:teuthology.orchestra.run.smithi169.stdout:kernel.core_pattern = core 2023-10-26T01:45:47.885 INFO:teuthology.orchestra.run.smithi170.stdout:kernel.core_pattern = core 2023-10-26T01:45:47.935 DEBUG:teuthology.orchestra.run.smithi169:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-26T01:45:47.986 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-26T01:45:47.987 DEBUG:teuthology.orchestra.run.smithi170:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-26T01:45:48.044 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-26T01:45:48.044 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-10-26T01:45:48.057 INFO:teuthology.task.internal:Transferring archived files... 2023-10-26T01:45:48.058 DEBUG:teuthology.misc:Transferring archived files from smithi169:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2023-10-24_19:52:40-fs-wip-batrick-testing-20231024.144545-distro-default-smithi/7436434/remote/smithi169 2023-10-26T01:45:48.058 DEBUG:teuthology.orchestra.run.smithi169:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-26T01:45:48.172 DEBUG:teuthology.misc:Transferring archived files from smithi170:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2023-10-24_19:52:40-fs-wip-batrick-testing-20231024.144545-distro-default-smithi/7436434/remote/smithi170 2023-10-26T01:45:48.173 DEBUG:teuthology.orchestra.run.smithi170:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-26T01:45:48.283 INFO:teuthology.task.internal:Removing archive directory... 2023-10-26T01:45:48.284 DEBUG:teuthology.orchestra.run.smithi169:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-26T01:45:48.293 DEBUG:teuthology.orchestra.run.smithi170:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-26T01:45:48.353 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-10-26T01:45:48.372 INFO:teuthology.task.internal:Not uploading archives. 2023-10-26T01:45:48.373 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-10-26T01:45:48.394 INFO:teuthology.task.internal:Tidying up after the test... 2023-10-26T01:45:48.395 DEBUG:teuthology.orchestra.run.smithi169:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-26T01:45:48.403 DEBUG:teuthology.orchestra.run.smithi170:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-26T01:45:48.454 INFO:teuthology.orchestra.run.smithi169.stdout: 524321 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 26 01:45 /home/ubuntu/cephtest 2023-10-26T01:45:48.464 INFO:teuthology.orchestra.run.smithi170.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 26 01:45 /home/ubuntu/cephtest 2023-10-26T01:45:48.467 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-10-26T01:45:48.477 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-10-26T01:45:48.565 INFO:teuthology.nuke:Checking targets against current locks 2023-10-26T01:45:48.592 DEBUG:teuthology.nuke:shortname: smithi169 2023-10-26T01:45:48.592 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-26T01:45:48.619 DEBUG:teuthology.nuke:shortname: smithi170 2023-10-26T01:45:48.620 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-26T01:45:48.624 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi169.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-10-24_19:52:40-fs-wip-batrick-testing-20231024.144545-distro-default-smithi/7436434', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-26 01:09:21.887394', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBMurhwyAOgLDZYp8KLlJfM2tGAae+jI6aesQjNLyzgOViSaU190GbQt3iYkIuSCQpGqFwejukievn06h7IOmAQ='} 2023-10-26T01:45:48.647 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi170.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-10-24_19:52:40-fs-wip-batrick-testing-20231024.144545-distro-default-smithi/7436434', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-26 01:09:21.888879', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOTrugeqpQFQkbL89tlvCCxYzOw91Gi/g8LcDRa/4kSdgb6tjnu8eLV09Jn06FyUdFgjJqFrxgYM07un/LnXr3g='} 2023-10-26T01:45:48.678 INFO:teuthology.orchestra.console.smithi169:Power off 2023-10-26T01:45:48.679 DEBUG:teuthology.orchestra.console.smithi169:pexpect command: ipmitool -H smithi169.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-26T01:45:48.696 INFO:teuthology.orchestra.console.smithi170:Power off 2023-10-26T01:45:48.697 DEBUG:teuthology.orchestra.console.smithi170:pexpect command: ipmitool -H smithi170.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-26T01:45:48.707 DEBUG:teuthology.orchestra.console.smithi169:power off output: Chassis Power Control: Down/Off 2023-10-26T01:45:48.708 DEBUG:teuthology.orchestra.console.smithi169:pexpect command: ipmitool -H smithi169.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-26T01:45:48.720 DEBUG:teuthology.orchestra.console.smithi170:power off output: Chassis Power Control: Down/Off 2023-10-26T01:45:48.721 DEBUG:teuthology.orchestra.console.smithi170:pexpect command: ipmitool -H smithi170.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-26T01:45:48.731 DEBUG:teuthology.orchestra.console.smithi169:check power output: Chassis Power is on 2023-10-26T01:45:48.743 DEBUG:teuthology.orchestra.console.smithi170:check power output: Chassis Power is on 2023-10-26T01:45:52.733 DEBUG:teuthology.orchestra.console.smithi169:pexpect command: ipmitool -H smithi169.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-26T01:45:52.744 DEBUG:teuthology.orchestra.console.smithi170:pexpect command: ipmitool -H smithi170.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-26T01:45:52.843 DEBUG:teuthology.orchestra.console.smithi169:check power output: Chassis Power is on 2023-10-26T01:45:52.853 DEBUG:teuthology.orchestra.console.smithi170:check power output: Chassis Power is on 2023-10-26T01:45:56.844 DEBUG:teuthology.orchestra.console.smithi169:pexpect command: ipmitool -H smithi169.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-26T01:45:56.855 DEBUG:teuthology.orchestra.console.smithi170:pexpect command: ipmitool -H smithi170.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-26T01:45:56.957 DEBUG:teuthology.orchestra.console.smithi169:check power output: Chassis Power is on 2023-10-26T01:45:56.964 DEBUG:teuthology.orchestra.console.smithi170:check power output: Chassis Power is off 2023-10-26T01:45:57.065 INFO:teuthology.orchestra.console.smithi170:Power off completed 2023-10-26T01:46:00.962 DEBUG:teuthology.orchestra.console.smithi169:pexpect command: ipmitool -H smithi169.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-26T01:46:01.073 DEBUG:teuthology.orchestra.console.smithi169:check power output: Chassis Power is off 2023-10-26T01:46:01.174 INFO:teuthology.orchestra.console.smithi169:Power off completed 2023-10-26T01:46:01.279 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 5b73517e0e2fa132ea79290e5fc06ea5349b2288 client.1-kernel-sha1: 5b73517e0e2fa132ea79290e5fc06ea5349b2288 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/mds-full} duration: 1630.3255362510681 failure_reason: '"2023-10-26T01:30:29.910396+0000 osd.1 (osd.1) 3 : cluster [WRN] OSD bench result of 97444.837208 IOPS exceeded the threshold limit of 500.000000 IOPS for osd.1. IOPS capacity is unchanged at 315.000000 IOPS. The recommendation is to establish the osd''s IOPS capacity using other benchmark tools (e.g. Fio) and then override osd_mclock_max_capacity_iops_[hdd|ssd]." in cluster log' flavor: default owner: scheduled_pdonnell@teuthology success: false 2023-10-26T01:46:01.280 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-26T01:46:01.364 INFO:teuthology.run:FAIL